Dec 03 11:20:08 crc systemd[1]: Starting Kubernetes Kubelet... Dec 03 11:20:08 crc restorecon[4694]: Relabeled /var/lib/kubelet/config.json from system_u:object_r:unlabeled_t:s0 to system_u:object_r:container_var_lib_t:s0 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/device-plugins not reset as customized by admin to system_u:object_r:container_file_t:s0 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/device-plugins/kubelet.sock not reset as customized by admin to system_u:object_r:container_file_t:s0 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/5fe485a1-e14f-4c09-b5b9-f252bc42b7e8/volumes/kubernetes.io~configmap/nginx-conf/..2025_02_23_05_40_35.4114275528/nginx.conf not reset as customized by admin to system_u:object_r:container_file_t:s0:c15,c25 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/5fe485a1-e14f-4c09-b5b9-f252bc42b7e8/etc-hosts not reset as customized by admin to system_u:object_r:container_file_t:s0:c15,c25 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/5fe485a1-e14f-4c09-b5b9-f252bc42b7e8/containers/networking-console-plugin/22e96971 not reset as customized by admin to system_u:object_r:container_file_t:s0:c15,c25 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/5fe485a1-e14f-4c09-b5b9-f252bc42b7e8/containers/networking-console-plugin/21c98286 not reset as customized by admin to system_u:object_r:container_file_t:s0:c15,c25 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/5fe485a1-e14f-4c09-b5b9-f252bc42b7e8/containers/networking-console-plugin/0f1869e1 not reset as customized by admin to system_u:object_r:container_file_t:s0:c15,c25 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/d1b160f5dda77d281dd8e69ec8d817f9/etc-hosts not reset as customized by admin to system_u:object_r:container_file_t:s0:c215,c682 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/d1b160f5dda77d281dd8e69ec8d817f9/containers/setup/46889d52 not reset as customized by admin to system_u:object_r:container_file_t:s0:c225,c458 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/d1b160f5dda77d281dd8e69ec8d817f9/containers/setup/5b6a5969 not reset as customized by admin to system_u:object_r:container_file_t:s0:c666,c963 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/d1b160f5dda77d281dd8e69ec8d817f9/containers/setup/6c7921f5 not reset as customized by admin to system_u:object_r:container_file_t:s0:c215,c682 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/d1b160f5dda77d281dd8e69ec8d817f9/containers/kube-rbac-proxy-crio/4804f443 not reset as customized by admin to system_u:object_r:container_file_t:s0:c225,c458 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/d1b160f5dda77d281dd8e69ec8d817f9/containers/kube-rbac-proxy-crio/2a46b283 not reset as customized by admin to system_u:object_r:container_file_t:s0:c225,c458 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/d1b160f5dda77d281dd8e69ec8d817f9/containers/kube-rbac-proxy-crio/a6b5573e not reset as customized by admin to system_u:object_r:container_file_t:s0:c225,c458 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/d1b160f5dda77d281dd8e69ec8d817f9/containers/kube-rbac-proxy-crio/4f88ee5b not reset as customized by admin to system_u:object_r:container_file_t:s0:c225,c458 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/d1b160f5dda77d281dd8e69ec8d817f9/containers/kube-rbac-proxy-crio/5a4eee4b not reset as customized by admin to system_u:object_r:container_file_t:s0:c666,c963 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/d1b160f5dda77d281dd8e69ec8d817f9/containers/kube-rbac-proxy-crio/cd87c521 not reset as customized by admin to system_u:object_r:container_file_t:s0:c215,c682 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/c03ee662-fb2f-4fc4-a2c1-af487c19d254/volumes/kubernetes.io~configmap/service-ca-bundle not reset as customized by admin to system_u:object_r:container_file_t:s0:c4,c24 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/c03ee662-fb2f-4fc4-a2c1-af487c19d254/volumes/kubernetes.io~configmap/service-ca-bundle/..2025_02_23_05_33_42.2574241751 not reset as customized by admin to system_u:object_r:container_file_t:s0:c4,c24 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/c03ee662-fb2f-4fc4-a2c1-af487c19d254/volumes/kubernetes.io~configmap/service-ca-bundle/..2025_02_23_05_33_42.2574241751/service-ca.crt not reset as customized by admin to system_u:object_r:container_file_t:s0:c4,c24 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/c03ee662-fb2f-4fc4-a2c1-af487c19d254/volumes/kubernetes.io~configmap/service-ca-bundle/..data not reset as customized by admin to system_u:object_r:container_file_t:s0:c4,c24 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/c03ee662-fb2f-4fc4-a2c1-af487c19d254/volumes/kubernetes.io~configmap/service-ca-bundle/service-ca.crt not reset as customized by admin to system_u:object_r:container_file_t:s0:c4,c24 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/c03ee662-fb2f-4fc4-a2c1-af487c19d254/etc-hosts not reset as customized by admin to system_u:object_r:container_file_t:s0:c4,c24 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/c03ee662-fb2f-4fc4-a2c1-af487c19d254/containers/router/38602af4 not reset as customized by admin to system_u:object_r:container_file_t:s0:c4,c24 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/c03ee662-fb2f-4fc4-a2c1-af487c19d254/containers/router/1483b002 not reset as customized by admin to system_u:object_r:container_file_t:s0:c4,c24 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/c03ee662-fb2f-4fc4-a2c1-af487c19d254/containers/router/0346718b not reset as customized by admin to system_u:object_r:container_file_t:s0:c4,c24 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/c03ee662-fb2f-4fc4-a2c1-af487c19d254/containers/router/d3ed4ada not reset as customized by admin to system_u:object_r:container_file_t:s0:c4,c24 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/c03ee662-fb2f-4fc4-a2c1-af487c19d254/containers/router/3bb473a5 not reset as customized by admin to system_u:object_r:container_file_t:s0:c4,c24 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/c03ee662-fb2f-4fc4-a2c1-af487c19d254/containers/router/8cd075a9 not reset as customized by admin to system_u:object_r:container_file_t:s0:c4,c24 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/c03ee662-fb2f-4fc4-a2c1-af487c19d254/containers/router/00ab4760 not reset as customized by admin to system_u:object_r:container_file_t:s0:c4,c24 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/c03ee662-fb2f-4fc4-a2c1-af487c19d254/containers/router/54a21c09 not reset as customized by admin to system_u:object_r:container_file_t:s0:c4,c24 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/37a5e44f-9a88-4405-be8a-b645485e7312/etc-hosts not reset as customized by admin to system_u:object_r:container_file_t:s0:c589,c726 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/37a5e44f-9a88-4405-be8a-b645485e7312/containers/network-operator/70478888 not reset as customized by admin to system_u:object_r:container_file_t:s0:c176,c499 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/37a5e44f-9a88-4405-be8a-b645485e7312/containers/network-operator/43802770 not reset as customized by admin to system_u:object_r:container_file_t:s0:c176,c499 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/37a5e44f-9a88-4405-be8a-b645485e7312/containers/network-operator/955a0edc not reset as customized by admin to system_u:object_r:container_file_t:s0:c176,c499 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/37a5e44f-9a88-4405-be8a-b645485e7312/containers/network-operator/bca2d009 not reset as customized by admin to system_u:object_r:container_file_t:s0:c140,c1009 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/37a5e44f-9a88-4405-be8a-b645485e7312/containers/network-operator/b295f9bd not reset as customized by admin to system_u:object_r:container_file_t:s0:c589,c726 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/7bb08738-c794-4ee8-9972-3a62ca171029/volumes/kubernetes.io~configmap/cni-binary-copy not reset as customized by admin to system_u:object_r:container_file_t:s0:c574,c582 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/7bb08738-c794-4ee8-9972-3a62ca171029/volumes/kubernetes.io~configmap/cni-binary-copy/..2025_02_23_05_21_22.3617465230 not reset as customized by admin to system_u:object_r:container_file_t:s0:c574,c582 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/7bb08738-c794-4ee8-9972-3a62ca171029/volumes/kubernetes.io~configmap/cni-binary-copy/..2025_02_23_05_21_22.3617465230/cnibincopy.sh not reset as customized by admin to system_u:object_r:container_file_t:s0:c574,c582 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/7bb08738-c794-4ee8-9972-3a62ca171029/volumes/kubernetes.io~configmap/cni-binary-copy/..data not reset as customized by admin to system_u:object_r:container_file_t:s0:c574,c582 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/7bb08738-c794-4ee8-9972-3a62ca171029/volumes/kubernetes.io~configmap/cni-binary-copy/cnibincopy.sh not reset as customized by admin to system_u:object_r:container_file_t:s0:c574,c582 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/7bb08738-c794-4ee8-9972-3a62ca171029/volumes/kubernetes.io~configmap/cni-sysctl-allowlist not reset as customized by admin to system_u:object_r:container_file_t:s0:c574,c582 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/7bb08738-c794-4ee8-9972-3a62ca171029/volumes/kubernetes.io~configmap/cni-sysctl-allowlist/..2025_02_23_05_21_22.2050650026 not reset as customized by admin to system_u:object_r:container_file_t:s0:c574,c582 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/7bb08738-c794-4ee8-9972-3a62ca171029/volumes/kubernetes.io~configmap/cni-sysctl-allowlist/..2025_02_23_05_21_22.2050650026/allowlist.conf not reset as customized by admin to system_u:object_r:container_file_t:s0:c574,c582 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/7bb08738-c794-4ee8-9972-3a62ca171029/volumes/kubernetes.io~configmap/cni-sysctl-allowlist/..data not reset as customized by admin to system_u:object_r:container_file_t:s0:c574,c582 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/7bb08738-c794-4ee8-9972-3a62ca171029/volumes/kubernetes.io~configmap/cni-sysctl-allowlist/allowlist.conf not reset as customized by admin to system_u:object_r:container_file_t:s0:c574,c582 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/7bb08738-c794-4ee8-9972-3a62ca171029/etc-hosts not reset as customized by admin to system_u:object_r:container_file_t:s0:c574,c582 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/7bb08738-c794-4ee8-9972-3a62ca171029/containers/egress-router-binary-copy/bc46ea27 not reset as customized by admin to system_u:object_r:container_file_t:s0:c203,c924 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/7bb08738-c794-4ee8-9972-3a62ca171029/containers/egress-router-binary-copy/5731fc1b not reset as customized by admin to system_u:object_r:container_file_t:s0:c138,c778 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/7bb08738-c794-4ee8-9972-3a62ca171029/containers/egress-router-binary-copy/5e1b2a3c not reset as customized by admin to system_u:object_r:container_file_t:s0:c574,c582 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/7bb08738-c794-4ee8-9972-3a62ca171029/containers/cni-plugins/943f0936 not reset as customized by admin to system_u:object_r:container_file_t:s0:c203,c924 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/7bb08738-c794-4ee8-9972-3a62ca171029/containers/cni-plugins/3f764ee4 not reset as customized by admin to system_u:object_r:container_file_t:s0:c138,c778 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/7bb08738-c794-4ee8-9972-3a62ca171029/containers/cni-plugins/8695e3f9 not reset as customized by admin to system_u:object_r:container_file_t:s0:c574,c582 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/7bb08738-c794-4ee8-9972-3a62ca171029/containers/bond-cni-plugin/aed7aa86 not reset as customized by admin to system_u:object_r:container_file_t:s0:c203,c924 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/7bb08738-c794-4ee8-9972-3a62ca171029/containers/bond-cni-plugin/c64d7448 not reset as customized by admin to system_u:object_r:container_file_t:s0:c138,c778 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/7bb08738-c794-4ee8-9972-3a62ca171029/containers/bond-cni-plugin/0ba16bd2 not reset as customized by admin to system_u:object_r:container_file_t:s0:c574,c582 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/7bb08738-c794-4ee8-9972-3a62ca171029/containers/routeoverride-cni/207a939f not reset as customized by admin to system_u:object_r:container_file_t:s0:c203,c924 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/7bb08738-c794-4ee8-9972-3a62ca171029/containers/routeoverride-cni/54aa8cdb not reset as customized by admin to system_u:object_r:container_file_t:s0:c138,c778 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/7bb08738-c794-4ee8-9972-3a62ca171029/containers/routeoverride-cni/1f5fa595 not reset as customized by admin to system_u:object_r:container_file_t:s0:c574,c582 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/7bb08738-c794-4ee8-9972-3a62ca171029/containers/whereabouts-cni-bincopy/bf9c8153 not reset as customized by admin to system_u:object_r:container_file_t:s0:c203,c924 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/7bb08738-c794-4ee8-9972-3a62ca171029/containers/whereabouts-cni-bincopy/47fba4ea not reset as customized by admin to system_u:object_r:container_file_t:s0:c138,c778 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/7bb08738-c794-4ee8-9972-3a62ca171029/containers/whereabouts-cni-bincopy/7ae55ce9 not reset as customized by admin to system_u:object_r:container_file_t:s0:c574,c582 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/7bb08738-c794-4ee8-9972-3a62ca171029/containers/whereabouts-cni/7906a268 not reset as customized by admin to system_u:object_r:container_file_t:s0:c203,c924 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/7bb08738-c794-4ee8-9972-3a62ca171029/containers/whereabouts-cni/ce43fa69 not reset as customized by admin to system_u:object_r:container_file_t:s0:c138,c778 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/7bb08738-c794-4ee8-9972-3a62ca171029/containers/whereabouts-cni/7fc7ea3a not reset as customized by admin to system_u:object_r:container_file_t:s0:c574,c582 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/7bb08738-c794-4ee8-9972-3a62ca171029/containers/kube-multus-additional-cni-plugins/d8c38b7d not reset as customized by admin to system_u:object_r:container_file_t:s0:c203,c924 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/7bb08738-c794-4ee8-9972-3a62ca171029/containers/kube-multus-additional-cni-plugins/9ef015fb not reset as customized by admin to system_u:object_r:container_file_t:s0:c138,c778 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/7bb08738-c794-4ee8-9972-3a62ca171029/containers/kube-multus-additional-cni-plugins/b9db6a41 not reset as customized by admin to system_u:object_r:container_file_t:s0:c574,c582 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/5b88f790-22fa-440e-b583-365168c0b23d/etc-hosts not reset as customized by admin to system_u:object_r:container_file_t:s0:c432,c991 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/5b88f790-22fa-440e-b583-365168c0b23d/containers/network-metrics-daemon/b1733d79 not reset as customized by admin to system_u:object_r:container_file_t:s0:c476,c820 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/5b88f790-22fa-440e-b583-365168c0b23d/containers/network-metrics-daemon/afccd338 not reset as customized by admin to system_u:object_r:container_file_t:s0:c272,c818 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/5b88f790-22fa-440e-b583-365168c0b23d/containers/network-metrics-daemon/9df0a185 not reset as customized by admin to system_u:object_r:container_file_t:s0:c432,c991 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/5b88f790-22fa-440e-b583-365168c0b23d/containers/kube-rbac-proxy/18938cf8 not reset as customized by admin to system_u:object_r:container_file_t:s0:c476,c820 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/5b88f790-22fa-440e-b583-365168c0b23d/containers/kube-rbac-proxy/7ab4eb23 not reset as customized by admin to system_u:object_r:container_file_t:s0:c272,c818 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/5b88f790-22fa-440e-b583-365168c0b23d/containers/kube-rbac-proxy/56930be6 not reset as customized by admin to system_u:object_r:container_file_t:s0:c432,c991 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/925f1c65-6136-48ba-85aa-3a3b50560753/volumes/kubernetes.io~configmap/env-overrides not reset as customized by admin to system_u:object_r:container_file_t:s0:c440,c975 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/925f1c65-6136-48ba-85aa-3a3b50560753/volumes/kubernetes.io~configmap/env-overrides/..2025_02_23_05_21_35.630010865 not reset as customized by admin to system_u:object_r:container_file_t:s0:c440,c975 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/925f1c65-6136-48ba-85aa-3a3b50560753/volumes/kubernetes.io~configmap/env-overrides/..data not reset as customized by admin to system_u:object_r:container_file_t:s0:c440,c975 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/925f1c65-6136-48ba-85aa-3a3b50560753/volumes/kubernetes.io~configmap/ovnkube-config not reset as customized by admin to system_u:object_r:container_file_t:s0:c440,c975 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/925f1c65-6136-48ba-85aa-3a3b50560753/volumes/kubernetes.io~configmap/ovnkube-config/..2025_02_23_05_21_35.1088506337 not reset as customized by admin to system_u:object_r:container_file_t:s0:c440,c975 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/925f1c65-6136-48ba-85aa-3a3b50560753/volumes/kubernetes.io~configmap/ovnkube-config/..2025_02_23_05_21_35.1088506337/ovnkube.conf not reset as customized by admin to system_u:object_r:container_file_t:s0:c440,c975 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/925f1c65-6136-48ba-85aa-3a3b50560753/volumes/kubernetes.io~configmap/ovnkube-config/..data not reset as customized by admin to system_u:object_r:container_file_t:s0:c440,c975 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/925f1c65-6136-48ba-85aa-3a3b50560753/volumes/kubernetes.io~configmap/ovnkube-config/ovnkube.conf not reset as customized by admin to system_u:object_r:container_file_t:s0:c440,c975 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/925f1c65-6136-48ba-85aa-3a3b50560753/etc-hosts not reset as customized by admin to system_u:object_r:container_file_t:s0:c440,c975 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/925f1c65-6136-48ba-85aa-3a3b50560753/containers/kube-rbac-proxy/0d8e3722 not reset as customized by admin to system_u:object_r:container_file_t:s0:c89,c211 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/925f1c65-6136-48ba-85aa-3a3b50560753/containers/kube-rbac-proxy/d22b2e76 not reset as customized by admin to system_u:object_r:container_file_t:s0:c382,c850 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/925f1c65-6136-48ba-85aa-3a3b50560753/containers/kube-rbac-proxy/e036759f not reset as customized by admin to system_u:object_r:container_file_t:s0:c440,c975 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/925f1c65-6136-48ba-85aa-3a3b50560753/containers/ovnkube-cluster-manager/2734c483 not reset as customized by admin to system_u:object_r:container_file_t:s0:c89,c211 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/925f1c65-6136-48ba-85aa-3a3b50560753/containers/ovnkube-cluster-manager/57878fe7 not reset as customized by admin to system_u:object_r:container_file_t:s0:c89,c211 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/925f1c65-6136-48ba-85aa-3a3b50560753/containers/ovnkube-cluster-manager/3f3c2e58 not reset as customized by admin to system_u:object_r:container_file_t:s0:c89,c211 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/925f1c65-6136-48ba-85aa-3a3b50560753/containers/ovnkube-cluster-manager/375bec3e not reset as customized by admin to system_u:object_r:container_file_t:s0:c382,c850 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/925f1c65-6136-48ba-85aa-3a3b50560753/containers/ovnkube-cluster-manager/7bc41e08 not reset as customized by admin to system_u:object_r:container_file_t:s0:c440,c975 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/cd70aa09-68dd-4d64-bd6f-156fe6d1dc6d/etc-hosts not reset as customized by admin to system_u:object_r:container_file_t:s0:c0,c25 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/cd70aa09-68dd-4d64-bd6f-156fe6d1dc6d/containers/download-server/48c7a72d not reset as customized by admin to system_u:object_r:container_file_t:s0:c0,c25 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/cd70aa09-68dd-4d64-bd6f-156fe6d1dc6d/containers/download-server/4b66701f not reset as customized by admin to system_u:object_r:container_file_t:s0:c0,c25 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/cd70aa09-68dd-4d64-bd6f-156fe6d1dc6d/containers/download-server/a5a1c202 not reset as customized by admin to system_u:object_r:container_file_t:s0:c0,c25 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/ef543e1b-8068-4ea3-b32a-61027b32e95d/volumes/kubernetes.io~configmap/ovnkube-identity-cm not reset as customized by admin to system_u:object_r:container_file_t:s0:c4,c22 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/ef543e1b-8068-4ea3-b32a-61027b32e95d/volumes/kubernetes.io~configmap/ovnkube-identity-cm/..2025_02_23_05_21_40.3350632666 not reset as customized by admin to system_u:object_r:container_file_t:s0:c4,c22 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/ef543e1b-8068-4ea3-b32a-61027b32e95d/volumes/kubernetes.io~configmap/ovnkube-identity-cm/..2025_02_23_05_21_40.3350632666/additional-cert-acceptance-cond.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c4,c22 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/ef543e1b-8068-4ea3-b32a-61027b32e95d/volumes/kubernetes.io~configmap/ovnkube-identity-cm/..2025_02_23_05_21_40.3350632666/additional-pod-admission-cond.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c4,c22 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/ef543e1b-8068-4ea3-b32a-61027b32e95d/volumes/kubernetes.io~configmap/ovnkube-identity-cm/..data not reset as customized by admin to system_u:object_r:container_file_t:s0:c4,c22 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/ef543e1b-8068-4ea3-b32a-61027b32e95d/volumes/kubernetes.io~configmap/ovnkube-identity-cm/additional-cert-acceptance-cond.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c4,c22 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/ef543e1b-8068-4ea3-b32a-61027b32e95d/volumes/kubernetes.io~configmap/ovnkube-identity-cm/additional-pod-admission-cond.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c4,c22 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/ef543e1b-8068-4ea3-b32a-61027b32e95d/volumes/kubernetes.io~configmap/env-overrides not reset as customized by admin to system_u:object_r:container_file_t:s0:c4,c22 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/ef543e1b-8068-4ea3-b32a-61027b32e95d/volumes/kubernetes.io~configmap/env-overrides/..2025_02_23_05_21_40.1388695756 not reset as customized by admin to system_u:object_r:container_file_t:s0:c4,c22 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/ef543e1b-8068-4ea3-b32a-61027b32e95d/volumes/kubernetes.io~configmap/env-overrides/..data not reset as customized by admin to system_u:object_r:container_file_t:s0:c4,c22 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/ef543e1b-8068-4ea3-b32a-61027b32e95d/etc-hosts not reset as customized by admin to system_u:object_r:container_file_t:s0:c4,c22 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/ef543e1b-8068-4ea3-b32a-61027b32e95d/containers/webhook/26f3df5b not reset as customized by admin to system_u:object_r:container_file_t:s0:c4,c22 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/ef543e1b-8068-4ea3-b32a-61027b32e95d/containers/webhook/6d8fb21d not reset as customized by admin to system_u:object_r:container_file_t:s0:c4,c22 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/ef543e1b-8068-4ea3-b32a-61027b32e95d/containers/webhook/50e94777 not reset as customized by admin to system_u:object_r:container_file_t:s0:c4,c22 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/ef543e1b-8068-4ea3-b32a-61027b32e95d/containers/approver/208473b3 not reset as customized by admin to system_u:object_r:container_file_t:s0:c4,c22 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/ef543e1b-8068-4ea3-b32a-61027b32e95d/containers/approver/ec9e08ba not reset as customized by admin to system_u:object_r:container_file_t:s0:c4,c22 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/ef543e1b-8068-4ea3-b32a-61027b32e95d/containers/approver/3b787c39 not reset as customized by admin to system_u:object_r:container_file_t:s0:c4,c22 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/ef543e1b-8068-4ea3-b32a-61027b32e95d/containers/approver/208eaed5 not reset as customized by admin to system_u:object_r:container_file_t:s0:c4,c22 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/ef543e1b-8068-4ea3-b32a-61027b32e95d/containers/approver/93aa3a2b not reset as customized by admin to system_u:object_r:container_file_t:s0:c4,c22 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/ef543e1b-8068-4ea3-b32a-61027b32e95d/containers/approver/3c697968 not reset as customized by admin to system_u:object_r:container_file_t:s0:c4,c22 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/3b6479f0-333b-4a96-9adf-2099afdc2447/etc-hosts not reset as customized by admin to system_u:object_r:container_file_t:s0:c20,c21 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/3b6479f0-333b-4a96-9adf-2099afdc2447/containers/network-check-target-container/ba950ec9 not reset as customized by admin to system_u:object_r:container_file_t:s0:c20,c21 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/3b6479f0-333b-4a96-9adf-2099afdc2447/containers/network-check-target-container/cb5cdb37 not reset as customized by admin to system_u:object_r:container_file_t:s0:c20,c21 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/3b6479f0-333b-4a96-9adf-2099afdc2447/containers/network-check-target-container/f2df9827 not reset as customized by admin to system_u:object_r:container_file_t:s0:c20,c21 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/31d8b7a1-420e-4252-a5b7-eebe8a111292/volumes/kubernetes.io~configmap/images not reset as customized by admin to system_u:object_r:container_file_t:s0:c4,c17 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/31d8b7a1-420e-4252-a5b7-eebe8a111292/volumes/kubernetes.io~configmap/images/..2025_02_23_05_22_30.473230615 not reset as customized by admin to system_u:object_r:container_file_t:s0:c4,c17 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/31d8b7a1-420e-4252-a5b7-eebe8a111292/volumes/kubernetes.io~configmap/images/..2025_02_23_05_22_30.473230615/images.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c4,c17 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/31d8b7a1-420e-4252-a5b7-eebe8a111292/volumes/kubernetes.io~configmap/images/..data not reset as customized by admin to system_u:object_r:container_file_t:s0:c4,c17 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/31d8b7a1-420e-4252-a5b7-eebe8a111292/volumes/kubernetes.io~configmap/images/images.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c4,c17 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/31d8b7a1-420e-4252-a5b7-eebe8a111292/volumes/kubernetes.io~configmap/auth-proxy-config not reset as customized by admin to system_u:object_r:container_file_t:s0:c4,c17 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/31d8b7a1-420e-4252-a5b7-eebe8a111292/volumes/kubernetes.io~configmap/auth-proxy-config/..data not reset as customized by admin to system_u:object_r:container_file_t:s0:c4,c17 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/31d8b7a1-420e-4252-a5b7-eebe8a111292/volumes/kubernetes.io~configmap/auth-proxy-config/config-file.yaml not reset as customized by admin to system_u:object_r:container_file_t:s0:c4,c17 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/31d8b7a1-420e-4252-a5b7-eebe8a111292/volumes/kubernetes.io~configmap/auth-proxy-config/..2025_02_24_06_22_02.1904938450 not reset as customized by admin to system_u:object_r:container_file_t:s0:c4,c17 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/31d8b7a1-420e-4252-a5b7-eebe8a111292/volumes/kubernetes.io~configmap/auth-proxy-config/..2025_02_24_06_22_02.1904938450/config-file.yaml not reset as customized by admin to system_u:object_r:container_file_t:s0:c4,c17 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/31d8b7a1-420e-4252-a5b7-eebe8a111292/etc-hosts not reset as customized by admin to system_u:object_r:container_file_t:s0:c4,c17 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/31d8b7a1-420e-4252-a5b7-eebe8a111292/containers/machine-config-operator/fedaa673 not reset as customized by admin to system_u:object_r:container_file_t:s0:c4,c17 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/31d8b7a1-420e-4252-a5b7-eebe8a111292/containers/machine-config-operator/9ca2df95 not reset as customized by admin to system_u:object_r:container_file_t:s0:c4,c17 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/31d8b7a1-420e-4252-a5b7-eebe8a111292/containers/machine-config-operator/b2d7460e not reset as customized by admin to system_u:object_r:container_file_t:s0:c4,c17 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/31d8b7a1-420e-4252-a5b7-eebe8a111292/containers/kube-rbac-proxy/2207853c not reset as customized by admin to system_u:object_r:container_file_t:s0:c4,c17 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/31d8b7a1-420e-4252-a5b7-eebe8a111292/containers/kube-rbac-proxy/241c1c29 not reset as customized by admin to system_u:object_r:container_file_t:s0:c4,c17 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/31d8b7a1-420e-4252-a5b7-eebe8a111292/containers/kube-rbac-proxy/2d910eaf not reset as customized by admin to system_u:object_r:container_file_t:s0:c4,c17 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/09efc573-dbb6-4249-bd59-9b87aba8dd28/volumes/kubernetes.io~configmap/etcd-ca not reset as customized by admin to system_u:object_r:container_file_t:s0:c84,c419 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/09efc573-dbb6-4249-bd59-9b87aba8dd28/volumes/kubernetes.io~configmap/etcd-ca/..data not reset as customized by admin to system_u:object_r:container_file_t:s0:c84,c419 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/09efc573-dbb6-4249-bd59-9b87aba8dd28/volumes/kubernetes.io~configmap/etcd-ca/..2025_02_23_05_23_49.3726007728 not reset as customized by admin to system_u:object_r:container_file_t:s0:c84,c419 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/09efc573-dbb6-4249-bd59-9b87aba8dd28/volumes/kubernetes.io~configmap/etcd-ca/..2025_02_23_05_23_49.3726007728/ca-bundle.crt not reset as customized by admin to system_u:object_r:container_file_t:s0:c84,c419 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/09efc573-dbb6-4249-bd59-9b87aba8dd28/volumes/kubernetes.io~configmap/etcd-ca/ca-bundle.crt not reset as customized by admin to system_u:object_r:container_file_t:s0:c84,c419 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/09efc573-dbb6-4249-bd59-9b87aba8dd28/volumes/kubernetes.io~configmap/etcd-service-ca not reset as customized by admin to system_u:object_r:container_file_t:s0:c84,c419 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/09efc573-dbb6-4249-bd59-9b87aba8dd28/volumes/kubernetes.io~configmap/etcd-service-ca/..data not reset as customized by admin to system_u:object_r:container_file_t:s0:c84,c419 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/09efc573-dbb6-4249-bd59-9b87aba8dd28/volumes/kubernetes.io~configmap/etcd-service-ca/..2025_02_23_05_23_49.841175008 not reset as customized by admin to system_u:object_r:container_file_t:s0:c84,c419 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/09efc573-dbb6-4249-bd59-9b87aba8dd28/volumes/kubernetes.io~configmap/etcd-service-ca/..2025_02_23_05_23_49.841175008/service-ca.crt not reset as customized by admin to system_u:object_r:container_file_t:s0:c84,c419 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/09efc573-dbb6-4249-bd59-9b87aba8dd28/volumes/kubernetes.io~configmap/etcd-service-ca/service-ca.crt not reset as customized by admin to system_u:object_r:container_file_t:s0:c84,c419 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/09efc573-dbb6-4249-bd59-9b87aba8dd28/volumes/kubernetes.io~configmap/config not reset as customized by admin to system_u:object_r:container_file_t:s0:c84,c419 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/09efc573-dbb6-4249-bd59-9b87aba8dd28/volumes/kubernetes.io~configmap/config/..2025_02_23_05_22_30.843437178 not reset as customized by admin to system_u:object_r:container_file_t:s0:c84,c419 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/09efc573-dbb6-4249-bd59-9b87aba8dd28/volumes/kubernetes.io~configmap/config/..2025_02_23_05_22_30.843437178/config.yaml not reset as customized by admin to system_u:object_r:container_file_t:s0:c84,c419 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/09efc573-dbb6-4249-bd59-9b87aba8dd28/volumes/kubernetes.io~configmap/config/..data not reset as customized by admin to system_u:object_r:container_file_t:s0:c84,c419 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/09efc573-dbb6-4249-bd59-9b87aba8dd28/volumes/kubernetes.io~configmap/config/config.yaml not reset as customized by admin to system_u:object_r:container_file_t:s0:c84,c419 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/09efc573-dbb6-4249-bd59-9b87aba8dd28/etc-hosts not reset as customized by admin to system_u:object_r:container_file_t:s0:c84,c419 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/09efc573-dbb6-4249-bd59-9b87aba8dd28/containers/etcd-operator/c6c0f2e7 not reset as customized by admin to system_u:object_r:container_file_t:s0:c263,c871 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/09efc573-dbb6-4249-bd59-9b87aba8dd28/containers/etcd-operator/399edc97 not reset as customized by admin to system_u:object_r:container_file_t:s0:c263,c871 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/09efc573-dbb6-4249-bd59-9b87aba8dd28/containers/etcd-operator/8049f7cc not reset as customized by admin to system_u:object_r:container_file_t:s0:c263,c871 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/09efc573-dbb6-4249-bd59-9b87aba8dd28/containers/etcd-operator/0cec5484 not reset as customized by admin to system_u:object_r:container_file_t:s0:c263,c871 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/09efc573-dbb6-4249-bd59-9b87aba8dd28/containers/etcd-operator/312446d0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c406,c828 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/09efc573-dbb6-4249-bd59-9b87aba8dd28/containers/etcd-operator/8e56a35d not reset as customized by admin to system_u:object_r:container_file_t:s0:c84,c419 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/1386a44e-36a2-460c-96d0-0359d2b6f0f5/volumes/kubernetes.io~configmap/config not reset as customized by admin to system_u:object_r:container_file_t:s0:c108,c511 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/1386a44e-36a2-460c-96d0-0359d2b6f0f5/volumes/kubernetes.io~configmap/config/..2025_02_23_05_22_30.133159589 not reset as customized by admin to system_u:object_r:container_file_t:s0:c108,c511 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/1386a44e-36a2-460c-96d0-0359d2b6f0f5/volumes/kubernetes.io~configmap/config/..2025_02_23_05_22_30.133159589/config.yaml not reset as customized by admin to system_u:object_r:container_file_t:s0:c108,c511 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/1386a44e-36a2-460c-96d0-0359d2b6f0f5/volumes/kubernetes.io~configmap/config/..data not reset as customized by admin to system_u:object_r:container_file_t:s0:c108,c511 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/1386a44e-36a2-460c-96d0-0359d2b6f0f5/volumes/kubernetes.io~configmap/config/config.yaml not reset as customized by admin to system_u:object_r:container_file_t:s0:c108,c511 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/1386a44e-36a2-460c-96d0-0359d2b6f0f5/etc-hosts not reset as customized by admin to system_u:object_r:container_file_t:s0:c108,c511 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/1386a44e-36a2-460c-96d0-0359d2b6f0f5/containers/kube-controller-manager-operator/2d30ddb9 not reset as customized by admin to system_u:object_r:container_file_t:s0:c380,c909 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/1386a44e-36a2-460c-96d0-0359d2b6f0f5/containers/kube-controller-manager-operator/eca8053d not reset as customized by admin to system_u:object_r:container_file_t:s0:c380,c909 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/1386a44e-36a2-460c-96d0-0359d2b6f0f5/containers/kube-controller-manager-operator/c3a25c9a not reset as customized by admin to system_u:object_r:container_file_t:s0:c168,c522 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/1386a44e-36a2-460c-96d0-0359d2b6f0f5/containers/kube-controller-manager-operator/b9609c22 not reset as customized by admin to system_u:object_r:container_file_t:s0:c108,c511 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/96b93a3a-6083-4aea-8eab-fe1aa8245ad9/etc-hosts not reset as customized by admin to system_u:object_r:container_file_t:s0:c968,c969 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/96b93a3a-6083-4aea-8eab-fe1aa8245ad9/containers/dns-operator/e8b0eca9 not reset as customized by admin to system_u:object_r:container_file_t:s0:c106,c418 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/96b93a3a-6083-4aea-8eab-fe1aa8245ad9/containers/dns-operator/b36a9c3f not reset as customized by admin to system_u:object_r:container_file_t:s0:c529,c711 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/96b93a3a-6083-4aea-8eab-fe1aa8245ad9/containers/dns-operator/38af7b07 not reset as customized by admin to system_u:object_r:container_file_t:s0:c968,c969 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/96b93a3a-6083-4aea-8eab-fe1aa8245ad9/containers/kube-rbac-proxy/ae821620 not reset as customized by admin to system_u:object_r:container_file_t:s0:c106,c418 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/96b93a3a-6083-4aea-8eab-fe1aa8245ad9/containers/kube-rbac-proxy/baa23338 not reset as customized by admin to system_u:object_r:container_file_t:s0:c529,c711 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/96b93a3a-6083-4aea-8eab-fe1aa8245ad9/containers/kube-rbac-proxy/2c534809 not reset as customized by admin to system_u:object_r:container_file_t:s0:c968,c969 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/496e6271-fb68-4057-954e-a0d97a4afa3f/volumes/kubernetes.io~configmap/config not reset as customized by admin to system_u:object_r:container_file_t:s0:c661,c999 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/496e6271-fb68-4057-954e-a0d97a4afa3f/volumes/kubernetes.io~configmap/config/..2025_02_23_05_22_30.3532625537 not reset as customized by admin to system_u:object_r:container_file_t:s0:c661,c999 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/496e6271-fb68-4057-954e-a0d97a4afa3f/volumes/kubernetes.io~configmap/config/..2025_02_23_05_22_30.3532625537/config.yaml not reset as customized by admin to system_u:object_r:container_file_t:s0:c661,c999 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/496e6271-fb68-4057-954e-a0d97a4afa3f/volumes/kubernetes.io~configmap/config/..data not reset as customized by admin to system_u:object_r:container_file_t:s0:c661,c999 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/496e6271-fb68-4057-954e-a0d97a4afa3f/volumes/kubernetes.io~configmap/config/config.yaml not reset as customized by admin to system_u:object_r:container_file_t:s0:c661,c999 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/496e6271-fb68-4057-954e-a0d97a4afa3f/etc-hosts not reset as customized by admin to system_u:object_r:container_file_t:s0:c661,c999 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/496e6271-fb68-4057-954e-a0d97a4afa3f/containers/kube-scheduler-operator-container/59b29eae not reset as customized by admin to system_u:object_r:container_file_t:s0:c338,c381 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/496e6271-fb68-4057-954e-a0d97a4afa3f/containers/kube-scheduler-operator-container/c91a8e4f not reset as customized by admin to system_u:object_r:container_file_t:s0:c338,c381 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/496e6271-fb68-4057-954e-a0d97a4afa3f/containers/kube-scheduler-operator-container/4d87494a not reset as customized by admin to system_u:object_r:container_file_t:s0:c442,c857 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/496e6271-fb68-4057-954e-a0d97a4afa3f/containers/kube-scheduler-operator-container/1e33ca63 not reset as customized by admin to system_u:object_r:container_file_t:s0:c661,c999 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/3ab1a177-2de0-46d9-b765-d0d0649bb42e/etc-hosts not reset as customized by admin to system_u:object_r:container_file_t:s0:c12,c18 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/3ab1a177-2de0-46d9-b765-d0d0649bb42e/containers/kube-rbac-proxy/8dea7be2 not reset as customized by admin to system_u:object_r:container_file_t:s0:c12,c18 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/3ab1a177-2de0-46d9-b765-d0d0649bb42e/containers/kube-rbac-proxy/d0b04a99 not reset as customized by admin to system_u:object_r:container_file_t:s0:c12,c18 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/3ab1a177-2de0-46d9-b765-d0d0649bb42e/containers/kube-rbac-proxy/d84f01e7 not reset as customized by admin to system_u:object_r:container_file_t:s0:c12,c18 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/3ab1a177-2de0-46d9-b765-d0d0649bb42e/containers/package-server-manager/4109059b not reset as customized by admin to system_u:object_r:container_file_t:s0:c12,c18 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/3ab1a177-2de0-46d9-b765-d0d0649bb42e/containers/package-server-manager/a7258a3e not reset as customized by admin to system_u:object_r:container_file_t:s0:c12,c18 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/3ab1a177-2de0-46d9-b765-d0d0649bb42e/containers/package-server-manager/05bdf2b6 not reset as customized by admin to system_u:object_r:container_file_t:s0:c12,c18 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/6731426b-95fe-49ff-bb5f-40441049fde2/etc-hosts not reset as customized by admin to system_u:object_r:container_file_t:s0:c0,c15 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/6731426b-95fe-49ff-bb5f-40441049fde2/containers/control-plane-machine-set-operator/f3261b51 not reset as customized by admin to system_u:object_r:container_file_t:s0:c0,c15 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/6731426b-95fe-49ff-bb5f-40441049fde2/containers/control-plane-machine-set-operator/315d045e not reset as customized by admin to system_u:object_r:container_file_t:s0:c0,c15 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/6731426b-95fe-49ff-bb5f-40441049fde2/containers/control-plane-machine-set-operator/5fdcf278 not reset as customized by admin to system_u:object_r:container_file_t:s0:c0,c15 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/6731426b-95fe-49ff-bb5f-40441049fde2/containers/control-plane-machine-set-operator/d053f757 not reset as customized by admin to system_u:object_r:container_file_t:s0:c0,c15 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/6731426b-95fe-49ff-bb5f-40441049fde2/containers/control-plane-machine-set-operator/c2850dc7 not reset as customized by admin to system_u:object_r:container_file_t:s0:c0,c15 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/b6cd30de-2eeb-49a2-ab40-9167f4560ff5/volumes/kubernetes.io~configmap/marketplace-trusted-ca not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/b6cd30de-2eeb-49a2-ab40-9167f4560ff5/volumes/kubernetes.io~configmap/marketplace-trusted-ca/..2025_02_23_05_22_30.2390596521 not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/b6cd30de-2eeb-49a2-ab40-9167f4560ff5/volumes/kubernetes.io~configmap/marketplace-trusted-ca/..2025_02_23_05_22_30.2390596521/tls-ca-bundle.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/b6cd30de-2eeb-49a2-ab40-9167f4560ff5/volumes/kubernetes.io~configmap/marketplace-trusted-ca/..data not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/b6cd30de-2eeb-49a2-ab40-9167f4560ff5/volumes/kubernetes.io~configmap/marketplace-trusted-ca/tls-ca-bundle.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/b6cd30de-2eeb-49a2-ab40-9167f4560ff5/etc-hosts not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/b6cd30de-2eeb-49a2-ab40-9167f4560ff5/containers/marketplace-operator/fcfb0b2b not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/b6cd30de-2eeb-49a2-ab40-9167f4560ff5/containers/marketplace-operator/c7ac9b7d not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/b6cd30de-2eeb-49a2-ab40-9167f4560ff5/containers/marketplace-operator/fa0c0d52 not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/b6cd30de-2eeb-49a2-ab40-9167f4560ff5/containers/marketplace-operator/c609b6ba not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/b6cd30de-2eeb-49a2-ab40-9167f4560ff5/containers/marketplace-operator/2be6c296 not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/b6cd30de-2eeb-49a2-ab40-9167f4560ff5/containers/marketplace-operator/89a32653 not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/b6cd30de-2eeb-49a2-ab40-9167f4560ff5/containers/marketplace-operator/4eb9afeb not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/b6cd30de-2eeb-49a2-ab40-9167f4560ff5/containers/marketplace-operator/13af6efa not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/b6312bbd-5731-4ea0-a20f-81d5a57df44a/etc-hosts not reset as customized by admin to system_u:object_r:container_file_t:s0:c12,c18 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/b6312bbd-5731-4ea0-a20f-81d5a57df44a/containers/olm-operator/b03f9724 not reset as customized by admin to system_u:object_r:container_file_t:s0:c12,c18 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/b6312bbd-5731-4ea0-a20f-81d5a57df44a/containers/olm-operator/e3d105cc not reset as customized by admin to system_u:object_r:container_file_t:s0:c12,c18 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/b6312bbd-5731-4ea0-a20f-81d5a57df44a/containers/olm-operator/3aed4d83 not reset as customized by admin to system_u:object_r:container_file_t:s0:c12,c18 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/210d8245-ebfc-4e3b-ac4a-e21ce76f9a7c/volumes/kubernetes.io~configmap/config not reset as customized by admin to system_u:object_r:container_file_t:s0:c2,c18 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/210d8245-ebfc-4e3b-ac4a-e21ce76f9a7c/volumes/kubernetes.io~configmap/config/..2025_02_23_05_22_30.1906041176 not reset as customized by admin to system_u:object_r:container_file_t:s0:c2,c18 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/210d8245-ebfc-4e3b-ac4a-e21ce76f9a7c/volumes/kubernetes.io~configmap/config/..2025_02_23_05_22_30.1906041176/config.yaml not reset as customized by admin to system_u:object_r:container_file_t:s0:c2,c18 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/210d8245-ebfc-4e3b-ac4a-e21ce76f9a7c/volumes/kubernetes.io~configmap/config/..data not reset as customized by admin to system_u:object_r:container_file_t:s0:c2,c18 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/210d8245-ebfc-4e3b-ac4a-e21ce76f9a7c/volumes/kubernetes.io~configmap/config/config.yaml not reset as customized by admin to system_u:object_r:container_file_t:s0:c2,c18 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/210d8245-ebfc-4e3b-ac4a-e21ce76f9a7c/etc-hosts not reset as customized by admin to system_u:object_r:container_file_t:s0:c2,c18 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/210d8245-ebfc-4e3b-ac4a-e21ce76f9a7c/containers/kube-storage-version-migrator-operator/0765fa6e not reset as customized by admin to system_u:object_r:container_file_t:s0:c2,c18 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/210d8245-ebfc-4e3b-ac4a-e21ce76f9a7c/containers/kube-storage-version-migrator-operator/2cefc627 not reset as customized by admin to system_u:object_r:container_file_t:s0:c2,c18 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/210d8245-ebfc-4e3b-ac4a-e21ce76f9a7c/containers/kube-storage-version-migrator-operator/3dcc6345 not reset as customized by admin to system_u:object_r:container_file_t:s0:c2,c18 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/210d8245-ebfc-4e3b-ac4a-e21ce76f9a7c/containers/kube-storage-version-migrator-operator/365af391 not reset as customized by admin to system_u:object_r:container_file_t:s0:c2,c18 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/bc5039c0-ea34-426b-a2b7-fbbc87b49a6d/volumes/kubernetes.io~empty-dir/available-featuregates not reset as customized by admin to system_u:object_r:container_file_t:s0:c9,c12 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/bc5039c0-ea34-426b-a2b7-fbbc87b49a6d/volumes/kubernetes.io~empty-dir/available-featuregates/featureGate-SelfManagedHA-Default.yaml not reset as customized by admin to system_u:object_r:container_file_t:s0:c9,c12 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/bc5039c0-ea34-426b-a2b7-fbbc87b49a6d/volumes/kubernetes.io~empty-dir/available-featuregates/featureGate-SelfManagedHA-TechPreviewNoUpgrade.yaml not reset as customized by admin to system_u:object_r:container_file_t:s0:c9,c12 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/bc5039c0-ea34-426b-a2b7-fbbc87b49a6d/volumes/kubernetes.io~empty-dir/available-featuregates/featureGate-SelfManagedHA-DevPreviewNoUpgrade.yaml not reset as customized by admin to system_u:object_r:container_file_t:s0:c9,c12 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/bc5039c0-ea34-426b-a2b7-fbbc87b49a6d/volumes/kubernetes.io~empty-dir/available-featuregates/featureGate-Hypershift-TechPreviewNoUpgrade.yaml not reset as customized by admin to system_u:object_r:container_file_t:s0:c9,c12 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/bc5039c0-ea34-426b-a2b7-fbbc87b49a6d/volumes/kubernetes.io~empty-dir/available-featuregates/featureGate-Hypershift-DevPreviewNoUpgrade.yaml not reset as customized by admin to system_u:object_r:container_file_t:s0:c9,c12 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/bc5039c0-ea34-426b-a2b7-fbbc87b49a6d/volumes/kubernetes.io~empty-dir/available-featuregates/featureGate-Hypershift-Default.yaml not reset as customized by admin to system_u:object_r:container_file_t:s0:c9,c12 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/bc5039c0-ea34-426b-a2b7-fbbc87b49a6d/etc-hosts not reset as customized by admin to system_u:object_r:container_file_t:s0:c9,c12 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/bc5039c0-ea34-426b-a2b7-fbbc87b49a6d/containers/openshift-api/b1130c0f not reset as customized by admin to system_u:object_r:container_file_t:s0:c9,c12 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/bc5039c0-ea34-426b-a2b7-fbbc87b49a6d/containers/openshift-api/236a5913 not reset as customized by admin to system_u:object_r:container_file_t:s0:c9,c12 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/bc5039c0-ea34-426b-a2b7-fbbc87b49a6d/containers/openshift-api/b9432e26 not reset as customized by admin to system_u:object_r:container_file_t:s0:c9,c12 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/bc5039c0-ea34-426b-a2b7-fbbc87b49a6d/containers/openshift-config-operator/5ddb0e3f not reset as customized by admin to system_u:object_r:container_file_t:s0:c9,c12 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/bc5039c0-ea34-426b-a2b7-fbbc87b49a6d/containers/openshift-config-operator/986dc4fd not reset as customized by admin to system_u:object_r:container_file_t:s0:c9,c12 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/bc5039c0-ea34-426b-a2b7-fbbc87b49a6d/containers/openshift-config-operator/8a23ff9a not reset as customized by admin to system_u:object_r:container_file_t:s0:c9,c12 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/bc5039c0-ea34-426b-a2b7-fbbc87b49a6d/containers/openshift-config-operator/9728ae68 not reset as customized by admin to system_u:object_r:container_file_t:s0:c9,c12 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/bc5039c0-ea34-426b-a2b7-fbbc87b49a6d/containers/openshift-config-operator/665f31d0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c9,c12 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/6509e943-70c6-444c-bc41-48a544e36fbd/volumes/kubernetes.io~configmap/config not reset as customized by admin to system_u:object_r:container_file_t:s0:c0,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/6509e943-70c6-444c-bc41-48a544e36fbd/volumes/kubernetes.io~configmap/config/..2025_02_23_05_22_30.1255385357 not reset as customized by admin to system_u:object_r:container_file_t:s0:c0,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/6509e943-70c6-444c-bc41-48a544e36fbd/volumes/kubernetes.io~configmap/config/..2025_02_23_05_22_30.1255385357/operator-config.yaml not reset as customized by admin to system_u:object_r:container_file_t:s0:c0,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/6509e943-70c6-444c-bc41-48a544e36fbd/volumes/kubernetes.io~configmap/config/..data not reset as customized by admin to system_u:object_r:container_file_t:s0:c0,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/6509e943-70c6-444c-bc41-48a544e36fbd/volumes/kubernetes.io~configmap/config/operator-config.yaml not reset as customized by admin to system_u:object_r:container_file_t:s0:c0,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/6509e943-70c6-444c-bc41-48a544e36fbd/volumes/kubernetes.io~configmap/service-ca-bundle not reset as customized by admin to system_u:object_r:container_file_t:s0:c0,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/6509e943-70c6-444c-bc41-48a544e36fbd/volumes/kubernetes.io~configmap/service-ca-bundle/..data not reset as customized by admin to system_u:object_r:container_file_t:s0:c0,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/6509e943-70c6-444c-bc41-48a544e36fbd/volumes/kubernetes.io~configmap/service-ca-bundle/..2025_02_23_05_23_57.573792656 not reset as customized by admin to system_u:object_r:container_file_t:s0:c0,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/6509e943-70c6-444c-bc41-48a544e36fbd/volumes/kubernetes.io~configmap/service-ca-bundle/..2025_02_23_05_23_57.573792656/service-ca.crt not reset as customized by admin to system_u:object_r:container_file_t:s0:c0,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/6509e943-70c6-444c-bc41-48a544e36fbd/volumes/kubernetes.io~configmap/service-ca-bundle/service-ca.crt not reset as customized by admin to system_u:object_r:container_file_t:s0:c0,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/6509e943-70c6-444c-bc41-48a544e36fbd/volumes/kubernetes.io~configmap/trusted-ca-bundle not reset as customized by admin to system_u:object_r:container_file_t:s0:c0,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/6509e943-70c6-444c-bc41-48a544e36fbd/volumes/kubernetes.io~configmap/trusted-ca-bundle/..2025_02_23_05_22_30.3254245399 not reset as customized by admin to system_u:object_r:container_file_t:s0:c0,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/6509e943-70c6-444c-bc41-48a544e36fbd/volumes/kubernetes.io~configmap/trusted-ca-bundle/..2025_02_23_05_22_30.3254245399/ca-bundle.crt not reset as customized by admin to system_u:object_r:container_file_t:s0:c0,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/6509e943-70c6-444c-bc41-48a544e36fbd/volumes/kubernetes.io~configmap/trusted-ca-bundle/..data not reset as customized by admin to system_u:object_r:container_file_t:s0:c0,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/6509e943-70c6-444c-bc41-48a544e36fbd/volumes/kubernetes.io~configmap/trusted-ca-bundle/ca-bundle.crt not reset as customized by admin to system_u:object_r:container_file_t:s0:c0,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/6509e943-70c6-444c-bc41-48a544e36fbd/etc-hosts not reset as customized by admin to system_u:object_r:container_file_t:s0:c0,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/6509e943-70c6-444c-bc41-48a544e36fbd/containers/authentication-operator/136c9b42 not reset as customized by admin to system_u:object_r:container_file_t:s0:c0,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/6509e943-70c6-444c-bc41-48a544e36fbd/containers/authentication-operator/98a1575b not reset as customized by admin to system_u:object_r:container_file_t:s0:c0,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/6509e943-70c6-444c-bc41-48a544e36fbd/containers/authentication-operator/cac69136 not reset as customized by admin to system_u:object_r:container_file_t:s0:c0,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/6509e943-70c6-444c-bc41-48a544e36fbd/containers/authentication-operator/5deb77a7 not reset as customized by admin to system_u:object_r:container_file_t:s0:c0,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/6509e943-70c6-444c-bc41-48a544e36fbd/containers/authentication-operator/2ae53400 not reset as customized by admin to system_u:object_r:container_file_t:s0:c0,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/01ab3dd5-8196-46d0-ad33-122e2ca51def/volumes/kubernetes.io~configmap/config not reset as customized by admin to system_u:object_r:container_file_t:s0:c5,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/01ab3dd5-8196-46d0-ad33-122e2ca51def/volumes/kubernetes.io~configmap/config/..2025_02_23_05_22_30.3608339744 not reset as customized by admin to system_u:object_r:container_file_t:s0:c5,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/01ab3dd5-8196-46d0-ad33-122e2ca51def/volumes/kubernetes.io~configmap/config/..2025_02_23_05_22_30.3608339744/operator-config.yaml not reset as customized by admin to system_u:object_r:container_file_t:s0:c5,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/01ab3dd5-8196-46d0-ad33-122e2ca51def/volumes/kubernetes.io~configmap/config/..data not reset as customized by admin to system_u:object_r:container_file_t:s0:c5,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/01ab3dd5-8196-46d0-ad33-122e2ca51def/volumes/kubernetes.io~configmap/config/operator-config.yaml not reset as customized by admin to system_u:object_r:container_file_t:s0:c5,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/01ab3dd5-8196-46d0-ad33-122e2ca51def/etc-hosts not reset as customized by admin to system_u:object_r:container_file_t:s0:c5,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/01ab3dd5-8196-46d0-ad33-122e2ca51def/containers/service-ca-operator/e46f2326 not reset as customized by admin to system_u:object_r:container_file_t:s0:c5,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/01ab3dd5-8196-46d0-ad33-122e2ca51def/containers/service-ca-operator/dc688d3c not reset as customized by admin to system_u:object_r:container_file_t:s0:c5,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/01ab3dd5-8196-46d0-ad33-122e2ca51def/containers/service-ca-operator/3497c3cd not reset as customized by admin to system_u:object_r:container_file_t:s0:c5,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/01ab3dd5-8196-46d0-ad33-122e2ca51def/containers/service-ca-operator/177eb008 not reset as customized by admin to system_u:object_r:container_file_t:s0:c5,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/7539238d-5fe0-46ed-884e-1c3b566537ec/volumes/kubernetes.io~configmap/config not reset as customized by admin to system_u:object_r:container_file_t:s0:c2,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/7539238d-5fe0-46ed-884e-1c3b566537ec/volumes/kubernetes.io~configmap/config/..2025_02_23_05_22_30.3819292994 not reset as customized by admin to system_u:object_r:container_file_t:s0:c2,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/7539238d-5fe0-46ed-884e-1c3b566537ec/volumes/kubernetes.io~configmap/config/..2025_02_23_05_22_30.3819292994/config.yaml not reset as customized by admin to system_u:object_r:container_file_t:s0:c2,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/7539238d-5fe0-46ed-884e-1c3b566537ec/volumes/kubernetes.io~configmap/config/..data not reset as customized by admin to system_u:object_r:container_file_t:s0:c2,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/7539238d-5fe0-46ed-884e-1c3b566537ec/volumes/kubernetes.io~configmap/config/config.yaml not reset as customized by admin to system_u:object_r:container_file_t:s0:c2,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/7539238d-5fe0-46ed-884e-1c3b566537ec/etc-hosts not reset as customized by admin to system_u:object_r:container_file_t:s0:c2,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/7539238d-5fe0-46ed-884e-1c3b566537ec/containers/openshift-apiserver-operator/af5a2afa not reset as customized by admin to system_u:object_r:container_file_t:s0:c2,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/7539238d-5fe0-46ed-884e-1c3b566537ec/containers/openshift-apiserver-operator/d780cb1f not reset as customized by admin to system_u:object_r:container_file_t:s0:c2,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/7539238d-5fe0-46ed-884e-1c3b566537ec/containers/openshift-apiserver-operator/49b0f374 not reset as customized by admin to system_u:object_r:container_file_t:s0:c2,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/7539238d-5fe0-46ed-884e-1c3b566537ec/containers/openshift-apiserver-operator/26fbb125 not reset as customized by admin to system_u:object_r:container_file_t:s0:c2,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/a31745f5-9847-4afe-82a5-3161cc66ca93/volumes/kubernetes.io~configmap/trusted-ca not reset as customized by admin to system_u:object_r:container_file_t:s0:c5,c11 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/a31745f5-9847-4afe-82a5-3161cc66ca93/volumes/kubernetes.io~configmap/trusted-ca/..2025_02_23_05_22_30.3244779536 not reset as customized by admin to system_u:object_r:container_file_t:s0:c5,c11 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/a31745f5-9847-4afe-82a5-3161cc66ca93/volumes/kubernetes.io~configmap/trusted-ca/..2025_02_23_05_22_30.3244779536/tls-ca-bundle.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c5,c11 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/a31745f5-9847-4afe-82a5-3161cc66ca93/volumes/kubernetes.io~configmap/trusted-ca/..data not reset as customized by admin to system_u:object_r:container_file_t:s0:c5,c11 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/a31745f5-9847-4afe-82a5-3161cc66ca93/volumes/kubernetes.io~configmap/trusted-ca/tls-ca-bundle.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c5,c11 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/a31745f5-9847-4afe-82a5-3161cc66ca93/etc-hosts not reset as customized by admin to system_u:object_r:container_file_t:s0:c5,c11 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/a31745f5-9847-4afe-82a5-3161cc66ca93/containers/ingress-operator/cf14125a not reset as customized by admin to system_u:object_r:container_file_t:s0:c5,c11 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/a31745f5-9847-4afe-82a5-3161cc66ca93/containers/ingress-operator/b7f86972 not reset as customized by admin to system_u:object_r:container_file_t:s0:c5,c11 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/a31745f5-9847-4afe-82a5-3161cc66ca93/containers/ingress-operator/e51d739c not reset as customized by admin to system_u:object_r:container_file_t:s0:c5,c11 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/a31745f5-9847-4afe-82a5-3161cc66ca93/containers/ingress-operator/88ba6a69 not reset as customized by admin to system_u:object_r:container_file_t:s0:c5,c11 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/a31745f5-9847-4afe-82a5-3161cc66ca93/containers/ingress-operator/669a9acf not reset as customized by admin to system_u:object_r:container_file_t:s0:c5,c11 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/a31745f5-9847-4afe-82a5-3161cc66ca93/containers/ingress-operator/5cd51231 not reset as customized by admin to system_u:object_r:container_file_t:s0:c5,c11 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/a31745f5-9847-4afe-82a5-3161cc66ca93/containers/ingress-operator/75349ec7 not reset as customized by admin to system_u:object_r:container_file_t:s0:c5,c11 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/a31745f5-9847-4afe-82a5-3161cc66ca93/containers/ingress-operator/15c26839 not reset as customized by admin to system_u:object_r:container_file_t:s0:c5,c11 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/a31745f5-9847-4afe-82a5-3161cc66ca93/containers/ingress-operator/45023dcd not reset as customized by admin to system_u:object_r:container_file_t:s0:c5,c11 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/a31745f5-9847-4afe-82a5-3161cc66ca93/containers/ingress-operator/2bb66a50 not reset as customized by admin to system_u:object_r:container_file_t:s0:c5,c11 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/a31745f5-9847-4afe-82a5-3161cc66ca93/containers/kube-rbac-proxy/64d03bdd not reset as customized by admin to system_u:object_r:container_file_t:s0:c5,c11 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/a31745f5-9847-4afe-82a5-3161cc66ca93/containers/kube-rbac-proxy/ab8e7ca0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c5,c11 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/a31745f5-9847-4afe-82a5-3161cc66ca93/containers/kube-rbac-proxy/bb9be25f not reset as customized by admin to system_u:object_r:container_file_t:s0:c5,c11 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/bf126b07-da06-4140-9a57-dfd54fc6b486/volumes/kubernetes.io~configmap/trusted-ca not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/bf126b07-da06-4140-9a57-dfd54fc6b486/volumes/kubernetes.io~configmap/trusted-ca/..2025_02_23_05_22_30.2034221258 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/bf126b07-da06-4140-9a57-dfd54fc6b486/volumes/kubernetes.io~configmap/trusted-ca/..2025_02_23_05_22_30.2034221258/tls-ca-bundle.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/bf126b07-da06-4140-9a57-dfd54fc6b486/volumes/kubernetes.io~configmap/trusted-ca/..data not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/bf126b07-da06-4140-9a57-dfd54fc6b486/volumes/kubernetes.io~configmap/trusted-ca/tls-ca-bundle.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/bf126b07-da06-4140-9a57-dfd54fc6b486/etc-hosts not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/bf126b07-da06-4140-9a57-dfd54fc6b486/containers/cluster-image-registry-operator/9a0b61d3 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/bf126b07-da06-4140-9a57-dfd54fc6b486/containers/cluster-image-registry-operator/d471b9d2 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/bf126b07-da06-4140-9a57-dfd54fc6b486/containers/cluster-image-registry-operator/8cb76b8e not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/f88749ec-7931-4ee7-b3fc-1ec5e11f92e9/etc-hosts not reset as customized by admin to system_u:object_r:container_file_t:s0:c12,c18 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/f88749ec-7931-4ee7-b3fc-1ec5e11f92e9/containers/catalog-operator/11a00840 not reset as customized by admin to system_u:object_r:container_file_t:s0:c12,c18 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/f88749ec-7931-4ee7-b3fc-1ec5e11f92e9/containers/catalog-operator/ec355a92 not reset as customized by admin to system_u:object_r:container_file_t:s0:c12,c18 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/f88749ec-7931-4ee7-b3fc-1ec5e11f92e9/containers/catalog-operator/992f735e not reset as customized by admin to system_u:object_r:container_file_t:s0:c12,c18 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8cea82b4-6893-4ddc-af9f-1bb5ae425c5b/volumes/kubernetes.io~configmap/config not reset as customized by admin to system_u:object_r:container_file_t:s0:c9,c14 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8cea82b4-6893-4ddc-af9f-1bb5ae425c5b/volumes/kubernetes.io~configmap/config/..2025_02_23_05_22_30.1782968797 not reset as customized by admin to system_u:object_r:container_file_t:s0:c9,c14 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8cea82b4-6893-4ddc-af9f-1bb5ae425c5b/volumes/kubernetes.io~configmap/config/..2025_02_23_05_22_30.1782968797/config.yaml not reset as customized by admin to system_u:object_r:container_file_t:s0:c9,c14 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8cea82b4-6893-4ddc-af9f-1bb5ae425c5b/volumes/kubernetes.io~configmap/config/..data not reset as customized by admin to system_u:object_r:container_file_t:s0:c9,c14 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8cea82b4-6893-4ddc-af9f-1bb5ae425c5b/volumes/kubernetes.io~configmap/config/config.yaml not reset as customized by admin to system_u:object_r:container_file_t:s0:c9,c14 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8cea82b4-6893-4ddc-af9f-1bb5ae425c5b/etc-hosts not reset as customized by admin to system_u:object_r:container_file_t:s0:c9,c14 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8cea82b4-6893-4ddc-af9f-1bb5ae425c5b/containers/openshift-controller-manager-operator/d59cdbbc not reset as customized by admin to system_u:object_r:container_file_t:s0:c9,c14 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8cea82b4-6893-4ddc-af9f-1bb5ae425c5b/containers/openshift-controller-manager-operator/72133ff0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c9,c14 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8cea82b4-6893-4ddc-af9f-1bb5ae425c5b/containers/openshift-controller-manager-operator/c56c834c not reset as customized by admin to system_u:object_r:container_file_t:s0:c9,c14 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8cea82b4-6893-4ddc-af9f-1bb5ae425c5b/containers/openshift-controller-manager-operator/d13724c7 not reset as customized by admin to system_u:object_r:container_file_t:s0:c9,c14 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8cea82b4-6893-4ddc-af9f-1bb5ae425c5b/containers/openshift-controller-manager-operator/0a498258 not reset as customized by admin to system_u:object_r:container_file_t:s0:c9,c14 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/5fe579f8-e8a6-4643-bce5-a661393c4dde/etc-hosts not reset as customized by admin to system_u:object_r:container_file_t:s0:c4,c17 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/5fe579f8-e8a6-4643-bce5-a661393c4dde/containers/machine-config-server/fa471982 not reset as customized by admin to system_u:object_r:container_file_t:s0:c4,c17 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/5fe579f8-e8a6-4643-bce5-a661393c4dde/containers/machine-config-server/fc900d92 not reset as customized by admin to system_u:object_r:container_file_t:s0:c4,c17 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/5fe579f8-e8a6-4643-bce5-a661393c4dde/containers/machine-config-server/fa7d68da not reset as customized by admin to system_u:object_r:container_file_t:s0:c4,c17 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/49ef4625-1d3a-4a9f-b595-c2433d32326d/etc-hosts not reset as customized by admin to system_u:object_r:container_file_t:s0:c9,c22 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/49ef4625-1d3a-4a9f-b595-c2433d32326d/containers/migrator/4bacf9b4 not reset as customized by admin to system_u:object_r:container_file_t:s0:c9,c22 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/49ef4625-1d3a-4a9f-b595-c2433d32326d/containers/migrator/424021b1 not reset as customized by admin to system_u:object_r:container_file_t:s0:c9,c22 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/49ef4625-1d3a-4a9f-b595-c2433d32326d/containers/migrator/fc2e31a3 not reset as customized by admin to system_u:object_r:container_file_t:s0:c9,c22 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/49ef4625-1d3a-4a9f-b595-c2433d32326d/containers/graceful-termination/f51eefac not reset as customized by admin to system_u:object_r:container_file_t:s0:c9,c22 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/49ef4625-1d3a-4a9f-b595-c2433d32326d/containers/graceful-termination/c8997f2f not reset as customized by admin to system_u:object_r:container_file_t:s0:c9,c22 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/49ef4625-1d3a-4a9f-b595-c2433d32326d/containers/graceful-termination/7481f599 not reset as customized by admin to system_u:object_r:container_file_t:s0:c9,c22 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/25e176fe-21b4-4974-b1ed-c8b94f112a7f/volumes/kubernetes.io~configmap/signing-cabundle not reset as customized by admin to system_u:object_r:container_file_t:s0:c19,c22 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/25e176fe-21b4-4974-b1ed-c8b94f112a7f/volumes/kubernetes.io~configmap/signing-cabundle/..2025_02_23_05_22_49.2255460704 not reset as customized by admin to system_u:object_r:container_file_t:s0:c19,c22 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/25e176fe-21b4-4974-b1ed-c8b94f112a7f/volumes/kubernetes.io~configmap/signing-cabundle/..2025_02_23_05_22_49.2255460704/ca-bundle.crt not reset as customized by admin to system_u:object_r:container_file_t:s0:c19,c22 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/25e176fe-21b4-4974-b1ed-c8b94f112a7f/volumes/kubernetes.io~configmap/signing-cabundle/..data not reset as customized by admin to system_u:object_r:container_file_t:s0:c19,c22 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/25e176fe-21b4-4974-b1ed-c8b94f112a7f/volumes/kubernetes.io~configmap/signing-cabundle/ca-bundle.crt not reset as customized by admin to system_u:object_r:container_file_t:s0:c19,c22 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/25e176fe-21b4-4974-b1ed-c8b94f112a7f/etc-hosts not reset as customized by admin to system_u:object_r:container_file_t:s0:c19,c22 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/25e176fe-21b4-4974-b1ed-c8b94f112a7f/containers/service-ca-controller/fdafea19 not reset as customized by admin to system_u:object_r:container_file_t:s0:c19,c22 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/25e176fe-21b4-4974-b1ed-c8b94f112a7f/containers/service-ca-controller/d0e1c571 not reset as customized by admin to system_u:object_r:container_file_t:s0:c19,c22 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/25e176fe-21b4-4974-b1ed-c8b94f112a7f/containers/service-ca-controller/ee398915 not reset as customized by admin to system_u:object_r:container_file_t:s0:c19,c22 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/25e176fe-21b4-4974-b1ed-c8b94f112a7f/containers/service-ca-controller/682bb6b8 not reset as customized by admin to system_u:object_r:container_file_t:s0:c19,c22 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/2139d3e2895fc6797b9c76a1b4c9886d/etc-hosts not reset as customized by admin to system_u:object_r:container_file_t:s0:c666,c920 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/2139d3e2895fc6797b9c76a1b4c9886d/containers/setup/a3e67855 not reset as customized by admin to system_u:object_r:container_file_t:s0:c294,c884 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/2139d3e2895fc6797b9c76a1b4c9886d/containers/setup/a989f289 not reset as customized by admin to system_u:object_r:container_file_t:s0:c336,c1016 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/2139d3e2895fc6797b9c76a1b4c9886d/containers/setup/915431bd not reset as customized by admin to system_u:object_r:container_file_t:s0:c666,c920 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/2139d3e2895fc6797b9c76a1b4c9886d/containers/etcd-ensure-env-vars/7796fdab not reset as customized by admin to system_u:object_r:container_file_t:s0:c294,c884 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/2139d3e2895fc6797b9c76a1b4c9886d/containers/etcd-ensure-env-vars/dcdb5f19 not reset as customized by admin to system_u:object_r:container_file_t:s0:c336,c1016 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/2139d3e2895fc6797b9c76a1b4c9886d/containers/etcd-ensure-env-vars/a3aaa88c not reset as customized by admin to system_u:object_r:container_file_t:s0:c666,c920 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/2139d3e2895fc6797b9c76a1b4c9886d/containers/etcd-resources-copy/5508e3e6 not reset as customized by admin to system_u:object_r:container_file_t:s0:c294,c884 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/2139d3e2895fc6797b9c76a1b4c9886d/containers/etcd-resources-copy/160585de not reset as customized by admin to system_u:object_r:container_file_t:s0:c336,c1016 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/2139d3e2895fc6797b9c76a1b4c9886d/containers/etcd-resources-copy/e99f8da3 not reset as customized by admin to system_u:object_r:container_file_t:s0:c666,c920 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/2139d3e2895fc6797b9c76a1b4c9886d/containers/etcdctl/8bc85570 not reset as customized by admin to system_u:object_r:container_file_t:s0:c294,c884 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/2139d3e2895fc6797b9c76a1b4c9886d/containers/etcdctl/a5861c91 not reset as customized by admin to system_u:object_r:container_file_t:s0:c336,c1016 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/2139d3e2895fc6797b9c76a1b4c9886d/containers/etcdctl/84db1135 not reset as customized by admin to system_u:object_r:container_file_t:s0:c666,c920 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/2139d3e2895fc6797b9c76a1b4c9886d/containers/etcd/9e1a6043 not reset as customized by admin to system_u:object_r:container_file_t:s0:c294,c884 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/2139d3e2895fc6797b9c76a1b4c9886d/containers/etcd/c1aba1c2 not reset as customized by admin to system_u:object_r:container_file_t:s0:c336,c1016 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/2139d3e2895fc6797b9c76a1b4c9886d/containers/etcd/d55ccd6d not reset as customized by admin to system_u:object_r:container_file_t:s0:c666,c920 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/2139d3e2895fc6797b9c76a1b4c9886d/containers/etcd-metrics/971cc9f6 not reset as customized by admin to system_u:object_r:container_file_t:s0:c294,c884 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/2139d3e2895fc6797b9c76a1b4c9886d/containers/etcd-metrics/8f2e3dcf not reset as customized by admin to system_u:object_r:container_file_t:s0:c336,c1016 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/2139d3e2895fc6797b9c76a1b4c9886d/containers/etcd-metrics/ceb35e9c not reset as customized by admin to system_u:object_r:container_file_t:s0:c666,c920 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/2139d3e2895fc6797b9c76a1b4c9886d/containers/etcd-readyz/1c192745 not reset as customized by admin to system_u:object_r:container_file_t:s0:c294,c884 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/2139d3e2895fc6797b9c76a1b4c9886d/containers/etcd-readyz/5209e501 not reset as customized by admin to system_u:object_r:container_file_t:s0:c336,c1016 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/2139d3e2895fc6797b9c76a1b4c9886d/containers/etcd-readyz/f83de4df not reset as customized by admin to system_u:object_r:container_file_t:s0:c666,c920 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/2139d3e2895fc6797b9c76a1b4c9886d/containers/etcd-rev/e7b978ac not reset as customized by admin to system_u:object_r:container_file_t:s0:c294,c884 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/2139d3e2895fc6797b9c76a1b4c9886d/containers/etcd-rev/c64304a1 not reset as customized by admin to system_u:object_r:container_file_t:s0:c336,c1016 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/2139d3e2895fc6797b9c76a1b4c9886d/containers/etcd-rev/5384386b not reset as customized by admin to system_u:object_r:container_file_t:s0:c666,c920 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/efdd0498-1daa-4136-9a4a-3b948c2293fc/etc-hosts not reset as customized by admin to system_u:object_r:container_file_t:s0:c268,c620 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/efdd0498-1daa-4136-9a4a-3b948c2293fc/containers/multus-admission-controller/cce3e3ff not reset as customized by admin to system_u:object_r:container_file_t:s0:c435,c756 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/efdd0498-1daa-4136-9a4a-3b948c2293fc/containers/multus-admission-controller/8fb75465 not reset as customized by admin to system_u:object_r:container_file_t:s0:c268,c620 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/efdd0498-1daa-4136-9a4a-3b948c2293fc/containers/kube-rbac-proxy/740f573e not reset as customized by admin to system_u:object_r:container_file_t:s0:c435,c756 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/efdd0498-1daa-4136-9a4a-3b948c2293fc/containers/kube-rbac-proxy/32fd1134 not reset as customized by admin to system_u:object_r:container_file_t:s0:c268,c620 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/20b0d48f-5fd6-431c-a545-e3c800c7b866/etc-hosts not reset as customized by admin to system_u:object_r:container_file_t:s0:c19,c24 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/20b0d48f-5fd6-431c-a545-e3c800c7b866/containers/serve-healthcheck-canary/0a861bd3 not reset as customized by admin to system_u:object_r:container_file_t:s0:c19,c24 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/20b0d48f-5fd6-431c-a545-e3c800c7b866/containers/serve-healthcheck-canary/80363026 not reset as customized by admin to system_u:object_r:container_file_t:s0:c19,c24 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/20b0d48f-5fd6-431c-a545-e3c800c7b866/containers/serve-healthcheck-canary/bfa952a8 not reset as customized by admin to system_u:object_r:container_file_t:s0:c19,c24 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/22c825df-677d-4ca6-82db-3454ed06e783/volumes/kubernetes.io~configmap/auth-proxy-config not reset as customized by admin to system_u:object_r:container_file_t:s0:c129,c158 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/22c825df-677d-4ca6-82db-3454ed06e783/volumes/kubernetes.io~configmap/auth-proxy-config/..2025_02_23_05_33_31.2122464563 not reset as customized by admin to system_u:object_r:container_file_t:s0:c129,c158 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/22c825df-677d-4ca6-82db-3454ed06e783/volumes/kubernetes.io~configmap/auth-proxy-config/..2025_02_23_05_33_31.2122464563/config-file.yaml not reset as customized by admin to system_u:object_r:container_file_t:s0:c129,c158 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/22c825df-677d-4ca6-82db-3454ed06e783/volumes/kubernetes.io~configmap/auth-proxy-config/..data not reset as customized by admin to system_u:object_r:container_file_t:s0:c129,c158 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/22c825df-677d-4ca6-82db-3454ed06e783/volumes/kubernetes.io~configmap/auth-proxy-config/config-file.yaml not reset as customized by admin to system_u:object_r:container_file_t:s0:c129,c158 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/22c825df-677d-4ca6-82db-3454ed06e783/volumes/kubernetes.io~configmap/config not reset as customized by admin to system_u:object_r:container_file_t:s0:c129,c158 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/22c825df-677d-4ca6-82db-3454ed06e783/volumes/kubernetes.io~configmap/config/..2025_02_23_05_33_31.333075221 not reset as customized by admin to system_u:object_r:container_file_t:s0:c129,c158 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/22c825df-677d-4ca6-82db-3454ed06e783/volumes/kubernetes.io~configmap/config/..data not reset as customized by admin to system_u:object_r:container_file_t:s0:c129,c158 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/22c825df-677d-4ca6-82db-3454ed06e783/etc-hosts not reset as customized by admin to system_u:object_r:container_file_t:s0:c129,c158 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/22c825df-677d-4ca6-82db-3454ed06e783/containers/kube-rbac-proxy/793bf43d not reset as customized by admin to system_u:object_r:container_file_t:s0:c381,c387 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/22c825df-677d-4ca6-82db-3454ed06e783/containers/kube-rbac-proxy/7db1bb6e not reset as customized by admin to system_u:object_r:container_file_t:s0:c142,c438 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/22c825df-677d-4ca6-82db-3454ed06e783/containers/kube-rbac-proxy/4f6a0368 not reset as customized by admin to system_u:object_r:container_file_t:s0:c129,c158 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/22c825df-677d-4ca6-82db-3454ed06e783/containers/machine-approver-controller/c12c7d86 not reset as customized by admin to system_u:object_r:container_file_t:s0:c381,c387 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/22c825df-677d-4ca6-82db-3454ed06e783/containers/machine-approver-controller/36c4a773 not reset as customized by admin to system_u:object_r:container_file_t:s0:c142,c438 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/22c825df-677d-4ca6-82db-3454ed06e783/containers/machine-approver-controller/4c1e98ae not reset as customized by admin to system_u:object_r:container_file_t:s0:c142,c438 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/22c825df-677d-4ca6-82db-3454ed06e783/containers/machine-approver-controller/a4c8115c not reset as customized by admin to system_u:object_r:container_file_t:s0:c129,c158 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/f4b27818a5e8e43d0dc095d08835c792/etc-hosts not reset as customized by admin to system_u:object_r:container_file_t:s0:c97,c980 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/f4b27818a5e8e43d0dc095d08835c792/containers/setup/7db1802e not reset as customized by admin to system_u:object_r:container_file_t:s0:c97,c980 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/f4b27818a5e8e43d0dc095d08835c792/containers/kube-apiserver/a008a7ab not reset as customized by admin to system_u:object_r:container_file_t:s0:c97,c980 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/f4b27818a5e8e43d0dc095d08835c792/containers/kube-apiserver-cert-syncer/2c836bac not reset as customized by admin to system_u:object_r:container_file_t:s0:c97,c980 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/f4b27818a5e8e43d0dc095d08835c792/containers/kube-apiserver-cert-regeneration-controller/0ce62299 not reset as customized by admin to system_u:object_r:container_file_t:s0:c97,c980 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/f4b27818a5e8e43d0dc095d08835c792/containers/kube-apiserver-insecure-readyz/945d2457 not reset as customized by admin to system_u:object_r:container_file_t:s0:c97,c980 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/f4b27818a5e8e43d0dc095d08835c792/containers/kube-apiserver-check-endpoints/7d5c1dd8 not reset as customized by admin to system_u:object_r:container_file_t:s0:c97,c980 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/utilities not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/utilities/copy-content not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/3scale-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/3scale-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/advanced-cluster-management not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/advanced-cluster-management/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/amq-broker-rhel8 not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/amq-broker-rhel8/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/amq-online not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/amq-online/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/amq-streams not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/amq-streams/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/amq-streams-console not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/amq-streams-console/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/amq7-interconnect-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/amq7-interconnect-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/ansible-automation-platform-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/ansible-automation-platform-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/ansible-cloud-addons-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/ansible-cloud-addons-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/apicast-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/apicast-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/apicurio-registry-3 not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/apicurio-registry-3/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/authorino-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/authorino-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/aws-load-balancer-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/aws-load-balancer-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/bamoe-businessautomation-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/bamoe-businessautomation-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/bamoe-kogito-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/bamoe-kogito-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/bpfman-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/bpfman-operator/index.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/businessautomation-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/businessautomation-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/cephcsi-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/cephcsi-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/cincinnati-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/cincinnati-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/cluster-kube-descheduler-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/cluster-kube-descheduler-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/cluster-logging not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/cluster-logging/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/cluster-observability-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/cluster-observability-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/compliance-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/compliance-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/container-security-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/container-security-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/costmanagement-metrics-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/costmanagement-metrics-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/cryostat-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/cryostat-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/datagrid not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/datagrid/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/devspaces not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/devspaces/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/devworkspace-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/devworkspace-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/dpu-network-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/dpu-network-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/eap not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/eap/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/elasticsearch-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/elasticsearch-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/external-dns-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/external-dns-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/fence-agents-remediation not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/fence-agents-remediation/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/file-integrity-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/file-integrity-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/fuse-apicurito not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/fuse-apicurito/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/fuse-console not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/fuse-console/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/fuse-online not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/fuse-online/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/gatekeeper-operator-product not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/gatekeeper-operator-product/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/jaeger-product not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/jaeger-product/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/jws-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/jws-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/kernel-module-management not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/kernel-module-management/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/kernel-module-management-hub not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/kernel-module-management-hub/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/kiali-ossm not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/kiali-ossm/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/kubevirt-hyperconverged not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/kubevirt-hyperconverged/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/logic-operator-rhel8 not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/logic-operator-rhel8/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/loki-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/loki-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/lvms-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/lvms-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/machine-deletion-remediation not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/machine-deletion-remediation/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/mcg-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/mcg-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/mta-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/mta-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/mtc-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/mtc-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/mtr-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/mtr-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/mtv-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/mtv-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/multicluster-engine not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/multicluster-engine/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/netobserv-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/netobserv-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/node-healthcheck-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/node-healthcheck-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/node-maintenance-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/node-maintenance-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/node-observability-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/node-observability-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/ocs-client-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/ocs-client-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/ocs-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/ocs-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/odf-csi-addons-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/odf-csi-addons-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/odf-multicluster-orchestrator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/odf-multicluster-orchestrator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/odf-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/odf-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/odf-prometheus-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/odf-prometheus-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/odr-cluster-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/odr-cluster-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/odr-hub-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/odr-hub-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/openshift-cert-manager-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/openshift-cert-manager-operator/bundle-v1.15.0.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/openshift-cert-manager-operator/channel.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/openshift-cert-manager-operator/package.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/openshift-custom-metrics-autoscaler-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/openshift-custom-metrics-autoscaler-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/openshift-gitops-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/openshift-gitops-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/openshift-pipelines-operator-rh not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/openshift-pipelines-operator-rh/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/openshift-secondary-scheduler-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/openshift-secondary-scheduler-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/opentelemetry-product not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/opentelemetry-product/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/quay-bridge-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/quay-bridge-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/quay-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/quay-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/recipe not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/recipe/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/red-hat-camel-k not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/red-hat-camel-k/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/red-hat-hawtio-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/red-hat-hawtio-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/redhat-oadp-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/redhat-oadp-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/rh-service-binding-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/rh-service-binding-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/rhacs-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/rhacs-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/rhbk-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/rhbk-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/rhdh not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/rhdh/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/rhods-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/rhods-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/rhods-prometheus-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/rhods-prometheus-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/rhpam-kogito-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/rhpam-kogito-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/rhsso-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/rhsso-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/rook-ceph-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/rook-ceph-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/run-once-duration-override-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/run-once-duration-override-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/sandboxed-containers-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/sandboxed-containers-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/security-profiles-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/security-profiles-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/self-node-remediation not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/self-node-remediation/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/serverless-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/serverless-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/service-registry-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/service-registry-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/servicemeshoperator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/servicemeshoperator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/servicemeshoperator3 not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/servicemeshoperator3/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/skupper-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/skupper-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/submariner not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/submariner/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/tang-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/tang-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/tempo-product not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/tempo-product/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/trustee-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/trustee-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/volsync-product not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/volsync-product/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/web-terminal not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/catalog/web-terminal/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/cache not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/cache/pogreb.v1 not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/cache/pogreb.v1/db not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/cache/pogreb.v1/db/00000-1.psg not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/cache/pogreb.v1/db/00000-1.psg.pmt not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/cache/pogreb.v1/db/db.pmt not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/cache/pogreb.v1/db/index.pmt not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/cache/pogreb.v1/db/main.pix not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/cache/pogreb.v1/db/overflow.pix not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes/kubernetes.io~empty-dir/catalog-content/cache/pogreb.v1/digest not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/etc-hosts not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/containers/extract-utilities/bc8d0691 not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/containers/extract-utilities/6b76097a not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/containers/extract-utilities/34d1af30 not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/containers/extract-content/312ba61c not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/containers/extract-content/645d5dd1 not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/containers/extract-content/16e825f0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/containers/registry-server/4cf51fc9 not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/containers/registry-server/2a23d348 not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/containers/registry-server/075dbd49 not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/3cb93b32-e0ae-4377-b9c8-fdb9842c6d59/volumes/kubernetes.io~configmap/serviceca not reset as customized by admin to system_u:object_r:container_file_t:s0:c842,c986 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/3cb93b32-e0ae-4377-b9c8-fdb9842c6d59/volumes/kubernetes.io~configmap/serviceca/..data not reset as customized by admin to system_u:object_r:container_file_t:s0:c842,c986 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/3cb93b32-e0ae-4377-b9c8-fdb9842c6d59/volumes/kubernetes.io~configmap/serviceca/..2025_02_24_06_09_13.3521195566 not reset as customized by admin to system_u:object_r:container_file_t:s0:c842,c986 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/3cb93b32-e0ae-4377-b9c8-fdb9842c6d59/volumes/kubernetes.io~configmap/serviceca/..2025_02_24_06_09_13.3521195566/image-registry.openshift-image-registry.svc..5000 not reset as customized by admin to system_u:object_r:container_file_t:s0:c842,c986 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/3cb93b32-e0ae-4377-b9c8-fdb9842c6d59/volumes/kubernetes.io~configmap/serviceca/..2025_02_24_06_09_13.3521195566/image-registry.openshift-image-registry.svc.cluster.local..5000 not reset as customized by admin to system_u:object_r:container_file_t:s0:c842,c986 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/3cb93b32-e0ae-4377-b9c8-fdb9842c6d59/volumes/kubernetes.io~configmap/serviceca/..2025_02_24_06_09_13.3521195566/default-route-openshift-image-registry.apps-crc.testing not reset as customized by admin to system_u:object_r:container_file_t:s0:c842,c986 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/3cb93b32-e0ae-4377-b9c8-fdb9842c6d59/volumes/kubernetes.io~configmap/serviceca/default-route-openshift-image-registry.apps-crc.testing not reset as customized by admin to system_u:object_r:container_file_t:s0:c842,c986 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/3cb93b32-e0ae-4377-b9c8-fdb9842c6d59/volumes/kubernetes.io~configmap/serviceca/image-registry.openshift-image-registry.svc..5000 not reset as customized by admin to system_u:object_r:container_file_t:s0:c842,c986 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/3cb93b32-e0ae-4377-b9c8-fdb9842c6d59/volumes/kubernetes.io~configmap/serviceca/image-registry.openshift-image-registry.svc.cluster.local..5000 not reset as customized by admin to system_u:object_r:container_file_t:s0:c842,c986 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/3cb93b32-e0ae-4377-b9c8-fdb9842c6d59/etc-hosts not reset as customized by admin to system_u:object_r:container_file_t:s0:c842,c986 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/3cb93b32-e0ae-4377-b9c8-fdb9842c6d59/containers/node-ca/dd585ddd not reset as customized by admin to system_u:object_r:container_file_t:s0:c377,c642 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/3cb93b32-e0ae-4377-b9c8-fdb9842c6d59/containers/node-ca/17ebd0ab not reset as customized by admin to system_u:object_r:container_file_t:s0:c338,c343 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/3cb93b32-e0ae-4377-b9c8-fdb9842c6d59/containers/node-ca/005579f4 not reset as customized by admin to system_u:object_r:container_file_t:s0:c842,c986 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/09ae3b1a-e8e7-4524-b54b-61eab6f9239a/volumes/kubernetes.io~configmap/etcd-serving-ca not reset as customized by admin to system_u:object_r:container_file_t:s0:c764,c897 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/09ae3b1a-e8e7-4524-b54b-61eab6f9239a/volumes/kubernetes.io~configmap/etcd-serving-ca/..2025_02_23_05_23_11.449897510 not reset as customized by admin to system_u:object_r:container_file_t:s0:c764,c897 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/09ae3b1a-e8e7-4524-b54b-61eab6f9239a/volumes/kubernetes.io~configmap/etcd-serving-ca/..2025_02_23_05_23_11.449897510/ca-bundle.crt not reset as customized by admin to system_u:object_r:container_file_t:s0:c764,c897 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/09ae3b1a-e8e7-4524-b54b-61eab6f9239a/volumes/kubernetes.io~configmap/etcd-serving-ca/..data not reset as customized by admin to system_u:object_r:container_file_t:s0:c764,c897 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/09ae3b1a-e8e7-4524-b54b-61eab6f9239a/volumes/kubernetes.io~configmap/etcd-serving-ca/ca-bundle.crt not reset as customized by admin to system_u:object_r:container_file_t:s0:c764,c897 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/09ae3b1a-e8e7-4524-b54b-61eab6f9239a/volumes/kubernetes.io~configmap/trusted-ca-bundle not reset as customized by admin to system_u:object_r:container_file_t:s0:c764,c897 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/09ae3b1a-e8e7-4524-b54b-61eab6f9239a/volumes/kubernetes.io~configmap/trusted-ca-bundle/..2025_02_23_05_23_11.1287037894 not reset as customized by admin to system_u:object_r:container_file_t:s0:c764,c897 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/09ae3b1a-e8e7-4524-b54b-61eab6f9239a/volumes/kubernetes.io~configmap/trusted-ca-bundle/..data not reset as customized by admin to system_u:object_r:container_file_t:s0:c764,c897 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/09ae3b1a-e8e7-4524-b54b-61eab6f9239a/volumes/kubernetes.io~configmap/audit-policies not reset as customized by admin to system_u:object_r:container_file_t:s0:c764,c897 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/09ae3b1a-e8e7-4524-b54b-61eab6f9239a/volumes/kubernetes.io~configmap/audit-policies/..2025_02_23_05_23_11.1301053334 not reset as customized by admin to system_u:object_r:container_file_t:s0:c764,c897 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/09ae3b1a-e8e7-4524-b54b-61eab6f9239a/volumes/kubernetes.io~configmap/audit-policies/..2025_02_23_05_23_11.1301053334/policy.yaml not reset as customized by admin to system_u:object_r:container_file_t:s0:c764,c897 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/09ae3b1a-e8e7-4524-b54b-61eab6f9239a/volumes/kubernetes.io~configmap/audit-policies/..data not reset as customized by admin to system_u:object_r:container_file_t:s0:c764,c897 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/09ae3b1a-e8e7-4524-b54b-61eab6f9239a/volumes/kubernetes.io~configmap/audit-policies/policy.yaml not reset as customized by admin to system_u:object_r:container_file_t:s0:c764,c897 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/09ae3b1a-e8e7-4524-b54b-61eab6f9239a/etc-hosts not reset as customized by admin to system_u:object_r:container_file_t:s0:c764,c897 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/09ae3b1a-e8e7-4524-b54b-61eab6f9239a/containers/fix-audit-permissions/bf5f3b9c not reset as customized by admin to system_u:object_r:container_file_t:s0:c49,c263 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/09ae3b1a-e8e7-4524-b54b-61eab6f9239a/containers/fix-audit-permissions/af276eb7 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c701 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/09ae3b1a-e8e7-4524-b54b-61eab6f9239a/containers/fix-audit-permissions/ea28e322 not reset as customized by admin to system_u:object_r:container_file_t:s0:c764,c897 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/09ae3b1a-e8e7-4524-b54b-61eab6f9239a/containers/oauth-apiserver/692e6683 not reset as customized by admin to system_u:object_r:container_file_t:s0:c49,c263 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/09ae3b1a-e8e7-4524-b54b-61eab6f9239a/containers/oauth-apiserver/871746a7 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c701 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/09ae3b1a-e8e7-4524-b54b-61eab6f9239a/containers/oauth-apiserver/4eb2e958 not reset as customized by admin to system_u:object_r:container_file_t:s0:c764,c897 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/43509403-f426-496e-be36-56cef71462f5/volumes/kubernetes.io~configmap/console-config not reset as customized by admin to system_u:object_r:container_file_t:s0:c0,c25 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/43509403-f426-496e-be36-56cef71462f5/volumes/kubernetes.io~configmap/console-config/..2025_02_24_06_09_06.2875086261 not reset as customized by admin to system_u:object_r:container_file_t:s0:c0,c25 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/43509403-f426-496e-be36-56cef71462f5/volumes/kubernetes.io~configmap/console-config/..2025_02_24_06_09_06.2875086261/console-config.yaml not reset as customized by admin to system_u:object_r:container_file_t:s0:c0,c25 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/43509403-f426-496e-be36-56cef71462f5/volumes/kubernetes.io~configmap/console-config/..data not reset as customized by admin to system_u:object_r:container_file_t:s0:c0,c25 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/43509403-f426-496e-be36-56cef71462f5/volumes/kubernetes.io~configmap/console-config/console-config.yaml not reset as customized by admin to system_u:object_r:container_file_t:s0:c0,c25 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/43509403-f426-496e-be36-56cef71462f5/volumes/kubernetes.io~configmap/trusted-ca-bundle not reset as customized by admin to system_u:object_r:container_file_t:s0:c0,c25 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/43509403-f426-496e-be36-56cef71462f5/volumes/kubernetes.io~configmap/trusted-ca-bundle/..2025_02_24_06_09_06.286118152 not reset as customized by admin to system_u:object_r:container_file_t:s0:c0,c25 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/43509403-f426-496e-be36-56cef71462f5/volumes/kubernetes.io~configmap/trusted-ca-bundle/..2025_02_24_06_09_06.286118152/tls-ca-bundle.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c0,c25 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/43509403-f426-496e-be36-56cef71462f5/volumes/kubernetes.io~configmap/trusted-ca-bundle/..data not reset as customized by admin to system_u:object_r:container_file_t:s0:c0,c25 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/43509403-f426-496e-be36-56cef71462f5/volumes/kubernetes.io~configmap/trusted-ca-bundle/tls-ca-bundle.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c0,c25 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/43509403-f426-496e-be36-56cef71462f5/volumes/kubernetes.io~configmap/oauth-serving-cert not reset as customized by admin to system_u:object_r:container_file_t:s0:c0,c25 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/43509403-f426-496e-be36-56cef71462f5/volumes/kubernetes.io~configmap/oauth-serving-cert/..2025_02_24_06_09_06.3865795478 not reset as customized by admin to system_u:object_r:container_file_t:s0:c0,c25 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/43509403-f426-496e-be36-56cef71462f5/volumes/kubernetes.io~configmap/oauth-serving-cert/..2025_02_24_06_09_06.3865795478/ca-bundle.crt not reset as customized by admin to system_u:object_r:container_file_t:s0:c0,c25 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/43509403-f426-496e-be36-56cef71462f5/volumes/kubernetes.io~configmap/oauth-serving-cert/..data not reset as customized by admin to system_u:object_r:container_file_t:s0:c0,c25 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/43509403-f426-496e-be36-56cef71462f5/volumes/kubernetes.io~configmap/oauth-serving-cert/ca-bundle.crt not reset as customized by admin to system_u:object_r:container_file_t:s0:c0,c25 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/43509403-f426-496e-be36-56cef71462f5/volumes/kubernetes.io~configmap/service-ca not reset as customized by admin to system_u:object_r:container_file_t:s0:c0,c25 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/43509403-f426-496e-be36-56cef71462f5/volumes/kubernetes.io~configmap/service-ca/..2025_02_24_06_09_06.584414814 not reset as customized by admin to system_u:object_r:container_file_t:s0:c0,c25 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/43509403-f426-496e-be36-56cef71462f5/volumes/kubernetes.io~configmap/service-ca/..2025_02_24_06_09_06.584414814/service-ca.crt not reset as customized by admin to system_u:object_r:container_file_t:s0:c0,c25 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/43509403-f426-496e-be36-56cef71462f5/volumes/kubernetes.io~configmap/service-ca/..data not reset as customized by admin to system_u:object_r:container_file_t:s0:c0,c25 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/43509403-f426-496e-be36-56cef71462f5/volumes/kubernetes.io~configmap/service-ca/service-ca.crt not reset as customized by admin to system_u:object_r:container_file_t:s0:c0,c25 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/43509403-f426-496e-be36-56cef71462f5/etc-hosts not reset as customized by admin to system_u:object_r:container_file_t:s0:c0,c25 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/43509403-f426-496e-be36-56cef71462f5/containers/console/ca9b62da not reset as customized by admin to system_u:object_r:container_file_t:s0:c0,c25 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/43509403-f426-496e-be36-56cef71462f5/containers/console/0edd6fce not reset as customized by admin to system_u:object_r:container_file_t:s0:c0,c25 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/7583ce53-e0fe-4a16-9e4d-50516596a136/volumes/kubernetes.io~configmap/config not reset as customized by admin to system_u:object_r:container_file_t:s0:c14,c22 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/7583ce53-e0fe-4a16-9e4d-50516596a136/volumes/kubernetes.io~configmap/config/..2025_02_24_06_20_07.2406383837 not reset as customized by admin to system_u:object_r:container_file_t:s0:c14,c22 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/7583ce53-e0fe-4a16-9e4d-50516596a136/volumes/kubernetes.io~configmap/config/..2025_02_24_06_20_07.2406383837/config.yaml not reset as customized by admin to system_u:object_r:container_file_t:s0:c14,c22 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/7583ce53-e0fe-4a16-9e4d-50516596a136/volumes/kubernetes.io~configmap/config/..2025_02_24_06_20_07.2406383837/openshift-controller-manager.client-ca.configmap not reset as customized by admin to system_u:object_r:container_file_t:s0:c14,c22 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/7583ce53-e0fe-4a16-9e4d-50516596a136/volumes/kubernetes.io~configmap/config/..2025_02_24_06_20_07.2406383837/openshift-controller-manager.openshift-global-ca.configmap not reset as customized by admin to system_u:object_r:container_file_t:s0:c14,c22 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/7583ce53-e0fe-4a16-9e4d-50516596a136/volumes/kubernetes.io~configmap/config/..2025_02_24_06_20_07.2406383837/openshift-controller-manager.serving-cert.secret not reset as customized by admin to system_u:object_r:container_file_t:s0:c14,c22 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/7583ce53-e0fe-4a16-9e4d-50516596a136/volumes/kubernetes.io~configmap/config/..data not reset as customized by admin to system_u:object_r:container_file_t:s0:c14,c22 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/7583ce53-e0fe-4a16-9e4d-50516596a136/volumes/kubernetes.io~configmap/config/config.yaml not reset as customized by admin to system_u:object_r:container_file_t:s0:c14,c22 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/7583ce53-e0fe-4a16-9e4d-50516596a136/volumes/kubernetes.io~configmap/config/openshift-controller-manager.client-ca.configmap not reset as customized by admin to system_u:object_r:container_file_t:s0:c14,c22 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/7583ce53-e0fe-4a16-9e4d-50516596a136/volumes/kubernetes.io~configmap/config/openshift-controller-manager.openshift-global-ca.configmap not reset as customized by admin to system_u:object_r:container_file_t:s0:c14,c22 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/7583ce53-e0fe-4a16-9e4d-50516596a136/volumes/kubernetes.io~configmap/config/openshift-controller-manager.serving-cert.secret not reset as customized by admin to system_u:object_r:container_file_t:s0:c14,c22 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/7583ce53-e0fe-4a16-9e4d-50516596a136/volumes/kubernetes.io~configmap/client-ca not reset as customized by admin to system_u:object_r:container_file_t:s0:c14,c22 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/7583ce53-e0fe-4a16-9e4d-50516596a136/volumes/kubernetes.io~configmap/client-ca/..2025_02_24_06_20_07.1071801880 not reset as customized by admin to system_u:object_r:container_file_t:s0:c14,c22 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/7583ce53-e0fe-4a16-9e4d-50516596a136/volumes/kubernetes.io~configmap/client-ca/..2025_02_24_06_20_07.1071801880/ca-bundle.crt not reset as customized by admin to system_u:object_r:container_file_t:s0:c14,c22 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/7583ce53-e0fe-4a16-9e4d-50516596a136/volumes/kubernetes.io~configmap/client-ca/..data not reset as customized by admin to system_u:object_r:container_file_t:s0:c14,c22 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/7583ce53-e0fe-4a16-9e4d-50516596a136/volumes/kubernetes.io~configmap/client-ca/ca-bundle.crt not reset as customized by admin to system_u:object_r:container_file_t:s0:c14,c22 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/7583ce53-e0fe-4a16-9e4d-50516596a136/volumes/kubernetes.io~configmap/proxy-ca-bundles not reset as customized by admin to system_u:object_r:container_file_t:s0:c14,c22 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/7583ce53-e0fe-4a16-9e4d-50516596a136/volumes/kubernetes.io~configmap/proxy-ca-bundles/..2025_02_24_06_20_07.2494444877 not reset as customized by admin to system_u:object_r:container_file_t:s0:c14,c22 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/7583ce53-e0fe-4a16-9e4d-50516596a136/volumes/kubernetes.io~configmap/proxy-ca-bundles/..2025_02_24_06_20_07.2494444877/tls-ca-bundle.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c14,c22 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/7583ce53-e0fe-4a16-9e4d-50516596a136/volumes/kubernetes.io~configmap/proxy-ca-bundles/..data not reset as customized by admin to system_u:object_r:container_file_t:s0:c14,c22 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/7583ce53-e0fe-4a16-9e4d-50516596a136/volumes/kubernetes.io~configmap/proxy-ca-bundles/tls-ca-bundle.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c14,c22 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/7583ce53-e0fe-4a16-9e4d-50516596a136/etc-hosts not reset as customized by admin to system_u:object_r:container_file_t:s0:c14,c22 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/7583ce53-e0fe-4a16-9e4d-50516596a136/containers/controller-manager/89b4555f not reset as customized by admin to system_u:object_r:container_file_t:s0:c14,c22 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/87cf06ed-a83f-41a7-828d-70653580a8cb/volumes/kubernetes.io~configmap/config-volume not reset as customized by admin to system_u:object_r:container_file_t:s0:c466,c972 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/87cf06ed-a83f-41a7-828d-70653580a8cb/volumes/kubernetes.io~configmap/config-volume/..2025_02_23_05_23_22.4071100442 not reset as customized by admin to system_u:object_r:container_file_t:s0:c466,c972 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/87cf06ed-a83f-41a7-828d-70653580a8cb/volumes/kubernetes.io~configmap/config-volume/..2025_02_23_05_23_22.4071100442/Corefile not reset as customized by admin to system_u:object_r:container_file_t:s0:c466,c972 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/87cf06ed-a83f-41a7-828d-70653580a8cb/volumes/kubernetes.io~configmap/config-volume/..data not reset as customized by admin to system_u:object_r:container_file_t:s0:c466,c972 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/87cf06ed-a83f-41a7-828d-70653580a8cb/volumes/kubernetes.io~configmap/config-volume/Corefile not reset as customized by admin to system_u:object_r:container_file_t:s0:c466,c972 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/87cf06ed-a83f-41a7-828d-70653580a8cb/etc-hosts not reset as customized by admin to system_u:object_r:container_file_t:s0:c466,c972 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/87cf06ed-a83f-41a7-828d-70653580a8cb/containers/dns/655fcd71 not reset as customized by admin to system_u:object_r:container_file_t:s0:c457,c841 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/87cf06ed-a83f-41a7-828d-70653580a8cb/containers/dns/0d43c002 not reset as customized by admin to system_u:object_r:container_file_t:s0:c55,c1022 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/87cf06ed-a83f-41a7-828d-70653580a8cb/containers/dns/e68efd17 not reset as customized by admin to system_u:object_r:container_file_t:s0:c466,c972 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/87cf06ed-a83f-41a7-828d-70653580a8cb/containers/kube-rbac-proxy/9acf9b65 not reset as customized by admin to system_u:object_r:container_file_t:s0:c457,c841 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/87cf06ed-a83f-41a7-828d-70653580a8cb/containers/kube-rbac-proxy/5ae3ff11 not reset as customized by admin to system_u:object_r:container_file_t:s0:c55,c1022 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/87cf06ed-a83f-41a7-828d-70653580a8cb/containers/kube-rbac-proxy/1e59206a not reset as customized by admin to system_u:object_r:container_file_t:s0:c466,c972 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/44663579-783b-4372-86d6-acf235a62d72/containers/dns-node-resolver/27af16d1 not reset as customized by admin to system_u:object_r:container_file_t:s0:c304,c1017 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/44663579-783b-4372-86d6-acf235a62d72/containers/dns-node-resolver/7918e729 not reset as customized by admin to system_u:object_r:container_file_t:s0:c853,c893 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/44663579-783b-4372-86d6-acf235a62d72/containers/dns-node-resolver/5d976d0e not reset as customized by admin to system_u:object_r:container_file_t:s0:c585,c981 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/9d4552c7-cd75-42dd-8880-30dd377c49a4/volumes/kubernetes.io~configmap/config not reset as customized by admin to system_u:object_r:container_file_t:s0:c5,c25 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/9d4552c7-cd75-42dd-8880-30dd377c49a4/volumes/kubernetes.io~configmap/config/..2025_02_23_05_38_56.1112187283 not reset as customized by admin to system_u:object_r:container_file_t:s0:c5,c25 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/9d4552c7-cd75-42dd-8880-30dd377c49a4/volumes/kubernetes.io~configmap/config/..2025_02_23_05_38_56.1112187283/controller-config.yaml not reset as customized by admin to system_u:object_r:container_file_t:s0:c5,c25 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/9d4552c7-cd75-42dd-8880-30dd377c49a4/volumes/kubernetes.io~configmap/config/..data not reset as customized by admin to system_u:object_r:container_file_t:s0:c5,c25 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/9d4552c7-cd75-42dd-8880-30dd377c49a4/volumes/kubernetes.io~configmap/config/controller-config.yaml not reset as customized by admin to system_u:object_r:container_file_t:s0:c5,c25 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/9d4552c7-cd75-42dd-8880-30dd377c49a4/volumes/kubernetes.io~configmap/trusted-ca not reset as customized by admin to system_u:object_r:container_file_t:s0:c5,c25 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/9d4552c7-cd75-42dd-8880-30dd377c49a4/volumes/kubernetes.io~configmap/trusted-ca/..2025_02_23_05_38_56.2839772658 not reset as customized by admin to system_u:object_r:container_file_t:s0:c5,c25 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/9d4552c7-cd75-42dd-8880-30dd377c49a4/volumes/kubernetes.io~configmap/trusted-ca/..2025_02_23_05_38_56.2839772658/tls-ca-bundle.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c5,c25 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/9d4552c7-cd75-42dd-8880-30dd377c49a4/volumes/kubernetes.io~configmap/trusted-ca/..data not reset as customized by admin to system_u:object_r:container_file_t:s0:c5,c25 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/9d4552c7-cd75-42dd-8880-30dd377c49a4/volumes/kubernetes.io~configmap/trusted-ca/tls-ca-bundle.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c5,c25 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/9d4552c7-cd75-42dd-8880-30dd377c49a4/etc-hosts not reset as customized by admin to system_u:object_r:container_file_t:s0:c5,c25 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/9d4552c7-cd75-42dd-8880-30dd377c49a4/containers/console-operator/d7f55cbb not reset as customized by admin to system_u:object_r:container_file_t:s0:c5,c25 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/9d4552c7-cd75-42dd-8880-30dd377c49a4/containers/console-operator/f0812073 not reset as customized by admin to system_u:object_r:container_file_t:s0:c5,c25 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/9d4552c7-cd75-42dd-8880-30dd377c49a4/containers/console-operator/1a56cbeb not reset as customized by admin to system_u:object_r:container_file_t:s0:c5,c25 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/9d4552c7-cd75-42dd-8880-30dd377c49a4/containers/console-operator/7fdd437e not reset as customized by admin to system_u:object_r:container_file_t:s0:c5,c25 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/9d4552c7-cd75-42dd-8880-30dd377c49a4/containers/console-operator/cdfb5652 not reset as customized by admin to system_u:object_r:container_file_t:s0:c5,c25 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/1bf7eb37-55a3-4c65-b768-a94c82151e69/volumes/kubernetes.io~configmap/etcd-serving-ca not reset as customized by admin to system_u:object_r:container_file_t:s0:c336,c787 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/1bf7eb37-55a3-4c65-b768-a94c82151e69/volumes/kubernetes.io~configmap/etcd-serving-ca/..2025_02_24_06_17_29.3844392896 not reset as customized by admin to system_u:object_r:container_file_t:s0:c336,c787 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/1bf7eb37-55a3-4c65-b768-a94c82151e69/volumes/kubernetes.io~configmap/etcd-serving-ca/..2025_02_24_06_17_29.3844392896/ca-bundle.crt not reset as customized by admin to system_u:object_r:container_file_t:s0:c336,c787 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/1bf7eb37-55a3-4c65-b768-a94c82151e69/volumes/kubernetes.io~configmap/etcd-serving-ca/..data not reset as customized by admin to system_u:object_r:container_file_t:s0:c336,c787 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/1bf7eb37-55a3-4c65-b768-a94c82151e69/volumes/kubernetes.io~configmap/etcd-serving-ca/ca-bundle.crt not reset as customized by admin to system_u:object_r:container_file_t:s0:c336,c787 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/1bf7eb37-55a3-4c65-b768-a94c82151e69/volumes/kubernetes.io~configmap/config not reset as customized by admin to system_u:object_r:container_file_t:s0:c336,c787 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/1bf7eb37-55a3-4c65-b768-a94c82151e69/volumes/kubernetes.io~configmap/config/..2025_02_24_06_17_29.848549803 not reset as customized by admin to system_u:object_r:container_file_t:s0:c336,c787 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/1bf7eb37-55a3-4c65-b768-a94c82151e69/volumes/kubernetes.io~configmap/config/..2025_02_24_06_17_29.848549803/config.yaml not reset as customized by admin to system_u:object_r:container_file_t:s0:c336,c787 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/1bf7eb37-55a3-4c65-b768-a94c82151e69/volumes/kubernetes.io~configmap/config/..data not reset as customized by admin to system_u:object_r:container_file_t:s0:c336,c787 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/1bf7eb37-55a3-4c65-b768-a94c82151e69/volumes/kubernetes.io~configmap/config/config.yaml not reset as customized by admin to system_u:object_r:container_file_t:s0:c336,c787 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/1bf7eb37-55a3-4c65-b768-a94c82151e69/volumes/kubernetes.io~configmap/audit not reset as customized by admin to system_u:object_r:container_file_t:s0:c336,c787 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/1bf7eb37-55a3-4c65-b768-a94c82151e69/volumes/kubernetes.io~configmap/audit/..2025_02_24_06_17_29.780046231 not reset as customized by admin to system_u:object_r:container_file_t:s0:c336,c787 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/1bf7eb37-55a3-4c65-b768-a94c82151e69/volumes/kubernetes.io~configmap/audit/..2025_02_24_06_17_29.780046231/policy.yaml not reset as customized by admin to system_u:object_r:container_file_t:s0:c336,c787 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/1bf7eb37-55a3-4c65-b768-a94c82151e69/volumes/kubernetes.io~configmap/audit/..data not reset as customized by admin to system_u:object_r:container_file_t:s0:c336,c787 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/1bf7eb37-55a3-4c65-b768-a94c82151e69/volumes/kubernetes.io~configmap/audit/policy.yaml not reset as customized by admin to system_u:object_r:container_file_t:s0:c336,c787 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/1bf7eb37-55a3-4c65-b768-a94c82151e69/volumes/kubernetes.io~configmap/image-import-ca not reset as customized by admin to system_u:object_r:container_file_t:s0:c336,c787 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/1bf7eb37-55a3-4c65-b768-a94c82151e69/volumes/kubernetes.io~configmap/image-import-ca/..2025_02_24_06_17_29.2926008347 not reset as customized by admin to system_u:object_r:container_file_t:s0:c336,c787 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/1bf7eb37-55a3-4c65-b768-a94c82151e69/volumes/kubernetes.io~configmap/image-import-ca/..2025_02_24_06_17_29.2926008347/image-registry.openshift-image-registry.svc..5000 not reset as customized by admin to system_u:object_r:container_file_t:s0:c336,c787 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/1bf7eb37-55a3-4c65-b768-a94c82151e69/volumes/kubernetes.io~configmap/image-import-ca/..2025_02_24_06_17_29.2926008347/image-registry.openshift-image-registry.svc.cluster.local..5000 not reset as customized by admin to system_u:object_r:container_file_t:s0:c336,c787 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/1bf7eb37-55a3-4c65-b768-a94c82151e69/volumes/kubernetes.io~configmap/image-import-ca/..2025_02_24_06_17_29.2926008347/default-route-openshift-image-registry.apps-crc.testing not reset as customized by admin to system_u:object_r:container_file_t:s0:c336,c787 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/1bf7eb37-55a3-4c65-b768-a94c82151e69/volumes/kubernetes.io~configmap/image-import-ca/..data not reset as customized by admin to system_u:object_r:container_file_t:s0:c336,c787 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/1bf7eb37-55a3-4c65-b768-a94c82151e69/volumes/kubernetes.io~configmap/image-import-ca/image-registry.openshift-image-registry.svc..5000 not reset as customized by admin to system_u:object_r:container_file_t:s0:c336,c787 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/1bf7eb37-55a3-4c65-b768-a94c82151e69/volumes/kubernetes.io~configmap/image-import-ca/image-registry.openshift-image-registry.svc.cluster.local..5000 not reset as customized by admin to system_u:object_r:container_file_t:s0:c336,c787 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/1bf7eb37-55a3-4c65-b768-a94c82151e69/volumes/kubernetes.io~configmap/image-import-ca/default-route-openshift-image-registry.apps-crc.testing not reset as customized by admin to system_u:object_r:container_file_t:s0:c336,c787 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/1bf7eb37-55a3-4c65-b768-a94c82151e69/volumes/kubernetes.io~configmap/trusted-ca-bundle not reset as customized by admin to system_u:object_r:container_file_t:s0:c336,c787 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/1bf7eb37-55a3-4c65-b768-a94c82151e69/volumes/kubernetes.io~configmap/trusted-ca-bundle/..2025_02_24_06_17_29.2729721485 not reset as customized by admin to system_u:object_r:container_file_t:s0:c336,c787 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/1bf7eb37-55a3-4c65-b768-a94c82151e69/volumes/kubernetes.io~configmap/trusted-ca-bundle/..2025_02_24_06_17_29.2729721485/tls-ca-bundle.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c336,c787 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/1bf7eb37-55a3-4c65-b768-a94c82151e69/volumes/kubernetes.io~configmap/trusted-ca-bundle/..data not reset as customized by admin to system_u:object_r:container_file_t:s0:c336,c787 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/1bf7eb37-55a3-4c65-b768-a94c82151e69/volumes/kubernetes.io~configmap/trusted-ca-bundle/tls-ca-bundle.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c336,c787 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/1bf7eb37-55a3-4c65-b768-a94c82151e69/etc-hosts not reset as customized by admin to system_u:object_r:container_file_t:s0:c336,c787 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/1bf7eb37-55a3-4c65-b768-a94c82151e69/containers/fix-audit-permissions/fb93119e not reset as customized by admin to system_u:object_r:container_file_t:s0:c336,c787 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/1bf7eb37-55a3-4c65-b768-a94c82151e69/containers/openshift-apiserver/f1e8fc0e not reset as customized by admin to system_u:object_r:container_file_t:s0:c336,c787 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/1bf7eb37-55a3-4c65-b768-a94c82151e69/containers/openshift-apiserver-check-endpoints/218511f3 not reset as customized by admin to system_u:object_r:container_file_t:s0:c336,c787 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/308be0ea-9f5f-4b29-aeb1-5abd31a0b17b/volumes/kubernetes.io~empty-dir/tmpfs not reset as customized by admin to system_u:object_r:container_file_t:s0:c12,c18 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/308be0ea-9f5f-4b29-aeb1-5abd31a0b17b/volumes/kubernetes.io~empty-dir/tmpfs/k8s-webhook-server not reset as customized by admin to system_u:object_r:container_file_t:s0:c12,c18 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/308be0ea-9f5f-4b29-aeb1-5abd31a0b17b/volumes/kubernetes.io~empty-dir/tmpfs/k8s-webhook-server/serving-certs not reset as customized by admin to system_u:object_r:container_file_t:s0:c12,c18 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/308be0ea-9f5f-4b29-aeb1-5abd31a0b17b/etc-hosts not reset as customized by admin to system_u:object_r:container_file_t:s0:c12,c18 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/308be0ea-9f5f-4b29-aeb1-5abd31a0b17b/containers/packageserver/ca8af7b3 not reset as customized by admin to system_u:object_r:container_file_t:s0:c12,c18 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/308be0ea-9f5f-4b29-aeb1-5abd31a0b17b/containers/packageserver/72cc8a75 not reset as customized by admin to system_u:object_r:container_file_t:s0:c12,c18 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/308be0ea-9f5f-4b29-aeb1-5abd31a0b17b/containers/packageserver/6e8a3760 not reset as customized by admin to system_u:object_r:container_file_t:s0:c12,c18 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/0b78653f-4ff9-4508-8672-245ed9b561e3/volumes/kubernetes.io~configmap/service-ca not reset as customized by admin to system_u:object_r:container_file_t:s0:c5,c6 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/0b78653f-4ff9-4508-8672-245ed9b561e3/volumes/kubernetes.io~configmap/service-ca/..2025_02_23_05_27_30.557428972 not reset as customized by admin to system_u:object_r:container_file_t:s0:c5,c6 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/0b78653f-4ff9-4508-8672-245ed9b561e3/volumes/kubernetes.io~configmap/service-ca/..2025_02_23_05_27_30.557428972/service-ca.crt not reset as customized by admin to system_u:object_r:container_file_t:s0:c5,c6 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/0b78653f-4ff9-4508-8672-245ed9b561e3/volumes/kubernetes.io~configmap/service-ca/..data not reset as customized by admin to system_u:object_r:container_file_t:s0:c5,c6 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/0b78653f-4ff9-4508-8672-245ed9b561e3/volumes/kubernetes.io~configmap/service-ca/service-ca.crt not reset as customized by admin to system_u:object_r:container_file_t:s0:c5,c6 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/0b78653f-4ff9-4508-8672-245ed9b561e3/etc-hosts not reset as customized by admin to system_u:object_r:container_file_t:s0:c5,c6 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/0b78653f-4ff9-4508-8672-245ed9b561e3/containers/cluster-version-operator/4c3455c0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c5,c6 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/0b78653f-4ff9-4508-8672-245ed9b561e3/containers/cluster-version-operator/2278acb0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c5,c6 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/0b78653f-4ff9-4508-8672-245ed9b561e3/containers/cluster-version-operator/4b453e4f not reset as customized by admin to system_u:object_r:container_file_t:s0:c5,c6 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/0b78653f-4ff9-4508-8672-245ed9b561e3/containers/cluster-version-operator/3ec09bda not reset as customized by admin to system_u:object_r:container_file_t:s0:c5,c6 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~configmap/trusted-ca not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~configmap/trusted-ca/..2025_02_24_06_25_03.422633132 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~configmap/trusted-ca/..2025_02_24_06_25_03.422633132/anchors not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~configmap/trusted-ca/..2025_02_24_06_25_03.422633132/anchors/ca-bundle.crt not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~configmap/trusted-ca/..data not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~configmap/trusted-ca/anchors not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~configmap/registry-certificates not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~configmap/registry-certificates/..2025_02_24_06_25_03.3594477318 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~configmap/registry-certificates/..2025_02_24_06_25_03.3594477318/image-registry.openshift-image-registry.svc..5000 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~configmap/registry-certificates/..2025_02_24_06_25_03.3594477318/image-registry.openshift-image-registry.svc.cluster.local..5000 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~configmap/registry-certificates/..2025_02_24_06_25_03.3594477318/default-route-openshift-image-registry.apps-crc.testing not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~configmap/registry-certificates/..data not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~configmap/registry-certificates/image-registry.openshift-image-registry.svc..5000 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~configmap/registry-certificates/image-registry.openshift-image-registry.svc.cluster.local..5000 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~configmap/registry-certificates/default-route-openshift-image-registry.apps-crc.testing not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/edk2 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/edk2/cacerts.bin not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/java not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/java/cacerts not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/openssl not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/openssl/ca-bundle.trust.crt not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/tls-ca-bundle.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/email-ca-bundle.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/objsign-ca-bundle.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/2ae6433e.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/fde84897.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/75680d2e.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/openshift-service-serving-signer_1740288168.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/facfc4fa.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/8f5a969c.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/CFCA_EV_ROOT.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/9ef4a08a.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/ingress-operator_1740288202.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/2f332aed.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/248c8271.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/8d10a21f.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/ACCVRAIZ1.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/a94d09e5.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/3c9a4d3b.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/40193066.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/AC_RAIZ_FNMT-RCM.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/cd8c0d63.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/b936d1c6.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/CA_Disig_Root_R2.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/4fd49c6c.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/AC_RAIZ_FNMT-RCM_SERVIDORES_SEGUROS.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/b81b93f0.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/5f9a69fa.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/Certigna.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/b30d5fda.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/ANF_Secure_Server_Root_CA.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/b433981b.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/93851c9e.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/9282e51c.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/e7dd1bc4.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/Actalis_Authentication_Root_CA.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/930ac5d2.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/5f47b495.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/e113c810.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/5931b5bc.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/AffirmTrust_Commercial.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/2b349938.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/e48193cf.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/302904dd.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/a716d4ed.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/AffirmTrust_Networking.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/93bc0acc.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/86212b19.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/Certigna_Root_CA.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/AffirmTrust_Premium.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/b727005e.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/dbc54cab.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/f51bb24c.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/c28a8a30.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/AffirmTrust_Premium_ECC.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/9c8dfbd4.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/ccc52f49.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/cb1c3204.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/Amazon_Root_CA_1.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/ce5e74ef.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/fd08c599.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/Certum_Trusted_Root_CA.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/Amazon_Root_CA_2.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/6d41d539.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/fb5fa911.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/e35234b1.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/Amazon_Root_CA_3.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/8cb5ee0f.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/7a7c655d.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/f8fc53da.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/Amazon_Root_CA_4.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/de6d66f3.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/d41b5e2a.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/41a3f684.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/1df5a75f.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/Atos_TrustedRoot_2011.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/e36a6752.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/b872f2b4.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/9576d26b.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/228f89db.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/Atos_TrustedRoot_Root_CA_ECC_TLS_2021.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/fb717492.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/2d21b73c.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/0b1b94ef.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/595e996b.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/Atos_TrustedRoot_Root_CA_RSA_TLS_2021.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/9b46e03d.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/128f4b91.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/Buypass_Class_3_Root_CA.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/81f2d2b1.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/Autoridad_de_Certificacion_Firmaprofesional_CIF_A62634068.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/3bde41ac.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/d16a5865.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/Certum_EC-384_CA.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/BJCA_Global_Root_CA1.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/0179095f.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/ffa7f1eb.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/9482e63a.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/d4dae3dd.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/BJCA_Global_Root_CA2.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/3e359ba6.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/7e067d03.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/95aff9e3.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/d7746a63.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/Baltimore_CyberTrust_Root.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/653b494a.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/3ad48a91.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/Certum_Trusted_Network_CA.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/Buypass_Class_2_Root_CA.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/54657681.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/82223c44.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/e8de2f56.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/2d9dafe4.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/d96b65e2.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/ee64a828.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/COMODO_Certification_Authority.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/40547a79.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/5a3f0ff8.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/7a780d93.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/34d996fb.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/COMODO_ECC_Certification_Authority.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/eed8c118.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/89c02a45.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/Certainly_Root_R1.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/b1159c4c.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/COMODO_RSA_Certification_Authority.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/d6325660.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/d4c339cb.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/8312c4c1.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/Certainly_Root_E1.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/8508e720.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/5fdd185d.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/48bec511.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/69105f4f.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/GlobalSign.1.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/0b9bc432.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/Certum_Trusted_Network_CA_2.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/GTS_Root_R3.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/32888f65.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/CommScope_Public_Trust_ECC_Root-01.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/6b03dec0.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/219d9499.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/CommScope_Public_Trust_ECC_Root-02.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/5acf816d.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/cbf06781.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/CommScope_Public_Trust_RSA_Root-01.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/GTS_Root_R4.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/dc99f41e.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/CommScope_Public_Trust_RSA_Root-02.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/GlobalSign.3.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/AAA_Certificate_Services.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/985c1f52.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/8794b4e3.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/D-TRUST_BR_Root_CA_1_2020.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/e7c037b4.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/ef954a4e.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/D-TRUST_EV_Root_CA_1_2020.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/2add47b6.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/90c5a3c8.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/D-TRUST_Root_Class_3_CA_2_2009.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/b0f3e76e.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/53a1b57a.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/D-TRUST_Root_Class_3_CA_2_EV_2009.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/GlobalSign_Root_CA.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/DigiCert_Assured_ID_Root_CA.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/5ad8a5d6.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/68dd7389.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/DigiCert_Assured_ID_Root_G2.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/9d04f354.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/8d6437c3.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/062cdee6.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/bd43e1dd.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/DigiCert_Assured_ID_Root_G3.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/7f3d5d1d.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/c491639e.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/GlobalSign_Root_E46.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/DigiCert_Global_Root_CA.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/3513523f.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/399e7759.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/feffd413.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/d18e9066.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/DigiCert_Global_Root_G2.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/607986c7.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/c90bc37d.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/1b0f7e5c.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/1e08bfd1.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/DigiCert_Global_Root_G3.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/dd8e9d41.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/ed39abd0.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/a3418fda.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/bc3f2570.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/DigiCert_High_Assurance_EV_Root_CA.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/244b5494.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/81b9768f.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/GlobalSign.2.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/4be590e0.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/DigiCert_TLS_ECC_P384_Root_G5.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/9846683b.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/252252d2.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/1e8e7201.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/ISRG_Root_X1.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/DigiCert_TLS_RSA4096_Root_G5.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/d52c538d.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/c44cc0c0.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/GlobalSign_Root_R46.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/DigiCert_Trusted_Root_G4.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/75d1b2ed.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/a2c66da8.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/GTS_Root_R2.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/ecccd8db.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/Entrust.net_Certification_Authority__2048_.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/aee5f10d.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/3e7271e8.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/b0e59380.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/4c3982f2.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/Entrust_Root_Certification_Authority.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/6b99d060.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/bf64f35b.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/0a775a30.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/002c0b4f.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/cc450945.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/Entrust_Root_Certification_Authority_-_EC1.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/106f3e4d.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/b3fb433b.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/GlobalSign.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/4042bcee.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/Entrust_Root_Certification_Authority_-_G2.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/02265526.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/455f1b52.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/0d69c7e1.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/9f727ac7.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/Entrust_Root_Certification_Authority_-_G4.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/5e98733a.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/f0cd152c.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/dc4d6a89.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/6187b673.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/FIRMAPROFESIONAL_CA_ROOT-A_WEB.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/ba8887ce.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/068570d1.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/f081611a.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/48a195d8.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/GDCA_TrustAUTH_R5_ROOT.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/0f6fa695.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/ab59055e.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/b92fd57f.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/GLOBALTRUST_2020.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/fa5da96b.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/1ec40989.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/7719f463.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/GTS_Root_R1.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/1001acf7.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/f013ecaf.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/626dceaf.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/c559d742.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/1d3472b9.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/9479c8c3.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/a81e292b.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/4bfab552.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/Go_Daddy_Class_2_Certification_Authority.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/Sectigo_Public_Server_Authentication_Root_E46.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/Go_Daddy_Root_Certificate_Authority_-_G2.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/e071171e.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/57bcb2da.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/HARICA_TLS_ECC_Root_CA_2021.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/ab5346f4.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/5046c355.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/HARICA_TLS_RSA_Root_CA_2021.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/865fbdf9.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/da0cfd1d.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/85cde254.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/Hellenic_Academic_and_Research_Institutions_ECC_RootCA_2015.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/cbb3f32b.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/SecureSign_RootCA11.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/Hellenic_Academic_and_Research_Institutions_RootCA_2015.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/5860aaa6.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/31188b5e.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/HiPKI_Root_CA_-_G1.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/c7f1359b.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/5f15c80c.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/Hongkong_Post_Root_CA_3.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/09789157.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/ISRG_Root_X2.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/18856ac4.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/1e09d511.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/IdenTrust_Commercial_Root_CA_1.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/cf701eeb.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/d06393bb.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/IdenTrust_Public_Sector_Root_CA_1.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/10531352.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/Izenpe.com.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/SecureTrust_CA.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/b0ed035a.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/Microsec_e-Szigno_Root_CA_2009.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/8160b96c.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/e8651083.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/2c63f966.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/Security_Communication_RootCA2.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/Microsoft_ECC_Root_Certificate_Authority_2017.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/8d89cda1.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/01419da9.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/SSL.com_TLS_RSA_Root_CA_2022.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/b7a5b843.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/Microsoft_RSA_Root_Certificate_Authority_2017.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/bf53fb88.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/9591a472.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/3afde786.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/SwissSign_Gold_CA_-_G2.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/NAVER_Global_Root_Certification_Authority.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/3fb36b73.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/d39b0a2c.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/a89d74c2.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/cd58d51e.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/b7db1890.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/NetLock_Arany__Class_Gold__F__tan__s__tv__ny.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/988a38cb.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/60afe812.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/f39fc864.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/5443e9e3.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/OISTE_WISeKey_Global_Root_GB_CA.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/e73d606e.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/dfc0fe80.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/b66938e9.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/1e1eab7c.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/OISTE_WISeKey_Global_Root_GC_CA.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/773e07ad.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/3c899c73.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/d59297b8.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/ddcda989.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/QuoVadis_Root_CA_1_G3.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/749e9e03.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/52b525c7.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/Security_Communication_RootCA3.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/QuoVadis_Root_CA_2.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/d7e8dc79.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/7a819ef2.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/08063a00.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:08 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/6b483515.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/QuoVadis_Root_CA_2_G3.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/064e0aa9.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/1f58a078.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/6f7454b3.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/7fa05551.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/QuoVadis_Root_CA_3.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/76faf6c0.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/9339512a.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/f387163d.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/ee37c333.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/QuoVadis_Root_CA_3_G3.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/e18bfb83.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/e442e424.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/fe8a2cd8.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/23f4c490.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/5cd81ad7.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/SSL.com_EV_Root_Certification_Authority_ECC.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/f0c70a8d.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/7892ad52.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/SZAFIR_ROOT_CA2.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/4f316efb.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/SSL.com_EV_Root_Certification_Authority_RSA_R2.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/06dc52d5.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/583d0756.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/Sectigo_Public_Server_Authentication_Root_R46.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/SSL.com_Root_Certification_Authority_ECC.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/0bf05006.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/88950faa.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/9046744a.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/3c860d51.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/SSL.com_Root_Certification_Authority_RSA.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/6fa5da56.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/33ee480d.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/Secure_Global_CA.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/63a2c897.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/SSL.com_TLS_ECC_Root_CA_2022.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/bdacca6f.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/ff34af3f.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/dbff3a01.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/Security_Communication_ECC_RootCA1.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/emSign_Root_CA_-_C1.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/Starfield_Class_2_Certification_Authority.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/406c9bb1.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/Starfield_Root_Certificate_Authority_-_G2.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/emSign_ECC_Root_CA_-_C3.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/Starfield_Services_Root_Certificate_Authority_-_G2.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/SwissSign_Silver_CA_-_G2.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/99e1b953.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/T-TeleSec_GlobalRoot_Class_2.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/vTrus_Root_CA.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/T-TeleSec_GlobalRoot_Class_3.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/14bc7599.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/TUBITAK_Kamu_SM_SSL_Kok_Sertifikasi_-_Surum_1.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/TWCA_Global_Root_CA.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/7a3adc42.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/TWCA_Root_Certification_Authority.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/f459871d.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/Telekom_Security_TLS_ECC_Root_2020.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/emSign_Root_CA_-_G1.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/Telekom_Security_TLS_RSA_Root_2023.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/TeliaSonera_Root_CA_v1.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/Telia_Root_CA_v2.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/8f103249.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/f058632f.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/ca-certificates.crt not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/TrustAsia_Global_Root_CA_G3.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/9bf03295.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/98aaf404.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/ca-bundle.crt not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/TrustAsia_Global_Root_CA_G4.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/1cef98f5.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/073bfcc5.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/2923b3f9.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/Trustwave_Global_Certification_Authority.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/f249de83.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/edcbddb5.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/emSign_ECC_Root_CA_-_G3.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/Trustwave_Global_ECC_P256_Certification_Authority.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/9b5697b0.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/1ae85e5e.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/b74d2bd5.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/Trustwave_Global_ECC_P384_Certification_Authority.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/d887a5bb.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/9aef356c.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/TunTrust_Root_CA.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/fd64f3fc.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/e13665f9.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/UCA_Extended_Validation_Root.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/0f5dc4f3.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/da7377f6.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/UCA_Global_G2_Root.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/c01eb047.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/304d27c3.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/ed858448.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/USERTrust_ECC_Certification_Authority.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/f30dd6ad.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/04f60c28.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/vTrus_ECC_Root_CA.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/USERTrust_RSA_Certification_Authority.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/fc5a8f99.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/35105088.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/ee532fd5.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/XRamp_Global_Certification_Authority.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/706f604c.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/76579174.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/certSIGN_ROOT_CA.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/8d86cdd1.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/882de061.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/certSIGN_ROOT_CA_G2.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/5f618aec.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/a9d40e02.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/e-Szigno_Root_CA_2017.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/e868b802.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/83e9984f.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/ePKI_Root_Certification_Authority.pem not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/ca6e4ad9.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/9d6523ce.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/4b718d9b.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes/kubernetes.io~empty-dir/ca-trust-extracted/pem/directory-hash/869fbf79.0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/etc-hosts not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/containers/registry/f8d22bdb not reset as customized by admin to system_u:object_r:container_file_t:s0:c10,c16 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/a0128f3a-b052-44ed-a84e-c4c8aaf17c13/etc-hosts not reset as customized by admin to system_u:object_r:container_file_t:s0:c9,c17 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/a0128f3a-b052-44ed-a84e-c4c8aaf17c13/containers/cluster-samples-operator/6e8bbfac not reset as customized by admin to system_u:object_r:container_file_t:s0:c9,c17 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/a0128f3a-b052-44ed-a84e-c4c8aaf17c13/containers/cluster-samples-operator/54dd7996 not reset as customized by admin to system_u:object_r:container_file_t:s0:c9,c17 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/a0128f3a-b052-44ed-a84e-c4c8aaf17c13/containers/cluster-samples-operator/a4f1bb05 not reset as customized by admin to system_u:object_r:container_file_t:s0:c9,c17 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/a0128f3a-b052-44ed-a84e-c4c8aaf17c13/containers/cluster-samples-operator-watch/207129da not reset as customized by admin to system_u:object_r:container_file_t:s0:c9,c17 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/a0128f3a-b052-44ed-a84e-c4c8aaf17c13/containers/cluster-samples-operator-watch/c1df39e1 not reset as customized by admin to system_u:object_r:container_file_t:s0:c9,c17 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/a0128f3a-b052-44ed-a84e-c4c8aaf17c13/containers/cluster-samples-operator-watch/15b8f1cd not reset as customized by admin to system_u:object_r:container_file_t:s0:c9,c17 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/6402fda4-df10-493c-b4e5-d0569419652d/volumes/kubernetes.io~configmap/config not reset as customized by admin to system_u:object_r:container_file_t:s0:c0,c15 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/6402fda4-df10-493c-b4e5-d0569419652d/volumes/kubernetes.io~configmap/config/..2025_02_23_05_27_49.3523263858 not reset as customized by admin to system_u:object_r:container_file_t:s0:c0,c15 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/6402fda4-df10-493c-b4e5-d0569419652d/volumes/kubernetes.io~configmap/config/..2025_02_23_05_27_49.3523263858/config-file.yaml not reset as customized by admin to system_u:object_r:container_file_t:s0:c0,c15 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/6402fda4-df10-493c-b4e5-d0569419652d/volumes/kubernetes.io~configmap/config/..data not reset as customized by admin to system_u:object_r:container_file_t:s0:c0,c15 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/6402fda4-df10-493c-b4e5-d0569419652d/volumes/kubernetes.io~configmap/config/config-file.yaml not reset as customized by admin to system_u:object_r:container_file_t:s0:c0,c15 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/6402fda4-df10-493c-b4e5-d0569419652d/volumes/kubernetes.io~configmap/images not reset as customized by admin to system_u:object_r:container_file_t:s0:c0,c15 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/6402fda4-df10-493c-b4e5-d0569419652d/volumes/kubernetes.io~configmap/images/..2025_02_23_05_27_49.3256605594 not reset as customized by admin to system_u:object_r:container_file_t:s0:c0,c15 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/6402fda4-df10-493c-b4e5-d0569419652d/volumes/kubernetes.io~configmap/images/..2025_02_23_05_27_49.3256605594/images.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c0,c15 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/6402fda4-df10-493c-b4e5-d0569419652d/volumes/kubernetes.io~configmap/images/..data not reset as customized by admin to system_u:object_r:container_file_t:s0:c0,c15 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/6402fda4-df10-493c-b4e5-d0569419652d/volumes/kubernetes.io~configmap/images/images.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c0,c15 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/6402fda4-df10-493c-b4e5-d0569419652d/etc-hosts not reset as customized by admin to system_u:object_r:container_file_t:s0:c0,c15 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/6402fda4-df10-493c-b4e5-d0569419652d/containers/kube-rbac-proxy/77bd6913 not reset as customized by admin to system_u:object_r:container_file_t:s0:c0,c15 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/6402fda4-df10-493c-b4e5-d0569419652d/containers/kube-rbac-proxy/2382c1b1 not reset as customized by admin to system_u:object_r:container_file_t:s0:c0,c15 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/6402fda4-df10-493c-b4e5-d0569419652d/containers/kube-rbac-proxy/704ce128 not reset as customized by admin to system_u:object_r:container_file_t:s0:c0,c15 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/6402fda4-df10-493c-b4e5-d0569419652d/containers/machine-api-operator/70d16fe0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c0,c15 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/6402fda4-df10-493c-b4e5-d0569419652d/containers/machine-api-operator/bfb95535 not reset as customized by admin to system_u:object_r:container_file_t:s0:c0,c15 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/6402fda4-df10-493c-b4e5-d0569419652d/containers/machine-api-operator/57a8e8e2 not reset as customized by admin to system_u:object_r:container_file_t:s0:c0,c15 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/e7e6199b-1264-4501-8953-767f51328d08/volumes/kubernetes.io~configmap/config not reset as customized by admin to system_u:object_r:container_file_t:s0:c219,c404 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/e7e6199b-1264-4501-8953-767f51328d08/volumes/kubernetes.io~configmap/config/..2025_02_23_05_27_49.3413793711 not reset as customized by admin to system_u:object_r:container_file_t:s0:c219,c404 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/e7e6199b-1264-4501-8953-767f51328d08/volumes/kubernetes.io~configmap/config/..2025_02_23_05_27_49.3413793711/config.yaml not reset as customized by admin to system_u:object_r:container_file_t:s0:c219,c404 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/e7e6199b-1264-4501-8953-767f51328d08/volumes/kubernetes.io~configmap/config/..data not reset as customized by admin to system_u:object_r:container_file_t:s0:c219,c404 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/e7e6199b-1264-4501-8953-767f51328d08/volumes/kubernetes.io~configmap/config/config.yaml not reset as customized by admin to system_u:object_r:container_file_t:s0:c219,c404 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/e7e6199b-1264-4501-8953-767f51328d08/etc-hosts not reset as customized by admin to system_u:object_r:container_file_t:s0:c219,c404 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/e7e6199b-1264-4501-8953-767f51328d08/containers/kube-apiserver-operator/1b9d3e5e not reset as customized by admin to system_u:object_r:container_file_t:s0:c107,c917 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/e7e6199b-1264-4501-8953-767f51328d08/containers/kube-apiserver-operator/fddb173c not reset as customized by admin to system_u:object_r:container_file_t:s0:c202,c983 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/e7e6199b-1264-4501-8953-767f51328d08/containers/kube-apiserver-operator/95d3c6c4 not reset as customized by admin to system_u:object_r:container_file_t:s0:c219,c404 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/9d751cbb-f2e2-430d-9754-c882a5e924a5/etc-hosts not reset as customized by admin to system_u:object_r:container_file_t:s0:c20,c21 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/9d751cbb-f2e2-430d-9754-c882a5e924a5/containers/check-endpoints/bfb5fff5 not reset as customized by admin to system_u:object_r:container_file_t:s0:c20,c21 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/9d751cbb-f2e2-430d-9754-c882a5e924a5/containers/check-endpoints/2aef40aa not reset as customized by admin to system_u:object_r:container_file_t:s0:c20,c21 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/9d751cbb-f2e2-430d-9754-c882a5e924a5/containers/check-endpoints/c0391cad not reset as customized by admin to system_u:object_r:container_file_t:s0:c20,c21 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/f614b9022728cf315e60c057852e563e/etc-hosts not reset as customized by admin to system_u:object_r:container_file_t:s0:c214,c928 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/f614b9022728cf315e60c057852e563e/containers/kube-controller-manager/1119e69d not reset as customized by admin to system_u:object_r:container_file_t:s0:c776,c1007 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/f614b9022728cf315e60c057852e563e/containers/kube-controller-manager/660608b4 not reset as customized by admin to system_u:object_r:container_file_t:s0:c214,c928 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/f614b9022728cf315e60c057852e563e/containers/kube-controller-manager/8220bd53 not reset as customized by admin to system_u:object_r:container_file_t:s0:c214,c928 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/f614b9022728cf315e60c057852e563e/containers/cluster-policy-controller/85f99d5c not reset as customized by admin to system_u:object_r:container_file_t:s0:c776,c1007 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/f614b9022728cf315e60c057852e563e/containers/cluster-policy-controller/4b0225f6 not reset as customized by admin to system_u:object_r:container_file_t:s0:c214,c928 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/f614b9022728cf315e60c057852e563e/containers/kube-controller-manager-cert-syncer/9c2a3394 not reset as customized by admin to system_u:object_r:container_file_t:s0:c776,c1007 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/f614b9022728cf315e60c057852e563e/containers/kube-controller-manager-cert-syncer/e820b243 not reset as customized by admin to system_u:object_r:container_file_t:s0:c214,c928 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/f614b9022728cf315e60c057852e563e/containers/kube-controller-manager-recovery-controller/1ca52ea0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c776,c1007 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/f614b9022728cf315e60c057852e563e/containers/kube-controller-manager-recovery-controller/e6988e45 not reset as customized by admin to system_u:object_r:container_file_t:s0:c214,c928 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/0b574797-001e-440a-8f4e-c0be86edad0f/volumes/kubernetes.io~configmap/mcc-auth-proxy-config not reset as customized by admin to system_u:object_r:container_file_t:s0:c4,c17 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/0b574797-001e-440a-8f4e-c0be86edad0f/volumes/kubernetes.io~configmap/mcc-auth-proxy-config/..data not reset as customized by admin to system_u:object_r:container_file_t:s0:c4,c17 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/0b574797-001e-440a-8f4e-c0be86edad0f/volumes/kubernetes.io~configmap/mcc-auth-proxy-config/config-file.yaml not reset as customized by admin to system_u:object_r:container_file_t:s0:c4,c17 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/0b574797-001e-440a-8f4e-c0be86edad0f/volumes/kubernetes.io~configmap/mcc-auth-proxy-config/..2025_02_24_06_09_21.2517297950 not reset as customized by admin to system_u:object_r:container_file_t:s0:c4,c17 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/0b574797-001e-440a-8f4e-c0be86edad0f/volumes/kubernetes.io~configmap/mcc-auth-proxy-config/..2025_02_24_06_09_21.2517297950/config-file.yaml not reset as customized by admin to system_u:object_r:container_file_t:s0:c4,c17 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/0b574797-001e-440a-8f4e-c0be86edad0f/etc-hosts not reset as customized by admin to system_u:object_r:container_file_t:s0:c4,c17 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/0b574797-001e-440a-8f4e-c0be86edad0f/containers/machine-config-controller/6655f00b not reset as customized by admin to system_u:object_r:container_file_t:s0:c4,c17 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/0b574797-001e-440a-8f4e-c0be86edad0f/containers/machine-config-controller/98bc3986 not reset as customized by admin to system_u:object_r:container_file_t:s0:c4,c17 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/0b574797-001e-440a-8f4e-c0be86edad0f/containers/machine-config-controller/08e3458a not reset as customized by admin to system_u:object_r:container_file_t:s0:c4,c17 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/0b574797-001e-440a-8f4e-c0be86edad0f/containers/kube-rbac-proxy/2a191cb0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c4,c17 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/0b574797-001e-440a-8f4e-c0be86edad0f/containers/kube-rbac-proxy/6c4eeefb not reset as customized by admin to system_u:object_r:container_file_t:s0:c4,c17 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/0b574797-001e-440a-8f4e-c0be86edad0f/containers/kube-rbac-proxy/f61a549c not reset as customized by admin to system_u:object_r:container_file_t:s0:c4,c17 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/bd23aa5c-e532-4e53-bccf-e79f130c5ae8/etc-hosts not reset as customized by admin to system_u:object_r:container_file_t:s0:c318,c553 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/bd23aa5c-e532-4e53-bccf-e79f130c5ae8/containers/hostpath-provisioner/24891863 not reset as customized by admin to system_u:object_r:container_file_t:s0:c37,c572 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/bd23aa5c-e532-4e53-bccf-e79f130c5ae8/containers/hostpath-provisioner/fbdfd89c not reset as customized by admin to system_u:object_r:container_file_t:s0:c318,c553 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/bd23aa5c-e532-4e53-bccf-e79f130c5ae8/containers/liveness-probe/9b63b3bc not reset as customized by admin to system_u:object_r:container_file_t:s0:c37,c572 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/bd23aa5c-e532-4e53-bccf-e79f130c5ae8/containers/liveness-probe/8acde6d6 not reset as customized by admin to system_u:object_r:container_file_t:s0:c318,c553 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/bd23aa5c-e532-4e53-bccf-e79f130c5ae8/containers/node-driver-registrar/59ecbba3 not reset as customized by admin to system_u:object_r:container_file_t:s0:c318,c553 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/bd23aa5c-e532-4e53-bccf-e79f130c5ae8/containers/csi-provisioner/685d4be3 not reset as customized by admin to system_u:object_r:container_file_t:s0:c318,c553 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5441d097-087c-4d9a-baa8-b210afa90fc9/volumes/kubernetes.io~configmap/config not reset as customized by admin to system_u:object_r:container_file_t:s0:c2,c23 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5441d097-087c-4d9a-baa8-b210afa90fc9/volumes/kubernetes.io~configmap/config/..2025_02_24_06_20_07.341639300 not reset as customized by admin to system_u:object_r:container_file_t:s0:c2,c23 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5441d097-087c-4d9a-baa8-b210afa90fc9/volumes/kubernetes.io~configmap/config/..2025_02_24_06_20_07.341639300/config.yaml not reset as customized by admin to system_u:object_r:container_file_t:s0:c2,c23 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5441d097-087c-4d9a-baa8-b210afa90fc9/volumes/kubernetes.io~configmap/config/..2025_02_24_06_20_07.341639300/openshift-route-controller-manager.client-ca.configmap not reset as customized by admin to system_u:object_r:container_file_t:s0:c2,c23 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5441d097-087c-4d9a-baa8-b210afa90fc9/volumes/kubernetes.io~configmap/config/..2025_02_24_06_20_07.341639300/openshift-route-controller-manager.serving-cert.secret not reset as customized by admin to system_u:object_r:container_file_t:s0:c2,c23 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5441d097-087c-4d9a-baa8-b210afa90fc9/volumes/kubernetes.io~configmap/config/..data not reset as customized by admin to system_u:object_r:container_file_t:s0:c2,c23 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5441d097-087c-4d9a-baa8-b210afa90fc9/volumes/kubernetes.io~configmap/config/config.yaml not reset as customized by admin to system_u:object_r:container_file_t:s0:c2,c23 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5441d097-087c-4d9a-baa8-b210afa90fc9/volumes/kubernetes.io~configmap/config/openshift-route-controller-manager.client-ca.configmap not reset as customized by admin to system_u:object_r:container_file_t:s0:c2,c23 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5441d097-087c-4d9a-baa8-b210afa90fc9/volumes/kubernetes.io~configmap/config/openshift-route-controller-manager.serving-cert.secret not reset as customized by admin to system_u:object_r:container_file_t:s0:c2,c23 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5441d097-087c-4d9a-baa8-b210afa90fc9/volumes/kubernetes.io~configmap/client-ca not reset as customized by admin to system_u:object_r:container_file_t:s0:c2,c23 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5441d097-087c-4d9a-baa8-b210afa90fc9/volumes/kubernetes.io~configmap/client-ca/..2025_02_24_06_20_07.2950937851 not reset as customized by admin to system_u:object_r:container_file_t:s0:c2,c23 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5441d097-087c-4d9a-baa8-b210afa90fc9/volumes/kubernetes.io~configmap/client-ca/..2025_02_24_06_20_07.2950937851/ca-bundle.crt not reset as customized by admin to system_u:object_r:container_file_t:s0:c2,c23 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5441d097-087c-4d9a-baa8-b210afa90fc9/volumes/kubernetes.io~configmap/client-ca/..data not reset as customized by admin to system_u:object_r:container_file_t:s0:c2,c23 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5441d097-087c-4d9a-baa8-b210afa90fc9/volumes/kubernetes.io~configmap/client-ca/ca-bundle.crt not reset as customized by admin to system_u:object_r:container_file_t:s0:c2,c23 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5441d097-087c-4d9a-baa8-b210afa90fc9/etc-hosts not reset as customized by admin to system_u:object_r:container_file_t:s0:c2,c23 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5441d097-087c-4d9a-baa8-b210afa90fc9/containers/route-controller-manager/feaea55e not reset as customized by admin to system_u:object_r:container_file_t:s0:c2,c23 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/abinitio-runtime-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/abinitio-runtime-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/accuknox-operator-certified not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/accuknox-operator-certified/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/aci-containers-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/aci-containers-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/aikit-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/aikit-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/airlock-microgateway not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/airlock-microgateway/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/ako-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/ako-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/alloy not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/alloy/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/anchore-engine not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/anchore-engine/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/anzo-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/anzo-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/anzograph-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/anzograph-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/anzounstructured-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/anzounstructured-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/appdynamics-cloud-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/appdynamics-cloud-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/appdynamics-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/appdynamics-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/aqua-operator-certified not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/aqua-operator-certified/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/cass-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/cass-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/ccm-node-agent-dcap-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/ccm-node-agent-dcap-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/ccm-node-agent-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/ccm-node-agent-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/cfm-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/cfm-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/cilium not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/cilium/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/cilium-enterprise not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/cilium-enterprise/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/cloud-native-postgresql not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/cloud-native-postgresql/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/cloudbees-ci not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/cloudbees-ci/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/cloudera-streams-messaging-kubernetes-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/cloudera-streams-messaging-kubernetes-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/cloudnative-pg not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/cloudnative-pg/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/cnfv-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/cnfv-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/cockroachdb-certified not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/cockroachdb-certified/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/conjur-follower-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/conjur-follower-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/coroot-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/coroot-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/crunchy-postgres-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/crunchy-postgres-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/cte-k8s-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/cte-k8s-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/datadog-operator-certified not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/datadog-operator-certified/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/dell-csm-operator-certified not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/dell-csm-operator-certified/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/digitalai-deploy-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/digitalai-deploy-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/digitalai-release-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/digitalai-release-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/dynatrace-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/dynatrace-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/edb-hcp-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/edb-hcp-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/eginnovations-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/eginnovations-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/elasticsearch-eck-operator-certified not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/elasticsearch-eck-operator-certified/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/falcon-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/falcon-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/federatorai-certified not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/federatorai-certified/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/fujitsu-enterprise-postgres-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/fujitsu-enterprise-postgres-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/function-mesh not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/function-mesh/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/harness-gitops-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/harness-gitops-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/hazelcast-platform-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/hazelcast-platform-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/hcp-terraform-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/hcp-terraform-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/hpe-ezmeral-csi-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/hpe-ezmeral-csi-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/ibm-application-gateway-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/ibm-application-gateway-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/ibm-block-csi-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/ibm-block-csi-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/ibm-security-verify-access-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/ibm-security-verify-access-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/ibm-security-verify-directory-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/ibm-security-verify-directory-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/ibm-security-verify-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/ibm-security-verify-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/infoscale-dr-manager not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/infoscale-dr-manager/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/infoscale-licensing-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/infoscale-licensing-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/infoscale-sds-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/infoscale-sds-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/infrastructure-asset-orchestrator-certified not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/infrastructure-asset-orchestrator-certified/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/instana-agent-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/instana-agent-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/intel-device-plugins-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/intel-device-plugins-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/intel-kubernetes-power-manager not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/intel-kubernetes-power-manager/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/iomesh-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/iomesh-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/joget-dx-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/joget-dx-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/joget-dx8-openshift-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/joget-dx8-openshift-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/joget-dx8-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/joget-dx8-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/k8s-triliovault not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/k8s-triliovault/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/keysight-ati-updates not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/keysight-ati-updates/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/keysight-kcos-framework not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/keysight-kcos-framework/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/keysight-kcos-ingress not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/keysight-kcos-ingress/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/keysight-kcos-licensing not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/keysight-kcos-licensing/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/keysight-kcos-sso not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/keysight-kcos-sso/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/keysight-keycloak-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/keysight-keycloak-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/keysight-load-core not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/keysight-load-core/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/keysight-loadcore-agents not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/keysight-loadcore-agents/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/keysight-nats-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/keysight-nats-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/keysight-nimbusmosaic-dusim not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/keysight-nimbusmosaic-dusim/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/keysight-rest-api-browser-v1 not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/keysight-rest-api-browser-v1/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/keysight-wap-appsec not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/keysight-wap-appsec/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/keysight-wap-core not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/keysight-wap-core/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/keysight-wap-db not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/keysight-wap-db/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/keysight-wap-diagnostics not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/keysight-wap-diagnostics/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/keysight-wap-logging not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/keysight-wap-logging/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/keysight-wap-migration not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/keysight-wap-migration/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/keysight-wap-msg-broker not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/keysight-wap-msg-broker/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/keysight-wap-notifications not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/keysight-wap-notifications/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/keysight-wap-stats-dashboards not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/keysight-wap-stats-dashboards/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/keysight-wap-storage not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/keysight-wap-storage/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/keysight-wap-test-core not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/keysight-wap-test-core/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/keysight-wap-ui not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/keysight-wap-ui/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/keysight-websocket-service not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/keysight-websocket-service/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/kong-gateway-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/kong-gateway-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/kubearmor-operator-certified not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/kubearmor-operator-certified/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/kubecost-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/kubecost-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/kubemq-operator-marketplace not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/kubemq-operator-marketplace/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/kubeturbo-certified not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/kubeturbo-certified/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/lenovo-locd-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/lenovo-locd-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/marketplace-games-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/marketplace-games-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/memcached-operator-ogaye not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/memcached-operator-ogaye/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/memory-machine-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/memory-machine-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/model-builder-for-vision-certified not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/model-builder-for-vision-certified/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/mongodb-atlas-kubernetes not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/mongodb-atlas-kubernetes/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/mongodb-enterprise not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/mongodb-enterprise/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/netapp-spark-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/netapp-spark-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/netscaler-adm-agent-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/netscaler-adm-agent-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/netscaler-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/netscaler-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/neuvector-certified-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/neuvector-certified-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/nexus-repository-ha-operator-certified not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/nexus-repository-ha-operator-certified/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/nginx-ingress-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/nginx-ingress-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/pcc-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/pcc-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/nim-operator-certified not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/nim-operator-certified/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/nxiq-operator-certified not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/nxiq-operator-certified/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/nxrm-operator-certified not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/nxrm-operator-certified/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/odigos-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/odigos-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/open-liberty-certified not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/open-liberty-certified/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/openshiftartifactoryha-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/openshiftartifactoryha-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/openshiftxray-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/openshiftxray-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/operator-certification-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/operator-certification-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/ovms-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/ovms-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/pachyderm-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/pachyderm-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/pmem-csi-operator-os not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/pmem-csi-operator-os/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/portworx-certified not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/portworx-certified/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/prometurbo-certified not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/prometurbo-certified/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/pubsubplus-eventbroker-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/pubsubplus-eventbroker-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/redis-enterprise-operator-cert not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/redis-enterprise-operator-cert/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/runtime-component-operator-certified not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/runtime-component-operator-certified/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/runtime-fabric-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/runtime-fabric-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/sanstoragecsi-operator-bundle not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/sanstoragecsi-operator-bundle/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/silicom-sts-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/silicom-sts-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/smilecdr-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/smilecdr-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/sriov-fec not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/sriov-fec/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/stackable-commons-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/stackable-commons-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/stackable-zookeeper-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/stackable-zookeeper-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/t8c-certified not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/t8c-certified/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/t8c-tsc-client-certified not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/t8c-tsc-client-certified/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/tawon-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/tawon-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/tigera-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/tigera-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/timemachine-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/timemachine-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/vault-secrets-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/vault-secrets-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/vcp-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/vcp-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/webotx-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/webotx-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/xcrypt-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/xcrypt-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/zabbix-operator-certified not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/catalog/zabbix-operator-certified/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/cache not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/cache/pogreb.v1 not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/cache/pogreb.v1/db not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/cache/pogreb.v1/db/00000-1.psg not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/cache/pogreb.v1/db/00000-1.psg.pmt not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/cache/pogreb.v1/db/db.pmt not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/cache/pogreb.v1/db/index.pmt not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/cache/pogreb.v1/db/main.pix not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/cache/pogreb.v1/db/overflow.pix not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/catalog-content/cache/pogreb.v1/digest not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/utilities not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes/kubernetes.io~empty-dir/utilities/copy-content not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/etc-hosts not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/containers/extract-utilities/63709497 not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/containers/extract-utilities/d966b7fd not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/containers/extract-utilities/f5773757 not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/containers/extract-content/81c9edb9 not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/containers/extract-content/57bf57ee not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/containers/extract-content/86f5e6aa not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/containers/registry-server/0aabe31d not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/containers/registry-server/d2af85c2 not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/containers/registry-server/09d157d9 not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/cache not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/cache/pogreb.v1 not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/cache/pogreb.v1/db not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/cache/pogreb.v1/db/00000-1.psg not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/cache/pogreb.v1/db/00000-1.psg.pmt not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/cache/pogreb.v1/db/db.pmt not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/cache/pogreb.v1/db/index.pmt not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/cache/pogreb.v1/db/main.pix not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/cache/pogreb.v1/db/overflow.pix not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/cache/pogreb.v1/digest not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/3scale-community-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/3scale-community-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/ack-acm-controller not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/ack-acm-controller/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/ack-acmpca-controller not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/ack-acmpca-controller/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/ack-apigateway-controller not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/ack-apigateway-controller/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/ack-apigatewayv2-controller not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/ack-apigatewayv2-controller/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/ack-applicationautoscaling-controller not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/ack-applicationautoscaling-controller/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/ack-athena-controller not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/ack-athena-controller/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/ack-cloudfront-controller not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/ack-cloudfront-controller/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/ack-cloudtrail-controller not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/ack-cloudtrail-controller/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/ack-cloudwatch-controller not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/ack-cloudwatch-controller/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/ack-cloudwatchlogs-controller not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/ack-cloudwatchlogs-controller/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/ack-documentdb-controller not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/ack-documentdb-controller/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/ack-dynamodb-controller not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/ack-dynamodb-controller/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/ack-ec2-controller not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/ack-ec2-controller/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/ack-ecr-controller not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/ack-ecr-controller/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/ack-ecs-controller not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/ack-ecs-controller/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/ack-efs-controller not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/ack-efs-controller/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/ack-eks-controller not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/ack-eks-controller/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/ack-elasticache-controller not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/ack-elasticache-controller/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/ack-elbv2-controller not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/ack-elbv2-controller/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/ack-emrcontainers-controller not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/ack-emrcontainers-controller/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/ack-eventbridge-controller not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/ack-eventbridge-controller/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/ack-iam-controller not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/ack-iam-controller/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/ack-kafka-controller not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/ack-kafka-controller/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/ack-keyspaces-controller not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/ack-keyspaces-controller/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/ack-kinesis-controller not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/ack-kinesis-controller/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/ack-kms-controller not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/ack-kms-controller/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/ack-lambda-controller not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/ack-lambda-controller/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/ack-memorydb-controller not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/ack-memorydb-controller/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/ack-mq-controller not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/ack-mq-controller/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/ack-networkfirewall-controller not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/ack-networkfirewall-controller/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/ack-opensearchservice-controller not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/ack-opensearchservice-controller/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/ack-organizations-controller not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/ack-organizations-controller/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/ack-pipes-controller not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/ack-pipes-controller/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/ack-prometheusservice-controller not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/ack-prometheusservice-controller/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/ack-rds-controller not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/ack-rds-controller/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/ack-recyclebin-controller not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/ack-recyclebin-controller/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/ack-route53-controller not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/ack-route53-controller/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/ack-route53resolver-controller not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/ack-route53resolver-controller/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/ack-s3-controller not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/ack-s3-controller/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/ack-sagemaker-controller not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/ack-sagemaker-controller/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/ack-secretsmanager-controller not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/ack-secretsmanager-controller/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/ack-ses-controller not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/ack-ses-controller/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/ack-sfn-controller not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/ack-sfn-controller/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/ack-sns-controller not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/ack-sns-controller/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/ack-sqs-controller not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/ack-sqs-controller/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/ack-ssm-controller not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/ack-ssm-controller/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/ack-wafv2-controller not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/ack-wafv2-controller/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/aerospike-kubernetes-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/aerospike-kubernetes-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/airflow-helm-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/airflow-helm-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/alloydb-omni-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/alloydb-omni-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/alvearie-imaging-ingestion not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/alvearie-imaging-ingestion/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/amd-gpu-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/amd-gpu-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/analytics-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/analytics-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/annotationlab not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/annotationlab/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/apicast-community-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/apicast-community-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/apicurio-api-controller not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/apicurio-api-controller/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/apicurio-registry not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/apicurio-registry/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/apicurito not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/apicurito/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/apimatic-kubernetes-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/apimatic-kubernetes-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/application-services-metering-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/application-services-metering-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/aqua not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/aqua/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/argocd-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/argocd-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/assisted-service-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/assisted-service-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/authorino-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/authorino-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/automotive-infra not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/automotive-infra/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/aws-efs-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/aws-efs-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/awss3-operator-registry not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/awss3-operator-registry/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/azure-service-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/azure-service-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/beegfs-csi-driver-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/beegfs-csi-driver-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/bpfman-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/bpfman-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/camel-k not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/camel-k/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/camel-karavan-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/camel-karavan-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/cass-operator-community not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/cass-operator-community/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/cert-manager not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/cert-manager/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/cert-utils-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/cert-utils-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/cluster-aas-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/cluster-aas-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/cluster-impairment-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/cluster-impairment-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/cluster-manager not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/cluster-manager/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/cockroachdb not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/cockroachdb/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/codeflare-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/codeflare-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/community-kubevirt-hyperconverged not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/community-kubevirt-hyperconverged/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/community-trivy-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/community-trivy-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/community-windows-machine-config-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/community-windows-machine-config-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/customized-user-remediation not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/customized-user-remediation/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/cxl-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/cxl-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/dapr-kubernetes-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/dapr-kubernetes-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/datadog-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/datadog-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/datatrucker-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/datatrucker-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/dbaas-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/dbaas-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/debezium-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/debezium-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/dell-csm-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/dell-csm-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/deployment-validation-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/deployment-validation-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/devopsinabox not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/devopsinabox/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/dns-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/dns-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/dynatrace-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/dynatrace-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/eclipse-amlen-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/eclipse-amlen-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/eclipse-che not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/eclipse-che/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/ecr-secret-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/ecr-secret-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/edp-keycloak-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/edp-keycloak-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/eginnovations-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/eginnovations-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/egressip-ipam-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/egressip-ipam-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/ember-csi-community-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/ember-csi-community-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/etcd not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/etcd/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/eventing-kogito not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/eventing-kogito/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/external-secrets-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/external-secrets-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/falcon-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/falcon-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/fence-agents-remediation not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/fence-agents-remediation/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/flink-kubernetes-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/flink-kubernetes-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/flux not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/flux/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/k8gb not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/k8gb/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/fossul-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/fossul-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/github-arc-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/github-arc-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/gitops-primer not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/gitops-primer/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/gitwebhook-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/gitwebhook-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/global-load-balancer-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/global-load-balancer-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/grafana-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/grafana-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/group-sync-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/group-sync-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/hawtio-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/hawtio-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/hazelcast-platform-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/hazelcast-platform-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/hedvig-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/hedvig-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/hive-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/hive-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/horreum-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/horreum-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/hyperfoil-bundle not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/hyperfoil-bundle/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/ibm-block-csi-operator-community not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/ibm-block-csi-operator-community/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/ibm-security-verify-access-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/ibm-security-verify-access-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/ibm-spectrum-scale-csi-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/ibm-spectrum-scale-csi-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/ibmcloud-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/ibmcloud-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/infinispan not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/infinispan/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/integrity-shield-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/integrity-shield-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/ipfs-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/ipfs-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/istio-workspace-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/istio-workspace-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/jaeger not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/jaeger/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/kaoto-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/kaoto-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/keda not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/keda/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/keepalived-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/keepalived-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/keycloak-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/keycloak-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/keycloak-permissions-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/keycloak-permissions-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/klusterlet not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/klusterlet/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/kogito-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/kogito-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/koku-metrics-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/koku-metrics-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/konveyor-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/konveyor-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/korrel8r not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/korrel8r/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/kuadrant-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/kuadrant-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/kube-green not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/kube-green/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/kubecost not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/kubecost/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/kubernetes-imagepuller-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/kubernetes-imagepuller-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/kubeturbo not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/kubeturbo/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/l5-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/l5-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/layer7-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/layer7-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/lbconfig-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/lbconfig-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/lib-bucket-provisioner not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/lib-bucket-provisioner/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/limitador-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/limitador-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/logging-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/logging-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/loki-helm-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/loki-helm-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/loki-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/loki-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/machine-deletion-remediation not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/machine-deletion-remediation/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/mariadb-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/mariadb-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/marin3r not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/marin3r/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/mercury-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/mercury-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/microcks not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/microcks/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/mongodb-atlas-kubernetes not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/mongodb-atlas-kubernetes/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/mongodb-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/mongodb-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/move2kube-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/move2kube-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/multi-nic-cni-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/multi-nic-cni-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/multicluster-global-hub-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/multicluster-global-hub-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/multicluster-operators-subscription not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/multicluster-operators-subscription/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/must-gather-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/must-gather-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/namespace-configuration-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/namespace-configuration-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/ncn-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/ncn-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/ndmspc-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/ndmspc-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/netobserv-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/netobserv-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/neuvector-community-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/neuvector-community-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/nexus-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/nexus-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/nexus-operator-m88i not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/nexus-operator-m88i/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/nfs-provisioner-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/nfs-provisioner-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/nlp-server not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/nlp-server/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/node-discovery-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/node-discovery-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/node-healthcheck-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/node-healthcheck-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/node-maintenance-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/node-maintenance-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/nsm-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/nsm-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/oadp-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/oadp-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/observability-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/observability-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/oci-ccm-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/oci-ccm-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/ocm-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/ocm-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/odoo-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/odoo-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/opendatahub-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/opendatahub-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/openebs not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/openebs/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/openshift-nfd-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/openshift-nfd-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/openshift-node-upgrade-mutex-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/openshift-node-upgrade-mutex-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/openshift-qiskit-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/openshift-qiskit-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/opentelemetry-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/opentelemetry-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/patch-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/patch-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/patterns-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/patterns-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/pcc-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/pcc-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/pelorus-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/pelorus-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/percona-xtradb-cluster-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/percona-xtradb-cluster-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/portworx-essentials not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/portworx-essentials/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/postgresql not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/postgresql/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/proactive-node-scaling-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/proactive-node-scaling-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/project-quay not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/project-quay/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/prometheus not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/prometheus/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/prometheus-exporter-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/prometheus-exporter-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/prometurbo not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/prometurbo/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/pubsubplus-eventbroker-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/pubsubplus-eventbroker-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/pulp-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/pulp-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/rabbitmq-cluster-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/rabbitmq-cluster-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/rabbitmq-messaging-topology-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/rabbitmq-messaging-topology-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/redis-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/redis-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/reportportal-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/reportportal-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/resource-locker-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/resource-locker-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/rhoas-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/rhoas-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/ripsaw not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/ripsaw/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/sailoperator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/sailoperator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/sap-commerce-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/sap-commerce-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/sap-data-intelligence-observer-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/sap-data-intelligence-observer-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/sap-hana-express-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/sap-hana-express-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/seldon-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/seldon-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/self-node-remediation not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/self-node-remediation/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/service-binding-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/service-binding-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/shipwright-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/shipwright-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/sigstore-helm-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/sigstore-helm-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/silicom-sts-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/silicom-sts-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/skupper-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/skupper-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/snapscheduler not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/snapscheduler/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/snyk-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/snyk-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/socmmd not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/socmmd/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/sonar-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/sonar-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/sosivio not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/sosivio/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/sonataflow-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/sonataflow-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/sosreport-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/sosreport-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/spark-helm-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/spark-helm-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/special-resource-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/special-resource-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/stolostron not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/stolostron/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/stolostron-engine not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/stolostron-engine/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/strimzi-kafka-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/strimzi-kafka-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/syndesis not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/syndesis/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/t8c not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/t8c/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/tagger not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/tagger/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/tempo-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/tempo-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/tf-controller not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/tf-controller/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/tidb-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/tidb-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/trident-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/trident-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/trustify-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/trustify-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/ucs-ci-solutions-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/ucs-ci-solutions-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/universal-crossplane not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/universal-crossplane/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/varnish-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/varnish-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/vault-config-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/vault-config-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/verticadb-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/verticadb-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/volume-expander-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/volume-expander-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/wandb-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/wandb-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/windup-operator not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/windup-operator/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/yaks not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/catalog-content/catalog/yaks/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/utilities not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes/kubernetes.io~empty-dir/utilities/copy-content not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/etc-hosts not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/containers/extract-utilities/c0fe7256 not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/containers/extract-utilities/c30319e4 not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/containers/extract-utilities/e6b1dd45 not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/containers/extract-content/2bb643f0 not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/containers/extract-content/920de426 not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/containers/extract-content/70fa1e87 not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/containers/registry-server/a1c12a2f not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/containers/registry-server/9442e6c7 not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/containers/registry-server/5b45ec72 not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/1d611f23-29be-4491-8495-bee1670e935f/volumes/kubernetes.io~empty-dir/catalog-content not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/1d611f23-29be-4491-8495-bee1670e935f/volumes/kubernetes.io~empty-dir/catalog-content/catalog not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/1d611f23-29be-4491-8495-bee1670e935f/volumes/kubernetes.io~empty-dir/catalog-content/catalog/abot-operator-rhmp not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/1d611f23-29be-4491-8495-bee1670e935f/volumes/kubernetes.io~empty-dir/catalog-content/catalog/abot-operator-rhmp/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/1d611f23-29be-4491-8495-bee1670e935f/volumes/kubernetes.io~empty-dir/catalog-content/catalog/aerospike-kubernetes-operator-rhmp not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/1d611f23-29be-4491-8495-bee1670e935f/volumes/kubernetes.io~empty-dir/catalog-content/catalog/aerospike-kubernetes-operator-rhmp/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/1d611f23-29be-4491-8495-bee1670e935f/volumes/kubernetes.io~empty-dir/catalog-content/catalog/aikit-operator-rhmp not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/1d611f23-29be-4491-8495-bee1670e935f/volumes/kubernetes.io~empty-dir/catalog-content/catalog/aikit-operator-rhmp/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/1d611f23-29be-4491-8495-bee1670e935f/volumes/kubernetes.io~empty-dir/catalog-content/catalog/anzo-operator-rhmp not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/1d611f23-29be-4491-8495-bee1670e935f/volumes/kubernetes.io~empty-dir/catalog-content/catalog/anzo-operator-rhmp/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/1d611f23-29be-4491-8495-bee1670e935f/volumes/kubernetes.io~empty-dir/catalog-content/catalog/anzograph-operator-rhmp not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/1d611f23-29be-4491-8495-bee1670e935f/volumes/kubernetes.io~empty-dir/catalog-content/catalog/anzograph-operator-rhmp/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/1d611f23-29be-4491-8495-bee1670e935f/volumes/kubernetes.io~empty-dir/catalog-content/catalog/anzounstructured-operator-rhmp not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/1d611f23-29be-4491-8495-bee1670e935f/volumes/kubernetes.io~empty-dir/catalog-content/catalog/anzounstructured-operator-rhmp/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/1d611f23-29be-4491-8495-bee1670e935f/volumes/kubernetes.io~empty-dir/catalog-content/catalog/cloudbees-ci-rhmp not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/1d611f23-29be-4491-8495-bee1670e935f/volumes/kubernetes.io~empty-dir/catalog-content/catalog/cloudbees-ci-rhmp/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/1d611f23-29be-4491-8495-bee1670e935f/volumes/kubernetes.io~empty-dir/catalog-content/catalog/cockroachdb-certified-rhmp not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/1d611f23-29be-4491-8495-bee1670e935f/volumes/kubernetes.io~empty-dir/catalog-content/catalog/cockroachdb-certified-rhmp/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/1d611f23-29be-4491-8495-bee1670e935f/volumes/kubernetes.io~empty-dir/catalog-content/catalog/crunchy-postgres-operator-rhmp not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/1d611f23-29be-4491-8495-bee1670e935f/volumes/kubernetes.io~empty-dir/catalog-content/catalog/crunchy-postgres-operator-rhmp/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/1d611f23-29be-4491-8495-bee1670e935f/volumes/kubernetes.io~empty-dir/catalog-content/catalog/datadog-operator-certified-rhmp not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/1d611f23-29be-4491-8495-bee1670e935f/volumes/kubernetes.io~empty-dir/catalog-content/catalog/datadog-operator-certified-rhmp/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/1d611f23-29be-4491-8495-bee1670e935f/volumes/kubernetes.io~empty-dir/catalog-content/catalog/dynatrace-operator-rhmp not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/1d611f23-29be-4491-8495-bee1670e935f/volumes/kubernetes.io~empty-dir/catalog-content/catalog/dynatrace-operator-rhmp/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/1d611f23-29be-4491-8495-bee1670e935f/volumes/kubernetes.io~empty-dir/catalog-content/catalog/entando-k8s-operator-rhmp not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/1d611f23-29be-4491-8495-bee1670e935f/volumes/kubernetes.io~empty-dir/catalog-content/catalog/entando-k8s-operator-rhmp/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/1d611f23-29be-4491-8495-bee1670e935f/volumes/kubernetes.io~empty-dir/catalog-content/catalog/flux not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/1d611f23-29be-4491-8495-bee1670e935f/volumes/kubernetes.io~empty-dir/catalog-content/catalog/flux/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/1d611f23-29be-4491-8495-bee1670e935f/volumes/kubernetes.io~empty-dir/catalog-content/catalog/instana-agent-operator-rhmp not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/1d611f23-29be-4491-8495-bee1670e935f/volumes/kubernetes.io~empty-dir/catalog-content/catalog/instana-agent-operator-rhmp/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/1d611f23-29be-4491-8495-bee1670e935f/volumes/kubernetes.io~empty-dir/catalog-content/catalog/iomesh-operator-rhmp not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/1d611f23-29be-4491-8495-bee1670e935f/volumes/kubernetes.io~empty-dir/catalog-content/catalog/iomesh-operator-rhmp/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/1d611f23-29be-4491-8495-bee1670e935f/volumes/kubernetes.io~empty-dir/catalog-content/catalog/joget-dx-operator-rhmp not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/1d611f23-29be-4491-8495-bee1670e935f/volumes/kubernetes.io~empty-dir/catalog-content/catalog/joget-dx-operator-rhmp/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/1d611f23-29be-4491-8495-bee1670e935f/volumes/kubernetes.io~empty-dir/catalog-content/catalog/joget-dx8-operator-rhmp not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/1d611f23-29be-4491-8495-bee1670e935f/volumes/kubernetes.io~empty-dir/catalog-content/catalog/joget-dx8-operator-rhmp/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/1d611f23-29be-4491-8495-bee1670e935f/volumes/kubernetes.io~empty-dir/catalog-content/catalog/k10-kasten-operator-paygo-rhmp not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/1d611f23-29be-4491-8495-bee1670e935f/volumes/kubernetes.io~empty-dir/catalog-content/catalog/k10-kasten-operator-paygo-rhmp/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/1d611f23-29be-4491-8495-bee1670e935f/volumes/kubernetes.io~empty-dir/catalog-content/catalog/k10-kasten-operator-rhmp not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/1d611f23-29be-4491-8495-bee1670e935f/volumes/kubernetes.io~empty-dir/catalog-content/catalog/k10-kasten-operator-rhmp/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/1d611f23-29be-4491-8495-bee1670e935f/volumes/kubernetes.io~empty-dir/catalog-content/catalog/k10-kasten-operator-term-rhmp not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/1d611f23-29be-4491-8495-bee1670e935f/volumes/kubernetes.io~empty-dir/catalog-content/catalog/k10-kasten-operator-term-rhmp/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/1d611f23-29be-4491-8495-bee1670e935f/volumes/kubernetes.io~empty-dir/catalog-content/catalog/kubemq-operator-marketplace-rhmp not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/1d611f23-29be-4491-8495-bee1670e935f/volumes/kubernetes.io~empty-dir/catalog-content/catalog/kubemq-operator-marketplace-rhmp/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/1d611f23-29be-4491-8495-bee1670e935f/volumes/kubernetes.io~empty-dir/catalog-content/catalog/kubeturbo-certified-rhmp not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/1d611f23-29be-4491-8495-bee1670e935f/volumes/kubernetes.io~empty-dir/catalog-content/catalog/kubeturbo-certified-rhmp/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/1d611f23-29be-4491-8495-bee1670e935f/volumes/kubernetes.io~empty-dir/catalog-content/catalog/linstor-operator-rhmp not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/1d611f23-29be-4491-8495-bee1670e935f/volumes/kubernetes.io~empty-dir/catalog-content/catalog/linstor-operator-rhmp/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/1d611f23-29be-4491-8495-bee1670e935f/volumes/kubernetes.io~empty-dir/catalog-content/catalog/marketplace-games-operator-rhmp not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/1d611f23-29be-4491-8495-bee1670e935f/volumes/kubernetes.io~empty-dir/catalog-content/catalog/marketplace-games-operator-rhmp/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/1d611f23-29be-4491-8495-bee1670e935f/volumes/kubernetes.io~empty-dir/catalog-content/catalog/model-builder-for-vision-certified-rhmp not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/1d611f23-29be-4491-8495-bee1670e935f/volumes/kubernetes.io~empty-dir/catalog-content/catalog/model-builder-for-vision-certified-rhmp/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/1d611f23-29be-4491-8495-bee1670e935f/volumes/kubernetes.io~empty-dir/catalog-content/catalog/neuvector-certified-operator-rhmp not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/1d611f23-29be-4491-8495-bee1670e935f/volumes/kubernetes.io~empty-dir/catalog-content/catalog/neuvector-certified-operator-rhmp/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/1d611f23-29be-4491-8495-bee1670e935f/volumes/kubernetes.io~empty-dir/catalog-content/catalog/ovms-operator-rhmp not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/1d611f23-29be-4491-8495-bee1670e935f/volumes/kubernetes.io~empty-dir/catalog-content/catalog/ovms-operator-rhmp/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/1d611f23-29be-4491-8495-bee1670e935f/volumes/kubernetes.io~empty-dir/catalog-content/catalog/pachyderm-operator-rhmp not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/1d611f23-29be-4491-8495-bee1670e935f/volumes/kubernetes.io~empty-dir/catalog-content/catalog/pachyderm-operator-rhmp/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/1d611f23-29be-4491-8495-bee1670e935f/volumes/kubernetes.io~empty-dir/catalog-content/catalog/redis-enterprise-operator-cert-rhmp not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/1d611f23-29be-4491-8495-bee1670e935f/volumes/kubernetes.io~empty-dir/catalog-content/catalog/redis-enterprise-operator-cert-rhmp/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/1d611f23-29be-4491-8495-bee1670e935f/volumes/kubernetes.io~empty-dir/catalog-content/catalog/seldon-deploy-operator-rhmp not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/1d611f23-29be-4491-8495-bee1670e935f/volumes/kubernetes.io~empty-dir/catalog-content/catalog/seldon-deploy-operator-rhmp/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/1d611f23-29be-4491-8495-bee1670e935f/volumes/kubernetes.io~empty-dir/catalog-content/catalog/starburst-enterprise-helm-operator-paygo-rhmp not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/1d611f23-29be-4491-8495-bee1670e935f/volumes/kubernetes.io~empty-dir/catalog-content/catalog/starburst-enterprise-helm-operator-paygo-rhmp/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/1d611f23-29be-4491-8495-bee1670e935f/volumes/kubernetes.io~empty-dir/catalog-content/catalog/starburst-enterprise-helm-operator-rhmp not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/1d611f23-29be-4491-8495-bee1670e935f/volumes/kubernetes.io~empty-dir/catalog-content/catalog/starburst-enterprise-helm-operator-rhmp/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/1d611f23-29be-4491-8495-bee1670e935f/volumes/kubernetes.io~empty-dir/catalog-content/catalog/t8c-certified-rhmp not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/1d611f23-29be-4491-8495-bee1670e935f/volumes/kubernetes.io~empty-dir/catalog-content/catalog/t8c-certified-rhmp/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/1d611f23-29be-4491-8495-bee1670e935f/volumes/kubernetes.io~empty-dir/catalog-content/catalog/timemachine-operator-rhmp not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/1d611f23-29be-4491-8495-bee1670e935f/volumes/kubernetes.io~empty-dir/catalog-content/catalog/timemachine-operator-rhmp/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/1d611f23-29be-4491-8495-bee1670e935f/volumes/kubernetes.io~empty-dir/catalog-content/catalog/vfunction-server-operator-rhmp not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/1d611f23-29be-4491-8495-bee1670e935f/volumes/kubernetes.io~empty-dir/catalog-content/catalog/vfunction-server-operator-rhmp/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/1d611f23-29be-4491-8495-bee1670e935f/volumes/kubernetes.io~empty-dir/catalog-content/catalog/xcrypt-operator-rhmp not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/1d611f23-29be-4491-8495-bee1670e935f/volumes/kubernetes.io~empty-dir/catalog-content/catalog/xcrypt-operator-rhmp/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/1d611f23-29be-4491-8495-bee1670e935f/volumes/kubernetes.io~empty-dir/catalog-content/catalog/yugabyte-platform-operator-bundle-rhmp not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/1d611f23-29be-4491-8495-bee1670e935f/volumes/kubernetes.io~empty-dir/catalog-content/catalog/yugabyte-platform-operator-bundle-rhmp/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/1d611f23-29be-4491-8495-bee1670e935f/volumes/kubernetes.io~empty-dir/catalog-content/catalog/zabbix-operator-certified-rhmp not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/1d611f23-29be-4491-8495-bee1670e935f/volumes/kubernetes.io~empty-dir/catalog-content/catalog/zabbix-operator-certified-rhmp/catalog.json not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/1d611f23-29be-4491-8495-bee1670e935f/volumes/kubernetes.io~empty-dir/catalog-content/cache not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/1d611f23-29be-4491-8495-bee1670e935f/volumes/kubernetes.io~empty-dir/catalog-content/cache/pogreb.v1 not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/1d611f23-29be-4491-8495-bee1670e935f/volumes/kubernetes.io~empty-dir/catalog-content/cache/pogreb.v1/db not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/1d611f23-29be-4491-8495-bee1670e935f/volumes/kubernetes.io~empty-dir/catalog-content/cache/pogreb.v1/db/00000-1.psg not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/1d611f23-29be-4491-8495-bee1670e935f/volumes/kubernetes.io~empty-dir/catalog-content/cache/pogreb.v1/db/00000-1.psg.pmt not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/1d611f23-29be-4491-8495-bee1670e935f/volumes/kubernetes.io~empty-dir/catalog-content/cache/pogreb.v1/db/db.pmt not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/1d611f23-29be-4491-8495-bee1670e935f/volumes/kubernetes.io~empty-dir/catalog-content/cache/pogreb.v1/db/index.pmt not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/1d611f23-29be-4491-8495-bee1670e935f/volumes/kubernetes.io~empty-dir/catalog-content/cache/pogreb.v1/db/main.pix not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/1d611f23-29be-4491-8495-bee1670e935f/volumes/kubernetes.io~empty-dir/catalog-content/cache/pogreb.v1/db/overflow.pix not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/1d611f23-29be-4491-8495-bee1670e935f/volumes/kubernetes.io~empty-dir/catalog-content/cache/pogreb.v1/digest not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/1d611f23-29be-4491-8495-bee1670e935f/volumes/kubernetes.io~empty-dir/utilities not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/1d611f23-29be-4491-8495-bee1670e935f/volumes/kubernetes.io~empty-dir/utilities/copy-content not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/1d611f23-29be-4491-8495-bee1670e935f/etc-hosts not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/1d611f23-29be-4491-8495-bee1670e935f/containers/extract-utilities/3c9f3a59 not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/1d611f23-29be-4491-8495-bee1670e935f/containers/extract-utilities/1091c11b not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/1d611f23-29be-4491-8495-bee1670e935f/containers/extract-utilities/9a6821c6 not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/1d611f23-29be-4491-8495-bee1670e935f/containers/extract-content/ec0c35e2 not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/1d611f23-29be-4491-8495-bee1670e935f/containers/extract-content/517f37e7 not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/1d611f23-29be-4491-8495-bee1670e935f/containers/extract-content/6214fe78 not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/1d611f23-29be-4491-8495-bee1670e935f/containers/registry-server/ba189c8b not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/1d611f23-29be-4491-8495-bee1670e935f/containers/registry-server/351e4f31 not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/1d611f23-29be-4491-8495-bee1670e935f/containers/registry-server/c0f219ff not reset as customized by admin to system_u:object_r:container_file_t:s0:c7,c13 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/3dcd261975c3d6b9a6ad6367fd4facd3/etc-hosts not reset as customized by admin to system_u:object_r:container_file_t:s0:c247,c522 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/3dcd261975c3d6b9a6ad6367fd4facd3/containers/wait-for-host-port/8069f607 not reset as customized by admin to system_u:object_r:container_file_t:s0:c378,c723 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/3dcd261975c3d6b9a6ad6367fd4facd3/containers/wait-for-host-port/559c3d82 not reset as customized by admin to system_u:object_r:container_file_t:s0:c133,c223 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/3dcd261975c3d6b9a6ad6367fd4facd3/containers/wait-for-host-port/605ad488 not reset as customized by admin to system_u:object_r:container_file_t:s0:c247,c522 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/3dcd261975c3d6b9a6ad6367fd4facd3/containers/kube-scheduler/148df488 not reset as customized by admin to system_u:object_r:container_file_t:s0:c378,c723 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/3dcd261975c3d6b9a6ad6367fd4facd3/containers/kube-scheduler/3bf6dcb4 not reset as customized by admin to system_u:object_r:container_file_t:s0:c133,c223 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/3dcd261975c3d6b9a6ad6367fd4facd3/containers/kube-scheduler/022a2feb not reset as customized by admin to system_u:object_r:container_file_t:s0:c247,c522 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/3dcd261975c3d6b9a6ad6367fd4facd3/containers/kube-scheduler-cert-syncer/938c3924 not reset as customized by admin to system_u:object_r:container_file_t:s0:c378,c723 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/3dcd261975c3d6b9a6ad6367fd4facd3/containers/kube-scheduler-cert-syncer/729fe23e not reset as customized by admin to system_u:object_r:container_file_t:s0:c133,c223 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/3dcd261975c3d6b9a6ad6367fd4facd3/containers/kube-scheduler-cert-syncer/1fd5cbd4 not reset as customized by admin to system_u:object_r:container_file_t:s0:c247,c522 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/3dcd261975c3d6b9a6ad6367fd4facd3/containers/kube-scheduler-recovery-controller/a96697e1 not reset as customized by admin to system_u:object_r:container_file_t:s0:c378,c723 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/3dcd261975c3d6b9a6ad6367fd4facd3/containers/kube-scheduler-recovery-controller/e155ddca not reset as customized by admin to system_u:object_r:container_file_t:s0:c133,c223 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/3dcd261975c3d6b9a6ad6367fd4facd3/containers/kube-scheduler-recovery-controller/10dd0e0f not reset as customized by admin to system_u:object_r:container_file_t:s0:c247,c522 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/49c341d1-5089-4bc2-86a0-a5e165cfcc6b/volumes/kubernetes.io~configmap/v4-0-config-system-trusted-ca-bundle not reset as customized by admin to system_u:object_r:container_file_t:s0:c682,c947 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/49c341d1-5089-4bc2-86a0-a5e165cfcc6b/volumes/kubernetes.io~configmap/v4-0-config-system-trusted-ca-bundle/..2025_02_24_06_09_35.3018472960 not reset as customized by admin to system_u:object_r:container_file_t:s0:c682,c947 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/49c341d1-5089-4bc2-86a0-a5e165cfcc6b/volumes/kubernetes.io~configmap/v4-0-config-system-trusted-ca-bundle/..2025_02_24_06_09_35.3018472960/ca-bundle.crt not reset as customized by admin to system_u:object_r:container_file_t:s0:c682,c947 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/49c341d1-5089-4bc2-86a0-a5e165cfcc6b/volumes/kubernetes.io~configmap/v4-0-config-system-trusted-ca-bundle/..data not reset as customized by admin to system_u:object_r:container_file_t:s0:c682,c947 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/49c341d1-5089-4bc2-86a0-a5e165cfcc6b/volumes/kubernetes.io~configmap/v4-0-config-system-trusted-ca-bundle/ca-bundle.crt not reset as customized by admin to system_u:object_r:container_file_t:s0:c682,c947 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/49c341d1-5089-4bc2-86a0-a5e165cfcc6b/volumes/kubernetes.io~configmap/audit-policies not reset as customized by admin to system_u:object_r:container_file_t:s0:c682,c947 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/49c341d1-5089-4bc2-86a0-a5e165cfcc6b/volumes/kubernetes.io~configmap/audit-policies/..2025_02_24_06_09_35.4262376737 not reset as customized by admin to system_u:object_r:container_file_t:s0:c682,c947 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/49c341d1-5089-4bc2-86a0-a5e165cfcc6b/volumes/kubernetes.io~configmap/audit-policies/..2025_02_24_06_09_35.4262376737/audit.yaml not reset as customized by admin to system_u:object_r:container_file_t:s0:c682,c947 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/49c341d1-5089-4bc2-86a0-a5e165cfcc6b/volumes/kubernetes.io~configmap/audit-policies/..data not reset as customized by admin to system_u:object_r:container_file_t:s0:c682,c947 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/49c341d1-5089-4bc2-86a0-a5e165cfcc6b/volumes/kubernetes.io~configmap/audit-policies/audit.yaml not reset as customized by admin to system_u:object_r:container_file_t:s0:c682,c947 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/49c341d1-5089-4bc2-86a0-a5e165cfcc6b/volumes/kubernetes.io~configmap/v4-0-config-system-cliconfig not reset as customized by admin to system_u:object_r:container_file_t:s0:c682,c947 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/49c341d1-5089-4bc2-86a0-a5e165cfcc6b/volumes/kubernetes.io~configmap/v4-0-config-system-cliconfig/..2025_02_24_06_09_35.2630275752 not reset as customized by admin to system_u:object_r:container_file_t:s0:c682,c947 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/49c341d1-5089-4bc2-86a0-a5e165cfcc6b/volumes/kubernetes.io~configmap/v4-0-config-system-cliconfig/..2025_02_24_06_09_35.2630275752/v4-0-config-system-cliconfig not reset as customized by admin to system_u:object_r:container_file_t:s0:c682,c947 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/49c341d1-5089-4bc2-86a0-a5e165cfcc6b/volumes/kubernetes.io~configmap/v4-0-config-system-cliconfig/..data not reset as customized by admin to system_u:object_r:container_file_t:s0:c682,c947 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/49c341d1-5089-4bc2-86a0-a5e165cfcc6b/volumes/kubernetes.io~configmap/v4-0-config-system-cliconfig/v4-0-config-system-cliconfig not reset as customized by admin to system_u:object_r:container_file_t:s0:c682,c947 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/49c341d1-5089-4bc2-86a0-a5e165cfcc6b/volumes/kubernetes.io~configmap/v4-0-config-system-service-ca not reset as customized by admin to system_u:object_r:container_file_t:s0:c682,c947 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/49c341d1-5089-4bc2-86a0-a5e165cfcc6b/volumes/kubernetes.io~configmap/v4-0-config-system-service-ca/..2025_02_24_06_09_35.2376963788 not reset as customized by admin to system_u:object_r:container_file_t:s0:c682,c947 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/49c341d1-5089-4bc2-86a0-a5e165cfcc6b/volumes/kubernetes.io~configmap/v4-0-config-system-service-ca/..2025_02_24_06_09_35.2376963788/service-ca.crt not reset as customized by admin to system_u:object_r:container_file_t:s0:c682,c947 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/49c341d1-5089-4bc2-86a0-a5e165cfcc6b/volumes/kubernetes.io~configmap/v4-0-config-system-service-ca/..data not reset as customized by admin to system_u:object_r:container_file_t:s0:c682,c947 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/49c341d1-5089-4bc2-86a0-a5e165cfcc6b/volumes/kubernetes.io~configmap/v4-0-config-system-service-ca/service-ca.crt not reset as customized by admin to system_u:object_r:container_file_t:s0:c682,c947 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/49c341d1-5089-4bc2-86a0-a5e165cfcc6b/etc-hosts not reset as customized by admin to system_u:object_r:container_file_t:s0:c682,c947 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/49c341d1-5089-4bc2-86a0-a5e165cfcc6b/containers/oauth-openshift/6f2c8392 not reset as customized by admin to system_u:object_r:container_file_t:s0:c267,c588 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/pods/49c341d1-5089-4bc2-86a0-a5e165cfcc6b/containers/oauth-openshift/bd241ad9 not reset as customized by admin to system_u:object_r:container_file_t:s0:c682,c947 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/plugins not reset as customized by admin to system_u:object_r:container_file_t:s0 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/plugins/csi-hostpath not reset as customized by admin to system_u:object_r:container_file_t:s0 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/plugins/csi-hostpath/csi.sock not reset as customized by admin to system_u:object_r:container_file_t:s0 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/plugins/kubernetes.io not reset as customized by admin to system_u:object_r:container_file_t:s0 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/plugins/kubernetes.io/csi not reset as customized by admin to system_u:object_r:container_file_t:s0 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/plugins/kubernetes.io/csi/kubevirt.io.hostpath-provisioner not reset as customized by admin to system_u:object_r:container_file_t:s0 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/plugins/kubernetes.io/csi/kubevirt.io.hostpath-provisioner/1f4776af88835e41c12b831b4c9fed40233456d14189815a54dbe7f892fc1983 not reset as customized by admin to system_u:object_r:container_file_t:s0 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/plugins/kubernetes.io/csi/kubevirt.io.hostpath-provisioner/1f4776af88835e41c12b831b4c9fed40233456d14189815a54dbe7f892fc1983/globalmount not reset as customized by admin to system_u:object_r:container_file_t:s0 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/plugins/kubernetes.io/csi/kubevirt.io.hostpath-provisioner/1f4776af88835e41c12b831b4c9fed40233456d14189815a54dbe7f892fc1983/vol_data.json not reset as customized by admin to system_u:object_r:container_file_t:s0 Dec 03 11:20:09 crc restorecon[4694]: /var/lib/kubelet/plugins_registry not reset as customized by admin to system_u:object_r:container_file_t:s0 Dec 03 11:20:09 crc restorecon[4694]: Relabeled /var/usrlocal/bin/kubenswrapper from system_u:object_r:bin_t:s0 to system_u:object_r:kubelet_exec_t:s0 Dec 03 11:20:09 crc kubenswrapper[5052]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Dec 03 11:20:09 crc kubenswrapper[5052]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Dec 03 11:20:09 crc kubenswrapper[5052]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Dec 03 11:20:09 crc kubenswrapper[5052]: Flag --register-with-taints has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Dec 03 11:20:09 crc kubenswrapper[5052]: Flag --pod-infra-container-image has been deprecated, will be removed in a future release. Image garbage collector will get sandbox image information from CRI. Dec 03 11:20:09 crc kubenswrapper[5052]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.521316 5052 server.go:211] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.525006 5052 feature_gate.go:330] unrecognized feature gate: ImageStreamImportMode Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.525031 5052 feature_gate.go:330] unrecognized feature gate: BuildCSIVolumes Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.525037 5052 feature_gate.go:330] unrecognized feature gate: CSIDriverSharedResource Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.525045 5052 feature_gate.go:330] unrecognized feature gate: NutanixMultiSubnets Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.525052 5052 feature_gate.go:330] unrecognized feature gate: ManagedBootImagesAWS Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.525058 5052 feature_gate.go:330] unrecognized feature gate: MultiArchInstallAWS Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.525064 5052 feature_gate.go:330] unrecognized feature gate: SetEIPForNLBIngressController Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.525069 5052 feature_gate.go:330] unrecognized feature gate: IngressControllerDynamicConfigurationManager Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.525074 5052 feature_gate.go:330] unrecognized feature gate: VSphereControlPlaneMachineSet Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.525085 5052 feature_gate.go:330] unrecognized feature gate: NewOLM Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.525089 5052 feature_gate.go:330] unrecognized feature gate: VSphereStaticIPs Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.525093 5052 feature_gate.go:330] unrecognized feature gate: MachineAPIProviderOpenStack Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.525098 5052 feature_gate.go:330] unrecognized feature gate: DNSNameResolver Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.525102 5052 feature_gate.go:330] unrecognized feature gate: AWSEFSDriverVolumeMetrics Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.525107 5052 feature_gate.go:330] unrecognized feature gate: MinimumKubeletVersion Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.525111 5052 feature_gate.go:330] unrecognized feature gate: SigstoreImageVerification Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.525116 5052 feature_gate.go:330] unrecognized feature gate: InsightsOnDemandDataGather Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.525122 5052 feature_gate.go:330] unrecognized feature gate: AdditionalRoutingCapabilities Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.525127 5052 feature_gate.go:330] unrecognized feature gate: NetworkDiagnosticsConfig Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.525134 5052 feature_gate.go:353] Setting GA feature gate CloudDualStackNodeIPs=true. It will be removed in a future release. Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.525140 5052 feature_gate.go:330] unrecognized feature gate: AWSClusterHostedDNS Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.525146 5052 feature_gate.go:330] unrecognized feature gate: AutomatedEtcdBackup Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.525151 5052 feature_gate.go:330] unrecognized feature gate: MultiArchInstallAzure Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.525156 5052 feature_gate.go:330] unrecognized feature gate: MultiArchInstallGCP Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.525161 5052 feature_gate.go:330] unrecognized feature gate: BootcNodeManagement Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.525165 5052 feature_gate.go:330] unrecognized feature gate: AlibabaPlatform Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.525170 5052 feature_gate.go:330] unrecognized feature gate: PinnedImages Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.525174 5052 feature_gate.go:330] unrecognized feature gate: GCPLabelsTags Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.525177 5052 feature_gate.go:330] unrecognized feature gate: MachineConfigNodes Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.525184 5052 feature_gate.go:353] Setting GA feature gate DisableKubeletCloudCredentialProviders=true. It will be removed in a future release. Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.525189 5052 feature_gate.go:330] unrecognized feature gate: InsightsConfigAPI Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.525194 5052 feature_gate.go:330] unrecognized feature gate: ChunkSizeMiB Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.525199 5052 feature_gate.go:330] unrecognized feature gate: MachineAPIMigration Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.525203 5052 feature_gate.go:330] unrecognized feature gate: ClusterAPIInstall Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.525207 5052 feature_gate.go:330] unrecognized feature gate: EtcdBackendQuota Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.525212 5052 feature_gate.go:330] unrecognized feature gate: NetworkSegmentation Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.525216 5052 feature_gate.go:330] unrecognized feature gate: Example Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.525221 5052 feature_gate.go:330] unrecognized feature gate: MetricsCollectionProfiles Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.525225 5052 feature_gate.go:330] unrecognized feature gate: RouteAdvertisements Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.525229 5052 feature_gate.go:330] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.525234 5052 feature_gate.go:330] unrecognized feature gate: GatewayAPI Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.525239 5052 feature_gate.go:330] unrecognized feature gate: VSphereDriverConfiguration Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.525242 5052 feature_gate.go:330] unrecognized feature gate: ManagedBootImages Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.525246 5052 feature_gate.go:330] unrecognized feature gate: MixedCPUsAllocation Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.525251 5052 feature_gate.go:330] unrecognized feature gate: UpgradeStatus Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.525256 5052 feature_gate.go:330] unrecognized feature gate: AzureWorkloadIdentity Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.525260 5052 feature_gate.go:330] unrecognized feature gate: HardwareSpeed Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.525264 5052 feature_gate.go:330] unrecognized feature gate: OnClusterBuild Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.525268 5052 feature_gate.go:330] unrecognized feature gate: InsightsConfig Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.525273 5052 feature_gate.go:330] unrecognized feature gate: NetworkLiveMigration Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.525278 5052 feature_gate.go:330] unrecognized feature gate: InsightsRuntimeExtractor Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.525282 5052 feature_gate.go:330] unrecognized feature gate: ConsolePluginContentSecurityPolicy Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.525288 5052 feature_gate.go:330] unrecognized feature gate: PlatformOperators Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.525295 5052 feature_gate.go:353] Setting GA feature gate ValidatingAdmissionPolicy=true. It will be removed in a future release. Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.525301 5052 feature_gate.go:330] unrecognized feature gate: ExternalOIDC Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.525307 5052 feature_gate.go:330] unrecognized feature gate: BareMetalLoadBalancer Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.525312 5052 feature_gate.go:330] unrecognized feature gate: SignatureStores Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.525317 5052 feature_gate.go:330] unrecognized feature gate: ClusterMonitoringConfig Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.525322 5052 feature_gate.go:330] unrecognized feature gate: GCPClusterHostedDNS Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.525326 5052 feature_gate.go:330] unrecognized feature gate: NodeDisruptionPolicy Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.525330 5052 feature_gate.go:330] unrecognized feature gate: PersistentIPsForVirtualization Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.525334 5052 feature_gate.go:330] unrecognized feature gate: PrivateHostedZoneAWS Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.525339 5052 feature_gate.go:330] unrecognized feature gate: VSphereMultiNetworks Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.525343 5052 feature_gate.go:330] unrecognized feature gate: IngressControllerLBSubnetsAWS Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.525347 5052 feature_gate.go:330] unrecognized feature gate: AdminNetworkPolicy Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.525351 5052 feature_gate.go:330] unrecognized feature gate: OVNObservability Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.525355 5052 feature_gate.go:330] unrecognized feature gate: VSphereMultiVCenters Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.525360 5052 feature_gate.go:330] unrecognized feature gate: OpenShiftPodSecurityAdmission Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.525365 5052 feature_gate.go:351] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.525371 5052 feature_gate.go:330] unrecognized feature gate: VolumeGroupSnapshot Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.525374 5052 feature_gate.go:330] unrecognized feature gate: ClusterAPIInstallIBMCloud Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.525476 5052 flags.go:64] FLAG: --address="0.0.0.0" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.525485 5052 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.525494 5052 flags.go:64] FLAG: --anonymous-auth="true" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.525501 5052 flags.go:64] FLAG: --application-metrics-count-limit="100" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.525525 5052 flags.go:64] FLAG: --authentication-token-webhook="false" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.525533 5052 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.525542 5052 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.525549 5052 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.525555 5052 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.525560 5052 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.525568 5052 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.525574 5052 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.525581 5052 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.525587 5052 flags.go:64] FLAG: --cgroup-root="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.525592 5052 flags.go:64] FLAG: --cgroups-per-qos="true" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.525597 5052 flags.go:64] FLAG: --client-ca-file="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.525601 5052 flags.go:64] FLAG: --cloud-config="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.525606 5052 flags.go:64] FLAG: --cloud-provider="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.525611 5052 flags.go:64] FLAG: --cluster-dns="[]" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.525619 5052 flags.go:64] FLAG: --cluster-domain="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.525624 5052 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.525629 5052 flags.go:64] FLAG: --config-dir="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.525634 5052 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.525639 5052 flags.go:64] FLAG: --container-log-max-files="5" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.525647 5052 flags.go:64] FLAG: --container-log-max-size="10Mi" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.525651 5052 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.525656 5052 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.525661 5052 flags.go:64] FLAG: --containerd-namespace="k8s.io" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.525666 5052 flags.go:64] FLAG: --contention-profiling="false" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.525670 5052 flags.go:64] FLAG: --cpu-cfs-quota="true" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.525675 5052 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.525680 5052 flags.go:64] FLAG: --cpu-manager-policy="none" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.525684 5052 flags.go:64] FLAG: --cpu-manager-policy-options="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.525691 5052 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.525696 5052 flags.go:64] FLAG: --enable-controller-attach-detach="true" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.525700 5052 flags.go:64] FLAG: --enable-debugging-handlers="true" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.525705 5052 flags.go:64] FLAG: --enable-load-reader="false" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.525711 5052 flags.go:64] FLAG: --enable-server="true" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.525715 5052 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.525721 5052 flags.go:64] FLAG: --event-burst="100" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.525727 5052 flags.go:64] FLAG: --event-qps="50" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.525732 5052 flags.go:64] FLAG: --event-storage-age-limit="default=0" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.525738 5052 flags.go:64] FLAG: --event-storage-event-limit="default=0" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.525742 5052 flags.go:64] FLAG: --eviction-hard="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.525749 5052 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.525754 5052 flags.go:64] FLAG: --eviction-minimum-reclaim="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.525759 5052 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.525764 5052 flags.go:64] FLAG: --eviction-soft="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.525768 5052 flags.go:64] FLAG: --eviction-soft-grace-period="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.525773 5052 flags.go:64] FLAG: --exit-on-lock-contention="false" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.525777 5052 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.525781 5052 flags.go:64] FLAG: --experimental-mounter-path="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.525787 5052 flags.go:64] FLAG: --fail-cgroupv1="false" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.525791 5052 flags.go:64] FLAG: --fail-swap-on="true" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.525796 5052 flags.go:64] FLAG: --feature-gates="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.525801 5052 flags.go:64] FLAG: --file-check-frequency="20s" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.525806 5052 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.525810 5052 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.525816 5052 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.525821 5052 flags.go:64] FLAG: --healthz-port="10248" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.525826 5052 flags.go:64] FLAG: --help="false" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.525831 5052 flags.go:64] FLAG: --hostname-override="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.525836 5052 flags.go:64] FLAG: --housekeeping-interval="10s" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.525840 5052 flags.go:64] FLAG: --http-check-frequency="20s" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.525845 5052 flags.go:64] FLAG: --image-credential-provider-bin-dir="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.525850 5052 flags.go:64] FLAG: --image-credential-provider-config="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.525855 5052 flags.go:64] FLAG: --image-gc-high-threshold="85" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.525859 5052 flags.go:64] FLAG: --image-gc-low-threshold="80" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.525863 5052 flags.go:64] FLAG: --image-service-endpoint="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.525867 5052 flags.go:64] FLAG: --kernel-memcg-notification="false" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.525872 5052 flags.go:64] FLAG: --kube-api-burst="100" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.525876 5052 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.525881 5052 flags.go:64] FLAG: --kube-api-qps="50" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.525885 5052 flags.go:64] FLAG: --kube-reserved="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.525890 5052 flags.go:64] FLAG: --kube-reserved-cgroup="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.525895 5052 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.525900 5052 flags.go:64] FLAG: --kubelet-cgroups="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.525904 5052 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.525909 5052 flags.go:64] FLAG: --lock-file="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.525913 5052 flags.go:64] FLAG: --log-cadvisor-usage="false" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.525917 5052 flags.go:64] FLAG: --log-flush-frequency="5s" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.525921 5052 flags.go:64] FLAG: --log-json-info-buffer-size="0" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.525928 5052 flags.go:64] FLAG: --log-json-split-stream="false" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.525932 5052 flags.go:64] FLAG: --log-text-info-buffer-size="0" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.525936 5052 flags.go:64] FLAG: --log-text-split-stream="false" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.525941 5052 flags.go:64] FLAG: --logging-format="text" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.525945 5052 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.525950 5052 flags.go:64] FLAG: --make-iptables-util-chains="true" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.525953 5052 flags.go:64] FLAG: --manifest-url="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.525958 5052 flags.go:64] FLAG: --manifest-url-header="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.525964 5052 flags.go:64] FLAG: --max-housekeeping-interval="15s" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.525968 5052 flags.go:64] FLAG: --max-open-files="1000000" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.525974 5052 flags.go:64] FLAG: --max-pods="110" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.525979 5052 flags.go:64] FLAG: --maximum-dead-containers="-1" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.525984 5052 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.525988 5052 flags.go:64] FLAG: --memory-manager-policy="None" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.525992 5052 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.525996 5052 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.526001 5052 flags.go:64] FLAG: --node-ip="192.168.126.11" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.526005 5052 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/control-plane=,node-role.kubernetes.io/master=,node.openshift.io/os_id=rhcos" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.526017 5052 flags.go:64] FLAG: --node-status-max-images="50" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.526022 5052 flags.go:64] FLAG: --node-status-update-frequency="10s" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.526026 5052 flags.go:64] FLAG: --oom-score-adj="-999" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.526030 5052 flags.go:64] FLAG: --pod-cidr="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.526037 5052 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:33549946e22a9ffa738fd94b1345f90921bc8f92fa6137784cb33c77ad806f9d" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.526045 5052 flags.go:64] FLAG: --pod-manifest-path="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.526049 5052 flags.go:64] FLAG: --pod-max-pids="-1" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.526054 5052 flags.go:64] FLAG: --pods-per-core="0" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.526058 5052 flags.go:64] FLAG: --port="10250" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.526064 5052 flags.go:64] FLAG: --protect-kernel-defaults="false" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.526069 5052 flags.go:64] FLAG: --provider-id="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.526073 5052 flags.go:64] FLAG: --qos-reserved="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.526079 5052 flags.go:64] FLAG: --read-only-port="10255" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.526084 5052 flags.go:64] FLAG: --register-node="true" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.526088 5052 flags.go:64] FLAG: --register-schedulable="true" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.526092 5052 flags.go:64] FLAG: --register-with-taints="node-role.kubernetes.io/master=:NoSchedule" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.526101 5052 flags.go:64] FLAG: --registry-burst="10" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.526105 5052 flags.go:64] FLAG: --registry-qps="5" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.526109 5052 flags.go:64] FLAG: --reserved-cpus="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.526113 5052 flags.go:64] FLAG: --reserved-memory="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.526119 5052 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.526124 5052 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.526129 5052 flags.go:64] FLAG: --rotate-certificates="false" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.526134 5052 flags.go:64] FLAG: --rotate-server-certificates="false" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.526139 5052 flags.go:64] FLAG: --runonce="false" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.526144 5052 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.526149 5052 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.526155 5052 flags.go:64] FLAG: --seccomp-default="false" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.526160 5052 flags.go:64] FLAG: --serialize-image-pulls="true" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.526164 5052 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.526169 5052 flags.go:64] FLAG: --storage-driver-db="cadvisor" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.526173 5052 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.526178 5052 flags.go:64] FLAG: --storage-driver-password="root" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.526183 5052 flags.go:64] FLAG: --storage-driver-secure="false" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.526188 5052 flags.go:64] FLAG: --storage-driver-table="stats" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.526192 5052 flags.go:64] FLAG: --storage-driver-user="root" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.526197 5052 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.526202 5052 flags.go:64] FLAG: --sync-frequency="1m0s" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.526206 5052 flags.go:64] FLAG: --system-cgroups="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.526211 5052 flags.go:64] FLAG: --system-reserved="cpu=200m,ephemeral-storage=350Mi,memory=350Mi" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.526218 5052 flags.go:64] FLAG: --system-reserved-cgroup="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.526224 5052 flags.go:64] FLAG: --tls-cert-file="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.526229 5052 flags.go:64] FLAG: --tls-cipher-suites="[]" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.526235 5052 flags.go:64] FLAG: --tls-min-version="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.526240 5052 flags.go:64] FLAG: --tls-private-key-file="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.526244 5052 flags.go:64] FLAG: --topology-manager-policy="none" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.526248 5052 flags.go:64] FLAG: --topology-manager-policy-options="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.526253 5052 flags.go:64] FLAG: --topology-manager-scope="container" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.526258 5052 flags.go:64] FLAG: --v="2" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.526264 5052 flags.go:64] FLAG: --version="false" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.526270 5052 flags.go:64] FLAG: --vmodule="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.526276 5052 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.526280 5052 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.526436 5052 feature_gate.go:330] unrecognized feature gate: PinnedImages Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.526443 5052 feature_gate.go:330] unrecognized feature gate: AWSClusterHostedDNS Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.526447 5052 feature_gate.go:330] unrecognized feature gate: AlibabaPlatform Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.526452 5052 feature_gate.go:330] unrecognized feature gate: HardwareSpeed Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.526456 5052 feature_gate.go:330] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.526460 5052 feature_gate.go:330] unrecognized feature gate: PersistentIPsForVirtualization Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.526464 5052 feature_gate.go:330] unrecognized feature gate: AWSEFSDriverVolumeMetrics Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.526467 5052 feature_gate.go:330] unrecognized feature gate: ClusterAPIInstall Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.526472 5052 feature_gate.go:353] Setting GA feature gate ValidatingAdmissionPolicy=true. It will be removed in a future release. Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.526477 5052 feature_gate.go:330] unrecognized feature gate: PrivateHostedZoneAWS Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.526481 5052 feature_gate.go:330] unrecognized feature gate: IngressControllerLBSubnetsAWS Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.526485 5052 feature_gate.go:330] unrecognized feature gate: IngressControllerDynamicConfigurationManager Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.526490 5052 feature_gate.go:353] Setting GA feature gate DisableKubeletCloudCredentialProviders=true. It will be removed in a future release. Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.526494 5052 feature_gate.go:330] unrecognized feature gate: BootcNodeManagement Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.526499 5052 feature_gate.go:351] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.526525 5052 feature_gate.go:330] unrecognized feature gate: MinimumKubeletVersion Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.526530 5052 feature_gate.go:330] unrecognized feature gate: GatewayAPI Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.526535 5052 feature_gate.go:330] unrecognized feature gate: InsightsOnDemandDataGather Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.526538 5052 feature_gate.go:330] unrecognized feature gate: EtcdBackendQuota Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.526544 5052 feature_gate.go:330] unrecognized feature gate: BareMetalLoadBalancer Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.526550 5052 feature_gate.go:353] Setting GA feature gate CloudDualStackNodeIPs=true. It will be removed in a future release. Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.526554 5052 feature_gate.go:330] unrecognized feature gate: RouteAdvertisements Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.526559 5052 feature_gate.go:330] unrecognized feature gate: InsightsRuntimeExtractor Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.526563 5052 feature_gate.go:330] unrecognized feature gate: NewOLM Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.526567 5052 feature_gate.go:330] unrecognized feature gate: AutomatedEtcdBackup Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.526571 5052 feature_gate.go:330] unrecognized feature gate: ChunkSizeMiB Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.526575 5052 feature_gate.go:330] unrecognized feature gate: OnClusterBuild Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.526579 5052 feature_gate.go:330] unrecognized feature gate: VSphereDriverConfiguration Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.526584 5052 feature_gate.go:330] unrecognized feature gate: MachineConfigNodes Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.526592 5052 feature_gate.go:330] unrecognized feature gate: SigstoreImageVerification Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.526600 5052 feature_gate.go:330] unrecognized feature gate: VolumeGroupSnapshot Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.526607 5052 feature_gate.go:330] unrecognized feature gate: MultiArchInstallGCP Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.526612 5052 feature_gate.go:330] unrecognized feature gate: VSphereStaticIPs Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.526616 5052 feature_gate.go:330] unrecognized feature gate: ImageStreamImportMode Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.526621 5052 feature_gate.go:330] unrecognized feature gate: VSphereMultiVCenters Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.526626 5052 feature_gate.go:330] unrecognized feature gate: UpgradeStatus Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.526631 5052 feature_gate.go:330] unrecognized feature gate: OpenShiftPodSecurityAdmission Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.526635 5052 feature_gate.go:330] unrecognized feature gate: GCPLabelsTags Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.526641 5052 feature_gate.go:330] unrecognized feature gate: ManagedBootImages Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.526645 5052 feature_gate.go:330] unrecognized feature gate: ExternalOIDC Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.526649 5052 feature_gate.go:330] unrecognized feature gate: AzureWorkloadIdentity Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.526653 5052 feature_gate.go:330] unrecognized feature gate: MultiArchInstallAzure Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.526658 5052 feature_gate.go:330] unrecognized feature gate: ConsolePluginContentSecurityPolicy Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.526662 5052 feature_gate.go:330] unrecognized feature gate: ClusterAPIInstallIBMCloud Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.526667 5052 feature_gate.go:330] unrecognized feature gate: NutanixMultiSubnets Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.526672 5052 feature_gate.go:330] unrecognized feature gate: SignatureStores Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.526676 5052 feature_gate.go:330] unrecognized feature gate: SetEIPForNLBIngressController Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.526681 5052 feature_gate.go:330] unrecognized feature gate: MetricsCollectionProfiles Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.526685 5052 feature_gate.go:330] unrecognized feature gate: PlatformOperators Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.526689 5052 feature_gate.go:330] unrecognized feature gate: MachineAPIMigration Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.526694 5052 feature_gate.go:330] unrecognized feature gate: ManagedBootImagesAWS Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.526698 5052 feature_gate.go:330] unrecognized feature gate: BuildCSIVolumes Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.526703 5052 feature_gate.go:330] unrecognized feature gate: AdditionalRoutingCapabilities Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.526707 5052 feature_gate.go:330] unrecognized feature gate: MachineAPIProviderOpenStack Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.526711 5052 feature_gate.go:330] unrecognized feature gate: NodeDisruptionPolicy Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.526715 5052 feature_gate.go:330] unrecognized feature gate: MultiArchInstallAWS Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.526719 5052 feature_gate.go:330] unrecognized feature gate: Example Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.526723 5052 feature_gate.go:330] unrecognized feature gate: NetworkDiagnosticsConfig Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.526727 5052 feature_gate.go:330] unrecognized feature gate: GCPClusterHostedDNS Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.526731 5052 feature_gate.go:330] unrecognized feature gate: DNSNameResolver Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.526735 5052 feature_gate.go:330] unrecognized feature gate: NetworkSegmentation Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.526739 5052 feature_gate.go:330] unrecognized feature gate: InsightsConfig Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.526743 5052 feature_gate.go:330] unrecognized feature gate: VSphereMultiNetworks Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.526747 5052 feature_gate.go:330] unrecognized feature gate: InsightsConfigAPI Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.526751 5052 feature_gate.go:330] unrecognized feature gate: ClusterMonitoringConfig Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.526760 5052 feature_gate.go:330] unrecognized feature gate: OVNObservability Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.526764 5052 feature_gate.go:330] unrecognized feature gate: CSIDriverSharedResource Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.526771 5052 feature_gate.go:330] unrecognized feature gate: AdminNetworkPolicy Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.526776 5052 feature_gate.go:330] unrecognized feature gate: NetworkLiveMigration Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.526781 5052 feature_gate.go:330] unrecognized feature gate: MixedCPUsAllocation Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.526784 5052 feature_gate.go:330] unrecognized feature gate: VSphereControlPlaneMachineSet Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.526792 5052 feature_gate.go:386] feature gates: {map[CloudDualStackNodeIPs:true DisableKubeletCloudCredentialProviders:true DynamicResourceAllocation:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false NodeSwap:false ProcMountType:false RouteExternalCertificate:false ServiceAccountTokenNodeBinding:false TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:false UserNamespacesSupport:false ValidatingAdmissionPolicy:true VolumeAttributesClass:false]} Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.536636 5052 server.go:491] "Kubelet version" kubeletVersion="v1.31.5" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.536679 5052 server.go:493] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.536769 5052 feature_gate.go:330] unrecognized feature gate: MixedCPUsAllocation Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.536781 5052 feature_gate.go:330] unrecognized feature gate: HardwareSpeed Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.536786 5052 feature_gate.go:330] unrecognized feature gate: VSphereMultiVCenters Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.536790 5052 feature_gate.go:330] unrecognized feature gate: InsightsConfigAPI Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.536795 5052 feature_gate.go:330] unrecognized feature gate: AzureWorkloadIdentity Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.536798 5052 feature_gate.go:330] unrecognized feature gate: ImageStreamImportMode Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.536802 5052 feature_gate.go:330] unrecognized feature gate: VSphereControlPlaneMachineSet Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.536806 5052 feature_gate.go:330] unrecognized feature gate: MultiArchInstallGCP Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.536812 5052 feature_gate.go:353] Setting GA feature gate DisableKubeletCloudCredentialProviders=true. It will be removed in a future release. Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.536819 5052 feature_gate.go:330] unrecognized feature gate: VSphereStaticIPs Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.536824 5052 feature_gate.go:330] unrecognized feature gate: OpenShiftPodSecurityAdmission Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.536829 5052 feature_gate.go:351] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.536834 5052 feature_gate.go:330] unrecognized feature gate: ChunkSizeMiB Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.536839 5052 feature_gate.go:353] Setting GA feature gate ValidatingAdmissionPolicy=true. It will be removed in a future release. Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.536844 5052 feature_gate.go:330] unrecognized feature gate: GCPClusterHostedDNS Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.536849 5052 feature_gate.go:330] unrecognized feature gate: PrivateHostedZoneAWS Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.536852 5052 feature_gate.go:330] unrecognized feature gate: NetworkSegmentation Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.536857 5052 feature_gate.go:330] unrecognized feature gate: ClusterAPIInstall Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.536860 5052 feature_gate.go:330] unrecognized feature gate: VSphereDriverConfiguration Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.536863 5052 feature_gate.go:330] unrecognized feature gate: OnClusterBuild Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.536868 5052 feature_gate.go:330] unrecognized feature gate: MachineAPIProviderOpenStack Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.536872 5052 feature_gate.go:330] unrecognized feature gate: DNSNameResolver Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.536875 5052 feature_gate.go:330] unrecognized feature gate: AWSClusterHostedDNS Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.536879 5052 feature_gate.go:330] unrecognized feature gate: EtcdBackendQuota Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.536883 5052 feature_gate.go:330] unrecognized feature gate: SetEIPForNLBIngressController Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.536887 5052 feature_gate.go:330] unrecognized feature gate: AutomatedEtcdBackup Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.536891 5052 feature_gate.go:330] unrecognized feature gate: MetricsCollectionProfiles Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.536895 5052 feature_gate.go:330] unrecognized feature gate: BareMetalLoadBalancer Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.536899 5052 feature_gate.go:330] unrecognized feature gate: IngressControllerLBSubnetsAWS Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.536903 5052 feature_gate.go:330] unrecognized feature gate: ClusterAPIInstallIBMCloud Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.536907 5052 feature_gate.go:330] unrecognized feature gate: MultiArchInstallAWS Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.536911 5052 feature_gate.go:330] unrecognized feature gate: NewOLM Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.536916 5052 feature_gate.go:330] unrecognized feature gate: Example Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.536920 5052 feature_gate.go:330] unrecognized feature gate: VSphereMultiNetworks Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.536925 5052 feature_gate.go:330] unrecognized feature gate: AWSEFSDriverVolumeMetrics Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.536929 5052 feature_gate.go:330] unrecognized feature gate: MultiArchInstallAzure Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.536932 5052 feature_gate.go:330] unrecognized feature gate: AlibabaPlatform Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.536936 5052 feature_gate.go:330] unrecognized feature gate: GatewayAPI Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.536940 5052 feature_gate.go:330] unrecognized feature gate: InsightsConfig Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.536943 5052 feature_gate.go:330] unrecognized feature gate: NodeDisruptionPolicy Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.536946 5052 feature_gate.go:330] unrecognized feature gate: MachineAPIMigration Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.536950 5052 feature_gate.go:330] unrecognized feature gate: RouteAdvertisements Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.536953 5052 feature_gate.go:330] unrecognized feature gate: ManagedBootImages Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.536957 5052 feature_gate.go:330] unrecognized feature gate: AdditionalRoutingCapabilities Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.536961 5052 feature_gate.go:330] unrecognized feature gate: GCPLabelsTags Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.536964 5052 feature_gate.go:330] unrecognized feature gate: VolumeGroupSnapshot Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.536968 5052 feature_gate.go:330] unrecognized feature gate: ManagedBootImagesAWS Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.536972 5052 feature_gate.go:330] unrecognized feature gate: NetworkDiagnosticsConfig Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.536976 5052 feature_gate.go:353] Setting GA feature gate CloudDualStackNodeIPs=true. It will be removed in a future release. Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.536981 5052 feature_gate.go:330] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.536985 5052 feature_gate.go:330] unrecognized feature gate: OVNObservability Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.536988 5052 feature_gate.go:330] unrecognized feature gate: SignatureStores Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.536992 5052 feature_gate.go:330] unrecognized feature gate: CSIDriverSharedResource Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.536996 5052 feature_gate.go:330] unrecognized feature gate: SigstoreImageVerification Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.537000 5052 feature_gate.go:330] unrecognized feature gate: UpgradeStatus Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.537003 5052 feature_gate.go:330] unrecognized feature gate: NetworkLiveMigration Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.537007 5052 feature_gate.go:330] unrecognized feature gate: IngressControllerDynamicConfigurationManager Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.537011 5052 feature_gate.go:330] unrecognized feature gate: PinnedImages Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.537015 5052 feature_gate.go:330] unrecognized feature gate: MachineConfigNodes Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.537019 5052 feature_gate.go:330] unrecognized feature gate: ExternalOIDC Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.537027 5052 feature_gate.go:330] unrecognized feature gate: AdminNetworkPolicy Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.537034 5052 feature_gate.go:330] unrecognized feature gate: PersistentIPsForVirtualization Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.537039 5052 feature_gate.go:330] unrecognized feature gate: ClusterMonitoringConfig Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.537046 5052 feature_gate.go:330] unrecognized feature gate: BuildCSIVolumes Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.537057 5052 feature_gate.go:330] unrecognized feature gate: InsightsOnDemandDataGather Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.537063 5052 feature_gate.go:330] unrecognized feature gate: InsightsRuntimeExtractor Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.537068 5052 feature_gate.go:330] unrecognized feature gate: PlatformOperators Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.537072 5052 feature_gate.go:330] unrecognized feature gate: MinimumKubeletVersion Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.537076 5052 feature_gate.go:330] unrecognized feature gate: BootcNodeManagement Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.537080 5052 feature_gate.go:330] unrecognized feature gate: ConsolePluginContentSecurityPolicy Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.537087 5052 feature_gate.go:330] unrecognized feature gate: NutanixMultiSubnets Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.537096 5052 feature_gate.go:386] feature gates: {map[CloudDualStackNodeIPs:true DisableKubeletCloudCredentialProviders:true DynamicResourceAllocation:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false NodeSwap:false ProcMountType:false RouteExternalCertificate:false ServiceAccountTokenNodeBinding:false TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:false UserNamespacesSupport:false ValidatingAdmissionPolicy:true VolumeAttributesClass:false]} Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.537221 5052 feature_gate.go:330] unrecognized feature gate: Example Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.537229 5052 feature_gate.go:330] unrecognized feature gate: ConsolePluginContentSecurityPolicy Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.537234 5052 feature_gate.go:330] unrecognized feature gate: MachineAPIMigration Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.537238 5052 feature_gate.go:330] unrecognized feature gate: GCPClusterHostedDNS Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.537241 5052 feature_gate.go:330] unrecognized feature gate: InsightsOnDemandDataGather Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.537245 5052 feature_gate.go:330] unrecognized feature gate: MachineConfigNodes Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.537248 5052 feature_gate.go:330] unrecognized feature gate: AWSClusterHostedDNS Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.537252 5052 feature_gate.go:330] unrecognized feature gate: NetworkSegmentation Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.537257 5052 feature_gate.go:353] Setting GA feature gate ValidatingAdmissionPolicy=true. It will be removed in a future release. Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.537266 5052 feature_gate.go:330] unrecognized feature gate: MultiArchInstallAzure Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.537271 5052 feature_gate.go:330] unrecognized feature gate: EtcdBackendQuota Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.537276 5052 feature_gate.go:330] unrecognized feature gate: MinimumKubeletVersion Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.537280 5052 feature_gate.go:330] unrecognized feature gate: VSphereDriverConfiguration Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.537285 5052 feature_gate.go:330] unrecognized feature gate: HardwareSpeed Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.537289 5052 feature_gate.go:330] unrecognized feature gate: NetworkLiveMigration Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.537295 5052 feature_gate.go:330] unrecognized feature gate: MultiArchInstallAWS Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.537299 5052 feature_gate.go:330] unrecognized feature gate: InsightsRuntimeExtractor Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.537307 5052 feature_gate.go:330] unrecognized feature gate: OVNObservability Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.537312 5052 feature_gate.go:330] unrecognized feature gate: OnClusterBuild Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.537316 5052 feature_gate.go:330] unrecognized feature gate: VSphereStaticIPs Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.537321 5052 feature_gate.go:330] unrecognized feature gate: PinnedImages Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.537326 5052 feature_gate.go:330] unrecognized feature gate: BootcNodeManagement Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.537330 5052 feature_gate.go:330] unrecognized feature gate: NetworkDiagnosticsConfig Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.537337 5052 feature_gate.go:353] Setting GA feature gate CloudDualStackNodeIPs=true. It will be removed in a future release. Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.537343 5052 feature_gate.go:330] unrecognized feature gate: VSphereMultiNetworks Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.537347 5052 feature_gate.go:330] unrecognized feature gate: MachineAPIProviderOpenStack Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.537353 5052 feature_gate.go:330] unrecognized feature gate: GatewayAPI Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.537357 5052 feature_gate.go:330] unrecognized feature gate: BareMetalLoadBalancer Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.537362 5052 feature_gate.go:330] unrecognized feature gate: AdditionalRoutingCapabilities Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.537368 5052 feature_gate.go:351] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.537374 5052 feature_gate.go:330] unrecognized feature gate: ImageStreamImportMode Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.537379 5052 feature_gate.go:353] Setting GA feature gate DisableKubeletCloudCredentialProviders=true. It will be removed in a future release. Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.537384 5052 feature_gate.go:330] unrecognized feature gate: MixedCPUsAllocation Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.537388 5052 feature_gate.go:330] unrecognized feature gate: SigstoreImageVerification Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.537395 5052 feature_gate.go:330] unrecognized feature gate: UpgradeStatus Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.537404 5052 feature_gate.go:330] unrecognized feature gate: CSIDriverSharedResource Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.537411 5052 feature_gate.go:330] unrecognized feature gate: ManagedBootImages Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.537416 5052 feature_gate.go:330] unrecognized feature gate: VSphereMultiVCenters Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.537421 5052 feature_gate.go:330] unrecognized feature gate: IngressControllerLBSubnetsAWS Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.537426 5052 feature_gate.go:330] unrecognized feature gate: SetEIPForNLBIngressController Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.537430 5052 feature_gate.go:330] unrecognized feature gate: PersistentIPsForVirtualization Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.537435 5052 feature_gate.go:330] unrecognized feature gate: ExternalOIDC Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.537439 5052 feature_gate.go:330] unrecognized feature gate: NewOLM Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.537444 5052 feature_gate.go:330] unrecognized feature gate: SignatureStores Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.537448 5052 feature_gate.go:330] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.537451 5052 feature_gate.go:330] unrecognized feature gate: PrivateHostedZoneAWS Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.537456 5052 feature_gate.go:330] unrecognized feature gate: RouteAdvertisements Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.537459 5052 feature_gate.go:330] unrecognized feature gate: BuildCSIVolumes Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.537463 5052 feature_gate.go:330] unrecognized feature gate: AutomatedEtcdBackup Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.537467 5052 feature_gate.go:330] unrecognized feature gate: MetricsCollectionProfiles Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.537471 5052 feature_gate.go:330] unrecognized feature gate: NodeDisruptionPolicy Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.537474 5052 feature_gate.go:330] unrecognized feature gate: AWSEFSDriverVolumeMetrics Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.537478 5052 feature_gate.go:330] unrecognized feature gate: AlibabaPlatform Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.537481 5052 feature_gate.go:330] unrecognized feature gate: InsightsConfig Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.537485 5052 feature_gate.go:330] unrecognized feature gate: VolumeGroupSnapshot Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.537489 5052 feature_gate.go:330] unrecognized feature gate: IngressControllerDynamicConfigurationManager Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.537493 5052 feature_gate.go:330] unrecognized feature gate: PlatformOperators Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.537497 5052 feature_gate.go:330] unrecognized feature gate: OpenShiftPodSecurityAdmission Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.537501 5052 feature_gate.go:330] unrecognized feature gate: InsightsConfigAPI Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.537526 5052 feature_gate.go:330] unrecognized feature gate: DNSNameResolver Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.537535 5052 feature_gate.go:330] unrecognized feature gate: GCPLabelsTags Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.537540 5052 feature_gate.go:330] unrecognized feature gate: MultiArchInstallGCP Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.537544 5052 feature_gate.go:330] unrecognized feature gate: ChunkSizeMiB Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.537548 5052 feature_gate.go:330] unrecognized feature gate: ClusterAPIInstallIBMCloud Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.537551 5052 feature_gate.go:330] unrecognized feature gate: ManagedBootImagesAWS Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.537555 5052 feature_gate.go:330] unrecognized feature gate: ClusterAPIInstall Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.537559 5052 feature_gate.go:330] unrecognized feature gate: ClusterMonitoringConfig Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.537562 5052 feature_gate.go:330] unrecognized feature gate: NutanixMultiSubnets Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.537567 5052 feature_gate.go:330] unrecognized feature gate: AzureWorkloadIdentity Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.537571 5052 feature_gate.go:330] unrecognized feature gate: VSphereControlPlaneMachineSet Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.537576 5052 feature_gate.go:330] unrecognized feature gate: AdminNetworkPolicy Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.537586 5052 feature_gate.go:386] feature gates: {map[CloudDualStackNodeIPs:true DisableKubeletCloudCredentialProviders:true DynamicResourceAllocation:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false NodeSwap:false ProcMountType:false RouteExternalCertificate:false ServiceAccountTokenNodeBinding:false TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:false UserNamespacesSupport:false ValidatingAdmissionPolicy:true VolumeAttributesClass:false]} Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.537830 5052 server.go:940] "Client rotation is on, will bootstrap in background" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.540793 5052 bootstrap.go:85] "Current kubeconfig file contents are still valid, no bootstrap necessary" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.540897 5052 certificate_store.go:130] Loading cert/key pair from "/var/lib/kubelet/pki/kubelet-client-current.pem". Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.541469 5052 server.go:997] "Starting client certificate rotation" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.541494 5052 certificate_manager.go:356] kubernetes.io/kube-apiserver-client-kubelet: Certificate rotation is enabled Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.542274 5052 certificate_manager.go:356] kubernetes.io/kube-apiserver-client-kubelet: Certificate expiration is 2026-02-24 05:52:08 +0000 UTC, rotation deadline is 2026-01-08 13:50:48.687118405 +0000 UTC Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.542471 5052 certificate_manager.go:356] kubernetes.io/kube-apiserver-client-kubelet: Waiting 866h30m39.144655693s for next certificate rotation Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.547616 5052 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.549572 5052 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.560615 5052 log.go:25] "Validated CRI v1 runtime API" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.575021 5052 log.go:25] "Validated CRI v1 image API" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.577107 5052 server.go:1437] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.579934 5052 fs.go:133] Filesystem UUIDs: map[0b076daa-c26a-46d2-b3a6-72a8dbc6e257:/dev/vda4 2025-12-03-11-16-07-00:/dev/sr0 7B77-95E7:/dev/vda2 de0497b0-db1b-465a-b278-03db02455c71:/dev/vda3] Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.580173 5052 fs.go:134] Filesystem partitions: map[/dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /dev/vda3:{mountpoint:/boot major:252 minor:3 fsType:ext4 blockSize:0} /dev/vda4:{mountpoint:/var major:252 minor:4 fsType:xfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /run/user/1000:{mountpoint:/run/user/1000 major:0 minor:42 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:30 fsType:tmpfs blockSize:0} /var/lib/etcd:{mountpoint:/var/lib/etcd major:0 minor:43 fsType:tmpfs blockSize:0}] Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.593936 5052 manager.go:217] Machine: {Timestamp:2025-12-03 11:20:09.592319591 +0000 UTC m=+0.180561375 CPUVendorID:AuthenticAMD NumCores:12 NumPhysicalCores:1 NumSockets:12 CpuFrequency:2800000 MemoryCapacity:33654128640 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:21801e6708c44f15b81395eb736a7cec SystemUUID:ad4477df-a9a6-40d9-b422-36e1612ca334 BootID:d2f3768d-f4ec-4a3e-a78b-995e22cb0cfd Filesystems:[{Device:/run/user/1000 DeviceMajor:0 DeviceMinor:42 Capacity:3365412864 Type:vfs Inodes:821634 HasInodes:true} {Device:/var/lib/etcd DeviceMajor:0 DeviceMinor:43 Capacity:1073741824 Type:vfs Inodes:4108170 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16827064320 Type:vfs Inodes:4108170 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6730825728 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/vda4 DeviceMajor:252 DeviceMinor:4 Capacity:85292941312 Type:vfs Inodes:41679680 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:30 Capacity:16827064320 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/vda3 DeviceMajor:252 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[252:0:{Name:vda Major:252 Minor:0 Size:214748364800 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:fa:16:3e:a8:bd:2a Speed:0 Mtu:1500} {Name:br-int MacAddress:d6:39:55:2e:22:71 Speed:0 Mtu:1400} {Name:ens3 MacAddress:fa:16:3e:a8:bd:2a Speed:-1 Mtu:1500} {Name:ens7 MacAddress:fa:16:3e:6b:93:89 Speed:-1 Mtu:1500} {Name:ens7.20 MacAddress:52:54:00:84:c8:52 Speed:-1 Mtu:1496} {Name:ens7.21 MacAddress:52:54:00:ec:7d:40 Speed:-1 Mtu:1496} {Name:ens7.22 MacAddress:52:54:00:c7:a3:df Speed:-1 Mtu:1496} {Name:eth10 MacAddress:fe:16:7a:ff:60:9d Speed:0 Mtu:1500} {Name:ovn-k8s-mp0 MacAddress:0a:58:0a:d9:00:02 Speed:0 Mtu:1400} {Name:ovs-system MacAddress:66:fe:44:08:04:fd Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33654128640 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:524288 Type:Unified Level:2}] UncoreCaches:[{Id:0 Size:16777216 Type:Unified Level:3}] SocketID:0 BookID: DrawerID:} {Id:0 Threads:[1] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:524288 Type:Unified Level:2}] UncoreCaches:[{Id:1 Size:16777216 Type:Unified Level:3}] SocketID:1 BookID: DrawerID:} {Id:0 Threads:[10] Caches:[{Id:10 Size:32768 Type:Data Level:1} {Id:10 Size:32768 Type:Instruction Level:1} {Id:10 Size:524288 Type:Unified Level:2}] UncoreCaches:[{Id:10 Size:16777216 Type:Unified Level:3}] SocketID:10 BookID: DrawerID:} {Id:0 Threads:[11] Caches:[{Id:11 Size:32768 Type:Data Level:1} {Id:11 Size:32768 Type:Instruction Level:1} {Id:11 Size:524288 Type:Unified Level:2}] UncoreCaches:[{Id:11 Size:16777216 Type:Unified Level:3}] SocketID:11 BookID: DrawerID:} {Id:0 Threads:[2] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:524288 Type:Unified Level:2}] UncoreCaches:[{Id:2 Size:16777216 Type:Unified Level:3}] SocketID:2 BookID: DrawerID:} {Id:0 Threads:[3] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:524288 Type:Unified Level:2}] UncoreCaches:[{Id:3 Size:16777216 Type:Unified Level:3}] SocketID:3 BookID: DrawerID:} {Id:0 Threads:[4] Caches:[{Id:4 Size:32768 Type:Data Level:1} {Id:4 Size:32768 Type:Instruction Level:1} {Id:4 Size:524288 Type:Unified Level:2}] UncoreCaches:[{Id:4 Size:16777216 Type:Unified Level:3}] SocketID:4 BookID: DrawerID:} {Id:0 Threads:[5] Caches:[{Id:5 Size:32768 Type:Data Level:1} {Id:5 Size:32768 Type:Instruction Level:1} {Id:5 Size:524288 Type:Unified Level:2}] UncoreCaches:[{Id:5 Size:16777216 Type:Unified Level:3}] SocketID:5 BookID: DrawerID:} {Id:0 Threads:[6] Caches:[{Id:6 Size:32768 Type:Data Level:1} {Id:6 Size:32768 Type:Instruction Level:1} {Id:6 Size:524288 Type:Unified Level:2}] UncoreCaches:[{Id:6 Size:16777216 Type:Unified Level:3}] SocketID:6 BookID: DrawerID:} {Id:0 Threads:[7] Caches:[{Id:7 Size:32768 Type:Data Level:1} {Id:7 Size:32768 Type:Instruction Level:1} {Id:7 Size:524288 Type:Unified Level:2}] UncoreCaches:[{Id:7 Size:16777216 Type:Unified Level:3}] SocketID:7 BookID: DrawerID:} {Id:0 Threads:[8] Caches:[{Id:8 Size:32768 Type:Data Level:1} {Id:8 Size:32768 Type:Instruction Level:1} {Id:8 Size:524288 Type:Unified Level:2}] UncoreCaches:[{Id:8 Size:16777216 Type:Unified Level:3}] SocketID:8 BookID: DrawerID:} {Id:0 Threads:[9] Caches:[{Id:9 Size:32768 Type:Data Level:1} {Id:9 Size:32768 Type:Instruction Level:1} {Id:9 Size:524288 Type:Unified Level:2}] UncoreCaches:[{Id:9 Size:16777216 Type:Unified Level:3}] SocketID:9 BookID: DrawerID:}] Caches:[] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.594500 5052 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.594893 5052 manager.go:233] Version: {KernelVersion:5.14.0-427.50.2.el9_4.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 418.94.202502100215-0 DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.595552 5052 swap_util.go:113] "Swap is on" /proc/swaps contents="Filename\t\t\t\tType\t\tSize\t\tUsed\t\tPriority" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.595845 5052 container_manager_linux.go:267] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.595904 5052 container_manager_linux.go:272] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"crc","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"200m","ephemeral-storage":"350Mi","memory":"350Mi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"ExperimentalMemoryManagerPolicy":"None","ExperimentalMemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.596158 5052 topology_manager.go:138] "Creating topology manager with none policy" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.596174 5052 container_manager_linux.go:303] "Creating device plugin manager" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.596377 5052 manager.go:142] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.596566 5052 server.go:66] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.596957 5052 state_mem.go:36] "Initialized new in-memory state store" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.597073 5052 server.go:1245] "Using root directory" path="/var/lib/kubelet" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.597800 5052 kubelet.go:418] "Attempting to sync node with API server" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.597823 5052 kubelet.go:313] "Adding static pod path" path="/etc/kubernetes/manifests" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.597872 5052 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.597891 5052 kubelet.go:324] "Adding apiserver pod source" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.597905 5052 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.600101 5052 kuberuntime_manager.go:262] "Container runtime initialized" containerRuntime="cri-o" version="1.31.5-4.rhaos4.18.gitdad78d5.el9" apiVersion="v1" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.600534 5052 certificate_store.go:130] Loading cert/key pair from "/var/lib/kubelet/pki/kubelet-server-current.pem". Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.601053 5052 reflector.go:561] k8s.io/client-go/informers/factory.go:160: failed to list *v1.Service: Get "https://api-int.crc.testing:6443/api/v1/services?fieldSelector=spec.clusterIP%21%3DNone&limit=500&resourceVersion=0": dial tcp 38.129.56.252:6443: connect: connection refused Dec 03 11:20:09 crc kubenswrapper[5052]: E1203 11:20:09.601252 5052 reflector.go:158] "Unhandled Error" err="k8s.io/client-go/informers/factory.go:160: Failed to watch *v1.Service: failed to list *v1.Service: Get \"https://api-int.crc.testing:6443/api/v1/services?fieldSelector=spec.clusterIP%21%3DNone&limit=500&resourceVersion=0\": dial tcp 38.129.56.252:6443: connect: connection refused" logger="UnhandledError" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.601382 5052 kubelet.go:854] "Not starting ClusterTrustBundle informer because we are in static kubelet mode" Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.601692 5052 reflector.go:561] k8s.io/client-go/informers/factory.go:160: failed to list *v1.Node: Get "https://api-int.crc.testing:6443/api/v1/nodes?fieldSelector=metadata.name%3Dcrc&limit=500&resourceVersion=0": dial tcp 38.129.56.252:6443: connect: connection refused Dec 03 11:20:09 crc kubenswrapper[5052]: E1203 11:20:09.601809 5052 reflector.go:158] "Unhandled Error" err="k8s.io/client-go/informers/factory.go:160: Failed to watch *v1.Node: failed to list *v1.Node: Get \"https://api-int.crc.testing:6443/api/v1/nodes?fieldSelector=metadata.name%3Dcrc&limit=500&resourceVersion=0\": dial tcp 38.129.56.252:6443: connect: connection refused" logger="UnhandledError" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.601926 5052 plugins.go:603] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.601951 5052 plugins.go:603] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.601959 5052 plugins.go:603] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.601968 5052 plugins.go:603] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.601991 5052 plugins.go:603] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.602000 5052 plugins.go:603] "Loaded volume plugin" pluginName="kubernetes.io/secret" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.602007 5052 plugins.go:603] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.602018 5052 plugins.go:603] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.602027 5052 plugins.go:603] "Loaded volume plugin" pluginName="kubernetes.io/fc" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.602034 5052 plugins.go:603] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.602064 5052 plugins.go:603] "Loaded volume plugin" pluginName="kubernetes.io/projected" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.602072 5052 plugins.go:603] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.602605 5052 plugins.go:603] "Loaded volume plugin" pluginName="kubernetes.io/csi" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.603106 5052 server.go:1280] "Started kubelet" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.603302 5052 csi_plugin.go:884] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.crc.testing:6443/apis/storage.k8s.io/v1/csinodes/crc?resourceVersion=0": dial tcp 38.129.56.252:6443: connect: connection refused Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.603499 5052 server.go:163] "Starting to listen" address="0.0.0.0" port=10250 Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.603877 5052 ratelimit.go:55] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Dec 03 11:20:09 crc systemd[1]: Started Kubernetes Kubelet. Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.605414 5052 server.go:236] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.606988 5052 server.go:460] "Adding debug handlers to kubelet server" Dec 03 11:20:09 crc kubenswrapper[5052]: E1203 11:20:09.607859 5052 event.go:368] "Unable to write event (may retry after sleeping)" err="Post \"https://api-int.crc.testing:6443/api/v1/namespaces/default/events\": dial tcp 38.129.56.252:6443: connect: connection refused" event="&Event{ObjectMeta:{crc.187db09b31ec0397 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:crc,UID:crc,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:crc,},FirstTimestamp:2025-12-03 11:20:09.603072919 +0000 UTC m=+0.191314673,LastTimestamp:2025-12-03 11:20:09.603072919 +0000 UTC m=+0.191314673,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:crc,}" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.614764 5052 certificate_manager.go:356] kubernetes.io/kubelet-serving: Certificate rotation is enabled Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.614836 5052 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.615367 5052 volume_manager.go:287] "The desired_state_of_world populator starts" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.615394 5052 volume_manager.go:289] "Starting Kubelet Volume Manager" Dec 03 11:20:09 crc kubenswrapper[5052]: E1203 11:20:09.615570 5052 kubelet_node_status.go:503] "Error getting the current node from lister" err="node \"crc\" not found" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.615907 5052 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.615111 5052 certificate_manager.go:356] kubernetes.io/kubelet-serving: Certificate expiration is 2026-02-24 05:53:03 +0000 UTC, rotation deadline is 2026-01-07 18:53:53.25756795 +0000 UTC Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.615938 5052 certificate_manager.go:356] kubernetes.io/kubelet-serving: Waiting 847h33m43.641633527s for next certificate rotation Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.618554 5052 factory.go:55] Registering systemd factory Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.618584 5052 factory.go:221] Registration of the systemd container factory successfully Dec 03 11:20:09 crc kubenswrapper[5052]: E1203 11:20:09.619559 5052 controller.go:145] "Failed to ensure lease exists, will retry" err="Get \"https://api-int.crc.testing:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/crc?timeout=10s\": dial tcp 38.129.56.252:6443: connect: connection refused" interval="200ms" Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.618458 5052 reflector.go:561] k8s.io/client-go/informers/factory.go:160: failed to list *v1.CSIDriver: Get "https://api-int.crc.testing:6443/apis/storage.k8s.io/v1/csidrivers?limit=500&resourceVersion=0": dial tcp 38.129.56.252:6443: connect: connection refused Dec 03 11:20:09 crc kubenswrapper[5052]: E1203 11:20:09.620488 5052 reflector.go:158] "Unhandled Error" err="k8s.io/client-go/informers/factory.go:160: Failed to watch *v1.CSIDriver: failed to list *v1.CSIDriver: Get \"https://api-int.crc.testing:6443/apis/storage.k8s.io/v1/csidrivers?limit=500&resourceVersion=0\": dial tcp 38.129.56.252:6443: connect: connection refused" logger="UnhandledError" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.620647 5052 factory.go:153] Registering CRI-O factory Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.620703 5052 factory.go:221] Registration of the crio container factory successfully Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.620803 5052 factory.go:219] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.620834 5052 factory.go:103] Registering Raw factory Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.620852 5052 manager.go:1196] Started watching for new ooms in manager Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.621761 5052 manager.go:319] Starting recovery of all containers Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.627427 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="3b6479f0-333b-4a96-9adf-2099afdc2447" volumeName="kubernetes.io/projected/3b6479f0-333b-4a96-9adf-2099afdc2447-kube-api-access-cqllr" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.627878 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="49c341d1-5089-4bc2-86a0-a5e165cfcc6b" volumeName="kubernetes.io/configmap/49c341d1-5089-4bc2-86a0-a5e165cfcc6b-audit-policies" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.627975 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="87cf06ed-a83f-41a7-828d-70653580a8cb" volumeName="kubernetes.io/configmap/87cf06ed-a83f-41a7-828d-70653580a8cb-config-volume" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.628083 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="96b93a3a-6083-4aea-8eab-fe1aa8245ad9" volumeName="kubernetes.io/projected/96b93a3a-6083-4aea-8eab-fe1aa8245ad9-kube-api-access-nzwt7" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.628170 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="01ab3dd5-8196-46d0-ad33-122e2ca51def" volumeName="kubernetes.io/configmap/01ab3dd5-8196-46d0-ad33-122e2ca51def-config" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.628263 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="1bf7eb37-55a3-4c65-b768-a94c82151e69" volumeName="kubernetes.io/secret/1bf7eb37-55a3-4c65-b768-a94c82151e69-encryption-config" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.628347 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="210d8245-ebfc-4e3b-ac4a-e21ce76f9a7c" volumeName="kubernetes.io/configmap/210d8245-ebfc-4e3b-ac4a-e21ce76f9a7c-config" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.628431 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="22c825df-677d-4ca6-82db-3454ed06e783" volumeName="kubernetes.io/configmap/22c825df-677d-4ca6-82db-3454ed06e783-auth-proxy-config" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.628562 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="ef543e1b-8068-4ea3-b32a-61027b32e95d" volumeName="kubernetes.io/projected/ef543e1b-8068-4ea3-b32a-61027b32e95d-kube-api-access-s2kz5" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.628654 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="a31745f5-9847-4afe-82a5-3161cc66ca93" volumeName="kubernetes.io/projected/a31745f5-9847-4afe-82a5-3161cc66ca93-bound-sa-token" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.628727 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="b6312bbd-5731-4ea0-a20f-81d5a57df44a" volumeName="kubernetes.io/projected/b6312bbd-5731-4ea0-a20f-81d5a57df44a-kube-api-access-249nr" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.628808 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="b6cd30de-2eeb-49a2-ab40-9167f4560ff5" volumeName="kubernetes.io/secret/b6cd30de-2eeb-49a2-ab40-9167f4560ff5-marketplace-operator-metrics" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.628906 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="bc5039c0-ea34-426b-a2b7-fbbc87b49a6d" volumeName="kubernetes.io/empty-dir/bc5039c0-ea34-426b-a2b7-fbbc87b49a6d-available-featuregates" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.629001 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="49c341d1-5089-4bc2-86a0-a5e165cfcc6b" volumeName="kubernetes.io/secret/49c341d1-5089-4bc2-86a0-a5e165cfcc6b-v4-0-config-user-template-error" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.629095 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="6402fda4-df10-493c-b4e5-d0569419652d" volumeName="kubernetes.io/configmap/6402fda4-df10-493c-b4e5-d0569419652d-images" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.629225 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="8f668bae-612b-4b75-9490-919e737c6a3b" volumeName="kubernetes.io/secret/8f668bae-612b-4b75-9490-919e737c6a3b-installation-pull-secrets" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.629312 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="a0128f3a-b052-44ed-a84e-c4c8aaf17c13" volumeName="kubernetes.io/projected/a0128f3a-b052-44ed-a84e-c4c8aaf17c13-kube-api-access-gf66m" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.629407 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="09efc573-dbb6-4249-bd59-9b87aba8dd28" volumeName="kubernetes.io/projected/09efc573-dbb6-4249-bd59-9b87aba8dd28-kube-api-access-8tdtz" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.629574 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="1386a44e-36a2-460c-96d0-0359d2b6f0f5" volumeName="kubernetes.io/projected/1386a44e-36a2-460c-96d0-0359d2b6f0f5-kube-api-access" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.629679 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="1d611f23-29be-4491-8495-bee1670e935f" volumeName="kubernetes.io/empty-dir/1d611f23-29be-4491-8495-bee1670e935f-utilities" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.629775 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="37a5e44f-9a88-4405-be8a-b645485e7312" volumeName="kubernetes.io/secret/37a5e44f-9a88-4405-be8a-b645485e7312-metrics-tls" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.629904 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="ef543e1b-8068-4ea3-b32a-61027b32e95d" volumeName="kubernetes.io/configmap/ef543e1b-8068-4ea3-b32a-61027b32e95d-ovnkube-identity-cm" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.630006 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="1386a44e-36a2-460c-96d0-0359d2b6f0f5" volumeName="kubernetes.io/configmap/1386a44e-36a2-460c-96d0-0359d2b6f0f5-config" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.630088 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="1bf7eb37-55a3-4c65-b768-a94c82151e69" volumeName="kubernetes.io/configmap/1bf7eb37-55a3-4c65-b768-a94c82151e69-image-import-ca" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.630167 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="49c341d1-5089-4bc2-86a0-a5e165cfcc6b" volumeName="kubernetes.io/projected/49c341d1-5089-4bc2-86a0-a5e165cfcc6b-kube-api-access-ngvvp" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.630241 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="57a731c4-ef35-47a8-b875-bfb08a7f8011" volumeName="kubernetes.io/empty-dir/57a731c4-ef35-47a8-b875-bfb08a7f8011-catalog-content" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.630322 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="8cea82b4-6893-4ddc-af9f-1bb5ae425c5b" volumeName="kubernetes.io/configmap/8cea82b4-6893-4ddc-af9f-1bb5ae425c5b-config" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.630430 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="bf126b07-da06-4140-9a57-dfd54fc6b486" volumeName="kubernetes.io/projected/bf126b07-da06-4140-9a57-dfd54fc6b486-kube-api-access-rnphk" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.630542 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="c03ee662-fb2f-4fc4-a2c1-af487c19d254" volumeName="kubernetes.io/secret/c03ee662-fb2f-4fc4-a2c1-af487c19d254-stats-auth" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.630641 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="09efc573-dbb6-4249-bd59-9b87aba8dd28" volumeName="kubernetes.io/secret/09efc573-dbb6-4249-bd59-9b87aba8dd28-serving-cert" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.630710 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="1bf7eb37-55a3-4c65-b768-a94c82151e69" volumeName="kubernetes.io/configmap/1bf7eb37-55a3-4c65-b768-a94c82151e69-etcd-serving-ca" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.630780 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="49c341d1-5089-4bc2-86a0-a5e165cfcc6b" volumeName="kubernetes.io/secret/49c341d1-5089-4bc2-86a0-a5e165cfcc6b-v4-0-config-user-template-login" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.630851 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="7583ce53-e0fe-4a16-9e4d-50516596a136" volumeName="kubernetes.io/configmap/7583ce53-e0fe-4a16-9e4d-50516596a136-client-ca" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.630923 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="cd70aa09-68dd-4d64-bd6f-156fe6d1dc6d" volumeName="kubernetes.io/projected/cd70aa09-68dd-4d64-bd6f-156fe6d1dc6d-kube-api-access-x2m85" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.630996 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="0b78653f-4ff9-4508-8672-245ed9b561e3" volumeName="kubernetes.io/projected/0b78653f-4ff9-4508-8672-245ed9b561e3-kube-api-access" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.631062 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="210d8245-ebfc-4e3b-ac4a-e21ce76f9a7c" volumeName="kubernetes.io/secret/210d8245-ebfc-4e3b-ac4a-e21ce76f9a7c-serving-cert" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.631129 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="5225d0e4-402f-4861-b410-819f433b1803" volumeName="kubernetes.io/projected/5225d0e4-402f-4861-b410-819f433b1803-kube-api-access-9xfj7" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.631201 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="bd23aa5c-e532-4e53-bccf-e79f130c5ae8" volumeName="kubernetes.io/projected/bd23aa5c-e532-4e53-bccf-e79f130c5ae8-kube-api-access-jhbk2" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.631271 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="96b93a3a-6083-4aea-8eab-fe1aa8245ad9" volumeName="kubernetes.io/secret/96b93a3a-6083-4aea-8eab-fe1aa8245ad9-metrics-tls" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.631342 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="b11524ee-3fca-4b1b-9cdf-6da289fdbc7d" volumeName="kubernetes.io/empty-dir/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d-utilities" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.631420 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="bc5039c0-ea34-426b-a2b7-fbbc87b49a6d" volumeName="kubernetes.io/secret/bc5039c0-ea34-426b-a2b7-fbbc87b49a6d-serving-cert" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.631494 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="fda69060-fa79-4696-b1a6-7980f124bf7c" volumeName="kubernetes.io/secret/fda69060-fa79-4696-b1a6-7980f124bf7c-proxy-tls" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.631597 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="09efc573-dbb6-4249-bd59-9b87aba8dd28" volumeName="kubernetes.io/configmap/09efc573-dbb6-4249-bd59-9b87aba8dd28-etcd-ca" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.631679 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="7bb08738-c794-4ee8-9972-3a62ca171029" volumeName="kubernetes.io/configmap/7bb08738-c794-4ee8-9972-3a62ca171029-cni-sysctl-allowlist" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.631760 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="87cf06ed-a83f-41a7-828d-70653580a8cb" volumeName="kubernetes.io/projected/87cf06ed-a83f-41a7-828d-70653580a8cb-kube-api-access-d6qdx" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.631953 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="8f668bae-612b-4b75-9490-919e737c6a3b" volumeName="kubernetes.io/empty-dir/8f668bae-612b-4b75-9490-919e737c6a3b-ca-trust-extracted" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.632038 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="6ea678ab-3438-413e-bfe3-290ae7725660" volumeName="kubernetes.io/configmap/6ea678ab-3438-413e-bfe3-290ae7725660-ovnkube-config" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.632122 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="7539238d-5fe0-46ed-884e-1c3b566537ec" volumeName="kubernetes.io/projected/7539238d-5fe0-46ed-884e-1c3b566537ec-kube-api-access-tk88c" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.632206 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="8cea82b4-6893-4ddc-af9f-1bb5ae425c5b" volumeName="kubernetes.io/secret/8cea82b4-6893-4ddc-af9f-1bb5ae425c5b-serving-cert" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.632278 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="efdd0498-1daa-4136-9a4a-3b948c2293fc" volumeName="kubernetes.io/secret/efdd0498-1daa-4136-9a4a-3b948c2293fc-webhook-certs" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.632358 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="1d611f23-29be-4491-8495-bee1670e935f" volumeName="kubernetes.io/empty-dir/1d611f23-29be-4491-8495-bee1670e935f-catalog-content" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.632429 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="49c341d1-5089-4bc2-86a0-a5e165cfcc6b" volumeName="kubernetes.io/secret/49c341d1-5089-4bc2-86a0-a5e165cfcc6b-v4-0-config-system-router-certs" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.632506 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="5225d0e4-402f-4861-b410-819f433b1803" volumeName="kubernetes.io/empty-dir/5225d0e4-402f-4861-b410-819f433b1803-catalog-content" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.632625 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="6731426b-95fe-49ff-bb5f-40441049fde2" volumeName="kubernetes.io/projected/6731426b-95fe-49ff-bb5f-40441049fde2-kube-api-access-x7zkh" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.632696 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="3cb93b32-e0ae-4377-b9c8-fdb9842c6d59" volumeName="kubernetes.io/projected/3cb93b32-e0ae-4377-b9c8-fdb9842c6d59-kube-api-access-wxkg8" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.632766 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="43509403-f426-496e-be36-56cef71462f5" volumeName="kubernetes.io/configmap/43509403-f426-496e-be36-56cef71462f5-console-config" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.632846 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="6402fda4-df10-493c-b4e5-d0569419652d" volumeName="kubernetes.io/configmap/6402fda4-df10-493c-b4e5-d0569419652d-config" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.632922 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="6ea678ab-3438-413e-bfe3-290ae7725660" volumeName="kubernetes.io/secret/6ea678ab-3438-413e-bfe3-290ae7725660-ovn-node-metrics-cert" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.632990 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="1bf7eb37-55a3-4c65-b768-a94c82151e69" volumeName="kubernetes.io/configmap/1bf7eb37-55a3-4c65-b768-a94c82151e69-trusted-ca-bundle" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.633062 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="1bf7eb37-55a3-4c65-b768-a94c82151e69" volumeName="kubernetes.io/secret/1bf7eb37-55a3-4c65-b768-a94c82151e69-etcd-client" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.633129 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="20b0d48f-5fd6-431c-a545-e3c800c7b866" volumeName="kubernetes.io/projected/20b0d48f-5fd6-431c-a545-e3c800c7b866-kube-api-access-w9rds" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.633203 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="308be0ea-9f5f-4b29-aeb1-5abd31a0b17b" volumeName="kubernetes.io/projected/308be0ea-9f5f-4b29-aeb1-5abd31a0b17b-kube-api-access-6ccd8" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.633273 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="6402fda4-df10-493c-b4e5-d0569419652d" volumeName="kubernetes.io/secret/6402fda4-df10-493c-b4e5-d0569419652d-machine-api-operator-tls" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.633349 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="a31745f5-9847-4afe-82a5-3161cc66ca93" volumeName="kubernetes.io/projected/a31745f5-9847-4afe-82a5-3161cc66ca93-kube-api-access-lz9wn" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.633436 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="09ae3b1a-e8e7-4524-b54b-61eab6f9239a" volumeName="kubernetes.io/secret/09ae3b1a-e8e7-4524-b54b-61eab6f9239a-etcd-client" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.633714 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="308be0ea-9f5f-4b29-aeb1-5abd31a0b17b" volumeName="kubernetes.io/secret/308be0ea-9f5f-4b29-aeb1-5abd31a0b17b-webhook-cert" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.633821 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="49c341d1-5089-4bc2-86a0-a5e165cfcc6b" volumeName="kubernetes.io/configmap/49c341d1-5089-4bc2-86a0-a5e165cfcc6b-v4-0-config-system-trusted-ca-bundle" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.633927 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="5225d0e4-402f-4861-b410-819f433b1803" volumeName="kubernetes.io/empty-dir/5225d0e4-402f-4861-b410-819f433b1803-utilities" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.634013 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="49c341d1-5089-4bc2-86a0-a5e165cfcc6b" volumeName="kubernetes.io/secret/49c341d1-5089-4bc2-86a0-a5e165cfcc6b-v4-0-config-system-ocp-branding-template" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.634097 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="5fe579f8-e8a6-4643-bce5-a661393c4dde" volumeName="kubernetes.io/secret/5fe579f8-e8a6-4643-bce5-a661393c4dde-node-bootstrap-token" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.634186 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="b6cd30de-2eeb-49a2-ab40-9167f4560ff5" volumeName="kubernetes.io/configmap/b6cd30de-2eeb-49a2-ab40-9167f4560ff5-marketplace-trusted-ca" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.634299 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="c03ee662-fb2f-4fc4-a2c1-af487c19d254" volumeName="kubernetes.io/projected/c03ee662-fb2f-4fc4-a2c1-af487c19d254-kube-api-access-v47cf" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.634408 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="09ae3b1a-e8e7-4524-b54b-61eab6f9239a" volumeName="kubernetes.io/secret/09ae3b1a-e8e7-4524-b54b-61eab6f9239a-encryption-config" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.634498 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="22c825df-677d-4ca6-82db-3454ed06e783" volumeName="kubernetes.io/configmap/22c825df-677d-4ca6-82db-3454ed06e783-config" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.634606 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="37a5e44f-9a88-4405-be8a-b645485e7312" volumeName="kubernetes.io/projected/37a5e44f-9a88-4405-be8a-b645485e7312-kube-api-access-rdwmf" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.634689 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="49c341d1-5089-4bc2-86a0-a5e165cfcc6b" volumeName="kubernetes.io/secret/49c341d1-5089-4bc2-86a0-a5e165cfcc6b-v4-0-config-user-template-provider-selection" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.634780 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="e7e6199b-1264-4501-8953-767f51328d08" volumeName="kubernetes.io/configmap/e7e6199b-1264-4501-8953-767f51328d08-config" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.634876 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="fda69060-fa79-4696-b1a6-7980f124bf7c" volumeName="kubernetes.io/projected/fda69060-fa79-4696-b1a6-7980f124bf7c-kube-api-access-xcgwh" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.634969 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="09ae3b1a-e8e7-4524-b54b-61eab6f9239a" volumeName="kubernetes.io/projected/09ae3b1a-e8e7-4524-b54b-61eab6f9239a-kube-api-access-zkvpv" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.635060 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="0b78653f-4ff9-4508-8672-245ed9b561e3" volumeName="kubernetes.io/configmap/0b78653f-4ff9-4508-8672-245ed9b561e3-service-ca" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.635152 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="31d8b7a1-420e-4252-a5b7-eebe8a111292" volumeName="kubernetes.io/configmap/31d8b7a1-420e-4252-a5b7-eebe8a111292-images" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.635228 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="d75a4c96-2883-4a0b-bab2-0fab2b6c0b49" volumeName="kubernetes.io/projected/d75a4c96-2883-4a0b-bab2-0fab2b6c0b49-kube-api-access-rczfb" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.635301 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="44663579-783b-4372-86d6-acf235a62d72" volumeName="kubernetes.io/projected/44663579-783b-4372-86d6-acf235a62d72-kube-api-access-vt5rc" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.635369 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="5441d097-087c-4d9a-baa8-b210afa90fc9" volumeName="kubernetes.io/projected/5441d097-087c-4d9a-baa8-b210afa90fc9-kube-api-access-2d4wz" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.635449 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="6509e943-70c6-444c-bc41-48a544e36fbd" volumeName="kubernetes.io/projected/6509e943-70c6-444c-bc41-48a544e36fbd-kube-api-access-6g6sz" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.635547 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="7539238d-5fe0-46ed-884e-1c3b566537ec" volumeName="kubernetes.io/secret/7539238d-5fe0-46ed-884e-1c3b566537ec-serving-cert" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.635622 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="01ab3dd5-8196-46d0-ad33-122e2ca51def" volumeName="kubernetes.io/projected/01ab3dd5-8196-46d0-ad33-122e2ca51def-kube-api-access-w7l8j" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.635699 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="09efc573-dbb6-4249-bd59-9b87aba8dd28" volumeName="kubernetes.io/secret/09efc573-dbb6-4249-bd59-9b87aba8dd28-etcd-client" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.635766 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="1bf7eb37-55a3-4c65-b768-a94c82151e69" volumeName="kubernetes.io/configmap/1bf7eb37-55a3-4c65-b768-a94c82151e69-audit" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.635844 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="22c825df-677d-4ca6-82db-3454ed06e783" volumeName="kubernetes.io/secret/22c825df-677d-4ca6-82db-3454ed06e783-machine-approver-tls" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.635923 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="8f668bae-612b-4b75-9490-919e737c6a3b" volumeName="kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.636957 5052 reconstruct.go:144] "Volume is marked device as uncertain and added into the actual state" volumeName="kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8" deviceMountPath="/var/lib/kubelet/plugins/kubernetes.io/csi/kubevirt.io.hostpath-provisioner/1f4776af88835e41c12b831b4c9fed40233456d14189815a54dbe7f892fc1983/globalmount" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.637058 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="7583ce53-e0fe-4a16-9e4d-50516596a136" volumeName="kubernetes.io/secret/7583ce53-e0fe-4a16-9e4d-50516596a136-serving-cert" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.637138 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="b6312bbd-5731-4ea0-a20f-81d5a57df44a" volumeName="kubernetes.io/secret/b6312bbd-5731-4ea0-a20f-81d5a57df44a-srv-cert" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.637228 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="c03ee662-fb2f-4fc4-a2c1-af487c19d254" volumeName="kubernetes.io/secret/c03ee662-fb2f-4fc4-a2c1-af487c19d254-default-certificate" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.637320 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="1bf7eb37-55a3-4c65-b768-a94c82151e69" volumeName="kubernetes.io/secret/1bf7eb37-55a3-4c65-b768-a94c82151e69-serving-cert" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.638669 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="308be0ea-9f5f-4b29-aeb1-5abd31a0b17b" volumeName="kubernetes.io/secret/308be0ea-9f5f-4b29-aeb1-5abd31a0b17b-apiservice-cert" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.638752 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="6ea678ab-3438-413e-bfe3-290ae7725660" volumeName="kubernetes.io/projected/6ea678ab-3438-413e-bfe3-290ae7725660-kube-api-access-htfz6" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.638797 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="7583ce53-e0fe-4a16-9e4d-50516596a136" volumeName="kubernetes.io/configmap/7583ce53-e0fe-4a16-9e4d-50516596a136-proxy-ca-bundles" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.638815 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="43509403-f426-496e-be36-56cef71462f5" volumeName="kubernetes.io/secret/43509403-f426-496e-be36-56cef71462f5-console-serving-cert" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.638831 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="6ea678ab-3438-413e-bfe3-290ae7725660" volumeName="kubernetes.io/configmap/6ea678ab-3438-413e-bfe3-290ae7725660-ovnkube-script-lib" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.638858 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="6402fda4-df10-493c-b4e5-d0569419652d" volumeName="kubernetes.io/projected/6402fda4-df10-493c-b4e5-d0569419652d-kube-api-access-mg5zb" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.638880 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="925f1c65-6136-48ba-85aa-3a3b50560753" volumeName="kubernetes.io/projected/925f1c65-6136-48ba-85aa-3a3b50560753-kube-api-access-s4n52" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.638905 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="ef543e1b-8068-4ea3-b32a-61027b32e95d" volumeName="kubernetes.io/secret/ef543e1b-8068-4ea3-b32a-61027b32e95d-webhook-cert" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.638925 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="1bf7eb37-55a3-4c65-b768-a94c82151e69" volumeName="kubernetes.io/projected/1bf7eb37-55a3-4c65-b768-a94c82151e69-kube-api-access-sb6h7" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.638970 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="25e176fe-21b4-4974-b1ed-c8b94f112a7f" volumeName="kubernetes.io/projected/25e176fe-21b4-4974-b1ed-c8b94f112a7f-kube-api-access-d4lsv" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.638999 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="3ab1a177-2de0-46d9-b765-d0d0649bb42e" volumeName="kubernetes.io/projected/3ab1a177-2de0-46d9-b765-d0d0649bb42e-kube-api-access-4d4hj" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.639021 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="57a731c4-ef35-47a8-b875-bfb08a7f8011" volumeName="kubernetes.io/projected/57a731c4-ef35-47a8-b875-bfb08a7f8011-kube-api-access-cfbct" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.639048 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="e7e6199b-1264-4501-8953-767f51328d08" volumeName="kubernetes.io/projected/e7e6199b-1264-4501-8953-767f51328d08-kube-api-access" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.639074 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="308be0ea-9f5f-4b29-aeb1-5abd31a0b17b" volumeName="kubernetes.io/empty-dir/308be0ea-9f5f-4b29-aeb1-5abd31a0b17b-tmpfs" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.639095 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="7bb08738-c794-4ee8-9972-3a62ca171029" volumeName="kubernetes.io/projected/7bb08738-c794-4ee8-9972-3a62ca171029-kube-api-access-279lb" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.639116 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="9d4552c7-cd75-42dd-8880-30dd377c49a4" volumeName="kubernetes.io/projected/9d4552c7-cd75-42dd-8880-30dd377c49a4-kube-api-access-pcxfs" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.639139 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="c03ee662-fb2f-4fc4-a2c1-af487c19d254" volumeName="kubernetes.io/secret/c03ee662-fb2f-4fc4-a2c1-af487c19d254-metrics-certs" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.639164 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="6731426b-95fe-49ff-bb5f-40441049fde2" volumeName="kubernetes.io/secret/6731426b-95fe-49ff-bb5f-40441049fde2-control-plane-machine-set-operator-tls" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.639206 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="a31745f5-9847-4afe-82a5-3161cc66ca93" volumeName="kubernetes.io/configmap/a31745f5-9847-4afe-82a5-3161cc66ca93-trusted-ca" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.639228 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="1386a44e-36a2-460c-96d0-0359d2b6f0f5" volumeName="kubernetes.io/secret/1386a44e-36a2-460c-96d0-0359d2b6f0f5-serving-cert" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.639245 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="25e176fe-21b4-4974-b1ed-c8b94f112a7f" volumeName="kubernetes.io/configmap/25e176fe-21b4-4974-b1ed-c8b94f112a7f-signing-cabundle" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.639269 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="25e176fe-21b4-4974-b1ed-c8b94f112a7f" volumeName="kubernetes.io/secret/25e176fe-21b4-4974-b1ed-c8b94f112a7f-signing-key" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.639282 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="5441d097-087c-4d9a-baa8-b210afa90fc9" volumeName="kubernetes.io/configmap/5441d097-087c-4d9a-baa8-b210afa90fc9-client-ca" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.639303 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="5fe579f8-e8a6-4643-bce5-a661393c4dde" volumeName="kubernetes.io/secret/5fe579f8-e8a6-4643-bce5-a661393c4dde-certs" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.639318 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="6509e943-70c6-444c-bc41-48a544e36fbd" volumeName="kubernetes.io/configmap/6509e943-70c6-444c-bc41-48a544e36fbd-config" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.639330 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="8cea82b4-6893-4ddc-af9f-1bb5ae425c5b" volumeName="kubernetes.io/projected/8cea82b4-6893-4ddc-af9f-1bb5ae425c5b-kube-api-access-w4xd4" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.639350 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="b11524ee-3fca-4b1b-9cdf-6da289fdbc7d" volumeName="kubernetes.io/empty-dir/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d-catalog-content" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.639362 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="01ab3dd5-8196-46d0-ad33-122e2ca51def" volumeName="kubernetes.io/secret/01ab3dd5-8196-46d0-ad33-122e2ca51def-serving-cert" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.639380 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="0b574797-001e-440a-8f4e-c0be86edad0f" volumeName="kubernetes.io/secret/0b574797-001e-440a-8f4e-c0be86edad0f-proxy-tls" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.639398 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="31d8b7a1-420e-4252-a5b7-eebe8a111292" volumeName="kubernetes.io/projected/31d8b7a1-420e-4252-a5b7-eebe8a111292-kube-api-access-zgdk5" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.639414 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="31d8b7a1-420e-4252-a5b7-eebe8a111292" volumeName="kubernetes.io/secret/31d8b7a1-420e-4252-a5b7-eebe8a111292-proxy-tls" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.639436 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="b6312bbd-5731-4ea0-a20f-81d5a57df44a" volumeName="kubernetes.io/secret/b6312bbd-5731-4ea0-a20f-81d5a57df44a-profile-collector-cert" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.639450 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="c03ee662-fb2f-4fc4-a2c1-af487c19d254" volumeName="kubernetes.io/configmap/c03ee662-fb2f-4fc4-a2c1-af487c19d254-service-ca-bundle" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.639475 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="efdd0498-1daa-4136-9a4a-3b948c2293fc" volumeName="kubernetes.io/projected/efdd0498-1daa-4136-9a4a-3b948c2293fc-kube-api-access-fqsjt" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.639493 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="fda69060-fa79-4696-b1a6-7980f124bf7c" volumeName="kubernetes.io/configmap/fda69060-fa79-4696-b1a6-7980f124bf7c-mcd-auth-proxy-config" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.639529 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="49c341d1-5089-4bc2-86a0-a5e165cfcc6b" volumeName="kubernetes.io/secret/49c341d1-5089-4bc2-86a0-a5e165cfcc6b-v4-0-config-user-idp-0-file-data" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.639550 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="5441d097-087c-4d9a-baa8-b210afa90fc9" volumeName="kubernetes.io/configmap/5441d097-087c-4d9a-baa8-b210afa90fc9-config" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.639563 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="8f668bae-612b-4b75-9490-919e737c6a3b" volumeName="kubernetes.io/configmap/8f668bae-612b-4b75-9490-919e737c6a3b-trusted-ca" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.639584 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="8f668bae-612b-4b75-9490-919e737c6a3b" volumeName="kubernetes.io/configmap/8f668bae-612b-4b75-9490-919e737c6a3b-registry-certificates" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.639601 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="f88749ec-7931-4ee7-b3fc-1ec5e11f92e9" volumeName="kubernetes.io/projected/f88749ec-7931-4ee7-b3fc-1ec5e11f92e9-kube-api-access-dbsvg" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.639617 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="f88749ec-7931-4ee7-b3fc-1ec5e11f92e9" volumeName="kubernetes.io/secret/f88749ec-7931-4ee7-b3fc-1ec5e11f92e9-profile-collector-cert" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.639639 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="43509403-f426-496e-be36-56cef71462f5" volumeName="kubernetes.io/configmap/43509403-f426-496e-be36-56cef71462f5-oauth-serving-cert" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.639651 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="49c341d1-5089-4bc2-86a0-a5e165cfcc6b" volumeName="kubernetes.io/configmap/49c341d1-5089-4bc2-86a0-a5e165cfcc6b-v4-0-config-system-cliconfig" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.639673 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="4bb40260-dbaa-4fb0-84df-5e680505d512" volumeName="kubernetes.io/projected/4bb40260-dbaa-4fb0-84df-5e680505d512-kube-api-access-2w9zh" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.639684 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="7583ce53-e0fe-4a16-9e4d-50516596a136" volumeName="kubernetes.io/projected/7583ce53-e0fe-4a16-9e4d-50516596a136-kube-api-access-xcphl" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.639694 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="a31745f5-9847-4afe-82a5-3161cc66ca93" volumeName="kubernetes.io/secret/a31745f5-9847-4afe-82a5-3161cc66ca93-metrics-tls" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.639714 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="3cb93b32-e0ae-4377-b9c8-fdb9842c6d59" volumeName="kubernetes.io/configmap/3cb93b32-e0ae-4377-b9c8-fdb9842c6d59-serviceca" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.639726 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="496e6271-fb68-4057-954e-a0d97a4afa3f" volumeName="kubernetes.io/secret/496e6271-fb68-4057-954e-a0d97a4afa3f-serving-cert" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.639739 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="7bb08738-c794-4ee8-9972-3a62ca171029" volumeName="kubernetes.io/configmap/7bb08738-c794-4ee8-9972-3a62ca171029-cni-binary-copy" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.639758 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="9d4552c7-cd75-42dd-8880-30dd377c49a4" volumeName="kubernetes.io/configmap/9d4552c7-cd75-42dd-8880-30dd377c49a4-trusted-ca" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.639769 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="8f668bae-612b-4b75-9490-919e737c6a3b" volumeName="kubernetes.io/projected/8f668bae-612b-4b75-9490-919e737c6a3b-kube-api-access-kfwg7" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.639787 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="3ab1a177-2de0-46d9-b765-d0d0649bb42e" volumeName="kubernetes.io/secret/3ab1a177-2de0-46d9-b765-d0d0649bb42e-package-server-manager-serving-cert" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.639802 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="57a731c4-ef35-47a8-b875-bfb08a7f8011" volumeName="kubernetes.io/empty-dir/57a731c4-ef35-47a8-b875-bfb08a7f8011-utilities" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.639817 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="5fe485a1-e14f-4c09-b5b9-f252bc42b7e8" volumeName="kubernetes.io/configmap/5fe485a1-e14f-4c09-b5b9-f252bc42b7e8-nginx-conf" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.639844 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="7539238d-5fe0-46ed-884e-1c3b566537ec" volumeName="kubernetes.io/configmap/7539238d-5fe0-46ed-884e-1c3b566537ec-config" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.639861 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="5441d097-087c-4d9a-baa8-b210afa90fc9" volumeName="kubernetes.io/secret/5441d097-087c-4d9a-baa8-b210afa90fc9-serving-cert" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.639885 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="5b88f790-22fa-440e-b583-365168c0b23d" volumeName="kubernetes.io/secret/5b88f790-22fa-440e-b583-365168c0b23d-metrics-certs" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.639897 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="09ae3b1a-e8e7-4524-b54b-61eab6f9239a" volumeName="kubernetes.io/configmap/09ae3b1a-e8e7-4524-b54b-61eab6f9239a-etcd-serving-ca" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.639909 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="0b78653f-4ff9-4508-8672-245ed9b561e3" volumeName="kubernetes.io/secret/0b78653f-4ff9-4508-8672-245ed9b561e3-serving-cert" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.639926 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="43509403-f426-496e-be36-56cef71462f5" volumeName="kubernetes.io/configmap/43509403-f426-496e-be36-56cef71462f5-trusted-ca-bundle" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.639937 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="4bb40260-dbaa-4fb0-84df-5e680505d512" volumeName="kubernetes.io/configmap/4bb40260-dbaa-4fb0-84df-5e680505d512-multus-daemon-config" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.639956 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="6ea678ab-3438-413e-bfe3-290ae7725660" volumeName="kubernetes.io/configmap/6ea678ab-3438-413e-bfe3-290ae7725660-env-overrides" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.639969 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="9d4552c7-cd75-42dd-8880-30dd377c49a4" volumeName="kubernetes.io/configmap/9d4552c7-cd75-42dd-8880-30dd377c49a4-config" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.639982 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="22c825df-677d-4ca6-82db-3454ed06e783" volumeName="kubernetes.io/projected/22c825df-677d-4ca6-82db-3454ed06e783-kube-api-access-7c4vf" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.640003 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="31d8b7a1-420e-4252-a5b7-eebe8a111292" volumeName="kubernetes.io/configmap/31d8b7a1-420e-4252-a5b7-eebe8a111292-auth-proxy-config" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.640017 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="496e6271-fb68-4057-954e-a0d97a4afa3f" volumeName="kubernetes.io/configmap/496e6271-fb68-4057-954e-a0d97a4afa3f-config" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.640060 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="5fe485a1-e14f-4c09-b5b9-f252bc42b7e8" volumeName="kubernetes.io/secret/5fe485a1-e14f-4c09-b5b9-f252bc42b7e8-networking-console-plugin-cert" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.640075 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="87cf06ed-a83f-41a7-828d-70653580a8cb" volumeName="kubernetes.io/secret/87cf06ed-a83f-41a7-828d-70653580a8cb-metrics-tls" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.640088 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="8f668bae-612b-4b75-9490-919e737c6a3b" volumeName="kubernetes.io/projected/8f668bae-612b-4b75-9490-919e737c6a3b-registry-tls" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.640106 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="9d4552c7-cd75-42dd-8880-30dd377c49a4" volumeName="kubernetes.io/secret/9d4552c7-cd75-42dd-8880-30dd377c49a4-serving-cert" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.640117 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="a0128f3a-b052-44ed-a84e-c4c8aaf17c13" volumeName="kubernetes.io/secret/a0128f3a-b052-44ed-a84e-c4c8aaf17c13-samples-operator-tls" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.640134 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="09efc573-dbb6-4249-bd59-9b87aba8dd28" volumeName="kubernetes.io/configmap/09efc573-dbb6-4249-bd59-9b87aba8dd28-etcd-service-ca" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.640146 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="43509403-f426-496e-be36-56cef71462f5" volumeName="kubernetes.io/secret/43509403-f426-496e-be36-56cef71462f5-console-oauth-config" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.640157 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="49c341d1-5089-4bc2-86a0-a5e165cfcc6b" volumeName="kubernetes.io/secret/49c341d1-5089-4bc2-86a0-a5e165cfcc6b-v4-0-config-system-serving-cert" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.640173 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="49c341d1-5089-4bc2-86a0-a5e165cfcc6b" volumeName="kubernetes.io/secret/49c341d1-5089-4bc2-86a0-a5e165cfcc6b-v4-0-config-system-session" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.640186 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="b6cd30de-2eeb-49a2-ab40-9167f4560ff5" volumeName="kubernetes.io/projected/b6cd30de-2eeb-49a2-ab40-9167f4560ff5-kube-api-access-pj782" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.640202 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="bf126b07-da06-4140-9a57-dfd54fc6b486" volumeName="kubernetes.io/projected/bf126b07-da06-4140-9a57-dfd54fc6b486-bound-sa-token" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.640225 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="f88749ec-7931-4ee7-b3fc-1ec5e11f92e9" volumeName="kubernetes.io/secret/f88749ec-7931-4ee7-b3fc-1ec5e11f92e9-srv-cert" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.640243 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="49ef4625-1d3a-4a9f-b595-c2433d32326d" volumeName="kubernetes.io/projected/49ef4625-1d3a-4a9f-b595-c2433d32326d-kube-api-access-pjr6v" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.640261 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="4bb40260-dbaa-4fb0-84df-5e680505d512" volumeName="kubernetes.io/configmap/4bb40260-dbaa-4fb0-84df-5e680505d512-cni-binary-copy" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.640272 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="5b88f790-22fa-440e-b583-365168c0b23d" volumeName="kubernetes.io/projected/5b88f790-22fa-440e-b583-365168c0b23d-kube-api-access-jkwtn" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.640286 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="6509e943-70c6-444c-bc41-48a544e36fbd" volumeName="kubernetes.io/secret/6509e943-70c6-444c-bc41-48a544e36fbd-serving-cert" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.640303 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="bc5039c0-ea34-426b-a2b7-fbbc87b49a6d" volumeName="kubernetes.io/projected/bc5039c0-ea34-426b-a2b7-fbbc87b49a6d-kube-api-access-mnrrd" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.640315 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="d75a4c96-2883-4a0b-bab2-0fab2b6c0b49" volumeName="kubernetes.io/configmap/d75a4c96-2883-4a0b-bab2-0fab2b6c0b49-iptables-alerter-script" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.640334 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="ef543e1b-8068-4ea3-b32a-61027b32e95d" volumeName="kubernetes.io/configmap/ef543e1b-8068-4ea3-b32a-61027b32e95d-env-overrides" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.640345 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="09ae3b1a-e8e7-4524-b54b-61eab6f9239a" volumeName="kubernetes.io/secret/09ae3b1a-e8e7-4524-b54b-61eab6f9239a-serving-cert" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.640356 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="09efc573-dbb6-4249-bd59-9b87aba8dd28" volumeName="kubernetes.io/configmap/09efc573-dbb6-4249-bd59-9b87aba8dd28-config" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.640373 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="210d8245-ebfc-4e3b-ac4a-e21ce76f9a7c" volumeName="kubernetes.io/projected/210d8245-ebfc-4e3b-ac4a-e21ce76f9a7c-kube-api-access-qs4fp" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.640389 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="b11524ee-3fca-4b1b-9cdf-6da289fdbc7d" volumeName="kubernetes.io/projected/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d-kube-api-access-x4zgh" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.640420 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="bf126b07-da06-4140-9a57-dfd54fc6b486" volumeName="kubernetes.io/configmap/bf126b07-da06-4140-9a57-dfd54fc6b486-trusted-ca" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.640437 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="09ae3b1a-e8e7-4524-b54b-61eab6f9239a" volumeName="kubernetes.io/configmap/09ae3b1a-e8e7-4524-b54b-61eab6f9239a-audit-policies" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.640449 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="0b574797-001e-440a-8f4e-c0be86edad0f" volumeName="kubernetes.io/configmap/0b574797-001e-440a-8f4e-c0be86edad0f-mcc-auth-proxy-config" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.640472 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="43509403-f426-496e-be36-56cef71462f5" volumeName="kubernetes.io/projected/43509403-f426-496e-be36-56cef71462f5-kube-api-access-qg5z5" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.640486 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="925f1c65-6136-48ba-85aa-3a3b50560753" volumeName="kubernetes.io/configmap/925f1c65-6136-48ba-85aa-3a3b50560753-ovnkube-config" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.640622 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="5fe579f8-e8a6-4643-bce5-a661393c4dde" volumeName="kubernetes.io/projected/5fe579f8-e8a6-4643-bce5-a661393c4dde-kube-api-access-fcqwp" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.640640 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="6509e943-70c6-444c-bc41-48a544e36fbd" volumeName="kubernetes.io/configmap/6509e943-70c6-444c-bc41-48a544e36fbd-service-ca-bundle" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.640654 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="6509e943-70c6-444c-bc41-48a544e36fbd" volumeName="kubernetes.io/configmap/6509e943-70c6-444c-bc41-48a544e36fbd-trusted-ca-bundle" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.640676 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="9d751cbb-f2e2-430d-9754-c882a5e924a5" volumeName="kubernetes.io/projected/9d751cbb-f2e2-430d-9754-c882a5e924a5-kube-api-access-s2dwl" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.640691 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="09ae3b1a-e8e7-4524-b54b-61eab6f9239a" volumeName="kubernetes.io/configmap/09ae3b1a-e8e7-4524-b54b-61eab6f9239a-trusted-ca-bundle" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.640705 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="0b574797-001e-440a-8f4e-c0be86edad0f" volumeName="kubernetes.io/projected/0b574797-001e-440a-8f4e-c0be86edad0f-kube-api-access-lzf88" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.640728 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="1d611f23-29be-4491-8495-bee1670e935f" volumeName="kubernetes.io/projected/1d611f23-29be-4491-8495-bee1670e935f-kube-api-access-bf2bz" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.640743 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="20b0d48f-5fd6-431c-a545-e3c800c7b866" volumeName="kubernetes.io/secret/20b0d48f-5fd6-431c-a545-e3c800c7b866-cert" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.640867 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="bf126b07-da06-4140-9a57-dfd54fc6b486" volumeName="kubernetes.io/secret/bf126b07-da06-4140-9a57-dfd54fc6b486-image-registry-operator-tls" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.640905 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="e7e6199b-1264-4501-8953-767f51328d08" volumeName="kubernetes.io/secret/e7e6199b-1264-4501-8953-767f51328d08-serving-cert" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.640929 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="925f1c65-6136-48ba-85aa-3a3b50560753" volumeName="kubernetes.io/configmap/925f1c65-6136-48ba-85aa-3a3b50560753-env-overrides" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.640972 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="925f1c65-6136-48ba-85aa-3a3b50560753" volumeName="kubernetes.io/secret/925f1c65-6136-48ba-85aa-3a3b50560753-ovn-control-plane-metrics-cert" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.641000 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="1bf7eb37-55a3-4c65-b768-a94c82151e69" volumeName="kubernetes.io/configmap/1bf7eb37-55a3-4c65-b768-a94c82151e69-config" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.641042 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="496e6271-fb68-4057-954e-a0d97a4afa3f" volumeName="kubernetes.io/projected/496e6271-fb68-4057-954e-a0d97a4afa3f-kube-api-access" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.641066 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="49c341d1-5089-4bc2-86a0-a5e165cfcc6b" volumeName="kubernetes.io/configmap/49c341d1-5089-4bc2-86a0-a5e165cfcc6b-v4-0-config-system-service-ca" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.641088 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="7583ce53-e0fe-4a16-9e4d-50516596a136" volumeName="kubernetes.io/configmap/7583ce53-e0fe-4a16-9e4d-50516596a136-config" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.641102 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="43509403-f426-496e-be36-56cef71462f5" volumeName="kubernetes.io/configmap/43509403-f426-496e-be36-56cef71462f5-service-ca" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.641114 5052 reconstruct.go:130] "Volume is marked as uncertain and added into the actual state" pod="" podName="8f668bae-612b-4b75-9490-919e737c6a3b" volumeName="kubernetes.io/projected/8f668bae-612b-4b75-9490-919e737c6a3b-bound-sa-token" seLinuxMountContext="" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.641129 5052 reconstruct.go:97] "Volume reconstruction finished" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.641144 5052 reconciler.go:26] "Reconciler: start to sync state" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.642647 5052 manager.go:324] Recovery completed Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.653529 5052 kubelet_node_status.go:401] "Setting node annotation to enable volume controller attach/detach" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.661211 5052 kubelet_node_status.go:724] "Recording event message for node" node="crc" event="NodeHasSufficientMemory" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.661282 5052 kubelet_node_status.go:724] "Recording event message for node" node="crc" event="NodeHasNoDiskPressure" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.661296 5052 kubelet_node_status.go:724] "Recording event message for node" node="crc" event="NodeHasSufficientPID" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.662173 5052 cpu_manager.go:225] "Starting CPU manager" policy="none" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.662188 5052 cpu_manager.go:226] "Reconciling" reconcilePeriod="10s" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.662215 5052 state_mem.go:36] "Initialized new in-memory state store" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.669198 5052 kubelet_network_linux.go:50] "Initialized iptables rules." protocol="IPv4" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.671502 5052 policy_none.go:49] "None policy: Start" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.671645 5052 kubelet_network_linux.go:50] "Initialized iptables rules." protocol="IPv6" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.671694 5052 status_manager.go:217] "Starting to sync pod status with apiserver" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.671732 5052 kubelet.go:2335] "Starting kubelet main sync loop" Dec 03 11:20:09 crc kubenswrapper[5052]: E1203 11:20:09.671779 5052 kubelet.go:2359] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.672893 5052 memory_manager.go:170] "Starting memorymanager" policy="None" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.672943 5052 state_mem.go:35] "Initializing new in-memory state store" Dec 03 11:20:09 crc kubenswrapper[5052]: W1203 11:20:09.673896 5052 reflector.go:561] k8s.io/client-go/informers/factory.go:160: failed to list *v1.RuntimeClass: Get "https://api-int.crc.testing:6443/apis/node.k8s.io/v1/runtimeclasses?limit=500&resourceVersion=0": dial tcp 38.129.56.252:6443: connect: connection refused Dec 03 11:20:09 crc kubenswrapper[5052]: E1203 11:20:09.673961 5052 reflector.go:158] "Unhandled Error" err="k8s.io/client-go/informers/factory.go:160: Failed to watch *v1.RuntimeClass: failed to list *v1.RuntimeClass: Get \"https://api-int.crc.testing:6443/apis/node.k8s.io/v1/runtimeclasses?limit=500&resourceVersion=0\": dial tcp 38.129.56.252:6443: connect: connection refused" logger="UnhandledError" Dec 03 11:20:09 crc kubenswrapper[5052]: E1203 11:20:09.715660 5052 kubelet_node_status.go:503] "Error getting the current node from lister" err="node \"crc\" not found" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.736191 5052 manager.go:334] "Starting Device Plugin manager" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.736278 5052 manager.go:513] "Failed to read data from checkpoint" checkpoint="kubelet_internal_checkpoint" err="checkpoint is not found" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.736297 5052 server.go:79] "Starting device plugin registration server" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.736932 5052 eviction_manager.go:189] "Eviction manager: starting control loop" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.736955 5052 container_log_manager.go:189] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.737125 5052 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.737257 5052 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.737274 5052 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Dec 03 11:20:09 crc kubenswrapper[5052]: E1203 11:20:09.749430 5052 eviction_manager.go:285] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"crc\" not found" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.772391 5052 kubelet.go:2421] "SyncLoop ADD" source="file" pods=["openshift-etcd/etcd-crc","openshift-kube-apiserver/kube-apiserver-crc","openshift-kube-controller-manager/kube-controller-manager-crc","openshift-kube-scheduler/openshift-kube-scheduler-crc","openshift-machine-config-operator/kube-rbac-proxy-crio-crc"] Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.772623 5052 kubelet_node_status.go:401] "Setting node annotation to enable volume controller attach/detach" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.774098 5052 kubelet_node_status.go:724] "Recording event message for node" node="crc" event="NodeHasSufficientMemory" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.774140 5052 kubelet_node_status.go:724] "Recording event message for node" node="crc" event="NodeHasNoDiskPressure" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.774153 5052 kubelet_node_status.go:724] "Recording event message for node" node="crc" event="NodeHasSufficientPID" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.774381 5052 kubelet_node_status.go:401] "Setting node annotation to enable volume controller attach/detach" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.774766 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-etcd/etcd-crc" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.774839 5052 kubelet_node_status.go:401] "Setting node annotation to enable volume controller attach/detach" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.775609 5052 kubelet_node_status.go:724] "Recording event message for node" node="crc" event="NodeHasSufficientMemory" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.775671 5052 kubelet_node_status.go:724] "Recording event message for node" node="crc" event="NodeHasNoDiskPressure" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.775692 5052 kubelet_node_status.go:724] "Recording event message for node" node="crc" event="NodeHasSufficientPID" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.775902 5052 kubelet_node_status.go:724] "Recording event message for node" node="crc" event="NodeHasSufficientMemory" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.775930 5052 kubelet_node_status.go:724] "Recording event message for node" node="crc" event="NodeHasNoDiskPressure" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.775941 5052 kubelet_node_status.go:724] "Recording event message for node" node="crc" event="NodeHasSufficientPID" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.775998 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-apiserver/kube-apiserver-crc" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.776045 5052 kubelet_node_status.go:401] "Setting node annotation to enable volume controller attach/detach" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.775911 5052 kubelet_node_status.go:401] "Setting node annotation to enable volume controller attach/detach" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.776998 5052 kubelet_node_status.go:724] "Recording event message for node" node="crc" event="NodeHasSufficientMemory" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.777030 5052 kubelet_node_status.go:724] "Recording event message for node" node="crc" event="NodeHasNoDiskPressure" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.777042 5052 kubelet_node_status.go:724] "Recording event message for node" node="crc" event="NodeHasSufficientPID" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.777282 5052 kubelet_node_status.go:724] "Recording event message for node" node="crc" event="NodeHasSufficientMemory" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.777314 5052 kubelet_node_status.go:724] "Recording event message for node" node="crc" event="NodeHasNoDiskPressure" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.777328 5052 kubelet_node_status.go:724] "Recording event message for node" node="crc" event="NodeHasSufficientPID" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.777434 5052 kubelet_node_status.go:401] "Setting node annotation to enable volume controller attach/detach" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.777647 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-controller-manager/kube-controller-manager-crc" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.777710 5052 kubelet_node_status.go:401] "Setting node annotation to enable volume controller attach/detach" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.778192 5052 kubelet_node_status.go:724] "Recording event message for node" node="crc" event="NodeHasSufficientMemory" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.778223 5052 kubelet_node_status.go:724] "Recording event message for node" node="crc" event="NodeHasNoDiskPressure" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.778236 5052 kubelet_node_status.go:724] "Recording event message for node" node="crc" event="NodeHasSufficientPID" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.778360 5052 kubelet_node_status.go:401] "Setting node annotation to enable volume controller attach/detach" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.778454 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-scheduler/openshift-kube-scheduler-crc" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.778485 5052 kubelet_node_status.go:401] "Setting node annotation to enable volume controller attach/detach" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.778838 5052 kubelet_node_status.go:724] "Recording event message for node" node="crc" event="NodeHasSufficientMemory" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.778876 5052 kubelet_node_status.go:724] "Recording event message for node" node="crc" event="NodeHasNoDiskPressure" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.778891 5052 kubelet_node_status.go:724] "Recording event message for node" node="crc" event="NodeHasSufficientPID" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.779199 5052 kubelet_node_status.go:724] "Recording event message for node" node="crc" event="NodeHasSufficientMemory" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.779237 5052 kubelet_node_status.go:724] "Recording event message for node" node="crc" event="NodeHasNoDiskPressure" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.779282 5052 kubelet_node_status.go:724] "Recording event message for node" node="crc" event="NodeHasSufficientPID" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.779300 5052 kubelet_node_status.go:724] "Recording event message for node" node="crc" event="NodeHasSufficientMemory" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.779318 5052 kubelet_node_status.go:724] "Recording event message for node" node="crc" event="NodeHasNoDiskPressure" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.779329 5052 kubelet_node_status.go:724] "Recording event message for node" node="crc" event="NodeHasSufficientPID" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.779475 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-crc" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.779502 5052 kubelet_node_status.go:401] "Setting node annotation to enable volume controller attach/detach" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.780084 5052 kubelet_node_status.go:724] "Recording event message for node" node="crc" event="NodeHasSufficientMemory" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.780126 5052 kubelet_node_status.go:724] "Recording event message for node" node="crc" event="NodeHasNoDiskPressure" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.780142 5052 kubelet_node_status.go:724] "Recording event message for node" node="crc" event="NodeHasSufficientPID" Dec 03 11:20:09 crc kubenswrapper[5052]: E1203 11:20:09.821102 5052 controller.go:145] "Failed to ensure lease exists, will retry" err="Get \"https://api-int.crc.testing:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/crc?timeout=10s\": dial tcp 38.129.56.252:6443: connect: connection refused" interval="400ms" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.838763 5052 kubelet_node_status.go:401] "Setting node annotation to enable volume controller attach/detach" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.841324 5052 kubelet_node_status.go:724] "Recording event message for node" node="crc" event="NodeHasSufficientMemory" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.841393 5052 kubelet_node_status.go:724] "Recording event message for node" node="crc" event="NodeHasNoDiskPressure" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.841408 5052 kubelet_node_status.go:724] "Recording event message for node" node="crc" event="NodeHasSufficientPID" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.841496 5052 kubelet_node_status.go:76] "Attempting to register node" node="crc" Dec 03 11:20:09 crc kubenswrapper[5052]: E1203 11:20:09.842718 5052 kubelet_node_status.go:99] "Unable to register node with API server" err="Post \"https://api-int.crc.testing:6443/api/v1/nodes\": dial tcp 38.129.56.252:6443: connect: connection refused" node="crc" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.844873 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d1b160f5dda77d281dd8e69ec8d817f9-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-crc\" (UID: \"d1b160f5dda77d281dd8e69ec8d817f9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-crc" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.844908 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert-dir\" (UniqueName: \"kubernetes.io/host-path/f4b27818a5e8e43d0dc095d08835c792-cert-dir\") pod \"kube-apiserver-crc\" (UID: \"f4b27818a5e8e43d0dc095d08835c792\") " pod="openshift-kube-apiserver/kube-apiserver-crc" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.844928 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert-dir\" (UniqueName: \"kubernetes.io/host-path/f614b9022728cf315e60c057852e563e-cert-dir\") pod \"kube-controller-manager-crc\" (UID: \"f614b9022728cf315e60c057852e563e\") " pod="openshift-kube-controller-manager/kube-controller-manager-crc" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.844945 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"resource-dir\" (UniqueName: \"kubernetes.io/host-path/3dcd261975c3d6b9a6ad6367fd4facd3-resource-dir\") pod \"openshift-kube-scheduler-crc\" (UID: \"3dcd261975c3d6b9a6ad6367fd4facd3\") " pod="openshift-kube-scheduler/openshift-kube-scheduler-crc" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.844963 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d1b160f5dda77d281dd8e69ec8d817f9-etc-kube\") pod \"kube-rbac-proxy-crio-crc\" (UID: \"d1b160f5dda77d281dd8e69ec8d817f9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-crc" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.844979 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert-dir\" (UniqueName: \"kubernetes.io/host-path/3dcd261975c3d6b9a6ad6367fd4facd3-cert-dir\") pod \"openshift-kube-scheduler-crc\" (UID: \"3dcd261975c3d6b9a6ad6367fd4facd3\") " pod="openshift-kube-scheduler/openshift-kube-scheduler-crc" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.844996 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-dir\" (UniqueName: \"kubernetes.io/host-path/2139d3e2895fc6797b9c76a1b4c9886d-data-dir\") pod \"etcd-crc\" (UID: \"2139d3e2895fc6797b9c76a1b4c9886d\") " pod="openshift-etcd/etcd-crc" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.845014 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"usr-local-bin\" (UniqueName: \"kubernetes.io/host-path/2139d3e2895fc6797b9c76a1b4c9886d-usr-local-bin\") pod \"etcd-crc\" (UID: \"2139d3e2895fc6797b9c76a1b4c9886d\") " pod="openshift-etcd/etcd-crc" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.845035 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-dir\" (UniqueName: \"kubernetes.io/host-path/f4b27818a5e8e43d0dc095d08835c792-audit-dir\") pod \"kube-apiserver-crc\" (UID: \"f4b27818a5e8e43d0dc095d08835c792\") " pod="openshift-kube-apiserver/kube-apiserver-crc" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.845063 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-dir\" (UniqueName: \"kubernetes.io/host-path/2139d3e2895fc6797b9c76a1b4c9886d-log-dir\") pod \"etcd-crc\" (UID: \"2139d3e2895fc6797b9c76a1b4c9886d\") " pod="openshift-etcd/etcd-crc" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.845085 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"resource-dir\" (UniqueName: \"kubernetes.io/host-path/f4b27818a5e8e43d0dc095d08835c792-resource-dir\") pod \"kube-apiserver-crc\" (UID: \"f4b27818a5e8e43d0dc095d08835c792\") " pod="openshift-kube-apiserver/kube-apiserver-crc" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.845116 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"resource-dir\" (UniqueName: \"kubernetes.io/host-path/f614b9022728cf315e60c057852e563e-resource-dir\") pod \"kube-controller-manager-crc\" (UID: \"f614b9022728cf315e60c057852e563e\") " pod="openshift-kube-controller-manager/kube-controller-manager-crc" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.845145 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"static-pod-dir\" (UniqueName: \"kubernetes.io/host-path/2139d3e2895fc6797b9c76a1b4c9886d-static-pod-dir\") pod \"etcd-crc\" (UID: \"2139d3e2895fc6797b9c76a1b4c9886d\") " pod="openshift-etcd/etcd-crc" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.845162 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"resource-dir\" (UniqueName: \"kubernetes.io/host-path/2139d3e2895fc6797b9c76a1b4c9886d-resource-dir\") pod \"etcd-crc\" (UID: \"2139d3e2895fc6797b9c76a1b4c9886d\") " pod="openshift-etcd/etcd-crc" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.845210 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert-dir\" (UniqueName: \"kubernetes.io/host-path/2139d3e2895fc6797b9c76a1b4c9886d-cert-dir\") pod \"etcd-crc\" (UID: \"2139d3e2895fc6797b9c76a1b4c9886d\") " pod="openshift-etcd/etcd-crc" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.947270 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d1b160f5dda77d281dd8e69ec8d817f9-etc-kube\") pod \"kube-rbac-proxy-crio-crc\" (UID: \"d1b160f5dda77d281dd8e69ec8d817f9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-crc" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.947390 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"data-dir\" (UniqueName: \"kubernetes.io/host-path/2139d3e2895fc6797b9c76a1b4c9886d-data-dir\") pod \"etcd-crc\" (UID: \"2139d3e2895fc6797b9c76a1b4c9886d\") " pod="openshift-etcd/etcd-crc" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.947413 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"usr-local-bin\" (UniqueName: \"kubernetes.io/host-path/2139d3e2895fc6797b9c76a1b4c9886d-usr-local-bin\") pod \"etcd-crc\" (UID: \"2139d3e2895fc6797b9c76a1b4c9886d\") " pod="openshift-etcd/etcd-crc" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.947431 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"audit-dir\" (UniqueName: \"kubernetes.io/host-path/f4b27818a5e8e43d0dc095d08835c792-audit-dir\") pod \"kube-apiserver-crc\" (UID: \"f4b27818a5e8e43d0dc095d08835c792\") " pod="openshift-kube-apiserver/kube-apiserver-crc" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.947465 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"cert-dir\" (UniqueName: \"kubernetes.io/host-path/3dcd261975c3d6b9a6ad6367fd4facd3-cert-dir\") pod \"openshift-kube-scheduler-crc\" (UID: \"3dcd261975c3d6b9a6ad6367fd4facd3\") " pod="openshift-kube-scheduler/openshift-kube-scheduler-crc" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.947487 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"resource-dir\" (UniqueName: \"kubernetes.io/host-path/f614b9022728cf315e60c057852e563e-resource-dir\") pod \"kube-controller-manager-crc\" (UID: \"f614b9022728cf315e60c057852e563e\") " pod="openshift-kube-controller-manager/kube-controller-manager-crc" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.947497 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"data-dir\" (UniqueName: \"kubernetes.io/host-path/2139d3e2895fc6797b9c76a1b4c9886d-data-dir\") pod \"etcd-crc\" (UID: \"2139d3e2895fc6797b9c76a1b4c9886d\") " pod="openshift-etcd/etcd-crc" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.947569 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d1b160f5dda77d281dd8e69ec8d817f9-etc-kube\") pod \"kube-rbac-proxy-crio-crc\" (UID: \"d1b160f5dda77d281dd8e69ec8d817f9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-crc" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.947616 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"resource-dir\" (UniqueName: \"kubernetes.io/host-path/f614b9022728cf315e60c057852e563e-resource-dir\") pod \"kube-controller-manager-crc\" (UID: \"f614b9022728cf315e60c057852e563e\") " pod="openshift-kube-controller-manager/kube-controller-manager-crc" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.947563 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"static-pod-dir\" (UniqueName: \"kubernetes.io/host-path/2139d3e2895fc6797b9c76a1b4c9886d-static-pod-dir\") pod \"etcd-crc\" (UID: \"2139d3e2895fc6797b9c76a1b4c9886d\") " pod="openshift-etcd/etcd-crc" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.947578 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"cert-dir\" (UniqueName: \"kubernetes.io/host-path/3dcd261975c3d6b9a6ad6367fd4facd3-cert-dir\") pod \"openshift-kube-scheduler-crc\" (UID: \"3dcd261975c3d6b9a6ad6367fd4facd3\") " pod="openshift-kube-scheduler/openshift-kube-scheduler-crc" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.947641 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"audit-dir\" (UniqueName: \"kubernetes.io/host-path/f4b27818a5e8e43d0dc095d08835c792-audit-dir\") pod \"kube-apiserver-crc\" (UID: \"f4b27818a5e8e43d0dc095d08835c792\") " pod="openshift-kube-apiserver/kube-apiserver-crc" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.947665 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"usr-local-bin\" (UniqueName: \"kubernetes.io/host-path/2139d3e2895fc6797b9c76a1b4c9886d-usr-local-bin\") pod \"etcd-crc\" (UID: \"2139d3e2895fc6797b9c76a1b4c9886d\") " pod="openshift-etcd/etcd-crc" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.947527 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"static-pod-dir\" (UniqueName: \"kubernetes.io/host-path/2139d3e2895fc6797b9c76a1b4c9886d-static-pod-dir\") pod \"etcd-crc\" (UID: \"2139d3e2895fc6797b9c76a1b4c9886d\") " pod="openshift-etcd/etcd-crc" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.947821 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"resource-dir\" (UniqueName: \"kubernetes.io/host-path/2139d3e2895fc6797b9c76a1b4c9886d-resource-dir\") pod \"etcd-crc\" (UID: \"2139d3e2895fc6797b9c76a1b4c9886d\") " pod="openshift-etcd/etcd-crc" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.947841 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"cert-dir\" (UniqueName: \"kubernetes.io/host-path/2139d3e2895fc6797b9c76a1b4c9886d-cert-dir\") pod \"etcd-crc\" (UID: \"2139d3e2895fc6797b9c76a1b4c9886d\") " pod="openshift-etcd/etcd-crc" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.947857 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"log-dir\" (UniqueName: \"kubernetes.io/host-path/2139d3e2895fc6797b9c76a1b4c9886d-log-dir\") pod \"etcd-crc\" (UID: \"2139d3e2895fc6797b9c76a1b4c9886d\") " pod="openshift-etcd/etcd-crc" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.947876 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"resource-dir\" (UniqueName: \"kubernetes.io/host-path/f4b27818a5e8e43d0dc095d08835c792-resource-dir\") pod \"kube-apiserver-crc\" (UID: \"f4b27818a5e8e43d0dc095d08835c792\") " pod="openshift-kube-apiserver/kube-apiserver-crc" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.947870 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"resource-dir\" (UniqueName: \"kubernetes.io/host-path/2139d3e2895fc6797b9c76a1b4c9886d-resource-dir\") pod \"etcd-crc\" (UID: \"2139d3e2895fc6797b9c76a1b4c9886d\") " pod="openshift-etcd/etcd-crc" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.947921 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"cert-dir\" (UniqueName: \"kubernetes.io/host-path/f4b27818a5e8e43d0dc095d08835c792-cert-dir\") pod \"kube-apiserver-crc\" (UID: \"f4b27818a5e8e43d0dc095d08835c792\") " pod="openshift-kube-apiserver/kube-apiserver-crc" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.947897 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"cert-dir\" (UniqueName: \"kubernetes.io/host-path/2139d3e2895fc6797b9c76a1b4c9886d-cert-dir\") pod \"etcd-crc\" (UID: \"2139d3e2895fc6797b9c76a1b4c9886d\") " pod="openshift-etcd/etcd-crc" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.947899 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"cert-dir\" (UniqueName: \"kubernetes.io/host-path/f4b27818a5e8e43d0dc095d08835c792-cert-dir\") pod \"kube-apiserver-crc\" (UID: \"f4b27818a5e8e43d0dc095d08835c792\") " pod="openshift-kube-apiserver/kube-apiserver-crc" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.947945 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"resource-dir\" (UniqueName: \"kubernetes.io/host-path/f4b27818a5e8e43d0dc095d08835c792-resource-dir\") pod \"kube-apiserver-crc\" (UID: \"f4b27818a5e8e43d0dc095d08835c792\") " pod="openshift-kube-apiserver/kube-apiserver-crc" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.947949 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"log-dir\" (UniqueName: \"kubernetes.io/host-path/2139d3e2895fc6797b9c76a1b4c9886d-log-dir\") pod \"etcd-crc\" (UID: \"2139d3e2895fc6797b9c76a1b4c9886d\") " pod="openshift-etcd/etcd-crc" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.947974 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"cert-dir\" (UniqueName: \"kubernetes.io/host-path/f614b9022728cf315e60c057852e563e-cert-dir\") pod \"kube-controller-manager-crc\" (UID: \"f614b9022728cf315e60c057852e563e\") " pod="openshift-kube-controller-manager/kube-controller-manager-crc" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.947962 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"cert-dir\" (UniqueName: \"kubernetes.io/host-path/f614b9022728cf315e60c057852e563e-cert-dir\") pod \"kube-controller-manager-crc\" (UID: \"f614b9022728cf315e60c057852e563e\") " pod="openshift-kube-controller-manager/kube-controller-manager-crc" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.948011 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"resource-dir\" (UniqueName: \"kubernetes.io/host-path/3dcd261975c3d6b9a6ad6367fd4facd3-resource-dir\") pod \"openshift-kube-scheduler-crc\" (UID: \"3dcd261975c3d6b9a6ad6367fd4facd3\") " pod="openshift-kube-scheduler/openshift-kube-scheduler-crc" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.948030 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d1b160f5dda77d281dd8e69ec8d817f9-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-crc\" (UID: \"d1b160f5dda77d281dd8e69ec8d817f9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-crc" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.948445 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"resource-dir\" (UniqueName: \"kubernetes.io/host-path/3dcd261975c3d6b9a6ad6367fd4facd3-resource-dir\") pod \"openshift-kube-scheduler-crc\" (UID: \"3dcd261975c3d6b9a6ad6367fd4facd3\") " pod="openshift-kube-scheduler/openshift-kube-scheduler-crc" Dec 03 11:20:09 crc kubenswrapper[5052]: I1203 11:20:09.948470 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d1b160f5dda77d281dd8e69ec8d817f9-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-crc\" (UID: \"d1b160f5dda77d281dd8e69ec8d817f9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-crc" Dec 03 11:20:10 crc kubenswrapper[5052]: I1203 11:20:10.043084 5052 kubelet_node_status.go:401] "Setting node annotation to enable volume controller attach/detach" Dec 03 11:20:10 crc kubenswrapper[5052]: I1203 11:20:10.045127 5052 kubelet_node_status.go:724] "Recording event message for node" node="crc" event="NodeHasSufficientMemory" Dec 03 11:20:10 crc kubenswrapper[5052]: I1203 11:20:10.045179 5052 kubelet_node_status.go:724] "Recording event message for node" node="crc" event="NodeHasNoDiskPressure" Dec 03 11:20:10 crc kubenswrapper[5052]: I1203 11:20:10.045194 5052 kubelet_node_status.go:724] "Recording event message for node" node="crc" event="NodeHasSufficientPID" Dec 03 11:20:10 crc kubenswrapper[5052]: I1203 11:20:10.045227 5052 kubelet_node_status.go:76] "Attempting to register node" node="crc" Dec 03 11:20:10 crc kubenswrapper[5052]: E1203 11:20:10.046011 5052 kubelet_node_status.go:99] "Unable to register node with API server" err="Post \"https://api-int.crc.testing:6443/api/v1/nodes\": dial tcp 38.129.56.252:6443: connect: connection refused" node="crc" Dec 03 11:20:10 crc kubenswrapper[5052]: I1203 11:20:10.110448 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-etcd/etcd-crc" Dec 03 11:20:10 crc kubenswrapper[5052]: I1203 11:20:10.126777 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-apiserver/kube-apiserver-crc" Dec 03 11:20:10 crc kubenswrapper[5052]: W1203 11:20:10.133976 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2139d3e2895fc6797b9c76a1b4c9886d.slice/crio-5b0a5bf81f11b3d111e12bf6747dd4a96a2cb3f21339298555fddec65732ba6a WatchSource:0}: Error finding container 5b0a5bf81f11b3d111e12bf6747dd4a96a2cb3f21339298555fddec65732ba6a: Status 404 returned error can't find the container with id 5b0a5bf81f11b3d111e12bf6747dd4a96a2cb3f21339298555fddec65732ba6a Dec 03 11:20:10 crc kubenswrapper[5052]: W1203 11:20:10.146654 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4b27818a5e8e43d0dc095d08835c792.slice/crio-e47abbbb4f85b3f6bec279aa17ceb7bf11db231ebfea2862405dfbd3b5701290 WatchSource:0}: Error finding container e47abbbb4f85b3f6bec279aa17ceb7bf11db231ebfea2862405dfbd3b5701290: Status 404 returned error can't find the container with id e47abbbb4f85b3f6bec279aa17ceb7bf11db231ebfea2862405dfbd3b5701290 Dec 03 11:20:10 crc kubenswrapper[5052]: I1203 11:20:10.149720 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-controller-manager/kube-controller-manager-crc" Dec 03 11:20:10 crc kubenswrapper[5052]: W1203 11:20:10.161590 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf614b9022728cf315e60c057852e563e.slice/crio-a8fa1076d6e416dac34df346f9b6ba87db01e82864fa92aec3e577f048095dc1 WatchSource:0}: Error finding container a8fa1076d6e416dac34df346f9b6ba87db01e82864fa92aec3e577f048095dc1: Status 404 returned error can't find the container with id a8fa1076d6e416dac34df346f9b6ba87db01e82864fa92aec3e577f048095dc1 Dec 03 11:20:10 crc kubenswrapper[5052]: I1203 11:20:10.164910 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-scheduler/openshift-kube-scheduler-crc" Dec 03 11:20:10 crc kubenswrapper[5052]: I1203 11:20:10.172076 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-crc" Dec 03 11:20:10 crc kubenswrapper[5052]: W1203 11:20:10.178449 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3dcd261975c3d6b9a6ad6367fd4facd3.slice/crio-6a0c049958928774d54bb4b576a3dffd14c08a19a8e85ae47666c2d6b300c27e WatchSource:0}: Error finding container 6a0c049958928774d54bb4b576a3dffd14c08a19a8e85ae47666c2d6b300c27e: Status 404 returned error can't find the container with id 6a0c049958928774d54bb4b576a3dffd14c08a19a8e85ae47666c2d6b300c27e Dec 03 11:20:10 crc kubenswrapper[5052]: E1203 11:20:10.222965 5052 controller.go:145] "Failed to ensure lease exists, will retry" err="Get \"https://api-int.crc.testing:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/crc?timeout=10s\": dial tcp 38.129.56.252:6443: connect: connection refused" interval="800ms" Dec 03 11:20:10 crc kubenswrapper[5052]: I1203 11:20:10.446492 5052 kubelet_node_status.go:401] "Setting node annotation to enable volume controller attach/detach" Dec 03 11:20:10 crc kubenswrapper[5052]: I1203 11:20:10.448464 5052 kubelet_node_status.go:724] "Recording event message for node" node="crc" event="NodeHasSufficientMemory" Dec 03 11:20:10 crc kubenswrapper[5052]: I1203 11:20:10.448537 5052 kubelet_node_status.go:724] "Recording event message for node" node="crc" event="NodeHasNoDiskPressure" Dec 03 11:20:10 crc kubenswrapper[5052]: I1203 11:20:10.448550 5052 kubelet_node_status.go:724] "Recording event message for node" node="crc" event="NodeHasSufficientPID" Dec 03 11:20:10 crc kubenswrapper[5052]: I1203 11:20:10.448581 5052 kubelet_node_status.go:76] "Attempting to register node" node="crc" Dec 03 11:20:10 crc kubenswrapper[5052]: E1203 11:20:10.449276 5052 kubelet_node_status.go:99] "Unable to register node with API server" err="Post \"https://api-int.crc.testing:6443/api/v1/nodes\": dial tcp 38.129.56.252:6443: connect: connection refused" node="crc" Dec 03 11:20:10 crc kubenswrapper[5052]: W1203 11:20:10.555146 5052 reflector.go:561] k8s.io/client-go/informers/factory.go:160: failed to list *v1.Service: Get "https://api-int.crc.testing:6443/api/v1/services?fieldSelector=spec.clusterIP%21%3DNone&limit=500&resourceVersion=0": dial tcp 38.129.56.252:6443: connect: connection refused Dec 03 11:20:10 crc kubenswrapper[5052]: E1203 11:20:10.555273 5052 reflector.go:158] "Unhandled Error" err="k8s.io/client-go/informers/factory.go:160: Failed to watch *v1.Service: failed to list *v1.Service: Get \"https://api-int.crc.testing:6443/api/v1/services?fieldSelector=spec.clusterIP%21%3DNone&limit=500&resourceVersion=0\": dial tcp 38.129.56.252:6443: connect: connection refused" logger="UnhandledError" Dec 03 11:20:10 crc kubenswrapper[5052]: I1203 11:20:10.604782 5052 csi_plugin.go:884] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.crc.testing:6443/apis/storage.k8s.io/v1/csinodes/crc?resourceVersion=0": dial tcp 38.129.56.252:6443: connect: connection refused Dec 03 11:20:10 crc kubenswrapper[5052]: I1203 11:20:10.679101 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-kube-controller-manager/kube-controller-manager-crc" event={"ID":"f614b9022728cf315e60c057852e563e","Type":"ContainerStarted","Data":"a8fa1076d6e416dac34df346f9b6ba87db01e82864fa92aec3e577f048095dc1"} Dec 03 11:20:10 crc kubenswrapper[5052]: I1203 11:20:10.680273 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-kube-apiserver/kube-apiserver-crc" event={"ID":"f4b27818a5e8e43d0dc095d08835c792","Type":"ContainerStarted","Data":"e47abbbb4f85b3f6bec279aa17ceb7bf11db231ebfea2862405dfbd3b5701290"} Dec 03 11:20:10 crc kubenswrapper[5052]: I1203 11:20:10.681529 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-etcd/etcd-crc" event={"ID":"2139d3e2895fc6797b9c76a1b4c9886d","Type":"ContainerStarted","Data":"5b0a5bf81f11b3d111e12bf6747dd4a96a2cb3f21339298555fddec65732ba6a"} Dec 03 11:20:10 crc kubenswrapper[5052]: I1203 11:20:10.685490 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-crc" event={"ID":"d1b160f5dda77d281dd8e69ec8d817f9","Type":"ContainerStarted","Data":"df1235d895b25b059d175fd77cb6458b76233b190a3c4abb3ed91aa44de244f7"} Dec 03 11:20:10 crc kubenswrapper[5052]: I1203 11:20:10.686353 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-kube-scheduler/openshift-kube-scheduler-crc" event={"ID":"3dcd261975c3d6b9a6ad6367fd4facd3","Type":"ContainerStarted","Data":"6a0c049958928774d54bb4b576a3dffd14c08a19a8e85ae47666c2d6b300c27e"} Dec 03 11:20:10 crc kubenswrapper[5052]: W1203 11:20:10.936915 5052 reflector.go:561] k8s.io/client-go/informers/factory.go:160: failed to list *v1.Node: Get "https://api-int.crc.testing:6443/api/v1/nodes?fieldSelector=metadata.name%3Dcrc&limit=500&resourceVersion=0": dial tcp 38.129.56.252:6443: connect: connection refused Dec 03 11:20:10 crc kubenswrapper[5052]: E1203 11:20:10.937028 5052 reflector.go:158] "Unhandled Error" err="k8s.io/client-go/informers/factory.go:160: Failed to watch *v1.Node: failed to list *v1.Node: Get \"https://api-int.crc.testing:6443/api/v1/nodes?fieldSelector=metadata.name%3Dcrc&limit=500&resourceVersion=0\": dial tcp 38.129.56.252:6443: connect: connection refused" logger="UnhandledError" Dec 03 11:20:11 crc kubenswrapper[5052]: W1203 11:20:11.018954 5052 reflector.go:561] k8s.io/client-go/informers/factory.go:160: failed to list *v1.CSIDriver: Get "https://api-int.crc.testing:6443/apis/storage.k8s.io/v1/csidrivers?limit=500&resourceVersion=0": dial tcp 38.129.56.252:6443: connect: connection refused Dec 03 11:20:11 crc kubenswrapper[5052]: E1203 11:20:11.019146 5052 reflector.go:158] "Unhandled Error" err="k8s.io/client-go/informers/factory.go:160: Failed to watch *v1.CSIDriver: failed to list *v1.CSIDriver: Get \"https://api-int.crc.testing:6443/apis/storage.k8s.io/v1/csidrivers?limit=500&resourceVersion=0\": dial tcp 38.129.56.252:6443: connect: connection refused" logger="UnhandledError" Dec 03 11:20:11 crc kubenswrapper[5052]: E1203 11:20:11.024703 5052 controller.go:145] "Failed to ensure lease exists, will retry" err="Get \"https://api-int.crc.testing:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/crc?timeout=10s\": dial tcp 38.129.56.252:6443: connect: connection refused" interval="1.6s" Dec 03 11:20:11 crc kubenswrapper[5052]: W1203 11:20:11.242984 5052 reflector.go:561] k8s.io/client-go/informers/factory.go:160: failed to list *v1.RuntimeClass: Get "https://api-int.crc.testing:6443/apis/node.k8s.io/v1/runtimeclasses?limit=500&resourceVersion=0": dial tcp 38.129.56.252:6443: connect: connection refused Dec 03 11:20:11 crc kubenswrapper[5052]: E1203 11:20:11.243619 5052 reflector.go:158] "Unhandled Error" err="k8s.io/client-go/informers/factory.go:160: Failed to watch *v1.RuntimeClass: failed to list *v1.RuntimeClass: Get \"https://api-int.crc.testing:6443/apis/node.k8s.io/v1/runtimeclasses?limit=500&resourceVersion=0\": dial tcp 38.129.56.252:6443: connect: connection refused" logger="UnhandledError" Dec 03 11:20:11 crc kubenswrapper[5052]: I1203 11:20:11.250044 5052 kubelet_node_status.go:401] "Setting node annotation to enable volume controller attach/detach" Dec 03 11:20:11 crc kubenswrapper[5052]: I1203 11:20:11.252261 5052 kubelet_node_status.go:724] "Recording event message for node" node="crc" event="NodeHasSufficientMemory" Dec 03 11:20:11 crc kubenswrapper[5052]: I1203 11:20:11.252304 5052 kubelet_node_status.go:724] "Recording event message for node" node="crc" event="NodeHasNoDiskPressure" Dec 03 11:20:11 crc kubenswrapper[5052]: I1203 11:20:11.252313 5052 kubelet_node_status.go:724] "Recording event message for node" node="crc" event="NodeHasSufficientPID" Dec 03 11:20:11 crc kubenswrapper[5052]: I1203 11:20:11.252336 5052 kubelet_node_status.go:76] "Attempting to register node" node="crc" Dec 03 11:20:11 crc kubenswrapper[5052]: E1203 11:20:11.252925 5052 kubelet_node_status.go:99] "Unable to register node with API server" err="Post \"https://api-int.crc.testing:6443/api/v1/nodes\": dial tcp 38.129.56.252:6443: connect: connection refused" node="crc" Dec 03 11:20:11 crc kubenswrapper[5052]: I1203 11:20:11.604487 5052 csi_plugin.go:884] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.crc.testing:6443/apis/storage.k8s.io/v1/csinodes/crc?resourceVersion=0": dial tcp 38.129.56.252:6443: connect: connection refused Dec 03 11:20:11 crc kubenswrapper[5052]: I1203 11:20:11.690603 5052 generic.go:334] "Generic (PLEG): container finished" podID="3dcd261975c3d6b9a6ad6367fd4facd3" containerID="502437c6e28a418a17bb8792d7e5cae005676c3b699c071d05fa5b0b3bc2681e" exitCode=0 Dec 03 11:20:11 crc kubenswrapper[5052]: I1203 11:20:11.690745 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-kube-scheduler/openshift-kube-scheduler-crc" event={"ID":"3dcd261975c3d6b9a6ad6367fd4facd3","Type":"ContainerDied","Data":"502437c6e28a418a17bb8792d7e5cae005676c3b699c071d05fa5b0b3bc2681e"} Dec 03 11:20:11 crc kubenswrapper[5052]: I1203 11:20:11.690830 5052 kubelet_node_status.go:401] "Setting node annotation to enable volume controller attach/detach" Dec 03 11:20:11 crc kubenswrapper[5052]: I1203 11:20:11.691846 5052 kubelet_node_status.go:724] "Recording event message for node" node="crc" event="NodeHasSufficientMemory" Dec 03 11:20:11 crc kubenswrapper[5052]: I1203 11:20:11.691879 5052 kubelet_node_status.go:724] "Recording event message for node" node="crc" event="NodeHasNoDiskPressure" Dec 03 11:20:11 crc kubenswrapper[5052]: I1203 11:20:11.691889 5052 kubelet_node_status.go:724] "Recording event message for node" node="crc" event="NodeHasSufficientPID" Dec 03 11:20:11 crc kubenswrapper[5052]: I1203 11:20:11.692632 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-kube-controller-manager/kube-controller-manager-crc" event={"ID":"f614b9022728cf315e60c057852e563e","Type":"ContainerStarted","Data":"f5d23c6160b5790e9272bd5ae83472267f4d1f60b67e0425a6bc2f5c1f9d1fb7"} Dec 03 11:20:11 crc kubenswrapper[5052]: I1203 11:20:11.694625 5052 generic.go:334] "Generic (PLEG): container finished" podID="f4b27818a5e8e43d0dc095d08835c792" containerID="7182c75773f069c72f87c39136222fd4a91b49add27352a1969645413e670f01" exitCode=0 Dec 03 11:20:11 crc kubenswrapper[5052]: I1203 11:20:11.694706 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-kube-apiserver/kube-apiserver-crc" event={"ID":"f4b27818a5e8e43d0dc095d08835c792","Type":"ContainerDied","Data":"7182c75773f069c72f87c39136222fd4a91b49add27352a1969645413e670f01"} Dec 03 11:20:11 crc kubenswrapper[5052]: I1203 11:20:11.694737 5052 kubelet_node_status.go:401] "Setting node annotation to enable volume controller attach/detach" Dec 03 11:20:11 crc kubenswrapper[5052]: I1203 11:20:11.695779 5052 kubelet_node_status.go:724] "Recording event message for node" node="crc" event="NodeHasSufficientMemory" Dec 03 11:20:11 crc kubenswrapper[5052]: I1203 11:20:11.695809 5052 kubelet_node_status.go:724] "Recording event message for node" node="crc" event="NodeHasNoDiskPressure" Dec 03 11:20:11 crc kubenswrapper[5052]: I1203 11:20:11.695821 5052 kubelet_node_status.go:724] "Recording event message for node" node="crc" event="NodeHasSufficientPID" Dec 03 11:20:11 crc kubenswrapper[5052]: I1203 11:20:11.696460 5052 generic.go:334] "Generic (PLEG): container finished" podID="2139d3e2895fc6797b9c76a1b4c9886d" containerID="7d7c61963a9597f2c9c159d743284313820acece17152be4d829b0e3d02434c1" exitCode=0 Dec 03 11:20:11 crc kubenswrapper[5052]: I1203 11:20:11.696549 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-etcd/etcd-crc" event={"ID":"2139d3e2895fc6797b9c76a1b4c9886d","Type":"ContainerDied","Data":"7d7c61963a9597f2c9c159d743284313820acece17152be4d829b0e3d02434c1"} Dec 03 11:20:11 crc kubenswrapper[5052]: I1203 11:20:11.696586 5052 kubelet_node_status.go:401] "Setting node annotation to enable volume controller attach/detach" Dec 03 11:20:11 crc kubenswrapper[5052]: I1203 11:20:11.697578 5052 kubelet_node_status.go:401] "Setting node annotation to enable volume controller attach/detach" Dec 03 11:20:11 crc kubenswrapper[5052]: I1203 11:20:11.697621 5052 kubelet_node_status.go:724] "Recording event message for node" node="crc" event="NodeHasSufficientMemory" Dec 03 11:20:11 crc kubenswrapper[5052]: I1203 11:20:11.697658 5052 kubelet_node_status.go:724] "Recording event message for node" node="crc" event="NodeHasNoDiskPressure" Dec 03 11:20:11 crc kubenswrapper[5052]: I1203 11:20:11.697673 5052 kubelet_node_status.go:724] "Recording event message for node" node="crc" event="NodeHasSufficientPID" Dec 03 11:20:11 crc kubenswrapper[5052]: I1203 11:20:11.698072 5052 generic.go:334] "Generic (PLEG): container finished" podID="d1b160f5dda77d281dd8e69ec8d817f9" containerID="ad02086a924ffe16f78515df78cdfd1f11a4661672c32c02e9e3fb4d4562d8df" exitCode=0 Dec 03 11:20:11 crc kubenswrapper[5052]: I1203 11:20:11.698121 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-crc" event={"ID":"d1b160f5dda77d281dd8e69ec8d817f9","Type":"ContainerDied","Data":"ad02086a924ffe16f78515df78cdfd1f11a4661672c32c02e9e3fb4d4562d8df"} Dec 03 11:20:11 crc kubenswrapper[5052]: I1203 11:20:11.698209 5052 kubelet_node_status.go:401] "Setting node annotation to enable volume controller attach/detach" Dec 03 11:20:11 crc kubenswrapper[5052]: I1203 11:20:11.699025 5052 kubelet_node_status.go:724] "Recording event message for node" node="crc" event="NodeHasSufficientMemory" Dec 03 11:20:11 crc kubenswrapper[5052]: I1203 11:20:11.699024 5052 kubelet_node_status.go:724] "Recording event message for node" node="crc" event="NodeHasSufficientMemory" Dec 03 11:20:11 crc kubenswrapper[5052]: I1203 11:20:11.699057 5052 kubelet_node_status.go:724] "Recording event message for node" node="crc" event="NodeHasNoDiskPressure" Dec 03 11:20:11 crc kubenswrapper[5052]: I1203 11:20:11.699067 5052 kubelet_node_status.go:724] "Recording event message for node" node="crc" event="NodeHasNoDiskPressure" Dec 03 11:20:11 crc kubenswrapper[5052]: I1203 11:20:11.699082 5052 kubelet_node_status.go:724] "Recording event message for node" node="crc" event="NodeHasSufficientPID" Dec 03 11:20:11 crc kubenswrapper[5052]: I1203 11:20:11.699070 5052 kubelet_node_status.go:724] "Recording event message for node" node="crc" event="NodeHasSufficientPID" Dec 03 11:20:12 crc kubenswrapper[5052]: I1203 11:20:12.605161 5052 csi_plugin.go:884] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.crc.testing:6443/apis/storage.k8s.io/v1/csinodes/crc?resourceVersion=0": dial tcp 38.129.56.252:6443: connect: connection refused Dec 03 11:20:12 crc kubenswrapper[5052]: E1203 11:20:12.626209 5052 controller.go:145] "Failed to ensure lease exists, will retry" err="Get \"https://api-int.crc.testing:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/crc?timeout=10s\": dial tcp 38.129.56.252:6443: connect: connection refused" interval="3.2s" Dec 03 11:20:12 crc kubenswrapper[5052]: I1203 11:20:12.704274 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-crc" event={"ID":"d1b160f5dda77d281dd8e69ec8d817f9","Type":"ContainerStarted","Data":"ce1d9a79582f9ec172300f78a9cdbc6658314603ef7d6413da353765a2d02f9f"} Dec 03 11:20:12 crc kubenswrapper[5052]: I1203 11:20:12.704375 5052 kubelet_node_status.go:401] "Setting node annotation to enable volume controller attach/detach" Dec 03 11:20:12 crc kubenswrapper[5052]: I1203 11:20:12.706503 5052 kubelet_node_status.go:724] "Recording event message for node" node="crc" event="NodeHasSufficientMemory" Dec 03 11:20:12 crc kubenswrapper[5052]: I1203 11:20:12.706847 5052 kubelet_node_status.go:724] "Recording event message for node" node="crc" event="NodeHasNoDiskPressure" Dec 03 11:20:12 crc kubenswrapper[5052]: I1203 11:20:12.706864 5052 kubelet_node_status.go:724] "Recording event message for node" node="crc" event="NodeHasSufficientPID" Dec 03 11:20:12 crc kubenswrapper[5052]: I1203 11:20:12.707697 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-kube-scheduler/openshift-kube-scheduler-crc" event={"ID":"3dcd261975c3d6b9a6ad6367fd4facd3","Type":"ContainerStarted","Data":"14fc78caa84c30c4b1e819d6da69b7e8515442f67c83cae420c24acccb9d4929"} Dec 03 11:20:12 crc kubenswrapper[5052]: I1203 11:20:12.707750 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-kube-scheduler/openshift-kube-scheduler-crc" event={"ID":"3dcd261975c3d6b9a6ad6367fd4facd3","Type":"ContainerStarted","Data":"8aead3aab1d1107dd7f1b5b343cf60cbfc3fac5ae853b074bda3f3b67caccd15"} Dec 03 11:20:12 crc kubenswrapper[5052]: I1203 11:20:12.720109 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-kube-controller-manager/kube-controller-manager-crc" event={"ID":"f614b9022728cf315e60c057852e563e","Type":"ContainerStarted","Data":"d05df6945d11d2abe73005efafe80220497a339008d4c28f7923c2dc7ac4a150"} Dec 03 11:20:12 crc kubenswrapper[5052]: I1203 11:20:12.720197 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-kube-controller-manager/kube-controller-manager-crc" event={"ID":"f614b9022728cf315e60c057852e563e","Type":"ContainerStarted","Data":"37485762ea1808cd3794c9641a4ead224b107e6c64c295de237f8ffd50aba4f4"} Dec 03 11:20:12 crc kubenswrapper[5052]: I1203 11:20:12.724768 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-kube-apiserver/kube-apiserver-crc" event={"ID":"f4b27818a5e8e43d0dc095d08835c792","Type":"ContainerStarted","Data":"8ceedd21ae3833d77fd440537f30434ee57f087fbefb2357b46231d3f778ad2a"} Dec 03 11:20:12 crc kubenswrapper[5052]: I1203 11:20:12.724829 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-kube-apiserver/kube-apiserver-crc" event={"ID":"f4b27818a5e8e43d0dc095d08835c792","Type":"ContainerStarted","Data":"09256543a9aa86da8bec948be040d36b6e4986b9232091d2638bb1fead5740a9"} Dec 03 11:20:12 crc kubenswrapper[5052]: I1203 11:20:12.729631 5052 generic.go:334] "Generic (PLEG): container finished" podID="2139d3e2895fc6797b9c76a1b4c9886d" containerID="e980e02c63b59597153749cc2ed54da5925e88a6a0ed2293f4075ff4b46bbe44" exitCode=0 Dec 03 11:20:12 crc kubenswrapper[5052]: I1203 11:20:12.729683 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-etcd/etcd-crc" event={"ID":"2139d3e2895fc6797b9c76a1b4c9886d","Type":"ContainerDied","Data":"e980e02c63b59597153749cc2ed54da5925e88a6a0ed2293f4075ff4b46bbe44"} Dec 03 11:20:12 crc kubenswrapper[5052]: I1203 11:20:12.729850 5052 kubelet_node_status.go:401] "Setting node annotation to enable volume controller attach/detach" Dec 03 11:20:12 crc kubenswrapper[5052]: I1203 11:20:12.731641 5052 kubelet_node_status.go:724] "Recording event message for node" node="crc" event="NodeHasSufficientMemory" Dec 03 11:20:12 crc kubenswrapper[5052]: I1203 11:20:12.731685 5052 kubelet_node_status.go:724] "Recording event message for node" node="crc" event="NodeHasNoDiskPressure" Dec 03 11:20:12 crc kubenswrapper[5052]: I1203 11:20:12.731697 5052 kubelet_node_status.go:724] "Recording event message for node" node="crc" event="NodeHasSufficientPID" Dec 03 11:20:12 crc kubenswrapper[5052]: I1203 11:20:12.853999 5052 kubelet_node_status.go:401] "Setting node annotation to enable volume controller attach/detach" Dec 03 11:20:12 crc kubenswrapper[5052]: I1203 11:20:12.855420 5052 kubelet_node_status.go:724] "Recording event message for node" node="crc" event="NodeHasSufficientMemory" Dec 03 11:20:12 crc kubenswrapper[5052]: I1203 11:20:12.855464 5052 kubelet_node_status.go:724] "Recording event message for node" node="crc" event="NodeHasNoDiskPressure" Dec 03 11:20:12 crc kubenswrapper[5052]: I1203 11:20:12.855478 5052 kubelet_node_status.go:724] "Recording event message for node" node="crc" event="NodeHasSufficientPID" Dec 03 11:20:12 crc kubenswrapper[5052]: I1203 11:20:12.855530 5052 kubelet_node_status.go:76] "Attempting to register node" node="crc" Dec 03 11:20:12 crc kubenswrapper[5052]: E1203 11:20:12.856180 5052 kubelet_node_status.go:99] "Unable to register node with API server" err="Post \"https://api-int.crc.testing:6443/api/v1/nodes\": dial tcp 38.129.56.252:6443: connect: connection refused" node="crc" Dec 03 11:20:13 crc kubenswrapper[5052]: W1203 11:20:13.228123 5052 reflector.go:561] k8s.io/client-go/informers/factory.go:160: failed to list *v1.RuntimeClass: Get "https://api-int.crc.testing:6443/apis/node.k8s.io/v1/runtimeclasses?limit=500&resourceVersion=0": dial tcp 38.129.56.252:6443: connect: connection refused Dec 03 11:20:13 crc kubenswrapper[5052]: E1203 11:20:13.228270 5052 reflector.go:158] "Unhandled Error" err="k8s.io/client-go/informers/factory.go:160: Failed to watch *v1.RuntimeClass: failed to list *v1.RuntimeClass: Get \"https://api-int.crc.testing:6443/apis/node.k8s.io/v1/runtimeclasses?limit=500&resourceVersion=0\": dial tcp 38.129.56.252:6443: connect: connection refused" logger="UnhandledError" Dec 03 11:20:13 crc kubenswrapper[5052]: W1203 11:20:13.388746 5052 reflector.go:561] k8s.io/client-go/informers/factory.go:160: failed to list *v1.Node: Get "https://api-int.crc.testing:6443/api/v1/nodes?fieldSelector=metadata.name%3Dcrc&limit=500&resourceVersion=0": dial tcp 38.129.56.252:6443: connect: connection refused Dec 03 11:20:13 crc kubenswrapper[5052]: E1203 11:20:13.388852 5052 reflector.go:158] "Unhandled Error" err="k8s.io/client-go/informers/factory.go:160: Failed to watch *v1.Node: failed to list *v1.Node: Get \"https://api-int.crc.testing:6443/api/v1/nodes?fieldSelector=metadata.name%3Dcrc&limit=500&resourceVersion=0\": dial tcp 38.129.56.252:6443: connect: connection refused" logger="UnhandledError" Dec 03 11:20:13 crc kubenswrapper[5052]: E1203 11:20:13.541390 5052 event.go:368] "Unable to write event (may retry after sleeping)" err="Post \"https://api-int.crc.testing:6443/api/v1/namespaces/default/events\": dial tcp 38.129.56.252:6443: connect: connection refused" event="&Event{ObjectMeta:{crc.187db09b31ec0397 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:crc,UID:crc,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:crc,},FirstTimestamp:2025-12-03 11:20:09.603072919 +0000 UTC m=+0.191314673,LastTimestamp:2025-12-03 11:20:09.603072919 +0000 UTC m=+0.191314673,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:crc,}" Dec 03 11:20:13 crc kubenswrapper[5052]: W1203 11:20:13.600376 5052 reflector.go:561] k8s.io/client-go/informers/factory.go:160: failed to list *v1.Service: Get "https://api-int.crc.testing:6443/api/v1/services?fieldSelector=spec.clusterIP%21%3DNone&limit=500&resourceVersion=0": dial tcp 38.129.56.252:6443: connect: connection refused Dec 03 11:20:13 crc kubenswrapper[5052]: E1203 11:20:13.600569 5052 reflector.go:158] "Unhandled Error" err="k8s.io/client-go/informers/factory.go:160: Failed to watch *v1.Service: failed to list *v1.Service: Get \"https://api-int.crc.testing:6443/api/v1/services?fieldSelector=spec.clusterIP%21%3DNone&limit=500&resourceVersion=0\": dial tcp 38.129.56.252:6443: connect: connection refused" logger="UnhandledError" Dec 03 11:20:13 crc kubenswrapper[5052]: I1203 11:20:13.604478 5052 csi_plugin.go:884] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.crc.testing:6443/apis/storage.k8s.io/v1/csinodes/crc?resourceVersion=0": dial tcp 38.129.56.252:6443: connect: connection refused Dec 03 11:20:13 crc kubenswrapper[5052]: I1203 11:20:13.732378 5052 kubelet_node_status.go:401] "Setting node annotation to enable volume controller attach/detach" Dec 03 11:20:13 crc kubenswrapper[5052]: I1203 11:20:13.733438 5052 kubelet_node_status.go:724] "Recording event message for node" node="crc" event="NodeHasSufficientMemory" Dec 03 11:20:13 crc kubenswrapper[5052]: I1203 11:20:13.733527 5052 kubelet_node_status.go:724] "Recording event message for node" node="crc" event="NodeHasNoDiskPressure" Dec 03 11:20:13 crc kubenswrapper[5052]: I1203 11:20:13.733538 5052 kubelet_node_status.go:724] "Recording event message for node" node="crc" event="NodeHasSufficientPID" Dec 03 11:20:13 crc kubenswrapper[5052]: W1203 11:20:13.946057 5052 reflector.go:561] k8s.io/client-go/informers/factory.go:160: failed to list *v1.CSIDriver: Get "https://api-int.crc.testing:6443/apis/storage.k8s.io/v1/csidrivers?limit=500&resourceVersion=0": dial tcp 38.129.56.252:6443: connect: connection refused Dec 03 11:20:13 crc kubenswrapper[5052]: E1203 11:20:13.946159 5052 reflector.go:158] "Unhandled Error" err="k8s.io/client-go/informers/factory.go:160: Failed to watch *v1.CSIDriver: failed to list *v1.CSIDriver: Get \"https://api-int.crc.testing:6443/apis/storage.k8s.io/v1/csidrivers?limit=500&resourceVersion=0\": dial tcp 38.129.56.252:6443: connect: connection refused" logger="UnhandledError" Dec 03 11:20:14 crc kubenswrapper[5052]: I1203 11:20:14.604877 5052 csi_plugin.go:884] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.crc.testing:6443/apis/storage.k8s.io/v1/csinodes/crc?resourceVersion=0": dial tcp 38.129.56.252:6443: connect: connection refused Dec 03 11:20:14 crc kubenswrapper[5052]: I1203 11:20:14.740742 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-kube-apiserver/kube-apiserver-crc" event={"ID":"f4b27818a5e8e43d0dc095d08835c792","Type":"ContainerStarted","Data":"15abb0d7fd5884519ed5ac454430dbb58f71ed1f459e5ed30f324f70befd243e"} Dec 03 11:20:15 crc kubenswrapper[5052]: I1203 11:20:15.746216 5052 generic.go:334] "Generic (PLEG): container finished" podID="2139d3e2895fc6797b9c76a1b4c9886d" containerID="6a0411ec57e822071155dd66aa1e30dcab88df41e229cf75208468e5dcf90358" exitCode=0 Dec 03 11:20:15 crc kubenswrapper[5052]: I1203 11:20:15.746305 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-etcd/etcd-crc" event={"ID":"2139d3e2895fc6797b9c76a1b4c9886d","Type":"ContainerDied","Data":"6a0411ec57e822071155dd66aa1e30dcab88df41e229cf75208468e5dcf90358"} Dec 03 11:20:15 crc kubenswrapper[5052]: I1203 11:20:15.746343 5052 kubelet_node_status.go:401] "Setting node annotation to enable volume controller attach/detach" Dec 03 11:20:15 crc kubenswrapper[5052]: I1203 11:20:15.747556 5052 kubelet_node_status.go:724] "Recording event message for node" node="crc" event="NodeHasSufficientMemory" Dec 03 11:20:15 crc kubenswrapper[5052]: I1203 11:20:15.747588 5052 kubelet_node_status.go:724] "Recording event message for node" node="crc" event="NodeHasNoDiskPressure" Dec 03 11:20:15 crc kubenswrapper[5052]: I1203 11:20:15.747598 5052 kubelet_node_status.go:724] "Recording event message for node" node="crc" event="NodeHasSufficientPID" Dec 03 11:20:15 crc kubenswrapper[5052]: I1203 11:20:15.750323 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-kube-scheduler/openshift-kube-scheduler-crc" event={"ID":"3dcd261975c3d6b9a6ad6367fd4facd3","Type":"ContainerStarted","Data":"d4d9c6be3d9b94b18f50d6480e2a0b11dc0cda5e7e295bad290d49233ebf2dda"} Dec 03 11:20:15 crc kubenswrapper[5052]: I1203 11:20:15.750374 5052 kubelet_node_status.go:401] "Setting node annotation to enable volume controller attach/detach" Dec 03 11:20:15 crc kubenswrapper[5052]: I1203 11:20:15.751230 5052 kubelet_node_status.go:724] "Recording event message for node" node="crc" event="NodeHasSufficientMemory" Dec 03 11:20:15 crc kubenswrapper[5052]: I1203 11:20:15.751270 5052 kubelet_node_status.go:724] "Recording event message for node" node="crc" event="NodeHasNoDiskPressure" Dec 03 11:20:15 crc kubenswrapper[5052]: I1203 11:20:15.751287 5052 kubelet_node_status.go:724] "Recording event message for node" node="crc" event="NodeHasSufficientPID" Dec 03 11:20:15 crc kubenswrapper[5052]: I1203 11:20:15.754063 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-kube-controller-manager/kube-controller-manager-crc" event={"ID":"f614b9022728cf315e60c057852e563e","Type":"ContainerStarted","Data":"8df2aa2470046bba37b41cb301e96f52b09b83124384b5dbc4942cd0bfeef1c6"} Dec 03 11:20:15 crc kubenswrapper[5052]: I1203 11:20:15.754112 5052 kubelet_node_status.go:401] "Setting node annotation to enable volume controller attach/detach" Dec 03 11:20:15 crc kubenswrapper[5052]: I1203 11:20:15.755595 5052 kubelet_node_status.go:724] "Recording event message for node" node="crc" event="NodeHasSufficientMemory" Dec 03 11:20:15 crc kubenswrapper[5052]: I1203 11:20:15.755636 5052 kubelet_node_status.go:724] "Recording event message for node" node="crc" event="NodeHasNoDiskPressure" Dec 03 11:20:15 crc kubenswrapper[5052]: I1203 11:20:15.755650 5052 kubelet_node_status.go:724] "Recording event message for node" node="crc" event="NodeHasSufficientPID" Dec 03 11:20:15 crc kubenswrapper[5052]: I1203 11:20:15.758463 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-kube-apiserver/kube-apiserver-crc" event={"ID":"f4b27818a5e8e43d0dc095d08835c792","Type":"ContainerStarted","Data":"b739f84524d09b27f6e052e21df8be3870a2136665704232f5a365d8a5bf41f5"} Dec 03 11:20:15 crc kubenswrapper[5052]: I1203 11:20:15.758527 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-kube-apiserver/kube-apiserver-crc" event={"ID":"f4b27818a5e8e43d0dc095d08835c792","Type":"ContainerStarted","Data":"22b8cb6aa04337d1420f97d3e128fce2b29a725ca6e5d157adeb1ba78f00073f"} Dec 03 11:20:15 crc kubenswrapper[5052]: I1203 11:20:15.758567 5052 kubelet_node_status.go:401] "Setting node annotation to enable volume controller attach/detach" Dec 03 11:20:15 crc kubenswrapper[5052]: I1203 11:20:15.759944 5052 kubelet_node_status.go:724] "Recording event message for node" node="crc" event="NodeHasSufficientMemory" Dec 03 11:20:15 crc kubenswrapper[5052]: I1203 11:20:15.759993 5052 kubelet_node_status.go:724] "Recording event message for node" node="crc" event="NodeHasNoDiskPressure" Dec 03 11:20:15 crc kubenswrapper[5052]: I1203 11:20:15.760009 5052 kubelet_node_status.go:724] "Recording event message for node" node="crc" event="NodeHasSufficientPID" Dec 03 11:20:15 crc kubenswrapper[5052]: I1203 11:20:15.802495 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openshift-kube-controller-manager/kube-controller-manager-crc" Dec 03 11:20:16 crc kubenswrapper[5052]: I1203 11:20:16.056873 5052 kubelet_node_status.go:401] "Setting node annotation to enable volume controller attach/detach" Dec 03 11:20:16 crc kubenswrapper[5052]: I1203 11:20:16.058722 5052 kubelet_node_status.go:724] "Recording event message for node" node="crc" event="NodeHasSufficientMemory" Dec 03 11:20:16 crc kubenswrapper[5052]: I1203 11:20:16.058766 5052 kubelet_node_status.go:724] "Recording event message for node" node="crc" event="NodeHasNoDiskPressure" Dec 03 11:20:16 crc kubenswrapper[5052]: I1203 11:20:16.058810 5052 kubelet_node_status.go:724] "Recording event message for node" node="crc" event="NodeHasSufficientPID" Dec 03 11:20:16 crc kubenswrapper[5052]: I1203 11:20:16.058837 5052 kubelet_node_status.go:76] "Attempting to register node" node="crc" Dec 03 11:20:16 crc kubenswrapper[5052]: I1203 11:20:16.763931 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-etcd/etcd-crc" event={"ID":"2139d3e2895fc6797b9c76a1b4c9886d","Type":"ContainerStarted","Data":"2409d840f8acda1e98e333c3b4d9ebd1b67f4a05fa3f662f6d7171dddab13846"} Dec 03 11:20:16 crc kubenswrapper[5052]: I1203 11:20:16.764002 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-etcd/etcd-crc" event={"ID":"2139d3e2895fc6797b9c76a1b4c9886d","Type":"ContainerStarted","Data":"c5dd3e47134939ea92cb1677d8d171eadd8c0c0743827ebee0a61daf261f4e30"} Dec 03 11:20:16 crc kubenswrapper[5052]: I1203 11:20:16.764020 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-etcd/etcd-crc" event={"ID":"2139d3e2895fc6797b9c76a1b4c9886d","Type":"ContainerStarted","Data":"f3bd857c0d35ed129a4fdc96deae82d6457f5f24c4d92a129972ce97a89b3488"} Dec 03 11:20:16 crc kubenswrapper[5052]: I1203 11:20:16.764022 5052 kubelet_node_status.go:401] "Setting node annotation to enable volume controller attach/detach" Dec 03 11:20:16 crc kubenswrapper[5052]: I1203 11:20:16.764031 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-etcd/etcd-crc" event={"ID":"2139d3e2895fc6797b9c76a1b4c9886d","Type":"ContainerStarted","Data":"d9e3c1335ee9e6fe5d5ca827757da511034a270ca681b3cef694e595539d1cd0"} Dec 03 11:20:16 crc kubenswrapper[5052]: I1203 11:20:16.764049 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openshift-kube-apiserver/kube-apiserver-crc" Dec 03 11:20:16 crc kubenswrapper[5052]: I1203 11:20:16.764065 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openshift-kube-scheduler/openshift-kube-scheduler-crc" Dec 03 11:20:16 crc kubenswrapper[5052]: I1203 11:20:16.764022 5052 kubelet_node_status.go:401] "Setting node annotation to enable volume controller attach/detach" Dec 03 11:20:16 crc kubenswrapper[5052]: I1203 11:20:16.764022 5052 kubelet_node_status.go:401] "Setting node annotation to enable volume controller attach/detach" Dec 03 11:20:16 crc kubenswrapper[5052]: I1203 11:20:16.765166 5052 kubelet_node_status.go:724] "Recording event message for node" node="crc" event="NodeHasSufficientMemory" Dec 03 11:20:16 crc kubenswrapper[5052]: I1203 11:20:16.765196 5052 kubelet_node_status.go:724] "Recording event message for node" node="crc" event="NodeHasNoDiskPressure" Dec 03 11:20:16 crc kubenswrapper[5052]: I1203 11:20:16.765209 5052 kubelet_node_status.go:724] "Recording event message for node" node="crc" event="NodeHasSufficientPID" Dec 03 11:20:16 crc kubenswrapper[5052]: I1203 11:20:16.765327 5052 kubelet_node_status.go:724] "Recording event message for node" node="crc" event="NodeHasSufficientMemory" Dec 03 11:20:16 crc kubenswrapper[5052]: I1203 11:20:16.765382 5052 kubelet_node_status.go:724] "Recording event message for node" node="crc" event="NodeHasNoDiskPressure" Dec 03 11:20:16 crc kubenswrapper[5052]: I1203 11:20:16.765400 5052 kubelet_node_status.go:724] "Recording event message for node" node="crc" event="NodeHasSufficientPID" Dec 03 11:20:16 crc kubenswrapper[5052]: I1203 11:20:16.765345 5052 kubelet_node_status.go:724] "Recording event message for node" node="crc" event="NodeHasSufficientMemory" Dec 03 11:20:16 crc kubenswrapper[5052]: I1203 11:20:16.765433 5052 kubelet_node_status.go:724] "Recording event message for node" node="crc" event="NodeHasNoDiskPressure" Dec 03 11:20:16 crc kubenswrapper[5052]: I1203 11:20:16.765445 5052 kubelet_node_status.go:724] "Recording event message for node" node="crc" event="NodeHasSufficientPID" Dec 03 11:20:17 crc kubenswrapper[5052]: I1203 11:20:17.771366 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-etcd/etcd-crc" event={"ID":"2139d3e2895fc6797b9c76a1b4c9886d","Type":"ContainerStarted","Data":"7e429db04e5ef00df1a3a67e1750865167bb14eef83a110aa54ce3ca2f4656c5"} Dec 03 11:20:17 crc kubenswrapper[5052]: I1203 11:20:17.771561 5052 kubelet_node_status.go:401] "Setting node annotation to enable volume controller attach/detach" Dec 03 11:20:17 crc kubenswrapper[5052]: I1203 11:20:17.771817 5052 kubelet_node_status.go:401] "Setting node annotation to enable volume controller attach/detach" Dec 03 11:20:17 crc kubenswrapper[5052]: I1203 11:20:17.771817 5052 kubelet_node_status.go:401] "Setting node annotation to enable volume controller attach/detach" Dec 03 11:20:17 crc kubenswrapper[5052]: I1203 11:20:17.772099 5052 kubelet_node_status.go:401] "Setting node annotation to enable volume controller attach/detach" Dec 03 11:20:17 crc kubenswrapper[5052]: I1203 11:20:17.772889 5052 kubelet_node_status.go:724] "Recording event message for node" node="crc" event="NodeHasSufficientMemory" Dec 03 11:20:17 crc kubenswrapper[5052]: I1203 11:20:17.772962 5052 kubelet_node_status.go:724] "Recording event message for node" node="crc" event="NodeHasNoDiskPressure" Dec 03 11:20:17 crc kubenswrapper[5052]: I1203 11:20:17.772989 5052 kubelet_node_status.go:724] "Recording event message for node" node="crc" event="NodeHasSufficientPID" Dec 03 11:20:17 crc kubenswrapper[5052]: I1203 11:20:17.773076 5052 kubelet_node_status.go:724] "Recording event message for node" node="crc" event="NodeHasSufficientMemory" Dec 03 11:20:17 crc kubenswrapper[5052]: I1203 11:20:17.773084 5052 kubelet_node_status.go:724] "Recording event message for node" node="crc" event="NodeHasSufficientMemory" Dec 03 11:20:17 crc kubenswrapper[5052]: I1203 11:20:17.773101 5052 kubelet_node_status.go:724] "Recording event message for node" node="crc" event="NodeHasNoDiskPressure" Dec 03 11:20:17 crc kubenswrapper[5052]: I1203 11:20:17.773114 5052 kubelet_node_status.go:724] "Recording event message for node" node="crc" event="NodeHasSufficientPID" Dec 03 11:20:17 crc kubenswrapper[5052]: I1203 11:20:17.773115 5052 kubelet_node_status.go:724] "Recording event message for node" node="crc" event="NodeHasNoDiskPressure" Dec 03 11:20:17 crc kubenswrapper[5052]: I1203 11:20:17.773148 5052 kubelet_node_status.go:724] "Recording event message for node" node="crc" event="NodeHasSufficientPID" Dec 03 11:20:17 crc kubenswrapper[5052]: I1203 11:20:17.773341 5052 kubelet_node_status.go:724] "Recording event message for node" node="crc" event="NodeHasSufficientMemory" Dec 03 11:20:17 crc kubenswrapper[5052]: I1203 11:20:17.773392 5052 kubelet_node_status.go:724] "Recording event message for node" node="crc" event="NodeHasNoDiskPressure" Dec 03 11:20:17 crc kubenswrapper[5052]: I1203 11:20:17.773405 5052 kubelet_node_status.go:724] "Recording event message for node" node="crc" event="NodeHasSufficientPID" Dec 03 11:20:18 crc kubenswrapper[5052]: I1203 11:20:18.112391 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openshift-kube-apiserver/kube-apiserver-crc" Dec 03 11:20:18 crc kubenswrapper[5052]: I1203 11:20:18.421272 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-kube-apiserver/kube-apiserver-crc" Dec 03 11:20:18 crc kubenswrapper[5052]: I1203 11:20:18.774605 5052 kubelet_node_status.go:401] "Setting node annotation to enable volume controller attach/detach" Dec 03 11:20:18 crc kubenswrapper[5052]: I1203 11:20:18.774605 5052 kubelet_node_status.go:401] "Setting node annotation to enable volume controller attach/detach" Dec 03 11:20:18 crc kubenswrapper[5052]: I1203 11:20:18.776012 5052 kubelet_node_status.go:724] "Recording event message for node" node="crc" event="NodeHasSufficientMemory" Dec 03 11:20:18 crc kubenswrapper[5052]: I1203 11:20:18.776057 5052 kubelet_node_status.go:724] "Recording event message for node" node="crc" event="NodeHasNoDiskPressure" Dec 03 11:20:18 crc kubenswrapper[5052]: I1203 11:20:18.776062 5052 kubelet_node_status.go:724] "Recording event message for node" node="crc" event="NodeHasSufficientMemory" Dec 03 11:20:18 crc kubenswrapper[5052]: I1203 11:20:18.776099 5052 kubelet_node_status.go:724] "Recording event message for node" node="crc" event="NodeHasNoDiskPressure" Dec 03 11:20:18 crc kubenswrapper[5052]: I1203 11:20:18.776112 5052 kubelet_node_status.go:724] "Recording event message for node" node="crc" event="NodeHasSufficientPID" Dec 03 11:20:18 crc kubenswrapper[5052]: I1203 11:20:18.776071 5052 kubelet_node_status.go:724] "Recording event message for node" node="crc" event="NodeHasSufficientPID" Dec 03 11:20:19 crc kubenswrapper[5052]: I1203 11:20:19.030101 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openshift-kube-controller-manager/kube-controller-manager-crc" Dec 03 11:20:19 crc kubenswrapper[5052]: I1203 11:20:19.030311 5052 kubelet_node_status.go:401] "Setting node annotation to enable volume controller attach/detach" Dec 03 11:20:19 crc kubenswrapper[5052]: I1203 11:20:19.031754 5052 kubelet_node_status.go:724] "Recording event message for node" node="crc" event="NodeHasSufficientMemory" Dec 03 11:20:19 crc kubenswrapper[5052]: I1203 11:20:19.031794 5052 kubelet_node_status.go:724] "Recording event message for node" node="crc" event="NodeHasNoDiskPressure" Dec 03 11:20:19 crc kubenswrapper[5052]: I1203 11:20:19.031806 5052 kubelet_node_status.go:724] "Recording event message for node" node="crc" event="NodeHasSufficientPID" Dec 03 11:20:19 crc kubenswrapper[5052]: I1203 11:20:19.119350 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-kube-controller-manager/kube-controller-manager-crc" Dec 03 11:20:19 crc kubenswrapper[5052]: E1203 11:20:19.750288 5052 eviction_manager.go:285] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"crc\" not found" Dec 03 11:20:19 crc kubenswrapper[5052]: I1203 11:20:19.777705 5052 kubelet_node_status.go:401] "Setting node annotation to enable volume controller attach/detach" Dec 03 11:20:19 crc kubenswrapper[5052]: I1203 11:20:19.777796 5052 kubelet_node_status.go:401] "Setting node annotation to enable volume controller attach/detach" Dec 03 11:20:19 crc kubenswrapper[5052]: I1203 11:20:19.778967 5052 kubelet_node_status.go:724] "Recording event message for node" node="crc" event="NodeHasSufficientMemory" Dec 03 11:20:19 crc kubenswrapper[5052]: I1203 11:20:19.778989 5052 kubelet_node_status.go:724] "Recording event message for node" node="crc" event="NodeHasSufficientMemory" Dec 03 11:20:19 crc kubenswrapper[5052]: I1203 11:20:19.779012 5052 kubelet_node_status.go:724] "Recording event message for node" node="crc" event="NodeHasNoDiskPressure" Dec 03 11:20:19 crc kubenswrapper[5052]: I1203 11:20:19.779039 5052 kubelet_node_status.go:724] "Recording event message for node" node="crc" event="NodeHasSufficientPID" Dec 03 11:20:19 crc kubenswrapper[5052]: I1203 11:20:19.779016 5052 kubelet_node_status.go:724] "Recording event message for node" node="crc" event="NodeHasNoDiskPressure" Dec 03 11:20:19 crc kubenswrapper[5052]: I1203 11:20:19.779173 5052 kubelet_node_status.go:724] "Recording event message for node" node="crc" event="NodeHasSufficientPID" Dec 03 11:20:21 crc kubenswrapper[5052]: I1203 11:20:21.178913 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openshift-etcd/etcd-crc" Dec 03 11:20:21 crc kubenswrapper[5052]: I1203 11:20:21.179160 5052 kubelet_node_status.go:401] "Setting node annotation to enable volume controller attach/detach" Dec 03 11:20:21 crc kubenswrapper[5052]: I1203 11:20:21.180680 5052 kubelet_node_status.go:724] "Recording event message for node" node="crc" event="NodeHasSufficientMemory" Dec 03 11:20:21 crc kubenswrapper[5052]: I1203 11:20:21.180714 5052 kubelet_node_status.go:724] "Recording event message for node" node="crc" event="NodeHasNoDiskPressure" Dec 03 11:20:21 crc kubenswrapper[5052]: I1203 11:20:21.180722 5052 kubelet_node_status.go:724] "Recording event message for node" node="crc" event="NodeHasSufficientPID" Dec 03 11:20:22 crc kubenswrapper[5052]: I1203 11:20:22.120715 5052 patch_prober.go:28] interesting pod/kube-controller-manager-crc container/cluster-policy-controller namespace/openshift-kube-controller-manager: Startup probe status=failure output="Get \"https://192.168.126.11:10357/healthz\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Dec 03 11:20:22 crc kubenswrapper[5052]: I1203 11:20:22.120840 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openshift-kube-controller-manager/kube-controller-manager-crc" podUID="f614b9022728cf315e60c057852e563e" containerName="cluster-policy-controller" probeResult="failure" output="Get \"https://192.168.126.11:10357/healthz\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Dec 03 11:20:22 crc kubenswrapper[5052]: I1203 11:20:22.956357 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-etcd/etcd-crc" Dec 03 11:20:22 crc kubenswrapper[5052]: I1203 11:20:22.956628 5052 kubelet_node_status.go:401] "Setting node annotation to enable volume controller attach/detach" Dec 03 11:20:22 crc kubenswrapper[5052]: I1203 11:20:22.958474 5052 kubelet_node_status.go:724] "Recording event message for node" node="crc" event="NodeHasSufficientMemory" Dec 03 11:20:22 crc kubenswrapper[5052]: I1203 11:20:22.958551 5052 kubelet_node_status.go:724] "Recording event message for node" node="crc" event="NodeHasNoDiskPressure" Dec 03 11:20:22 crc kubenswrapper[5052]: I1203 11:20:22.958563 5052 kubelet_node_status.go:724] "Recording event message for node" node="crc" event="NodeHasSufficientPID" Dec 03 11:20:24 crc kubenswrapper[5052]: I1203 11:20:24.386771 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-kube-controller-manager/kube-controller-manager-crc" Dec 03 11:20:24 crc kubenswrapper[5052]: I1203 11:20:24.387035 5052 kubelet_node_status.go:401] "Setting node annotation to enable volume controller attach/detach" Dec 03 11:20:24 crc kubenswrapper[5052]: I1203 11:20:24.388804 5052 kubelet_node_status.go:724] "Recording event message for node" node="crc" event="NodeHasSufficientMemory" Dec 03 11:20:24 crc kubenswrapper[5052]: I1203 11:20:24.388906 5052 kubelet_node_status.go:724] "Recording event message for node" node="crc" event="NodeHasNoDiskPressure" Dec 03 11:20:24 crc kubenswrapper[5052]: I1203 11:20:24.388924 5052 kubelet_node_status.go:724] "Recording event message for node" node="crc" event="NodeHasSufficientPID" Dec 03 11:20:24 crc kubenswrapper[5052]: I1203 11:20:24.394027 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-kube-controller-manager/kube-controller-manager-crc" Dec 03 11:20:24 crc kubenswrapper[5052]: I1203 11:20:24.789712 5052 kubelet_node_status.go:401] "Setting node annotation to enable volume controller attach/detach" Dec 03 11:20:24 crc kubenswrapper[5052]: I1203 11:20:24.791029 5052 kubelet_node_status.go:724] "Recording event message for node" node="crc" event="NodeHasSufficientMemory" Dec 03 11:20:24 crc kubenswrapper[5052]: I1203 11:20:24.791093 5052 kubelet_node_status.go:724] "Recording event message for node" node="crc" event="NodeHasNoDiskPressure" Dec 03 11:20:24 crc kubenswrapper[5052]: I1203 11:20:24.791114 5052 kubelet_node_status.go:724] "Recording event message for node" node="crc" event="NodeHasSufficientPID" Dec 03 11:20:24 crc kubenswrapper[5052]: I1203 11:20:24.794944 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-kube-controller-manager/kube-controller-manager-crc" Dec 03 11:20:25 crc kubenswrapper[5052]: I1203 11:20:25.605979 5052 csi_plugin.go:884] Failed to contact API server when waiting for CSINode publishing: Get "https://api-int.crc.testing:6443/apis/storage.k8s.io/v1/csinodes/crc?resourceVersion=0": net/http: TLS handshake timeout Dec 03 11:20:25 crc kubenswrapper[5052]: I1203 11:20:25.792556 5052 kubelet_node_status.go:401] "Setting node annotation to enable volume controller attach/detach" Dec 03 11:20:25 crc kubenswrapper[5052]: I1203 11:20:25.793614 5052 kubelet_node_status.go:724] "Recording event message for node" node="crc" event="NodeHasSufficientMemory" Dec 03 11:20:25 crc kubenswrapper[5052]: I1203 11:20:25.793678 5052 kubelet_node_status.go:724] "Recording event message for node" node="crc" event="NodeHasNoDiskPressure" Dec 03 11:20:25 crc kubenswrapper[5052]: I1203 11:20:25.793699 5052 kubelet_node_status.go:724] "Recording event message for node" node="crc" event="NodeHasSufficientPID" Dec 03 11:20:25 crc kubenswrapper[5052]: E1203 11:20:25.827309 5052 controller.go:145] "Failed to ensure lease exists, will retry" err="Get \"https://api-int.crc.testing:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/crc?timeout=10s\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" interval="6.4s" Dec 03 11:20:25 crc kubenswrapper[5052]: I1203 11:20:25.944704 5052 patch_prober.go:28] interesting pod/kube-apiserver-crc container/kube-apiserver namespace/openshift-kube-apiserver: Startup probe status=failure output="HTTP probe failed with statuscode: 403" start-of-body={"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"forbidden: User \"system:anonymous\" cannot get path \"/livez\"","reason":"Forbidden","details":{},"code":403} Dec 03 11:20:25 crc kubenswrapper[5052]: I1203 11:20:25.944811 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openshift-kube-apiserver/kube-apiserver-crc" podUID="f4b27818a5e8e43d0dc095d08835c792" containerName="kube-apiserver" probeResult="failure" output="HTTP probe failed with statuscode: 403" Dec 03 11:20:25 crc kubenswrapper[5052]: I1203 11:20:25.961537 5052 patch_prober.go:28] interesting pod/kube-apiserver-crc container/kube-apiserver namespace/openshift-kube-apiserver: Startup probe status=failure output="HTTP probe failed with statuscode: 403" start-of-body={"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"forbidden: User \"system:anonymous\" cannot get path \"/livez\": RBAC: [clusterrole.rbac.authorization.k8s.io \"system:openshift:public-info-viewer\" not found, clusterrole.rbac.authorization.k8s.io \"system:public-info-viewer\" not found]","reason":"Forbidden","details":{},"code":403} Dec 03 11:20:25 crc kubenswrapper[5052]: I1203 11:20:25.961642 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openshift-kube-apiserver/kube-apiserver-crc" podUID="f4b27818a5e8e43d0dc095d08835c792" containerName="kube-apiserver" probeResult="failure" output="HTTP probe failed with statuscode: 403" Dec 03 11:20:28 crc kubenswrapper[5052]: I1203 11:20:28.426616 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-kube-apiserver/kube-apiserver-crc" Dec 03 11:20:28 crc kubenswrapper[5052]: I1203 11:20:28.427276 5052 kubelet_node_status.go:401] "Setting node annotation to enable volume controller attach/detach" Dec 03 11:20:28 crc kubenswrapper[5052]: I1203 11:20:28.434230 5052 kubelet_node_status.go:724] "Recording event message for node" node="crc" event="NodeHasSufficientMemory" Dec 03 11:20:28 crc kubenswrapper[5052]: I1203 11:20:28.434329 5052 kubelet_node_status.go:724] "Recording event message for node" node="crc" event="NodeHasNoDiskPressure" Dec 03 11:20:28 crc kubenswrapper[5052]: I1203 11:20:28.434379 5052 kubelet_node_status.go:724] "Recording event message for node" node="crc" event="NodeHasSufficientPID" Dec 03 11:20:28 crc kubenswrapper[5052]: I1203 11:20:28.440707 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-kube-apiserver/kube-apiserver-crc" Dec 03 11:20:28 crc kubenswrapper[5052]: I1203 11:20:28.800314 5052 kubelet_node_status.go:401] "Setting node annotation to enable volume controller attach/detach" Dec 03 11:20:28 crc kubenswrapper[5052]: I1203 11:20:28.801190 5052 kubelet_node_status.go:724] "Recording event message for node" node="crc" event="NodeHasSufficientMemory" Dec 03 11:20:28 crc kubenswrapper[5052]: I1203 11:20:28.801239 5052 kubelet_node_status.go:724] "Recording event message for node" node="crc" event="NodeHasNoDiskPressure" Dec 03 11:20:28 crc kubenswrapper[5052]: I1203 11:20:28.801251 5052 kubelet_node_status.go:724] "Recording event message for node" node="crc" event="NodeHasSufficientPID" Dec 03 11:20:29 crc kubenswrapper[5052]: E1203 11:20:29.750729 5052 eviction_manager.go:285] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"crc\" not found" Dec 03 11:20:30 crc kubenswrapper[5052]: I1203 11:20:30.941856 5052 trace.go:236] Trace[1347031738]: "Reflector ListAndWatch" name:k8s.io/client-go/informers/factory.go:160 (03-Dec-2025 11:20:18.381) (total time: 12559ms): Dec 03 11:20:30 crc kubenswrapper[5052]: Trace[1347031738]: ---"Objects listed" error: 12559ms (11:20:30.941) Dec 03 11:20:30 crc kubenswrapper[5052]: Trace[1347031738]: [12.559979995s] [12.559979995s] END Dec 03 11:20:30 crc kubenswrapper[5052]: I1203 11:20:30.941893 5052 reflector.go:368] Caches populated for *v1.RuntimeClass from k8s.io/client-go/informers/factory.go:160 Dec 03 11:20:30 crc kubenswrapper[5052]: I1203 11:20:30.942714 5052 trace.go:236] Trace[1382570905]: "Reflector ListAndWatch" name:k8s.io/client-go/informers/factory.go:160 (03-Dec-2025 11:20:19.411) (total time: 11531ms): Dec 03 11:20:30 crc kubenswrapper[5052]: Trace[1382570905]: ---"Objects listed" error: 11531ms (11:20:30.942) Dec 03 11:20:30 crc kubenswrapper[5052]: Trace[1382570905]: [11.531054098s] [11.531054098s] END Dec 03 11:20:30 crc kubenswrapper[5052]: I1203 11:20:30.942743 5052 reflector.go:368] Caches populated for *v1.CSIDriver from k8s.io/client-go/informers/factory.go:160 Dec 03 11:20:30 crc kubenswrapper[5052]: E1203 11:20:30.943747 5052 kubelet_node_status.go:99] "Unable to register node with API server" err="nodes \"crc\" is forbidden: autoscaling.openshift.io/ManagedNode infra config cache not synchronized" node="crc" Dec 03 11:20:30 crc kubenswrapper[5052]: I1203 11:20:30.943978 5052 trace.go:236] Trace[55997705]: "Reflector ListAndWatch" name:k8s.io/client-go/informers/factory.go:160 (03-Dec-2025 11:20:17.367) (total time: 13576ms): Dec 03 11:20:30 crc kubenswrapper[5052]: Trace[55997705]: ---"Objects listed" error: 13576ms (11:20:30.943) Dec 03 11:20:30 crc kubenswrapper[5052]: Trace[55997705]: [13.576505678s] [13.576505678s] END Dec 03 11:20:30 crc kubenswrapper[5052]: I1203 11:20:30.944020 5052 reflector.go:368] Caches populated for *v1.Node from k8s.io/client-go/informers/factory.go:160 Dec 03 11:20:30 crc kubenswrapper[5052]: I1203 11:20:30.943980 5052 trace.go:236] Trace[792845787]: "Reflector ListAndWatch" name:k8s.io/client-go/informers/factory.go:160 (03-Dec-2025 11:20:19.935) (total time: 11008ms): Dec 03 11:20:30 crc kubenswrapper[5052]: Trace[792845787]: ---"Objects listed" error: 11008ms (11:20:30.943) Dec 03 11:20:30 crc kubenswrapper[5052]: Trace[792845787]: [11.008292581s] [11.008292581s] END Dec 03 11:20:30 crc kubenswrapper[5052]: I1203 11:20:30.944063 5052 reflector.go:368] Caches populated for *v1.Service from k8s.io/client-go/informers/factory.go:160 Dec 03 11:20:30 crc kubenswrapper[5052]: I1203 11:20:30.945330 5052 reconstruct.go:205] "DevicePaths of reconstructed volumes updated" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.347155 5052 patch_prober.go:28] interesting pod/kube-apiserver-crc container/kube-apiserver-check-endpoints namespace/openshift-kube-apiserver: Readiness probe status=failure output="Get \"https://192.168.126.11:17697/healthz\": read tcp 192.168.126.11:52526->192.168.126.11:17697: read: connection reset by peer" start-of-body= Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.348105 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openshift-kube-apiserver/kube-apiserver-crc" podUID="f4b27818a5e8e43d0dc095d08835c792" containerName="kube-apiserver-check-endpoints" probeResult="failure" output="Get \"https://192.168.126.11:17697/healthz\": read tcp 192.168.126.11:52526->192.168.126.11:17697: read: connection reset by peer" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.349917 5052 patch_prober.go:28] interesting pod/kube-apiserver-crc container/kube-apiserver-check-endpoints namespace/openshift-kube-apiserver: Readiness probe status=failure output="Get \"https://192.168.126.11:17697/healthz\": dial tcp 192.168.126.11:17697: connect: connection refused" start-of-body= Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.349992 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openshift-kube-apiserver/kube-apiserver-crc" podUID="f4b27818a5e8e43d0dc095d08835c792" containerName="kube-apiserver-check-endpoints" probeResult="failure" output="Get \"https://192.168.126.11:17697/healthz\": dial tcp 192.168.126.11:17697: connect: connection refused" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.350393 5052 patch_prober.go:28] interesting pod/kube-apiserver-crc container/kube-apiserver-check-endpoints namespace/openshift-kube-apiserver: Readiness probe status=failure output="Get \"https://192.168.126.11:17697/healthz\": dial tcp 192.168.126.11:17697: connect: connection refused" start-of-body= Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.350469 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openshift-kube-apiserver/kube-apiserver-crc" podUID="f4b27818a5e8e43d0dc095d08835c792" containerName="kube-apiserver-check-endpoints" probeResult="failure" output="Get \"https://192.168.126.11:17697/healthz\": dial tcp 192.168.126.11:17697: connect: connection refused" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.611121 5052 apiserver.go:52] "Watching apiserver" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.615913 5052 reflector.go:368] Caches populated for *v1.Pod from pkg/kubelet/config/apiserver.go:66 Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.616442 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openshift-dns/node-resolver-xph69","openshift-machine-config-operator/machine-config-daemon-74bj6","openshift-network-diagnostics/network-check-source-55646444c4-trplf","openshift-network-operator/iptables-alerter-4ln5h","openshift-multus/multus-additional-cni-plugins-2lrql","openshift-multus/multus-hx657","openshift-network-console/networking-console-plugin-85b44fc459-gdk6g","openshift-network-diagnostics/network-check-target-xd92c","openshift-network-node-identity/network-node-identity-vrzqb","openshift-network-operator/network-operator-58b4c7f79c-55gtf","openshift-ovn-kubernetes/ovnkube-node-kg55h"] Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.616910 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/network-operator-58b4c7f79c-55gtf" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.616984 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-55646444c4-trplf" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.617021 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-4ln5h" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.617049 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-node-identity/network-node-identity-vrzqb" Dec 03 11:20:31 crc kubenswrapper[5052]: E1203 11:20:31.617087 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-source-55646444c4-trplf" podUID="9d751cbb-f2e2-430d-9754-c882a5e924a5" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.617150 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xd92c" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.617396 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-85b44fc459-gdk6g" Dec 03 11:20:31 crc kubenswrapper[5052]: E1203 11:20:31.617437 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-console/networking-console-plugin-85b44fc459-gdk6g" podUID="5fe485a1-e14f-4c09-b5b9-f252bc42b7e8" Dec 03 11:20:31 crc kubenswrapper[5052]: E1203 11:20:31.617524 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xd92c" podUID="3b6479f0-333b-4a96-9adf-2099afdc2447" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.617700 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-xph69" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.617951 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.618136 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-2lrql" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.618188 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-hx657" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.618614 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-kg55h" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.621325 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-network-node-identity"/"openshift-service-ca.crt" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.621669 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-multus"/"openshift-service-ca.crt" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.622086 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-machine-config-operator"/"kube-rbac-proxy" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.622277 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-dns"/"openshift-service-ca.crt" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.622325 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-machine-config-operator"/"machine-config-daemon-dockercfg-r5tcq" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.622476 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-network-operator"/"iptables-alerter-script" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.622610 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-multus"/"default-dockercfg-2q5b6" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.622738 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-network-node-identity"/"kube-root-ca.crt" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.622766 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-multus"/"default-cni-sysctl-allowlist" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.622632 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-multus"/"cni-copy-resources" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.622834 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-machine-config-operator"/"proxy-tls" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.622913 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-network-operator"/"openshift-service-ca.crt" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.622846 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-multus"/"multus-ancillary-tools-dockercfg-vnmsz" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.622956 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-ovn-kubernetes"/"ovn-node-metrics-cert" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.622978 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-machine-config-operator"/"openshift-service-ca.crt" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.623243 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-network-operator"/"metrics-tls" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.624724 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-network-operator"/"kube-root-ca.crt" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.624842 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-multus"/"kube-root-ca.crt" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.625068 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-ovn-kubernetes"/"openshift-service-ca.crt" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.625191 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-ovn-kubernetes"/"env-overrides" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.625346 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-ovn-kubernetes"/"ovnkube-config" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.625426 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-ovn-kubernetes"/"kube-root-ca.crt" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.625592 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-network-node-identity"/"ovnkube-identity-cm" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.625720 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-network-node-identity"/"env-overrides" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.628017 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-multus"/"multus-daemon-config" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.628351 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-dns"/"node-resolver-dockercfg-kz9s7" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.628779 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-dns"/"kube-root-ca.crt" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.629211 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-ovn-kubernetes"/"ovn-kubernetes-node-dockercfg-pwtwl" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.629816 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-ovn-kubernetes"/"ovnkube-script-lib" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.630100 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-network-node-identity"/"network-node-identity-cert" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.632836 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-machine-config-operator"/"kube-root-ca.crt" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.643456 5052 status_manager.go:875] "Failed to update status for pod" pod="openshift-network-operator/network-operator-58b4c7f79c-55gtf" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"37a5e44f-9a88-4405-be8a-b645485e7312\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"message\\\":\\\"containers with unready status: [network-operator]\\\",\\\"reason\\\":\\\"ContainersNotReady\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"message\\\":\\\"containers with unready status: [network-operator]\\\",\\\"reason\\\":\\\"ContainersNotReady\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"ContainersReady\\\"}],\\\"containerStatuses\\\":[{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e1baa38811c04bd8909e01a1f3be7421a1cb99d608d3dc4cf86d95b17de2ab8b\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{\\\"terminated\\\":{\\\"exitCode\\\":137,\\\"finishedAt\\\":null,\\\"message\\\":\\\"The container could not be located when the pod was deleted. The container used to be Running\\\",\\\"reason\\\":\\\"ContainerStatusUnknown\\\",\\\"startedAt\\\":null}},\\\"name\\\":\\\"network-operator\\\",\\\"ready\\\":false,\\\"restartCount\\\":5,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"ContainerCreating\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/etc/kubernetes\\\",\\\"name\\\":\\\"host-etc-kube\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/serving-cert\\\",\\\"name\\\":\\\"metrics-tls\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-rdwmf\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}]}}\" for pod \"openshift-network-operator\"/\"network-operator-58b4c7f79c-55gtf\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://127.0.0.1:9743/pod?timeout=10s\": dial tcp 127.0.0.1:9743: connect: connection refused" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.648459 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-kube-controller-manager/kube-controller-manager-crc" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.652775 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-kube-controller-manager/kube-controller-manager-crc" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.671622 5052 status_manager.go:875] "Failed to update status for pod" pod="openshift-network-diagnostics/network-check-source-55646444c4-trplf" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"9d751cbb-f2e2-430d-9754-c882a5e924a5\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"message\\\":\\\"containers with unready status: [check-endpoints]\\\",\\\"reason\\\":\\\"ContainersNotReady\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"message\\\":\\\"containers with unready status: [check-endpoints]\\\",\\\"reason\\\":\\\"ContainersNotReady\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"ContainersReady\\\"}],\\\"containerStatuses\\\":[{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e1baa38811c04bd8909e01a1f3be7421a1cb99d608d3dc4cf86d95b17de2ab8b\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{\\\"terminated\\\":{\\\"exitCode\\\":137,\\\"finishedAt\\\":null,\\\"message\\\":\\\"The container could not be located when the pod was deleted. The container used to be Running\\\",\\\"reason\\\":\\\"ContainerStatusUnknown\\\",\\\"startedAt\\\":null}},\\\"name\\\":\\\"check-endpoints\\\",\\\"ready\\\":false,\\\"restartCount\\\":3,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"ContainerCreating\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-s2dwl\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}],\\\"podIP\\\":null,\\\"podIPs\\\":null}}\" for pod \"openshift-network-diagnostics\"/\"network-check-source-55646444c4-trplf\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://127.0.0.1:9743/pod?timeout=10s\": dial tcp 127.0.0.1:9743: connect: connection refused" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.684320 5052 status_manager.go:875] "Failed to update status for pod" pod="openshift-network-operator/network-operator-58b4c7f79c-55gtf" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"37a5e44f-9a88-4405-be8a-b645485e7312\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"message\\\":\\\"containers with unready status: [network-operator]\\\",\\\"reason\\\":\\\"ContainersNotReady\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"message\\\":\\\"containers with unready status: [network-operator]\\\",\\\"reason\\\":\\\"ContainersNotReady\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"ContainersReady\\\"}],\\\"containerStatuses\\\":[{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e1baa38811c04bd8909e01a1f3be7421a1cb99d608d3dc4cf86d95b17de2ab8b\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{\\\"terminated\\\":{\\\"exitCode\\\":137,\\\"finishedAt\\\":null,\\\"message\\\":\\\"The container could not be located when the pod was deleted. The container used to be Running\\\",\\\"reason\\\":\\\"ContainerStatusUnknown\\\",\\\"startedAt\\\":null}},\\\"name\\\":\\\"network-operator\\\",\\\"ready\\\":false,\\\"restartCount\\\":5,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"ContainerCreating\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/etc/kubernetes\\\",\\\"name\\\":\\\"host-etc-kube\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/serving-cert\\\",\\\"name\\\":\\\"metrics-tls\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-rdwmf\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}]}}\" for pod \"openshift-network-operator\"/\"network-operator-58b4c7f79c-55gtf\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://127.0.0.1:9743/pod?timeout=10s\": dial tcp 127.0.0.1:9743: connect: connection refused" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.701972 5052 status_manager.go:875] "Failed to update status for pod" pod="openshift-network-node-identity/network-node-identity-vrzqb" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"ef543e1b-8068-4ea3-b32a-61027b32e95d\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"message\\\":\\\"containers with unready status: [webhook approver]\\\",\\\"reason\\\":\\\"ContainersNotReady\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"message\\\":\\\"containers with unready status: [webhook approver]\\\",\\\"reason\\\":\\\"ContainersNotReady\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"ContainersReady\\\"}],\\\"containerStatuses\\\":[{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:174f36cdd47ef0d1d2099482919d773257453265a2af0b17b154edc32fa41ac2\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{\\\"terminated\\\":{\\\"exitCode\\\":137,\\\"finishedAt\\\":null,\\\"message\\\":\\\"The container could not be located when the pod was deleted. The container used to be Running\\\",\\\"reason\\\":\\\"ContainerStatusUnknown\\\",\\\"startedAt\\\":null}},\\\"name\\\":\\\"approver\\\",\\\"ready\\\":false,\\\"restartCount\\\":6,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"ContainerCreating\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/env\\\",\\\"name\\\":\\\"env-overrides\\\"},{\\\"mountPath\\\":\\\"/var/run/ovnkube-identity-config\\\",\\\"name\\\":\\\"ovnkube-identity-cm\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-s2kz5\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:174f36cdd47ef0d1d2099482919d773257453265a2af0b17b154edc32fa41ac2\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{\\\"terminated\\\":{\\\"exitCode\\\":137,\\\"finishedAt\\\":null,\\\"message\\\":\\\"The container could not be located when the pod was deleted. The container used to be Running\\\",\\\"reason\\\":\\\"ContainerStatusUnknown\\\",\\\"startedAt\\\":null}},\\\"name\\\":\\\"webhook\\\",\\\"ready\\\":false,\\\"restartCount\\\":3,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"ContainerCreating\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/etc/webhook-cert/\\\",\\\"name\\\":\\\"webhook-cert\\\"},{\\\"mountPath\\\":\\\"/env\\\",\\\"name\\\":\\\"env-overrides\\\"},{\\\"mountPath\\\":\\\"/var/run/ovnkube-identity-config\\\",\\\"name\\\":\\\"ovnkube-identity-cm\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-s2kz5\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}]}}\" for pod \"openshift-network-node-identity\"/\"network-node-identity-vrzqb\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://127.0.0.1:9743/pod?timeout=10s\": dial tcp 127.0.0.1:9743: connect: connection refused" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.714862 5052 status_manager.go:875] "Failed to update status for pod" pod="openshift-network-console/networking-console-plugin-85b44fc459-gdk6g" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"5fe485a1-e14f-4c09-b5b9-f252bc42b7e8\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"message\\\":\\\"containers with unready status: [networking-console-plugin]\\\",\\\"reason\\\":\\\"ContainersNotReady\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"message\\\":\\\"containers with unready status: [networking-console-plugin]\\\",\\\"reason\\\":\\\"ContainersNotReady\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"ContainersReady\\\"}],\\\"containerStatuses\\\":[{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ae647598ec35cda5766806d3d44a91e3b9d4dee48ff154f3d8490165399873fd\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{\\\"terminated\\\":{\\\"exitCode\\\":137,\\\"finishedAt\\\":null,\\\"message\\\":\\\"The container could not be located when the pod was deleted. The container used to be Running\\\",\\\"reason\\\":\\\"ContainerStatusUnknown\\\",\\\"startedAt\\\":null}},\\\"name\\\":\\\"networking-console-plugin\\\",\\\"ready\\\":false,\\\"restartCount\\\":3,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"ContainerCreating\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/var/cert\\\",\\\"name\\\":\\\"networking-console-plugin-cert\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/etc/nginx/nginx.conf\\\",\\\"name\\\":\\\"nginx-conf\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}],\\\"podIP\\\":null,\\\"podIPs\\\":null}}\" for pod \"openshift-network-console\"/\"networking-console-plugin-85b44fc459-gdk6g\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://127.0.0.1:9743/pod?timeout=10s\": dial tcp 127.0.0.1:9743: connect: connection refused" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.717852 5052 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.725448 5052 status_manager.go:875] "Failed to update status for pod" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"10c237bc-72e7-4da3-9f2c-b1688395bf6e\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"Initialized\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"message\\\":\\\"containers with unready status: [machine-config-daemon kube-rbac-proxy]\\\",\\\"reason\\\":\\\"ContainersNotReady\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"message\\\":\\\"containers with unready status: [machine-config-daemon kube-rbac-proxy]\\\",\\\"reason\\\":\\\"ContainersNotReady\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"ContainersReady\\\"}],\\\"containerStatuses\\\":[{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:242b3d66438c42745f4ef318bdeaf3d793426f12962a42ea83e18d06c08aaf09\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"kube-rbac-proxy\\\",\\\"ready\\\":false,\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"ContainerCreating\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/etc/tls/private\\\",\\\"name\\\":\\\"proxy-tls\\\"},{\\\"mountPath\\\":\\\"/etc/kube-rbac-proxy\\\",\\\"name\\\":\\\"mcd-auth-proxy-config\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-9vkjv\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c915fb8ba96e911699a1ae34a8e95ca8a9fbe1bf8c28fea177225c63a8bdfc0a\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"machine-config-daemon\\\",\\\"ready\\\":false,\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"ContainerCreating\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/rootfs\\\",\\\"name\\\":\\\"rootfs\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-9vkjv\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}],\\\"hostIP\\\":\\\"192.168.126.11\\\",\\\"hostIPs\\\":[{\\\"ip\\\":\\\"192.168.126.11\\\"}],\\\"podIP\\\":\\\"192.168.126.11\\\",\\\"podIPs\\\":[{\\\"ip\\\":\\\"192.168.126.11\\\"}],\\\"startTime\\\":\\\"2025-12-03T11:20:31Z\\\"}}\" for pod \"openshift-machine-config-operator\"/\"machine-config-daemon-74bj6\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://127.0.0.1:9743/pod?timeout=10s\": dial tcp 127.0.0.1:9743: connect: connection refused" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.730895 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openshift-kube-controller-manager/kube-controller-manager-crc"] Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.742947 5052 status_manager.go:875] "Failed to update status for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kg55h" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"bc569bba-c020-40d6-af68-e617625fe0e4\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"message\\\":\\\"containers with incomplete status: [kubecfg-setup]\\\",\\\"reason\\\":\\\"ContainersNotInitialized\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"Initialized\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"message\\\":\\\"containers with unready status: [ovn-controller ovn-acl-logging kube-rbac-proxy-node kube-rbac-proxy-ovn-metrics northd nbdb sbdb ovnkube-controller]\\\",\\\"reason\\\":\\\"ContainersNotReady\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"message\\\":\\\"containers with unready status: [ovn-controller ovn-acl-logging kube-rbac-proxy-node kube-rbac-proxy-ovn-metrics northd nbdb sbdb ovnkube-controller]\\\",\\\"reason\\\":\\\"ContainersNotReady\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"ContainersReady\\\"}],\\\"containerStatuses\\\":[{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:242b3d66438c42745f4ef318bdeaf3d793426f12962a42ea83e18d06c08aaf09\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"kube-rbac-proxy-node\\\",\\\"ready\\\":false,\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"PodInitializing\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/ovnkube-lib\\\",\\\"name\\\":\\\"ovnkube-script-lib\\\"},{\\\"mountPath\\\":\\\"/etc/pki/tls/metrics-cert\\\",\\\"name\\\":\\\"ovn-node-metrics-cert\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-x92h2\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:242b3d66438c42745f4ef318bdeaf3d793426f12962a42ea83e18d06c08aaf09\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"kube-rbac-proxy-ovn-metrics\\\",\\\"ready\\\":false,\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"PodInitializing\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/ovnkube-lib\\\",\\\"name\\\":\\\"ovnkube-script-lib\\\"},{\\\"mountPath\\\":\\\"/etc/pki/tls/metrics-cert\\\",\\\"name\\\":\\\"ovn-node-metrics-cert\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-x92h2\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:174f36cdd47ef0d1d2099482919d773257453265a2af0b17b154edc32fa41ac2\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"nbdb\\\",\\\"ready\\\":false,\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"PodInitializing\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/ovnkube-lib\\\",\\\"name\\\":\\\"ovnkube-script-lib\\\"},{\\\"mountPath\\\":\\\"/etc/ovn/\\\",\\\"name\\\":\\\"etc-openvswitch\\\"},{\\\"mountPath\\\":\\\"/var/log/ovn\\\",\\\"name\\\":\\\"node-log\\\"},{\\\"mountPath\\\":\\\"/run/ovn/\\\",\\\"name\\\":\\\"run-ovn\\\"},{\\\"mountPath\\\":\\\"/env\\\",\\\"name\\\":\\\"env-overrides\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-x92h2\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:174f36cdd47ef0d1d2099482919d773257453265a2af0b17b154edc32fa41ac2\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"northd\\\",\\\"ready\\\":false,\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"PodInitializing\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/ovnkube-lib\\\",\\\"name\\\":\\\"ovnkube-script-lib\\\"},{\\\"mountPath\\\":\\\"/etc/ovn\\\",\\\"name\\\":\\\"etc-openvswitch\\\"},{\\\"mountPath\\\":\\\"/var/log/ovn\\\",\\\"name\\\":\\\"node-log\\\"},{\\\"mountPath\\\":\\\"/run/ovn/\\\",\\\"name\\\":\\\"run-ovn\\\"},{\\\"mountPath\\\":\\\"/env\\\",\\\"name\\\":\\\"env-overrides\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-x92h2\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:174f36cdd47ef0d1d2099482919d773257453265a2af0b17b154edc32fa41ac2\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"ovn-acl-logging\\\",\\\"ready\\\":false,\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"PodInitializing\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/ovnkube-lib\\\",\\\"name\\\":\\\"ovnkube-script-lib\\\"},{\\\"mountPath\\\":\\\"/var/log/ovn/\\\",\\\"name\\\":\\\"node-log\\\"},{\\\"mountPath\\\":\\\"/run/ovn/\\\",\\\"name\\\":\\\"run-ovn\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-x92h2\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:174f36cdd47ef0d1d2099482919d773257453265a2af0b17b154edc32fa41ac2\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"ovn-controller\\\",\\\"ready\\\":false,\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"PodInitializing\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/ovnkube-lib\\\",\\\"name\\\":\\\"ovnkube-script-lib\\\"},{\\\"mountPath\\\":\\\"/run/openvswitch\\\",\\\"name\\\":\\\"run-openvswitch\\\"},{\\\"mountPath\\\":\\\"/run/ovn/\\\",\\\"name\\\":\\\"run-ovn\\\"},{\\\"mountPath\\\":\\\"/etc/openvswitch\\\",\\\"name\\\":\\\"etc-openvswitch\\\"},{\\\"mountPath\\\":\\\"/etc/ovn/\\\",\\\"name\\\":\\\"etc-openvswitch\\\"},{\\\"mountPath\\\":\\\"/var/lib/openvswitch\\\",\\\"name\\\":\\\"var-lib-openvswitch\\\"},{\\\"mountPath\\\":\\\"/env\\\",\\\"name\\\":\\\"env-overrides\\\"},{\\\"mountPath\\\":\\\"/var/log/ovn/\\\",\\\"name\\\":\\\"node-log\\\"},{\\\"mountPath\\\":\\\"/dev/log\\\",\\\"name\\\":\\\"log-socket\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-x92h2\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:174f36cdd47ef0d1d2099482919d773257453265a2af0b17b154edc32fa41ac2\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"ovnkube-controller\\\",\\\"ready\\\":false,\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"PodInitializing\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/ovnkube-lib\\\",\\\"name\\\":\\\"ovnkube-script-lib\\\"},{\\\"mountPath\\\":\\\"/var/lib/kubelet\\\",\\\"name\\\":\\\"host-kubelet\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/etc/systemd/system\\\",\\\"name\\\":\\\"systemd-units\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/host\\\",\\\"name\\\":\\\"host-slash\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/run/ovn-kubernetes/\\\",\\\"name\\\":\\\"host-run-ovn-kubernetes\\\"},{\\\"mountPath\\\":\\\"/run/netns\\\",\\\"name\\\":\\\"host-run-netns\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/run/systemd/private\\\",\\\"name\\\":\\\"run-systemd\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/cni-bin-dir\\\",\\\"name\\\":\\\"host-cni-bin\\\"},{\\\"mountPath\\\":\\\"/etc/cni/net.d\\\",\\\"name\\\":\\\"host-cni-netd\\\"},{\\\"mountPath\\\":\\\"/var/lib/cni/networks/ovn-k8s-cni-overlay\\\",\\\"name\\\":\\\"host-var-lib-cni-networks-ovn-kubernetes\\\"},{\\\"mountPath\\\":\\\"/run/openvswitch\\\",\\\"name\\\":\\\"run-openvswitch\\\"},{\\\"mountPath\\\":\\\"/var/log/ovnkube/\\\",\\\"name\\\":\\\"etc-openvswitch\\\"},{\\\"mountPath\\\":\\\"/run/ovn/\\\",\\\"name\\\":\\\"run-ovn\\\"},{\\\"mountPath\\\":\\\"/etc/openvswitch\\\",\\\"name\\\":\\\"etc-openvswitch\\\"},{\\\"mountPath\\\":\\\"/etc/ovn/\\\",\\\"name\\\":\\\"etc-openvswitch\\\"},{\\\"mountPath\\\":\\\"/var/lib/openvswitch\\\",\\\"name\\\":\\\"var-lib-openvswitch\\\"},{\\\"mountPath\\\":\\\"/run/ovnkube-config/\\\",\\\"name\\\":\\\"ovnkube-config\\\"},{\\\"mountPath\\\":\\\"/env\\\",\\\"name\\\":\\\"env-overrides\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-x92h2\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:174f36cdd47ef0d1d2099482919d773257453265a2af0b17b154edc32fa41ac2\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"sbdb\\\",\\\"ready\\\":false,\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"PodInitializing\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/ovnkube-lib\\\",\\\"name\\\":\\\"ovnkube-script-lib\\\"},{\\\"mountPath\\\":\\\"/etc/ovn/\\\",\\\"name\\\":\\\"etc-openvswitch\\\"},{\\\"mountPath\\\":\\\"/run/ovn/\\\",\\\"name\\\":\\\"run-ovn\\\"},{\\\"mountPath\\\":\\\"/var/log/ovn\\\",\\\"name\\\":\\\"node-log\\\"},{\\\"mountPath\\\":\\\"/env\\\",\\\"name\\\":\\\"env-overrides\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-x92h2\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}],\\\"hostIP\\\":\\\"192.168.126.11\\\",\\\"hostIPs\\\":[{\\\"ip\\\":\\\"192.168.126.11\\\"}],\\\"initContainerStatuses\\\":[{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:174f36cdd47ef0d1d2099482919d773257453265a2af0b17b154edc32fa41ac2\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"kubecfg-setup\\\",\\\"ready\\\":false,\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"PodInitializing\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/etc/ovn/\\\",\\\"name\\\":\\\"etc-openvswitch\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-x92h2\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}],\\\"podIP\\\":\\\"192.168.126.11\\\",\\\"podIPs\\\":[{\\\"ip\\\":\\\"192.168.126.11\\\"}],\\\"startTime\\\":\\\"2025-12-03T11:20:31Z\\\"}}\" for pod \"openshift-ovn-kubernetes\"/\"ovnkube-node-kg55h\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://127.0.0.1:9743/pod?timeout=10s\": dial tcp 127.0.0.1:9743: connect: connection refused" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.749554 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"auth-proxy-config\" (UniqueName: \"kubernetes.io/configmap/22c825df-677d-4ca6-82db-3454ed06e783-auth-proxy-config\") pod \"22c825df-677d-4ca6-82db-3454ed06e783\" (UID: \"22c825df-677d-4ca6-82db-3454ed06e783\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.749628 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/c03ee662-fb2f-4fc4-a2c1-af487c19d254-stats-auth\") pod \"c03ee662-fb2f-4fc4-a2c1-af487c19d254\" (UID: \"c03ee662-fb2f-4fc4-a2c1-af487c19d254\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.749659 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"etcd-serving-ca\" (UniqueName: \"kubernetes.io/configmap/1bf7eb37-55a3-4c65-b768-a94c82151e69-etcd-serving-ca\") pod \"1bf7eb37-55a3-4c65-b768-a94c82151e69\" (UID: \"1bf7eb37-55a3-4c65-b768-a94c82151e69\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.749683 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"client-ca\" (UniqueName: \"kubernetes.io/configmap/7583ce53-e0fe-4a16-9e4d-50516596a136-client-ca\") pod \"7583ce53-e0fe-4a16-9e4d-50516596a136\" (UID: \"7583ce53-e0fe-4a16-9e4d-50516596a136\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.749707 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/20b0d48f-5fd6-431c-a545-e3c800c7b866-cert\") pod \"20b0d48f-5fd6-431c-a545-e3c800c7b866\" (UID: \"20b0d48f-5fd6-431c-a545-e3c800c7b866\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.749733 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d6qdx\" (UniqueName: \"kubernetes.io/projected/87cf06ed-a83f-41a7-828d-70653580a8cb-kube-api-access-d6qdx\") pod \"87cf06ed-a83f-41a7-828d-70653580a8cb\" (UID: \"87cf06ed-a83f-41a7-828d-70653580a8cb\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.749756 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/43509403-f426-496e-be36-56cef71462f5-oauth-serving-cert\") pod \"43509403-f426-496e-be36-56cef71462f5\" (UID: \"43509403-f426-496e-be36-56cef71462f5\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.749785 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e7e6199b-1264-4501-8953-767f51328d08-config\") pod \"e7e6199b-1264-4501-8953-767f51328d08\" (UID: \"e7e6199b-1264-4501-8953-767f51328d08\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.749808 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/3cb93b32-e0ae-4377-b9c8-fdb9842c6d59-serviceca\") pod \"3cb93b32-e0ae-4377-b9c8-fdb9842c6d59\" (UID: \"3cb93b32-e0ae-4377-b9c8-fdb9842c6d59\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.749829 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"srv-cert\" (UniqueName: \"kubernetes.io/secret/b6312bbd-5731-4ea0-a20f-81d5a57df44a-srv-cert\") pod \"b6312bbd-5731-4ea0-a20f-81d5a57df44a\" (UID: \"b6312bbd-5731-4ea0-a20f-81d5a57df44a\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.749850 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/496e6271-fb68-4057-954e-a0d97a4afa3f-serving-cert\") pod \"496e6271-fb68-4057-954e-a0d97a4afa3f\" (UID: \"496e6271-fb68-4057-954e-a0d97a4afa3f\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.749878 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"auth-proxy-config\" (UniqueName: \"kubernetes.io/configmap/31d8b7a1-420e-4252-a5b7-eebe8a111292-auth-proxy-config\") pod \"31d8b7a1-420e-4252-a5b7-eebe8a111292\" (UID: \"31d8b7a1-420e-4252-a5b7-eebe8a111292\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.749906 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"marketplace-operator-metrics\" (UniqueName: \"kubernetes.io/secret/b6cd30de-2eeb-49a2-ab40-9167f4560ff5-marketplace-operator-metrics\") pod \"b6cd30de-2eeb-49a2-ab40-9167f4560ff5\" (UID: \"b6cd30de-2eeb-49a2-ab40-9167f4560ff5\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.749930 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"v4-0-config-system-ocp-branding-template\" (UniqueName: \"kubernetes.io/secret/49c341d1-5089-4bc2-86a0-a5e165cfcc6b-v4-0-config-system-ocp-branding-template\") pod \"49c341d1-5089-4bc2-86a0-a5e165cfcc6b\" (UID: \"49c341d1-5089-4bc2-86a0-a5e165cfcc6b\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.749954 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qs4fp\" (UniqueName: \"kubernetes.io/projected/210d8245-ebfc-4e3b-ac4a-e21ce76f9a7c-kube-api-access-qs4fp\") pod \"210d8245-ebfc-4e3b-ac4a-e21ce76f9a7c\" (UID: \"210d8245-ebfc-4e3b-ac4a-e21ce76f9a7c\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.749980 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/308be0ea-9f5f-4b29-aeb1-5abd31a0b17b-apiservice-cert\") pod \"308be0ea-9f5f-4b29-aeb1-5abd31a0b17b\" (UID: \"308be0ea-9f5f-4b29-aeb1-5abd31a0b17b\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.750004 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"audit-policies\" (UniqueName: \"kubernetes.io/configmap/49c341d1-5089-4bc2-86a0-a5e165cfcc6b-audit-policies\") pod \"49c341d1-5089-4bc2-86a0-a5e165cfcc6b\" (UID: \"49c341d1-5089-4bc2-86a0-a5e165cfcc6b\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.750030 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8f668bae-612b-4b75-9490-919e737c6a3b-installation-pull-secrets\") pod \"8f668bae-612b-4b75-9490-919e737c6a3b\" (UID: \"8f668bae-612b-4b75-9490-919e737c6a3b\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.750089 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ngvvp\" (UniqueName: \"kubernetes.io/projected/49c341d1-5089-4bc2-86a0-a5e165cfcc6b-kube-api-access-ngvvp\") pod \"49c341d1-5089-4bc2-86a0-a5e165cfcc6b\" (UID: \"49c341d1-5089-4bc2-86a0-a5e165cfcc6b\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.750118 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5b88f790-22fa-440e-b583-365168c0b23d-metrics-certs\") pod \"5b88f790-22fa-440e-b583-365168c0b23d\" (UID: \"5b88f790-22fa-440e-b583-365168c0b23d\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.750145 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a0128f3a-b052-44ed-a84e-c4c8aaf17c13-samples-operator-tls\") pod \"a0128f3a-b052-44ed-a84e-c4c8aaf17c13\" (UID: \"a0128f3a-b052-44ed-a84e-c4c8aaf17c13\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.750185 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x7zkh\" (UniqueName: \"kubernetes.io/projected/6731426b-95fe-49ff-bb5f-40441049fde2-kube-api-access-x7zkh\") pod \"6731426b-95fe-49ff-bb5f-40441049fde2\" (UID: \"6731426b-95fe-49ff-bb5f-40441049fde2\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.750211 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w9rds\" (UniqueName: \"kubernetes.io/projected/20b0d48f-5fd6-431c-a545-e3c800c7b866-kube-api-access-w9rds\") pod \"20b0d48f-5fd6-431c-a545-e3c800c7b866\" (UID: \"20b0d48f-5fd6-431c-a545-e3c800c7b866\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.750237 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7539238d-5fe0-46ed-884e-1c3b566537ec-serving-cert\") pod \"7539238d-5fe0-46ed-884e-1c3b566537ec\" (UID: \"7539238d-5fe0-46ed-884e-1c3b566537ec\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.750259 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dbsvg\" (UniqueName: \"kubernetes.io/projected/f88749ec-7931-4ee7-b3fc-1ec5e11f92e9-kube-api-access-dbsvg\") pod \"f88749ec-7931-4ee7-b3fc-1ec5e11f92e9\" (UID: \"f88749ec-7931-4ee7-b3fc-1ec5e11f92e9\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.750285 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5441d097-087c-4d9a-baa8-b210afa90fc9-serving-cert\") pod \"5441d097-087c-4d9a-baa8-b210afa90fc9\" (UID: \"5441d097-087c-4d9a-baa8-b210afa90fc9\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.750310 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"v4-0-config-user-template-provider-selection\" (UniqueName: \"kubernetes.io/secret/49c341d1-5089-4bc2-86a0-a5e165cfcc6b-v4-0-config-user-template-provider-selection\") pod \"49c341d1-5089-4bc2-86a0-a5e165cfcc6b\" (UID: \"49c341d1-5089-4bc2-86a0-a5e165cfcc6b\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.750334 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/6ea678ab-3438-413e-bfe3-290ae7725660-ovnkube-script-lib\") pod \"6ea678ab-3438-413e-bfe3-290ae7725660\" (UID: \"6ea678ab-3438-413e-bfe3-290ae7725660\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.750357 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jhbk2\" (UniqueName: \"kubernetes.io/projected/bd23aa5c-e532-4e53-bccf-e79f130c5ae8-kube-api-access-jhbk2\") pod \"bd23aa5c-e532-4e53-bccf-e79f130c5ae8\" (UID: \"bd23aa5c-e532-4e53-bccf-e79f130c5ae8\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.750397 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/308be0ea-9f5f-4b29-aeb1-5abd31a0b17b-webhook-cert\") pod \"308be0ea-9f5f-4b29-aeb1-5abd31a0b17b\" (UID: \"308be0ea-9f5f-4b29-aeb1-5abd31a0b17b\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.750421 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xcphl\" (UniqueName: \"kubernetes.io/projected/7583ce53-e0fe-4a16-9e4d-50516596a136-kube-api-access-xcphl\") pod \"7583ce53-e0fe-4a16-9e4d-50516596a136\" (UID: \"7583ce53-e0fe-4a16-9e4d-50516596a136\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.750443 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bf2bz\" (UniqueName: \"kubernetes.io/projected/1d611f23-29be-4491-8495-bee1670e935f-kube-api-access-bf2bz\") pod \"1d611f23-29be-4491-8495-bee1670e935f\" (UID: \"1d611f23-29be-4491-8495-bee1670e935f\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.750469 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e7e6199b-1264-4501-8953-767f51328d08-serving-cert\") pod \"e7e6199b-1264-4501-8953-767f51328d08\" (UID: \"e7e6199b-1264-4501-8953-767f51328d08\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.750520 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8cea82b4-6893-4ddc-af9f-1bb5ae425c5b-config\") pod \"8cea82b4-6893-4ddc-af9f-1bb5ae425c5b\" (UID: \"8cea82b4-6893-4ddc-af9f-1bb5ae425c5b\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.750565 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/1d611f23-29be-4491-8495-bee1670e935f-catalog-content\") pod \"1d611f23-29be-4491-8495-bee1670e935f\" (UID: \"1d611f23-29be-4491-8495-bee1670e935f\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.750588 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/6402fda4-df10-493c-b4e5-d0569419652d-config\") pod \"6402fda4-df10-493c-b4e5-d0569419652d\" (UID: \"6402fda4-df10-493c-b4e5-d0569419652d\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.750612 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"webhook-certs\" (UniqueName: \"kubernetes.io/secret/efdd0498-1daa-4136-9a4a-3b948c2293fc-webhook-certs\") pod \"efdd0498-1daa-4136-9a4a-3b948c2293fc\" (UID: \"efdd0498-1daa-4136-9a4a-3b948c2293fc\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.750640 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/5225d0e4-402f-4861-b410-819f433b1803-catalog-content\") pod \"5225d0e4-402f-4861-b410-819f433b1803\" (UID: \"5225d0e4-402f-4861-b410-819f433b1803\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.750664 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1386a44e-36a2-460c-96d0-0359d2b6f0f5-serving-cert\") pod \"1386a44e-36a2-460c-96d0-0359d2b6f0f5\" (UID: \"1386a44e-36a2-460c-96d0-0359d2b6f0f5\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.750691 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pj782\" (UniqueName: \"kubernetes.io/projected/b6cd30de-2eeb-49a2-ab40-9167f4560ff5-kube-api-access-pj782\") pod \"b6cd30de-2eeb-49a2-ab40-9167f4560ff5\" (UID: \"b6cd30de-2eeb-49a2-ab40-9167f4560ff5\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.750720 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/43509403-f426-496e-be36-56cef71462f5-console-serving-cert\") pod \"43509403-f426-496e-be36-56cef71462f5\" (UID: \"43509403-f426-496e-be36-56cef71462f5\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.750744 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d-catalog-content\") pod \"b11524ee-3fca-4b1b-9cdf-6da289fdbc7d\" (UID: \"b11524ee-3fca-4b1b-9cdf-6da289fdbc7d\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.750769 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"v4-0-config-user-idp-0-file-data\" (UniqueName: \"kubernetes.io/secret/49c341d1-5089-4bc2-86a0-a5e165cfcc6b-v4-0-config-user-idp-0-file-data\") pod \"49c341d1-5089-4bc2-86a0-a5e165cfcc6b\" (UID: \"49c341d1-5089-4bc2-86a0-a5e165cfcc6b\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.750767 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/22c825df-677d-4ca6-82db-3454ed06e783-auth-proxy-config" (OuterVolumeSpecName: "auth-proxy-config") pod "22c825df-677d-4ca6-82db-3454ed06e783" (UID: "22c825df-677d-4ca6-82db-3454ed06e783"). InnerVolumeSpecName "auth-proxy-config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.750792 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8f668bae-612b-4b75-9490-919e737c6a3b-registry-tls\") pod \"8f668bae-612b-4b75-9490-919e737c6a3b\" (UID: \"8f668bae-612b-4b75-9490-919e737c6a3b\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.750890 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d-utilities\") pod \"b11524ee-3fca-4b1b-9cdf-6da289fdbc7d\" (UID: \"b11524ee-3fca-4b1b-9cdf-6da289fdbc7d\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.750918 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"machine-api-operator-tls\" (UniqueName: \"kubernetes.io/secret/6402fda4-df10-493c-b4e5-d0569419652d-machine-api-operator-tls\") pod \"6402fda4-df10-493c-b4e5-d0569419652d\" (UID: \"6402fda4-df10-493c-b4e5-d0569419652d\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.750941 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a31745f5-9847-4afe-82a5-3161cc66ca93-trusted-ca\") pod \"a31745f5-9847-4afe-82a5-3161cc66ca93\" (UID: \"a31745f5-9847-4afe-82a5-3161cc66ca93\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.750961 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fcqwp\" (UniqueName: \"kubernetes.io/projected/5fe579f8-e8a6-4643-bce5-a661393c4dde-kube-api-access-fcqwp\") pod \"5fe579f8-e8a6-4643-bce5-a661393c4dde\" (UID: \"5fe579f8-e8a6-4643-bce5-a661393c4dde\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.750979 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-279lb\" (UniqueName: \"kubernetes.io/projected/7bb08738-c794-4ee8-9972-3a62ca171029-kube-api-access-279lb\") pod \"7bb08738-c794-4ee8-9972-3a62ca171029\" (UID: \"7bb08738-c794-4ee8-9972-3a62ca171029\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.750967 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8f668bae-612b-4b75-9490-919e737c6a3b-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "8f668bae-612b-4b75-9490-919e737c6a3b" (UID: "8f668bae-612b-4b75-9490-919e737c6a3b"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.750998 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/96b93a3a-6083-4aea-8eab-fe1aa8245ad9-metrics-tls\") pod \"96b93a3a-6083-4aea-8eab-fe1aa8245ad9\" (UID: \"96b93a3a-6083-4aea-8eab-fe1aa8245ad9\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.751027 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6509e943-70c6-444c-bc41-48a544e36fbd-service-ca-bundle\") pod \"6509e943-70c6-444c-bc41-48a544e36fbd\" (UID: \"6509e943-70c6-444c-bc41-48a544e36fbd\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.751053 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/43509403-f426-496e-be36-56cef71462f5-service-ca\") pod \"43509403-f426-496e-be36-56cef71462f5\" (UID: \"43509403-f426-496e-be36-56cef71462f5\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.751073 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"encryption-config\" (UniqueName: \"kubernetes.io/secret/1bf7eb37-55a3-4c65-b768-a94c82151e69-encryption-config\") pod \"1bf7eb37-55a3-4c65-b768-a94c82151e69\" (UID: \"1bf7eb37-55a3-4c65-b768-a94c82151e69\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.751090 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"machine-approver-tls\" (UniqueName: \"kubernetes.io/secret/22c825df-677d-4ca6-82db-3454ed06e783-machine-approver-tls\") pod \"22c825df-677d-4ca6-82db-3454ed06e783\" (UID: \"22c825df-677d-4ca6-82db-3454ed06e783\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.751109 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"srv-cert\" (UniqueName: \"kubernetes.io/secret/f88749ec-7931-4ee7-b3fc-1ec5e11f92e9-srv-cert\") pod \"f88749ec-7931-4ee7-b3fc-1ec5e11f92e9\" (UID: \"f88749ec-7931-4ee7-b3fc-1ec5e11f92e9\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.751128 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w4xd4\" (UniqueName: \"kubernetes.io/projected/8cea82b4-6893-4ddc-af9f-1bb5ae425c5b-kube-api-access-w4xd4\") pod \"8cea82b4-6893-4ddc-af9f-1bb5ae425c5b\" (UID: \"8cea82b4-6893-4ddc-af9f-1bb5ae425c5b\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.751124 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8f668bae-612b-4b75-9490-919e737c6a3b-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "8f668bae-612b-4b75-9490-919e737c6a3b" (UID: "8f668bae-612b-4b75-9490-919e737c6a3b"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.751148 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lzf88\" (UniqueName: \"kubernetes.io/projected/0b574797-001e-440a-8f4e-c0be86edad0f-kube-api-access-lzf88\") pod \"0b574797-001e-440a-8f4e-c0be86edad0f\" (UID: \"0b574797-001e-440a-8f4e-c0be86edad0f\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.751158 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c03ee662-fb2f-4fc4-a2c1-af487c19d254-stats-auth" (OuterVolumeSpecName: "stats-auth") pod "c03ee662-fb2f-4fc4-a2c1-af487c19d254" (UID: "c03ee662-fb2f-4fc4-a2c1-af487c19d254"). InnerVolumeSpecName "stats-auth". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.751178 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-249nr\" (UniqueName: \"kubernetes.io/projected/b6312bbd-5731-4ea0-a20f-81d5a57df44a-kube-api-access-249nr\") pod \"b6312bbd-5731-4ea0-a20f-81d5a57df44a\" (UID: \"b6312bbd-5731-4ea0-a20f-81d5a57df44a\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.751201 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"encryption-config\" (UniqueName: \"kubernetes.io/secret/09ae3b1a-e8e7-4524-b54b-61eab6f9239a-encryption-config\") pod \"09ae3b1a-e8e7-4524-b54b-61eab6f9239a\" (UID: \"09ae3b1a-e8e7-4524-b54b-61eab6f9239a\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.751218 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vt5rc\" (UniqueName: \"kubernetes.io/projected/44663579-783b-4372-86d6-acf235a62d72-kube-api-access-vt5rc\") pod \"44663579-783b-4372-86d6-acf235a62d72\" (UID: \"44663579-783b-4372-86d6-acf235a62d72\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.751238 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/4bb40260-dbaa-4fb0-84df-5e680505d512-multus-daemon-config\") pod \"4bb40260-dbaa-4fb0-84df-5e680505d512\" (UID: \"4bb40260-dbaa-4fb0-84df-5e680505d512\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.751260 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/09efc573-dbb6-4249-bd59-9b87aba8dd28-config\") pod \"09efc573-dbb6-4249-bd59-9b87aba8dd28\" (UID: \"09efc573-dbb6-4249-bd59-9b87aba8dd28\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.751280 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/7583ce53-e0fe-4a16-9e4d-50516596a136-config\") pod \"7583ce53-e0fe-4a16-9e4d-50516596a136\" (UID: \"7583ce53-e0fe-4a16-9e4d-50516596a136\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.751298 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/6ea678ab-3438-413e-bfe3-290ae7725660-env-overrides\") pod \"6ea678ab-3438-413e-bfe3-290ae7725660\" (UID: \"6ea678ab-3438-413e-bfe3-290ae7725660\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.751316 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x4zgh\" (UniqueName: \"kubernetes.io/projected/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d-kube-api-access-x4zgh\") pod \"b11524ee-3fca-4b1b-9cdf-6da289fdbc7d\" (UID: \"b11524ee-3fca-4b1b-9cdf-6da289fdbc7d\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.751333 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/210d8245-ebfc-4e3b-ac4a-e21ce76f9a7c-config\") pod \"210d8245-ebfc-4e3b-ac4a-e21ce76f9a7c\" (UID: \"210d8245-ebfc-4e3b-ac4a-e21ce76f9a7c\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.751354 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/1d611f23-29be-4491-8495-bee1670e935f-utilities\") pod \"1d611f23-29be-4491-8495-bee1670e935f\" (UID: \"1d611f23-29be-4491-8495-bee1670e935f\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.751371 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"etcd-ca\" (UniqueName: \"kubernetes.io/configmap/09efc573-dbb6-4249-bd59-9b87aba8dd28-etcd-ca\") pod \"09efc573-dbb6-4249-bd59-9b87aba8dd28\" (UID: \"09efc573-dbb6-4249-bd59-9b87aba8dd28\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.751385 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/49c341d1-5089-4bc2-86a0-a5e165cfcc6b-kube-api-access-ngvvp" (OuterVolumeSpecName: "kube-api-access-ngvvp") pod "49c341d1-5089-4bc2-86a0-a5e165cfcc6b" (UID: "49c341d1-5089-4bc2-86a0-a5e165cfcc6b"). InnerVolumeSpecName "kube-api-access-ngvvp". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.751390 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/6ea678ab-3438-413e-bfe3-290ae7725660-ovn-node-metrics-cert\") pod \"6ea678ab-3438-413e-bfe3-290ae7725660\" (UID: \"6ea678ab-3438-413e-bfe3-290ae7725660\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.751438 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"tmpfs\" (UniqueName: \"kubernetes.io/empty-dir/308be0ea-9f5f-4b29-aeb1-5abd31a0b17b-tmpfs\") pod \"308be0ea-9f5f-4b29-aeb1-5abd31a0b17b\" (UID: \"308be0ea-9f5f-4b29-aeb1-5abd31a0b17b\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.751468 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/fda69060-fa79-4696-b1a6-7980f124bf7c-proxy-tls\") pod \"fda69060-fa79-4696-b1a6-7980f124bf7c\" (UID: \"fda69060-fa79-4696-b1a6-7980f124bf7c\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.751500 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/496e6271-fb68-4057-954e-a0d97a4afa3f-config\") pod \"496e6271-fb68-4057-954e-a0d97a4afa3f\" (UID: \"496e6271-fb68-4057-954e-a0d97a4afa3f\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.751548 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/6509e943-70c6-444c-bc41-48a544e36fbd-config\") pod \"6509e943-70c6-444c-bc41-48a544e36fbd\" (UID: \"6509e943-70c6-444c-bc41-48a544e36fbd\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.751575 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zgdk5\" (UniqueName: \"kubernetes.io/projected/31d8b7a1-420e-4252-a5b7-eebe8a111292-kube-api-access-zgdk5\") pod \"31d8b7a1-420e-4252-a5b7-eebe8a111292\" (UID: \"31d8b7a1-420e-4252-a5b7-eebe8a111292\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.751598 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/43509403-f426-496e-be36-56cef71462f5-trusted-ca-bundle\") pod \"43509403-f426-496e-be36-56cef71462f5\" (UID: \"43509403-f426-496e-be36-56cef71462f5\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.751628 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access\" (UniqueName: \"kubernetes.io/projected/e7e6199b-1264-4501-8953-767f51328d08-kube-api-access\") pod \"e7e6199b-1264-4501-8953-767f51328d08\" (UID: \"e7e6199b-1264-4501-8953-767f51328d08\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.751653 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7bb08738-c794-4ee8-9972-3a62ca171029-cni-binary-copy\") pod \"7bb08738-c794-4ee8-9972-3a62ca171029\" (UID: \"7bb08738-c794-4ee8-9972-3a62ca171029\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.751677 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gf66m\" (UniqueName: \"kubernetes.io/projected/a0128f3a-b052-44ed-a84e-c4c8aaf17c13-kube-api-access-gf66m\") pod \"a0128f3a-b052-44ed-a84e-c4c8aaf17c13\" (UID: \"a0128f3a-b052-44ed-a84e-c4c8aaf17c13\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.751725 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"client-ca\" (UniqueName: \"kubernetes.io/configmap/5441d097-087c-4d9a-baa8-b210afa90fc9-client-ca\") pod \"5441d097-087c-4d9a-baa8-b210afa90fc9\" (UID: \"5441d097-087c-4d9a-baa8-b210afa90fc9\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.751780 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/31d8b7a1-420e-4252-a5b7-eebe8a111292-proxy-tls\") pod \"31d8b7a1-420e-4252-a5b7-eebe8a111292\" (UID: \"31d8b7a1-420e-4252-a5b7-eebe8a111292\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.751805 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8f668bae-612b-4b75-9490-919e737c6a3b-registry-certificates\") pod \"8f668bae-612b-4b75-9490-919e737c6a3b\" (UID: \"8f668bae-612b-4b75-9490-919e737c6a3b\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.751835 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"etcd-client\" (UniqueName: \"kubernetes.io/secret/1bf7eb37-55a3-4c65-b768-a94c82151e69-etcd-client\") pod \"1bf7eb37-55a3-4c65-b768-a94c82151e69\" (UID: \"1bf7eb37-55a3-4c65-b768-a94c82151e69\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.751857 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"proxy-ca-bundles\" (UniqueName: \"kubernetes.io/configmap/7583ce53-e0fe-4a16-9e4d-50516596a136-proxy-ca-bundles\") pod \"7583ce53-e0fe-4a16-9e4d-50516596a136\" (UID: \"7583ce53-e0fe-4a16-9e4d-50516596a136\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.751881 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access\" (UniqueName: \"kubernetes.io/projected/496e6271-fb68-4057-954e-a0d97a4afa3f-kube-api-access\") pod \"496e6271-fb68-4057-954e-a0d97a4afa3f\" (UID: \"496e6271-fb68-4057-954e-a0d97a4afa3f\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.751906 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jkwtn\" (UniqueName: \"kubernetes.io/projected/5b88f790-22fa-440e-b583-365168c0b23d-kube-api-access-jkwtn\") pod \"5b88f790-22fa-440e-b583-365168c0b23d\" (UID: \"5b88f790-22fa-440e-b583-365168c0b23d\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.751930 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pjr6v\" (UniqueName: \"kubernetes.io/projected/49ef4625-1d3a-4a9f-b595-c2433d32326d-kube-api-access-pjr6v\") pod \"49ef4625-1d3a-4a9f-b595-c2433d32326d\" (UID: \"49ef4625-1d3a-4a9f-b595-c2433d32326d\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.751955 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/09ae3b1a-e8e7-4524-b54b-61eab6f9239a-trusted-ca-bundle\") pod \"09ae3b1a-e8e7-4524-b54b-61eab6f9239a\" (UID: \"09ae3b1a-e8e7-4524-b54b-61eab6f9239a\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.751984 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1bf7eb37-55a3-4c65-b768-a94c82151e69-trusted-ca-bundle\") pod \"1bf7eb37-55a3-4c65-b768-a94c82151e69\" (UID: \"1bf7eb37-55a3-4c65-b768-a94c82151e69\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.752050 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"v4-0-config-system-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/49c341d1-5089-4bc2-86a0-a5e165cfcc6b-v4-0-config-system-trusted-ca-bundle\") pod \"49c341d1-5089-4bc2-86a0-a5e165cfcc6b\" (UID: \"49c341d1-5089-4bc2-86a0-a5e165cfcc6b\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.752077 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"images\" (UniqueName: \"kubernetes.io/configmap/31d8b7a1-420e-4252-a5b7-eebe8a111292-images\") pod \"31d8b7a1-420e-4252-a5b7-eebe8a111292\" (UID: \"31d8b7a1-420e-4252-a5b7-eebe8a111292\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.752103 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d4lsv\" (UniqueName: \"kubernetes.io/projected/25e176fe-21b4-4974-b1ed-c8b94f112a7f-kube-api-access-d4lsv\") pod \"25e176fe-21b4-4974-b1ed-c8b94f112a7f\" (UID: \"25e176fe-21b4-4974-b1ed-c8b94f112a7f\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.752138 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2w9zh\" (UniqueName: \"kubernetes.io/projected/4bb40260-dbaa-4fb0-84df-5e680505d512-kube-api-access-2w9zh\") pod \"4bb40260-dbaa-4fb0-84df-5e680505d512\" (UID: \"4bb40260-dbaa-4fb0-84df-5e680505d512\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.752172 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/5225d0e4-402f-4861-b410-819f433b1803-utilities\") pod \"5225d0e4-402f-4861-b410-819f433b1803\" (UID: \"5225d0e4-402f-4861-b410-819f433b1803\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.752197 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mg5zb\" (UniqueName: \"kubernetes.io/projected/6402fda4-df10-493c-b4e5-d0569419652d-kube-api-access-mg5zb\") pod \"6402fda4-df10-493c-b4e5-d0569419652d\" (UID: \"6402fda4-df10-493c-b4e5-d0569419652d\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.752221 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1bf7eb37-55a3-4c65-b768-a94c82151e69-config\") pod \"1bf7eb37-55a3-4c65-b768-a94c82151e69\" (UID: \"1bf7eb37-55a3-4c65-b768-a94c82151e69\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.752244 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bf126b07-da06-4140-9a57-dfd54fc6b486-bound-sa-token\") pod \"bf126b07-da06-4140-9a57-dfd54fc6b486\" (UID: \"bf126b07-da06-4140-9a57-dfd54fc6b486\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.752268 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"etcd-client\" (UniqueName: \"kubernetes.io/secret/09ae3b1a-e8e7-4524-b54b-61eab6f9239a-etcd-client\") pod \"09ae3b1a-e8e7-4524-b54b-61eab6f9239a\" (UID: \"09ae3b1a-e8e7-4524-b54b-61eab6f9239a\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.752290 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/09efc573-dbb6-4249-bd59-9b87aba8dd28-serving-cert\") pod \"09efc573-dbb6-4249-bd59-9b87aba8dd28\" (UID: \"09efc573-dbb6-4249-bd59-9b87aba8dd28\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.752315 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"profile-collector-cert\" (UniqueName: \"kubernetes.io/secret/f88749ec-7931-4ee7-b3fc-1ec5e11f92e9-profile-collector-cert\") pod \"f88749ec-7931-4ee7-b3fc-1ec5e11f92e9\" (UID: \"f88749ec-7931-4ee7-b3fc-1ec5e11f92e9\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.752339 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"images\" (UniqueName: \"kubernetes.io/configmap/6402fda4-df10-493c-b4e5-d0569419652d-images\") pod \"6402fda4-df10-493c-b4e5-d0569419652d\" (UID: \"6402fda4-df10-493c-b4e5-d0569419652d\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.752357 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9xfj7\" (UniqueName: \"kubernetes.io/projected/5225d0e4-402f-4861-b410-819f433b1803-kube-api-access-9xfj7\") pod \"5225d0e4-402f-4861-b410-819f433b1803\" (UID: \"5225d0e4-402f-4861-b410-819f433b1803\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.752377 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9d4552c7-cd75-42dd-8880-30dd377c49a4-trusted-ca\") pod \"9d4552c7-cd75-42dd-8880-30dd377c49a4\" (UID: \"9d4552c7-cd75-42dd-8880-30dd377c49a4\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.752403 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"audit-policies\" (UniqueName: \"kubernetes.io/configmap/09ae3b1a-e8e7-4524-b54b-61eab6f9239a-audit-policies\") pod \"09ae3b1a-e8e7-4524-b54b-61eab6f9239a\" (UID: \"09ae3b1a-e8e7-4524-b54b-61eab6f9239a\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.752427 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"v4-0-config-system-cliconfig\" (UniqueName: \"kubernetes.io/configmap/49c341d1-5089-4bc2-86a0-a5e165cfcc6b-v4-0-config-system-cliconfig\") pod \"49c341d1-5089-4bc2-86a0-a5e165cfcc6b\" (UID: \"49c341d1-5089-4bc2-86a0-a5e165cfcc6b\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.752450 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rnphk\" (UniqueName: \"kubernetes.io/projected/bf126b07-da06-4140-9a57-dfd54fc6b486-kube-api-access-rnphk\") pod \"bf126b07-da06-4140-9a57-dfd54fc6b486\" (UID: \"bf126b07-da06-4140-9a57-dfd54fc6b486\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.752475 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1bf7eb37-55a3-4c65-b768-a94c82151e69-serving-cert\") pod \"1bf7eb37-55a3-4c65-b768-a94c82151e69\" (UID: \"1bf7eb37-55a3-4c65-b768-a94c82151e69\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.752497 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/01ab3dd5-8196-46d0-ad33-122e2ca51def-serving-cert\") pod \"01ab3dd5-8196-46d0-ad33-122e2ca51def\" (UID: \"01ab3dd5-8196-46d0-ad33-122e2ca51def\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.752541 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"package-server-manager-serving-cert\" (UniqueName: \"kubernetes.io/secret/3ab1a177-2de0-46d9-b765-d0d0649bb42e-package-server-manager-serving-cert\") pod \"3ab1a177-2de0-46d9-b765-d0d0649bb42e\" (UID: \"3ab1a177-2de0-46d9-b765-d0d0649bb42e\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.752568 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/01ab3dd5-8196-46d0-ad33-122e2ca51def-config\") pod \"01ab3dd5-8196-46d0-ad33-122e2ca51def\" (UID: \"01ab3dd5-8196-46d0-ad33-122e2ca51def\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.752590 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/43509403-f426-496e-be36-56cef71462f5-console-config\") pod \"43509403-f426-496e-be36-56cef71462f5\" (UID: \"43509403-f426-496e-be36-56cef71462f5\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.752613 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"node-bootstrap-token\" (UniqueName: \"kubernetes.io/secret/5fe579f8-e8a6-4643-bce5-a661393c4dde-node-bootstrap-token\") pod \"5fe579f8-e8a6-4643-bce5-a661393c4dde\" (UID: \"5fe579f8-e8a6-4643-bce5-a661393c4dde\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.752639 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mnrrd\" (UniqueName: \"kubernetes.io/projected/bc5039c0-ea34-426b-a2b7-fbbc87b49a6d-kube-api-access-mnrrd\") pod \"bc5039c0-ea34-426b-a2b7-fbbc87b49a6d\" (UID: \"bc5039c0-ea34-426b-a2b7-fbbc87b49a6d\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.752665 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w7l8j\" (UniqueName: \"kubernetes.io/projected/01ab3dd5-8196-46d0-ad33-122e2ca51def-kube-api-access-w7l8j\") pod \"01ab3dd5-8196-46d0-ad33-122e2ca51def\" (UID: \"01ab3dd5-8196-46d0-ad33-122e2ca51def\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.752697 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0b78653f-4ff9-4508-8672-245ed9b561e3-serving-cert\") pod \"0b78653f-4ff9-4508-8672-245ed9b561e3\" (UID: \"0b78653f-4ff9-4508-8672-245ed9b561e3\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.752720 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9d4552c7-cd75-42dd-8880-30dd377c49a4-serving-cert\") pod \"9d4552c7-cd75-42dd-8880-30dd377c49a4\" (UID: \"9d4552c7-cd75-42dd-8880-30dd377c49a4\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.752742 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/925f1c65-6136-48ba-85aa-3a3b50560753-ovnkube-config\") pod \"925f1c65-6136-48ba-85aa-3a3b50560753\" (UID: \"925f1c65-6136-48ba-85aa-3a3b50560753\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.752785 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6509e943-70c6-444c-bc41-48a544e36fbd-trusted-ca-bundle\") pod \"6509e943-70c6-444c-bc41-48a544e36fbd\" (UID: \"6509e943-70c6-444c-bc41-48a544e36fbd\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.752804 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/7bb08738-c794-4ee8-9972-3a62ca171029-cni-sysctl-allowlist\") pod \"7bb08738-c794-4ee8-9972-3a62ca171029\" (UID: \"7bb08738-c794-4ee8-9972-3a62ca171029\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.752822 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tk88c\" (UniqueName: \"kubernetes.io/projected/7539238d-5fe0-46ed-884e-1c3b566537ec-kube-api-access-tk88c\") pod \"7539238d-5fe0-46ed-884e-1c3b566537ec\" (UID: \"7539238d-5fe0-46ed-884e-1c3b566537ec\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.752850 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sb6h7\" (UniqueName: \"kubernetes.io/projected/1bf7eb37-55a3-4c65-b768-a94c82151e69-kube-api-access-sb6h7\") pod \"1bf7eb37-55a3-4c65-b768-a94c82151e69\" (UID: \"1bf7eb37-55a3-4c65-b768-a94c82151e69\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.752877 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4d4hj\" (UniqueName: \"kubernetes.io/projected/3ab1a177-2de0-46d9-b765-d0d0649bb42e-kube-api-access-4d4hj\") pod \"3ab1a177-2de0-46d9-b765-d0d0649bb42e\" (UID: \"3ab1a177-2de0-46d9-b765-d0d0649bb42e\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.752901 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/9d4552c7-cd75-42dd-8880-30dd377c49a4-config\") pod \"9d4552c7-cd75-42dd-8880-30dd377c49a4\" (UID: \"9d4552c7-cd75-42dd-8880-30dd377c49a4\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.752928 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovn-control-plane-metrics-cert\" (UniqueName: \"kubernetes.io/secret/925f1c65-6136-48ba-85aa-3a3b50560753-ovn-control-plane-metrics-cert\") pod \"925f1c65-6136-48ba-85aa-3a3b50560753\" (UID: \"925f1c65-6136-48ba-85aa-3a3b50560753\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.753483 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/87cf06ed-a83f-41a7-828d-70653580a8cb-config-volume\") pod \"87cf06ed-a83f-41a7-828d-70653580a8cb\" (UID: \"87cf06ed-a83f-41a7-828d-70653580a8cb\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.753533 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8tdtz\" (UniqueName: \"kubernetes.io/projected/09efc573-dbb6-4249-bd59-9b87aba8dd28-kube-api-access-8tdtz\") pod \"09efc573-dbb6-4249-bd59-9b87aba8dd28\" (UID: \"09efc573-dbb6-4249-bd59-9b87aba8dd28\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.753934 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x2m85\" (UniqueName: \"kubernetes.io/projected/cd70aa09-68dd-4d64-bd6f-156fe6d1dc6d-kube-api-access-x2m85\") pod \"cd70aa09-68dd-4d64-bd6f-156fe6d1dc6d\" (UID: \"cd70aa09-68dd-4d64-bd6f-156fe6d1dc6d\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.753972 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"marketplace-trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b6cd30de-2eeb-49a2-ab40-9167f4560ff5-marketplace-trusted-ca\") pod \"b6cd30de-2eeb-49a2-ab40-9167f4560ff5\" (UID: \"b6cd30de-2eeb-49a2-ab40-9167f4560ff5\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.753999 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"v4-0-config-system-session\" (UniqueName: \"kubernetes.io/secret/49c341d1-5089-4bc2-86a0-a5e165cfcc6b-v4-0-config-system-session\") pod \"49c341d1-5089-4bc2-86a0-a5e165cfcc6b\" (UID: \"49c341d1-5089-4bc2-86a0-a5e165cfcc6b\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.754025 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8cea82b4-6893-4ddc-af9f-1bb5ae425c5b-serving-cert\") pod \"8cea82b4-6893-4ddc-af9f-1bb5ae425c5b\" (UID: \"8cea82b4-6893-4ddc-af9f-1bb5ae425c5b\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.754046 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/43509403-f426-496e-be36-56cef71462f5-console-oauth-config\") pod \"43509403-f426-496e-be36-56cef71462f5\" (UID: \"43509403-f426-496e-be36-56cef71462f5\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.754072 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"v4-0-config-system-serving-cert\" (UniqueName: \"kubernetes.io/secret/49c341d1-5089-4bc2-86a0-a5e165cfcc6b-v4-0-config-system-serving-cert\") pod \"49c341d1-5089-4bc2-86a0-a5e165cfcc6b\" (UID: \"49c341d1-5089-4bc2-86a0-a5e165cfcc6b\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.754098 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8f668bae-612b-4b75-9490-919e737c6a3b-ca-trust-extracted\") pod \"8f668bae-612b-4b75-9490-919e737c6a3b\" (UID: \"8f668bae-612b-4b75-9490-919e737c6a3b\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.754129 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"v4-0-config-user-template-error\" (UniqueName: \"kubernetes.io/secret/49c341d1-5089-4bc2-86a0-a5e165cfcc6b-v4-0-config-user-template-error\") pod \"49c341d1-5089-4bc2-86a0-a5e165cfcc6b\" (UID: \"49c341d1-5089-4bc2-86a0-a5e165cfcc6b\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.754159 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"image-import-ca\" (UniqueName: \"kubernetes.io/configmap/1bf7eb37-55a3-4c65-b768-a94c82151e69-image-import-ca\") pod \"1bf7eb37-55a3-4c65-b768-a94c82151e69\" (UID: \"1bf7eb37-55a3-4c65-b768-a94c82151e69\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.754187 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/210d8245-ebfc-4e3b-ac4a-e21ce76f9a7c-serving-cert\") pod \"210d8245-ebfc-4e3b-ac4a-e21ce76f9a7c\" (UID: \"210d8245-ebfc-4e3b-ac4a-e21ce76f9a7c\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.754255 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/bc5039c0-ea34-426b-a2b7-fbbc87b49a6d-serving-cert\") pod \"bc5039c0-ea34-426b-a2b7-fbbc87b49a6d\" (UID: \"bc5039c0-ea34-426b-a2b7-fbbc87b49a6d\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.754289 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"mcc-auth-proxy-config\" (UniqueName: \"kubernetes.io/configmap/0b574797-001e-440a-8f4e-c0be86edad0f-mcc-auth-proxy-config\") pod \"0b574797-001e-440a-8f4e-c0be86edad0f\" (UID: \"0b574797-001e-440a-8f4e-c0be86edad0f\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.754316 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fqsjt\" (UniqueName: \"kubernetes.io/projected/efdd0498-1daa-4136-9a4a-3b948c2293fc-kube-api-access-fqsjt\") pod \"efdd0498-1daa-4136-9a4a-3b948c2293fc\" (UID: \"efdd0498-1daa-4136-9a4a-3b948c2293fc\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.754343 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"available-featuregates\" (UniqueName: \"kubernetes.io/empty-dir/bc5039c0-ea34-426b-a2b7-fbbc87b49a6d-available-featuregates\") pod \"bc5039c0-ea34-426b-a2b7-fbbc87b49a6d\" (UID: \"bc5039c0-ea34-426b-a2b7-fbbc87b49a6d\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.754371 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wxkg8\" (UniqueName: \"kubernetes.io/projected/3cb93b32-e0ae-4377-b9c8-fdb9842c6d59-kube-api-access-wxkg8\") pod \"3cb93b32-e0ae-4377-b9c8-fdb9842c6d59\" (UID: \"3cb93b32-e0ae-4377-b9c8-fdb9842c6d59\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.754393 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xcgwh\" (UniqueName: \"kubernetes.io/projected/fda69060-fa79-4696-b1a6-7980f124bf7c-kube-api-access-xcgwh\") pod \"fda69060-fa79-4696-b1a6-7980f124bf7c\" (UID: \"fda69060-fa79-4696-b1a6-7980f124bf7c\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.754420 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7583ce53-e0fe-4a16-9e4d-50516596a136-serving-cert\") pod \"7583ce53-e0fe-4a16-9e4d-50516596a136\" (UID: \"7583ce53-e0fe-4a16-9e4d-50516596a136\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.754446 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/0b574797-001e-440a-8f4e-c0be86edad0f-proxy-tls\") pod \"0b574797-001e-440a-8f4e-c0be86edad0f\" (UID: \"0b574797-001e-440a-8f4e-c0be86edad0f\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.754469 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access\" (UniqueName: \"kubernetes.io/projected/0b78653f-4ff9-4508-8672-245ed9b561e3-kube-api-access\") pod \"0b78653f-4ff9-4508-8672-245ed9b561e3\" (UID: \"0b78653f-4ff9-4508-8672-245ed9b561e3\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.754494 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2d4wz\" (UniqueName: \"kubernetes.io/projected/5441d097-087c-4d9a-baa8-b210afa90fc9-kube-api-access-2d4wz\") pod \"5441d097-087c-4d9a-baa8-b210afa90fc9\" (UID: \"5441d097-087c-4d9a-baa8-b210afa90fc9\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.754540 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/87cf06ed-a83f-41a7-828d-70653580a8cb-metrics-tls\") pod \"87cf06ed-a83f-41a7-828d-70653580a8cb\" (UID: \"87cf06ed-a83f-41a7-828d-70653580a8cb\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.754566 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/09ae3b1a-e8e7-4524-b54b-61eab6f9239a-serving-cert\") pod \"09ae3b1a-e8e7-4524-b54b-61eab6f9239a\" (UID: \"09ae3b1a-e8e7-4524-b54b-61eab6f9239a\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.754593 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/c03ee662-fb2f-4fc4-a2c1-af487c19d254-default-certificate\") pod \"c03ee662-fb2f-4fc4-a2c1-af487c19d254\" (UID: \"c03ee662-fb2f-4fc4-a2c1-af487c19d254\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.754615 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4bb40260-dbaa-4fb0-84df-5e680505d512-cni-binary-copy\") pod \"4bb40260-dbaa-4fb0-84df-5e680505d512\" (UID: \"4bb40260-dbaa-4fb0-84df-5e680505d512\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.754637 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6509e943-70c6-444c-bc41-48a544e36fbd-serving-cert\") pod \"6509e943-70c6-444c-bc41-48a544e36fbd\" (UID: \"6509e943-70c6-444c-bc41-48a544e36fbd\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.754659 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a31745f5-9847-4afe-82a5-3161cc66ca93-bound-sa-token\") pod \"a31745f5-9847-4afe-82a5-3161cc66ca93\" (UID: \"a31745f5-9847-4afe-82a5-3161cc66ca93\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.754680 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lz9wn\" (UniqueName: \"kubernetes.io/projected/a31745f5-9847-4afe-82a5-3161cc66ca93-kube-api-access-lz9wn\") pod \"a31745f5-9847-4afe-82a5-3161cc66ca93\" (UID: \"a31745f5-9847-4afe-82a5-3161cc66ca93\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.754700 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"v4-0-config-system-service-ca\" (UniqueName: \"kubernetes.io/configmap/49c341d1-5089-4bc2-86a0-a5e165cfcc6b-v4-0-config-system-service-ca\") pod \"49c341d1-5089-4bc2-86a0-a5e165cfcc6b\" (UID: \"49c341d1-5089-4bc2-86a0-a5e165cfcc6b\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.754720 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8f668bae-612b-4b75-9490-919e737c6a3b-bound-sa-token\") pod \"8f668bae-612b-4b75-9490-919e737c6a3b\" (UID: \"8f668bae-612b-4b75-9490-919e737c6a3b\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.754741 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"v4-0-config-system-router-certs\" (UniqueName: \"kubernetes.io/secret/49c341d1-5089-4bc2-86a0-a5e165cfcc6b-v4-0-config-system-router-certs\") pod \"49c341d1-5089-4bc2-86a0-a5e165cfcc6b\" (UID: \"49c341d1-5089-4bc2-86a0-a5e165cfcc6b\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.754767 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pcxfs\" (UniqueName: \"kubernetes.io/projected/9d4552c7-cd75-42dd-8880-30dd377c49a4-kube-api-access-pcxfs\") pod \"9d4552c7-cd75-42dd-8880-30dd377c49a4\" (UID: \"9d4552c7-cd75-42dd-8880-30dd377c49a4\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.754870 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\") pod \"8f668bae-612b-4b75-9490-919e737c6a3b\" (UID: \"8f668bae-612b-4b75-9490-919e737c6a3b\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.754900 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/bf126b07-da06-4140-9a57-dfd54fc6b486-trusted-ca\") pod \"bf126b07-da06-4140-9a57-dfd54fc6b486\" (UID: \"bf126b07-da06-4140-9a57-dfd54fc6b486\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.754918 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"image-registry-operator-tls\" (UniqueName: \"kubernetes.io/secret/bf126b07-da06-4140-9a57-dfd54fc6b486-image-registry-operator-tls\") pod \"bf126b07-da06-4140-9a57-dfd54fc6b486\" (UID: \"bf126b07-da06-4140-9a57-dfd54fc6b486\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.754946 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/22c825df-677d-4ca6-82db-3454ed06e783-config\") pod \"22c825df-677d-4ca6-82db-3454ed06e783\" (UID: \"22c825df-677d-4ca6-82db-3454ed06e783\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.754967 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0b78653f-4ff9-4508-8672-245ed9b561e3-service-ca\") pod \"0b78653f-4ff9-4508-8672-245ed9b561e3\" (UID: \"0b78653f-4ff9-4508-8672-245ed9b561e3\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.754989 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/6ea678ab-3438-413e-bfe3-290ae7725660-ovnkube-config\") pod \"6ea678ab-3438-413e-bfe3-290ae7725660\" (UID: \"6ea678ab-3438-413e-bfe3-290ae7725660\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.755013 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6ccd8\" (UniqueName: \"kubernetes.io/projected/308be0ea-9f5f-4b29-aeb1-5abd31a0b17b-kube-api-access-6ccd8\") pod \"308be0ea-9f5f-4b29-aeb1-5abd31a0b17b\" (UID: \"308be0ea-9f5f-4b29-aeb1-5abd31a0b17b\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.755039 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c03ee662-fb2f-4fc4-a2c1-af487c19d254-metrics-certs\") pod \"c03ee662-fb2f-4fc4-a2c1-af487c19d254\" (UID: \"c03ee662-fb2f-4fc4-a2c1-af487c19d254\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.755060 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/25e176fe-21b4-4974-b1ed-c8b94f112a7f-signing-cabundle\") pod \"25e176fe-21b4-4974-b1ed-c8b94f112a7f\" (UID: \"25e176fe-21b4-4974-b1ed-c8b94f112a7f\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.755083 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kfwg7\" (UniqueName: \"kubernetes.io/projected/8f668bae-612b-4b75-9490-919e737c6a3b-kube-api-access-kfwg7\") pod \"8f668bae-612b-4b75-9490-919e737c6a3b\" (UID: \"8f668bae-612b-4b75-9490-919e737c6a3b\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.755107 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"audit\" (UniqueName: \"kubernetes.io/configmap/1bf7eb37-55a3-4c65-b768-a94c82151e69-audit\") pod \"1bf7eb37-55a3-4c65-b768-a94c82151e69\" (UID: \"1bf7eb37-55a3-4c65-b768-a94c82151e69\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.755735 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"etcd-service-ca\" (UniqueName: \"kubernetes.io/configmap/09efc573-dbb6-4249-bd59-9b87aba8dd28-etcd-service-ca\") pod \"09efc573-dbb6-4249-bd59-9b87aba8dd28\" (UID: \"09efc573-dbb6-4249-bd59-9b87aba8dd28\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.755771 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/925f1c65-6136-48ba-85aa-3a3b50560753-env-overrides\") pod \"925f1c65-6136-48ba-85aa-3a3b50560753\" (UID: \"925f1c65-6136-48ba-85aa-3a3b50560753\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.755798 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"mcd-auth-proxy-config\" (UniqueName: \"kubernetes.io/configmap/fda69060-fa79-4696-b1a6-7980f124bf7c-mcd-auth-proxy-config\") pod \"fda69060-fa79-4696-b1a6-7980f124bf7c\" (UID: \"fda69060-fa79-4696-b1a6-7980f124bf7c\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.755821 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/57a731c4-ef35-47a8-b875-bfb08a7f8011-utilities\") pod \"57a731c4-ef35-47a8-b875-bfb08a7f8011\" (UID: \"57a731c4-ef35-47a8-b875-bfb08a7f8011\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.755846 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"etcd-serving-ca\" (UniqueName: \"kubernetes.io/configmap/09ae3b1a-e8e7-4524-b54b-61eab6f9239a-etcd-serving-ca\") pod \"09ae3b1a-e8e7-4524-b54b-61eab6f9239a\" (UID: \"09ae3b1a-e8e7-4524-b54b-61eab6f9239a\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.755872 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qg5z5\" (UniqueName: \"kubernetes.io/projected/43509403-f426-496e-be36-56cef71462f5-kube-api-access-qg5z5\") pod \"43509403-f426-496e-be36-56cef71462f5\" (UID: \"43509403-f426-496e-be36-56cef71462f5\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.755898 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c03ee662-fb2f-4fc4-a2c1-af487c19d254-service-ca-bundle\") pod \"c03ee662-fb2f-4fc4-a2c1-af487c19d254\" (UID: \"c03ee662-fb2f-4fc4-a2c1-af487c19d254\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.755926 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nzwt7\" (UniqueName: \"kubernetes.io/projected/96b93a3a-6083-4aea-8eab-fe1aa8245ad9-kube-api-access-nzwt7\") pod \"96b93a3a-6083-4aea-8eab-fe1aa8245ad9\" (UID: \"96b93a3a-6083-4aea-8eab-fe1aa8245ad9\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.755954 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v47cf\" (UniqueName: \"kubernetes.io/projected/c03ee662-fb2f-4fc4-a2c1-af487c19d254-kube-api-access-v47cf\") pod \"c03ee662-fb2f-4fc4-a2c1-af487c19d254\" (UID: \"c03ee662-fb2f-4fc4-a2c1-af487c19d254\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.755980 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/7539238d-5fe0-46ed-884e-1c3b566537ec-config\") pod \"7539238d-5fe0-46ed-884e-1c3b566537ec\" (UID: \"7539238d-5fe0-46ed-884e-1c3b566537ec\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.756006 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access\" (UniqueName: \"kubernetes.io/projected/1386a44e-36a2-460c-96d0-0359d2b6f0f5-kube-api-access\") pod \"1386a44e-36a2-460c-96d0-0359d2b6f0f5\" (UID: \"1386a44e-36a2-460c-96d0-0359d2b6f0f5\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.756036 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/57a731c4-ef35-47a8-b875-bfb08a7f8011-catalog-content\") pod \"57a731c4-ef35-47a8-b875-bfb08a7f8011\" (UID: \"57a731c4-ef35-47a8-b875-bfb08a7f8011\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.756063 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-htfz6\" (UniqueName: \"kubernetes.io/projected/6ea678ab-3438-413e-bfe3-290ae7725660-kube-api-access-htfz6\") pod \"6ea678ab-3438-413e-bfe3-290ae7725660\" (UID: \"6ea678ab-3438-413e-bfe3-290ae7725660\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.756089 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"v4-0-config-user-template-login\" (UniqueName: \"kubernetes.io/secret/49c341d1-5089-4bc2-86a0-a5e165cfcc6b-v4-0-config-user-template-login\") pod \"49c341d1-5089-4bc2-86a0-a5e165cfcc6b\" (UID: \"49c341d1-5089-4bc2-86a0-a5e165cfcc6b\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.756115 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7c4vf\" (UniqueName: \"kubernetes.io/projected/22c825df-677d-4ca6-82db-3454ed06e783-kube-api-access-7c4vf\") pod \"22c825df-677d-4ca6-82db-3454ed06e783\" (UID: \"22c825df-677d-4ca6-82db-3454ed06e783\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.756144 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zkvpv\" (UniqueName: \"kubernetes.io/projected/09ae3b1a-e8e7-4524-b54b-61eab6f9239a-kube-api-access-zkvpv\") pod \"09ae3b1a-e8e7-4524-b54b-61eab6f9239a\" (UID: \"09ae3b1a-e8e7-4524-b54b-61eab6f9239a\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.756168 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"etcd-client\" (UniqueName: \"kubernetes.io/secret/09efc573-dbb6-4249-bd59-9b87aba8dd28-etcd-client\") pod \"09efc573-dbb6-4249-bd59-9b87aba8dd28\" (UID: \"09efc573-dbb6-4249-bd59-9b87aba8dd28\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.756191 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8f668bae-612b-4b75-9490-919e737c6a3b-trusted-ca\") pod \"8f668bae-612b-4b75-9490-919e737c6a3b\" (UID: \"8f668bae-612b-4b75-9490-919e737c6a3b\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.756214 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a31745f5-9847-4afe-82a5-3161cc66ca93-metrics-tls\") pod \"a31745f5-9847-4afe-82a5-3161cc66ca93\" (UID: \"a31745f5-9847-4afe-82a5-3161cc66ca93\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.756242 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1386a44e-36a2-460c-96d0-0359d2b6f0f5-config\") pod \"1386a44e-36a2-460c-96d0-0359d2b6f0f5\" (UID: \"1386a44e-36a2-460c-96d0-0359d2b6f0f5\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.756267 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6g6sz\" (UniqueName: \"kubernetes.io/projected/6509e943-70c6-444c-bc41-48a544e36fbd-kube-api-access-6g6sz\") pod \"6509e943-70c6-444c-bc41-48a544e36fbd\" (UID: \"6509e943-70c6-444c-bc41-48a544e36fbd\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.756290 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/25e176fe-21b4-4974-b1ed-c8b94f112a7f-signing-key\") pod \"25e176fe-21b4-4974-b1ed-c8b94f112a7f\" (UID: \"25e176fe-21b4-4974-b1ed-c8b94f112a7f\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.756310 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5441d097-087c-4d9a-baa8-b210afa90fc9-config\") pod \"5441d097-087c-4d9a-baa8-b210afa90fc9\" (UID: \"5441d097-087c-4d9a-baa8-b210afa90fc9\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.756331 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s4n52\" (UniqueName: \"kubernetes.io/projected/925f1c65-6136-48ba-85aa-3a3b50560753-kube-api-access-s4n52\") pod \"925f1c65-6136-48ba-85aa-3a3b50560753\" (UID: \"925f1c65-6136-48ba-85aa-3a3b50560753\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.756348 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cfbct\" (UniqueName: \"kubernetes.io/projected/57a731c4-ef35-47a8-b875-bfb08a7f8011-kube-api-access-cfbct\") pod \"57a731c4-ef35-47a8-b875-bfb08a7f8011\" (UID: \"57a731c4-ef35-47a8-b875-bfb08a7f8011\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.756366 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"control-plane-machine-set-operator-tls\" (UniqueName: \"kubernetes.io/secret/6731426b-95fe-49ff-bb5f-40441049fde2-control-plane-machine-set-operator-tls\") pod \"6731426b-95fe-49ff-bb5f-40441049fde2\" (UID: \"6731426b-95fe-49ff-bb5f-40441049fde2\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.756383 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"certs\" (UniqueName: \"kubernetes.io/secret/5fe579f8-e8a6-4643-bce5-a661393c4dde-certs\") pod \"5fe579f8-e8a6-4643-bce5-a661393c4dde\" (UID: \"5fe579f8-e8a6-4643-bce5-a661393c4dde\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.756400 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"profile-collector-cert\" (UniqueName: \"kubernetes.io/secret/b6312bbd-5731-4ea0-a20f-81d5a57df44a-profile-collector-cert\") pod \"b6312bbd-5731-4ea0-a20f-81d5a57df44a\" (UID: \"b6312bbd-5731-4ea0-a20f-81d5a57df44a\") " Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.756471 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/c30c02db-f4e7-4432-941c-5448fd71f3d9-hostroot\") pod \"multus-hx657\" (UID: \"c30c02db-f4e7-4432-941c-5448fd71f3d9\") " pod="openshift-multus/multus-hx657" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.756629 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bc569bba-c020-40d6-af68-e617625fe0e4-run-openvswitch\") pod \"ovnkube-node-kg55h\" (UID: \"bc569bba-c020-40d6-af68-e617625fe0e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-kg55h" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.756662 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-rdwmf\" (UniqueName: \"kubernetes.io/projected/37a5e44f-9a88-4405-be8a-b645485e7312-kube-api-access-rdwmf\") pod \"network-operator-58b4c7f79c-55gtf\" (UID: \"37a5e44f-9a88-4405-be8a-b645485e7312\") " pod="openshift-network-operator/network-operator-58b4c7f79c-55gtf" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.756683 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bc569bba-c020-40d6-af68-e617625fe0e4-host-run-ovn-kubernetes\") pod \"ovnkube-node-kg55h\" (UID: \"bc569bba-c020-40d6-af68-e617625fe0e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-kg55h" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.756699 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/bc569bba-c020-40d6-af68-e617625fe0e4-host-cni-netd\") pod \"ovnkube-node-kg55h\" (UID: \"bc569bba-c020-40d6-af68-e617625fe0e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-kg55h" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.756721 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bc569bba-c020-40d6-af68-e617625fe0e4-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-kg55h\" (UID: \"bc569bba-c020-40d6-af68-e617625fe0e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-kg55h" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.757572 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c30c02db-f4e7-4432-941c-5448fd71f3d9-cni-binary-copy\") pod \"multus-hx657\" (UID: \"c30c02db-f4e7-4432-941c-5448fd71f3d9\") " pod="openshift-multus/multus-hx657" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.757676 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c30c02db-f4e7-4432-941c-5448fd71f3d9-host-var-lib-kubelet\") pod \"multus-hx657\" (UID: \"c30c02db-f4e7-4432-941c-5448fd71f3d9\") " pod="openshift-multus/multus-hx657" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.757709 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/bc569bba-c020-40d6-af68-e617625fe0e4-run-systemd\") pod \"ovnkube-node-kg55h\" (UID: \"bc569bba-c020-40d6-af68-e617625fe0e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-kg55h" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.757742 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/bc569bba-c020-40d6-af68-e617625fe0e4-log-socket\") pod \"ovnkube-node-kg55h\" (UID: \"bc569bba-c020-40d6-af68-e617625fe0e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-kg55h" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.757769 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e5c587e6-866a-4427-8a2c-3469cf0fc495-hosts-file\") pod \"node-resolver-xph69\" (UID: \"e5c587e6-866a-4427-8a2c-3469cf0fc495\") " pod="openshift-dns/node-resolver-xph69" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.757800 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c30c02db-f4e7-4432-941c-5448fd71f3d9-cnibin\") pod \"multus-hx657\" (UID: \"c30c02db-f4e7-4432-941c-5448fd71f3d9\") " pod="openshift-multus/multus-hx657" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.757835 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tk7n6\" (UniqueName: \"kubernetes.io/projected/e5c587e6-866a-4427-8a2c-3469cf0fc495-kube-api-access-tk7n6\") pod \"node-resolver-xph69\" (UID: \"e5c587e6-866a-4427-8a2c-3469cf0fc495\") " pod="openshift-dns/node-resolver-xph69" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.757865 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c30c02db-f4e7-4432-941c-5448fd71f3d9-host-run-netns\") pod \"multus-hx657\" (UID: \"c30c02db-f4e7-4432-941c-5448fd71f3d9\") " pod="openshift-multus/multus-hx657" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.757917 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-cqllr\" (UniqueName: \"kubernetes.io/projected/3b6479f0-333b-4a96-9adf-2099afdc2447-kube-api-access-cqllr\") pod \"network-check-target-xd92c\" (UID: \"3b6479f0-333b-4a96-9adf-2099afdc2447\") " pod="openshift-network-diagnostics/network-check-target-xd92c" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.757955 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/17265a68-bcb5-455c-b7df-c148a6517db9-cni-binary-copy\") pod \"multus-additional-cni-plugins-2lrql\" (UID: \"17265a68-bcb5-455c-b7df-c148a6517db9\") " pod="openshift-multus/multus-additional-cni-plugins-2lrql" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.758622 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d75a4c96-2883-4a0b-bab2-0fab2b6c0b49-iptables-alerter-script\") pod \"iptables-alerter-4ln5h\" (UID: \"d75a4c96-2883-4a0b-bab2-0fab2b6c0b49\") " pod="openshift-network-operator/iptables-alerter-4ln5h" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.758661 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/5fe485a1-e14f-4c09-b5b9-f252bc42b7e8-nginx-conf\") pod \"networking-console-plugin-85b44fc459-gdk6g\" (UID: \"5fe485a1-e14f-4c09-b5b9-f252bc42b7e8\") " pod="openshift-network-console/networking-console-plugin-85b44fc459-gdk6g" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.758694 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c30c02db-f4e7-4432-941c-5448fd71f3d9-host-var-lib-cni-bin\") pod \"multus-hx657\" (UID: \"c30c02db-f4e7-4432-941c-5448fd71f3d9\") " pod="openshift-multus/multus-hx657" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.758728 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bc569bba-c020-40d6-af68-e617625fe0e4-etc-openvswitch\") pod \"ovnkube-node-kg55h\" (UID: \"bc569bba-c020-40d6-af68-e617625fe0e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-kg55h" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.758769 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/bc569bba-c020-40d6-af68-e617625fe0e4-run-ovn\") pod \"ovnkube-node-kg55h\" (UID: \"bc569bba-c020-40d6-af68-e617625fe0e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-kg55h" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.758800 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/17265a68-bcb5-455c-b7df-c148a6517db9-system-cni-dir\") pod \"multus-additional-cni-plugins-2lrql\" (UID: \"17265a68-bcb5-455c-b7df-c148a6517db9\") " pod="openshift-multus/multus-additional-cni-plugins-2lrql" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.758823 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/17265a68-bcb5-455c-b7df-c148a6517db9-tuning-conf-dir\") pod \"multus-additional-cni-plugins-2lrql\" (UID: \"17265a68-bcb5-455c-b7df-c148a6517db9\") " pod="openshift-multus/multus-additional-cni-plugins-2lrql" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.758860 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-rczfb\" (UniqueName: \"kubernetes.io/projected/d75a4c96-2883-4a0b-bab2-0fab2b6c0b49-kube-api-access-rczfb\") pod \"iptables-alerter-4ln5h\" (UID: \"d75a4c96-2883-4a0b-bab2-0fab2b6c0b49\") " pod="openshift-network-operator/iptables-alerter-4ln5h" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.758900 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/bc569bba-c020-40d6-af68-e617625fe0e4-host-kubelet\") pod \"ovnkube-node-kg55h\" (UID: \"bc569bba-c020-40d6-af68-e617625fe0e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-kg55h" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.758925 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bc569bba-c020-40d6-af68-e617625fe0e4-host-cni-bin\") pod \"ovnkube-node-kg55h\" (UID: \"bc569bba-c020-40d6-af68-e617625fe0e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-kg55h" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.758956 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/bc569bba-c020-40d6-af68-e617625fe0e4-ovnkube-script-lib\") pod \"ovnkube-node-kg55h\" (UID: \"bc569bba-c020-40d6-af68-e617625fe0e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-kg55h" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.758996 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-s2kz5\" (UniqueName: \"kubernetes.io/projected/ef543e1b-8068-4ea3-b32a-61027b32e95d-kube-api-access-s2kz5\") pod \"network-node-identity-vrzqb\" (UID: \"ef543e1b-8068-4ea3-b32a-61027b32e95d\") " pod="openshift-network-node-identity/network-node-identity-vrzqb" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.759022 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bc569bba-c020-40d6-af68-e617625fe0e4-host-run-netns\") pod \"ovnkube-node-kg55h\" (UID: \"bc569bba-c020-40d6-af68-e617625fe0e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-kg55h" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.759047 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/bc569bba-c020-40d6-af68-e617625fe0e4-node-log\") pod \"ovnkube-node-kg55h\" (UID: \"bc569bba-c020-40d6-af68-e617625fe0e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-kg55h" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.759067 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"rootfs\" (UniqueName: \"kubernetes.io/host-path/10c237bc-72e7-4da3-9f2c-b1688395bf6e-rootfs\") pod \"machine-config-daemon-74bj6\" (UID: \"10c237bc-72e7-4da3-9f2c-b1688395bf6e\") " pod="openshift-machine-config-operator/machine-config-daemon-74bj6" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.759096 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ef543e1b-8068-4ea3-b32a-61027b32e95d-webhook-cert\") pod \"network-node-identity-vrzqb\" (UID: \"ef543e1b-8068-4ea3-b32a-61027b32e95d\") " pod="openshift-network-node-identity/network-node-identity-vrzqb" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.759122 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ef543e1b-8068-4ea3-b32a-61027b32e95d-env-overrides\") pod \"network-node-identity-vrzqb\" (UID: \"ef543e1b-8068-4ea3-b32a-61027b32e95d\") " pod="openshift-network-node-identity/network-node-identity-vrzqb" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.759146 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovnkube-identity-cm\" (UniqueName: \"kubernetes.io/configmap/ef543e1b-8068-4ea3-b32a-61027b32e95d-ovnkube-identity-cm\") pod \"network-node-identity-vrzqb\" (UID: \"ef543e1b-8068-4ea3-b32a-61027b32e95d\") " pod="openshift-network-node-identity/network-node-identity-vrzqb" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.759178 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/bc569bba-c020-40d6-af68-e617625fe0e4-systemd-units\") pod \"ovnkube-node-kg55h\" (UID: \"bc569bba-c020-40d6-af68-e617625fe0e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-kg55h" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.759209 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/5fe485a1-e14f-4c09-b5b9-f252bc42b7e8-networking-console-plugin-cert\") pod \"networking-console-plugin-85b44fc459-gdk6g\" (UID: \"5fe485a1-e14f-4c09-b5b9-f252bc42b7e8\") " pod="openshift-network-console/networking-console-plugin-85b44fc459-gdk6g" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.759249 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/c30c02db-f4e7-4432-941c-5448fd71f3d9-host-run-k8s-cni-cncf-io\") pod \"multus-hx657\" (UID: \"c30c02db-f4e7-4432-941c-5448fd71f3d9\") " pod="openshift-multus/multus-hx657" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.759288 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/c30c02db-f4e7-4432-941c-5448fd71f3d9-multus-daemon-config\") pod \"multus-hx657\" (UID: \"c30c02db-f4e7-4432-941c-5448fd71f3d9\") " pod="openshift-multus/multus-hx657" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.759326 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/10c237bc-72e7-4da3-9f2c-b1688395bf6e-proxy-tls\") pod \"machine-config-daemon-74bj6\" (UID: \"10c237bc-72e7-4da3-9f2c-b1688395bf6e\") " pod="openshift-machine-config-operator/machine-config-daemon-74bj6" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.759362 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w4zf5\" (UniqueName: \"kubernetes.io/projected/17265a68-bcb5-455c-b7df-c148a6517db9-kube-api-access-w4zf5\") pod \"multus-additional-cni-plugins-2lrql\" (UID: \"17265a68-bcb5-455c-b7df-c148a6517db9\") " pod="openshift-multus/multus-additional-cni-plugins-2lrql" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.759401 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c30c02db-f4e7-4432-941c-5448fd71f3d9-os-release\") pod \"multus-hx657\" (UID: \"c30c02db-f4e7-4432-941c-5448fd71f3d9\") " pod="openshift-multus/multus-hx657" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.759433 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/c30c02db-f4e7-4432-941c-5448fd71f3d9-multus-socket-dir-parent\") pod \"multus-hx657\" (UID: \"c30c02db-f4e7-4432-941c-5448fd71f3d9\") " pod="openshift-multus/multus-hx657" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.759909 5052 status_manager.go:875] "Failed to update status for pod" pod="openshift-network-diagnostics/network-check-source-55646444c4-trplf" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"9d751cbb-f2e2-430d-9754-c882a5e924a5\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"message\\\":\\\"containers with unready status: [check-endpoints]\\\",\\\"reason\\\":\\\"ContainersNotReady\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"message\\\":\\\"containers with unready status: [check-endpoints]\\\",\\\"reason\\\":\\\"ContainersNotReady\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"ContainersReady\\\"}],\\\"containerStatuses\\\":[{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e1baa38811c04bd8909e01a1f3be7421a1cb99d608d3dc4cf86d95b17de2ab8b\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{\\\"terminated\\\":{\\\"exitCode\\\":137,\\\"finishedAt\\\":null,\\\"message\\\":\\\"The container could not be located when the pod was deleted. The container used to be Running\\\",\\\"reason\\\":\\\"ContainerStatusUnknown\\\",\\\"startedAt\\\":null}},\\\"name\\\":\\\"check-endpoints\\\",\\\"ready\\\":false,\\\"restartCount\\\":3,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"ContainerCreating\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-s2dwl\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}],\\\"podIP\\\":null,\\\"podIPs\\\":null}}\" for pod \"openshift-network-diagnostics\"/\"network-check-source-55646444c4-trplf\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://127.0.0.1:9743/pod?timeout=10s\": dial tcp 127.0.0.1:9743: connect: connection refused" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.760626 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d75a4c96-2883-4a0b-bab2-0fab2b6c0b49-iptables-alerter-script\") pod \"iptables-alerter-4ln5h\" (UID: \"d75a4c96-2883-4a0b-bab2-0fab2b6c0b49\") " pod="openshift-network-operator/iptables-alerter-4ln5h" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.762675 5052 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.767435 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ef543e1b-8068-4ea3-b32a-61027b32e95d-env-overrides\") pod \"network-node-identity-vrzqb\" (UID: \"ef543e1b-8068-4ea3-b32a-61027b32e95d\") " pod="openshift-network-node-identity/network-node-identity-vrzqb" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.751598 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6ea678ab-3438-413e-bfe3-290ae7725660-ovn-node-metrics-cert" (OuterVolumeSpecName: "ovn-node-metrics-cert") pod "6ea678ab-3438-413e-bfe3-290ae7725660" (UID: "6ea678ab-3438-413e-bfe3-290ae7725660"). InnerVolumeSpecName "ovn-node-metrics-cert". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.751924 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d-utilities" (OuterVolumeSpecName: "utilities") pod "b11524ee-3fca-4b1b-9cdf-6da289fdbc7d" (UID: "b11524ee-3fca-4b1b-9cdf-6da289fdbc7d"). InnerVolumeSpecName "utilities". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.752016 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1bf7eb37-55a3-4c65-b768-a94c82151e69-etcd-serving-ca" (OuterVolumeSpecName: "etcd-serving-ca") pod "1bf7eb37-55a3-4c65-b768-a94c82151e69" (UID: "1bf7eb37-55a3-4c65-b768-a94c82151e69"). InnerVolumeSpecName "etcd-serving-ca". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.752148 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6402fda4-df10-493c-b4e5-d0569419652d-machine-api-operator-tls" (OuterVolumeSpecName: "machine-api-operator-tls") pod "6402fda4-df10-493c-b4e5-d0569419652d" (UID: "6402fda4-df10-493c-b4e5-d0569419652d"). InnerVolumeSpecName "machine-api-operator-tls". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.752212 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0b574797-001e-440a-8f4e-c0be86edad0f-kube-api-access-lzf88" (OuterVolumeSpecName: "kube-api-access-lzf88") pod "0b574797-001e-440a-8f4e-c0be86edad0f" (UID: "0b574797-001e-440a-8f4e-c0be86edad0f"). InnerVolumeSpecName "kube-api-access-lzf88". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.769808 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovnkube-identity-cm\" (UniqueName: \"kubernetes.io/configmap/ef543e1b-8068-4ea3-b32a-61027b32e95d-ovnkube-identity-cm\") pod \"network-node-identity-vrzqb\" (UID: \"ef543e1b-8068-4ea3-b32a-61027b32e95d\") " pod="openshift-network-node-identity/network-node-identity-vrzqb" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.752867 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b6312bbd-5731-4ea0-a20f-81d5a57df44a-kube-api-access-249nr" (OuterVolumeSpecName: "kube-api-access-249nr") pod "b6312bbd-5731-4ea0-a20f-81d5a57df44a" (UID: "b6312bbd-5731-4ea0-a20f-81d5a57df44a"). InnerVolumeSpecName "kube-api-access-249nr". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.753335 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/20b0d48f-5fd6-431c-a545-e3c800c7b866-cert" (OuterVolumeSpecName: "cert") pod "20b0d48f-5fd6-431c-a545-e3c800c7b866" (UID: "20b0d48f-5fd6-431c-a545-e3c800c7b866"). InnerVolumeSpecName "cert". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.753548 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/87cf06ed-a83f-41a7-828d-70653580a8cb-kube-api-access-d6qdx" (OuterVolumeSpecName: "kube-api-access-d6qdx") pod "87cf06ed-a83f-41a7-828d-70653580a8cb" (UID: "87cf06ed-a83f-41a7-828d-70653580a8cb"). InnerVolumeSpecName "kube-api-access-d6qdx". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.753662 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/22c825df-677d-4ca6-82db-3454ed06e783-machine-approver-tls" (OuterVolumeSpecName: "machine-approver-tls") pod "22c825df-677d-4ca6-82db-3454ed06e783" (UID: "22c825df-677d-4ca6-82db-3454ed06e783"). InnerVolumeSpecName "machine-approver-tls". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.770017 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8f668bae-612b-4b75-9490-919e737c6a3b-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "8f668bae-612b-4b75-9490-919e737c6a3b" (UID: "8f668bae-612b-4b75-9490-919e737c6a3b"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.770090 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/25e176fe-21b4-4974-b1ed-c8b94f112a7f-signing-cabundle" (OuterVolumeSpecName: "signing-cabundle") pod "25e176fe-21b4-4974-b1ed-c8b94f112a7f" (UID: "25e176fe-21b4-4974-b1ed-c8b94f112a7f"). InnerVolumeSpecName "signing-cabundle". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.753225 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6ea678ab-3438-413e-bfe3-290ae7725660-env-overrides" (OuterVolumeSpecName: "env-overrides") pod "6ea678ab-3438-413e-bfe3-290ae7725660" (UID: "6ea678ab-3438-413e-bfe3-290ae7725660"). InnerVolumeSpecName "env-overrides". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.754078 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b6cd30de-2eeb-49a2-ab40-9167f4560ff5-marketplace-operator-metrics" (OuterVolumeSpecName: "marketplace-operator-metrics") pod "b6cd30de-2eeb-49a2-ab40-9167f4560ff5" (UID: "b6cd30de-2eeb-49a2-ab40-9167f4560ff5"). InnerVolumeSpecName "marketplace-operator-metrics". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.754328 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1d611f23-29be-4491-8495-bee1670e935f-utilities" (OuterVolumeSpecName: "utilities") pod "1d611f23-29be-4491-8495-bee1670e935f" (UID: "1d611f23-29be-4491-8495-bee1670e935f"). InnerVolumeSpecName "utilities". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.754523 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/49c341d1-5089-4bc2-86a0-a5e165cfcc6b-v4-0-config-system-ocp-branding-template" (OuterVolumeSpecName: "v4-0-config-system-ocp-branding-template") pod "49c341d1-5089-4bc2-86a0-a5e165cfcc6b" (UID: "49c341d1-5089-4bc2-86a0-a5e165cfcc6b"). InnerVolumeSpecName "v4-0-config-system-ocp-branding-template". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.754724 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/31d8b7a1-420e-4252-a5b7-eebe8a111292-auth-proxy-config" (OuterVolumeSpecName: "auth-proxy-config") pod "31d8b7a1-420e-4252-a5b7-eebe8a111292" (UID: "31d8b7a1-420e-4252-a5b7-eebe8a111292"). InnerVolumeSpecName "auth-proxy-config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.755120 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/09efc573-dbb6-4249-bd59-9b87aba8dd28-etcd-ca" (OuterVolumeSpecName: "etcd-ca") pod "09efc573-dbb6-4249-bd59-9b87aba8dd28" (UID: "09efc573-dbb6-4249-bd59-9b87aba8dd28"). InnerVolumeSpecName "etcd-ca". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.755145 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f88749ec-7931-4ee7-b3fc-1ec5e11f92e9-srv-cert" (OuterVolumeSpecName: "srv-cert") pod "f88749ec-7931-4ee7-b3fc-1ec5e11f92e9" (UID: "f88749ec-7931-4ee7-b3fc-1ec5e11f92e9"). InnerVolumeSpecName "srv-cert". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.755200 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7583ce53-e0fe-4a16-9e4d-50516596a136-client-ca" (OuterVolumeSpecName: "client-ca") pod "7583ce53-e0fe-4a16-9e4d-50516596a136" (UID: "7583ce53-e0fe-4a16-9e4d-50516596a136"). InnerVolumeSpecName "client-ca". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.754052 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/09efc573-dbb6-4249-bd59-9b87aba8dd28-config" (OuterVolumeSpecName: "config") pod "09efc573-dbb6-4249-bd59-9b87aba8dd28" (UID: "09efc573-dbb6-4249-bd59-9b87aba8dd28"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.755308 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5fe579f8-e8a6-4643-bce5-a661393c4dde-kube-api-access-fcqwp" (OuterVolumeSpecName: "kube-api-access-fcqwp") pod "5fe579f8-e8a6-4643-bce5-a661393c4dde" (UID: "5fe579f8-e8a6-4643-bce5-a661393c4dde"). InnerVolumeSpecName "kube-api-access-fcqwp". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.755407 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/210d8245-ebfc-4e3b-ac4a-e21ce76f9a7c-kube-api-access-qs4fp" (OuterVolumeSpecName: "kube-api-access-qs4fp") pod "210d8245-ebfc-4e3b-ac4a-e21ce76f9a7c" (UID: "210d8245-ebfc-4e3b-ac4a-e21ce76f9a7c"). InnerVolumeSpecName "kube-api-access-qs4fp". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.755435 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/43509403-f426-496e-be36-56cef71462f5-service-ca" (OuterVolumeSpecName: "service-ca") pod "43509403-f426-496e-be36-56cef71462f5" (UID: "43509403-f426-496e-be36-56cef71462f5"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.755654 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7bb08738-c794-4ee8-9972-3a62ca171029-kube-api-access-279lb" (OuterVolumeSpecName: "kube-api-access-279lb") pod "7bb08738-c794-4ee8-9972-3a62ca171029" (UID: "7bb08738-c794-4ee8-9972-3a62ca171029"). InnerVolumeSpecName "kube-api-access-279lb". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.755707 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/44663579-783b-4372-86d6-acf235a62d72-kube-api-access-vt5rc" (OuterVolumeSpecName: "kube-api-access-vt5rc") pod "44663579-783b-4372-86d6-acf235a62d72" (UID: "44663579-783b-4372-86d6-acf235a62d72"). InnerVolumeSpecName "kube-api-access-vt5rc". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.755722 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1bf7eb37-55a3-4c65-b768-a94c82151e69-encryption-config" (OuterVolumeSpecName: "encryption-config") pod "1bf7eb37-55a3-4c65-b768-a94c82151e69" (UID: "1bf7eb37-55a3-4c65-b768-a94c82151e69"). InnerVolumeSpecName "encryption-config". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.755731 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7583ce53-e0fe-4a16-9e4d-50516596a136-config" (OuterVolumeSpecName: "config") pod "7583ce53-e0fe-4a16-9e4d-50516596a136" (UID: "7583ce53-e0fe-4a16-9e4d-50516596a136"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.755806 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/43509403-f426-496e-be36-56cef71462f5-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "43509403-f426-496e-be36-56cef71462f5" (UID: "43509403-f426-496e-be36-56cef71462f5"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.755770 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4bb40260-dbaa-4fb0-84df-5e680505d512-multus-daemon-config" (OuterVolumeSpecName: "multus-daemon-config") pod "4bb40260-dbaa-4fb0-84df-5e680505d512" (UID: "4bb40260-dbaa-4fb0-84df-5e680505d512"). InnerVolumeSpecName "multus-daemon-config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.755930 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/308be0ea-9f5f-4b29-aeb1-5abd31a0b17b-apiservice-cert" (OuterVolumeSpecName: "apiservice-cert") pod "308be0ea-9f5f-4b29-aeb1-5abd31a0b17b" (UID: "308be0ea-9f5f-4b29-aeb1-5abd31a0b17b"). InnerVolumeSpecName "apiservice-cert". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.756111 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/96b93a3a-6083-4aea-8eab-fe1aa8245ad9-metrics-tls" (OuterVolumeSpecName: "metrics-tls") pod "96b93a3a-6083-4aea-8eab-fe1aa8245ad9" (UID: "96b93a3a-6083-4aea-8eab-fe1aa8245ad9"). InnerVolumeSpecName "metrics-tls". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.756416 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/01ab3dd5-8196-46d0-ad33-122e2ca51def-kube-api-access-w7l8j" (OuterVolumeSpecName: "kube-api-access-w7l8j") pod "01ab3dd5-8196-46d0-ad33-122e2ca51def" (UID: "01ab3dd5-8196-46d0-ad33-122e2ca51def"). InnerVolumeSpecName "kube-api-access-w7l8j". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.756407 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/09ae3b1a-e8e7-4524-b54b-61eab6f9239a-encryption-config" (OuterVolumeSpecName: "encryption-config") pod "09ae3b1a-e8e7-4524-b54b-61eab6f9239a" (UID: "09ae3b1a-e8e7-4524-b54b-61eab6f9239a"). InnerVolumeSpecName "encryption-config". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.756462 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a31745f5-9847-4afe-82a5-3161cc66ca93-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "a31745f5-9847-4afe-82a5-3161cc66ca93" (UID: "a31745f5-9847-4afe-82a5-3161cc66ca93"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.756709 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5b88f790-22fa-440e-b583-365168c0b23d-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "5b88f790-22fa-440e-b583-365168c0b23d" (UID: "5b88f790-22fa-440e-b583-365168c0b23d"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.757271 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3cb93b32-e0ae-4377-b9c8-fdb9842c6d59-serviceca" (OuterVolumeSpecName: "serviceca") pod "3cb93b32-e0ae-4377-b9c8-fdb9842c6d59" (UID: "3cb93b32-e0ae-4377-b9c8-fdb9842c6d59"). InnerVolumeSpecName "serviceca". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.757297 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3cb93b32-e0ae-4377-b9c8-fdb9842c6d59-kube-api-access-wxkg8" (OuterVolumeSpecName: "kube-api-access-wxkg8") pod "3cb93b32-e0ae-4377-b9c8-fdb9842c6d59" (UID: "3cb93b32-e0ae-4377-b9c8-fdb9842c6d59"). InnerVolumeSpecName "kube-api-access-wxkg8". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.757329 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8cea82b4-6893-4ddc-af9f-1bb5ae425c5b-kube-api-access-w4xd4" (OuterVolumeSpecName: "kube-api-access-w4xd4") pod "8cea82b4-6893-4ddc-af9f-1bb5ae425c5b" (UID: "8cea82b4-6893-4ddc-af9f-1bb5ae425c5b"). InnerVolumeSpecName "kube-api-access-w4xd4". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.757668 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fda69060-fa79-4696-b1a6-7980f124bf7c-kube-api-access-xcgwh" (OuterVolumeSpecName: "kube-api-access-xcgwh") pod "fda69060-fa79-4696-b1a6-7980f124bf7c" (UID: "fda69060-fa79-4696-b1a6-7980f124bf7c"). InnerVolumeSpecName "kube-api-access-xcgwh". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.758030 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6509e943-70c6-444c-bc41-48a544e36fbd-service-ca-bundle" (OuterVolumeSpecName: "service-ca-bundle") pod "6509e943-70c6-444c-bc41-48a544e36fbd" (UID: "6509e943-70c6-444c-bc41-48a544e36fbd"). InnerVolumeSpecName "service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.758084 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e7e6199b-1264-4501-8953-767f51328d08-config" (OuterVolumeSpecName: "config") pod "e7e6199b-1264-4501-8953-767f51328d08" (UID: "e7e6199b-1264-4501-8953-767f51328d08"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.758218 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a0128f3a-b052-44ed-a84e-c4c8aaf17c13-samples-operator-tls" (OuterVolumeSpecName: "samples-operator-tls") pod "a0128f3a-b052-44ed-a84e-c4c8aaf17c13" (UID: "a0128f3a-b052-44ed-a84e-c4c8aaf17c13"). InnerVolumeSpecName "samples-operator-tls". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.758532 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7583ce53-e0fe-4a16-9e4d-50516596a136-serving-cert" (OuterVolumeSpecName: "serving-cert") pod "7583ce53-e0fe-4a16-9e4d-50516596a136" (UID: "7583ce53-e0fe-4a16-9e4d-50516596a136"). InnerVolumeSpecName "serving-cert". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.758640 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9d4552c7-cd75-42dd-8880-30dd377c49a4-config" (OuterVolumeSpecName: "config") pod "9d4552c7-cd75-42dd-8880-30dd377c49a4" (UID: "9d4552c7-cd75-42dd-8880-30dd377c49a4"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.758757 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/49c341d1-5089-4bc2-86a0-a5e165cfcc6b-audit-policies" (OuterVolumeSpecName: "audit-policies") pod "49c341d1-5089-4bc2-86a0-a5e165cfcc6b" (UID: "49c341d1-5089-4bc2-86a0-a5e165cfcc6b"). InnerVolumeSpecName "audit-policies". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.758809 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c03ee662-fb2f-4fc4-a2c1-af487c19d254-kube-api-access-v47cf" (OuterVolumeSpecName: "kube-api-access-v47cf") pod "c03ee662-fb2f-4fc4-a2c1-af487c19d254" (UID: "c03ee662-fb2f-4fc4-a2c1-af487c19d254"). InnerVolumeSpecName "kube-api-access-v47cf". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.758801 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6509e943-70c6-444c-bc41-48a544e36fbd-serving-cert" (OuterVolumeSpecName: "serving-cert") pod "6509e943-70c6-444c-bc41-48a544e36fbd" (UID: "6509e943-70c6-444c-bc41-48a544e36fbd"). InnerVolumeSpecName "serving-cert". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: E1203 11:20:31.759585 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8 podName:8f668bae-612b-4b75-9490-919e737c6a3b nodeName:}" failed. No retries permitted until 2025-12-03 11:20:32.259438388 +0000 UTC m=+22.847680152 (durationBeforeRetry 500ms). Error: UnmountVolume.TearDown failed for volume "pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8" (UniqueName: "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8") pod "8f668bae-612b-4b75-9490-919e737c6a3b" (UID: "8f668bae-612b-4b75-9490-919e737c6a3b") : kubernetes.io/csi: Unmounter.TearDownAt failed to get CSI client: driver name kubevirt.io.hostpath-provisioner not found in the list of registered CSI drivers Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.760230 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9d4552c7-cd75-42dd-8880-30dd377c49a4-kube-api-access-pcxfs" (OuterVolumeSpecName: "kube-api-access-pcxfs") pod "9d4552c7-cd75-42dd-8880-30dd377c49a4" (UID: "9d4552c7-cd75-42dd-8880-30dd377c49a4"). InnerVolumeSpecName "kube-api-access-pcxfs". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.760435 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/308be0ea-9f5f-4b29-aeb1-5abd31a0b17b-kube-api-access-6ccd8" (OuterVolumeSpecName: "kube-api-access-6ccd8") pod "308be0ea-9f5f-4b29-aeb1-5abd31a0b17b" (UID: "308be0ea-9f5f-4b29-aeb1-5abd31a0b17b"). InnerVolumeSpecName "kube-api-access-6ccd8". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.760490 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b6312bbd-5731-4ea0-a20f-81d5a57df44a-srv-cert" (OuterVolumeSpecName: "srv-cert") pod "b6312bbd-5731-4ea0-a20f-81d5a57df44a" (UID: "b6312bbd-5731-4ea0-a20f-81d5a57df44a"). InnerVolumeSpecName "srv-cert". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: E1203 11:20:31.760794 5052 configmap.go:193] Couldn't get configMap openshift-network-console/networking-console-plugin: object "openshift-network-console"/"networking-console-plugin" not registered Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.760869 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0b574797-001e-440a-8f4e-c0be86edad0f-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "0b574797-001e-440a-8f4e-c0be86edad0f" (UID: "0b574797-001e-440a-8f4e-c0be86edad0f"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.760994 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1bf7eb37-55a3-4c65-b768-a94c82151e69-image-import-ca" (OuterVolumeSpecName: "image-import-ca") pod "1bf7eb37-55a3-4c65-b768-a94c82151e69" (UID: "1bf7eb37-55a3-4c65-b768-a94c82151e69"). InnerVolumeSpecName "image-import-ca". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.761276 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5441d097-087c-4d9a-baa8-b210afa90fc9-kube-api-access-2d4wz" (OuterVolumeSpecName: "kube-api-access-2d4wz") pod "5441d097-087c-4d9a-baa8-b210afa90fc9" (UID: "5441d097-087c-4d9a-baa8-b210afa90fc9"). InnerVolumeSpecName "kube-api-access-2d4wz". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.761371 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/57a731c4-ef35-47a8-b875-bfb08a7f8011-utilities" (OuterVolumeSpecName: "utilities") pod "57a731c4-ef35-47a8-b875-bfb08a7f8011" (UID: "57a731c4-ef35-47a8-b875-bfb08a7f8011"). InnerVolumeSpecName "utilities". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.761415 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/49c341d1-5089-4bc2-86a0-a5e165cfcc6b-v4-0-config-system-service-ca" (OuterVolumeSpecName: "v4-0-config-system-service-ca") pod "49c341d1-5089-4bc2-86a0-a5e165cfcc6b" (UID: "49c341d1-5089-4bc2-86a0-a5e165cfcc6b"). InnerVolumeSpecName "v4-0-config-system-service-ca". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.761601 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/87cf06ed-a83f-41a7-828d-70653580a8cb-metrics-tls" (OuterVolumeSpecName: "metrics-tls") pod "87cf06ed-a83f-41a7-828d-70653580a8cb" (UID: "87cf06ed-a83f-41a7-828d-70653580a8cb"). InnerVolumeSpecName "metrics-tls". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.761665 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bf126b07-da06-4140-9a57-dfd54fc6b486-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "bf126b07-da06-4140-9a57-dfd54fc6b486" (UID: "bf126b07-da06-4140-9a57-dfd54fc6b486"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.761961 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/49c341d1-5089-4bc2-86a0-a5e165cfcc6b-v4-0-config-system-serving-cert" (OuterVolumeSpecName: "v4-0-config-system-serving-cert") pod "49c341d1-5089-4bc2-86a0-a5e165cfcc6b" (UID: "49c341d1-5089-4bc2-86a0-a5e165cfcc6b"). InnerVolumeSpecName "v4-0-config-system-serving-cert". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.762031 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a31745f5-9847-4afe-82a5-3161cc66ca93-kube-api-access-lz9wn" (OuterVolumeSpecName: "kube-api-access-lz9wn") pod "a31745f5-9847-4afe-82a5-3161cc66ca93" (UID: "a31745f5-9847-4afe-82a5-3161cc66ca93"). InnerVolumeSpecName "kube-api-access-lz9wn". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.762267 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c03ee662-fb2f-4fc4-a2c1-af487c19d254-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "c03ee662-fb2f-4fc4-a2c1-af487c19d254" (UID: "c03ee662-fb2f-4fc4-a2c1-af487c19d254"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.762305 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6731426b-95fe-49ff-bb5f-40441049fde2-kube-api-access-x7zkh" (OuterVolumeSpecName: "kube-api-access-x7zkh") pod "6731426b-95fe-49ff-bb5f-40441049fde2" (UID: "6731426b-95fe-49ff-bb5f-40441049fde2"). InnerVolumeSpecName "kube-api-access-x7zkh". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.762345 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/49c341d1-5089-4bc2-86a0-a5e165cfcc6b-v4-0-config-system-router-certs" (OuterVolumeSpecName: "v4-0-config-system-router-certs") pod "49c341d1-5089-4bc2-86a0-a5e165cfcc6b" (UID: "49c341d1-5089-4bc2-86a0-a5e165cfcc6b"). InnerVolumeSpecName "v4-0-config-system-router-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.762440 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/09ae3b1a-e8e7-4524-b54b-61eab6f9239a-etcd-serving-ca" (OuterVolumeSpecName: "etcd-serving-ca") pod "09ae3b1a-e8e7-4524-b54b-61eab6f9239a" (UID: "09ae3b1a-e8e7-4524-b54b-61eab6f9239a"). InnerVolumeSpecName "etcd-serving-ca". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.762447 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/210d8245-ebfc-4e3b-ac4a-e21ce76f9a7c-serving-cert" (OuterVolumeSpecName: "serving-cert") pod "210d8245-ebfc-4e3b-ac4a-e21ce76f9a7c" (UID: "210d8245-ebfc-4e3b-ac4a-e21ce76f9a7c"). InnerVolumeSpecName "serving-cert". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.762562 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d-kube-api-access-x4zgh" (OuterVolumeSpecName: "kube-api-access-x4zgh") pod "b11524ee-3fca-4b1b-9cdf-6da289fdbc7d" (UID: "b11524ee-3fca-4b1b-9cdf-6da289fdbc7d"). InnerVolumeSpecName "kube-api-access-x4zgh". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.762738 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/20b0d48f-5fd6-431c-a545-e3c800c7b866-kube-api-access-w9rds" (OuterVolumeSpecName: "kube-api-access-w9rds") pod "20b0d48f-5fd6-431c-a545-e3c800c7b866" (UID: "20b0d48f-5fd6-431c-a545-e3c800c7b866"). InnerVolumeSpecName "kube-api-access-w9rds". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.763094 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/49c341d1-5089-4bc2-86a0-a5e165cfcc6b-v4-0-config-user-template-error" (OuterVolumeSpecName: "v4-0-config-user-template-error") pod "49c341d1-5089-4bc2-86a0-a5e165cfcc6b" (UID: "49c341d1-5089-4bc2-86a0-a5e165cfcc6b"). InnerVolumeSpecName "v4-0-config-user-template-error". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.763070 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/210d8245-ebfc-4e3b-ac4a-e21ce76f9a7c-config" (OuterVolumeSpecName: "config") pod "210d8245-ebfc-4e3b-ac4a-e21ce76f9a7c" (UID: "210d8245-ebfc-4e3b-ac4a-e21ce76f9a7c"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.763328 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7539238d-5fe0-46ed-884e-1c3b566537ec-serving-cert" (OuterVolumeSpecName: "serving-cert") pod "7539238d-5fe0-46ed-884e-1c3b566537ec" (UID: "7539238d-5fe0-46ed-884e-1c3b566537ec"). InnerVolumeSpecName "serving-cert". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.763345 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/43509403-f426-496e-be36-56cef71462f5-kube-api-access-qg5z5" (OuterVolumeSpecName: "kube-api-access-qg5z5") pod "43509403-f426-496e-be36-56cef71462f5" (UID: "43509403-f426-496e-be36-56cef71462f5"). InnerVolumeSpecName "kube-api-access-qg5z5". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.763798 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cd70aa09-68dd-4d64-bd6f-156fe6d1dc6d-kube-api-access-x2m85" (OuterVolumeSpecName: "kube-api-access-x2m85") pod "cd70aa09-68dd-4d64-bd6f-156fe6d1dc6d" (UID: "cd70aa09-68dd-4d64-bd6f-156fe6d1dc6d"). InnerVolumeSpecName "kube-api-access-x2m85". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.763981 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/49c341d1-5089-4bc2-86a0-a5e165cfcc6b-v4-0-config-system-session" (OuterVolumeSpecName: "v4-0-config-system-session") pod "49c341d1-5089-4bc2-86a0-a5e165cfcc6b" (UID: "49c341d1-5089-4bc2-86a0-a5e165cfcc6b"). InnerVolumeSpecName "v4-0-config-system-session". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.765382 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bd23aa5c-e532-4e53-bccf-e79f130c5ae8-kube-api-access-jhbk2" (OuterVolumeSpecName: "kube-api-access-jhbk2") pod "bd23aa5c-e532-4e53-bccf-e79f130c5ae8" (UID: "bd23aa5c-e532-4e53-bccf-e79f130c5ae8"). InnerVolumeSpecName "kube-api-access-jhbk2". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.765408 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1386a44e-36a2-460c-96d0-0359d2b6f0f5-kube-api-access" (OuterVolumeSpecName: "kube-api-access") pod "1386a44e-36a2-460c-96d0-0359d2b6f0f5" (UID: "1386a44e-36a2-460c-96d0-0359d2b6f0f5"). InnerVolumeSpecName "kube-api-access". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.765704 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/308be0ea-9f5f-4b29-aeb1-5abd31a0b17b-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "308be0ea-9f5f-4b29-aeb1-5abd31a0b17b" (UID: "308be0ea-9f5f-4b29-aeb1-5abd31a0b17b"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.766224 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9d4552c7-cd75-42dd-8880-30dd377c49a4-serving-cert" (OuterVolumeSpecName: "serving-cert") pod "9d4552c7-cd75-42dd-8880-30dd377c49a4" (UID: "9d4552c7-cd75-42dd-8880-30dd377c49a4"). InnerVolumeSpecName "serving-cert". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.766433 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0b78653f-4ff9-4508-8672-245ed9b561e3-serving-cert" (OuterVolumeSpecName: "serving-cert") pod "0b78653f-4ff9-4508-8672-245ed9b561e3" (UID: "0b78653f-4ff9-4508-8672-245ed9b561e3"). InnerVolumeSpecName "serving-cert". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.766444 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1bf7eb37-55a3-4c65-b768-a94c82151e69-audit" (OuterVolumeSpecName: "audit") pod "1bf7eb37-55a3-4c65-b768-a94c82151e69" (UID: "1bf7eb37-55a3-4c65-b768-a94c82151e69"). InnerVolumeSpecName "audit". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: E1203 11:20:31.767030 5052 secret.go:188] Couldn't get secret openshift-network-console/networking-console-plugin-cert: object "openshift-network-console"/"networking-console-plugin-cert" not registered Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.767086 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a31745f5-9847-4afe-82a5-3161cc66ca93-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "a31745f5-9847-4afe-82a5-3161cc66ca93" (UID: "a31745f5-9847-4afe-82a5-3161cc66ca93"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.767109 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/09efc573-dbb6-4249-bd59-9b87aba8dd28-kube-api-access-8tdtz" (OuterVolumeSpecName: "kube-api-access-8tdtz") pod "09efc573-dbb6-4249-bd59-9b87aba8dd28" (UID: "09efc573-dbb6-4249-bd59-9b87aba8dd28"). InnerVolumeSpecName "kube-api-access-8tdtz". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.767940 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/09ae3b1a-e8e7-4524-b54b-61eab6f9239a-kube-api-access-zkvpv" (OuterVolumeSpecName: "kube-api-access-zkvpv") pod "09ae3b1a-e8e7-4524-b54b-61eab6f9239a" (UID: "09ae3b1a-e8e7-4524-b54b-61eab6f9239a"). InnerVolumeSpecName "kube-api-access-zkvpv". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.763341 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/925f1c65-6136-48ba-85aa-3a3b50560753-ovn-control-plane-metrics-cert" (OuterVolumeSpecName: "ovn-control-plane-metrics-cert") pod "925f1c65-6136-48ba-85aa-3a3b50560753" (UID: "925f1c65-6136-48ba-85aa-3a3b50560753"). InnerVolumeSpecName "ovn-control-plane-metrics-cert". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.767967 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1d611f23-29be-4491-8495-bee1670e935f-kube-api-access-bf2bz" (OuterVolumeSpecName: "kube-api-access-bf2bz") pod "1d611f23-29be-4491-8495-bee1670e935f" (UID: "1d611f23-29be-4491-8495-bee1670e935f"). InnerVolumeSpecName "kube-api-access-bf2bz". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.768360 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8f668bae-612b-4b75-9490-919e737c6a3b-kube-api-access-kfwg7" (OuterVolumeSpecName: "kube-api-access-kfwg7") pod "8f668bae-612b-4b75-9490-919e737c6a3b" (UID: "8f668bae-612b-4b75-9490-919e737c6a3b"). InnerVolumeSpecName "kube-api-access-kfwg7". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.768457 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/22c825df-677d-4ca6-82db-3454ed06e783-kube-api-access-7c4vf" (OuterVolumeSpecName: "kube-api-access-7c4vf") pod "22c825df-677d-4ca6-82db-3454ed06e783" (UID: "22c825df-677d-4ca6-82db-3454ed06e783"). InnerVolumeSpecName "kube-api-access-7c4vf". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.768700 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0b78653f-4ff9-4508-8672-245ed9b561e3-kube-api-access" (OuterVolumeSpecName: "kube-api-access") pod "0b78653f-4ff9-4508-8672-245ed9b561e3" (UID: "0b78653f-4ff9-4508-8672-245ed9b561e3"). InnerVolumeSpecName "kube-api-access". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.768878 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7539238d-5fe0-46ed-884e-1c3b566537ec-config" (OuterVolumeSpecName: "config") pod "7539238d-5fe0-46ed-884e-1c3b566537ec" (UID: "7539238d-5fe0-46ed-884e-1c3b566537ec"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.769020 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1386a44e-36a2-460c-96d0-0359d2b6f0f5-config" (OuterVolumeSpecName: "config") pod "1386a44e-36a2-460c-96d0-0359d2b6f0f5" (UID: "1386a44e-36a2-460c-96d0-0359d2b6f0f5"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.769113 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/09ae3b1a-e8e7-4524-b54b-61eab6f9239a-serving-cert" (OuterVolumeSpecName: "serving-cert") pod "09ae3b1a-e8e7-4524-b54b-61eab6f9239a" (UID: "09ae3b1a-e8e7-4524-b54b-61eab6f9239a"). InnerVolumeSpecName "serving-cert". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.769199 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fda69060-fa79-4696-b1a6-7980f124bf7c-mcd-auth-proxy-config" (OuterVolumeSpecName: "mcd-auth-proxy-config") pod "fda69060-fa79-4696-b1a6-7980f124bf7c" (UID: "fda69060-fa79-4696-b1a6-7980f124bf7c"). InnerVolumeSpecName "mcd-auth-proxy-config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.769252 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/87cf06ed-a83f-41a7-828d-70653580a8cb-config-volume" (OuterVolumeSpecName: "config-volume") pod "87cf06ed-a83f-41a7-828d-70653580a8cb" (UID: "87cf06ed-a83f-41a7-828d-70653580a8cb"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.769463 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a31745f5-9847-4afe-82a5-3161cc66ca93-metrics-tls" (OuterVolumeSpecName: "metrics-tls") pod "a31745f5-9847-4afe-82a5-3161cc66ca93" (UID: "a31745f5-9847-4afe-82a5-3161cc66ca93"). InnerVolumeSpecName "metrics-tls". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.768318 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4bb40260-dbaa-4fb0-84df-5e680505d512-cni-binary-copy" (OuterVolumeSpecName: "cni-binary-copy") pod "4bb40260-dbaa-4fb0-84df-5e680505d512" (UID: "4bb40260-dbaa-4fb0-84df-5e680505d512"). InnerVolumeSpecName "cni-binary-copy". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.769656 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bf126b07-da06-4140-9a57-dfd54fc6b486-image-registry-operator-tls" (OuterVolumeSpecName: "image-registry-operator-tls") pod "bf126b07-da06-4140-9a57-dfd54fc6b486" (UID: "bf126b07-da06-4140-9a57-dfd54fc6b486"). InnerVolumeSpecName "image-registry-operator-tls". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.769712 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e7e6199b-1264-4501-8953-767f51328d08-serving-cert" (OuterVolumeSpecName: "serving-cert") pod "e7e6199b-1264-4501-8953-767f51328d08" (UID: "e7e6199b-1264-4501-8953-767f51328d08"). InnerVolumeSpecName "serving-cert". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.769789 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6509e943-70c6-444c-bc41-48a544e36fbd-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "6509e943-70c6-444c-bc41-48a544e36fbd" (UID: "6509e943-70c6-444c-bc41-48a544e36fbd"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.770229 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/496e6271-fb68-4057-954e-a0d97a4afa3f-serving-cert" (OuterVolumeSpecName: "serving-cert") pod "496e6271-fb68-4057-954e-a0d97a4afa3f" (UID: "496e6271-fb68-4057-954e-a0d97a4afa3f"). InnerVolumeSpecName "serving-cert". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.770363 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7539238d-5fe0-46ed-884e-1c3b566537ec-kube-api-access-tk88c" (OuterVolumeSpecName: "kube-api-access-tk88c") pod "7539238d-5fe0-46ed-884e-1c3b566537ec" (UID: "7539238d-5fe0-46ed-884e-1c3b566537ec"). InnerVolumeSpecName "kube-api-access-tk88c". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.770379 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1bf7eb37-55a3-4c65-b768-a94c82151e69-kube-api-access-sb6h7" (OuterVolumeSpecName: "kube-api-access-sb6h7") pod "1bf7eb37-55a3-4c65-b768-a94c82151e69" (UID: "1bf7eb37-55a3-4c65-b768-a94c82151e69"). InnerVolumeSpecName "kube-api-access-sb6h7". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.770428 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/09efc573-dbb6-4249-bd59-9b87aba8dd28-etcd-client" (OuterVolumeSpecName: "etcd-client") pod "09efc573-dbb6-4249-bd59-9b87aba8dd28" (UID: "09efc573-dbb6-4249-bd59-9b87aba8dd28"). InnerVolumeSpecName "etcd-client". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.770747 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8cea82b4-6893-4ddc-af9f-1bb5ae425c5b-serving-cert" (OuterVolumeSpecName: "serving-cert") pod "8cea82b4-6893-4ddc-af9f-1bb5ae425c5b" (UID: "8cea82b4-6893-4ddc-af9f-1bb5ae425c5b"). InnerVolumeSpecName "serving-cert". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.771271 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8cea82b4-6893-4ddc-af9f-1bb5ae425c5b-config" (OuterVolumeSpecName: "config") pod "8cea82b4-6893-4ddc-af9f-1bb5ae425c5b" (UID: "8cea82b4-6893-4ddc-af9f-1bb5ae425c5b"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.771525 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/43509403-f426-496e-be36-56cef71462f5-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "43509403-f426-496e-be36-56cef71462f5" (UID: "43509403-f426-496e-be36-56cef71462f5"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.771875 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/96b93a3a-6083-4aea-8eab-fe1aa8245ad9-kube-api-access-nzwt7" (OuterVolumeSpecName: "kube-api-access-nzwt7") pod "96b93a3a-6083-4aea-8eab-fe1aa8245ad9" (UID: "96b93a3a-6083-4aea-8eab-fe1aa8245ad9"). InnerVolumeSpecName "kube-api-access-nzwt7". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.771911 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b6cd30de-2eeb-49a2-ab40-9167f4560ff5-marketplace-trusted-ca" (OuterVolumeSpecName: "marketplace-trusted-ca") pod "b6cd30de-2eeb-49a2-ab40-9167f4560ff5" (UID: "b6cd30de-2eeb-49a2-ab40-9167f4560ff5"). InnerVolumeSpecName "marketplace-trusted-ca". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.772159 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6ea678ab-3438-413e-bfe3-290ae7725660-ovnkube-config" (OuterVolumeSpecName: "ovnkube-config") pod "6ea678ab-3438-413e-bfe3-290ae7725660" (UID: "6ea678ab-3438-413e-bfe3-290ae7725660"). InnerVolumeSpecName "ovnkube-config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.772251 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bc5039c0-ea34-426b-a2b7-fbbc87b49a6d-serving-cert" (OuterVolumeSpecName: "serving-cert") pod "bc5039c0-ea34-426b-a2b7-fbbc87b49a6d" (UID: "bc5039c0-ea34-426b-a2b7-fbbc87b49a6d"). InnerVolumeSpecName "serving-cert". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.772351 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/49c341d1-5089-4bc2-86a0-a5e165cfcc6b-v4-0-config-user-template-provider-selection" (OuterVolumeSpecName: "v4-0-config-user-template-provider-selection") pod "49c341d1-5089-4bc2-86a0-a5e165cfcc6b" (UID: "49c341d1-5089-4bc2-86a0-a5e165cfcc6b"). InnerVolumeSpecName "v4-0-config-user-template-provider-selection". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.772270 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/49c341d1-5089-4bc2-86a0-a5e165cfcc6b-v4-0-config-user-template-login" (OuterVolumeSpecName: "v4-0-config-user-template-login") pod "49c341d1-5089-4bc2-86a0-a5e165cfcc6b" (UID: "49c341d1-5089-4bc2-86a0-a5e165cfcc6b"). InnerVolumeSpecName "v4-0-config-user-template-login". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.772671 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bc5039c0-ea34-426b-a2b7-fbbc87b49a6d-available-featuregates" (OuterVolumeSpecName: "available-featuregates") pod "bc5039c0-ea34-426b-a2b7-fbbc87b49a6d" (UID: "bc5039c0-ea34-426b-a2b7-fbbc87b49a6d"). InnerVolumeSpecName "available-featuregates". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.774326 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/efdd0498-1daa-4136-9a4a-3b948c2293fc-kube-api-access-fqsjt" (OuterVolumeSpecName: "kube-api-access-fqsjt") pod "efdd0498-1daa-4136-9a4a-3b948c2293fc" (UID: "efdd0498-1daa-4136-9a4a-3b948c2293fc"). InnerVolumeSpecName "kube-api-access-fqsjt". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.774661 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6ea678ab-3438-413e-bfe3-290ae7725660-kube-api-access-htfz6" (OuterVolumeSpecName: "kube-api-access-htfz6") pod "6ea678ab-3438-413e-bfe3-290ae7725660" (UID: "6ea678ab-3438-413e-bfe3-290ae7725660"). InnerVolumeSpecName "kube-api-access-htfz6". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.774928 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f88749ec-7931-4ee7-b3fc-1ec5e11f92e9-kube-api-access-dbsvg" (OuterVolumeSpecName: "kube-api-access-dbsvg") pod "f88749ec-7931-4ee7-b3fc-1ec5e11f92e9" (UID: "f88749ec-7931-4ee7-b3fc-1ec5e11f92e9"). InnerVolumeSpecName "kube-api-access-dbsvg". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.774942 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c03ee662-fb2f-4fc4-a2c1-af487c19d254-default-certificate" (OuterVolumeSpecName: "default-certificate") pod "c03ee662-fb2f-4fc4-a2c1-af487c19d254" (UID: "c03ee662-fb2f-4fc4-a2c1-af487c19d254"). InnerVolumeSpecName "default-certificate". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.775017 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7583ce53-e0fe-4a16-9e4d-50516596a136-kube-api-access-xcphl" (OuterVolumeSpecName: "kube-api-access-xcphl") pod "7583ce53-e0fe-4a16-9e4d-50516596a136" (UID: "7583ce53-e0fe-4a16-9e4d-50516596a136"). InnerVolumeSpecName "kube-api-access-xcphl". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.775160 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/22c825df-677d-4ca6-82db-3454ed06e783-config" (OuterVolumeSpecName: "config") pod "22c825df-677d-4ca6-82db-3454ed06e783" (UID: "22c825df-677d-4ca6-82db-3454ed06e783"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.775426 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5441d097-087c-4d9a-baa8-b210afa90fc9-serving-cert" (OuterVolumeSpecName: "serving-cert") pod "5441d097-087c-4d9a-baa8-b210afa90fc9" (UID: "5441d097-087c-4d9a-baa8-b210afa90fc9"). InnerVolumeSpecName "serving-cert". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.775426 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/09efc573-dbb6-4249-bd59-9b87aba8dd28-etcd-service-ca" (OuterVolumeSpecName: "etcd-service-ca") pod "09efc573-dbb6-4249-bd59-9b87aba8dd28" (UID: "09efc573-dbb6-4249-bd59-9b87aba8dd28"). InnerVolumeSpecName "etcd-service-ca". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.775826 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/925f1c65-6136-48ba-85aa-3a3b50560753-env-overrides" (OuterVolumeSpecName: "env-overrides") pod "925f1c65-6136-48ba-85aa-3a3b50560753" (UID: "925f1c65-6136-48ba-85aa-3a3b50560753"). InnerVolumeSpecName "env-overrides". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.775888 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0b78653f-4ff9-4508-8672-245ed9b561e3-service-ca" (OuterVolumeSpecName: "service-ca") pod "0b78653f-4ff9-4508-8672-245ed9b561e3" (UID: "0b78653f-4ff9-4508-8672-245ed9b561e3"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.776917 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6509e943-70c6-444c-bc41-48a544e36fbd-kube-api-access-6g6sz" (OuterVolumeSpecName: "kube-api-access-6g6sz") pod "6509e943-70c6-444c-bc41-48a544e36fbd" (UID: "6509e943-70c6-444c-bc41-48a544e36fbd"). InnerVolumeSpecName "kube-api-access-6g6sz". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.777860 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7bb08738-c794-4ee8-9972-3a62ca171029-cni-sysctl-allowlist" (OuterVolumeSpecName: "cni-sysctl-allowlist") pod "7bb08738-c794-4ee8-9972-3a62ca171029" (UID: "7bb08738-c794-4ee8-9972-3a62ca171029"). InnerVolumeSpecName "cni-sysctl-allowlist". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.777917 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0b574797-001e-440a-8f4e-c0be86edad0f-mcc-auth-proxy-config" (OuterVolumeSpecName: "mcc-auth-proxy-config") pod "0b574797-001e-440a-8f4e-c0be86edad0f" (UID: "0b574797-001e-440a-8f4e-c0be86edad0f"). InnerVolumeSpecName "mcc-auth-proxy-config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.777981 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6ea678ab-3438-413e-bfe3-290ae7725660-ovnkube-script-lib" (OuterVolumeSpecName: "ovnkube-script-lib") pod "6ea678ab-3438-413e-bfe3-290ae7725660" (UID: "6ea678ab-3438-413e-bfe3-290ae7725660"). InnerVolumeSpecName "ovnkube-script-lib". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.778344 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8f668bae-612b-4b75-9490-919e737c6a3b-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "8f668bae-612b-4b75-9490-919e737c6a3b" (UID: "8f668bae-612b-4b75-9490-919e737c6a3b"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.778887 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/925f1c65-6136-48ba-85aa-3a3b50560753-ovnkube-config" (OuterVolumeSpecName: "ovnkube-config") pod "925f1c65-6136-48ba-85aa-3a3b50560753" (UID: "925f1c65-6136-48ba-85aa-3a3b50560753"). InnerVolumeSpecName "ovnkube-config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.780139 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6731426b-95fe-49ff-bb5f-40441049fde2-control-plane-machine-set-operator-tls" (OuterVolumeSpecName: "control-plane-machine-set-operator-tls") pod "6731426b-95fe-49ff-bb5f-40441049fde2" (UID: "6731426b-95fe-49ff-bb5f-40441049fde2"). InnerVolumeSpecName "control-plane-machine-set-operator-tls". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.780284 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a0128f3a-b052-44ed-a84e-c4c8aaf17c13-kube-api-access-gf66m" (OuterVolumeSpecName: "kube-api-access-gf66m") pod "a0128f3a-b052-44ed-a84e-c4c8aaf17c13" (UID: "a0128f3a-b052-44ed-a84e-c4c8aaf17c13"). InnerVolumeSpecName "kube-api-access-gf66m". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.780887 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/09ae3b1a-e8e7-4524-b54b-61eab6f9239a-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "09ae3b1a-e8e7-4524-b54b-61eab6f9239a" (UID: "09ae3b1a-e8e7-4524-b54b-61eab6f9239a"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.781148 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c03ee662-fb2f-4fc4-a2c1-af487c19d254-service-ca-bundle" (OuterVolumeSpecName: "service-ca-bundle") pod "c03ee662-fb2f-4fc4-a2c1-af487c19d254" (UID: "c03ee662-fb2f-4fc4-a2c1-af487c19d254"). InnerVolumeSpecName "service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.781197 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/57a731c4-ef35-47a8-b875-bfb08a7f8011-kube-api-access-cfbct" (OuterVolumeSpecName: "kube-api-access-cfbct") pod "57a731c4-ef35-47a8-b875-bfb08a7f8011" (UID: "57a731c4-ef35-47a8-b875-bfb08a7f8011"). InnerVolumeSpecName "kube-api-access-cfbct". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.781172 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5fe579f8-e8a6-4643-bce5-a661393c4dde-certs" (OuterVolumeSpecName: "certs") pod "5fe579f8-e8a6-4643-bce5-a661393c4dde" (UID: "5fe579f8-e8a6-4643-bce5-a661393c4dde"). InnerVolumeSpecName "certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.781355 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/31d8b7a1-420e-4252-a5b7-eebe8a111292-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "31d8b7a1-420e-4252-a5b7-eebe8a111292" (UID: "31d8b7a1-420e-4252-a5b7-eebe8a111292"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: E1203 11:20:31.781479 5052 projected.go:288] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.782032 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/43509403-f426-496e-be36-56cef71462f5-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "43509403-f426-496e-be36-56cef71462f5" (UID: "43509403-f426-496e-be36-56cef71462f5"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: E1203 11:20:31.782118 5052 projected.go:288] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.782199 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/25e176fe-21b4-4974-b1ed-c8b94f112a7f-signing-key" (OuterVolumeSpecName: "signing-key") pod "25e176fe-21b4-4974-b1ed-c8b94f112a7f" (UID: "25e176fe-21b4-4974-b1ed-c8b94f112a7f"). InnerVolumeSpecName "signing-key". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.782230 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5b88f790-22fa-440e-b583-365168c0b23d-kube-api-access-jkwtn" (OuterVolumeSpecName: "kube-api-access-jkwtn") pod "5b88f790-22fa-440e-b583-365168c0b23d" (UID: "5b88f790-22fa-440e-b583-365168c0b23d"). InnerVolumeSpecName "kube-api-access-jkwtn". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.781400 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e7e6199b-1264-4501-8953-767f51328d08-kube-api-access" (OuterVolumeSpecName: "kube-api-access") pod "e7e6199b-1264-4501-8953-767f51328d08" (UID: "e7e6199b-1264-4501-8953-767f51328d08"). InnerVolumeSpecName "kube-api-access". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.781558 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3ab1a177-2de0-46d9-b765-d0d0649bb42e-kube-api-access-4d4hj" (OuterVolumeSpecName: "kube-api-access-4d4hj") pod "3ab1a177-2de0-46d9-b765-d0d0649bb42e" (UID: "3ab1a177-2de0-46d9-b765-d0d0649bb42e"). InnerVolumeSpecName "kube-api-access-4d4hj". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.782325 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9d4552c7-cd75-42dd-8880-30dd377c49a4-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "9d4552c7-cd75-42dd-8880-30dd377c49a4" (UID: "9d4552c7-cd75-42dd-8880-30dd377c49a4"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.781779 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/496e6271-fb68-4057-954e-a0d97a4afa3f-kube-api-access" (OuterVolumeSpecName: "kube-api-access") pod "496e6271-fb68-4057-954e-a0d97a4afa3f" (UID: "496e6271-fb68-4057-954e-a0d97a4afa3f"). InnerVolumeSpecName "kube-api-access". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.781805 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b6312bbd-5731-4ea0-a20f-81d5a57df44a-profile-collector-cert" (OuterVolumeSpecName: "profile-collector-cert") pod "b6312bbd-5731-4ea0-a20f-81d5a57df44a" (UID: "b6312bbd-5731-4ea0-a20f-81d5a57df44a"). InnerVolumeSpecName "profile-collector-cert". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: E1203 11:20:31.782321 5052 projected.go:194] Error preparing data for projected volume kube-api-access-cqllr for pod openshift-network-diagnostics/network-check-target-xd92c: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.782915 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/925f1c65-6136-48ba-85aa-3a3b50560753-kube-api-access-s4n52" (OuterVolumeSpecName: "kube-api-access-s4n52") pod "925f1c65-6136-48ba-85aa-3a3b50560753" (UID: "925f1c65-6136-48ba-85aa-3a3b50560753"). InnerVolumeSpecName "kube-api-access-s4n52". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.783095 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/31d8b7a1-420e-4252-a5b7-eebe8a111292-images" (OuterVolumeSpecName: "images") pod "31d8b7a1-420e-4252-a5b7-eebe8a111292" (UID: "31d8b7a1-420e-4252-a5b7-eebe8a111292"). InnerVolumeSpecName "images". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.783419 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bf126b07-da06-4140-9a57-dfd54fc6b486-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "bf126b07-da06-4140-9a57-dfd54fc6b486" (UID: "bf126b07-da06-4140-9a57-dfd54fc6b486"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.783462 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4bb40260-dbaa-4fb0-84df-5e680505d512-kube-api-access-2w9zh" (OuterVolumeSpecName: "kube-api-access-2w9zh") pod "4bb40260-dbaa-4fb0-84df-5e680505d512" (UID: "4bb40260-dbaa-4fb0-84df-5e680505d512"). InnerVolumeSpecName "kube-api-access-2w9zh". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.783700 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/09ae3b1a-e8e7-4524-b54b-61eab6f9239a-etcd-client" (OuterVolumeSpecName: "etcd-client") pod "09ae3b1a-e8e7-4524-b54b-61eab6f9239a" (UID: "09ae3b1a-e8e7-4524-b54b-61eab6f9239a"). InnerVolumeSpecName "etcd-client". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.783731 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5441d097-087c-4d9a-baa8-b210afa90fc9-config" (OuterVolumeSpecName: "config") pod "5441d097-087c-4d9a-baa8-b210afa90fc9" (UID: "5441d097-087c-4d9a-baa8-b210afa90fc9"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.784249 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/49c341d1-5089-4bc2-86a0-a5e165cfcc6b-v4-0-config-user-idp-0-file-data" (OuterVolumeSpecName: "v4-0-config-user-idp-0-file-data") pod "49c341d1-5089-4bc2-86a0-a5e165cfcc6b" (UID: "49c341d1-5089-4bc2-86a0-a5e165cfcc6b"). InnerVolumeSpecName "v4-0-config-user-idp-0-file-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: E1203 11:20:31.784367 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5fe485a1-e14f-4c09-b5b9-f252bc42b7e8-nginx-conf podName:5fe485a1-e14f-4c09-b5b9-f252bc42b7e8 nodeName:}" failed. No retries permitted until 2025-12-03 11:20:32.284332184 +0000 UTC m=+22.872574138 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "nginx-conf" (UniqueName: "kubernetes.io/configmap/5fe485a1-e14f-4c09-b5b9-f252bc42b7e8-nginx-conf") pod "networking-console-plugin-85b44fc459-gdk6g" (UID: "5fe485a1-e14f-4c09-b5b9-f252bc42b7e8") : object "openshift-network-console"/"networking-console-plugin" not registered Dec 03 11:20:31 crc kubenswrapper[5052]: E1203 11:20:31.785266 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5fe485a1-e14f-4c09-b5b9-f252bc42b7e8-networking-console-plugin-cert podName:5fe485a1-e14f-4c09-b5b9-f252bc42b7e8 nodeName:}" failed. No retries permitted until 2025-12-03 11:20:32.285227608 +0000 UTC m=+22.873469522 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/5fe485a1-e14f-4c09-b5b9-f252bc42b7e8-networking-console-plugin-cert") pod "networking-console-plugin-85b44fc459-gdk6g" (UID: "5fe485a1-e14f-4c09-b5b9-f252bc42b7e8") : object "openshift-network-console"/"networking-console-plugin-cert" not registered Dec 03 11:20:31 crc kubenswrapper[5052]: E1203 11:20:31.785294 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3b6479f0-333b-4a96-9adf-2099afdc2447-kube-api-access-cqllr podName:3b6479f0-333b-4a96-9adf-2099afdc2447 nodeName:}" failed. No retries permitted until 2025-12-03 11:20:32.28528233 +0000 UTC m=+22.873524244 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-cqllr" (UniqueName: "kubernetes.io/projected/3b6479f0-333b-4a96-9adf-2099afdc2447-kube-api-access-cqllr") pod "network-check-target-xd92c" (UID: "3b6479f0-333b-4a96-9adf-2099afdc2447") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.784985 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/09ae3b1a-e8e7-4524-b54b-61eab6f9239a-audit-policies" (OuterVolumeSpecName: "audit-policies") pod "09ae3b1a-e8e7-4524-b54b-61eab6f9239a" (UID: "09ae3b1a-e8e7-4524-b54b-61eab6f9239a"). InnerVolumeSpecName "audit-policies". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.785565 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bf126b07-da06-4140-9a57-dfd54fc6b486-kube-api-access-rnphk" (OuterVolumeSpecName: "kube-api-access-rnphk") pod "bf126b07-da06-4140-9a57-dfd54fc6b486" (UID: "bf126b07-da06-4140-9a57-dfd54fc6b486"). InnerVolumeSpecName "kube-api-access-rnphk". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.785690 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6509e943-70c6-444c-bc41-48a544e36fbd-config" (OuterVolumeSpecName: "config") pod "6509e943-70c6-444c-bc41-48a544e36fbd" (UID: "6509e943-70c6-444c-bc41-48a544e36fbd"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.785703 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fda69060-fa79-4696-b1a6-7980f124bf7c-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "fda69060-fa79-4696-b1a6-7980f124bf7c" (UID: "fda69060-fa79-4696-b1a6-7980f124bf7c"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.786211 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/43509403-f426-496e-be36-56cef71462f5-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "43509403-f426-496e-be36-56cef71462f5" (UID: "43509403-f426-496e-be36-56cef71462f5"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.786543 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/496e6271-fb68-4057-954e-a0d97a4afa3f-config" (OuterVolumeSpecName: "config") pod "496e6271-fb68-4057-954e-a0d97a4afa3f" (UID: "496e6271-fb68-4057-954e-a0d97a4afa3f"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.786621 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7583ce53-e0fe-4a16-9e4d-50516596a136-proxy-ca-bundles" (OuterVolumeSpecName: "proxy-ca-bundles") pod "7583ce53-e0fe-4a16-9e4d-50516596a136" (UID: "7583ce53-e0fe-4a16-9e4d-50516596a136"). InnerVolumeSpecName "proxy-ca-bundles". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.786953 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/31d8b7a1-420e-4252-a5b7-eebe8a111292-kube-api-access-zgdk5" (OuterVolumeSpecName: "kube-api-access-zgdk5") pod "31d8b7a1-420e-4252-a5b7-eebe8a111292" (UID: "31d8b7a1-420e-4252-a5b7-eebe8a111292"). InnerVolumeSpecName "kube-api-access-zgdk5". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.786982 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/01ab3dd5-8196-46d0-ad33-122e2ca51def-config" (OuterVolumeSpecName: "config") pod "01ab3dd5-8196-46d0-ad33-122e2ca51def" (UID: "01ab3dd5-8196-46d0-ad33-122e2ca51def"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.787076 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/49ef4625-1d3a-4a9f-b595-c2433d32326d-kube-api-access-pjr6v" (OuterVolumeSpecName: "kube-api-access-pjr6v") pod "49ef4625-1d3a-4a9f-b595-c2433d32326d" (UID: "49ef4625-1d3a-4a9f-b595-c2433d32326d"). InnerVolumeSpecName "kube-api-access-pjr6v". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.787276 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3ab1a177-2de0-46d9-b765-d0d0649bb42e-package-server-manager-serving-cert" (OuterVolumeSpecName: "package-server-manager-serving-cert") pod "3ab1a177-2de0-46d9-b765-d0d0649bb42e" (UID: "3ab1a177-2de0-46d9-b765-d0d0649bb42e"). InnerVolumeSpecName "package-server-manager-serving-cert". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.787290 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/09efc573-dbb6-4249-bd59-9b87aba8dd28-serving-cert" (OuterVolumeSpecName: "serving-cert") pod "09efc573-dbb6-4249-bd59-9b87aba8dd28" (UID: "09efc573-dbb6-4249-bd59-9b87aba8dd28"). InnerVolumeSpecName "serving-cert". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.787316 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/01ab3dd5-8196-46d0-ad33-122e2ca51def-serving-cert" (OuterVolumeSpecName: "serving-cert") pod "01ab3dd5-8196-46d0-ad33-122e2ca51def" (UID: "01ab3dd5-8196-46d0-ad33-122e2ca51def"). InnerVolumeSpecName "serving-cert". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.787779 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6402fda4-df10-493c-b4e5-d0569419652d-config" (OuterVolumeSpecName: "config") pod "6402fda4-df10-493c-b4e5-d0569419652d" (UID: "6402fda4-df10-493c-b4e5-d0569419652d"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.787850 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/43509403-f426-496e-be36-56cef71462f5-console-config" (OuterVolumeSpecName: "console-config") pod "43509403-f426-496e-be36-56cef71462f5" (UID: "43509403-f426-496e-be36-56cef71462f5"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.787882 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/bc569bba-c020-40d6-af68-e617625fe0e4-ovn-node-metrics-cert\") pod \"ovnkube-node-kg55h\" (UID: \"bc569bba-c020-40d6-af68-e617625fe0e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-kg55h" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.787939 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d75a4c96-2883-4a0b-bab2-0fab2b6c0b49-host-slash\") pod \"iptables-alerter-4ln5h\" (UID: \"d75a4c96-2883-4a0b-bab2-0fab2b6c0b49\") " pod="openshift-network-operator/iptables-alerter-4ln5h" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.788114 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bc5039c0-ea34-426b-a2b7-fbbc87b49a6d-kube-api-access-mnrrd" (OuterVolumeSpecName: "kube-api-access-mnrrd") pod "bc5039c0-ea34-426b-a2b7-fbbc87b49a6d" (UID: "bc5039c0-ea34-426b-a2b7-fbbc87b49a6d"). InnerVolumeSpecName "kube-api-access-mnrrd". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.788148 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-s2dwl\" (UniqueName: \"kubernetes.io/projected/9d751cbb-f2e2-430d-9754-c882a5e924a5-kube-api-access-s2dwl\") pod \"network-check-source-55646444c4-trplf\" (UID: \"9d751cbb-f2e2-430d-9754-c882a5e924a5\") " pod="openshift-network-diagnostics/network-check-source-55646444c4-trplf" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.788258 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/17265a68-bcb5-455c-b7df-c148a6517db9-os-release\") pod \"multus-additional-cni-plugins-2lrql\" (UID: \"17265a68-bcb5-455c-b7df-c148a6517db9\") " pod="openshift-multus/multus-additional-cni-plugins-2lrql" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.788358 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/c30c02db-f4e7-4432-941c-5448fd71f3d9-host-run-multus-certs\") pod \"multus-hx657\" (UID: \"c30c02db-f4e7-4432-941c-5448fd71f3d9\") " pod="openshift-multus/multus-hx657" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.788389 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fhsjl\" (UniqueName: \"kubernetes.io/projected/c30c02db-f4e7-4432-941c-5448fd71f3d9-kube-api-access-fhsjl\") pod \"multus-hx657\" (UID: \"c30c02db-f4e7-4432-941c-5448fd71f3d9\") " pod="openshift-multus/multus-hx657" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.788423 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x92h2\" (UniqueName: \"kubernetes.io/projected/bc569bba-c020-40d6-af68-e617625fe0e4-kube-api-access-x92h2\") pod \"ovnkube-node-kg55h\" (UID: \"bc569bba-c020-40d6-af68-e617625fe0e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-kg55h" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.788455 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/c30c02db-f4e7-4432-941c-5448fd71f3d9-host-var-lib-cni-multus\") pod \"multus-hx657\" (UID: \"c30c02db-f4e7-4432-941c-5448fd71f3d9\") " pod="openshift-multus/multus-hx657" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.788487 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c30c02db-f4e7-4432-941c-5448fd71f3d9-multus-cni-dir\") pod \"multus-hx657\" (UID: \"c30c02db-f4e7-4432-941c-5448fd71f3d9\") " pod="openshift-multus/multus-hx657" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.788548 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9vkjv\" (UniqueName: \"kubernetes.io/projected/10c237bc-72e7-4da3-9f2c-b1688395bf6e-kube-api-access-9vkjv\") pod \"machine-config-daemon-74bj6\" (UID: \"10c237bc-72e7-4da3-9f2c-b1688395bf6e\") " pod="openshift-machine-config-operator/machine-config-daemon-74bj6" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.788589 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bc569bba-c020-40d6-af68-e617625fe0e4-var-lib-openvswitch\") pod \"ovnkube-node-kg55h\" (UID: \"bc569bba-c020-40d6-af68-e617625fe0e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-kg55h" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.788635 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/bc569bba-c020-40d6-af68-e617625fe0e4-env-overrides\") pod \"ovnkube-node-kg55h\" (UID: \"bc569bba-c020-40d6-af68-e617625fe0e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-kg55h" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.788672 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/17265a68-bcb5-455c-b7df-c148a6517db9-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-2lrql\" (UID: \"17265a68-bcb5-455c-b7df-c148a6517db9\") " pod="openshift-multus/multus-additional-cni-plugins-2lrql" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.788715 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-etc-kube\" (UniqueName: \"kubernetes.io/host-path/37a5e44f-9a88-4405-be8a-b645485e7312-host-etc-kube\") pod \"network-operator-58b4c7f79c-55gtf\" (UID: \"37a5e44f-9a88-4405-be8a-b645485e7312\") " pod="openshift-network-operator/network-operator-58b4c7f79c-55gtf" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.788778 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/37a5e44f-9a88-4405-be8a-b645485e7312-metrics-tls\") pod \"network-operator-58b4c7f79c-55gtf\" (UID: \"37a5e44f-9a88-4405-be8a-b645485e7312\") " pod="openshift-network-operator/network-operator-58b4c7f79c-55gtf" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.788809 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/17265a68-bcb5-455c-b7df-c148a6517db9-cnibin\") pod \"multus-additional-cni-plugins-2lrql\" (UID: \"17265a68-bcb5-455c-b7df-c148a6517db9\") " pod="openshift-multus/multus-additional-cni-plugins-2lrql" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.788836 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c30c02db-f4e7-4432-941c-5448fd71f3d9-etc-kubernetes\") pod \"multus-hx657\" (UID: \"c30c02db-f4e7-4432-941c-5448fd71f3d9\") " pod="openshift-multus/multus-hx657" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.788901 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c30c02db-f4e7-4432-941c-5448fd71f3d9-system-cni-dir\") pod \"multus-hx657\" (UID: \"c30c02db-f4e7-4432-941c-5448fd71f3d9\") " pod="openshift-multus/multus-hx657" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.788933 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c30c02db-f4e7-4432-941c-5448fd71f3d9-multus-conf-dir\") pod \"multus-hx657\" (UID: \"c30c02db-f4e7-4432-941c-5448fd71f3d9\") " pod="openshift-multus/multus-hx657" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.788943 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6402fda4-df10-493c-b4e5-d0569419652d-images" (OuterVolumeSpecName: "images") pod "6402fda4-df10-493c-b4e5-d0569419652d" (UID: "6402fda4-df10-493c-b4e5-d0569419652d"). InnerVolumeSpecName "images". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.788965 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bc569bba-c020-40d6-af68-e617625fe0e4-host-slash\") pod \"ovnkube-node-kg55h\" (UID: \"bc569bba-c020-40d6-af68-e617625fe0e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-kg55h" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.789185 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5fe579f8-e8a6-4643-bce5-a661393c4dde-node-bootstrap-token" (OuterVolumeSpecName: "node-bootstrap-token") pod "5fe579f8-e8a6-4643-bce5-a661393c4dde" (UID: "5fe579f8-e8a6-4643-bce5-a661393c4dde"). InnerVolumeSpecName "node-bootstrap-token". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.789303 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/bc569bba-c020-40d6-af68-e617625fe0e4-ovnkube-config\") pod \"ovnkube-node-kg55h\" (UID: \"bc569bba-c020-40d6-af68-e617625fe0e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-kg55h" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.789414 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"mcd-auth-proxy-config\" (UniqueName: \"kubernetes.io/configmap/10c237bc-72e7-4da3-9f2c-b1688395bf6e-mcd-auth-proxy-config\") pod \"machine-config-daemon-74bj6\" (UID: \"10c237bc-72e7-4da3-9f2c-b1688395bf6e\") " pod="openshift-machine-config-operator/machine-config-daemon-74bj6" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.789682 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1386a44e-36a2-460c-96d0-0359d2b6f0f5-config\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.789768 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-6g6sz\" (UniqueName: \"kubernetes.io/projected/6509e943-70c6-444c-bc41-48a544e36fbd-kube-api-access-6g6sz\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.789857 5052 reconciler_common.go:293] "Volume detached for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/25e176fe-21b4-4974-b1ed-c8b94f112a7f-signing-key\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.789934 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5441d097-087c-4d9a-baa8-b210afa90fc9-config\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.789999 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-s4n52\" (UniqueName: \"kubernetes.io/projected/925f1c65-6136-48ba-85aa-3a3b50560753-kube-api-access-s4n52\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.790064 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-cfbct\" (UniqueName: \"kubernetes.io/projected/57a731c4-ef35-47a8-b875-bfb08a7f8011-kube-api-access-cfbct\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.790129 5052 reconciler_common.go:293] "Volume detached for volume \"control-plane-machine-set-operator-tls\" (UniqueName: \"kubernetes.io/secret/6731426b-95fe-49ff-bb5f-40441049fde2-control-plane-machine-set-operator-tls\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.790194 5052 reconciler_common.go:293] "Volume detached for volume \"certs\" (UniqueName: \"kubernetes.io/secret/5fe579f8-e8a6-4643-bce5-a661393c4dde-certs\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.790268 5052 reconciler_common.go:293] "Volume detached for volume \"profile-collector-cert\" (UniqueName: \"kubernetes.io/secret/b6312bbd-5731-4ea0-a20f-81d5a57df44a-profile-collector-cert\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.790340 5052 reconciler_common.go:293] "Volume detached for volume \"auth-proxy-config\" (UniqueName: \"kubernetes.io/configmap/22c825df-677d-4ca6-82db-3454ed06e783-auth-proxy-config\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.790414 5052 reconciler_common.go:293] "Volume detached for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/c03ee662-fb2f-4fc4-a2c1-af487c19d254-stats-auth\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.789640 5052 status_manager.go:875] "Failed to update status for pod" pod="openshift-network-operator/iptables-alerter-4ln5h" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"d75a4c96-2883-4a0b-bab2-0fab2b6c0b49\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"message\\\":\\\"containers with unready status: [iptables-alerter]\\\",\\\"reason\\\":\\\"ContainersNotReady\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"message\\\":\\\"containers with unready status: [iptables-alerter]\\\",\\\"reason\\\":\\\"ContainersNotReady\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"ContainersReady\\\"}],\\\"containerStatuses\\\":[{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:35512335ac39aed0f55b7f799f416f4f6445c20c1b19888cf2bb72bb276703f2\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{\\\"terminated\\\":{\\\"exitCode\\\":137,\\\"finishedAt\\\":null,\\\"message\\\":\\\"The container could not be located when the pod was deleted. The container used to be Running\\\",\\\"reason\\\":\\\"ContainerStatusUnknown\\\",\\\"startedAt\\\":null}},\\\"name\\\":\\\"iptables-alerter\\\",\\\"ready\\\":false,\\\"restartCount\\\":3,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"ContainerCreating\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/iptables-alerter\\\",\\\"name\\\":\\\"iptables-alerter-script\\\"},{\\\"mountPath\\\":\\\"/host\\\",\\\"name\\\":\\\"host-slash\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-rczfb\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}]}}\" for pod \"openshift-network-operator\"/\"iptables-alerter-4ln5h\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://127.0.0.1:9743/pod?timeout=10s\": dial tcp 127.0.0.1:9743: connect: connection refused" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.790879 5052 reconciler_common.go:293] "Volume detached for volume \"etcd-serving-ca\" (UniqueName: \"kubernetes.io/configmap/1bf7eb37-55a3-4c65-b768-a94c82151e69-etcd-serving-ca\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.790988 5052 reconciler_common.go:293] "Volume detached for volume \"client-ca\" (UniqueName: \"kubernetes.io/configmap/7583ce53-e0fe-4a16-9e4d-50516596a136-client-ca\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.791082 5052 reconciler_common.go:293] "Volume detached for volume \"cert\" (UniqueName: \"kubernetes.io/secret/20b0d48f-5fd6-431c-a545-e3c800c7b866-cert\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.791160 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-d6qdx\" (UniqueName: \"kubernetes.io/projected/87cf06ed-a83f-41a7-828d-70653580a8cb-kube-api-access-d6qdx\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.791239 5052 reconciler_common.go:293] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/43509403-f426-496e-be36-56cef71462f5-oauth-serving-cert\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.791314 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e7e6199b-1264-4501-8953-767f51328d08-config\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.791399 5052 reconciler_common.go:293] "Volume detached for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/3cb93b32-e0ae-4377-b9c8-fdb9842c6d59-serviceca\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.791477 5052 reconciler_common.go:293] "Volume detached for volume \"srv-cert\" (UniqueName: \"kubernetes.io/secret/b6312bbd-5731-4ea0-a20f-81d5a57df44a-srv-cert\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.791575 5052 reconciler_common.go:293] "Volume detached for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/496e6271-fb68-4057-954e-a0d97a4afa3f-serving-cert\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.791663 5052 reconciler_common.go:293] "Volume detached for volume \"auth-proxy-config\" (UniqueName: \"kubernetes.io/configmap/31d8b7a1-420e-4252-a5b7-eebe8a111292-auth-proxy-config\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.791748 5052 reconciler_common.go:293] "Volume detached for volume \"marketplace-operator-metrics\" (UniqueName: \"kubernetes.io/secret/b6cd30de-2eeb-49a2-ab40-9167f4560ff5-marketplace-operator-metrics\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.791825 5052 reconciler_common.go:293] "Volume detached for volume \"v4-0-config-system-ocp-branding-template\" (UniqueName: \"kubernetes.io/secret/49c341d1-5089-4bc2-86a0-a5e165cfcc6b-v4-0-config-system-ocp-branding-template\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.793392 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-qs4fp\" (UniqueName: \"kubernetes.io/projected/210d8245-ebfc-4e3b-ac4a-e21ce76f9a7c-kube-api-access-qs4fp\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.793495 5052 reconciler_common.go:293] "Volume detached for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/308be0ea-9f5f-4b29-aeb1-5abd31a0b17b-apiservice-cert\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.793710 5052 reconciler_common.go:293] "Volume detached for volume \"audit-policies\" (UniqueName: \"kubernetes.io/configmap/49c341d1-5089-4bc2-86a0-a5e165cfcc6b-audit-policies\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.793780 5052 reconciler_common.go:293] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8f668bae-612b-4b75-9490-919e737c6a3b-installation-pull-secrets\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.793845 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-ngvvp\" (UniqueName: \"kubernetes.io/projected/49c341d1-5089-4bc2-86a0-a5e165cfcc6b-kube-api-access-ngvvp\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.793904 5052 reconciler_common.go:293] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5b88f790-22fa-440e-b583-365168c0b23d-metrics-certs\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.793961 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-x7zkh\" (UniqueName: \"kubernetes.io/projected/6731426b-95fe-49ff-bb5f-40441049fde2-kube-api-access-x7zkh\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.794013 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-w9rds\" (UniqueName: \"kubernetes.io/projected/20b0d48f-5fd6-431c-a545-e3c800c7b866-kube-api-access-w9rds\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.794069 5052 reconciler_common.go:293] "Volume detached for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7539238d-5fe0-46ed-884e-1c3b566537ec-serving-cert\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.794141 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-dbsvg\" (UniqueName: \"kubernetes.io/projected/f88749ec-7931-4ee7-b3fc-1ec5e11f92e9-kube-api-access-dbsvg\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.794219 5052 reconciler_common.go:293] "Volume detached for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5441d097-087c-4d9a-baa8-b210afa90fc9-serving-cert\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.794287 5052 reconciler_common.go:293] "Volume detached for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a0128f3a-b052-44ed-a84e-c4c8aaf17c13-samples-operator-tls\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.794348 5052 reconciler_common.go:293] "Volume detached for volume \"v4-0-config-user-template-provider-selection\" (UniqueName: \"kubernetes.io/secret/49c341d1-5089-4bc2-86a0-a5e165cfcc6b-v4-0-config-user-template-provider-selection\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.794431 5052 reconciler_common.go:293] "Volume detached for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/6ea678ab-3438-413e-bfe3-290ae7725660-ovnkube-script-lib\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.794530 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-jhbk2\" (UniqueName: \"kubernetes.io/projected/bd23aa5c-e532-4e53-bccf-e79f130c5ae8-kube-api-access-jhbk2\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.794617 5052 reconciler_common.go:293] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/308be0ea-9f5f-4b29-aeb1-5abd31a0b17b-webhook-cert\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.794681 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-xcphl\" (UniqueName: \"kubernetes.io/projected/7583ce53-e0fe-4a16-9e4d-50516596a136-kube-api-access-xcphl\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.795589 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-bf2bz\" (UniqueName: \"kubernetes.io/projected/1d611f23-29be-4491-8495-bee1670e935f-kube-api-access-bf2bz\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.795683 5052 reconciler_common.go:293] "Volume detached for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e7e6199b-1264-4501-8953-767f51328d08-serving-cert\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.795960 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8cea82b4-6893-4ddc-af9f-1bb5ae425c5b-config\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.796057 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/6402fda4-df10-493c-b4e5-d0569419652d-config\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.796143 5052 reconciler_common.go:293] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/43509403-f426-496e-be36-56cef71462f5-console-serving-cert\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.796724 5052 reconciler_common.go:293] "Volume detached for volume \"v4-0-config-user-idp-0-file-data\" (UniqueName: \"kubernetes.io/secret/49c341d1-5089-4bc2-86a0-a5e165cfcc6b-v4-0-config-user-idp-0-file-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.796929 5052 reconciler_common.go:293] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8f668bae-612b-4b75-9490-919e737c6a3b-registry-tls\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.797038 5052 reconciler_common.go:293] "Volume detached for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d-utilities\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.797132 5052 reconciler_common.go:293] "Volume detached for volume \"machine-api-operator-tls\" (UniqueName: \"kubernetes.io/secret/6402fda4-df10-493c-b4e5-d0569419652d-machine-api-operator-tls\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.797226 5052 reconciler_common.go:293] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a31745f5-9847-4afe-82a5-3161cc66ca93-trusted-ca\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.792942 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/37a5e44f-9a88-4405-be8a-b645485e7312-metrics-tls\") pod \"network-operator-58b4c7f79c-55gtf\" (UID: \"37a5e44f-9a88-4405-be8a-b645485e7312\") " pod="openshift-network-operator/network-operator-58b4c7f79c-55gtf" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.792957 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/57a731c4-ef35-47a8-b875-bfb08a7f8011-catalog-content" (OuterVolumeSpecName: "catalog-content") pod "57a731c4-ef35-47a8-b875-bfb08a7f8011" (UID: "57a731c4-ef35-47a8-b875-bfb08a7f8011"). InnerVolumeSpecName "catalog-content". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.795495 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1d611f23-29be-4491-8495-bee1670e935f-catalog-content" (OuterVolumeSpecName: "catalog-content") pod "1d611f23-29be-4491-8495-bee1670e935f" (UID: "1d611f23-29be-4491-8495-bee1670e935f"). InnerVolumeSpecName "catalog-content". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.797303 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-fcqwp\" (UniqueName: \"kubernetes.io/projected/5fe579f8-e8a6-4643-bce5-a661393c4dde-kube-api-access-fcqwp\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.796075 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ef543e1b-8068-4ea3-b32a-61027b32e95d-webhook-cert\") pod \"network-node-identity-vrzqb\" (UID: \"ef543e1b-8068-4ea3-b32a-61027b32e95d\") " pod="openshift-network-node-identity/network-node-identity-vrzqb" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.797607 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-279lb\" (UniqueName: \"kubernetes.io/projected/7bb08738-c794-4ee8-9972-3a62ca171029-kube-api-access-279lb\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.797629 5052 reconciler_common.go:293] "Volume detached for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/96b93a3a-6083-4aea-8eab-fe1aa8245ad9-metrics-tls\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.797669 5052 reconciler_common.go:293] "Volume detached for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6509e943-70c6-444c-bc41-48a544e36fbd-service-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.797687 5052 reconciler_common.go:293] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/43509403-f426-496e-be36-56cef71462f5-service-ca\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.797702 5052 reconciler_common.go:293] "Volume detached for volume \"encryption-config\" (UniqueName: \"kubernetes.io/secret/1bf7eb37-55a3-4c65-b768-a94c82151e69-encryption-config\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.797716 5052 reconciler_common.go:293] "Volume detached for volume \"machine-approver-tls\" (UniqueName: \"kubernetes.io/secret/22c825df-677d-4ca6-82db-3454ed06e783-machine-approver-tls\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.797732 5052 reconciler_common.go:293] "Volume detached for volume \"srv-cert\" (UniqueName: \"kubernetes.io/secret/f88749ec-7931-4ee7-b3fc-1ec5e11f92e9-srv-cert\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.797748 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-w4xd4\" (UniqueName: \"kubernetes.io/projected/8cea82b4-6893-4ddc-af9f-1bb5ae425c5b-kube-api-access-w4xd4\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.797764 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-lzf88\" (UniqueName: \"kubernetes.io/projected/0b574797-001e-440a-8f4e-c0be86edad0f-kube-api-access-lzf88\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.797786 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-249nr\" (UniqueName: \"kubernetes.io/projected/b6312bbd-5731-4ea0-a20f-81d5a57df44a-kube-api-access-249nr\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.797801 5052 reconciler_common.go:293] "Volume detached for volume \"encryption-config\" (UniqueName: \"kubernetes.io/secret/09ae3b1a-e8e7-4524-b54b-61eab6f9239a-encryption-config\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.797818 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-vt5rc\" (UniqueName: \"kubernetes.io/projected/44663579-783b-4372-86d6-acf235a62d72-kube-api-access-vt5rc\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.797832 5052 reconciler_common.go:293] "Volume detached for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/4bb40260-dbaa-4fb0-84df-5e680505d512-multus-daemon-config\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.797846 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/09efc573-dbb6-4249-bd59-9b87aba8dd28-config\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.797860 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/7583ce53-e0fe-4a16-9e4d-50516596a136-config\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.797873 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/210d8245-ebfc-4e3b-ac4a-e21ce76f9a7c-config\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.797886 5052 reconciler_common.go:293] "Volume detached for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/1d611f23-29be-4491-8495-bee1670e935f-utilities\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.797899 5052 reconciler_common.go:293] "Volume detached for volume \"etcd-ca\" (UniqueName: \"kubernetes.io/configmap/09efc573-dbb6-4249-bd59-9b87aba8dd28-etcd-ca\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.797913 5052 reconciler_common.go:293] "Volume detached for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/6ea678ab-3438-413e-bfe3-290ae7725660-ovn-node-metrics-cert\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.797926 5052 reconciler_common.go:293] "Volume detached for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/6ea678ab-3438-413e-bfe3-290ae7725660-env-overrides\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.797940 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-x4zgh\" (UniqueName: \"kubernetes.io/projected/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d-kube-api-access-x4zgh\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.797955 5052 reconciler_common.go:293] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/fda69060-fa79-4696-b1a6-7980f124bf7c-proxy-tls\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.797969 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/496e6271-fb68-4057-954e-a0d97a4afa3f-config\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.797100 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-rdwmf\" (UniqueName: \"kubernetes.io/projected/37a5e44f-9a88-4405-be8a-b645485e7312-kube-api-access-rdwmf\") pod \"network-operator-58b4c7f79c-55gtf\" (UID: \"37a5e44f-9a88-4405-be8a-b645485e7312\") " pod="openshift-network-operator/network-operator-58b4c7f79c-55gtf" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.799134 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-s2kz5\" (UniqueName: \"kubernetes.io/projected/ef543e1b-8068-4ea3-b32a-61027b32e95d-kube-api-access-s2kz5\") pod \"network-node-identity-vrzqb\" (UID: \"ef543e1b-8068-4ea3-b32a-61027b32e95d\") " pod="openshift-network-node-identity/network-node-identity-vrzqb" Dec 03 11:20:31 crc kubenswrapper[5052]: E1203 11:20:31.800485 5052 projected.go:288] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Dec 03 11:20:31 crc kubenswrapper[5052]: E1203 11:20:31.800533 5052 projected.go:288] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Dec 03 11:20:31 crc kubenswrapper[5052]: E1203 11:20:31.800553 5052 projected.go:194] Error preparing data for projected volume kube-api-access-s2dwl for pod openshift-network-diagnostics/network-check-source-55646444c4-trplf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Dec 03 11:20:31 crc kubenswrapper[5052]: E1203 11:20:31.800636 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9d751cbb-f2e2-430d-9754-c882a5e924a5-kube-api-access-s2dwl podName:9d751cbb-f2e2-430d-9754-c882a5e924a5 nodeName:}" failed. No retries permitted until 2025-12-03 11:20:32.300610132 +0000 UTC m=+22.888852086 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-s2dwl" (UniqueName: "kubernetes.io/projected/9d751cbb-f2e2-430d-9754-c882a5e924a5-kube-api-access-s2dwl") pod "network-check-source-55646444c4-trplf" (UID: "9d751cbb-f2e2-430d-9754-c882a5e924a5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.797985 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/6509e943-70c6-444c-bc41-48a544e36fbd-config\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.800967 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-zgdk5\" (UniqueName: \"kubernetes.io/projected/31d8b7a1-420e-4252-a5b7-eebe8a111292-kube-api-access-zgdk5\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.800986 5052 reconciler_common.go:293] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/43509403-f426-496e-be36-56cef71462f5-trusted-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.801001 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access\" (UniqueName: \"kubernetes.io/projected/e7e6199b-1264-4501-8953-767f51328d08-kube-api-access\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.801017 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-gf66m\" (UniqueName: \"kubernetes.io/projected/a0128f3a-b052-44ed-a84e-c4c8aaf17c13-kube-api-access-gf66m\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.801034 5052 reconciler_common.go:293] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/31d8b7a1-420e-4252-a5b7-eebe8a111292-proxy-tls\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.801050 5052 reconciler_common.go:293] "Volume detached for volume \"proxy-ca-bundles\" (UniqueName: \"kubernetes.io/configmap/7583ce53-e0fe-4a16-9e4d-50516596a136-proxy-ca-bundles\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.801063 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access\" (UniqueName: \"kubernetes.io/projected/496e6271-fb68-4057-954e-a0d97a4afa3f-kube-api-access\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.801077 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-jkwtn\" (UniqueName: \"kubernetes.io/projected/5b88f790-22fa-440e-b583-365168c0b23d-kube-api-access-jkwtn\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.801091 5052 reconciler_common.go:293] "Volume detached for volume \"images\" (UniqueName: \"kubernetes.io/configmap/31d8b7a1-420e-4252-a5b7-eebe8a111292-images\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.801105 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-2w9zh\" (UniqueName: \"kubernetes.io/projected/4bb40260-dbaa-4fb0-84df-5e680505d512-kube-api-access-2w9zh\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.801121 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-pjr6v\" (UniqueName: \"kubernetes.io/projected/49ef4625-1d3a-4a9f-b595-c2433d32326d-kube-api-access-pjr6v\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.801138 5052 reconciler_common.go:293] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/09ae3b1a-e8e7-4524-b54b-61eab6f9239a-trusted-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.801154 5052 reconciler_common.go:293] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bf126b07-da06-4140-9a57-dfd54fc6b486-bound-sa-token\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.801167 5052 reconciler_common.go:293] "Volume detached for volume \"etcd-client\" (UniqueName: \"kubernetes.io/secret/09ae3b1a-e8e7-4524-b54b-61eab6f9239a-etcd-client\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.801184 5052 reconciler_common.go:293] "Volume detached for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/09efc573-dbb6-4249-bd59-9b87aba8dd28-serving-cert\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.801197 5052 reconciler_common.go:293] "Volume detached for volume \"images\" (UniqueName: \"kubernetes.io/configmap/6402fda4-df10-493c-b4e5-d0569419652d-images\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.801210 5052 reconciler_common.go:293] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9d4552c7-cd75-42dd-8880-30dd377c49a4-trusted-ca\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.801222 5052 reconciler_common.go:293] "Volume detached for volume \"audit-policies\" (UniqueName: \"kubernetes.io/configmap/09ae3b1a-e8e7-4524-b54b-61eab6f9239a-audit-policies\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.801236 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-rnphk\" (UniqueName: \"kubernetes.io/projected/bf126b07-da06-4140-9a57-dfd54fc6b486-kube-api-access-rnphk\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.801252 5052 reconciler_common.go:293] "Volume detached for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/01ab3dd5-8196-46d0-ad33-122e2ca51def-serving-cert\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.801266 5052 reconciler_common.go:293] "Volume detached for volume \"package-server-manager-serving-cert\" (UniqueName: \"kubernetes.io/secret/3ab1a177-2de0-46d9-b765-d0d0649bb42e-package-server-manager-serving-cert\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.801280 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/01ab3dd5-8196-46d0-ad33-122e2ca51def-config\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.801295 5052 reconciler_common.go:293] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/43509403-f426-496e-be36-56cef71462f5-console-config\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.807498 5052 reconciler_common.go:293] "Volume detached for volume \"node-bootstrap-token\" (UniqueName: \"kubernetes.io/secret/5fe579f8-e8a6-4643-bce5-a661393c4dde-node-bootstrap-token\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.807548 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-mnrrd\" (UniqueName: \"kubernetes.io/projected/bc5039c0-ea34-426b-a2b7-fbbc87b49a6d-kube-api-access-mnrrd\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.807566 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-w7l8j\" (UniqueName: \"kubernetes.io/projected/01ab3dd5-8196-46d0-ad33-122e2ca51def-kube-api-access-w7l8j\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.807580 5052 reconciler_common.go:293] "Volume detached for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0b78653f-4ff9-4508-8672-245ed9b561e3-serving-cert\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.807597 5052 reconciler_common.go:293] "Volume detached for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/7bb08738-c794-4ee8-9972-3a62ca171029-cni-sysctl-allowlist\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.807612 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-tk88c\" (UniqueName: \"kubernetes.io/projected/7539238d-5fe0-46ed-884e-1c3b566537ec-kube-api-access-tk88c\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.807629 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-sb6h7\" (UniqueName: \"kubernetes.io/projected/1bf7eb37-55a3-4c65-b768-a94c82151e69-kube-api-access-sb6h7\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.807643 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-4d4hj\" (UniqueName: \"kubernetes.io/projected/3ab1a177-2de0-46d9-b765-d0d0649bb42e-kube-api-access-4d4hj\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.807658 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/9d4552c7-cd75-42dd-8880-30dd377c49a4-config\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.807672 5052 reconciler_common.go:293] "Volume detached for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9d4552c7-cd75-42dd-8880-30dd377c49a4-serving-cert\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.807687 5052 reconciler_common.go:293] "Volume detached for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/925f1c65-6136-48ba-85aa-3a3b50560753-ovnkube-config\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.807700 5052 reconciler_common.go:293] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6509e943-70c6-444c-bc41-48a544e36fbd-trusted-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.807715 5052 reconciler_common.go:293] "Volume detached for volume \"ovn-control-plane-metrics-cert\" (UniqueName: \"kubernetes.io/secret/925f1c65-6136-48ba-85aa-3a3b50560753-ovn-control-plane-metrics-cert\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.807730 5052 reconciler_common.go:293] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/87cf06ed-a83f-41a7-828d-70653580a8cb-config-volume\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.807744 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-8tdtz\" (UniqueName: \"kubernetes.io/projected/09efc573-dbb6-4249-bd59-9b87aba8dd28-kube-api-access-8tdtz\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.807764 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-x2m85\" (UniqueName: \"kubernetes.io/projected/cd70aa09-68dd-4d64-bd6f-156fe6d1dc6d-kube-api-access-x2m85\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.807776 5052 reconciler_common.go:293] "Volume detached for volume \"marketplace-trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b6cd30de-2eeb-49a2-ab40-9167f4560ff5-marketplace-trusted-ca\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.807789 5052 reconciler_common.go:293] "Volume detached for volume \"v4-0-config-system-session\" (UniqueName: \"kubernetes.io/secret/49c341d1-5089-4bc2-86a0-a5e165cfcc6b-v4-0-config-system-session\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.807802 5052 reconciler_common.go:293] "Volume detached for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8cea82b4-6893-4ddc-af9f-1bb5ae425c5b-serving-cert\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.807814 5052 reconciler_common.go:293] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/43509403-f426-496e-be36-56cef71462f5-console-oauth-config\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.807827 5052 reconciler_common.go:293] "Volume detached for volume \"v4-0-config-system-serving-cert\" (UniqueName: \"kubernetes.io/secret/49c341d1-5089-4bc2-86a0-a5e165cfcc6b-v4-0-config-system-serving-cert\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.807842 5052 reconciler_common.go:293] "Volume detached for volume \"v4-0-config-user-template-error\" (UniqueName: \"kubernetes.io/secret/49c341d1-5089-4bc2-86a0-a5e165cfcc6b-v4-0-config-user-template-error\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.807856 5052 reconciler_common.go:293] "Volume detached for volume \"image-import-ca\" (UniqueName: \"kubernetes.io/configmap/1bf7eb37-55a3-4c65-b768-a94c82151e69-image-import-ca\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.807870 5052 reconciler_common.go:293] "Volume detached for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/210d8245-ebfc-4e3b-ac4a-e21ce76f9a7c-serving-cert\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.807886 5052 reconciler_common.go:293] "Volume detached for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/bc5039c0-ea34-426b-a2b7-fbbc87b49a6d-serving-cert\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.807934 5052 reconciler_common.go:293] "Volume detached for volume \"mcc-auth-proxy-config\" (UniqueName: \"kubernetes.io/configmap/0b574797-001e-440a-8f4e-c0be86edad0f-mcc-auth-proxy-config\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.807951 5052 reconciler_common.go:293] "Volume detached for volume \"available-featuregates\" (UniqueName: \"kubernetes.io/empty-dir/bc5039c0-ea34-426b-a2b7-fbbc87b49a6d-available-featuregates\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.807964 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-wxkg8\" (UniqueName: \"kubernetes.io/projected/3cb93b32-e0ae-4377-b9c8-fdb9842c6d59-kube-api-access-wxkg8\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.807979 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-xcgwh\" (UniqueName: \"kubernetes.io/projected/fda69060-fa79-4696-b1a6-7980f124bf7c-kube-api-access-xcgwh\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.807992 5052 reconciler_common.go:293] "Volume detached for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7583ce53-e0fe-4a16-9e4d-50516596a136-serving-cert\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.808004 5052 reconciler_common.go:293] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/0b574797-001e-440a-8f4e-c0be86edad0f-proxy-tls\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.808018 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-fqsjt\" (UniqueName: \"kubernetes.io/projected/efdd0498-1daa-4136-9a4a-3b948c2293fc-kube-api-access-fqsjt\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.808030 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access\" (UniqueName: \"kubernetes.io/projected/0b78653f-4ff9-4508-8672-245ed9b561e3-kube-api-access\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.808043 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-2d4wz\" (UniqueName: \"kubernetes.io/projected/5441d097-087c-4d9a-baa8-b210afa90fc9-kube-api-access-2d4wz\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.808059 5052 reconciler_common.go:293] "Volume detached for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/87cf06ed-a83f-41a7-828d-70653580a8cb-metrics-tls\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.808071 5052 reconciler_common.go:293] "Volume detached for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/09ae3b1a-e8e7-4524-b54b-61eab6f9239a-serving-cert\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.808083 5052 reconciler_common.go:293] "Volume detached for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/c03ee662-fb2f-4fc4-a2c1-af487c19d254-default-certificate\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.808097 5052 reconciler_common.go:293] "Volume detached for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4bb40260-dbaa-4fb0-84df-5e680505d512-cni-binary-copy\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.802673 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/efdd0498-1daa-4136-9a4a-3b948c2293fc-webhook-certs" (OuterVolumeSpecName: "webhook-certs") pod "efdd0498-1daa-4136-9a4a-3b948c2293fc" (UID: "efdd0498-1daa-4136-9a4a-3b948c2293fc"). InnerVolumeSpecName "webhook-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.804759 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6402fda4-df10-493c-b4e5-d0569419652d-kube-api-access-mg5zb" (OuterVolumeSpecName: "kube-api-access-mg5zb") pod "6402fda4-df10-493c-b4e5-d0569419652d" (UID: "6402fda4-df10-493c-b4e5-d0569419652d"). InnerVolumeSpecName "kube-api-access-mg5zb". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.805909 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5225d0e4-402f-4861-b410-819f433b1803-utilities" (OuterVolumeSpecName: "utilities") pod "5225d0e4-402f-4861-b410-819f433b1803" (UID: "5225d0e4-402f-4861-b410-819f433b1803"). InnerVolumeSpecName "utilities". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.805613 5052 status_manager.go:875] "Failed to update status for pod" pod="openshift-multus/multus-hx657" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"c30c02db-f4e7-4432-941c-5448fd71f3d9\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"Initialized\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"message\\\":\\\"containers with unready status: [kube-multus]\\\",\\\"reason\\\":\\\"ContainersNotReady\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"message\\\":\\\"containers with unready status: [kube-multus]\\\",\\\"reason\\\":\\\"ContainersNotReady\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"ContainersReady\\\"}],\\\"containerStatuses\\\":[{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7eeaee65f2808b819eedb413bdcabb9144e12f0dd97f13fd1afba93a95b67b26\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"kube-multus\\\",\\\"ready\\\":false,\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"ContainerCreating\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/entrypoint\\\",\\\"name\\\":\\\"cni-binary-copy\\\"},{\\\"mountPath\\\":\\\"/host/etc/os-release\\\",\\\"name\\\":\\\"os-release\\\"},{\\\"mountPath\\\":\\\"/host/etc/cni/net.d\\\",\\\"name\\\":\\\"system-cni-dir\\\"},{\\\"mountPath\\\":\\\"/host/run/multus/cni/net.d\\\",\\\"name\\\":\\\"multus-cni-dir\\\"},{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/run/multus\\\",\\\"name\\\":\\\"multus-socket-dir-parent\\\"},{\\\"mountPath\\\":\\\"/run/k8s.cni.cncf.io\\\",\\\"name\\\":\\\"host-run-k8s-cni-cncf-io\\\"},{\\\"mountPath\\\":\\\"/run/netns\\\",\\\"name\\\":\\\"host-run-netns\\\"},{\\\"mountPath\\\":\\\"/var/lib/cni/bin\\\",\\\"name\\\":\\\"host-var-lib-cni-bin\\\"},{\\\"mountPath\\\":\\\"/var/lib/cni/multus\\\",\\\"name\\\":\\\"host-var-lib-cni-multus\\\"},{\\\"mountPath\\\":\\\"/var/lib/kubelet\\\",\\\"name\\\":\\\"host-var-lib-kubelet\\\"},{\\\"mountPath\\\":\\\"/hostroot\\\",\\\"name\\\":\\\"hostroot\\\"},{\\\"mountPath\\\":\\\"/etc/cni/multus/net.d\\\",\\\"name\\\":\\\"multus-conf-dir\\\"},{\\\"mountPath\\\":\\\"/etc/cni/net.d/multus.d\\\",\\\"name\\\":\\\"multus-daemon-config\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/etc/cni/multus/certs\\\",\\\"name\\\":\\\"host-run-multus-certs\\\"},{\\\"mountPath\\\":\\\"/etc/kubernetes\\\",\\\"name\\\":\\\"etc-kubernetes\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-fhsjl\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}],\\\"hostIP\\\":\\\"192.168.126.11\\\",\\\"hostIPs\\\":[{\\\"ip\\\":\\\"192.168.126.11\\\"}],\\\"podIP\\\":\\\"192.168.126.11\\\",\\\"podIPs\\\":[{\\\"ip\\\":\\\"192.168.126.11\\\"}],\\\"startTime\\\":\\\"2025-12-03T11:20:31Z\\\"}}\" for pod \"openshift-multus\"/\"multus-hx657\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://127.0.0.1:9743/pod?timeout=10s\": dial tcp 127.0.0.1:9743: connect: connection refused" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.808110 5052 reconciler_common.go:293] "Volume detached for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6509e943-70c6-444c-bc41-48a544e36fbd-serving-cert\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.808342 5052 reconciler_common.go:293] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a31745f5-9847-4afe-82a5-3161cc66ca93-bound-sa-token\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.806283 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-rczfb\" (UniqueName: \"kubernetes.io/projected/d75a4c96-2883-4a0b-bab2-0fab2b6c0b49-kube-api-access-rczfb\") pod \"iptables-alerter-4ln5h\" (UID: \"d75a4c96-2883-4a0b-bab2-0fab2b6c0b49\") " pod="openshift-network-operator/iptables-alerter-4ln5h" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.807369 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1386a44e-36a2-460c-96d0-0359d2b6f0f5-serving-cert" (OuterVolumeSpecName: "serving-cert") pod "1386a44e-36a2-460c-96d0-0359d2b6f0f5" (UID: "1386a44e-36a2-460c-96d0-0359d2b6f0f5"). InnerVolumeSpecName "serving-cert". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.808287 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/49c341d1-5089-4bc2-86a0-a5e165cfcc6b-v4-0-config-system-cliconfig" (OuterVolumeSpecName: "v4-0-config-system-cliconfig") pod "49c341d1-5089-4bc2-86a0-a5e165cfcc6b" (UID: "49c341d1-5089-4bc2-86a0-a5e165cfcc6b"). InnerVolumeSpecName "v4-0-config-system-cliconfig". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.808360 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-lz9wn\" (UniqueName: \"kubernetes.io/projected/a31745f5-9847-4afe-82a5-3161cc66ca93-kube-api-access-lz9wn\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.808472 5052 reconciler_common.go:293] "Volume detached for volume \"v4-0-config-system-service-ca\" (UniqueName: \"kubernetes.io/configmap/49c341d1-5089-4bc2-86a0-a5e165cfcc6b-v4-0-config-system-service-ca\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.808497 5052 reconciler_common.go:293] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8f668bae-612b-4b75-9490-919e737c6a3b-bound-sa-token\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.808534 5052 reconciler_common.go:293] "Volume detached for volume \"v4-0-config-system-router-certs\" (UniqueName: \"kubernetes.io/secret/49c341d1-5089-4bc2-86a0-a5e165cfcc6b-v4-0-config-system-router-certs\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.808552 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-pcxfs\" (UniqueName: \"kubernetes.io/projected/9d4552c7-cd75-42dd-8880-30dd377c49a4-kube-api-access-pcxfs\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.808566 5052 reconciler_common.go:293] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/bf126b07-da06-4140-9a57-dfd54fc6b486-trusted-ca\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.808580 5052 reconciler_common.go:293] "Volume detached for volume \"image-registry-operator-tls\" (UniqueName: \"kubernetes.io/secret/bf126b07-da06-4140-9a57-dfd54fc6b486-image-registry-operator-tls\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.808595 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/22c825df-677d-4ca6-82db-3454ed06e783-config\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.808607 5052 reconciler_common.go:293] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0b78653f-4ff9-4508-8672-245ed9b561e3-service-ca\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.808621 5052 reconciler_common.go:293] "Volume detached for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/6ea678ab-3438-413e-bfe3-290ae7725660-ovnkube-config\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.808636 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-6ccd8\" (UniqueName: \"kubernetes.io/projected/308be0ea-9f5f-4b29-aeb1-5abd31a0b17b-kube-api-access-6ccd8\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.808653 5052 reconciler_common.go:293] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c03ee662-fb2f-4fc4-a2c1-af487c19d254-metrics-certs\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.808666 5052 reconciler_common.go:293] "Volume detached for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/25e176fe-21b4-4974-b1ed-c8b94f112a7f-signing-cabundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.808679 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-kfwg7\" (UniqueName: \"kubernetes.io/projected/8f668bae-612b-4b75-9490-919e737c6a3b-kube-api-access-kfwg7\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.808697 5052 reconciler_common.go:293] "Volume detached for volume \"audit\" (UniqueName: \"kubernetes.io/configmap/1bf7eb37-55a3-4c65-b768-a94c82151e69-audit\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.808712 5052 reconciler_common.go:293] "Volume detached for volume \"etcd-service-ca\" (UniqueName: \"kubernetes.io/configmap/09efc573-dbb6-4249-bd59-9b87aba8dd28-etcd-service-ca\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.808725 5052 reconciler_common.go:293] "Volume detached for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/925f1c65-6136-48ba-85aa-3a3b50560753-env-overrides\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.808738 5052 reconciler_common.go:293] "Volume detached for volume \"mcd-auth-proxy-config\" (UniqueName: \"kubernetes.io/configmap/fda69060-fa79-4696-b1a6-7980f124bf7c-mcd-auth-proxy-config\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.808779 5052 reconciler_common.go:293] "Volume detached for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/57a731c4-ef35-47a8-b875-bfb08a7f8011-utilities\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.808795 5052 reconciler_common.go:293] "Volume detached for volume \"etcd-serving-ca\" (UniqueName: \"kubernetes.io/configmap/09ae3b1a-e8e7-4524-b54b-61eab6f9239a-etcd-serving-ca\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.808810 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-qg5z5\" (UniqueName: \"kubernetes.io/projected/43509403-f426-496e-be36-56cef71462f5-kube-api-access-qg5z5\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.808826 5052 reconciler_common.go:293] "Volume detached for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c03ee662-fb2f-4fc4-a2c1-af487c19d254-service-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.808842 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-nzwt7\" (UniqueName: \"kubernetes.io/projected/96b93a3a-6083-4aea-8eab-fe1aa8245ad9-kube-api-access-nzwt7\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.808865 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-v47cf\" (UniqueName: \"kubernetes.io/projected/c03ee662-fb2f-4fc4-a2c1-af487c19d254-kube-api-access-v47cf\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.808878 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access\" (UniqueName: \"kubernetes.io/projected/1386a44e-36a2-460c-96d0-0359d2b6f0f5-kube-api-access\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.808891 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-htfz6\" (UniqueName: \"kubernetes.io/projected/6ea678ab-3438-413e-bfe3-290ae7725660-kube-api-access-htfz6\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.808816 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/49c341d1-5089-4bc2-86a0-a5e165cfcc6b-v4-0-config-system-trusted-ca-bundle" (OuterVolumeSpecName: "v4-0-config-system-trusted-ca-bundle") pod "49c341d1-5089-4bc2-86a0-a5e165cfcc6b" (UID: "49c341d1-5089-4bc2-86a0-a5e165cfcc6b"). InnerVolumeSpecName "v4-0-config-system-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.808907 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/7539238d-5fe0-46ed-884e-1c3b566537ec-config\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.808945 5052 reconciler_common.go:293] "Volume detached for volume \"v4-0-config-user-template-login\" (UniqueName: \"kubernetes.io/secret/49c341d1-5089-4bc2-86a0-a5e165cfcc6b-v4-0-config-user-template-login\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.808966 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-7c4vf\" (UniqueName: \"kubernetes.io/projected/22c825df-677d-4ca6-82db-3454ed06e783-kube-api-access-7c4vf\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.808984 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-zkvpv\" (UniqueName: \"kubernetes.io/projected/09ae3b1a-e8e7-4524-b54b-61eab6f9239a-kube-api-access-zkvpv\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.808997 5052 reconciler_common.go:293] "Volume detached for volume \"etcd-client\" (UniqueName: \"kubernetes.io/secret/09efc573-dbb6-4249-bd59-9b87aba8dd28-etcd-client\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.809012 5052 reconciler_common.go:293] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8f668bae-612b-4b75-9490-919e737c6a3b-trusted-ca\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.809026 5052 reconciler_common.go:293] "Volume detached for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a31745f5-9847-4afe-82a5-3161cc66ca93-metrics-tls\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.809681 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7bb08738-c794-4ee8-9972-3a62ca171029-cni-binary-copy" (OuterVolumeSpecName: "cni-binary-copy") pod "7bb08738-c794-4ee8-9972-3a62ca171029" (UID: "7bb08738-c794-4ee8-9972-3a62ca171029"). InnerVolumeSpecName "cni-binary-copy". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.809776 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8f668bae-612b-4b75-9490-919e737c6a3b-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "8f668bae-612b-4b75-9490-919e737c6a3b" (UID: "8f668bae-612b-4b75-9490-919e737c6a3b"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.810116 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/25e176fe-21b4-4974-b1ed-c8b94f112a7f-kube-api-access-d4lsv" (OuterVolumeSpecName: "kube-api-access-d4lsv") pod "25e176fe-21b4-4974-b1ed-c8b94f112a7f" (UID: "25e176fe-21b4-4974-b1ed-c8b94f112a7f"). InnerVolumeSpecName "kube-api-access-d4lsv". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.810145 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5225d0e4-402f-4861-b410-819f433b1803-kube-api-access-9xfj7" (OuterVolumeSpecName: "kube-api-access-9xfj7") pod "5225d0e4-402f-4861-b410-819f433b1803" (UID: "5225d0e4-402f-4861-b410-819f433b1803"). InnerVolumeSpecName "kube-api-access-9xfj7". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.810418 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1bf7eb37-55a3-4c65-b768-a94c82151e69-serving-cert" (OuterVolumeSpecName: "serving-cert") pod "1bf7eb37-55a3-4c65-b768-a94c82151e69" (UID: "1bf7eb37-55a3-4c65-b768-a94c82151e69"). InnerVolumeSpecName "serving-cert". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.810446 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1bf7eb37-55a3-4c65-b768-a94c82151e69-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "1bf7eb37-55a3-4c65-b768-a94c82151e69" (UID: "1bf7eb37-55a3-4c65-b768-a94c82151e69"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.810804 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/308be0ea-9f5f-4b29-aeb1-5abd31a0b17b-tmpfs" (OuterVolumeSpecName: "tmpfs") pod "308be0ea-9f5f-4b29-aeb1-5abd31a0b17b" (UID: "308be0ea-9f5f-4b29-aeb1-5abd31a0b17b"). InnerVolumeSpecName "tmpfs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.810794 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f88749ec-7931-4ee7-b3fc-1ec5e11f92e9-profile-collector-cert" (OuterVolumeSpecName: "profile-collector-cert") pod "f88749ec-7931-4ee7-b3fc-1ec5e11f92e9" (UID: "f88749ec-7931-4ee7-b3fc-1ec5e11f92e9"). InnerVolumeSpecName "profile-collector-cert". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.810866 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5441d097-087c-4d9a-baa8-b210afa90fc9-client-ca" (OuterVolumeSpecName: "client-ca") pod "5441d097-087c-4d9a-baa8-b210afa90fc9" (UID: "5441d097-087c-4d9a-baa8-b210afa90fc9"). InnerVolumeSpecName "client-ca". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.810870 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1bf7eb37-55a3-4c65-b768-a94c82151e69-etcd-client" (OuterVolumeSpecName: "etcd-client") pod "1bf7eb37-55a3-4c65-b768-a94c82151e69" (UID: "1bf7eb37-55a3-4c65-b768-a94c82151e69"). InnerVolumeSpecName "etcd-client". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.810964 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1bf7eb37-55a3-4c65-b768-a94c82151e69-config" (OuterVolumeSpecName: "config") pod "1bf7eb37-55a3-4c65-b768-a94c82151e69" (UID: "1bf7eb37-55a3-4c65-b768-a94c82151e69"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.811155 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b6cd30de-2eeb-49a2-ab40-9167f4560ff5-kube-api-access-pj782" (OuterVolumeSpecName: "kube-api-access-pj782") pod "b6cd30de-2eeb-49a2-ab40-9167f4560ff5" (UID: "b6cd30de-2eeb-49a2-ab40-9167f4560ff5"). InnerVolumeSpecName "kube-api-access-pj782". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.814144 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-apiserver_kube-apiserver-crc_f4b27818a5e8e43d0dc095d08835c792/kube-apiserver-check-endpoints/0.log" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.819985 5052 generic.go:334] "Generic (PLEG): container finished" podID="f4b27818a5e8e43d0dc095d08835c792" containerID="b739f84524d09b27f6e052e21df8be3870a2136665704232f5a365d8a5bf41f5" exitCode=255 Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.820793 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-kube-apiserver/kube-apiserver-crc" event={"ID":"f4b27818a5e8e43d0dc095d08835c792","Type":"ContainerDied","Data":"b739f84524d09b27f6e052e21df8be3870a2136665704232f5a365d8a5bf41f5"} Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.821431 5052 status_manager.go:875] "Failed to update status for pod" pod="openshift-network-diagnostics/network-check-target-xd92c" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"3b6479f0-333b-4a96-9adf-2099afdc2447\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"message\\\":\\\"containers with unready status: [network-check-target-container]\\\",\\\"reason\\\":\\\"ContainersNotReady\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"message\\\":\\\"containers with unready status: [network-check-target-container]\\\",\\\"reason\\\":\\\"ContainersNotReady\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"ContainersReady\\\"}],\\\"containerStatuses\\\":[{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e1baa38811c04bd8909e01a1f3be7421a1cb99d608d3dc4cf86d95b17de2ab8b\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{\\\"terminated\\\":{\\\"exitCode\\\":137,\\\"finishedAt\\\":null,\\\"message\\\":\\\"The container could not be located when the pod was deleted. The container used to be Running\\\",\\\"reason\\\":\\\"ContainerStatusUnknown\\\",\\\"startedAt\\\":null}},\\\"name\\\":\\\"network-check-target-container\\\",\\\"ready\\\":false,\\\"restartCount\\\":3,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"ContainerCreating\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-cqllr\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}],\\\"podIP\\\":null,\\\"podIPs\\\":null}}\" for pod \"openshift-network-diagnostics\"/\"network-check-target-xd92c\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://127.0.0.1:9743/pod?timeout=10s\": dial tcp 127.0.0.1:9743: connect: connection refused" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.824300 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d-catalog-content" (OuterVolumeSpecName: "catalog-content") pod "b11524ee-3fca-4b1b-9cdf-6da289fdbc7d" (UID: "b11524ee-3fca-4b1b-9cdf-6da289fdbc7d"). InnerVolumeSpecName "catalog-content". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.830152 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openshift-kube-apiserver/kube-apiserver-crc"] Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.830580 5052 scope.go:117] "RemoveContainer" containerID="b739f84524d09b27f6e052e21df8be3870a2136665704232f5a365d8a5bf41f5" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.831044 5052 status_manager.go:875] "Failed to update status for pod" pod="openshift-dns/node-resolver-xph69" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"e5c587e6-866a-4427-8a2c-3469cf0fc495\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"Initialized\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"message\\\":\\\"containers with unready status: [dns-node-resolver]\\\",\\\"reason\\\":\\\"ContainersNotReady\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"message\\\":\\\"containers with unready status: [dns-node-resolver]\\\",\\\"reason\\\":\\\"ContainersNotReady\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"ContainersReady\\\"}],\\\"containerStatuses\\\":[{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:35512335ac39aed0f55b7f799f416f4f6445c20c1b19888cf2bb72bb276703f2\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"dns-node-resolver\\\",\\\"ready\\\":false,\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"ContainerCreating\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/etc/hosts\\\",\\\"name\\\":\\\"hosts-file\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-tk7n6\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}],\\\"hostIP\\\":\\\"192.168.126.11\\\",\\\"hostIPs\\\":[{\\\"ip\\\":\\\"192.168.126.11\\\"}],\\\"podIP\\\":\\\"192.168.126.11\\\",\\\"podIPs\\\":[{\\\"ip\\\":\\\"192.168.126.11\\\"}],\\\"startTime\\\":\\\"2025-12-03T11:20:31Z\\\"}}\" for pod \"openshift-dns\"/\"node-resolver-xph69\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://127.0.0.1:9743/pod?timeout=10s\": dial tcp 127.0.0.1:9743: connect: connection refused" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.837411 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8f668bae-612b-4b75-9490-919e737c6a3b-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "8f668bae-612b-4b75-9490-919e737c6a3b" (UID: "8f668bae-612b-4b75-9490-919e737c6a3b"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.838440 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5225d0e4-402f-4861-b410-819f433b1803-catalog-content" (OuterVolumeSpecName: "catalog-content") pod "5225d0e4-402f-4861-b410-819f433b1803" (UID: "5225d0e4-402f-4861-b410-819f433b1803"). InnerVolumeSpecName "catalog-content". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.842705 5052 status_manager.go:875] "Failed to update status for pod" pod="openshift-multus/multus-additional-cni-plugins-2lrql" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"17265a68-bcb5-455c-b7df-c148a6517db9\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"message\\\":\\\"containers with incomplete status: [egress-router-binary-copy cni-plugins bond-cni-plugin routeoverride-cni whereabouts-cni-bincopy whereabouts-cni]\\\",\\\"reason\\\":\\\"ContainersNotInitialized\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"Initialized\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"message\\\":\\\"containers with unready status: [kube-multus-additional-cni-plugins]\\\",\\\"reason\\\":\\\"ContainersNotReady\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"message\\\":\\\"containers with unready status: [kube-multus-additional-cni-plugins]\\\",\\\"reason\\\":\\\"ContainersNotReady\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"ContainersReady\\\"}],\\\"containerStatuses\\\":[{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7eeaee65f2808b819eedb413bdcabb9144e12f0dd97f13fd1afba93a95b67b26\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"kube-multus-additional-cni-plugins\\\",\\\"ready\\\":false,\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"PodInitializing\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-w4zf5\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}],\\\"hostIP\\\":\\\"192.168.126.11\\\",\\\"hostIPs\\\":[{\\\"ip\\\":\\\"192.168.126.11\\\"}],\\\"initContainerStatuses\\\":[{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e40792096b162f0f9ce5f8362f51e5f8dea2c1ce4b1447235388416b5db7708c\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"egress-router-binary-copy\\\",\\\"ready\\\":false,\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"PodInitializing\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/entrypoint\\\",\\\"name\\\":\\\"cni-binary-copy\\\"},{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/os-release\\\",\\\"name\\\":\\\"os-release\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-w4zf5\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:687fddfbb085a1688df312ce4ec8c857df9b2daed8ff4a7ed6163a1154afa2cc\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"cni-plugins\\\",\\\"ready\\\":false,\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"PodInitializing\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/entrypoint\\\",\\\"name\\\":\\\"cni-binary-copy\\\"},{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/os-release\\\",\\\"name\\\":\\\"os-release\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/host/etc/cni/tuning/\\\",\\\"name\\\":\\\"tuning-conf-dir\\\"},{\\\"mountPath\\\":\\\"/sysctls\\\",\\\"name\\\":\\\"cni-sysctl-allowlist\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-w4zf5\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:383f4cceeeaead203bb2327fdd367c64b64d729d7fa93089f249e496fcef0c78\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"bond-cni-plugin\\\",\\\"ready\\\":false,\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"PodInitializing\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/entrypoint\\\",\\\"name\\\":\\\"cni-binary-copy\\\"},{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/os-release\\\",\\\"name\\\":\\\"os-release\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-w4zf5\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f567acb85146b5ed81451ec3e79f2de0c62e28c69b2eeade0abdf5d0c388e7aa\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"routeoverride-cni\\\",\\\"ready\\\":false,\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"PodInitializing\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/entrypoint\\\",\\\"name\\\":\\\"cni-binary-copy\\\"},{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/os-release\\\",\\\"name\\\":\\\"os-release\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-w4zf5\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:98100674616e54319f6713d742fd0c3bdbc84e6e6173e8ccf4a2473a714c2bc4\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"whereabouts-cni-bincopy\\\",\\\"ready\\\":false,\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"PodInitializing\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/entrypoint\\\",\\\"name\\\":\\\"cni-binary-copy\\\"},{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/os-release\\\",\\\"name\\\":\\\"os-release\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-w4zf5\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:98100674616e54319f6713d742fd0c3bdbc84e6e6173e8ccf4a2473a714c2bc4\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"whereabouts-cni\\\",\\\"ready\\\":false,\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"PodInitializing\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/cni/net.d\\\",\\\"name\\\":\\\"system-cni-dir\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-w4zf5\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}],\\\"podIP\\\":\\\"192.168.126.11\\\",\\\"podIPs\\\":[{\\\"ip\\\":\\\"192.168.126.11\\\"}],\\\"startTime\\\":\\\"2025-12-03T11:20:31Z\\\"}}\" for pod \"openshift-multus\"/\"multus-additional-cni-plugins-2lrql\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://127.0.0.1:9743/pod?timeout=10s\": dial tcp 127.0.0.1:9743: connect: connection refused" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.853907 5052 status_manager.go:875] "Failed to update status for pod" pod="openshift-network-diagnostics/network-check-source-55646444c4-trplf" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"9d751cbb-f2e2-430d-9754-c882a5e924a5\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"message\\\":\\\"containers with unready status: [check-endpoints]\\\",\\\"reason\\\":\\\"ContainersNotReady\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"message\\\":\\\"containers with unready status: [check-endpoints]\\\",\\\"reason\\\":\\\"ContainersNotReady\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"ContainersReady\\\"}],\\\"containerStatuses\\\":[{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e1baa38811c04bd8909e01a1f3be7421a1cb99d608d3dc4cf86d95b17de2ab8b\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{\\\"terminated\\\":{\\\"exitCode\\\":137,\\\"finishedAt\\\":null,\\\"message\\\":\\\"The container could not be located when the pod was deleted. The container used to be Running\\\",\\\"reason\\\":\\\"ContainerStatusUnknown\\\",\\\"startedAt\\\":null}},\\\"name\\\":\\\"check-endpoints\\\",\\\"ready\\\":false,\\\"restartCount\\\":3,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"ContainerCreating\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-s2dwl\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}],\\\"podIP\\\":null,\\\"podIPs\\\":null}}\" for pod \"openshift-network-diagnostics\"/\"network-check-source-55646444c4-trplf\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://127.0.0.1:9743/pod?timeout=10s\": dial tcp 127.0.0.1:9743: connect: connection refused" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.866464 5052 status_manager.go:875] "Failed to update status for pod" pod="openshift-network-operator/iptables-alerter-4ln5h" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"d75a4c96-2883-4a0b-bab2-0fab2b6c0b49\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"message\\\":\\\"containers with unready status: [iptables-alerter]\\\",\\\"reason\\\":\\\"ContainersNotReady\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"message\\\":\\\"containers with unready status: [iptables-alerter]\\\",\\\"reason\\\":\\\"ContainersNotReady\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"ContainersReady\\\"}],\\\"containerStatuses\\\":[{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:35512335ac39aed0f55b7f799f416f4f6445c20c1b19888cf2bb72bb276703f2\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{\\\"terminated\\\":{\\\"exitCode\\\":137,\\\"finishedAt\\\":null,\\\"message\\\":\\\"The container could not be located when the pod was deleted. The container used to be Running\\\",\\\"reason\\\":\\\"ContainerStatusUnknown\\\",\\\"startedAt\\\":null}},\\\"name\\\":\\\"iptables-alerter\\\",\\\"ready\\\":false,\\\"restartCount\\\":3,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"ContainerCreating\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/iptables-alerter\\\",\\\"name\\\":\\\"iptables-alerter-script\\\"},{\\\"mountPath\\\":\\\"/host\\\",\\\"name\\\":\\\"host-slash\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-rczfb\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}]}}\" for pod \"openshift-network-operator\"/\"iptables-alerter-4ln5h\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://127.0.0.1:9743/pod?timeout=10s\": dial tcp 127.0.0.1:9743: connect: connection refused" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.888161 5052 status_manager.go:875] "Failed to update status for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kg55h" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"bc569bba-c020-40d6-af68-e617625fe0e4\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"message\\\":\\\"containers with incomplete status: [kubecfg-setup]\\\",\\\"reason\\\":\\\"ContainersNotInitialized\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"Initialized\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"message\\\":\\\"containers with unready status: [ovn-controller ovn-acl-logging kube-rbac-proxy-node kube-rbac-proxy-ovn-metrics northd nbdb sbdb ovnkube-controller]\\\",\\\"reason\\\":\\\"ContainersNotReady\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"message\\\":\\\"containers with unready status: [ovn-controller ovn-acl-logging kube-rbac-proxy-node kube-rbac-proxy-ovn-metrics northd nbdb sbdb ovnkube-controller]\\\",\\\"reason\\\":\\\"ContainersNotReady\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"ContainersReady\\\"}],\\\"containerStatuses\\\":[{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:242b3d66438c42745f4ef318bdeaf3d793426f12962a42ea83e18d06c08aaf09\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"kube-rbac-proxy-node\\\",\\\"ready\\\":false,\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"PodInitializing\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/ovnkube-lib\\\",\\\"name\\\":\\\"ovnkube-script-lib\\\"},{\\\"mountPath\\\":\\\"/etc/pki/tls/metrics-cert\\\",\\\"name\\\":\\\"ovn-node-metrics-cert\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-x92h2\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:242b3d66438c42745f4ef318bdeaf3d793426f12962a42ea83e18d06c08aaf09\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"kube-rbac-proxy-ovn-metrics\\\",\\\"ready\\\":false,\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"PodInitializing\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/ovnkube-lib\\\",\\\"name\\\":\\\"ovnkube-script-lib\\\"},{\\\"mountPath\\\":\\\"/etc/pki/tls/metrics-cert\\\",\\\"name\\\":\\\"ovn-node-metrics-cert\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-x92h2\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:174f36cdd47ef0d1d2099482919d773257453265a2af0b17b154edc32fa41ac2\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"nbdb\\\",\\\"ready\\\":false,\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"PodInitializing\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/ovnkube-lib\\\",\\\"name\\\":\\\"ovnkube-script-lib\\\"},{\\\"mountPath\\\":\\\"/etc/ovn/\\\",\\\"name\\\":\\\"etc-openvswitch\\\"},{\\\"mountPath\\\":\\\"/var/log/ovn\\\",\\\"name\\\":\\\"node-log\\\"},{\\\"mountPath\\\":\\\"/run/ovn/\\\",\\\"name\\\":\\\"run-ovn\\\"},{\\\"mountPath\\\":\\\"/env\\\",\\\"name\\\":\\\"env-overrides\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-x92h2\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:174f36cdd47ef0d1d2099482919d773257453265a2af0b17b154edc32fa41ac2\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"northd\\\",\\\"ready\\\":false,\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"PodInitializing\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/ovnkube-lib\\\",\\\"name\\\":\\\"ovnkube-script-lib\\\"},{\\\"mountPath\\\":\\\"/etc/ovn\\\",\\\"name\\\":\\\"etc-openvswitch\\\"},{\\\"mountPath\\\":\\\"/var/log/ovn\\\",\\\"name\\\":\\\"node-log\\\"},{\\\"mountPath\\\":\\\"/run/ovn/\\\",\\\"name\\\":\\\"run-ovn\\\"},{\\\"mountPath\\\":\\\"/env\\\",\\\"name\\\":\\\"env-overrides\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-x92h2\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:174f36cdd47ef0d1d2099482919d773257453265a2af0b17b154edc32fa41ac2\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"ovn-acl-logging\\\",\\\"ready\\\":false,\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"PodInitializing\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/ovnkube-lib\\\",\\\"name\\\":\\\"ovnkube-script-lib\\\"},{\\\"mountPath\\\":\\\"/var/log/ovn/\\\",\\\"name\\\":\\\"node-log\\\"},{\\\"mountPath\\\":\\\"/run/ovn/\\\",\\\"name\\\":\\\"run-ovn\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-x92h2\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:174f36cdd47ef0d1d2099482919d773257453265a2af0b17b154edc32fa41ac2\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"ovn-controller\\\",\\\"ready\\\":false,\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"PodInitializing\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/ovnkube-lib\\\",\\\"name\\\":\\\"ovnkube-script-lib\\\"},{\\\"mountPath\\\":\\\"/run/openvswitch\\\",\\\"name\\\":\\\"run-openvswitch\\\"},{\\\"mountPath\\\":\\\"/run/ovn/\\\",\\\"name\\\":\\\"run-ovn\\\"},{\\\"mountPath\\\":\\\"/etc/openvswitch\\\",\\\"name\\\":\\\"etc-openvswitch\\\"},{\\\"mountPath\\\":\\\"/etc/ovn/\\\",\\\"name\\\":\\\"etc-openvswitch\\\"},{\\\"mountPath\\\":\\\"/var/lib/openvswitch\\\",\\\"name\\\":\\\"var-lib-openvswitch\\\"},{\\\"mountPath\\\":\\\"/env\\\",\\\"name\\\":\\\"env-overrides\\\"},{\\\"mountPath\\\":\\\"/var/log/ovn/\\\",\\\"name\\\":\\\"node-log\\\"},{\\\"mountPath\\\":\\\"/dev/log\\\",\\\"name\\\":\\\"log-socket\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-x92h2\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:174f36cdd47ef0d1d2099482919d773257453265a2af0b17b154edc32fa41ac2\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"ovnkube-controller\\\",\\\"ready\\\":false,\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"PodInitializing\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/ovnkube-lib\\\",\\\"name\\\":\\\"ovnkube-script-lib\\\"},{\\\"mountPath\\\":\\\"/var/lib/kubelet\\\",\\\"name\\\":\\\"host-kubelet\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/etc/systemd/system\\\",\\\"name\\\":\\\"systemd-units\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/host\\\",\\\"name\\\":\\\"host-slash\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/run/ovn-kubernetes/\\\",\\\"name\\\":\\\"host-run-ovn-kubernetes\\\"},{\\\"mountPath\\\":\\\"/run/netns\\\",\\\"name\\\":\\\"host-run-netns\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/run/systemd/private\\\",\\\"name\\\":\\\"run-systemd\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/cni-bin-dir\\\",\\\"name\\\":\\\"host-cni-bin\\\"},{\\\"mountPath\\\":\\\"/etc/cni/net.d\\\",\\\"name\\\":\\\"host-cni-netd\\\"},{\\\"mountPath\\\":\\\"/var/lib/cni/networks/ovn-k8s-cni-overlay\\\",\\\"name\\\":\\\"host-var-lib-cni-networks-ovn-kubernetes\\\"},{\\\"mountPath\\\":\\\"/run/openvswitch\\\",\\\"name\\\":\\\"run-openvswitch\\\"},{\\\"mountPath\\\":\\\"/var/log/ovnkube/\\\",\\\"name\\\":\\\"etc-openvswitch\\\"},{\\\"mountPath\\\":\\\"/run/ovn/\\\",\\\"name\\\":\\\"run-ovn\\\"},{\\\"mountPath\\\":\\\"/etc/openvswitch\\\",\\\"name\\\":\\\"etc-openvswitch\\\"},{\\\"mountPath\\\":\\\"/etc/ovn/\\\",\\\"name\\\":\\\"etc-openvswitch\\\"},{\\\"mountPath\\\":\\\"/var/lib/openvswitch\\\",\\\"name\\\":\\\"var-lib-openvswitch\\\"},{\\\"mountPath\\\":\\\"/run/ovnkube-config/\\\",\\\"name\\\":\\\"ovnkube-config\\\"},{\\\"mountPath\\\":\\\"/env\\\",\\\"name\\\":\\\"env-overrides\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-x92h2\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:174f36cdd47ef0d1d2099482919d773257453265a2af0b17b154edc32fa41ac2\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"sbdb\\\",\\\"ready\\\":false,\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"PodInitializing\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/ovnkube-lib\\\",\\\"name\\\":\\\"ovnkube-script-lib\\\"},{\\\"mountPath\\\":\\\"/etc/ovn/\\\",\\\"name\\\":\\\"etc-openvswitch\\\"},{\\\"mountPath\\\":\\\"/run/ovn/\\\",\\\"name\\\":\\\"run-ovn\\\"},{\\\"mountPath\\\":\\\"/var/log/ovn\\\",\\\"name\\\":\\\"node-log\\\"},{\\\"mountPath\\\":\\\"/env\\\",\\\"name\\\":\\\"env-overrides\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-x92h2\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}],\\\"hostIP\\\":\\\"192.168.126.11\\\",\\\"hostIPs\\\":[{\\\"ip\\\":\\\"192.168.126.11\\\"}],\\\"initContainerStatuses\\\":[{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:174f36cdd47ef0d1d2099482919d773257453265a2af0b17b154edc32fa41ac2\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"kubecfg-setup\\\",\\\"ready\\\":false,\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"PodInitializing\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/etc/ovn/\\\",\\\"name\\\":\\\"etc-openvswitch\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-x92h2\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}],\\\"podIP\\\":\\\"192.168.126.11\\\",\\\"podIPs\\\":[{\\\"ip\\\":\\\"192.168.126.11\\\"}],\\\"startTime\\\":\\\"2025-12-03T11:20:31Z\\\"}}\" for pod \"openshift-ovn-kubernetes\"/\"ovnkube-node-kg55h\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://127.0.0.1:9743/pod?timeout=10s\": dial tcp 127.0.0.1:9743: connect: connection refused" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.902313 5052 status_manager.go:875] "Failed to update status for pod" pod="openshift-kube-controller-manager/kube-controller-manager-crc" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"f31e1704-9c1d-4044-b318-2aaccb5896e2\\\"},\\\"status\\\":{\\\"conditions\\\":[{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:15Z\\\",\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:09Z\\\",\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"Initialized\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"ContainersReady\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:09Z\\\",\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"PodScheduled\\\"}],\\\"containerStatuses\\\":[{\\\"containerID\\\":\\\"cri-o://37485762ea1808cd3794c9641a4ead224b107e6c64c295de237f8ffd50aba4f4\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c0f9da410c07372b6c9ad6a79379b491cd10fdee88051c026b084652d85aed21\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c0f9da410c07372b6c9ad6a79379b491cd10fdee88051c026b084652d85aed21\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"cluster-policy-controller\\\",\\\"ready\\\":true,\\\"restartCount\\\":0,\\\"started\\\":true,\\\"state\\\":{\\\"running\\\":{\\\"startedAt\\\":\\\"2025-12-03T11:20:12Z\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/etc/kubernetes/static-pod-resources\\\",\\\"name\\\":\\\"resource-dir\\\"},{\\\"mountPath\\\":\\\"/etc/kubernetes/static-pod-certs\\\",\\\"name\\\":\\\"cert-dir\\\"}]},{\\\"containerID\\\":\\\"cri-o://f5d23c6160b5790e9272bd5ae83472267f4d1f60b67e0425a6bc2f5c1f9d1fb7\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:06bc35825771aee1220d34720243b89c4ba8a8b335e6de2597126bd791fd90d4\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:06bc35825771aee1220d34720243b89c4ba8a8b335e6de2597126bd791fd90d4\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"kube-controller-manager\\\",\\\"ready\\\":true,\\\"restartCount\\\":0,\\\"started\\\":true,\\\"state\\\":{\\\"running\\\":{\\\"startedAt\\\":\\\"2025-12-03T11:20:11Z\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/etc/kubernetes/static-pod-resources\\\",\\\"name\\\":\\\"resource-dir\\\"},{\\\"mountPath\\\":\\\"/etc/kubernetes/static-pod-certs\\\",\\\"name\\\":\\\"cert-dir\\\"}]},{\\\"containerID\\\":\\\"cri-o://d05df6945d11d2abe73005efafe80220497a339008d4c28f7923c2dc7ac4a150\\\",\\\"image\\\":\\\"quay.io/crcont/openshift-crc-cluster-kube-controller-manager-operator@sha256:8506ce0a578bc18fac117eb2b82799488ffac0bed08287faaf92edaf5d17ab95\\\",\\\"imageID\\\":\\\"quay.io/crcont/openshift-crc-cluster-kube-controller-manager-operator@sha256:8506ce0a578bc18fac117eb2b82799488ffac0bed08287faaf92edaf5d17ab95\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"kube-controller-manager-cert-syncer\\\",\\\"ready\\\":true,\\\"restartCount\\\":0,\\\"started\\\":true,\\\"state\\\":{\\\"running\\\":{\\\"startedAt\\\":\\\"2025-12-03T11:20:12Z\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/etc/kubernetes/static-pod-resources\\\",\\\"name\\\":\\\"resource-dir\\\"},{\\\"mountPath\\\":\\\"/etc/kubernetes/static-pod-certs\\\",\\\"name\\\":\\\"cert-dir\\\"}]},{\\\"containerID\\\":\\\"cri-o://8df2aa2470046bba37b41cb301e96f52b09b83124384b5dbc4942cd0bfeef1c6\\\",\\\"image\\\":\\\"quay.io/crcont/openshift-crc-cluster-kube-controller-manager-operator@sha256:8506ce0a578bc18fac117eb2b82799488ffac0bed08287faaf92edaf5d17ab95\\\",\\\"imageID\\\":\\\"quay.io/crcont/openshift-crc-cluster-kube-controller-manager-operator@sha256:8506ce0a578bc18fac117eb2b82799488ffac0bed08287faaf92edaf5d17ab95\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"kube-controller-manager-recovery-controller\\\",\\\"ready\\\":true,\\\"restartCount\\\":0,\\\"started\\\":true,\\\"state\\\":{\\\"running\\\":{\\\"startedAt\\\":\\\"2025-12-03T11:20:15Z\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/etc/kubernetes/static-pod-resources\\\",\\\"name\\\":\\\"resource-dir\\\"},{\\\"mountPath\\\":\\\"/etc/kubernetes/static-pod-certs\\\",\\\"name\\\":\\\"cert-dir\\\"}]}],\\\"hostIP\\\":\\\"192.168.126.11\\\",\\\"hostIPs\\\":[{\\\"ip\\\":\\\"192.168.126.11\\\"}],\\\"phase\\\":\\\"Running\\\",\\\"podIP\\\":\\\"192.168.126.11\\\",\\\"podIPs\\\":[{\\\"ip\\\":\\\"192.168.126.11\\\"}],\\\"startTime\\\":\\\"2025-12-03T11:20:09Z\\\"}}\" for pod \"openshift-kube-controller-manager\"/\"kube-controller-manager-crc\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://127.0.0.1:9743/pod?timeout=10s\": dial tcp 127.0.0.1:9743: connect: connection refused" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.910669 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/17265a68-bcb5-455c-b7df-c148a6517db9-cnibin\") pod \"multus-additional-cni-plugins-2lrql\" (UID: \"17265a68-bcb5-455c-b7df-c148a6517db9\") " pod="openshift-multus/multus-additional-cni-plugins-2lrql" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.910715 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c30c02db-f4e7-4432-941c-5448fd71f3d9-etc-kubernetes\") pod \"multus-hx657\" (UID: \"c30c02db-f4e7-4432-941c-5448fd71f3d9\") " pod="openshift-multus/multus-hx657" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.910730 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c30c02db-f4e7-4432-941c-5448fd71f3d9-system-cni-dir\") pod \"multus-hx657\" (UID: \"c30c02db-f4e7-4432-941c-5448fd71f3d9\") " pod="openshift-multus/multus-hx657" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.910754 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c30c02db-f4e7-4432-941c-5448fd71f3d9-multus-conf-dir\") pod \"multus-hx657\" (UID: \"c30c02db-f4e7-4432-941c-5448fd71f3d9\") " pod="openshift-multus/multus-hx657" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.910770 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"mcd-auth-proxy-config\" (UniqueName: \"kubernetes.io/configmap/10c237bc-72e7-4da3-9f2c-b1688395bf6e-mcd-auth-proxy-config\") pod \"machine-config-daemon-74bj6\" (UID: \"10c237bc-72e7-4da3-9f2c-b1688395bf6e\") " pod="openshift-machine-config-operator/machine-config-daemon-74bj6" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.910789 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bc569bba-c020-40d6-af68-e617625fe0e4-host-slash\") pod \"ovnkube-node-kg55h\" (UID: \"bc569bba-c020-40d6-af68-e617625fe0e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-kg55h" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.910807 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/bc569bba-c020-40d6-af68-e617625fe0e4-ovnkube-config\") pod \"ovnkube-node-kg55h\" (UID: \"bc569bba-c020-40d6-af68-e617625fe0e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-kg55h" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.910821 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/c30c02db-f4e7-4432-941c-5448fd71f3d9-hostroot\") pod \"multus-hx657\" (UID: \"c30c02db-f4e7-4432-941c-5448fd71f3d9\") " pod="openshift-multus/multus-hx657" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.910838 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bc569bba-c020-40d6-af68-e617625fe0e4-run-openvswitch\") pod \"ovnkube-node-kg55h\" (UID: \"bc569bba-c020-40d6-af68-e617625fe0e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-kg55h" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.910858 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c30c02db-f4e7-4432-941c-5448fd71f3d9-host-var-lib-kubelet\") pod \"multus-hx657\" (UID: \"c30c02db-f4e7-4432-941c-5448fd71f3d9\") " pod="openshift-multus/multus-hx657" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.910880 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bc569bba-c020-40d6-af68-e617625fe0e4-host-run-ovn-kubernetes\") pod \"ovnkube-node-kg55h\" (UID: \"bc569bba-c020-40d6-af68-e617625fe0e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-kg55h" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.910898 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/bc569bba-c020-40d6-af68-e617625fe0e4-host-cni-netd\") pod \"ovnkube-node-kg55h\" (UID: \"bc569bba-c020-40d6-af68-e617625fe0e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-kg55h" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.910919 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bc569bba-c020-40d6-af68-e617625fe0e4-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-kg55h\" (UID: \"bc569bba-c020-40d6-af68-e617625fe0e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-kg55h" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.910964 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c30c02db-f4e7-4432-941c-5448fd71f3d9-cni-binary-copy\") pod \"multus-hx657\" (UID: \"c30c02db-f4e7-4432-941c-5448fd71f3d9\") " pod="openshift-multus/multus-hx657" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.910978 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c30c02db-f4e7-4432-941c-5448fd71f3d9-cnibin\") pod \"multus-hx657\" (UID: \"c30c02db-f4e7-4432-941c-5448fd71f3d9\") " pod="openshift-multus/multus-hx657" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.910994 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/bc569bba-c020-40d6-af68-e617625fe0e4-run-systemd\") pod \"ovnkube-node-kg55h\" (UID: \"bc569bba-c020-40d6-af68-e617625fe0e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-kg55h" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.911012 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/bc569bba-c020-40d6-af68-e617625fe0e4-log-socket\") pod \"ovnkube-node-kg55h\" (UID: \"bc569bba-c020-40d6-af68-e617625fe0e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-kg55h" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.911028 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e5c587e6-866a-4427-8a2c-3469cf0fc495-hosts-file\") pod \"node-resolver-xph69\" (UID: \"e5c587e6-866a-4427-8a2c-3469cf0fc495\") " pod="openshift-dns/node-resolver-xph69" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.911055 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-tk7n6\" (UniqueName: \"kubernetes.io/projected/e5c587e6-866a-4427-8a2c-3469cf0fc495-kube-api-access-tk7n6\") pod \"node-resolver-xph69\" (UID: \"e5c587e6-866a-4427-8a2c-3469cf0fc495\") " pod="openshift-dns/node-resolver-xph69" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.911070 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c30c02db-f4e7-4432-941c-5448fd71f3d9-host-run-netns\") pod \"multus-hx657\" (UID: \"c30c02db-f4e7-4432-941c-5448fd71f3d9\") " pod="openshift-multus/multus-hx657" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.911091 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/17265a68-bcb5-455c-b7df-c148a6517db9-cni-binary-copy\") pod \"multus-additional-cni-plugins-2lrql\" (UID: \"17265a68-bcb5-455c-b7df-c148a6517db9\") " pod="openshift-multus/multus-additional-cni-plugins-2lrql" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.911108 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c30c02db-f4e7-4432-941c-5448fd71f3d9-host-var-lib-cni-bin\") pod \"multus-hx657\" (UID: \"c30c02db-f4e7-4432-941c-5448fd71f3d9\") " pod="openshift-multus/multus-hx657" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.911123 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bc569bba-c020-40d6-af68-e617625fe0e4-etc-openvswitch\") pod \"ovnkube-node-kg55h\" (UID: \"bc569bba-c020-40d6-af68-e617625fe0e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-kg55h" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.911141 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/bc569bba-c020-40d6-af68-e617625fe0e4-run-ovn\") pod \"ovnkube-node-kg55h\" (UID: \"bc569bba-c020-40d6-af68-e617625fe0e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-kg55h" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.911172 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/17265a68-bcb5-455c-b7df-c148a6517db9-system-cni-dir\") pod \"multus-additional-cni-plugins-2lrql\" (UID: \"17265a68-bcb5-455c-b7df-c148a6517db9\") " pod="openshift-multus/multus-additional-cni-plugins-2lrql" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.911190 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/17265a68-bcb5-455c-b7df-c148a6517db9-tuning-conf-dir\") pod \"multus-additional-cni-plugins-2lrql\" (UID: \"17265a68-bcb5-455c-b7df-c148a6517db9\") " pod="openshift-multus/multus-additional-cni-plugins-2lrql" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.911209 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/bc569bba-c020-40d6-af68-e617625fe0e4-host-kubelet\") pod \"ovnkube-node-kg55h\" (UID: \"bc569bba-c020-40d6-af68-e617625fe0e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-kg55h" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.911225 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bc569bba-c020-40d6-af68-e617625fe0e4-host-cni-bin\") pod \"ovnkube-node-kg55h\" (UID: \"bc569bba-c020-40d6-af68-e617625fe0e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-kg55h" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.911242 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/bc569bba-c020-40d6-af68-e617625fe0e4-ovnkube-script-lib\") pod \"ovnkube-node-kg55h\" (UID: \"bc569bba-c020-40d6-af68-e617625fe0e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-kg55h" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.911259 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bc569bba-c020-40d6-af68-e617625fe0e4-host-run-netns\") pod \"ovnkube-node-kg55h\" (UID: \"bc569bba-c020-40d6-af68-e617625fe0e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-kg55h" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.911275 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/bc569bba-c020-40d6-af68-e617625fe0e4-node-log\") pod \"ovnkube-node-kg55h\" (UID: \"bc569bba-c020-40d6-af68-e617625fe0e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-kg55h" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.911290 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"rootfs\" (UniqueName: \"kubernetes.io/host-path/10c237bc-72e7-4da3-9f2c-b1688395bf6e-rootfs\") pod \"machine-config-daemon-74bj6\" (UID: \"10c237bc-72e7-4da3-9f2c-b1688395bf6e\") " pod="openshift-machine-config-operator/machine-config-daemon-74bj6" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.911305 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/c30c02db-f4e7-4432-941c-5448fd71f3d9-multus-daemon-config\") pod \"multus-hx657\" (UID: \"c30c02db-f4e7-4432-941c-5448fd71f3d9\") " pod="openshift-multus/multus-hx657" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.911321 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/bc569bba-c020-40d6-af68-e617625fe0e4-systemd-units\") pod \"ovnkube-node-kg55h\" (UID: \"bc569bba-c020-40d6-af68-e617625fe0e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-kg55h" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.911345 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/c30c02db-f4e7-4432-941c-5448fd71f3d9-host-run-k8s-cni-cncf-io\") pod \"multus-hx657\" (UID: \"c30c02db-f4e7-4432-941c-5448fd71f3d9\") " pod="openshift-multus/multus-hx657" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.911373 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/10c237bc-72e7-4da3-9f2c-b1688395bf6e-proxy-tls\") pod \"machine-config-daemon-74bj6\" (UID: \"10c237bc-72e7-4da3-9f2c-b1688395bf6e\") " pod="openshift-machine-config-operator/machine-config-daemon-74bj6" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.911430 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-w4zf5\" (UniqueName: \"kubernetes.io/projected/17265a68-bcb5-455c-b7df-c148a6517db9-kube-api-access-w4zf5\") pod \"multus-additional-cni-plugins-2lrql\" (UID: \"17265a68-bcb5-455c-b7df-c148a6517db9\") " pod="openshift-multus/multus-additional-cni-plugins-2lrql" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.911448 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c30c02db-f4e7-4432-941c-5448fd71f3d9-os-release\") pod \"multus-hx657\" (UID: \"c30c02db-f4e7-4432-941c-5448fd71f3d9\") " pod="openshift-multus/multus-hx657" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.911467 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/c30c02db-f4e7-4432-941c-5448fd71f3d9-multus-socket-dir-parent\") pod \"multus-hx657\" (UID: \"c30c02db-f4e7-4432-941c-5448fd71f3d9\") " pod="openshift-multus/multus-hx657" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.911483 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/bc569bba-c020-40d6-af68-e617625fe0e4-ovn-node-metrics-cert\") pod \"ovnkube-node-kg55h\" (UID: \"bc569bba-c020-40d6-af68-e617625fe0e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-kg55h" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.911498 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d75a4c96-2883-4a0b-bab2-0fab2b6c0b49-host-slash\") pod \"iptables-alerter-4ln5h\" (UID: \"d75a4c96-2883-4a0b-bab2-0fab2b6c0b49\") " pod="openshift-network-operator/iptables-alerter-4ln5h" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.911536 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-fhsjl\" (UniqueName: \"kubernetes.io/projected/c30c02db-f4e7-4432-941c-5448fd71f3d9-kube-api-access-fhsjl\") pod \"multus-hx657\" (UID: \"c30c02db-f4e7-4432-941c-5448fd71f3d9\") " pod="openshift-multus/multus-hx657" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.911560 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/17265a68-bcb5-455c-b7df-c148a6517db9-os-release\") pod \"multus-additional-cni-plugins-2lrql\" (UID: \"17265a68-bcb5-455c-b7df-c148a6517db9\") " pod="openshift-multus/multus-additional-cni-plugins-2lrql" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.911576 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/c30c02db-f4e7-4432-941c-5448fd71f3d9-host-run-multus-certs\") pod \"multus-hx657\" (UID: \"c30c02db-f4e7-4432-941c-5448fd71f3d9\") " pod="openshift-multus/multus-hx657" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.911591 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-x92h2\" (UniqueName: \"kubernetes.io/projected/bc569bba-c020-40d6-af68-e617625fe0e4-kube-api-access-x92h2\") pod \"ovnkube-node-kg55h\" (UID: \"bc569bba-c020-40d6-af68-e617625fe0e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-kg55h" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.911605 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/c30c02db-f4e7-4432-941c-5448fd71f3d9-host-var-lib-cni-multus\") pod \"multus-hx657\" (UID: \"c30c02db-f4e7-4432-941c-5448fd71f3d9\") " pod="openshift-multus/multus-hx657" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.911621 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c30c02db-f4e7-4432-941c-5448fd71f3d9-multus-cni-dir\") pod \"multus-hx657\" (UID: \"c30c02db-f4e7-4432-941c-5448fd71f3d9\") " pod="openshift-multus/multus-hx657" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.911637 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-9vkjv\" (UniqueName: \"kubernetes.io/projected/10c237bc-72e7-4da3-9f2c-b1688395bf6e-kube-api-access-9vkjv\") pod \"machine-config-daemon-74bj6\" (UID: \"10c237bc-72e7-4da3-9f2c-b1688395bf6e\") " pod="openshift-machine-config-operator/machine-config-daemon-74bj6" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.911651 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bc569bba-c020-40d6-af68-e617625fe0e4-var-lib-openvswitch\") pod \"ovnkube-node-kg55h\" (UID: \"bc569bba-c020-40d6-af68-e617625fe0e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-kg55h" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.911665 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/bc569bba-c020-40d6-af68-e617625fe0e4-env-overrides\") pod \"ovnkube-node-kg55h\" (UID: \"bc569bba-c020-40d6-af68-e617625fe0e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-kg55h" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.911699 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/17265a68-bcb5-455c-b7df-c148a6517db9-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-2lrql\" (UID: \"17265a68-bcb5-455c-b7df-c148a6517db9\") " pod="openshift-multus/multus-additional-cni-plugins-2lrql" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.911716 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"host-etc-kube\" (UniqueName: \"kubernetes.io/host-path/37a5e44f-9a88-4405-be8a-b645485e7312-host-etc-kube\") pod \"network-operator-58b4c7f79c-55gtf\" (UID: \"37a5e44f-9a88-4405-be8a-b645485e7312\") " pod="openshift-network-operator/network-operator-58b4c7f79c-55gtf" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.911747 5052 reconciler_common.go:293] "Volume detached for volume \"tmpfs\" (UniqueName: \"kubernetes.io/empty-dir/308be0ea-9f5f-4b29-aeb1-5abd31a0b17b-tmpfs\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.911757 5052 reconciler_common.go:293] "Volume detached for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7bb08738-c794-4ee8-9972-3a62ca171029-cni-binary-copy\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.911767 5052 reconciler_common.go:293] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8f668bae-612b-4b75-9490-919e737c6a3b-registry-certificates\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.911780 5052 reconciler_common.go:293] "Volume detached for volume \"client-ca\" (UniqueName: \"kubernetes.io/configmap/5441d097-087c-4d9a-baa8-b210afa90fc9-client-ca\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.911790 5052 reconciler_common.go:293] "Volume detached for volume \"etcd-client\" (UniqueName: \"kubernetes.io/secret/1bf7eb37-55a3-4c65-b768-a94c82151e69-etcd-client\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.911799 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-d4lsv\" (UniqueName: \"kubernetes.io/projected/25e176fe-21b4-4974-b1ed-c8b94f112a7f-kube-api-access-d4lsv\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.911811 5052 reconciler_common.go:293] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1bf7eb37-55a3-4c65-b768-a94c82151e69-trusted-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.911823 5052 reconciler_common.go:293] "Volume detached for volume \"v4-0-config-system-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/49c341d1-5089-4bc2-86a0-a5e165cfcc6b-v4-0-config-system-trusted-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.911835 5052 reconciler_common.go:293] "Volume detached for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/5225d0e4-402f-4861-b410-819f433b1803-utilities\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.911850 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-mg5zb\" (UniqueName: \"kubernetes.io/projected/6402fda4-df10-493c-b4e5-d0569419652d-kube-api-access-mg5zb\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.911862 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1bf7eb37-55a3-4c65-b768-a94c82151e69-config\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.911874 5052 reconciler_common.go:293] "Volume detached for volume \"profile-collector-cert\" (UniqueName: \"kubernetes.io/secret/f88749ec-7931-4ee7-b3fc-1ec5e11f92e9-profile-collector-cert\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.911886 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-9xfj7\" (UniqueName: \"kubernetes.io/projected/5225d0e4-402f-4861-b410-819f433b1803-kube-api-access-9xfj7\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.911874 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bc569bba-c020-40d6-af68-e617625fe0e4-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-kg55h\" (UID: \"bc569bba-c020-40d6-af68-e617625fe0e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-kg55h" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.911897 5052 reconciler_common.go:293] "Volume detached for volume \"v4-0-config-system-cliconfig\" (UniqueName: \"kubernetes.io/configmap/49c341d1-5089-4bc2-86a0-a5e165cfcc6b-v4-0-config-system-cliconfig\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.911939 5052 reconciler_common.go:293] "Volume detached for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1bf7eb37-55a3-4c65-b768-a94c82151e69-serving-cert\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.911954 5052 reconciler_common.go:293] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8f668bae-612b-4b75-9490-919e737c6a3b-ca-trust-extracted\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.911962 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/17265a68-bcb5-455c-b7df-c148a6517db9-cnibin\") pod \"multus-additional-cni-plugins-2lrql\" (UID: \"17265a68-bcb5-455c-b7df-c148a6517db9\") " pod="openshift-multus/multus-additional-cni-plugins-2lrql" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.911970 5052 reconciler_common.go:293] "Volume detached for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/57a731c4-ef35-47a8-b875-bfb08a7f8011-catalog-content\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.911991 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/c30c02db-f4e7-4432-941c-5448fd71f3d9-multus-socket-dir-parent\") pod \"multus-hx657\" (UID: \"c30c02db-f4e7-4432-941c-5448fd71f3d9\") " pod="openshift-multus/multus-hx657" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.912016 5052 reconciler_common.go:293] "Volume detached for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/1d611f23-29be-4491-8495-bee1670e935f-catalog-content\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.912035 5052 reconciler_common.go:293] "Volume detached for volume \"webhook-certs\" (UniqueName: \"kubernetes.io/secret/efdd0498-1daa-4136-9a4a-3b948c2293fc-webhook-certs\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.912048 5052 reconciler_common.go:293] "Volume detached for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/5225d0e4-402f-4861-b410-819f433b1803-catalog-content\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.912061 5052 reconciler_common.go:293] "Volume detached for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1386a44e-36a2-460c-96d0-0359d2b6f0f5-serving-cert\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.912065 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c30c02db-f4e7-4432-941c-5448fd71f3d9-host-var-lib-cni-bin\") pod \"multus-hx657\" (UID: \"c30c02db-f4e7-4432-941c-5448fd71f3d9\") " pod="openshift-multus/multus-hx657" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.912076 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-pj782\" (UniqueName: \"kubernetes.io/projected/b6cd30de-2eeb-49a2-ab40-9167f4560ff5-kube-api-access-pj782\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.912095 5052 reconciler_common.go:293] "Volume detached for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d-catalog-content\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.912104 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c30c02db-f4e7-4432-941c-5448fd71f3d9-etc-kubernetes\") pod \"multus-hx657\" (UID: \"c30c02db-f4e7-4432-941c-5448fd71f3d9\") " pod="openshift-multus/multus-hx657" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.912145 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"host-etc-kube\" (UniqueName: \"kubernetes.io/host-path/37a5e44f-9a88-4405-be8a-b645485e7312-host-etc-kube\") pod \"network-operator-58b4c7f79c-55gtf\" (UID: \"37a5e44f-9a88-4405-be8a-b645485e7312\") " pod="openshift-network-operator/network-operator-58b4c7f79c-55gtf" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.912131 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"rootfs\" (UniqueName: \"kubernetes.io/host-path/10c237bc-72e7-4da3-9f2c-b1688395bf6e-rootfs\") pod \"machine-config-daemon-74bj6\" (UID: \"10c237bc-72e7-4da3-9f2c-b1688395bf6e\") " pod="openshift-machine-config-operator/machine-config-daemon-74bj6" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.912176 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/bc569bba-c020-40d6-af68-e617625fe0e4-host-kubelet\") pod \"ovnkube-node-kg55h\" (UID: \"bc569bba-c020-40d6-af68-e617625fe0e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-kg55h" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.912207 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bc569bba-c020-40d6-af68-e617625fe0e4-host-cni-bin\") pod \"ovnkube-node-kg55h\" (UID: \"bc569bba-c020-40d6-af68-e617625fe0e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-kg55h" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.912233 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c30c02db-f4e7-4432-941c-5448fd71f3d9-system-cni-dir\") pod \"multus-hx657\" (UID: \"c30c02db-f4e7-4432-941c-5448fd71f3d9\") " pod="openshift-multus/multus-hx657" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.912254 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c30c02db-f4e7-4432-941c-5448fd71f3d9-multus-conf-dir\") pod \"multus-hx657\" (UID: \"c30c02db-f4e7-4432-941c-5448fd71f3d9\") " pod="openshift-multus/multus-hx657" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.913081 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"mcd-auth-proxy-config\" (UniqueName: \"kubernetes.io/configmap/10c237bc-72e7-4da3-9f2c-b1688395bf6e-mcd-auth-proxy-config\") pod \"machine-config-daemon-74bj6\" (UID: \"10c237bc-72e7-4da3-9f2c-b1688395bf6e\") " pod="openshift-machine-config-operator/machine-config-daemon-74bj6" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.913111 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bc569bba-c020-40d6-af68-e617625fe0e4-host-slash\") pod \"ovnkube-node-kg55h\" (UID: \"bc569bba-c020-40d6-af68-e617625fe0e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-kg55h" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.913284 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/bc569bba-c020-40d6-af68-e617625fe0e4-ovnkube-script-lib\") pod \"ovnkube-node-kg55h\" (UID: \"bc569bba-c020-40d6-af68-e617625fe0e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-kg55h" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.913324 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bc569bba-c020-40d6-af68-e617625fe0e4-host-run-netns\") pod \"ovnkube-node-kg55h\" (UID: \"bc569bba-c020-40d6-af68-e617625fe0e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-kg55h" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.913351 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/bc569bba-c020-40d6-af68-e617625fe0e4-node-log\") pod \"ovnkube-node-kg55h\" (UID: \"bc569bba-c020-40d6-af68-e617625fe0e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-kg55h" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.913385 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/bc569bba-c020-40d6-af68-e617625fe0e4-run-systemd\") pod \"ovnkube-node-kg55h\" (UID: \"bc569bba-c020-40d6-af68-e617625fe0e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-kg55h" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.913726 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/bc569bba-c020-40d6-af68-e617625fe0e4-ovnkube-config\") pod \"ovnkube-node-kg55h\" (UID: \"bc569bba-c020-40d6-af68-e617625fe0e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-kg55h" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.913760 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/c30c02db-f4e7-4432-941c-5448fd71f3d9-hostroot\") pod \"multus-hx657\" (UID: \"c30c02db-f4e7-4432-941c-5448fd71f3d9\") " pod="openshift-multus/multus-hx657" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.913792 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bc569bba-c020-40d6-af68-e617625fe0e4-run-openvswitch\") pod \"ovnkube-node-kg55h\" (UID: \"bc569bba-c020-40d6-af68-e617625fe0e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-kg55h" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.913817 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c30c02db-f4e7-4432-941c-5448fd71f3d9-host-var-lib-kubelet\") pod \"multus-hx657\" (UID: \"c30c02db-f4e7-4432-941c-5448fd71f3d9\") " pod="openshift-multus/multus-hx657" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.913846 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bc569bba-c020-40d6-af68-e617625fe0e4-host-run-ovn-kubernetes\") pod \"ovnkube-node-kg55h\" (UID: \"bc569bba-c020-40d6-af68-e617625fe0e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-kg55h" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.913872 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/bc569bba-c020-40d6-af68-e617625fe0e4-host-cni-netd\") pod \"ovnkube-node-kg55h\" (UID: \"bc569bba-c020-40d6-af68-e617625fe0e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-kg55h" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.914015 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c30c02db-f4e7-4432-941c-5448fd71f3d9-cni-binary-copy\") pod \"multus-hx657\" (UID: \"c30c02db-f4e7-4432-941c-5448fd71f3d9\") " pod="openshift-multus/multus-hx657" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.914059 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c30c02db-f4e7-4432-941c-5448fd71f3d9-cnibin\") pod \"multus-hx657\" (UID: \"c30c02db-f4e7-4432-941c-5448fd71f3d9\") " pod="openshift-multus/multus-hx657" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.914086 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c30c02db-f4e7-4432-941c-5448fd71f3d9-host-run-netns\") pod \"multus-hx657\" (UID: \"c30c02db-f4e7-4432-941c-5448fd71f3d9\") " pod="openshift-multus/multus-hx657" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.914107 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/bc569bba-c020-40d6-af68-e617625fe0e4-systemd-units\") pod \"ovnkube-node-kg55h\" (UID: \"bc569bba-c020-40d6-af68-e617625fe0e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-kg55h" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.914126 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/bc569bba-c020-40d6-af68-e617625fe0e4-log-socket\") pod \"ovnkube-node-kg55h\" (UID: \"bc569bba-c020-40d6-af68-e617625fe0e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-kg55h" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.914315 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e5c587e6-866a-4427-8a2c-3469cf0fc495-hosts-file\") pod \"node-resolver-xph69\" (UID: \"e5c587e6-866a-4427-8a2c-3469cf0fc495\") " pod="openshift-dns/node-resolver-xph69" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.914357 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/c30c02db-f4e7-4432-941c-5448fd71f3d9-host-run-k8s-cni-cncf-io\") pod \"multus-hx657\" (UID: \"c30c02db-f4e7-4432-941c-5448fd71f3d9\") " pod="openshift-multus/multus-hx657" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.914390 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/bc569bba-c020-40d6-af68-e617625fe0e4-run-ovn\") pod \"ovnkube-node-kg55h\" (UID: \"bc569bba-c020-40d6-af68-e617625fe0e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-kg55h" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.914414 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bc569bba-c020-40d6-af68-e617625fe0e4-etc-openvswitch\") pod \"ovnkube-node-kg55h\" (UID: \"bc569bba-c020-40d6-af68-e617625fe0e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-kg55h" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.914437 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/c30c02db-f4e7-4432-941c-5448fd71f3d9-host-var-lib-cni-multus\") pod \"multus-hx657\" (UID: \"c30c02db-f4e7-4432-941c-5448fd71f3d9\") " pod="openshift-multus/multus-hx657" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.914467 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d75a4c96-2883-4a0b-bab2-0fab2b6c0b49-host-slash\") pod \"iptables-alerter-4ln5h\" (UID: \"d75a4c96-2883-4a0b-bab2-0fab2b6c0b49\") " pod="openshift-network-operator/iptables-alerter-4ln5h" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.914583 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/c30c02db-f4e7-4432-941c-5448fd71f3d9-host-run-multus-certs\") pod \"multus-hx657\" (UID: \"c30c02db-f4e7-4432-941c-5448fd71f3d9\") " pod="openshift-multus/multus-hx657" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.914666 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/17265a68-bcb5-455c-b7df-c148a6517db9-tuning-conf-dir\") pod \"multus-additional-cni-plugins-2lrql\" (UID: \"17265a68-bcb5-455c-b7df-c148a6517db9\") " pod="openshift-multus/multus-additional-cni-plugins-2lrql" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.914669 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/17265a68-bcb5-455c-b7df-c148a6517db9-os-release\") pod \"multus-additional-cni-plugins-2lrql\" (UID: \"17265a68-bcb5-455c-b7df-c148a6517db9\") " pod="openshift-multus/multus-additional-cni-plugins-2lrql" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.914700 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/17265a68-bcb5-455c-b7df-c148a6517db9-system-cni-dir\") pod \"multus-additional-cni-plugins-2lrql\" (UID: \"17265a68-bcb5-455c-b7df-c148a6517db9\") " pod="openshift-multus/multus-additional-cni-plugins-2lrql" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.914719 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/17265a68-bcb5-455c-b7df-c148a6517db9-cni-binary-copy\") pod \"multus-additional-cni-plugins-2lrql\" (UID: \"17265a68-bcb5-455c-b7df-c148a6517db9\") " pod="openshift-multus/multus-additional-cni-plugins-2lrql" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.914724 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bc569bba-c020-40d6-af68-e617625fe0e4-var-lib-openvswitch\") pod \"ovnkube-node-kg55h\" (UID: \"bc569bba-c020-40d6-af68-e617625fe0e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-kg55h" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.914766 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c30c02db-f4e7-4432-941c-5448fd71f3d9-multus-cni-dir\") pod \"multus-hx657\" (UID: \"c30c02db-f4e7-4432-941c-5448fd71f3d9\") " pod="openshift-multus/multus-hx657" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.914954 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c30c02db-f4e7-4432-941c-5448fd71f3d9-os-release\") pod \"multus-hx657\" (UID: \"c30c02db-f4e7-4432-941c-5448fd71f3d9\") " pod="openshift-multus/multus-hx657" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.917697 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/c30c02db-f4e7-4432-941c-5448fd71f3d9-multus-daemon-config\") pod \"multus-hx657\" (UID: \"c30c02db-f4e7-4432-941c-5448fd71f3d9\") " pod="openshift-multus/multus-hx657" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.917770 5052 status_manager.go:875] "Failed to update status for pod" pod="openshift-network-diagnostics/network-check-target-xd92c" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"3b6479f0-333b-4a96-9adf-2099afdc2447\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"message\\\":\\\"containers with unready status: [network-check-target-container]\\\",\\\"reason\\\":\\\"ContainersNotReady\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"message\\\":\\\"containers with unready status: [network-check-target-container]\\\",\\\"reason\\\":\\\"ContainersNotReady\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"ContainersReady\\\"}],\\\"containerStatuses\\\":[{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e1baa38811c04bd8909e01a1f3be7421a1cb99d608d3dc4cf86d95b17de2ab8b\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{\\\"terminated\\\":{\\\"exitCode\\\":137,\\\"finishedAt\\\":null,\\\"message\\\":\\\"The container could not be located when the pod was deleted. The container used to be Running\\\",\\\"reason\\\":\\\"ContainerStatusUnknown\\\",\\\"startedAt\\\":null}},\\\"name\\\":\\\"network-check-target-container\\\",\\\"ready\\\":false,\\\"restartCount\\\":3,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"ContainerCreating\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-cqllr\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}],\\\"podIP\\\":null,\\\"podIPs\\\":null}}\" for pod \"openshift-network-diagnostics\"/\"network-check-target-xd92c\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://127.0.0.1:9743/pod?timeout=10s\": dial tcp 127.0.0.1:9743: connect: connection refused" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.918289 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/bc569bba-c020-40d6-af68-e617625fe0e4-env-overrides\") pod \"ovnkube-node-kg55h\" (UID: \"bc569bba-c020-40d6-af68-e617625fe0e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-kg55h" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.922119 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/bc569bba-c020-40d6-af68-e617625fe0e4-ovn-node-metrics-cert\") pod \"ovnkube-node-kg55h\" (UID: \"bc569bba-c020-40d6-af68-e617625fe0e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-kg55h" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.927125 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/17265a68-bcb5-455c-b7df-c148a6517db9-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-2lrql\" (UID: \"17265a68-bcb5-455c-b7df-c148a6517db9\") " pod="openshift-multus/multus-additional-cni-plugins-2lrql" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.931142 5052 status_manager.go:875] "Failed to update status for pod" pod="openshift-dns/node-resolver-xph69" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"e5c587e6-866a-4427-8a2c-3469cf0fc495\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"Initialized\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"message\\\":\\\"containers with unready status: [dns-node-resolver]\\\",\\\"reason\\\":\\\"ContainersNotReady\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"message\\\":\\\"containers with unready status: [dns-node-resolver]\\\",\\\"reason\\\":\\\"ContainersNotReady\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"ContainersReady\\\"}],\\\"containerStatuses\\\":[{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:35512335ac39aed0f55b7f799f416f4f6445c20c1b19888cf2bb72bb276703f2\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"dns-node-resolver\\\",\\\"ready\\\":false,\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"ContainerCreating\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/etc/hosts\\\",\\\"name\\\":\\\"hosts-file\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-tk7n6\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}],\\\"hostIP\\\":\\\"192.168.126.11\\\",\\\"hostIPs\\\":[{\\\"ip\\\":\\\"192.168.126.11\\\"}],\\\"podIP\\\":\\\"192.168.126.11\\\",\\\"podIPs\\\":[{\\\"ip\\\":\\\"192.168.126.11\\\"}],\\\"startTime\\\":\\\"2025-12-03T11:20:31Z\\\"}}\" for pod \"openshift-dns\"/\"node-resolver-xph69\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://127.0.0.1:9743/pod?timeout=10s\": dial tcp 127.0.0.1:9743: connect: connection refused" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.932037 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/10c237bc-72e7-4da3-9f2c-b1688395bf6e-proxy-tls\") pod \"machine-config-daemon-74bj6\" (UID: \"10c237bc-72e7-4da3-9f2c-b1688395bf6e\") " pod="openshift-machine-config-operator/machine-config-daemon-74bj6" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.933409 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-fhsjl\" (UniqueName: \"kubernetes.io/projected/c30c02db-f4e7-4432-941c-5448fd71f3d9-kube-api-access-fhsjl\") pod \"multus-hx657\" (UID: \"c30c02db-f4e7-4432-941c-5448fd71f3d9\") " pod="openshift-multus/multus-hx657" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.934209 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-x92h2\" (UniqueName: \"kubernetes.io/projected/bc569bba-c020-40d6-af68-e617625fe0e4-kube-api-access-x92h2\") pod \"ovnkube-node-kg55h\" (UID: \"bc569bba-c020-40d6-af68-e617625fe0e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-kg55h" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.935235 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-tk7n6\" (UniqueName: \"kubernetes.io/projected/e5c587e6-866a-4427-8a2c-3469cf0fc495-kube-api-access-tk7n6\") pod \"node-resolver-xph69\" (UID: \"e5c587e6-866a-4427-8a2c-3469cf0fc495\") " pod="openshift-dns/node-resolver-xph69" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.936240 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-w4zf5\" (UniqueName: \"kubernetes.io/projected/17265a68-bcb5-455c-b7df-c148a6517db9-kube-api-access-w4zf5\") pod \"multus-additional-cni-plugins-2lrql\" (UID: \"17265a68-bcb5-455c-b7df-c148a6517db9\") " pod="openshift-multus/multus-additional-cni-plugins-2lrql" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.938936 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/network-operator-58b4c7f79c-55gtf" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.942297 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-9vkjv\" (UniqueName: \"kubernetes.io/projected/10c237bc-72e7-4da3-9f2c-b1688395bf6e-kube-api-access-9vkjv\") pod \"machine-config-daemon-74bj6\" (UID: \"10c237bc-72e7-4da3-9f2c-b1688395bf6e\") " pod="openshift-machine-config-operator/machine-config-daemon-74bj6" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.948592 5052 status_manager.go:875] "Failed to update status for pod" pod="openshift-multus/multus-additional-cni-plugins-2lrql" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"17265a68-bcb5-455c-b7df-c148a6517db9\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"message\\\":\\\"containers with incomplete status: [egress-router-binary-copy cni-plugins bond-cni-plugin routeoverride-cni whereabouts-cni-bincopy whereabouts-cni]\\\",\\\"reason\\\":\\\"ContainersNotInitialized\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"Initialized\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"message\\\":\\\"containers with unready status: [kube-multus-additional-cni-plugins]\\\",\\\"reason\\\":\\\"ContainersNotReady\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"message\\\":\\\"containers with unready status: [kube-multus-additional-cni-plugins]\\\",\\\"reason\\\":\\\"ContainersNotReady\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"ContainersReady\\\"}],\\\"containerStatuses\\\":[{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7eeaee65f2808b819eedb413bdcabb9144e12f0dd97f13fd1afba93a95b67b26\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"kube-multus-additional-cni-plugins\\\",\\\"ready\\\":false,\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"PodInitializing\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-w4zf5\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}],\\\"hostIP\\\":\\\"192.168.126.11\\\",\\\"hostIPs\\\":[{\\\"ip\\\":\\\"192.168.126.11\\\"}],\\\"initContainerStatuses\\\":[{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e40792096b162f0f9ce5f8362f51e5f8dea2c1ce4b1447235388416b5db7708c\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"egress-router-binary-copy\\\",\\\"ready\\\":false,\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"PodInitializing\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/entrypoint\\\",\\\"name\\\":\\\"cni-binary-copy\\\"},{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/os-release\\\",\\\"name\\\":\\\"os-release\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-w4zf5\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:687fddfbb085a1688df312ce4ec8c857df9b2daed8ff4a7ed6163a1154afa2cc\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"cni-plugins\\\",\\\"ready\\\":false,\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"PodInitializing\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/entrypoint\\\",\\\"name\\\":\\\"cni-binary-copy\\\"},{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/os-release\\\",\\\"name\\\":\\\"os-release\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/host/etc/cni/tuning/\\\",\\\"name\\\":\\\"tuning-conf-dir\\\"},{\\\"mountPath\\\":\\\"/sysctls\\\",\\\"name\\\":\\\"cni-sysctl-allowlist\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-w4zf5\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:383f4cceeeaead203bb2327fdd367c64b64d729d7fa93089f249e496fcef0c78\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"bond-cni-plugin\\\",\\\"ready\\\":false,\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"PodInitializing\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/entrypoint\\\",\\\"name\\\":\\\"cni-binary-copy\\\"},{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/os-release\\\",\\\"name\\\":\\\"os-release\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-w4zf5\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f567acb85146b5ed81451ec3e79f2de0c62e28c69b2eeade0abdf5d0c388e7aa\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"routeoverride-cni\\\",\\\"ready\\\":false,\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"PodInitializing\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/entrypoint\\\",\\\"name\\\":\\\"cni-binary-copy\\\"},{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/os-release\\\",\\\"name\\\":\\\"os-release\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-w4zf5\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:98100674616e54319f6713d742fd0c3bdbc84e6e6173e8ccf4a2473a714c2bc4\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"whereabouts-cni-bincopy\\\",\\\"ready\\\":false,\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"PodInitializing\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/entrypoint\\\",\\\"name\\\":\\\"cni-binary-copy\\\"},{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/os-release\\\",\\\"name\\\":\\\"os-release\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-w4zf5\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:98100674616e54319f6713d742fd0c3bdbc84e6e6173e8ccf4a2473a714c2bc4\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"whereabouts-cni\\\",\\\"ready\\\":false,\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"PodInitializing\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/cni/net.d\\\",\\\"name\\\":\\\"system-cni-dir\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-w4zf5\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}],\\\"podIP\\\":\\\"192.168.126.11\\\",\\\"podIPs\\\":[{\\\"ip\\\":\\\"192.168.126.11\\\"}],\\\"startTime\\\":\\\"2025-12-03T11:20:31Z\\\"}}\" for pod \"openshift-multus\"/\"multus-additional-cni-plugins-2lrql\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://127.0.0.1:9743/pod?timeout=10s\": dial tcp 127.0.0.1:9743: connect: connection refused" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.950718 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-4ln5h" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.958042 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-node-identity/network-node-identity-vrzqb" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.963563 5052 status_manager.go:875] "Failed to update status for pod" pod="openshift-multus/multus-hx657" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"c30c02db-f4e7-4432-941c-5448fd71f3d9\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"Initialized\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"message\\\":\\\"containers with unready status: [kube-multus]\\\",\\\"reason\\\":\\\"ContainersNotReady\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"message\\\":\\\"containers with unready status: [kube-multus]\\\",\\\"reason\\\":\\\"ContainersNotReady\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"ContainersReady\\\"}],\\\"containerStatuses\\\":[{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7eeaee65f2808b819eedb413bdcabb9144e12f0dd97f13fd1afba93a95b67b26\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"kube-multus\\\",\\\"ready\\\":false,\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"ContainerCreating\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/entrypoint\\\",\\\"name\\\":\\\"cni-binary-copy\\\"},{\\\"mountPath\\\":\\\"/host/etc/os-release\\\",\\\"name\\\":\\\"os-release\\\"},{\\\"mountPath\\\":\\\"/host/etc/cni/net.d\\\",\\\"name\\\":\\\"system-cni-dir\\\"},{\\\"mountPath\\\":\\\"/host/run/multus/cni/net.d\\\",\\\"name\\\":\\\"multus-cni-dir\\\"},{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/run/multus\\\",\\\"name\\\":\\\"multus-socket-dir-parent\\\"},{\\\"mountPath\\\":\\\"/run/k8s.cni.cncf.io\\\",\\\"name\\\":\\\"host-run-k8s-cni-cncf-io\\\"},{\\\"mountPath\\\":\\\"/run/netns\\\",\\\"name\\\":\\\"host-run-netns\\\"},{\\\"mountPath\\\":\\\"/var/lib/cni/bin\\\",\\\"name\\\":\\\"host-var-lib-cni-bin\\\"},{\\\"mountPath\\\":\\\"/var/lib/cni/multus\\\",\\\"name\\\":\\\"host-var-lib-cni-multus\\\"},{\\\"mountPath\\\":\\\"/var/lib/kubelet\\\",\\\"name\\\":\\\"host-var-lib-kubelet\\\"},{\\\"mountPath\\\":\\\"/hostroot\\\",\\\"name\\\":\\\"hostroot\\\"},{\\\"mountPath\\\":\\\"/etc/cni/multus/net.d\\\",\\\"name\\\":\\\"multus-conf-dir\\\"},{\\\"mountPath\\\":\\\"/etc/cni/net.d/multus.d\\\",\\\"name\\\":\\\"multus-daemon-config\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/etc/cni/multus/certs\\\",\\\"name\\\":\\\"host-run-multus-certs\\\"},{\\\"mountPath\\\":\\\"/etc/kubernetes\\\",\\\"name\\\":\\\"etc-kubernetes\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-fhsjl\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}],\\\"hostIP\\\":\\\"192.168.126.11\\\",\\\"hostIPs\\\":[{\\\"ip\\\":\\\"192.168.126.11\\\"}],\\\"podIP\\\":\\\"192.168.126.11\\\",\\\"podIPs\\\":[{\\\"ip\\\":\\\"192.168.126.11\\\"}],\\\"startTime\\\":\\\"2025-12-03T11:20:31Z\\\"}}\" for pod \"openshift-multus\"/\"multus-hx657\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://127.0.0.1:9743/pod?timeout=10s\": dial tcp 127.0.0.1:9743: connect: connection refused" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.965575 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-xph69" Dec 03 11:20:31 crc kubenswrapper[5052]: W1203 11:20:31.967388 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd75a4c96_2883_4a0b_bab2_0fab2b6c0b49.slice/crio-ea19d4efb14c351de0dec5bbee1220a317d2eb6a0b0043dbc40cc61704064bec WatchSource:0}: Error finding container ea19d4efb14c351de0dec5bbee1220a317d2eb6a0b0043dbc40cc61704064bec: Status 404 returned error can't find the container with id ea19d4efb14c351de0dec5bbee1220a317d2eb6a0b0043dbc40cc61704064bec Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.973172 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.977198 5052 status_manager.go:875] "Failed to update status for pod" pod="openshift-kube-apiserver/kube-apiserver-crc" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"6c8df12a-3d11-4bb9-b398-67091c1918e4\\\"},\\\"status\\\":{\\\"conditions\\\":[{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:11Z\\\",\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:11Z\\\",\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"Initialized\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:09Z\\\",\\\"message\\\":\\\"containers with unready status: [kube-apiserver-check-endpoints]\\\",\\\"reason\\\":\\\"ContainersNotReady\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:09Z\\\",\\\"message\\\":\\\"containers with unready status: [kube-apiserver-check-endpoints]\\\",\\\"reason\\\":\\\"ContainersNotReady\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"ContainersReady\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:09Z\\\",\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"PodScheduled\\\"}],\\\"containerStatuses\\\":[{\\\"containerID\\\":\\\"cri-o://09256543a9aa86da8bec948be040d36b6e4986b9232091d2638bb1fead5740a9\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:06bc35825771aee1220d34720243b89c4ba8a8b335e6de2597126bd791fd90d4\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:06bc35825771aee1220d34720243b89c4ba8a8b335e6de2597126bd791fd90d4\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"kube-apiserver\\\",\\\"ready\\\":true,\\\"restartCount\\\":0,\\\"started\\\":true,\\\"state\\\":{\\\"running\\\":{\\\"startedAt\\\":\\\"2025-12-03T11:20:12Z\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/etc/kubernetes/static-pod-resources\\\",\\\"name\\\":\\\"resource-dir\\\"},{\\\"mountPath\\\":\\\"/etc/kubernetes/static-pod-certs\\\",\\\"name\\\":\\\"cert-dir\\\"},{\\\"mountPath\\\":\\\"/var/log/kube-apiserver\\\",\\\"name\\\":\\\"audit-dir\\\"}]},{\\\"containerID\\\":\\\"cri-o://15abb0d7fd5884519ed5ac454430dbb58f71ed1f459e5ed30f324f70befd243e\\\",\\\"image\\\":\\\"quay.io/crcont/openshift-crc-cluster-kube-apiserver-operator@sha256:9f36dc276e27753fc478274c7f7814a4f8945c987117ee1ea3b8e6355e6d7462\\\",\\\"imageID\\\":\\\"quay.io/crcont/openshift-crc-cluster-kube-apiserver-operator@sha256:9f36dc276e27753fc478274c7f7814a4f8945c987117ee1ea3b8e6355e6d7462\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"kube-apiserver-cert-regeneration-controller\\\",\\\"ready\\\":true,\\\"restartCount\\\":0,\\\"started\\\":true,\\\"state\\\":{\\\"running\\\":{\\\"startedAt\\\":\\\"2025-12-03T11:20:14Z\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/etc/kubernetes/static-pod-resources\\\",\\\"name\\\":\\\"resource-dir\\\"}]},{\\\"containerID\\\":\\\"cri-o://8ceedd21ae3833d77fd440537f30434ee57f087fbefb2357b46231d3f778ad2a\\\",\\\"image\\\":\\\"quay.io/crcont/openshift-crc-cluster-kube-apiserver-operator@sha256:9f36dc276e27753fc478274c7f7814a4f8945c987117ee1ea3b8e6355e6d7462\\\",\\\"imageID\\\":\\\"quay.io/crcont/openshift-crc-cluster-kube-apiserver-operator@sha256:9f36dc276e27753fc478274c7f7814a4f8945c987117ee1ea3b8e6355e6d7462\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"kube-apiserver-cert-syncer\\\",\\\"ready\\\":true,\\\"restartCount\\\":0,\\\"started\\\":true,\\\"state\\\":{\\\"running\\\":{\\\"startedAt\\\":\\\"2025-12-03T11:20:12Z\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/etc/kubernetes/static-pod-resources\\\",\\\"name\\\":\\\"resource-dir\\\"},{\\\"mountPath\\\":\\\"/etc/kubernetes/static-pod-certs\\\",\\\"name\\\":\\\"cert-dir\\\"}]},{\\\"containerID\\\":\\\"cri-o://b739f84524d09b27f6e052e21df8be3870a2136665704232f5a365d8a5bf41f5\\\",\\\"image\\\":\\\"quay.io/crcont/openshift-crc-cluster-kube-apiserver-operator@sha256:9f36dc276e27753fc478274c7f7814a4f8945c987117ee1ea3b8e6355e6d7462\\\",\\\"imageID\\\":\\\"quay.io/crcont/openshift-crc-cluster-kube-apiserver-operator@sha256:9f36dc276e27753fc478274c7f7814a4f8945c987117ee1ea3b8e6355e6d7462\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"kube-apiserver-check-endpoints\\\",\\\"ready\\\":false,\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://b739f84524d09b27f6e052e21df8be3870a2136665704232f5a365d8a5bf41f5\\\",\\\"exitCode\\\":255,\\\"finishedAt\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"message\\\":\\\"ing back to namespace): Get \\\\\\\"https://localhost:6443/api/v1/namespaces/openshift-kube-apiserver/pods/kube-apiserver-crc\\\\\\\": net/http: TLS handshake timeout\\\\nI1203 11:20:25.911385 1 builder.go:304] check-endpoints version 4.18.0-202502101302.p0.g763313c.assembly.stream.el9-763313c-763313c860ea43fcfc9b1ac00ebae096b57c078e\\\\nI1203 11:20:25.912713 1 dynamic_serving_content.go:116] \\\\\\\"Loaded a new cert/key pair\\\\\\\" name=\\\\\\\"serving-cert::/tmp/serving-cert-3310779835/tls.crt::/tmp/serving-cert-3310779835/tls.key\\\\\\\"\\\\nI1203 11:20:31.324803 1 requestheader_controller.go:247] Loaded a new request header values for RequestHeaderAuthRequestController\\\\nI1203 11:20:31.327725 1 maxinflight.go:139] \\\\\\\"Initialized nonMutatingChan\\\\\\\" len=400\\\\nI1203 11:20:31.327838 1 maxinflight.go:145] \\\\\\\"Initialized mutatingChan\\\\\\\" len=200\\\\nI1203 11:20:31.327886 1 maxinflight.go:116] \\\\\\\"Set denominator for readonly requests\\\\\\\" limit=400\\\\nI1203 11:20:31.327915 1 maxinflight.go:120] \\\\\\\"Set denominator for mutating requests\\\\\\\" limit=200\\\\nI1203 11:20:31.335573 1 secure_serving.go:57] Forcing use of http/1.1 only\\\\nW1203 11:20:31.335607 1 secure_serving.go:69] Use of insecure cipher 'TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256' detected.\\\\nW1203 11:20:31.335613 1 secure_serving.go:69] Use of insecure cipher 'TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256' detected.\\\\nW1203 11:20:31.335619 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_128_GCM_SHA256' detected.\\\\nW1203 11:20:31.335623 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_256_GCM_SHA384' detected.\\\\nW1203 11:20:31.335626 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_128_CBC_SHA' detected.\\\\nW1203 11:20:31.335630 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_256_CBC_SHA' detected.\\\\nI1203 11:20:31.335893 1 genericapiserver.go:533] MuxAndDiscoveryComplete has all endpoints registered and discovery information is complete\\\\nF1203 11:20:31.338035 1 cmd.go:182] pods \\\\\\\"kube-apiserver-crc\\\\\\\" not found\\\\n\\\",\\\"reason\\\":\\\"Error\\\",\\\"startedAt\\\":\\\"2025-12-03T11:20:15Z\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/etc/kubernetes/static-pod-resources\\\",\\\"name\\\":\\\"resource-dir\\\"},{\\\"mountPath\\\":\\\"/etc/kubernetes/static-pod-certs\\\",\\\"name\\\":\\\"cert-dir\\\"}]},{\\\"containerID\\\":\\\"cri-o://22b8cb6aa04337d1420f97d3e128fce2b29a725ca6e5d157adeb1ba78f00073f\\\",\\\"image\\\":\\\"quay.io/crcont/openshift-crc-cluster-kube-apiserver-operator@sha256:9f36dc276e27753fc478274c7f7814a4f8945c987117ee1ea3b8e6355e6d7462\\\",\\\"imageID\\\":\\\"quay.io/crcont/openshift-crc-cluster-kube-apiserver-operator@sha256:9f36dc276e27753fc478274c7f7814a4f8945c987117ee1ea3b8e6355e6d7462\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"kube-apiserver-insecure-readyz\\\",\\\"ready\\\":true,\\\"restartCount\\\":0,\\\"started\\\":true,\\\"state\\\":{\\\"running\\\":{\\\"startedAt\\\":\\\"2025-12-03T11:20:15Z\\\"}}}],\\\"hostIP\\\":\\\"192.168.126.11\\\",\\\"hostIPs\\\":[{\\\"ip\\\":\\\"192.168.126.11\\\"}],\\\"initContainerStatuses\\\":[{\\\"containerID\\\":\\\"cri-o://7182c75773f069c72f87c39136222fd4a91b49add27352a1969645413e670f01\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:06bc35825771aee1220d34720243b89c4ba8a8b335e6de2597126bd791fd90d4\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:06bc35825771aee1220d34720243b89c4ba8a8b335e6de2597126bd791fd90d4\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"setup\\\",\\\"ready\\\":true,\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://7182c75773f069c72f87c39136222fd4a91b49add27352a1969645413e670f01\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2025-12-03T11:20:11Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2025-12-03T11:20:11Z\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/var/log/kube-apiserver\\\",\\\"name\\\":\\\"audit-dir\\\"}]}],\\\"phase\\\":\\\"Running\\\",\\\"podIP\\\":\\\"192.168.126.11\\\",\\\"podIPs\\\":[{\\\"ip\\\":\\\"192.168.126.11\\\"}],\\\"startTime\\\":\\\"2025-12-03T11:20:09Z\\\"}}\" for pod \"openshift-kube-apiserver\"/\"kube-apiserver-crc\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://127.0.0.1:9743/pod?timeout=10s\": dial tcp 127.0.0.1:9743: connect: connection refused" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.983066 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-2lrql" Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.989021 5052 status_manager.go:875] "Failed to update status for pod" pod="openshift-network-console/networking-console-plugin-85b44fc459-gdk6g" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"5fe485a1-e14f-4c09-b5b9-f252bc42b7e8\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"message\\\":\\\"containers with unready status: [networking-console-plugin]\\\",\\\"reason\\\":\\\"ContainersNotReady\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"message\\\":\\\"containers with unready status: [networking-console-plugin]\\\",\\\"reason\\\":\\\"ContainersNotReady\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"ContainersReady\\\"}],\\\"containerStatuses\\\":[{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ae647598ec35cda5766806d3d44a91e3b9d4dee48ff154f3d8490165399873fd\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{\\\"terminated\\\":{\\\"exitCode\\\":137,\\\"finishedAt\\\":null,\\\"message\\\":\\\"The container could not be located when the pod was deleted. The container used to be Running\\\",\\\"reason\\\":\\\"ContainerStatusUnknown\\\",\\\"startedAt\\\":null}},\\\"name\\\":\\\"networking-console-plugin\\\",\\\"ready\\\":false,\\\"restartCount\\\":3,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"ContainerCreating\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/var/cert\\\",\\\"name\\\":\\\"networking-console-plugin-cert\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/etc/nginx/nginx.conf\\\",\\\"name\\\":\\\"nginx-conf\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}],\\\"podIP\\\":null,\\\"podIPs\\\":null}}\" for pod \"openshift-network-console\"/\"networking-console-plugin-85b44fc459-gdk6g\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://127.0.0.1:9743/pod?timeout=10s\": dial tcp 127.0.0.1:9743: connect: connection refused" Dec 03 11:20:31 crc kubenswrapper[5052]: W1203 11:20:31.990986 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef543e1b_8068_4ea3_b32a_61027b32e95d.slice/crio-30b27fe40764d373dce5c76f374c2270b94074453439cad92d16e85720a0819d WatchSource:0}: Error finding container 30b27fe40764d373dce5c76f374c2270b94074453439cad92d16e85720a0819d: Status 404 returned error can't find the container with id 30b27fe40764d373dce5c76f374c2270b94074453439cad92d16e85720a0819d Dec 03 11:20:31 crc kubenswrapper[5052]: I1203 11:20:31.993128 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-hx657" Dec 03 11:20:31 crc kubenswrapper[5052]: W1203 11:20:31.994885 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5c587e6_866a_4427_8a2c_3469cf0fc495.slice/crio-761a0a8204c9a90e4082321610aa71863eccdcad216f22f8935ee82d9185a59e WatchSource:0}: Error finding container 761a0a8204c9a90e4082321610aa71863eccdcad216f22f8935ee82d9185a59e: Status 404 returned error can't find the container with id 761a0a8204c9a90e4082321610aa71863eccdcad216f22f8935ee82d9185a59e Dec 03 11:20:32 crc kubenswrapper[5052]: I1203 11:20:32.004272 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-kg55h" Dec 03 11:20:32 crc kubenswrapper[5052]: I1203 11:20:32.006613 5052 status_manager.go:875] "Failed to update status for pod" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"10c237bc-72e7-4da3-9f2c-b1688395bf6e\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"Initialized\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"message\\\":\\\"containers with unready status: [machine-config-daemon kube-rbac-proxy]\\\",\\\"reason\\\":\\\"ContainersNotReady\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"message\\\":\\\"containers with unready status: [machine-config-daemon kube-rbac-proxy]\\\",\\\"reason\\\":\\\"ContainersNotReady\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"ContainersReady\\\"}],\\\"containerStatuses\\\":[{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:242b3d66438c42745f4ef318bdeaf3d793426f12962a42ea83e18d06c08aaf09\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"kube-rbac-proxy\\\",\\\"ready\\\":false,\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"ContainerCreating\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/etc/tls/private\\\",\\\"name\\\":\\\"proxy-tls\\\"},{\\\"mountPath\\\":\\\"/etc/kube-rbac-proxy\\\",\\\"name\\\":\\\"mcd-auth-proxy-config\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-9vkjv\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c915fb8ba96e911699a1ae34a8e95ca8a9fbe1bf8c28fea177225c63a8bdfc0a\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"machine-config-daemon\\\",\\\"ready\\\":false,\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"ContainerCreating\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/rootfs\\\",\\\"name\\\":\\\"rootfs\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-9vkjv\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}],\\\"hostIP\\\":\\\"192.168.126.11\\\",\\\"hostIPs\\\":[{\\\"ip\\\":\\\"192.168.126.11\\\"}],\\\"podIP\\\":\\\"192.168.126.11\\\",\\\"podIPs\\\":[{\\\"ip\\\":\\\"192.168.126.11\\\"}],\\\"startTime\\\":\\\"2025-12-03T11:20:31Z\\\"}}\" for pod \"openshift-machine-config-operator\"/\"machine-config-daemon-74bj6\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://127.0.0.1:9743/pod?timeout=10s\": dial tcp 127.0.0.1:9743: connect: connection refused" Dec 03 11:20:32 crc kubenswrapper[5052]: I1203 11:20:32.019478 5052 status_manager.go:875] "Failed to update status for pod" pod="openshift-network-operator/network-operator-58b4c7f79c-55gtf" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"37a5e44f-9a88-4405-be8a-b645485e7312\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"message\\\":\\\"containers with unready status: [network-operator]\\\",\\\"reason\\\":\\\"ContainersNotReady\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"message\\\":\\\"containers with unready status: [network-operator]\\\",\\\"reason\\\":\\\"ContainersNotReady\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"ContainersReady\\\"}],\\\"containerStatuses\\\":[{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e1baa38811c04bd8909e01a1f3be7421a1cb99d608d3dc4cf86d95b17de2ab8b\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{\\\"terminated\\\":{\\\"exitCode\\\":137,\\\"finishedAt\\\":null,\\\"message\\\":\\\"The container could not be located when the pod was deleted. The container used to be Running\\\",\\\"reason\\\":\\\"ContainerStatusUnknown\\\",\\\"startedAt\\\":null}},\\\"name\\\":\\\"network-operator\\\",\\\"ready\\\":false,\\\"restartCount\\\":5,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"ContainerCreating\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/etc/kubernetes\\\",\\\"name\\\":\\\"host-etc-kube\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/serving-cert\\\",\\\"name\\\":\\\"metrics-tls\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-rdwmf\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}]}}\" for pod \"openshift-network-operator\"/\"network-operator-58b4c7f79c-55gtf\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://127.0.0.1:9743/pod?timeout=10s\": dial tcp 127.0.0.1:9743: connect: connection refused" Dec 03 11:20:32 crc kubenswrapper[5052]: W1203 11:20:32.020150 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10c237bc_72e7_4da3_9f2c_b1688395bf6e.slice/crio-c20b30bf9e037ac3ae8754c6cd2b01b620e952ed0834eb65171ab3a6cb32e43d WatchSource:0}: Error finding container c20b30bf9e037ac3ae8754c6cd2b01b620e952ed0834eb65171ab3a6cb32e43d: Status 404 returned error can't find the container with id c20b30bf9e037ac3ae8754c6cd2b01b620e952ed0834eb65171ab3a6cb32e43d Dec 03 11:20:32 crc kubenswrapper[5052]: W1203 11:20:32.040423 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17265a68_bcb5_455c_b7df_c148a6517db9.slice/crio-1efd6de654cc021dae19ff2b6880d6ca41f569617f2a48119c5594a6dc7ab7a8 WatchSource:0}: Error finding container 1efd6de654cc021dae19ff2b6880d6ca41f569617f2a48119c5594a6dc7ab7a8: Status 404 returned error can't find the container with id 1efd6de654cc021dae19ff2b6880d6ca41f569617f2a48119c5594a6dc7ab7a8 Dec 03 11:20:32 crc kubenswrapper[5052]: I1203 11:20:32.044084 5052 status_manager.go:875] "Failed to update status for pod" pod="openshift-network-node-identity/network-node-identity-vrzqb" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"ef543e1b-8068-4ea3-b32a-61027b32e95d\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"message\\\":\\\"containers with unready status: [webhook approver]\\\",\\\"reason\\\":\\\"ContainersNotReady\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"message\\\":\\\"containers with unready status: [webhook approver]\\\",\\\"reason\\\":\\\"ContainersNotReady\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"ContainersReady\\\"}],\\\"containerStatuses\\\":[{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:174f36cdd47ef0d1d2099482919d773257453265a2af0b17b154edc32fa41ac2\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{\\\"terminated\\\":{\\\"exitCode\\\":137,\\\"finishedAt\\\":null,\\\"message\\\":\\\"The container could not be located when the pod was deleted. The container used to be Running\\\",\\\"reason\\\":\\\"ContainerStatusUnknown\\\",\\\"startedAt\\\":null}},\\\"name\\\":\\\"approver\\\",\\\"ready\\\":false,\\\"restartCount\\\":6,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"ContainerCreating\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/env\\\",\\\"name\\\":\\\"env-overrides\\\"},{\\\"mountPath\\\":\\\"/var/run/ovnkube-identity-config\\\",\\\"name\\\":\\\"ovnkube-identity-cm\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-s2kz5\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:174f36cdd47ef0d1d2099482919d773257453265a2af0b17b154edc32fa41ac2\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{\\\"terminated\\\":{\\\"exitCode\\\":137,\\\"finishedAt\\\":null,\\\"message\\\":\\\"The container could not be located when the pod was deleted. The container used to be Running\\\",\\\"reason\\\":\\\"ContainerStatusUnknown\\\",\\\"startedAt\\\":null}},\\\"name\\\":\\\"webhook\\\",\\\"ready\\\":false,\\\"restartCount\\\":3,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"ContainerCreating\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/etc/webhook-cert/\\\",\\\"name\\\":\\\"webhook-cert\\\"},{\\\"mountPath\\\":\\\"/env\\\",\\\"name\\\":\\\"env-overrides\\\"},{\\\"mountPath\\\":\\\"/var/run/ovnkube-identity-config\\\",\\\"name\\\":\\\"ovnkube-identity-cm\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-s2kz5\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}]}}\" for pod \"openshift-network-node-identity\"/\"network-node-identity-vrzqb\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://127.0.0.1:9743/pod?timeout=10s\": dial tcp 127.0.0.1:9743: connect: connection refused" Dec 03 11:20:32 crc kubenswrapper[5052]: W1203 11:20:32.057982 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc30c02db_f4e7_4432_941c_5448fd71f3d9.slice/crio-2559bc144590f35ee37d8b35b2709355307645823f808866b1f7e29e57e15459 WatchSource:0}: Error finding container 2559bc144590f35ee37d8b35b2709355307645823f808866b1f7e29e57e15459: Status 404 returned error can't find the container with id 2559bc144590f35ee37d8b35b2709355307645823f808866b1f7e29e57e15459 Dec 03 11:20:32 crc kubenswrapper[5052]: I1203 11:20:32.321809 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\") pod \"8f668bae-612b-4b75-9490-919e737c6a3b\" (UID: \"8f668bae-612b-4b75-9490-919e737c6a3b\") " Dec 03 11:20:32 crc kubenswrapper[5052]: I1203 11:20:32.321987 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/5fe485a1-e14f-4c09-b5b9-f252bc42b7e8-networking-console-plugin-cert\") pod \"networking-console-plugin-85b44fc459-gdk6g\" (UID: \"5fe485a1-e14f-4c09-b5b9-f252bc42b7e8\") " pod="openshift-network-console/networking-console-plugin-85b44fc459-gdk6g" Dec 03 11:20:32 crc kubenswrapper[5052]: E1203 11:20:32.322083 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8 podName:8f668bae-612b-4b75-9490-919e737c6a3b nodeName:}" failed. No retries permitted until 2025-12-03 11:20:33.322037732 +0000 UTC m=+23.910279496 (durationBeforeRetry 1s). Error: UnmountVolume.TearDown failed for volume "pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8" (UniqueName: "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8") pod "8f668bae-612b-4b75-9490-919e737c6a3b" (UID: "8f668bae-612b-4b75-9490-919e737c6a3b") : kubernetes.io/csi: Unmounter.TearDownAt failed to get CSI client: driver name kubevirt.io.hostpath-provisioner not found in the list of registered CSI drivers Dec 03 11:20:32 crc kubenswrapper[5052]: E1203 11:20:32.322142 5052 secret.go:188] Couldn't get secret openshift-network-console/networking-console-plugin-cert: object "openshift-network-console"/"networking-console-plugin-cert" not registered Dec 03 11:20:32 crc kubenswrapper[5052]: E1203 11:20:32.322217 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5fe485a1-e14f-4c09-b5b9-f252bc42b7e8-networking-console-plugin-cert podName:5fe485a1-e14f-4c09-b5b9-f252bc42b7e8 nodeName:}" failed. No retries permitted until 2025-12-03 11:20:33.322198537 +0000 UTC m=+23.910440351 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/5fe485a1-e14f-4c09-b5b9-f252bc42b7e8-networking-console-plugin-cert") pod "networking-console-plugin-85b44fc459-gdk6g" (UID: "5fe485a1-e14f-4c09-b5b9-f252bc42b7e8") : object "openshift-network-console"/"networking-console-plugin-cert" not registered Dec 03 11:20:32 crc kubenswrapper[5052]: I1203 11:20:32.322270 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-s2dwl\" (UniqueName: \"kubernetes.io/projected/9d751cbb-f2e2-430d-9754-c882a5e924a5-kube-api-access-s2dwl\") pod \"network-check-source-55646444c4-trplf\" (UID: \"9d751cbb-f2e2-430d-9754-c882a5e924a5\") " pod="openshift-network-diagnostics/network-check-source-55646444c4-trplf" Dec 03 11:20:32 crc kubenswrapper[5052]: I1203 11:20:32.322402 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/5fe485a1-e14f-4c09-b5b9-f252bc42b7e8-nginx-conf\") pod \"networking-console-plugin-85b44fc459-gdk6g\" (UID: \"5fe485a1-e14f-4c09-b5b9-f252bc42b7e8\") " pod="openshift-network-console/networking-console-plugin-85b44fc459-gdk6g" Dec 03 11:20:32 crc kubenswrapper[5052]: E1203 11:20:32.322430 5052 projected.go:288] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Dec 03 11:20:32 crc kubenswrapper[5052]: I1203 11:20:32.322439 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-cqllr\" (UniqueName: \"kubernetes.io/projected/3b6479f0-333b-4a96-9adf-2099afdc2447-kube-api-access-cqllr\") pod \"network-check-target-xd92c\" (UID: \"3b6479f0-333b-4a96-9adf-2099afdc2447\") " pod="openshift-network-diagnostics/network-check-target-xd92c" Dec 03 11:20:32 crc kubenswrapper[5052]: E1203 11:20:32.322449 5052 projected.go:288] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Dec 03 11:20:32 crc kubenswrapper[5052]: E1203 11:20:32.322467 5052 projected.go:194] Error preparing data for projected volume kube-api-access-s2dwl for pod openshift-network-diagnostics/network-check-source-55646444c4-trplf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Dec 03 11:20:32 crc kubenswrapper[5052]: E1203 11:20:32.322528 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9d751cbb-f2e2-430d-9754-c882a5e924a5-kube-api-access-s2dwl podName:9d751cbb-f2e2-430d-9754-c882a5e924a5 nodeName:}" failed. No retries permitted until 2025-12-03 11:20:33.322500085 +0000 UTC m=+23.910741849 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-s2dwl" (UniqueName: "kubernetes.io/projected/9d751cbb-f2e2-430d-9754-c882a5e924a5-kube-api-access-s2dwl") pod "network-check-source-55646444c4-trplf" (UID: "9d751cbb-f2e2-430d-9754-c882a5e924a5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Dec 03 11:20:32 crc kubenswrapper[5052]: E1203 11:20:32.322580 5052 configmap.go:193] Couldn't get configMap openshift-network-console/networking-console-plugin: object "openshift-network-console"/"networking-console-plugin" not registered Dec 03 11:20:32 crc kubenswrapper[5052]: E1203 11:20:32.322625 5052 projected.go:288] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Dec 03 11:20:32 crc kubenswrapper[5052]: E1203 11:20:32.322645 5052 projected.go:288] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Dec 03 11:20:32 crc kubenswrapper[5052]: E1203 11:20:32.322656 5052 projected.go:194] Error preparing data for projected volume kube-api-access-cqllr for pod openshift-network-diagnostics/network-check-target-xd92c: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Dec 03 11:20:32 crc kubenswrapper[5052]: E1203 11:20:32.322709 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5fe485a1-e14f-4c09-b5b9-f252bc42b7e8-nginx-conf podName:5fe485a1-e14f-4c09-b5b9-f252bc42b7e8 nodeName:}" failed. No retries permitted until 2025-12-03 11:20:33.32267172 +0000 UTC m=+23.910913484 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "nginx-conf" (UniqueName: "kubernetes.io/configmap/5fe485a1-e14f-4c09-b5b9-f252bc42b7e8-nginx-conf") pod "networking-console-plugin-85b44fc459-gdk6g" (UID: "5fe485a1-e14f-4c09-b5b9-f252bc42b7e8") : object "openshift-network-console"/"networking-console-plugin" not registered Dec 03 11:20:32 crc kubenswrapper[5052]: E1203 11:20:32.322731 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3b6479f0-333b-4a96-9adf-2099afdc2447-kube-api-access-cqllr podName:3b6479f0-333b-4a96-9adf-2099afdc2447 nodeName:}" failed. No retries permitted until 2025-12-03 11:20:33.322723591 +0000 UTC m=+23.910965355 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-cqllr" (UniqueName: "kubernetes.io/projected/3b6479f0-333b-4a96-9adf-2099afdc2447-kube-api-access-cqllr") pod "network-check-target-xd92c" (UID: "3b6479f0-333b-4a96-9adf-2099afdc2447") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Dec 03 11:20:32 crc kubenswrapper[5052]: I1203 11:20:32.672805 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xd92c" Dec 03 11:20:32 crc kubenswrapper[5052]: E1203 11:20:32.672974 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xd92c" podUID="3b6479f0-333b-4a96-9adf-2099afdc2447" Dec 03 11:20:32 crc kubenswrapper[5052]: I1203 11:20:32.824988 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-4ln5h" event={"ID":"d75a4c96-2883-4a0b-bab2-0fab2b6c0b49","Type":"ContainerStarted","Data":"ea19d4efb14c351de0dec5bbee1220a317d2eb6a0b0043dbc40cc61704064bec"} Dec 03 11:20:32 crc kubenswrapper[5052]: I1203 11:20:32.826506 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" event={"ID":"10c237bc-72e7-4da3-9f2c-b1688395bf6e","Type":"ContainerStarted","Data":"1413108bd79cded55a270a3437cda42aab7c3db3cfb81894fade4363bbb28f2d"} Dec 03 11:20:32 crc kubenswrapper[5052]: I1203 11:20:32.826593 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" event={"ID":"10c237bc-72e7-4da3-9f2c-b1688395bf6e","Type":"ContainerStarted","Data":"c20b30bf9e037ac3ae8754c6cd2b01b620e952ed0834eb65171ab3a6cb32e43d"} Dec 03 11:20:32 crc kubenswrapper[5052]: I1203 11:20:32.829262 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-apiserver_kube-apiserver-crc_f4b27818a5e8e43d0dc095d08835c792/kube-apiserver-check-endpoints/0.log" Dec 03 11:20:32 crc kubenswrapper[5052]: I1203 11:20:32.831917 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-kube-apiserver/kube-apiserver-crc" event={"ID":"f4b27818a5e8e43d0dc095d08835c792","Type":"ContainerStarted","Data":"ee9c5c61cfd6dee9fd44ebc4c7ef4d20b5fde82b12a8147c3c871e2615da7940"} Dec 03 11:20:32 crc kubenswrapper[5052]: I1203 11:20:32.833914 5052 generic.go:334] "Generic (PLEG): container finished" podID="bc569bba-c020-40d6-af68-e617625fe0e4" containerID="4def7c1172d2ae52e275b2371cf88c9f93c3f0c747c18de7d1073d50b03b19bc" exitCode=0 Dec 03 11:20:32 crc kubenswrapper[5052]: I1203 11:20:32.833977 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kg55h" event={"ID":"bc569bba-c020-40d6-af68-e617625fe0e4","Type":"ContainerDied","Data":"4def7c1172d2ae52e275b2371cf88c9f93c3f0c747c18de7d1073d50b03b19bc"} Dec 03 11:20:32 crc kubenswrapper[5052]: I1203 11:20:32.834017 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kg55h" event={"ID":"bc569bba-c020-40d6-af68-e617625fe0e4","Type":"ContainerStarted","Data":"08118e254f2650de17c8e49017cae4a774b2c748d9cd9b8b2993b973913c0567"} Dec 03 11:20:32 crc kubenswrapper[5052]: I1203 11:20:32.838303 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2lrql" event={"ID":"17265a68-bcb5-455c-b7df-c148a6517db9","Type":"ContainerStarted","Data":"ce6a0f2a385f3ad9f34f3a09e960700dc4d3941341c08997b95c648c82d11749"} Dec 03 11:20:32 crc kubenswrapper[5052]: I1203 11:20:32.838366 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2lrql" event={"ID":"17265a68-bcb5-455c-b7df-c148a6517db9","Type":"ContainerStarted","Data":"1efd6de654cc021dae19ff2b6880d6ca41f569617f2a48119c5594a6dc7ab7a8"} Dec 03 11:20:32 crc kubenswrapper[5052]: I1203 11:20:32.840246 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-xph69" event={"ID":"e5c587e6-866a-4427-8a2c-3469cf0fc495","Type":"ContainerStarted","Data":"8cbacdbdef2832bba669718580c35080702354f16ec06e7d2680a6ee5b63e8b3"} Dec 03 11:20:32 crc kubenswrapper[5052]: I1203 11:20:32.841044 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-xph69" event={"ID":"e5c587e6-866a-4427-8a2c-3469cf0fc495","Type":"ContainerStarted","Data":"761a0a8204c9a90e4082321610aa71863eccdcad216f22f8935ee82d9185a59e"} Dec 03 11:20:32 crc kubenswrapper[5052]: I1203 11:20:32.842269 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-network-node-identity/network-node-identity-vrzqb" event={"ID":"ef543e1b-8068-4ea3-b32a-61027b32e95d","Type":"ContainerStarted","Data":"13aaf5ec87e48b83ec6508ce67110db3b2800f9fa1ca96fd64160d3761248fdd"} Dec 03 11:20:32 crc kubenswrapper[5052]: I1203 11:20:32.842294 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-network-node-identity/network-node-identity-vrzqb" event={"ID":"ef543e1b-8068-4ea3-b32a-61027b32e95d","Type":"ContainerStarted","Data":"30b27fe40764d373dce5c76f374c2270b94074453439cad92d16e85720a0819d"} Dec 03 11:20:32 crc kubenswrapper[5052]: I1203 11:20:32.844044 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-hx657" event={"ID":"c30c02db-f4e7-4432-941c-5448fd71f3d9","Type":"ContainerStarted","Data":"388fefa2a0e0ba8ca09e3fd39371e442358e9547cc2789ca8fa4b7b7237f6ac5"} Dec 03 11:20:32 crc kubenswrapper[5052]: I1203 11:20:32.844081 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-hx657" event={"ID":"c30c02db-f4e7-4432-941c-5448fd71f3d9","Type":"ContainerStarted","Data":"2559bc144590f35ee37d8b35b2709355307645823f808866b1f7e29e57e15459"} Dec 03 11:20:32 crc kubenswrapper[5052]: I1203 11:20:32.845609 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/network-operator-58b4c7f79c-55gtf" event={"ID":"37a5e44f-9a88-4405-be8a-b645485e7312","Type":"ContainerStarted","Data":"177058541a1de284ec4275037302f876e3345d48400f5edc9466508d43518412"} Dec 03 11:20:32 crc kubenswrapper[5052]: I1203 11:20:32.845643 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/network-operator-58b4c7f79c-55gtf" event={"ID":"37a5e44f-9a88-4405-be8a-b645485e7312","Type":"ContainerStarted","Data":"f089b4078d5dc7da8036731e00c58e3824c27db4dcdcbc095298ab3117d69f83"} Dec 03 11:20:32 crc kubenswrapper[5052]: I1203 11:20:32.854473 5052 status_manager.go:875] "Failed to update status for pod" pod="openshift-network-operator/network-operator-58b4c7f79c-55gtf" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"37a5e44f-9a88-4405-be8a-b645485e7312\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"message\\\":\\\"containers with unready status: [network-operator]\\\",\\\"reason\\\":\\\"ContainersNotReady\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"message\\\":\\\"containers with unready status: [network-operator]\\\",\\\"reason\\\":\\\"ContainersNotReady\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"ContainersReady\\\"}],\\\"containerStatuses\\\":[{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e1baa38811c04bd8909e01a1f3be7421a1cb99d608d3dc4cf86d95b17de2ab8b\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{\\\"terminated\\\":{\\\"exitCode\\\":137,\\\"finishedAt\\\":null,\\\"message\\\":\\\"The container could not be located when the pod was deleted. The container used to be Running\\\",\\\"reason\\\":\\\"ContainerStatusUnknown\\\",\\\"startedAt\\\":null}},\\\"name\\\":\\\"network-operator\\\",\\\"ready\\\":false,\\\"restartCount\\\":5,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"ContainerCreating\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/etc/kubernetes\\\",\\\"name\\\":\\\"host-etc-kube\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/serving-cert\\\",\\\"name\\\":\\\"metrics-tls\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-rdwmf\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}]}}\" for pod \"openshift-network-operator\"/\"network-operator-58b4c7f79c-55gtf\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://127.0.0.1:9743/pod?timeout=10s\": dial tcp 127.0.0.1:9743: connect: connection refused" Dec 03 11:20:32 crc kubenswrapper[5052]: I1203 11:20:32.870153 5052 status_manager.go:875] "Failed to update status for pod" pod="openshift-network-node-identity/network-node-identity-vrzqb" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"ef543e1b-8068-4ea3-b32a-61027b32e95d\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"message\\\":\\\"containers with unready status: [webhook approver]\\\",\\\"reason\\\":\\\"ContainersNotReady\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"message\\\":\\\"containers with unready status: [webhook approver]\\\",\\\"reason\\\":\\\"ContainersNotReady\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"ContainersReady\\\"}],\\\"containerStatuses\\\":[{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:174f36cdd47ef0d1d2099482919d773257453265a2af0b17b154edc32fa41ac2\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{\\\"terminated\\\":{\\\"exitCode\\\":137,\\\"finishedAt\\\":null,\\\"message\\\":\\\"The container could not be located when the pod was deleted. The container used to be Running\\\",\\\"reason\\\":\\\"ContainerStatusUnknown\\\",\\\"startedAt\\\":null}},\\\"name\\\":\\\"approver\\\",\\\"ready\\\":false,\\\"restartCount\\\":6,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"ContainerCreating\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/env\\\",\\\"name\\\":\\\"env-overrides\\\"},{\\\"mountPath\\\":\\\"/var/run/ovnkube-identity-config\\\",\\\"name\\\":\\\"ovnkube-identity-cm\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-s2kz5\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:174f36cdd47ef0d1d2099482919d773257453265a2af0b17b154edc32fa41ac2\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{\\\"terminated\\\":{\\\"exitCode\\\":137,\\\"finishedAt\\\":null,\\\"message\\\":\\\"The container could not be located when the pod was deleted. The container used to be Running\\\",\\\"reason\\\":\\\"ContainerStatusUnknown\\\",\\\"startedAt\\\":null}},\\\"name\\\":\\\"webhook\\\",\\\"ready\\\":false,\\\"restartCount\\\":3,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"ContainerCreating\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/etc/webhook-cert/\\\",\\\"name\\\":\\\"webhook-cert\\\"},{\\\"mountPath\\\":\\\"/env\\\",\\\"name\\\":\\\"env-overrides\\\"},{\\\"mountPath\\\":\\\"/var/run/ovnkube-identity-config\\\",\\\"name\\\":\\\"ovnkube-identity-cm\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-s2kz5\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}]}}\" for pod \"openshift-network-node-identity\"/\"network-node-identity-vrzqb\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://127.0.0.1:9743/pod?timeout=10s\": dial tcp 127.0.0.1:9743: connect: connection refused" Dec 03 11:20:32 crc kubenswrapper[5052]: I1203 11:20:32.884638 5052 status_manager.go:875] "Failed to update status for pod" pod="openshift-network-console/networking-console-plugin-85b44fc459-gdk6g" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"5fe485a1-e14f-4c09-b5b9-f252bc42b7e8\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"message\\\":\\\"containers with unready status: [networking-console-plugin]\\\",\\\"reason\\\":\\\"ContainersNotReady\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"message\\\":\\\"containers with unready status: [networking-console-plugin]\\\",\\\"reason\\\":\\\"ContainersNotReady\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"ContainersReady\\\"}],\\\"containerStatuses\\\":[{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ae647598ec35cda5766806d3d44a91e3b9d4dee48ff154f3d8490165399873fd\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{\\\"terminated\\\":{\\\"exitCode\\\":137,\\\"finishedAt\\\":null,\\\"message\\\":\\\"The container could not be located when the pod was deleted. The container used to be Running\\\",\\\"reason\\\":\\\"ContainerStatusUnknown\\\",\\\"startedAt\\\":null}},\\\"name\\\":\\\"networking-console-plugin\\\",\\\"ready\\\":false,\\\"restartCount\\\":3,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"ContainerCreating\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/var/cert\\\",\\\"name\\\":\\\"networking-console-plugin-cert\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/etc/nginx/nginx.conf\\\",\\\"name\\\":\\\"nginx-conf\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}],\\\"podIP\\\":null,\\\"podIPs\\\":null}}\" for pod \"openshift-network-console\"/\"networking-console-plugin-85b44fc459-gdk6g\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://127.0.0.1:9743/pod?timeout=10s\": dial tcp 127.0.0.1:9743: connect: connection refused" Dec 03 11:20:32 crc kubenswrapper[5052]: I1203 11:20:32.895240 5052 status_manager.go:875] "Failed to update status for pod" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"10c237bc-72e7-4da3-9f2c-b1688395bf6e\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"Initialized\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"message\\\":\\\"containers with unready status: [machine-config-daemon kube-rbac-proxy]\\\",\\\"reason\\\":\\\"ContainersNotReady\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"message\\\":\\\"containers with unready status: [machine-config-daemon kube-rbac-proxy]\\\",\\\"reason\\\":\\\"ContainersNotReady\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"ContainersReady\\\"}],\\\"containerStatuses\\\":[{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:242b3d66438c42745f4ef318bdeaf3d793426f12962a42ea83e18d06c08aaf09\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"kube-rbac-proxy\\\",\\\"ready\\\":false,\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"ContainerCreating\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/etc/tls/private\\\",\\\"name\\\":\\\"proxy-tls\\\"},{\\\"mountPath\\\":\\\"/etc/kube-rbac-proxy\\\",\\\"name\\\":\\\"mcd-auth-proxy-config\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-9vkjv\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c915fb8ba96e911699a1ae34a8e95ca8a9fbe1bf8c28fea177225c63a8bdfc0a\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"machine-config-daemon\\\",\\\"ready\\\":false,\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"ContainerCreating\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/rootfs\\\",\\\"name\\\":\\\"rootfs\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-9vkjv\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}],\\\"hostIP\\\":\\\"192.168.126.11\\\",\\\"hostIPs\\\":[{\\\"ip\\\":\\\"192.168.126.11\\\"}],\\\"podIP\\\":\\\"192.168.126.11\\\",\\\"podIPs\\\":[{\\\"ip\\\":\\\"192.168.126.11\\\"}],\\\"startTime\\\":\\\"2025-12-03T11:20:31Z\\\"}}\" for pod \"openshift-machine-config-operator\"/\"machine-config-daemon-74bj6\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://127.0.0.1:9743/pod?timeout=10s\": dial tcp 127.0.0.1:9743: connect: connection refused" Dec 03 11:20:32 crc kubenswrapper[5052]: I1203 11:20:32.914569 5052 status_manager.go:875] "Failed to update status for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kg55h" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"bc569bba-c020-40d6-af68-e617625fe0e4\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"message\\\":\\\"containers with incomplete status: [kubecfg-setup]\\\",\\\"reason\\\":\\\"ContainersNotInitialized\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"Initialized\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"message\\\":\\\"containers with unready status: [ovn-controller ovn-acl-logging kube-rbac-proxy-node kube-rbac-proxy-ovn-metrics northd nbdb sbdb ovnkube-controller]\\\",\\\"reason\\\":\\\"ContainersNotReady\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"message\\\":\\\"containers with unready status: [ovn-controller ovn-acl-logging kube-rbac-proxy-node kube-rbac-proxy-ovn-metrics northd nbdb sbdb ovnkube-controller]\\\",\\\"reason\\\":\\\"ContainersNotReady\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"ContainersReady\\\"}],\\\"containerStatuses\\\":[{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:242b3d66438c42745f4ef318bdeaf3d793426f12962a42ea83e18d06c08aaf09\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"kube-rbac-proxy-node\\\",\\\"ready\\\":false,\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"PodInitializing\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/ovnkube-lib\\\",\\\"name\\\":\\\"ovnkube-script-lib\\\"},{\\\"mountPath\\\":\\\"/etc/pki/tls/metrics-cert\\\",\\\"name\\\":\\\"ovn-node-metrics-cert\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-x92h2\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:242b3d66438c42745f4ef318bdeaf3d793426f12962a42ea83e18d06c08aaf09\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"kube-rbac-proxy-ovn-metrics\\\",\\\"ready\\\":false,\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"PodInitializing\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/ovnkube-lib\\\",\\\"name\\\":\\\"ovnkube-script-lib\\\"},{\\\"mountPath\\\":\\\"/etc/pki/tls/metrics-cert\\\",\\\"name\\\":\\\"ovn-node-metrics-cert\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-x92h2\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:174f36cdd47ef0d1d2099482919d773257453265a2af0b17b154edc32fa41ac2\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"nbdb\\\",\\\"ready\\\":false,\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"PodInitializing\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/ovnkube-lib\\\",\\\"name\\\":\\\"ovnkube-script-lib\\\"},{\\\"mountPath\\\":\\\"/etc/ovn/\\\",\\\"name\\\":\\\"etc-openvswitch\\\"},{\\\"mountPath\\\":\\\"/var/log/ovn\\\",\\\"name\\\":\\\"node-log\\\"},{\\\"mountPath\\\":\\\"/run/ovn/\\\",\\\"name\\\":\\\"run-ovn\\\"},{\\\"mountPath\\\":\\\"/env\\\",\\\"name\\\":\\\"env-overrides\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-x92h2\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:174f36cdd47ef0d1d2099482919d773257453265a2af0b17b154edc32fa41ac2\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"northd\\\",\\\"ready\\\":false,\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"PodInitializing\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/ovnkube-lib\\\",\\\"name\\\":\\\"ovnkube-script-lib\\\"},{\\\"mountPath\\\":\\\"/etc/ovn\\\",\\\"name\\\":\\\"etc-openvswitch\\\"},{\\\"mountPath\\\":\\\"/var/log/ovn\\\",\\\"name\\\":\\\"node-log\\\"},{\\\"mountPath\\\":\\\"/run/ovn/\\\",\\\"name\\\":\\\"run-ovn\\\"},{\\\"mountPath\\\":\\\"/env\\\",\\\"name\\\":\\\"env-overrides\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-x92h2\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:174f36cdd47ef0d1d2099482919d773257453265a2af0b17b154edc32fa41ac2\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"ovn-acl-logging\\\",\\\"ready\\\":false,\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"PodInitializing\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/ovnkube-lib\\\",\\\"name\\\":\\\"ovnkube-script-lib\\\"},{\\\"mountPath\\\":\\\"/var/log/ovn/\\\",\\\"name\\\":\\\"node-log\\\"},{\\\"mountPath\\\":\\\"/run/ovn/\\\",\\\"name\\\":\\\"run-ovn\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-x92h2\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:174f36cdd47ef0d1d2099482919d773257453265a2af0b17b154edc32fa41ac2\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"ovn-controller\\\",\\\"ready\\\":false,\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"PodInitializing\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/ovnkube-lib\\\",\\\"name\\\":\\\"ovnkube-script-lib\\\"},{\\\"mountPath\\\":\\\"/run/openvswitch\\\",\\\"name\\\":\\\"run-openvswitch\\\"},{\\\"mountPath\\\":\\\"/run/ovn/\\\",\\\"name\\\":\\\"run-ovn\\\"},{\\\"mountPath\\\":\\\"/etc/openvswitch\\\",\\\"name\\\":\\\"etc-openvswitch\\\"},{\\\"mountPath\\\":\\\"/etc/ovn/\\\",\\\"name\\\":\\\"etc-openvswitch\\\"},{\\\"mountPath\\\":\\\"/var/lib/openvswitch\\\",\\\"name\\\":\\\"var-lib-openvswitch\\\"},{\\\"mountPath\\\":\\\"/env\\\",\\\"name\\\":\\\"env-overrides\\\"},{\\\"mountPath\\\":\\\"/var/log/ovn/\\\",\\\"name\\\":\\\"node-log\\\"},{\\\"mountPath\\\":\\\"/dev/log\\\",\\\"name\\\":\\\"log-socket\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-x92h2\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:174f36cdd47ef0d1d2099482919d773257453265a2af0b17b154edc32fa41ac2\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"ovnkube-controller\\\",\\\"ready\\\":false,\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"PodInitializing\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/ovnkube-lib\\\",\\\"name\\\":\\\"ovnkube-script-lib\\\"},{\\\"mountPath\\\":\\\"/var/lib/kubelet\\\",\\\"name\\\":\\\"host-kubelet\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/etc/systemd/system\\\",\\\"name\\\":\\\"systemd-units\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/host\\\",\\\"name\\\":\\\"host-slash\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/run/ovn-kubernetes/\\\",\\\"name\\\":\\\"host-run-ovn-kubernetes\\\"},{\\\"mountPath\\\":\\\"/run/netns\\\",\\\"name\\\":\\\"host-run-netns\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/run/systemd/private\\\",\\\"name\\\":\\\"run-systemd\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/cni-bin-dir\\\",\\\"name\\\":\\\"host-cni-bin\\\"},{\\\"mountPath\\\":\\\"/etc/cni/net.d\\\",\\\"name\\\":\\\"host-cni-netd\\\"},{\\\"mountPath\\\":\\\"/var/lib/cni/networks/ovn-k8s-cni-overlay\\\",\\\"name\\\":\\\"host-var-lib-cni-networks-ovn-kubernetes\\\"},{\\\"mountPath\\\":\\\"/run/openvswitch\\\",\\\"name\\\":\\\"run-openvswitch\\\"},{\\\"mountPath\\\":\\\"/var/log/ovnkube/\\\",\\\"name\\\":\\\"etc-openvswitch\\\"},{\\\"mountPath\\\":\\\"/run/ovn/\\\",\\\"name\\\":\\\"run-ovn\\\"},{\\\"mountPath\\\":\\\"/etc/openvswitch\\\",\\\"name\\\":\\\"etc-openvswitch\\\"},{\\\"mountPath\\\":\\\"/etc/ovn/\\\",\\\"name\\\":\\\"etc-openvswitch\\\"},{\\\"mountPath\\\":\\\"/var/lib/openvswitch\\\",\\\"name\\\":\\\"var-lib-openvswitch\\\"},{\\\"mountPath\\\":\\\"/run/ovnkube-config/\\\",\\\"name\\\":\\\"ovnkube-config\\\"},{\\\"mountPath\\\":\\\"/env\\\",\\\"name\\\":\\\"env-overrides\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-x92h2\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:174f36cdd47ef0d1d2099482919d773257453265a2af0b17b154edc32fa41ac2\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"sbdb\\\",\\\"ready\\\":false,\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"PodInitializing\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/ovnkube-lib\\\",\\\"name\\\":\\\"ovnkube-script-lib\\\"},{\\\"mountPath\\\":\\\"/etc/ovn/\\\",\\\"name\\\":\\\"etc-openvswitch\\\"},{\\\"mountPath\\\":\\\"/run/ovn/\\\",\\\"name\\\":\\\"run-ovn\\\"},{\\\"mountPath\\\":\\\"/var/log/ovn\\\",\\\"name\\\":\\\"node-log\\\"},{\\\"mountPath\\\":\\\"/env\\\",\\\"name\\\":\\\"env-overrides\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-x92h2\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}],\\\"hostIP\\\":\\\"192.168.126.11\\\",\\\"hostIPs\\\":[{\\\"ip\\\":\\\"192.168.126.11\\\"}],\\\"initContainerStatuses\\\":[{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:174f36cdd47ef0d1d2099482919d773257453265a2af0b17b154edc32fa41ac2\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"kubecfg-setup\\\",\\\"ready\\\":false,\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"PodInitializing\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/etc/ovn/\\\",\\\"name\\\":\\\"etc-openvswitch\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-x92h2\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}],\\\"podIP\\\":\\\"192.168.126.11\\\",\\\"podIPs\\\":[{\\\"ip\\\":\\\"192.168.126.11\\\"}],\\\"startTime\\\":\\\"2025-12-03T11:20:31Z\\\"}}\" for pod \"openshift-ovn-kubernetes\"/\"ovnkube-node-kg55h\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://127.0.0.1:9743/pod?timeout=10s\": dial tcp 127.0.0.1:9743: connect: connection refused" Dec 03 11:20:32 crc kubenswrapper[5052]: I1203 11:20:32.932750 5052 status_manager.go:875] "Failed to update status for pod" pod="openshift-kube-controller-manager/kube-controller-manager-crc" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"f31e1704-9c1d-4044-b318-2aaccb5896e2\\\"},\\\"status\\\":{\\\"conditions\\\":[{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:15Z\\\",\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:09Z\\\",\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"Initialized\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"ContainersReady\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:09Z\\\",\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"PodScheduled\\\"}],\\\"containerStatuses\\\":[{\\\"containerID\\\":\\\"cri-o://37485762ea1808cd3794c9641a4ead224b107e6c64c295de237f8ffd50aba4f4\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c0f9da410c07372b6c9ad6a79379b491cd10fdee88051c026b084652d85aed21\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c0f9da410c07372b6c9ad6a79379b491cd10fdee88051c026b084652d85aed21\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"cluster-policy-controller\\\",\\\"ready\\\":true,\\\"restartCount\\\":0,\\\"started\\\":true,\\\"state\\\":{\\\"running\\\":{\\\"startedAt\\\":\\\"2025-12-03T11:20:12Z\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/etc/kubernetes/static-pod-resources\\\",\\\"name\\\":\\\"resource-dir\\\"},{\\\"mountPath\\\":\\\"/etc/kubernetes/static-pod-certs\\\",\\\"name\\\":\\\"cert-dir\\\"}]},{\\\"containerID\\\":\\\"cri-o://f5d23c6160b5790e9272bd5ae83472267f4d1f60b67e0425a6bc2f5c1f9d1fb7\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:06bc35825771aee1220d34720243b89c4ba8a8b335e6de2597126bd791fd90d4\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:06bc35825771aee1220d34720243b89c4ba8a8b335e6de2597126bd791fd90d4\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"kube-controller-manager\\\",\\\"ready\\\":true,\\\"restartCount\\\":0,\\\"started\\\":true,\\\"state\\\":{\\\"running\\\":{\\\"startedAt\\\":\\\"2025-12-03T11:20:11Z\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/etc/kubernetes/static-pod-resources\\\",\\\"name\\\":\\\"resource-dir\\\"},{\\\"mountPath\\\":\\\"/etc/kubernetes/static-pod-certs\\\",\\\"name\\\":\\\"cert-dir\\\"}]},{\\\"containerID\\\":\\\"cri-o://d05df6945d11d2abe73005efafe80220497a339008d4c28f7923c2dc7ac4a150\\\",\\\"image\\\":\\\"quay.io/crcont/openshift-crc-cluster-kube-controller-manager-operator@sha256:8506ce0a578bc18fac117eb2b82799488ffac0bed08287faaf92edaf5d17ab95\\\",\\\"imageID\\\":\\\"quay.io/crcont/openshift-crc-cluster-kube-controller-manager-operator@sha256:8506ce0a578bc18fac117eb2b82799488ffac0bed08287faaf92edaf5d17ab95\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"kube-controller-manager-cert-syncer\\\",\\\"ready\\\":true,\\\"restartCount\\\":0,\\\"started\\\":true,\\\"state\\\":{\\\"running\\\":{\\\"startedAt\\\":\\\"2025-12-03T11:20:12Z\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/etc/kubernetes/static-pod-resources\\\",\\\"name\\\":\\\"resource-dir\\\"},{\\\"mountPath\\\":\\\"/etc/kubernetes/static-pod-certs\\\",\\\"name\\\":\\\"cert-dir\\\"}]},{\\\"containerID\\\":\\\"cri-o://8df2aa2470046bba37b41cb301e96f52b09b83124384b5dbc4942cd0bfeef1c6\\\",\\\"image\\\":\\\"quay.io/crcont/openshift-crc-cluster-kube-controller-manager-operator@sha256:8506ce0a578bc18fac117eb2b82799488ffac0bed08287faaf92edaf5d17ab95\\\",\\\"imageID\\\":\\\"quay.io/crcont/openshift-crc-cluster-kube-controller-manager-operator@sha256:8506ce0a578bc18fac117eb2b82799488ffac0bed08287faaf92edaf5d17ab95\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"kube-controller-manager-recovery-controller\\\",\\\"ready\\\":true,\\\"restartCount\\\":0,\\\"started\\\":true,\\\"state\\\":{\\\"running\\\":{\\\"startedAt\\\":\\\"2025-12-03T11:20:15Z\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/etc/kubernetes/static-pod-resources\\\",\\\"name\\\":\\\"resource-dir\\\"},{\\\"mountPath\\\":\\\"/etc/kubernetes/static-pod-certs\\\",\\\"name\\\":\\\"cert-dir\\\"}]}],\\\"hostIP\\\":\\\"192.168.126.11\\\",\\\"hostIPs\\\":[{\\\"ip\\\":\\\"192.168.126.11\\\"}],\\\"phase\\\":\\\"Running\\\",\\\"podIP\\\":\\\"192.168.126.11\\\",\\\"podIPs\\\":[{\\\"ip\\\":\\\"192.168.126.11\\\"}],\\\"startTime\\\":\\\"2025-12-03T11:20:09Z\\\"}}\" for pod \"openshift-kube-controller-manager\"/\"kube-controller-manager-crc\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://127.0.0.1:9743/pod?timeout=10s\": dial tcp 127.0.0.1:9743: connect: connection refused" Dec 03 11:20:32 crc kubenswrapper[5052]: I1203 11:20:32.955764 5052 status_manager.go:875] "Failed to update status for pod" pod="openshift-network-diagnostics/network-check-source-55646444c4-trplf" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"9d751cbb-f2e2-430d-9754-c882a5e924a5\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"message\\\":\\\"containers with unready status: [check-endpoints]\\\",\\\"reason\\\":\\\"ContainersNotReady\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"message\\\":\\\"containers with unready status: [check-endpoints]\\\",\\\"reason\\\":\\\"ContainersNotReady\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"ContainersReady\\\"}],\\\"containerStatuses\\\":[{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e1baa38811c04bd8909e01a1f3be7421a1cb99d608d3dc4cf86d95b17de2ab8b\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{\\\"terminated\\\":{\\\"exitCode\\\":137,\\\"finishedAt\\\":null,\\\"message\\\":\\\"The container could not be located when the pod was deleted. The container used to be Running\\\",\\\"reason\\\":\\\"ContainerStatusUnknown\\\",\\\"startedAt\\\":null}},\\\"name\\\":\\\"check-endpoints\\\",\\\"ready\\\":false,\\\"restartCount\\\":3,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"ContainerCreating\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-s2dwl\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}],\\\"podIP\\\":null,\\\"podIPs\\\":null}}\" for pod \"openshift-network-diagnostics\"/\"network-check-source-55646444c4-trplf\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://127.0.0.1:9743/pod?timeout=10s\": dial tcp 127.0.0.1:9743: connect: connection refused" Dec 03 11:20:32 crc kubenswrapper[5052]: I1203 11:20:32.970263 5052 status_manager.go:875] "Failed to update status for pod" pod="openshift-network-operator/iptables-alerter-4ln5h" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"d75a4c96-2883-4a0b-bab2-0fab2b6c0b49\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"message\\\":\\\"containers with unready status: [iptables-alerter]\\\",\\\"reason\\\":\\\"ContainersNotReady\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"message\\\":\\\"containers with unready status: [iptables-alerter]\\\",\\\"reason\\\":\\\"ContainersNotReady\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"ContainersReady\\\"}],\\\"containerStatuses\\\":[{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:35512335ac39aed0f55b7f799f416f4f6445c20c1b19888cf2bb72bb276703f2\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{\\\"terminated\\\":{\\\"exitCode\\\":137,\\\"finishedAt\\\":null,\\\"message\\\":\\\"The container could not be located when the pod was deleted. The container used to be Running\\\",\\\"reason\\\":\\\"ContainerStatusUnknown\\\",\\\"startedAt\\\":null}},\\\"name\\\":\\\"iptables-alerter\\\",\\\"ready\\\":false,\\\"restartCount\\\":3,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"ContainerCreating\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/iptables-alerter\\\",\\\"name\\\":\\\"iptables-alerter-script\\\"},{\\\"mountPath\\\":\\\"/host\\\",\\\"name\\\":\\\"host-slash\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-rczfb\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}]}}\" for pod \"openshift-network-operator\"/\"iptables-alerter-4ln5h\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://127.0.0.1:9743/pod?timeout=10s\": dial tcp 127.0.0.1:9743: connect: connection refused" Dec 03 11:20:32 crc kubenswrapper[5052]: I1203 11:20:32.982372 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-etcd/etcd-crc" Dec 03 11:20:32 crc kubenswrapper[5052]: I1203 11:20:32.983538 5052 status_manager.go:875] "Failed to update status for pod" pod="openshift-multus/multus-hx657" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"c30c02db-f4e7-4432-941c-5448fd71f3d9\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"Initialized\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"message\\\":\\\"containers with unready status: [kube-multus]\\\",\\\"reason\\\":\\\"ContainersNotReady\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"message\\\":\\\"containers with unready status: [kube-multus]\\\",\\\"reason\\\":\\\"ContainersNotReady\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"ContainersReady\\\"}],\\\"containerStatuses\\\":[{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7eeaee65f2808b819eedb413bdcabb9144e12f0dd97f13fd1afba93a95b67b26\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"kube-multus\\\",\\\"ready\\\":false,\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"ContainerCreating\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/entrypoint\\\",\\\"name\\\":\\\"cni-binary-copy\\\"},{\\\"mountPath\\\":\\\"/host/etc/os-release\\\",\\\"name\\\":\\\"os-release\\\"},{\\\"mountPath\\\":\\\"/host/etc/cni/net.d\\\",\\\"name\\\":\\\"system-cni-dir\\\"},{\\\"mountPath\\\":\\\"/host/run/multus/cni/net.d\\\",\\\"name\\\":\\\"multus-cni-dir\\\"},{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/run/multus\\\",\\\"name\\\":\\\"multus-socket-dir-parent\\\"},{\\\"mountPath\\\":\\\"/run/k8s.cni.cncf.io\\\",\\\"name\\\":\\\"host-run-k8s-cni-cncf-io\\\"},{\\\"mountPath\\\":\\\"/run/netns\\\",\\\"name\\\":\\\"host-run-netns\\\"},{\\\"mountPath\\\":\\\"/var/lib/cni/bin\\\",\\\"name\\\":\\\"host-var-lib-cni-bin\\\"},{\\\"mountPath\\\":\\\"/var/lib/cni/multus\\\",\\\"name\\\":\\\"host-var-lib-cni-multus\\\"},{\\\"mountPath\\\":\\\"/var/lib/kubelet\\\",\\\"name\\\":\\\"host-var-lib-kubelet\\\"},{\\\"mountPath\\\":\\\"/hostroot\\\",\\\"name\\\":\\\"hostroot\\\"},{\\\"mountPath\\\":\\\"/etc/cni/multus/net.d\\\",\\\"name\\\":\\\"multus-conf-dir\\\"},{\\\"mountPath\\\":\\\"/etc/cni/net.d/multus.d\\\",\\\"name\\\":\\\"multus-daemon-config\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/etc/cni/multus/certs\\\",\\\"name\\\":\\\"host-run-multus-certs\\\"},{\\\"mountPath\\\":\\\"/etc/kubernetes\\\",\\\"name\\\":\\\"etc-kubernetes\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-fhsjl\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}],\\\"hostIP\\\":\\\"192.168.126.11\\\",\\\"hostIPs\\\":[{\\\"ip\\\":\\\"192.168.126.11\\\"}],\\\"podIP\\\":\\\"192.168.126.11\\\",\\\"podIPs\\\":[{\\\"ip\\\":\\\"192.168.126.11\\\"}],\\\"startTime\\\":\\\"2025-12-03T11:20:31Z\\\"}}\" for pod \"openshift-multus\"/\"multus-hx657\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://127.0.0.1:9743/pod?timeout=10s\": dial tcp 127.0.0.1:9743: connect: connection refused" Dec 03 11:20:32 crc kubenswrapper[5052]: I1203 11:20:32.994056 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-etcd/etcd-crc" Dec 03 11:20:32 crc kubenswrapper[5052]: I1203 11:20:32.996560 5052 status_manager.go:875] "Failed to update status for pod" pod="openshift-kube-apiserver/kube-apiserver-crc" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"6c8df12a-3d11-4bb9-b398-67091c1918e4\\\"},\\\"status\\\":{\\\"conditions\\\":[{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:11Z\\\",\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:11Z\\\",\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"Initialized\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:09Z\\\",\\\"message\\\":\\\"containers with unready status: [kube-apiserver-check-endpoints]\\\",\\\"reason\\\":\\\"ContainersNotReady\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:09Z\\\",\\\"message\\\":\\\"containers with unready status: [kube-apiserver-check-endpoints]\\\",\\\"reason\\\":\\\"ContainersNotReady\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"ContainersReady\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:09Z\\\",\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"PodScheduled\\\"}],\\\"containerStatuses\\\":[{\\\"containerID\\\":\\\"cri-o://09256543a9aa86da8bec948be040d36b6e4986b9232091d2638bb1fead5740a9\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:06bc35825771aee1220d34720243b89c4ba8a8b335e6de2597126bd791fd90d4\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:06bc35825771aee1220d34720243b89c4ba8a8b335e6de2597126bd791fd90d4\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"kube-apiserver\\\",\\\"ready\\\":true,\\\"restartCount\\\":0,\\\"started\\\":true,\\\"state\\\":{\\\"running\\\":{\\\"startedAt\\\":\\\"2025-12-03T11:20:12Z\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/etc/kubernetes/static-pod-resources\\\",\\\"name\\\":\\\"resource-dir\\\"},{\\\"mountPath\\\":\\\"/etc/kubernetes/static-pod-certs\\\",\\\"name\\\":\\\"cert-dir\\\"},{\\\"mountPath\\\":\\\"/var/log/kube-apiserver\\\",\\\"name\\\":\\\"audit-dir\\\"}]},{\\\"containerID\\\":\\\"cri-o://15abb0d7fd5884519ed5ac454430dbb58f71ed1f459e5ed30f324f70befd243e\\\",\\\"image\\\":\\\"quay.io/crcont/openshift-crc-cluster-kube-apiserver-operator@sha256:9f36dc276e27753fc478274c7f7814a4f8945c987117ee1ea3b8e6355e6d7462\\\",\\\"imageID\\\":\\\"quay.io/crcont/openshift-crc-cluster-kube-apiserver-operator@sha256:9f36dc276e27753fc478274c7f7814a4f8945c987117ee1ea3b8e6355e6d7462\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"kube-apiserver-cert-regeneration-controller\\\",\\\"ready\\\":true,\\\"restartCount\\\":0,\\\"started\\\":true,\\\"state\\\":{\\\"running\\\":{\\\"startedAt\\\":\\\"2025-12-03T11:20:14Z\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/etc/kubernetes/static-pod-resources\\\",\\\"name\\\":\\\"resource-dir\\\"}]},{\\\"containerID\\\":\\\"cri-o://8ceedd21ae3833d77fd440537f30434ee57f087fbefb2357b46231d3f778ad2a\\\",\\\"image\\\":\\\"quay.io/crcont/openshift-crc-cluster-kube-apiserver-operator@sha256:9f36dc276e27753fc478274c7f7814a4f8945c987117ee1ea3b8e6355e6d7462\\\",\\\"imageID\\\":\\\"quay.io/crcont/openshift-crc-cluster-kube-apiserver-operator@sha256:9f36dc276e27753fc478274c7f7814a4f8945c987117ee1ea3b8e6355e6d7462\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"kube-apiserver-cert-syncer\\\",\\\"ready\\\":true,\\\"restartCount\\\":0,\\\"started\\\":true,\\\"state\\\":{\\\"running\\\":{\\\"startedAt\\\":\\\"2025-12-03T11:20:12Z\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/etc/kubernetes/static-pod-resources\\\",\\\"name\\\":\\\"resource-dir\\\"},{\\\"mountPath\\\":\\\"/etc/kubernetes/static-pod-certs\\\",\\\"name\\\":\\\"cert-dir\\\"}]},{\\\"containerID\\\":\\\"cri-o://ee9c5c61cfd6dee9fd44ebc4c7ef4d20b5fde82b12a8147c3c871e2615da7940\\\",\\\"image\\\":\\\"quay.io/crcont/openshift-crc-cluster-kube-apiserver-operator@sha256:9f36dc276e27753fc478274c7f7814a4f8945c987117ee1ea3b8e6355e6d7462\\\",\\\"imageID\\\":\\\"quay.io/crcont/openshift-crc-cluster-kube-apiserver-operator@sha256:9f36dc276e27753fc478274c7f7814a4f8945c987117ee1ea3b8e6355e6d7462\\\",\\\"lastState\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://b739f84524d09b27f6e052e21df8be3870a2136665704232f5a365d8a5bf41f5\\\",\\\"exitCode\\\":255,\\\"finishedAt\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"message\\\":\\\"ing back to namespace): Get \\\\\\\"https://localhost:6443/api/v1/namespaces/openshift-kube-apiserver/pods/kube-apiserver-crc\\\\\\\": net/http: TLS handshake timeout\\\\nI1203 11:20:25.911385 1 builder.go:304] check-endpoints version 4.18.0-202502101302.p0.g763313c.assembly.stream.el9-763313c-763313c860ea43fcfc9b1ac00ebae096b57c078e\\\\nI1203 11:20:25.912713 1 dynamic_serving_content.go:116] \\\\\\\"Loaded a new cert/key pair\\\\\\\" name=\\\\\\\"serving-cert::/tmp/serving-cert-3310779835/tls.crt::/tmp/serving-cert-3310779835/tls.key\\\\\\\"\\\\nI1203 11:20:31.324803 1 requestheader_controller.go:247] Loaded a new request header values for RequestHeaderAuthRequestController\\\\nI1203 11:20:31.327725 1 maxinflight.go:139] \\\\\\\"Initialized nonMutatingChan\\\\\\\" len=400\\\\nI1203 11:20:31.327838 1 maxinflight.go:145] \\\\\\\"Initialized mutatingChan\\\\\\\" len=200\\\\nI1203 11:20:31.327886 1 maxinflight.go:116] \\\\\\\"Set denominator for readonly requests\\\\\\\" limit=400\\\\nI1203 11:20:31.327915 1 maxinflight.go:120] \\\\\\\"Set denominator for mutating requests\\\\\\\" limit=200\\\\nI1203 11:20:31.335573 1 secure_serving.go:57] Forcing use of http/1.1 only\\\\nW1203 11:20:31.335607 1 secure_serving.go:69] Use of insecure cipher 'TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256' detected.\\\\nW1203 11:20:31.335613 1 secure_serving.go:69] Use of insecure cipher 'TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256' detected.\\\\nW1203 11:20:31.335619 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_128_GCM_SHA256' detected.\\\\nW1203 11:20:31.335623 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_256_GCM_SHA384' detected.\\\\nW1203 11:20:31.335626 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_128_CBC_SHA' detected.\\\\nW1203 11:20:31.335630 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_256_CBC_SHA' detected.\\\\nI1203 11:20:31.335893 1 genericapiserver.go:533] MuxAndDiscoveryComplete has all endpoints registered and discovery information is complete\\\\nF1203 11:20:31.338035 1 cmd.go:182] pods \\\\\\\"kube-apiserver-crc\\\\\\\" not found\\\\n\\\",\\\"reason\\\":\\\"Error\\\",\\\"startedAt\\\":\\\"2025-12-03T11:20:15Z\\\"}},\\\"name\\\":\\\"kube-apiserver-check-endpoints\\\",\\\"ready\\\":false,\\\"restartCount\\\":1,\\\"started\\\":true,\\\"state\\\":{\\\"running\\\":{\\\"startedAt\\\":\\\"2025-12-03T11:20:32Z\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/etc/kubernetes/static-pod-resources\\\",\\\"name\\\":\\\"resource-dir\\\"},{\\\"mountPath\\\":\\\"/etc/kubernetes/static-pod-certs\\\",\\\"name\\\":\\\"cert-dir\\\"}]},{\\\"containerID\\\":\\\"cri-o://22b8cb6aa04337d1420f97d3e128fce2b29a725ca6e5d157adeb1ba78f00073f\\\",\\\"image\\\":\\\"quay.io/crcont/openshift-crc-cluster-kube-apiserver-operator@sha256:9f36dc276e27753fc478274c7f7814a4f8945c987117ee1ea3b8e6355e6d7462\\\",\\\"imageID\\\":\\\"quay.io/crcont/openshift-crc-cluster-kube-apiserver-operator@sha256:9f36dc276e27753fc478274c7f7814a4f8945c987117ee1ea3b8e6355e6d7462\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"kube-apiserver-insecure-readyz\\\",\\\"ready\\\":true,\\\"restartCount\\\":0,\\\"started\\\":true,\\\"state\\\":{\\\"running\\\":{\\\"startedAt\\\":\\\"2025-12-03T11:20:15Z\\\"}}}],\\\"hostIP\\\":\\\"192.168.126.11\\\",\\\"hostIPs\\\":[{\\\"ip\\\":\\\"192.168.126.11\\\"}],\\\"initContainerStatuses\\\":[{\\\"containerID\\\":\\\"cri-o://7182c75773f069c72f87c39136222fd4a91b49add27352a1969645413e670f01\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:06bc35825771aee1220d34720243b89c4ba8a8b335e6de2597126bd791fd90d4\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:06bc35825771aee1220d34720243b89c4ba8a8b335e6de2597126bd791fd90d4\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"setup\\\",\\\"ready\\\":true,\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://7182c75773f069c72f87c39136222fd4a91b49add27352a1969645413e670f01\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2025-12-03T11:20:11Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2025-12-03T11:20:11Z\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/var/log/kube-apiserver\\\",\\\"name\\\":\\\"audit-dir\\\"}]}],\\\"phase\\\":\\\"Running\\\",\\\"podIP\\\":\\\"192.168.126.11\\\",\\\"podIPs\\\":[{\\\"ip\\\":\\\"192.168.126.11\\\"}],\\\"startTime\\\":\\\"2025-12-03T11:20:09Z\\\"}}\" for pod \"openshift-kube-apiserver\"/\"kube-apiserver-crc\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://127.0.0.1:9743/pod?timeout=10s\": dial tcp 127.0.0.1:9743: connect: connection refused" Dec 03 11:20:33 crc kubenswrapper[5052]: I1203 11:20:33.001884 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openshift-etcd/etcd-crc"] Dec 03 11:20:33 crc kubenswrapper[5052]: I1203 11:20:33.010467 5052 status_manager.go:875] "Failed to update status for pod" pod="openshift-network-diagnostics/network-check-target-xd92c" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"3b6479f0-333b-4a96-9adf-2099afdc2447\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"message\\\":\\\"containers with unready status: [network-check-target-container]\\\",\\\"reason\\\":\\\"ContainersNotReady\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"message\\\":\\\"containers with unready status: [network-check-target-container]\\\",\\\"reason\\\":\\\"ContainersNotReady\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"ContainersReady\\\"}],\\\"containerStatuses\\\":[{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e1baa38811c04bd8909e01a1f3be7421a1cb99d608d3dc4cf86d95b17de2ab8b\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{\\\"terminated\\\":{\\\"exitCode\\\":137,\\\"finishedAt\\\":null,\\\"message\\\":\\\"The container could not be located when the pod was deleted. The container used to be Running\\\",\\\"reason\\\":\\\"ContainerStatusUnknown\\\",\\\"startedAt\\\":null}},\\\"name\\\":\\\"network-check-target-container\\\",\\\"ready\\\":false,\\\"restartCount\\\":3,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"ContainerCreating\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-cqllr\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}],\\\"podIP\\\":null,\\\"podIPs\\\":null}}\" for pod \"openshift-network-diagnostics\"/\"network-check-target-xd92c\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://127.0.0.1:9743/pod?timeout=10s\": dial tcp 127.0.0.1:9743: connect: connection refused" Dec 03 11:20:33 crc kubenswrapper[5052]: I1203 11:20:33.021352 5052 status_manager.go:875] "Failed to update status for pod" pod="openshift-dns/node-resolver-xph69" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"e5c587e6-866a-4427-8a2c-3469cf0fc495\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"Initialized\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"message\\\":\\\"containers with unready status: [dns-node-resolver]\\\",\\\"reason\\\":\\\"ContainersNotReady\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"message\\\":\\\"containers with unready status: [dns-node-resolver]\\\",\\\"reason\\\":\\\"ContainersNotReady\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"ContainersReady\\\"}],\\\"containerStatuses\\\":[{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:35512335ac39aed0f55b7f799f416f4f6445c20c1b19888cf2bb72bb276703f2\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"dns-node-resolver\\\",\\\"ready\\\":false,\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"ContainerCreating\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/etc/hosts\\\",\\\"name\\\":\\\"hosts-file\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-tk7n6\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}],\\\"hostIP\\\":\\\"192.168.126.11\\\",\\\"hostIPs\\\":[{\\\"ip\\\":\\\"192.168.126.11\\\"}],\\\"podIP\\\":\\\"192.168.126.11\\\",\\\"podIPs\\\":[{\\\"ip\\\":\\\"192.168.126.11\\\"}],\\\"startTime\\\":\\\"2025-12-03T11:20:31Z\\\"}}\" for pod \"openshift-dns\"/\"node-resolver-xph69\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://127.0.0.1:9743/pod?timeout=10s\": dial tcp 127.0.0.1:9743: connect: connection refused" Dec 03 11:20:33 crc kubenswrapper[5052]: I1203 11:20:33.058798 5052 status_manager.go:875] "Failed to update status for pod" pod="openshift-multus/multus-additional-cni-plugins-2lrql" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"17265a68-bcb5-455c-b7df-c148a6517db9\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"message\\\":\\\"containers with incomplete status: [egress-router-binary-copy cni-plugins bond-cni-plugin routeoverride-cni whereabouts-cni-bincopy whereabouts-cni]\\\",\\\"reason\\\":\\\"ContainersNotInitialized\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"Initialized\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"message\\\":\\\"containers with unready status: [kube-multus-additional-cni-plugins]\\\",\\\"reason\\\":\\\"ContainersNotReady\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"message\\\":\\\"containers with unready status: [kube-multus-additional-cni-plugins]\\\",\\\"reason\\\":\\\"ContainersNotReady\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"ContainersReady\\\"}],\\\"containerStatuses\\\":[{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7eeaee65f2808b819eedb413bdcabb9144e12f0dd97f13fd1afba93a95b67b26\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"kube-multus-additional-cni-plugins\\\",\\\"ready\\\":false,\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"PodInitializing\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-w4zf5\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}],\\\"hostIP\\\":\\\"192.168.126.11\\\",\\\"hostIPs\\\":[{\\\"ip\\\":\\\"192.168.126.11\\\"}],\\\"initContainerStatuses\\\":[{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e40792096b162f0f9ce5f8362f51e5f8dea2c1ce4b1447235388416b5db7708c\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"egress-router-binary-copy\\\",\\\"ready\\\":false,\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"PodInitializing\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/entrypoint\\\",\\\"name\\\":\\\"cni-binary-copy\\\"},{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/os-release\\\",\\\"name\\\":\\\"os-release\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-w4zf5\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:687fddfbb085a1688df312ce4ec8c857df9b2daed8ff4a7ed6163a1154afa2cc\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"cni-plugins\\\",\\\"ready\\\":false,\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"PodInitializing\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/entrypoint\\\",\\\"name\\\":\\\"cni-binary-copy\\\"},{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/os-release\\\",\\\"name\\\":\\\"os-release\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/host/etc/cni/tuning/\\\",\\\"name\\\":\\\"tuning-conf-dir\\\"},{\\\"mountPath\\\":\\\"/sysctls\\\",\\\"name\\\":\\\"cni-sysctl-allowlist\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-w4zf5\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:383f4cceeeaead203bb2327fdd367c64b64d729d7fa93089f249e496fcef0c78\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"bond-cni-plugin\\\",\\\"ready\\\":false,\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"PodInitializing\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/entrypoint\\\",\\\"name\\\":\\\"cni-binary-copy\\\"},{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/os-release\\\",\\\"name\\\":\\\"os-release\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-w4zf5\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f567acb85146b5ed81451ec3e79f2de0c62e28c69b2eeade0abdf5d0c388e7aa\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"routeoverride-cni\\\",\\\"ready\\\":false,\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"PodInitializing\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/entrypoint\\\",\\\"name\\\":\\\"cni-binary-copy\\\"},{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/os-release\\\",\\\"name\\\":\\\"os-release\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-w4zf5\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:98100674616e54319f6713d742fd0c3bdbc84e6e6173e8ccf4a2473a714c2bc4\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"whereabouts-cni-bincopy\\\",\\\"ready\\\":false,\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"PodInitializing\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/entrypoint\\\",\\\"name\\\":\\\"cni-binary-copy\\\"},{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/os-release\\\",\\\"name\\\":\\\"os-release\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-w4zf5\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:98100674616e54319f6713d742fd0c3bdbc84e6e6173e8ccf4a2473a714c2bc4\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"whereabouts-cni\\\",\\\"ready\\\":false,\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"PodInitializing\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/cni/net.d\\\",\\\"name\\\":\\\"system-cni-dir\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-w4zf5\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}],\\\"podIP\\\":\\\"192.168.126.11\\\",\\\"podIPs\\\":[{\\\"ip\\\":\\\"192.168.126.11\\\"}],\\\"startTime\\\":\\\"2025-12-03T11:20:31Z\\\"}}\" for pod \"openshift-multus\"/\"multus-additional-cni-plugins-2lrql\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://127.0.0.1:9743/pod?timeout=10s\": dial tcp 127.0.0.1:9743: connect: connection refused" Dec 03 11:20:33 crc kubenswrapper[5052]: I1203 11:20:33.075669 5052 status_manager.go:875] "Failed to update status for pod" pod="openshift-kube-apiserver/kube-apiserver-crc" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"6c8df12a-3d11-4bb9-b398-67091c1918e4\\\"},\\\"status\\\":{\\\"conditions\\\":[{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:11Z\\\",\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:11Z\\\",\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"Initialized\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:09Z\\\",\\\"message\\\":\\\"containers with unready status: [kube-apiserver-check-endpoints]\\\",\\\"reason\\\":\\\"ContainersNotReady\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:09Z\\\",\\\"message\\\":\\\"containers with unready status: [kube-apiserver-check-endpoints]\\\",\\\"reason\\\":\\\"ContainersNotReady\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"ContainersReady\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:09Z\\\",\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"PodScheduled\\\"}],\\\"containerStatuses\\\":[{\\\"containerID\\\":\\\"cri-o://09256543a9aa86da8bec948be040d36b6e4986b9232091d2638bb1fead5740a9\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:06bc35825771aee1220d34720243b89c4ba8a8b335e6de2597126bd791fd90d4\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:06bc35825771aee1220d34720243b89c4ba8a8b335e6de2597126bd791fd90d4\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"kube-apiserver\\\",\\\"ready\\\":true,\\\"restartCount\\\":0,\\\"started\\\":true,\\\"state\\\":{\\\"running\\\":{\\\"startedAt\\\":\\\"2025-12-03T11:20:12Z\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/etc/kubernetes/static-pod-resources\\\",\\\"name\\\":\\\"resource-dir\\\"},{\\\"mountPath\\\":\\\"/etc/kubernetes/static-pod-certs\\\",\\\"name\\\":\\\"cert-dir\\\"},{\\\"mountPath\\\":\\\"/var/log/kube-apiserver\\\",\\\"name\\\":\\\"audit-dir\\\"}]},{\\\"containerID\\\":\\\"cri-o://15abb0d7fd5884519ed5ac454430dbb58f71ed1f459e5ed30f324f70befd243e\\\",\\\"image\\\":\\\"quay.io/crcont/openshift-crc-cluster-kube-apiserver-operator@sha256:9f36dc276e27753fc478274c7f7814a4f8945c987117ee1ea3b8e6355e6d7462\\\",\\\"imageID\\\":\\\"quay.io/crcont/openshift-crc-cluster-kube-apiserver-operator@sha256:9f36dc276e27753fc478274c7f7814a4f8945c987117ee1ea3b8e6355e6d7462\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"kube-apiserver-cert-regeneration-controller\\\",\\\"ready\\\":true,\\\"restartCount\\\":0,\\\"started\\\":true,\\\"state\\\":{\\\"running\\\":{\\\"startedAt\\\":\\\"2025-12-03T11:20:14Z\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/etc/kubernetes/static-pod-resources\\\",\\\"name\\\":\\\"resource-dir\\\"}]},{\\\"containerID\\\":\\\"cri-o://8ceedd21ae3833d77fd440537f30434ee57f087fbefb2357b46231d3f778ad2a\\\",\\\"image\\\":\\\"quay.io/crcont/openshift-crc-cluster-kube-apiserver-operator@sha256:9f36dc276e27753fc478274c7f7814a4f8945c987117ee1ea3b8e6355e6d7462\\\",\\\"imageID\\\":\\\"quay.io/crcont/openshift-crc-cluster-kube-apiserver-operator@sha256:9f36dc276e27753fc478274c7f7814a4f8945c987117ee1ea3b8e6355e6d7462\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"kube-apiserver-cert-syncer\\\",\\\"ready\\\":true,\\\"restartCount\\\":0,\\\"started\\\":true,\\\"state\\\":{\\\"running\\\":{\\\"startedAt\\\":\\\"2025-12-03T11:20:12Z\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/etc/kubernetes/static-pod-resources\\\",\\\"name\\\":\\\"resource-dir\\\"},{\\\"mountPath\\\":\\\"/etc/kubernetes/static-pod-certs\\\",\\\"name\\\":\\\"cert-dir\\\"}]},{\\\"containerID\\\":\\\"cri-o://ee9c5c61cfd6dee9fd44ebc4c7ef4d20b5fde82b12a8147c3c871e2615da7940\\\",\\\"image\\\":\\\"quay.io/crcont/openshift-crc-cluster-kube-apiserver-operator@sha256:9f36dc276e27753fc478274c7f7814a4f8945c987117ee1ea3b8e6355e6d7462\\\",\\\"imageID\\\":\\\"quay.io/crcont/openshift-crc-cluster-kube-apiserver-operator@sha256:9f36dc276e27753fc478274c7f7814a4f8945c987117ee1ea3b8e6355e6d7462\\\",\\\"lastState\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://b739f84524d09b27f6e052e21df8be3870a2136665704232f5a365d8a5bf41f5\\\",\\\"exitCode\\\":255,\\\"finishedAt\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"message\\\":\\\"ing back to namespace): Get \\\\\\\"https://localhost:6443/api/v1/namespaces/openshift-kube-apiserver/pods/kube-apiserver-crc\\\\\\\": net/http: TLS handshake timeout\\\\nI1203 11:20:25.911385 1 builder.go:304] check-endpoints version 4.18.0-202502101302.p0.g763313c.assembly.stream.el9-763313c-763313c860ea43fcfc9b1ac00ebae096b57c078e\\\\nI1203 11:20:25.912713 1 dynamic_serving_content.go:116] \\\\\\\"Loaded a new cert/key pair\\\\\\\" name=\\\\\\\"serving-cert::/tmp/serving-cert-3310779835/tls.crt::/tmp/serving-cert-3310779835/tls.key\\\\\\\"\\\\nI1203 11:20:31.324803 1 requestheader_controller.go:247] Loaded a new request header values for RequestHeaderAuthRequestController\\\\nI1203 11:20:31.327725 1 maxinflight.go:139] \\\\\\\"Initialized nonMutatingChan\\\\\\\" len=400\\\\nI1203 11:20:31.327838 1 maxinflight.go:145] \\\\\\\"Initialized mutatingChan\\\\\\\" len=200\\\\nI1203 11:20:31.327886 1 maxinflight.go:116] \\\\\\\"Set denominator for readonly requests\\\\\\\" limit=400\\\\nI1203 11:20:31.327915 1 maxinflight.go:120] \\\\\\\"Set denominator for mutating requests\\\\\\\" limit=200\\\\nI1203 11:20:31.335573 1 secure_serving.go:57] Forcing use of http/1.1 only\\\\nW1203 11:20:31.335607 1 secure_serving.go:69] Use of insecure cipher 'TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256' detected.\\\\nW1203 11:20:31.335613 1 secure_serving.go:69] Use of insecure cipher 'TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256' detected.\\\\nW1203 11:20:31.335619 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_128_GCM_SHA256' detected.\\\\nW1203 11:20:31.335623 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_256_GCM_SHA384' detected.\\\\nW1203 11:20:31.335626 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_128_CBC_SHA' detected.\\\\nW1203 11:20:31.335630 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_256_CBC_SHA' detected.\\\\nI1203 11:20:31.335893 1 genericapiserver.go:533] MuxAndDiscoveryComplete has all endpoints registered and discovery information is complete\\\\nF1203 11:20:31.338035 1 cmd.go:182] pods \\\\\\\"kube-apiserver-crc\\\\\\\" not found\\\\n\\\",\\\"reason\\\":\\\"Error\\\",\\\"startedAt\\\":\\\"2025-12-03T11:20:15Z\\\"}},\\\"name\\\":\\\"kube-apiserver-check-endpoints\\\",\\\"ready\\\":false,\\\"restartCount\\\":1,\\\"started\\\":true,\\\"state\\\":{\\\"running\\\":{\\\"startedAt\\\":\\\"2025-12-03T11:20:32Z\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/etc/kubernetes/static-pod-resources\\\",\\\"name\\\":\\\"resource-dir\\\"},{\\\"mountPath\\\":\\\"/etc/kubernetes/static-pod-certs\\\",\\\"name\\\":\\\"cert-dir\\\"}]},{\\\"containerID\\\":\\\"cri-o://22b8cb6aa04337d1420f97d3e128fce2b29a725ca6e5d157adeb1ba78f00073f\\\",\\\"image\\\":\\\"quay.io/crcont/openshift-crc-cluster-kube-apiserver-operator@sha256:9f36dc276e27753fc478274c7f7814a4f8945c987117ee1ea3b8e6355e6d7462\\\",\\\"imageID\\\":\\\"quay.io/crcont/openshift-crc-cluster-kube-apiserver-operator@sha256:9f36dc276e27753fc478274c7f7814a4f8945c987117ee1ea3b8e6355e6d7462\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"kube-apiserver-insecure-readyz\\\",\\\"ready\\\":true,\\\"restartCount\\\":0,\\\"started\\\":true,\\\"state\\\":{\\\"running\\\":{\\\"startedAt\\\":\\\"2025-12-03T11:20:15Z\\\"}}}],\\\"hostIP\\\":\\\"192.168.126.11\\\",\\\"hostIPs\\\":[{\\\"ip\\\":\\\"192.168.126.11\\\"}],\\\"initContainerStatuses\\\":[{\\\"containerID\\\":\\\"cri-o://7182c75773f069c72f87c39136222fd4a91b49add27352a1969645413e670f01\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:06bc35825771aee1220d34720243b89c4ba8a8b335e6de2597126bd791fd90d4\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:06bc35825771aee1220d34720243b89c4ba8a8b335e6de2597126bd791fd90d4\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"setup\\\",\\\"ready\\\":true,\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://7182c75773f069c72f87c39136222fd4a91b49add27352a1969645413e670f01\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2025-12-03T11:20:11Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2025-12-03T11:20:11Z\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/var/log/kube-apiserver\\\",\\\"name\\\":\\\"audit-dir\\\"}]}],\\\"phase\\\":\\\"Running\\\",\\\"podIP\\\":\\\"192.168.126.11\\\",\\\"podIPs\\\":[{\\\"ip\\\":\\\"192.168.126.11\\\"}],\\\"startTime\\\":\\\"2025-12-03T11:20:09Z\\\"}}\" for pod \"openshift-kube-apiserver\"/\"kube-apiserver-crc\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://127.0.0.1:9743/pod?timeout=10s\": tls: failed to verify certificate: x509: certificate has expired or is not yet valid: current time 2025-12-03T11:20:33Z is after 2025-08-24T17:21:41Z" Dec 03 11:20:33 crc kubenswrapper[5052]: I1203 11:20:33.089256 5052 status_manager.go:875] "Failed to update status for pod" pod="openshift-network-diagnostics/network-check-target-xd92c" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"3b6479f0-333b-4a96-9adf-2099afdc2447\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"message\\\":\\\"containers with unready status: [network-check-target-container]\\\",\\\"reason\\\":\\\"ContainersNotReady\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"message\\\":\\\"containers with unready status: [network-check-target-container]\\\",\\\"reason\\\":\\\"ContainersNotReady\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"ContainersReady\\\"}],\\\"containerStatuses\\\":[{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e1baa38811c04bd8909e01a1f3be7421a1cb99d608d3dc4cf86d95b17de2ab8b\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{\\\"terminated\\\":{\\\"exitCode\\\":137,\\\"finishedAt\\\":null,\\\"message\\\":\\\"The container could not be located when the pod was deleted. The container used to be Running\\\",\\\"reason\\\":\\\"ContainerStatusUnknown\\\",\\\"startedAt\\\":null}},\\\"name\\\":\\\"network-check-target-container\\\",\\\"ready\\\":false,\\\"restartCount\\\":3,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"ContainerCreating\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-cqllr\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}],\\\"podIP\\\":null,\\\"podIPs\\\":null}}\" for pod \"openshift-network-diagnostics\"/\"network-check-target-xd92c\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://127.0.0.1:9743/pod?timeout=10s\": tls: failed to verify certificate: x509: certificate has expired or is not yet valid: current time 2025-12-03T11:20:33Z is after 2025-08-24T17:21:41Z" Dec 03 11:20:33 crc kubenswrapper[5052]: I1203 11:20:33.100801 5052 status_manager.go:875] "Failed to update status for pod" pod="openshift-dns/node-resolver-xph69" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"e5c587e6-866a-4427-8a2c-3469cf0fc495\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:32Z\\\",\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"Initialized\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:32Z\\\",\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:32Z\\\",\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"ContainersReady\\\"}],\\\"containerStatuses\\\":[{\\\"containerID\\\":\\\"cri-o://8cbacdbdef2832bba669718580c35080702354f16ec06e7d2680a6ee5b63e8b3\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:35512335ac39aed0f55b7f799f416f4f6445c20c1b19888cf2bb72bb276703f2\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:35512335ac39aed0f55b7f799f416f4f6445c20c1b19888cf2bb72bb276703f2\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"dns-node-resolver\\\",\\\"ready\\\":true,\\\"restartCount\\\":0,\\\"started\\\":true,\\\"state\\\":{\\\"running\\\":{\\\"startedAt\\\":\\\"2025-12-03T11:20:32Z\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/etc/hosts\\\",\\\"name\\\":\\\"hosts-file\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-tk7n6\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}],\\\"hostIP\\\":\\\"192.168.126.11\\\",\\\"hostIPs\\\":[{\\\"ip\\\":\\\"192.168.126.11\\\"}],\\\"phase\\\":\\\"Running\\\",\\\"podIP\\\":\\\"192.168.126.11\\\",\\\"podIPs\\\":[{\\\"ip\\\":\\\"192.168.126.11\\\"}],\\\"startTime\\\":\\\"2025-12-03T11:20:31Z\\\"}}\" for pod \"openshift-dns\"/\"node-resolver-xph69\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://127.0.0.1:9743/pod?timeout=10s\": tls: failed to verify certificate: x509: certificate has expired or is not yet valid: current time 2025-12-03T11:20:33Z is after 2025-08-24T17:21:41Z" Dec 03 11:20:33 crc kubenswrapper[5052]: I1203 11:20:33.117941 5052 status_manager.go:875] "Failed to update status for pod" pod="openshift-multus/multus-additional-cni-plugins-2lrql" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"17265a68-bcb5-455c-b7df-c148a6517db9\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:32Z\\\",\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"message\\\":\\\"containers with incomplete status: [egress-router-binary-copy cni-plugins bond-cni-plugin routeoverride-cni whereabouts-cni-bincopy whereabouts-cni]\\\",\\\"reason\\\":\\\"ContainersNotInitialized\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"Initialized\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"message\\\":\\\"containers with unready status: [kube-multus-additional-cni-plugins]\\\",\\\"reason\\\":\\\"ContainersNotReady\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"message\\\":\\\"containers with unready status: [kube-multus-additional-cni-plugins]\\\",\\\"reason\\\":\\\"ContainersNotReady\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"ContainersReady\\\"}],\\\"containerStatuses\\\":[{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7eeaee65f2808b819eedb413bdcabb9144e12f0dd97f13fd1afba93a95b67b26\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"kube-multus-additional-cni-plugins\\\",\\\"ready\\\":false,\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"PodInitializing\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-w4zf5\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}],\\\"hostIP\\\":\\\"192.168.126.11\\\",\\\"hostIPs\\\":[{\\\"ip\\\":\\\"192.168.126.11\\\"}],\\\"initContainerStatuses\\\":[{\\\"containerID\\\":\\\"cri-o://ce6a0f2a385f3ad9f34f3a09e960700dc4d3941341c08997b95c648c82d11749\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e40792096b162f0f9ce5f8362f51e5f8dea2c1ce4b1447235388416b5db7708c\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e40792096b162f0f9ce5f8362f51e5f8dea2c1ce4b1447235388416b5db7708c\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"egress-router-binary-copy\\\",\\\"ready\\\":false,\\\"restartCount\\\":0,\\\"started\\\":true,\\\"state\\\":{\\\"running\\\":{\\\"startedAt\\\":\\\"2025-12-03T11:20:32Z\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/entrypoint\\\",\\\"name\\\":\\\"cni-binary-copy\\\"},{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/os-release\\\",\\\"name\\\":\\\"os-release\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-w4zf5\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:687fddfbb085a1688df312ce4ec8c857df9b2daed8ff4a7ed6163a1154afa2cc\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"cni-plugins\\\",\\\"ready\\\":false,\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"PodInitializing\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/entrypoint\\\",\\\"name\\\":\\\"cni-binary-copy\\\"},{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/os-release\\\",\\\"name\\\":\\\"os-release\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/host/etc/cni/tuning/\\\",\\\"name\\\":\\\"tuning-conf-dir\\\"},{\\\"mountPath\\\":\\\"/sysctls\\\",\\\"name\\\":\\\"cni-sysctl-allowlist\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-w4zf5\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:383f4cceeeaead203bb2327fdd367c64b64d729d7fa93089f249e496fcef0c78\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"bond-cni-plugin\\\",\\\"ready\\\":false,\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"PodInitializing\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/entrypoint\\\",\\\"name\\\":\\\"cni-binary-copy\\\"},{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/os-release\\\",\\\"name\\\":\\\"os-release\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-w4zf5\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f567acb85146b5ed81451ec3e79f2de0c62e28c69b2eeade0abdf5d0c388e7aa\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"routeoverride-cni\\\",\\\"ready\\\":false,\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"PodInitializing\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/entrypoint\\\",\\\"name\\\":\\\"cni-binary-copy\\\"},{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/os-release\\\",\\\"name\\\":\\\"os-release\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-w4zf5\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:98100674616e54319f6713d742fd0c3bdbc84e6e6173e8ccf4a2473a714c2bc4\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"whereabouts-cni-bincopy\\\",\\\"ready\\\":false,\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"PodInitializing\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/entrypoint\\\",\\\"name\\\":\\\"cni-binary-copy\\\"},{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/os-release\\\",\\\"name\\\":\\\"os-release\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-w4zf5\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:98100674616e54319f6713d742fd0c3bdbc84e6e6173e8ccf4a2473a714c2bc4\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"whereabouts-cni\\\",\\\"ready\\\":false,\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"PodInitializing\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/cni/net.d\\\",\\\"name\\\":\\\"system-cni-dir\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-w4zf5\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}],\\\"podIP\\\":\\\"192.168.126.11\\\",\\\"podIPs\\\":[{\\\"ip\\\":\\\"192.168.126.11\\\"}],\\\"startTime\\\":\\\"2025-12-03T11:20:31Z\\\"}}\" for pod \"openshift-multus\"/\"multus-additional-cni-plugins-2lrql\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://127.0.0.1:9743/pod?timeout=10s\": tls: failed to verify certificate: x509: certificate has expired or is not yet valid: current time 2025-12-03T11:20:33Z is after 2025-08-24T17:21:41Z" Dec 03 11:20:33 crc kubenswrapper[5052]: I1203 11:20:33.136177 5052 status_manager.go:875] "Failed to update status for pod" pod="openshift-multus/multus-hx657" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"c30c02db-f4e7-4432-941c-5448fd71f3d9\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:32Z\\\",\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"Initialized\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:32Z\\\",\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:32Z\\\",\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"ContainersReady\\\"}],\\\"containerStatuses\\\":[{\\\"containerID\\\":\\\"cri-o://388fefa2a0e0ba8ca09e3fd39371e442358e9547cc2789ca8fa4b7b7237f6ac5\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7eeaee65f2808b819eedb413bdcabb9144e12f0dd97f13fd1afba93a95b67b26\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7eeaee65f2808b819eedb413bdcabb9144e12f0dd97f13fd1afba93a95b67b26\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"kube-multus\\\",\\\"ready\\\":true,\\\"restartCount\\\":0,\\\"started\\\":true,\\\"state\\\":{\\\"running\\\":{\\\"startedAt\\\":\\\"2025-12-03T11:20:32Z\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/entrypoint\\\",\\\"name\\\":\\\"cni-binary-copy\\\"},{\\\"mountPath\\\":\\\"/host/etc/os-release\\\",\\\"name\\\":\\\"os-release\\\"},{\\\"mountPath\\\":\\\"/host/etc/cni/net.d\\\",\\\"name\\\":\\\"system-cni-dir\\\"},{\\\"mountPath\\\":\\\"/host/run/multus/cni/net.d\\\",\\\"name\\\":\\\"multus-cni-dir\\\"},{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/run/multus\\\",\\\"name\\\":\\\"multus-socket-dir-parent\\\"},{\\\"mountPath\\\":\\\"/run/k8s.cni.cncf.io\\\",\\\"name\\\":\\\"host-run-k8s-cni-cncf-io\\\"},{\\\"mountPath\\\":\\\"/run/netns\\\",\\\"name\\\":\\\"host-run-netns\\\"},{\\\"mountPath\\\":\\\"/var/lib/cni/bin\\\",\\\"name\\\":\\\"host-var-lib-cni-bin\\\"},{\\\"mountPath\\\":\\\"/var/lib/cni/multus\\\",\\\"name\\\":\\\"host-var-lib-cni-multus\\\"},{\\\"mountPath\\\":\\\"/var/lib/kubelet\\\",\\\"name\\\":\\\"host-var-lib-kubelet\\\"},{\\\"mountPath\\\":\\\"/hostroot\\\",\\\"name\\\":\\\"hostroot\\\"},{\\\"mountPath\\\":\\\"/etc/cni/multus/net.d\\\",\\\"name\\\":\\\"multus-conf-dir\\\"},{\\\"mountPath\\\":\\\"/etc/cni/net.d/multus.d\\\",\\\"name\\\":\\\"multus-daemon-config\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/etc/cni/multus/certs\\\",\\\"name\\\":\\\"host-run-multus-certs\\\"},{\\\"mountPath\\\":\\\"/etc/kubernetes\\\",\\\"name\\\":\\\"etc-kubernetes\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-fhsjl\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}],\\\"hostIP\\\":\\\"192.168.126.11\\\",\\\"hostIPs\\\":[{\\\"ip\\\":\\\"192.168.126.11\\\"}],\\\"phase\\\":\\\"Running\\\",\\\"podIP\\\":\\\"192.168.126.11\\\",\\\"podIPs\\\":[{\\\"ip\\\":\\\"192.168.126.11\\\"}],\\\"startTime\\\":\\\"2025-12-03T11:20:31Z\\\"}}\" for pod \"openshift-multus\"/\"multus-hx657\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://127.0.0.1:9743/pod?timeout=10s\": tls: failed to verify certificate: x509: certificate has expired or is not yet valid: current time 2025-12-03T11:20:33Z is after 2025-08-24T17:21:41Z" Dec 03 11:20:33 crc kubenswrapper[5052]: I1203 11:20:33.162376 5052 status_manager.go:875] "Failed to update status for pod" pod="openshift-etcd/etcd-crc" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"bc6c0f8f-e9c0-47cd-89fa-974113e2c13a\\\"},\\\"status\\\":{\\\"conditions\\\":[{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:11Z\\\",\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:15Z\\\",\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"Initialized\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:32Z\\\",\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:32Z\\\",\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"ContainersReady\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:09Z\\\",\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"PodScheduled\\\"}],\\\"containerStatuses\\\":[{\\\"containerID\\\":\\\"cri-o://f3bd857c0d35ed129a4fdc96deae82d6457f5f24c4d92a129972ce97a89b3488\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:07b7c6877441ecd6a5646fb68e33e9be8b90092272e49117b54b4a67314731ca\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:07b7c6877441ecd6a5646fb68e33e9be8b90092272e49117b54b4a67314731ca\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"etcd\\\",\\\"ready\\\":true,\\\"restartCount\\\":0,\\\"started\\\":true,\\\"state\\\":{\\\"running\\\":{\\\"startedAt\\\":\\\"2025-12-03T11:20:16Z\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/etc/kubernetes/manifests\\\",\\\"name\\\":\\\"static-pod-dir\\\"},{\\\"mountPath\\\":\\\"/etc/kubernetes/static-pod-resources\\\",\\\"name\\\":\\\"resource-dir\\\"},{\\\"mountPath\\\":\\\"/etc/kubernetes/static-pod-certs\\\",\\\"name\\\":\\\"cert-dir\\\"},{\\\"mountPath\\\":\\\"/var/lib/etcd/\\\",\\\"name\\\":\\\"data-dir\\\"}]},{\\\"containerID\\\":\\\"cri-o://c5dd3e47134939ea92cb1677d8d171eadd8c0c0743827ebee0a61daf261f4e30\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:07b7c6877441ecd6a5646fb68e33e9be8b90092272e49117b54b4a67314731ca\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:07b7c6877441ecd6a5646fb68e33e9be8b90092272e49117b54b4a67314731ca\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"etcd-metrics\\\",\\\"ready\\\":true,\\\"restartCount\\\":0,\\\"started\\\":true,\\\"state\\\":{\\\"running\\\":{\\\"startedAt\\\":\\\"2025-12-03T11:20:16Z\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/etc/kubernetes/static-pod-resources\\\",\\\"name\\\":\\\"resource-dir\\\"},{\\\"mountPath\\\":\\\"/etc/kubernetes/static-pod-certs\\\",\\\"name\\\":\\\"cert-dir\\\"},{\\\"mountPath\\\":\\\"/var/lib/etcd/\\\",\\\"name\\\":\\\"data-dir\\\"}]},{\\\"containerID\\\":\\\"cri-o://2409d840f8acda1e98e333c3b4d9ebd1b67f4a05fa3f662f6d7171dddab13846\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a0fa3723269019bee1847b26702f42928e779036cc2f58408f8ee7866be30a93\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a0fa3723269019bee1847b26702f42928e779036cc2f58408f8ee7866be30a93\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"etcd-readyz\\\",\\\"ready\\\":true,\\\"restartCount\\\":0,\\\"started\\\":true,\\\"state\\\":{\\\"running\\\":{\\\"startedAt\\\":\\\"2025-12-03T11:20:16Z\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/var/log/etcd/\\\",\\\"name\\\":\\\"log-dir\\\"},{\\\"mountPath\\\":\\\"/etc/kubernetes/static-pod-certs\\\",\\\"name\\\":\\\"cert-dir\\\"}]},{\\\"containerID\\\":\\\"cri-o://7e429db04e5ef00df1a3a67e1750865167bb14eef83a110aa54ce3ca2f4656c5\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a0fa3723269019bee1847b26702f42928e779036cc2f58408f8ee7866be30a93\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a0fa3723269019bee1847b26702f42928e779036cc2f58408f8ee7866be30a93\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"etcd-rev\\\",\\\"ready\\\":true,\\\"restartCount\\\":0,\\\"started\\\":true,\\\"state\\\":{\\\"running\\\":{\\\"startedAt\\\":\\\"2025-12-03T11:20:16Z\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/var/lib/etcd\\\",\\\"name\\\":\\\"data-dir\\\"},{\\\"mountPath\\\":\\\"/etc/kubernetes/static-pod-certs\\\",\\\"name\\\":\\\"cert-dir\\\"}]},{\\\"containerID\\\":\\\"cri-o://d9e3c1335ee9e6fe5d5ca827757da511034a270ca681b3cef694e595539d1cd0\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:07b7c6877441ecd6a5646fb68e33e9be8b90092272e49117b54b4a67314731ca\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:07b7c6877441ecd6a5646fb68e33e9be8b90092272e49117b54b4a67314731ca\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"etcdctl\\\",\\\"ready\\\":true,\\\"restartCount\\\":0,\\\"started\\\":true,\\\"state\\\":{\\\"running\\\":{\\\"startedAt\\\":\\\"2025-12-03T11:20:15Z\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/etc/kubernetes/manifests\\\",\\\"name\\\":\\\"static-pod-dir\\\"},{\\\"mountPath\\\":\\\"/etc/kubernetes/static-pod-resources\\\",\\\"name\\\":\\\"resource-dir\\\"},{\\\"mountPath\\\":\\\"/etc/kubernetes/static-pod-certs\\\",\\\"name\\\":\\\"cert-dir\\\"},{\\\"mountPath\\\":\\\"/var/lib/etcd/\\\",\\\"name\\\":\\\"data-dir\\\"}]}],\\\"hostIP\\\":\\\"192.168.126.11\\\",\\\"hostIPs\\\":[{\\\"ip\\\":\\\"192.168.126.11\\\"}],\\\"initContainerStatuses\\\":[{\\\"containerID\\\":\\\"cri-o://7d7c61963a9597f2c9c159d743284313820acece17152be4d829b0e3d02434c1\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:07b7c6877441ecd6a5646fb68e33e9be8b90092272e49117b54b4a67314731ca\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:07b7c6877441ecd6a5646fb68e33e9be8b90092272e49117b54b4a67314731ca\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"setup\\\",\\\"ready\\\":true,\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://7d7c61963a9597f2c9c159d743284313820acece17152be4d829b0e3d02434c1\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2025-12-03T11:20:11Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2025-12-03T11:20:11Z\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/var/log/etcd\\\",\\\"name\\\":\\\"log-dir\\\"}]},{\\\"containerID\\\":\\\"cri-o://e980e02c63b59597153749cc2ed54da5925e88a6a0ed2293f4075ff4b46bbe44\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:07b7c6877441ecd6a5646fb68e33e9be8b90092272e49117b54b4a67314731ca\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:07b7c6877441ecd6a5646fb68e33e9be8b90092272e49117b54b4a67314731ca\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"etcd-ensure-env-vars\\\",\\\"ready\\\":true,\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://e980e02c63b59597153749cc2ed54da5925e88a6a0ed2293f4075ff4b46bbe44\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2025-12-03T11:20:12Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2025-12-03T11:20:12Z\\\"}}},{\\\"containerID\\\":\\\"cri-o://6a0411ec57e822071155dd66aa1e30dcab88df41e229cf75208468e5dcf90358\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:07b7c6877441ecd6a5646fb68e33e9be8b90092272e49117b54b4a67314731ca\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:07b7c6877441ecd6a5646fb68e33e9be8b90092272e49117b54b4a67314731ca\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"etcd-resources-copy\\\",\\\"ready\\\":true,\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://6a0411ec57e822071155dd66aa1e30dcab88df41e229cf75208468e5dcf90358\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2025-12-03T11:20:15Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2025-12-03T11:20:15Z\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/etc/kubernetes/static-pod-resources\\\",\\\"name\\\":\\\"resource-dir\\\"},{\\\"mountPath\\\":\\\"/etc/kubernetes/static-pod-certs\\\",\\\"name\\\":\\\"cert-dir\\\"},{\\\"mountPath\\\":\\\"/usr/local/bin\\\",\\\"name\\\":\\\"usr-local-bin\\\"}]}],\\\"phase\\\":\\\"Running\\\",\\\"podIP\\\":\\\"192.168.126.11\\\",\\\"podIPs\\\":[{\\\"ip\\\":\\\"192.168.126.11\\\"}],\\\"startTime\\\":\\\"2025-12-03T11:20:09Z\\\"}}\" for pod \"openshift-etcd\"/\"etcd-crc\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://127.0.0.1:9743/pod?timeout=10s\": tls: failed to verify certificate: x509: certificate has expired or is not yet valid: current time 2025-12-03T11:20:33Z is after 2025-08-24T17:21:41Z" Dec 03 11:20:33 crc kubenswrapper[5052]: I1203 11:20:33.176463 5052 status_manager.go:875] "Failed to update status for pod" pod="openshift-network-node-identity/network-node-identity-vrzqb" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"ef543e1b-8068-4ea3-b32a-61027b32e95d\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"message\\\":\\\"containers with unready status: [webhook approver]\\\",\\\"reason\\\":\\\"ContainersNotReady\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"message\\\":\\\"containers with unready status: [webhook approver]\\\",\\\"reason\\\":\\\"ContainersNotReady\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"ContainersReady\\\"}],\\\"containerStatuses\\\":[{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:174f36cdd47ef0d1d2099482919d773257453265a2af0b17b154edc32fa41ac2\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{\\\"terminated\\\":{\\\"exitCode\\\":137,\\\"finishedAt\\\":null,\\\"message\\\":\\\"The container could not be located when the pod was deleted. The container used to be Running\\\",\\\"reason\\\":\\\"ContainerStatusUnknown\\\",\\\"startedAt\\\":null}},\\\"name\\\":\\\"approver\\\",\\\"ready\\\":false,\\\"restartCount\\\":6,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"ContainerCreating\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/env\\\",\\\"name\\\":\\\"env-overrides\\\"},{\\\"mountPath\\\":\\\"/var/run/ovnkube-identity-config\\\",\\\"name\\\":\\\"ovnkube-identity-cm\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-s2kz5\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:174f36cdd47ef0d1d2099482919d773257453265a2af0b17b154edc32fa41ac2\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{\\\"terminated\\\":{\\\"exitCode\\\":137,\\\"finishedAt\\\":null,\\\"message\\\":\\\"The container could not be located when the pod was deleted. The container used to be Running\\\",\\\"reason\\\":\\\"ContainerStatusUnknown\\\",\\\"startedAt\\\":null}},\\\"name\\\":\\\"webhook\\\",\\\"ready\\\":false,\\\"restartCount\\\":3,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"ContainerCreating\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/etc/webhook-cert/\\\",\\\"name\\\":\\\"webhook-cert\\\"},{\\\"mountPath\\\":\\\"/env\\\",\\\"name\\\":\\\"env-overrides\\\"},{\\\"mountPath\\\":\\\"/var/run/ovnkube-identity-config\\\",\\\"name\\\":\\\"ovnkube-identity-cm\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-s2kz5\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}]}}\" for pod \"openshift-network-node-identity\"/\"network-node-identity-vrzqb\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://127.0.0.1:9743/pod?timeout=10s\": tls: failed to verify certificate: x509: certificate has expired or is not yet valid: current time 2025-12-03T11:20:33Z is after 2025-08-24T17:21:41Z" Dec 03 11:20:33 crc kubenswrapper[5052]: I1203 11:20:33.191606 5052 status_manager.go:875] "Failed to update status for pod" pod="openshift-network-console/networking-console-plugin-85b44fc459-gdk6g" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"5fe485a1-e14f-4c09-b5b9-f252bc42b7e8\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"message\\\":\\\"containers with unready status: [networking-console-plugin]\\\",\\\"reason\\\":\\\"ContainersNotReady\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"message\\\":\\\"containers with unready status: [networking-console-plugin]\\\",\\\"reason\\\":\\\"ContainersNotReady\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"ContainersReady\\\"}],\\\"containerStatuses\\\":[{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ae647598ec35cda5766806d3d44a91e3b9d4dee48ff154f3d8490165399873fd\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{\\\"terminated\\\":{\\\"exitCode\\\":137,\\\"finishedAt\\\":null,\\\"message\\\":\\\"The container could not be located when the pod was deleted. The container used to be Running\\\",\\\"reason\\\":\\\"ContainerStatusUnknown\\\",\\\"startedAt\\\":null}},\\\"name\\\":\\\"networking-console-plugin\\\",\\\"ready\\\":false,\\\"restartCount\\\":3,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"ContainerCreating\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/var/cert\\\",\\\"name\\\":\\\"networking-console-plugin-cert\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/etc/nginx/nginx.conf\\\",\\\"name\\\":\\\"nginx-conf\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}],\\\"podIP\\\":null,\\\"podIPs\\\":null}}\" for pod \"openshift-network-console\"/\"networking-console-plugin-85b44fc459-gdk6g\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://127.0.0.1:9743/pod?timeout=10s\": tls: failed to verify certificate: x509: certificate has expired or is not yet valid: current time 2025-12-03T11:20:33Z is after 2025-08-24T17:21:41Z" Dec 03 11:20:33 crc kubenswrapper[5052]: I1203 11:20:33.204882 5052 status_manager.go:875] "Failed to update status for pod" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"10c237bc-72e7-4da3-9f2c-b1688395bf6e\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"Initialized\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"message\\\":\\\"containers with unready status: [machine-config-daemon kube-rbac-proxy]\\\",\\\"reason\\\":\\\"ContainersNotReady\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"message\\\":\\\"containers with unready status: [machine-config-daemon kube-rbac-proxy]\\\",\\\"reason\\\":\\\"ContainersNotReady\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"ContainersReady\\\"}],\\\"containerStatuses\\\":[{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:242b3d66438c42745f4ef318bdeaf3d793426f12962a42ea83e18d06c08aaf09\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"kube-rbac-proxy\\\",\\\"ready\\\":false,\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"ContainerCreating\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/etc/tls/private\\\",\\\"name\\\":\\\"proxy-tls\\\"},{\\\"mountPath\\\":\\\"/etc/kube-rbac-proxy\\\",\\\"name\\\":\\\"mcd-auth-proxy-config\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-9vkjv\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c915fb8ba96e911699a1ae34a8e95ca8a9fbe1bf8c28fea177225c63a8bdfc0a\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"machine-config-daemon\\\",\\\"ready\\\":false,\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"ContainerCreating\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/rootfs\\\",\\\"name\\\":\\\"rootfs\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-9vkjv\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}],\\\"hostIP\\\":\\\"192.168.126.11\\\",\\\"hostIPs\\\":[{\\\"ip\\\":\\\"192.168.126.11\\\"}],\\\"podIP\\\":\\\"192.168.126.11\\\",\\\"podIPs\\\":[{\\\"ip\\\":\\\"192.168.126.11\\\"}],\\\"startTime\\\":\\\"2025-12-03T11:20:31Z\\\"}}\" for pod \"openshift-machine-config-operator\"/\"machine-config-daemon-74bj6\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://127.0.0.1:9743/pod?timeout=10s\": tls: failed to verify certificate: x509: certificate has expired or is not yet valid: current time 2025-12-03T11:20:33Z is after 2025-08-24T17:21:41Z" Dec 03 11:20:33 crc kubenswrapper[5052]: I1203 11:20:33.220153 5052 status_manager.go:875] "Failed to update status for pod" pod="openshift-network-operator/network-operator-58b4c7f79c-55gtf" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"37a5e44f-9a88-4405-be8a-b645485e7312\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:32Z\\\",\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:32Z\\\",\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:32Z\\\",\\\"type\\\":\\\"ContainersReady\\\"}],\\\"containerStatuses\\\":[{\\\"containerID\\\":\\\"cri-o://177058541a1de284ec4275037302f876e3345d48400f5edc9466508d43518412\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e1baa38811c04bd8909e01a1f3be7421a1cb99d608d3dc4cf86d95b17de2ab8b\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e1baa38811c04bd8909e01a1f3be7421a1cb99d608d3dc4cf86d95b17de2ab8b\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"network-operator\\\",\\\"ready\\\":true,\\\"restartCount\\\":0,\\\"started\\\":true,\\\"state\\\":{\\\"running\\\":{\\\"startedAt\\\":\\\"2025-12-03T11:20:32Z\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/etc/kubernetes\\\",\\\"name\\\":\\\"host-etc-kube\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/serving-cert\\\",\\\"name\\\":\\\"metrics-tls\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-rdwmf\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}]}}\" for pod \"openshift-network-operator\"/\"network-operator-58b4c7f79c-55gtf\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://127.0.0.1:9743/pod?timeout=10s\": tls: failed to verify certificate: x509: certificate has expired or is not yet valid: current time 2025-12-03T11:20:33Z is after 2025-08-24T17:21:41Z" Dec 03 11:20:33 crc kubenswrapper[5052]: I1203 11:20:33.234751 5052 status_manager.go:875] "Failed to update status for pod" pod="openshift-network-diagnostics/network-check-source-55646444c4-trplf" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"9d751cbb-f2e2-430d-9754-c882a5e924a5\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"message\\\":\\\"containers with unready status: [check-endpoints]\\\",\\\"reason\\\":\\\"ContainersNotReady\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"message\\\":\\\"containers with unready status: [check-endpoints]\\\",\\\"reason\\\":\\\"ContainersNotReady\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"ContainersReady\\\"}],\\\"containerStatuses\\\":[{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e1baa38811c04bd8909e01a1f3be7421a1cb99d608d3dc4cf86d95b17de2ab8b\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{\\\"terminated\\\":{\\\"exitCode\\\":137,\\\"finishedAt\\\":null,\\\"message\\\":\\\"The container could not be located when the pod was deleted. The container used to be Running\\\",\\\"reason\\\":\\\"ContainerStatusUnknown\\\",\\\"startedAt\\\":null}},\\\"name\\\":\\\"check-endpoints\\\",\\\"ready\\\":false,\\\"restartCount\\\":3,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"ContainerCreating\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-s2dwl\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}],\\\"podIP\\\":null,\\\"podIPs\\\":null}}\" for pod \"openshift-network-diagnostics\"/\"network-check-source-55646444c4-trplf\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://127.0.0.1:9743/pod?timeout=10s\": tls: failed to verify certificate: x509: certificate has expired or is not yet valid: current time 2025-12-03T11:20:33Z is after 2025-08-24T17:21:41Z" Dec 03 11:20:33 crc kubenswrapper[5052]: I1203 11:20:33.255502 5052 status_manager.go:875] "Failed to update status for pod" pod="openshift-network-operator/iptables-alerter-4ln5h" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"d75a4c96-2883-4a0b-bab2-0fab2b6c0b49\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"message\\\":\\\"containers with unready status: [iptables-alerter]\\\",\\\"reason\\\":\\\"ContainersNotReady\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"message\\\":\\\"containers with unready status: [iptables-alerter]\\\",\\\"reason\\\":\\\"ContainersNotReady\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"ContainersReady\\\"}],\\\"containerStatuses\\\":[{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:35512335ac39aed0f55b7f799f416f4f6445c20c1b19888cf2bb72bb276703f2\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{\\\"terminated\\\":{\\\"exitCode\\\":137,\\\"finishedAt\\\":null,\\\"message\\\":\\\"The container could not be located when the pod was deleted. The container used to be Running\\\",\\\"reason\\\":\\\"ContainerStatusUnknown\\\",\\\"startedAt\\\":null}},\\\"name\\\":\\\"iptables-alerter\\\",\\\"ready\\\":false,\\\"restartCount\\\":3,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"ContainerCreating\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/iptables-alerter\\\",\\\"name\\\":\\\"iptables-alerter-script\\\"},{\\\"mountPath\\\":\\\"/host\\\",\\\"name\\\":\\\"host-slash\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-rczfb\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}]}}\" for pod \"openshift-network-operator\"/\"iptables-alerter-4ln5h\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://127.0.0.1:9743/pod?timeout=10s\": tls: failed to verify certificate: x509: certificate has expired or is not yet valid: current time 2025-12-03T11:20:33Z is after 2025-08-24T17:21:41Z" Dec 03 11:20:33 crc kubenswrapper[5052]: I1203 11:20:33.280029 5052 status_manager.go:875] "Failed to update status for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kg55h" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"bc569bba-c020-40d6-af68-e617625fe0e4\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:32Z\\\",\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:32Z\\\",\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"Initialized\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"message\\\":\\\"containers with unready status: [ovn-controller ovn-acl-logging kube-rbac-proxy-node kube-rbac-proxy-ovn-metrics northd nbdb sbdb ovnkube-controller]\\\",\\\"reason\\\":\\\"ContainersNotReady\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"message\\\":\\\"containers with unready status: [ovn-controller ovn-acl-logging kube-rbac-proxy-node kube-rbac-proxy-ovn-metrics northd nbdb sbdb ovnkube-controller]\\\",\\\"reason\\\":\\\"ContainersNotReady\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"ContainersReady\\\"}],\\\"containerStatuses\\\":[{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:242b3d66438c42745f4ef318bdeaf3d793426f12962a42ea83e18d06c08aaf09\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"kube-rbac-proxy-node\\\",\\\"ready\\\":false,\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"PodInitializing\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/ovnkube-lib\\\",\\\"name\\\":\\\"ovnkube-script-lib\\\"},{\\\"mountPath\\\":\\\"/etc/pki/tls/metrics-cert\\\",\\\"name\\\":\\\"ovn-node-metrics-cert\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-x92h2\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:242b3d66438c42745f4ef318bdeaf3d793426f12962a42ea83e18d06c08aaf09\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"kube-rbac-proxy-ovn-metrics\\\",\\\"ready\\\":false,\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"PodInitializing\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/ovnkube-lib\\\",\\\"name\\\":\\\"ovnkube-script-lib\\\"},{\\\"mountPath\\\":\\\"/etc/pki/tls/metrics-cert\\\",\\\"name\\\":\\\"ovn-node-metrics-cert\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-x92h2\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:174f36cdd47ef0d1d2099482919d773257453265a2af0b17b154edc32fa41ac2\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"nbdb\\\",\\\"ready\\\":false,\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"PodInitializing\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/ovnkube-lib\\\",\\\"name\\\":\\\"ovnkube-script-lib\\\"},{\\\"mountPath\\\":\\\"/etc/ovn/\\\",\\\"name\\\":\\\"etc-openvswitch\\\"},{\\\"mountPath\\\":\\\"/var/log/ovn\\\",\\\"name\\\":\\\"node-log\\\"},{\\\"mountPath\\\":\\\"/run/ovn/\\\",\\\"name\\\":\\\"run-ovn\\\"},{\\\"mountPath\\\":\\\"/env\\\",\\\"name\\\":\\\"env-overrides\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-x92h2\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:174f36cdd47ef0d1d2099482919d773257453265a2af0b17b154edc32fa41ac2\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"northd\\\",\\\"ready\\\":false,\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"PodInitializing\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/ovnkube-lib\\\",\\\"name\\\":\\\"ovnkube-script-lib\\\"},{\\\"mountPath\\\":\\\"/etc/ovn\\\",\\\"name\\\":\\\"etc-openvswitch\\\"},{\\\"mountPath\\\":\\\"/var/log/ovn\\\",\\\"name\\\":\\\"node-log\\\"},{\\\"mountPath\\\":\\\"/run/ovn/\\\",\\\"name\\\":\\\"run-ovn\\\"},{\\\"mountPath\\\":\\\"/env\\\",\\\"name\\\":\\\"env-overrides\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-x92h2\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:174f36cdd47ef0d1d2099482919d773257453265a2af0b17b154edc32fa41ac2\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"ovn-acl-logging\\\",\\\"ready\\\":false,\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"PodInitializing\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/ovnkube-lib\\\",\\\"name\\\":\\\"ovnkube-script-lib\\\"},{\\\"mountPath\\\":\\\"/var/log/ovn/\\\",\\\"name\\\":\\\"node-log\\\"},{\\\"mountPath\\\":\\\"/run/ovn/\\\",\\\"name\\\":\\\"run-ovn\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-x92h2\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:174f36cdd47ef0d1d2099482919d773257453265a2af0b17b154edc32fa41ac2\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"ovn-controller\\\",\\\"ready\\\":false,\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"PodInitializing\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/ovnkube-lib\\\",\\\"name\\\":\\\"ovnkube-script-lib\\\"},{\\\"mountPath\\\":\\\"/run/openvswitch\\\",\\\"name\\\":\\\"run-openvswitch\\\"},{\\\"mountPath\\\":\\\"/run/ovn/\\\",\\\"name\\\":\\\"run-ovn\\\"},{\\\"mountPath\\\":\\\"/etc/openvswitch\\\",\\\"name\\\":\\\"etc-openvswitch\\\"},{\\\"mountPath\\\":\\\"/etc/ovn/\\\",\\\"name\\\":\\\"etc-openvswitch\\\"},{\\\"mountPath\\\":\\\"/var/lib/openvswitch\\\",\\\"name\\\":\\\"var-lib-openvswitch\\\"},{\\\"mountPath\\\":\\\"/env\\\",\\\"name\\\":\\\"env-overrides\\\"},{\\\"mountPath\\\":\\\"/var/log/ovn/\\\",\\\"name\\\":\\\"node-log\\\"},{\\\"mountPath\\\":\\\"/dev/log\\\",\\\"name\\\":\\\"log-socket\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-x92h2\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:174f36cdd47ef0d1d2099482919d773257453265a2af0b17b154edc32fa41ac2\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"ovnkube-controller\\\",\\\"ready\\\":false,\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"PodInitializing\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/ovnkube-lib\\\",\\\"name\\\":\\\"ovnkube-script-lib\\\"},{\\\"mountPath\\\":\\\"/var/lib/kubelet\\\",\\\"name\\\":\\\"host-kubelet\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/etc/systemd/system\\\",\\\"name\\\":\\\"systemd-units\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/host\\\",\\\"name\\\":\\\"host-slash\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/run/ovn-kubernetes/\\\",\\\"name\\\":\\\"host-run-ovn-kubernetes\\\"},{\\\"mountPath\\\":\\\"/run/netns\\\",\\\"name\\\":\\\"host-run-netns\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/run/systemd/private\\\",\\\"name\\\":\\\"run-systemd\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/cni-bin-dir\\\",\\\"name\\\":\\\"host-cni-bin\\\"},{\\\"mountPath\\\":\\\"/etc/cni/net.d\\\",\\\"name\\\":\\\"host-cni-netd\\\"},{\\\"mountPath\\\":\\\"/var/lib/cni/networks/ovn-k8s-cni-overlay\\\",\\\"name\\\":\\\"host-var-lib-cni-networks-ovn-kubernetes\\\"},{\\\"mountPath\\\":\\\"/run/openvswitch\\\",\\\"name\\\":\\\"run-openvswitch\\\"},{\\\"mountPath\\\":\\\"/var/log/ovnkube/\\\",\\\"name\\\":\\\"etc-openvswitch\\\"},{\\\"mountPath\\\":\\\"/run/ovn/\\\",\\\"name\\\":\\\"run-ovn\\\"},{\\\"mountPath\\\":\\\"/etc/openvswitch\\\",\\\"name\\\":\\\"etc-openvswitch\\\"},{\\\"mountPath\\\":\\\"/etc/ovn/\\\",\\\"name\\\":\\\"etc-openvswitch\\\"},{\\\"mountPath\\\":\\\"/var/lib/openvswitch\\\",\\\"name\\\":\\\"var-lib-openvswitch\\\"},{\\\"mountPath\\\":\\\"/run/ovnkube-config/\\\",\\\"name\\\":\\\"ovnkube-config\\\"},{\\\"mountPath\\\":\\\"/env\\\",\\\"name\\\":\\\"env-overrides\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-x92h2\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:174f36cdd47ef0d1d2099482919d773257453265a2af0b17b154edc32fa41ac2\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"sbdb\\\",\\\"ready\\\":false,\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"PodInitializing\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/ovnkube-lib\\\",\\\"name\\\":\\\"ovnkube-script-lib\\\"},{\\\"mountPath\\\":\\\"/etc/ovn/\\\",\\\"name\\\":\\\"etc-openvswitch\\\"},{\\\"mountPath\\\":\\\"/run/ovn/\\\",\\\"name\\\":\\\"run-ovn\\\"},{\\\"mountPath\\\":\\\"/var/log/ovn\\\",\\\"name\\\":\\\"node-log\\\"},{\\\"mountPath\\\":\\\"/env\\\",\\\"name\\\":\\\"env-overrides\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-x92h2\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}],\\\"hostIP\\\":\\\"192.168.126.11\\\",\\\"hostIPs\\\":[{\\\"ip\\\":\\\"192.168.126.11\\\"}],\\\"initContainerStatuses\\\":[{\\\"containerID\\\":\\\"cri-o://4def7c1172d2ae52e275b2371cf88c9f93c3f0c747c18de7d1073d50b03b19bc\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:174f36cdd47ef0d1d2099482919d773257453265a2af0b17b154edc32fa41ac2\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:174f36cdd47ef0d1d2099482919d773257453265a2af0b17b154edc32fa41ac2\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"kubecfg-setup\\\",\\\"ready\\\":true,\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://4def7c1172d2ae52e275b2371cf88c9f93c3f0c747c18de7d1073d50b03b19bc\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2025-12-03T11:20:32Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2025-12-03T11:20:32Z\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/etc/ovn/\\\",\\\"name\\\":\\\"etc-openvswitch\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-x92h2\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}],\\\"podIP\\\":\\\"192.168.126.11\\\",\\\"podIPs\\\":[{\\\"ip\\\":\\\"192.168.126.11\\\"}],\\\"startTime\\\":\\\"2025-12-03T11:20:31Z\\\"}}\" for pod \"openshift-ovn-kubernetes\"/\"ovnkube-node-kg55h\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://127.0.0.1:9743/pod?timeout=10s\": tls: failed to verify certificate: x509: certificate has expired or is not yet valid: current time 2025-12-03T11:20:33Z is after 2025-08-24T17:21:41Z" Dec 03 11:20:33 crc kubenswrapper[5052]: I1203 11:20:33.308664 5052 status_manager.go:875] "Failed to update status for pod" pod="openshift-kube-controller-manager/kube-controller-manager-crc" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"f31e1704-9c1d-4044-b318-2aaccb5896e2\\\"},\\\"status\\\":{\\\"conditions\\\":[{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:15Z\\\",\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:09Z\\\",\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"Initialized\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"ContainersReady\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:09Z\\\",\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"PodScheduled\\\"}],\\\"containerStatuses\\\":[{\\\"containerID\\\":\\\"cri-o://37485762ea1808cd3794c9641a4ead224b107e6c64c295de237f8ffd50aba4f4\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c0f9da410c07372b6c9ad6a79379b491cd10fdee88051c026b084652d85aed21\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c0f9da410c07372b6c9ad6a79379b491cd10fdee88051c026b084652d85aed21\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"cluster-policy-controller\\\",\\\"ready\\\":true,\\\"restartCount\\\":0,\\\"started\\\":true,\\\"state\\\":{\\\"running\\\":{\\\"startedAt\\\":\\\"2025-12-03T11:20:12Z\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/etc/kubernetes/static-pod-resources\\\",\\\"name\\\":\\\"resource-dir\\\"},{\\\"mountPath\\\":\\\"/etc/kubernetes/static-pod-certs\\\",\\\"name\\\":\\\"cert-dir\\\"}]},{\\\"containerID\\\":\\\"cri-o://f5d23c6160b5790e9272bd5ae83472267f4d1f60b67e0425a6bc2f5c1f9d1fb7\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:06bc35825771aee1220d34720243b89c4ba8a8b335e6de2597126bd791fd90d4\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:06bc35825771aee1220d34720243b89c4ba8a8b335e6de2597126bd791fd90d4\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"kube-controller-manager\\\",\\\"ready\\\":true,\\\"restartCount\\\":0,\\\"started\\\":true,\\\"state\\\":{\\\"running\\\":{\\\"startedAt\\\":\\\"2025-12-03T11:20:11Z\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/etc/kubernetes/static-pod-resources\\\",\\\"name\\\":\\\"resource-dir\\\"},{\\\"mountPath\\\":\\\"/etc/kubernetes/static-pod-certs\\\",\\\"name\\\":\\\"cert-dir\\\"}]},{\\\"containerID\\\":\\\"cri-o://d05df6945d11d2abe73005efafe80220497a339008d4c28f7923c2dc7ac4a150\\\",\\\"image\\\":\\\"quay.io/crcont/openshift-crc-cluster-kube-controller-manager-operator@sha256:8506ce0a578bc18fac117eb2b82799488ffac0bed08287faaf92edaf5d17ab95\\\",\\\"imageID\\\":\\\"quay.io/crcont/openshift-crc-cluster-kube-controller-manager-operator@sha256:8506ce0a578bc18fac117eb2b82799488ffac0bed08287faaf92edaf5d17ab95\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"kube-controller-manager-cert-syncer\\\",\\\"ready\\\":true,\\\"restartCount\\\":0,\\\"started\\\":true,\\\"state\\\":{\\\"running\\\":{\\\"startedAt\\\":\\\"2025-12-03T11:20:12Z\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/etc/kubernetes/static-pod-resources\\\",\\\"name\\\":\\\"resource-dir\\\"},{\\\"mountPath\\\":\\\"/etc/kubernetes/static-pod-certs\\\",\\\"name\\\":\\\"cert-dir\\\"}]},{\\\"containerID\\\":\\\"cri-o://8df2aa2470046bba37b41cb301e96f52b09b83124384b5dbc4942cd0bfeef1c6\\\",\\\"image\\\":\\\"quay.io/crcont/openshift-crc-cluster-kube-controller-manager-operator@sha256:8506ce0a578bc18fac117eb2b82799488ffac0bed08287faaf92edaf5d17ab95\\\",\\\"imageID\\\":\\\"quay.io/crcont/openshift-crc-cluster-kube-controller-manager-operator@sha256:8506ce0a578bc18fac117eb2b82799488ffac0bed08287faaf92edaf5d17ab95\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"kube-controller-manager-recovery-controller\\\",\\\"ready\\\":true,\\\"restartCount\\\":0,\\\"started\\\":true,\\\"state\\\":{\\\"running\\\":{\\\"startedAt\\\":\\\"2025-12-03T11:20:15Z\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/etc/kubernetes/static-pod-resources\\\",\\\"name\\\":\\\"resource-dir\\\"},{\\\"mountPath\\\":\\\"/etc/kubernetes/static-pod-certs\\\",\\\"name\\\":\\\"cert-dir\\\"}]}],\\\"hostIP\\\":\\\"192.168.126.11\\\",\\\"hostIPs\\\":[{\\\"ip\\\":\\\"192.168.126.11\\\"}],\\\"phase\\\":\\\"Running\\\",\\\"podIP\\\":\\\"192.168.126.11\\\",\\\"podIPs\\\":[{\\\"ip\\\":\\\"192.168.126.11\\\"}],\\\"startTime\\\":\\\"2025-12-03T11:20:09Z\\\"}}\" for pod \"openshift-kube-controller-manager\"/\"kube-controller-manager-crc\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://127.0.0.1:9743/pod?timeout=10s\": tls: failed to verify certificate: x509: certificate has expired or is not yet valid: current time 2025-12-03T11:20:33Z is after 2025-08-24T17:21:41Z" Dec 03 11:20:33 crc kubenswrapper[5052]: I1203 11:20:33.331869 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\") pod \"8f668bae-612b-4b75-9490-919e737c6a3b\" (UID: \"8f668bae-612b-4b75-9490-919e737c6a3b\") " Dec 03 11:20:33 crc kubenswrapper[5052]: I1203 11:20:33.331987 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-s2dwl\" (UniqueName: \"kubernetes.io/projected/9d751cbb-f2e2-430d-9754-c882a5e924a5-kube-api-access-s2dwl\") pod \"network-check-source-55646444c4-trplf\" (UID: \"9d751cbb-f2e2-430d-9754-c882a5e924a5\") " pod="openshift-network-diagnostics/network-check-source-55646444c4-trplf" Dec 03 11:20:33 crc kubenswrapper[5052]: E1203 11:20:33.332014 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8 podName:8f668bae-612b-4b75-9490-919e737c6a3b nodeName:}" failed. No retries permitted until 2025-12-03 11:20:35.331990765 +0000 UTC m=+25.920232529 (durationBeforeRetry 2s). Error: UnmountVolume.TearDown failed for volume "pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8" (UniqueName: "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8") pod "8f668bae-612b-4b75-9490-919e737c6a3b" (UID: "8f668bae-612b-4b75-9490-919e737c6a3b") : kubernetes.io/csi: Unmounter.TearDownAt failed to get CSI client: driver name kubevirt.io.hostpath-provisioner not found in the list of registered CSI drivers Dec 03 11:20:33 crc kubenswrapper[5052]: I1203 11:20:33.332039 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/5fe485a1-e14f-4c09-b5b9-f252bc42b7e8-nginx-conf\") pod \"networking-console-plugin-85b44fc459-gdk6g\" (UID: \"5fe485a1-e14f-4c09-b5b9-f252bc42b7e8\") " pod="openshift-network-console/networking-console-plugin-85b44fc459-gdk6g" Dec 03 11:20:33 crc kubenswrapper[5052]: I1203 11:20:33.332066 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-cqllr\" (UniqueName: \"kubernetes.io/projected/3b6479f0-333b-4a96-9adf-2099afdc2447-kube-api-access-cqllr\") pod \"network-check-target-xd92c\" (UID: \"3b6479f0-333b-4a96-9adf-2099afdc2447\") " pod="openshift-network-diagnostics/network-check-target-xd92c" Dec 03 11:20:33 crc kubenswrapper[5052]: I1203 11:20:33.332090 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/5fe485a1-e14f-4c09-b5b9-f252bc42b7e8-networking-console-plugin-cert\") pod \"networking-console-plugin-85b44fc459-gdk6g\" (UID: \"5fe485a1-e14f-4c09-b5b9-f252bc42b7e8\") " pod="openshift-network-console/networking-console-plugin-85b44fc459-gdk6g" Dec 03 11:20:33 crc kubenswrapper[5052]: E1203 11:20:33.332147 5052 projected.go:288] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Dec 03 11:20:33 crc kubenswrapper[5052]: E1203 11:20:33.332158 5052 secret.go:188] Couldn't get secret openshift-network-console/networking-console-plugin-cert: object "openshift-network-console"/"networking-console-plugin-cert" not registered Dec 03 11:20:33 crc kubenswrapper[5052]: E1203 11:20:33.332164 5052 projected.go:288] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Dec 03 11:20:33 crc kubenswrapper[5052]: E1203 11:20:33.332180 5052 projected.go:194] Error preparing data for projected volume kube-api-access-s2dwl for pod openshift-network-diagnostics/network-check-source-55646444c4-trplf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Dec 03 11:20:33 crc kubenswrapper[5052]: E1203 11:20:33.332191 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5fe485a1-e14f-4c09-b5b9-f252bc42b7e8-networking-console-plugin-cert podName:5fe485a1-e14f-4c09-b5b9-f252bc42b7e8 nodeName:}" failed. No retries permitted until 2025-12-03 11:20:35.332184841 +0000 UTC m=+25.920426605 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/5fe485a1-e14f-4c09-b5b9-f252bc42b7e8-networking-console-plugin-cert") pod "networking-console-plugin-85b44fc459-gdk6g" (UID: "5fe485a1-e14f-4c09-b5b9-f252bc42b7e8") : object "openshift-network-console"/"networking-console-plugin-cert" not registered Dec 03 11:20:33 crc kubenswrapper[5052]: E1203 11:20:33.332216 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9d751cbb-f2e2-430d-9754-c882a5e924a5-kube-api-access-s2dwl podName:9d751cbb-f2e2-430d-9754-c882a5e924a5 nodeName:}" failed. No retries permitted until 2025-12-03 11:20:35.332205711 +0000 UTC m=+25.920447475 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-s2dwl" (UniqueName: "kubernetes.io/projected/9d751cbb-f2e2-430d-9754-c882a5e924a5-kube-api-access-s2dwl") pod "network-check-source-55646444c4-trplf" (UID: "9d751cbb-f2e2-430d-9754-c882a5e924a5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Dec 03 11:20:33 crc kubenswrapper[5052]: E1203 11:20:33.332258 5052 configmap.go:193] Couldn't get configMap openshift-network-console/networking-console-plugin: object "openshift-network-console"/"networking-console-plugin" not registered Dec 03 11:20:33 crc kubenswrapper[5052]: E1203 11:20:33.332365 5052 projected.go:288] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Dec 03 11:20:33 crc kubenswrapper[5052]: E1203 11:20:33.332402 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5fe485a1-e14f-4c09-b5b9-f252bc42b7e8-nginx-conf podName:5fe485a1-e14f-4c09-b5b9-f252bc42b7e8 nodeName:}" failed. No retries permitted until 2025-12-03 11:20:35.332369856 +0000 UTC m=+25.920611620 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "nginx-conf" (UniqueName: "kubernetes.io/configmap/5fe485a1-e14f-4c09-b5b9-f252bc42b7e8-nginx-conf") pod "networking-console-plugin-85b44fc459-gdk6g" (UID: "5fe485a1-e14f-4c09-b5b9-f252bc42b7e8") : object "openshift-network-console"/"networking-console-plugin" not registered Dec 03 11:20:33 crc kubenswrapper[5052]: E1203 11:20:33.332404 5052 projected.go:288] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Dec 03 11:20:33 crc kubenswrapper[5052]: E1203 11:20:33.332435 5052 projected.go:194] Error preparing data for projected volume kube-api-access-cqllr for pod openshift-network-diagnostics/network-check-target-xd92c: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Dec 03 11:20:33 crc kubenswrapper[5052]: E1203 11:20:33.332468 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3b6479f0-333b-4a96-9adf-2099afdc2447-kube-api-access-cqllr podName:3b6479f0-333b-4a96-9adf-2099afdc2447 nodeName:}" failed. No retries permitted until 2025-12-03 11:20:35.332461018 +0000 UTC m=+25.920703062 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-cqllr" (UniqueName: "kubernetes.io/projected/3b6479f0-333b-4a96-9adf-2099afdc2447-kube-api-access-cqllr") pod "network-check-target-xd92c" (UID: "3b6479f0-333b-4a96-9adf-2099afdc2447") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Dec 03 11:20:33 crc kubenswrapper[5052]: I1203 11:20:33.673001 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-55646444c4-trplf" Dec 03 11:20:33 crc kubenswrapper[5052]: I1203 11:20:33.673005 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-85b44fc459-gdk6g" Dec 03 11:20:33 crc kubenswrapper[5052]: E1203 11:20:33.673223 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-source-55646444c4-trplf" podUID="9d751cbb-f2e2-430d-9754-c882a5e924a5" Dec 03 11:20:33 crc kubenswrapper[5052]: E1203 11:20:33.673335 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-console/networking-console-plugin-85b44fc459-gdk6g" podUID="5fe485a1-e14f-4c09-b5b9-f252bc42b7e8" Dec 03 11:20:33 crc kubenswrapper[5052]: I1203 11:20:33.678680 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="01ab3dd5-8196-46d0-ad33-122e2ca51def" path="/var/lib/kubelet/pods/01ab3dd5-8196-46d0-ad33-122e2ca51def/volumes" Dec 03 11:20:33 crc kubenswrapper[5052]: I1203 11:20:33.680143 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="09ae3b1a-e8e7-4524-b54b-61eab6f9239a" path="/var/lib/kubelet/pods/09ae3b1a-e8e7-4524-b54b-61eab6f9239a/volumes" Dec 03 11:20:33 crc kubenswrapper[5052]: I1203 11:20:33.684170 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="09efc573-dbb6-4249-bd59-9b87aba8dd28" path="/var/lib/kubelet/pods/09efc573-dbb6-4249-bd59-9b87aba8dd28/volumes" Dec 03 11:20:33 crc kubenswrapper[5052]: I1203 11:20:33.685700 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0b574797-001e-440a-8f4e-c0be86edad0f" path="/var/lib/kubelet/pods/0b574797-001e-440a-8f4e-c0be86edad0f/volumes" Dec 03 11:20:33 crc kubenswrapper[5052]: I1203 11:20:33.687089 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0b78653f-4ff9-4508-8672-245ed9b561e3" path="/var/lib/kubelet/pods/0b78653f-4ff9-4508-8672-245ed9b561e3/volumes" Dec 03 11:20:33 crc kubenswrapper[5052]: I1203 11:20:33.687903 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1386a44e-36a2-460c-96d0-0359d2b6f0f5" path="/var/lib/kubelet/pods/1386a44e-36a2-460c-96d0-0359d2b6f0f5/volumes" Dec 03 11:20:33 crc kubenswrapper[5052]: I1203 11:20:33.688622 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1bf7eb37-55a3-4c65-b768-a94c82151e69" path="/var/lib/kubelet/pods/1bf7eb37-55a3-4c65-b768-a94c82151e69/volumes" Dec 03 11:20:33 crc kubenswrapper[5052]: I1203 11:20:33.689775 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1d611f23-29be-4491-8495-bee1670e935f" path="/var/lib/kubelet/pods/1d611f23-29be-4491-8495-bee1670e935f/volumes" Dec 03 11:20:33 crc kubenswrapper[5052]: I1203 11:20:33.690459 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="20b0d48f-5fd6-431c-a545-e3c800c7b866" path="/var/lib/kubelet/pods/20b0d48f-5fd6-431c-a545-e3c800c7b866/volumes" Dec 03 11:20:33 crc kubenswrapper[5052]: I1203 11:20:33.693425 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="210d8245-ebfc-4e3b-ac4a-e21ce76f9a7c" path="/var/lib/kubelet/pods/210d8245-ebfc-4e3b-ac4a-e21ce76f9a7c/volumes" Dec 03 11:20:33 crc kubenswrapper[5052]: I1203 11:20:33.694335 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="22c825df-677d-4ca6-82db-3454ed06e783" path="/var/lib/kubelet/pods/22c825df-677d-4ca6-82db-3454ed06e783/volumes" Dec 03 11:20:33 crc kubenswrapper[5052]: I1203 11:20:33.695997 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="25e176fe-21b4-4974-b1ed-c8b94f112a7f" path="/var/lib/kubelet/pods/25e176fe-21b4-4974-b1ed-c8b94f112a7f/volumes" Dec 03 11:20:33 crc kubenswrapper[5052]: I1203 11:20:33.697372 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="308be0ea-9f5f-4b29-aeb1-5abd31a0b17b" path="/var/lib/kubelet/pods/308be0ea-9f5f-4b29-aeb1-5abd31a0b17b/volumes" Dec 03 11:20:33 crc kubenswrapper[5052]: I1203 11:20:33.699252 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="31d8b7a1-420e-4252-a5b7-eebe8a111292" path="/var/lib/kubelet/pods/31d8b7a1-420e-4252-a5b7-eebe8a111292/volumes" Dec 03 11:20:33 crc kubenswrapper[5052]: I1203 11:20:33.700814 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3ab1a177-2de0-46d9-b765-d0d0649bb42e" path="/var/lib/kubelet/pods/3ab1a177-2de0-46d9-b765-d0d0649bb42e/volumes" Dec 03 11:20:33 crc kubenswrapper[5052]: I1203 11:20:33.701486 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3cb93b32-e0ae-4377-b9c8-fdb9842c6d59" path="/var/lib/kubelet/pods/3cb93b32-e0ae-4377-b9c8-fdb9842c6d59/volumes" Dec 03 11:20:33 crc kubenswrapper[5052]: I1203 11:20:33.702916 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="43509403-f426-496e-be36-56cef71462f5" path="/var/lib/kubelet/pods/43509403-f426-496e-be36-56cef71462f5/volumes" Dec 03 11:20:33 crc kubenswrapper[5052]: I1203 11:20:33.703353 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="44663579-783b-4372-86d6-acf235a62d72" path="/var/lib/kubelet/pods/44663579-783b-4372-86d6-acf235a62d72/volumes" Dec 03 11:20:33 crc kubenswrapper[5052]: I1203 11:20:33.704301 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="496e6271-fb68-4057-954e-a0d97a4afa3f" path="/var/lib/kubelet/pods/496e6271-fb68-4057-954e-a0d97a4afa3f/volumes" Dec 03 11:20:33 crc kubenswrapper[5052]: I1203 11:20:33.705612 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="49c341d1-5089-4bc2-86a0-a5e165cfcc6b" path="/var/lib/kubelet/pods/49c341d1-5089-4bc2-86a0-a5e165cfcc6b/volumes" Dec 03 11:20:33 crc kubenswrapper[5052]: I1203 11:20:33.706143 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="49ef4625-1d3a-4a9f-b595-c2433d32326d" path="/var/lib/kubelet/pods/49ef4625-1d3a-4a9f-b595-c2433d32326d/volumes" Dec 03 11:20:33 crc kubenswrapper[5052]: I1203 11:20:33.707201 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4bb40260-dbaa-4fb0-84df-5e680505d512" path="/var/lib/kubelet/pods/4bb40260-dbaa-4fb0-84df-5e680505d512/volumes" Dec 03 11:20:33 crc kubenswrapper[5052]: I1203 11:20:33.707814 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5225d0e4-402f-4861-b410-819f433b1803" path="/var/lib/kubelet/pods/5225d0e4-402f-4861-b410-819f433b1803/volumes" Dec 03 11:20:33 crc kubenswrapper[5052]: I1203 11:20:33.708876 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5441d097-087c-4d9a-baa8-b210afa90fc9" path="/var/lib/kubelet/pods/5441d097-087c-4d9a-baa8-b210afa90fc9/volumes" Dec 03 11:20:33 crc kubenswrapper[5052]: I1203 11:20:33.709349 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="57a731c4-ef35-47a8-b875-bfb08a7f8011" path="/var/lib/kubelet/pods/57a731c4-ef35-47a8-b875-bfb08a7f8011/volumes" Dec 03 11:20:33 crc kubenswrapper[5052]: I1203 11:20:33.710001 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5b88f790-22fa-440e-b583-365168c0b23d" path="/var/lib/kubelet/pods/5b88f790-22fa-440e-b583-365168c0b23d/volumes" Dec 03 11:20:33 crc kubenswrapper[5052]: I1203 11:20:33.711183 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5fe579f8-e8a6-4643-bce5-a661393c4dde" path="/var/lib/kubelet/pods/5fe579f8-e8a6-4643-bce5-a661393c4dde/volumes" Dec 03 11:20:33 crc kubenswrapper[5052]: I1203 11:20:33.711748 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6402fda4-df10-493c-b4e5-d0569419652d" path="/var/lib/kubelet/pods/6402fda4-df10-493c-b4e5-d0569419652d/volumes" Dec 03 11:20:33 crc kubenswrapper[5052]: I1203 11:20:33.712817 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6509e943-70c6-444c-bc41-48a544e36fbd" path="/var/lib/kubelet/pods/6509e943-70c6-444c-bc41-48a544e36fbd/volumes" Dec 03 11:20:33 crc kubenswrapper[5052]: I1203 11:20:33.713353 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6731426b-95fe-49ff-bb5f-40441049fde2" path="/var/lib/kubelet/pods/6731426b-95fe-49ff-bb5f-40441049fde2/volumes" Dec 03 11:20:33 crc kubenswrapper[5052]: I1203 11:20:33.714445 5052 kubelet_volumes.go:152] "Cleaned up orphaned volume subpath from pod" podUID="6ea678ab-3438-413e-bfe3-290ae7725660" path="/var/lib/kubelet/pods/6ea678ab-3438-413e-bfe3-290ae7725660/volume-subpaths/run-systemd/ovnkube-controller/6" Dec 03 11:20:33 crc kubenswrapper[5052]: I1203 11:20:33.714610 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6ea678ab-3438-413e-bfe3-290ae7725660" path="/var/lib/kubelet/pods/6ea678ab-3438-413e-bfe3-290ae7725660/volumes" Dec 03 11:20:33 crc kubenswrapper[5052]: I1203 11:20:33.716646 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7539238d-5fe0-46ed-884e-1c3b566537ec" path="/var/lib/kubelet/pods/7539238d-5fe0-46ed-884e-1c3b566537ec/volumes" Dec 03 11:20:33 crc kubenswrapper[5052]: I1203 11:20:33.717795 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7583ce53-e0fe-4a16-9e4d-50516596a136" path="/var/lib/kubelet/pods/7583ce53-e0fe-4a16-9e4d-50516596a136/volumes" Dec 03 11:20:33 crc kubenswrapper[5052]: I1203 11:20:33.718306 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7bb08738-c794-4ee8-9972-3a62ca171029" path="/var/lib/kubelet/pods/7bb08738-c794-4ee8-9972-3a62ca171029/volumes" Dec 03 11:20:33 crc kubenswrapper[5052]: I1203 11:20:33.719943 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="87cf06ed-a83f-41a7-828d-70653580a8cb" path="/var/lib/kubelet/pods/87cf06ed-a83f-41a7-828d-70653580a8cb/volumes" Dec 03 11:20:33 crc kubenswrapper[5052]: I1203 11:20:33.720729 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8cea82b4-6893-4ddc-af9f-1bb5ae425c5b" path="/var/lib/kubelet/pods/8cea82b4-6893-4ddc-af9f-1bb5ae425c5b/volumes" Dec 03 11:20:33 crc kubenswrapper[5052]: I1203 11:20:33.721794 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="925f1c65-6136-48ba-85aa-3a3b50560753" path="/var/lib/kubelet/pods/925f1c65-6136-48ba-85aa-3a3b50560753/volumes" Dec 03 11:20:33 crc kubenswrapper[5052]: I1203 11:20:33.723446 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="96b93a3a-6083-4aea-8eab-fe1aa8245ad9" path="/var/lib/kubelet/pods/96b93a3a-6083-4aea-8eab-fe1aa8245ad9/volumes" Dec 03 11:20:33 crc kubenswrapper[5052]: I1203 11:20:33.725617 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9d4552c7-cd75-42dd-8880-30dd377c49a4" path="/var/lib/kubelet/pods/9d4552c7-cd75-42dd-8880-30dd377c49a4/volumes" Dec 03 11:20:33 crc kubenswrapper[5052]: I1203 11:20:33.726286 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a0128f3a-b052-44ed-a84e-c4c8aaf17c13" path="/var/lib/kubelet/pods/a0128f3a-b052-44ed-a84e-c4c8aaf17c13/volumes" Dec 03 11:20:33 crc kubenswrapper[5052]: I1203 11:20:33.727434 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a31745f5-9847-4afe-82a5-3161cc66ca93" path="/var/lib/kubelet/pods/a31745f5-9847-4afe-82a5-3161cc66ca93/volumes" Dec 03 11:20:33 crc kubenswrapper[5052]: I1203 11:20:33.728200 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b11524ee-3fca-4b1b-9cdf-6da289fdbc7d" path="/var/lib/kubelet/pods/b11524ee-3fca-4b1b-9cdf-6da289fdbc7d/volumes" Dec 03 11:20:33 crc kubenswrapper[5052]: I1203 11:20:33.729638 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b6312bbd-5731-4ea0-a20f-81d5a57df44a" path="/var/lib/kubelet/pods/b6312bbd-5731-4ea0-a20f-81d5a57df44a/volumes" Dec 03 11:20:33 crc kubenswrapper[5052]: I1203 11:20:33.730367 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b6cd30de-2eeb-49a2-ab40-9167f4560ff5" path="/var/lib/kubelet/pods/b6cd30de-2eeb-49a2-ab40-9167f4560ff5/volumes" Dec 03 11:20:33 crc kubenswrapper[5052]: I1203 11:20:33.731446 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bc5039c0-ea34-426b-a2b7-fbbc87b49a6d" path="/var/lib/kubelet/pods/bc5039c0-ea34-426b-a2b7-fbbc87b49a6d/volumes" Dec 03 11:20:33 crc kubenswrapper[5052]: I1203 11:20:33.732451 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bd23aa5c-e532-4e53-bccf-e79f130c5ae8" path="/var/lib/kubelet/pods/bd23aa5c-e532-4e53-bccf-e79f130c5ae8/volumes" Dec 03 11:20:33 crc kubenswrapper[5052]: I1203 11:20:33.733961 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bf126b07-da06-4140-9a57-dfd54fc6b486" path="/var/lib/kubelet/pods/bf126b07-da06-4140-9a57-dfd54fc6b486/volumes" Dec 03 11:20:33 crc kubenswrapper[5052]: I1203 11:20:33.734554 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c03ee662-fb2f-4fc4-a2c1-af487c19d254" path="/var/lib/kubelet/pods/c03ee662-fb2f-4fc4-a2c1-af487c19d254/volumes" Dec 03 11:20:33 crc kubenswrapper[5052]: I1203 11:20:33.735929 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cd70aa09-68dd-4d64-bd6f-156fe6d1dc6d" path="/var/lib/kubelet/pods/cd70aa09-68dd-4d64-bd6f-156fe6d1dc6d/volumes" Dec 03 11:20:33 crc kubenswrapper[5052]: I1203 11:20:33.736697 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e7e6199b-1264-4501-8953-767f51328d08" path="/var/lib/kubelet/pods/e7e6199b-1264-4501-8953-767f51328d08/volumes" Dec 03 11:20:33 crc kubenswrapper[5052]: I1203 11:20:33.738568 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="efdd0498-1daa-4136-9a4a-3b948c2293fc" path="/var/lib/kubelet/pods/efdd0498-1daa-4136-9a4a-3b948c2293fc/volumes" Dec 03 11:20:33 crc kubenswrapper[5052]: I1203 11:20:33.739500 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f88749ec-7931-4ee7-b3fc-1ec5e11f92e9" path="/var/lib/kubelet/pods/f88749ec-7931-4ee7-b3fc-1ec5e11f92e9/volumes" Dec 03 11:20:33 crc kubenswrapper[5052]: I1203 11:20:33.740120 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fda69060-fa79-4696-b1a6-7980f124bf7c" path="/var/lib/kubelet/pods/fda69060-fa79-4696-b1a6-7980f124bf7c/volumes" Dec 03 11:20:33 crc kubenswrapper[5052]: I1203 11:20:33.853181 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kg55h" event={"ID":"bc569bba-c020-40d6-af68-e617625fe0e4","Type":"ContainerStarted","Data":"2363c55c03e9159893688a52c5ba318a997f018c828f223626074a58918ccc24"} Dec 03 11:20:33 crc kubenswrapper[5052]: I1203 11:20:33.854802 5052 generic.go:334] "Generic (PLEG): container finished" podID="17265a68-bcb5-455c-b7df-c148a6517db9" containerID="ce6a0f2a385f3ad9f34f3a09e960700dc4d3941341c08997b95c648c82d11749" exitCode=0 Dec 03 11:20:33 crc kubenswrapper[5052]: I1203 11:20:33.854866 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2lrql" event={"ID":"17265a68-bcb5-455c-b7df-c148a6517db9","Type":"ContainerDied","Data":"ce6a0f2a385f3ad9f34f3a09e960700dc4d3941341c08997b95c648c82d11749"} Dec 03 11:20:33 crc kubenswrapper[5052]: I1203 11:20:33.861663 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" event={"ID":"10c237bc-72e7-4da3-9f2c-b1688395bf6e","Type":"ContainerStarted","Data":"b9c20844de46e525f8cdce935c43fc2e236f7bd4980cab6e4b13a967e18ffb0a"} Dec 03 11:20:33 crc kubenswrapper[5052]: I1203 11:20:33.863645 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-network-node-identity/network-node-identity-vrzqb" event={"ID":"ef543e1b-8068-4ea3-b32a-61027b32e95d","Type":"ContainerStarted","Data":"e42188cae70b530a8f9b32c15799070a3c56d867fe86f378b8edef8b93deed45"} Dec 03 11:20:33 crc kubenswrapper[5052]: I1203 11:20:33.864555 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openshift-kube-apiserver/kube-apiserver-crc" Dec 03 11:20:33 crc kubenswrapper[5052]: I1203 11:20:33.868165 5052 status_manager.go:875] "Failed to update status for pod" pod="openshift-dns/node-resolver-xph69" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"e5c587e6-866a-4427-8a2c-3469cf0fc495\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:32Z\\\",\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"Initialized\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:32Z\\\",\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:32Z\\\",\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"ContainersReady\\\"}],\\\"containerStatuses\\\":[{\\\"containerID\\\":\\\"cri-o://8cbacdbdef2832bba669718580c35080702354f16ec06e7d2680a6ee5b63e8b3\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:35512335ac39aed0f55b7f799f416f4f6445c20c1b19888cf2bb72bb276703f2\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:35512335ac39aed0f55b7f799f416f4f6445c20c1b19888cf2bb72bb276703f2\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"dns-node-resolver\\\",\\\"ready\\\":true,\\\"restartCount\\\":0,\\\"started\\\":true,\\\"state\\\":{\\\"running\\\":{\\\"startedAt\\\":\\\"2025-12-03T11:20:32Z\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/etc/hosts\\\",\\\"name\\\":\\\"hosts-file\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-tk7n6\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}],\\\"hostIP\\\":\\\"192.168.126.11\\\",\\\"hostIPs\\\":[{\\\"ip\\\":\\\"192.168.126.11\\\"}],\\\"phase\\\":\\\"Running\\\",\\\"podIP\\\":\\\"192.168.126.11\\\",\\\"podIPs\\\":[{\\\"ip\\\":\\\"192.168.126.11\\\"}],\\\"startTime\\\":\\\"2025-12-03T11:20:31Z\\\"}}\" for pod \"openshift-dns\"/\"node-resolver-xph69\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://127.0.0.1:9743/pod?timeout=10s\": tls: failed to verify certificate: x509: certificate has expired or is not yet valid: current time 2025-12-03T11:20:33Z is after 2025-08-24T17:21:41Z" Dec 03 11:20:33 crc kubenswrapper[5052]: I1203 11:20:33.886418 5052 status_manager.go:875] "Failed to update status for pod" pod="openshift-multus/multus-additional-cni-plugins-2lrql" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"17265a68-bcb5-455c-b7df-c148a6517db9\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:32Z\\\",\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"message\\\":\\\"containers with incomplete status: [cni-plugins bond-cni-plugin routeoverride-cni whereabouts-cni-bincopy whereabouts-cni]\\\",\\\"reason\\\":\\\"ContainersNotInitialized\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"Initialized\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"message\\\":\\\"containers with unready status: [kube-multus-additional-cni-plugins]\\\",\\\"reason\\\":\\\"ContainersNotReady\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"message\\\":\\\"containers with unready status: [kube-multus-additional-cni-plugins]\\\",\\\"reason\\\":\\\"ContainersNotReady\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"ContainersReady\\\"}],\\\"containerStatuses\\\":[{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7eeaee65f2808b819eedb413bdcabb9144e12f0dd97f13fd1afba93a95b67b26\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"kube-multus-additional-cni-plugins\\\",\\\"ready\\\":false,\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"PodInitializing\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-w4zf5\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}],\\\"hostIP\\\":\\\"192.168.126.11\\\",\\\"hostIPs\\\":[{\\\"ip\\\":\\\"192.168.126.11\\\"}],\\\"initContainerStatuses\\\":[{\\\"containerID\\\":\\\"cri-o://ce6a0f2a385f3ad9f34f3a09e960700dc4d3941341c08997b95c648c82d11749\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e40792096b162f0f9ce5f8362f51e5f8dea2c1ce4b1447235388416b5db7708c\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e40792096b162f0f9ce5f8362f51e5f8dea2c1ce4b1447235388416b5db7708c\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"egress-router-binary-copy\\\",\\\"ready\\\":true,\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://ce6a0f2a385f3ad9f34f3a09e960700dc4d3941341c08997b95c648c82d11749\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2025-12-03T11:20:33Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2025-12-03T11:20:32Z\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/entrypoint\\\",\\\"name\\\":\\\"cni-binary-copy\\\"},{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/os-release\\\",\\\"name\\\":\\\"os-release\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-w4zf5\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:687fddfbb085a1688df312ce4ec8c857df9b2daed8ff4a7ed6163a1154afa2cc\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"cni-plugins\\\",\\\"ready\\\":false,\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"PodInitializing\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/entrypoint\\\",\\\"name\\\":\\\"cni-binary-copy\\\"},{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/os-release\\\",\\\"name\\\":\\\"os-release\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/host/etc/cni/tuning/\\\",\\\"name\\\":\\\"tuning-conf-dir\\\"},{\\\"mountPath\\\":\\\"/sysctls\\\",\\\"name\\\":\\\"cni-sysctl-allowlist\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-w4zf5\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:383f4cceeeaead203bb2327fdd367c64b64d729d7fa93089f249e496fcef0c78\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"bond-cni-plugin\\\",\\\"ready\\\":false,\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"PodInitializing\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/entrypoint\\\",\\\"name\\\":\\\"cni-binary-copy\\\"},{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/os-release\\\",\\\"name\\\":\\\"os-release\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-w4zf5\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f567acb85146b5ed81451ec3e79f2de0c62e28c69b2eeade0abdf5d0c388e7aa\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"routeoverride-cni\\\",\\\"ready\\\":false,\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"PodInitializing\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/entrypoint\\\",\\\"name\\\":\\\"cni-binary-copy\\\"},{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/os-release\\\",\\\"name\\\":\\\"os-release\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-w4zf5\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:98100674616e54319f6713d742fd0c3bdbc84e6e6173e8ccf4a2473a714c2bc4\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"whereabouts-cni-bincopy\\\",\\\"ready\\\":false,\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"PodInitializing\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/entrypoint\\\",\\\"name\\\":\\\"cni-binary-copy\\\"},{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/os-release\\\",\\\"name\\\":\\\"os-release\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-w4zf5\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:98100674616e54319f6713d742fd0c3bdbc84e6e6173e8ccf4a2473a714c2bc4\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"whereabouts-cni\\\",\\\"ready\\\":false,\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"PodInitializing\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/cni/net.d\\\",\\\"name\\\":\\\"system-cni-dir\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-w4zf5\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}],\\\"podIP\\\":\\\"192.168.126.11\\\",\\\"podIPs\\\":[{\\\"ip\\\":\\\"192.168.126.11\\\"}],\\\"startTime\\\":\\\"2025-12-03T11:20:31Z\\\"}}\" for pod \"openshift-multus\"/\"multus-additional-cni-plugins-2lrql\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://127.0.0.1:9743/pod?timeout=10s\": tls: failed to verify certificate: x509: certificate has expired or is not yet valid: current time 2025-12-03T11:20:33Z is after 2025-08-24T17:21:41Z" Dec 03 11:20:33 crc kubenswrapper[5052]: I1203 11:20:33.903239 5052 status_manager.go:875] "Failed to update status for pod" pod="openshift-multus/multus-hx657" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"c30c02db-f4e7-4432-941c-5448fd71f3d9\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:32Z\\\",\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"Initialized\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:32Z\\\",\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:32Z\\\",\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"ContainersReady\\\"}],\\\"containerStatuses\\\":[{\\\"containerID\\\":\\\"cri-o://388fefa2a0e0ba8ca09e3fd39371e442358e9547cc2789ca8fa4b7b7237f6ac5\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7eeaee65f2808b819eedb413bdcabb9144e12f0dd97f13fd1afba93a95b67b26\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7eeaee65f2808b819eedb413bdcabb9144e12f0dd97f13fd1afba93a95b67b26\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"kube-multus\\\",\\\"ready\\\":true,\\\"restartCount\\\":0,\\\"started\\\":true,\\\"state\\\":{\\\"running\\\":{\\\"startedAt\\\":\\\"2025-12-03T11:20:32Z\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/entrypoint\\\",\\\"name\\\":\\\"cni-binary-copy\\\"},{\\\"mountPath\\\":\\\"/host/etc/os-release\\\",\\\"name\\\":\\\"os-release\\\"},{\\\"mountPath\\\":\\\"/host/etc/cni/net.d\\\",\\\"name\\\":\\\"system-cni-dir\\\"},{\\\"mountPath\\\":\\\"/host/run/multus/cni/net.d\\\",\\\"name\\\":\\\"multus-cni-dir\\\"},{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/run/multus\\\",\\\"name\\\":\\\"multus-socket-dir-parent\\\"},{\\\"mountPath\\\":\\\"/run/k8s.cni.cncf.io\\\",\\\"name\\\":\\\"host-run-k8s-cni-cncf-io\\\"},{\\\"mountPath\\\":\\\"/run/netns\\\",\\\"name\\\":\\\"host-run-netns\\\"},{\\\"mountPath\\\":\\\"/var/lib/cni/bin\\\",\\\"name\\\":\\\"host-var-lib-cni-bin\\\"},{\\\"mountPath\\\":\\\"/var/lib/cni/multus\\\",\\\"name\\\":\\\"host-var-lib-cni-multus\\\"},{\\\"mountPath\\\":\\\"/var/lib/kubelet\\\",\\\"name\\\":\\\"host-var-lib-kubelet\\\"},{\\\"mountPath\\\":\\\"/hostroot\\\",\\\"name\\\":\\\"hostroot\\\"},{\\\"mountPath\\\":\\\"/etc/cni/multus/net.d\\\",\\\"name\\\":\\\"multus-conf-dir\\\"},{\\\"mountPath\\\":\\\"/etc/cni/net.d/multus.d\\\",\\\"name\\\":\\\"multus-daemon-config\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/etc/cni/multus/certs\\\",\\\"name\\\":\\\"host-run-multus-certs\\\"},{\\\"mountPath\\\":\\\"/etc/kubernetes\\\",\\\"name\\\":\\\"etc-kubernetes\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-fhsjl\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}],\\\"hostIP\\\":\\\"192.168.126.11\\\",\\\"hostIPs\\\":[{\\\"ip\\\":\\\"192.168.126.11\\\"}],\\\"phase\\\":\\\"Running\\\",\\\"podIP\\\":\\\"192.168.126.11\\\",\\\"podIPs\\\":[{\\\"ip\\\":\\\"192.168.126.11\\\"}],\\\"startTime\\\":\\\"2025-12-03T11:20:31Z\\\"}}\" for pod \"openshift-multus\"/\"multus-hx657\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://127.0.0.1:9743/pod?timeout=10s\": tls: failed to verify certificate: x509: certificate has expired or is not yet valid: current time 2025-12-03T11:20:33Z is after 2025-08-24T17:21:41Z" Dec 03 11:20:33 crc kubenswrapper[5052]: I1203 11:20:33.926412 5052 status_manager.go:875] "Failed to update status for pod" pod="openshift-etcd/etcd-crc" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"bc6c0f8f-e9c0-47cd-89fa-974113e2c13a\\\"},\\\"status\\\":{\\\"conditions\\\":[{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:11Z\\\",\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:15Z\\\",\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"Initialized\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:32Z\\\",\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:32Z\\\",\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"ContainersReady\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:09Z\\\",\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"PodScheduled\\\"}],\\\"containerStatuses\\\":[{\\\"containerID\\\":\\\"cri-o://f3bd857c0d35ed129a4fdc96deae82d6457f5f24c4d92a129972ce97a89b3488\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:07b7c6877441ecd6a5646fb68e33e9be8b90092272e49117b54b4a67314731ca\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:07b7c6877441ecd6a5646fb68e33e9be8b90092272e49117b54b4a67314731ca\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"etcd\\\",\\\"ready\\\":true,\\\"restartCount\\\":0,\\\"started\\\":true,\\\"state\\\":{\\\"running\\\":{\\\"startedAt\\\":\\\"2025-12-03T11:20:16Z\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/etc/kubernetes/manifests\\\",\\\"name\\\":\\\"static-pod-dir\\\"},{\\\"mountPath\\\":\\\"/etc/kubernetes/static-pod-resources\\\",\\\"name\\\":\\\"resource-dir\\\"},{\\\"mountPath\\\":\\\"/etc/kubernetes/static-pod-certs\\\",\\\"name\\\":\\\"cert-dir\\\"},{\\\"mountPath\\\":\\\"/var/lib/etcd/\\\",\\\"name\\\":\\\"data-dir\\\"}]},{\\\"containerID\\\":\\\"cri-o://c5dd3e47134939ea92cb1677d8d171eadd8c0c0743827ebee0a61daf261f4e30\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:07b7c6877441ecd6a5646fb68e33e9be8b90092272e49117b54b4a67314731ca\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:07b7c6877441ecd6a5646fb68e33e9be8b90092272e49117b54b4a67314731ca\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"etcd-metrics\\\",\\\"ready\\\":true,\\\"restartCount\\\":0,\\\"started\\\":true,\\\"state\\\":{\\\"running\\\":{\\\"startedAt\\\":\\\"2025-12-03T11:20:16Z\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/etc/kubernetes/static-pod-resources\\\",\\\"name\\\":\\\"resource-dir\\\"},{\\\"mountPath\\\":\\\"/etc/kubernetes/static-pod-certs\\\",\\\"name\\\":\\\"cert-dir\\\"},{\\\"mountPath\\\":\\\"/var/lib/etcd/\\\",\\\"name\\\":\\\"data-dir\\\"}]},{\\\"containerID\\\":\\\"cri-o://2409d840f8acda1e98e333c3b4d9ebd1b67f4a05fa3f662f6d7171dddab13846\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a0fa3723269019bee1847b26702f42928e779036cc2f58408f8ee7866be30a93\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a0fa3723269019bee1847b26702f42928e779036cc2f58408f8ee7866be30a93\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"etcd-readyz\\\",\\\"ready\\\":true,\\\"restartCount\\\":0,\\\"started\\\":true,\\\"state\\\":{\\\"running\\\":{\\\"startedAt\\\":\\\"2025-12-03T11:20:16Z\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/var/log/etcd/\\\",\\\"name\\\":\\\"log-dir\\\"},{\\\"mountPath\\\":\\\"/etc/kubernetes/static-pod-certs\\\",\\\"name\\\":\\\"cert-dir\\\"}]},{\\\"containerID\\\":\\\"cri-o://7e429db04e5ef00df1a3a67e1750865167bb14eef83a110aa54ce3ca2f4656c5\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a0fa3723269019bee1847b26702f42928e779036cc2f58408f8ee7866be30a93\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a0fa3723269019bee1847b26702f42928e779036cc2f58408f8ee7866be30a93\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"etcd-rev\\\",\\\"ready\\\":true,\\\"restartCount\\\":0,\\\"started\\\":true,\\\"state\\\":{\\\"running\\\":{\\\"startedAt\\\":\\\"2025-12-03T11:20:16Z\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/var/lib/etcd\\\",\\\"name\\\":\\\"data-dir\\\"},{\\\"mountPath\\\":\\\"/etc/kubernetes/static-pod-certs\\\",\\\"name\\\":\\\"cert-dir\\\"}]},{\\\"containerID\\\":\\\"cri-o://d9e3c1335ee9e6fe5d5ca827757da511034a270ca681b3cef694e595539d1cd0\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:07b7c6877441ecd6a5646fb68e33e9be8b90092272e49117b54b4a67314731ca\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:07b7c6877441ecd6a5646fb68e33e9be8b90092272e49117b54b4a67314731ca\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"etcdctl\\\",\\\"ready\\\":true,\\\"restartCount\\\":0,\\\"started\\\":true,\\\"state\\\":{\\\"running\\\":{\\\"startedAt\\\":\\\"2025-12-03T11:20:15Z\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/etc/kubernetes/manifests\\\",\\\"name\\\":\\\"static-pod-dir\\\"},{\\\"mountPath\\\":\\\"/etc/kubernetes/static-pod-resources\\\",\\\"name\\\":\\\"resource-dir\\\"},{\\\"mountPath\\\":\\\"/etc/kubernetes/static-pod-certs\\\",\\\"name\\\":\\\"cert-dir\\\"},{\\\"mountPath\\\":\\\"/var/lib/etcd/\\\",\\\"name\\\":\\\"data-dir\\\"}]}],\\\"hostIP\\\":\\\"192.168.126.11\\\",\\\"hostIPs\\\":[{\\\"ip\\\":\\\"192.168.126.11\\\"}],\\\"initContainerStatuses\\\":[{\\\"containerID\\\":\\\"cri-o://7d7c61963a9597f2c9c159d743284313820acece17152be4d829b0e3d02434c1\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:07b7c6877441ecd6a5646fb68e33e9be8b90092272e49117b54b4a67314731ca\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:07b7c6877441ecd6a5646fb68e33e9be8b90092272e49117b54b4a67314731ca\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"setup\\\",\\\"ready\\\":true,\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://7d7c61963a9597f2c9c159d743284313820acece17152be4d829b0e3d02434c1\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2025-12-03T11:20:11Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2025-12-03T11:20:11Z\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/var/log/etcd\\\",\\\"name\\\":\\\"log-dir\\\"}]},{\\\"containerID\\\":\\\"cri-o://e980e02c63b59597153749cc2ed54da5925e88a6a0ed2293f4075ff4b46bbe44\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:07b7c6877441ecd6a5646fb68e33e9be8b90092272e49117b54b4a67314731ca\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:07b7c6877441ecd6a5646fb68e33e9be8b90092272e49117b54b4a67314731ca\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"etcd-ensure-env-vars\\\",\\\"ready\\\":true,\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://e980e02c63b59597153749cc2ed54da5925e88a6a0ed2293f4075ff4b46bbe44\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2025-12-03T11:20:12Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2025-12-03T11:20:12Z\\\"}}},{\\\"containerID\\\":\\\"cri-o://6a0411ec57e822071155dd66aa1e30dcab88df41e229cf75208468e5dcf90358\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:07b7c6877441ecd6a5646fb68e33e9be8b90092272e49117b54b4a67314731ca\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:07b7c6877441ecd6a5646fb68e33e9be8b90092272e49117b54b4a67314731ca\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"etcd-resources-copy\\\",\\\"ready\\\":true,\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://6a0411ec57e822071155dd66aa1e30dcab88df41e229cf75208468e5dcf90358\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2025-12-03T11:20:15Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2025-12-03T11:20:15Z\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/etc/kubernetes/static-pod-resources\\\",\\\"name\\\":\\\"resource-dir\\\"},{\\\"mountPath\\\":\\\"/etc/kubernetes/static-pod-certs\\\",\\\"name\\\":\\\"cert-dir\\\"},{\\\"mountPath\\\":\\\"/usr/local/bin\\\",\\\"name\\\":\\\"usr-local-bin\\\"}]}],\\\"phase\\\":\\\"Running\\\",\\\"podIP\\\":\\\"192.168.126.11\\\",\\\"podIPs\\\":[{\\\"ip\\\":\\\"192.168.126.11\\\"}],\\\"startTime\\\":\\\"2025-12-03T11:20:09Z\\\"}}\" for pod \"openshift-etcd\"/\"etcd-crc\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://127.0.0.1:9743/pod?timeout=10s\": tls: failed to verify certificate: x509: certificate has expired or is not yet valid: current time 2025-12-03T11:20:33Z is after 2025-08-24T17:21:41Z" Dec 03 11:20:33 crc kubenswrapper[5052]: I1203 11:20:33.942417 5052 status_manager.go:875] "Failed to update status for pod" pod="openshift-kube-apiserver/kube-apiserver-crc" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"6c8df12a-3d11-4bb9-b398-67091c1918e4\\\"},\\\"status\\\":{\\\"conditions\\\":[{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:11Z\\\",\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:11Z\\\",\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"Initialized\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:09Z\\\",\\\"message\\\":\\\"containers with unready status: [kube-apiserver-check-endpoints]\\\",\\\"reason\\\":\\\"ContainersNotReady\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:09Z\\\",\\\"message\\\":\\\"containers with unready status: [kube-apiserver-check-endpoints]\\\",\\\"reason\\\":\\\"ContainersNotReady\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"ContainersReady\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:09Z\\\",\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"PodScheduled\\\"}],\\\"containerStatuses\\\":[{\\\"containerID\\\":\\\"cri-o://09256543a9aa86da8bec948be040d36b6e4986b9232091d2638bb1fead5740a9\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:06bc35825771aee1220d34720243b89c4ba8a8b335e6de2597126bd791fd90d4\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:06bc35825771aee1220d34720243b89c4ba8a8b335e6de2597126bd791fd90d4\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"kube-apiserver\\\",\\\"ready\\\":true,\\\"restartCount\\\":0,\\\"started\\\":true,\\\"state\\\":{\\\"running\\\":{\\\"startedAt\\\":\\\"2025-12-03T11:20:12Z\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/etc/kubernetes/static-pod-resources\\\",\\\"name\\\":\\\"resource-dir\\\"},{\\\"mountPath\\\":\\\"/etc/kubernetes/static-pod-certs\\\",\\\"name\\\":\\\"cert-dir\\\"},{\\\"mountPath\\\":\\\"/var/log/kube-apiserver\\\",\\\"name\\\":\\\"audit-dir\\\"}]},{\\\"containerID\\\":\\\"cri-o://15abb0d7fd5884519ed5ac454430dbb58f71ed1f459e5ed30f324f70befd243e\\\",\\\"image\\\":\\\"quay.io/crcont/openshift-crc-cluster-kube-apiserver-operator@sha256:9f36dc276e27753fc478274c7f7814a4f8945c987117ee1ea3b8e6355e6d7462\\\",\\\"imageID\\\":\\\"quay.io/crcont/openshift-crc-cluster-kube-apiserver-operator@sha256:9f36dc276e27753fc478274c7f7814a4f8945c987117ee1ea3b8e6355e6d7462\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"kube-apiserver-cert-regeneration-controller\\\",\\\"ready\\\":true,\\\"restartCount\\\":0,\\\"started\\\":true,\\\"state\\\":{\\\"running\\\":{\\\"startedAt\\\":\\\"2025-12-03T11:20:14Z\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/etc/kubernetes/static-pod-resources\\\",\\\"name\\\":\\\"resource-dir\\\"}]},{\\\"containerID\\\":\\\"cri-o://8ceedd21ae3833d77fd440537f30434ee57f087fbefb2357b46231d3f778ad2a\\\",\\\"image\\\":\\\"quay.io/crcont/openshift-crc-cluster-kube-apiserver-operator@sha256:9f36dc276e27753fc478274c7f7814a4f8945c987117ee1ea3b8e6355e6d7462\\\",\\\"imageID\\\":\\\"quay.io/crcont/openshift-crc-cluster-kube-apiserver-operator@sha256:9f36dc276e27753fc478274c7f7814a4f8945c987117ee1ea3b8e6355e6d7462\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"kube-apiserver-cert-syncer\\\",\\\"ready\\\":true,\\\"restartCount\\\":0,\\\"started\\\":true,\\\"state\\\":{\\\"running\\\":{\\\"startedAt\\\":\\\"2025-12-03T11:20:12Z\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/etc/kubernetes/static-pod-resources\\\",\\\"name\\\":\\\"resource-dir\\\"},{\\\"mountPath\\\":\\\"/etc/kubernetes/static-pod-certs\\\",\\\"name\\\":\\\"cert-dir\\\"}]},{\\\"containerID\\\":\\\"cri-o://ee9c5c61cfd6dee9fd44ebc4c7ef4d20b5fde82b12a8147c3c871e2615da7940\\\",\\\"image\\\":\\\"quay.io/crcont/openshift-crc-cluster-kube-apiserver-operator@sha256:9f36dc276e27753fc478274c7f7814a4f8945c987117ee1ea3b8e6355e6d7462\\\",\\\"imageID\\\":\\\"quay.io/crcont/openshift-crc-cluster-kube-apiserver-operator@sha256:9f36dc276e27753fc478274c7f7814a4f8945c987117ee1ea3b8e6355e6d7462\\\",\\\"lastState\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://b739f84524d09b27f6e052e21df8be3870a2136665704232f5a365d8a5bf41f5\\\",\\\"exitCode\\\":255,\\\"finishedAt\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"message\\\":\\\"ing back to namespace): Get \\\\\\\"https://localhost:6443/api/v1/namespaces/openshift-kube-apiserver/pods/kube-apiserver-crc\\\\\\\": net/http: TLS handshake timeout\\\\nI1203 11:20:25.911385 1 builder.go:304] check-endpoints version 4.18.0-202502101302.p0.g763313c.assembly.stream.el9-763313c-763313c860ea43fcfc9b1ac00ebae096b57c078e\\\\nI1203 11:20:25.912713 1 dynamic_serving_content.go:116] \\\\\\\"Loaded a new cert/key pair\\\\\\\" name=\\\\\\\"serving-cert::/tmp/serving-cert-3310779835/tls.crt::/tmp/serving-cert-3310779835/tls.key\\\\\\\"\\\\nI1203 11:20:31.324803 1 requestheader_controller.go:247] Loaded a new request header values for RequestHeaderAuthRequestController\\\\nI1203 11:20:31.327725 1 maxinflight.go:139] \\\\\\\"Initialized nonMutatingChan\\\\\\\" len=400\\\\nI1203 11:20:31.327838 1 maxinflight.go:145] \\\\\\\"Initialized mutatingChan\\\\\\\" len=200\\\\nI1203 11:20:31.327886 1 maxinflight.go:116] \\\\\\\"Set denominator for readonly requests\\\\\\\" limit=400\\\\nI1203 11:20:31.327915 1 maxinflight.go:120] \\\\\\\"Set denominator for mutating requests\\\\\\\" limit=200\\\\nI1203 11:20:31.335573 1 secure_serving.go:57] Forcing use of http/1.1 only\\\\nW1203 11:20:31.335607 1 secure_serving.go:69] Use of insecure cipher 'TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256' detected.\\\\nW1203 11:20:31.335613 1 secure_serving.go:69] Use of insecure cipher 'TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256' detected.\\\\nW1203 11:20:31.335619 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_128_GCM_SHA256' detected.\\\\nW1203 11:20:31.335623 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_256_GCM_SHA384' detected.\\\\nW1203 11:20:31.335626 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_128_CBC_SHA' detected.\\\\nW1203 11:20:31.335630 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_256_CBC_SHA' detected.\\\\nI1203 11:20:31.335893 1 genericapiserver.go:533] MuxAndDiscoveryComplete has all endpoints registered and discovery information is complete\\\\nF1203 11:20:31.338035 1 cmd.go:182] pods \\\\\\\"kube-apiserver-crc\\\\\\\" not found\\\\n\\\",\\\"reason\\\":\\\"Error\\\",\\\"startedAt\\\":\\\"2025-12-03T11:20:15Z\\\"}},\\\"name\\\":\\\"kube-apiserver-check-endpoints\\\",\\\"ready\\\":false,\\\"restartCount\\\":1,\\\"started\\\":true,\\\"state\\\":{\\\"running\\\":{\\\"startedAt\\\":\\\"2025-12-03T11:20:32Z\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/etc/kubernetes/static-pod-resources\\\",\\\"name\\\":\\\"resource-dir\\\"},{\\\"mountPath\\\":\\\"/etc/kubernetes/static-pod-certs\\\",\\\"name\\\":\\\"cert-dir\\\"}]},{\\\"containerID\\\":\\\"cri-o://22b8cb6aa04337d1420f97d3e128fce2b29a725ca6e5d157adeb1ba78f00073f\\\",\\\"image\\\":\\\"quay.io/crcont/openshift-crc-cluster-kube-apiserver-operator@sha256:9f36dc276e27753fc478274c7f7814a4f8945c987117ee1ea3b8e6355e6d7462\\\",\\\"imageID\\\":\\\"quay.io/crcont/openshift-crc-cluster-kube-apiserver-operator@sha256:9f36dc276e27753fc478274c7f7814a4f8945c987117ee1ea3b8e6355e6d7462\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"kube-apiserver-insecure-readyz\\\",\\\"ready\\\":true,\\\"restartCount\\\":0,\\\"started\\\":true,\\\"state\\\":{\\\"running\\\":{\\\"startedAt\\\":\\\"2025-12-03T11:20:15Z\\\"}}}],\\\"hostIP\\\":\\\"192.168.126.11\\\",\\\"hostIPs\\\":[{\\\"ip\\\":\\\"192.168.126.11\\\"}],\\\"initContainerStatuses\\\":[{\\\"containerID\\\":\\\"cri-o://7182c75773f069c72f87c39136222fd4a91b49add27352a1969645413e670f01\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:06bc35825771aee1220d34720243b89c4ba8a8b335e6de2597126bd791fd90d4\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:06bc35825771aee1220d34720243b89c4ba8a8b335e6de2597126bd791fd90d4\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"setup\\\",\\\"ready\\\":true,\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://7182c75773f069c72f87c39136222fd4a91b49add27352a1969645413e670f01\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2025-12-03T11:20:11Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2025-12-03T11:20:11Z\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/var/log/kube-apiserver\\\",\\\"name\\\":\\\"audit-dir\\\"}]}],\\\"phase\\\":\\\"Running\\\",\\\"podIP\\\":\\\"192.168.126.11\\\",\\\"podIPs\\\":[{\\\"ip\\\":\\\"192.168.126.11\\\"}],\\\"startTime\\\":\\\"2025-12-03T11:20:09Z\\\"}}\" for pod \"openshift-kube-apiserver\"/\"kube-apiserver-crc\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://127.0.0.1:9743/pod?timeout=10s\": tls: failed to verify certificate: x509: certificate has expired or is not yet valid: current time 2025-12-03T11:20:33Z is after 2025-08-24T17:21:41Z" Dec 03 11:20:33 crc kubenswrapper[5052]: I1203 11:20:33.959431 5052 status_manager.go:875] "Failed to update status for pod" pod="openshift-network-diagnostics/network-check-target-xd92c" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"3b6479f0-333b-4a96-9adf-2099afdc2447\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"message\\\":\\\"containers with unready status: [network-check-target-container]\\\",\\\"reason\\\":\\\"ContainersNotReady\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"message\\\":\\\"containers with unready status: [network-check-target-container]\\\",\\\"reason\\\":\\\"ContainersNotReady\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"ContainersReady\\\"}],\\\"containerStatuses\\\":[{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e1baa38811c04bd8909e01a1f3be7421a1cb99d608d3dc4cf86d95b17de2ab8b\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{\\\"terminated\\\":{\\\"exitCode\\\":137,\\\"finishedAt\\\":null,\\\"message\\\":\\\"The container could not be located when the pod was deleted. The container used to be Running\\\",\\\"reason\\\":\\\"ContainerStatusUnknown\\\",\\\"startedAt\\\":null}},\\\"name\\\":\\\"network-check-target-container\\\",\\\"ready\\\":false,\\\"restartCount\\\":3,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"ContainerCreating\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-cqllr\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}],\\\"podIP\\\":null,\\\"podIPs\\\":null}}\" for pod \"openshift-network-diagnostics\"/\"network-check-target-xd92c\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://127.0.0.1:9743/pod?timeout=10s\": tls: failed to verify certificate: x509: certificate has expired or is not yet valid: current time 2025-12-03T11:20:33Z is after 2025-08-24T17:21:41Z" Dec 03 11:20:33 crc kubenswrapper[5052]: I1203 11:20:33.972283 5052 status_manager.go:875] "Failed to update status for pod" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"10c237bc-72e7-4da3-9f2c-b1688395bf6e\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"Initialized\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"message\\\":\\\"containers with unready status: [machine-config-daemon kube-rbac-proxy]\\\",\\\"reason\\\":\\\"ContainersNotReady\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"message\\\":\\\"containers with unready status: [machine-config-daemon kube-rbac-proxy]\\\",\\\"reason\\\":\\\"ContainersNotReady\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"ContainersReady\\\"}],\\\"containerStatuses\\\":[{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:242b3d66438c42745f4ef318bdeaf3d793426f12962a42ea83e18d06c08aaf09\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"kube-rbac-proxy\\\",\\\"ready\\\":false,\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"ContainerCreating\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/etc/tls/private\\\",\\\"name\\\":\\\"proxy-tls\\\"},{\\\"mountPath\\\":\\\"/etc/kube-rbac-proxy\\\",\\\"name\\\":\\\"mcd-auth-proxy-config\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-9vkjv\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c915fb8ba96e911699a1ae34a8e95ca8a9fbe1bf8c28fea177225c63a8bdfc0a\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"machine-config-daemon\\\",\\\"ready\\\":false,\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"ContainerCreating\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/rootfs\\\",\\\"name\\\":\\\"rootfs\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-9vkjv\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}],\\\"hostIP\\\":\\\"192.168.126.11\\\",\\\"hostIPs\\\":[{\\\"ip\\\":\\\"192.168.126.11\\\"}],\\\"podIP\\\":\\\"192.168.126.11\\\",\\\"podIPs\\\":[{\\\"ip\\\":\\\"192.168.126.11\\\"}],\\\"startTime\\\":\\\"2025-12-03T11:20:31Z\\\"}}\" for pod \"openshift-machine-config-operator\"/\"machine-config-daemon-74bj6\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://127.0.0.1:9743/pod?timeout=10s\": tls: failed to verify certificate: x509: certificate has expired or is not yet valid: current time 2025-12-03T11:20:33Z is after 2025-08-24T17:21:41Z" Dec 03 11:20:33 crc kubenswrapper[5052]: I1203 11:20:33.988237 5052 status_manager.go:875] "Failed to update status for pod" pod="openshift-network-operator/network-operator-58b4c7f79c-55gtf" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"37a5e44f-9a88-4405-be8a-b645485e7312\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:32Z\\\",\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:32Z\\\",\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:32Z\\\",\\\"type\\\":\\\"ContainersReady\\\"}],\\\"containerStatuses\\\":[{\\\"containerID\\\":\\\"cri-o://177058541a1de284ec4275037302f876e3345d48400f5edc9466508d43518412\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e1baa38811c04bd8909e01a1f3be7421a1cb99d608d3dc4cf86d95b17de2ab8b\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e1baa38811c04bd8909e01a1f3be7421a1cb99d608d3dc4cf86d95b17de2ab8b\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"network-operator\\\",\\\"ready\\\":true,\\\"restartCount\\\":0,\\\"started\\\":true,\\\"state\\\":{\\\"running\\\":{\\\"startedAt\\\":\\\"2025-12-03T11:20:32Z\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/etc/kubernetes\\\",\\\"name\\\":\\\"host-etc-kube\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/serving-cert\\\",\\\"name\\\":\\\"metrics-tls\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-rdwmf\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}]}}\" for pod \"openshift-network-operator\"/\"network-operator-58b4c7f79c-55gtf\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://127.0.0.1:9743/pod?timeout=10s\": tls: failed to verify certificate: x509: certificate has expired or is not yet valid: current time 2025-12-03T11:20:33Z is after 2025-08-24T17:21:41Z" Dec 03 11:20:34 crc kubenswrapper[5052]: I1203 11:20:34.005883 5052 status_manager.go:875] "Failed to update status for pod" pod="openshift-network-node-identity/network-node-identity-vrzqb" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"ef543e1b-8068-4ea3-b32a-61027b32e95d\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"message\\\":\\\"containers with unready status: [webhook approver]\\\",\\\"reason\\\":\\\"ContainersNotReady\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"message\\\":\\\"containers with unready status: [webhook approver]\\\",\\\"reason\\\":\\\"ContainersNotReady\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"ContainersReady\\\"}],\\\"containerStatuses\\\":[{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:174f36cdd47ef0d1d2099482919d773257453265a2af0b17b154edc32fa41ac2\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{\\\"terminated\\\":{\\\"exitCode\\\":137,\\\"finishedAt\\\":null,\\\"message\\\":\\\"The container could not be located when the pod was deleted. The container used to be Running\\\",\\\"reason\\\":\\\"ContainerStatusUnknown\\\",\\\"startedAt\\\":null}},\\\"name\\\":\\\"approver\\\",\\\"ready\\\":false,\\\"restartCount\\\":6,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"ContainerCreating\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/env\\\",\\\"name\\\":\\\"env-overrides\\\"},{\\\"mountPath\\\":\\\"/var/run/ovnkube-identity-config\\\",\\\"name\\\":\\\"ovnkube-identity-cm\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-s2kz5\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:174f36cdd47ef0d1d2099482919d773257453265a2af0b17b154edc32fa41ac2\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{\\\"terminated\\\":{\\\"exitCode\\\":137,\\\"finishedAt\\\":null,\\\"message\\\":\\\"The container could not be located when the pod was deleted. The container used to be Running\\\",\\\"reason\\\":\\\"ContainerStatusUnknown\\\",\\\"startedAt\\\":null}},\\\"name\\\":\\\"webhook\\\",\\\"ready\\\":false,\\\"restartCount\\\":3,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"ContainerCreating\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/etc/webhook-cert/\\\",\\\"name\\\":\\\"webhook-cert\\\"},{\\\"mountPath\\\":\\\"/env\\\",\\\"name\\\":\\\"env-overrides\\\"},{\\\"mountPath\\\":\\\"/var/run/ovnkube-identity-config\\\",\\\"name\\\":\\\"ovnkube-identity-cm\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-s2kz5\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}]}}\" for pod \"openshift-network-node-identity\"/\"network-node-identity-vrzqb\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://127.0.0.1:9743/pod?timeout=10s\": tls: failed to verify certificate: x509: certificate has expired or is not yet valid: current time 2025-12-03T11:20:34Z is after 2025-08-24T17:21:41Z" Dec 03 11:20:34 crc kubenswrapper[5052]: I1203 11:20:34.021733 5052 status_manager.go:875] "Failed to update status for pod" pod="openshift-network-console/networking-console-plugin-85b44fc459-gdk6g" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"5fe485a1-e14f-4c09-b5b9-f252bc42b7e8\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"message\\\":\\\"containers with unready status: [networking-console-plugin]\\\",\\\"reason\\\":\\\"ContainersNotReady\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"message\\\":\\\"containers with unready status: [networking-console-plugin]\\\",\\\"reason\\\":\\\"ContainersNotReady\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"ContainersReady\\\"}],\\\"containerStatuses\\\":[{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ae647598ec35cda5766806d3d44a91e3b9d4dee48ff154f3d8490165399873fd\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{\\\"terminated\\\":{\\\"exitCode\\\":137,\\\"finishedAt\\\":null,\\\"message\\\":\\\"The container could not be located when the pod was deleted. The container used to be Running\\\",\\\"reason\\\":\\\"ContainerStatusUnknown\\\",\\\"startedAt\\\":null}},\\\"name\\\":\\\"networking-console-plugin\\\",\\\"ready\\\":false,\\\"restartCount\\\":3,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"ContainerCreating\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/var/cert\\\",\\\"name\\\":\\\"networking-console-plugin-cert\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/etc/nginx/nginx.conf\\\",\\\"name\\\":\\\"nginx-conf\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}],\\\"podIP\\\":null,\\\"podIPs\\\":null}}\" for pod \"openshift-network-console\"/\"networking-console-plugin-85b44fc459-gdk6g\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://127.0.0.1:9743/pod?timeout=10s\": tls: failed to verify certificate: x509: certificate has expired or is not yet valid: current time 2025-12-03T11:20:34Z is after 2025-08-24T17:21:41Z" Dec 03 11:20:34 crc kubenswrapper[5052]: I1203 11:20:34.038428 5052 status_manager.go:875] "Failed to update status for pod" pod="openshift-network-diagnostics/network-check-source-55646444c4-trplf" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"9d751cbb-f2e2-430d-9754-c882a5e924a5\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"message\\\":\\\"containers with unready status: [check-endpoints]\\\",\\\"reason\\\":\\\"ContainersNotReady\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"message\\\":\\\"containers with unready status: [check-endpoints]\\\",\\\"reason\\\":\\\"ContainersNotReady\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"ContainersReady\\\"}],\\\"containerStatuses\\\":[{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e1baa38811c04bd8909e01a1f3be7421a1cb99d608d3dc4cf86d95b17de2ab8b\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{\\\"terminated\\\":{\\\"exitCode\\\":137,\\\"finishedAt\\\":null,\\\"message\\\":\\\"The container could not be located when the pod was deleted. The container used to be Running\\\",\\\"reason\\\":\\\"ContainerStatusUnknown\\\",\\\"startedAt\\\":null}},\\\"name\\\":\\\"check-endpoints\\\",\\\"ready\\\":false,\\\"restartCount\\\":3,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"ContainerCreating\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-s2dwl\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}],\\\"podIP\\\":null,\\\"podIPs\\\":null}}\" for pod \"openshift-network-diagnostics\"/\"network-check-source-55646444c4-trplf\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://127.0.0.1:9743/pod?timeout=10s\": tls: failed to verify certificate: x509: certificate has expired or is not yet valid: current time 2025-12-03T11:20:34Z is after 2025-08-24T17:21:41Z" Dec 03 11:20:34 crc kubenswrapper[5052]: I1203 11:20:34.052398 5052 status_manager.go:875] "Failed to update status for pod" pod="openshift-network-operator/iptables-alerter-4ln5h" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"d75a4c96-2883-4a0b-bab2-0fab2b6c0b49\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"message\\\":\\\"containers with unready status: [iptables-alerter]\\\",\\\"reason\\\":\\\"ContainersNotReady\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"message\\\":\\\"containers with unready status: [iptables-alerter]\\\",\\\"reason\\\":\\\"ContainersNotReady\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"ContainersReady\\\"}],\\\"containerStatuses\\\":[{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:35512335ac39aed0f55b7f799f416f4f6445c20c1b19888cf2bb72bb276703f2\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{\\\"terminated\\\":{\\\"exitCode\\\":137,\\\"finishedAt\\\":null,\\\"message\\\":\\\"The container could not be located when the pod was deleted. The container used to be Running\\\",\\\"reason\\\":\\\"ContainerStatusUnknown\\\",\\\"startedAt\\\":null}},\\\"name\\\":\\\"iptables-alerter\\\",\\\"ready\\\":false,\\\"restartCount\\\":3,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"ContainerCreating\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/iptables-alerter\\\",\\\"name\\\":\\\"iptables-alerter-script\\\"},{\\\"mountPath\\\":\\\"/host\\\",\\\"name\\\":\\\"host-slash\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-rczfb\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}]}}\" for pod \"openshift-network-operator\"/\"iptables-alerter-4ln5h\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://127.0.0.1:9743/pod?timeout=10s\": tls: failed to verify certificate: x509: certificate has expired or is not yet valid: current time 2025-12-03T11:20:34Z is after 2025-08-24T17:21:41Z" Dec 03 11:20:34 crc kubenswrapper[5052]: I1203 11:20:34.073151 5052 status_manager.go:875] "Failed to update status for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kg55h" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"bc569bba-c020-40d6-af68-e617625fe0e4\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:32Z\\\",\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:32Z\\\",\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"Initialized\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"message\\\":\\\"containers with unready status: [ovn-controller ovn-acl-logging kube-rbac-proxy-node kube-rbac-proxy-ovn-metrics northd nbdb sbdb ovnkube-controller]\\\",\\\"reason\\\":\\\"ContainersNotReady\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"message\\\":\\\"containers with unready status: [ovn-controller ovn-acl-logging kube-rbac-proxy-node kube-rbac-proxy-ovn-metrics northd nbdb sbdb ovnkube-controller]\\\",\\\"reason\\\":\\\"ContainersNotReady\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"ContainersReady\\\"}],\\\"containerStatuses\\\":[{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:242b3d66438c42745f4ef318bdeaf3d793426f12962a42ea83e18d06c08aaf09\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"kube-rbac-proxy-node\\\",\\\"ready\\\":false,\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"PodInitializing\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/ovnkube-lib\\\",\\\"name\\\":\\\"ovnkube-script-lib\\\"},{\\\"mountPath\\\":\\\"/etc/pki/tls/metrics-cert\\\",\\\"name\\\":\\\"ovn-node-metrics-cert\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-x92h2\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:242b3d66438c42745f4ef318bdeaf3d793426f12962a42ea83e18d06c08aaf09\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"kube-rbac-proxy-ovn-metrics\\\",\\\"ready\\\":false,\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"PodInitializing\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/ovnkube-lib\\\",\\\"name\\\":\\\"ovnkube-script-lib\\\"},{\\\"mountPath\\\":\\\"/etc/pki/tls/metrics-cert\\\",\\\"name\\\":\\\"ovn-node-metrics-cert\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-x92h2\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:174f36cdd47ef0d1d2099482919d773257453265a2af0b17b154edc32fa41ac2\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"nbdb\\\",\\\"ready\\\":false,\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"PodInitializing\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/ovnkube-lib\\\",\\\"name\\\":\\\"ovnkube-script-lib\\\"},{\\\"mountPath\\\":\\\"/etc/ovn/\\\",\\\"name\\\":\\\"etc-openvswitch\\\"},{\\\"mountPath\\\":\\\"/var/log/ovn\\\",\\\"name\\\":\\\"node-log\\\"},{\\\"mountPath\\\":\\\"/run/ovn/\\\",\\\"name\\\":\\\"run-ovn\\\"},{\\\"mountPath\\\":\\\"/env\\\",\\\"name\\\":\\\"env-overrides\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-x92h2\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:174f36cdd47ef0d1d2099482919d773257453265a2af0b17b154edc32fa41ac2\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"northd\\\",\\\"ready\\\":false,\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"PodInitializing\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/ovnkube-lib\\\",\\\"name\\\":\\\"ovnkube-script-lib\\\"},{\\\"mountPath\\\":\\\"/etc/ovn\\\",\\\"name\\\":\\\"etc-openvswitch\\\"},{\\\"mountPath\\\":\\\"/var/log/ovn\\\",\\\"name\\\":\\\"node-log\\\"},{\\\"mountPath\\\":\\\"/run/ovn/\\\",\\\"name\\\":\\\"run-ovn\\\"},{\\\"mountPath\\\":\\\"/env\\\",\\\"name\\\":\\\"env-overrides\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-x92h2\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:174f36cdd47ef0d1d2099482919d773257453265a2af0b17b154edc32fa41ac2\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"ovn-acl-logging\\\",\\\"ready\\\":false,\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"PodInitializing\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/ovnkube-lib\\\",\\\"name\\\":\\\"ovnkube-script-lib\\\"},{\\\"mountPath\\\":\\\"/var/log/ovn/\\\",\\\"name\\\":\\\"node-log\\\"},{\\\"mountPath\\\":\\\"/run/ovn/\\\",\\\"name\\\":\\\"run-ovn\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-x92h2\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:174f36cdd47ef0d1d2099482919d773257453265a2af0b17b154edc32fa41ac2\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"ovn-controller\\\",\\\"ready\\\":false,\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"PodInitializing\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/ovnkube-lib\\\",\\\"name\\\":\\\"ovnkube-script-lib\\\"},{\\\"mountPath\\\":\\\"/run/openvswitch\\\",\\\"name\\\":\\\"run-openvswitch\\\"},{\\\"mountPath\\\":\\\"/run/ovn/\\\",\\\"name\\\":\\\"run-ovn\\\"},{\\\"mountPath\\\":\\\"/etc/openvswitch\\\",\\\"name\\\":\\\"etc-openvswitch\\\"},{\\\"mountPath\\\":\\\"/etc/ovn/\\\",\\\"name\\\":\\\"etc-openvswitch\\\"},{\\\"mountPath\\\":\\\"/var/lib/openvswitch\\\",\\\"name\\\":\\\"var-lib-openvswitch\\\"},{\\\"mountPath\\\":\\\"/env\\\",\\\"name\\\":\\\"env-overrides\\\"},{\\\"mountPath\\\":\\\"/var/log/ovn/\\\",\\\"name\\\":\\\"node-log\\\"},{\\\"mountPath\\\":\\\"/dev/log\\\",\\\"name\\\":\\\"log-socket\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-x92h2\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:174f36cdd47ef0d1d2099482919d773257453265a2af0b17b154edc32fa41ac2\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"ovnkube-controller\\\",\\\"ready\\\":false,\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"PodInitializing\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/ovnkube-lib\\\",\\\"name\\\":\\\"ovnkube-script-lib\\\"},{\\\"mountPath\\\":\\\"/var/lib/kubelet\\\",\\\"name\\\":\\\"host-kubelet\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/etc/systemd/system\\\",\\\"name\\\":\\\"systemd-units\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/host\\\",\\\"name\\\":\\\"host-slash\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/run/ovn-kubernetes/\\\",\\\"name\\\":\\\"host-run-ovn-kubernetes\\\"},{\\\"mountPath\\\":\\\"/run/netns\\\",\\\"name\\\":\\\"host-run-netns\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/run/systemd/private\\\",\\\"name\\\":\\\"run-systemd\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/cni-bin-dir\\\",\\\"name\\\":\\\"host-cni-bin\\\"},{\\\"mountPath\\\":\\\"/etc/cni/net.d\\\",\\\"name\\\":\\\"host-cni-netd\\\"},{\\\"mountPath\\\":\\\"/var/lib/cni/networks/ovn-k8s-cni-overlay\\\",\\\"name\\\":\\\"host-var-lib-cni-networks-ovn-kubernetes\\\"},{\\\"mountPath\\\":\\\"/run/openvswitch\\\",\\\"name\\\":\\\"run-openvswitch\\\"},{\\\"mountPath\\\":\\\"/var/log/ovnkube/\\\",\\\"name\\\":\\\"etc-openvswitch\\\"},{\\\"mountPath\\\":\\\"/run/ovn/\\\",\\\"name\\\":\\\"run-ovn\\\"},{\\\"mountPath\\\":\\\"/etc/openvswitch\\\",\\\"name\\\":\\\"etc-openvswitch\\\"},{\\\"mountPath\\\":\\\"/etc/ovn/\\\",\\\"name\\\":\\\"etc-openvswitch\\\"},{\\\"mountPath\\\":\\\"/var/lib/openvswitch\\\",\\\"name\\\":\\\"var-lib-openvswitch\\\"},{\\\"mountPath\\\":\\\"/run/ovnkube-config/\\\",\\\"name\\\":\\\"ovnkube-config\\\"},{\\\"mountPath\\\":\\\"/env\\\",\\\"name\\\":\\\"env-overrides\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-x92h2\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:174f36cdd47ef0d1d2099482919d773257453265a2af0b17b154edc32fa41ac2\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"sbdb\\\",\\\"ready\\\":false,\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"PodInitializing\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/ovnkube-lib\\\",\\\"name\\\":\\\"ovnkube-script-lib\\\"},{\\\"mountPath\\\":\\\"/etc/ovn/\\\",\\\"name\\\":\\\"etc-openvswitch\\\"},{\\\"mountPath\\\":\\\"/run/ovn/\\\",\\\"name\\\":\\\"run-ovn\\\"},{\\\"mountPath\\\":\\\"/var/log/ovn\\\",\\\"name\\\":\\\"node-log\\\"},{\\\"mountPath\\\":\\\"/env\\\",\\\"name\\\":\\\"env-overrides\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-x92h2\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}],\\\"hostIP\\\":\\\"192.168.126.11\\\",\\\"hostIPs\\\":[{\\\"ip\\\":\\\"192.168.126.11\\\"}],\\\"initContainerStatuses\\\":[{\\\"containerID\\\":\\\"cri-o://4def7c1172d2ae52e275b2371cf88c9f93c3f0c747c18de7d1073d50b03b19bc\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:174f36cdd47ef0d1d2099482919d773257453265a2af0b17b154edc32fa41ac2\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:174f36cdd47ef0d1d2099482919d773257453265a2af0b17b154edc32fa41ac2\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"kubecfg-setup\\\",\\\"ready\\\":true,\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://4def7c1172d2ae52e275b2371cf88c9f93c3f0c747c18de7d1073d50b03b19bc\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2025-12-03T11:20:32Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2025-12-03T11:20:32Z\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/etc/ovn/\\\",\\\"name\\\":\\\"etc-openvswitch\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-x92h2\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}],\\\"podIP\\\":\\\"192.168.126.11\\\",\\\"podIPs\\\":[{\\\"ip\\\":\\\"192.168.126.11\\\"}],\\\"startTime\\\":\\\"2025-12-03T11:20:31Z\\\"}}\" for pod \"openshift-ovn-kubernetes\"/\"ovnkube-node-kg55h\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://127.0.0.1:9743/pod?timeout=10s\": tls: failed to verify certificate: x509: certificate has expired or is not yet valid: current time 2025-12-03T11:20:34Z is after 2025-08-24T17:21:41Z" Dec 03 11:20:34 crc kubenswrapper[5052]: I1203 11:20:34.091457 5052 status_manager.go:875] "Failed to update status for pod" pod="openshift-kube-controller-manager/kube-controller-manager-crc" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"f31e1704-9c1d-4044-b318-2aaccb5896e2\\\"},\\\"status\\\":{\\\"conditions\\\":[{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:15Z\\\",\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:09Z\\\",\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"Initialized\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"ContainersReady\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:09Z\\\",\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"PodScheduled\\\"}],\\\"containerStatuses\\\":[{\\\"containerID\\\":\\\"cri-o://37485762ea1808cd3794c9641a4ead224b107e6c64c295de237f8ffd50aba4f4\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c0f9da410c07372b6c9ad6a79379b491cd10fdee88051c026b084652d85aed21\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c0f9da410c07372b6c9ad6a79379b491cd10fdee88051c026b084652d85aed21\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"cluster-policy-controller\\\",\\\"ready\\\":true,\\\"restartCount\\\":0,\\\"started\\\":true,\\\"state\\\":{\\\"running\\\":{\\\"startedAt\\\":\\\"2025-12-03T11:20:12Z\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/etc/kubernetes/static-pod-resources\\\",\\\"name\\\":\\\"resource-dir\\\"},{\\\"mountPath\\\":\\\"/etc/kubernetes/static-pod-certs\\\",\\\"name\\\":\\\"cert-dir\\\"}]},{\\\"containerID\\\":\\\"cri-o://f5d23c6160b5790e9272bd5ae83472267f4d1f60b67e0425a6bc2f5c1f9d1fb7\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:06bc35825771aee1220d34720243b89c4ba8a8b335e6de2597126bd791fd90d4\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:06bc35825771aee1220d34720243b89c4ba8a8b335e6de2597126bd791fd90d4\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"kube-controller-manager\\\",\\\"ready\\\":true,\\\"restartCount\\\":0,\\\"started\\\":true,\\\"state\\\":{\\\"running\\\":{\\\"startedAt\\\":\\\"2025-12-03T11:20:11Z\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/etc/kubernetes/static-pod-resources\\\",\\\"name\\\":\\\"resource-dir\\\"},{\\\"mountPath\\\":\\\"/etc/kubernetes/static-pod-certs\\\",\\\"name\\\":\\\"cert-dir\\\"}]},{\\\"containerID\\\":\\\"cri-o://d05df6945d11d2abe73005efafe80220497a339008d4c28f7923c2dc7ac4a150\\\",\\\"image\\\":\\\"quay.io/crcont/openshift-crc-cluster-kube-controller-manager-operator@sha256:8506ce0a578bc18fac117eb2b82799488ffac0bed08287faaf92edaf5d17ab95\\\",\\\"imageID\\\":\\\"quay.io/crcont/openshift-crc-cluster-kube-controller-manager-operator@sha256:8506ce0a578bc18fac117eb2b82799488ffac0bed08287faaf92edaf5d17ab95\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"kube-controller-manager-cert-syncer\\\",\\\"ready\\\":true,\\\"restartCount\\\":0,\\\"started\\\":true,\\\"state\\\":{\\\"running\\\":{\\\"startedAt\\\":\\\"2025-12-03T11:20:12Z\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/etc/kubernetes/static-pod-resources\\\",\\\"name\\\":\\\"resource-dir\\\"},{\\\"mountPath\\\":\\\"/etc/kubernetes/static-pod-certs\\\",\\\"name\\\":\\\"cert-dir\\\"}]},{\\\"containerID\\\":\\\"cri-o://8df2aa2470046bba37b41cb301e96f52b09b83124384b5dbc4942cd0bfeef1c6\\\",\\\"image\\\":\\\"quay.io/crcont/openshift-crc-cluster-kube-controller-manager-operator@sha256:8506ce0a578bc18fac117eb2b82799488ffac0bed08287faaf92edaf5d17ab95\\\",\\\"imageID\\\":\\\"quay.io/crcont/openshift-crc-cluster-kube-controller-manager-operator@sha256:8506ce0a578bc18fac117eb2b82799488ffac0bed08287faaf92edaf5d17ab95\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"kube-controller-manager-recovery-controller\\\",\\\"ready\\\":true,\\\"restartCount\\\":0,\\\"started\\\":true,\\\"state\\\":{\\\"running\\\":{\\\"startedAt\\\":\\\"2025-12-03T11:20:15Z\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/etc/kubernetes/static-pod-resources\\\",\\\"name\\\":\\\"resource-dir\\\"},{\\\"mountPath\\\":\\\"/etc/kubernetes/static-pod-certs\\\",\\\"name\\\":\\\"cert-dir\\\"}]}],\\\"hostIP\\\":\\\"192.168.126.11\\\",\\\"hostIPs\\\":[{\\\"ip\\\":\\\"192.168.126.11\\\"}],\\\"phase\\\":\\\"Running\\\",\\\"podIP\\\":\\\"192.168.126.11\\\",\\\"podIPs\\\":[{\\\"ip\\\":\\\"192.168.126.11\\\"}],\\\"startTime\\\":\\\"2025-12-03T11:20:09Z\\\"}}\" for pod \"openshift-kube-controller-manager\"/\"kube-controller-manager-crc\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://127.0.0.1:9743/pod?timeout=10s\": tls: failed to verify certificate: x509: certificate has expired or is not yet valid: current time 2025-12-03T11:20:34Z is after 2025-08-24T17:21:41Z" Dec 03 11:20:34 crc kubenswrapper[5052]: I1203 11:20:34.114002 5052 status_manager.go:875] "Failed to update status for pod" pod="openshift-etcd/etcd-crc" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"bc6c0f8f-e9c0-47cd-89fa-974113e2c13a\\\"},\\\"status\\\":{\\\"conditions\\\":[{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:11Z\\\",\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:15Z\\\",\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"Initialized\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:32Z\\\",\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:32Z\\\",\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"ContainersReady\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:09Z\\\",\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"PodScheduled\\\"}],\\\"containerStatuses\\\":[{\\\"containerID\\\":\\\"cri-o://f3bd857c0d35ed129a4fdc96deae82d6457f5f24c4d92a129972ce97a89b3488\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:07b7c6877441ecd6a5646fb68e33e9be8b90092272e49117b54b4a67314731ca\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:07b7c6877441ecd6a5646fb68e33e9be8b90092272e49117b54b4a67314731ca\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"etcd\\\",\\\"ready\\\":true,\\\"restartCount\\\":0,\\\"started\\\":true,\\\"state\\\":{\\\"running\\\":{\\\"startedAt\\\":\\\"2025-12-03T11:20:16Z\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/etc/kubernetes/manifests\\\",\\\"name\\\":\\\"static-pod-dir\\\"},{\\\"mountPath\\\":\\\"/etc/kubernetes/static-pod-resources\\\",\\\"name\\\":\\\"resource-dir\\\"},{\\\"mountPath\\\":\\\"/etc/kubernetes/static-pod-certs\\\",\\\"name\\\":\\\"cert-dir\\\"},{\\\"mountPath\\\":\\\"/var/lib/etcd/\\\",\\\"name\\\":\\\"data-dir\\\"}]},{\\\"containerID\\\":\\\"cri-o://c5dd3e47134939ea92cb1677d8d171eadd8c0c0743827ebee0a61daf261f4e30\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:07b7c6877441ecd6a5646fb68e33e9be8b90092272e49117b54b4a67314731ca\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:07b7c6877441ecd6a5646fb68e33e9be8b90092272e49117b54b4a67314731ca\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"etcd-metrics\\\",\\\"ready\\\":true,\\\"restartCount\\\":0,\\\"started\\\":true,\\\"state\\\":{\\\"running\\\":{\\\"startedAt\\\":\\\"2025-12-03T11:20:16Z\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/etc/kubernetes/static-pod-resources\\\",\\\"name\\\":\\\"resource-dir\\\"},{\\\"mountPath\\\":\\\"/etc/kubernetes/static-pod-certs\\\",\\\"name\\\":\\\"cert-dir\\\"},{\\\"mountPath\\\":\\\"/var/lib/etcd/\\\",\\\"name\\\":\\\"data-dir\\\"}]},{\\\"containerID\\\":\\\"cri-o://2409d840f8acda1e98e333c3b4d9ebd1b67f4a05fa3f662f6d7171dddab13846\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a0fa3723269019bee1847b26702f42928e779036cc2f58408f8ee7866be30a93\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a0fa3723269019bee1847b26702f42928e779036cc2f58408f8ee7866be30a93\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"etcd-readyz\\\",\\\"ready\\\":true,\\\"restartCount\\\":0,\\\"started\\\":true,\\\"state\\\":{\\\"running\\\":{\\\"startedAt\\\":\\\"2025-12-03T11:20:16Z\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/var/log/etcd/\\\",\\\"name\\\":\\\"log-dir\\\"},{\\\"mountPath\\\":\\\"/etc/kubernetes/static-pod-certs\\\",\\\"name\\\":\\\"cert-dir\\\"}]},{\\\"containerID\\\":\\\"cri-o://7e429db04e5ef00df1a3a67e1750865167bb14eef83a110aa54ce3ca2f4656c5\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a0fa3723269019bee1847b26702f42928e779036cc2f58408f8ee7866be30a93\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a0fa3723269019bee1847b26702f42928e779036cc2f58408f8ee7866be30a93\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"etcd-rev\\\",\\\"ready\\\":true,\\\"restartCount\\\":0,\\\"started\\\":true,\\\"state\\\":{\\\"running\\\":{\\\"startedAt\\\":\\\"2025-12-03T11:20:16Z\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/var/lib/etcd\\\",\\\"name\\\":\\\"data-dir\\\"},{\\\"mountPath\\\":\\\"/etc/kubernetes/static-pod-certs\\\",\\\"name\\\":\\\"cert-dir\\\"}]},{\\\"containerID\\\":\\\"cri-o://d9e3c1335ee9e6fe5d5ca827757da511034a270ca681b3cef694e595539d1cd0\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:07b7c6877441ecd6a5646fb68e33e9be8b90092272e49117b54b4a67314731ca\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:07b7c6877441ecd6a5646fb68e33e9be8b90092272e49117b54b4a67314731ca\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"etcdctl\\\",\\\"ready\\\":true,\\\"restartCount\\\":0,\\\"started\\\":true,\\\"state\\\":{\\\"running\\\":{\\\"startedAt\\\":\\\"2025-12-03T11:20:15Z\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/etc/kubernetes/manifests\\\",\\\"name\\\":\\\"static-pod-dir\\\"},{\\\"mountPath\\\":\\\"/etc/kubernetes/static-pod-resources\\\",\\\"name\\\":\\\"resource-dir\\\"},{\\\"mountPath\\\":\\\"/etc/kubernetes/static-pod-certs\\\",\\\"name\\\":\\\"cert-dir\\\"},{\\\"mountPath\\\":\\\"/var/lib/etcd/\\\",\\\"name\\\":\\\"data-dir\\\"}]}],\\\"hostIP\\\":\\\"192.168.126.11\\\",\\\"hostIPs\\\":[{\\\"ip\\\":\\\"192.168.126.11\\\"}],\\\"initContainerStatuses\\\":[{\\\"containerID\\\":\\\"cri-o://7d7c61963a9597f2c9c159d743284313820acece17152be4d829b0e3d02434c1\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:07b7c6877441ecd6a5646fb68e33e9be8b90092272e49117b54b4a67314731ca\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:07b7c6877441ecd6a5646fb68e33e9be8b90092272e49117b54b4a67314731ca\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"setup\\\",\\\"ready\\\":true,\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://7d7c61963a9597f2c9c159d743284313820acece17152be4d829b0e3d02434c1\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2025-12-03T11:20:11Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2025-12-03T11:20:11Z\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/var/log/etcd\\\",\\\"name\\\":\\\"log-dir\\\"}]},{\\\"containerID\\\":\\\"cri-o://e980e02c63b59597153749cc2ed54da5925e88a6a0ed2293f4075ff4b46bbe44\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:07b7c6877441ecd6a5646fb68e33e9be8b90092272e49117b54b4a67314731ca\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:07b7c6877441ecd6a5646fb68e33e9be8b90092272e49117b54b4a67314731ca\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"etcd-ensure-env-vars\\\",\\\"ready\\\":true,\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://e980e02c63b59597153749cc2ed54da5925e88a6a0ed2293f4075ff4b46bbe44\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2025-12-03T11:20:12Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2025-12-03T11:20:12Z\\\"}}},{\\\"containerID\\\":\\\"cri-o://6a0411ec57e822071155dd66aa1e30dcab88df41e229cf75208468e5dcf90358\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:07b7c6877441ecd6a5646fb68e33e9be8b90092272e49117b54b4a67314731ca\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:07b7c6877441ecd6a5646fb68e33e9be8b90092272e49117b54b4a67314731ca\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"etcd-resources-copy\\\",\\\"ready\\\":true,\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://6a0411ec57e822071155dd66aa1e30dcab88df41e229cf75208468e5dcf90358\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2025-12-03T11:20:15Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2025-12-03T11:20:15Z\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/etc/kubernetes/static-pod-resources\\\",\\\"name\\\":\\\"resource-dir\\\"},{\\\"mountPath\\\":\\\"/etc/kubernetes/static-pod-certs\\\",\\\"name\\\":\\\"cert-dir\\\"},{\\\"mountPath\\\":\\\"/usr/local/bin\\\",\\\"name\\\":\\\"usr-local-bin\\\"}]}],\\\"phase\\\":\\\"Running\\\",\\\"podIP\\\":\\\"192.168.126.11\\\",\\\"podIPs\\\":[{\\\"ip\\\":\\\"192.168.126.11\\\"}],\\\"startTime\\\":\\\"2025-12-03T11:20:09Z\\\"}}\" for pod \"openshift-etcd\"/\"etcd-crc\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://127.0.0.1:9743/pod?timeout=10s\": tls: failed to verify certificate: x509: certificate has expired or is not yet valid: current time 2025-12-03T11:20:34Z is after 2025-08-24T17:21:41Z" Dec 03 11:20:34 crc kubenswrapper[5052]: I1203 11:20:34.131710 5052 status_manager.go:875] "Failed to update status for pod" pod="openshift-kube-apiserver/kube-apiserver-crc" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"6c8df12a-3d11-4bb9-b398-67091c1918e4\\\"},\\\"status\\\":{\\\"conditions\\\":[{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:11Z\\\",\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:11Z\\\",\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"Initialized\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:09Z\\\",\\\"message\\\":\\\"containers with unready status: [kube-apiserver-check-endpoints]\\\",\\\"reason\\\":\\\"ContainersNotReady\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:09Z\\\",\\\"message\\\":\\\"containers with unready status: [kube-apiserver-check-endpoints]\\\",\\\"reason\\\":\\\"ContainersNotReady\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"ContainersReady\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:09Z\\\",\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"PodScheduled\\\"}],\\\"containerStatuses\\\":[{\\\"containerID\\\":\\\"cri-o://09256543a9aa86da8bec948be040d36b6e4986b9232091d2638bb1fead5740a9\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:06bc35825771aee1220d34720243b89c4ba8a8b335e6de2597126bd791fd90d4\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:06bc35825771aee1220d34720243b89c4ba8a8b335e6de2597126bd791fd90d4\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"kube-apiserver\\\",\\\"ready\\\":true,\\\"restartCount\\\":0,\\\"started\\\":true,\\\"state\\\":{\\\"running\\\":{\\\"startedAt\\\":\\\"2025-12-03T11:20:12Z\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/etc/kubernetes/static-pod-resources\\\",\\\"name\\\":\\\"resource-dir\\\"},{\\\"mountPath\\\":\\\"/etc/kubernetes/static-pod-certs\\\",\\\"name\\\":\\\"cert-dir\\\"},{\\\"mountPath\\\":\\\"/var/log/kube-apiserver\\\",\\\"name\\\":\\\"audit-dir\\\"}]},{\\\"containerID\\\":\\\"cri-o://15abb0d7fd5884519ed5ac454430dbb58f71ed1f459e5ed30f324f70befd243e\\\",\\\"image\\\":\\\"quay.io/crcont/openshift-crc-cluster-kube-apiserver-operator@sha256:9f36dc276e27753fc478274c7f7814a4f8945c987117ee1ea3b8e6355e6d7462\\\",\\\"imageID\\\":\\\"quay.io/crcont/openshift-crc-cluster-kube-apiserver-operator@sha256:9f36dc276e27753fc478274c7f7814a4f8945c987117ee1ea3b8e6355e6d7462\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"kube-apiserver-cert-regeneration-controller\\\",\\\"ready\\\":true,\\\"restartCount\\\":0,\\\"started\\\":true,\\\"state\\\":{\\\"running\\\":{\\\"startedAt\\\":\\\"2025-12-03T11:20:14Z\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/etc/kubernetes/static-pod-resources\\\",\\\"name\\\":\\\"resource-dir\\\"}]},{\\\"containerID\\\":\\\"cri-o://8ceedd21ae3833d77fd440537f30434ee57f087fbefb2357b46231d3f778ad2a\\\",\\\"image\\\":\\\"quay.io/crcont/openshift-crc-cluster-kube-apiserver-operator@sha256:9f36dc276e27753fc478274c7f7814a4f8945c987117ee1ea3b8e6355e6d7462\\\",\\\"imageID\\\":\\\"quay.io/crcont/openshift-crc-cluster-kube-apiserver-operator@sha256:9f36dc276e27753fc478274c7f7814a4f8945c987117ee1ea3b8e6355e6d7462\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"kube-apiserver-cert-syncer\\\",\\\"ready\\\":true,\\\"restartCount\\\":0,\\\"started\\\":true,\\\"state\\\":{\\\"running\\\":{\\\"startedAt\\\":\\\"2025-12-03T11:20:12Z\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/etc/kubernetes/static-pod-resources\\\",\\\"name\\\":\\\"resource-dir\\\"},{\\\"mountPath\\\":\\\"/etc/kubernetes/static-pod-certs\\\",\\\"name\\\":\\\"cert-dir\\\"}]},{\\\"containerID\\\":\\\"cri-o://ee9c5c61cfd6dee9fd44ebc4c7ef4d20b5fde82b12a8147c3c871e2615da7940\\\",\\\"image\\\":\\\"quay.io/crcont/openshift-crc-cluster-kube-apiserver-operator@sha256:9f36dc276e27753fc478274c7f7814a4f8945c987117ee1ea3b8e6355e6d7462\\\",\\\"imageID\\\":\\\"quay.io/crcont/openshift-crc-cluster-kube-apiserver-operator@sha256:9f36dc276e27753fc478274c7f7814a4f8945c987117ee1ea3b8e6355e6d7462\\\",\\\"lastState\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://b739f84524d09b27f6e052e21df8be3870a2136665704232f5a365d8a5bf41f5\\\",\\\"exitCode\\\":255,\\\"finishedAt\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"message\\\":\\\"ing back to namespace): Get \\\\\\\"https://localhost:6443/api/v1/namespaces/openshift-kube-apiserver/pods/kube-apiserver-crc\\\\\\\": net/http: TLS handshake timeout\\\\nI1203 11:20:25.911385 1 builder.go:304] check-endpoints version 4.18.0-202502101302.p0.g763313c.assembly.stream.el9-763313c-763313c860ea43fcfc9b1ac00ebae096b57c078e\\\\nI1203 11:20:25.912713 1 dynamic_serving_content.go:116] \\\\\\\"Loaded a new cert/key pair\\\\\\\" name=\\\\\\\"serving-cert::/tmp/serving-cert-3310779835/tls.crt::/tmp/serving-cert-3310779835/tls.key\\\\\\\"\\\\nI1203 11:20:31.324803 1 requestheader_controller.go:247] Loaded a new request header values for RequestHeaderAuthRequestController\\\\nI1203 11:20:31.327725 1 maxinflight.go:139] \\\\\\\"Initialized nonMutatingChan\\\\\\\" len=400\\\\nI1203 11:20:31.327838 1 maxinflight.go:145] \\\\\\\"Initialized mutatingChan\\\\\\\" len=200\\\\nI1203 11:20:31.327886 1 maxinflight.go:116] \\\\\\\"Set denominator for readonly requests\\\\\\\" limit=400\\\\nI1203 11:20:31.327915 1 maxinflight.go:120] \\\\\\\"Set denominator for mutating requests\\\\\\\" limit=200\\\\nI1203 11:20:31.335573 1 secure_serving.go:57] Forcing use of http/1.1 only\\\\nW1203 11:20:31.335607 1 secure_serving.go:69] Use of insecure cipher 'TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256' detected.\\\\nW1203 11:20:31.335613 1 secure_serving.go:69] Use of insecure cipher 'TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256' detected.\\\\nW1203 11:20:31.335619 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_128_GCM_SHA256' detected.\\\\nW1203 11:20:31.335623 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_256_GCM_SHA384' detected.\\\\nW1203 11:20:31.335626 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_128_CBC_SHA' detected.\\\\nW1203 11:20:31.335630 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_256_CBC_SHA' detected.\\\\nI1203 11:20:31.335893 1 genericapiserver.go:533] MuxAndDiscoveryComplete has all endpoints registered and discovery information is complete\\\\nF1203 11:20:31.338035 1 cmd.go:182] pods \\\\\\\"kube-apiserver-crc\\\\\\\" not found\\\\n\\\",\\\"reason\\\":\\\"Error\\\",\\\"startedAt\\\":\\\"2025-12-03T11:20:15Z\\\"}},\\\"name\\\":\\\"kube-apiserver-check-endpoints\\\",\\\"ready\\\":false,\\\"restartCount\\\":1,\\\"started\\\":true,\\\"state\\\":{\\\"running\\\":{\\\"startedAt\\\":\\\"2025-12-03T11:20:32Z\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/etc/kubernetes/static-pod-resources\\\",\\\"name\\\":\\\"resource-dir\\\"},{\\\"mountPath\\\":\\\"/etc/kubernetes/static-pod-certs\\\",\\\"name\\\":\\\"cert-dir\\\"}]},{\\\"containerID\\\":\\\"cri-o://22b8cb6aa04337d1420f97d3e128fce2b29a725ca6e5d157adeb1ba78f00073f\\\",\\\"image\\\":\\\"quay.io/crcont/openshift-crc-cluster-kube-apiserver-operator@sha256:9f36dc276e27753fc478274c7f7814a4f8945c987117ee1ea3b8e6355e6d7462\\\",\\\"imageID\\\":\\\"quay.io/crcont/openshift-crc-cluster-kube-apiserver-operator@sha256:9f36dc276e27753fc478274c7f7814a4f8945c987117ee1ea3b8e6355e6d7462\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"kube-apiserver-insecure-readyz\\\",\\\"ready\\\":true,\\\"restartCount\\\":0,\\\"started\\\":true,\\\"state\\\":{\\\"running\\\":{\\\"startedAt\\\":\\\"2025-12-03T11:20:15Z\\\"}}}],\\\"hostIP\\\":\\\"192.168.126.11\\\",\\\"hostIPs\\\":[{\\\"ip\\\":\\\"192.168.126.11\\\"}],\\\"initContainerStatuses\\\":[{\\\"containerID\\\":\\\"cri-o://7182c75773f069c72f87c39136222fd4a91b49add27352a1969645413e670f01\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:06bc35825771aee1220d34720243b89c4ba8a8b335e6de2597126bd791fd90d4\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:06bc35825771aee1220d34720243b89c4ba8a8b335e6de2597126bd791fd90d4\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"setup\\\",\\\"ready\\\":true,\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://7182c75773f069c72f87c39136222fd4a91b49add27352a1969645413e670f01\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2025-12-03T11:20:11Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2025-12-03T11:20:11Z\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/var/log/kube-apiserver\\\",\\\"name\\\":\\\"audit-dir\\\"}]}],\\\"phase\\\":\\\"Running\\\",\\\"podIP\\\":\\\"192.168.126.11\\\",\\\"podIPs\\\":[{\\\"ip\\\":\\\"192.168.126.11\\\"}],\\\"startTime\\\":\\\"2025-12-03T11:20:09Z\\\"}}\" for pod \"openshift-kube-apiserver\"/\"kube-apiserver-crc\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://127.0.0.1:9743/pod?timeout=10s\": tls: failed to verify certificate: x509: certificate has expired or is not yet valid: current time 2025-12-03T11:20:34Z is after 2025-08-24T17:21:41Z" Dec 03 11:20:34 crc kubenswrapper[5052]: I1203 11:20:34.151721 5052 status_manager.go:875] "Failed to update status for pod" pod="openshift-network-diagnostics/network-check-target-xd92c" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"3b6479f0-333b-4a96-9adf-2099afdc2447\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"message\\\":\\\"containers with unready status: [network-check-target-container]\\\",\\\"reason\\\":\\\"ContainersNotReady\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"message\\\":\\\"containers with unready status: [network-check-target-container]\\\",\\\"reason\\\":\\\"ContainersNotReady\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"ContainersReady\\\"}],\\\"containerStatuses\\\":[{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e1baa38811c04bd8909e01a1f3be7421a1cb99d608d3dc4cf86d95b17de2ab8b\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{\\\"terminated\\\":{\\\"exitCode\\\":137,\\\"finishedAt\\\":null,\\\"message\\\":\\\"The container could not be located when the pod was deleted. The container used to be Running\\\",\\\"reason\\\":\\\"ContainerStatusUnknown\\\",\\\"startedAt\\\":null}},\\\"name\\\":\\\"network-check-target-container\\\",\\\"ready\\\":false,\\\"restartCount\\\":3,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"ContainerCreating\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-cqllr\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}],\\\"podIP\\\":null,\\\"podIPs\\\":null}}\" for pod \"openshift-network-diagnostics\"/\"network-check-target-xd92c\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://127.0.0.1:9743/pod?timeout=10s\": tls: failed to verify certificate: x509: certificate has expired or is not yet valid: current time 2025-12-03T11:20:34Z is after 2025-08-24T17:21:41Z" Dec 03 11:20:34 crc kubenswrapper[5052]: I1203 11:20:34.164281 5052 status_manager.go:875] "Failed to update status for pod" pod="openshift-dns/node-resolver-xph69" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"e5c587e6-866a-4427-8a2c-3469cf0fc495\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:32Z\\\",\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"Initialized\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:32Z\\\",\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:32Z\\\",\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"ContainersReady\\\"}],\\\"containerStatuses\\\":[{\\\"containerID\\\":\\\"cri-o://8cbacdbdef2832bba669718580c35080702354f16ec06e7d2680a6ee5b63e8b3\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:35512335ac39aed0f55b7f799f416f4f6445c20c1b19888cf2bb72bb276703f2\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:35512335ac39aed0f55b7f799f416f4f6445c20c1b19888cf2bb72bb276703f2\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"dns-node-resolver\\\",\\\"ready\\\":true,\\\"restartCount\\\":0,\\\"started\\\":true,\\\"state\\\":{\\\"running\\\":{\\\"startedAt\\\":\\\"2025-12-03T11:20:32Z\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/etc/hosts\\\",\\\"name\\\":\\\"hosts-file\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-tk7n6\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}],\\\"hostIP\\\":\\\"192.168.126.11\\\",\\\"hostIPs\\\":[{\\\"ip\\\":\\\"192.168.126.11\\\"}],\\\"phase\\\":\\\"Running\\\",\\\"podIP\\\":\\\"192.168.126.11\\\",\\\"podIPs\\\":[{\\\"ip\\\":\\\"192.168.126.11\\\"}],\\\"startTime\\\":\\\"2025-12-03T11:20:31Z\\\"}}\" for pod \"openshift-dns\"/\"node-resolver-xph69\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://127.0.0.1:9743/pod?timeout=10s\": tls: failed to verify certificate: x509: certificate has expired or is not yet valid: current time 2025-12-03T11:20:34Z is after 2025-08-24T17:21:41Z" Dec 03 11:20:34 crc kubenswrapper[5052]: I1203 11:20:34.184545 5052 status_manager.go:875] "Failed to update status for pod" pod="openshift-multus/multus-additional-cni-plugins-2lrql" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"17265a68-bcb5-455c-b7df-c148a6517db9\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:32Z\\\",\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"message\\\":\\\"containers with incomplete status: [cni-plugins bond-cni-plugin routeoverride-cni whereabouts-cni-bincopy whereabouts-cni]\\\",\\\"reason\\\":\\\"ContainersNotInitialized\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"Initialized\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"message\\\":\\\"containers with unready status: [kube-multus-additional-cni-plugins]\\\",\\\"reason\\\":\\\"ContainersNotReady\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"message\\\":\\\"containers with unready status: [kube-multus-additional-cni-plugins]\\\",\\\"reason\\\":\\\"ContainersNotReady\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"ContainersReady\\\"}],\\\"containerStatuses\\\":[{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7eeaee65f2808b819eedb413bdcabb9144e12f0dd97f13fd1afba93a95b67b26\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"kube-multus-additional-cni-plugins\\\",\\\"ready\\\":false,\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"PodInitializing\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-w4zf5\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}],\\\"hostIP\\\":\\\"192.168.126.11\\\",\\\"hostIPs\\\":[{\\\"ip\\\":\\\"192.168.126.11\\\"}],\\\"initContainerStatuses\\\":[{\\\"containerID\\\":\\\"cri-o://ce6a0f2a385f3ad9f34f3a09e960700dc4d3941341c08997b95c648c82d11749\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e40792096b162f0f9ce5f8362f51e5f8dea2c1ce4b1447235388416b5db7708c\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e40792096b162f0f9ce5f8362f51e5f8dea2c1ce4b1447235388416b5db7708c\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"egress-router-binary-copy\\\",\\\"ready\\\":true,\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://ce6a0f2a385f3ad9f34f3a09e960700dc4d3941341c08997b95c648c82d11749\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2025-12-03T11:20:33Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2025-12-03T11:20:32Z\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/entrypoint\\\",\\\"name\\\":\\\"cni-binary-copy\\\"},{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/os-release\\\",\\\"name\\\":\\\"os-release\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-w4zf5\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:687fddfbb085a1688df312ce4ec8c857df9b2daed8ff4a7ed6163a1154afa2cc\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"cni-plugins\\\",\\\"ready\\\":false,\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"PodInitializing\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/entrypoint\\\",\\\"name\\\":\\\"cni-binary-copy\\\"},{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/os-release\\\",\\\"name\\\":\\\"os-release\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/host/etc/cni/tuning/\\\",\\\"name\\\":\\\"tuning-conf-dir\\\"},{\\\"mountPath\\\":\\\"/sysctls\\\",\\\"name\\\":\\\"cni-sysctl-allowlist\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-w4zf5\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:383f4cceeeaead203bb2327fdd367c64b64d729d7fa93089f249e496fcef0c78\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"bond-cni-plugin\\\",\\\"ready\\\":false,\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"PodInitializing\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/entrypoint\\\",\\\"name\\\":\\\"cni-binary-copy\\\"},{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/os-release\\\",\\\"name\\\":\\\"os-release\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-w4zf5\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f567acb85146b5ed81451ec3e79f2de0c62e28c69b2eeade0abdf5d0c388e7aa\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"routeoverride-cni\\\",\\\"ready\\\":false,\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"PodInitializing\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/entrypoint\\\",\\\"name\\\":\\\"cni-binary-copy\\\"},{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/os-release\\\",\\\"name\\\":\\\"os-release\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-w4zf5\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:98100674616e54319f6713d742fd0c3bdbc84e6e6173e8ccf4a2473a714c2bc4\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"whereabouts-cni-bincopy\\\",\\\"ready\\\":false,\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"PodInitializing\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/entrypoint\\\",\\\"name\\\":\\\"cni-binary-copy\\\"},{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/os-release\\\",\\\"name\\\":\\\"os-release\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-w4zf5\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:98100674616e54319f6713d742fd0c3bdbc84e6e6173e8ccf4a2473a714c2bc4\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"whereabouts-cni\\\",\\\"ready\\\":false,\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"PodInitializing\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/etc/cni/net.d\\\",\\\"name\\\":\\\"system-cni-dir\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-w4zf5\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}],\\\"podIP\\\":\\\"192.168.126.11\\\",\\\"podIPs\\\":[{\\\"ip\\\":\\\"192.168.126.11\\\"}],\\\"startTime\\\":\\\"2025-12-03T11:20:31Z\\\"}}\" for pod \"openshift-multus\"/\"multus-additional-cni-plugins-2lrql\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://127.0.0.1:9743/pod?timeout=10s\": tls: failed to verify certificate: x509: certificate has expired or is not yet valid: current time 2025-12-03T11:20:34Z is after 2025-08-24T17:21:41Z" Dec 03 11:20:34 crc kubenswrapper[5052]: I1203 11:20:34.200500 5052 status_manager.go:875] "Failed to update status for pod" pod="openshift-multus/multus-hx657" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"c30c02db-f4e7-4432-941c-5448fd71f3d9\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:32Z\\\",\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"Initialized\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:32Z\\\",\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:32Z\\\",\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"ContainersReady\\\"}],\\\"containerStatuses\\\":[{\\\"containerID\\\":\\\"cri-o://388fefa2a0e0ba8ca09e3fd39371e442358e9547cc2789ca8fa4b7b7237f6ac5\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7eeaee65f2808b819eedb413bdcabb9144e12f0dd97f13fd1afba93a95b67b26\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7eeaee65f2808b819eedb413bdcabb9144e12f0dd97f13fd1afba93a95b67b26\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"kube-multus\\\",\\\"ready\\\":true,\\\"restartCount\\\":0,\\\"started\\\":true,\\\"state\\\":{\\\"running\\\":{\\\"startedAt\\\":\\\"2025-12-03T11:20:32Z\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/entrypoint\\\",\\\"name\\\":\\\"cni-binary-copy\\\"},{\\\"mountPath\\\":\\\"/host/etc/os-release\\\",\\\"name\\\":\\\"os-release\\\"},{\\\"mountPath\\\":\\\"/host/etc/cni/net.d\\\",\\\"name\\\":\\\"system-cni-dir\\\"},{\\\"mountPath\\\":\\\"/host/run/multus/cni/net.d\\\",\\\"name\\\":\\\"multus-cni-dir\\\"},{\\\"mountPath\\\":\\\"/host/opt/cni/bin\\\",\\\"name\\\":\\\"cnibin\\\"},{\\\"mountPath\\\":\\\"/host/run/multus\\\",\\\"name\\\":\\\"multus-socket-dir-parent\\\"},{\\\"mountPath\\\":\\\"/run/k8s.cni.cncf.io\\\",\\\"name\\\":\\\"host-run-k8s-cni-cncf-io\\\"},{\\\"mountPath\\\":\\\"/run/netns\\\",\\\"name\\\":\\\"host-run-netns\\\"},{\\\"mountPath\\\":\\\"/var/lib/cni/bin\\\",\\\"name\\\":\\\"host-var-lib-cni-bin\\\"},{\\\"mountPath\\\":\\\"/var/lib/cni/multus\\\",\\\"name\\\":\\\"host-var-lib-cni-multus\\\"},{\\\"mountPath\\\":\\\"/var/lib/kubelet\\\",\\\"name\\\":\\\"host-var-lib-kubelet\\\"},{\\\"mountPath\\\":\\\"/hostroot\\\",\\\"name\\\":\\\"hostroot\\\"},{\\\"mountPath\\\":\\\"/etc/cni/multus/net.d\\\",\\\"name\\\":\\\"multus-conf-dir\\\"},{\\\"mountPath\\\":\\\"/etc/cni/net.d/multus.d\\\",\\\"name\\\":\\\"multus-daemon-config\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/etc/cni/multus/certs\\\",\\\"name\\\":\\\"host-run-multus-certs\\\"},{\\\"mountPath\\\":\\\"/etc/kubernetes\\\",\\\"name\\\":\\\"etc-kubernetes\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-fhsjl\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}],\\\"hostIP\\\":\\\"192.168.126.11\\\",\\\"hostIPs\\\":[{\\\"ip\\\":\\\"192.168.126.11\\\"}],\\\"phase\\\":\\\"Running\\\",\\\"podIP\\\":\\\"192.168.126.11\\\",\\\"podIPs\\\":[{\\\"ip\\\":\\\"192.168.126.11\\\"}],\\\"startTime\\\":\\\"2025-12-03T11:20:31Z\\\"}}\" for pod \"openshift-multus\"/\"multus-hx657\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://127.0.0.1:9743/pod?timeout=10s\": tls: failed to verify certificate: x509: certificate has expired or is not yet valid: current time 2025-12-03T11:20:34Z is after 2025-08-24T17:21:41Z" Dec 03 11:20:34 crc kubenswrapper[5052]: I1203 11:20:34.220838 5052 status_manager.go:875] "Failed to update status for pod" pod="openshift-network-operator/network-operator-58b4c7f79c-55gtf" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"37a5e44f-9a88-4405-be8a-b645485e7312\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:32Z\\\",\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:32Z\\\",\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:32Z\\\",\\\"type\\\":\\\"ContainersReady\\\"}],\\\"containerStatuses\\\":[{\\\"containerID\\\":\\\"cri-o://177058541a1de284ec4275037302f876e3345d48400f5edc9466508d43518412\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e1baa38811c04bd8909e01a1f3be7421a1cb99d608d3dc4cf86d95b17de2ab8b\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e1baa38811c04bd8909e01a1f3be7421a1cb99d608d3dc4cf86d95b17de2ab8b\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"network-operator\\\",\\\"ready\\\":true,\\\"restartCount\\\":0,\\\"started\\\":true,\\\"state\\\":{\\\"running\\\":{\\\"startedAt\\\":\\\"2025-12-03T11:20:32Z\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/etc/kubernetes\\\",\\\"name\\\":\\\"host-etc-kube\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/serving-cert\\\",\\\"name\\\":\\\"metrics-tls\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-rdwmf\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}]}}\" for pod \"openshift-network-operator\"/\"network-operator-58b4c7f79c-55gtf\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://127.0.0.1:9743/pod?timeout=10s\": tls: failed to verify certificate: x509: certificate has expired or is not yet valid: current time 2025-12-03T11:20:34Z is after 2025-08-24T17:21:41Z" Dec 03 11:20:34 crc kubenswrapper[5052]: I1203 11:20:34.237605 5052 status_manager.go:875] "Failed to update status for pod" pod="openshift-network-node-identity/network-node-identity-vrzqb" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"ef543e1b-8068-4ea3-b32a-61027b32e95d\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:33Z\\\",\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:33Z\\\",\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:33Z\\\",\\\"type\\\":\\\"ContainersReady\\\"}],\\\"containerStatuses\\\":[{\\\"containerID\\\":\\\"cri-o://e42188cae70b530a8f9b32c15799070a3c56d867fe86f378b8edef8b93deed45\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:174f36cdd47ef0d1d2099482919d773257453265a2af0b17b154edc32fa41ac2\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:174f36cdd47ef0d1d2099482919d773257453265a2af0b17b154edc32fa41ac2\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"approver\\\",\\\"ready\\\":true,\\\"restartCount\\\":0,\\\"started\\\":true,\\\"state\\\":{\\\"running\\\":{\\\"startedAt\\\":\\\"2025-12-03T11:20:33Z\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/env\\\",\\\"name\\\":\\\"env-overrides\\\"},{\\\"mountPath\\\":\\\"/var/run/ovnkube-identity-config\\\",\\\"name\\\":\\\"ovnkube-identity-cm\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-s2kz5\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"containerID\\\":\\\"cri-o://13aaf5ec87e48b83ec6508ce67110db3b2800f9fa1ca96fd64160d3761248fdd\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:174f36cdd47ef0d1d2099482919d773257453265a2af0b17b154edc32fa41ac2\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:174f36cdd47ef0d1d2099482919d773257453265a2af0b17b154edc32fa41ac2\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"webhook\\\",\\\"ready\\\":true,\\\"restartCount\\\":0,\\\"started\\\":true,\\\"state\\\":{\\\"running\\\":{\\\"startedAt\\\":\\\"2025-12-03T11:20:32Z\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/etc/webhook-cert/\\\",\\\"name\\\":\\\"webhook-cert\\\"},{\\\"mountPath\\\":\\\"/env\\\",\\\"name\\\":\\\"env-overrides\\\"},{\\\"mountPath\\\":\\\"/var/run/ovnkube-identity-config\\\",\\\"name\\\":\\\"ovnkube-identity-cm\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-s2kz5\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}]}}\" for pod \"openshift-network-node-identity\"/\"network-node-identity-vrzqb\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://127.0.0.1:9743/pod?timeout=10s\": tls: failed to verify certificate: x509: certificate has expired or is not yet valid: current time 2025-12-03T11:20:34Z is after 2025-08-24T17:21:41Z" Dec 03 11:20:34 crc kubenswrapper[5052]: I1203 11:20:34.253335 5052 status_manager.go:875] "Failed to update status for pod" pod="openshift-network-console/networking-console-plugin-85b44fc459-gdk6g" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"5fe485a1-e14f-4c09-b5b9-f252bc42b7e8\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"message\\\":\\\"containers with unready status: [networking-console-plugin]\\\",\\\"reason\\\":\\\"ContainersNotReady\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"message\\\":\\\"containers with unready status: [networking-console-plugin]\\\",\\\"reason\\\":\\\"ContainersNotReady\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"ContainersReady\\\"}],\\\"containerStatuses\\\":[{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ae647598ec35cda5766806d3d44a91e3b9d4dee48ff154f3d8490165399873fd\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{\\\"terminated\\\":{\\\"exitCode\\\":137,\\\"finishedAt\\\":null,\\\"message\\\":\\\"The container could not be located when the pod was deleted. The container used to be Running\\\",\\\"reason\\\":\\\"ContainerStatusUnknown\\\",\\\"startedAt\\\":null}},\\\"name\\\":\\\"networking-console-plugin\\\",\\\"ready\\\":false,\\\"restartCount\\\":3,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"ContainerCreating\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/var/cert\\\",\\\"name\\\":\\\"networking-console-plugin-cert\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/etc/nginx/nginx.conf\\\",\\\"name\\\":\\\"nginx-conf\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}],\\\"podIP\\\":null,\\\"podIPs\\\":null}}\" for pod \"openshift-network-console\"/\"networking-console-plugin-85b44fc459-gdk6g\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://127.0.0.1:9743/pod?timeout=10s\": tls: failed to verify certificate: x509: certificate has expired or is not yet valid: current time 2025-12-03T11:20:34Z is after 2025-08-24T17:21:41Z" Dec 03 11:20:34 crc kubenswrapper[5052]: I1203 11:20:34.274915 5052 status_manager.go:875] "Failed to update status for pod" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"10c237bc-72e7-4da3-9f2c-b1688395bf6e\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:33Z\\\",\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"Initialized\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:33Z\\\",\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:33Z\\\",\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"ContainersReady\\\"}],\\\"containerStatuses\\\":[{\\\"containerID\\\":\\\"cri-o://b9c20844de46e525f8cdce935c43fc2e236f7bd4980cab6e4b13a967e18ffb0a\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:242b3d66438c42745f4ef318bdeaf3d793426f12962a42ea83e18d06c08aaf09\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:242b3d66438c42745f4ef318bdeaf3d793426f12962a42ea83e18d06c08aaf09\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"kube-rbac-proxy\\\",\\\"ready\\\":true,\\\"restartCount\\\":0,\\\"started\\\":true,\\\"state\\\":{\\\"running\\\":{\\\"startedAt\\\":\\\"2025-12-03T11:20:33Z\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/etc/tls/private\\\",\\\"name\\\":\\\"proxy-tls\\\"},{\\\"mountPath\\\":\\\"/etc/kube-rbac-proxy\\\",\\\"name\\\":\\\"mcd-auth-proxy-config\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-9vkjv\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"containerID\\\":\\\"cri-o://1413108bd79cded55a270a3437cda42aab7c3db3cfb81894fade4363bbb28f2d\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c915fb8ba96e911699a1ae34a8e95ca8a9fbe1bf8c28fea177225c63a8bdfc0a\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c915fb8ba96e911699a1ae34a8e95ca8a9fbe1bf8c28fea177225c63a8bdfc0a\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"machine-config-daemon\\\",\\\"ready\\\":true,\\\"restartCount\\\":0,\\\"started\\\":true,\\\"state\\\":{\\\"running\\\":{\\\"startedAt\\\":\\\"2025-12-03T11:20:32Z\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/rootfs\\\",\\\"name\\\":\\\"rootfs\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-9vkjv\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}],\\\"hostIP\\\":\\\"192.168.126.11\\\",\\\"hostIPs\\\":[{\\\"ip\\\":\\\"192.168.126.11\\\"}],\\\"phase\\\":\\\"Running\\\",\\\"podIP\\\":\\\"192.168.126.11\\\",\\\"podIPs\\\":[{\\\"ip\\\":\\\"192.168.126.11\\\"}],\\\"startTime\\\":\\\"2025-12-03T11:20:31Z\\\"}}\" for pod \"openshift-machine-config-operator\"/\"machine-config-daemon-74bj6\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://127.0.0.1:9743/pod?timeout=10s\": tls: failed to verify certificate: x509: certificate has expired or is not yet valid: current time 2025-12-03T11:20:34Z is after 2025-08-24T17:21:41Z" Dec 03 11:20:34 crc kubenswrapper[5052]: I1203 11:20:34.297338 5052 status_manager.go:875] "Failed to update status for pod" pod="openshift-kube-controller-manager/kube-controller-manager-crc" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"f31e1704-9c1d-4044-b318-2aaccb5896e2\\\"},\\\"status\\\":{\\\"conditions\\\":[{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:15Z\\\",\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:09Z\\\",\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"Initialized\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"ContainersReady\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:09Z\\\",\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"PodScheduled\\\"}],\\\"containerStatuses\\\":[{\\\"containerID\\\":\\\"cri-o://37485762ea1808cd3794c9641a4ead224b107e6c64c295de237f8ffd50aba4f4\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c0f9da410c07372b6c9ad6a79379b491cd10fdee88051c026b084652d85aed21\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c0f9da410c07372b6c9ad6a79379b491cd10fdee88051c026b084652d85aed21\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"cluster-policy-controller\\\",\\\"ready\\\":true,\\\"restartCount\\\":0,\\\"started\\\":true,\\\"state\\\":{\\\"running\\\":{\\\"startedAt\\\":\\\"2025-12-03T11:20:12Z\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/etc/kubernetes/static-pod-resources\\\",\\\"name\\\":\\\"resource-dir\\\"},{\\\"mountPath\\\":\\\"/etc/kubernetes/static-pod-certs\\\",\\\"name\\\":\\\"cert-dir\\\"}]},{\\\"containerID\\\":\\\"cri-o://f5d23c6160b5790e9272bd5ae83472267f4d1f60b67e0425a6bc2f5c1f9d1fb7\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:06bc35825771aee1220d34720243b89c4ba8a8b335e6de2597126bd791fd90d4\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:06bc35825771aee1220d34720243b89c4ba8a8b335e6de2597126bd791fd90d4\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"kube-controller-manager\\\",\\\"ready\\\":true,\\\"restartCount\\\":0,\\\"started\\\":true,\\\"state\\\":{\\\"running\\\":{\\\"startedAt\\\":\\\"2025-12-03T11:20:11Z\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/etc/kubernetes/static-pod-resources\\\",\\\"name\\\":\\\"resource-dir\\\"},{\\\"mountPath\\\":\\\"/etc/kubernetes/static-pod-certs\\\",\\\"name\\\":\\\"cert-dir\\\"}]},{\\\"containerID\\\":\\\"cri-o://d05df6945d11d2abe73005efafe80220497a339008d4c28f7923c2dc7ac4a150\\\",\\\"image\\\":\\\"quay.io/crcont/openshift-crc-cluster-kube-controller-manager-operator@sha256:8506ce0a578bc18fac117eb2b82799488ffac0bed08287faaf92edaf5d17ab95\\\",\\\"imageID\\\":\\\"quay.io/crcont/openshift-crc-cluster-kube-controller-manager-operator@sha256:8506ce0a578bc18fac117eb2b82799488ffac0bed08287faaf92edaf5d17ab95\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"kube-controller-manager-cert-syncer\\\",\\\"ready\\\":true,\\\"restartCount\\\":0,\\\"started\\\":true,\\\"state\\\":{\\\"running\\\":{\\\"startedAt\\\":\\\"2025-12-03T11:20:12Z\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/etc/kubernetes/static-pod-resources\\\",\\\"name\\\":\\\"resource-dir\\\"},{\\\"mountPath\\\":\\\"/etc/kubernetes/static-pod-certs\\\",\\\"name\\\":\\\"cert-dir\\\"}]},{\\\"containerID\\\":\\\"cri-o://8df2aa2470046bba37b41cb301e96f52b09b83124384b5dbc4942cd0bfeef1c6\\\",\\\"image\\\":\\\"quay.io/crcont/openshift-crc-cluster-kube-controller-manager-operator@sha256:8506ce0a578bc18fac117eb2b82799488ffac0bed08287faaf92edaf5d17ab95\\\",\\\"imageID\\\":\\\"quay.io/crcont/openshift-crc-cluster-kube-controller-manager-operator@sha256:8506ce0a578bc18fac117eb2b82799488ffac0bed08287faaf92edaf5d17ab95\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"kube-controller-manager-recovery-controller\\\",\\\"ready\\\":true,\\\"restartCount\\\":0,\\\"started\\\":true,\\\"state\\\":{\\\"running\\\":{\\\"startedAt\\\":\\\"2025-12-03T11:20:15Z\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/etc/kubernetes/static-pod-resources\\\",\\\"name\\\":\\\"resource-dir\\\"},{\\\"mountPath\\\":\\\"/etc/kubernetes/static-pod-certs\\\",\\\"name\\\":\\\"cert-dir\\\"}]}],\\\"hostIP\\\":\\\"192.168.126.11\\\",\\\"hostIPs\\\":[{\\\"ip\\\":\\\"192.168.126.11\\\"}],\\\"phase\\\":\\\"Running\\\",\\\"podIP\\\":\\\"192.168.126.11\\\",\\\"podIPs\\\":[{\\\"ip\\\":\\\"192.168.126.11\\\"}],\\\"startTime\\\":\\\"2025-12-03T11:20:09Z\\\"}}\" for pod \"openshift-kube-controller-manager\"/\"kube-controller-manager-crc\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://127.0.0.1:9743/pod?timeout=10s\": tls: failed to verify certificate: x509: certificate has expired or is not yet valid: current time 2025-12-03T11:20:34Z is after 2025-08-24T17:21:41Z" Dec 03 11:20:34 crc kubenswrapper[5052]: I1203 11:20:34.323839 5052 status_manager.go:875] "Failed to update status for pod" pod="openshift-network-diagnostics/network-check-source-55646444c4-trplf" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"9d751cbb-f2e2-430d-9754-c882a5e924a5\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"message\\\":\\\"containers with unready status: [check-endpoints]\\\",\\\"reason\\\":\\\"ContainersNotReady\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"message\\\":\\\"containers with unready status: [check-endpoints]\\\",\\\"reason\\\":\\\"ContainersNotReady\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"ContainersReady\\\"}],\\\"containerStatuses\\\":[{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e1baa38811c04bd8909e01a1f3be7421a1cb99d608d3dc4cf86d95b17de2ab8b\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{\\\"terminated\\\":{\\\"exitCode\\\":137,\\\"finishedAt\\\":null,\\\"message\\\":\\\"The container could not be located when the pod was deleted. The container used to be Running\\\",\\\"reason\\\":\\\"ContainerStatusUnknown\\\",\\\"startedAt\\\":null}},\\\"name\\\":\\\"check-endpoints\\\",\\\"ready\\\":false,\\\"restartCount\\\":3,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"ContainerCreating\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-s2dwl\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}],\\\"podIP\\\":null,\\\"podIPs\\\":null}}\" for pod \"openshift-network-diagnostics\"/\"network-check-source-55646444c4-trplf\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://127.0.0.1:9743/pod?timeout=10s\": tls: failed to verify certificate: x509: certificate has expired or is not yet valid: current time 2025-12-03T11:20:34Z is after 2025-08-24T17:21:41Z" Dec 03 11:20:34 crc kubenswrapper[5052]: I1203 11:20:34.353501 5052 status_manager.go:875] "Failed to update status for pod" pod="openshift-network-operator/iptables-alerter-4ln5h" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"d75a4c96-2883-4a0b-bab2-0fab2b6c0b49\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"message\\\":\\\"containers with unready status: [iptables-alerter]\\\",\\\"reason\\\":\\\"ContainersNotReady\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"message\\\":\\\"containers with unready status: [iptables-alerter]\\\",\\\"reason\\\":\\\"ContainersNotReady\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"ContainersReady\\\"}],\\\"containerStatuses\\\":[{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:35512335ac39aed0f55b7f799f416f4f6445c20c1b19888cf2bb72bb276703f2\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{\\\"terminated\\\":{\\\"exitCode\\\":137,\\\"finishedAt\\\":null,\\\"message\\\":\\\"The container could not be located when the pod was deleted. The container used to be Running\\\",\\\"reason\\\":\\\"ContainerStatusUnknown\\\",\\\"startedAt\\\":null}},\\\"name\\\":\\\"iptables-alerter\\\",\\\"ready\\\":false,\\\"restartCount\\\":3,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"ContainerCreating\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/iptables-alerter\\\",\\\"name\\\":\\\"iptables-alerter-script\\\"},{\\\"mountPath\\\":\\\"/host\\\",\\\"name\\\":\\\"host-slash\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-rczfb\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}]}}\" for pod \"openshift-network-operator\"/\"iptables-alerter-4ln5h\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://127.0.0.1:9743/pod?timeout=10s\": tls: failed to verify certificate: x509: certificate has expired or is not yet valid: current time 2025-12-03T11:20:34Z is after 2025-08-24T17:21:41Z" Dec 03 11:20:34 crc kubenswrapper[5052]: I1203 11:20:34.397707 5052 status_manager.go:875] "Failed to update status for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kg55h" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"bc569bba-c020-40d6-af68-e617625fe0e4\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:32Z\\\",\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:32Z\\\",\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"Initialized\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"message\\\":\\\"containers with unready status: [ovn-controller ovn-acl-logging kube-rbac-proxy-node kube-rbac-proxy-ovn-metrics northd nbdb sbdb ovnkube-controller]\\\",\\\"reason\\\":\\\"ContainersNotReady\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"message\\\":\\\"containers with unready status: [ovn-controller ovn-acl-logging kube-rbac-proxy-node kube-rbac-proxy-ovn-metrics northd nbdb sbdb ovnkube-controller]\\\",\\\"reason\\\":\\\"ContainersNotReady\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"ContainersReady\\\"}],\\\"containerStatuses\\\":[{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:242b3d66438c42745f4ef318bdeaf3d793426f12962a42ea83e18d06c08aaf09\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"kube-rbac-proxy-node\\\",\\\"ready\\\":false,\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"PodInitializing\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/ovnkube-lib\\\",\\\"name\\\":\\\"ovnkube-script-lib\\\"},{\\\"mountPath\\\":\\\"/etc/pki/tls/metrics-cert\\\",\\\"name\\\":\\\"ovn-node-metrics-cert\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-x92h2\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:242b3d66438c42745f4ef318bdeaf3d793426f12962a42ea83e18d06c08aaf09\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"kube-rbac-proxy-ovn-metrics\\\",\\\"ready\\\":false,\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"PodInitializing\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/ovnkube-lib\\\",\\\"name\\\":\\\"ovnkube-script-lib\\\"},{\\\"mountPath\\\":\\\"/etc/pki/tls/metrics-cert\\\",\\\"name\\\":\\\"ovn-node-metrics-cert\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-x92h2\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:174f36cdd47ef0d1d2099482919d773257453265a2af0b17b154edc32fa41ac2\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"nbdb\\\",\\\"ready\\\":false,\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"PodInitializing\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/ovnkube-lib\\\",\\\"name\\\":\\\"ovnkube-script-lib\\\"},{\\\"mountPath\\\":\\\"/etc/ovn/\\\",\\\"name\\\":\\\"etc-openvswitch\\\"},{\\\"mountPath\\\":\\\"/var/log/ovn\\\",\\\"name\\\":\\\"node-log\\\"},{\\\"mountPath\\\":\\\"/run/ovn/\\\",\\\"name\\\":\\\"run-ovn\\\"},{\\\"mountPath\\\":\\\"/env\\\",\\\"name\\\":\\\"env-overrides\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-x92h2\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:174f36cdd47ef0d1d2099482919d773257453265a2af0b17b154edc32fa41ac2\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"northd\\\",\\\"ready\\\":false,\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"PodInitializing\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/ovnkube-lib\\\",\\\"name\\\":\\\"ovnkube-script-lib\\\"},{\\\"mountPath\\\":\\\"/etc/ovn\\\",\\\"name\\\":\\\"etc-openvswitch\\\"},{\\\"mountPath\\\":\\\"/var/log/ovn\\\",\\\"name\\\":\\\"node-log\\\"},{\\\"mountPath\\\":\\\"/run/ovn/\\\",\\\"name\\\":\\\"run-ovn\\\"},{\\\"mountPath\\\":\\\"/env\\\",\\\"name\\\":\\\"env-overrides\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-x92h2\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:174f36cdd47ef0d1d2099482919d773257453265a2af0b17b154edc32fa41ac2\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"ovn-acl-logging\\\",\\\"ready\\\":false,\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"PodInitializing\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/ovnkube-lib\\\",\\\"name\\\":\\\"ovnkube-script-lib\\\"},{\\\"mountPath\\\":\\\"/var/log/ovn/\\\",\\\"name\\\":\\\"node-log\\\"},{\\\"mountPath\\\":\\\"/run/ovn/\\\",\\\"name\\\":\\\"run-ovn\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-x92h2\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:174f36cdd47ef0d1d2099482919d773257453265a2af0b17b154edc32fa41ac2\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"ovn-controller\\\",\\\"ready\\\":false,\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"PodInitializing\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/ovnkube-lib\\\",\\\"name\\\":\\\"ovnkube-script-lib\\\"},{\\\"mountPath\\\":\\\"/run/openvswitch\\\",\\\"name\\\":\\\"run-openvswitch\\\"},{\\\"mountPath\\\":\\\"/run/ovn/\\\",\\\"name\\\":\\\"run-ovn\\\"},{\\\"mountPath\\\":\\\"/etc/openvswitch\\\",\\\"name\\\":\\\"etc-openvswitch\\\"},{\\\"mountPath\\\":\\\"/etc/ovn/\\\",\\\"name\\\":\\\"etc-openvswitch\\\"},{\\\"mountPath\\\":\\\"/var/lib/openvswitch\\\",\\\"name\\\":\\\"var-lib-openvswitch\\\"},{\\\"mountPath\\\":\\\"/env\\\",\\\"name\\\":\\\"env-overrides\\\"},{\\\"mountPath\\\":\\\"/var/log/ovn/\\\",\\\"name\\\":\\\"node-log\\\"},{\\\"mountPath\\\":\\\"/dev/log\\\",\\\"name\\\":\\\"log-socket\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-x92h2\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:174f36cdd47ef0d1d2099482919d773257453265a2af0b17b154edc32fa41ac2\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"ovnkube-controller\\\",\\\"ready\\\":false,\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"PodInitializing\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/ovnkube-lib\\\",\\\"name\\\":\\\"ovnkube-script-lib\\\"},{\\\"mountPath\\\":\\\"/var/lib/kubelet\\\",\\\"name\\\":\\\"host-kubelet\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/etc/systemd/system\\\",\\\"name\\\":\\\"systemd-units\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/host\\\",\\\"name\\\":\\\"host-slash\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/run/ovn-kubernetes/\\\",\\\"name\\\":\\\"host-run-ovn-kubernetes\\\"},{\\\"mountPath\\\":\\\"/run/netns\\\",\\\"name\\\":\\\"host-run-netns\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/run/systemd/private\\\",\\\"name\\\":\\\"run-systemd\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/cni-bin-dir\\\",\\\"name\\\":\\\"host-cni-bin\\\"},{\\\"mountPath\\\":\\\"/etc/cni/net.d\\\",\\\"name\\\":\\\"host-cni-netd\\\"},{\\\"mountPath\\\":\\\"/var/lib/cni/networks/ovn-k8s-cni-overlay\\\",\\\"name\\\":\\\"host-var-lib-cni-networks-ovn-kubernetes\\\"},{\\\"mountPath\\\":\\\"/run/openvswitch\\\",\\\"name\\\":\\\"run-openvswitch\\\"},{\\\"mountPath\\\":\\\"/var/log/ovnkube/\\\",\\\"name\\\":\\\"etc-openvswitch\\\"},{\\\"mountPath\\\":\\\"/run/ovn/\\\",\\\"name\\\":\\\"run-ovn\\\"},{\\\"mountPath\\\":\\\"/etc/openvswitch\\\",\\\"name\\\":\\\"etc-openvswitch\\\"},{\\\"mountPath\\\":\\\"/etc/ovn/\\\",\\\"name\\\":\\\"etc-openvswitch\\\"},{\\\"mountPath\\\":\\\"/var/lib/openvswitch\\\",\\\"name\\\":\\\"var-lib-openvswitch\\\"},{\\\"mountPath\\\":\\\"/run/ovnkube-config/\\\",\\\"name\\\":\\\"ovnkube-config\\\"},{\\\"mountPath\\\":\\\"/env\\\",\\\"name\\\":\\\"env-overrides\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-x92h2\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:174f36cdd47ef0d1d2099482919d773257453265a2af0b17b154edc32fa41ac2\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"sbdb\\\",\\\"ready\\\":false,\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"PodInitializing\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/ovnkube-lib\\\",\\\"name\\\":\\\"ovnkube-script-lib\\\"},{\\\"mountPath\\\":\\\"/etc/ovn/\\\",\\\"name\\\":\\\"etc-openvswitch\\\"},{\\\"mountPath\\\":\\\"/run/ovn/\\\",\\\"name\\\":\\\"run-ovn\\\"},{\\\"mountPath\\\":\\\"/var/log/ovn\\\",\\\"name\\\":\\\"node-log\\\"},{\\\"mountPath\\\":\\\"/env\\\",\\\"name\\\":\\\"env-overrides\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-x92h2\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}],\\\"hostIP\\\":\\\"192.168.126.11\\\",\\\"hostIPs\\\":[{\\\"ip\\\":\\\"192.168.126.11\\\"}],\\\"initContainerStatuses\\\":[{\\\"containerID\\\":\\\"cri-o://4def7c1172d2ae52e275b2371cf88c9f93c3f0c747c18de7d1073d50b03b19bc\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:174f36cdd47ef0d1d2099482919d773257453265a2af0b17b154edc32fa41ac2\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:174f36cdd47ef0d1d2099482919d773257453265a2af0b17b154edc32fa41ac2\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"kubecfg-setup\\\",\\\"ready\\\":true,\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://4def7c1172d2ae52e275b2371cf88c9f93c3f0c747c18de7d1073d50b03b19bc\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2025-12-03T11:20:32Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2025-12-03T11:20:32Z\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/etc/ovn/\\\",\\\"name\\\":\\\"etc-openvswitch\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-x92h2\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}],\\\"podIP\\\":\\\"192.168.126.11\\\",\\\"podIPs\\\":[{\\\"ip\\\":\\\"192.168.126.11\\\"}],\\\"startTime\\\":\\\"2025-12-03T11:20:31Z\\\"}}\" for pod \"openshift-ovn-kubernetes\"/\"ovnkube-node-kg55h\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://127.0.0.1:9743/pod?timeout=10s\": tls: failed to verify certificate: x509: certificate has expired or is not yet valid: current time 2025-12-03T11:20:34Z is after 2025-08-24T17:21:41Z" Dec 03 11:20:34 crc kubenswrapper[5052]: I1203 11:20:34.672630 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xd92c" Dec 03 11:20:34 crc kubenswrapper[5052]: E1203 11:20:34.673279 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xd92c" podUID="3b6479f0-333b-4a96-9adf-2099afdc2447" Dec 03 11:20:34 crc kubenswrapper[5052]: I1203 11:20:34.872486 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kg55h" event={"ID":"bc569bba-c020-40d6-af68-e617625fe0e4","Type":"ContainerStarted","Data":"c16097e9c17ec2d194aef1c4af913073743ff9b3e54b20999b8b48ce26176ecb"} Dec 03 11:20:34 crc kubenswrapper[5052]: I1203 11:20:34.872564 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kg55h" event={"ID":"bc569bba-c020-40d6-af68-e617625fe0e4","Type":"ContainerStarted","Data":"5b3ab2b483c488f2669acaf42538650e9c65cd6da9b9a5296572ed8f4702a685"} Dec 03 11:20:34 crc kubenswrapper[5052]: I1203 11:20:34.872578 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kg55h" event={"ID":"bc569bba-c020-40d6-af68-e617625fe0e4","Type":"ContainerStarted","Data":"9201f740cc3848eee706f3415f90bbffac06b7df3bb60f3495eff3c1d0a7f3a8"} Dec 03 11:20:34 crc kubenswrapper[5052]: I1203 11:20:34.872589 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kg55h" event={"ID":"bc569bba-c020-40d6-af68-e617625fe0e4","Type":"ContainerStarted","Data":"bde78769f16ee1d0101355c5c8eb4a48437c04ccc42ca5104d1b98538b2545be"} Dec 03 11:20:34 crc kubenswrapper[5052]: I1203 11:20:34.872600 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kg55h" event={"ID":"bc569bba-c020-40d6-af68-e617625fe0e4","Type":"ContainerStarted","Data":"896c25e34b1cb8871fd92cc6c8c908bfce14a9713345f571a672b2156a1765bf"} Dec 03 11:20:34 crc kubenswrapper[5052]: I1203 11:20:34.874678 5052 generic.go:334] "Generic (PLEG): container finished" podID="17265a68-bcb5-455c-b7df-c148a6517db9" containerID="74bfb6b47f5a7bb15524fbcc8ff1ada01ca06c82a90689c3c6221049bb37ab8d" exitCode=0 Dec 03 11:20:34 crc kubenswrapper[5052]: I1203 11:20:34.874805 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2lrql" event={"ID":"17265a68-bcb5-455c-b7df-c148a6517db9","Type":"ContainerDied","Data":"74bfb6b47f5a7bb15524fbcc8ff1ada01ca06c82a90689c3c6221049bb37ab8d"} Dec 03 11:20:34 crc kubenswrapper[5052]: I1203 11:20:34.890395 5052 status_manager.go:875] "Failed to update status for pod" pod="openshift-network-diagnostics/network-check-source-55646444c4-trplf" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"9d751cbb-f2e2-430d-9754-c882a5e924a5\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"message\\\":\\\"containers with unready status: [check-endpoints]\\\",\\\"reason\\\":\\\"ContainersNotReady\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"message\\\":\\\"containers with unready status: [check-endpoints]\\\",\\\"reason\\\":\\\"ContainersNotReady\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"ContainersReady\\\"}],\\\"containerStatuses\\\":[{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e1baa38811c04bd8909e01a1f3be7421a1cb99d608d3dc4cf86d95b17de2ab8b\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{\\\"terminated\\\":{\\\"exitCode\\\":137,\\\"finishedAt\\\":null,\\\"message\\\":\\\"The container could not be located when the pod was deleted. The container used to be Running\\\",\\\"reason\\\":\\\"ContainerStatusUnknown\\\",\\\"startedAt\\\":null}},\\\"name\\\":\\\"check-endpoints\\\",\\\"ready\\\":false,\\\"restartCount\\\":3,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"ContainerCreating\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-s2dwl\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}],\\\"podIP\\\":null,\\\"podIPs\\\":null}}\" for pod \"openshift-network-diagnostics\"/\"network-check-source-55646444c4-trplf\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://127.0.0.1:9743/pod?timeout=10s\": tls: failed to verify certificate: x509: certificate has expired or is not yet valid: current time 2025-12-03T11:20:34Z is after 2025-08-24T17:21:41Z" Dec 03 11:20:34 crc kubenswrapper[5052]: I1203 11:20:34.904956 5052 status_manager.go:875] "Failed to update status for pod" pod="openshift-network-operator/iptables-alerter-4ln5h" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"d75a4c96-2883-4a0b-bab2-0fab2b6c0b49\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"message\\\":\\\"containers with unready status: [iptables-alerter]\\\",\\\"reason\\\":\\\"ContainersNotReady\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"message\\\":\\\"containers with unready status: [iptables-alerter]\\\",\\\"reason\\\":\\\"ContainersNotReady\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"ContainersReady\\\"}],\\\"containerStatuses\\\":[{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:35512335ac39aed0f55b7f799f416f4f6445c20c1b19888cf2bb72bb276703f2\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{\\\"terminated\\\":{\\\"exitCode\\\":137,\\\"finishedAt\\\":null,\\\"message\\\":\\\"The container could not be located when the pod was deleted. The container used to be Running\\\",\\\"reason\\\":\\\"ContainerStatusUnknown\\\",\\\"startedAt\\\":null}},\\\"name\\\":\\\"iptables-alerter\\\",\\\"ready\\\":false,\\\"restartCount\\\":3,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"ContainerCreating\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/iptables-alerter\\\",\\\"name\\\":\\\"iptables-alerter-script\\\"},{\\\"mountPath\\\":\\\"/host\\\",\\\"name\\\":\\\"host-slash\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-rczfb\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}]}}\" for pod \"openshift-network-operator\"/\"iptables-alerter-4ln5h\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://127.0.0.1:9743/pod?timeout=10s\": tls: failed to verify certificate: x509: certificate has expired or is not yet valid: current time 2025-12-03T11:20:34Z is after 2025-08-24T17:21:41Z" Dec 03 11:20:34 crc kubenswrapper[5052]: I1203 11:20:34.923410 5052 status_manager.go:875] "Failed to update status for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kg55h" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"bc569bba-c020-40d6-af68-e617625fe0e4\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:32Z\\\",\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:32Z\\\",\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"Initialized\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"message\\\":\\\"containers with unready status: [ovn-controller ovn-acl-logging kube-rbac-proxy-node kube-rbac-proxy-ovn-metrics northd nbdb sbdb ovnkube-controller]\\\",\\\"reason\\\":\\\"ContainersNotReady\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"message\\\":\\\"containers with unready status: [ovn-controller ovn-acl-logging kube-rbac-proxy-node kube-rbac-proxy-ovn-metrics northd nbdb sbdb ovnkube-controller]\\\",\\\"reason\\\":\\\"ContainersNotReady\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"ContainersReady\\\"}],\\\"containerStatuses\\\":[{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:242b3d66438c42745f4ef318bdeaf3d793426f12962a42ea83e18d06c08aaf09\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"kube-rbac-proxy-node\\\",\\\"ready\\\":false,\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"PodInitializing\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/ovnkube-lib\\\",\\\"name\\\":\\\"ovnkube-script-lib\\\"},{\\\"mountPath\\\":\\\"/etc/pki/tls/metrics-cert\\\",\\\"name\\\":\\\"ovn-node-metrics-cert\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-x92h2\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:242b3d66438c42745f4ef318bdeaf3d793426f12962a42ea83e18d06c08aaf09\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"kube-rbac-proxy-ovn-metrics\\\",\\\"ready\\\":false,\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"PodInitializing\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/ovnkube-lib\\\",\\\"name\\\":\\\"ovnkube-script-lib\\\"},{\\\"mountPath\\\":\\\"/etc/pki/tls/metrics-cert\\\",\\\"name\\\":\\\"ovn-node-metrics-cert\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-x92h2\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:174f36cdd47ef0d1d2099482919d773257453265a2af0b17b154edc32fa41ac2\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"nbdb\\\",\\\"ready\\\":false,\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"PodInitializing\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/ovnkube-lib\\\",\\\"name\\\":\\\"ovnkube-script-lib\\\"},{\\\"mountPath\\\":\\\"/etc/ovn/\\\",\\\"name\\\":\\\"etc-openvswitch\\\"},{\\\"mountPath\\\":\\\"/var/log/ovn\\\",\\\"name\\\":\\\"node-log\\\"},{\\\"mountPath\\\":\\\"/run/ovn/\\\",\\\"name\\\":\\\"run-ovn\\\"},{\\\"mountPath\\\":\\\"/env\\\",\\\"name\\\":\\\"env-overrides\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-x92h2\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:174f36cdd47ef0d1d2099482919d773257453265a2af0b17b154edc32fa41ac2\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"northd\\\",\\\"ready\\\":false,\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"PodInitializing\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/ovnkube-lib\\\",\\\"name\\\":\\\"ovnkube-script-lib\\\"},{\\\"mountPath\\\":\\\"/etc/ovn\\\",\\\"name\\\":\\\"etc-openvswitch\\\"},{\\\"mountPath\\\":\\\"/var/log/ovn\\\",\\\"name\\\":\\\"node-log\\\"},{\\\"mountPath\\\":\\\"/run/ovn/\\\",\\\"name\\\":\\\"run-ovn\\\"},{\\\"mountPath\\\":\\\"/env\\\",\\\"name\\\":\\\"env-overrides\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-x92h2\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:174f36cdd47ef0d1d2099482919d773257453265a2af0b17b154edc32fa41ac2\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"ovn-acl-logging\\\",\\\"ready\\\":false,\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"PodInitializing\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/ovnkube-lib\\\",\\\"name\\\":\\\"ovnkube-script-lib\\\"},{\\\"mountPath\\\":\\\"/var/log/ovn/\\\",\\\"name\\\":\\\"node-log\\\"},{\\\"mountPath\\\":\\\"/run/ovn/\\\",\\\"name\\\":\\\"run-ovn\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-x92h2\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:174f36cdd47ef0d1d2099482919d773257453265a2af0b17b154edc32fa41ac2\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"ovn-controller\\\",\\\"ready\\\":false,\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"PodInitializing\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/ovnkube-lib\\\",\\\"name\\\":\\\"ovnkube-script-lib\\\"},{\\\"mountPath\\\":\\\"/run/openvswitch\\\",\\\"name\\\":\\\"run-openvswitch\\\"},{\\\"mountPath\\\":\\\"/run/ovn/\\\",\\\"name\\\":\\\"run-ovn\\\"},{\\\"mountPath\\\":\\\"/etc/openvswitch\\\",\\\"name\\\":\\\"etc-openvswitch\\\"},{\\\"mountPath\\\":\\\"/etc/ovn/\\\",\\\"name\\\":\\\"etc-openvswitch\\\"},{\\\"mountPath\\\":\\\"/var/lib/openvswitch\\\",\\\"name\\\":\\\"var-lib-openvswitch\\\"},{\\\"mountPath\\\":\\\"/env\\\",\\\"name\\\":\\\"env-overrides\\\"},{\\\"mountPath\\\":\\\"/var/log/ovn/\\\",\\\"name\\\":\\\"node-log\\\"},{\\\"mountPath\\\":\\\"/dev/log\\\",\\\"name\\\":\\\"log-socket\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-x92h2\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:174f36cdd47ef0d1d2099482919d773257453265a2af0b17b154edc32fa41ac2\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"ovnkube-controller\\\",\\\"ready\\\":false,\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"PodInitializing\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/ovnkube-lib\\\",\\\"name\\\":\\\"ovnkube-script-lib\\\"},{\\\"mountPath\\\":\\\"/var/lib/kubelet\\\",\\\"name\\\":\\\"host-kubelet\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/etc/systemd/system\\\",\\\"name\\\":\\\"systemd-units\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/host\\\",\\\"name\\\":\\\"host-slash\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/run/ovn-kubernetes/\\\",\\\"name\\\":\\\"host-run-ovn-kubernetes\\\"},{\\\"mountPath\\\":\\\"/run/netns\\\",\\\"name\\\":\\\"host-run-netns\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/run/systemd/private\\\",\\\"name\\\":\\\"run-systemd\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/cni-bin-dir\\\",\\\"name\\\":\\\"host-cni-bin\\\"},{\\\"mountPath\\\":\\\"/etc/cni/net.d\\\",\\\"name\\\":\\\"host-cni-netd\\\"},{\\\"mountPath\\\":\\\"/var/lib/cni/networks/ovn-k8s-cni-overlay\\\",\\\"name\\\":\\\"host-var-lib-cni-networks-ovn-kubernetes\\\"},{\\\"mountPath\\\":\\\"/run/openvswitch\\\",\\\"name\\\":\\\"run-openvswitch\\\"},{\\\"mountPath\\\":\\\"/var/log/ovnkube/\\\",\\\"name\\\":\\\"etc-openvswitch\\\"},{\\\"mountPath\\\":\\\"/run/ovn/\\\",\\\"name\\\":\\\"run-ovn\\\"},{\\\"mountPath\\\":\\\"/etc/openvswitch\\\",\\\"name\\\":\\\"etc-openvswitch\\\"},{\\\"mountPath\\\":\\\"/etc/ovn/\\\",\\\"name\\\":\\\"etc-openvswitch\\\"},{\\\"mountPath\\\":\\\"/var/lib/openvswitch\\\",\\\"name\\\":\\\"var-lib-openvswitch\\\"},{\\\"mountPath\\\":\\\"/run/ovnkube-config/\\\",\\\"name\\\":\\\"ovnkube-config\\\"},{\\\"mountPath\\\":\\\"/env\\\",\\\"name\\\":\\\"env-overrides\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-x92h2\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]},{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:174f36cdd47ef0d1d2099482919d773257453265a2af0b17b154edc32fa41ac2\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"sbdb\\\",\\\"ready\\\":false,\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"PodInitializing\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/ovnkube-lib\\\",\\\"name\\\":\\\"ovnkube-script-lib\\\"},{\\\"mountPath\\\":\\\"/etc/ovn/\\\",\\\"name\\\":\\\"etc-openvswitch\\\"},{\\\"mountPath\\\":\\\"/run/ovn/\\\",\\\"name\\\":\\\"run-ovn\\\"},{\\\"mountPath\\\":\\\"/var/log/ovn\\\",\\\"name\\\":\\\"node-log\\\"},{\\\"mountPath\\\":\\\"/env\\\",\\\"name\\\":\\\"env-overrides\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-x92h2\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}],\\\"hostIP\\\":\\\"192.168.126.11\\\",\\\"hostIPs\\\":[{\\\"ip\\\":\\\"192.168.126.11\\\"}],\\\"initContainerStatuses\\\":[{\\\"containerID\\\":\\\"cri-o://4def7c1172d2ae52e275b2371cf88c9f93c3f0c747c18de7d1073d50b03b19bc\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:174f36cdd47ef0d1d2099482919d773257453265a2af0b17b154edc32fa41ac2\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:174f36cdd47ef0d1d2099482919d773257453265a2af0b17b154edc32fa41ac2\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"kubecfg-setup\\\",\\\"ready\\\":true,\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://4def7c1172d2ae52e275b2371cf88c9f93c3f0c747c18de7d1073d50b03b19bc\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2025-12-03T11:20:32Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2025-12-03T11:20:32Z\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/etc/ovn/\\\",\\\"name\\\":\\\"etc-openvswitch\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-x92h2\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}],\\\"podIP\\\":\\\"192.168.126.11\\\",\\\"podIPs\\\":[{\\\"ip\\\":\\\"192.168.126.11\\\"}],\\\"startTime\\\":\\\"2025-12-03T11:20:31Z\\\"}}\" for pod \"openshift-ovn-kubernetes\"/\"ovnkube-node-kg55h\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://127.0.0.1:9743/pod?timeout=10s\": tls: failed to verify certificate: x509: certificate has expired or is not yet valid: current time 2025-12-03T11:20:34Z is after 2025-08-24T17:21:41Z" Dec 03 11:20:34 crc kubenswrapper[5052]: I1203 11:20:34.939342 5052 status_manager.go:875] "Failed to update status for pod" pod="openshift-kube-controller-manager/kube-controller-manager-crc" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"f31e1704-9c1d-4044-b318-2aaccb5896e2\\\"},\\\"status\\\":{\\\"conditions\\\":[{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:15Z\\\",\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:09Z\\\",\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"Initialized\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"ContainersReady\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:09Z\\\",\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"PodScheduled\\\"}],\\\"containerStatuses\\\":[{\\\"containerID\\\":\\\"cri-o://37485762ea1808cd3794c9641a4ead224b107e6c64c295de237f8ffd50aba4f4\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c0f9da410c07372b6c9ad6a79379b491cd10fdee88051c026b084652d85aed21\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c0f9da410c07372b6c9ad6a79379b491cd10fdee88051c026b084652d85aed21\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"cluster-policy-controller\\\",\\\"ready\\\":true,\\\"restartCount\\\":0,\\\"started\\\":true,\\\"state\\\":{\\\"running\\\":{\\\"startedAt\\\":\\\"2025-12-03T11:20:12Z\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/etc/kubernetes/static-pod-resources\\\",\\\"name\\\":\\\"resource-dir\\\"},{\\\"mountPath\\\":\\\"/etc/kubernetes/static-pod-certs\\\",\\\"name\\\":\\\"cert-dir\\\"}]},{\\\"containerID\\\":\\\"cri-o://f5d23c6160b5790e9272bd5ae83472267f4d1f60b67e0425a6bc2f5c1f9d1fb7\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:06bc35825771aee1220d34720243b89c4ba8a8b335e6de2597126bd791fd90d4\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:06bc35825771aee1220d34720243b89c4ba8a8b335e6de2597126bd791fd90d4\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"kube-controller-manager\\\",\\\"ready\\\":true,\\\"restartCount\\\":0,\\\"started\\\":true,\\\"state\\\":{\\\"running\\\":{\\\"startedAt\\\":\\\"2025-12-03T11:20:11Z\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/etc/kubernetes/static-pod-resources\\\",\\\"name\\\":\\\"resource-dir\\\"},{\\\"mountPath\\\":\\\"/etc/kubernetes/static-pod-certs\\\",\\\"name\\\":\\\"cert-dir\\\"}]},{\\\"containerID\\\":\\\"cri-o://d05df6945d11d2abe73005efafe80220497a339008d4c28f7923c2dc7ac4a150\\\",\\\"image\\\":\\\"quay.io/crcont/openshift-crc-cluster-kube-controller-manager-operator@sha256:8506ce0a578bc18fac117eb2b82799488ffac0bed08287faaf92edaf5d17ab95\\\",\\\"imageID\\\":\\\"quay.io/crcont/openshift-crc-cluster-kube-controller-manager-operator@sha256:8506ce0a578bc18fac117eb2b82799488ffac0bed08287faaf92edaf5d17ab95\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"kube-controller-manager-cert-syncer\\\",\\\"ready\\\":true,\\\"restartCount\\\":0,\\\"started\\\":true,\\\"state\\\":{\\\"running\\\":{\\\"startedAt\\\":\\\"2025-12-03T11:20:12Z\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/etc/kubernetes/static-pod-resources\\\",\\\"name\\\":\\\"resource-dir\\\"},{\\\"mountPath\\\":\\\"/etc/kubernetes/static-pod-certs\\\",\\\"name\\\":\\\"cert-dir\\\"}]},{\\\"containerID\\\":\\\"cri-o://8df2aa2470046bba37b41cb301e96f52b09b83124384b5dbc4942cd0bfeef1c6\\\",\\\"image\\\":\\\"quay.io/crcont/openshift-crc-cluster-kube-controller-manager-operator@sha256:8506ce0a578bc18fac117eb2b82799488ffac0bed08287faaf92edaf5d17ab95\\\",\\\"imageID\\\":\\\"quay.io/crcont/openshift-crc-cluster-kube-controller-manager-operator@sha256:8506ce0a578bc18fac117eb2b82799488ffac0bed08287faaf92edaf5d17ab95\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"kube-controller-manager-recovery-controller\\\",\\\"ready\\\":true,\\\"restartCount\\\":0,\\\"started\\\":true,\\\"state\\\":{\\\"running\\\":{\\\"startedAt\\\":\\\"2025-12-03T11:20:15Z\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/etc/kubernetes/static-pod-resources\\\",\\\"name\\\":\\\"resource-dir\\\"},{\\\"mountPath\\\":\\\"/etc/kubernetes/static-pod-certs\\\",\\\"name\\\":\\\"cert-dir\\\"}]}],\\\"hostIP\\\":\\\"192.168.126.11\\\",\\\"hostIPs\\\":[{\\\"ip\\\":\\\"192.168.126.11\\\"}],\\\"phase\\\":\\\"Running\\\",\\\"podIP\\\":\\\"192.168.126.11\\\",\\\"podIPs\\\":[{\\\"ip\\\":\\\"192.168.126.11\\\"}],\\\"startTime\\\":\\\"2025-12-03T11:20:09Z\\\"}}\" for pod \"openshift-kube-controller-manager\"/\"kube-controller-manager-crc\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://127.0.0.1:9743/pod?timeout=10s\": tls: failed to verify certificate: x509: certificate has expired or is not yet valid: current time 2025-12-03T11:20:34Z is after 2025-08-24T17:21:41Z" Dec 03 11:20:34 crc kubenswrapper[5052]: I1203 11:20:34.955399 5052 status_manager.go:875] "Failed to update status for pod" pod="openshift-dns/node-resolver-xph69" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"e5c587e6-866a-4427-8a2c-3469cf0fc495\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:32Z\\\",\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:31Z\\\",\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"Initialized\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:32Z\\\",\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2025-12-03T11:20:32Z\\\",\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"ContainersReady\\\"}],\\\"containerStatuses\\\":[{\\\"containerID\\\":\\\"cri-o://8cbacdbdef2832bba669718580c35080702354f16ec06e7d2680a6ee5b63e8b3\\\",\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:35512335ac39aed0f55b7f799f416f4f6445c20c1b19888cf2bb72bb276703f2\\\",\\\"imageID\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:35512335ac39aed0f55b7f799f416f4f6445c20c1b19888cf2bb72bb276703f2\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"dns-node-resolver\\\",\\\"ready\\\":true,\\\"restartCount\\\":0,\\\"started\\\":true,\\\"state\\\":{\\\"running\\\":{\\\"startedAt\\\":\\\"2025-12-03T11:20:32Z\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/etc/hosts\\\",\\\"name\\\":\\\"hosts-file\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-tk7n6\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}],\\\"hostIP\\\":\\\"192.168.126.11\\\",\\\"hostIPs\\\":[{\\\"ip\\\":\\\"192.168.126.11\\\"}],\\\"phase\\\":\\\"Running\\\",\\\"podIP\\\":\\\"192.168.126.11\\\",\\\"podIPs\\\":[{\\\"ip\\\":\\\"192.168.126.11\\\"}],\\\"startTime\\\":\\\"2025-12-03T11:20:31Z\\\"}}\" for pod \"openshift-dns\"/\"node-resolver-xph69\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://127.0.0.1:9743/pod?timeout=10s\": tls: failed to verify certificate: x509: certificate has expired or is not yet valid: current time 2025-12-03T11:20:34Z is after 2025-08-24T17:21:41Z" Dec 03 11:20:35 crc kubenswrapper[5052]: I1203 11:20:35.034545 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-hx657" podStartSLOduration=4.034503914 podStartE2EDuration="4.034503914s" podCreationTimestamp="2025-12-03 11:20:31 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:20:35.001076902 +0000 UTC m=+25.589318686" watchObservedRunningTime="2025-12-03 11:20:35.034503914 +0000 UTC m=+25.622745678" Dec 03 11:20:35 crc kubenswrapper[5052]: I1203 11:20:35.034758 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-etcd/etcd-crc" podStartSLOduration=3.03475368 podStartE2EDuration="3.03475368s" podCreationTimestamp="2025-12-03 11:20:32 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:20:35.033392643 +0000 UTC m=+25.621634427" watchObservedRunningTime="2025-12-03 11:20:35.03475368 +0000 UTC m=+25.622995444" Dec 03 11:20:35 crc kubenswrapper[5052]: I1203 11:20:35.047623 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openshift-image-registry/node-ca-m9wcb"] Dec 03 11:20:35 crc kubenswrapper[5052]: I1203 11:20:35.048219 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-m9wcb" Dec 03 11:20:35 crc kubenswrapper[5052]: I1203 11:20:35.051203 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-image-registry"/"openshift-service-ca.crt" Dec 03 11:20:35 crc kubenswrapper[5052]: I1203 11:20:35.051347 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-image-registry"/"node-ca-dockercfg-4777p" Dec 03 11:20:35 crc kubenswrapper[5052]: I1203 11:20:35.051682 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-image-registry"/"image-registry-certificates" Dec 03 11:20:35 crc kubenswrapper[5052]: I1203 11:20:35.052037 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-image-registry"/"kube-root-ca.crt" Dec 03 11:20:35 crc kubenswrapper[5052]: I1203 11:20:35.053296 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5e64e946-286b-4821-9f0f-e4f545d0b11d-host\") pod \"node-ca-m9wcb\" (UID: \"5e64e946-286b-4821-9f0f-e4f545d0b11d\") " pod="openshift-image-registry/node-ca-m9wcb" Dec 03 11:20:35 crc kubenswrapper[5052]: I1203 11:20:35.053416 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5e64e946-286b-4821-9f0f-e4f545d0b11d-serviceca\") pod \"node-ca-m9wcb\" (UID: \"5e64e946-286b-4821-9f0f-e4f545d0b11d\") " pod="openshift-image-registry/node-ca-m9wcb" Dec 03 11:20:35 crc kubenswrapper[5052]: I1203 11:20:35.053468 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p6xdk\" (UniqueName: \"kubernetes.io/projected/5e64e946-286b-4821-9f0f-e4f545d0b11d-kube-api-access-p6xdk\") pod \"node-ca-m9wcb\" (UID: \"5e64e946-286b-4821-9f0f-e4f545d0b11d\") " pod="openshift-image-registry/node-ca-m9wcb" Dec 03 11:20:35 crc kubenswrapper[5052]: I1203 11:20:35.057629 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-kube-apiserver/kube-apiserver-crc" podStartSLOduration=4.05760059 podStartE2EDuration="4.05760059s" podCreationTimestamp="2025-12-03 11:20:31 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:20:35.05759683 +0000 UTC m=+25.645838594" watchObservedRunningTime="2025-12-03 11:20:35.05760059 +0000 UTC m=+25.645842354" Dec 03 11:20:35 crc kubenswrapper[5052]: I1203 11:20:35.103822 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podStartSLOduration=4.103799183 podStartE2EDuration="4.103799183s" podCreationTimestamp="2025-12-03 11:20:31 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:20:35.086037754 +0000 UTC m=+25.674279518" watchObservedRunningTime="2025-12-03 11:20:35.103799183 +0000 UTC m=+25.692040947" Dec 03 11:20:35 crc kubenswrapper[5052]: I1203 11:20:35.154481 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-p6xdk\" (UniqueName: \"kubernetes.io/projected/5e64e946-286b-4821-9f0f-e4f545d0b11d-kube-api-access-p6xdk\") pod \"node-ca-m9wcb\" (UID: \"5e64e946-286b-4821-9f0f-e4f545d0b11d\") " pod="openshift-image-registry/node-ca-m9wcb" Dec 03 11:20:35 crc kubenswrapper[5052]: I1203 11:20:35.154580 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5e64e946-286b-4821-9f0f-e4f545d0b11d-host\") pod \"node-ca-m9wcb\" (UID: \"5e64e946-286b-4821-9f0f-e4f545d0b11d\") " pod="openshift-image-registry/node-ca-m9wcb" Dec 03 11:20:35 crc kubenswrapper[5052]: I1203 11:20:35.154676 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5e64e946-286b-4821-9f0f-e4f545d0b11d-serviceca\") pod \"node-ca-m9wcb\" (UID: \"5e64e946-286b-4821-9f0f-e4f545d0b11d\") " pod="openshift-image-registry/node-ca-m9wcb" Dec 03 11:20:35 crc kubenswrapper[5052]: I1203 11:20:35.154785 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5e64e946-286b-4821-9f0f-e4f545d0b11d-host\") pod \"node-ca-m9wcb\" (UID: \"5e64e946-286b-4821-9f0f-e4f545d0b11d\") " pod="openshift-image-registry/node-ca-m9wcb" Dec 03 11:20:35 crc kubenswrapper[5052]: I1203 11:20:35.155840 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5e64e946-286b-4821-9f0f-e4f545d0b11d-serviceca\") pod \"node-ca-m9wcb\" (UID: \"5e64e946-286b-4821-9f0f-e4f545d0b11d\") " pod="openshift-image-registry/node-ca-m9wcb" Dec 03 11:20:35 crc kubenswrapper[5052]: I1203 11:20:35.178283 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-kube-controller-manager/kube-controller-manager-crc" podStartSLOduration=4.178246725 podStartE2EDuration="4.178246725s" podCreationTimestamp="2025-12-03 11:20:31 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:20:35.178146282 +0000 UTC m=+25.766388046" watchObservedRunningTime="2025-12-03 11:20:35.178246725 +0000 UTC m=+25.766488489" Dec 03 11:20:35 crc kubenswrapper[5052]: I1203 11:20:35.180462 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-p6xdk\" (UniqueName: \"kubernetes.io/projected/5e64e946-286b-4821-9f0f-e4f545d0b11d-kube-api-access-p6xdk\") pod \"node-ca-m9wcb\" (UID: \"5e64e946-286b-4821-9f0f-e4f545d0b11d\") " pod="openshift-image-registry/node-ca-m9wcb" Dec 03 11:20:35 crc kubenswrapper[5052]: I1203 11:20:35.237567 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/node-resolver-xph69" podStartSLOduration=5.237539099 podStartE2EDuration="5.237539099s" podCreationTimestamp="2025-12-03 11:20:30 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:20:35.236782558 +0000 UTC m=+25.825024332" watchObservedRunningTime="2025-12-03 11:20:35.237539099 +0000 UTC m=+25.825780863" Dec 03 11:20:35 crc kubenswrapper[5052]: I1203 11:20:35.273736 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openshift-ovn-kubernetes/ovnkube-control-plane-749d76644c-hczk6"] Dec 03 11:20:35 crc kubenswrapper[5052]: I1203 11:20:35.274260 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-control-plane-749d76644c-hczk6" Dec 03 11:20:35 crc kubenswrapper[5052]: I1203 11:20:35.276106 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-ovn-kubernetes"/"ovn-control-plane-metrics-cert" Dec 03 11:20:35 crc kubenswrapper[5052]: I1203 11:20:35.276329 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-ovn-kubernetes"/"ovn-kubernetes-control-plane-dockercfg-gs7dd" Dec 03 11:20:35 crc kubenswrapper[5052]: I1203 11:20:35.317151 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openshift-multus/network-metrics-daemon-kf52k"] Dec 03 11:20:35 crc kubenswrapper[5052]: I1203 11:20:35.317629 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kf52k" Dec 03 11:20:35 crc kubenswrapper[5052]: E1203 11:20:35.317692 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kf52k" podUID="a3f2b85d-9b43-4c1c-bc3a-602405b96efc" Dec 03 11:20:35 crc kubenswrapper[5052]: I1203 11:20:35.356591 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\") pod \"8f668bae-612b-4b75-9490-919e737c6a3b\" (UID: \"8f668bae-612b-4b75-9490-919e737c6a3b\") " Dec 03 11:20:35 crc kubenswrapper[5052]: I1203 11:20:35.356692 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a3f2b85d-9b43-4c1c-bc3a-602405b96efc-metrics-certs\") pod \"network-metrics-daemon-kf52k\" (UID: \"a3f2b85d-9b43-4c1c-bc3a-602405b96efc\") " pod="openshift-multus/network-metrics-daemon-kf52k" Dec 03 11:20:35 crc kubenswrapper[5052]: I1203 11:20:35.356730 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-s2dwl\" (UniqueName: \"kubernetes.io/projected/9d751cbb-f2e2-430d-9754-c882a5e924a5-kube-api-access-s2dwl\") pod \"network-check-source-55646444c4-trplf\" (UID: \"9d751cbb-f2e2-430d-9754-c882a5e924a5\") " pod="openshift-network-diagnostics/network-check-source-55646444c4-trplf" Dec 03 11:20:35 crc kubenswrapper[5052]: I1203 11:20:35.356776 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2wsl8\" (UniqueName: \"kubernetes.io/projected/a3f2b85d-9b43-4c1c-bc3a-602405b96efc-kube-api-access-2wsl8\") pod \"network-metrics-daemon-kf52k\" (UID: \"a3f2b85d-9b43-4c1c-bc3a-602405b96efc\") " pod="openshift-multus/network-metrics-daemon-kf52k" Dec 03 11:20:35 crc kubenswrapper[5052]: I1203 11:20:35.356809 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-control-plane-metrics-cert\" (UniqueName: \"kubernetes.io/secret/4f4eb9b9-2cbd-435c-a98b-eb94a919d8ce-ovn-control-plane-metrics-cert\") pod \"ovnkube-control-plane-749d76644c-hczk6\" (UID: \"4f4eb9b9-2cbd-435c-a98b-eb94a919d8ce\") " pod="openshift-ovn-kubernetes/ovnkube-control-plane-749d76644c-hczk6" Dec 03 11:20:35 crc kubenswrapper[5052]: I1203 11:20:35.356832 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8xt8l\" (UniqueName: \"kubernetes.io/projected/4f4eb9b9-2cbd-435c-a98b-eb94a919d8ce-kube-api-access-8xt8l\") pod \"ovnkube-control-plane-749d76644c-hczk6\" (UID: \"4f4eb9b9-2cbd-435c-a98b-eb94a919d8ce\") " pod="openshift-ovn-kubernetes/ovnkube-control-plane-749d76644c-hczk6" Dec 03 11:20:35 crc kubenswrapper[5052]: E1203 11:20:35.356865 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8 podName:8f668bae-612b-4b75-9490-919e737c6a3b nodeName:}" failed. No retries permitted until 2025-12-03 11:20:39.356831477 +0000 UTC m=+29.945073251 (durationBeforeRetry 4s). Error: UnmountVolume.TearDown failed for volume "pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8" (UniqueName: "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8") pod "8f668bae-612b-4b75-9490-919e737c6a3b" (UID: "8f668bae-612b-4b75-9490-919e737c6a3b") : kubernetes.io/csi: Unmounter.TearDownAt failed to get CSI client: driver name kubevirt.io.hostpath-provisioner not found in the list of registered CSI drivers Dec 03 11:20:35 crc kubenswrapper[5052]: E1203 11:20:35.356883 5052 projected.go:288] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Dec 03 11:20:35 crc kubenswrapper[5052]: E1203 11:20:35.356914 5052 projected.go:288] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Dec 03 11:20:35 crc kubenswrapper[5052]: E1203 11:20:35.356930 5052 projected.go:194] Error preparing data for projected volume kube-api-access-s2dwl for pod openshift-network-diagnostics/network-check-source-55646444c4-trplf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Dec 03 11:20:35 crc kubenswrapper[5052]: I1203 11:20:35.356939 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/4f4eb9b9-2cbd-435c-a98b-eb94a919d8ce-ovnkube-config\") pod \"ovnkube-control-plane-749d76644c-hczk6\" (UID: \"4f4eb9b9-2cbd-435c-a98b-eb94a919d8ce\") " pod="openshift-ovn-kubernetes/ovnkube-control-plane-749d76644c-hczk6" Dec 03 11:20:35 crc kubenswrapper[5052]: E1203 11:20:35.356995 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9d751cbb-f2e2-430d-9754-c882a5e924a5-kube-api-access-s2dwl podName:9d751cbb-f2e2-430d-9754-c882a5e924a5 nodeName:}" failed. No retries permitted until 2025-12-03 11:20:39.356975871 +0000 UTC m=+29.945217745 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-s2dwl" (UniqueName: "kubernetes.io/projected/9d751cbb-f2e2-430d-9754-c882a5e924a5-kube-api-access-s2dwl") pod "network-check-source-55646444c4-trplf" (UID: "9d751cbb-f2e2-430d-9754-c882a5e924a5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Dec 03 11:20:35 crc kubenswrapper[5052]: I1203 11:20:35.357021 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/4f4eb9b9-2cbd-435c-a98b-eb94a919d8ce-env-overrides\") pod \"ovnkube-control-plane-749d76644c-hczk6\" (UID: \"4f4eb9b9-2cbd-435c-a98b-eb94a919d8ce\") " pod="openshift-ovn-kubernetes/ovnkube-control-plane-749d76644c-hczk6" Dec 03 11:20:35 crc kubenswrapper[5052]: I1203 11:20:35.357060 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-cqllr\" (UniqueName: \"kubernetes.io/projected/3b6479f0-333b-4a96-9adf-2099afdc2447-kube-api-access-cqllr\") pod \"network-check-target-xd92c\" (UID: \"3b6479f0-333b-4a96-9adf-2099afdc2447\") " pod="openshift-network-diagnostics/network-check-target-xd92c" Dec 03 11:20:35 crc kubenswrapper[5052]: I1203 11:20:35.357089 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/5fe485a1-e14f-4c09-b5b9-f252bc42b7e8-nginx-conf\") pod \"networking-console-plugin-85b44fc459-gdk6g\" (UID: \"5fe485a1-e14f-4c09-b5b9-f252bc42b7e8\") " pod="openshift-network-console/networking-console-plugin-85b44fc459-gdk6g" Dec 03 11:20:35 crc kubenswrapper[5052]: I1203 11:20:35.357119 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/5fe485a1-e14f-4c09-b5b9-f252bc42b7e8-networking-console-plugin-cert\") pod \"networking-console-plugin-85b44fc459-gdk6g\" (UID: \"5fe485a1-e14f-4c09-b5b9-f252bc42b7e8\") " pod="openshift-network-console/networking-console-plugin-85b44fc459-gdk6g" Dec 03 11:20:35 crc kubenswrapper[5052]: E1203 11:20:35.357140 5052 projected.go:288] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Dec 03 11:20:35 crc kubenswrapper[5052]: E1203 11:20:35.357161 5052 projected.go:288] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Dec 03 11:20:35 crc kubenswrapper[5052]: E1203 11:20:35.357171 5052 projected.go:194] Error preparing data for projected volume kube-api-access-cqllr for pod openshift-network-diagnostics/network-check-target-xd92c: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Dec 03 11:20:35 crc kubenswrapper[5052]: E1203 11:20:35.357192 5052 secret.go:188] Couldn't get secret openshift-network-console/networking-console-plugin-cert: object "openshift-network-console"/"networking-console-plugin-cert" not registered Dec 03 11:20:35 crc kubenswrapper[5052]: E1203 11:20:35.357202 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3b6479f0-333b-4a96-9adf-2099afdc2447-kube-api-access-cqllr podName:3b6479f0-333b-4a96-9adf-2099afdc2447 nodeName:}" failed. No retries permitted until 2025-12-03 11:20:39.357191676 +0000 UTC m=+29.945433440 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-cqllr" (UniqueName: "kubernetes.io/projected/3b6479f0-333b-4a96-9adf-2099afdc2447-kube-api-access-cqllr") pod "network-check-target-xd92c" (UID: "3b6479f0-333b-4a96-9adf-2099afdc2447") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Dec 03 11:20:35 crc kubenswrapper[5052]: E1203 11:20:35.357216 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5fe485a1-e14f-4c09-b5b9-f252bc42b7e8-networking-console-plugin-cert podName:5fe485a1-e14f-4c09-b5b9-f252bc42b7e8 nodeName:}" failed. No retries permitted until 2025-12-03 11:20:39.357209717 +0000 UTC m=+29.945451481 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/5fe485a1-e14f-4c09-b5b9-f252bc42b7e8-networking-console-plugin-cert") pod "networking-console-plugin-85b44fc459-gdk6g" (UID: "5fe485a1-e14f-4c09-b5b9-f252bc42b7e8") : object "openshift-network-console"/"networking-console-plugin-cert" not registered Dec 03 11:20:35 crc kubenswrapper[5052]: E1203 11:20:35.357261 5052 configmap.go:193] Couldn't get configMap openshift-network-console/networking-console-plugin: object "openshift-network-console"/"networking-console-plugin" not registered Dec 03 11:20:35 crc kubenswrapper[5052]: E1203 11:20:35.357337 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5fe485a1-e14f-4c09-b5b9-f252bc42b7e8-nginx-conf podName:5fe485a1-e14f-4c09-b5b9-f252bc42b7e8 nodeName:}" failed. No retries permitted until 2025-12-03 11:20:39.35732985 +0000 UTC m=+29.945571614 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "nginx-conf" (UniqueName: "kubernetes.io/configmap/5fe485a1-e14f-4c09-b5b9-f252bc42b7e8-nginx-conf") pod "networking-console-plugin-85b44fc459-gdk6g" (UID: "5fe485a1-e14f-4c09-b5b9-f252bc42b7e8") : object "openshift-network-console"/"networking-console-plugin" not registered Dec 03 11:20:35 crc kubenswrapper[5052]: I1203 11:20:35.367670 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-m9wcb" Dec 03 11:20:35 crc kubenswrapper[5052]: I1203 11:20:35.457726 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a3f2b85d-9b43-4c1c-bc3a-602405b96efc-metrics-certs\") pod \"network-metrics-daemon-kf52k\" (UID: \"a3f2b85d-9b43-4c1c-bc3a-602405b96efc\") " pod="openshift-multus/network-metrics-daemon-kf52k" Dec 03 11:20:35 crc kubenswrapper[5052]: I1203 11:20:35.458162 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-2wsl8\" (UniqueName: \"kubernetes.io/projected/a3f2b85d-9b43-4c1c-bc3a-602405b96efc-kube-api-access-2wsl8\") pod \"network-metrics-daemon-kf52k\" (UID: \"a3f2b85d-9b43-4c1c-bc3a-602405b96efc\") " pod="openshift-multus/network-metrics-daemon-kf52k" Dec 03 11:20:35 crc kubenswrapper[5052]: I1203 11:20:35.458183 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovn-control-plane-metrics-cert\" (UniqueName: \"kubernetes.io/secret/4f4eb9b9-2cbd-435c-a98b-eb94a919d8ce-ovn-control-plane-metrics-cert\") pod \"ovnkube-control-plane-749d76644c-hczk6\" (UID: \"4f4eb9b9-2cbd-435c-a98b-eb94a919d8ce\") " pod="openshift-ovn-kubernetes/ovnkube-control-plane-749d76644c-hczk6" Dec 03 11:20:35 crc kubenswrapper[5052]: I1203 11:20:35.458199 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-8xt8l\" (UniqueName: \"kubernetes.io/projected/4f4eb9b9-2cbd-435c-a98b-eb94a919d8ce-kube-api-access-8xt8l\") pod \"ovnkube-control-plane-749d76644c-hczk6\" (UID: \"4f4eb9b9-2cbd-435c-a98b-eb94a919d8ce\") " pod="openshift-ovn-kubernetes/ovnkube-control-plane-749d76644c-hczk6" Dec 03 11:20:35 crc kubenswrapper[5052]: I1203 11:20:35.458221 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/4f4eb9b9-2cbd-435c-a98b-eb94a919d8ce-ovnkube-config\") pod \"ovnkube-control-plane-749d76644c-hczk6\" (UID: \"4f4eb9b9-2cbd-435c-a98b-eb94a919d8ce\") " pod="openshift-ovn-kubernetes/ovnkube-control-plane-749d76644c-hczk6" Dec 03 11:20:35 crc kubenswrapper[5052]: I1203 11:20:35.458245 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/4f4eb9b9-2cbd-435c-a98b-eb94a919d8ce-env-overrides\") pod \"ovnkube-control-plane-749d76644c-hczk6\" (UID: \"4f4eb9b9-2cbd-435c-a98b-eb94a919d8ce\") " pod="openshift-ovn-kubernetes/ovnkube-control-plane-749d76644c-hczk6" Dec 03 11:20:35 crc kubenswrapper[5052]: E1203 11:20:35.457951 5052 secret.go:188] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Dec 03 11:20:35 crc kubenswrapper[5052]: E1203 11:20:35.458480 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3f2b85d-9b43-4c1c-bc3a-602405b96efc-metrics-certs podName:a3f2b85d-9b43-4c1c-bc3a-602405b96efc nodeName:}" failed. No retries permitted until 2025-12-03 11:20:35.958452407 +0000 UTC m=+26.546694171 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a3f2b85d-9b43-4c1c-bc3a-602405b96efc-metrics-certs") pod "network-metrics-daemon-kf52k" (UID: "a3f2b85d-9b43-4c1c-bc3a-602405b96efc") : object "openshift-multus"/"metrics-daemon-secret" not registered Dec 03 11:20:35 crc kubenswrapper[5052]: I1203 11:20:35.459258 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/4f4eb9b9-2cbd-435c-a98b-eb94a919d8ce-env-overrides\") pod \"ovnkube-control-plane-749d76644c-hczk6\" (UID: \"4f4eb9b9-2cbd-435c-a98b-eb94a919d8ce\") " pod="openshift-ovn-kubernetes/ovnkube-control-plane-749d76644c-hczk6" Dec 03 11:20:35 crc kubenswrapper[5052]: I1203 11:20:35.459388 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/4f4eb9b9-2cbd-435c-a98b-eb94a919d8ce-ovnkube-config\") pod \"ovnkube-control-plane-749d76644c-hczk6\" (UID: \"4f4eb9b9-2cbd-435c-a98b-eb94a919d8ce\") " pod="openshift-ovn-kubernetes/ovnkube-control-plane-749d76644c-hczk6" Dec 03 11:20:35 crc kubenswrapper[5052]: I1203 11:20:35.464253 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovn-control-plane-metrics-cert\" (UniqueName: \"kubernetes.io/secret/4f4eb9b9-2cbd-435c-a98b-eb94a919d8ce-ovn-control-plane-metrics-cert\") pod \"ovnkube-control-plane-749d76644c-hczk6\" (UID: \"4f4eb9b9-2cbd-435c-a98b-eb94a919d8ce\") " pod="openshift-ovn-kubernetes/ovnkube-control-plane-749d76644c-hczk6" Dec 03 11:20:35 crc kubenswrapper[5052]: I1203 11:20:35.476351 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-2wsl8\" (UniqueName: \"kubernetes.io/projected/a3f2b85d-9b43-4c1c-bc3a-602405b96efc-kube-api-access-2wsl8\") pod \"network-metrics-daemon-kf52k\" (UID: \"a3f2b85d-9b43-4c1c-bc3a-602405b96efc\") " pod="openshift-multus/network-metrics-daemon-kf52k" Dec 03 11:20:35 crc kubenswrapper[5052]: I1203 11:20:35.479873 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-8xt8l\" (UniqueName: \"kubernetes.io/projected/4f4eb9b9-2cbd-435c-a98b-eb94a919d8ce-kube-api-access-8xt8l\") pod \"ovnkube-control-plane-749d76644c-hczk6\" (UID: \"4f4eb9b9-2cbd-435c-a98b-eb94a919d8ce\") " pod="openshift-ovn-kubernetes/ovnkube-control-plane-749d76644c-hczk6" Dec 03 11:20:35 crc kubenswrapper[5052]: I1203 11:20:35.587940 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-control-plane-749d76644c-hczk6" Dec 03 11:20:35 crc kubenswrapper[5052]: W1203 11:20:35.601186 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f4eb9b9_2cbd_435c_a98b_eb94a919d8ce.slice/crio-1c86cbf515df3d10f61648a1fb6f4313215a442c51018dc82566d06358757fa1 WatchSource:0}: Error finding container 1c86cbf515df3d10f61648a1fb6f4313215a442c51018dc82566d06358757fa1: Status 404 returned error can't find the container with id 1c86cbf515df3d10f61648a1fb6f4313215a442c51018dc82566d06358757fa1 Dec 03 11:20:35 crc kubenswrapper[5052]: I1203 11:20:35.672279 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-85b44fc459-gdk6g" Dec 03 11:20:35 crc kubenswrapper[5052]: I1203 11:20:35.672353 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-55646444c4-trplf" Dec 03 11:20:35 crc kubenswrapper[5052]: E1203 11:20:35.672460 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-console/networking-console-plugin-85b44fc459-gdk6g" podUID="5fe485a1-e14f-4c09-b5b9-f252bc42b7e8" Dec 03 11:20:35 crc kubenswrapper[5052]: E1203 11:20:35.672652 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-source-55646444c4-trplf" podUID="9d751cbb-f2e2-430d-9754-c882a5e924a5" Dec 03 11:20:35 crc kubenswrapper[5052]: I1203 11:20:35.880199 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-control-plane-749d76644c-hczk6" event={"ID":"4f4eb9b9-2cbd-435c-a98b-eb94a919d8ce","Type":"ContainerStarted","Data":"1c86cbf515df3d10f61648a1fb6f4313215a442c51018dc82566d06358757fa1"} Dec 03 11:20:35 crc kubenswrapper[5052]: I1203 11:20:35.882233 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-m9wcb" event={"ID":"5e64e946-286b-4821-9f0f-e4f545d0b11d","Type":"ContainerStarted","Data":"c6beac6155fe7599695c687c2a94de287d8da6030b2b33d9ccae2808c82f91e7"} Dec 03 11:20:35 crc kubenswrapper[5052]: I1203 11:20:35.882289 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-m9wcb" event={"ID":"5e64e946-286b-4821-9f0f-e4f545d0b11d","Type":"ContainerStarted","Data":"fbc97e5bfd02de0b81eef4a5cdec7aad274a08f71f3c74ef9e50a5cd2832844c"} Dec 03 11:20:35 crc kubenswrapper[5052]: I1203 11:20:35.883903 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-4ln5h" event={"ID":"d75a4c96-2883-4a0b-bab2-0fab2b6c0b49","Type":"ContainerStarted","Data":"cdfbfbc1aac14de4705d3e9e2cc479c23d0bf1041eaa3f33a69d26cefe6f3206"} Dec 03 11:20:35 crc kubenswrapper[5052]: I1203 11:20:35.887472 5052 generic.go:334] "Generic (PLEG): container finished" podID="17265a68-bcb5-455c-b7df-c148a6517db9" containerID="d632802002441ca913dbc0776ef1b6b79edb13cae94f47faa9b315d3875eb5c1" exitCode=0 Dec 03 11:20:35 crc kubenswrapper[5052]: I1203 11:20:35.887532 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2lrql" event={"ID":"17265a68-bcb5-455c-b7df-c148a6517db9","Type":"ContainerDied","Data":"d632802002441ca913dbc0776ef1b6b79edb13cae94f47faa9b315d3875eb5c1"} Dec 03 11:20:35 crc kubenswrapper[5052]: I1203 11:20:35.895771 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/node-ca-m9wcb" podStartSLOduration=4.895751888 podStartE2EDuration="4.895751888s" podCreationTimestamp="2025-12-03 11:20:31 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:20:35.894910435 +0000 UTC m=+26.483152209" watchObservedRunningTime="2025-12-03 11:20:35.895751888 +0000 UTC m=+26.483993652" Dec 03 11:20:35 crc kubenswrapper[5052]: I1203 11:20:35.964929 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a3f2b85d-9b43-4c1c-bc3a-602405b96efc-metrics-certs\") pod \"network-metrics-daemon-kf52k\" (UID: \"a3f2b85d-9b43-4c1c-bc3a-602405b96efc\") " pod="openshift-multus/network-metrics-daemon-kf52k" Dec 03 11:20:35 crc kubenswrapper[5052]: E1203 11:20:35.965117 5052 secret.go:188] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Dec 03 11:20:35 crc kubenswrapper[5052]: E1203 11:20:35.965305 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3f2b85d-9b43-4c1c-bc3a-602405b96efc-metrics-certs podName:a3f2b85d-9b43-4c1c-bc3a-602405b96efc nodeName:}" failed. No retries permitted until 2025-12-03 11:20:36.965284645 +0000 UTC m=+27.553526409 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a3f2b85d-9b43-4c1c-bc3a-602405b96efc-metrics-certs") pod "network-metrics-daemon-kf52k" (UID: "a3f2b85d-9b43-4c1c-bc3a-602405b96efc") : object "openshift-multus"/"metrics-daemon-secret" not registered Dec 03 11:20:36 crc kubenswrapper[5052]: I1203 11:20:36.672110 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kf52k" Dec 03 11:20:36 crc kubenswrapper[5052]: I1203 11:20:36.672251 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xd92c" Dec 03 11:20:36 crc kubenswrapper[5052]: E1203 11:20:36.672385 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kf52k" podUID="a3f2b85d-9b43-4c1c-bc3a-602405b96efc" Dec 03 11:20:36 crc kubenswrapper[5052]: E1203 11:20:36.672586 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xd92c" podUID="3b6479f0-333b-4a96-9adf-2099afdc2447" Dec 03 11:20:36 crc kubenswrapper[5052]: I1203 11:20:36.895072 5052 generic.go:334] "Generic (PLEG): container finished" podID="17265a68-bcb5-455c-b7df-c148a6517db9" containerID="130fb6147f9ba2f55ec17d8150581d9aac78ad1e84bac5d0d86e22e52d5fb71a" exitCode=0 Dec 03 11:20:36 crc kubenswrapper[5052]: I1203 11:20:36.895162 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2lrql" event={"ID":"17265a68-bcb5-455c-b7df-c148a6517db9","Type":"ContainerDied","Data":"130fb6147f9ba2f55ec17d8150581d9aac78ad1e84bac5d0d86e22e52d5fb71a"} Dec 03 11:20:36 crc kubenswrapper[5052]: I1203 11:20:36.897862 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-control-plane-749d76644c-hczk6" event={"ID":"4f4eb9b9-2cbd-435c-a98b-eb94a919d8ce","Type":"ContainerStarted","Data":"51a69af47455dba0cbcc1a758e833d78c9bc385c5f80c5329c4a701899b96389"} Dec 03 11:20:36 crc kubenswrapper[5052]: I1203 11:20:36.897916 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-control-plane-749d76644c-hczk6" event={"ID":"4f4eb9b9-2cbd-435c-a98b-eb94a919d8ce","Type":"ContainerStarted","Data":"c17c78ebc70562275e8c5b67d607868fc54a1345b2042d230cd7ab15527a24e7"} Dec 03 11:20:36 crc kubenswrapper[5052]: I1203 11:20:36.940729 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-control-plane-749d76644c-hczk6" podStartSLOduration=4.940704267 podStartE2EDuration="4.940704267s" podCreationTimestamp="2025-12-03 11:20:32 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:20:36.939907465 +0000 UTC m=+27.528149229" watchObservedRunningTime="2025-12-03 11:20:36.940704267 +0000 UTC m=+27.528946031" Dec 03 11:20:36 crc kubenswrapper[5052]: I1203 11:20:36.974724 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a3f2b85d-9b43-4c1c-bc3a-602405b96efc-metrics-certs\") pod \"network-metrics-daemon-kf52k\" (UID: \"a3f2b85d-9b43-4c1c-bc3a-602405b96efc\") " pod="openshift-multus/network-metrics-daemon-kf52k" Dec 03 11:20:36 crc kubenswrapper[5052]: E1203 11:20:36.976136 5052 secret.go:188] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Dec 03 11:20:36 crc kubenswrapper[5052]: E1203 11:20:36.976232 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3f2b85d-9b43-4c1c-bc3a-602405b96efc-metrics-certs podName:a3f2b85d-9b43-4c1c-bc3a-602405b96efc nodeName:}" failed. No retries permitted until 2025-12-03 11:20:38.976210286 +0000 UTC m=+29.564452050 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a3f2b85d-9b43-4c1c-bc3a-602405b96efc-metrics-certs") pod "network-metrics-daemon-kf52k" (UID: "a3f2b85d-9b43-4c1c-bc3a-602405b96efc") : object "openshift-multus"/"metrics-daemon-secret" not registered Dec 03 11:20:37 crc kubenswrapper[5052]: I1203 11:20:37.344479 5052 kubelet_node_status.go:401] "Setting node annotation to enable volume controller attach/detach" Dec 03 11:20:37 crc kubenswrapper[5052]: I1203 11:20:37.347133 5052 kubelet_node_status.go:724] "Recording event message for node" node="crc" event="NodeHasSufficientMemory" Dec 03 11:20:37 crc kubenswrapper[5052]: I1203 11:20:37.347186 5052 kubelet_node_status.go:724] "Recording event message for node" node="crc" event="NodeHasNoDiskPressure" Dec 03 11:20:37 crc kubenswrapper[5052]: I1203 11:20:37.347195 5052 kubelet_node_status.go:724] "Recording event message for node" node="crc" event="NodeHasSufficientPID" Dec 03 11:20:37 crc kubenswrapper[5052]: I1203 11:20:37.347340 5052 kubelet_node_status.go:76] "Attempting to register node" node="crc" Dec 03 11:20:37 crc kubenswrapper[5052]: I1203 11:20:37.353017 5052 kubelet_node_status.go:115] "Node was previously registered" node="crc" Dec 03 11:20:37 crc kubenswrapper[5052]: I1203 11:20:37.353256 5052 kubelet_node_status.go:79] "Successfully registered node" node="crc" Dec 03 11:20:37 crc kubenswrapper[5052]: I1203 11:20:37.354247 5052 kubelet_node_status.go:724] "Recording event message for node" node="crc" event="NodeHasSufficientMemory" Dec 03 11:20:37 crc kubenswrapper[5052]: I1203 11:20:37.354279 5052 kubelet_node_status.go:724] "Recording event message for node" node="crc" event="NodeHasNoDiskPressure" Dec 03 11:20:37 crc kubenswrapper[5052]: I1203 11:20:37.354289 5052 kubelet_node_status.go:724] "Recording event message for node" node="crc" event="NodeHasSufficientPID" Dec 03 11:20:37 crc kubenswrapper[5052]: I1203 11:20:37.354303 5052 kubelet_node_status.go:724] "Recording event message for node" node="crc" event="NodeNotReady" Dec 03 11:20:37 crc kubenswrapper[5052]: I1203 11:20:37.354315 5052 setters.go:603] "Node became not ready" node="crc" condition={"type":"Ready","status":"False","lastHeartbeatTime":"2025-12-03T11:20:37Z","lastTransitionTime":"2025-12-03T11:20:37Z","reason":"KubeletNotReady","message":"container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?"} Dec 03 11:20:37 crc kubenswrapper[5052]: I1203 11:20:37.376105 5052 kubelet_node_status.go:724] "Recording event message for node" node="crc" event="NodeHasSufficientMemory" Dec 03 11:20:37 crc kubenswrapper[5052]: I1203 11:20:37.376465 5052 kubelet_node_status.go:724] "Recording event message for node" node="crc" event="NodeHasNoDiskPressure" Dec 03 11:20:37 crc kubenswrapper[5052]: I1203 11:20:37.376586 5052 kubelet_node_status.go:724] "Recording event message for node" node="crc" event="NodeHasSufficientPID" Dec 03 11:20:37 crc kubenswrapper[5052]: I1203 11:20:37.376677 5052 kubelet_node_status.go:724] "Recording event message for node" node="crc" event="NodeNotReady" Dec 03 11:20:37 crc kubenswrapper[5052]: I1203 11:20:37.376820 5052 setters.go:603] "Node became not ready" node="crc" condition={"type":"Ready","status":"False","lastHeartbeatTime":"2025-12-03T11:20:37Z","lastTransitionTime":"2025-12-03T11:20:37Z","reason":"KubeletNotReady","message":"container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?"} Dec 03 11:20:37 crc kubenswrapper[5052]: I1203 11:20:37.404533 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openshift-cluster-version/cluster-version-operator-5c965bbfc6-8glnw"] Dec 03 11:20:37 crc kubenswrapper[5052]: I1203 11:20:37.405795 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-version/cluster-version-operator-5c965bbfc6-8glnw" Dec 03 11:20:37 crc kubenswrapper[5052]: I1203 11:20:37.407598 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-cluster-version"/"default-dockercfg-gxtc4" Dec 03 11:20:37 crc kubenswrapper[5052]: I1203 11:20:37.407909 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-cluster-version"/"openshift-service-ca.crt" Dec 03 11:20:37 crc kubenswrapper[5052]: I1203 11:20:37.408907 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-cluster-version"/"kube-root-ca.crt" Dec 03 11:20:37 crc kubenswrapper[5052]: I1203 11:20:37.409527 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-cluster-version"/"cluster-version-operator-serving-cert" Dec 03 11:20:37 crc kubenswrapper[5052]: I1203 11:20:37.479970 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e81d7ac0-e117-4ddb-88fb-fde5b3947080-serving-cert\") pod \"cluster-version-operator-5c965bbfc6-8glnw\" (UID: \"e81d7ac0-e117-4ddb-88fb-fde5b3947080\") " pod="openshift-cluster-version/cluster-version-operator-5c965bbfc6-8glnw" Dec 03 11:20:37 crc kubenswrapper[5052]: I1203 11:20:37.480036 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-ssl-certs\" (UniqueName: \"kubernetes.io/host-path/e81d7ac0-e117-4ddb-88fb-fde5b3947080-etc-ssl-certs\") pod \"cluster-version-operator-5c965bbfc6-8glnw\" (UID: \"e81d7ac0-e117-4ddb-88fb-fde5b3947080\") " pod="openshift-cluster-version/cluster-version-operator-5c965bbfc6-8glnw" Dec 03 11:20:37 crc kubenswrapper[5052]: I1203 11:20:37.480054 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-cvo-updatepayloads\" (UniqueName: \"kubernetes.io/host-path/e81d7ac0-e117-4ddb-88fb-fde5b3947080-etc-cvo-updatepayloads\") pod \"cluster-version-operator-5c965bbfc6-8glnw\" (UID: \"e81d7ac0-e117-4ddb-88fb-fde5b3947080\") " pod="openshift-cluster-version/cluster-version-operator-5c965bbfc6-8glnw" Dec 03 11:20:37 crc kubenswrapper[5052]: I1203 11:20:37.480179 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access\" (UniqueName: \"kubernetes.io/projected/e81d7ac0-e117-4ddb-88fb-fde5b3947080-kube-api-access\") pod \"cluster-version-operator-5c965bbfc6-8glnw\" (UID: \"e81d7ac0-e117-4ddb-88fb-fde5b3947080\") " pod="openshift-cluster-version/cluster-version-operator-5c965bbfc6-8glnw" Dec 03 11:20:37 crc kubenswrapper[5052]: I1203 11:20:37.480234 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e81d7ac0-e117-4ddb-88fb-fde5b3947080-service-ca\") pod \"cluster-version-operator-5c965bbfc6-8glnw\" (UID: \"e81d7ac0-e117-4ddb-88fb-fde5b3947080\") " pod="openshift-cluster-version/cluster-version-operator-5c965bbfc6-8glnw" Dec 03 11:20:37 crc kubenswrapper[5052]: I1203 11:20:37.581450 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e81d7ac0-e117-4ddb-88fb-fde5b3947080-serving-cert\") pod \"cluster-version-operator-5c965bbfc6-8glnw\" (UID: \"e81d7ac0-e117-4ddb-88fb-fde5b3947080\") " pod="openshift-cluster-version/cluster-version-operator-5c965bbfc6-8glnw" Dec 03 11:20:37 crc kubenswrapper[5052]: I1203 11:20:37.581592 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-ssl-certs\" (UniqueName: \"kubernetes.io/host-path/e81d7ac0-e117-4ddb-88fb-fde5b3947080-etc-ssl-certs\") pod \"cluster-version-operator-5c965bbfc6-8glnw\" (UID: \"e81d7ac0-e117-4ddb-88fb-fde5b3947080\") " pod="openshift-cluster-version/cluster-version-operator-5c965bbfc6-8glnw" Dec 03 11:20:37 crc kubenswrapper[5052]: I1203 11:20:37.581612 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-cvo-updatepayloads\" (UniqueName: \"kubernetes.io/host-path/e81d7ac0-e117-4ddb-88fb-fde5b3947080-etc-cvo-updatepayloads\") pod \"cluster-version-operator-5c965bbfc6-8glnw\" (UID: \"e81d7ac0-e117-4ddb-88fb-fde5b3947080\") " pod="openshift-cluster-version/cluster-version-operator-5c965bbfc6-8glnw" Dec 03 11:20:37 crc kubenswrapper[5052]: I1203 11:20:37.581645 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access\" (UniqueName: \"kubernetes.io/projected/e81d7ac0-e117-4ddb-88fb-fde5b3947080-kube-api-access\") pod \"cluster-version-operator-5c965bbfc6-8glnw\" (UID: \"e81d7ac0-e117-4ddb-88fb-fde5b3947080\") " pod="openshift-cluster-version/cluster-version-operator-5c965bbfc6-8glnw" Dec 03 11:20:37 crc kubenswrapper[5052]: I1203 11:20:37.581722 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"etc-ssl-certs\" (UniqueName: \"kubernetes.io/host-path/e81d7ac0-e117-4ddb-88fb-fde5b3947080-etc-ssl-certs\") pod \"cluster-version-operator-5c965bbfc6-8glnw\" (UID: \"e81d7ac0-e117-4ddb-88fb-fde5b3947080\") " pod="openshift-cluster-version/cluster-version-operator-5c965bbfc6-8glnw" Dec 03 11:20:37 crc kubenswrapper[5052]: I1203 11:20:37.581708 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e81d7ac0-e117-4ddb-88fb-fde5b3947080-service-ca\") pod \"cluster-version-operator-5c965bbfc6-8glnw\" (UID: \"e81d7ac0-e117-4ddb-88fb-fde5b3947080\") " pod="openshift-cluster-version/cluster-version-operator-5c965bbfc6-8glnw" Dec 03 11:20:37 crc kubenswrapper[5052]: I1203 11:20:37.581866 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"etc-cvo-updatepayloads\" (UniqueName: \"kubernetes.io/host-path/e81d7ac0-e117-4ddb-88fb-fde5b3947080-etc-cvo-updatepayloads\") pod \"cluster-version-operator-5c965bbfc6-8glnw\" (UID: \"e81d7ac0-e117-4ddb-88fb-fde5b3947080\") " pod="openshift-cluster-version/cluster-version-operator-5c965bbfc6-8glnw" Dec 03 11:20:37 crc kubenswrapper[5052]: I1203 11:20:37.582814 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e81d7ac0-e117-4ddb-88fb-fde5b3947080-service-ca\") pod \"cluster-version-operator-5c965bbfc6-8glnw\" (UID: \"e81d7ac0-e117-4ddb-88fb-fde5b3947080\") " pod="openshift-cluster-version/cluster-version-operator-5c965bbfc6-8glnw" Dec 03 11:20:37 crc kubenswrapper[5052]: I1203 11:20:37.586688 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e81d7ac0-e117-4ddb-88fb-fde5b3947080-serving-cert\") pod \"cluster-version-operator-5c965bbfc6-8glnw\" (UID: \"e81d7ac0-e117-4ddb-88fb-fde5b3947080\") " pod="openshift-cluster-version/cluster-version-operator-5c965bbfc6-8glnw" Dec 03 11:20:37 crc kubenswrapper[5052]: I1203 11:20:37.601754 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access\" (UniqueName: \"kubernetes.io/projected/e81d7ac0-e117-4ddb-88fb-fde5b3947080-kube-api-access\") pod \"cluster-version-operator-5c965bbfc6-8glnw\" (UID: \"e81d7ac0-e117-4ddb-88fb-fde5b3947080\") " pod="openshift-cluster-version/cluster-version-operator-5c965bbfc6-8glnw" Dec 03 11:20:37 crc kubenswrapper[5052]: I1203 11:20:37.672741 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-55646444c4-trplf" Dec 03 11:20:37 crc kubenswrapper[5052]: I1203 11:20:37.672784 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-85b44fc459-gdk6g" Dec 03 11:20:37 crc kubenswrapper[5052]: E1203 11:20:37.672914 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-source-55646444c4-trplf" podUID="9d751cbb-f2e2-430d-9754-c882a5e924a5" Dec 03 11:20:37 crc kubenswrapper[5052]: E1203 11:20:37.673081 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-console/networking-console-plugin-85b44fc459-gdk6g" podUID="5fe485a1-e14f-4c09-b5b9-f252bc42b7e8" Dec 03 11:20:37 crc kubenswrapper[5052]: I1203 11:20:37.720209 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-version/cluster-version-operator-5c965bbfc6-8glnw" Dec 03 11:20:37 crc kubenswrapper[5052]: W1203 11:20:37.734145 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode81d7ac0_e117_4ddb_88fb_fde5b3947080.slice/crio-63ad865c4c3c580af5ac978407ecbb1fce24e9777d56589ae3826907835a018c WatchSource:0}: Error finding container 63ad865c4c3c580af5ac978407ecbb1fce24e9777d56589ae3826907835a018c: Status 404 returned error can't find the container with id 63ad865c4c3c580af5ac978407ecbb1fce24e9777d56589ae3826907835a018c Dec 03 11:20:37 crc kubenswrapper[5052]: I1203 11:20:37.902694 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-version/cluster-version-operator-5c965bbfc6-8glnw" event={"ID":"e81d7ac0-e117-4ddb-88fb-fde5b3947080","Type":"ContainerStarted","Data":"8f644c1b527c222babaa56515af482734157305f80f901a138a67ee8b037ce00"} Dec 03 11:20:37 crc kubenswrapper[5052]: I1203 11:20:37.902762 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-version/cluster-version-operator-5c965bbfc6-8glnw" event={"ID":"e81d7ac0-e117-4ddb-88fb-fde5b3947080","Type":"ContainerStarted","Data":"63ad865c4c3c580af5ac978407ecbb1fce24e9777d56589ae3826907835a018c"} Dec 03 11:20:37 crc kubenswrapper[5052]: I1203 11:20:37.906125 5052 generic.go:334] "Generic (PLEG): container finished" podID="17265a68-bcb5-455c-b7df-c148a6517db9" containerID="b451824d762d9a88d49c0acafc0ea9215f1b256d5188cb5388443a8e0b2f92bd" exitCode=0 Dec 03 11:20:37 crc kubenswrapper[5052]: I1203 11:20:37.906183 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2lrql" event={"ID":"17265a68-bcb5-455c-b7df-c148a6517db9","Type":"ContainerDied","Data":"b451824d762d9a88d49c0acafc0ea9215f1b256d5188cb5388443a8e0b2f92bd"} Dec 03 11:20:37 crc kubenswrapper[5052]: I1203 11:20:37.914558 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kg55h" event={"ID":"bc569bba-c020-40d6-af68-e617625fe0e4","Type":"ContainerStarted","Data":"affc8d8f111572fbded3dbcc4c9f374ca5ba664cd6989bb1e717c163a1d88417"} Dec 03 11:20:38 crc kubenswrapper[5052]: I1203 11:20:38.672594 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xd92c" Dec 03 11:20:38 crc kubenswrapper[5052]: E1203 11:20:38.673263 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xd92c" podUID="3b6479f0-333b-4a96-9adf-2099afdc2447" Dec 03 11:20:38 crc kubenswrapper[5052]: I1203 11:20:38.672718 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kf52k" Dec 03 11:20:38 crc kubenswrapper[5052]: E1203 11:20:38.673684 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kf52k" podUID="a3f2b85d-9b43-4c1c-bc3a-602405b96efc" Dec 03 11:20:38 crc kubenswrapper[5052]: I1203 11:20:38.925074 5052 generic.go:334] "Generic (PLEG): container finished" podID="17265a68-bcb5-455c-b7df-c148a6517db9" containerID="915ffb06684530d1d2625c7551b3ad7f88c7ce7aae2d1646f19902655be34a6d" exitCode=0 Dec 03 11:20:38 crc kubenswrapper[5052]: I1203 11:20:38.925164 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2lrql" event={"ID":"17265a68-bcb5-455c-b7df-c148a6517db9","Type":"ContainerDied","Data":"915ffb06684530d1d2625c7551b3ad7f88c7ce7aae2d1646f19902655be34a6d"} Dec 03 11:20:38 crc kubenswrapper[5052]: I1203 11:20:38.943253 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-version/cluster-version-operator-5c965bbfc6-8glnw" podStartSLOduration=7.943229964 podStartE2EDuration="7.943229964s" podCreationTimestamp="2025-12-03 11:20:31 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:20:38.942973597 +0000 UTC m=+29.531215361" watchObservedRunningTime="2025-12-03 11:20:38.943229964 +0000 UTC m=+29.531471728" Dec 03 11:20:39 crc kubenswrapper[5052]: I1203 11:20:39.030260 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a3f2b85d-9b43-4c1c-bc3a-602405b96efc-metrics-certs\") pod \"network-metrics-daemon-kf52k\" (UID: \"a3f2b85d-9b43-4c1c-bc3a-602405b96efc\") " pod="openshift-multus/network-metrics-daemon-kf52k" Dec 03 11:20:39 crc kubenswrapper[5052]: E1203 11:20:39.030949 5052 secret.go:188] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Dec 03 11:20:39 crc kubenswrapper[5052]: E1203 11:20:39.031141 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3f2b85d-9b43-4c1c-bc3a-602405b96efc-metrics-certs podName:a3f2b85d-9b43-4c1c-bc3a-602405b96efc nodeName:}" failed. No retries permitted until 2025-12-03 11:20:43.031101385 +0000 UTC m=+33.619343329 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a3f2b85d-9b43-4c1c-bc3a-602405b96efc-metrics-certs") pod "network-metrics-daemon-kf52k" (UID: "a3f2b85d-9b43-4c1c-bc3a-602405b96efc") : object "openshift-multus"/"metrics-daemon-secret" not registered Dec 03 11:20:39 crc kubenswrapper[5052]: I1203 11:20:39.433551 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\") pod \"8f668bae-612b-4b75-9490-919e737c6a3b\" (UID: \"8f668bae-612b-4b75-9490-919e737c6a3b\") " Dec 03 11:20:39 crc kubenswrapper[5052]: I1203 11:20:39.433750 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/5fe485a1-e14f-4c09-b5b9-f252bc42b7e8-nginx-conf\") pod \"networking-console-plugin-85b44fc459-gdk6g\" (UID: \"5fe485a1-e14f-4c09-b5b9-f252bc42b7e8\") " pod="openshift-network-console/networking-console-plugin-85b44fc459-gdk6g" Dec 03 11:20:39 crc kubenswrapper[5052]: I1203 11:20:39.433784 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-cqllr\" (UniqueName: \"kubernetes.io/projected/3b6479f0-333b-4a96-9adf-2099afdc2447-kube-api-access-cqllr\") pod \"network-check-target-xd92c\" (UID: \"3b6479f0-333b-4a96-9adf-2099afdc2447\") " pod="openshift-network-diagnostics/network-check-target-xd92c" Dec 03 11:20:39 crc kubenswrapper[5052]: I1203 11:20:39.433819 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/5fe485a1-e14f-4c09-b5b9-f252bc42b7e8-networking-console-plugin-cert\") pod \"networking-console-plugin-85b44fc459-gdk6g\" (UID: \"5fe485a1-e14f-4c09-b5b9-f252bc42b7e8\") " pod="openshift-network-console/networking-console-plugin-85b44fc459-gdk6g" Dec 03 11:20:39 crc kubenswrapper[5052]: I1203 11:20:39.433856 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-s2dwl\" (UniqueName: \"kubernetes.io/projected/9d751cbb-f2e2-430d-9754-c882a5e924a5-kube-api-access-s2dwl\") pod \"network-check-source-55646444c4-trplf\" (UID: \"9d751cbb-f2e2-430d-9754-c882a5e924a5\") " pod="openshift-network-diagnostics/network-check-source-55646444c4-trplf" Dec 03 11:20:39 crc kubenswrapper[5052]: E1203 11:20:39.434012 5052 projected.go:288] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Dec 03 11:20:39 crc kubenswrapper[5052]: E1203 11:20:39.434035 5052 projected.go:288] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Dec 03 11:20:39 crc kubenswrapper[5052]: E1203 11:20:39.434049 5052 projected.go:194] Error preparing data for projected volume kube-api-access-s2dwl for pod openshift-network-diagnostics/network-check-source-55646444c4-trplf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Dec 03 11:20:39 crc kubenswrapper[5052]: E1203 11:20:39.434101 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9d751cbb-f2e2-430d-9754-c882a5e924a5-kube-api-access-s2dwl podName:9d751cbb-f2e2-430d-9754-c882a5e924a5 nodeName:}" failed. No retries permitted until 2025-12-03 11:20:47.434086281 +0000 UTC m=+38.022328045 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-s2dwl" (UniqueName: "kubernetes.io/projected/9d751cbb-f2e2-430d-9754-c882a5e924a5-kube-api-access-s2dwl") pod "network-check-source-55646444c4-trplf" (UID: "9d751cbb-f2e2-430d-9754-c882a5e924a5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Dec 03 11:20:39 crc kubenswrapper[5052]: E1203 11:20:39.434164 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8 podName:8f668bae-612b-4b75-9490-919e737c6a3b nodeName:}" failed. No retries permitted until 2025-12-03 11:20:47.434153493 +0000 UTC m=+38.022395247 (durationBeforeRetry 8s). Error: UnmountVolume.TearDown failed for volume "pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8" (UniqueName: "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8") pod "8f668bae-612b-4b75-9490-919e737c6a3b" (UID: "8f668bae-612b-4b75-9490-919e737c6a3b") : kubernetes.io/csi: Unmounter.TearDownAt failed to get CSI client: driver name kubevirt.io.hostpath-provisioner not found in the list of registered CSI drivers Dec 03 11:20:39 crc kubenswrapper[5052]: E1203 11:20:39.434204 5052 configmap.go:193] Couldn't get configMap openshift-network-console/networking-console-plugin: object "openshift-network-console"/"networking-console-plugin" not registered Dec 03 11:20:39 crc kubenswrapper[5052]: E1203 11:20:39.434229 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5fe485a1-e14f-4c09-b5b9-f252bc42b7e8-nginx-conf podName:5fe485a1-e14f-4c09-b5b9-f252bc42b7e8 nodeName:}" failed. No retries permitted until 2025-12-03 11:20:47.434222605 +0000 UTC m=+38.022464369 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "nginx-conf" (UniqueName: "kubernetes.io/configmap/5fe485a1-e14f-4c09-b5b9-f252bc42b7e8-nginx-conf") pod "networking-console-plugin-85b44fc459-gdk6g" (UID: "5fe485a1-e14f-4c09-b5b9-f252bc42b7e8") : object "openshift-network-console"/"networking-console-plugin" not registered Dec 03 11:20:39 crc kubenswrapper[5052]: E1203 11:20:39.434286 5052 projected.go:288] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Dec 03 11:20:39 crc kubenswrapper[5052]: E1203 11:20:39.434297 5052 projected.go:288] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Dec 03 11:20:39 crc kubenswrapper[5052]: E1203 11:20:39.434306 5052 projected.go:194] Error preparing data for projected volume kube-api-access-cqllr for pod openshift-network-diagnostics/network-check-target-xd92c: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Dec 03 11:20:39 crc kubenswrapper[5052]: E1203 11:20:39.434332 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3b6479f0-333b-4a96-9adf-2099afdc2447-kube-api-access-cqllr podName:3b6479f0-333b-4a96-9adf-2099afdc2447 nodeName:}" failed. No retries permitted until 2025-12-03 11:20:47.434322228 +0000 UTC m=+38.022563992 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-cqllr" (UniqueName: "kubernetes.io/projected/3b6479f0-333b-4a96-9adf-2099afdc2447-kube-api-access-cqllr") pod "network-check-target-xd92c" (UID: "3b6479f0-333b-4a96-9adf-2099afdc2447") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Dec 03 11:20:39 crc kubenswrapper[5052]: E1203 11:20:39.434374 5052 secret.go:188] Couldn't get secret openshift-network-console/networking-console-plugin-cert: object "openshift-network-console"/"networking-console-plugin-cert" not registered Dec 03 11:20:39 crc kubenswrapper[5052]: E1203 11:20:39.434393 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5fe485a1-e14f-4c09-b5b9-f252bc42b7e8-networking-console-plugin-cert podName:5fe485a1-e14f-4c09-b5b9-f252bc42b7e8 nodeName:}" failed. No retries permitted until 2025-12-03 11:20:47.434388099 +0000 UTC m=+38.022629863 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/5fe485a1-e14f-4c09-b5b9-f252bc42b7e8-networking-console-plugin-cert") pod "networking-console-plugin-85b44fc459-gdk6g" (UID: "5fe485a1-e14f-4c09-b5b9-f252bc42b7e8") : object "openshift-network-console"/"networking-console-plugin-cert" not registered Dec 03 11:20:39 crc kubenswrapper[5052]: I1203 11:20:39.681257 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-85b44fc459-gdk6g" Dec 03 11:20:39 crc kubenswrapper[5052]: E1203 11:20:39.681364 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-console/networking-console-plugin-85b44fc459-gdk6g" podUID="5fe485a1-e14f-4c09-b5b9-f252bc42b7e8" Dec 03 11:20:39 crc kubenswrapper[5052]: I1203 11:20:39.681605 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-55646444c4-trplf" Dec 03 11:20:39 crc kubenswrapper[5052]: E1203 11:20:39.681674 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-source-55646444c4-trplf" podUID="9d751cbb-f2e2-430d-9754-c882a5e924a5" Dec 03 11:20:39 crc kubenswrapper[5052]: I1203 11:20:39.934197 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2lrql" event={"ID":"17265a68-bcb5-455c-b7df-c148a6517db9","Type":"ContainerStarted","Data":"1b407c9ba391dc2d2ed4b007a559f050253a94dc2299dcb18c3fc9cadea74066"} Dec 03 11:20:40 crc kubenswrapper[5052]: I1203 11:20:40.672890 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kf52k" Dec 03 11:20:40 crc kubenswrapper[5052]: I1203 11:20:40.672902 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xd92c" Dec 03 11:20:40 crc kubenswrapper[5052]: E1203 11:20:40.673055 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kf52k" podUID="a3f2b85d-9b43-4c1c-bc3a-602405b96efc" Dec 03 11:20:40 crc kubenswrapper[5052]: E1203 11:20:40.673126 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xd92c" podUID="3b6479f0-333b-4a96-9adf-2099afdc2447" Dec 03 11:20:40 crc kubenswrapper[5052]: I1203 11:20:40.944798 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kg55h" event={"ID":"bc569bba-c020-40d6-af68-e617625fe0e4","Type":"ContainerStarted","Data":"7548847d4c67dfe4acdf79ad3da86e54d4766ed7657a83a725c1ecc4a22c44dd"} Dec 03 11:20:40 crc kubenswrapper[5052]: I1203 11:20:40.969187 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-2lrql" podStartSLOduration=9.969165986 podStartE2EDuration="9.969165986s" podCreationTimestamp="2025-12-03 11:20:31 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:20:40.968553179 +0000 UTC m=+31.556794953" watchObservedRunningTime="2025-12-03 11:20:40.969165986 +0000 UTC m=+31.557407760" Dec 03 11:20:41 crc kubenswrapper[5052]: I1203 11:20:41.672675 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-55646444c4-trplf" Dec 03 11:20:41 crc kubenswrapper[5052]: I1203 11:20:41.672749 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-85b44fc459-gdk6g" Dec 03 11:20:41 crc kubenswrapper[5052]: E1203 11:20:41.672906 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-source-55646444c4-trplf" podUID="9d751cbb-f2e2-430d-9754-c882a5e924a5" Dec 03 11:20:41 crc kubenswrapper[5052]: E1203 11:20:41.673028 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-console/networking-console-plugin-85b44fc459-gdk6g" podUID="5fe485a1-e14f-4c09-b5b9-f252bc42b7e8" Dec 03 11:20:41 crc kubenswrapper[5052]: I1203 11:20:41.947187 5052 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Dec 03 11:20:41 crc kubenswrapper[5052]: I1203 11:20:41.947746 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openshift-ovn-kubernetes/ovnkube-node-kg55h" Dec 03 11:20:41 crc kubenswrapper[5052]: I1203 11:20:41.947795 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openshift-ovn-kubernetes/ovnkube-node-kg55h" Dec 03 11:20:41 crc kubenswrapper[5052]: I1203 11:20:41.973059 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-kg55h" Dec 03 11:20:41 crc kubenswrapper[5052]: I1203 11:20:41.974836 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-kg55h" Dec 03 11:20:42 crc kubenswrapper[5052]: I1203 11:20:42.002925 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-kg55h" podStartSLOduration=11.002905534 podStartE2EDuration="11.002905534s" podCreationTimestamp="2025-12-03 11:20:31 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:20:41.005899279 +0000 UTC m=+31.594141043" watchObservedRunningTime="2025-12-03 11:20:42.002905534 +0000 UTC m=+32.591147298" Dec 03 11:20:42 crc kubenswrapper[5052]: I1203 11:20:42.408900 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-kf52k"] Dec 03 11:20:42 crc kubenswrapper[5052]: I1203 11:20:42.409085 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kf52k" Dec 03 11:20:42 crc kubenswrapper[5052]: E1203 11:20:42.409240 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kf52k" podUID="a3f2b85d-9b43-4c1c-bc3a-602405b96efc" Dec 03 11:20:42 crc kubenswrapper[5052]: I1203 11:20:42.671935 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xd92c" Dec 03 11:20:42 crc kubenswrapper[5052]: E1203 11:20:42.672054 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xd92c" podUID="3b6479f0-333b-4a96-9adf-2099afdc2447" Dec 03 11:20:42 crc kubenswrapper[5052]: I1203 11:20:42.950317 5052 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Dec 03 11:20:43 crc kubenswrapper[5052]: I1203 11:20:43.074352 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a3f2b85d-9b43-4c1c-bc3a-602405b96efc-metrics-certs\") pod \"network-metrics-daemon-kf52k\" (UID: \"a3f2b85d-9b43-4c1c-bc3a-602405b96efc\") " pod="openshift-multus/network-metrics-daemon-kf52k" Dec 03 11:20:43 crc kubenswrapper[5052]: E1203 11:20:43.075125 5052 secret.go:188] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Dec 03 11:20:43 crc kubenswrapper[5052]: E1203 11:20:43.075362 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3f2b85d-9b43-4c1c-bc3a-602405b96efc-metrics-certs podName:a3f2b85d-9b43-4c1c-bc3a-602405b96efc nodeName:}" failed. No retries permitted until 2025-12-03 11:20:51.075171314 +0000 UTC m=+41.663413078 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a3f2b85d-9b43-4c1c-bc3a-602405b96efc-metrics-certs") pod "network-metrics-daemon-kf52k" (UID: "a3f2b85d-9b43-4c1c-bc3a-602405b96efc") : object "openshift-multus"/"metrics-daemon-secret" not registered Dec 03 11:20:43 crc kubenswrapper[5052]: I1203 11:20:43.672717 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-55646444c4-trplf" Dec 03 11:20:43 crc kubenswrapper[5052]: E1203 11:20:43.672970 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-source-55646444c4-trplf" podUID="9d751cbb-f2e2-430d-9754-c882a5e924a5" Dec 03 11:20:43 crc kubenswrapper[5052]: I1203 11:20:43.673409 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-85b44fc459-gdk6g" Dec 03 11:20:43 crc kubenswrapper[5052]: I1203 11:20:43.673455 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kf52k" Dec 03 11:20:43 crc kubenswrapper[5052]: E1203 11:20:43.673639 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-console/networking-console-plugin-85b44fc459-gdk6g" podUID="5fe485a1-e14f-4c09-b5b9-f252bc42b7e8" Dec 03 11:20:43 crc kubenswrapper[5052]: E1203 11:20:43.673741 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kf52k" podUID="a3f2b85d-9b43-4c1c-bc3a-602405b96efc" Dec 03 11:20:43 crc kubenswrapper[5052]: I1203 11:20:43.955177 5052 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Dec 03 11:20:44 crc kubenswrapper[5052]: I1203 11:20:44.672980 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xd92c" Dec 03 11:20:44 crc kubenswrapper[5052]: E1203 11:20:44.673188 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xd92c" podUID="3b6479f0-333b-4a96-9adf-2099afdc2447" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.573353 5052 kubelet_node_status.go:724] "Recording event message for node" node="crc" event="NodeReady" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.574167 5052 kubelet_node_status.go:538] "Fast updating node status as it just became ready" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.620706 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openshift-dns-operator/dns-operator-744455d44c-zvvfc"] Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.621340 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns-operator/dns-operator-744455d44c-zvvfc" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.622131 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openshift-oauth-apiserver/apiserver-7bbb656c7d-86vx8"] Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.622764 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-oauth-apiserver/apiserver-7bbb656c7d-86vx8" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.623202 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-665b6dd947-7wkt6"] Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.623620 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-665b6dd947-7wkt6" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.624157 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openshift-apiserver/apiserver-76f77b778f-dmvst"] Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.625144 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-apiserver/apiserver-76f77b778f-dmvst" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.625165 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openshift-authentication-operator/authentication-operator-69f744f599-qjq5f"] Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.625680 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-authentication-operator/authentication-operator-69f744f599-qjq5f" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.626301 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openshift-config-operator/openshift-config-operator-7777fb866f-bkztv"] Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.626788 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-config-operator/openshift-config-operator-7777fb866f-bkztv" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.627438 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openshift-image-registry/cluster-image-registry-operator-dc59b4c8b-4v96x"] Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.628028 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/cluster-image-registry-operator-dc59b4c8b-4v96x" Dec 03 11:20:45 crc kubenswrapper[5052]: W1203 11:20:45.628528 5052 reflector.go:561] object-"openshift-apiserver"/"image-import-ca": failed to list *v1.ConfigMap: configmaps "image-import-ca" is forbidden: User "system:node:crc" cannot list resource "configmaps" in API group "" in the namespace "openshift-apiserver": no relationship found between node 'crc' and this object Dec 03 11:20:45 crc kubenswrapper[5052]: E1203 11:20:45.628579 5052 reflector.go:158] "Unhandled Error" err="object-\"openshift-apiserver\"/\"image-import-ca\": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps \"image-import-ca\" is forbidden: User \"system:node:crc\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-apiserver\": no relationship found between node 'crc' and this object" logger="UnhandledError" Dec 03 11:20:45 crc kubenswrapper[5052]: W1203 11:20:45.628903 5052 reflector.go:561] object-"openshift-apiserver"/"etcd-client": failed to list *v1.Secret: secrets "etcd-client" is forbidden: User "system:node:crc" cannot list resource "secrets" in API group "" in the namespace "openshift-apiserver": no relationship found between node 'crc' and this object Dec 03 11:20:45 crc kubenswrapper[5052]: E1203 11:20:45.628928 5052 reflector.go:158] "Unhandled Error" err="object-\"openshift-apiserver\"/\"etcd-client\": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets \"etcd-client\" is forbidden: User \"system:node:crc\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-apiserver\": no relationship found between node 'crc' and this object" logger="UnhandledError" Dec 03 11:20:45 crc kubenswrapper[5052]: W1203 11:20:45.628904 5052 reflector.go:561] object-"openshift-apiserver"/"serving-cert": failed to list *v1.Secret: secrets "serving-cert" is forbidden: User "system:node:crc" cannot list resource "secrets" in API group "" in the namespace "openshift-apiserver": no relationship found between node 'crc' and this object Dec 03 11:20:45 crc kubenswrapper[5052]: W1203 11:20:45.628943 5052 reflector.go:561] object-"openshift-apiserver"/"etcd-serving-ca": failed to list *v1.ConfigMap: configmaps "etcd-serving-ca" is forbidden: User "system:node:crc" cannot list resource "configmaps" in API group "" in the namespace "openshift-apiserver": no relationship found between node 'crc' and this object Dec 03 11:20:45 crc kubenswrapper[5052]: E1203 11:20:45.628985 5052 reflector.go:158] "Unhandled Error" err="object-\"openshift-apiserver\"/\"etcd-serving-ca\": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps \"etcd-serving-ca\" is forbidden: User \"system:node:crc\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-apiserver\": no relationship found between node 'crc' and this object" logger="UnhandledError" Dec 03 11:20:45 crc kubenswrapper[5052]: E1203 11:20:45.628952 5052 reflector.go:158] "Unhandled Error" err="object-\"openshift-apiserver\"/\"serving-cert\": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets \"serving-cert\" is forbidden: User \"system:node:crc\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-apiserver\": no relationship found between node 'crc' and this object" logger="UnhandledError" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.629104 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openshift-authentication/oauth-openshift-558db77b4-99jsx"] Dec 03 11:20:45 crc kubenswrapper[5052]: W1203 11:20:45.629130 5052 reflector.go:561] object-"openshift-authentication-operator"/"serving-cert": failed to list *v1.Secret: secrets "serving-cert" is forbidden: User "system:node:crc" cannot list resource "secrets" in API group "" in the namespace "openshift-authentication-operator": no relationship found between node 'crc' and this object Dec 03 11:20:45 crc kubenswrapper[5052]: E1203 11:20:45.629150 5052 reflector.go:158] "Unhandled Error" err="object-\"openshift-authentication-operator\"/\"serving-cert\": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets \"serving-cert\" is forbidden: User \"system:node:crc\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-authentication-operator\": no relationship found between node 'crc' and this object" logger="UnhandledError" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.629162 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-dns-operator"/"dns-operator-dockercfg-9mqw5" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.629263 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-dns-operator"/"kube-root-ca.crt" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.629626 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-authentication/oauth-openshift-558db77b4-99jsx" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.629834 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-cluster-samples-operator"/"kube-root-ca.crt" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.630045 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-dns-operator"/"metrics-tls" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.631047 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-697d97f7c8-p77bx"] Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.631349 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-authentication-operator"/"openshift-service-ca.crt" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.631616 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-dns-operator"/"openshift-service-ca.crt" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.631662 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-697d97f7c8-p77bx" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.631815 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-oauth-apiserver"/"serving-cert" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.632062 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-config-operator"/"openshift-service-ca.crt" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.632166 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-cluster-samples-operator"/"samples-operator-tls" Dec 03 11:20:45 crc kubenswrapper[5052]: W1203 11:20:45.632214 5052 reflector.go:561] object-"openshift-authentication-operator"/"authentication-operator-config": failed to list *v1.ConfigMap: configmaps "authentication-operator-config" is forbidden: User "system:node:crc" cannot list resource "configmaps" in API group "" in the namespace "openshift-authentication-operator": no relationship found between node 'crc' and this object Dec 03 11:20:45 crc kubenswrapper[5052]: E1203 11:20:45.632248 5052 reflector.go:158] "Unhandled Error" err="object-\"openshift-authentication-operator\"/\"authentication-operator-config\": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps \"authentication-operator-config\" is forbidden: User \"system:node:crc\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-authentication-operator\": no relationship found between node 'crc' and this object" logger="UnhandledError" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.632322 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-oauth-apiserver"/"trusted-ca-bundle" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.632362 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-apiserver"/"config" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.632544 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-authentication-operator"/"authentication-operator-dockercfg-mz9bj" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.632741 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openshift-ingress-operator/ingress-operator-5b745b69d9-hc8vv"] Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.633393 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-operator/ingress-operator-5b745b69d9-hc8vv" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.634912 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-b67b599dd-5h545"] Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.635289 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-b67b599dd-5h545" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.638204 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/machine-config-controller-84d6567774-rxwdl"] Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.638336 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-authentication-operator"/"service-ca-bundle" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.638568 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/machine-config-operator-74547568cd-tr2xc"] Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.639282 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/machine-config-controller-84d6567774-rxwdl" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.639315 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/machine-config-operator-74547568cd-tr2xc" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.640329 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openshift-route-controller-manager/route-controller-manager-6576b87f9c-9fpqn"] Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.640690 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-route-controller-manager/route-controller-manager-6576b87f9c-9fpqn" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.642300 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openshift-marketplace/marketplace-operator-79b997595-kdr8l"] Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.642823 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/marketplace-operator-79b997595-kdr8l" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.656433 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-oauth-apiserver"/"encryption-config-1" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.656613 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-cluster-samples-operator"/"openshift-service-ca.crt" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.656435 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-oauth-apiserver"/"openshift-service-ca.crt" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.656795 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-cluster-samples-operator"/"cluster-samples-operator-dockercfg-xpp9w" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.658377 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-apiserver"/"encryption-config-1" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.658710 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-apiserver"/"openshift-service-ca.crt" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.660034 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openshift-kube-scheduler-operator/openshift-kube-scheduler-operator-5fdd9b5758-rxfb5"] Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.660412 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-apiserver"/"audit-1" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.660675 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-scheduler-operator/openshift-kube-scheduler-operator-5fdd9b5758-rxfb5" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.661234 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-config-operator"/"kube-root-ca.crt" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.661437 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-kube-storage-version-migrator-operator"/"openshift-service-ca.crt" Dec 03 11:20:45 crc kubenswrapper[5052]: W1203 11:20:45.661556 5052 reflector.go:561] object-"openshift-image-registry"/"installation-pull-secrets": failed to list *v1.Secret: secrets "installation-pull-secrets" is forbidden: User "system:node:crc" cannot list resource "secrets" in API group "" in the namespace "openshift-image-registry": no relationship found between node 'crc' and this object Dec 03 11:20:45 crc kubenswrapper[5052]: E1203 11:20:45.661587 5052 reflector.go:158] "Unhandled Error" err="object-\"openshift-image-registry\"/\"installation-pull-secrets\": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets \"installation-pull-secrets\" is forbidden: User \"system:node:crc\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-image-registry\": no relationship found between node 'crc' and this object" logger="UnhandledError" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.661728 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-config-operator"/"config-operator-serving-cert" Dec 03 11:20:45 crc kubenswrapper[5052]: W1203 11:20:45.661790 5052 reflector.go:561] object-"openshift-ingress-operator"/"openshift-service-ca.crt": failed to list *v1.ConfigMap: configmaps "openshift-service-ca.crt" is forbidden: User "system:node:crc" cannot list resource "configmaps" in API group "" in the namespace "openshift-ingress-operator": no relationship found between node 'crc' and this object Dec 03 11:20:45 crc kubenswrapper[5052]: E1203 11:20:45.661814 5052 reflector.go:158] "Unhandled Error" err="object-\"openshift-ingress-operator\"/\"openshift-service-ca.crt\": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps \"openshift-service-ca.crt\" is forbidden: User \"system:node:crc\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-ingress-operator\": no relationship found between node 'crc' and this object" logger="UnhandledError" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.661868 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-oauth-apiserver"/"kube-root-ca.crt" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.667471 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-authentication"/"v4-0-config-system-service-ca" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.667598 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-authentication"/"v4-0-config-user-template-provider-selection" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.668117 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-authentication"/"v4-0-config-system-session" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.668262 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-authentication"/"v4-0-config-user-idp-0-file-data" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.668386 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-authentication"/"v4-0-config-system-cliconfig" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.668635 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-oauth-apiserver"/"etcd-client" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.669165 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-apiserver"/"openshift-apiserver-sa-dockercfg-djjff" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.669413 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-oauth-apiserver"/"etcd-serving-ca" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.669645 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-route-controller-manager"/"config" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.669901 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-machine-config-operator"/"machine-config-operator-images" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.670968 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-oauth-apiserver"/"audit-1" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.672611 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-55646444c4-trplf" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.673242 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-85b44fc459-gdk6g" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.681302 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-authentication"/"oauth-openshift-dockercfg-znhcc" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.682012 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kf52k" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.682728 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-authentication"/"v4-0-config-user-template-error" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.683429 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-route-controller-manager"/"openshift-service-ca.crt" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.683649 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-marketplace"/"openshift-service-ca.crt" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.683810 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-authentication"/"openshift-service-ca.crt" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.683832 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-kube-storage-version-migrator-operator"/"config" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.683424 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-authentication-operator"/"kube-root-ca.crt" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.684830 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-route-controller-manager"/"route-controller-manager-sa-dockercfg-h2zr2" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.685168 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-image-registry"/"registry-dockercfg-kzzsd" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.685210 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-machine-config-operator"/"mcc-proxy-tls" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.685283 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-image-registry"/"image-registry-tls" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.685372 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-authentication"/"kube-root-ca.crt" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.685686 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-oauth-apiserver"/"oauth-apiserver-sa-dockercfg-6r2bq" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.685889 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-config-operator"/"openshift-config-operator-dockercfg-7pc5z" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.686019 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-kube-scheduler-operator"/"kube-root-ca.crt" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.686053 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-marketplace"/"marketplace-operator-dockercfg-5nsgg" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.686137 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-authentication"/"v4-0-config-system-serving-cert" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.686162 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-machine-config-operator"/"machine-config-operator-dockercfg-98p87" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.686246 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-authentication"/"v4-0-config-system-router-certs" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.686288 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-image-registry"/"cluster-image-registry-operator-dockercfg-m4qtx" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.686287 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-marketplace"/"kube-root-ca.crt" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.686342 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-ingress-operator"/"kube-root-ca.crt" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.686354 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-image-registry"/"image-registry-operator-tls" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.686386 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-kube-storage-version-migrator-operator"/"serving-cert" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.686286 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-authentication"/"audit" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.685965 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-route-controller-manager"/"client-ca" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.686466 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-apiserver"/"kube-root-ca.crt" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.685923 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-ingress-operator"/"metrics-tls" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.686528 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-kube-storage-version-migrator-operator"/"kube-root-ca.crt" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.686503 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-route-controller-manager"/"serving-cert" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.687489 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-kube-storage-version-migrator-operator"/"kube-storage-version-migrator-operator-dockercfg-2bh8d" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.688493 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-kube-scheduler-operator"/"openshift-kube-scheduler-operator-config" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.688769 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-ingress-operator"/"ingress-operator-dockercfg-7lnqk" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.689173 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-network-diagnostics"/"openshift-service-ca.crt" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.689455 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-kube-scheduler-operator"/"kube-scheduler-operator-serving-cert" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.689778 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-machine-config-operator"/"machine-config-controller-dockercfg-c2lfx" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.690145 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-route-controller-manager"/"kube-root-ca.crt" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.690288 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-marketplace"/"marketplace-operator-metrics" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.690545 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-kube-scheduler-operator"/"openshift-kube-scheduler-operator-dockercfg-qt55r" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.690650 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-machine-config-operator"/"mco-proxy-tls" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.693076 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-network-console"/"networking-console-plugin-cert" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.693792 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-network-console"/"networking-console-plugin" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.694017 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-multus"/"metrics-daemon-secret" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.694142 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-network-diagnostics"/"kube-root-ca.crt" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.694272 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-marketplace"/"marketplace-trusted-ca" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.702864 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-59844c95c7-4qts8"] Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.703835 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-7954f5f757-l7xtl"] Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.704176 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-59844c95c7-4qts8" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.704356 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openshift-controller-manager/controller-manager-879f6c89f-46rm8"] Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.704730 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-7954f5f757-l7xtl" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.706593 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-image-registry"/"trusted-ca" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.706979 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-multus"/"metrics-daemon-sa-dockercfg-d427c" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.714431 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-operator-tls\" (UniqueName: \"kubernetes.io/secret/407f9c0b-88d2-46f9-9f47-d3af857b0f30-image-registry-operator-tls\") pod \"cluster-image-registry-operator-dc59b4c8b-4v96x\" (UID: \"407f9c0b-88d2-46f9-9f47-d3af857b0f30\") " pod="openshift-image-registry/cluster-image-registry-operator-dc59b4c8b-4v96x" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.714484 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cabc2063-04d9-4491-be9f-f7781b5f02cf-metrics-tls\") pod \"dns-operator-744455d44c-zvvfc\" (UID: \"cabc2063-04d9-4491-be9f-f7781b5f02cf\") " pod="openshift-dns-operator/dns-operator-744455d44c-zvvfc" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.714554 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x6dnv\" (UniqueName: \"kubernetes.io/projected/8fae1d72-fba4-4999-bea5-39d6472d7ea7-kube-api-access-x6dnv\") pod \"apiserver-7bbb656c7d-86vx8\" (UID: \"8fae1d72-fba4-4999-bea5-39d6472d7ea7\") " pod="openshift-oauth-apiserver/apiserver-7bbb656c7d-86vx8" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.714581 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e0bf52c7-bf23-4c65-b7cb-0d8d466aa8f2-serving-cert\") pod \"authentication-operator-69f744f599-qjq5f\" (UID: \"e0bf52c7-bf23-4c65-b7cb-0d8d466aa8f2\") " pod="openshift-authentication-operator/authentication-operator-69f744f599-qjq5f" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.714605 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"encryption-config\" (UniqueName: \"kubernetes.io/secret/8fae1d72-fba4-4999-bea5-39d6472d7ea7-encryption-config\") pod \"apiserver-7bbb656c7d-86vx8\" (UID: \"8fae1d72-fba4-4999-bea5-39d6472d7ea7\") " pod="openshift-oauth-apiserver/apiserver-7bbb656c7d-86vx8" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.714624 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etcd-serving-ca\" (UniqueName: \"kubernetes.io/configmap/41016a7b-cc79-4f32-860c-cb75e405cf42-etcd-serving-ca\") pod \"apiserver-76f77b778f-dmvst\" (UID: \"41016a7b-cc79-4f32-860c-cb75e405cf42\") " pod="openshift-apiserver/apiserver-76f77b778f-dmvst" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.714649 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/407f9c0b-88d2-46f9-9f47-d3af857b0f30-bound-sa-token\") pod \"cluster-image-registry-operator-dc59b4c8b-4v96x\" (UID: \"407f9c0b-88d2-46f9-9f47-d3af857b0f30\") " pod="openshift-image-registry/cluster-image-registry-operator-dc59b4c8b-4v96x" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.714675 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nv988\" (UniqueName: \"kubernetes.io/projected/353b0eac-95ba-420e-bf67-a26b472fcdc2-kube-api-access-nv988\") pod \"route-controller-manager-6576b87f9c-9fpqn\" (UID: \"353b0eac-95ba-420e-bf67-a26b472fcdc2\") " pod="openshift-route-controller-manager/route-controller-manager-6576b87f9c-9fpqn" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.714700 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/efa98345-cafa-483a-976b-d772a3bd94e7-proxy-tls\") pod \"machine-config-controller-84d6567774-rxwdl\" (UID: \"efa98345-cafa-483a-976b-d772a3bd94e7\") " pod="openshift-machine-config-operator/machine-config-controller-84d6567774-rxwdl" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.714720 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"mcc-auth-proxy-config\" (UniqueName: \"kubernetes.io/configmap/efa98345-cafa-483a-976b-d772a3bd94e7-mcc-auth-proxy-config\") pod \"machine-config-controller-84d6567774-rxwdl\" (UID: \"efa98345-cafa-483a-976b-d772a3bd94e7\") " pod="openshift-machine-config-operator/machine-config-controller-84d6567774-rxwdl" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.714738 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/353b0eac-95ba-420e-bf67-a26b472fcdc2-config\") pod \"route-controller-manager-6576b87f9c-9fpqn\" (UID: \"353b0eac-95ba-420e-bf67-a26b472fcdc2\") " pod="openshift-route-controller-manager/route-controller-manager-6576b87f9c-9fpqn" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.714759 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-pullsecrets\" (UniqueName: \"kubernetes.io/host-path/41016a7b-cc79-4f32-860c-cb75e405cf42-node-pullsecrets\") pod \"apiserver-76f77b778f-dmvst\" (UID: \"41016a7b-cc79-4f32-860c-cb75e405cf42\") " pod="openshift-apiserver/apiserver-76f77b778f-dmvst" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.714783 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8vm6t\" (UniqueName: \"kubernetes.io/projected/cabc2063-04d9-4491-be9f-f7781b5f02cf-kube-api-access-8vm6t\") pod \"dns-operator-744455d44c-zvvfc\" (UID: \"cabc2063-04d9-4491-be9f-f7781b5f02cf\") " pod="openshift-dns-operator/dns-operator-744455d44c-zvvfc" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.714801 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-dir\" (UniqueName: \"kubernetes.io/host-path/41016a7b-cc79-4f32-860c-cb75e405cf42-audit-dir\") pod \"apiserver-76f77b778f-dmvst\" (UID: \"41016a7b-cc79-4f32-860c-cb75e405cf42\") " pod="openshift-apiserver/apiserver-76f77b778f-dmvst" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.714834 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etcd-serving-ca\" (UniqueName: \"kubernetes.io/configmap/8fae1d72-fba4-4999-bea5-39d6472d7ea7-etcd-serving-ca\") pod \"apiserver-7bbb656c7d-86vx8\" (UID: \"8fae1d72-fba4-4999-bea5-39d6472d7ea7\") " pod="openshift-oauth-apiserver/apiserver-7bbb656c7d-86vx8" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.714854 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e0bf52c7-bf23-4c65-b7cb-0d8d466aa8f2-service-ca-bundle\") pod \"authentication-operator-69f744f599-qjq5f\" (UID: \"e0bf52c7-bf23-4c65-b7cb-0d8d466aa8f2\") " pod="openshift-authentication-operator/authentication-operator-69f744f599-qjq5f" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.714880 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mh48n\" (UniqueName: \"kubernetes.io/projected/59e1467d-d303-4f32-a968-b66eb94953af-kube-api-access-mh48n\") pod \"cluster-samples-operator-665b6dd947-7wkt6\" (UID: \"59e1467d-d303-4f32-a968-b66eb94953af\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-665b6dd947-7wkt6" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.714903 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8fae1d72-fba4-4999-bea5-39d6472d7ea7-trusted-ca-bundle\") pod \"apiserver-7bbb656c7d-86vx8\" (UID: \"8fae1d72-fba4-4999-bea5-39d6472d7ea7\") " pod="openshift-oauth-apiserver/apiserver-7bbb656c7d-86vx8" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.714919 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c68wk\" (UniqueName: \"kubernetes.io/projected/41016a7b-cc79-4f32-860c-cb75e405cf42-kube-api-access-c68wk\") pod \"apiserver-76f77b778f-dmvst\" (UID: \"41016a7b-cc79-4f32-860c-cb75e405cf42\") " pod="openshift-apiserver/apiserver-76f77b778f-dmvst" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.714941 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t9z59\" (UniqueName: \"kubernetes.io/projected/407f9c0b-88d2-46f9-9f47-d3af857b0f30-kube-api-access-t9z59\") pod \"cluster-image-registry-operator-dc59b4c8b-4v96x\" (UID: \"407f9c0b-88d2-46f9-9f47-d3af857b0f30\") " pod="openshift-image-registry/cluster-image-registry-operator-dc59b4c8b-4v96x" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.714966 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/41016a7b-cc79-4f32-860c-cb75e405cf42-serving-cert\") pod \"apiserver-76f77b778f-dmvst\" (UID: \"41016a7b-cc79-4f32-860c-cb75e405cf42\") " pod="openshift-apiserver/apiserver-76f77b778f-dmvst" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.714995 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qxsnm\" (UniqueName: \"kubernetes.io/projected/833bb55c-5c9b-450c-9287-30f146a2d16a-kube-api-access-qxsnm\") pod \"ingress-operator-5b745b69d9-hc8vv\" (UID: \"833bb55c-5c9b-450c-9287-30f146a2d16a\") " pod="openshift-ingress-operator/ingress-operator-5b745b69d9-hc8vv" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.715026 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e0bf52c7-bf23-4c65-b7cb-0d8d466aa8f2-config\") pod \"authentication-operator-69f744f599-qjq5f\" (UID: \"e0bf52c7-bf23-4c65-b7cb-0d8d466aa8f2\") " pod="openshift-authentication-operator/authentication-operator-69f744f599-qjq5f" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.715067 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8fae1d72-fba4-4999-bea5-39d6472d7ea7-serving-cert\") pod \"apiserver-7bbb656c7d-86vx8\" (UID: \"8fae1d72-fba4-4999-bea5-39d6472d7ea7\") " pod="openshift-oauth-apiserver/apiserver-7bbb656c7d-86vx8" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.715093 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca\" (UniqueName: \"kubernetes.io/configmap/353b0eac-95ba-420e-bf67-a26b472fcdc2-client-ca\") pod \"route-controller-manager-6576b87f9c-9fpqn\" (UID: \"353b0eac-95ba-420e-bf67-a26b472fcdc2\") " pod="openshift-route-controller-manager/route-controller-manager-6576b87f9c-9fpqn" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.715120 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/833bb55c-5c9b-450c-9287-30f146a2d16a-trusted-ca\") pod \"ingress-operator-5b745b69d9-hc8vv\" (UID: \"833bb55c-5c9b-450c-9287-30f146a2d16a\") " pod="openshift-ingress-operator/ingress-operator-5b745b69d9-hc8vv" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.715143 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit\" (UniqueName: \"kubernetes.io/configmap/41016a7b-cc79-4f32-860c-cb75e405cf42-audit\") pod \"apiserver-76f77b778f-dmvst\" (UID: \"41016a7b-cc79-4f32-860c-cb75e405cf42\") " pod="openshift-apiserver/apiserver-76f77b778f-dmvst" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.715211 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/833bb55c-5c9b-450c-9287-30f146a2d16a-metrics-tls\") pod \"ingress-operator-5b745b69d9-hc8vv\" (UID: \"833bb55c-5c9b-450c-9287-30f146a2d16a\") " pod="openshift-ingress-operator/ingress-operator-5b745b69d9-hc8vv" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.715252 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"encryption-config\" (UniqueName: \"kubernetes.io/secret/41016a7b-cc79-4f32-860c-cb75e405cf42-encryption-config\") pod \"apiserver-76f77b778f-dmvst\" (UID: \"41016a7b-cc79-4f32-860c-cb75e405cf42\") " pod="openshift-apiserver/apiserver-76f77b778f-dmvst" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.715281 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/41016a7b-cc79-4f32-860c-cb75e405cf42-trusted-ca-bundle\") pod \"apiserver-76f77b778f-dmvst\" (UID: \"41016a7b-cc79-4f32-860c-cb75e405cf42\") " pod="openshift-apiserver/apiserver-76f77b778f-dmvst" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.715309 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-dir\" (UniqueName: \"kubernetes.io/host-path/8fae1d72-fba4-4999-bea5-39d6472d7ea7-audit-dir\") pod \"apiserver-7bbb656c7d-86vx8\" (UID: \"8fae1d72-fba4-4999-bea5-39d6472d7ea7\") " pod="openshift-oauth-apiserver/apiserver-7bbb656c7d-86vx8" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.715386 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-import-ca\" (UniqueName: \"kubernetes.io/configmap/41016a7b-cc79-4f32-860c-cb75e405cf42-image-import-ca\") pod \"apiserver-76f77b778f-dmvst\" (UID: \"41016a7b-cc79-4f32-860c-cb75e405cf42\") " pod="openshift-apiserver/apiserver-76f77b778f-dmvst" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.715409 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/59e1467d-d303-4f32-a968-b66eb94953af-samples-operator-tls\") pod \"cluster-samples-operator-665b6dd947-7wkt6\" (UID: \"59e1467d-d303-4f32-a968-b66eb94953af\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-665b6dd947-7wkt6" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.715436 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etcd-client\" (UniqueName: \"kubernetes.io/secret/8fae1d72-fba4-4999-bea5-39d6472d7ea7-etcd-client\") pod \"apiserver-7bbb656c7d-86vx8\" (UID: \"8fae1d72-fba4-4999-bea5-39d6472d7ea7\") " pod="openshift-oauth-apiserver/apiserver-7bbb656c7d-86vx8" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.715465 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/353b0eac-95ba-420e-bf67-a26b472fcdc2-serving-cert\") pod \"route-controller-manager-6576b87f9c-9fpqn\" (UID: \"353b0eac-95ba-420e-bf67-a26b472fcdc2\") " pod="openshift-route-controller-manager/route-controller-manager-6576b87f9c-9fpqn" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.715496 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e0bf52c7-bf23-4c65-b7cb-0d8d466aa8f2-trusted-ca-bundle\") pod \"authentication-operator-69f744f599-qjq5f\" (UID: \"e0bf52c7-bf23-4c65-b7cb-0d8d466aa8f2\") " pod="openshift-authentication-operator/authentication-operator-69f744f599-qjq5f" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.715548 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-policies\" (UniqueName: \"kubernetes.io/configmap/8fae1d72-fba4-4999-bea5-39d6472d7ea7-audit-policies\") pod \"apiserver-7bbb656c7d-86vx8\" (UID: \"8fae1d72-fba4-4999-bea5-39d6472d7ea7\") " pod="openshift-oauth-apiserver/apiserver-7bbb656c7d-86vx8" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.715571 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/833bb55c-5c9b-450c-9287-30f146a2d16a-bound-sa-token\") pod \"ingress-operator-5b745b69d9-hc8vv\" (UID: \"833bb55c-5c9b-450c-9287-30f146a2d16a\") " pod="openshift-ingress-operator/ingress-operator-5b745b69d9-hc8vv" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.715608 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gz6dn\" (UniqueName: \"kubernetes.io/projected/efa98345-cafa-483a-976b-d772a3bd94e7-kube-api-access-gz6dn\") pod \"machine-config-controller-84d6567774-rxwdl\" (UID: \"efa98345-cafa-483a-976b-d772a3bd94e7\") " pod="openshift-machine-config-operator/machine-config-controller-84d6567774-rxwdl" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.715632 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etcd-client\" (UniqueName: \"kubernetes.io/secret/41016a7b-cc79-4f32-860c-cb75e405cf42-etcd-client\") pod \"apiserver-76f77b778f-dmvst\" (UID: \"41016a7b-cc79-4f32-860c-cb75e405cf42\") " pod="openshift-apiserver/apiserver-76f77b778f-dmvst" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.715673 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ng77f\" (UniqueName: \"kubernetes.io/projected/e0bf52c7-bf23-4c65-b7cb-0d8d466aa8f2-kube-api-access-ng77f\") pod \"authentication-operator-69f744f599-qjq5f\" (UID: \"e0bf52c7-bf23-4c65-b7cb-0d8d466aa8f2\") " pod="openshift-authentication-operator/authentication-operator-69f744f599-qjq5f" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.715718 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/41016a7b-cc79-4f32-860c-cb75e405cf42-config\") pod \"apiserver-76f77b778f-dmvst\" (UID: \"41016a7b-cc79-4f32-860c-cb75e405cf42\") " pod="openshift-apiserver/apiserver-76f77b778f-dmvst" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.715745 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/407f9c0b-88d2-46f9-9f47-d3af857b0f30-trusted-ca\") pod \"cluster-image-registry-operator-dc59b4c8b-4v96x\" (UID: \"407f9c0b-88d2-46f9-9f47-d3af857b0f30\") " pod="openshift-image-registry/cluster-image-registry-operator-dc59b4c8b-4v96x" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.717431 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-ingress-operator"/"trusted-ca" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.733482 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-authentication"/"v4-0-config-user-template-login" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.733715 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openshift-cluster-machine-approver/machine-approver-56656f9798-6t9dp"] Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.734296 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-58897d9998-7vpns"] Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.734572 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-controller-manager/controller-manager-879f6c89f-46rm8" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.734997 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-machine-approver/machine-approver-56656f9798-6t9dp" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.738809 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-authentication"/"v4-0-config-system-ocp-branding-template" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.740974 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openshift-controller-manager-operator/openshift-controller-manager-operator-756b6f6bc6-v7lmf"] Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.741656 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openshift-console/console-f9d7485db-568dj"] Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.742238 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openshift-kube-apiserver-operator/kube-apiserver-operator-766d6c64bb-xwlk8"] Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.744419 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-admission-controller-857f4d67dd-q8n58"] Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.751581 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openshift-etcd-operator/etcd-operator-b45778765-z2lp6"] Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.743267 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-controller-manager-operator/openshift-controller-manager-operator-756b6f6bc6-v7lmf" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.743211 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-58897d9998-7vpns" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.752001 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-admission-controller-857f4d67dd-q8n58" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.744820 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-apiserver-operator/kube-apiserver-operator-766d6c64bb-xwlk8" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.744553 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-authentication"/"v4-0-config-system-trusted-ca-bundle" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.743310 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-f9d7485db-568dj" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.746981 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-apiserver"/"trusted-ca-bundle" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.749965 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-authentication-operator"/"trusted-ca-bundle" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.751395 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-console"/"openshift-service-ca.crt" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.751439 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-console"/"default-dockercfg-chnjx" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.751553 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-kube-storage-version-migrator"/"openshift-service-ca.crt" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.751667 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-console"/"kube-root-ca.crt" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.751797 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-kube-storage-version-migrator"/"kube-root-ca.crt" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.751919 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-kube-storage-version-migrator"/"kube-storage-version-migrator-sa-dockercfg-5xfcg" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.753224 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-etcd-operator/etcd-operator-b45778765-z2lp6" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.754372 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openshift-machine-api/machine-api-operator-5694c8668f-l5lrg"] Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.755136 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openshift-kube-controller-manager-operator/kube-controller-manager-operator-78b949d7b-h6b78"] Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.755153 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-api/machine-api-operator-5694c8668f-l5lrg" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.756208 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-controller-manager-operator/kube-controller-manager-operator-78b949d7b-h6b78" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.756579 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-5444994796-rtl8t"] Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.757023 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-5444994796-rtl8t" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.757393 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openshift-operator-lifecycle-manager/package-server-manager-789f6589d5-577rn"] Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.757955 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-operator-lifecycle-manager/package-server-manager-789f6589d5-577rn" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.762284 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openshift-apiserver-operator/openshift-apiserver-operator-796bbdcf4f-jskzg"] Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.763098 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-apiserver-operator/openshift-apiserver-operator-796bbdcf4f-jskzg" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.763546 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openshift-operator-lifecycle-manager/catalog-operator-68c6474976-rqhkg"] Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.763879 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-operator-lifecycle-manager/catalog-operator-68c6474976-rqhkg" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.763877 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openshift-operator-lifecycle-manager/packageserver-d55dfcdfc-7s97q"] Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.764438 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-controller-manager"/"openshift-service-ca.crt" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.764561 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-operator-lifecycle-manager/packageserver-d55dfcdfc-7s97q" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.774602 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openshift-operator-lifecycle-manager/olm-operator-6b444d44fb-6lm8x"] Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.775796 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openshift-operator-lifecycle-manager/collect-profiles-29412675-hzd4n"] Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.776189 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-9c57cc56f-w7g95"] Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.776642 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-9c57cc56f-w7g95" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.776944 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-operator-lifecycle-manager/olm-operator-6b444d44fb-6lm8x" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.777124 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-operator-lifecycle-manager/collect-profiles-29412675-hzd4n" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.791199 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-cluster-machine-approver"/"kube-rbac-proxy" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.791960 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openshift-machine-api/control-plane-machine-set-operator-78cbb6b69f-p4b4l"] Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.792767 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-api/control-plane-machine-set-operator-78cbb6b69f-p4b4l" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.792969 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-777779d784-9jvh6"] Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.793781 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-777779d784-9jvh6" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.820984 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xk4gt\" (UniqueName: \"kubernetes.io/projected/6e31bff9-7ac9-47a7-a895-db2cc3051d85-kube-api-access-xk4gt\") pod \"kube-storage-version-migrator-operator-b67b599dd-5h545\" (UID: \"6e31bff9-7ac9-47a7-a895-db2cc3051d85\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-b67b599dd-5h545" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.821132 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e0bf52c7-bf23-4c65-b7cb-0d8d466aa8f2-serving-cert\") pod \"authentication-operator-69f744f599-qjq5f\" (UID: \"e0bf52c7-bf23-4c65-b7cb-0d8d466aa8f2\") " pod="openshift-authentication-operator/authentication-operator-69f744f599-qjq5f" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.821270 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-x6dnv\" (UniqueName: \"kubernetes.io/projected/8fae1d72-fba4-4999-bea5-39d6472d7ea7-kube-api-access-x6dnv\") pod \"apiserver-7bbb656c7d-86vx8\" (UID: \"8fae1d72-fba4-4999-bea5-39d6472d7ea7\") " pod="openshift-oauth-apiserver/apiserver-7bbb656c7d-86vx8" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.821337 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f5012254-068c-4360-b310-39ff3d309199-ca-trust-extracted\") pod \"image-registry-697d97f7c8-p77bx\" (UID: \"f5012254-068c-4360-b310-39ff3d309199\") " pod="openshift-image-registry/image-registry-697d97f7c8-p77bx" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.821385 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"encryption-config\" (UniqueName: \"kubernetes.io/secret/8fae1d72-fba4-4999-bea5-39d6472d7ea7-encryption-config\") pod \"apiserver-7bbb656c7d-86vx8\" (UID: \"8fae1d72-fba4-4999-bea5-39d6472d7ea7\") " pod="openshift-oauth-apiserver/apiserver-7bbb656c7d-86vx8" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.821418 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etcd-serving-ca\" (UniqueName: \"kubernetes.io/configmap/41016a7b-cc79-4f32-860c-cb75e405cf42-etcd-serving-ca\") pod \"apiserver-76f77b778f-dmvst\" (UID: \"41016a7b-cc79-4f32-860c-cb75e405cf42\") " pod="openshift-apiserver/apiserver-76f77b778f-dmvst" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.821578 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l5k2l\" (UniqueName: \"kubernetes.io/projected/75f62784-7818-4a74-9c8c-4777d6db0611-kube-api-access-l5k2l\") pod \"oauth-openshift-558db77b4-99jsx\" (UID: \"75f62784-7818-4a74-9c8c-4777d6db0611\") " pod="openshift-authentication/oauth-openshift-558db77b4-99jsx" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.821622 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/407f9c0b-88d2-46f9-9f47-d3af857b0f30-bound-sa-token\") pod \"cluster-image-registry-operator-dc59b4c8b-4v96x\" (UID: \"407f9c0b-88d2-46f9-9f47-d3af857b0f30\") " pod="openshift-image-registry/cluster-image-registry-operator-dc59b4c8b-4v96x" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.821670 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-nv988\" (UniqueName: \"kubernetes.io/projected/353b0eac-95ba-420e-bf67-a26b472fcdc2-kube-api-access-nv988\") pod \"route-controller-manager-6576b87f9c-9fpqn\" (UID: \"353b0eac-95ba-420e-bf67-a26b472fcdc2\") " pod="openshift-route-controller-manager/route-controller-manager-6576b87f9c-9fpqn" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.821708 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\") pod \"image-registry-697d97f7c8-p77bx\" (UID: \"f5012254-068c-4360-b310-39ff3d309199\") " pod="openshift-image-registry/image-registry-697d97f7c8-p77bx" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.821748 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/efa98345-cafa-483a-976b-d772a3bd94e7-proxy-tls\") pod \"machine-config-controller-84d6567774-rxwdl\" (UID: \"efa98345-cafa-483a-976b-d772a3bd94e7\") " pod="openshift-machine-config-operator/machine-config-controller-84d6567774-rxwdl" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.821779 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"mcc-auth-proxy-config\" (UniqueName: \"kubernetes.io/configmap/efa98345-cafa-483a-976b-d772a3bd94e7-mcc-auth-proxy-config\") pod \"machine-config-controller-84d6567774-rxwdl\" (UID: \"efa98345-cafa-483a-976b-d772a3bd94e7\") " pod="openshift-machine-config-operator/machine-config-controller-84d6567774-rxwdl" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.821869 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/353b0eac-95ba-420e-bf67-a26b472fcdc2-config\") pod \"route-controller-manager-6576b87f9c-9fpqn\" (UID: \"353b0eac-95ba-420e-bf67-a26b472fcdc2\") " pod="openshift-route-controller-manager/route-controller-manager-6576b87f9c-9fpqn" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.822127 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"node-pullsecrets\" (UniqueName: \"kubernetes.io/host-path/41016a7b-cc79-4f32-860c-cb75e405cf42-node-pullsecrets\") pod \"apiserver-76f77b778f-dmvst\" (UID: \"41016a7b-cc79-4f32-860c-cb75e405cf42\") " pod="openshift-apiserver/apiserver-76f77b778f-dmvst" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.822171 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"available-featuregates\" (UniqueName: \"kubernetes.io/empty-dir/0a110335-c2e5-40a2-b565-a4791146d53b-available-featuregates\") pod \"openshift-config-operator-7777fb866f-bkztv\" (UID: \"0a110335-c2e5-40a2-b565-a4791146d53b\") " pod="openshift-config-operator/openshift-config-operator-7777fb866f-bkztv" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.822205 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-8vm6t\" (UniqueName: \"kubernetes.io/projected/cabc2063-04d9-4491-be9f-f7781b5f02cf-kube-api-access-8vm6t\") pod \"dns-operator-744455d44c-zvvfc\" (UID: \"cabc2063-04d9-4491-be9f-f7781b5f02cf\") " pod="openshift-dns-operator/dns-operator-744455d44c-zvvfc" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.822239 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"audit-dir\" (UniqueName: \"kubernetes.io/host-path/41016a7b-cc79-4f32-860c-cb75e405cf42-audit-dir\") pod \"apiserver-76f77b778f-dmvst\" (UID: \"41016a7b-cc79-4f32-860c-cb75e405cf42\") " pod="openshift-apiserver/apiserver-76f77b778f-dmvst" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.822268 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f5012254-068c-4360-b310-39ff3d309199-bound-sa-token\") pod \"image-registry-697d97f7c8-p77bx\" (UID: \"f5012254-068c-4360-b310-39ff3d309199\") " pod="openshift-image-registry/image-registry-697d97f7c8-p77bx" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.822302 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etcd-serving-ca\" (UniqueName: \"kubernetes.io/configmap/8fae1d72-fba4-4999-bea5-39d6472d7ea7-etcd-serving-ca\") pod \"apiserver-7bbb656c7d-86vx8\" (UID: \"8fae1d72-fba4-4999-bea5-39d6472d7ea7\") " pod="openshift-oauth-apiserver/apiserver-7bbb656c7d-86vx8" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.822324 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e0bf52c7-bf23-4c65-b7cb-0d8d466aa8f2-service-ca-bundle\") pod \"authentication-operator-69f744f599-qjq5f\" (UID: \"e0bf52c7-bf23-4c65-b7cb-0d8d466aa8f2\") " pod="openshift-authentication-operator/authentication-operator-69f744f599-qjq5f" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.822354 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"v4-0-config-system-ocp-branding-template\" (UniqueName: \"kubernetes.io/secret/75f62784-7818-4a74-9c8c-4777d6db0611-v4-0-config-system-ocp-branding-template\") pod \"oauth-openshift-558db77b4-99jsx\" (UID: \"75f62784-7818-4a74-9c8c-4777d6db0611\") " pod="openshift-authentication/oauth-openshift-558db77b4-99jsx" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.822383 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/6e31bff9-7ac9-47a7-a895-db2cc3051d85-config\") pod \"kube-storage-version-migrator-operator-b67b599dd-5h545\" (UID: \"6e31bff9-7ac9-47a7-a895-db2cc3051d85\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-b67b599dd-5h545" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.822411 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-t9z59\" (UniqueName: \"kubernetes.io/projected/407f9c0b-88d2-46f9-9f47-d3af857b0f30-kube-api-access-t9z59\") pod \"cluster-image-registry-operator-dc59b4c8b-4v96x\" (UID: \"407f9c0b-88d2-46f9-9f47-d3af857b0f30\") " pod="openshift-image-registry/cluster-image-registry-operator-dc59b4c8b-4v96x" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.822447 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-mh48n\" (UniqueName: \"kubernetes.io/projected/59e1467d-d303-4f32-a968-b66eb94953af-kube-api-access-mh48n\") pod \"cluster-samples-operator-665b6dd947-7wkt6\" (UID: \"59e1467d-d303-4f32-a968-b66eb94953af\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-665b6dd947-7wkt6" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.822468 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8fae1d72-fba4-4999-bea5-39d6472d7ea7-trusted-ca-bundle\") pod \"apiserver-7bbb656c7d-86vx8\" (UID: \"8fae1d72-fba4-4999-bea5-39d6472d7ea7\") " pod="openshift-oauth-apiserver/apiserver-7bbb656c7d-86vx8" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.822495 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-c68wk\" (UniqueName: \"kubernetes.io/projected/41016a7b-cc79-4f32-860c-cb75e405cf42-kube-api-access-c68wk\") pod \"apiserver-76f77b778f-dmvst\" (UID: \"41016a7b-cc79-4f32-860c-cb75e405cf42\") " pod="openshift-apiserver/apiserver-76f77b778f-dmvst" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.822560 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f5012254-068c-4360-b310-39ff3d309199-registry-certificates\") pod \"image-registry-697d97f7c8-p77bx\" (UID: \"f5012254-068c-4360-b310-39ff3d309199\") " pod="openshift-image-registry/image-registry-697d97f7c8-p77bx" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.822591 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4cgvv\" (UniqueName: \"kubernetes.io/projected/cac70dab-6e13-4693-ac66-c7552b62dfd7-kube-api-access-4cgvv\") pod \"marketplace-operator-79b997595-kdr8l\" (UID: \"cac70dab-6e13-4693-ac66-c7552b62dfd7\") " pod="openshift-marketplace/marketplace-operator-79b997595-kdr8l" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.822615 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"v4-0-config-system-serving-cert\" (UniqueName: \"kubernetes.io/secret/75f62784-7818-4a74-9c8c-4777d6db0611-v4-0-config-system-serving-cert\") pod \"oauth-openshift-558db77b4-99jsx\" (UID: \"75f62784-7818-4a74-9c8c-4777d6db0611\") " pod="openshift-authentication/oauth-openshift-558db77b4-99jsx" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.822645 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/41016a7b-cc79-4f32-860c-cb75e405cf42-serving-cert\") pod \"apiserver-76f77b778f-dmvst\" (UID: \"41016a7b-cc79-4f32-860c-cb75e405cf42\") " pod="openshift-apiserver/apiserver-76f77b778f-dmvst" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.822673 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"v4-0-config-system-session\" (UniqueName: \"kubernetes.io/secret/75f62784-7818-4a74-9c8c-4777d6db0611-v4-0-config-system-session\") pod \"oauth-openshift-558db77b4-99jsx\" (UID: \"75f62784-7818-4a74-9c8c-4777d6db0611\") " pod="openshift-authentication/oauth-openshift-558db77b4-99jsx" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.822717 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-qxsnm\" (UniqueName: \"kubernetes.io/projected/833bb55c-5c9b-450c-9287-30f146a2d16a-kube-api-access-qxsnm\") pod \"ingress-operator-5b745b69d9-hc8vv\" (UID: \"833bb55c-5c9b-450c-9287-30f146a2d16a\") " pod="openshift-ingress-operator/ingress-operator-5b745b69d9-hc8vv" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.822745 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e0bf52c7-bf23-4c65-b7cb-0d8d466aa8f2-config\") pod \"authentication-operator-69f744f599-qjq5f\" (UID: \"e0bf52c7-bf23-4c65-b7cb-0d8d466aa8f2\") " pod="openshift-authentication-operator/authentication-operator-69f744f599-qjq5f" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.822772 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ztr7w\" (UniqueName: \"kubernetes.io/projected/0a110335-c2e5-40a2-b565-a4791146d53b-kube-api-access-ztr7w\") pod \"openshift-config-operator-7777fb866f-bkztv\" (UID: \"0a110335-c2e5-40a2-b565-a4791146d53b\") " pod="openshift-config-operator/openshift-config-operator-7777fb866f-bkztv" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.822798 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"marketplace-trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cac70dab-6e13-4693-ac66-c7552b62dfd7-marketplace-trusted-ca\") pod \"marketplace-operator-79b997595-kdr8l\" (UID: \"cac70dab-6e13-4693-ac66-c7552b62dfd7\") " pod="openshift-marketplace/marketplace-operator-79b997595-kdr8l" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.822822 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8fae1d72-fba4-4999-bea5-39d6472d7ea7-serving-cert\") pod \"apiserver-7bbb656c7d-86vx8\" (UID: \"8fae1d72-fba4-4999-bea5-39d6472d7ea7\") " pod="openshift-oauth-apiserver/apiserver-7bbb656c7d-86vx8" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.822859 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"client-ca\" (UniqueName: \"kubernetes.io/configmap/353b0eac-95ba-420e-bf67-a26b472fcdc2-client-ca\") pod \"route-controller-manager-6576b87f9c-9fpqn\" (UID: \"353b0eac-95ba-420e-bf67-a26b472fcdc2\") " pod="openshift-route-controller-manager/route-controller-manager-6576b87f9c-9fpqn" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.822893 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f5012254-068c-4360-b310-39ff3d309199-trusted-ca\") pod \"image-registry-697d97f7c8-p77bx\" (UID: \"f5012254-068c-4360-b310-39ff3d309199\") " pod="openshift-image-registry/image-registry-697d97f7c8-p77bx" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.822921 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"v4-0-config-system-cliconfig\" (UniqueName: \"kubernetes.io/configmap/75f62784-7818-4a74-9c8c-4777d6db0611-v4-0-config-system-cliconfig\") pod \"oauth-openshift-558db77b4-99jsx\" (UID: \"75f62784-7818-4a74-9c8c-4777d6db0611\") " pod="openshift-authentication/oauth-openshift-558db77b4-99jsx" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.822956 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/833bb55c-5c9b-450c-9287-30f146a2d16a-trusted-ca\") pod \"ingress-operator-5b745b69d9-hc8vv\" (UID: \"833bb55c-5c9b-450c-9287-30f146a2d16a\") " pod="openshift-ingress-operator/ingress-operator-5b745b69d9-hc8vv" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.822993 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"audit\" (UniqueName: \"kubernetes.io/configmap/41016a7b-cc79-4f32-860c-cb75e405cf42-audit\") pod \"apiserver-76f77b778f-dmvst\" (UID: \"41016a7b-cc79-4f32-860c-cb75e405cf42\") " pod="openshift-apiserver/apiserver-76f77b778f-dmvst" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.823024 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"v4-0-config-user-template-error\" (UniqueName: \"kubernetes.io/secret/75f62784-7818-4a74-9c8c-4777d6db0611-v4-0-config-user-template-error\") pod \"oauth-openshift-558db77b4-99jsx\" (UID: \"75f62784-7818-4a74-9c8c-4777d6db0611\") " pod="openshift-authentication/oauth-openshift-558db77b4-99jsx" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.823053 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/833bb55c-5c9b-450c-9287-30f146a2d16a-metrics-tls\") pod \"ingress-operator-5b745b69d9-hc8vv\" (UID: \"833bb55c-5c9b-450c-9287-30f146a2d16a\") " pod="openshift-ingress-operator/ingress-operator-5b745b69d9-hc8vv" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.823097 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/41016a7b-cc79-4f32-860c-cb75e405cf42-trusted-ca-bundle\") pod \"apiserver-76f77b778f-dmvst\" (UID: \"41016a7b-cc79-4f32-860c-cb75e405cf42\") " pod="openshift-apiserver/apiserver-76f77b778f-dmvst" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.823126 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"encryption-config\" (UniqueName: \"kubernetes.io/secret/41016a7b-cc79-4f32-860c-cb75e405cf42-encryption-config\") pod \"apiserver-76f77b778f-dmvst\" (UID: \"41016a7b-cc79-4f32-860c-cb75e405cf42\") " pod="openshift-apiserver/apiserver-76f77b778f-dmvst" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.823153 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f5012254-068c-4360-b310-39ff3d309199-registry-tls\") pod \"image-registry-697d97f7c8-p77bx\" (UID: \"f5012254-068c-4360-b310-39ff3d309199\") " pod="openshift-image-registry/image-registry-697d97f7c8-p77bx" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.823200 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"v4-0-config-user-idp-0-file-data\" (UniqueName: \"kubernetes.io/secret/75f62784-7818-4a74-9c8c-4777d6db0611-v4-0-config-user-idp-0-file-data\") pod \"oauth-openshift-558db77b4-99jsx\" (UID: \"75f62784-7818-4a74-9c8c-4777d6db0611\") " pod="openshift-authentication/oauth-openshift-558db77b4-99jsx" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.823204 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"etcd-serving-ca\" (UniqueName: \"kubernetes.io/configmap/8fae1d72-fba4-4999-bea5-39d6472d7ea7-etcd-serving-ca\") pod \"apiserver-7bbb656c7d-86vx8\" (UID: \"8fae1d72-fba4-4999-bea5-39d6472d7ea7\") " pod="openshift-oauth-apiserver/apiserver-7bbb656c7d-86vx8" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.823243 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"audit-dir\" (UniqueName: \"kubernetes.io/host-path/8fae1d72-fba4-4999-bea5-39d6472d7ea7-audit-dir\") pod \"apiserver-7bbb656c7d-86vx8\" (UID: \"8fae1d72-fba4-4999-bea5-39d6472d7ea7\") " pod="openshift-oauth-apiserver/apiserver-7bbb656c7d-86vx8" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.823272 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cvfz9\" (UniqueName: \"kubernetes.io/projected/f5012254-068c-4360-b310-39ff3d309199-kube-api-access-cvfz9\") pod \"image-registry-697d97f7c8-p77bx\" (UID: \"f5012254-068c-4360-b310-39ff3d309199\") " pod="openshift-image-registry/image-registry-697d97f7c8-p77bx" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.823299 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0a110335-c2e5-40a2-b565-a4791146d53b-serving-cert\") pod \"openshift-config-operator-7777fb866f-bkztv\" (UID: \"0a110335-c2e5-40a2-b565-a4791146d53b\") " pod="openshift-config-operator/openshift-config-operator-7777fb866f-bkztv" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.823329 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"images\" (UniqueName: \"kubernetes.io/configmap/cc99b3a0-fa04-4128-b730-1d24da63d08e-images\") pod \"machine-config-operator-74547568cd-tr2xc\" (UID: \"cc99b3a0-fa04-4128-b730-1d24da63d08e\") " pod="openshift-machine-config-operator/machine-config-operator-74547568cd-tr2xc" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.824040 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"image-import-ca\" (UniqueName: \"kubernetes.io/configmap/41016a7b-cc79-4f32-860c-cb75e405cf42-image-import-ca\") pod \"apiserver-76f77b778f-dmvst\" (UID: \"41016a7b-cc79-4f32-860c-cb75e405cf42\") " pod="openshift-apiserver/apiserver-76f77b778f-dmvst" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.824170 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-dir\" (UniqueName: \"kubernetes.io/host-path/75f62784-7818-4a74-9c8c-4777d6db0611-audit-dir\") pod \"oauth-openshift-558db77b4-99jsx\" (UID: \"75f62784-7818-4a74-9c8c-4777d6db0611\") " pod="openshift-authentication/oauth-openshift-558db77b4-99jsx" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.824292 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/59e1467d-d303-4f32-a968-b66eb94953af-samples-operator-tls\") pod \"cluster-samples-operator-665b6dd947-7wkt6\" (UID: \"59e1467d-d303-4f32-a968-b66eb94953af\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-665b6dd947-7wkt6" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.824390 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etcd-client\" (UniqueName: \"kubernetes.io/secret/8fae1d72-fba4-4999-bea5-39d6472d7ea7-etcd-client\") pod \"apiserver-7bbb656c7d-86vx8\" (UID: \"8fae1d72-fba4-4999-bea5-39d6472d7ea7\") " pod="openshift-oauth-apiserver/apiserver-7bbb656c7d-86vx8" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.824490 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/353b0eac-95ba-420e-bf67-a26b472fcdc2-serving-cert\") pod \"route-controller-manager-6576b87f9c-9fpqn\" (UID: \"353b0eac-95ba-420e-bf67-a26b472fcdc2\") " pod="openshift-route-controller-manager/route-controller-manager-6576b87f9c-9fpqn" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.824621 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e0bf52c7-bf23-4c65-b7cb-0d8d466aa8f2-trusted-ca-bundle\") pod \"authentication-operator-69f744f599-qjq5f\" (UID: \"e0bf52c7-bf23-4c65-b7cb-0d8d466aa8f2\") " pod="openshift-authentication-operator/authentication-operator-69f744f599-qjq5f" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.824727 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"auth-proxy-config\" (UniqueName: \"kubernetes.io/configmap/cc99b3a0-fa04-4128-b730-1d24da63d08e-auth-proxy-config\") pod \"machine-config-operator-74547568cd-tr2xc\" (UID: \"cc99b3a0-fa04-4128-b730-1d24da63d08e\") " pod="openshift-machine-config-operator/machine-config-operator-74547568cd-tr2xc" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.824836 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"audit-policies\" (UniqueName: \"kubernetes.io/configmap/8fae1d72-fba4-4999-bea5-39d6472d7ea7-audit-policies\") pod \"apiserver-7bbb656c7d-86vx8\" (UID: \"8fae1d72-fba4-4999-bea5-39d6472d7ea7\") " pod="openshift-oauth-apiserver/apiserver-7bbb656c7d-86vx8" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.824934 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/833bb55c-5c9b-450c-9287-30f146a2d16a-bound-sa-token\") pod \"ingress-operator-5b745b69d9-hc8vv\" (UID: \"833bb55c-5c9b-450c-9287-30f146a2d16a\") " pod="openshift-ingress-operator/ingress-operator-5b745b69d9-hc8vv" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.825059 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"v4-0-config-user-template-login\" (UniqueName: \"kubernetes.io/secret/75f62784-7818-4a74-9c8c-4777d6db0611-v4-0-config-user-template-login\") pod \"oauth-openshift-558db77b4-99jsx\" (UID: \"75f62784-7818-4a74-9c8c-4777d6db0611\") " pod="openshift-authentication/oauth-openshift-558db77b4-99jsx" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.825155 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etcd-client\" (UniqueName: \"kubernetes.io/secret/41016a7b-cc79-4f32-860c-cb75e405cf42-etcd-client\") pod \"apiserver-76f77b778f-dmvst\" (UID: \"41016a7b-cc79-4f32-860c-cb75e405cf42\") " pod="openshift-apiserver/apiserver-76f77b778f-dmvst" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.825263 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-gz6dn\" (UniqueName: \"kubernetes.io/projected/efa98345-cafa-483a-976b-d772a3bd94e7-kube-api-access-gz6dn\") pod \"machine-config-controller-84d6567774-rxwdl\" (UID: \"efa98345-cafa-483a-976b-d772a3bd94e7\") " pod="openshift-machine-config-operator/machine-config-controller-84d6567774-rxwdl" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.825356 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6e31bff9-7ac9-47a7-a895-db2cc3051d85-serving-cert\") pod \"kube-storage-version-migrator-operator-b67b599dd-5h545\" (UID: \"6e31bff9-7ac9-47a7-a895-db2cc3051d85\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-b67b599dd-5h545" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.825449 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"v4-0-config-system-service-ca\" (UniqueName: \"kubernetes.io/configmap/75f62784-7818-4a74-9c8c-4777d6db0611-v4-0-config-system-service-ca\") pod \"oauth-openshift-558db77b4-99jsx\" (UID: \"75f62784-7818-4a74-9c8c-4777d6db0611\") " pod="openshift-authentication/oauth-openshift-558db77b4-99jsx" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.825565 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-ng77f\" (UniqueName: \"kubernetes.io/projected/e0bf52c7-bf23-4c65-b7cb-0d8d466aa8f2-kube-api-access-ng77f\") pod \"authentication-operator-69f744f599-qjq5f\" (UID: \"e0bf52c7-bf23-4c65-b7cb-0d8d466aa8f2\") " pod="openshift-authentication-operator/authentication-operator-69f744f599-qjq5f" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.825664 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"v4-0-config-system-router-certs\" (UniqueName: \"kubernetes.io/secret/75f62784-7818-4a74-9c8c-4777d6db0611-v4-0-config-system-router-certs\") pod \"oauth-openshift-558db77b4-99jsx\" (UID: \"75f62784-7818-4a74-9c8c-4777d6db0611\") " pod="openshift-authentication/oauth-openshift-558db77b4-99jsx" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.825799 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"v4-0-config-system-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/75f62784-7818-4a74-9c8c-4777d6db0611-v4-0-config-system-trusted-ca-bundle\") pod \"oauth-openshift-558db77b4-99jsx\" (UID: \"75f62784-7818-4a74-9c8c-4777d6db0611\") " pod="openshift-authentication/oauth-openshift-558db77b4-99jsx" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.825898 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/cc99b3a0-fa04-4128-b730-1d24da63d08e-proxy-tls\") pod \"machine-config-operator-74547568cd-tr2xc\" (UID: \"cc99b3a0-fa04-4128-b730-1d24da63d08e\") " pod="openshift-machine-config-operator/machine-config-operator-74547568cd-tr2xc" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.825997 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h4gph\" (UniqueName: \"kubernetes.io/projected/cc99b3a0-fa04-4128-b730-1d24da63d08e-kube-api-access-h4gph\") pod \"machine-config-operator-74547568cd-tr2xc\" (UID: \"cc99b3a0-fa04-4128-b730-1d24da63d08e\") " pod="openshift-machine-config-operator/machine-config-operator-74547568cd-tr2xc" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.826108 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/407f9c0b-88d2-46f9-9f47-d3af857b0f30-trusted-ca\") pod \"cluster-image-registry-operator-dc59b4c8b-4v96x\" (UID: \"407f9c0b-88d2-46f9-9f47-d3af857b0f30\") " pod="openshift-image-registry/cluster-image-registry-operator-dc59b4c8b-4v96x" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.826223 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/41016a7b-cc79-4f32-860c-cb75e405cf42-config\") pod \"apiserver-76f77b778f-dmvst\" (UID: \"41016a7b-cc79-4f32-860c-cb75e405cf42\") " pod="openshift-apiserver/apiserver-76f77b778f-dmvst" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.826323 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"marketplace-operator-metrics\" (UniqueName: \"kubernetes.io/secret/cac70dab-6e13-4693-ac66-c7552b62dfd7-marketplace-operator-metrics\") pod \"marketplace-operator-79b997595-kdr8l\" (UID: \"cac70dab-6e13-4693-ac66-c7552b62dfd7\") " pod="openshift-marketplace/marketplace-operator-79b997595-kdr8l" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.826442 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-policies\" (UniqueName: \"kubernetes.io/configmap/75f62784-7818-4a74-9c8c-4777d6db0611-audit-policies\") pod \"oauth-openshift-558db77b4-99jsx\" (UID: \"75f62784-7818-4a74-9c8c-4777d6db0611\") " pod="openshift-authentication/oauth-openshift-558db77b4-99jsx" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.826565 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f5012254-068c-4360-b310-39ff3d309199-installation-pull-secrets\") pod \"image-registry-697d97f7c8-p77bx\" (UID: \"f5012254-068c-4360-b310-39ff3d309199\") " pod="openshift-image-registry/image-registry-697d97f7c8-p77bx" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.826695 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"image-registry-operator-tls\" (UniqueName: \"kubernetes.io/secret/407f9c0b-88d2-46f9-9f47-d3af857b0f30-image-registry-operator-tls\") pod \"cluster-image-registry-operator-dc59b4c8b-4v96x\" (UID: \"407f9c0b-88d2-46f9-9f47-d3af857b0f30\") " pod="openshift-image-registry/cluster-image-registry-operator-dc59b4c8b-4v96x" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.826812 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cabc2063-04d9-4491-be9f-f7781b5f02cf-metrics-tls\") pod \"dns-operator-744455d44c-zvvfc\" (UID: \"cabc2063-04d9-4491-be9f-f7781b5f02cf\") " pod="openshift-dns-operator/dns-operator-744455d44c-zvvfc" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.826893 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"v4-0-config-user-template-provider-selection\" (UniqueName: \"kubernetes.io/secret/75f62784-7818-4a74-9c8c-4777d6db0611-v4-0-config-user-template-provider-selection\") pod \"oauth-openshift-558db77b4-99jsx\" (UID: \"75f62784-7818-4a74-9c8c-4777d6db0611\") " pod="openshift-authentication/oauth-openshift-558db77b4-99jsx" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.823824 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"audit-dir\" (UniqueName: \"kubernetes.io/host-path/41016a7b-cc79-4f32-860c-cb75e405cf42-audit-dir\") pod \"apiserver-76f77b778f-dmvst\" (UID: \"41016a7b-cc79-4f32-860c-cb75e405cf42\") " pod="openshift-apiserver/apiserver-76f77b778f-dmvst" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.827102 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e0bf52c7-bf23-4c65-b7cb-0d8d466aa8f2-service-ca-bundle\") pod \"authentication-operator-69f744f599-qjq5f\" (UID: \"e0bf52c7-bf23-4c65-b7cb-0d8d466aa8f2\") " pod="openshift-authentication-operator/authentication-operator-69f744f599-qjq5f" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.830147 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"encryption-config\" (UniqueName: \"kubernetes.io/secret/8fae1d72-fba4-4999-bea5-39d6472d7ea7-encryption-config\") pod \"apiserver-7bbb656c7d-86vx8\" (UID: \"8fae1d72-fba4-4999-bea5-39d6472d7ea7\") " pod="openshift-oauth-apiserver/apiserver-7bbb656c7d-86vx8" Dec 03 11:20:45 crc kubenswrapper[5052]: E1203 11:20:45.830279 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8 podName: nodeName:}" failed. No retries permitted until 2025-12-03 11:20:46.330152358 +0000 UTC m=+36.918394122 (durationBeforeRetry 500ms). Error: MountVolume.MountDevice failed for volume "pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8" (UniqueName: "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8") pod "image-registry-697d97f7c8-p77bx" (UID: "f5012254-068c-4360-b310-39ff3d309199") : kubernetes.io/csi: attacher.MountDevice failed to create newCsiDriverClient: driver name kubevirt.io.hostpath-provisioner not found in the list of registered CSI drivers Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.832034 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-controller-manager"/"openshift-controller-manager-sa-dockercfg-msq4c" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.832449 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e0bf52c7-bf23-4c65-b7cb-0d8d466aa8f2-trusted-ca-bundle\") pod \"authentication-operator-69f744f599-qjq5f\" (UID: \"e0bf52c7-bf23-4c65-b7cb-0d8d466aa8f2\") " pod="openshift-authentication-operator/authentication-operator-69f744f599-qjq5f" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.832632 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"mcc-auth-proxy-config\" (UniqueName: \"kubernetes.io/configmap/efa98345-cafa-483a-976b-d772a3bd94e7-mcc-auth-proxy-config\") pod \"machine-config-controller-84d6567774-rxwdl\" (UID: \"efa98345-cafa-483a-976b-d772a3bd94e7\") " pod="openshift-machine-config-operator/machine-config-controller-84d6567774-rxwdl" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.832905 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"node-pullsecrets\" (UniqueName: \"kubernetes.io/host-path/41016a7b-cc79-4f32-860c-cb75e405cf42-node-pullsecrets\") pod \"apiserver-76f77b778f-dmvst\" (UID: \"41016a7b-cc79-4f32-860c-cb75e405cf42\") " pod="openshift-apiserver/apiserver-76f77b778f-dmvst" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.833104 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"client-ca\" (UniqueName: \"kubernetes.io/configmap/353b0eac-95ba-420e-bf67-a26b472fcdc2-client-ca\") pod \"route-controller-manager-6576b87f9c-9fpqn\" (UID: \"353b0eac-95ba-420e-bf67-a26b472fcdc2\") " pod="openshift-route-controller-manager/route-controller-manager-6576b87f9c-9fpqn" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.833372 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-controller-manager"/"serving-cert" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.822128 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-6m2jn"] Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.826610 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/353b0eac-95ba-420e-bf67-a26b472fcdc2-config\") pod \"route-controller-manager-6576b87f9c-9fpqn\" (UID: \"353b0eac-95ba-420e-bf67-a26b472fcdc2\") " pod="openshift-route-controller-manager/route-controller-manager-6576b87f9c-9fpqn" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.837199 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8fae1d72-fba4-4999-bea5-39d6472d7ea7-trusted-ca-bundle\") pod \"apiserver-7bbb656c7d-86vx8\" (UID: \"8fae1d72-fba4-4999-bea5-39d6472d7ea7\") " pod="openshift-oauth-apiserver/apiserver-7bbb656c7d-86vx8" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.851291 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8fae1d72-fba4-4999-bea5-39d6472d7ea7-serving-cert\") pod \"apiserver-7bbb656c7d-86vx8\" (UID: \"8fae1d72-fba4-4999-bea5-39d6472d7ea7\") " pod="openshift-oauth-apiserver/apiserver-7bbb656c7d-86vx8" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.851412 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/41016a7b-cc79-4f32-860c-cb75e405cf42-trusted-ca-bundle\") pod \"apiserver-76f77b778f-dmvst\" (UID: \"41016a7b-cc79-4f32-860c-cb75e405cf42\") " pod="openshift-apiserver/apiserver-76f77b778f-dmvst" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.857787 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/833bb55c-5c9b-450c-9287-30f146a2d16a-metrics-tls\") pod \"ingress-operator-5b745b69d9-hc8vv\" (UID: \"833bb55c-5c9b-450c-9287-30f146a2d16a\") " pod="openshift-ingress-operator/ingress-operator-5b745b69d9-hc8vv" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.858301 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"audit\" (UniqueName: \"kubernetes.io/configmap/41016a7b-cc79-4f32-860c-cb75e405cf42-audit\") pod \"apiserver-76f77b778f-dmvst\" (UID: \"41016a7b-cc79-4f32-860c-cb75e405cf42\") " pod="openshift-apiserver/apiserver-76f77b778f-dmvst" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.860925 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"audit-dir\" (UniqueName: \"kubernetes.io/host-path/8fae1d72-fba4-4999-bea5-39d6472d7ea7-audit-dir\") pod \"apiserver-7bbb656c7d-86vx8\" (UID: \"8fae1d72-fba4-4999-bea5-39d6472d7ea7\") " pod="openshift-oauth-apiserver/apiserver-7bbb656c7d-86vx8" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.863914 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/833bb55c-5c9b-450c-9287-30f146a2d16a-trusted-ca\") pod \"ingress-operator-5b745b69d9-hc8vv\" (UID: \"833bb55c-5c9b-450c-9287-30f146a2d16a\") " pod="openshift-ingress-operator/ingress-operator-5b745b69d9-hc8vv" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.864859 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openshift-multus/cni-sysctl-allowlist-ds-8l9gm"] Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.865332 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-dns-operator/dns-operator-744455d44c-zvvfc"] Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.865365 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-665b6dd947-7wkt6"] Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.865378 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-authentication-operator/authentication-operator-69f744f599-qjq5f"] Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.865390 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-oauth-apiserver/apiserver-7bbb656c7d-86vx8"] Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.865401 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-apiserver/apiserver-76f77b778f-dmvst"] Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.865497 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/cni-sysctl-allowlist-ds-8l9gm" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.865735 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-6m2jn" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.866230 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"audit-policies\" (UniqueName: \"kubernetes.io/configmap/8fae1d72-fba4-4999-bea5-39d6472d7ea7-audit-policies\") pod \"apiserver-7bbb656c7d-86vx8\" (UID: \"8fae1d72-fba4-4999-bea5-39d6472d7ea7\") " pod="openshift-oauth-apiserver/apiserver-7bbb656c7d-86vx8" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.866418 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"etcd-client\" (UniqueName: \"kubernetes.io/secret/8fae1d72-fba4-4999-bea5-39d6472d7ea7-etcd-client\") pod \"apiserver-7bbb656c7d-86vx8\" (UID: \"8fae1d72-fba4-4999-bea5-39d6472d7ea7\") " pod="openshift-oauth-apiserver/apiserver-7bbb656c7d-86vx8" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.866802 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-cluster-machine-approver"/"machine-approver-sa-dockercfg-nl2j4" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.867022 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/407f9c0b-88d2-46f9-9f47-d3af857b0f30-trusted-ca\") pod \"cluster-image-registry-operator-dc59b4c8b-4v96x\" (UID: \"407f9c0b-88d2-46f9-9f47-d3af857b0f30\") " pod="openshift-image-registry/cluster-image-registry-operator-dc59b4c8b-4v96x" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.867042 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/353b0eac-95ba-420e-bf67-a26b472fcdc2-serving-cert\") pod \"route-controller-manager-6576b87f9c-9fpqn\" (UID: \"353b0eac-95ba-420e-bf67-a26b472fcdc2\") " pod="openshift-route-controller-manager/route-controller-manager-6576b87f9c-9fpqn" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.867175 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/59e1467d-d303-4f32-a968-b66eb94953af-samples-operator-tls\") pod \"cluster-samples-operator-665b6dd947-7wkt6\" (UID: \"59e1467d-d303-4f32-a968-b66eb94953af\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-665b6dd947-7wkt6" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.867280 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-tvr2s"] Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.867589 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/efa98345-cafa-483a-976b-d772a3bd94e7-proxy-tls\") pod \"machine-config-controller-84d6567774-rxwdl\" (UID: \"efa98345-cafa-483a-976b-d772a3bd94e7\") " pod="openshift-machine-config-operator/machine-config-controller-84d6567774-rxwdl" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.868134 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-tvr2s" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.869168 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["hostpath-provisioner/csi-hostpathplugin-5tlrr"] Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.870967 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/machine-config-server-kzd49"] Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.871579 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/machine-config-server-kzd49" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.871927 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="hostpath-provisioner/csi-hostpathplugin-5tlrr" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.875116 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/41016a7b-cc79-4f32-860c-cb75e405cf42-config\") pod \"apiserver-76f77b778f-dmvst\" (UID: \"41016a7b-cc79-4f32-860c-cb75e405cf42\") " pod="openshift-apiserver/apiserver-76f77b778f-dmvst" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.875477 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-controller-manager"/"config" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.875600 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cabc2063-04d9-4491-be9f-f7781b5f02cf-metrics-tls\") pod \"dns-operator-744455d44c-zvvfc\" (UID: \"cabc2063-04d9-4491-be9f-f7781b5f02cf\") " pod="openshift-dns-operator/dns-operator-744455d44c-zvvfc" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.875892 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"encryption-config\" (UniqueName: \"kubernetes.io/secret/41016a7b-cc79-4f32-860c-cb75e405cf42-encryption-config\") pod \"apiserver-76f77b778f-dmvst\" (UID: \"41016a7b-cc79-4f32-860c-cb75e405cf42\") " pod="openshift-apiserver/apiserver-76f77b778f-dmvst" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.879596 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-machine-config-operator/machine-config-controller-84d6567774-rxwdl"] Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.879708 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-697d97f7c8-p77bx"] Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.880127 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-b67b599dd-5h545"] Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.882389 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"image-registry-operator-tls\" (UniqueName: \"kubernetes.io/secret/407f9c0b-88d2-46f9-9f47-d3af857b0f30-image-registry-operator-tls\") pod \"cluster-image-registry-operator-dc59b4c8b-4v96x\" (UID: \"407f9c0b-88d2-46f9-9f47-d3af857b0f30\") " pod="openshift-image-registry/cluster-image-registry-operator-dc59b4c8b-4v96x" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.890472 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/cluster-image-registry-operator-dc59b4c8b-4v96x"] Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.891685 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-config-operator/openshift-config-operator-7777fb866f-bkztv"] Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.895413 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-kube-scheduler-operator/openshift-kube-scheduler-operator-5fdd9b5758-rxfb5"] Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.895480 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-7954f5f757-l7xtl"] Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.895493 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-operator/ingress-operator-5b745b69d9-hc8vv"] Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.898629 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-controller-manager"/"openshift-global-ca" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.900585 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-58897d9998-7vpns"] Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.901615 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-route-controller-manager/route-controller-manager-6576b87f9c-9fpqn"] Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.902490 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-controller-manager/controller-manager-879f6c89f-46rm8"] Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.904046 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-f9d7485db-568dj"] Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.908780 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-controller-manager-operator/openshift-controller-manager-operator-756b6f6bc6-v7lmf"] Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.908831 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-operator-lifecycle-manager/catalog-operator-68c6474976-rqhkg"] Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.908843 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-operator-lifecycle-manager/olm-operator-6b444d44fb-6lm8x"] Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.909983 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-controller-manager"/"kube-root-ca.crt" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.911561 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-kube-apiserver-operator/kube-apiserver-operator-766d6c64bb-xwlk8"] Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.913318 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-777779d784-9jvh6"] Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.915039 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-59844c95c7-4qts8"] Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.917046 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-machine-api/machine-api-operator-5694c8668f-l5lrg"] Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.919207 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-apiserver-operator/openshift-apiserver-operator-796bbdcf4f-jskzg"] Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.920245 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-authentication/oauth-openshift-558db77b4-99jsx"] Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.921315 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-machine-config-operator/machine-config-operator-74547568cd-tr2xc"] Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.922610 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-kube-controller-manager-operator/kube-controller-manager-operator-78b949d7b-h6b78"] Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.924031 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-multus/multus-admission-controller-857f4d67dd-q8n58"] Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.925478 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-operator-lifecycle-manager/package-server-manager-789f6589d5-577rn"] Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.926929 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-cluster-machine-approver"/"machine-approver-config" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.927393 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-etcd-operator/etcd-operator-b45778765-z2lp6"] Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.927469 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\") pod \"8f668bae-612b-4b75-9490-919e737c6a3b\" (UID: \"8f668bae-612b-4b75-9490-919e737c6a3b\") " Dec 03 11:20:45 crc kubenswrapper[5052]: E1203 11:20:45.927728 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8 podName:8f668bae-612b-4b75-9490-919e737c6a3b nodeName:}" failed. No retries permitted until 2025-12-03 11:20:46.427701946 +0000 UTC m=+37.015943710 (durationBeforeRetry 500ms). Error: UnmountVolume.TearDown failed for volume "pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8" (UniqueName: "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8") pod "8f668bae-612b-4b75-9490-919e737c6a3b" (UID: "8f668bae-612b-4b75-9490-919e737c6a3b") : kubernetes.io/csi: Unmounter.TearDownAt failed to get CSI client: driver name kubevirt.io.hostpath-provisioner not found in the list of registered CSI drivers Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.927783 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\") pod \"image-registry-697d97f7c8-p77bx\" (UID: \"f5012254-068c-4360-b310-39ff3d309199\") " pod="openshift-image-registry/image-registry-697d97f7c8-p77bx" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.927831 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/42cd7f06-f712-41e0-9fa5-c8ec9f7af997-config\") pod \"openshift-apiserver-operator-796bbdcf4f-jskzg\" (UID: \"42cd7f06-f712-41e0-9fa5-c8ec9f7af997\") " pod="openshift-apiserver-operator/openshift-apiserver-operator-796bbdcf4f-jskzg" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.927879 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c67q4\" (UniqueName: \"kubernetes.io/projected/3afb93c1-9cbc-4588-a3bf-2ae1e774a2d1-kube-api-access-c67q4\") pod \"console-operator-58897d9998-7vpns\" (UID: \"3afb93c1-9cbc-4588-a3bf-2ae1e774a2d1\") " pod="openshift-console-operator/console-operator-58897d9998-7vpns" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.927910 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ljgxw\" (UniqueName: \"kubernetes.io/projected/43fca84c-422e-4e3b-b3ca-b787a4e3f677-kube-api-access-ljgxw\") pod \"openshift-controller-manager-operator-756b6f6bc6-v7lmf\" (UID: \"43fca84c-422e-4e3b-b3ca-b787a4e3f677\") " pod="openshift-controller-manager-operator/openshift-controller-manager-operator-756b6f6bc6-v7lmf" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.927958 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"v4-0-config-system-serving-cert\" (UniqueName: \"kubernetes.io/secret/75f62784-7818-4a74-9c8c-4777d6db0611-v4-0-config-system-serving-cert\") pod \"oauth-openshift-558db77b4-99jsx\" (UID: \"75f62784-7818-4a74-9c8c-4777d6db0611\") " pod="openshift-authentication/oauth-openshift-558db77b4-99jsx" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.927997 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/71ea0624-9fb5-415a-96e1-d5ada83c0fe4-config\") pod \"etcd-operator-b45778765-z2lp6\" (UID: \"71ea0624-9fb5-415a-96e1-d5ada83c0fe4\") " pod="openshift-etcd-operator/etcd-operator-b45778765-z2lp6" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.928024 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/eb07cd6b-71c7-4dc7-b658-40f32c63f373-signing-key\") pod \"service-ca-9c57cc56f-w7g95\" (UID: \"eb07cd6b-71c7-4dc7-b658-40f32c63f373\") " pod="openshift-service-ca/service-ca-9c57cc56f-w7g95" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.928070 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/42cd7f06-f712-41e0-9fa5-c8ec9f7af997-serving-cert\") pod \"openshift-apiserver-operator-796bbdcf4f-jskzg\" (UID: \"42cd7f06-f712-41e0-9fa5-c8ec9f7af997\") " pod="openshift-apiserver-operator/openshift-apiserver-operator-796bbdcf4f-jskzg" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.928114 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"v4-0-config-system-session\" (UniqueName: \"kubernetes.io/secret/75f62784-7818-4a74-9c8c-4777d6db0611-v4-0-config-system-session\") pod \"oauth-openshift-558db77b4-99jsx\" (UID: \"75f62784-7818-4a74-9c8c-4777d6db0611\") " pod="openshift-authentication/oauth-openshift-558db77b4-99jsx" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.928146 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"srv-cert\" (UniqueName: \"kubernetes.io/secret/663b7426-fde0-4b59-b857-b434b4a88f4b-srv-cert\") pod \"olm-operator-6b444d44fb-6lm8x\" (UID: \"663b7426-fde0-4b59-b857-b434b4a88f4b\") " pod="openshift-operator-lifecycle-manager/olm-operator-6b444d44fb-6lm8x" Dec 03 11:20:45 crc kubenswrapper[5052]: E1203 11:20:45.928170 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8 podName: nodeName:}" failed. No retries permitted until 2025-12-03 11:20:46.428157449 +0000 UTC m=+37.016399213 (durationBeforeRetry 500ms). Error: MountVolume.MountDevice failed for volume "pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8" (UniqueName: "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8") pod "image-registry-697d97f7c8-p77bx" (UID: "f5012254-068c-4360-b310-39ff3d309199") : kubernetes.io/csi: attacher.MountDevice failed to create newCsiDriverClient: driver name kubevirt.io.hostpath-provisioner not found in the list of registered CSI drivers Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.928257 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f5012254-068c-4360-b310-39ff3d309199-trusted-ca\") pod \"image-registry-697d97f7c8-p77bx\" (UID: \"f5012254-068c-4360-b310-39ff3d309199\") " pod="openshift-image-registry/image-registry-697d97f7c8-p77bx" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.928301 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"v4-0-config-system-cliconfig\" (UniqueName: \"kubernetes.io/configmap/75f62784-7818-4a74-9c8c-4777d6db0611-v4-0-config-system-cliconfig\") pod \"oauth-openshift-558db77b4-99jsx\" (UID: \"75f62784-7818-4a74-9c8c-4777d6db0611\") " pod="openshift-authentication/oauth-openshift-558db77b4-99jsx" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.928330 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"v4-0-config-user-template-error\" (UniqueName: \"kubernetes.io/secret/75f62784-7818-4a74-9c8c-4777d6db0611-v4-0-config-user-template-error\") pod \"oauth-openshift-558db77b4-99jsx\" (UID: \"75f62784-7818-4a74-9c8c-4777d6db0611\") " pod="openshift-authentication/oauth-openshift-558db77b4-99jsx" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.928365 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mdldv\" (UniqueName: \"kubernetes.io/projected/8e3a111c-05ff-4f33-b5ba-81dae833b529-kube-api-access-mdldv\") pod \"catalog-operator-68c6474976-rqhkg\" (UID: \"8e3a111c-05ff-4f33-b5ba-81dae833b529\") " pod="openshift-operator-lifecycle-manager/catalog-operator-68c6474976-rqhkg" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.928391 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-volume\" (UniqueName: \"kubernetes.io/secret/e43f14fc-f3eb-4f58-b202-cdfc641ac08a-secret-volume\") pod \"collect-profiles-29412675-hzd4n\" (UID: \"e43f14fc-f3eb-4f58-b202-cdfc641ac08a\") " pod="openshift-operator-lifecycle-manager/collect-profiles-29412675-hzd4n" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.928440 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/71ea0624-9fb5-415a-96e1-d5ada83c0fe4-serving-cert\") pod \"etcd-operator-b45778765-z2lp6\" (UID: \"71ea0624-9fb5-415a-96e1-d5ada83c0fe4\") " pod="openshift-etcd-operator/etcd-operator-b45778765-z2lp6" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.928465 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3afb93c1-9cbc-4588-a3bf-2ae1e774a2d1-config\") pod \"console-operator-58897d9998-7vpns\" (UID: \"3afb93c1-9cbc-4588-a3bf-2ae1e774a2d1\") " pod="openshift-console-operator/console-operator-58897d9998-7vpns" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.928566 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f5012254-068c-4360-b310-39ff3d309199-registry-tls\") pod \"image-registry-697d97f7c8-p77bx\" (UID: \"f5012254-068c-4360-b310-39ff3d309199\") " pod="openshift-image-registry/image-registry-697d97f7c8-p77bx" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.928604 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0a110335-c2e5-40a2-b565-a4791146d53b-serving-cert\") pod \"openshift-config-operator-7777fb866f-bkztv\" (UID: \"0a110335-c2e5-40a2-b565-a4791146d53b\") " pod="openshift-config-operator/openshift-config-operator-7777fb866f-bkztv" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.928636 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"images\" (UniqueName: \"kubernetes.io/configmap/cc99b3a0-fa04-4128-b730-1d24da63d08e-images\") pod \"machine-config-operator-74547568cd-tr2xc\" (UID: \"cc99b3a0-fa04-4128-b730-1d24da63d08e\") " pod="openshift-machine-config-operator/machine-config-operator-74547568cd-tr2xc" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.928670 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"srv-cert\" (UniqueName: \"kubernetes.io/secret/8e3a111c-05ff-4f33-b5ba-81dae833b529-srv-cert\") pod \"catalog-operator-68c6474976-rqhkg\" (UID: \"8e3a111c-05ff-4f33-b5ba-81dae833b529\") " pod="openshift-operator-lifecycle-manager/catalog-operator-68c6474976-rqhkg" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.928700 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"audit-dir\" (UniqueName: \"kubernetes.io/host-path/75f62784-7818-4a74-9c8c-4777d6db0611-audit-dir\") pod \"oauth-openshift-558db77b4-99jsx\" (UID: \"75f62784-7818-4a74-9c8c-4777d6db0611\") " pod="openshift-authentication/oauth-openshift-558db77b4-99jsx" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.928732 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"profile-collector-cert\" (UniqueName: \"kubernetes.io/secret/8e3a111c-05ff-4f33-b5ba-81dae833b529-profile-collector-cert\") pod \"catalog-operator-68c6474976-rqhkg\" (UID: \"8e3a111c-05ff-4f33-b5ba-81dae833b529\") " pod="openshift-operator-lifecycle-manager/catalog-operator-68c6474976-rqhkg" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.928763 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hmnk9\" (UniqueName: \"kubernetes.io/projected/e43f14fc-f3eb-4f58-b202-cdfc641ac08a-kube-api-access-hmnk9\") pod \"collect-profiles-29412675-hzd4n\" (UID: \"e43f14fc-f3eb-4f58-b202-cdfc641ac08a\") " pod="openshift-operator-lifecycle-manager/collect-profiles-29412675-hzd4n" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.928802 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmpfs\" (UniqueName: \"kubernetes.io/empty-dir/6413eb91-7b61-4660-888c-14da224c7dfa-tmpfs\") pod \"packageserver-d55dfcdfc-7s97q\" (UID: \"6413eb91-7b61-4660-888c-14da224c7dfa\") " pod="openshift-operator-lifecycle-manager/packageserver-d55dfcdfc-7s97q" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.928837 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-znx74\" (UniqueName: \"kubernetes.io/projected/42cd7f06-f712-41e0-9fa5-c8ec9f7af997-kube-api-access-znx74\") pod \"openshift-apiserver-operator-796bbdcf4f-jskzg\" (UID: \"42cd7f06-f712-41e0-9fa5-c8ec9f7af997\") " pod="openshift-apiserver-operator/openshift-apiserver-operator-796bbdcf4f-jskzg" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.928866 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"machine-approver-tls\" (UniqueName: \"kubernetes.io/secret/8c1b25f5-2d98-4852-bc7a-d5cbfa5cb6aa-machine-approver-tls\") pod \"machine-approver-56656f9798-6t9dp\" (UID: \"8c1b25f5-2d98-4852-bc7a-d5cbfa5cb6aa\") " pod="openshift-cluster-machine-approver/machine-approver-56656f9798-6t9dp" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.928893 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"control-plane-machine-set-operator-tls\" (UniqueName: \"kubernetes.io/secret/693b46e6-42c3-4d07-bfbd-bd83d6982199-control-plane-machine-set-operator-tls\") pod \"control-plane-machine-set-operator-78cbb6b69f-p4b4l\" (UID: \"693b46e6-42c3-4d07-bfbd-bd83d6982199\") " pod="openshift-machine-api/control-plane-machine-set-operator-78cbb6b69f-p4b4l" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.928926 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6e31bff9-7ac9-47a7-a895-db2cc3051d85-serving-cert\") pod \"kube-storage-version-migrator-operator-b67b599dd-5h545\" (UID: \"6e31bff9-7ac9-47a7-a895-db2cc3051d85\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-b67b599dd-5h545" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.928962 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"v4-0-config-system-service-ca\" (UniqueName: \"kubernetes.io/configmap/75f62784-7818-4a74-9c8c-4777d6db0611-v4-0-config-system-service-ca\") pod \"oauth-openshift-558db77b4-99jsx\" (UID: \"75f62784-7818-4a74-9c8c-4777d6db0611\") " pod="openshift-authentication/oauth-openshift-558db77b4-99jsx" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.929026 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/287ad4d4-bf40-4cc4-9856-580b65a2e88d-config\") pod \"kube-apiserver-operator-766d6c64bb-xwlk8\" (UID: \"287ad4d4-bf40-4cc4-9856-580b65a2e88d\") " pod="openshift-kube-apiserver-operator/kube-apiserver-operator-766d6c64bb-xwlk8" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.929060 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access\" (UniqueName: \"kubernetes.io/projected/287ad4d4-bf40-4cc4-9856-580b65a2e88d-kube-api-access\") pod \"kube-apiserver-operator-766d6c64bb-xwlk8\" (UID: \"287ad4d4-bf40-4cc4-9856-580b65a2e88d\") " pod="openshift-kube-apiserver-operator/kube-apiserver-operator-766d6c64bb-xwlk8" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.929087 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/43fca84c-422e-4e3b-b3ca-b787a4e3f677-serving-cert\") pod \"openshift-controller-manager-operator-756b6f6bc6-v7lmf\" (UID: \"43fca84c-422e-4e3b-b3ca-b787a4e3f677\") " pod="openshift-controller-manager-operator/openshift-controller-manager-operator-756b6f6bc6-v7lmf" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.929116 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"profile-collector-cert\" (UniqueName: \"kubernetes.io/secret/663b7426-fde0-4b59-b857-b434b4a88f4b-profile-collector-cert\") pod \"olm-operator-6b444d44fb-6lm8x\" (UID: \"663b7426-fde0-4b59-b857-b434b4a88f4b\") " pod="openshift-operator-lifecycle-manager/olm-operator-6b444d44fb-6lm8x" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.929151 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"audit-policies\" (UniqueName: \"kubernetes.io/configmap/75f62784-7818-4a74-9c8c-4777d6db0611-audit-policies\") pod \"oauth-openshift-558db77b4-99jsx\" (UID: \"75f62784-7818-4a74-9c8c-4777d6db0611\") " pod="openshift-authentication/oauth-openshift-558db77b4-99jsx" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.929182 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6413eb91-7b61-4660-888c-14da224c7dfa-webhook-cert\") pod \"packageserver-d55dfcdfc-7s97q\" (UID: \"6413eb91-7b61-4660-888c-14da224c7dfa\") " pod="openshift-operator-lifecycle-manager/packageserver-d55dfcdfc-7s97q" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.929212 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/287ad4d4-bf40-4cc4-9856-580b65a2e88d-serving-cert\") pod \"kube-apiserver-operator-766d6c64bb-xwlk8\" (UID: \"287ad4d4-bf40-4cc4-9856-580b65a2e88d\") " pod="openshift-kube-apiserver-operator/kube-apiserver-operator-766d6c64bb-xwlk8" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.929276 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xxzql\" (UniqueName: \"kubernetes.io/projected/71ea0624-9fb5-415a-96e1-d5ada83c0fe4-kube-api-access-xxzql\") pod \"etcd-operator-b45778765-z2lp6\" (UID: \"71ea0624-9fb5-415a-96e1-d5ada83c0fe4\") " pod="openshift-etcd-operator/etcd-operator-b45778765-z2lp6" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.929349 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-xk4gt\" (UniqueName: \"kubernetes.io/projected/6e31bff9-7ac9-47a7-a895-db2cc3051d85-kube-api-access-xk4gt\") pod \"kube-storage-version-migrator-operator-b67b599dd-5h545\" (UID: \"6e31bff9-7ac9-47a7-a895-db2cc3051d85\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-b67b599dd-5h545" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.930145 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/48040744-7fc1-482e-9b68-3a904ba03a8d-default-certificate\") pod \"router-default-5444994796-rtl8t\" (UID: \"48040744-7fc1-482e-9b68-3a904ba03a8d\") " pod="openshift-ingress/router-default-5444994796-rtl8t" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.929333 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"audit-dir\" (UniqueName: \"kubernetes.io/host-path/75f62784-7818-4a74-9c8c-4777d6db0611-audit-dir\") pod \"oauth-openshift-558db77b4-99jsx\" (UID: \"75f62784-7818-4a74-9c8c-4777d6db0611\") " pod="openshift-authentication/oauth-openshift-558db77b4-99jsx" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.930217 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/48040744-7fc1-482e-9b68-3a904ba03a8d-metrics-certs\") pod \"router-default-5444994796-rtl8t\" (UID: \"48040744-7fc1-482e-9b68-3a904ba03a8d\") " pod="openshift-ingress/router-default-5444994796-rtl8t" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.930305 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zdbwv\" (UniqueName: \"kubernetes.io/projected/3e0657c6-3f3c-4300-8cfb-b3db773dba75-kube-api-access-zdbwv\") pod \"service-ca-operator-777779d784-9jvh6\" (UID: \"3e0657c6-3f3c-4300-8cfb-b3db773dba75\") " pod="openshift-service-ca-operator/service-ca-operator-777779d784-9jvh6" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.930583 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-88b82\" (UniqueName: \"kubernetes.io/projected/eb07cd6b-71c7-4dc7-b658-40f32c63f373-kube-api-access-88b82\") pod \"service-ca-9c57cc56f-w7g95\" (UID: \"eb07cd6b-71c7-4dc7-b658-40f32c63f373\") " pod="openshift-service-ca/service-ca-9c57cc56f-w7g95" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.930657 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3e0657c6-3f3c-4300-8cfb-b3db773dba75-config\") pod \"service-ca-operator-777779d784-9jvh6\" (UID: \"3e0657c6-3f3c-4300-8cfb-b3db773dba75\") " pod="openshift-service-ca-operator/service-ca-operator-777779d784-9jvh6" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.930038 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"v4-0-config-system-cliconfig\" (UniqueName: \"kubernetes.io/configmap/75f62784-7818-4a74-9c8c-4777d6db0611-v4-0-config-system-cliconfig\") pod \"oauth-openshift-558db77b4-99jsx\" (UID: \"75f62784-7818-4a74-9c8c-4777d6db0611\") " pod="openshift-authentication/oauth-openshift-558db77b4-99jsx" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.931194 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"images\" (UniqueName: \"kubernetes.io/configmap/cc99b3a0-fa04-4128-b730-1d24da63d08e-images\") pod \"machine-config-operator-74547568cd-tr2xc\" (UID: \"cc99b3a0-fa04-4128-b730-1d24da63d08e\") " pod="openshift-machine-config-operator/machine-config-operator-74547568cd-tr2xc" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.931913 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"v4-0-config-system-service-ca\" (UniqueName: \"kubernetes.io/configmap/75f62784-7818-4a74-9c8c-4777d6db0611-v4-0-config-system-service-ca\") pod \"oauth-openshift-558db77b4-99jsx\" (UID: \"75f62784-7818-4a74-9c8c-4777d6db0611\") " pod="openshift-authentication/oauth-openshift-558db77b4-99jsx" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.932029 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f5012254-068c-4360-b310-39ff3d309199-trusted-ca\") pod \"image-registry-697d97f7c8-p77bx\" (UID: \"f5012254-068c-4360-b310-39ff3d309199\") " pod="openshift-image-registry/image-registry-697d97f7c8-p77bx" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.932025 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"v4-0-config-system-session\" (UniqueName: \"kubernetes.io/secret/75f62784-7818-4a74-9c8c-4777d6db0611-v4-0-config-system-session\") pod \"oauth-openshift-558db77b4-99jsx\" (UID: \"75f62784-7818-4a74-9c8c-4777d6db0611\") " pod="openshift-authentication/oauth-openshift-558db77b4-99jsx" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.932413 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"audit-policies\" (UniqueName: \"kubernetes.io/configmap/75f62784-7818-4a74-9c8c-4777d6db0611-audit-policies\") pod \"oauth-openshift-558db77b4-99jsx\" (UID: \"75f62784-7818-4a74-9c8c-4777d6db0611\") " pod="openshift-authentication/oauth-openshift-558db77b4-99jsx" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.932430 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"v4-0-config-system-serving-cert\" (UniqueName: \"kubernetes.io/secret/75f62784-7818-4a74-9c8c-4777d6db0611-v4-0-config-system-serving-cert\") pod \"oauth-openshift-558db77b4-99jsx\" (UID: \"75f62784-7818-4a74-9c8c-4777d6db0611\") " pod="openshift-authentication/oauth-openshift-558db77b4-99jsx" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.933005 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f5012254-068c-4360-b310-39ff3d309199-registry-tls\") pod \"image-registry-697d97f7c8-p77bx\" (UID: \"f5012254-068c-4360-b310-39ff3d309199\") " pod="openshift-image-registry/image-registry-697d97f7c8-p77bx" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.933806 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6e31bff9-7ac9-47a7-a895-db2cc3051d85-serving-cert\") pod \"kube-storage-version-migrator-operator-b67b599dd-5h545\" (UID: \"6e31bff9-7ac9-47a7-a895-db2cc3051d85\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-b67b599dd-5h545" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.934144 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"v4-0-config-user-template-error\" (UniqueName: \"kubernetes.io/secret/75f62784-7818-4a74-9c8c-4777d6db0611-v4-0-config-user-template-error\") pod \"oauth-openshift-558db77b4-99jsx\" (UID: \"75f62784-7818-4a74-9c8c-4777d6db0611\") " pod="openshift-authentication/oauth-openshift-558db77b4-99jsx" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.938363 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/marketplace-operator-79b997595-kdr8l"] Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.939202 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0a110335-c2e5-40a2-b565-a4791146d53b-serving-cert\") pod \"openshift-config-operator-7777fb866f-bkztv\" (UID: \"0a110335-c2e5-40a2-b565-a4791146d53b\") " pod="openshift-config-operator/openshift-config-operator-7777fb866f-bkztv" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.939400 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"available-featuregates\" (UniqueName: \"kubernetes.io/empty-dir/0a110335-c2e5-40a2-b565-a4791146d53b-available-featuregates\") pod \"openshift-config-operator-7777fb866f-bkztv\" (UID: \"0a110335-c2e5-40a2-b565-a4791146d53b\") " pod="openshift-config-operator/openshift-config-operator-7777fb866f-bkztv" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.939474 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etcd-service-ca\" (UniqueName: \"kubernetes.io/configmap/71ea0624-9fb5-415a-96e1-d5ada83c0fe4-etcd-service-ca\") pod \"etcd-operator-b45778765-z2lp6\" (UID: \"71ea0624-9fb5-415a-96e1-d5ada83c0fe4\") " pod="openshift-etcd-operator/etcd-operator-b45778765-z2lp6" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.939546 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f5012254-068c-4360-b310-39ff3d309199-bound-sa-token\") pod \"image-registry-697d97f7c8-p77bx\" (UID: \"f5012254-068c-4360-b310-39ff3d309199\") " pod="openshift-image-registry/image-registry-697d97f7c8-p77bx" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.939600 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"v4-0-config-system-ocp-branding-template\" (UniqueName: \"kubernetes.io/secret/75f62784-7818-4a74-9c8c-4777d6db0611-v4-0-config-system-ocp-branding-template\") pod \"oauth-openshift-558db77b4-99jsx\" (UID: \"75f62784-7818-4a74-9c8c-4777d6db0611\") " pod="openshift-authentication/oauth-openshift-558db77b4-99jsx" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.939646 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/6e31bff9-7ac9-47a7-a895-db2cc3051d85-config\") pod \"kube-storage-version-migrator-operator-b67b599dd-5h545\" (UID: \"6e31bff9-7ac9-47a7-a895-db2cc3051d85\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-b67b599dd-5h545" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.939688 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-4cgvv\" (UniqueName: \"kubernetes.io/projected/cac70dab-6e13-4693-ac66-c7552b62dfd7-kube-api-access-4cgvv\") pod \"marketplace-operator-79b997595-kdr8l\" (UID: \"cac70dab-6e13-4693-ac66-c7552b62dfd7\") " pod="openshift-marketplace/marketplace-operator-79b997595-kdr8l" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.939723 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/48040744-7fc1-482e-9b68-3a904ba03a8d-service-ca-bundle\") pod \"router-default-5444994796-rtl8t\" (UID: \"48040744-7fc1-482e-9b68-3a904ba03a8d\") " pod="openshift-ingress/router-default-5444994796-rtl8t" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.939776 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f5012254-068c-4360-b310-39ff3d309199-registry-certificates\") pod \"image-registry-697d97f7c8-p77bx\" (UID: \"f5012254-068c-4360-b310-39ff3d309199\") " pod="openshift-image-registry/image-registry-697d97f7c8-p77bx" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.939807 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4q7pc\" (UniqueName: \"kubernetes.io/projected/693b46e6-42c3-4d07-bfbd-bd83d6982199-kube-api-access-4q7pc\") pod \"control-plane-machine-set-operator-78cbb6b69f-p4b4l\" (UID: \"693b46e6-42c3-4d07-bfbd-bd83d6982199\") " pod="openshift-machine-api/control-plane-machine-set-operator-78cbb6b69f-p4b4l" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.939865 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-ztr7w\" (UniqueName: \"kubernetes.io/projected/0a110335-c2e5-40a2-b565-a4791146d53b-kube-api-access-ztr7w\") pod \"openshift-config-operator-7777fb866f-bkztv\" (UID: \"0a110335-c2e5-40a2-b565-a4791146d53b\") " pod="openshift-config-operator/openshift-config-operator-7777fb866f-bkztv" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.939901 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3afb93c1-9cbc-4588-a3bf-2ae1e774a2d1-serving-cert\") pod \"console-operator-58897d9998-7vpns\" (UID: \"3afb93c1-9cbc-4588-a3bf-2ae1e774a2d1\") " pod="openshift-console-operator/console-operator-58897d9998-7vpns" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.940000 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"marketplace-trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cac70dab-6e13-4693-ac66-c7552b62dfd7-marketplace-trusted-ca\") pod \"marketplace-operator-79b997595-kdr8l\" (UID: \"cac70dab-6e13-4693-ac66-c7552b62dfd7\") " pod="openshift-marketplace/marketplace-operator-79b997595-kdr8l" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.940053 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"v4-0-config-user-idp-0-file-data\" (UniqueName: \"kubernetes.io/secret/75f62784-7818-4a74-9c8c-4777d6db0611-v4-0-config-user-idp-0-file-data\") pod \"oauth-openshift-558db77b4-99jsx\" (UID: \"75f62784-7818-4a74-9c8c-4777d6db0611\") " pod="openshift-authentication/oauth-openshift-558db77b4-99jsx" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.940093 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5s2zd\" (UniqueName: \"kubernetes.io/projected/663b7426-fde0-4b59-b857-b434b4a88f4b-kube-api-access-5s2zd\") pod \"olm-operator-6b444d44fb-6lm8x\" (UID: \"663b7426-fde0-4b59-b857-b434b4a88f4b\") " pod="openshift-operator-lifecycle-manager/olm-operator-6b444d44fb-6lm8x" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.940130 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-cvfz9\" (UniqueName: \"kubernetes.io/projected/f5012254-068c-4360-b310-39ff3d309199-kube-api-access-cvfz9\") pod \"image-registry-697d97f7c8-p77bx\" (UID: \"f5012254-068c-4360-b310-39ff3d309199\") " pod="openshift-image-registry/image-registry-697d97f7c8-p77bx" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.940301 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/43fca84c-422e-4e3b-b3ca-b787a4e3f677-config\") pod \"openshift-controller-manager-operator-756b6f6bc6-v7lmf\" (UID: \"43fca84c-422e-4e3b-b3ca-b787a4e3f677\") " pod="openshift-controller-manager-operator/openshift-controller-manager-operator-756b6f6bc6-v7lmf" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.940356 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etcd-client\" (UniqueName: \"kubernetes.io/secret/71ea0624-9fb5-415a-96e1-d5ada83c0fe4-etcd-client\") pod \"etcd-operator-b45778765-z2lp6\" (UID: \"71ea0624-9fb5-415a-96e1-d5ada83c0fe4\") " pod="openshift-etcd-operator/etcd-operator-b45778765-z2lp6" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.940393 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"auth-proxy-config\" (UniqueName: \"kubernetes.io/configmap/cc99b3a0-fa04-4128-b730-1d24da63d08e-auth-proxy-config\") pod \"machine-config-operator-74547568cd-tr2xc\" (UID: \"cc99b3a0-fa04-4128-b730-1d24da63d08e\") " pod="openshift-machine-config-operator/machine-config-operator-74547568cd-tr2xc" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.940471 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3afb93c1-9cbc-4588-a3bf-2ae1e774a2d1-trusted-ca\") pod \"console-operator-58897d9998-7vpns\" (UID: \"3afb93c1-9cbc-4588-a3bf-2ae1e774a2d1\") " pod="openshift-console-operator/console-operator-58897d9998-7vpns" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.940586 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vkn2m\" (UniqueName: \"kubernetes.io/projected/48040744-7fc1-482e-9b68-3a904ba03a8d-kube-api-access-vkn2m\") pod \"router-default-5444994796-rtl8t\" (UID: \"48040744-7fc1-482e-9b68-3a904ba03a8d\") " pod="openshift-ingress/router-default-5444994796-rtl8t" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.940638 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8c1b25f5-2d98-4852-bc7a-d5cbfa5cb6aa-config\") pod \"machine-approver-56656f9798-6t9dp\" (UID: \"8c1b25f5-2d98-4852-bc7a-d5cbfa5cb6aa\") " pod="openshift-cluster-machine-approver/machine-approver-56656f9798-6t9dp" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.940669 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etcd-ca\" (UniqueName: \"kubernetes.io/configmap/71ea0624-9fb5-415a-96e1-d5ada83c0fe4-etcd-ca\") pod \"etcd-operator-b45778765-z2lp6\" (UID: \"71ea0624-9fb5-415a-96e1-d5ada83c0fe4\") " pod="openshift-etcd-operator/etcd-operator-b45778765-z2lp6" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.940700 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"v4-0-config-user-template-login\" (UniqueName: \"kubernetes.io/secret/75f62784-7818-4a74-9c8c-4777d6db0611-v4-0-config-user-template-login\") pod \"oauth-openshift-558db77b4-99jsx\" (UID: \"75f62784-7818-4a74-9c8c-4777d6db0611\") " pod="openshift-authentication/oauth-openshift-558db77b4-99jsx" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.940955 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qpvck\" (UniqueName: \"kubernetes.io/projected/6413eb91-7b61-4660-888c-14da224c7dfa-kube-api-access-qpvck\") pod \"packageserver-d55dfcdfc-7s97q\" (UID: \"6413eb91-7b61-4660-888c-14da224c7dfa\") " pod="openshift-operator-lifecycle-manager/packageserver-d55dfcdfc-7s97q" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.941084 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"auth-proxy-config\" (UniqueName: \"kubernetes.io/configmap/8c1b25f5-2d98-4852-bc7a-d5cbfa5cb6aa-auth-proxy-config\") pod \"machine-approver-56656f9798-6t9dp\" (UID: \"8c1b25f5-2d98-4852-bc7a-d5cbfa5cb6aa\") " pod="openshift-cluster-machine-approver/machine-approver-56656f9798-6t9dp" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.941151 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3e0657c6-3f3c-4300-8cfb-b3db773dba75-serving-cert\") pod \"service-ca-operator-777779d784-9jvh6\" (UID: \"3e0657c6-3f3c-4300-8cfb-b3db773dba75\") " pod="openshift-service-ca-operator/service-ca-operator-777779d784-9jvh6" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.941192 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/6413eb91-7b61-4660-888c-14da224c7dfa-apiservice-cert\") pod \"packageserver-d55dfcdfc-7s97q\" (UID: \"6413eb91-7b61-4660-888c-14da224c7dfa\") " pod="openshift-operator-lifecycle-manager/packageserver-d55dfcdfc-7s97q" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.941258 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"v4-0-config-system-router-certs\" (UniqueName: \"kubernetes.io/secret/75f62784-7818-4a74-9c8c-4777d6db0611-v4-0-config-system-router-certs\") pod \"oauth-openshift-558db77b4-99jsx\" (UID: \"75f62784-7818-4a74-9c8c-4777d6db0611\") " pod="openshift-authentication/oauth-openshift-558db77b4-99jsx" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.941349 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"v4-0-config-system-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/75f62784-7818-4a74-9c8c-4777d6db0611-v4-0-config-system-trusted-ca-bundle\") pod \"oauth-openshift-558db77b4-99jsx\" (UID: \"75f62784-7818-4a74-9c8c-4777d6db0611\") " pod="openshift-authentication/oauth-openshift-558db77b4-99jsx" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.941395 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/cc99b3a0-fa04-4128-b730-1d24da63d08e-proxy-tls\") pod \"machine-config-operator-74547568cd-tr2xc\" (UID: \"cc99b3a0-fa04-4128-b730-1d24da63d08e\") " pod="openshift-machine-config-operator/machine-config-operator-74547568cd-tr2xc" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.941452 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/eb07cd6b-71c7-4dc7-b658-40f32c63f373-signing-cabundle\") pod \"service-ca-9c57cc56f-w7g95\" (UID: \"eb07cd6b-71c7-4dc7-b658-40f32c63f373\") " pod="openshift-service-ca/service-ca-9c57cc56f-w7g95" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.941635 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-h4gph\" (UniqueName: \"kubernetes.io/projected/cc99b3a0-fa04-4128-b730-1d24da63d08e-kube-api-access-h4gph\") pod \"machine-config-operator-74547568cd-tr2xc\" (UID: \"cc99b3a0-fa04-4128-b730-1d24da63d08e\") " pod="openshift-machine-config-operator/machine-config-operator-74547568cd-tr2xc" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.941686 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"marketplace-operator-metrics\" (UniqueName: \"kubernetes.io/secret/cac70dab-6e13-4693-ac66-c7552b62dfd7-marketplace-operator-metrics\") pod \"marketplace-operator-79b997595-kdr8l\" (UID: \"cac70dab-6e13-4693-ac66-c7552b62dfd7\") " pod="openshift-marketplace/marketplace-operator-79b997595-kdr8l" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.941743 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xkxbr\" (UniqueName: \"kubernetes.io/projected/ec2ea657-018b-4626-bf79-5c818f573a4b-kube-api-access-xkxbr\") pod \"downloads-7954f5f757-l7xtl\" (UID: \"ec2ea657-018b-4626-bf79-5c818f573a4b\") " pod="openshift-console/downloads-7954f5f757-l7xtl" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.941792 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"available-featuregates\" (UniqueName: \"kubernetes.io/empty-dir/0a110335-c2e5-40a2-b565-a4791146d53b-available-featuregates\") pod \"openshift-config-operator-7777fb866f-bkztv\" (UID: \"0a110335-c2e5-40a2-b565-a4791146d53b\") " pod="openshift-config-operator/openshift-config-operator-7777fb866f-bkztv" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.944583 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/6e31bff9-7ac9-47a7-a895-db2cc3051d85-config\") pod \"kube-storage-version-migrator-operator-b67b599dd-5h545\" (UID: \"6e31bff9-7ac9-47a7-a895-db2cc3051d85\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-b67b599dd-5h545" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.945747 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"v4-0-config-system-router-certs\" (UniqueName: \"kubernetes.io/secret/75f62784-7818-4a74-9c8c-4777d6db0611-v4-0-config-system-router-certs\") pod \"oauth-openshift-558db77b4-99jsx\" (UID: \"75f62784-7818-4a74-9c8c-4777d6db0611\") " pod="openshift-authentication/oauth-openshift-558db77b4-99jsx" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.946034 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"v4-0-config-system-ocp-branding-template\" (UniqueName: \"kubernetes.io/secret/75f62784-7818-4a74-9c8c-4777d6db0611-v4-0-config-system-ocp-branding-template\") pod \"oauth-openshift-558db77b4-99jsx\" (UID: \"75f62784-7818-4a74-9c8c-4777d6db0611\") " pod="openshift-authentication/oauth-openshift-558db77b4-99jsx" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.946271 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-operator-lifecycle-manager/packageserver-d55dfcdfc-7s97q"] Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.941808 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/e43f14fc-f3eb-4f58-b202-cdfc641ac08a-config-volume\") pod \"collect-profiles-29412675-hzd4n\" (UID: \"e43f14fc-f3eb-4f58-b202-cdfc641ac08a\") " pod="openshift-operator-lifecycle-manager/collect-profiles-29412675-hzd4n" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.948335 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"v4-0-config-system-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/75f62784-7818-4a74-9c8c-4777d6db0611-v4-0-config-system-trusted-ca-bundle\") pod \"oauth-openshift-558db77b4-99jsx\" (UID: \"75f62784-7818-4a74-9c8c-4777d6db0611\") " pod="openshift-authentication/oauth-openshift-558db77b4-99jsx" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.948459 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f5012254-068c-4360-b310-39ff3d309199-installation-pull-secrets\") pod \"image-registry-697d97f7c8-p77bx\" (UID: \"f5012254-068c-4360-b310-39ff3d309199\") " pod="openshift-image-registry/image-registry-697d97f7c8-p77bx" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.948519 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"v4-0-config-user-template-provider-selection\" (UniqueName: \"kubernetes.io/secret/75f62784-7818-4a74-9c8c-4777d6db0611-v4-0-config-user-template-provider-selection\") pod \"oauth-openshift-558db77b4-99jsx\" (UID: \"75f62784-7818-4a74-9c8c-4777d6db0611\") " pod="openshift-authentication/oauth-openshift-558db77b4-99jsx" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.948618 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7tbg4\" (UniqueName: \"kubernetes.io/projected/8c1b25f5-2d98-4852-bc7a-d5cbfa5cb6aa-kube-api-access-7tbg4\") pod \"machine-approver-56656f9798-6t9dp\" (UID: \"8c1b25f5-2d98-4852-bc7a-d5cbfa5cb6aa\") " pod="openshift-cluster-machine-approver/machine-approver-56656f9798-6t9dp" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.948698 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f5012254-068c-4360-b310-39ff3d309199-ca-trust-extracted\") pod \"image-registry-697d97f7c8-p77bx\" (UID: \"f5012254-068c-4360-b310-39ff3d309199\") " pod="openshift-image-registry/image-registry-697d97f7c8-p77bx" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.948739 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/48040744-7fc1-482e-9b68-3a904ba03a8d-stats-auth\") pod \"router-default-5444994796-rtl8t\" (UID: \"48040744-7fc1-482e-9b68-3a904ba03a8d\") " pod="openshift-ingress/router-default-5444994796-rtl8t" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.948780 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-l5k2l\" (UniqueName: \"kubernetes.io/projected/75f62784-7818-4a74-9c8c-4777d6db0611-kube-api-access-l5k2l\") pod \"oauth-openshift-558db77b4-99jsx\" (UID: \"75f62784-7818-4a74-9c8c-4777d6db0611\") " pod="openshift-authentication/oauth-openshift-558db77b4-99jsx" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.949067 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"auth-proxy-config\" (UniqueName: \"kubernetes.io/configmap/cc99b3a0-fa04-4128-b730-1d24da63d08e-auth-proxy-config\") pod \"machine-config-operator-74547568cd-tr2xc\" (UID: \"cc99b3a0-fa04-4128-b730-1d24da63d08e\") " pod="openshift-machine-config-operator/machine-config-operator-74547568cd-tr2xc" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.949142 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-9c57cc56f-w7g95"] Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.951282 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f5012254-068c-4360-b310-39ff3d309199-registry-certificates\") pod \"image-registry-697d97f7c8-p77bx\" (UID: \"f5012254-068c-4360-b310-39ff3d309199\") " pod="openshift-image-registry/image-registry-697d97f7c8-p77bx" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.952399 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-6m2jn"] Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.953040 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"marketplace-operator-metrics\" (UniqueName: \"kubernetes.io/secret/cac70dab-6e13-4693-ac66-c7552b62dfd7-marketplace-operator-metrics\") pod \"marketplace-operator-79b997595-kdr8l\" (UID: \"cac70dab-6e13-4693-ac66-c7552b62dfd7\") " pod="openshift-marketplace/marketplace-operator-79b997595-kdr8l" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.955814 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f5012254-068c-4360-b310-39ff3d309199-ca-trust-extracted\") pod \"image-registry-697d97f7c8-p77bx\" (UID: \"f5012254-068c-4360-b310-39ff3d309199\") " pod="openshift-image-registry/image-registry-697d97f7c8-p77bx" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.956638 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/cc99b3a0-fa04-4128-b730-1d24da63d08e-proxy-tls\") pod \"machine-config-operator-74547568cd-tr2xc\" (UID: \"cc99b3a0-fa04-4128-b730-1d24da63d08e\") " pod="openshift-machine-config-operator/machine-config-operator-74547568cd-tr2xc" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.957180 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-operator-lifecycle-manager/collect-profiles-29412675-hzd4n"] Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.957333 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"marketplace-trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cac70dab-6e13-4693-ac66-c7552b62dfd7-marketplace-trusted-ca\") pod \"marketplace-operator-79b997595-kdr8l\" (UID: \"cac70dab-6e13-4693-ac66-c7552b62dfd7\") " pod="openshift-marketplace/marketplace-operator-79b997595-kdr8l" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.957703 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"v4-0-config-user-template-login\" (UniqueName: \"kubernetes.io/secret/75f62784-7818-4a74-9c8c-4777d6db0611-v4-0-config-user-template-login\") pod \"oauth-openshift-558db77b4-99jsx\" (UID: \"75f62784-7818-4a74-9c8c-4777d6db0611\") " pod="openshift-authentication/oauth-openshift-558db77b4-99jsx" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.959006 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-cluster-machine-approver"/"machine-approver-tls" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.962420 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"v4-0-config-user-idp-0-file-data\" (UniqueName: \"kubernetes.io/secret/75f62784-7818-4a74-9c8c-4777d6db0611-v4-0-config-user-idp-0-file-data\") pod \"oauth-openshift-558db77b4-99jsx\" (UID: \"75f62784-7818-4a74-9c8c-4777d6db0611\") " pod="openshift-authentication/oauth-openshift-558db77b4-99jsx" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.965413 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["hostpath-provisioner/csi-hostpathplugin-5tlrr"] Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.966888 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-cluster-machine-approver"/"kube-root-ca.crt" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.967576 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-machine-api/control-plane-machine-set-operator-78cbb6b69f-p4b4l"] Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.969466 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-tvr2s"] Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.970070 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"v4-0-config-user-template-provider-selection\" (UniqueName: \"kubernetes.io/secret/75f62784-7818-4a74-9c8c-4777d6db0611-v4-0-config-user-template-provider-selection\") pod \"oauth-openshift-558db77b4-99jsx\" (UID: \"75f62784-7818-4a74-9c8c-4777d6db0611\") " pod="openshift-authentication/oauth-openshift-558db77b4-99jsx" Dec 03 11:20:45 crc kubenswrapper[5052]: I1203 11:20:45.984426 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-cluster-machine-approver"/"openshift-service-ca.crt" Dec 03 11:20:46 crc kubenswrapper[5052]: I1203 11:20:46.003342 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-controller-manager"/"client-ca" Dec 03 11:20:46 crc kubenswrapper[5052]: I1203 11:20:46.023559 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-controller-manager-operator"/"openshift-service-ca.crt" Dec 03 11:20:46 crc kubenswrapper[5052]: I1203 11:20:46.049621 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\") pod \"8f668bae-612b-4b75-9490-919e737c6a3b\" (UID: \"8f668bae-612b-4b75-9490-919e737c6a3b\") " Dec 03 11:20:46 crc kubenswrapper[5052]: I1203 11:20:46.049895 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-qpvck\" (UniqueName: \"kubernetes.io/projected/6413eb91-7b61-4660-888c-14da224c7dfa-kube-api-access-qpvck\") pod \"packageserver-d55dfcdfc-7s97q\" (UID: \"6413eb91-7b61-4660-888c-14da224c7dfa\") " pod="openshift-operator-lifecycle-manager/packageserver-d55dfcdfc-7s97q" Dec 03 11:20:46 crc kubenswrapper[5052]: I1203 11:20:46.049959 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"auth-proxy-config\" (UniqueName: \"kubernetes.io/configmap/8c1b25f5-2d98-4852-bc7a-d5cbfa5cb6aa-auth-proxy-config\") pod \"machine-approver-56656f9798-6t9dp\" (UID: \"8c1b25f5-2d98-4852-bc7a-d5cbfa5cb6aa\") " pod="openshift-cluster-machine-approver/machine-approver-56656f9798-6t9dp" Dec 03 11:20:46 crc kubenswrapper[5052]: I1203 11:20:46.049992 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3e0657c6-3f3c-4300-8cfb-b3db773dba75-serving-cert\") pod \"service-ca-operator-777779d784-9jvh6\" (UID: \"3e0657c6-3f3c-4300-8cfb-b3db773dba75\") " pod="openshift-service-ca-operator/service-ca-operator-777779d784-9jvh6" Dec 03 11:20:46 crc kubenswrapper[5052]: I1203 11:20:46.050027 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/6413eb91-7b61-4660-888c-14da224c7dfa-apiservice-cert\") pod \"packageserver-d55dfcdfc-7s97q\" (UID: \"6413eb91-7b61-4660-888c-14da224c7dfa\") " pod="openshift-operator-lifecycle-manager/packageserver-d55dfcdfc-7s97q" Dec 03 11:20:46 crc kubenswrapper[5052]: I1203 11:20:46.050060 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/eb07cd6b-71c7-4dc7-b658-40f32c63f373-signing-cabundle\") pod \"service-ca-9c57cc56f-w7g95\" (UID: \"eb07cd6b-71c7-4dc7-b658-40f32c63f373\") " pod="openshift-service-ca/service-ca-9c57cc56f-w7g95" Dec 03 11:20:46 crc kubenswrapper[5052]: I1203 11:20:46.050098 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-xkxbr\" (UniqueName: \"kubernetes.io/projected/ec2ea657-018b-4626-bf79-5c818f573a4b-kube-api-access-xkxbr\") pod \"downloads-7954f5f757-l7xtl\" (UID: \"ec2ea657-018b-4626-bf79-5c818f573a4b\") " pod="openshift-console/downloads-7954f5f757-l7xtl" Dec 03 11:20:46 crc kubenswrapper[5052]: I1203 11:20:46.050122 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/e43f14fc-f3eb-4f58-b202-cdfc641ac08a-config-volume\") pod \"collect-profiles-29412675-hzd4n\" (UID: \"e43f14fc-f3eb-4f58-b202-cdfc641ac08a\") " pod="openshift-operator-lifecycle-manager/collect-profiles-29412675-hzd4n" Dec 03 11:20:46 crc kubenswrapper[5052]: I1203 11:20:46.050144 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-7tbg4\" (UniqueName: \"kubernetes.io/projected/8c1b25f5-2d98-4852-bc7a-d5cbfa5cb6aa-kube-api-access-7tbg4\") pod \"machine-approver-56656f9798-6t9dp\" (UID: \"8c1b25f5-2d98-4852-bc7a-d5cbfa5cb6aa\") " pod="openshift-cluster-machine-approver/machine-approver-56656f9798-6t9dp" Dec 03 11:20:46 crc kubenswrapper[5052]: I1203 11:20:46.050182 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/48040744-7fc1-482e-9b68-3a904ba03a8d-stats-auth\") pod \"router-default-5444994796-rtl8t\" (UID: \"48040744-7fc1-482e-9b68-3a904ba03a8d\") " pod="openshift-ingress/router-default-5444994796-rtl8t" Dec 03 11:20:46 crc kubenswrapper[5052]: I1203 11:20:46.050218 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/42cd7f06-f712-41e0-9fa5-c8ec9f7af997-config\") pod \"openshift-apiserver-operator-796bbdcf4f-jskzg\" (UID: \"42cd7f06-f712-41e0-9fa5-c8ec9f7af997\") " pod="openshift-apiserver-operator/openshift-apiserver-operator-796bbdcf4f-jskzg" Dec 03 11:20:46 crc kubenswrapper[5052]: I1203 11:20:46.050260 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-c67q4\" (UniqueName: \"kubernetes.io/projected/3afb93c1-9cbc-4588-a3bf-2ae1e774a2d1-kube-api-access-c67q4\") pod \"console-operator-58897d9998-7vpns\" (UID: \"3afb93c1-9cbc-4588-a3bf-2ae1e774a2d1\") " pod="openshift-console-operator/console-operator-58897d9998-7vpns" Dec 03 11:20:46 crc kubenswrapper[5052]: I1203 11:20:46.050280 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-ljgxw\" (UniqueName: \"kubernetes.io/projected/43fca84c-422e-4e3b-b3ca-b787a4e3f677-kube-api-access-ljgxw\") pod \"openshift-controller-manager-operator-756b6f6bc6-v7lmf\" (UID: \"43fca84c-422e-4e3b-b3ca-b787a4e3f677\") " pod="openshift-controller-manager-operator/openshift-controller-manager-operator-756b6f6bc6-v7lmf" Dec 03 11:20:46 crc kubenswrapper[5052]: I1203 11:20:46.050300 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/eb07cd6b-71c7-4dc7-b658-40f32c63f373-signing-key\") pod \"service-ca-9c57cc56f-w7g95\" (UID: \"eb07cd6b-71c7-4dc7-b658-40f32c63f373\") " pod="openshift-service-ca/service-ca-9c57cc56f-w7g95" Dec 03 11:20:46 crc kubenswrapper[5052]: I1203 11:20:46.050350 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/71ea0624-9fb5-415a-96e1-d5ada83c0fe4-config\") pod \"etcd-operator-b45778765-z2lp6\" (UID: \"71ea0624-9fb5-415a-96e1-d5ada83c0fe4\") " pod="openshift-etcd-operator/etcd-operator-b45778765-z2lp6" Dec 03 11:20:46 crc kubenswrapper[5052]: I1203 11:20:46.050111 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-console-operator"/"trusted-ca" Dec 03 11:20:46 crc kubenswrapper[5052]: I1203 11:20:46.050501 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/42cd7f06-f712-41e0-9fa5-c8ec9f7af997-serving-cert\") pod \"openshift-apiserver-operator-796bbdcf4f-jskzg\" (UID: \"42cd7f06-f712-41e0-9fa5-c8ec9f7af997\") " pod="openshift-apiserver-operator/openshift-apiserver-operator-796bbdcf4f-jskzg" Dec 03 11:20:46 crc kubenswrapper[5052]: I1203 11:20:46.050591 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"srv-cert\" (UniqueName: \"kubernetes.io/secret/663b7426-fde0-4b59-b857-b434b4a88f4b-srv-cert\") pod \"olm-operator-6b444d44fb-6lm8x\" (UID: \"663b7426-fde0-4b59-b857-b434b4a88f4b\") " pod="openshift-operator-lifecycle-manager/olm-operator-6b444d44fb-6lm8x" Dec 03 11:20:46 crc kubenswrapper[5052]: I1203 11:20:46.050633 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-mdldv\" (UniqueName: \"kubernetes.io/projected/8e3a111c-05ff-4f33-b5ba-81dae833b529-kube-api-access-mdldv\") pod \"catalog-operator-68c6474976-rqhkg\" (UID: \"8e3a111c-05ff-4f33-b5ba-81dae833b529\") " pod="openshift-operator-lifecycle-manager/catalog-operator-68c6474976-rqhkg" Dec 03 11:20:46 crc kubenswrapper[5052]: I1203 11:20:46.050653 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"secret-volume\" (UniqueName: \"kubernetes.io/secret/e43f14fc-f3eb-4f58-b202-cdfc641ac08a-secret-volume\") pod \"collect-profiles-29412675-hzd4n\" (UID: \"e43f14fc-f3eb-4f58-b202-cdfc641ac08a\") " pod="openshift-operator-lifecycle-manager/collect-profiles-29412675-hzd4n" Dec 03 11:20:46 crc kubenswrapper[5052]: I1203 11:20:46.050699 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/71ea0624-9fb5-415a-96e1-d5ada83c0fe4-serving-cert\") pod \"etcd-operator-b45778765-z2lp6\" (UID: \"71ea0624-9fb5-415a-96e1-d5ada83c0fe4\") " pod="openshift-etcd-operator/etcd-operator-b45778765-z2lp6" Dec 03 11:20:46 crc kubenswrapper[5052]: I1203 11:20:46.050717 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3afb93c1-9cbc-4588-a3bf-2ae1e774a2d1-config\") pod \"console-operator-58897d9998-7vpns\" (UID: \"3afb93c1-9cbc-4588-a3bf-2ae1e774a2d1\") " pod="openshift-console-operator/console-operator-58897d9998-7vpns" Dec 03 11:20:46 crc kubenswrapper[5052]: I1203 11:20:46.050735 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"srv-cert\" (UniqueName: \"kubernetes.io/secret/8e3a111c-05ff-4f33-b5ba-81dae833b529-srv-cert\") pod \"catalog-operator-68c6474976-rqhkg\" (UID: \"8e3a111c-05ff-4f33-b5ba-81dae833b529\") " pod="openshift-operator-lifecycle-manager/catalog-operator-68c6474976-rqhkg" Dec 03 11:20:46 crc kubenswrapper[5052]: I1203 11:20:46.050781 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"profile-collector-cert\" (UniqueName: \"kubernetes.io/secret/8e3a111c-05ff-4f33-b5ba-81dae833b529-profile-collector-cert\") pod \"catalog-operator-68c6474976-rqhkg\" (UID: \"8e3a111c-05ff-4f33-b5ba-81dae833b529\") " pod="openshift-operator-lifecycle-manager/catalog-operator-68c6474976-rqhkg" Dec 03 11:20:46 crc kubenswrapper[5052]: I1203 11:20:46.050804 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-hmnk9\" (UniqueName: \"kubernetes.io/projected/e43f14fc-f3eb-4f58-b202-cdfc641ac08a-kube-api-access-hmnk9\") pod \"collect-profiles-29412675-hzd4n\" (UID: \"e43f14fc-f3eb-4f58-b202-cdfc641ac08a\") " pod="openshift-operator-lifecycle-manager/collect-profiles-29412675-hzd4n" Dec 03 11:20:46 crc kubenswrapper[5052]: I1203 11:20:46.050820 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"tmpfs\" (UniqueName: \"kubernetes.io/empty-dir/6413eb91-7b61-4660-888c-14da224c7dfa-tmpfs\") pod \"packageserver-d55dfcdfc-7s97q\" (UID: \"6413eb91-7b61-4660-888c-14da224c7dfa\") " pod="openshift-operator-lifecycle-manager/packageserver-d55dfcdfc-7s97q" Dec 03 11:20:46 crc kubenswrapper[5052]: I1203 11:20:46.050855 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-znx74\" (UniqueName: \"kubernetes.io/projected/42cd7f06-f712-41e0-9fa5-c8ec9f7af997-kube-api-access-znx74\") pod \"openshift-apiserver-operator-796bbdcf4f-jskzg\" (UID: \"42cd7f06-f712-41e0-9fa5-c8ec9f7af997\") " pod="openshift-apiserver-operator/openshift-apiserver-operator-796bbdcf4f-jskzg" Dec 03 11:20:46 crc kubenswrapper[5052]: I1203 11:20:46.050875 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"machine-approver-tls\" (UniqueName: \"kubernetes.io/secret/8c1b25f5-2d98-4852-bc7a-d5cbfa5cb6aa-machine-approver-tls\") pod \"machine-approver-56656f9798-6t9dp\" (UID: \"8c1b25f5-2d98-4852-bc7a-d5cbfa5cb6aa\") " pod="openshift-cluster-machine-approver/machine-approver-56656f9798-6t9dp" Dec 03 11:20:46 crc kubenswrapper[5052]: I1203 11:20:46.050898 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"control-plane-machine-set-operator-tls\" (UniqueName: \"kubernetes.io/secret/693b46e6-42c3-4d07-bfbd-bd83d6982199-control-plane-machine-set-operator-tls\") pod \"control-plane-machine-set-operator-78cbb6b69f-p4b4l\" (UID: \"693b46e6-42c3-4d07-bfbd-bd83d6982199\") " pod="openshift-machine-api/control-plane-machine-set-operator-78cbb6b69f-p4b4l" Dec 03 11:20:46 crc kubenswrapper[5052]: E1203 11:20:46.051127 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8 podName:8f668bae-612b-4b75-9490-919e737c6a3b nodeName:}" failed. No retries permitted until 2025-12-03 11:20:46.551041286 +0000 UTC m=+37.139283050 (durationBeforeRetry 500ms). Error: UnmountVolume.TearDown failed for volume "pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8" (UniqueName: "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8") pod "8f668bae-612b-4b75-9490-919e737c6a3b" (UID: "8f668bae-612b-4b75-9490-919e737c6a3b") : kubernetes.io/csi: Unmounter.TearDownAt failed to get CSI client: driver name kubevirt.io.hostpath-provisioner not found in the list of registered CSI drivers Dec 03 11:20:46 crc kubenswrapper[5052]: I1203 11:20:46.051150 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"auth-proxy-config\" (UniqueName: \"kubernetes.io/configmap/8c1b25f5-2d98-4852-bc7a-d5cbfa5cb6aa-auth-proxy-config\") pod \"machine-approver-56656f9798-6t9dp\" (UID: \"8c1b25f5-2d98-4852-bc7a-d5cbfa5cb6aa\") " pod="openshift-cluster-machine-approver/machine-approver-56656f9798-6t9dp" Dec 03 11:20:46 crc kubenswrapper[5052]: I1203 11:20:46.051243 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/287ad4d4-bf40-4cc4-9856-580b65a2e88d-config\") pod \"kube-apiserver-operator-766d6c64bb-xwlk8\" (UID: \"287ad4d4-bf40-4cc4-9856-580b65a2e88d\") " pod="openshift-kube-apiserver-operator/kube-apiserver-operator-766d6c64bb-xwlk8" Dec 03 11:20:46 crc kubenswrapper[5052]: I1203 11:20:46.051328 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/43fca84c-422e-4e3b-b3ca-b787a4e3f677-serving-cert\") pod \"openshift-controller-manager-operator-756b6f6bc6-v7lmf\" (UID: \"43fca84c-422e-4e3b-b3ca-b787a4e3f677\") " pod="openshift-controller-manager-operator/openshift-controller-manager-operator-756b6f6bc6-v7lmf" Dec 03 11:20:46 crc kubenswrapper[5052]: I1203 11:20:46.051358 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access\" (UniqueName: \"kubernetes.io/projected/287ad4d4-bf40-4cc4-9856-580b65a2e88d-kube-api-access\") pod \"kube-apiserver-operator-766d6c64bb-xwlk8\" (UID: \"287ad4d4-bf40-4cc4-9856-580b65a2e88d\") " pod="openshift-kube-apiserver-operator/kube-apiserver-operator-766d6c64bb-xwlk8" Dec 03 11:20:46 crc kubenswrapper[5052]: I1203 11:20:46.051382 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"profile-collector-cert\" (UniqueName: \"kubernetes.io/secret/663b7426-fde0-4b59-b857-b434b4a88f4b-profile-collector-cert\") pod \"olm-operator-6b444d44fb-6lm8x\" (UID: \"663b7426-fde0-4b59-b857-b434b4a88f4b\") " pod="openshift-operator-lifecycle-manager/olm-operator-6b444d44fb-6lm8x" Dec 03 11:20:46 crc kubenswrapper[5052]: I1203 11:20:46.051407 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6413eb91-7b61-4660-888c-14da224c7dfa-webhook-cert\") pod \"packageserver-d55dfcdfc-7s97q\" (UID: \"6413eb91-7b61-4660-888c-14da224c7dfa\") " pod="openshift-operator-lifecycle-manager/packageserver-d55dfcdfc-7s97q" Dec 03 11:20:46 crc kubenswrapper[5052]: I1203 11:20:46.051424 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/287ad4d4-bf40-4cc4-9856-580b65a2e88d-serving-cert\") pod \"kube-apiserver-operator-766d6c64bb-xwlk8\" (UID: \"287ad4d4-bf40-4cc4-9856-580b65a2e88d\") " pod="openshift-kube-apiserver-operator/kube-apiserver-operator-766d6c64bb-xwlk8" Dec 03 11:20:46 crc kubenswrapper[5052]: I1203 11:20:46.051447 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-xxzql\" (UniqueName: \"kubernetes.io/projected/71ea0624-9fb5-415a-96e1-d5ada83c0fe4-kube-api-access-xxzql\") pod \"etcd-operator-b45778765-z2lp6\" (UID: \"71ea0624-9fb5-415a-96e1-d5ada83c0fe4\") " pod="openshift-etcd-operator/etcd-operator-b45778765-z2lp6" Dec 03 11:20:46 crc kubenswrapper[5052]: I1203 11:20:46.051487 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/48040744-7fc1-482e-9b68-3a904ba03a8d-default-certificate\") pod \"router-default-5444994796-rtl8t\" (UID: \"48040744-7fc1-482e-9b68-3a904ba03a8d\") " pod="openshift-ingress/router-default-5444994796-rtl8t" Dec 03 11:20:46 crc kubenswrapper[5052]: I1203 11:20:46.051519 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/48040744-7fc1-482e-9b68-3a904ba03a8d-metrics-certs\") pod \"router-default-5444994796-rtl8t\" (UID: \"48040744-7fc1-482e-9b68-3a904ba03a8d\") " pod="openshift-ingress/router-default-5444994796-rtl8t" Dec 03 11:20:46 crc kubenswrapper[5052]: I1203 11:20:46.051561 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-zdbwv\" (UniqueName: \"kubernetes.io/projected/3e0657c6-3f3c-4300-8cfb-b3db773dba75-kube-api-access-zdbwv\") pod \"service-ca-operator-777779d784-9jvh6\" (UID: \"3e0657c6-3f3c-4300-8cfb-b3db773dba75\") " pod="openshift-service-ca-operator/service-ca-operator-777779d784-9jvh6" Dec 03 11:20:46 crc kubenswrapper[5052]: I1203 11:20:46.051578 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-88b82\" (UniqueName: \"kubernetes.io/projected/eb07cd6b-71c7-4dc7-b658-40f32c63f373-kube-api-access-88b82\") pod \"service-ca-9c57cc56f-w7g95\" (UID: \"eb07cd6b-71c7-4dc7-b658-40f32c63f373\") " pod="openshift-service-ca/service-ca-9c57cc56f-w7g95" Dec 03 11:20:46 crc kubenswrapper[5052]: I1203 11:20:46.051602 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3e0657c6-3f3c-4300-8cfb-b3db773dba75-config\") pod \"service-ca-operator-777779d784-9jvh6\" (UID: \"3e0657c6-3f3c-4300-8cfb-b3db773dba75\") " pod="openshift-service-ca-operator/service-ca-operator-777779d784-9jvh6" Dec 03 11:20:46 crc kubenswrapper[5052]: I1203 11:20:46.051641 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etcd-service-ca\" (UniqueName: \"kubernetes.io/configmap/71ea0624-9fb5-415a-96e1-d5ada83c0fe4-etcd-service-ca\") pod \"etcd-operator-b45778765-z2lp6\" (UID: \"71ea0624-9fb5-415a-96e1-d5ada83c0fe4\") " pod="openshift-etcd-operator/etcd-operator-b45778765-z2lp6" Dec 03 11:20:46 crc kubenswrapper[5052]: I1203 11:20:46.051687 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/48040744-7fc1-482e-9b68-3a904ba03a8d-service-ca-bundle\") pod \"router-default-5444994796-rtl8t\" (UID: \"48040744-7fc1-482e-9b68-3a904ba03a8d\") " pod="openshift-ingress/router-default-5444994796-rtl8t" Dec 03 11:20:46 crc kubenswrapper[5052]: I1203 11:20:46.051707 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-4q7pc\" (UniqueName: \"kubernetes.io/projected/693b46e6-42c3-4d07-bfbd-bd83d6982199-kube-api-access-4q7pc\") pod \"control-plane-machine-set-operator-78cbb6b69f-p4b4l\" (UID: \"693b46e6-42c3-4d07-bfbd-bd83d6982199\") " pod="openshift-machine-api/control-plane-machine-set-operator-78cbb6b69f-p4b4l" Dec 03 11:20:46 crc kubenswrapper[5052]: I1203 11:20:46.051742 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3afb93c1-9cbc-4588-a3bf-2ae1e774a2d1-serving-cert\") pod \"console-operator-58897d9998-7vpns\" (UID: \"3afb93c1-9cbc-4588-a3bf-2ae1e774a2d1\") " pod="openshift-console-operator/console-operator-58897d9998-7vpns" Dec 03 11:20:46 crc kubenswrapper[5052]: I1203 11:20:46.051777 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-5s2zd\" (UniqueName: \"kubernetes.io/projected/663b7426-fde0-4b59-b857-b434b4a88f4b-kube-api-access-5s2zd\") pod \"olm-operator-6b444d44fb-6lm8x\" (UID: \"663b7426-fde0-4b59-b857-b434b4a88f4b\") " pod="openshift-operator-lifecycle-manager/olm-operator-6b444d44fb-6lm8x" Dec 03 11:20:46 crc kubenswrapper[5052]: I1203 11:20:46.051806 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etcd-client\" (UniqueName: \"kubernetes.io/secret/71ea0624-9fb5-415a-96e1-d5ada83c0fe4-etcd-client\") pod \"etcd-operator-b45778765-z2lp6\" (UID: \"71ea0624-9fb5-415a-96e1-d5ada83c0fe4\") " pod="openshift-etcd-operator/etcd-operator-b45778765-z2lp6" Dec 03 11:20:46 crc kubenswrapper[5052]: I1203 11:20:46.051825 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/43fca84c-422e-4e3b-b3ca-b787a4e3f677-config\") pod \"openshift-controller-manager-operator-756b6f6bc6-v7lmf\" (UID: \"43fca84c-422e-4e3b-b3ca-b787a4e3f677\") " pod="openshift-controller-manager-operator/openshift-controller-manager-operator-756b6f6bc6-v7lmf" Dec 03 11:20:46 crc kubenswrapper[5052]: I1203 11:20:46.051852 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3afb93c1-9cbc-4588-a3bf-2ae1e774a2d1-trusted-ca\") pod \"console-operator-58897d9998-7vpns\" (UID: \"3afb93c1-9cbc-4588-a3bf-2ae1e774a2d1\") " pod="openshift-console-operator/console-operator-58897d9998-7vpns" Dec 03 11:20:46 crc kubenswrapper[5052]: I1203 11:20:46.051900 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-vkn2m\" (UniqueName: \"kubernetes.io/projected/48040744-7fc1-482e-9b68-3a904ba03a8d-kube-api-access-vkn2m\") pod \"router-default-5444994796-rtl8t\" (UID: \"48040744-7fc1-482e-9b68-3a904ba03a8d\") " pod="openshift-ingress/router-default-5444994796-rtl8t" Dec 03 11:20:46 crc kubenswrapper[5052]: I1203 11:20:46.051918 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etcd-ca\" (UniqueName: \"kubernetes.io/configmap/71ea0624-9fb5-415a-96e1-d5ada83c0fe4-etcd-ca\") pod \"etcd-operator-b45778765-z2lp6\" (UID: \"71ea0624-9fb5-415a-96e1-d5ada83c0fe4\") " pod="openshift-etcd-operator/etcd-operator-b45778765-z2lp6" Dec 03 11:20:46 crc kubenswrapper[5052]: I1203 11:20:46.051933 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8c1b25f5-2d98-4852-bc7a-d5cbfa5cb6aa-config\") pod \"machine-approver-56656f9798-6t9dp\" (UID: \"8c1b25f5-2d98-4852-bc7a-d5cbfa5cb6aa\") " pod="openshift-cluster-machine-approver/machine-approver-56656f9798-6t9dp" Dec 03 11:20:46 crc kubenswrapper[5052]: I1203 11:20:46.052060 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"tmpfs\" (UniqueName: \"kubernetes.io/empty-dir/6413eb91-7b61-4660-888c-14da224c7dfa-tmpfs\") pod \"packageserver-d55dfcdfc-7s97q\" (UID: \"6413eb91-7b61-4660-888c-14da224c7dfa\") " pod="openshift-operator-lifecycle-manager/packageserver-d55dfcdfc-7s97q" Dec 03 11:20:46 crc kubenswrapper[5052]: I1203 11:20:46.052361 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8c1b25f5-2d98-4852-bc7a-d5cbfa5cb6aa-config\") pod \"machine-approver-56656f9798-6t9dp\" (UID: \"8c1b25f5-2d98-4852-bc7a-d5cbfa5cb6aa\") " pod="openshift-cluster-machine-approver/machine-approver-56656f9798-6t9dp" Dec 03 11:20:46 crc kubenswrapper[5052]: I1203 11:20:46.053250 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3afb93c1-9cbc-4588-a3bf-2ae1e774a2d1-trusted-ca\") pod \"console-operator-58897d9998-7vpns\" (UID: \"3afb93c1-9cbc-4588-a3bf-2ae1e774a2d1\") " pod="openshift-console-operator/console-operator-58897d9998-7vpns" Dec 03 11:20:46 crc kubenswrapper[5052]: I1203 11:20:46.054700 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"machine-approver-tls\" (UniqueName: \"kubernetes.io/secret/8c1b25f5-2d98-4852-bc7a-d5cbfa5cb6aa-machine-approver-tls\") pod \"machine-approver-56656f9798-6t9dp\" (UID: \"8c1b25f5-2d98-4852-bc7a-d5cbfa5cb6aa\") " pod="openshift-cluster-machine-approver/machine-approver-56656f9798-6t9dp" Dec 03 11:20:46 crc kubenswrapper[5052]: I1203 11:20:46.063936 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-controller-manager-operator"/"openshift-controller-manager-operator-dockercfg-vw8fw" Dec 03 11:20:46 crc kubenswrapper[5052]: I1203 11:20:46.084781 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-controller-manager-operator"/"openshift-controller-manager-operator-serving-cert" Dec 03 11:20:46 crc kubenswrapper[5052]: I1203 11:20:46.094990 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/43fca84c-422e-4e3b-b3ca-b787a4e3f677-serving-cert\") pod \"openshift-controller-manager-operator-756b6f6bc6-v7lmf\" (UID: \"43fca84c-422e-4e3b-b3ca-b787a4e3f677\") " pod="openshift-controller-manager-operator/openshift-controller-manager-operator-756b6f6bc6-v7lmf" Dec 03 11:20:46 crc kubenswrapper[5052]: I1203 11:20:46.104528 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-controller-manager-operator"/"openshift-controller-manager-operator-config" Dec 03 11:20:46 crc kubenswrapper[5052]: I1203 11:20:46.113560 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/43fca84c-422e-4e3b-b3ca-b787a4e3f677-config\") pod \"openshift-controller-manager-operator-756b6f6bc6-v7lmf\" (UID: \"43fca84c-422e-4e3b-b3ca-b787a4e3f677\") " pod="openshift-controller-manager-operator/openshift-controller-manager-operator-756b6f6bc6-v7lmf" Dec 03 11:20:46 crc kubenswrapper[5052]: I1203 11:20:46.128350 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-console-operator"/"kube-root-ca.crt" Dec 03 11:20:46 crc kubenswrapper[5052]: I1203 11:20:46.143589 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-multus"/"multus-admission-controller-secret" Dec 03 11:20:46 crc kubenswrapper[5052]: I1203 11:20:46.153845 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\") pod \"image-registry-697d97f7c8-p77bx\" (UID: \"f5012254-068c-4360-b310-39ff3d309199\") " pod="openshift-image-registry/image-registry-697d97f7c8-p77bx" Dec 03 11:20:46 crc kubenswrapper[5052]: E1203 11:20:46.155917 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8 podName: nodeName:}" failed. No retries permitted until 2025-12-03 11:20:46.655898585 +0000 UTC m=+37.244140349 (durationBeforeRetry 500ms). Error: MountVolume.MountDevice failed for volume "pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8" (UniqueName: "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8") pod "image-registry-697d97f7c8-p77bx" (UID: "f5012254-068c-4360-b310-39ff3d309199") : kubernetes.io/csi: attacher.MountDevice failed to create newCsiDriverClient: driver name kubevirt.io.hostpath-provisioner not found in the list of registered CSI drivers Dec 03 11:20:46 crc kubenswrapper[5052]: I1203 11:20:46.163429 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-console-operator"/"openshift-service-ca.crt" Dec 03 11:20:46 crc kubenswrapper[5052]: I1203 11:20:46.183916 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-multus"/"multus-ac-dockercfg-9lkdf" Dec 03 11:20:46 crc kubenswrapper[5052]: I1203 11:20:46.203271 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-console-operator"/"console-operator-config" Dec 03 11:20:46 crc kubenswrapper[5052]: I1203 11:20:46.211932 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3afb93c1-9cbc-4588-a3bf-2ae1e774a2d1-config\") pod \"console-operator-58897d9998-7vpns\" (UID: \"3afb93c1-9cbc-4588-a3bf-2ae1e774a2d1\") " pod="openshift-console-operator/console-operator-58897d9998-7vpns" Dec 03 11:20:46 crc kubenswrapper[5052]: I1203 11:20:46.224548 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-controller-manager-operator"/"kube-root-ca.crt" Dec 03 11:20:46 crc kubenswrapper[5052]: I1203 11:20:46.243735 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-console-operator"/"console-operator-dockercfg-4xjcr" Dec 03 11:20:46 crc kubenswrapper[5052]: I1203 11:20:46.255768 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\") pod \"8f668bae-612b-4b75-9490-919e737c6a3b\" (UID: \"8f668bae-612b-4b75-9490-919e737c6a3b\") " Dec 03 11:20:46 crc kubenswrapper[5052]: E1203 11:20:46.255936 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8 podName:8f668bae-612b-4b75-9490-919e737c6a3b nodeName:}" failed. No retries permitted until 2025-12-03 11:20:46.755912972 +0000 UTC m=+37.344154746 (durationBeforeRetry 500ms). Error: UnmountVolume.TearDown failed for volume "pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8" (UniqueName: "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8") pod "8f668bae-612b-4b75-9490-919e737c6a3b" (UID: "8f668bae-612b-4b75-9490-919e737c6a3b") : kubernetes.io/csi: Unmounter.TearDownAt failed to get CSI client: driver name kubevirt.io.hostpath-provisioner not found in the list of registered CSI drivers Dec 03 11:20:46 crc kubenswrapper[5052]: I1203 11:20:46.256208 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\") pod \"image-registry-697d97f7c8-p77bx\" (UID: \"f5012254-068c-4360-b310-39ff3d309199\") " pod="openshift-image-registry/image-registry-697d97f7c8-p77bx" Dec 03 11:20:46 crc kubenswrapper[5052]: E1203 11:20:46.256549 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8 podName: nodeName:}" failed. No retries permitted until 2025-12-03 11:20:46.756538899 +0000 UTC m=+37.344780663 (durationBeforeRetry 500ms). Error: MountVolume.MountDevice failed for volume "pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8" (UniqueName: "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8") pod "image-registry-697d97f7c8-p77bx" (UID: "f5012254-068c-4360-b310-39ff3d309199") : kubernetes.io/csi: attacher.MountDevice failed to create newCsiDriverClient: driver name kubevirt.io.hostpath-provisioner not found in the list of registered CSI drivers Dec 03 11:20:46 crc kubenswrapper[5052]: I1203 11:20:46.264673 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-kube-apiserver-operator"/"kube-apiserver-operator-dockercfg-x57mr" Dec 03 11:20:46 crc kubenswrapper[5052]: I1203 11:20:46.283501 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-console-operator"/"serving-cert" Dec 03 11:20:46 crc kubenswrapper[5052]: I1203 11:20:46.296412 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3afb93c1-9cbc-4588-a3bf-2ae1e774a2d1-serving-cert\") pod \"console-operator-58897d9998-7vpns\" (UID: \"3afb93c1-9cbc-4588-a3bf-2ae1e774a2d1\") " pod="openshift-console-operator/console-operator-58897d9998-7vpns" Dec 03 11:20:46 crc kubenswrapper[5052]: I1203 11:20:46.303732 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-kube-apiserver-operator"/"kube-apiserver-operator-serving-cert" Dec 03 11:20:46 crc kubenswrapper[5052]: I1203 11:20:46.315559 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/287ad4d4-bf40-4cc4-9856-580b65a2e88d-serving-cert\") pod \"kube-apiserver-operator-766d6c64bb-xwlk8\" (UID: \"287ad4d4-bf40-4cc4-9856-580b65a2e88d\") " pod="openshift-kube-apiserver-operator/kube-apiserver-operator-766d6c64bb-xwlk8" Dec 03 11:20:46 crc kubenswrapper[5052]: I1203 11:20:46.324256 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-kube-apiserver-operator"/"kube-apiserver-operator-config" Dec 03 11:20:46 crc kubenswrapper[5052]: I1203 11:20:46.332964 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/287ad4d4-bf40-4cc4-9856-580b65a2e88d-config\") pod \"kube-apiserver-operator-766d6c64bb-xwlk8\" (UID: \"287ad4d4-bf40-4cc4-9856-580b65a2e88d\") " pod="openshift-kube-apiserver-operator/kube-apiserver-operator-766d6c64bb-xwlk8" Dec 03 11:20:46 crc kubenswrapper[5052]: I1203 11:20:46.344161 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-console"/"console-dockercfg-f62pw" Dec 03 11:20:46 crc kubenswrapper[5052]: I1203 11:20:46.357952 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\") pod \"8f668bae-612b-4b75-9490-919e737c6a3b\" (UID: \"8f668bae-612b-4b75-9490-919e737c6a3b\") " Dec 03 11:20:46 crc kubenswrapper[5052]: E1203 11:20:46.358246 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8 podName:8f668bae-612b-4b75-9490-919e737c6a3b nodeName:}" failed. No retries permitted until 2025-12-03 11:20:46.858207251 +0000 UTC m=+37.446449035 (durationBeforeRetry 500ms). Error: UnmountVolume.TearDown failed for volume "pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8" (UniqueName: "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8") pod "8f668bae-612b-4b75-9490-919e737c6a3b" (UID: "8f668bae-612b-4b75-9490-919e737c6a3b") : kubernetes.io/csi: Unmounter.TearDownAt failed to get CSI client: driver name kubevirt.io.hostpath-provisioner not found in the list of registered CSI drivers Dec 03 11:20:46 crc kubenswrapper[5052]: I1203 11:20:46.364004 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-console"/"oauth-serving-cert" Dec 03 11:20:46 crc kubenswrapper[5052]: I1203 11:20:46.383862 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-console"/"console-serving-cert" Dec 03 11:20:46 crc kubenswrapper[5052]: I1203 11:20:46.404295 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-console"/"console-config" Dec 03 11:20:46 crc kubenswrapper[5052]: I1203 11:20:46.423960 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-console"/"console-oauth-config" Dec 03 11:20:46 crc kubenswrapper[5052]: I1203 11:20:46.444199 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-console"/"service-ca" Dec 03 11:20:46 crc kubenswrapper[5052]: I1203 11:20:46.460458 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\") pod \"image-registry-697d97f7c8-p77bx\" (UID: \"f5012254-068c-4360-b310-39ff3d309199\") " pod="openshift-image-registry/image-registry-697d97f7c8-p77bx" Dec 03 11:20:46 crc kubenswrapper[5052]: E1203 11:20:46.460930 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8 podName: nodeName:}" failed. No retries permitted until 2025-12-03 11:20:46.960911921 +0000 UTC m=+37.549153685 (durationBeforeRetry 500ms). Error: MountVolume.MountDevice failed for volume "pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8" (UniqueName: "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8") pod "image-registry-697d97f7c8-p77bx" (UID: "f5012254-068c-4360-b310-39ff3d309199") : kubernetes.io/csi: attacher.MountDevice failed to create newCsiDriverClient: driver name kubevirt.io.hostpath-provisioner not found in the list of registered CSI drivers Dec 03 11:20:46 crc kubenswrapper[5052]: I1203 11:20:46.471159 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-console"/"trusted-ca-bundle" Dec 03 11:20:46 crc kubenswrapper[5052]: I1203 11:20:46.483863 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-kube-apiserver-operator"/"kube-root-ca.crt" Dec 03 11:20:46 crc kubenswrapper[5052]: I1203 11:20:46.503958 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-etcd-operator"/"etcd-client" Dec 03 11:20:46 crc kubenswrapper[5052]: I1203 11:20:46.517457 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"etcd-client\" (UniqueName: \"kubernetes.io/secret/71ea0624-9fb5-415a-96e1-d5ada83c0fe4-etcd-client\") pod \"etcd-operator-b45778765-z2lp6\" (UID: \"71ea0624-9fb5-415a-96e1-d5ada83c0fe4\") " pod="openshift-etcd-operator/etcd-operator-b45778765-z2lp6" Dec 03 11:20:46 crc kubenswrapper[5052]: I1203 11:20:46.523652 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-etcd-operator"/"etcd-ca-bundle" Dec 03 11:20:46 crc kubenswrapper[5052]: I1203 11:20:46.534213 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"etcd-ca\" (UniqueName: \"kubernetes.io/configmap/71ea0624-9fb5-415a-96e1-d5ada83c0fe4-etcd-ca\") pod \"etcd-operator-b45778765-z2lp6\" (UID: \"71ea0624-9fb5-415a-96e1-d5ada83c0fe4\") " pod="openshift-etcd-operator/etcd-operator-b45778765-z2lp6" Dec 03 11:20:46 crc kubenswrapper[5052]: I1203 11:20:46.544599 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-etcd-operator"/"etcd-operator-dockercfg-r9srn" Dec 03 11:20:46 crc kubenswrapper[5052]: I1203 11:20:46.561334 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\") pod \"8f668bae-612b-4b75-9490-919e737c6a3b\" (UID: \"8f668bae-612b-4b75-9490-919e737c6a3b\") " Dec 03 11:20:46 crc kubenswrapper[5052]: E1203 11:20:46.561581 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8 podName:8f668bae-612b-4b75-9490-919e737c6a3b nodeName:}" failed. No retries permitted until 2025-12-03 11:20:47.061559115 +0000 UTC m=+37.649800879 (durationBeforeRetry 500ms). Error: UnmountVolume.TearDown failed for volume "pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8" (UniqueName: "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8") pod "8f668bae-612b-4b75-9490-919e737c6a3b" (UID: "8f668bae-612b-4b75-9490-919e737c6a3b") : kubernetes.io/csi: Unmounter.TearDownAt failed to get CSI client: driver name kubevirt.io.hostpath-provisioner not found in the list of registered CSI drivers Dec 03 11:20:46 crc kubenswrapper[5052]: I1203 11:20:46.562030 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\") pod \"image-registry-697d97f7c8-p77bx\" (UID: \"f5012254-068c-4360-b310-39ff3d309199\") " pod="openshift-image-registry/image-registry-697d97f7c8-p77bx" Dec 03 11:20:46 crc kubenswrapper[5052]: E1203 11:20:46.562692 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8 podName: nodeName:}" failed. No retries permitted until 2025-12-03 11:20:47.062669515 +0000 UTC m=+37.650911279 (durationBeforeRetry 500ms). Error: MountVolume.MountDevice failed for volume "pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8" (UniqueName: "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8") pod "image-registry-697d97f7c8-p77bx" (UID: "f5012254-068c-4360-b310-39ff3d309199") : kubernetes.io/csi: attacher.MountDevice failed to create newCsiDriverClient: driver name kubevirt.io.hostpath-provisioner not found in the list of registered CSI drivers Dec 03 11:20:46 crc kubenswrapper[5052]: I1203 11:20:46.564533 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-etcd-operator"/"etcd-service-ca-bundle" Dec 03 11:20:46 crc kubenswrapper[5052]: I1203 11:20:46.573062 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"etcd-service-ca\" (UniqueName: \"kubernetes.io/configmap/71ea0624-9fb5-415a-96e1-d5ada83c0fe4-etcd-service-ca\") pod \"etcd-operator-b45778765-z2lp6\" (UID: \"71ea0624-9fb5-415a-96e1-d5ada83c0fe4\") " pod="openshift-etcd-operator/etcd-operator-b45778765-z2lp6" Dec 03 11:20:46 crc kubenswrapper[5052]: I1203 11:20:46.584693 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-etcd-operator"/"etcd-operator-serving-cert" Dec 03 11:20:46 crc kubenswrapper[5052]: I1203 11:20:46.594209 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/71ea0624-9fb5-415a-96e1-d5ada83c0fe4-serving-cert\") pod \"etcd-operator-b45778765-z2lp6\" (UID: \"71ea0624-9fb5-415a-96e1-d5ada83c0fe4\") " pod="openshift-etcd-operator/etcd-operator-b45778765-z2lp6" Dec 03 11:20:46 crc kubenswrapper[5052]: I1203 11:20:46.603462 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-etcd-operator"/"kube-root-ca.crt" Dec 03 11:20:46 crc kubenswrapper[5052]: I1203 11:20:46.624058 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-etcd-operator"/"openshift-service-ca.crt" Dec 03 11:20:46 crc kubenswrapper[5052]: I1203 11:20:46.645030 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-etcd-operator"/"etcd-operator-config" Dec 03 11:20:46 crc kubenswrapper[5052]: I1203 11:20:46.652107 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/71ea0624-9fb5-415a-96e1-d5ada83c0fe4-config\") pod \"etcd-operator-b45778765-z2lp6\" (UID: \"71ea0624-9fb5-415a-96e1-d5ada83c0fe4\") " pod="openshift-etcd-operator/etcd-operator-b45778765-z2lp6" Dec 03 11:20:46 crc kubenswrapper[5052]: I1203 11:20:46.664216 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-machine-api"/"machine-api-operator-tls" Dec 03 11:20:46 crc kubenswrapper[5052]: I1203 11:20:46.664309 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\") pod \"8f668bae-612b-4b75-9490-919e737c6a3b\" (UID: \"8f668bae-612b-4b75-9490-919e737c6a3b\") " Dec 03 11:20:46 crc kubenswrapper[5052]: E1203 11:20:46.665876 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8 podName:8f668bae-612b-4b75-9490-919e737c6a3b nodeName:}" failed. No retries permitted until 2025-12-03 11:20:47.165839479 +0000 UTC m=+37.754081243 (durationBeforeRetry 500ms). Error: UnmountVolume.TearDown failed for volume "pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8" (UniqueName: "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8") pod "8f668bae-612b-4b75-9490-919e737c6a3b" (UID: "8f668bae-612b-4b75-9490-919e737c6a3b") : kubernetes.io/csi: Unmounter.TearDownAt failed to get CSI client: driver name kubevirt.io.hostpath-provisioner not found in the list of registered CSI drivers Dec 03 11:20:46 crc kubenswrapper[5052]: I1203 11:20:46.666069 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\") pod \"image-registry-697d97f7c8-p77bx\" (UID: \"f5012254-068c-4360-b310-39ff3d309199\") " pod="openshift-image-registry/image-registry-697d97f7c8-p77bx" Dec 03 11:20:46 crc kubenswrapper[5052]: E1203 11:20:46.666461 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8 podName: nodeName:}" failed. No retries permitted until 2025-12-03 11:20:47.166445315 +0000 UTC m=+37.754687079 (durationBeforeRetry 500ms). Error: MountVolume.MountDevice failed for volume "pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8" (UniqueName: "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8") pod "image-registry-697d97f7c8-p77bx" (UID: "f5012254-068c-4360-b310-39ff3d309199") : kubernetes.io/csi: attacher.MountDevice failed to create newCsiDriverClient: driver name kubevirt.io.hostpath-provisioner not found in the list of registered CSI drivers Dec 03 11:20:46 crc kubenswrapper[5052]: I1203 11:20:46.673094 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xd92c" Dec 03 11:20:46 crc kubenswrapper[5052]: I1203 11:20:46.684492 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-machine-api"/"openshift-service-ca.crt" Dec 03 11:20:46 crc kubenswrapper[5052]: I1203 11:20:46.704589 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-machine-api"/"kube-rbac-proxy" Dec 03 11:20:46 crc kubenswrapper[5052]: I1203 11:20:46.724190 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-machine-api"/"machine-api-operator-dockercfg-mfbb7" Dec 03 11:20:46 crc kubenswrapper[5052]: I1203 11:20:46.745336 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-machine-api"/"kube-root-ca.crt" Dec 03 11:20:46 crc kubenswrapper[5052]: I1203 11:20:46.762219 5052 request.go:700] Waited for 1.00659243s due to client-side throttling, not priority and fairness, request: GET:https://api-int.crc.testing:6443/api/v1/namespaces/openshift-machine-api/configmaps?fieldSelector=metadata.name%3Dmachine-api-operator-images&limit=500&resourceVersion=0 Dec 03 11:20:46 crc kubenswrapper[5052]: I1203 11:20:46.763964 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-machine-api"/"machine-api-operator-images" Dec 03 11:20:46 crc kubenswrapper[5052]: I1203 11:20:46.767260 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\") pod \"8f668bae-612b-4b75-9490-919e737c6a3b\" (UID: \"8f668bae-612b-4b75-9490-919e737c6a3b\") " Dec 03 11:20:46 crc kubenswrapper[5052]: E1203 11:20:46.767449 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8 podName:8f668bae-612b-4b75-9490-919e737c6a3b nodeName:}" failed. No retries permitted until 2025-12-03 11:20:47.267408088 +0000 UTC m=+37.855649852 (durationBeforeRetry 500ms). Error: UnmountVolume.TearDown failed for volume "pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8" (UniqueName: "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8") pod "8f668bae-612b-4b75-9490-919e737c6a3b" (UID: "8f668bae-612b-4b75-9490-919e737c6a3b") : kubernetes.io/csi: Unmounter.TearDownAt failed to get CSI client: driver name kubevirt.io.hostpath-provisioner not found in the list of registered CSI drivers Dec 03 11:20:46 crc kubenswrapper[5052]: I1203 11:20:46.768130 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\") pod \"image-registry-697d97f7c8-p77bx\" (UID: \"f5012254-068c-4360-b310-39ff3d309199\") " pod="openshift-image-registry/image-registry-697d97f7c8-p77bx" Dec 03 11:20:46 crc kubenswrapper[5052]: E1203 11:20:46.768553 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8 podName: nodeName:}" failed. No retries permitted until 2025-12-03 11:20:47.268541069 +0000 UTC m=+37.856782833 (durationBeforeRetry 500ms). Error: MountVolume.MountDevice failed for volume "pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8" (UniqueName: "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8") pod "image-registry-697d97f7c8-p77bx" (UID: "f5012254-068c-4360-b310-39ff3d309199") : kubernetes.io/csi: attacher.MountDevice failed to create newCsiDriverClient: driver name kubevirt.io.hostpath-provisioner not found in the list of registered CSI drivers Dec 03 11:20:46 crc kubenswrapper[5052]: I1203 11:20:46.783717 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-kube-controller-manager-operator"/"kube-root-ca.crt" Dec 03 11:20:46 crc kubenswrapper[5052]: I1203 11:20:46.804068 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-kube-controller-manager-operator"/"kube-controller-manager-operator-dockercfg-gkqpw" Dec 03 11:20:46 crc kubenswrapper[5052]: E1203 11:20:46.822118 5052 secret.go:188] Couldn't get secret openshift-authentication-operator/serving-cert: failed to sync secret cache: timed out waiting for the condition Dec 03 11:20:46 crc kubenswrapper[5052]: E1203 11:20:46.823474 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e0bf52c7-bf23-4c65-b7cb-0d8d466aa8f2-serving-cert podName:e0bf52c7-bf23-4c65-b7cb-0d8d466aa8f2 nodeName:}" failed. No retries permitted until 2025-12-03 11:20:47.322197848 +0000 UTC m=+37.910439612 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "serving-cert" (UniqueName: "kubernetes.io/secret/e0bf52c7-bf23-4c65-b7cb-0d8d466aa8f2-serving-cert") pod "authentication-operator-69f744f599-qjq5f" (UID: "e0bf52c7-bf23-4c65-b7cb-0d8d466aa8f2") : failed to sync secret cache: timed out waiting for the condition Dec 03 11:20:46 crc kubenswrapper[5052]: I1203 11:20:46.824037 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-kube-controller-manager-operator"/"kube-controller-manager-operator-serving-cert" Dec 03 11:20:46 crc kubenswrapper[5052]: E1203 11:20:46.827480 5052 configmap.go:193] Couldn't get configMap openshift-apiserver/image-import-ca: failed to sync configmap cache: timed out waiting for the condition Dec 03 11:20:46 crc kubenswrapper[5052]: E1203 11:20:46.827570 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/41016a7b-cc79-4f32-860c-cb75e405cf42-image-import-ca podName:41016a7b-cc79-4f32-860c-cb75e405cf42 nodeName:}" failed. No retries permitted until 2025-12-03 11:20:47.327553265 +0000 UTC m=+37.915795029 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "image-import-ca" (UniqueName: "kubernetes.io/configmap/41016a7b-cc79-4f32-860c-cb75e405cf42-image-import-ca") pod "apiserver-76f77b778f-dmvst" (UID: "41016a7b-cc79-4f32-860c-cb75e405cf42") : failed to sync configmap cache: timed out waiting for the condition Dec 03 11:20:46 crc kubenswrapper[5052]: E1203 11:20:46.834755 5052 configmap.go:193] Couldn't get configMap openshift-authentication-operator/authentication-operator-config: failed to sync configmap cache: timed out waiting for the condition Dec 03 11:20:46 crc kubenswrapper[5052]: E1203 11:20:46.834776 5052 configmap.go:193] Couldn't get configMap openshift-apiserver/etcd-serving-ca: failed to sync configmap cache: timed out waiting for the condition Dec 03 11:20:46 crc kubenswrapper[5052]: E1203 11:20:46.834807 5052 secret.go:188] Couldn't get secret openshift-apiserver/serving-cert: failed to sync secret cache: timed out waiting for the condition Dec 03 11:20:46 crc kubenswrapper[5052]: E1203 11:20:46.834813 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e0bf52c7-bf23-4c65-b7cb-0d8d466aa8f2-config podName:e0bf52c7-bf23-4c65-b7cb-0d8d466aa8f2 nodeName:}" failed. No retries permitted until 2025-12-03 11:20:47.334800175 +0000 UTC m=+37.923041929 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "config" (UniqueName: "kubernetes.io/configmap/e0bf52c7-bf23-4c65-b7cb-0d8d466aa8f2-config") pod "authentication-operator-69f744f599-qjq5f" (UID: "e0bf52c7-bf23-4c65-b7cb-0d8d466aa8f2") : failed to sync configmap cache: timed out waiting for the condition Dec 03 11:20:46 crc kubenswrapper[5052]: E1203 11:20:46.834929 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/41016a7b-cc79-4f32-860c-cb75e405cf42-etcd-serving-ca podName:41016a7b-cc79-4f32-860c-cb75e405cf42 nodeName:}" failed. No retries permitted until 2025-12-03 11:20:47.334903068 +0000 UTC m=+37.923144832 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "etcd-serving-ca" (UniqueName: "kubernetes.io/configmap/41016a7b-cc79-4f32-860c-cb75e405cf42-etcd-serving-ca") pod "apiserver-76f77b778f-dmvst" (UID: "41016a7b-cc79-4f32-860c-cb75e405cf42") : failed to sync configmap cache: timed out waiting for the condition Dec 03 11:20:46 crc kubenswrapper[5052]: E1203 11:20:46.834947 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/41016a7b-cc79-4f32-860c-cb75e405cf42-serving-cert podName:41016a7b-cc79-4f32-860c-cb75e405cf42 nodeName:}" failed. No retries permitted until 2025-12-03 11:20:47.334941909 +0000 UTC m=+37.923183673 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "serving-cert" (UniqueName: "kubernetes.io/secret/41016a7b-cc79-4f32-860c-cb75e405cf42-serving-cert") pod "apiserver-76f77b778f-dmvst" (UID: "41016a7b-cc79-4f32-860c-cb75e405cf42") : failed to sync secret cache: timed out waiting for the condition Dec 03 11:20:46 crc kubenswrapper[5052]: I1203 11:20:46.843285 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-kube-controller-manager-operator"/"kube-controller-manager-operator-config" Dec 03 11:20:46 crc kubenswrapper[5052]: E1203 11:20:46.863093 5052 secret.go:188] Couldn't get secret openshift-apiserver/etcd-client: failed to sync secret cache: timed out waiting for the condition Dec 03 11:20:46 crc kubenswrapper[5052]: E1203 11:20:46.863228 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/41016a7b-cc79-4f32-860c-cb75e405cf42-etcd-client podName:41016a7b-cc79-4f32-860c-cb75e405cf42 nodeName:}" failed. No retries permitted until 2025-12-03 11:20:47.363199198 +0000 UTC m=+37.951440962 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "etcd-client" (UniqueName: "kubernetes.io/secret/41016a7b-cc79-4f32-860c-cb75e405cf42-etcd-client") pod "apiserver-76f77b778f-dmvst" (UID: "41016a7b-cc79-4f32-860c-cb75e405cf42") : failed to sync secret cache: timed out waiting for the condition Dec 03 11:20:46 crc kubenswrapper[5052]: I1203 11:20:46.864454 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-ingress"/"openshift-service-ca.crt" Dec 03 11:20:46 crc kubenswrapper[5052]: I1203 11:20:46.868990 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\") pod \"8f668bae-612b-4b75-9490-919e737c6a3b\" (UID: \"8f668bae-612b-4b75-9490-919e737c6a3b\") " Dec 03 11:20:46 crc kubenswrapper[5052]: E1203 11:20:46.869211 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8 podName:8f668bae-612b-4b75-9490-919e737c6a3b nodeName:}" failed. No retries permitted until 2025-12-03 11:20:47.369183122 +0000 UTC m=+37.957424896 (durationBeforeRetry 500ms). Error: UnmountVolume.TearDown failed for volume "pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8" (UniqueName: "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8") pod "8f668bae-612b-4b75-9490-919e737c6a3b" (UID: "8f668bae-612b-4b75-9490-919e737c6a3b") : kubernetes.io/csi: Unmounter.TearDownAt failed to get CSI client: driver name kubevirt.io.hostpath-provisioner not found in the list of registered CSI drivers Dec 03 11:20:46 crc kubenswrapper[5052]: I1203 11:20:46.869577 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\") pod \"image-registry-697d97f7c8-p77bx\" (UID: \"f5012254-068c-4360-b310-39ff3d309199\") " pod="openshift-image-registry/image-registry-697d97f7c8-p77bx" Dec 03 11:20:46 crc kubenswrapper[5052]: E1203 11:20:46.870121 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8 podName: nodeName:}" failed. No retries permitted until 2025-12-03 11:20:47.370108098 +0000 UTC m=+37.958349872 (durationBeforeRetry 500ms). Error: MountVolume.MountDevice failed for volume "pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8" (UniqueName: "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8") pod "image-registry-697d97f7c8-p77bx" (UID: "f5012254-068c-4360-b310-39ff3d309199") : kubernetes.io/csi: attacher.MountDevice failed to create newCsiDriverClient: driver name kubevirt.io.hostpath-provisioner not found in the list of registered CSI drivers Dec 03 11:20:46 crc kubenswrapper[5052]: I1203 11:20:46.884706 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-ingress"/"router-certs-default" Dec 03 11:20:46 crc kubenswrapper[5052]: I1203 11:20:46.895797 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/48040744-7fc1-482e-9b68-3a904ba03a8d-default-certificate\") pod \"router-default-5444994796-rtl8t\" (UID: \"48040744-7fc1-482e-9b68-3a904ba03a8d\") " pod="openshift-ingress/router-default-5444994796-rtl8t" Dec 03 11:20:46 crc kubenswrapper[5052]: I1203 11:20:46.903756 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-ingress"/"router-stats-default" Dec 03 11:20:46 crc kubenswrapper[5052]: I1203 11:20:46.915746 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/48040744-7fc1-482e-9b68-3a904ba03a8d-stats-auth\") pod \"router-default-5444994796-rtl8t\" (UID: \"48040744-7fc1-482e-9b68-3a904ba03a8d\") " pod="openshift-ingress/router-default-5444994796-rtl8t" Dec 03 11:20:46 crc kubenswrapper[5052]: I1203 11:20:46.924129 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-ingress"/"router-metrics-certs-default" Dec 03 11:20:46 crc kubenswrapper[5052]: I1203 11:20:46.934733 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/48040744-7fc1-482e-9b68-3a904ba03a8d-metrics-certs\") pod \"router-default-5444994796-rtl8t\" (UID: \"48040744-7fc1-482e-9b68-3a904ba03a8d\") " pod="openshift-ingress/router-default-5444994796-rtl8t" Dec 03 11:20:46 crc kubenswrapper[5052]: I1203 11:20:46.944072 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-ingress"/"router-dockercfg-zdk86" Dec 03 11:20:46 crc kubenswrapper[5052]: E1203 11:20:46.956156 5052 secret.go:188] Couldn't get secret openshift-image-registry/installation-pull-secrets: failed to sync secret cache: timed out waiting for the condition Dec 03 11:20:46 crc kubenswrapper[5052]: E1203 11:20:46.956262 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f5012254-068c-4360-b310-39ff3d309199-installation-pull-secrets podName:f5012254-068c-4360-b310-39ff3d309199 nodeName:}" failed. No retries permitted until 2025-12-03 11:20:47.456236692 +0000 UTC m=+38.044478526 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "installation-pull-secrets" (UniqueName: "kubernetes.io/secret/f5012254-068c-4360-b310-39ff3d309199-installation-pull-secrets") pod "image-registry-697d97f7c8-p77bx" (UID: "f5012254-068c-4360-b310-39ff3d309199") : failed to sync secret cache: timed out waiting for the condition Dec 03 11:20:46 crc kubenswrapper[5052]: I1203 11:20:46.963325 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-ingress"/"service-ca-bundle" Dec 03 11:20:46 crc kubenswrapper[5052]: I1203 11:20:46.970823 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\") pod \"8f668bae-612b-4b75-9490-919e737c6a3b\" (UID: \"8f668bae-612b-4b75-9490-919e737c6a3b\") " Dec 03 11:20:46 crc kubenswrapper[5052]: E1203 11:20:46.971135 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8 podName:8f668bae-612b-4b75-9490-919e737c6a3b nodeName:}" failed. No retries permitted until 2025-12-03 11:20:47.471099481 +0000 UTC m=+38.059341245 (durationBeforeRetry 500ms). Error: UnmountVolume.TearDown failed for volume "pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8" (UniqueName: "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8") pod "8f668bae-612b-4b75-9490-919e737c6a3b" (UID: "8f668bae-612b-4b75-9490-919e737c6a3b") : kubernetes.io/csi: Unmounter.TearDownAt failed to get CSI client: driver name kubevirt.io.hostpath-provisioner not found in the list of registered CSI drivers Dec 03 11:20:46 crc kubenswrapper[5052]: I1203 11:20:46.971421 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\") pod \"image-registry-697d97f7c8-p77bx\" (UID: \"f5012254-068c-4360-b310-39ff3d309199\") " pod="openshift-image-registry/image-registry-697d97f7c8-p77bx" Dec 03 11:20:46 crc kubenswrapper[5052]: E1203 11:20:46.971824 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8 podName: nodeName:}" failed. No retries permitted until 2025-12-03 11:20:47.47180201 +0000 UTC m=+38.060043784 (durationBeforeRetry 500ms). Error: MountVolume.MountDevice failed for volume "pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8" (UniqueName: "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8") pod "image-registry-697d97f7c8-p77bx" (UID: "f5012254-068c-4360-b310-39ff3d309199") : kubernetes.io/csi: attacher.MountDevice failed to create newCsiDriverClient: driver name kubevirt.io.hostpath-provisioner not found in the list of registered CSI drivers Dec 03 11:20:46 crc kubenswrapper[5052]: I1203 11:20:46.973025 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/48040744-7fc1-482e-9b68-3a904ba03a8d-service-ca-bundle\") pod \"router-default-5444994796-rtl8t\" (UID: \"48040744-7fc1-482e-9b68-3a904ba03a8d\") " pod="openshift-ingress/router-default-5444994796-rtl8t" Dec 03 11:20:46 crc kubenswrapper[5052]: I1203 11:20:46.983593 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-ingress"/"kube-root-ca.crt" Dec 03 11:20:47 crc kubenswrapper[5052]: I1203 11:20:47.003265 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-operator-lifecycle-manager"/"openshift-service-ca.crt" Dec 03 11:20:47 crc kubenswrapper[5052]: I1203 11:20:47.025092 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-operator-lifecycle-manager"/"olm-operator-serviceaccount-dockercfg-rq7zk" Dec 03 11:20:47 crc kubenswrapper[5052]: I1203 11:20:47.044191 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-operator-lifecycle-manager"/"package-server-manager-serving-cert" Dec 03 11:20:47 crc kubenswrapper[5052]: E1203 11:20:47.050775 5052 secret.go:188] Couldn't get secret openshift-service-ca-operator/serving-cert: failed to sync secret cache: timed out waiting for the condition Dec 03 11:20:47 crc kubenswrapper[5052]: E1203 11:20:47.050968 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3e0657c6-3f3c-4300-8cfb-b3db773dba75-serving-cert podName:3e0657c6-3f3c-4300-8cfb-b3db773dba75 nodeName:}" failed. No retries permitted until 2025-12-03 11:20:47.55090441 +0000 UTC m=+38.139146214 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "serving-cert" (UniqueName: "kubernetes.io/secret/3e0657c6-3f3c-4300-8cfb-b3db773dba75-serving-cert") pod "service-ca-operator-777779d784-9jvh6" (UID: "3e0657c6-3f3c-4300-8cfb-b3db773dba75") : failed to sync secret cache: timed out waiting for the condition Dec 03 11:20:47 crc kubenswrapper[5052]: E1203 11:20:47.051632 5052 secret.go:188] Couldn't get secret openshift-operator-lifecycle-manager/catalog-operator-serving-cert: failed to sync secret cache: timed out waiting for the condition Dec 03 11:20:47 crc kubenswrapper[5052]: E1203 11:20:47.051688 5052 secret.go:188] Couldn't get secret openshift-service-ca/signing-key: failed to sync secret cache: timed out waiting for the condition Dec 03 11:20:47 crc kubenswrapper[5052]: E1203 11:20:47.051770 5052 configmap.go:193] Couldn't get configMap openshift-apiserver-operator/openshift-apiserver-operator-config: failed to sync configmap cache: timed out waiting for the condition Dec 03 11:20:47 crc kubenswrapper[5052]: E1203 11:20:47.051838 5052 secret.go:188] Couldn't get secret openshift-apiserver-operator/openshift-apiserver-operator-serving-cert: failed to sync secret cache: timed out waiting for the condition Dec 03 11:20:47 crc kubenswrapper[5052]: E1203 11:20:47.051870 5052 secret.go:188] Couldn't get secret openshift-operator-lifecycle-manager/olm-operator-serving-cert: failed to sync secret cache: timed out waiting for the condition Dec 03 11:20:47 crc kubenswrapper[5052]: E1203 11:20:47.051881 5052 configmap.go:193] Couldn't get configMap openshift-service-ca/signing-cabundle: failed to sync configmap cache: timed out waiting for the condition Dec 03 11:20:47 crc kubenswrapper[5052]: E1203 11:20:47.051894 5052 secret.go:188] Couldn't get secret openshift-operator-lifecycle-manager/pprof-cert: failed to sync secret cache: timed out waiting for the condition Dec 03 11:20:47 crc kubenswrapper[5052]: E1203 11:20:47.051615 5052 secret.go:188] Couldn't get secret openshift-machine-api/control-plane-machine-set-operator-tls: failed to sync secret cache: timed out waiting for the condition Dec 03 11:20:47 crc kubenswrapper[5052]: E1203 11:20:47.051807 5052 secret.go:188] Couldn't get secret openshift-operator-lifecycle-manager/packageserver-service-cert: failed to sync secret cache: timed out waiting for the condition Dec 03 11:20:47 crc kubenswrapper[5052]: E1203 11:20:47.051700 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8e3a111c-05ff-4f33-b5ba-81dae833b529-srv-cert podName:8e3a111c-05ff-4f33-b5ba-81dae833b529 nodeName:}" failed. No retries permitted until 2025-12-03 11:20:47.551691062 +0000 UTC m=+38.139932826 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "srv-cert" (UniqueName: "kubernetes.io/secret/8e3a111c-05ff-4f33-b5ba-81dae833b529-srv-cert") pod "catalog-operator-68c6474976-rqhkg" (UID: "8e3a111c-05ff-4f33-b5ba-81dae833b529") : failed to sync secret cache: timed out waiting for the condition Dec 03 11:20:47 crc kubenswrapper[5052]: E1203 11:20:47.051973 5052 configmap.go:193] Couldn't get configMap openshift-operator-lifecycle-manager/collect-profiles-config: failed to sync configmap cache: timed out waiting for the condition Dec 03 11:20:47 crc kubenswrapper[5052]: E1203 11:20:47.051994 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eb07cd6b-71c7-4dc7-b658-40f32c63f373-signing-key podName:eb07cd6b-71c7-4dc7-b658-40f32c63f373 nodeName:}" failed. No retries permitted until 2025-12-03 11:20:47.5519681 +0000 UTC m=+38.140209854 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "signing-key" (UniqueName: "kubernetes.io/secret/eb07cd6b-71c7-4dc7-b658-40f32c63f373-signing-key") pod "service-ca-9c57cc56f-w7g95" (UID: "eb07cd6b-71c7-4dc7-b658-40f32c63f373") : failed to sync secret cache: timed out waiting for the condition Dec 03 11:20:47 crc kubenswrapper[5052]: E1203 11:20:47.052007 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/42cd7f06-f712-41e0-9fa5-c8ec9f7af997-config podName:42cd7f06-f712-41e0-9fa5-c8ec9f7af997 nodeName:}" failed. No retries permitted until 2025-12-03 11:20:47.552001481 +0000 UTC m=+38.140243245 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "config" (UniqueName: "kubernetes.io/configmap/42cd7f06-f712-41e0-9fa5-c8ec9f7af997-config") pod "openshift-apiserver-operator-796bbdcf4f-jskzg" (UID: "42cd7f06-f712-41e0-9fa5-c8ec9f7af997") : failed to sync configmap cache: timed out waiting for the condition Dec 03 11:20:47 crc kubenswrapper[5052]: E1203 11:20:47.052019 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/42cd7f06-f712-41e0-9fa5-c8ec9f7af997-serving-cert podName:42cd7f06-f712-41e0-9fa5-c8ec9f7af997 nodeName:}" failed. No retries permitted until 2025-12-03 11:20:47.552013011 +0000 UTC m=+38.140254775 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "serving-cert" (UniqueName: "kubernetes.io/secret/42cd7f06-f712-41e0-9fa5-c8ec9f7af997-serving-cert") pod "openshift-apiserver-operator-796bbdcf4f-jskzg" (UID: "42cd7f06-f712-41e0-9fa5-c8ec9f7af997") : failed to sync secret cache: timed out waiting for the condition Dec 03 11:20:47 crc kubenswrapper[5052]: E1203 11:20:47.052030 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/663b7426-fde0-4b59-b857-b434b4a88f4b-srv-cert podName:663b7426-fde0-4b59-b857-b434b4a88f4b nodeName:}" failed. No retries permitted until 2025-12-03 11:20:47.552024481 +0000 UTC m=+38.140266245 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "srv-cert" (UniqueName: "kubernetes.io/secret/663b7426-fde0-4b59-b857-b434b4a88f4b-srv-cert") pod "olm-operator-6b444d44fb-6lm8x" (UID: "663b7426-fde0-4b59-b857-b434b4a88f4b") : failed to sync secret cache: timed out waiting for the condition Dec 03 11:20:47 crc kubenswrapper[5052]: E1203 11:20:47.052042 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e43f14fc-f3eb-4f58-b202-cdfc641ac08a-secret-volume podName:e43f14fc-f3eb-4f58-b202-cdfc641ac08a nodeName:}" failed. No retries permitted until 2025-12-03 11:20:47.552038402 +0000 UTC m=+38.140280166 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-volume" (UniqueName: "kubernetes.io/secret/e43f14fc-f3eb-4f58-b202-cdfc641ac08a-secret-volume") pod "collect-profiles-29412675-hzd4n" (UID: "e43f14fc-f3eb-4f58-b202-cdfc641ac08a") : failed to sync secret cache: timed out waiting for the condition Dec 03 11:20:47 crc kubenswrapper[5052]: E1203 11:20:47.052052 5052 secret.go:188] Couldn't get secret openshift-operator-lifecycle-manager/pprof-cert: failed to sync secret cache: timed out waiting for the condition Dec 03 11:20:47 crc kubenswrapper[5052]: E1203 11:20:47.052074 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/eb07cd6b-71c7-4dc7-b658-40f32c63f373-signing-cabundle podName:eb07cd6b-71c7-4dc7-b658-40f32c63f373 nodeName:}" failed. No retries permitted until 2025-12-03 11:20:47.552050652 +0000 UTC m=+38.140292416 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "signing-cabundle" (UniqueName: "kubernetes.io/configmap/eb07cd6b-71c7-4dc7-b658-40f32c63f373-signing-cabundle") pod "service-ca-9c57cc56f-w7g95" (UID: "eb07cd6b-71c7-4dc7-b658-40f32c63f373") : failed to sync configmap cache: timed out waiting for the condition Dec 03 11:20:47 crc kubenswrapper[5052]: E1203 11:20:47.052085 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/693b46e6-42c3-4d07-bfbd-bd83d6982199-control-plane-machine-set-operator-tls podName:693b46e6-42c3-4d07-bfbd-bd83d6982199 nodeName:}" failed. No retries permitted until 2025-12-03 11:20:47.552080653 +0000 UTC m=+38.140322417 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "control-plane-machine-set-operator-tls" (UniqueName: "kubernetes.io/secret/693b46e6-42c3-4d07-bfbd-bd83d6982199-control-plane-machine-set-operator-tls") pod "control-plane-machine-set-operator-78cbb6b69f-p4b4l" (UID: "693b46e6-42c3-4d07-bfbd-bd83d6982199") : failed to sync secret cache: timed out waiting for the condition Dec 03 11:20:47 crc kubenswrapper[5052]: E1203 11:20:47.052097 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6413eb91-7b61-4660-888c-14da224c7dfa-apiservice-cert podName:6413eb91-7b61-4660-888c-14da224c7dfa nodeName:}" failed. No retries permitted until 2025-12-03 11:20:47.552091733 +0000 UTC m=+38.140333497 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "apiservice-cert" (UniqueName: "kubernetes.io/secret/6413eb91-7b61-4660-888c-14da224c7dfa-apiservice-cert") pod "packageserver-d55dfcdfc-7s97q" (UID: "6413eb91-7b61-4660-888c-14da224c7dfa") : failed to sync secret cache: timed out waiting for the condition Dec 03 11:20:47 crc kubenswrapper[5052]: E1203 11:20:47.052106 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e43f14fc-f3eb-4f58-b202-cdfc641ac08a-config-volume podName:e43f14fc-f3eb-4f58-b202-cdfc641ac08a nodeName:}" failed. No retries permitted until 2025-12-03 11:20:47.552102123 +0000 UTC m=+38.140343887 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "config-volume" (UniqueName: "kubernetes.io/configmap/e43f14fc-f3eb-4f58-b202-cdfc641ac08a-config-volume") pod "collect-profiles-29412675-hzd4n" (UID: "e43f14fc-f3eb-4f58-b202-cdfc641ac08a") : failed to sync configmap cache: timed out waiting for the condition Dec 03 11:20:47 crc kubenswrapper[5052]: E1203 11:20:47.052116 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8e3a111c-05ff-4f33-b5ba-81dae833b529-profile-collector-cert podName:8e3a111c-05ff-4f33-b5ba-81dae833b529 nodeName:}" failed. No retries permitted until 2025-12-03 11:20:47.552112154 +0000 UTC m=+38.140353908 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "profile-collector-cert" (UniqueName: "kubernetes.io/secret/8e3a111c-05ff-4f33-b5ba-81dae833b529-profile-collector-cert") pod "catalog-operator-68c6474976-rqhkg" (UID: "8e3a111c-05ff-4f33-b5ba-81dae833b529") : failed to sync secret cache: timed out waiting for the condition Dec 03 11:20:47 crc kubenswrapper[5052]: E1203 11:20:47.052161 5052 configmap.go:193] Couldn't get configMap openshift-service-ca-operator/service-ca-operator-config: failed to sync configmap cache: timed out waiting for the condition Dec 03 11:20:47 crc kubenswrapper[5052]: E1203 11:20:47.052260 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3e0657c6-3f3c-4300-8cfb-b3db773dba75-config podName:3e0657c6-3f3c-4300-8cfb-b3db773dba75 nodeName:}" failed. No retries permitted until 2025-12-03 11:20:47.552236397 +0000 UTC m=+38.140478201 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "config" (UniqueName: "kubernetes.io/configmap/3e0657c6-3f3c-4300-8cfb-b3db773dba75-config") pod "service-ca-operator-777779d784-9jvh6" (UID: "3e0657c6-3f3c-4300-8cfb-b3db773dba75") : failed to sync configmap cache: timed out waiting for the condition Dec 03 11:20:47 crc kubenswrapper[5052]: E1203 11:20:47.052340 5052 secret.go:188] Couldn't get secret openshift-operator-lifecycle-manager/pprof-cert: failed to sync secret cache: timed out waiting for the condition Dec 03 11:20:47 crc kubenswrapper[5052]: E1203 11:20:47.052394 5052 secret.go:188] Couldn't get secret openshift-operator-lifecycle-manager/packageserver-service-cert: failed to sync secret cache: timed out waiting for the condition Dec 03 11:20:47 crc kubenswrapper[5052]: E1203 11:20:47.052455 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/663b7426-fde0-4b59-b857-b434b4a88f4b-profile-collector-cert podName:663b7426-fde0-4b59-b857-b434b4a88f4b nodeName:}" failed. No retries permitted until 2025-12-03 11:20:47.552432703 +0000 UTC m=+38.140674507 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "profile-collector-cert" (UniqueName: "kubernetes.io/secret/663b7426-fde0-4b59-b857-b434b4a88f4b-profile-collector-cert") pod "olm-operator-6b444d44fb-6lm8x" (UID: "663b7426-fde0-4b59-b857-b434b4a88f4b") : failed to sync secret cache: timed out waiting for the condition Dec 03 11:20:47 crc kubenswrapper[5052]: E1203 11:20:47.052548 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6413eb91-7b61-4660-888c-14da224c7dfa-webhook-cert podName:6413eb91-7b61-4660-888c-14da224c7dfa nodeName:}" failed. No retries permitted until 2025-12-03 11:20:47.552506935 +0000 UTC m=+38.140748739 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/6413eb91-7b61-4660-888c-14da224c7dfa-webhook-cert") pod "packageserver-d55dfcdfc-7s97q" (UID: "6413eb91-7b61-4660-888c-14da224c7dfa") : failed to sync secret cache: timed out waiting for the condition Dec 03 11:20:47 crc kubenswrapper[5052]: I1203 11:20:47.063417 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-operator-lifecycle-manager"/"kube-root-ca.crt" Dec 03 11:20:47 crc kubenswrapper[5052]: I1203 11:20:47.073296 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\") pod \"8f668bae-612b-4b75-9490-919e737c6a3b\" (UID: \"8f668bae-612b-4b75-9490-919e737c6a3b\") " Dec 03 11:20:47 crc kubenswrapper[5052]: E1203 11:20:47.073460 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8 podName:8f668bae-612b-4b75-9490-919e737c6a3b nodeName:}" failed. No retries permitted until 2025-12-03 11:20:47.573434731 +0000 UTC m=+38.161676515 (durationBeforeRetry 500ms). Error: UnmountVolume.TearDown failed for volume "pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8" (UniqueName: "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8") pod "8f668bae-612b-4b75-9490-919e737c6a3b" (UID: "8f668bae-612b-4b75-9490-919e737c6a3b") : kubernetes.io/csi: Unmounter.TearDownAt failed to get CSI client: driver name kubevirt.io.hostpath-provisioner not found in the list of registered CSI drivers Dec 03 11:20:47 crc kubenswrapper[5052]: I1203 11:20:47.074301 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\") pod \"image-registry-697d97f7c8-p77bx\" (UID: \"f5012254-068c-4360-b310-39ff3d309199\") " pod="openshift-image-registry/image-registry-697d97f7c8-p77bx" Dec 03 11:20:47 crc kubenswrapper[5052]: E1203 11:20:47.074725 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8 podName: nodeName:}" failed. No retries permitted until 2025-12-03 11:20:47.574711727 +0000 UTC m=+38.162953491 (durationBeforeRetry 500ms). Error: MountVolume.MountDevice failed for volume "pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8" (UniqueName: "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8") pod "image-registry-697d97f7c8-p77bx" (UID: "f5012254-068c-4360-b310-39ff3d309199") : kubernetes.io/csi: attacher.MountDevice failed to create newCsiDriverClient: driver name kubevirt.io.hostpath-provisioner not found in the list of registered CSI drivers Dec 03 11:20:47 crc kubenswrapper[5052]: I1203 11:20:47.084285 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-apiserver-operator"/"openshift-service-ca.crt" Dec 03 11:20:47 crc kubenswrapper[5052]: I1203 11:20:47.103438 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-apiserver-operator"/"openshift-apiserver-operator-dockercfg-xtcjv" Dec 03 11:20:47 crc kubenswrapper[5052]: I1203 11:20:47.124114 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-apiserver-operator"/"openshift-apiserver-operator-serving-cert" Dec 03 11:20:47 crc kubenswrapper[5052]: I1203 11:20:47.144303 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-apiserver-operator"/"openshift-apiserver-operator-config" Dec 03 11:20:47 crc kubenswrapper[5052]: I1203 11:20:47.163726 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-apiserver-operator"/"kube-root-ca.crt" Dec 03 11:20:47 crc kubenswrapper[5052]: I1203 11:20:47.176616 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\") pod \"8f668bae-612b-4b75-9490-919e737c6a3b\" (UID: \"8f668bae-612b-4b75-9490-919e737c6a3b\") " Dec 03 11:20:47 crc kubenswrapper[5052]: E1203 11:20:47.176923 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8 podName:8f668bae-612b-4b75-9490-919e737c6a3b nodeName:}" failed. No retries permitted until 2025-12-03 11:20:47.676881632 +0000 UTC m=+38.265123406 (durationBeforeRetry 500ms). Error: UnmountVolume.TearDown failed for volume "pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8" (UniqueName: "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8") pod "8f668bae-612b-4b75-9490-919e737c6a3b" (UID: "8f668bae-612b-4b75-9490-919e737c6a3b") : kubernetes.io/csi: Unmounter.TearDownAt failed to get CSI client: driver name kubevirt.io.hostpath-provisioner not found in the list of registered CSI drivers Dec 03 11:20:47 crc kubenswrapper[5052]: I1203 11:20:47.177085 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\") pod \"image-registry-697d97f7c8-p77bx\" (UID: \"f5012254-068c-4360-b310-39ff3d309199\") " pod="openshift-image-registry/image-registry-697d97f7c8-p77bx" Dec 03 11:20:47 crc kubenswrapper[5052]: E1203 11:20:47.177558 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8 podName: nodeName:}" failed. No retries permitted until 2025-12-03 11:20:47.67754118 +0000 UTC m=+38.265782944 (durationBeforeRetry 500ms). Error: MountVolume.MountDevice failed for volume "pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8" (UniqueName: "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8") pod "image-registry-697d97f7c8-p77bx" (UID: "f5012254-068c-4360-b310-39ff3d309199") : kubernetes.io/csi: attacher.MountDevice failed to create newCsiDriverClient: driver name kubevirt.io.hostpath-provisioner not found in the list of registered CSI drivers Dec 03 11:20:47 crc kubenswrapper[5052]: I1203 11:20:47.184046 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-operator-lifecycle-manager"/"pprof-cert" Dec 03 11:20:47 crc kubenswrapper[5052]: I1203 11:20:47.203958 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-operator-lifecycle-manager"/"packageserver-service-cert" Dec 03 11:20:47 crc kubenswrapper[5052]: I1203 11:20:47.224021 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-operator-lifecycle-manager"/"catalog-operator-serving-cert" Dec 03 11:20:47 crc kubenswrapper[5052]: I1203 11:20:47.243488 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-service-ca"/"openshift-service-ca.crt" Dec 03 11:20:47 crc kubenswrapper[5052]: I1203 11:20:47.263558 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-operator-lifecycle-manager"/"olm-operator-serving-cert" Dec 03 11:20:47 crc kubenswrapper[5052]: I1203 11:20:47.279155 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\") pod \"8f668bae-612b-4b75-9490-919e737c6a3b\" (UID: \"8f668bae-612b-4b75-9490-919e737c6a3b\") " Dec 03 11:20:47 crc kubenswrapper[5052]: E1203 11:20:47.279433 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8 podName:8f668bae-612b-4b75-9490-919e737c6a3b nodeName:}" failed. No retries permitted until 2025-12-03 11:20:47.779398307 +0000 UTC m=+38.367640091 (durationBeforeRetry 500ms). Error: UnmountVolume.TearDown failed for volume "pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8" (UniqueName: "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8") pod "8f668bae-612b-4b75-9490-919e737c6a3b" (UID: "8f668bae-612b-4b75-9490-919e737c6a3b") : kubernetes.io/csi: Unmounter.TearDownAt failed to get CSI client: driver name kubevirt.io.hostpath-provisioner not found in the list of registered CSI drivers Dec 03 11:20:47 crc kubenswrapper[5052]: I1203 11:20:47.279726 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\") pod \"image-registry-697d97f7c8-p77bx\" (UID: \"f5012254-068c-4360-b310-39ff3d309199\") " pod="openshift-image-registry/image-registry-697d97f7c8-p77bx" Dec 03 11:20:47 crc kubenswrapper[5052]: E1203 11:20:47.280181 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8 podName: nodeName:}" failed. No retries permitted until 2025-12-03 11:20:47.780162779 +0000 UTC m=+38.368404543 (durationBeforeRetry 500ms). Error: MountVolume.MountDevice failed for volume "pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8" (UniqueName: "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8") pod "image-registry-697d97f7c8-p77bx" (UID: "f5012254-068c-4360-b310-39ff3d309199") : kubernetes.io/csi: attacher.MountDevice failed to create newCsiDriverClient: driver name kubevirt.io.hostpath-provisioner not found in the list of registered CSI drivers Dec 03 11:20:47 crc kubenswrapper[5052]: I1203 11:20:47.284051 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-operator-lifecycle-manager"/"collect-profiles-config" Dec 03 11:20:47 crc kubenswrapper[5052]: I1203 11:20:47.304798 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-service-ca"/"service-ca-dockercfg-pn86c" Dec 03 11:20:47 crc kubenswrapper[5052]: I1203 11:20:47.324496 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-service-ca"/"signing-key" Dec 03 11:20:47 crc kubenswrapper[5052]: I1203 11:20:47.344756 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-service-ca"/"signing-cabundle" Dec 03 11:20:47 crc kubenswrapper[5052]: I1203 11:20:47.364144 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-service-ca"/"kube-root-ca.crt" Dec 03 11:20:47 crc kubenswrapper[5052]: I1203 11:20:47.381096 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\") pod \"8f668bae-612b-4b75-9490-919e737c6a3b\" (UID: \"8f668bae-612b-4b75-9490-919e737c6a3b\") " Dec 03 11:20:47 crc kubenswrapper[5052]: E1203 11:20:47.381257 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8 podName:8f668bae-612b-4b75-9490-919e737c6a3b nodeName:}" failed. No retries permitted until 2025-12-03 11:20:47.881234524 +0000 UTC m=+38.469476288 (durationBeforeRetry 500ms). Error: UnmountVolume.TearDown failed for volume "pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8" (UniqueName: "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8") pod "8f668bae-612b-4b75-9490-919e737c6a3b" (UID: "8f668bae-612b-4b75-9490-919e737c6a3b") : kubernetes.io/csi: Unmounter.TearDownAt failed to get CSI client: driver name kubevirt.io.hostpath-provisioner not found in the list of registered CSI drivers Dec 03 11:20:47 crc kubenswrapper[5052]: I1203 11:20:47.381344 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e0bf52c7-bf23-4c65-b7cb-0d8d466aa8f2-serving-cert\") pod \"authentication-operator-69f744f599-qjq5f\" (UID: \"e0bf52c7-bf23-4c65-b7cb-0d8d466aa8f2\") " pod="openshift-authentication-operator/authentication-operator-69f744f599-qjq5f" Dec 03 11:20:47 crc kubenswrapper[5052]: I1203 11:20:47.381381 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etcd-serving-ca\" (UniqueName: \"kubernetes.io/configmap/41016a7b-cc79-4f32-860c-cb75e405cf42-etcd-serving-ca\") pod \"apiserver-76f77b778f-dmvst\" (UID: \"41016a7b-cc79-4f32-860c-cb75e405cf42\") " pod="openshift-apiserver/apiserver-76f77b778f-dmvst" Dec 03 11:20:47 crc kubenswrapper[5052]: I1203 11:20:47.381453 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e0bf52c7-bf23-4c65-b7cb-0d8d466aa8f2-config\") pod \"authentication-operator-69f744f599-qjq5f\" (UID: \"e0bf52c7-bf23-4c65-b7cb-0d8d466aa8f2\") " pod="openshift-authentication-operator/authentication-operator-69f744f599-qjq5f" Dec 03 11:20:47 crc kubenswrapper[5052]: I1203 11:20:47.381494 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"image-import-ca\" (UniqueName: \"kubernetes.io/configmap/41016a7b-cc79-4f32-860c-cb75e405cf42-image-import-ca\") pod \"apiserver-76f77b778f-dmvst\" (UID: \"41016a7b-cc79-4f32-860c-cb75e405cf42\") " pod="openshift-apiserver/apiserver-76f77b778f-dmvst" Dec 03 11:20:47 crc kubenswrapper[5052]: I1203 11:20:47.381552 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etcd-client\" (UniqueName: \"kubernetes.io/secret/41016a7b-cc79-4f32-860c-cb75e405cf42-etcd-client\") pod \"apiserver-76f77b778f-dmvst\" (UID: \"41016a7b-cc79-4f32-860c-cb75e405cf42\") " pod="openshift-apiserver/apiserver-76f77b778f-dmvst" Dec 03 11:20:47 crc kubenswrapper[5052]: I1203 11:20:47.381707 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\") pod \"image-registry-697d97f7c8-p77bx\" (UID: \"f5012254-068c-4360-b310-39ff3d309199\") " pod="openshift-image-registry/image-registry-697d97f7c8-p77bx" Dec 03 11:20:47 crc kubenswrapper[5052]: I1203 11:20:47.381795 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/41016a7b-cc79-4f32-860c-cb75e405cf42-serving-cert\") pod \"apiserver-76f77b778f-dmvst\" (UID: \"41016a7b-cc79-4f32-860c-cb75e405cf42\") " pod="openshift-apiserver/apiserver-76f77b778f-dmvst" Dec 03 11:20:47 crc kubenswrapper[5052]: E1203 11:20:47.382203 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8 podName: nodeName:}" failed. No retries permitted until 2025-12-03 11:20:47.88218535 +0000 UTC m=+38.470427114 (durationBeforeRetry 500ms). Error: MountVolume.MountDevice failed for volume "pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8" (UniqueName: "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8") pod "image-registry-697d97f7c8-p77bx" (UID: "f5012254-068c-4360-b310-39ff3d309199") : kubernetes.io/csi: attacher.MountDevice failed to create newCsiDriverClient: driver name kubevirt.io.hostpath-provisioner not found in the list of registered CSI drivers Dec 03 11:20:47 crc kubenswrapper[5052]: I1203 11:20:47.383992 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-operator-lifecycle-manager"/"collect-profiles-dockercfg-kzf4t" Dec 03 11:20:47 crc kubenswrapper[5052]: I1203 11:20:47.423997 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-machine-api"/"control-plane-machine-set-operator-dockercfg-k9rxt" Dec 03 11:20:47 crc kubenswrapper[5052]: I1203 11:20:47.444448 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-machine-api"/"control-plane-machine-set-operator-tls" Dec 03 11:20:47 crc kubenswrapper[5052]: I1203 11:20:47.464290 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-service-ca-operator"/"openshift-service-ca.crt" Dec 03 11:20:47 crc kubenswrapper[5052]: I1203 11:20:47.482640 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\") pod \"8f668bae-612b-4b75-9490-919e737c6a3b\" (UID: \"8f668bae-612b-4b75-9490-919e737c6a3b\") " Dec 03 11:20:47 crc kubenswrapper[5052]: E1203 11:20:47.482882 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8 podName:8f668bae-612b-4b75-9490-919e737c6a3b nodeName:}" failed. No retries permitted until 2025-12-03 11:20:47.982818913 +0000 UTC m=+38.571060677 (durationBeforeRetry 500ms). Error: UnmountVolume.TearDown failed for volume "pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8" (UniqueName: "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8") pod "8f668bae-612b-4b75-9490-919e737c6a3b" (UID: "8f668bae-612b-4b75-9490-919e737c6a3b") : kubernetes.io/csi: Unmounter.TearDownAt failed to get CSI client: driver name kubevirt.io.hostpath-provisioner not found in the list of registered CSI drivers Dec 03 11:20:47 crc kubenswrapper[5052]: I1203 11:20:47.483139 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-s2dwl\" (UniqueName: \"kubernetes.io/projected/9d751cbb-f2e2-430d-9754-c882a5e924a5-kube-api-access-s2dwl\") pod \"network-check-source-55646444c4-trplf\" (UID: \"9d751cbb-f2e2-430d-9754-c882a5e924a5\") " pod="openshift-network-diagnostics/network-check-source-55646444c4-trplf" Dec 03 11:20:47 crc kubenswrapper[5052]: I1203 11:20:47.483479 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/5fe485a1-e14f-4c09-b5b9-f252bc42b7e8-nginx-conf\") pod \"networking-console-plugin-85b44fc459-gdk6g\" (UID: \"5fe485a1-e14f-4c09-b5b9-f252bc42b7e8\") " pod="openshift-network-console/networking-console-plugin-85b44fc459-gdk6g" Dec 03 11:20:47 crc kubenswrapper[5052]: I1203 11:20:47.483595 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f5012254-068c-4360-b310-39ff3d309199-installation-pull-secrets\") pod \"image-registry-697d97f7c8-p77bx\" (UID: \"f5012254-068c-4360-b310-39ff3d309199\") " pod="openshift-image-registry/image-registry-697d97f7c8-p77bx" Dec 03 11:20:47 crc kubenswrapper[5052]: I1203 11:20:47.483686 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\") pod \"image-registry-697d97f7c8-p77bx\" (UID: \"f5012254-068c-4360-b310-39ff3d309199\") " pod="openshift-image-registry/image-registry-697d97f7c8-p77bx" Dec 03 11:20:47 crc kubenswrapper[5052]: I1203 11:20:47.483859 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-cqllr\" (UniqueName: \"kubernetes.io/projected/3b6479f0-333b-4a96-9adf-2099afdc2447-kube-api-access-cqllr\") pod \"network-check-target-xd92c\" (UID: \"3b6479f0-333b-4a96-9adf-2099afdc2447\") " pod="openshift-network-diagnostics/network-check-target-xd92c" Dec 03 11:20:47 crc kubenswrapper[5052]: I1203 11:20:47.483972 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/5fe485a1-e14f-4c09-b5b9-f252bc42b7e8-networking-console-plugin-cert\") pod \"networking-console-plugin-85b44fc459-gdk6g\" (UID: \"5fe485a1-e14f-4c09-b5b9-f252bc42b7e8\") " pod="openshift-network-console/networking-console-plugin-85b44fc459-gdk6g" Dec 03 11:20:47 crc kubenswrapper[5052]: E1203 11:20:47.484223 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8 podName: nodeName:}" failed. No retries permitted until 2025-12-03 11:20:47.984210962 +0000 UTC m=+38.572452726 (durationBeforeRetry 500ms). Error: MountVolume.MountDevice failed for volume "pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8" (UniqueName: "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8") pod "image-registry-697d97f7c8-p77bx" (UID: "f5012254-068c-4360-b310-39ff3d309199") : kubernetes.io/csi: attacher.MountDevice failed to create newCsiDriverClient: driver name kubevirt.io.hostpath-provisioner not found in the list of registered CSI drivers Dec 03 11:20:47 crc kubenswrapper[5052]: I1203 11:20:47.484397 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-service-ca-operator"/"service-ca-operator-dockercfg-rg9jl" Dec 03 11:20:47 crc kubenswrapper[5052]: I1203 11:20:47.484470 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/5fe485a1-e14f-4c09-b5b9-f252bc42b7e8-nginx-conf\") pod \"networking-console-plugin-85b44fc459-gdk6g\" (UID: \"5fe485a1-e14f-4c09-b5b9-f252bc42b7e8\") " pod="openshift-network-console/networking-console-plugin-85b44fc459-gdk6g" Dec 03 11:20:47 crc kubenswrapper[5052]: I1203 11:20:47.487483 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/5fe485a1-e14f-4c09-b5b9-f252bc42b7e8-networking-console-plugin-cert\") pod \"networking-console-plugin-85b44fc459-gdk6g\" (UID: \"5fe485a1-e14f-4c09-b5b9-f252bc42b7e8\") " pod="openshift-network-console/networking-console-plugin-85b44fc459-gdk6g" Dec 03 11:20:47 crc kubenswrapper[5052]: I1203 11:20:47.489565 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-cqllr\" (UniqueName: \"kubernetes.io/projected/3b6479f0-333b-4a96-9adf-2099afdc2447-kube-api-access-cqllr\") pod \"network-check-target-xd92c\" (UID: \"3b6479f0-333b-4a96-9adf-2099afdc2447\") " pod="openshift-network-diagnostics/network-check-target-xd92c" Dec 03 11:20:47 crc kubenswrapper[5052]: I1203 11:20:47.491006 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-s2dwl\" (UniqueName: \"kubernetes.io/projected/9d751cbb-f2e2-430d-9754-c882a5e924a5-kube-api-access-s2dwl\") pod \"network-check-source-55646444c4-trplf\" (UID: \"9d751cbb-f2e2-430d-9754-c882a5e924a5\") " pod="openshift-network-diagnostics/network-check-source-55646444c4-trplf" Dec 03 11:20:47 crc kubenswrapper[5052]: I1203 11:20:47.503807 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-service-ca-operator"/"serving-cert" Dec 03 11:20:47 crc kubenswrapper[5052]: I1203 11:20:47.523966 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-service-ca-operator"/"service-ca-operator-config" Dec 03 11:20:47 crc kubenswrapper[5052]: I1203 11:20:47.544066 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-service-ca-operator"/"kube-root-ca.crt" Dec 03 11:20:47 crc kubenswrapper[5052]: I1203 11:20:47.578824 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-x6dnv\" (UniqueName: \"kubernetes.io/projected/8fae1d72-fba4-4999-bea5-39d6472d7ea7-kube-api-access-x6dnv\") pod \"apiserver-7bbb656c7d-86vx8\" (UID: \"8fae1d72-fba4-4999-bea5-39d6472d7ea7\") " pod="openshift-oauth-apiserver/apiserver-7bbb656c7d-86vx8" Dec 03 11:20:47 crc kubenswrapper[5052]: I1203 11:20:47.584980 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\") pod \"8f668bae-612b-4b75-9490-919e737c6a3b\" (UID: \"8f668bae-612b-4b75-9490-919e737c6a3b\") " Dec 03 11:20:47 crc kubenswrapper[5052]: I1203 11:20:47.585188 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"secret-volume\" (UniqueName: \"kubernetes.io/secret/e43f14fc-f3eb-4f58-b202-cdfc641ac08a-secret-volume\") pod \"collect-profiles-29412675-hzd4n\" (UID: \"e43f14fc-f3eb-4f58-b202-cdfc641ac08a\") " pod="openshift-operator-lifecycle-manager/collect-profiles-29412675-hzd4n" Dec 03 11:20:47 crc kubenswrapper[5052]: E1203 11:20:47.585257 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8 podName:8f668bae-612b-4b75-9490-919e737c6a3b nodeName:}" failed. No retries permitted until 2025-12-03 11:20:48.085222956 +0000 UTC m=+38.673464740 (durationBeforeRetry 500ms). Error: UnmountVolume.TearDown failed for volume "pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8" (UniqueName: "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8") pod "8f668bae-612b-4b75-9490-919e737c6a3b" (UID: "8f668bae-612b-4b75-9490-919e737c6a3b") : kubernetes.io/csi: Unmounter.TearDownAt failed to get CSI client: driver name kubevirt.io.hostpath-provisioner not found in the list of registered CSI drivers Dec 03 11:20:47 crc kubenswrapper[5052]: I1203 11:20:47.585320 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"srv-cert\" (UniqueName: \"kubernetes.io/secret/8e3a111c-05ff-4f33-b5ba-81dae833b529-srv-cert\") pod \"catalog-operator-68c6474976-rqhkg\" (UID: \"8e3a111c-05ff-4f33-b5ba-81dae833b529\") " pod="openshift-operator-lifecycle-manager/catalog-operator-68c6474976-rqhkg" Dec 03 11:20:47 crc kubenswrapper[5052]: I1203 11:20:47.585387 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"profile-collector-cert\" (UniqueName: \"kubernetes.io/secret/8e3a111c-05ff-4f33-b5ba-81dae833b529-profile-collector-cert\") pod \"catalog-operator-68c6474976-rqhkg\" (UID: \"8e3a111c-05ff-4f33-b5ba-81dae833b529\") " pod="openshift-operator-lifecycle-manager/catalog-operator-68c6474976-rqhkg" Dec 03 11:20:47 crc kubenswrapper[5052]: I1203 11:20:47.585578 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"control-plane-machine-set-operator-tls\" (UniqueName: \"kubernetes.io/secret/693b46e6-42c3-4d07-bfbd-bd83d6982199-control-plane-machine-set-operator-tls\") pod \"control-plane-machine-set-operator-78cbb6b69f-p4b4l\" (UID: \"693b46e6-42c3-4d07-bfbd-bd83d6982199\") " pod="openshift-machine-api/control-plane-machine-set-operator-78cbb6b69f-p4b4l" Dec 03 11:20:47 crc kubenswrapper[5052]: I1203 11:20:47.585666 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"profile-collector-cert\" (UniqueName: \"kubernetes.io/secret/663b7426-fde0-4b59-b857-b434b4a88f4b-profile-collector-cert\") pod \"olm-operator-6b444d44fb-6lm8x\" (UID: \"663b7426-fde0-4b59-b857-b434b4a88f4b\") " pod="openshift-operator-lifecycle-manager/olm-operator-6b444d44fb-6lm8x" Dec 03 11:20:47 crc kubenswrapper[5052]: I1203 11:20:47.585729 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6413eb91-7b61-4660-888c-14da224c7dfa-webhook-cert\") pod \"packageserver-d55dfcdfc-7s97q\" (UID: \"6413eb91-7b61-4660-888c-14da224c7dfa\") " pod="openshift-operator-lifecycle-manager/packageserver-d55dfcdfc-7s97q" Dec 03 11:20:47 crc kubenswrapper[5052]: I1203 11:20:47.585902 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3e0657c6-3f3c-4300-8cfb-b3db773dba75-config\") pod \"service-ca-operator-777779d784-9jvh6\" (UID: \"3e0657c6-3f3c-4300-8cfb-b3db773dba75\") " pod="openshift-service-ca-operator/service-ca-operator-777779d784-9jvh6" Dec 03 11:20:47 crc kubenswrapper[5052]: I1203 11:20:47.586217 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3e0657c6-3f3c-4300-8cfb-b3db773dba75-serving-cert\") pod \"service-ca-operator-777779d784-9jvh6\" (UID: \"3e0657c6-3f3c-4300-8cfb-b3db773dba75\") " pod="openshift-service-ca-operator/service-ca-operator-777779d784-9jvh6" Dec 03 11:20:47 crc kubenswrapper[5052]: I1203 11:20:47.586242 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/6413eb91-7b61-4660-888c-14da224c7dfa-apiservice-cert\") pod \"packageserver-d55dfcdfc-7s97q\" (UID: \"6413eb91-7b61-4660-888c-14da224c7dfa\") " pod="openshift-operator-lifecycle-manager/packageserver-d55dfcdfc-7s97q" Dec 03 11:20:47 crc kubenswrapper[5052]: I1203 11:20:47.586270 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/eb07cd6b-71c7-4dc7-b658-40f32c63f373-signing-cabundle\") pod \"service-ca-9c57cc56f-w7g95\" (UID: \"eb07cd6b-71c7-4dc7-b658-40f32c63f373\") " pod="openshift-service-ca/service-ca-9c57cc56f-w7g95" Dec 03 11:20:47 crc kubenswrapper[5052]: I1203 11:20:47.586801 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/e43f14fc-f3eb-4f58-b202-cdfc641ac08a-config-volume\") pod \"collect-profiles-29412675-hzd4n\" (UID: \"e43f14fc-f3eb-4f58-b202-cdfc641ac08a\") " pod="openshift-operator-lifecycle-manager/collect-profiles-29412675-hzd4n" Dec 03 11:20:47 crc kubenswrapper[5052]: I1203 11:20:47.586867 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\") pod \"image-registry-697d97f7c8-p77bx\" (UID: \"f5012254-068c-4360-b310-39ff3d309199\") " pod="openshift-image-registry/image-registry-697d97f7c8-p77bx" Dec 03 11:20:47 crc kubenswrapper[5052]: I1203 11:20:47.586868 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3e0657c6-3f3c-4300-8cfb-b3db773dba75-config\") pod \"service-ca-operator-777779d784-9jvh6\" (UID: \"3e0657c6-3f3c-4300-8cfb-b3db773dba75\") " pod="openshift-service-ca-operator/service-ca-operator-777779d784-9jvh6" Dec 03 11:20:47 crc kubenswrapper[5052]: I1203 11:20:47.586896 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/42cd7f06-f712-41e0-9fa5-c8ec9f7af997-config\") pod \"openshift-apiserver-operator-796bbdcf4f-jskzg\" (UID: \"42cd7f06-f712-41e0-9fa5-c8ec9f7af997\") " pod="openshift-apiserver-operator/openshift-apiserver-operator-796bbdcf4f-jskzg" Dec 03 11:20:47 crc kubenswrapper[5052]: I1203 11:20:47.586992 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/eb07cd6b-71c7-4dc7-b658-40f32c63f373-signing-key\") pod \"service-ca-9c57cc56f-w7g95\" (UID: \"eb07cd6b-71c7-4dc7-b658-40f32c63f373\") " pod="openshift-service-ca/service-ca-9c57cc56f-w7g95" Dec 03 11:20:47 crc kubenswrapper[5052]: I1203 11:20:47.587035 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/42cd7f06-f712-41e0-9fa5-c8ec9f7af997-serving-cert\") pod \"openshift-apiserver-operator-796bbdcf4f-jskzg\" (UID: \"42cd7f06-f712-41e0-9fa5-c8ec9f7af997\") " pod="openshift-apiserver-operator/openshift-apiserver-operator-796bbdcf4f-jskzg" Dec 03 11:20:47 crc kubenswrapper[5052]: I1203 11:20:47.587071 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"srv-cert\" (UniqueName: \"kubernetes.io/secret/663b7426-fde0-4b59-b857-b434b4a88f4b-srv-cert\") pod \"olm-operator-6b444d44fb-6lm8x\" (UID: \"663b7426-fde0-4b59-b857-b434b4a88f4b\") " pod="openshift-operator-lifecycle-manager/olm-operator-6b444d44fb-6lm8x" Dec 03 11:20:47 crc kubenswrapper[5052]: I1203 11:20:47.587690 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/eb07cd6b-71c7-4dc7-b658-40f32c63f373-signing-cabundle\") pod \"service-ca-9c57cc56f-w7g95\" (UID: \"eb07cd6b-71c7-4dc7-b658-40f32c63f373\") " pod="openshift-service-ca/service-ca-9c57cc56f-w7g95" Dec 03 11:20:47 crc kubenswrapper[5052]: I1203 11:20:47.587734 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/42cd7f06-f712-41e0-9fa5-c8ec9f7af997-config\") pod \"openshift-apiserver-operator-796bbdcf4f-jskzg\" (UID: \"42cd7f06-f712-41e0-9fa5-c8ec9f7af997\") " pod="openshift-apiserver-operator/openshift-apiserver-operator-796bbdcf4f-jskzg" Dec 03 11:20:47 crc kubenswrapper[5052]: I1203 11:20:47.587992 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/e43f14fc-f3eb-4f58-b202-cdfc641ac08a-config-volume\") pod \"collect-profiles-29412675-hzd4n\" (UID: \"e43f14fc-f3eb-4f58-b202-cdfc641ac08a\") " pod="openshift-operator-lifecycle-manager/collect-profiles-29412675-hzd4n" Dec 03 11:20:47 crc kubenswrapper[5052]: E1203 11:20:47.588139 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8 podName: nodeName:}" failed. No retries permitted until 2025-12-03 11:20:48.088117915 +0000 UTC m=+38.676359689 (durationBeforeRetry 500ms). Error: MountVolume.MountDevice failed for volume "pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8" (UniqueName: "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8") pod "image-registry-697d97f7c8-p77bx" (UID: "f5012254-068c-4360-b310-39ff3d309199") : kubernetes.io/csi: attacher.MountDevice failed to create newCsiDriverClient: driver name kubevirt.io.hostpath-provisioner not found in the list of registered CSI drivers Dec 03 11:20:47 crc kubenswrapper[5052]: I1203 11:20:47.589086 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3e0657c6-3f3c-4300-8cfb-b3db773dba75-serving-cert\") pod \"service-ca-operator-777779d784-9jvh6\" (UID: \"3e0657c6-3f3c-4300-8cfb-b3db773dba75\") " pod="openshift-service-ca-operator/service-ca-operator-777779d784-9jvh6" Dec 03 11:20:47 crc kubenswrapper[5052]: I1203 11:20:47.589461 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"secret-volume\" (UniqueName: \"kubernetes.io/secret/e43f14fc-f3eb-4f58-b202-cdfc641ac08a-secret-volume\") pod \"collect-profiles-29412675-hzd4n\" (UID: \"e43f14fc-f3eb-4f58-b202-cdfc641ac08a\") " pod="openshift-operator-lifecycle-manager/collect-profiles-29412675-hzd4n" Dec 03 11:20:47 crc kubenswrapper[5052]: I1203 11:20:47.589758 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6413eb91-7b61-4660-888c-14da224c7dfa-webhook-cert\") pod \"packageserver-d55dfcdfc-7s97q\" (UID: \"6413eb91-7b61-4660-888c-14da224c7dfa\") " pod="openshift-operator-lifecycle-manager/packageserver-d55dfcdfc-7s97q" Dec 03 11:20:47 crc kubenswrapper[5052]: I1203 11:20:47.589940 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"profile-collector-cert\" (UniqueName: \"kubernetes.io/secret/663b7426-fde0-4b59-b857-b434b4a88f4b-profile-collector-cert\") pod \"olm-operator-6b444d44fb-6lm8x\" (UID: \"663b7426-fde0-4b59-b857-b434b4a88f4b\") " pod="openshift-operator-lifecycle-manager/olm-operator-6b444d44fb-6lm8x" Dec 03 11:20:47 crc kubenswrapper[5052]: I1203 11:20:47.590434 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/6413eb91-7b61-4660-888c-14da224c7dfa-apiservice-cert\") pod \"packageserver-d55dfcdfc-7s97q\" (UID: \"6413eb91-7b61-4660-888c-14da224c7dfa\") " pod="openshift-operator-lifecycle-manager/packageserver-d55dfcdfc-7s97q" Dec 03 11:20:47 crc kubenswrapper[5052]: I1203 11:20:47.590567 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xd92c" Dec 03 11:20:47 crc kubenswrapper[5052]: I1203 11:20:47.591018 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"profile-collector-cert\" (UniqueName: \"kubernetes.io/secret/8e3a111c-05ff-4f33-b5ba-81dae833b529-profile-collector-cert\") pod \"catalog-operator-68c6474976-rqhkg\" (UID: \"8e3a111c-05ff-4f33-b5ba-81dae833b529\") " pod="openshift-operator-lifecycle-manager/catalog-operator-68c6474976-rqhkg" Dec 03 11:20:47 crc kubenswrapper[5052]: I1203 11:20:47.591190 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"srv-cert\" (UniqueName: \"kubernetes.io/secret/8e3a111c-05ff-4f33-b5ba-81dae833b529-srv-cert\") pod \"catalog-operator-68c6474976-rqhkg\" (UID: \"8e3a111c-05ff-4f33-b5ba-81dae833b529\") " pod="openshift-operator-lifecycle-manager/catalog-operator-68c6474976-rqhkg" Dec 03 11:20:47 crc kubenswrapper[5052]: I1203 11:20:47.591601 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"srv-cert\" (UniqueName: \"kubernetes.io/secret/663b7426-fde0-4b59-b857-b434b4a88f4b-srv-cert\") pod \"olm-operator-6b444d44fb-6lm8x\" (UID: \"663b7426-fde0-4b59-b857-b434b4a88f4b\") " pod="openshift-operator-lifecycle-manager/olm-operator-6b444d44fb-6lm8x" Dec 03 11:20:47 crc kubenswrapper[5052]: I1203 11:20:47.592844 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"control-plane-machine-set-operator-tls\" (UniqueName: \"kubernetes.io/secret/693b46e6-42c3-4d07-bfbd-bd83d6982199-control-plane-machine-set-operator-tls\") pod \"control-plane-machine-set-operator-78cbb6b69f-p4b4l\" (UID: \"693b46e6-42c3-4d07-bfbd-bd83d6982199\") " pod="openshift-machine-api/control-plane-machine-set-operator-78cbb6b69f-p4b4l" Dec 03 11:20:47 crc kubenswrapper[5052]: I1203 11:20:47.593888 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/eb07cd6b-71c7-4dc7-b658-40f32c63f373-signing-key\") pod \"service-ca-9c57cc56f-w7g95\" (UID: \"eb07cd6b-71c7-4dc7-b658-40f32c63f373\") " pod="openshift-service-ca/service-ca-9c57cc56f-w7g95" Dec 03 11:20:47 crc kubenswrapper[5052]: I1203 11:20:47.597150 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/42cd7f06-f712-41e0-9fa5-c8ec9f7af997-serving-cert\") pod \"openshift-apiserver-operator-796bbdcf4f-jskzg\" (UID: \"42cd7f06-f712-41e0-9fa5-c8ec9f7af997\") " pod="openshift-apiserver-operator/openshift-apiserver-operator-796bbdcf4f-jskzg" Dec 03 11:20:47 crc kubenswrapper[5052]: I1203 11:20:47.598690 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/407f9c0b-88d2-46f9-9f47-d3af857b0f30-bound-sa-token\") pod \"cluster-image-registry-operator-dc59b4c8b-4v96x\" (UID: \"407f9c0b-88d2-46f9-9f47-d3af857b0f30\") " pod="openshift-image-registry/cluster-image-registry-operator-dc59b4c8b-4v96x" Dec 03 11:20:47 crc kubenswrapper[5052]: I1203 11:20:47.618798 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-nv988\" (UniqueName: \"kubernetes.io/projected/353b0eac-95ba-420e-bf67-a26b472fcdc2-kube-api-access-nv988\") pod \"route-controller-manager-6576b87f9c-9fpqn\" (UID: \"353b0eac-95ba-420e-bf67-a26b472fcdc2\") " pod="openshift-route-controller-manager/route-controller-manager-6576b87f9c-9fpqn" Dec 03 11:20:47 crc kubenswrapper[5052]: I1203 11:20:47.639326 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-8vm6t\" (UniqueName: \"kubernetes.io/projected/cabc2063-04d9-4491-be9f-f7781b5f02cf-kube-api-access-8vm6t\") pod \"dns-operator-744455d44c-zvvfc\" (UID: \"cabc2063-04d9-4491-be9f-f7781b5f02cf\") " pod="openshift-dns-operator/dns-operator-744455d44c-zvvfc" Dec 03 11:20:47 crc kubenswrapper[5052]: I1203 11:20:47.658092 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-mh48n\" (UniqueName: \"kubernetes.io/projected/59e1467d-d303-4f32-a968-b66eb94953af-kube-api-access-mh48n\") pod \"cluster-samples-operator-665b6dd947-7wkt6\" (UID: \"59e1467d-d303-4f32-a968-b66eb94953af\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-665b6dd947-7wkt6" Dec 03 11:20:47 crc kubenswrapper[5052]: I1203 11:20:47.685433 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-t9z59\" (UniqueName: \"kubernetes.io/projected/407f9c0b-88d2-46f9-9f47-d3af857b0f30-kube-api-access-t9z59\") pod \"cluster-image-registry-operator-dc59b4c8b-4v96x\" (UID: \"407f9c0b-88d2-46f9-9f47-d3af857b0f30\") " pod="openshift-image-registry/cluster-image-registry-operator-dc59b4c8b-4v96x" Dec 03 11:20:47 crc kubenswrapper[5052]: I1203 11:20:47.688903 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\") pod \"8f668bae-612b-4b75-9490-919e737c6a3b\" (UID: \"8f668bae-612b-4b75-9490-919e737c6a3b\") " Dec 03 11:20:47 crc kubenswrapper[5052]: E1203 11:20:47.689129 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8 podName:8f668bae-612b-4b75-9490-919e737c6a3b nodeName:}" failed. No retries permitted until 2025-12-03 11:20:48.189096038 +0000 UTC m=+38.777337802 (durationBeforeRetry 500ms). Error: UnmountVolume.TearDown failed for volume "pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8" (UniqueName: "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8") pod "8f668bae-612b-4b75-9490-919e737c6a3b" (UID: "8f668bae-612b-4b75-9490-919e737c6a3b") : kubernetes.io/csi: Unmounter.TearDownAt failed to get CSI client: driver name kubevirt.io.hostpath-provisioner not found in the list of registered CSI drivers Dec 03 11:20:47 crc kubenswrapper[5052]: I1203 11:20:47.689518 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\") pod \"image-registry-697d97f7c8-p77bx\" (UID: \"f5012254-068c-4360-b310-39ff3d309199\") " pod="openshift-image-registry/image-registry-697d97f7c8-p77bx" Dec 03 11:20:47 crc kubenswrapper[5052]: E1203 11:20:47.690397 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8 podName: nodeName:}" failed. No retries permitted until 2025-12-03 11:20:48.190377254 +0000 UTC m=+38.778619018 (durationBeforeRetry 500ms). Error: MountVolume.MountDevice failed for volume "pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8" (UniqueName: "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8") pod "image-registry-697d97f7c8-p77bx" (UID: "f5012254-068c-4360-b310-39ff3d309199") : kubernetes.io/csi: attacher.MountDevice failed to create newCsiDriverClient: driver name kubevirt.io.hostpath-provisioner not found in the list of registered CSI drivers Dec 03 11:20:47 crc kubenswrapper[5052]: I1203 11:20:47.698898 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-c68wk\" (UniqueName: \"kubernetes.io/projected/41016a7b-cc79-4f32-860c-cb75e405cf42-kube-api-access-c68wk\") pod \"apiserver-76f77b778f-dmvst\" (UID: \"41016a7b-cc79-4f32-860c-cb75e405cf42\") " pod="openshift-apiserver/apiserver-76f77b778f-dmvst" Dec 03 11:20:47 crc kubenswrapper[5052]: I1203 11:20:47.735172 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-route-controller-manager/route-controller-manager-6576b87f9c-9fpqn" Dec 03 11:20:47 crc kubenswrapper[5052]: I1203 11:20:47.740486 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-ng77f\" (UniqueName: \"kubernetes.io/projected/e0bf52c7-bf23-4c65-b7cb-0d8d466aa8f2-kube-api-access-ng77f\") pod \"authentication-operator-69f744f599-qjq5f\" (UID: \"e0bf52c7-bf23-4c65-b7cb-0d8d466aa8f2\") " pod="openshift-authentication-operator/authentication-operator-69f744f599-qjq5f" Dec 03 11:20:47 crc kubenswrapper[5052]: I1203 11:20:47.746285 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns-operator/dns-operator-744455d44c-zvvfc" Dec 03 11:20:47 crc kubenswrapper[5052]: I1203 11:20:47.755479 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-oauth-apiserver/apiserver-7bbb656c7d-86vx8" Dec 03 11:20:47 crc kubenswrapper[5052]: I1203 11:20:47.762372 5052 request.go:700] Waited for 1.899624061s due to client-side throttling, not priority and fairness, request: POST:https://api-int.crc.testing:6443/api/v1/namespaces/openshift-ingress-operator/serviceaccounts/ingress-operator/token Dec 03 11:20:47 crc kubenswrapper[5052]: I1203 11:20:47.767866 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-55646444c4-trplf" Dec 03 11:20:47 crc kubenswrapper[5052]: I1203 11:20:47.773809 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-665b6dd947-7wkt6" Dec 03 11:20:47 crc kubenswrapper[5052]: I1203 11:20:47.778047 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-85b44fc459-gdk6g" Dec 03 11:20:47 crc kubenswrapper[5052]: I1203 11:20:47.786282 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/833bb55c-5c9b-450c-9287-30f146a2d16a-bound-sa-token\") pod \"ingress-operator-5b745b69d9-hc8vv\" (UID: \"833bb55c-5c9b-450c-9287-30f146a2d16a\") " pod="openshift-ingress-operator/ingress-operator-5b745b69d9-hc8vv" Dec 03 11:20:47 crc kubenswrapper[5052]: I1203 11:20:47.790621 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\") pod \"8f668bae-612b-4b75-9490-919e737c6a3b\" (UID: \"8f668bae-612b-4b75-9490-919e737c6a3b\") " Dec 03 11:20:47 crc kubenswrapper[5052]: E1203 11:20:47.790786 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8 podName:8f668bae-612b-4b75-9490-919e737c6a3b nodeName:}" failed. No retries permitted until 2025-12-03 11:20:48.29076006 +0000 UTC m=+38.879001824 (durationBeforeRetry 500ms). Error: UnmountVolume.TearDown failed for volume "pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8" (UniqueName: "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8") pod "8f668bae-612b-4b75-9490-919e737c6a3b" (UID: "8f668bae-612b-4b75-9490-919e737c6a3b") : kubernetes.io/csi: Unmounter.TearDownAt failed to get CSI client: driver name kubevirt.io.hostpath-provisioner not found in the list of registered CSI drivers Dec 03 11:20:47 crc kubenswrapper[5052]: I1203 11:20:47.791014 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\") pod \"image-registry-697d97f7c8-p77bx\" (UID: \"f5012254-068c-4360-b310-39ff3d309199\") " pod="openshift-image-registry/image-registry-697d97f7c8-p77bx" Dec 03 11:20:47 crc kubenswrapper[5052]: E1203 11:20:47.791440 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8 podName: nodeName:}" failed. No retries permitted until 2025-12-03 11:20:48.291425398 +0000 UTC m=+38.879667162 (durationBeforeRetry 500ms). Error: MountVolume.MountDevice failed for volume "pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8" (UniqueName: "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8") pod "image-registry-697d97f7c8-p77bx" (UID: "f5012254-068c-4360-b310-39ff3d309199") : kubernetes.io/csi: attacher.MountDevice failed to create newCsiDriverClient: driver name kubevirt.io.hostpath-provisioner not found in the list of registered CSI drivers Dec 03 11:20:47 crc kubenswrapper[5052]: I1203 11:20:47.799760 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-gz6dn\" (UniqueName: \"kubernetes.io/projected/efa98345-cafa-483a-976b-d772a3bd94e7-kube-api-access-gz6dn\") pod \"machine-config-controller-84d6567774-rxwdl\" (UID: \"efa98345-cafa-483a-976b-d772a3bd94e7\") " pod="openshift-machine-config-operator/machine-config-controller-84d6567774-rxwdl" Dec 03 11:20:47 crc kubenswrapper[5052]: I1203 11:20:47.805574 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-multus"/"cni-sysctl-allowlist" Dec 03 11:20:47 crc kubenswrapper[5052]: I1203 11:20:47.824819 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-dns"/"dns-dockercfg-jwfmh" Dec 03 11:20:47 crc kubenswrapper[5052]: I1203 11:20:47.846054 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-dns"/"dns-default" Dec 03 11:20:47 crc kubenswrapper[5052]: I1203 11:20:47.858814 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/cluster-image-registry-operator-dc59b4c8b-4v96x" Dec 03 11:20:47 crc kubenswrapper[5052]: I1203 11:20:47.864679 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-dns"/"dns-default-metrics-tls" Dec 03 11:20:47 crc kubenswrapper[5052]: I1203 11:20:47.887869 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-ingress-canary"/"openshift-service-ca.crt" Dec 03 11:20:47 crc kubenswrapper[5052]: I1203 11:20:47.901382 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\") pod \"8f668bae-612b-4b75-9490-919e737c6a3b\" (UID: \"8f668bae-612b-4b75-9490-919e737c6a3b\") " Dec 03 11:20:47 crc kubenswrapper[5052]: E1203 11:20:47.902174 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8 podName:8f668bae-612b-4b75-9490-919e737c6a3b nodeName:}" failed. No retries permitted until 2025-12-03 11:20:48.40214958 +0000 UTC m=+38.990391344 (durationBeforeRetry 500ms). Error: UnmountVolume.TearDown failed for volume "pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8" (UniqueName: "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8") pod "8f668bae-612b-4b75-9490-919e737c6a3b" (UID: "8f668bae-612b-4b75-9490-919e737c6a3b") : kubernetes.io/csi: Unmounter.TearDownAt failed to get CSI client: driver name kubevirt.io.hostpath-provisioner not found in the list of registered CSI drivers Dec 03 11:20:47 crc kubenswrapper[5052]: I1203 11:20:47.905502 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-ingress-canary"/"default-dockercfg-2llfx" Dec 03 11:20:47 crc kubenswrapper[5052]: I1203 11:20:47.926664 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-ingress-canary"/"canary-serving-cert" Dec 03 11:20:47 crc kubenswrapper[5052]: I1203 11:20:47.947057 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-ingress-canary"/"kube-root-ca.crt" Dec 03 11:20:47 crc kubenswrapper[5052]: I1203 11:20:47.964800 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-machine-config-operator"/"machine-config-server-tls" Dec 03 11:20:47 crc kubenswrapper[5052]: I1203 11:20:47.973987 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-xd92c" event={"ID":"3b6479f0-333b-4a96-9adf-2099afdc2447","Type":"ContainerStarted","Data":"7a80435c81d945ee222a59a9ac15292978e41f1659b14c2128f8da4ca451268c"} Dec 03 11:20:47 crc kubenswrapper[5052]: I1203 11:20:47.984805 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"hostpath-provisioner"/"openshift-service-ca.crt" Dec 03 11:20:48 crc kubenswrapper[5052]: I1203 11:20:48.003979 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\") pod \"image-registry-697d97f7c8-p77bx\" (UID: \"f5012254-068c-4360-b310-39ff3d309199\") " pod="openshift-image-registry/image-registry-697d97f7c8-p77bx" Dec 03 11:20:48 crc kubenswrapper[5052]: I1203 11:20:48.003983 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-machine-config-operator"/"machine-config-server-dockercfg-qx5rd" Dec 03 11:20:48 crc kubenswrapper[5052]: E1203 11:20:48.004575 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8 podName: nodeName:}" failed. No retries permitted until 2025-12-03 11:20:48.504539642 +0000 UTC m=+39.092781396 (durationBeforeRetry 500ms). Error: MountVolume.MountDevice failed for volume "pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8" (UniqueName: "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8") pod "image-registry-697d97f7c8-p77bx" (UID: "f5012254-068c-4360-b310-39ff3d309199") : kubernetes.io/csi: attacher.MountDevice failed to create newCsiDriverClient: driver name kubevirt.io.hostpath-provisioner not found in the list of registered CSI drivers Dec 03 11:20:48 crc kubenswrapper[5052]: I1203 11:20:48.006812 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-route-controller-manager/route-controller-manager-6576b87f9c-9fpqn"] Dec 03 11:20:48 crc kubenswrapper[5052]: W1203 11:20:48.020989 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod353b0eac_95ba_420e_bf67_a26b472fcdc2.slice/crio-ac445a00288813c296aca72aa0f9283d19329a7fb1a7ba6f66cddb2fe3f5494f WatchSource:0}: Error finding container ac445a00288813c296aca72aa0f9283d19329a7fb1a7ba6f66cddb2fe3f5494f: Status 404 returned error can't find the container with id ac445a00288813c296aca72aa0f9283d19329a7fb1a7ba6f66cddb2fe3f5494f Dec 03 11:20:48 crc kubenswrapper[5052]: I1203 11:20:48.021848 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/machine-config-controller-84d6567774-rxwdl" Dec 03 11:20:48 crc kubenswrapper[5052]: I1203 11:20:48.025191 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-machine-config-operator"/"node-bootstrapper-token" Dec 03 11:20:48 crc kubenswrapper[5052]: I1203 11:20:48.046529 5052 reflector.go:368] Caches populated for *v1.Secret from object-"hostpath-provisioner"/"csi-hostpath-provisioner-sa-dockercfg-qd74k" Dec 03 11:20:48 crc kubenswrapper[5052]: I1203 11:20:48.053227 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-oauth-apiserver/apiserver-7bbb656c7d-86vx8"] Dec 03 11:20:48 crc kubenswrapper[5052]: I1203 11:20:48.063869 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"hostpath-provisioner"/"kube-root-ca.crt" Dec 03 11:20:48 crc kubenswrapper[5052]: I1203 11:20:48.104703 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\") pod \"8f668bae-612b-4b75-9490-919e737c6a3b\" (UID: \"8f668bae-612b-4b75-9490-919e737c6a3b\") " Dec 03 11:20:48 crc kubenswrapper[5052]: E1203 11:20:48.105373 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8 podName:8f668bae-612b-4b75-9490-919e737c6a3b nodeName:}" failed. No retries permitted until 2025-12-03 11:20:48.60534214 +0000 UTC m=+39.193583904 (durationBeforeRetry 500ms). Error: UnmountVolume.TearDown failed for volume "pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8" (UniqueName: "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8") pod "8f668bae-612b-4b75-9490-919e737c6a3b" (UID: "8f668bae-612b-4b75-9490-919e737c6a3b") : kubernetes.io/csi: Unmounter.TearDownAt failed to get CSI client: driver name kubevirt.io.hostpath-provisioner not found in the list of registered CSI drivers Dec 03 11:20:48 crc kubenswrapper[5052]: I1203 11:20:48.107608 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-665b6dd947-7wkt6"] Dec 03 11:20:48 crc kubenswrapper[5052]: I1203 11:20:48.120823 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f5012254-068c-4360-b310-39ff3d309199-bound-sa-token\") pod \"image-registry-697d97f7c8-p77bx\" (UID: \"f5012254-068c-4360-b310-39ff3d309199\") " pod="openshift-image-registry/image-registry-697d97f7c8-p77bx" Dec 03 11:20:48 crc kubenswrapper[5052]: I1203 11:20:48.133995 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/cluster-image-registry-operator-dc59b4c8b-4v96x"] Dec 03 11:20:48 crc kubenswrapper[5052]: I1203 11:20:48.139543 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-cvfz9\" (UniqueName: \"kubernetes.io/projected/f5012254-068c-4360-b310-39ff3d309199-kube-api-access-cvfz9\") pod \"image-registry-697d97f7c8-p77bx\" (UID: \"f5012254-068c-4360-b310-39ff3d309199\") " pod="openshift-image-registry/image-registry-697d97f7c8-p77bx" Dec 03 11:20:48 crc kubenswrapper[5052]: I1203 11:20:48.162917 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-4cgvv\" (UniqueName: \"kubernetes.io/projected/cac70dab-6e13-4693-ac66-c7552b62dfd7-kube-api-access-4cgvv\") pod \"marketplace-operator-79b997595-kdr8l\" (UID: \"cac70dab-6e13-4693-ac66-c7552b62dfd7\") " pod="openshift-marketplace/marketplace-operator-79b997595-kdr8l" Dec 03 11:20:48 crc kubenswrapper[5052]: I1203 11:20:48.180806 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-ztr7w\" (UniqueName: \"kubernetes.io/projected/0a110335-c2e5-40a2-b565-a4791146d53b-kube-api-access-ztr7w\") pod \"openshift-config-operator-7777fb866f-bkztv\" (UID: \"0a110335-c2e5-40a2-b565-a4791146d53b\") " pod="openshift-config-operator/openshift-config-operator-7777fb866f-bkztv" Dec 03 11:20:48 crc kubenswrapper[5052]: I1203 11:20:48.198480 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-h4gph\" (UniqueName: \"kubernetes.io/projected/cc99b3a0-fa04-4128-b730-1d24da63d08e-kube-api-access-h4gph\") pod \"machine-config-operator-74547568cd-tr2xc\" (UID: \"cc99b3a0-fa04-4128-b730-1d24da63d08e\") " pod="openshift-machine-config-operator/machine-config-operator-74547568cd-tr2xc" Dec 03 11:20:48 crc kubenswrapper[5052]: I1203 11:20:48.206772 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\") pod \"image-registry-697d97f7c8-p77bx\" (UID: \"f5012254-068c-4360-b310-39ff3d309199\") " pod="openshift-image-registry/image-registry-697d97f7c8-p77bx" Dec 03 11:20:48 crc kubenswrapper[5052]: E1203 11:20:48.207277 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8 podName: nodeName:}" failed. No retries permitted until 2025-12-03 11:20:48.707245898 +0000 UTC m=+39.295487672 (durationBeforeRetry 500ms). Error: MountVolume.MountDevice failed for volume "pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8" (UniqueName: "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8") pod "image-registry-697d97f7c8-p77bx" (UID: "f5012254-068c-4360-b310-39ff3d309199") : kubernetes.io/csi: attacher.MountDevice failed to create newCsiDriverClient: driver name kubevirt.io.hostpath-provisioner not found in the list of registered CSI drivers Dec 03 11:20:48 crc kubenswrapper[5052]: I1203 11:20:48.226084 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-l5k2l\" (UniqueName: \"kubernetes.io/projected/75f62784-7818-4a74-9c8c-4777d6db0611-kube-api-access-l5k2l\") pod \"oauth-openshift-558db77b4-99jsx\" (UID: \"75f62784-7818-4a74-9c8c-4777d6db0611\") " pod="openshift-authentication/oauth-openshift-558db77b4-99jsx" Dec 03 11:20:48 crc kubenswrapper[5052]: I1203 11:20:48.236625 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-dns-operator/dns-operator-744455d44c-zvvfc"] Dec 03 11:20:48 crc kubenswrapper[5052]: I1203 11:20:48.262078 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-xkxbr\" (UniqueName: \"kubernetes.io/projected/ec2ea657-018b-4626-bf79-5c818f573a4b-kube-api-access-xkxbr\") pod \"downloads-7954f5f757-l7xtl\" (UID: \"ec2ea657-018b-4626-bf79-5c818f573a4b\") " pod="openshift-console/downloads-7954f5f757-l7xtl" Dec 03 11:20:48 crc kubenswrapper[5052]: I1203 11:20:48.283618 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-7tbg4\" (UniqueName: \"kubernetes.io/projected/8c1b25f5-2d98-4852-bc7a-d5cbfa5cb6aa-kube-api-access-7tbg4\") pod \"machine-approver-56656f9798-6t9dp\" (UID: \"8c1b25f5-2d98-4852-bc7a-d5cbfa5cb6aa\") " pod="openshift-cluster-machine-approver/machine-approver-56656f9798-6t9dp" Dec 03 11:20:48 crc kubenswrapper[5052]: I1203 11:20:48.298362 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-c67q4\" (UniqueName: \"kubernetes.io/projected/3afb93c1-9cbc-4588-a3bf-2ae1e774a2d1-kube-api-access-c67q4\") pod \"console-operator-58897d9998-7vpns\" (UID: \"3afb93c1-9cbc-4588-a3bf-2ae1e774a2d1\") " pod="openshift-console-operator/console-operator-58897d9998-7vpns" Dec 03 11:20:48 crc kubenswrapper[5052]: I1203 11:20:48.308102 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\") pod \"8f668bae-612b-4b75-9490-919e737c6a3b\" (UID: \"8f668bae-612b-4b75-9490-919e737c6a3b\") " Dec 03 11:20:48 crc kubenswrapper[5052]: E1203 11:20:48.308547 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8 podName:8f668bae-612b-4b75-9490-919e737c6a3b nodeName:}" failed. No retries permitted until 2025-12-03 11:20:48.808521639 +0000 UTC m=+39.396763413 (durationBeforeRetry 500ms). Error: UnmountVolume.TearDown failed for volume "pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8" (UniqueName: "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8") pod "8f668bae-612b-4b75-9490-919e737c6a3b" (UID: "8f668bae-612b-4b75-9490-919e737c6a3b") : kubernetes.io/csi: Unmounter.TearDownAt failed to get CSI client: driver name kubevirt.io.hostpath-provisioner not found in the list of registered CSI drivers Dec 03 11:20:48 crc kubenswrapper[5052]: I1203 11:20:48.308799 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\") pod \"image-registry-697d97f7c8-p77bx\" (UID: \"f5012254-068c-4360-b310-39ff3d309199\") " pod="openshift-image-registry/image-registry-697d97f7c8-p77bx" Dec 03 11:20:48 crc kubenswrapper[5052]: E1203 11:20:48.309212 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8 podName: nodeName:}" failed. No retries permitted until 2025-12-03 11:20:48.809202128 +0000 UTC m=+39.397443892 (durationBeforeRetry 500ms). Error: MountVolume.MountDevice failed for volume "pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8" (UniqueName: "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8") pod "image-registry-697d97f7c8-p77bx" (UID: "f5012254-068c-4360-b310-39ff3d309199") : kubernetes.io/csi: attacher.MountDevice failed to create newCsiDriverClient: driver name kubevirt.io.hostpath-provisioner not found in the list of registered CSI drivers Dec 03 11:20:48 crc kubenswrapper[5052]: I1203 11:20:48.314690 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/machine-config-operator-74547568cd-tr2xc" Dec 03 11:20:48 crc kubenswrapper[5052]: I1203 11:20:48.321493 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-ljgxw\" (UniqueName: \"kubernetes.io/projected/43fca84c-422e-4e3b-b3ca-b787a4e3f677-kube-api-access-ljgxw\") pod \"openshift-controller-manager-operator-756b6f6bc6-v7lmf\" (UID: \"43fca84c-422e-4e3b-b3ca-b787a4e3f677\") " pod="openshift-controller-manager-operator/openshift-controller-manager-operator-756b6f6bc6-v7lmf" Dec 03 11:20:48 crc kubenswrapper[5052]: I1203 11:20:48.342811 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-mdldv\" (UniqueName: \"kubernetes.io/projected/8e3a111c-05ff-4f33-b5ba-81dae833b529-kube-api-access-mdldv\") pod \"catalog-operator-68c6474976-rqhkg\" (UID: \"8e3a111c-05ff-4f33-b5ba-81dae833b529\") " pod="openshift-operator-lifecycle-manager/catalog-operator-68c6474976-rqhkg" Dec 03 11:20:48 crc kubenswrapper[5052]: I1203 11:20:48.345840 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/marketplace-operator-79b997595-kdr8l" Dec 03 11:20:48 crc kubenswrapper[5052]: I1203 11:20:48.379142 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-znx74\" (UniqueName: \"kubernetes.io/projected/42cd7f06-f712-41e0-9fa5-c8ec9f7af997-kube-api-access-znx74\") pod \"openshift-apiserver-operator-796bbdcf4f-jskzg\" (UID: \"42cd7f06-f712-41e0-9fa5-c8ec9f7af997\") " pod="openshift-apiserver-operator/openshift-apiserver-operator-796bbdcf4f-jskzg" Dec 03 11:20:48 crc kubenswrapper[5052]: E1203 11:20:48.381980 5052 secret.go:188] Couldn't get secret openshift-apiserver/etcd-client: failed to sync secret cache: timed out waiting for the condition Dec 03 11:20:48 crc kubenswrapper[5052]: E1203 11:20:48.382064 5052 configmap.go:193] Couldn't get configMap openshift-apiserver/etcd-serving-ca: failed to sync configmap cache: timed out waiting for the condition Dec 03 11:20:48 crc kubenswrapper[5052]: E1203 11:20:48.382056 5052 configmap.go:193] Couldn't get configMap openshift-authentication-operator/authentication-operator-config: failed to sync configmap cache: timed out waiting for the condition Dec 03 11:20:48 crc kubenswrapper[5052]: E1203 11:20:48.382121 5052 secret.go:188] Couldn't get secret openshift-authentication-operator/serving-cert: failed to sync secret cache: timed out waiting for the condition Dec 03 11:20:48 crc kubenswrapper[5052]: E1203 11:20:48.382121 5052 configmap.go:193] Couldn't get configMap openshift-apiserver/image-import-ca: failed to sync configmap cache: timed out waiting for the condition Dec 03 11:20:48 crc kubenswrapper[5052]: E1203 11:20:48.382128 5052 secret.go:188] Couldn't get secret openshift-apiserver/serving-cert: failed to sync secret cache: timed out waiting for the condition Dec 03 11:20:48 crc kubenswrapper[5052]: E1203 11:20:48.382820 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/41016a7b-cc79-4f32-860c-cb75e405cf42-etcd-client podName:41016a7b-cc79-4f32-860c-cb75e405cf42 nodeName:}" failed. No retries permitted until 2025-12-03 11:20:49.382749755 +0000 UTC m=+39.970991519 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "etcd-client" (UniqueName: "kubernetes.io/secret/41016a7b-cc79-4f32-860c-cb75e405cf42-etcd-client") pod "apiserver-76f77b778f-dmvst" (UID: "41016a7b-cc79-4f32-860c-cb75e405cf42") : failed to sync secret cache: timed out waiting for the condition Dec 03 11:20:48 crc kubenswrapper[5052]: E1203 11:20:48.383474 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/41016a7b-cc79-4f32-860c-cb75e405cf42-etcd-serving-ca podName:41016a7b-cc79-4f32-860c-cb75e405cf42 nodeName:}" failed. No retries permitted until 2025-12-03 11:20:49.383416263 +0000 UTC m=+39.971658027 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "etcd-serving-ca" (UniqueName: "kubernetes.io/configmap/41016a7b-cc79-4f32-860c-cb75e405cf42-etcd-serving-ca") pod "apiserver-76f77b778f-dmvst" (UID: "41016a7b-cc79-4f32-860c-cb75e405cf42") : failed to sync configmap cache: timed out waiting for the condition Dec 03 11:20:48 crc kubenswrapper[5052]: E1203 11:20:48.383583 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e0bf52c7-bf23-4c65-b7cb-0d8d466aa8f2-config podName:e0bf52c7-bf23-4c65-b7cb-0d8d466aa8f2 nodeName:}" failed. No retries permitted until 2025-12-03 11:20:49.383571697 +0000 UTC m=+39.971813461 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "config" (UniqueName: "kubernetes.io/configmap/e0bf52c7-bf23-4c65-b7cb-0d8d466aa8f2-config") pod "authentication-operator-69f744f599-qjq5f" (UID: "e0bf52c7-bf23-4c65-b7cb-0d8d466aa8f2") : failed to sync configmap cache: timed out waiting for the condition Dec 03 11:20:48 crc kubenswrapper[5052]: E1203 11:20:48.383670 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e0bf52c7-bf23-4c65-b7cb-0d8d466aa8f2-serving-cert podName:e0bf52c7-bf23-4c65-b7cb-0d8d466aa8f2 nodeName:}" failed. No retries permitted until 2025-12-03 11:20:49.383654219 +0000 UTC m=+39.971895983 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "serving-cert" (UniqueName: "kubernetes.io/secret/e0bf52c7-bf23-4c65-b7cb-0d8d466aa8f2-serving-cert") pod "authentication-operator-69f744f599-qjq5f" (UID: "e0bf52c7-bf23-4c65-b7cb-0d8d466aa8f2") : failed to sync secret cache: timed out waiting for the condition Dec 03 11:20:48 crc kubenswrapper[5052]: E1203 11:20:48.383756 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/41016a7b-cc79-4f32-860c-cb75e405cf42-image-import-ca podName:41016a7b-cc79-4f32-860c-cb75e405cf42 nodeName:}" failed. No retries permitted until 2025-12-03 11:20:49.383745532 +0000 UTC m=+39.971987296 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "image-import-ca" (UniqueName: "kubernetes.io/configmap/41016a7b-cc79-4f32-860c-cb75e405cf42-image-import-ca") pod "apiserver-76f77b778f-dmvst" (UID: "41016a7b-cc79-4f32-860c-cb75e405cf42") : failed to sync configmap cache: timed out waiting for the condition Dec 03 11:20:48 crc kubenswrapper[5052]: E1203 11:20:48.383842 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/41016a7b-cc79-4f32-860c-cb75e405cf42-serving-cert podName:41016a7b-cc79-4f32-860c-cb75e405cf42 nodeName:}" failed. No retries permitted until 2025-12-03 11:20:49.383831984 +0000 UTC m=+39.972073758 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "serving-cert" (UniqueName: "kubernetes.io/secret/41016a7b-cc79-4f32-860c-cb75e405cf42-serving-cert") pod "apiserver-76f77b778f-dmvst" (UID: "41016a7b-cc79-4f32-860c-cb75e405cf42") : failed to sync secret cache: timed out waiting for the condition Dec 03 11:20:48 crc kubenswrapper[5052]: I1203 11:20:48.399449 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access\" (UniqueName: \"kubernetes.io/projected/287ad4d4-bf40-4cc4-9856-580b65a2e88d-kube-api-access\") pod \"kube-apiserver-operator-766d6c64bb-xwlk8\" (UID: \"287ad4d4-bf40-4cc4-9856-580b65a2e88d\") " pod="openshift-kube-apiserver-operator/kube-apiserver-operator-766d6c64bb-xwlk8" Dec 03 11:20:48 crc kubenswrapper[5052]: I1203 11:20:48.401864 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-7954f5f757-l7xtl" Dec 03 11:20:48 crc kubenswrapper[5052]: I1203 11:20:48.410262 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\") pod \"8f668bae-612b-4b75-9490-919e737c6a3b\" (UID: \"8f668bae-612b-4b75-9490-919e737c6a3b\") " Dec 03 11:20:48 crc kubenswrapper[5052]: E1203 11:20:48.410412 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8 podName:8f668bae-612b-4b75-9490-919e737c6a3b nodeName:}" failed. No retries permitted until 2025-12-03 11:20:48.910395096 +0000 UTC m=+39.498636860 (durationBeforeRetry 500ms). Error: UnmountVolume.TearDown failed for volume "pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8" (UniqueName: "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8") pod "8f668bae-612b-4b75-9490-919e737c6a3b" (UID: "8f668bae-612b-4b75-9490-919e737c6a3b") : kubernetes.io/csi: Unmounter.TearDownAt failed to get CSI client: driver name kubevirt.io.hostpath-provisioner not found in the list of registered CSI drivers Dec 03 11:20:48 crc kubenswrapper[5052]: I1203 11:20:48.410794 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\") pod \"image-registry-697d97f7c8-p77bx\" (UID: \"f5012254-068c-4360-b310-39ff3d309199\") " pod="openshift-image-registry/image-registry-697d97f7c8-p77bx" Dec 03 11:20:48 crc kubenswrapper[5052]: E1203 11:20:48.411344 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8 podName: nodeName:}" failed. No retries permitted until 2025-12-03 11:20:48.911321692 +0000 UTC m=+39.499563456 (durationBeforeRetry 500ms). Error: MountVolume.MountDevice failed for volume "pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8" (UniqueName: "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8") pod "image-registry-697d97f7c8-p77bx" (UID: "f5012254-068c-4360-b310-39ff3d309199") : kubernetes.io/csi: attacher.MountDevice failed to create newCsiDriverClient: driver name kubevirt.io.hostpath-provisioner not found in the list of registered CSI drivers Dec 03 11:20:48 crc kubenswrapper[5052]: I1203 11:20:48.418022 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-xxzql\" (UniqueName: \"kubernetes.io/projected/71ea0624-9fb5-415a-96e1-d5ada83c0fe4-kube-api-access-xxzql\") pod \"etcd-operator-b45778765-z2lp6\" (UID: \"71ea0624-9fb5-415a-96e1-d5ada83c0fe4\") " pod="openshift-etcd-operator/etcd-operator-b45778765-z2lp6" Dec 03 11:20:48 crc kubenswrapper[5052]: I1203 11:20:48.418285 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-machine-approver/machine-approver-56656f9798-6t9dp" Dec 03 11:20:48 crc kubenswrapper[5052]: I1203 11:20:48.424948 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-controller-manager-operator/openshift-controller-manager-operator-756b6f6bc6-v7lmf" Dec 03 11:20:48 crc kubenswrapper[5052]: I1203 11:20:48.431569 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-58897d9998-7vpns" Dec 03 11:20:48 crc kubenswrapper[5052]: I1203 11:20:48.438172 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-zdbwv\" (UniqueName: \"kubernetes.io/projected/3e0657c6-3f3c-4300-8cfb-b3db773dba75-kube-api-access-zdbwv\") pod \"service-ca-operator-777779d784-9jvh6\" (UID: \"3e0657c6-3f3c-4300-8cfb-b3db773dba75\") " pod="openshift-service-ca-operator/service-ca-operator-777779d784-9jvh6" Dec 03 11:20:48 crc kubenswrapper[5052]: I1203 11:20:48.445349 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-config-operator/openshift-config-operator-7777fb866f-bkztv" Dec 03 11:20:48 crc kubenswrapper[5052]: I1203 11:20:48.454540 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-apiserver-operator/kube-apiserver-operator-766d6c64bb-xwlk8" Dec 03 11:20:48 crc kubenswrapper[5052]: I1203 11:20:48.456271 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-88b82\" (UniqueName: \"kubernetes.io/projected/eb07cd6b-71c7-4dc7-b658-40f32c63f373-kube-api-access-88b82\") pod \"service-ca-9c57cc56f-w7g95\" (UID: \"eb07cd6b-71c7-4dc7-b658-40f32c63f373\") " pod="openshift-service-ca/service-ca-9c57cc56f-w7g95" Dec 03 11:20:48 crc kubenswrapper[5052]: I1203 11:20:48.462014 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-etcd-operator/etcd-operator-b45778765-z2lp6" Dec 03 11:20:48 crc kubenswrapper[5052]: I1203 11:20:48.482298 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-4q7pc\" (UniqueName: \"kubernetes.io/projected/693b46e6-42c3-4d07-bfbd-bd83d6982199-kube-api-access-4q7pc\") pod \"control-plane-machine-set-operator-78cbb6b69f-p4b4l\" (UID: \"693b46e6-42c3-4d07-bfbd-bd83d6982199\") " pod="openshift-machine-api/control-plane-machine-set-operator-78cbb6b69f-p4b4l" Dec 03 11:20:48 crc kubenswrapper[5052]: I1203 11:20:48.482799 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-authentication/oauth-openshift-558db77b4-99jsx" Dec 03 11:20:48 crc kubenswrapper[5052]: E1203 11:20:48.484096 5052 secret.go:188] Couldn't get secret openshift-image-registry/installation-pull-secrets: failed to sync secret cache: timed out waiting for the condition Dec 03 11:20:48 crc kubenswrapper[5052]: E1203 11:20:48.484253 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f5012254-068c-4360-b310-39ff3d309199-installation-pull-secrets podName:f5012254-068c-4360-b310-39ff3d309199 nodeName:}" failed. No retries permitted until 2025-12-03 11:20:49.484230191 +0000 UTC m=+40.072471965 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "installation-pull-secrets" (UniqueName: "kubernetes.io/secret/f5012254-068c-4360-b310-39ff3d309199-installation-pull-secrets") pod "image-registry-697d97f7c8-p77bx" (UID: "f5012254-068c-4360-b310-39ff3d309199") : failed to sync secret cache: timed out waiting for the condition Dec 03 11:20:48 crc kubenswrapper[5052]: I1203 11:20:48.500836 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-vkn2m\" (UniqueName: \"kubernetes.io/projected/48040744-7fc1-482e-9b68-3a904ba03a8d-kube-api-access-vkn2m\") pod \"router-default-5444994796-rtl8t\" (UID: \"48040744-7fc1-482e-9b68-3a904ba03a8d\") " pod="openshift-ingress/router-default-5444994796-rtl8t" Dec 03 11:20:48 crc kubenswrapper[5052]: I1203 11:20:48.503659 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-apiserver-operator/openshift-apiserver-operator-796bbdcf4f-jskzg" Dec 03 11:20:48 crc kubenswrapper[5052]: I1203 11:20:48.512237 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\") pod \"8f668bae-612b-4b75-9490-919e737c6a3b\" (UID: \"8f668bae-612b-4b75-9490-919e737c6a3b\") " Dec 03 11:20:48 crc kubenswrapper[5052]: E1203 11:20:48.512835 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8 podName:8f668bae-612b-4b75-9490-919e737c6a3b nodeName:}" failed. No retries permitted until 2025-12-03 11:20:49.012821719 +0000 UTC m=+39.601063483 (durationBeforeRetry 500ms). Error: UnmountVolume.TearDown failed for volume "pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8" (UniqueName: "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8") pod "8f668bae-612b-4b75-9490-919e737c6a3b" (UID: "8f668bae-612b-4b75-9490-919e737c6a3b") : kubernetes.io/csi: Unmounter.TearDownAt failed to get CSI client: driver name kubevirt.io.hostpath-provisioner not found in the list of registered CSI drivers Dec 03 11:20:48 crc kubenswrapper[5052]: I1203 11:20:48.513018 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-operator-lifecycle-manager/catalog-operator-68c6474976-rqhkg" Dec 03 11:20:48 crc kubenswrapper[5052]: I1203 11:20:48.524428 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-apiserver"/"serving-cert" Dec 03 11:20:48 crc kubenswrapper[5052]: I1203 11:20:48.537943 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-5s2zd\" (UniqueName: \"kubernetes.io/projected/663b7426-fde0-4b59-b857-b434b4a88f4b-kube-api-access-5s2zd\") pod \"olm-operator-6b444d44fb-6lm8x\" (UID: \"663b7426-fde0-4b59-b857-b434b4a88f4b\") " pod="openshift-operator-lifecycle-manager/olm-operator-6b444d44fb-6lm8x" Dec 03 11:20:48 crc kubenswrapper[5052]: I1203 11:20:48.543834 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-authentication-operator"/"serving-cert" Dec 03 11:20:48 crc kubenswrapper[5052]: I1203 11:20:48.547385 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-9c57cc56f-w7g95" Dec 03 11:20:48 crc kubenswrapper[5052]: I1203 11:20:48.556983 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-operator-lifecycle-manager/olm-operator-6b444d44fb-6lm8x" Dec 03 11:20:48 crc kubenswrapper[5052]: I1203 11:20:48.564000 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-image-registry"/"installation-pull-secrets" Dec 03 11:20:48 crc kubenswrapper[5052]: I1203 11:20:48.574938 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-api/control-plane-machine-set-operator-78cbb6b69f-p4b4l" Dec 03 11:20:48 crc kubenswrapper[5052]: I1203 11:20:48.585201 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-apiserver"/"etcd-client" Dec 03 11:20:48 crc kubenswrapper[5052]: I1203 11:20:48.585542 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-777779d784-9jvh6" Dec 03 11:20:48 crc kubenswrapper[5052]: I1203 11:20:48.604220 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-ingress-operator"/"openshift-service-ca.crt" Dec 03 11:20:48 crc kubenswrapper[5052]: I1203 11:20:48.614444 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\") pod \"image-registry-697d97f7c8-p77bx\" (UID: \"f5012254-068c-4360-b310-39ff3d309199\") " pod="openshift-image-registry/image-registry-697d97f7c8-p77bx" Dec 03 11:20:48 crc kubenswrapper[5052]: E1203 11:20:48.615256 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8 podName: nodeName:}" failed. No retries permitted until 2025-12-03 11:20:49.115225731 +0000 UTC m=+39.703467525 (durationBeforeRetry 500ms). Error: MountVolume.MountDevice failed for volume "pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8" (UniqueName: "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8") pod "image-registry-697d97f7c8-p77bx" (UID: "f5012254-068c-4360-b310-39ff3d309199") : kubernetes.io/csi: attacher.MountDevice failed to create newCsiDriverClient: driver name kubevirt.io.hostpath-provisioner not found in the list of registered CSI drivers Dec 03 11:20:48 crc kubenswrapper[5052]: I1203 11:20:48.617689 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-qxsnm\" (UniqueName: \"kubernetes.io/projected/833bb55c-5c9b-450c-9287-30f146a2d16a-kube-api-access-qxsnm\") pod \"ingress-operator-5b745b69d9-hc8vv\" (UID: \"833bb55c-5c9b-450c-9287-30f146a2d16a\") " pod="openshift-ingress-operator/ingress-operator-5b745b69d9-hc8vv" Dec 03 11:20:48 crc kubenswrapper[5052]: I1203 11:20:48.624564 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-apiserver"/"etcd-serving-ca" Dec 03 11:20:48 crc kubenswrapper[5052]: I1203 11:20:48.643615 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-authentication-operator"/"authentication-operator-config" Dec 03 11:20:48 crc kubenswrapper[5052]: I1203 11:20:48.667703 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-apiserver"/"image-import-ca" Dec 03 11:20:48 crc kubenswrapper[5052]: I1203 11:20:48.716264 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\") pod \"8f668bae-612b-4b75-9490-919e737c6a3b\" (UID: \"8f668bae-612b-4b75-9490-919e737c6a3b\") " Dec 03 11:20:48 crc kubenswrapper[5052]: E1203 11:20:48.716503 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8 podName:8f668bae-612b-4b75-9490-919e737c6a3b nodeName:}" failed. No retries permitted until 2025-12-03 11:20:49.216458 +0000 UTC m=+39.804699774 (durationBeforeRetry 500ms). Error: UnmountVolume.TearDown failed for volume "pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8" (UniqueName: "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8") pod "8f668bae-612b-4b75-9490-919e737c6a3b" (UID: "8f668bae-612b-4b75-9490-919e737c6a3b") : kubernetes.io/csi: Unmounter.TearDownAt failed to get CSI client: driver name kubevirt.io.hostpath-provisioner not found in the list of registered CSI drivers Dec 03 11:20:48 crc kubenswrapper[5052]: I1203 11:20:48.716819 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b1d2ff08-d58b-48d0-a0e5-6cfd867a63de-service-ca\") pod \"console-f9d7485db-568dj\" (UID: \"b1d2ff08-d58b-48d0-a0e5-6cfd867a63de\") " pod="openshift-console/console-f9d7485db-568dj" Dec 03 11:20:48 crc kubenswrapper[5052]: I1203 11:20:48.716865 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w4cbl\" (UniqueName: \"kubernetes.io/projected/b1d2ff08-d58b-48d0-a0e5-6cfd867a63de-kube-api-access-w4cbl\") pod \"console-f9d7485db-568dj\" (UID: \"b1d2ff08-d58b-48d0-a0e5-6cfd867a63de\") " pod="openshift-console/console-f9d7485db-568dj" Dec 03 11:20:48 crc kubenswrapper[5052]: I1203 11:20:48.717024 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-krszl\" (UniqueName: \"kubernetes.io/projected/199a472c-800b-45e2-a507-7e6cb2c7427f-kube-api-access-krszl\") pod \"migrator-59844c95c7-4qts8\" (UID: \"199a472c-800b-45e2-a507-7e6cb2c7427f\") " pod="openshift-kube-storage-version-migrator/migrator-59844c95c7-4qts8" Dec 03 11:20:48 crc kubenswrapper[5052]: I1203 11:20:48.717128 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j4nkw\" (UniqueName: \"kubernetes.io/projected/efbeac4d-779b-4eb2-acf5-241c83a46eb7-kube-api-access-j4nkw\") pod \"dns-default-6m2jn\" (UID: \"efbeac4d-779b-4eb2-acf5-241c83a46eb7\") " pod="openshift-dns/dns-default-6m2jn" Dec 03 11:20:48 crc kubenswrapper[5052]: I1203 11:20:48.717201 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-ca-bundles\" (UniqueName: \"kubernetes.io/configmap/1c570f3f-3090-48e1-9b7e-f66c99310dbd-proxy-ca-bundles\") pod \"controller-manager-879f6c89f-46rm8\" (UID: \"1c570f3f-3090-48e1-9b7e-f66c99310dbd\") " pod="openshift-controller-manager/controller-manager-879f6c89f-46rm8" Dec 03 11:20:48 crc kubenswrapper[5052]: I1203 11:20:48.717243 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1c570f3f-3090-48e1-9b7e-f66c99310dbd-serving-cert\") pod \"controller-manager-879f6c89f-46rm8\" (UID: \"1c570f3f-3090-48e1-9b7e-f66c99310dbd\") " pod="openshift-controller-manager/controller-manager-879f6c89f-46rm8" Dec 03 11:20:48 crc kubenswrapper[5052]: I1203 11:20:48.717302 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3dd904b5-266d-4a49-ad58-805139eef151-config\") pod \"kube-controller-manager-operator-78b949d7b-h6b78\" (UID: \"3dd904b5-266d-4a49-ad58-805139eef151\") " pod="openshift-kube-controller-manager-operator/kube-controller-manager-operator-78b949d7b-h6b78" Dec 03 11:20:48 crc kubenswrapper[5052]: I1203 11:20:48.717342 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hmnh8\" (UniqueName: \"kubernetes.io/projected/daf5492e-2286-42c8-8c08-88200b27e2fb-kube-api-access-hmnh8\") pod \"multus-admission-controller-857f4d67dd-q8n58\" (UID: \"daf5492e-2286-42c8-8c08-88200b27e2fb\") " pod="openshift-multus/multus-admission-controller-857f4d67dd-q8n58" Dec 03 11:20:48 crc kubenswrapper[5052]: I1203 11:20:48.717567 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1c570f3f-3090-48e1-9b7e-f66c99310dbd-config\") pod \"controller-manager-879f6c89f-46rm8\" (UID: \"1c570f3f-3090-48e1-9b7e-f66c99310dbd\") " pod="openshift-controller-manager/controller-manager-879f6c89f-46rm8" Dec 03 11:20:48 crc kubenswrapper[5052]: I1203 11:20:48.717818 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"machine-api-operator-tls\" (UniqueName: \"kubernetes.io/secret/3d483046-743f-43c6-b73d-31f4f1bcca8c-machine-api-operator-tls\") pod \"machine-api-operator-5694c8668f-l5lrg\" (UID: \"3d483046-743f-43c6-b73d-31f4f1bcca8c\") " pod="openshift-machine-api/machine-api-operator-5694c8668f-l5lrg" Dec 03 11:20:48 crc kubenswrapper[5052]: I1203 11:20:48.717882 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/efbeac4d-779b-4eb2-acf5-241c83a46eb7-metrics-tls\") pod \"dns-default-6m2jn\" (UID: \"efbeac4d-779b-4eb2-acf5-241c83a46eb7\") " pod="openshift-dns/dns-default-6m2jn" Dec 03 11:20:48 crc kubenswrapper[5052]: I1203 11:20:48.718046 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/efbeac4d-779b-4eb2-acf5-241c83a46eb7-config-volume\") pod \"dns-default-6m2jn\" (UID: \"efbeac4d-779b-4eb2-acf5-241c83a46eb7\") " pod="openshift-dns/dns-default-6m2jn" Dec 03 11:20:48 crc kubenswrapper[5052]: I1203 11:20:48.718088 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cpcp4\" (UniqueName: \"kubernetes.io/projected/3d483046-743f-43c6-b73d-31f4f1bcca8c-kube-api-access-cpcp4\") pod \"machine-api-operator-5694c8668f-l5lrg\" (UID: \"3d483046-743f-43c6-b73d-31f4f1bcca8c\") " pod="openshift-machine-api/machine-api-operator-5694c8668f-l5lrg" Dec 03 11:20:48 crc kubenswrapper[5052]: I1203 11:20:48.718256 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b1d2ff08-d58b-48d0-a0e5-6cfd867a63de-console-serving-cert\") pod \"console-f9d7485db-568dj\" (UID: \"b1d2ff08-d58b-48d0-a0e5-6cfd867a63de\") " pod="openshift-console/console-f9d7485db-568dj" Dec 03 11:20:48 crc kubenswrapper[5052]: I1203 11:20:48.718326 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3d483046-743f-43c6-b73d-31f4f1bcca8c-config\") pod \"machine-api-operator-5694c8668f-l5lrg\" (UID: \"3d483046-743f-43c6-b73d-31f4f1bcca8c\") " pod="openshift-machine-api/machine-api-operator-5694c8668f-l5lrg" Dec 03 11:20:48 crc kubenswrapper[5052]: I1203 11:20:48.718676 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b1d2ff08-d58b-48d0-a0e5-6cfd867a63de-console-config\") pod \"console-f9d7485db-568dj\" (UID: \"b1d2ff08-d58b-48d0-a0e5-6cfd867a63de\") " pod="openshift-console/console-f9d7485db-568dj" Dec 03 11:20:48 crc kubenswrapper[5052]: I1203 11:20:48.718735 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-certs\" (UniqueName: \"kubernetes.io/secret/daf5492e-2286-42c8-8c08-88200b27e2fb-webhook-certs\") pod \"multus-admission-controller-857f4d67dd-q8n58\" (UID: \"daf5492e-2286-42c8-8c08-88200b27e2fb\") " pod="openshift-multus/multus-admission-controller-857f4d67dd-q8n58" Dec 03 11:20:48 crc kubenswrapper[5052]: I1203 11:20:48.718841 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b1d2ff08-d58b-48d0-a0e5-6cfd867a63de-trusted-ca-bundle\") pod \"console-f9d7485db-568dj\" (UID: \"b1d2ff08-d58b-48d0-a0e5-6cfd867a63de\") " pod="openshift-console/console-f9d7485db-568dj" Dec 03 11:20:48 crc kubenswrapper[5052]: I1203 11:20:48.718882 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3dd904b5-266d-4a49-ad58-805139eef151-serving-cert\") pod \"kube-controller-manager-operator-78b949d7b-h6b78\" (UID: \"3dd904b5-266d-4a49-ad58-805139eef151\") " pod="openshift-kube-controller-manager-operator/kube-controller-manager-operator-78b949d7b-h6b78" Dec 03 11:20:48 crc kubenswrapper[5052]: I1203 11:20:48.719018 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b1d2ff08-d58b-48d0-a0e5-6cfd867a63de-oauth-serving-cert\") pod \"console-f9d7485db-568dj\" (UID: \"b1d2ff08-d58b-48d0-a0e5-6cfd867a63de\") " pod="openshift-console/console-f9d7485db-568dj" Dec 03 11:20:48 crc kubenswrapper[5052]: I1203 11:20:48.719075 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access\" (UniqueName: \"kubernetes.io/projected/3dd904b5-266d-4a49-ad58-805139eef151-kube-api-access\") pod \"kube-controller-manager-operator-78b949d7b-h6b78\" (UID: \"3dd904b5-266d-4a49-ad58-805139eef151\") " pod="openshift-kube-controller-manager-operator/kube-controller-manager-operator-78b949d7b-h6b78" Dec 03 11:20:48 crc kubenswrapper[5052]: I1203 11:20:48.719222 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b1d2ff08-d58b-48d0-a0e5-6cfd867a63de-console-oauth-config\") pod \"console-f9d7485db-568dj\" (UID: \"b1d2ff08-d58b-48d0-a0e5-6cfd867a63de\") " pod="openshift-console/console-f9d7485db-568dj" Dec 03 11:20:48 crc kubenswrapper[5052]: I1203 11:20:48.719489 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5d079af1-8d26-4a35-89f6-1312e1d5c2dc-config\") pod \"openshift-kube-scheduler-operator-5fdd9b5758-rxfb5\" (UID: \"5d079af1-8d26-4a35-89f6-1312e1d5c2dc\") " pod="openshift-kube-scheduler-operator/openshift-kube-scheduler-operator-5fdd9b5758-rxfb5" Dec 03 11:20:48 crc kubenswrapper[5052]: I1203 11:20:48.719792 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"images\" (UniqueName: \"kubernetes.io/configmap/3d483046-743f-43c6-b73d-31f4f1bcca8c-images\") pod \"machine-api-operator-5694c8668f-l5lrg\" (UID: \"3d483046-743f-43c6-b73d-31f4f1bcca8c\") " pod="openshift-machine-api/machine-api-operator-5694c8668f-l5lrg" Dec 03 11:20:48 crc kubenswrapper[5052]: I1203 11:20:48.719859 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rgsw9\" (UniqueName: \"kubernetes.io/projected/3f6f3c17-944e-4afb-8cd7-3ca95f787057-kube-api-access-rgsw9\") pod \"package-server-manager-789f6589d5-577rn\" (UID: \"3f6f3c17-944e-4afb-8cd7-3ca95f787057\") " pod="openshift-operator-lifecycle-manager/package-server-manager-789f6589d5-577rn" Dec 03 11:20:48 crc kubenswrapper[5052]: I1203 11:20:48.719980 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\") pod \"image-registry-697d97f7c8-p77bx\" (UID: \"f5012254-068c-4360-b310-39ff3d309199\") " pod="openshift-image-registry/image-registry-697d97f7c8-p77bx" Dec 03 11:20:48 crc kubenswrapper[5052]: I1203 11:20:48.720026 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca\" (UniqueName: \"kubernetes.io/configmap/1c570f3f-3090-48e1-9b7e-f66c99310dbd-client-ca\") pod \"controller-manager-879f6c89f-46rm8\" (UID: \"1c570f3f-3090-48e1-9b7e-f66c99310dbd\") " pod="openshift-controller-manager/controller-manager-879f6c89f-46rm8" Dec 03 11:20:48 crc kubenswrapper[5052]: I1203 11:20:48.720583 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5d079af1-8d26-4a35-89f6-1312e1d5c2dc-serving-cert\") pod \"openshift-kube-scheduler-operator-5fdd9b5758-rxfb5\" (UID: \"5d079af1-8d26-4a35-89f6-1312e1d5c2dc\") " pod="openshift-kube-scheduler-operator/openshift-kube-scheduler-operator-5fdd9b5758-rxfb5" Dec 03 11:20:48 crc kubenswrapper[5052]: I1203 11:20:48.720728 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2jbxs\" (UniqueName: \"kubernetes.io/projected/1c570f3f-3090-48e1-9b7e-f66c99310dbd-kube-api-access-2jbxs\") pod \"controller-manager-879f6c89f-46rm8\" (UID: \"1c570f3f-3090-48e1-9b7e-f66c99310dbd\") " pod="openshift-controller-manager/controller-manager-879f6c89f-46rm8" Dec 03 11:20:48 crc kubenswrapper[5052]: I1203 11:20:48.720938 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access\" (UniqueName: \"kubernetes.io/projected/5d079af1-8d26-4a35-89f6-1312e1d5c2dc-kube-api-access\") pod \"openshift-kube-scheduler-operator-5fdd9b5758-rxfb5\" (UID: \"5d079af1-8d26-4a35-89f6-1312e1d5c2dc\") " pod="openshift-kube-scheduler-operator/openshift-kube-scheduler-operator-5fdd9b5758-rxfb5" Dec 03 11:20:48 crc kubenswrapper[5052]: I1203 11:20:48.721043 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"package-server-manager-serving-cert\" (UniqueName: \"kubernetes.io/secret/3f6f3c17-944e-4afb-8cd7-3ca95f787057-package-server-manager-serving-cert\") pod \"package-server-manager-789f6589d5-577rn\" (UID: \"3f6f3c17-944e-4afb-8cd7-3ca95f787057\") " pod="openshift-operator-lifecycle-manager/package-server-manager-789f6589d5-577rn" Dec 03 11:20:48 crc kubenswrapper[5052]: I1203 11:20:48.786356 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-5444994796-rtl8t" Dec 03 11:20:49 crc kubenswrapper[5052]: I1203 11:20:49.242965 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-operator/ingress-operator-5b745b69d9-hc8vv" Dec 03 11:20:49 crc kubenswrapper[5052]: I1203 11:20:49.246387 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qq2t2\" (UniqueName: \"kubernetes.io/projected/ed6dae78-3358-4be5-84c2-142e4d45fe9a-kube-api-access-qq2t2\") pod \"ingress-canary-tvr2s\" (UID: \"ed6dae78-3358-4be5-84c2-142e4d45fe9a\") " pod="openshift-ingress-canary/ingress-canary-tvr2s" Dec 03 11:20:49 crc kubenswrapper[5052]: I1203 11:20:49.246638 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b1d2ff08-d58b-48d0-a0e5-6cfd867a63de-service-ca\") pod \"console-f9d7485db-568dj\" (UID: \"b1d2ff08-d58b-48d0-a0e5-6cfd867a63de\") " pod="openshift-console/console-f9d7485db-568dj" Dec 03 11:20:49 crc kubenswrapper[5052]: I1203 11:20:49.247094 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-w4cbl\" (UniqueName: \"kubernetes.io/projected/b1d2ff08-d58b-48d0-a0e5-6cfd867a63de-kube-api-access-w4cbl\") pod \"console-f9d7485db-568dj\" (UID: \"b1d2ff08-d58b-48d0-a0e5-6cfd867a63de\") " pod="openshift-console/console-f9d7485db-568dj" Dec 03 11:20:49 crc kubenswrapper[5052]: I1203 11:20:49.247131 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-krszl\" (UniqueName: \"kubernetes.io/projected/199a472c-800b-45e2-a507-7e6cb2c7427f-kube-api-access-krszl\") pod \"migrator-59844c95c7-4qts8\" (UID: \"199a472c-800b-45e2-a507-7e6cb2c7427f\") " pod="openshift-kube-storage-version-migrator/migrator-59844c95c7-4qts8" Dec 03 11:20:49 crc kubenswrapper[5052]: I1203 11:20:49.247157 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-j4nkw\" (UniqueName: \"kubernetes.io/projected/efbeac4d-779b-4eb2-acf5-241c83a46eb7-kube-api-access-j4nkw\") pod \"dns-default-6m2jn\" (UID: \"efbeac4d-779b-4eb2-acf5-241c83a46eb7\") " pod="openshift-dns/dns-default-6m2jn" Dec 03 11:20:49 crc kubenswrapper[5052]: I1203 11:20:49.247176 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"proxy-ca-bundles\" (UniqueName: \"kubernetes.io/configmap/1c570f3f-3090-48e1-9b7e-f66c99310dbd-proxy-ca-bundles\") pod \"controller-manager-879f6c89f-46rm8\" (UID: \"1c570f3f-3090-48e1-9b7e-f66c99310dbd\") " pod="openshift-controller-manager/controller-manager-879f6c89f-46rm8" Dec 03 11:20:49 crc kubenswrapper[5052]: I1203 11:20:49.247194 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1c570f3f-3090-48e1-9b7e-f66c99310dbd-serving-cert\") pod \"controller-manager-879f6c89f-46rm8\" (UID: \"1c570f3f-3090-48e1-9b7e-f66c99310dbd\") " pod="openshift-controller-manager/controller-manager-879f6c89f-46rm8" Dec 03 11:20:49 crc kubenswrapper[5052]: I1203 11:20:49.247213 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3dd904b5-266d-4a49-ad58-805139eef151-config\") pod \"kube-controller-manager-operator-78b949d7b-h6b78\" (UID: \"3dd904b5-266d-4a49-ad58-805139eef151\") " pod="openshift-kube-controller-manager-operator/kube-controller-manager-operator-78b949d7b-h6b78" Dec 03 11:20:49 crc kubenswrapper[5052]: I1203 11:20:49.248434 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-hmnh8\" (UniqueName: \"kubernetes.io/projected/daf5492e-2286-42c8-8c08-88200b27e2fb-kube-api-access-hmnh8\") pod \"multus-admission-controller-857f4d67dd-q8n58\" (UID: \"daf5492e-2286-42c8-8c08-88200b27e2fb\") " pod="openshift-multus/multus-admission-controller-857f4d67dd-q8n58" Dec 03 11:20:49 crc kubenswrapper[5052]: I1203 11:20:49.249100 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b1d2ff08-d58b-48d0-a0e5-6cfd867a63de-service-ca\") pod \"console-f9d7485db-568dj\" (UID: \"b1d2ff08-d58b-48d0-a0e5-6cfd867a63de\") " pod="openshift-console/console-f9d7485db-568dj" Dec 03 11:20:49 crc kubenswrapper[5052]: I1203 11:20:49.249141 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1c570f3f-3090-48e1-9b7e-f66c99310dbd-config\") pod \"controller-manager-879f6c89f-46rm8\" (UID: \"1c570f3f-3090-48e1-9b7e-f66c99310dbd\") " pod="openshift-controller-manager/controller-manager-879f6c89f-46rm8" Dec 03 11:20:49 crc kubenswrapper[5052]: I1203 11:20:49.249264 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3dd904b5-266d-4a49-ad58-805139eef151-config\") pod \"kube-controller-manager-operator-78b949d7b-h6b78\" (UID: \"3dd904b5-266d-4a49-ad58-805139eef151\") " pod="openshift-kube-controller-manager-operator/kube-controller-manager-operator-78b949d7b-h6b78" Dec 03 11:20:49 crc kubenswrapper[5052]: I1203 11:20:49.249558 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"machine-api-operator-tls\" (UniqueName: \"kubernetes.io/secret/3d483046-743f-43c6-b73d-31f4f1bcca8c-machine-api-operator-tls\") pod \"machine-api-operator-5694c8668f-l5lrg\" (UID: \"3d483046-743f-43c6-b73d-31f4f1bcca8c\") " pod="openshift-machine-api/machine-api-operator-5694c8668f-l5lrg" Dec 03 11:20:49 crc kubenswrapper[5052]: I1203 11:20:49.249594 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/efbeac4d-779b-4eb2-acf5-241c83a46eb7-metrics-tls\") pod \"dns-default-6m2jn\" (UID: \"efbeac4d-779b-4eb2-acf5-241c83a46eb7\") " pod="openshift-dns/dns-default-6m2jn" Dec 03 11:20:49 crc kubenswrapper[5052]: I1203 11:20:49.249668 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-cpcp4\" (UniqueName: \"kubernetes.io/projected/3d483046-743f-43c6-b73d-31f4f1bcca8c-kube-api-access-cpcp4\") pod \"machine-api-operator-5694c8668f-l5lrg\" (UID: \"3d483046-743f-43c6-b73d-31f4f1bcca8c\") " pod="openshift-machine-api/machine-api-operator-5694c8668f-l5lrg" Dec 03 11:20:49 crc kubenswrapper[5052]: I1203 11:20:49.250950 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3d483046-743f-43c6-b73d-31f4f1bcca8c-config\") pod \"machine-api-operator-5694c8668f-l5lrg\" (UID: \"3d483046-743f-43c6-b73d-31f4f1bcca8c\") " pod="openshift-machine-api/machine-api-operator-5694c8668f-l5lrg" Dec 03 11:20:49 crc kubenswrapper[5052]: I1203 11:20:49.251786 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3d483046-743f-43c6-b73d-31f4f1bcca8c-config\") pod \"machine-api-operator-5694c8668f-l5lrg\" (UID: \"3d483046-743f-43c6-b73d-31f4f1bcca8c\") " pod="openshift-machine-api/machine-api-operator-5694c8668f-l5lrg" Dec 03 11:20:49 crc kubenswrapper[5052]: I1203 11:20:49.251883 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b1d2ff08-d58b-48d0-a0e5-6cfd867a63de-console-config\") pod \"console-f9d7485db-568dj\" (UID: \"b1d2ff08-d58b-48d0-a0e5-6cfd867a63de\") " pod="openshift-console/console-f9d7485db-568dj" Dec 03 11:20:49 crc kubenswrapper[5052]: I1203 11:20:49.251929 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"webhook-certs\" (UniqueName: \"kubernetes.io/secret/daf5492e-2286-42c8-8c08-88200b27e2fb-webhook-certs\") pod \"multus-admission-controller-857f4d67dd-q8n58\" (UID: \"daf5492e-2286-42c8-8c08-88200b27e2fb\") " pod="openshift-multus/multus-admission-controller-857f4d67dd-q8n58" Dec 03 11:20:49 crc kubenswrapper[5052]: I1203 11:20:49.251985 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b1d2ff08-d58b-48d0-a0e5-6cfd867a63de-trusted-ca-bundle\") pod \"console-f9d7485db-568dj\" (UID: \"b1d2ff08-d58b-48d0-a0e5-6cfd867a63de\") " pod="openshift-console/console-f9d7485db-568dj" Dec 03 11:20:49 crc kubenswrapper[5052]: I1203 11:20:49.253053 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b1d2ff08-d58b-48d0-a0e5-6cfd867a63de-console-config\") pod \"console-f9d7485db-568dj\" (UID: \"b1d2ff08-d58b-48d0-a0e5-6cfd867a63de\") " pod="openshift-console/console-f9d7485db-568dj" Dec 03 11:20:49 crc kubenswrapper[5052]: I1203 11:20:49.254554 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/efbeac4d-779b-4eb2-acf5-241c83a46eb7-metrics-tls\") pod \"dns-default-6m2jn\" (UID: \"efbeac4d-779b-4eb2-acf5-241c83a46eb7\") " pod="openshift-dns/dns-default-6m2jn" Dec 03 11:20:49 crc kubenswrapper[5052]: I1203 11:20:49.254655 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1c570f3f-3090-48e1-9b7e-f66c99310dbd-config\") pod \"controller-manager-879f6c89f-46rm8\" (UID: \"1c570f3f-3090-48e1-9b7e-f66c99310dbd\") " pod="openshift-controller-manager/controller-manager-879f6c89f-46rm8" Dec 03 11:20:49 crc kubenswrapper[5052]: I1203 11:20:49.255641 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3dd904b5-266d-4a49-ad58-805139eef151-serving-cert\") pod \"kube-controller-manager-operator-78b949d7b-h6b78\" (UID: \"3dd904b5-266d-4a49-ad58-805139eef151\") " pod="openshift-kube-controller-manager-operator/kube-controller-manager-operator-78b949d7b-h6b78" Dec 03 11:20:49 crc kubenswrapper[5052]: I1203 11:20:49.256153 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"proxy-ca-bundles\" (UniqueName: \"kubernetes.io/configmap/1c570f3f-3090-48e1-9b7e-f66c99310dbd-proxy-ca-bundles\") pod \"controller-manager-879f6c89f-46rm8\" (UID: \"1c570f3f-3090-48e1-9b7e-f66c99310dbd\") " pod="openshift-controller-manager/controller-manager-879f6c89f-46rm8" Dec 03 11:20:49 crc kubenswrapper[5052]: I1203 11:20:49.257872 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"machine-api-operator-tls\" (UniqueName: \"kubernetes.io/secret/3d483046-743f-43c6-b73d-31f4f1bcca8c-machine-api-operator-tls\") pod \"machine-api-operator-5694c8668f-l5lrg\" (UID: \"3d483046-743f-43c6-b73d-31f4f1bcca8c\") " pod="openshift-machine-api/machine-api-operator-5694c8668f-l5lrg" Dec 03 11:20:49 crc kubenswrapper[5052]: I1203 11:20:49.259181 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-qpvck\" (UniqueName: \"kubernetes.io/projected/6413eb91-7b61-4660-888c-14da224c7dfa-kube-api-access-qpvck\") pod \"packageserver-d55dfcdfc-7s97q\" (UID: \"6413eb91-7b61-4660-888c-14da224c7dfa\") " pod="openshift-operator-lifecycle-manager/packageserver-d55dfcdfc-7s97q" Dec 03 11:20:49 crc kubenswrapper[5052]: E1203 11:20:49.260258 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8 podName: nodeName:}" failed. No retries permitted until 2025-12-03 11:20:49.760233456 +0000 UTC m=+40.348475300 (durationBeforeRetry 500ms). Error: MountVolume.MountDevice failed for volume "pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8" (UniqueName: "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8") pod "image-registry-697d97f7c8-p77bx" (UID: "f5012254-068c-4360-b310-39ff3d309199") : kubernetes.io/csi: attacher.MountDevice failed to create newCsiDriverClient: driver name kubevirt.io.hostpath-provisioner not found in the list of registered CSI drivers Dec 03 11:20:49 crc kubenswrapper[5052]: I1203 11:20:49.260882 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b1d2ff08-d58b-48d0-a0e5-6cfd867a63de-oauth-serving-cert\") pod \"console-f9d7485db-568dj\" (UID: \"b1d2ff08-d58b-48d0-a0e5-6cfd867a63de\") " pod="openshift-console/console-f9d7485db-568dj" Dec 03 11:20:49 crc kubenswrapper[5052]: I1203 11:20:49.261309 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b1d2ff08-d58b-48d0-a0e5-6cfd867a63de-console-oauth-config\") pod \"console-f9d7485db-568dj\" (UID: \"b1d2ff08-d58b-48d0-a0e5-6cfd867a63de\") " pod="openshift-console/console-f9d7485db-568dj" Dec 03 11:20:49 crc kubenswrapper[5052]: I1203 11:20:49.261425 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access\" (UniqueName: \"kubernetes.io/projected/3dd904b5-266d-4a49-ad58-805139eef151-kube-api-access\") pod \"kube-controller-manager-operator-78b949d7b-h6b78\" (UID: \"3dd904b5-266d-4a49-ad58-805139eef151\") " pod="openshift-kube-controller-manager-operator/kube-controller-manager-operator-78b949d7b-h6b78" Dec 03 11:20:49 crc kubenswrapper[5052]: I1203 11:20:49.264153 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b1d2ff08-d58b-48d0-a0e5-6cfd867a63de-trusted-ca-bundle\") pod \"console-f9d7485db-568dj\" (UID: \"b1d2ff08-d58b-48d0-a0e5-6cfd867a63de\") " pod="openshift-console/console-f9d7485db-568dj" Dec 03 11:20:49 crc kubenswrapper[5052]: I1203 11:20:49.265668 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/bf761cb9-e024-4acd-a947-0f8064267243-cni-sysctl-allowlist\") pod \"cni-sysctl-allowlist-ds-8l9gm\" (UID: \"bf761cb9-e024-4acd-a947-0f8064267243\") " pod="openshift-multus/cni-sysctl-allowlist-ds-8l9gm" Dec 03 11:20:49 crc kubenswrapper[5052]: I1203 11:20:49.265733 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ed6dae78-3358-4be5-84c2-142e4d45fe9a-cert\") pod \"ingress-canary-tvr2s\" (UID: \"ed6dae78-3358-4be5-84c2-142e4d45fe9a\") " pod="openshift-ingress-canary/ingress-canary-tvr2s" Dec 03 11:20:49 crc kubenswrapper[5052]: I1203 11:20:49.265771 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"images\" (UniqueName: \"kubernetes.io/configmap/3d483046-743f-43c6-b73d-31f4f1bcca8c-images\") pod \"machine-api-operator-5694c8668f-l5lrg\" (UID: \"3d483046-743f-43c6-b73d-31f4f1bcca8c\") " pod="openshift-machine-api/machine-api-operator-5694c8668f-l5lrg" Dec 03 11:20:49 crc kubenswrapper[5052]: I1203 11:20:49.266281 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-hmnk9\" (UniqueName: \"kubernetes.io/projected/e43f14fc-f3eb-4f58-b202-cdfc641ac08a-kube-api-access-hmnk9\") pod \"collect-profiles-29412675-hzd4n\" (UID: \"e43f14fc-f3eb-4f58-b202-cdfc641ac08a\") " pod="openshift-operator-lifecycle-manager/collect-profiles-29412675-hzd4n" Dec 03 11:20:49 crc kubenswrapper[5052]: I1203 11:20:49.266405 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"images\" (UniqueName: \"kubernetes.io/configmap/3d483046-743f-43c6-b73d-31f4f1bcca8c-images\") pod \"machine-api-operator-5694c8668f-l5lrg\" (UID: \"3d483046-743f-43c6-b73d-31f4f1bcca8c\") " pod="openshift-machine-api/machine-api-operator-5694c8668f-l5lrg" Dec 03 11:20:49 crc kubenswrapper[5052]: I1203 11:20:49.266840 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b1d2ff08-d58b-48d0-a0e5-6cfd867a63de-oauth-serving-cert\") pod \"console-f9d7485db-568dj\" (UID: \"b1d2ff08-d58b-48d0-a0e5-6cfd867a63de\") " pod="openshift-console/console-f9d7485db-568dj" Dec 03 11:20:49 crc kubenswrapper[5052]: W1203 11:20:49.268801 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5fe485a1_e14f_4c09_b5b9_f252bc42b7e8.slice/crio-ee956411efbf969f87faded1406176dc05bdbd0ae40eac9eacf261a6432125b6 WatchSource:0}: Error finding container ee956411efbf969f87faded1406176dc05bdbd0ae40eac9eacf261a6432125b6: Status 404 returned error can't find the container with id ee956411efbf969f87faded1406176dc05bdbd0ae40eac9eacf261a6432125b6 Dec 03 11:20:49 crc kubenswrapper[5052]: W1203 11:20:49.271704 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod407f9c0b_88d2_46f9_9f47_d3af857b0f30.slice/crio-87d4d86647ea9d6b06f3ae5723bb9f40c7470e8c8f1b0fdb3a050825ae3a6b45 WatchSource:0}: Error finding container 87d4d86647ea9d6b06f3ae5723bb9f40c7470e8c8f1b0fdb3a050825ae3a6b45: Status 404 returned error can't find the container with id 87d4d86647ea9d6b06f3ae5723bb9f40c7470e8c8f1b0fdb3a050825ae3a6b45 Dec 03 11:20:49 crc kubenswrapper[5052]: I1203 11:20:49.273550 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1c570f3f-3090-48e1-9b7e-f66c99310dbd-serving-cert\") pod \"controller-manager-879f6c89f-46rm8\" (UID: \"1c570f3f-3090-48e1-9b7e-f66c99310dbd\") " pod="openshift-controller-manager/controller-manager-879f6c89f-46rm8" Dec 03 11:20:49 crc kubenswrapper[5052]: I1203 11:20:49.276498 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"webhook-certs\" (UniqueName: \"kubernetes.io/secret/daf5492e-2286-42c8-8c08-88200b27e2fb-webhook-certs\") pod \"multus-admission-controller-857f4d67dd-q8n58\" (UID: \"daf5492e-2286-42c8-8c08-88200b27e2fb\") " pod="openshift-multus/multus-admission-controller-857f4d67dd-q8n58" Dec 03 11:20:49 crc kubenswrapper[5052]: I1203 11:20:49.279145 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-xk4gt\" (UniqueName: \"kubernetes.io/projected/6e31bff9-7ac9-47a7-a895-db2cc3051d85-kube-api-access-xk4gt\") pod \"kube-storage-version-migrator-operator-b67b599dd-5h545\" (UID: \"6e31bff9-7ac9-47a7-a895-db2cc3051d85\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-b67b599dd-5h545" Dec 03 11:20:49 crc kubenswrapper[5052]: I1203 11:20:49.279978 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3dd904b5-266d-4a49-ad58-805139eef151-serving-cert\") pod \"kube-controller-manager-operator-78b949d7b-h6b78\" (UID: \"3dd904b5-266d-4a49-ad58-805139eef151\") " pod="openshift-kube-controller-manager-operator/kube-controller-manager-operator-78b949d7b-h6b78" Dec 03 11:20:49 crc kubenswrapper[5052]: I1203 11:20:49.282249 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-route-controller-manager/route-controller-manager-6576b87f9c-9fpqn" event={"ID":"353b0eac-95ba-420e-bf67-a26b472fcdc2","Type":"ContainerStarted","Data":"ac445a00288813c296aca72aa0f9283d19329a7fb1a7ba6f66cddb2fe3f5494f"} Dec 03 11:20:49 crc kubenswrapper[5052]: I1203 11:20:49.282551 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-w4cbl\" (UniqueName: \"kubernetes.io/projected/b1d2ff08-d58b-48d0-a0e5-6cfd867a63de-kube-api-access-w4cbl\") pod \"console-f9d7485db-568dj\" (UID: \"b1d2ff08-d58b-48d0-a0e5-6cfd867a63de\") " pod="openshift-console/console-f9d7485db-568dj" Dec 03 11:20:49 crc kubenswrapper[5052]: I1203 11:20:49.283042 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-krszl\" (UniqueName: \"kubernetes.io/projected/199a472c-800b-45e2-a507-7e6cb2c7427f-kube-api-access-krszl\") pod \"migrator-59844c95c7-4qts8\" (UID: \"199a472c-800b-45e2-a507-7e6cb2c7427f\") " pod="openshift-kube-storage-version-migrator/migrator-59844c95c7-4qts8" Dec 03 11:20:49 crc kubenswrapper[5052]: I1203 11:20:49.285046 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-55646444c4-trplf" event={"ID":"9d751cbb-f2e2-430d-9754-c882a5e924a5","Type":"ContainerStarted","Data":"d63a159f7af3651ae6d68cc8bd528d208d51101136bfffdeff7f2e7ac9bf1d20"} Dec 03 11:20:49 crc kubenswrapper[5052]: I1203 11:20:49.287133 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b1d2ff08-d58b-48d0-a0e5-6cfd867a63de-console-oauth-config\") pod \"console-f9d7485db-568dj\" (UID: \"b1d2ff08-d58b-48d0-a0e5-6cfd867a63de\") " pod="openshift-console/console-f9d7485db-568dj" Dec 03 11:20:49 crc kubenswrapper[5052]: I1203 11:20:49.288424 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-cpcp4\" (UniqueName: \"kubernetes.io/projected/3d483046-743f-43c6-b73d-31f4f1bcca8c-kube-api-access-cpcp4\") pod \"machine-api-operator-5694c8668f-l5lrg\" (UID: \"3d483046-743f-43c6-b73d-31f4f1bcca8c\") " pod="openshift-machine-api/machine-api-operator-5694c8668f-l5lrg" Dec 03 11:20:49 crc kubenswrapper[5052]: I1203 11:20:49.288493 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-oauth-apiserver/apiserver-7bbb656c7d-86vx8" event={"ID":"8fae1d72-fba4-4999-bea5-39d6472d7ea7","Type":"ContainerStarted","Data":"d011647f541c60136fb8f1d202fb08826d24098687d4c0f6bde7e771038a9705"} Dec 03 11:20:49 crc kubenswrapper[5052]: I1203 11:20:49.290566 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access\" (UniqueName: \"kubernetes.io/projected/3dd904b5-266d-4a49-ad58-805139eef151-kube-api-access\") pod \"kube-controller-manager-operator-78b949d7b-h6b78\" (UID: \"3dd904b5-266d-4a49-ad58-805139eef151\") " pod="openshift-kube-controller-manager-operator/kube-controller-manager-operator-78b949d7b-h6b78" Dec 03 11:20:49 crc kubenswrapper[5052]: I1203 11:20:49.294193 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-59844c95c7-4qts8" Dec 03 11:20:49 crc kubenswrapper[5052]: I1203 11:20:49.298323 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-j4nkw\" (UniqueName: \"kubernetes.io/projected/efbeac4d-779b-4eb2-acf5-241c83a46eb7-kube-api-access-j4nkw\") pod \"dns-default-6m2jn\" (UID: \"efbeac4d-779b-4eb2-acf5-241c83a46eb7\") " pod="openshift-dns/dns-default-6m2jn" Dec 03 11:20:49 crc kubenswrapper[5052]: I1203 11:20:49.301868 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-hmnh8\" (UniqueName: \"kubernetes.io/projected/daf5492e-2286-42c8-8c08-88200b27e2fb-kube-api-access-hmnh8\") pod \"multus-admission-controller-857f4d67dd-q8n58\" (UID: \"daf5492e-2286-42c8-8c08-88200b27e2fb\") " pod="openshift-multus/multus-admission-controller-857f4d67dd-q8n58" Dec 03 11:20:49 crc kubenswrapper[5052]: I1203 11:20:49.338455 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-admission-controller-857f4d67dd-q8n58" Dec 03 11:20:49 crc kubenswrapper[5052]: I1203 11:20:49.369697 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\") pod \"8f668bae-612b-4b75-9490-919e737c6a3b\" (UID: \"8f668bae-612b-4b75-9490-919e737c6a3b\") " Dec 03 11:20:49 crc kubenswrapper[5052]: I1203 11:20:49.369993 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-rgsw9\" (UniqueName: \"kubernetes.io/projected/3f6f3c17-944e-4afb-8cd7-3ca95f787057-kube-api-access-rgsw9\") pod \"package-server-manager-789f6589d5-577rn\" (UID: \"3f6f3c17-944e-4afb-8cd7-3ca95f787057\") " pod="openshift-operator-lifecycle-manager/package-server-manager-789f6589d5-577rn" Dec 03 11:20:49 crc kubenswrapper[5052]: I1203 11:20:49.370028 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"client-ca\" (UniqueName: \"kubernetes.io/configmap/1c570f3f-3090-48e1-9b7e-f66c99310dbd-client-ca\") pod \"controller-manager-879f6c89f-46rm8\" (UID: \"1c570f3f-3090-48e1-9b7e-f66c99310dbd\") " pod="openshift-controller-manager/controller-manager-879f6c89f-46rm8" Dec 03 11:20:49 crc kubenswrapper[5052]: I1203 11:20:49.370052 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5d079af1-8d26-4a35-89f6-1312e1d5c2dc-serving-cert\") pod \"openshift-kube-scheduler-operator-5fdd9b5758-rxfb5\" (UID: \"5d079af1-8d26-4a35-89f6-1312e1d5c2dc\") " pod="openshift-kube-scheduler-operator/openshift-kube-scheduler-operator-5fdd9b5758-rxfb5" Dec 03 11:20:49 crc kubenswrapper[5052]: I1203 11:20:49.370112 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-2jbxs\" (UniqueName: \"kubernetes.io/projected/1c570f3f-3090-48e1-9b7e-f66c99310dbd-kube-api-access-2jbxs\") pod \"controller-manager-879f6c89f-46rm8\" (UID: \"1c570f3f-3090-48e1-9b7e-f66c99310dbd\") " pod="openshift-controller-manager/controller-manager-879f6c89f-46rm8" Dec 03 11:20:49 crc kubenswrapper[5052]: I1203 11:20:49.370140 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-qq2t2\" (UniqueName: \"kubernetes.io/projected/ed6dae78-3358-4be5-84c2-142e4d45fe9a-kube-api-access-qq2t2\") pod \"ingress-canary-tvr2s\" (UID: \"ed6dae78-3358-4be5-84c2-142e4d45fe9a\") " pod="openshift-ingress-canary/ingress-canary-tvr2s" Dec 03 11:20:49 crc kubenswrapper[5052]: I1203 11:20:49.370211 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access\" (UniqueName: \"kubernetes.io/projected/5d079af1-8d26-4a35-89f6-1312e1d5c2dc-kube-api-access\") pod \"openshift-kube-scheduler-operator-5fdd9b5758-rxfb5\" (UID: \"5d079af1-8d26-4a35-89f6-1312e1d5c2dc\") " pod="openshift-kube-scheduler-operator/openshift-kube-scheduler-operator-5fdd9b5758-rxfb5" Dec 03 11:20:49 crc kubenswrapper[5052]: I1203 11:20:49.370239 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/bf761cb9-e024-4acd-a947-0f8064267243-tuning-conf-dir\") pod \"cni-sysctl-allowlist-ds-8l9gm\" (UID: \"bf761cb9-e024-4acd-a947-0f8064267243\") " pod="openshift-multus/cni-sysctl-allowlist-ds-8l9gm" Dec 03 11:20:49 crc kubenswrapper[5052]: I1203 11:20:49.370271 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"package-server-manager-serving-cert\" (UniqueName: \"kubernetes.io/secret/3f6f3c17-944e-4afb-8cd7-3ca95f787057-package-server-manager-serving-cert\") pod \"package-server-manager-789f6589d5-577rn\" (UID: \"3f6f3c17-944e-4afb-8cd7-3ca95f787057\") " pod="openshift-operator-lifecycle-manager/package-server-manager-789f6589d5-577rn" Dec 03 11:20:49 crc kubenswrapper[5052]: I1203 11:20:49.370291 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certs\" (UniqueName: \"kubernetes.io/secret/c3a89abc-ba9f-4c13-a73e-2b9e66e9345a-certs\") pod \"machine-config-server-kzd49\" (UID: \"c3a89abc-ba9f-4c13-a73e-2b9e66e9345a\") " pod="openshift-machine-config-operator/machine-config-server-kzd49" Dec 03 11:20:49 crc kubenswrapper[5052]: I1203 11:20:49.370357 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/60b355dc-6de7-4a73-a2d2-3a0f0befd049-registration-dir\") pod \"csi-hostpathplugin-5tlrr\" (UID: \"60b355dc-6de7-4a73-a2d2-3a0f0befd049\") " pod="hostpath-provisioner/csi-hostpathplugin-5tlrr" Dec 03 11:20:49 crc kubenswrapper[5052]: I1203 11:20:49.370437 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"csi-data-dir\" (UniqueName: \"kubernetes.io/host-path/60b355dc-6de7-4a73-a2d2-3a0f0befd049-csi-data-dir\") pod \"csi-hostpathplugin-5tlrr\" (UID: \"60b355dc-6de7-4a73-a2d2-3a0f0befd049\") " pod="hostpath-provisioner/csi-hostpathplugin-5tlrr" Dec 03 11:20:49 crc kubenswrapper[5052]: I1203 11:20:49.370614 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/60b355dc-6de7-4a73-a2d2-3a0f0befd049-socket-dir\") pod \"csi-hostpathplugin-5tlrr\" (UID: \"60b355dc-6de7-4a73-a2d2-3a0f0befd049\") " pod="hostpath-provisioner/csi-hostpathplugin-5tlrr" Dec 03 11:20:49 crc kubenswrapper[5052]: I1203 11:20:49.370858 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/efbeac4d-779b-4eb2-acf5-241c83a46eb7-config-volume\") pod \"dns-default-6m2jn\" (UID: \"efbeac4d-779b-4eb2-acf5-241c83a46eb7\") " pod="openshift-dns/dns-default-6m2jn" Dec 03 11:20:49 crc kubenswrapper[5052]: I1203 11:20:49.370961 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c5vsd\" (UniqueName: \"kubernetes.io/projected/bf761cb9-e024-4acd-a947-0f8064267243-kube-api-access-c5vsd\") pod \"cni-sysctl-allowlist-ds-8l9gm\" (UID: \"bf761cb9-e024-4acd-a947-0f8064267243\") " pod="openshift-multus/cni-sysctl-allowlist-ds-8l9gm" Dec 03 11:20:49 crc kubenswrapper[5052]: I1203 11:20:49.370994 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-bootstrap-token\" (UniqueName: \"kubernetes.io/secret/c3a89abc-ba9f-4c13-a73e-2b9e66e9345a-node-bootstrap-token\") pod \"machine-config-server-kzd49\" (UID: \"c3a89abc-ba9f-4c13-a73e-2b9e66e9345a\") " pod="openshift-machine-config-operator/machine-config-server-kzd49" Dec 03 11:20:49 crc kubenswrapper[5052]: I1203 11:20:49.371025 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b1d2ff08-d58b-48d0-a0e5-6cfd867a63de-console-serving-cert\") pod \"console-f9d7485db-568dj\" (UID: \"b1d2ff08-d58b-48d0-a0e5-6cfd867a63de\") " pod="openshift-console/console-f9d7485db-568dj" Dec 03 11:20:49 crc kubenswrapper[5052]: I1203 11:20:49.371250 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ghdbm\" (UniqueName: \"kubernetes.io/projected/c3a89abc-ba9f-4c13-a73e-2b9e66e9345a-kube-api-access-ghdbm\") pod \"machine-config-server-kzd49\" (UID: \"c3a89abc-ba9f-4c13-a73e-2b9e66e9345a\") " pod="openshift-machine-config-operator/machine-config-server-kzd49" Dec 03 11:20:49 crc kubenswrapper[5052]: I1203 11:20:49.371309 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wlxdz\" (UniqueName: \"kubernetes.io/projected/60b355dc-6de7-4a73-a2d2-3a0f0befd049-kube-api-access-wlxdz\") pod \"csi-hostpathplugin-5tlrr\" (UID: \"60b355dc-6de7-4a73-a2d2-3a0f0befd049\") " pod="hostpath-provisioner/csi-hostpathplugin-5tlrr" Dec 03 11:20:49 crc kubenswrapper[5052]: I1203 11:20:49.371367 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5d079af1-8d26-4a35-89f6-1312e1d5c2dc-config\") pod \"openshift-kube-scheduler-operator-5fdd9b5758-rxfb5\" (UID: \"5d079af1-8d26-4a35-89f6-1312e1d5c2dc\") " pod="openshift-kube-scheduler-operator/openshift-kube-scheduler-operator-5fdd9b5758-rxfb5" Dec 03 11:20:49 crc kubenswrapper[5052]: I1203 11:20:49.371403 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ready\" (UniqueName: \"kubernetes.io/empty-dir/bf761cb9-e024-4acd-a947-0f8064267243-ready\") pod \"cni-sysctl-allowlist-ds-8l9gm\" (UID: \"bf761cb9-e024-4acd-a947-0f8064267243\") " pod="openshift-multus/cni-sysctl-allowlist-ds-8l9gm" Dec 03 11:20:49 crc kubenswrapper[5052]: I1203 11:20:49.371459 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"mountpoint-dir\" (UniqueName: \"kubernetes.io/host-path/60b355dc-6de7-4a73-a2d2-3a0f0befd049-mountpoint-dir\") pod \"csi-hostpathplugin-5tlrr\" (UID: \"60b355dc-6de7-4a73-a2d2-3a0f0befd049\") " pod="hostpath-provisioner/csi-hostpathplugin-5tlrr" Dec 03 11:20:49 crc kubenswrapper[5052]: I1203 11:20:49.371483 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-dir\" (UniqueName: \"kubernetes.io/host-path/60b355dc-6de7-4a73-a2d2-3a0f0befd049-plugins-dir\") pod \"csi-hostpathplugin-5tlrr\" (UID: \"60b355dc-6de7-4a73-a2d2-3a0f0befd049\") " pod="hostpath-provisioner/csi-hostpathplugin-5tlrr" Dec 03 11:20:49 crc kubenswrapper[5052]: I1203 11:20:49.371537 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/bf761cb9-e024-4acd-a947-0f8064267243-cni-sysctl-allowlist\") pod \"cni-sysctl-allowlist-ds-8l9gm\" (UID: \"bf761cb9-e024-4acd-a947-0f8064267243\") " pod="openshift-multus/cni-sysctl-allowlist-ds-8l9gm" Dec 03 11:20:49 crc kubenswrapper[5052]: I1203 11:20:49.371598 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ed6dae78-3358-4be5-84c2-142e4d45fe9a-cert\") pod \"ingress-canary-tvr2s\" (UID: \"ed6dae78-3358-4be5-84c2-142e4d45fe9a\") " pod="openshift-ingress-canary/ingress-canary-tvr2s" Dec 03 11:20:49 crc kubenswrapper[5052]: E1203 11:20:49.372691 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8 podName:8f668bae-612b-4b75-9490-919e737c6a3b nodeName:}" failed. No retries permitted until 2025-12-03 11:20:49.872669064 +0000 UTC m=+40.460910858 (durationBeforeRetry 500ms). Error: UnmountVolume.TearDown failed for volume "pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8" (UniqueName: "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8") pod "8f668bae-612b-4b75-9490-919e737c6a3b" (UID: "8f668bae-612b-4b75-9490-919e737c6a3b") : kubernetes.io/csi: Unmounter.TearDownAt failed to get CSI client: driver name kubevirt.io.hostpath-provisioner not found in the list of registered CSI drivers Dec 03 11:20:49 crc kubenswrapper[5052]: I1203 11:20:49.372706 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-api/machine-api-operator-5694c8668f-l5lrg" Dec 03 11:20:49 crc kubenswrapper[5052]: I1203 11:20:49.372890 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"client-ca\" (UniqueName: \"kubernetes.io/configmap/1c570f3f-3090-48e1-9b7e-f66c99310dbd-client-ca\") pod \"controller-manager-879f6c89f-46rm8\" (UID: \"1c570f3f-3090-48e1-9b7e-f66c99310dbd\") " pod="openshift-controller-manager/controller-manager-879f6c89f-46rm8" Dec 03 11:20:49 crc kubenswrapper[5052]: I1203 11:20:49.373539 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/efbeac4d-779b-4eb2-acf5-241c83a46eb7-config-volume\") pod \"dns-default-6m2jn\" (UID: \"efbeac4d-779b-4eb2-acf5-241c83a46eb7\") " pod="openshift-dns/dns-default-6m2jn" Dec 03 11:20:49 crc kubenswrapper[5052]: I1203 11:20:49.375927 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5d079af1-8d26-4a35-89f6-1312e1d5c2dc-config\") pod \"openshift-kube-scheduler-operator-5fdd9b5758-rxfb5\" (UID: \"5d079af1-8d26-4a35-89f6-1312e1d5c2dc\") " pod="openshift-kube-scheduler-operator/openshift-kube-scheduler-operator-5fdd9b5758-rxfb5" Dec 03 11:20:49 crc kubenswrapper[5052]: I1203 11:20:49.376858 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/bf761cb9-e024-4acd-a947-0f8064267243-cni-sysctl-allowlist\") pod \"cni-sysctl-allowlist-ds-8l9gm\" (UID: \"bf761cb9-e024-4acd-a947-0f8064267243\") " pod="openshift-multus/cni-sysctl-allowlist-ds-8l9gm" Dec 03 11:20:49 crc kubenswrapper[5052]: I1203 11:20:49.379041 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-controller-manager-operator/kube-controller-manager-operator-78b949d7b-h6b78" Dec 03 11:20:49 crc kubenswrapper[5052]: I1203 11:20:49.390086 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5d079af1-8d26-4a35-89f6-1312e1d5c2dc-serving-cert\") pod \"openshift-kube-scheduler-operator-5fdd9b5758-rxfb5\" (UID: \"5d079af1-8d26-4a35-89f6-1312e1d5c2dc\") " pod="openshift-kube-scheduler-operator/openshift-kube-scheduler-operator-5fdd9b5758-rxfb5" Dec 03 11:20:49 crc kubenswrapper[5052]: I1203 11:20:49.394973 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"package-server-manager-serving-cert\" (UniqueName: \"kubernetes.io/secret/3f6f3c17-944e-4afb-8cd7-3ca95f787057-package-server-manager-serving-cert\") pod \"package-server-manager-789f6589d5-577rn\" (UID: \"3f6f3c17-944e-4afb-8cd7-3ca95f787057\") " pod="openshift-operator-lifecycle-manager/package-server-manager-789f6589d5-577rn" Dec 03 11:20:49 crc kubenswrapper[5052]: I1203 11:20:49.396142 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-2jbxs\" (UniqueName: \"kubernetes.io/projected/1c570f3f-3090-48e1-9b7e-f66c99310dbd-kube-api-access-2jbxs\") pod \"controller-manager-879f6c89f-46rm8\" (UID: \"1c570f3f-3090-48e1-9b7e-f66c99310dbd\") " pod="openshift-controller-manager/controller-manager-879f6c89f-46rm8" Dec 03 11:20:49 crc kubenswrapper[5052]: I1203 11:20:49.397799 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-qq2t2\" (UniqueName: \"kubernetes.io/projected/ed6dae78-3358-4be5-84c2-142e4d45fe9a-kube-api-access-qq2t2\") pod \"ingress-canary-tvr2s\" (UID: \"ed6dae78-3358-4be5-84c2-142e4d45fe9a\") " pod="openshift-ingress-canary/ingress-canary-tvr2s" Dec 03 11:20:49 crc kubenswrapper[5052]: I1203 11:20:49.397897 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-rgsw9\" (UniqueName: \"kubernetes.io/projected/3f6f3c17-944e-4afb-8cd7-3ca95f787057-kube-api-access-rgsw9\") pod \"package-server-manager-789f6589d5-577rn\" (UID: \"3f6f3c17-944e-4afb-8cd7-3ca95f787057\") " pod="openshift-operator-lifecycle-manager/package-server-manager-789f6589d5-577rn" Dec 03 11:20:49 crc kubenswrapper[5052]: I1203 11:20:49.400886 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access\" (UniqueName: \"kubernetes.io/projected/5d079af1-8d26-4a35-89f6-1312e1d5c2dc-kube-api-access\") pod \"openshift-kube-scheduler-operator-5fdd9b5758-rxfb5\" (UID: \"5d079af1-8d26-4a35-89f6-1312e1d5c2dc\") " pod="openshift-kube-scheduler-operator/openshift-kube-scheduler-operator-5fdd9b5758-rxfb5" Dec 03 11:20:49 crc kubenswrapper[5052]: I1203 11:20:49.402161 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b1d2ff08-d58b-48d0-a0e5-6cfd867a63de-console-serving-cert\") pod \"console-f9d7485db-568dj\" (UID: \"b1d2ff08-d58b-48d0-a0e5-6cfd867a63de\") " pod="openshift-console/console-f9d7485db-568dj" Dec 03 11:20:49 crc kubenswrapper[5052]: I1203 11:20:49.409411 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ed6dae78-3358-4be5-84c2-142e4d45fe9a-cert\") pod \"ingress-canary-tvr2s\" (UID: \"ed6dae78-3358-4be5-84c2-142e4d45fe9a\") " pod="openshift-ingress-canary/ingress-canary-tvr2s" Dec 03 11:20:49 crc kubenswrapper[5052]: I1203 11:20:49.433183 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-operator-lifecycle-manager/packageserver-d55dfcdfc-7s97q" Dec 03 11:20:49 crc kubenswrapper[5052]: I1203 11:20:49.452168 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-machine-config-operator/machine-config-controller-84d6567774-rxwdl"] Dec 03 11:20:49 crc kubenswrapper[5052]: I1203 11:20:49.466279 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-operator-lifecycle-manager/collect-profiles-29412675-hzd4n" Dec 03 11:20:49 crc kubenswrapper[5052]: I1203 11:20:49.475265 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e0bf52c7-bf23-4c65-b7cb-0d8d466aa8f2-serving-cert\") pod \"authentication-operator-69f744f599-qjq5f\" (UID: \"e0bf52c7-bf23-4c65-b7cb-0d8d466aa8f2\") " pod="openshift-authentication-operator/authentication-operator-69f744f599-qjq5f" Dec 03 11:20:49 crc kubenswrapper[5052]: I1203 11:20:49.475295 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/60b355dc-6de7-4a73-a2d2-3a0f0befd049-socket-dir\") pod \"csi-hostpathplugin-5tlrr\" (UID: \"60b355dc-6de7-4a73-a2d2-3a0f0befd049\") " pod="hostpath-provisioner/csi-hostpathplugin-5tlrr" Dec 03 11:20:49 crc kubenswrapper[5052]: I1203 11:20:49.475318 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etcd-serving-ca\" (UniqueName: \"kubernetes.io/configmap/41016a7b-cc79-4f32-860c-cb75e405cf42-etcd-serving-ca\") pod \"apiserver-76f77b778f-dmvst\" (UID: \"41016a7b-cc79-4f32-860c-cb75e405cf42\") " pod="openshift-apiserver/apiserver-76f77b778f-dmvst" Dec 03 11:20:49 crc kubenswrapper[5052]: I1203 11:20:49.475372 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e0bf52c7-bf23-4c65-b7cb-0d8d466aa8f2-config\") pod \"authentication-operator-69f744f599-qjq5f\" (UID: \"e0bf52c7-bf23-4c65-b7cb-0d8d466aa8f2\") " pod="openshift-authentication-operator/authentication-operator-69f744f599-qjq5f" Dec 03 11:20:49 crc kubenswrapper[5052]: I1203 11:20:49.475397 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-c5vsd\" (UniqueName: \"kubernetes.io/projected/bf761cb9-e024-4acd-a947-0f8064267243-kube-api-access-c5vsd\") pod \"cni-sysctl-allowlist-ds-8l9gm\" (UID: \"bf761cb9-e024-4acd-a947-0f8064267243\") " pod="openshift-multus/cni-sysctl-allowlist-ds-8l9gm" Dec 03 11:20:49 crc kubenswrapper[5052]: I1203 11:20:49.475415 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"node-bootstrap-token\" (UniqueName: \"kubernetes.io/secret/c3a89abc-ba9f-4c13-a73e-2b9e66e9345a-node-bootstrap-token\") pod \"machine-config-server-kzd49\" (UID: \"c3a89abc-ba9f-4c13-a73e-2b9e66e9345a\") " pod="openshift-machine-config-operator/machine-config-server-kzd49" Dec 03 11:20:49 crc kubenswrapper[5052]: I1203 11:20:49.475434 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"image-import-ca\" (UniqueName: \"kubernetes.io/configmap/41016a7b-cc79-4f32-860c-cb75e405cf42-image-import-ca\") pod \"apiserver-76f77b778f-dmvst\" (UID: \"41016a7b-cc79-4f32-860c-cb75e405cf42\") " pod="openshift-apiserver/apiserver-76f77b778f-dmvst" Dec 03 11:20:49 crc kubenswrapper[5052]: I1203 11:20:49.475457 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etcd-client\" (UniqueName: \"kubernetes.io/secret/41016a7b-cc79-4f32-860c-cb75e405cf42-etcd-client\") pod \"apiserver-76f77b778f-dmvst\" (UID: \"41016a7b-cc79-4f32-860c-cb75e405cf42\") " pod="openshift-apiserver/apiserver-76f77b778f-dmvst" Dec 03 11:20:49 crc kubenswrapper[5052]: I1203 11:20:49.475476 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-ghdbm\" (UniqueName: \"kubernetes.io/projected/c3a89abc-ba9f-4c13-a73e-2b9e66e9345a-kube-api-access-ghdbm\") pod \"machine-config-server-kzd49\" (UID: \"c3a89abc-ba9f-4c13-a73e-2b9e66e9345a\") " pod="openshift-machine-config-operator/machine-config-server-kzd49" Dec 03 11:20:49 crc kubenswrapper[5052]: I1203 11:20:49.475491 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-wlxdz\" (UniqueName: \"kubernetes.io/projected/60b355dc-6de7-4a73-a2d2-3a0f0befd049-kube-api-access-wlxdz\") pod \"csi-hostpathplugin-5tlrr\" (UID: \"60b355dc-6de7-4a73-a2d2-3a0f0befd049\") " pod="hostpath-provisioner/csi-hostpathplugin-5tlrr" Dec 03 11:20:49 crc kubenswrapper[5052]: I1203 11:20:49.475581 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ready\" (UniqueName: \"kubernetes.io/empty-dir/bf761cb9-e024-4acd-a947-0f8064267243-ready\") pod \"cni-sysctl-allowlist-ds-8l9gm\" (UID: \"bf761cb9-e024-4acd-a947-0f8064267243\") " pod="openshift-multus/cni-sysctl-allowlist-ds-8l9gm" Dec 03 11:20:49 crc kubenswrapper[5052]: I1203 11:20:49.475602 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"mountpoint-dir\" (UniqueName: \"kubernetes.io/host-path/60b355dc-6de7-4a73-a2d2-3a0f0befd049-mountpoint-dir\") pod \"csi-hostpathplugin-5tlrr\" (UID: \"60b355dc-6de7-4a73-a2d2-3a0f0befd049\") " pod="hostpath-provisioner/csi-hostpathplugin-5tlrr" Dec 03 11:20:49 crc kubenswrapper[5052]: I1203 11:20:49.476228 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"plugins-dir\" (UniqueName: \"kubernetes.io/host-path/60b355dc-6de7-4a73-a2d2-3a0f0befd049-plugins-dir\") pod \"csi-hostpathplugin-5tlrr\" (UID: \"60b355dc-6de7-4a73-a2d2-3a0f0befd049\") " pod="hostpath-provisioner/csi-hostpathplugin-5tlrr" Dec 03 11:20:49 crc kubenswrapper[5052]: I1203 11:20:49.476252 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\") pod \"image-registry-697d97f7c8-p77bx\" (UID: \"f5012254-068c-4360-b310-39ff3d309199\") " pod="openshift-image-registry/image-registry-697d97f7c8-p77bx" Dec 03 11:20:49 crc kubenswrapper[5052]: I1203 11:20:49.476276 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/41016a7b-cc79-4f32-860c-cb75e405cf42-serving-cert\") pod \"apiserver-76f77b778f-dmvst\" (UID: \"41016a7b-cc79-4f32-860c-cb75e405cf42\") " pod="openshift-apiserver/apiserver-76f77b778f-dmvst" Dec 03 11:20:49 crc kubenswrapper[5052]: I1203 11:20:49.476299 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/bf761cb9-e024-4acd-a947-0f8064267243-tuning-conf-dir\") pod \"cni-sysctl-allowlist-ds-8l9gm\" (UID: \"bf761cb9-e024-4acd-a947-0f8064267243\") " pod="openshift-multus/cni-sysctl-allowlist-ds-8l9gm" Dec 03 11:20:49 crc kubenswrapper[5052]: I1203 11:20:49.476315 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"certs\" (UniqueName: \"kubernetes.io/secret/c3a89abc-ba9f-4c13-a73e-2b9e66e9345a-certs\") pod \"machine-config-server-kzd49\" (UID: \"c3a89abc-ba9f-4c13-a73e-2b9e66e9345a\") " pod="openshift-machine-config-operator/machine-config-server-kzd49" Dec 03 11:20:49 crc kubenswrapper[5052]: I1203 11:20:49.476336 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/60b355dc-6de7-4a73-a2d2-3a0f0befd049-registration-dir\") pod \"csi-hostpathplugin-5tlrr\" (UID: \"60b355dc-6de7-4a73-a2d2-3a0f0befd049\") " pod="hostpath-provisioner/csi-hostpathplugin-5tlrr" Dec 03 11:20:49 crc kubenswrapper[5052]: I1203 11:20:49.476356 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"csi-data-dir\" (UniqueName: \"kubernetes.io/host-path/60b355dc-6de7-4a73-a2d2-3a0f0befd049-csi-data-dir\") pod \"csi-hostpathplugin-5tlrr\" (UID: \"60b355dc-6de7-4a73-a2d2-3a0f0befd049\") " pod="hostpath-provisioner/csi-hostpathplugin-5tlrr" Dec 03 11:20:49 crc kubenswrapper[5052]: I1203 11:20:49.476500 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"csi-data-dir\" (UniqueName: \"kubernetes.io/host-path/60b355dc-6de7-4a73-a2d2-3a0f0befd049-csi-data-dir\") pod \"csi-hostpathplugin-5tlrr\" (UID: \"60b355dc-6de7-4a73-a2d2-3a0f0befd049\") " pod="hostpath-provisioner/csi-hostpathplugin-5tlrr" Dec 03 11:20:49 crc kubenswrapper[5052]: I1203 11:20:49.477210 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/60b355dc-6de7-4a73-a2d2-3a0f0befd049-socket-dir\") pod \"csi-hostpathplugin-5tlrr\" (UID: \"60b355dc-6de7-4a73-a2d2-3a0f0befd049\") " pod="hostpath-provisioner/csi-hostpathplugin-5tlrr" Dec 03 11:20:49 crc kubenswrapper[5052]: I1203 11:20:49.477289 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"plugins-dir\" (UniqueName: \"kubernetes.io/host-path/60b355dc-6de7-4a73-a2d2-3a0f0befd049-plugins-dir\") pod \"csi-hostpathplugin-5tlrr\" (UID: \"60b355dc-6de7-4a73-a2d2-3a0f0befd049\") " pod="hostpath-provisioner/csi-hostpathplugin-5tlrr" Dec 03 11:20:49 crc kubenswrapper[5052]: I1203 11:20:49.477867 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ready\" (UniqueName: \"kubernetes.io/empty-dir/bf761cb9-e024-4acd-a947-0f8064267243-ready\") pod \"cni-sysctl-allowlist-ds-8l9gm\" (UID: \"bf761cb9-e024-4acd-a947-0f8064267243\") " pod="openshift-multus/cni-sysctl-allowlist-ds-8l9gm" Dec 03 11:20:49 crc kubenswrapper[5052]: I1203 11:20:49.478685 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"mountpoint-dir\" (UniqueName: \"kubernetes.io/host-path/60b355dc-6de7-4a73-a2d2-3a0f0befd049-mountpoint-dir\") pod \"csi-hostpathplugin-5tlrr\" (UID: \"60b355dc-6de7-4a73-a2d2-3a0f0befd049\") " pod="hostpath-provisioner/csi-hostpathplugin-5tlrr" Dec 03 11:20:49 crc kubenswrapper[5052]: I1203 11:20:49.478738 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/bf761cb9-e024-4acd-a947-0f8064267243-tuning-conf-dir\") pod \"cni-sysctl-allowlist-ds-8l9gm\" (UID: \"bf761cb9-e024-4acd-a947-0f8064267243\") " pod="openshift-multus/cni-sysctl-allowlist-ds-8l9gm" Dec 03 11:20:49 crc kubenswrapper[5052]: E1203 11:20:49.478999 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8 podName: nodeName:}" failed. No retries permitted until 2025-12-03 11:20:49.978985194 +0000 UTC m=+40.567226958 (durationBeforeRetry 500ms). Error: MountVolume.MountDevice failed for volume "pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8" (UniqueName: "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8") pod "image-registry-697d97f7c8-p77bx" (UID: "f5012254-068c-4360-b310-39ff3d309199") : kubernetes.io/csi: attacher.MountDevice failed to create newCsiDriverClient: driver name kubevirt.io.hostpath-provisioner not found in the list of registered CSI drivers Dec 03 11:20:49 crc kubenswrapper[5052]: I1203 11:20:49.479408 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e0bf52c7-bf23-4c65-b7cb-0d8d466aa8f2-config\") pod \"authentication-operator-69f744f599-qjq5f\" (UID: \"e0bf52c7-bf23-4c65-b7cb-0d8d466aa8f2\") " pod="openshift-authentication-operator/authentication-operator-69f744f599-qjq5f" Dec 03 11:20:49 crc kubenswrapper[5052]: I1203 11:20:49.481285 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"node-bootstrap-token\" (UniqueName: \"kubernetes.io/secret/c3a89abc-ba9f-4c13-a73e-2b9e66e9345a-node-bootstrap-token\") pod \"machine-config-server-kzd49\" (UID: \"c3a89abc-ba9f-4c13-a73e-2b9e66e9345a\") " pod="openshift-machine-config-operator/machine-config-server-kzd49" Dec 03 11:20:49 crc kubenswrapper[5052]: I1203 11:20:49.481823 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"etcd-serving-ca\" (UniqueName: \"kubernetes.io/configmap/41016a7b-cc79-4f32-860c-cb75e405cf42-etcd-serving-ca\") pod \"apiserver-76f77b778f-dmvst\" (UID: \"41016a7b-cc79-4f32-860c-cb75e405cf42\") " pod="openshift-apiserver/apiserver-76f77b778f-dmvst" Dec 03 11:20:49 crc kubenswrapper[5052]: I1203 11:20:49.481964 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/60b355dc-6de7-4a73-a2d2-3a0f0befd049-registration-dir\") pod \"csi-hostpathplugin-5tlrr\" (UID: \"60b355dc-6de7-4a73-a2d2-3a0f0befd049\") " pod="hostpath-provisioner/csi-hostpathplugin-5tlrr" Dec 03 11:20:49 crc kubenswrapper[5052]: I1203 11:20:49.482679 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"certs\" (UniqueName: \"kubernetes.io/secret/c3a89abc-ba9f-4c13-a73e-2b9e66e9345a-certs\") pod \"machine-config-server-kzd49\" (UID: \"c3a89abc-ba9f-4c13-a73e-2b9e66e9345a\") " pod="openshift-machine-config-operator/machine-config-server-kzd49" Dec 03 11:20:49 crc kubenswrapper[5052]: I1203 11:20:49.483996 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"etcd-client\" (UniqueName: \"kubernetes.io/secret/41016a7b-cc79-4f32-860c-cb75e405cf42-etcd-client\") pod \"apiserver-76f77b778f-dmvst\" (UID: \"41016a7b-cc79-4f32-860c-cb75e405cf42\") " pod="openshift-apiserver/apiserver-76f77b778f-dmvst" Dec 03 11:20:49 crc kubenswrapper[5052]: I1203 11:20:49.484419 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/41016a7b-cc79-4f32-860c-cb75e405cf42-serving-cert\") pod \"apiserver-76f77b778f-dmvst\" (UID: \"41016a7b-cc79-4f32-860c-cb75e405cf42\") " pod="openshift-apiserver/apiserver-76f77b778f-dmvst" Dec 03 11:20:49 crc kubenswrapper[5052]: I1203 11:20:49.485699 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e0bf52c7-bf23-4c65-b7cb-0d8d466aa8f2-serving-cert\") pod \"authentication-operator-69f744f599-qjq5f\" (UID: \"e0bf52c7-bf23-4c65-b7cb-0d8d466aa8f2\") " pod="openshift-authentication-operator/authentication-operator-69f744f599-qjq5f" Dec 03 11:20:49 crc kubenswrapper[5052]: I1203 11:20:49.486360 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"image-import-ca\" (UniqueName: \"kubernetes.io/configmap/41016a7b-cc79-4f32-860c-cb75e405cf42-image-import-ca\") pod \"apiserver-76f77b778f-dmvst\" (UID: \"41016a7b-cc79-4f32-860c-cb75e405cf42\") " pod="openshift-apiserver/apiserver-76f77b778f-dmvst" Dec 03 11:20:49 crc kubenswrapper[5052]: I1203 11:20:49.495263 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-c5vsd\" (UniqueName: \"kubernetes.io/projected/bf761cb9-e024-4acd-a947-0f8064267243-kube-api-access-c5vsd\") pod \"cni-sysctl-allowlist-ds-8l9gm\" (UID: \"bf761cb9-e024-4acd-a947-0f8064267243\") " pod="openshift-multus/cni-sysctl-allowlist-ds-8l9gm" Dec 03 11:20:49 crc kubenswrapper[5052]: I1203 11:20:49.495286 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-wlxdz\" (UniqueName: \"kubernetes.io/projected/60b355dc-6de7-4a73-a2d2-3a0f0befd049-kube-api-access-wlxdz\") pod \"csi-hostpathplugin-5tlrr\" (UID: \"60b355dc-6de7-4a73-a2d2-3a0f0befd049\") " pod="hostpath-provisioner/csi-hostpathplugin-5tlrr" Dec 03 11:20:49 crc kubenswrapper[5052]: I1203 11:20:49.495659 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-6m2jn" Dec 03 11:20:49 crc kubenswrapper[5052]: I1203 11:20:49.496571 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-b67b599dd-5h545" Dec 03 11:20:49 crc kubenswrapper[5052]: I1203 11:20:49.505647 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-ghdbm\" (UniqueName: \"kubernetes.io/projected/c3a89abc-ba9f-4c13-a73e-2b9e66e9345a-kube-api-access-ghdbm\") pod \"machine-config-server-kzd49\" (UID: \"c3a89abc-ba9f-4c13-a73e-2b9e66e9345a\") " pod="openshift-machine-config-operator/machine-config-server-kzd49" Dec 03 11:20:49 crc kubenswrapper[5052]: I1203 11:20:49.505968 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/cni-sysctl-allowlist-ds-8l9gm" Dec 03 11:20:49 crc kubenswrapper[5052]: I1203 11:20:49.517076 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-tvr2s" Dec 03 11:20:49 crc kubenswrapper[5052]: I1203 11:20:49.525779 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/machine-config-server-kzd49" Dec 03 11:20:49 crc kubenswrapper[5052]: I1203 11:20:49.538167 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="hostpath-provisioner/csi-hostpathplugin-5tlrr" Dec 03 11:20:49 crc kubenswrapper[5052]: I1203 11:20:49.559412 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-scheduler-operator/openshift-kube-scheduler-operator-5fdd9b5758-rxfb5" Dec 03 11:20:49 crc kubenswrapper[5052]: I1203 11:20:49.586242 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\") pod \"8f668bae-612b-4b75-9490-919e737c6a3b\" (UID: \"8f668bae-612b-4b75-9490-919e737c6a3b\") " Dec 03 11:20:49 crc kubenswrapper[5052]: I1203 11:20:49.586614 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f5012254-068c-4360-b310-39ff3d309199-installation-pull-secrets\") pod \"image-registry-697d97f7c8-p77bx\" (UID: \"f5012254-068c-4360-b310-39ff3d309199\") " pod="openshift-image-registry/image-registry-697d97f7c8-p77bx" Dec 03 11:20:49 crc kubenswrapper[5052]: E1203 11:20:49.587541 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8 podName:8f668bae-612b-4b75-9490-919e737c6a3b nodeName:}" failed. No retries permitted until 2025-12-03 11:20:50.087498365 +0000 UTC m=+40.675740209 (durationBeforeRetry 500ms). Error: UnmountVolume.TearDown failed for volume "pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8" (UniqueName: "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8") pod "8f668bae-612b-4b75-9490-919e737c6a3b" (UID: "8f668bae-612b-4b75-9490-919e737c6a3b") : kubernetes.io/csi: Unmounter.TearDownAt failed to get CSI client: driver name kubevirt.io.hostpath-provisioner not found in the list of registered CSI drivers Dec 03 11:20:49 crc kubenswrapper[5052]: I1203 11:20:49.587867 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-apiserver/apiserver-76f77b778f-dmvst" Dec 03 11:20:49 crc kubenswrapper[5052]: I1203 11:20:49.588428 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-machine-config-operator/machine-config-operator-74547568cd-tr2xc"] Dec 03 11:20:49 crc kubenswrapper[5052]: I1203 11:20:49.593463 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f5012254-068c-4360-b310-39ff3d309199-installation-pull-secrets\") pod \"image-registry-697d97f7c8-p77bx\" (UID: \"f5012254-068c-4360-b310-39ff3d309199\") " pod="openshift-image-registry/image-registry-697d97f7c8-p77bx" Dec 03 11:20:49 crc kubenswrapper[5052]: I1203 11:20:49.596400 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-authentication-operator/authentication-operator-69f744f599-qjq5f" Dec 03 11:20:49 crc kubenswrapper[5052]: I1203 11:20:49.610633 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-controller-manager/controller-manager-879f6c89f-46rm8" Dec 03 11:20:49 crc kubenswrapper[5052]: I1203 11:20:49.620902 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-7954f5f757-l7xtl"] Dec 03 11:20:49 crc kubenswrapper[5052]: I1203 11:20:49.648312 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-f9d7485db-568dj" Dec 03 11:20:49 crc kubenswrapper[5052]: I1203 11:20:49.688259 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\") pod \"image-registry-697d97f7c8-p77bx\" (UID: \"f5012254-068c-4360-b310-39ff3d309199\") " pod="openshift-image-registry/image-registry-697d97f7c8-p77bx" Dec 03 11:20:49 crc kubenswrapper[5052]: E1203 11:20:49.689115 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8 podName: nodeName:}" failed. No retries permitted until 2025-12-03 11:20:50.189102665 +0000 UTC m=+40.777344429 (durationBeforeRetry 500ms). Error: MountVolume.MountDevice failed for volume "pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8" (UniqueName: "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8") pod "image-registry-697d97f7c8-p77bx" (UID: "f5012254-068c-4360-b310-39ff3d309199") : kubernetes.io/csi: attacher.MountDevice failed to create newCsiDriverClient: driver name kubevirt.io.hostpath-provisioner not found in the list of registered CSI drivers Dec 03 11:20:49 crc kubenswrapper[5052]: I1203 11:20:49.694759 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-operator-lifecycle-manager/package-server-manager-789f6589d5-577rn" Dec 03 11:20:49 crc kubenswrapper[5052]: I1203 11:20:49.710024 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/marketplace-operator-79b997595-kdr8l"] Dec 03 11:20:49 crc kubenswrapper[5052]: I1203 11:20:49.791273 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\") pod \"8f668bae-612b-4b75-9490-919e737c6a3b\" (UID: \"8f668bae-612b-4b75-9490-919e737c6a3b\") " Dec 03 11:20:49 crc kubenswrapper[5052]: E1203 11:20:49.791721 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8 podName:8f668bae-612b-4b75-9490-919e737c6a3b nodeName:}" failed. No retries permitted until 2025-12-03 11:20:50.291699172 +0000 UTC m=+40.879940936 (durationBeforeRetry 500ms). Error: UnmountVolume.TearDown failed for volume "pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8" (UniqueName: "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8") pod "8f668bae-612b-4b75-9490-919e737c6a3b" (UID: "8f668bae-612b-4b75-9490-919e737c6a3b") : kubernetes.io/csi: Unmounter.TearDownAt failed to get CSI client: driver name kubevirt.io.hostpath-provisioner not found in the list of registered CSI drivers Dec 03 11:20:49 crc kubenswrapper[5052]: I1203 11:20:49.800617 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-authentication/oauth-openshift-558db77b4-99jsx"] Dec 03 11:20:49 crc kubenswrapper[5052]: I1203 11:20:49.854908 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-operator/ingress-operator-5b745b69d9-hc8vv"] Dec 03 11:20:49 crc kubenswrapper[5052]: I1203 11:20:49.892296 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\") pod \"image-registry-697d97f7c8-p77bx\" (UID: \"f5012254-068c-4360-b310-39ff3d309199\") " pod="openshift-image-registry/image-registry-697d97f7c8-p77bx" Dec 03 11:20:49 crc kubenswrapper[5052]: E1203 11:20:49.892776 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8 podName: nodeName:}" failed. No retries permitted until 2025-12-03 11:20:50.392760208 +0000 UTC m=+40.981001972 (durationBeforeRetry 500ms). Error: MountVolume.MountDevice failed for volume "pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8" (UniqueName: "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8") pod "image-registry-697d97f7c8-p77bx" (UID: "f5012254-068c-4360-b310-39ff3d309199") : kubernetes.io/csi: attacher.MountDevice failed to create newCsiDriverClient: driver name kubevirt.io.hostpath-provisioner not found in the list of registered CSI drivers Dec 03 11:20:49 crc kubenswrapper[5052]: I1203 11:20:49.993633 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\") pod \"8f668bae-612b-4b75-9490-919e737c6a3b\" (UID: \"8f668bae-612b-4b75-9490-919e737c6a3b\") " Dec 03 11:20:49 crc kubenswrapper[5052]: E1203 11:20:49.993765 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8 podName:8f668bae-612b-4b75-9490-919e737c6a3b nodeName:}" failed. No retries permitted until 2025-12-03 11:20:50.493746031 +0000 UTC m=+41.081987795 (durationBeforeRetry 500ms). Error: UnmountVolume.TearDown failed for volume "pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8" (UniqueName: "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8") pod "8f668bae-612b-4b75-9490-919e737c6a3b" (UID: "8f668bae-612b-4b75-9490-919e737c6a3b") : kubernetes.io/csi: Unmounter.TearDownAt failed to get CSI client: driver name kubevirt.io.hostpath-provisioner not found in the list of registered CSI drivers Dec 03 11:20:49 crc kubenswrapper[5052]: I1203 11:20:49.993961 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\") pod \"image-registry-697d97f7c8-p77bx\" (UID: \"f5012254-068c-4360-b310-39ff3d309199\") " pod="openshift-image-registry/image-registry-697d97f7c8-p77bx" Dec 03 11:20:49 crc kubenswrapper[5052]: E1203 11:20:49.994329 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8 podName: nodeName:}" failed. No retries permitted until 2025-12-03 11:20:50.494323146 +0000 UTC m=+41.082564910 (durationBeforeRetry 500ms). Error: MountVolume.MountDevice failed for volume "pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8" (UniqueName: "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8") pod "image-registry-697d97f7c8-p77bx" (UID: "f5012254-068c-4360-b310-39ff3d309199") : kubernetes.io/csi: attacher.MountDevice failed to create newCsiDriverClient: driver name kubevirt.io.hostpath-provisioner not found in the list of registered CSI drivers Dec 03 11:20:50 crc kubenswrapper[5052]: W1203 11:20:50.060430 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod833bb55c_5c9b_450c_9287_30f146a2d16a.slice/crio-0427de16d89c2b0460318d208e3759018038e8f8d1f257113d0b0c73abac9076 WatchSource:0}: Error finding container 0427de16d89c2b0460318d208e3759018038e8f8d1f257113d0b0c73abac9076: Status 404 returned error can't find the container with id 0427de16d89c2b0460318d208e3759018038e8f8d1f257113d0b0c73abac9076 Dec 03 11:20:50 crc kubenswrapper[5052]: I1203 11:20:50.095078 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\") pod \"8f668bae-612b-4b75-9490-919e737c6a3b\" (UID: \"8f668bae-612b-4b75-9490-919e737c6a3b\") " Dec 03 11:20:50 crc kubenswrapper[5052]: E1203 11:20:50.095614 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8 podName:8f668bae-612b-4b75-9490-919e737c6a3b nodeName:}" failed. No retries permitted until 2025-12-03 11:20:50.595593337 +0000 UTC m=+41.183835101 (durationBeforeRetry 500ms). Error: UnmountVolume.TearDown failed for volume "pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8" (UniqueName: "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8") pod "8f668bae-612b-4b75-9490-919e737c6a3b" (UID: "8f668bae-612b-4b75-9490-919e737c6a3b") : kubernetes.io/csi: Unmounter.TearDownAt failed to get CSI client: driver name kubevirt.io.hostpath-provisioner not found in the list of registered CSI drivers Dec 03 11:20:50 crc kubenswrapper[5052]: E1203 11:20:50.192968 5052 cadvisor_stats_provider.go:516] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5fe485a1_e14f_4c09_b5b9_f252bc42b7e8.slice/crio-33fca63dd845eae4c12886dc990d984dff16c77109a4e2866862c00f3c21f733.scope\": RecentStats: unable to find data in memory cache]" Dec 03 11:20:50 crc kubenswrapper[5052]: I1203 11:20:50.196801 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\") pod \"image-registry-697d97f7c8-p77bx\" (UID: \"f5012254-068c-4360-b310-39ff3d309199\") " pod="openshift-image-registry/image-registry-697d97f7c8-p77bx" Dec 03 11:20:50 crc kubenswrapper[5052]: E1203 11:20:50.197226 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8 podName: nodeName:}" failed. No retries permitted until 2025-12-03 11:20:50.697208848 +0000 UTC m=+41.285450612 (durationBeforeRetry 500ms). Error: MountVolume.MountDevice failed for volume "pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8" (UniqueName: "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8") pod "image-registry-697d97f7c8-p77bx" (UID: "f5012254-068c-4360-b310-39ff3d309199") : kubernetes.io/csi: attacher.MountDevice failed to create newCsiDriverClient: driver name kubevirt.io.hostpath-provisioner not found in the list of registered CSI drivers Dec 03 11:20:50 crc kubenswrapper[5052]: I1203 11:20:50.297335 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\") pod \"8f668bae-612b-4b75-9490-919e737c6a3b\" (UID: \"8f668bae-612b-4b75-9490-919e737c6a3b\") " Dec 03 11:20:50 crc kubenswrapper[5052]: E1203 11:20:50.297660 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8 podName:8f668bae-612b-4b75-9490-919e737c6a3b nodeName:}" failed. No retries permitted until 2025-12-03 11:20:50.797644486 +0000 UTC m=+41.385886250 (durationBeforeRetry 500ms). Error: UnmountVolume.TearDown failed for volume "pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8" (UniqueName: "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8") pod "8f668bae-612b-4b75-9490-919e737c6a3b" (UID: "8f668bae-612b-4b75-9490-919e737c6a3b") : kubernetes.io/csi: Unmounter.TearDownAt failed to get CSI client: driver name kubevirt.io.hostpath-provisioner not found in the list of registered CSI drivers Dec 03 11:20:50 crc kubenswrapper[5052]: I1203 11:20:50.313621 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-665b6dd947-7wkt6" event={"ID":"59e1467d-d303-4f32-a968-b66eb94953af","Type":"ContainerStarted","Data":"82b94b8c6bb1422375583c42cf7805dc5a411c7d0514ea250087a9d675a919c6"} Dec 03 11:20:50 crc kubenswrapper[5052]: I1203 11:20:50.315228 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-authentication/oauth-openshift-558db77b4-99jsx" event={"ID":"75f62784-7818-4a74-9c8c-4777d6db0611","Type":"ContainerStarted","Data":"f27d0c5e51bbc4e9b04fabcea475200348e7624f94cfe69d2c6bd6690b326182"} Dec 03 11:20:50 crc kubenswrapper[5052]: I1203 11:20:50.319365 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-5444994796-rtl8t" event={"ID":"48040744-7fc1-482e-9b68-3a904ba03a8d","Type":"ContainerStarted","Data":"17e814edb5e3c9c305985ebcc3beb3ac00d132dc54fae39e65b62756157bbd32"} Dec 03 11:20:50 crc kubenswrapper[5052]: I1203 11:20:50.319780 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-59844c95c7-4qts8"] Dec 03 11:20:50 crc kubenswrapper[5052]: I1203 11:20:50.322011 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/cluster-image-registry-operator-dc59b4c8b-4v96x" event={"ID":"407f9c0b-88d2-46f9-9f47-d3af857b0f30","Type":"ContainerStarted","Data":"87d4d86647ea9d6b06f3ae5723bb9f40c7470e8c8f1b0fdb3a050825ae3a6b45"} Dec 03 11:20:50 crc kubenswrapper[5052]: I1203 11:20:50.324614 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-operator-lifecycle-manager/olm-operator-6b444d44fb-6lm8x"] Dec 03 11:20:50 crc kubenswrapper[5052]: I1203 11:20:50.325960 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/machine-config-server-kzd49" event={"ID":"c3a89abc-ba9f-4c13-a73e-2b9e66e9345a","Type":"ContainerStarted","Data":"e33f88b9dec6e70e166c38dfd11dd8ae6255114cebe348c1a9a28cc9b8210961"} Dec 03 11:20:50 crc kubenswrapper[5052]: I1203 11:20:50.327691 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-dns-operator/dns-operator-744455d44c-zvvfc" event={"ID":"cabc2063-04d9-4491-be9f-f7781b5f02cf","Type":"ContainerStarted","Data":"c72e3ba99e5bc4e01e5970baa84c65aa1156ba3c56e0ea77bc2cb33bc00bf68a"} Dec 03 11:20:50 crc kubenswrapper[5052]: I1203 11:20:50.328943 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-operator-lifecycle-manager/packageserver-d55dfcdfc-7s97q"] Dec 03 11:20:50 crc kubenswrapper[5052]: I1203 11:20:50.331984 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/machine-config-controller-84d6567774-rxwdl" event={"ID":"efa98345-cafa-483a-976b-d772a3bd94e7","Type":"ContainerStarted","Data":"c4af2c618f03289f9173e7bcad9f692b301aff88d7ac9bb4d9bc5d9458f4cb62"} Dec 03 11:20:50 crc kubenswrapper[5052]: I1203 11:20:50.333623 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-multus/cni-sysctl-allowlist-ds-8l9gm" event={"ID":"bf761cb9-e024-4acd-a947-0f8064267243","Type":"ContainerStarted","Data":"aaadb8e89911de41a45561de348fecea7160e3a0227674f4f4516c5d5ef9a151"} Dec 03 11:20:50 crc kubenswrapper[5052]: I1203 11:20:50.335077 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-machine-approver/machine-approver-56656f9798-6t9dp" event={"ID":"8c1b25f5-2d98-4852-bc7a-d5cbfa5cb6aa","Type":"ContainerStarted","Data":"1869e1c3ac63697c712528ef4a7f3a2cd3ccfc49517d6b6bffbb25f41e5c1070"} Dec 03 11:20:50 crc kubenswrapper[5052]: I1203 11:20:50.335904 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-7954f5f757-l7xtl" event={"ID":"ec2ea657-018b-4626-bf79-5c818f573a4b","Type":"ContainerStarted","Data":"95d8f9e662e649e39bf4c019cf31fb742f3041cc895f2691960794881a179d8a"} Dec 03 11:20:50 crc kubenswrapper[5052]: I1203 11:20:50.336866 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-85b44fc459-gdk6g" event={"ID":"5fe485a1-e14f-4c09-b5b9-f252bc42b7e8","Type":"ContainerStarted","Data":"33fca63dd845eae4c12886dc990d984dff16c77109a4e2866862c00f3c21f733"} Dec 03 11:20:50 crc kubenswrapper[5052]: I1203 11:20:50.336892 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-85b44fc459-gdk6g" event={"ID":"5fe485a1-e14f-4c09-b5b9-f252bc42b7e8","Type":"ContainerStarted","Data":"ee956411efbf969f87faded1406176dc05bdbd0ae40eac9eacf261a6432125b6"} Dec 03 11:20:50 crc kubenswrapper[5052]: I1203 11:20:50.342634 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-xd92c" event={"ID":"3b6479f0-333b-4a96-9adf-2099afdc2447","Type":"ContainerStarted","Data":"a4b0e3a72f90dbf3afb733238388f7289d3b73e76422ce496bc423b604d2bc93"} Dec 03 11:20:50 crc kubenswrapper[5052]: I1203 11:20:50.343009 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openshift-network-diagnostics/network-check-target-xd92c" Dec 03 11:20:50 crc kubenswrapper[5052]: I1203 11:20:50.343716 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/marketplace-operator-79b997595-kdr8l" event={"ID":"cac70dab-6e13-4693-ac66-c7552b62dfd7","Type":"ContainerStarted","Data":"31ae09a223e991501ffa4143ce1afff7c0f5b63af7a50b309ae6188a2dc5ce6e"} Dec 03 11:20:50 crc kubenswrapper[5052]: I1203 11:20:50.345245 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-operator/ingress-operator-5b745b69d9-hc8vv" event={"ID":"833bb55c-5c9b-450c-9287-30f146a2d16a","Type":"ContainerStarted","Data":"0427de16d89c2b0460318d208e3759018038e8f8d1f257113d0b0c73abac9076"} Dec 03 11:20:50 crc kubenswrapper[5052]: I1203 11:20:50.346700 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-route-controller-manager/route-controller-manager-6576b87f9c-9fpqn" event={"ID":"353b0eac-95ba-420e-bf67-a26b472fcdc2","Type":"ContainerStarted","Data":"bf6c8c8e1f17e65818a13d1e6546cd80d767fd889d2c115fa92c485276bc40f2"} Dec 03 11:20:50 crc kubenswrapper[5052]: I1203 11:20:50.347094 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openshift-route-controller-manager/route-controller-manager-6576b87f9c-9fpqn" Dec 03 11:20:50 crc kubenswrapper[5052]: I1203 11:20:50.348213 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-55646444c4-trplf" event={"ID":"9d751cbb-f2e2-430d-9754-c882a5e924a5","Type":"ContainerStarted","Data":"e516c59c7fea0a1a02e5faa0938de36f7c2b9ebcd091bfbf77355e955d95404e"} Dec 03 11:20:50 crc kubenswrapper[5052]: I1203 11:20:50.349689 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/machine-config-operator-74547568cd-tr2xc" event={"ID":"cc99b3a0-fa04-4128-b730-1d24da63d08e","Type":"ContainerStarted","Data":"44f81948e797051a3da30c55e52a0b42bc84672d74839e66200727481d623fd7"} Dec 03 11:20:50 crc kubenswrapper[5052]: I1203 11:20:50.369049 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-route-controller-manager/route-controller-manager-6576b87f9c-9fpqn" Dec 03 11:20:50 crc kubenswrapper[5052]: I1203 11:20:50.398438 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\") pod \"image-registry-697d97f7c8-p77bx\" (UID: \"f5012254-068c-4360-b310-39ff3d309199\") " pod="openshift-image-registry/image-registry-697d97f7c8-p77bx" Dec 03 11:20:50 crc kubenswrapper[5052]: E1203 11:20:50.399936 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8 podName: nodeName:}" failed. No retries permitted until 2025-12-03 11:20:50.899919374 +0000 UTC m=+41.488161138 (durationBeforeRetry 500ms). Error: MountVolume.MountDevice failed for volume "pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8" (UniqueName: "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8") pod "image-registry-697d97f7c8-p77bx" (UID: "f5012254-068c-4360-b310-39ff3d309199") : kubernetes.io/csi: attacher.MountDevice failed to create newCsiDriverClient: driver name kubevirt.io.hostpath-provisioner not found in the list of registered CSI drivers Dec 03 11:20:50 crc kubenswrapper[5052]: I1203 11:20:50.499271 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\") pod \"8f668bae-612b-4b75-9490-919e737c6a3b\" (UID: \"8f668bae-612b-4b75-9490-919e737c6a3b\") " Dec 03 11:20:50 crc kubenswrapper[5052]: E1203 11:20:50.499644 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8 podName:8f668bae-612b-4b75-9490-919e737c6a3b nodeName:}" failed. No retries permitted until 2025-12-03 11:20:50.999625412 +0000 UTC m=+41.587867176 (durationBeforeRetry 500ms). Error: UnmountVolume.TearDown failed for volume "pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8" (UniqueName: "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8") pod "8f668bae-612b-4b75-9490-919e737c6a3b" (UID: "8f668bae-612b-4b75-9490-919e737c6a3b") : kubernetes.io/csi: Unmounter.TearDownAt failed to get CSI client: driver name kubevirt.io.hostpath-provisioner not found in the list of registered CSI drivers Dec 03 11:20:50 crc kubenswrapper[5052]: I1203 11:20:50.600424 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\") pod \"image-registry-697d97f7c8-p77bx\" (UID: \"f5012254-068c-4360-b310-39ff3d309199\") " pod="openshift-image-registry/image-registry-697d97f7c8-p77bx" Dec 03 11:20:50 crc kubenswrapper[5052]: E1203 11:20:50.601357 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8 podName: nodeName:}" failed. No retries permitted until 2025-12-03 11:20:51.101334195 +0000 UTC m=+41.689576029 (durationBeforeRetry 500ms). Error: MountVolume.MountDevice failed for volume "pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8" (UniqueName: "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8") pod "image-registry-697d97f7c8-p77bx" (UID: "f5012254-068c-4360-b310-39ff3d309199") : kubernetes.io/csi: attacher.MountDevice failed to create newCsiDriverClient: driver name kubevirt.io.hostpath-provisioner not found in the list of registered CSI drivers Dec 03 11:20:50 crc kubenswrapper[5052]: I1203 11:20:50.702187 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\") pod \"8f668bae-612b-4b75-9490-919e737c6a3b\" (UID: \"8f668bae-612b-4b75-9490-919e737c6a3b\") " Dec 03 11:20:50 crc kubenswrapper[5052]: E1203 11:20:50.702535 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8 podName:8f668bae-612b-4b75-9490-919e737c6a3b nodeName:}" failed. No retries permitted until 2025-12-03 11:20:51.202495283 +0000 UTC m=+41.790737047 (durationBeforeRetry 500ms). Error: UnmountVolume.TearDown failed for volume "pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8" (UniqueName: "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8") pod "8f668bae-612b-4b75-9490-919e737c6a3b" (UID: "8f668bae-612b-4b75-9490-919e737c6a3b") : kubernetes.io/csi: Unmounter.TearDownAt failed to get CSI client: driver name kubevirt.io.hostpath-provisioner not found in the list of registered CSI drivers Dec 03 11:20:50 crc kubenswrapper[5052]: I1203 11:20:50.803074 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\") pod \"image-registry-697d97f7c8-p77bx\" (UID: \"f5012254-068c-4360-b310-39ff3d309199\") " pod="openshift-image-registry/image-registry-697d97f7c8-p77bx" Dec 03 11:20:50 crc kubenswrapper[5052]: E1203 11:20:50.803528 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8 podName: nodeName:}" failed. No retries permitted until 2025-12-03 11:20:51.303494686 +0000 UTC m=+41.891736450 (durationBeforeRetry 500ms). Error: MountVolume.MountDevice failed for volume "pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8" (UniqueName: "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8") pod "image-registry-697d97f7c8-p77bx" (UID: "f5012254-068c-4360-b310-39ff3d309199") : kubernetes.io/csi: attacher.MountDevice failed to create newCsiDriverClient: driver name kubevirt.io.hostpath-provisioner not found in the list of registered CSI drivers Dec 03 11:20:50 crc kubenswrapper[5052]: I1203 11:20:50.904740 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\") pod \"8f668bae-612b-4b75-9490-919e737c6a3b\" (UID: \"8f668bae-612b-4b75-9490-919e737c6a3b\") " Dec 03 11:20:50 crc kubenswrapper[5052]: E1203 11:20:50.904932 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8 podName:8f668bae-612b-4b75-9490-919e737c6a3b nodeName:}" failed. No retries permitted until 2025-12-03 11:20:51.40487934 +0000 UTC m=+41.993121424 (durationBeforeRetry 500ms). Error: UnmountVolume.TearDown failed for volume "pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8" (UniqueName: "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8") pod "8f668bae-612b-4b75-9490-919e737c6a3b" (UID: "8f668bae-612b-4b75-9490-919e737c6a3b") : kubernetes.io/csi: Unmounter.TearDownAt failed to get CSI client: driver name kubevirt.io.hostpath-provisioner not found in the list of registered CSI drivers Dec 03 11:20:50 crc kubenswrapper[5052]: I1203 11:20:50.905588 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\") pod \"image-registry-697d97f7c8-p77bx\" (UID: \"f5012254-068c-4360-b310-39ff3d309199\") " pod="openshift-image-registry/image-registry-697d97f7c8-p77bx" Dec 03 11:20:50 crc kubenswrapper[5052]: E1203 11:20:50.908523 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8 podName: nodeName:}" failed. No retries permitted until 2025-12-03 11:20:51.40848337 +0000 UTC m=+41.996725134 (durationBeforeRetry 500ms). Error: MountVolume.MountDevice failed for volume "pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8" (UniqueName: "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8") pod "image-registry-697d97f7c8-p77bx" (UID: "f5012254-068c-4360-b310-39ff3d309199") : kubernetes.io/csi: attacher.MountDevice failed to create newCsiDriverClient: driver name kubevirt.io.hostpath-provisioner not found in the list of registered CSI drivers Dec 03 11:20:50 crc kubenswrapper[5052]: I1203 11:20:50.979729 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-route-controller-manager/route-controller-manager-6576b87f9c-9fpqn" podStartSLOduration=18.979582319 podStartE2EDuration="18.979582319s" podCreationTimestamp="2025-12-03 11:20:32 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:20:50.976372161 +0000 UTC m=+41.564613925" watchObservedRunningTime="2025-12-03 11:20:50.979582319 +0000 UTC m=+41.567824083" Dec 03 11:20:51 crc kubenswrapper[5052]: I1203 11:20:51.009261 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\") pod \"8f668bae-612b-4b75-9490-919e737c6a3b\" (UID: \"8f668bae-612b-4b75-9490-919e737c6a3b\") " Dec 03 11:20:51 crc kubenswrapper[5052]: E1203 11:20:51.009541 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8 podName:8f668bae-612b-4b75-9490-919e737c6a3b nodeName:}" failed. No retries permitted until 2025-12-03 11:20:51.509496643 +0000 UTC m=+42.097738407 (durationBeforeRetry 500ms). Error: UnmountVolume.TearDown failed for volume "pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8" (UniqueName: "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8") pod "8f668bae-612b-4b75-9490-919e737c6a3b" (UID: "8f668bae-612b-4b75-9490-919e737c6a3b") : kubernetes.io/csi: Unmounter.TearDownAt failed to get CSI client: driver name kubevirt.io.hostpath-provisioner not found in the list of registered CSI drivers Dec 03 11:20:51 crc kubenswrapper[5052]: I1203 11:20:51.009918 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\") pod \"image-registry-697d97f7c8-p77bx\" (UID: \"f5012254-068c-4360-b310-39ff3d309199\") " pod="openshift-image-registry/image-registry-697d97f7c8-p77bx" Dec 03 11:20:51 crc kubenswrapper[5052]: E1203 11:20:51.010502 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8 podName: nodeName:}" failed. No retries permitted until 2025-12-03 11:20:51.510484611 +0000 UTC m=+42.098726375 (durationBeforeRetry 500ms). Error: MountVolume.MountDevice failed for volume "pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8" (UniqueName: "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8") pod "image-registry-697d97f7c8-p77bx" (UID: "f5012254-068c-4360-b310-39ff3d309199") : kubernetes.io/csi: attacher.MountDevice failed to create newCsiDriverClient: driver name kubevirt.io.hostpath-provisioner not found in the list of registered CSI drivers Dec 03 11:20:51 crc kubenswrapper[5052]: I1203 11:20:51.111548 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\") pod \"8f668bae-612b-4b75-9490-919e737c6a3b\" (UID: \"8f668bae-612b-4b75-9490-919e737c6a3b\") " Dec 03 11:20:51 crc kubenswrapper[5052]: E1203 11:20:51.111997 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8 podName:8f668bae-612b-4b75-9490-919e737c6a3b nodeName:}" failed. No retries permitted until 2025-12-03 11:20:51.611959197 +0000 UTC m=+42.200200971 (durationBeforeRetry 500ms). Error: UnmountVolume.TearDown failed for volume "pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8" (UniqueName: "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8") pod "8f668bae-612b-4b75-9490-919e737c6a3b" (UID: "8f668bae-612b-4b75-9490-919e737c6a3b") : kubernetes.io/csi: Unmounter.TearDownAt failed to get CSI client: driver name kubevirt.io.hostpath-provisioner not found in the list of registered CSI drivers Dec 03 11:20:51 crc kubenswrapper[5052]: I1203 11:20:51.112219 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\") pod \"image-registry-697d97f7c8-p77bx\" (UID: \"f5012254-068c-4360-b310-39ff3d309199\") " pod="openshift-image-registry/image-registry-697d97f7c8-p77bx" Dec 03 11:20:51 crc kubenswrapper[5052]: I1203 11:20:51.112290 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a3f2b85d-9b43-4c1c-bc3a-602405b96efc-metrics-certs\") pod \"network-metrics-daemon-kf52k\" (UID: \"a3f2b85d-9b43-4c1c-bc3a-602405b96efc\") " pod="openshift-multus/network-metrics-daemon-kf52k" Dec 03 11:20:51 crc kubenswrapper[5052]: E1203 11:20:51.112813 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8 podName: nodeName:}" failed. No retries permitted until 2025-12-03 11:20:51.61280157 +0000 UTC m=+42.201043414 (durationBeforeRetry 500ms). Error: MountVolume.MountDevice failed for volume "pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8" (UniqueName: "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8") pod "image-registry-697d97f7c8-p77bx" (UID: "f5012254-068c-4360-b310-39ff3d309199") : kubernetes.io/csi: attacher.MountDevice failed to create newCsiDriverClient: driver name kubevirt.io.hostpath-provisioner not found in the list of registered CSI drivers Dec 03 11:20:51 crc kubenswrapper[5052]: I1203 11:20:51.138969 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a3f2b85d-9b43-4c1c-bc3a-602405b96efc-metrics-certs\") pod \"network-metrics-daemon-kf52k\" (UID: \"a3f2b85d-9b43-4c1c-bc3a-602405b96efc\") " pod="openshift-multus/network-metrics-daemon-kf52k" Dec 03 11:20:51 crc kubenswrapper[5052]: I1203 11:20:51.212941 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\") pod \"8f668bae-612b-4b75-9490-919e737c6a3b\" (UID: \"8f668bae-612b-4b75-9490-919e737c6a3b\") " Dec 03 11:20:51 crc kubenswrapper[5052]: E1203 11:20:51.213143 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8 podName:8f668bae-612b-4b75-9490-919e737c6a3b nodeName:}" failed. No retries permitted until 2025-12-03 11:20:51.713099645 +0000 UTC m=+42.301341409 (durationBeforeRetry 500ms). Error: UnmountVolume.TearDown failed for volume "pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8" (UniqueName: "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8") pod "8f668bae-612b-4b75-9490-919e737c6a3b" (UID: "8f668bae-612b-4b75-9490-919e737c6a3b") : kubernetes.io/csi: Unmounter.TearDownAt failed to get CSI client: driver name kubevirt.io.hostpath-provisioner not found in the list of registered CSI drivers Dec 03 11:20:51 crc kubenswrapper[5052]: I1203 11:20:51.213292 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\") pod \"image-registry-697d97f7c8-p77bx\" (UID: \"f5012254-068c-4360-b310-39ff3d309199\") " pod="openshift-image-registry/image-registry-697d97f7c8-p77bx" Dec 03 11:20:51 crc kubenswrapper[5052]: E1203 11:20:51.213751 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8 podName: nodeName:}" failed. No retries permitted until 2025-12-03 11:20:51.713729802 +0000 UTC m=+42.301971566 (durationBeforeRetry 500ms). Error: MountVolume.MountDevice failed for volume "pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8" (UniqueName: "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8") pod "image-registry-697d97f7c8-p77bx" (UID: "f5012254-068c-4360-b310-39ff3d309199") : kubernetes.io/csi: attacher.MountDevice failed to create newCsiDriverClient: driver name kubevirt.io.hostpath-provisioner not found in the list of registered CSI drivers Dec 03 11:20:51 crc kubenswrapper[5052]: I1203 11:20:51.221764 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-kube-apiserver/kube-apiserver-crc" Dec 03 11:20:51 crc kubenswrapper[5052]: I1203 11:20:51.314674 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\") pod \"8f668bae-612b-4b75-9490-919e737c6a3b\" (UID: \"8f668bae-612b-4b75-9490-919e737c6a3b\") " Dec 03 11:20:51 crc kubenswrapper[5052]: E1203 11:20:51.315047 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8 podName:8f668bae-612b-4b75-9490-919e737c6a3b nodeName:}" failed. No retries permitted until 2025-12-03 11:20:51.815026394 +0000 UTC m=+42.403268158 (durationBeforeRetry 500ms). Error: UnmountVolume.TearDown failed for volume "pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8" (UniqueName: "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8") pod "8f668bae-612b-4b75-9490-919e737c6a3b" (UID: "8f668bae-612b-4b75-9490-919e737c6a3b") : kubernetes.io/csi: Unmounter.TearDownAt failed to get CSI client: driver name kubevirt.io.hostpath-provisioner not found in the list of registered CSI drivers Dec 03 11:20:51 crc kubenswrapper[5052]: I1203 11:20:51.361371 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/marketplace-operator-79b997595-kdr8l" event={"ID":"cac70dab-6e13-4693-ac66-c7552b62dfd7","Type":"ContainerStarted","Data":"85ba752630aa58b2a1baa94ed4769d91d59327218485bf8f7d30a5cf17fe7c22"} Dec 03 11:20:51 crc kubenswrapper[5052]: I1203 11:20:51.362872 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openshift-marketplace/marketplace-operator-79b997595-kdr8l" Dec 03 11:20:51 crc kubenswrapper[5052]: I1203 11:20:51.368614 5052 patch_prober.go:28] interesting pod/marketplace-operator-79b997595-kdr8l container/marketplace-operator namespace/openshift-marketplace: Readiness probe status=failure output="Get \"http://10.217.0.31:8080/healthz\": dial tcp 10.217.0.31:8080: connect: connection refused" start-of-body= Dec 03 11:20:51 crc kubenswrapper[5052]: I1203 11:20:51.368681 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openshift-marketplace/marketplace-operator-79b997595-kdr8l" podUID="cac70dab-6e13-4693-ac66-c7552b62dfd7" containerName="marketplace-operator" probeResult="failure" output="Get \"http://10.217.0.31:8080/healthz\": dial tcp 10.217.0.31:8080: connect: connection refused" Dec 03 11:20:51 crc kubenswrapper[5052]: I1203 11:20:51.369312 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/cluster-image-registry-operator-dc59b4c8b-4v96x" event={"ID":"407f9c0b-88d2-46f9-9f47-d3af857b0f30","Type":"ContainerStarted","Data":"09b735c3d6b9239ac5b04e45a0caa6f01781354eb9dfad6b2245ce039f6d8ba4"} Dec 03 11:20:51 crc kubenswrapper[5052]: I1203 11:20:51.373017 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-operator-lifecycle-manager/packageserver-d55dfcdfc-7s97q" event={"ID":"6413eb91-7b61-4660-888c-14da224c7dfa","Type":"ContainerStarted","Data":"042473d467b5c45d6165f738a4f2f9c21622fa3786948ad044dc432284bdf99e"} Dec 03 11:20:51 crc kubenswrapper[5052]: I1203 11:20:51.374620 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/machine-config-operator-74547568cd-tr2xc" event={"ID":"cc99b3a0-fa04-4128-b730-1d24da63d08e","Type":"ContainerStarted","Data":"2ec396890638cd2c8fd7015c18bf6bc8f6fffcc4a5f2be92ba38b5dd5b94ea77"} Dec 03 11:20:51 crc kubenswrapper[5052]: I1203 11:20:51.382196 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-operator-lifecycle-manager/olm-operator-6b444d44fb-6lm8x" event={"ID":"663b7426-fde0-4b59-b857-b434b4a88f4b","Type":"ContainerStarted","Data":"0de9c7b35aec36de2c6b1e4ffb9ca60702f857286a626d614fd44b8d03398667"} Dec 03 11:20:51 crc kubenswrapper[5052]: I1203 11:20:51.384758 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-5444994796-rtl8t" event={"ID":"48040744-7fc1-482e-9b68-3a904ba03a8d","Type":"ContainerStarted","Data":"71901005ccf6a02a5a033b3a70cfa8f069ebba69d834fc253bcb4ea3962b0c1c"} Dec 03 11:20:51 crc kubenswrapper[5052]: I1203 11:20:51.389206 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kf52k" Dec 03 11:20:51 crc kubenswrapper[5052]: I1203 11:20:51.390217 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-marketplace/marketplace-operator-79b997595-kdr8l" podStartSLOduration=19.390201255 podStartE2EDuration="19.390201255s" podCreationTimestamp="2025-12-03 11:20:32 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:20:51.388580591 +0000 UTC m=+41.976822375" watchObservedRunningTime="2025-12-03 11:20:51.390201255 +0000 UTC m=+41.978443019" Dec 03 11:20:51 crc kubenswrapper[5052]: I1203 11:20:51.398914 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-665b6dd947-7wkt6" event={"ID":"59e1467d-d303-4f32-a968-b66eb94953af","Type":"ContainerStarted","Data":"aeba6ce16b5eb3aec23df441b5e20c6e38041ac8194859977572fcd1beaac98d"} Dec 03 11:20:51 crc kubenswrapper[5052]: I1203 11:20:51.401797 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-dns-operator/dns-operator-744455d44c-zvvfc" event={"ID":"cabc2063-04d9-4491-be9f-f7781b5f02cf","Type":"ContainerStarted","Data":"64f581b73ad78736f812ec077ae699396197df891cfa398e56329dee95b8420d"} Dec 03 11:20:51 crc kubenswrapper[5052]: I1203 11:20:51.416207 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress/router-default-5444994796-rtl8t" podStartSLOduration=19.416185451 podStartE2EDuration="19.416185451s" podCreationTimestamp="2025-12-03 11:20:32 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:20:51.415209044 +0000 UTC m=+42.003450808" watchObservedRunningTime="2025-12-03 11:20:51.416185451 +0000 UTC m=+42.004427215" Dec 03 11:20:51 crc kubenswrapper[5052]: I1203 11:20:51.419770 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\") pod \"image-registry-697d97f7c8-p77bx\" (UID: \"f5012254-068c-4360-b310-39ff3d309199\") " pod="openshift-image-registry/image-registry-697d97f7c8-p77bx" Dec 03 11:20:51 crc kubenswrapper[5052]: E1203 11:20:51.421717 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8 podName: nodeName:}" failed. No retries permitted until 2025-12-03 11:20:51.921704254 +0000 UTC m=+42.509946018 (durationBeforeRetry 500ms). Error: MountVolume.MountDevice failed for volume "pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8" (UniqueName: "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8") pod "image-registry-697d97f7c8-p77bx" (UID: "f5012254-068c-4360-b310-39ff3d309199") : kubernetes.io/csi: attacher.MountDevice failed to create newCsiDriverClient: driver name kubevirt.io.hostpath-provisioner not found in the list of registered CSI drivers Dec 03 11:20:51 crc kubenswrapper[5052]: I1203 11:20:51.421782 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-7954f5f757-l7xtl" event={"ID":"ec2ea657-018b-4626-bf79-5c818f573a4b","Type":"ContainerStarted","Data":"b70cadb5c4d599092898ac0f6233f41fe3a6b61c9dd488fa803569714579f823"} Dec 03 11:20:51 crc kubenswrapper[5052]: I1203 11:20:51.422544 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openshift-console/downloads-7954f5f757-l7xtl" Dec 03 11:20:51 crc kubenswrapper[5052]: I1203 11:20:51.429983 5052 patch_prober.go:28] interesting pod/downloads-7954f5f757-l7xtl container/download-server namespace/openshift-console: Readiness probe status=failure output="Get \"http://10.217.0.18:8080/\": dial tcp 10.217.0.18:8080: connect: connection refused" start-of-body= Dec 03 11:20:51 crc kubenswrapper[5052]: I1203 11:20:51.430019 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openshift-console/downloads-7954f5f757-l7xtl" podUID="ec2ea657-018b-4626-bf79-5c818f573a4b" containerName="download-server" probeResult="failure" output="Get \"http://10.217.0.18:8080/\": dial tcp 10.217.0.18:8080: connect: connection refused" Dec 03 11:20:51 crc kubenswrapper[5052]: I1203 11:20:51.443808 5052 generic.go:334] "Generic (PLEG): container finished" podID="8fae1d72-fba4-4999-bea5-39d6472d7ea7" containerID="f17adfa1c878a38ab6a4ec8434309eac1f9964f19600d4180bcdc69483543180" exitCode=0 Dec 03 11:20:51 crc kubenswrapper[5052]: I1203 11:20:51.444004 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-oauth-apiserver/apiserver-7bbb656c7d-86vx8" event={"ID":"8fae1d72-fba4-4999-bea5-39d6472d7ea7","Type":"ContainerDied","Data":"f17adfa1c878a38ab6a4ec8434309eac1f9964f19600d4180bcdc69483543180"} Dec 03 11:20:51 crc kubenswrapper[5052]: I1203 11:20:51.453822 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-59844c95c7-4qts8" event={"ID":"199a472c-800b-45e2-a507-7e6cb2c7427f","Type":"ContainerStarted","Data":"89ba05099ca8908a22f3655380ffa47466441a7d7880b257a9716edff23ce3a7"} Dec 03 11:20:51 crc kubenswrapper[5052]: I1203 11:20:51.524306 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\") pod \"8f668bae-612b-4b75-9490-919e737c6a3b\" (UID: \"8f668bae-612b-4b75-9490-919e737c6a3b\") " Dec 03 11:20:51 crc kubenswrapper[5052]: E1203 11:20:51.524666 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8 podName:8f668bae-612b-4b75-9490-919e737c6a3b nodeName:}" failed. No retries permitted until 2025-12-03 11:20:52.02463523 +0000 UTC m=+42.612876994 (durationBeforeRetry 500ms). Error: UnmountVolume.TearDown failed for volume "pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8" (UniqueName: "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8") pod "8f668bae-612b-4b75-9490-919e737c6a3b" (UID: "8f668bae-612b-4b75-9490-919e737c6a3b") : kubernetes.io/csi: Unmounter.TearDownAt failed to get CSI client: driver name kubevirt.io.hostpath-provisioner not found in the list of registered CSI drivers Dec 03 11:20:51 crc kubenswrapper[5052]: I1203 11:20:51.524764 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\") pod \"image-registry-697d97f7c8-p77bx\" (UID: \"f5012254-068c-4360-b310-39ff3d309199\") " pod="openshift-image-registry/image-registry-697d97f7c8-p77bx" Dec 03 11:20:51 crc kubenswrapper[5052]: E1203 11:20:51.525943 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8 podName: nodeName:}" failed. No retries permitted until 2025-12-03 11:20:52.025933976 +0000 UTC m=+42.614175820 (durationBeforeRetry 500ms). Error: MountVolume.MountDevice failed for volume "pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8" (UniqueName: "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8") pod "image-registry-697d97f7c8-p77bx" (UID: "f5012254-068c-4360-b310-39ff3d309199") : kubernetes.io/csi: attacher.MountDevice failed to create newCsiDriverClient: driver name kubevirt.io.hostpath-provisioner not found in the list of registered CSI drivers Dec 03 11:20:51 crc kubenswrapper[5052]: I1203 11:20:51.638306 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/downloads-7954f5f757-l7xtl" podStartSLOduration=20.638284432 podStartE2EDuration="20.638284432s" podCreationTimestamp="2025-12-03 11:20:31 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:20:51.612120871 +0000 UTC m=+42.200362635" watchObservedRunningTime="2025-12-03 11:20:51.638284432 +0000 UTC m=+42.226526186" Dec 03 11:20:51 crc kubenswrapper[5052]: I1203 11:20:51.639767 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/cluster-image-registry-operator-dc59b4c8b-4v96x" podStartSLOduration=19.639761613 podStartE2EDuration="19.639761613s" podCreationTimestamp="2025-12-03 11:20:32 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:20:51.47382077 +0000 UTC m=+42.062062534" watchObservedRunningTime="2025-12-03 11:20:51.639761613 +0000 UTC m=+42.228003377" Dec 03 11:20:51 crc kubenswrapper[5052]: I1203 11:20:51.642209 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\") pod \"8f668bae-612b-4b75-9490-919e737c6a3b\" (UID: \"8f668bae-612b-4b75-9490-919e737c6a3b\") " Dec 03 11:20:51 crc kubenswrapper[5052]: E1203 11:20:51.642837 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8 podName:8f668bae-612b-4b75-9490-919e737c6a3b nodeName:}" failed. No retries permitted until 2025-12-03 11:20:52.142820307 +0000 UTC m=+42.731062071 (durationBeforeRetry 500ms). Error: UnmountVolume.TearDown failed for volume "pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8" (UniqueName: "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8") pod "8f668bae-612b-4b75-9490-919e737c6a3b" (UID: "8f668bae-612b-4b75-9490-919e737c6a3b") : kubernetes.io/csi: Unmounter.TearDownAt failed to get CSI client: driver name kubevirt.io.hostpath-provisioner not found in the list of registered CSI drivers Dec 03 11:20:51 crc kubenswrapper[5052]: I1203 11:20:51.747529 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\") pod \"image-registry-697d97f7c8-p77bx\" (UID: \"f5012254-068c-4360-b310-39ff3d309199\") " pod="openshift-image-registry/image-registry-697d97f7c8-p77bx" Dec 03 11:20:51 crc kubenswrapper[5052]: E1203 11:20:51.748145 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8 podName: nodeName:}" failed. No retries permitted until 2025-12-03 11:20:52.248131039 +0000 UTC m=+42.836372803 (durationBeforeRetry 500ms). Error: MountVolume.MountDevice failed for volume "pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8" (UniqueName: "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8") pod "image-registry-697d97f7c8-p77bx" (UID: "f5012254-068c-4360-b310-39ff3d309199") : kubernetes.io/csi: attacher.MountDevice failed to create newCsiDriverClient: driver name kubevirt.io.hostpath-provisioner not found in the list of registered CSI drivers Dec 03 11:20:51 crc kubenswrapper[5052]: I1203 11:20:51.849229 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\") pod \"8f668bae-612b-4b75-9490-919e737c6a3b\" (UID: \"8f668bae-612b-4b75-9490-919e737c6a3b\") " Dec 03 11:20:51 crc kubenswrapper[5052]: E1203 11:20:51.849598 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8 podName:8f668bae-612b-4b75-9490-919e737c6a3b nodeName:}" failed. No retries permitted until 2025-12-03 11:20:52.349578945 +0000 UTC m=+42.937820709 (durationBeforeRetry 500ms). Error: UnmountVolume.TearDown failed for volume "pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8" (UniqueName: "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8") pod "8f668bae-612b-4b75-9490-919e737c6a3b" (UID: "8f668bae-612b-4b75-9490-919e737c6a3b") : kubernetes.io/csi: Unmounter.TearDownAt failed to get CSI client: driver name kubevirt.io.hostpath-provisioner not found in the list of registered CSI drivers Dec 03 11:20:51 crc kubenswrapper[5052]: I1203 11:20:51.878452 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-machine-api/control-plane-machine-set-operator-78cbb6b69f-p4b4l"] Dec 03 11:20:51 crc kubenswrapper[5052]: I1203 11:20:51.898469 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-controller-manager-operator/openshift-controller-manager-operator-756b6f6bc6-v7lmf"] Dec 03 11:20:51 crc kubenswrapper[5052]: I1203 11:20:51.905445 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-kube-apiserver-operator/kube-apiserver-operator-766d6c64bb-xwlk8"] Dec 03 11:20:51 crc kubenswrapper[5052]: I1203 11:20:51.906184 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-777779d784-9jvh6"] Dec 03 11:20:51 crc kubenswrapper[5052]: I1203 11:20:51.951147 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\") pod \"image-registry-697d97f7c8-p77bx\" (UID: \"f5012254-068c-4360-b310-39ff3d309199\") " pod="openshift-image-registry/image-registry-697d97f7c8-p77bx" Dec 03 11:20:51 crc kubenswrapper[5052]: E1203 11:20:51.951719 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8 podName: nodeName:}" failed. No retries permitted until 2025-12-03 11:20:52.45170303 +0000 UTC m=+43.039944794 (durationBeforeRetry 500ms). Error: MountVolume.MountDevice failed for volume "pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8" (UniqueName: "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8") pod "image-registry-697d97f7c8-p77bx" (UID: "f5012254-068c-4360-b310-39ff3d309199") : kubernetes.io/csi: attacher.MountDevice failed to create newCsiDriverClient: driver name kubevirt.io.hostpath-provisioner not found in the list of registered CSI drivers Dec 03 11:20:52 crc kubenswrapper[5052]: W1203 11:20:52.017692 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e0657c6_3f3c_4300_8cfb_b3db773dba75.slice/crio-76cd17f055a2d819a2006f39b12f4e1af3fd73831fdbc6c06e0abd1fa48b25f7 WatchSource:0}: Error finding container 76cd17f055a2d819a2006f39b12f4e1af3fd73831fdbc6c06e0abd1fa48b25f7: Status 404 returned error can't find the container with id 76cd17f055a2d819a2006f39b12f4e1af3fd73831fdbc6c06e0abd1fa48b25f7 Dec 03 11:20:52 crc kubenswrapper[5052]: I1203 11:20:52.020961 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-kube-controller-manager-operator/kube-controller-manager-operator-78b949d7b-h6b78"] Dec 03 11:20:52 crc kubenswrapper[5052]: I1203 11:20:52.053161 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-operator-lifecycle-manager/catalog-operator-68c6474976-rqhkg"] Dec 03 11:20:52 crc kubenswrapper[5052]: I1203 11:20:52.059940 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\") pod \"8f668bae-612b-4b75-9490-919e737c6a3b\" (UID: \"8f668bae-612b-4b75-9490-919e737c6a3b\") " Dec 03 11:20:52 crc kubenswrapper[5052]: E1203 11:20:52.062159 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8 podName:8f668bae-612b-4b75-9490-919e737c6a3b nodeName:}" failed. No retries permitted until 2025-12-03 11:20:52.562122523 +0000 UTC m=+43.150364287 (durationBeforeRetry 500ms). Error: UnmountVolume.TearDown failed for volume "pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8" (UniqueName: "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8") pod "8f668bae-612b-4b75-9490-919e737c6a3b" (UID: "8f668bae-612b-4b75-9490-919e737c6a3b") : kubernetes.io/csi: Unmounter.TearDownAt failed to get CSI client: driver name kubevirt.io.hostpath-provisioner not found in the list of registered CSI drivers Dec 03 11:20:52 crc kubenswrapper[5052]: I1203 11:20:52.067443 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-9c57cc56f-w7g95"] Dec 03 11:20:52 crc kubenswrapper[5052]: I1203 11:20:52.069126 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\") pod \"image-registry-697d97f7c8-p77bx\" (UID: \"f5012254-068c-4360-b310-39ff3d309199\") " pod="openshift-image-registry/image-registry-697d97f7c8-p77bx" Dec 03 11:20:52 crc kubenswrapper[5052]: E1203 11:20:52.071465 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8 podName: nodeName:}" failed. No retries permitted until 2025-12-03 11:20:52.57144995 +0000 UTC m=+43.159691714 (durationBeforeRetry 500ms). Error: MountVolume.MountDevice failed for volume "pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8" (UniqueName: "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8") pod "image-registry-697d97f7c8-p77bx" (UID: "f5012254-068c-4360-b310-39ff3d309199") : kubernetes.io/csi: attacher.MountDevice failed to create newCsiDriverClient: driver name kubevirt.io.hostpath-provisioner not found in the list of registered CSI drivers Dec 03 11:20:52 crc kubenswrapper[5052]: I1203 11:20:52.126590 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-config-operator/openshift-config-operator-7777fb866f-bkztv"] Dec 03 11:20:52 crc kubenswrapper[5052]: I1203 11:20:52.158309 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-etcd-operator/etcd-operator-b45778765-z2lp6"] Dec 03 11:20:52 crc kubenswrapper[5052]: I1203 11:20:52.160643 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-apiserver-operator/openshift-apiserver-operator-796bbdcf4f-jskzg"] Dec 03 11:20:52 crc kubenswrapper[5052]: I1203 11:20:52.167078 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-58897d9998-7vpns"] Dec 03 11:20:52 crc kubenswrapper[5052]: I1203 11:20:52.181389 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\") pod \"8f668bae-612b-4b75-9490-919e737c6a3b\" (UID: \"8f668bae-612b-4b75-9490-919e737c6a3b\") " Dec 03 11:20:52 crc kubenswrapper[5052]: E1203 11:20:52.181917 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8 podName:8f668bae-612b-4b75-9490-919e737c6a3b nodeName:}" failed. No retries permitted until 2025-12-03 11:20:52.681881323 +0000 UTC m=+43.270123087 (durationBeforeRetry 500ms). Error: UnmountVolume.TearDown failed for volume "pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8" (UniqueName: "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8") pod "8f668bae-612b-4b75-9490-919e737c6a3b" (UID: "8f668bae-612b-4b75-9490-919e737c6a3b") : kubernetes.io/csi: Unmounter.TearDownAt failed to get CSI client: driver name kubevirt.io.hostpath-provisioner not found in the list of registered CSI drivers Dec 03 11:20:52 crc kubenswrapper[5052]: I1203 11:20:52.206064 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-kube-scheduler-operator/openshift-kube-scheduler-operator-5fdd9b5758-rxfb5"] Dec 03 11:20:52 crc kubenswrapper[5052]: I1203 11:20:52.215350 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-multus/multus-admission-controller-857f4d67dd-q8n58"] Dec 03 11:20:52 crc kubenswrapper[5052]: I1203 11:20:52.246192 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-5444994796-rtl8t" Dec 03 11:20:52 crc kubenswrapper[5052]: I1203 11:20:52.256698 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-operator-lifecycle-manager/collect-profiles-29412675-hzd4n"] Dec 03 11:20:52 crc kubenswrapper[5052]: I1203 11:20:52.264101 5052 patch_prober.go:28] interesting pod/router-default-5444994796-rtl8t container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-http failed: reason withheld Dec 03 11:20:52 crc kubenswrapper[5052]: [-]has-synced failed: reason withheld Dec 03 11:20:52 crc kubenswrapper[5052]: [+]process-running ok Dec 03 11:20:52 crc kubenswrapper[5052]: healthz check failed Dec 03 11:20:52 crc kubenswrapper[5052]: I1203 11:20:52.264218 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-5444994796-rtl8t" podUID="48040744-7fc1-482e-9b68-3a904ba03a8d" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500" Dec 03 11:20:52 crc kubenswrapper[5052]: W1203 11:20:52.268230 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a110335_c2e5_40a2_b565_a4791146d53b.slice/crio-5cfc098995ed3eab105030845dafc82ac0782e8a25572b88ec76854e98a928db WatchSource:0}: Error finding container 5cfc098995ed3eab105030845dafc82ac0782e8a25572b88ec76854e98a928db: Status 404 returned error can't find the container with id 5cfc098995ed3eab105030845dafc82ac0782e8a25572b88ec76854e98a928db Dec 03 11:20:52 crc kubenswrapper[5052]: I1203 11:20:52.276312 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-controller-manager/controller-manager-879f6c89f-46rm8"] Dec 03 11:20:52 crc kubenswrapper[5052]: I1203 11:20:52.283163 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\") pod \"image-registry-697d97f7c8-p77bx\" (UID: \"f5012254-068c-4360-b310-39ff3d309199\") " pod="openshift-image-registry/image-registry-697d97f7c8-p77bx" Dec 03 11:20:52 crc kubenswrapper[5052]: E1203 11:20:52.283691 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8 podName: nodeName:}" failed. No retries permitted until 2025-12-03 11:20:52.783677008 +0000 UTC m=+43.371918772 (durationBeforeRetry 500ms). Error: MountVolume.MountDevice failed for volume "pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8" (UniqueName: "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8") pod "image-registry-697d97f7c8-p77bx" (UID: "f5012254-068c-4360-b310-39ff3d309199") : kubernetes.io/csi: attacher.MountDevice failed to create newCsiDriverClient: driver name kubevirt.io.hostpath-provisioner not found in the list of registered CSI drivers Dec 03 11:20:52 crc kubenswrapper[5052]: I1203 11:20:52.287865 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-authentication-operator/authentication-operator-69f744f599-qjq5f"] Dec 03 11:20:52 crc kubenswrapper[5052]: W1203 11:20:52.297268 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42cd7f06_f712_41e0_9fa5_c8ec9f7af997.slice/crio-b3919e1210bcd9f746eea3aafcfd8f8f1da750ba2ad12dde596c86a45ca1d3d7 WatchSource:0}: Error finding container b3919e1210bcd9f746eea3aafcfd8f8f1da750ba2ad12dde596c86a45ca1d3d7: Status 404 returned error can't find the container with id b3919e1210bcd9f746eea3aafcfd8f8f1da750ba2ad12dde596c86a45ca1d3d7 Dec 03 11:20:52 crc kubenswrapper[5052]: I1203 11:20:52.335647 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-6m2jn"] Dec 03 11:20:52 crc kubenswrapper[5052]: I1203 11:20:52.352839 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-tvr2s"] Dec 03 11:20:52 crc kubenswrapper[5052]: I1203 11:20:52.352916 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-machine-api/machine-api-operator-5694c8668f-l5lrg"] Dec 03 11:20:52 crc kubenswrapper[5052]: I1203 11:20:52.359580 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-apiserver/apiserver-76f77b778f-dmvst"] Dec 03 11:20:52 crc kubenswrapper[5052]: I1203 11:20:52.361757 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-kf52k"] Dec 03 11:20:52 crc kubenswrapper[5052]: I1203 11:20:52.364106 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-f9d7485db-568dj"] Dec 03 11:20:52 crc kubenswrapper[5052]: I1203 11:20:52.387439 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\") pod \"8f668bae-612b-4b75-9490-919e737c6a3b\" (UID: \"8f668bae-612b-4b75-9490-919e737c6a3b\") " Dec 03 11:20:52 crc kubenswrapper[5052]: E1203 11:20:52.388549 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8 podName:8f668bae-612b-4b75-9490-919e737c6a3b nodeName:}" failed. No retries permitted until 2025-12-03 11:20:52.888503346 +0000 UTC m=+43.476745110 (durationBeforeRetry 500ms). Error: UnmountVolume.TearDown failed for volume "pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8" (UniqueName: "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8") pod "8f668bae-612b-4b75-9490-919e737c6a3b" (UID: "8f668bae-612b-4b75-9490-919e737c6a3b") : kubernetes.io/csi: Unmounter.TearDownAt failed to get CSI client: driver name kubevirt.io.hostpath-provisioner not found in the list of registered CSI drivers Dec 03 11:20:52 crc kubenswrapper[5052]: W1203 11:20:52.428544 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3afb93c1_9cbc_4588_a3bf_2ae1e774a2d1.slice/crio-17a34e820b3eb25dc8b4ce7590dfb86e7981efed8ab6ab3e3a8f17268d4db893 WatchSource:0}: Error finding container 17a34e820b3eb25dc8b4ce7590dfb86e7981efed8ab6ab3e3a8f17268d4db893: Status 404 returned error can't find the container with id 17a34e820b3eb25dc8b4ce7590dfb86e7981efed8ab6ab3e3a8f17268d4db893 Dec 03 11:20:52 crc kubenswrapper[5052]: I1203 11:20:52.431662 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-b67b599dd-5h545"] Dec 03 11:20:52 crc kubenswrapper[5052]: I1203 11:20:52.444432 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-operator-lifecycle-manager/package-server-manager-789f6589d5-577rn"] Dec 03 11:20:52 crc kubenswrapper[5052]: I1203 11:20:52.453988 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["hostpath-provisioner/csi-hostpathplugin-5tlrr"] Dec 03 11:20:52 crc kubenswrapper[5052]: I1203 11:20:52.470696 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-controller-manager-operator/openshift-controller-manager-operator-756b6f6bc6-v7lmf" event={"ID":"43fca84c-422e-4e3b-b3ca-b787a4e3f677","Type":"ContainerStarted","Data":"9d0b514c8e44f66fc435892bbeeb3fb65fcff32172a042350b9d8f5846c3dfc1"} Dec 03 11:20:52 crc kubenswrapper[5052]: W1203 11:20:52.484691 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode43f14fc_f3eb_4f58_b202_cdfc641ac08a.slice/crio-4c7da6eda6ba0ce1724c7cd12ef13c0bc571d89291e3b3beb10c29f534bad15c WatchSource:0}: Error finding container 4c7da6eda6ba0ce1724c7cd12ef13c0bc571d89291e3b3beb10c29f534bad15c: Status 404 returned error can't find the container with id 4c7da6eda6ba0ce1724c7cd12ef13c0bc571d89291e3b3beb10c29f534bad15c Dec 03 11:20:52 crc kubenswrapper[5052]: I1203 11:20:52.490261 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\") pod \"image-registry-697d97f7c8-p77bx\" (UID: \"f5012254-068c-4360-b310-39ff3d309199\") " pod="openshift-image-registry/image-registry-697d97f7c8-p77bx" Dec 03 11:20:52 crc kubenswrapper[5052]: E1203 11:20:52.490711 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8 podName: nodeName:}" failed. No retries permitted until 2025-12-03 11:20:52.990693003 +0000 UTC m=+43.578934767 (durationBeforeRetry 500ms). Error: MountVolume.MountDevice failed for volume "pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8" (UniqueName: "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8") pod "image-registry-697d97f7c8-p77bx" (UID: "f5012254-068c-4360-b310-39ff3d309199") : kubernetes.io/csi: attacher.MountDevice failed to create newCsiDriverClient: driver name kubevirt.io.hostpath-provisioner not found in the list of registered CSI drivers Dec 03 11:20:52 crc kubenswrapper[5052]: W1203 11:20:52.493490 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0bf52c7_bf23_4c65_b7cb_0d8d466aa8f2.slice/crio-9daff7089b44abb5c2c2b6c654f7aebe9ac3f395b6a5bdcd59e616c9d2b6f1fb WatchSource:0}: Error finding container 9daff7089b44abb5c2c2b6c654f7aebe9ac3f395b6a5bdcd59e616c9d2b6f1fb: Status 404 returned error can't find the container with id 9daff7089b44abb5c2c2b6c654f7aebe9ac3f395b6a5bdcd59e616c9d2b6f1fb Dec 03 11:20:52 crc kubenswrapper[5052]: I1203 11:20:52.500228 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/machine-config-operator-74547568cd-tr2xc" event={"ID":"cc99b3a0-fa04-4128-b730-1d24da63d08e","Type":"ContainerStarted","Data":"64d00ee78842b06ad70862c129edb7be2b98d0f8fb9f35bd6dd64d4dcd8b41c8"} Dec 03 11:20:52 crc kubenswrapper[5052]: I1203 11:20:52.526855 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-machine-config-operator/machine-config-operator-74547568cd-tr2xc" podStartSLOduration=20.526827678 podStartE2EDuration="20.526827678s" podCreationTimestamp="2025-12-03 11:20:32 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:20:52.524297999 +0000 UTC m=+43.112539763" watchObservedRunningTime="2025-12-03 11:20:52.526827678 +0000 UTC m=+43.115069442" Dec 03 11:20:52 crc kubenswrapper[5052]: I1203 11:20:52.551752 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-58897d9998-7vpns" event={"ID":"3afb93c1-9cbc-4588-a3bf-2ae1e774a2d1","Type":"ContainerStarted","Data":"17a34e820b3eb25dc8b4ce7590dfb86e7981efed8ab6ab3e3a8f17268d4db893"} Dec 03 11:20:52 crc kubenswrapper[5052]: W1203 11:20:52.563434 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3f2b85d_9b43_4c1c_bc3a_602405b96efc.slice/crio-69f36ab583623e3144a71b4eadc9ea854fc3e15d49e4498df773402a1a196571 WatchSource:0}: Error finding container 69f36ab583623e3144a71b4eadc9ea854fc3e15d49e4498df773402a1a196571: Status 404 returned error can't find the container with id 69f36ab583623e3144a71b4eadc9ea854fc3e15d49e4498df773402a1a196571 Dec 03 11:20:52 crc kubenswrapper[5052]: W1203 11:20:52.578718 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e31bff9_7ac9_47a7_a895_db2cc3051d85.slice/crio-be447860efa147cd171f704464b0733a3a443a6db7998c8b2cfb1e13553daf3d WatchSource:0}: Error finding container be447860efa147cd171f704464b0733a3a443a6db7998c8b2cfb1e13553daf3d: Status 404 returned error can't find the container with id be447860efa147cd171f704464b0733a3a443a6db7998c8b2cfb1e13553daf3d Dec 03 11:20:52 crc kubenswrapper[5052]: I1203 11:20:52.591603 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-59844c95c7-4qts8" event={"ID":"199a472c-800b-45e2-a507-7e6cb2c7427f","Type":"ContainerStarted","Data":"b8004470df05c0459b12ed0d128c95fd086944ce94a12263f27132518dd21bb4"} Dec 03 11:20:52 crc kubenswrapper[5052]: I1203 11:20:52.591648 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-59844c95c7-4qts8" event={"ID":"199a472c-800b-45e2-a507-7e6cb2c7427f","Type":"ContainerStarted","Data":"c78fbce086aaf8f95bb8d92835b2ba18d1b541a14a0c9d8461cd362818b380f0"} Dec 03 11:20:52 crc kubenswrapper[5052]: I1203 11:20:52.594834 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\") pod \"8f668bae-612b-4b75-9490-919e737c6a3b\" (UID: \"8f668bae-612b-4b75-9490-919e737c6a3b\") " Dec 03 11:20:52 crc kubenswrapper[5052]: E1203 11:20:52.596697 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8 podName:8f668bae-612b-4b75-9490-919e737c6a3b nodeName:}" failed. No retries permitted until 2025-12-03 11:20:53.096673413 +0000 UTC m=+43.684915177 (durationBeforeRetry 500ms). Error: UnmountVolume.TearDown failed for volume "pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8" (UniqueName: "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8") pod "8f668bae-612b-4b75-9490-919e737c6a3b" (UID: "8f668bae-612b-4b75-9490-919e737c6a3b") : kubernetes.io/csi: Unmounter.TearDownAt failed to get CSI client: driver name kubevirt.io.hostpath-provisioner not found in the list of registered CSI drivers Dec 03 11:20:52 crc kubenswrapper[5052]: I1203 11:20:52.617323 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-apiserver-operator/openshift-apiserver-operator-796bbdcf4f-jskzg" event={"ID":"42cd7f06-f712-41e0-9fa5-c8ec9f7af997","Type":"ContainerStarted","Data":"b3919e1210bcd9f746eea3aafcfd8f8f1da750ba2ad12dde596c86a45ca1d3d7"} Dec 03 11:20:52 crc kubenswrapper[5052]: I1203 11:20:52.631162 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-59844c95c7-4qts8" podStartSLOduration=20.631141553 podStartE2EDuration="20.631141553s" podCreationTimestamp="2025-12-03 11:20:32 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:20:52.629163499 +0000 UTC m=+43.217405263" watchObservedRunningTime="2025-12-03 11:20:52.631141553 +0000 UTC m=+43.219383317" Dec 03 11:20:52 crc kubenswrapper[5052]: I1203 11:20:52.633145 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-etcd-operator/etcd-operator-b45778765-z2lp6" event={"ID":"71ea0624-9fb5-415a-96e1-d5ada83c0fe4","Type":"ContainerStarted","Data":"d250bf02f8cfb9dd1ecb340aed486ae62450509a62d16b4c9b92c98f1d68409a"} Dec 03 11:20:52 crc kubenswrapper[5052]: I1203 11:20:52.659644 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/machine-config-controller-84d6567774-rxwdl" event={"ID":"efa98345-cafa-483a-976b-d772a3bd94e7","Type":"ContainerStarted","Data":"1aa2215a8cb7619207c76eee2492be981edaa3b457b4f254f98b5b2c4b601878"} Dec 03 11:20:52 crc kubenswrapper[5052]: I1203 11:20:52.660882 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/machine-config-controller-84d6567774-rxwdl" event={"ID":"efa98345-cafa-483a-976b-d772a3bd94e7","Type":"ContainerStarted","Data":"a50a2c36fc46e62b364dd61254d3a0d0f1b65c42cd61fbb746b2bc05a961dbd2"} Dec 03 11:20:52 crc kubenswrapper[5052]: I1203 11:20:52.680573 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-oauth-apiserver/apiserver-7bbb656c7d-86vx8" event={"ID":"8fae1d72-fba4-4999-bea5-39d6472d7ea7","Type":"ContainerStarted","Data":"7e64fdfb090e045b541b497795caeaaa0e2a44869d087e8261fc27946eb969f3"} Dec 03 11:20:52 crc kubenswrapper[5052]: I1203 11:20:52.685718 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-operator/ingress-operator-5b745b69d9-hc8vv" event={"ID":"833bb55c-5c9b-450c-9287-30f146a2d16a","Type":"ContainerStarted","Data":"1ac97f7be8053d6ef207f12eb89e77036360752d93fe2678ae81766230152512"} Dec 03 11:20:52 crc kubenswrapper[5052]: I1203 11:20:52.685794 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-operator/ingress-operator-5b745b69d9-hc8vv" event={"ID":"833bb55c-5c9b-450c-9287-30f146a2d16a","Type":"ContainerStarted","Data":"ffe64175a47261e1f141c78077d2931d610d2f980a0cda668f43a3779b1b1745"} Dec 03 11:20:52 crc kubenswrapper[5052]: I1203 11:20:52.691007 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-kube-controller-manager-operator/kube-controller-manager-operator-78b949d7b-h6b78" event={"ID":"3dd904b5-266d-4a49-ad58-805139eef151","Type":"ContainerStarted","Data":"9b3eb1d728688a443a626578b57c62ca644f3c148ce9bd88f3f01b37d0366fec"} Dec 03 11:20:52 crc kubenswrapper[5052]: I1203 11:20:52.691683 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-machine-config-operator/machine-config-controller-84d6567774-rxwdl" podStartSLOduration=20.691658781 podStartE2EDuration="20.691658781s" podCreationTimestamp="2025-12-03 11:20:32 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:20:52.690653983 +0000 UTC m=+43.278895767" watchObservedRunningTime="2025-12-03 11:20:52.691658781 +0000 UTC m=+43.279900545" Dec 03 11:20:52 crc kubenswrapper[5052]: I1203 11:20:52.697805 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\") pod \"image-registry-697d97f7c8-p77bx\" (UID: \"f5012254-068c-4360-b310-39ff3d309199\") " pod="openshift-image-registry/image-registry-697d97f7c8-p77bx" Dec 03 11:20:52 crc kubenswrapper[5052]: E1203 11:20:52.698286 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8 podName: nodeName:}" failed. No retries permitted until 2025-12-03 11:20:53.198267853 +0000 UTC m=+43.786509617 (durationBeforeRetry 500ms). Error: MountVolume.MountDevice failed for volume "pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8" (UniqueName: "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8") pod "image-registry-697d97f7c8-p77bx" (UID: "f5012254-068c-4360-b310-39ff3d309199") : kubernetes.io/csi: attacher.MountDevice failed to create newCsiDriverClient: driver name kubevirt.io.hostpath-provisioner not found in the list of registered CSI drivers Dec 03 11:20:52 crc kubenswrapper[5052]: I1203 11:20:52.722944 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-multus/cni-sysctl-allowlist-ds-8l9gm" event={"ID":"bf761cb9-e024-4acd-a947-0f8064267243","Type":"ContainerStarted","Data":"b0103da56f9128a13b28c52c5f8402db976339938594d6eae2a65c5f008b27ef"} Dec 03 11:20:52 crc kubenswrapper[5052]: I1203 11:20:52.723737 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openshift-multus/cni-sysctl-allowlist-ds-8l9gm" Dec 03 11:20:52 crc kubenswrapper[5052]: I1203 11:20:52.757176 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-oauth-apiserver/apiserver-7bbb656c7d-86vx8" Dec 03 11:20:52 crc kubenswrapper[5052]: I1203 11:20:52.757286 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openshift-oauth-apiserver/apiserver-7bbb656c7d-86vx8" Dec 03 11:20:52 crc kubenswrapper[5052]: I1203 11:20:52.758713 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-kube-apiserver-operator/kube-apiserver-operator-766d6c64bb-xwlk8" event={"ID":"287ad4d4-bf40-4cc4-9856-580b65a2e88d","Type":"ContainerStarted","Data":"ed7a21898c079b9dcff054477b658df5137432c447944599d21e1a812a670b9b"} Dec 03 11:20:52 crc kubenswrapper[5052]: I1203 11:20:52.767543 5052 patch_prober.go:28] interesting pod/apiserver-7bbb656c7d-86vx8 container/oauth-apiserver namespace/openshift-oauth-apiserver: Startup probe status=failure output="Get \"https://10.217.0.6:8443/livez\": dial tcp 10.217.0.6:8443: connect: connection refused" start-of-body= Dec 03 11:20:52 crc kubenswrapper[5052]: I1203 11:20:52.767615 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openshift-oauth-apiserver/apiserver-7bbb656c7d-86vx8" podUID="8fae1d72-fba4-4999-bea5-39d6472d7ea7" containerName="oauth-apiserver" probeResult="failure" output="Get \"https://10.217.0.6:8443/livez\": dial tcp 10.217.0.6:8443: connect: connection refused" Dec 03 11:20:52 crc kubenswrapper[5052]: I1203 11:20:52.775501 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-operator-lifecycle-manager/olm-operator-6b444d44fb-6lm8x" event={"ID":"663b7426-fde0-4b59-b857-b434b4a88f4b","Type":"ContainerStarted","Data":"461981054a84cbd921113edbdebea82b159638a6688cee69e652d68e51ab5edb"} Dec 03 11:20:52 crc kubenswrapper[5052]: I1203 11:20:52.777082 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openshift-operator-lifecycle-manager/olm-operator-6b444d44fb-6lm8x" Dec 03 11:20:52 crc kubenswrapper[5052]: I1203 11:20:52.783110 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-multus/cni-sysctl-allowlist-ds-8l9gm" Dec 03 11:20:52 crc kubenswrapper[5052]: I1203 11:20:52.783321 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-oauth-apiserver/apiserver-7bbb656c7d-86vx8" podStartSLOduration=20.783279426 podStartE2EDuration="20.783279426s" podCreationTimestamp="2025-12-03 11:20:32 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:20:52.779910643 +0000 UTC m=+43.368152407" watchObservedRunningTime="2025-12-03 11:20:52.783279426 +0000 UTC m=+43.371521210" Dec 03 11:20:52 crc kubenswrapper[5052]: I1203 11:20:52.783677 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress-operator/ingress-operator-5b745b69d9-hc8vv" podStartSLOduration=20.783670727 podStartE2EDuration="20.783670727s" podCreationTimestamp="2025-12-03 11:20:32 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:20:52.746661867 +0000 UTC m=+43.334903641" watchObservedRunningTime="2025-12-03 11:20:52.783670727 +0000 UTC m=+43.371912491" Dec 03 11:20:52 crc kubenswrapper[5052]: I1203 11:20:52.785809 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-machine-api/control-plane-machine-set-operator-78cbb6b69f-p4b4l" event={"ID":"693b46e6-42c3-4d07-bfbd-bd83d6982199","Type":"ContainerStarted","Data":"d281a41994089b5554d92957d363f5de49369c5c7288e72d9e49d28ccea23568"} Dec 03 11:20:52 crc kubenswrapper[5052]: I1203 11:20:52.785879 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-machine-api/control-plane-machine-set-operator-78cbb6b69f-p4b4l" event={"ID":"693b46e6-42c3-4d07-bfbd-bd83d6982199","Type":"ContainerStarted","Data":"fc9c83f999a3873ed92ffc535634e53dc8d11e5a27a92f2bf69a8f3a57f27a11"} Dec 03 11:20:52 crc kubenswrapper[5052]: I1203 11:20:52.794064 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-admission-controller-857f4d67dd-q8n58" event={"ID":"daf5492e-2286-42c8-8c08-88200b27e2fb","Type":"ContainerStarted","Data":"507933fcc27ef72ff0424ce01ee08bd946c52de5d2bc9d1563070330f2a51195"} Dec 03 11:20:52 crc kubenswrapper[5052]: I1203 11:20:52.798738 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\") pod \"8f668bae-612b-4b75-9490-919e737c6a3b\" (UID: \"8f668bae-612b-4b75-9490-919e737c6a3b\") " Dec 03 11:20:52 crc kubenswrapper[5052]: E1203 11:20:52.799757 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8 podName:8f668bae-612b-4b75-9490-919e737c6a3b nodeName:}" failed. No retries permitted until 2025-12-03 11:20:53.299738609 +0000 UTC m=+43.887980373 (durationBeforeRetry 500ms). Error: UnmountVolume.TearDown failed for volume "pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8" (UniqueName: "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8") pod "8f668bae-612b-4b75-9490-919e737c6a3b" (UID: "8f668bae-612b-4b75-9490-919e737c6a3b") : kubernetes.io/csi: Unmounter.TearDownAt failed to get CSI client: driver name kubevirt.io.hostpath-provisioner not found in the list of registered CSI drivers Dec 03 11:20:52 crc kubenswrapper[5052]: I1203 11:20:52.809957 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-operator-lifecycle-manager/olm-operator-6b444d44fb-6lm8x" Dec 03 11:20:52 crc kubenswrapper[5052]: I1203 11:20:52.810757 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-operator-lifecycle-manager/olm-operator-6b444d44fb-6lm8x" podStartSLOduration=20.810730142 podStartE2EDuration="20.810730142s" podCreationTimestamp="2025-12-03 11:20:32 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:20:52.810193227 +0000 UTC m=+43.398434991" watchObservedRunningTime="2025-12-03 11:20:52.810730142 +0000 UTC m=+43.398971906" Dec 03 11:20:52 crc kubenswrapper[5052]: I1203 11:20:52.811445 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-665b6dd947-7wkt6" event={"ID":"59e1467d-d303-4f32-a968-b66eb94953af","Type":"ContainerStarted","Data":"ee5640a471adaaaefd208ad6ba01777e3e520f984cf69e6053e7f39c37ab488d"} Dec 03 11:20:52 crc kubenswrapper[5052]: I1203 11:20:52.815756 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-dns-operator/dns-operator-744455d44c-zvvfc" event={"ID":"cabc2063-04d9-4491-be9f-f7781b5f02cf","Type":"ContainerStarted","Data":"b967b7bb788ffe46e2d0cdc0210115b83b94a23bc4f810bc8baaf14ce3a452ea"} Dec 03 11:20:52 crc kubenswrapper[5052]: I1203 11:20:52.819358 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-config-operator/openshift-config-operator-7777fb866f-bkztv" event={"ID":"0a110335-c2e5-40a2-b565-a4791146d53b","Type":"ContainerStarted","Data":"5cfc098995ed3eab105030845dafc82ac0782e8a25572b88ec76854e98a928db"} Dec 03 11:20:52 crc kubenswrapper[5052]: I1203 11:20:52.841250 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/cni-sysctl-allowlist-ds-8l9gm" podStartSLOduration=7.841223873 podStartE2EDuration="7.841223873s" podCreationTimestamp="2025-12-03 11:20:45 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:20:52.839076183 +0000 UTC m=+43.427317947" watchObservedRunningTime="2025-12-03 11:20:52.841223873 +0000 UTC m=+43.429465637" Dec 03 11:20:52 crc kubenswrapper[5052]: I1203 11:20:52.842172 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/machine-config-server-kzd49" event={"ID":"c3a89abc-ba9f-4c13-a73e-2b9e66e9345a","Type":"ContainerStarted","Data":"91161cca30034bfe267a97e536066efa4297a0aca067f7306ec4c7643e3bc404"} Dec 03 11:20:52 crc kubenswrapper[5052]: I1203 11:20:52.846170 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-controller-manager/controller-manager-879f6c89f-46rm8" event={"ID":"1c570f3f-3090-48e1-9b7e-f66c99310dbd","Type":"ContainerStarted","Data":"13df0bb2759655c12e4f73784433bcd7dc9c684b4ff40da2bbafe3119c9eb255"} Dec 03 11:20:52 crc kubenswrapper[5052]: I1203 11:20:52.849160 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-777779d784-9jvh6" event={"ID":"3e0657c6-3f3c-4300-8cfb-b3db773dba75","Type":"ContainerStarted","Data":"76cd17f055a2d819a2006f39b12f4e1af3fd73831fdbc6c06e0abd1fa48b25f7"} Dec 03 11:20:52 crc kubenswrapper[5052]: I1203 11:20:52.859971 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-operator-lifecycle-manager/packageserver-d55dfcdfc-7s97q" event={"ID":"6413eb91-7b61-4660-888c-14da224c7dfa","Type":"ContainerStarted","Data":"e2e2448904e9f22dd904993e750117c6ced295bbf54bae71334c8efee36557af"} Dec 03 11:20:52 crc kubenswrapper[5052]: I1203 11:20:52.866437 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openshift-operator-lifecycle-manager/packageserver-d55dfcdfc-7s97q" Dec 03 11:20:52 crc kubenswrapper[5052]: I1203 11:20:52.875439 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-operator-lifecycle-manager/catalog-operator-68c6474976-rqhkg" event={"ID":"8e3a111c-05ff-4f33-b5ba-81dae833b529","Type":"ContainerStarted","Data":"12cd205661baa25b45dd7985ef3f39204e6ce820d45b562b480f47cec8716c99"} Dec 03 11:20:52 crc kubenswrapper[5052]: I1203 11:20:52.895297 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-9c57cc56f-w7g95" event={"ID":"eb07cd6b-71c7-4dc7-b658-40f32c63f373","Type":"ContainerStarted","Data":"3ae4671a1f4cf40ba632d8ff74f88d60250227e65d0c2fdc03eec21fe433d583"} Dec 03 11:20:52 crc kubenswrapper[5052]: I1203 11:20:52.901478 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\") pod \"image-registry-697d97f7c8-p77bx\" (UID: \"f5012254-068c-4360-b310-39ff3d309199\") " pod="openshift-image-registry/image-registry-697d97f7c8-p77bx" Dec 03 11:20:52 crc kubenswrapper[5052]: E1203 11:20:52.904143 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8 podName: nodeName:}" failed. No retries permitted until 2025-12-03 11:20:53.404122286 +0000 UTC m=+43.992364120 (durationBeforeRetry 500ms). Error: MountVolume.MountDevice failed for volume "pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8" (UniqueName: "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8") pod "image-registry-697d97f7c8-p77bx" (UID: "f5012254-068c-4360-b310-39ff3d309199") : kubernetes.io/csi: attacher.MountDevice failed to create newCsiDriverClient: driver name kubevirt.io.hostpath-provisioner not found in the list of registered CSI drivers Dec 03 11:20:52 crc kubenswrapper[5052]: I1203 11:20:52.940869 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns-operator/dns-operator-744455d44c-zvvfc" podStartSLOduration=21.940846188 podStartE2EDuration="21.940846188s" podCreationTimestamp="2025-12-03 11:20:31 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:20:52.940146289 +0000 UTC m=+43.528388053" watchObservedRunningTime="2025-12-03 11:20:52.940846188 +0000 UTC m=+43.529087952" Dec 03 11:20:52 crc kubenswrapper[5052]: I1203 11:20:52.967723 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-machine-approver/machine-approver-56656f9798-6t9dp" event={"ID":"8c1b25f5-2d98-4852-bc7a-d5cbfa5cb6aa","Type":"ContainerStarted","Data":"b3d4acf97370edf296939d110f26e460a341ac0cd5274c197d14c312bb94e25f"} Dec 03 11:20:53 crc kubenswrapper[5052]: I1203 11:20:53.003236 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\") pod \"8f668bae-612b-4b75-9490-919e737c6a3b\" (UID: \"8f668bae-612b-4b75-9490-919e737c6a3b\") " Dec 03 11:20:53 crc kubenswrapper[5052]: E1203 11:20:53.006324 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8 podName:8f668bae-612b-4b75-9490-919e737c6a3b nodeName:}" failed. No retries permitted until 2025-12-03 11:20:53.506277411 +0000 UTC m=+44.094519175 (durationBeforeRetry 500ms). Error: UnmountVolume.TearDown failed for volume "pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8" (UniqueName: "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8") pod "8f668bae-612b-4b75-9490-919e737c6a3b" (UID: "8f668bae-612b-4b75-9490-919e737c6a3b") : kubernetes.io/csi: Unmounter.TearDownAt failed to get CSI client: driver name kubevirt.io.hostpath-provisioner not found in the list of registered CSI drivers Dec 03 11:20:53 crc kubenswrapper[5052]: I1203 11:20:53.045016 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-authentication/oauth-openshift-558db77b4-99jsx" event={"ID":"75f62784-7818-4a74-9c8c-4777d6db0611","Type":"ContainerStarted","Data":"4670adfe678989d2721baf3507dccbda083f63c965b655522db5099c5d66de54"} Dec 03 11:20:53 crc kubenswrapper[5052]: I1203 11:20:53.045067 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openshift-authentication/oauth-openshift-558db77b4-99jsx" Dec 03 11:20:53 crc kubenswrapper[5052]: I1203 11:20:53.049103 5052 patch_prober.go:28] interesting pod/downloads-7954f5f757-l7xtl container/download-server namespace/openshift-console: Readiness probe status=failure output="Get \"http://10.217.0.18:8080/\": dial tcp 10.217.0.18:8080: connect: connection refused" start-of-body= Dec 03 11:20:53 crc kubenswrapper[5052]: I1203 11:20:53.049191 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openshift-console/downloads-7954f5f757-l7xtl" podUID="ec2ea657-018b-4626-bf79-5c818f573a4b" containerName="download-server" probeResult="failure" output="Get \"http://10.217.0.18:8080/\": dial tcp 10.217.0.18:8080: connect: connection refused" Dec 03 11:20:53 crc kubenswrapper[5052]: I1203 11:20:53.049103 5052 patch_prober.go:28] interesting pod/marketplace-operator-79b997595-kdr8l container/marketplace-operator namespace/openshift-marketplace: Readiness probe status=failure output="Get \"http://10.217.0.31:8080/healthz\": dial tcp 10.217.0.31:8080: connect: connection refused" start-of-body= Dec 03 11:20:53 crc kubenswrapper[5052]: I1203 11:20:53.049271 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openshift-marketplace/marketplace-operator-79b997595-kdr8l" podUID="cac70dab-6e13-4693-ac66-c7552b62dfd7" containerName="marketplace-operator" probeResult="failure" output="Get \"http://10.217.0.31:8080/healthz\": dial tcp 10.217.0.31:8080: connect: connection refused" Dec 03 11:20:53 crc kubenswrapper[5052]: I1203 11:20:53.072775 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-machine-api/control-plane-machine-set-operator-78cbb6b69f-p4b4l" podStartSLOduration=21.072754363 podStartE2EDuration="21.072754363s" podCreationTimestamp="2025-12-03 11:20:32 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:20:53.022294893 +0000 UTC m=+43.610536657" watchObservedRunningTime="2025-12-03 11:20:53.072754363 +0000 UTC m=+43.660996127" Dec 03 11:20:53 crc kubenswrapper[5052]: I1203 11:20:53.122907 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\") pod \"image-registry-697d97f7c8-p77bx\" (UID: \"f5012254-068c-4360-b310-39ff3d309199\") " pod="openshift-image-registry/image-registry-697d97f7c8-p77bx" Dec 03 11:20:53 crc kubenswrapper[5052]: E1203 11:20:53.129806 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8 podName: nodeName:}" failed. No retries permitted until 2025-12-03 11:20:53.629783815 +0000 UTC m=+44.218025769 (durationBeforeRetry 500ms). Error: MountVolume.MountDevice failed for volume "pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8" (UniqueName: "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8") pod "image-registry-697d97f7c8-p77bx" (UID: "f5012254-068c-4360-b310-39ff3d309199") : kubernetes.io/csi: attacher.MountDevice failed to create newCsiDriverClient: driver name kubevirt.io.hostpath-provisioner not found in the list of registered CSI drivers Dec 03 11:20:53 crc kubenswrapper[5052]: I1203 11:20:53.138370 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-operator-lifecycle-manager/packageserver-d55dfcdfc-7s97q" Dec 03 11:20:53 crc kubenswrapper[5052]: I1203 11:20:53.156708 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-machine-config-operator/machine-config-server-kzd49" podStartSLOduration=8.156685196 podStartE2EDuration="8.156685196s" podCreationTimestamp="2025-12-03 11:20:45 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:20:53.156503001 +0000 UTC m=+43.744744765" watchObservedRunningTime="2025-12-03 11:20:53.156685196 +0000 UTC m=+43.744926960" Dec 03 11:20:53 crc kubenswrapper[5052]: I1203 11:20:53.157789 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-665b6dd947-7wkt6" podStartSLOduration=22.157778987 podStartE2EDuration="22.157778987s" podCreationTimestamp="2025-12-03 11:20:31 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:20:53.080071605 +0000 UTC m=+43.668313369" watchObservedRunningTime="2025-12-03 11:20:53.157778987 +0000 UTC m=+43.746020741" Dec 03 11:20:53 crc kubenswrapper[5052]: I1203 11:20:53.224326 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\") pod \"8f668bae-612b-4b75-9490-919e737c6a3b\" (UID: \"8f668bae-612b-4b75-9490-919e737c6a3b\") " Dec 03 11:20:53 crc kubenswrapper[5052]: E1203 11:20:53.224844 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8 podName:8f668bae-612b-4b75-9490-919e737c6a3b nodeName:}" failed. No retries permitted until 2025-12-03 11:20:53.724821174 +0000 UTC m=+44.313062938 (durationBeforeRetry 500ms). Error: UnmountVolume.TearDown failed for volume "pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8" (UniqueName: "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8") pod "8f668bae-612b-4b75-9490-919e737c6a3b" (UID: "8f668bae-612b-4b75-9490-919e737c6a3b") : kubernetes.io/csi: Unmounter.TearDownAt failed to get CSI client: driver name kubevirt.io.hostpath-provisioner not found in the list of registered CSI drivers Dec 03 11:20:53 crc kubenswrapper[5052]: I1203 11:20:53.227194 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-machine-approver/machine-approver-56656f9798-6t9dp" podStartSLOduration=23.227170109 podStartE2EDuration="23.227170109s" podCreationTimestamp="2025-12-03 11:20:30 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:20:53.18984416 +0000 UTC m=+43.778085924" watchObservedRunningTime="2025-12-03 11:20:53.227170109 +0000 UTC m=+43.815411873" Dec 03 11:20:53 crc kubenswrapper[5052]: I1203 11:20:53.231894 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openshift-multus/cni-sysctl-allowlist-ds-8l9gm"] Dec 03 11:20:53 crc kubenswrapper[5052]: I1203 11:20:53.238484 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-authentication/oauth-openshift-558db77b4-99jsx" Dec 03 11:20:53 crc kubenswrapper[5052]: I1203 11:20:53.242803 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-777779d784-9jvh6" podStartSLOduration=21.242777199 podStartE2EDuration="21.242777199s" podCreationTimestamp="2025-12-03 11:20:32 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:20:53.238546082 +0000 UTC m=+43.826787856" watchObservedRunningTime="2025-12-03 11:20:53.242777199 +0000 UTC m=+43.831018963" Dec 03 11:20:53 crc kubenswrapper[5052]: I1203 11:20:53.256411 5052 patch_prober.go:28] interesting pod/router-default-5444994796-rtl8t container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-http failed: reason withheld Dec 03 11:20:53 crc kubenswrapper[5052]: [-]has-synced failed: reason withheld Dec 03 11:20:53 crc kubenswrapper[5052]: [+]process-running ok Dec 03 11:20:53 crc kubenswrapper[5052]: healthz check failed Dec 03 11:20:53 crc kubenswrapper[5052]: I1203 11:20:53.256583 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-5444994796-rtl8t" podUID="48040744-7fc1-482e-9b68-3a904ba03a8d" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500" Dec 03 11:20:53 crc kubenswrapper[5052]: I1203 11:20:53.319276 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-operator-lifecycle-manager/packageserver-d55dfcdfc-7s97q" podStartSLOduration=21.319235416 podStartE2EDuration="21.319235416s" podCreationTimestamp="2025-12-03 11:20:32 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:20:53.278538194 +0000 UTC m=+43.866779958" watchObservedRunningTime="2025-12-03 11:20:53.319235416 +0000 UTC m=+43.907477180" Dec 03 11:20:53 crc kubenswrapper[5052]: I1203 11:20:53.319772 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-authentication/oauth-openshift-558db77b4-99jsx" podStartSLOduration=22.319765211 podStartE2EDuration="22.319765211s" podCreationTimestamp="2025-12-03 11:20:31 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:20:53.316542402 +0000 UTC m=+43.904784186" watchObservedRunningTime="2025-12-03 11:20:53.319765211 +0000 UTC m=+43.908006975" Dec 03 11:20:53 crc kubenswrapper[5052]: I1203 11:20:53.325981 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\") pod \"image-registry-697d97f7c8-p77bx\" (UID: \"f5012254-068c-4360-b310-39ff3d309199\") " pod="openshift-image-registry/image-registry-697d97f7c8-p77bx" Dec 03 11:20:53 crc kubenswrapper[5052]: E1203 11:20:53.326372 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8 podName: nodeName:}" failed. No retries permitted until 2025-12-03 11:20:53.826356212 +0000 UTC m=+44.414597976 (durationBeforeRetry 500ms). Error: MountVolume.MountDevice failed for volume "pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8" (UniqueName: "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8") pod "image-registry-697d97f7c8-p77bx" (UID: "f5012254-068c-4360-b310-39ff3d309199") : kubernetes.io/csi: attacher.MountDevice failed to create newCsiDriverClient: driver name kubevirt.io.hostpath-provisioner not found in the list of registered CSI drivers Dec 03 11:20:53 crc kubenswrapper[5052]: I1203 11:20:53.431025 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\") pod \"8f668bae-612b-4b75-9490-919e737c6a3b\" (UID: \"8f668bae-612b-4b75-9490-919e737c6a3b\") " Dec 03 11:20:53 crc kubenswrapper[5052]: E1203 11:20:53.431780 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8 podName:8f668bae-612b-4b75-9490-919e737c6a3b nodeName:}" failed. No retries permitted until 2025-12-03 11:20:53.931762797 +0000 UTC m=+44.520004561 (durationBeforeRetry 500ms). Error: UnmountVolume.TearDown failed for volume "pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8" (UniqueName: "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8") pod "8f668bae-612b-4b75-9490-919e737c6a3b" (UID: "8f668bae-612b-4b75-9490-919e737c6a3b") : kubernetes.io/csi: Unmounter.TearDownAt failed to get CSI client: driver name kubevirt.io.hostpath-provisioner not found in the list of registered CSI drivers Dec 03 11:20:53 crc kubenswrapper[5052]: I1203 11:20:53.539433 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\") pod \"image-registry-697d97f7c8-p77bx\" (UID: \"f5012254-068c-4360-b310-39ff3d309199\") " pod="openshift-image-registry/image-registry-697d97f7c8-p77bx" Dec 03 11:20:53 crc kubenswrapper[5052]: E1203 11:20:53.556784 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8 podName: nodeName:}" failed. No retries permitted until 2025-12-03 11:20:54.056750872 +0000 UTC m=+44.644992636 (durationBeforeRetry 500ms). Error: MountVolume.MountDevice failed for volume "pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8" (UniqueName: "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8") pod "image-registry-697d97f7c8-p77bx" (UID: "f5012254-068c-4360-b310-39ff3d309199") : kubernetes.io/csi: attacher.MountDevice failed to create newCsiDriverClient: driver name kubevirt.io.hostpath-provisioner not found in the list of registered CSI drivers Dec 03 11:20:53 crc kubenswrapper[5052]: I1203 11:20:53.640256 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\") pod \"8f668bae-612b-4b75-9490-919e737c6a3b\" (UID: \"8f668bae-612b-4b75-9490-919e737c6a3b\") " Dec 03 11:20:53 crc kubenswrapper[5052]: E1203 11:20:53.640406 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8 podName:8f668bae-612b-4b75-9490-919e737c6a3b nodeName:}" failed. No retries permitted until 2025-12-03 11:20:54.140383786 +0000 UTC m=+44.728625560 (durationBeforeRetry 500ms). Error: UnmountVolume.TearDown failed for volume "pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8" (UniqueName: "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8") pod "8f668bae-612b-4b75-9490-919e737c6a3b" (UID: "8f668bae-612b-4b75-9490-919e737c6a3b") : kubernetes.io/csi: Unmounter.TearDownAt failed to get CSI client: driver name kubevirt.io.hostpath-provisioner not found in the list of registered CSI drivers Dec 03 11:20:53 crc kubenswrapper[5052]: I1203 11:20:53.640563 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\") pod \"image-registry-697d97f7c8-p77bx\" (UID: \"f5012254-068c-4360-b310-39ff3d309199\") " pod="openshift-image-registry/image-registry-697d97f7c8-p77bx" Dec 03 11:20:53 crc kubenswrapper[5052]: E1203 11:20:53.640959 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8 podName: nodeName:}" failed. No retries permitted until 2025-12-03 11:20:54.140949922 +0000 UTC m=+44.729191686 (durationBeforeRetry 500ms). Error: MountVolume.MountDevice failed for volume "pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8" (UniqueName: "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8") pod "image-registry-697d97f7c8-p77bx" (UID: "f5012254-068c-4360-b310-39ff3d309199") : kubernetes.io/csi: attacher.MountDevice failed to create newCsiDriverClient: driver name kubevirt.io.hostpath-provisioner not found in the list of registered CSI drivers Dec 03 11:20:53 crc kubenswrapper[5052]: I1203 11:20:53.742598 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\") pod \"8f668bae-612b-4b75-9490-919e737c6a3b\" (UID: \"8f668bae-612b-4b75-9490-919e737c6a3b\") " Dec 03 11:20:53 crc kubenswrapper[5052]: E1203 11:20:53.743291 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8 podName:8f668bae-612b-4b75-9490-919e737c6a3b nodeName:}" failed. No retries permitted until 2025-12-03 11:20:54.243274272 +0000 UTC m=+44.831516036 (durationBeforeRetry 500ms). Error: UnmountVolume.TearDown failed for volume "pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8" (UniqueName: "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8") pod "8f668bae-612b-4b75-9490-919e737c6a3b" (UID: "8f668bae-612b-4b75-9490-919e737c6a3b") : kubernetes.io/csi: Unmounter.TearDownAt failed to get CSI client: driver name kubevirt.io.hostpath-provisioner not found in the list of registered CSI drivers Dec 03 11:20:53 crc kubenswrapper[5052]: I1203 11:20:53.844368 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\") pod \"image-registry-697d97f7c8-p77bx\" (UID: \"f5012254-068c-4360-b310-39ff3d309199\") " pod="openshift-image-registry/image-registry-697d97f7c8-p77bx" Dec 03 11:20:53 crc kubenswrapper[5052]: E1203 11:20:53.844763 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8 podName: nodeName:}" failed. No retries permitted until 2025-12-03 11:20:54.344744708 +0000 UTC m=+44.932986472 (durationBeforeRetry 500ms). Error: MountVolume.MountDevice failed for volume "pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8" (UniqueName: "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8") pod "image-registry-697d97f7c8-p77bx" (UID: "f5012254-068c-4360-b310-39ff3d309199") : kubernetes.io/csi: attacher.MountDevice failed to create newCsiDriverClient: driver name kubevirt.io.hostpath-provisioner not found in the list of registered CSI drivers Dec 03 11:20:53 crc kubenswrapper[5052]: I1203 11:20:53.945798 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\") pod \"8f668bae-612b-4b75-9490-919e737c6a3b\" (UID: \"8f668bae-612b-4b75-9490-919e737c6a3b\") " Dec 03 11:20:53 crc kubenswrapper[5052]: E1203 11:20:53.945963 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8 podName:8f668bae-612b-4b75-9490-919e737c6a3b nodeName:}" failed. No retries permitted until 2025-12-03 11:20:54.445936007 +0000 UTC m=+45.034177771 (durationBeforeRetry 500ms). Error: UnmountVolume.TearDown failed for volume "pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8" (UniqueName: "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8") pod "8f668bae-612b-4b75-9490-919e737c6a3b" (UID: "8f668bae-612b-4b75-9490-919e737c6a3b") : kubernetes.io/csi: Unmounter.TearDownAt failed to get CSI client: driver name kubevirt.io.hostpath-provisioner not found in the list of registered CSI drivers Dec 03 11:20:53 crc kubenswrapper[5052]: I1203 11:20:53.946058 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\") pod \"image-registry-697d97f7c8-p77bx\" (UID: \"f5012254-068c-4360-b310-39ff3d309199\") " pod="openshift-image-registry/image-registry-697d97f7c8-p77bx" Dec 03 11:20:53 crc kubenswrapper[5052]: E1203 11:20:53.948162 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8 podName: nodeName:}" failed. No retries permitted until 2025-12-03 11:20:54.448121357 +0000 UTC m=+45.036363121 (durationBeforeRetry 500ms). Error: MountVolume.MountDevice failed for volume "pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8" (UniqueName: "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8") pod "image-registry-697d97f7c8-p77bx" (UID: "f5012254-068c-4360-b310-39ff3d309199") : kubernetes.io/csi: attacher.MountDevice failed to create newCsiDriverClient: driver name kubevirt.io.hostpath-provisioner not found in the list of registered CSI drivers Dec 03 11:20:54 crc kubenswrapper[5052]: I1203 11:20:54.047133 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\") pod \"8f668bae-612b-4b75-9490-919e737c6a3b\" (UID: \"8f668bae-612b-4b75-9490-919e737c6a3b\") " Dec 03 11:20:54 crc kubenswrapper[5052]: E1203 11:20:54.047274 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8 podName:8f668bae-612b-4b75-9490-919e737c6a3b nodeName:}" failed. No retries permitted until 2025-12-03 11:20:54.547246509 +0000 UTC m=+45.135488273 (durationBeforeRetry 500ms). Error: UnmountVolume.TearDown failed for volume "pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8" (UniqueName: "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8") pod "8f668bae-612b-4b75-9490-919e737c6a3b" (UID: "8f668bae-612b-4b75-9490-919e737c6a3b") : kubernetes.io/csi: Unmounter.TearDownAt failed to get CSI client: driver name kubevirt.io.hostpath-provisioner not found in the list of registered CSI drivers Dec 03 11:20:54 crc kubenswrapper[5052]: I1203 11:20:54.047782 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\") pod \"image-registry-697d97f7c8-p77bx\" (UID: \"f5012254-068c-4360-b310-39ff3d309199\") " pod="openshift-image-registry/image-registry-697d97f7c8-p77bx" Dec 03 11:20:54 crc kubenswrapper[5052]: E1203 11:20:54.048140 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8 podName: nodeName:}" failed. No retries permitted until 2025-12-03 11:20:54.548126534 +0000 UTC m=+45.136368298 (durationBeforeRetry 500ms). Error: MountVolume.MountDevice failed for volume "pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8" (UniqueName: "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8") pod "image-registry-697d97f7c8-p77bx" (UID: "f5012254-068c-4360-b310-39ff3d309199") : kubernetes.io/csi: attacher.MountDevice failed to create newCsiDriverClient: driver name kubevirt.io.hostpath-provisioner not found in the list of registered CSI drivers Dec 03 11:20:54 crc kubenswrapper[5052]: I1203 11:20:54.105471 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-operator-lifecycle-manager/package-server-manager-789f6589d5-577rn" event={"ID":"3f6f3c17-944e-4afb-8cd7-3ca95f787057","Type":"ContainerStarted","Data":"0fd84a32e3b43ff083ea1e2ad1026219a503ccdf43d6a356caba18c4954f219d"} Dec 03 11:20:54 crc kubenswrapper[5052]: I1203 11:20:54.142393 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-tvr2s" event={"ID":"ed6dae78-3358-4be5-84c2-142e4d45fe9a","Type":"ContainerStarted","Data":"97ffaa14d95a591a13ccf3a203f4c73afe822b01d1cc79d09eb98d2125a765b2"} Dec 03 11:20:54 crc kubenswrapper[5052]: I1203 11:20:54.142457 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-tvr2s" event={"ID":"ed6dae78-3358-4be5-84c2-142e4d45fe9a","Type":"ContainerStarted","Data":"c2a43819eebf1ba8561eadbe8898a6863694b98e3f1a8943e637db53bc227619"} Dec 03 11:20:54 crc kubenswrapper[5052]: I1203 11:20:54.149688 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\") pod \"8f668bae-612b-4b75-9490-919e737c6a3b\" (UID: \"8f668bae-612b-4b75-9490-919e737c6a3b\") " Dec 03 11:20:54 crc kubenswrapper[5052]: E1203 11:20:54.149834 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8 podName:8f668bae-612b-4b75-9490-919e737c6a3b nodeName:}" failed. No retries permitted until 2025-12-03 11:20:54.649801086 +0000 UTC m=+45.238042850 (durationBeforeRetry 500ms). Error: UnmountVolume.TearDown failed for volume "pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8" (UniqueName: "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8") pod "8f668bae-612b-4b75-9490-919e737c6a3b" (UID: "8f668bae-612b-4b75-9490-919e737c6a3b") : kubernetes.io/csi: Unmounter.TearDownAt failed to get CSI client: driver name kubevirt.io.hostpath-provisioner not found in the list of registered CSI drivers Dec 03 11:20:54 crc kubenswrapper[5052]: I1203 11:20:54.150019 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\") pod \"image-registry-697d97f7c8-p77bx\" (UID: \"f5012254-068c-4360-b310-39ff3d309199\") " pod="openshift-image-registry/image-registry-697d97f7c8-p77bx" Dec 03 11:20:54 crc kubenswrapper[5052]: E1203 11:20:54.150374 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8 podName: nodeName:}" failed. No retries permitted until 2025-12-03 11:20:54.650364462 +0000 UTC m=+45.238606226 (durationBeforeRetry 500ms). Error: MountVolume.MountDevice failed for volume "pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8" (UniqueName: "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8") pod "image-registry-697d97f7c8-p77bx" (UID: "f5012254-068c-4360-b310-39ff3d309199") : kubernetes.io/csi: attacher.MountDevice failed to create newCsiDriverClient: driver name kubevirt.io.hostpath-provisioner not found in the list of registered CSI drivers Dec 03 11:20:54 crc kubenswrapper[5052]: I1203 11:20:54.174530 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-tvr2s" podStartSLOduration=9.174470925 podStartE2EDuration="9.174470925s" podCreationTimestamp="2025-12-03 11:20:45 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:20:54.170456255 +0000 UTC m=+44.758698019" watchObservedRunningTime="2025-12-03 11:20:54.174470925 +0000 UTC m=+44.762712689" Dec 03 11:20:54 crc kubenswrapper[5052]: I1203 11:20:54.194962 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-9c57cc56f-w7g95" event={"ID":"eb07cd6b-71c7-4dc7-b658-40f32c63f373","Type":"ContainerStarted","Data":"ca2ec86d0a8cee11f7d0457a2c319fdf3076960ad30500a6ef7aca29637bdc40"} Dec 03 11:20:54 crc kubenswrapper[5052]: I1203 11:20:54.205112 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-controller-manager/controller-manager-879f6c89f-46rm8" event={"ID":"1c570f3f-3090-48e1-9b7e-f66c99310dbd","Type":"ContainerStarted","Data":"1f634a2dcccde0222d0716fd2f6329e6bc2208c388e57ed224043363bc2ca391"} Dec 03 11:20:54 crc kubenswrapper[5052]: I1203 11:20:54.205945 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openshift-controller-manager/controller-manager-879f6c89f-46rm8" Dec 03 11:20:54 crc kubenswrapper[5052]: I1203 11:20:54.209087 5052 patch_prober.go:28] interesting pod/controller-manager-879f6c89f-46rm8 container/controller-manager namespace/openshift-controller-manager: Readiness probe status=failure output="Get \"https://10.217.0.19:8443/healthz\": dial tcp 10.217.0.19:8443: connect: connection refused" start-of-body= Dec 03 11:20:54 crc kubenswrapper[5052]: I1203 11:20:54.209554 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openshift-controller-manager/controller-manager-879f6c89f-46rm8" podUID="1c570f3f-3090-48e1-9b7e-f66c99310dbd" containerName="controller-manager" probeResult="failure" output="Get \"https://10.217.0.19:8443/healthz\": dial tcp 10.217.0.19:8443: connect: connection refused" Dec 03 11:20:54 crc kubenswrapper[5052]: I1203 11:20:54.215489 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-controller-manager-operator/openshift-controller-manager-operator-756b6f6bc6-v7lmf" event={"ID":"43fca84c-422e-4e3b-b3ca-b787a4e3f677","Type":"ContainerStarted","Data":"0dcbdb55b3e6d898e3844483461026f2e38406571ce3e5fe7bb763f5fec55725"} Dec 03 11:20:54 crc kubenswrapper[5052]: I1203 11:20:54.234319 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-service-ca/service-ca-9c57cc56f-w7g95" podStartSLOduration=22.234287944 podStartE2EDuration="22.234287944s" podCreationTimestamp="2025-12-03 11:20:32 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:20:54.231756144 +0000 UTC m=+44.819997918" watchObservedRunningTime="2025-12-03 11:20:54.234287944 +0000 UTC m=+44.822529708" Dec 03 11:20:54 crc kubenswrapper[5052]: I1203 11:20:54.253250 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\") pod \"8f668bae-612b-4b75-9490-919e737c6a3b\" (UID: \"8f668bae-612b-4b75-9490-919e737c6a3b\") " Dec 03 11:20:54 crc kubenswrapper[5052]: E1203 11:20:54.255323 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8 podName:8f668bae-612b-4b75-9490-919e737c6a3b nodeName:}" failed. No retries permitted until 2025-12-03 11:20:54.755295543 +0000 UTC m=+45.343537307 (durationBeforeRetry 500ms). Error: UnmountVolume.TearDown failed for volume "pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8" (UniqueName: "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8") pod "8f668bae-612b-4b75-9490-919e737c6a3b" (UID: "8f668bae-612b-4b75-9490-919e737c6a3b") : kubernetes.io/csi: Unmounter.TearDownAt failed to get CSI client: driver name kubevirt.io.hostpath-provisioner not found in the list of registered CSI drivers Dec 03 11:20:54 crc kubenswrapper[5052]: I1203 11:20:54.257097 5052 patch_prober.go:28] interesting pod/router-default-5444994796-rtl8t container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-http failed: reason withheld Dec 03 11:20:54 crc kubenswrapper[5052]: [-]has-synced failed: reason withheld Dec 03 11:20:54 crc kubenswrapper[5052]: [+]process-running ok Dec 03 11:20:54 crc kubenswrapper[5052]: healthz check failed Dec 03 11:20:54 crc kubenswrapper[5052]: I1203 11:20:54.257160 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-5444994796-rtl8t" podUID="48040744-7fc1-482e-9b68-3a904ba03a8d" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500" Dec 03 11:20:54 crc kubenswrapper[5052]: I1203 11:20:54.296824 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-authentication-operator/authentication-operator-69f744f599-qjq5f" event={"ID":"e0bf52c7-bf23-4c65-b7cb-0d8d466aa8f2","Type":"ContainerStarted","Data":"a2f1f2c434fd382b4e2e1cfa1be018661a5512be84965a552c038e6d4e87ad62"} Dec 03 11:20:54 crc kubenswrapper[5052]: I1203 11:20:54.296908 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-authentication-operator/authentication-operator-69f744f599-qjq5f" event={"ID":"e0bf52c7-bf23-4c65-b7cb-0d8d466aa8f2","Type":"ContainerStarted","Data":"9daff7089b44abb5c2c2b6c654f7aebe9ac3f395b6a5bdcd59e616c9d2b6f1fb"} Dec 03 11:20:54 crc kubenswrapper[5052]: I1203 11:20:54.322208 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-controller-manager-operator/openshift-controller-manager-operator-756b6f6bc6-v7lmf" podStartSLOduration=23.322181566 podStartE2EDuration="23.322181566s" podCreationTimestamp="2025-12-03 11:20:31 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:20:54.279412918 +0000 UTC m=+44.867654672" watchObservedRunningTime="2025-12-03 11:20:54.322181566 +0000 UTC m=+44.910423330" Dec 03 11:20:54 crc kubenswrapper[5052]: I1203 11:20:54.322864 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-admission-controller-857f4d67dd-q8n58" event={"ID":"daf5492e-2286-42c8-8c08-88200b27e2fb","Type":"ContainerStarted","Data":"83bdad1fe0600686f229fc7b7af7f76563058a68f211300b58eb6db4a40717fe"} Dec 03 11:20:54 crc kubenswrapper[5052]: I1203 11:20:54.336773 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-kube-controller-manager-operator/kube-controller-manager-operator-78b949d7b-h6b78" event={"ID":"3dd904b5-266d-4a49-ad58-805139eef151","Type":"ContainerStarted","Data":"ea8b6ae1797131a4e3065514a7275b7081b551b161e5566be23cd8852b812989"} Dec 03 11:20:54 crc kubenswrapper[5052]: I1203 11:20:54.353435 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-operator-lifecycle-manager/collect-profiles-29412675-hzd4n" event={"ID":"e43f14fc-f3eb-4f58-b202-cdfc641ac08a","Type":"ContainerStarted","Data":"4c7da6eda6ba0ce1724c7cd12ef13c0bc571d89291e3b3beb10c29f534bad15c"} Dec 03 11:20:54 crc kubenswrapper[5052]: I1203 11:20:54.355000 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\") pod \"image-registry-697d97f7c8-p77bx\" (UID: \"f5012254-068c-4360-b310-39ff3d309199\") " pod="openshift-image-registry/image-registry-697d97f7c8-p77bx" Dec 03 11:20:54 crc kubenswrapper[5052]: E1203 11:20:54.356107 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8 podName: nodeName:}" failed. No retries permitted until 2025-12-03 11:20:54.856084951 +0000 UTC m=+45.444326725 (durationBeforeRetry 500ms). Error: MountVolume.MountDevice failed for volume "pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8" (UniqueName: "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8") pod "image-registry-697d97f7c8-p77bx" (UID: "f5012254-068c-4360-b310-39ff3d309199") : kubernetes.io/csi: attacher.MountDevice failed to create newCsiDriverClient: driver name kubevirt.io.hostpath-provisioner not found in the list of registered CSI drivers Dec 03 11:20:54 crc kubenswrapper[5052]: I1203 11:20:54.363358 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-controller-manager/controller-manager-879f6c89f-46rm8" podStartSLOduration=23.362999651 podStartE2EDuration="23.362999651s" podCreationTimestamp="2025-12-03 11:20:31 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:20:54.321642751 +0000 UTC m=+44.909884515" watchObservedRunningTime="2025-12-03 11:20:54.362999651 +0000 UTC m=+44.951241415" Dec 03 11:20:54 crc kubenswrapper[5052]: I1203 11:20:54.364701 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-authentication-operator/authentication-operator-69f744f599-qjq5f" podStartSLOduration=23.364688137999998 podStartE2EDuration="23.364688138s" podCreationTimestamp="2025-12-03 11:20:31 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:20:54.352267325 +0000 UTC m=+44.940509099" watchObservedRunningTime="2025-12-03 11:20:54.364688138 +0000 UTC m=+44.952929922" Dec 03 11:20:54 crc kubenswrapper[5052]: I1203 11:20:54.365596 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-58897d9998-7vpns" event={"ID":"3afb93c1-9cbc-4588-a3bf-2ae1e774a2d1","Type":"ContainerStarted","Data":"9dd1cbc61710283d5c585116dadeb54c2c49de605fd7b38ce31b362b38594852"} Dec 03 11:20:54 crc kubenswrapper[5052]: I1203 11:20:54.366558 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openshift-console-operator/console-operator-58897d9998-7vpns" Dec 03 11:20:54 crc kubenswrapper[5052]: I1203 11:20:54.384999 5052 patch_prober.go:28] interesting pod/console-operator-58897d9998-7vpns container/console-operator namespace/openshift-console-operator: Readiness probe status=failure output="Get \"https://10.217.0.20:8443/readyz\": dial tcp 10.217.0.20:8443: connect: connection refused" start-of-body= Dec 03 11:20:54 crc kubenswrapper[5052]: I1203 11:20:54.385268 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openshift-console-operator/console-operator-58897d9998-7vpns" podUID="3afb93c1-9cbc-4588-a3bf-2ae1e774a2d1" containerName="console-operator" probeResult="failure" output="Get \"https://10.217.0.20:8443/readyz\": dial tcp 10.217.0.20:8443: connect: connection refused" Dec 03 11:20:54 crc kubenswrapper[5052]: I1203 11:20:54.393461 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-f9d7485db-568dj" event={"ID":"b1d2ff08-d58b-48d0-a0e5-6cfd867a63de","Type":"ContainerStarted","Data":"d5c5a4b903b9b98d3670cf389863431252c01e50350a0b60b093433302d7adea"} Dec 03 11:20:54 crc kubenswrapper[5052]: I1203 11:20:54.398436 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="hostpath-provisioner/csi-hostpathplugin-5tlrr" event={"ID":"60b355dc-6de7-4a73-a2d2-3a0f0befd049","Type":"ContainerStarted","Data":"83218e89a1f6993d10fabaddd4c7bddcfec408a3c43057d2a33be96fceeb8047"} Dec 03 11:20:54 crc kubenswrapper[5052]: I1203 11:20:54.398817 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-kube-controller-manager-operator/kube-controller-manager-operator-78b949d7b-h6b78" podStartSLOduration=22.398784467 podStartE2EDuration="22.398784467s" podCreationTimestamp="2025-12-03 11:20:32 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:20:54.390710985 +0000 UTC m=+44.978952779" watchObservedRunningTime="2025-12-03 11:20:54.398784467 +0000 UTC m=+44.987026231" Dec 03 11:20:54 crc kubenswrapper[5052]: I1203 11:20:54.413221 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-apiserver-operator/openshift-apiserver-operator-796bbdcf4f-jskzg" event={"ID":"42cd7f06-f712-41e0-9fa5-c8ec9f7af997","Type":"ContainerStarted","Data":"3029b1a89e7ab333c6ac0900bb193835b44eb6217f738959ee9d3a56a73c1c2e"} Dec 03 11:20:54 crc kubenswrapper[5052]: I1203 11:20:54.422900 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-operator-lifecycle-manager/collect-profiles-29412675-hzd4n" podStartSLOduration=23.422880661 podStartE2EDuration="23.422880661s" podCreationTimestamp="2025-12-03 11:20:31 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:20:54.420838135 +0000 UTC m=+45.009079899" watchObservedRunningTime="2025-12-03 11:20:54.422880661 +0000 UTC m=+45.011122425" Dec 03 11:20:54 crc kubenswrapper[5052]: I1203 11:20:54.429600 5052 generic.go:334] "Generic (PLEG): container finished" podID="0a110335-c2e5-40a2-b565-a4791146d53b" containerID="36c99b00684b76c4a182ff540f39fe5a2235bc86225581b9e6bdcbb467443cb3" exitCode=0 Dec 03 11:20:54 crc kubenswrapper[5052]: I1203 11:20:54.429701 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-config-operator/openshift-config-operator-7777fb866f-bkztv" event={"ID":"0a110335-c2e5-40a2-b565-a4791146d53b","Type":"ContainerDied","Data":"36c99b00684b76c4a182ff540f39fe5a2235bc86225581b9e6bdcbb467443cb3"} Dec 03 11:20:54 crc kubenswrapper[5052]: I1203 11:20:54.445907 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-machine-api/machine-api-operator-5694c8668f-l5lrg" event={"ID":"3d483046-743f-43c6-b73d-31f4f1bcca8c","Type":"ContainerStarted","Data":"c5ae0ac628e86ae1559f7cf47e006fa3eaf03c1570380546cd13ffe881914b3e"} Dec 03 11:20:54 crc kubenswrapper[5052]: I1203 11:20:54.446351 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-machine-api/machine-api-operator-5694c8668f-l5lrg" event={"ID":"3d483046-743f-43c6-b73d-31f4f1bcca8c","Type":"ContainerStarted","Data":"7a64031273b14f2a8b61b7cdd7af7422fbc1a73aef39d6d2a42d2d0d77d3d927"} Dec 03 11:20:54 crc kubenswrapper[5052]: I1203 11:20:54.457163 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\") pod \"8f668bae-612b-4b75-9490-919e737c6a3b\" (UID: \"8f668bae-612b-4b75-9490-919e737c6a3b\") " Dec 03 11:20:54 crc kubenswrapper[5052]: E1203 11:20:54.458779 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8 podName:8f668bae-612b-4b75-9490-919e737c6a3b nodeName:}" failed. No retries permitted until 2025-12-03 11:20:54.9587535 +0000 UTC m=+45.546995264 (durationBeforeRetry 500ms). Error: UnmountVolume.TearDown failed for volume "pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8" (UniqueName: "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8") pod "8f668bae-612b-4b75-9490-919e737c6a3b" (UID: "8f668bae-612b-4b75-9490-919e737c6a3b") : kubernetes.io/csi: Unmounter.TearDownAt failed to get CSI client: driver name kubevirt.io.hostpath-provisioner not found in the list of registered CSI drivers Dec 03 11:20:54 crc kubenswrapper[5052]: I1203 11:20:54.473851 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console-operator/console-operator-58897d9998-7vpns" podStartSLOduration=23.473817865 podStartE2EDuration="23.473817865s" podCreationTimestamp="2025-12-03 11:20:31 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:20:54.465050003 +0000 UTC m=+45.053291787" watchObservedRunningTime="2025-12-03 11:20:54.473817865 +0000 UTC m=+45.062059629" Dec 03 11:20:54 crc kubenswrapper[5052]: I1203 11:20:54.485293 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-kf52k" event={"ID":"a3f2b85d-9b43-4c1c-bc3a-602405b96efc","Type":"ContainerStarted","Data":"1549241964ebb90c5c33bb1eac5e4946cca9ae25750102a4e72100325eaba0f9"} Dec 03 11:20:54 crc kubenswrapper[5052]: I1203 11:20:54.485348 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-kf52k" event={"ID":"a3f2b85d-9b43-4c1c-bc3a-602405b96efc","Type":"ContainerStarted","Data":"69f36ab583623e3144a71b4eadc9ea854fc3e15d49e4498df773402a1a196571"} Dec 03 11:20:54 crc kubenswrapper[5052]: I1203 11:20:54.531596 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-6m2jn" event={"ID":"efbeac4d-779b-4eb2-acf5-241c83a46eb7","Type":"ContainerStarted","Data":"c208d39bc5c299c88ba702262c768d874cdfade896816006a506455136868878"} Dec 03 11:20:54 crc kubenswrapper[5052]: I1203 11:20:54.546752 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-apiserver-operator/openshift-apiserver-operator-796bbdcf4f-jskzg" podStartSLOduration=23.546730084 podStartE2EDuration="23.546730084s" podCreationTimestamp="2025-12-03 11:20:31 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:20:54.545574573 +0000 UTC m=+45.133816337" watchObservedRunningTime="2025-12-03 11:20:54.546730084 +0000 UTC m=+45.134971848" Dec 03 11:20:54 crc kubenswrapper[5052]: I1203 11:20:54.554583 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-777779d784-9jvh6" event={"ID":"3e0657c6-3f3c-4300-8cfb-b3db773dba75","Type":"ContainerStarted","Data":"c951a9b48226d13c8f902e410042072e955c59ac3990edc77e30b4c866994b6a"} Dec 03 11:20:54 crc kubenswrapper[5052]: I1203 11:20:54.558976 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\") pod \"image-registry-697d97f7c8-p77bx\" (UID: \"f5012254-068c-4360-b310-39ff3d309199\") " pod="openshift-image-registry/image-registry-697d97f7c8-p77bx" Dec 03 11:20:54 crc kubenswrapper[5052]: E1203 11:20:54.560977 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8 podName: nodeName:}" failed. No retries permitted until 2025-12-03 11:20:55.060961267 +0000 UTC m=+45.649203031 (durationBeforeRetry 500ms). Error: MountVolume.MountDevice failed for volume "pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8" (UniqueName: "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8") pod "image-registry-697d97f7c8-p77bx" (UID: "f5012254-068c-4360-b310-39ff3d309199") : kubernetes.io/csi: attacher.MountDevice failed to create newCsiDriverClient: driver name kubevirt.io.hostpath-provisioner not found in the list of registered CSI drivers Dec 03 11:20:54 crc kubenswrapper[5052]: I1203 11:20:54.568104 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-kube-apiserver-operator/kube-apiserver-operator-766d6c64bb-xwlk8" event={"ID":"287ad4d4-bf40-4cc4-9856-580b65a2e88d","Type":"ContainerStarted","Data":"a6ac061e4cf748351be2c666fe3126e715a772c152b903eb16f8e3723a5e3a9d"} Dec 03 11:20:54 crc kubenswrapper[5052]: I1203 11:20:54.595949 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/console-f9d7485db-568dj" podStartSLOduration=23.59591855 podStartE2EDuration="23.59591855s" podCreationTimestamp="2025-12-03 11:20:31 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:20:54.585308778 +0000 UTC m=+45.173550562" watchObservedRunningTime="2025-12-03 11:20:54.59591855 +0000 UTC m=+45.184160314" Dec 03 11:20:54 crc kubenswrapper[5052]: I1203 11:20:54.608203 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-apiserver/apiserver-76f77b778f-dmvst" event={"ID":"41016a7b-cc79-4f32-860c-cb75e405cf42","Type":"ContainerStarted","Data":"9b870ba873178115d65a9003417991c1107a5cf27778a851d654c2472e79548a"} Dec 03 11:20:54 crc kubenswrapper[5052]: I1203 11:20:54.650668 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-kube-apiserver-operator/kube-apiserver-operator-766d6c64bb-xwlk8" podStartSLOduration=22.650627678 podStartE2EDuration="22.650627678s" podCreationTimestamp="2025-12-03 11:20:32 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:20:54.610390029 +0000 UTC m=+45.198631803" watchObservedRunningTime="2025-12-03 11:20:54.650627678 +0000 UTC m=+45.238869442" Dec 03 11:20:54 crc kubenswrapper[5052]: I1203 11:20:54.654943 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-kube-scheduler-operator/openshift-kube-scheduler-operator-5fdd9b5758-rxfb5" event={"ID":"5d079af1-8d26-4a35-89f6-1312e1d5c2dc","Type":"ContainerStarted","Data":"8a21f30c2af44fdb7902dcfb214924235234baa6c40ef3c6f74b5863ab281b50"} Dec 03 11:20:54 crc kubenswrapper[5052]: I1203 11:20:54.660068 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\") pod \"8f668bae-612b-4b75-9490-919e737c6a3b\" (UID: \"8f668bae-612b-4b75-9490-919e737c6a3b\") " Dec 03 11:20:54 crc kubenswrapper[5052]: E1203 11:20:54.660633 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8 podName:8f668bae-612b-4b75-9490-919e737c6a3b nodeName:}" failed. No retries permitted until 2025-12-03 11:20:55.160602343 +0000 UTC m=+45.748844287 (durationBeforeRetry 500ms). Error: UnmountVolume.TearDown failed for volume "pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8" (UniqueName: "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8") pod "8f668bae-612b-4b75-9490-919e737c6a3b" (UID: "8f668bae-612b-4b75-9490-919e737c6a3b") : kubernetes.io/csi: Unmounter.TearDownAt failed to get CSI client: driver name kubevirt.io.hostpath-provisioner not found in the list of registered CSI drivers Dec 03 11:20:54 crc kubenswrapper[5052]: I1203 11:20:54.714299 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-machine-approver/machine-approver-56656f9798-6t9dp" event={"ID":"8c1b25f5-2d98-4852-bc7a-d5cbfa5cb6aa","Type":"ContainerStarted","Data":"b4e29b7cd1991b63ac854bdb2c251a9895ecfa53b789f5cf1a0c693bc0293b09"} Dec 03 11:20:54 crc kubenswrapper[5052]: I1203 11:20:54.714690 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-kube-scheduler-operator/openshift-kube-scheduler-operator-5fdd9b5758-rxfb5" podStartSLOduration=22.714675763 podStartE2EDuration="22.714675763s" podCreationTimestamp="2025-12-03 11:20:32 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:20:54.714245751 +0000 UTC m=+45.302487515" watchObservedRunningTime="2025-12-03 11:20:54.714675763 +0000 UTC m=+45.302917527" Dec 03 11:20:54 crc kubenswrapper[5052]: I1203 11:20:54.736773 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-b67b599dd-5h545" event={"ID":"6e31bff9-7ac9-47a7-a895-db2cc3051d85","Type":"ContainerStarted","Data":"be447860efa147cd171f704464b0733a3a443a6db7998c8b2cfb1e13553daf3d"} Dec 03 11:20:54 crc kubenswrapper[5052]: I1203 11:20:54.757317 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-operator-lifecycle-manager/catalog-operator-68c6474976-rqhkg" event={"ID":"8e3a111c-05ff-4f33-b5ba-81dae833b529","Type":"ContainerStarted","Data":"edb81e6e71130e49a423a2c464a5faa8c37e762714ea709cca8094b91821f832"} Dec 03 11:20:54 crc kubenswrapper[5052]: I1203 11:20:54.757363 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openshift-operator-lifecycle-manager/catalog-operator-68c6474976-rqhkg" Dec 03 11:20:54 crc kubenswrapper[5052]: I1203 11:20:54.763664 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\") pod \"image-registry-697d97f7c8-p77bx\" (UID: \"f5012254-068c-4360-b310-39ff3d309199\") " pod="openshift-image-registry/image-registry-697d97f7c8-p77bx" Dec 03 11:20:54 crc kubenswrapper[5052]: I1203 11:20:54.766422 5052 patch_prober.go:28] interesting pod/downloads-7954f5f757-l7xtl container/download-server namespace/openshift-console: Readiness probe status=failure output="Get \"http://10.217.0.18:8080/\": dial tcp 10.217.0.18:8080: connect: connection refused" start-of-body= Dec 03 11:20:54 crc kubenswrapper[5052]: I1203 11:20:54.766476 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openshift-console/downloads-7954f5f757-l7xtl" podUID="ec2ea657-018b-4626-bf79-5c818f573a4b" containerName="download-server" probeResult="failure" output="Get \"http://10.217.0.18:8080/\": dial tcp 10.217.0.18:8080: connect: connection refused" Dec 03 11:20:54 crc kubenswrapper[5052]: I1203 11:20:54.769406 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-marketplace/marketplace-operator-79b997595-kdr8l" Dec 03 11:20:54 crc kubenswrapper[5052]: E1203 11:20:54.771414 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8 podName: nodeName:}" failed. No retries permitted until 2025-12-03 11:20:55.271391426 +0000 UTC m=+45.859633360 (durationBeforeRetry 500ms). Error: MountVolume.MountDevice failed for volume "pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8" (UniqueName: "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8") pod "image-registry-697d97f7c8-p77bx" (UID: "f5012254-068c-4360-b310-39ff3d309199") : kubernetes.io/csi: attacher.MountDevice failed to create newCsiDriverClient: driver name kubevirt.io.hostpath-provisioner not found in the list of registered CSI drivers Dec 03 11:20:54 crc kubenswrapper[5052]: I1203 11:20:54.779378 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-b67b599dd-5h545" podStartSLOduration=22.779347775 podStartE2EDuration="22.779347775s" podCreationTimestamp="2025-12-03 11:20:32 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:20:54.768804675 +0000 UTC m=+45.357046439" watchObservedRunningTime="2025-12-03 11:20:54.779347775 +0000 UTC m=+45.367589539" Dec 03 11:20:54 crc kubenswrapper[5052]: I1203 11:20:54.800082 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-operator-lifecycle-manager/catalog-operator-68c6474976-rqhkg" Dec 03 11:20:54 crc kubenswrapper[5052]: I1203 11:20:54.828009 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-operator-lifecycle-manager/catalog-operator-68c6474976-rqhkg" podStartSLOduration=22.827968195 podStartE2EDuration="22.827968195s" podCreationTimestamp="2025-12-03 11:20:32 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:20:54.825702533 +0000 UTC m=+45.413944307" watchObservedRunningTime="2025-12-03 11:20:54.827968195 +0000 UTC m=+45.416209959" Dec 03 11:20:54 crc kubenswrapper[5052]: I1203 11:20:54.869185 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\") pod \"8f668bae-612b-4b75-9490-919e737c6a3b\" (UID: \"8f668bae-612b-4b75-9490-919e737c6a3b\") " Dec 03 11:20:54 crc kubenswrapper[5052]: E1203 11:20:54.871085 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8 podName:8f668bae-612b-4b75-9490-919e737c6a3b nodeName:}" failed. No retries permitted until 2025-12-03 11:20:55.371070403 +0000 UTC m=+45.959312167 (durationBeforeRetry 500ms). Error: UnmountVolume.TearDown failed for volume "pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8" (UniqueName: "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8") pod "8f668bae-612b-4b75-9490-919e737c6a3b" (UID: "8f668bae-612b-4b75-9490-919e737c6a3b") : kubernetes.io/csi: Unmounter.TearDownAt failed to get CSI client: driver name kubevirt.io.hostpath-provisioner not found in the list of registered CSI drivers Dec 03 11:20:54 crc kubenswrapper[5052]: I1203 11:20:54.986376 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\") pod \"image-registry-697d97f7c8-p77bx\" (UID: \"f5012254-068c-4360-b310-39ff3d309199\") " pod="openshift-image-registry/image-registry-697d97f7c8-p77bx" Dec 03 11:20:54 crc kubenswrapper[5052]: E1203 11:20:54.986737 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8 podName: nodeName:}" failed. No retries permitted until 2025-12-03 11:20:55.48672282 +0000 UTC m=+46.074964584 (durationBeforeRetry 500ms). Error: MountVolume.MountDevice failed for volume "pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8" (UniqueName: "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8") pod "image-registry-697d97f7c8-p77bx" (UID: "f5012254-068c-4360-b310-39ff3d309199") : kubernetes.io/csi: attacher.MountDevice failed to create newCsiDriverClient: driver name kubevirt.io.hostpath-provisioner not found in the list of registered CSI drivers Dec 03 11:20:55 crc kubenswrapper[5052]: I1203 11:20:55.097140 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\") pod \"8f668bae-612b-4b75-9490-919e737c6a3b\" (UID: \"8f668bae-612b-4b75-9490-919e737c6a3b\") " Dec 03 11:20:55 crc kubenswrapper[5052]: E1203 11:20:55.097697 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8 podName:8f668bae-612b-4b75-9490-919e737c6a3b nodeName:}" failed. No retries permitted until 2025-12-03 11:20:55.597674928 +0000 UTC m=+46.185916702 (durationBeforeRetry 500ms). Error: UnmountVolume.TearDown failed for volume "pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8" (UniqueName: "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8") pod "8f668bae-612b-4b75-9490-919e737c6a3b" (UID: "8f668bae-612b-4b75-9490-919e737c6a3b") : kubernetes.io/csi: Unmounter.TearDownAt failed to get CSI client: driver name kubevirt.io.hostpath-provisioner not found in the list of registered CSI drivers Dec 03 11:20:55 crc kubenswrapper[5052]: I1203 11:20:55.202369 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\") pod \"image-registry-697d97f7c8-p77bx\" (UID: \"f5012254-068c-4360-b310-39ff3d309199\") " pod="openshift-image-registry/image-registry-697d97f7c8-p77bx" Dec 03 11:20:55 crc kubenswrapper[5052]: E1203 11:20:55.202759 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8 podName: nodeName:}" failed. No retries permitted until 2025-12-03 11:20:55.702745893 +0000 UTC m=+46.290987657 (durationBeforeRetry 500ms). Error: MountVolume.MountDevice failed for volume "pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8" (UniqueName: "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8") pod "image-registry-697d97f7c8-p77bx" (UID: "f5012254-068c-4360-b310-39ff3d309199") : kubernetes.io/csi: attacher.MountDevice failed to create newCsiDriverClient: driver name kubevirt.io.hostpath-provisioner not found in the list of registered CSI drivers Dec 03 11:20:55 crc kubenswrapper[5052]: I1203 11:20:55.256775 5052 patch_prober.go:28] interesting pod/router-default-5444994796-rtl8t container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-http failed: reason withheld Dec 03 11:20:55 crc kubenswrapper[5052]: [-]has-synced failed: reason withheld Dec 03 11:20:55 crc kubenswrapper[5052]: [+]process-running ok Dec 03 11:20:55 crc kubenswrapper[5052]: healthz check failed Dec 03 11:20:55 crc kubenswrapper[5052]: I1203 11:20:55.256842 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-5444994796-rtl8t" podUID="48040744-7fc1-482e-9b68-3a904ba03a8d" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500" Dec 03 11:20:55 crc kubenswrapper[5052]: I1203 11:20:55.303844 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\") pod \"8f668bae-612b-4b75-9490-919e737c6a3b\" (UID: \"8f668bae-612b-4b75-9490-919e737c6a3b\") " Dec 03 11:20:55 crc kubenswrapper[5052]: E1203 11:20:55.304480 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8 podName:8f668bae-612b-4b75-9490-919e737c6a3b nodeName:}" failed. No retries permitted until 2025-12-03 11:20:55.804450736 +0000 UTC m=+46.392692500 (durationBeforeRetry 500ms). Error: UnmountVolume.TearDown failed for volume "pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8" (UniqueName: "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8") pod "8f668bae-612b-4b75-9490-919e737c6a3b" (UID: "8f668bae-612b-4b75-9490-919e737c6a3b") : kubernetes.io/csi: Unmounter.TearDownAt failed to get CSI client: driver name kubevirt.io.hostpath-provisioner not found in the list of registered CSI drivers Dec 03 11:20:55 crc kubenswrapper[5052]: I1203 11:20:55.304586 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\") pod \"image-registry-697d97f7c8-p77bx\" (UID: \"f5012254-068c-4360-b310-39ff3d309199\") " pod="openshift-image-registry/image-registry-697d97f7c8-p77bx" Dec 03 11:20:55 crc kubenswrapper[5052]: E1203 11:20:55.305075 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8 podName: nodeName:}" failed. No retries permitted until 2025-12-03 11:20:55.805057303 +0000 UTC m=+46.393299077 (durationBeforeRetry 500ms). Error: MountVolume.MountDevice failed for volume "pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8" (UniqueName: "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8") pod "image-registry-697d97f7c8-p77bx" (UID: "f5012254-068c-4360-b310-39ff3d309199") : kubernetes.io/csi: attacher.MountDevice failed to create newCsiDriverClient: driver name kubevirt.io.hostpath-provisioner not found in the list of registered CSI drivers Dec 03 11:20:55 crc kubenswrapper[5052]: I1203 11:20:55.405884 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\") pod \"8f668bae-612b-4b75-9490-919e737c6a3b\" (UID: \"8f668bae-612b-4b75-9490-919e737c6a3b\") " Dec 03 11:20:55 crc kubenswrapper[5052]: E1203 11:20:55.406148 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8 podName:8f668bae-612b-4b75-9490-919e737c6a3b nodeName:}" failed. No retries permitted until 2025-12-03 11:20:55.906118728 +0000 UTC m=+46.494360492 (durationBeforeRetry 500ms). Error: UnmountVolume.TearDown failed for volume "pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8" (UniqueName: "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8") pod "8f668bae-612b-4b75-9490-919e737c6a3b" (UID: "8f668bae-612b-4b75-9490-919e737c6a3b") : kubernetes.io/csi: Unmounter.TearDownAt failed to get CSI client: driver name kubevirt.io.hostpath-provisioner not found in the list of registered CSI drivers Dec 03 11:20:55 crc kubenswrapper[5052]: I1203 11:20:55.406288 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\") pod \"image-registry-697d97f7c8-p77bx\" (UID: \"f5012254-068c-4360-b310-39ff3d309199\") " pod="openshift-image-registry/image-registry-697d97f7c8-p77bx" Dec 03 11:20:55 crc kubenswrapper[5052]: E1203 11:20:55.406714 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8 podName: nodeName:}" failed. No retries permitted until 2025-12-03 11:20:55.906697874 +0000 UTC m=+46.494939638 (durationBeforeRetry 500ms). Error: MountVolume.MountDevice failed for volume "pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8" (UniqueName: "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8") pod "image-registry-697d97f7c8-p77bx" (UID: "f5012254-068c-4360-b310-39ff3d309199") : kubernetes.io/csi: attacher.MountDevice failed to create newCsiDriverClient: driver name kubevirt.io.hostpath-provisioner not found in the list of registered CSI drivers Dec 03 11:20:55 crc kubenswrapper[5052]: I1203 11:20:55.508144 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\") pod \"8f668bae-612b-4b75-9490-919e737c6a3b\" (UID: \"8f668bae-612b-4b75-9490-919e737c6a3b\") " Dec 03 11:20:55 crc kubenswrapper[5052]: E1203 11:20:55.508408 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8 podName:8f668bae-612b-4b75-9490-919e737c6a3b nodeName:}" failed. No retries permitted until 2025-12-03 11:20:56.008369096 +0000 UTC m=+46.596610860 (durationBeforeRetry 500ms). Error: UnmountVolume.TearDown failed for volume "pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8" (UniqueName: "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8") pod "8f668bae-612b-4b75-9490-919e737c6a3b" (UID: "8f668bae-612b-4b75-9490-919e737c6a3b") : kubernetes.io/csi: Unmounter.TearDownAt failed to get CSI client: driver name kubevirt.io.hostpath-provisioner not found in the list of registered CSI drivers Dec 03 11:20:55 crc kubenswrapper[5052]: I1203 11:20:55.508487 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\") pod \"image-registry-697d97f7c8-p77bx\" (UID: \"f5012254-068c-4360-b310-39ff3d309199\") " pod="openshift-image-registry/image-registry-697d97f7c8-p77bx" Dec 03 11:20:55 crc kubenswrapper[5052]: E1203 11:20:55.508976 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8 podName: nodeName:}" failed. No retries permitted until 2025-12-03 11:20:56.008968793 +0000 UTC m=+46.597210557 (durationBeforeRetry 500ms). Error: MountVolume.MountDevice failed for volume "pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8" (UniqueName: "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8") pod "image-registry-697d97f7c8-p77bx" (UID: "f5012254-068c-4360-b310-39ff3d309199") : kubernetes.io/csi: attacher.MountDevice failed to create newCsiDriverClient: driver name kubevirt.io.hostpath-provisioner not found in the list of registered CSI drivers Dec 03 11:20:55 crc kubenswrapper[5052]: I1203 11:20:55.609825 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\") pod \"8f668bae-612b-4b75-9490-919e737c6a3b\" (UID: \"8f668bae-612b-4b75-9490-919e737c6a3b\") " Dec 03 11:20:55 crc kubenswrapper[5052]: E1203 11:20:55.610067 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8 podName:8f668bae-612b-4b75-9490-919e737c6a3b nodeName:}" failed. No retries permitted until 2025-12-03 11:20:56.110035848 +0000 UTC m=+46.698277612 (durationBeforeRetry 500ms). Error: UnmountVolume.TearDown failed for volume "pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8" (UniqueName: "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8") pod "8f668bae-612b-4b75-9490-919e737c6a3b" (UID: "8f668bae-612b-4b75-9490-919e737c6a3b") : kubernetes.io/csi: Unmounter.TearDownAt failed to get CSI client: driver name kubevirt.io.hostpath-provisioner not found in the list of registered CSI drivers Dec 03 11:20:55 crc kubenswrapper[5052]: I1203 11:20:55.610131 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\") pod \"image-registry-697d97f7c8-p77bx\" (UID: \"f5012254-068c-4360-b310-39ff3d309199\") " pod="openshift-image-registry/image-registry-697d97f7c8-p77bx" Dec 03 11:20:55 crc kubenswrapper[5052]: E1203 11:20:55.610491 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8 podName: nodeName:}" failed. No retries permitted until 2025-12-03 11:20:56.11047724 +0000 UTC m=+46.698719004 (durationBeforeRetry 500ms). Error: MountVolume.MountDevice failed for volume "pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8" (UniqueName: "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8") pod "image-registry-697d97f7c8-p77bx" (UID: "f5012254-068c-4360-b310-39ff3d309199") : kubernetes.io/csi: attacher.MountDevice failed to create newCsiDriverClient: driver name kubevirt.io.hostpath-provisioner not found in the list of registered CSI drivers Dec 03 11:20:55 crc kubenswrapper[5052]: I1203 11:20:55.704571 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openshift-marketplace/community-operators-pxx88"] Dec 03 11:20:55 crc kubenswrapper[5052]: I1203 11:20:55.705618 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/community-operators-pxx88" Dec 03 11:20:55 crc kubenswrapper[5052]: I1203 11:20:55.713070 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-marketplace"/"community-operators-dockercfg-dmngl" Dec 03 11:20:55 crc kubenswrapper[5052]: I1203 11:20:55.713718 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\") pod \"8f668bae-612b-4b75-9490-919e737c6a3b\" (UID: \"8f668bae-612b-4b75-9490-919e737c6a3b\") " Dec 03 11:20:55 crc kubenswrapper[5052]: E1203 11:20:55.714189 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8 podName:8f668bae-612b-4b75-9490-919e737c6a3b nodeName:}" failed. No retries permitted until 2025-12-03 11:20:56.214173178 +0000 UTC m=+46.802414952 (durationBeforeRetry 500ms). Error: UnmountVolume.TearDown failed for volume "pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8" (UniqueName: "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8") pod "8f668bae-612b-4b75-9490-919e737c6a3b" (UID: "8f668bae-612b-4b75-9490-919e737c6a3b") : kubernetes.io/csi: Unmounter.TearDownAt failed to get CSI client: driver name kubevirt.io.hostpath-provisioner not found in the list of registered CSI drivers Dec 03 11:20:55 crc kubenswrapper[5052]: I1203 11:20:55.727923 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/community-operators-pxx88"] Dec 03 11:20:55 crc kubenswrapper[5052]: I1203 11:20:55.797756 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-kf52k" event={"ID":"a3f2b85d-9b43-4c1c-bc3a-602405b96efc","Type":"ContainerStarted","Data":"17ac2ebfbb304b846c8582efa00e83522ca08bdadc7f857950bcb8c7a53ab8ea"} Dec 03 11:20:55 crc kubenswrapper[5052]: I1203 11:20:55.803636 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-b67b599dd-5h545" event={"ID":"6e31bff9-7ac9-47a7-a895-db2cc3051d85","Type":"ContainerStarted","Data":"6a75d8eadeef9d85822f7d98d181720101ca89ff5ee305a9d728fd202de44d5a"} Dec 03 11:20:55 crc kubenswrapper[5052]: I1203 11:20:55.813935 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-f9d7485db-568dj" event={"ID":"b1d2ff08-d58b-48d0-a0e5-6cfd867a63de","Type":"ContainerStarted","Data":"26c41ad67f1bd2c1f945f4259e4ebfeaef4d6c83d4ff3d8f2f1ec262785569b6"} Dec 03 11:20:55 crc kubenswrapper[5052]: I1203 11:20:55.815402 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\") pod \"image-registry-697d97f7c8-p77bx\" (UID: \"f5012254-068c-4360-b310-39ff3d309199\") " pod="openshift-image-registry/image-registry-697d97f7c8-p77bx" Dec 03 11:20:55 crc kubenswrapper[5052]: I1203 11:20:55.815452 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/a7228f2a-1d6d-4307-80e7-deebd47d995a-utilities\") pod \"community-operators-pxx88\" (UID: \"a7228f2a-1d6d-4307-80e7-deebd47d995a\") " pod="openshift-marketplace/community-operators-pxx88" Dec 03 11:20:55 crc kubenswrapper[5052]: I1203 11:20:55.815483 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4xqqz\" (UniqueName: \"kubernetes.io/projected/a7228f2a-1d6d-4307-80e7-deebd47d995a-kube-api-access-4xqqz\") pod \"community-operators-pxx88\" (UID: \"a7228f2a-1d6d-4307-80e7-deebd47d995a\") " pod="openshift-marketplace/community-operators-pxx88" Dec 03 11:20:55 crc kubenswrapper[5052]: I1203 11:20:55.815540 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/a7228f2a-1d6d-4307-80e7-deebd47d995a-catalog-content\") pod \"community-operators-pxx88\" (UID: \"a7228f2a-1d6d-4307-80e7-deebd47d995a\") " pod="openshift-marketplace/community-operators-pxx88" Dec 03 11:20:55 crc kubenswrapper[5052]: E1203 11:20:55.816127 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8 podName: nodeName:}" failed. No retries permitted until 2025-12-03 11:20:56.316099167 +0000 UTC m=+46.904341051 (durationBeforeRetry 500ms). Error: MountVolume.MountDevice failed for volume "pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8" (UniqueName: "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8") pod "image-registry-697d97f7c8-p77bx" (UID: "f5012254-068c-4360-b310-39ff3d309199") : kubernetes.io/csi: attacher.MountDevice failed to create newCsiDriverClient: driver name kubevirt.io.hostpath-provisioner not found in the list of registered CSI drivers Dec 03 11:20:55 crc kubenswrapper[5052]: I1203 11:20:55.822524 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-etcd-operator/etcd-operator-b45778765-z2lp6" event={"ID":"71ea0624-9fb5-415a-96e1-d5ada83c0fe4","Type":"ContainerStarted","Data":"1dd45d2052f8f7b1288237fa910448c46fd33857808024e5d0d2376c44a58a89"} Dec 03 11:20:55 crc kubenswrapper[5052]: I1203 11:20:55.829876 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-kf52k" podStartSLOduration=24.829855615 podStartE2EDuration="24.829855615s" podCreationTimestamp="2025-12-03 11:20:31 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:20:55.828379674 +0000 UTC m=+46.416621438" watchObservedRunningTime="2025-12-03 11:20:55.829855615 +0000 UTC m=+46.418097379" Dec 03 11:20:55 crc kubenswrapper[5052]: I1203 11:20:55.830815 5052 generic.go:334] "Generic (PLEG): container finished" podID="41016a7b-cc79-4f32-860c-cb75e405cf42" containerID="c46762ab1787f9ebb49a8bfcbf273a0beb9f8a943c47340148e08302884731f0" exitCode=0 Dec 03 11:20:55 crc kubenswrapper[5052]: I1203 11:20:55.830908 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-apiserver/apiserver-76f77b778f-dmvst" event={"ID":"41016a7b-cc79-4f32-860c-cb75e405cf42","Type":"ContainerStarted","Data":"88f8495a0724e1134b0e50e7ba657723e3b10f21a961930af7c13e83db7fce12"} Dec 03 11:20:55 crc kubenswrapper[5052]: I1203 11:20:55.830948 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-apiserver/apiserver-76f77b778f-dmvst" event={"ID":"41016a7b-cc79-4f32-860c-cb75e405cf42","Type":"ContainerDied","Data":"c46762ab1787f9ebb49a8bfcbf273a0beb9f8a943c47340148e08302884731f0"} Dec 03 11:20:55 crc kubenswrapper[5052]: I1203 11:20:55.854662 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-kube-scheduler-operator/openshift-kube-scheduler-operator-5fdd9b5758-rxfb5" event={"ID":"5d079af1-8d26-4a35-89f6-1312e1d5c2dc","Type":"ContainerStarted","Data":"ed84622527e422f2aa66b874ed9866e9340fa7c06fc9a159aadb1e9102557478"} Dec 03 11:20:55 crc kubenswrapper[5052]: I1203 11:20:55.867129 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-config-operator/openshift-config-operator-7777fb866f-bkztv" event={"ID":"0a110335-c2e5-40a2-b565-a4791146d53b","Type":"ContainerStarted","Data":"a88b198251211eaa1d40b34c7dabcecc0e8d3de4f0281299fa865879ac29bcd9"} Dec 03 11:20:55 crc kubenswrapper[5052]: I1203 11:20:55.867306 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openshift-config-operator/openshift-config-operator-7777fb866f-bkztv" Dec 03 11:20:55 crc kubenswrapper[5052]: I1203 11:20:55.875683 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-etcd-operator/etcd-operator-b45778765-z2lp6" podStartSLOduration=24.875655337 podStartE2EDuration="24.875655337s" podCreationTimestamp="2025-12-03 11:20:31 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:20:55.873111457 +0000 UTC m=+46.461353231" watchObservedRunningTime="2025-12-03 11:20:55.875655337 +0000 UTC m=+46.463897101" Dec 03 11:20:55 crc kubenswrapper[5052]: I1203 11:20:55.887190 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-operator-lifecycle-manager/collect-profiles-29412675-hzd4n" event={"ID":"e43f14fc-f3eb-4f58-b202-cdfc641ac08a","Type":"ContainerStarted","Data":"e2f05729fcded9f0fd3a94445c561f0dc54d8da986e5b0a038ffa712510d5162"} Dec 03 11:20:55 crc kubenswrapper[5052]: I1203 11:20:55.896951 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openshift-marketplace/certified-operators-ghqsf"] Dec 03 11:20:55 crc kubenswrapper[5052]: I1203 11:20:55.910546 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-machine-api/machine-api-operator-5694c8668f-l5lrg" event={"ID":"3d483046-743f-43c6-b73d-31f4f1bcca8c","Type":"ContainerStarted","Data":"c696f6efd0b47cb0ba233a8604376de52ffd4c369e02c55ce8c8ed073e6f16c4"} Dec 03 11:20:55 crc kubenswrapper[5052]: I1203 11:20:55.910708 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/certified-operators-ghqsf" Dec 03 11:20:55 crc kubenswrapper[5052]: I1203 11:20:55.912956 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-config-operator/openshift-config-operator-7777fb866f-bkztv" podStartSLOduration=24.912939584 podStartE2EDuration="24.912939584s" podCreationTimestamp="2025-12-03 11:20:31 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:20:55.910363263 +0000 UTC m=+46.498605017" watchObservedRunningTime="2025-12-03 11:20:55.912939584 +0000 UTC m=+46.501181348" Dec 03 11:20:55 crc kubenswrapper[5052]: I1203 11:20:55.923285 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-marketplace"/"certified-operators-dockercfg-4rs5g" Dec 03 11:20:55 crc kubenswrapper[5052]: I1203 11:20:55.924201 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\") pod \"8f668bae-612b-4b75-9490-919e737c6a3b\" (UID: \"8f668bae-612b-4b75-9490-919e737c6a3b\") " Dec 03 11:20:55 crc kubenswrapper[5052]: I1203 11:20:55.924413 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/certified-operators-ghqsf"] Dec 03 11:20:55 crc kubenswrapper[5052]: I1203 11:20:55.924558 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/a7228f2a-1d6d-4307-80e7-deebd47d995a-utilities\") pod \"community-operators-pxx88\" (UID: \"a7228f2a-1d6d-4307-80e7-deebd47d995a\") " pod="openshift-marketplace/community-operators-pxx88" Dec 03 11:20:55 crc kubenswrapper[5052]: I1203 11:20:55.924627 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-4xqqz\" (UniqueName: \"kubernetes.io/projected/a7228f2a-1d6d-4307-80e7-deebd47d995a-kube-api-access-4xqqz\") pod \"community-operators-pxx88\" (UID: \"a7228f2a-1d6d-4307-80e7-deebd47d995a\") " pod="openshift-marketplace/community-operators-pxx88" Dec 03 11:20:55 crc kubenswrapper[5052]: E1203 11:20:55.924738 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8 podName:8f668bae-612b-4b75-9490-919e737c6a3b nodeName:}" failed. No retries permitted until 2025-12-03 11:20:56.424712689 +0000 UTC m=+47.012954453 (durationBeforeRetry 500ms). Error: UnmountVolume.TearDown failed for volume "pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8" (UniqueName: "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8") pod "8f668bae-612b-4b75-9490-919e737c6a3b" (UID: "8f668bae-612b-4b75-9490-919e737c6a3b") : kubernetes.io/csi: Unmounter.TearDownAt failed to get CSI client: driver name kubevirt.io.hostpath-provisioner not found in the list of registered CSI drivers Dec 03 11:20:55 crc kubenswrapper[5052]: I1203 11:20:55.924971 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/a7228f2a-1d6d-4307-80e7-deebd47d995a-catalog-content\") pod \"community-operators-pxx88\" (UID: \"a7228f2a-1d6d-4307-80e7-deebd47d995a\") " pod="openshift-marketplace/community-operators-pxx88" Dec 03 11:20:55 crc kubenswrapper[5052]: I1203 11:20:55.926092 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/a7228f2a-1d6d-4307-80e7-deebd47d995a-catalog-content\") pod \"community-operators-pxx88\" (UID: \"a7228f2a-1d6d-4307-80e7-deebd47d995a\") " pod="openshift-marketplace/community-operators-pxx88" Dec 03 11:20:55 crc kubenswrapper[5052]: I1203 11:20:55.927577 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/a7228f2a-1d6d-4307-80e7-deebd47d995a-utilities\") pod \"community-operators-pxx88\" (UID: \"a7228f2a-1d6d-4307-80e7-deebd47d995a\") " pod="openshift-marketplace/community-operators-pxx88" Dec 03 11:20:55 crc kubenswrapper[5052]: I1203 11:20:55.931430 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-admission-controller-857f4d67dd-q8n58" event={"ID":"daf5492e-2286-42c8-8c08-88200b27e2fb","Type":"ContainerStarted","Data":"930f6eafe7c2dea15b0a66179a1ef1d9de5425744e8115e1b7cb1e022574d814"} Dec 03 11:20:55 crc kubenswrapper[5052]: I1203 11:20:55.943965 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-6m2jn" event={"ID":"efbeac4d-779b-4eb2-acf5-241c83a46eb7","Type":"ContainerStarted","Data":"aa7bb309bb95405e1bca61006fb0e33d55a58ff08d7b7f149daf59bd5270990a"} Dec 03 11:20:55 crc kubenswrapper[5052]: I1203 11:20:55.944038 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-6m2jn" event={"ID":"efbeac4d-779b-4eb2-acf5-241c83a46eb7","Type":"ContainerStarted","Data":"60e6b80b9bd91210ab596c58a5107e898dcb028d4ee9af2b03eae2a7c402463d"} Dec 03 11:20:55 crc kubenswrapper[5052]: I1203 11:20:55.944751 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openshift-dns/dns-default-6m2jn" Dec 03 11:20:55 crc kubenswrapper[5052]: I1203 11:20:55.970039 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-machine-api/machine-api-operator-5694c8668f-l5lrg" podStartSLOduration=23.970018857 podStartE2EDuration="23.970018857s" podCreationTimestamp="2025-12-03 11:20:32 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:20:55.968138546 +0000 UTC m=+46.556380310" watchObservedRunningTime="2025-12-03 11:20:55.970018857 +0000 UTC m=+46.558260631" Dec 03 11:20:55 crc kubenswrapper[5052]: I1203 11:20:55.979861 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-operator-lifecycle-manager/package-server-manager-789f6589d5-577rn" event={"ID":"3f6f3c17-944e-4afb-8cd7-3ca95f787057","Type":"ContainerStarted","Data":"61483a3cb45b5e04ef23b0b52d957af59af0852f4b4af07a697b755e35f0c55a"} Dec 03 11:20:55 crc kubenswrapper[5052]: I1203 11:20:55.979928 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-operator-lifecycle-manager/package-server-manager-789f6589d5-577rn" event={"ID":"3f6f3c17-944e-4afb-8cd7-3ca95f787057","Type":"ContainerStarted","Data":"c7ac38ae365b3c4897db38434607870903161d0961a751c170b7c13ec28eb70e"} Dec 03 11:20:55 crc kubenswrapper[5052]: I1203 11:20:55.980559 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openshift-operator-lifecycle-manager/package-server-manager-789f6589d5-577rn" Dec 03 11:20:56 crc kubenswrapper[5052]: I1203 11:20:56.007432 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="hostpath-provisioner/csi-hostpathplugin-5tlrr" event={"ID":"60b355dc-6de7-4a73-a2d2-3a0f0befd049","Type":"ContainerStarted","Data":"8f32065c890c444ed1f5e3ae1f1641770b6604b77cb7f6571a937c4c7ec8ff73"} Dec 03 11:20:56 crc kubenswrapper[5052]: I1203 11:20:56.013697 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openshift-multus/cni-sysctl-allowlist-ds-8l9gm" podUID="bf761cb9-e024-4acd-a947-0f8064267243" containerName="kube-multus-additional-cni-plugins" containerID="cri-o://b0103da56f9128a13b28c52c5f8402db976339938594d6eae2a65c5f008b27ef" gracePeriod=30 Dec 03 11:20:56 crc kubenswrapper[5052]: I1203 11:20:56.023872 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-4xqqz\" (UniqueName: \"kubernetes.io/projected/a7228f2a-1d6d-4307-80e7-deebd47d995a-kube-api-access-4xqqz\") pod \"community-operators-pxx88\" (UID: \"a7228f2a-1d6d-4307-80e7-deebd47d995a\") " pod="openshift-marketplace/community-operators-pxx88" Dec 03 11:20:56 crc kubenswrapper[5052]: I1203 11:20:56.026497 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/dc6b87f0-562b-4fa9-85b3-ccdd0867374b-utilities\") pod \"certified-operators-ghqsf\" (UID: \"dc6b87f0-562b-4fa9-85b3-ccdd0867374b\") " pod="openshift-marketplace/certified-operators-ghqsf" Dec 03 11:20:56 crc kubenswrapper[5052]: I1203 11:20:56.026634 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/dc6b87f0-562b-4fa9-85b3-ccdd0867374b-catalog-content\") pod \"certified-operators-ghqsf\" (UID: \"dc6b87f0-562b-4fa9-85b3-ccdd0867374b\") " pod="openshift-marketplace/certified-operators-ghqsf" Dec 03 11:20:56 crc kubenswrapper[5052]: I1203 11:20:56.026825 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z6xw8\" (UniqueName: \"kubernetes.io/projected/dc6b87f0-562b-4fa9-85b3-ccdd0867374b-kube-api-access-z6xw8\") pod \"certified-operators-ghqsf\" (UID: \"dc6b87f0-562b-4fa9-85b3-ccdd0867374b\") " pod="openshift-marketplace/certified-operators-ghqsf" Dec 03 11:20:56 crc kubenswrapper[5052]: I1203 11:20:56.026937 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\") pod \"image-registry-697d97f7c8-p77bx\" (UID: \"f5012254-068c-4360-b310-39ff3d309199\") " pod="openshift-image-registry/image-registry-697d97f7c8-p77bx" Dec 03 11:20:56 crc kubenswrapper[5052]: E1203 11:20:56.027446 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8 podName: nodeName:}" failed. No retries permitted until 2025-12-03 11:20:56.52743017 +0000 UTC m=+47.115671934 (durationBeforeRetry 500ms). Error: MountVolume.MountDevice failed for volume "pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8" (UniqueName: "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8") pod "image-registry-697d97f7c8-p77bx" (UID: "f5012254-068c-4360-b310-39ff3d309199") : kubernetes.io/csi: attacher.MountDevice failed to create newCsiDriverClient: driver name kubevirt.io.hostpath-provisioner not found in the list of registered CSI drivers Dec 03 11:20:56 crc kubenswrapper[5052]: I1203 11:20:56.034870 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/community-operators-pxx88" Dec 03 11:20:56 crc kubenswrapper[5052]: I1203 11:20:56.035862 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-controller-manager/controller-manager-879f6c89f-46rm8" Dec 03 11:20:56 crc kubenswrapper[5052]: I1203 11:20:56.063773 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-58897d9998-7vpns" Dec 03 11:20:56 crc kubenswrapper[5052]: I1203 11:20:56.066787 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-admission-controller-857f4d67dd-q8n58" podStartSLOduration=24.066770334 podStartE2EDuration="24.066770334s" podCreationTimestamp="2025-12-03 11:20:32 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:20:56.063870154 +0000 UTC m=+46.652111938" watchObservedRunningTime="2025-12-03 11:20:56.066770334 +0000 UTC m=+46.655012098" Dec 03 11:20:56 crc kubenswrapper[5052]: I1203 11:20:56.110113 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openshift-marketplace/community-operators-r8txz"] Dec 03 11:20:56 crc kubenswrapper[5052]: I1203 11:20:56.111376 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/community-operators-r8txz" Dec 03 11:20:56 crc kubenswrapper[5052]: I1203 11:20:56.154591 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\") pod \"8f668bae-612b-4b75-9490-919e737c6a3b\" (UID: \"8f668bae-612b-4b75-9490-919e737c6a3b\") " Dec 03 11:20:56 crc kubenswrapper[5052]: I1203 11:20:56.155142 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/dc6b87f0-562b-4fa9-85b3-ccdd0867374b-utilities\") pod \"certified-operators-ghqsf\" (UID: \"dc6b87f0-562b-4fa9-85b3-ccdd0867374b\") " pod="openshift-marketplace/certified-operators-ghqsf" Dec 03 11:20:56 crc kubenswrapper[5052]: I1203 11:20:56.155460 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/dc6b87f0-562b-4fa9-85b3-ccdd0867374b-catalog-content\") pod \"certified-operators-ghqsf\" (UID: \"dc6b87f0-562b-4fa9-85b3-ccdd0867374b\") " pod="openshift-marketplace/certified-operators-ghqsf" Dec 03 11:20:56 crc kubenswrapper[5052]: I1203 11:20:56.155999 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-z6xw8\" (UniqueName: \"kubernetes.io/projected/dc6b87f0-562b-4fa9-85b3-ccdd0867374b-kube-api-access-z6xw8\") pod \"certified-operators-ghqsf\" (UID: \"dc6b87f0-562b-4fa9-85b3-ccdd0867374b\") " pod="openshift-marketplace/certified-operators-ghqsf" Dec 03 11:20:56 crc kubenswrapper[5052]: E1203 11:20:56.157971 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8 podName:8f668bae-612b-4b75-9490-919e737c6a3b nodeName:}" failed. No retries permitted until 2025-12-03 11:20:56.657946846 +0000 UTC m=+47.246188610 (durationBeforeRetry 500ms). Error: UnmountVolume.TearDown failed for volume "pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8" (UniqueName: "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8") pod "8f668bae-612b-4b75-9490-919e737c6a3b" (UID: "8f668bae-612b-4b75-9490-919e737c6a3b") : kubernetes.io/csi: Unmounter.TearDownAt failed to get CSI client: driver name kubevirt.io.hostpath-provisioner not found in the list of registered CSI drivers Dec 03 11:20:56 crc kubenswrapper[5052]: I1203 11:20:56.161382 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/dc6b87f0-562b-4fa9-85b3-ccdd0867374b-utilities\") pod \"certified-operators-ghqsf\" (UID: \"dc6b87f0-562b-4fa9-85b3-ccdd0867374b\") " pod="openshift-marketplace/certified-operators-ghqsf" Dec 03 11:20:56 crc kubenswrapper[5052]: I1203 11:20:56.171946 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/dc6b87f0-562b-4fa9-85b3-ccdd0867374b-catalog-content\") pod \"certified-operators-ghqsf\" (UID: \"dc6b87f0-562b-4fa9-85b3-ccdd0867374b\") " pod="openshift-marketplace/certified-operators-ghqsf" Dec 03 11:20:56 crc kubenswrapper[5052]: I1203 11:20:56.249095 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/community-operators-r8txz"] Dec 03 11:20:56 crc kubenswrapper[5052]: I1203 11:20:56.255107 5052 patch_prober.go:28] interesting pod/router-default-5444994796-rtl8t container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-http failed: reason withheld Dec 03 11:20:56 crc kubenswrapper[5052]: [-]has-synced failed: reason withheld Dec 03 11:20:56 crc kubenswrapper[5052]: [+]process-running ok Dec 03 11:20:56 crc kubenswrapper[5052]: healthz check failed Dec 03 11:20:56 crc kubenswrapper[5052]: I1203 11:20:56.255174 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-5444994796-rtl8t" podUID="48040744-7fc1-482e-9b68-3a904ba03a8d" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500" Dec 03 11:20:56 crc kubenswrapper[5052]: I1203 11:20:56.258354 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/f1ea0ffb-1062-4713-908b-55c0b07c49ee-catalog-content\") pod \"community-operators-r8txz\" (UID: \"f1ea0ffb-1062-4713-908b-55c0b07c49ee\") " pod="openshift-marketplace/community-operators-r8txz" Dec 03 11:20:56 crc kubenswrapper[5052]: I1203 11:20:56.258409 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g4qvc\" (UniqueName: \"kubernetes.io/projected/f1ea0ffb-1062-4713-908b-55c0b07c49ee-kube-api-access-g4qvc\") pod \"community-operators-r8txz\" (UID: \"f1ea0ffb-1062-4713-908b-55c0b07c49ee\") " pod="openshift-marketplace/community-operators-r8txz" Dec 03 11:20:56 crc kubenswrapper[5052]: I1203 11:20:56.258462 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/f1ea0ffb-1062-4713-908b-55c0b07c49ee-utilities\") pod \"community-operators-r8txz\" (UID: \"f1ea0ffb-1062-4713-908b-55c0b07c49ee\") " pod="openshift-marketplace/community-operators-r8txz" Dec 03 11:20:56 crc kubenswrapper[5052]: I1203 11:20:56.258492 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\") pod \"image-registry-697d97f7c8-p77bx\" (UID: \"f5012254-068c-4360-b310-39ff3d309199\") " pod="openshift-image-registry/image-registry-697d97f7c8-p77bx" Dec 03 11:20:56 crc kubenswrapper[5052]: E1203 11:20:56.258973 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8 podName: nodeName:}" failed. No retries permitted until 2025-12-03 11:20:56.75895328 +0000 UTC m=+47.347195054 (durationBeforeRetry 500ms). Error: MountVolume.MountDevice failed for volume "pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8" (UniqueName: "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8") pod "image-registry-697d97f7c8-p77bx" (UID: "f5012254-068c-4360-b310-39ff3d309199") : kubernetes.io/csi: attacher.MountDevice failed to create newCsiDriverClient: driver name kubevirt.io.hostpath-provisioner not found in the list of registered CSI drivers Dec 03 11:20:56 crc kubenswrapper[5052]: I1203 11:20:56.260869 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/dns-default-6m2jn" podStartSLOduration=11.260849632 podStartE2EDuration="11.260849632s" podCreationTimestamp="2025-12-03 11:20:45 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:20:56.227174774 +0000 UTC m=+46.815416528" watchObservedRunningTime="2025-12-03 11:20:56.260849632 +0000 UTC m=+46.849091396" Dec 03 11:20:56 crc kubenswrapper[5052]: I1203 11:20:56.296980 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-z6xw8\" (UniqueName: \"kubernetes.io/projected/dc6b87f0-562b-4fa9-85b3-ccdd0867374b-kube-api-access-z6xw8\") pod \"certified-operators-ghqsf\" (UID: \"dc6b87f0-562b-4fa9-85b3-ccdd0867374b\") " pod="openshift-marketplace/certified-operators-ghqsf" Dec 03 11:20:56 crc kubenswrapper[5052]: I1203 11:20:56.309154 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openshift-marketplace/certified-operators-99svv"] Dec 03 11:20:56 crc kubenswrapper[5052]: I1203 11:20:56.310490 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/certified-operators-99svv" Dec 03 11:20:56 crc kubenswrapper[5052]: I1203 11:20:56.323261 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/certified-operators-99svv"] Dec 03 11:20:56 crc kubenswrapper[5052]: I1203 11:20:56.361813 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\") pod \"8f668bae-612b-4b75-9490-919e737c6a3b\" (UID: \"8f668bae-612b-4b75-9490-919e737c6a3b\") " Dec 03 11:20:56 crc kubenswrapper[5052]: I1203 11:20:56.362036 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/3b1b1a53-eff1-4339-a82e-9b8400dd5604-utilities\") pod \"certified-operators-99svv\" (UID: \"3b1b1a53-eff1-4339-a82e-9b8400dd5604\") " pod="openshift-marketplace/certified-operators-99svv" Dec 03 11:20:56 crc kubenswrapper[5052]: I1203 11:20:56.362090 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/f1ea0ffb-1062-4713-908b-55c0b07c49ee-catalog-content\") pod \"community-operators-r8txz\" (UID: \"f1ea0ffb-1062-4713-908b-55c0b07c49ee\") " pod="openshift-marketplace/community-operators-r8txz" Dec 03 11:20:56 crc kubenswrapper[5052]: I1203 11:20:56.362114 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/3b1b1a53-eff1-4339-a82e-9b8400dd5604-catalog-content\") pod \"certified-operators-99svv\" (UID: \"3b1b1a53-eff1-4339-a82e-9b8400dd5604\") " pod="openshift-marketplace/certified-operators-99svv" Dec 03 11:20:56 crc kubenswrapper[5052]: I1203 11:20:56.362140 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-g4qvc\" (UniqueName: \"kubernetes.io/projected/f1ea0ffb-1062-4713-908b-55c0b07c49ee-kube-api-access-g4qvc\") pod \"community-operators-r8txz\" (UID: \"f1ea0ffb-1062-4713-908b-55c0b07c49ee\") " pod="openshift-marketplace/community-operators-r8txz" Dec 03 11:20:56 crc kubenswrapper[5052]: I1203 11:20:56.362178 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zn4pm\" (UniqueName: \"kubernetes.io/projected/3b1b1a53-eff1-4339-a82e-9b8400dd5604-kube-api-access-zn4pm\") pod \"certified-operators-99svv\" (UID: \"3b1b1a53-eff1-4339-a82e-9b8400dd5604\") " pod="openshift-marketplace/certified-operators-99svv" Dec 03 11:20:56 crc kubenswrapper[5052]: I1203 11:20:56.362210 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/f1ea0ffb-1062-4713-908b-55c0b07c49ee-utilities\") pod \"community-operators-r8txz\" (UID: \"f1ea0ffb-1062-4713-908b-55c0b07c49ee\") " pod="openshift-marketplace/community-operators-r8txz" Dec 03 11:20:56 crc kubenswrapper[5052]: E1203 11:20:56.362653 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8 podName:8f668bae-612b-4b75-9490-919e737c6a3b nodeName:}" failed. No retries permitted until 2025-12-03 11:20:56.862614377 +0000 UTC m=+47.450856141 (durationBeforeRetry 500ms). Error: UnmountVolume.TearDown failed for volume "pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8" (UniqueName: "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8") pod "8f668bae-612b-4b75-9490-919e737c6a3b" (UID: "8f668bae-612b-4b75-9490-919e737c6a3b") : kubernetes.io/csi: Unmounter.TearDownAt failed to get CSI client: driver name kubevirt.io.hostpath-provisioner not found in the list of registered CSI drivers Dec 03 11:20:56 crc kubenswrapper[5052]: I1203 11:20:56.363186 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/f1ea0ffb-1062-4713-908b-55c0b07c49ee-utilities\") pod \"community-operators-r8txz\" (UID: \"f1ea0ffb-1062-4713-908b-55c0b07c49ee\") " pod="openshift-marketplace/community-operators-r8txz" Dec 03 11:20:56 crc kubenswrapper[5052]: I1203 11:20:56.363543 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/f1ea0ffb-1062-4713-908b-55c0b07c49ee-catalog-content\") pod \"community-operators-r8txz\" (UID: \"f1ea0ffb-1062-4713-908b-55c0b07c49ee\") " pod="openshift-marketplace/community-operators-r8txz" Dec 03 11:20:56 crc kubenswrapper[5052]: I1203 11:20:56.464098 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/3b1b1a53-eff1-4339-a82e-9b8400dd5604-utilities\") pod \"certified-operators-99svv\" (UID: \"3b1b1a53-eff1-4339-a82e-9b8400dd5604\") " pod="openshift-marketplace/certified-operators-99svv" Dec 03 11:20:56 crc kubenswrapper[5052]: I1203 11:20:56.464436 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/3b1b1a53-eff1-4339-a82e-9b8400dd5604-catalog-content\") pod \"certified-operators-99svv\" (UID: \"3b1b1a53-eff1-4339-a82e-9b8400dd5604\") " pod="openshift-marketplace/certified-operators-99svv" Dec 03 11:20:56 crc kubenswrapper[5052]: I1203 11:20:56.464596 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-zn4pm\" (UniqueName: \"kubernetes.io/projected/3b1b1a53-eff1-4339-a82e-9b8400dd5604-kube-api-access-zn4pm\") pod \"certified-operators-99svv\" (UID: \"3b1b1a53-eff1-4339-a82e-9b8400dd5604\") " pod="openshift-marketplace/certified-operators-99svv" Dec 03 11:20:56 crc kubenswrapper[5052]: I1203 11:20:56.464620 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/3b1b1a53-eff1-4339-a82e-9b8400dd5604-utilities\") pod \"certified-operators-99svv\" (UID: \"3b1b1a53-eff1-4339-a82e-9b8400dd5604\") " pod="openshift-marketplace/certified-operators-99svv" Dec 03 11:20:56 crc kubenswrapper[5052]: I1203 11:20:56.464817 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/3b1b1a53-eff1-4339-a82e-9b8400dd5604-catalog-content\") pod \"certified-operators-99svv\" (UID: \"3b1b1a53-eff1-4339-a82e-9b8400dd5604\") " pod="openshift-marketplace/certified-operators-99svv" Dec 03 11:20:56 crc kubenswrapper[5052]: I1203 11:20:56.465106 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\") pod \"image-registry-697d97f7c8-p77bx\" (UID: \"f5012254-068c-4360-b310-39ff3d309199\") " pod="openshift-image-registry/image-registry-697d97f7c8-p77bx" Dec 03 11:20:56 crc kubenswrapper[5052]: E1203 11:20:56.465463 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8 podName: nodeName:}" failed. No retries permitted until 2025-12-03 11:20:56.965446431 +0000 UTC m=+47.553688195 (durationBeforeRetry 500ms). Error: MountVolume.MountDevice failed for volume "pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8" (UniqueName: "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8") pod "image-registry-697d97f7c8-p77bx" (UID: "f5012254-068c-4360-b310-39ff3d309199") : kubernetes.io/csi: attacher.MountDevice failed to create newCsiDriverClient: driver name kubevirt.io.hostpath-provisioner not found in the list of registered CSI drivers Dec 03 11:20:56 crc kubenswrapper[5052]: I1203 11:20:56.550385 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/certified-operators-ghqsf" Dec 03 11:20:56 crc kubenswrapper[5052]: I1203 11:20:56.567453 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\") pod \"8f668bae-612b-4b75-9490-919e737c6a3b\" (UID: \"8f668bae-612b-4b75-9490-919e737c6a3b\") " Dec 03 11:20:56 crc kubenswrapper[5052]: E1203 11:20:56.567972 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8 podName:8f668bae-612b-4b75-9490-919e737c6a3b nodeName:}" failed. No retries permitted until 2025-12-03 11:20:57.067951906 +0000 UTC m=+47.656193670 (durationBeforeRetry 500ms). Error: UnmountVolume.TearDown failed for volume "pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8" (UniqueName: "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8") pod "8f668bae-612b-4b75-9490-919e737c6a3b" (UID: "8f668bae-612b-4b75-9490-919e737c6a3b") : kubernetes.io/csi: Unmounter.TearDownAt failed to get CSI client: driver name kubevirt.io.hostpath-provisioner not found in the list of registered CSI drivers Dec 03 11:20:56 crc kubenswrapper[5052]: I1203 11:20:56.603112 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-zn4pm\" (UniqueName: \"kubernetes.io/projected/3b1b1a53-eff1-4339-a82e-9b8400dd5604-kube-api-access-zn4pm\") pod \"certified-operators-99svv\" (UID: \"3b1b1a53-eff1-4339-a82e-9b8400dd5604\") " pod="openshift-marketplace/certified-operators-99svv" Dec 03 11:20:56 crc kubenswrapper[5052]: I1203 11:20:56.631580 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-operator-lifecycle-manager/package-server-manager-789f6589d5-577rn" podStartSLOduration=24.631550698 podStartE2EDuration="24.631550698s" podCreationTimestamp="2025-12-03 11:20:32 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:20:56.629994565 +0000 UTC m=+47.218236329" watchObservedRunningTime="2025-12-03 11:20:56.631550698 +0000 UTC m=+47.219792462" Dec 03 11:20:56 crc kubenswrapper[5052]: I1203 11:20:56.636448 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/community-operators-pxx88"] Dec 03 11:20:56 crc kubenswrapper[5052]: I1203 11:20:56.642210 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-g4qvc\" (UniqueName: \"kubernetes.io/projected/f1ea0ffb-1062-4713-908b-55c0b07c49ee-kube-api-access-g4qvc\") pod \"community-operators-r8txz\" (UID: \"f1ea0ffb-1062-4713-908b-55c0b07c49ee\") " pod="openshift-marketplace/community-operators-r8txz" Dec 03 11:20:56 crc kubenswrapper[5052]: I1203 11:20:56.648878 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/certified-operators-99svv" Dec 03 11:20:56 crc kubenswrapper[5052]: I1203 11:20:56.669496 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\") pod \"image-registry-697d97f7c8-p77bx\" (UID: \"f5012254-068c-4360-b310-39ff3d309199\") " pod="openshift-image-registry/image-registry-697d97f7c8-p77bx" Dec 03 11:20:56 crc kubenswrapper[5052]: E1203 11:20:56.669936 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8 podName: nodeName:}" failed. No retries permitted until 2025-12-03 11:20:57.169915416 +0000 UTC m=+47.758157190 (durationBeforeRetry 500ms). Error: MountVolume.MountDevice failed for volume "pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8" (UniqueName: "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8") pod "image-registry-697d97f7c8-p77bx" (UID: "f5012254-068c-4360-b310-39ff3d309199") : kubernetes.io/csi: attacher.MountDevice failed to create newCsiDriverClient: driver name kubevirt.io.hostpath-provisioner not found in the list of registered CSI drivers Dec 03 11:20:56 crc kubenswrapper[5052]: I1203 11:20:56.778415 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\") pod \"8f668bae-612b-4b75-9490-919e737c6a3b\" (UID: \"8f668bae-612b-4b75-9490-919e737c6a3b\") " Dec 03 11:20:56 crc kubenswrapper[5052]: E1203 11:20:56.778800 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8 podName:8f668bae-612b-4b75-9490-919e737c6a3b nodeName:}" failed. No retries permitted until 2025-12-03 11:20:57.278755775 +0000 UTC m=+47.866997549 (durationBeforeRetry 500ms). Error: UnmountVolume.TearDown failed for volume "pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8" (UniqueName: "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8") pod "8f668bae-612b-4b75-9490-919e737c6a3b" (UID: "8f668bae-612b-4b75-9490-919e737c6a3b") : kubernetes.io/csi: Unmounter.TearDownAt failed to get CSI client: driver name kubevirt.io.hostpath-provisioner not found in the list of registered CSI drivers Dec 03 11:20:56 crc kubenswrapper[5052]: I1203 11:20:56.779178 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\") pod \"image-registry-697d97f7c8-p77bx\" (UID: \"f5012254-068c-4360-b310-39ff3d309199\") " pod="openshift-image-registry/image-registry-697d97f7c8-p77bx" Dec 03 11:20:56 crc kubenswrapper[5052]: E1203 11:20:56.779669 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8 podName: nodeName:}" failed. No retries permitted until 2025-12-03 11:20:57.27965291 +0000 UTC m=+47.867894674 (durationBeforeRetry 500ms). Error: MountVolume.MountDevice failed for volume "pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8" (UniqueName: "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8") pod "image-registry-697d97f7c8-p77bx" (UID: "f5012254-068c-4360-b310-39ff3d309199") : kubernetes.io/csi: attacher.MountDevice failed to create newCsiDriverClient: driver name kubevirt.io.hostpath-provisioner not found in the list of registered CSI drivers Dec 03 11:20:56 crc kubenswrapper[5052]: I1203 11:20:56.880272 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\") pod \"8f668bae-612b-4b75-9490-919e737c6a3b\" (UID: \"8f668bae-612b-4b75-9490-919e737c6a3b\") " Dec 03 11:20:56 crc kubenswrapper[5052]: E1203 11:20:56.880580 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8 podName:8f668bae-612b-4b75-9490-919e737c6a3b nodeName:}" failed. No retries permitted until 2025-12-03 11:20:57.38054341 +0000 UTC m=+47.968785174 (durationBeforeRetry 500ms). Error: UnmountVolume.TearDown failed for volume "pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8" (UniqueName: "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8") pod "8f668bae-612b-4b75-9490-919e737c6a3b" (UID: "8f668bae-612b-4b75-9490-919e737c6a3b") : kubernetes.io/csi: Unmounter.TearDownAt failed to get CSI client: driver name kubevirt.io.hostpath-provisioner not found in the list of registered CSI drivers Dec 03 11:20:56 crc kubenswrapper[5052]: I1203 11:20:56.880647 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\") pod \"image-registry-697d97f7c8-p77bx\" (UID: \"f5012254-068c-4360-b310-39ff3d309199\") " pod="openshift-image-registry/image-registry-697d97f7c8-p77bx" Dec 03 11:20:56 crc kubenswrapper[5052]: I1203 11:20:56.880698 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/community-operators-r8txz" Dec 03 11:20:56 crc kubenswrapper[5052]: E1203 11:20:56.881129 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8 podName: nodeName:}" failed. No retries permitted until 2025-12-03 11:20:57.381121686 +0000 UTC m=+47.969363450 (durationBeforeRetry 500ms). Error: MountVolume.MountDevice failed for volume "pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8" (UniqueName: "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8") pod "image-registry-697d97f7c8-p77bx" (UID: "f5012254-068c-4360-b310-39ff3d309199") : kubernetes.io/csi: attacher.MountDevice failed to create newCsiDriverClient: driver name kubevirt.io.hostpath-provisioner not found in the list of registered CSI drivers Dec 03 11:20:56 crc kubenswrapper[5052]: I1203 11:20:56.982928 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\") pod \"8f668bae-612b-4b75-9490-919e737c6a3b\" (UID: \"8f668bae-612b-4b75-9490-919e737c6a3b\") " Dec 03 11:20:56 crc kubenswrapper[5052]: E1203 11:20:56.983888 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8 podName:8f668bae-612b-4b75-9490-919e737c6a3b nodeName:}" failed. No retries permitted until 2025-12-03 11:20:57.483866088 +0000 UTC m=+48.072107852 (durationBeforeRetry 500ms). Error: UnmountVolume.TearDown failed for volume "pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8" (UniqueName: "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8") pod "8f668bae-612b-4b75-9490-919e737c6a3b" (UID: "8f668bae-612b-4b75-9490-919e737c6a3b") : kubernetes.io/csi: Unmounter.TearDownAt failed to get CSI client: driver name kubevirt.io.hostpath-provisioner not found in the list of registered CSI drivers Dec 03 11:20:57 crc kubenswrapper[5052]: I1203 11:20:57.063708 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/community-operators-pxx88" event={"ID":"a7228f2a-1d6d-4307-80e7-deebd47d995a","Type":"ContainerStarted","Data":"cdba1e921c314ba078fb0f85a22db6f04bafcb29d83bb480e5669a56a272b521"} Dec 03 11:20:57 crc kubenswrapper[5052]: I1203 11:20:57.085534 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\") pod \"image-registry-697d97f7c8-p77bx\" (UID: \"f5012254-068c-4360-b310-39ff3d309199\") " pod="openshift-image-registry/image-registry-697d97f7c8-p77bx" Dec 03 11:20:57 crc kubenswrapper[5052]: E1203 11:20:57.085886 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8 podName: nodeName:}" failed. No retries permitted until 2025-12-03 11:20:57.585871729 +0000 UTC m=+48.174113493 (durationBeforeRetry 500ms). Error: MountVolume.MountDevice failed for volume "pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8" (UniqueName: "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8") pod "image-registry-697d97f7c8-p77bx" (UID: "f5012254-068c-4360-b310-39ff3d309199") : kubernetes.io/csi: attacher.MountDevice failed to create newCsiDriverClient: driver name kubevirt.io.hostpath-provisioner not found in the list of registered CSI drivers Dec 03 11:20:57 crc kubenswrapper[5052]: I1203 11:20:57.177222 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="hostpath-provisioner/csi-hostpathplugin-5tlrr" event={"ID":"60b355dc-6de7-4a73-a2d2-3a0f0befd049","Type":"ContainerStarted","Data":"8f4435134434b007197e9a3d0b8f2f05c9055c3d30e4c87c29b44a1d61c85f57"} Dec 03 11:20:57 crc kubenswrapper[5052]: I1203 11:20:57.183304 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-apiserver/apiserver-76f77b778f-dmvst" event={"ID":"41016a7b-cc79-4f32-860c-cb75e405cf42","Type":"ContainerStarted","Data":"e98de3f62c6e71fa5a88afe759a7bfbab280cfecea975e275be00add9ba2d6cd"} Dec 03 11:20:57 crc kubenswrapper[5052]: I1203 11:20:57.195463 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\") pod \"8f668bae-612b-4b75-9490-919e737c6a3b\" (UID: \"8f668bae-612b-4b75-9490-919e737c6a3b\") " Dec 03 11:20:57 crc kubenswrapper[5052]: E1203 11:20:57.196007 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8 podName:8f668bae-612b-4b75-9490-919e737c6a3b nodeName:}" failed. No retries permitted until 2025-12-03 11:20:57.695966513 +0000 UTC m=+48.284208337 (durationBeforeRetry 500ms). Error: UnmountVolume.TearDown failed for volume "pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8" (UniqueName: "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8") pod "8f668bae-612b-4b75-9490-919e737c6a3b" (UID: "8f668bae-612b-4b75-9490-919e737c6a3b") : kubernetes.io/csi: Unmounter.TearDownAt failed to get CSI client: driver name kubevirt.io.hostpath-provisioner not found in the list of registered CSI drivers Dec 03 11:20:57 crc kubenswrapper[5052]: I1203 11:20:57.196623 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\") pod \"image-registry-697d97f7c8-p77bx\" (UID: \"f5012254-068c-4360-b310-39ff3d309199\") " pod="openshift-image-registry/image-registry-697d97f7c8-p77bx" Dec 03 11:20:57 crc kubenswrapper[5052]: E1203 11:20:57.200076 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8 podName: nodeName:}" failed. No retries permitted until 2025-12-03 11:20:57.700048305 +0000 UTC m=+48.288290239 (durationBeforeRetry 500ms). Error: MountVolume.MountDevice failed for volume "pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8" (UniqueName: "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8") pod "image-registry-697d97f7c8-p77bx" (UID: "f5012254-068c-4360-b310-39ff3d309199") : kubernetes.io/csi: attacher.MountDevice failed to create newCsiDriverClient: driver name kubevirt.io.hostpath-provisioner not found in the list of registered CSI drivers Dec 03 11:20:57 crc kubenswrapper[5052]: I1203 11:20:57.212487 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-config-operator/openshift-config-operator-7777fb866f-bkztv" Dec 03 11:20:57 crc kubenswrapper[5052]: I1203 11:20:57.285882 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-apiserver/apiserver-76f77b778f-dmvst" podStartSLOduration=25.28585371 podStartE2EDuration="25.28585371s" podCreationTimestamp="2025-12-03 11:20:32 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:20:57.226889495 +0000 UTC m=+47.815131259" watchObservedRunningTime="2025-12-03 11:20:57.28585371 +0000 UTC m=+47.874095474" Dec 03 11:20:57 crc kubenswrapper[5052]: I1203 11:20:57.286746 5052 patch_prober.go:28] interesting pod/router-default-5444994796-rtl8t container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-http failed: reason withheld Dec 03 11:20:57 crc kubenswrapper[5052]: [-]has-synced failed: reason withheld Dec 03 11:20:57 crc kubenswrapper[5052]: [+]process-running ok Dec 03 11:20:57 crc kubenswrapper[5052]: healthz check failed Dec 03 11:20:57 crc kubenswrapper[5052]: I1203 11:20:57.286813 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/certified-operators-ghqsf"] Dec 03 11:20:57 crc kubenswrapper[5052]: I1203 11:20:57.286863 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-5444994796-rtl8t" podUID="48040744-7fc1-482e-9b68-3a904ba03a8d" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500" Dec 03 11:20:57 crc kubenswrapper[5052]: I1203 11:20:57.304700 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/certified-operators-99svv"] Dec 03 11:20:57 crc kubenswrapper[5052]: I1203 11:20:57.320028 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\") pod \"8f668bae-612b-4b75-9490-919e737c6a3b\" (UID: \"8f668bae-612b-4b75-9490-919e737c6a3b\") " Dec 03 11:20:57 crc kubenswrapper[5052]: E1203 11:20:57.320939 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8 podName:8f668bae-612b-4b75-9490-919e737c6a3b nodeName:}" failed. No retries permitted until 2025-12-03 11:20:57.820922817 +0000 UTC m=+48.409164581 (durationBeforeRetry 500ms). Error: UnmountVolume.TearDown failed for volume "pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8" (UniqueName: "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8") pod "8f668bae-612b-4b75-9490-919e737c6a3b" (UID: "8f668bae-612b-4b75-9490-919e737c6a3b") : kubernetes.io/csi: Unmounter.TearDownAt failed to get CSI client: driver name kubevirt.io.hostpath-provisioner not found in the list of registered CSI drivers Dec 03 11:20:57 crc kubenswrapper[5052]: I1203 11:20:57.321097 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\") pod \"image-registry-697d97f7c8-p77bx\" (UID: \"f5012254-068c-4360-b310-39ff3d309199\") " pod="openshift-image-registry/image-registry-697d97f7c8-p77bx" Dec 03 11:20:57 crc kubenswrapper[5052]: E1203 11:20:57.322350 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8 podName: nodeName:}" failed. No retries permitted until 2025-12-03 11:20:57.822342066 +0000 UTC m=+48.410583830 (durationBeforeRetry 500ms). Error: MountVolume.MountDevice failed for volume "pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8" (UniqueName: "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8") pod "image-registry-697d97f7c8-p77bx" (UID: "f5012254-068c-4360-b310-39ff3d309199") : kubernetes.io/csi: attacher.MountDevice failed to create newCsiDriverClient: driver name kubevirt.io.hostpath-provisioner not found in the list of registered CSI drivers Dec 03 11:20:57 crc kubenswrapper[5052]: I1203 11:20:57.425466 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\") pod \"8f668bae-612b-4b75-9490-919e737c6a3b\" (UID: \"8f668bae-612b-4b75-9490-919e737c6a3b\") " Dec 03 11:20:57 crc kubenswrapper[5052]: E1203 11:20:57.425718 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8 podName:8f668bae-612b-4b75-9490-919e737c6a3b nodeName:}" failed. No retries permitted until 2025-12-03 11:20:57.925684904 +0000 UTC m=+48.513926668 (durationBeforeRetry 500ms). Error: UnmountVolume.TearDown failed for volume "pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8" (UniqueName: "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8") pod "8f668bae-612b-4b75-9490-919e737c6a3b" (UID: "8f668bae-612b-4b75-9490-919e737c6a3b") : kubernetes.io/csi: Unmounter.TearDownAt failed to get CSI client: driver name kubevirt.io.hostpath-provisioner not found in the list of registered CSI drivers Dec 03 11:20:57 crc kubenswrapper[5052]: I1203 11:20:57.425789 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\") pod \"image-registry-697d97f7c8-p77bx\" (UID: \"f5012254-068c-4360-b310-39ff3d309199\") " pod="openshift-image-registry/image-registry-697d97f7c8-p77bx" Dec 03 11:20:57 crc kubenswrapper[5052]: E1203 11:20:57.426476 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8 podName: nodeName:}" failed. No retries permitted until 2025-12-03 11:20:57.926466875 +0000 UTC m=+48.514708639 (durationBeforeRetry 500ms). Error: MountVolume.MountDevice failed for volume "pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8" (UniqueName: "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8") pod "image-registry-697d97f7c8-p77bx" (UID: "f5012254-068c-4360-b310-39ff3d309199") : kubernetes.io/csi: attacher.MountDevice failed to create newCsiDriverClient: driver name kubevirt.io.hostpath-provisioner not found in the list of registered CSI drivers Dec 03 11:20:57 crc kubenswrapper[5052]: I1203 11:20:57.528903 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\") pod \"8f668bae-612b-4b75-9490-919e737c6a3b\" (UID: \"8f668bae-612b-4b75-9490-919e737c6a3b\") " Dec 03 11:20:57 crc kubenswrapper[5052]: E1203 11:20:57.529503 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8 podName:8f668bae-612b-4b75-9490-919e737c6a3b nodeName:}" failed. No retries permitted until 2025-12-03 11:20:58.029481224 +0000 UTC m=+48.617722988 (durationBeforeRetry 500ms). Error: UnmountVolume.TearDown failed for volume "pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8" (UniqueName: "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8") pod "8f668bae-612b-4b75-9490-919e737c6a3b" (UID: "8f668bae-612b-4b75-9490-919e737c6a3b") : kubernetes.io/csi: Unmounter.TearDownAt failed to get CSI client: driver name kubevirt.io.hostpath-provisioner not found in the list of registered CSI drivers Dec 03 11:20:57 crc kubenswrapper[5052]: I1203 11:20:57.581899 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/community-operators-r8txz"] Dec 03 11:20:57 crc kubenswrapper[5052]: W1203 11:20:57.614995 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1ea0ffb_1062_4713_908b_55c0b07c49ee.slice/crio-f0c85f21f278dcf751b76f25d9cea56b5c872ef44a26cff908d2223a33ee9fe0 WatchSource:0}: Error finding container f0c85f21f278dcf751b76f25d9cea56b5c872ef44a26cff908d2223a33ee9fe0: Status 404 returned error can't find the container with id f0c85f21f278dcf751b76f25d9cea56b5c872ef44a26cff908d2223a33ee9fe0 Dec 03 11:20:57 crc kubenswrapper[5052]: I1203 11:20:57.630408 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\") pod \"image-registry-697d97f7c8-p77bx\" (UID: \"f5012254-068c-4360-b310-39ff3d309199\") " pod="openshift-image-registry/image-registry-697d97f7c8-p77bx" Dec 03 11:20:57 crc kubenswrapper[5052]: E1203 11:20:57.630916 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8 podName: nodeName:}" failed. No retries permitted until 2025-12-03 11:20:58.130899049 +0000 UTC m=+48.719140813 (durationBeforeRetry 500ms). Error: MountVolume.MountDevice failed for volume "pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8" (UniqueName: "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8") pod "image-registry-697d97f7c8-p77bx" (UID: "f5012254-068c-4360-b310-39ff3d309199") : kubernetes.io/csi: attacher.MountDevice failed to create newCsiDriverClient: driver name kubevirt.io.hostpath-provisioner not found in the list of registered CSI drivers Dec 03 11:20:57 crc kubenswrapper[5052]: I1203 11:20:57.732342 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\") pod \"8f668bae-612b-4b75-9490-919e737c6a3b\" (UID: \"8f668bae-612b-4b75-9490-919e737c6a3b\") " Dec 03 11:20:57 crc kubenswrapper[5052]: E1203 11:20:57.732582 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8 podName:8f668bae-612b-4b75-9490-919e737c6a3b nodeName:}" failed. No retries permitted until 2025-12-03 11:20:58.2325425 +0000 UTC m=+48.820784264 (durationBeforeRetry 500ms). Error: UnmountVolume.TearDown failed for volume "pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8" (UniqueName: "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8") pod "8f668bae-612b-4b75-9490-919e737c6a3b" (UID: "8f668bae-612b-4b75-9490-919e737c6a3b") : kubernetes.io/csi: Unmounter.TearDownAt failed to get CSI client: driver name kubevirt.io.hostpath-provisioner not found in the list of registered CSI drivers Dec 03 11:20:57 crc kubenswrapper[5052]: I1203 11:20:57.732753 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\") pod \"image-registry-697d97f7c8-p77bx\" (UID: \"f5012254-068c-4360-b310-39ff3d309199\") " pod="openshift-image-registry/image-registry-697d97f7c8-p77bx" Dec 03 11:20:57 crc kubenswrapper[5052]: E1203 11:20:57.733234 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8 podName: nodeName:}" failed. No retries permitted until 2025-12-03 11:20:58.233217439 +0000 UTC m=+48.821459203 (durationBeforeRetry 500ms). Error: MountVolume.MountDevice failed for volume "pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8" (UniqueName: "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8") pod "image-registry-697d97f7c8-p77bx" (UID: "f5012254-068c-4360-b310-39ff3d309199") : kubernetes.io/csi: attacher.MountDevice failed to create newCsiDriverClient: driver name kubevirt.io.hostpath-provisioner not found in the list of registered CSI drivers Dec 03 11:20:57 crc kubenswrapper[5052]: I1203 11:20:57.768321 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-oauth-apiserver/apiserver-7bbb656c7d-86vx8" Dec 03 11:20:57 crc kubenswrapper[5052]: I1203 11:20:57.775922 5052 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/kubevirt.io.hostpath-provisioner-reg.sock" Dec 03 11:20:57 crc kubenswrapper[5052]: I1203 11:20:57.778784 5052 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/kubevirt.io.hostpath-provisioner-reg.sock","Timestamp":"2025-12-03T11:20:57.775952887Z","Handler":null,"Name":""} Dec 03 11:20:57 crc kubenswrapper[5052]: I1203 11:20:57.794985 5052 csi_plugin.go:100] kubernetes.io/csi: Trying to validate a new CSI Driver with name: kubevirt.io.hostpath-provisioner endpoint: /var/lib/kubelet/plugins/csi-hostpath/csi.sock versions: 1.0.0 Dec 03 11:20:57 crc kubenswrapper[5052]: I1203 11:20:57.795069 5052 csi_plugin.go:113] kubernetes.io/csi: Register new plugin with name: kubevirt.io.hostpath-provisioner at endpoint: /var/lib/kubelet/plugins/csi-hostpath/csi.sock Dec 03 11:20:57 crc kubenswrapper[5052]: I1203 11:20:57.798432 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-oauth-apiserver/apiserver-7bbb656c7d-86vx8" Dec 03 11:20:57 crc kubenswrapper[5052]: I1203 11:20:57.834080 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\") pod \"8f668bae-612b-4b75-9490-919e737c6a3b\" (UID: \"8f668bae-612b-4b75-9490-919e737c6a3b\") " Dec 03 11:20:57 crc kubenswrapper[5052]: I1203 11:20:57.839872 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8" (OuterVolumeSpecName: "pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8") pod "8f668bae-612b-4b75-9490-919e737c6a3b" (UID: "8f668bae-612b-4b75-9490-919e737c6a3b"). InnerVolumeSpecName "pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8". PluginName "kubernetes.io/csi", VolumeGidValue "" Dec 03 11:20:57 crc kubenswrapper[5052]: I1203 11:20:57.895614 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openshift-marketplace/redhat-marketplace-gqhzd"] Dec 03 11:20:57 crc kubenswrapper[5052]: I1203 11:20:57.896870 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/redhat-marketplace-gqhzd" Dec 03 11:20:57 crc kubenswrapper[5052]: I1203 11:20:57.900301 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-marketplace"/"redhat-marketplace-dockercfg-x2ctb" Dec 03 11:20:57 crc kubenswrapper[5052]: I1203 11:20:57.909712 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/redhat-marketplace-gqhzd"] Dec 03 11:20:57 crc kubenswrapper[5052]: I1203 11:20:57.936311 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\") pod \"image-registry-697d97f7c8-p77bx\" (UID: \"f5012254-068c-4360-b310-39ff3d309199\") " pod="openshift-image-registry/image-registry-697d97f7c8-p77bx" Dec 03 11:20:57 crc kubenswrapper[5052]: I1203 11:20:57.936405 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/355008cc-8721-44c4-a1e0-3714b0086286-catalog-content\") pod \"redhat-marketplace-gqhzd\" (UID: \"355008cc-8721-44c4-a1e0-3714b0086286\") " pod="openshift-marketplace/redhat-marketplace-gqhzd" Dec 03 11:20:57 crc kubenswrapper[5052]: I1203 11:20:57.936430 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/355008cc-8721-44c4-a1e0-3714b0086286-utilities\") pod \"redhat-marketplace-gqhzd\" (UID: \"355008cc-8721-44c4-a1e0-3714b0086286\") " pod="openshift-marketplace/redhat-marketplace-gqhzd" Dec 03 11:20:57 crc kubenswrapper[5052]: I1203 11:20:57.936446 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qdx45\" (UniqueName: \"kubernetes.io/projected/355008cc-8721-44c4-a1e0-3714b0086286-kube-api-access-qdx45\") pod \"redhat-marketplace-gqhzd\" (UID: \"355008cc-8721-44c4-a1e0-3714b0086286\") " pod="openshift-marketplace/redhat-marketplace-gqhzd" Dec 03 11:20:57 crc kubenswrapper[5052]: I1203 11:20:57.957356 5052 csi_attacher.go:380] kubernetes.io/csi: attacher.MountDevice STAGE_UNSTAGE_VOLUME capability not set. Skipping MountDevice... Dec 03 11:20:57 crc kubenswrapper[5052]: I1203 11:20:57.957969 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\") pod \"image-registry-697d97f7c8-p77bx\" (UID: \"f5012254-068c-4360-b310-39ff3d309199\") device mount path \"/var/lib/kubelet/plugins/kubernetes.io/csi/kubevirt.io.hostpath-provisioner/1f4776af88835e41c12b831b4c9fed40233456d14189815a54dbe7f892fc1983/globalmount\"" pod="openshift-image-registry/image-registry-697d97f7c8-p77bx" Dec 03 11:20:58 crc kubenswrapper[5052]: I1203 11:20:58.037383 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/355008cc-8721-44c4-a1e0-3714b0086286-catalog-content\") pod \"redhat-marketplace-gqhzd\" (UID: \"355008cc-8721-44c4-a1e0-3714b0086286\") " pod="openshift-marketplace/redhat-marketplace-gqhzd" Dec 03 11:20:58 crc kubenswrapper[5052]: I1203 11:20:58.037434 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/355008cc-8721-44c4-a1e0-3714b0086286-utilities\") pod \"redhat-marketplace-gqhzd\" (UID: \"355008cc-8721-44c4-a1e0-3714b0086286\") " pod="openshift-marketplace/redhat-marketplace-gqhzd" Dec 03 11:20:58 crc kubenswrapper[5052]: I1203 11:20:58.037459 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-qdx45\" (UniqueName: \"kubernetes.io/projected/355008cc-8721-44c4-a1e0-3714b0086286-kube-api-access-qdx45\") pod \"redhat-marketplace-gqhzd\" (UID: \"355008cc-8721-44c4-a1e0-3714b0086286\") " pod="openshift-marketplace/redhat-marketplace-gqhzd" Dec 03 11:20:58 crc kubenswrapper[5052]: I1203 11:20:58.037943 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/355008cc-8721-44c4-a1e0-3714b0086286-catalog-content\") pod \"redhat-marketplace-gqhzd\" (UID: \"355008cc-8721-44c4-a1e0-3714b0086286\") " pod="openshift-marketplace/redhat-marketplace-gqhzd" Dec 03 11:20:58 crc kubenswrapper[5052]: I1203 11:20:58.037957 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/355008cc-8721-44c4-a1e0-3714b0086286-utilities\") pod \"redhat-marketplace-gqhzd\" (UID: \"355008cc-8721-44c4-a1e0-3714b0086286\") " pod="openshift-marketplace/redhat-marketplace-gqhzd" Dec 03 11:20:58 crc kubenswrapper[5052]: I1203 11:20:58.071812 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-qdx45\" (UniqueName: \"kubernetes.io/projected/355008cc-8721-44c4-a1e0-3714b0086286-kube-api-access-qdx45\") pod \"redhat-marketplace-gqhzd\" (UID: \"355008cc-8721-44c4-a1e0-3714b0086286\") " pod="openshift-marketplace/redhat-marketplace-gqhzd" Dec 03 11:20:58 crc kubenswrapper[5052]: I1203 11:20:58.203971 5052 generic.go:334] "Generic (PLEG): container finished" podID="e43f14fc-f3eb-4f58-b202-cdfc641ac08a" containerID="e2f05729fcded9f0fd3a94445c561f0dc54d8da986e5b0a038ffa712510d5162" exitCode=0 Dec 03 11:20:58 crc kubenswrapper[5052]: I1203 11:20:58.204087 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-operator-lifecycle-manager/collect-profiles-29412675-hzd4n" event={"ID":"e43f14fc-f3eb-4f58-b202-cdfc641ac08a","Type":"ContainerDied","Data":"e2f05729fcded9f0fd3a94445c561f0dc54d8da986e5b0a038ffa712510d5162"} Dec 03 11:20:58 crc kubenswrapper[5052]: I1203 11:20:58.206279 5052 generic.go:334] "Generic (PLEG): container finished" podID="f1ea0ffb-1062-4713-908b-55c0b07c49ee" containerID="96f8f369bca7603b9c66f19aa6ff55cd19d2bae1429e5887b389cf9727e18387" exitCode=0 Dec 03 11:20:58 crc kubenswrapper[5052]: I1203 11:20:58.206433 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/community-operators-r8txz" event={"ID":"f1ea0ffb-1062-4713-908b-55c0b07c49ee","Type":"ContainerDied","Data":"96f8f369bca7603b9c66f19aa6ff55cd19d2bae1429e5887b389cf9727e18387"} Dec 03 11:20:58 crc kubenswrapper[5052]: I1203 11:20:58.206546 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/community-operators-r8txz" event={"ID":"f1ea0ffb-1062-4713-908b-55c0b07c49ee","Type":"ContainerStarted","Data":"f0c85f21f278dcf751b76f25d9cea56b5c872ef44a26cff908d2223a33ee9fe0"} Dec 03 11:20:58 crc kubenswrapper[5052]: I1203 11:20:58.208968 5052 provider.go:102] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Dec 03 11:20:58 crc kubenswrapper[5052]: I1203 11:20:58.209102 5052 generic.go:334] "Generic (PLEG): container finished" podID="3b1b1a53-eff1-4339-a82e-9b8400dd5604" containerID="09adf77efb9328693f484a82c4a75a64e2aa5685f237cfbbad8e3146adf0c914" exitCode=0 Dec 03 11:20:58 crc kubenswrapper[5052]: I1203 11:20:58.209222 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/certified-operators-99svv" event={"ID":"3b1b1a53-eff1-4339-a82e-9b8400dd5604","Type":"ContainerDied","Data":"09adf77efb9328693f484a82c4a75a64e2aa5685f237cfbbad8e3146adf0c914"} Dec 03 11:20:58 crc kubenswrapper[5052]: I1203 11:20:58.209258 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/certified-operators-99svv" event={"ID":"3b1b1a53-eff1-4339-a82e-9b8400dd5604","Type":"ContainerStarted","Data":"73ff6ff9fa66dca9a11a38cc566ec67d005940268452df96cdfc40a37dd7b59d"} Dec 03 11:20:58 crc kubenswrapper[5052]: I1203 11:20:58.217790 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/redhat-marketplace-gqhzd" Dec 03 11:20:58 crc kubenswrapper[5052]: I1203 11:20:58.236628 5052 generic.go:334] "Generic (PLEG): container finished" podID="dc6b87f0-562b-4fa9-85b3-ccdd0867374b" containerID="1767d68b4de95fcf8cb420d32b3fc4b35d9a8358bcee6bbbab6a4fe5d7edbcbd" exitCode=0 Dec 03 11:20:58 crc kubenswrapper[5052]: I1203 11:20:58.237604 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\") pod \"image-registry-697d97f7c8-p77bx\" (UID: \"f5012254-068c-4360-b310-39ff3d309199\") " pod="openshift-image-registry/image-registry-697d97f7c8-p77bx" Dec 03 11:20:58 crc kubenswrapper[5052]: I1203 11:20:58.238026 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/certified-operators-ghqsf" event={"ID":"dc6b87f0-562b-4fa9-85b3-ccdd0867374b","Type":"ContainerDied","Data":"1767d68b4de95fcf8cb420d32b3fc4b35d9a8358bcee6bbbab6a4fe5d7edbcbd"} Dec 03 11:20:58 crc kubenswrapper[5052]: I1203 11:20:58.238127 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/certified-operators-ghqsf" event={"ID":"dc6b87f0-562b-4fa9-85b3-ccdd0867374b","Type":"ContainerStarted","Data":"53738140aeb647312cd091a13c101026a4b498818809d4b57a5c4e077ed6151a"} Dec 03 11:20:58 crc kubenswrapper[5052]: I1203 11:20:58.246183 5052 generic.go:334] "Generic (PLEG): container finished" podID="a7228f2a-1d6d-4307-80e7-deebd47d995a" containerID="f165c5dd9b74405cb0258ec415d7cf2852fda1d2ba6fadfe03c30d3730fa500f" exitCode=0 Dec 03 11:20:58 crc kubenswrapper[5052]: I1203 11:20:58.246305 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/community-operators-pxx88" event={"ID":"a7228f2a-1d6d-4307-80e7-deebd47d995a","Type":"ContainerDied","Data":"f165c5dd9b74405cb0258ec415d7cf2852fda1d2ba6fadfe03c30d3730fa500f"} Dec 03 11:20:58 crc kubenswrapper[5052]: I1203 11:20:58.252447 5052 patch_prober.go:28] interesting pod/router-default-5444994796-rtl8t container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-http failed: reason withheld Dec 03 11:20:58 crc kubenswrapper[5052]: [-]has-synced failed: reason withheld Dec 03 11:20:58 crc kubenswrapper[5052]: [+]process-running ok Dec 03 11:20:58 crc kubenswrapper[5052]: healthz check failed Dec 03 11:20:58 crc kubenswrapper[5052]: I1203 11:20:58.252589 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-5444994796-rtl8t" podUID="48040744-7fc1-482e-9b68-3a904ba03a8d" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500" Dec 03 11:20:58 crc kubenswrapper[5052]: I1203 11:20:58.268289 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="hostpath-provisioner/csi-hostpathplugin-5tlrr" event={"ID":"60b355dc-6de7-4a73-a2d2-3a0f0befd049","Type":"ContainerStarted","Data":"84909c1499034958d0611da661b6d2664bd47171a75fa134fa0945528cd705c2"} Dec 03 11:20:58 crc kubenswrapper[5052]: I1203 11:20:58.292336 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openshift-marketplace/redhat-marketplace-qttdj"] Dec 03 11:20:58 crc kubenswrapper[5052]: I1203 11:20:58.294221 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/redhat-marketplace-qttdj" Dec 03 11:20:58 crc kubenswrapper[5052]: I1203 11:20:58.324170 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/redhat-marketplace-qttdj"] Dec 03 11:20:58 crc kubenswrapper[5052]: I1203 11:20:58.393404 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-697d97f7c8-p77bx" Dec 03 11:20:58 crc kubenswrapper[5052]: I1203 11:20:58.407192 5052 patch_prober.go:28] interesting pod/downloads-7954f5f757-l7xtl container/download-server namespace/openshift-console: Readiness probe status=failure output="Get \"http://10.217.0.18:8080/\": dial tcp 10.217.0.18:8080: connect: connection refused" start-of-body= Dec 03 11:20:58 crc kubenswrapper[5052]: I1203 11:20:58.407289 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openshift-console/downloads-7954f5f757-l7xtl" podUID="ec2ea657-018b-4626-bf79-5c818f573a4b" containerName="download-server" probeResult="failure" output="Get \"http://10.217.0.18:8080/\": dial tcp 10.217.0.18:8080: connect: connection refused" Dec 03 11:20:58 crc kubenswrapper[5052]: I1203 11:20:58.407640 5052 patch_prober.go:28] interesting pod/downloads-7954f5f757-l7xtl container/download-server namespace/openshift-console: Liveness probe status=failure output="Get \"http://10.217.0.18:8080/\": dial tcp 10.217.0.18:8080: connect: connection refused" start-of-body= Dec 03 11:20:58 crc kubenswrapper[5052]: I1203 11:20:58.407753 5052 prober.go:107] "Probe failed" probeType="Liveness" pod="openshift-console/downloads-7954f5f757-l7xtl" podUID="ec2ea657-018b-4626-bf79-5c818f573a4b" containerName="download-server" probeResult="failure" output="Get \"http://10.217.0.18:8080/\": dial tcp 10.217.0.18:8080: connect: connection refused" Dec 03 11:20:58 crc kubenswrapper[5052]: I1203 11:20:58.457380 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lssp4\" (UniqueName: \"kubernetes.io/projected/8cf9b98e-2ea9-47ee-bd1b-a913714b7668-kube-api-access-lssp4\") pod \"redhat-marketplace-qttdj\" (UID: \"8cf9b98e-2ea9-47ee-bd1b-a913714b7668\") " pod="openshift-marketplace/redhat-marketplace-qttdj" Dec 03 11:20:58 crc kubenswrapper[5052]: I1203 11:20:58.457438 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/8cf9b98e-2ea9-47ee-bd1b-a913714b7668-catalog-content\") pod \"redhat-marketplace-qttdj\" (UID: \"8cf9b98e-2ea9-47ee-bd1b-a913714b7668\") " pod="openshift-marketplace/redhat-marketplace-qttdj" Dec 03 11:20:58 crc kubenswrapper[5052]: I1203 11:20:58.457905 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/8cf9b98e-2ea9-47ee-bd1b-a913714b7668-utilities\") pod \"redhat-marketplace-qttdj\" (UID: \"8cf9b98e-2ea9-47ee-bd1b-a913714b7668\") " pod="openshift-marketplace/redhat-marketplace-qttdj" Dec 03 11:20:58 crc kubenswrapper[5052]: I1203 11:20:58.556264 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openshift-ovn-kubernetes/ovnkube-node-kg55h" Dec 03 11:20:58 crc kubenswrapper[5052]: I1203 11:20:58.556485 5052 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Dec 03 11:20:58 crc kubenswrapper[5052]: I1203 11:20:58.560019 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/8cf9b98e-2ea9-47ee-bd1b-a913714b7668-utilities\") pod \"redhat-marketplace-qttdj\" (UID: \"8cf9b98e-2ea9-47ee-bd1b-a913714b7668\") " pod="openshift-marketplace/redhat-marketplace-qttdj" Dec 03 11:20:58 crc kubenswrapper[5052]: I1203 11:20:58.560066 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-lssp4\" (UniqueName: \"kubernetes.io/projected/8cf9b98e-2ea9-47ee-bd1b-a913714b7668-kube-api-access-lssp4\") pod \"redhat-marketplace-qttdj\" (UID: \"8cf9b98e-2ea9-47ee-bd1b-a913714b7668\") " pod="openshift-marketplace/redhat-marketplace-qttdj" Dec 03 11:20:58 crc kubenswrapper[5052]: I1203 11:20:58.560090 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/8cf9b98e-2ea9-47ee-bd1b-a913714b7668-catalog-content\") pod \"redhat-marketplace-qttdj\" (UID: \"8cf9b98e-2ea9-47ee-bd1b-a913714b7668\") " pod="openshift-marketplace/redhat-marketplace-qttdj" Dec 03 11:20:58 crc kubenswrapper[5052]: I1203 11:20:58.560500 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/8cf9b98e-2ea9-47ee-bd1b-a913714b7668-utilities\") pod \"redhat-marketplace-qttdj\" (UID: \"8cf9b98e-2ea9-47ee-bd1b-a913714b7668\") " pod="openshift-marketplace/redhat-marketplace-qttdj" Dec 03 11:20:58 crc kubenswrapper[5052]: I1203 11:20:58.560834 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/8cf9b98e-2ea9-47ee-bd1b-a913714b7668-catalog-content\") pod \"redhat-marketplace-qttdj\" (UID: \"8cf9b98e-2ea9-47ee-bd1b-a913714b7668\") " pod="openshift-marketplace/redhat-marketplace-qttdj" Dec 03 11:20:58 crc kubenswrapper[5052]: I1203 11:20:58.613422 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-lssp4\" (UniqueName: \"kubernetes.io/projected/8cf9b98e-2ea9-47ee-bd1b-a913714b7668-kube-api-access-lssp4\") pod \"redhat-marketplace-qttdj\" (UID: \"8cf9b98e-2ea9-47ee-bd1b-a913714b7668\") " pod="openshift-marketplace/redhat-marketplace-qttdj" Dec 03 11:20:58 crc kubenswrapper[5052]: I1203 11:20:58.618876 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/redhat-marketplace-qttdj" Dec 03 11:20:58 crc kubenswrapper[5052]: I1203 11:20:58.620939 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-kg55h" Dec 03 11:20:58 crc kubenswrapper[5052]: I1203 11:20:58.724319 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/redhat-marketplace-gqhzd"] Dec 03 11:20:58 crc kubenswrapper[5052]: W1203 11:20:58.770718 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod355008cc_8721_44c4_a1e0_3714b0086286.slice/crio-11321015dcd20484cfaee0feff5586e629baa83de93c0b75a8a93885ed4f02a7 WatchSource:0}: Error finding container 11321015dcd20484cfaee0feff5586e629baa83de93c0b75a8a93885ed4f02a7: Status 404 returned error can't find the container with id 11321015dcd20484cfaee0feff5586e629baa83de93c0b75a8a93885ed4f02a7 Dec 03 11:20:58 crc kubenswrapper[5052]: I1203 11:20:58.887168 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-697d97f7c8-p77bx"] Dec 03 11:20:58 crc kubenswrapper[5052]: I1203 11:20:58.895585 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openshift-marketplace/redhat-operators-dwt2v"] Dec 03 11:20:58 crc kubenswrapper[5052]: I1203 11:20:58.899004 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/redhat-operators-dwt2v" Dec 03 11:20:58 crc kubenswrapper[5052]: I1203 11:20:58.901468 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-marketplace"/"redhat-operators-dockercfg-ct8rh" Dec 03 11:20:58 crc kubenswrapper[5052]: I1203 11:20:58.917429 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/redhat-operators-dwt2v"] Dec 03 11:20:58 crc kubenswrapper[5052]: I1203 11:20:58.965505 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/afb87a28-1b7c-4b09-b186-f05da232f501-utilities\") pod \"redhat-operators-dwt2v\" (UID: \"afb87a28-1b7c-4b09-b186-f05da232f501\") " pod="openshift-marketplace/redhat-operators-dwt2v" Dec 03 11:20:58 crc kubenswrapper[5052]: I1203 11:20:58.965641 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dm25c\" (UniqueName: \"kubernetes.io/projected/afb87a28-1b7c-4b09-b186-f05da232f501-kube-api-access-dm25c\") pod \"redhat-operators-dwt2v\" (UID: \"afb87a28-1b7c-4b09-b186-f05da232f501\") " pod="openshift-marketplace/redhat-operators-dwt2v" Dec 03 11:20:58 crc kubenswrapper[5052]: I1203 11:20:58.965669 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/afb87a28-1b7c-4b09-b186-f05da232f501-catalog-content\") pod \"redhat-operators-dwt2v\" (UID: \"afb87a28-1b7c-4b09-b186-f05da232f501\") " pod="openshift-marketplace/redhat-operators-dwt2v" Dec 03 11:20:59 crc kubenswrapper[5052]: I1203 11:20:59.066614 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-dm25c\" (UniqueName: \"kubernetes.io/projected/afb87a28-1b7c-4b09-b186-f05da232f501-kube-api-access-dm25c\") pod \"redhat-operators-dwt2v\" (UID: \"afb87a28-1b7c-4b09-b186-f05da232f501\") " pod="openshift-marketplace/redhat-operators-dwt2v" Dec 03 11:20:59 crc kubenswrapper[5052]: I1203 11:20:59.066658 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/afb87a28-1b7c-4b09-b186-f05da232f501-catalog-content\") pod \"redhat-operators-dwt2v\" (UID: \"afb87a28-1b7c-4b09-b186-f05da232f501\") " pod="openshift-marketplace/redhat-operators-dwt2v" Dec 03 11:20:59 crc kubenswrapper[5052]: I1203 11:20:59.066702 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/afb87a28-1b7c-4b09-b186-f05da232f501-utilities\") pod \"redhat-operators-dwt2v\" (UID: \"afb87a28-1b7c-4b09-b186-f05da232f501\") " pod="openshift-marketplace/redhat-operators-dwt2v" Dec 03 11:20:59 crc kubenswrapper[5052]: I1203 11:20:59.067195 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/afb87a28-1b7c-4b09-b186-f05da232f501-utilities\") pod \"redhat-operators-dwt2v\" (UID: \"afb87a28-1b7c-4b09-b186-f05da232f501\") " pod="openshift-marketplace/redhat-operators-dwt2v" Dec 03 11:20:59 crc kubenswrapper[5052]: I1203 11:20:59.067718 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/afb87a28-1b7c-4b09-b186-f05da232f501-catalog-content\") pod \"redhat-operators-dwt2v\" (UID: \"afb87a28-1b7c-4b09-b186-f05da232f501\") " pod="openshift-marketplace/redhat-operators-dwt2v" Dec 03 11:20:59 crc kubenswrapper[5052]: I1203 11:20:59.080566 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/redhat-marketplace-qttdj"] Dec 03 11:20:59 crc kubenswrapper[5052]: I1203 11:20:59.089754 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-dm25c\" (UniqueName: \"kubernetes.io/projected/afb87a28-1b7c-4b09-b186-f05da232f501-kube-api-access-dm25c\") pod \"redhat-operators-dwt2v\" (UID: \"afb87a28-1b7c-4b09-b186-f05da232f501\") " pod="openshift-marketplace/redhat-operators-dwt2v" Dec 03 11:20:59 crc kubenswrapper[5052]: I1203 11:20:59.194060 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openshift-kube-controller-manager/revision-pruner-9-crc"] Dec 03 11:20:59 crc kubenswrapper[5052]: I1203 11:20:59.195433 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-controller-manager/revision-pruner-9-crc" Dec 03 11:20:59 crc kubenswrapper[5052]: I1203 11:20:59.198566 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-kube-controller-manager"/"kube-root-ca.crt" Dec 03 11:20:59 crc kubenswrapper[5052]: I1203 11:20:59.198654 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-kube-controller-manager"/"installer-sa-dockercfg-kjl2n" Dec 03 11:20:59 crc kubenswrapper[5052]: I1203 11:20:59.204307 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-kube-controller-manager/revision-pruner-9-crc"] Dec 03 11:20:59 crc kubenswrapper[5052]: I1203 11:20:59.244840 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openshift-ingress/router-default-5444994796-rtl8t" Dec 03 11:20:59 crc kubenswrapper[5052]: I1203 11:20:59.254397 5052 patch_prober.go:28] interesting pod/router-default-5444994796-rtl8t container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-http failed: reason withheld Dec 03 11:20:59 crc kubenswrapper[5052]: [-]has-synced failed: reason withheld Dec 03 11:20:59 crc kubenswrapper[5052]: [+]process-running ok Dec 03 11:20:59 crc kubenswrapper[5052]: healthz check failed Dec 03 11:20:59 crc kubenswrapper[5052]: I1203 11:20:59.254463 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-5444994796-rtl8t" podUID="48040744-7fc1-482e-9b68-3a904ba03a8d" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500" Dec 03 11:20:59 crc kubenswrapper[5052]: I1203 11:20:59.269332 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/976897c5-e4db-4f3d-bd44-0e4061bdf72b-kubelet-dir\") pod \"revision-pruner-9-crc\" (UID: \"976897c5-e4db-4f3d-bd44-0e4061bdf72b\") " pod="openshift-kube-controller-manager/revision-pruner-9-crc" Dec 03 11:20:59 crc kubenswrapper[5052]: I1203 11:20:59.269400 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access\" (UniqueName: \"kubernetes.io/projected/976897c5-e4db-4f3d-bd44-0e4061bdf72b-kube-api-access\") pod \"revision-pruner-9-crc\" (UID: \"976897c5-e4db-4f3d-bd44-0e4061bdf72b\") " pod="openshift-kube-controller-manager/revision-pruner-9-crc" Dec 03 11:20:59 crc kubenswrapper[5052]: I1203 11:20:59.273281 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/redhat-operators-dwt2v" Dec 03 11:20:59 crc kubenswrapper[5052]: I1203 11:20:59.276342 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-marketplace-qttdj" event={"ID":"8cf9b98e-2ea9-47ee-bd1b-a913714b7668","Type":"ContainerStarted","Data":"8a054a130d2fc5a4999a05716d05febba7f18f9a4fdb1f6757f8a0424ee6fb4d"} Dec 03 11:20:59 crc kubenswrapper[5052]: I1203 11:20:59.278141 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-marketplace-gqhzd" event={"ID":"355008cc-8721-44c4-a1e0-3714b0086286","Type":"ContainerStarted","Data":"11321015dcd20484cfaee0feff5586e629baa83de93c0b75a8a93885ed4f02a7"} Dec 03 11:20:59 crc kubenswrapper[5052]: I1203 11:20:59.280955 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="hostpath-provisioner/csi-hostpathplugin-5tlrr" event={"ID":"60b355dc-6de7-4a73-a2d2-3a0f0befd049","Type":"ContainerStarted","Data":"4ad88cd38b4fd9176595d42235d3b8a93467cda37f5b8061a546fa685a6b665c"} Dec 03 11:20:59 crc kubenswrapper[5052]: I1203 11:20:59.282455 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-697d97f7c8-p77bx" event={"ID":"f5012254-068c-4360-b310-39ff3d309199","Type":"ContainerStarted","Data":"2aac92f4f9ceffe73b9baac1c04f7002e948025594b6c4534ec1782755c026b1"} Dec 03 11:20:59 crc kubenswrapper[5052]: I1203 11:20:59.299677 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openshift-marketplace/redhat-operators-nq2qk"] Dec 03 11:20:59 crc kubenswrapper[5052]: I1203 11:20:59.301199 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/redhat-operators-nq2qk" Dec 03 11:20:59 crc kubenswrapper[5052]: I1203 11:20:59.306546 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/redhat-operators-nq2qk"] Dec 03 11:20:59 crc kubenswrapper[5052]: I1203 11:20:59.312155 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="hostpath-provisioner/csi-hostpathplugin-5tlrr" podStartSLOduration=14.312130282 podStartE2EDuration="14.312130282s" podCreationTimestamp="2025-12-03 11:20:45 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:20:59.310151577 +0000 UTC m=+49.898393361" watchObservedRunningTime="2025-12-03 11:20:59.312130282 +0000 UTC m=+49.900372046" Dec 03 11:20:59 crc kubenswrapper[5052]: I1203 11:20:59.370473 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/976897c5-e4db-4f3d-bd44-0e4061bdf72b-kubelet-dir\") pod \"revision-pruner-9-crc\" (UID: \"976897c5-e4db-4f3d-bd44-0e4061bdf72b\") " pod="openshift-kube-controller-manager/revision-pruner-9-crc" Dec 03 11:20:59 crc kubenswrapper[5052]: I1203 11:20:59.370566 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access\" (UniqueName: \"kubernetes.io/projected/976897c5-e4db-4f3d-bd44-0e4061bdf72b-kube-api-access\") pod \"revision-pruner-9-crc\" (UID: \"976897c5-e4db-4f3d-bd44-0e4061bdf72b\") " pod="openshift-kube-controller-manager/revision-pruner-9-crc" Dec 03 11:20:59 crc kubenswrapper[5052]: I1203 11:20:59.370610 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/1e97c072-bdcc-4095-9a87-ddf7d4f2d22b-catalog-content\") pod \"redhat-operators-nq2qk\" (UID: \"1e97c072-bdcc-4095-9a87-ddf7d4f2d22b\") " pod="openshift-marketplace/redhat-operators-nq2qk" Dec 03 11:20:59 crc kubenswrapper[5052]: I1203 11:20:59.370660 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kgznh\" (UniqueName: \"kubernetes.io/projected/1e97c072-bdcc-4095-9a87-ddf7d4f2d22b-kube-api-access-kgznh\") pod \"redhat-operators-nq2qk\" (UID: \"1e97c072-bdcc-4095-9a87-ddf7d4f2d22b\") " pod="openshift-marketplace/redhat-operators-nq2qk" Dec 03 11:20:59 crc kubenswrapper[5052]: I1203 11:20:59.370696 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/1e97c072-bdcc-4095-9a87-ddf7d4f2d22b-utilities\") pod \"redhat-operators-nq2qk\" (UID: \"1e97c072-bdcc-4095-9a87-ddf7d4f2d22b\") " pod="openshift-marketplace/redhat-operators-nq2qk" Dec 03 11:20:59 crc kubenswrapper[5052]: I1203 11:20:59.371842 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/976897c5-e4db-4f3d-bd44-0e4061bdf72b-kubelet-dir\") pod \"revision-pruner-9-crc\" (UID: \"976897c5-e4db-4f3d-bd44-0e4061bdf72b\") " pod="openshift-kube-controller-manager/revision-pruner-9-crc" Dec 03 11:20:59 crc kubenswrapper[5052]: I1203 11:20:59.390704 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access\" (UniqueName: \"kubernetes.io/projected/976897c5-e4db-4f3d-bd44-0e4061bdf72b-kube-api-access\") pod \"revision-pruner-9-crc\" (UID: \"976897c5-e4db-4f3d-bd44-0e4061bdf72b\") " pod="openshift-kube-controller-manager/revision-pruner-9-crc" Dec 03 11:20:59 crc kubenswrapper[5052]: I1203 11:20:59.472145 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/1e97c072-bdcc-4095-9a87-ddf7d4f2d22b-catalog-content\") pod \"redhat-operators-nq2qk\" (UID: \"1e97c072-bdcc-4095-9a87-ddf7d4f2d22b\") " pod="openshift-marketplace/redhat-operators-nq2qk" Dec 03 11:20:59 crc kubenswrapper[5052]: I1203 11:20:59.472211 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-kgznh\" (UniqueName: \"kubernetes.io/projected/1e97c072-bdcc-4095-9a87-ddf7d4f2d22b-kube-api-access-kgznh\") pod \"redhat-operators-nq2qk\" (UID: \"1e97c072-bdcc-4095-9a87-ddf7d4f2d22b\") " pod="openshift-marketplace/redhat-operators-nq2qk" Dec 03 11:20:59 crc kubenswrapper[5052]: I1203 11:20:59.472250 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/1e97c072-bdcc-4095-9a87-ddf7d4f2d22b-utilities\") pod \"redhat-operators-nq2qk\" (UID: \"1e97c072-bdcc-4095-9a87-ddf7d4f2d22b\") " pod="openshift-marketplace/redhat-operators-nq2qk" Dec 03 11:20:59 crc kubenswrapper[5052]: I1203 11:20:59.473402 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/1e97c072-bdcc-4095-9a87-ddf7d4f2d22b-utilities\") pod \"redhat-operators-nq2qk\" (UID: \"1e97c072-bdcc-4095-9a87-ddf7d4f2d22b\") " pod="openshift-marketplace/redhat-operators-nq2qk" Dec 03 11:20:59 crc kubenswrapper[5052]: I1203 11:20:59.473711 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/1e97c072-bdcc-4095-9a87-ddf7d4f2d22b-catalog-content\") pod \"redhat-operators-nq2qk\" (UID: \"1e97c072-bdcc-4095-9a87-ddf7d4f2d22b\") " pod="openshift-marketplace/redhat-operators-nq2qk" Dec 03 11:20:59 crc kubenswrapper[5052]: I1203 11:20:59.514874 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-kgznh\" (UniqueName: \"kubernetes.io/projected/1e97c072-bdcc-4095-9a87-ddf7d4f2d22b-kube-api-access-kgznh\") pod \"redhat-operators-nq2qk\" (UID: \"1e97c072-bdcc-4095-9a87-ddf7d4f2d22b\") " pod="openshift-marketplace/redhat-operators-nq2qk" Dec 03 11:20:59 crc kubenswrapper[5052]: E1203 11:20:59.514923 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="b0103da56f9128a13b28c52c5f8402db976339938594d6eae2a65c5f008b27ef" cmd=["/bin/bash","-c","test -f /ready/ready"] Dec 03 11:20:59 crc kubenswrapper[5052]: E1203 11:20:59.532713 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="b0103da56f9128a13b28c52c5f8402db976339938594d6eae2a65c5f008b27ef" cmd=["/bin/bash","-c","test -f /ready/ready"] Dec 03 11:20:59 crc kubenswrapper[5052]: E1203 11:20:59.534341 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="b0103da56f9128a13b28c52c5f8402db976339938594d6eae2a65c5f008b27ef" cmd=["/bin/bash","-c","test -f /ready/ready"] Dec 03 11:20:59 crc kubenswrapper[5052]: E1203 11:20:59.534380 5052 prober.go:104] "Probe errored" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" probeType="Readiness" pod="openshift-multus/cni-sysctl-allowlist-ds-8l9gm" podUID="bf761cb9-e024-4acd-a947-0f8064267243" containerName="kube-multus-additional-cni-plugins" Dec 03 11:20:59 crc kubenswrapper[5052]: I1203 11:20:59.557224 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-controller-manager/revision-pruner-9-crc" Dec 03 11:20:59 crc kubenswrapper[5052]: I1203 11:20:59.573837 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/redhat-operators-dwt2v"] Dec 03 11:20:59 crc kubenswrapper[5052]: I1203 11:20:59.588588 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-apiserver/apiserver-76f77b778f-dmvst" Dec 03 11:20:59 crc kubenswrapper[5052]: I1203 11:20:59.588665 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openshift-apiserver/apiserver-76f77b778f-dmvst" Dec 03 11:20:59 crc kubenswrapper[5052]: I1203 11:20:59.596302 5052 patch_prober.go:28] interesting pod/apiserver-76f77b778f-dmvst container/openshift-apiserver namespace/openshift-apiserver: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[+]ping ok Dec 03 11:20:59 crc kubenswrapper[5052]: [+]log ok Dec 03 11:20:59 crc kubenswrapper[5052]: [+]etcd ok Dec 03 11:20:59 crc kubenswrapper[5052]: [+]poststarthook/start-apiserver-admission-initializer ok Dec 03 11:20:59 crc kubenswrapper[5052]: [+]poststarthook/generic-apiserver-start-informers ok Dec 03 11:20:59 crc kubenswrapper[5052]: [+]poststarthook/max-in-flight-filter ok Dec 03 11:20:59 crc kubenswrapper[5052]: [+]poststarthook/storage-object-count-tracker-hook ok Dec 03 11:20:59 crc kubenswrapper[5052]: [+]poststarthook/image.openshift.io-apiserver-caches ok Dec 03 11:20:59 crc kubenswrapper[5052]: [-]poststarthook/authorization.openshift.io-bootstrapclusterroles failed: reason withheld Dec 03 11:20:59 crc kubenswrapper[5052]: [-]poststarthook/authorization.openshift.io-ensurenodebootstrap-sa failed: reason withheld Dec 03 11:20:59 crc kubenswrapper[5052]: [+]poststarthook/project.openshift.io-projectcache ok Dec 03 11:20:59 crc kubenswrapper[5052]: [+]poststarthook/project.openshift.io-projectauthorizationcache ok Dec 03 11:20:59 crc kubenswrapper[5052]: [+]poststarthook/openshift.io-startinformers ok Dec 03 11:20:59 crc kubenswrapper[5052]: [+]poststarthook/openshift.io-restmapperupdater ok Dec 03 11:20:59 crc kubenswrapper[5052]: [+]poststarthook/quota.openshift.io-clusterquotamapping ok Dec 03 11:20:59 crc kubenswrapper[5052]: livez check failed Dec 03 11:20:59 crc kubenswrapper[5052]: I1203 11:20:59.596395 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openshift-apiserver/apiserver-76f77b778f-dmvst" podUID="41016a7b-cc79-4f32-860c-cb75e405cf42" containerName="openshift-apiserver" probeResult="failure" output="HTTP probe failed with statuscode: 500" Dec 03 11:20:59 crc kubenswrapper[5052]: I1203 11:20:59.603307 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-operator-lifecycle-manager/collect-profiles-29412675-hzd4n" Dec 03 11:20:59 crc kubenswrapper[5052]: I1203 11:20:59.644219 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/redhat-operators-nq2qk" Dec 03 11:20:59 crc kubenswrapper[5052]: I1203 11:20:59.649653 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-f9d7485db-568dj" Dec 03 11:20:59 crc kubenswrapper[5052]: I1203 11:20:59.649716 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openshift-console/console-f9d7485db-568dj" Dec 03 11:20:59 crc kubenswrapper[5052]: I1203 11:20:59.651581 5052 patch_prober.go:28] interesting pod/console-f9d7485db-568dj container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.217.0.22:8443/health\": dial tcp 10.217.0.22:8443: connect: connection refused" start-of-body= Dec 03 11:20:59 crc kubenswrapper[5052]: I1203 11:20:59.651632 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openshift-console/console-f9d7485db-568dj" podUID="b1d2ff08-d58b-48d0-a0e5-6cfd867a63de" containerName="console" probeResult="failure" output="Get \"https://10.217.0.22:8443/health\": dial tcp 10.217.0.22:8443: connect: connection refused" Dec 03 11:20:59 crc kubenswrapper[5052]: I1203 11:20:59.688139 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8f668bae-612b-4b75-9490-919e737c6a3b" path="/var/lib/kubelet/pods/8f668bae-612b-4b75-9490-919e737c6a3b/volumes" Dec 03 11:20:59 crc kubenswrapper[5052]: I1203 11:20:59.779126 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/e43f14fc-f3eb-4f58-b202-cdfc641ac08a-config-volume\") pod \"e43f14fc-f3eb-4f58-b202-cdfc641ac08a\" (UID: \"e43f14fc-f3eb-4f58-b202-cdfc641ac08a\") " Dec 03 11:20:59 crc kubenswrapper[5052]: I1203 11:20:59.779634 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"secret-volume\" (UniqueName: \"kubernetes.io/secret/e43f14fc-f3eb-4f58-b202-cdfc641ac08a-secret-volume\") pod \"e43f14fc-f3eb-4f58-b202-cdfc641ac08a\" (UID: \"e43f14fc-f3eb-4f58-b202-cdfc641ac08a\") " Dec 03 11:20:59 crc kubenswrapper[5052]: I1203 11:20:59.779756 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hmnk9\" (UniqueName: \"kubernetes.io/projected/e43f14fc-f3eb-4f58-b202-cdfc641ac08a-kube-api-access-hmnk9\") pod \"e43f14fc-f3eb-4f58-b202-cdfc641ac08a\" (UID: \"e43f14fc-f3eb-4f58-b202-cdfc641ac08a\") " Dec 03 11:20:59 crc kubenswrapper[5052]: I1203 11:20:59.781063 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e43f14fc-f3eb-4f58-b202-cdfc641ac08a-config-volume" (OuterVolumeSpecName: "config-volume") pod "e43f14fc-f3eb-4f58-b202-cdfc641ac08a" (UID: "e43f14fc-f3eb-4f58-b202-cdfc641ac08a"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:20:59 crc kubenswrapper[5052]: I1203 11:20:59.784940 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e43f14fc-f3eb-4f58-b202-cdfc641ac08a-secret-volume" (OuterVolumeSpecName: "secret-volume") pod "e43f14fc-f3eb-4f58-b202-cdfc641ac08a" (UID: "e43f14fc-f3eb-4f58-b202-cdfc641ac08a"). InnerVolumeSpecName "secret-volume". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:20:59 crc kubenswrapper[5052]: I1203 11:20:59.786043 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e43f14fc-f3eb-4f58-b202-cdfc641ac08a-kube-api-access-hmnk9" (OuterVolumeSpecName: "kube-api-access-hmnk9") pod "e43f14fc-f3eb-4f58-b202-cdfc641ac08a" (UID: "e43f14fc-f3eb-4f58-b202-cdfc641ac08a"). InnerVolumeSpecName "kube-api-access-hmnk9". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:20:59 crc kubenswrapper[5052]: I1203 11:20:59.891566 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-kube-controller-manager/revision-pruner-9-crc"] Dec 03 11:20:59 crc kubenswrapper[5052]: I1203 11:20:59.893712 5052 reconciler_common.go:293] "Volume detached for volume \"secret-volume\" (UniqueName: \"kubernetes.io/secret/e43f14fc-f3eb-4f58-b202-cdfc641ac08a-secret-volume\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:59 crc kubenswrapper[5052]: I1203 11:20:59.893744 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-hmnk9\" (UniqueName: \"kubernetes.io/projected/e43f14fc-f3eb-4f58-b202-cdfc641ac08a-kube-api-access-hmnk9\") on node \"crc\" DevicePath \"\"" Dec 03 11:20:59 crc kubenswrapper[5052]: I1203 11:20:59.893754 5052 reconciler_common.go:293] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/e43f14fc-f3eb-4f58-b202-cdfc641ac08a-config-volume\") on node \"crc\" DevicePath \"\"" Dec 03 11:21:00 crc kubenswrapper[5052]: I1203 11:21:00.057109 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/redhat-operators-nq2qk"] Dec 03 11:21:00 crc kubenswrapper[5052]: I1203 11:21:00.250031 5052 patch_prober.go:28] interesting pod/router-default-5444994796-rtl8t container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-http failed: reason withheld Dec 03 11:21:00 crc kubenswrapper[5052]: [-]has-synced failed: reason withheld Dec 03 11:21:00 crc kubenswrapper[5052]: [+]process-running ok Dec 03 11:21:00 crc kubenswrapper[5052]: healthz check failed Dec 03 11:21:00 crc kubenswrapper[5052]: I1203 11:21:00.250103 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-5444994796-rtl8t" podUID="48040744-7fc1-482e-9b68-3a904ba03a8d" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500" Dec 03 11:21:00 crc kubenswrapper[5052]: I1203 11:21:00.307807 5052 generic.go:334] "Generic (PLEG): container finished" podID="355008cc-8721-44c4-a1e0-3714b0086286" containerID="c5ecedf32f587c013691fd61e77a5a2a4ef4e820227c1e4e265c3ef4d180defb" exitCode=0 Dec 03 11:21:00 crc kubenswrapper[5052]: I1203 11:21:00.307909 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-marketplace-gqhzd" event={"ID":"355008cc-8721-44c4-a1e0-3714b0086286","Type":"ContainerDied","Data":"c5ecedf32f587c013691fd61e77a5a2a4ef4e820227c1e4e265c3ef4d180defb"} Dec 03 11:21:00 crc kubenswrapper[5052]: I1203 11:21:00.323481 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-operator-lifecycle-manager/collect-profiles-29412675-hzd4n" Dec 03 11:21:00 crc kubenswrapper[5052]: I1203 11:21:00.323480 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-operator-lifecycle-manager/collect-profiles-29412675-hzd4n" event={"ID":"e43f14fc-f3eb-4f58-b202-cdfc641ac08a","Type":"ContainerDied","Data":"4c7da6eda6ba0ce1724c7cd12ef13c0bc571d89291e3b3beb10c29f534bad15c"} Dec 03 11:21:00 crc kubenswrapper[5052]: I1203 11:21:00.323745 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4c7da6eda6ba0ce1724c7cd12ef13c0bc571d89291e3b3beb10c29f534bad15c" Dec 03 11:21:00 crc kubenswrapper[5052]: I1203 11:21:00.332254 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-697d97f7c8-p77bx" event={"ID":"f5012254-068c-4360-b310-39ff3d309199","Type":"ContainerStarted","Data":"d21421fd913291ac3ac9b67e9a506128685f1a0d80fcab53e873bc0124e4cfb1"} Dec 03 11:21:00 crc kubenswrapper[5052]: I1203 11:21:00.332547 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openshift-image-registry/image-registry-697d97f7c8-p77bx" Dec 03 11:21:00 crc kubenswrapper[5052]: I1203 11:21:00.336780 5052 generic.go:334] "Generic (PLEG): container finished" podID="afb87a28-1b7c-4b09-b186-f05da232f501" containerID="948e728d629565ee5dd9e42189bfbad641f1d3720cf7169d41a2fb418d4f1258" exitCode=0 Dec 03 11:21:00 crc kubenswrapper[5052]: I1203 11:21:00.337220 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-operators-dwt2v" event={"ID":"afb87a28-1b7c-4b09-b186-f05da232f501","Type":"ContainerDied","Data":"948e728d629565ee5dd9e42189bfbad641f1d3720cf7169d41a2fb418d4f1258"} Dec 03 11:21:00 crc kubenswrapper[5052]: I1203 11:21:00.337240 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-operators-dwt2v" event={"ID":"afb87a28-1b7c-4b09-b186-f05da232f501","Type":"ContainerStarted","Data":"6378464b7415353cf4fc4b0eb25f32886db80f0cb73e423e6879d3bfab9cb9ee"} Dec 03 11:21:00 crc kubenswrapper[5052]: I1203 11:21:00.352224 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/image-registry-697d97f7c8-p77bx" podStartSLOduration=28.352204504 podStartE2EDuration="28.352204504s" podCreationTimestamp="2025-12-03 11:20:32 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:21:00.351166255 +0000 UTC m=+50.939408019" watchObservedRunningTime="2025-12-03 11:21:00.352204504 +0000 UTC m=+50.940446268" Dec 03 11:21:00 crc kubenswrapper[5052]: I1203 11:21:00.378725 5052 generic.go:334] "Generic (PLEG): container finished" podID="8cf9b98e-2ea9-47ee-bd1b-a913714b7668" containerID="b7b104beab26c080f6cd35706b17f3334ca22c3f8adfcf8f8aa965994d754fe3" exitCode=0 Dec 03 11:21:00 crc kubenswrapper[5052]: I1203 11:21:00.378826 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-marketplace-qttdj" event={"ID":"8cf9b98e-2ea9-47ee-bd1b-a913714b7668","Type":"ContainerDied","Data":"b7b104beab26c080f6cd35706b17f3334ca22c3f8adfcf8f8aa965994d754fe3"} Dec 03 11:21:00 crc kubenswrapper[5052]: I1203 11:21:00.388796 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-operators-nq2qk" event={"ID":"1e97c072-bdcc-4095-9a87-ddf7d4f2d22b","Type":"ContainerStarted","Data":"6d126c9db48ca5aa9702f1a934a254b89f7620454dfb955777e1ae91618746bc"} Dec 03 11:21:00 crc kubenswrapper[5052]: I1203 11:21:00.394058 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-kube-controller-manager/revision-pruner-9-crc" event={"ID":"976897c5-e4db-4f3d-bd44-0e4061bdf72b","Type":"ContainerStarted","Data":"170387cfb74b132f74642d91e20ed7cd74c8275edd8c5a21bdbe5a90c14b6a83"} Dec 03 11:21:01 crc kubenswrapper[5052]: I1203 11:21:01.247745 5052 patch_prober.go:28] interesting pod/router-default-5444994796-rtl8t container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-http failed: reason withheld Dec 03 11:21:01 crc kubenswrapper[5052]: [-]has-synced failed: reason withheld Dec 03 11:21:01 crc kubenswrapper[5052]: [+]process-running ok Dec 03 11:21:01 crc kubenswrapper[5052]: healthz check failed Dec 03 11:21:01 crc kubenswrapper[5052]: I1203 11:21:01.248446 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-5444994796-rtl8t" podUID="48040744-7fc1-482e-9b68-3a904ba03a8d" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500" Dec 03 11:21:01 crc kubenswrapper[5052]: I1203 11:21:01.447954 5052 generic.go:334] "Generic (PLEG): container finished" podID="1e97c072-bdcc-4095-9a87-ddf7d4f2d22b" containerID="934b128bed629ee32297db05e254a0d31930e08ad1f1bbdc95695969db8a1c44" exitCode=0 Dec 03 11:21:01 crc kubenswrapper[5052]: I1203 11:21:01.448104 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-operators-nq2qk" event={"ID":"1e97c072-bdcc-4095-9a87-ddf7d4f2d22b","Type":"ContainerDied","Data":"934b128bed629ee32297db05e254a0d31930e08ad1f1bbdc95695969db8a1c44"} Dec 03 11:21:01 crc kubenswrapper[5052]: I1203 11:21:01.455285 5052 generic.go:334] "Generic (PLEG): container finished" podID="976897c5-e4db-4f3d-bd44-0e4061bdf72b" containerID="a7a636ce19173de0f67bf080e664580e7fadbb4b7f039f2262df551f79a4f5c0" exitCode=0 Dec 03 11:21:01 crc kubenswrapper[5052]: I1203 11:21:01.455928 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-kube-controller-manager/revision-pruner-9-crc" event={"ID":"976897c5-e4db-4f3d-bd44-0e4061bdf72b","Type":"ContainerDied","Data":"a7a636ce19173de0f67bf080e664580e7fadbb4b7f039f2262df551f79a4f5c0"} Dec 03 11:21:01 crc kubenswrapper[5052]: I1203 11:21:01.578066 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openshift-kube-apiserver/revision-pruner-8-crc"] Dec 03 11:21:01 crc kubenswrapper[5052]: E1203 11:21:01.579041 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="e43f14fc-f3eb-4f58-b202-cdfc641ac08a" containerName="collect-profiles" Dec 03 11:21:01 crc kubenswrapper[5052]: I1203 11:21:01.579069 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="e43f14fc-f3eb-4f58-b202-cdfc641ac08a" containerName="collect-profiles" Dec 03 11:21:01 crc kubenswrapper[5052]: I1203 11:21:01.579435 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="e43f14fc-f3eb-4f58-b202-cdfc641ac08a" containerName="collect-profiles" Dec 03 11:21:01 crc kubenswrapper[5052]: I1203 11:21:01.580217 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-apiserver/revision-pruner-8-crc" Dec 03 11:21:01 crc kubenswrapper[5052]: I1203 11:21:01.582964 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-kube-apiserver"/"installer-sa-dockercfg-5pr6n" Dec 03 11:21:01 crc kubenswrapper[5052]: I1203 11:21:01.583553 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-kube-apiserver"/"kube-root-ca.crt" Dec 03 11:21:01 crc kubenswrapper[5052]: I1203 11:21:01.587958 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-kube-apiserver/revision-pruner-8-crc"] Dec 03 11:21:01 crc kubenswrapper[5052]: I1203 11:21:01.725211 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c5cfc694-6aed-4138-98e4-2842e1a8f7c8-kubelet-dir\") pod \"revision-pruner-8-crc\" (UID: \"c5cfc694-6aed-4138-98e4-2842e1a8f7c8\") " pod="openshift-kube-apiserver/revision-pruner-8-crc" Dec 03 11:21:01 crc kubenswrapper[5052]: I1203 11:21:01.725380 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access\" (UniqueName: \"kubernetes.io/projected/c5cfc694-6aed-4138-98e4-2842e1a8f7c8-kube-api-access\") pod \"revision-pruner-8-crc\" (UID: \"c5cfc694-6aed-4138-98e4-2842e1a8f7c8\") " pod="openshift-kube-apiserver/revision-pruner-8-crc" Dec 03 11:21:01 crc kubenswrapper[5052]: I1203 11:21:01.828147 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c5cfc694-6aed-4138-98e4-2842e1a8f7c8-kubelet-dir\") pod \"revision-pruner-8-crc\" (UID: \"c5cfc694-6aed-4138-98e4-2842e1a8f7c8\") " pod="openshift-kube-apiserver/revision-pruner-8-crc" Dec 03 11:21:01 crc kubenswrapper[5052]: I1203 11:21:01.828254 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access\" (UniqueName: \"kubernetes.io/projected/c5cfc694-6aed-4138-98e4-2842e1a8f7c8-kube-api-access\") pod \"revision-pruner-8-crc\" (UID: \"c5cfc694-6aed-4138-98e4-2842e1a8f7c8\") " pod="openshift-kube-apiserver/revision-pruner-8-crc" Dec 03 11:21:01 crc kubenswrapper[5052]: I1203 11:21:01.828324 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c5cfc694-6aed-4138-98e4-2842e1a8f7c8-kubelet-dir\") pod \"revision-pruner-8-crc\" (UID: \"c5cfc694-6aed-4138-98e4-2842e1a8f7c8\") " pod="openshift-kube-apiserver/revision-pruner-8-crc" Dec 03 11:21:01 crc kubenswrapper[5052]: I1203 11:21:01.854614 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access\" (UniqueName: \"kubernetes.io/projected/c5cfc694-6aed-4138-98e4-2842e1a8f7c8-kube-api-access\") pod \"revision-pruner-8-crc\" (UID: \"c5cfc694-6aed-4138-98e4-2842e1a8f7c8\") " pod="openshift-kube-apiserver/revision-pruner-8-crc" Dec 03 11:21:01 crc kubenswrapper[5052]: I1203 11:21:01.914806 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-apiserver/revision-pruner-8-crc" Dec 03 11:21:02 crc kubenswrapper[5052]: I1203 11:21:02.247751 5052 patch_prober.go:28] interesting pod/router-default-5444994796-rtl8t container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-http failed: reason withheld Dec 03 11:21:02 crc kubenswrapper[5052]: [-]has-synced failed: reason withheld Dec 03 11:21:02 crc kubenswrapper[5052]: [+]process-running ok Dec 03 11:21:02 crc kubenswrapper[5052]: healthz check failed Dec 03 11:21:02 crc kubenswrapper[5052]: I1203 11:21:02.247831 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-5444994796-rtl8t" podUID="48040744-7fc1-482e-9b68-3a904ba03a8d" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500" Dec 03 11:21:02 crc kubenswrapper[5052]: I1203 11:21:02.639825 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-kube-apiserver/revision-pruner-8-crc"] Dec 03 11:21:02 crc kubenswrapper[5052]: I1203 11:21:02.795872 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-kube-controller-manager/revision-pruner-9-crc" Dec 03 11:21:02 crc kubenswrapper[5052]: I1203 11:21:02.956835 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access\" (UniqueName: \"kubernetes.io/projected/976897c5-e4db-4f3d-bd44-0e4061bdf72b-kube-api-access\") pod \"976897c5-e4db-4f3d-bd44-0e4061bdf72b\" (UID: \"976897c5-e4db-4f3d-bd44-0e4061bdf72b\") " Dec 03 11:21:02 crc kubenswrapper[5052]: I1203 11:21:02.959385 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/976897c5-e4db-4f3d-bd44-0e4061bdf72b-kubelet-dir\") pod \"976897c5-e4db-4f3d-bd44-0e4061bdf72b\" (UID: \"976897c5-e4db-4f3d-bd44-0e4061bdf72b\") " Dec 03 11:21:02 crc kubenswrapper[5052]: I1203 11:21:02.959591 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/976897c5-e4db-4f3d-bd44-0e4061bdf72b-kubelet-dir" (OuterVolumeSpecName: "kubelet-dir") pod "976897c5-e4db-4f3d-bd44-0e4061bdf72b" (UID: "976897c5-e4db-4f3d-bd44-0e4061bdf72b"). InnerVolumeSpecName "kubelet-dir". PluginName "kubernetes.io/host-path", VolumeGidValue "" Dec 03 11:21:02 crc kubenswrapper[5052]: I1203 11:21:02.959964 5052 reconciler_common.go:293] "Volume detached for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/976897c5-e4db-4f3d-bd44-0e4061bdf72b-kubelet-dir\") on node \"crc\" DevicePath \"\"" Dec 03 11:21:02 crc kubenswrapper[5052]: I1203 11:21:02.973827 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/976897c5-e4db-4f3d-bd44-0e4061bdf72b-kube-api-access" (OuterVolumeSpecName: "kube-api-access") pod "976897c5-e4db-4f3d-bd44-0e4061bdf72b" (UID: "976897c5-e4db-4f3d-bd44-0e4061bdf72b"). InnerVolumeSpecName "kube-api-access". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:21:03 crc kubenswrapper[5052]: I1203 11:21:03.061772 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access\" (UniqueName: \"kubernetes.io/projected/976897c5-e4db-4f3d-bd44-0e4061bdf72b-kube-api-access\") on node \"crc\" DevicePath \"\"" Dec 03 11:21:03 crc kubenswrapper[5052]: I1203 11:21:03.248189 5052 patch_prober.go:28] interesting pod/router-default-5444994796-rtl8t container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-http failed: reason withheld Dec 03 11:21:03 crc kubenswrapper[5052]: [-]has-synced failed: reason withheld Dec 03 11:21:03 crc kubenswrapper[5052]: [+]process-running ok Dec 03 11:21:03 crc kubenswrapper[5052]: healthz check failed Dec 03 11:21:03 crc kubenswrapper[5052]: I1203 11:21:03.248266 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-5444994796-rtl8t" podUID="48040744-7fc1-482e-9b68-3a904ba03a8d" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500" Dec 03 11:21:03 crc kubenswrapper[5052]: I1203 11:21:03.476112 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-kube-controller-manager/revision-pruner-9-crc" event={"ID":"976897c5-e4db-4f3d-bd44-0e4061bdf72b","Type":"ContainerDied","Data":"170387cfb74b132f74642d91e20ed7cd74c8275edd8c5a21bdbe5a90c14b6a83"} Dec 03 11:21:03 crc kubenswrapper[5052]: I1203 11:21:03.476481 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="170387cfb74b132f74642d91e20ed7cd74c8275edd8c5a21bdbe5a90c14b6a83" Dec 03 11:21:03 crc kubenswrapper[5052]: I1203 11:21:03.476238 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-kube-controller-manager/revision-pruner-9-crc" Dec 03 11:21:03 crc kubenswrapper[5052]: I1203 11:21:03.484262 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-kube-apiserver/revision-pruner-8-crc" event={"ID":"c5cfc694-6aed-4138-98e4-2842e1a8f7c8","Type":"ContainerStarted","Data":"286b424e6e3c9046b692b7d7588d54265710a7e452c9d64cbb25bb6d54126d31"} Dec 03 11:21:03 crc kubenswrapper[5052]: I1203 11:21:03.538324 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-kube-scheduler/openshift-kube-scheduler-crc" Dec 03 11:21:03 crc kubenswrapper[5052]: I1203 11:21:03.567010 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openshift-kube-scheduler/openshift-kube-scheduler-crc"] Dec 03 11:21:04 crc kubenswrapper[5052]: I1203 11:21:04.249184 5052 patch_prober.go:28] interesting pod/router-default-5444994796-rtl8t container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-http failed: reason withheld Dec 03 11:21:04 crc kubenswrapper[5052]: [-]has-synced failed: reason withheld Dec 03 11:21:04 crc kubenswrapper[5052]: [+]process-running ok Dec 03 11:21:04 crc kubenswrapper[5052]: healthz check failed Dec 03 11:21:04 crc kubenswrapper[5052]: I1203 11:21:04.249259 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-5444994796-rtl8t" podUID="48040744-7fc1-482e-9b68-3a904ba03a8d" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500" Dec 03 11:21:04 crc kubenswrapper[5052]: I1203 11:21:04.498946 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-6m2jn" Dec 03 11:21:04 crc kubenswrapper[5052]: I1203 11:21:04.503104 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-kube-apiserver/revision-pruner-8-crc" event={"ID":"c5cfc694-6aed-4138-98e4-2842e1a8f7c8","Type":"ContainerStarted","Data":"efb1290421afdf5cda42d0e6e5b7ece6af358fc8da87c7c63196bf6317331dd1"} Dec 03 11:21:04 crc kubenswrapper[5052]: I1203 11:21:04.528119 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-kube-scheduler/openshift-kube-scheduler-crc" podStartSLOduration=1.528091486 podStartE2EDuration="1.528091486s" podCreationTimestamp="2025-12-03 11:21:03 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:21:04.527202791 +0000 UTC m=+55.115444555" watchObservedRunningTime="2025-12-03 11:21:04.528091486 +0000 UTC m=+55.116333250" Dec 03 11:21:04 crc kubenswrapper[5052]: I1203 11:21:04.566986 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-kube-apiserver/revision-pruner-8-crc" podStartSLOduration=3.5669634869999998 podStartE2EDuration="3.566963487s" podCreationTimestamp="2025-12-03 11:21:01 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:21:04.56527309 +0000 UTC m=+55.153514854" watchObservedRunningTime="2025-12-03 11:21:04.566963487 +0000 UTC m=+55.155205271" Dec 03 11:21:04 crc kubenswrapper[5052]: I1203 11:21:04.595921 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-apiserver/apiserver-76f77b778f-dmvst" Dec 03 11:21:04 crc kubenswrapper[5052]: I1203 11:21:04.602570 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-apiserver/apiserver-76f77b778f-dmvst" Dec 03 11:21:05 crc kubenswrapper[5052]: I1203 11:21:05.256500 5052 patch_prober.go:28] interesting pod/router-default-5444994796-rtl8t container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-http failed: reason withheld Dec 03 11:21:05 crc kubenswrapper[5052]: [-]has-synced failed: reason withheld Dec 03 11:21:05 crc kubenswrapper[5052]: [+]process-running ok Dec 03 11:21:05 crc kubenswrapper[5052]: healthz check failed Dec 03 11:21:05 crc kubenswrapper[5052]: I1203 11:21:05.256602 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-5444994796-rtl8t" podUID="48040744-7fc1-482e-9b68-3a904ba03a8d" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500" Dec 03 11:21:05 crc kubenswrapper[5052]: I1203 11:21:05.544090 5052 generic.go:334] "Generic (PLEG): container finished" podID="c5cfc694-6aed-4138-98e4-2842e1a8f7c8" containerID="efb1290421afdf5cda42d0e6e5b7ece6af358fc8da87c7c63196bf6317331dd1" exitCode=0 Dec 03 11:21:05 crc kubenswrapper[5052]: I1203 11:21:05.544151 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-kube-apiserver/revision-pruner-8-crc" event={"ID":"c5cfc694-6aed-4138-98e4-2842e1a8f7c8","Type":"ContainerDied","Data":"efb1290421afdf5cda42d0e6e5b7ece6af358fc8da87c7c63196bf6317331dd1"} Dec 03 11:21:06 crc kubenswrapper[5052]: I1203 11:21:06.246913 5052 patch_prober.go:28] interesting pod/router-default-5444994796-rtl8t container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-http failed: reason withheld Dec 03 11:21:06 crc kubenswrapper[5052]: [-]has-synced failed: reason withheld Dec 03 11:21:06 crc kubenswrapper[5052]: [+]process-running ok Dec 03 11:21:06 crc kubenswrapper[5052]: healthz check failed Dec 03 11:21:06 crc kubenswrapper[5052]: I1203 11:21:06.246992 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-5444994796-rtl8t" podUID="48040744-7fc1-482e-9b68-3a904ba03a8d" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500" Dec 03 11:21:06 crc kubenswrapper[5052]: I1203 11:21:06.915950 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-kube-apiserver/revision-pruner-8-crc" Dec 03 11:21:07 crc kubenswrapper[5052]: I1203 11:21:07.032121 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c5cfc694-6aed-4138-98e4-2842e1a8f7c8-kubelet-dir\") pod \"c5cfc694-6aed-4138-98e4-2842e1a8f7c8\" (UID: \"c5cfc694-6aed-4138-98e4-2842e1a8f7c8\") " Dec 03 11:21:07 crc kubenswrapper[5052]: I1203 11:21:07.032632 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access\" (UniqueName: \"kubernetes.io/projected/c5cfc694-6aed-4138-98e4-2842e1a8f7c8-kube-api-access\") pod \"c5cfc694-6aed-4138-98e4-2842e1a8f7c8\" (UID: \"c5cfc694-6aed-4138-98e4-2842e1a8f7c8\") " Dec 03 11:21:07 crc kubenswrapper[5052]: I1203 11:21:07.032257 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/c5cfc694-6aed-4138-98e4-2842e1a8f7c8-kubelet-dir" (OuterVolumeSpecName: "kubelet-dir") pod "c5cfc694-6aed-4138-98e4-2842e1a8f7c8" (UID: "c5cfc694-6aed-4138-98e4-2842e1a8f7c8"). InnerVolumeSpecName "kubelet-dir". PluginName "kubernetes.io/host-path", VolumeGidValue "" Dec 03 11:21:07 crc kubenswrapper[5052]: I1203 11:21:07.032969 5052 reconciler_common.go:293] "Volume detached for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c5cfc694-6aed-4138-98e4-2842e1a8f7c8-kubelet-dir\") on node \"crc\" DevicePath \"\"" Dec 03 11:21:07 crc kubenswrapper[5052]: I1203 11:21:07.041802 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c5cfc694-6aed-4138-98e4-2842e1a8f7c8-kube-api-access" (OuterVolumeSpecName: "kube-api-access") pod "c5cfc694-6aed-4138-98e4-2842e1a8f7c8" (UID: "c5cfc694-6aed-4138-98e4-2842e1a8f7c8"). InnerVolumeSpecName "kube-api-access". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:21:07 crc kubenswrapper[5052]: I1203 11:21:07.136061 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access\" (UniqueName: \"kubernetes.io/projected/c5cfc694-6aed-4138-98e4-2842e1a8f7c8-kube-api-access\") on node \"crc\" DevicePath \"\"" Dec 03 11:21:07 crc kubenswrapper[5052]: I1203 11:21:07.248724 5052 patch_prober.go:28] interesting pod/router-default-5444994796-rtl8t container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-http failed: reason withheld Dec 03 11:21:07 crc kubenswrapper[5052]: [-]has-synced failed: reason withheld Dec 03 11:21:07 crc kubenswrapper[5052]: [+]process-running ok Dec 03 11:21:07 crc kubenswrapper[5052]: healthz check failed Dec 03 11:21:07 crc kubenswrapper[5052]: I1203 11:21:07.248799 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-5444994796-rtl8t" podUID="48040744-7fc1-482e-9b68-3a904ba03a8d" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500" Dec 03 11:21:07 crc kubenswrapper[5052]: I1203 11:21:07.566310 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-kube-apiserver/revision-pruner-8-crc" event={"ID":"c5cfc694-6aed-4138-98e4-2842e1a8f7c8","Type":"ContainerDied","Data":"286b424e6e3c9046b692b7d7588d54265710a7e452c9d64cbb25bb6d54126d31"} Dec 03 11:21:07 crc kubenswrapper[5052]: I1203 11:21:07.566374 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="286b424e6e3c9046b692b7d7588d54265710a7e452c9d64cbb25bb6d54126d31" Dec 03 11:21:07 crc kubenswrapper[5052]: I1203 11:21:07.566462 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-kube-apiserver/revision-pruner-8-crc" Dec 03 11:21:08 crc kubenswrapper[5052]: I1203 11:21:08.414083 5052 patch_prober.go:28] interesting pod/router-default-5444994796-rtl8t container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-http failed: reason withheld Dec 03 11:21:08 crc kubenswrapper[5052]: [+]has-synced ok Dec 03 11:21:08 crc kubenswrapper[5052]: [+]process-running ok Dec 03 11:21:08 crc kubenswrapper[5052]: healthz check failed Dec 03 11:21:08 crc kubenswrapper[5052]: I1203 11:21:08.414160 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-5444994796-rtl8t" podUID="48040744-7fc1-482e-9b68-3a904ba03a8d" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500" Dec 03 11:21:08 crc kubenswrapper[5052]: I1203 11:21:08.437257 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-7954f5f757-l7xtl" Dec 03 11:21:09 crc kubenswrapper[5052]: I1203 11:21:09.251474 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-5444994796-rtl8t" Dec 03 11:21:09 crc kubenswrapper[5052]: I1203 11:21:09.255403 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-5444994796-rtl8t" Dec 03 11:21:09 crc kubenswrapper[5052]: E1203 11:21:09.509468 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="b0103da56f9128a13b28c52c5f8402db976339938594d6eae2a65c5f008b27ef" cmd=["/bin/bash","-c","test -f /ready/ready"] Dec 03 11:21:09 crc kubenswrapper[5052]: E1203 11:21:09.513831 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="b0103da56f9128a13b28c52c5f8402db976339938594d6eae2a65c5f008b27ef" cmd=["/bin/bash","-c","test -f /ready/ready"] Dec 03 11:21:09 crc kubenswrapper[5052]: E1203 11:21:09.515412 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="b0103da56f9128a13b28c52c5f8402db976339938594d6eae2a65c5f008b27ef" cmd=["/bin/bash","-c","test -f /ready/ready"] Dec 03 11:21:09 crc kubenswrapper[5052]: E1203 11:21:09.515596 5052 prober.go:104] "Probe errored" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" probeType="Readiness" pod="openshift-multus/cni-sysctl-allowlist-ds-8l9gm" podUID="bf761cb9-e024-4acd-a947-0f8064267243" containerName="kube-multus-additional-cni-plugins" Dec 03 11:21:09 crc kubenswrapper[5052]: I1203 11:21:09.649894 5052 patch_prober.go:28] interesting pod/console-f9d7485db-568dj container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.217.0.22:8443/health\": dial tcp 10.217.0.22:8443: connect: connection refused" start-of-body= Dec 03 11:21:09 crc kubenswrapper[5052]: I1203 11:21:09.649960 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openshift-console/console-f9d7485db-568dj" podUID="b1d2ff08-d58b-48d0-a0e5-6cfd867a63de" containerName="console" probeResult="failure" output="Get \"https://10.217.0.22:8443/health\": dial tcp 10.217.0.22:8443: connect: connection refused" Dec 03 11:21:18 crc kubenswrapper[5052]: I1203 11:21:18.414270 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-697d97f7c8-p77bx" Dec 03 11:21:19 crc kubenswrapper[5052]: E1203 11:21:19.511088 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="b0103da56f9128a13b28c52c5f8402db976339938594d6eae2a65c5f008b27ef" cmd=["/bin/bash","-c","test -f /ready/ready"] Dec 03 11:21:19 crc kubenswrapper[5052]: E1203 11:21:19.512798 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="b0103da56f9128a13b28c52c5f8402db976339938594d6eae2a65c5f008b27ef" cmd=["/bin/bash","-c","test -f /ready/ready"] Dec 03 11:21:19 crc kubenswrapper[5052]: E1203 11:21:19.513969 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="b0103da56f9128a13b28c52c5f8402db976339938594d6eae2a65c5f008b27ef" cmd=["/bin/bash","-c","test -f /ready/ready"] Dec 03 11:21:19 crc kubenswrapper[5052]: E1203 11:21:19.514016 5052 prober.go:104] "Probe errored" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" probeType="Readiness" pod="openshift-multus/cni-sysctl-allowlist-ds-8l9gm" podUID="bf761cb9-e024-4acd-a947-0f8064267243" containerName="kube-multus-additional-cni-plugins" Dec 03 11:21:19 crc kubenswrapper[5052]: I1203 11:21:19.654575 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-f9d7485db-568dj" Dec 03 11:21:19 crc kubenswrapper[5052]: I1203 11:21:19.658396 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-f9d7485db-568dj" Dec 03 11:21:23 crc kubenswrapper[5052]: I1203 11:21:23.690575 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-crc"] Dec 03 11:21:27 crc kubenswrapper[5052]: I1203 11:21:27.784111 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_cni-sysctl-allowlist-ds-8l9gm_bf761cb9-e024-4acd-a947-0f8064267243/kube-multus-additional-cni-plugins/0.log" Dec 03 11:21:27 crc kubenswrapper[5052]: I1203 11:21:27.784446 5052 generic.go:334] "Generic (PLEG): container finished" podID="bf761cb9-e024-4acd-a947-0f8064267243" containerID="b0103da56f9128a13b28c52c5f8402db976339938594d6eae2a65c5f008b27ef" exitCode=137 Dec 03 11:21:27 crc kubenswrapper[5052]: I1203 11:21:27.784489 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-multus/cni-sysctl-allowlist-ds-8l9gm" event={"ID":"bf761cb9-e024-4acd-a947-0f8064267243","Type":"ContainerDied","Data":"b0103da56f9128a13b28c52c5f8402db976339938594d6eae2a65c5f008b27ef"} Dec 03 11:21:28 crc kubenswrapper[5052]: I1203 11:21:28.614603 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-xd92c" Dec 03 11:21:28 crc kubenswrapper[5052]: I1203 11:21:28.641201 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-crc" podStartSLOduration=5.641177161 podStartE2EDuration="5.641177161s" podCreationTimestamp="2025-12-03 11:21:23 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:21:28.635501114 +0000 UTC m=+79.223742888" watchObservedRunningTime="2025-12-03 11:21:28.641177161 +0000 UTC m=+79.229418925" Dec 03 11:21:29 crc kubenswrapper[5052]: E1203 11:21:29.507774 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of b0103da56f9128a13b28c52c5f8402db976339938594d6eae2a65c5f008b27ef is running failed: container process not found" containerID="b0103da56f9128a13b28c52c5f8402db976339938594d6eae2a65c5f008b27ef" cmd=["/bin/bash","-c","test -f /ready/ready"] Dec 03 11:21:29 crc kubenswrapper[5052]: E1203 11:21:29.508238 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of b0103da56f9128a13b28c52c5f8402db976339938594d6eae2a65c5f008b27ef is running failed: container process not found" containerID="b0103da56f9128a13b28c52c5f8402db976339938594d6eae2a65c5f008b27ef" cmd=["/bin/bash","-c","test -f /ready/ready"] Dec 03 11:21:29 crc kubenswrapper[5052]: E1203 11:21:29.508554 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of b0103da56f9128a13b28c52c5f8402db976339938594d6eae2a65c5f008b27ef is running failed: container process not found" containerID="b0103da56f9128a13b28c52c5f8402db976339938594d6eae2a65c5f008b27ef" cmd=["/bin/bash","-c","test -f /ready/ready"] Dec 03 11:21:29 crc kubenswrapper[5052]: E1203 11:21:29.508581 5052 prober.go:104] "Probe errored" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of b0103da56f9128a13b28c52c5f8402db976339938594d6eae2a65c5f008b27ef is running failed: container process not found" probeType="Readiness" pod="openshift-multus/cni-sysctl-allowlist-ds-8l9gm" podUID="bf761cb9-e024-4acd-a947-0f8064267243" containerName="kube-multus-additional-cni-plugins" Dec 03 11:21:29 crc kubenswrapper[5052]: I1203 11:21:29.700507 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-operator-lifecycle-manager/package-server-manager-789f6589d5-577rn" Dec 03 11:21:37 crc kubenswrapper[5052]: I1203 11:21:37.168639 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openshift-kube-apiserver/revision-pruner-9-crc"] Dec 03 11:21:37 crc kubenswrapper[5052]: E1203 11:21:37.169614 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="c5cfc694-6aed-4138-98e4-2842e1a8f7c8" containerName="pruner" Dec 03 11:21:37 crc kubenswrapper[5052]: I1203 11:21:37.169632 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="c5cfc694-6aed-4138-98e4-2842e1a8f7c8" containerName="pruner" Dec 03 11:21:37 crc kubenswrapper[5052]: E1203 11:21:37.169648 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="976897c5-e4db-4f3d-bd44-0e4061bdf72b" containerName="pruner" Dec 03 11:21:37 crc kubenswrapper[5052]: I1203 11:21:37.169657 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="976897c5-e4db-4f3d-bd44-0e4061bdf72b" containerName="pruner" Dec 03 11:21:37 crc kubenswrapper[5052]: I1203 11:21:37.169775 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="976897c5-e4db-4f3d-bd44-0e4061bdf72b" containerName="pruner" Dec 03 11:21:37 crc kubenswrapper[5052]: I1203 11:21:37.169793 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="c5cfc694-6aed-4138-98e4-2842e1a8f7c8" containerName="pruner" Dec 03 11:21:37 crc kubenswrapper[5052]: I1203 11:21:37.170364 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-apiserver/revision-pruner-9-crc" Dec 03 11:21:37 crc kubenswrapper[5052]: I1203 11:21:37.172777 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-kube-apiserver"/"kube-root-ca.crt" Dec 03 11:21:37 crc kubenswrapper[5052]: I1203 11:21:37.175824 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-kube-apiserver"/"installer-sa-dockercfg-5pr6n" Dec 03 11:21:37 crc kubenswrapper[5052]: I1203 11:21:37.180167 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-kube-apiserver/revision-pruner-9-crc"] Dec 03 11:21:37 crc kubenswrapper[5052]: I1203 11:21:37.235712 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access\" (UniqueName: \"kubernetes.io/projected/0b5fd713-7392-4060-a6bb-0dc6938a16e0-kube-api-access\") pod \"revision-pruner-9-crc\" (UID: \"0b5fd713-7392-4060-a6bb-0dc6938a16e0\") " pod="openshift-kube-apiserver/revision-pruner-9-crc" Dec 03 11:21:37 crc kubenswrapper[5052]: I1203 11:21:37.235791 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/0b5fd713-7392-4060-a6bb-0dc6938a16e0-kubelet-dir\") pod \"revision-pruner-9-crc\" (UID: \"0b5fd713-7392-4060-a6bb-0dc6938a16e0\") " pod="openshift-kube-apiserver/revision-pruner-9-crc" Dec 03 11:21:37 crc kubenswrapper[5052]: I1203 11:21:37.337230 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access\" (UniqueName: \"kubernetes.io/projected/0b5fd713-7392-4060-a6bb-0dc6938a16e0-kube-api-access\") pod \"revision-pruner-9-crc\" (UID: \"0b5fd713-7392-4060-a6bb-0dc6938a16e0\") " pod="openshift-kube-apiserver/revision-pruner-9-crc" Dec 03 11:21:37 crc kubenswrapper[5052]: I1203 11:21:37.337306 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/0b5fd713-7392-4060-a6bb-0dc6938a16e0-kubelet-dir\") pod \"revision-pruner-9-crc\" (UID: \"0b5fd713-7392-4060-a6bb-0dc6938a16e0\") " pod="openshift-kube-apiserver/revision-pruner-9-crc" Dec 03 11:21:37 crc kubenswrapper[5052]: I1203 11:21:37.337444 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/0b5fd713-7392-4060-a6bb-0dc6938a16e0-kubelet-dir\") pod \"revision-pruner-9-crc\" (UID: \"0b5fd713-7392-4060-a6bb-0dc6938a16e0\") " pod="openshift-kube-apiserver/revision-pruner-9-crc" Dec 03 11:21:37 crc kubenswrapper[5052]: I1203 11:21:37.364615 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access\" (UniqueName: \"kubernetes.io/projected/0b5fd713-7392-4060-a6bb-0dc6938a16e0-kube-api-access\") pod \"revision-pruner-9-crc\" (UID: \"0b5fd713-7392-4060-a6bb-0dc6938a16e0\") " pod="openshift-kube-apiserver/revision-pruner-9-crc" Dec 03 11:21:37 crc kubenswrapper[5052]: I1203 11:21:37.610946 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-apiserver/revision-pruner-9-crc" Dec 03 11:21:39 crc kubenswrapper[5052]: E1203 11:21:39.507435 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of b0103da56f9128a13b28c52c5f8402db976339938594d6eae2a65c5f008b27ef is running failed: container process not found" containerID="b0103da56f9128a13b28c52c5f8402db976339938594d6eae2a65c5f008b27ef" cmd=["/bin/bash","-c","test -f /ready/ready"] Dec 03 11:21:39 crc kubenswrapper[5052]: E1203 11:21:39.508166 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of b0103da56f9128a13b28c52c5f8402db976339938594d6eae2a65c5f008b27ef is running failed: container process not found" containerID="b0103da56f9128a13b28c52c5f8402db976339938594d6eae2a65c5f008b27ef" cmd=["/bin/bash","-c","test -f /ready/ready"] Dec 03 11:21:39 crc kubenswrapper[5052]: E1203 11:21:39.508567 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of b0103da56f9128a13b28c52c5f8402db976339938594d6eae2a65c5f008b27ef is running failed: container process not found" containerID="b0103da56f9128a13b28c52c5f8402db976339938594d6eae2a65c5f008b27ef" cmd=["/bin/bash","-c","test -f /ready/ready"] Dec 03 11:21:39 crc kubenswrapper[5052]: E1203 11:21:39.508660 5052 prober.go:104] "Probe errored" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of b0103da56f9128a13b28c52c5f8402db976339938594d6eae2a65c5f008b27ef is running failed: container process not found" probeType="Readiness" pod="openshift-multus/cni-sysctl-allowlist-ds-8l9gm" podUID="bf761cb9-e024-4acd-a947-0f8064267243" containerName="kube-multus-additional-cni-plugins" Dec 03 11:21:41 crc kubenswrapper[5052]: I1203 11:21:41.009957 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_cni-sysctl-allowlist-ds-8l9gm_bf761cb9-e024-4acd-a947-0f8064267243/kube-multus-additional-cni-plugins/0.log" Dec 03 11:21:41 crc kubenswrapper[5052]: I1203 11:21:41.010328 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-multus/cni-sysctl-allowlist-ds-8l9gm" Dec 03 11:21:41 crc kubenswrapper[5052]: I1203 11:21:41.061705 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c5vsd\" (UniqueName: \"kubernetes.io/projected/bf761cb9-e024-4acd-a947-0f8064267243-kube-api-access-c5vsd\") pod \"bf761cb9-e024-4acd-a947-0f8064267243\" (UID: \"bf761cb9-e024-4acd-a947-0f8064267243\") " Dec 03 11:21:41 crc kubenswrapper[5052]: I1203 11:21:41.061773 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/bf761cb9-e024-4acd-a947-0f8064267243-tuning-conf-dir\") pod \"bf761cb9-e024-4acd-a947-0f8064267243\" (UID: \"bf761cb9-e024-4acd-a947-0f8064267243\") " Dec 03 11:21:41 crc kubenswrapper[5052]: I1203 11:21:41.061929 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/bf761cb9-e024-4acd-a947-0f8064267243-cni-sysctl-allowlist\") pod \"bf761cb9-e024-4acd-a947-0f8064267243\" (UID: \"bf761cb9-e024-4acd-a947-0f8064267243\") " Dec 03 11:21:41 crc kubenswrapper[5052]: I1203 11:21:41.061950 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/bf761cb9-e024-4acd-a947-0f8064267243-tuning-conf-dir" (OuterVolumeSpecName: "tuning-conf-dir") pod "bf761cb9-e024-4acd-a947-0f8064267243" (UID: "bf761cb9-e024-4acd-a947-0f8064267243"). InnerVolumeSpecName "tuning-conf-dir". PluginName "kubernetes.io/host-path", VolumeGidValue "" Dec 03 11:21:41 crc kubenswrapper[5052]: I1203 11:21:41.062030 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ready\" (UniqueName: \"kubernetes.io/empty-dir/bf761cb9-e024-4acd-a947-0f8064267243-ready\") pod \"bf761cb9-e024-4acd-a947-0f8064267243\" (UID: \"bf761cb9-e024-4acd-a947-0f8064267243\") " Dec 03 11:21:41 crc kubenswrapper[5052]: I1203 11:21:41.062408 5052 reconciler_common.go:293] "Volume detached for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/bf761cb9-e024-4acd-a947-0f8064267243-tuning-conf-dir\") on node \"crc\" DevicePath \"\"" Dec 03 11:21:41 crc kubenswrapper[5052]: I1203 11:21:41.062459 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bf761cb9-e024-4acd-a947-0f8064267243-ready" (OuterVolumeSpecName: "ready") pod "bf761cb9-e024-4acd-a947-0f8064267243" (UID: "bf761cb9-e024-4acd-a947-0f8064267243"). InnerVolumeSpecName "ready". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:21:41 crc kubenswrapper[5052]: I1203 11:21:41.062834 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bf761cb9-e024-4acd-a947-0f8064267243-cni-sysctl-allowlist" (OuterVolumeSpecName: "cni-sysctl-allowlist") pod "bf761cb9-e024-4acd-a947-0f8064267243" (UID: "bf761cb9-e024-4acd-a947-0f8064267243"). InnerVolumeSpecName "cni-sysctl-allowlist". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:21:41 crc kubenswrapper[5052]: I1203 11:21:41.075459 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bf761cb9-e024-4acd-a947-0f8064267243-kube-api-access-c5vsd" (OuterVolumeSpecName: "kube-api-access-c5vsd") pod "bf761cb9-e024-4acd-a947-0f8064267243" (UID: "bf761cb9-e024-4acd-a947-0f8064267243"). InnerVolumeSpecName "kube-api-access-c5vsd". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:21:41 crc kubenswrapper[5052]: I1203 11:21:41.164171 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-c5vsd\" (UniqueName: \"kubernetes.io/projected/bf761cb9-e024-4acd-a947-0f8064267243-kube-api-access-c5vsd\") on node \"crc\" DevicePath \"\"" Dec 03 11:21:41 crc kubenswrapper[5052]: I1203 11:21:41.164226 5052 reconciler_common.go:293] "Volume detached for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/bf761cb9-e024-4acd-a947-0f8064267243-cni-sysctl-allowlist\") on node \"crc\" DevicePath \"\"" Dec 03 11:21:41 crc kubenswrapper[5052]: I1203 11:21:41.164241 5052 reconciler_common.go:293] "Volume detached for volume \"ready\" (UniqueName: \"kubernetes.io/empty-dir/bf761cb9-e024-4acd-a947-0f8064267243-ready\") on node \"crc\" DevicePath \"\"" Dec 03 11:21:41 crc kubenswrapper[5052]: I1203 11:21:41.559964 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openshift-kube-apiserver/installer-9-crc"] Dec 03 11:21:41 crc kubenswrapper[5052]: E1203 11:21:41.560348 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="bf761cb9-e024-4acd-a947-0f8064267243" containerName="kube-multus-additional-cni-plugins" Dec 03 11:21:41 crc kubenswrapper[5052]: I1203 11:21:41.560379 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="bf761cb9-e024-4acd-a947-0f8064267243" containerName="kube-multus-additional-cni-plugins" Dec 03 11:21:41 crc kubenswrapper[5052]: I1203 11:21:41.560501 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="bf761cb9-e024-4acd-a947-0f8064267243" containerName="kube-multus-additional-cni-plugins" Dec 03 11:21:41 crc kubenswrapper[5052]: I1203 11:21:41.561132 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-apiserver/installer-9-crc" Dec 03 11:21:41 crc kubenswrapper[5052]: I1203 11:21:41.569972 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access\" (UniqueName: \"kubernetes.io/projected/68a245d3-ddaf-40a0-92f6-92760173b7a3-kube-api-access\") pod \"installer-9-crc\" (UID: \"68a245d3-ddaf-40a0-92f6-92760173b7a3\") " pod="openshift-kube-apiserver/installer-9-crc" Dec 03 11:21:41 crc kubenswrapper[5052]: I1203 11:21:41.570024 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/68a245d3-ddaf-40a0-92f6-92760173b7a3-kubelet-dir\") pod \"installer-9-crc\" (UID: \"68a245d3-ddaf-40a0-92f6-92760173b7a3\") " pod="openshift-kube-apiserver/installer-9-crc" Dec 03 11:21:41 crc kubenswrapper[5052]: I1203 11:21:41.570132 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lock\" (UniqueName: \"kubernetes.io/host-path/68a245d3-ddaf-40a0-92f6-92760173b7a3-var-lock\") pod \"installer-9-crc\" (UID: \"68a245d3-ddaf-40a0-92f6-92760173b7a3\") " pod="openshift-kube-apiserver/installer-9-crc" Dec 03 11:21:41 crc kubenswrapper[5052]: I1203 11:21:41.577216 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-kube-apiserver/installer-9-crc"] Dec 03 11:21:41 crc kubenswrapper[5052]: I1203 11:21:41.671440 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"var-lock\" (UniqueName: \"kubernetes.io/host-path/68a245d3-ddaf-40a0-92f6-92760173b7a3-var-lock\") pod \"installer-9-crc\" (UID: \"68a245d3-ddaf-40a0-92f6-92760173b7a3\") " pod="openshift-kube-apiserver/installer-9-crc" Dec 03 11:21:41 crc kubenswrapper[5052]: I1203 11:21:41.671528 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access\" (UniqueName: \"kubernetes.io/projected/68a245d3-ddaf-40a0-92f6-92760173b7a3-kube-api-access\") pod \"installer-9-crc\" (UID: \"68a245d3-ddaf-40a0-92f6-92760173b7a3\") " pod="openshift-kube-apiserver/installer-9-crc" Dec 03 11:21:41 crc kubenswrapper[5052]: I1203 11:21:41.671564 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/68a245d3-ddaf-40a0-92f6-92760173b7a3-kubelet-dir\") pod \"installer-9-crc\" (UID: \"68a245d3-ddaf-40a0-92f6-92760173b7a3\") " pod="openshift-kube-apiserver/installer-9-crc" Dec 03 11:21:41 crc kubenswrapper[5052]: I1203 11:21:41.671684 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/68a245d3-ddaf-40a0-92f6-92760173b7a3-kubelet-dir\") pod \"installer-9-crc\" (UID: \"68a245d3-ddaf-40a0-92f6-92760173b7a3\") " pod="openshift-kube-apiserver/installer-9-crc" Dec 03 11:21:41 crc kubenswrapper[5052]: I1203 11:21:41.671672 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"var-lock\" (UniqueName: \"kubernetes.io/host-path/68a245d3-ddaf-40a0-92f6-92760173b7a3-var-lock\") pod \"installer-9-crc\" (UID: \"68a245d3-ddaf-40a0-92f6-92760173b7a3\") " pod="openshift-kube-apiserver/installer-9-crc" Dec 03 11:21:41 crc kubenswrapper[5052]: I1203 11:21:41.689844 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access\" (UniqueName: \"kubernetes.io/projected/68a245d3-ddaf-40a0-92f6-92760173b7a3-kube-api-access\") pod \"installer-9-crc\" (UID: \"68a245d3-ddaf-40a0-92f6-92760173b7a3\") " pod="openshift-kube-apiserver/installer-9-crc" Dec 03 11:21:41 crc kubenswrapper[5052]: I1203 11:21:41.913036 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-apiserver/installer-9-crc" Dec 03 11:21:41 crc kubenswrapper[5052]: I1203 11:21:41.936600 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_cni-sysctl-allowlist-ds-8l9gm_bf761cb9-e024-4acd-a947-0f8064267243/kube-multus-additional-cni-plugins/0.log" Dec 03 11:21:41 crc kubenswrapper[5052]: I1203 11:21:41.936693 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-multus/cni-sysctl-allowlist-ds-8l9gm" event={"ID":"bf761cb9-e024-4acd-a947-0f8064267243","Type":"ContainerDied","Data":"aaadb8e89911de41a45561de348fecea7160e3a0227674f4f4516c5d5ef9a151"} Dec 03 11:21:41 crc kubenswrapper[5052]: I1203 11:21:41.936778 5052 scope.go:117] "RemoveContainer" containerID="b0103da56f9128a13b28c52c5f8402db976339938594d6eae2a65c5f008b27ef" Dec 03 11:21:41 crc kubenswrapper[5052]: I1203 11:21:41.936947 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-multus/cni-sysctl-allowlist-ds-8l9gm" Dec 03 11:21:41 crc kubenswrapper[5052]: I1203 11:21:41.967149 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openshift-multus/cni-sysctl-allowlist-ds-8l9gm"] Dec 03 11:21:41 crc kubenswrapper[5052]: I1203 11:21:41.971187 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openshift-multus/cni-sysctl-allowlist-ds-8l9gm"] Dec 03 11:21:42 crc kubenswrapper[5052]: E1203 11:21:42.525267 5052 log.go:32] "PullImage from image service failed" err="rpc error: code = Canceled desc = copying system image from manifest list: copying config: context canceled" image="registry.redhat.io/redhat/redhat-marketplace-index:v4.18" Dec 03 11:21:42 crc kubenswrapper[5052]: E1203 11:21:42.526082 5052 kuberuntime_manager.go:1274] "Unhandled Error" err="init container &Container{Name:extract-content,Image:registry.redhat.io/redhat/redhat-marketplace-index:v4.18,Command:[/utilities/copy-content],Args:[--catalog.from=/configs --catalog.to=/extracted-catalog/catalog --cache.from=/tmp/cache --cache.to=/extracted-catalog/cache],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:utilities,ReadOnly:false,MountPath:/utilities,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:catalog-content,ReadOnly:false,MountPath:/extracted-catalog,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-lssp4,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000170000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:nil,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:FallbackToLogsOnError,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod redhat-marketplace-qttdj_openshift-marketplace(8cf9b98e-2ea9-47ee-bd1b-a913714b7668): ErrImagePull: rpc error: code = Canceled desc = copying system image from manifest list: copying config: context canceled" logger="UnhandledError" Dec 03 11:21:42 crc kubenswrapper[5052]: E1203 11:21:42.527376 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"extract-content\" with ErrImagePull: \"rpc error: code = Canceled desc = copying system image from manifest list: copying config: context canceled\"" pod="openshift-marketplace/redhat-marketplace-qttdj" podUID="8cf9b98e-2ea9-47ee-bd1b-a913714b7668" Dec 03 11:21:43 crc kubenswrapper[5052]: I1203 11:21:43.682761 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bf761cb9-e024-4acd-a947-0f8064267243" path="/var/lib/kubelet/pods/bf761cb9-e024-4acd-a947-0f8064267243/volumes" Dec 03 11:21:44 crc kubenswrapper[5052]: E1203 11:21:44.344873 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"extract-content\" with ImagePullBackOff: \"Back-off pulling image \\\"registry.redhat.io/redhat/redhat-marketplace-index:v4.18\\\"\"" pod="openshift-marketplace/redhat-marketplace-qttdj" podUID="8cf9b98e-2ea9-47ee-bd1b-a913714b7668" Dec 03 11:21:44 crc kubenswrapper[5052]: E1203 11:21:44.474380 5052 log.go:32] "PullImage from image service failed" err="rpc error: code = Canceled desc = copying system image from manifest list: copying config: context canceled" image="registry.redhat.io/redhat/certified-operator-index:v4.18" Dec 03 11:21:44 crc kubenswrapper[5052]: E1203 11:21:44.475093 5052 kuberuntime_manager.go:1274] "Unhandled Error" err="init container &Container{Name:extract-content,Image:registry.redhat.io/redhat/certified-operator-index:v4.18,Command:[/utilities/copy-content],Args:[--catalog.from=/configs --catalog.to=/extracted-catalog/catalog --cache.from=/tmp/cache --cache.to=/extracted-catalog/cache],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:utilities,ReadOnly:false,MountPath:/utilities,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:catalog-content,ReadOnly:false,MountPath:/extracted-catalog,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-zn4pm,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000170000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:nil,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:FallbackToLogsOnError,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod certified-operators-99svv_openshift-marketplace(3b1b1a53-eff1-4339-a82e-9b8400dd5604): ErrImagePull: rpc error: code = Canceled desc = copying system image from manifest list: copying config: context canceled" logger="UnhandledError" Dec 03 11:21:44 crc kubenswrapper[5052]: E1203 11:21:44.476324 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"extract-content\" with ErrImagePull: \"rpc error: code = Canceled desc = copying system image from manifest list: copying config: context canceled\"" pod="openshift-marketplace/certified-operators-99svv" podUID="3b1b1a53-eff1-4339-a82e-9b8400dd5604" Dec 03 11:21:46 crc kubenswrapper[5052]: E1203 11:21:46.071492 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"extract-content\" with ImagePullBackOff: \"Back-off pulling image \\\"registry.redhat.io/redhat/certified-operator-index:v4.18\\\"\"" pod="openshift-marketplace/certified-operators-99svv" podUID="3b1b1a53-eff1-4339-a82e-9b8400dd5604" Dec 03 11:21:46 crc kubenswrapper[5052]: E1203 11:21:46.219884 5052 log.go:32] "PullImage from image service failed" err="rpc error: code = Canceled desc = copying system image from manifest list: copying config: context canceled" image="registry.redhat.io/redhat/community-operator-index:v4.18" Dec 03 11:21:46 crc kubenswrapper[5052]: E1203 11:21:46.220125 5052 kuberuntime_manager.go:1274] "Unhandled Error" err="init container &Container{Name:extract-content,Image:registry.redhat.io/redhat/community-operator-index:v4.18,Command:[/utilities/copy-content],Args:[--catalog.from=/configs --catalog.to=/extracted-catalog/catalog --cache.from=/tmp/cache --cache.to=/extracted-catalog/cache],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:utilities,ReadOnly:false,MountPath:/utilities,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:catalog-content,ReadOnly:false,MountPath:/extracted-catalog,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-g4qvc,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000170000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:nil,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:FallbackToLogsOnError,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod community-operators-r8txz_openshift-marketplace(f1ea0ffb-1062-4713-908b-55c0b07c49ee): ErrImagePull: rpc error: code = Canceled desc = copying system image from manifest list: copying config: context canceled" logger="UnhandledError" Dec 03 11:21:46 crc kubenswrapper[5052]: E1203 11:21:46.221368 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"extract-content\" with ErrImagePull: \"rpc error: code = Canceled desc = copying system image from manifest list: copying config: context canceled\"" pod="openshift-marketplace/community-operators-r8txz" podUID="f1ea0ffb-1062-4713-908b-55c0b07c49ee" Dec 03 11:21:46 crc kubenswrapper[5052]: E1203 11:21:46.269836 5052 log.go:32] "PullImage from image service failed" err="rpc error: code = Canceled desc = copying system image from manifest list: copying config: context canceled" image="registry.redhat.io/redhat/redhat-marketplace-index:v4.18" Dec 03 11:21:46 crc kubenswrapper[5052]: E1203 11:21:46.270338 5052 kuberuntime_manager.go:1274] "Unhandled Error" err="init container &Container{Name:extract-content,Image:registry.redhat.io/redhat/redhat-marketplace-index:v4.18,Command:[/utilities/copy-content],Args:[--catalog.from=/configs --catalog.to=/extracted-catalog/catalog --cache.from=/tmp/cache --cache.to=/extracted-catalog/cache],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:utilities,ReadOnly:false,MountPath:/utilities,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:catalog-content,ReadOnly:false,MountPath:/extracted-catalog,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-qdx45,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000170000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:nil,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:FallbackToLogsOnError,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod redhat-marketplace-gqhzd_openshift-marketplace(355008cc-8721-44c4-a1e0-3714b0086286): ErrImagePull: rpc error: code = Canceled desc = copying system image from manifest list: copying config: context canceled" logger="UnhandledError" Dec 03 11:21:46 crc kubenswrapper[5052]: E1203 11:21:46.271605 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"extract-content\" with ErrImagePull: \"rpc error: code = Canceled desc = copying system image from manifest list: copying config: context canceled\"" pod="openshift-marketplace/redhat-marketplace-gqhzd" podUID="355008cc-8721-44c4-a1e0-3714b0086286" Dec 03 11:21:49 crc kubenswrapper[5052]: E1203 11:21:49.625025 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"extract-content\" with ImagePullBackOff: \"Back-off pulling image \\\"registry.redhat.io/redhat/redhat-marketplace-index:v4.18\\\"\"" pod="openshift-marketplace/redhat-marketplace-gqhzd" podUID="355008cc-8721-44c4-a1e0-3714b0086286" Dec 03 11:21:49 crc kubenswrapper[5052]: E1203 11:21:49.625145 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"extract-content\" with ImagePullBackOff: \"Back-off pulling image \\\"registry.redhat.io/redhat/community-operator-index:v4.18\\\"\"" pod="openshift-marketplace/community-operators-r8txz" podUID="f1ea0ffb-1062-4713-908b-55c0b07c49ee" Dec 03 11:21:49 crc kubenswrapper[5052]: E1203 11:21:49.709772 5052 log.go:32] "PullImage from image service failed" err="rpc error: code = Canceled desc = copying system image from manifest list: copying config: context canceled" image="registry.redhat.io/redhat/redhat-operator-index:v4.18" Dec 03 11:21:49 crc kubenswrapper[5052]: E1203 11:21:49.709949 5052 kuberuntime_manager.go:1274] "Unhandled Error" err="init container &Container{Name:extract-content,Image:registry.redhat.io/redhat/redhat-operator-index:v4.18,Command:[/utilities/copy-content],Args:[--catalog.from=/configs --catalog.to=/extracted-catalog/catalog --cache.from=/tmp/cache --cache.to=/extracted-catalog/cache],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:utilities,ReadOnly:false,MountPath:/utilities,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:catalog-content,ReadOnly:false,MountPath:/extracted-catalog,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-kgznh,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000170000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:nil,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:FallbackToLogsOnError,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod redhat-operators-nq2qk_openshift-marketplace(1e97c072-bdcc-4095-9a87-ddf7d4f2d22b): ErrImagePull: rpc error: code = Canceled desc = copying system image from manifest list: copying config: context canceled" logger="UnhandledError" Dec 03 11:21:49 crc kubenswrapper[5052]: E1203 11:21:49.711110 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"extract-content\" with ErrImagePull: \"rpc error: code = Canceled desc = copying system image from manifest list: copying config: context canceled\"" pod="openshift-marketplace/redhat-operators-nq2qk" podUID="1e97c072-bdcc-4095-9a87-ddf7d4f2d22b" Dec 03 11:21:49 crc kubenswrapper[5052]: E1203 11:21:49.729761 5052 log.go:32] "PullImage from image service failed" err="rpc error: code = Canceled desc = copying system image from manifest list: copying config: context canceled" image="registry.redhat.io/redhat/redhat-operator-index:v4.18" Dec 03 11:21:49 crc kubenswrapper[5052]: E1203 11:21:49.729968 5052 kuberuntime_manager.go:1274] "Unhandled Error" err="init container &Container{Name:extract-content,Image:registry.redhat.io/redhat/redhat-operator-index:v4.18,Command:[/utilities/copy-content],Args:[--catalog.from=/configs --catalog.to=/extracted-catalog/catalog --cache.from=/tmp/cache --cache.to=/extracted-catalog/cache],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:utilities,ReadOnly:false,MountPath:/utilities,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:catalog-content,ReadOnly:false,MountPath:/extracted-catalog,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-dm25c,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000170000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:nil,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:FallbackToLogsOnError,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod redhat-operators-dwt2v_openshift-marketplace(afb87a28-1b7c-4b09-b186-f05da232f501): ErrImagePull: rpc error: code = Canceled desc = copying system image from manifest list: copying config: context canceled" logger="UnhandledError" Dec 03 11:21:49 crc kubenswrapper[5052]: E1203 11:21:49.731626 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"extract-content\" with ErrImagePull: \"rpc error: code = Canceled desc = copying system image from manifest list: copying config: context canceled\"" pod="openshift-marketplace/redhat-operators-dwt2v" podUID="afb87a28-1b7c-4b09-b186-f05da232f501" Dec 03 11:21:49 crc kubenswrapper[5052]: E1203 11:21:49.741719 5052 log.go:32] "PullImage from image service failed" err="rpc error: code = Canceled desc = copying system image from manifest list: copying config: context canceled" image="registry.redhat.io/redhat/community-operator-index:v4.18" Dec 03 11:21:49 crc kubenswrapper[5052]: E1203 11:21:49.741921 5052 kuberuntime_manager.go:1274] "Unhandled Error" err="init container &Container{Name:extract-content,Image:registry.redhat.io/redhat/community-operator-index:v4.18,Command:[/utilities/copy-content],Args:[--catalog.from=/configs --catalog.to=/extracted-catalog/catalog --cache.from=/tmp/cache --cache.to=/extracted-catalog/cache],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:utilities,ReadOnly:false,MountPath:/utilities,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:catalog-content,ReadOnly:false,MountPath:/extracted-catalog,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-4xqqz,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000170000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:nil,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:FallbackToLogsOnError,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod community-operators-pxx88_openshift-marketplace(a7228f2a-1d6d-4307-80e7-deebd47d995a): ErrImagePull: rpc error: code = Canceled desc = copying system image from manifest list: copying config: context canceled" logger="UnhandledError" Dec 03 11:21:49 crc kubenswrapper[5052]: E1203 11:21:49.743146 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"extract-content\" with ErrImagePull: \"rpc error: code = Canceled desc = copying system image from manifest list: copying config: context canceled\"" pod="openshift-marketplace/community-operators-pxx88" podUID="a7228f2a-1d6d-4307-80e7-deebd47d995a" Dec 03 11:21:49 crc kubenswrapper[5052]: E1203 11:21:49.764220 5052 log.go:32] "PullImage from image service failed" err="rpc error: code = Canceled desc = copying system image from manifest list: copying config: context canceled" image="registry.redhat.io/redhat/certified-operator-index:v4.18" Dec 03 11:21:49 crc kubenswrapper[5052]: E1203 11:21:49.764392 5052 kuberuntime_manager.go:1274] "Unhandled Error" err="init container &Container{Name:extract-content,Image:registry.redhat.io/redhat/certified-operator-index:v4.18,Command:[/utilities/copy-content],Args:[--catalog.from=/configs --catalog.to=/extracted-catalog/catalog --cache.from=/tmp/cache --cache.to=/extracted-catalog/cache],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:utilities,ReadOnly:false,MountPath:/utilities,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:catalog-content,ReadOnly:false,MountPath:/extracted-catalog,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-z6xw8,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000170000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:nil,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:FallbackToLogsOnError,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod certified-operators-ghqsf_openshift-marketplace(dc6b87f0-562b-4fa9-85b3-ccdd0867374b): ErrImagePull: rpc error: code = Canceled desc = copying system image from manifest list: copying config: context canceled" logger="UnhandledError" Dec 03 11:21:49 crc kubenswrapper[5052]: E1203 11:21:49.765752 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"extract-content\" with ErrImagePull: \"rpc error: code = Canceled desc = copying system image from manifest list: copying config: context canceled\"" pod="openshift-marketplace/certified-operators-ghqsf" podUID="dc6b87f0-562b-4fa9-85b3-ccdd0867374b" Dec 03 11:21:49 crc kubenswrapper[5052]: I1203 11:21:49.985745 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-kube-apiserver/revision-pruner-9-crc"] Dec 03 11:21:50 crc kubenswrapper[5052]: E1203 11:21:50.012377 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"extract-content\" with ImagePullBackOff: \"Back-off pulling image \\\"registry.redhat.io/redhat/redhat-operator-index:v4.18\\\"\"" pod="openshift-marketplace/redhat-operators-dwt2v" podUID="afb87a28-1b7c-4b09-b186-f05da232f501" Dec 03 11:21:50 crc kubenswrapper[5052]: E1203 11:21:50.012730 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"extract-content\" with ImagePullBackOff: \"Back-off pulling image \\\"registry.redhat.io/redhat/community-operator-index:v4.18\\\"\"" pod="openshift-marketplace/community-operators-pxx88" podUID="a7228f2a-1d6d-4307-80e7-deebd47d995a" Dec 03 11:21:50 crc kubenswrapper[5052]: E1203 11:21:50.012813 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"extract-content\" with ImagePullBackOff: \"Back-off pulling image \\\"registry.redhat.io/redhat/certified-operator-index:v4.18\\\"\"" pod="openshift-marketplace/certified-operators-ghqsf" podUID="dc6b87f0-562b-4fa9-85b3-ccdd0867374b" Dec 03 11:21:50 crc kubenswrapper[5052]: E1203 11:21:50.012939 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"extract-content\" with ImagePullBackOff: \"Back-off pulling image \\\"registry.redhat.io/redhat/redhat-operator-index:v4.18\\\"\"" pod="openshift-marketplace/redhat-operators-nq2qk" podUID="1e97c072-bdcc-4095-9a87-ddf7d4f2d22b" Dec 03 11:21:50 crc kubenswrapper[5052]: W1203 11:21:50.017104 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod0b5fd713_7392_4060_a6bb_0dc6938a16e0.slice/crio-c60be5b5bacbc297be324d1d2f5aebad5db28b05d279c35d224f9a319ac02acc WatchSource:0}: Error finding container c60be5b5bacbc297be324d1d2f5aebad5db28b05d279c35d224f9a319ac02acc: Status 404 returned error can't find the container with id c60be5b5bacbc297be324d1d2f5aebad5db28b05d279c35d224f9a319ac02acc Dec 03 11:21:50 crc kubenswrapper[5052]: I1203 11:21:50.134981 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-kube-apiserver/installer-9-crc"] Dec 03 11:21:51 crc kubenswrapper[5052]: I1203 11:21:51.008215 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-kube-apiserver/installer-9-crc" event={"ID":"68a245d3-ddaf-40a0-92f6-92760173b7a3","Type":"ContainerStarted","Data":"1f566de232b91a5d9b8cc04d57b9441b861075111c8d838e330f0a5a1294f19a"} Dec 03 11:21:51 crc kubenswrapper[5052]: I1203 11:21:51.008619 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-kube-apiserver/installer-9-crc" event={"ID":"68a245d3-ddaf-40a0-92f6-92760173b7a3","Type":"ContainerStarted","Data":"cf6c37cbe08d8376ca83da9d1dd8026b2962ef84c13de494526fbe9016df27fd"} Dec 03 11:21:51 crc kubenswrapper[5052]: I1203 11:21:51.010827 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-kube-apiserver/revision-pruner-9-crc" event={"ID":"0b5fd713-7392-4060-a6bb-0dc6938a16e0","Type":"ContainerStarted","Data":"3451ad0d6ddad59190a2157295ede0f54ba547564bd05d6e948b25a31cb236e7"} Dec 03 11:21:51 crc kubenswrapper[5052]: I1203 11:21:51.010854 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-kube-apiserver/revision-pruner-9-crc" event={"ID":"0b5fd713-7392-4060-a6bb-0dc6938a16e0","Type":"ContainerStarted","Data":"c60be5b5bacbc297be324d1d2f5aebad5db28b05d279c35d224f9a319ac02acc"} Dec 03 11:21:51 crc kubenswrapper[5052]: I1203 11:21:51.035115 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-kube-apiserver/installer-9-crc" podStartSLOduration=10.035094019 podStartE2EDuration="10.035094019s" podCreationTimestamp="2025-12-03 11:21:41 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:21:51.034813831 +0000 UTC m=+101.623055595" watchObservedRunningTime="2025-12-03 11:21:51.035094019 +0000 UTC m=+101.623335793" Dec 03 11:21:52 crc kubenswrapper[5052]: I1203 11:21:52.021541 5052 generic.go:334] "Generic (PLEG): container finished" podID="0b5fd713-7392-4060-a6bb-0dc6938a16e0" containerID="3451ad0d6ddad59190a2157295ede0f54ba547564bd05d6e948b25a31cb236e7" exitCode=0 Dec 03 11:21:52 crc kubenswrapper[5052]: I1203 11:21:52.021678 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-kube-apiserver/revision-pruner-9-crc" event={"ID":"0b5fd713-7392-4060-a6bb-0dc6938a16e0","Type":"ContainerDied","Data":"3451ad0d6ddad59190a2157295ede0f54ba547564bd05d6e948b25a31cb236e7"} Dec 03 11:21:53 crc kubenswrapper[5052]: I1203 11:21:53.300797 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-kube-apiserver/revision-pruner-9-crc" Dec 03 11:21:53 crc kubenswrapper[5052]: I1203 11:21:53.398053 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access\" (UniqueName: \"kubernetes.io/projected/0b5fd713-7392-4060-a6bb-0dc6938a16e0-kube-api-access\") pod \"0b5fd713-7392-4060-a6bb-0dc6938a16e0\" (UID: \"0b5fd713-7392-4060-a6bb-0dc6938a16e0\") " Dec 03 11:21:53 crc kubenswrapper[5052]: I1203 11:21:53.398732 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/0b5fd713-7392-4060-a6bb-0dc6938a16e0-kubelet-dir\") pod \"0b5fd713-7392-4060-a6bb-0dc6938a16e0\" (UID: \"0b5fd713-7392-4060-a6bb-0dc6938a16e0\") " Dec 03 11:21:53 crc kubenswrapper[5052]: I1203 11:21:53.398897 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/0b5fd713-7392-4060-a6bb-0dc6938a16e0-kubelet-dir" (OuterVolumeSpecName: "kubelet-dir") pod "0b5fd713-7392-4060-a6bb-0dc6938a16e0" (UID: "0b5fd713-7392-4060-a6bb-0dc6938a16e0"). InnerVolumeSpecName "kubelet-dir". PluginName "kubernetes.io/host-path", VolumeGidValue "" Dec 03 11:21:53 crc kubenswrapper[5052]: I1203 11:21:53.399306 5052 reconciler_common.go:293] "Volume detached for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/0b5fd713-7392-4060-a6bb-0dc6938a16e0-kubelet-dir\") on node \"crc\" DevicePath \"\"" Dec 03 11:21:53 crc kubenswrapper[5052]: I1203 11:21:53.405340 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0b5fd713-7392-4060-a6bb-0dc6938a16e0-kube-api-access" (OuterVolumeSpecName: "kube-api-access") pod "0b5fd713-7392-4060-a6bb-0dc6938a16e0" (UID: "0b5fd713-7392-4060-a6bb-0dc6938a16e0"). InnerVolumeSpecName "kube-api-access". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:21:53 crc kubenswrapper[5052]: I1203 11:21:53.501036 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access\" (UniqueName: \"kubernetes.io/projected/0b5fd713-7392-4060-a6bb-0dc6938a16e0-kube-api-access\") on node \"crc\" DevicePath \"\"" Dec 03 11:21:54 crc kubenswrapper[5052]: I1203 11:21:54.037320 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-kube-apiserver/revision-pruner-9-crc" event={"ID":"0b5fd713-7392-4060-a6bb-0dc6938a16e0","Type":"ContainerDied","Data":"c60be5b5bacbc297be324d1d2f5aebad5db28b05d279c35d224f9a319ac02acc"} Dec 03 11:21:54 crc kubenswrapper[5052]: I1203 11:21:54.037377 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c60be5b5bacbc297be324d1d2f5aebad5db28b05d279c35d224f9a319ac02acc" Dec 03 11:21:54 crc kubenswrapper[5052]: I1203 11:21:54.037377 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-kube-apiserver/revision-pruner-9-crc" Dec 03 11:21:59 crc kubenswrapper[5052]: I1203 11:21:59.343768 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openshift-authentication/oauth-openshift-558db77b4-99jsx"] Dec 03 11:22:13 crc kubenswrapper[5052]: I1203 11:22:13.167037 5052 generic.go:334] "Generic (PLEG): container finished" podID="8cf9b98e-2ea9-47ee-bd1b-a913714b7668" containerID="02ecb4effb412198bc8c98b05696cad847e453c295fb6a2f92dc892af695a3fa" exitCode=0 Dec 03 11:22:13 crc kubenswrapper[5052]: I1203 11:22:13.167667 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-marketplace-qttdj" event={"ID":"8cf9b98e-2ea9-47ee-bd1b-a913714b7668","Type":"ContainerDied","Data":"02ecb4effb412198bc8c98b05696cad847e453c295fb6a2f92dc892af695a3fa"} Dec 03 11:22:13 crc kubenswrapper[5052]: I1203 11:22:13.169364 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/certified-operators-ghqsf" event={"ID":"dc6b87f0-562b-4fa9-85b3-ccdd0867374b","Type":"ContainerStarted","Data":"10f0e44d53a1f23dd54e713776978c8be6ae2d56d9821498094ebf59154f9352"} Dec 03 11:22:14 crc kubenswrapper[5052]: I1203 11:22:14.179010 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/community-operators-r8txz" event={"ID":"f1ea0ffb-1062-4713-908b-55c0b07c49ee","Type":"ContainerStarted","Data":"f022e6c81efcef390965c816054aeb1d50773210788dd511dedc372063167377"} Dec 03 11:22:14 crc kubenswrapper[5052]: I1203 11:22:14.182129 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/certified-operators-99svv" event={"ID":"3b1b1a53-eff1-4339-a82e-9b8400dd5604","Type":"ContainerStarted","Data":"8e3aeed37fefe08d41b2774a5527250f33728938e9f2024a90283dd7bf21c220"} Dec 03 11:22:14 crc kubenswrapper[5052]: I1203 11:22:14.184631 5052 generic.go:334] "Generic (PLEG): container finished" podID="dc6b87f0-562b-4fa9-85b3-ccdd0867374b" containerID="10f0e44d53a1f23dd54e713776978c8be6ae2d56d9821498094ebf59154f9352" exitCode=0 Dec 03 11:22:14 crc kubenswrapper[5052]: I1203 11:22:14.185342 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/certified-operators-ghqsf" event={"ID":"dc6b87f0-562b-4fa9-85b3-ccdd0867374b","Type":"ContainerDied","Data":"10f0e44d53a1f23dd54e713776978c8be6ae2d56d9821498094ebf59154f9352"} Dec 03 11:22:15 crc kubenswrapper[5052]: I1203 11:22:15.194630 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-operators-dwt2v" event={"ID":"afb87a28-1b7c-4b09-b186-f05da232f501","Type":"ContainerStarted","Data":"9043d6a4663d99a931a2821703919380aec3be2f62eb4b00ecf3fb5846d1902e"} Dec 03 11:22:15 crc kubenswrapper[5052]: I1203 11:22:15.198099 5052 generic.go:334] "Generic (PLEG): container finished" podID="f1ea0ffb-1062-4713-908b-55c0b07c49ee" containerID="f022e6c81efcef390965c816054aeb1d50773210788dd511dedc372063167377" exitCode=0 Dec 03 11:22:15 crc kubenswrapper[5052]: I1203 11:22:15.198184 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/community-operators-r8txz" event={"ID":"f1ea0ffb-1062-4713-908b-55c0b07c49ee","Type":"ContainerDied","Data":"f022e6c81efcef390965c816054aeb1d50773210788dd511dedc372063167377"} Dec 03 11:22:15 crc kubenswrapper[5052]: I1203 11:22:15.200638 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-operators-nq2qk" event={"ID":"1e97c072-bdcc-4095-9a87-ddf7d4f2d22b","Type":"ContainerStarted","Data":"911c84a397ed38c66cc5476f39883872f2440ea2cd24b75a8427853d654bdc4c"} Dec 03 11:22:15 crc kubenswrapper[5052]: I1203 11:22:15.202834 5052 generic.go:334] "Generic (PLEG): container finished" podID="3b1b1a53-eff1-4339-a82e-9b8400dd5604" containerID="8e3aeed37fefe08d41b2774a5527250f33728938e9f2024a90283dd7bf21c220" exitCode=0 Dec 03 11:22:15 crc kubenswrapper[5052]: I1203 11:22:15.202881 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/certified-operators-99svv" event={"ID":"3b1b1a53-eff1-4339-a82e-9b8400dd5604","Type":"ContainerDied","Data":"8e3aeed37fefe08d41b2774a5527250f33728938e9f2024a90283dd7bf21c220"} Dec 03 11:22:15 crc kubenswrapper[5052]: I1203 11:22:15.205106 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-marketplace-gqhzd" event={"ID":"355008cc-8721-44c4-a1e0-3714b0086286","Type":"ContainerStarted","Data":"33250962c095d7fb6ff9072a7d197c57c2128c134927c39a68e1d0e51ff9db6e"} Dec 03 11:22:15 crc kubenswrapper[5052]: I1203 11:22:15.207003 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/community-operators-pxx88" event={"ID":"a7228f2a-1d6d-4307-80e7-deebd47d995a","Type":"ContainerStarted","Data":"0443c848089499f8994f9881003129bf06532281a1ddd16281246884dd44408f"} Dec 03 11:22:16 crc kubenswrapper[5052]: I1203 11:22:16.217050 5052 generic.go:334] "Generic (PLEG): container finished" podID="355008cc-8721-44c4-a1e0-3714b0086286" containerID="33250962c095d7fb6ff9072a7d197c57c2128c134927c39a68e1d0e51ff9db6e" exitCode=0 Dec 03 11:22:16 crc kubenswrapper[5052]: I1203 11:22:16.217098 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-marketplace-gqhzd" event={"ID":"355008cc-8721-44c4-a1e0-3714b0086286","Type":"ContainerDied","Data":"33250962c095d7fb6ff9072a7d197c57c2128c134927c39a68e1d0e51ff9db6e"} Dec 03 11:22:16 crc kubenswrapper[5052]: I1203 11:22:16.220344 5052 generic.go:334] "Generic (PLEG): container finished" podID="a7228f2a-1d6d-4307-80e7-deebd47d995a" containerID="0443c848089499f8994f9881003129bf06532281a1ddd16281246884dd44408f" exitCode=0 Dec 03 11:22:16 crc kubenswrapper[5052]: I1203 11:22:16.220458 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/community-operators-pxx88" event={"ID":"a7228f2a-1d6d-4307-80e7-deebd47d995a","Type":"ContainerDied","Data":"0443c848089499f8994f9881003129bf06532281a1ddd16281246884dd44408f"} Dec 03 11:22:17 crc kubenswrapper[5052]: I1203 11:22:17.229408 5052 generic.go:334] "Generic (PLEG): container finished" podID="afb87a28-1b7c-4b09-b186-f05da232f501" containerID="9043d6a4663d99a931a2821703919380aec3be2f62eb4b00ecf3fb5846d1902e" exitCode=0 Dec 03 11:22:17 crc kubenswrapper[5052]: I1203 11:22:17.229452 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-operators-dwt2v" event={"ID":"afb87a28-1b7c-4b09-b186-f05da232f501","Type":"ContainerDied","Data":"9043d6a4663d99a931a2821703919380aec3be2f62eb4b00ecf3fb5846d1902e"} Dec 03 11:22:17 crc kubenswrapper[5052]: I1203 11:22:17.233827 5052 generic.go:334] "Generic (PLEG): container finished" podID="1e97c072-bdcc-4095-9a87-ddf7d4f2d22b" containerID="911c84a397ed38c66cc5476f39883872f2440ea2cd24b75a8427853d654bdc4c" exitCode=0 Dec 03 11:22:17 crc kubenswrapper[5052]: I1203 11:22:17.233857 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-operators-nq2qk" event={"ID":"1e97c072-bdcc-4095-9a87-ddf7d4f2d22b","Type":"ContainerDied","Data":"911c84a397ed38c66cc5476f39883872f2440ea2cd24b75a8427853d654bdc4c"} Dec 03 11:22:18 crc kubenswrapper[5052]: I1203 11:22:18.242035 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-marketplace-qttdj" event={"ID":"8cf9b98e-2ea9-47ee-bd1b-a913714b7668","Type":"ContainerStarted","Data":"1e2172f73a4eb24ef8a5086bfa2b3d1c4e398d6dcf33426b05c5a0016d33f018"} Dec 03 11:22:18 crc kubenswrapper[5052]: I1203 11:22:18.245167 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/certified-operators-99svv" event={"ID":"3b1b1a53-eff1-4339-a82e-9b8400dd5604","Type":"ContainerStarted","Data":"e764472b38e59af009aaebaecca6572e1999fbdc2ba38646e6f0c78ecc104706"} Dec 03 11:22:18 crc kubenswrapper[5052]: I1203 11:22:18.248597 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/certified-operators-ghqsf" event={"ID":"dc6b87f0-562b-4fa9-85b3-ccdd0867374b","Type":"ContainerStarted","Data":"c3facd8f5aa5277fa03efbb624aef1a61ea9479ddb54b075d4bae1376f0a3224"} Dec 03 11:22:18 crc kubenswrapper[5052]: I1203 11:22:18.269213 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-marketplace/redhat-marketplace-qttdj" podStartSLOduration=3.618906571 podStartE2EDuration="1m20.269184462s" podCreationTimestamp="2025-12-03 11:20:58 +0000 UTC" firstStartedPulling="2025-12-03 11:21:00.387805955 +0000 UTC m=+50.976047719" lastFinishedPulling="2025-12-03 11:22:17.038083846 +0000 UTC m=+127.626325610" observedRunningTime="2025-12-03 11:22:18.264313449 +0000 UTC m=+128.852555213" watchObservedRunningTime="2025-12-03 11:22:18.269184462 +0000 UTC m=+128.857426226" Dec 03 11:22:18 crc kubenswrapper[5052]: I1203 11:22:18.290144 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-marketplace/certified-operators-ghqsf" podStartSLOduration=4.210422068 podStartE2EDuration="1m23.290118457s" podCreationTimestamp="2025-12-03 11:20:55 +0000 UTC" firstStartedPulling="2025-12-03 11:20:58.241112396 +0000 UTC m=+48.829354160" lastFinishedPulling="2025-12-03 11:22:17.320808785 +0000 UTC m=+127.909050549" observedRunningTime="2025-12-03 11:22:18.289095387 +0000 UTC m=+128.877337151" watchObservedRunningTime="2025-12-03 11:22:18.290118457 +0000 UTC m=+128.878360221" Dec 03 11:22:18 crc kubenswrapper[5052]: I1203 11:22:18.309864 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-marketplace/certified-operators-99svv" podStartSLOduration=3.180379379 podStartE2EDuration="1m22.309844796s" podCreationTimestamp="2025-12-03 11:20:56 +0000 UTC" firstStartedPulling="2025-12-03 11:20:58.21695995 +0000 UTC m=+48.805201714" lastFinishedPulling="2025-12-03 11:22:17.346425367 +0000 UTC m=+127.934667131" observedRunningTime="2025-12-03 11:22:18.308869987 +0000 UTC m=+128.897111751" watchObservedRunningTime="2025-12-03 11:22:18.309844796 +0000 UTC m=+128.898086550" Dec 03 11:22:18 crc kubenswrapper[5052]: I1203 11:22:18.620388 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-marketplace/redhat-marketplace-qttdj" Dec 03 11:22:18 crc kubenswrapper[5052]: I1203 11:22:18.620455 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openshift-marketplace/redhat-marketplace-qttdj" Dec 03 11:22:19 crc kubenswrapper[5052]: I1203 11:22:19.260870 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/community-operators-r8txz" event={"ID":"f1ea0ffb-1062-4713-908b-55c0b07c49ee","Type":"ContainerStarted","Data":"84a10027d57c8f6d30f6e4c7f8065d6749b27b1a001972b6421953cced8ece00"} Dec 03 11:22:19 crc kubenswrapper[5052]: I1203 11:22:19.262865 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-operators-nq2qk" event={"ID":"1e97c072-bdcc-4095-9a87-ddf7d4f2d22b","Type":"ContainerStarted","Data":"4d0e97a334afc8b927d8b98a681cfab0c25dceea8bee70eac0ac81cdb025e397"} Dec 03 11:22:19 crc kubenswrapper[5052]: I1203 11:22:19.265070 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-marketplace-gqhzd" event={"ID":"355008cc-8721-44c4-a1e0-3714b0086286","Type":"ContainerStarted","Data":"4c0a6dfbfc2568cb3f97d4312237572cc0135491d1360e4adda4523a9ddb4f70"} Dec 03 11:22:19 crc kubenswrapper[5052]: I1203 11:22:19.268160 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/community-operators-pxx88" event={"ID":"a7228f2a-1d6d-4307-80e7-deebd47d995a","Type":"ContainerStarted","Data":"f9dbb7ff933337d17d118274887a9a69ba98abc650bca5b8b4185da0ab78b339"} Dec 03 11:22:19 crc kubenswrapper[5052]: I1203 11:22:19.271051 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-operators-dwt2v" event={"ID":"afb87a28-1b7c-4b09-b186-f05da232f501","Type":"ContainerStarted","Data":"72b81e1c6f6bd6fb2bc6844709bf2e26f33ff52dc4552dfe11690d4fab2eed25"} Dec 03 11:22:19 crc kubenswrapper[5052]: I1203 11:22:19.274493 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-marketplace/redhat-operators-dwt2v" Dec 03 11:22:19 crc kubenswrapper[5052]: I1203 11:22:19.274598 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openshift-marketplace/redhat-operators-dwt2v" Dec 03 11:22:19 crc kubenswrapper[5052]: I1203 11:22:19.296094 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-marketplace/community-operators-r8txz" podStartSLOduration=2.514805268 podStartE2EDuration="1m23.296075675s" podCreationTimestamp="2025-12-03 11:20:56 +0000 UTC" firstStartedPulling="2025-12-03 11:20:58.208566999 +0000 UTC m=+48.796808763" lastFinishedPulling="2025-12-03 11:22:18.989837416 +0000 UTC m=+129.578079170" observedRunningTime="2025-12-03 11:22:19.294713975 +0000 UTC m=+129.882955759" watchObservedRunningTime="2025-12-03 11:22:19.296075675 +0000 UTC m=+129.884317439" Dec 03 11:22:19 crc kubenswrapper[5052]: I1203 11:22:19.320708 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-marketplace/redhat-operators-dwt2v" podStartSLOduration=3.019100743 podStartE2EDuration="1m21.320684227s" podCreationTimestamp="2025-12-03 11:20:58 +0000 UTC" firstStartedPulling="2025-12-03 11:21:00.344733508 +0000 UTC m=+50.932975272" lastFinishedPulling="2025-12-03 11:22:18.646316992 +0000 UTC m=+129.234558756" observedRunningTime="2025-12-03 11:22:19.316216876 +0000 UTC m=+129.904458640" watchObservedRunningTime="2025-12-03 11:22:19.320684227 +0000 UTC m=+129.908925981" Dec 03 11:22:19 crc kubenswrapper[5052]: I1203 11:22:19.360630 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-marketplace/community-operators-pxx88" podStartSLOduration=4.100676536 podStartE2EDuration="1m24.360601659s" podCreationTimestamp="2025-12-03 11:20:55 +0000 UTC" firstStartedPulling="2025-12-03 11:20:58.252818568 +0000 UTC m=+48.841060332" lastFinishedPulling="2025-12-03 11:22:18.512743691 +0000 UTC m=+129.100985455" observedRunningTime="2025-12-03 11:22:19.348550335 +0000 UTC m=+129.936792099" watchObservedRunningTime="2025-12-03 11:22:19.360601659 +0000 UTC m=+129.948843423" Dec 03 11:22:19 crc kubenswrapper[5052]: I1203 11:22:19.375761 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-marketplace/redhat-operators-nq2qk" podStartSLOduration=3.354008709 podStartE2EDuration="1m20.375738464s" podCreationTimestamp="2025-12-03 11:20:59 +0000 UTC" firstStartedPulling="2025-12-03 11:21:01.452678361 +0000 UTC m=+52.040920125" lastFinishedPulling="2025-12-03 11:22:18.474408116 +0000 UTC m=+129.062649880" observedRunningTime="2025-12-03 11:22:19.373092286 +0000 UTC m=+129.961334050" watchObservedRunningTime="2025-12-03 11:22:19.375738464 +0000 UTC m=+129.963980228" Dec 03 11:22:19 crc kubenswrapper[5052]: I1203 11:22:19.399735 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-marketplace/redhat-marketplace-gqhzd" podStartSLOduration=4.081252745 podStartE2EDuration="1m22.399708857s" podCreationTimestamp="2025-12-03 11:20:57 +0000 UTC" firstStartedPulling="2025-12-03 11:21:00.312830559 +0000 UTC m=+50.901072323" lastFinishedPulling="2025-12-03 11:22:18.631286671 +0000 UTC m=+129.219528435" observedRunningTime="2025-12-03 11:22:19.395478623 +0000 UTC m=+129.983720397" watchObservedRunningTime="2025-12-03 11:22:19.399708857 +0000 UTC m=+129.987950621" Dec 03 11:22:19 crc kubenswrapper[5052]: I1203 11:22:19.644646 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openshift-marketplace/redhat-operators-nq2qk" Dec 03 11:22:19 crc kubenswrapper[5052]: I1203 11:22:19.645098 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-marketplace/redhat-operators-nq2qk" Dec 03 11:22:19 crc kubenswrapper[5052]: I1203 11:22:19.834797 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openshift-marketplace/redhat-marketplace-qttdj" podUID="8cf9b98e-2ea9-47ee-bd1b-a913714b7668" containerName="registry-server" probeResult="failure" output=< Dec 03 11:22:19 crc kubenswrapper[5052]: timeout: failed to connect service ":50051" within 1s Dec 03 11:22:19 crc kubenswrapper[5052]: > Dec 03 11:22:20 crc kubenswrapper[5052]: I1203 11:22:20.401339 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openshift-marketplace/redhat-operators-dwt2v" podUID="afb87a28-1b7c-4b09-b186-f05da232f501" containerName="registry-server" probeResult="failure" output=< Dec 03 11:22:20 crc kubenswrapper[5052]: timeout: failed to connect service ":50051" within 1s Dec 03 11:22:20 crc kubenswrapper[5052]: > Dec 03 11:22:20 crc kubenswrapper[5052]: I1203 11:22:20.859406 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openshift-marketplace/redhat-operators-nq2qk" podUID="1e97c072-bdcc-4095-9a87-ddf7d4f2d22b" containerName="registry-server" probeResult="failure" output=< Dec 03 11:22:20 crc kubenswrapper[5052]: timeout: failed to connect service ":50051" within 1s Dec 03 11:22:20 crc kubenswrapper[5052]: > Dec 03 11:22:24 crc kubenswrapper[5052]: I1203 11:22:24.373555 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openshift-authentication/oauth-openshift-558db77b4-99jsx" podUID="75f62784-7818-4a74-9c8c-4777d6db0611" containerName="oauth-openshift" containerID="cri-o://4670adfe678989d2721baf3507dccbda083f63c965b655522db5099c5d66de54" gracePeriod=15 Dec 03 11:22:26 crc kubenswrapper[5052]: I1203 11:22:26.036602 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openshift-marketplace/community-operators-pxx88" Dec 03 11:22:26 crc kubenswrapper[5052]: I1203 11:22:26.036676 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-marketplace/community-operators-pxx88" Dec 03 11:22:26 crc kubenswrapper[5052]: I1203 11:22:26.114360 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-marketplace/community-operators-pxx88" Dec 03 11:22:26 crc kubenswrapper[5052]: I1203 11:22:26.414090 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-marketplace/community-operators-pxx88" Dec 03 11:22:26 crc kubenswrapper[5052]: I1203 11:22:26.551577 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-marketplace/certified-operators-ghqsf" Dec 03 11:22:26 crc kubenswrapper[5052]: I1203 11:22:26.551648 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openshift-marketplace/certified-operators-ghqsf" Dec 03 11:22:26 crc kubenswrapper[5052]: I1203 11:22:26.593860 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-marketplace/certified-operators-ghqsf" Dec 03 11:22:26 crc kubenswrapper[5052]: I1203 11:22:26.650141 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openshift-marketplace/certified-operators-99svv" Dec 03 11:22:26 crc kubenswrapper[5052]: I1203 11:22:26.650575 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-marketplace/certified-operators-99svv" Dec 03 11:22:26 crc kubenswrapper[5052]: I1203 11:22:26.689183 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-marketplace/certified-operators-99svv" Dec 03 11:22:26 crc kubenswrapper[5052]: I1203 11:22:26.881749 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-marketplace/community-operators-r8txz" Dec 03 11:22:26 crc kubenswrapper[5052]: I1203 11:22:26.881914 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openshift-marketplace/community-operators-r8txz" Dec 03 11:22:26 crc kubenswrapper[5052]: I1203 11:22:26.929774 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-marketplace/community-operators-r8txz" Dec 03 11:22:27 crc kubenswrapper[5052]: I1203 11:22:27.319752 5052 generic.go:334] "Generic (PLEG): container finished" podID="75f62784-7818-4a74-9c8c-4777d6db0611" containerID="4670adfe678989d2721baf3507dccbda083f63c965b655522db5099c5d66de54" exitCode=0 Dec 03 11:22:27 crc kubenswrapper[5052]: I1203 11:22:27.319934 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-authentication/oauth-openshift-558db77b4-99jsx" event={"ID":"75f62784-7818-4a74-9c8c-4777d6db0611","Type":"ContainerDied","Data":"4670adfe678989d2721baf3507dccbda083f63c965b655522db5099c5d66de54"} Dec 03 11:22:27 crc kubenswrapper[5052]: I1203 11:22:27.360662 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-marketplace/certified-operators-ghqsf" Dec 03 11:22:27 crc kubenswrapper[5052]: I1203 11:22:27.361589 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-marketplace/community-operators-r8txz" Dec 03 11:22:27 crc kubenswrapper[5052]: I1203 11:22:27.362255 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-marketplace/certified-operators-99svv" Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.021670 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-authentication/oauth-openshift-558db77b4-99jsx" Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.066239 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openshift-authentication/oauth-openshift-6f85f596d6-nhvd7"] Dec 03 11:22:28 crc kubenswrapper[5052]: E1203 11:22:28.066535 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="75f62784-7818-4a74-9c8c-4777d6db0611" containerName="oauth-openshift" Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.066549 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="75f62784-7818-4a74-9c8c-4777d6db0611" containerName="oauth-openshift" Dec 03 11:22:28 crc kubenswrapper[5052]: E1203 11:22:28.066573 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="0b5fd713-7392-4060-a6bb-0dc6938a16e0" containerName="pruner" Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.066584 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="0b5fd713-7392-4060-a6bb-0dc6938a16e0" containerName="pruner" Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.066719 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="75f62784-7818-4a74-9c8c-4777d6db0611" containerName="oauth-openshift" Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.066737 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="0b5fd713-7392-4060-a6bb-0dc6938a16e0" containerName="pruner" Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.067239 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-authentication/oauth-openshift-6f85f596d6-nhvd7" Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.104162 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-authentication/oauth-openshift-6f85f596d6-nhvd7"] Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.158862 5052 kubelet.go:2421] "SyncLoop ADD" source="file" pods=["openshift-kube-apiserver/kube-apiserver-startup-monitor-crc"] Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.159957 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-apiserver/kube-apiserver-startup-monitor-crc" Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.160162 5052 kubelet.go:2431] "SyncLoop REMOVE" source="file" pods=["openshift-kube-apiserver/kube-apiserver-crc"] Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.160542 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openshift-kube-apiserver/kube-apiserver-crc" podUID="f4b27818a5e8e43d0dc095d08835c792" containerName="kube-apiserver" containerID="cri-o://09256543a9aa86da8bec948be040d36b6e4986b9232091d2638bb1fead5740a9" gracePeriod=15 Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.160649 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openshift-kube-apiserver/kube-apiserver-crc" podUID="f4b27818a5e8e43d0dc095d08835c792" containerName="kube-apiserver-cert-regeneration-controller" containerID="cri-o://15abb0d7fd5884519ed5ac454430dbb58f71ed1f459e5ed30f324f70befd243e" gracePeriod=15 Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.160680 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openshift-kube-apiserver/kube-apiserver-crc" podUID="f4b27818a5e8e43d0dc095d08835c792" containerName="kube-apiserver-cert-syncer" containerID="cri-o://8ceedd21ae3833d77fd440537f30434ee57f087fbefb2357b46231d3f778ad2a" gracePeriod=15 Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.160737 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openshift-kube-apiserver/kube-apiserver-crc" podUID="f4b27818a5e8e43d0dc095d08835c792" containerName="kube-apiserver-check-endpoints" containerID="cri-o://ee9c5c61cfd6dee9fd44ebc4c7ef4d20b5fde82b12a8147c3c871e2615da7940" gracePeriod=15 Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.160588 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openshift-kube-apiserver/kube-apiserver-crc" podUID="f4b27818a5e8e43d0dc095d08835c792" containerName="kube-apiserver-insecure-readyz" containerID="cri-o://22b8cb6aa04337d1420f97d3e128fce2b29a725ca6e5d157adeb1ba78f00073f" gracePeriod=15 Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.161276 5052 kubelet.go:2421] "SyncLoop ADD" source="file" pods=["openshift-kube-apiserver/kube-apiserver-crc"] Dec 03 11:22:28 crc kubenswrapper[5052]: E1203 11:22:28.161482 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="f4b27818a5e8e43d0dc095d08835c792" containerName="kube-apiserver-check-endpoints" Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.161496 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="f4b27818a5e8e43d0dc095d08835c792" containerName="kube-apiserver-check-endpoints" Dec 03 11:22:28 crc kubenswrapper[5052]: E1203 11:22:28.161527 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="f4b27818a5e8e43d0dc095d08835c792" containerName="kube-apiserver-check-endpoints" Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.161537 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="f4b27818a5e8e43d0dc095d08835c792" containerName="kube-apiserver-check-endpoints" Dec 03 11:22:28 crc kubenswrapper[5052]: E1203 11:22:28.161547 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="f4b27818a5e8e43d0dc095d08835c792" containerName="kube-apiserver" Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.161554 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="f4b27818a5e8e43d0dc095d08835c792" containerName="kube-apiserver" Dec 03 11:22:28 crc kubenswrapper[5052]: E1203 11:22:28.161564 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="f4b27818a5e8e43d0dc095d08835c792" containerName="kube-apiserver-cert-syncer" Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.161570 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="f4b27818a5e8e43d0dc095d08835c792" containerName="kube-apiserver-cert-syncer" Dec 03 11:22:28 crc kubenswrapper[5052]: E1203 11:22:28.161581 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="f4b27818a5e8e43d0dc095d08835c792" containerName="setup" Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.161588 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="f4b27818a5e8e43d0dc095d08835c792" containerName="setup" Dec 03 11:22:28 crc kubenswrapper[5052]: E1203 11:22:28.161599 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="f4b27818a5e8e43d0dc095d08835c792" containerName="kube-apiserver-insecure-readyz" Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.161606 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="f4b27818a5e8e43d0dc095d08835c792" containerName="kube-apiserver-insecure-readyz" Dec 03 11:22:28 crc kubenswrapper[5052]: E1203 11:22:28.161615 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="f4b27818a5e8e43d0dc095d08835c792" containerName="kube-apiserver-cert-regeneration-controller" Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.161624 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="f4b27818a5e8e43d0dc095d08835c792" containerName="kube-apiserver-cert-regeneration-controller" Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.161756 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="f4b27818a5e8e43d0dc095d08835c792" containerName="kube-apiserver-cert-syncer" Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.161770 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="f4b27818a5e8e43d0dc095d08835c792" containerName="kube-apiserver-cert-regeneration-controller" Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.161779 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="f4b27818a5e8e43d0dc095d08835c792" containerName="kube-apiserver-check-endpoints" Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.161788 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="f4b27818a5e8e43d0dc095d08835c792" containerName="kube-apiserver-check-endpoints" Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.161795 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="f4b27818a5e8e43d0dc095d08835c792" containerName="kube-apiserver" Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.161806 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="f4b27818a5e8e43d0dc095d08835c792" containerName="kube-apiserver-insecure-readyz" Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.209732 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"v4-0-config-system-router-certs\" (UniqueName: \"kubernetes.io/secret/75f62784-7818-4a74-9c8c-4777d6db0611-v4-0-config-system-router-certs\") pod \"75f62784-7818-4a74-9c8c-4777d6db0611\" (UID: \"75f62784-7818-4a74-9c8c-4777d6db0611\") " Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.209802 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"v4-0-config-system-service-ca\" (UniqueName: \"kubernetes.io/configmap/75f62784-7818-4a74-9c8c-4777d6db0611-v4-0-config-system-service-ca\") pod \"75f62784-7818-4a74-9c8c-4777d6db0611\" (UID: \"75f62784-7818-4a74-9c8c-4777d6db0611\") " Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.209846 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"audit-policies\" (UniqueName: \"kubernetes.io/configmap/75f62784-7818-4a74-9c8c-4777d6db0611-audit-policies\") pod \"75f62784-7818-4a74-9c8c-4777d6db0611\" (UID: \"75f62784-7818-4a74-9c8c-4777d6db0611\") " Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.209871 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"v4-0-config-system-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/75f62784-7818-4a74-9c8c-4777d6db0611-v4-0-config-system-trusted-ca-bundle\") pod \"75f62784-7818-4a74-9c8c-4777d6db0611\" (UID: \"75f62784-7818-4a74-9c8c-4777d6db0611\") " Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.209959 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"v4-0-config-system-cliconfig\" (UniqueName: \"kubernetes.io/configmap/75f62784-7818-4a74-9c8c-4777d6db0611-v4-0-config-system-cliconfig\") pod \"75f62784-7818-4a74-9c8c-4777d6db0611\" (UID: \"75f62784-7818-4a74-9c8c-4777d6db0611\") " Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.210007 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"v4-0-config-user-template-error\" (UniqueName: \"kubernetes.io/secret/75f62784-7818-4a74-9c8c-4777d6db0611-v4-0-config-user-template-error\") pod \"75f62784-7818-4a74-9c8c-4777d6db0611\" (UID: \"75f62784-7818-4a74-9c8c-4777d6db0611\") " Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.210038 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l5k2l\" (UniqueName: \"kubernetes.io/projected/75f62784-7818-4a74-9c8c-4777d6db0611-kube-api-access-l5k2l\") pod \"75f62784-7818-4a74-9c8c-4777d6db0611\" (UID: \"75f62784-7818-4a74-9c8c-4777d6db0611\") " Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.210062 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"v4-0-config-system-serving-cert\" (UniqueName: \"kubernetes.io/secret/75f62784-7818-4a74-9c8c-4777d6db0611-v4-0-config-system-serving-cert\") pod \"75f62784-7818-4a74-9c8c-4777d6db0611\" (UID: \"75f62784-7818-4a74-9c8c-4777d6db0611\") " Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.210090 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"v4-0-config-user-idp-0-file-data\" (UniqueName: \"kubernetes.io/secret/75f62784-7818-4a74-9c8c-4777d6db0611-v4-0-config-user-idp-0-file-data\") pod \"75f62784-7818-4a74-9c8c-4777d6db0611\" (UID: \"75f62784-7818-4a74-9c8c-4777d6db0611\") " Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.210138 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"v4-0-config-system-session\" (UniqueName: \"kubernetes.io/secret/75f62784-7818-4a74-9c8c-4777d6db0611-v4-0-config-system-session\") pod \"75f62784-7818-4a74-9c8c-4777d6db0611\" (UID: \"75f62784-7818-4a74-9c8c-4777d6db0611\") " Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.210216 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"v4-0-config-user-template-login\" (UniqueName: \"kubernetes.io/secret/75f62784-7818-4a74-9c8c-4777d6db0611-v4-0-config-user-template-login\") pod \"75f62784-7818-4a74-9c8c-4777d6db0611\" (UID: \"75f62784-7818-4a74-9c8c-4777d6db0611\") " Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.210246 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"audit-dir\" (UniqueName: \"kubernetes.io/host-path/75f62784-7818-4a74-9c8c-4777d6db0611-audit-dir\") pod \"75f62784-7818-4a74-9c8c-4777d6db0611\" (UID: \"75f62784-7818-4a74-9c8c-4777d6db0611\") " Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.210275 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"v4-0-config-system-ocp-branding-template\" (UniqueName: \"kubernetes.io/secret/75f62784-7818-4a74-9c8c-4777d6db0611-v4-0-config-system-ocp-branding-template\") pod \"75f62784-7818-4a74-9c8c-4777d6db0611\" (UID: \"75f62784-7818-4a74-9c8c-4777d6db0611\") " Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.210323 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"v4-0-config-user-template-provider-selection\" (UniqueName: \"kubernetes.io/secret/75f62784-7818-4a74-9c8c-4777d6db0611-v4-0-config-user-template-provider-selection\") pod \"75f62784-7818-4a74-9c8c-4777d6db0611\" (UID: \"75f62784-7818-4a74-9c8c-4777d6db0611\") " Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.210556 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"v4-0-config-user-template-error\" (UniqueName: \"kubernetes.io/secret/c712fba2-c530-4f64-836d-16706a2e2b50-v4-0-config-user-template-error\") pod \"oauth-openshift-6f85f596d6-nhvd7\" (UID: \"c712fba2-c530-4f64-836d-16706a2e2b50\") " pod="openshift-authentication/oauth-openshift-6f85f596d6-nhvd7" Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.210639 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"v4-0-config-user-template-login\" (UniqueName: \"kubernetes.io/secret/c712fba2-c530-4f64-836d-16706a2e2b50-v4-0-config-user-template-login\") pod \"oauth-openshift-6f85f596d6-nhvd7\" (UID: \"c712fba2-c530-4f64-836d-16706a2e2b50\") " pod="openshift-authentication/oauth-openshift-6f85f596d6-nhvd7" Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.210687 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"v4-0-config-system-router-certs\" (UniqueName: \"kubernetes.io/secret/c712fba2-c530-4f64-836d-16706a2e2b50-v4-0-config-system-router-certs\") pod \"oauth-openshift-6f85f596d6-nhvd7\" (UID: \"c712fba2-c530-4f64-836d-16706a2e2b50\") " pod="openshift-authentication/oauth-openshift-6f85f596d6-nhvd7" Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.210726 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"v4-0-config-user-template-provider-selection\" (UniqueName: \"kubernetes.io/secret/c712fba2-c530-4f64-836d-16706a2e2b50-v4-0-config-user-template-provider-selection\") pod \"oauth-openshift-6f85f596d6-nhvd7\" (UID: \"c712fba2-c530-4f64-836d-16706a2e2b50\") " pod="openshift-authentication/oauth-openshift-6f85f596d6-nhvd7" Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.210770 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"v4-0-config-system-session\" (UniqueName: \"kubernetes.io/secret/c712fba2-c530-4f64-836d-16706a2e2b50-v4-0-config-system-session\") pod \"oauth-openshift-6f85f596d6-nhvd7\" (UID: \"c712fba2-c530-4f64-836d-16706a2e2b50\") " pod="openshift-authentication/oauth-openshift-6f85f596d6-nhvd7" Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.211283 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openshift-kube-apiserver/kube-apiserver-startup-monitor-crc"] Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.211405 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"v4-0-config-system-ocp-branding-template\" (UniqueName: \"kubernetes.io/secret/c712fba2-c530-4f64-836d-16706a2e2b50-v4-0-config-system-ocp-branding-template\") pod \"oauth-openshift-6f85f596d6-nhvd7\" (UID: \"c712fba2-c530-4f64-836d-16706a2e2b50\") " pod="openshift-authentication/oauth-openshift-6f85f596d6-nhvd7" Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.211474 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zbr2b\" (UniqueName: \"kubernetes.io/projected/c712fba2-c530-4f64-836d-16706a2e2b50-kube-api-access-zbr2b\") pod \"oauth-openshift-6f85f596d6-nhvd7\" (UID: \"c712fba2-c530-4f64-836d-16706a2e2b50\") " pod="openshift-authentication/oauth-openshift-6f85f596d6-nhvd7" Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.211534 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-dir\" (UniqueName: \"kubernetes.io/host-path/c712fba2-c530-4f64-836d-16706a2e2b50-audit-dir\") pod \"oauth-openshift-6f85f596d6-nhvd7\" (UID: \"c712fba2-c530-4f64-836d-16706a2e2b50\") " pod="openshift-authentication/oauth-openshift-6f85f596d6-nhvd7" Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.211577 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"v4-0-config-user-idp-0-file-data\" (UniqueName: \"kubernetes.io/secret/c712fba2-c530-4f64-836d-16706a2e2b50-v4-0-config-user-idp-0-file-data\") pod \"oauth-openshift-6f85f596d6-nhvd7\" (UID: \"c712fba2-c530-4f64-836d-16706a2e2b50\") " pod="openshift-authentication/oauth-openshift-6f85f596d6-nhvd7" Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.211640 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"v4-0-config-system-cliconfig\" (UniqueName: \"kubernetes.io/configmap/c712fba2-c530-4f64-836d-16706a2e2b50-v4-0-config-system-cliconfig\") pod \"oauth-openshift-6f85f596d6-nhvd7\" (UID: \"c712fba2-c530-4f64-836d-16706a2e2b50\") " pod="openshift-authentication/oauth-openshift-6f85f596d6-nhvd7" Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.211663 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-policies\" (UniqueName: \"kubernetes.io/configmap/c712fba2-c530-4f64-836d-16706a2e2b50-audit-policies\") pod \"oauth-openshift-6f85f596d6-nhvd7\" (UID: \"c712fba2-c530-4f64-836d-16706a2e2b50\") " pod="openshift-authentication/oauth-openshift-6f85f596d6-nhvd7" Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.211693 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"v4-0-config-system-service-ca\" (UniqueName: \"kubernetes.io/configmap/c712fba2-c530-4f64-836d-16706a2e2b50-v4-0-config-system-service-ca\") pod \"oauth-openshift-6f85f596d6-nhvd7\" (UID: \"c712fba2-c530-4f64-836d-16706a2e2b50\") " pod="openshift-authentication/oauth-openshift-6f85f596d6-nhvd7" Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.211752 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"v4-0-config-system-serving-cert\" (UniqueName: \"kubernetes.io/secret/c712fba2-c530-4f64-836d-16706a2e2b50-v4-0-config-system-serving-cert\") pod \"oauth-openshift-6f85f596d6-nhvd7\" (UID: \"c712fba2-c530-4f64-836d-16706a2e2b50\") " pod="openshift-authentication/oauth-openshift-6f85f596d6-nhvd7" Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.211779 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"v4-0-config-system-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c712fba2-c530-4f64-836d-16706a2e2b50-v4-0-config-system-trusted-ca-bundle\") pod \"oauth-openshift-6f85f596d6-nhvd7\" (UID: \"c712fba2-c530-4f64-836d-16706a2e2b50\") " pod="openshift-authentication/oauth-openshift-6f85f596d6-nhvd7" Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.212245 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/75f62784-7818-4a74-9c8c-4777d6db0611-audit-dir" (OuterVolumeSpecName: "audit-dir") pod "75f62784-7818-4a74-9c8c-4777d6db0611" (UID: "75f62784-7818-4a74-9c8c-4777d6db0611"). InnerVolumeSpecName "audit-dir". PluginName "kubernetes.io/host-path", VolumeGidValue "" Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.212963 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/75f62784-7818-4a74-9c8c-4777d6db0611-v4-0-config-system-cliconfig" (OuterVolumeSpecName: "v4-0-config-system-cliconfig") pod "75f62784-7818-4a74-9c8c-4777d6db0611" (UID: "75f62784-7818-4a74-9c8c-4777d6db0611"). InnerVolumeSpecName "v4-0-config-system-cliconfig". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.213118 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/75f62784-7818-4a74-9c8c-4777d6db0611-v4-0-config-system-trusted-ca-bundle" (OuterVolumeSpecName: "v4-0-config-system-trusted-ca-bundle") pod "75f62784-7818-4a74-9c8c-4777d6db0611" (UID: "75f62784-7818-4a74-9c8c-4777d6db0611"). InnerVolumeSpecName "v4-0-config-system-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.213695 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/75f62784-7818-4a74-9c8c-4777d6db0611-v4-0-config-system-service-ca" (OuterVolumeSpecName: "v4-0-config-system-service-ca") pod "75f62784-7818-4a74-9c8c-4777d6db0611" (UID: "75f62784-7818-4a74-9c8c-4777d6db0611"). InnerVolumeSpecName "v4-0-config-system-service-ca". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.214122 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/75f62784-7818-4a74-9c8c-4777d6db0611-audit-policies" (OuterVolumeSpecName: "audit-policies") pod "75f62784-7818-4a74-9c8c-4777d6db0611" (UID: "75f62784-7818-4a74-9c8c-4777d6db0611"). InnerVolumeSpecName "audit-policies". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.219081 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-marketplace/redhat-marketplace-gqhzd" Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.219158 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openshift-marketplace/redhat-marketplace-gqhzd" Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.225737 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/75f62784-7818-4a74-9c8c-4777d6db0611-v4-0-config-user-idp-0-file-data" (OuterVolumeSpecName: "v4-0-config-user-idp-0-file-data") pod "75f62784-7818-4a74-9c8c-4777d6db0611" (UID: "75f62784-7818-4a74-9c8c-4777d6db0611"). InnerVolumeSpecName "v4-0-config-user-idp-0-file-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.226542 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/75f62784-7818-4a74-9c8c-4777d6db0611-kube-api-access-l5k2l" (OuterVolumeSpecName: "kube-api-access-l5k2l") pod "75f62784-7818-4a74-9c8c-4777d6db0611" (UID: "75f62784-7818-4a74-9c8c-4777d6db0611"). InnerVolumeSpecName "kube-api-access-l5k2l". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.226820 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/75f62784-7818-4a74-9c8c-4777d6db0611-v4-0-config-system-session" (OuterVolumeSpecName: "v4-0-config-system-session") pod "75f62784-7818-4a74-9c8c-4777d6db0611" (UID: "75f62784-7818-4a74-9c8c-4777d6db0611"). InnerVolumeSpecName "v4-0-config-system-session". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.229360 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/75f62784-7818-4a74-9c8c-4777d6db0611-v4-0-config-user-template-login" (OuterVolumeSpecName: "v4-0-config-user-template-login") pod "75f62784-7818-4a74-9c8c-4777d6db0611" (UID: "75f62784-7818-4a74-9c8c-4777d6db0611"). InnerVolumeSpecName "v4-0-config-user-template-login". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.229914 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/75f62784-7818-4a74-9c8c-4777d6db0611-v4-0-config-system-router-certs" (OuterVolumeSpecName: "v4-0-config-system-router-certs") pod "75f62784-7818-4a74-9c8c-4777d6db0611" (UID: "75f62784-7818-4a74-9c8c-4777d6db0611"). InnerVolumeSpecName "v4-0-config-system-router-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.238917 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/75f62784-7818-4a74-9c8c-4777d6db0611-v4-0-config-user-template-provider-selection" (OuterVolumeSpecName: "v4-0-config-user-template-provider-selection") pod "75f62784-7818-4a74-9c8c-4777d6db0611" (UID: "75f62784-7818-4a74-9c8c-4777d6db0611"). InnerVolumeSpecName "v4-0-config-user-template-provider-selection". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.245621 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/75f62784-7818-4a74-9c8c-4777d6db0611-v4-0-config-user-template-error" (OuterVolumeSpecName: "v4-0-config-user-template-error") pod "75f62784-7818-4a74-9c8c-4777d6db0611" (UID: "75f62784-7818-4a74-9c8c-4777d6db0611"). InnerVolumeSpecName "v4-0-config-user-template-error". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.253752 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/75f62784-7818-4a74-9c8c-4777d6db0611-v4-0-config-system-ocp-branding-template" (OuterVolumeSpecName: "v4-0-config-system-ocp-branding-template") pod "75f62784-7818-4a74-9c8c-4777d6db0611" (UID: "75f62784-7818-4a74-9c8c-4777d6db0611"). InnerVolumeSpecName "v4-0-config-system-ocp-branding-template". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.256569 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/75f62784-7818-4a74-9c8c-4777d6db0611-v4-0-config-system-serving-cert" (OuterVolumeSpecName: "v4-0-config-system-serving-cert") pod "75f62784-7818-4a74-9c8c-4777d6db0611" (UID: "75f62784-7818-4a74-9c8c-4777d6db0611"). InnerVolumeSpecName "v4-0-config-system-serving-cert". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.290870 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-marketplace/redhat-marketplace-gqhzd" Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.292060 5052 status_manager.go:851] "Failed to get status for pod" podUID="f4b27818a5e8e43d0dc095d08835c792" pod="openshift-kube-apiserver/kube-apiserver-crc" err="Get \"https://api-int.crc.testing:6443/api/v1/namespaces/openshift-kube-apiserver/pods/kube-apiserver-crc\": dial tcp 38.129.56.252:6443: connect: connection refused" Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.292256 5052 status_manager.go:851] "Failed to get status for pod" podUID="f85e55b1a89d02b0cb034b1ea31ed45a" pod="openshift-kube-apiserver/kube-apiserver-startup-monitor-crc" err="Get \"https://api-int.crc.testing:6443/api/v1/namespaces/openshift-kube-apiserver/pods/kube-apiserver-startup-monitor-crc\": dial tcp 38.129.56.252:6443: connect: connection refused" Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.292422 5052 status_manager.go:851] "Failed to get status for pod" podUID="355008cc-8721-44c4-a1e0-3714b0086286" pod="openshift-marketplace/redhat-marketplace-gqhzd" err="Get \"https://api-int.crc.testing:6443/api/v1/namespaces/openshift-marketplace/pods/redhat-marketplace-gqhzd\": dial tcp 38.129.56.252:6443: connect: connection refused" Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.313239 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"audit-policies\" (UniqueName: \"kubernetes.io/configmap/c712fba2-c530-4f64-836d-16706a2e2b50-audit-policies\") pod \"oauth-openshift-6f85f596d6-nhvd7\" (UID: \"c712fba2-c530-4f64-836d-16706a2e2b50\") " pod="openshift-authentication/oauth-openshift-6f85f596d6-nhvd7" Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.313301 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-log\" (UniqueName: \"kubernetes.io/host-path/f85e55b1a89d02b0cb034b1ea31ed45a-var-log\") pod \"kube-apiserver-startup-monitor-crc\" (UID: \"f85e55b1a89d02b0cb034b1ea31ed45a\") " pod="openshift-kube-apiserver/kube-apiserver-startup-monitor-crc" Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.313334 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"v4-0-config-system-service-ca\" (UniqueName: \"kubernetes.io/configmap/c712fba2-c530-4f64-836d-16706a2e2b50-v4-0-config-system-service-ca\") pod \"oauth-openshift-6f85f596d6-nhvd7\" (UID: \"c712fba2-c530-4f64-836d-16706a2e2b50\") " pod="openshift-authentication/oauth-openshift-6f85f596d6-nhvd7" Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.313388 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lock\" (UniqueName: \"kubernetes.io/host-path/f85e55b1a89d02b0cb034b1ea31ed45a-var-lock\") pod \"kube-apiserver-startup-monitor-crc\" (UID: \"f85e55b1a89d02b0cb034b1ea31ed45a\") " pod="openshift-kube-apiserver/kube-apiserver-startup-monitor-crc" Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.313437 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"v4-0-config-system-serving-cert\" (UniqueName: \"kubernetes.io/secret/c712fba2-c530-4f64-836d-16706a2e2b50-v4-0-config-system-serving-cert\") pod \"oauth-openshift-6f85f596d6-nhvd7\" (UID: \"c712fba2-c530-4f64-836d-16706a2e2b50\") " pod="openshift-authentication/oauth-openshift-6f85f596d6-nhvd7" Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.313452 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"resource-dir\" (UniqueName: \"kubernetes.io/host-path/f85e55b1a89d02b0cb034b1ea31ed45a-resource-dir\") pod \"kube-apiserver-startup-monitor-crc\" (UID: \"f85e55b1a89d02b0cb034b1ea31ed45a\") " pod="openshift-kube-apiserver/kube-apiserver-startup-monitor-crc" Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.313467 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/host-path/f85e55b1a89d02b0cb034b1ea31ed45a-manifests\") pod \"kube-apiserver-startup-monitor-crc\" (UID: \"f85e55b1a89d02b0cb034b1ea31ed45a\") " pod="openshift-kube-apiserver/kube-apiserver-startup-monitor-crc" Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.313494 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert-dir\" (UniqueName: \"kubernetes.io/host-path/71bb4a3aecc4ba5b26c4b7318770ce13-cert-dir\") pod \"kube-apiserver-crc\" (UID: \"71bb4a3aecc4ba5b26c4b7318770ce13\") " pod="openshift-kube-apiserver/kube-apiserver-crc" Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.313541 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"v4-0-config-system-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c712fba2-c530-4f64-836d-16706a2e2b50-v4-0-config-system-trusted-ca-bundle\") pod \"oauth-openshift-6f85f596d6-nhvd7\" (UID: \"c712fba2-c530-4f64-836d-16706a2e2b50\") " pod="openshift-authentication/oauth-openshift-6f85f596d6-nhvd7" Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.313581 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-dir\" (UniqueName: \"kubernetes.io/host-path/71bb4a3aecc4ba5b26c4b7318770ce13-audit-dir\") pod \"kube-apiserver-crc\" (UID: \"71bb4a3aecc4ba5b26c4b7318770ce13\") " pod="openshift-kube-apiserver/kube-apiserver-crc" Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.313608 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"v4-0-config-user-template-error\" (UniqueName: \"kubernetes.io/secret/c712fba2-c530-4f64-836d-16706a2e2b50-v4-0-config-user-template-error\") pod \"oauth-openshift-6f85f596d6-nhvd7\" (UID: \"c712fba2-c530-4f64-836d-16706a2e2b50\") " pod="openshift-authentication/oauth-openshift-6f85f596d6-nhvd7" Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.313627 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"v4-0-config-user-template-login\" (UniqueName: \"kubernetes.io/secret/c712fba2-c530-4f64-836d-16706a2e2b50-v4-0-config-user-template-login\") pod \"oauth-openshift-6f85f596d6-nhvd7\" (UID: \"c712fba2-c530-4f64-836d-16706a2e2b50\") " pod="openshift-authentication/oauth-openshift-6f85f596d6-nhvd7" Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.313643 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"pod-resource-dir\" (UniqueName: \"kubernetes.io/host-path/f85e55b1a89d02b0cb034b1ea31ed45a-pod-resource-dir\") pod \"kube-apiserver-startup-monitor-crc\" (UID: \"f85e55b1a89d02b0cb034b1ea31ed45a\") " pod="openshift-kube-apiserver/kube-apiserver-startup-monitor-crc" Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.313669 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"v4-0-config-system-router-certs\" (UniqueName: \"kubernetes.io/secret/c712fba2-c530-4f64-836d-16706a2e2b50-v4-0-config-system-router-certs\") pod \"oauth-openshift-6f85f596d6-nhvd7\" (UID: \"c712fba2-c530-4f64-836d-16706a2e2b50\") " pod="openshift-authentication/oauth-openshift-6f85f596d6-nhvd7" Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.313687 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"v4-0-config-user-template-provider-selection\" (UniqueName: \"kubernetes.io/secret/c712fba2-c530-4f64-836d-16706a2e2b50-v4-0-config-user-template-provider-selection\") pod \"oauth-openshift-6f85f596d6-nhvd7\" (UID: \"c712fba2-c530-4f64-836d-16706a2e2b50\") " pod="openshift-authentication/oauth-openshift-6f85f596d6-nhvd7" Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.313724 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"v4-0-config-system-session\" (UniqueName: \"kubernetes.io/secret/c712fba2-c530-4f64-836d-16706a2e2b50-v4-0-config-system-session\") pod \"oauth-openshift-6f85f596d6-nhvd7\" (UID: \"c712fba2-c530-4f64-836d-16706a2e2b50\") " pod="openshift-authentication/oauth-openshift-6f85f596d6-nhvd7" Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.313743 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"resource-dir\" (UniqueName: \"kubernetes.io/host-path/71bb4a3aecc4ba5b26c4b7318770ce13-resource-dir\") pod \"kube-apiserver-crc\" (UID: \"71bb4a3aecc4ba5b26c4b7318770ce13\") " pod="openshift-kube-apiserver/kube-apiserver-crc" Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.313766 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"v4-0-config-system-ocp-branding-template\" (UniqueName: \"kubernetes.io/secret/c712fba2-c530-4f64-836d-16706a2e2b50-v4-0-config-system-ocp-branding-template\") pod \"oauth-openshift-6f85f596d6-nhvd7\" (UID: \"c712fba2-c530-4f64-836d-16706a2e2b50\") " pod="openshift-authentication/oauth-openshift-6f85f596d6-nhvd7" Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.313782 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-zbr2b\" (UniqueName: \"kubernetes.io/projected/c712fba2-c530-4f64-836d-16706a2e2b50-kube-api-access-zbr2b\") pod \"oauth-openshift-6f85f596d6-nhvd7\" (UID: \"c712fba2-c530-4f64-836d-16706a2e2b50\") " pod="openshift-authentication/oauth-openshift-6f85f596d6-nhvd7" Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.313801 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"audit-dir\" (UniqueName: \"kubernetes.io/host-path/c712fba2-c530-4f64-836d-16706a2e2b50-audit-dir\") pod \"oauth-openshift-6f85f596d6-nhvd7\" (UID: \"c712fba2-c530-4f64-836d-16706a2e2b50\") " pod="openshift-authentication/oauth-openshift-6f85f596d6-nhvd7" Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.313830 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"v4-0-config-user-idp-0-file-data\" (UniqueName: \"kubernetes.io/secret/c712fba2-c530-4f64-836d-16706a2e2b50-v4-0-config-user-idp-0-file-data\") pod \"oauth-openshift-6f85f596d6-nhvd7\" (UID: \"c712fba2-c530-4f64-836d-16706a2e2b50\") " pod="openshift-authentication/oauth-openshift-6f85f596d6-nhvd7" Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.313860 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"v4-0-config-system-cliconfig\" (UniqueName: \"kubernetes.io/configmap/c712fba2-c530-4f64-836d-16706a2e2b50-v4-0-config-system-cliconfig\") pod \"oauth-openshift-6f85f596d6-nhvd7\" (UID: \"c712fba2-c530-4f64-836d-16706a2e2b50\") " pod="openshift-authentication/oauth-openshift-6f85f596d6-nhvd7" Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.313899 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-l5k2l\" (UniqueName: \"kubernetes.io/projected/75f62784-7818-4a74-9c8c-4777d6db0611-kube-api-access-l5k2l\") on node \"crc\" DevicePath \"\"" Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.313910 5052 reconciler_common.go:293] "Volume detached for volume \"v4-0-config-system-serving-cert\" (UniqueName: \"kubernetes.io/secret/75f62784-7818-4a74-9c8c-4777d6db0611-v4-0-config-system-serving-cert\") on node \"crc\" DevicePath \"\"" Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.313921 5052 reconciler_common.go:293] "Volume detached for volume \"v4-0-config-user-idp-0-file-data\" (UniqueName: \"kubernetes.io/secret/75f62784-7818-4a74-9c8c-4777d6db0611-v4-0-config-user-idp-0-file-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.313931 5052 reconciler_common.go:293] "Volume detached for volume \"v4-0-config-system-session\" (UniqueName: \"kubernetes.io/secret/75f62784-7818-4a74-9c8c-4777d6db0611-v4-0-config-system-session\") on node \"crc\" DevicePath \"\"" Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.313942 5052 reconciler_common.go:293] "Volume detached for volume \"v4-0-config-user-template-login\" (UniqueName: \"kubernetes.io/secret/75f62784-7818-4a74-9c8c-4777d6db0611-v4-0-config-user-template-login\") on node \"crc\" DevicePath \"\"" Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.313951 5052 reconciler_common.go:293] "Volume detached for volume \"audit-dir\" (UniqueName: \"kubernetes.io/host-path/75f62784-7818-4a74-9c8c-4777d6db0611-audit-dir\") on node \"crc\" DevicePath \"\"" Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.313962 5052 reconciler_common.go:293] "Volume detached for volume \"v4-0-config-system-ocp-branding-template\" (UniqueName: \"kubernetes.io/secret/75f62784-7818-4a74-9c8c-4777d6db0611-v4-0-config-system-ocp-branding-template\") on node \"crc\" DevicePath \"\"" Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.313973 5052 reconciler_common.go:293] "Volume detached for volume \"v4-0-config-user-template-provider-selection\" (UniqueName: \"kubernetes.io/secret/75f62784-7818-4a74-9c8c-4777d6db0611-v4-0-config-user-template-provider-selection\") on node \"crc\" DevicePath \"\"" Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.313982 5052 reconciler_common.go:293] "Volume detached for volume \"v4-0-config-system-router-certs\" (UniqueName: \"kubernetes.io/secret/75f62784-7818-4a74-9c8c-4777d6db0611-v4-0-config-system-router-certs\") on node \"crc\" DevicePath \"\"" Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.313992 5052 reconciler_common.go:293] "Volume detached for volume \"v4-0-config-system-service-ca\" (UniqueName: \"kubernetes.io/configmap/75f62784-7818-4a74-9c8c-4777d6db0611-v4-0-config-system-service-ca\") on node \"crc\" DevicePath \"\"" Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.314000 5052 reconciler_common.go:293] "Volume detached for volume \"v4-0-config-system-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/75f62784-7818-4a74-9c8c-4777d6db0611-v4-0-config-system-trusted-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.314010 5052 reconciler_common.go:293] "Volume detached for volume \"audit-policies\" (UniqueName: \"kubernetes.io/configmap/75f62784-7818-4a74-9c8c-4777d6db0611-audit-policies\") on node \"crc\" DevicePath \"\"" Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.314020 5052 reconciler_common.go:293] "Volume detached for volume \"v4-0-config-system-cliconfig\" (UniqueName: \"kubernetes.io/configmap/75f62784-7818-4a74-9c8c-4777d6db0611-v4-0-config-system-cliconfig\") on node \"crc\" DevicePath \"\"" Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.314029 5052 reconciler_common.go:293] "Volume detached for volume \"v4-0-config-user-template-error\" (UniqueName: \"kubernetes.io/secret/75f62784-7818-4a74-9c8c-4777d6db0611-v4-0-config-user-template-error\") on node \"crc\" DevicePath \"\"" Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.314873 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"v4-0-config-system-cliconfig\" (UniqueName: \"kubernetes.io/configmap/c712fba2-c530-4f64-836d-16706a2e2b50-v4-0-config-system-cliconfig\") pod \"oauth-openshift-6f85f596d6-nhvd7\" (UID: \"c712fba2-c530-4f64-836d-16706a2e2b50\") " pod="openshift-authentication/oauth-openshift-6f85f596d6-nhvd7" Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.316069 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"v4-0-config-system-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c712fba2-c530-4f64-836d-16706a2e2b50-v4-0-config-system-trusted-ca-bundle\") pod \"oauth-openshift-6f85f596d6-nhvd7\" (UID: \"c712fba2-c530-4f64-836d-16706a2e2b50\") " pod="openshift-authentication/oauth-openshift-6f85f596d6-nhvd7" Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.316269 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"v4-0-config-system-service-ca\" (UniqueName: \"kubernetes.io/configmap/c712fba2-c530-4f64-836d-16706a2e2b50-v4-0-config-system-service-ca\") pod \"oauth-openshift-6f85f596d6-nhvd7\" (UID: \"c712fba2-c530-4f64-836d-16706a2e2b50\") " pod="openshift-authentication/oauth-openshift-6f85f596d6-nhvd7" Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.317030 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"audit-policies\" (UniqueName: \"kubernetes.io/configmap/c712fba2-c530-4f64-836d-16706a2e2b50-audit-policies\") pod \"oauth-openshift-6f85f596d6-nhvd7\" (UID: \"c712fba2-c530-4f64-836d-16706a2e2b50\") " pod="openshift-authentication/oauth-openshift-6f85f596d6-nhvd7" Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.317849 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"v4-0-config-system-serving-cert\" (UniqueName: \"kubernetes.io/secret/c712fba2-c530-4f64-836d-16706a2e2b50-v4-0-config-system-serving-cert\") pod \"oauth-openshift-6f85f596d6-nhvd7\" (UID: \"c712fba2-c530-4f64-836d-16706a2e2b50\") " pod="openshift-authentication/oauth-openshift-6f85f596d6-nhvd7" Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.319009 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"v4-0-config-user-template-error\" (UniqueName: \"kubernetes.io/secret/c712fba2-c530-4f64-836d-16706a2e2b50-v4-0-config-user-template-error\") pod \"oauth-openshift-6f85f596d6-nhvd7\" (UID: \"c712fba2-c530-4f64-836d-16706a2e2b50\") " pod="openshift-authentication/oauth-openshift-6f85f596d6-nhvd7" Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.319077 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"audit-dir\" (UniqueName: \"kubernetes.io/host-path/c712fba2-c530-4f64-836d-16706a2e2b50-audit-dir\") pod \"oauth-openshift-6f85f596d6-nhvd7\" (UID: \"c712fba2-c530-4f64-836d-16706a2e2b50\") " pod="openshift-authentication/oauth-openshift-6f85f596d6-nhvd7" Dec 03 11:22:28 crc kubenswrapper[5052]: E1203 11:22:28.321277 5052 projected.go:194] Error preparing data for projected volume kube-api-access-zbr2b for pod openshift-authentication/oauth-openshift-6f85f596d6-nhvd7: failed to fetch token: Post "https://api-int.crc.testing:6443/api/v1/namespaces/openshift-authentication/serviceaccounts/oauth-openshift/token": dial tcp 38.129.56.252:6443: connect: connection refused Dec 03 11:22:28 crc kubenswrapper[5052]: E1203 11:22:28.321357 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c712fba2-c530-4f64-836d-16706a2e2b50-kube-api-access-zbr2b podName:c712fba2-c530-4f64-836d-16706a2e2b50 nodeName:}" failed. No retries permitted until 2025-12-03 11:22:28.82133496 +0000 UTC m=+139.409576724 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-zbr2b" (UniqueName: "kubernetes.io/projected/c712fba2-c530-4f64-836d-16706a2e2b50-kube-api-access-zbr2b") pod "oauth-openshift-6f85f596d6-nhvd7" (UID: "c712fba2-c530-4f64-836d-16706a2e2b50") : failed to fetch token: Post "https://api-int.crc.testing:6443/api/v1/namespaces/openshift-authentication/serviceaccounts/oauth-openshift/token": dial tcp 38.129.56.252:6443: connect: connection refused Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.321616 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"v4-0-config-system-session\" (UniqueName: \"kubernetes.io/secret/c712fba2-c530-4f64-836d-16706a2e2b50-v4-0-config-system-session\") pod \"oauth-openshift-6f85f596d6-nhvd7\" (UID: \"c712fba2-c530-4f64-836d-16706a2e2b50\") " pod="openshift-authentication/oauth-openshift-6f85f596d6-nhvd7" Dec 03 11:22:28 crc kubenswrapper[5052]: E1203 11:22:28.321962 5052 event.go:368] "Unable to write event (may retry after sleeping)" err="Post \"https://api-int.crc.testing:6443/api/v1/namespaces/openshift-authentication/events\": dial tcp 38.129.56.252:6443: connect: connection refused" event="&Event{ObjectMeta:{oauth-openshift-6f85f596d6-nhvd7.187db0bb7e2c92de openshift-authentication 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:openshift-authentication,Name:oauth-openshift-6f85f596d6-nhvd7,UID:c712fba2-c530-4f64-836d-16706a2e2b50,APIVersion:v1,ResourceVersion:29195,FieldPath:,},Reason:FailedMount,Message:MountVolume.SetUp failed for volume \"kube-api-access-zbr2b\" : failed to fetch token: Post \"https://api-int.crc.testing:6443/api/v1/namespaces/openshift-authentication/serviceaccounts/oauth-openshift/token\": dial tcp 38.129.56.252:6443: connect: connection refused,Source:EventSource{Component:kubelet,Host:crc,},FirstTimestamp:2025-12-03 11:22:28.32132579 +0000 UTC m=+138.909567554,LastTimestamp:2025-12-03 11:22:28.32132579 +0000 UTC m=+138.909567554,Count:1,Type:Warning,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:crc,}" Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.323415 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"v4-0-config-system-ocp-branding-template\" (UniqueName: \"kubernetes.io/secret/c712fba2-c530-4f64-836d-16706a2e2b50-v4-0-config-system-ocp-branding-template\") pod \"oauth-openshift-6f85f596d6-nhvd7\" (UID: \"c712fba2-c530-4f64-836d-16706a2e2b50\") " pod="openshift-authentication/oauth-openshift-6f85f596d6-nhvd7" Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.323757 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"v4-0-config-user-idp-0-file-data\" (UniqueName: \"kubernetes.io/secret/c712fba2-c530-4f64-836d-16706a2e2b50-v4-0-config-user-idp-0-file-data\") pod \"oauth-openshift-6f85f596d6-nhvd7\" (UID: \"c712fba2-c530-4f64-836d-16706a2e2b50\") " pod="openshift-authentication/oauth-openshift-6f85f596d6-nhvd7" Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.323880 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"v4-0-config-user-template-login\" (UniqueName: \"kubernetes.io/secret/c712fba2-c530-4f64-836d-16706a2e2b50-v4-0-config-user-template-login\") pod \"oauth-openshift-6f85f596d6-nhvd7\" (UID: \"c712fba2-c530-4f64-836d-16706a2e2b50\") " pod="openshift-authentication/oauth-openshift-6f85f596d6-nhvd7" Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.325274 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"v4-0-config-system-router-certs\" (UniqueName: \"kubernetes.io/secret/c712fba2-c530-4f64-836d-16706a2e2b50-v4-0-config-system-router-certs\") pod \"oauth-openshift-6f85f596d6-nhvd7\" (UID: \"c712fba2-c530-4f64-836d-16706a2e2b50\") " pod="openshift-authentication/oauth-openshift-6f85f596d6-nhvd7" Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.326251 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"v4-0-config-user-template-provider-selection\" (UniqueName: \"kubernetes.io/secret/c712fba2-c530-4f64-836d-16706a2e2b50-v4-0-config-user-template-provider-selection\") pod \"oauth-openshift-6f85f596d6-nhvd7\" (UID: \"c712fba2-c530-4f64-836d-16706a2e2b50\") " pod="openshift-authentication/oauth-openshift-6f85f596d6-nhvd7" Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.329884 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-authentication/oauth-openshift-558db77b4-99jsx" Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.330680 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-authentication/oauth-openshift-558db77b4-99jsx" event={"ID":"75f62784-7818-4a74-9c8c-4777d6db0611","Type":"ContainerDied","Data":"f27d0c5e51bbc4e9b04fabcea475200348e7624f94cfe69d2c6bd6690b326182"} Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.330733 5052 scope.go:117] "RemoveContainer" containerID="4670adfe678989d2721baf3507dccbda083f63c965b655522db5099c5d66de54" Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.332857 5052 status_manager.go:851] "Failed to get status for pod" podUID="f4b27818a5e8e43d0dc095d08835c792" pod="openshift-kube-apiserver/kube-apiserver-crc" err="Get \"https://api-int.crc.testing:6443/api/v1/namespaces/openshift-kube-apiserver/pods/kube-apiserver-crc\": dial tcp 38.129.56.252:6443: connect: connection refused" Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.333472 5052 status_manager.go:851] "Failed to get status for pod" podUID="f85e55b1a89d02b0cb034b1ea31ed45a" pod="openshift-kube-apiserver/kube-apiserver-startup-monitor-crc" err="Get \"https://api-int.crc.testing:6443/api/v1/namespaces/openshift-kube-apiserver/pods/kube-apiserver-startup-monitor-crc\": dial tcp 38.129.56.252:6443: connect: connection refused" Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.333811 5052 status_manager.go:851] "Failed to get status for pod" podUID="355008cc-8721-44c4-a1e0-3714b0086286" pod="openshift-marketplace/redhat-marketplace-gqhzd" err="Get \"https://api-int.crc.testing:6443/api/v1/namespaces/openshift-marketplace/pods/redhat-marketplace-gqhzd\": dial tcp 38.129.56.252:6443: connect: connection refused" Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.334050 5052 status_manager.go:851] "Failed to get status for pod" podUID="75f62784-7818-4a74-9c8c-4777d6db0611" pod="openshift-authentication/oauth-openshift-558db77b4-99jsx" err="Get \"https://api-int.crc.testing:6443/api/v1/namespaces/openshift-authentication/pods/oauth-openshift-558db77b4-99jsx\": dial tcp 38.129.56.252:6443: connect: connection refused" Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.345804 5052 status_manager.go:851] "Failed to get status for pod" podUID="75f62784-7818-4a74-9c8c-4777d6db0611" pod="openshift-authentication/oauth-openshift-558db77b4-99jsx" err="Get \"https://api-int.crc.testing:6443/api/v1/namespaces/openshift-authentication/pods/oauth-openshift-558db77b4-99jsx\": dial tcp 38.129.56.252:6443: connect: connection refused" Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.345993 5052 status_manager.go:851] "Failed to get status for pod" podUID="f4b27818a5e8e43d0dc095d08835c792" pod="openshift-kube-apiserver/kube-apiserver-crc" err="Get \"https://api-int.crc.testing:6443/api/v1/namespaces/openshift-kube-apiserver/pods/kube-apiserver-crc\": dial tcp 38.129.56.252:6443: connect: connection refused" Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.346200 5052 status_manager.go:851] "Failed to get status for pod" podUID="f85e55b1a89d02b0cb034b1ea31ed45a" pod="openshift-kube-apiserver/kube-apiserver-startup-monitor-crc" err="Get \"https://api-int.crc.testing:6443/api/v1/namespaces/openshift-kube-apiserver/pods/kube-apiserver-startup-monitor-crc\": dial tcp 38.129.56.252:6443: connect: connection refused" Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.346341 5052 status_manager.go:851] "Failed to get status for pod" podUID="355008cc-8721-44c4-a1e0-3714b0086286" pod="openshift-marketplace/redhat-marketplace-gqhzd" err="Get \"https://api-int.crc.testing:6443/api/v1/namespaces/openshift-marketplace/pods/redhat-marketplace-gqhzd\": dial tcp 38.129.56.252:6443: connect: connection refused" Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.373849 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-marketplace/redhat-marketplace-gqhzd" Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.374838 5052 status_manager.go:851] "Failed to get status for pod" podUID="f4b27818a5e8e43d0dc095d08835c792" pod="openshift-kube-apiserver/kube-apiserver-crc" err="Get \"https://api-int.crc.testing:6443/api/v1/namespaces/openshift-kube-apiserver/pods/kube-apiserver-crc\": dial tcp 38.129.56.252:6443: connect: connection refused" Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.375474 5052 status_manager.go:851] "Failed to get status for pod" podUID="f85e55b1a89d02b0cb034b1ea31ed45a" pod="openshift-kube-apiserver/kube-apiserver-startup-monitor-crc" err="Get \"https://api-int.crc.testing:6443/api/v1/namespaces/openshift-kube-apiserver/pods/kube-apiserver-startup-monitor-crc\": dial tcp 38.129.56.252:6443: connect: connection refused" Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.376086 5052 status_manager.go:851] "Failed to get status for pod" podUID="355008cc-8721-44c4-a1e0-3714b0086286" pod="openshift-marketplace/redhat-marketplace-gqhzd" err="Get \"https://api-int.crc.testing:6443/api/v1/namespaces/openshift-marketplace/pods/redhat-marketplace-gqhzd\": dial tcp 38.129.56.252:6443: connect: connection refused" Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.376406 5052 status_manager.go:851] "Failed to get status for pod" podUID="75f62784-7818-4a74-9c8c-4777d6db0611" pod="openshift-authentication/oauth-openshift-558db77b4-99jsx" err="Get \"https://api-int.crc.testing:6443/api/v1/namespaces/openshift-authentication/pods/oauth-openshift-558db77b4-99jsx\": dial tcp 38.129.56.252:6443: connect: connection refused" Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.414964 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"audit-dir\" (UniqueName: \"kubernetes.io/host-path/71bb4a3aecc4ba5b26c4b7318770ce13-audit-dir\") pod \"kube-apiserver-crc\" (UID: \"71bb4a3aecc4ba5b26c4b7318770ce13\") " pod="openshift-kube-apiserver/kube-apiserver-crc" Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.415039 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"pod-resource-dir\" (UniqueName: \"kubernetes.io/host-path/f85e55b1a89d02b0cb034b1ea31ed45a-pod-resource-dir\") pod \"kube-apiserver-startup-monitor-crc\" (UID: \"f85e55b1a89d02b0cb034b1ea31ed45a\") " pod="openshift-kube-apiserver/kube-apiserver-startup-monitor-crc" Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.415083 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"resource-dir\" (UniqueName: \"kubernetes.io/host-path/71bb4a3aecc4ba5b26c4b7318770ce13-resource-dir\") pod \"kube-apiserver-crc\" (UID: \"71bb4a3aecc4ba5b26c4b7318770ce13\") " pod="openshift-kube-apiserver/kube-apiserver-crc" Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.415113 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"audit-dir\" (UniqueName: \"kubernetes.io/host-path/71bb4a3aecc4ba5b26c4b7318770ce13-audit-dir\") pod \"kube-apiserver-crc\" (UID: \"71bb4a3aecc4ba5b26c4b7318770ce13\") " pod="openshift-kube-apiserver/kube-apiserver-crc" Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.415145 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"var-log\" (UniqueName: \"kubernetes.io/host-path/f85e55b1a89d02b0cb034b1ea31ed45a-var-log\") pod \"kube-apiserver-startup-monitor-crc\" (UID: \"f85e55b1a89d02b0cb034b1ea31ed45a\") " pod="openshift-kube-apiserver/kube-apiserver-startup-monitor-crc" Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.415182 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"pod-resource-dir\" (UniqueName: \"kubernetes.io/host-path/f85e55b1a89d02b0cb034b1ea31ed45a-pod-resource-dir\") pod \"kube-apiserver-startup-monitor-crc\" (UID: \"f85e55b1a89d02b0cb034b1ea31ed45a\") " pod="openshift-kube-apiserver/kube-apiserver-startup-monitor-crc" Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.415213 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"var-lock\" (UniqueName: \"kubernetes.io/host-path/f85e55b1a89d02b0cb034b1ea31ed45a-var-lock\") pod \"kube-apiserver-startup-monitor-crc\" (UID: \"f85e55b1a89d02b0cb034b1ea31ed45a\") " pod="openshift-kube-apiserver/kube-apiserver-startup-monitor-crc" Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.415266 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"var-lock\" (UniqueName: \"kubernetes.io/host-path/f85e55b1a89d02b0cb034b1ea31ed45a-var-lock\") pod \"kube-apiserver-startup-monitor-crc\" (UID: \"f85e55b1a89d02b0cb034b1ea31ed45a\") " pod="openshift-kube-apiserver/kube-apiserver-startup-monitor-crc" Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.415300 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"resource-dir\" (UniqueName: \"kubernetes.io/host-path/71bb4a3aecc4ba5b26c4b7318770ce13-resource-dir\") pod \"kube-apiserver-crc\" (UID: \"71bb4a3aecc4ba5b26c4b7318770ce13\") " pod="openshift-kube-apiserver/kube-apiserver-crc" Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.415333 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"var-log\" (UniqueName: \"kubernetes.io/host-path/f85e55b1a89d02b0cb034b1ea31ed45a-var-log\") pod \"kube-apiserver-startup-monitor-crc\" (UID: \"f85e55b1a89d02b0cb034b1ea31ed45a\") " pod="openshift-kube-apiserver/kube-apiserver-startup-monitor-crc" Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.415338 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"resource-dir\" (UniqueName: \"kubernetes.io/host-path/f85e55b1a89d02b0cb034b1ea31ed45a-resource-dir\") pod \"kube-apiserver-startup-monitor-crc\" (UID: \"f85e55b1a89d02b0cb034b1ea31ed45a\") " pod="openshift-kube-apiserver/kube-apiserver-startup-monitor-crc" Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.415362 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/host-path/f85e55b1a89d02b0cb034b1ea31ed45a-manifests\") pod \"kube-apiserver-startup-monitor-crc\" (UID: \"f85e55b1a89d02b0cb034b1ea31ed45a\") " pod="openshift-kube-apiserver/kube-apiserver-startup-monitor-crc" Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.415403 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"cert-dir\" (UniqueName: \"kubernetes.io/host-path/71bb4a3aecc4ba5b26c4b7318770ce13-cert-dir\") pod \"kube-apiserver-crc\" (UID: \"71bb4a3aecc4ba5b26c4b7318770ce13\") " pod="openshift-kube-apiserver/kube-apiserver-crc" Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.416047 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/host-path/f85e55b1a89d02b0cb034b1ea31ed45a-manifests\") pod \"kube-apiserver-startup-monitor-crc\" (UID: \"f85e55b1a89d02b0cb034b1ea31ed45a\") " pod="openshift-kube-apiserver/kube-apiserver-startup-monitor-crc" Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.416259 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"cert-dir\" (UniqueName: \"kubernetes.io/host-path/71bb4a3aecc4ba5b26c4b7318770ce13-cert-dir\") pod \"kube-apiserver-crc\" (UID: \"71bb4a3aecc4ba5b26c4b7318770ce13\") " pod="openshift-kube-apiserver/kube-apiserver-crc" Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.416297 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"resource-dir\" (UniqueName: \"kubernetes.io/host-path/f85e55b1a89d02b0cb034b1ea31ed45a-resource-dir\") pod \"kube-apiserver-startup-monitor-crc\" (UID: \"f85e55b1a89d02b0cb034b1ea31ed45a\") " pod="openshift-kube-apiserver/kube-apiserver-startup-monitor-crc" Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.661426 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-marketplace/redhat-marketplace-qttdj" Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.662470 5052 status_manager.go:851] "Failed to get status for pod" podUID="355008cc-8721-44c4-a1e0-3714b0086286" pod="openshift-marketplace/redhat-marketplace-gqhzd" err="Get \"https://api-int.crc.testing:6443/api/v1/namespaces/openshift-marketplace/pods/redhat-marketplace-gqhzd\": dial tcp 38.129.56.252:6443: connect: connection refused" Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.662886 5052 status_manager.go:851] "Failed to get status for pod" podUID="75f62784-7818-4a74-9c8c-4777d6db0611" pod="openshift-authentication/oauth-openshift-558db77b4-99jsx" err="Get \"https://api-int.crc.testing:6443/api/v1/namespaces/openshift-authentication/pods/oauth-openshift-558db77b4-99jsx\": dial tcp 38.129.56.252:6443: connect: connection refused" Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.663255 5052 status_manager.go:851] "Failed to get status for pod" podUID="f4b27818a5e8e43d0dc095d08835c792" pod="openshift-kube-apiserver/kube-apiserver-crc" err="Get \"https://api-int.crc.testing:6443/api/v1/namespaces/openshift-kube-apiserver/pods/kube-apiserver-crc\": dial tcp 38.129.56.252:6443: connect: connection refused" Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.664079 5052 status_manager.go:851] "Failed to get status for pod" podUID="8cf9b98e-2ea9-47ee-bd1b-a913714b7668" pod="openshift-marketplace/redhat-marketplace-qttdj" err="Get \"https://api-int.crc.testing:6443/api/v1/namespaces/openshift-marketplace/pods/redhat-marketplace-qttdj\": dial tcp 38.129.56.252:6443: connect: connection refused" Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.665194 5052 status_manager.go:851] "Failed to get status for pod" podUID="f85e55b1a89d02b0cb034b1ea31ed45a" pod="openshift-kube-apiserver/kube-apiserver-startup-monitor-crc" err="Get \"https://api-int.crc.testing:6443/api/v1/namespaces/openshift-kube-apiserver/pods/kube-apiserver-startup-monitor-crc\": dial tcp 38.129.56.252:6443: connect: connection refused" Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.702469 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-marketplace/redhat-marketplace-qttdj" Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.703298 5052 status_manager.go:851] "Failed to get status for pod" podUID="355008cc-8721-44c4-a1e0-3714b0086286" pod="openshift-marketplace/redhat-marketplace-gqhzd" err="Get \"https://api-int.crc.testing:6443/api/v1/namespaces/openshift-marketplace/pods/redhat-marketplace-gqhzd\": dial tcp 38.129.56.252:6443: connect: connection refused" Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.704014 5052 status_manager.go:851] "Failed to get status for pod" podUID="75f62784-7818-4a74-9c8c-4777d6db0611" pod="openshift-authentication/oauth-openshift-558db77b4-99jsx" err="Get \"https://api-int.crc.testing:6443/api/v1/namespaces/openshift-authentication/pods/oauth-openshift-558db77b4-99jsx\": dial tcp 38.129.56.252:6443: connect: connection refused" Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.704702 5052 status_manager.go:851] "Failed to get status for pod" podUID="f4b27818a5e8e43d0dc095d08835c792" pod="openshift-kube-apiserver/kube-apiserver-crc" err="Get \"https://api-int.crc.testing:6443/api/v1/namespaces/openshift-kube-apiserver/pods/kube-apiserver-crc\": dial tcp 38.129.56.252:6443: connect: connection refused" Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.705069 5052 status_manager.go:851] "Failed to get status for pod" podUID="8cf9b98e-2ea9-47ee-bd1b-a913714b7668" pod="openshift-marketplace/redhat-marketplace-qttdj" err="Get \"https://api-int.crc.testing:6443/api/v1/namespaces/openshift-marketplace/pods/redhat-marketplace-qttdj\": dial tcp 38.129.56.252:6443: connect: connection refused" Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.705422 5052 status_manager.go:851] "Failed to get status for pod" podUID="f85e55b1a89d02b0cb034b1ea31ed45a" pod="openshift-kube-apiserver/kube-apiserver-startup-monitor-crc" err="Get \"https://api-int.crc.testing:6443/api/v1/namespaces/openshift-kube-apiserver/pods/kube-apiserver-startup-monitor-crc\": dial tcp 38.129.56.252:6443: connect: connection refused" Dec 03 11:22:28 crc kubenswrapper[5052]: I1203 11:22:28.921870 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-zbr2b\" (UniqueName: \"kubernetes.io/projected/c712fba2-c530-4f64-836d-16706a2e2b50-kube-api-access-zbr2b\") pod \"oauth-openshift-6f85f596d6-nhvd7\" (UID: \"c712fba2-c530-4f64-836d-16706a2e2b50\") " pod="openshift-authentication/oauth-openshift-6f85f596d6-nhvd7" Dec 03 11:22:28 crc kubenswrapper[5052]: E1203 11:22:28.922955 5052 projected.go:194] Error preparing data for projected volume kube-api-access-zbr2b for pod openshift-authentication/oauth-openshift-6f85f596d6-nhvd7: failed to fetch token: Post "https://api-int.crc.testing:6443/api/v1/namespaces/openshift-authentication/serviceaccounts/oauth-openshift/token": dial tcp 38.129.56.252:6443: connect: connection refused Dec 03 11:22:28 crc kubenswrapper[5052]: E1203 11:22:28.923079 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c712fba2-c530-4f64-836d-16706a2e2b50-kube-api-access-zbr2b podName:c712fba2-c530-4f64-836d-16706a2e2b50 nodeName:}" failed. No retries permitted until 2025-12-03 11:22:29.923054858 +0000 UTC m=+140.511296622 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-zbr2b" (UniqueName: "kubernetes.io/projected/c712fba2-c530-4f64-836d-16706a2e2b50-kube-api-access-zbr2b") pod "oauth-openshift-6f85f596d6-nhvd7" (UID: "c712fba2-c530-4f64-836d-16706a2e2b50") : failed to fetch token: Post "https://api-int.crc.testing:6443/api/v1/namespaces/openshift-authentication/serviceaccounts/oauth-openshift/token": dial tcp 38.129.56.252:6443: connect: connection refused Dec 03 11:22:29 crc kubenswrapper[5052]: I1203 11:22:29.314934 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-marketplace/redhat-operators-dwt2v" Dec 03 11:22:29 crc kubenswrapper[5052]: I1203 11:22:29.315712 5052 status_manager.go:851] "Failed to get status for pod" podUID="f4b27818a5e8e43d0dc095d08835c792" pod="openshift-kube-apiserver/kube-apiserver-crc" err="Get \"https://api-int.crc.testing:6443/api/v1/namespaces/openshift-kube-apiserver/pods/kube-apiserver-crc\": dial tcp 38.129.56.252:6443: connect: connection refused" Dec 03 11:22:29 crc kubenswrapper[5052]: I1203 11:22:29.316077 5052 status_manager.go:851] "Failed to get status for pod" podUID="8cf9b98e-2ea9-47ee-bd1b-a913714b7668" pod="openshift-marketplace/redhat-marketplace-qttdj" err="Get \"https://api-int.crc.testing:6443/api/v1/namespaces/openshift-marketplace/pods/redhat-marketplace-qttdj\": dial tcp 38.129.56.252:6443: connect: connection refused" Dec 03 11:22:29 crc kubenswrapper[5052]: I1203 11:22:29.316415 5052 status_manager.go:851] "Failed to get status for pod" podUID="f85e55b1a89d02b0cb034b1ea31ed45a" pod="openshift-kube-apiserver/kube-apiserver-startup-monitor-crc" err="Get \"https://api-int.crc.testing:6443/api/v1/namespaces/openshift-kube-apiserver/pods/kube-apiserver-startup-monitor-crc\": dial tcp 38.129.56.252:6443: connect: connection refused" Dec 03 11:22:29 crc kubenswrapper[5052]: I1203 11:22:29.316776 5052 status_manager.go:851] "Failed to get status for pod" podUID="afb87a28-1b7c-4b09-b186-f05da232f501" pod="openshift-marketplace/redhat-operators-dwt2v" err="Get \"https://api-int.crc.testing:6443/api/v1/namespaces/openshift-marketplace/pods/redhat-operators-dwt2v\": dial tcp 38.129.56.252:6443: connect: connection refused" Dec 03 11:22:29 crc kubenswrapper[5052]: I1203 11:22:29.317062 5052 status_manager.go:851] "Failed to get status for pod" podUID="355008cc-8721-44c4-a1e0-3714b0086286" pod="openshift-marketplace/redhat-marketplace-gqhzd" err="Get \"https://api-int.crc.testing:6443/api/v1/namespaces/openshift-marketplace/pods/redhat-marketplace-gqhzd\": dial tcp 38.129.56.252:6443: connect: connection refused" Dec 03 11:22:29 crc kubenswrapper[5052]: I1203 11:22:29.317398 5052 status_manager.go:851] "Failed to get status for pod" podUID="75f62784-7818-4a74-9c8c-4777d6db0611" pod="openshift-authentication/oauth-openshift-558db77b4-99jsx" err="Get \"https://api-int.crc.testing:6443/api/v1/namespaces/openshift-authentication/pods/oauth-openshift-558db77b4-99jsx\": dial tcp 38.129.56.252:6443: connect: connection refused" Dec 03 11:22:29 crc kubenswrapper[5052]: I1203 11:22:29.336010 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-apiserver/kube-apiserver-startup-monitor-crc" Dec 03 11:22:29 crc kubenswrapper[5052]: I1203 11:22:29.367835 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-marketplace/redhat-operators-dwt2v" Dec 03 11:22:29 crc kubenswrapper[5052]: I1203 11:22:29.368847 5052 status_manager.go:851] "Failed to get status for pod" podUID="75f62784-7818-4a74-9c8c-4777d6db0611" pod="openshift-authentication/oauth-openshift-558db77b4-99jsx" err="Get \"https://api-int.crc.testing:6443/api/v1/namespaces/openshift-authentication/pods/oauth-openshift-558db77b4-99jsx\": dial tcp 38.129.56.252:6443: connect: connection refused" Dec 03 11:22:29 crc kubenswrapper[5052]: I1203 11:22:29.369415 5052 status_manager.go:851] "Failed to get status for pod" podUID="f4b27818a5e8e43d0dc095d08835c792" pod="openshift-kube-apiserver/kube-apiserver-crc" err="Get \"https://api-int.crc.testing:6443/api/v1/namespaces/openshift-kube-apiserver/pods/kube-apiserver-crc\": dial tcp 38.129.56.252:6443: connect: connection refused" Dec 03 11:22:29 crc kubenswrapper[5052]: I1203 11:22:29.369701 5052 status_manager.go:851] "Failed to get status for pod" podUID="8cf9b98e-2ea9-47ee-bd1b-a913714b7668" pod="openshift-marketplace/redhat-marketplace-qttdj" err="Get \"https://api-int.crc.testing:6443/api/v1/namespaces/openshift-marketplace/pods/redhat-marketplace-qttdj\": dial tcp 38.129.56.252:6443: connect: connection refused" Dec 03 11:22:29 crc kubenswrapper[5052]: I1203 11:22:29.369971 5052 status_manager.go:851] "Failed to get status for pod" podUID="f85e55b1a89d02b0cb034b1ea31ed45a" pod="openshift-kube-apiserver/kube-apiserver-startup-monitor-crc" err="Get \"https://api-int.crc.testing:6443/api/v1/namespaces/openshift-kube-apiserver/pods/kube-apiserver-startup-monitor-crc\": dial tcp 38.129.56.252:6443: connect: connection refused" Dec 03 11:22:29 crc kubenswrapper[5052]: I1203 11:22:29.370359 5052 status_manager.go:851] "Failed to get status for pod" podUID="afb87a28-1b7c-4b09-b186-f05da232f501" pod="openshift-marketplace/redhat-operators-dwt2v" err="Get \"https://api-int.crc.testing:6443/api/v1/namespaces/openshift-marketplace/pods/redhat-operators-dwt2v\": dial tcp 38.129.56.252:6443: connect: connection refused" Dec 03 11:22:29 crc kubenswrapper[5052]: I1203 11:22:29.370799 5052 status_manager.go:851] "Failed to get status for pod" podUID="355008cc-8721-44c4-a1e0-3714b0086286" pod="openshift-marketplace/redhat-marketplace-gqhzd" err="Get \"https://api-int.crc.testing:6443/api/v1/namespaces/openshift-marketplace/pods/redhat-marketplace-gqhzd\": dial tcp 38.129.56.252:6443: connect: connection refused" Dec 03 11:22:29 crc kubenswrapper[5052]: W1203 11:22:29.376605 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf85e55b1a89d02b0cb034b1ea31ed45a.slice/crio-8db0a9436e0123a1b4be924130b4607275f4bfed911879f2ee5fb9ad4293bc76 WatchSource:0}: Error finding container 8db0a9436e0123a1b4be924130b4607275f4bfed911879f2ee5fb9ad4293bc76: Status 404 returned error can't find the container with id 8db0a9436e0123a1b4be924130b4607275f4bfed911879f2ee5fb9ad4293bc76 Dec 03 11:22:29 crc kubenswrapper[5052]: I1203 11:22:29.679259 5052 status_manager.go:851] "Failed to get status for pod" podUID="75f62784-7818-4a74-9c8c-4777d6db0611" pod="openshift-authentication/oauth-openshift-558db77b4-99jsx" err="Get \"https://api-int.crc.testing:6443/api/v1/namespaces/openshift-authentication/pods/oauth-openshift-558db77b4-99jsx\": dial tcp 38.129.56.252:6443: connect: connection refused" Dec 03 11:22:29 crc kubenswrapper[5052]: I1203 11:22:29.682094 5052 status_manager.go:851] "Failed to get status for pod" podUID="f4b27818a5e8e43d0dc095d08835c792" pod="openshift-kube-apiserver/kube-apiserver-crc" err="Get \"https://api-int.crc.testing:6443/api/v1/namespaces/openshift-kube-apiserver/pods/kube-apiserver-crc\": dial tcp 38.129.56.252:6443: connect: connection refused" Dec 03 11:22:29 crc kubenswrapper[5052]: I1203 11:22:29.682569 5052 status_manager.go:851] "Failed to get status for pod" podUID="8cf9b98e-2ea9-47ee-bd1b-a913714b7668" pod="openshift-marketplace/redhat-marketplace-qttdj" err="Get \"https://api-int.crc.testing:6443/api/v1/namespaces/openshift-marketplace/pods/redhat-marketplace-qttdj\": dial tcp 38.129.56.252:6443: connect: connection refused" Dec 03 11:22:29 crc kubenswrapper[5052]: I1203 11:22:29.682899 5052 status_manager.go:851] "Failed to get status for pod" podUID="f85e55b1a89d02b0cb034b1ea31ed45a" pod="openshift-kube-apiserver/kube-apiserver-startup-monitor-crc" err="Get \"https://api-int.crc.testing:6443/api/v1/namespaces/openshift-kube-apiserver/pods/kube-apiserver-startup-monitor-crc\": dial tcp 38.129.56.252:6443: connect: connection refused" Dec 03 11:22:29 crc kubenswrapper[5052]: I1203 11:22:29.683466 5052 status_manager.go:851] "Failed to get status for pod" podUID="afb87a28-1b7c-4b09-b186-f05da232f501" pod="openshift-marketplace/redhat-operators-dwt2v" err="Get \"https://api-int.crc.testing:6443/api/v1/namespaces/openshift-marketplace/pods/redhat-operators-dwt2v\": dial tcp 38.129.56.252:6443: connect: connection refused" Dec 03 11:22:29 crc kubenswrapper[5052]: I1203 11:22:29.683861 5052 status_manager.go:851] "Failed to get status for pod" podUID="355008cc-8721-44c4-a1e0-3714b0086286" pod="openshift-marketplace/redhat-marketplace-gqhzd" err="Get \"https://api-int.crc.testing:6443/api/v1/namespaces/openshift-marketplace/pods/redhat-marketplace-gqhzd\": dial tcp 38.129.56.252:6443: connect: connection refused" Dec 03 11:22:29 crc kubenswrapper[5052]: I1203 11:22:29.684424 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-marketplace/redhat-operators-nq2qk" Dec 03 11:22:29 crc kubenswrapper[5052]: I1203 11:22:29.684916 5052 status_manager.go:851] "Failed to get status for pod" podUID="8cf9b98e-2ea9-47ee-bd1b-a913714b7668" pod="openshift-marketplace/redhat-marketplace-qttdj" err="Get \"https://api-int.crc.testing:6443/api/v1/namespaces/openshift-marketplace/pods/redhat-marketplace-qttdj\": dial tcp 38.129.56.252:6443: connect: connection refused" Dec 03 11:22:29 crc kubenswrapper[5052]: I1203 11:22:29.685426 5052 status_manager.go:851] "Failed to get status for pod" podUID="f85e55b1a89d02b0cb034b1ea31ed45a" pod="openshift-kube-apiserver/kube-apiserver-startup-monitor-crc" err="Get \"https://api-int.crc.testing:6443/api/v1/namespaces/openshift-kube-apiserver/pods/kube-apiserver-startup-monitor-crc\": dial tcp 38.129.56.252:6443: connect: connection refused" Dec 03 11:22:29 crc kubenswrapper[5052]: I1203 11:22:29.685798 5052 status_manager.go:851] "Failed to get status for pod" podUID="afb87a28-1b7c-4b09-b186-f05da232f501" pod="openshift-marketplace/redhat-operators-dwt2v" err="Get \"https://api-int.crc.testing:6443/api/v1/namespaces/openshift-marketplace/pods/redhat-operators-dwt2v\": dial tcp 38.129.56.252:6443: connect: connection refused" Dec 03 11:22:29 crc kubenswrapper[5052]: I1203 11:22:29.686133 5052 status_manager.go:851] "Failed to get status for pod" podUID="1e97c072-bdcc-4095-9a87-ddf7d4f2d22b" pod="openshift-marketplace/redhat-operators-nq2qk" err="Get \"https://api-int.crc.testing:6443/api/v1/namespaces/openshift-marketplace/pods/redhat-operators-nq2qk\": dial tcp 38.129.56.252:6443: connect: connection refused" Dec 03 11:22:29 crc kubenswrapper[5052]: I1203 11:22:29.686373 5052 status_manager.go:851] "Failed to get status for pod" podUID="355008cc-8721-44c4-a1e0-3714b0086286" pod="openshift-marketplace/redhat-marketplace-gqhzd" err="Get \"https://api-int.crc.testing:6443/api/v1/namespaces/openshift-marketplace/pods/redhat-marketplace-gqhzd\": dial tcp 38.129.56.252:6443: connect: connection refused" Dec 03 11:22:29 crc kubenswrapper[5052]: I1203 11:22:29.686766 5052 status_manager.go:851] "Failed to get status for pod" podUID="75f62784-7818-4a74-9c8c-4777d6db0611" pod="openshift-authentication/oauth-openshift-558db77b4-99jsx" err="Get \"https://api-int.crc.testing:6443/api/v1/namespaces/openshift-authentication/pods/oauth-openshift-558db77b4-99jsx\": dial tcp 38.129.56.252:6443: connect: connection refused" Dec 03 11:22:29 crc kubenswrapper[5052]: I1203 11:22:29.720534 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-marketplace/redhat-operators-nq2qk" Dec 03 11:22:29 crc kubenswrapper[5052]: I1203 11:22:29.721316 5052 status_manager.go:851] "Failed to get status for pod" podUID="f85e55b1a89d02b0cb034b1ea31ed45a" pod="openshift-kube-apiserver/kube-apiserver-startup-monitor-crc" err="Get \"https://api-int.crc.testing:6443/api/v1/namespaces/openshift-kube-apiserver/pods/kube-apiserver-startup-monitor-crc\": dial tcp 38.129.56.252:6443: connect: connection refused" Dec 03 11:22:29 crc kubenswrapper[5052]: I1203 11:22:29.721662 5052 status_manager.go:851] "Failed to get status for pod" podUID="afb87a28-1b7c-4b09-b186-f05da232f501" pod="openshift-marketplace/redhat-operators-dwt2v" err="Get \"https://api-int.crc.testing:6443/api/v1/namespaces/openshift-marketplace/pods/redhat-operators-dwt2v\": dial tcp 38.129.56.252:6443: connect: connection refused" Dec 03 11:22:29 crc kubenswrapper[5052]: I1203 11:22:29.722123 5052 status_manager.go:851] "Failed to get status for pod" podUID="1e97c072-bdcc-4095-9a87-ddf7d4f2d22b" pod="openshift-marketplace/redhat-operators-nq2qk" err="Get \"https://api-int.crc.testing:6443/api/v1/namespaces/openshift-marketplace/pods/redhat-operators-nq2qk\": dial tcp 38.129.56.252:6443: connect: connection refused" Dec 03 11:22:29 crc kubenswrapper[5052]: I1203 11:22:29.722296 5052 status_manager.go:851] "Failed to get status for pod" podUID="355008cc-8721-44c4-a1e0-3714b0086286" pod="openshift-marketplace/redhat-marketplace-gqhzd" err="Get \"https://api-int.crc.testing:6443/api/v1/namespaces/openshift-marketplace/pods/redhat-marketplace-gqhzd\": dial tcp 38.129.56.252:6443: connect: connection refused" Dec 03 11:22:29 crc kubenswrapper[5052]: I1203 11:22:29.722457 5052 status_manager.go:851] "Failed to get status for pod" podUID="75f62784-7818-4a74-9c8c-4777d6db0611" pod="openshift-authentication/oauth-openshift-558db77b4-99jsx" err="Get \"https://api-int.crc.testing:6443/api/v1/namespaces/openshift-authentication/pods/oauth-openshift-558db77b4-99jsx\": dial tcp 38.129.56.252:6443: connect: connection refused" Dec 03 11:22:29 crc kubenswrapper[5052]: I1203 11:22:29.722704 5052 status_manager.go:851] "Failed to get status for pod" podUID="8cf9b98e-2ea9-47ee-bd1b-a913714b7668" pod="openshift-marketplace/redhat-marketplace-qttdj" err="Get \"https://api-int.crc.testing:6443/api/v1/namespaces/openshift-marketplace/pods/redhat-marketplace-qttdj\": dial tcp 38.129.56.252:6443: connect: connection refused" Dec 03 11:22:29 crc kubenswrapper[5052]: E1203 11:22:29.793077 5052 kubelet_node_status.go:585] "Error updating node status, will retry" err="failed to patch status \"{\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"type\\\":\\\"DiskPressure\\\"},{\\\"type\\\":\\\"PIDPressure\\\"},{\\\"type\\\":\\\"Ready\\\"}],\\\"conditions\\\":[{\\\"lastHeartbeatTime\\\":\\\"2025-12-03T11:22:29Z\\\",\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2025-12-03T11:22:29Z\\\",\\\"type\\\":\\\"DiskPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2025-12-03T11:22:29Z\\\",\\\"type\\\":\\\"PIDPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2025-12-03T11:22:29Z\\\",\\\"type\\\":\\\"Ready\\\"}],\\\"images\\\":[{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b9ea248f8ca33258fe1683da51d2b16b94630be1b361c65f68a16c1a34b94887\\\"],\\\"sizeBytes\\\":2887430265},{\\\"names\\\":[\\\"registry.redhat.io/redhat/redhat-operator-index@sha256:416a7dc57b2b95775e679e0ab93111baaa063e55a4c6d73856a248d85a2debbd\\\",\\\"registry.redhat.io/redhat/redhat-operator-index@sha256:5cdd97eed164a2eda9842fb91d284f06d2e63d69af9a98001fca2d6cebd0b52a\\\",\\\"registry.redhat.io/redhat/redhat-operator-index:v4.18\\\"],\\\"sizeBytes\\\":1609873225},{\\\"names\\\":[\\\"registry.redhat.io/redhat/redhat-operator-index@sha256:4a62fa1c0091f6d94e8fb7258470b9a532d78364b6b51a05341592041d598562\\\"],\\\"sizeBytes\\\":1523204510},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:174f36cdd47ef0d1d2099482919d773257453265a2af0b17b154edc32fa41ac2\\\"],\\\"sizeBytes\\\":1498102846},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7eeaee65f2808b819eedb413bdcabb9144e12f0dd97f13fd1afba93a95b67b26\\\"],\\\"sizeBytes\\\":1232839934},{\\\"names\\\":[\\\"registry.redhat.io/redhat/certified-operator-index@sha256:2041f196d215ed63fc3218d5a0e0cb8409dd5b575612d999cffc9817ca7ee2c4\\\",\\\"registry.redhat.io/redhat/certified-operator-index@sha256:9f4887ff6cc4a4c64b413cf3ffbbceababd44911f3d138b5dc10dcb622ccd23c\\\",\\\"registry.redhat.io/redhat/certified-operator-index:v4.18\\\"],\\\"sizeBytes\\\":1204906320},{\\\"names\\\":[\\\"registry.redhat.io/redhat/community-operator-index@sha256:dba4a8e0293f7b6e1459b74484a7126274bdc9efa75f808eb15e5a3896a3c818\\\",\\\"registry.redhat.io/redhat/community-operator-index@sha256:eee64597300e249e4bead6abda4d235cc4fc3a87be82e3e9f582609602ed87d7\\\",\\\"registry.redhat.io/redhat/community-operator-index:v4.18\\\"],\\\"sizeBytes\\\":1201319250},{\\\"names\\\":[\\\"registry.redhat.io/redhat/community-operator-index@sha256:8ff55cdb2367f5011074d2f5ebdc153b8885e7495e14ae00f99d2b7ab3584ade\\\"],\\\"sizeBytes\\\":1151049424},{\\\"names\\\":[\\\"registry.redhat.io/redhat/redhat-marketplace-index@sha256:e8990432556acad31519b1a73ec32f32d27c2034cf9e5cc4db8980efc7331594\\\",\\\"registry.redhat.io/redhat/redhat-marketplace-index@sha256:ebe9f523f5c211a3a0f2570331dddcd5be15b12c1fecd9b8b121f881bfaad029\\\",\\\"registry.redhat.io/redhat/redhat-marketplace-index:v4.18\\\"],\\\"sizeBytes\\\":1129027903},{\\\"names\\\":[\\\"registry.redhat.io/redhat/certified-operator-index@sha256:7688bce5eb0d153adff87fc9f7a47642465c0b88208efb236880197969931b37\\\"],\\\"sizeBytes\\\":1032059094},{\\\"names\\\":[\\\"registry.redhat.io/redhat/redhat-marketplace-index@sha256:1dc15c170ebf462dacaef75511740ed94ca1da210f3980f66d77f91ba201c875\\\"],\\\"sizeBytes\\\":1001152198},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c915fb8ba96e911699a1ae34a8e95ca8a9fbe1bf8c28fea177225c63a8bdfc0a\\\"],\\\"sizeBytes\\\":964552795},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:06bc35825771aee1220d34720243b89c4ba8a8b335e6de2597126bd791fd90d4\\\"],\\\"sizeBytes\\\":947616130},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c3cc3840d7a81ce1b420f06e07a923861faf37d9c10688aa3aa0b7b76c8706ad\\\"],\\\"sizeBytes\\\":907837715},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:101f295e2eae0755ae1865f7de885db1f17b9368e4120a713bb5f79e17ce8f93\\\"],\\\"sizeBytes\\\":854694423},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:47b0670fa1051335fd2d2c9e8361e4ed77c7760c33a2180b136f7c7f59863ec2\\\"],\\\"sizeBytes\\\":852490370},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:862f4a4bed52f372056b6d368e2498ebfb063075b31cf48dbdaaeedfcf0396cb\\\"],\\\"sizeBytes\\\":772592048},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:98100674616e54319f6713d742fd0c3bdbc84e6e6173e8ccf4a2473a714c2bc4\\\"],\\\"sizeBytes\\\":705793115},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:687fddfbb085a1688df312ce4ec8c857df9b2daed8ff4a7ed6163a1154afa2cc\\\"],\\\"sizeBytes\\\":687915987},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f247257b0885cf5d303e3612c7714b33ae51404cfa2429822060c6c025eb17dd\\\"],\\\"sizeBytes\\\":668060419},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e1baa38811c04bd8909e01a1f3be7421a1cb99d608d3dc4cf86d95b17de2ab8b\\\"],\\\"sizeBytes\\\":613826183},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7e3e9dc0b02b9351edf7c46b1d46d724abd1ac38ecbd6bc541cee84a209258d8\\\"],\\\"sizeBytes\\\":581863411},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:35512335ac39aed0f55b7f799f416f4f6445c20c1b19888cf2bb72bb276703f2\\\"],\\\"sizeBytes\\\":574606365},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ee8d8f089ec1488067444c7e276c4e47cc93840280f3b3295484d67af2232002\\\"],\\\"sizeBytes\\\":550676059},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:10f20a39f16ae3019c62261eda8beb9e4d8c36cbb7b500b3bae1312987f0685d\\\"],\\\"sizeBytes\\\":541458174},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e40792096b162f0f9ce5f8362f51e5f8dea2c1ce4b1447235388416b5db7708c\\\"],\\\"sizeBytes\\\":533092226},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:07b7c6877441ecd6a5646fb68e33e9be8b90092272e49117b54b4a67314731ca\\\"],\\\"sizeBytes\\\":528023732},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a0fa3723269019bee1847b26702f42928e779036cc2f58408f8ee7866be30a93\\\"],\\\"sizeBytes\\\":510867594},{\\\"names\\\":[\\\"quay.io/crcont/ocp-release@sha256:0b6ae0d091d2bf49f9b3a3aff54aabdc49e70c783780f118789f49d8f95a9e03\\\"],\\\"sizeBytes\\\":510526836},{\\\"names\\\":[\\\"quay.io/crcont/openshift-crc-cluster-kube-apiserver-operator@sha256:9f36dc276e27753fc478274c7f7814a4f8945c987117ee1ea3b8e6355e6d7462\\\"],\\\"sizeBytes\\\":507459597},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7e9e7dd2b1a8394b7490ca6df8a3ee8cdfc6193ecc6fb6173ed9a1868116a207\\\"],\\\"sizeBytes\\\":505721947},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:094bb6a6641b4edbaf932f0551bcda20b0d4e012cbe84207348b24eeabd351e9\\\"],\\\"sizeBytes\\\":504778226},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c69fe7a98a744b7a7b61b2a8db81a338f373cd2b1d46c6d3f02864b30c37e46c\\\"],\\\"sizeBytes\\\":504735878},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e51e6f78ec20ef91c82e94a49f950e427e77894e582dcc406eec4df807ddd76e\\\"],\\\"sizeBytes\\\":502943148},{\\\"names\\\":[\\\"quay.io/crcont/openshift-crc-cluster-kube-controller-manager-operator@sha256:8506ce0a578bc18fac117eb2b82799488ffac0bed08287faaf92edaf5d17ab95\\\"],\\\"sizeBytes\\\":501379880},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3a741253807c962189819d879b8fef94a9452fb3f5f3969ec3207eb2d9862205\\\"],\\\"sizeBytes\\\":500472212},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:5b881c97aa8e440c6b3ca001edfd789a9380066b8f11f35a8dd8d88c5c7dbf86\\\"],\\\"sizeBytes\\\":498888951},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:5aa9e5379bfeb63f4e517fb45168eb6820138041641bbdfc6f4db6427032fa37\\\"],\\\"sizeBytes\\\":497832828},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c0f9da410c07372b6c9ad6a79379b491cd10fdee88051c026b084652d85aed21\\\"],\\\"sizeBytes\\\":497742284},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:88b1f0a05a1b1c91e1212b40f0e7d04c9351ec9d34c52097bfdc5897b46f2f0e\\\"],\\\"sizeBytes\\\":497120598},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:737e9019a072c74321e0a909ca95481f5c545044dd4f151a34d0e1c8b9cf273f\\\"],\\\"sizeBytes\\\":488494681},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:fe009d03910e18795e3bd60a3fd84938311d464d2730a2af5ded5b24e4d05a6b\\\"],\\\"sizeBytes\\\":487097366},{\\\"names\\\":[\\\"registry.redhat.io/openshift4/ose-csi-external-provisioner@sha256:66760a53b64d381940757ca9f0d05f523a61f943f8da03ce9791e5d05264a736\\\",\\\"registry.redhat.io/openshift4/ose-csi-external-provisioner@sha256:e97a0cb5b6119a9735efe0ac24630a8912fcad89a1dddfa76dc10edac4ec9815\\\",\\\"registry.redhat.io/openshift4/ose-csi-external-provisioner:latest\\\"],\\\"sizeBytes\\\":485998616},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:9fa29d188c85a8b1e1bd15c9c18e96f1b235da9bd4a45dbc086a4a69520ed63f\\\"],\\\"sizeBytes\\\":485767738},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:898cae57123c5006d397b24af21b0f24a0c42c9b0be5ee8251e1824711f65820\\\"],\\\"sizeBytes\\\":485535312},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:1eda5ad6a6c5b9cd94b4b456e9116f4a0517241b614de1a99df14baee20c3e6a\\\"],\\\"sizeBytes\\\":479585218},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:487c0a8d5200bcdce484ab1169229d8fcb8e91a934be45afff7819c4f7612f57\\\"],\\\"sizeBytes\\\":476681373},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b641ed0d63034b23d07eb0b2cd455390e83b186e77375e2d3f37633c1ddb0495\\\"],\\\"sizeBytes\\\":473958144},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:32f9e10dfb8a7c812ea8b3e71a42bed9cef05305be18cc368b666df4643ba717\\\"],\\\"sizeBytes\\\":463179365},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8fdf28927b06a42ea8af3985d558c84d9efd142bb32d3892c4fa9f5e0d98133c\\\"],\\\"sizeBytes\\\":460774792}]}}\" for node \"crc\": Patch \"https://api-int.crc.testing:6443/api/v1/nodes/crc/status?timeout=10s\": dial tcp 38.129.56.252:6443: connect: connection refused" Dec 03 11:22:29 crc kubenswrapper[5052]: E1203 11:22:29.793874 5052 kubelet_node_status.go:585] "Error updating node status, will retry" err="error getting node \"crc\": Get \"https://api-int.crc.testing:6443/api/v1/nodes/crc?timeout=10s\": dial tcp 38.129.56.252:6443: connect: connection refused" Dec 03 11:22:29 crc kubenswrapper[5052]: E1203 11:22:29.794840 5052 kubelet_node_status.go:585] "Error updating node status, will retry" err="error getting node \"crc\": Get \"https://api-int.crc.testing:6443/api/v1/nodes/crc?timeout=10s\": dial tcp 38.129.56.252:6443: connect: connection refused" Dec 03 11:22:29 crc kubenswrapper[5052]: E1203 11:22:29.795106 5052 kubelet_node_status.go:585] "Error updating node status, will retry" err="error getting node \"crc\": Get \"https://api-int.crc.testing:6443/api/v1/nodes/crc?timeout=10s\": dial tcp 38.129.56.252:6443: connect: connection refused" Dec 03 11:22:29 crc kubenswrapper[5052]: E1203 11:22:29.795416 5052 kubelet_node_status.go:585] "Error updating node status, will retry" err="error getting node \"crc\": Get \"https://api-int.crc.testing:6443/api/v1/nodes/crc?timeout=10s\": dial tcp 38.129.56.252:6443: connect: connection refused" Dec 03 11:22:29 crc kubenswrapper[5052]: E1203 11:22:29.795450 5052 kubelet_node_status.go:572] "Unable to update node status" err="update node status exceeds retry count" Dec 03 11:22:29 crc kubenswrapper[5052]: I1203 11:22:29.935961 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-zbr2b\" (UniqueName: \"kubernetes.io/projected/c712fba2-c530-4f64-836d-16706a2e2b50-kube-api-access-zbr2b\") pod \"oauth-openshift-6f85f596d6-nhvd7\" (UID: \"c712fba2-c530-4f64-836d-16706a2e2b50\") " pod="openshift-authentication/oauth-openshift-6f85f596d6-nhvd7" Dec 03 11:22:29 crc kubenswrapper[5052]: E1203 11:22:29.936744 5052 projected.go:194] Error preparing data for projected volume kube-api-access-zbr2b for pod openshift-authentication/oauth-openshift-6f85f596d6-nhvd7: failed to fetch token: Post "https://api-int.crc.testing:6443/api/v1/namespaces/openshift-authentication/serviceaccounts/oauth-openshift/token": dial tcp 38.129.56.252:6443: connect: connection refused Dec 03 11:22:29 crc kubenswrapper[5052]: E1203 11:22:29.936808 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c712fba2-c530-4f64-836d-16706a2e2b50-kube-api-access-zbr2b podName:c712fba2-c530-4f64-836d-16706a2e2b50 nodeName:}" failed. No retries permitted until 2025-12-03 11:22:31.936789857 +0000 UTC m=+142.525031621 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-zbr2b" (UniqueName: "kubernetes.io/projected/c712fba2-c530-4f64-836d-16706a2e2b50-kube-api-access-zbr2b") pod "oauth-openshift-6f85f596d6-nhvd7" (UID: "c712fba2-c530-4f64-836d-16706a2e2b50") : failed to fetch token: Post "https://api-int.crc.testing:6443/api/v1/namespaces/openshift-authentication/serviceaccounts/oauth-openshift/token": dial tcp 38.129.56.252:6443: connect: connection refused Dec 03 11:22:30 crc kubenswrapper[5052]: I1203 11:22:30.346034 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-apiserver_kube-apiserver-crc_f4b27818a5e8e43d0dc095d08835c792/kube-apiserver-check-endpoints/0.log" Dec 03 11:22:30 crc kubenswrapper[5052]: I1203 11:22:30.348706 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-apiserver_kube-apiserver-crc_f4b27818a5e8e43d0dc095d08835c792/kube-apiserver-cert-syncer/0.log" Dec 03 11:22:30 crc kubenswrapper[5052]: I1203 11:22:30.349635 5052 generic.go:334] "Generic (PLEG): container finished" podID="f4b27818a5e8e43d0dc095d08835c792" containerID="ee9c5c61cfd6dee9fd44ebc4c7ef4d20b5fde82b12a8147c3c871e2615da7940" exitCode=0 Dec 03 11:22:30 crc kubenswrapper[5052]: I1203 11:22:30.349671 5052 generic.go:334] "Generic (PLEG): container finished" podID="f4b27818a5e8e43d0dc095d08835c792" containerID="22b8cb6aa04337d1420f97d3e128fce2b29a725ca6e5d157adeb1ba78f00073f" exitCode=0 Dec 03 11:22:30 crc kubenswrapper[5052]: I1203 11:22:30.349682 5052 generic.go:334] "Generic (PLEG): container finished" podID="f4b27818a5e8e43d0dc095d08835c792" containerID="15abb0d7fd5884519ed5ac454430dbb58f71ed1f459e5ed30f324f70befd243e" exitCode=0 Dec 03 11:22:30 crc kubenswrapper[5052]: I1203 11:22:30.349691 5052 generic.go:334] "Generic (PLEG): container finished" podID="f4b27818a5e8e43d0dc095d08835c792" containerID="8ceedd21ae3833d77fd440537f30434ee57f087fbefb2357b46231d3f778ad2a" exitCode=2 Dec 03 11:22:30 crc kubenswrapper[5052]: I1203 11:22:30.349777 5052 scope.go:117] "RemoveContainer" containerID="b739f84524d09b27f6e052e21df8be3870a2136665704232f5a365d8a5bf41f5" Dec 03 11:22:30 crc kubenswrapper[5052]: I1203 11:22:30.354517 5052 generic.go:334] "Generic (PLEG): container finished" podID="68a245d3-ddaf-40a0-92f6-92760173b7a3" containerID="1f566de232b91a5d9b8cc04d57b9441b861075111c8d838e330f0a5a1294f19a" exitCode=0 Dec 03 11:22:30 crc kubenswrapper[5052]: I1203 11:22:30.354542 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-kube-apiserver/installer-9-crc" event={"ID":"68a245d3-ddaf-40a0-92f6-92760173b7a3","Type":"ContainerDied","Data":"1f566de232b91a5d9b8cc04d57b9441b861075111c8d838e330f0a5a1294f19a"} Dec 03 11:22:30 crc kubenswrapper[5052]: I1203 11:22:30.355281 5052 status_manager.go:851] "Failed to get status for pod" podUID="1e97c072-bdcc-4095-9a87-ddf7d4f2d22b" pod="openshift-marketplace/redhat-operators-nq2qk" err="Get \"https://api-int.crc.testing:6443/api/v1/namespaces/openshift-marketplace/pods/redhat-operators-nq2qk\": dial tcp 38.129.56.252:6443: connect: connection refused" Dec 03 11:22:30 crc kubenswrapper[5052]: I1203 11:22:30.355703 5052 status_manager.go:851] "Failed to get status for pod" podUID="68a245d3-ddaf-40a0-92f6-92760173b7a3" pod="openshift-kube-apiserver/installer-9-crc" err="Get \"https://api-int.crc.testing:6443/api/v1/namespaces/openshift-kube-apiserver/pods/installer-9-crc\": dial tcp 38.129.56.252:6443: connect: connection refused" Dec 03 11:22:30 crc kubenswrapper[5052]: I1203 11:22:30.356188 5052 status_manager.go:851] "Failed to get status for pod" podUID="355008cc-8721-44c4-a1e0-3714b0086286" pod="openshift-marketplace/redhat-marketplace-gqhzd" err="Get \"https://api-int.crc.testing:6443/api/v1/namespaces/openshift-marketplace/pods/redhat-marketplace-gqhzd\": dial tcp 38.129.56.252:6443: connect: connection refused" Dec 03 11:22:30 crc kubenswrapper[5052]: I1203 11:22:30.356564 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-kube-apiserver/kube-apiserver-startup-monitor-crc" event={"ID":"f85e55b1a89d02b0cb034b1ea31ed45a","Type":"ContainerStarted","Data":"d689877e29d83970d21d5d92e2788fd82d91a2fad5604a21a43190e8e96a0ed6"} Dec 03 11:22:30 crc kubenswrapper[5052]: I1203 11:22:30.356593 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-kube-apiserver/kube-apiserver-startup-monitor-crc" event={"ID":"f85e55b1a89d02b0cb034b1ea31ed45a","Type":"ContainerStarted","Data":"8db0a9436e0123a1b4be924130b4607275f4bfed911879f2ee5fb9ad4293bc76"} Dec 03 11:22:30 crc kubenswrapper[5052]: I1203 11:22:30.356624 5052 status_manager.go:851] "Failed to get status for pod" podUID="75f62784-7818-4a74-9c8c-4777d6db0611" pod="openshift-authentication/oauth-openshift-558db77b4-99jsx" err="Get \"https://api-int.crc.testing:6443/api/v1/namespaces/openshift-authentication/pods/oauth-openshift-558db77b4-99jsx\": dial tcp 38.129.56.252:6443: connect: connection refused" Dec 03 11:22:30 crc kubenswrapper[5052]: I1203 11:22:30.356848 5052 status_manager.go:851] "Failed to get status for pod" podUID="8cf9b98e-2ea9-47ee-bd1b-a913714b7668" pod="openshift-marketplace/redhat-marketplace-qttdj" err="Get \"https://api-int.crc.testing:6443/api/v1/namespaces/openshift-marketplace/pods/redhat-marketplace-qttdj\": dial tcp 38.129.56.252:6443: connect: connection refused" Dec 03 11:22:30 crc kubenswrapper[5052]: I1203 11:22:30.357045 5052 status_manager.go:851] "Failed to get status for pod" podUID="f85e55b1a89d02b0cb034b1ea31ed45a" pod="openshift-kube-apiserver/kube-apiserver-startup-monitor-crc" err="Get \"https://api-int.crc.testing:6443/api/v1/namespaces/openshift-kube-apiserver/pods/kube-apiserver-startup-monitor-crc\": dial tcp 38.129.56.252:6443: connect: connection refused" Dec 03 11:22:30 crc kubenswrapper[5052]: I1203 11:22:30.357208 5052 status_manager.go:851] "Failed to get status for pod" podUID="afb87a28-1b7c-4b09-b186-f05da232f501" pod="openshift-marketplace/redhat-operators-dwt2v" err="Get \"https://api-int.crc.testing:6443/api/v1/namespaces/openshift-marketplace/pods/redhat-operators-dwt2v\": dial tcp 38.129.56.252:6443: connect: connection refused" Dec 03 11:22:30 crc kubenswrapper[5052]: E1203 11:22:30.397065 5052 event.go:368] "Unable to write event (may retry after sleeping)" err="Post \"https://api-int.crc.testing:6443/api/v1/namespaces/openshift-authentication/events\": dial tcp 38.129.56.252:6443: connect: connection refused" event="&Event{ObjectMeta:{oauth-openshift-6f85f596d6-nhvd7.187db0bb7e2c92de openshift-authentication 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:openshift-authentication,Name:oauth-openshift-6f85f596d6-nhvd7,UID:c712fba2-c530-4f64-836d-16706a2e2b50,APIVersion:v1,ResourceVersion:29195,FieldPath:,},Reason:FailedMount,Message:MountVolume.SetUp failed for volume \"kube-api-access-zbr2b\" : failed to fetch token: Post \"https://api-int.crc.testing:6443/api/v1/namespaces/openshift-authentication/serviceaccounts/oauth-openshift/token\": dial tcp 38.129.56.252:6443: connect: connection refused,Source:EventSource{Component:kubelet,Host:crc,},FirstTimestamp:2025-12-03 11:22:28.32132579 +0000 UTC m=+138.909567554,LastTimestamp:2025-12-03 11:22:28.32132579 +0000 UTC m=+138.909567554,Count:1,Type:Warning,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:crc,}" Dec 03 11:22:31 crc kubenswrapper[5052]: I1203 11:22:31.366108 5052 status_manager.go:851] "Failed to get status for pod" podUID="1e97c072-bdcc-4095-9a87-ddf7d4f2d22b" pod="openshift-marketplace/redhat-operators-nq2qk" err="Get \"https://api-int.crc.testing:6443/api/v1/namespaces/openshift-marketplace/pods/redhat-operators-nq2qk\": dial tcp 38.129.56.252:6443: connect: connection refused" Dec 03 11:22:31 crc kubenswrapper[5052]: I1203 11:22:31.367278 5052 status_manager.go:851] "Failed to get status for pod" podUID="68a245d3-ddaf-40a0-92f6-92760173b7a3" pod="openshift-kube-apiserver/installer-9-crc" err="Get \"https://api-int.crc.testing:6443/api/v1/namespaces/openshift-kube-apiserver/pods/installer-9-crc\": dial tcp 38.129.56.252:6443: connect: connection refused" Dec 03 11:22:31 crc kubenswrapper[5052]: I1203 11:22:31.368081 5052 status_manager.go:851] "Failed to get status for pod" podUID="355008cc-8721-44c4-a1e0-3714b0086286" pod="openshift-marketplace/redhat-marketplace-gqhzd" err="Get \"https://api-int.crc.testing:6443/api/v1/namespaces/openshift-marketplace/pods/redhat-marketplace-gqhzd\": dial tcp 38.129.56.252:6443: connect: connection refused" Dec 03 11:22:31 crc kubenswrapper[5052]: I1203 11:22:31.368758 5052 status_manager.go:851] "Failed to get status for pod" podUID="75f62784-7818-4a74-9c8c-4777d6db0611" pod="openshift-authentication/oauth-openshift-558db77b4-99jsx" err="Get \"https://api-int.crc.testing:6443/api/v1/namespaces/openshift-authentication/pods/oauth-openshift-558db77b4-99jsx\": dial tcp 38.129.56.252:6443: connect: connection refused" Dec 03 11:22:31 crc kubenswrapper[5052]: I1203 11:22:31.369197 5052 status_manager.go:851] "Failed to get status for pod" podUID="8cf9b98e-2ea9-47ee-bd1b-a913714b7668" pod="openshift-marketplace/redhat-marketplace-qttdj" err="Get \"https://api-int.crc.testing:6443/api/v1/namespaces/openshift-marketplace/pods/redhat-marketplace-qttdj\": dial tcp 38.129.56.252:6443: connect: connection refused" Dec 03 11:22:31 crc kubenswrapper[5052]: I1203 11:22:31.369596 5052 status_manager.go:851] "Failed to get status for pod" podUID="f85e55b1a89d02b0cb034b1ea31ed45a" pod="openshift-kube-apiserver/kube-apiserver-startup-monitor-crc" err="Get \"https://api-int.crc.testing:6443/api/v1/namespaces/openshift-kube-apiserver/pods/kube-apiserver-startup-monitor-crc\": dial tcp 38.129.56.252:6443: connect: connection refused" Dec 03 11:22:31 crc kubenswrapper[5052]: I1203 11:22:31.369948 5052 status_manager.go:851] "Failed to get status for pod" podUID="afb87a28-1b7c-4b09-b186-f05da232f501" pod="openshift-marketplace/redhat-operators-dwt2v" err="Get \"https://api-int.crc.testing:6443/api/v1/namespaces/openshift-marketplace/pods/redhat-operators-dwt2v\": dial tcp 38.129.56.252:6443: connect: connection refused" Dec 03 11:22:31 crc kubenswrapper[5052]: I1203 11:22:31.747531 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-kube-apiserver/installer-9-crc" Dec 03 11:22:31 crc kubenswrapper[5052]: I1203 11:22:31.748387 5052 status_manager.go:851] "Failed to get status for pod" podUID="8cf9b98e-2ea9-47ee-bd1b-a913714b7668" pod="openshift-marketplace/redhat-marketplace-qttdj" err="Get \"https://api-int.crc.testing:6443/api/v1/namespaces/openshift-marketplace/pods/redhat-marketplace-qttdj\": dial tcp 38.129.56.252:6443: connect: connection refused" Dec 03 11:22:31 crc kubenswrapper[5052]: I1203 11:22:31.748609 5052 status_manager.go:851] "Failed to get status for pod" podUID="f85e55b1a89d02b0cb034b1ea31ed45a" pod="openshift-kube-apiserver/kube-apiserver-startup-monitor-crc" err="Get \"https://api-int.crc.testing:6443/api/v1/namespaces/openshift-kube-apiserver/pods/kube-apiserver-startup-monitor-crc\": dial tcp 38.129.56.252:6443: connect: connection refused" Dec 03 11:22:31 crc kubenswrapper[5052]: I1203 11:22:31.748862 5052 status_manager.go:851] "Failed to get status for pod" podUID="afb87a28-1b7c-4b09-b186-f05da232f501" pod="openshift-marketplace/redhat-operators-dwt2v" err="Get \"https://api-int.crc.testing:6443/api/v1/namespaces/openshift-marketplace/pods/redhat-operators-dwt2v\": dial tcp 38.129.56.252:6443: connect: connection refused" Dec 03 11:22:31 crc kubenswrapper[5052]: I1203 11:22:31.749542 5052 status_manager.go:851] "Failed to get status for pod" podUID="1e97c072-bdcc-4095-9a87-ddf7d4f2d22b" pod="openshift-marketplace/redhat-operators-nq2qk" err="Get \"https://api-int.crc.testing:6443/api/v1/namespaces/openshift-marketplace/pods/redhat-operators-nq2qk\": dial tcp 38.129.56.252:6443: connect: connection refused" Dec 03 11:22:31 crc kubenswrapper[5052]: I1203 11:22:31.750143 5052 status_manager.go:851] "Failed to get status for pod" podUID="68a245d3-ddaf-40a0-92f6-92760173b7a3" pod="openshift-kube-apiserver/installer-9-crc" err="Get \"https://api-int.crc.testing:6443/api/v1/namespaces/openshift-kube-apiserver/pods/installer-9-crc\": dial tcp 38.129.56.252:6443: connect: connection refused" Dec 03 11:22:31 crc kubenswrapper[5052]: I1203 11:22:31.750504 5052 status_manager.go:851] "Failed to get status for pod" podUID="355008cc-8721-44c4-a1e0-3714b0086286" pod="openshift-marketplace/redhat-marketplace-gqhzd" err="Get \"https://api-int.crc.testing:6443/api/v1/namespaces/openshift-marketplace/pods/redhat-marketplace-gqhzd\": dial tcp 38.129.56.252:6443: connect: connection refused" Dec 03 11:22:31 crc kubenswrapper[5052]: I1203 11:22:31.750855 5052 status_manager.go:851] "Failed to get status for pod" podUID="75f62784-7818-4a74-9c8c-4777d6db0611" pod="openshift-authentication/oauth-openshift-558db77b4-99jsx" err="Get \"https://api-int.crc.testing:6443/api/v1/namespaces/openshift-authentication/pods/oauth-openshift-558db77b4-99jsx\": dial tcp 38.129.56.252:6443: connect: connection refused" Dec 03 11:22:31 crc kubenswrapper[5052]: I1203 11:22:31.873062 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access\" (UniqueName: \"kubernetes.io/projected/68a245d3-ddaf-40a0-92f6-92760173b7a3-kube-api-access\") pod \"68a245d3-ddaf-40a0-92f6-92760173b7a3\" (UID: \"68a245d3-ddaf-40a0-92f6-92760173b7a3\") " Dec 03 11:22:31 crc kubenswrapper[5052]: I1203 11:22:31.873148 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/68a245d3-ddaf-40a0-92f6-92760173b7a3-kubelet-dir\") pod \"68a245d3-ddaf-40a0-92f6-92760173b7a3\" (UID: \"68a245d3-ddaf-40a0-92f6-92760173b7a3\") " Dec 03 11:22:31 crc kubenswrapper[5052]: I1203 11:22:31.873164 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"var-lock\" (UniqueName: \"kubernetes.io/host-path/68a245d3-ddaf-40a0-92f6-92760173b7a3-var-lock\") pod \"68a245d3-ddaf-40a0-92f6-92760173b7a3\" (UID: \"68a245d3-ddaf-40a0-92f6-92760173b7a3\") " Dec 03 11:22:31 crc kubenswrapper[5052]: I1203 11:22:31.873252 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/68a245d3-ddaf-40a0-92f6-92760173b7a3-kubelet-dir" (OuterVolumeSpecName: "kubelet-dir") pod "68a245d3-ddaf-40a0-92f6-92760173b7a3" (UID: "68a245d3-ddaf-40a0-92f6-92760173b7a3"). InnerVolumeSpecName "kubelet-dir". PluginName "kubernetes.io/host-path", VolumeGidValue "" Dec 03 11:22:31 crc kubenswrapper[5052]: I1203 11:22:31.873355 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/68a245d3-ddaf-40a0-92f6-92760173b7a3-var-lock" (OuterVolumeSpecName: "var-lock") pod "68a245d3-ddaf-40a0-92f6-92760173b7a3" (UID: "68a245d3-ddaf-40a0-92f6-92760173b7a3"). InnerVolumeSpecName "var-lock". PluginName "kubernetes.io/host-path", VolumeGidValue "" Dec 03 11:22:31 crc kubenswrapper[5052]: I1203 11:22:31.873593 5052 reconciler_common.go:293] "Volume detached for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/68a245d3-ddaf-40a0-92f6-92760173b7a3-kubelet-dir\") on node \"crc\" DevicePath \"\"" Dec 03 11:22:31 crc kubenswrapper[5052]: I1203 11:22:31.873624 5052 reconciler_common.go:293] "Volume detached for volume \"var-lock\" (UniqueName: \"kubernetes.io/host-path/68a245d3-ddaf-40a0-92f6-92760173b7a3-var-lock\") on node \"crc\" DevicePath \"\"" Dec 03 11:22:31 crc kubenswrapper[5052]: I1203 11:22:31.878673 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/68a245d3-ddaf-40a0-92f6-92760173b7a3-kube-api-access" (OuterVolumeSpecName: "kube-api-access") pod "68a245d3-ddaf-40a0-92f6-92760173b7a3" (UID: "68a245d3-ddaf-40a0-92f6-92760173b7a3"). InnerVolumeSpecName "kube-api-access". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:22:31 crc kubenswrapper[5052]: I1203 11:22:31.975000 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-zbr2b\" (UniqueName: \"kubernetes.io/projected/c712fba2-c530-4f64-836d-16706a2e2b50-kube-api-access-zbr2b\") pod \"oauth-openshift-6f85f596d6-nhvd7\" (UID: \"c712fba2-c530-4f64-836d-16706a2e2b50\") " pod="openshift-authentication/oauth-openshift-6f85f596d6-nhvd7" Dec 03 11:22:31 crc kubenswrapper[5052]: I1203 11:22:31.975169 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access\" (UniqueName: \"kubernetes.io/projected/68a245d3-ddaf-40a0-92f6-92760173b7a3-kube-api-access\") on node \"crc\" DevicePath \"\"" Dec 03 11:22:31 crc kubenswrapper[5052]: E1203 11:22:31.975754 5052 projected.go:194] Error preparing data for projected volume kube-api-access-zbr2b for pod openshift-authentication/oauth-openshift-6f85f596d6-nhvd7: failed to fetch token: Post "https://api-int.crc.testing:6443/api/v1/namespaces/openshift-authentication/serviceaccounts/oauth-openshift/token": dial tcp 38.129.56.252:6443: connect: connection refused Dec 03 11:22:31 crc kubenswrapper[5052]: E1203 11:22:31.975852 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c712fba2-c530-4f64-836d-16706a2e2b50-kube-api-access-zbr2b podName:c712fba2-c530-4f64-836d-16706a2e2b50 nodeName:}" failed. No retries permitted until 2025-12-03 11:22:35.975824255 +0000 UTC m=+146.564066019 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-zbr2b" (UniqueName: "kubernetes.io/projected/c712fba2-c530-4f64-836d-16706a2e2b50-kube-api-access-zbr2b") pod "oauth-openshift-6f85f596d6-nhvd7" (UID: "c712fba2-c530-4f64-836d-16706a2e2b50") : failed to fetch token: Post "https://api-int.crc.testing:6443/api/v1/namespaces/openshift-authentication/serviceaccounts/oauth-openshift/token": dial tcp 38.129.56.252:6443: connect: connection refused Dec 03 11:22:32 crc kubenswrapper[5052]: I1203 11:22:32.370335 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-kube-apiserver/installer-9-crc" event={"ID":"68a245d3-ddaf-40a0-92f6-92760173b7a3","Type":"ContainerDied","Data":"cf6c37cbe08d8376ca83da9d1dd8026b2962ef84c13de494526fbe9016df27fd"} Dec 03 11:22:32 crc kubenswrapper[5052]: I1203 11:22:32.370696 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cf6c37cbe08d8376ca83da9d1dd8026b2962ef84c13de494526fbe9016df27fd" Dec 03 11:22:32 crc kubenswrapper[5052]: I1203 11:22:32.370400 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-kube-apiserver/installer-9-crc" Dec 03 11:22:32 crc kubenswrapper[5052]: I1203 11:22:32.382817 5052 status_manager.go:851] "Failed to get status for pod" podUID="8cf9b98e-2ea9-47ee-bd1b-a913714b7668" pod="openshift-marketplace/redhat-marketplace-qttdj" err="Get \"https://api-int.crc.testing:6443/api/v1/namespaces/openshift-marketplace/pods/redhat-marketplace-qttdj\": dial tcp 38.129.56.252:6443: connect: connection refused" Dec 03 11:22:32 crc kubenswrapper[5052]: I1203 11:22:32.383141 5052 status_manager.go:851] "Failed to get status for pod" podUID="f85e55b1a89d02b0cb034b1ea31ed45a" pod="openshift-kube-apiserver/kube-apiserver-startup-monitor-crc" err="Get \"https://api-int.crc.testing:6443/api/v1/namespaces/openshift-kube-apiserver/pods/kube-apiserver-startup-monitor-crc\": dial tcp 38.129.56.252:6443: connect: connection refused" Dec 03 11:22:32 crc kubenswrapper[5052]: I1203 11:22:32.383395 5052 status_manager.go:851] "Failed to get status for pod" podUID="afb87a28-1b7c-4b09-b186-f05da232f501" pod="openshift-marketplace/redhat-operators-dwt2v" err="Get \"https://api-int.crc.testing:6443/api/v1/namespaces/openshift-marketplace/pods/redhat-operators-dwt2v\": dial tcp 38.129.56.252:6443: connect: connection refused" Dec 03 11:22:32 crc kubenswrapper[5052]: I1203 11:22:32.383669 5052 status_manager.go:851] "Failed to get status for pod" podUID="1e97c072-bdcc-4095-9a87-ddf7d4f2d22b" pod="openshift-marketplace/redhat-operators-nq2qk" err="Get \"https://api-int.crc.testing:6443/api/v1/namespaces/openshift-marketplace/pods/redhat-operators-nq2qk\": dial tcp 38.129.56.252:6443: connect: connection refused" Dec 03 11:22:32 crc kubenswrapper[5052]: I1203 11:22:32.383991 5052 status_manager.go:851] "Failed to get status for pod" podUID="68a245d3-ddaf-40a0-92f6-92760173b7a3" pod="openshift-kube-apiserver/installer-9-crc" err="Get \"https://api-int.crc.testing:6443/api/v1/namespaces/openshift-kube-apiserver/pods/installer-9-crc\": dial tcp 38.129.56.252:6443: connect: connection refused" Dec 03 11:22:32 crc kubenswrapper[5052]: I1203 11:22:32.384247 5052 status_manager.go:851] "Failed to get status for pod" podUID="355008cc-8721-44c4-a1e0-3714b0086286" pod="openshift-marketplace/redhat-marketplace-gqhzd" err="Get \"https://api-int.crc.testing:6443/api/v1/namespaces/openshift-marketplace/pods/redhat-marketplace-gqhzd\": dial tcp 38.129.56.252:6443: connect: connection refused" Dec 03 11:22:32 crc kubenswrapper[5052]: I1203 11:22:32.384566 5052 status_manager.go:851] "Failed to get status for pod" podUID="75f62784-7818-4a74-9c8c-4777d6db0611" pod="openshift-authentication/oauth-openshift-558db77b4-99jsx" err="Get \"https://api-int.crc.testing:6443/api/v1/namespaces/openshift-authentication/pods/oauth-openshift-558db77b4-99jsx\": dial tcp 38.129.56.252:6443: connect: connection refused" Dec 03 11:22:33 crc kubenswrapper[5052]: I1203 11:22:33.385237 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-apiserver_kube-apiserver-crc_f4b27818a5e8e43d0dc095d08835c792/kube-apiserver-cert-syncer/0.log" Dec 03 11:22:34 crc kubenswrapper[5052]: I1203 11:22:34.304717 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-apiserver_kube-apiserver-crc_f4b27818a5e8e43d0dc095d08835c792/kube-apiserver-cert-syncer/0.log" Dec 03 11:22:34 crc kubenswrapper[5052]: I1203 11:22:34.306137 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-kube-apiserver/kube-apiserver-crc" Dec 03 11:22:34 crc kubenswrapper[5052]: I1203 11:22:34.307592 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"cert-dir\" (UniqueName: \"kubernetes.io/host-path/f4b27818a5e8e43d0dc095d08835c792-cert-dir\") pod \"f4b27818a5e8e43d0dc095d08835c792\" (UID: \"f4b27818a5e8e43d0dc095d08835c792\") " Dec 03 11:22:34 crc kubenswrapper[5052]: I1203 11:22:34.307839 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"audit-dir\" (UniqueName: \"kubernetes.io/host-path/f4b27818a5e8e43d0dc095d08835c792-audit-dir\") pod \"f4b27818a5e8e43d0dc095d08835c792\" (UID: \"f4b27818a5e8e43d0dc095d08835c792\") " Dec 03 11:22:34 crc kubenswrapper[5052]: I1203 11:22:34.307595 5052 status_manager.go:851] "Failed to get status for pod" podUID="f4b27818a5e8e43d0dc095d08835c792" pod="openshift-kube-apiserver/kube-apiserver-crc" err="Get \"https://api-int.crc.testing:6443/api/v1/namespaces/openshift-kube-apiserver/pods/kube-apiserver-crc\": dial tcp 38.129.56.252:6443: connect: connection refused" Dec 03 11:22:34 crc kubenswrapper[5052]: I1203 11:22:34.307930 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"resource-dir\" (UniqueName: \"kubernetes.io/host-path/f4b27818a5e8e43d0dc095d08835c792-resource-dir\") pod \"f4b27818a5e8e43d0dc095d08835c792\" (UID: \"f4b27818a5e8e43d0dc095d08835c792\") " Dec 03 11:22:34 crc kubenswrapper[5052]: I1203 11:22:34.307659 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/f4b27818a5e8e43d0dc095d08835c792-cert-dir" (OuterVolumeSpecName: "cert-dir") pod "f4b27818a5e8e43d0dc095d08835c792" (UID: "f4b27818a5e8e43d0dc095d08835c792"). InnerVolumeSpecName "cert-dir". PluginName "kubernetes.io/host-path", VolumeGidValue "" Dec 03 11:22:34 crc kubenswrapper[5052]: I1203 11:22:34.308112 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/f4b27818a5e8e43d0dc095d08835c792-resource-dir" (OuterVolumeSpecName: "resource-dir") pod "f4b27818a5e8e43d0dc095d08835c792" (UID: "f4b27818a5e8e43d0dc095d08835c792"). InnerVolumeSpecName "resource-dir". PluginName "kubernetes.io/host-path", VolumeGidValue "" Dec 03 11:22:34 crc kubenswrapper[5052]: I1203 11:22:34.308014 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/f4b27818a5e8e43d0dc095d08835c792-audit-dir" (OuterVolumeSpecName: "audit-dir") pod "f4b27818a5e8e43d0dc095d08835c792" (UID: "f4b27818a5e8e43d0dc095d08835c792"). InnerVolumeSpecName "audit-dir". PluginName "kubernetes.io/host-path", VolumeGidValue "" Dec 03 11:22:34 crc kubenswrapper[5052]: I1203 11:22:34.308154 5052 status_manager.go:851] "Failed to get status for pod" podUID="8cf9b98e-2ea9-47ee-bd1b-a913714b7668" pod="openshift-marketplace/redhat-marketplace-qttdj" err="Get \"https://api-int.crc.testing:6443/api/v1/namespaces/openshift-marketplace/pods/redhat-marketplace-qttdj\": dial tcp 38.129.56.252:6443: connect: connection refused" Dec 03 11:22:34 crc kubenswrapper[5052]: I1203 11:22:34.308360 5052 reconciler_common.go:293] "Volume detached for volume \"cert-dir\" (UniqueName: \"kubernetes.io/host-path/f4b27818a5e8e43d0dc095d08835c792-cert-dir\") on node \"crc\" DevicePath \"\"" Dec 03 11:22:34 crc kubenswrapper[5052]: I1203 11:22:34.308392 5052 reconciler_common.go:293] "Volume detached for volume \"audit-dir\" (UniqueName: \"kubernetes.io/host-path/f4b27818a5e8e43d0dc095d08835c792-audit-dir\") on node \"crc\" DevicePath \"\"" Dec 03 11:22:34 crc kubenswrapper[5052]: I1203 11:22:34.308406 5052 reconciler_common.go:293] "Volume detached for volume \"resource-dir\" (UniqueName: \"kubernetes.io/host-path/f4b27818a5e8e43d0dc095d08835c792-resource-dir\") on node \"crc\" DevicePath \"\"" Dec 03 11:22:34 crc kubenswrapper[5052]: I1203 11:22:34.308581 5052 status_manager.go:851] "Failed to get status for pod" podUID="f85e55b1a89d02b0cb034b1ea31ed45a" pod="openshift-kube-apiserver/kube-apiserver-startup-monitor-crc" err="Get \"https://api-int.crc.testing:6443/api/v1/namespaces/openshift-kube-apiserver/pods/kube-apiserver-startup-monitor-crc\": dial tcp 38.129.56.252:6443: connect: connection refused" Dec 03 11:22:34 crc kubenswrapper[5052]: I1203 11:22:34.309118 5052 status_manager.go:851] "Failed to get status for pod" podUID="afb87a28-1b7c-4b09-b186-f05da232f501" pod="openshift-marketplace/redhat-operators-dwt2v" err="Get \"https://api-int.crc.testing:6443/api/v1/namespaces/openshift-marketplace/pods/redhat-operators-dwt2v\": dial tcp 38.129.56.252:6443: connect: connection refused" Dec 03 11:22:34 crc kubenswrapper[5052]: I1203 11:22:34.309640 5052 status_manager.go:851] "Failed to get status for pod" podUID="1e97c072-bdcc-4095-9a87-ddf7d4f2d22b" pod="openshift-marketplace/redhat-operators-nq2qk" err="Get \"https://api-int.crc.testing:6443/api/v1/namespaces/openshift-marketplace/pods/redhat-operators-nq2qk\": dial tcp 38.129.56.252:6443: connect: connection refused" Dec 03 11:22:34 crc kubenswrapper[5052]: I1203 11:22:34.310241 5052 status_manager.go:851] "Failed to get status for pod" podUID="68a245d3-ddaf-40a0-92f6-92760173b7a3" pod="openshift-kube-apiserver/installer-9-crc" err="Get \"https://api-int.crc.testing:6443/api/v1/namespaces/openshift-kube-apiserver/pods/installer-9-crc\": dial tcp 38.129.56.252:6443: connect: connection refused" Dec 03 11:22:34 crc kubenswrapper[5052]: I1203 11:22:34.310628 5052 status_manager.go:851] "Failed to get status for pod" podUID="355008cc-8721-44c4-a1e0-3714b0086286" pod="openshift-marketplace/redhat-marketplace-gqhzd" err="Get \"https://api-int.crc.testing:6443/api/v1/namespaces/openshift-marketplace/pods/redhat-marketplace-gqhzd\": dial tcp 38.129.56.252:6443: connect: connection refused" Dec 03 11:22:34 crc kubenswrapper[5052]: I1203 11:22:34.311394 5052 status_manager.go:851] "Failed to get status for pod" podUID="75f62784-7818-4a74-9c8c-4777d6db0611" pod="openshift-authentication/oauth-openshift-558db77b4-99jsx" err="Get \"https://api-int.crc.testing:6443/api/v1/namespaces/openshift-authentication/pods/oauth-openshift-558db77b4-99jsx\": dial tcp 38.129.56.252:6443: connect: connection refused" Dec 03 11:22:34 crc kubenswrapper[5052]: I1203 11:22:34.394768 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-apiserver_kube-apiserver-crc_f4b27818a5e8e43d0dc095d08835c792/kube-apiserver-cert-syncer/0.log" Dec 03 11:22:34 crc kubenswrapper[5052]: I1203 11:22:34.395664 5052 generic.go:334] "Generic (PLEG): container finished" podID="f4b27818a5e8e43d0dc095d08835c792" containerID="09256543a9aa86da8bec948be040d36b6e4986b9232091d2638bb1fead5740a9" exitCode=0 Dec 03 11:22:34 crc kubenswrapper[5052]: I1203 11:22:34.395772 5052 scope.go:117] "RemoveContainer" containerID="ee9c5c61cfd6dee9fd44ebc4c7ef4d20b5fde82b12a8147c3c871e2615da7940" Dec 03 11:22:34 crc kubenswrapper[5052]: I1203 11:22:34.395813 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-kube-apiserver/kube-apiserver-crc" Dec 03 11:22:34 crc kubenswrapper[5052]: I1203 11:22:34.411425 5052 scope.go:117] "RemoveContainer" containerID="22b8cb6aa04337d1420f97d3e128fce2b29a725ca6e5d157adeb1ba78f00073f" Dec 03 11:22:34 crc kubenswrapper[5052]: I1203 11:22:34.414399 5052 status_manager.go:851] "Failed to get status for pod" podUID="f4b27818a5e8e43d0dc095d08835c792" pod="openshift-kube-apiserver/kube-apiserver-crc" err="Get \"https://api-int.crc.testing:6443/api/v1/namespaces/openshift-kube-apiserver/pods/kube-apiserver-crc\": dial tcp 38.129.56.252:6443: connect: connection refused" Dec 03 11:22:34 crc kubenswrapper[5052]: I1203 11:22:34.415084 5052 status_manager.go:851] "Failed to get status for pod" podUID="8cf9b98e-2ea9-47ee-bd1b-a913714b7668" pod="openshift-marketplace/redhat-marketplace-qttdj" err="Get \"https://api-int.crc.testing:6443/api/v1/namespaces/openshift-marketplace/pods/redhat-marketplace-qttdj\": dial tcp 38.129.56.252:6443: connect: connection refused" Dec 03 11:22:34 crc kubenswrapper[5052]: I1203 11:22:34.415470 5052 status_manager.go:851] "Failed to get status for pod" podUID="f85e55b1a89d02b0cb034b1ea31ed45a" pod="openshift-kube-apiserver/kube-apiserver-startup-monitor-crc" err="Get \"https://api-int.crc.testing:6443/api/v1/namespaces/openshift-kube-apiserver/pods/kube-apiserver-startup-monitor-crc\": dial tcp 38.129.56.252:6443: connect: connection refused" Dec 03 11:22:34 crc kubenswrapper[5052]: I1203 11:22:34.415786 5052 status_manager.go:851] "Failed to get status for pod" podUID="afb87a28-1b7c-4b09-b186-f05da232f501" pod="openshift-marketplace/redhat-operators-dwt2v" err="Get \"https://api-int.crc.testing:6443/api/v1/namespaces/openshift-marketplace/pods/redhat-operators-dwt2v\": dial tcp 38.129.56.252:6443: connect: connection refused" Dec 03 11:22:34 crc kubenswrapper[5052]: I1203 11:22:34.416036 5052 status_manager.go:851] "Failed to get status for pod" podUID="1e97c072-bdcc-4095-9a87-ddf7d4f2d22b" pod="openshift-marketplace/redhat-operators-nq2qk" err="Get \"https://api-int.crc.testing:6443/api/v1/namespaces/openshift-marketplace/pods/redhat-operators-nq2qk\": dial tcp 38.129.56.252:6443: connect: connection refused" Dec 03 11:22:34 crc kubenswrapper[5052]: I1203 11:22:34.416311 5052 status_manager.go:851] "Failed to get status for pod" podUID="68a245d3-ddaf-40a0-92f6-92760173b7a3" pod="openshift-kube-apiserver/installer-9-crc" err="Get \"https://api-int.crc.testing:6443/api/v1/namespaces/openshift-kube-apiserver/pods/installer-9-crc\": dial tcp 38.129.56.252:6443: connect: connection refused" Dec 03 11:22:34 crc kubenswrapper[5052]: I1203 11:22:34.416618 5052 status_manager.go:851] "Failed to get status for pod" podUID="355008cc-8721-44c4-a1e0-3714b0086286" pod="openshift-marketplace/redhat-marketplace-gqhzd" err="Get \"https://api-int.crc.testing:6443/api/v1/namespaces/openshift-marketplace/pods/redhat-marketplace-gqhzd\": dial tcp 38.129.56.252:6443: connect: connection refused" Dec 03 11:22:34 crc kubenswrapper[5052]: I1203 11:22:34.416868 5052 status_manager.go:851] "Failed to get status for pod" podUID="75f62784-7818-4a74-9c8c-4777d6db0611" pod="openshift-authentication/oauth-openshift-558db77b4-99jsx" err="Get \"https://api-int.crc.testing:6443/api/v1/namespaces/openshift-authentication/pods/oauth-openshift-558db77b4-99jsx\": dial tcp 38.129.56.252:6443: connect: connection refused" Dec 03 11:22:34 crc kubenswrapper[5052]: I1203 11:22:34.428954 5052 scope.go:117] "RemoveContainer" containerID="15abb0d7fd5884519ed5ac454430dbb58f71ed1f459e5ed30f324f70befd243e" Dec 03 11:22:34 crc kubenswrapper[5052]: I1203 11:22:34.445764 5052 scope.go:117] "RemoveContainer" containerID="8ceedd21ae3833d77fd440537f30434ee57f087fbefb2357b46231d3f778ad2a" Dec 03 11:22:34 crc kubenswrapper[5052]: I1203 11:22:34.463951 5052 scope.go:117] "RemoveContainer" containerID="09256543a9aa86da8bec948be040d36b6e4986b9232091d2638bb1fead5740a9" Dec 03 11:22:34 crc kubenswrapper[5052]: I1203 11:22:34.480588 5052 scope.go:117] "RemoveContainer" containerID="7182c75773f069c72f87c39136222fd4a91b49add27352a1969645413e670f01" Dec 03 11:22:34 crc kubenswrapper[5052]: I1203 11:22:34.499666 5052 scope.go:117] "RemoveContainer" containerID="ee9c5c61cfd6dee9fd44ebc4c7ef4d20b5fde82b12a8147c3c871e2615da7940" Dec 03 11:22:34 crc kubenswrapper[5052]: E1203 11:22:34.500263 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ee9c5c61cfd6dee9fd44ebc4c7ef4d20b5fde82b12a8147c3c871e2615da7940\": container with ID starting with ee9c5c61cfd6dee9fd44ebc4c7ef4d20b5fde82b12a8147c3c871e2615da7940 not found: ID does not exist" containerID="ee9c5c61cfd6dee9fd44ebc4c7ef4d20b5fde82b12a8147c3c871e2615da7940" Dec 03 11:22:34 crc kubenswrapper[5052]: I1203 11:22:34.500314 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ee9c5c61cfd6dee9fd44ebc4c7ef4d20b5fde82b12a8147c3c871e2615da7940"} err="failed to get container status \"ee9c5c61cfd6dee9fd44ebc4c7ef4d20b5fde82b12a8147c3c871e2615da7940\": rpc error: code = NotFound desc = could not find container \"ee9c5c61cfd6dee9fd44ebc4c7ef4d20b5fde82b12a8147c3c871e2615da7940\": container with ID starting with ee9c5c61cfd6dee9fd44ebc4c7ef4d20b5fde82b12a8147c3c871e2615da7940 not found: ID does not exist" Dec 03 11:22:34 crc kubenswrapper[5052]: I1203 11:22:34.500379 5052 scope.go:117] "RemoveContainer" containerID="22b8cb6aa04337d1420f97d3e128fce2b29a725ca6e5d157adeb1ba78f00073f" Dec 03 11:22:34 crc kubenswrapper[5052]: E1203 11:22:34.501045 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"22b8cb6aa04337d1420f97d3e128fce2b29a725ca6e5d157adeb1ba78f00073f\": container with ID starting with 22b8cb6aa04337d1420f97d3e128fce2b29a725ca6e5d157adeb1ba78f00073f not found: ID does not exist" containerID="22b8cb6aa04337d1420f97d3e128fce2b29a725ca6e5d157adeb1ba78f00073f" Dec 03 11:22:34 crc kubenswrapper[5052]: I1203 11:22:34.501084 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"22b8cb6aa04337d1420f97d3e128fce2b29a725ca6e5d157adeb1ba78f00073f"} err="failed to get container status \"22b8cb6aa04337d1420f97d3e128fce2b29a725ca6e5d157adeb1ba78f00073f\": rpc error: code = NotFound desc = could not find container \"22b8cb6aa04337d1420f97d3e128fce2b29a725ca6e5d157adeb1ba78f00073f\": container with ID starting with 22b8cb6aa04337d1420f97d3e128fce2b29a725ca6e5d157adeb1ba78f00073f not found: ID does not exist" Dec 03 11:22:34 crc kubenswrapper[5052]: I1203 11:22:34.501116 5052 scope.go:117] "RemoveContainer" containerID="15abb0d7fd5884519ed5ac454430dbb58f71ed1f459e5ed30f324f70befd243e" Dec 03 11:22:34 crc kubenswrapper[5052]: E1203 11:22:34.501412 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"15abb0d7fd5884519ed5ac454430dbb58f71ed1f459e5ed30f324f70befd243e\": container with ID starting with 15abb0d7fd5884519ed5ac454430dbb58f71ed1f459e5ed30f324f70befd243e not found: ID does not exist" containerID="15abb0d7fd5884519ed5ac454430dbb58f71ed1f459e5ed30f324f70befd243e" Dec 03 11:22:34 crc kubenswrapper[5052]: I1203 11:22:34.501448 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"15abb0d7fd5884519ed5ac454430dbb58f71ed1f459e5ed30f324f70befd243e"} err="failed to get container status \"15abb0d7fd5884519ed5ac454430dbb58f71ed1f459e5ed30f324f70befd243e\": rpc error: code = NotFound desc = could not find container \"15abb0d7fd5884519ed5ac454430dbb58f71ed1f459e5ed30f324f70befd243e\": container with ID starting with 15abb0d7fd5884519ed5ac454430dbb58f71ed1f459e5ed30f324f70befd243e not found: ID does not exist" Dec 03 11:22:34 crc kubenswrapper[5052]: I1203 11:22:34.501465 5052 scope.go:117] "RemoveContainer" containerID="8ceedd21ae3833d77fd440537f30434ee57f087fbefb2357b46231d3f778ad2a" Dec 03 11:22:34 crc kubenswrapper[5052]: E1203 11:22:34.501756 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8ceedd21ae3833d77fd440537f30434ee57f087fbefb2357b46231d3f778ad2a\": container with ID starting with 8ceedd21ae3833d77fd440537f30434ee57f087fbefb2357b46231d3f778ad2a not found: ID does not exist" containerID="8ceedd21ae3833d77fd440537f30434ee57f087fbefb2357b46231d3f778ad2a" Dec 03 11:22:34 crc kubenswrapper[5052]: I1203 11:22:34.501789 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8ceedd21ae3833d77fd440537f30434ee57f087fbefb2357b46231d3f778ad2a"} err="failed to get container status \"8ceedd21ae3833d77fd440537f30434ee57f087fbefb2357b46231d3f778ad2a\": rpc error: code = NotFound desc = could not find container \"8ceedd21ae3833d77fd440537f30434ee57f087fbefb2357b46231d3f778ad2a\": container with ID starting with 8ceedd21ae3833d77fd440537f30434ee57f087fbefb2357b46231d3f778ad2a not found: ID does not exist" Dec 03 11:22:34 crc kubenswrapper[5052]: I1203 11:22:34.501812 5052 scope.go:117] "RemoveContainer" containerID="09256543a9aa86da8bec948be040d36b6e4986b9232091d2638bb1fead5740a9" Dec 03 11:22:34 crc kubenswrapper[5052]: E1203 11:22:34.502118 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"09256543a9aa86da8bec948be040d36b6e4986b9232091d2638bb1fead5740a9\": container with ID starting with 09256543a9aa86da8bec948be040d36b6e4986b9232091d2638bb1fead5740a9 not found: ID does not exist" containerID="09256543a9aa86da8bec948be040d36b6e4986b9232091d2638bb1fead5740a9" Dec 03 11:22:34 crc kubenswrapper[5052]: I1203 11:22:34.502149 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"09256543a9aa86da8bec948be040d36b6e4986b9232091d2638bb1fead5740a9"} err="failed to get container status \"09256543a9aa86da8bec948be040d36b6e4986b9232091d2638bb1fead5740a9\": rpc error: code = NotFound desc = could not find container \"09256543a9aa86da8bec948be040d36b6e4986b9232091d2638bb1fead5740a9\": container with ID starting with 09256543a9aa86da8bec948be040d36b6e4986b9232091d2638bb1fead5740a9 not found: ID does not exist" Dec 03 11:22:34 crc kubenswrapper[5052]: I1203 11:22:34.502168 5052 scope.go:117] "RemoveContainer" containerID="7182c75773f069c72f87c39136222fd4a91b49add27352a1969645413e670f01" Dec 03 11:22:34 crc kubenswrapper[5052]: E1203 11:22:34.502447 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7182c75773f069c72f87c39136222fd4a91b49add27352a1969645413e670f01\": container with ID starting with 7182c75773f069c72f87c39136222fd4a91b49add27352a1969645413e670f01 not found: ID does not exist" containerID="7182c75773f069c72f87c39136222fd4a91b49add27352a1969645413e670f01" Dec 03 11:22:34 crc kubenswrapper[5052]: I1203 11:22:34.502474 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7182c75773f069c72f87c39136222fd4a91b49add27352a1969645413e670f01"} err="failed to get container status \"7182c75773f069c72f87c39136222fd4a91b49add27352a1969645413e670f01\": rpc error: code = NotFound desc = could not find container \"7182c75773f069c72f87c39136222fd4a91b49add27352a1969645413e670f01\": container with ID starting with 7182c75773f069c72f87c39136222fd4a91b49add27352a1969645413e670f01 not found: ID does not exist" Dec 03 11:22:35 crc kubenswrapper[5052]: E1203 11:22:35.302804 5052 controller.go:195] "Failed to update lease" err="Put \"https://api-int.crc.testing:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/crc?timeout=10s\": dial tcp 38.129.56.252:6443: connect: connection refused" Dec 03 11:22:35 crc kubenswrapper[5052]: E1203 11:22:35.303270 5052 controller.go:195] "Failed to update lease" err="Put \"https://api-int.crc.testing:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/crc?timeout=10s\": dial tcp 38.129.56.252:6443: connect: connection refused" Dec 03 11:22:35 crc kubenswrapper[5052]: E1203 11:22:35.303863 5052 controller.go:195] "Failed to update lease" err="Put \"https://api-int.crc.testing:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/crc?timeout=10s\": dial tcp 38.129.56.252:6443: connect: connection refused" Dec 03 11:22:35 crc kubenswrapper[5052]: E1203 11:22:35.304166 5052 controller.go:195] "Failed to update lease" err="Put \"https://api-int.crc.testing:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/crc?timeout=10s\": dial tcp 38.129.56.252:6443: connect: connection refused" Dec 03 11:22:35 crc kubenswrapper[5052]: E1203 11:22:35.304464 5052 controller.go:195] "Failed to update lease" err="Put \"https://api-int.crc.testing:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/crc?timeout=10s\": dial tcp 38.129.56.252:6443: connect: connection refused" Dec 03 11:22:35 crc kubenswrapper[5052]: I1203 11:22:35.304518 5052 controller.go:115] "failed to update lease using latest lease, fallback to ensure lease" err="failed 5 attempts to update lease" Dec 03 11:22:35 crc kubenswrapper[5052]: E1203 11:22:35.304776 5052 controller.go:145] "Failed to ensure lease exists, will retry" err="Get \"https://api-int.crc.testing:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/crc?timeout=10s\": dial tcp 38.129.56.252:6443: connect: connection refused" interval="200ms" Dec 03 11:22:35 crc kubenswrapper[5052]: E1203 11:22:35.506686 5052 controller.go:145] "Failed to ensure lease exists, will retry" err="Get \"https://api-int.crc.testing:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/crc?timeout=10s\": dial tcp 38.129.56.252:6443: connect: connection refused" interval="400ms" Dec 03 11:22:35 crc kubenswrapper[5052]: I1203 11:22:35.681492 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f4b27818a5e8e43d0dc095d08835c792" path="/var/lib/kubelet/pods/f4b27818a5e8e43d0dc095d08835c792/volumes" Dec 03 11:22:35 crc kubenswrapper[5052]: E1203 11:22:35.908330 5052 controller.go:145] "Failed to ensure lease exists, will retry" err="Get \"https://api-int.crc.testing:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/crc?timeout=10s\": dial tcp 38.129.56.252:6443: connect: connection refused" interval="800ms" Dec 03 11:22:36 crc kubenswrapper[5052]: I1203 11:22:36.029351 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-zbr2b\" (UniqueName: \"kubernetes.io/projected/c712fba2-c530-4f64-836d-16706a2e2b50-kube-api-access-zbr2b\") pod \"oauth-openshift-6f85f596d6-nhvd7\" (UID: \"c712fba2-c530-4f64-836d-16706a2e2b50\") " pod="openshift-authentication/oauth-openshift-6f85f596d6-nhvd7" Dec 03 11:22:36 crc kubenswrapper[5052]: E1203 11:22:36.030326 5052 projected.go:194] Error preparing data for projected volume kube-api-access-zbr2b for pod openshift-authentication/oauth-openshift-6f85f596d6-nhvd7: failed to fetch token: Post "https://api-int.crc.testing:6443/api/v1/namespaces/openshift-authentication/serviceaccounts/oauth-openshift/token": dial tcp 38.129.56.252:6443: connect: connection refused Dec 03 11:22:36 crc kubenswrapper[5052]: E1203 11:22:36.030457 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c712fba2-c530-4f64-836d-16706a2e2b50-kube-api-access-zbr2b podName:c712fba2-c530-4f64-836d-16706a2e2b50 nodeName:}" failed. No retries permitted until 2025-12-03 11:22:44.030425237 +0000 UTC m=+154.618667001 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-zbr2b" (UniqueName: "kubernetes.io/projected/c712fba2-c530-4f64-836d-16706a2e2b50-kube-api-access-zbr2b") pod "oauth-openshift-6f85f596d6-nhvd7" (UID: "c712fba2-c530-4f64-836d-16706a2e2b50") : failed to fetch token: Post "https://api-int.crc.testing:6443/api/v1/namespaces/openshift-authentication/serviceaccounts/oauth-openshift/token": dial tcp 38.129.56.252:6443: connect: connection refused Dec 03 11:22:36 crc kubenswrapper[5052]: E1203 11:22:36.709655 5052 controller.go:145] "Failed to ensure lease exists, will retry" err="Get \"https://api-int.crc.testing:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/crc?timeout=10s\": dial tcp 38.129.56.252:6443: connect: connection refused" interval="1.6s" Dec 03 11:22:38 crc kubenswrapper[5052]: E1203 11:22:38.311062 5052 controller.go:145] "Failed to ensure lease exists, will retry" err="Get \"https://api-int.crc.testing:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/crc?timeout=10s\": dial tcp 38.129.56.252:6443: connect: connection refused" interval="3.2s" Dec 03 11:22:39 crc kubenswrapper[5052]: I1203 11:22:39.678119 5052 status_manager.go:851] "Failed to get status for pod" podUID="8cf9b98e-2ea9-47ee-bd1b-a913714b7668" pod="openshift-marketplace/redhat-marketplace-qttdj" err="Get \"https://api-int.crc.testing:6443/api/v1/namespaces/openshift-marketplace/pods/redhat-marketplace-qttdj\": dial tcp 38.129.56.252:6443: connect: connection refused" Dec 03 11:22:39 crc kubenswrapper[5052]: I1203 11:22:39.679501 5052 status_manager.go:851] "Failed to get status for pod" podUID="f85e55b1a89d02b0cb034b1ea31ed45a" pod="openshift-kube-apiserver/kube-apiserver-startup-monitor-crc" err="Get \"https://api-int.crc.testing:6443/api/v1/namespaces/openshift-kube-apiserver/pods/kube-apiserver-startup-monitor-crc\": dial tcp 38.129.56.252:6443: connect: connection refused" Dec 03 11:22:39 crc kubenswrapper[5052]: I1203 11:22:39.680290 5052 status_manager.go:851] "Failed to get status for pod" podUID="afb87a28-1b7c-4b09-b186-f05da232f501" pod="openshift-marketplace/redhat-operators-dwt2v" err="Get \"https://api-int.crc.testing:6443/api/v1/namespaces/openshift-marketplace/pods/redhat-operators-dwt2v\": dial tcp 38.129.56.252:6443: connect: connection refused" Dec 03 11:22:39 crc kubenswrapper[5052]: I1203 11:22:39.686651 5052 status_manager.go:851] "Failed to get status for pod" podUID="1e97c072-bdcc-4095-9a87-ddf7d4f2d22b" pod="openshift-marketplace/redhat-operators-nq2qk" err="Get \"https://api-int.crc.testing:6443/api/v1/namespaces/openshift-marketplace/pods/redhat-operators-nq2qk\": dial tcp 38.129.56.252:6443: connect: connection refused" Dec 03 11:22:39 crc kubenswrapper[5052]: I1203 11:22:39.687101 5052 status_manager.go:851] "Failed to get status for pod" podUID="68a245d3-ddaf-40a0-92f6-92760173b7a3" pod="openshift-kube-apiserver/installer-9-crc" err="Get \"https://api-int.crc.testing:6443/api/v1/namespaces/openshift-kube-apiserver/pods/installer-9-crc\": dial tcp 38.129.56.252:6443: connect: connection refused" Dec 03 11:22:39 crc kubenswrapper[5052]: I1203 11:22:39.687385 5052 status_manager.go:851] "Failed to get status for pod" podUID="355008cc-8721-44c4-a1e0-3714b0086286" pod="openshift-marketplace/redhat-marketplace-gqhzd" err="Get \"https://api-int.crc.testing:6443/api/v1/namespaces/openshift-marketplace/pods/redhat-marketplace-gqhzd\": dial tcp 38.129.56.252:6443: connect: connection refused" Dec 03 11:22:39 crc kubenswrapper[5052]: I1203 11:22:39.687666 5052 status_manager.go:851] "Failed to get status for pod" podUID="75f62784-7818-4a74-9c8c-4777d6db0611" pod="openshift-authentication/oauth-openshift-558db77b4-99jsx" err="Get \"https://api-int.crc.testing:6443/api/v1/namespaces/openshift-authentication/pods/oauth-openshift-558db77b4-99jsx\": dial tcp 38.129.56.252:6443: connect: connection refused" Dec 03 11:22:40 crc kubenswrapper[5052]: E1203 11:22:40.021852 5052 kubelet_node_status.go:585] "Error updating node status, will retry" err="failed to patch status \"{\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"type\\\":\\\"DiskPressure\\\"},{\\\"type\\\":\\\"PIDPressure\\\"},{\\\"type\\\":\\\"Ready\\\"}],\\\"conditions\\\":[{\\\"lastHeartbeatTime\\\":\\\"2025-12-03T11:22:40Z\\\",\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2025-12-03T11:22:40Z\\\",\\\"type\\\":\\\"DiskPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2025-12-03T11:22:40Z\\\",\\\"type\\\":\\\"PIDPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2025-12-03T11:22:40Z\\\",\\\"type\\\":\\\"Ready\\\"}],\\\"images\\\":[{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b9ea248f8ca33258fe1683da51d2b16b94630be1b361c65f68a16c1a34b94887\\\"],\\\"sizeBytes\\\":2887430265},{\\\"names\\\":[\\\"registry.redhat.io/redhat/redhat-operator-index@sha256:416a7dc57b2b95775e679e0ab93111baaa063e55a4c6d73856a248d85a2debbd\\\",\\\"registry.redhat.io/redhat/redhat-operator-index@sha256:5cdd97eed164a2eda9842fb91d284f06d2e63d69af9a98001fca2d6cebd0b52a\\\",\\\"registry.redhat.io/redhat/redhat-operator-index:v4.18\\\"],\\\"sizeBytes\\\":1609873225},{\\\"names\\\":[\\\"registry.redhat.io/redhat/redhat-operator-index@sha256:4a62fa1c0091f6d94e8fb7258470b9a532d78364b6b51a05341592041d598562\\\"],\\\"sizeBytes\\\":1523204510},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:174f36cdd47ef0d1d2099482919d773257453265a2af0b17b154edc32fa41ac2\\\"],\\\"sizeBytes\\\":1498102846},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7eeaee65f2808b819eedb413bdcabb9144e12f0dd97f13fd1afba93a95b67b26\\\"],\\\"sizeBytes\\\":1232839934},{\\\"names\\\":[\\\"registry.redhat.io/redhat/certified-operator-index@sha256:2041f196d215ed63fc3218d5a0e0cb8409dd5b575612d999cffc9817ca7ee2c4\\\",\\\"registry.redhat.io/redhat/certified-operator-index@sha256:9f4887ff6cc4a4c64b413cf3ffbbceababd44911f3d138b5dc10dcb622ccd23c\\\",\\\"registry.redhat.io/redhat/certified-operator-index:v4.18\\\"],\\\"sizeBytes\\\":1204906320},{\\\"names\\\":[\\\"registry.redhat.io/redhat/community-operator-index@sha256:dba4a8e0293f7b6e1459b74484a7126274bdc9efa75f808eb15e5a3896a3c818\\\",\\\"registry.redhat.io/redhat/community-operator-index@sha256:eee64597300e249e4bead6abda4d235cc4fc3a87be82e3e9f582609602ed87d7\\\",\\\"registry.redhat.io/redhat/community-operator-index:v4.18\\\"],\\\"sizeBytes\\\":1201319250},{\\\"names\\\":[\\\"registry.redhat.io/redhat/community-operator-index@sha256:8ff55cdb2367f5011074d2f5ebdc153b8885e7495e14ae00f99d2b7ab3584ade\\\"],\\\"sizeBytes\\\":1151049424},{\\\"names\\\":[\\\"registry.redhat.io/redhat/redhat-marketplace-index@sha256:e8990432556acad31519b1a73ec32f32d27c2034cf9e5cc4db8980efc7331594\\\",\\\"registry.redhat.io/redhat/redhat-marketplace-index@sha256:ebe9f523f5c211a3a0f2570331dddcd5be15b12c1fecd9b8b121f881bfaad029\\\",\\\"registry.redhat.io/redhat/redhat-marketplace-index:v4.18\\\"],\\\"sizeBytes\\\":1129027903},{\\\"names\\\":[\\\"registry.redhat.io/redhat/certified-operator-index@sha256:7688bce5eb0d153adff87fc9f7a47642465c0b88208efb236880197969931b37\\\"],\\\"sizeBytes\\\":1032059094},{\\\"names\\\":[\\\"registry.redhat.io/redhat/redhat-marketplace-index@sha256:1dc15c170ebf462dacaef75511740ed94ca1da210f3980f66d77f91ba201c875\\\"],\\\"sizeBytes\\\":1001152198},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c915fb8ba96e911699a1ae34a8e95ca8a9fbe1bf8c28fea177225c63a8bdfc0a\\\"],\\\"sizeBytes\\\":964552795},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:06bc35825771aee1220d34720243b89c4ba8a8b335e6de2597126bd791fd90d4\\\"],\\\"sizeBytes\\\":947616130},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c3cc3840d7a81ce1b420f06e07a923861faf37d9c10688aa3aa0b7b76c8706ad\\\"],\\\"sizeBytes\\\":907837715},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:101f295e2eae0755ae1865f7de885db1f17b9368e4120a713bb5f79e17ce8f93\\\"],\\\"sizeBytes\\\":854694423},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:47b0670fa1051335fd2d2c9e8361e4ed77c7760c33a2180b136f7c7f59863ec2\\\"],\\\"sizeBytes\\\":852490370},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:862f4a4bed52f372056b6d368e2498ebfb063075b31cf48dbdaaeedfcf0396cb\\\"],\\\"sizeBytes\\\":772592048},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:98100674616e54319f6713d742fd0c3bdbc84e6e6173e8ccf4a2473a714c2bc4\\\"],\\\"sizeBytes\\\":705793115},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:687fddfbb085a1688df312ce4ec8c857df9b2daed8ff4a7ed6163a1154afa2cc\\\"],\\\"sizeBytes\\\":687915987},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f247257b0885cf5d303e3612c7714b33ae51404cfa2429822060c6c025eb17dd\\\"],\\\"sizeBytes\\\":668060419},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e1baa38811c04bd8909e01a1f3be7421a1cb99d608d3dc4cf86d95b17de2ab8b\\\"],\\\"sizeBytes\\\":613826183},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7e3e9dc0b02b9351edf7c46b1d46d724abd1ac38ecbd6bc541cee84a209258d8\\\"],\\\"sizeBytes\\\":581863411},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:35512335ac39aed0f55b7f799f416f4f6445c20c1b19888cf2bb72bb276703f2\\\"],\\\"sizeBytes\\\":574606365},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ee8d8f089ec1488067444c7e276c4e47cc93840280f3b3295484d67af2232002\\\"],\\\"sizeBytes\\\":550676059},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:10f20a39f16ae3019c62261eda8beb9e4d8c36cbb7b500b3bae1312987f0685d\\\"],\\\"sizeBytes\\\":541458174},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e40792096b162f0f9ce5f8362f51e5f8dea2c1ce4b1447235388416b5db7708c\\\"],\\\"sizeBytes\\\":533092226},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:07b7c6877441ecd6a5646fb68e33e9be8b90092272e49117b54b4a67314731ca\\\"],\\\"sizeBytes\\\":528023732},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a0fa3723269019bee1847b26702f42928e779036cc2f58408f8ee7866be30a93\\\"],\\\"sizeBytes\\\":510867594},{\\\"names\\\":[\\\"quay.io/crcont/ocp-release@sha256:0b6ae0d091d2bf49f9b3a3aff54aabdc49e70c783780f118789f49d8f95a9e03\\\"],\\\"sizeBytes\\\":510526836},{\\\"names\\\":[\\\"quay.io/crcont/openshift-crc-cluster-kube-apiserver-operator@sha256:9f36dc276e27753fc478274c7f7814a4f8945c987117ee1ea3b8e6355e6d7462\\\"],\\\"sizeBytes\\\":507459597},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7e9e7dd2b1a8394b7490ca6df8a3ee8cdfc6193ecc6fb6173ed9a1868116a207\\\"],\\\"sizeBytes\\\":505721947},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:094bb6a6641b4edbaf932f0551bcda20b0d4e012cbe84207348b24eeabd351e9\\\"],\\\"sizeBytes\\\":504778226},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c69fe7a98a744b7a7b61b2a8db81a338f373cd2b1d46c6d3f02864b30c37e46c\\\"],\\\"sizeBytes\\\":504735878},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e51e6f78ec20ef91c82e94a49f950e427e77894e582dcc406eec4df807ddd76e\\\"],\\\"sizeBytes\\\":502943148},{\\\"names\\\":[\\\"quay.io/crcont/openshift-crc-cluster-kube-controller-manager-operator@sha256:8506ce0a578bc18fac117eb2b82799488ffac0bed08287faaf92edaf5d17ab95\\\"],\\\"sizeBytes\\\":501379880},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3a741253807c962189819d879b8fef94a9452fb3f5f3969ec3207eb2d9862205\\\"],\\\"sizeBytes\\\":500472212},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:5b881c97aa8e440c6b3ca001edfd789a9380066b8f11f35a8dd8d88c5c7dbf86\\\"],\\\"sizeBytes\\\":498888951},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:5aa9e5379bfeb63f4e517fb45168eb6820138041641bbdfc6f4db6427032fa37\\\"],\\\"sizeBytes\\\":497832828},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c0f9da410c07372b6c9ad6a79379b491cd10fdee88051c026b084652d85aed21\\\"],\\\"sizeBytes\\\":497742284},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:88b1f0a05a1b1c91e1212b40f0e7d04c9351ec9d34c52097bfdc5897b46f2f0e\\\"],\\\"sizeBytes\\\":497120598},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:737e9019a072c74321e0a909ca95481f5c545044dd4f151a34d0e1c8b9cf273f\\\"],\\\"sizeBytes\\\":488494681},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:fe009d03910e18795e3bd60a3fd84938311d464d2730a2af5ded5b24e4d05a6b\\\"],\\\"sizeBytes\\\":487097366},{\\\"names\\\":[\\\"registry.redhat.io/openshift4/ose-csi-external-provisioner@sha256:66760a53b64d381940757ca9f0d05f523a61f943f8da03ce9791e5d05264a736\\\",\\\"registry.redhat.io/openshift4/ose-csi-external-provisioner@sha256:e97a0cb5b6119a9735efe0ac24630a8912fcad89a1dddfa76dc10edac4ec9815\\\",\\\"registry.redhat.io/openshift4/ose-csi-external-provisioner:latest\\\"],\\\"sizeBytes\\\":485998616},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:9fa29d188c85a8b1e1bd15c9c18e96f1b235da9bd4a45dbc086a4a69520ed63f\\\"],\\\"sizeBytes\\\":485767738},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:898cae57123c5006d397b24af21b0f24a0c42c9b0be5ee8251e1824711f65820\\\"],\\\"sizeBytes\\\":485535312},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:1eda5ad6a6c5b9cd94b4b456e9116f4a0517241b614de1a99df14baee20c3e6a\\\"],\\\"sizeBytes\\\":479585218},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:487c0a8d5200bcdce484ab1169229d8fcb8e91a934be45afff7819c4f7612f57\\\"],\\\"sizeBytes\\\":476681373},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b641ed0d63034b23d07eb0b2cd455390e83b186e77375e2d3f37633c1ddb0495\\\"],\\\"sizeBytes\\\":473958144},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:32f9e10dfb8a7c812ea8b3e71a42bed9cef05305be18cc368b666df4643ba717\\\"],\\\"sizeBytes\\\":463179365},{\\\"names\\\":[\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8fdf28927b06a42ea8af3985d558c84d9efd142bb32d3892c4fa9f5e0d98133c\\\"],\\\"sizeBytes\\\":460774792}]}}\" for node \"crc\": Patch \"https://api-int.crc.testing:6443/api/v1/nodes/crc/status?timeout=10s\": dial tcp 38.129.56.252:6443: connect: connection refused" Dec 03 11:22:40 crc kubenswrapper[5052]: E1203 11:22:40.022675 5052 kubelet_node_status.go:585] "Error updating node status, will retry" err="error getting node \"crc\": Get \"https://api-int.crc.testing:6443/api/v1/nodes/crc?timeout=10s\": dial tcp 38.129.56.252:6443: connect: connection refused" Dec 03 11:22:40 crc kubenswrapper[5052]: E1203 11:22:40.022977 5052 kubelet_node_status.go:585] "Error updating node status, will retry" err="error getting node \"crc\": Get \"https://api-int.crc.testing:6443/api/v1/nodes/crc?timeout=10s\": dial tcp 38.129.56.252:6443: connect: connection refused" Dec 03 11:22:40 crc kubenswrapper[5052]: E1203 11:22:40.023267 5052 kubelet_node_status.go:585] "Error updating node status, will retry" err="error getting node \"crc\": Get \"https://api-int.crc.testing:6443/api/v1/nodes/crc?timeout=10s\": dial tcp 38.129.56.252:6443: connect: connection refused" Dec 03 11:22:40 crc kubenswrapper[5052]: E1203 11:22:40.023625 5052 kubelet_node_status.go:585] "Error updating node status, will retry" err="error getting node \"crc\": Get \"https://api-int.crc.testing:6443/api/v1/nodes/crc?timeout=10s\": dial tcp 38.129.56.252:6443: connect: connection refused" Dec 03 11:22:40 crc kubenswrapper[5052]: E1203 11:22:40.023658 5052 kubelet_node_status.go:572] "Unable to update node status" err="update node status exceeds retry count" Dec 03 11:22:40 crc kubenswrapper[5052]: E1203 11:22:40.398077 5052 event.go:368] "Unable to write event (may retry after sleeping)" err="Post \"https://api-int.crc.testing:6443/api/v1/namespaces/openshift-authentication/events\": dial tcp 38.129.56.252:6443: connect: connection refused" event="&Event{ObjectMeta:{oauth-openshift-6f85f596d6-nhvd7.187db0bb7e2c92de openshift-authentication 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Pod,Namespace:openshift-authentication,Name:oauth-openshift-6f85f596d6-nhvd7,UID:c712fba2-c530-4f64-836d-16706a2e2b50,APIVersion:v1,ResourceVersion:29195,FieldPath:,},Reason:FailedMount,Message:MountVolume.SetUp failed for volume \"kube-api-access-zbr2b\" : failed to fetch token: Post \"https://api-int.crc.testing:6443/api/v1/namespaces/openshift-authentication/serviceaccounts/oauth-openshift/token\": dial tcp 38.129.56.252:6443: connect: connection refused,Source:EventSource{Component:kubelet,Host:crc,},FirstTimestamp:2025-12-03 11:22:28.32132579 +0000 UTC m=+138.909567554,LastTimestamp:2025-12-03 11:22:28.32132579 +0000 UTC m=+138.909567554,Count:1,Type:Warning,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:crc,}" Dec 03 11:22:41 crc kubenswrapper[5052]: E1203 11:22:41.512885 5052 controller.go:145] "Failed to ensure lease exists, will retry" err="Get \"https://api-int.crc.testing:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/crc?timeout=10s\": dial tcp 38.129.56.252:6443: connect: connection refused" interval="6.4s" Dec 03 11:22:42 crc kubenswrapper[5052]: I1203 11:22:42.672196 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-apiserver/kube-apiserver-crc" Dec 03 11:22:42 crc kubenswrapper[5052]: I1203 11:22:42.673463 5052 status_manager.go:851] "Failed to get status for pod" podUID="68a245d3-ddaf-40a0-92f6-92760173b7a3" pod="openshift-kube-apiserver/installer-9-crc" err="Get \"https://api-int.crc.testing:6443/api/v1/namespaces/openshift-kube-apiserver/pods/installer-9-crc\": dial tcp 38.129.56.252:6443: connect: connection refused" Dec 03 11:22:42 crc kubenswrapper[5052]: I1203 11:22:42.673834 5052 status_manager.go:851] "Failed to get status for pod" podUID="355008cc-8721-44c4-a1e0-3714b0086286" pod="openshift-marketplace/redhat-marketplace-gqhzd" err="Get \"https://api-int.crc.testing:6443/api/v1/namespaces/openshift-marketplace/pods/redhat-marketplace-gqhzd\": dial tcp 38.129.56.252:6443: connect: connection refused" Dec 03 11:22:42 crc kubenswrapper[5052]: I1203 11:22:42.674077 5052 status_manager.go:851] "Failed to get status for pod" podUID="75f62784-7818-4a74-9c8c-4777d6db0611" pod="openshift-authentication/oauth-openshift-558db77b4-99jsx" err="Get \"https://api-int.crc.testing:6443/api/v1/namespaces/openshift-authentication/pods/oauth-openshift-558db77b4-99jsx\": dial tcp 38.129.56.252:6443: connect: connection refused" Dec 03 11:22:42 crc kubenswrapper[5052]: I1203 11:22:42.675574 5052 status_manager.go:851] "Failed to get status for pod" podUID="8cf9b98e-2ea9-47ee-bd1b-a913714b7668" pod="openshift-marketplace/redhat-marketplace-qttdj" err="Get \"https://api-int.crc.testing:6443/api/v1/namespaces/openshift-marketplace/pods/redhat-marketplace-qttdj\": dial tcp 38.129.56.252:6443: connect: connection refused" Dec 03 11:22:42 crc kubenswrapper[5052]: I1203 11:22:42.675886 5052 status_manager.go:851] "Failed to get status for pod" podUID="f85e55b1a89d02b0cb034b1ea31ed45a" pod="openshift-kube-apiserver/kube-apiserver-startup-monitor-crc" err="Get \"https://api-int.crc.testing:6443/api/v1/namespaces/openshift-kube-apiserver/pods/kube-apiserver-startup-monitor-crc\": dial tcp 38.129.56.252:6443: connect: connection refused" Dec 03 11:22:42 crc kubenswrapper[5052]: I1203 11:22:42.676110 5052 status_manager.go:851] "Failed to get status for pod" podUID="afb87a28-1b7c-4b09-b186-f05da232f501" pod="openshift-marketplace/redhat-operators-dwt2v" err="Get \"https://api-int.crc.testing:6443/api/v1/namespaces/openshift-marketplace/pods/redhat-operators-dwt2v\": dial tcp 38.129.56.252:6443: connect: connection refused" Dec 03 11:22:42 crc kubenswrapper[5052]: I1203 11:22:42.676550 5052 status_manager.go:851] "Failed to get status for pod" podUID="1e97c072-bdcc-4095-9a87-ddf7d4f2d22b" pod="openshift-marketplace/redhat-operators-nq2qk" err="Get \"https://api-int.crc.testing:6443/api/v1/namespaces/openshift-marketplace/pods/redhat-operators-nq2qk\": dial tcp 38.129.56.252:6443: connect: connection refused" Dec 03 11:22:42 crc kubenswrapper[5052]: I1203 11:22:42.690019 5052 kubelet.go:1909] "Trying to delete pod" pod="openshift-kube-apiserver/kube-apiserver-crc" podUID="6c8df12a-3d11-4bb9-b398-67091c1918e4" Dec 03 11:22:42 crc kubenswrapper[5052]: I1203 11:22:42.690062 5052 mirror_client.go:130] "Deleting a mirror pod" pod="openshift-kube-apiserver/kube-apiserver-crc" podUID="6c8df12a-3d11-4bb9-b398-67091c1918e4" Dec 03 11:22:42 crc kubenswrapper[5052]: E1203 11:22:42.690754 5052 mirror_client.go:138] "Failed deleting a mirror pod" err="Delete \"https://api-int.crc.testing:6443/api/v1/namespaces/openshift-kube-apiserver/pods/kube-apiserver-crc\": dial tcp 38.129.56.252:6443: connect: connection refused" pod="openshift-kube-apiserver/kube-apiserver-crc" Dec 03 11:22:42 crc kubenswrapper[5052]: I1203 11:22:42.691255 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-apiserver/kube-apiserver-crc" Dec 03 11:22:43 crc kubenswrapper[5052]: I1203 11:22:43.450589 5052 generic.go:334] "Generic (PLEG): container finished" podID="71bb4a3aecc4ba5b26c4b7318770ce13" containerID="89384a2624c4de8b69d9cbfc43f2c152ccb9de3d6f021726240b80060f50c122" exitCode=0 Dec 03 11:22:43 crc kubenswrapper[5052]: I1203 11:22:43.450697 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-kube-apiserver/kube-apiserver-crc" event={"ID":"71bb4a3aecc4ba5b26c4b7318770ce13","Type":"ContainerDied","Data":"89384a2624c4de8b69d9cbfc43f2c152ccb9de3d6f021726240b80060f50c122"} Dec 03 11:22:43 crc kubenswrapper[5052]: I1203 11:22:43.450971 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-kube-apiserver/kube-apiserver-crc" event={"ID":"71bb4a3aecc4ba5b26c4b7318770ce13","Type":"ContainerStarted","Data":"0f3c792060ea2a73a522685aca9fb7f812484a19102eb1f35705623c40e927c6"} Dec 03 11:22:43 crc kubenswrapper[5052]: I1203 11:22:43.451393 5052 kubelet.go:1909] "Trying to delete pod" pod="openshift-kube-apiserver/kube-apiserver-crc" podUID="6c8df12a-3d11-4bb9-b398-67091c1918e4" Dec 03 11:22:43 crc kubenswrapper[5052]: I1203 11:22:43.451444 5052 mirror_client.go:130] "Deleting a mirror pod" pod="openshift-kube-apiserver/kube-apiserver-crc" podUID="6c8df12a-3d11-4bb9-b398-67091c1918e4" Dec 03 11:22:43 crc kubenswrapper[5052]: E1203 11:22:43.452070 5052 mirror_client.go:138] "Failed deleting a mirror pod" err="Delete \"https://api-int.crc.testing:6443/api/v1/namespaces/openshift-kube-apiserver/pods/kube-apiserver-crc\": dial tcp 38.129.56.252:6443: connect: connection refused" pod="openshift-kube-apiserver/kube-apiserver-crc" Dec 03 11:22:43 crc kubenswrapper[5052]: I1203 11:22:43.452104 5052 status_manager.go:851] "Failed to get status for pod" podUID="8cf9b98e-2ea9-47ee-bd1b-a913714b7668" pod="openshift-marketplace/redhat-marketplace-qttdj" err="Get \"https://api-int.crc.testing:6443/api/v1/namespaces/openshift-marketplace/pods/redhat-marketplace-qttdj\": dial tcp 38.129.56.252:6443: connect: connection refused" Dec 03 11:22:43 crc kubenswrapper[5052]: I1203 11:22:43.452723 5052 status_manager.go:851] "Failed to get status for pod" podUID="f85e55b1a89d02b0cb034b1ea31ed45a" pod="openshift-kube-apiserver/kube-apiserver-startup-monitor-crc" err="Get \"https://api-int.crc.testing:6443/api/v1/namespaces/openshift-kube-apiserver/pods/kube-apiserver-startup-monitor-crc\": dial tcp 38.129.56.252:6443: connect: connection refused" Dec 03 11:22:43 crc kubenswrapper[5052]: I1203 11:22:43.453661 5052 status_manager.go:851] "Failed to get status for pod" podUID="afb87a28-1b7c-4b09-b186-f05da232f501" pod="openshift-marketplace/redhat-operators-dwt2v" err="Get \"https://api-int.crc.testing:6443/api/v1/namespaces/openshift-marketplace/pods/redhat-operators-dwt2v\": dial tcp 38.129.56.252:6443: connect: connection refused" Dec 03 11:22:43 crc kubenswrapper[5052]: I1203 11:22:43.454335 5052 status_manager.go:851] "Failed to get status for pod" podUID="1e97c072-bdcc-4095-9a87-ddf7d4f2d22b" pod="openshift-marketplace/redhat-operators-nq2qk" err="Get \"https://api-int.crc.testing:6443/api/v1/namespaces/openshift-marketplace/pods/redhat-operators-nq2qk\": dial tcp 38.129.56.252:6443: connect: connection refused" Dec 03 11:22:43 crc kubenswrapper[5052]: I1203 11:22:43.455101 5052 status_manager.go:851] "Failed to get status for pod" podUID="68a245d3-ddaf-40a0-92f6-92760173b7a3" pod="openshift-kube-apiserver/installer-9-crc" err="Get \"https://api-int.crc.testing:6443/api/v1/namespaces/openshift-kube-apiserver/pods/installer-9-crc\": dial tcp 38.129.56.252:6443: connect: connection refused" Dec 03 11:22:43 crc kubenswrapper[5052]: I1203 11:22:43.455538 5052 status_manager.go:851] "Failed to get status for pod" podUID="355008cc-8721-44c4-a1e0-3714b0086286" pod="openshift-marketplace/redhat-marketplace-gqhzd" err="Get \"https://api-int.crc.testing:6443/api/v1/namespaces/openshift-marketplace/pods/redhat-marketplace-gqhzd\": dial tcp 38.129.56.252:6443: connect: connection refused" Dec 03 11:22:43 crc kubenswrapper[5052]: I1203 11:22:43.455876 5052 status_manager.go:851] "Failed to get status for pod" podUID="75f62784-7818-4a74-9c8c-4777d6db0611" pod="openshift-authentication/oauth-openshift-558db77b4-99jsx" err="Get \"https://api-int.crc.testing:6443/api/v1/namespaces/openshift-authentication/pods/oauth-openshift-558db77b4-99jsx\": dial tcp 38.129.56.252:6443: connect: connection refused" Dec 03 11:22:43 crc kubenswrapper[5052]: I1203 11:22:43.457305 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-controller-manager_kube-controller-manager-crc_f614b9022728cf315e60c057852e563e/kube-controller-manager/0.log" Dec 03 11:22:43 crc kubenswrapper[5052]: I1203 11:22:43.457370 5052 generic.go:334] "Generic (PLEG): container finished" podID="f614b9022728cf315e60c057852e563e" containerID="f5d23c6160b5790e9272bd5ae83472267f4d1f60b67e0425a6bc2f5c1f9d1fb7" exitCode=1 Dec 03 11:22:43 crc kubenswrapper[5052]: I1203 11:22:43.457408 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-kube-controller-manager/kube-controller-manager-crc" event={"ID":"f614b9022728cf315e60c057852e563e","Type":"ContainerDied","Data":"f5d23c6160b5790e9272bd5ae83472267f4d1f60b67e0425a6bc2f5c1f9d1fb7"} Dec 03 11:22:43 crc kubenswrapper[5052]: I1203 11:22:43.458043 5052 scope.go:117] "RemoveContainer" containerID="f5d23c6160b5790e9272bd5ae83472267f4d1f60b67e0425a6bc2f5c1f9d1fb7" Dec 03 11:22:43 crc kubenswrapper[5052]: I1203 11:22:43.458232 5052 status_manager.go:851] "Failed to get status for pod" podUID="1e97c072-bdcc-4095-9a87-ddf7d4f2d22b" pod="openshift-marketplace/redhat-operators-nq2qk" err="Get \"https://api-int.crc.testing:6443/api/v1/namespaces/openshift-marketplace/pods/redhat-operators-nq2qk\": dial tcp 38.129.56.252:6443: connect: connection refused" Dec 03 11:22:43 crc kubenswrapper[5052]: I1203 11:22:43.458546 5052 status_manager.go:851] "Failed to get status for pod" podUID="68a245d3-ddaf-40a0-92f6-92760173b7a3" pod="openshift-kube-apiserver/installer-9-crc" err="Get \"https://api-int.crc.testing:6443/api/v1/namespaces/openshift-kube-apiserver/pods/installer-9-crc\": dial tcp 38.129.56.252:6443: connect: connection refused" Dec 03 11:22:43 crc kubenswrapper[5052]: I1203 11:22:43.458805 5052 status_manager.go:851] "Failed to get status for pod" podUID="355008cc-8721-44c4-a1e0-3714b0086286" pod="openshift-marketplace/redhat-marketplace-gqhzd" err="Get \"https://api-int.crc.testing:6443/api/v1/namespaces/openshift-marketplace/pods/redhat-marketplace-gqhzd\": dial tcp 38.129.56.252:6443: connect: connection refused" Dec 03 11:22:43 crc kubenswrapper[5052]: I1203 11:22:43.459127 5052 status_manager.go:851] "Failed to get status for pod" podUID="75f62784-7818-4a74-9c8c-4777d6db0611" pod="openshift-authentication/oauth-openshift-558db77b4-99jsx" err="Get \"https://api-int.crc.testing:6443/api/v1/namespaces/openshift-authentication/pods/oauth-openshift-558db77b4-99jsx\": dial tcp 38.129.56.252:6443: connect: connection refused" Dec 03 11:22:43 crc kubenswrapper[5052]: I1203 11:22:43.459664 5052 status_manager.go:851] "Failed to get status for pod" podUID="f614b9022728cf315e60c057852e563e" pod="openshift-kube-controller-manager/kube-controller-manager-crc" err="Get \"https://api-int.crc.testing:6443/api/v1/namespaces/openshift-kube-controller-manager/pods/kube-controller-manager-crc\": dial tcp 38.129.56.252:6443: connect: connection refused" Dec 03 11:22:43 crc kubenswrapper[5052]: I1203 11:22:43.459947 5052 status_manager.go:851] "Failed to get status for pod" podUID="8cf9b98e-2ea9-47ee-bd1b-a913714b7668" pod="openshift-marketplace/redhat-marketplace-qttdj" err="Get \"https://api-int.crc.testing:6443/api/v1/namespaces/openshift-marketplace/pods/redhat-marketplace-qttdj\": dial tcp 38.129.56.252:6443: connect: connection refused" Dec 03 11:22:43 crc kubenswrapper[5052]: I1203 11:22:43.460218 5052 status_manager.go:851] "Failed to get status for pod" podUID="f85e55b1a89d02b0cb034b1ea31ed45a" pod="openshift-kube-apiserver/kube-apiserver-startup-monitor-crc" err="Get \"https://api-int.crc.testing:6443/api/v1/namespaces/openshift-kube-apiserver/pods/kube-apiserver-startup-monitor-crc\": dial tcp 38.129.56.252:6443: connect: connection refused" Dec 03 11:22:43 crc kubenswrapper[5052]: I1203 11:22:43.460493 5052 status_manager.go:851] "Failed to get status for pod" podUID="afb87a28-1b7c-4b09-b186-f05da232f501" pod="openshift-marketplace/redhat-operators-dwt2v" err="Get \"https://api-int.crc.testing:6443/api/v1/namespaces/openshift-marketplace/pods/redhat-operators-dwt2v\": dial tcp 38.129.56.252:6443: connect: connection refused" Dec 03 11:22:44 crc kubenswrapper[5052]: I1203 11:22:44.041200 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-zbr2b\" (UniqueName: \"kubernetes.io/projected/c712fba2-c530-4f64-836d-16706a2e2b50-kube-api-access-zbr2b\") pod \"oauth-openshift-6f85f596d6-nhvd7\" (UID: \"c712fba2-c530-4f64-836d-16706a2e2b50\") " pod="openshift-authentication/oauth-openshift-6f85f596d6-nhvd7" Dec 03 11:22:44 crc kubenswrapper[5052]: I1203 11:22:44.387102 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-kube-controller-manager/kube-controller-manager-crc" Dec 03 11:22:44 crc kubenswrapper[5052]: I1203 11:22:44.468378 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-kube-apiserver/kube-apiserver-crc" event={"ID":"71bb4a3aecc4ba5b26c4b7318770ce13","Type":"ContainerStarted","Data":"a9c801835d2bdb7469233ab5601a37b2dfb4294b5a6d9337e6f41cdb63a8bc12"} Dec 03 11:22:44 crc kubenswrapper[5052]: I1203 11:22:44.468446 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-kube-apiserver/kube-apiserver-crc" event={"ID":"71bb4a3aecc4ba5b26c4b7318770ce13","Type":"ContainerStarted","Data":"038dc830a541ea12639d1f96bed6b1a8b23006c8b34cbe1443b2e306adb63412"} Dec 03 11:22:44 crc kubenswrapper[5052]: I1203 11:22:44.468463 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-kube-apiserver/kube-apiserver-crc" event={"ID":"71bb4a3aecc4ba5b26c4b7318770ce13","Type":"ContainerStarted","Data":"da5087cc9d242000bc3c96ae0564a3c190671d47c322c6e1c40e284b9ae15967"} Dec 03 11:22:44 crc kubenswrapper[5052]: I1203 11:22:44.468476 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-kube-apiserver/kube-apiserver-crc" event={"ID":"71bb4a3aecc4ba5b26c4b7318770ce13","Type":"ContainerStarted","Data":"300e27e77107f7911a51a0e044570f754e17f83f1fe4167817e467b0c49d8452"} Dec 03 11:22:44 crc kubenswrapper[5052]: I1203 11:22:44.472827 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-controller-manager_kube-controller-manager-crc_f614b9022728cf315e60c057852e563e/kube-controller-manager/0.log" Dec 03 11:22:44 crc kubenswrapper[5052]: I1203 11:22:44.472892 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-kube-controller-manager/kube-controller-manager-crc" event={"ID":"f614b9022728cf315e60c057852e563e","Type":"ContainerStarted","Data":"e4e1d023d2a5f295163847116f66a34781ded04d6a211cee2c628d937ddba05b"} Dec 03 11:22:45 crc kubenswrapper[5052]: I1203 11:22:45.482690 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-kube-apiserver/kube-apiserver-crc" event={"ID":"71bb4a3aecc4ba5b26c4b7318770ce13","Type":"ContainerStarted","Data":"86f136a0c6cfccfbe9c44ab53c3a763669e4401160cf077584ab51f2a2638846"} Dec 03 11:22:45 crc kubenswrapper[5052]: I1203 11:22:45.483124 5052 kubelet.go:1909] "Trying to delete pod" pod="openshift-kube-apiserver/kube-apiserver-crc" podUID="6c8df12a-3d11-4bb9-b398-67091c1918e4" Dec 03 11:22:45 crc kubenswrapper[5052]: I1203 11:22:45.483160 5052 mirror_client.go:130] "Deleting a mirror pod" pod="openshift-kube-apiserver/kube-apiserver-crc" podUID="6c8df12a-3d11-4bb9-b398-67091c1918e4" Dec 03 11:22:45 crc kubenswrapper[5052]: I1203 11:22:45.802753 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openshift-kube-controller-manager/kube-controller-manager-crc" Dec 03 11:22:47 crc kubenswrapper[5052]: I1203 11:22:47.692558 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-kube-apiserver/kube-apiserver-crc" Dec 03 11:22:47 crc kubenswrapper[5052]: I1203 11:22:47.692897 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openshift-kube-apiserver/kube-apiserver-crc" Dec 03 11:22:47 crc kubenswrapper[5052]: I1203 11:22:47.698302 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-kube-apiserver/kube-apiserver-crc" Dec 03 11:22:50 crc kubenswrapper[5052]: I1203 11:22:50.123574 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-zbr2b\" (UniqueName: \"kubernetes.io/projected/c712fba2-c530-4f64-836d-16706a2e2b50-kube-api-access-zbr2b\") pod \"oauth-openshift-6f85f596d6-nhvd7\" (UID: \"c712fba2-c530-4f64-836d-16706a2e2b50\") " pod="openshift-authentication/oauth-openshift-6f85f596d6-nhvd7" Dec 03 11:22:50 crc kubenswrapper[5052]: I1203 11:22:50.322060 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-authentication/oauth-openshift-6f85f596d6-nhvd7" Dec 03 11:22:50 crc kubenswrapper[5052]: I1203 11:22:50.491216 5052 kubelet.go:1914] "Deleted mirror pod because it is outdated" pod="openshift-kube-apiserver/kube-apiserver-crc" Dec 03 11:22:50 crc kubenswrapper[5052]: I1203 11:22:50.514378 5052 kubelet.go:1909] "Trying to delete pod" pod="openshift-kube-apiserver/kube-apiserver-crc" podUID="6c8df12a-3d11-4bb9-b398-67091c1918e4" Dec 03 11:22:50 crc kubenswrapper[5052]: I1203 11:22:50.514409 5052 mirror_client.go:130] "Deleting a mirror pod" pod="openshift-kube-apiserver/kube-apiserver-crc" podUID="6c8df12a-3d11-4bb9-b398-67091c1918e4" Dec 03 11:22:50 crc kubenswrapper[5052]: I1203 11:22:50.514583 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openshift-kube-apiserver/kube-apiserver-crc" Dec 03 11:22:50 crc kubenswrapper[5052]: I1203 11:22:50.518794 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-kube-apiserver/kube-apiserver-crc" Dec 03 11:22:50 crc kubenswrapper[5052]: I1203 11:22:50.520948 5052 status_manager.go:861] "Pod was deleted and then recreated, skipping status update" pod="openshift-kube-apiserver/kube-apiserver-crc" oldPodUID="71bb4a3aecc4ba5b26c4b7318770ce13" podUID="230d5086-806e-4794-9c25-533c9b75cbbd" Dec 03 11:22:51 crc kubenswrapper[5052]: I1203 11:22:51.521272 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-authentication_oauth-openshift-6f85f596d6-nhvd7_c712fba2-c530-4f64-836d-16706a2e2b50/oauth-openshift/0.log" Dec 03 11:22:51 crc kubenswrapper[5052]: I1203 11:22:51.521348 5052 generic.go:334] "Generic (PLEG): container finished" podID="c712fba2-c530-4f64-836d-16706a2e2b50" containerID="15b4d89a400bb02640f3e55c6672cf0c21450924b3423214669babd681bccca4" exitCode=255 Dec 03 11:22:51 crc kubenswrapper[5052]: I1203 11:22:51.521417 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-authentication/oauth-openshift-6f85f596d6-nhvd7" event={"ID":"c712fba2-c530-4f64-836d-16706a2e2b50","Type":"ContainerDied","Data":"15b4d89a400bb02640f3e55c6672cf0c21450924b3423214669babd681bccca4"} Dec 03 11:22:51 crc kubenswrapper[5052]: I1203 11:22:51.521473 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-authentication/oauth-openshift-6f85f596d6-nhvd7" event={"ID":"c712fba2-c530-4f64-836d-16706a2e2b50","Type":"ContainerStarted","Data":"425743fb775c17a1acd518db3ea1c5b6caff781a21bf94b797df4b8c24a2a29b"} Dec 03 11:22:51 crc kubenswrapper[5052]: I1203 11:22:51.521755 5052 kubelet.go:1909] "Trying to delete pod" pod="openshift-kube-apiserver/kube-apiserver-crc" podUID="6c8df12a-3d11-4bb9-b398-67091c1918e4" Dec 03 11:22:51 crc kubenswrapper[5052]: I1203 11:22:51.521774 5052 mirror_client.go:130] "Deleting a mirror pod" pod="openshift-kube-apiserver/kube-apiserver-crc" podUID="6c8df12a-3d11-4bb9-b398-67091c1918e4" Dec 03 11:22:51 crc kubenswrapper[5052]: I1203 11:22:51.522127 5052 scope.go:117] "RemoveContainer" containerID="15b4d89a400bb02640f3e55c6672cf0c21450924b3423214669babd681bccca4" Dec 03 11:22:52 crc kubenswrapper[5052]: I1203 11:22:52.528461 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-authentication_oauth-openshift-6f85f596d6-nhvd7_c712fba2-c530-4f64-836d-16706a2e2b50/oauth-openshift/1.log" Dec 03 11:22:52 crc kubenswrapper[5052]: I1203 11:22:52.529746 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-authentication_oauth-openshift-6f85f596d6-nhvd7_c712fba2-c530-4f64-836d-16706a2e2b50/oauth-openshift/0.log" Dec 03 11:22:52 crc kubenswrapper[5052]: I1203 11:22:52.529813 5052 generic.go:334] "Generic (PLEG): container finished" podID="c712fba2-c530-4f64-836d-16706a2e2b50" containerID="85580d54fd89ac968072a74b3bc437499dc423732ffbab32cff36f191dfae7b2" exitCode=255 Dec 03 11:22:52 crc kubenswrapper[5052]: I1203 11:22:52.529920 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-authentication/oauth-openshift-6f85f596d6-nhvd7" event={"ID":"c712fba2-c530-4f64-836d-16706a2e2b50","Type":"ContainerDied","Data":"85580d54fd89ac968072a74b3bc437499dc423732ffbab32cff36f191dfae7b2"} Dec 03 11:22:52 crc kubenswrapper[5052]: I1203 11:22:52.529996 5052 scope.go:117] "RemoveContainer" containerID="15b4d89a400bb02640f3e55c6672cf0c21450924b3423214669babd681bccca4" Dec 03 11:22:52 crc kubenswrapper[5052]: I1203 11:22:52.530288 5052 kubelet.go:1909] "Trying to delete pod" pod="openshift-kube-apiserver/kube-apiserver-crc" podUID="6c8df12a-3d11-4bb9-b398-67091c1918e4" Dec 03 11:22:52 crc kubenswrapper[5052]: I1203 11:22:52.530309 5052 mirror_client.go:130] "Deleting a mirror pod" pod="openshift-kube-apiserver/kube-apiserver-crc" podUID="6c8df12a-3d11-4bb9-b398-67091c1918e4" Dec 03 11:22:52 crc kubenswrapper[5052]: I1203 11:22:52.533156 5052 scope.go:117] "RemoveContainer" containerID="85580d54fd89ac968072a74b3bc437499dc423732ffbab32cff36f191dfae7b2" Dec 03 11:22:52 crc kubenswrapper[5052]: E1203 11:22:52.533596 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"oauth-openshift\" with CrashLoopBackOff: \"back-off 10s restarting failed container=oauth-openshift pod=oauth-openshift-6f85f596d6-nhvd7_openshift-authentication(c712fba2-c530-4f64-836d-16706a2e2b50)\"" pod="openshift-authentication/oauth-openshift-6f85f596d6-nhvd7" podUID="c712fba2-c530-4f64-836d-16706a2e2b50" Dec 03 11:22:53 crc kubenswrapper[5052]: I1203 11:22:53.542480 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-authentication_oauth-openshift-6f85f596d6-nhvd7_c712fba2-c530-4f64-836d-16706a2e2b50/oauth-openshift/1.log" Dec 03 11:22:53 crc kubenswrapper[5052]: I1203 11:22:53.543390 5052 scope.go:117] "RemoveContainer" containerID="85580d54fd89ac968072a74b3bc437499dc423732ffbab32cff36f191dfae7b2" Dec 03 11:22:53 crc kubenswrapper[5052]: E1203 11:22:53.543655 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"oauth-openshift\" with CrashLoopBackOff: \"back-off 10s restarting failed container=oauth-openshift pod=oauth-openshift-6f85f596d6-nhvd7_openshift-authentication(c712fba2-c530-4f64-836d-16706a2e2b50)\"" pod="openshift-authentication/oauth-openshift-6f85f596d6-nhvd7" podUID="c712fba2-c530-4f64-836d-16706a2e2b50" Dec 03 11:22:54 crc kubenswrapper[5052]: I1203 11:22:54.387011 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-kube-controller-manager/kube-controller-manager-crc" Dec 03 11:22:54 crc kubenswrapper[5052]: I1203 11:22:54.394016 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-kube-controller-manager/kube-controller-manager-crc" Dec 03 11:22:54 crc kubenswrapper[5052]: I1203 11:22:54.555367 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-kube-controller-manager/kube-controller-manager-crc" Dec 03 11:22:59 crc kubenswrapper[5052]: I1203 11:22:59.380304 5052 patch_prober.go:28] interesting pod/machine-config-daemon-74bj6 container/machine-config-daemon namespace/openshift-machine-config-operator: Liveness probe status=failure output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" start-of-body= Dec 03 11:22:59 crc kubenswrapper[5052]: I1203 11:22:59.380940 5052 prober.go:107] "Probe failed" probeType="Liveness" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" containerName="machine-config-daemon" probeResult="failure" output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" Dec 03 11:22:59 crc kubenswrapper[5052]: I1203 11:22:59.691620 5052 status_manager.go:861] "Pod was deleted and then recreated, skipping status update" pod="openshift-kube-apiserver/kube-apiserver-crc" oldPodUID="71bb4a3aecc4ba5b26c4b7318770ce13" podUID="230d5086-806e-4794-9c25-533c9b75cbbd" Dec 03 11:23:00 crc kubenswrapper[5052]: I1203 11:23:00.322544 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openshift-authentication/oauth-openshift-6f85f596d6-nhvd7" Dec 03 11:23:00 crc kubenswrapper[5052]: I1203 11:23:00.322609 5052 kubelet.go:2542] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-authentication/oauth-openshift-6f85f596d6-nhvd7" Dec 03 11:23:00 crc kubenswrapper[5052]: I1203 11:23:00.323235 5052 scope.go:117] "RemoveContainer" containerID="85580d54fd89ac968072a74b3bc437499dc423732ffbab32cff36f191dfae7b2" Dec 03 11:23:00 crc kubenswrapper[5052]: E1203 11:23:00.323491 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"oauth-openshift\" with CrashLoopBackOff: \"back-off 10s restarting failed container=oauth-openshift pod=oauth-openshift-6f85f596d6-nhvd7_openshift-authentication(c712fba2-c530-4f64-836d-16706a2e2b50)\"" pod="openshift-authentication/oauth-openshift-6f85f596d6-nhvd7" podUID="c712fba2-c530-4f64-836d-16706a2e2b50" Dec 03 11:23:00 crc kubenswrapper[5052]: I1203 11:23:00.561216 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-authentication"/"v4-0-config-system-cliconfig" Dec 03 11:23:00 crc kubenswrapper[5052]: I1203 11:23:00.756696 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-ovn-kubernetes"/"ovnkube-script-lib" Dec 03 11:23:00 crc kubenswrapper[5052]: I1203 11:23:00.774113 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-dns-operator"/"openshift-service-ca.crt" Dec 03 11:23:00 crc kubenswrapper[5052]: I1203 11:23:00.792399 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-console"/"service-ca" Dec 03 11:23:01 crc kubenswrapper[5052]: I1203 11:23:01.398876 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-operator-lifecycle-manager"/"kube-root-ca.crt" Dec 03 11:23:01 crc kubenswrapper[5052]: I1203 11:23:01.411821 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-oauth-apiserver"/"oauth-apiserver-sa-dockercfg-6r2bq" Dec 03 11:23:01 crc kubenswrapper[5052]: I1203 11:23:01.481130 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-network-operator"/"openshift-service-ca.crt" Dec 03 11:23:01 crc kubenswrapper[5052]: I1203 11:23:01.582442 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-console"/"console-dockercfg-f62pw" Dec 03 11:23:01 crc kubenswrapper[5052]: I1203 11:23:01.636942 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-authentication"/"v4-0-config-system-session" Dec 03 11:23:01 crc kubenswrapper[5052]: I1203 11:23:01.958598 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-ovn-kubernetes"/"ovn-kubernetes-control-plane-dockercfg-gs7dd" Dec 03 11:23:02 crc kubenswrapper[5052]: I1203 11:23:02.447897 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-machine-api"/"kube-rbac-proxy" Dec 03 11:23:02 crc kubenswrapper[5052]: I1203 11:23:02.461020 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-apiserver"/"openshift-apiserver-sa-dockercfg-djjff" Dec 03 11:23:02 crc kubenswrapper[5052]: I1203 11:23:02.530263 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-config-operator"/"openshift-config-operator-dockercfg-7pc5z" Dec 03 11:23:02 crc kubenswrapper[5052]: I1203 11:23:02.669725 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-apiserver-operator"/"openshift-apiserver-operator-serving-cert" Dec 03 11:23:02 crc kubenswrapper[5052]: I1203 11:23:02.716810 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-kube-apiserver-operator"/"kube-apiserver-operator-serving-cert" Dec 03 11:23:02 crc kubenswrapper[5052]: I1203 11:23:02.741462 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-oauth-apiserver"/"etcd-client" Dec 03 11:23:02 crc kubenswrapper[5052]: I1203 11:23:02.861236 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-ingress"/"router-stats-default" Dec 03 11:23:02 crc kubenswrapper[5052]: I1203 11:23:02.886269 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-machine-config-operator"/"machine-config-server-dockercfg-qx5rd" Dec 03 11:23:02 crc kubenswrapper[5052]: I1203 11:23:02.940998 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-kube-storage-version-migrator-operator"/"kube-storage-version-migrator-operator-dockercfg-2bh8d" Dec 03 11:23:03 crc kubenswrapper[5052]: I1203 11:23:03.113591 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-controller-manager"/"openshift-service-ca.crt" Dec 03 11:23:03 crc kubenswrapper[5052]: I1203 11:23:03.203703 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-authentication"/"kube-root-ca.crt" Dec 03 11:23:03 crc kubenswrapper[5052]: I1203 11:23:03.247611 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-machine-api"/"machine-api-operator-dockercfg-mfbb7" Dec 03 11:23:03 crc kubenswrapper[5052]: I1203 11:23:03.288837 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-controller-manager"/"openshift-global-ca" Dec 03 11:23:03 crc kubenswrapper[5052]: I1203 11:23:03.371148 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-apiserver-operator"/"openshift-service-ca.crt" Dec 03 11:23:03 crc kubenswrapper[5052]: I1203 11:23:03.386495 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-console"/"kube-root-ca.crt" Dec 03 11:23:03 crc kubenswrapper[5052]: I1203 11:23:03.405830 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-machine-api"/"control-plane-machine-set-operator-tls" Dec 03 11:23:03 crc kubenswrapper[5052]: I1203 11:23:03.505290 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-kube-storage-version-migrator"/"kube-root-ca.crt" Dec 03 11:23:03 crc kubenswrapper[5052]: I1203 11:23:03.507731 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-controller-manager-operator"/"openshift-service-ca.crt" Dec 03 11:23:03 crc kubenswrapper[5052]: I1203 11:23:03.557730 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-ingress-canary"/"openshift-service-ca.crt" Dec 03 11:23:03 crc kubenswrapper[5052]: I1203 11:23:03.640028 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-image-registry"/"installation-pull-secrets" Dec 03 11:23:03 crc kubenswrapper[5052]: I1203 11:23:03.677277 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-ovn-kubernetes"/"ovn-control-plane-metrics-cert" Dec 03 11:23:03 crc kubenswrapper[5052]: I1203 11:23:03.683791 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-route-controller-manager"/"route-controller-manager-sa-dockercfg-h2zr2" Dec 03 11:23:03 crc kubenswrapper[5052]: I1203 11:23:03.722399 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"hostpath-provisioner"/"kube-root-ca.crt" Dec 03 11:23:03 crc kubenswrapper[5052]: I1203 11:23:03.906618 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-kube-apiserver-operator"/"kube-root-ca.crt" Dec 03 11:23:03 crc kubenswrapper[5052]: I1203 11:23:03.961443 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-console-operator"/"console-operator-dockercfg-4xjcr" Dec 03 11:23:04 crc kubenswrapper[5052]: I1203 11:23:04.004196 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-authentication"/"v4-0-config-system-ocp-branding-template" Dec 03 11:23:04 crc kubenswrapper[5052]: I1203 11:23:04.097958 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-network-operator"/"kube-root-ca.crt" Dec 03 11:23:04 crc kubenswrapper[5052]: I1203 11:23:04.159832 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-ingress"/"service-ca-bundle" Dec 03 11:23:04 crc kubenswrapper[5052]: I1203 11:23:04.175049 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-service-ca"/"signing-cabundle" Dec 03 11:23:04 crc kubenswrapper[5052]: I1203 11:23:04.199101 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-console-operator"/"kube-root-ca.crt" Dec 03 11:23:04 crc kubenswrapper[5052]: I1203 11:23:04.250516 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-ovn-kubernetes"/"ovn-kubernetes-node-dockercfg-pwtwl" Dec 03 11:23:04 crc kubenswrapper[5052]: I1203 11:23:04.290572 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-authentication-operator"/"serving-cert" Dec 03 11:23:04 crc kubenswrapper[5052]: I1203 11:23:04.293398 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-controller-manager-operator"/"kube-root-ca.crt" Dec 03 11:23:04 crc kubenswrapper[5052]: I1203 11:23:04.297285 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-cluster-machine-approver"/"kube-root-ca.crt" Dec 03 11:23:04 crc kubenswrapper[5052]: I1203 11:23:04.608218 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-console"/"openshift-service-ca.crt" Dec 03 11:23:04 crc kubenswrapper[5052]: I1203 11:23:04.733094 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-operator-lifecycle-manager"/"package-server-manager-serving-cert" Dec 03 11:23:04 crc kubenswrapper[5052]: I1203 11:23:04.739997 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-kube-storage-version-migrator-operator"/"openshift-service-ca.crt" Dec 03 11:23:04 crc kubenswrapper[5052]: I1203 11:23:04.916556 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-authentication"/"audit" Dec 03 11:23:04 crc kubenswrapper[5052]: I1203 11:23:04.973374 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-ingress-operator"/"trusted-ca" Dec 03 11:23:04 crc kubenswrapper[5052]: I1203 11:23:04.990295 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-marketplace"/"redhat-operators-dockercfg-ct8rh" Dec 03 11:23:05 crc kubenswrapper[5052]: I1203 11:23:05.048364 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-cluster-machine-approver"/"machine-approver-tls" Dec 03 11:23:05 crc kubenswrapper[5052]: I1203 11:23:05.062692 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-service-ca-operator"/"service-ca-operator-dockercfg-rg9jl" Dec 03 11:23:05 crc kubenswrapper[5052]: I1203 11:23:05.067277 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-service-ca"/"service-ca-dockercfg-pn86c" Dec 03 11:23:05 crc kubenswrapper[5052]: I1203 11:23:05.076412 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-oauth-apiserver"/"serving-cert" Dec 03 11:23:05 crc kubenswrapper[5052]: I1203 11:23:05.082394 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-etcd-operator"/"kube-root-ca.crt" Dec 03 11:23:05 crc kubenswrapper[5052]: I1203 11:23:05.110741 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-etcd-operator"/"etcd-operator-serving-cert" Dec 03 11:23:05 crc kubenswrapper[5052]: I1203 11:23:05.212658 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-controller-manager-operator"/"openshift-controller-manager-operator-config" Dec 03 11:23:05 crc kubenswrapper[5052]: I1203 11:23:05.249191 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-cluster-machine-approver"/"kube-rbac-proxy" Dec 03 11:23:05 crc kubenswrapper[5052]: I1203 11:23:05.454278 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-cluster-version"/"kube-root-ca.crt" Dec 03 11:23:05 crc kubenswrapper[5052]: I1203 11:23:05.494180 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-network-diagnostics"/"kube-root-ca.crt" Dec 03 11:23:05 crc kubenswrapper[5052]: I1203 11:23:05.529990 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-marketplace"/"kube-root-ca.crt" Dec 03 11:23:05 crc kubenswrapper[5052]: I1203 11:23:05.544214 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-machine-config-operator"/"machine-config-operator-images" Dec 03 11:23:05 crc kubenswrapper[5052]: I1203 11:23:05.560125 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-service-ca-operator"/"kube-root-ca.crt" Dec 03 11:23:05 crc kubenswrapper[5052]: I1203 11:23:05.562347 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-apiserver"/"image-import-ca" Dec 03 11:23:05 crc kubenswrapper[5052]: I1203 11:23:05.595324 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-image-registry"/"cluster-image-registry-operator-dockercfg-m4qtx" Dec 03 11:23:05 crc kubenswrapper[5052]: I1203 11:23:05.604708 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-console"/"console-oauth-config" Dec 03 11:23:05 crc kubenswrapper[5052]: I1203 11:23:05.643843 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-dns"/"dns-dockercfg-jwfmh" Dec 03 11:23:05 crc kubenswrapper[5052]: I1203 11:23:05.756617 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-kube-storage-version-migrator"/"kube-storage-version-migrator-sa-dockercfg-5xfcg" Dec 03 11:23:05 crc kubenswrapper[5052]: I1203 11:23:05.757291 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-network-node-identity"/"ovnkube-identity-cm" Dec 03 11:23:05 crc kubenswrapper[5052]: I1203 11:23:05.783842 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-console"/"trusted-ca-bundle" Dec 03 11:23:05 crc kubenswrapper[5052]: I1203 11:23:05.922750 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-marketplace"/"marketplace-operator-metrics" Dec 03 11:23:05 crc kubenswrapper[5052]: I1203 11:23:05.956613 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-dns-operator"/"kube-root-ca.crt" Dec 03 11:23:05 crc kubenswrapper[5052]: I1203 11:23:05.992272 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-kube-storage-version-migrator"/"openshift-service-ca.crt" Dec 03 11:23:06 crc kubenswrapper[5052]: I1203 11:23:06.003752 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-ingress"/"router-dockercfg-zdk86" Dec 03 11:23:06 crc kubenswrapper[5052]: I1203 11:23:06.041029 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-machine-api"/"machine-api-operator-images" Dec 03 11:23:06 crc kubenswrapper[5052]: I1203 11:23:06.049070 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-image-registry"/"image-registry-tls" Dec 03 11:23:06 crc kubenswrapper[5052]: I1203 11:23:06.242296 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-dns-operator"/"dns-operator-dockercfg-9mqw5" Dec 03 11:23:06 crc kubenswrapper[5052]: I1203 11:23:06.304494 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-cluster-version"/"openshift-service-ca.crt" Dec 03 11:23:06 crc kubenswrapper[5052]: I1203 11:23:06.392141 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-machine-config-operator"/"machine-config-controller-dockercfg-c2lfx" Dec 03 11:23:06 crc kubenswrapper[5052]: I1203 11:23:06.433781 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-cluster-machine-approver"/"openshift-service-ca.crt" Dec 03 11:23:06 crc kubenswrapper[5052]: I1203 11:23:06.442248 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-network-console"/"networking-console-plugin-cert" Dec 03 11:23:06 crc kubenswrapper[5052]: I1203 11:23:06.517849 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-marketplace"/"redhat-marketplace-dockercfg-x2ctb" Dec 03 11:23:06 crc kubenswrapper[5052]: I1203 11:23:06.571116 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-network-operator"/"iptables-alerter-script" Dec 03 11:23:06 crc kubenswrapper[5052]: I1203 11:23:06.596430 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-multus"/"openshift-service-ca.crt" Dec 03 11:23:06 crc kubenswrapper[5052]: I1203 11:23:06.637739 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-authentication"/"v4-0-config-system-router-certs" Dec 03 11:23:06 crc kubenswrapper[5052]: I1203 11:23:06.670002 5052 reflector.go:368] Caches populated for *v1.Service from k8s.io/client-go/informers/factory.go:160 Dec 03 11:23:06 crc kubenswrapper[5052]: I1203 11:23:06.780149 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-cluster-machine-approver"/"machine-approver-sa-dockercfg-nl2j4" Dec 03 11:23:06 crc kubenswrapper[5052]: I1203 11:23:06.878243 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-dns-operator"/"metrics-tls" Dec 03 11:23:06 crc kubenswrapper[5052]: I1203 11:23:06.946442 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-etcd-operator"/"etcd-service-ca-bundle" Dec 03 11:23:06 crc kubenswrapper[5052]: I1203 11:23:06.954398 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-ingress-canary"/"default-dockercfg-2llfx" Dec 03 11:23:06 crc kubenswrapper[5052]: I1203 11:23:06.986132 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-machine-config-operator"/"proxy-tls" Dec 03 11:23:07 crc kubenswrapper[5052]: I1203 11:23:07.070630 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-ingress-operator"/"openshift-service-ca.crt" Dec 03 11:23:07 crc kubenswrapper[5052]: I1203 11:23:07.081955 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-etcd-operator"/"openshift-service-ca.crt" Dec 03 11:23:07 crc kubenswrapper[5052]: I1203 11:23:07.101112 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-multus"/"default-cni-sysctl-allowlist" Dec 03 11:23:07 crc kubenswrapper[5052]: I1203 11:23:07.102465 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-etcd-operator"/"etcd-ca-bundle" Dec 03 11:23:07 crc kubenswrapper[5052]: I1203 11:23:07.323695 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-machine-config-operator"/"node-bootstrapper-token" Dec 03 11:23:07 crc kubenswrapper[5052]: I1203 11:23:07.324032 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-dns"/"openshift-service-ca.crt" Dec 03 11:23:07 crc kubenswrapper[5052]: I1203 11:23:07.420704 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-controller-manager"/"client-ca" Dec 03 11:23:07 crc kubenswrapper[5052]: I1203 11:23:07.484024 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"hostpath-provisioner"/"openshift-service-ca.crt" Dec 03 11:23:07 crc kubenswrapper[5052]: I1203 11:23:07.565817 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-console"/"console-serving-cert" Dec 03 11:23:07 crc kubenswrapper[5052]: I1203 11:23:07.703173 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-apiserver"/"trusted-ca-bundle" Dec 03 11:23:07 crc kubenswrapper[5052]: I1203 11:23:07.770841 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-image-registry"/"image-registry-certificates" Dec 03 11:23:07 crc kubenswrapper[5052]: I1203 11:23:07.782229 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-operator-lifecycle-manager"/"olm-operator-serving-cert" Dec 03 11:23:07 crc kubenswrapper[5052]: I1203 11:23:07.801889 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-controller-manager"/"serving-cert" Dec 03 11:23:08 crc kubenswrapper[5052]: I1203 11:23:08.140175 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-machine-config-operator"/"kube-root-ca.crt" Dec 03 11:23:08 crc kubenswrapper[5052]: I1203 11:23:08.162289 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-console-operator"/"serving-cert" Dec 03 11:23:08 crc kubenswrapper[5052]: I1203 11:23:08.182802 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-authentication-operator"/"service-ca-bundle" Dec 03 11:23:08 crc kubenswrapper[5052]: I1203 11:23:08.220002 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-authentication"/"oauth-openshift-dockercfg-znhcc" Dec 03 11:23:08 crc kubenswrapper[5052]: I1203 11:23:08.311125 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-kube-storage-version-migrator-operator"/"serving-cert" Dec 03 11:23:08 crc kubenswrapper[5052]: I1203 11:23:08.354563 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-console"/"default-dockercfg-chnjx" Dec 03 11:23:08 crc kubenswrapper[5052]: I1203 11:23:08.363475 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-kube-controller-manager-operator"/"kube-root-ca.crt" Dec 03 11:23:08 crc kubenswrapper[5052]: I1203 11:23:08.370405 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-network-node-identity"/"network-node-identity-cert" Dec 03 11:23:08 crc kubenswrapper[5052]: I1203 11:23:08.408758 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-multus"/"multus-admission-controller-secret" Dec 03 11:23:08 crc kubenswrapper[5052]: I1203 11:23:08.408803 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-apiserver"/"etcd-serving-ca" Dec 03 11:23:08 crc kubenswrapper[5052]: I1203 11:23:08.536541 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-network-diagnostics"/"openshift-service-ca.crt" Dec 03 11:23:08 crc kubenswrapper[5052]: I1203 11:23:08.573148 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-apiserver"/"serving-cert" Dec 03 11:23:08 crc kubenswrapper[5052]: I1203 11:23:08.585410 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-route-controller-manager"/"kube-root-ca.crt" Dec 03 11:23:08 crc kubenswrapper[5052]: I1203 11:23:08.623534 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-marketplace"/"certified-operators-dockercfg-4rs5g" Dec 03 11:23:08 crc kubenswrapper[5052]: I1203 11:23:08.683444 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-cluster-version"/"default-dockercfg-gxtc4" Dec 03 11:23:08 crc kubenswrapper[5052]: I1203 11:23:08.686975 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-service-ca"/"kube-root-ca.crt" Dec 03 11:23:08 crc kubenswrapper[5052]: I1203 11:23:08.702438 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-authentication-operator"/"trusted-ca-bundle" Dec 03 11:23:08 crc kubenswrapper[5052]: I1203 11:23:08.750906 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-ingress-operator"/"metrics-tls" Dec 03 11:23:08 crc kubenswrapper[5052]: I1203 11:23:08.824946 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-kube-scheduler-operator"/"openshift-kube-scheduler-operator-dockercfg-qt55r" Dec 03 11:23:08 crc kubenswrapper[5052]: I1203 11:23:08.863013 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-ingress"/"openshift-service-ca.crt" Dec 03 11:23:08 crc kubenswrapper[5052]: I1203 11:23:08.888110 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-route-controller-manager"/"client-ca" Dec 03 11:23:08 crc kubenswrapper[5052]: I1203 11:23:08.900939 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-authentication"/"v4-0-config-user-template-login" Dec 03 11:23:08 crc kubenswrapper[5052]: I1203 11:23:08.903111 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-kube-storage-version-migrator-operator"/"config" Dec 03 11:23:08 crc kubenswrapper[5052]: I1203 11:23:08.944989 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-authentication-operator"/"openshift-service-ca.crt" Dec 03 11:23:09 crc kubenswrapper[5052]: I1203 11:23:09.083112 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-kube-apiserver-operator"/"kube-apiserver-operator-config" Dec 03 11:23:09 crc kubenswrapper[5052]: I1203 11:23:09.097086 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-ingress"/"router-certs-default" Dec 03 11:23:09 crc kubenswrapper[5052]: I1203 11:23:09.146542 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-ingress-operator"/"ingress-operator-dockercfg-7lnqk" Dec 03 11:23:09 crc kubenswrapper[5052]: I1203 11:23:09.392492 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-image-registry"/"openshift-service-ca.crt" Dec 03 11:23:09 crc kubenswrapper[5052]: I1203 11:23:09.433852 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-machine-config-operator"/"machine-config-operator-dockercfg-98p87" Dec 03 11:23:09 crc kubenswrapper[5052]: I1203 11:23:09.460742 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-controller-manager"/"config" Dec 03 11:23:09 crc kubenswrapper[5052]: I1203 11:23:09.537353 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-machine-api"/"openshift-service-ca.crt" Dec 03 11:23:09 crc kubenswrapper[5052]: I1203 11:23:09.626628 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-apiserver"/"audit-1" Dec 03 11:23:09 crc kubenswrapper[5052]: I1203 11:23:09.670628 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-config-operator"/"config-operator-serving-cert" Dec 03 11:23:09 crc kubenswrapper[5052]: I1203 11:23:09.679692 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-multus"/"multus-ac-dockercfg-9lkdf" Dec 03 11:23:09 crc kubenswrapper[5052]: I1203 11:23:09.694973 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-console"/"oauth-serving-cert" Dec 03 11:23:09 crc kubenswrapper[5052]: I1203 11:23:09.712123 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-service-ca"/"openshift-service-ca.crt" Dec 03 11:23:09 crc kubenswrapper[5052]: I1203 11:23:09.730483 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-ingress"/"kube-root-ca.crt" Dec 03 11:23:09 crc kubenswrapper[5052]: I1203 11:23:09.753715 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-machine-config-operator"/"machine-config-server-tls" Dec 03 11:23:09 crc kubenswrapper[5052]: I1203 11:23:09.818610 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-machine-api"/"machine-api-operator-tls" Dec 03 11:23:10 crc kubenswrapper[5052]: I1203 11:23:10.008212 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-machine-api"/"control-plane-machine-set-operator-dockercfg-k9rxt" Dec 03 11:23:10 crc kubenswrapper[5052]: I1203 11:23:10.141727 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-image-registry"/"image-registry-operator-tls" Dec 03 11:23:10 crc kubenswrapper[5052]: I1203 11:23:10.142324 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-marketplace"/"community-operators-dockercfg-dmngl" Dec 03 11:23:10 crc kubenswrapper[5052]: I1203 11:23:10.168569 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-console"/"console-config" Dec 03 11:23:10 crc kubenswrapper[5052]: I1203 11:23:10.229471 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-service-ca"/"signing-key" Dec 03 11:23:10 crc kubenswrapper[5052]: I1203 11:23:10.329052 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-oauth-apiserver"/"etcd-serving-ca" Dec 03 11:23:10 crc kubenswrapper[5052]: I1203 11:23:10.338740 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-kube-apiserver-operator"/"kube-apiserver-operator-dockercfg-x57mr" Dec 03 11:23:10 crc kubenswrapper[5052]: I1203 11:23:10.414394 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-apiserver-operator"/"openshift-apiserver-operator-config" Dec 03 11:23:10 crc kubenswrapper[5052]: I1203 11:23:10.434182 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-dns"/"dns-default-metrics-tls" Dec 03 11:23:10 crc kubenswrapper[5052]: I1203 11:23:10.459311 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-oauth-apiserver"/"trusted-ca-bundle" Dec 03 11:23:10 crc kubenswrapper[5052]: I1203 11:23:10.565842 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-service-ca-operator"/"openshift-service-ca.crt" Dec 03 11:23:10 crc kubenswrapper[5052]: I1203 11:23:10.589653 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-image-registry"/"registry-dockercfg-kzzsd" Dec 03 11:23:10 crc kubenswrapper[5052]: I1203 11:23:10.723871 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-marketplace"/"openshift-service-ca.crt" Dec 03 11:23:10 crc kubenswrapper[5052]: I1203 11:23:10.813932 5052 reflector.go:368] Caches populated for *v1.Secret from object-"hostpath-provisioner"/"csi-hostpath-provisioner-sa-dockercfg-qd74k" Dec 03 11:23:10 crc kubenswrapper[5052]: I1203 11:23:10.891051 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-machine-config-operator"/"openshift-service-ca.crt" Dec 03 11:23:10 crc kubenswrapper[5052]: I1203 11:23:10.915020 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-authentication"/"v4-0-config-system-service-ca" Dec 03 11:23:11 crc kubenswrapper[5052]: I1203 11:23:11.066658 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-network-node-identity"/"openshift-service-ca.crt" Dec 03 11:23:11 crc kubenswrapper[5052]: I1203 11:23:11.183577 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-authentication"/"openshift-service-ca.crt" Dec 03 11:23:11 crc kubenswrapper[5052]: I1203 11:23:11.197861 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-ingress-operator"/"kube-root-ca.crt" Dec 03 11:23:11 crc kubenswrapper[5052]: I1203 11:23:11.258060 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-apiserver-operator"/"openshift-apiserver-operator-dockercfg-xtcjv" Dec 03 11:23:11 crc kubenswrapper[5052]: I1203 11:23:11.321696 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-ingress-canary"/"kube-root-ca.crt" Dec 03 11:23:11 crc kubenswrapper[5052]: I1203 11:23:11.378338 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-route-controller-manager"/"openshift-service-ca.crt" Dec 03 11:23:11 crc kubenswrapper[5052]: I1203 11:23:11.420289 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-network-operator"/"metrics-tls" Dec 03 11:23:11 crc kubenswrapper[5052]: I1203 11:23:11.455440 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-kube-scheduler-operator"/"kube-root-ca.crt" Dec 03 11:23:11 crc kubenswrapper[5052]: I1203 11:23:11.468464 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-dns"/"dns-default" Dec 03 11:23:11 crc kubenswrapper[5052]: I1203 11:23:11.550631 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-ovn-kubernetes"/"ovn-node-metrics-cert" Dec 03 11:23:11 crc kubenswrapper[5052]: I1203 11:23:11.591411 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-oauth-apiserver"/"audit-1" Dec 03 11:23:11 crc kubenswrapper[5052]: I1203 11:23:11.643668 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-cluster-samples-operator"/"samples-operator-tls" Dec 03 11:23:11 crc kubenswrapper[5052]: I1203 11:23:11.647605 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-etcd-operator"/"etcd-client" Dec 03 11:23:11 crc kubenswrapper[5052]: I1203 11:23:11.742989 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-dns"/"node-resolver-dockercfg-kz9s7" Dec 03 11:23:11 crc kubenswrapper[5052]: I1203 11:23:11.752144 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-machine-config-operator"/"kube-rbac-proxy" Dec 03 11:23:11 crc kubenswrapper[5052]: I1203 11:23:11.826574 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-operator-lifecycle-manager"/"catalog-operator-serving-cert" Dec 03 11:23:11 crc kubenswrapper[5052]: I1203 11:23:11.832761 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-config-operator"/"kube-root-ca.crt" Dec 03 11:23:11 crc kubenswrapper[5052]: I1203 11:23:11.876079 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-apiserver"/"kube-root-ca.crt" Dec 03 11:23:11 crc kubenswrapper[5052]: I1203 11:23:11.947091 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-network-node-identity"/"env-overrides" Dec 03 11:23:12 crc kubenswrapper[5052]: I1203 11:23:12.006709 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-kube-storage-version-migrator-operator"/"kube-root-ca.crt" Dec 03 11:23:12 crc kubenswrapper[5052]: I1203 11:23:12.073808 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-controller-manager-operator"/"openshift-controller-manager-operator-dockercfg-vw8fw" Dec 03 11:23:12 crc kubenswrapper[5052]: I1203 11:23:12.141584 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-oauth-apiserver"/"encryption-config-1" Dec 03 11:23:12 crc kubenswrapper[5052]: I1203 11:23:12.146726 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-image-registry"/"node-ca-dockercfg-4777p" Dec 03 11:23:12 crc kubenswrapper[5052]: I1203 11:23:12.162559 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-kube-controller-manager-operator"/"kube-controller-manager-operator-serving-cert" Dec 03 11:23:12 crc kubenswrapper[5052]: I1203 11:23:12.227642 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-oauth-apiserver"/"kube-root-ca.crt" Dec 03 11:23:12 crc kubenswrapper[5052]: I1203 11:23:12.242706 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-controller-manager-operator"/"openshift-controller-manager-operator-serving-cert" Dec 03 11:23:12 crc kubenswrapper[5052]: I1203 11:23:12.259736 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-ingress"/"router-metrics-certs-default" Dec 03 11:23:12 crc kubenswrapper[5052]: I1203 11:23:12.295546 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-config-operator"/"openshift-service-ca.crt" Dec 03 11:23:12 crc kubenswrapper[5052]: I1203 11:23:12.400236 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-multus"/"default-dockercfg-2q5b6" Dec 03 11:23:12 crc kubenswrapper[5052]: I1203 11:23:12.410557 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-ovn-kubernetes"/"env-overrides" Dec 03 11:23:12 crc kubenswrapper[5052]: I1203 11:23:12.429320 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-marketplace"/"marketplace-operator-dockercfg-5nsgg" Dec 03 11:23:12 crc kubenswrapper[5052]: I1203 11:23:12.504990 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-apiserver"/"encryption-config-1" Dec 03 11:23:12 crc kubenswrapper[5052]: I1203 11:23:12.625267 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-image-registry"/"kube-root-ca.crt" Dec 03 11:23:12 crc kubenswrapper[5052]: I1203 11:23:12.628645 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-cluster-samples-operator"/"kube-root-ca.crt" Dec 03 11:23:12 crc kubenswrapper[5052]: I1203 11:23:12.662005 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-operator-lifecycle-manager"/"openshift-service-ca.crt" Dec 03 11:23:12 crc kubenswrapper[5052]: I1203 11:23:12.670135 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-multus"/"metrics-daemon-secret" Dec 03 11:23:12 crc kubenswrapper[5052]: I1203 11:23:12.753424 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-machine-config-operator"/"machine-config-daemon-dockercfg-r5tcq" Dec 03 11:23:12 crc kubenswrapper[5052]: I1203 11:23:12.851963 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-apiserver"/"openshift-service-ca.crt" Dec 03 11:23:12 crc kubenswrapper[5052]: I1203 11:23:12.898913 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-multus"/"kube-root-ca.crt" Dec 03 11:23:12 crc kubenswrapper[5052]: I1203 11:23:12.920463 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-multus"/"multus-daemon-config" Dec 03 11:23:12 crc kubenswrapper[5052]: I1203 11:23:12.947881 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-authentication"/"v4-0-config-system-trusted-ca-bundle" Dec 03 11:23:12 crc kubenswrapper[5052]: I1203 11:23:12.987772 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-console-operator"/"openshift-service-ca.crt" Dec 03 11:23:12 crc kubenswrapper[5052]: I1203 11:23:12.997655 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-authentication"/"v4-0-config-system-serving-cert" Dec 03 11:23:13 crc kubenswrapper[5052]: I1203 11:23:13.157050 5052 reflector.go:368] Caches populated for *v1.Pod from pkg/kubelet/config/apiserver.go:66 Dec 03 11:23:13 crc kubenswrapper[5052]: I1203 11:23:13.159687 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-kube-apiserver/kube-apiserver-startup-monitor-crc" podStartSLOduration=45.159668361 podStartE2EDuration="45.159668361s" podCreationTimestamp="2025-12-03 11:22:28 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:22:50.1354343 +0000 UTC m=+160.723676064" watchObservedRunningTime="2025-12-03 11:23:13.159668361 +0000 UTC m=+183.747910125" Dec 03 11:23:13 crc kubenswrapper[5052]: I1203 11:23:13.162586 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openshift-authentication/oauth-openshift-558db77b4-99jsx","openshift-kube-apiserver/kube-apiserver-crc"] Dec 03 11:23:13 crc kubenswrapper[5052]: I1203 11:23:13.162651 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openshift-kube-apiserver/kube-apiserver-crc"] Dec 03 11:23:13 crc kubenswrapper[5052]: I1203 11:23:13.162674 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-authentication/oauth-openshift-6f85f596d6-nhvd7"] Dec 03 11:23:13 crc kubenswrapper[5052]: I1203 11:23:13.163119 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-kube-controller-manager-operator"/"kube-controller-manager-operator-config" Dec 03 11:23:13 crc kubenswrapper[5052]: I1203 11:23:13.163368 5052 scope.go:117] "RemoveContainer" containerID="85580d54fd89ac968072a74b3bc437499dc423732ffbab32cff36f191dfae7b2" Dec 03 11:23:13 crc kubenswrapper[5052]: I1203 11:23:13.167867 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-kube-apiserver/kube-apiserver-crc" Dec 03 11:23:13 crc kubenswrapper[5052]: I1203 11:23:13.186650 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-kube-apiserver/kube-apiserver-crc" podStartSLOduration=23.186628136 podStartE2EDuration="23.186628136s" podCreationTimestamp="2025-12-03 11:22:50 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:23:13.184861185 +0000 UTC m=+183.773102959" watchObservedRunningTime="2025-12-03 11:23:13.186628136 +0000 UTC m=+183.774869900" Dec 03 11:23:13 crc kubenswrapper[5052]: I1203 11:23:13.230202 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-route-controller-manager"/"config" Dec 03 11:23:13 crc kubenswrapper[5052]: I1203 11:23:13.278414 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-console-operator"/"console-operator-config" Dec 03 11:23:13 crc kubenswrapper[5052]: I1203 11:23:13.483244 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-controller-manager"/"kube-root-ca.crt" Dec 03 11:23:13 crc kubenswrapper[5052]: I1203 11:23:13.546973 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-kube-controller-manager-operator"/"kube-controller-manager-operator-dockercfg-gkqpw" Dec 03 11:23:13 crc kubenswrapper[5052]: I1203 11:23:13.655709 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-cluster-samples-operator"/"cluster-samples-operator-dockercfg-xpp9w" Dec 03 11:23:13 crc kubenswrapper[5052]: I1203 11:23:13.660702 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-authentication_oauth-openshift-6f85f596d6-nhvd7_c712fba2-c530-4f64-836d-16706a2e2b50/oauth-openshift/1.log" Dec 03 11:23:13 crc kubenswrapper[5052]: I1203 11:23:13.660815 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-authentication/oauth-openshift-6f85f596d6-nhvd7" event={"ID":"c712fba2-c530-4f64-836d-16706a2e2b50","Type":"ContainerStarted","Data":"a88b3638cce9b73f5a3fff0ccf068c38e7539a5c2fe3baca7d3e906643349464"} Dec 03 11:23:13 crc kubenswrapper[5052]: I1203 11:23:13.661605 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openshift-authentication/oauth-openshift-6f85f596d6-nhvd7" Dec 03 11:23:13 crc kubenswrapper[5052]: I1203 11:23:13.662045 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-oauth-apiserver"/"openshift-service-ca.crt" Dec 03 11:23:13 crc kubenswrapper[5052]: I1203 11:23:13.683380 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="75f62784-7818-4a74-9c8c-4777d6db0611" path="/var/lib/kubelet/pods/75f62784-7818-4a74-9c8c-4777d6db0611/volumes" Dec 03 11:23:13 crc kubenswrapper[5052]: I1203 11:23:13.687054 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-authentication/oauth-openshift-6f85f596d6-nhvd7" podStartSLOduration=74.687030124 podStartE2EDuration="1m14.687030124s" podCreationTimestamp="2025-12-03 11:21:59 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:23:13.686164639 +0000 UTC m=+184.274406413" watchObservedRunningTime="2025-12-03 11:23:13.687030124 +0000 UTC m=+184.275271888" Dec 03 11:23:13 crc kubenswrapper[5052]: I1203 11:23:13.754996 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-ovn-kubernetes"/"ovnkube-config" Dec 03 11:23:13 crc kubenswrapper[5052]: I1203 11:23:13.763029 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-kube-scheduler-operator"/"kube-scheduler-operator-serving-cert" Dec 03 11:23:13 crc kubenswrapper[5052]: I1203 11:23:13.780006 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-ovn-kubernetes"/"kube-root-ca.crt" Dec 03 11:23:13 crc kubenswrapper[5052]: I1203 11:23:13.838978 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-apiserver"/"config" Dec 03 11:23:13 crc kubenswrapper[5052]: I1203 11:23:13.844300 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-authentication/oauth-openshift-6f85f596d6-nhvd7" Dec 03 11:23:13 crc kubenswrapper[5052]: I1203 11:23:13.846668 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-operator-lifecycle-manager"/"pprof-cert" Dec 03 11:23:13 crc kubenswrapper[5052]: I1203 11:23:13.860943 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-marketplace"/"marketplace-trusted-ca" Dec 03 11:23:13 crc kubenswrapper[5052]: I1203 11:23:13.918533 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-etcd-operator"/"etcd-operator-config" Dec 03 11:23:13 crc kubenswrapper[5052]: I1203 11:23:13.922182 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-cluster-samples-operator"/"openshift-service-ca.crt" Dec 03 11:23:14 crc kubenswrapper[5052]: I1203 11:23:14.010384 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-operator-lifecycle-manager"/"olm-operator-serviceaccount-dockercfg-rq7zk" Dec 03 11:23:14 crc kubenswrapper[5052]: I1203 11:23:14.154215 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-network-node-identity"/"kube-root-ca.crt" Dec 03 11:23:14 crc kubenswrapper[5052]: I1203 11:23:14.154533 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-controller-manager"/"openshift-controller-manager-sa-dockercfg-msq4c" Dec 03 11:23:14 crc kubenswrapper[5052]: I1203 11:23:14.258056 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-machine-config-operator"/"mcc-proxy-tls" Dec 03 11:23:14 crc kubenswrapper[5052]: I1203 11:23:14.628461 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-multus"/"multus-ancillary-tools-dockercfg-vnmsz" Dec 03 11:23:14 crc kubenswrapper[5052]: I1203 11:23:14.711048 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-ovn-kubernetes"/"openshift-service-ca.crt" Dec 03 11:23:14 crc kubenswrapper[5052]: I1203 11:23:14.744417 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-apiserver-operator"/"kube-root-ca.crt" Dec 03 11:23:14 crc kubenswrapper[5052]: I1203 11:23:14.795673 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-authentication"/"v4-0-config-user-template-provider-selection" Dec 03 11:23:14 crc kubenswrapper[5052]: I1203 11:23:14.885173 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-operator-lifecycle-manager"/"packageserver-service-cert" Dec 03 11:23:14 crc kubenswrapper[5052]: I1203 11:23:14.885254 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-cluster-machine-approver"/"machine-approver-config" Dec 03 11:23:14 crc kubenswrapper[5052]: I1203 11:23:14.911534 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-etcd-operator"/"etcd-operator-dockercfg-r9srn" Dec 03 11:23:14 crc kubenswrapper[5052]: I1203 11:23:14.950305 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-network-console"/"networking-console-plugin" Dec 03 11:23:15 crc kubenswrapper[5052]: I1203 11:23:15.035793 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-apiserver"/"etcd-client" Dec 03 11:23:15 crc kubenswrapper[5052]: I1203 11:23:15.216630 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-cluster-version"/"cluster-version-operator-serving-cert" Dec 03 11:23:15 crc kubenswrapper[5052]: I1203 11:23:15.228109 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-service-ca-operator"/"service-ca-operator-config" Dec 03 11:23:15 crc kubenswrapper[5052]: I1203 11:23:15.232863 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-image-registry"/"trusted-ca" Dec 03 11:23:15 crc kubenswrapper[5052]: I1203 11:23:15.387082 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-route-controller-manager"/"serving-cert" Dec 03 11:23:15 crc kubenswrapper[5052]: I1203 11:23:15.734140 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-authentication"/"v4-0-config-user-template-error" Dec 03 11:23:15 crc kubenswrapper[5052]: I1203 11:23:15.979829 5052 reflector.go:368] Caches populated for *v1.CSIDriver from k8s.io/client-go/informers/factory.go:160 Dec 03 11:23:16 crc kubenswrapper[5052]: I1203 11:23:16.085769 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-multus"/"cni-copy-resources" Dec 03 11:23:16 crc kubenswrapper[5052]: I1203 11:23:16.246188 5052 reflector.go:368] Caches populated for *v1.Node from k8s.io/client-go/informers/factory.go:160 Dec 03 11:23:16 crc kubenswrapper[5052]: I1203 11:23:16.250761 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-service-ca-operator"/"serving-cert" Dec 03 11:23:16 crc kubenswrapper[5052]: I1203 11:23:16.345219 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-dns"/"kube-root-ca.crt" Dec 03 11:23:16 crc kubenswrapper[5052]: I1203 11:23:16.463478 5052 reflector.go:368] Caches populated for *v1.RuntimeClass from k8s.io/client-go/informers/factory.go:160 Dec 03 11:23:16 crc kubenswrapper[5052]: I1203 11:23:16.490829 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-machine-api"/"kube-root-ca.crt" Dec 03 11:23:16 crc kubenswrapper[5052]: I1203 11:23:16.663044 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-authentication"/"v4-0-config-user-idp-0-file-data" Dec 03 11:23:17 crc kubenswrapper[5052]: I1203 11:23:17.374034 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-machine-config-operator"/"mco-proxy-tls" Dec 03 11:23:17 crc kubenswrapper[5052]: I1203 11:23:17.488953 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-multus"/"metrics-daemon-sa-dockercfg-d427c" Dec 03 11:23:24 crc kubenswrapper[5052]: I1203 11:23:24.147465 5052 kubelet.go:2431] "SyncLoop REMOVE" source="file" pods=["openshift-kube-apiserver/kube-apiserver-startup-monitor-crc"] Dec 03 11:23:24 crc kubenswrapper[5052]: I1203 11:23:24.149336 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openshift-kube-apiserver/kube-apiserver-startup-monitor-crc" podUID="f85e55b1a89d02b0cb034b1ea31ed45a" containerName="startup-monitor" containerID="cri-o://d689877e29d83970d21d5d92e2788fd82d91a2fad5604a21a43190e8e96a0ed6" gracePeriod=5 Dec 03 11:23:29 crc kubenswrapper[5052]: I1203 11:23:29.380310 5052 patch_prober.go:28] interesting pod/machine-config-daemon-74bj6 container/machine-config-daemon namespace/openshift-machine-config-operator: Liveness probe status=failure output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" start-of-body= Dec 03 11:23:29 crc kubenswrapper[5052]: I1203 11:23:29.380858 5052 prober.go:107] "Probe failed" probeType="Liveness" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" containerName="machine-config-daemon" probeResult="failure" output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" Dec 03 11:23:30 crc kubenswrapper[5052]: I1203 11:23:30.542581 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-authentication-operator"/"authentication-operator-dockercfg-mz9bj" Dec 03 11:23:30 crc kubenswrapper[5052]: I1203 11:23:30.712407 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-apiserver_kube-apiserver-startup-monitor-crc_f85e55b1a89d02b0cb034b1ea31ed45a/startup-monitor/0.log" Dec 03 11:23:30 crc kubenswrapper[5052]: I1203 11:23:30.712922 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-kube-apiserver/kube-apiserver-startup-monitor-crc" Dec 03 11:23:30 crc kubenswrapper[5052]: I1203 11:23:30.770203 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-apiserver_kube-apiserver-startup-monitor-crc_f85e55b1a89d02b0cb034b1ea31ed45a/startup-monitor/0.log" Dec 03 11:23:30 crc kubenswrapper[5052]: I1203 11:23:30.770280 5052 generic.go:334] "Generic (PLEG): container finished" podID="f85e55b1a89d02b0cb034b1ea31ed45a" containerID="d689877e29d83970d21d5d92e2788fd82d91a2fad5604a21a43190e8e96a0ed6" exitCode=137 Dec 03 11:23:30 crc kubenswrapper[5052]: I1203 11:23:30.770391 5052 scope.go:117] "RemoveContainer" containerID="d689877e29d83970d21d5d92e2788fd82d91a2fad5604a21a43190e8e96a0ed6" Dec 03 11:23:30 crc kubenswrapper[5052]: I1203 11:23:30.770611 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-kube-apiserver/kube-apiserver-startup-monitor-crc" Dec 03 11:23:30 crc kubenswrapper[5052]: I1203 11:23:30.792793 5052 scope.go:117] "RemoveContainer" containerID="d689877e29d83970d21d5d92e2788fd82d91a2fad5604a21a43190e8e96a0ed6" Dec 03 11:23:30 crc kubenswrapper[5052]: E1203 11:23:30.793384 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d689877e29d83970d21d5d92e2788fd82d91a2fad5604a21a43190e8e96a0ed6\": container with ID starting with d689877e29d83970d21d5d92e2788fd82d91a2fad5604a21a43190e8e96a0ed6 not found: ID does not exist" containerID="d689877e29d83970d21d5d92e2788fd82d91a2fad5604a21a43190e8e96a0ed6" Dec 03 11:23:30 crc kubenswrapper[5052]: I1203 11:23:30.793435 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d689877e29d83970d21d5d92e2788fd82d91a2fad5604a21a43190e8e96a0ed6"} err="failed to get container status \"d689877e29d83970d21d5d92e2788fd82d91a2fad5604a21a43190e8e96a0ed6\": rpc error: code = NotFound desc = could not find container \"d689877e29d83970d21d5d92e2788fd82d91a2fad5604a21a43190e8e96a0ed6\": container with ID starting with d689877e29d83970d21d5d92e2788fd82d91a2fad5604a21a43190e8e96a0ed6 not found: ID does not exist" Dec 03 11:23:30 crc kubenswrapper[5052]: I1203 11:23:30.857696 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"pod-resource-dir\" (UniqueName: \"kubernetes.io/host-path/f85e55b1a89d02b0cb034b1ea31ed45a-pod-resource-dir\") pod \"f85e55b1a89d02b0cb034b1ea31ed45a\" (UID: \"f85e55b1a89d02b0cb034b1ea31ed45a\") " Dec 03 11:23:30 crc kubenswrapper[5052]: I1203 11:23:30.857841 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"resource-dir\" (UniqueName: \"kubernetes.io/host-path/f85e55b1a89d02b0cb034b1ea31ed45a-resource-dir\") pod \"f85e55b1a89d02b0cb034b1ea31ed45a\" (UID: \"f85e55b1a89d02b0cb034b1ea31ed45a\") " Dec 03 11:23:30 crc kubenswrapper[5052]: I1203 11:23:30.857948 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/host-path/f85e55b1a89d02b0cb034b1ea31ed45a-manifests\") pod \"f85e55b1a89d02b0cb034b1ea31ed45a\" (UID: \"f85e55b1a89d02b0cb034b1ea31ed45a\") " Dec 03 11:23:30 crc kubenswrapper[5052]: I1203 11:23:30.857982 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"var-log\" (UniqueName: \"kubernetes.io/host-path/f85e55b1a89d02b0cb034b1ea31ed45a-var-log\") pod \"f85e55b1a89d02b0cb034b1ea31ed45a\" (UID: \"f85e55b1a89d02b0cb034b1ea31ed45a\") " Dec 03 11:23:30 crc kubenswrapper[5052]: I1203 11:23:30.858001 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"var-lock\" (UniqueName: \"kubernetes.io/host-path/f85e55b1a89d02b0cb034b1ea31ed45a-var-lock\") pod \"f85e55b1a89d02b0cb034b1ea31ed45a\" (UID: \"f85e55b1a89d02b0cb034b1ea31ed45a\") " Dec 03 11:23:30 crc kubenswrapper[5052]: I1203 11:23:30.857977 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/f85e55b1a89d02b0cb034b1ea31ed45a-resource-dir" (OuterVolumeSpecName: "resource-dir") pod "f85e55b1a89d02b0cb034b1ea31ed45a" (UID: "f85e55b1a89d02b0cb034b1ea31ed45a"). InnerVolumeSpecName "resource-dir". PluginName "kubernetes.io/host-path", VolumeGidValue "" Dec 03 11:23:30 crc kubenswrapper[5052]: I1203 11:23:30.858060 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/f85e55b1a89d02b0cb034b1ea31ed45a-var-log" (OuterVolumeSpecName: "var-log") pod "f85e55b1a89d02b0cb034b1ea31ed45a" (UID: "f85e55b1a89d02b0cb034b1ea31ed45a"). InnerVolumeSpecName "var-log". PluginName "kubernetes.io/host-path", VolumeGidValue "" Dec 03 11:23:30 crc kubenswrapper[5052]: I1203 11:23:30.858131 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/f85e55b1a89d02b0cb034b1ea31ed45a-manifests" (OuterVolumeSpecName: "manifests") pod "f85e55b1a89d02b0cb034b1ea31ed45a" (UID: "f85e55b1a89d02b0cb034b1ea31ed45a"). InnerVolumeSpecName "manifests". PluginName "kubernetes.io/host-path", VolumeGidValue "" Dec 03 11:23:30 crc kubenswrapper[5052]: I1203 11:23:30.858155 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/f85e55b1a89d02b0cb034b1ea31ed45a-var-lock" (OuterVolumeSpecName: "var-lock") pod "f85e55b1a89d02b0cb034b1ea31ed45a" (UID: "f85e55b1a89d02b0cb034b1ea31ed45a"). InnerVolumeSpecName "var-lock". PluginName "kubernetes.io/host-path", VolumeGidValue "" Dec 03 11:23:30 crc kubenswrapper[5052]: I1203 11:23:30.858395 5052 reconciler_common.go:293] "Volume detached for volume \"resource-dir\" (UniqueName: \"kubernetes.io/host-path/f85e55b1a89d02b0cb034b1ea31ed45a-resource-dir\") on node \"crc\" DevicePath \"\"" Dec 03 11:23:30 crc kubenswrapper[5052]: I1203 11:23:30.858413 5052 reconciler_common.go:293] "Volume detached for volume \"manifests\" (UniqueName: \"kubernetes.io/host-path/f85e55b1a89d02b0cb034b1ea31ed45a-manifests\") on node \"crc\" DevicePath \"\"" Dec 03 11:23:30 crc kubenswrapper[5052]: I1203 11:23:30.858426 5052 reconciler_common.go:293] "Volume detached for volume \"var-log\" (UniqueName: \"kubernetes.io/host-path/f85e55b1a89d02b0cb034b1ea31ed45a-var-log\") on node \"crc\" DevicePath \"\"" Dec 03 11:23:30 crc kubenswrapper[5052]: I1203 11:23:30.858438 5052 reconciler_common.go:293] "Volume detached for volume \"var-lock\" (UniqueName: \"kubernetes.io/host-path/f85e55b1a89d02b0cb034b1ea31ed45a-var-lock\") on node \"crc\" DevicePath \"\"" Dec 03 11:23:30 crc kubenswrapper[5052]: I1203 11:23:30.867902 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/f85e55b1a89d02b0cb034b1ea31ed45a-pod-resource-dir" (OuterVolumeSpecName: "pod-resource-dir") pod "f85e55b1a89d02b0cb034b1ea31ed45a" (UID: "f85e55b1a89d02b0cb034b1ea31ed45a"). InnerVolumeSpecName "pod-resource-dir". PluginName "kubernetes.io/host-path", VolumeGidValue "" Dec 03 11:23:30 crc kubenswrapper[5052]: I1203 11:23:30.960099 5052 reconciler_common.go:293] "Volume detached for volume \"pod-resource-dir\" (UniqueName: \"kubernetes.io/host-path/f85e55b1a89d02b0cb034b1ea31ed45a-pod-resource-dir\") on node \"crc\" DevicePath \"\"" Dec 03 11:23:31 crc kubenswrapper[5052]: I1203 11:23:31.449848 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-authentication-operator"/"kube-root-ca.crt" Dec 03 11:23:31 crc kubenswrapper[5052]: I1203 11:23:31.681934 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f85e55b1a89d02b0cb034b1ea31ed45a" path="/var/lib/kubelet/pods/f85e55b1a89d02b0cb034b1ea31ed45a/volumes" Dec 03 11:23:31 crc kubenswrapper[5052]: I1203 11:23:31.682281 5052 mirror_client.go:130] "Deleting a mirror pod" pod="openshift-kube-apiserver/kube-apiserver-startup-monitor-crc" podUID="" Dec 03 11:23:31 crc kubenswrapper[5052]: I1203 11:23:31.699019 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openshift-kube-apiserver/kube-apiserver-startup-monitor-crc"] Dec 03 11:23:31 crc kubenswrapper[5052]: I1203 11:23:31.699090 5052 kubelet.go:2649] "Unable to find pod for mirror pod, skipping" mirrorPod="openshift-kube-apiserver/kube-apiserver-startup-monitor-crc" mirrorPodUID="e58cb7a4-1f83-4f80-8f3e-918ffeafcc35" Dec 03 11:23:31 crc kubenswrapper[5052]: I1203 11:23:31.706063 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openshift-kube-apiserver/kube-apiserver-startup-monitor-crc"] Dec 03 11:23:31 crc kubenswrapper[5052]: I1203 11:23:31.706112 5052 kubelet.go:2673] "Unable to find pod for mirror pod, skipping" mirrorPod="openshift-kube-apiserver/kube-apiserver-startup-monitor-crc" mirrorPodUID="e58cb7a4-1f83-4f80-8f3e-918ffeafcc35" Dec 03 11:23:35 crc kubenswrapper[5052]: I1203 11:23:35.876497 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-authentication-operator"/"authentication-operator-config" Dec 03 11:23:37 crc kubenswrapper[5052]: I1203 11:23:37.610685 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-kube-scheduler-operator"/"openshift-kube-scheduler-operator-config" Dec 03 11:23:41 crc kubenswrapper[5052]: I1203 11:23:41.597000 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-console-operator"/"trusted-ca" Dec 03 11:23:48 crc kubenswrapper[5052]: I1203 11:23:48.118863 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-ingress-canary"/"canary-serving-cert" Dec 03 11:23:59 crc kubenswrapper[5052]: I1203 11:23:59.379898 5052 patch_prober.go:28] interesting pod/machine-config-daemon-74bj6 container/machine-config-daemon namespace/openshift-machine-config-operator: Liveness probe status=failure output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" start-of-body= Dec 03 11:23:59 crc kubenswrapper[5052]: I1203 11:23:59.380758 5052 prober.go:107] "Probe failed" probeType="Liveness" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" containerName="machine-config-daemon" probeResult="failure" output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" Dec 03 11:23:59 crc kubenswrapper[5052]: I1203 11:23:59.380833 5052 kubelet.go:2542] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" Dec 03 11:23:59 crc kubenswrapper[5052]: I1203 11:23:59.381785 5052 kuberuntime_manager.go:1027] "Message for Container of pod" containerName="machine-config-daemon" containerStatusID={"Type":"cri-o","ID":"1413108bd79cded55a270a3437cda42aab7c3db3cfb81894fade4363bbb28f2d"} pod="openshift-machine-config-operator/machine-config-daemon-74bj6" containerMessage="Container machine-config-daemon failed liveness probe, will be restarted" Dec 03 11:23:59 crc kubenswrapper[5052]: I1203 11:23:59.381856 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" containerName="machine-config-daemon" containerID="cri-o://1413108bd79cded55a270a3437cda42aab7c3db3cfb81894fade4363bbb28f2d" gracePeriod=600 Dec 03 11:23:59 crc kubenswrapper[5052]: I1203 11:23:59.950490 5052 generic.go:334] "Generic (PLEG): container finished" podID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" containerID="1413108bd79cded55a270a3437cda42aab7c3db3cfb81894fade4363bbb28f2d" exitCode=0 Dec 03 11:23:59 crc kubenswrapper[5052]: I1203 11:23:59.950576 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" event={"ID":"10c237bc-72e7-4da3-9f2c-b1688395bf6e","Type":"ContainerDied","Data":"1413108bd79cded55a270a3437cda42aab7c3db3cfb81894fade4363bbb28f2d"} Dec 03 11:23:59 crc kubenswrapper[5052]: I1203 11:23:59.950882 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" event={"ID":"10c237bc-72e7-4da3-9f2c-b1688395bf6e","Type":"ContainerStarted","Data":"17b2fe4901a430b4e6aeb90b23c1429860b29094991387152438ff2505fc2e22"} Dec 03 11:24:01 crc kubenswrapper[5052]: I1203 11:24:01.303196 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openshift-controller-manager/controller-manager-879f6c89f-46rm8"] Dec 03 11:24:01 crc kubenswrapper[5052]: I1203 11:24:01.304924 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openshift-controller-manager/controller-manager-879f6c89f-46rm8" podUID="1c570f3f-3090-48e1-9b7e-f66c99310dbd" containerName="controller-manager" containerID="cri-o://1f634a2dcccde0222d0716fd2f6329e6bc2208c388e57ed224043363bc2ca391" gracePeriod=30 Dec 03 11:24:01 crc kubenswrapper[5052]: I1203 11:24:01.401820 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openshift-route-controller-manager/route-controller-manager-6576b87f9c-9fpqn"] Dec 03 11:24:01 crc kubenswrapper[5052]: I1203 11:24:01.402416 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openshift-route-controller-manager/route-controller-manager-6576b87f9c-9fpqn" podUID="353b0eac-95ba-420e-bf67-a26b472fcdc2" containerName="route-controller-manager" containerID="cri-o://bf6c8c8e1f17e65818a13d1e6546cd80d767fd889d2c115fa92c485276bc40f2" gracePeriod=30 Dec 03 11:24:01 crc kubenswrapper[5052]: I1203 11:24:01.685848 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-controller-manager/controller-manager-879f6c89f-46rm8" Dec 03 11:24:01 crc kubenswrapper[5052]: I1203 11:24:01.755575 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-route-controller-manager/route-controller-manager-6576b87f9c-9fpqn" Dec 03 11:24:01 crc kubenswrapper[5052]: I1203 11:24:01.857631 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"client-ca\" (UniqueName: \"kubernetes.io/configmap/1c570f3f-3090-48e1-9b7e-f66c99310dbd-client-ca\") pod \"1c570f3f-3090-48e1-9b7e-f66c99310dbd\" (UID: \"1c570f3f-3090-48e1-9b7e-f66c99310dbd\") " Dec 03 11:24:01 crc kubenswrapper[5052]: I1203 11:24:01.857736 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"proxy-ca-bundles\" (UniqueName: \"kubernetes.io/configmap/1c570f3f-3090-48e1-9b7e-f66c99310dbd-proxy-ca-bundles\") pod \"1c570f3f-3090-48e1-9b7e-f66c99310dbd\" (UID: \"1c570f3f-3090-48e1-9b7e-f66c99310dbd\") " Dec 03 11:24:01 crc kubenswrapper[5052]: I1203 11:24:01.857784 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/353b0eac-95ba-420e-bf67-a26b472fcdc2-serving-cert\") pod \"353b0eac-95ba-420e-bf67-a26b472fcdc2\" (UID: \"353b0eac-95ba-420e-bf67-a26b472fcdc2\") " Dec 03 11:24:01 crc kubenswrapper[5052]: I1203 11:24:01.857836 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"client-ca\" (UniqueName: \"kubernetes.io/configmap/353b0eac-95ba-420e-bf67-a26b472fcdc2-client-ca\") pod \"353b0eac-95ba-420e-bf67-a26b472fcdc2\" (UID: \"353b0eac-95ba-420e-bf67-a26b472fcdc2\") " Dec 03 11:24:01 crc kubenswrapper[5052]: I1203 11:24:01.857881 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1c570f3f-3090-48e1-9b7e-f66c99310dbd-serving-cert\") pod \"1c570f3f-3090-48e1-9b7e-f66c99310dbd\" (UID: \"1c570f3f-3090-48e1-9b7e-f66c99310dbd\") " Dec 03 11:24:01 crc kubenswrapper[5052]: I1203 11:24:01.857925 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2jbxs\" (UniqueName: \"kubernetes.io/projected/1c570f3f-3090-48e1-9b7e-f66c99310dbd-kube-api-access-2jbxs\") pod \"1c570f3f-3090-48e1-9b7e-f66c99310dbd\" (UID: \"1c570f3f-3090-48e1-9b7e-f66c99310dbd\") " Dec 03 11:24:01 crc kubenswrapper[5052]: I1203 11:24:01.857959 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/353b0eac-95ba-420e-bf67-a26b472fcdc2-config\") pod \"353b0eac-95ba-420e-bf67-a26b472fcdc2\" (UID: \"353b0eac-95ba-420e-bf67-a26b472fcdc2\") " Dec 03 11:24:01 crc kubenswrapper[5052]: I1203 11:24:01.858004 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1c570f3f-3090-48e1-9b7e-f66c99310dbd-config\") pod \"1c570f3f-3090-48e1-9b7e-f66c99310dbd\" (UID: \"1c570f3f-3090-48e1-9b7e-f66c99310dbd\") " Dec 03 11:24:01 crc kubenswrapper[5052]: I1203 11:24:01.858033 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nv988\" (UniqueName: \"kubernetes.io/projected/353b0eac-95ba-420e-bf67-a26b472fcdc2-kube-api-access-nv988\") pod \"353b0eac-95ba-420e-bf67-a26b472fcdc2\" (UID: \"353b0eac-95ba-420e-bf67-a26b472fcdc2\") " Dec 03 11:24:01 crc kubenswrapper[5052]: I1203 11:24:01.858674 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1c570f3f-3090-48e1-9b7e-f66c99310dbd-client-ca" (OuterVolumeSpecName: "client-ca") pod "1c570f3f-3090-48e1-9b7e-f66c99310dbd" (UID: "1c570f3f-3090-48e1-9b7e-f66c99310dbd"). InnerVolumeSpecName "client-ca". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:24:01 crc kubenswrapper[5052]: I1203 11:24:01.858828 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1c570f3f-3090-48e1-9b7e-f66c99310dbd-proxy-ca-bundles" (OuterVolumeSpecName: "proxy-ca-bundles") pod "1c570f3f-3090-48e1-9b7e-f66c99310dbd" (UID: "1c570f3f-3090-48e1-9b7e-f66c99310dbd"). InnerVolumeSpecName "proxy-ca-bundles". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:24:01 crc kubenswrapper[5052]: I1203 11:24:01.859307 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/353b0eac-95ba-420e-bf67-a26b472fcdc2-client-ca" (OuterVolumeSpecName: "client-ca") pod "353b0eac-95ba-420e-bf67-a26b472fcdc2" (UID: "353b0eac-95ba-420e-bf67-a26b472fcdc2"). InnerVolumeSpecName "client-ca". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:24:01 crc kubenswrapper[5052]: I1203 11:24:01.859349 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/353b0eac-95ba-420e-bf67-a26b472fcdc2-config" (OuterVolumeSpecName: "config") pod "353b0eac-95ba-420e-bf67-a26b472fcdc2" (UID: "353b0eac-95ba-420e-bf67-a26b472fcdc2"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:24:01 crc kubenswrapper[5052]: I1203 11:24:01.860394 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1c570f3f-3090-48e1-9b7e-f66c99310dbd-config" (OuterVolumeSpecName: "config") pod "1c570f3f-3090-48e1-9b7e-f66c99310dbd" (UID: "1c570f3f-3090-48e1-9b7e-f66c99310dbd"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:24:01 crc kubenswrapper[5052]: I1203 11:24:01.864693 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/353b0eac-95ba-420e-bf67-a26b472fcdc2-kube-api-access-nv988" (OuterVolumeSpecName: "kube-api-access-nv988") pod "353b0eac-95ba-420e-bf67-a26b472fcdc2" (UID: "353b0eac-95ba-420e-bf67-a26b472fcdc2"). InnerVolumeSpecName "kube-api-access-nv988". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:24:01 crc kubenswrapper[5052]: I1203 11:24:01.864852 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1c570f3f-3090-48e1-9b7e-f66c99310dbd-kube-api-access-2jbxs" (OuterVolumeSpecName: "kube-api-access-2jbxs") pod "1c570f3f-3090-48e1-9b7e-f66c99310dbd" (UID: "1c570f3f-3090-48e1-9b7e-f66c99310dbd"). InnerVolumeSpecName "kube-api-access-2jbxs". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:24:01 crc kubenswrapper[5052]: I1203 11:24:01.864867 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/353b0eac-95ba-420e-bf67-a26b472fcdc2-serving-cert" (OuterVolumeSpecName: "serving-cert") pod "353b0eac-95ba-420e-bf67-a26b472fcdc2" (UID: "353b0eac-95ba-420e-bf67-a26b472fcdc2"). InnerVolumeSpecName "serving-cert". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:24:01 crc kubenswrapper[5052]: I1203 11:24:01.865111 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1c570f3f-3090-48e1-9b7e-f66c99310dbd-serving-cert" (OuterVolumeSpecName: "serving-cert") pod "1c570f3f-3090-48e1-9b7e-f66c99310dbd" (UID: "1c570f3f-3090-48e1-9b7e-f66c99310dbd"). InnerVolumeSpecName "serving-cert". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:24:01 crc kubenswrapper[5052]: I1203 11:24:01.960107 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1c570f3f-3090-48e1-9b7e-f66c99310dbd-config\") on node \"crc\" DevicePath \"\"" Dec 03 11:24:01 crc kubenswrapper[5052]: I1203 11:24:01.960155 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-nv988\" (UniqueName: \"kubernetes.io/projected/353b0eac-95ba-420e-bf67-a26b472fcdc2-kube-api-access-nv988\") on node \"crc\" DevicePath \"\"" Dec 03 11:24:01 crc kubenswrapper[5052]: I1203 11:24:01.960172 5052 reconciler_common.go:293] "Volume detached for volume \"client-ca\" (UniqueName: \"kubernetes.io/configmap/1c570f3f-3090-48e1-9b7e-f66c99310dbd-client-ca\") on node \"crc\" DevicePath \"\"" Dec 03 11:24:01 crc kubenswrapper[5052]: I1203 11:24:01.960183 5052 reconciler_common.go:293] "Volume detached for volume \"proxy-ca-bundles\" (UniqueName: \"kubernetes.io/configmap/1c570f3f-3090-48e1-9b7e-f66c99310dbd-proxy-ca-bundles\") on node \"crc\" DevicePath \"\"" Dec 03 11:24:01 crc kubenswrapper[5052]: I1203 11:24:01.960192 5052 reconciler_common.go:293] "Volume detached for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/353b0eac-95ba-420e-bf67-a26b472fcdc2-serving-cert\") on node \"crc\" DevicePath \"\"" Dec 03 11:24:01 crc kubenswrapper[5052]: I1203 11:24:01.960200 5052 reconciler_common.go:293] "Volume detached for volume \"client-ca\" (UniqueName: \"kubernetes.io/configmap/353b0eac-95ba-420e-bf67-a26b472fcdc2-client-ca\") on node \"crc\" DevicePath \"\"" Dec 03 11:24:01 crc kubenswrapper[5052]: I1203 11:24:01.960207 5052 reconciler_common.go:293] "Volume detached for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1c570f3f-3090-48e1-9b7e-f66c99310dbd-serving-cert\") on node \"crc\" DevicePath \"\"" Dec 03 11:24:01 crc kubenswrapper[5052]: I1203 11:24:01.960215 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-2jbxs\" (UniqueName: \"kubernetes.io/projected/1c570f3f-3090-48e1-9b7e-f66c99310dbd-kube-api-access-2jbxs\") on node \"crc\" DevicePath \"\"" Dec 03 11:24:01 crc kubenswrapper[5052]: I1203 11:24:01.960243 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/353b0eac-95ba-420e-bf67-a26b472fcdc2-config\") on node \"crc\" DevicePath \"\"" Dec 03 11:24:01 crc kubenswrapper[5052]: I1203 11:24:01.967963 5052 generic.go:334] "Generic (PLEG): container finished" podID="1c570f3f-3090-48e1-9b7e-f66c99310dbd" containerID="1f634a2dcccde0222d0716fd2f6329e6bc2208c388e57ed224043363bc2ca391" exitCode=0 Dec 03 11:24:01 crc kubenswrapper[5052]: I1203 11:24:01.968051 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-controller-manager/controller-manager-879f6c89f-46rm8" event={"ID":"1c570f3f-3090-48e1-9b7e-f66c99310dbd","Type":"ContainerDied","Data":"1f634a2dcccde0222d0716fd2f6329e6bc2208c388e57ed224043363bc2ca391"} Dec 03 11:24:01 crc kubenswrapper[5052]: I1203 11:24:01.968063 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-controller-manager/controller-manager-879f6c89f-46rm8" Dec 03 11:24:01 crc kubenswrapper[5052]: I1203 11:24:01.968086 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-controller-manager/controller-manager-879f6c89f-46rm8" event={"ID":"1c570f3f-3090-48e1-9b7e-f66c99310dbd","Type":"ContainerDied","Data":"13df0bb2759655c12e4f73784433bcd7dc9c684b4ff40da2bbafe3119c9eb255"} Dec 03 11:24:01 crc kubenswrapper[5052]: I1203 11:24:01.968110 5052 scope.go:117] "RemoveContainer" containerID="1f634a2dcccde0222d0716fd2f6329e6bc2208c388e57ed224043363bc2ca391" Dec 03 11:24:01 crc kubenswrapper[5052]: I1203 11:24:01.971020 5052 generic.go:334] "Generic (PLEG): container finished" podID="353b0eac-95ba-420e-bf67-a26b472fcdc2" containerID="bf6c8c8e1f17e65818a13d1e6546cd80d767fd889d2c115fa92c485276bc40f2" exitCode=0 Dec 03 11:24:01 crc kubenswrapper[5052]: I1203 11:24:01.971056 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-route-controller-manager/route-controller-manager-6576b87f9c-9fpqn" event={"ID":"353b0eac-95ba-420e-bf67-a26b472fcdc2","Type":"ContainerDied","Data":"bf6c8c8e1f17e65818a13d1e6546cd80d767fd889d2c115fa92c485276bc40f2"} Dec 03 11:24:01 crc kubenswrapper[5052]: I1203 11:24:01.971080 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-route-controller-manager/route-controller-manager-6576b87f9c-9fpqn" event={"ID":"353b0eac-95ba-420e-bf67-a26b472fcdc2","Type":"ContainerDied","Data":"ac445a00288813c296aca72aa0f9283d19329a7fb1a7ba6f66cddb2fe3f5494f"} Dec 03 11:24:01 crc kubenswrapper[5052]: I1203 11:24:01.971130 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-route-controller-manager/route-controller-manager-6576b87f9c-9fpqn" Dec 03 11:24:01 crc kubenswrapper[5052]: I1203 11:24:01.989011 5052 scope.go:117] "RemoveContainer" containerID="1f634a2dcccde0222d0716fd2f6329e6bc2208c388e57ed224043363bc2ca391" Dec 03 11:24:01 crc kubenswrapper[5052]: E1203 11:24:01.989670 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1f634a2dcccde0222d0716fd2f6329e6bc2208c388e57ed224043363bc2ca391\": container with ID starting with 1f634a2dcccde0222d0716fd2f6329e6bc2208c388e57ed224043363bc2ca391 not found: ID does not exist" containerID="1f634a2dcccde0222d0716fd2f6329e6bc2208c388e57ed224043363bc2ca391" Dec 03 11:24:01 crc kubenswrapper[5052]: I1203 11:24:01.989718 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1f634a2dcccde0222d0716fd2f6329e6bc2208c388e57ed224043363bc2ca391"} err="failed to get container status \"1f634a2dcccde0222d0716fd2f6329e6bc2208c388e57ed224043363bc2ca391\": rpc error: code = NotFound desc = could not find container \"1f634a2dcccde0222d0716fd2f6329e6bc2208c388e57ed224043363bc2ca391\": container with ID starting with 1f634a2dcccde0222d0716fd2f6329e6bc2208c388e57ed224043363bc2ca391 not found: ID does not exist" Dec 03 11:24:01 crc kubenswrapper[5052]: I1203 11:24:01.989751 5052 scope.go:117] "RemoveContainer" containerID="bf6c8c8e1f17e65818a13d1e6546cd80d767fd889d2c115fa92c485276bc40f2" Dec 03 11:24:02 crc kubenswrapper[5052]: I1203 11:24:02.009294 5052 scope.go:117] "RemoveContainer" containerID="bf6c8c8e1f17e65818a13d1e6546cd80d767fd889d2c115fa92c485276bc40f2" Dec 03 11:24:02 crc kubenswrapper[5052]: E1203 11:24:02.010024 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bf6c8c8e1f17e65818a13d1e6546cd80d767fd889d2c115fa92c485276bc40f2\": container with ID starting with bf6c8c8e1f17e65818a13d1e6546cd80d767fd889d2c115fa92c485276bc40f2 not found: ID does not exist" containerID="bf6c8c8e1f17e65818a13d1e6546cd80d767fd889d2c115fa92c485276bc40f2" Dec 03 11:24:02 crc kubenswrapper[5052]: I1203 11:24:02.010115 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bf6c8c8e1f17e65818a13d1e6546cd80d767fd889d2c115fa92c485276bc40f2"} err="failed to get container status \"bf6c8c8e1f17e65818a13d1e6546cd80d767fd889d2c115fa92c485276bc40f2\": rpc error: code = NotFound desc = could not find container \"bf6c8c8e1f17e65818a13d1e6546cd80d767fd889d2c115fa92c485276bc40f2\": container with ID starting with bf6c8c8e1f17e65818a13d1e6546cd80d767fd889d2c115fa92c485276bc40f2 not found: ID does not exist" Dec 03 11:24:02 crc kubenswrapper[5052]: I1203 11:24:02.013596 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openshift-route-controller-manager/route-controller-manager-6576b87f9c-9fpqn"] Dec 03 11:24:02 crc kubenswrapper[5052]: I1203 11:24:02.018556 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openshift-route-controller-manager/route-controller-manager-6576b87f9c-9fpqn"] Dec 03 11:24:02 crc kubenswrapper[5052]: I1203 11:24:02.029626 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openshift-controller-manager/controller-manager-879f6c89f-46rm8"] Dec 03 11:24:02 crc kubenswrapper[5052]: I1203 11:24:02.034158 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openshift-controller-manager/controller-manager-879f6c89f-46rm8"] Dec 03 11:24:03 crc kubenswrapper[5052]: I1203 11:24:03.292748 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openshift-controller-manager/controller-manager-85c48b9894-nfp9p"] Dec 03 11:24:03 crc kubenswrapper[5052]: E1203 11:24:03.293049 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="1c570f3f-3090-48e1-9b7e-f66c99310dbd" containerName="controller-manager" Dec 03 11:24:03 crc kubenswrapper[5052]: I1203 11:24:03.293064 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="1c570f3f-3090-48e1-9b7e-f66c99310dbd" containerName="controller-manager" Dec 03 11:24:03 crc kubenswrapper[5052]: E1203 11:24:03.293084 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="f85e55b1a89d02b0cb034b1ea31ed45a" containerName="startup-monitor" Dec 03 11:24:03 crc kubenswrapper[5052]: I1203 11:24:03.293092 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="f85e55b1a89d02b0cb034b1ea31ed45a" containerName="startup-monitor" Dec 03 11:24:03 crc kubenswrapper[5052]: E1203 11:24:03.293105 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="68a245d3-ddaf-40a0-92f6-92760173b7a3" containerName="installer" Dec 03 11:24:03 crc kubenswrapper[5052]: I1203 11:24:03.293114 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="68a245d3-ddaf-40a0-92f6-92760173b7a3" containerName="installer" Dec 03 11:24:03 crc kubenswrapper[5052]: E1203 11:24:03.293136 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="353b0eac-95ba-420e-bf67-a26b472fcdc2" containerName="route-controller-manager" Dec 03 11:24:03 crc kubenswrapper[5052]: I1203 11:24:03.293144 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="353b0eac-95ba-420e-bf67-a26b472fcdc2" containerName="route-controller-manager" Dec 03 11:24:03 crc kubenswrapper[5052]: I1203 11:24:03.293254 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="f85e55b1a89d02b0cb034b1ea31ed45a" containerName="startup-monitor" Dec 03 11:24:03 crc kubenswrapper[5052]: I1203 11:24:03.293268 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="68a245d3-ddaf-40a0-92f6-92760173b7a3" containerName="installer" Dec 03 11:24:03 crc kubenswrapper[5052]: I1203 11:24:03.293278 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="353b0eac-95ba-420e-bf67-a26b472fcdc2" containerName="route-controller-manager" Dec 03 11:24:03 crc kubenswrapper[5052]: I1203 11:24:03.293287 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="1c570f3f-3090-48e1-9b7e-f66c99310dbd" containerName="controller-manager" Dec 03 11:24:03 crc kubenswrapper[5052]: I1203 11:24:03.293790 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-controller-manager/controller-manager-85c48b9894-nfp9p" Dec 03 11:24:03 crc kubenswrapper[5052]: I1203 11:24:03.296229 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-controller-manager"/"kube-root-ca.crt" Dec 03 11:24:03 crc kubenswrapper[5052]: I1203 11:24:03.296459 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openshift-route-controller-manager/route-controller-manager-5966cc58bc-hlf6h"] Dec 03 11:24:03 crc kubenswrapper[5052]: I1203 11:24:03.297311 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-route-controller-manager/route-controller-manager-5966cc58bc-hlf6h" Dec 03 11:24:03 crc kubenswrapper[5052]: I1203 11:24:03.297343 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-controller-manager"/"serving-cert" Dec 03 11:24:03 crc kubenswrapper[5052]: I1203 11:24:03.297470 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-controller-manager"/"config" Dec 03 11:24:03 crc kubenswrapper[5052]: I1203 11:24:03.297674 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-controller-manager"/"openshift-service-ca.crt" Dec 03 11:24:03 crc kubenswrapper[5052]: I1203 11:24:03.298454 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-controller-manager"/"openshift-controller-manager-sa-dockercfg-msq4c" Dec 03 11:24:03 crc kubenswrapper[5052]: I1203 11:24:03.302317 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-route-controller-manager"/"config" Dec 03 11:24:03 crc kubenswrapper[5052]: I1203 11:24:03.302761 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-controller-manager"/"client-ca" Dec 03 11:24:03 crc kubenswrapper[5052]: I1203 11:24:03.302862 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-route-controller-manager"/"serving-cert" Dec 03 11:24:03 crc kubenswrapper[5052]: I1203 11:24:03.303014 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-route-controller-manager"/"openshift-service-ca.crt" Dec 03 11:24:03 crc kubenswrapper[5052]: I1203 11:24:03.302534 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-route-controller-manager"/"kube-root-ca.crt" Dec 03 11:24:03 crc kubenswrapper[5052]: I1203 11:24:03.303846 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-route-controller-manager"/"client-ca" Dec 03 11:24:03 crc kubenswrapper[5052]: I1203 11:24:03.304127 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-route-controller-manager"/"route-controller-manager-sa-dockercfg-h2zr2" Dec 03 11:24:03 crc kubenswrapper[5052]: I1203 11:24:03.308241 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-controller-manager"/"openshift-global-ca" Dec 03 11:24:03 crc kubenswrapper[5052]: I1203 11:24:03.310854 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-controller-manager/controller-manager-85c48b9894-nfp9p"] Dec 03 11:24:03 crc kubenswrapper[5052]: I1203 11:24:03.317786 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-route-controller-manager/route-controller-manager-5966cc58bc-hlf6h"] Dec 03 11:24:03 crc kubenswrapper[5052]: I1203 11:24:03.381676 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bgdb6\" (UniqueName: \"kubernetes.io/projected/7489ae0d-ae13-4ae8-9ec0-f1c2c1233562-kube-api-access-bgdb6\") pod \"route-controller-manager-5966cc58bc-hlf6h\" (UID: \"7489ae0d-ae13-4ae8-9ec0-f1c2c1233562\") " pod="openshift-route-controller-manager/route-controller-manager-5966cc58bc-hlf6h" Dec 03 11:24:03 crc kubenswrapper[5052]: I1203 11:24:03.381746 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c9317f31-e609-4397-9b9f-dcdce33196ad-serving-cert\") pod \"controller-manager-85c48b9894-nfp9p\" (UID: \"c9317f31-e609-4397-9b9f-dcdce33196ad\") " pod="openshift-controller-manager/controller-manager-85c48b9894-nfp9p" Dec 03 11:24:03 crc kubenswrapper[5052]: I1203 11:24:03.381872 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-852nw\" (UniqueName: \"kubernetes.io/projected/c9317f31-e609-4397-9b9f-dcdce33196ad-kube-api-access-852nw\") pod \"controller-manager-85c48b9894-nfp9p\" (UID: \"c9317f31-e609-4397-9b9f-dcdce33196ad\") " pod="openshift-controller-manager/controller-manager-85c48b9894-nfp9p" Dec 03 11:24:03 crc kubenswrapper[5052]: I1203 11:24:03.381917 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c9317f31-e609-4397-9b9f-dcdce33196ad-config\") pod \"controller-manager-85c48b9894-nfp9p\" (UID: \"c9317f31-e609-4397-9b9f-dcdce33196ad\") " pod="openshift-controller-manager/controller-manager-85c48b9894-nfp9p" Dec 03 11:24:03 crc kubenswrapper[5052]: I1203 11:24:03.381961 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-ca-bundles\" (UniqueName: \"kubernetes.io/configmap/c9317f31-e609-4397-9b9f-dcdce33196ad-proxy-ca-bundles\") pod \"controller-manager-85c48b9894-nfp9p\" (UID: \"c9317f31-e609-4397-9b9f-dcdce33196ad\") " pod="openshift-controller-manager/controller-manager-85c48b9894-nfp9p" Dec 03 11:24:03 crc kubenswrapper[5052]: I1203 11:24:03.381994 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7489ae0d-ae13-4ae8-9ec0-f1c2c1233562-serving-cert\") pod \"route-controller-manager-5966cc58bc-hlf6h\" (UID: \"7489ae0d-ae13-4ae8-9ec0-f1c2c1233562\") " pod="openshift-route-controller-manager/route-controller-manager-5966cc58bc-hlf6h" Dec 03 11:24:03 crc kubenswrapper[5052]: I1203 11:24:03.382025 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca\" (UniqueName: \"kubernetes.io/configmap/c9317f31-e609-4397-9b9f-dcdce33196ad-client-ca\") pod \"controller-manager-85c48b9894-nfp9p\" (UID: \"c9317f31-e609-4397-9b9f-dcdce33196ad\") " pod="openshift-controller-manager/controller-manager-85c48b9894-nfp9p" Dec 03 11:24:03 crc kubenswrapper[5052]: I1203 11:24:03.382126 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/7489ae0d-ae13-4ae8-9ec0-f1c2c1233562-config\") pod \"route-controller-manager-5966cc58bc-hlf6h\" (UID: \"7489ae0d-ae13-4ae8-9ec0-f1c2c1233562\") " pod="openshift-route-controller-manager/route-controller-manager-5966cc58bc-hlf6h" Dec 03 11:24:03 crc kubenswrapper[5052]: I1203 11:24:03.382372 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca\" (UniqueName: \"kubernetes.io/configmap/7489ae0d-ae13-4ae8-9ec0-f1c2c1233562-client-ca\") pod \"route-controller-manager-5966cc58bc-hlf6h\" (UID: \"7489ae0d-ae13-4ae8-9ec0-f1c2c1233562\") " pod="openshift-route-controller-manager/route-controller-manager-5966cc58bc-hlf6h" Dec 03 11:24:03 crc kubenswrapper[5052]: I1203 11:24:03.483323 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-bgdb6\" (UniqueName: \"kubernetes.io/projected/7489ae0d-ae13-4ae8-9ec0-f1c2c1233562-kube-api-access-bgdb6\") pod \"route-controller-manager-5966cc58bc-hlf6h\" (UID: \"7489ae0d-ae13-4ae8-9ec0-f1c2c1233562\") " pod="openshift-route-controller-manager/route-controller-manager-5966cc58bc-hlf6h" Dec 03 11:24:03 crc kubenswrapper[5052]: I1203 11:24:03.483391 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c9317f31-e609-4397-9b9f-dcdce33196ad-serving-cert\") pod \"controller-manager-85c48b9894-nfp9p\" (UID: \"c9317f31-e609-4397-9b9f-dcdce33196ad\") " pod="openshift-controller-manager/controller-manager-85c48b9894-nfp9p" Dec 03 11:24:03 crc kubenswrapper[5052]: I1203 11:24:03.483439 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-852nw\" (UniqueName: \"kubernetes.io/projected/c9317f31-e609-4397-9b9f-dcdce33196ad-kube-api-access-852nw\") pod \"controller-manager-85c48b9894-nfp9p\" (UID: \"c9317f31-e609-4397-9b9f-dcdce33196ad\") " pod="openshift-controller-manager/controller-manager-85c48b9894-nfp9p" Dec 03 11:24:03 crc kubenswrapper[5052]: I1203 11:24:03.483472 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c9317f31-e609-4397-9b9f-dcdce33196ad-config\") pod \"controller-manager-85c48b9894-nfp9p\" (UID: \"c9317f31-e609-4397-9b9f-dcdce33196ad\") " pod="openshift-controller-manager/controller-manager-85c48b9894-nfp9p" Dec 03 11:24:03 crc kubenswrapper[5052]: I1203 11:24:03.483530 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"proxy-ca-bundles\" (UniqueName: \"kubernetes.io/configmap/c9317f31-e609-4397-9b9f-dcdce33196ad-proxy-ca-bundles\") pod \"controller-manager-85c48b9894-nfp9p\" (UID: \"c9317f31-e609-4397-9b9f-dcdce33196ad\") " pod="openshift-controller-manager/controller-manager-85c48b9894-nfp9p" Dec 03 11:24:03 crc kubenswrapper[5052]: I1203 11:24:03.483560 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7489ae0d-ae13-4ae8-9ec0-f1c2c1233562-serving-cert\") pod \"route-controller-manager-5966cc58bc-hlf6h\" (UID: \"7489ae0d-ae13-4ae8-9ec0-f1c2c1233562\") " pod="openshift-route-controller-manager/route-controller-manager-5966cc58bc-hlf6h" Dec 03 11:24:03 crc kubenswrapper[5052]: I1203 11:24:03.483583 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"client-ca\" (UniqueName: \"kubernetes.io/configmap/c9317f31-e609-4397-9b9f-dcdce33196ad-client-ca\") pod \"controller-manager-85c48b9894-nfp9p\" (UID: \"c9317f31-e609-4397-9b9f-dcdce33196ad\") " pod="openshift-controller-manager/controller-manager-85c48b9894-nfp9p" Dec 03 11:24:03 crc kubenswrapper[5052]: I1203 11:24:03.483611 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/7489ae0d-ae13-4ae8-9ec0-f1c2c1233562-config\") pod \"route-controller-manager-5966cc58bc-hlf6h\" (UID: \"7489ae0d-ae13-4ae8-9ec0-f1c2c1233562\") " pod="openshift-route-controller-manager/route-controller-manager-5966cc58bc-hlf6h" Dec 03 11:24:03 crc kubenswrapper[5052]: I1203 11:24:03.483651 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"client-ca\" (UniqueName: \"kubernetes.io/configmap/7489ae0d-ae13-4ae8-9ec0-f1c2c1233562-client-ca\") pod \"route-controller-manager-5966cc58bc-hlf6h\" (UID: \"7489ae0d-ae13-4ae8-9ec0-f1c2c1233562\") " pod="openshift-route-controller-manager/route-controller-manager-5966cc58bc-hlf6h" Dec 03 11:24:03 crc kubenswrapper[5052]: I1203 11:24:03.484752 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"client-ca\" (UniqueName: \"kubernetes.io/configmap/7489ae0d-ae13-4ae8-9ec0-f1c2c1233562-client-ca\") pod \"route-controller-manager-5966cc58bc-hlf6h\" (UID: \"7489ae0d-ae13-4ae8-9ec0-f1c2c1233562\") " pod="openshift-route-controller-manager/route-controller-manager-5966cc58bc-hlf6h" Dec 03 11:24:03 crc kubenswrapper[5052]: I1203 11:24:03.485104 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/7489ae0d-ae13-4ae8-9ec0-f1c2c1233562-config\") pod \"route-controller-manager-5966cc58bc-hlf6h\" (UID: \"7489ae0d-ae13-4ae8-9ec0-f1c2c1233562\") " pod="openshift-route-controller-manager/route-controller-manager-5966cc58bc-hlf6h" Dec 03 11:24:03 crc kubenswrapper[5052]: I1203 11:24:03.485467 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"client-ca\" (UniqueName: \"kubernetes.io/configmap/c9317f31-e609-4397-9b9f-dcdce33196ad-client-ca\") pod \"controller-manager-85c48b9894-nfp9p\" (UID: \"c9317f31-e609-4397-9b9f-dcdce33196ad\") " pod="openshift-controller-manager/controller-manager-85c48b9894-nfp9p" Dec 03 11:24:03 crc kubenswrapper[5052]: I1203 11:24:03.487674 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c9317f31-e609-4397-9b9f-dcdce33196ad-config\") pod \"controller-manager-85c48b9894-nfp9p\" (UID: \"c9317f31-e609-4397-9b9f-dcdce33196ad\") " pod="openshift-controller-manager/controller-manager-85c48b9894-nfp9p" Dec 03 11:24:03 crc kubenswrapper[5052]: I1203 11:24:03.488407 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7489ae0d-ae13-4ae8-9ec0-f1c2c1233562-serving-cert\") pod \"route-controller-manager-5966cc58bc-hlf6h\" (UID: \"7489ae0d-ae13-4ae8-9ec0-f1c2c1233562\") " pod="openshift-route-controller-manager/route-controller-manager-5966cc58bc-hlf6h" Dec 03 11:24:03 crc kubenswrapper[5052]: I1203 11:24:03.489052 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"proxy-ca-bundles\" (UniqueName: \"kubernetes.io/configmap/c9317f31-e609-4397-9b9f-dcdce33196ad-proxy-ca-bundles\") pod \"controller-manager-85c48b9894-nfp9p\" (UID: \"c9317f31-e609-4397-9b9f-dcdce33196ad\") " pod="openshift-controller-manager/controller-manager-85c48b9894-nfp9p" Dec 03 11:24:03 crc kubenswrapper[5052]: I1203 11:24:03.490589 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c9317f31-e609-4397-9b9f-dcdce33196ad-serving-cert\") pod \"controller-manager-85c48b9894-nfp9p\" (UID: \"c9317f31-e609-4397-9b9f-dcdce33196ad\") " pod="openshift-controller-manager/controller-manager-85c48b9894-nfp9p" Dec 03 11:24:03 crc kubenswrapper[5052]: I1203 11:24:03.502423 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-bgdb6\" (UniqueName: \"kubernetes.io/projected/7489ae0d-ae13-4ae8-9ec0-f1c2c1233562-kube-api-access-bgdb6\") pod \"route-controller-manager-5966cc58bc-hlf6h\" (UID: \"7489ae0d-ae13-4ae8-9ec0-f1c2c1233562\") " pod="openshift-route-controller-manager/route-controller-manager-5966cc58bc-hlf6h" Dec 03 11:24:03 crc kubenswrapper[5052]: I1203 11:24:03.503530 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-852nw\" (UniqueName: \"kubernetes.io/projected/c9317f31-e609-4397-9b9f-dcdce33196ad-kube-api-access-852nw\") pod \"controller-manager-85c48b9894-nfp9p\" (UID: \"c9317f31-e609-4397-9b9f-dcdce33196ad\") " pod="openshift-controller-manager/controller-manager-85c48b9894-nfp9p" Dec 03 11:24:03 crc kubenswrapper[5052]: I1203 11:24:03.612167 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-controller-manager/controller-manager-85c48b9894-nfp9p" Dec 03 11:24:03 crc kubenswrapper[5052]: I1203 11:24:03.629305 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-route-controller-manager/route-controller-manager-5966cc58bc-hlf6h" Dec 03 11:24:03 crc kubenswrapper[5052]: I1203 11:24:03.681966 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1c570f3f-3090-48e1-9b7e-f66c99310dbd" path="/var/lib/kubelet/pods/1c570f3f-3090-48e1-9b7e-f66c99310dbd/volumes" Dec 03 11:24:03 crc kubenswrapper[5052]: I1203 11:24:03.683250 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="353b0eac-95ba-420e-bf67-a26b472fcdc2" path="/var/lib/kubelet/pods/353b0eac-95ba-420e-bf67-a26b472fcdc2/volumes" Dec 03 11:24:03 crc kubenswrapper[5052]: I1203 11:24:03.853172 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-route-controller-manager/route-controller-manager-5966cc58bc-hlf6h"] Dec 03 11:24:03 crc kubenswrapper[5052]: I1203 11:24:03.890790 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-controller-manager/controller-manager-85c48b9894-nfp9p"] Dec 03 11:24:03 crc kubenswrapper[5052]: W1203 11:24:03.899067 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9317f31_e609_4397_9b9f_dcdce33196ad.slice/crio-c7e00e9762c4d58919f30bf1f891b54978bfd911c6cf12da7b215ea8b988b0c8 WatchSource:0}: Error finding container c7e00e9762c4d58919f30bf1f891b54978bfd911c6cf12da7b215ea8b988b0c8: Status 404 returned error can't find the container with id c7e00e9762c4d58919f30bf1f891b54978bfd911c6cf12da7b215ea8b988b0c8 Dec 03 11:24:03 crc kubenswrapper[5052]: I1203 11:24:03.935830 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openshift-controller-manager/controller-manager-85c48b9894-nfp9p"] Dec 03 11:24:03 crc kubenswrapper[5052]: I1203 11:24:03.958252 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openshift-route-controller-manager/route-controller-manager-5966cc58bc-hlf6h"] Dec 03 11:24:04 crc kubenswrapper[5052]: I1203 11:24:04.017546 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-route-controller-manager/route-controller-manager-5966cc58bc-hlf6h" event={"ID":"7489ae0d-ae13-4ae8-9ec0-f1c2c1233562","Type":"ContainerStarted","Data":"a42d091a1560ef9b10734106f8bc3bdff22178b75741c641b8b29a1d8cd6e7df"} Dec 03 11:24:04 crc kubenswrapper[5052]: I1203 11:24:04.032115 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-controller-manager/controller-manager-85c48b9894-nfp9p" event={"ID":"c9317f31-e609-4397-9b9f-dcdce33196ad","Type":"ContainerStarted","Data":"c7e00e9762c4d58919f30bf1f891b54978bfd911c6cf12da7b215ea8b988b0c8"} Dec 03 11:24:05 crc kubenswrapper[5052]: I1203 11:24:05.041235 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-route-controller-manager/route-controller-manager-5966cc58bc-hlf6h" event={"ID":"7489ae0d-ae13-4ae8-9ec0-f1c2c1233562","Type":"ContainerStarted","Data":"63707d96b0610619cff15f854584047cae6ba9db1c3b62b169a6b70758c15693"} Dec 03 11:24:05 crc kubenswrapper[5052]: I1203 11:24:05.042959 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openshift-route-controller-manager/route-controller-manager-5966cc58bc-hlf6h" Dec 03 11:24:05 crc kubenswrapper[5052]: I1203 11:24:05.041406 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openshift-route-controller-manager/route-controller-manager-5966cc58bc-hlf6h" podUID="7489ae0d-ae13-4ae8-9ec0-f1c2c1233562" containerName="route-controller-manager" containerID="cri-o://63707d96b0610619cff15f854584047cae6ba9db1c3b62b169a6b70758c15693" gracePeriod=30 Dec 03 11:24:05 crc kubenswrapper[5052]: I1203 11:24:05.045363 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-controller-manager/controller-manager-85c48b9894-nfp9p" event={"ID":"c9317f31-e609-4397-9b9f-dcdce33196ad","Type":"ContainerStarted","Data":"7b5839439f359f05f1efa0a71d30d9a7033932c9b3e7af34d31e872ba2a88633"} Dec 03 11:24:05 crc kubenswrapper[5052]: I1203 11:24:05.045638 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openshift-controller-manager/controller-manager-85c48b9894-nfp9p" podUID="c9317f31-e609-4397-9b9f-dcdce33196ad" containerName="controller-manager" containerID="cri-o://7b5839439f359f05f1efa0a71d30d9a7033932c9b3e7af34d31e872ba2a88633" gracePeriod=30 Dec 03 11:24:05 crc kubenswrapper[5052]: I1203 11:24:05.045815 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openshift-controller-manager/controller-manager-85c48b9894-nfp9p" Dec 03 11:24:05 crc kubenswrapper[5052]: I1203 11:24:05.053807 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-controller-manager/controller-manager-85c48b9894-nfp9p" Dec 03 11:24:05 crc kubenswrapper[5052]: I1203 11:24:05.063277 5052 patch_prober.go:28] interesting pod/route-controller-manager-5966cc58bc-hlf6h container/route-controller-manager namespace/openshift-route-controller-manager: Readiness probe status=failure output="Get \"https://10.217.0.58:8443/healthz\": read tcp 10.217.0.2:59474->10.217.0.58:8443: read: connection reset by peer" start-of-body= Dec 03 11:24:05 crc kubenswrapper[5052]: I1203 11:24:05.063352 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openshift-route-controller-manager/route-controller-manager-5966cc58bc-hlf6h" podUID="7489ae0d-ae13-4ae8-9ec0-f1c2c1233562" containerName="route-controller-manager" probeResult="failure" output="Get \"https://10.217.0.58:8443/healthz\": read tcp 10.217.0.2:59474->10.217.0.58:8443: read: connection reset by peer" Dec 03 11:24:05 crc kubenswrapper[5052]: I1203 11:24:05.067712 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-route-controller-manager/route-controller-manager-5966cc58bc-hlf6h" podStartSLOduration=4.067670735 podStartE2EDuration="4.067670735s" podCreationTimestamp="2025-12-03 11:24:01 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:24:05.06284421 +0000 UTC m=+235.651085974" watchObservedRunningTime="2025-12-03 11:24:05.067670735 +0000 UTC m=+235.655912499" Dec 03 11:24:05 crc kubenswrapper[5052]: I1203 11:24:05.091193 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-controller-manager/controller-manager-85c48b9894-nfp9p" podStartSLOduration=4.091170574 podStartE2EDuration="4.091170574s" podCreationTimestamp="2025-12-03 11:24:01 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:24:05.087875432 +0000 UTC m=+235.676117216" watchObservedRunningTime="2025-12-03 11:24:05.091170574 +0000 UTC m=+235.679412338" Dec 03 11:24:05 crc kubenswrapper[5052]: I1203 11:24:05.444218 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-route-controller-manager_route-controller-manager-5966cc58bc-hlf6h_7489ae0d-ae13-4ae8-9ec0-f1c2c1233562/route-controller-manager/0.log" Dec 03 11:24:05 crc kubenswrapper[5052]: I1203 11:24:05.444298 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-route-controller-manager/route-controller-manager-5966cc58bc-hlf6h" Dec 03 11:24:05 crc kubenswrapper[5052]: I1203 11:24:05.450013 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-controller-manager/controller-manager-85c48b9894-nfp9p" Dec 03 11:24:05 crc kubenswrapper[5052]: I1203 11:24:05.474014 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openshift-route-controller-manager/route-controller-manager-57cb97bb6b-s82qv"] Dec 03 11:24:05 crc kubenswrapper[5052]: E1203 11:24:05.474342 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="7489ae0d-ae13-4ae8-9ec0-f1c2c1233562" containerName="route-controller-manager" Dec 03 11:24:05 crc kubenswrapper[5052]: I1203 11:24:05.474362 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="7489ae0d-ae13-4ae8-9ec0-f1c2c1233562" containerName="route-controller-manager" Dec 03 11:24:05 crc kubenswrapper[5052]: E1203 11:24:05.474381 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="c9317f31-e609-4397-9b9f-dcdce33196ad" containerName="controller-manager" Dec 03 11:24:05 crc kubenswrapper[5052]: I1203 11:24:05.474393 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="c9317f31-e609-4397-9b9f-dcdce33196ad" containerName="controller-manager" Dec 03 11:24:05 crc kubenswrapper[5052]: I1203 11:24:05.474548 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="c9317f31-e609-4397-9b9f-dcdce33196ad" containerName="controller-manager" Dec 03 11:24:05 crc kubenswrapper[5052]: I1203 11:24:05.474582 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="7489ae0d-ae13-4ae8-9ec0-f1c2c1233562" containerName="route-controller-manager" Dec 03 11:24:05 crc kubenswrapper[5052]: I1203 11:24:05.475179 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-route-controller-manager/route-controller-manager-57cb97bb6b-s82qv" Dec 03 11:24:05 crc kubenswrapper[5052]: I1203 11:24:05.487811 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-route-controller-manager/route-controller-manager-57cb97bb6b-s82qv"] Dec 03 11:24:05 crc kubenswrapper[5052]: I1203 11:24:05.516412 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c9317f31-e609-4397-9b9f-dcdce33196ad-config\") pod \"c9317f31-e609-4397-9b9f-dcdce33196ad\" (UID: \"c9317f31-e609-4397-9b9f-dcdce33196ad\") " Dec 03 11:24:05 crc kubenswrapper[5052]: I1203 11:24:05.516500 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"proxy-ca-bundles\" (UniqueName: \"kubernetes.io/configmap/c9317f31-e609-4397-9b9f-dcdce33196ad-proxy-ca-bundles\") pod \"c9317f31-e609-4397-9b9f-dcdce33196ad\" (UID: \"c9317f31-e609-4397-9b9f-dcdce33196ad\") " Dec 03 11:24:05 crc kubenswrapper[5052]: I1203 11:24:05.516566 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c9317f31-e609-4397-9b9f-dcdce33196ad-serving-cert\") pod \"c9317f31-e609-4397-9b9f-dcdce33196ad\" (UID: \"c9317f31-e609-4397-9b9f-dcdce33196ad\") " Dec 03 11:24:05 crc kubenswrapper[5052]: I1203 11:24:05.516594 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7489ae0d-ae13-4ae8-9ec0-f1c2c1233562-serving-cert\") pod \"7489ae0d-ae13-4ae8-9ec0-f1c2c1233562\" (UID: \"7489ae0d-ae13-4ae8-9ec0-f1c2c1233562\") " Dec 03 11:24:05 crc kubenswrapper[5052]: I1203 11:24:05.517007 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca\" (UniqueName: \"kubernetes.io/configmap/e9823b73-36f5-435f-947d-aa7cd015ff2e-client-ca\") pod \"route-controller-manager-57cb97bb6b-s82qv\" (UID: \"e9823b73-36f5-435f-947d-aa7cd015ff2e\") " pod="openshift-route-controller-manager/route-controller-manager-57cb97bb6b-s82qv" Dec 03 11:24:05 crc kubenswrapper[5052]: I1203 11:24:05.517080 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e9823b73-36f5-435f-947d-aa7cd015ff2e-serving-cert\") pod \"route-controller-manager-57cb97bb6b-s82qv\" (UID: \"e9823b73-36f5-435f-947d-aa7cd015ff2e\") " pod="openshift-route-controller-manager/route-controller-manager-57cb97bb6b-s82qv" Dec 03 11:24:05 crc kubenswrapper[5052]: I1203 11:24:05.517144 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e9823b73-36f5-435f-947d-aa7cd015ff2e-config\") pod \"route-controller-manager-57cb97bb6b-s82qv\" (UID: \"e9823b73-36f5-435f-947d-aa7cd015ff2e\") " pod="openshift-route-controller-manager/route-controller-manager-57cb97bb6b-s82qv" Dec 03 11:24:05 crc kubenswrapper[5052]: I1203 11:24:05.517283 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rsjp2\" (UniqueName: \"kubernetes.io/projected/e9823b73-36f5-435f-947d-aa7cd015ff2e-kube-api-access-rsjp2\") pod \"route-controller-manager-57cb97bb6b-s82qv\" (UID: \"e9823b73-36f5-435f-947d-aa7cd015ff2e\") " pod="openshift-route-controller-manager/route-controller-manager-57cb97bb6b-s82qv" Dec 03 11:24:05 crc kubenswrapper[5052]: I1203 11:24:05.518623 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c9317f31-e609-4397-9b9f-dcdce33196ad-config" (OuterVolumeSpecName: "config") pod "c9317f31-e609-4397-9b9f-dcdce33196ad" (UID: "c9317f31-e609-4397-9b9f-dcdce33196ad"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:24:05 crc kubenswrapper[5052]: I1203 11:24:05.519226 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c9317f31-e609-4397-9b9f-dcdce33196ad-proxy-ca-bundles" (OuterVolumeSpecName: "proxy-ca-bundles") pod "c9317f31-e609-4397-9b9f-dcdce33196ad" (UID: "c9317f31-e609-4397-9b9f-dcdce33196ad"). InnerVolumeSpecName "proxy-ca-bundles". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:24:05 crc kubenswrapper[5052]: I1203 11:24:05.525640 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7489ae0d-ae13-4ae8-9ec0-f1c2c1233562-serving-cert" (OuterVolumeSpecName: "serving-cert") pod "7489ae0d-ae13-4ae8-9ec0-f1c2c1233562" (UID: "7489ae0d-ae13-4ae8-9ec0-f1c2c1233562"). InnerVolumeSpecName "serving-cert". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:24:05 crc kubenswrapper[5052]: I1203 11:24:05.526160 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c9317f31-e609-4397-9b9f-dcdce33196ad-serving-cert" (OuterVolumeSpecName: "serving-cert") pod "c9317f31-e609-4397-9b9f-dcdce33196ad" (UID: "c9317f31-e609-4397-9b9f-dcdce33196ad"). InnerVolumeSpecName "serving-cert". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:24:05 crc kubenswrapper[5052]: I1203 11:24:05.618308 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"client-ca\" (UniqueName: \"kubernetes.io/configmap/7489ae0d-ae13-4ae8-9ec0-f1c2c1233562-client-ca\") pod \"7489ae0d-ae13-4ae8-9ec0-f1c2c1233562\" (UID: \"7489ae0d-ae13-4ae8-9ec0-f1c2c1233562\") " Dec 03 11:24:05 crc kubenswrapper[5052]: I1203 11:24:05.618384 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bgdb6\" (UniqueName: \"kubernetes.io/projected/7489ae0d-ae13-4ae8-9ec0-f1c2c1233562-kube-api-access-bgdb6\") pod \"7489ae0d-ae13-4ae8-9ec0-f1c2c1233562\" (UID: \"7489ae0d-ae13-4ae8-9ec0-f1c2c1233562\") " Dec 03 11:24:05 crc kubenswrapper[5052]: I1203 11:24:05.618465 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"client-ca\" (UniqueName: \"kubernetes.io/configmap/c9317f31-e609-4397-9b9f-dcdce33196ad-client-ca\") pod \"c9317f31-e609-4397-9b9f-dcdce33196ad\" (UID: \"c9317f31-e609-4397-9b9f-dcdce33196ad\") " Dec 03 11:24:05 crc kubenswrapper[5052]: I1203 11:24:05.618491 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/7489ae0d-ae13-4ae8-9ec0-f1c2c1233562-config\") pod \"7489ae0d-ae13-4ae8-9ec0-f1c2c1233562\" (UID: \"7489ae0d-ae13-4ae8-9ec0-f1c2c1233562\") " Dec 03 11:24:05 crc kubenswrapper[5052]: I1203 11:24:05.618532 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-852nw\" (UniqueName: \"kubernetes.io/projected/c9317f31-e609-4397-9b9f-dcdce33196ad-kube-api-access-852nw\") pod \"c9317f31-e609-4397-9b9f-dcdce33196ad\" (UID: \"c9317f31-e609-4397-9b9f-dcdce33196ad\") " Dec 03 11:24:05 crc kubenswrapper[5052]: I1203 11:24:05.618677 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-rsjp2\" (UniqueName: \"kubernetes.io/projected/e9823b73-36f5-435f-947d-aa7cd015ff2e-kube-api-access-rsjp2\") pod \"route-controller-manager-57cb97bb6b-s82qv\" (UID: \"e9823b73-36f5-435f-947d-aa7cd015ff2e\") " pod="openshift-route-controller-manager/route-controller-manager-57cb97bb6b-s82qv" Dec 03 11:24:05 crc kubenswrapper[5052]: I1203 11:24:05.618723 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"client-ca\" (UniqueName: \"kubernetes.io/configmap/e9823b73-36f5-435f-947d-aa7cd015ff2e-client-ca\") pod \"route-controller-manager-57cb97bb6b-s82qv\" (UID: \"e9823b73-36f5-435f-947d-aa7cd015ff2e\") " pod="openshift-route-controller-manager/route-controller-manager-57cb97bb6b-s82qv" Dec 03 11:24:05 crc kubenswrapper[5052]: I1203 11:24:05.618758 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e9823b73-36f5-435f-947d-aa7cd015ff2e-serving-cert\") pod \"route-controller-manager-57cb97bb6b-s82qv\" (UID: \"e9823b73-36f5-435f-947d-aa7cd015ff2e\") " pod="openshift-route-controller-manager/route-controller-manager-57cb97bb6b-s82qv" Dec 03 11:24:05 crc kubenswrapper[5052]: I1203 11:24:05.618795 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e9823b73-36f5-435f-947d-aa7cd015ff2e-config\") pod \"route-controller-manager-57cb97bb6b-s82qv\" (UID: \"e9823b73-36f5-435f-947d-aa7cd015ff2e\") " pod="openshift-route-controller-manager/route-controller-manager-57cb97bb6b-s82qv" Dec 03 11:24:05 crc kubenswrapper[5052]: I1203 11:24:05.618839 5052 reconciler_common.go:293] "Volume detached for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c9317f31-e609-4397-9b9f-dcdce33196ad-serving-cert\") on node \"crc\" DevicePath \"\"" Dec 03 11:24:05 crc kubenswrapper[5052]: I1203 11:24:05.618852 5052 reconciler_common.go:293] "Volume detached for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7489ae0d-ae13-4ae8-9ec0-f1c2c1233562-serving-cert\") on node \"crc\" DevicePath \"\"" Dec 03 11:24:05 crc kubenswrapper[5052]: I1203 11:24:05.618867 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c9317f31-e609-4397-9b9f-dcdce33196ad-config\") on node \"crc\" DevicePath \"\"" Dec 03 11:24:05 crc kubenswrapper[5052]: I1203 11:24:05.618879 5052 reconciler_common.go:293] "Volume detached for volume \"proxy-ca-bundles\" (UniqueName: \"kubernetes.io/configmap/c9317f31-e609-4397-9b9f-dcdce33196ad-proxy-ca-bundles\") on node \"crc\" DevicePath \"\"" Dec 03 11:24:05 crc kubenswrapper[5052]: I1203 11:24:05.619176 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c9317f31-e609-4397-9b9f-dcdce33196ad-client-ca" (OuterVolumeSpecName: "client-ca") pod "c9317f31-e609-4397-9b9f-dcdce33196ad" (UID: "c9317f31-e609-4397-9b9f-dcdce33196ad"). InnerVolumeSpecName "client-ca". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:24:05 crc kubenswrapper[5052]: I1203 11:24:05.619259 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7489ae0d-ae13-4ae8-9ec0-f1c2c1233562-client-ca" (OuterVolumeSpecName: "client-ca") pod "7489ae0d-ae13-4ae8-9ec0-f1c2c1233562" (UID: "7489ae0d-ae13-4ae8-9ec0-f1c2c1233562"). InnerVolumeSpecName "client-ca". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:24:05 crc kubenswrapper[5052]: I1203 11:24:05.619435 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7489ae0d-ae13-4ae8-9ec0-f1c2c1233562-config" (OuterVolumeSpecName: "config") pod "7489ae0d-ae13-4ae8-9ec0-f1c2c1233562" (UID: "7489ae0d-ae13-4ae8-9ec0-f1c2c1233562"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:24:05 crc kubenswrapper[5052]: I1203 11:24:05.620261 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"client-ca\" (UniqueName: \"kubernetes.io/configmap/e9823b73-36f5-435f-947d-aa7cd015ff2e-client-ca\") pod \"route-controller-manager-57cb97bb6b-s82qv\" (UID: \"e9823b73-36f5-435f-947d-aa7cd015ff2e\") " pod="openshift-route-controller-manager/route-controller-manager-57cb97bb6b-s82qv" Dec 03 11:24:05 crc kubenswrapper[5052]: I1203 11:24:05.620414 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e9823b73-36f5-435f-947d-aa7cd015ff2e-config\") pod \"route-controller-manager-57cb97bb6b-s82qv\" (UID: \"e9823b73-36f5-435f-947d-aa7cd015ff2e\") " pod="openshift-route-controller-manager/route-controller-manager-57cb97bb6b-s82qv" Dec 03 11:24:05 crc kubenswrapper[5052]: I1203 11:24:05.621952 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7489ae0d-ae13-4ae8-9ec0-f1c2c1233562-kube-api-access-bgdb6" (OuterVolumeSpecName: "kube-api-access-bgdb6") pod "7489ae0d-ae13-4ae8-9ec0-f1c2c1233562" (UID: "7489ae0d-ae13-4ae8-9ec0-f1c2c1233562"). InnerVolumeSpecName "kube-api-access-bgdb6". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:24:05 crc kubenswrapper[5052]: I1203 11:24:05.622410 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c9317f31-e609-4397-9b9f-dcdce33196ad-kube-api-access-852nw" (OuterVolumeSpecName: "kube-api-access-852nw") pod "c9317f31-e609-4397-9b9f-dcdce33196ad" (UID: "c9317f31-e609-4397-9b9f-dcdce33196ad"). InnerVolumeSpecName "kube-api-access-852nw". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:24:05 crc kubenswrapper[5052]: I1203 11:24:05.623218 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e9823b73-36f5-435f-947d-aa7cd015ff2e-serving-cert\") pod \"route-controller-manager-57cb97bb6b-s82qv\" (UID: \"e9823b73-36f5-435f-947d-aa7cd015ff2e\") " pod="openshift-route-controller-manager/route-controller-manager-57cb97bb6b-s82qv" Dec 03 11:24:05 crc kubenswrapper[5052]: I1203 11:24:05.638180 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-rsjp2\" (UniqueName: \"kubernetes.io/projected/e9823b73-36f5-435f-947d-aa7cd015ff2e-kube-api-access-rsjp2\") pod \"route-controller-manager-57cb97bb6b-s82qv\" (UID: \"e9823b73-36f5-435f-947d-aa7cd015ff2e\") " pod="openshift-route-controller-manager/route-controller-manager-57cb97bb6b-s82qv" Dec 03 11:24:05 crc kubenswrapper[5052]: I1203 11:24:05.720077 5052 reconciler_common.go:293] "Volume detached for volume \"client-ca\" (UniqueName: \"kubernetes.io/configmap/7489ae0d-ae13-4ae8-9ec0-f1c2c1233562-client-ca\") on node \"crc\" DevicePath \"\"" Dec 03 11:24:05 crc kubenswrapper[5052]: I1203 11:24:05.720116 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-bgdb6\" (UniqueName: \"kubernetes.io/projected/7489ae0d-ae13-4ae8-9ec0-f1c2c1233562-kube-api-access-bgdb6\") on node \"crc\" DevicePath \"\"" Dec 03 11:24:05 crc kubenswrapper[5052]: I1203 11:24:05.720126 5052 reconciler_common.go:293] "Volume detached for volume \"client-ca\" (UniqueName: \"kubernetes.io/configmap/c9317f31-e609-4397-9b9f-dcdce33196ad-client-ca\") on node \"crc\" DevicePath \"\"" Dec 03 11:24:05 crc kubenswrapper[5052]: I1203 11:24:05.720135 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/7489ae0d-ae13-4ae8-9ec0-f1c2c1233562-config\") on node \"crc\" DevicePath \"\"" Dec 03 11:24:05 crc kubenswrapper[5052]: I1203 11:24:05.720177 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-852nw\" (UniqueName: \"kubernetes.io/projected/c9317f31-e609-4397-9b9f-dcdce33196ad-kube-api-access-852nw\") on node \"crc\" DevicePath \"\"" Dec 03 11:24:05 crc kubenswrapper[5052]: I1203 11:24:05.796449 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-route-controller-manager/route-controller-manager-57cb97bb6b-s82qv" Dec 03 11:24:06 crc kubenswrapper[5052]: I1203 11:24:06.030019 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-route-controller-manager/route-controller-manager-57cb97bb6b-s82qv"] Dec 03 11:24:06 crc kubenswrapper[5052]: I1203 11:24:06.064730 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-route-controller-manager_route-controller-manager-5966cc58bc-hlf6h_7489ae0d-ae13-4ae8-9ec0-f1c2c1233562/route-controller-manager/0.log" Dec 03 11:24:06 crc kubenswrapper[5052]: I1203 11:24:06.065147 5052 generic.go:334] "Generic (PLEG): container finished" podID="7489ae0d-ae13-4ae8-9ec0-f1c2c1233562" containerID="63707d96b0610619cff15f854584047cae6ba9db1c3b62b169a6b70758c15693" exitCode=255 Dec 03 11:24:06 crc kubenswrapper[5052]: I1203 11:24:06.065269 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-route-controller-manager/route-controller-manager-5966cc58bc-hlf6h" Dec 03 11:24:06 crc kubenswrapper[5052]: I1203 11:24:06.065314 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-route-controller-manager/route-controller-manager-5966cc58bc-hlf6h" event={"ID":"7489ae0d-ae13-4ae8-9ec0-f1c2c1233562","Type":"ContainerDied","Data":"63707d96b0610619cff15f854584047cae6ba9db1c3b62b169a6b70758c15693"} Dec 03 11:24:06 crc kubenswrapper[5052]: I1203 11:24:06.065429 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-route-controller-manager/route-controller-manager-5966cc58bc-hlf6h" event={"ID":"7489ae0d-ae13-4ae8-9ec0-f1c2c1233562","Type":"ContainerDied","Data":"a42d091a1560ef9b10734106f8bc3bdff22178b75741c641b8b29a1d8cd6e7df"} Dec 03 11:24:06 crc kubenswrapper[5052]: I1203 11:24:06.065469 5052 scope.go:117] "RemoveContainer" containerID="63707d96b0610619cff15f854584047cae6ba9db1c3b62b169a6b70758c15693" Dec 03 11:24:06 crc kubenswrapper[5052]: I1203 11:24:06.067703 5052 generic.go:334] "Generic (PLEG): container finished" podID="c9317f31-e609-4397-9b9f-dcdce33196ad" containerID="7b5839439f359f05f1efa0a71d30d9a7033932c9b3e7af34d31e872ba2a88633" exitCode=0 Dec 03 11:24:06 crc kubenswrapper[5052]: I1203 11:24:06.067860 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-controller-manager/controller-manager-85c48b9894-nfp9p" Dec 03 11:24:06 crc kubenswrapper[5052]: I1203 11:24:06.068724 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-controller-manager/controller-manager-85c48b9894-nfp9p" event={"ID":"c9317f31-e609-4397-9b9f-dcdce33196ad","Type":"ContainerDied","Data":"7b5839439f359f05f1efa0a71d30d9a7033932c9b3e7af34d31e872ba2a88633"} Dec 03 11:24:06 crc kubenswrapper[5052]: I1203 11:24:06.068760 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-controller-manager/controller-manager-85c48b9894-nfp9p" event={"ID":"c9317f31-e609-4397-9b9f-dcdce33196ad","Type":"ContainerDied","Data":"c7e00e9762c4d58919f30bf1f891b54978bfd911c6cf12da7b215ea8b988b0c8"} Dec 03 11:24:06 crc kubenswrapper[5052]: I1203 11:24:06.070802 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-route-controller-manager/route-controller-manager-57cb97bb6b-s82qv" event={"ID":"e9823b73-36f5-435f-947d-aa7cd015ff2e","Type":"ContainerStarted","Data":"3c28785c46abd1c3a58f9fff0c628eb121141bc4d77ff8888ad76190df653f3f"} Dec 03 11:24:06 crc kubenswrapper[5052]: I1203 11:24:06.090228 5052 scope.go:117] "RemoveContainer" containerID="63707d96b0610619cff15f854584047cae6ba9db1c3b62b169a6b70758c15693" Dec 03 11:24:06 crc kubenswrapper[5052]: E1203 11:24:06.091606 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"63707d96b0610619cff15f854584047cae6ba9db1c3b62b169a6b70758c15693\": container with ID starting with 63707d96b0610619cff15f854584047cae6ba9db1c3b62b169a6b70758c15693 not found: ID does not exist" containerID="63707d96b0610619cff15f854584047cae6ba9db1c3b62b169a6b70758c15693" Dec 03 11:24:06 crc kubenswrapper[5052]: I1203 11:24:06.091661 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"63707d96b0610619cff15f854584047cae6ba9db1c3b62b169a6b70758c15693"} err="failed to get container status \"63707d96b0610619cff15f854584047cae6ba9db1c3b62b169a6b70758c15693\": rpc error: code = NotFound desc = could not find container \"63707d96b0610619cff15f854584047cae6ba9db1c3b62b169a6b70758c15693\": container with ID starting with 63707d96b0610619cff15f854584047cae6ba9db1c3b62b169a6b70758c15693 not found: ID does not exist" Dec 03 11:24:06 crc kubenswrapper[5052]: I1203 11:24:06.091700 5052 scope.go:117] "RemoveContainer" containerID="7b5839439f359f05f1efa0a71d30d9a7033932c9b3e7af34d31e872ba2a88633" Dec 03 11:24:06 crc kubenswrapper[5052]: I1203 11:24:06.096390 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openshift-route-controller-manager/route-controller-manager-5966cc58bc-hlf6h"] Dec 03 11:24:06 crc kubenswrapper[5052]: I1203 11:24:06.101292 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openshift-route-controller-manager/route-controller-manager-5966cc58bc-hlf6h"] Dec 03 11:24:06 crc kubenswrapper[5052]: I1203 11:24:06.115104 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openshift-controller-manager/controller-manager-85c48b9894-nfp9p"] Dec 03 11:24:06 crc kubenswrapper[5052]: I1203 11:24:06.118405 5052 scope.go:117] "RemoveContainer" containerID="7b5839439f359f05f1efa0a71d30d9a7033932c9b3e7af34d31e872ba2a88633" Dec 03 11:24:06 crc kubenswrapper[5052]: I1203 11:24:06.118781 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openshift-controller-manager/controller-manager-85c48b9894-nfp9p"] Dec 03 11:24:06 crc kubenswrapper[5052]: E1203 11:24:06.119118 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7b5839439f359f05f1efa0a71d30d9a7033932c9b3e7af34d31e872ba2a88633\": container with ID starting with 7b5839439f359f05f1efa0a71d30d9a7033932c9b3e7af34d31e872ba2a88633 not found: ID does not exist" containerID="7b5839439f359f05f1efa0a71d30d9a7033932c9b3e7af34d31e872ba2a88633" Dec 03 11:24:06 crc kubenswrapper[5052]: I1203 11:24:06.119168 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7b5839439f359f05f1efa0a71d30d9a7033932c9b3e7af34d31e872ba2a88633"} err="failed to get container status \"7b5839439f359f05f1efa0a71d30d9a7033932c9b3e7af34d31e872ba2a88633\": rpc error: code = NotFound desc = could not find container \"7b5839439f359f05f1efa0a71d30d9a7033932c9b3e7af34d31e872ba2a88633\": container with ID starting with 7b5839439f359f05f1efa0a71d30d9a7033932c9b3e7af34d31e872ba2a88633 not found: ID does not exist" Dec 03 11:24:07 crc kubenswrapper[5052]: I1203 11:24:07.082426 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-route-controller-manager/route-controller-manager-57cb97bb6b-s82qv" event={"ID":"e9823b73-36f5-435f-947d-aa7cd015ff2e","Type":"ContainerStarted","Data":"7f3ba9d5a3bafceff4079c57523368508c99be5d94f9663ff24e5e742c9ad96c"} Dec 03 11:24:07 crc kubenswrapper[5052]: I1203 11:24:07.082924 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openshift-route-controller-manager/route-controller-manager-57cb97bb6b-s82qv" Dec 03 11:24:07 crc kubenswrapper[5052]: I1203 11:24:07.089760 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-route-controller-manager/route-controller-manager-57cb97bb6b-s82qv" Dec 03 11:24:07 crc kubenswrapper[5052]: I1203 11:24:07.106032 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-route-controller-manager/route-controller-manager-57cb97bb6b-s82qv" podStartSLOduration=4.106009201 podStartE2EDuration="4.106009201s" podCreationTimestamp="2025-12-03 11:24:03 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:24:07.10599259 +0000 UTC m=+237.694234354" watchObservedRunningTime="2025-12-03 11:24:07.106009201 +0000 UTC m=+237.694250965" Dec 03 11:24:07 crc kubenswrapper[5052]: I1203 11:24:07.681659 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7489ae0d-ae13-4ae8-9ec0-f1c2c1233562" path="/var/lib/kubelet/pods/7489ae0d-ae13-4ae8-9ec0-f1c2c1233562/volumes" Dec 03 11:24:07 crc kubenswrapper[5052]: I1203 11:24:07.682795 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c9317f31-e609-4397-9b9f-dcdce33196ad" path="/var/lib/kubelet/pods/c9317f31-e609-4397-9b9f-dcdce33196ad/volumes" Dec 03 11:24:08 crc kubenswrapper[5052]: I1203 11:24:08.294772 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openshift-controller-manager/controller-manager-7bd8d5968-hddbl"] Dec 03 11:24:08 crc kubenswrapper[5052]: I1203 11:24:08.295673 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-controller-manager/controller-manager-7bd8d5968-hddbl" Dec 03 11:24:08 crc kubenswrapper[5052]: I1203 11:24:08.298742 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-controller-manager"/"openshift-service-ca.crt" Dec 03 11:24:08 crc kubenswrapper[5052]: I1203 11:24:08.299212 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-controller-manager"/"openshift-controller-manager-sa-dockercfg-msq4c" Dec 03 11:24:08 crc kubenswrapper[5052]: I1203 11:24:08.299495 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-controller-manager"/"kube-root-ca.crt" Dec 03 11:24:08 crc kubenswrapper[5052]: I1203 11:24:08.299267 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-controller-manager"/"config" Dec 03 11:24:08 crc kubenswrapper[5052]: I1203 11:24:08.299298 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-controller-manager"/"serving-cert" Dec 03 11:24:08 crc kubenswrapper[5052]: I1203 11:24:08.301870 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-controller-manager"/"client-ca" Dec 03 11:24:08 crc kubenswrapper[5052]: I1203 11:24:08.306355 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-controller-manager"/"openshift-global-ca" Dec 03 11:24:08 crc kubenswrapper[5052]: I1203 11:24:08.317213 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-controller-manager/controller-manager-7bd8d5968-hddbl"] Dec 03 11:24:08 crc kubenswrapper[5052]: I1203 11:24:08.372104 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-ca-bundles\" (UniqueName: \"kubernetes.io/configmap/94a23d8f-e983-48c4-8ad4-37404a325973-proxy-ca-bundles\") pod \"controller-manager-7bd8d5968-hddbl\" (UID: \"94a23d8f-e983-48c4-8ad4-37404a325973\") " pod="openshift-controller-manager/controller-manager-7bd8d5968-hddbl" Dec 03 11:24:08 crc kubenswrapper[5052]: I1203 11:24:08.372176 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/94a23d8f-e983-48c4-8ad4-37404a325973-serving-cert\") pod \"controller-manager-7bd8d5968-hddbl\" (UID: \"94a23d8f-e983-48c4-8ad4-37404a325973\") " pod="openshift-controller-manager/controller-manager-7bd8d5968-hddbl" Dec 03 11:24:08 crc kubenswrapper[5052]: I1203 11:24:08.372265 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l75pp\" (UniqueName: \"kubernetes.io/projected/94a23d8f-e983-48c4-8ad4-37404a325973-kube-api-access-l75pp\") pod \"controller-manager-7bd8d5968-hddbl\" (UID: \"94a23d8f-e983-48c4-8ad4-37404a325973\") " pod="openshift-controller-manager/controller-manager-7bd8d5968-hddbl" Dec 03 11:24:08 crc kubenswrapper[5052]: I1203 11:24:08.372335 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca\" (UniqueName: \"kubernetes.io/configmap/94a23d8f-e983-48c4-8ad4-37404a325973-client-ca\") pod \"controller-manager-7bd8d5968-hddbl\" (UID: \"94a23d8f-e983-48c4-8ad4-37404a325973\") " pod="openshift-controller-manager/controller-manager-7bd8d5968-hddbl" Dec 03 11:24:08 crc kubenswrapper[5052]: I1203 11:24:08.372414 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/94a23d8f-e983-48c4-8ad4-37404a325973-config\") pod \"controller-manager-7bd8d5968-hddbl\" (UID: \"94a23d8f-e983-48c4-8ad4-37404a325973\") " pod="openshift-controller-manager/controller-manager-7bd8d5968-hddbl" Dec 03 11:24:08 crc kubenswrapper[5052]: I1203 11:24:08.473316 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-l75pp\" (UniqueName: \"kubernetes.io/projected/94a23d8f-e983-48c4-8ad4-37404a325973-kube-api-access-l75pp\") pod \"controller-manager-7bd8d5968-hddbl\" (UID: \"94a23d8f-e983-48c4-8ad4-37404a325973\") " pod="openshift-controller-manager/controller-manager-7bd8d5968-hddbl" Dec 03 11:24:08 crc kubenswrapper[5052]: I1203 11:24:08.473422 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"client-ca\" (UniqueName: \"kubernetes.io/configmap/94a23d8f-e983-48c4-8ad4-37404a325973-client-ca\") pod \"controller-manager-7bd8d5968-hddbl\" (UID: \"94a23d8f-e983-48c4-8ad4-37404a325973\") " pod="openshift-controller-manager/controller-manager-7bd8d5968-hddbl" Dec 03 11:24:08 crc kubenswrapper[5052]: I1203 11:24:08.473466 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/94a23d8f-e983-48c4-8ad4-37404a325973-config\") pod \"controller-manager-7bd8d5968-hddbl\" (UID: \"94a23d8f-e983-48c4-8ad4-37404a325973\") " pod="openshift-controller-manager/controller-manager-7bd8d5968-hddbl" Dec 03 11:24:08 crc kubenswrapper[5052]: I1203 11:24:08.473548 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"proxy-ca-bundles\" (UniqueName: \"kubernetes.io/configmap/94a23d8f-e983-48c4-8ad4-37404a325973-proxy-ca-bundles\") pod \"controller-manager-7bd8d5968-hddbl\" (UID: \"94a23d8f-e983-48c4-8ad4-37404a325973\") " pod="openshift-controller-manager/controller-manager-7bd8d5968-hddbl" Dec 03 11:24:08 crc kubenswrapper[5052]: I1203 11:24:08.474263 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/94a23d8f-e983-48c4-8ad4-37404a325973-serving-cert\") pod \"controller-manager-7bd8d5968-hddbl\" (UID: \"94a23d8f-e983-48c4-8ad4-37404a325973\") " pod="openshift-controller-manager/controller-manager-7bd8d5968-hddbl" Dec 03 11:24:08 crc kubenswrapper[5052]: I1203 11:24:08.475249 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"proxy-ca-bundles\" (UniqueName: \"kubernetes.io/configmap/94a23d8f-e983-48c4-8ad4-37404a325973-proxy-ca-bundles\") pod \"controller-manager-7bd8d5968-hddbl\" (UID: \"94a23d8f-e983-48c4-8ad4-37404a325973\") " pod="openshift-controller-manager/controller-manager-7bd8d5968-hddbl" Dec 03 11:24:08 crc kubenswrapper[5052]: I1203 11:24:08.475290 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"client-ca\" (UniqueName: \"kubernetes.io/configmap/94a23d8f-e983-48c4-8ad4-37404a325973-client-ca\") pod \"controller-manager-7bd8d5968-hddbl\" (UID: \"94a23d8f-e983-48c4-8ad4-37404a325973\") " pod="openshift-controller-manager/controller-manager-7bd8d5968-hddbl" Dec 03 11:24:08 crc kubenswrapper[5052]: I1203 11:24:08.475428 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/94a23d8f-e983-48c4-8ad4-37404a325973-config\") pod \"controller-manager-7bd8d5968-hddbl\" (UID: \"94a23d8f-e983-48c4-8ad4-37404a325973\") " pod="openshift-controller-manager/controller-manager-7bd8d5968-hddbl" Dec 03 11:24:08 crc kubenswrapper[5052]: I1203 11:24:08.481524 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/94a23d8f-e983-48c4-8ad4-37404a325973-serving-cert\") pod \"controller-manager-7bd8d5968-hddbl\" (UID: \"94a23d8f-e983-48c4-8ad4-37404a325973\") " pod="openshift-controller-manager/controller-manager-7bd8d5968-hddbl" Dec 03 11:24:08 crc kubenswrapper[5052]: I1203 11:24:08.498586 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-l75pp\" (UniqueName: \"kubernetes.io/projected/94a23d8f-e983-48c4-8ad4-37404a325973-kube-api-access-l75pp\") pod \"controller-manager-7bd8d5968-hddbl\" (UID: \"94a23d8f-e983-48c4-8ad4-37404a325973\") " pod="openshift-controller-manager/controller-manager-7bd8d5968-hddbl" Dec 03 11:24:08 crc kubenswrapper[5052]: I1203 11:24:08.616012 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-controller-manager/controller-manager-7bd8d5968-hddbl" Dec 03 11:24:08 crc kubenswrapper[5052]: I1203 11:24:08.865409 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-controller-manager/controller-manager-7bd8d5968-hddbl"] Dec 03 11:24:08 crc kubenswrapper[5052]: W1203 11:24:08.880893 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod94a23d8f_e983_48c4_8ad4_37404a325973.slice/crio-5e8849bc370c802fe0f8131f2ea52d5a43bc5df8738236c7ef6c115b04b55574 WatchSource:0}: Error finding container 5e8849bc370c802fe0f8131f2ea52d5a43bc5df8738236c7ef6c115b04b55574: Status 404 returned error can't find the container with id 5e8849bc370c802fe0f8131f2ea52d5a43bc5df8738236c7ef6c115b04b55574 Dec 03 11:24:09 crc kubenswrapper[5052]: I1203 11:24:09.095665 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-controller-manager/controller-manager-7bd8d5968-hddbl" event={"ID":"94a23d8f-e983-48c4-8ad4-37404a325973","Type":"ContainerStarted","Data":"06be2622fe2fd858b7a245a2b20ce8ef132f8e1b4ef5bdc4c125056c471e3a96"} Dec 03 11:24:09 crc kubenswrapper[5052]: I1203 11:24:09.095730 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-controller-manager/controller-manager-7bd8d5968-hddbl" event={"ID":"94a23d8f-e983-48c4-8ad4-37404a325973","Type":"ContainerStarted","Data":"5e8849bc370c802fe0f8131f2ea52d5a43bc5df8738236c7ef6c115b04b55574"} Dec 03 11:24:09 crc kubenswrapper[5052]: I1203 11:24:09.125071 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-controller-manager/controller-manager-7bd8d5968-hddbl" podStartSLOduration=6.125034615 podStartE2EDuration="6.125034615s" podCreationTimestamp="2025-12-03 11:24:03 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:24:09.119161261 +0000 UTC m=+239.707403025" watchObservedRunningTime="2025-12-03 11:24:09.125034615 +0000 UTC m=+239.713276379" Dec 03 11:24:10 crc kubenswrapper[5052]: I1203 11:24:10.101433 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openshift-controller-manager/controller-manager-7bd8d5968-hddbl" Dec 03 11:24:10 crc kubenswrapper[5052]: I1203 11:24:10.107840 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-controller-manager/controller-manager-7bd8d5968-hddbl" Dec 03 11:24:16 crc kubenswrapper[5052]: I1203 11:24:16.819227 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openshift-controller-manager/controller-manager-7bd8d5968-hddbl"] Dec 03 11:24:16 crc kubenswrapper[5052]: I1203 11:24:16.820134 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openshift-controller-manager/controller-manager-7bd8d5968-hddbl" podUID="94a23d8f-e983-48c4-8ad4-37404a325973" containerName="controller-manager" containerID="cri-o://06be2622fe2fd858b7a245a2b20ce8ef132f8e1b4ef5bdc4c125056c471e3a96" gracePeriod=30 Dec 03 11:24:18 crc kubenswrapper[5052]: I1203 11:24:18.145109 5052 generic.go:334] "Generic (PLEG): container finished" podID="94a23d8f-e983-48c4-8ad4-37404a325973" containerID="06be2622fe2fd858b7a245a2b20ce8ef132f8e1b4ef5bdc4c125056c471e3a96" exitCode=0 Dec 03 11:24:18 crc kubenswrapper[5052]: I1203 11:24:18.145206 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-controller-manager/controller-manager-7bd8d5968-hddbl" event={"ID":"94a23d8f-e983-48c4-8ad4-37404a325973","Type":"ContainerDied","Data":"06be2622fe2fd858b7a245a2b20ce8ef132f8e1b4ef5bdc4c125056c471e3a96"} Dec 03 11:24:18 crc kubenswrapper[5052]: I1203 11:24:18.617582 5052 patch_prober.go:28] interesting pod/controller-manager-7bd8d5968-hddbl container/controller-manager namespace/openshift-controller-manager: Readiness probe status=failure output="Get \"https://10.217.0.61:8443/healthz\": dial tcp 10.217.0.61:8443: connect: connection refused" start-of-body= Dec 03 11:24:18 crc kubenswrapper[5052]: I1203 11:24:18.617963 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openshift-controller-manager/controller-manager-7bd8d5968-hddbl" podUID="94a23d8f-e983-48c4-8ad4-37404a325973" containerName="controller-manager" probeResult="failure" output="Get \"https://10.217.0.61:8443/healthz\": dial tcp 10.217.0.61:8443: connect: connection refused" Dec 03 11:24:19 crc kubenswrapper[5052]: I1203 11:24:19.154089 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-controller-manager/controller-manager-7bd8d5968-hddbl" event={"ID":"94a23d8f-e983-48c4-8ad4-37404a325973","Type":"ContainerDied","Data":"5e8849bc370c802fe0f8131f2ea52d5a43bc5df8738236c7ef6c115b04b55574"} Dec 03 11:24:19 crc kubenswrapper[5052]: I1203 11:24:19.154157 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5e8849bc370c802fe0f8131f2ea52d5a43bc5df8738236c7ef6c115b04b55574" Dec 03 11:24:19 crc kubenswrapper[5052]: I1203 11:24:19.173187 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-controller-manager/controller-manager-7bd8d5968-hddbl" Dec 03 11:24:19 crc kubenswrapper[5052]: I1203 11:24:19.204240 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openshift-controller-manager/controller-manager-546b59f455-zx647"] Dec 03 11:24:19 crc kubenswrapper[5052]: E1203 11:24:19.204768 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="94a23d8f-e983-48c4-8ad4-37404a325973" containerName="controller-manager" Dec 03 11:24:19 crc kubenswrapper[5052]: I1203 11:24:19.204840 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="94a23d8f-e983-48c4-8ad4-37404a325973" containerName="controller-manager" Dec 03 11:24:19 crc kubenswrapper[5052]: I1203 11:24:19.205082 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="94a23d8f-e983-48c4-8ad4-37404a325973" containerName="controller-manager" Dec 03 11:24:19 crc kubenswrapper[5052]: I1203 11:24:19.205679 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-controller-manager/controller-manager-546b59f455-zx647" Dec 03 11:24:19 crc kubenswrapper[5052]: I1203 11:24:19.215290 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-controller-manager/controller-manager-546b59f455-zx647"] Dec 03 11:24:19 crc kubenswrapper[5052]: I1203 11:24:19.343736 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/94a23d8f-e983-48c4-8ad4-37404a325973-serving-cert\") pod \"94a23d8f-e983-48c4-8ad4-37404a325973\" (UID: \"94a23d8f-e983-48c4-8ad4-37404a325973\") " Dec 03 11:24:19 crc kubenswrapper[5052]: I1203 11:24:19.343806 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/94a23d8f-e983-48c4-8ad4-37404a325973-config\") pod \"94a23d8f-e983-48c4-8ad4-37404a325973\" (UID: \"94a23d8f-e983-48c4-8ad4-37404a325973\") " Dec 03 11:24:19 crc kubenswrapper[5052]: I1203 11:24:19.343889 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l75pp\" (UniqueName: \"kubernetes.io/projected/94a23d8f-e983-48c4-8ad4-37404a325973-kube-api-access-l75pp\") pod \"94a23d8f-e983-48c4-8ad4-37404a325973\" (UID: \"94a23d8f-e983-48c4-8ad4-37404a325973\") " Dec 03 11:24:19 crc kubenswrapper[5052]: I1203 11:24:19.343922 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"proxy-ca-bundles\" (UniqueName: \"kubernetes.io/configmap/94a23d8f-e983-48c4-8ad4-37404a325973-proxy-ca-bundles\") pod \"94a23d8f-e983-48c4-8ad4-37404a325973\" (UID: \"94a23d8f-e983-48c4-8ad4-37404a325973\") " Dec 03 11:24:19 crc kubenswrapper[5052]: I1203 11:24:19.343967 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"client-ca\" (UniqueName: \"kubernetes.io/configmap/94a23d8f-e983-48c4-8ad4-37404a325973-client-ca\") pod \"94a23d8f-e983-48c4-8ad4-37404a325973\" (UID: \"94a23d8f-e983-48c4-8ad4-37404a325973\") " Dec 03 11:24:19 crc kubenswrapper[5052]: I1203 11:24:19.344197 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca\" (UniqueName: \"kubernetes.io/configmap/fc655546-130a-49a7-90d6-b76d8e033028-client-ca\") pod \"controller-manager-546b59f455-zx647\" (UID: \"fc655546-130a-49a7-90d6-b76d8e033028\") " pod="openshift-controller-manager/controller-manager-546b59f455-zx647" Dec 03 11:24:19 crc kubenswrapper[5052]: I1203 11:24:19.344231 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/fc655546-130a-49a7-90d6-b76d8e033028-serving-cert\") pod \"controller-manager-546b59f455-zx647\" (UID: \"fc655546-130a-49a7-90d6-b76d8e033028\") " pod="openshift-controller-manager/controller-manager-546b59f455-zx647" Dec 03 11:24:19 crc kubenswrapper[5052]: I1203 11:24:19.344258 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-ca-bundles\" (UniqueName: \"kubernetes.io/configmap/fc655546-130a-49a7-90d6-b76d8e033028-proxy-ca-bundles\") pod \"controller-manager-546b59f455-zx647\" (UID: \"fc655546-130a-49a7-90d6-b76d8e033028\") " pod="openshift-controller-manager/controller-manager-546b59f455-zx647" Dec 03 11:24:19 crc kubenswrapper[5052]: I1203 11:24:19.344309 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/fc655546-130a-49a7-90d6-b76d8e033028-config\") pod \"controller-manager-546b59f455-zx647\" (UID: \"fc655546-130a-49a7-90d6-b76d8e033028\") " pod="openshift-controller-manager/controller-manager-546b59f455-zx647" Dec 03 11:24:19 crc kubenswrapper[5052]: I1203 11:24:19.344350 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bdskz\" (UniqueName: \"kubernetes.io/projected/fc655546-130a-49a7-90d6-b76d8e033028-kube-api-access-bdskz\") pod \"controller-manager-546b59f455-zx647\" (UID: \"fc655546-130a-49a7-90d6-b76d8e033028\") " pod="openshift-controller-manager/controller-manager-546b59f455-zx647" Dec 03 11:24:19 crc kubenswrapper[5052]: I1203 11:24:19.345311 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/94a23d8f-e983-48c4-8ad4-37404a325973-proxy-ca-bundles" (OuterVolumeSpecName: "proxy-ca-bundles") pod "94a23d8f-e983-48c4-8ad4-37404a325973" (UID: "94a23d8f-e983-48c4-8ad4-37404a325973"). InnerVolumeSpecName "proxy-ca-bundles". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:24:19 crc kubenswrapper[5052]: I1203 11:24:19.345322 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/94a23d8f-e983-48c4-8ad4-37404a325973-client-ca" (OuterVolumeSpecName: "client-ca") pod "94a23d8f-e983-48c4-8ad4-37404a325973" (UID: "94a23d8f-e983-48c4-8ad4-37404a325973"). InnerVolumeSpecName "client-ca". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:24:19 crc kubenswrapper[5052]: I1203 11:24:19.345362 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/94a23d8f-e983-48c4-8ad4-37404a325973-config" (OuterVolumeSpecName: "config") pod "94a23d8f-e983-48c4-8ad4-37404a325973" (UID: "94a23d8f-e983-48c4-8ad4-37404a325973"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:24:19 crc kubenswrapper[5052]: I1203 11:24:19.350405 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/94a23d8f-e983-48c4-8ad4-37404a325973-serving-cert" (OuterVolumeSpecName: "serving-cert") pod "94a23d8f-e983-48c4-8ad4-37404a325973" (UID: "94a23d8f-e983-48c4-8ad4-37404a325973"). InnerVolumeSpecName "serving-cert". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:24:19 crc kubenswrapper[5052]: I1203 11:24:19.352704 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/94a23d8f-e983-48c4-8ad4-37404a325973-kube-api-access-l75pp" (OuterVolumeSpecName: "kube-api-access-l75pp") pod "94a23d8f-e983-48c4-8ad4-37404a325973" (UID: "94a23d8f-e983-48c4-8ad4-37404a325973"). InnerVolumeSpecName "kube-api-access-l75pp". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:24:19 crc kubenswrapper[5052]: I1203 11:24:19.445177 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"client-ca\" (UniqueName: \"kubernetes.io/configmap/fc655546-130a-49a7-90d6-b76d8e033028-client-ca\") pod \"controller-manager-546b59f455-zx647\" (UID: \"fc655546-130a-49a7-90d6-b76d8e033028\") " pod="openshift-controller-manager/controller-manager-546b59f455-zx647" Dec 03 11:24:19 crc kubenswrapper[5052]: I1203 11:24:19.445401 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/fc655546-130a-49a7-90d6-b76d8e033028-serving-cert\") pod \"controller-manager-546b59f455-zx647\" (UID: \"fc655546-130a-49a7-90d6-b76d8e033028\") " pod="openshift-controller-manager/controller-manager-546b59f455-zx647" Dec 03 11:24:19 crc kubenswrapper[5052]: I1203 11:24:19.445433 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"proxy-ca-bundles\" (UniqueName: \"kubernetes.io/configmap/fc655546-130a-49a7-90d6-b76d8e033028-proxy-ca-bundles\") pod \"controller-manager-546b59f455-zx647\" (UID: \"fc655546-130a-49a7-90d6-b76d8e033028\") " pod="openshift-controller-manager/controller-manager-546b59f455-zx647" Dec 03 11:24:19 crc kubenswrapper[5052]: I1203 11:24:19.445490 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/fc655546-130a-49a7-90d6-b76d8e033028-config\") pod \"controller-manager-546b59f455-zx647\" (UID: \"fc655546-130a-49a7-90d6-b76d8e033028\") " pod="openshift-controller-manager/controller-manager-546b59f455-zx647" Dec 03 11:24:19 crc kubenswrapper[5052]: I1203 11:24:19.445566 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-bdskz\" (UniqueName: \"kubernetes.io/projected/fc655546-130a-49a7-90d6-b76d8e033028-kube-api-access-bdskz\") pod \"controller-manager-546b59f455-zx647\" (UID: \"fc655546-130a-49a7-90d6-b76d8e033028\") " pod="openshift-controller-manager/controller-manager-546b59f455-zx647" Dec 03 11:24:19 crc kubenswrapper[5052]: I1203 11:24:19.445644 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-l75pp\" (UniqueName: \"kubernetes.io/projected/94a23d8f-e983-48c4-8ad4-37404a325973-kube-api-access-l75pp\") on node \"crc\" DevicePath \"\"" Dec 03 11:24:19 crc kubenswrapper[5052]: I1203 11:24:19.445657 5052 reconciler_common.go:293] "Volume detached for volume \"proxy-ca-bundles\" (UniqueName: \"kubernetes.io/configmap/94a23d8f-e983-48c4-8ad4-37404a325973-proxy-ca-bundles\") on node \"crc\" DevicePath \"\"" Dec 03 11:24:19 crc kubenswrapper[5052]: I1203 11:24:19.445670 5052 reconciler_common.go:293] "Volume detached for volume \"client-ca\" (UniqueName: \"kubernetes.io/configmap/94a23d8f-e983-48c4-8ad4-37404a325973-client-ca\") on node \"crc\" DevicePath \"\"" Dec 03 11:24:19 crc kubenswrapper[5052]: I1203 11:24:19.445681 5052 reconciler_common.go:293] "Volume detached for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/94a23d8f-e983-48c4-8ad4-37404a325973-serving-cert\") on node \"crc\" DevicePath \"\"" Dec 03 11:24:19 crc kubenswrapper[5052]: I1203 11:24:19.445691 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/94a23d8f-e983-48c4-8ad4-37404a325973-config\") on node \"crc\" DevicePath \"\"" Dec 03 11:24:19 crc kubenswrapper[5052]: I1203 11:24:19.446471 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"client-ca\" (UniqueName: \"kubernetes.io/configmap/fc655546-130a-49a7-90d6-b76d8e033028-client-ca\") pod \"controller-manager-546b59f455-zx647\" (UID: \"fc655546-130a-49a7-90d6-b76d8e033028\") " pod="openshift-controller-manager/controller-manager-546b59f455-zx647" Dec 03 11:24:19 crc kubenswrapper[5052]: I1203 11:24:19.447200 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"proxy-ca-bundles\" (UniqueName: \"kubernetes.io/configmap/fc655546-130a-49a7-90d6-b76d8e033028-proxy-ca-bundles\") pod \"controller-manager-546b59f455-zx647\" (UID: \"fc655546-130a-49a7-90d6-b76d8e033028\") " pod="openshift-controller-manager/controller-manager-546b59f455-zx647" Dec 03 11:24:19 crc kubenswrapper[5052]: I1203 11:24:19.447838 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/fc655546-130a-49a7-90d6-b76d8e033028-config\") pod \"controller-manager-546b59f455-zx647\" (UID: \"fc655546-130a-49a7-90d6-b76d8e033028\") " pod="openshift-controller-manager/controller-manager-546b59f455-zx647" Dec 03 11:24:19 crc kubenswrapper[5052]: I1203 11:24:19.451037 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/fc655546-130a-49a7-90d6-b76d8e033028-serving-cert\") pod \"controller-manager-546b59f455-zx647\" (UID: \"fc655546-130a-49a7-90d6-b76d8e033028\") " pod="openshift-controller-manager/controller-manager-546b59f455-zx647" Dec 03 11:24:19 crc kubenswrapper[5052]: I1203 11:24:19.463532 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-bdskz\" (UniqueName: \"kubernetes.io/projected/fc655546-130a-49a7-90d6-b76d8e033028-kube-api-access-bdskz\") pod \"controller-manager-546b59f455-zx647\" (UID: \"fc655546-130a-49a7-90d6-b76d8e033028\") " pod="openshift-controller-manager/controller-manager-546b59f455-zx647" Dec 03 11:24:19 crc kubenswrapper[5052]: I1203 11:24:19.522327 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-controller-manager/controller-manager-546b59f455-zx647" Dec 03 11:24:19 crc kubenswrapper[5052]: I1203 11:24:19.741709 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-controller-manager/controller-manager-546b59f455-zx647"] Dec 03 11:24:19 crc kubenswrapper[5052]: W1203 11:24:19.746685 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc655546_130a_49a7_90d6_b76d8e033028.slice/crio-49d37a016376b9b42e5cd9ef365a50889d393c53d4296173e1036da1b0117d79 WatchSource:0}: Error finding container 49d37a016376b9b42e5cd9ef365a50889d393c53d4296173e1036da1b0117d79: Status 404 returned error can't find the container with id 49d37a016376b9b42e5cd9ef365a50889d393c53d4296173e1036da1b0117d79 Dec 03 11:24:20 crc kubenswrapper[5052]: I1203 11:24:20.159683 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-controller-manager/controller-manager-546b59f455-zx647" event={"ID":"fc655546-130a-49a7-90d6-b76d8e033028","Type":"ContainerStarted","Data":"e6c788482682fb366ce6e34ffc9ac2979cc3f5c6f1c24cd4137c7e5818224561"} Dec 03 11:24:20 crc kubenswrapper[5052]: I1203 11:24:20.160063 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openshift-controller-manager/controller-manager-546b59f455-zx647" Dec 03 11:24:20 crc kubenswrapper[5052]: I1203 11:24:20.160076 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-controller-manager/controller-manager-546b59f455-zx647" event={"ID":"fc655546-130a-49a7-90d6-b76d8e033028","Type":"ContainerStarted","Data":"49d37a016376b9b42e5cd9ef365a50889d393c53d4296173e1036da1b0117d79"} Dec 03 11:24:20 crc kubenswrapper[5052]: I1203 11:24:20.159722 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-controller-manager/controller-manager-7bd8d5968-hddbl" Dec 03 11:24:20 crc kubenswrapper[5052]: I1203 11:24:20.168794 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-controller-manager/controller-manager-546b59f455-zx647" Dec 03 11:24:20 crc kubenswrapper[5052]: I1203 11:24:20.177933 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-controller-manager/controller-manager-546b59f455-zx647" podStartSLOduration=4.177916706 podStartE2EDuration="4.177916706s" podCreationTimestamp="2025-12-03 11:24:16 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:24:20.175677463 +0000 UTC m=+250.763919227" watchObservedRunningTime="2025-12-03 11:24:20.177916706 +0000 UTC m=+250.766158470" Dec 03 11:24:20 crc kubenswrapper[5052]: I1203 11:24:20.191343 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openshift-controller-manager/controller-manager-7bd8d5968-hddbl"] Dec 03 11:24:20 crc kubenswrapper[5052]: I1203 11:24:20.196836 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openshift-controller-manager/controller-manager-7bd8d5968-hddbl"] Dec 03 11:24:21 crc kubenswrapper[5052]: I1203 11:24:21.681149 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="94a23d8f-e983-48c4-8ad4-37404a325973" path="/var/lib/kubelet/pods/94a23d8f-e983-48c4-8ad4-37404a325973/volumes" Dec 03 11:24:21 crc kubenswrapper[5052]: I1203 11:24:21.805216 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openshift-controller-manager/controller-manager-546b59f455-zx647"] Dec 03 11:24:23 crc kubenswrapper[5052]: I1203 11:24:23.173548 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openshift-controller-manager/controller-manager-546b59f455-zx647" podUID="fc655546-130a-49a7-90d6-b76d8e033028" containerName="controller-manager" containerID="cri-o://e6c788482682fb366ce6e34ffc9ac2979cc3f5c6f1c24cd4137c7e5818224561" gracePeriod=30 Dec 03 11:24:24 crc kubenswrapper[5052]: I1203 11:24:24.160606 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-controller-manager/controller-manager-546b59f455-zx647" Dec 03 11:24:24 crc kubenswrapper[5052]: I1203 11:24:24.181344 5052 generic.go:334] "Generic (PLEG): container finished" podID="fc655546-130a-49a7-90d6-b76d8e033028" containerID="e6c788482682fb366ce6e34ffc9ac2979cc3f5c6f1c24cd4137c7e5818224561" exitCode=0 Dec 03 11:24:24 crc kubenswrapper[5052]: I1203 11:24:24.181411 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-controller-manager/controller-manager-546b59f455-zx647" Dec 03 11:24:24 crc kubenswrapper[5052]: I1203 11:24:24.181396 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-controller-manager/controller-manager-546b59f455-zx647" event={"ID":"fc655546-130a-49a7-90d6-b76d8e033028","Type":"ContainerDied","Data":"e6c788482682fb366ce6e34ffc9ac2979cc3f5c6f1c24cd4137c7e5818224561"} Dec 03 11:24:24 crc kubenswrapper[5052]: I1203 11:24:24.181669 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-controller-manager/controller-manager-546b59f455-zx647" event={"ID":"fc655546-130a-49a7-90d6-b76d8e033028","Type":"ContainerDied","Data":"49d37a016376b9b42e5cd9ef365a50889d393c53d4296173e1036da1b0117d79"} Dec 03 11:24:24 crc kubenswrapper[5052]: I1203 11:24:24.181713 5052 scope.go:117] "RemoveContainer" containerID="e6c788482682fb366ce6e34ffc9ac2979cc3f5c6f1c24cd4137c7e5818224561" Dec 03 11:24:24 crc kubenswrapper[5052]: I1203 11:24:24.190713 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openshift-controller-manager/controller-manager-7bd8d5968-24t6k"] Dec 03 11:24:24 crc kubenswrapper[5052]: E1203 11:24:24.191050 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="fc655546-130a-49a7-90d6-b76d8e033028" containerName="controller-manager" Dec 03 11:24:24 crc kubenswrapper[5052]: I1203 11:24:24.191069 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="fc655546-130a-49a7-90d6-b76d8e033028" containerName="controller-manager" Dec 03 11:24:24 crc kubenswrapper[5052]: I1203 11:24:24.191176 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="fc655546-130a-49a7-90d6-b76d8e033028" containerName="controller-manager" Dec 03 11:24:24 crc kubenswrapper[5052]: I1203 11:24:24.191701 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-controller-manager/controller-manager-7bd8d5968-24t6k" Dec 03 11:24:24 crc kubenswrapper[5052]: I1203 11:24:24.199613 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-controller-manager/controller-manager-7bd8d5968-24t6k"] Dec 03 11:24:24 crc kubenswrapper[5052]: I1203 11:24:24.201834 5052 scope.go:117] "RemoveContainer" containerID="e6c788482682fb366ce6e34ffc9ac2979cc3f5c6f1c24cd4137c7e5818224561" Dec 03 11:24:24 crc kubenswrapper[5052]: E1203 11:24:24.202348 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e6c788482682fb366ce6e34ffc9ac2979cc3f5c6f1c24cd4137c7e5818224561\": container with ID starting with e6c788482682fb366ce6e34ffc9ac2979cc3f5c6f1c24cd4137c7e5818224561 not found: ID does not exist" containerID="e6c788482682fb366ce6e34ffc9ac2979cc3f5c6f1c24cd4137c7e5818224561" Dec 03 11:24:24 crc kubenswrapper[5052]: I1203 11:24:24.202393 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e6c788482682fb366ce6e34ffc9ac2979cc3f5c6f1c24cd4137c7e5818224561"} err="failed to get container status \"e6c788482682fb366ce6e34ffc9ac2979cc3f5c6f1c24cd4137c7e5818224561\": rpc error: code = NotFound desc = could not find container \"e6c788482682fb366ce6e34ffc9ac2979cc3f5c6f1c24cd4137c7e5818224561\": container with ID starting with e6c788482682fb366ce6e34ffc9ac2979cc3f5c6f1c24cd4137c7e5818224561 not found: ID does not exist" Dec 03 11:24:24 crc kubenswrapper[5052]: I1203 11:24:24.310183 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"client-ca\" (UniqueName: \"kubernetes.io/configmap/fc655546-130a-49a7-90d6-b76d8e033028-client-ca\") pod \"fc655546-130a-49a7-90d6-b76d8e033028\" (UID: \"fc655546-130a-49a7-90d6-b76d8e033028\") " Dec 03 11:24:24 crc kubenswrapper[5052]: I1203 11:24:24.310286 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/fc655546-130a-49a7-90d6-b76d8e033028-config\") pod \"fc655546-130a-49a7-90d6-b76d8e033028\" (UID: \"fc655546-130a-49a7-90d6-b76d8e033028\") " Dec 03 11:24:24 crc kubenswrapper[5052]: I1203 11:24:24.310314 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"proxy-ca-bundles\" (UniqueName: \"kubernetes.io/configmap/fc655546-130a-49a7-90d6-b76d8e033028-proxy-ca-bundles\") pod \"fc655546-130a-49a7-90d6-b76d8e033028\" (UID: \"fc655546-130a-49a7-90d6-b76d8e033028\") " Dec 03 11:24:24 crc kubenswrapper[5052]: I1203 11:24:24.310363 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/fc655546-130a-49a7-90d6-b76d8e033028-serving-cert\") pod \"fc655546-130a-49a7-90d6-b76d8e033028\" (UID: \"fc655546-130a-49a7-90d6-b76d8e033028\") " Dec 03 11:24:24 crc kubenswrapper[5052]: I1203 11:24:24.310412 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bdskz\" (UniqueName: \"kubernetes.io/projected/fc655546-130a-49a7-90d6-b76d8e033028-kube-api-access-bdskz\") pod \"fc655546-130a-49a7-90d6-b76d8e033028\" (UID: \"fc655546-130a-49a7-90d6-b76d8e033028\") " Dec 03 11:24:24 crc kubenswrapper[5052]: I1203 11:24:24.310619 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/11451007-f98a-42ee-8f83-9601d0a18fa3-serving-cert\") pod \"controller-manager-7bd8d5968-24t6k\" (UID: \"11451007-f98a-42ee-8f83-9601d0a18fa3\") " pod="openshift-controller-manager/controller-manager-7bd8d5968-24t6k" Dec 03 11:24:24 crc kubenswrapper[5052]: I1203 11:24:24.310647 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-ca-bundles\" (UniqueName: \"kubernetes.io/configmap/11451007-f98a-42ee-8f83-9601d0a18fa3-proxy-ca-bundles\") pod \"controller-manager-7bd8d5968-24t6k\" (UID: \"11451007-f98a-42ee-8f83-9601d0a18fa3\") " pod="openshift-controller-manager/controller-manager-7bd8d5968-24t6k" Dec 03 11:24:24 crc kubenswrapper[5052]: I1203 11:24:24.310701 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bq29b\" (UniqueName: \"kubernetes.io/projected/11451007-f98a-42ee-8f83-9601d0a18fa3-kube-api-access-bq29b\") pod \"controller-manager-7bd8d5968-24t6k\" (UID: \"11451007-f98a-42ee-8f83-9601d0a18fa3\") " pod="openshift-controller-manager/controller-manager-7bd8d5968-24t6k" Dec 03 11:24:24 crc kubenswrapper[5052]: I1203 11:24:24.310728 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/11451007-f98a-42ee-8f83-9601d0a18fa3-config\") pod \"controller-manager-7bd8d5968-24t6k\" (UID: \"11451007-f98a-42ee-8f83-9601d0a18fa3\") " pod="openshift-controller-manager/controller-manager-7bd8d5968-24t6k" Dec 03 11:24:24 crc kubenswrapper[5052]: I1203 11:24:24.310879 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca\" (UniqueName: \"kubernetes.io/configmap/11451007-f98a-42ee-8f83-9601d0a18fa3-client-ca\") pod \"controller-manager-7bd8d5968-24t6k\" (UID: \"11451007-f98a-42ee-8f83-9601d0a18fa3\") " pod="openshift-controller-manager/controller-manager-7bd8d5968-24t6k" Dec 03 11:24:24 crc kubenswrapper[5052]: I1203 11:24:24.311396 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fc655546-130a-49a7-90d6-b76d8e033028-proxy-ca-bundles" (OuterVolumeSpecName: "proxy-ca-bundles") pod "fc655546-130a-49a7-90d6-b76d8e033028" (UID: "fc655546-130a-49a7-90d6-b76d8e033028"). InnerVolumeSpecName "proxy-ca-bundles". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:24:24 crc kubenswrapper[5052]: I1203 11:24:24.311449 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fc655546-130a-49a7-90d6-b76d8e033028-config" (OuterVolumeSpecName: "config") pod "fc655546-130a-49a7-90d6-b76d8e033028" (UID: "fc655546-130a-49a7-90d6-b76d8e033028"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:24:24 crc kubenswrapper[5052]: I1203 11:24:24.311466 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fc655546-130a-49a7-90d6-b76d8e033028-client-ca" (OuterVolumeSpecName: "client-ca") pod "fc655546-130a-49a7-90d6-b76d8e033028" (UID: "fc655546-130a-49a7-90d6-b76d8e033028"). InnerVolumeSpecName "client-ca". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:24:24 crc kubenswrapper[5052]: I1203 11:24:24.316410 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fc655546-130a-49a7-90d6-b76d8e033028-kube-api-access-bdskz" (OuterVolumeSpecName: "kube-api-access-bdskz") pod "fc655546-130a-49a7-90d6-b76d8e033028" (UID: "fc655546-130a-49a7-90d6-b76d8e033028"). InnerVolumeSpecName "kube-api-access-bdskz". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:24:24 crc kubenswrapper[5052]: I1203 11:24:24.317800 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fc655546-130a-49a7-90d6-b76d8e033028-serving-cert" (OuterVolumeSpecName: "serving-cert") pod "fc655546-130a-49a7-90d6-b76d8e033028" (UID: "fc655546-130a-49a7-90d6-b76d8e033028"). InnerVolumeSpecName "serving-cert". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:24:24 crc kubenswrapper[5052]: I1203 11:24:24.412017 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"proxy-ca-bundles\" (UniqueName: \"kubernetes.io/configmap/11451007-f98a-42ee-8f83-9601d0a18fa3-proxy-ca-bundles\") pod \"controller-manager-7bd8d5968-24t6k\" (UID: \"11451007-f98a-42ee-8f83-9601d0a18fa3\") " pod="openshift-controller-manager/controller-manager-7bd8d5968-24t6k" Dec 03 11:24:24 crc kubenswrapper[5052]: I1203 11:24:24.412083 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/11451007-f98a-42ee-8f83-9601d0a18fa3-serving-cert\") pod \"controller-manager-7bd8d5968-24t6k\" (UID: \"11451007-f98a-42ee-8f83-9601d0a18fa3\") " pod="openshift-controller-manager/controller-manager-7bd8d5968-24t6k" Dec 03 11:24:24 crc kubenswrapper[5052]: I1203 11:24:24.412144 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-bq29b\" (UniqueName: \"kubernetes.io/projected/11451007-f98a-42ee-8f83-9601d0a18fa3-kube-api-access-bq29b\") pod \"controller-manager-7bd8d5968-24t6k\" (UID: \"11451007-f98a-42ee-8f83-9601d0a18fa3\") " pod="openshift-controller-manager/controller-manager-7bd8d5968-24t6k" Dec 03 11:24:24 crc kubenswrapper[5052]: I1203 11:24:24.412175 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/11451007-f98a-42ee-8f83-9601d0a18fa3-config\") pod \"controller-manager-7bd8d5968-24t6k\" (UID: \"11451007-f98a-42ee-8f83-9601d0a18fa3\") " pod="openshift-controller-manager/controller-manager-7bd8d5968-24t6k" Dec 03 11:24:24 crc kubenswrapper[5052]: I1203 11:24:24.412208 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"client-ca\" (UniqueName: \"kubernetes.io/configmap/11451007-f98a-42ee-8f83-9601d0a18fa3-client-ca\") pod \"controller-manager-7bd8d5968-24t6k\" (UID: \"11451007-f98a-42ee-8f83-9601d0a18fa3\") " pod="openshift-controller-manager/controller-manager-7bd8d5968-24t6k" Dec 03 11:24:24 crc kubenswrapper[5052]: I1203 11:24:24.412304 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/fc655546-130a-49a7-90d6-b76d8e033028-config\") on node \"crc\" DevicePath \"\"" Dec 03 11:24:24 crc kubenswrapper[5052]: I1203 11:24:24.412321 5052 reconciler_common.go:293] "Volume detached for volume \"proxy-ca-bundles\" (UniqueName: \"kubernetes.io/configmap/fc655546-130a-49a7-90d6-b76d8e033028-proxy-ca-bundles\") on node \"crc\" DevicePath \"\"" Dec 03 11:24:24 crc kubenswrapper[5052]: I1203 11:24:24.412336 5052 reconciler_common.go:293] "Volume detached for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/fc655546-130a-49a7-90d6-b76d8e033028-serving-cert\") on node \"crc\" DevicePath \"\"" Dec 03 11:24:24 crc kubenswrapper[5052]: I1203 11:24:24.412348 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-bdskz\" (UniqueName: \"kubernetes.io/projected/fc655546-130a-49a7-90d6-b76d8e033028-kube-api-access-bdskz\") on node \"crc\" DevicePath \"\"" Dec 03 11:24:24 crc kubenswrapper[5052]: I1203 11:24:24.412359 5052 reconciler_common.go:293] "Volume detached for volume \"client-ca\" (UniqueName: \"kubernetes.io/configmap/fc655546-130a-49a7-90d6-b76d8e033028-client-ca\") on node \"crc\" DevicePath \"\"" Dec 03 11:24:24 crc kubenswrapper[5052]: I1203 11:24:24.413406 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"client-ca\" (UniqueName: \"kubernetes.io/configmap/11451007-f98a-42ee-8f83-9601d0a18fa3-client-ca\") pod \"controller-manager-7bd8d5968-24t6k\" (UID: \"11451007-f98a-42ee-8f83-9601d0a18fa3\") " pod="openshift-controller-manager/controller-manager-7bd8d5968-24t6k" Dec 03 11:24:24 crc kubenswrapper[5052]: I1203 11:24:24.414145 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/11451007-f98a-42ee-8f83-9601d0a18fa3-config\") pod \"controller-manager-7bd8d5968-24t6k\" (UID: \"11451007-f98a-42ee-8f83-9601d0a18fa3\") " pod="openshift-controller-manager/controller-manager-7bd8d5968-24t6k" Dec 03 11:24:24 crc kubenswrapper[5052]: I1203 11:24:24.416760 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/11451007-f98a-42ee-8f83-9601d0a18fa3-serving-cert\") pod \"controller-manager-7bd8d5968-24t6k\" (UID: \"11451007-f98a-42ee-8f83-9601d0a18fa3\") " pod="openshift-controller-manager/controller-manager-7bd8d5968-24t6k" Dec 03 11:24:24 crc kubenswrapper[5052]: I1203 11:24:24.417094 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"proxy-ca-bundles\" (UniqueName: \"kubernetes.io/configmap/11451007-f98a-42ee-8f83-9601d0a18fa3-proxy-ca-bundles\") pod \"controller-manager-7bd8d5968-24t6k\" (UID: \"11451007-f98a-42ee-8f83-9601d0a18fa3\") " pod="openshift-controller-manager/controller-manager-7bd8d5968-24t6k" Dec 03 11:24:24 crc kubenswrapper[5052]: I1203 11:24:24.431054 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-bq29b\" (UniqueName: \"kubernetes.io/projected/11451007-f98a-42ee-8f83-9601d0a18fa3-kube-api-access-bq29b\") pod \"controller-manager-7bd8d5968-24t6k\" (UID: \"11451007-f98a-42ee-8f83-9601d0a18fa3\") " pod="openshift-controller-manager/controller-manager-7bd8d5968-24t6k" Dec 03 11:24:24 crc kubenswrapper[5052]: I1203 11:24:24.505786 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-controller-manager/controller-manager-7bd8d5968-24t6k" Dec 03 11:24:24 crc kubenswrapper[5052]: I1203 11:24:24.506928 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openshift-controller-manager/controller-manager-546b59f455-zx647"] Dec 03 11:24:24 crc kubenswrapper[5052]: I1203 11:24:24.518418 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openshift-controller-manager/controller-manager-546b59f455-zx647"] Dec 03 11:24:24 crc kubenswrapper[5052]: I1203 11:24:24.729343 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-controller-manager/controller-manager-7bd8d5968-24t6k"] Dec 03 11:24:25 crc kubenswrapper[5052]: I1203 11:24:25.188966 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-controller-manager/controller-manager-7bd8d5968-24t6k" event={"ID":"11451007-f98a-42ee-8f83-9601d0a18fa3","Type":"ContainerStarted","Data":"04bd7e9979b16ba99287e352c7fa8c531599d31554b1fd06f743a642096c143c"} Dec 03 11:24:25 crc kubenswrapper[5052]: I1203 11:24:25.189286 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-controller-manager/controller-manager-7bd8d5968-24t6k" event={"ID":"11451007-f98a-42ee-8f83-9601d0a18fa3","Type":"ContainerStarted","Data":"c81fadfda24d9d04b62f4eb34e05c1dff650dd1bb5365d8ddc30805410b23e3b"} Dec 03 11:24:25 crc kubenswrapper[5052]: I1203 11:24:25.189304 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openshift-controller-manager/controller-manager-7bd8d5968-24t6k" Dec 03 11:24:25 crc kubenswrapper[5052]: I1203 11:24:25.198826 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-controller-manager/controller-manager-7bd8d5968-24t6k" Dec 03 11:24:25 crc kubenswrapper[5052]: I1203 11:24:25.244042 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-controller-manager/controller-manager-7bd8d5968-24t6k" podStartSLOduration=4.244017493 podStartE2EDuration="4.244017493s" podCreationTimestamp="2025-12-03 11:24:21 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:24:25.217112918 +0000 UTC m=+255.805354692" watchObservedRunningTime="2025-12-03 11:24:25.244017493 +0000 UTC m=+255.832259257" Dec 03 11:24:25 crc kubenswrapper[5052]: I1203 11:24:25.682726 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fc655546-130a-49a7-90d6-b76d8e033028" path="/var/lib/kubelet/pods/fc655546-130a-49a7-90d6-b76d8e033028/volumes" Dec 03 11:24:31 crc kubenswrapper[5052]: I1203 11:24:31.025577 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openshift-marketplace/community-operators-r8txz"] Dec 03 11:24:31 crc kubenswrapper[5052]: I1203 11:24:31.026696 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openshift-marketplace/community-operators-r8txz" podUID="f1ea0ffb-1062-4713-908b-55c0b07c49ee" containerName="registry-server" containerID="cri-o://84a10027d57c8f6d30f6e4c7f8065d6749b27b1a001972b6421953cced8ece00" gracePeriod=2 Dec 03 11:24:31 crc kubenswrapper[5052]: I1203 11:24:31.825214 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openshift-marketplace/certified-operators-99svv"] Dec 03 11:24:31 crc kubenswrapper[5052]: I1203 11:24:31.825465 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openshift-marketplace/certified-operators-99svv" podUID="3b1b1a53-eff1-4339-a82e-9b8400dd5604" containerName="registry-server" containerID="cri-o://e764472b38e59af009aaebaecca6572e1999fbdc2ba38646e6f0c78ecc104706" gracePeriod=2 Dec 03 11:24:33 crc kubenswrapper[5052]: I1203 11:24:33.112115 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-66df7c8f76-tth5j"] Dec 03 11:24:33 crc kubenswrapper[5052]: I1203 11:24:33.114097 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-66df7c8f76-tth5j" Dec 03 11:24:33 crc kubenswrapper[5052]: I1203 11:24:33.126830 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-66df7c8f76-tth5j"] Dec 03 11:24:33 crc kubenswrapper[5052]: I1203 11:24:33.242074 5052 generic.go:334] "Generic (PLEG): container finished" podID="f1ea0ffb-1062-4713-908b-55c0b07c49ee" containerID="84a10027d57c8f6d30f6e4c7f8065d6749b27b1a001972b6421953cced8ece00" exitCode=0 Dec 03 11:24:33 crc kubenswrapper[5052]: I1203 11:24:33.242180 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/community-operators-r8txz" event={"ID":"f1ea0ffb-1062-4713-908b-55c0b07c49ee","Type":"ContainerDied","Data":"84a10027d57c8f6d30f6e4c7f8065d6749b27b1a001972b6421953cced8ece00"} Dec 03 11:24:33 crc kubenswrapper[5052]: I1203 11:24:33.244055 5052 generic.go:334] "Generic (PLEG): container finished" podID="3b1b1a53-eff1-4339-a82e-9b8400dd5604" containerID="e764472b38e59af009aaebaecca6572e1999fbdc2ba38646e6f0c78ecc104706" exitCode=0 Dec 03 11:24:33 crc kubenswrapper[5052]: I1203 11:24:33.244075 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/certified-operators-99svv" event={"ID":"3b1b1a53-eff1-4339-a82e-9b8400dd5604","Type":"ContainerDied","Data":"e764472b38e59af009aaebaecca6572e1999fbdc2ba38646e6f0c78ecc104706"} Dec 03 11:24:33 crc kubenswrapper[5052]: I1203 11:24:33.264356 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0a3d2d53-efdd-49bb-bca9-90285d88a220-ca-trust-extracted\") pod \"image-registry-66df7c8f76-tth5j\" (UID: \"0a3d2d53-efdd-49bb-bca9-90285d88a220\") " pod="openshift-image-registry/image-registry-66df7c8f76-tth5j" Dec 03 11:24:33 crc kubenswrapper[5052]: I1203 11:24:33.264872 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0a3d2d53-efdd-49bb-bca9-90285d88a220-installation-pull-secrets\") pod \"image-registry-66df7c8f76-tth5j\" (UID: \"0a3d2d53-efdd-49bb-bca9-90285d88a220\") " pod="openshift-image-registry/image-registry-66df7c8f76-tth5j" Dec 03 11:24:33 crc kubenswrapper[5052]: I1203 11:24:33.264907 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0a3d2d53-efdd-49bb-bca9-90285d88a220-registry-tls\") pod \"image-registry-66df7c8f76-tth5j\" (UID: \"0a3d2d53-efdd-49bb-bca9-90285d88a220\") " pod="openshift-image-registry/image-registry-66df7c8f76-tth5j" Dec 03 11:24:33 crc kubenswrapper[5052]: I1203 11:24:33.264944 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0a3d2d53-efdd-49bb-bca9-90285d88a220-registry-certificates\") pod \"image-registry-66df7c8f76-tth5j\" (UID: \"0a3d2d53-efdd-49bb-bca9-90285d88a220\") " pod="openshift-image-registry/image-registry-66df7c8f76-tth5j" Dec 03 11:24:33 crc kubenswrapper[5052]: I1203 11:24:33.264968 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0a3d2d53-efdd-49bb-bca9-90285d88a220-bound-sa-token\") pod \"image-registry-66df7c8f76-tth5j\" (UID: \"0a3d2d53-efdd-49bb-bca9-90285d88a220\") " pod="openshift-image-registry/image-registry-66df7c8f76-tth5j" Dec 03 11:24:33 crc kubenswrapper[5052]: I1203 11:24:33.265012 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\") pod \"image-registry-66df7c8f76-tth5j\" (UID: \"0a3d2d53-efdd-49bb-bca9-90285d88a220\") " pod="openshift-image-registry/image-registry-66df7c8f76-tth5j" Dec 03 11:24:33 crc kubenswrapper[5052]: I1203 11:24:33.265045 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nskrd\" (UniqueName: \"kubernetes.io/projected/0a3d2d53-efdd-49bb-bca9-90285d88a220-kube-api-access-nskrd\") pod \"image-registry-66df7c8f76-tth5j\" (UID: \"0a3d2d53-efdd-49bb-bca9-90285d88a220\") " pod="openshift-image-registry/image-registry-66df7c8f76-tth5j" Dec 03 11:24:33 crc kubenswrapper[5052]: I1203 11:24:33.265080 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0a3d2d53-efdd-49bb-bca9-90285d88a220-trusted-ca\") pod \"image-registry-66df7c8f76-tth5j\" (UID: \"0a3d2d53-efdd-49bb-bca9-90285d88a220\") " pod="openshift-image-registry/image-registry-66df7c8f76-tth5j" Dec 03 11:24:33 crc kubenswrapper[5052]: I1203 11:24:33.297187 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\") pod \"image-registry-66df7c8f76-tth5j\" (UID: \"0a3d2d53-efdd-49bb-bca9-90285d88a220\") " pod="openshift-image-registry/image-registry-66df7c8f76-tth5j" Dec 03 11:24:33 crc kubenswrapper[5052]: I1203 11:24:33.366559 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0a3d2d53-efdd-49bb-bca9-90285d88a220-ca-trust-extracted\") pod \"image-registry-66df7c8f76-tth5j\" (UID: \"0a3d2d53-efdd-49bb-bca9-90285d88a220\") " pod="openshift-image-registry/image-registry-66df7c8f76-tth5j" Dec 03 11:24:33 crc kubenswrapper[5052]: I1203 11:24:33.366622 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0a3d2d53-efdd-49bb-bca9-90285d88a220-installation-pull-secrets\") pod \"image-registry-66df7c8f76-tth5j\" (UID: \"0a3d2d53-efdd-49bb-bca9-90285d88a220\") " pod="openshift-image-registry/image-registry-66df7c8f76-tth5j" Dec 03 11:24:33 crc kubenswrapper[5052]: I1203 11:24:33.366643 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0a3d2d53-efdd-49bb-bca9-90285d88a220-registry-tls\") pod \"image-registry-66df7c8f76-tth5j\" (UID: \"0a3d2d53-efdd-49bb-bca9-90285d88a220\") " pod="openshift-image-registry/image-registry-66df7c8f76-tth5j" Dec 03 11:24:33 crc kubenswrapper[5052]: I1203 11:24:33.366671 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0a3d2d53-efdd-49bb-bca9-90285d88a220-registry-certificates\") pod \"image-registry-66df7c8f76-tth5j\" (UID: \"0a3d2d53-efdd-49bb-bca9-90285d88a220\") " pod="openshift-image-registry/image-registry-66df7c8f76-tth5j" Dec 03 11:24:33 crc kubenswrapper[5052]: I1203 11:24:33.366687 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0a3d2d53-efdd-49bb-bca9-90285d88a220-bound-sa-token\") pod \"image-registry-66df7c8f76-tth5j\" (UID: \"0a3d2d53-efdd-49bb-bca9-90285d88a220\") " pod="openshift-image-registry/image-registry-66df7c8f76-tth5j" Dec 03 11:24:33 crc kubenswrapper[5052]: I1203 11:24:33.366873 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-nskrd\" (UniqueName: \"kubernetes.io/projected/0a3d2d53-efdd-49bb-bca9-90285d88a220-kube-api-access-nskrd\") pod \"image-registry-66df7c8f76-tth5j\" (UID: \"0a3d2d53-efdd-49bb-bca9-90285d88a220\") " pod="openshift-image-registry/image-registry-66df7c8f76-tth5j" Dec 03 11:24:33 crc kubenswrapper[5052]: I1203 11:24:33.366922 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0a3d2d53-efdd-49bb-bca9-90285d88a220-trusted-ca\") pod \"image-registry-66df7c8f76-tth5j\" (UID: \"0a3d2d53-efdd-49bb-bca9-90285d88a220\") " pod="openshift-image-registry/image-registry-66df7c8f76-tth5j" Dec 03 11:24:33 crc kubenswrapper[5052]: I1203 11:24:33.367280 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0a3d2d53-efdd-49bb-bca9-90285d88a220-ca-trust-extracted\") pod \"image-registry-66df7c8f76-tth5j\" (UID: \"0a3d2d53-efdd-49bb-bca9-90285d88a220\") " pod="openshift-image-registry/image-registry-66df7c8f76-tth5j" Dec 03 11:24:33 crc kubenswrapper[5052]: I1203 11:24:33.368221 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0a3d2d53-efdd-49bb-bca9-90285d88a220-trusted-ca\") pod \"image-registry-66df7c8f76-tth5j\" (UID: \"0a3d2d53-efdd-49bb-bca9-90285d88a220\") " pod="openshift-image-registry/image-registry-66df7c8f76-tth5j" Dec 03 11:24:33 crc kubenswrapper[5052]: I1203 11:24:33.368360 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0a3d2d53-efdd-49bb-bca9-90285d88a220-registry-certificates\") pod \"image-registry-66df7c8f76-tth5j\" (UID: \"0a3d2d53-efdd-49bb-bca9-90285d88a220\") " pod="openshift-image-registry/image-registry-66df7c8f76-tth5j" Dec 03 11:24:33 crc kubenswrapper[5052]: I1203 11:24:33.373076 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0a3d2d53-efdd-49bb-bca9-90285d88a220-installation-pull-secrets\") pod \"image-registry-66df7c8f76-tth5j\" (UID: \"0a3d2d53-efdd-49bb-bca9-90285d88a220\") " pod="openshift-image-registry/image-registry-66df7c8f76-tth5j" Dec 03 11:24:33 crc kubenswrapper[5052]: I1203 11:24:33.373178 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0a3d2d53-efdd-49bb-bca9-90285d88a220-registry-tls\") pod \"image-registry-66df7c8f76-tth5j\" (UID: \"0a3d2d53-efdd-49bb-bca9-90285d88a220\") " pod="openshift-image-registry/image-registry-66df7c8f76-tth5j" Dec 03 11:24:33 crc kubenswrapper[5052]: I1203 11:24:33.382911 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-nskrd\" (UniqueName: \"kubernetes.io/projected/0a3d2d53-efdd-49bb-bca9-90285d88a220-kube-api-access-nskrd\") pod \"image-registry-66df7c8f76-tth5j\" (UID: \"0a3d2d53-efdd-49bb-bca9-90285d88a220\") " pod="openshift-image-registry/image-registry-66df7c8f76-tth5j" Dec 03 11:24:33 crc kubenswrapper[5052]: I1203 11:24:33.394976 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0a3d2d53-efdd-49bb-bca9-90285d88a220-bound-sa-token\") pod \"image-registry-66df7c8f76-tth5j\" (UID: \"0a3d2d53-efdd-49bb-bca9-90285d88a220\") " pod="openshift-image-registry/image-registry-66df7c8f76-tth5j" Dec 03 11:24:33 crc kubenswrapper[5052]: I1203 11:24:33.436367 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/certified-operators-99svv" Dec 03 11:24:33 crc kubenswrapper[5052]: I1203 11:24:33.439467 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-66df7c8f76-tth5j" Dec 03 11:24:33 crc kubenswrapper[5052]: I1203 11:24:33.569694 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/3b1b1a53-eff1-4339-a82e-9b8400dd5604-utilities\") pod \"3b1b1a53-eff1-4339-a82e-9b8400dd5604\" (UID: \"3b1b1a53-eff1-4339-a82e-9b8400dd5604\") " Dec 03 11:24:33 crc kubenswrapper[5052]: I1203 11:24:33.570144 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/3b1b1a53-eff1-4339-a82e-9b8400dd5604-catalog-content\") pod \"3b1b1a53-eff1-4339-a82e-9b8400dd5604\" (UID: \"3b1b1a53-eff1-4339-a82e-9b8400dd5604\") " Dec 03 11:24:33 crc kubenswrapper[5052]: I1203 11:24:33.570182 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zn4pm\" (UniqueName: \"kubernetes.io/projected/3b1b1a53-eff1-4339-a82e-9b8400dd5604-kube-api-access-zn4pm\") pod \"3b1b1a53-eff1-4339-a82e-9b8400dd5604\" (UID: \"3b1b1a53-eff1-4339-a82e-9b8400dd5604\") " Dec 03 11:24:33 crc kubenswrapper[5052]: I1203 11:24:33.571161 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b1b1a53-eff1-4339-a82e-9b8400dd5604-utilities" (OuterVolumeSpecName: "utilities") pod "3b1b1a53-eff1-4339-a82e-9b8400dd5604" (UID: "3b1b1a53-eff1-4339-a82e-9b8400dd5604"). InnerVolumeSpecName "utilities". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:24:33 crc kubenswrapper[5052]: I1203 11:24:33.594083 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3b1b1a53-eff1-4339-a82e-9b8400dd5604-kube-api-access-zn4pm" (OuterVolumeSpecName: "kube-api-access-zn4pm") pod "3b1b1a53-eff1-4339-a82e-9b8400dd5604" (UID: "3b1b1a53-eff1-4339-a82e-9b8400dd5604"). InnerVolumeSpecName "kube-api-access-zn4pm". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:24:33 crc kubenswrapper[5052]: I1203 11:24:33.623945 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b1b1a53-eff1-4339-a82e-9b8400dd5604-catalog-content" (OuterVolumeSpecName: "catalog-content") pod "3b1b1a53-eff1-4339-a82e-9b8400dd5604" (UID: "3b1b1a53-eff1-4339-a82e-9b8400dd5604"). InnerVolumeSpecName "catalog-content". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:24:33 crc kubenswrapper[5052]: I1203 11:24:33.628954 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openshift-marketplace/redhat-marketplace-qttdj"] Dec 03 11:24:33 crc kubenswrapper[5052]: I1203 11:24:33.629264 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openshift-marketplace/redhat-marketplace-qttdj" podUID="8cf9b98e-2ea9-47ee-bd1b-a913714b7668" containerName="registry-server" containerID="cri-o://1e2172f73a4eb24ef8a5086bfa2b3d1c4e398d6dcf33426b05c5a0016d33f018" gracePeriod=2 Dec 03 11:24:33 crc kubenswrapper[5052]: I1203 11:24:33.671579 5052 reconciler_common.go:293] "Volume detached for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/3b1b1a53-eff1-4339-a82e-9b8400dd5604-utilities\") on node \"crc\" DevicePath \"\"" Dec 03 11:24:33 crc kubenswrapper[5052]: I1203 11:24:33.671625 5052 reconciler_common.go:293] "Volume detached for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/3b1b1a53-eff1-4339-a82e-9b8400dd5604-catalog-content\") on node \"crc\" DevicePath \"\"" Dec 03 11:24:33 crc kubenswrapper[5052]: I1203 11:24:33.671642 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-zn4pm\" (UniqueName: \"kubernetes.io/projected/3b1b1a53-eff1-4339-a82e-9b8400dd5604-kube-api-access-zn4pm\") on node \"crc\" DevicePath \"\"" Dec 03 11:24:33 crc kubenswrapper[5052]: I1203 11:24:33.941381 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-66df7c8f76-tth5j"] Dec 03 11:24:34 crc kubenswrapper[5052]: I1203 11:24:34.232389 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openshift-marketplace/redhat-operators-nq2qk"] Dec 03 11:24:34 crc kubenswrapper[5052]: I1203 11:24:34.233014 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openshift-marketplace/redhat-operators-nq2qk" podUID="1e97c072-bdcc-4095-9a87-ddf7d4f2d22b" containerName="registry-server" containerID="cri-o://4d0e97a334afc8b927d8b98a681cfab0c25dceea8bee70eac0ac81cdb025e397" gracePeriod=2 Dec 03 11:24:34 crc kubenswrapper[5052]: I1203 11:24:34.251073 5052 generic.go:334] "Generic (PLEG): container finished" podID="8cf9b98e-2ea9-47ee-bd1b-a913714b7668" containerID="1e2172f73a4eb24ef8a5086bfa2b3d1c4e398d6dcf33426b05c5a0016d33f018" exitCode=0 Dec 03 11:24:34 crc kubenswrapper[5052]: I1203 11:24:34.251161 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-marketplace-qttdj" event={"ID":"8cf9b98e-2ea9-47ee-bd1b-a913714b7668","Type":"ContainerDied","Data":"1e2172f73a4eb24ef8a5086bfa2b3d1c4e398d6dcf33426b05c5a0016d33f018"} Dec 03 11:24:34 crc kubenswrapper[5052]: I1203 11:24:34.252414 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-66df7c8f76-tth5j" event={"ID":"0a3d2d53-efdd-49bb-bca9-90285d88a220","Type":"ContainerStarted","Data":"d4d7ad54a611796dceada27e3c1f7a24ebb8ef725f6831df83631ab8c6cca5c2"} Dec 03 11:24:34 crc kubenswrapper[5052]: I1203 11:24:34.254573 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/certified-operators-99svv" event={"ID":"3b1b1a53-eff1-4339-a82e-9b8400dd5604","Type":"ContainerDied","Data":"73ff6ff9fa66dca9a11a38cc566ec67d005940268452df96cdfc40a37dd7b59d"} Dec 03 11:24:34 crc kubenswrapper[5052]: I1203 11:24:34.254609 5052 scope.go:117] "RemoveContainer" containerID="e764472b38e59af009aaebaecca6572e1999fbdc2ba38646e6f0c78ecc104706" Dec 03 11:24:34 crc kubenswrapper[5052]: I1203 11:24:34.254673 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/certified-operators-99svv" Dec 03 11:24:34 crc kubenswrapper[5052]: I1203 11:24:34.274601 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openshift-marketplace/certified-operators-99svv"] Dec 03 11:24:34 crc kubenswrapper[5052]: I1203 11:24:34.275034 5052 scope.go:117] "RemoveContainer" containerID="8e3aeed37fefe08d41b2774a5527250f33728938e9f2024a90283dd7bf21c220" Dec 03 11:24:34 crc kubenswrapper[5052]: I1203 11:24:34.280553 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openshift-marketplace/certified-operators-99svv"] Dec 03 11:24:34 crc kubenswrapper[5052]: I1203 11:24:34.306826 5052 scope.go:117] "RemoveContainer" containerID="09adf77efb9328693f484a82c4a75a64e2aa5685f237cfbbad8e3146adf0c914" Dec 03 11:24:34 crc kubenswrapper[5052]: I1203 11:24:34.372819 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/community-operators-r8txz" Dec 03 11:24:34 crc kubenswrapper[5052]: I1203 11:24:34.482716 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/f1ea0ffb-1062-4713-908b-55c0b07c49ee-catalog-content\") pod \"f1ea0ffb-1062-4713-908b-55c0b07c49ee\" (UID: \"f1ea0ffb-1062-4713-908b-55c0b07c49ee\") " Dec 03 11:24:34 crc kubenswrapper[5052]: I1203 11:24:34.482791 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/f1ea0ffb-1062-4713-908b-55c0b07c49ee-utilities\") pod \"f1ea0ffb-1062-4713-908b-55c0b07c49ee\" (UID: \"f1ea0ffb-1062-4713-908b-55c0b07c49ee\") " Dec 03 11:24:34 crc kubenswrapper[5052]: I1203 11:24:34.482880 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g4qvc\" (UniqueName: \"kubernetes.io/projected/f1ea0ffb-1062-4713-908b-55c0b07c49ee-kube-api-access-g4qvc\") pod \"f1ea0ffb-1062-4713-908b-55c0b07c49ee\" (UID: \"f1ea0ffb-1062-4713-908b-55c0b07c49ee\") " Dec 03 11:24:34 crc kubenswrapper[5052]: I1203 11:24:34.484552 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f1ea0ffb-1062-4713-908b-55c0b07c49ee-utilities" (OuterVolumeSpecName: "utilities") pod "f1ea0ffb-1062-4713-908b-55c0b07c49ee" (UID: "f1ea0ffb-1062-4713-908b-55c0b07c49ee"). InnerVolumeSpecName "utilities". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:24:34 crc kubenswrapper[5052]: I1203 11:24:34.496795 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f1ea0ffb-1062-4713-908b-55c0b07c49ee-kube-api-access-g4qvc" (OuterVolumeSpecName: "kube-api-access-g4qvc") pod "f1ea0ffb-1062-4713-908b-55c0b07c49ee" (UID: "f1ea0ffb-1062-4713-908b-55c0b07c49ee"). InnerVolumeSpecName "kube-api-access-g4qvc". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:24:34 crc kubenswrapper[5052]: I1203 11:24:34.552669 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f1ea0ffb-1062-4713-908b-55c0b07c49ee-catalog-content" (OuterVolumeSpecName: "catalog-content") pod "f1ea0ffb-1062-4713-908b-55c0b07c49ee" (UID: "f1ea0ffb-1062-4713-908b-55c0b07c49ee"). InnerVolumeSpecName "catalog-content". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:24:34 crc kubenswrapper[5052]: I1203 11:24:34.584580 5052 reconciler_common.go:293] "Volume detached for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/f1ea0ffb-1062-4713-908b-55c0b07c49ee-catalog-content\") on node \"crc\" DevicePath \"\"" Dec 03 11:24:34 crc kubenswrapper[5052]: I1203 11:24:34.584985 5052 reconciler_common.go:293] "Volume detached for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/f1ea0ffb-1062-4713-908b-55c0b07c49ee-utilities\") on node \"crc\" DevicePath \"\"" Dec 03 11:24:34 crc kubenswrapper[5052]: I1203 11:24:34.585050 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-g4qvc\" (UniqueName: \"kubernetes.io/projected/f1ea0ffb-1062-4713-908b-55c0b07c49ee-kube-api-access-g4qvc\") on node \"crc\" DevicePath \"\"" Dec 03 11:24:35 crc kubenswrapper[5052]: I1203 11:24:35.233287 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/redhat-marketplace-qttdj" Dec 03 11:24:35 crc kubenswrapper[5052]: I1203 11:24:35.261657 5052 generic.go:334] "Generic (PLEG): container finished" podID="1e97c072-bdcc-4095-9a87-ddf7d4f2d22b" containerID="4d0e97a334afc8b927d8b98a681cfab0c25dceea8bee70eac0ac81cdb025e397" exitCode=0 Dec 03 11:24:35 crc kubenswrapper[5052]: I1203 11:24:35.261731 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-operators-nq2qk" event={"ID":"1e97c072-bdcc-4095-9a87-ddf7d4f2d22b","Type":"ContainerDied","Data":"4d0e97a334afc8b927d8b98a681cfab0c25dceea8bee70eac0ac81cdb025e397"} Dec 03 11:24:35 crc kubenswrapper[5052]: I1203 11:24:35.263112 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-66df7c8f76-tth5j" event={"ID":"0a3d2d53-efdd-49bb-bca9-90285d88a220","Type":"ContainerStarted","Data":"bb9c6abd9612c905dbcef734629480242b5a684d6720770a76efcb24fe627c8a"} Dec 03 11:24:35 crc kubenswrapper[5052]: I1203 11:24:35.267262 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/community-operators-r8txz" event={"ID":"f1ea0ffb-1062-4713-908b-55c0b07c49ee","Type":"ContainerDied","Data":"f0c85f21f278dcf751b76f25d9cea56b5c872ef44a26cff908d2223a33ee9fe0"} Dec 03 11:24:35 crc kubenswrapper[5052]: I1203 11:24:35.267309 5052 scope.go:117] "RemoveContainer" containerID="84a10027d57c8f6d30f6e4c7f8065d6749b27b1a001972b6421953cced8ece00" Dec 03 11:24:35 crc kubenswrapper[5052]: I1203 11:24:35.267394 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/community-operators-r8txz" Dec 03 11:24:35 crc kubenswrapper[5052]: I1203 11:24:35.272464 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-marketplace-qttdj" event={"ID":"8cf9b98e-2ea9-47ee-bd1b-a913714b7668","Type":"ContainerDied","Data":"8a054a130d2fc5a4999a05716d05febba7f18f9a4fdb1f6757f8a0424ee6fb4d"} Dec 03 11:24:35 crc kubenswrapper[5052]: I1203 11:24:35.272567 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/redhat-marketplace-qttdj" Dec 03 11:24:35 crc kubenswrapper[5052]: I1203 11:24:35.313186 5052 scope.go:117] "RemoveContainer" containerID="f022e6c81efcef390965c816054aeb1d50773210788dd511dedc372063167377" Dec 03 11:24:35 crc kubenswrapper[5052]: I1203 11:24:35.317592 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openshift-marketplace/community-operators-r8txz"] Dec 03 11:24:35 crc kubenswrapper[5052]: I1203 11:24:35.321176 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openshift-marketplace/community-operators-r8txz"] Dec 03 11:24:35 crc kubenswrapper[5052]: I1203 11:24:35.350166 5052 scope.go:117] "RemoveContainer" containerID="96f8f369bca7603b9c66f19aa6ff55cd19d2bae1429e5887b389cf9727e18387" Dec 03 11:24:35 crc kubenswrapper[5052]: I1203 11:24:35.371788 5052 scope.go:117] "RemoveContainer" containerID="1e2172f73a4eb24ef8a5086bfa2b3d1c4e398d6dcf33426b05c5a0016d33f018" Dec 03 11:24:35 crc kubenswrapper[5052]: I1203 11:24:35.392937 5052 scope.go:117] "RemoveContainer" containerID="02ecb4effb412198bc8c98b05696cad847e453c295fb6a2f92dc892af695a3fa" Dec 03 11:24:35 crc kubenswrapper[5052]: I1203 11:24:35.398864 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/8cf9b98e-2ea9-47ee-bd1b-a913714b7668-catalog-content\") pod \"8cf9b98e-2ea9-47ee-bd1b-a913714b7668\" (UID: \"8cf9b98e-2ea9-47ee-bd1b-a913714b7668\") " Dec 03 11:24:35 crc kubenswrapper[5052]: I1203 11:24:35.398959 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lssp4\" (UniqueName: \"kubernetes.io/projected/8cf9b98e-2ea9-47ee-bd1b-a913714b7668-kube-api-access-lssp4\") pod \"8cf9b98e-2ea9-47ee-bd1b-a913714b7668\" (UID: \"8cf9b98e-2ea9-47ee-bd1b-a913714b7668\") " Dec 03 11:24:35 crc kubenswrapper[5052]: I1203 11:24:35.399014 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/8cf9b98e-2ea9-47ee-bd1b-a913714b7668-utilities\") pod \"8cf9b98e-2ea9-47ee-bd1b-a913714b7668\" (UID: \"8cf9b98e-2ea9-47ee-bd1b-a913714b7668\") " Dec 03 11:24:35 crc kubenswrapper[5052]: I1203 11:24:35.399930 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8cf9b98e-2ea9-47ee-bd1b-a913714b7668-utilities" (OuterVolumeSpecName: "utilities") pod "8cf9b98e-2ea9-47ee-bd1b-a913714b7668" (UID: "8cf9b98e-2ea9-47ee-bd1b-a913714b7668"). InnerVolumeSpecName "utilities". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:24:35 crc kubenswrapper[5052]: I1203 11:24:35.407770 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8cf9b98e-2ea9-47ee-bd1b-a913714b7668-kube-api-access-lssp4" (OuterVolumeSpecName: "kube-api-access-lssp4") pod "8cf9b98e-2ea9-47ee-bd1b-a913714b7668" (UID: "8cf9b98e-2ea9-47ee-bd1b-a913714b7668"). InnerVolumeSpecName "kube-api-access-lssp4". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:24:35 crc kubenswrapper[5052]: I1203 11:24:35.413439 5052 scope.go:117] "RemoveContainer" containerID="b7b104beab26c080f6cd35706b17f3334ca22c3f8adfcf8f8aa965994d754fe3" Dec 03 11:24:35 crc kubenswrapper[5052]: I1203 11:24:35.420566 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8cf9b98e-2ea9-47ee-bd1b-a913714b7668-catalog-content" (OuterVolumeSpecName: "catalog-content") pod "8cf9b98e-2ea9-47ee-bd1b-a913714b7668" (UID: "8cf9b98e-2ea9-47ee-bd1b-a913714b7668"). InnerVolumeSpecName "catalog-content". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:24:35 crc kubenswrapper[5052]: I1203 11:24:35.500564 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-lssp4\" (UniqueName: \"kubernetes.io/projected/8cf9b98e-2ea9-47ee-bd1b-a913714b7668-kube-api-access-lssp4\") on node \"crc\" DevicePath \"\"" Dec 03 11:24:35 crc kubenswrapper[5052]: I1203 11:24:35.500604 5052 reconciler_common.go:293] "Volume detached for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/8cf9b98e-2ea9-47ee-bd1b-a913714b7668-utilities\") on node \"crc\" DevicePath \"\"" Dec 03 11:24:35 crc kubenswrapper[5052]: I1203 11:24:35.500614 5052 reconciler_common.go:293] "Volume detached for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/8cf9b98e-2ea9-47ee-bd1b-a913714b7668-catalog-content\") on node \"crc\" DevicePath \"\"" Dec 03 11:24:35 crc kubenswrapper[5052]: I1203 11:24:35.599078 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openshift-marketplace/redhat-marketplace-qttdj"] Dec 03 11:24:35 crc kubenswrapper[5052]: I1203 11:24:35.603183 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openshift-marketplace/redhat-marketplace-qttdj"] Dec 03 11:24:35 crc kubenswrapper[5052]: I1203 11:24:35.681664 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3b1b1a53-eff1-4339-a82e-9b8400dd5604" path="/var/lib/kubelet/pods/3b1b1a53-eff1-4339-a82e-9b8400dd5604/volumes" Dec 03 11:24:35 crc kubenswrapper[5052]: I1203 11:24:35.682747 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8cf9b98e-2ea9-47ee-bd1b-a913714b7668" path="/var/lib/kubelet/pods/8cf9b98e-2ea9-47ee-bd1b-a913714b7668/volumes" Dec 03 11:24:35 crc kubenswrapper[5052]: I1203 11:24:35.683550 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f1ea0ffb-1062-4713-908b-55c0b07c49ee" path="/var/lib/kubelet/pods/f1ea0ffb-1062-4713-908b-55c0b07c49ee/volumes" Dec 03 11:24:36 crc kubenswrapper[5052]: I1203 11:24:36.282136 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openshift-image-registry/image-registry-66df7c8f76-tth5j" Dec 03 11:24:36 crc kubenswrapper[5052]: I1203 11:24:36.324072 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/image-registry-66df7c8f76-tth5j" podStartSLOduration=3.324053667 podStartE2EDuration="3.324053667s" podCreationTimestamp="2025-12-03 11:24:33 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:24:36.320673351 +0000 UTC m=+266.908915115" watchObservedRunningTime="2025-12-03 11:24:36.324053667 +0000 UTC m=+266.912295431" Dec 03 11:24:36 crc kubenswrapper[5052]: I1203 11:24:36.553452 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/redhat-operators-nq2qk" Dec 03 11:24:36 crc kubenswrapper[5052]: I1203 11:24:36.717273 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kgznh\" (UniqueName: \"kubernetes.io/projected/1e97c072-bdcc-4095-9a87-ddf7d4f2d22b-kube-api-access-kgznh\") pod \"1e97c072-bdcc-4095-9a87-ddf7d4f2d22b\" (UID: \"1e97c072-bdcc-4095-9a87-ddf7d4f2d22b\") " Dec 03 11:24:36 crc kubenswrapper[5052]: I1203 11:24:36.717358 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/1e97c072-bdcc-4095-9a87-ddf7d4f2d22b-catalog-content\") pod \"1e97c072-bdcc-4095-9a87-ddf7d4f2d22b\" (UID: \"1e97c072-bdcc-4095-9a87-ddf7d4f2d22b\") " Dec 03 11:24:36 crc kubenswrapper[5052]: I1203 11:24:36.717394 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/1e97c072-bdcc-4095-9a87-ddf7d4f2d22b-utilities\") pod \"1e97c072-bdcc-4095-9a87-ddf7d4f2d22b\" (UID: \"1e97c072-bdcc-4095-9a87-ddf7d4f2d22b\") " Dec 03 11:24:36 crc kubenswrapper[5052]: I1203 11:24:36.718429 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1e97c072-bdcc-4095-9a87-ddf7d4f2d22b-utilities" (OuterVolumeSpecName: "utilities") pod "1e97c072-bdcc-4095-9a87-ddf7d4f2d22b" (UID: "1e97c072-bdcc-4095-9a87-ddf7d4f2d22b"). InnerVolumeSpecName "utilities". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:24:36 crc kubenswrapper[5052]: I1203 11:24:36.719164 5052 reconciler_common.go:293] "Volume detached for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/1e97c072-bdcc-4095-9a87-ddf7d4f2d22b-utilities\") on node \"crc\" DevicePath \"\"" Dec 03 11:24:36 crc kubenswrapper[5052]: I1203 11:24:36.724797 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1e97c072-bdcc-4095-9a87-ddf7d4f2d22b-kube-api-access-kgznh" (OuterVolumeSpecName: "kube-api-access-kgznh") pod "1e97c072-bdcc-4095-9a87-ddf7d4f2d22b" (UID: "1e97c072-bdcc-4095-9a87-ddf7d4f2d22b"). InnerVolumeSpecName "kube-api-access-kgznh". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:24:36 crc kubenswrapper[5052]: I1203 11:24:36.821111 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-kgznh\" (UniqueName: \"kubernetes.io/projected/1e97c072-bdcc-4095-9a87-ddf7d4f2d22b-kube-api-access-kgznh\") on node \"crc\" DevicePath \"\"" Dec 03 11:24:36 crc kubenswrapper[5052]: I1203 11:24:36.851428 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1e97c072-bdcc-4095-9a87-ddf7d4f2d22b-catalog-content" (OuterVolumeSpecName: "catalog-content") pod "1e97c072-bdcc-4095-9a87-ddf7d4f2d22b" (UID: "1e97c072-bdcc-4095-9a87-ddf7d4f2d22b"). InnerVolumeSpecName "catalog-content". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:24:36 crc kubenswrapper[5052]: I1203 11:24:36.922281 5052 reconciler_common.go:293] "Volume detached for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/1e97c072-bdcc-4095-9a87-ddf7d4f2d22b-catalog-content\") on node \"crc\" DevicePath \"\"" Dec 03 11:24:37 crc kubenswrapper[5052]: I1203 11:24:37.288194 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-operators-nq2qk" event={"ID":"1e97c072-bdcc-4095-9a87-ddf7d4f2d22b","Type":"ContainerDied","Data":"6d126c9db48ca5aa9702f1a934a254b89f7620454dfb955777e1ae91618746bc"} Dec 03 11:24:37 crc kubenswrapper[5052]: I1203 11:24:37.288250 5052 scope.go:117] "RemoveContainer" containerID="4d0e97a334afc8b927d8b98a681cfab0c25dceea8bee70eac0ac81cdb025e397" Dec 03 11:24:37 crc kubenswrapper[5052]: I1203 11:24:37.288283 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/redhat-operators-nq2qk" Dec 03 11:24:37 crc kubenswrapper[5052]: I1203 11:24:37.303293 5052 scope.go:117] "RemoveContainer" containerID="911c84a397ed38c66cc5476f39883872f2440ea2cd24b75a8427853d654bdc4c" Dec 03 11:24:37 crc kubenswrapper[5052]: I1203 11:24:37.320850 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openshift-marketplace/redhat-operators-nq2qk"] Dec 03 11:24:37 crc kubenswrapper[5052]: I1203 11:24:37.324592 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openshift-marketplace/redhat-operators-nq2qk"] Dec 03 11:24:37 crc kubenswrapper[5052]: I1203 11:24:37.348826 5052 scope.go:117] "RemoveContainer" containerID="934b128bed629ee32297db05e254a0d31930e08ad1f1bbdc95695969db8a1c44" Dec 03 11:24:37 crc kubenswrapper[5052]: I1203 11:24:37.679181 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1e97c072-bdcc-4095-9a87-ddf7d4f2d22b" path="/var/lib/kubelet/pods/1e97c072-bdcc-4095-9a87-ddf7d4f2d22b/volumes" Dec 03 11:24:47 crc kubenswrapper[5052]: I1203 11:24:47.419519 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openshift-marketplace/certified-operators-ghqsf"] Dec 03 11:24:47 crc kubenswrapper[5052]: I1203 11:24:47.422221 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openshift-marketplace/certified-operators-ghqsf" podUID="dc6b87f0-562b-4fa9-85b3-ccdd0867374b" containerName="registry-server" containerID="cri-o://c3facd8f5aa5277fa03efbb624aef1a61ea9479ddb54b075d4bae1376f0a3224" gracePeriod=30 Dec 03 11:24:47 crc kubenswrapper[5052]: I1203 11:24:47.431631 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openshift-marketplace/community-operators-pxx88"] Dec 03 11:24:47 crc kubenswrapper[5052]: I1203 11:24:47.431925 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openshift-marketplace/community-operators-pxx88" podUID="a7228f2a-1d6d-4307-80e7-deebd47d995a" containerName="registry-server" containerID="cri-o://f9dbb7ff933337d17d118274887a9a69ba98abc650bca5b8b4185da0ab78b339" gracePeriod=30 Dec 03 11:24:47 crc kubenswrapper[5052]: I1203 11:24:47.444459 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openshift-marketplace/marketplace-operator-79b997595-kdr8l"] Dec 03 11:24:47 crc kubenswrapper[5052]: I1203 11:24:47.444741 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openshift-marketplace/marketplace-operator-79b997595-kdr8l" podUID="cac70dab-6e13-4693-ac66-c7552b62dfd7" containerName="marketplace-operator" containerID="cri-o://85ba752630aa58b2a1baa94ed4769d91d59327218485bf8f7d30a5cf17fe7c22" gracePeriod=30 Dec 03 11:24:47 crc kubenswrapper[5052]: I1203 11:24:47.450668 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openshift-marketplace/redhat-marketplace-gqhzd"] Dec 03 11:24:47 crc kubenswrapper[5052]: I1203 11:24:47.450986 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openshift-marketplace/redhat-marketplace-gqhzd" podUID="355008cc-8721-44c4-a1e0-3714b0086286" containerName="registry-server" containerID="cri-o://4c0a6dfbfc2568cb3f97d4312237572cc0135491d1360e4adda4523a9ddb4f70" gracePeriod=30 Dec 03 11:24:47 crc kubenswrapper[5052]: I1203 11:24:47.464383 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openshift-marketplace/redhat-operators-dwt2v"] Dec 03 11:24:47 crc kubenswrapper[5052]: I1203 11:24:47.464695 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openshift-marketplace/redhat-operators-dwt2v" podUID="afb87a28-1b7c-4b09-b186-f05da232f501" containerName="registry-server" containerID="cri-o://72b81e1c6f6bd6fb2bc6844709bf2e26f33ff52dc4552dfe11690d4fab2eed25" gracePeriod=30 Dec 03 11:24:47 crc kubenswrapper[5052]: I1203 11:24:47.470610 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openshift-marketplace/marketplace-operator-79b997595-sbjmz"] Dec 03 11:24:47 crc kubenswrapper[5052]: E1203 11:24:47.470920 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="f1ea0ffb-1062-4713-908b-55c0b07c49ee" containerName="extract-utilities" Dec 03 11:24:47 crc kubenswrapper[5052]: I1203 11:24:47.470939 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="f1ea0ffb-1062-4713-908b-55c0b07c49ee" containerName="extract-utilities" Dec 03 11:24:47 crc kubenswrapper[5052]: E1203 11:24:47.470950 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="8cf9b98e-2ea9-47ee-bd1b-a913714b7668" containerName="registry-server" Dec 03 11:24:47 crc kubenswrapper[5052]: I1203 11:24:47.470957 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="8cf9b98e-2ea9-47ee-bd1b-a913714b7668" containerName="registry-server" Dec 03 11:24:47 crc kubenswrapper[5052]: E1203 11:24:47.470966 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="3b1b1a53-eff1-4339-a82e-9b8400dd5604" containerName="extract-content" Dec 03 11:24:47 crc kubenswrapper[5052]: I1203 11:24:47.470972 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b1b1a53-eff1-4339-a82e-9b8400dd5604" containerName="extract-content" Dec 03 11:24:47 crc kubenswrapper[5052]: E1203 11:24:47.470979 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="f1ea0ffb-1062-4713-908b-55c0b07c49ee" containerName="extract-content" Dec 03 11:24:47 crc kubenswrapper[5052]: I1203 11:24:47.470986 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="f1ea0ffb-1062-4713-908b-55c0b07c49ee" containerName="extract-content" Dec 03 11:24:47 crc kubenswrapper[5052]: E1203 11:24:47.470993 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="f1ea0ffb-1062-4713-908b-55c0b07c49ee" containerName="registry-server" Dec 03 11:24:47 crc kubenswrapper[5052]: I1203 11:24:47.470998 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="f1ea0ffb-1062-4713-908b-55c0b07c49ee" containerName="registry-server" Dec 03 11:24:47 crc kubenswrapper[5052]: E1203 11:24:47.471011 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="8cf9b98e-2ea9-47ee-bd1b-a913714b7668" containerName="extract-content" Dec 03 11:24:47 crc kubenswrapper[5052]: I1203 11:24:47.471018 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="8cf9b98e-2ea9-47ee-bd1b-a913714b7668" containerName="extract-content" Dec 03 11:24:47 crc kubenswrapper[5052]: E1203 11:24:47.471030 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="8cf9b98e-2ea9-47ee-bd1b-a913714b7668" containerName="extract-utilities" Dec 03 11:24:47 crc kubenswrapper[5052]: I1203 11:24:47.471036 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="8cf9b98e-2ea9-47ee-bd1b-a913714b7668" containerName="extract-utilities" Dec 03 11:24:47 crc kubenswrapper[5052]: E1203 11:24:47.471054 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="1e97c072-bdcc-4095-9a87-ddf7d4f2d22b" containerName="extract-content" Dec 03 11:24:47 crc kubenswrapper[5052]: I1203 11:24:47.471061 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="1e97c072-bdcc-4095-9a87-ddf7d4f2d22b" containerName="extract-content" Dec 03 11:24:47 crc kubenswrapper[5052]: E1203 11:24:47.471068 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="1e97c072-bdcc-4095-9a87-ddf7d4f2d22b" containerName="extract-utilities" Dec 03 11:24:47 crc kubenswrapper[5052]: I1203 11:24:47.471073 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="1e97c072-bdcc-4095-9a87-ddf7d4f2d22b" containerName="extract-utilities" Dec 03 11:24:47 crc kubenswrapper[5052]: E1203 11:24:47.471082 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="3b1b1a53-eff1-4339-a82e-9b8400dd5604" containerName="extract-utilities" Dec 03 11:24:47 crc kubenswrapper[5052]: I1203 11:24:47.471088 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b1b1a53-eff1-4339-a82e-9b8400dd5604" containerName="extract-utilities" Dec 03 11:24:47 crc kubenswrapper[5052]: E1203 11:24:47.471096 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="3b1b1a53-eff1-4339-a82e-9b8400dd5604" containerName="registry-server" Dec 03 11:24:47 crc kubenswrapper[5052]: I1203 11:24:47.471109 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b1b1a53-eff1-4339-a82e-9b8400dd5604" containerName="registry-server" Dec 03 11:24:47 crc kubenswrapper[5052]: E1203 11:24:47.471120 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="1e97c072-bdcc-4095-9a87-ddf7d4f2d22b" containerName="registry-server" Dec 03 11:24:47 crc kubenswrapper[5052]: I1203 11:24:47.471126 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="1e97c072-bdcc-4095-9a87-ddf7d4f2d22b" containerName="registry-server" Dec 03 11:24:47 crc kubenswrapper[5052]: I1203 11:24:47.471233 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="8cf9b98e-2ea9-47ee-bd1b-a913714b7668" containerName="registry-server" Dec 03 11:24:47 crc kubenswrapper[5052]: I1203 11:24:47.471247 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="1e97c072-bdcc-4095-9a87-ddf7d4f2d22b" containerName="registry-server" Dec 03 11:24:47 crc kubenswrapper[5052]: I1203 11:24:47.471256 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="f1ea0ffb-1062-4713-908b-55c0b07c49ee" containerName="registry-server" Dec 03 11:24:47 crc kubenswrapper[5052]: I1203 11:24:47.471268 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="3b1b1a53-eff1-4339-a82e-9b8400dd5604" containerName="registry-server" Dec 03 11:24:47 crc kubenswrapper[5052]: I1203 11:24:47.471790 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/marketplace-operator-79b997595-sbjmz" Dec 03 11:24:47 crc kubenswrapper[5052]: I1203 11:24:47.478314 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/marketplace-operator-79b997595-sbjmz"] Dec 03 11:24:47 crc kubenswrapper[5052]: I1203 11:24:47.488977 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"marketplace-trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ce27ce03-4ec0-418f-95d8-c74102f26bbd-marketplace-trusted-ca\") pod \"marketplace-operator-79b997595-sbjmz\" (UID: \"ce27ce03-4ec0-418f-95d8-c74102f26bbd\") " pod="openshift-marketplace/marketplace-operator-79b997595-sbjmz" Dec 03 11:24:47 crc kubenswrapper[5052]: I1203 11:24:47.590346 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"marketplace-operator-metrics\" (UniqueName: \"kubernetes.io/secret/ce27ce03-4ec0-418f-95d8-c74102f26bbd-marketplace-operator-metrics\") pod \"marketplace-operator-79b997595-sbjmz\" (UID: \"ce27ce03-4ec0-418f-95d8-c74102f26bbd\") " pod="openshift-marketplace/marketplace-operator-79b997595-sbjmz" Dec 03 11:24:47 crc kubenswrapper[5052]: I1203 11:24:47.590653 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kp45d\" (UniqueName: \"kubernetes.io/projected/ce27ce03-4ec0-418f-95d8-c74102f26bbd-kube-api-access-kp45d\") pod \"marketplace-operator-79b997595-sbjmz\" (UID: \"ce27ce03-4ec0-418f-95d8-c74102f26bbd\") " pod="openshift-marketplace/marketplace-operator-79b997595-sbjmz" Dec 03 11:24:47 crc kubenswrapper[5052]: I1203 11:24:47.590692 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"marketplace-trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ce27ce03-4ec0-418f-95d8-c74102f26bbd-marketplace-trusted-ca\") pod \"marketplace-operator-79b997595-sbjmz\" (UID: \"ce27ce03-4ec0-418f-95d8-c74102f26bbd\") " pod="openshift-marketplace/marketplace-operator-79b997595-sbjmz" Dec 03 11:24:47 crc kubenswrapper[5052]: I1203 11:24:47.592653 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"marketplace-trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ce27ce03-4ec0-418f-95d8-c74102f26bbd-marketplace-trusted-ca\") pod \"marketplace-operator-79b997595-sbjmz\" (UID: \"ce27ce03-4ec0-418f-95d8-c74102f26bbd\") " pod="openshift-marketplace/marketplace-operator-79b997595-sbjmz" Dec 03 11:24:47 crc kubenswrapper[5052]: I1203 11:24:47.691291 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-kp45d\" (UniqueName: \"kubernetes.io/projected/ce27ce03-4ec0-418f-95d8-c74102f26bbd-kube-api-access-kp45d\") pod \"marketplace-operator-79b997595-sbjmz\" (UID: \"ce27ce03-4ec0-418f-95d8-c74102f26bbd\") " pod="openshift-marketplace/marketplace-operator-79b997595-sbjmz" Dec 03 11:24:47 crc kubenswrapper[5052]: I1203 11:24:47.691408 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"marketplace-operator-metrics\" (UniqueName: \"kubernetes.io/secret/ce27ce03-4ec0-418f-95d8-c74102f26bbd-marketplace-operator-metrics\") pod \"marketplace-operator-79b997595-sbjmz\" (UID: \"ce27ce03-4ec0-418f-95d8-c74102f26bbd\") " pod="openshift-marketplace/marketplace-operator-79b997595-sbjmz" Dec 03 11:24:47 crc kubenswrapper[5052]: I1203 11:24:47.698891 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"marketplace-operator-metrics\" (UniqueName: \"kubernetes.io/secret/ce27ce03-4ec0-418f-95d8-c74102f26bbd-marketplace-operator-metrics\") pod \"marketplace-operator-79b997595-sbjmz\" (UID: \"ce27ce03-4ec0-418f-95d8-c74102f26bbd\") " pod="openshift-marketplace/marketplace-operator-79b997595-sbjmz" Dec 03 11:24:47 crc kubenswrapper[5052]: I1203 11:24:47.711142 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-kp45d\" (UniqueName: \"kubernetes.io/projected/ce27ce03-4ec0-418f-95d8-c74102f26bbd-kube-api-access-kp45d\") pod \"marketplace-operator-79b997595-sbjmz\" (UID: \"ce27ce03-4ec0-418f-95d8-c74102f26bbd\") " pod="openshift-marketplace/marketplace-operator-79b997595-sbjmz" Dec 03 11:24:47 crc kubenswrapper[5052]: I1203 11:24:47.789234 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/marketplace-operator-79b997595-sbjmz" Dec 03 11:24:47 crc kubenswrapper[5052]: I1203 11:24:47.948793 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/redhat-marketplace-gqhzd" Dec 03 11:24:48 crc kubenswrapper[5052]: I1203 11:24:48.096133 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qdx45\" (UniqueName: \"kubernetes.io/projected/355008cc-8721-44c4-a1e0-3714b0086286-kube-api-access-qdx45\") pod \"355008cc-8721-44c4-a1e0-3714b0086286\" (UID: \"355008cc-8721-44c4-a1e0-3714b0086286\") " Dec 03 11:24:48 crc kubenswrapper[5052]: I1203 11:24:48.096294 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/355008cc-8721-44c4-a1e0-3714b0086286-utilities\") pod \"355008cc-8721-44c4-a1e0-3714b0086286\" (UID: \"355008cc-8721-44c4-a1e0-3714b0086286\") " Dec 03 11:24:48 crc kubenswrapper[5052]: I1203 11:24:48.096331 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/355008cc-8721-44c4-a1e0-3714b0086286-catalog-content\") pod \"355008cc-8721-44c4-a1e0-3714b0086286\" (UID: \"355008cc-8721-44c4-a1e0-3714b0086286\") " Dec 03 11:24:48 crc kubenswrapper[5052]: I1203 11:24:48.097988 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/355008cc-8721-44c4-a1e0-3714b0086286-utilities" (OuterVolumeSpecName: "utilities") pod "355008cc-8721-44c4-a1e0-3714b0086286" (UID: "355008cc-8721-44c4-a1e0-3714b0086286"). InnerVolumeSpecName "utilities". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:24:48 crc kubenswrapper[5052]: I1203 11:24:48.110861 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/355008cc-8721-44c4-a1e0-3714b0086286-kube-api-access-qdx45" (OuterVolumeSpecName: "kube-api-access-qdx45") pod "355008cc-8721-44c4-a1e0-3714b0086286" (UID: "355008cc-8721-44c4-a1e0-3714b0086286"). InnerVolumeSpecName "kube-api-access-qdx45". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:24:48 crc kubenswrapper[5052]: I1203 11:24:48.125382 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/355008cc-8721-44c4-a1e0-3714b0086286-catalog-content" (OuterVolumeSpecName: "catalog-content") pod "355008cc-8721-44c4-a1e0-3714b0086286" (UID: "355008cc-8721-44c4-a1e0-3714b0086286"). InnerVolumeSpecName "catalog-content". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:24:48 crc kubenswrapper[5052]: I1203 11:24:48.146070 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/certified-operators-ghqsf" Dec 03 11:24:48 crc kubenswrapper[5052]: I1203 11:24:48.151152 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/marketplace-operator-79b997595-kdr8l" Dec 03 11:24:48 crc kubenswrapper[5052]: I1203 11:24:48.157011 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/community-operators-pxx88" Dec 03 11:24:48 crc kubenswrapper[5052]: I1203 11:24:48.197694 5052 reconciler_common.go:293] "Volume detached for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/355008cc-8721-44c4-a1e0-3714b0086286-utilities\") on node \"crc\" DevicePath \"\"" Dec 03 11:24:48 crc kubenswrapper[5052]: I1203 11:24:48.197713 5052 reconciler_common.go:293] "Volume detached for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/355008cc-8721-44c4-a1e0-3714b0086286-catalog-content\") on node \"crc\" DevicePath \"\"" Dec 03 11:24:48 crc kubenswrapper[5052]: I1203 11:24:48.197722 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-qdx45\" (UniqueName: \"kubernetes.io/projected/355008cc-8721-44c4-a1e0-3714b0086286-kube-api-access-qdx45\") on node \"crc\" DevicePath \"\"" Dec 03 11:24:48 crc kubenswrapper[5052]: I1203 11:24:48.238418 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/redhat-operators-dwt2v" Dec 03 11:24:48 crc kubenswrapper[5052]: I1203 11:24:48.298753 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"marketplace-operator-metrics\" (UniqueName: \"kubernetes.io/secret/cac70dab-6e13-4693-ac66-c7552b62dfd7-marketplace-operator-metrics\") pod \"cac70dab-6e13-4693-ac66-c7552b62dfd7\" (UID: \"cac70dab-6e13-4693-ac66-c7552b62dfd7\") " Dec 03 11:24:48 crc kubenswrapper[5052]: I1203 11:24:48.298840 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4cgvv\" (UniqueName: \"kubernetes.io/projected/cac70dab-6e13-4693-ac66-c7552b62dfd7-kube-api-access-4cgvv\") pod \"cac70dab-6e13-4693-ac66-c7552b62dfd7\" (UID: \"cac70dab-6e13-4693-ac66-c7552b62dfd7\") " Dec 03 11:24:48 crc kubenswrapper[5052]: I1203 11:24:48.298885 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/a7228f2a-1d6d-4307-80e7-deebd47d995a-utilities\") pod \"a7228f2a-1d6d-4307-80e7-deebd47d995a\" (UID: \"a7228f2a-1d6d-4307-80e7-deebd47d995a\") " Dec 03 11:24:48 crc kubenswrapper[5052]: I1203 11:24:48.298936 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z6xw8\" (UniqueName: \"kubernetes.io/projected/dc6b87f0-562b-4fa9-85b3-ccdd0867374b-kube-api-access-z6xw8\") pod \"dc6b87f0-562b-4fa9-85b3-ccdd0867374b\" (UID: \"dc6b87f0-562b-4fa9-85b3-ccdd0867374b\") " Dec 03 11:24:48 crc kubenswrapper[5052]: I1203 11:24:48.299021 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/afb87a28-1b7c-4b09-b186-f05da232f501-catalog-content\") pod \"afb87a28-1b7c-4b09-b186-f05da232f501\" (UID: \"afb87a28-1b7c-4b09-b186-f05da232f501\") " Dec 03 11:24:48 crc kubenswrapper[5052]: I1203 11:24:48.299036 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/afb87a28-1b7c-4b09-b186-f05da232f501-utilities\") pod \"afb87a28-1b7c-4b09-b186-f05da232f501\" (UID: \"afb87a28-1b7c-4b09-b186-f05da232f501\") " Dec 03 11:24:48 crc kubenswrapper[5052]: I1203 11:24:48.299065 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"marketplace-trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cac70dab-6e13-4693-ac66-c7552b62dfd7-marketplace-trusted-ca\") pod \"cac70dab-6e13-4693-ac66-c7552b62dfd7\" (UID: \"cac70dab-6e13-4693-ac66-c7552b62dfd7\") " Dec 03 11:24:48 crc kubenswrapper[5052]: I1203 11:24:48.299090 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/dc6b87f0-562b-4fa9-85b3-ccdd0867374b-catalog-content\") pod \"dc6b87f0-562b-4fa9-85b3-ccdd0867374b\" (UID: \"dc6b87f0-562b-4fa9-85b3-ccdd0867374b\") " Dec 03 11:24:48 crc kubenswrapper[5052]: I1203 11:24:48.299112 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4xqqz\" (UniqueName: \"kubernetes.io/projected/a7228f2a-1d6d-4307-80e7-deebd47d995a-kube-api-access-4xqqz\") pod \"a7228f2a-1d6d-4307-80e7-deebd47d995a\" (UID: \"a7228f2a-1d6d-4307-80e7-deebd47d995a\") " Dec 03 11:24:48 crc kubenswrapper[5052]: I1203 11:24:48.299150 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/dc6b87f0-562b-4fa9-85b3-ccdd0867374b-utilities\") pod \"dc6b87f0-562b-4fa9-85b3-ccdd0867374b\" (UID: \"dc6b87f0-562b-4fa9-85b3-ccdd0867374b\") " Dec 03 11:24:48 crc kubenswrapper[5052]: I1203 11:24:48.299174 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/a7228f2a-1d6d-4307-80e7-deebd47d995a-catalog-content\") pod \"a7228f2a-1d6d-4307-80e7-deebd47d995a\" (UID: \"a7228f2a-1d6d-4307-80e7-deebd47d995a\") " Dec 03 11:24:48 crc kubenswrapper[5052]: I1203 11:24:48.299690 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a7228f2a-1d6d-4307-80e7-deebd47d995a-utilities" (OuterVolumeSpecName: "utilities") pod "a7228f2a-1d6d-4307-80e7-deebd47d995a" (UID: "a7228f2a-1d6d-4307-80e7-deebd47d995a"). InnerVolumeSpecName "utilities". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:24:48 crc kubenswrapper[5052]: I1203 11:24:48.299728 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cac70dab-6e13-4693-ac66-c7552b62dfd7-marketplace-trusted-ca" (OuterVolumeSpecName: "marketplace-trusted-ca") pod "cac70dab-6e13-4693-ac66-c7552b62dfd7" (UID: "cac70dab-6e13-4693-ac66-c7552b62dfd7"). InnerVolumeSpecName "marketplace-trusted-ca". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:24:48 crc kubenswrapper[5052]: I1203 11:24:48.300234 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/afb87a28-1b7c-4b09-b186-f05da232f501-utilities" (OuterVolumeSpecName: "utilities") pod "afb87a28-1b7c-4b09-b186-f05da232f501" (UID: "afb87a28-1b7c-4b09-b186-f05da232f501"). InnerVolumeSpecName "utilities". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:24:48 crc kubenswrapper[5052]: I1203 11:24:48.300747 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc6b87f0-562b-4fa9-85b3-ccdd0867374b-utilities" (OuterVolumeSpecName: "utilities") pod "dc6b87f0-562b-4fa9-85b3-ccdd0867374b" (UID: "dc6b87f0-562b-4fa9-85b3-ccdd0867374b"). InnerVolumeSpecName "utilities". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:24:48 crc kubenswrapper[5052]: I1203 11:24:48.303074 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cac70dab-6e13-4693-ac66-c7552b62dfd7-marketplace-operator-metrics" (OuterVolumeSpecName: "marketplace-operator-metrics") pod "cac70dab-6e13-4693-ac66-c7552b62dfd7" (UID: "cac70dab-6e13-4693-ac66-c7552b62dfd7"). InnerVolumeSpecName "marketplace-operator-metrics". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:24:48 crc kubenswrapper[5052]: I1203 11:24:48.306829 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dc6b87f0-562b-4fa9-85b3-ccdd0867374b-kube-api-access-z6xw8" (OuterVolumeSpecName: "kube-api-access-z6xw8") pod "dc6b87f0-562b-4fa9-85b3-ccdd0867374b" (UID: "dc6b87f0-562b-4fa9-85b3-ccdd0867374b"). InnerVolumeSpecName "kube-api-access-z6xw8". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:24:48 crc kubenswrapper[5052]: I1203 11:24:48.309823 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a7228f2a-1d6d-4307-80e7-deebd47d995a-kube-api-access-4xqqz" (OuterVolumeSpecName: "kube-api-access-4xqqz") pod "a7228f2a-1d6d-4307-80e7-deebd47d995a" (UID: "a7228f2a-1d6d-4307-80e7-deebd47d995a"). InnerVolumeSpecName "kube-api-access-4xqqz". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:24:48 crc kubenswrapper[5052]: I1203 11:24:48.310088 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cac70dab-6e13-4693-ac66-c7552b62dfd7-kube-api-access-4cgvv" (OuterVolumeSpecName: "kube-api-access-4cgvv") pod "cac70dab-6e13-4693-ac66-c7552b62dfd7" (UID: "cac70dab-6e13-4693-ac66-c7552b62dfd7"). InnerVolumeSpecName "kube-api-access-4cgvv". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:24:48 crc kubenswrapper[5052]: I1203 11:24:48.349232 5052 generic.go:334] "Generic (PLEG): container finished" podID="cac70dab-6e13-4693-ac66-c7552b62dfd7" containerID="85ba752630aa58b2a1baa94ed4769d91d59327218485bf8f7d30a5cf17fe7c22" exitCode=0 Dec 03 11:24:48 crc kubenswrapper[5052]: I1203 11:24:48.349286 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/marketplace-operator-79b997595-kdr8l" Dec 03 11:24:48 crc kubenswrapper[5052]: I1203 11:24:48.349337 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/marketplace-operator-79b997595-kdr8l" event={"ID":"cac70dab-6e13-4693-ac66-c7552b62dfd7","Type":"ContainerDied","Data":"85ba752630aa58b2a1baa94ed4769d91d59327218485bf8f7d30a5cf17fe7c22"} Dec 03 11:24:48 crc kubenswrapper[5052]: I1203 11:24:48.349535 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/marketplace-operator-79b997595-kdr8l" event={"ID":"cac70dab-6e13-4693-ac66-c7552b62dfd7","Type":"ContainerDied","Data":"31ae09a223e991501ffa4143ce1afff7c0f5b63af7a50b309ae6188a2dc5ce6e"} Dec 03 11:24:48 crc kubenswrapper[5052]: I1203 11:24:48.349565 5052 scope.go:117] "RemoveContainer" containerID="85ba752630aa58b2a1baa94ed4769d91d59327218485bf8f7d30a5cf17fe7c22" Dec 03 11:24:48 crc kubenswrapper[5052]: I1203 11:24:48.352903 5052 generic.go:334] "Generic (PLEG): container finished" podID="dc6b87f0-562b-4fa9-85b3-ccdd0867374b" containerID="c3facd8f5aa5277fa03efbb624aef1a61ea9479ddb54b075d4bae1376f0a3224" exitCode=0 Dec 03 11:24:48 crc kubenswrapper[5052]: I1203 11:24:48.353006 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/certified-operators-ghqsf" event={"ID":"dc6b87f0-562b-4fa9-85b3-ccdd0867374b","Type":"ContainerDied","Data":"c3facd8f5aa5277fa03efbb624aef1a61ea9479ddb54b075d4bae1376f0a3224"} Dec 03 11:24:48 crc kubenswrapper[5052]: I1203 11:24:48.353062 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/certified-operators-ghqsf" event={"ID":"dc6b87f0-562b-4fa9-85b3-ccdd0867374b","Type":"ContainerDied","Data":"53738140aeb647312cd091a13c101026a4b498818809d4b57a5c4e077ed6151a"} Dec 03 11:24:48 crc kubenswrapper[5052]: I1203 11:24:48.353169 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/certified-operators-ghqsf" Dec 03 11:24:48 crc kubenswrapper[5052]: I1203 11:24:48.361709 5052 generic.go:334] "Generic (PLEG): container finished" podID="355008cc-8721-44c4-a1e0-3714b0086286" containerID="4c0a6dfbfc2568cb3f97d4312237572cc0135491d1360e4adda4523a9ddb4f70" exitCode=0 Dec 03 11:24:48 crc kubenswrapper[5052]: I1203 11:24:48.361899 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/redhat-marketplace-gqhzd" Dec 03 11:24:48 crc kubenswrapper[5052]: I1203 11:24:48.361940 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-marketplace-gqhzd" event={"ID":"355008cc-8721-44c4-a1e0-3714b0086286","Type":"ContainerDied","Data":"4c0a6dfbfc2568cb3f97d4312237572cc0135491d1360e4adda4523a9ddb4f70"} Dec 03 11:24:48 crc kubenswrapper[5052]: I1203 11:24:48.362003 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-marketplace-gqhzd" event={"ID":"355008cc-8721-44c4-a1e0-3714b0086286","Type":"ContainerDied","Data":"11321015dcd20484cfaee0feff5586e629baa83de93c0b75a8a93885ed4f02a7"} Dec 03 11:24:48 crc kubenswrapper[5052]: I1203 11:24:48.375494 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc6b87f0-562b-4fa9-85b3-ccdd0867374b-catalog-content" (OuterVolumeSpecName: "catalog-content") pod "dc6b87f0-562b-4fa9-85b3-ccdd0867374b" (UID: "dc6b87f0-562b-4fa9-85b3-ccdd0867374b"). InnerVolumeSpecName "catalog-content". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:24:48 crc kubenswrapper[5052]: I1203 11:24:48.382025 5052 generic.go:334] "Generic (PLEG): container finished" podID="a7228f2a-1d6d-4307-80e7-deebd47d995a" containerID="f9dbb7ff933337d17d118274887a9a69ba98abc650bca5b8b4185da0ab78b339" exitCode=0 Dec 03 11:24:48 crc kubenswrapper[5052]: I1203 11:24:48.382111 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/community-operators-pxx88" event={"ID":"a7228f2a-1d6d-4307-80e7-deebd47d995a","Type":"ContainerDied","Data":"f9dbb7ff933337d17d118274887a9a69ba98abc650bca5b8b4185da0ab78b339"} Dec 03 11:24:48 crc kubenswrapper[5052]: I1203 11:24:48.382129 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/community-operators-pxx88" Dec 03 11:24:48 crc kubenswrapper[5052]: I1203 11:24:48.382150 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/community-operators-pxx88" event={"ID":"a7228f2a-1d6d-4307-80e7-deebd47d995a","Type":"ContainerDied","Data":"cdba1e921c314ba078fb0f85a22db6f04bafcb29d83bb480e5669a56a272b521"} Dec 03 11:24:48 crc kubenswrapper[5052]: I1203 11:24:48.386228 5052 scope.go:117] "RemoveContainer" containerID="85ba752630aa58b2a1baa94ed4769d91d59327218485bf8f7d30a5cf17fe7c22" Dec 03 11:24:48 crc kubenswrapper[5052]: E1203 11:24:48.386685 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"85ba752630aa58b2a1baa94ed4769d91d59327218485bf8f7d30a5cf17fe7c22\": container with ID starting with 85ba752630aa58b2a1baa94ed4769d91d59327218485bf8f7d30a5cf17fe7c22 not found: ID does not exist" containerID="85ba752630aa58b2a1baa94ed4769d91d59327218485bf8f7d30a5cf17fe7c22" Dec 03 11:24:48 crc kubenswrapper[5052]: I1203 11:24:48.386719 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"85ba752630aa58b2a1baa94ed4769d91d59327218485bf8f7d30a5cf17fe7c22"} err="failed to get container status \"85ba752630aa58b2a1baa94ed4769d91d59327218485bf8f7d30a5cf17fe7c22\": rpc error: code = NotFound desc = could not find container \"85ba752630aa58b2a1baa94ed4769d91d59327218485bf8f7d30a5cf17fe7c22\": container with ID starting with 85ba752630aa58b2a1baa94ed4769d91d59327218485bf8f7d30a5cf17fe7c22 not found: ID does not exist" Dec 03 11:24:48 crc kubenswrapper[5052]: I1203 11:24:48.386774 5052 scope.go:117] "RemoveContainer" containerID="c3facd8f5aa5277fa03efbb624aef1a61ea9479ddb54b075d4bae1376f0a3224" Dec 03 11:24:48 crc kubenswrapper[5052]: I1203 11:24:48.390000 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a7228f2a-1d6d-4307-80e7-deebd47d995a-catalog-content" (OuterVolumeSpecName: "catalog-content") pod "a7228f2a-1d6d-4307-80e7-deebd47d995a" (UID: "a7228f2a-1d6d-4307-80e7-deebd47d995a"). InnerVolumeSpecName "catalog-content". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:24:48 crc kubenswrapper[5052]: I1203 11:24:48.391139 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openshift-marketplace/marketplace-operator-79b997595-kdr8l"] Dec 03 11:24:48 crc kubenswrapper[5052]: I1203 11:24:48.401545 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dm25c\" (UniqueName: \"kubernetes.io/projected/afb87a28-1b7c-4b09-b186-f05da232f501-kube-api-access-dm25c\") pod \"afb87a28-1b7c-4b09-b186-f05da232f501\" (UID: \"afb87a28-1b7c-4b09-b186-f05da232f501\") " Dec 03 11:24:48 crc kubenswrapper[5052]: I1203 11:24:48.402003 5052 reconciler_common.go:293] "Volume detached for volume \"marketplace-operator-metrics\" (UniqueName: \"kubernetes.io/secret/cac70dab-6e13-4693-ac66-c7552b62dfd7-marketplace-operator-metrics\") on node \"crc\" DevicePath \"\"" Dec 03 11:24:48 crc kubenswrapper[5052]: I1203 11:24:48.402030 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-4cgvv\" (UniqueName: \"kubernetes.io/projected/cac70dab-6e13-4693-ac66-c7552b62dfd7-kube-api-access-4cgvv\") on node \"crc\" DevicePath \"\"" Dec 03 11:24:48 crc kubenswrapper[5052]: I1203 11:24:48.402043 5052 reconciler_common.go:293] "Volume detached for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/a7228f2a-1d6d-4307-80e7-deebd47d995a-utilities\") on node \"crc\" DevicePath \"\"" Dec 03 11:24:48 crc kubenswrapper[5052]: I1203 11:24:48.402057 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-z6xw8\" (UniqueName: \"kubernetes.io/projected/dc6b87f0-562b-4fa9-85b3-ccdd0867374b-kube-api-access-z6xw8\") on node \"crc\" DevicePath \"\"" Dec 03 11:24:48 crc kubenswrapper[5052]: I1203 11:24:48.402068 5052 reconciler_common.go:293] "Volume detached for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/afb87a28-1b7c-4b09-b186-f05da232f501-utilities\") on node \"crc\" DevicePath \"\"" Dec 03 11:24:48 crc kubenswrapper[5052]: I1203 11:24:48.402079 5052 reconciler_common.go:293] "Volume detached for volume \"marketplace-trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cac70dab-6e13-4693-ac66-c7552b62dfd7-marketplace-trusted-ca\") on node \"crc\" DevicePath \"\"" Dec 03 11:24:48 crc kubenswrapper[5052]: I1203 11:24:48.402091 5052 reconciler_common.go:293] "Volume detached for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/dc6b87f0-562b-4fa9-85b3-ccdd0867374b-catalog-content\") on node \"crc\" DevicePath \"\"" Dec 03 11:24:48 crc kubenswrapper[5052]: I1203 11:24:48.402106 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-4xqqz\" (UniqueName: \"kubernetes.io/projected/a7228f2a-1d6d-4307-80e7-deebd47d995a-kube-api-access-4xqqz\") on node \"crc\" DevicePath \"\"" Dec 03 11:24:48 crc kubenswrapper[5052]: I1203 11:24:48.402118 5052 reconciler_common.go:293] "Volume detached for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/dc6b87f0-562b-4fa9-85b3-ccdd0867374b-utilities\") on node \"crc\" DevicePath \"\"" Dec 03 11:24:48 crc kubenswrapper[5052]: I1203 11:24:48.402131 5052 reconciler_common.go:293] "Volume detached for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/a7228f2a-1d6d-4307-80e7-deebd47d995a-catalog-content\") on node \"crc\" DevicePath \"\"" Dec 03 11:24:48 crc kubenswrapper[5052]: I1203 11:24:48.408101 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openshift-marketplace/marketplace-operator-79b997595-kdr8l"] Dec 03 11:24:48 crc kubenswrapper[5052]: I1203 11:24:48.410059 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/afb87a28-1b7c-4b09-b186-f05da232f501-kube-api-access-dm25c" (OuterVolumeSpecName: "kube-api-access-dm25c") pod "afb87a28-1b7c-4b09-b186-f05da232f501" (UID: "afb87a28-1b7c-4b09-b186-f05da232f501"). InnerVolumeSpecName "kube-api-access-dm25c". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:24:48 crc kubenswrapper[5052]: I1203 11:24:48.412672 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/marketplace-operator-79b997595-sbjmz"] Dec 03 11:24:48 crc kubenswrapper[5052]: I1203 11:24:48.415347 5052 generic.go:334] "Generic (PLEG): container finished" podID="afb87a28-1b7c-4b09-b186-f05da232f501" containerID="72b81e1c6f6bd6fb2bc6844709bf2e26f33ff52dc4552dfe11690d4fab2eed25" exitCode=0 Dec 03 11:24:48 crc kubenswrapper[5052]: I1203 11:24:48.415488 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/redhat-operators-dwt2v" Dec 03 11:24:48 crc kubenswrapper[5052]: I1203 11:24:48.415584 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-operators-dwt2v" event={"ID":"afb87a28-1b7c-4b09-b186-f05da232f501","Type":"ContainerDied","Data":"72b81e1c6f6bd6fb2bc6844709bf2e26f33ff52dc4552dfe11690d4fab2eed25"} Dec 03 11:24:48 crc kubenswrapper[5052]: I1203 11:24:48.415787 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-operators-dwt2v" event={"ID":"afb87a28-1b7c-4b09-b186-f05da232f501","Type":"ContainerDied","Data":"6378464b7415353cf4fc4b0eb25f32886db80f0cb73e423e6879d3bfab9cb9ee"} Dec 03 11:24:48 crc kubenswrapper[5052]: I1203 11:24:48.421632 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openshift-marketplace/redhat-marketplace-gqhzd"] Dec 03 11:24:48 crc kubenswrapper[5052]: I1203 11:24:48.425799 5052 scope.go:117] "RemoveContainer" containerID="10f0e44d53a1f23dd54e713776978c8be6ae2d56d9821498094ebf59154f9352" Dec 03 11:24:48 crc kubenswrapper[5052]: I1203 11:24:48.437615 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/afb87a28-1b7c-4b09-b186-f05da232f501-catalog-content" (OuterVolumeSpecName: "catalog-content") pod "afb87a28-1b7c-4b09-b186-f05da232f501" (UID: "afb87a28-1b7c-4b09-b186-f05da232f501"). InnerVolumeSpecName "catalog-content". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:24:48 crc kubenswrapper[5052]: I1203 11:24:48.450714 5052 scope.go:117] "RemoveContainer" containerID="1767d68b4de95fcf8cb420d32b3fc4b35d9a8358bcee6bbbab6a4fe5d7edbcbd" Dec 03 11:24:48 crc kubenswrapper[5052]: I1203 11:24:48.453415 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openshift-marketplace/redhat-marketplace-gqhzd"] Dec 03 11:24:48 crc kubenswrapper[5052]: I1203 11:24:48.468331 5052 scope.go:117] "RemoveContainer" containerID="c3facd8f5aa5277fa03efbb624aef1a61ea9479ddb54b075d4bae1376f0a3224" Dec 03 11:24:48 crc kubenswrapper[5052]: E1203 11:24:48.468871 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c3facd8f5aa5277fa03efbb624aef1a61ea9479ddb54b075d4bae1376f0a3224\": container with ID starting with c3facd8f5aa5277fa03efbb624aef1a61ea9479ddb54b075d4bae1376f0a3224 not found: ID does not exist" containerID="c3facd8f5aa5277fa03efbb624aef1a61ea9479ddb54b075d4bae1376f0a3224" Dec 03 11:24:48 crc kubenswrapper[5052]: I1203 11:24:48.468911 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c3facd8f5aa5277fa03efbb624aef1a61ea9479ddb54b075d4bae1376f0a3224"} err="failed to get container status \"c3facd8f5aa5277fa03efbb624aef1a61ea9479ddb54b075d4bae1376f0a3224\": rpc error: code = NotFound desc = could not find container \"c3facd8f5aa5277fa03efbb624aef1a61ea9479ddb54b075d4bae1376f0a3224\": container with ID starting with c3facd8f5aa5277fa03efbb624aef1a61ea9479ddb54b075d4bae1376f0a3224 not found: ID does not exist" Dec 03 11:24:48 crc kubenswrapper[5052]: I1203 11:24:48.468940 5052 scope.go:117] "RemoveContainer" containerID="10f0e44d53a1f23dd54e713776978c8be6ae2d56d9821498094ebf59154f9352" Dec 03 11:24:48 crc kubenswrapper[5052]: E1203 11:24:48.469818 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"10f0e44d53a1f23dd54e713776978c8be6ae2d56d9821498094ebf59154f9352\": container with ID starting with 10f0e44d53a1f23dd54e713776978c8be6ae2d56d9821498094ebf59154f9352 not found: ID does not exist" containerID="10f0e44d53a1f23dd54e713776978c8be6ae2d56d9821498094ebf59154f9352" Dec 03 11:24:48 crc kubenswrapper[5052]: I1203 11:24:48.469857 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"10f0e44d53a1f23dd54e713776978c8be6ae2d56d9821498094ebf59154f9352"} err="failed to get container status \"10f0e44d53a1f23dd54e713776978c8be6ae2d56d9821498094ebf59154f9352\": rpc error: code = NotFound desc = could not find container \"10f0e44d53a1f23dd54e713776978c8be6ae2d56d9821498094ebf59154f9352\": container with ID starting with 10f0e44d53a1f23dd54e713776978c8be6ae2d56d9821498094ebf59154f9352 not found: ID does not exist" Dec 03 11:24:48 crc kubenswrapper[5052]: I1203 11:24:48.469886 5052 scope.go:117] "RemoveContainer" containerID="1767d68b4de95fcf8cb420d32b3fc4b35d9a8358bcee6bbbab6a4fe5d7edbcbd" Dec 03 11:24:48 crc kubenswrapper[5052]: E1203 11:24:48.470175 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1767d68b4de95fcf8cb420d32b3fc4b35d9a8358bcee6bbbab6a4fe5d7edbcbd\": container with ID starting with 1767d68b4de95fcf8cb420d32b3fc4b35d9a8358bcee6bbbab6a4fe5d7edbcbd not found: ID does not exist" containerID="1767d68b4de95fcf8cb420d32b3fc4b35d9a8358bcee6bbbab6a4fe5d7edbcbd" Dec 03 11:24:48 crc kubenswrapper[5052]: I1203 11:24:48.470200 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1767d68b4de95fcf8cb420d32b3fc4b35d9a8358bcee6bbbab6a4fe5d7edbcbd"} err="failed to get container status \"1767d68b4de95fcf8cb420d32b3fc4b35d9a8358bcee6bbbab6a4fe5d7edbcbd\": rpc error: code = NotFound desc = could not find container \"1767d68b4de95fcf8cb420d32b3fc4b35d9a8358bcee6bbbab6a4fe5d7edbcbd\": container with ID starting with 1767d68b4de95fcf8cb420d32b3fc4b35d9a8358bcee6bbbab6a4fe5d7edbcbd not found: ID does not exist" Dec 03 11:24:48 crc kubenswrapper[5052]: I1203 11:24:48.470215 5052 scope.go:117] "RemoveContainer" containerID="4c0a6dfbfc2568cb3f97d4312237572cc0135491d1360e4adda4523a9ddb4f70" Dec 03 11:24:48 crc kubenswrapper[5052]: I1203 11:24:48.485568 5052 scope.go:117] "RemoveContainer" containerID="33250962c095d7fb6ff9072a7d197c57c2128c134927c39a68e1d0e51ff9db6e" Dec 03 11:24:48 crc kubenswrapper[5052]: I1203 11:24:48.508914 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-dm25c\" (UniqueName: \"kubernetes.io/projected/afb87a28-1b7c-4b09-b186-f05da232f501-kube-api-access-dm25c\") on node \"crc\" DevicePath \"\"" Dec 03 11:24:48 crc kubenswrapper[5052]: I1203 11:24:48.508943 5052 reconciler_common.go:293] "Volume detached for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/afb87a28-1b7c-4b09-b186-f05da232f501-catalog-content\") on node \"crc\" DevicePath \"\"" Dec 03 11:24:48 crc kubenswrapper[5052]: I1203 11:24:48.510975 5052 scope.go:117] "RemoveContainer" containerID="c5ecedf32f587c013691fd61e77a5a2a4ef4e820227c1e4e265c3ef4d180defb" Dec 03 11:24:48 crc kubenswrapper[5052]: I1203 11:24:48.526303 5052 scope.go:117] "RemoveContainer" containerID="4c0a6dfbfc2568cb3f97d4312237572cc0135491d1360e4adda4523a9ddb4f70" Dec 03 11:24:48 crc kubenswrapper[5052]: E1203 11:24:48.526805 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4c0a6dfbfc2568cb3f97d4312237572cc0135491d1360e4adda4523a9ddb4f70\": container with ID starting with 4c0a6dfbfc2568cb3f97d4312237572cc0135491d1360e4adda4523a9ddb4f70 not found: ID does not exist" containerID="4c0a6dfbfc2568cb3f97d4312237572cc0135491d1360e4adda4523a9ddb4f70" Dec 03 11:24:48 crc kubenswrapper[5052]: I1203 11:24:48.526843 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4c0a6dfbfc2568cb3f97d4312237572cc0135491d1360e4adda4523a9ddb4f70"} err="failed to get container status \"4c0a6dfbfc2568cb3f97d4312237572cc0135491d1360e4adda4523a9ddb4f70\": rpc error: code = NotFound desc = could not find container \"4c0a6dfbfc2568cb3f97d4312237572cc0135491d1360e4adda4523a9ddb4f70\": container with ID starting with 4c0a6dfbfc2568cb3f97d4312237572cc0135491d1360e4adda4523a9ddb4f70 not found: ID does not exist" Dec 03 11:24:48 crc kubenswrapper[5052]: I1203 11:24:48.526873 5052 scope.go:117] "RemoveContainer" containerID="33250962c095d7fb6ff9072a7d197c57c2128c134927c39a68e1d0e51ff9db6e" Dec 03 11:24:48 crc kubenswrapper[5052]: E1203 11:24:48.527218 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"33250962c095d7fb6ff9072a7d197c57c2128c134927c39a68e1d0e51ff9db6e\": container with ID starting with 33250962c095d7fb6ff9072a7d197c57c2128c134927c39a68e1d0e51ff9db6e not found: ID does not exist" containerID="33250962c095d7fb6ff9072a7d197c57c2128c134927c39a68e1d0e51ff9db6e" Dec 03 11:24:48 crc kubenswrapper[5052]: I1203 11:24:48.527250 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"33250962c095d7fb6ff9072a7d197c57c2128c134927c39a68e1d0e51ff9db6e"} err="failed to get container status \"33250962c095d7fb6ff9072a7d197c57c2128c134927c39a68e1d0e51ff9db6e\": rpc error: code = NotFound desc = could not find container \"33250962c095d7fb6ff9072a7d197c57c2128c134927c39a68e1d0e51ff9db6e\": container with ID starting with 33250962c095d7fb6ff9072a7d197c57c2128c134927c39a68e1d0e51ff9db6e not found: ID does not exist" Dec 03 11:24:48 crc kubenswrapper[5052]: I1203 11:24:48.527280 5052 scope.go:117] "RemoveContainer" containerID="c5ecedf32f587c013691fd61e77a5a2a4ef4e820227c1e4e265c3ef4d180defb" Dec 03 11:24:48 crc kubenswrapper[5052]: E1203 11:24:48.527570 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c5ecedf32f587c013691fd61e77a5a2a4ef4e820227c1e4e265c3ef4d180defb\": container with ID starting with c5ecedf32f587c013691fd61e77a5a2a4ef4e820227c1e4e265c3ef4d180defb not found: ID does not exist" containerID="c5ecedf32f587c013691fd61e77a5a2a4ef4e820227c1e4e265c3ef4d180defb" Dec 03 11:24:48 crc kubenswrapper[5052]: I1203 11:24:48.527593 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c5ecedf32f587c013691fd61e77a5a2a4ef4e820227c1e4e265c3ef4d180defb"} err="failed to get container status \"c5ecedf32f587c013691fd61e77a5a2a4ef4e820227c1e4e265c3ef4d180defb\": rpc error: code = NotFound desc = could not find container \"c5ecedf32f587c013691fd61e77a5a2a4ef4e820227c1e4e265c3ef4d180defb\": container with ID starting with c5ecedf32f587c013691fd61e77a5a2a4ef4e820227c1e4e265c3ef4d180defb not found: ID does not exist" Dec 03 11:24:48 crc kubenswrapper[5052]: I1203 11:24:48.527607 5052 scope.go:117] "RemoveContainer" containerID="f9dbb7ff933337d17d118274887a9a69ba98abc650bca5b8b4185da0ab78b339" Dec 03 11:24:48 crc kubenswrapper[5052]: I1203 11:24:48.540633 5052 scope.go:117] "RemoveContainer" containerID="0443c848089499f8994f9881003129bf06532281a1ddd16281246884dd44408f" Dec 03 11:24:48 crc kubenswrapper[5052]: I1203 11:24:48.553301 5052 scope.go:117] "RemoveContainer" containerID="f165c5dd9b74405cb0258ec415d7cf2852fda1d2ba6fadfe03c30d3730fa500f" Dec 03 11:24:48 crc kubenswrapper[5052]: I1203 11:24:48.566689 5052 scope.go:117] "RemoveContainer" containerID="f9dbb7ff933337d17d118274887a9a69ba98abc650bca5b8b4185da0ab78b339" Dec 03 11:24:48 crc kubenswrapper[5052]: E1203 11:24:48.567076 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f9dbb7ff933337d17d118274887a9a69ba98abc650bca5b8b4185da0ab78b339\": container with ID starting with f9dbb7ff933337d17d118274887a9a69ba98abc650bca5b8b4185da0ab78b339 not found: ID does not exist" containerID="f9dbb7ff933337d17d118274887a9a69ba98abc650bca5b8b4185da0ab78b339" Dec 03 11:24:48 crc kubenswrapper[5052]: I1203 11:24:48.567109 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f9dbb7ff933337d17d118274887a9a69ba98abc650bca5b8b4185da0ab78b339"} err="failed to get container status \"f9dbb7ff933337d17d118274887a9a69ba98abc650bca5b8b4185da0ab78b339\": rpc error: code = NotFound desc = could not find container \"f9dbb7ff933337d17d118274887a9a69ba98abc650bca5b8b4185da0ab78b339\": container with ID starting with f9dbb7ff933337d17d118274887a9a69ba98abc650bca5b8b4185da0ab78b339 not found: ID does not exist" Dec 03 11:24:48 crc kubenswrapper[5052]: I1203 11:24:48.567133 5052 scope.go:117] "RemoveContainer" containerID="0443c848089499f8994f9881003129bf06532281a1ddd16281246884dd44408f" Dec 03 11:24:48 crc kubenswrapper[5052]: E1203 11:24:48.567787 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0443c848089499f8994f9881003129bf06532281a1ddd16281246884dd44408f\": container with ID starting with 0443c848089499f8994f9881003129bf06532281a1ddd16281246884dd44408f not found: ID does not exist" containerID="0443c848089499f8994f9881003129bf06532281a1ddd16281246884dd44408f" Dec 03 11:24:48 crc kubenswrapper[5052]: I1203 11:24:48.567899 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0443c848089499f8994f9881003129bf06532281a1ddd16281246884dd44408f"} err="failed to get container status \"0443c848089499f8994f9881003129bf06532281a1ddd16281246884dd44408f\": rpc error: code = NotFound desc = could not find container \"0443c848089499f8994f9881003129bf06532281a1ddd16281246884dd44408f\": container with ID starting with 0443c848089499f8994f9881003129bf06532281a1ddd16281246884dd44408f not found: ID does not exist" Dec 03 11:24:48 crc kubenswrapper[5052]: I1203 11:24:48.567975 5052 scope.go:117] "RemoveContainer" containerID="f165c5dd9b74405cb0258ec415d7cf2852fda1d2ba6fadfe03c30d3730fa500f" Dec 03 11:24:48 crc kubenswrapper[5052]: E1203 11:24:48.568393 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f165c5dd9b74405cb0258ec415d7cf2852fda1d2ba6fadfe03c30d3730fa500f\": container with ID starting with f165c5dd9b74405cb0258ec415d7cf2852fda1d2ba6fadfe03c30d3730fa500f not found: ID does not exist" containerID="f165c5dd9b74405cb0258ec415d7cf2852fda1d2ba6fadfe03c30d3730fa500f" Dec 03 11:24:48 crc kubenswrapper[5052]: I1203 11:24:48.568420 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f165c5dd9b74405cb0258ec415d7cf2852fda1d2ba6fadfe03c30d3730fa500f"} err="failed to get container status \"f165c5dd9b74405cb0258ec415d7cf2852fda1d2ba6fadfe03c30d3730fa500f\": rpc error: code = NotFound desc = could not find container \"f165c5dd9b74405cb0258ec415d7cf2852fda1d2ba6fadfe03c30d3730fa500f\": container with ID starting with f165c5dd9b74405cb0258ec415d7cf2852fda1d2ba6fadfe03c30d3730fa500f not found: ID does not exist" Dec 03 11:24:48 crc kubenswrapper[5052]: I1203 11:24:48.568436 5052 scope.go:117] "RemoveContainer" containerID="72b81e1c6f6bd6fb2bc6844709bf2e26f33ff52dc4552dfe11690d4fab2eed25" Dec 03 11:24:48 crc kubenswrapper[5052]: I1203 11:24:48.581925 5052 scope.go:117] "RemoveContainer" containerID="9043d6a4663d99a931a2821703919380aec3be2f62eb4b00ecf3fb5846d1902e" Dec 03 11:24:48 crc kubenswrapper[5052]: I1203 11:24:48.629924 5052 scope.go:117] "RemoveContainer" containerID="948e728d629565ee5dd9e42189bfbad641f1d3720cf7169d41a2fb418d4f1258" Dec 03 11:24:48 crc kubenswrapper[5052]: I1203 11:24:48.650792 5052 scope.go:117] "RemoveContainer" containerID="72b81e1c6f6bd6fb2bc6844709bf2e26f33ff52dc4552dfe11690d4fab2eed25" Dec 03 11:24:48 crc kubenswrapper[5052]: E1203 11:24:48.651243 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"72b81e1c6f6bd6fb2bc6844709bf2e26f33ff52dc4552dfe11690d4fab2eed25\": container with ID starting with 72b81e1c6f6bd6fb2bc6844709bf2e26f33ff52dc4552dfe11690d4fab2eed25 not found: ID does not exist" containerID="72b81e1c6f6bd6fb2bc6844709bf2e26f33ff52dc4552dfe11690d4fab2eed25" Dec 03 11:24:48 crc kubenswrapper[5052]: I1203 11:24:48.651278 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"72b81e1c6f6bd6fb2bc6844709bf2e26f33ff52dc4552dfe11690d4fab2eed25"} err="failed to get container status \"72b81e1c6f6bd6fb2bc6844709bf2e26f33ff52dc4552dfe11690d4fab2eed25\": rpc error: code = NotFound desc = could not find container \"72b81e1c6f6bd6fb2bc6844709bf2e26f33ff52dc4552dfe11690d4fab2eed25\": container with ID starting with 72b81e1c6f6bd6fb2bc6844709bf2e26f33ff52dc4552dfe11690d4fab2eed25 not found: ID does not exist" Dec 03 11:24:48 crc kubenswrapper[5052]: I1203 11:24:48.651305 5052 scope.go:117] "RemoveContainer" containerID="9043d6a4663d99a931a2821703919380aec3be2f62eb4b00ecf3fb5846d1902e" Dec 03 11:24:48 crc kubenswrapper[5052]: E1203 11:24:48.651763 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9043d6a4663d99a931a2821703919380aec3be2f62eb4b00ecf3fb5846d1902e\": container with ID starting with 9043d6a4663d99a931a2821703919380aec3be2f62eb4b00ecf3fb5846d1902e not found: ID does not exist" containerID="9043d6a4663d99a931a2821703919380aec3be2f62eb4b00ecf3fb5846d1902e" Dec 03 11:24:48 crc kubenswrapper[5052]: I1203 11:24:48.651841 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9043d6a4663d99a931a2821703919380aec3be2f62eb4b00ecf3fb5846d1902e"} err="failed to get container status \"9043d6a4663d99a931a2821703919380aec3be2f62eb4b00ecf3fb5846d1902e\": rpc error: code = NotFound desc = could not find container \"9043d6a4663d99a931a2821703919380aec3be2f62eb4b00ecf3fb5846d1902e\": container with ID starting with 9043d6a4663d99a931a2821703919380aec3be2f62eb4b00ecf3fb5846d1902e not found: ID does not exist" Dec 03 11:24:48 crc kubenswrapper[5052]: I1203 11:24:48.651879 5052 scope.go:117] "RemoveContainer" containerID="948e728d629565ee5dd9e42189bfbad641f1d3720cf7169d41a2fb418d4f1258" Dec 03 11:24:48 crc kubenswrapper[5052]: E1203 11:24:48.652316 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"948e728d629565ee5dd9e42189bfbad641f1d3720cf7169d41a2fb418d4f1258\": container with ID starting with 948e728d629565ee5dd9e42189bfbad641f1d3720cf7169d41a2fb418d4f1258 not found: ID does not exist" containerID="948e728d629565ee5dd9e42189bfbad641f1d3720cf7169d41a2fb418d4f1258" Dec 03 11:24:48 crc kubenswrapper[5052]: I1203 11:24:48.652340 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"948e728d629565ee5dd9e42189bfbad641f1d3720cf7169d41a2fb418d4f1258"} err="failed to get container status \"948e728d629565ee5dd9e42189bfbad641f1d3720cf7169d41a2fb418d4f1258\": rpc error: code = NotFound desc = could not find container \"948e728d629565ee5dd9e42189bfbad641f1d3720cf7169d41a2fb418d4f1258\": container with ID starting with 948e728d629565ee5dd9e42189bfbad641f1d3720cf7169d41a2fb418d4f1258 not found: ID does not exist" Dec 03 11:24:48 crc kubenswrapper[5052]: I1203 11:24:48.691879 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openshift-marketplace/certified-operators-ghqsf"] Dec 03 11:24:48 crc kubenswrapper[5052]: I1203 11:24:48.696330 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openshift-marketplace/certified-operators-ghqsf"] Dec 03 11:24:48 crc kubenswrapper[5052]: I1203 11:24:48.717764 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openshift-marketplace/community-operators-pxx88"] Dec 03 11:24:48 crc kubenswrapper[5052]: I1203 11:24:48.719899 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openshift-marketplace/community-operators-pxx88"] Dec 03 11:24:48 crc kubenswrapper[5052]: I1203 11:24:48.745564 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openshift-marketplace/redhat-operators-dwt2v"] Dec 03 11:24:48 crc kubenswrapper[5052]: I1203 11:24:48.749636 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openshift-marketplace/redhat-operators-dwt2v"] Dec 03 11:24:49 crc kubenswrapper[5052]: I1203 11:24:49.424977 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/marketplace-operator-79b997595-sbjmz" event={"ID":"ce27ce03-4ec0-418f-95d8-c74102f26bbd","Type":"ContainerStarted","Data":"d730545c54df27728c5dfdbd91b02797763c9b002571455af33634dc6d9c836c"} Dec 03 11:24:49 crc kubenswrapper[5052]: I1203 11:24:49.425356 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/marketplace-operator-79b997595-sbjmz" event={"ID":"ce27ce03-4ec0-418f-95d8-c74102f26bbd","Type":"ContainerStarted","Data":"c975af0ae614f7be5ff7ed050c793675d2b0024930b5cf5a02fa1fb4cf269d33"} Dec 03 11:24:49 crc kubenswrapper[5052]: I1203 11:24:49.425379 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openshift-marketplace/marketplace-operator-79b997595-sbjmz" Dec 03 11:24:49 crc kubenswrapper[5052]: I1203 11:24:49.432831 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-marketplace/marketplace-operator-79b997595-sbjmz" Dec 03 11:24:49 crc kubenswrapper[5052]: I1203 11:24:49.442562 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-marketplace/marketplace-operator-79b997595-sbjmz" podStartSLOduration=2.442546406 podStartE2EDuration="2.442546406s" podCreationTimestamp="2025-12-03 11:24:47 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:24:49.439955263 +0000 UTC m=+280.028197027" watchObservedRunningTime="2025-12-03 11:24:49.442546406 +0000 UTC m=+280.030788180" Dec 03 11:24:49 crc kubenswrapper[5052]: I1203 11:24:49.684857 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="355008cc-8721-44c4-a1e0-3714b0086286" path="/var/lib/kubelet/pods/355008cc-8721-44c4-a1e0-3714b0086286/volumes" Dec 03 11:24:49 crc kubenswrapper[5052]: I1203 11:24:49.685610 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a7228f2a-1d6d-4307-80e7-deebd47d995a" path="/var/lib/kubelet/pods/a7228f2a-1d6d-4307-80e7-deebd47d995a/volumes" Dec 03 11:24:49 crc kubenswrapper[5052]: I1203 11:24:49.686259 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="afb87a28-1b7c-4b09-b186-f05da232f501" path="/var/lib/kubelet/pods/afb87a28-1b7c-4b09-b186-f05da232f501/volumes" Dec 03 11:24:49 crc kubenswrapper[5052]: I1203 11:24:49.687385 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cac70dab-6e13-4693-ac66-c7552b62dfd7" path="/var/lib/kubelet/pods/cac70dab-6e13-4693-ac66-c7552b62dfd7/volumes" Dec 03 11:24:49 crc kubenswrapper[5052]: I1203 11:24:49.687965 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dc6b87f0-562b-4fa9-85b3-ccdd0867374b" path="/var/lib/kubelet/pods/dc6b87f0-562b-4fa9-85b3-ccdd0867374b/volumes" Dec 03 11:24:50 crc kubenswrapper[5052]: I1203 11:24:50.233364 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openshift-marketplace/redhat-marketplace-5ppmc"] Dec 03 11:24:50 crc kubenswrapper[5052]: E1203 11:24:50.233757 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="dc6b87f0-562b-4fa9-85b3-ccdd0867374b" containerName="extract-utilities" Dec 03 11:24:50 crc kubenswrapper[5052]: I1203 11:24:50.233804 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="dc6b87f0-562b-4fa9-85b3-ccdd0867374b" containerName="extract-utilities" Dec 03 11:24:50 crc kubenswrapper[5052]: E1203 11:24:50.233815 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="afb87a28-1b7c-4b09-b186-f05da232f501" containerName="extract-utilities" Dec 03 11:24:50 crc kubenswrapper[5052]: I1203 11:24:50.233821 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="afb87a28-1b7c-4b09-b186-f05da232f501" containerName="extract-utilities" Dec 03 11:24:50 crc kubenswrapper[5052]: E1203 11:24:50.233829 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="afb87a28-1b7c-4b09-b186-f05da232f501" containerName="registry-server" Dec 03 11:24:50 crc kubenswrapper[5052]: I1203 11:24:50.233835 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="afb87a28-1b7c-4b09-b186-f05da232f501" containerName="registry-server" Dec 03 11:24:50 crc kubenswrapper[5052]: E1203 11:24:50.233846 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="a7228f2a-1d6d-4307-80e7-deebd47d995a" containerName="extract-content" Dec 03 11:24:50 crc kubenswrapper[5052]: I1203 11:24:50.233879 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="a7228f2a-1d6d-4307-80e7-deebd47d995a" containerName="extract-content" Dec 03 11:24:50 crc kubenswrapper[5052]: E1203 11:24:50.233891 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="a7228f2a-1d6d-4307-80e7-deebd47d995a" containerName="extract-utilities" Dec 03 11:24:50 crc kubenswrapper[5052]: I1203 11:24:50.233897 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="a7228f2a-1d6d-4307-80e7-deebd47d995a" containerName="extract-utilities" Dec 03 11:24:50 crc kubenswrapper[5052]: E1203 11:24:50.233905 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="afb87a28-1b7c-4b09-b186-f05da232f501" containerName="extract-content" Dec 03 11:24:50 crc kubenswrapper[5052]: I1203 11:24:50.233911 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="afb87a28-1b7c-4b09-b186-f05da232f501" containerName="extract-content" Dec 03 11:24:50 crc kubenswrapper[5052]: E1203 11:24:50.233920 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="355008cc-8721-44c4-a1e0-3714b0086286" containerName="extract-utilities" Dec 03 11:24:50 crc kubenswrapper[5052]: I1203 11:24:50.233929 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="355008cc-8721-44c4-a1e0-3714b0086286" containerName="extract-utilities" Dec 03 11:24:50 crc kubenswrapper[5052]: E1203 11:24:50.233961 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="355008cc-8721-44c4-a1e0-3714b0086286" containerName="extract-content" Dec 03 11:24:50 crc kubenswrapper[5052]: I1203 11:24:50.233968 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="355008cc-8721-44c4-a1e0-3714b0086286" containerName="extract-content" Dec 03 11:24:50 crc kubenswrapper[5052]: E1203 11:24:50.233979 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="a7228f2a-1d6d-4307-80e7-deebd47d995a" containerName="registry-server" Dec 03 11:24:50 crc kubenswrapper[5052]: I1203 11:24:50.233984 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="a7228f2a-1d6d-4307-80e7-deebd47d995a" containerName="registry-server" Dec 03 11:24:50 crc kubenswrapper[5052]: E1203 11:24:50.233993 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="dc6b87f0-562b-4fa9-85b3-ccdd0867374b" containerName="registry-server" Dec 03 11:24:50 crc kubenswrapper[5052]: I1203 11:24:50.234001 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="dc6b87f0-562b-4fa9-85b3-ccdd0867374b" containerName="registry-server" Dec 03 11:24:50 crc kubenswrapper[5052]: E1203 11:24:50.234010 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="dc6b87f0-562b-4fa9-85b3-ccdd0867374b" containerName="extract-content" Dec 03 11:24:50 crc kubenswrapper[5052]: I1203 11:24:50.234038 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="dc6b87f0-562b-4fa9-85b3-ccdd0867374b" containerName="extract-content" Dec 03 11:24:50 crc kubenswrapper[5052]: E1203 11:24:50.234047 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="cac70dab-6e13-4693-ac66-c7552b62dfd7" containerName="marketplace-operator" Dec 03 11:24:50 crc kubenswrapper[5052]: I1203 11:24:50.234053 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="cac70dab-6e13-4693-ac66-c7552b62dfd7" containerName="marketplace-operator" Dec 03 11:24:50 crc kubenswrapper[5052]: E1203 11:24:50.234062 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="355008cc-8721-44c4-a1e0-3714b0086286" containerName="registry-server" Dec 03 11:24:50 crc kubenswrapper[5052]: I1203 11:24:50.234067 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="355008cc-8721-44c4-a1e0-3714b0086286" containerName="registry-server" Dec 03 11:24:50 crc kubenswrapper[5052]: I1203 11:24:50.234156 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="355008cc-8721-44c4-a1e0-3714b0086286" containerName="registry-server" Dec 03 11:24:50 crc kubenswrapper[5052]: I1203 11:24:50.234167 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="afb87a28-1b7c-4b09-b186-f05da232f501" containerName="registry-server" Dec 03 11:24:50 crc kubenswrapper[5052]: I1203 11:24:50.234181 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="a7228f2a-1d6d-4307-80e7-deebd47d995a" containerName="registry-server" Dec 03 11:24:50 crc kubenswrapper[5052]: I1203 11:24:50.234188 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="dc6b87f0-562b-4fa9-85b3-ccdd0867374b" containerName="registry-server" Dec 03 11:24:50 crc kubenswrapper[5052]: I1203 11:24:50.234201 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="cac70dab-6e13-4693-ac66-c7552b62dfd7" containerName="marketplace-operator" Dec 03 11:24:50 crc kubenswrapper[5052]: I1203 11:24:50.234926 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/redhat-marketplace-5ppmc" Dec 03 11:24:50 crc kubenswrapper[5052]: I1203 11:24:50.237202 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-marketplace"/"redhat-marketplace-dockercfg-x2ctb" Dec 03 11:24:50 crc kubenswrapper[5052]: I1203 11:24:50.244110 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/redhat-marketplace-5ppmc"] Dec 03 11:24:50 crc kubenswrapper[5052]: I1203 11:24:50.432309 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/7074b488-71b1-4f8a-b798-b878a519cf7c-catalog-content\") pod \"redhat-marketplace-5ppmc\" (UID: \"7074b488-71b1-4f8a-b798-b878a519cf7c\") " pod="openshift-marketplace/redhat-marketplace-5ppmc" Dec 03 11:24:50 crc kubenswrapper[5052]: I1203 11:24:50.432402 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k4ldv\" (UniqueName: \"kubernetes.io/projected/7074b488-71b1-4f8a-b798-b878a519cf7c-kube-api-access-k4ldv\") pod \"redhat-marketplace-5ppmc\" (UID: \"7074b488-71b1-4f8a-b798-b878a519cf7c\") " pod="openshift-marketplace/redhat-marketplace-5ppmc" Dec 03 11:24:50 crc kubenswrapper[5052]: I1203 11:24:50.432444 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/7074b488-71b1-4f8a-b798-b878a519cf7c-utilities\") pod \"redhat-marketplace-5ppmc\" (UID: \"7074b488-71b1-4f8a-b798-b878a519cf7c\") " pod="openshift-marketplace/redhat-marketplace-5ppmc" Dec 03 11:24:50 crc kubenswrapper[5052]: I1203 11:24:50.533854 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/7074b488-71b1-4f8a-b798-b878a519cf7c-utilities\") pod \"redhat-marketplace-5ppmc\" (UID: \"7074b488-71b1-4f8a-b798-b878a519cf7c\") " pod="openshift-marketplace/redhat-marketplace-5ppmc" Dec 03 11:24:50 crc kubenswrapper[5052]: I1203 11:24:50.533923 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/7074b488-71b1-4f8a-b798-b878a519cf7c-catalog-content\") pod \"redhat-marketplace-5ppmc\" (UID: \"7074b488-71b1-4f8a-b798-b878a519cf7c\") " pod="openshift-marketplace/redhat-marketplace-5ppmc" Dec 03 11:24:50 crc kubenswrapper[5052]: I1203 11:24:50.534104 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-k4ldv\" (UniqueName: \"kubernetes.io/projected/7074b488-71b1-4f8a-b798-b878a519cf7c-kube-api-access-k4ldv\") pod \"redhat-marketplace-5ppmc\" (UID: \"7074b488-71b1-4f8a-b798-b878a519cf7c\") " pod="openshift-marketplace/redhat-marketplace-5ppmc" Dec 03 11:24:50 crc kubenswrapper[5052]: I1203 11:24:50.534503 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/7074b488-71b1-4f8a-b798-b878a519cf7c-utilities\") pod \"redhat-marketplace-5ppmc\" (UID: \"7074b488-71b1-4f8a-b798-b878a519cf7c\") " pod="openshift-marketplace/redhat-marketplace-5ppmc" Dec 03 11:24:50 crc kubenswrapper[5052]: I1203 11:24:50.534682 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/7074b488-71b1-4f8a-b798-b878a519cf7c-catalog-content\") pod \"redhat-marketplace-5ppmc\" (UID: \"7074b488-71b1-4f8a-b798-b878a519cf7c\") " pod="openshift-marketplace/redhat-marketplace-5ppmc" Dec 03 11:24:50 crc kubenswrapper[5052]: I1203 11:24:50.556259 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-k4ldv\" (UniqueName: \"kubernetes.io/projected/7074b488-71b1-4f8a-b798-b878a519cf7c-kube-api-access-k4ldv\") pod \"redhat-marketplace-5ppmc\" (UID: \"7074b488-71b1-4f8a-b798-b878a519cf7c\") " pod="openshift-marketplace/redhat-marketplace-5ppmc" Dec 03 11:24:50 crc kubenswrapper[5052]: I1203 11:24:50.593688 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/redhat-marketplace-5ppmc" Dec 03 11:24:51 crc kubenswrapper[5052]: I1203 11:24:51.000652 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/redhat-marketplace-5ppmc"] Dec 03 11:24:51 crc kubenswrapper[5052]: I1203 11:24:51.237758 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openshift-marketplace/redhat-operators-xtmc5"] Dec 03 11:24:51 crc kubenswrapper[5052]: I1203 11:24:51.239818 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/redhat-operators-xtmc5" Dec 03 11:24:51 crc kubenswrapper[5052]: I1203 11:24:51.244969 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-marketplace"/"redhat-operators-dockercfg-ct8rh" Dec 03 11:24:51 crc kubenswrapper[5052]: I1203 11:24:51.253786 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/redhat-operators-xtmc5"] Dec 03 11:24:51 crc kubenswrapper[5052]: I1203 11:24:51.345185 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gjqg9\" (UniqueName: \"kubernetes.io/projected/3045e915-e20b-4e46-a4bc-55783cb77448-kube-api-access-gjqg9\") pod \"redhat-operators-xtmc5\" (UID: \"3045e915-e20b-4e46-a4bc-55783cb77448\") " pod="openshift-marketplace/redhat-operators-xtmc5" Dec 03 11:24:51 crc kubenswrapper[5052]: I1203 11:24:51.345252 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/3045e915-e20b-4e46-a4bc-55783cb77448-utilities\") pod \"redhat-operators-xtmc5\" (UID: \"3045e915-e20b-4e46-a4bc-55783cb77448\") " pod="openshift-marketplace/redhat-operators-xtmc5" Dec 03 11:24:51 crc kubenswrapper[5052]: I1203 11:24:51.345293 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/3045e915-e20b-4e46-a4bc-55783cb77448-catalog-content\") pod \"redhat-operators-xtmc5\" (UID: \"3045e915-e20b-4e46-a4bc-55783cb77448\") " pod="openshift-marketplace/redhat-operators-xtmc5" Dec 03 11:24:51 crc kubenswrapper[5052]: I1203 11:24:51.446500 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/3045e915-e20b-4e46-a4bc-55783cb77448-utilities\") pod \"redhat-operators-xtmc5\" (UID: \"3045e915-e20b-4e46-a4bc-55783cb77448\") " pod="openshift-marketplace/redhat-operators-xtmc5" Dec 03 11:24:51 crc kubenswrapper[5052]: I1203 11:24:51.446643 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/3045e915-e20b-4e46-a4bc-55783cb77448-catalog-content\") pod \"redhat-operators-xtmc5\" (UID: \"3045e915-e20b-4e46-a4bc-55783cb77448\") " pod="openshift-marketplace/redhat-operators-xtmc5" Dec 03 11:24:51 crc kubenswrapper[5052]: I1203 11:24:51.446735 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-gjqg9\" (UniqueName: \"kubernetes.io/projected/3045e915-e20b-4e46-a4bc-55783cb77448-kube-api-access-gjqg9\") pod \"redhat-operators-xtmc5\" (UID: \"3045e915-e20b-4e46-a4bc-55783cb77448\") " pod="openshift-marketplace/redhat-operators-xtmc5" Dec 03 11:24:51 crc kubenswrapper[5052]: I1203 11:24:51.447147 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/3045e915-e20b-4e46-a4bc-55783cb77448-catalog-content\") pod \"redhat-operators-xtmc5\" (UID: \"3045e915-e20b-4e46-a4bc-55783cb77448\") " pod="openshift-marketplace/redhat-operators-xtmc5" Dec 03 11:24:51 crc kubenswrapper[5052]: I1203 11:24:51.448050 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/3045e915-e20b-4e46-a4bc-55783cb77448-utilities\") pod \"redhat-operators-xtmc5\" (UID: \"3045e915-e20b-4e46-a4bc-55783cb77448\") " pod="openshift-marketplace/redhat-operators-xtmc5" Dec 03 11:24:51 crc kubenswrapper[5052]: I1203 11:24:51.449829 5052 generic.go:334] "Generic (PLEG): container finished" podID="7074b488-71b1-4f8a-b798-b878a519cf7c" containerID="46661e58a53aa8093440e4b39cade5425fdd951e57bcaf5caef61adf03490f7d" exitCode=0 Dec 03 11:24:51 crc kubenswrapper[5052]: I1203 11:24:51.449943 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-marketplace-5ppmc" event={"ID":"7074b488-71b1-4f8a-b798-b878a519cf7c","Type":"ContainerDied","Data":"46661e58a53aa8093440e4b39cade5425fdd951e57bcaf5caef61adf03490f7d"} Dec 03 11:24:51 crc kubenswrapper[5052]: I1203 11:24:51.449994 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-marketplace-5ppmc" event={"ID":"7074b488-71b1-4f8a-b798-b878a519cf7c","Type":"ContainerStarted","Data":"c94fde85fa445d8d3e0a6e7539bd1e21d7888ac65c58344e56c1bad797d47e95"} Dec 03 11:24:51 crc kubenswrapper[5052]: I1203 11:24:51.473332 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-gjqg9\" (UniqueName: \"kubernetes.io/projected/3045e915-e20b-4e46-a4bc-55783cb77448-kube-api-access-gjqg9\") pod \"redhat-operators-xtmc5\" (UID: \"3045e915-e20b-4e46-a4bc-55783cb77448\") " pod="openshift-marketplace/redhat-operators-xtmc5" Dec 03 11:24:51 crc kubenswrapper[5052]: I1203 11:24:51.570645 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/redhat-operators-xtmc5" Dec 03 11:24:52 crc kubenswrapper[5052]: I1203 11:24:52.302561 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/redhat-operators-xtmc5"] Dec 03 11:24:52 crc kubenswrapper[5052]: I1203 11:24:52.457842 5052 generic.go:334] "Generic (PLEG): container finished" podID="7074b488-71b1-4f8a-b798-b878a519cf7c" containerID="d5a4516b2454725c58465bf1b58b2f0ffb61dbf2847e4d34aeba6c33eea011e3" exitCode=0 Dec 03 11:24:52 crc kubenswrapper[5052]: I1203 11:24:52.457930 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-marketplace-5ppmc" event={"ID":"7074b488-71b1-4f8a-b798-b878a519cf7c","Type":"ContainerDied","Data":"d5a4516b2454725c58465bf1b58b2f0ffb61dbf2847e4d34aeba6c33eea011e3"} Dec 03 11:24:52 crc kubenswrapper[5052]: I1203 11:24:52.460546 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-operators-xtmc5" event={"ID":"3045e915-e20b-4e46-a4bc-55783cb77448","Type":"ContainerStarted","Data":"42ac4d1fdee0fcbcbb4f92bf4ed784d97d9b66b4b2f48fe855b0f7e669fac5f7"} Dec 03 11:24:52 crc kubenswrapper[5052]: I1203 11:24:52.635445 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openshift-marketplace/certified-operators-7wd6v"] Dec 03 11:24:52 crc kubenswrapper[5052]: I1203 11:24:52.637482 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/certified-operators-7wd6v" Dec 03 11:24:52 crc kubenswrapper[5052]: I1203 11:24:52.640578 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-marketplace"/"certified-operators-dockercfg-4rs5g" Dec 03 11:24:52 crc kubenswrapper[5052]: I1203 11:24:52.646886 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/certified-operators-7wd6v"] Dec 03 11:24:52 crc kubenswrapper[5052]: I1203 11:24:52.764912 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/fb0eab4c-4728-4b3f-9df0-70bd099cb8c9-catalog-content\") pod \"certified-operators-7wd6v\" (UID: \"fb0eab4c-4728-4b3f-9df0-70bd099cb8c9\") " pod="openshift-marketplace/certified-operators-7wd6v" Dec 03 11:24:52 crc kubenswrapper[5052]: I1203 11:24:52.765961 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/fb0eab4c-4728-4b3f-9df0-70bd099cb8c9-utilities\") pod \"certified-operators-7wd6v\" (UID: \"fb0eab4c-4728-4b3f-9df0-70bd099cb8c9\") " pod="openshift-marketplace/certified-operators-7wd6v" Dec 03 11:24:52 crc kubenswrapper[5052]: I1203 11:24:52.765993 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6zcn4\" (UniqueName: \"kubernetes.io/projected/fb0eab4c-4728-4b3f-9df0-70bd099cb8c9-kube-api-access-6zcn4\") pod \"certified-operators-7wd6v\" (UID: \"fb0eab4c-4728-4b3f-9df0-70bd099cb8c9\") " pod="openshift-marketplace/certified-operators-7wd6v" Dec 03 11:24:52 crc kubenswrapper[5052]: I1203 11:24:52.867214 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/fb0eab4c-4728-4b3f-9df0-70bd099cb8c9-utilities\") pod \"certified-operators-7wd6v\" (UID: \"fb0eab4c-4728-4b3f-9df0-70bd099cb8c9\") " pod="openshift-marketplace/certified-operators-7wd6v" Dec 03 11:24:52 crc kubenswrapper[5052]: I1203 11:24:52.867303 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-6zcn4\" (UniqueName: \"kubernetes.io/projected/fb0eab4c-4728-4b3f-9df0-70bd099cb8c9-kube-api-access-6zcn4\") pod \"certified-operators-7wd6v\" (UID: \"fb0eab4c-4728-4b3f-9df0-70bd099cb8c9\") " pod="openshift-marketplace/certified-operators-7wd6v" Dec 03 11:24:52 crc kubenswrapper[5052]: I1203 11:24:52.867386 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/fb0eab4c-4728-4b3f-9df0-70bd099cb8c9-catalog-content\") pod \"certified-operators-7wd6v\" (UID: \"fb0eab4c-4728-4b3f-9df0-70bd099cb8c9\") " pod="openshift-marketplace/certified-operators-7wd6v" Dec 03 11:24:52 crc kubenswrapper[5052]: I1203 11:24:52.868034 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/fb0eab4c-4728-4b3f-9df0-70bd099cb8c9-catalog-content\") pod \"certified-operators-7wd6v\" (UID: \"fb0eab4c-4728-4b3f-9df0-70bd099cb8c9\") " pod="openshift-marketplace/certified-operators-7wd6v" Dec 03 11:24:52 crc kubenswrapper[5052]: I1203 11:24:52.868283 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/fb0eab4c-4728-4b3f-9df0-70bd099cb8c9-utilities\") pod \"certified-operators-7wd6v\" (UID: \"fb0eab4c-4728-4b3f-9df0-70bd099cb8c9\") " pod="openshift-marketplace/certified-operators-7wd6v" Dec 03 11:24:52 crc kubenswrapper[5052]: I1203 11:24:52.890206 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-6zcn4\" (UniqueName: \"kubernetes.io/projected/fb0eab4c-4728-4b3f-9df0-70bd099cb8c9-kube-api-access-6zcn4\") pod \"certified-operators-7wd6v\" (UID: \"fb0eab4c-4728-4b3f-9df0-70bd099cb8c9\") " pod="openshift-marketplace/certified-operators-7wd6v" Dec 03 11:24:52 crc kubenswrapper[5052]: I1203 11:24:52.969909 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/certified-operators-7wd6v" Dec 03 11:24:53 crc kubenswrapper[5052]: I1203 11:24:53.384158 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/certified-operators-7wd6v"] Dec 03 11:24:53 crc kubenswrapper[5052]: I1203 11:24:53.446658 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-66df7c8f76-tth5j" Dec 03 11:24:53 crc kubenswrapper[5052]: I1203 11:24:53.471332 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/certified-operators-7wd6v" event={"ID":"fb0eab4c-4728-4b3f-9df0-70bd099cb8c9","Type":"ContainerStarted","Data":"8137e47f9747a51ee4e1a48fa4930b93a7adf74fa53a8f3a4f8fcb986e0447f0"} Dec 03 11:24:53 crc kubenswrapper[5052]: I1203 11:24:53.509220 5052 generic.go:334] "Generic (PLEG): container finished" podID="3045e915-e20b-4e46-a4bc-55783cb77448" containerID="89e8f138dd47abf09e30d789abd9af05e16bfb3000449040f6651ad45ab008c2" exitCode=0 Dec 03 11:24:53 crc kubenswrapper[5052]: I1203 11:24:53.509404 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-operators-xtmc5" event={"ID":"3045e915-e20b-4e46-a4bc-55783cb77448","Type":"ContainerDied","Data":"89e8f138dd47abf09e30d789abd9af05e16bfb3000449040f6651ad45ab008c2"} Dec 03 11:24:53 crc kubenswrapper[5052]: I1203 11:24:53.517970 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-697d97f7c8-p77bx"] Dec 03 11:24:53 crc kubenswrapper[5052]: I1203 11:24:53.525464 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-marketplace-5ppmc" event={"ID":"7074b488-71b1-4f8a-b798-b878a519cf7c","Type":"ContainerStarted","Data":"8cf4d95147039ebc9f4eca7f3e313a35b14b9f137078e858421477044a91b888"} Dec 03 11:24:53 crc kubenswrapper[5052]: E1203 11:24:53.635848 5052 cadvisor_stats_provider.go:516] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb0eab4c_4728_4b3f_9df0_70bd099cb8c9.slice/crio-205156a1a270a68d4c328e186a20aed900af4b6ef2b3b532a3cae8e5047b5110.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb0eab4c_4728_4b3f_9df0_70bd099cb8c9.slice/crio-conmon-205156a1a270a68d4c328e186a20aed900af4b6ef2b3b532a3cae8e5047b5110.scope\": RecentStats: unable to find data in memory cache]" Dec 03 11:24:53 crc kubenswrapper[5052]: I1203 11:24:53.641313 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-marketplace/redhat-marketplace-5ppmc" podStartSLOduration=2.158482027 podStartE2EDuration="3.641290314s" podCreationTimestamp="2025-12-03 11:24:50 +0000 UTC" firstStartedPulling="2025-12-03 11:24:51.451186162 +0000 UTC m=+282.039427926" lastFinishedPulling="2025-12-03 11:24:52.933994449 +0000 UTC m=+283.522236213" observedRunningTime="2025-12-03 11:24:53.583612475 +0000 UTC m=+284.171854239" watchObservedRunningTime="2025-12-03 11:24:53.641290314 +0000 UTC m=+284.229532078" Dec 03 11:24:53 crc kubenswrapper[5052]: I1203 11:24:53.643012 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openshift-marketplace/community-operators-jxmjl"] Dec 03 11:24:53 crc kubenswrapper[5052]: I1203 11:24:53.644275 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/community-operators-jxmjl" Dec 03 11:24:53 crc kubenswrapper[5052]: I1203 11:24:53.650121 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-marketplace"/"community-operators-dockercfg-dmngl" Dec 03 11:24:53 crc kubenswrapper[5052]: I1203 11:24:53.654447 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/community-operators-jxmjl"] Dec 03 11:24:53 crc kubenswrapper[5052]: I1203 11:24:53.785558 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/3e53b2f1-00e5-49d7-a6a6-5bcae4e9befb-catalog-content\") pod \"community-operators-jxmjl\" (UID: \"3e53b2f1-00e5-49d7-a6a6-5bcae4e9befb\") " pod="openshift-marketplace/community-operators-jxmjl" Dec 03 11:24:53 crc kubenswrapper[5052]: I1203 11:24:53.785633 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/3e53b2f1-00e5-49d7-a6a6-5bcae4e9befb-utilities\") pod \"community-operators-jxmjl\" (UID: \"3e53b2f1-00e5-49d7-a6a6-5bcae4e9befb\") " pod="openshift-marketplace/community-operators-jxmjl" Dec 03 11:24:53 crc kubenswrapper[5052]: I1203 11:24:53.785748 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-46qmn\" (UniqueName: \"kubernetes.io/projected/3e53b2f1-00e5-49d7-a6a6-5bcae4e9befb-kube-api-access-46qmn\") pod \"community-operators-jxmjl\" (UID: \"3e53b2f1-00e5-49d7-a6a6-5bcae4e9befb\") " pod="openshift-marketplace/community-operators-jxmjl" Dec 03 11:24:53 crc kubenswrapper[5052]: I1203 11:24:53.886946 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-46qmn\" (UniqueName: \"kubernetes.io/projected/3e53b2f1-00e5-49d7-a6a6-5bcae4e9befb-kube-api-access-46qmn\") pod \"community-operators-jxmjl\" (UID: \"3e53b2f1-00e5-49d7-a6a6-5bcae4e9befb\") " pod="openshift-marketplace/community-operators-jxmjl" Dec 03 11:24:53 crc kubenswrapper[5052]: I1203 11:24:53.887035 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/3e53b2f1-00e5-49d7-a6a6-5bcae4e9befb-catalog-content\") pod \"community-operators-jxmjl\" (UID: \"3e53b2f1-00e5-49d7-a6a6-5bcae4e9befb\") " pod="openshift-marketplace/community-operators-jxmjl" Dec 03 11:24:53 crc kubenswrapper[5052]: I1203 11:24:53.887064 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/3e53b2f1-00e5-49d7-a6a6-5bcae4e9befb-utilities\") pod \"community-operators-jxmjl\" (UID: \"3e53b2f1-00e5-49d7-a6a6-5bcae4e9befb\") " pod="openshift-marketplace/community-operators-jxmjl" Dec 03 11:24:53 crc kubenswrapper[5052]: I1203 11:24:53.887660 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/3e53b2f1-00e5-49d7-a6a6-5bcae4e9befb-utilities\") pod \"community-operators-jxmjl\" (UID: \"3e53b2f1-00e5-49d7-a6a6-5bcae4e9befb\") " pod="openshift-marketplace/community-operators-jxmjl" Dec 03 11:24:53 crc kubenswrapper[5052]: I1203 11:24:53.887951 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/3e53b2f1-00e5-49d7-a6a6-5bcae4e9befb-catalog-content\") pod \"community-operators-jxmjl\" (UID: \"3e53b2f1-00e5-49d7-a6a6-5bcae4e9befb\") " pod="openshift-marketplace/community-operators-jxmjl" Dec 03 11:24:53 crc kubenswrapper[5052]: I1203 11:24:53.910115 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-46qmn\" (UniqueName: \"kubernetes.io/projected/3e53b2f1-00e5-49d7-a6a6-5bcae4e9befb-kube-api-access-46qmn\") pod \"community-operators-jxmjl\" (UID: \"3e53b2f1-00e5-49d7-a6a6-5bcae4e9befb\") " pod="openshift-marketplace/community-operators-jxmjl" Dec 03 11:24:54 crc kubenswrapper[5052]: I1203 11:24:54.006944 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/community-operators-jxmjl" Dec 03 11:24:54 crc kubenswrapper[5052]: I1203 11:24:54.450751 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/community-operators-jxmjl"] Dec 03 11:24:54 crc kubenswrapper[5052]: W1203 11:24:54.454403 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e53b2f1_00e5_49d7_a6a6_5bcae4e9befb.slice/crio-30ded7c822d22511ff5ee2b50b795816fb1f7a3ac5eb0d640c5c0b975ae04021 WatchSource:0}: Error finding container 30ded7c822d22511ff5ee2b50b795816fb1f7a3ac5eb0d640c5c0b975ae04021: Status 404 returned error can't find the container with id 30ded7c822d22511ff5ee2b50b795816fb1f7a3ac5eb0d640c5c0b975ae04021 Dec 03 11:24:54 crc kubenswrapper[5052]: I1203 11:24:54.533999 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/community-operators-jxmjl" event={"ID":"3e53b2f1-00e5-49d7-a6a6-5bcae4e9befb","Type":"ContainerStarted","Data":"30ded7c822d22511ff5ee2b50b795816fb1f7a3ac5eb0d640c5c0b975ae04021"} Dec 03 11:24:54 crc kubenswrapper[5052]: I1203 11:24:54.535854 5052 generic.go:334] "Generic (PLEG): container finished" podID="fb0eab4c-4728-4b3f-9df0-70bd099cb8c9" containerID="205156a1a270a68d4c328e186a20aed900af4b6ef2b3b532a3cae8e5047b5110" exitCode=0 Dec 03 11:24:54 crc kubenswrapper[5052]: I1203 11:24:54.535951 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/certified-operators-7wd6v" event={"ID":"fb0eab4c-4728-4b3f-9df0-70bd099cb8c9","Type":"ContainerDied","Data":"205156a1a270a68d4c328e186a20aed900af4b6ef2b3b532a3cae8e5047b5110"} Dec 03 11:24:54 crc kubenswrapper[5052]: I1203 11:24:54.538622 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-operators-xtmc5" event={"ID":"3045e915-e20b-4e46-a4bc-55783cb77448","Type":"ContainerStarted","Data":"027faf1cbff6bfd9a57014de652c9e64b9959f538b04469d6c04fba73a412699"} Dec 03 11:24:55 crc kubenswrapper[5052]: I1203 11:24:55.548315 5052 generic.go:334] "Generic (PLEG): container finished" podID="3045e915-e20b-4e46-a4bc-55783cb77448" containerID="027faf1cbff6bfd9a57014de652c9e64b9959f538b04469d6c04fba73a412699" exitCode=0 Dec 03 11:24:55 crc kubenswrapper[5052]: I1203 11:24:55.549041 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-operators-xtmc5" event={"ID":"3045e915-e20b-4e46-a4bc-55783cb77448","Type":"ContainerDied","Data":"027faf1cbff6bfd9a57014de652c9e64b9959f538b04469d6c04fba73a412699"} Dec 03 11:24:55 crc kubenswrapper[5052]: I1203 11:24:55.565073 5052 generic.go:334] "Generic (PLEG): container finished" podID="3e53b2f1-00e5-49d7-a6a6-5bcae4e9befb" containerID="b1e7fd15d95a20884e5214fc37df165775f1f32afd7417efb71318b82748d4d7" exitCode=0 Dec 03 11:24:55 crc kubenswrapper[5052]: I1203 11:24:55.565166 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/community-operators-jxmjl" event={"ID":"3e53b2f1-00e5-49d7-a6a6-5bcae4e9befb","Type":"ContainerDied","Data":"b1e7fd15d95a20884e5214fc37df165775f1f32afd7417efb71318b82748d4d7"} Dec 03 11:24:55 crc kubenswrapper[5052]: I1203 11:24:55.570641 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/certified-operators-7wd6v" event={"ID":"fb0eab4c-4728-4b3f-9df0-70bd099cb8c9","Type":"ContainerStarted","Data":"19b85075f2eda47e6aec0c9055473935cf58cfcdbf1f8ce63045e089415617fe"} Dec 03 11:24:56 crc kubenswrapper[5052]: I1203 11:24:56.580081 5052 generic.go:334] "Generic (PLEG): container finished" podID="fb0eab4c-4728-4b3f-9df0-70bd099cb8c9" containerID="19b85075f2eda47e6aec0c9055473935cf58cfcdbf1f8ce63045e089415617fe" exitCode=0 Dec 03 11:24:56 crc kubenswrapper[5052]: I1203 11:24:56.580183 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/certified-operators-7wd6v" event={"ID":"fb0eab4c-4728-4b3f-9df0-70bd099cb8c9","Type":"ContainerDied","Data":"19b85075f2eda47e6aec0c9055473935cf58cfcdbf1f8ce63045e089415617fe"} Dec 03 11:24:57 crc kubenswrapper[5052]: I1203 11:24:57.593529 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-operators-xtmc5" event={"ID":"3045e915-e20b-4e46-a4bc-55783cb77448","Type":"ContainerStarted","Data":"877e8dfde4cc74b22ae57a30d901fe4ff83c6d6d91657bed5734791b0fc4b501"} Dec 03 11:24:57 crc kubenswrapper[5052]: I1203 11:24:57.596591 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/community-operators-jxmjl" event={"ID":"3e53b2f1-00e5-49d7-a6a6-5bcae4e9befb","Type":"ContainerStarted","Data":"6ac973e18ac00d9025535900924ca7001bb39371d0c2414a4f4c3560c6579eed"} Dec 03 11:24:57 crc kubenswrapper[5052]: I1203 11:24:57.599333 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/certified-operators-7wd6v" event={"ID":"fb0eab4c-4728-4b3f-9df0-70bd099cb8c9","Type":"ContainerStarted","Data":"e9e06f4cae7c528e5ad942a4d6fd8849b4e5b0d0000bfe3132ef7697b540b97b"} Dec 03 11:24:57 crc kubenswrapper[5052]: I1203 11:24:57.613445 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-marketplace/redhat-operators-xtmc5" podStartSLOduration=4.041075636 podStartE2EDuration="6.613423919s" podCreationTimestamp="2025-12-03 11:24:51 +0000 UTC" firstStartedPulling="2025-12-03 11:24:53.518997319 +0000 UTC m=+284.107239083" lastFinishedPulling="2025-12-03 11:24:56.091345602 +0000 UTC m=+286.679587366" observedRunningTime="2025-12-03 11:24:57.612878804 +0000 UTC m=+288.201120578" watchObservedRunningTime="2025-12-03 11:24:57.613423919 +0000 UTC m=+288.201665683" Dec 03 11:24:57 crc kubenswrapper[5052]: I1203 11:24:57.633689 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-marketplace/certified-operators-7wd6v" podStartSLOduration=3.9024784759999998 podStartE2EDuration="5.633668021s" podCreationTimestamp="2025-12-03 11:24:52 +0000 UTC" firstStartedPulling="2025-12-03 11:24:54.538090354 +0000 UTC m=+285.126332118" lastFinishedPulling="2025-12-03 11:24:56.269279899 +0000 UTC m=+286.857521663" observedRunningTime="2025-12-03 11:24:57.630830161 +0000 UTC m=+288.219071945" watchObservedRunningTime="2025-12-03 11:24:57.633668021 +0000 UTC m=+288.221909785" Dec 03 11:24:58 crc kubenswrapper[5052]: I1203 11:24:58.606663 5052 generic.go:334] "Generic (PLEG): container finished" podID="3e53b2f1-00e5-49d7-a6a6-5bcae4e9befb" containerID="6ac973e18ac00d9025535900924ca7001bb39371d0c2414a4f4c3560c6579eed" exitCode=0 Dec 03 11:24:58 crc kubenswrapper[5052]: I1203 11:24:58.607582 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/community-operators-jxmjl" event={"ID":"3e53b2f1-00e5-49d7-a6a6-5bcae4e9befb","Type":"ContainerDied","Data":"6ac973e18ac00d9025535900924ca7001bb39371d0c2414a4f4c3560c6579eed"} Dec 03 11:24:59 crc kubenswrapper[5052]: I1203 11:24:59.614381 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/community-operators-jxmjl" event={"ID":"3e53b2f1-00e5-49d7-a6a6-5bcae4e9befb","Type":"ContainerStarted","Data":"e3c8f6cc7f02135fad4fef7877e15a154d9b313da7f2737783e864e2e3d449e5"} Dec 03 11:25:00 crc kubenswrapper[5052]: I1203 11:25:00.594616 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openshift-marketplace/redhat-marketplace-5ppmc" Dec 03 11:25:00 crc kubenswrapper[5052]: I1203 11:25:00.594693 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-marketplace/redhat-marketplace-5ppmc" Dec 03 11:25:00 crc kubenswrapper[5052]: I1203 11:25:00.639624 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-marketplace/community-operators-jxmjl" podStartSLOduration=3.849785894 podStartE2EDuration="7.639608416s" podCreationTimestamp="2025-12-03 11:24:53 +0000 UTC" firstStartedPulling="2025-12-03 11:24:55.56922146 +0000 UTC m=+286.157463224" lastFinishedPulling="2025-12-03 11:24:59.359043982 +0000 UTC m=+289.947285746" observedRunningTime="2025-12-03 11:25:00.637619079 +0000 UTC m=+291.225860853" watchObservedRunningTime="2025-12-03 11:25:00.639608416 +0000 UTC m=+291.227850170" Dec 03 11:25:00 crc kubenswrapper[5052]: I1203 11:25:00.645497 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-marketplace/redhat-marketplace-5ppmc" Dec 03 11:25:00 crc kubenswrapper[5052]: I1203 11:25:00.684000 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-marketplace/redhat-marketplace-5ppmc" Dec 03 11:25:01 crc kubenswrapper[5052]: I1203 11:25:01.571673 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-marketplace/redhat-operators-xtmc5" Dec 03 11:25:01 crc kubenswrapper[5052]: I1203 11:25:01.571948 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openshift-marketplace/redhat-operators-xtmc5" Dec 03 11:25:02 crc kubenswrapper[5052]: I1203 11:25:02.612404 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openshift-marketplace/redhat-operators-xtmc5" podUID="3045e915-e20b-4e46-a4bc-55783cb77448" containerName="registry-server" probeResult="failure" output=< Dec 03 11:25:02 crc kubenswrapper[5052]: timeout: failed to connect service ":50051" within 1s Dec 03 11:25:02 crc kubenswrapper[5052]: > Dec 03 11:25:02 crc kubenswrapper[5052]: I1203 11:25:02.971151 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-marketplace/certified-operators-7wd6v" Dec 03 11:25:02 crc kubenswrapper[5052]: I1203 11:25:02.971219 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openshift-marketplace/certified-operators-7wd6v" Dec 03 11:25:03 crc kubenswrapper[5052]: I1203 11:25:03.013947 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-marketplace/certified-operators-7wd6v" Dec 03 11:25:03 crc kubenswrapper[5052]: I1203 11:25:03.682875 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-marketplace/certified-operators-7wd6v" Dec 03 11:25:04 crc kubenswrapper[5052]: I1203 11:25:04.008018 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openshift-marketplace/community-operators-jxmjl" Dec 03 11:25:04 crc kubenswrapper[5052]: I1203 11:25:04.008094 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-marketplace/community-operators-jxmjl" Dec 03 11:25:04 crc kubenswrapper[5052]: I1203 11:25:04.051222 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-marketplace/community-operators-jxmjl" Dec 03 11:25:04 crc kubenswrapper[5052]: I1203 11:25:04.690927 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-marketplace/community-operators-jxmjl" Dec 03 11:25:11 crc kubenswrapper[5052]: I1203 11:25:11.622988 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-marketplace/redhat-operators-xtmc5" Dec 03 11:25:11 crc kubenswrapper[5052]: I1203 11:25:11.670094 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-marketplace/redhat-operators-xtmc5" Dec 03 11:25:18 crc kubenswrapper[5052]: I1203 11:25:18.554342 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openshift-image-registry/image-registry-697d97f7c8-p77bx" podUID="f5012254-068c-4360-b310-39ff3d309199" containerName="registry" containerID="cri-o://d21421fd913291ac3ac9b67e9a506128685f1a0d80fcab53e873bc0124e4cfb1" gracePeriod=30 Dec 03 11:25:19 crc kubenswrapper[5052]: I1203 11:25:19.453662 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-697d97f7c8-p77bx" Dec 03 11:25:19 crc kubenswrapper[5052]: I1203 11:25:19.564056 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f5012254-068c-4360-b310-39ff3d309199-ca-trust-extracted\") pod \"f5012254-068c-4360-b310-39ff3d309199\" (UID: \"f5012254-068c-4360-b310-39ff3d309199\") " Dec 03 11:25:19 crc kubenswrapper[5052]: I1203 11:25:19.564134 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f5012254-068c-4360-b310-39ff3d309199-registry-tls\") pod \"f5012254-068c-4360-b310-39ff3d309199\" (UID: \"f5012254-068c-4360-b310-39ff3d309199\") " Dec 03 11:25:19 crc kubenswrapper[5052]: I1203 11:25:19.564179 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f5012254-068c-4360-b310-39ff3d309199-registry-certificates\") pod \"f5012254-068c-4360-b310-39ff3d309199\" (UID: \"f5012254-068c-4360-b310-39ff3d309199\") " Dec 03 11:25:19 crc kubenswrapper[5052]: I1203 11:25:19.564218 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f5012254-068c-4360-b310-39ff3d309199-trusted-ca\") pod \"f5012254-068c-4360-b310-39ff3d309199\" (UID: \"f5012254-068c-4360-b310-39ff3d309199\") " Dec 03 11:25:19 crc kubenswrapper[5052]: I1203 11:25:19.564495 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"registry-storage\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8\") pod \"f5012254-068c-4360-b310-39ff3d309199\" (UID: \"f5012254-068c-4360-b310-39ff3d309199\") " Dec 03 11:25:19 crc kubenswrapper[5052]: I1203 11:25:19.564603 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cvfz9\" (UniqueName: \"kubernetes.io/projected/f5012254-068c-4360-b310-39ff3d309199-kube-api-access-cvfz9\") pod \"f5012254-068c-4360-b310-39ff3d309199\" (UID: \"f5012254-068c-4360-b310-39ff3d309199\") " Dec 03 11:25:19 crc kubenswrapper[5052]: I1203 11:25:19.564635 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f5012254-068c-4360-b310-39ff3d309199-installation-pull-secrets\") pod \"f5012254-068c-4360-b310-39ff3d309199\" (UID: \"f5012254-068c-4360-b310-39ff3d309199\") " Dec 03 11:25:19 crc kubenswrapper[5052]: I1203 11:25:19.564695 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f5012254-068c-4360-b310-39ff3d309199-bound-sa-token\") pod \"f5012254-068c-4360-b310-39ff3d309199\" (UID: \"f5012254-068c-4360-b310-39ff3d309199\") " Dec 03 11:25:19 crc kubenswrapper[5052]: I1203 11:25:19.564916 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f5012254-068c-4360-b310-39ff3d309199-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "f5012254-068c-4360-b310-39ff3d309199" (UID: "f5012254-068c-4360-b310-39ff3d309199"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:25:19 crc kubenswrapper[5052]: I1203 11:25:19.565278 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f5012254-068c-4360-b310-39ff3d309199-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "f5012254-068c-4360-b310-39ff3d309199" (UID: "f5012254-068c-4360-b310-39ff3d309199"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:25:19 crc kubenswrapper[5052]: I1203 11:25:19.571962 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f5012254-068c-4360-b310-39ff3d309199-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "f5012254-068c-4360-b310-39ff3d309199" (UID: "f5012254-068c-4360-b310-39ff3d309199"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:25:19 crc kubenswrapper[5052]: I1203 11:25:19.572209 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f5012254-068c-4360-b310-39ff3d309199-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "f5012254-068c-4360-b310-39ff3d309199" (UID: "f5012254-068c-4360-b310-39ff3d309199"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:25:19 crc kubenswrapper[5052]: I1203 11:25:19.576562 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8" (OuterVolumeSpecName: "registry-storage") pod "f5012254-068c-4360-b310-39ff3d309199" (UID: "f5012254-068c-4360-b310-39ff3d309199"). InnerVolumeSpecName "pvc-657094db-63f1-4ba8-9a24-edca0e80b7a8". PluginName "kubernetes.io/csi", VolumeGidValue "" Dec 03 11:25:19 crc kubenswrapper[5052]: I1203 11:25:19.582070 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f5012254-068c-4360-b310-39ff3d309199-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "f5012254-068c-4360-b310-39ff3d309199" (UID: "f5012254-068c-4360-b310-39ff3d309199"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:25:19 crc kubenswrapper[5052]: I1203 11:25:19.582448 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f5012254-068c-4360-b310-39ff3d309199-kube-api-access-cvfz9" (OuterVolumeSpecName: "kube-api-access-cvfz9") pod "f5012254-068c-4360-b310-39ff3d309199" (UID: "f5012254-068c-4360-b310-39ff3d309199"). InnerVolumeSpecName "kube-api-access-cvfz9". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:25:19 crc kubenswrapper[5052]: I1203 11:25:19.583220 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f5012254-068c-4360-b310-39ff3d309199-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "f5012254-068c-4360-b310-39ff3d309199" (UID: "f5012254-068c-4360-b310-39ff3d309199"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:25:19 crc kubenswrapper[5052]: I1203 11:25:19.666609 5052 reconciler_common.go:293] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f5012254-068c-4360-b310-39ff3d309199-ca-trust-extracted\") on node \"crc\" DevicePath \"\"" Dec 03 11:25:19 crc kubenswrapper[5052]: I1203 11:25:19.666667 5052 reconciler_common.go:293] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f5012254-068c-4360-b310-39ff3d309199-registry-tls\") on node \"crc\" DevicePath \"\"" Dec 03 11:25:19 crc kubenswrapper[5052]: I1203 11:25:19.666684 5052 reconciler_common.go:293] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f5012254-068c-4360-b310-39ff3d309199-registry-certificates\") on node \"crc\" DevicePath \"\"" Dec 03 11:25:19 crc kubenswrapper[5052]: I1203 11:25:19.666697 5052 reconciler_common.go:293] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f5012254-068c-4360-b310-39ff3d309199-trusted-ca\") on node \"crc\" DevicePath \"\"" Dec 03 11:25:19 crc kubenswrapper[5052]: I1203 11:25:19.666707 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-cvfz9\" (UniqueName: \"kubernetes.io/projected/f5012254-068c-4360-b310-39ff3d309199-kube-api-access-cvfz9\") on node \"crc\" DevicePath \"\"" Dec 03 11:25:19 crc kubenswrapper[5052]: I1203 11:25:19.666719 5052 reconciler_common.go:293] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f5012254-068c-4360-b310-39ff3d309199-installation-pull-secrets\") on node \"crc\" DevicePath \"\"" Dec 03 11:25:19 crc kubenswrapper[5052]: I1203 11:25:19.666730 5052 reconciler_common.go:293] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f5012254-068c-4360-b310-39ff3d309199-bound-sa-token\") on node \"crc\" DevicePath \"\"" Dec 03 11:25:19 crc kubenswrapper[5052]: I1203 11:25:19.753878 5052 generic.go:334] "Generic (PLEG): container finished" podID="f5012254-068c-4360-b310-39ff3d309199" containerID="d21421fd913291ac3ac9b67e9a506128685f1a0d80fcab53e873bc0124e4cfb1" exitCode=0 Dec 03 11:25:19 crc kubenswrapper[5052]: I1203 11:25:19.753946 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-697d97f7c8-p77bx" event={"ID":"f5012254-068c-4360-b310-39ff3d309199","Type":"ContainerDied","Data":"d21421fd913291ac3ac9b67e9a506128685f1a0d80fcab53e873bc0124e4cfb1"} Dec 03 11:25:19 crc kubenswrapper[5052]: I1203 11:25:19.753986 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-697d97f7c8-p77bx" event={"ID":"f5012254-068c-4360-b310-39ff3d309199","Type":"ContainerDied","Data":"2aac92f4f9ceffe73b9baac1c04f7002e948025594b6c4534ec1782755c026b1"} Dec 03 11:25:19 crc kubenswrapper[5052]: I1203 11:25:19.754006 5052 scope.go:117] "RemoveContainer" containerID="d21421fd913291ac3ac9b67e9a506128685f1a0d80fcab53e873bc0124e4cfb1" Dec 03 11:25:19 crc kubenswrapper[5052]: I1203 11:25:19.754156 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-697d97f7c8-p77bx" Dec 03 11:25:19 crc kubenswrapper[5052]: I1203 11:25:19.777917 5052 scope.go:117] "RemoveContainer" containerID="d21421fd913291ac3ac9b67e9a506128685f1a0d80fcab53e873bc0124e4cfb1" Dec 03 11:25:19 crc kubenswrapper[5052]: I1203 11:25:19.778789 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-697d97f7c8-p77bx"] Dec 03 11:25:19 crc kubenswrapper[5052]: E1203 11:25:19.779835 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d21421fd913291ac3ac9b67e9a506128685f1a0d80fcab53e873bc0124e4cfb1\": container with ID starting with d21421fd913291ac3ac9b67e9a506128685f1a0d80fcab53e873bc0124e4cfb1 not found: ID does not exist" containerID="d21421fd913291ac3ac9b67e9a506128685f1a0d80fcab53e873bc0124e4cfb1" Dec 03 11:25:19 crc kubenswrapper[5052]: I1203 11:25:19.779890 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d21421fd913291ac3ac9b67e9a506128685f1a0d80fcab53e873bc0124e4cfb1"} err="failed to get container status \"d21421fd913291ac3ac9b67e9a506128685f1a0d80fcab53e873bc0124e4cfb1\": rpc error: code = NotFound desc = could not find container \"d21421fd913291ac3ac9b67e9a506128685f1a0d80fcab53e873bc0124e4cfb1\": container with ID starting with d21421fd913291ac3ac9b67e9a506128685f1a0d80fcab53e873bc0124e4cfb1 not found: ID does not exist" Dec 03 11:25:19 crc kubenswrapper[5052]: I1203 11:25:19.783994 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-697d97f7c8-p77bx"] Dec 03 11:25:21 crc kubenswrapper[5052]: I1203 11:25:21.680830 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f5012254-068c-4360-b310-39ff3d309199" path="/var/lib/kubelet/pods/f5012254-068c-4360-b310-39ff3d309199/volumes" Dec 03 11:25:59 crc kubenswrapper[5052]: I1203 11:25:59.380237 5052 patch_prober.go:28] interesting pod/machine-config-daemon-74bj6 container/machine-config-daemon namespace/openshift-machine-config-operator: Liveness probe status=failure output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" start-of-body= Dec 03 11:25:59 crc kubenswrapper[5052]: I1203 11:25:59.381216 5052 prober.go:107] "Probe failed" probeType="Liveness" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" containerName="machine-config-daemon" probeResult="failure" output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" Dec 03 11:26:29 crc kubenswrapper[5052]: I1203 11:26:29.379526 5052 patch_prober.go:28] interesting pod/machine-config-daemon-74bj6 container/machine-config-daemon namespace/openshift-machine-config-operator: Liveness probe status=failure output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" start-of-body= Dec 03 11:26:29 crc kubenswrapper[5052]: I1203 11:26:29.380099 5052 prober.go:107] "Probe failed" probeType="Liveness" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" containerName="machine-config-daemon" probeResult="failure" output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" Dec 03 11:26:59 crc kubenswrapper[5052]: I1203 11:26:59.379315 5052 patch_prober.go:28] interesting pod/machine-config-daemon-74bj6 container/machine-config-daemon namespace/openshift-machine-config-operator: Liveness probe status=failure output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" start-of-body= Dec 03 11:26:59 crc kubenswrapper[5052]: I1203 11:26:59.379884 5052 prober.go:107] "Probe failed" probeType="Liveness" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" containerName="machine-config-daemon" probeResult="failure" output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" Dec 03 11:26:59 crc kubenswrapper[5052]: I1203 11:26:59.379933 5052 kubelet.go:2542] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" Dec 03 11:26:59 crc kubenswrapper[5052]: I1203 11:26:59.380500 5052 kuberuntime_manager.go:1027] "Message for Container of pod" containerName="machine-config-daemon" containerStatusID={"Type":"cri-o","ID":"17b2fe4901a430b4e6aeb90b23c1429860b29094991387152438ff2505fc2e22"} pod="openshift-machine-config-operator/machine-config-daemon-74bj6" containerMessage="Container machine-config-daemon failed liveness probe, will be restarted" Dec 03 11:26:59 crc kubenswrapper[5052]: I1203 11:26:59.380566 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" containerName="machine-config-daemon" containerID="cri-o://17b2fe4901a430b4e6aeb90b23c1429860b29094991387152438ff2505fc2e22" gracePeriod=600 Dec 03 11:27:00 crc kubenswrapper[5052]: I1203 11:27:00.365932 5052 generic.go:334] "Generic (PLEG): container finished" podID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" containerID="17b2fe4901a430b4e6aeb90b23c1429860b29094991387152438ff2505fc2e22" exitCode=0 Dec 03 11:27:00 crc kubenswrapper[5052]: I1203 11:27:00.366168 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" event={"ID":"10c237bc-72e7-4da3-9f2c-b1688395bf6e","Type":"ContainerDied","Data":"17b2fe4901a430b4e6aeb90b23c1429860b29094991387152438ff2505fc2e22"} Dec 03 11:27:00 crc kubenswrapper[5052]: I1203 11:27:00.366703 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" event={"ID":"10c237bc-72e7-4da3-9f2c-b1688395bf6e","Type":"ContainerStarted","Data":"2d08ec4ed134e9ab1a30c6d8ed980a535d93d2cc12d3d85579e39538d5086343"} Dec 03 11:27:00 crc kubenswrapper[5052]: I1203 11:27:00.366732 5052 scope.go:117] "RemoveContainer" containerID="1413108bd79cded55a270a3437cda42aab7c3db3cfb81894fade4363bbb28f2d" Dec 03 11:28:59 crc kubenswrapper[5052]: I1203 11:28:59.379305 5052 patch_prober.go:28] interesting pod/machine-config-daemon-74bj6 container/machine-config-daemon namespace/openshift-machine-config-operator: Liveness probe status=failure output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" start-of-body= Dec 03 11:28:59 crc kubenswrapper[5052]: I1203 11:28:59.379795 5052 prober.go:107] "Probe failed" probeType="Liveness" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" containerName="machine-config-daemon" probeResult="failure" output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" Dec 03 11:29:29 crc kubenswrapper[5052]: I1203 11:29:29.379900 5052 patch_prober.go:28] interesting pod/machine-config-daemon-74bj6 container/machine-config-daemon namespace/openshift-machine-config-operator: Liveness probe status=failure output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" start-of-body= Dec 03 11:29:29 crc kubenswrapper[5052]: I1203 11:29:29.380824 5052 prober.go:107] "Probe failed" probeType="Liveness" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" containerName="machine-config-daemon" probeResult="failure" output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" Dec 03 11:29:59 crc kubenswrapper[5052]: I1203 11:29:59.379653 5052 patch_prober.go:28] interesting pod/machine-config-daemon-74bj6 container/machine-config-daemon namespace/openshift-machine-config-operator: Liveness probe status=failure output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" start-of-body= Dec 03 11:29:59 crc kubenswrapper[5052]: I1203 11:29:59.380233 5052 prober.go:107] "Probe failed" probeType="Liveness" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" containerName="machine-config-daemon" probeResult="failure" output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" Dec 03 11:29:59 crc kubenswrapper[5052]: I1203 11:29:59.380284 5052 kubelet.go:2542] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" Dec 03 11:29:59 crc kubenswrapper[5052]: I1203 11:29:59.380916 5052 kuberuntime_manager.go:1027] "Message for Container of pod" containerName="machine-config-daemon" containerStatusID={"Type":"cri-o","ID":"2d08ec4ed134e9ab1a30c6d8ed980a535d93d2cc12d3d85579e39538d5086343"} pod="openshift-machine-config-operator/machine-config-daemon-74bj6" containerMessage="Container machine-config-daemon failed liveness probe, will be restarted" Dec 03 11:29:59 crc kubenswrapper[5052]: I1203 11:29:59.380966 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" containerName="machine-config-daemon" containerID="cri-o://2d08ec4ed134e9ab1a30c6d8ed980a535d93d2cc12d3d85579e39538d5086343" gracePeriod=600 Dec 03 11:30:00 crc kubenswrapper[5052]: I1203 11:30:00.169962 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openshift-operator-lifecycle-manager/collect-profiles-29412690-ql6kh"] Dec 03 11:30:00 crc kubenswrapper[5052]: E1203 11:30:00.170909 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="f5012254-068c-4360-b310-39ff3d309199" containerName="registry" Dec 03 11:30:00 crc kubenswrapper[5052]: I1203 11:30:00.170934 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="f5012254-068c-4360-b310-39ff3d309199" containerName="registry" Dec 03 11:30:00 crc kubenswrapper[5052]: I1203 11:30:00.171076 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="f5012254-068c-4360-b310-39ff3d309199" containerName="registry" Dec 03 11:30:00 crc kubenswrapper[5052]: I1203 11:30:00.171588 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-operator-lifecycle-manager/collect-profiles-29412690-ql6kh" Dec 03 11:30:00 crc kubenswrapper[5052]: I1203 11:30:00.174079 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-operator-lifecycle-manager"/"collect-profiles-dockercfg-kzf4t" Dec 03 11:30:00 crc kubenswrapper[5052]: I1203 11:30:00.174787 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-operator-lifecycle-manager"/"collect-profiles-config" Dec 03 11:30:00 crc kubenswrapper[5052]: I1203 11:30:00.176313 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-operator-lifecycle-manager/collect-profiles-29412690-ql6kh"] Dec 03 11:30:00 crc kubenswrapper[5052]: I1203 11:30:00.330455 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/1ae0ed52-3a96-42c1-97ed-ae40c39997e1-config-volume\") pod \"collect-profiles-29412690-ql6kh\" (UID: \"1ae0ed52-3a96-42c1-97ed-ae40c39997e1\") " pod="openshift-operator-lifecycle-manager/collect-profiles-29412690-ql6kh" Dec 03 11:30:00 crc kubenswrapper[5052]: I1203 11:30:00.330494 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-volume\" (UniqueName: \"kubernetes.io/secret/1ae0ed52-3a96-42c1-97ed-ae40c39997e1-secret-volume\") pod \"collect-profiles-29412690-ql6kh\" (UID: \"1ae0ed52-3a96-42c1-97ed-ae40c39997e1\") " pod="openshift-operator-lifecycle-manager/collect-profiles-29412690-ql6kh" Dec 03 11:30:00 crc kubenswrapper[5052]: I1203 11:30:00.330580 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5h68m\" (UniqueName: \"kubernetes.io/projected/1ae0ed52-3a96-42c1-97ed-ae40c39997e1-kube-api-access-5h68m\") pod \"collect-profiles-29412690-ql6kh\" (UID: \"1ae0ed52-3a96-42c1-97ed-ae40c39997e1\") " pod="openshift-operator-lifecycle-manager/collect-profiles-29412690-ql6kh" Dec 03 11:30:00 crc kubenswrapper[5052]: I1203 11:30:00.362481 5052 generic.go:334] "Generic (PLEG): container finished" podID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" containerID="2d08ec4ed134e9ab1a30c6d8ed980a535d93d2cc12d3d85579e39538d5086343" exitCode=0 Dec 03 11:30:00 crc kubenswrapper[5052]: I1203 11:30:00.362538 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" event={"ID":"10c237bc-72e7-4da3-9f2c-b1688395bf6e","Type":"ContainerDied","Data":"2d08ec4ed134e9ab1a30c6d8ed980a535d93d2cc12d3d85579e39538d5086343"} Dec 03 11:30:00 crc kubenswrapper[5052]: I1203 11:30:00.362568 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" event={"ID":"10c237bc-72e7-4da3-9f2c-b1688395bf6e","Type":"ContainerStarted","Data":"a8ea894737897a85494be3992d10d75dae39643d0a55aa5bfc62a4269324bdcd"} Dec 03 11:30:00 crc kubenswrapper[5052]: I1203 11:30:00.362585 5052 scope.go:117] "RemoveContainer" containerID="17b2fe4901a430b4e6aeb90b23c1429860b29094991387152438ff2505fc2e22" Dec 03 11:30:00 crc kubenswrapper[5052]: I1203 11:30:00.431673 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-5h68m\" (UniqueName: \"kubernetes.io/projected/1ae0ed52-3a96-42c1-97ed-ae40c39997e1-kube-api-access-5h68m\") pod \"collect-profiles-29412690-ql6kh\" (UID: \"1ae0ed52-3a96-42c1-97ed-ae40c39997e1\") " pod="openshift-operator-lifecycle-manager/collect-profiles-29412690-ql6kh" Dec 03 11:30:00 crc kubenswrapper[5052]: I1203 11:30:00.431767 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/1ae0ed52-3a96-42c1-97ed-ae40c39997e1-config-volume\") pod \"collect-profiles-29412690-ql6kh\" (UID: \"1ae0ed52-3a96-42c1-97ed-ae40c39997e1\") " pod="openshift-operator-lifecycle-manager/collect-profiles-29412690-ql6kh" Dec 03 11:30:00 crc kubenswrapper[5052]: I1203 11:30:00.431797 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"secret-volume\" (UniqueName: \"kubernetes.io/secret/1ae0ed52-3a96-42c1-97ed-ae40c39997e1-secret-volume\") pod \"collect-profiles-29412690-ql6kh\" (UID: \"1ae0ed52-3a96-42c1-97ed-ae40c39997e1\") " pod="openshift-operator-lifecycle-manager/collect-profiles-29412690-ql6kh" Dec 03 11:30:00 crc kubenswrapper[5052]: I1203 11:30:00.433119 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/1ae0ed52-3a96-42c1-97ed-ae40c39997e1-config-volume\") pod \"collect-profiles-29412690-ql6kh\" (UID: \"1ae0ed52-3a96-42c1-97ed-ae40c39997e1\") " pod="openshift-operator-lifecycle-manager/collect-profiles-29412690-ql6kh" Dec 03 11:30:00 crc kubenswrapper[5052]: I1203 11:30:00.437975 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"secret-volume\" (UniqueName: \"kubernetes.io/secret/1ae0ed52-3a96-42c1-97ed-ae40c39997e1-secret-volume\") pod \"collect-profiles-29412690-ql6kh\" (UID: \"1ae0ed52-3a96-42c1-97ed-ae40c39997e1\") " pod="openshift-operator-lifecycle-manager/collect-profiles-29412690-ql6kh" Dec 03 11:30:00 crc kubenswrapper[5052]: I1203 11:30:00.448151 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-5h68m\" (UniqueName: \"kubernetes.io/projected/1ae0ed52-3a96-42c1-97ed-ae40c39997e1-kube-api-access-5h68m\") pod \"collect-profiles-29412690-ql6kh\" (UID: \"1ae0ed52-3a96-42c1-97ed-ae40c39997e1\") " pod="openshift-operator-lifecycle-manager/collect-profiles-29412690-ql6kh" Dec 03 11:30:00 crc kubenswrapper[5052]: I1203 11:30:00.487666 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-operator-lifecycle-manager/collect-profiles-29412690-ql6kh" Dec 03 11:30:00 crc kubenswrapper[5052]: I1203 11:30:00.673377 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-operator-lifecycle-manager/collect-profiles-29412690-ql6kh"] Dec 03 11:30:01 crc kubenswrapper[5052]: I1203 11:30:01.373043 5052 generic.go:334] "Generic (PLEG): container finished" podID="1ae0ed52-3a96-42c1-97ed-ae40c39997e1" containerID="015b047fdd13acbbfc273d6c1ab47adc405079b2a4c2c1faa5e8cf0065aec661" exitCode=0 Dec 03 11:30:01 crc kubenswrapper[5052]: I1203 11:30:01.373155 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-operator-lifecycle-manager/collect-profiles-29412690-ql6kh" event={"ID":"1ae0ed52-3a96-42c1-97ed-ae40c39997e1","Type":"ContainerDied","Data":"015b047fdd13acbbfc273d6c1ab47adc405079b2a4c2c1faa5e8cf0065aec661"} Dec 03 11:30:01 crc kubenswrapper[5052]: I1203 11:30:01.373391 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-operator-lifecycle-manager/collect-profiles-29412690-ql6kh" event={"ID":"1ae0ed52-3a96-42c1-97ed-ae40c39997e1","Type":"ContainerStarted","Data":"dedd6ce5db2cd9b5fb457d63ed6690e36a50cabc55b2b74047d9582a206898ad"} Dec 03 11:30:02 crc kubenswrapper[5052]: I1203 11:30:02.566710 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-operator-lifecycle-manager/collect-profiles-29412690-ql6kh" Dec 03 11:30:02 crc kubenswrapper[5052]: I1203 11:30:02.757813 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"secret-volume\" (UniqueName: \"kubernetes.io/secret/1ae0ed52-3a96-42c1-97ed-ae40c39997e1-secret-volume\") pod \"1ae0ed52-3a96-42c1-97ed-ae40c39997e1\" (UID: \"1ae0ed52-3a96-42c1-97ed-ae40c39997e1\") " Dec 03 11:30:02 crc kubenswrapper[5052]: I1203 11:30:02.758175 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5h68m\" (UniqueName: \"kubernetes.io/projected/1ae0ed52-3a96-42c1-97ed-ae40c39997e1-kube-api-access-5h68m\") pod \"1ae0ed52-3a96-42c1-97ed-ae40c39997e1\" (UID: \"1ae0ed52-3a96-42c1-97ed-ae40c39997e1\") " Dec 03 11:30:02 crc kubenswrapper[5052]: I1203 11:30:02.758233 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/1ae0ed52-3a96-42c1-97ed-ae40c39997e1-config-volume\") pod \"1ae0ed52-3a96-42c1-97ed-ae40c39997e1\" (UID: \"1ae0ed52-3a96-42c1-97ed-ae40c39997e1\") " Dec 03 11:30:02 crc kubenswrapper[5052]: I1203 11:30:02.759032 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1ae0ed52-3a96-42c1-97ed-ae40c39997e1-config-volume" (OuterVolumeSpecName: "config-volume") pod "1ae0ed52-3a96-42c1-97ed-ae40c39997e1" (UID: "1ae0ed52-3a96-42c1-97ed-ae40c39997e1"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:30:02 crc kubenswrapper[5052]: I1203 11:30:02.763648 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1ae0ed52-3a96-42c1-97ed-ae40c39997e1-kube-api-access-5h68m" (OuterVolumeSpecName: "kube-api-access-5h68m") pod "1ae0ed52-3a96-42c1-97ed-ae40c39997e1" (UID: "1ae0ed52-3a96-42c1-97ed-ae40c39997e1"). InnerVolumeSpecName "kube-api-access-5h68m". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:30:02 crc kubenswrapper[5052]: I1203 11:30:02.770775 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1ae0ed52-3a96-42c1-97ed-ae40c39997e1-secret-volume" (OuterVolumeSpecName: "secret-volume") pod "1ae0ed52-3a96-42c1-97ed-ae40c39997e1" (UID: "1ae0ed52-3a96-42c1-97ed-ae40c39997e1"). InnerVolumeSpecName "secret-volume". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:30:02 crc kubenswrapper[5052]: I1203 11:30:02.858966 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-5h68m\" (UniqueName: \"kubernetes.io/projected/1ae0ed52-3a96-42c1-97ed-ae40c39997e1-kube-api-access-5h68m\") on node \"crc\" DevicePath \"\"" Dec 03 11:30:02 crc kubenswrapper[5052]: I1203 11:30:02.859004 5052 reconciler_common.go:293] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/1ae0ed52-3a96-42c1-97ed-ae40c39997e1-config-volume\") on node \"crc\" DevicePath \"\"" Dec 03 11:30:02 crc kubenswrapper[5052]: I1203 11:30:02.859013 5052 reconciler_common.go:293] "Volume detached for volume \"secret-volume\" (UniqueName: \"kubernetes.io/secret/1ae0ed52-3a96-42c1-97ed-ae40c39997e1-secret-volume\") on node \"crc\" DevicePath \"\"" Dec 03 11:30:03 crc kubenswrapper[5052]: I1203 11:30:03.383604 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-operator-lifecycle-manager/collect-profiles-29412690-ql6kh" event={"ID":"1ae0ed52-3a96-42c1-97ed-ae40c39997e1","Type":"ContainerDied","Data":"dedd6ce5db2cd9b5fb457d63ed6690e36a50cabc55b2b74047d9582a206898ad"} Dec 03 11:30:03 crc kubenswrapper[5052]: I1203 11:30:03.383670 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="dedd6ce5db2cd9b5fb457d63ed6690e36a50cabc55b2b74047d9582a206898ad" Dec 03 11:30:03 crc kubenswrapper[5052]: I1203 11:30:03.383732 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-operator-lifecycle-manager/collect-profiles-29412690-ql6kh" Dec 03 11:30:12 crc kubenswrapper[5052]: I1203 11:30:12.400113 5052 scope.go:117] "RemoveContainer" containerID="06be2622fe2fd858b7a245a2b20ce8ef132f8e1b4ef5bdc4c125056c471e3a96" Dec 03 11:30:59 crc kubenswrapper[5052]: I1203 11:30:59.456074 5052 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.070302 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openshift-ovn-kubernetes/ovnkube-node-kg55h"] Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.072671 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openshift-ovn-kubernetes/ovnkube-node-kg55h" podUID="bc569bba-c020-40d6-af68-e617625fe0e4" containerName="ovn-controller" containerID="cri-o://2363c55c03e9159893688a52c5ba318a997f018c828f223626074a58918ccc24" gracePeriod=30 Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.072719 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openshift-ovn-kubernetes/ovnkube-node-kg55h" podUID="bc569bba-c020-40d6-af68-e617625fe0e4" containerName="nbdb" containerID="cri-o://c16097e9c17ec2d194aef1c4af913073743ff9b3e54b20999b8b48ce26176ecb" gracePeriod=30 Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.072829 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openshift-ovn-kubernetes/ovnkube-node-kg55h" podUID="bc569bba-c020-40d6-af68-e617625fe0e4" containerName="northd" containerID="cri-o://5b3ab2b483c488f2669acaf42538650e9c65cd6da9b9a5296572ed8f4702a685" gracePeriod=30 Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.072854 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openshift-ovn-kubernetes/ovnkube-node-kg55h" podUID="bc569bba-c020-40d6-af68-e617625fe0e4" containerName="kube-rbac-proxy-node" containerID="cri-o://bde78769f16ee1d0101355c5c8eb4a48437c04ccc42ca5104d1b98538b2545be" gracePeriod=30 Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.072798 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openshift-ovn-kubernetes/ovnkube-node-kg55h" podUID="bc569bba-c020-40d6-af68-e617625fe0e4" containerName="kube-rbac-proxy-ovn-metrics" containerID="cri-o://9201f740cc3848eee706f3415f90bbffac06b7df3bb60f3495eff3c1d0a7f3a8" gracePeriod=30 Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.072895 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openshift-ovn-kubernetes/ovnkube-node-kg55h" podUID="bc569bba-c020-40d6-af68-e617625fe0e4" containerName="ovn-acl-logging" containerID="cri-o://896c25e34b1cb8871fd92cc6c8c908bfce14a9713345f571a672b2156a1765bf" gracePeriod=30 Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.072876 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openshift-ovn-kubernetes/ovnkube-node-kg55h" podUID="bc569bba-c020-40d6-af68-e617625fe0e4" containerName="sbdb" containerID="cri-o://affc8d8f111572fbded3dbcc4c9f374ca5ba664cd6989bb1e717c163a1d88417" gracePeriod=30 Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.122713 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openshift-ovn-kubernetes/ovnkube-node-kg55h" podUID="bc569bba-c020-40d6-af68-e617625fe0e4" containerName="ovnkube-controller" containerID="cri-o://7548847d4c67dfe4acdf79ad3da86e54d4766ed7657a83a725c1ecc4a22c44dd" gracePeriod=30 Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.411197 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kg55h_bc569bba-c020-40d6-af68-e617625fe0e4/ovn-acl-logging/0.log" Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.412663 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kg55h_bc569bba-c020-40d6-af68-e617625fe0e4/ovn-controller/0.log" Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.413132 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-kg55h" Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.467573 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openshift-ovn-kubernetes/ovnkube-node-rkzqc"] Dec 03 11:31:37 crc kubenswrapper[5052]: E1203 11:31:37.467820 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="bc569bba-c020-40d6-af68-e617625fe0e4" containerName="sbdb" Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.467834 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="bc569bba-c020-40d6-af68-e617625fe0e4" containerName="sbdb" Dec 03 11:31:37 crc kubenswrapper[5052]: E1203 11:31:37.467849 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="bc569bba-c020-40d6-af68-e617625fe0e4" containerName="ovnkube-controller" Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.467856 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="bc569bba-c020-40d6-af68-e617625fe0e4" containerName="ovnkube-controller" Dec 03 11:31:37 crc kubenswrapper[5052]: E1203 11:31:37.467866 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="bc569bba-c020-40d6-af68-e617625fe0e4" containerName="northd" Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.467875 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="bc569bba-c020-40d6-af68-e617625fe0e4" containerName="northd" Dec 03 11:31:37 crc kubenswrapper[5052]: E1203 11:31:37.467884 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="bc569bba-c020-40d6-af68-e617625fe0e4" containerName="ovn-controller" Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.467893 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="bc569bba-c020-40d6-af68-e617625fe0e4" containerName="ovn-controller" Dec 03 11:31:37 crc kubenswrapper[5052]: E1203 11:31:37.467904 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="bc569bba-c020-40d6-af68-e617625fe0e4" containerName="ovn-acl-logging" Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.467911 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="bc569bba-c020-40d6-af68-e617625fe0e4" containerName="ovn-acl-logging" Dec 03 11:31:37 crc kubenswrapper[5052]: E1203 11:31:37.467926 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="bc569bba-c020-40d6-af68-e617625fe0e4" containerName="nbdb" Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.467933 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="bc569bba-c020-40d6-af68-e617625fe0e4" containerName="nbdb" Dec 03 11:31:37 crc kubenswrapper[5052]: E1203 11:31:37.467945 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="1ae0ed52-3a96-42c1-97ed-ae40c39997e1" containerName="collect-profiles" Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.467952 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="1ae0ed52-3a96-42c1-97ed-ae40c39997e1" containerName="collect-profiles" Dec 03 11:31:37 crc kubenswrapper[5052]: E1203 11:31:37.467962 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="bc569bba-c020-40d6-af68-e617625fe0e4" containerName="kube-rbac-proxy-ovn-metrics" Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.467970 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="bc569bba-c020-40d6-af68-e617625fe0e4" containerName="kube-rbac-proxy-ovn-metrics" Dec 03 11:31:37 crc kubenswrapper[5052]: E1203 11:31:37.467982 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="bc569bba-c020-40d6-af68-e617625fe0e4" containerName="kubecfg-setup" Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.467990 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="bc569bba-c020-40d6-af68-e617625fe0e4" containerName="kubecfg-setup" Dec 03 11:31:37 crc kubenswrapper[5052]: E1203 11:31:37.468001 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="bc569bba-c020-40d6-af68-e617625fe0e4" containerName="kube-rbac-proxy-node" Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.468008 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="bc569bba-c020-40d6-af68-e617625fe0e4" containerName="kube-rbac-proxy-node" Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.468108 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="bc569bba-c020-40d6-af68-e617625fe0e4" containerName="kube-rbac-proxy-node" Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.468121 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="bc569bba-c020-40d6-af68-e617625fe0e4" containerName="kube-rbac-proxy-ovn-metrics" Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.468132 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="bc569bba-c020-40d6-af68-e617625fe0e4" containerName="sbdb" Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.468146 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="bc569bba-c020-40d6-af68-e617625fe0e4" containerName="ovnkube-controller" Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.468155 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="bc569bba-c020-40d6-af68-e617625fe0e4" containerName="northd" Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.468165 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="1ae0ed52-3a96-42c1-97ed-ae40c39997e1" containerName="collect-profiles" Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.468175 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="bc569bba-c020-40d6-af68-e617625fe0e4" containerName="ovn-acl-logging" Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.468184 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="bc569bba-c020-40d6-af68-e617625fe0e4" containerName="nbdb" Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.468193 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="bc569bba-c020-40d6-af68-e617625fe0e4" containerName="ovn-controller" Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.469999 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-rkzqc" Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.528149 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/bc569bba-c020-40d6-af68-e617625fe0e4-run-ovn\") pod \"bc569bba-c020-40d6-af68-e617625fe0e4\" (UID: \"bc569bba-c020-40d6-af68-e617625fe0e4\") " Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.528205 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/bc569bba-c020-40d6-af68-e617625fe0e4-host-kubelet\") pod \"bc569bba-c020-40d6-af68-e617625fe0e4\" (UID: \"bc569bba-c020-40d6-af68-e617625fe0e4\") " Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.528223 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bc569bba-c020-40d6-af68-e617625fe0e4-var-lib-openvswitch\") pod \"bc569bba-c020-40d6-af68-e617625fe0e4\" (UID: \"bc569bba-c020-40d6-af68-e617625fe0e4\") " Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.528235 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/bc569bba-c020-40d6-af68-e617625fe0e4-run-ovn" (OuterVolumeSpecName: "run-ovn") pod "bc569bba-c020-40d6-af68-e617625fe0e4" (UID: "bc569bba-c020-40d6-af68-e617625fe0e4"). InnerVolumeSpecName "run-ovn". PluginName "kubernetes.io/host-path", VolumeGidValue "" Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.528249 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bc569bba-c020-40d6-af68-e617625fe0e4-run-openvswitch\") pod \"bc569bba-c020-40d6-af68-e617625fe0e4\" (UID: \"bc569bba-c020-40d6-af68-e617625fe0e4\") " Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.528273 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/bc569bba-c020-40d6-af68-e617625fe0e4-run-openvswitch" (OuterVolumeSpecName: "run-openvswitch") pod "bc569bba-c020-40d6-af68-e617625fe0e4" (UID: "bc569bba-c020-40d6-af68-e617625fe0e4"). InnerVolumeSpecName "run-openvswitch". PluginName "kubernetes.io/host-path", VolumeGidValue "" Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.528294 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bc569bba-c020-40d6-af68-e617625fe0e4-host-cni-bin\") pod \"bc569bba-c020-40d6-af68-e617625fe0e4\" (UID: \"bc569bba-c020-40d6-af68-e617625fe0e4\") " Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.528295 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/bc569bba-c020-40d6-af68-e617625fe0e4-var-lib-openvswitch" (OuterVolumeSpecName: "var-lib-openvswitch") pod "bc569bba-c020-40d6-af68-e617625fe0e4" (UID: "bc569bba-c020-40d6-af68-e617625fe0e4"). InnerVolumeSpecName "var-lib-openvswitch". PluginName "kubernetes.io/host-path", VolumeGidValue "" Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.528325 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/bc569bba-c020-40d6-af68-e617625fe0e4-systemd-units\") pod \"bc569bba-c020-40d6-af68-e617625fe0e4\" (UID: \"bc569bba-c020-40d6-af68-e617625fe0e4\") " Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.528324 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/bc569bba-c020-40d6-af68-e617625fe0e4-host-kubelet" (OuterVolumeSpecName: "host-kubelet") pod "bc569bba-c020-40d6-af68-e617625fe0e4" (UID: "bc569bba-c020-40d6-af68-e617625fe0e4"). InnerVolumeSpecName "host-kubelet". PluginName "kubernetes.io/host-path", VolumeGidValue "" Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.528352 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x92h2\" (UniqueName: \"kubernetes.io/projected/bc569bba-c020-40d6-af68-e617625fe0e4-kube-api-access-x92h2\") pod \"bc569bba-c020-40d6-af68-e617625fe0e4\" (UID: \"bc569bba-c020-40d6-af68-e617625fe0e4\") " Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.528363 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/bc569bba-c020-40d6-af68-e617625fe0e4-systemd-units" (OuterVolumeSpecName: "systemd-units") pod "bc569bba-c020-40d6-af68-e617625fe0e4" (UID: "bc569bba-c020-40d6-af68-e617625fe0e4"). InnerVolumeSpecName "systemd-units". PluginName "kubernetes.io/host-path", VolumeGidValue "" Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.528378 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/bc569bba-c020-40d6-af68-e617625fe0e4-ovn-node-metrics-cert\") pod \"bc569bba-c020-40d6-af68-e617625fe0e4\" (UID: \"bc569bba-c020-40d6-af68-e617625fe0e4\") " Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.528403 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/bc569bba-c020-40d6-af68-e617625fe0e4-ovnkube-script-lib\") pod \"bc569bba-c020-40d6-af68-e617625fe0e4\" (UID: \"bc569bba-c020-40d6-af68-e617625fe0e4\") " Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.528425 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/bc569bba-c020-40d6-af68-e617625fe0e4-node-log\") pod \"bc569bba-c020-40d6-af68-e617625fe0e4\" (UID: \"bc569bba-c020-40d6-af68-e617625fe0e4\") " Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.528476 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bc569bba-c020-40d6-af68-e617625fe0e4-host-slash\") pod \"bc569bba-c020-40d6-af68-e617625fe0e4\" (UID: \"bc569bba-c020-40d6-af68-e617625fe0e4\") " Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.528407 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/bc569bba-c020-40d6-af68-e617625fe0e4-host-cni-bin" (OuterVolumeSpecName: "host-cni-bin") pod "bc569bba-c020-40d6-af68-e617625fe0e4" (UID: "bc569bba-c020-40d6-af68-e617625fe0e4"). InnerVolumeSpecName "host-cni-bin". PluginName "kubernetes.io/host-path", VolumeGidValue "" Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.528452 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/bc569bba-c020-40d6-af68-e617625fe0e4-node-log" (OuterVolumeSpecName: "node-log") pod "bc569bba-c020-40d6-af68-e617625fe0e4" (UID: "bc569bba-c020-40d6-af68-e617625fe0e4"). InnerVolumeSpecName "node-log". PluginName "kubernetes.io/host-path", VolumeGidValue "" Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.528567 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/bc569bba-c020-40d6-af68-e617625fe0e4-log-socket" (OuterVolumeSpecName: "log-socket") pod "bc569bba-c020-40d6-af68-e617625fe0e4" (UID: "bc569bba-c020-40d6-af68-e617625fe0e4"). InnerVolumeSpecName "log-socket". PluginName "kubernetes.io/host-path", VolumeGidValue "" Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.528501 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/bc569bba-c020-40d6-af68-e617625fe0e4-log-socket\") pod \"bc569bba-c020-40d6-af68-e617625fe0e4\" (UID: \"bc569bba-c020-40d6-af68-e617625fe0e4\") " Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.528631 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bc569bba-c020-40d6-af68-e617625fe0e4-host-var-lib-cni-networks-ovn-kubernetes\") pod \"bc569bba-c020-40d6-af68-e617625fe0e4\" (UID: \"bc569bba-c020-40d6-af68-e617625fe0e4\") " Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.528588 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/bc569bba-c020-40d6-af68-e617625fe0e4-host-slash" (OuterVolumeSpecName: "host-slash") pod "bc569bba-c020-40d6-af68-e617625fe0e4" (UID: "bc569bba-c020-40d6-af68-e617625fe0e4"). InnerVolumeSpecName "host-slash". PluginName "kubernetes.io/host-path", VolumeGidValue "" Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.528679 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bc569bba-c020-40d6-af68-e617625fe0e4-host-run-ovn-kubernetes\") pod \"bc569bba-c020-40d6-af68-e617625fe0e4\" (UID: \"bc569bba-c020-40d6-af68-e617625fe0e4\") " Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.528706 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/bc569bba-c020-40d6-af68-e617625fe0e4-host-var-lib-cni-networks-ovn-kubernetes" (OuterVolumeSpecName: "host-var-lib-cni-networks-ovn-kubernetes") pod "bc569bba-c020-40d6-af68-e617625fe0e4" (UID: "bc569bba-c020-40d6-af68-e617625fe0e4"). InnerVolumeSpecName "host-var-lib-cni-networks-ovn-kubernetes". PluginName "kubernetes.io/host-path", VolumeGidValue "" Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.528722 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/bc569bba-c020-40d6-af68-e617625fe0e4-env-overrides\") pod \"bc569bba-c020-40d6-af68-e617625fe0e4\" (UID: \"bc569bba-c020-40d6-af68-e617625fe0e4\") " Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.528748 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/bc569bba-c020-40d6-af68-e617625fe0e4-host-cni-netd\") pod \"bc569bba-c020-40d6-af68-e617625fe0e4\" (UID: \"bc569bba-c020-40d6-af68-e617625fe0e4\") " Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.528754 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/bc569bba-c020-40d6-af68-e617625fe0e4-host-run-ovn-kubernetes" (OuterVolumeSpecName: "host-run-ovn-kubernetes") pod "bc569bba-c020-40d6-af68-e617625fe0e4" (UID: "bc569bba-c020-40d6-af68-e617625fe0e4"). InnerVolumeSpecName "host-run-ovn-kubernetes". PluginName "kubernetes.io/host-path", VolumeGidValue "" Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.528773 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bc569bba-c020-40d6-af68-e617625fe0e4-etc-openvswitch\") pod \"bc569bba-c020-40d6-af68-e617625fe0e4\" (UID: \"bc569bba-c020-40d6-af68-e617625fe0e4\") " Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.528806 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/bc569bba-c020-40d6-af68-e617625fe0e4-ovnkube-config\") pod \"bc569bba-c020-40d6-af68-e617625fe0e4\" (UID: \"bc569bba-c020-40d6-af68-e617625fe0e4\") " Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.528828 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/bc569bba-c020-40d6-af68-e617625fe0e4-run-systemd\") pod \"bc569bba-c020-40d6-af68-e617625fe0e4\" (UID: \"bc569bba-c020-40d6-af68-e617625fe0e4\") " Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.528848 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bc569bba-c020-40d6-af68-e617625fe0e4-host-run-netns\") pod \"bc569bba-c020-40d6-af68-e617625fe0e4\" (UID: \"bc569bba-c020-40d6-af68-e617625fe0e4\") " Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.528943 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/bc569bba-c020-40d6-af68-e617625fe0e4-etc-openvswitch" (OuterVolumeSpecName: "etc-openvswitch") pod "bc569bba-c020-40d6-af68-e617625fe0e4" (UID: "bc569bba-c020-40d6-af68-e617625fe0e4"). InnerVolumeSpecName "etc-openvswitch". PluginName "kubernetes.io/host-path", VolumeGidValue "" Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.528986 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/bc569bba-c020-40d6-af68-e617625fe0e4-host-cni-netd" (OuterVolumeSpecName: "host-cni-netd") pod "bc569bba-c020-40d6-af68-e617625fe0e4" (UID: "bc569bba-c020-40d6-af68-e617625fe0e4"). InnerVolumeSpecName "host-cni-netd". PluginName "kubernetes.io/host-path", VolumeGidValue "" Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.529055 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/bc569bba-c020-40d6-af68-e617625fe0e4-host-run-netns" (OuterVolumeSpecName: "host-run-netns") pod "bc569bba-c020-40d6-af68-e617625fe0e4" (UID: "bc569bba-c020-40d6-af68-e617625fe0e4"). InnerVolumeSpecName "host-run-netns". PluginName "kubernetes.io/host-path", VolumeGidValue "" Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.529224 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bc569bba-c020-40d6-af68-e617625fe0e4-env-overrides" (OuterVolumeSpecName: "env-overrides") pod "bc569bba-c020-40d6-af68-e617625fe0e4" (UID: "bc569bba-c020-40d6-af68-e617625fe0e4"). InnerVolumeSpecName "env-overrides". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.529262 5052 reconciler_common.go:293] "Volume detached for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/bc569bba-c020-40d6-af68-e617625fe0e4-host-cni-netd\") on node \"crc\" DevicePath \"\"" Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.529280 5052 reconciler_common.go:293] "Volume detached for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bc569bba-c020-40d6-af68-e617625fe0e4-etc-openvswitch\") on node \"crc\" DevicePath \"\"" Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.529293 5052 reconciler_common.go:293] "Volume detached for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bc569bba-c020-40d6-af68-e617625fe0e4-host-run-netns\") on node \"crc\" DevicePath \"\"" Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.529307 5052 reconciler_common.go:293] "Volume detached for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/bc569bba-c020-40d6-af68-e617625fe0e4-run-ovn\") on node \"crc\" DevicePath \"\"" Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.529319 5052 reconciler_common.go:293] "Volume detached for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/bc569bba-c020-40d6-af68-e617625fe0e4-host-kubelet\") on node \"crc\" DevicePath \"\"" Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.529330 5052 reconciler_common.go:293] "Volume detached for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bc569bba-c020-40d6-af68-e617625fe0e4-var-lib-openvswitch\") on node \"crc\" DevicePath \"\"" Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.529343 5052 reconciler_common.go:293] "Volume detached for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bc569bba-c020-40d6-af68-e617625fe0e4-run-openvswitch\") on node \"crc\" DevicePath \"\"" Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.529354 5052 reconciler_common.go:293] "Volume detached for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bc569bba-c020-40d6-af68-e617625fe0e4-host-cni-bin\") on node \"crc\" DevicePath \"\"" Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.529366 5052 reconciler_common.go:293] "Volume detached for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/bc569bba-c020-40d6-af68-e617625fe0e4-systemd-units\") on node \"crc\" DevicePath \"\"" Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.529377 5052 reconciler_common.go:293] "Volume detached for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/bc569bba-c020-40d6-af68-e617625fe0e4-node-log\") on node \"crc\" DevicePath \"\"" Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.529388 5052 reconciler_common.go:293] "Volume detached for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bc569bba-c020-40d6-af68-e617625fe0e4-host-slash\") on node \"crc\" DevicePath \"\"" Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.529399 5052 reconciler_common.go:293] "Volume detached for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/bc569bba-c020-40d6-af68-e617625fe0e4-log-socket\") on node \"crc\" DevicePath \"\"" Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.529395 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bc569bba-c020-40d6-af68-e617625fe0e4-ovnkube-script-lib" (OuterVolumeSpecName: "ovnkube-script-lib") pod "bc569bba-c020-40d6-af68-e617625fe0e4" (UID: "bc569bba-c020-40d6-af68-e617625fe0e4"). InnerVolumeSpecName "ovnkube-script-lib". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.529412 5052 reconciler_common.go:293] "Volume detached for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bc569bba-c020-40d6-af68-e617625fe0e4-host-var-lib-cni-networks-ovn-kubernetes\") on node \"crc\" DevicePath \"\"" Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.529443 5052 reconciler_common.go:293] "Volume detached for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bc569bba-c020-40d6-af68-e617625fe0e4-host-run-ovn-kubernetes\") on node \"crc\" DevicePath \"\"" Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.529634 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bc569bba-c020-40d6-af68-e617625fe0e4-ovnkube-config" (OuterVolumeSpecName: "ovnkube-config") pod "bc569bba-c020-40d6-af68-e617625fe0e4" (UID: "bc569bba-c020-40d6-af68-e617625fe0e4"). InnerVolumeSpecName "ovnkube-config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.534399 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bc569bba-c020-40d6-af68-e617625fe0e4-ovn-node-metrics-cert" (OuterVolumeSpecName: "ovn-node-metrics-cert") pod "bc569bba-c020-40d6-af68-e617625fe0e4" (UID: "bc569bba-c020-40d6-af68-e617625fe0e4"). InnerVolumeSpecName "ovn-node-metrics-cert". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.534425 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bc569bba-c020-40d6-af68-e617625fe0e4-kube-api-access-x92h2" (OuterVolumeSpecName: "kube-api-access-x92h2") pod "bc569bba-c020-40d6-af68-e617625fe0e4" (UID: "bc569bba-c020-40d6-af68-e617625fe0e4"). InnerVolumeSpecName "kube-api-access-x92h2". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.542298 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/bc569bba-c020-40d6-af68-e617625fe0e4-run-systemd" (OuterVolumeSpecName: "run-systemd") pod "bc569bba-c020-40d6-af68-e617625fe0e4" (UID: "bc569bba-c020-40d6-af68-e617625fe0e4"). InnerVolumeSpecName "run-systemd". PluginName "kubernetes.io/host-path", VolumeGidValue "" Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.631159 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/17e134b4-4012-46f9-9824-0e3df9873d09-systemd-units\") pod \"ovnkube-node-rkzqc\" (UID: \"17e134b4-4012-46f9-9824-0e3df9873d09\") " pod="openshift-ovn-kubernetes/ovnkube-node-rkzqc" Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.631454 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/17e134b4-4012-46f9-9824-0e3df9873d09-host-run-ovn-kubernetes\") pod \"ovnkube-node-rkzqc\" (UID: \"17e134b4-4012-46f9-9824-0e3df9873d09\") " pod="openshift-ovn-kubernetes/ovnkube-node-rkzqc" Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.631633 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/17e134b4-4012-46f9-9824-0e3df9873d09-var-lib-openvswitch\") pod \"ovnkube-node-rkzqc\" (UID: \"17e134b4-4012-46f9-9824-0e3df9873d09\") " pod="openshift-ovn-kubernetes/ovnkube-node-rkzqc" Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.631671 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/17e134b4-4012-46f9-9824-0e3df9873d09-node-log\") pod \"ovnkube-node-rkzqc\" (UID: \"17e134b4-4012-46f9-9824-0e3df9873d09\") " pod="openshift-ovn-kubernetes/ovnkube-node-rkzqc" Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.631740 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/17e134b4-4012-46f9-9824-0e3df9873d09-host-kubelet\") pod \"ovnkube-node-rkzqc\" (UID: \"17e134b4-4012-46f9-9824-0e3df9873d09\") " pod="openshift-ovn-kubernetes/ovnkube-node-rkzqc" Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.631820 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/17e134b4-4012-46f9-9824-0e3df9873d09-env-overrides\") pod \"ovnkube-node-rkzqc\" (UID: \"17e134b4-4012-46f9-9824-0e3df9873d09\") " pod="openshift-ovn-kubernetes/ovnkube-node-rkzqc" Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.631888 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/17e134b4-4012-46f9-9824-0e3df9873d09-run-ovn\") pod \"ovnkube-node-rkzqc\" (UID: \"17e134b4-4012-46f9-9824-0e3df9873d09\") " pod="openshift-ovn-kubernetes/ovnkube-node-rkzqc" Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.632048 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/17e134b4-4012-46f9-9824-0e3df9873d09-run-systemd\") pod \"ovnkube-node-rkzqc\" (UID: \"17e134b4-4012-46f9-9824-0e3df9873d09\") " pod="openshift-ovn-kubernetes/ovnkube-node-rkzqc" Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.632095 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/17e134b4-4012-46f9-9824-0e3df9873d09-host-cni-bin\") pod \"ovnkube-node-rkzqc\" (UID: \"17e134b4-4012-46f9-9824-0e3df9873d09\") " pod="openshift-ovn-kubernetes/ovnkube-node-rkzqc" Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.632139 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/17e134b4-4012-46f9-9824-0e3df9873d09-log-socket\") pod \"ovnkube-node-rkzqc\" (UID: \"17e134b4-4012-46f9-9824-0e3df9873d09\") " pod="openshift-ovn-kubernetes/ovnkube-node-rkzqc" Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.632295 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/17e134b4-4012-46f9-9824-0e3df9873d09-etc-openvswitch\") pod \"ovnkube-node-rkzqc\" (UID: \"17e134b4-4012-46f9-9824-0e3df9873d09\") " pod="openshift-ovn-kubernetes/ovnkube-node-rkzqc" Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.632435 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/17e134b4-4012-46f9-9824-0e3df9873d09-ovnkube-script-lib\") pod \"ovnkube-node-rkzqc\" (UID: \"17e134b4-4012-46f9-9824-0e3df9873d09\") " pod="openshift-ovn-kubernetes/ovnkube-node-rkzqc" Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.632746 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/17e134b4-4012-46f9-9824-0e3df9873d09-host-run-netns\") pod \"ovnkube-node-rkzqc\" (UID: \"17e134b4-4012-46f9-9824-0e3df9873d09\") " pod="openshift-ovn-kubernetes/ovnkube-node-rkzqc" Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.632875 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/17e134b4-4012-46f9-9824-0e3df9873d09-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-rkzqc\" (UID: \"17e134b4-4012-46f9-9824-0e3df9873d09\") " pod="openshift-ovn-kubernetes/ovnkube-node-rkzqc" Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.633076 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/17e134b4-4012-46f9-9824-0e3df9873d09-ovn-node-metrics-cert\") pod \"ovnkube-node-rkzqc\" (UID: \"17e134b4-4012-46f9-9824-0e3df9873d09\") " pod="openshift-ovn-kubernetes/ovnkube-node-rkzqc" Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.633202 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/17e134b4-4012-46f9-9824-0e3df9873d09-ovnkube-config\") pod \"ovnkube-node-rkzqc\" (UID: \"17e134b4-4012-46f9-9824-0e3df9873d09\") " pod="openshift-ovn-kubernetes/ovnkube-node-rkzqc" Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.633473 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/17e134b4-4012-46f9-9824-0e3df9873d09-run-openvswitch\") pod \"ovnkube-node-rkzqc\" (UID: \"17e134b4-4012-46f9-9824-0e3df9873d09\") " pod="openshift-ovn-kubernetes/ovnkube-node-rkzqc" Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.633604 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/17e134b4-4012-46f9-9824-0e3df9873d09-host-slash\") pod \"ovnkube-node-rkzqc\" (UID: \"17e134b4-4012-46f9-9824-0e3df9873d09\") " pod="openshift-ovn-kubernetes/ovnkube-node-rkzqc" Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.633674 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r5fmk\" (UniqueName: \"kubernetes.io/projected/17e134b4-4012-46f9-9824-0e3df9873d09-kube-api-access-r5fmk\") pod \"ovnkube-node-rkzqc\" (UID: \"17e134b4-4012-46f9-9824-0e3df9873d09\") " pod="openshift-ovn-kubernetes/ovnkube-node-rkzqc" Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.633720 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/17e134b4-4012-46f9-9824-0e3df9873d09-host-cni-netd\") pod \"ovnkube-node-rkzqc\" (UID: \"17e134b4-4012-46f9-9824-0e3df9873d09\") " pod="openshift-ovn-kubernetes/ovnkube-node-rkzqc" Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.633931 5052 reconciler_common.go:293] "Volume detached for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/bc569bba-c020-40d6-af68-e617625fe0e4-env-overrides\") on node \"crc\" DevicePath \"\"" Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.633967 5052 reconciler_common.go:293] "Volume detached for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/bc569bba-c020-40d6-af68-e617625fe0e4-ovnkube-config\") on node \"crc\" DevicePath \"\"" Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.633986 5052 reconciler_common.go:293] "Volume detached for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/bc569bba-c020-40d6-af68-e617625fe0e4-run-systemd\") on node \"crc\" DevicePath \"\"" Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.634006 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-x92h2\" (UniqueName: \"kubernetes.io/projected/bc569bba-c020-40d6-af68-e617625fe0e4-kube-api-access-x92h2\") on node \"crc\" DevicePath \"\"" Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.634030 5052 reconciler_common.go:293] "Volume detached for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/bc569bba-c020-40d6-af68-e617625fe0e4-ovn-node-metrics-cert\") on node \"crc\" DevicePath \"\"" Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.634053 5052 reconciler_common.go:293] "Volume detached for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/bc569bba-c020-40d6-af68-e617625fe0e4-ovnkube-script-lib\") on node \"crc\" DevicePath \"\"" Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.735467 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/17e134b4-4012-46f9-9824-0e3df9873d09-log-socket\") pod \"ovnkube-node-rkzqc\" (UID: \"17e134b4-4012-46f9-9824-0e3df9873d09\") " pod="openshift-ovn-kubernetes/ovnkube-node-rkzqc" Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.735582 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/17e134b4-4012-46f9-9824-0e3df9873d09-ovnkube-script-lib\") pod \"ovnkube-node-rkzqc\" (UID: \"17e134b4-4012-46f9-9824-0e3df9873d09\") " pod="openshift-ovn-kubernetes/ovnkube-node-rkzqc" Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.735610 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/17e134b4-4012-46f9-9824-0e3df9873d09-log-socket\") pod \"ovnkube-node-rkzqc\" (UID: \"17e134b4-4012-46f9-9824-0e3df9873d09\") " pod="openshift-ovn-kubernetes/ovnkube-node-rkzqc" Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.735643 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/17e134b4-4012-46f9-9824-0e3df9873d09-etc-openvswitch\") pod \"ovnkube-node-rkzqc\" (UID: \"17e134b4-4012-46f9-9824-0e3df9873d09\") " pod="openshift-ovn-kubernetes/ovnkube-node-rkzqc" Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.735615 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/17e134b4-4012-46f9-9824-0e3df9873d09-etc-openvswitch\") pod \"ovnkube-node-rkzqc\" (UID: \"17e134b4-4012-46f9-9824-0e3df9873d09\") " pod="openshift-ovn-kubernetes/ovnkube-node-rkzqc" Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.735690 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/17e134b4-4012-46f9-9824-0e3df9873d09-host-run-netns\") pod \"ovnkube-node-rkzqc\" (UID: \"17e134b4-4012-46f9-9824-0e3df9873d09\") " pod="openshift-ovn-kubernetes/ovnkube-node-rkzqc" Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.735783 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/17e134b4-4012-46f9-9824-0e3df9873d09-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-rkzqc\" (UID: \"17e134b4-4012-46f9-9824-0e3df9873d09\") " pod="openshift-ovn-kubernetes/ovnkube-node-rkzqc" Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.735813 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/17e134b4-4012-46f9-9824-0e3df9873d09-ovn-node-metrics-cert\") pod \"ovnkube-node-rkzqc\" (UID: \"17e134b4-4012-46f9-9824-0e3df9873d09\") " pod="openshift-ovn-kubernetes/ovnkube-node-rkzqc" Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.735834 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/17e134b4-4012-46f9-9824-0e3df9873d09-ovnkube-config\") pod \"ovnkube-node-rkzqc\" (UID: \"17e134b4-4012-46f9-9824-0e3df9873d09\") " pod="openshift-ovn-kubernetes/ovnkube-node-rkzqc" Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.735854 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/17e134b4-4012-46f9-9824-0e3df9873d09-run-openvswitch\") pod \"ovnkube-node-rkzqc\" (UID: \"17e134b4-4012-46f9-9824-0e3df9873d09\") " pod="openshift-ovn-kubernetes/ovnkube-node-rkzqc" Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.735871 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/17e134b4-4012-46f9-9824-0e3df9873d09-host-slash\") pod \"ovnkube-node-rkzqc\" (UID: \"17e134b4-4012-46f9-9824-0e3df9873d09\") " pod="openshift-ovn-kubernetes/ovnkube-node-rkzqc" Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.735879 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/17e134b4-4012-46f9-9824-0e3df9873d09-host-run-netns\") pod \"ovnkube-node-rkzqc\" (UID: \"17e134b4-4012-46f9-9824-0e3df9873d09\") " pod="openshift-ovn-kubernetes/ovnkube-node-rkzqc" Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.735892 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-r5fmk\" (UniqueName: \"kubernetes.io/projected/17e134b4-4012-46f9-9824-0e3df9873d09-kube-api-access-r5fmk\") pod \"ovnkube-node-rkzqc\" (UID: \"17e134b4-4012-46f9-9824-0e3df9873d09\") " pod="openshift-ovn-kubernetes/ovnkube-node-rkzqc" Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.735972 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/17e134b4-4012-46f9-9824-0e3df9873d09-host-cni-netd\") pod \"ovnkube-node-rkzqc\" (UID: \"17e134b4-4012-46f9-9824-0e3df9873d09\") " pod="openshift-ovn-kubernetes/ovnkube-node-rkzqc" Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.736014 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/17e134b4-4012-46f9-9824-0e3df9873d09-systemd-units\") pod \"ovnkube-node-rkzqc\" (UID: \"17e134b4-4012-46f9-9824-0e3df9873d09\") " pod="openshift-ovn-kubernetes/ovnkube-node-rkzqc" Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.736093 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/17e134b4-4012-46f9-9824-0e3df9873d09-host-run-ovn-kubernetes\") pod \"ovnkube-node-rkzqc\" (UID: \"17e134b4-4012-46f9-9824-0e3df9873d09\") " pod="openshift-ovn-kubernetes/ovnkube-node-rkzqc" Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.736126 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/17e134b4-4012-46f9-9824-0e3df9873d09-var-lib-openvswitch\") pod \"ovnkube-node-rkzqc\" (UID: \"17e134b4-4012-46f9-9824-0e3df9873d09\") " pod="openshift-ovn-kubernetes/ovnkube-node-rkzqc" Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.736149 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/17e134b4-4012-46f9-9824-0e3df9873d09-node-log\") pod \"ovnkube-node-rkzqc\" (UID: \"17e134b4-4012-46f9-9824-0e3df9873d09\") " pod="openshift-ovn-kubernetes/ovnkube-node-rkzqc" Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.736176 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/17e134b4-4012-46f9-9824-0e3df9873d09-env-overrides\") pod \"ovnkube-node-rkzqc\" (UID: \"17e134b4-4012-46f9-9824-0e3df9873d09\") " pod="openshift-ovn-kubernetes/ovnkube-node-rkzqc" Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.736201 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/17e134b4-4012-46f9-9824-0e3df9873d09-host-kubelet\") pod \"ovnkube-node-rkzqc\" (UID: \"17e134b4-4012-46f9-9824-0e3df9873d09\") " pod="openshift-ovn-kubernetes/ovnkube-node-rkzqc" Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.736225 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/17e134b4-4012-46f9-9824-0e3df9873d09-run-ovn\") pod \"ovnkube-node-rkzqc\" (UID: \"17e134b4-4012-46f9-9824-0e3df9873d09\") " pod="openshift-ovn-kubernetes/ovnkube-node-rkzqc" Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.736280 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/17e134b4-4012-46f9-9824-0e3df9873d09-run-systemd\") pod \"ovnkube-node-rkzqc\" (UID: \"17e134b4-4012-46f9-9824-0e3df9873d09\") " pod="openshift-ovn-kubernetes/ovnkube-node-rkzqc" Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.736305 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/17e134b4-4012-46f9-9824-0e3df9873d09-host-cni-bin\") pod \"ovnkube-node-rkzqc\" (UID: \"17e134b4-4012-46f9-9824-0e3df9873d09\") " pod="openshift-ovn-kubernetes/ovnkube-node-rkzqc" Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.736399 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/17e134b4-4012-46f9-9824-0e3df9873d09-host-cni-bin\") pod \"ovnkube-node-rkzqc\" (UID: \"17e134b4-4012-46f9-9824-0e3df9873d09\") " pod="openshift-ovn-kubernetes/ovnkube-node-rkzqc" Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.736430 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/17e134b4-4012-46f9-9824-0e3df9873d09-host-cni-netd\") pod \"ovnkube-node-rkzqc\" (UID: \"17e134b4-4012-46f9-9824-0e3df9873d09\") " pod="openshift-ovn-kubernetes/ovnkube-node-rkzqc" Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.736460 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/17e134b4-4012-46f9-9824-0e3df9873d09-systemd-units\") pod \"ovnkube-node-rkzqc\" (UID: \"17e134b4-4012-46f9-9824-0e3df9873d09\") " pod="openshift-ovn-kubernetes/ovnkube-node-rkzqc" Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.736488 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/17e134b4-4012-46f9-9824-0e3df9873d09-host-run-ovn-kubernetes\") pod \"ovnkube-node-rkzqc\" (UID: \"17e134b4-4012-46f9-9824-0e3df9873d09\") " pod="openshift-ovn-kubernetes/ovnkube-node-rkzqc" Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.736544 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/17e134b4-4012-46f9-9824-0e3df9873d09-var-lib-openvswitch\") pod \"ovnkube-node-rkzqc\" (UID: \"17e134b4-4012-46f9-9824-0e3df9873d09\") " pod="openshift-ovn-kubernetes/ovnkube-node-rkzqc" Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.736577 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/17e134b4-4012-46f9-9824-0e3df9873d09-node-log\") pod \"ovnkube-node-rkzqc\" (UID: \"17e134b4-4012-46f9-9824-0e3df9873d09\") " pod="openshift-ovn-kubernetes/ovnkube-node-rkzqc" Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.736636 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/17e134b4-4012-46f9-9824-0e3df9873d09-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-rkzqc\" (UID: \"17e134b4-4012-46f9-9824-0e3df9873d09\") " pod="openshift-ovn-kubernetes/ovnkube-node-rkzqc" Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.736672 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/17e134b4-4012-46f9-9824-0e3df9873d09-host-slash\") pod \"ovnkube-node-rkzqc\" (UID: \"17e134b4-4012-46f9-9824-0e3df9873d09\") " pod="openshift-ovn-kubernetes/ovnkube-node-rkzqc" Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.736686 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/17e134b4-4012-46f9-9824-0e3df9873d09-run-openvswitch\") pod \"ovnkube-node-rkzqc\" (UID: \"17e134b4-4012-46f9-9824-0e3df9873d09\") " pod="openshift-ovn-kubernetes/ovnkube-node-rkzqc" Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.736698 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/17e134b4-4012-46f9-9824-0e3df9873d09-run-ovn\") pod \"ovnkube-node-rkzqc\" (UID: \"17e134b4-4012-46f9-9824-0e3df9873d09\") " pod="openshift-ovn-kubernetes/ovnkube-node-rkzqc" Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.736726 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/17e134b4-4012-46f9-9824-0e3df9873d09-run-systemd\") pod \"ovnkube-node-rkzqc\" (UID: \"17e134b4-4012-46f9-9824-0e3df9873d09\") " pod="openshift-ovn-kubernetes/ovnkube-node-rkzqc" Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.736789 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/17e134b4-4012-46f9-9824-0e3df9873d09-host-kubelet\") pod \"ovnkube-node-rkzqc\" (UID: \"17e134b4-4012-46f9-9824-0e3df9873d09\") " pod="openshift-ovn-kubernetes/ovnkube-node-rkzqc" Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.738387 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/17e134b4-4012-46f9-9824-0e3df9873d09-ovnkube-script-lib\") pod \"ovnkube-node-rkzqc\" (UID: \"17e134b4-4012-46f9-9824-0e3df9873d09\") " pod="openshift-ovn-kubernetes/ovnkube-node-rkzqc" Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.738535 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/17e134b4-4012-46f9-9824-0e3df9873d09-env-overrides\") pod \"ovnkube-node-rkzqc\" (UID: \"17e134b4-4012-46f9-9824-0e3df9873d09\") " pod="openshift-ovn-kubernetes/ovnkube-node-rkzqc" Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.738818 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/17e134b4-4012-46f9-9824-0e3df9873d09-ovnkube-config\") pod \"ovnkube-node-rkzqc\" (UID: \"17e134b4-4012-46f9-9824-0e3df9873d09\") " pod="openshift-ovn-kubernetes/ovnkube-node-rkzqc" Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.742089 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/17e134b4-4012-46f9-9824-0e3df9873d09-ovn-node-metrics-cert\") pod \"ovnkube-node-rkzqc\" (UID: \"17e134b4-4012-46f9-9824-0e3df9873d09\") " pod="openshift-ovn-kubernetes/ovnkube-node-rkzqc" Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.755945 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-r5fmk\" (UniqueName: \"kubernetes.io/projected/17e134b4-4012-46f9-9824-0e3df9873d09-kube-api-access-r5fmk\") pod \"ovnkube-node-rkzqc\" (UID: \"17e134b4-4012-46f9-9824-0e3df9873d09\") " pod="openshift-ovn-kubernetes/ovnkube-node-rkzqc" Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.783020 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-rkzqc" Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.917254 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-hx657_c30c02db-f4e7-4432-941c-5448fd71f3d9/kube-multus/0.log" Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.917584 5052 generic.go:334] "Generic (PLEG): container finished" podID="c30c02db-f4e7-4432-941c-5448fd71f3d9" containerID="388fefa2a0e0ba8ca09e3fd39371e442358e9547cc2789ca8fa4b7b7237f6ac5" exitCode=2 Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.917689 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-hx657" event={"ID":"c30c02db-f4e7-4432-941c-5448fd71f3d9","Type":"ContainerDied","Data":"388fefa2a0e0ba8ca09e3fd39371e442358e9547cc2789ca8fa4b7b7237f6ac5"} Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.918597 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rkzqc" event={"ID":"17e134b4-4012-46f9-9824-0e3df9873d09","Type":"ContainerStarted","Data":"bcb396f15544560431fcedf5fbf86db2dbdd4f36a34696923be914d860d45b1a"} Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.919651 5052 scope.go:117] "RemoveContainer" containerID="388fefa2a0e0ba8ca09e3fd39371e442358e9547cc2789ca8fa4b7b7237f6ac5" Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.921980 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kg55h_bc569bba-c020-40d6-af68-e617625fe0e4/ovn-acl-logging/0.log" Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.923205 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kg55h_bc569bba-c020-40d6-af68-e617625fe0e4/ovn-controller/0.log" Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.924884 5052 generic.go:334] "Generic (PLEG): container finished" podID="bc569bba-c020-40d6-af68-e617625fe0e4" containerID="7548847d4c67dfe4acdf79ad3da86e54d4766ed7657a83a725c1ecc4a22c44dd" exitCode=0 Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.924915 5052 generic.go:334] "Generic (PLEG): container finished" podID="bc569bba-c020-40d6-af68-e617625fe0e4" containerID="affc8d8f111572fbded3dbcc4c9f374ca5ba664cd6989bb1e717c163a1d88417" exitCode=0 Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.924928 5052 generic.go:334] "Generic (PLEG): container finished" podID="bc569bba-c020-40d6-af68-e617625fe0e4" containerID="c16097e9c17ec2d194aef1c4af913073743ff9b3e54b20999b8b48ce26176ecb" exitCode=0 Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.924941 5052 generic.go:334] "Generic (PLEG): container finished" podID="bc569bba-c020-40d6-af68-e617625fe0e4" containerID="5b3ab2b483c488f2669acaf42538650e9c65cd6da9b9a5296572ed8f4702a685" exitCode=0 Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.924951 5052 generic.go:334] "Generic (PLEG): container finished" podID="bc569bba-c020-40d6-af68-e617625fe0e4" containerID="9201f740cc3848eee706f3415f90bbffac06b7df3bb60f3495eff3c1d0a7f3a8" exitCode=0 Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.924960 5052 generic.go:334] "Generic (PLEG): container finished" podID="bc569bba-c020-40d6-af68-e617625fe0e4" containerID="bde78769f16ee1d0101355c5c8eb4a48437c04ccc42ca5104d1b98538b2545be" exitCode=0 Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.924968 5052 generic.go:334] "Generic (PLEG): container finished" podID="bc569bba-c020-40d6-af68-e617625fe0e4" containerID="896c25e34b1cb8871fd92cc6c8c908bfce14a9713345f571a672b2156a1765bf" exitCode=143 Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.924973 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-kg55h" Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.924979 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kg55h" event={"ID":"bc569bba-c020-40d6-af68-e617625fe0e4","Type":"ContainerDied","Data":"7548847d4c67dfe4acdf79ad3da86e54d4766ed7657a83a725c1ecc4a22c44dd"} Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.925013 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kg55h" event={"ID":"bc569bba-c020-40d6-af68-e617625fe0e4","Type":"ContainerDied","Data":"affc8d8f111572fbded3dbcc4c9f374ca5ba664cd6989bb1e717c163a1d88417"} Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.925046 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kg55h" event={"ID":"bc569bba-c020-40d6-af68-e617625fe0e4","Type":"ContainerDied","Data":"c16097e9c17ec2d194aef1c4af913073743ff9b3e54b20999b8b48ce26176ecb"} Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.925059 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kg55h" event={"ID":"bc569bba-c020-40d6-af68-e617625fe0e4","Type":"ContainerDied","Data":"5b3ab2b483c488f2669acaf42538650e9c65cd6da9b9a5296572ed8f4702a685"} Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.925069 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kg55h" event={"ID":"bc569bba-c020-40d6-af68-e617625fe0e4","Type":"ContainerDied","Data":"9201f740cc3848eee706f3415f90bbffac06b7df3bb60f3495eff3c1d0a7f3a8"} Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.924978 5052 generic.go:334] "Generic (PLEG): container finished" podID="bc569bba-c020-40d6-af68-e617625fe0e4" containerID="2363c55c03e9159893688a52c5ba318a997f018c828f223626074a58918ccc24" exitCode=143 Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.925086 5052 scope.go:117] "RemoveContainer" containerID="7548847d4c67dfe4acdf79ad3da86e54d4766ed7657a83a725c1ecc4a22c44dd" Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.925077 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kg55h" event={"ID":"bc569bba-c020-40d6-af68-e617625fe0e4","Type":"ContainerDied","Data":"bde78769f16ee1d0101355c5c8eb4a48437c04ccc42ca5104d1b98538b2545be"} Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.925120 5052 pod_container_deletor.go:114] "Failed to issue the request to remove container" containerID={"Type":"cri-o","ID":"896c25e34b1cb8871fd92cc6c8c908bfce14a9713345f571a672b2156a1765bf"} Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.925136 5052 pod_container_deletor.go:114] "Failed to issue the request to remove container" containerID={"Type":"cri-o","ID":"2363c55c03e9159893688a52c5ba318a997f018c828f223626074a58918ccc24"} Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.925147 5052 pod_container_deletor.go:114] "Failed to issue the request to remove container" containerID={"Type":"cri-o","ID":"4def7c1172d2ae52e275b2371cf88c9f93c3f0c747c18de7d1073d50b03b19bc"} Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.925162 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kg55h" event={"ID":"bc569bba-c020-40d6-af68-e617625fe0e4","Type":"ContainerDied","Data":"896c25e34b1cb8871fd92cc6c8c908bfce14a9713345f571a672b2156a1765bf"} Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.925177 5052 pod_container_deletor.go:114] "Failed to issue the request to remove container" containerID={"Type":"cri-o","ID":"7548847d4c67dfe4acdf79ad3da86e54d4766ed7657a83a725c1ecc4a22c44dd"} Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.925189 5052 pod_container_deletor.go:114] "Failed to issue the request to remove container" containerID={"Type":"cri-o","ID":"affc8d8f111572fbded3dbcc4c9f374ca5ba664cd6989bb1e717c163a1d88417"} Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.925196 5052 pod_container_deletor.go:114] "Failed to issue the request to remove container" containerID={"Type":"cri-o","ID":"c16097e9c17ec2d194aef1c4af913073743ff9b3e54b20999b8b48ce26176ecb"} Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.925205 5052 pod_container_deletor.go:114] "Failed to issue the request to remove container" containerID={"Type":"cri-o","ID":"5b3ab2b483c488f2669acaf42538650e9c65cd6da9b9a5296572ed8f4702a685"} Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.925213 5052 pod_container_deletor.go:114] "Failed to issue the request to remove container" containerID={"Type":"cri-o","ID":"9201f740cc3848eee706f3415f90bbffac06b7df3bb60f3495eff3c1d0a7f3a8"} Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.925222 5052 pod_container_deletor.go:114] "Failed to issue the request to remove container" containerID={"Type":"cri-o","ID":"bde78769f16ee1d0101355c5c8eb4a48437c04ccc42ca5104d1b98538b2545be"} Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.925229 5052 pod_container_deletor.go:114] "Failed to issue the request to remove container" containerID={"Type":"cri-o","ID":"896c25e34b1cb8871fd92cc6c8c908bfce14a9713345f571a672b2156a1765bf"} Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.925236 5052 pod_container_deletor.go:114] "Failed to issue the request to remove container" containerID={"Type":"cri-o","ID":"2363c55c03e9159893688a52c5ba318a997f018c828f223626074a58918ccc24"} Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.925243 5052 pod_container_deletor.go:114] "Failed to issue the request to remove container" containerID={"Type":"cri-o","ID":"4def7c1172d2ae52e275b2371cf88c9f93c3f0c747c18de7d1073d50b03b19bc"} Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.925252 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kg55h" event={"ID":"bc569bba-c020-40d6-af68-e617625fe0e4","Type":"ContainerDied","Data":"2363c55c03e9159893688a52c5ba318a997f018c828f223626074a58918ccc24"} Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.925263 5052 pod_container_deletor.go:114] "Failed to issue the request to remove container" containerID={"Type":"cri-o","ID":"7548847d4c67dfe4acdf79ad3da86e54d4766ed7657a83a725c1ecc4a22c44dd"} Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.925270 5052 pod_container_deletor.go:114] "Failed to issue the request to remove container" containerID={"Type":"cri-o","ID":"affc8d8f111572fbded3dbcc4c9f374ca5ba664cd6989bb1e717c163a1d88417"} Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.925278 5052 pod_container_deletor.go:114] "Failed to issue the request to remove container" containerID={"Type":"cri-o","ID":"c16097e9c17ec2d194aef1c4af913073743ff9b3e54b20999b8b48ce26176ecb"} Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.925285 5052 pod_container_deletor.go:114] "Failed to issue the request to remove container" containerID={"Type":"cri-o","ID":"5b3ab2b483c488f2669acaf42538650e9c65cd6da9b9a5296572ed8f4702a685"} Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.925292 5052 pod_container_deletor.go:114] "Failed to issue the request to remove container" containerID={"Type":"cri-o","ID":"9201f740cc3848eee706f3415f90bbffac06b7df3bb60f3495eff3c1d0a7f3a8"} Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.925299 5052 pod_container_deletor.go:114] "Failed to issue the request to remove container" containerID={"Type":"cri-o","ID":"bde78769f16ee1d0101355c5c8eb4a48437c04ccc42ca5104d1b98538b2545be"} Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.925306 5052 pod_container_deletor.go:114] "Failed to issue the request to remove container" containerID={"Type":"cri-o","ID":"896c25e34b1cb8871fd92cc6c8c908bfce14a9713345f571a672b2156a1765bf"} Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.925313 5052 pod_container_deletor.go:114] "Failed to issue the request to remove container" containerID={"Type":"cri-o","ID":"2363c55c03e9159893688a52c5ba318a997f018c828f223626074a58918ccc24"} Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.925320 5052 pod_container_deletor.go:114] "Failed to issue the request to remove container" containerID={"Type":"cri-o","ID":"4def7c1172d2ae52e275b2371cf88c9f93c3f0c747c18de7d1073d50b03b19bc"} Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.925329 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kg55h" event={"ID":"bc569bba-c020-40d6-af68-e617625fe0e4","Type":"ContainerDied","Data":"08118e254f2650de17c8e49017cae4a774b2c748d9cd9b8b2993b973913c0567"} Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.925340 5052 pod_container_deletor.go:114] "Failed to issue the request to remove container" containerID={"Type":"cri-o","ID":"7548847d4c67dfe4acdf79ad3da86e54d4766ed7657a83a725c1ecc4a22c44dd"} Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.925348 5052 pod_container_deletor.go:114] "Failed to issue the request to remove container" containerID={"Type":"cri-o","ID":"affc8d8f111572fbded3dbcc4c9f374ca5ba664cd6989bb1e717c163a1d88417"} Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.925354 5052 pod_container_deletor.go:114] "Failed to issue the request to remove container" containerID={"Type":"cri-o","ID":"c16097e9c17ec2d194aef1c4af913073743ff9b3e54b20999b8b48ce26176ecb"} Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.925361 5052 pod_container_deletor.go:114] "Failed to issue the request to remove container" containerID={"Type":"cri-o","ID":"5b3ab2b483c488f2669acaf42538650e9c65cd6da9b9a5296572ed8f4702a685"} Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.925367 5052 pod_container_deletor.go:114] "Failed to issue the request to remove container" containerID={"Type":"cri-o","ID":"9201f740cc3848eee706f3415f90bbffac06b7df3bb60f3495eff3c1d0a7f3a8"} Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.925374 5052 pod_container_deletor.go:114] "Failed to issue the request to remove container" containerID={"Type":"cri-o","ID":"bde78769f16ee1d0101355c5c8eb4a48437c04ccc42ca5104d1b98538b2545be"} Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.925381 5052 pod_container_deletor.go:114] "Failed to issue the request to remove container" containerID={"Type":"cri-o","ID":"896c25e34b1cb8871fd92cc6c8c908bfce14a9713345f571a672b2156a1765bf"} Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.925387 5052 pod_container_deletor.go:114] "Failed to issue the request to remove container" containerID={"Type":"cri-o","ID":"2363c55c03e9159893688a52c5ba318a997f018c828f223626074a58918ccc24"} Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.925394 5052 pod_container_deletor.go:114] "Failed to issue the request to remove container" containerID={"Type":"cri-o","ID":"4def7c1172d2ae52e275b2371cf88c9f93c3f0c747c18de7d1073d50b03b19bc"} Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.947678 5052 scope.go:117] "RemoveContainer" containerID="affc8d8f111572fbded3dbcc4c9f374ca5ba664cd6989bb1e717c163a1d88417" Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.975735 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openshift-ovn-kubernetes/ovnkube-node-kg55h"] Dec 03 11:31:37 crc kubenswrapper[5052]: I1203 11:31:37.977749 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openshift-ovn-kubernetes/ovnkube-node-kg55h"] Dec 03 11:31:38 crc kubenswrapper[5052]: I1203 11:31:38.026731 5052 scope.go:117] "RemoveContainer" containerID="c16097e9c17ec2d194aef1c4af913073743ff9b3e54b20999b8b48ce26176ecb" Dec 03 11:31:38 crc kubenswrapper[5052]: I1203 11:31:38.059557 5052 scope.go:117] "RemoveContainer" containerID="5b3ab2b483c488f2669acaf42538650e9c65cd6da9b9a5296572ed8f4702a685" Dec 03 11:31:38 crc kubenswrapper[5052]: I1203 11:31:38.075605 5052 scope.go:117] "RemoveContainer" containerID="9201f740cc3848eee706f3415f90bbffac06b7df3bb60f3495eff3c1d0a7f3a8" Dec 03 11:31:38 crc kubenswrapper[5052]: I1203 11:31:38.091668 5052 scope.go:117] "RemoveContainer" containerID="bde78769f16ee1d0101355c5c8eb4a48437c04ccc42ca5104d1b98538b2545be" Dec 03 11:31:38 crc kubenswrapper[5052]: I1203 11:31:38.108608 5052 scope.go:117] "RemoveContainer" containerID="896c25e34b1cb8871fd92cc6c8c908bfce14a9713345f571a672b2156a1765bf" Dec 03 11:31:38 crc kubenswrapper[5052]: I1203 11:31:38.122584 5052 scope.go:117] "RemoveContainer" containerID="2363c55c03e9159893688a52c5ba318a997f018c828f223626074a58918ccc24" Dec 03 11:31:38 crc kubenswrapper[5052]: I1203 11:31:38.141612 5052 scope.go:117] "RemoveContainer" containerID="4def7c1172d2ae52e275b2371cf88c9f93c3f0c747c18de7d1073d50b03b19bc" Dec 03 11:31:38 crc kubenswrapper[5052]: I1203 11:31:38.155834 5052 scope.go:117] "RemoveContainer" containerID="7548847d4c67dfe4acdf79ad3da86e54d4766ed7657a83a725c1ecc4a22c44dd" Dec 03 11:31:38 crc kubenswrapper[5052]: E1203 11:31:38.156347 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7548847d4c67dfe4acdf79ad3da86e54d4766ed7657a83a725c1ecc4a22c44dd\": container with ID starting with 7548847d4c67dfe4acdf79ad3da86e54d4766ed7657a83a725c1ecc4a22c44dd not found: ID does not exist" containerID="7548847d4c67dfe4acdf79ad3da86e54d4766ed7657a83a725c1ecc4a22c44dd" Dec 03 11:31:38 crc kubenswrapper[5052]: I1203 11:31:38.156423 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7548847d4c67dfe4acdf79ad3da86e54d4766ed7657a83a725c1ecc4a22c44dd"} err="failed to get container status \"7548847d4c67dfe4acdf79ad3da86e54d4766ed7657a83a725c1ecc4a22c44dd\": rpc error: code = NotFound desc = could not find container \"7548847d4c67dfe4acdf79ad3da86e54d4766ed7657a83a725c1ecc4a22c44dd\": container with ID starting with 7548847d4c67dfe4acdf79ad3da86e54d4766ed7657a83a725c1ecc4a22c44dd not found: ID does not exist" Dec 03 11:31:38 crc kubenswrapper[5052]: I1203 11:31:38.156480 5052 scope.go:117] "RemoveContainer" containerID="affc8d8f111572fbded3dbcc4c9f374ca5ba664cd6989bb1e717c163a1d88417" Dec 03 11:31:38 crc kubenswrapper[5052]: E1203 11:31:38.156896 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"affc8d8f111572fbded3dbcc4c9f374ca5ba664cd6989bb1e717c163a1d88417\": container with ID starting with affc8d8f111572fbded3dbcc4c9f374ca5ba664cd6989bb1e717c163a1d88417 not found: ID does not exist" containerID="affc8d8f111572fbded3dbcc4c9f374ca5ba664cd6989bb1e717c163a1d88417" Dec 03 11:31:38 crc kubenswrapper[5052]: I1203 11:31:38.156926 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"affc8d8f111572fbded3dbcc4c9f374ca5ba664cd6989bb1e717c163a1d88417"} err="failed to get container status \"affc8d8f111572fbded3dbcc4c9f374ca5ba664cd6989bb1e717c163a1d88417\": rpc error: code = NotFound desc = could not find container \"affc8d8f111572fbded3dbcc4c9f374ca5ba664cd6989bb1e717c163a1d88417\": container with ID starting with affc8d8f111572fbded3dbcc4c9f374ca5ba664cd6989bb1e717c163a1d88417 not found: ID does not exist" Dec 03 11:31:38 crc kubenswrapper[5052]: I1203 11:31:38.156952 5052 scope.go:117] "RemoveContainer" containerID="c16097e9c17ec2d194aef1c4af913073743ff9b3e54b20999b8b48ce26176ecb" Dec 03 11:31:38 crc kubenswrapper[5052]: E1203 11:31:38.157190 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c16097e9c17ec2d194aef1c4af913073743ff9b3e54b20999b8b48ce26176ecb\": container with ID starting with c16097e9c17ec2d194aef1c4af913073743ff9b3e54b20999b8b48ce26176ecb not found: ID does not exist" containerID="c16097e9c17ec2d194aef1c4af913073743ff9b3e54b20999b8b48ce26176ecb" Dec 03 11:31:38 crc kubenswrapper[5052]: I1203 11:31:38.157208 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c16097e9c17ec2d194aef1c4af913073743ff9b3e54b20999b8b48ce26176ecb"} err="failed to get container status \"c16097e9c17ec2d194aef1c4af913073743ff9b3e54b20999b8b48ce26176ecb\": rpc error: code = NotFound desc = could not find container \"c16097e9c17ec2d194aef1c4af913073743ff9b3e54b20999b8b48ce26176ecb\": container with ID starting with c16097e9c17ec2d194aef1c4af913073743ff9b3e54b20999b8b48ce26176ecb not found: ID does not exist" Dec 03 11:31:38 crc kubenswrapper[5052]: I1203 11:31:38.157220 5052 scope.go:117] "RemoveContainer" containerID="5b3ab2b483c488f2669acaf42538650e9c65cd6da9b9a5296572ed8f4702a685" Dec 03 11:31:38 crc kubenswrapper[5052]: E1203 11:31:38.157659 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5b3ab2b483c488f2669acaf42538650e9c65cd6da9b9a5296572ed8f4702a685\": container with ID starting with 5b3ab2b483c488f2669acaf42538650e9c65cd6da9b9a5296572ed8f4702a685 not found: ID does not exist" containerID="5b3ab2b483c488f2669acaf42538650e9c65cd6da9b9a5296572ed8f4702a685" Dec 03 11:31:38 crc kubenswrapper[5052]: I1203 11:31:38.157703 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5b3ab2b483c488f2669acaf42538650e9c65cd6da9b9a5296572ed8f4702a685"} err="failed to get container status \"5b3ab2b483c488f2669acaf42538650e9c65cd6da9b9a5296572ed8f4702a685\": rpc error: code = NotFound desc = could not find container \"5b3ab2b483c488f2669acaf42538650e9c65cd6da9b9a5296572ed8f4702a685\": container with ID starting with 5b3ab2b483c488f2669acaf42538650e9c65cd6da9b9a5296572ed8f4702a685 not found: ID does not exist" Dec 03 11:31:38 crc kubenswrapper[5052]: I1203 11:31:38.157745 5052 scope.go:117] "RemoveContainer" containerID="9201f740cc3848eee706f3415f90bbffac06b7df3bb60f3495eff3c1d0a7f3a8" Dec 03 11:31:38 crc kubenswrapper[5052]: E1203 11:31:38.158128 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9201f740cc3848eee706f3415f90bbffac06b7df3bb60f3495eff3c1d0a7f3a8\": container with ID starting with 9201f740cc3848eee706f3415f90bbffac06b7df3bb60f3495eff3c1d0a7f3a8 not found: ID does not exist" containerID="9201f740cc3848eee706f3415f90bbffac06b7df3bb60f3495eff3c1d0a7f3a8" Dec 03 11:31:38 crc kubenswrapper[5052]: I1203 11:31:38.158155 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9201f740cc3848eee706f3415f90bbffac06b7df3bb60f3495eff3c1d0a7f3a8"} err="failed to get container status \"9201f740cc3848eee706f3415f90bbffac06b7df3bb60f3495eff3c1d0a7f3a8\": rpc error: code = NotFound desc = could not find container \"9201f740cc3848eee706f3415f90bbffac06b7df3bb60f3495eff3c1d0a7f3a8\": container with ID starting with 9201f740cc3848eee706f3415f90bbffac06b7df3bb60f3495eff3c1d0a7f3a8 not found: ID does not exist" Dec 03 11:31:38 crc kubenswrapper[5052]: I1203 11:31:38.158174 5052 scope.go:117] "RemoveContainer" containerID="bde78769f16ee1d0101355c5c8eb4a48437c04ccc42ca5104d1b98538b2545be" Dec 03 11:31:38 crc kubenswrapper[5052]: E1203 11:31:38.158378 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bde78769f16ee1d0101355c5c8eb4a48437c04ccc42ca5104d1b98538b2545be\": container with ID starting with bde78769f16ee1d0101355c5c8eb4a48437c04ccc42ca5104d1b98538b2545be not found: ID does not exist" containerID="bde78769f16ee1d0101355c5c8eb4a48437c04ccc42ca5104d1b98538b2545be" Dec 03 11:31:38 crc kubenswrapper[5052]: I1203 11:31:38.158433 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bde78769f16ee1d0101355c5c8eb4a48437c04ccc42ca5104d1b98538b2545be"} err="failed to get container status \"bde78769f16ee1d0101355c5c8eb4a48437c04ccc42ca5104d1b98538b2545be\": rpc error: code = NotFound desc = could not find container \"bde78769f16ee1d0101355c5c8eb4a48437c04ccc42ca5104d1b98538b2545be\": container with ID starting with bde78769f16ee1d0101355c5c8eb4a48437c04ccc42ca5104d1b98538b2545be not found: ID does not exist" Dec 03 11:31:38 crc kubenswrapper[5052]: I1203 11:31:38.158455 5052 scope.go:117] "RemoveContainer" containerID="896c25e34b1cb8871fd92cc6c8c908bfce14a9713345f571a672b2156a1765bf" Dec 03 11:31:38 crc kubenswrapper[5052]: E1203 11:31:38.158725 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"896c25e34b1cb8871fd92cc6c8c908bfce14a9713345f571a672b2156a1765bf\": container with ID starting with 896c25e34b1cb8871fd92cc6c8c908bfce14a9713345f571a672b2156a1765bf not found: ID does not exist" containerID="896c25e34b1cb8871fd92cc6c8c908bfce14a9713345f571a672b2156a1765bf" Dec 03 11:31:38 crc kubenswrapper[5052]: I1203 11:31:38.158750 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"896c25e34b1cb8871fd92cc6c8c908bfce14a9713345f571a672b2156a1765bf"} err="failed to get container status \"896c25e34b1cb8871fd92cc6c8c908bfce14a9713345f571a672b2156a1765bf\": rpc error: code = NotFound desc = could not find container \"896c25e34b1cb8871fd92cc6c8c908bfce14a9713345f571a672b2156a1765bf\": container with ID starting with 896c25e34b1cb8871fd92cc6c8c908bfce14a9713345f571a672b2156a1765bf not found: ID does not exist" Dec 03 11:31:38 crc kubenswrapper[5052]: I1203 11:31:38.158764 5052 scope.go:117] "RemoveContainer" containerID="2363c55c03e9159893688a52c5ba318a997f018c828f223626074a58918ccc24" Dec 03 11:31:38 crc kubenswrapper[5052]: E1203 11:31:38.158987 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2363c55c03e9159893688a52c5ba318a997f018c828f223626074a58918ccc24\": container with ID starting with 2363c55c03e9159893688a52c5ba318a997f018c828f223626074a58918ccc24 not found: ID does not exist" containerID="2363c55c03e9159893688a52c5ba318a997f018c828f223626074a58918ccc24" Dec 03 11:31:38 crc kubenswrapper[5052]: I1203 11:31:38.159009 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2363c55c03e9159893688a52c5ba318a997f018c828f223626074a58918ccc24"} err="failed to get container status \"2363c55c03e9159893688a52c5ba318a997f018c828f223626074a58918ccc24\": rpc error: code = NotFound desc = could not find container \"2363c55c03e9159893688a52c5ba318a997f018c828f223626074a58918ccc24\": container with ID starting with 2363c55c03e9159893688a52c5ba318a997f018c828f223626074a58918ccc24 not found: ID does not exist" Dec 03 11:31:38 crc kubenswrapper[5052]: I1203 11:31:38.159054 5052 scope.go:117] "RemoveContainer" containerID="4def7c1172d2ae52e275b2371cf88c9f93c3f0c747c18de7d1073d50b03b19bc" Dec 03 11:31:38 crc kubenswrapper[5052]: E1203 11:31:38.159396 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4def7c1172d2ae52e275b2371cf88c9f93c3f0c747c18de7d1073d50b03b19bc\": container with ID starting with 4def7c1172d2ae52e275b2371cf88c9f93c3f0c747c18de7d1073d50b03b19bc not found: ID does not exist" containerID="4def7c1172d2ae52e275b2371cf88c9f93c3f0c747c18de7d1073d50b03b19bc" Dec 03 11:31:38 crc kubenswrapper[5052]: I1203 11:31:38.159417 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4def7c1172d2ae52e275b2371cf88c9f93c3f0c747c18de7d1073d50b03b19bc"} err="failed to get container status \"4def7c1172d2ae52e275b2371cf88c9f93c3f0c747c18de7d1073d50b03b19bc\": rpc error: code = NotFound desc = could not find container \"4def7c1172d2ae52e275b2371cf88c9f93c3f0c747c18de7d1073d50b03b19bc\": container with ID starting with 4def7c1172d2ae52e275b2371cf88c9f93c3f0c747c18de7d1073d50b03b19bc not found: ID does not exist" Dec 03 11:31:38 crc kubenswrapper[5052]: I1203 11:31:38.159429 5052 scope.go:117] "RemoveContainer" containerID="7548847d4c67dfe4acdf79ad3da86e54d4766ed7657a83a725c1ecc4a22c44dd" Dec 03 11:31:38 crc kubenswrapper[5052]: I1203 11:31:38.159659 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7548847d4c67dfe4acdf79ad3da86e54d4766ed7657a83a725c1ecc4a22c44dd"} err="failed to get container status \"7548847d4c67dfe4acdf79ad3da86e54d4766ed7657a83a725c1ecc4a22c44dd\": rpc error: code = NotFound desc = could not find container \"7548847d4c67dfe4acdf79ad3da86e54d4766ed7657a83a725c1ecc4a22c44dd\": container with ID starting with 7548847d4c67dfe4acdf79ad3da86e54d4766ed7657a83a725c1ecc4a22c44dd not found: ID does not exist" Dec 03 11:31:38 crc kubenswrapper[5052]: I1203 11:31:38.159701 5052 scope.go:117] "RemoveContainer" containerID="affc8d8f111572fbded3dbcc4c9f374ca5ba664cd6989bb1e717c163a1d88417" Dec 03 11:31:38 crc kubenswrapper[5052]: I1203 11:31:38.159935 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"affc8d8f111572fbded3dbcc4c9f374ca5ba664cd6989bb1e717c163a1d88417"} err="failed to get container status \"affc8d8f111572fbded3dbcc4c9f374ca5ba664cd6989bb1e717c163a1d88417\": rpc error: code = NotFound desc = could not find container \"affc8d8f111572fbded3dbcc4c9f374ca5ba664cd6989bb1e717c163a1d88417\": container with ID starting with affc8d8f111572fbded3dbcc4c9f374ca5ba664cd6989bb1e717c163a1d88417 not found: ID does not exist" Dec 03 11:31:38 crc kubenswrapper[5052]: I1203 11:31:38.159951 5052 scope.go:117] "RemoveContainer" containerID="c16097e9c17ec2d194aef1c4af913073743ff9b3e54b20999b8b48ce26176ecb" Dec 03 11:31:38 crc kubenswrapper[5052]: I1203 11:31:38.160173 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c16097e9c17ec2d194aef1c4af913073743ff9b3e54b20999b8b48ce26176ecb"} err="failed to get container status \"c16097e9c17ec2d194aef1c4af913073743ff9b3e54b20999b8b48ce26176ecb\": rpc error: code = NotFound desc = could not find container \"c16097e9c17ec2d194aef1c4af913073743ff9b3e54b20999b8b48ce26176ecb\": container with ID starting with c16097e9c17ec2d194aef1c4af913073743ff9b3e54b20999b8b48ce26176ecb not found: ID does not exist" Dec 03 11:31:38 crc kubenswrapper[5052]: I1203 11:31:38.160192 5052 scope.go:117] "RemoveContainer" containerID="5b3ab2b483c488f2669acaf42538650e9c65cd6da9b9a5296572ed8f4702a685" Dec 03 11:31:38 crc kubenswrapper[5052]: I1203 11:31:38.160401 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5b3ab2b483c488f2669acaf42538650e9c65cd6da9b9a5296572ed8f4702a685"} err="failed to get container status \"5b3ab2b483c488f2669acaf42538650e9c65cd6da9b9a5296572ed8f4702a685\": rpc error: code = NotFound desc = could not find container \"5b3ab2b483c488f2669acaf42538650e9c65cd6da9b9a5296572ed8f4702a685\": container with ID starting with 5b3ab2b483c488f2669acaf42538650e9c65cd6da9b9a5296572ed8f4702a685 not found: ID does not exist" Dec 03 11:31:38 crc kubenswrapper[5052]: I1203 11:31:38.160421 5052 scope.go:117] "RemoveContainer" containerID="9201f740cc3848eee706f3415f90bbffac06b7df3bb60f3495eff3c1d0a7f3a8" Dec 03 11:31:38 crc kubenswrapper[5052]: I1203 11:31:38.160857 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9201f740cc3848eee706f3415f90bbffac06b7df3bb60f3495eff3c1d0a7f3a8"} err="failed to get container status \"9201f740cc3848eee706f3415f90bbffac06b7df3bb60f3495eff3c1d0a7f3a8\": rpc error: code = NotFound desc = could not find container \"9201f740cc3848eee706f3415f90bbffac06b7df3bb60f3495eff3c1d0a7f3a8\": container with ID starting with 9201f740cc3848eee706f3415f90bbffac06b7df3bb60f3495eff3c1d0a7f3a8 not found: ID does not exist" Dec 03 11:31:38 crc kubenswrapper[5052]: I1203 11:31:38.160882 5052 scope.go:117] "RemoveContainer" containerID="bde78769f16ee1d0101355c5c8eb4a48437c04ccc42ca5104d1b98538b2545be" Dec 03 11:31:38 crc kubenswrapper[5052]: I1203 11:31:38.161161 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bde78769f16ee1d0101355c5c8eb4a48437c04ccc42ca5104d1b98538b2545be"} err="failed to get container status \"bde78769f16ee1d0101355c5c8eb4a48437c04ccc42ca5104d1b98538b2545be\": rpc error: code = NotFound desc = could not find container \"bde78769f16ee1d0101355c5c8eb4a48437c04ccc42ca5104d1b98538b2545be\": container with ID starting with bde78769f16ee1d0101355c5c8eb4a48437c04ccc42ca5104d1b98538b2545be not found: ID does not exist" Dec 03 11:31:38 crc kubenswrapper[5052]: I1203 11:31:38.161182 5052 scope.go:117] "RemoveContainer" containerID="896c25e34b1cb8871fd92cc6c8c908bfce14a9713345f571a672b2156a1765bf" Dec 03 11:31:38 crc kubenswrapper[5052]: I1203 11:31:38.161482 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"896c25e34b1cb8871fd92cc6c8c908bfce14a9713345f571a672b2156a1765bf"} err="failed to get container status \"896c25e34b1cb8871fd92cc6c8c908bfce14a9713345f571a672b2156a1765bf\": rpc error: code = NotFound desc = could not find container \"896c25e34b1cb8871fd92cc6c8c908bfce14a9713345f571a672b2156a1765bf\": container with ID starting with 896c25e34b1cb8871fd92cc6c8c908bfce14a9713345f571a672b2156a1765bf not found: ID does not exist" Dec 03 11:31:38 crc kubenswrapper[5052]: I1203 11:31:38.161526 5052 scope.go:117] "RemoveContainer" containerID="2363c55c03e9159893688a52c5ba318a997f018c828f223626074a58918ccc24" Dec 03 11:31:38 crc kubenswrapper[5052]: I1203 11:31:38.162428 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2363c55c03e9159893688a52c5ba318a997f018c828f223626074a58918ccc24"} err="failed to get container status \"2363c55c03e9159893688a52c5ba318a997f018c828f223626074a58918ccc24\": rpc error: code = NotFound desc = could not find container \"2363c55c03e9159893688a52c5ba318a997f018c828f223626074a58918ccc24\": container with ID starting with 2363c55c03e9159893688a52c5ba318a997f018c828f223626074a58918ccc24 not found: ID does not exist" Dec 03 11:31:38 crc kubenswrapper[5052]: I1203 11:31:38.162481 5052 scope.go:117] "RemoveContainer" containerID="4def7c1172d2ae52e275b2371cf88c9f93c3f0c747c18de7d1073d50b03b19bc" Dec 03 11:31:38 crc kubenswrapper[5052]: I1203 11:31:38.162714 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4def7c1172d2ae52e275b2371cf88c9f93c3f0c747c18de7d1073d50b03b19bc"} err="failed to get container status \"4def7c1172d2ae52e275b2371cf88c9f93c3f0c747c18de7d1073d50b03b19bc\": rpc error: code = NotFound desc = could not find container \"4def7c1172d2ae52e275b2371cf88c9f93c3f0c747c18de7d1073d50b03b19bc\": container with ID starting with 4def7c1172d2ae52e275b2371cf88c9f93c3f0c747c18de7d1073d50b03b19bc not found: ID does not exist" Dec 03 11:31:38 crc kubenswrapper[5052]: I1203 11:31:38.162744 5052 scope.go:117] "RemoveContainer" containerID="7548847d4c67dfe4acdf79ad3da86e54d4766ed7657a83a725c1ecc4a22c44dd" Dec 03 11:31:38 crc kubenswrapper[5052]: I1203 11:31:38.162974 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7548847d4c67dfe4acdf79ad3da86e54d4766ed7657a83a725c1ecc4a22c44dd"} err="failed to get container status \"7548847d4c67dfe4acdf79ad3da86e54d4766ed7657a83a725c1ecc4a22c44dd\": rpc error: code = NotFound desc = could not find container \"7548847d4c67dfe4acdf79ad3da86e54d4766ed7657a83a725c1ecc4a22c44dd\": container with ID starting with 7548847d4c67dfe4acdf79ad3da86e54d4766ed7657a83a725c1ecc4a22c44dd not found: ID does not exist" Dec 03 11:31:38 crc kubenswrapper[5052]: I1203 11:31:38.162998 5052 scope.go:117] "RemoveContainer" containerID="affc8d8f111572fbded3dbcc4c9f374ca5ba664cd6989bb1e717c163a1d88417" Dec 03 11:31:38 crc kubenswrapper[5052]: I1203 11:31:38.163296 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"affc8d8f111572fbded3dbcc4c9f374ca5ba664cd6989bb1e717c163a1d88417"} err="failed to get container status \"affc8d8f111572fbded3dbcc4c9f374ca5ba664cd6989bb1e717c163a1d88417\": rpc error: code = NotFound desc = could not find container \"affc8d8f111572fbded3dbcc4c9f374ca5ba664cd6989bb1e717c163a1d88417\": container with ID starting with affc8d8f111572fbded3dbcc4c9f374ca5ba664cd6989bb1e717c163a1d88417 not found: ID does not exist" Dec 03 11:31:38 crc kubenswrapper[5052]: I1203 11:31:38.163349 5052 scope.go:117] "RemoveContainer" containerID="c16097e9c17ec2d194aef1c4af913073743ff9b3e54b20999b8b48ce26176ecb" Dec 03 11:31:38 crc kubenswrapper[5052]: I1203 11:31:38.163611 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c16097e9c17ec2d194aef1c4af913073743ff9b3e54b20999b8b48ce26176ecb"} err="failed to get container status \"c16097e9c17ec2d194aef1c4af913073743ff9b3e54b20999b8b48ce26176ecb\": rpc error: code = NotFound desc = could not find container \"c16097e9c17ec2d194aef1c4af913073743ff9b3e54b20999b8b48ce26176ecb\": container with ID starting with c16097e9c17ec2d194aef1c4af913073743ff9b3e54b20999b8b48ce26176ecb not found: ID does not exist" Dec 03 11:31:38 crc kubenswrapper[5052]: I1203 11:31:38.163631 5052 scope.go:117] "RemoveContainer" containerID="5b3ab2b483c488f2669acaf42538650e9c65cd6da9b9a5296572ed8f4702a685" Dec 03 11:31:38 crc kubenswrapper[5052]: I1203 11:31:38.163878 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5b3ab2b483c488f2669acaf42538650e9c65cd6da9b9a5296572ed8f4702a685"} err="failed to get container status \"5b3ab2b483c488f2669acaf42538650e9c65cd6da9b9a5296572ed8f4702a685\": rpc error: code = NotFound desc = could not find container \"5b3ab2b483c488f2669acaf42538650e9c65cd6da9b9a5296572ed8f4702a685\": container with ID starting with 5b3ab2b483c488f2669acaf42538650e9c65cd6da9b9a5296572ed8f4702a685 not found: ID does not exist" Dec 03 11:31:38 crc kubenswrapper[5052]: I1203 11:31:38.163905 5052 scope.go:117] "RemoveContainer" containerID="9201f740cc3848eee706f3415f90bbffac06b7df3bb60f3495eff3c1d0a7f3a8" Dec 03 11:31:38 crc kubenswrapper[5052]: I1203 11:31:38.164155 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9201f740cc3848eee706f3415f90bbffac06b7df3bb60f3495eff3c1d0a7f3a8"} err="failed to get container status \"9201f740cc3848eee706f3415f90bbffac06b7df3bb60f3495eff3c1d0a7f3a8\": rpc error: code = NotFound desc = could not find container \"9201f740cc3848eee706f3415f90bbffac06b7df3bb60f3495eff3c1d0a7f3a8\": container with ID starting with 9201f740cc3848eee706f3415f90bbffac06b7df3bb60f3495eff3c1d0a7f3a8 not found: ID does not exist" Dec 03 11:31:38 crc kubenswrapper[5052]: I1203 11:31:38.164192 5052 scope.go:117] "RemoveContainer" containerID="bde78769f16ee1d0101355c5c8eb4a48437c04ccc42ca5104d1b98538b2545be" Dec 03 11:31:38 crc kubenswrapper[5052]: I1203 11:31:38.164390 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bde78769f16ee1d0101355c5c8eb4a48437c04ccc42ca5104d1b98538b2545be"} err="failed to get container status \"bde78769f16ee1d0101355c5c8eb4a48437c04ccc42ca5104d1b98538b2545be\": rpc error: code = NotFound desc = could not find container \"bde78769f16ee1d0101355c5c8eb4a48437c04ccc42ca5104d1b98538b2545be\": container with ID starting with bde78769f16ee1d0101355c5c8eb4a48437c04ccc42ca5104d1b98538b2545be not found: ID does not exist" Dec 03 11:31:38 crc kubenswrapper[5052]: I1203 11:31:38.164411 5052 scope.go:117] "RemoveContainer" containerID="896c25e34b1cb8871fd92cc6c8c908bfce14a9713345f571a672b2156a1765bf" Dec 03 11:31:38 crc kubenswrapper[5052]: I1203 11:31:38.164667 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"896c25e34b1cb8871fd92cc6c8c908bfce14a9713345f571a672b2156a1765bf"} err="failed to get container status \"896c25e34b1cb8871fd92cc6c8c908bfce14a9713345f571a672b2156a1765bf\": rpc error: code = NotFound desc = could not find container \"896c25e34b1cb8871fd92cc6c8c908bfce14a9713345f571a672b2156a1765bf\": container with ID starting with 896c25e34b1cb8871fd92cc6c8c908bfce14a9713345f571a672b2156a1765bf not found: ID does not exist" Dec 03 11:31:38 crc kubenswrapper[5052]: I1203 11:31:38.164687 5052 scope.go:117] "RemoveContainer" containerID="2363c55c03e9159893688a52c5ba318a997f018c828f223626074a58918ccc24" Dec 03 11:31:38 crc kubenswrapper[5052]: I1203 11:31:38.164877 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2363c55c03e9159893688a52c5ba318a997f018c828f223626074a58918ccc24"} err="failed to get container status \"2363c55c03e9159893688a52c5ba318a997f018c828f223626074a58918ccc24\": rpc error: code = NotFound desc = could not find container \"2363c55c03e9159893688a52c5ba318a997f018c828f223626074a58918ccc24\": container with ID starting with 2363c55c03e9159893688a52c5ba318a997f018c828f223626074a58918ccc24 not found: ID does not exist" Dec 03 11:31:38 crc kubenswrapper[5052]: I1203 11:31:38.164913 5052 scope.go:117] "RemoveContainer" containerID="4def7c1172d2ae52e275b2371cf88c9f93c3f0c747c18de7d1073d50b03b19bc" Dec 03 11:31:38 crc kubenswrapper[5052]: I1203 11:31:38.165095 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4def7c1172d2ae52e275b2371cf88c9f93c3f0c747c18de7d1073d50b03b19bc"} err="failed to get container status \"4def7c1172d2ae52e275b2371cf88c9f93c3f0c747c18de7d1073d50b03b19bc\": rpc error: code = NotFound desc = could not find container \"4def7c1172d2ae52e275b2371cf88c9f93c3f0c747c18de7d1073d50b03b19bc\": container with ID starting with 4def7c1172d2ae52e275b2371cf88c9f93c3f0c747c18de7d1073d50b03b19bc not found: ID does not exist" Dec 03 11:31:38 crc kubenswrapper[5052]: I1203 11:31:38.165114 5052 scope.go:117] "RemoveContainer" containerID="7548847d4c67dfe4acdf79ad3da86e54d4766ed7657a83a725c1ecc4a22c44dd" Dec 03 11:31:38 crc kubenswrapper[5052]: I1203 11:31:38.165305 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7548847d4c67dfe4acdf79ad3da86e54d4766ed7657a83a725c1ecc4a22c44dd"} err="failed to get container status \"7548847d4c67dfe4acdf79ad3da86e54d4766ed7657a83a725c1ecc4a22c44dd\": rpc error: code = NotFound desc = could not find container \"7548847d4c67dfe4acdf79ad3da86e54d4766ed7657a83a725c1ecc4a22c44dd\": container with ID starting with 7548847d4c67dfe4acdf79ad3da86e54d4766ed7657a83a725c1ecc4a22c44dd not found: ID does not exist" Dec 03 11:31:38 crc kubenswrapper[5052]: I1203 11:31:38.165353 5052 scope.go:117] "RemoveContainer" containerID="affc8d8f111572fbded3dbcc4c9f374ca5ba664cd6989bb1e717c163a1d88417" Dec 03 11:31:38 crc kubenswrapper[5052]: I1203 11:31:38.165874 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"affc8d8f111572fbded3dbcc4c9f374ca5ba664cd6989bb1e717c163a1d88417"} err="failed to get container status \"affc8d8f111572fbded3dbcc4c9f374ca5ba664cd6989bb1e717c163a1d88417\": rpc error: code = NotFound desc = could not find container \"affc8d8f111572fbded3dbcc4c9f374ca5ba664cd6989bb1e717c163a1d88417\": container with ID starting with affc8d8f111572fbded3dbcc4c9f374ca5ba664cd6989bb1e717c163a1d88417 not found: ID does not exist" Dec 03 11:31:38 crc kubenswrapper[5052]: I1203 11:31:38.166010 5052 scope.go:117] "RemoveContainer" containerID="c16097e9c17ec2d194aef1c4af913073743ff9b3e54b20999b8b48ce26176ecb" Dec 03 11:31:38 crc kubenswrapper[5052]: I1203 11:31:38.166218 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c16097e9c17ec2d194aef1c4af913073743ff9b3e54b20999b8b48ce26176ecb"} err="failed to get container status \"c16097e9c17ec2d194aef1c4af913073743ff9b3e54b20999b8b48ce26176ecb\": rpc error: code = NotFound desc = could not find container \"c16097e9c17ec2d194aef1c4af913073743ff9b3e54b20999b8b48ce26176ecb\": container with ID starting with c16097e9c17ec2d194aef1c4af913073743ff9b3e54b20999b8b48ce26176ecb not found: ID does not exist" Dec 03 11:31:38 crc kubenswrapper[5052]: I1203 11:31:38.166257 5052 scope.go:117] "RemoveContainer" containerID="5b3ab2b483c488f2669acaf42538650e9c65cd6da9b9a5296572ed8f4702a685" Dec 03 11:31:38 crc kubenswrapper[5052]: I1203 11:31:38.166461 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5b3ab2b483c488f2669acaf42538650e9c65cd6da9b9a5296572ed8f4702a685"} err="failed to get container status \"5b3ab2b483c488f2669acaf42538650e9c65cd6da9b9a5296572ed8f4702a685\": rpc error: code = NotFound desc = could not find container \"5b3ab2b483c488f2669acaf42538650e9c65cd6da9b9a5296572ed8f4702a685\": container with ID starting with 5b3ab2b483c488f2669acaf42538650e9c65cd6da9b9a5296572ed8f4702a685 not found: ID does not exist" Dec 03 11:31:38 crc kubenswrapper[5052]: I1203 11:31:38.166490 5052 scope.go:117] "RemoveContainer" containerID="9201f740cc3848eee706f3415f90bbffac06b7df3bb60f3495eff3c1d0a7f3a8" Dec 03 11:31:38 crc kubenswrapper[5052]: I1203 11:31:38.166705 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9201f740cc3848eee706f3415f90bbffac06b7df3bb60f3495eff3c1d0a7f3a8"} err="failed to get container status \"9201f740cc3848eee706f3415f90bbffac06b7df3bb60f3495eff3c1d0a7f3a8\": rpc error: code = NotFound desc = could not find container \"9201f740cc3848eee706f3415f90bbffac06b7df3bb60f3495eff3c1d0a7f3a8\": container with ID starting with 9201f740cc3848eee706f3415f90bbffac06b7df3bb60f3495eff3c1d0a7f3a8 not found: ID does not exist" Dec 03 11:31:38 crc kubenswrapper[5052]: I1203 11:31:38.166724 5052 scope.go:117] "RemoveContainer" containerID="bde78769f16ee1d0101355c5c8eb4a48437c04ccc42ca5104d1b98538b2545be" Dec 03 11:31:38 crc kubenswrapper[5052]: I1203 11:31:38.166905 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bde78769f16ee1d0101355c5c8eb4a48437c04ccc42ca5104d1b98538b2545be"} err="failed to get container status \"bde78769f16ee1d0101355c5c8eb4a48437c04ccc42ca5104d1b98538b2545be\": rpc error: code = NotFound desc = could not find container \"bde78769f16ee1d0101355c5c8eb4a48437c04ccc42ca5104d1b98538b2545be\": container with ID starting with bde78769f16ee1d0101355c5c8eb4a48437c04ccc42ca5104d1b98538b2545be not found: ID does not exist" Dec 03 11:31:38 crc kubenswrapper[5052]: I1203 11:31:38.166923 5052 scope.go:117] "RemoveContainer" containerID="896c25e34b1cb8871fd92cc6c8c908bfce14a9713345f571a672b2156a1765bf" Dec 03 11:31:38 crc kubenswrapper[5052]: I1203 11:31:38.167727 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"896c25e34b1cb8871fd92cc6c8c908bfce14a9713345f571a672b2156a1765bf"} err="failed to get container status \"896c25e34b1cb8871fd92cc6c8c908bfce14a9713345f571a672b2156a1765bf\": rpc error: code = NotFound desc = could not find container \"896c25e34b1cb8871fd92cc6c8c908bfce14a9713345f571a672b2156a1765bf\": container with ID starting with 896c25e34b1cb8871fd92cc6c8c908bfce14a9713345f571a672b2156a1765bf not found: ID does not exist" Dec 03 11:31:38 crc kubenswrapper[5052]: I1203 11:31:38.167756 5052 scope.go:117] "RemoveContainer" containerID="2363c55c03e9159893688a52c5ba318a997f018c828f223626074a58918ccc24" Dec 03 11:31:38 crc kubenswrapper[5052]: I1203 11:31:38.168008 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2363c55c03e9159893688a52c5ba318a997f018c828f223626074a58918ccc24"} err="failed to get container status \"2363c55c03e9159893688a52c5ba318a997f018c828f223626074a58918ccc24\": rpc error: code = NotFound desc = could not find container \"2363c55c03e9159893688a52c5ba318a997f018c828f223626074a58918ccc24\": container with ID starting with 2363c55c03e9159893688a52c5ba318a997f018c828f223626074a58918ccc24 not found: ID does not exist" Dec 03 11:31:38 crc kubenswrapper[5052]: I1203 11:31:38.168065 5052 scope.go:117] "RemoveContainer" containerID="4def7c1172d2ae52e275b2371cf88c9f93c3f0c747c18de7d1073d50b03b19bc" Dec 03 11:31:38 crc kubenswrapper[5052]: I1203 11:31:38.168295 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4def7c1172d2ae52e275b2371cf88c9f93c3f0c747c18de7d1073d50b03b19bc"} err="failed to get container status \"4def7c1172d2ae52e275b2371cf88c9f93c3f0c747c18de7d1073d50b03b19bc\": rpc error: code = NotFound desc = could not find container \"4def7c1172d2ae52e275b2371cf88c9f93c3f0c747c18de7d1073d50b03b19bc\": container with ID starting with 4def7c1172d2ae52e275b2371cf88c9f93c3f0c747c18de7d1073d50b03b19bc not found: ID does not exist" Dec 03 11:31:38 crc kubenswrapper[5052]: I1203 11:31:38.168317 5052 scope.go:117] "RemoveContainer" containerID="7548847d4c67dfe4acdf79ad3da86e54d4766ed7657a83a725c1ecc4a22c44dd" Dec 03 11:31:38 crc kubenswrapper[5052]: I1203 11:31:38.168548 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7548847d4c67dfe4acdf79ad3da86e54d4766ed7657a83a725c1ecc4a22c44dd"} err="failed to get container status \"7548847d4c67dfe4acdf79ad3da86e54d4766ed7657a83a725c1ecc4a22c44dd\": rpc error: code = NotFound desc = could not find container \"7548847d4c67dfe4acdf79ad3da86e54d4766ed7657a83a725c1ecc4a22c44dd\": container with ID starting with 7548847d4c67dfe4acdf79ad3da86e54d4766ed7657a83a725c1ecc4a22c44dd not found: ID does not exist" Dec 03 11:31:38 crc kubenswrapper[5052]: I1203 11:31:38.168568 5052 scope.go:117] "RemoveContainer" containerID="affc8d8f111572fbded3dbcc4c9f374ca5ba664cd6989bb1e717c163a1d88417" Dec 03 11:31:38 crc kubenswrapper[5052]: I1203 11:31:38.168812 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"affc8d8f111572fbded3dbcc4c9f374ca5ba664cd6989bb1e717c163a1d88417"} err="failed to get container status \"affc8d8f111572fbded3dbcc4c9f374ca5ba664cd6989bb1e717c163a1d88417\": rpc error: code = NotFound desc = could not find container \"affc8d8f111572fbded3dbcc4c9f374ca5ba664cd6989bb1e717c163a1d88417\": container with ID starting with affc8d8f111572fbded3dbcc4c9f374ca5ba664cd6989bb1e717c163a1d88417 not found: ID does not exist" Dec 03 11:31:38 crc kubenswrapper[5052]: I1203 11:31:38.168853 5052 scope.go:117] "RemoveContainer" containerID="c16097e9c17ec2d194aef1c4af913073743ff9b3e54b20999b8b48ce26176ecb" Dec 03 11:31:38 crc kubenswrapper[5052]: I1203 11:31:38.169091 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c16097e9c17ec2d194aef1c4af913073743ff9b3e54b20999b8b48ce26176ecb"} err="failed to get container status \"c16097e9c17ec2d194aef1c4af913073743ff9b3e54b20999b8b48ce26176ecb\": rpc error: code = NotFound desc = could not find container \"c16097e9c17ec2d194aef1c4af913073743ff9b3e54b20999b8b48ce26176ecb\": container with ID starting with c16097e9c17ec2d194aef1c4af913073743ff9b3e54b20999b8b48ce26176ecb not found: ID does not exist" Dec 03 11:31:38 crc kubenswrapper[5052]: I1203 11:31:38.169140 5052 scope.go:117] "RemoveContainer" containerID="5b3ab2b483c488f2669acaf42538650e9c65cd6da9b9a5296572ed8f4702a685" Dec 03 11:31:38 crc kubenswrapper[5052]: I1203 11:31:38.169473 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5b3ab2b483c488f2669acaf42538650e9c65cd6da9b9a5296572ed8f4702a685"} err="failed to get container status \"5b3ab2b483c488f2669acaf42538650e9c65cd6da9b9a5296572ed8f4702a685\": rpc error: code = NotFound desc = could not find container \"5b3ab2b483c488f2669acaf42538650e9c65cd6da9b9a5296572ed8f4702a685\": container with ID starting with 5b3ab2b483c488f2669acaf42538650e9c65cd6da9b9a5296572ed8f4702a685 not found: ID does not exist" Dec 03 11:31:38 crc kubenswrapper[5052]: I1203 11:31:38.169563 5052 scope.go:117] "RemoveContainer" containerID="9201f740cc3848eee706f3415f90bbffac06b7df3bb60f3495eff3c1d0a7f3a8" Dec 03 11:31:38 crc kubenswrapper[5052]: I1203 11:31:38.169787 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9201f740cc3848eee706f3415f90bbffac06b7df3bb60f3495eff3c1d0a7f3a8"} err="failed to get container status \"9201f740cc3848eee706f3415f90bbffac06b7df3bb60f3495eff3c1d0a7f3a8\": rpc error: code = NotFound desc = could not find container \"9201f740cc3848eee706f3415f90bbffac06b7df3bb60f3495eff3c1d0a7f3a8\": container with ID starting with 9201f740cc3848eee706f3415f90bbffac06b7df3bb60f3495eff3c1d0a7f3a8 not found: ID does not exist" Dec 03 11:31:38 crc kubenswrapper[5052]: I1203 11:31:38.169808 5052 scope.go:117] "RemoveContainer" containerID="bde78769f16ee1d0101355c5c8eb4a48437c04ccc42ca5104d1b98538b2545be" Dec 03 11:31:38 crc kubenswrapper[5052]: I1203 11:31:38.169975 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bde78769f16ee1d0101355c5c8eb4a48437c04ccc42ca5104d1b98538b2545be"} err="failed to get container status \"bde78769f16ee1d0101355c5c8eb4a48437c04ccc42ca5104d1b98538b2545be\": rpc error: code = NotFound desc = could not find container \"bde78769f16ee1d0101355c5c8eb4a48437c04ccc42ca5104d1b98538b2545be\": container with ID starting with bde78769f16ee1d0101355c5c8eb4a48437c04ccc42ca5104d1b98538b2545be not found: ID does not exist" Dec 03 11:31:38 crc kubenswrapper[5052]: I1203 11:31:38.934812 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-hx657_c30c02db-f4e7-4432-941c-5448fd71f3d9/kube-multus/0.log" Dec 03 11:31:38 crc kubenswrapper[5052]: I1203 11:31:38.935337 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-hx657" event={"ID":"c30c02db-f4e7-4432-941c-5448fd71f3d9","Type":"ContainerStarted","Data":"cd57f51366b45dae8138d897518460f88d1761bc21693c42cbe01b7ad987912a"} Dec 03 11:31:38 crc kubenswrapper[5052]: I1203 11:31:38.940020 5052 generic.go:334] "Generic (PLEG): container finished" podID="17e134b4-4012-46f9-9824-0e3df9873d09" containerID="3f7aa024ee4bc7a309e51cd9c00c9658939bd2cbb8598681dda90e77ad2c9687" exitCode=0 Dec 03 11:31:38 crc kubenswrapper[5052]: I1203 11:31:38.940122 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rkzqc" event={"ID":"17e134b4-4012-46f9-9824-0e3df9873d09","Type":"ContainerDied","Data":"3f7aa024ee4bc7a309e51cd9c00c9658939bd2cbb8598681dda90e77ad2c9687"} Dec 03 11:31:39 crc kubenswrapper[5052]: I1203 11:31:39.679830 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bc569bba-c020-40d6-af68-e617625fe0e4" path="/var/lib/kubelet/pods/bc569bba-c020-40d6-af68-e617625fe0e4/volumes" Dec 03 11:31:39 crc kubenswrapper[5052]: I1203 11:31:39.955656 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rkzqc" event={"ID":"17e134b4-4012-46f9-9824-0e3df9873d09","Type":"ContainerStarted","Data":"9ac29664b5bebac5705ec542a5ebd4412feb15361c90dda7aa52bd82e4de6a2e"} Dec 03 11:31:39 crc kubenswrapper[5052]: I1203 11:31:39.956058 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rkzqc" event={"ID":"17e134b4-4012-46f9-9824-0e3df9873d09","Type":"ContainerStarted","Data":"1220474b49d97653b56f22fb24846e83745580b7508a255ead3c8c9173a5dc81"} Dec 03 11:31:39 crc kubenswrapper[5052]: I1203 11:31:39.956073 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rkzqc" event={"ID":"17e134b4-4012-46f9-9824-0e3df9873d09","Type":"ContainerStarted","Data":"ed40e4155a29b4e0ffbba4822f1dacc4c206b78c9968475b26db0e1c1cc9c7cb"} Dec 03 11:31:39 crc kubenswrapper[5052]: I1203 11:31:39.956086 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rkzqc" event={"ID":"17e134b4-4012-46f9-9824-0e3df9873d09","Type":"ContainerStarted","Data":"66bf2d779397d44730a81ec22bea30c51e4eddd3e8fa17750bc2282302dd94b1"} Dec 03 11:31:39 crc kubenswrapper[5052]: I1203 11:31:39.956101 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rkzqc" event={"ID":"17e134b4-4012-46f9-9824-0e3df9873d09","Type":"ContainerStarted","Data":"e963e98e88801c42235ccb30e03d192bea424e63257e1bfcdd4f2e183fa97cf0"} Dec 03 11:31:39 crc kubenswrapper[5052]: I1203 11:31:39.956112 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rkzqc" event={"ID":"17e134b4-4012-46f9-9824-0e3df9873d09","Type":"ContainerStarted","Data":"22667f9e3ae5e64c2296b6efbf95f003c0eea1fc20dab6db61cbf3c1b61ec58d"} Dec 03 11:31:41 crc kubenswrapper[5052]: I1203 11:31:41.971075 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rkzqc" event={"ID":"17e134b4-4012-46f9-9824-0e3df9873d09","Type":"ContainerStarted","Data":"e0e5dbb4627fa870d8344931950f1ff8ed79815303adf8dcfa77e106f0d573d5"} Dec 03 11:31:44 crc kubenswrapper[5052]: I1203 11:31:44.998996 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rkzqc" event={"ID":"17e134b4-4012-46f9-9824-0e3df9873d09","Type":"ContainerStarted","Data":"c8d552fe40591f1338dcbf48f3b67532abff55caa5f2f7874ac3940af765c474"} Dec 03 11:31:45 crc kubenswrapper[5052]: I1203 11:31:44.999596 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openshift-ovn-kubernetes/ovnkube-node-rkzqc" Dec 03 11:31:45 crc kubenswrapper[5052]: I1203 11:31:44.999620 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openshift-ovn-kubernetes/ovnkube-node-rkzqc" Dec 03 11:31:45 crc kubenswrapper[5052]: I1203 11:31:45.034040 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-rkzqc" podStartSLOduration=8.034023682 podStartE2EDuration="8.034023682s" podCreationTimestamp="2025-12-03 11:31:37 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:31:45.032711014 +0000 UTC m=+695.620952788" watchObservedRunningTime="2025-12-03 11:31:45.034023682 +0000 UTC m=+695.622265446" Dec 03 11:31:45 crc kubenswrapper[5052]: I1203 11:31:45.042900 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-rkzqc" Dec 03 11:31:46 crc kubenswrapper[5052]: I1203 11:31:46.004394 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openshift-ovn-kubernetes/ovnkube-node-rkzqc" Dec 03 11:31:46 crc kubenswrapper[5052]: I1203 11:31:46.037705 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-rkzqc" Dec 03 11:31:53 crc kubenswrapper[5052]: I1203 11:31:53.605489 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openshift-marketplace/5064f9f8917b246f69f5d7fc025e7e6c34236c02bca31167615d38212frd48v"] Dec 03 11:31:53 crc kubenswrapper[5052]: I1203 11:31:53.607156 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/5064f9f8917b246f69f5d7fc025e7e6c34236c02bca31167615d38212frd48v" Dec 03 11:31:53 crc kubenswrapper[5052]: I1203 11:31:53.610883 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-marketplace"/"default-dockercfg-vmwhc" Dec 03 11:31:53 crc kubenswrapper[5052]: I1203 11:31:53.615165 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/5064f9f8917b246f69f5d7fc025e7e6c34236c02bca31167615d38212frd48v"] Dec 03 11:31:53 crc kubenswrapper[5052]: I1203 11:31:53.739251 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/58e476bb-2b81-464d-8331-b31e848ed4b4-util\") pod \"5064f9f8917b246f69f5d7fc025e7e6c34236c02bca31167615d38212frd48v\" (UID: \"58e476bb-2b81-464d-8331-b31e848ed4b4\") " pod="openshift-marketplace/5064f9f8917b246f69f5d7fc025e7e6c34236c02bca31167615d38212frd48v" Dec 03 11:31:53 crc kubenswrapper[5052]: I1203 11:31:53.739309 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/58e476bb-2b81-464d-8331-b31e848ed4b4-bundle\") pod \"5064f9f8917b246f69f5d7fc025e7e6c34236c02bca31167615d38212frd48v\" (UID: \"58e476bb-2b81-464d-8331-b31e848ed4b4\") " pod="openshift-marketplace/5064f9f8917b246f69f5d7fc025e7e6c34236c02bca31167615d38212frd48v" Dec 03 11:31:53 crc kubenswrapper[5052]: I1203 11:31:53.739344 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2n9fm\" (UniqueName: \"kubernetes.io/projected/58e476bb-2b81-464d-8331-b31e848ed4b4-kube-api-access-2n9fm\") pod \"5064f9f8917b246f69f5d7fc025e7e6c34236c02bca31167615d38212frd48v\" (UID: \"58e476bb-2b81-464d-8331-b31e848ed4b4\") " pod="openshift-marketplace/5064f9f8917b246f69f5d7fc025e7e6c34236c02bca31167615d38212frd48v" Dec 03 11:31:53 crc kubenswrapper[5052]: I1203 11:31:53.840373 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-2n9fm\" (UniqueName: \"kubernetes.io/projected/58e476bb-2b81-464d-8331-b31e848ed4b4-kube-api-access-2n9fm\") pod \"5064f9f8917b246f69f5d7fc025e7e6c34236c02bca31167615d38212frd48v\" (UID: \"58e476bb-2b81-464d-8331-b31e848ed4b4\") " pod="openshift-marketplace/5064f9f8917b246f69f5d7fc025e7e6c34236c02bca31167615d38212frd48v" Dec 03 11:31:53 crc kubenswrapper[5052]: I1203 11:31:53.840503 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/58e476bb-2b81-464d-8331-b31e848ed4b4-util\") pod \"5064f9f8917b246f69f5d7fc025e7e6c34236c02bca31167615d38212frd48v\" (UID: \"58e476bb-2b81-464d-8331-b31e848ed4b4\") " pod="openshift-marketplace/5064f9f8917b246f69f5d7fc025e7e6c34236c02bca31167615d38212frd48v" Dec 03 11:31:53 crc kubenswrapper[5052]: I1203 11:31:53.840606 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/58e476bb-2b81-464d-8331-b31e848ed4b4-bundle\") pod \"5064f9f8917b246f69f5d7fc025e7e6c34236c02bca31167615d38212frd48v\" (UID: \"58e476bb-2b81-464d-8331-b31e848ed4b4\") " pod="openshift-marketplace/5064f9f8917b246f69f5d7fc025e7e6c34236c02bca31167615d38212frd48v" Dec 03 11:31:53 crc kubenswrapper[5052]: I1203 11:31:53.841239 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/58e476bb-2b81-464d-8331-b31e848ed4b4-bundle\") pod \"5064f9f8917b246f69f5d7fc025e7e6c34236c02bca31167615d38212frd48v\" (UID: \"58e476bb-2b81-464d-8331-b31e848ed4b4\") " pod="openshift-marketplace/5064f9f8917b246f69f5d7fc025e7e6c34236c02bca31167615d38212frd48v" Dec 03 11:31:53 crc kubenswrapper[5052]: I1203 11:31:53.841412 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/58e476bb-2b81-464d-8331-b31e848ed4b4-util\") pod \"5064f9f8917b246f69f5d7fc025e7e6c34236c02bca31167615d38212frd48v\" (UID: \"58e476bb-2b81-464d-8331-b31e848ed4b4\") " pod="openshift-marketplace/5064f9f8917b246f69f5d7fc025e7e6c34236c02bca31167615d38212frd48v" Dec 03 11:31:53 crc kubenswrapper[5052]: I1203 11:31:53.868483 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-2n9fm\" (UniqueName: \"kubernetes.io/projected/58e476bb-2b81-464d-8331-b31e848ed4b4-kube-api-access-2n9fm\") pod \"5064f9f8917b246f69f5d7fc025e7e6c34236c02bca31167615d38212frd48v\" (UID: \"58e476bb-2b81-464d-8331-b31e848ed4b4\") " pod="openshift-marketplace/5064f9f8917b246f69f5d7fc025e7e6c34236c02bca31167615d38212frd48v" Dec 03 11:31:53 crc kubenswrapper[5052]: I1203 11:31:53.924563 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/5064f9f8917b246f69f5d7fc025e7e6c34236c02bca31167615d38212frd48v" Dec 03 11:31:54 crc kubenswrapper[5052]: I1203 11:31:54.177520 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/5064f9f8917b246f69f5d7fc025e7e6c34236c02bca31167615d38212frd48v"] Dec 03 11:31:54 crc kubenswrapper[5052]: W1203 11:31:54.184397 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58e476bb_2b81_464d_8331_b31e848ed4b4.slice/crio-cde0053e0ec935eecb8d9df7c62fe90098ecb6252df80a5aca2ca7072520dfe3 WatchSource:0}: Error finding container cde0053e0ec935eecb8d9df7c62fe90098ecb6252df80a5aca2ca7072520dfe3: Status 404 returned error can't find the container with id cde0053e0ec935eecb8d9df7c62fe90098ecb6252df80a5aca2ca7072520dfe3 Dec 03 11:31:55 crc kubenswrapper[5052]: I1203 11:31:55.058623 5052 generic.go:334] "Generic (PLEG): container finished" podID="58e476bb-2b81-464d-8331-b31e848ed4b4" containerID="293d2cf10ba9a3b46a508b19009e17fa3168a0641bbbb9b37f6d5f1083f891ab" exitCode=0 Dec 03 11:31:55 crc kubenswrapper[5052]: I1203 11:31:55.058718 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/5064f9f8917b246f69f5d7fc025e7e6c34236c02bca31167615d38212frd48v" event={"ID":"58e476bb-2b81-464d-8331-b31e848ed4b4","Type":"ContainerDied","Data":"293d2cf10ba9a3b46a508b19009e17fa3168a0641bbbb9b37f6d5f1083f891ab"} Dec 03 11:31:55 crc kubenswrapper[5052]: I1203 11:31:55.058976 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/5064f9f8917b246f69f5d7fc025e7e6c34236c02bca31167615d38212frd48v" event={"ID":"58e476bb-2b81-464d-8331-b31e848ed4b4","Type":"ContainerStarted","Data":"cde0053e0ec935eecb8d9df7c62fe90098ecb6252df80a5aca2ca7072520dfe3"} Dec 03 11:31:55 crc kubenswrapper[5052]: I1203 11:31:55.061280 5052 provider.go:102] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Dec 03 11:31:55 crc kubenswrapper[5052]: I1203 11:31:55.958690 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openshift-marketplace/redhat-operators-jszsg"] Dec 03 11:31:55 crc kubenswrapper[5052]: I1203 11:31:55.960308 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/redhat-operators-jszsg" Dec 03 11:31:55 crc kubenswrapper[5052]: I1203 11:31:55.977728 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/redhat-operators-jszsg"] Dec 03 11:31:56 crc kubenswrapper[5052]: I1203 11:31:56.111950 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/f29decad-dadb-4aeb-ab03-8e038f00deaf-catalog-content\") pod \"redhat-operators-jszsg\" (UID: \"f29decad-dadb-4aeb-ab03-8e038f00deaf\") " pod="openshift-marketplace/redhat-operators-jszsg" Dec 03 11:31:56 crc kubenswrapper[5052]: I1203 11:31:56.111997 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/f29decad-dadb-4aeb-ab03-8e038f00deaf-utilities\") pod \"redhat-operators-jszsg\" (UID: \"f29decad-dadb-4aeb-ab03-8e038f00deaf\") " pod="openshift-marketplace/redhat-operators-jszsg" Dec 03 11:31:56 crc kubenswrapper[5052]: I1203 11:31:56.112064 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nxsj6\" (UniqueName: \"kubernetes.io/projected/f29decad-dadb-4aeb-ab03-8e038f00deaf-kube-api-access-nxsj6\") pod \"redhat-operators-jszsg\" (UID: \"f29decad-dadb-4aeb-ab03-8e038f00deaf\") " pod="openshift-marketplace/redhat-operators-jszsg" Dec 03 11:31:56 crc kubenswrapper[5052]: I1203 11:31:56.213602 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-nxsj6\" (UniqueName: \"kubernetes.io/projected/f29decad-dadb-4aeb-ab03-8e038f00deaf-kube-api-access-nxsj6\") pod \"redhat-operators-jszsg\" (UID: \"f29decad-dadb-4aeb-ab03-8e038f00deaf\") " pod="openshift-marketplace/redhat-operators-jszsg" Dec 03 11:31:56 crc kubenswrapper[5052]: I1203 11:31:56.213707 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/f29decad-dadb-4aeb-ab03-8e038f00deaf-catalog-content\") pod \"redhat-operators-jszsg\" (UID: \"f29decad-dadb-4aeb-ab03-8e038f00deaf\") " pod="openshift-marketplace/redhat-operators-jszsg" Dec 03 11:31:56 crc kubenswrapper[5052]: I1203 11:31:56.213734 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/f29decad-dadb-4aeb-ab03-8e038f00deaf-utilities\") pod \"redhat-operators-jszsg\" (UID: \"f29decad-dadb-4aeb-ab03-8e038f00deaf\") " pod="openshift-marketplace/redhat-operators-jszsg" Dec 03 11:31:56 crc kubenswrapper[5052]: I1203 11:31:56.214272 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/f29decad-dadb-4aeb-ab03-8e038f00deaf-utilities\") pod \"redhat-operators-jszsg\" (UID: \"f29decad-dadb-4aeb-ab03-8e038f00deaf\") " pod="openshift-marketplace/redhat-operators-jszsg" Dec 03 11:31:56 crc kubenswrapper[5052]: I1203 11:31:56.214340 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/f29decad-dadb-4aeb-ab03-8e038f00deaf-catalog-content\") pod \"redhat-operators-jszsg\" (UID: \"f29decad-dadb-4aeb-ab03-8e038f00deaf\") " pod="openshift-marketplace/redhat-operators-jszsg" Dec 03 11:31:56 crc kubenswrapper[5052]: I1203 11:31:56.235368 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-nxsj6\" (UniqueName: \"kubernetes.io/projected/f29decad-dadb-4aeb-ab03-8e038f00deaf-kube-api-access-nxsj6\") pod \"redhat-operators-jszsg\" (UID: \"f29decad-dadb-4aeb-ab03-8e038f00deaf\") " pod="openshift-marketplace/redhat-operators-jszsg" Dec 03 11:31:56 crc kubenswrapper[5052]: I1203 11:31:56.274359 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/redhat-operators-jszsg" Dec 03 11:31:56 crc kubenswrapper[5052]: I1203 11:31:56.466795 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/redhat-operators-jszsg"] Dec 03 11:31:57 crc kubenswrapper[5052]: I1203 11:31:57.070591 5052 generic.go:334] "Generic (PLEG): container finished" podID="f29decad-dadb-4aeb-ab03-8e038f00deaf" containerID="e76f612a08f067f47a669848f19fe82b0c5c9924b8cb1601359fb01bf5dbc412" exitCode=0 Dec 03 11:31:57 crc kubenswrapper[5052]: I1203 11:31:57.070640 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-operators-jszsg" event={"ID":"f29decad-dadb-4aeb-ab03-8e038f00deaf","Type":"ContainerDied","Data":"e76f612a08f067f47a669848f19fe82b0c5c9924b8cb1601359fb01bf5dbc412"} Dec 03 11:31:57 crc kubenswrapper[5052]: I1203 11:31:57.070676 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-operators-jszsg" event={"ID":"f29decad-dadb-4aeb-ab03-8e038f00deaf","Type":"ContainerStarted","Data":"e9b13cb5a9dd4a54d705634b2fadd738a0c0d97afb89811b399ffc3d3a9a0032"} Dec 03 11:31:57 crc kubenswrapper[5052]: I1203 11:31:57.072491 5052 generic.go:334] "Generic (PLEG): container finished" podID="58e476bb-2b81-464d-8331-b31e848ed4b4" containerID="20300c0cb6d19e6df5e10d8550e67d0f1f50d5fc00aa0e3cf9b2829b38f8831d" exitCode=0 Dec 03 11:31:57 crc kubenswrapper[5052]: I1203 11:31:57.072529 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/5064f9f8917b246f69f5d7fc025e7e6c34236c02bca31167615d38212frd48v" event={"ID":"58e476bb-2b81-464d-8331-b31e848ed4b4","Type":"ContainerDied","Data":"20300c0cb6d19e6df5e10d8550e67d0f1f50d5fc00aa0e3cf9b2829b38f8831d"} Dec 03 11:31:58 crc kubenswrapper[5052]: I1203 11:31:58.082326 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-operators-jszsg" event={"ID":"f29decad-dadb-4aeb-ab03-8e038f00deaf","Type":"ContainerStarted","Data":"34b0851f40d459998977ca2e64786548f4c4e4d34c7d55a1cdf0f89c0aed531f"} Dec 03 11:31:58 crc kubenswrapper[5052]: I1203 11:31:58.084994 5052 generic.go:334] "Generic (PLEG): container finished" podID="58e476bb-2b81-464d-8331-b31e848ed4b4" containerID="71db9f9cb52cf8e6fda007eb7e5977c10a0b035ee0865d289cdfb42b763cd740" exitCode=0 Dec 03 11:31:58 crc kubenswrapper[5052]: I1203 11:31:58.085037 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/5064f9f8917b246f69f5d7fc025e7e6c34236c02bca31167615d38212frd48v" event={"ID":"58e476bb-2b81-464d-8331-b31e848ed4b4","Type":"ContainerDied","Data":"71db9f9cb52cf8e6fda007eb7e5977c10a0b035ee0865d289cdfb42b763cd740"} Dec 03 11:31:59 crc kubenswrapper[5052]: I1203 11:31:59.093099 5052 generic.go:334] "Generic (PLEG): container finished" podID="f29decad-dadb-4aeb-ab03-8e038f00deaf" containerID="34b0851f40d459998977ca2e64786548f4c4e4d34c7d55a1cdf0f89c0aed531f" exitCode=0 Dec 03 11:31:59 crc kubenswrapper[5052]: I1203 11:31:59.093186 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-operators-jszsg" event={"ID":"f29decad-dadb-4aeb-ab03-8e038f00deaf","Type":"ContainerDied","Data":"34b0851f40d459998977ca2e64786548f4c4e4d34c7d55a1cdf0f89c0aed531f"} Dec 03 11:31:59 crc kubenswrapper[5052]: I1203 11:31:59.333208 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/5064f9f8917b246f69f5d7fc025e7e6c34236c02bca31167615d38212frd48v" Dec 03 11:31:59 crc kubenswrapper[5052]: I1203 11:31:59.379346 5052 patch_prober.go:28] interesting pod/machine-config-daemon-74bj6 container/machine-config-daemon namespace/openshift-machine-config-operator: Liveness probe status=failure output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" start-of-body= Dec 03 11:31:59 crc kubenswrapper[5052]: I1203 11:31:59.379420 5052 prober.go:107] "Probe failed" probeType="Liveness" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" containerName="machine-config-daemon" probeResult="failure" output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" Dec 03 11:31:59 crc kubenswrapper[5052]: I1203 11:31:59.492746 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/58e476bb-2b81-464d-8331-b31e848ed4b4-util\") pod \"58e476bb-2b81-464d-8331-b31e848ed4b4\" (UID: \"58e476bb-2b81-464d-8331-b31e848ed4b4\") " Dec 03 11:31:59 crc kubenswrapper[5052]: I1203 11:31:59.492867 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/58e476bb-2b81-464d-8331-b31e848ed4b4-bundle\") pod \"58e476bb-2b81-464d-8331-b31e848ed4b4\" (UID: \"58e476bb-2b81-464d-8331-b31e848ed4b4\") " Dec 03 11:31:59 crc kubenswrapper[5052]: I1203 11:31:59.492939 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2n9fm\" (UniqueName: \"kubernetes.io/projected/58e476bb-2b81-464d-8331-b31e848ed4b4-kube-api-access-2n9fm\") pod \"58e476bb-2b81-464d-8331-b31e848ed4b4\" (UID: \"58e476bb-2b81-464d-8331-b31e848ed4b4\") " Dec 03 11:31:59 crc kubenswrapper[5052]: I1203 11:31:59.493748 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/58e476bb-2b81-464d-8331-b31e848ed4b4-bundle" (OuterVolumeSpecName: "bundle") pod "58e476bb-2b81-464d-8331-b31e848ed4b4" (UID: "58e476bb-2b81-464d-8331-b31e848ed4b4"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:31:59 crc kubenswrapper[5052]: I1203 11:31:59.503159 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/58e476bb-2b81-464d-8331-b31e848ed4b4-kube-api-access-2n9fm" (OuterVolumeSpecName: "kube-api-access-2n9fm") pod "58e476bb-2b81-464d-8331-b31e848ed4b4" (UID: "58e476bb-2b81-464d-8331-b31e848ed4b4"). InnerVolumeSpecName "kube-api-access-2n9fm". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:31:59 crc kubenswrapper[5052]: I1203 11:31:59.526575 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/58e476bb-2b81-464d-8331-b31e848ed4b4-util" (OuterVolumeSpecName: "util") pod "58e476bb-2b81-464d-8331-b31e848ed4b4" (UID: "58e476bb-2b81-464d-8331-b31e848ed4b4"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:31:59 crc kubenswrapper[5052]: I1203 11:31:59.595798 5052 reconciler_common.go:293] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/58e476bb-2b81-464d-8331-b31e848ed4b4-util\") on node \"crc\" DevicePath \"\"" Dec 03 11:31:59 crc kubenswrapper[5052]: I1203 11:31:59.595826 5052 reconciler_common.go:293] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/58e476bb-2b81-464d-8331-b31e848ed4b4-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:31:59 crc kubenswrapper[5052]: I1203 11:31:59.595835 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-2n9fm\" (UniqueName: \"kubernetes.io/projected/58e476bb-2b81-464d-8331-b31e848ed4b4-kube-api-access-2n9fm\") on node \"crc\" DevicePath \"\"" Dec 03 11:32:00 crc kubenswrapper[5052]: I1203 11:32:00.100523 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-operators-jszsg" event={"ID":"f29decad-dadb-4aeb-ab03-8e038f00deaf","Type":"ContainerStarted","Data":"4325f1a87badead06be67a05cfe54eeab515ee18d331bc6c1a38540e45b42abc"} Dec 03 11:32:00 crc kubenswrapper[5052]: I1203 11:32:00.102605 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/5064f9f8917b246f69f5d7fc025e7e6c34236c02bca31167615d38212frd48v" event={"ID":"58e476bb-2b81-464d-8331-b31e848ed4b4","Type":"ContainerDied","Data":"cde0053e0ec935eecb8d9df7c62fe90098ecb6252df80a5aca2ca7072520dfe3"} Dec 03 11:32:00 crc kubenswrapper[5052]: I1203 11:32:00.102632 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cde0053e0ec935eecb8d9df7c62fe90098ecb6252df80a5aca2ca7072520dfe3" Dec 03 11:32:00 crc kubenswrapper[5052]: I1203 11:32:00.102653 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/5064f9f8917b246f69f5d7fc025e7e6c34236c02bca31167615d38212frd48v" Dec 03 11:32:00 crc kubenswrapper[5052]: I1203 11:32:00.120713 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-marketplace/redhat-operators-jszsg" podStartSLOduration=2.641642962 podStartE2EDuration="5.120685827s" podCreationTimestamp="2025-12-03 11:31:55 +0000 UTC" firstStartedPulling="2025-12-03 11:31:57.07196975 +0000 UTC m=+707.660211514" lastFinishedPulling="2025-12-03 11:31:59.551012615 +0000 UTC m=+710.139254379" observedRunningTime="2025-12-03 11:32:00.116012783 +0000 UTC m=+710.704254547" watchObservedRunningTime="2025-12-03 11:32:00.120685827 +0000 UTC m=+710.708927611" Dec 03 11:32:03 crc kubenswrapper[5052]: I1203 11:32:03.855178 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openshift-nmstate/nmstate-operator-5b5b58f5c8-8bhqp"] Dec 03 11:32:03 crc kubenswrapper[5052]: E1203 11:32:03.855400 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="58e476bb-2b81-464d-8331-b31e848ed4b4" containerName="pull" Dec 03 11:32:03 crc kubenswrapper[5052]: I1203 11:32:03.855413 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="58e476bb-2b81-464d-8331-b31e848ed4b4" containerName="pull" Dec 03 11:32:03 crc kubenswrapper[5052]: E1203 11:32:03.855423 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="58e476bb-2b81-464d-8331-b31e848ed4b4" containerName="util" Dec 03 11:32:03 crc kubenswrapper[5052]: I1203 11:32:03.855429 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="58e476bb-2b81-464d-8331-b31e848ed4b4" containerName="util" Dec 03 11:32:03 crc kubenswrapper[5052]: E1203 11:32:03.855448 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="58e476bb-2b81-464d-8331-b31e848ed4b4" containerName="extract" Dec 03 11:32:03 crc kubenswrapper[5052]: I1203 11:32:03.855454 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="58e476bb-2b81-464d-8331-b31e848ed4b4" containerName="extract" Dec 03 11:32:03 crc kubenswrapper[5052]: I1203 11:32:03.855555 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="58e476bb-2b81-464d-8331-b31e848ed4b4" containerName="extract" Dec 03 11:32:03 crc kubenswrapper[5052]: I1203 11:32:03.855938 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-nmstate/nmstate-operator-5b5b58f5c8-8bhqp" Dec 03 11:32:03 crc kubenswrapper[5052]: I1203 11:32:03.859498 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-nmstate"/"nmstate-operator-dockercfg-hslz2" Dec 03 11:32:03 crc kubenswrapper[5052]: I1203 11:32:03.859588 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-nmstate"/"openshift-service-ca.crt" Dec 03 11:32:03 crc kubenswrapper[5052]: I1203 11:32:03.861966 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-nmstate"/"kube-root-ca.crt" Dec 03 11:32:03 crc kubenswrapper[5052]: I1203 11:32:03.875583 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-nmstate/nmstate-operator-5b5b58f5c8-8bhqp"] Dec 03 11:32:04 crc kubenswrapper[5052]: I1203 11:32:04.044377 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t2fwm\" (UniqueName: \"kubernetes.io/projected/11ca0a66-5f66-4ff4-a6d2-77048290159d-kube-api-access-t2fwm\") pod \"nmstate-operator-5b5b58f5c8-8bhqp\" (UID: \"11ca0a66-5f66-4ff4-a6d2-77048290159d\") " pod="openshift-nmstate/nmstate-operator-5b5b58f5c8-8bhqp" Dec 03 11:32:04 crc kubenswrapper[5052]: I1203 11:32:04.145279 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-t2fwm\" (UniqueName: \"kubernetes.io/projected/11ca0a66-5f66-4ff4-a6d2-77048290159d-kube-api-access-t2fwm\") pod \"nmstate-operator-5b5b58f5c8-8bhqp\" (UID: \"11ca0a66-5f66-4ff4-a6d2-77048290159d\") " pod="openshift-nmstate/nmstate-operator-5b5b58f5c8-8bhqp" Dec 03 11:32:04 crc kubenswrapper[5052]: I1203 11:32:04.164853 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-t2fwm\" (UniqueName: \"kubernetes.io/projected/11ca0a66-5f66-4ff4-a6d2-77048290159d-kube-api-access-t2fwm\") pod \"nmstate-operator-5b5b58f5c8-8bhqp\" (UID: \"11ca0a66-5f66-4ff4-a6d2-77048290159d\") " pod="openshift-nmstate/nmstate-operator-5b5b58f5c8-8bhqp" Dec 03 11:32:04 crc kubenswrapper[5052]: I1203 11:32:04.171298 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-nmstate/nmstate-operator-5b5b58f5c8-8bhqp" Dec 03 11:32:04 crc kubenswrapper[5052]: I1203 11:32:04.381399 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-nmstate/nmstate-operator-5b5b58f5c8-8bhqp"] Dec 03 11:32:05 crc kubenswrapper[5052]: I1203 11:32:05.131040 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-nmstate/nmstate-operator-5b5b58f5c8-8bhqp" event={"ID":"11ca0a66-5f66-4ff4-a6d2-77048290159d","Type":"ContainerStarted","Data":"261551930be7459a15b8d5f632a701cd9aad157fa327e64b7a357e7343896e5c"} Dec 03 11:32:06 crc kubenswrapper[5052]: I1203 11:32:06.275049 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-marketplace/redhat-operators-jszsg" Dec 03 11:32:06 crc kubenswrapper[5052]: I1203 11:32:06.275327 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openshift-marketplace/redhat-operators-jszsg" Dec 03 11:32:06 crc kubenswrapper[5052]: I1203 11:32:06.323780 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-marketplace/redhat-operators-jszsg" Dec 03 11:32:07 crc kubenswrapper[5052]: I1203 11:32:07.187373 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-marketplace/redhat-operators-jszsg" Dec 03 11:32:07 crc kubenswrapper[5052]: I1203 11:32:07.806707 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-rkzqc" Dec 03 11:32:08 crc kubenswrapper[5052]: I1203 11:32:08.949694 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openshift-marketplace/redhat-operators-jszsg"] Dec 03 11:32:09 crc kubenswrapper[5052]: I1203 11:32:09.156489 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openshift-marketplace/redhat-operators-jszsg" podUID="f29decad-dadb-4aeb-ab03-8e038f00deaf" containerName="registry-server" containerID="cri-o://4325f1a87badead06be67a05cfe54eeab515ee18d331bc6c1a38540e45b42abc" gracePeriod=2 Dec 03 11:32:09 crc kubenswrapper[5052]: I1203 11:32:09.157013 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-nmstate/nmstate-operator-5b5b58f5c8-8bhqp" event={"ID":"11ca0a66-5f66-4ff4-a6d2-77048290159d","Type":"ContainerStarted","Data":"b6764b1021ca9e5c6ecd0996580ba7f8eeece876ed77670d7987f1d54f93eb28"} Dec 03 11:32:09 crc kubenswrapper[5052]: I1203 11:32:09.181283 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-nmstate/nmstate-operator-5b5b58f5c8-8bhqp" podStartSLOduration=1.848214886 podStartE2EDuration="6.18126603s" podCreationTimestamp="2025-12-03 11:32:03 +0000 UTC" firstStartedPulling="2025-12-03 11:32:04.3929888 +0000 UTC m=+714.981230564" lastFinishedPulling="2025-12-03 11:32:08.726039934 +0000 UTC m=+719.314281708" observedRunningTime="2025-12-03 11:32:09.178329076 +0000 UTC m=+719.766570860" watchObservedRunningTime="2025-12-03 11:32:09.18126603 +0000 UTC m=+719.769507794" Dec 03 11:32:12 crc kubenswrapper[5052]: I1203 11:32:12.184547 5052 generic.go:334] "Generic (PLEG): container finished" podID="f29decad-dadb-4aeb-ab03-8e038f00deaf" containerID="4325f1a87badead06be67a05cfe54eeab515ee18d331bc6c1a38540e45b42abc" exitCode=0 Dec 03 11:32:12 crc kubenswrapper[5052]: I1203 11:32:12.184597 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-operators-jszsg" event={"ID":"f29decad-dadb-4aeb-ab03-8e038f00deaf","Type":"ContainerDied","Data":"4325f1a87badead06be67a05cfe54eeab515ee18d331bc6c1a38540e45b42abc"} Dec 03 11:32:12 crc kubenswrapper[5052]: I1203 11:32:12.311076 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/redhat-operators-jszsg" Dec 03 11:32:12 crc kubenswrapper[5052]: I1203 11:32:12.374732 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/f29decad-dadb-4aeb-ab03-8e038f00deaf-utilities\") pod \"f29decad-dadb-4aeb-ab03-8e038f00deaf\" (UID: \"f29decad-dadb-4aeb-ab03-8e038f00deaf\") " Dec 03 11:32:12 crc kubenswrapper[5052]: I1203 11:32:12.374866 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nxsj6\" (UniqueName: \"kubernetes.io/projected/f29decad-dadb-4aeb-ab03-8e038f00deaf-kube-api-access-nxsj6\") pod \"f29decad-dadb-4aeb-ab03-8e038f00deaf\" (UID: \"f29decad-dadb-4aeb-ab03-8e038f00deaf\") " Dec 03 11:32:12 crc kubenswrapper[5052]: I1203 11:32:12.374925 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/f29decad-dadb-4aeb-ab03-8e038f00deaf-catalog-content\") pod \"f29decad-dadb-4aeb-ab03-8e038f00deaf\" (UID: \"f29decad-dadb-4aeb-ab03-8e038f00deaf\") " Dec 03 11:32:12 crc kubenswrapper[5052]: I1203 11:32:12.375845 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f29decad-dadb-4aeb-ab03-8e038f00deaf-utilities" (OuterVolumeSpecName: "utilities") pod "f29decad-dadb-4aeb-ab03-8e038f00deaf" (UID: "f29decad-dadb-4aeb-ab03-8e038f00deaf"). InnerVolumeSpecName "utilities". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:32:12 crc kubenswrapper[5052]: I1203 11:32:12.415147 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f29decad-dadb-4aeb-ab03-8e038f00deaf-kube-api-access-nxsj6" (OuterVolumeSpecName: "kube-api-access-nxsj6") pod "f29decad-dadb-4aeb-ab03-8e038f00deaf" (UID: "f29decad-dadb-4aeb-ab03-8e038f00deaf"). InnerVolumeSpecName "kube-api-access-nxsj6". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:32:12 crc kubenswrapper[5052]: I1203 11:32:12.475793 5052 reconciler_common.go:293] "Volume detached for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/f29decad-dadb-4aeb-ab03-8e038f00deaf-utilities\") on node \"crc\" DevicePath \"\"" Dec 03 11:32:12 crc kubenswrapper[5052]: I1203 11:32:12.475822 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-nxsj6\" (UniqueName: \"kubernetes.io/projected/f29decad-dadb-4aeb-ab03-8e038f00deaf-kube-api-access-nxsj6\") on node \"crc\" DevicePath \"\"" Dec 03 11:32:12 crc kubenswrapper[5052]: I1203 11:32:12.477600 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f29decad-dadb-4aeb-ab03-8e038f00deaf-catalog-content" (OuterVolumeSpecName: "catalog-content") pod "f29decad-dadb-4aeb-ab03-8e038f00deaf" (UID: "f29decad-dadb-4aeb-ab03-8e038f00deaf"). InnerVolumeSpecName "catalog-content". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:32:12 crc kubenswrapper[5052]: I1203 11:32:12.577370 5052 reconciler_common.go:293] "Volume detached for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/f29decad-dadb-4aeb-ab03-8e038f00deaf-catalog-content\") on node \"crc\" DevicePath \"\"" Dec 03 11:32:13 crc kubenswrapper[5052]: I1203 11:32:13.191506 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-operators-jszsg" event={"ID":"f29decad-dadb-4aeb-ab03-8e038f00deaf","Type":"ContainerDied","Data":"e9b13cb5a9dd4a54d705634b2fadd738a0c0d97afb89811b399ffc3d3a9a0032"} Dec 03 11:32:13 crc kubenswrapper[5052]: I1203 11:32:13.191570 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/redhat-operators-jszsg" Dec 03 11:32:13 crc kubenswrapper[5052]: I1203 11:32:13.191587 5052 scope.go:117] "RemoveContainer" containerID="4325f1a87badead06be67a05cfe54eeab515ee18d331bc6c1a38540e45b42abc" Dec 03 11:32:13 crc kubenswrapper[5052]: I1203 11:32:13.205344 5052 scope.go:117] "RemoveContainer" containerID="34b0851f40d459998977ca2e64786548f4c4e4d34c7d55a1cdf0f89c0aed531f" Dec 03 11:32:13 crc kubenswrapper[5052]: I1203 11:32:13.221711 5052 scope.go:117] "RemoveContainer" containerID="e76f612a08f067f47a669848f19fe82b0c5c9924b8cb1601359fb01bf5dbc412" Dec 03 11:32:13 crc kubenswrapper[5052]: I1203 11:32:13.223295 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openshift-marketplace/redhat-operators-jszsg"] Dec 03 11:32:13 crc kubenswrapper[5052]: I1203 11:32:13.226956 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openshift-marketplace/redhat-operators-jszsg"] Dec 03 11:32:13 crc kubenswrapper[5052]: I1203 11:32:13.680301 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f29decad-dadb-4aeb-ab03-8e038f00deaf" path="/var/lib/kubelet/pods/f29decad-dadb-4aeb-ab03-8e038f00deaf/volumes" Dec 03 11:32:13 crc kubenswrapper[5052]: I1203 11:32:13.917010 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openshift-nmstate/nmstate-metrics-7f946cbc9-ffkk9"] Dec 03 11:32:13 crc kubenswrapper[5052]: E1203 11:32:13.917508 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="f29decad-dadb-4aeb-ab03-8e038f00deaf" containerName="registry-server" Dec 03 11:32:13 crc kubenswrapper[5052]: I1203 11:32:13.917524 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="f29decad-dadb-4aeb-ab03-8e038f00deaf" containerName="registry-server" Dec 03 11:32:13 crc kubenswrapper[5052]: E1203 11:32:13.917554 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="f29decad-dadb-4aeb-ab03-8e038f00deaf" containerName="extract-content" Dec 03 11:32:13 crc kubenswrapper[5052]: I1203 11:32:13.917560 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="f29decad-dadb-4aeb-ab03-8e038f00deaf" containerName="extract-content" Dec 03 11:32:13 crc kubenswrapper[5052]: E1203 11:32:13.917569 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="f29decad-dadb-4aeb-ab03-8e038f00deaf" containerName="extract-utilities" Dec 03 11:32:13 crc kubenswrapper[5052]: I1203 11:32:13.917575 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="f29decad-dadb-4aeb-ab03-8e038f00deaf" containerName="extract-utilities" Dec 03 11:32:13 crc kubenswrapper[5052]: I1203 11:32:13.917665 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="f29decad-dadb-4aeb-ab03-8e038f00deaf" containerName="registry-server" Dec 03 11:32:13 crc kubenswrapper[5052]: I1203 11:32:13.918178 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-nmstate/nmstate-metrics-7f946cbc9-ffkk9" Dec 03 11:32:13 crc kubenswrapper[5052]: I1203 11:32:13.955649 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-nmstate"/"nmstate-handler-dockercfg-jsj68" Dec 03 11:32:13 crc kubenswrapper[5052]: I1203 11:32:13.963511 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-nmstate/nmstate-metrics-7f946cbc9-ffkk9"] Dec 03 11:32:13 crc kubenswrapper[5052]: I1203 11:32:13.972489 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openshift-nmstate/nmstate-webhook-5f6d4c5ccb-gbkms"] Dec 03 11:32:13 crc kubenswrapper[5052]: I1203 11:32:13.973426 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-nmstate/nmstate-webhook-5f6d4c5ccb-gbkms" Dec 03 11:32:13 crc kubenswrapper[5052]: I1203 11:32:13.976613 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-nmstate"/"openshift-nmstate-webhook" Dec 03 11:32:13 crc kubenswrapper[5052]: I1203 11:32:13.989749 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-nmstate/nmstate-webhook-5f6d4c5ccb-gbkms"] Dec 03 11:32:13 crc kubenswrapper[5052]: I1203 11:32:13.995303 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-key-pair\" (UniqueName: \"kubernetes.io/secret/f60b6277-113c-459c-801b-aa5c94de641a-tls-key-pair\") pod \"nmstate-webhook-5f6d4c5ccb-gbkms\" (UID: \"f60b6277-113c-459c-801b-aa5c94de641a\") " pod="openshift-nmstate/nmstate-webhook-5f6d4c5ccb-gbkms" Dec 03 11:32:13 crc kubenswrapper[5052]: I1203 11:32:13.995366 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dcn6r\" (UniqueName: \"kubernetes.io/projected/bdc64cdc-4e09-4eff-b1bc-23b9dd86ab8b-kube-api-access-dcn6r\") pod \"nmstate-metrics-7f946cbc9-ffkk9\" (UID: \"bdc64cdc-4e09-4eff-b1bc-23b9dd86ab8b\") " pod="openshift-nmstate/nmstate-metrics-7f946cbc9-ffkk9" Dec 03 11:32:13 crc kubenswrapper[5052]: I1203 11:32:13.995432 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l98pf\" (UniqueName: \"kubernetes.io/projected/f60b6277-113c-459c-801b-aa5c94de641a-kube-api-access-l98pf\") pod \"nmstate-webhook-5f6d4c5ccb-gbkms\" (UID: \"f60b6277-113c-459c-801b-aa5c94de641a\") " pod="openshift-nmstate/nmstate-webhook-5f6d4c5ccb-gbkms" Dec 03 11:32:14 crc kubenswrapper[5052]: I1203 11:32:14.001561 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openshift-nmstate/nmstate-handler-qggff"] Dec 03 11:32:14 crc kubenswrapper[5052]: I1203 11:32:14.002436 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-nmstate/nmstate-handler-qggff" Dec 03 11:32:14 crc kubenswrapper[5052]: I1203 11:32:14.097063 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus-socket\" (UniqueName: \"kubernetes.io/host-path/318304dc-2b11-480e-8683-7627f288faa8-dbus-socket\") pod \"nmstate-handler-qggff\" (UID: \"318304dc-2b11-480e-8683-7627f288faa8\") " pod="openshift-nmstate/nmstate-handler-qggff" Dec 03 11:32:14 crc kubenswrapper[5052]: I1203 11:32:14.097113 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nmstate-lock\" (UniqueName: \"kubernetes.io/host-path/318304dc-2b11-480e-8683-7627f288faa8-nmstate-lock\") pod \"nmstate-handler-qggff\" (UID: \"318304dc-2b11-480e-8683-7627f288faa8\") " pod="openshift-nmstate/nmstate-handler-qggff" Dec 03 11:32:14 crc kubenswrapper[5052]: I1203 11:32:14.097146 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m6vml\" (UniqueName: \"kubernetes.io/projected/318304dc-2b11-480e-8683-7627f288faa8-kube-api-access-m6vml\") pod \"nmstate-handler-qggff\" (UID: \"318304dc-2b11-480e-8683-7627f288faa8\") " pod="openshift-nmstate/nmstate-handler-qggff" Dec 03 11:32:14 crc kubenswrapper[5052]: I1203 11:32:14.097170 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"tls-key-pair\" (UniqueName: \"kubernetes.io/secret/f60b6277-113c-459c-801b-aa5c94de641a-tls-key-pair\") pod \"nmstate-webhook-5f6d4c5ccb-gbkms\" (UID: \"f60b6277-113c-459c-801b-aa5c94de641a\") " pod="openshift-nmstate/nmstate-webhook-5f6d4c5ccb-gbkms" Dec 03 11:32:14 crc kubenswrapper[5052]: I1203 11:32:14.097204 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-dcn6r\" (UniqueName: \"kubernetes.io/projected/bdc64cdc-4e09-4eff-b1bc-23b9dd86ab8b-kube-api-access-dcn6r\") pod \"nmstate-metrics-7f946cbc9-ffkk9\" (UID: \"bdc64cdc-4e09-4eff-b1bc-23b9dd86ab8b\") " pod="openshift-nmstate/nmstate-metrics-7f946cbc9-ffkk9" Dec 03 11:32:14 crc kubenswrapper[5052]: E1203 11:32:14.097325 5052 secret.go:188] Couldn't get secret openshift-nmstate/openshift-nmstate-webhook: secret "openshift-nmstate-webhook" not found Dec 03 11:32:14 crc kubenswrapper[5052]: I1203 11:32:14.097329 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovs-socket\" (UniqueName: \"kubernetes.io/host-path/318304dc-2b11-480e-8683-7627f288faa8-ovs-socket\") pod \"nmstate-handler-qggff\" (UID: \"318304dc-2b11-480e-8683-7627f288faa8\") " pod="openshift-nmstate/nmstate-handler-qggff" Dec 03 11:32:14 crc kubenswrapper[5052]: E1203 11:32:14.097402 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f60b6277-113c-459c-801b-aa5c94de641a-tls-key-pair podName:f60b6277-113c-459c-801b-aa5c94de641a nodeName:}" failed. No retries permitted until 2025-12-03 11:32:14.597384148 +0000 UTC m=+725.185625912 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-key-pair" (UniqueName: "kubernetes.io/secret/f60b6277-113c-459c-801b-aa5c94de641a-tls-key-pair") pod "nmstate-webhook-5f6d4c5ccb-gbkms" (UID: "f60b6277-113c-459c-801b-aa5c94de641a") : secret "openshift-nmstate-webhook" not found Dec 03 11:32:14 crc kubenswrapper[5052]: I1203 11:32:14.097604 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-l98pf\" (UniqueName: \"kubernetes.io/projected/f60b6277-113c-459c-801b-aa5c94de641a-kube-api-access-l98pf\") pod \"nmstate-webhook-5f6d4c5ccb-gbkms\" (UID: \"f60b6277-113c-459c-801b-aa5c94de641a\") " pod="openshift-nmstate/nmstate-webhook-5f6d4c5ccb-gbkms" Dec 03 11:32:14 crc kubenswrapper[5052]: I1203 11:32:14.104241 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openshift-nmstate/nmstate-console-plugin-7fbb5f6569-br7vf"] Dec 03 11:32:14 crc kubenswrapper[5052]: I1203 11:32:14.104990 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-nmstate/nmstate-console-plugin-7fbb5f6569-br7vf" Dec 03 11:32:14 crc kubenswrapper[5052]: I1203 11:32:14.107735 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-nmstate"/"nginx-conf" Dec 03 11:32:14 crc kubenswrapper[5052]: I1203 11:32:14.108147 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-nmstate"/"default-dockercfg-9g9cg" Dec 03 11:32:14 crc kubenswrapper[5052]: I1203 11:32:14.109432 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-nmstate"/"plugin-serving-cert" Dec 03 11:32:14 crc kubenswrapper[5052]: I1203 11:32:14.114439 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-nmstate/nmstate-console-plugin-7fbb5f6569-br7vf"] Dec 03 11:32:14 crc kubenswrapper[5052]: I1203 11:32:14.123587 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-dcn6r\" (UniqueName: \"kubernetes.io/projected/bdc64cdc-4e09-4eff-b1bc-23b9dd86ab8b-kube-api-access-dcn6r\") pod \"nmstate-metrics-7f946cbc9-ffkk9\" (UID: \"bdc64cdc-4e09-4eff-b1bc-23b9dd86ab8b\") " pod="openshift-nmstate/nmstate-metrics-7f946cbc9-ffkk9" Dec 03 11:32:14 crc kubenswrapper[5052]: I1203 11:32:14.130479 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-l98pf\" (UniqueName: \"kubernetes.io/projected/f60b6277-113c-459c-801b-aa5c94de641a-kube-api-access-l98pf\") pod \"nmstate-webhook-5f6d4c5ccb-gbkms\" (UID: \"f60b6277-113c-459c-801b-aa5c94de641a\") " pod="openshift-nmstate/nmstate-webhook-5f6d4c5ccb-gbkms" Dec 03 11:32:14 crc kubenswrapper[5052]: I1203 11:32:14.199160 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/04ad7b19-29ef-46b9-8668-b5437b6c88cc-plugin-serving-cert\") pod \"nmstate-console-plugin-7fbb5f6569-br7vf\" (UID: \"04ad7b19-29ef-46b9-8668-b5437b6c88cc\") " pod="openshift-nmstate/nmstate-console-plugin-7fbb5f6569-br7vf" Dec 03 11:32:14 crc kubenswrapper[5052]: I1203 11:32:14.199215 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p6x5s\" (UniqueName: \"kubernetes.io/projected/04ad7b19-29ef-46b9-8668-b5437b6c88cc-kube-api-access-p6x5s\") pod \"nmstate-console-plugin-7fbb5f6569-br7vf\" (UID: \"04ad7b19-29ef-46b9-8668-b5437b6c88cc\") " pod="openshift-nmstate/nmstate-console-plugin-7fbb5f6569-br7vf" Dec 03 11:32:14 crc kubenswrapper[5052]: I1203 11:32:14.199249 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"dbus-socket\" (UniqueName: \"kubernetes.io/host-path/318304dc-2b11-480e-8683-7627f288faa8-dbus-socket\") pod \"nmstate-handler-qggff\" (UID: \"318304dc-2b11-480e-8683-7627f288faa8\") " pod="openshift-nmstate/nmstate-handler-qggff" Dec 03 11:32:14 crc kubenswrapper[5052]: I1203 11:32:14.199269 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"nmstate-lock\" (UniqueName: \"kubernetes.io/host-path/318304dc-2b11-480e-8683-7627f288faa8-nmstate-lock\") pod \"nmstate-handler-qggff\" (UID: \"318304dc-2b11-480e-8683-7627f288faa8\") " pod="openshift-nmstate/nmstate-handler-qggff" Dec 03 11:32:14 crc kubenswrapper[5052]: I1203 11:32:14.199342 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"nmstate-lock\" (UniqueName: \"kubernetes.io/host-path/318304dc-2b11-480e-8683-7627f288faa8-nmstate-lock\") pod \"nmstate-handler-qggff\" (UID: \"318304dc-2b11-480e-8683-7627f288faa8\") " pod="openshift-nmstate/nmstate-handler-qggff" Dec 03 11:32:14 crc kubenswrapper[5052]: I1203 11:32:14.199386 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-m6vml\" (UniqueName: \"kubernetes.io/projected/318304dc-2b11-480e-8683-7627f288faa8-kube-api-access-m6vml\") pod \"nmstate-handler-qggff\" (UID: \"318304dc-2b11-480e-8683-7627f288faa8\") " pod="openshift-nmstate/nmstate-handler-qggff" Dec 03 11:32:14 crc kubenswrapper[5052]: I1203 11:32:14.199412 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/04ad7b19-29ef-46b9-8668-b5437b6c88cc-nginx-conf\") pod \"nmstate-console-plugin-7fbb5f6569-br7vf\" (UID: \"04ad7b19-29ef-46b9-8668-b5437b6c88cc\") " pod="openshift-nmstate/nmstate-console-plugin-7fbb5f6569-br7vf" Dec 03 11:32:14 crc kubenswrapper[5052]: I1203 11:32:14.199432 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovs-socket\" (UniqueName: \"kubernetes.io/host-path/318304dc-2b11-480e-8683-7627f288faa8-ovs-socket\") pod \"nmstate-handler-qggff\" (UID: \"318304dc-2b11-480e-8683-7627f288faa8\") " pod="openshift-nmstate/nmstate-handler-qggff" Dec 03 11:32:14 crc kubenswrapper[5052]: I1203 11:32:14.199480 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovs-socket\" (UniqueName: \"kubernetes.io/host-path/318304dc-2b11-480e-8683-7627f288faa8-ovs-socket\") pod \"nmstate-handler-qggff\" (UID: \"318304dc-2b11-480e-8683-7627f288faa8\") " pod="openshift-nmstate/nmstate-handler-qggff" Dec 03 11:32:14 crc kubenswrapper[5052]: I1203 11:32:14.199492 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"dbus-socket\" (UniqueName: \"kubernetes.io/host-path/318304dc-2b11-480e-8683-7627f288faa8-dbus-socket\") pod \"nmstate-handler-qggff\" (UID: \"318304dc-2b11-480e-8683-7627f288faa8\") " pod="openshift-nmstate/nmstate-handler-qggff" Dec 03 11:32:14 crc kubenswrapper[5052]: I1203 11:32:14.216759 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-m6vml\" (UniqueName: \"kubernetes.io/projected/318304dc-2b11-480e-8683-7627f288faa8-kube-api-access-m6vml\") pod \"nmstate-handler-qggff\" (UID: \"318304dc-2b11-480e-8683-7627f288faa8\") " pod="openshift-nmstate/nmstate-handler-qggff" Dec 03 11:32:14 crc kubenswrapper[5052]: I1203 11:32:14.264851 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-nmstate/nmstate-metrics-7f946cbc9-ffkk9" Dec 03 11:32:14 crc kubenswrapper[5052]: I1203 11:32:14.283878 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openshift-console/console-55fbcbdddd-nmbpx"] Dec 03 11:32:14 crc kubenswrapper[5052]: I1203 11:32:14.284696 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-55fbcbdddd-nmbpx" Dec 03 11:32:14 crc kubenswrapper[5052]: I1203 11:32:14.300577 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-p6x5s\" (UniqueName: \"kubernetes.io/projected/04ad7b19-29ef-46b9-8668-b5437b6c88cc-kube-api-access-p6x5s\") pod \"nmstate-console-plugin-7fbb5f6569-br7vf\" (UID: \"04ad7b19-29ef-46b9-8668-b5437b6c88cc\") " pod="openshift-nmstate/nmstate-console-plugin-7fbb5f6569-br7vf" Dec 03 11:32:14 crc kubenswrapper[5052]: I1203 11:32:14.300687 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/04ad7b19-29ef-46b9-8668-b5437b6c88cc-nginx-conf\") pod \"nmstate-console-plugin-7fbb5f6569-br7vf\" (UID: \"04ad7b19-29ef-46b9-8668-b5437b6c88cc\") " pod="openshift-nmstate/nmstate-console-plugin-7fbb5f6569-br7vf" Dec 03 11:32:14 crc kubenswrapper[5052]: I1203 11:32:14.300730 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/04ad7b19-29ef-46b9-8668-b5437b6c88cc-plugin-serving-cert\") pod \"nmstate-console-plugin-7fbb5f6569-br7vf\" (UID: \"04ad7b19-29ef-46b9-8668-b5437b6c88cc\") " pod="openshift-nmstate/nmstate-console-plugin-7fbb5f6569-br7vf" Dec 03 11:32:14 crc kubenswrapper[5052]: I1203 11:32:14.302397 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/04ad7b19-29ef-46b9-8668-b5437b6c88cc-nginx-conf\") pod \"nmstate-console-plugin-7fbb5f6569-br7vf\" (UID: \"04ad7b19-29ef-46b9-8668-b5437b6c88cc\") " pod="openshift-nmstate/nmstate-console-plugin-7fbb5f6569-br7vf" Dec 03 11:32:14 crc kubenswrapper[5052]: I1203 11:32:14.302438 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-55fbcbdddd-nmbpx"] Dec 03 11:32:14 crc kubenswrapper[5052]: I1203 11:32:14.305912 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/04ad7b19-29ef-46b9-8668-b5437b6c88cc-plugin-serving-cert\") pod \"nmstate-console-plugin-7fbb5f6569-br7vf\" (UID: \"04ad7b19-29ef-46b9-8668-b5437b6c88cc\") " pod="openshift-nmstate/nmstate-console-plugin-7fbb5f6569-br7vf" Dec 03 11:32:14 crc kubenswrapper[5052]: I1203 11:32:14.335830 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-nmstate/nmstate-handler-qggff" Dec 03 11:32:14 crc kubenswrapper[5052]: I1203 11:32:14.340290 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-p6x5s\" (UniqueName: \"kubernetes.io/projected/04ad7b19-29ef-46b9-8668-b5437b6c88cc-kube-api-access-p6x5s\") pod \"nmstate-console-plugin-7fbb5f6569-br7vf\" (UID: \"04ad7b19-29ef-46b9-8668-b5437b6c88cc\") " pod="openshift-nmstate/nmstate-console-plugin-7fbb5f6569-br7vf" Dec 03 11:32:14 crc kubenswrapper[5052]: I1203 11:32:14.402325 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d058b2ea-5596-4f8a-b623-a65c7aded545-service-ca\") pod \"console-55fbcbdddd-nmbpx\" (UID: \"d058b2ea-5596-4f8a-b623-a65c7aded545\") " pod="openshift-console/console-55fbcbdddd-nmbpx" Dec 03 11:32:14 crc kubenswrapper[5052]: I1203 11:32:14.402650 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d058b2ea-5596-4f8a-b623-a65c7aded545-oauth-serving-cert\") pod \"console-55fbcbdddd-nmbpx\" (UID: \"d058b2ea-5596-4f8a-b623-a65c7aded545\") " pod="openshift-console/console-55fbcbdddd-nmbpx" Dec 03 11:32:14 crc kubenswrapper[5052]: I1203 11:32:14.402712 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d058b2ea-5596-4f8a-b623-a65c7aded545-console-config\") pod \"console-55fbcbdddd-nmbpx\" (UID: \"d058b2ea-5596-4f8a-b623-a65c7aded545\") " pod="openshift-console/console-55fbcbdddd-nmbpx" Dec 03 11:32:14 crc kubenswrapper[5052]: I1203 11:32:14.402739 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d058b2ea-5596-4f8a-b623-a65c7aded545-trusted-ca-bundle\") pod \"console-55fbcbdddd-nmbpx\" (UID: \"d058b2ea-5596-4f8a-b623-a65c7aded545\") " pod="openshift-console/console-55fbcbdddd-nmbpx" Dec 03 11:32:14 crc kubenswrapper[5052]: I1203 11:32:14.402768 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d058b2ea-5596-4f8a-b623-a65c7aded545-console-oauth-config\") pod \"console-55fbcbdddd-nmbpx\" (UID: \"d058b2ea-5596-4f8a-b623-a65c7aded545\") " pod="openshift-console/console-55fbcbdddd-nmbpx" Dec 03 11:32:14 crc kubenswrapper[5052]: I1203 11:32:14.402800 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d058b2ea-5596-4f8a-b623-a65c7aded545-console-serving-cert\") pod \"console-55fbcbdddd-nmbpx\" (UID: \"d058b2ea-5596-4f8a-b623-a65c7aded545\") " pod="openshift-console/console-55fbcbdddd-nmbpx" Dec 03 11:32:14 crc kubenswrapper[5052]: I1203 11:32:14.402829 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5mjdt\" (UniqueName: \"kubernetes.io/projected/d058b2ea-5596-4f8a-b623-a65c7aded545-kube-api-access-5mjdt\") pod \"console-55fbcbdddd-nmbpx\" (UID: \"d058b2ea-5596-4f8a-b623-a65c7aded545\") " pod="openshift-console/console-55fbcbdddd-nmbpx" Dec 03 11:32:14 crc kubenswrapper[5052]: I1203 11:32:14.420918 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-nmstate/nmstate-console-plugin-7fbb5f6569-br7vf" Dec 03 11:32:14 crc kubenswrapper[5052]: I1203 11:32:14.546012 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-5mjdt\" (UniqueName: \"kubernetes.io/projected/d058b2ea-5596-4f8a-b623-a65c7aded545-kube-api-access-5mjdt\") pod \"console-55fbcbdddd-nmbpx\" (UID: \"d058b2ea-5596-4f8a-b623-a65c7aded545\") " pod="openshift-console/console-55fbcbdddd-nmbpx" Dec 03 11:32:14 crc kubenswrapper[5052]: I1203 11:32:14.546085 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d058b2ea-5596-4f8a-b623-a65c7aded545-service-ca\") pod \"console-55fbcbdddd-nmbpx\" (UID: \"d058b2ea-5596-4f8a-b623-a65c7aded545\") " pod="openshift-console/console-55fbcbdddd-nmbpx" Dec 03 11:32:14 crc kubenswrapper[5052]: I1203 11:32:14.546141 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d058b2ea-5596-4f8a-b623-a65c7aded545-oauth-serving-cert\") pod \"console-55fbcbdddd-nmbpx\" (UID: \"d058b2ea-5596-4f8a-b623-a65c7aded545\") " pod="openshift-console/console-55fbcbdddd-nmbpx" Dec 03 11:32:14 crc kubenswrapper[5052]: I1203 11:32:14.546205 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d058b2ea-5596-4f8a-b623-a65c7aded545-console-config\") pod \"console-55fbcbdddd-nmbpx\" (UID: \"d058b2ea-5596-4f8a-b623-a65c7aded545\") " pod="openshift-console/console-55fbcbdddd-nmbpx" Dec 03 11:32:14 crc kubenswrapper[5052]: I1203 11:32:14.546233 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d058b2ea-5596-4f8a-b623-a65c7aded545-trusted-ca-bundle\") pod \"console-55fbcbdddd-nmbpx\" (UID: \"d058b2ea-5596-4f8a-b623-a65c7aded545\") " pod="openshift-console/console-55fbcbdddd-nmbpx" Dec 03 11:32:14 crc kubenswrapper[5052]: I1203 11:32:14.546270 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d058b2ea-5596-4f8a-b623-a65c7aded545-console-oauth-config\") pod \"console-55fbcbdddd-nmbpx\" (UID: \"d058b2ea-5596-4f8a-b623-a65c7aded545\") " pod="openshift-console/console-55fbcbdddd-nmbpx" Dec 03 11:32:14 crc kubenswrapper[5052]: I1203 11:32:14.546309 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d058b2ea-5596-4f8a-b623-a65c7aded545-console-serving-cert\") pod \"console-55fbcbdddd-nmbpx\" (UID: \"d058b2ea-5596-4f8a-b623-a65c7aded545\") " pod="openshift-console/console-55fbcbdddd-nmbpx" Dec 03 11:32:14 crc kubenswrapper[5052]: I1203 11:32:14.547624 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d058b2ea-5596-4f8a-b623-a65c7aded545-console-config\") pod \"console-55fbcbdddd-nmbpx\" (UID: \"d058b2ea-5596-4f8a-b623-a65c7aded545\") " pod="openshift-console/console-55fbcbdddd-nmbpx" Dec 03 11:32:14 crc kubenswrapper[5052]: I1203 11:32:14.547976 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d058b2ea-5596-4f8a-b623-a65c7aded545-oauth-serving-cert\") pod \"console-55fbcbdddd-nmbpx\" (UID: \"d058b2ea-5596-4f8a-b623-a65c7aded545\") " pod="openshift-console/console-55fbcbdddd-nmbpx" Dec 03 11:32:14 crc kubenswrapper[5052]: I1203 11:32:14.549832 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d058b2ea-5596-4f8a-b623-a65c7aded545-service-ca\") pod \"console-55fbcbdddd-nmbpx\" (UID: \"d058b2ea-5596-4f8a-b623-a65c7aded545\") " pod="openshift-console/console-55fbcbdddd-nmbpx" Dec 03 11:32:14 crc kubenswrapper[5052]: I1203 11:32:14.551004 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d058b2ea-5596-4f8a-b623-a65c7aded545-trusted-ca-bundle\") pod \"console-55fbcbdddd-nmbpx\" (UID: \"d058b2ea-5596-4f8a-b623-a65c7aded545\") " pod="openshift-console/console-55fbcbdddd-nmbpx" Dec 03 11:32:14 crc kubenswrapper[5052]: I1203 11:32:14.551755 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d058b2ea-5596-4f8a-b623-a65c7aded545-console-serving-cert\") pod \"console-55fbcbdddd-nmbpx\" (UID: \"d058b2ea-5596-4f8a-b623-a65c7aded545\") " pod="openshift-console/console-55fbcbdddd-nmbpx" Dec 03 11:32:14 crc kubenswrapper[5052]: I1203 11:32:14.553464 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d058b2ea-5596-4f8a-b623-a65c7aded545-console-oauth-config\") pod \"console-55fbcbdddd-nmbpx\" (UID: \"d058b2ea-5596-4f8a-b623-a65c7aded545\") " pod="openshift-console/console-55fbcbdddd-nmbpx" Dec 03 11:32:14 crc kubenswrapper[5052]: I1203 11:32:14.566954 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-5mjdt\" (UniqueName: \"kubernetes.io/projected/d058b2ea-5596-4f8a-b623-a65c7aded545-kube-api-access-5mjdt\") pod \"console-55fbcbdddd-nmbpx\" (UID: \"d058b2ea-5596-4f8a-b623-a65c7aded545\") " pod="openshift-console/console-55fbcbdddd-nmbpx" Dec 03 11:32:14 crc kubenswrapper[5052]: I1203 11:32:14.580432 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-nmstate/nmstate-metrics-7f946cbc9-ffkk9"] Dec 03 11:32:14 crc kubenswrapper[5052]: W1203 11:32:14.584980 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbdc64cdc_4e09_4eff_b1bc_23b9dd86ab8b.slice/crio-a00b3e9ce71cf82a5faa6eab167d717db9495a2be43b75d29f3703ed60aa337b WatchSource:0}: Error finding container a00b3e9ce71cf82a5faa6eab167d717db9495a2be43b75d29f3703ed60aa337b: Status 404 returned error can't find the container with id a00b3e9ce71cf82a5faa6eab167d717db9495a2be43b75d29f3703ed60aa337b Dec 03 11:32:14 crc kubenswrapper[5052]: I1203 11:32:14.647347 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"tls-key-pair\" (UniqueName: \"kubernetes.io/secret/f60b6277-113c-459c-801b-aa5c94de641a-tls-key-pair\") pod \"nmstate-webhook-5f6d4c5ccb-gbkms\" (UID: \"f60b6277-113c-459c-801b-aa5c94de641a\") " pod="openshift-nmstate/nmstate-webhook-5f6d4c5ccb-gbkms" Dec 03 11:32:14 crc kubenswrapper[5052]: I1203 11:32:14.651165 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"tls-key-pair\" (UniqueName: \"kubernetes.io/secret/f60b6277-113c-459c-801b-aa5c94de641a-tls-key-pair\") pod \"nmstate-webhook-5f6d4c5ccb-gbkms\" (UID: \"f60b6277-113c-459c-801b-aa5c94de641a\") " pod="openshift-nmstate/nmstate-webhook-5f6d4c5ccb-gbkms" Dec 03 11:32:14 crc kubenswrapper[5052]: I1203 11:32:14.651425 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-55fbcbdddd-nmbpx" Dec 03 11:32:14 crc kubenswrapper[5052]: I1203 11:32:14.670072 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-nmstate/nmstate-console-plugin-7fbb5f6569-br7vf"] Dec 03 11:32:14 crc kubenswrapper[5052]: W1203 11:32:14.677769 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04ad7b19_29ef_46b9_8668_b5437b6c88cc.slice/crio-5add47f39d14754ce157d27975b1db2fb7022c82cec9a65d53f029d63e246f1d WatchSource:0}: Error finding container 5add47f39d14754ce157d27975b1db2fb7022c82cec9a65d53f029d63e246f1d: Status 404 returned error can't find the container with id 5add47f39d14754ce157d27975b1db2fb7022c82cec9a65d53f029d63e246f1d Dec 03 11:32:14 crc kubenswrapper[5052]: W1203 11:32:14.842004 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd058b2ea_5596_4f8a_b623_a65c7aded545.slice/crio-0bec31aea232845de2d505762c2e49b82d349dd122282783647aa999a824c70b WatchSource:0}: Error finding container 0bec31aea232845de2d505762c2e49b82d349dd122282783647aa999a824c70b: Status 404 returned error can't find the container with id 0bec31aea232845de2d505762c2e49b82d349dd122282783647aa999a824c70b Dec 03 11:32:14 crc kubenswrapper[5052]: I1203 11:32:14.843641 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-55fbcbdddd-nmbpx"] Dec 03 11:32:14 crc kubenswrapper[5052]: I1203 11:32:14.890488 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-nmstate/nmstate-webhook-5f6d4c5ccb-gbkms" Dec 03 11:32:15 crc kubenswrapper[5052]: I1203 11:32:15.080631 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-nmstate/nmstate-webhook-5f6d4c5ccb-gbkms"] Dec 03 11:32:15 crc kubenswrapper[5052]: I1203 11:32:15.216171 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-55fbcbdddd-nmbpx" event={"ID":"d058b2ea-5596-4f8a-b623-a65c7aded545","Type":"ContainerStarted","Data":"5479cd515e66cb70ce0be484d258f71375763253b24a2542b39a5b7ad70171f7"} Dec 03 11:32:15 crc kubenswrapper[5052]: I1203 11:32:15.216404 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-55fbcbdddd-nmbpx" event={"ID":"d058b2ea-5596-4f8a-b623-a65c7aded545","Type":"ContainerStarted","Data":"0bec31aea232845de2d505762c2e49b82d349dd122282783647aa999a824c70b"} Dec 03 11:32:15 crc kubenswrapper[5052]: I1203 11:32:15.217008 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-nmstate/nmstate-metrics-7f946cbc9-ffkk9" event={"ID":"bdc64cdc-4e09-4eff-b1bc-23b9dd86ab8b","Type":"ContainerStarted","Data":"a00b3e9ce71cf82a5faa6eab167d717db9495a2be43b75d29f3703ed60aa337b"} Dec 03 11:32:15 crc kubenswrapper[5052]: I1203 11:32:15.217823 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-nmstate/nmstate-webhook-5f6d4c5ccb-gbkms" event={"ID":"f60b6277-113c-459c-801b-aa5c94de641a","Type":"ContainerStarted","Data":"2ea601b6450acccedef61a3b9b6988e3225d8a49fcc015a30a73146cd2bf22a4"} Dec 03 11:32:15 crc kubenswrapper[5052]: I1203 11:32:15.218678 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-nmstate/nmstate-handler-qggff" event={"ID":"318304dc-2b11-480e-8683-7627f288faa8","Type":"ContainerStarted","Data":"5ca954adefa92f000a9c4257bc19f194b2dc99b8e70470760b448666a59b010f"} Dec 03 11:32:15 crc kubenswrapper[5052]: I1203 11:32:15.219373 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-nmstate/nmstate-console-plugin-7fbb5f6569-br7vf" event={"ID":"04ad7b19-29ef-46b9-8668-b5437b6c88cc","Type":"ContainerStarted","Data":"5add47f39d14754ce157d27975b1db2fb7022c82cec9a65d53f029d63e246f1d"} Dec 03 11:32:18 crc kubenswrapper[5052]: I1203 11:32:18.243084 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-nmstate/nmstate-webhook-5f6d4c5ccb-gbkms" event={"ID":"f60b6277-113c-459c-801b-aa5c94de641a","Type":"ContainerStarted","Data":"2f893b3f576b3ed6a9e5616d400fecf4d259c5bf193f9c6c807996d6a23c52f5"} Dec 03 11:32:18 crc kubenswrapper[5052]: I1203 11:32:18.243980 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openshift-nmstate/nmstate-webhook-5f6d4c5ccb-gbkms" Dec 03 11:32:18 crc kubenswrapper[5052]: I1203 11:32:18.246072 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-nmstate/nmstate-handler-qggff" event={"ID":"318304dc-2b11-480e-8683-7627f288faa8","Type":"ContainerStarted","Data":"e2de9e4331461a30be81d0abe60de4fe17a8b2cca47cea8c223f4fba500916d4"} Dec 03 11:32:18 crc kubenswrapper[5052]: I1203 11:32:18.246168 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openshift-nmstate/nmstate-handler-qggff" Dec 03 11:32:18 crc kubenswrapper[5052]: I1203 11:32:18.248390 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-nmstate/nmstate-console-plugin-7fbb5f6569-br7vf" event={"ID":"04ad7b19-29ef-46b9-8668-b5437b6c88cc","Type":"ContainerStarted","Data":"0fb3f84b834eb9a5859365e7a4502ac209cb6a1c46dc750e78f5c514af15d4f8"} Dec 03 11:32:18 crc kubenswrapper[5052]: I1203 11:32:18.255039 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-nmstate/nmstate-metrics-7f946cbc9-ffkk9" event={"ID":"bdc64cdc-4e09-4eff-b1bc-23b9dd86ab8b","Type":"ContainerStarted","Data":"45a9a573aef93222fbb45d2061c45b37eacc33761635adf1e879d666a2e098e8"} Dec 03 11:32:18 crc kubenswrapper[5052]: I1203 11:32:18.266986 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/console-55fbcbdddd-nmbpx" podStartSLOduration=4.266967435 podStartE2EDuration="4.266967435s" podCreationTimestamp="2025-12-03 11:32:14 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:32:15.238285634 +0000 UTC m=+725.826527398" watchObservedRunningTime="2025-12-03 11:32:18.266967435 +0000 UTC m=+728.855209199" Dec 03 11:32:18 crc kubenswrapper[5052]: I1203 11:32:18.268128 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-nmstate/nmstate-webhook-5f6d4c5ccb-gbkms" podStartSLOduration=2.497774289 podStartE2EDuration="5.268119588s" podCreationTimestamp="2025-12-03 11:32:13 +0000 UTC" firstStartedPulling="2025-12-03 11:32:15.087767557 +0000 UTC m=+725.676009321" lastFinishedPulling="2025-12-03 11:32:17.858112836 +0000 UTC m=+728.446354620" observedRunningTime="2025-12-03 11:32:18.264313149 +0000 UTC m=+728.852554933" watchObservedRunningTime="2025-12-03 11:32:18.268119588 +0000 UTC m=+728.856361342" Dec 03 11:32:18 crc kubenswrapper[5052]: I1203 11:32:18.309299 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-nmstate/nmstate-console-plugin-7fbb5f6569-br7vf" podStartSLOduration=1.179297226 podStartE2EDuration="4.309268965s" podCreationTimestamp="2025-12-03 11:32:14 +0000 UTC" firstStartedPulling="2025-12-03 11:32:14.68211604 +0000 UTC m=+725.270357804" lastFinishedPulling="2025-12-03 11:32:17.812087779 +0000 UTC m=+728.400329543" observedRunningTime="2025-12-03 11:32:18.289228702 +0000 UTC m=+728.877470476" watchObservedRunningTime="2025-12-03 11:32:18.309268965 +0000 UTC m=+728.897510749" Dec 03 11:32:18 crc kubenswrapper[5052]: I1203 11:32:18.324786 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-nmstate/nmstate-handler-qggff" podStartSLOduration=1.8804810349999999 podStartE2EDuration="5.324755438s" podCreationTimestamp="2025-12-03 11:32:13 +0000 UTC" firstStartedPulling="2025-12-03 11:32:14.366857699 +0000 UTC m=+724.955099463" lastFinishedPulling="2025-12-03 11:32:17.811132102 +0000 UTC m=+728.399373866" observedRunningTime="2025-12-03 11:32:18.30908899 +0000 UTC m=+728.897330754" watchObservedRunningTime="2025-12-03 11:32:18.324755438 +0000 UTC m=+728.912997202" Dec 03 11:32:21 crc kubenswrapper[5052]: I1203 11:32:21.276736 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-nmstate/nmstate-metrics-7f946cbc9-ffkk9" event={"ID":"bdc64cdc-4e09-4eff-b1bc-23b9dd86ab8b","Type":"ContainerStarted","Data":"8c11846c63889c4f8832022c63a0fd84072759a90e66f5f619ff5a25bd376ff7"} Dec 03 11:32:21 crc kubenswrapper[5052]: I1203 11:32:21.295392 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-nmstate/nmstate-metrics-7f946cbc9-ffkk9" podStartSLOduration=2.745245469 podStartE2EDuration="8.295363628s" podCreationTimestamp="2025-12-03 11:32:13 +0000 UTC" firstStartedPulling="2025-12-03 11:32:14.58743018 +0000 UTC m=+725.175671944" lastFinishedPulling="2025-12-03 11:32:20.137548339 +0000 UTC m=+730.725790103" observedRunningTime="2025-12-03 11:32:21.293104074 +0000 UTC m=+731.881345918" watchObservedRunningTime="2025-12-03 11:32:21.295363628 +0000 UTC m=+731.883605422" Dec 03 11:32:24 crc kubenswrapper[5052]: I1203 11:32:24.361396 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-nmstate/nmstate-handler-qggff" Dec 03 11:32:24 crc kubenswrapper[5052]: I1203 11:32:24.652448 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openshift-console/console-55fbcbdddd-nmbpx" Dec 03 11:32:24 crc kubenswrapper[5052]: I1203 11:32:24.652539 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-55fbcbdddd-nmbpx" Dec 03 11:32:24 crc kubenswrapper[5052]: I1203 11:32:24.657660 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-55fbcbdddd-nmbpx" Dec 03 11:32:25 crc kubenswrapper[5052]: I1203 11:32:25.304386 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-55fbcbdddd-nmbpx" Dec 03 11:32:25 crc kubenswrapper[5052]: I1203 11:32:25.366999 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openshift-console/console-f9d7485db-568dj"] Dec 03 11:32:29 crc kubenswrapper[5052]: I1203 11:32:29.379737 5052 patch_prober.go:28] interesting pod/machine-config-daemon-74bj6 container/machine-config-daemon namespace/openshift-machine-config-operator: Liveness probe status=failure output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" start-of-body= Dec 03 11:32:29 crc kubenswrapper[5052]: I1203 11:32:29.380076 5052 prober.go:107] "Probe failed" probeType="Liveness" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" containerName="machine-config-daemon" probeResult="failure" output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" Dec 03 11:32:34 crc kubenswrapper[5052]: I1203 11:32:34.898508 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-nmstate/nmstate-webhook-5f6d4c5ccb-gbkms" Dec 03 11:32:47 crc kubenswrapper[5052]: I1203 11:32:47.438683 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openshift-marketplace/af69698b82fdf008f5ff9e195cf808a654240e16b13dcd924b74994f83dzz4m"] Dec 03 11:32:47 crc kubenswrapper[5052]: I1203 11:32:47.440132 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/af69698b82fdf008f5ff9e195cf808a654240e16b13dcd924b74994f83dzz4m" Dec 03 11:32:47 crc kubenswrapper[5052]: I1203 11:32:47.443207 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-marketplace"/"default-dockercfg-vmwhc" Dec 03 11:32:47 crc kubenswrapper[5052]: I1203 11:32:47.450122 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/af69698b82fdf008f5ff9e195cf808a654240e16b13dcd924b74994f83dzz4m"] Dec 03 11:32:47 crc kubenswrapper[5052]: I1203 11:32:47.613801 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/aeaaf048-0037-4550-b1f5-54cfe153dcd2-util\") pod \"af69698b82fdf008f5ff9e195cf808a654240e16b13dcd924b74994f83dzz4m\" (UID: \"aeaaf048-0037-4550-b1f5-54cfe153dcd2\") " pod="openshift-marketplace/af69698b82fdf008f5ff9e195cf808a654240e16b13dcd924b74994f83dzz4m" Dec 03 11:32:47 crc kubenswrapper[5052]: I1203 11:32:47.613859 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vfdbc\" (UniqueName: \"kubernetes.io/projected/aeaaf048-0037-4550-b1f5-54cfe153dcd2-kube-api-access-vfdbc\") pod \"af69698b82fdf008f5ff9e195cf808a654240e16b13dcd924b74994f83dzz4m\" (UID: \"aeaaf048-0037-4550-b1f5-54cfe153dcd2\") " pod="openshift-marketplace/af69698b82fdf008f5ff9e195cf808a654240e16b13dcd924b74994f83dzz4m" Dec 03 11:32:47 crc kubenswrapper[5052]: I1203 11:32:47.613891 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/aeaaf048-0037-4550-b1f5-54cfe153dcd2-bundle\") pod \"af69698b82fdf008f5ff9e195cf808a654240e16b13dcd924b74994f83dzz4m\" (UID: \"aeaaf048-0037-4550-b1f5-54cfe153dcd2\") " pod="openshift-marketplace/af69698b82fdf008f5ff9e195cf808a654240e16b13dcd924b74994f83dzz4m" Dec 03 11:32:47 crc kubenswrapper[5052]: I1203 11:32:47.716309 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/aeaaf048-0037-4550-b1f5-54cfe153dcd2-bundle\") pod \"af69698b82fdf008f5ff9e195cf808a654240e16b13dcd924b74994f83dzz4m\" (UID: \"aeaaf048-0037-4550-b1f5-54cfe153dcd2\") " pod="openshift-marketplace/af69698b82fdf008f5ff9e195cf808a654240e16b13dcd924b74994f83dzz4m" Dec 03 11:32:47 crc kubenswrapper[5052]: I1203 11:32:47.716635 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/aeaaf048-0037-4550-b1f5-54cfe153dcd2-util\") pod \"af69698b82fdf008f5ff9e195cf808a654240e16b13dcd924b74994f83dzz4m\" (UID: \"aeaaf048-0037-4550-b1f5-54cfe153dcd2\") " pod="openshift-marketplace/af69698b82fdf008f5ff9e195cf808a654240e16b13dcd924b74994f83dzz4m" Dec 03 11:32:47 crc kubenswrapper[5052]: I1203 11:32:47.716700 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-vfdbc\" (UniqueName: \"kubernetes.io/projected/aeaaf048-0037-4550-b1f5-54cfe153dcd2-kube-api-access-vfdbc\") pod \"af69698b82fdf008f5ff9e195cf808a654240e16b13dcd924b74994f83dzz4m\" (UID: \"aeaaf048-0037-4550-b1f5-54cfe153dcd2\") " pod="openshift-marketplace/af69698b82fdf008f5ff9e195cf808a654240e16b13dcd924b74994f83dzz4m" Dec 03 11:32:47 crc kubenswrapper[5052]: I1203 11:32:47.717239 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/aeaaf048-0037-4550-b1f5-54cfe153dcd2-bundle\") pod \"af69698b82fdf008f5ff9e195cf808a654240e16b13dcd924b74994f83dzz4m\" (UID: \"aeaaf048-0037-4550-b1f5-54cfe153dcd2\") " pod="openshift-marketplace/af69698b82fdf008f5ff9e195cf808a654240e16b13dcd924b74994f83dzz4m" Dec 03 11:32:47 crc kubenswrapper[5052]: I1203 11:32:47.717260 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/aeaaf048-0037-4550-b1f5-54cfe153dcd2-util\") pod \"af69698b82fdf008f5ff9e195cf808a654240e16b13dcd924b74994f83dzz4m\" (UID: \"aeaaf048-0037-4550-b1f5-54cfe153dcd2\") " pod="openshift-marketplace/af69698b82fdf008f5ff9e195cf808a654240e16b13dcd924b74994f83dzz4m" Dec 03 11:32:47 crc kubenswrapper[5052]: I1203 11:32:47.737778 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-vfdbc\" (UniqueName: \"kubernetes.io/projected/aeaaf048-0037-4550-b1f5-54cfe153dcd2-kube-api-access-vfdbc\") pod \"af69698b82fdf008f5ff9e195cf808a654240e16b13dcd924b74994f83dzz4m\" (UID: \"aeaaf048-0037-4550-b1f5-54cfe153dcd2\") " pod="openshift-marketplace/af69698b82fdf008f5ff9e195cf808a654240e16b13dcd924b74994f83dzz4m" Dec 03 11:32:47 crc kubenswrapper[5052]: I1203 11:32:47.761090 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/af69698b82fdf008f5ff9e195cf808a654240e16b13dcd924b74994f83dzz4m" Dec 03 11:32:47 crc kubenswrapper[5052]: I1203 11:32:47.999094 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/af69698b82fdf008f5ff9e195cf808a654240e16b13dcd924b74994f83dzz4m"] Dec 03 11:32:48 crc kubenswrapper[5052]: I1203 11:32:48.471119 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/af69698b82fdf008f5ff9e195cf808a654240e16b13dcd924b74994f83dzz4m" event={"ID":"aeaaf048-0037-4550-b1f5-54cfe153dcd2","Type":"ContainerStarted","Data":"7963c2f1b03aed925b5502313f61b412edede70836e1b082fdf91ec54daa12f2"} Dec 03 11:32:48 crc kubenswrapper[5052]: I1203 11:32:48.471631 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/af69698b82fdf008f5ff9e195cf808a654240e16b13dcd924b74994f83dzz4m" event={"ID":"aeaaf048-0037-4550-b1f5-54cfe153dcd2","Type":"ContainerStarted","Data":"f780ac761682fe1a0511a6261ab36fe6e6938264407b9ea2d6c9037007bf6b7f"} Dec 03 11:32:49 crc kubenswrapper[5052]: I1203 11:32:49.477788 5052 generic.go:334] "Generic (PLEG): container finished" podID="aeaaf048-0037-4550-b1f5-54cfe153dcd2" containerID="7963c2f1b03aed925b5502313f61b412edede70836e1b082fdf91ec54daa12f2" exitCode=0 Dec 03 11:32:49 crc kubenswrapper[5052]: I1203 11:32:49.477837 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/af69698b82fdf008f5ff9e195cf808a654240e16b13dcd924b74994f83dzz4m" event={"ID":"aeaaf048-0037-4550-b1f5-54cfe153dcd2","Type":"ContainerDied","Data":"7963c2f1b03aed925b5502313f61b412edede70836e1b082fdf91ec54daa12f2"} Dec 03 11:32:50 crc kubenswrapper[5052]: I1203 11:32:50.435281 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openshift-console/console-f9d7485db-568dj" podUID="b1d2ff08-d58b-48d0-a0e5-6cfd867a63de" containerName="console" containerID="cri-o://26c41ad67f1bd2c1f945f4259e4ebfeaef4d6c83d4ff3d8f2f1ec262785569b6" gracePeriod=15 Dec 03 11:32:51 crc kubenswrapper[5052]: I1203 11:32:51.040242 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-f9d7485db-568dj_b1d2ff08-d58b-48d0-a0e5-6cfd867a63de/console/0.log" Dec 03 11:32:51 crc kubenswrapper[5052]: I1203 11:32:51.040481 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-f9d7485db-568dj" Dec 03 11:32:51 crc kubenswrapper[5052]: I1203 11:32:51.160869 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b1d2ff08-d58b-48d0-a0e5-6cfd867a63de-console-oauth-config\") pod \"b1d2ff08-d58b-48d0-a0e5-6cfd867a63de\" (UID: \"b1d2ff08-d58b-48d0-a0e5-6cfd867a63de\") " Dec 03 11:32:51 crc kubenswrapper[5052]: I1203 11:32:51.160994 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b1d2ff08-d58b-48d0-a0e5-6cfd867a63de-service-ca\") pod \"b1d2ff08-d58b-48d0-a0e5-6cfd867a63de\" (UID: \"b1d2ff08-d58b-48d0-a0e5-6cfd867a63de\") " Dec 03 11:32:51 crc kubenswrapper[5052]: I1203 11:32:51.161057 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w4cbl\" (UniqueName: \"kubernetes.io/projected/b1d2ff08-d58b-48d0-a0e5-6cfd867a63de-kube-api-access-w4cbl\") pod \"b1d2ff08-d58b-48d0-a0e5-6cfd867a63de\" (UID: \"b1d2ff08-d58b-48d0-a0e5-6cfd867a63de\") " Dec 03 11:32:51 crc kubenswrapper[5052]: I1203 11:32:51.161098 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b1d2ff08-d58b-48d0-a0e5-6cfd867a63de-console-config\") pod \"b1d2ff08-d58b-48d0-a0e5-6cfd867a63de\" (UID: \"b1d2ff08-d58b-48d0-a0e5-6cfd867a63de\") " Dec 03 11:32:51 crc kubenswrapper[5052]: I1203 11:32:51.161160 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b1d2ff08-d58b-48d0-a0e5-6cfd867a63de-console-serving-cert\") pod \"b1d2ff08-d58b-48d0-a0e5-6cfd867a63de\" (UID: \"b1d2ff08-d58b-48d0-a0e5-6cfd867a63de\") " Dec 03 11:32:51 crc kubenswrapper[5052]: I1203 11:32:51.161186 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b1d2ff08-d58b-48d0-a0e5-6cfd867a63de-oauth-serving-cert\") pod \"b1d2ff08-d58b-48d0-a0e5-6cfd867a63de\" (UID: \"b1d2ff08-d58b-48d0-a0e5-6cfd867a63de\") " Dec 03 11:32:51 crc kubenswrapper[5052]: I1203 11:32:51.161254 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b1d2ff08-d58b-48d0-a0e5-6cfd867a63de-trusted-ca-bundle\") pod \"b1d2ff08-d58b-48d0-a0e5-6cfd867a63de\" (UID: \"b1d2ff08-d58b-48d0-a0e5-6cfd867a63de\") " Dec 03 11:32:51 crc kubenswrapper[5052]: I1203 11:32:51.161684 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b1d2ff08-d58b-48d0-a0e5-6cfd867a63de-console-config" (OuterVolumeSpecName: "console-config") pod "b1d2ff08-d58b-48d0-a0e5-6cfd867a63de" (UID: "b1d2ff08-d58b-48d0-a0e5-6cfd867a63de"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:32:51 crc kubenswrapper[5052]: I1203 11:32:51.161849 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b1d2ff08-d58b-48d0-a0e5-6cfd867a63de-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "b1d2ff08-d58b-48d0-a0e5-6cfd867a63de" (UID: "b1d2ff08-d58b-48d0-a0e5-6cfd867a63de"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:32:51 crc kubenswrapper[5052]: I1203 11:32:51.161737 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b1d2ff08-d58b-48d0-a0e5-6cfd867a63de-service-ca" (OuterVolumeSpecName: "service-ca") pod "b1d2ff08-d58b-48d0-a0e5-6cfd867a63de" (UID: "b1d2ff08-d58b-48d0-a0e5-6cfd867a63de"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:32:51 crc kubenswrapper[5052]: I1203 11:32:51.161990 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b1d2ff08-d58b-48d0-a0e5-6cfd867a63de-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "b1d2ff08-d58b-48d0-a0e5-6cfd867a63de" (UID: "b1d2ff08-d58b-48d0-a0e5-6cfd867a63de"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:32:51 crc kubenswrapper[5052]: I1203 11:32:51.167563 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b1d2ff08-d58b-48d0-a0e5-6cfd867a63de-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "b1d2ff08-d58b-48d0-a0e5-6cfd867a63de" (UID: "b1d2ff08-d58b-48d0-a0e5-6cfd867a63de"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:32:51 crc kubenswrapper[5052]: I1203 11:32:51.168371 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b1d2ff08-d58b-48d0-a0e5-6cfd867a63de-kube-api-access-w4cbl" (OuterVolumeSpecName: "kube-api-access-w4cbl") pod "b1d2ff08-d58b-48d0-a0e5-6cfd867a63de" (UID: "b1d2ff08-d58b-48d0-a0e5-6cfd867a63de"). InnerVolumeSpecName "kube-api-access-w4cbl". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:32:51 crc kubenswrapper[5052]: I1203 11:32:51.168955 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b1d2ff08-d58b-48d0-a0e5-6cfd867a63de-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "b1d2ff08-d58b-48d0-a0e5-6cfd867a63de" (UID: "b1d2ff08-d58b-48d0-a0e5-6cfd867a63de"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:32:51 crc kubenswrapper[5052]: I1203 11:32:51.263802 5052 reconciler_common.go:293] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b1d2ff08-d58b-48d0-a0e5-6cfd867a63de-trusted-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:32:51 crc kubenswrapper[5052]: I1203 11:32:51.263858 5052 reconciler_common.go:293] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b1d2ff08-d58b-48d0-a0e5-6cfd867a63de-console-oauth-config\") on node \"crc\" DevicePath \"\"" Dec 03 11:32:51 crc kubenswrapper[5052]: I1203 11:32:51.263869 5052 reconciler_common.go:293] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b1d2ff08-d58b-48d0-a0e5-6cfd867a63de-service-ca\") on node \"crc\" DevicePath \"\"" Dec 03 11:32:51 crc kubenswrapper[5052]: I1203 11:32:51.263882 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-w4cbl\" (UniqueName: \"kubernetes.io/projected/b1d2ff08-d58b-48d0-a0e5-6cfd867a63de-kube-api-access-w4cbl\") on node \"crc\" DevicePath \"\"" Dec 03 11:32:51 crc kubenswrapper[5052]: I1203 11:32:51.263898 5052 reconciler_common.go:293] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b1d2ff08-d58b-48d0-a0e5-6cfd867a63de-console-config\") on node \"crc\" DevicePath \"\"" Dec 03 11:32:51 crc kubenswrapper[5052]: I1203 11:32:51.263909 5052 reconciler_common.go:293] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b1d2ff08-d58b-48d0-a0e5-6cfd867a63de-console-serving-cert\") on node \"crc\" DevicePath \"\"" Dec 03 11:32:51 crc kubenswrapper[5052]: I1203 11:32:51.263924 5052 reconciler_common.go:293] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b1d2ff08-d58b-48d0-a0e5-6cfd867a63de-oauth-serving-cert\") on node \"crc\" DevicePath \"\"" Dec 03 11:32:51 crc kubenswrapper[5052]: I1203 11:32:51.504568 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/af69698b82fdf008f5ff9e195cf808a654240e16b13dcd924b74994f83dzz4m" event={"ID":"aeaaf048-0037-4550-b1f5-54cfe153dcd2","Type":"ContainerStarted","Data":"0abf607ba75de58f14c0e7d83ff85b608a6dc0aaac4b36809e250eda3b861bd0"} Dec 03 11:32:51 crc kubenswrapper[5052]: I1203 11:32:51.506483 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-f9d7485db-568dj_b1d2ff08-d58b-48d0-a0e5-6cfd867a63de/console/0.log" Dec 03 11:32:51 crc kubenswrapper[5052]: I1203 11:32:51.506688 5052 generic.go:334] "Generic (PLEG): container finished" podID="b1d2ff08-d58b-48d0-a0e5-6cfd867a63de" containerID="26c41ad67f1bd2c1f945f4259e4ebfeaef4d6c83d4ff3d8f2f1ec262785569b6" exitCode=2 Dec 03 11:32:51 crc kubenswrapper[5052]: I1203 11:32:51.506771 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-f9d7485db-568dj" Dec 03 11:32:51 crc kubenswrapper[5052]: I1203 11:32:51.506778 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-f9d7485db-568dj" event={"ID":"b1d2ff08-d58b-48d0-a0e5-6cfd867a63de","Type":"ContainerDied","Data":"26c41ad67f1bd2c1f945f4259e4ebfeaef4d6c83d4ff3d8f2f1ec262785569b6"} Dec 03 11:32:51 crc kubenswrapper[5052]: I1203 11:32:51.507746 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-f9d7485db-568dj" event={"ID":"b1d2ff08-d58b-48d0-a0e5-6cfd867a63de","Type":"ContainerDied","Data":"d5c5a4b903b9b98d3670cf389863431252c01e50350a0b60b093433302d7adea"} Dec 03 11:32:51 crc kubenswrapper[5052]: I1203 11:32:51.507780 5052 scope.go:117] "RemoveContainer" containerID="26c41ad67f1bd2c1f945f4259e4ebfeaef4d6c83d4ff3d8f2f1ec262785569b6" Dec 03 11:32:51 crc kubenswrapper[5052]: I1203 11:32:51.557827 5052 scope.go:117] "RemoveContainer" containerID="26c41ad67f1bd2c1f945f4259e4ebfeaef4d6c83d4ff3d8f2f1ec262785569b6" Dec 03 11:32:51 crc kubenswrapper[5052]: E1203 11:32:51.562742 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"26c41ad67f1bd2c1f945f4259e4ebfeaef4d6c83d4ff3d8f2f1ec262785569b6\": container with ID starting with 26c41ad67f1bd2c1f945f4259e4ebfeaef4d6c83d4ff3d8f2f1ec262785569b6 not found: ID does not exist" containerID="26c41ad67f1bd2c1f945f4259e4ebfeaef4d6c83d4ff3d8f2f1ec262785569b6" Dec 03 11:32:51 crc kubenswrapper[5052]: I1203 11:32:51.562811 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"26c41ad67f1bd2c1f945f4259e4ebfeaef4d6c83d4ff3d8f2f1ec262785569b6"} err="failed to get container status \"26c41ad67f1bd2c1f945f4259e4ebfeaef4d6c83d4ff3d8f2f1ec262785569b6\": rpc error: code = NotFound desc = could not find container \"26c41ad67f1bd2c1f945f4259e4ebfeaef4d6c83d4ff3d8f2f1ec262785569b6\": container with ID starting with 26c41ad67f1bd2c1f945f4259e4ebfeaef4d6c83d4ff3d8f2f1ec262785569b6 not found: ID does not exist" Dec 03 11:32:51 crc kubenswrapper[5052]: I1203 11:32:51.564878 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openshift-console/console-f9d7485db-568dj"] Dec 03 11:32:51 crc kubenswrapper[5052]: I1203 11:32:51.568849 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-f9d7485db-568dj"] Dec 03 11:32:51 crc kubenswrapper[5052]: I1203 11:32:51.681026 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b1d2ff08-d58b-48d0-a0e5-6cfd867a63de" path="/var/lib/kubelet/pods/b1d2ff08-d58b-48d0-a0e5-6cfd867a63de/volumes" Dec 03 11:32:52 crc kubenswrapper[5052]: I1203 11:32:52.516020 5052 generic.go:334] "Generic (PLEG): container finished" podID="aeaaf048-0037-4550-b1f5-54cfe153dcd2" containerID="0abf607ba75de58f14c0e7d83ff85b608a6dc0aaac4b36809e250eda3b861bd0" exitCode=0 Dec 03 11:32:52 crc kubenswrapper[5052]: I1203 11:32:52.516077 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/af69698b82fdf008f5ff9e195cf808a654240e16b13dcd924b74994f83dzz4m" event={"ID":"aeaaf048-0037-4550-b1f5-54cfe153dcd2","Type":"ContainerDied","Data":"0abf607ba75de58f14c0e7d83ff85b608a6dc0aaac4b36809e250eda3b861bd0"} Dec 03 11:32:53 crc kubenswrapper[5052]: I1203 11:32:53.524909 5052 generic.go:334] "Generic (PLEG): container finished" podID="aeaaf048-0037-4550-b1f5-54cfe153dcd2" containerID="a8589c44bdcbb1031a75d0e1387770875f3b54c30b68ef57db16174a7035c791" exitCode=0 Dec 03 11:32:53 crc kubenswrapper[5052]: I1203 11:32:53.524958 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/af69698b82fdf008f5ff9e195cf808a654240e16b13dcd924b74994f83dzz4m" event={"ID":"aeaaf048-0037-4550-b1f5-54cfe153dcd2","Type":"ContainerDied","Data":"a8589c44bdcbb1031a75d0e1387770875f3b54c30b68ef57db16174a7035c791"} Dec 03 11:32:54 crc kubenswrapper[5052]: I1203 11:32:54.758671 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/af69698b82fdf008f5ff9e195cf808a654240e16b13dcd924b74994f83dzz4m" Dec 03 11:32:54 crc kubenswrapper[5052]: I1203 11:32:54.912659 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/aeaaf048-0037-4550-b1f5-54cfe153dcd2-util\") pod \"aeaaf048-0037-4550-b1f5-54cfe153dcd2\" (UID: \"aeaaf048-0037-4550-b1f5-54cfe153dcd2\") " Dec 03 11:32:54 crc kubenswrapper[5052]: I1203 11:32:54.912790 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vfdbc\" (UniqueName: \"kubernetes.io/projected/aeaaf048-0037-4550-b1f5-54cfe153dcd2-kube-api-access-vfdbc\") pod \"aeaaf048-0037-4550-b1f5-54cfe153dcd2\" (UID: \"aeaaf048-0037-4550-b1f5-54cfe153dcd2\") " Dec 03 11:32:54 crc kubenswrapper[5052]: I1203 11:32:54.912826 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/aeaaf048-0037-4550-b1f5-54cfe153dcd2-bundle\") pod \"aeaaf048-0037-4550-b1f5-54cfe153dcd2\" (UID: \"aeaaf048-0037-4550-b1f5-54cfe153dcd2\") " Dec 03 11:32:54 crc kubenswrapper[5052]: I1203 11:32:54.913755 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aeaaf048-0037-4550-b1f5-54cfe153dcd2-bundle" (OuterVolumeSpecName: "bundle") pod "aeaaf048-0037-4550-b1f5-54cfe153dcd2" (UID: "aeaaf048-0037-4550-b1f5-54cfe153dcd2"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:32:54 crc kubenswrapper[5052]: I1203 11:32:54.918145 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aeaaf048-0037-4550-b1f5-54cfe153dcd2-kube-api-access-vfdbc" (OuterVolumeSpecName: "kube-api-access-vfdbc") pod "aeaaf048-0037-4550-b1f5-54cfe153dcd2" (UID: "aeaaf048-0037-4550-b1f5-54cfe153dcd2"). InnerVolumeSpecName "kube-api-access-vfdbc". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:32:54 crc kubenswrapper[5052]: I1203 11:32:54.925961 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aeaaf048-0037-4550-b1f5-54cfe153dcd2-util" (OuterVolumeSpecName: "util") pod "aeaaf048-0037-4550-b1f5-54cfe153dcd2" (UID: "aeaaf048-0037-4550-b1f5-54cfe153dcd2"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:32:55 crc kubenswrapper[5052]: I1203 11:32:55.014215 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-vfdbc\" (UniqueName: \"kubernetes.io/projected/aeaaf048-0037-4550-b1f5-54cfe153dcd2-kube-api-access-vfdbc\") on node \"crc\" DevicePath \"\"" Dec 03 11:32:55 crc kubenswrapper[5052]: I1203 11:32:55.014254 5052 reconciler_common.go:293] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/aeaaf048-0037-4550-b1f5-54cfe153dcd2-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:32:55 crc kubenswrapper[5052]: I1203 11:32:55.014267 5052 reconciler_common.go:293] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/aeaaf048-0037-4550-b1f5-54cfe153dcd2-util\") on node \"crc\" DevicePath \"\"" Dec 03 11:32:55 crc kubenswrapper[5052]: I1203 11:32:55.537649 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/af69698b82fdf008f5ff9e195cf808a654240e16b13dcd924b74994f83dzz4m" event={"ID":"aeaaf048-0037-4550-b1f5-54cfe153dcd2","Type":"ContainerDied","Data":"f780ac761682fe1a0511a6261ab36fe6e6938264407b9ea2d6c9037007bf6b7f"} Dec 03 11:32:55 crc kubenswrapper[5052]: I1203 11:32:55.537704 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f780ac761682fe1a0511a6261ab36fe6e6938264407b9ea2d6c9037007bf6b7f" Dec 03 11:32:55 crc kubenswrapper[5052]: I1203 11:32:55.537788 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/af69698b82fdf008f5ff9e195cf808a654240e16b13dcd924b74994f83dzz4m" Dec 03 11:32:59 crc kubenswrapper[5052]: I1203 11:32:59.380315 5052 patch_prober.go:28] interesting pod/machine-config-daemon-74bj6 container/machine-config-daemon namespace/openshift-machine-config-operator: Liveness probe status=failure output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" start-of-body= Dec 03 11:32:59 crc kubenswrapper[5052]: I1203 11:32:59.380395 5052 prober.go:107] "Probe failed" probeType="Liveness" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" containerName="machine-config-daemon" probeResult="failure" output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" Dec 03 11:32:59 crc kubenswrapper[5052]: I1203 11:32:59.380445 5052 kubelet.go:2542] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" Dec 03 11:32:59 crc kubenswrapper[5052]: I1203 11:32:59.381126 5052 kuberuntime_manager.go:1027] "Message for Container of pod" containerName="machine-config-daemon" containerStatusID={"Type":"cri-o","ID":"a8ea894737897a85494be3992d10d75dae39643d0a55aa5bfc62a4269324bdcd"} pod="openshift-machine-config-operator/machine-config-daemon-74bj6" containerMessage="Container machine-config-daemon failed liveness probe, will be restarted" Dec 03 11:32:59 crc kubenswrapper[5052]: I1203 11:32:59.381195 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" containerName="machine-config-daemon" containerID="cri-o://a8ea894737897a85494be3992d10d75dae39643d0a55aa5bfc62a4269324bdcd" gracePeriod=600 Dec 03 11:33:00 crc kubenswrapper[5052]: I1203 11:33:00.566104 5052 generic.go:334] "Generic (PLEG): container finished" podID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" containerID="a8ea894737897a85494be3992d10d75dae39643d0a55aa5bfc62a4269324bdcd" exitCode=0 Dec 03 11:33:00 crc kubenswrapper[5052]: I1203 11:33:00.566188 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" event={"ID":"10c237bc-72e7-4da3-9f2c-b1688395bf6e","Type":"ContainerDied","Data":"a8ea894737897a85494be3992d10d75dae39643d0a55aa5bfc62a4269324bdcd"} Dec 03 11:33:00 crc kubenswrapper[5052]: I1203 11:33:00.566536 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" event={"ID":"10c237bc-72e7-4da3-9f2c-b1688395bf6e","Type":"ContainerStarted","Data":"f37a0f397f498a9429565aa0bada2c6757e002bd2d8af169475a45c6c645d73e"} Dec 03 11:33:00 crc kubenswrapper[5052]: I1203 11:33:00.566563 5052 scope.go:117] "RemoveContainer" containerID="2d08ec4ed134e9ab1a30c6d8ed980a535d93d2cc12d3d85579e39538d5086343" Dec 03 11:33:06 crc kubenswrapper[5052]: I1203 11:33:06.369372 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["metallb-system/metallb-operator-controller-manager-b9dd667db-j4qd7"] Dec 03 11:33:06 crc kubenswrapper[5052]: E1203 11:33:06.370088 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="aeaaf048-0037-4550-b1f5-54cfe153dcd2" containerName="extract" Dec 03 11:33:06 crc kubenswrapper[5052]: I1203 11:33:06.370100 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="aeaaf048-0037-4550-b1f5-54cfe153dcd2" containerName="extract" Dec 03 11:33:06 crc kubenswrapper[5052]: E1203 11:33:06.370120 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="aeaaf048-0037-4550-b1f5-54cfe153dcd2" containerName="pull" Dec 03 11:33:06 crc kubenswrapper[5052]: I1203 11:33:06.370125 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="aeaaf048-0037-4550-b1f5-54cfe153dcd2" containerName="pull" Dec 03 11:33:06 crc kubenswrapper[5052]: E1203 11:33:06.370135 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="aeaaf048-0037-4550-b1f5-54cfe153dcd2" containerName="util" Dec 03 11:33:06 crc kubenswrapper[5052]: I1203 11:33:06.370140 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="aeaaf048-0037-4550-b1f5-54cfe153dcd2" containerName="util" Dec 03 11:33:06 crc kubenswrapper[5052]: E1203 11:33:06.370150 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="b1d2ff08-d58b-48d0-a0e5-6cfd867a63de" containerName="console" Dec 03 11:33:06 crc kubenswrapper[5052]: I1203 11:33:06.370155 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="b1d2ff08-d58b-48d0-a0e5-6cfd867a63de" containerName="console" Dec 03 11:33:06 crc kubenswrapper[5052]: I1203 11:33:06.370235 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="b1d2ff08-d58b-48d0-a0e5-6cfd867a63de" containerName="console" Dec 03 11:33:06 crc kubenswrapper[5052]: I1203 11:33:06.370252 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="aeaaf048-0037-4550-b1f5-54cfe153dcd2" containerName="extract" Dec 03 11:33:06 crc kubenswrapper[5052]: I1203 11:33:06.370626 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="metallb-system/metallb-operator-controller-manager-b9dd667db-j4qd7" Dec 03 11:33:06 crc kubenswrapper[5052]: I1203 11:33:06.373029 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"metallb-system"/"openshift-service-ca.crt" Dec 03 11:33:06 crc kubenswrapper[5052]: I1203 11:33:06.373075 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"metallb-system"/"kube-root-ca.crt" Dec 03 11:33:06 crc kubenswrapper[5052]: I1203 11:33:06.373247 5052 reflector.go:368] Caches populated for *v1.Secret from object-"metallb-system"/"manager-account-dockercfg-vzs4c" Dec 03 11:33:06 crc kubenswrapper[5052]: I1203 11:33:06.373588 5052 reflector.go:368] Caches populated for *v1.Secret from object-"metallb-system"/"metallb-operator-webhook-server-cert" Dec 03 11:33:06 crc kubenswrapper[5052]: I1203 11:33:06.374908 5052 reflector.go:368] Caches populated for *v1.Secret from object-"metallb-system"/"metallb-operator-controller-manager-service-cert" Dec 03 11:33:06 crc kubenswrapper[5052]: I1203 11:33:06.393632 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["metallb-system/metallb-operator-controller-manager-b9dd667db-j4qd7"] Dec 03 11:33:06 crc kubenswrapper[5052]: I1203 11:33:06.462873 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ba0d3ff2-b21c-4788-9dad-6fb7f16a6cb8-webhook-cert\") pod \"metallb-operator-controller-manager-b9dd667db-j4qd7\" (UID: \"ba0d3ff2-b21c-4788-9dad-6fb7f16a6cb8\") " pod="metallb-system/metallb-operator-controller-manager-b9dd667db-j4qd7" Dec 03 11:33:06 crc kubenswrapper[5052]: I1203 11:33:06.463138 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ba0d3ff2-b21c-4788-9dad-6fb7f16a6cb8-apiservice-cert\") pod \"metallb-operator-controller-manager-b9dd667db-j4qd7\" (UID: \"ba0d3ff2-b21c-4788-9dad-6fb7f16a6cb8\") " pod="metallb-system/metallb-operator-controller-manager-b9dd667db-j4qd7" Dec 03 11:33:06 crc kubenswrapper[5052]: I1203 11:33:06.463187 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6bx4g\" (UniqueName: \"kubernetes.io/projected/ba0d3ff2-b21c-4788-9dad-6fb7f16a6cb8-kube-api-access-6bx4g\") pod \"metallb-operator-controller-manager-b9dd667db-j4qd7\" (UID: \"ba0d3ff2-b21c-4788-9dad-6fb7f16a6cb8\") " pod="metallb-system/metallb-operator-controller-manager-b9dd667db-j4qd7" Dec 03 11:33:06 crc kubenswrapper[5052]: I1203 11:33:06.564960 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-6bx4g\" (UniqueName: \"kubernetes.io/projected/ba0d3ff2-b21c-4788-9dad-6fb7f16a6cb8-kube-api-access-6bx4g\") pod \"metallb-operator-controller-manager-b9dd667db-j4qd7\" (UID: \"ba0d3ff2-b21c-4788-9dad-6fb7f16a6cb8\") " pod="metallb-system/metallb-operator-controller-manager-b9dd667db-j4qd7" Dec 03 11:33:06 crc kubenswrapper[5052]: I1203 11:33:06.565068 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ba0d3ff2-b21c-4788-9dad-6fb7f16a6cb8-webhook-cert\") pod \"metallb-operator-controller-manager-b9dd667db-j4qd7\" (UID: \"ba0d3ff2-b21c-4788-9dad-6fb7f16a6cb8\") " pod="metallb-system/metallb-operator-controller-manager-b9dd667db-j4qd7" Dec 03 11:33:06 crc kubenswrapper[5052]: I1203 11:33:06.565117 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ba0d3ff2-b21c-4788-9dad-6fb7f16a6cb8-apiservice-cert\") pod \"metallb-operator-controller-manager-b9dd667db-j4qd7\" (UID: \"ba0d3ff2-b21c-4788-9dad-6fb7f16a6cb8\") " pod="metallb-system/metallb-operator-controller-manager-b9dd667db-j4qd7" Dec 03 11:33:06 crc kubenswrapper[5052]: I1203 11:33:06.571006 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ba0d3ff2-b21c-4788-9dad-6fb7f16a6cb8-apiservice-cert\") pod \"metallb-operator-controller-manager-b9dd667db-j4qd7\" (UID: \"ba0d3ff2-b21c-4788-9dad-6fb7f16a6cb8\") " pod="metallb-system/metallb-operator-controller-manager-b9dd667db-j4qd7" Dec 03 11:33:06 crc kubenswrapper[5052]: I1203 11:33:06.572084 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ba0d3ff2-b21c-4788-9dad-6fb7f16a6cb8-webhook-cert\") pod \"metallb-operator-controller-manager-b9dd667db-j4qd7\" (UID: \"ba0d3ff2-b21c-4788-9dad-6fb7f16a6cb8\") " pod="metallb-system/metallb-operator-controller-manager-b9dd667db-j4qd7" Dec 03 11:33:06 crc kubenswrapper[5052]: I1203 11:33:06.585189 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-6bx4g\" (UniqueName: \"kubernetes.io/projected/ba0d3ff2-b21c-4788-9dad-6fb7f16a6cb8-kube-api-access-6bx4g\") pod \"metallb-operator-controller-manager-b9dd667db-j4qd7\" (UID: \"ba0d3ff2-b21c-4788-9dad-6fb7f16a6cb8\") " pod="metallb-system/metallb-operator-controller-manager-b9dd667db-j4qd7" Dec 03 11:33:06 crc kubenswrapper[5052]: I1203 11:33:06.613606 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["metallb-system/metallb-operator-webhook-server-5b4bcb5c5f-ttnfj"] Dec 03 11:33:06 crc kubenswrapper[5052]: I1203 11:33:06.614428 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="metallb-system/metallb-operator-webhook-server-5b4bcb5c5f-ttnfj" Dec 03 11:33:06 crc kubenswrapper[5052]: I1203 11:33:06.616615 5052 reflector.go:368] Caches populated for *v1.Secret from object-"metallb-system"/"metallb-operator-webhook-server-service-cert" Dec 03 11:33:06 crc kubenswrapper[5052]: I1203 11:33:06.616939 5052 reflector.go:368] Caches populated for *v1.Secret from object-"metallb-system"/"controller-dockercfg-8mblh" Dec 03 11:33:06 crc kubenswrapper[5052]: I1203 11:33:06.624911 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["metallb-system/metallb-operator-webhook-server-5b4bcb5c5f-ttnfj"] Dec 03 11:33:06 crc kubenswrapper[5052]: I1203 11:33:06.625333 5052 reflector.go:368] Caches populated for *v1.Secret from object-"metallb-system"/"metallb-webhook-cert" Dec 03 11:33:06 crc kubenswrapper[5052]: I1203 11:33:06.666757 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/74df1e0f-e3ec-4206-9e30-a99b914dd282-apiservice-cert\") pod \"metallb-operator-webhook-server-5b4bcb5c5f-ttnfj\" (UID: \"74df1e0f-e3ec-4206-9e30-a99b914dd282\") " pod="metallb-system/metallb-operator-webhook-server-5b4bcb5c5f-ttnfj" Dec 03 11:33:06 crc kubenswrapper[5052]: I1203 11:33:06.666905 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/74df1e0f-e3ec-4206-9e30-a99b914dd282-webhook-cert\") pod \"metallb-operator-webhook-server-5b4bcb5c5f-ttnfj\" (UID: \"74df1e0f-e3ec-4206-9e30-a99b914dd282\") " pod="metallb-system/metallb-operator-webhook-server-5b4bcb5c5f-ttnfj" Dec 03 11:33:06 crc kubenswrapper[5052]: I1203 11:33:06.666954 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7jhct\" (UniqueName: \"kubernetes.io/projected/74df1e0f-e3ec-4206-9e30-a99b914dd282-kube-api-access-7jhct\") pod \"metallb-operator-webhook-server-5b4bcb5c5f-ttnfj\" (UID: \"74df1e0f-e3ec-4206-9e30-a99b914dd282\") " pod="metallb-system/metallb-operator-webhook-server-5b4bcb5c5f-ttnfj" Dec 03 11:33:06 crc kubenswrapper[5052]: I1203 11:33:06.687473 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="metallb-system/metallb-operator-controller-manager-b9dd667db-j4qd7" Dec 03 11:33:06 crc kubenswrapper[5052]: I1203 11:33:06.768389 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/74df1e0f-e3ec-4206-9e30-a99b914dd282-apiservice-cert\") pod \"metallb-operator-webhook-server-5b4bcb5c5f-ttnfj\" (UID: \"74df1e0f-e3ec-4206-9e30-a99b914dd282\") " pod="metallb-system/metallb-operator-webhook-server-5b4bcb5c5f-ttnfj" Dec 03 11:33:06 crc kubenswrapper[5052]: I1203 11:33:06.768481 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/74df1e0f-e3ec-4206-9e30-a99b914dd282-webhook-cert\") pod \"metallb-operator-webhook-server-5b4bcb5c5f-ttnfj\" (UID: \"74df1e0f-e3ec-4206-9e30-a99b914dd282\") " pod="metallb-system/metallb-operator-webhook-server-5b4bcb5c5f-ttnfj" Dec 03 11:33:06 crc kubenswrapper[5052]: I1203 11:33:06.768591 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-7jhct\" (UniqueName: \"kubernetes.io/projected/74df1e0f-e3ec-4206-9e30-a99b914dd282-kube-api-access-7jhct\") pod \"metallb-operator-webhook-server-5b4bcb5c5f-ttnfj\" (UID: \"74df1e0f-e3ec-4206-9e30-a99b914dd282\") " pod="metallb-system/metallb-operator-webhook-server-5b4bcb5c5f-ttnfj" Dec 03 11:33:06 crc kubenswrapper[5052]: I1203 11:33:06.778855 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/74df1e0f-e3ec-4206-9e30-a99b914dd282-webhook-cert\") pod \"metallb-operator-webhook-server-5b4bcb5c5f-ttnfj\" (UID: \"74df1e0f-e3ec-4206-9e30-a99b914dd282\") " pod="metallb-system/metallb-operator-webhook-server-5b4bcb5c5f-ttnfj" Dec 03 11:33:06 crc kubenswrapper[5052]: I1203 11:33:06.779810 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/74df1e0f-e3ec-4206-9e30-a99b914dd282-apiservice-cert\") pod \"metallb-operator-webhook-server-5b4bcb5c5f-ttnfj\" (UID: \"74df1e0f-e3ec-4206-9e30-a99b914dd282\") " pod="metallb-system/metallb-operator-webhook-server-5b4bcb5c5f-ttnfj" Dec 03 11:33:06 crc kubenswrapper[5052]: I1203 11:33:06.797469 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-7jhct\" (UniqueName: \"kubernetes.io/projected/74df1e0f-e3ec-4206-9e30-a99b914dd282-kube-api-access-7jhct\") pod \"metallb-operator-webhook-server-5b4bcb5c5f-ttnfj\" (UID: \"74df1e0f-e3ec-4206-9e30-a99b914dd282\") " pod="metallb-system/metallb-operator-webhook-server-5b4bcb5c5f-ttnfj" Dec 03 11:33:06 crc kubenswrapper[5052]: I1203 11:33:06.930784 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="metallb-system/metallb-operator-webhook-server-5b4bcb5c5f-ttnfj" Dec 03 11:33:06 crc kubenswrapper[5052]: I1203 11:33:06.979618 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["metallb-system/metallb-operator-controller-manager-b9dd667db-j4qd7"] Dec 03 11:33:07 crc kubenswrapper[5052]: W1203 11:33:07.000401 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba0d3ff2_b21c_4788_9dad_6fb7f16a6cb8.slice/crio-4b1bb9971fc7e28ebff9d86fc12afb2909837782d6cc093966cf722cd9f292bf WatchSource:0}: Error finding container 4b1bb9971fc7e28ebff9d86fc12afb2909837782d6cc093966cf722cd9f292bf: Status 404 returned error can't find the container with id 4b1bb9971fc7e28ebff9d86fc12afb2909837782d6cc093966cf722cd9f292bf Dec 03 11:33:07 crc kubenswrapper[5052]: I1203 11:33:07.145386 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["metallb-system/metallb-operator-webhook-server-5b4bcb5c5f-ttnfj"] Dec 03 11:33:07 crc kubenswrapper[5052]: W1203 11:33:07.150822 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod74df1e0f_e3ec_4206_9e30_a99b914dd282.slice/crio-ead088d08f6ded3fca3a0f5137adea2242aa94b172ca6dbcb2e1b8c8920bbb03 WatchSource:0}: Error finding container ead088d08f6ded3fca3a0f5137adea2242aa94b172ca6dbcb2e1b8c8920bbb03: Status 404 returned error can't find the container with id ead088d08f6ded3fca3a0f5137adea2242aa94b172ca6dbcb2e1b8c8920bbb03 Dec 03 11:33:07 crc kubenswrapper[5052]: I1203 11:33:07.611420 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="metallb-system/metallb-operator-controller-manager-b9dd667db-j4qd7" event={"ID":"ba0d3ff2-b21c-4788-9dad-6fb7f16a6cb8","Type":"ContainerStarted","Data":"4b1bb9971fc7e28ebff9d86fc12afb2909837782d6cc093966cf722cd9f292bf"} Dec 03 11:33:07 crc kubenswrapper[5052]: I1203 11:33:07.612250 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="metallb-system/metallb-operator-webhook-server-5b4bcb5c5f-ttnfj" event={"ID":"74df1e0f-e3ec-4206-9e30-a99b914dd282","Type":"ContainerStarted","Data":"ead088d08f6ded3fca3a0f5137adea2242aa94b172ca6dbcb2e1b8c8920bbb03"} Dec 03 11:33:12 crc kubenswrapper[5052]: I1203 11:33:12.659003 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="metallb-system/metallb-operator-controller-manager-b9dd667db-j4qd7" event={"ID":"ba0d3ff2-b21c-4788-9dad-6fb7f16a6cb8","Type":"ContainerStarted","Data":"e1e929e12b7b6974932f801d79c4141acfe1fcdfa9d5fa4538640471892b8c15"} Dec 03 11:33:12 crc kubenswrapper[5052]: I1203 11:33:12.659797 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="metallb-system/metallb-operator-controller-manager-b9dd667db-j4qd7" Dec 03 11:33:12 crc kubenswrapper[5052]: I1203 11:33:12.660792 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="metallb-system/metallb-operator-webhook-server-5b4bcb5c5f-ttnfj" event={"ID":"74df1e0f-e3ec-4206-9e30-a99b914dd282","Type":"ContainerStarted","Data":"8d62655a62ac4a06d1e3e802d3b9d7245e133cfe7701045d58fd869c50d27d3b"} Dec 03 11:33:12 crc kubenswrapper[5052]: I1203 11:33:12.661219 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="metallb-system/metallb-operator-webhook-server-5b4bcb5c5f-ttnfj" Dec 03 11:33:12 crc kubenswrapper[5052]: I1203 11:33:12.686874 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="metallb-system/metallb-operator-controller-manager-b9dd667db-j4qd7" podStartSLOduration=1.7983050550000002 podStartE2EDuration="6.686853865s" podCreationTimestamp="2025-12-03 11:33:06 +0000 UTC" firstStartedPulling="2025-12-03 11:33:07.014597311 +0000 UTC m=+777.602839065" lastFinishedPulling="2025-12-03 11:33:11.903146111 +0000 UTC m=+782.491387875" observedRunningTime="2025-12-03 11:33:12.681335838 +0000 UTC m=+783.269577612" watchObservedRunningTime="2025-12-03 11:33:12.686853865 +0000 UTC m=+783.275095629" Dec 03 11:33:12 crc kubenswrapper[5052]: I1203 11:33:12.718026 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="metallb-system/metallb-operator-webhook-server-5b4bcb5c5f-ttnfj" podStartSLOduration=1.965345806 podStartE2EDuration="6.717998787s" podCreationTimestamp="2025-12-03 11:33:06 +0000 UTC" firstStartedPulling="2025-12-03 11:33:07.159792516 +0000 UTC m=+777.748034270" lastFinishedPulling="2025-12-03 11:33:11.912445487 +0000 UTC m=+782.500687251" observedRunningTime="2025-12-03 11:33:12.717147422 +0000 UTC m=+783.305389186" watchObservedRunningTime="2025-12-03 11:33:12.717998787 +0000 UTC m=+783.306240591" Dec 03 11:33:26 crc kubenswrapper[5052]: I1203 11:33:26.936263 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="metallb-system/metallb-operator-webhook-server-5b4bcb5c5f-ttnfj" Dec 03 11:33:46 crc kubenswrapper[5052]: I1203 11:33:46.690574 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="metallb-system/metallb-operator-controller-manager-b9dd667db-j4qd7" Dec 03 11:33:47 crc kubenswrapper[5052]: I1203 11:33:47.445061 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["metallb-system/frr-k8s-47pp5"] Dec 03 11:33:47 crc kubenswrapper[5052]: I1203 11:33:47.448331 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="metallb-system/frr-k8s-47pp5" Dec 03 11:33:47 crc kubenswrapper[5052]: I1203 11:33:47.451907 5052 reflector.go:368] Caches populated for *v1.Secret from object-"metallb-system"/"frr-k8s-daemon-dockercfg-pbbgz" Dec 03 11:33:47 crc kubenswrapper[5052]: I1203 11:33:47.452262 5052 reflector.go:368] Caches populated for *v1.Secret from object-"metallb-system"/"frr-k8s-certs-secret" Dec 03 11:33:47 crc kubenswrapper[5052]: I1203 11:33:47.452557 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"metallb-system"/"frr-startup" Dec 03 11:33:47 crc kubenswrapper[5052]: I1203 11:33:47.461218 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["metallb-system/frr-k8s-webhook-server-7fcb986d4-66rvv"] Dec 03 11:33:47 crc kubenswrapper[5052]: I1203 11:33:47.462097 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="metallb-system/frr-k8s-webhook-server-7fcb986d4-66rvv" Dec 03 11:33:47 crc kubenswrapper[5052]: I1203 11:33:47.466034 5052 reflector.go:368] Caches populated for *v1.Secret from object-"metallb-system"/"frr-k8s-webhook-server-cert" Dec 03 11:33:47 crc kubenswrapper[5052]: I1203 11:33:47.471869 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["metallb-system/frr-k8s-webhook-server-7fcb986d4-66rvv"] Dec 03 11:33:47 crc kubenswrapper[5052]: I1203 11:33:47.518417 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"frr-conf\" (UniqueName: \"kubernetes.io/empty-dir/76bb2cd3-e75d-40f1-a9e6-63f7bd725eea-frr-conf\") pod \"frr-k8s-47pp5\" (UID: \"76bb2cd3-e75d-40f1-a9e6-63f7bd725eea\") " pod="metallb-system/frr-k8s-47pp5" Dec 03 11:33:47 crc kubenswrapper[5052]: I1203 11:33:47.518462 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"reloader\" (UniqueName: \"kubernetes.io/empty-dir/76bb2cd3-e75d-40f1-a9e6-63f7bd725eea-reloader\") pod \"frr-k8s-47pp5\" (UID: \"76bb2cd3-e75d-40f1-a9e6-63f7bd725eea\") " pod="metallb-system/frr-k8s-47pp5" Dec 03 11:33:47 crc kubenswrapper[5052]: I1203 11:33:47.518496 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/76bb2cd3-e75d-40f1-a9e6-63f7bd725eea-metrics-certs\") pod \"frr-k8s-47pp5\" (UID: \"76bb2cd3-e75d-40f1-a9e6-63f7bd725eea\") " pod="metallb-system/frr-k8s-47pp5" Dec 03 11:33:47 crc kubenswrapper[5052]: I1203 11:33:47.518541 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dbgk5\" (UniqueName: \"kubernetes.io/projected/c89171df-ff1e-4bc6-8e4e-9c24dc8ee436-kube-api-access-dbgk5\") pod \"frr-k8s-webhook-server-7fcb986d4-66rvv\" (UID: \"c89171df-ff1e-4bc6-8e4e-9c24dc8ee436\") " pod="metallb-system/frr-k8s-webhook-server-7fcb986d4-66rvv" Dec 03 11:33:47 crc kubenswrapper[5052]: I1203 11:33:47.518592 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"frr-sockets\" (UniqueName: \"kubernetes.io/empty-dir/76bb2cd3-e75d-40f1-a9e6-63f7bd725eea-frr-sockets\") pod \"frr-k8s-47pp5\" (UID: \"76bb2cd3-e75d-40f1-a9e6-63f7bd725eea\") " pod="metallb-system/frr-k8s-47pp5" Dec 03 11:33:47 crc kubenswrapper[5052]: I1203 11:33:47.518749 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xk5tc\" (UniqueName: \"kubernetes.io/projected/76bb2cd3-e75d-40f1-a9e6-63f7bd725eea-kube-api-access-xk5tc\") pod \"frr-k8s-47pp5\" (UID: \"76bb2cd3-e75d-40f1-a9e6-63f7bd725eea\") " pod="metallb-system/frr-k8s-47pp5" Dec 03 11:33:47 crc kubenswrapper[5052]: I1203 11:33:47.518824 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics\" (UniqueName: \"kubernetes.io/empty-dir/76bb2cd3-e75d-40f1-a9e6-63f7bd725eea-metrics\") pod \"frr-k8s-47pp5\" (UID: \"76bb2cd3-e75d-40f1-a9e6-63f7bd725eea\") " pod="metallb-system/frr-k8s-47pp5" Dec 03 11:33:47 crc kubenswrapper[5052]: I1203 11:33:47.518859 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"frr-startup\" (UniqueName: \"kubernetes.io/configmap/76bb2cd3-e75d-40f1-a9e6-63f7bd725eea-frr-startup\") pod \"frr-k8s-47pp5\" (UID: \"76bb2cd3-e75d-40f1-a9e6-63f7bd725eea\") " pod="metallb-system/frr-k8s-47pp5" Dec 03 11:33:47 crc kubenswrapper[5052]: I1203 11:33:47.518882 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c89171df-ff1e-4bc6-8e4e-9c24dc8ee436-cert\") pod \"frr-k8s-webhook-server-7fcb986d4-66rvv\" (UID: \"c89171df-ff1e-4bc6-8e4e-9c24dc8ee436\") " pod="metallb-system/frr-k8s-webhook-server-7fcb986d4-66rvv" Dec 03 11:33:47 crc kubenswrapper[5052]: I1203 11:33:47.556949 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["metallb-system/speaker-56dnr"] Dec 03 11:33:47 crc kubenswrapper[5052]: I1203 11:33:47.557804 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="metallb-system/speaker-56dnr" Dec 03 11:33:47 crc kubenswrapper[5052]: I1203 11:33:47.559868 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"metallb-system"/"metallb-excludel2" Dec 03 11:33:47 crc kubenswrapper[5052]: I1203 11:33:47.560495 5052 reflector.go:368] Caches populated for *v1.Secret from object-"metallb-system"/"speaker-certs-secret" Dec 03 11:33:47 crc kubenswrapper[5052]: I1203 11:33:47.560909 5052 reflector.go:368] Caches populated for *v1.Secret from object-"metallb-system"/"speaker-dockercfg-24dsq" Dec 03 11:33:47 crc kubenswrapper[5052]: I1203 11:33:47.560957 5052 reflector.go:368] Caches populated for *v1.Secret from object-"metallb-system"/"metallb-memberlist" Dec 03 11:33:47 crc kubenswrapper[5052]: I1203 11:33:47.579396 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["metallb-system/controller-f8648f98b-nc9sj"] Dec 03 11:33:47 crc kubenswrapper[5052]: I1203 11:33:47.580234 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="metallb-system/controller-f8648f98b-nc9sj" Dec 03 11:33:47 crc kubenswrapper[5052]: I1203 11:33:47.587320 5052 reflector.go:368] Caches populated for *v1.Secret from object-"metallb-system"/"controller-certs-secret" Dec 03 11:33:47 crc kubenswrapper[5052]: I1203 11:33:47.602964 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["metallb-system/controller-f8648f98b-nc9sj"] Dec 03 11:33:47 crc kubenswrapper[5052]: I1203 11:33:47.624973 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-498bd\" (UniqueName: \"kubernetes.io/projected/3d2671fe-8f61-40ab-8ebf-3c7d8fe99644-kube-api-access-498bd\") pod \"controller-f8648f98b-nc9sj\" (UID: \"3d2671fe-8f61-40ab-8ebf-3c7d8fe99644\") " pod="metallb-system/controller-f8648f98b-nc9sj" Dec 03 11:33:47 crc kubenswrapper[5052]: I1203 11:33:47.625018 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/76bb2cd3-e75d-40f1-a9e6-63f7bd725eea-metrics-certs\") pod \"frr-k8s-47pp5\" (UID: \"76bb2cd3-e75d-40f1-a9e6-63f7bd725eea\") " pod="metallb-system/frr-k8s-47pp5" Dec 03 11:33:47 crc kubenswrapper[5052]: I1203 11:33:47.625046 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-dbgk5\" (UniqueName: \"kubernetes.io/projected/c89171df-ff1e-4bc6-8e4e-9c24dc8ee436-kube-api-access-dbgk5\") pod \"frr-k8s-webhook-server-7fcb986d4-66rvv\" (UID: \"c89171df-ff1e-4bc6-8e4e-9c24dc8ee436\") " pod="metallb-system/frr-k8s-webhook-server-7fcb986d4-66rvv" Dec 03 11:33:47 crc kubenswrapper[5052]: I1203 11:33:47.625063 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gk79j\" (UniqueName: \"kubernetes.io/projected/92c3adf7-3dc8-4644-9fb7-f68994747973-kube-api-access-gk79j\") pod \"speaker-56dnr\" (UID: \"92c3adf7-3dc8-4644-9fb7-f68994747973\") " pod="metallb-system/speaker-56dnr" Dec 03 11:33:47 crc kubenswrapper[5052]: I1203 11:33:47.625084 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3d2671fe-8f61-40ab-8ebf-3c7d8fe99644-metrics-certs\") pod \"controller-f8648f98b-nc9sj\" (UID: \"3d2671fe-8f61-40ab-8ebf-3c7d8fe99644\") " pod="metallb-system/controller-f8648f98b-nc9sj" Dec 03 11:33:47 crc kubenswrapper[5052]: I1203 11:33:47.625116 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"frr-sockets\" (UniqueName: \"kubernetes.io/empty-dir/76bb2cd3-e75d-40f1-a9e6-63f7bd725eea-frr-sockets\") pod \"frr-k8s-47pp5\" (UID: \"76bb2cd3-e75d-40f1-a9e6-63f7bd725eea\") " pod="metallb-system/frr-k8s-47pp5" Dec 03 11:33:47 crc kubenswrapper[5052]: I1203 11:33:47.625136 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/92c3adf7-3dc8-4644-9fb7-f68994747973-metrics-certs\") pod \"speaker-56dnr\" (UID: \"92c3adf7-3dc8-4644-9fb7-f68994747973\") " pod="metallb-system/speaker-56dnr" Dec 03 11:33:47 crc kubenswrapper[5052]: I1203 11:33:47.625161 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-xk5tc\" (UniqueName: \"kubernetes.io/projected/76bb2cd3-e75d-40f1-a9e6-63f7bd725eea-kube-api-access-xk5tc\") pod \"frr-k8s-47pp5\" (UID: \"76bb2cd3-e75d-40f1-a9e6-63f7bd725eea\") " pod="metallb-system/frr-k8s-47pp5" Dec 03 11:33:47 crc kubenswrapper[5052]: I1203 11:33:47.625187 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"metrics\" (UniqueName: \"kubernetes.io/empty-dir/76bb2cd3-e75d-40f1-a9e6-63f7bd725eea-metrics\") pod \"frr-k8s-47pp5\" (UID: \"76bb2cd3-e75d-40f1-a9e6-63f7bd725eea\") " pod="metallb-system/frr-k8s-47pp5" Dec 03 11:33:47 crc kubenswrapper[5052]: I1203 11:33:47.625205 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3d2671fe-8f61-40ab-8ebf-3c7d8fe99644-cert\") pod \"controller-f8648f98b-nc9sj\" (UID: \"3d2671fe-8f61-40ab-8ebf-3c7d8fe99644\") " pod="metallb-system/controller-f8648f98b-nc9sj" Dec 03 11:33:47 crc kubenswrapper[5052]: I1203 11:33:47.625221 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"frr-startup\" (UniqueName: \"kubernetes.io/configmap/76bb2cd3-e75d-40f1-a9e6-63f7bd725eea-frr-startup\") pod \"frr-k8s-47pp5\" (UID: \"76bb2cd3-e75d-40f1-a9e6-63f7bd725eea\") " pod="metallb-system/frr-k8s-47pp5" Dec 03 11:33:47 crc kubenswrapper[5052]: I1203 11:33:47.625238 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metallb-excludel2\" (UniqueName: \"kubernetes.io/configmap/92c3adf7-3dc8-4644-9fb7-f68994747973-metallb-excludel2\") pod \"speaker-56dnr\" (UID: \"92c3adf7-3dc8-4644-9fb7-f68994747973\") " pod="metallb-system/speaker-56dnr" Dec 03 11:33:47 crc kubenswrapper[5052]: I1203 11:33:47.625253 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c89171df-ff1e-4bc6-8e4e-9c24dc8ee436-cert\") pod \"frr-k8s-webhook-server-7fcb986d4-66rvv\" (UID: \"c89171df-ff1e-4bc6-8e4e-9c24dc8ee436\") " pod="metallb-system/frr-k8s-webhook-server-7fcb986d4-66rvv" Dec 03 11:33:47 crc kubenswrapper[5052]: I1203 11:33:47.625268 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"memberlist\" (UniqueName: \"kubernetes.io/secret/92c3adf7-3dc8-4644-9fb7-f68994747973-memberlist\") pod \"speaker-56dnr\" (UID: \"92c3adf7-3dc8-4644-9fb7-f68994747973\") " pod="metallb-system/speaker-56dnr" Dec 03 11:33:47 crc kubenswrapper[5052]: I1203 11:33:47.625284 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"frr-conf\" (UniqueName: \"kubernetes.io/empty-dir/76bb2cd3-e75d-40f1-a9e6-63f7bd725eea-frr-conf\") pod \"frr-k8s-47pp5\" (UID: \"76bb2cd3-e75d-40f1-a9e6-63f7bd725eea\") " pod="metallb-system/frr-k8s-47pp5" Dec 03 11:33:47 crc kubenswrapper[5052]: I1203 11:33:47.625301 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"reloader\" (UniqueName: \"kubernetes.io/empty-dir/76bb2cd3-e75d-40f1-a9e6-63f7bd725eea-reloader\") pod \"frr-k8s-47pp5\" (UID: \"76bb2cd3-e75d-40f1-a9e6-63f7bd725eea\") " pod="metallb-system/frr-k8s-47pp5" Dec 03 11:33:47 crc kubenswrapper[5052]: I1203 11:33:47.625777 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"reloader\" (UniqueName: \"kubernetes.io/empty-dir/76bb2cd3-e75d-40f1-a9e6-63f7bd725eea-reloader\") pod \"frr-k8s-47pp5\" (UID: \"76bb2cd3-e75d-40f1-a9e6-63f7bd725eea\") " pod="metallb-system/frr-k8s-47pp5" Dec 03 11:33:47 crc kubenswrapper[5052]: I1203 11:33:47.625975 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"metrics\" (UniqueName: \"kubernetes.io/empty-dir/76bb2cd3-e75d-40f1-a9e6-63f7bd725eea-metrics\") pod \"frr-k8s-47pp5\" (UID: \"76bb2cd3-e75d-40f1-a9e6-63f7bd725eea\") " pod="metallb-system/frr-k8s-47pp5" Dec 03 11:33:47 crc kubenswrapper[5052]: E1203 11:33:47.626261 5052 secret.go:188] Couldn't get secret metallb-system/frr-k8s-certs-secret: secret "frr-k8s-certs-secret" not found Dec 03 11:33:47 crc kubenswrapper[5052]: E1203 11:33:47.626370 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/76bb2cd3-e75d-40f1-a9e6-63f7bd725eea-metrics-certs podName:76bb2cd3-e75d-40f1-a9e6-63f7bd725eea nodeName:}" failed. No retries permitted until 2025-12-03 11:33:48.126354202 +0000 UTC m=+818.714595966 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/76bb2cd3-e75d-40f1-a9e6-63f7bd725eea-metrics-certs") pod "frr-k8s-47pp5" (UID: "76bb2cd3-e75d-40f1-a9e6-63f7bd725eea") : secret "frr-k8s-certs-secret" not found Dec 03 11:33:47 crc kubenswrapper[5052]: I1203 11:33:47.626729 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"frr-sockets\" (UniqueName: \"kubernetes.io/empty-dir/76bb2cd3-e75d-40f1-a9e6-63f7bd725eea-frr-sockets\") pod \"frr-k8s-47pp5\" (UID: \"76bb2cd3-e75d-40f1-a9e6-63f7bd725eea\") " pod="metallb-system/frr-k8s-47pp5" Dec 03 11:33:47 crc kubenswrapper[5052]: E1203 11:33:47.626803 5052 secret.go:188] Couldn't get secret metallb-system/frr-k8s-webhook-server-cert: secret "frr-k8s-webhook-server-cert" not found Dec 03 11:33:47 crc kubenswrapper[5052]: E1203 11:33:47.626848 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c89171df-ff1e-4bc6-8e4e-9c24dc8ee436-cert podName:c89171df-ff1e-4bc6-8e4e-9c24dc8ee436 nodeName:}" failed. No retries permitted until 2025-12-03 11:33:48.126835526 +0000 UTC m=+818.715077290 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/c89171df-ff1e-4bc6-8e4e-9c24dc8ee436-cert") pod "frr-k8s-webhook-server-7fcb986d4-66rvv" (UID: "c89171df-ff1e-4bc6-8e4e-9c24dc8ee436") : secret "frr-k8s-webhook-server-cert" not found Dec 03 11:33:47 crc kubenswrapper[5052]: I1203 11:33:47.626735 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"frr-startup\" (UniqueName: \"kubernetes.io/configmap/76bb2cd3-e75d-40f1-a9e6-63f7bd725eea-frr-startup\") pod \"frr-k8s-47pp5\" (UID: \"76bb2cd3-e75d-40f1-a9e6-63f7bd725eea\") " pod="metallb-system/frr-k8s-47pp5" Dec 03 11:33:47 crc kubenswrapper[5052]: I1203 11:33:47.627027 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"frr-conf\" (UniqueName: \"kubernetes.io/empty-dir/76bb2cd3-e75d-40f1-a9e6-63f7bd725eea-frr-conf\") pod \"frr-k8s-47pp5\" (UID: \"76bb2cd3-e75d-40f1-a9e6-63f7bd725eea\") " pod="metallb-system/frr-k8s-47pp5" Dec 03 11:33:47 crc kubenswrapper[5052]: I1203 11:33:47.654567 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-xk5tc\" (UniqueName: \"kubernetes.io/projected/76bb2cd3-e75d-40f1-a9e6-63f7bd725eea-kube-api-access-xk5tc\") pod \"frr-k8s-47pp5\" (UID: \"76bb2cd3-e75d-40f1-a9e6-63f7bd725eea\") " pod="metallb-system/frr-k8s-47pp5" Dec 03 11:33:47 crc kubenswrapper[5052]: I1203 11:33:47.654998 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-dbgk5\" (UniqueName: \"kubernetes.io/projected/c89171df-ff1e-4bc6-8e4e-9c24dc8ee436-kube-api-access-dbgk5\") pod \"frr-k8s-webhook-server-7fcb986d4-66rvv\" (UID: \"c89171df-ff1e-4bc6-8e4e-9c24dc8ee436\") " pod="metallb-system/frr-k8s-webhook-server-7fcb986d4-66rvv" Dec 03 11:33:47 crc kubenswrapper[5052]: I1203 11:33:47.729206 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-498bd\" (UniqueName: \"kubernetes.io/projected/3d2671fe-8f61-40ab-8ebf-3c7d8fe99644-kube-api-access-498bd\") pod \"controller-f8648f98b-nc9sj\" (UID: \"3d2671fe-8f61-40ab-8ebf-3c7d8fe99644\") " pod="metallb-system/controller-f8648f98b-nc9sj" Dec 03 11:33:47 crc kubenswrapper[5052]: I1203 11:33:47.729280 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-gk79j\" (UniqueName: \"kubernetes.io/projected/92c3adf7-3dc8-4644-9fb7-f68994747973-kube-api-access-gk79j\") pod \"speaker-56dnr\" (UID: \"92c3adf7-3dc8-4644-9fb7-f68994747973\") " pod="metallb-system/speaker-56dnr" Dec 03 11:33:47 crc kubenswrapper[5052]: I1203 11:33:47.729305 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3d2671fe-8f61-40ab-8ebf-3c7d8fe99644-metrics-certs\") pod \"controller-f8648f98b-nc9sj\" (UID: \"3d2671fe-8f61-40ab-8ebf-3c7d8fe99644\") " pod="metallb-system/controller-f8648f98b-nc9sj" Dec 03 11:33:47 crc kubenswrapper[5052]: I1203 11:33:47.729344 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/92c3adf7-3dc8-4644-9fb7-f68994747973-metrics-certs\") pod \"speaker-56dnr\" (UID: \"92c3adf7-3dc8-4644-9fb7-f68994747973\") " pod="metallb-system/speaker-56dnr" Dec 03 11:33:47 crc kubenswrapper[5052]: I1203 11:33:47.729387 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3d2671fe-8f61-40ab-8ebf-3c7d8fe99644-cert\") pod \"controller-f8648f98b-nc9sj\" (UID: \"3d2671fe-8f61-40ab-8ebf-3c7d8fe99644\") " pod="metallb-system/controller-f8648f98b-nc9sj" Dec 03 11:33:47 crc kubenswrapper[5052]: I1203 11:33:47.729405 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"metallb-excludel2\" (UniqueName: \"kubernetes.io/configmap/92c3adf7-3dc8-4644-9fb7-f68994747973-metallb-excludel2\") pod \"speaker-56dnr\" (UID: \"92c3adf7-3dc8-4644-9fb7-f68994747973\") " pod="metallb-system/speaker-56dnr" Dec 03 11:33:47 crc kubenswrapper[5052]: I1203 11:33:47.729424 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"memberlist\" (UniqueName: \"kubernetes.io/secret/92c3adf7-3dc8-4644-9fb7-f68994747973-memberlist\") pod \"speaker-56dnr\" (UID: \"92c3adf7-3dc8-4644-9fb7-f68994747973\") " pod="metallb-system/speaker-56dnr" Dec 03 11:33:47 crc kubenswrapper[5052]: E1203 11:33:47.729557 5052 secret.go:188] Couldn't get secret metallb-system/metallb-memberlist: secret "metallb-memberlist" not found Dec 03 11:33:47 crc kubenswrapper[5052]: E1203 11:33:47.729604 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/92c3adf7-3dc8-4644-9fb7-f68994747973-memberlist podName:92c3adf7-3dc8-4644-9fb7-f68994747973 nodeName:}" failed. No retries permitted until 2025-12-03 11:33:48.229588906 +0000 UTC m=+818.817830670 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "memberlist" (UniqueName: "kubernetes.io/secret/92c3adf7-3dc8-4644-9fb7-f68994747973-memberlist") pod "speaker-56dnr" (UID: "92c3adf7-3dc8-4644-9fb7-f68994747973") : secret "metallb-memberlist" not found Dec 03 11:33:47 crc kubenswrapper[5052]: E1203 11:33:47.730677 5052 secret.go:188] Couldn't get secret metallb-system/speaker-certs-secret: secret "speaker-certs-secret" not found Dec 03 11:33:47 crc kubenswrapper[5052]: E1203 11:33:47.730832 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/92c3adf7-3dc8-4644-9fb7-f68994747973-metrics-certs podName:92c3adf7-3dc8-4644-9fb7-f68994747973 nodeName:}" failed. No retries permitted until 2025-12-03 11:33:48.23079763 +0000 UTC m=+818.819039464 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/92c3adf7-3dc8-4644-9fb7-f68994747973-metrics-certs") pod "speaker-56dnr" (UID: "92c3adf7-3dc8-4644-9fb7-f68994747973") : secret "speaker-certs-secret" not found Dec 03 11:33:47 crc kubenswrapper[5052]: I1203 11:33:47.731730 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"metallb-excludel2\" (UniqueName: \"kubernetes.io/configmap/92c3adf7-3dc8-4644-9fb7-f68994747973-metallb-excludel2\") pod \"speaker-56dnr\" (UID: \"92c3adf7-3dc8-4644-9fb7-f68994747973\") " pod="metallb-system/speaker-56dnr" Dec 03 11:33:47 crc kubenswrapper[5052]: I1203 11:33:47.735787 5052 reflector.go:368] Caches populated for *v1.Secret from object-"metallb-system"/"metallb-webhook-cert" Dec 03 11:33:47 crc kubenswrapper[5052]: I1203 11:33:47.746162 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3d2671fe-8f61-40ab-8ebf-3c7d8fe99644-cert\") pod \"controller-f8648f98b-nc9sj\" (UID: \"3d2671fe-8f61-40ab-8ebf-3c7d8fe99644\") " pod="metallb-system/controller-f8648f98b-nc9sj" Dec 03 11:33:47 crc kubenswrapper[5052]: I1203 11:33:47.747649 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3d2671fe-8f61-40ab-8ebf-3c7d8fe99644-metrics-certs\") pod \"controller-f8648f98b-nc9sj\" (UID: \"3d2671fe-8f61-40ab-8ebf-3c7d8fe99644\") " pod="metallb-system/controller-f8648f98b-nc9sj" Dec 03 11:33:47 crc kubenswrapper[5052]: I1203 11:33:47.749789 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-gk79j\" (UniqueName: \"kubernetes.io/projected/92c3adf7-3dc8-4644-9fb7-f68994747973-kube-api-access-gk79j\") pod \"speaker-56dnr\" (UID: \"92c3adf7-3dc8-4644-9fb7-f68994747973\") " pod="metallb-system/speaker-56dnr" Dec 03 11:33:47 crc kubenswrapper[5052]: I1203 11:33:47.750427 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-498bd\" (UniqueName: \"kubernetes.io/projected/3d2671fe-8f61-40ab-8ebf-3c7d8fe99644-kube-api-access-498bd\") pod \"controller-f8648f98b-nc9sj\" (UID: \"3d2671fe-8f61-40ab-8ebf-3c7d8fe99644\") " pod="metallb-system/controller-f8648f98b-nc9sj" Dec 03 11:33:47 crc kubenswrapper[5052]: I1203 11:33:47.894988 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="metallb-system/controller-f8648f98b-nc9sj" Dec 03 11:33:48 crc kubenswrapper[5052]: I1203 11:33:48.135179 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c89171df-ff1e-4bc6-8e4e-9c24dc8ee436-cert\") pod \"frr-k8s-webhook-server-7fcb986d4-66rvv\" (UID: \"c89171df-ff1e-4bc6-8e4e-9c24dc8ee436\") " pod="metallb-system/frr-k8s-webhook-server-7fcb986d4-66rvv" Dec 03 11:33:48 crc kubenswrapper[5052]: I1203 11:33:48.135662 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/76bb2cd3-e75d-40f1-a9e6-63f7bd725eea-metrics-certs\") pod \"frr-k8s-47pp5\" (UID: \"76bb2cd3-e75d-40f1-a9e6-63f7bd725eea\") " pod="metallb-system/frr-k8s-47pp5" Dec 03 11:33:48 crc kubenswrapper[5052]: I1203 11:33:48.142292 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c89171df-ff1e-4bc6-8e4e-9c24dc8ee436-cert\") pod \"frr-k8s-webhook-server-7fcb986d4-66rvv\" (UID: \"c89171df-ff1e-4bc6-8e4e-9c24dc8ee436\") " pod="metallb-system/frr-k8s-webhook-server-7fcb986d4-66rvv" Dec 03 11:33:48 crc kubenswrapper[5052]: I1203 11:33:48.143808 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/76bb2cd3-e75d-40f1-a9e6-63f7bd725eea-metrics-certs\") pod \"frr-k8s-47pp5\" (UID: \"76bb2cd3-e75d-40f1-a9e6-63f7bd725eea\") " pod="metallb-system/frr-k8s-47pp5" Dec 03 11:33:48 crc kubenswrapper[5052]: I1203 11:33:48.237078 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/92c3adf7-3dc8-4644-9fb7-f68994747973-metrics-certs\") pod \"speaker-56dnr\" (UID: \"92c3adf7-3dc8-4644-9fb7-f68994747973\") " pod="metallb-system/speaker-56dnr" Dec 03 11:33:48 crc kubenswrapper[5052]: I1203 11:33:48.237170 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"memberlist\" (UniqueName: \"kubernetes.io/secret/92c3adf7-3dc8-4644-9fb7-f68994747973-memberlist\") pod \"speaker-56dnr\" (UID: \"92c3adf7-3dc8-4644-9fb7-f68994747973\") " pod="metallb-system/speaker-56dnr" Dec 03 11:33:48 crc kubenswrapper[5052]: E1203 11:33:48.237320 5052 secret.go:188] Couldn't get secret metallb-system/metallb-memberlist: secret "metallb-memberlist" not found Dec 03 11:33:48 crc kubenswrapper[5052]: E1203 11:33:48.237379 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/92c3adf7-3dc8-4644-9fb7-f68994747973-memberlist podName:92c3adf7-3dc8-4644-9fb7-f68994747973 nodeName:}" failed. No retries permitted until 2025-12-03 11:33:49.237364095 +0000 UTC m=+819.825605859 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "memberlist" (UniqueName: "kubernetes.io/secret/92c3adf7-3dc8-4644-9fb7-f68994747973-memberlist") pod "speaker-56dnr" (UID: "92c3adf7-3dc8-4644-9fb7-f68994747973") : secret "metallb-memberlist" not found Dec 03 11:33:48 crc kubenswrapper[5052]: I1203 11:33:48.241047 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/92c3adf7-3dc8-4644-9fb7-f68994747973-metrics-certs\") pod \"speaker-56dnr\" (UID: \"92c3adf7-3dc8-4644-9fb7-f68994747973\") " pod="metallb-system/speaker-56dnr" Dec 03 11:33:48 crc kubenswrapper[5052]: I1203 11:33:48.337662 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["metallb-system/controller-f8648f98b-nc9sj"] Dec 03 11:33:48 crc kubenswrapper[5052]: I1203 11:33:48.374312 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="metallb-system/frr-k8s-47pp5" Dec 03 11:33:48 crc kubenswrapper[5052]: I1203 11:33:48.385657 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="metallb-system/frr-k8s-webhook-server-7fcb986d4-66rvv" Dec 03 11:33:48 crc kubenswrapper[5052]: I1203 11:33:48.617568 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["metallb-system/frr-k8s-webhook-server-7fcb986d4-66rvv"] Dec 03 11:33:48 crc kubenswrapper[5052]: W1203 11:33:48.630546 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc89171df_ff1e_4bc6_8e4e_9c24dc8ee436.slice/crio-6943b2b44a02c95440b834e4dcc68431635223d9ffb5a0337febe3243fe4a097 WatchSource:0}: Error finding container 6943b2b44a02c95440b834e4dcc68431635223d9ffb5a0337febe3243fe4a097: Status 404 returned error can't find the container with id 6943b2b44a02c95440b834e4dcc68431635223d9ffb5a0337febe3243fe4a097 Dec 03 11:33:48 crc kubenswrapper[5052]: I1203 11:33:48.863422 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="metallb-system/frr-k8s-47pp5" event={"ID":"76bb2cd3-e75d-40f1-a9e6-63f7bd725eea","Type":"ContainerStarted","Data":"89b8e79e474df53ac54aec37ab83de10faad74527ed21c71308b064e241403ae"} Dec 03 11:33:48 crc kubenswrapper[5052]: I1203 11:33:48.865395 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="metallb-system/controller-f8648f98b-nc9sj" event={"ID":"3d2671fe-8f61-40ab-8ebf-3c7d8fe99644","Type":"ContainerStarted","Data":"89b55ad385b42f24fb80a7eb55a8858d14f32b4e7281baf2b3ec2e5b47c4c971"} Dec 03 11:33:48 crc kubenswrapper[5052]: I1203 11:33:48.865456 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="metallb-system/controller-f8648f98b-nc9sj" event={"ID":"3d2671fe-8f61-40ab-8ebf-3c7d8fe99644","Type":"ContainerStarted","Data":"286028861a0b848652d91f355479334b46175d3d8c4088e8799a371bd7cd263c"} Dec 03 11:33:48 crc kubenswrapper[5052]: I1203 11:33:48.865469 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="metallb-system/controller-f8648f98b-nc9sj" event={"ID":"3d2671fe-8f61-40ab-8ebf-3c7d8fe99644","Type":"ContainerStarted","Data":"6684564e48de70d9f4e3428a43e723353907c0f217789da5e987e903a5aa4ce6"} Dec 03 11:33:48 crc kubenswrapper[5052]: I1203 11:33:48.865620 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="metallb-system/controller-f8648f98b-nc9sj" Dec 03 11:33:48 crc kubenswrapper[5052]: I1203 11:33:48.866736 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="metallb-system/frr-k8s-webhook-server-7fcb986d4-66rvv" event={"ID":"c89171df-ff1e-4bc6-8e4e-9c24dc8ee436","Type":"ContainerStarted","Data":"6943b2b44a02c95440b834e4dcc68431635223d9ffb5a0337febe3243fe4a097"} Dec 03 11:33:48 crc kubenswrapper[5052]: I1203 11:33:48.885143 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="metallb-system/controller-f8648f98b-nc9sj" podStartSLOduration=1.88511492 podStartE2EDuration="1.88511492s" podCreationTimestamp="2025-12-03 11:33:47 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:33:48.882116344 +0000 UTC m=+819.470358118" watchObservedRunningTime="2025-12-03 11:33:48.88511492 +0000 UTC m=+819.473356684" Dec 03 11:33:49 crc kubenswrapper[5052]: I1203 11:33:49.255152 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"memberlist\" (UniqueName: \"kubernetes.io/secret/92c3adf7-3dc8-4644-9fb7-f68994747973-memberlist\") pod \"speaker-56dnr\" (UID: \"92c3adf7-3dc8-4644-9fb7-f68994747973\") " pod="metallb-system/speaker-56dnr" Dec 03 11:33:49 crc kubenswrapper[5052]: I1203 11:33:49.261694 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"memberlist\" (UniqueName: \"kubernetes.io/secret/92c3adf7-3dc8-4644-9fb7-f68994747973-memberlist\") pod \"speaker-56dnr\" (UID: \"92c3adf7-3dc8-4644-9fb7-f68994747973\") " pod="metallb-system/speaker-56dnr" Dec 03 11:33:49 crc kubenswrapper[5052]: I1203 11:33:49.372946 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="metallb-system/speaker-56dnr" Dec 03 11:33:49 crc kubenswrapper[5052]: W1203 11:33:49.397996 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92c3adf7_3dc8_4644_9fb7_f68994747973.slice/crio-b05c70953eb88feac792df0ceb57b8ebf3eb1b006633631dc87e198c47b3b0a2 WatchSource:0}: Error finding container b05c70953eb88feac792df0ceb57b8ebf3eb1b006633631dc87e198c47b3b0a2: Status 404 returned error can't find the container with id b05c70953eb88feac792df0ceb57b8ebf3eb1b006633631dc87e198c47b3b0a2 Dec 03 11:33:49 crc kubenswrapper[5052]: I1203 11:33:49.745285 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openshift-marketplace/certified-operators-48pv8"] Dec 03 11:33:49 crc kubenswrapper[5052]: I1203 11:33:49.752847 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/certified-operators-48pv8" Dec 03 11:33:49 crc kubenswrapper[5052]: I1203 11:33:49.771556 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/certified-operators-48pv8"] Dec 03 11:33:49 crc kubenswrapper[5052]: I1203 11:33:49.863334 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sgbkq\" (UniqueName: \"kubernetes.io/projected/e5586a9d-6e6b-47f5-994b-80f9238d8a77-kube-api-access-sgbkq\") pod \"certified-operators-48pv8\" (UID: \"e5586a9d-6e6b-47f5-994b-80f9238d8a77\") " pod="openshift-marketplace/certified-operators-48pv8" Dec 03 11:33:49 crc kubenswrapper[5052]: I1203 11:33:49.863378 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/e5586a9d-6e6b-47f5-994b-80f9238d8a77-catalog-content\") pod \"certified-operators-48pv8\" (UID: \"e5586a9d-6e6b-47f5-994b-80f9238d8a77\") " pod="openshift-marketplace/certified-operators-48pv8" Dec 03 11:33:49 crc kubenswrapper[5052]: I1203 11:33:49.863700 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/e5586a9d-6e6b-47f5-994b-80f9238d8a77-utilities\") pod \"certified-operators-48pv8\" (UID: \"e5586a9d-6e6b-47f5-994b-80f9238d8a77\") " pod="openshift-marketplace/certified-operators-48pv8" Dec 03 11:33:49 crc kubenswrapper[5052]: I1203 11:33:49.899317 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="metallb-system/speaker-56dnr" event={"ID":"92c3adf7-3dc8-4644-9fb7-f68994747973","Type":"ContainerStarted","Data":"91e2b544ddf9c2ee14953c16077b318fc9052bc36d70bcd6896d2f89a7f1beb2"} Dec 03 11:33:49 crc kubenswrapper[5052]: I1203 11:33:49.899371 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="metallb-system/speaker-56dnr" event={"ID":"92c3adf7-3dc8-4644-9fb7-f68994747973","Type":"ContainerStarted","Data":"b05c70953eb88feac792df0ceb57b8ebf3eb1b006633631dc87e198c47b3b0a2"} Dec 03 11:33:49 crc kubenswrapper[5052]: I1203 11:33:49.965088 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/e5586a9d-6e6b-47f5-994b-80f9238d8a77-utilities\") pod \"certified-operators-48pv8\" (UID: \"e5586a9d-6e6b-47f5-994b-80f9238d8a77\") " pod="openshift-marketplace/certified-operators-48pv8" Dec 03 11:33:49 crc kubenswrapper[5052]: I1203 11:33:49.965144 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-sgbkq\" (UniqueName: \"kubernetes.io/projected/e5586a9d-6e6b-47f5-994b-80f9238d8a77-kube-api-access-sgbkq\") pod \"certified-operators-48pv8\" (UID: \"e5586a9d-6e6b-47f5-994b-80f9238d8a77\") " pod="openshift-marketplace/certified-operators-48pv8" Dec 03 11:33:49 crc kubenswrapper[5052]: I1203 11:33:49.965161 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/e5586a9d-6e6b-47f5-994b-80f9238d8a77-catalog-content\") pod \"certified-operators-48pv8\" (UID: \"e5586a9d-6e6b-47f5-994b-80f9238d8a77\") " pod="openshift-marketplace/certified-operators-48pv8" Dec 03 11:33:49 crc kubenswrapper[5052]: I1203 11:33:49.965880 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/e5586a9d-6e6b-47f5-994b-80f9238d8a77-utilities\") pod \"certified-operators-48pv8\" (UID: \"e5586a9d-6e6b-47f5-994b-80f9238d8a77\") " pod="openshift-marketplace/certified-operators-48pv8" Dec 03 11:33:49 crc kubenswrapper[5052]: I1203 11:33:49.965941 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/e5586a9d-6e6b-47f5-994b-80f9238d8a77-catalog-content\") pod \"certified-operators-48pv8\" (UID: \"e5586a9d-6e6b-47f5-994b-80f9238d8a77\") " pod="openshift-marketplace/certified-operators-48pv8" Dec 03 11:33:49 crc kubenswrapper[5052]: I1203 11:33:49.987491 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-sgbkq\" (UniqueName: \"kubernetes.io/projected/e5586a9d-6e6b-47f5-994b-80f9238d8a77-kube-api-access-sgbkq\") pod \"certified-operators-48pv8\" (UID: \"e5586a9d-6e6b-47f5-994b-80f9238d8a77\") " pod="openshift-marketplace/certified-operators-48pv8" Dec 03 11:33:50 crc kubenswrapper[5052]: I1203 11:33:50.083388 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/certified-operators-48pv8" Dec 03 11:33:50 crc kubenswrapper[5052]: I1203 11:33:50.422912 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/certified-operators-48pv8"] Dec 03 11:33:50 crc kubenswrapper[5052]: I1203 11:33:50.911481 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="metallb-system/speaker-56dnr" event={"ID":"92c3adf7-3dc8-4644-9fb7-f68994747973","Type":"ContainerStarted","Data":"c9fb26c505123247a99715a1981b154f06713192756a7e35c0ba273ddb424953"} Dec 03 11:33:50 crc kubenswrapper[5052]: I1203 11:33:50.911781 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="metallb-system/speaker-56dnr" Dec 03 11:33:50 crc kubenswrapper[5052]: I1203 11:33:50.912722 5052 generic.go:334] "Generic (PLEG): container finished" podID="e5586a9d-6e6b-47f5-994b-80f9238d8a77" containerID="dff418811651fc032eae48f384fbbef75d30c3038ec6eca898a6a02edf1ab09a" exitCode=0 Dec 03 11:33:50 crc kubenswrapper[5052]: I1203 11:33:50.912747 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/certified-operators-48pv8" event={"ID":"e5586a9d-6e6b-47f5-994b-80f9238d8a77","Type":"ContainerDied","Data":"dff418811651fc032eae48f384fbbef75d30c3038ec6eca898a6a02edf1ab09a"} Dec 03 11:33:50 crc kubenswrapper[5052]: I1203 11:33:50.912770 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/certified-operators-48pv8" event={"ID":"e5586a9d-6e6b-47f5-994b-80f9238d8a77","Type":"ContainerStarted","Data":"49d380be2b61797fc052c8a7e8a98d84b41782bb8ad744aab768e53357277ddc"} Dec 03 11:33:50 crc kubenswrapper[5052]: I1203 11:33:50.940153 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="metallb-system/speaker-56dnr" podStartSLOduration=3.940133221 podStartE2EDuration="3.940133221s" podCreationTimestamp="2025-12-03 11:33:47 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:33:50.937482865 +0000 UTC m=+821.525724629" watchObservedRunningTime="2025-12-03 11:33:50.940133221 +0000 UTC m=+821.528374985" Dec 03 11:33:51 crc kubenswrapper[5052]: I1203 11:33:51.922441 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/certified-operators-48pv8" event={"ID":"e5586a9d-6e6b-47f5-994b-80f9238d8a77","Type":"ContainerStarted","Data":"d16a650d469fe0e6e3287e66a9edd55b461588461aab1708ace956fd769655c6"} Dec 03 11:33:52 crc kubenswrapper[5052]: I1203 11:33:52.935465 5052 generic.go:334] "Generic (PLEG): container finished" podID="e5586a9d-6e6b-47f5-994b-80f9238d8a77" containerID="d16a650d469fe0e6e3287e66a9edd55b461588461aab1708ace956fd769655c6" exitCode=0 Dec 03 11:33:52 crc kubenswrapper[5052]: I1203 11:33:52.935546 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/certified-operators-48pv8" event={"ID":"e5586a9d-6e6b-47f5-994b-80f9238d8a77","Type":"ContainerDied","Data":"d16a650d469fe0e6e3287e66a9edd55b461588461aab1708ace956fd769655c6"} Dec 03 11:33:55 crc kubenswrapper[5052]: I1203 11:33:55.954232 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="metallb-system/frr-k8s-webhook-server-7fcb986d4-66rvv" event={"ID":"c89171df-ff1e-4bc6-8e4e-9c24dc8ee436","Type":"ContainerStarted","Data":"1fb74867102e58a761053d25382a3ee0a4beeb8cd0a380a33bff95a575f0913d"} Dec 03 11:33:55 crc kubenswrapper[5052]: I1203 11:33:55.954904 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="metallb-system/frr-k8s-webhook-server-7fcb986d4-66rvv" Dec 03 11:33:55 crc kubenswrapper[5052]: I1203 11:33:55.956567 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/certified-operators-48pv8" event={"ID":"e5586a9d-6e6b-47f5-994b-80f9238d8a77","Type":"ContainerStarted","Data":"96dac7dba0cb90d44baf4546143fc1006073b0cfe86eecb8fedb368b7f43d716"} Dec 03 11:33:55 crc kubenswrapper[5052]: I1203 11:33:55.958340 5052 generic.go:334] "Generic (PLEG): container finished" podID="76bb2cd3-e75d-40f1-a9e6-63f7bd725eea" containerID="682922bfb3b8246d522284842f678a61e9b815776cd3c48e03364c35b0a7f5c2" exitCode=0 Dec 03 11:33:55 crc kubenswrapper[5052]: I1203 11:33:55.958371 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="metallb-system/frr-k8s-47pp5" event={"ID":"76bb2cd3-e75d-40f1-a9e6-63f7bd725eea","Type":"ContainerDied","Data":"682922bfb3b8246d522284842f678a61e9b815776cd3c48e03364c35b0a7f5c2"} Dec 03 11:33:55 crc kubenswrapper[5052]: I1203 11:33:55.972223 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="metallb-system/frr-k8s-webhook-server-7fcb986d4-66rvv" podStartSLOduration=2.017741854 podStartE2EDuration="8.972197307s" podCreationTimestamp="2025-12-03 11:33:47 +0000 UTC" firstStartedPulling="2025-12-03 11:33:48.636008932 +0000 UTC m=+819.224250696" lastFinishedPulling="2025-12-03 11:33:55.590464385 +0000 UTC m=+826.178706149" observedRunningTime="2025-12-03 11:33:55.9670353 +0000 UTC m=+826.555277064" watchObservedRunningTime="2025-12-03 11:33:55.972197307 +0000 UTC m=+826.560439071" Dec 03 11:33:56 crc kubenswrapper[5052]: I1203 11:33:56.012558 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-marketplace/certified-operators-48pv8" podStartSLOduration=2.362301351 podStartE2EDuration="7.012540812s" podCreationTimestamp="2025-12-03 11:33:49 +0000 UTC" firstStartedPulling="2025-12-03 11:33:50.913987753 +0000 UTC m=+821.502229527" lastFinishedPulling="2025-12-03 11:33:55.564227214 +0000 UTC m=+826.152468988" observedRunningTime="2025-12-03 11:33:56.00898989 +0000 UTC m=+826.597231654" watchObservedRunningTime="2025-12-03 11:33:56.012540812 +0000 UTC m=+826.600782576" Dec 03 11:33:56 crc kubenswrapper[5052]: I1203 11:33:56.966683 5052 generic.go:334] "Generic (PLEG): container finished" podID="76bb2cd3-e75d-40f1-a9e6-63f7bd725eea" containerID="bc7ce8046df0ccb82d63076728436da473ef5c15280398ca980642937541d98a" exitCode=0 Dec 03 11:33:56 crc kubenswrapper[5052]: I1203 11:33:56.967700 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="metallb-system/frr-k8s-47pp5" event={"ID":"76bb2cd3-e75d-40f1-a9e6-63f7bd725eea","Type":"ContainerDied","Data":"bc7ce8046df0ccb82d63076728436da473ef5c15280398ca980642937541d98a"} Dec 03 11:33:57 crc kubenswrapper[5052]: I1203 11:33:57.973127 5052 generic.go:334] "Generic (PLEG): container finished" podID="76bb2cd3-e75d-40f1-a9e6-63f7bd725eea" containerID="504756964c1d5ed42931cc38027dd5454f241fab432b8d43194042720df095c1" exitCode=0 Dec 03 11:33:57 crc kubenswrapper[5052]: I1203 11:33:57.973166 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="metallb-system/frr-k8s-47pp5" event={"ID":"76bb2cd3-e75d-40f1-a9e6-63f7bd725eea","Type":"ContainerDied","Data":"504756964c1d5ed42931cc38027dd5454f241fab432b8d43194042720df095c1"} Dec 03 11:33:58 crc kubenswrapper[5052]: I1203 11:33:58.983403 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="metallb-system/frr-k8s-47pp5" event={"ID":"76bb2cd3-e75d-40f1-a9e6-63f7bd725eea","Type":"ContainerStarted","Data":"18c6e84e43960f52698caa01523568f7b33ff01da1ade8c26f3880e46bea15bc"} Dec 03 11:33:59 crc kubenswrapper[5052]: I1203 11:33:59.376812 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="metallb-system/speaker-56dnr" Dec 03 11:33:59 crc kubenswrapper[5052]: I1203 11:33:59.993900 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="metallb-system/frr-k8s-47pp5" event={"ID":"76bb2cd3-e75d-40f1-a9e6-63f7bd725eea","Type":"ContainerStarted","Data":"9e0a224ad4cbbac3d1051955a108e4ff045f37b6c3d5db0fca76018901e85a18"} Dec 03 11:33:59 crc kubenswrapper[5052]: I1203 11:33:59.994158 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="metallb-system/frr-k8s-47pp5" event={"ID":"76bb2cd3-e75d-40f1-a9e6-63f7bd725eea","Type":"ContainerStarted","Data":"e9474a2509541797501b89bf9fc316562cb1cd9a65600bceff9ad3186f99a439"} Dec 03 11:33:59 crc kubenswrapper[5052]: I1203 11:33:59.994173 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="metallb-system/frr-k8s-47pp5" Dec 03 11:33:59 crc kubenswrapper[5052]: I1203 11:33:59.994183 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="metallb-system/frr-k8s-47pp5" event={"ID":"76bb2cd3-e75d-40f1-a9e6-63f7bd725eea","Type":"ContainerStarted","Data":"83ba5a69c1bbb0e406a0fc79c42ffb8d6a1a7b8384a8bf1377cd677616db1263"} Dec 03 11:33:59 crc kubenswrapper[5052]: I1203 11:33:59.994193 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="metallb-system/frr-k8s-47pp5" event={"ID":"76bb2cd3-e75d-40f1-a9e6-63f7bd725eea","Type":"ContainerStarted","Data":"62820c3c55256682920209f9dc76d03c1435c746ce6776a8764e63f9cc16c90c"} Dec 03 11:33:59 crc kubenswrapper[5052]: I1203 11:33:59.994202 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="metallb-system/frr-k8s-47pp5" event={"ID":"76bb2cd3-e75d-40f1-a9e6-63f7bd725eea","Type":"ContainerStarted","Data":"5fc41dbfb1aab898dd48504c39d67ab034d3d7dc452b97a482e71b3e902ec447"} Dec 03 11:34:00 crc kubenswrapper[5052]: I1203 11:34:00.017109 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="metallb-system/frr-k8s-47pp5" podStartSLOduration=6.013516578 podStartE2EDuration="13.017088766s" podCreationTimestamp="2025-12-03 11:33:47 +0000 UTC" firstStartedPulling="2025-12-03 11:33:48.560592694 +0000 UTC m=+819.148834458" lastFinishedPulling="2025-12-03 11:33:55.564164882 +0000 UTC m=+826.152406646" observedRunningTime="2025-12-03 11:34:00.011869066 +0000 UTC m=+830.600110840" watchObservedRunningTime="2025-12-03 11:34:00.017088766 +0000 UTC m=+830.605330540" Dec 03 11:34:00 crc kubenswrapper[5052]: I1203 11:34:00.083789 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openshift-marketplace/certified-operators-48pv8" Dec 03 11:34:00 crc kubenswrapper[5052]: I1203 11:34:00.083839 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-marketplace/certified-operators-48pv8" Dec 03 11:34:00 crc kubenswrapper[5052]: I1203 11:34:00.125765 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-marketplace/certified-operators-48pv8" Dec 03 11:34:00 crc kubenswrapper[5052]: I1203 11:34:00.796253 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openshift-marketplace/1f59f640c8a0eb1a7b0f26c81382bbdde784d03eb439a940bb8da3931aj27fz"] Dec 03 11:34:00 crc kubenswrapper[5052]: I1203 11:34:00.799252 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/1f59f640c8a0eb1a7b0f26c81382bbdde784d03eb439a940bb8da3931aj27fz" Dec 03 11:34:00 crc kubenswrapper[5052]: I1203 11:34:00.803173 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/1f59f640c8a0eb1a7b0f26c81382bbdde784d03eb439a940bb8da3931aj27fz"] Dec 03 11:34:00 crc kubenswrapper[5052]: I1203 11:34:00.807931 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-marketplace"/"default-dockercfg-vmwhc" Dec 03 11:34:00 crc kubenswrapper[5052]: I1203 11:34:00.860913 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c121fea4-2ada-4c07-8cbb-06d0ab09b325-bundle\") pod \"1f59f640c8a0eb1a7b0f26c81382bbdde784d03eb439a940bb8da3931aj27fz\" (UID: \"c121fea4-2ada-4c07-8cbb-06d0ab09b325\") " pod="openshift-marketplace/1f59f640c8a0eb1a7b0f26c81382bbdde784d03eb439a940bb8da3931aj27fz" Dec 03 11:34:00 crc kubenswrapper[5052]: I1203 11:34:00.861019 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mdgk5\" (UniqueName: \"kubernetes.io/projected/c121fea4-2ada-4c07-8cbb-06d0ab09b325-kube-api-access-mdgk5\") pod \"1f59f640c8a0eb1a7b0f26c81382bbdde784d03eb439a940bb8da3931aj27fz\" (UID: \"c121fea4-2ada-4c07-8cbb-06d0ab09b325\") " pod="openshift-marketplace/1f59f640c8a0eb1a7b0f26c81382bbdde784d03eb439a940bb8da3931aj27fz" Dec 03 11:34:00 crc kubenswrapper[5052]: I1203 11:34:00.861044 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c121fea4-2ada-4c07-8cbb-06d0ab09b325-util\") pod \"1f59f640c8a0eb1a7b0f26c81382bbdde784d03eb439a940bb8da3931aj27fz\" (UID: \"c121fea4-2ada-4c07-8cbb-06d0ab09b325\") " pod="openshift-marketplace/1f59f640c8a0eb1a7b0f26c81382bbdde784d03eb439a940bb8da3931aj27fz" Dec 03 11:34:00 crc kubenswrapper[5052]: I1203 11:34:00.962056 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-mdgk5\" (UniqueName: \"kubernetes.io/projected/c121fea4-2ada-4c07-8cbb-06d0ab09b325-kube-api-access-mdgk5\") pod \"1f59f640c8a0eb1a7b0f26c81382bbdde784d03eb439a940bb8da3931aj27fz\" (UID: \"c121fea4-2ada-4c07-8cbb-06d0ab09b325\") " pod="openshift-marketplace/1f59f640c8a0eb1a7b0f26c81382bbdde784d03eb439a940bb8da3931aj27fz" Dec 03 11:34:00 crc kubenswrapper[5052]: I1203 11:34:00.962116 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c121fea4-2ada-4c07-8cbb-06d0ab09b325-util\") pod \"1f59f640c8a0eb1a7b0f26c81382bbdde784d03eb439a940bb8da3931aj27fz\" (UID: \"c121fea4-2ada-4c07-8cbb-06d0ab09b325\") " pod="openshift-marketplace/1f59f640c8a0eb1a7b0f26c81382bbdde784d03eb439a940bb8da3931aj27fz" Dec 03 11:34:00 crc kubenswrapper[5052]: I1203 11:34:00.962172 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c121fea4-2ada-4c07-8cbb-06d0ab09b325-bundle\") pod \"1f59f640c8a0eb1a7b0f26c81382bbdde784d03eb439a940bb8da3931aj27fz\" (UID: \"c121fea4-2ada-4c07-8cbb-06d0ab09b325\") " pod="openshift-marketplace/1f59f640c8a0eb1a7b0f26c81382bbdde784d03eb439a940bb8da3931aj27fz" Dec 03 11:34:00 crc kubenswrapper[5052]: I1203 11:34:00.962682 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c121fea4-2ada-4c07-8cbb-06d0ab09b325-bundle\") pod \"1f59f640c8a0eb1a7b0f26c81382bbdde784d03eb439a940bb8da3931aj27fz\" (UID: \"c121fea4-2ada-4c07-8cbb-06d0ab09b325\") " pod="openshift-marketplace/1f59f640c8a0eb1a7b0f26c81382bbdde784d03eb439a940bb8da3931aj27fz" Dec 03 11:34:00 crc kubenswrapper[5052]: I1203 11:34:00.963054 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c121fea4-2ada-4c07-8cbb-06d0ab09b325-util\") pod \"1f59f640c8a0eb1a7b0f26c81382bbdde784d03eb439a940bb8da3931aj27fz\" (UID: \"c121fea4-2ada-4c07-8cbb-06d0ab09b325\") " pod="openshift-marketplace/1f59f640c8a0eb1a7b0f26c81382bbdde784d03eb439a940bb8da3931aj27fz" Dec 03 11:34:00 crc kubenswrapper[5052]: I1203 11:34:00.983818 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-mdgk5\" (UniqueName: \"kubernetes.io/projected/c121fea4-2ada-4c07-8cbb-06d0ab09b325-kube-api-access-mdgk5\") pod \"1f59f640c8a0eb1a7b0f26c81382bbdde784d03eb439a940bb8da3931aj27fz\" (UID: \"c121fea4-2ada-4c07-8cbb-06d0ab09b325\") " pod="openshift-marketplace/1f59f640c8a0eb1a7b0f26c81382bbdde784d03eb439a940bb8da3931aj27fz" Dec 03 11:34:01 crc kubenswrapper[5052]: I1203 11:34:01.036842 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-marketplace/certified-operators-48pv8" Dec 03 11:34:01 crc kubenswrapper[5052]: I1203 11:34:01.120779 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/1f59f640c8a0eb1a7b0f26c81382bbdde784d03eb439a940bb8da3931aj27fz" Dec 03 11:34:01 crc kubenswrapper[5052]: I1203 11:34:01.517986 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/1f59f640c8a0eb1a7b0f26c81382bbdde784d03eb439a940bb8da3931aj27fz"] Dec 03 11:34:02 crc kubenswrapper[5052]: I1203 11:34:02.005774 5052 generic.go:334] "Generic (PLEG): container finished" podID="c121fea4-2ada-4c07-8cbb-06d0ab09b325" containerID="38583b9d7bf4be96decf5afaf0cebe2637c317f5c748c1e59812f12f61012144" exitCode=0 Dec 03 11:34:02 crc kubenswrapper[5052]: I1203 11:34:02.005830 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/1f59f640c8a0eb1a7b0f26c81382bbdde784d03eb439a940bb8da3931aj27fz" event={"ID":"c121fea4-2ada-4c07-8cbb-06d0ab09b325","Type":"ContainerDied","Data":"38583b9d7bf4be96decf5afaf0cebe2637c317f5c748c1e59812f12f61012144"} Dec 03 11:34:02 crc kubenswrapper[5052]: I1203 11:34:02.006073 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/1f59f640c8a0eb1a7b0f26c81382bbdde784d03eb439a940bb8da3931aj27fz" event={"ID":"c121fea4-2ada-4c07-8cbb-06d0ab09b325","Type":"ContainerStarted","Data":"7cfb7776318aaeec5e8bddf6a4eb67c66a2302b5cd47b60ec8f91e421f1284cd"} Dec 03 11:34:03 crc kubenswrapper[5052]: I1203 11:34:03.375261 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="metallb-system/frr-k8s-47pp5" Dec 03 11:34:03 crc kubenswrapper[5052]: I1203 11:34:03.409797 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="metallb-system/frr-k8s-47pp5" Dec 03 11:34:04 crc kubenswrapper[5052]: I1203 11:34:04.554186 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openshift-marketplace/certified-operators-48pv8"] Dec 03 11:34:04 crc kubenswrapper[5052]: I1203 11:34:04.554813 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openshift-marketplace/certified-operators-48pv8" podUID="e5586a9d-6e6b-47f5-994b-80f9238d8a77" containerName="registry-server" containerID="cri-o://96dac7dba0cb90d44baf4546143fc1006073b0cfe86eecb8fedb368b7f43d716" gracePeriod=2 Dec 03 11:34:04 crc kubenswrapper[5052]: I1203 11:34:04.897868 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/certified-operators-48pv8" Dec 03 11:34:05 crc kubenswrapper[5052]: I1203 11:34:05.026999 5052 generic.go:334] "Generic (PLEG): container finished" podID="e5586a9d-6e6b-47f5-994b-80f9238d8a77" containerID="96dac7dba0cb90d44baf4546143fc1006073b0cfe86eecb8fedb368b7f43d716" exitCode=0 Dec 03 11:34:05 crc kubenswrapper[5052]: I1203 11:34:05.027062 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/certified-operators-48pv8" event={"ID":"e5586a9d-6e6b-47f5-994b-80f9238d8a77","Type":"ContainerDied","Data":"96dac7dba0cb90d44baf4546143fc1006073b0cfe86eecb8fedb368b7f43d716"} Dec 03 11:34:05 crc kubenswrapper[5052]: I1203 11:34:05.027088 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/certified-operators-48pv8" event={"ID":"e5586a9d-6e6b-47f5-994b-80f9238d8a77","Type":"ContainerDied","Data":"49d380be2b61797fc052c8a7e8a98d84b41782bb8ad744aab768e53357277ddc"} Dec 03 11:34:05 crc kubenswrapper[5052]: I1203 11:34:05.027104 5052 scope.go:117] "RemoveContainer" containerID="96dac7dba0cb90d44baf4546143fc1006073b0cfe86eecb8fedb368b7f43d716" Dec 03 11:34:05 crc kubenswrapper[5052]: I1203 11:34:05.027251 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/certified-operators-48pv8" Dec 03 11:34:05 crc kubenswrapper[5052]: I1203 11:34:05.029237 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/e5586a9d-6e6b-47f5-994b-80f9238d8a77-catalog-content\") pod \"e5586a9d-6e6b-47f5-994b-80f9238d8a77\" (UID: \"e5586a9d-6e6b-47f5-994b-80f9238d8a77\") " Dec 03 11:34:05 crc kubenswrapper[5052]: I1203 11:34:05.029545 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sgbkq\" (UniqueName: \"kubernetes.io/projected/e5586a9d-6e6b-47f5-994b-80f9238d8a77-kube-api-access-sgbkq\") pod \"e5586a9d-6e6b-47f5-994b-80f9238d8a77\" (UID: \"e5586a9d-6e6b-47f5-994b-80f9238d8a77\") " Dec 03 11:34:05 crc kubenswrapper[5052]: I1203 11:34:05.029593 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/e5586a9d-6e6b-47f5-994b-80f9238d8a77-utilities\") pod \"e5586a9d-6e6b-47f5-994b-80f9238d8a77\" (UID: \"e5586a9d-6e6b-47f5-994b-80f9238d8a77\") " Dec 03 11:34:05 crc kubenswrapper[5052]: I1203 11:34:05.031773 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e5586a9d-6e6b-47f5-994b-80f9238d8a77-utilities" (OuterVolumeSpecName: "utilities") pod "e5586a9d-6e6b-47f5-994b-80f9238d8a77" (UID: "e5586a9d-6e6b-47f5-994b-80f9238d8a77"). InnerVolumeSpecName "utilities". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:34:05 crc kubenswrapper[5052]: I1203 11:34:05.050072 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e5586a9d-6e6b-47f5-994b-80f9238d8a77-kube-api-access-sgbkq" (OuterVolumeSpecName: "kube-api-access-sgbkq") pod "e5586a9d-6e6b-47f5-994b-80f9238d8a77" (UID: "e5586a9d-6e6b-47f5-994b-80f9238d8a77"). InnerVolumeSpecName "kube-api-access-sgbkq". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:34:05 crc kubenswrapper[5052]: I1203 11:34:05.080708 5052 scope.go:117] "RemoveContainer" containerID="d16a650d469fe0e6e3287e66a9edd55b461588461aab1708ace956fd769655c6" Dec 03 11:34:05 crc kubenswrapper[5052]: I1203 11:34:05.081469 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e5586a9d-6e6b-47f5-994b-80f9238d8a77-catalog-content" (OuterVolumeSpecName: "catalog-content") pod "e5586a9d-6e6b-47f5-994b-80f9238d8a77" (UID: "e5586a9d-6e6b-47f5-994b-80f9238d8a77"). InnerVolumeSpecName "catalog-content". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:34:05 crc kubenswrapper[5052]: I1203 11:34:05.101412 5052 scope.go:117] "RemoveContainer" containerID="dff418811651fc032eae48f384fbbef75d30c3038ec6eca898a6a02edf1ab09a" Dec 03 11:34:05 crc kubenswrapper[5052]: I1203 11:34:05.121821 5052 scope.go:117] "RemoveContainer" containerID="96dac7dba0cb90d44baf4546143fc1006073b0cfe86eecb8fedb368b7f43d716" Dec 03 11:34:05 crc kubenswrapper[5052]: E1203 11:34:05.122254 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"96dac7dba0cb90d44baf4546143fc1006073b0cfe86eecb8fedb368b7f43d716\": container with ID starting with 96dac7dba0cb90d44baf4546143fc1006073b0cfe86eecb8fedb368b7f43d716 not found: ID does not exist" containerID="96dac7dba0cb90d44baf4546143fc1006073b0cfe86eecb8fedb368b7f43d716" Dec 03 11:34:05 crc kubenswrapper[5052]: I1203 11:34:05.122295 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"96dac7dba0cb90d44baf4546143fc1006073b0cfe86eecb8fedb368b7f43d716"} err="failed to get container status \"96dac7dba0cb90d44baf4546143fc1006073b0cfe86eecb8fedb368b7f43d716\": rpc error: code = NotFound desc = could not find container \"96dac7dba0cb90d44baf4546143fc1006073b0cfe86eecb8fedb368b7f43d716\": container with ID starting with 96dac7dba0cb90d44baf4546143fc1006073b0cfe86eecb8fedb368b7f43d716 not found: ID does not exist" Dec 03 11:34:05 crc kubenswrapper[5052]: I1203 11:34:05.122321 5052 scope.go:117] "RemoveContainer" containerID="d16a650d469fe0e6e3287e66a9edd55b461588461aab1708ace956fd769655c6" Dec 03 11:34:05 crc kubenswrapper[5052]: E1203 11:34:05.123597 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d16a650d469fe0e6e3287e66a9edd55b461588461aab1708ace956fd769655c6\": container with ID starting with d16a650d469fe0e6e3287e66a9edd55b461588461aab1708ace956fd769655c6 not found: ID does not exist" containerID="d16a650d469fe0e6e3287e66a9edd55b461588461aab1708ace956fd769655c6" Dec 03 11:34:05 crc kubenswrapper[5052]: I1203 11:34:05.123629 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d16a650d469fe0e6e3287e66a9edd55b461588461aab1708ace956fd769655c6"} err="failed to get container status \"d16a650d469fe0e6e3287e66a9edd55b461588461aab1708ace956fd769655c6\": rpc error: code = NotFound desc = could not find container \"d16a650d469fe0e6e3287e66a9edd55b461588461aab1708ace956fd769655c6\": container with ID starting with d16a650d469fe0e6e3287e66a9edd55b461588461aab1708ace956fd769655c6 not found: ID does not exist" Dec 03 11:34:05 crc kubenswrapper[5052]: I1203 11:34:05.123650 5052 scope.go:117] "RemoveContainer" containerID="dff418811651fc032eae48f384fbbef75d30c3038ec6eca898a6a02edf1ab09a" Dec 03 11:34:05 crc kubenswrapper[5052]: E1203 11:34:05.124018 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dff418811651fc032eae48f384fbbef75d30c3038ec6eca898a6a02edf1ab09a\": container with ID starting with dff418811651fc032eae48f384fbbef75d30c3038ec6eca898a6a02edf1ab09a not found: ID does not exist" containerID="dff418811651fc032eae48f384fbbef75d30c3038ec6eca898a6a02edf1ab09a" Dec 03 11:34:05 crc kubenswrapper[5052]: I1203 11:34:05.124052 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dff418811651fc032eae48f384fbbef75d30c3038ec6eca898a6a02edf1ab09a"} err="failed to get container status \"dff418811651fc032eae48f384fbbef75d30c3038ec6eca898a6a02edf1ab09a\": rpc error: code = NotFound desc = could not find container \"dff418811651fc032eae48f384fbbef75d30c3038ec6eca898a6a02edf1ab09a\": container with ID starting with dff418811651fc032eae48f384fbbef75d30c3038ec6eca898a6a02edf1ab09a not found: ID does not exist" Dec 03 11:34:05 crc kubenswrapper[5052]: I1203 11:34:05.130979 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-sgbkq\" (UniqueName: \"kubernetes.io/projected/e5586a9d-6e6b-47f5-994b-80f9238d8a77-kube-api-access-sgbkq\") on node \"crc\" DevicePath \"\"" Dec 03 11:34:05 crc kubenswrapper[5052]: I1203 11:34:05.131016 5052 reconciler_common.go:293] "Volume detached for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/e5586a9d-6e6b-47f5-994b-80f9238d8a77-utilities\") on node \"crc\" DevicePath \"\"" Dec 03 11:34:05 crc kubenswrapper[5052]: I1203 11:34:05.131032 5052 reconciler_common.go:293] "Volume detached for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/e5586a9d-6e6b-47f5-994b-80f9238d8a77-catalog-content\") on node \"crc\" DevicePath \"\"" Dec 03 11:34:05 crc kubenswrapper[5052]: I1203 11:34:05.355543 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openshift-marketplace/certified-operators-48pv8"] Dec 03 11:34:05 crc kubenswrapper[5052]: I1203 11:34:05.359627 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openshift-marketplace/certified-operators-48pv8"] Dec 03 11:34:05 crc kubenswrapper[5052]: I1203 11:34:05.683215 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e5586a9d-6e6b-47f5-994b-80f9238d8a77" path="/var/lib/kubelet/pods/e5586a9d-6e6b-47f5-994b-80f9238d8a77/volumes" Dec 03 11:34:07 crc kubenswrapper[5052]: I1203 11:34:07.042504 5052 generic.go:334] "Generic (PLEG): container finished" podID="c121fea4-2ada-4c07-8cbb-06d0ab09b325" containerID="57238964fc9f8aa4366036c7892ddd8b616df7ad3ba5b150d4569637b29228ca" exitCode=0 Dec 03 11:34:07 crc kubenswrapper[5052]: I1203 11:34:07.042680 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/1f59f640c8a0eb1a7b0f26c81382bbdde784d03eb439a940bb8da3931aj27fz" event={"ID":"c121fea4-2ada-4c07-8cbb-06d0ab09b325","Type":"ContainerDied","Data":"57238964fc9f8aa4366036c7892ddd8b616df7ad3ba5b150d4569637b29228ca"} Dec 03 11:34:07 crc kubenswrapper[5052]: I1203 11:34:07.172924 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openshift-marketplace/community-operators-dll6j"] Dec 03 11:34:07 crc kubenswrapper[5052]: E1203 11:34:07.173187 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="e5586a9d-6e6b-47f5-994b-80f9238d8a77" containerName="registry-server" Dec 03 11:34:07 crc kubenswrapper[5052]: I1203 11:34:07.173206 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="e5586a9d-6e6b-47f5-994b-80f9238d8a77" containerName="registry-server" Dec 03 11:34:07 crc kubenswrapper[5052]: E1203 11:34:07.173223 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="e5586a9d-6e6b-47f5-994b-80f9238d8a77" containerName="extract-content" Dec 03 11:34:07 crc kubenswrapper[5052]: I1203 11:34:07.173232 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="e5586a9d-6e6b-47f5-994b-80f9238d8a77" containerName="extract-content" Dec 03 11:34:07 crc kubenswrapper[5052]: E1203 11:34:07.173246 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="e5586a9d-6e6b-47f5-994b-80f9238d8a77" containerName="extract-utilities" Dec 03 11:34:07 crc kubenswrapper[5052]: I1203 11:34:07.173253 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="e5586a9d-6e6b-47f5-994b-80f9238d8a77" containerName="extract-utilities" Dec 03 11:34:07 crc kubenswrapper[5052]: I1203 11:34:07.173401 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="e5586a9d-6e6b-47f5-994b-80f9238d8a77" containerName="registry-server" Dec 03 11:34:07 crc kubenswrapper[5052]: I1203 11:34:07.174338 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/community-operators-dll6j" Dec 03 11:34:07 crc kubenswrapper[5052]: I1203 11:34:07.194387 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/community-operators-dll6j"] Dec 03 11:34:07 crc kubenswrapper[5052]: I1203 11:34:07.261717 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/2f8b0974-b7fd-4a8f-97d0-4bd784e11890-utilities\") pod \"community-operators-dll6j\" (UID: \"2f8b0974-b7fd-4a8f-97d0-4bd784e11890\") " pod="openshift-marketplace/community-operators-dll6j" Dec 03 11:34:07 crc kubenswrapper[5052]: I1203 11:34:07.261775 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rhl7d\" (UniqueName: \"kubernetes.io/projected/2f8b0974-b7fd-4a8f-97d0-4bd784e11890-kube-api-access-rhl7d\") pod \"community-operators-dll6j\" (UID: \"2f8b0974-b7fd-4a8f-97d0-4bd784e11890\") " pod="openshift-marketplace/community-operators-dll6j" Dec 03 11:34:07 crc kubenswrapper[5052]: I1203 11:34:07.261806 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/2f8b0974-b7fd-4a8f-97d0-4bd784e11890-catalog-content\") pod \"community-operators-dll6j\" (UID: \"2f8b0974-b7fd-4a8f-97d0-4bd784e11890\") " pod="openshift-marketplace/community-operators-dll6j" Dec 03 11:34:07 crc kubenswrapper[5052]: I1203 11:34:07.363210 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/2f8b0974-b7fd-4a8f-97d0-4bd784e11890-utilities\") pod \"community-operators-dll6j\" (UID: \"2f8b0974-b7fd-4a8f-97d0-4bd784e11890\") " pod="openshift-marketplace/community-operators-dll6j" Dec 03 11:34:07 crc kubenswrapper[5052]: I1203 11:34:07.363267 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-rhl7d\" (UniqueName: \"kubernetes.io/projected/2f8b0974-b7fd-4a8f-97d0-4bd784e11890-kube-api-access-rhl7d\") pod \"community-operators-dll6j\" (UID: \"2f8b0974-b7fd-4a8f-97d0-4bd784e11890\") " pod="openshift-marketplace/community-operators-dll6j" Dec 03 11:34:07 crc kubenswrapper[5052]: I1203 11:34:07.363299 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/2f8b0974-b7fd-4a8f-97d0-4bd784e11890-catalog-content\") pod \"community-operators-dll6j\" (UID: \"2f8b0974-b7fd-4a8f-97d0-4bd784e11890\") " pod="openshift-marketplace/community-operators-dll6j" Dec 03 11:34:07 crc kubenswrapper[5052]: I1203 11:34:07.363793 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/2f8b0974-b7fd-4a8f-97d0-4bd784e11890-utilities\") pod \"community-operators-dll6j\" (UID: \"2f8b0974-b7fd-4a8f-97d0-4bd784e11890\") " pod="openshift-marketplace/community-operators-dll6j" Dec 03 11:34:07 crc kubenswrapper[5052]: I1203 11:34:07.363826 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/2f8b0974-b7fd-4a8f-97d0-4bd784e11890-catalog-content\") pod \"community-operators-dll6j\" (UID: \"2f8b0974-b7fd-4a8f-97d0-4bd784e11890\") " pod="openshift-marketplace/community-operators-dll6j" Dec 03 11:34:07 crc kubenswrapper[5052]: I1203 11:34:07.379570 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-rhl7d\" (UniqueName: \"kubernetes.io/projected/2f8b0974-b7fd-4a8f-97d0-4bd784e11890-kube-api-access-rhl7d\") pod \"community-operators-dll6j\" (UID: \"2f8b0974-b7fd-4a8f-97d0-4bd784e11890\") " pod="openshift-marketplace/community-operators-dll6j" Dec 03 11:34:07 crc kubenswrapper[5052]: I1203 11:34:07.490838 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/community-operators-dll6j" Dec 03 11:34:07 crc kubenswrapper[5052]: I1203 11:34:07.780814 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/community-operators-dll6j"] Dec 03 11:34:07 crc kubenswrapper[5052]: I1203 11:34:07.899788 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="metallb-system/controller-f8648f98b-nc9sj" Dec 03 11:34:08 crc kubenswrapper[5052]: I1203 11:34:08.051886 5052 generic.go:334] "Generic (PLEG): container finished" podID="c121fea4-2ada-4c07-8cbb-06d0ab09b325" containerID="0707cf556877bf2b118279717f2c734bce24ee7f6418054f1bf2807f48fa54fc" exitCode=0 Dec 03 11:34:08 crc kubenswrapper[5052]: I1203 11:34:08.051926 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/1f59f640c8a0eb1a7b0f26c81382bbdde784d03eb439a940bb8da3931aj27fz" event={"ID":"c121fea4-2ada-4c07-8cbb-06d0ab09b325","Type":"ContainerDied","Data":"0707cf556877bf2b118279717f2c734bce24ee7f6418054f1bf2807f48fa54fc"} Dec 03 11:34:08 crc kubenswrapper[5052]: I1203 11:34:08.053655 5052 generic.go:334] "Generic (PLEG): container finished" podID="2f8b0974-b7fd-4a8f-97d0-4bd784e11890" containerID="3ad92e32a5c89995484db309c2d9ed3532d36dc612f720b170c34bd757148280" exitCode=0 Dec 03 11:34:08 crc kubenswrapper[5052]: I1203 11:34:08.053704 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/community-operators-dll6j" event={"ID":"2f8b0974-b7fd-4a8f-97d0-4bd784e11890","Type":"ContainerDied","Data":"3ad92e32a5c89995484db309c2d9ed3532d36dc612f720b170c34bd757148280"} Dec 03 11:34:08 crc kubenswrapper[5052]: I1203 11:34:08.053732 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/community-operators-dll6j" event={"ID":"2f8b0974-b7fd-4a8f-97d0-4bd784e11890","Type":"ContainerStarted","Data":"201cb812b360b9d3b63fd36c4479117550f46101a0186d44896fec830664aa02"} Dec 03 11:34:08 crc kubenswrapper[5052]: I1203 11:34:08.379618 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="metallb-system/frr-k8s-47pp5" Dec 03 11:34:08 crc kubenswrapper[5052]: I1203 11:34:08.390462 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="metallb-system/frr-k8s-webhook-server-7fcb986d4-66rvv" Dec 03 11:34:09 crc kubenswrapper[5052]: I1203 11:34:09.329442 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/1f59f640c8a0eb1a7b0f26c81382bbdde784d03eb439a940bb8da3931aj27fz" Dec 03 11:34:09 crc kubenswrapper[5052]: I1203 11:34:09.391274 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c121fea4-2ada-4c07-8cbb-06d0ab09b325-bundle\") pod \"c121fea4-2ada-4c07-8cbb-06d0ab09b325\" (UID: \"c121fea4-2ada-4c07-8cbb-06d0ab09b325\") " Dec 03 11:34:09 crc kubenswrapper[5052]: I1203 11:34:09.391322 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mdgk5\" (UniqueName: \"kubernetes.io/projected/c121fea4-2ada-4c07-8cbb-06d0ab09b325-kube-api-access-mdgk5\") pod \"c121fea4-2ada-4c07-8cbb-06d0ab09b325\" (UID: \"c121fea4-2ada-4c07-8cbb-06d0ab09b325\") " Dec 03 11:34:09 crc kubenswrapper[5052]: I1203 11:34:09.391404 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c121fea4-2ada-4c07-8cbb-06d0ab09b325-util\") pod \"c121fea4-2ada-4c07-8cbb-06d0ab09b325\" (UID: \"c121fea4-2ada-4c07-8cbb-06d0ab09b325\") " Dec 03 11:34:09 crc kubenswrapper[5052]: I1203 11:34:09.392450 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c121fea4-2ada-4c07-8cbb-06d0ab09b325-bundle" (OuterVolumeSpecName: "bundle") pod "c121fea4-2ada-4c07-8cbb-06d0ab09b325" (UID: "c121fea4-2ada-4c07-8cbb-06d0ab09b325"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:34:09 crc kubenswrapper[5052]: I1203 11:34:09.404737 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c121fea4-2ada-4c07-8cbb-06d0ab09b325-kube-api-access-mdgk5" (OuterVolumeSpecName: "kube-api-access-mdgk5") pod "c121fea4-2ada-4c07-8cbb-06d0ab09b325" (UID: "c121fea4-2ada-4c07-8cbb-06d0ab09b325"). InnerVolumeSpecName "kube-api-access-mdgk5". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:34:09 crc kubenswrapper[5052]: I1203 11:34:09.406042 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c121fea4-2ada-4c07-8cbb-06d0ab09b325-util" (OuterVolumeSpecName: "util") pod "c121fea4-2ada-4c07-8cbb-06d0ab09b325" (UID: "c121fea4-2ada-4c07-8cbb-06d0ab09b325"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:34:09 crc kubenswrapper[5052]: I1203 11:34:09.493400 5052 reconciler_common.go:293] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c121fea4-2ada-4c07-8cbb-06d0ab09b325-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:34:09 crc kubenswrapper[5052]: I1203 11:34:09.493466 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-mdgk5\" (UniqueName: \"kubernetes.io/projected/c121fea4-2ada-4c07-8cbb-06d0ab09b325-kube-api-access-mdgk5\") on node \"crc\" DevicePath \"\"" Dec 03 11:34:09 crc kubenswrapper[5052]: I1203 11:34:09.493480 5052 reconciler_common.go:293] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c121fea4-2ada-4c07-8cbb-06d0ab09b325-util\") on node \"crc\" DevicePath \"\"" Dec 03 11:34:10 crc kubenswrapper[5052]: I1203 11:34:10.084775 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/1f59f640c8a0eb1a7b0f26c81382bbdde784d03eb439a940bb8da3931aj27fz" event={"ID":"c121fea4-2ada-4c07-8cbb-06d0ab09b325","Type":"ContainerDied","Data":"7cfb7776318aaeec5e8bddf6a4eb67c66a2302b5cd47b60ec8f91e421f1284cd"} Dec 03 11:34:10 crc kubenswrapper[5052]: I1203 11:34:10.085208 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7cfb7776318aaeec5e8bddf6a4eb67c66a2302b5cd47b60ec8f91e421f1284cd" Dec 03 11:34:10 crc kubenswrapper[5052]: I1203 11:34:10.084843 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/1f59f640c8a0eb1a7b0f26c81382bbdde784d03eb439a940bb8da3931aj27fz" Dec 03 11:34:10 crc kubenswrapper[5052]: I1203 11:34:10.088246 5052 generic.go:334] "Generic (PLEG): container finished" podID="2f8b0974-b7fd-4a8f-97d0-4bd784e11890" containerID="5dc3691559f95f90aa2fbb00935a450325fcf4c54f11c6a158205e492efe6fff" exitCode=0 Dec 03 11:34:10 crc kubenswrapper[5052]: I1203 11:34:10.088320 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/community-operators-dll6j" event={"ID":"2f8b0974-b7fd-4a8f-97d0-4bd784e11890","Type":"ContainerDied","Data":"5dc3691559f95f90aa2fbb00935a450325fcf4c54f11c6a158205e492efe6fff"} Dec 03 11:34:11 crc kubenswrapper[5052]: I1203 11:34:11.095233 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/community-operators-dll6j" event={"ID":"2f8b0974-b7fd-4a8f-97d0-4bd784e11890","Type":"ContainerStarted","Data":"3d0a11780f6311729df0727ab9e68a7857c0b4bfcdfa754a2b2d212a7cc9c670"} Dec 03 11:34:11 crc kubenswrapper[5052]: I1203 11:34:11.119038 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-marketplace/community-operators-dll6j" podStartSLOduration=1.709039174 podStartE2EDuration="4.119017122s" podCreationTimestamp="2025-12-03 11:34:07 +0000 UTC" firstStartedPulling="2025-12-03 11:34:08.055950807 +0000 UTC m=+838.644192571" lastFinishedPulling="2025-12-03 11:34:10.465928745 +0000 UTC m=+841.054170519" observedRunningTime="2025-12-03 11:34:11.114467202 +0000 UTC m=+841.702708966" watchObservedRunningTime="2025-12-03 11:34:11.119017122 +0000 UTC m=+841.707258886" Dec 03 11:34:13 crc kubenswrapper[5052]: I1203 11:34:13.959350 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-64cf6dff88-bzrrx"] Dec 03 11:34:13 crc kubenswrapper[5052]: E1203 11:34:13.959708 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="c121fea4-2ada-4c07-8cbb-06d0ab09b325" containerName="pull" Dec 03 11:34:13 crc kubenswrapper[5052]: I1203 11:34:13.959728 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="c121fea4-2ada-4c07-8cbb-06d0ab09b325" containerName="pull" Dec 03 11:34:13 crc kubenswrapper[5052]: E1203 11:34:13.959750 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="c121fea4-2ada-4c07-8cbb-06d0ab09b325" containerName="util" Dec 03 11:34:13 crc kubenswrapper[5052]: I1203 11:34:13.959761 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="c121fea4-2ada-4c07-8cbb-06d0ab09b325" containerName="util" Dec 03 11:34:13 crc kubenswrapper[5052]: E1203 11:34:13.959795 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="c121fea4-2ada-4c07-8cbb-06d0ab09b325" containerName="extract" Dec 03 11:34:13 crc kubenswrapper[5052]: I1203 11:34:13.959807 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="c121fea4-2ada-4c07-8cbb-06d0ab09b325" containerName="extract" Dec 03 11:34:13 crc kubenswrapper[5052]: I1203 11:34:13.959982 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="c121fea4-2ada-4c07-8cbb-06d0ab09b325" containerName="extract" Dec 03 11:34:13 crc kubenswrapper[5052]: I1203 11:34:13.960624 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-64cf6dff88-bzrrx" Dec 03 11:34:13 crc kubenswrapper[5052]: I1203 11:34:13.962739 5052 reflector.go:368] Caches populated for *v1.Secret from object-"cert-manager-operator"/"cert-manager-operator-controller-manager-dockercfg-jhqcq" Dec 03 11:34:13 crc kubenswrapper[5052]: I1203 11:34:13.962958 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"cert-manager-operator"/"openshift-service-ca.crt" Dec 03 11:34:13 crc kubenswrapper[5052]: I1203 11:34:13.963148 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"cert-manager-operator"/"kube-root-ca.crt" Dec 03 11:34:13 crc kubenswrapper[5052]: I1203 11:34:13.976667 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-64cf6dff88-bzrrx"] Dec 03 11:34:14 crc kubenswrapper[5052]: I1203 11:34:14.052235 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9b7700d3-21d0-40ba-87c7-e5125b298577-tmp\") pod \"cert-manager-operator-controller-manager-64cf6dff88-bzrrx\" (UID: \"9b7700d3-21d0-40ba-87c7-e5125b298577\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-64cf6dff88-bzrrx" Dec 03 11:34:14 crc kubenswrapper[5052]: I1203 11:34:14.052371 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9kh66\" (UniqueName: \"kubernetes.io/projected/9b7700d3-21d0-40ba-87c7-e5125b298577-kube-api-access-9kh66\") pod \"cert-manager-operator-controller-manager-64cf6dff88-bzrrx\" (UID: \"9b7700d3-21d0-40ba-87c7-e5125b298577\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-64cf6dff88-bzrrx" Dec 03 11:34:14 crc kubenswrapper[5052]: I1203 11:34:14.153159 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9b7700d3-21d0-40ba-87c7-e5125b298577-tmp\") pod \"cert-manager-operator-controller-manager-64cf6dff88-bzrrx\" (UID: \"9b7700d3-21d0-40ba-87c7-e5125b298577\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-64cf6dff88-bzrrx" Dec 03 11:34:14 crc kubenswrapper[5052]: I1203 11:34:14.153252 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-9kh66\" (UniqueName: \"kubernetes.io/projected/9b7700d3-21d0-40ba-87c7-e5125b298577-kube-api-access-9kh66\") pod \"cert-manager-operator-controller-manager-64cf6dff88-bzrrx\" (UID: \"9b7700d3-21d0-40ba-87c7-e5125b298577\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-64cf6dff88-bzrrx" Dec 03 11:34:14 crc kubenswrapper[5052]: I1203 11:34:14.153766 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9b7700d3-21d0-40ba-87c7-e5125b298577-tmp\") pod \"cert-manager-operator-controller-manager-64cf6dff88-bzrrx\" (UID: \"9b7700d3-21d0-40ba-87c7-e5125b298577\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-64cf6dff88-bzrrx" Dec 03 11:34:14 crc kubenswrapper[5052]: I1203 11:34:14.174206 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-9kh66\" (UniqueName: \"kubernetes.io/projected/9b7700d3-21d0-40ba-87c7-e5125b298577-kube-api-access-9kh66\") pod \"cert-manager-operator-controller-manager-64cf6dff88-bzrrx\" (UID: \"9b7700d3-21d0-40ba-87c7-e5125b298577\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-64cf6dff88-bzrrx" Dec 03 11:34:14 crc kubenswrapper[5052]: I1203 11:34:14.279969 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-64cf6dff88-bzrrx" Dec 03 11:34:14 crc kubenswrapper[5052]: I1203 11:34:14.769870 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-64cf6dff88-bzrrx"] Dec 03 11:34:15 crc kubenswrapper[5052]: I1203 11:34:15.115262 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-64cf6dff88-bzrrx" event={"ID":"9b7700d3-21d0-40ba-87c7-e5125b298577","Type":"ContainerStarted","Data":"b1b8c7cf85349bd3934613a79b287b8e094dfd4c8f74270c076347a31c90e725"} Dec 03 11:34:16 crc kubenswrapper[5052]: I1203 11:34:16.760871 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openshift-marketplace/redhat-marketplace-tk9vg"] Dec 03 11:34:16 crc kubenswrapper[5052]: I1203 11:34:16.762256 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/redhat-marketplace-tk9vg" Dec 03 11:34:16 crc kubenswrapper[5052]: I1203 11:34:16.787835 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/redhat-marketplace-tk9vg"] Dec 03 11:34:16 crc kubenswrapper[5052]: I1203 11:34:16.896064 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qm87r\" (UniqueName: \"kubernetes.io/projected/f297bb86-edee-4810-bc3f-c06f14a948b4-kube-api-access-qm87r\") pod \"redhat-marketplace-tk9vg\" (UID: \"f297bb86-edee-4810-bc3f-c06f14a948b4\") " pod="openshift-marketplace/redhat-marketplace-tk9vg" Dec 03 11:34:16 crc kubenswrapper[5052]: I1203 11:34:16.896136 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/f297bb86-edee-4810-bc3f-c06f14a948b4-utilities\") pod \"redhat-marketplace-tk9vg\" (UID: \"f297bb86-edee-4810-bc3f-c06f14a948b4\") " pod="openshift-marketplace/redhat-marketplace-tk9vg" Dec 03 11:34:16 crc kubenswrapper[5052]: I1203 11:34:16.896161 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/f297bb86-edee-4810-bc3f-c06f14a948b4-catalog-content\") pod \"redhat-marketplace-tk9vg\" (UID: \"f297bb86-edee-4810-bc3f-c06f14a948b4\") " pod="openshift-marketplace/redhat-marketplace-tk9vg" Dec 03 11:34:16 crc kubenswrapper[5052]: I1203 11:34:16.997540 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/f297bb86-edee-4810-bc3f-c06f14a948b4-utilities\") pod \"redhat-marketplace-tk9vg\" (UID: \"f297bb86-edee-4810-bc3f-c06f14a948b4\") " pod="openshift-marketplace/redhat-marketplace-tk9vg" Dec 03 11:34:16 crc kubenswrapper[5052]: I1203 11:34:16.997588 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/f297bb86-edee-4810-bc3f-c06f14a948b4-catalog-content\") pod \"redhat-marketplace-tk9vg\" (UID: \"f297bb86-edee-4810-bc3f-c06f14a948b4\") " pod="openshift-marketplace/redhat-marketplace-tk9vg" Dec 03 11:34:16 crc kubenswrapper[5052]: I1203 11:34:16.997681 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-qm87r\" (UniqueName: \"kubernetes.io/projected/f297bb86-edee-4810-bc3f-c06f14a948b4-kube-api-access-qm87r\") pod \"redhat-marketplace-tk9vg\" (UID: \"f297bb86-edee-4810-bc3f-c06f14a948b4\") " pod="openshift-marketplace/redhat-marketplace-tk9vg" Dec 03 11:34:16 crc kubenswrapper[5052]: I1203 11:34:16.998441 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/f297bb86-edee-4810-bc3f-c06f14a948b4-utilities\") pod \"redhat-marketplace-tk9vg\" (UID: \"f297bb86-edee-4810-bc3f-c06f14a948b4\") " pod="openshift-marketplace/redhat-marketplace-tk9vg" Dec 03 11:34:16 crc kubenswrapper[5052]: I1203 11:34:16.998779 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/f297bb86-edee-4810-bc3f-c06f14a948b4-catalog-content\") pod \"redhat-marketplace-tk9vg\" (UID: \"f297bb86-edee-4810-bc3f-c06f14a948b4\") " pod="openshift-marketplace/redhat-marketplace-tk9vg" Dec 03 11:34:17 crc kubenswrapper[5052]: I1203 11:34:17.048325 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-qm87r\" (UniqueName: \"kubernetes.io/projected/f297bb86-edee-4810-bc3f-c06f14a948b4-kube-api-access-qm87r\") pod \"redhat-marketplace-tk9vg\" (UID: \"f297bb86-edee-4810-bc3f-c06f14a948b4\") " pod="openshift-marketplace/redhat-marketplace-tk9vg" Dec 03 11:34:17 crc kubenswrapper[5052]: I1203 11:34:17.098335 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/redhat-marketplace-tk9vg" Dec 03 11:34:17 crc kubenswrapper[5052]: I1203 11:34:17.317329 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/redhat-marketplace-tk9vg"] Dec 03 11:34:17 crc kubenswrapper[5052]: I1203 11:34:17.491701 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openshift-marketplace/community-operators-dll6j" Dec 03 11:34:17 crc kubenswrapper[5052]: I1203 11:34:17.491751 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-marketplace/community-operators-dll6j" Dec 03 11:34:17 crc kubenswrapper[5052]: I1203 11:34:17.532135 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-marketplace/community-operators-dll6j" Dec 03 11:34:18 crc kubenswrapper[5052]: I1203 11:34:18.150102 5052 generic.go:334] "Generic (PLEG): container finished" podID="f297bb86-edee-4810-bc3f-c06f14a948b4" containerID="236fd310e3f77946ec8dc2a427332b92676fabb309409a1c75e816ac477da677" exitCode=0 Dec 03 11:34:18 crc kubenswrapper[5052]: I1203 11:34:18.151646 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-marketplace-tk9vg" event={"ID":"f297bb86-edee-4810-bc3f-c06f14a948b4","Type":"ContainerDied","Data":"236fd310e3f77946ec8dc2a427332b92676fabb309409a1c75e816ac477da677"} Dec 03 11:34:18 crc kubenswrapper[5052]: I1203 11:34:18.151682 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-marketplace-tk9vg" event={"ID":"f297bb86-edee-4810-bc3f-c06f14a948b4","Type":"ContainerStarted","Data":"3bc47de67725b9a2d0750f104399c4e9614be7e85c75ac841a1ffceb2adc0540"} Dec 03 11:34:18 crc kubenswrapper[5052]: I1203 11:34:18.198912 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-marketplace/community-operators-dll6j" Dec 03 11:34:21 crc kubenswrapper[5052]: I1203 11:34:21.154620 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openshift-marketplace/community-operators-dll6j"] Dec 03 11:34:21 crc kubenswrapper[5052]: I1203 11:34:21.155237 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openshift-marketplace/community-operators-dll6j" podUID="2f8b0974-b7fd-4a8f-97d0-4bd784e11890" containerName="registry-server" containerID="cri-o://3d0a11780f6311729df0727ab9e68a7857c0b4bfcdfa754a2b2d212a7cc9c670" gracePeriod=2 Dec 03 11:34:22 crc kubenswrapper[5052]: I1203 11:34:22.175869 5052 generic.go:334] "Generic (PLEG): container finished" podID="2f8b0974-b7fd-4a8f-97d0-4bd784e11890" containerID="3d0a11780f6311729df0727ab9e68a7857c0b4bfcdfa754a2b2d212a7cc9c670" exitCode=0 Dec 03 11:34:22 crc kubenswrapper[5052]: I1203 11:34:22.176323 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/community-operators-dll6j" event={"ID":"2f8b0974-b7fd-4a8f-97d0-4bd784e11890","Type":"ContainerDied","Data":"3d0a11780f6311729df0727ab9e68a7857c0b4bfcdfa754a2b2d212a7cc9c670"} Dec 03 11:34:22 crc kubenswrapper[5052]: I1203 11:34:22.628588 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/community-operators-dll6j" Dec 03 11:34:22 crc kubenswrapper[5052]: I1203 11:34:22.680564 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rhl7d\" (UniqueName: \"kubernetes.io/projected/2f8b0974-b7fd-4a8f-97d0-4bd784e11890-kube-api-access-rhl7d\") pod \"2f8b0974-b7fd-4a8f-97d0-4bd784e11890\" (UID: \"2f8b0974-b7fd-4a8f-97d0-4bd784e11890\") " Dec 03 11:34:22 crc kubenswrapper[5052]: I1203 11:34:22.680694 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/2f8b0974-b7fd-4a8f-97d0-4bd784e11890-catalog-content\") pod \"2f8b0974-b7fd-4a8f-97d0-4bd784e11890\" (UID: \"2f8b0974-b7fd-4a8f-97d0-4bd784e11890\") " Dec 03 11:34:22 crc kubenswrapper[5052]: I1203 11:34:22.680781 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/2f8b0974-b7fd-4a8f-97d0-4bd784e11890-utilities\") pod \"2f8b0974-b7fd-4a8f-97d0-4bd784e11890\" (UID: \"2f8b0974-b7fd-4a8f-97d0-4bd784e11890\") " Dec 03 11:34:22 crc kubenswrapper[5052]: I1203 11:34:22.681747 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f8b0974-b7fd-4a8f-97d0-4bd784e11890-utilities" (OuterVolumeSpecName: "utilities") pod "2f8b0974-b7fd-4a8f-97d0-4bd784e11890" (UID: "2f8b0974-b7fd-4a8f-97d0-4bd784e11890"). InnerVolumeSpecName "utilities". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:34:22 crc kubenswrapper[5052]: I1203 11:34:22.686378 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2f8b0974-b7fd-4a8f-97d0-4bd784e11890-kube-api-access-rhl7d" (OuterVolumeSpecName: "kube-api-access-rhl7d") pod "2f8b0974-b7fd-4a8f-97d0-4bd784e11890" (UID: "2f8b0974-b7fd-4a8f-97d0-4bd784e11890"). InnerVolumeSpecName "kube-api-access-rhl7d". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:34:22 crc kubenswrapper[5052]: I1203 11:34:22.733438 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f8b0974-b7fd-4a8f-97d0-4bd784e11890-catalog-content" (OuterVolumeSpecName: "catalog-content") pod "2f8b0974-b7fd-4a8f-97d0-4bd784e11890" (UID: "2f8b0974-b7fd-4a8f-97d0-4bd784e11890"). InnerVolumeSpecName "catalog-content". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:34:22 crc kubenswrapper[5052]: I1203 11:34:22.781885 5052 reconciler_common.go:293] "Volume detached for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/2f8b0974-b7fd-4a8f-97d0-4bd784e11890-catalog-content\") on node \"crc\" DevicePath \"\"" Dec 03 11:34:22 crc kubenswrapper[5052]: I1203 11:34:22.781933 5052 reconciler_common.go:293] "Volume detached for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/2f8b0974-b7fd-4a8f-97d0-4bd784e11890-utilities\") on node \"crc\" DevicePath \"\"" Dec 03 11:34:22 crc kubenswrapper[5052]: I1203 11:34:22.781944 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-rhl7d\" (UniqueName: \"kubernetes.io/projected/2f8b0974-b7fd-4a8f-97d0-4bd784e11890-kube-api-access-rhl7d\") on node \"crc\" DevicePath \"\"" Dec 03 11:34:23 crc kubenswrapper[5052]: I1203 11:34:23.183100 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/community-operators-dll6j" event={"ID":"2f8b0974-b7fd-4a8f-97d0-4bd784e11890","Type":"ContainerDied","Data":"201cb812b360b9d3b63fd36c4479117550f46101a0186d44896fec830664aa02"} Dec 03 11:34:23 crc kubenswrapper[5052]: I1203 11:34:23.183457 5052 scope.go:117] "RemoveContainer" containerID="3d0a11780f6311729df0727ab9e68a7857c0b4bfcdfa754a2b2d212a7cc9c670" Dec 03 11:34:23 crc kubenswrapper[5052]: I1203 11:34:23.183149 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/community-operators-dll6j" Dec 03 11:34:23 crc kubenswrapper[5052]: I1203 11:34:23.196284 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-64cf6dff88-bzrrx" event={"ID":"9b7700d3-21d0-40ba-87c7-e5125b298577","Type":"ContainerStarted","Data":"082bb6df009022131a57882a9d0912efe5f9af4d3cd2be52302207dfb36458a5"} Dec 03 11:34:23 crc kubenswrapper[5052]: I1203 11:34:23.199759 5052 generic.go:334] "Generic (PLEG): container finished" podID="f297bb86-edee-4810-bc3f-c06f14a948b4" containerID="36ff07bffd1a5968305005e1cfbeb683b230bebdaac6e58c2cecf8794106a72e" exitCode=0 Dec 03 11:34:23 crc kubenswrapper[5052]: I1203 11:34:23.199792 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-marketplace-tk9vg" event={"ID":"f297bb86-edee-4810-bc3f-c06f14a948b4","Type":"ContainerDied","Data":"36ff07bffd1a5968305005e1cfbeb683b230bebdaac6e58c2cecf8794106a72e"} Dec 03 11:34:23 crc kubenswrapper[5052]: I1203 11:34:23.209595 5052 scope.go:117] "RemoveContainer" containerID="5dc3691559f95f90aa2fbb00935a450325fcf4c54f11c6a158205e492efe6fff" Dec 03 11:34:23 crc kubenswrapper[5052]: I1203 11:34:23.214686 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-64cf6dff88-bzrrx" podStartSLOduration=2.295039771 podStartE2EDuration="10.214668326s" podCreationTimestamp="2025-12-03 11:34:13 +0000 UTC" firstStartedPulling="2025-12-03 11:34:14.785622919 +0000 UTC m=+845.373864683" lastFinishedPulling="2025-12-03 11:34:22.705251434 +0000 UTC m=+853.293493238" observedRunningTime="2025-12-03 11:34:23.214424619 +0000 UTC m=+853.802666403" watchObservedRunningTime="2025-12-03 11:34:23.214668326 +0000 UTC m=+853.802910090" Dec 03 11:34:23 crc kubenswrapper[5052]: I1203 11:34:23.230065 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openshift-marketplace/community-operators-dll6j"] Dec 03 11:34:23 crc kubenswrapper[5052]: I1203 11:34:23.233837 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openshift-marketplace/community-operators-dll6j"] Dec 03 11:34:23 crc kubenswrapper[5052]: I1203 11:34:23.261283 5052 scope.go:117] "RemoveContainer" containerID="3ad92e32a5c89995484db309c2d9ed3532d36dc612f720b170c34bd757148280" Dec 03 11:34:23 crc kubenswrapper[5052]: I1203 11:34:23.680198 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2f8b0974-b7fd-4a8f-97d0-4bd784e11890" path="/var/lib/kubelet/pods/2f8b0974-b7fd-4a8f-97d0-4bd784e11890/volumes" Dec 03 11:34:24 crc kubenswrapper[5052]: I1203 11:34:24.217953 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-marketplace-tk9vg" event={"ID":"f297bb86-edee-4810-bc3f-c06f14a948b4","Type":"ContainerStarted","Data":"1cce53245fe88f95f161830f068e2fcd4daa5221956a178a0a6e935bf7d56f98"} Dec 03 11:34:24 crc kubenswrapper[5052]: I1203 11:34:24.269168 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-marketplace/redhat-marketplace-tk9vg" podStartSLOduration=2.481270987 podStartE2EDuration="8.269145059s" podCreationTimestamp="2025-12-03 11:34:16 +0000 UTC" firstStartedPulling="2025-12-03 11:34:18.153244353 +0000 UTC m=+848.741486117" lastFinishedPulling="2025-12-03 11:34:23.941118425 +0000 UTC m=+854.529360189" observedRunningTime="2025-12-03 11:34:24.264876286 +0000 UTC m=+854.853118050" watchObservedRunningTime="2025-12-03 11:34:24.269145059 +0000 UTC m=+854.857386823" Dec 03 11:34:27 crc kubenswrapper[5052]: I1203 11:34:27.098998 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-marketplace/redhat-marketplace-tk9vg" Dec 03 11:34:27 crc kubenswrapper[5052]: I1203 11:34:27.100257 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openshift-marketplace/redhat-marketplace-tk9vg" Dec 03 11:34:27 crc kubenswrapper[5052]: I1203 11:34:27.150590 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-marketplace/redhat-marketplace-tk9vg" Dec 03 11:34:27 crc kubenswrapper[5052]: I1203 11:34:27.207801 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-f4fb5df64-z8l9s"] Dec 03 11:34:27 crc kubenswrapper[5052]: E1203 11:34:27.208071 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="2f8b0974-b7fd-4a8f-97d0-4bd784e11890" containerName="extract-content" Dec 03 11:34:27 crc kubenswrapper[5052]: I1203 11:34:27.208089 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f8b0974-b7fd-4a8f-97d0-4bd784e11890" containerName="extract-content" Dec 03 11:34:27 crc kubenswrapper[5052]: E1203 11:34:27.208108 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="2f8b0974-b7fd-4a8f-97d0-4bd784e11890" containerName="registry-server" Dec 03 11:34:27 crc kubenswrapper[5052]: I1203 11:34:27.208114 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f8b0974-b7fd-4a8f-97d0-4bd784e11890" containerName="registry-server" Dec 03 11:34:27 crc kubenswrapper[5052]: E1203 11:34:27.208124 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="2f8b0974-b7fd-4a8f-97d0-4bd784e11890" containerName="extract-utilities" Dec 03 11:34:27 crc kubenswrapper[5052]: I1203 11:34:27.208131 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f8b0974-b7fd-4a8f-97d0-4bd784e11890" containerName="extract-utilities" Dec 03 11:34:27 crc kubenswrapper[5052]: I1203 11:34:27.208230 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="2f8b0974-b7fd-4a8f-97d0-4bd784e11890" containerName="registry-server" Dec 03 11:34:27 crc kubenswrapper[5052]: I1203 11:34:27.208609 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-f4fb5df64-z8l9s" Dec 03 11:34:27 crc kubenswrapper[5052]: I1203 11:34:27.210729 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"cert-manager"/"openshift-service-ca.crt" Dec 03 11:34:27 crc kubenswrapper[5052]: I1203 11:34:27.210903 5052 reflector.go:368] Caches populated for *v1.Secret from object-"cert-manager"/"cert-manager-webhook-dockercfg-wdr8r" Dec 03 11:34:27 crc kubenswrapper[5052]: I1203 11:34:27.210848 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"cert-manager"/"kube-root-ca.crt" Dec 03 11:34:27 crc kubenswrapper[5052]: I1203 11:34:27.221583 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-f4fb5df64-z8l9s"] Dec 03 11:34:27 crc kubenswrapper[5052]: I1203 11:34:27.264686 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cgrqb\" (UniqueName: \"kubernetes.io/projected/cca488df-be0b-41f9-bd69-edf11dd061be-kube-api-access-cgrqb\") pod \"cert-manager-webhook-f4fb5df64-z8l9s\" (UID: \"cca488df-be0b-41f9-bd69-edf11dd061be\") " pod="cert-manager/cert-manager-webhook-f4fb5df64-z8l9s" Dec 03 11:34:27 crc kubenswrapper[5052]: I1203 11:34:27.264741 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cca488df-be0b-41f9-bd69-edf11dd061be-bound-sa-token\") pod \"cert-manager-webhook-f4fb5df64-z8l9s\" (UID: \"cca488df-be0b-41f9-bd69-edf11dd061be\") " pod="cert-manager/cert-manager-webhook-f4fb5df64-z8l9s" Dec 03 11:34:27 crc kubenswrapper[5052]: I1203 11:34:27.329038 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-855d9ccff4-5pthb"] Dec 03 11:34:27 crc kubenswrapper[5052]: I1203 11:34:27.329808 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-855d9ccff4-5pthb" Dec 03 11:34:27 crc kubenswrapper[5052]: I1203 11:34:27.331553 5052 reflector.go:368] Caches populated for *v1.Secret from object-"cert-manager"/"cert-manager-cainjector-dockercfg-gl6lp" Dec 03 11:34:27 crc kubenswrapper[5052]: I1203 11:34:27.341736 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-855d9ccff4-5pthb"] Dec 03 11:34:27 crc kubenswrapper[5052]: I1203 11:34:27.366398 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-cgrqb\" (UniqueName: \"kubernetes.io/projected/cca488df-be0b-41f9-bd69-edf11dd061be-kube-api-access-cgrqb\") pod \"cert-manager-webhook-f4fb5df64-z8l9s\" (UID: \"cca488df-be0b-41f9-bd69-edf11dd061be\") " pod="cert-manager/cert-manager-webhook-f4fb5df64-z8l9s" Dec 03 11:34:27 crc kubenswrapper[5052]: I1203 11:34:27.366468 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cca488df-be0b-41f9-bd69-edf11dd061be-bound-sa-token\") pod \"cert-manager-webhook-f4fb5df64-z8l9s\" (UID: \"cca488df-be0b-41f9-bd69-edf11dd061be\") " pod="cert-manager/cert-manager-webhook-f4fb5df64-z8l9s" Dec 03 11:34:27 crc kubenswrapper[5052]: I1203 11:34:27.386242 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-cgrqb\" (UniqueName: \"kubernetes.io/projected/cca488df-be0b-41f9-bd69-edf11dd061be-kube-api-access-cgrqb\") pod \"cert-manager-webhook-f4fb5df64-z8l9s\" (UID: \"cca488df-be0b-41f9-bd69-edf11dd061be\") " pod="cert-manager/cert-manager-webhook-f4fb5df64-z8l9s" Dec 03 11:34:27 crc kubenswrapper[5052]: I1203 11:34:27.391405 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cca488df-be0b-41f9-bd69-edf11dd061be-bound-sa-token\") pod \"cert-manager-webhook-f4fb5df64-z8l9s\" (UID: \"cca488df-be0b-41f9-bd69-edf11dd061be\") " pod="cert-manager/cert-manager-webhook-f4fb5df64-z8l9s" Dec 03 11:34:27 crc kubenswrapper[5052]: I1203 11:34:27.467594 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/03732802-639c-4591-b3bf-07bc550527f3-bound-sa-token\") pod \"cert-manager-cainjector-855d9ccff4-5pthb\" (UID: \"03732802-639c-4591-b3bf-07bc550527f3\") " pod="cert-manager/cert-manager-cainjector-855d9ccff4-5pthb" Dec 03 11:34:27 crc kubenswrapper[5052]: I1203 11:34:27.467648 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8gn9k\" (UniqueName: \"kubernetes.io/projected/03732802-639c-4591-b3bf-07bc550527f3-kube-api-access-8gn9k\") pod \"cert-manager-cainjector-855d9ccff4-5pthb\" (UID: \"03732802-639c-4591-b3bf-07bc550527f3\") " pod="cert-manager/cert-manager-cainjector-855d9ccff4-5pthb" Dec 03 11:34:27 crc kubenswrapper[5052]: I1203 11:34:27.562342 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-f4fb5df64-z8l9s" Dec 03 11:34:27 crc kubenswrapper[5052]: I1203 11:34:27.568724 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/03732802-639c-4591-b3bf-07bc550527f3-bound-sa-token\") pod \"cert-manager-cainjector-855d9ccff4-5pthb\" (UID: \"03732802-639c-4591-b3bf-07bc550527f3\") " pod="cert-manager/cert-manager-cainjector-855d9ccff4-5pthb" Dec 03 11:34:27 crc kubenswrapper[5052]: I1203 11:34:27.568767 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-8gn9k\" (UniqueName: \"kubernetes.io/projected/03732802-639c-4591-b3bf-07bc550527f3-kube-api-access-8gn9k\") pod \"cert-manager-cainjector-855d9ccff4-5pthb\" (UID: \"03732802-639c-4591-b3bf-07bc550527f3\") " pod="cert-manager/cert-manager-cainjector-855d9ccff4-5pthb" Dec 03 11:34:27 crc kubenswrapper[5052]: I1203 11:34:27.591444 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-8gn9k\" (UniqueName: \"kubernetes.io/projected/03732802-639c-4591-b3bf-07bc550527f3-kube-api-access-8gn9k\") pod \"cert-manager-cainjector-855d9ccff4-5pthb\" (UID: \"03732802-639c-4591-b3bf-07bc550527f3\") " pod="cert-manager/cert-manager-cainjector-855d9ccff4-5pthb" Dec 03 11:34:27 crc kubenswrapper[5052]: I1203 11:34:27.591506 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/03732802-639c-4591-b3bf-07bc550527f3-bound-sa-token\") pod \"cert-manager-cainjector-855d9ccff4-5pthb\" (UID: \"03732802-639c-4591-b3bf-07bc550527f3\") " pod="cert-manager/cert-manager-cainjector-855d9ccff4-5pthb" Dec 03 11:34:27 crc kubenswrapper[5052]: I1203 11:34:27.658219 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-855d9ccff4-5pthb" Dec 03 11:34:27 crc kubenswrapper[5052]: I1203 11:34:27.909968 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-855d9ccff4-5pthb"] Dec 03 11:34:27 crc kubenswrapper[5052]: W1203 11:34:27.913234 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod03732802_639c_4591_b3bf_07bc550527f3.slice/crio-ba38aefd0537e173e797fd1391f253d4d0fab3ad0175ebbfe095e6c57493afab WatchSource:0}: Error finding container ba38aefd0537e173e797fd1391f253d4d0fab3ad0175ebbfe095e6c57493afab: Status 404 returned error can't find the container with id ba38aefd0537e173e797fd1391f253d4d0fab3ad0175ebbfe095e6c57493afab Dec 03 11:34:28 crc kubenswrapper[5052]: I1203 11:34:28.034583 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-f4fb5df64-z8l9s"] Dec 03 11:34:28 crc kubenswrapper[5052]: W1203 11:34:28.037600 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcca488df_be0b_41f9_bd69_edf11dd061be.slice/crio-b91350af77a93cabac2ac7f35062284607b3972235a99acb4c58ce28304cf099 WatchSource:0}: Error finding container b91350af77a93cabac2ac7f35062284607b3972235a99acb4c58ce28304cf099: Status 404 returned error can't find the container with id b91350af77a93cabac2ac7f35062284607b3972235a99acb4c58ce28304cf099 Dec 03 11:34:28 crc kubenswrapper[5052]: I1203 11:34:28.245007 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-855d9ccff4-5pthb" event={"ID":"03732802-639c-4591-b3bf-07bc550527f3","Type":"ContainerStarted","Data":"ba38aefd0537e173e797fd1391f253d4d0fab3ad0175ebbfe095e6c57493afab"} Dec 03 11:34:28 crc kubenswrapper[5052]: I1203 11:34:28.246173 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-f4fb5df64-z8l9s" event={"ID":"cca488df-be0b-41f9-bd69-edf11dd061be","Type":"ContainerStarted","Data":"b91350af77a93cabac2ac7f35062284607b3972235a99acb4c58ce28304cf099"} Dec 03 11:34:37 crc kubenswrapper[5052]: I1203 11:34:37.153025 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-marketplace/redhat-marketplace-tk9vg" Dec 03 11:34:37 crc kubenswrapper[5052]: I1203 11:34:37.305990 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-855d9ccff4-5pthb" event={"ID":"03732802-639c-4591-b3bf-07bc550527f3","Type":"ContainerStarted","Data":"9d5fcb6e77b473538eb90364a3051c8370fe18afcf0bb8b2591c18121b7ca2dd"} Dec 03 11:34:37 crc kubenswrapper[5052]: I1203 11:34:37.307470 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-f4fb5df64-z8l9s" event={"ID":"cca488df-be0b-41f9-bd69-edf11dd061be","Type":"ContainerStarted","Data":"c5c2a7372dbef40a92eb0415999da0153084c907aa193325e365ba6cd6971811"} Dec 03 11:34:37 crc kubenswrapper[5052]: I1203 11:34:37.307610 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="cert-manager/cert-manager-webhook-f4fb5df64-z8l9s" Dec 03 11:34:37 crc kubenswrapper[5052]: I1203 11:34:37.318732 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-855d9ccff4-5pthb" podStartSLOduration=1.767491863 podStartE2EDuration="10.318711112s" podCreationTimestamp="2025-12-03 11:34:27 +0000 UTC" firstStartedPulling="2025-12-03 11:34:27.91597381 +0000 UTC m=+858.504215574" lastFinishedPulling="2025-12-03 11:34:36.467193059 +0000 UTC m=+867.055434823" observedRunningTime="2025-12-03 11:34:37.318446135 +0000 UTC m=+867.906687899" watchObservedRunningTime="2025-12-03 11:34:37.318711112 +0000 UTC m=+867.906952886" Dec 03 11:34:37 crc kubenswrapper[5052]: I1203 11:34:37.343598 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-f4fb5df64-z8l9s" podStartSLOduration=1.885446223 podStartE2EDuration="10.343579209s" podCreationTimestamp="2025-12-03 11:34:27 +0000 UTC" firstStartedPulling="2025-12-03 11:34:28.0405325 +0000 UTC m=+858.628774264" lastFinishedPulling="2025-12-03 11:34:36.498665486 +0000 UTC m=+867.086907250" observedRunningTime="2025-12-03 11:34:37.341741486 +0000 UTC m=+867.929983240" watchObservedRunningTime="2025-12-03 11:34:37.343579209 +0000 UTC m=+867.931820973" Dec 03 11:34:37 crc kubenswrapper[5052]: I1203 11:34:37.370333 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openshift-marketplace/redhat-marketplace-tk9vg"] Dec 03 11:34:37 crc kubenswrapper[5052]: I1203 11:34:37.370560 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openshift-marketplace/redhat-marketplace-tk9vg" podUID="f297bb86-edee-4810-bc3f-c06f14a948b4" containerName="registry-server" containerID="cri-o://1cce53245fe88f95f161830f068e2fcd4daa5221956a178a0a6e935bf7d56f98" gracePeriod=2 Dec 03 11:34:37 crc kubenswrapper[5052]: I1203 11:34:37.780057 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/redhat-marketplace-tk9vg" Dec 03 11:34:37 crc kubenswrapper[5052]: I1203 11:34:37.947312 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/f297bb86-edee-4810-bc3f-c06f14a948b4-utilities\") pod \"f297bb86-edee-4810-bc3f-c06f14a948b4\" (UID: \"f297bb86-edee-4810-bc3f-c06f14a948b4\") " Dec 03 11:34:37 crc kubenswrapper[5052]: I1203 11:34:37.947393 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qm87r\" (UniqueName: \"kubernetes.io/projected/f297bb86-edee-4810-bc3f-c06f14a948b4-kube-api-access-qm87r\") pod \"f297bb86-edee-4810-bc3f-c06f14a948b4\" (UID: \"f297bb86-edee-4810-bc3f-c06f14a948b4\") " Dec 03 11:34:37 crc kubenswrapper[5052]: I1203 11:34:37.947429 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/f297bb86-edee-4810-bc3f-c06f14a948b4-catalog-content\") pod \"f297bb86-edee-4810-bc3f-c06f14a948b4\" (UID: \"f297bb86-edee-4810-bc3f-c06f14a948b4\") " Dec 03 11:34:37 crc kubenswrapper[5052]: I1203 11:34:37.949098 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f297bb86-edee-4810-bc3f-c06f14a948b4-utilities" (OuterVolumeSpecName: "utilities") pod "f297bb86-edee-4810-bc3f-c06f14a948b4" (UID: "f297bb86-edee-4810-bc3f-c06f14a948b4"). InnerVolumeSpecName "utilities". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:34:37 crc kubenswrapper[5052]: I1203 11:34:37.953277 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f297bb86-edee-4810-bc3f-c06f14a948b4-kube-api-access-qm87r" (OuterVolumeSpecName: "kube-api-access-qm87r") pod "f297bb86-edee-4810-bc3f-c06f14a948b4" (UID: "f297bb86-edee-4810-bc3f-c06f14a948b4"). InnerVolumeSpecName "kube-api-access-qm87r". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:34:37 crc kubenswrapper[5052]: I1203 11:34:37.967348 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f297bb86-edee-4810-bc3f-c06f14a948b4-catalog-content" (OuterVolumeSpecName: "catalog-content") pod "f297bb86-edee-4810-bc3f-c06f14a948b4" (UID: "f297bb86-edee-4810-bc3f-c06f14a948b4"). InnerVolumeSpecName "catalog-content". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:34:38 crc kubenswrapper[5052]: I1203 11:34:38.049279 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-qm87r\" (UniqueName: \"kubernetes.io/projected/f297bb86-edee-4810-bc3f-c06f14a948b4-kube-api-access-qm87r\") on node \"crc\" DevicePath \"\"" Dec 03 11:34:38 crc kubenswrapper[5052]: I1203 11:34:38.049323 5052 reconciler_common.go:293] "Volume detached for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/f297bb86-edee-4810-bc3f-c06f14a948b4-catalog-content\") on node \"crc\" DevicePath \"\"" Dec 03 11:34:38 crc kubenswrapper[5052]: I1203 11:34:38.049335 5052 reconciler_common.go:293] "Volume detached for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/f297bb86-edee-4810-bc3f-c06f14a948b4-utilities\") on node \"crc\" DevicePath \"\"" Dec 03 11:34:38 crc kubenswrapper[5052]: I1203 11:34:38.315663 5052 generic.go:334] "Generic (PLEG): container finished" podID="f297bb86-edee-4810-bc3f-c06f14a948b4" containerID="1cce53245fe88f95f161830f068e2fcd4daa5221956a178a0a6e935bf7d56f98" exitCode=0 Dec 03 11:34:38 crc kubenswrapper[5052]: I1203 11:34:38.315725 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/redhat-marketplace-tk9vg" Dec 03 11:34:38 crc kubenswrapper[5052]: I1203 11:34:38.315752 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-marketplace-tk9vg" event={"ID":"f297bb86-edee-4810-bc3f-c06f14a948b4","Type":"ContainerDied","Data":"1cce53245fe88f95f161830f068e2fcd4daa5221956a178a0a6e935bf7d56f98"} Dec 03 11:34:38 crc kubenswrapper[5052]: I1203 11:34:38.315798 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-marketplace-tk9vg" event={"ID":"f297bb86-edee-4810-bc3f-c06f14a948b4","Type":"ContainerDied","Data":"3bc47de67725b9a2d0750f104399c4e9614be7e85c75ac841a1ffceb2adc0540"} Dec 03 11:34:38 crc kubenswrapper[5052]: I1203 11:34:38.315816 5052 scope.go:117] "RemoveContainer" containerID="1cce53245fe88f95f161830f068e2fcd4daa5221956a178a0a6e935bf7d56f98" Dec 03 11:34:38 crc kubenswrapper[5052]: I1203 11:34:38.351939 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openshift-marketplace/redhat-marketplace-tk9vg"] Dec 03 11:34:38 crc kubenswrapper[5052]: I1203 11:34:38.352560 5052 scope.go:117] "RemoveContainer" containerID="36ff07bffd1a5968305005e1cfbeb683b230bebdaac6e58c2cecf8794106a72e" Dec 03 11:34:38 crc kubenswrapper[5052]: I1203 11:34:38.380789 5052 scope.go:117] "RemoveContainer" containerID="236fd310e3f77946ec8dc2a427332b92676fabb309409a1c75e816ac477da677" Dec 03 11:34:38 crc kubenswrapper[5052]: I1203 11:34:38.382041 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openshift-marketplace/redhat-marketplace-tk9vg"] Dec 03 11:34:38 crc kubenswrapper[5052]: I1203 11:34:38.400179 5052 scope.go:117] "RemoveContainer" containerID="1cce53245fe88f95f161830f068e2fcd4daa5221956a178a0a6e935bf7d56f98" Dec 03 11:34:38 crc kubenswrapper[5052]: E1203 11:34:38.401070 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1cce53245fe88f95f161830f068e2fcd4daa5221956a178a0a6e935bf7d56f98\": container with ID starting with 1cce53245fe88f95f161830f068e2fcd4daa5221956a178a0a6e935bf7d56f98 not found: ID does not exist" containerID="1cce53245fe88f95f161830f068e2fcd4daa5221956a178a0a6e935bf7d56f98" Dec 03 11:34:38 crc kubenswrapper[5052]: I1203 11:34:38.401108 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1cce53245fe88f95f161830f068e2fcd4daa5221956a178a0a6e935bf7d56f98"} err="failed to get container status \"1cce53245fe88f95f161830f068e2fcd4daa5221956a178a0a6e935bf7d56f98\": rpc error: code = NotFound desc = could not find container \"1cce53245fe88f95f161830f068e2fcd4daa5221956a178a0a6e935bf7d56f98\": container with ID starting with 1cce53245fe88f95f161830f068e2fcd4daa5221956a178a0a6e935bf7d56f98 not found: ID does not exist" Dec 03 11:34:38 crc kubenswrapper[5052]: I1203 11:34:38.401134 5052 scope.go:117] "RemoveContainer" containerID="36ff07bffd1a5968305005e1cfbeb683b230bebdaac6e58c2cecf8794106a72e" Dec 03 11:34:38 crc kubenswrapper[5052]: E1203 11:34:38.401492 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"36ff07bffd1a5968305005e1cfbeb683b230bebdaac6e58c2cecf8794106a72e\": container with ID starting with 36ff07bffd1a5968305005e1cfbeb683b230bebdaac6e58c2cecf8794106a72e not found: ID does not exist" containerID="36ff07bffd1a5968305005e1cfbeb683b230bebdaac6e58c2cecf8794106a72e" Dec 03 11:34:38 crc kubenswrapper[5052]: I1203 11:34:38.401535 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"36ff07bffd1a5968305005e1cfbeb683b230bebdaac6e58c2cecf8794106a72e"} err="failed to get container status \"36ff07bffd1a5968305005e1cfbeb683b230bebdaac6e58c2cecf8794106a72e\": rpc error: code = NotFound desc = could not find container \"36ff07bffd1a5968305005e1cfbeb683b230bebdaac6e58c2cecf8794106a72e\": container with ID starting with 36ff07bffd1a5968305005e1cfbeb683b230bebdaac6e58c2cecf8794106a72e not found: ID does not exist" Dec 03 11:34:38 crc kubenswrapper[5052]: I1203 11:34:38.401552 5052 scope.go:117] "RemoveContainer" containerID="236fd310e3f77946ec8dc2a427332b92676fabb309409a1c75e816ac477da677" Dec 03 11:34:38 crc kubenswrapper[5052]: E1203 11:34:38.401809 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"236fd310e3f77946ec8dc2a427332b92676fabb309409a1c75e816ac477da677\": container with ID starting with 236fd310e3f77946ec8dc2a427332b92676fabb309409a1c75e816ac477da677 not found: ID does not exist" containerID="236fd310e3f77946ec8dc2a427332b92676fabb309409a1c75e816ac477da677" Dec 03 11:34:38 crc kubenswrapper[5052]: I1203 11:34:38.401837 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"236fd310e3f77946ec8dc2a427332b92676fabb309409a1c75e816ac477da677"} err="failed to get container status \"236fd310e3f77946ec8dc2a427332b92676fabb309409a1c75e816ac477da677\": rpc error: code = NotFound desc = could not find container \"236fd310e3f77946ec8dc2a427332b92676fabb309409a1c75e816ac477da677\": container with ID starting with 236fd310e3f77946ec8dc2a427332b92676fabb309409a1c75e816ac477da677 not found: ID does not exist" Dec 03 11:34:39 crc kubenswrapper[5052]: I1203 11:34:39.224200 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-86cb77c54b-jdfq2"] Dec 03 11:34:39 crc kubenswrapper[5052]: E1203 11:34:39.224496 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="f297bb86-edee-4810-bc3f-c06f14a948b4" containerName="registry-server" Dec 03 11:34:39 crc kubenswrapper[5052]: I1203 11:34:39.224537 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="f297bb86-edee-4810-bc3f-c06f14a948b4" containerName="registry-server" Dec 03 11:34:39 crc kubenswrapper[5052]: E1203 11:34:39.224566 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="f297bb86-edee-4810-bc3f-c06f14a948b4" containerName="extract-content" Dec 03 11:34:39 crc kubenswrapper[5052]: I1203 11:34:39.224574 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="f297bb86-edee-4810-bc3f-c06f14a948b4" containerName="extract-content" Dec 03 11:34:39 crc kubenswrapper[5052]: E1203 11:34:39.224584 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="f297bb86-edee-4810-bc3f-c06f14a948b4" containerName="extract-utilities" Dec 03 11:34:39 crc kubenswrapper[5052]: I1203 11:34:39.224593 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="f297bb86-edee-4810-bc3f-c06f14a948b4" containerName="extract-utilities" Dec 03 11:34:39 crc kubenswrapper[5052]: I1203 11:34:39.224745 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="f297bb86-edee-4810-bc3f-c06f14a948b4" containerName="registry-server" Dec 03 11:34:39 crc kubenswrapper[5052]: I1203 11:34:39.225247 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-86cb77c54b-jdfq2" Dec 03 11:34:39 crc kubenswrapper[5052]: I1203 11:34:39.229139 5052 reflector.go:368] Caches populated for *v1.Secret from object-"cert-manager"/"cert-manager-dockercfg-jqvdg" Dec 03 11:34:39 crc kubenswrapper[5052]: I1203 11:34:39.239361 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-86cb77c54b-jdfq2"] Dec 03 11:34:39 crc kubenswrapper[5052]: I1203 11:34:39.366178 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/15312175-c609-4674-a597-a520517b3d0f-bound-sa-token\") pod \"cert-manager-86cb77c54b-jdfq2\" (UID: \"15312175-c609-4674-a597-a520517b3d0f\") " pod="cert-manager/cert-manager-86cb77c54b-jdfq2" Dec 03 11:34:39 crc kubenswrapper[5052]: I1203 11:34:39.366218 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qsqnx\" (UniqueName: \"kubernetes.io/projected/15312175-c609-4674-a597-a520517b3d0f-kube-api-access-qsqnx\") pod \"cert-manager-86cb77c54b-jdfq2\" (UID: \"15312175-c609-4674-a597-a520517b3d0f\") " pod="cert-manager/cert-manager-86cb77c54b-jdfq2" Dec 03 11:34:39 crc kubenswrapper[5052]: I1203 11:34:39.467944 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/15312175-c609-4674-a597-a520517b3d0f-bound-sa-token\") pod \"cert-manager-86cb77c54b-jdfq2\" (UID: \"15312175-c609-4674-a597-a520517b3d0f\") " pod="cert-manager/cert-manager-86cb77c54b-jdfq2" Dec 03 11:34:39 crc kubenswrapper[5052]: I1203 11:34:39.468020 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-qsqnx\" (UniqueName: \"kubernetes.io/projected/15312175-c609-4674-a597-a520517b3d0f-kube-api-access-qsqnx\") pod \"cert-manager-86cb77c54b-jdfq2\" (UID: \"15312175-c609-4674-a597-a520517b3d0f\") " pod="cert-manager/cert-manager-86cb77c54b-jdfq2" Dec 03 11:34:39 crc kubenswrapper[5052]: I1203 11:34:39.486940 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-qsqnx\" (UniqueName: \"kubernetes.io/projected/15312175-c609-4674-a597-a520517b3d0f-kube-api-access-qsqnx\") pod \"cert-manager-86cb77c54b-jdfq2\" (UID: \"15312175-c609-4674-a597-a520517b3d0f\") " pod="cert-manager/cert-manager-86cb77c54b-jdfq2" Dec 03 11:34:39 crc kubenswrapper[5052]: I1203 11:34:39.489066 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/15312175-c609-4674-a597-a520517b3d0f-bound-sa-token\") pod \"cert-manager-86cb77c54b-jdfq2\" (UID: \"15312175-c609-4674-a597-a520517b3d0f\") " pod="cert-manager/cert-manager-86cb77c54b-jdfq2" Dec 03 11:34:39 crc kubenswrapper[5052]: I1203 11:34:39.539116 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-86cb77c54b-jdfq2" Dec 03 11:34:39 crc kubenswrapper[5052]: I1203 11:34:39.704500 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f297bb86-edee-4810-bc3f-c06f14a948b4" path="/var/lib/kubelet/pods/f297bb86-edee-4810-bc3f-c06f14a948b4/volumes" Dec 03 11:34:39 crc kubenswrapper[5052]: I1203 11:34:39.943004 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-86cb77c54b-jdfq2"] Dec 03 11:34:40 crc kubenswrapper[5052]: I1203 11:34:40.329623 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-86cb77c54b-jdfq2" event={"ID":"15312175-c609-4674-a597-a520517b3d0f","Type":"ContainerStarted","Data":"3430d8267d7c0096b026740eb11f0dab70194490e996b890aebeba5e64550097"} Dec 03 11:34:42 crc kubenswrapper[5052]: I1203 11:34:42.342187 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-86cb77c54b-jdfq2" event={"ID":"15312175-c609-4674-a597-a520517b3d0f","Type":"ContainerStarted","Data":"6b64cdbee88cda301aeebe71cd2eaaab2966630dcf005b511c3476587b17430e"} Dec 03 11:34:42 crc kubenswrapper[5052]: I1203 11:34:42.365985 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="cert-manager/cert-manager-86cb77c54b-jdfq2" podStartSLOduration=3.365960889 podStartE2EDuration="3.365960889s" podCreationTimestamp="2025-12-03 11:34:39 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:34:42.36116629 +0000 UTC m=+872.949408084" watchObservedRunningTime="2025-12-03 11:34:42.365960889 +0000 UTC m=+872.954202693" Dec 03 11:34:42 crc kubenswrapper[5052]: I1203 11:34:42.565362 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-f4fb5df64-z8l9s" Dec 03 11:34:45 crc kubenswrapper[5052]: I1203 11:34:45.171395 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-operators/openstack-operator-index-wpwlf"] Dec 03 11:34:45 crc kubenswrapper[5052]: I1203 11:34:45.172677 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-operators/openstack-operator-index-wpwlf" Dec 03 11:34:45 crc kubenswrapper[5052]: I1203 11:34:45.174485 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-operators"/"openshift-service-ca.crt" Dec 03 11:34:45 crc kubenswrapper[5052]: I1203 11:34:45.175138 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-operators"/"kube-root-ca.crt" Dec 03 11:34:45 crc kubenswrapper[5052]: I1203 11:34:45.177733 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-operators"/"openstack-operator-index-dockercfg-wrgsd" Dec 03 11:34:45 crc kubenswrapper[5052]: I1203 11:34:45.182724 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-operators/openstack-operator-index-wpwlf"] Dec 03 11:34:45 crc kubenswrapper[5052]: I1203 11:34:45.349236 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c8sh8\" (UniqueName: \"kubernetes.io/projected/85e42274-fd16-4e65-80f8-7b691fc6a9b8-kube-api-access-c8sh8\") pod \"openstack-operator-index-wpwlf\" (UID: \"85e42274-fd16-4e65-80f8-7b691fc6a9b8\") " pod="openstack-operators/openstack-operator-index-wpwlf" Dec 03 11:34:45 crc kubenswrapper[5052]: I1203 11:34:45.450858 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-c8sh8\" (UniqueName: \"kubernetes.io/projected/85e42274-fd16-4e65-80f8-7b691fc6a9b8-kube-api-access-c8sh8\") pod \"openstack-operator-index-wpwlf\" (UID: \"85e42274-fd16-4e65-80f8-7b691fc6a9b8\") " pod="openstack-operators/openstack-operator-index-wpwlf" Dec 03 11:34:45 crc kubenswrapper[5052]: I1203 11:34:45.470867 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-c8sh8\" (UniqueName: \"kubernetes.io/projected/85e42274-fd16-4e65-80f8-7b691fc6a9b8-kube-api-access-c8sh8\") pod \"openstack-operator-index-wpwlf\" (UID: \"85e42274-fd16-4e65-80f8-7b691fc6a9b8\") " pod="openstack-operators/openstack-operator-index-wpwlf" Dec 03 11:34:45 crc kubenswrapper[5052]: I1203 11:34:45.552011 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-operators/openstack-operator-index-wpwlf" Dec 03 11:34:45 crc kubenswrapper[5052]: I1203 11:34:45.731022 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-operators/openstack-operator-index-wpwlf"] Dec 03 11:34:45 crc kubenswrapper[5052]: W1203 11:34:45.740029 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85e42274_fd16_4e65_80f8_7b691fc6a9b8.slice/crio-b3a20a84d763adb14f8ecd25ff89fbb64e584e05f461446966ac4cc5080751b3 WatchSource:0}: Error finding container b3a20a84d763adb14f8ecd25ff89fbb64e584e05f461446966ac4cc5080751b3: Status 404 returned error can't find the container with id b3a20a84d763adb14f8ecd25ff89fbb64e584e05f461446966ac4cc5080751b3 Dec 03 11:34:46 crc kubenswrapper[5052]: I1203 11:34:46.368444 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/openstack-operator-index-wpwlf" event={"ID":"85e42274-fd16-4e65-80f8-7b691fc6a9b8","Type":"ContainerStarted","Data":"b3a20a84d763adb14f8ecd25ff89fbb64e584e05f461446966ac4cc5080751b3"} Dec 03 11:34:47 crc kubenswrapper[5052]: I1203 11:34:47.375714 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/openstack-operator-index-wpwlf" event={"ID":"85e42274-fd16-4e65-80f8-7b691fc6a9b8","Type":"ContainerStarted","Data":"87c5be32a26e2a2b0fdd19457c26d55c1dd0e449144c4bf0666c8d799a47558f"} Dec 03 11:34:47 crc kubenswrapper[5052]: I1203 11:34:47.386978 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-operators/openstack-operator-index-wpwlf" podStartSLOduration=1.49088401 podStartE2EDuration="2.386958977s" podCreationTimestamp="2025-12-03 11:34:45 +0000 UTC" firstStartedPulling="2025-12-03 11:34:45.743629672 +0000 UTC m=+876.331871436" lastFinishedPulling="2025-12-03 11:34:46.639704629 +0000 UTC m=+877.227946403" observedRunningTime="2025-12-03 11:34:47.385984319 +0000 UTC m=+877.974226093" watchObservedRunningTime="2025-12-03 11:34:47.386958977 +0000 UTC m=+877.975200751" Dec 03 11:34:48 crc kubenswrapper[5052]: I1203 11:34:48.553404 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-operators/openstack-operator-index-wpwlf"] Dec 03 11:34:49 crc kubenswrapper[5052]: I1203 11:34:49.159984 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-operators/openstack-operator-index-x6glm"] Dec 03 11:34:49 crc kubenswrapper[5052]: I1203 11:34:49.160861 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-operators/openstack-operator-index-x6glm" Dec 03 11:34:49 crc kubenswrapper[5052]: I1203 11:34:49.174815 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-operators/openstack-operator-index-x6glm"] Dec 03 11:34:49 crc kubenswrapper[5052]: I1203 11:34:49.297200 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-42lr8\" (UniqueName: \"kubernetes.io/projected/6fe52b82-e00b-4c40-8ba1-bb52e7897667-kube-api-access-42lr8\") pod \"openstack-operator-index-x6glm\" (UID: \"6fe52b82-e00b-4c40-8ba1-bb52e7897667\") " pod="openstack-operators/openstack-operator-index-x6glm" Dec 03 11:34:49 crc kubenswrapper[5052]: I1203 11:34:49.387000 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-operators/openstack-operator-index-wpwlf" podUID="85e42274-fd16-4e65-80f8-7b691fc6a9b8" containerName="registry-server" containerID="cri-o://87c5be32a26e2a2b0fdd19457c26d55c1dd0e449144c4bf0666c8d799a47558f" gracePeriod=2 Dec 03 11:34:49 crc kubenswrapper[5052]: I1203 11:34:49.398589 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-42lr8\" (UniqueName: \"kubernetes.io/projected/6fe52b82-e00b-4c40-8ba1-bb52e7897667-kube-api-access-42lr8\") pod \"openstack-operator-index-x6glm\" (UID: \"6fe52b82-e00b-4c40-8ba1-bb52e7897667\") " pod="openstack-operators/openstack-operator-index-x6glm" Dec 03 11:34:49 crc kubenswrapper[5052]: I1203 11:34:49.420114 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-42lr8\" (UniqueName: \"kubernetes.io/projected/6fe52b82-e00b-4c40-8ba1-bb52e7897667-kube-api-access-42lr8\") pod \"openstack-operator-index-x6glm\" (UID: \"6fe52b82-e00b-4c40-8ba1-bb52e7897667\") " pod="openstack-operators/openstack-operator-index-x6glm" Dec 03 11:34:49 crc kubenswrapper[5052]: I1203 11:34:49.481934 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-operators/openstack-operator-index-x6glm" Dec 03 11:34:49 crc kubenswrapper[5052]: I1203 11:34:49.762877 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-operators/openstack-operator-index-x6glm"] Dec 03 11:34:49 crc kubenswrapper[5052]: I1203 11:34:49.817997 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-operators/openstack-operator-index-wpwlf" Dec 03 11:34:49 crc kubenswrapper[5052]: I1203 11:34:49.903107 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c8sh8\" (UniqueName: \"kubernetes.io/projected/85e42274-fd16-4e65-80f8-7b691fc6a9b8-kube-api-access-c8sh8\") pod \"85e42274-fd16-4e65-80f8-7b691fc6a9b8\" (UID: \"85e42274-fd16-4e65-80f8-7b691fc6a9b8\") " Dec 03 11:34:49 crc kubenswrapper[5052]: I1203 11:34:49.907684 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/85e42274-fd16-4e65-80f8-7b691fc6a9b8-kube-api-access-c8sh8" (OuterVolumeSpecName: "kube-api-access-c8sh8") pod "85e42274-fd16-4e65-80f8-7b691fc6a9b8" (UID: "85e42274-fd16-4e65-80f8-7b691fc6a9b8"). InnerVolumeSpecName "kube-api-access-c8sh8". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:34:50 crc kubenswrapper[5052]: I1203 11:34:50.004259 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-c8sh8\" (UniqueName: \"kubernetes.io/projected/85e42274-fd16-4e65-80f8-7b691fc6a9b8-kube-api-access-c8sh8\") on node \"crc\" DevicePath \"\"" Dec 03 11:34:50 crc kubenswrapper[5052]: I1203 11:34:50.393987 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/openstack-operator-index-x6glm" event={"ID":"6fe52b82-e00b-4c40-8ba1-bb52e7897667","Type":"ContainerStarted","Data":"409bd09ed8d8c06816bb54f36a222437bc97be11de3a87273fc8a642cd7acdb8"} Dec 03 11:34:50 crc kubenswrapper[5052]: I1203 11:34:50.394042 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/openstack-operator-index-x6glm" event={"ID":"6fe52b82-e00b-4c40-8ba1-bb52e7897667","Type":"ContainerStarted","Data":"ec3b6d0b8463f153bb7a7b108aac3ac84c5a218111f60f535adc1948f70e1e20"} Dec 03 11:34:50 crc kubenswrapper[5052]: I1203 11:34:50.396042 5052 generic.go:334] "Generic (PLEG): container finished" podID="85e42274-fd16-4e65-80f8-7b691fc6a9b8" containerID="87c5be32a26e2a2b0fdd19457c26d55c1dd0e449144c4bf0666c8d799a47558f" exitCode=0 Dec 03 11:34:50 crc kubenswrapper[5052]: I1203 11:34:50.396093 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/openstack-operator-index-wpwlf" event={"ID":"85e42274-fd16-4e65-80f8-7b691fc6a9b8","Type":"ContainerDied","Data":"87c5be32a26e2a2b0fdd19457c26d55c1dd0e449144c4bf0666c8d799a47558f"} Dec 03 11:34:50 crc kubenswrapper[5052]: I1203 11:34:50.396131 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/openstack-operator-index-wpwlf" event={"ID":"85e42274-fd16-4e65-80f8-7b691fc6a9b8","Type":"ContainerDied","Data":"b3a20a84d763adb14f8ecd25ff89fbb64e584e05f461446966ac4cc5080751b3"} Dec 03 11:34:50 crc kubenswrapper[5052]: I1203 11:34:50.396110 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-operators/openstack-operator-index-wpwlf" Dec 03 11:34:50 crc kubenswrapper[5052]: I1203 11:34:50.396151 5052 scope.go:117] "RemoveContainer" containerID="87c5be32a26e2a2b0fdd19457c26d55c1dd0e449144c4bf0666c8d799a47558f" Dec 03 11:34:50 crc kubenswrapper[5052]: I1203 11:34:50.422722 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-operators/openstack-operator-index-x6glm" podStartSLOduration=0.975044213 podStartE2EDuration="1.422704935s" podCreationTimestamp="2025-12-03 11:34:49 +0000 UTC" firstStartedPulling="2025-12-03 11:34:49.776677006 +0000 UTC m=+880.364918770" lastFinishedPulling="2025-12-03 11:34:50.224337718 +0000 UTC m=+880.812579492" observedRunningTime="2025-12-03 11:34:50.4201015 +0000 UTC m=+881.008343264" watchObservedRunningTime="2025-12-03 11:34:50.422704935 +0000 UTC m=+881.010946699" Dec 03 11:34:50 crc kubenswrapper[5052]: I1203 11:34:50.424065 5052 scope.go:117] "RemoveContainer" containerID="87c5be32a26e2a2b0fdd19457c26d55c1dd0e449144c4bf0666c8d799a47558f" Dec 03 11:34:50 crc kubenswrapper[5052]: E1203 11:34:50.424991 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"87c5be32a26e2a2b0fdd19457c26d55c1dd0e449144c4bf0666c8d799a47558f\": container with ID starting with 87c5be32a26e2a2b0fdd19457c26d55c1dd0e449144c4bf0666c8d799a47558f not found: ID does not exist" containerID="87c5be32a26e2a2b0fdd19457c26d55c1dd0e449144c4bf0666c8d799a47558f" Dec 03 11:34:50 crc kubenswrapper[5052]: I1203 11:34:50.425021 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"87c5be32a26e2a2b0fdd19457c26d55c1dd0e449144c4bf0666c8d799a47558f"} err="failed to get container status \"87c5be32a26e2a2b0fdd19457c26d55c1dd0e449144c4bf0666c8d799a47558f\": rpc error: code = NotFound desc = could not find container \"87c5be32a26e2a2b0fdd19457c26d55c1dd0e449144c4bf0666c8d799a47558f\": container with ID starting with 87c5be32a26e2a2b0fdd19457c26d55c1dd0e449144c4bf0666c8d799a47558f not found: ID does not exist" Dec 03 11:34:50 crc kubenswrapper[5052]: I1203 11:34:50.449548 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-operators/openstack-operator-index-wpwlf"] Dec 03 11:34:50 crc kubenswrapper[5052]: I1203 11:34:50.453687 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-operators/openstack-operator-index-wpwlf"] Dec 03 11:34:51 crc kubenswrapper[5052]: I1203 11:34:51.689499 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="85e42274-fd16-4e65-80f8-7b691fc6a9b8" path="/var/lib/kubelet/pods/85e42274-fd16-4e65-80f8-7b691fc6a9b8/volumes" Dec 03 11:34:59 crc kubenswrapper[5052]: I1203 11:34:59.482539 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-operators/openstack-operator-index-x6glm" Dec 03 11:34:59 crc kubenswrapper[5052]: I1203 11:34:59.483972 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-operators/openstack-operator-index-x6glm" Dec 03 11:34:59 crc kubenswrapper[5052]: I1203 11:34:59.504920 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-operators/openstack-operator-index-x6glm" Dec 03 11:35:00 crc kubenswrapper[5052]: I1203 11:35:00.475892 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-operators/openstack-operator-index-x6glm" Dec 03 11:35:02 crc kubenswrapper[5052]: I1203 11:35:02.201180 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-operators/98dc3bd0b5c63de8bc52e3558b9d3e72fafafb6fd127fd2510d2206864t722l"] Dec 03 11:35:02 crc kubenswrapper[5052]: E1203 11:35:02.201880 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="85e42274-fd16-4e65-80f8-7b691fc6a9b8" containerName="registry-server" Dec 03 11:35:02 crc kubenswrapper[5052]: I1203 11:35:02.201902 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="85e42274-fd16-4e65-80f8-7b691fc6a9b8" containerName="registry-server" Dec 03 11:35:02 crc kubenswrapper[5052]: I1203 11:35:02.202093 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="85e42274-fd16-4e65-80f8-7b691fc6a9b8" containerName="registry-server" Dec 03 11:35:02 crc kubenswrapper[5052]: I1203 11:35:02.203426 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-operators/98dc3bd0b5c63de8bc52e3558b9d3e72fafafb6fd127fd2510d2206864t722l" Dec 03 11:35:02 crc kubenswrapper[5052]: I1203 11:35:02.205468 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-operators"/"default-dockercfg-7fklr" Dec 03 11:35:02 crc kubenswrapper[5052]: I1203 11:35:02.222249 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-operators/98dc3bd0b5c63de8bc52e3558b9d3e72fafafb6fd127fd2510d2206864t722l"] Dec 03 11:35:02 crc kubenswrapper[5052]: I1203 11:35:02.375230 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8ad91c8d-10f1-465d-b835-74904f775d32-util\") pod \"98dc3bd0b5c63de8bc52e3558b9d3e72fafafb6fd127fd2510d2206864t722l\" (UID: \"8ad91c8d-10f1-465d-b835-74904f775d32\") " pod="openstack-operators/98dc3bd0b5c63de8bc52e3558b9d3e72fafafb6fd127fd2510d2206864t722l" Dec 03 11:35:02 crc kubenswrapper[5052]: I1203 11:35:02.375448 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jcvwd\" (UniqueName: \"kubernetes.io/projected/8ad91c8d-10f1-465d-b835-74904f775d32-kube-api-access-jcvwd\") pod \"98dc3bd0b5c63de8bc52e3558b9d3e72fafafb6fd127fd2510d2206864t722l\" (UID: \"8ad91c8d-10f1-465d-b835-74904f775d32\") " pod="openstack-operators/98dc3bd0b5c63de8bc52e3558b9d3e72fafafb6fd127fd2510d2206864t722l" Dec 03 11:35:02 crc kubenswrapper[5052]: I1203 11:35:02.375504 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8ad91c8d-10f1-465d-b835-74904f775d32-bundle\") pod \"98dc3bd0b5c63de8bc52e3558b9d3e72fafafb6fd127fd2510d2206864t722l\" (UID: \"8ad91c8d-10f1-465d-b835-74904f775d32\") " pod="openstack-operators/98dc3bd0b5c63de8bc52e3558b9d3e72fafafb6fd127fd2510d2206864t722l" Dec 03 11:35:02 crc kubenswrapper[5052]: I1203 11:35:02.476891 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-jcvwd\" (UniqueName: \"kubernetes.io/projected/8ad91c8d-10f1-465d-b835-74904f775d32-kube-api-access-jcvwd\") pod \"98dc3bd0b5c63de8bc52e3558b9d3e72fafafb6fd127fd2510d2206864t722l\" (UID: \"8ad91c8d-10f1-465d-b835-74904f775d32\") " pod="openstack-operators/98dc3bd0b5c63de8bc52e3558b9d3e72fafafb6fd127fd2510d2206864t722l" Dec 03 11:35:02 crc kubenswrapper[5052]: I1203 11:35:02.476967 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8ad91c8d-10f1-465d-b835-74904f775d32-bundle\") pod \"98dc3bd0b5c63de8bc52e3558b9d3e72fafafb6fd127fd2510d2206864t722l\" (UID: \"8ad91c8d-10f1-465d-b835-74904f775d32\") " pod="openstack-operators/98dc3bd0b5c63de8bc52e3558b9d3e72fafafb6fd127fd2510d2206864t722l" Dec 03 11:35:02 crc kubenswrapper[5052]: I1203 11:35:02.477036 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8ad91c8d-10f1-465d-b835-74904f775d32-util\") pod \"98dc3bd0b5c63de8bc52e3558b9d3e72fafafb6fd127fd2510d2206864t722l\" (UID: \"8ad91c8d-10f1-465d-b835-74904f775d32\") " pod="openstack-operators/98dc3bd0b5c63de8bc52e3558b9d3e72fafafb6fd127fd2510d2206864t722l" Dec 03 11:35:02 crc kubenswrapper[5052]: I1203 11:35:02.477670 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8ad91c8d-10f1-465d-b835-74904f775d32-util\") pod \"98dc3bd0b5c63de8bc52e3558b9d3e72fafafb6fd127fd2510d2206864t722l\" (UID: \"8ad91c8d-10f1-465d-b835-74904f775d32\") " pod="openstack-operators/98dc3bd0b5c63de8bc52e3558b9d3e72fafafb6fd127fd2510d2206864t722l" Dec 03 11:35:02 crc kubenswrapper[5052]: I1203 11:35:02.478462 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8ad91c8d-10f1-465d-b835-74904f775d32-bundle\") pod \"98dc3bd0b5c63de8bc52e3558b9d3e72fafafb6fd127fd2510d2206864t722l\" (UID: \"8ad91c8d-10f1-465d-b835-74904f775d32\") " pod="openstack-operators/98dc3bd0b5c63de8bc52e3558b9d3e72fafafb6fd127fd2510d2206864t722l" Dec 03 11:35:02 crc kubenswrapper[5052]: I1203 11:35:02.499578 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-jcvwd\" (UniqueName: \"kubernetes.io/projected/8ad91c8d-10f1-465d-b835-74904f775d32-kube-api-access-jcvwd\") pod \"98dc3bd0b5c63de8bc52e3558b9d3e72fafafb6fd127fd2510d2206864t722l\" (UID: \"8ad91c8d-10f1-465d-b835-74904f775d32\") " pod="openstack-operators/98dc3bd0b5c63de8bc52e3558b9d3e72fafafb6fd127fd2510d2206864t722l" Dec 03 11:35:02 crc kubenswrapper[5052]: I1203 11:35:02.529941 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-operators/98dc3bd0b5c63de8bc52e3558b9d3e72fafafb6fd127fd2510d2206864t722l" Dec 03 11:35:02 crc kubenswrapper[5052]: W1203 11:35:02.952140 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ad91c8d_10f1_465d_b835_74904f775d32.slice/crio-141fe66bcb4714f3edead40de0304ee43088ff97c59a633122b57af45ccb89f0 WatchSource:0}: Error finding container 141fe66bcb4714f3edead40de0304ee43088ff97c59a633122b57af45ccb89f0: Status 404 returned error can't find the container with id 141fe66bcb4714f3edead40de0304ee43088ff97c59a633122b57af45ccb89f0 Dec 03 11:35:02 crc kubenswrapper[5052]: I1203 11:35:02.952384 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-operators/98dc3bd0b5c63de8bc52e3558b9d3e72fafafb6fd127fd2510d2206864t722l"] Dec 03 11:35:03 crc kubenswrapper[5052]: I1203 11:35:03.472796 5052 generic.go:334] "Generic (PLEG): container finished" podID="8ad91c8d-10f1-465d-b835-74904f775d32" containerID="68d019501383a405411c007622c2abbfc555ddefac71b2db656127bfbb35778d" exitCode=0 Dec 03 11:35:03 crc kubenswrapper[5052]: I1203 11:35:03.472915 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/98dc3bd0b5c63de8bc52e3558b9d3e72fafafb6fd127fd2510d2206864t722l" event={"ID":"8ad91c8d-10f1-465d-b835-74904f775d32","Type":"ContainerDied","Data":"68d019501383a405411c007622c2abbfc555ddefac71b2db656127bfbb35778d"} Dec 03 11:35:03 crc kubenswrapper[5052]: I1203 11:35:03.473077 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/98dc3bd0b5c63de8bc52e3558b9d3e72fafafb6fd127fd2510d2206864t722l" event={"ID":"8ad91c8d-10f1-465d-b835-74904f775d32","Type":"ContainerStarted","Data":"141fe66bcb4714f3edead40de0304ee43088ff97c59a633122b57af45ccb89f0"} Dec 03 11:35:04 crc kubenswrapper[5052]: I1203 11:35:04.481717 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/98dc3bd0b5c63de8bc52e3558b9d3e72fafafb6fd127fd2510d2206864t722l" event={"ID":"8ad91c8d-10f1-465d-b835-74904f775d32","Type":"ContainerStarted","Data":"aef3df5f8fff4b308899303eaa804c7403a302b0281ea9d04dd205a1a36bc5e6"} Dec 03 11:35:05 crc kubenswrapper[5052]: I1203 11:35:05.491141 5052 generic.go:334] "Generic (PLEG): container finished" podID="8ad91c8d-10f1-465d-b835-74904f775d32" containerID="aef3df5f8fff4b308899303eaa804c7403a302b0281ea9d04dd205a1a36bc5e6" exitCode=0 Dec 03 11:35:05 crc kubenswrapper[5052]: I1203 11:35:05.492166 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/98dc3bd0b5c63de8bc52e3558b9d3e72fafafb6fd127fd2510d2206864t722l" event={"ID":"8ad91c8d-10f1-465d-b835-74904f775d32","Type":"ContainerDied","Data":"aef3df5f8fff4b308899303eaa804c7403a302b0281ea9d04dd205a1a36bc5e6"} Dec 03 11:35:06 crc kubenswrapper[5052]: I1203 11:35:06.503006 5052 generic.go:334] "Generic (PLEG): container finished" podID="8ad91c8d-10f1-465d-b835-74904f775d32" containerID="4581166bdfee51112c24884b2492e7341e87ef27c9a5126a5746e43910a66aa4" exitCode=0 Dec 03 11:35:06 crc kubenswrapper[5052]: I1203 11:35:06.503129 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/98dc3bd0b5c63de8bc52e3558b9d3e72fafafb6fd127fd2510d2206864t722l" event={"ID":"8ad91c8d-10f1-465d-b835-74904f775d32","Type":"ContainerDied","Data":"4581166bdfee51112c24884b2492e7341e87ef27c9a5126a5746e43910a66aa4"} Dec 03 11:35:07 crc kubenswrapper[5052]: I1203 11:35:07.843210 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-operators/98dc3bd0b5c63de8bc52e3558b9d3e72fafafb6fd127fd2510d2206864t722l" Dec 03 11:35:07 crc kubenswrapper[5052]: I1203 11:35:07.954533 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jcvwd\" (UniqueName: \"kubernetes.io/projected/8ad91c8d-10f1-465d-b835-74904f775d32-kube-api-access-jcvwd\") pod \"8ad91c8d-10f1-465d-b835-74904f775d32\" (UID: \"8ad91c8d-10f1-465d-b835-74904f775d32\") " Dec 03 11:35:07 crc kubenswrapper[5052]: I1203 11:35:07.954667 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8ad91c8d-10f1-465d-b835-74904f775d32-bundle\") pod \"8ad91c8d-10f1-465d-b835-74904f775d32\" (UID: \"8ad91c8d-10f1-465d-b835-74904f775d32\") " Dec 03 11:35:07 crc kubenswrapper[5052]: I1203 11:35:07.954735 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8ad91c8d-10f1-465d-b835-74904f775d32-util\") pod \"8ad91c8d-10f1-465d-b835-74904f775d32\" (UID: \"8ad91c8d-10f1-465d-b835-74904f775d32\") " Dec 03 11:35:07 crc kubenswrapper[5052]: I1203 11:35:07.955426 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8ad91c8d-10f1-465d-b835-74904f775d32-bundle" (OuterVolumeSpecName: "bundle") pod "8ad91c8d-10f1-465d-b835-74904f775d32" (UID: "8ad91c8d-10f1-465d-b835-74904f775d32"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:35:07 crc kubenswrapper[5052]: I1203 11:35:07.960493 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8ad91c8d-10f1-465d-b835-74904f775d32-kube-api-access-jcvwd" (OuterVolumeSpecName: "kube-api-access-jcvwd") pod "8ad91c8d-10f1-465d-b835-74904f775d32" (UID: "8ad91c8d-10f1-465d-b835-74904f775d32"). InnerVolumeSpecName "kube-api-access-jcvwd". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:35:07 crc kubenswrapper[5052]: I1203 11:35:07.971294 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8ad91c8d-10f1-465d-b835-74904f775d32-util" (OuterVolumeSpecName: "util") pod "8ad91c8d-10f1-465d-b835-74904f775d32" (UID: "8ad91c8d-10f1-465d-b835-74904f775d32"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:35:08 crc kubenswrapper[5052]: I1203 11:35:08.056636 5052 reconciler_common.go:293] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8ad91c8d-10f1-465d-b835-74904f775d32-util\") on node \"crc\" DevicePath \"\"" Dec 03 11:35:08 crc kubenswrapper[5052]: I1203 11:35:08.056668 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-jcvwd\" (UniqueName: \"kubernetes.io/projected/8ad91c8d-10f1-465d-b835-74904f775d32-kube-api-access-jcvwd\") on node \"crc\" DevicePath \"\"" Dec 03 11:35:08 crc kubenswrapper[5052]: I1203 11:35:08.056681 5052 reconciler_common.go:293] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8ad91c8d-10f1-465d-b835-74904f775d32-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:35:08 crc kubenswrapper[5052]: I1203 11:35:08.521183 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/98dc3bd0b5c63de8bc52e3558b9d3e72fafafb6fd127fd2510d2206864t722l" event={"ID":"8ad91c8d-10f1-465d-b835-74904f775d32","Type":"ContainerDied","Data":"141fe66bcb4714f3edead40de0304ee43088ff97c59a633122b57af45ccb89f0"} Dec 03 11:35:08 crc kubenswrapper[5052]: I1203 11:35:08.521248 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="141fe66bcb4714f3edead40de0304ee43088ff97c59a633122b57af45ccb89f0" Dec 03 11:35:08 crc kubenswrapper[5052]: I1203 11:35:08.521345 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-operators/98dc3bd0b5c63de8bc52e3558b9d3e72fafafb6fd127fd2510d2206864t722l" Dec 03 11:35:14 crc kubenswrapper[5052]: I1203 11:35:14.376858 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-operators/openstack-operator-controller-operator-7dd5c7bb7c-rjflp"] Dec 03 11:35:14 crc kubenswrapper[5052]: E1203 11:35:14.377762 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="8ad91c8d-10f1-465d-b835-74904f775d32" containerName="util" Dec 03 11:35:14 crc kubenswrapper[5052]: I1203 11:35:14.377777 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="8ad91c8d-10f1-465d-b835-74904f775d32" containerName="util" Dec 03 11:35:14 crc kubenswrapper[5052]: E1203 11:35:14.377790 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="8ad91c8d-10f1-465d-b835-74904f775d32" containerName="pull" Dec 03 11:35:14 crc kubenswrapper[5052]: I1203 11:35:14.377798 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="8ad91c8d-10f1-465d-b835-74904f775d32" containerName="pull" Dec 03 11:35:14 crc kubenswrapper[5052]: E1203 11:35:14.377808 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="8ad91c8d-10f1-465d-b835-74904f775d32" containerName="extract" Dec 03 11:35:14 crc kubenswrapper[5052]: I1203 11:35:14.377815 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="8ad91c8d-10f1-465d-b835-74904f775d32" containerName="extract" Dec 03 11:35:14 crc kubenswrapper[5052]: I1203 11:35:14.377950 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="8ad91c8d-10f1-465d-b835-74904f775d32" containerName="extract" Dec 03 11:35:14 crc kubenswrapper[5052]: I1203 11:35:14.378424 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-operators/openstack-operator-controller-operator-7dd5c7bb7c-rjflp" Dec 03 11:35:14 crc kubenswrapper[5052]: I1203 11:35:14.380783 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-operators"/"openstack-operator-controller-operator-dockercfg-t5rjk" Dec 03 11:35:14 crc kubenswrapper[5052]: I1203 11:35:14.416974 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-operators/openstack-operator-controller-operator-7dd5c7bb7c-rjflp"] Dec 03 11:35:14 crc kubenswrapper[5052]: I1203 11:35:14.552876 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-658xp\" (UniqueName: \"kubernetes.io/projected/cd2c5c4c-7ea8-4f95-9556-58b87d14a2be-kube-api-access-658xp\") pod \"openstack-operator-controller-operator-7dd5c7bb7c-rjflp\" (UID: \"cd2c5c4c-7ea8-4f95-9556-58b87d14a2be\") " pod="openstack-operators/openstack-operator-controller-operator-7dd5c7bb7c-rjflp" Dec 03 11:35:14 crc kubenswrapper[5052]: I1203 11:35:14.654071 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-658xp\" (UniqueName: \"kubernetes.io/projected/cd2c5c4c-7ea8-4f95-9556-58b87d14a2be-kube-api-access-658xp\") pod \"openstack-operator-controller-operator-7dd5c7bb7c-rjflp\" (UID: \"cd2c5c4c-7ea8-4f95-9556-58b87d14a2be\") " pod="openstack-operators/openstack-operator-controller-operator-7dd5c7bb7c-rjflp" Dec 03 11:35:14 crc kubenswrapper[5052]: I1203 11:35:14.678355 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-658xp\" (UniqueName: \"kubernetes.io/projected/cd2c5c4c-7ea8-4f95-9556-58b87d14a2be-kube-api-access-658xp\") pod \"openstack-operator-controller-operator-7dd5c7bb7c-rjflp\" (UID: \"cd2c5c4c-7ea8-4f95-9556-58b87d14a2be\") " pod="openstack-operators/openstack-operator-controller-operator-7dd5c7bb7c-rjflp" Dec 03 11:35:14 crc kubenswrapper[5052]: I1203 11:35:14.697468 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-operators/openstack-operator-controller-operator-7dd5c7bb7c-rjflp" Dec 03 11:35:15 crc kubenswrapper[5052]: I1203 11:35:15.209235 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-operators/openstack-operator-controller-operator-7dd5c7bb7c-rjflp"] Dec 03 11:35:15 crc kubenswrapper[5052]: I1203 11:35:15.566381 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/openstack-operator-controller-operator-7dd5c7bb7c-rjflp" event={"ID":"cd2c5c4c-7ea8-4f95-9556-58b87d14a2be","Type":"ContainerStarted","Data":"a34afd81a3a9a587cceb4cb7cd37efc7470c49f3210edaf5369224a9cae78e67"} Dec 03 11:35:20 crc kubenswrapper[5052]: I1203 11:35:20.639642 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/openstack-operator-controller-operator-7dd5c7bb7c-rjflp" event={"ID":"cd2c5c4c-7ea8-4f95-9556-58b87d14a2be","Type":"ContainerStarted","Data":"5a4c5c6a86ebb9b257e5e311d545f7f9899b5f426c04b2d6f5217d3cce6845e8"} Dec 03 11:35:20 crc kubenswrapper[5052]: I1203 11:35:20.641028 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-operators/openstack-operator-controller-operator-7dd5c7bb7c-rjflp" Dec 03 11:35:20 crc kubenswrapper[5052]: I1203 11:35:20.686726 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-operators/openstack-operator-controller-operator-7dd5c7bb7c-rjflp" podStartSLOduration=1.9246822080000001 podStartE2EDuration="6.686704393s" podCreationTimestamp="2025-12-03 11:35:14 +0000 UTC" firstStartedPulling="2025-12-03 11:35:15.234034152 +0000 UTC m=+905.822275916" lastFinishedPulling="2025-12-03 11:35:19.996056337 +0000 UTC m=+910.584298101" observedRunningTime="2025-12-03 11:35:20.684284674 +0000 UTC m=+911.272526458" watchObservedRunningTime="2025-12-03 11:35:20.686704393 +0000 UTC m=+911.274946177" Dec 03 11:35:29 crc kubenswrapper[5052]: I1203 11:35:29.379036 5052 patch_prober.go:28] interesting pod/machine-config-daemon-74bj6 container/machine-config-daemon namespace/openshift-machine-config-operator: Liveness probe status=failure output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" start-of-body= Dec 03 11:35:29 crc kubenswrapper[5052]: I1203 11:35:29.379719 5052 prober.go:107] "Probe failed" probeType="Liveness" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" containerName="machine-config-daemon" probeResult="failure" output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" Dec 03 11:35:34 crc kubenswrapper[5052]: I1203 11:35:34.701749 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-operators/openstack-operator-controller-operator-7dd5c7bb7c-rjflp" Dec 03 11:35:52 crc kubenswrapper[5052]: I1203 11:35:52.545834 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-operators/barbican-operator-controller-manager-7d9dfd778-cqfdk"] Dec 03 11:35:52 crc kubenswrapper[5052]: I1203 11:35:52.547417 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-operators/barbican-operator-controller-manager-7d9dfd778-cqfdk" Dec 03 11:35:52 crc kubenswrapper[5052]: I1203 11:35:52.549645 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-operators"/"barbican-operator-controller-manager-dockercfg-t27jg" Dec 03 11:35:52 crc kubenswrapper[5052]: I1203 11:35:52.553733 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-operators/cinder-operator-controller-manager-859b6ccc6-vt9d2"] Dec 03 11:35:52 crc kubenswrapper[5052]: I1203 11:35:52.555039 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-operators/cinder-operator-controller-manager-859b6ccc6-vt9d2" Dec 03 11:35:52 crc kubenswrapper[5052]: I1203 11:35:52.556989 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-operators"/"cinder-operator-controller-manager-dockercfg-cfw7j" Dec 03 11:35:52 crc kubenswrapper[5052]: I1203 11:35:52.558497 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-operators/barbican-operator-controller-manager-7d9dfd778-cqfdk"] Dec 03 11:35:52 crc kubenswrapper[5052]: I1203 11:35:52.567264 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-operators/cinder-operator-controller-manager-859b6ccc6-vt9d2"] Dec 03 11:35:52 crc kubenswrapper[5052]: I1203 11:35:52.585170 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-operators/designate-operator-controller-manager-78b4bc895b-874g2"] Dec 03 11:35:52 crc kubenswrapper[5052]: I1203 11:35:52.586446 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-operators/designate-operator-controller-manager-78b4bc895b-874g2" Dec 03 11:35:52 crc kubenswrapper[5052]: I1203 11:35:52.591948 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-operators"/"designate-operator-controller-manager-dockercfg-98kg4" Dec 03 11:35:52 crc kubenswrapper[5052]: I1203 11:35:52.618585 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-operators/designate-operator-controller-manager-78b4bc895b-874g2"] Dec 03 11:35:52 crc kubenswrapper[5052]: I1203 11:35:52.626404 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-operators/glance-operator-controller-manager-77987cd8cd-btn2l"] Dec 03 11:35:52 crc kubenswrapper[5052]: I1203 11:35:52.626842 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qtl9v\" (UniqueName: \"kubernetes.io/projected/2e2dc077-c02f-4f98-8fa6-9f72890faeaf-kube-api-access-qtl9v\") pod \"designate-operator-controller-manager-78b4bc895b-874g2\" (UID: \"2e2dc077-c02f-4f98-8fa6-9f72890faeaf\") " pod="openstack-operators/designate-operator-controller-manager-78b4bc895b-874g2" Dec 03 11:35:52 crc kubenswrapper[5052]: I1203 11:35:52.626933 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8vjcl\" (UniqueName: \"kubernetes.io/projected/8dbd25a1-9de9-431e-be37-d239e3f64728-kube-api-access-8vjcl\") pod \"barbican-operator-controller-manager-7d9dfd778-cqfdk\" (UID: \"8dbd25a1-9de9-431e-be37-d239e3f64728\") " pod="openstack-operators/barbican-operator-controller-manager-7d9dfd778-cqfdk" Dec 03 11:35:52 crc kubenswrapper[5052]: I1203 11:35:52.626962 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fkq98\" (UniqueName: \"kubernetes.io/projected/5968d273-33dd-45cc-aab0-e83628bcd15c-kube-api-access-fkq98\") pod \"cinder-operator-controller-manager-859b6ccc6-vt9d2\" (UID: \"5968d273-33dd-45cc-aab0-e83628bcd15c\") " pod="openstack-operators/cinder-operator-controller-manager-859b6ccc6-vt9d2" Dec 03 11:35:52 crc kubenswrapper[5052]: I1203 11:35:52.627784 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-operators/glance-operator-controller-manager-77987cd8cd-btn2l" Dec 03 11:35:52 crc kubenswrapper[5052]: I1203 11:35:52.630818 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-operators"/"glance-operator-controller-manager-dockercfg-5wcvt" Dec 03 11:35:52 crc kubenswrapper[5052]: I1203 11:35:52.655349 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-operators/glance-operator-controller-manager-77987cd8cd-btn2l"] Dec 03 11:35:52 crc kubenswrapper[5052]: I1203 11:35:52.662884 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-operators/heat-operator-controller-manager-5f64f6f8bb-48fnb"] Dec 03 11:35:52 crc kubenswrapper[5052]: I1203 11:35:52.664123 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-operators/heat-operator-controller-manager-5f64f6f8bb-48fnb" Dec 03 11:35:52 crc kubenswrapper[5052]: I1203 11:35:52.667961 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-operators"/"heat-operator-controller-manager-dockercfg-prgxj" Dec 03 11:35:52 crc kubenswrapper[5052]: I1203 11:35:52.688307 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-operators/heat-operator-controller-manager-5f64f6f8bb-48fnb"] Dec 03 11:35:52 crc kubenswrapper[5052]: I1203 11:35:52.697190 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-operators/horizon-operator-controller-manager-68c6d99b8f-mjdvh"] Dec 03 11:35:52 crc kubenswrapper[5052]: I1203 11:35:52.701498 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-operators/horizon-operator-controller-manager-68c6d99b8f-mjdvh" Dec 03 11:35:52 crc kubenswrapper[5052]: I1203 11:35:52.708113 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-operators"/"horizon-operator-controller-manager-dockercfg-tr8r7" Dec 03 11:35:52 crc kubenswrapper[5052]: I1203 11:35:52.719626 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-operators/infra-operator-controller-manager-57548d458d-9fkll"] Dec 03 11:35:52 crc kubenswrapper[5052]: I1203 11:35:52.720969 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-operators/infra-operator-controller-manager-57548d458d-9fkll" Dec 03 11:35:52 crc kubenswrapper[5052]: I1203 11:35:52.724718 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-operators"/"infra-operator-webhook-server-cert" Dec 03 11:35:52 crc kubenswrapper[5052]: I1203 11:35:52.724759 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-operators"/"infra-operator-controller-manager-dockercfg-xgv5g" Dec 03 11:35:52 crc kubenswrapper[5052]: I1203 11:35:52.731149 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-klcr8\" (UniqueName: \"kubernetes.io/projected/81f5f8e1-015a-4b45-83e6-13e88efb8e80-kube-api-access-klcr8\") pod \"infra-operator-controller-manager-57548d458d-9fkll\" (UID: \"81f5f8e1-015a-4b45-83e6-13e88efb8e80\") " pod="openstack-operators/infra-operator-controller-manager-57548d458d-9fkll" Dec 03 11:35:52 crc kubenswrapper[5052]: I1203 11:35:52.731260 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kvnlb\" (UniqueName: \"kubernetes.io/projected/d429f7ab-9a88-4411-a7a5-88bc8ab76285-kube-api-access-kvnlb\") pod \"glance-operator-controller-manager-77987cd8cd-btn2l\" (UID: \"d429f7ab-9a88-4411-a7a5-88bc8ab76285\") " pod="openstack-operators/glance-operator-controller-manager-77987cd8cd-btn2l" Dec 03 11:35:52 crc kubenswrapper[5052]: I1203 11:35:52.731371 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-8vjcl\" (UniqueName: \"kubernetes.io/projected/8dbd25a1-9de9-431e-be37-d239e3f64728-kube-api-access-8vjcl\") pod \"barbican-operator-controller-manager-7d9dfd778-cqfdk\" (UID: \"8dbd25a1-9de9-431e-be37-d239e3f64728\") " pod="openstack-operators/barbican-operator-controller-manager-7d9dfd778-cqfdk" Dec 03 11:35:52 crc kubenswrapper[5052]: I1203 11:35:52.731430 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-fkq98\" (UniqueName: \"kubernetes.io/projected/5968d273-33dd-45cc-aab0-e83628bcd15c-kube-api-access-fkq98\") pod \"cinder-operator-controller-manager-859b6ccc6-vt9d2\" (UID: \"5968d273-33dd-45cc-aab0-e83628bcd15c\") " pod="openstack-operators/cinder-operator-controller-manager-859b6ccc6-vt9d2" Dec 03 11:35:52 crc kubenswrapper[5052]: I1203 11:35:52.731477 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7shls\" (UniqueName: \"kubernetes.io/projected/ce3d34f6-e545-4eb2-bcc5-999377931601-kube-api-access-7shls\") pod \"horizon-operator-controller-manager-68c6d99b8f-mjdvh\" (UID: \"ce3d34f6-e545-4eb2-bcc5-999377931601\") " pod="openstack-operators/horizon-operator-controller-manager-68c6d99b8f-mjdvh" Dec 03 11:35:52 crc kubenswrapper[5052]: I1203 11:35:52.731590 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/81f5f8e1-015a-4b45-83e6-13e88efb8e80-cert\") pod \"infra-operator-controller-manager-57548d458d-9fkll\" (UID: \"81f5f8e1-015a-4b45-83e6-13e88efb8e80\") " pod="openstack-operators/infra-operator-controller-manager-57548d458d-9fkll" Dec 03 11:35:52 crc kubenswrapper[5052]: I1203 11:35:52.731651 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wgp7x\" (UniqueName: \"kubernetes.io/projected/0cbd234c-ecb3-4e54-af86-a2b15fdd94a3-kube-api-access-wgp7x\") pod \"heat-operator-controller-manager-5f64f6f8bb-48fnb\" (UID: \"0cbd234c-ecb3-4e54-af86-a2b15fdd94a3\") " pod="openstack-operators/heat-operator-controller-manager-5f64f6f8bb-48fnb" Dec 03 11:35:52 crc kubenswrapper[5052]: I1203 11:35:52.731752 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-qtl9v\" (UniqueName: \"kubernetes.io/projected/2e2dc077-c02f-4f98-8fa6-9f72890faeaf-kube-api-access-qtl9v\") pod \"designate-operator-controller-manager-78b4bc895b-874g2\" (UID: \"2e2dc077-c02f-4f98-8fa6-9f72890faeaf\") " pod="openstack-operators/designate-operator-controller-manager-78b4bc895b-874g2" Dec 03 11:35:52 crc kubenswrapper[5052]: I1203 11:35:52.755658 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-operators/ironic-operator-controller-manager-6c548fd776-pqfnd"] Dec 03 11:35:52 crc kubenswrapper[5052]: I1203 11:35:52.774623 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-operators/ironic-operator-controller-manager-6c548fd776-pqfnd" Dec 03 11:35:52 crc kubenswrapper[5052]: I1203 11:35:52.784192 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-operators"/"ironic-operator-controller-manager-dockercfg-ln872" Dec 03 11:35:52 crc kubenswrapper[5052]: I1203 11:35:52.792997 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-qtl9v\" (UniqueName: \"kubernetes.io/projected/2e2dc077-c02f-4f98-8fa6-9f72890faeaf-kube-api-access-qtl9v\") pod \"designate-operator-controller-manager-78b4bc895b-874g2\" (UID: \"2e2dc077-c02f-4f98-8fa6-9f72890faeaf\") " pod="openstack-operators/designate-operator-controller-manager-78b4bc895b-874g2" Dec 03 11:35:52 crc kubenswrapper[5052]: I1203 11:35:52.810654 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-8vjcl\" (UniqueName: \"kubernetes.io/projected/8dbd25a1-9de9-431e-be37-d239e3f64728-kube-api-access-8vjcl\") pod \"barbican-operator-controller-manager-7d9dfd778-cqfdk\" (UID: \"8dbd25a1-9de9-431e-be37-d239e3f64728\") " pod="openstack-operators/barbican-operator-controller-manager-7d9dfd778-cqfdk" Dec 03 11:35:52 crc kubenswrapper[5052]: I1203 11:35:52.846979 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-operators/horizon-operator-controller-manager-68c6d99b8f-mjdvh"] Dec 03 11:35:52 crc kubenswrapper[5052]: I1203 11:35:52.847892 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-kvnlb\" (UniqueName: \"kubernetes.io/projected/d429f7ab-9a88-4411-a7a5-88bc8ab76285-kube-api-access-kvnlb\") pod \"glance-operator-controller-manager-77987cd8cd-btn2l\" (UID: \"d429f7ab-9a88-4411-a7a5-88bc8ab76285\") " pod="openstack-operators/glance-operator-controller-manager-77987cd8cd-btn2l" Dec 03 11:35:52 crc kubenswrapper[5052]: I1203 11:35:52.847959 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h6smm\" (UniqueName: \"kubernetes.io/projected/7017b3f7-1eb9-4dfe-8ef3-6db27cf3a1f1-kube-api-access-h6smm\") pod \"ironic-operator-controller-manager-6c548fd776-pqfnd\" (UID: \"7017b3f7-1eb9-4dfe-8ef3-6db27cf3a1f1\") " pod="openstack-operators/ironic-operator-controller-manager-6c548fd776-pqfnd" Dec 03 11:35:52 crc kubenswrapper[5052]: I1203 11:35:52.848002 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-7shls\" (UniqueName: \"kubernetes.io/projected/ce3d34f6-e545-4eb2-bcc5-999377931601-kube-api-access-7shls\") pod \"horizon-operator-controller-manager-68c6d99b8f-mjdvh\" (UID: \"ce3d34f6-e545-4eb2-bcc5-999377931601\") " pod="openstack-operators/horizon-operator-controller-manager-68c6d99b8f-mjdvh" Dec 03 11:35:52 crc kubenswrapper[5052]: I1203 11:35:52.848025 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/81f5f8e1-015a-4b45-83e6-13e88efb8e80-cert\") pod \"infra-operator-controller-manager-57548d458d-9fkll\" (UID: \"81f5f8e1-015a-4b45-83e6-13e88efb8e80\") " pod="openstack-operators/infra-operator-controller-manager-57548d458d-9fkll" Dec 03 11:35:52 crc kubenswrapper[5052]: I1203 11:35:52.848056 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-wgp7x\" (UniqueName: \"kubernetes.io/projected/0cbd234c-ecb3-4e54-af86-a2b15fdd94a3-kube-api-access-wgp7x\") pod \"heat-operator-controller-manager-5f64f6f8bb-48fnb\" (UID: \"0cbd234c-ecb3-4e54-af86-a2b15fdd94a3\") " pod="openstack-operators/heat-operator-controller-manager-5f64f6f8bb-48fnb" Dec 03 11:35:52 crc kubenswrapper[5052]: I1203 11:35:52.848099 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-klcr8\" (UniqueName: \"kubernetes.io/projected/81f5f8e1-015a-4b45-83e6-13e88efb8e80-kube-api-access-klcr8\") pod \"infra-operator-controller-manager-57548d458d-9fkll\" (UID: \"81f5f8e1-015a-4b45-83e6-13e88efb8e80\") " pod="openstack-operators/infra-operator-controller-manager-57548d458d-9fkll" Dec 03 11:35:52 crc kubenswrapper[5052]: E1203 11:35:52.850769 5052 secret.go:188] Couldn't get secret openstack-operators/infra-operator-webhook-server-cert: secret "infra-operator-webhook-server-cert" not found Dec 03 11:35:52 crc kubenswrapper[5052]: E1203 11:35:52.850831 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/81f5f8e1-015a-4b45-83e6-13e88efb8e80-cert podName:81f5f8e1-015a-4b45-83e6-13e88efb8e80 nodeName:}" failed. No retries permitted until 2025-12-03 11:35:53.350810167 +0000 UTC m=+943.939051931 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/81f5f8e1-015a-4b45-83e6-13e88efb8e80-cert") pod "infra-operator-controller-manager-57548d458d-9fkll" (UID: "81f5f8e1-015a-4b45-83e6-13e88efb8e80") : secret "infra-operator-webhook-server-cert" not found Dec 03 11:35:52 crc kubenswrapper[5052]: I1203 11:35:52.870259 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-fkq98\" (UniqueName: \"kubernetes.io/projected/5968d273-33dd-45cc-aab0-e83628bcd15c-kube-api-access-fkq98\") pod \"cinder-operator-controller-manager-859b6ccc6-vt9d2\" (UID: \"5968d273-33dd-45cc-aab0-e83628bcd15c\") " pod="openstack-operators/cinder-operator-controller-manager-859b6ccc6-vt9d2" Dec 03 11:35:52 crc kubenswrapper[5052]: I1203 11:35:52.877730 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-operators/barbican-operator-controller-manager-7d9dfd778-cqfdk" Dec 03 11:35:52 crc kubenswrapper[5052]: I1203 11:35:52.885464 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-klcr8\" (UniqueName: \"kubernetes.io/projected/81f5f8e1-015a-4b45-83e6-13e88efb8e80-kube-api-access-klcr8\") pod \"infra-operator-controller-manager-57548d458d-9fkll\" (UID: \"81f5f8e1-015a-4b45-83e6-13e88efb8e80\") " pod="openstack-operators/infra-operator-controller-manager-57548d458d-9fkll" Dec 03 11:35:52 crc kubenswrapper[5052]: I1203 11:35:52.898158 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-wgp7x\" (UniqueName: \"kubernetes.io/projected/0cbd234c-ecb3-4e54-af86-a2b15fdd94a3-kube-api-access-wgp7x\") pod \"heat-operator-controller-manager-5f64f6f8bb-48fnb\" (UID: \"0cbd234c-ecb3-4e54-af86-a2b15fdd94a3\") " pod="openstack-operators/heat-operator-controller-manager-5f64f6f8bb-48fnb" Dec 03 11:35:52 crc kubenswrapper[5052]: I1203 11:35:52.898465 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-operators/cinder-operator-controller-manager-859b6ccc6-vt9d2" Dec 03 11:35:52 crc kubenswrapper[5052]: I1203 11:35:52.908454 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-operators/infra-operator-controller-manager-57548d458d-9fkll"] Dec 03 11:35:52 crc kubenswrapper[5052]: I1203 11:35:52.915989 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-7shls\" (UniqueName: \"kubernetes.io/projected/ce3d34f6-e545-4eb2-bcc5-999377931601-kube-api-access-7shls\") pod \"horizon-operator-controller-manager-68c6d99b8f-mjdvh\" (UID: \"ce3d34f6-e545-4eb2-bcc5-999377931601\") " pod="openstack-operators/horizon-operator-controller-manager-68c6d99b8f-mjdvh" Dec 03 11:35:52 crc kubenswrapper[5052]: I1203 11:35:52.916274 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-operators/designate-operator-controller-manager-78b4bc895b-874g2" Dec 03 11:35:52 crc kubenswrapper[5052]: I1203 11:35:52.917496 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-kvnlb\" (UniqueName: \"kubernetes.io/projected/d429f7ab-9a88-4411-a7a5-88bc8ab76285-kube-api-access-kvnlb\") pod \"glance-operator-controller-manager-77987cd8cd-btn2l\" (UID: \"d429f7ab-9a88-4411-a7a5-88bc8ab76285\") " pod="openstack-operators/glance-operator-controller-manager-77987cd8cd-btn2l" Dec 03 11:35:52 crc kubenswrapper[5052]: I1203 11:35:52.931576 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-operators/ironic-operator-controller-manager-6c548fd776-pqfnd"] Dec 03 11:35:52 crc kubenswrapper[5052]: I1203 11:35:52.946601 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-operators/keystone-operator-controller-manager-7765d96ddf-nlrl9"] Dec 03 11:35:52 crc kubenswrapper[5052]: I1203 11:35:52.947696 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-operators/keystone-operator-controller-manager-7765d96ddf-nlrl9" Dec 03 11:35:52 crc kubenswrapper[5052]: I1203 11:35:52.947835 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-operators/glance-operator-controller-manager-77987cd8cd-btn2l" Dec 03 11:35:52 crc kubenswrapper[5052]: I1203 11:35:52.956806 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sw44h\" (UniqueName: \"kubernetes.io/projected/95b50980-fd5a-4a19-b1cd-13df98d257fe-kube-api-access-sw44h\") pod \"keystone-operator-controller-manager-7765d96ddf-nlrl9\" (UID: \"95b50980-fd5a-4a19-b1cd-13df98d257fe\") " pod="openstack-operators/keystone-operator-controller-manager-7765d96ddf-nlrl9" Dec 03 11:35:52 crc kubenswrapper[5052]: I1203 11:35:52.956851 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-h6smm\" (UniqueName: \"kubernetes.io/projected/7017b3f7-1eb9-4dfe-8ef3-6db27cf3a1f1-kube-api-access-h6smm\") pod \"ironic-operator-controller-manager-6c548fd776-pqfnd\" (UID: \"7017b3f7-1eb9-4dfe-8ef3-6db27cf3a1f1\") " pod="openstack-operators/ironic-operator-controller-manager-6c548fd776-pqfnd" Dec 03 11:35:52 crc kubenswrapper[5052]: I1203 11:35:52.966982 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-operators/keystone-operator-controller-manager-7765d96ddf-nlrl9"] Dec 03 11:35:52 crc kubenswrapper[5052]: I1203 11:35:52.992160 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-h6smm\" (UniqueName: \"kubernetes.io/projected/7017b3f7-1eb9-4dfe-8ef3-6db27cf3a1f1-kube-api-access-h6smm\") pod \"ironic-operator-controller-manager-6c548fd776-pqfnd\" (UID: \"7017b3f7-1eb9-4dfe-8ef3-6db27cf3a1f1\") " pod="openstack-operators/ironic-operator-controller-manager-6c548fd776-pqfnd" Dec 03 11:35:52 crc kubenswrapper[5052]: I1203 11:35:52.993111 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-operators/heat-operator-controller-manager-5f64f6f8bb-48fnb" Dec 03 11:35:52 crc kubenswrapper[5052]: I1203 11:35:52.993205 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-operators"/"keystone-operator-controller-manager-dockercfg-kcds5" Dec 03 11:35:52 crc kubenswrapper[5052]: I1203 11:35:52.993587 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-operators/mariadb-operator-controller-manager-56bbcc9d85-sr7zw"] Dec 03 11:35:52 crc kubenswrapper[5052]: I1203 11:35:52.994656 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-operators/mariadb-operator-controller-manager-56bbcc9d85-sr7zw" Dec 03 11:35:53 crc kubenswrapper[5052]: I1203 11:35:53.002325 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-operators"/"mariadb-operator-controller-manager-dockercfg-ptwqt" Dec 03 11:35:53 crc kubenswrapper[5052]: I1203 11:35:53.008887 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-operators/manila-operator-controller-manager-7c79b5df47-ccbqq"] Dec 03 11:35:53 crc kubenswrapper[5052]: I1203 11:35:53.012658 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-operators/manila-operator-controller-manager-7c79b5df47-ccbqq" Dec 03 11:35:53 crc kubenswrapper[5052]: I1203 11:35:53.015643 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-operators"/"manila-operator-controller-manager-dockercfg-n9llg" Dec 03 11:35:53 crc kubenswrapper[5052]: I1203 11:35:53.022784 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-operators/mariadb-operator-controller-manager-56bbcc9d85-sr7zw"] Dec 03 11:35:53 crc kubenswrapper[5052]: I1203 11:35:53.039150 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-operators/horizon-operator-controller-manager-68c6d99b8f-mjdvh" Dec 03 11:35:53 crc kubenswrapper[5052]: I1203 11:35:53.041251 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-operators/neutron-operator-controller-manager-5fdfd5b6b5-q48nd"] Dec 03 11:35:53 crc kubenswrapper[5052]: I1203 11:35:53.042712 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-operators/neutron-operator-controller-manager-5fdfd5b6b5-q48nd" Dec 03 11:35:53 crc kubenswrapper[5052]: I1203 11:35:53.054192 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-operators"/"neutron-operator-controller-manager-dockercfg-vdzfp" Dec 03 11:35:53 crc kubenswrapper[5052]: I1203 11:35:53.058833 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cpczm\" (UniqueName: \"kubernetes.io/projected/f0e70c0d-10fe-43b7-a08c-0ce411f223a6-kube-api-access-cpczm\") pod \"mariadb-operator-controller-manager-56bbcc9d85-sr7zw\" (UID: \"f0e70c0d-10fe-43b7-a08c-0ce411f223a6\") " pod="openstack-operators/mariadb-operator-controller-manager-56bbcc9d85-sr7zw" Dec 03 11:35:53 crc kubenswrapper[5052]: I1203 11:35:53.058892 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-sw44h\" (UniqueName: \"kubernetes.io/projected/95b50980-fd5a-4a19-b1cd-13df98d257fe-kube-api-access-sw44h\") pod \"keystone-operator-controller-manager-7765d96ddf-nlrl9\" (UID: \"95b50980-fd5a-4a19-b1cd-13df98d257fe\") " pod="openstack-operators/keystone-operator-controller-manager-7765d96ddf-nlrl9" Dec 03 11:35:53 crc kubenswrapper[5052]: I1203 11:35:53.058941 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j8lx5\" (UniqueName: \"kubernetes.io/projected/ea1e91de-7e7d-498a-a17b-551f2bee325a-kube-api-access-j8lx5\") pod \"manila-operator-controller-manager-7c79b5df47-ccbqq\" (UID: \"ea1e91de-7e7d-498a-a17b-551f2bee325a\") " pod="openstack-operators/manila-operator-controller-manager-7c79b5df47-ccbqq" Dec 03 11:35:53 crc kubenswrapper[5052]: I1203 11:35:53.058986 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j2rjq\" (UniqueName: \"kubernetes.io/projected/53c88ca3-39f7-4a81-b305-40047bcaeb88-kube-api-access-j2rjq\") pod \"neutron-operator-controller-manager-5fdfd5b6b5-q48nd\" (UID: \"53c88ca3-39f7-4a81-b305-40047bcaeb88\") " pod="openstack-operators/neutron-operator-controller-manager-5fdfd5b6b5-q48nd" Dec 03 11:35:53 crc kubenswrapper[5052]: I1203 11:35:53.069633 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-operators/manila-operator-controller-manager-7c79b5df47-ccbqq"] Dec 03 11:35:53 crc kubenswrapper[5052]: I1203 11:35:53.075827 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-operators/neutron-operator-controller-manager-5fdfd5b6b5-q48nd"] Dec 03 11:35:53 crc kubenswrapper[5052]: I1203 11:35:53.083851 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-sw44h\" (UniqueName: \"kubernetes.io/projected/95b50980-fd5a-4a19-b1cd-13df98d257fe-kube-api-access-sw44h\") pod \"keystone-operator-controller-manager-7765d96ddf-nlrl9\" (UID: \"95b50980-fd5a-4a19-b1cd-13df98d257fe\") " pod="openstack-operators/keystone-operator-controller-manager-7765d96ddf-nlrl9" Dec 03 11:35:53 crc kubenswrapper[5052]: I1203 11:35:53.099404 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-operators/nova-operator-controller-manager-697bc559fc-gcj5d"] Dec 03 11:35:53 crc kubenswrapper[5052]: I1203 11:35:53.100798 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-operators/nova-operator-controller-manager-697bc559fc-gcj5d" Dec 03 11:35:53 crc kubenswrapper[5052]: I1203 11:35:53.103289 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-operators"/"nova-operator-controller-manager-dockercfg-hm7ll" Dec 03 11:35:53 crc kubenswrapper[5052]: I1203 11:35:53.119105 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-operators/octavia-operator-controller-manager-998648c74-z6cw9"] Dec 03 11:35:53 crc kubenswrapper[5052]: I1203 11:35:53.120433 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-operators/octavia-operator-controller-manager-998648c74-z6cw9" Dec 03 11:35:53 crc kubenswrapper[5052]: I1203 11:35:53.127598 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-operators"/"octavia-operator-controller-manager-dockercfg-x6ddf" Dec 03 11:35:53 crc kubenswrapper[5052]: I1203 11:35:53.137501 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-operators/nova-operator-controller-manager-697bc559fc-gcj5d"] Dec 03 11:35:53 crc kubenswrapper[5052]: I1203 11:35:53.143465 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-operators/octavia-operator-controller-manager-998648c74-z6cw9"] Dec 03 11:35:53 crc kubenswrapper[5052]: I1203 11:35:53.155439 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-operators/openstack-baremetal-operator-controller-manager-55d86b6686f2mft"] Dec 03 11:35:53 crc kubenswrapper[5052]: I1203 11:35:53.156662 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-operators/openstack-baremetal-operator-controller-manager-55d86b6686f2mft" Dec 03 11:35:53 crc kubenswrapper[5052]: I1203 11:35:53.161389 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-j8lx5\" (UniqueName: \"kubernetes.io/projected/ea1e91de-7e7d-498a-a17b-551f2bee325a-kube-api-access-j8lx5\") pod \"manila-operator-controller-manager-7c79b5df47-ccbqq\" (UID: \"ea1e91de-7e7d-498a-a17b-551f2bee325a\") " pod="openstack-operators/manila-operator-controller-manager-7c79b5df47-ccbqq" Dec 03 11:35:53 crc kubenswrapper[5052]: I1203 11:35:53.161524 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6pqvt\" (UniqueName: \"kubernetes.io/projected/3e6bc3f6-1f1a-4b99-8835-98252369d364-kube-api-access-6pqvt\") pod \"nova-operator-controller-manager-697bc559fc-gcj5d\" (UID: \"3e6bc3f6-1f1a-4b99-8835-98252369d364\") " pod="openstack-operators/nova-operator-controller-manager-697bc559fc-gcj5d" Dec 03 11:35:53 crc kubenswrapper[5052]: I1203 11:35:53.161568 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-j2rjq\" (UniqueName: \"kubernetes.io/projected/53c88ca3-39f7-4a81-b305-40047bcaeb88-kube-api-access-j2rjq\") pod \"neutron-operator-controller-manager-5fdfd5b6b5-q48nd\" (UID: \"53c88ca3-39f7-4a81-b305-40047bcaeb88\") " pod="openstack-operators/neutron-operator-controller-manager-5fdfd5b6b5-q48nd" Dec 03 11:35:53 crc kubenswrapper[5052]: I1203 11:35:53.161588 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-operators/ovn-operator-controller-manager-b6456fdb6-sggpb"] Dec 03 11:35:53 crc kubenswrapper[5052]: I1203 11:35:53.161637 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f44vw\" (UniqueName: \"kubernetes.io/projected/50c6e918-7f1f-4d6f-ba5f-940e5f485825-kube-api-access-f44vw\") pod \"octavia-operator-controller-manager-998648c74-z6cw9\" (UID: \"50c6e918-7f1f-4d6f-ba5f-940e5f485825\") " pod="openstack-operators/octavia-operator-controller-manager-998648c74-z6cw9" Dec 03 11:35:53 crc kubenswrapper[5052]: I1203 11:35:53.161734 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-cpczm\" (UniqueName: \"kubernetes.io/projected/f0e70c0d-10fe-43b7-a08c-0ce411f223a6-kube-api-access-cpczm\") pod \"mariadb-operator-controller-manager-56bbcc9d85-sr7zw\" (UID: \"f0e70c0d-10fe-43b7-a08c-0ce411f223a6\") " pod="openstack-operators/mariadb-operator-controller-manager-56bbcc9d85-sr7zw" Dec 03 11:35:53 crc kubenswrapper[5052]: I1203 11:35:53.162891 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-operators/ovn-operator-controller-manager-b6456fdb6-sggpb" Dec 03 11:35:53 crc kubenswrapper[5052]: I1203 11:35:53.162983 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-operators"/"openstack-baremetal-operator-controller-manager-dockercfg-5qffs" Dec 03 11:35:53 crc kubenswrapper[5052]: I1203 11:35:53.163225 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-operators"/"openstack-baremetal-operator-webhook-server-cert" Dec 03 11:35:53 crc kubenswrapper[5052]: I1203 11:35:53.169278 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-operators"/"ovn-operator-controller-manager-dockercfg-k88xx" Dec 03 11:35:53 crc kubenswrapper[5052]: I1203 11:35:53.169477 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-operators/ovn-operator-controller-manager-b6456fdb6-sggpb"] Dec 03 11:35:53 crc kubenswrapper[5052]: I1203 11:35:53.176592 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-operators/openstack-baremetal-operator-controller-manager-55d86b6686f2mft"] Dec 03 11:35:53 crc kubenswrapper[5052]: I1203 11:35:53.189310 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-j2rjq\" (UniqueName: \"kubernetes.io/projected/53c88ca3-39f7-4a81-b305-40047bcaeb88-kube-api-access-j2rjq\") pod \"neutron-operator-controller-manager-5fdfd5b6b5-q48nd\" (UID: \"53c88ca3-39f7-4a81-b305-40047bcaeb88\") " pod="openstack-operators/neutron-operator-controller-manager-5fdfd5b6b5-q48nd" Dec 03 11:35:53 crc kubenswrapper[5052]: I1203 11:35:53.197335 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-j8lx5\" (UniqueName: \"kubernetes.io/projected/ea1e91de-7e7d-498a-a17b-551f2bee325a-kube-api-access-j8lx5\") pod \"manila-operator-controller-manager-7c79b5df47-ccbqq\" (UID: \"ea1e91de-7e7d-498a-a17b-551f2bee325a\") " pod="openstack-operators/manila-operator-controller-manager-7c79b5df47-ccbqq" Dec 03 11:35:53 crc kubenswrapper[5052]: I1203 11:35:53.202456 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-cpczm\" (UniqueName: \"kubernetes.io/projected/f0e70c0d-10fe-43b7-a08c-0ce411f223a6-kube-api-access-cpczm\") pod \"mariadb-operator-controller-manager-56bbcc9d85-sr7zw\" (UID: \"f0e70c0d-10fe-43b7-a08c-0ce411f223a6\") " pod="openstack-operators/mariadb-operator-controller-manager-56bbcc9d85-sr7zw" Dec 03 11:35:53 crc kubenswrapper[5052]: I1203 11:35:53.202592 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-operators/placement-operator-controller-manager-78f8948974-j62hd"] Dec 03 11:35:53 crc kubenswrapper[5052]: I1203 11:35:53.206279 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-operators/placement-operator-controller-manager-78f8948974-j62hd" Dec 03 11:35:53 crc kubenswrapper[5052]: I1203 11:35:53.212877 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-operators"/"placement-operator-controller-manager-dockercfg-jmqn9" Dec 03 11:35:53 crc kubenswrapper[5052]: I1203 11:35:53.257792 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-operators/ironic-operator-controller-manager-6c548fd776-pqfnd" Dec 03 11:35:53 crc kubenswrapper[5052]: I1203 11:35:53.263205 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e10da887-70df-43aa-aa79-4a14231e5ec9-cert\") pod \"openstack-baremetal-operator-controller-manager-55d86b6686f2mft\" (UID: \"e10da887-70df-43aa-aa79-4a14231e5ec9\") " pod="openstack-operators/openstack-baremetal-operator-controller-manager-55d86b6686f2mft" Dec 03 11:35:53 crc kubenswrapper[5052]: I1203 11:35:53.263250 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6jkbw\" (UniqueName: \"kubernetes.io/projected/a605d632-d9a5-4ce0-9449-e3d57105df1b-kube-api-access-6jkbw\") pod \"ovn-operator-controller-manager-b6456fdb6-sggpb\" (UID: \"a605d632-d9a5-4ce0-9449-e3d57105df1b\") " pod="openstack-operators/ovn-operator-controller-manager-b6456fdb6-sggpb" Dec 03 11:35:53 crc kubenswrapper[5052]: I1203 11:35:53.263275 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-f44vw\" (UniqueName: \"kubernetes.io/projected/50c6e918-7f1f-4d6f-ba5f-940e5f485825-kube-api-access-f44vw\") pod \"octavia-operator-controller-manager-998648c74-z6cw9\" (UID: \"50c6e918-7f1f-4d6f-ba5f-940e5f485825\") " pod="openstack-operators/octavia-operator-controller-manager-998648c74-z6cw9" Dec 03 11:35:53 crc kubenswrapper[5052]: I1203 11:35:53.263311 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bpxzs\" (UniqueName: \"kubernetes.io/projected/37ab421b-0ad3-4d9d-bf47-8bf908c4aa57-kube-api-access-bpxzs\") pod \"placement-operator-controller-manager-78f8948974-j62hd\" (UID: \"37ab421b-0ad3-4d9d-bf47-8bf908c4aa57\") " pod="openstack-operators/placement-operator-controller-manager-78f8948974-j62hd" Dec 03 11:35:53 crc kubenswrapper[5052]: I1203 11:35:53.263391 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2cfz6\" (UniqueName: \"kubernetes.io/projected/e10da887-70df-43aa-aa79-4a14231e5ec9-kube-api-access-2cfz6\") pod \"openstack-baremetal-operator-controller-manager-55d86b6686f2mft\" (UID: \"e10da887-70df-43aa-aa79-4a14231e5ec9\") " pod="openstack-operators/openstack-baremetal-operator-controller-manager-55d86b6686f2mft" Dec 03 11:35:53 crc kubenswrapper[5052]: I1203 11:35:53.263428 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-6pqvt\" (UniqueName: \"kubernetes.io/projected/3e6bc3f6-1f1a-4b99-8835-98252369d364-kube-api-access-6pqvt\") pod \"nova-operator-controller-manager-697bc559fc-gcj5d\" (UID: \"3e6bc3f6-1f1a-4b99-8835-98252369d364\") " pod="openstack-operators/nova-operator-controller-manager-697bc559fc-gcj5d" Dec 03 11:35:53 crc kubenswrapper[5052]: I1203 11:35:53.289305 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-f44vw\" (UniqueName: \"kubernetes.io/projected/50c6e918-7f1f-4d6f-ba5f-940e5f485825-kube-api-access-f44vw\") pod \"octavia-operator-controller-manager-998648c74-z6cw9\" (UID: \"50c6e918-7f1f-4d6f-ba5f-940e5f485825\") " pod="openstack-operators/octavia-operator-controller-manager-998648c74-z6cw9" Dec 03 11:35:53 crc kubenswrapper[5052]: I1203 11:35:53.290288 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-6pqvt\" (UniqueName: \"kubernetes.io/projected/3e6bc3f6-1f1a-4b99-8835-98252369d364-kube-api-access-6pqvt\") pod \"nova-operator-controller-manager-697bc559fc-gcj5d\" (UID: \"3e6bc3f6-1f1a-4b99-8835-98252369d364\") " pod="openstack-operators/nova-operator-controller-manager-697bc559fc-gcj5d" Dec 03 11:35:53 crc kubenswrapper[5052]: I1203 11:35:53.299331 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-operators/keystone-operator-controller-manager-7765d96ddf-nlrl9" Dec 03 11:35:53 crc kubenswrapper[5052]: I1203 11:35:53.328230 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-operators/placement-operator-controller-manager-78f8948974-j62hd"] Dec 03 11:35:53 crc kubenswrapper[5052]: I1203 11:35:53.352634 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-operators/telemetry-operator-controller-manager-76cc84c6bb-zl7px"] Dec 03 11:35:53 crc kubenswrapper[5052]: I1203 11:35:53.356214 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-operators/mariadb-operator-controller-manager-56bbcc9d85-sr7zw" Dec 03 11:35:53 crc kubenswrapper[5052]: I1203 11:35:53.360800 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-operators/telemetry-operator-controller-manager-76cc84c6bb-zl7px" Dec 03 11:35:53 crc kubenswrapper[5052]: I1203 11:35:53.364894 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-2cfz6\" (UniqueName: \"kubernetes.io/projected/e10da887-70df-43aa-aa79-4a14231e5ec9-kube-api-access-2cfz6\") pod \"openstack-baremetal-operator-controller-manager-55d86b6686f2mft\" (UID: \"e10da887-70df-43aa-aa79-4a14231e5ec9\") " pod="openstack-operators/openstack-baremetal-operator-controller-manager-55d86b6686f2mft" Dec 03 11:35:53 crc kubenswrapper[5052]: I1203 11:35:53.364928 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/81f5f8e1-015a-4b45-83e6-13e88efb8e80-cert\") pod \"infra-operator-controller-manager-57548d458d-9fkll\" (UID: \"81f5f8e1-015a-4b45-83e6-13e88efb8e80\") " pod="openstack-operators/infra-operator-controller-manager-57548d458d-9fkll" Dec 03 11:35:53 crc kubenswrapper[5052]: I1203 11:35:53.364972 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e10da887-70df-43aa-aa79-4a14231e5ec9-cert\") pod \"openstack-baremetal-operator-controller-manager-55d86b6686f2mft\" (UID: \"e10da887-70df-43aa-aa79-4a14231e5ec9\") " pod="openstack-operators/openstack-baremetal-operator-controller-manager-55d86b6686f2mft" Dec 03 11:35:53 crc kubenswrapper[5052]: I1203 11:35:53.364991 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-6jkbw\" (UniqueName: \"kubernetes.io/projected/a605d632-d9a5-4ce0-9449-e3d57105df1b-kube-api-access-6jkbw\") pod \"ovn-operator-controller-manager-b6456fdb6-sggpb\" (UID: \"a605d632-d9a5-4ce0-9449-e3d57105df1b\") " pod="openstack-operators/ovn-operator-controller-manager-b6456fdb6-sggpb" Dec 03 11:35:53 crc kubenswrapper[5052]: I1203 11:35:53.365016 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-bpxzs\" (UniqueName: \"kubernetes.io/projected/37ab421b-0ad3-4d9d-bf47-8bf908c4aa57-kube-api-access-bpxzs\") pod \"placement-operator-controller-manager-78f8948974-j62hd\" (UID: \"37ab421b-0ad3-4d9d-bf47-8bf908c4aa57\") " pod="openstack-operators/placement-operator-controller-manager-78f8948974-j62hd" Dec 03 11:35:53 crc kubenswrapper[5052]: I1203 11:35:53.366391 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-operators"/"telemetry-operator-controller-manager-dockercfg-5b6xg" Dec 03 11:35:53 crc kubenswrapper[5052]: E1203 11:35:53.368147 5052 secret.go:188] Couldn't get secret openstack-operators/openstack-baremetal-operator-webhook-server-cert: secret "openstack-baremetal-operator-webhook-server-cert" not found Dec 03 11:35:53 crc kubenswrapper[5052]: E1203 11:35:53.368210 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e10da887-70df-43aa-aa79-4a14231e5ec9-cert podName:e10da887-70df-43aa-aa79-4a14231e5ec9 nodeName:}" failed. No retries permitted until 2025-12-03 11:35:53.8681861 +0000 UTC m=+944.456428054 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/e10da887-70df-43aa-aa79-4a14231e5ec9-cert") pod "openstack-baremetal-operator-controller-manager-55d86b6686f2mft" (UID: "e10da887-70df-43aa-aa79-4a14231e5ec9") : secret "openstack-baremetal-operator-webhook-server-cert" not found Dec 03 11:35:53 crc kubenswrapper[5052]: E1203 11:35:53.368692 5052 secret.go:188] Couldn't get secret openstack-operators/infra-operator-webhook-server-cert: secret "infra-operator-webhook-server-cert" not found Dec 03 11:35:53 crc kubenswrapper[5052]: E1203 11:35:53.368725 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/81f5f8e1-015a-4b45-83e6-13e88efb8e80-cert podName:81f5f8e1-015a-4b45-83e6-13e88efb8e80 nodeName:}" failed. No retries permitted until 2025-12-03 11:35:54.368715725 +0000 UTC m=+944.956957489 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/81f5f8e1-015a-4b45-83e6-13e88efb8e80-cert") pod "infra-operator-controller-manager-57548d458d-9fkll" (UID: "81f5f8e1-015a-4b45-83e6-13e88efb8e80") : secret "infra-operator-webhook-server-cert" not found Dec 03 11:35:53 crc kubenswrapper[5052]: I1203 11:35:53.378523 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-operators/swift-operator-controller-manager-5f8c65bbfc-2cgdz"] Dec 03 11:35:53 crc kubenswrapper[5052]: I1203 11:35:53.385247 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-operators/swift-operator-controller-manager-5f8c65bbfc-2cgdz" Dec 03 11:35:53 crc kubenswrapper[5052]: I1203 11:35:53.388276 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-operators"/"swift-operator-controller-manager-dockercfg-wztz8" Dec 03 11:35:53 crc kubenswrapper[5052]: I1203 11:35:53.388792 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-operators/manila-operator-controller-manager-7c79b5df47-ccbqq" Dec 03 11:35:53 crc kubenswrapper[5052]: I1203 11:35:53.408219 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-bpxzs\" (UniqueName: \"kubernetes.io/projected/37ab421b-0ad3-4d9d-bf47-8bf908c4aa57-kube-api-access-bpxzs\") pod \"placement-operator-controller-manager-78f8948974-j62hd\" (UID: \"37ab421b-0ad3-4d9d-bf47-8bf908c4aa57\") " pod="openstack-operators/placement-operator-controller-manager-78f8948974-j62hd" Dec 03 11:35:53 crc kubenswrapper[5052]: I1203 11:35:53.413314 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-6jkbw\" (UniqueName: \"kubernetes.io/projected/a605d632-d9a5-4ce0-9449-e3d57105df1b-kube-api-access-6jkbw\") pod \"ovn-operator-controller-manager-b6456fdb6-sggpb\" (UID: \"a605d632-d9a5-4ce0-9449-e3d57105df1b\") " pod="openstack-operators/ovn-operator-controller-manager-b6456fdb6-sggpb" Dec 03 11:35:53 crc kubenswrapper[5052]: I1203 11:35:53.415196 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-2cfz6\" (UniqueName: \"kubernetes.io/projected/e10da887-70df-43aa-aa79-4a14231e5ec9-kube-api-access-2cfz6\") pod \"openstack-baremetal-operator-controller-manager-55d86b6686f2mft\" (UID: \"e10da887-70df-43aa-aa79-4a14231e5ec9\") " pod="openstack-operators/openstack-baremetal-operator-controller-manager-55d86b6686f2mft" Dec 03 11:35:53 crc kubenswrapper[5052]: I1203 11:35:53.417171 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-operators/swift-operator-controller-manager-5f8c65bbfc-2cgdz"] Dec 03 11:35:53 crc kubenswrapper[5052]: I1203 11:35:53.424688 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-operators/neutron-operator-controller-manager-5fdfd5b6b5-q48nd" Dec 03 11:35:53 crc kubenswrapper[5052]: I1203 11:35:53.434751 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-operators/nova-operator-controller-manager-697bc559fc-gcj5d" Dec 03 11:35:53 crc kubenswrapper[5052]: I1203 11:35:53.477594 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-operators/test-operator-controller-manager-5854674fcc-jltb6"] Dec 03 11:35:53 crc kubenswrapper[5052]: I1203 11:35:53.481244 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vpllc\" (UniqueName: \"kubernetes.io/projected/8fd08931-7f0b-4d5b-bfb1-f27980648561-kube-api-access-vpllc\") pod \"swift-operator-controller-manager-5f8c65bbfc-2cgdz\" (UID: \"8fd08931-7f0b-4d5b-bfb1-f27980648561\") " pod="openstack-operators/swift-operator-controller-manager-5f8c65bbfc-2cgdz" Dec 03 11:35:53 crc kubenswrapper[5052]: I1203 11:35:53.481363 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pmdm6\" (UniqueName: \"kubernetes.io/projected/3894016b-8242-4b12-8407-bcf78af00851-kube-api-access-pmdm6\") pod \"telemetry-operator-controller-manager-76cc84c6bb-zl7px\" (UID: \"3894016b-8242-4b12-8407-bcf78af00851\") " pod="openstack-operators/telemetry-operator-controller-manager-76cc84c6bb-zl7px" Dec 03 11:35:53 crc kubenswrapper[5052]: I1203 11:35:53.490434 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-operators/octavia-operator-controller-manager-998648c74-z6cw9" Dec 03 11:35:53 crc kubenswrapper[5052]: I1203 11:35:53.493553 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-operators/test-operator-controller-manager-5854674fcc-jltb6" Dec 03 11:35:53 crc kubenswrapper[5052]: I1203 11:35:53.496695 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-operators"/"test-operator-controller-manager-dockercfg-hhjpf" Dec 03 11:35:53 crc kubenswrapper[5052]: I1203 11:35:53.497460 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-operators/telemetry-operator-controller-manager-76cc84c6bb-zl7px"] Dec 03 11:35:53 crc kubenswrapper[5052]: I1203 11:35:53.511787 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-operators/test-operator-controller-manager-5854674fcc-jltb6"] Dec 03 11:35:53 crc kubenswrapper[5052]: I1203 11:35:53.525962 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-operators/watcher-operator-controller-manager-769dc69bc-7rhg6"] Dec 03 11:35:53 crc kubenswrapper[5052]: I1203 11:35:53.527103 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-operators/watcher-operator-controller-manager-769dc69bc-7rhg6" Dec 03 11:35:53 crc kubenswrapper[5052]: I1203 11:35:53.527396 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-operators/ovn-operator-controller-manager-b6456fdb6-sggpb" Dec 03 11:35:53 crc kubenswrapper[5052]: I1203 11:35:53.530059 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-operators"/"watcher-operator-controller-manager-dockercfg-mz86b" Dec 03 11:35:53 crc kubenswrapper[5052]: I1203 11:35:53.533306 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-operators/watcher-operator-controller-manager-769dc69bc-7rhg6"] Dec 03 11:35:53 crc kubenswrapper[5052]: I1203 11:35:53.551214 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-operators/placement-operator-controller-manager-78f8948974-j62hd" Dec 03 11:35:53 crc kubenswrapper[5052]: I1203 11:35:53.579313 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-operators/openstack-operator-controller-manager-9f56fc979-b8jv8"] Dec 03 11:35:53 crc kubenswrapper[5052]: I1203 11:35:53.580930 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-operators/openstack-operator-controller-manager-9f56fc979-b8jv8" Dec 03 11:35:53 crc kubenswrapper[5052]: I1203 11:35:53.582266 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-vpllc\" (UniqueName: \"kubernetes.io/projected/8fd08931-7f0b-4d5b-bfb1-f27980648561-kube-api-access-vpllc\") pod \"swift-operator-controller-manager-5f8c65bbfc-2cgdz\" (UID: \"8fd08931-7f0b-4d5b-bfb1-f27980648561\") " pod="openstack-operators/swift-operator-controller-manager-5f8c65bbfc-2cgdz" Dec 03 11:35:53 crc kubenswrapper[5052]: I1203 11:35:53.582467 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-pmdm6\" (UniqueName: \"kubernetes.io/projected/3894016b-8242-4b12-8407-bcf78af00851-kube-api-access-pmdm6\") pod \"telemetry-operator-controller-manager-76cc84c6bb-zl7px\" (UID: \"3894016b-8242-4b12-8407-bcf78af00851\") " pod="openstack-operators/telemetry-operator-controller-manager-76cc84c6bb-zl7px" Dec 03 11:35:53 crc kubenswrapper[5052]: I1203 11:35:53.583228 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-operators"/"metrics-server-cert" Dec 03 11:35:53 crc kubenswrapper[5052]: I1203 11:35:53.583440 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-operators"/"openstack-operator-controller-manager-dockercfg-st26m" Dec 03 11:35:53 crc kubenswrapper[5052]: I1203 11:35:53.583807 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-operators"/"webhook-server-cert" Dec 03 11:35:53 crc kubenswrapper[5052]: I1203 11:35:53.586353 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-operators/openstack-operator-controller-manager-9f56fc979-b8jv8"] Dec 03 11:35:53 crc kubenswrapper[5052]: I1203 11:35:53.598677 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-operators/rabbitmq-cluster-operator-manager-668c99d594-ksw2p"] Dec 03 11:35:53 crc kubenswrapper[5052]: I1203 11:35:53.600010 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-operators/rabbitmq-cluster-operator-manager-668c99d594-ksw2p" Dec 03 11:35:53 crc kubenswrapper[5052]: I1203 11:35:53.602954 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-operators"/"rabbitmq-cluster-operator-controller-manager-dockercfg-b95zf" Dec 03 11:35:53 crc kubenswrapper[5052]: I1203 11:35:53.614882 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-operators/rabbitmq-cluster-operator-manager-668c99d594-ksw2p"] Dec 03 11:35:53 crc kubenswrapper[5052]: I1203 11:35:53.623289 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-pmdm6\" (UniqueName: \"kubernetes.io/projected/3894016b-8242-4b12-8407-bcf78af00851-kube-api-access-pmdm6\") pod \"telemetry-operator-controller-manager-76cc84c6bb-zl7px\" (UID: \"3894016b-8242-4b12-8407-bcf78af00851\") " pod="openstack-operators/telemetry-operator-controller-manager-76cc84c6bb-zl7px" Dec 03 11:35:53 crc kubenswrapper[5052]: I1203 11:35:53.624154 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-vpllc\" (UniqueName: \"kubernetes.io/projected/8fd08931-7f0b-4d5b-bfb1-f27980648561-kube-api-access-vpllc\") pod \"swift-operator-controller-manager-5f8c65bbfc-2cgdz\" (UID: \"8fd08931-7f0b-4d5b-bfb1-f27980648561\") " pod="openstack-operators/swift-operator-controller-manager-5f8c65bbfc-2cgdz" Dec 03 11:35:53 crc kubenswrapper[5052]: I1203 11:35:53.687047 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-certs\" (UniqueName: \"kubernetes.io/secret/527bcd50-6d8d-4d4f-bc11-5382eb9adfc2-webhook-certs\") pod \"openstack-operator-controller-manager-9f56fc979-b8jv8\" (UID: \"527bcd50-6d8d-4d4f-bc11-5382eb9adfc2\") " pod="openstack-operators/openstack-operator-controller-manager-9f56fc979-b8jv8" Dec 03 11:35:53 crc kubenswrapper[5052]: I1203 11:35:53.687121 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dsx9h\" (UniqueName: \"kubernetes.io/projected/1cb3d739-e4d6-4058-9456-a3c2e1c09130-kube-api-access-dsx9h\") pod \"test-operator-controller-manager-5854674fcc-jltb6\" (UID: \"1cb3d739-e4d6-4058-9456-a3c2e1c09130\") " pod="openstack-operators/test-operator-controller-manager-5854674fcc-jltb6" Dec 03 11:35:53 crc kubenswrapper[5052]: I1203 11:35:53.687243 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/527bcd50-6d8d-4d4f-bc11-5382eb9adfc2-metrics-certs\") pod \"openstack-operator-controller-manager-9f56fc979-b8jv8\" (UID: \"527bcd50-6d8d-4d4f-bc11-5382eb9adfc2\") " pod="openstack-operators/openstack-operator-controller-manager-9f56fc979-b8jv8" Dec 03 11:35:53 crc kubenswrapper[5052]: I1203 11:35:53.687269 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gxjqd\" (UniqueName: \"kubernetes.io/projected/6bea5f42-82da-4148-9996-dc05121212f7-kube-api-access-gxjqd\") pod \"watcher-operator-controller-manager-769dc69bc-7rhg6\" (UID: \"6bea5f42-82da-4148-9996-dc05121212f7\") " pod="openstack-operators/watcher-operator-controller-manager-769dc69bc-7rhg6" Dec 03 11:35:53 crc kubenswrapper[5052]: I1203 11:35:53.687324 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rwsmp\" (UniqueName: \"kubernetes.io/projected/527bcd50-6d8d-4d4f-bc11-5382eb9adfc2-kube-api-access-rwsmp\") pod \"openstack-operator-controller-manager-9f56fc979-b8jv8\" (UID: \"527bcd50-6d8d-4d4f-bc11-5382eb9adfc2\") " pod="openstack-operators/openstack-operator-controller-manager-9f56fc979-b8jv8" Dec 03 11:35:53 crc kubenswrapper[5052]: I1203 11:35:53.788752 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"webhook-certs\" (UniqueName: \"kubernetes.io/secret/527bcd50-6d8d-4d4f-bc11-5382eb9adfc2-webhook-certs\") pod \"openstack-operator-controller-manager-9f56fc979-b8jv8\" (UID: \"527bcd50-6d8d-4d4f-bc11-5382eb9adfc2\") " pod="openstack-operators/openstack-operator-controller-manager-9f56fc979-b8jv8" Dec 03 11:35:53 crc kubenswrapper[5052]: I1203 11:35:53.788842 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7dk26\" (UniqueName: \"kubernetes.io/projected/459f5555-efae-4c19-84d0-2af917b9560f-kube-api-access-7dk26\") pod \"rabbitmq-cluster-operator-manager-668c99d594-ksw2p\" (UID: \"459f5555-efae-4c19-84d0-2af917b9560f\") " pod="openstack-operators/rabbitmq-cluster-operator-manager-668c99d594-ksw2p" Dec 03 11:35:53 crc kubenswrapper[5052]: I1203 11:35:53.788882 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-dsx9h\" (UniqueName: \"kubernetes.io/projected/1cb3d739-e4d6-4058-9456-a3c2e1c09130-kube-api-access-dsx9h\") pod \"test-operator-controller-manager-5854674fcc-jltb6\" (UID: \"1cb3d739-e4d6-4058-9456-a3c2e1c09130\") " pod="openstack-operators/test-operator-controller-manager-5854674fcc-jltb6" Dec 03 11:35:53 crc kubenswrapper[5052]: I1203 11:35:53.788962 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-gxjqd\" (UniqueName: \"kubernetes.io/projected/6bea5f42-82da-4148-9996-dc05121212f7-kube-api-access-gxjqd\") pod \"watcher-operator-controller-manager-769dc69bc-7rhg6\" (UID: \"6bea5f42-82da-4148-9996-dc05121212f7\") " pod="openstack-operators/watcher-operator-controller-manager-769dc69bc-7rhg6" Dec 03 11:35:53 crc kubenswrapper[5052]: I1203 11:35:53.788992 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/527bcd50-6d8d-4d4f-bc11-5382eb9adfc2-metrics-certs\") pod \"openstack-operator-controller-manager-9f56fc979-b8jv8\" (UID: \"527bcd50-6d8d-4d4f-bc11-5382eb9adfc2\") " pod="openstack-operators/openstack-operator-controller-manager-9f56fc979-b8jv8" Dec 03 11:35:53 crc kubenswrapper[5052]: I1203 11:35:53.789029 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-rwsmp\" (UniqueName: \"kubernetes.io/projected/527bcd50-6d8d-4d4f-bc11-5382eb9adfc2-kube-api-access-rwsmp\") pod \"openstack-operator-controller-manager-9f56fc979-b8jv8\" (UID: \"527bcd50-6d8d-4d4f-bc11-5382eb9adfc2\") " pod="openstack-operators/openstack-operator-controller-manager-9f56fc979-b8jv8" Dec 03 11:35:53 crc kubenswrapper[5052]: E1203 11:35:53.789411 5052 secret.go:188] Couldn't get secret openstack-operators/webhook-server-cert: secret "webhook-server-cert" not found Dec 03 11:35:53 crc kubenswrapper[5052]: E1203 11:35:53.789569 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/527bcd50-6d8d-4d4f-bc11-5382eb9adfc2-webhook-certs podName:527bcd50-6d8d-4d4f-bc11-5382eb9adfc2 nodeName:}" failed. No retries permitted until 2025-12-03 11:35:54.289545245 +0000 UTC m=+944.877787009 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-certs" (UniqueName: "kubernetes.io/secret/527bcd50-6d8d-4d4f-bc11-5382eb9adfc2-webhook-certs") pod "openstack-operator-controller-manager-9f56fc979-b8jv8" (UID: "527bcd50-6d8d-4d4f-bc11-5382eb9adfc2") : secret "webhook-server-cert" not found Dec 03 11:35:53 crc kubenswrapper[5052]: E1203 11:35:53.789593 5052 secret.go:188] Couldn't get secret openstack-operators/metrics-server-cert: secret "metrics-server-cert" not found Dec 03 11:35:53 crc kubenswrapper[5052]: E1203 11:35:53.789806 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/527bcd50-6d8d-4d4f-bc11-5382eb9adfc2-metrics-certs podName:527bcd50-6d8d-4d4f-bc11-5382eb9adfc2 nodeName:}" failed. No retries permitted until 2025-12-03 11:35:54.289771351 +0000 UTC m=+944.878013315 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/527bcd50-6d8d-4d4f-bc11-5382eb9adfc2-metrics-certs") pod "openstack-operator-controller-manager-9f56fc979-b8jv8" (UID: "527bcd50-6d8d-4d4f-bc11-5382eb9adfc2") : secret "metrics-server-cert" not found Dec 03 11:35:53 crc kubenswrapper[5052]: I1203 11:35:53.817467 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-gxjqd\" (UniqueName: \"kubernetes.io/projected/6bea5f42-82da-4148-9996-dc05121212f7-kube-api-access-gxjqd\") pod \"watcher-operator-controller-manager-769dc69bc-7rhg6\" (UID: \"6bea5f42-82da-4148-9996-dc05121212f7\") " pod="openstack-operators/watcher-operator-controller-manager-769dc69bc-7rhg6" Dec 03 11:35:53 crc kubenswrapper[5052]: I1203 11:35:53.818947 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-rwsmp\" (UniqueName: \"kubernetes.io/projected/527bcd50-6d8d-4d4f-bc11-5382eb9adfc2-kube-api-access-rwsmp\") pod \"openstack-operator-controller-manager-9f56fc979-b8jv8\" (UID: \"527bcd50-6d8d-4d4f-bc11-5382eb9adfc2\") " pod="openstack-operators/openstack-operator-controller-manager-9f56fc979-b8jv8" Dec 03 11:35:53 crc kubenswrapper[5052]: I1203 11:35:53.820296 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-dsx9h\" (UniqueName: \"kubernetes.io/projected/1cb3d739-e4d6-4058-9456-a3c2e1c09130-kube-api-access-dsx9h\") pod \"test-operator-controller-manager-5854674fcc-jltb6\" (UID: \"1cb3d739-e4d6-4058-9456-a3c2e1c09130\") " pod="openstack-operators/test-operator-controller-manager-5854674fcc-jltb6" Dec 03 11:35:53 crc kubenswrapper[5052]: I1203 11:35:53.839214 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-operators/telemetry-operator-controller-manager-76cc84c6bb-zl7px" Dec 03 11:35:53 crc kubenswrapper[5052]: I1203 11:35:53.872296 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-operators/swift-operator-controller-manager-5f8c65bbfc-2cgdz" Dec 03 11:35:53 crc kubenswrapper[5052]: I1203 11:35:53.878169 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-operators/test-operator-controller-manager-5854674fcc-jltb6" Dec 03 11:35:53 crc kubenswrapper[5052]: I1203 11:35:53.891219 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e10da887-70df-43aa-aa79-4a14231e5ec9-cert\") pod \"openstack-baremetal-operator-controller-manager-55d86b6686f2mft\" (UID: \"e10da887-70df-43aa-aa79-4a14231e5ec9\") " pod="openstack-operators/openstack-baremetal-operator-controller-manager-55d86b6686f2mft" Dec 03 11:35:53 crc kubenswrapper[5052]: I1203 11:35:53.891271 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-7dk26\" (UniqueName: \"kubernetes.io/projected/459f5555-efae-4c19-84d0-2af917b9560f-kube-api-access-7dk26\") pod \"rabbitmq-cluster-operator-manager-668c99d594-ksw2p\" (UID: \"459f5555-efae-4c19-84d0-2af917b9560f\") " pod="openstack-operators/rabbitmq-cluster-operator-manager-668c99d594-ksw2p" Dec 03 11:35:53 crc kubenswrapper[5052]: E1203 11:35:53.891784 5052 secret.go:188] Couldn't get secret openstack-operators/openstack-baremetal-operator-webhook-server-cert: secret "openstack-baremetal-operator-webhook-server-cert" not found Dec 03 11:35:53 crc kubenswrapper[5052]: E1203 11:35:53.891837 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e10da887-70df-43aa-aa79-4a14231e5ec9-cert podName:e10da887-70df-43aa-aa79-4a14231e5ec9 nodeName:}" failed. No retries permitted until 2025-12-03 11:35:54.891823942 +0000 UTC m=+945.480065706 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/e10da887-70df-43aa-aa79-4a14231e5ec9-cert") pod "openstack-baremetal-operator-controller-manager-55d86b6686f2mft" (UID: "e10da887-70df-43aa-aa79-4a14231e5ec9") : secret "openstack-baremetal-operator-webhook-server-cert" not found Dec 03 11:35:53 crc kubenswrapper[5052]: I1203 11:35:53.917942 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-7dk26\" (UniqueName: \"kubernetes.io/projected/459f5555-efae-4c19-84d0-2af917b9560f-kube-api-access-7dk26\") pod \"rabbitmq-cluster-operator-manager-668c99d594-ksw2p\" (UID: \"459f5555-efae-4c19-84d0-2af917b9560f\") " pod="openstack-operators/rabbitmq-cluster-operator-manager-668c99d594-ksw2p" Dec 03 11:35:53 crc kubenswrapper[5052]: I1203 11:35:53.930676 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-operators/watcher-operator-controller-manager-769dc69bc-7rhg6" Dec 03 11:35:53 crc kubenswrapper[5052]: I1203 11:35:53.963457 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-operators/rabbitmq-cluster-operator-manager-668c99d594-ksw2p" Dec 03 11:35:53 crc kubenswrapper[5052]: I1203 11:35:53.999678 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-operators/barbican-operator-controller-manager-7d9dfd778-cqfdk"] Dec 03 11:35:54 crc kubenswrapper[5052]: I1203 11:35:54.018278 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-operators/heat-operator-controller-manager-5f64f6f8bb-48fnb"] Dec 03 11:35:54 crc kubenswrapper[5052]: I1203 11:35:54.042560 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-operators/designate-operator-controller-manager-78b4bc895b-874g2"] Dec 03 11:35:54 crc kubenswrapper[5052]: I1203 11:35:54.049793 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-operators/horizon-operator-controller-manager-68c6d99b8f-mjdvh"] Dec 03 11:35:54 crc kubenswrapper[5052]: I1203 11:35:54.059536 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-operators/cinder-operator-controller-manager-859b6ccc6-vt9d2"] Dec 03 11:35:54 crc kubenswrapper[5052]: I1203 11:35:54.319576 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-operators/manila-operator-controller-manager-7c79b5df47-ccbqq"] Dec 03 11:35:54 crc kubenswrapper[5052]: I1203 11:35:54.328204 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/527bcd50-6d8d-4d4f-bc11-5382eb9adfc2-metrics-certs\") pod \"openstack-operator-controller-manager-9f56fc979-b8jv8\" (UID: \"527bcd50-6d8d-4d4f-bc11-5382eb9adfc2\") " pod="openstack-operators/openstack-operator-controller-manager-9f56fc979-b8jv8" Dec 03 11:35:54 crc kubenswrapper[5052]: I1203 11:35:54.328362 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"webhook-certs\" (UniqueName: \"kubernetes.io/secret/527bcd50-6d8d-4d4f-bc11-5382eb9adfc2-webhook-certs\") pod \"openstack-operator-controller-manager-9f56fc979-b8jv8\" (UID: \"527bcd50-6d8d-4d4f-bc11-5382eb9adfc2\") " pod="openstack-operators/openstack-operator-controller-manager-9f56fc979-b8jv8" Dec 03 11:35:54 crc kubenswrapper[5052]: E1203 11:35:54.328648 5052 secret.go:188] Couldn't get secret openstack-operators/webhook-server-cert: secret "webhook-server-cert" not found Dec 03 11:35:54 crc kubenswrapper[5052]: E1203 11:35:54.328732 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/527bcd50-6d8d-4d4f-bc11-5382eb9adfc2-webhook-certs podName:527bcd50-6d8d-4d4f-bc11-5382eb9adfc2 nodeName:}" failed. No retries permitted until 2025-12-03 11:35:55.328708394 +0000 UTC m=+945.916950158 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "webhook-certs" (UniqueName: "kubernetes.io/secret/527bcd50-6d8d-4d4f-bc11-5382eb9adfc2-webhook-certs") pod "openstack-operator-controller-manager-9f56fc979-b8jv8" (UID: "527bcd50-6d8d-4d4f-bc11-5382eb9adfc2") : secret "webhook-server-cert" not found Dec 03 11:35:54 crc kubenswrapper[5052]: E1203 11:35:54.328969 5052 secret.go:188] Couldn't get secret openstack-operators/metrics-server-cert: secret "metrics-server-cert" not found Dec 03 11:35:54 crc kubenswrapper[5052]: E1203 11:35:54.329079 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/527bcd50-6d8d-4d4f-bc11-5382eb9adfc2-metrics-certs podName:527bcd50-6d8d-4d4f-bc11-5382eb9adfc2 nodeName:}" failed. No retries permitted until 2025-12-03 11:35:55.329050984 +0000 UTC m=+945.917292748 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/527bcd50-6d8d-4d4f-bc11-5382eb9adfc2-metrics-certs") pod "openstack-operator-controller-manager-9f56fc979-b8jv8" (UID: "527bcd50-6d8d-4d4f-bc11-5382eb9adfc2") : secret "metrics-server-cert" not found Dec 03 11:35:54 crc kubenswrapper[5052]: I1203 11:35:54.344841 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-operators/glance-operator-controller-manager-77987cd8cd-btn2l"] Dec 03 11:35:54 crc kubenswrapper[5052]: I1203 11:35:54.352415 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-operators/keystone-operator-controller-manager-7765d96ddf-nlrl9"] Dec 03 11:35:54 crc kubenswrapper[5052]: W1203 11:35:54.360917 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea1e91de_7e7d_498a_a17b_551f2bee325a.slice/crio-de9266c7e68b4ad9a6760a4efe0699ed57cc7eaba3f772d4b1d0f34a3589e5a0 WatchSource:0}: Error finding container de9266c7e68b4ad9a6760a4efe0699ed57cc7eaba3f772d4b1d0f34a3589e5a0: Status 404 returned error can't find the container with id de9266c7e68b4ad9a6760a4efe0699ed57cc7eaba3f772d4b1d0f34a3589e5a0 Dec 03 11:35:54 crc kubenswrapper[5052]: I1203 11:35:54.391747 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-operators/octavia-operator-controller-manager-998648c74-z6cw9"] Dec 03 11:35:54 crc kubenswrapper[5052]: I1203 11:35:54.398340 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-operators/ironic-operator-controller-manager-6c548fd776-pqfnd"] Dec 03 11:35:54 crc kubenswrapper[5052]: I1203 11:35:54.433462 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/81f5f8e1-015a-4b45-83e6-13e88efb8e80-cert\") pod \"infra-operator-controller-manager-57548d458d-9fkll\" (UID: \"81f5f8e1-015a-4b45-83e6-13e88efb8e80\") " pod="openstack-operators/infra-operator-controller-manager-57548d458d-9fkll" Dec 03 11:35:54 crc kubenswrapper[5052]: E1203 11:35:54.433713 5052 secret.go:188] Couldn't get secret openstack-operators/infra-operator-webhook-server-cert: secret "infra-operator-webhook-server-cert" not found Dec 03 11:35:54 crc kubenswrapper[5052]: E1203 11:35:54.433808 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/81f5f8e1-015a-4b45-83e6-13e88efb8e80-cert podName:81f5f8e1-015a-4b45-83e6-13e88efb8e80 nodeName:}" failed. No retries permitted until 2025-12-03 11:35:56.433786913 +0000 UTC m=+947.022028677 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/81f5f8e1-015a-4b45-83e6-13e88efb8e80-cert") pod "infra-operator-controller-manager-57548d458d-9fkll" (UID: "81f5f8e1-015a-4b45-83e6-13e88efb8e80") : secret "infra-operator-webhook-server-cert" not found Dec 03 11:35:54 crc kubenswrapper[5052]: W1203 11:35:54.546124 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3894016b_8242_4b12_8407_bcf78af00851.slice/crio-0f99870dc82861c793dbad4149f80170c0c9dc2378102d4f80a980b605b8485f WatchSource:0}: Error finding container 0f99870dc82861c793dbad4149f80170c0c9dc2378102d4f80a980b605b8485f: Status 404 returned error can't find the container with id 0f99870dc82861c793dbad4149f80170c0c9dc2378102d4f80a980b605b8485f Dec 03 11:35:54 crc kubenswrapper[5052]: I1203 11:35:54.551611 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-operators/neutron-operator-controller-manager-5fdfd5b6b5-q48nd"] Dec 03 11:35:54 crc kubenswrapper[5052]: E1203 11:35:54.568462 5052 kuberuntime_manager.go:1274] "Unhandled Error" err="container &Container{Name:manager,Image:quay.io/openstack-k8s-operators/nova-operator@sha256:779f0cee6024d0fb8f259b036fe790e62aa5a3b0431ea9bf15a6e7d02e2e5670,Command:[/manager],Args:[--leader-elect --health-probe-bind-address=:8081 --metrics-bind-address=127.0.0.1:8080],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{EnvVar{Name:LEASE_DURATION,Value:30,ValueFrom:nil,},EnvVar{Name:RENEW_DEADLINE,Value:20,ValueFrom:nil,},EnvVar{Name:RETRY_PERIOD,Value:5,ValueFrom:nil,},EnvVar{Name:ENABLE_WEBHOOKS,Value:false,ValueFrom:nil,},EnvVar{Name:METRICS_CERTS,Value:false,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{500 -3} {} 500m DecimalSI},memory: {{536870912 0} {} BinarySI},},Requests:ResourceList{cpu: {{10 -3} {} 10m DecimalSI},memory: {{268435456 0} {} BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:kube-api-access-6pqvt,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/healthz,Port:{0 8081 },Host:,Scheme:HTTP,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:15,TimeoutSeconds:1,PeriodSeconds:20,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/readyz,Port:{0 8081 },Host:,Scheme:HTTP,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:1,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[MKNOD],},Privileged:nil,SELinuxOptions:nil,RunAsUser:nil,RunAsNonRoot:nil,ReadOnlyRootFilesystem:nil,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod nova-operator-controller-manager-697bc559fc-gcj5d_openstack-operators(3e6bc3f6-1f1a-4b99-8835-98252369d364): ErrImagePull: pull QPS exceeded" logger="UnhandledError" Dec 03 11:35:54 crc kubenswrapper[5052]: E1203 11:35:54.576368 5052 kuberuntime_manager.go:1274] "Unhandled Error" err="container &Container{Name:kube-rbac-proxy,Image:quay.io/openstack-k8s-operators/kube-rbac-proxy:v0.16.0,Command:[],Args:[--secure-listen-address=0.0.0.0:8443 --upstream=http://127.0.0.1:8080/ --logtostderr=true --v=0],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{500 -3} {} 500m DecimalSI},memory: {{134217728 0} {} BinarySI},},Requests:ResourceList{cpu: {{5 -3} {} 5m DecimalSI},memory: {{67108864 0} {} BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:kube-api-access-6pqvt,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[MKNOD],},Privileged:nil,SELinuxOptions:nil,RunAsUser:nil,RunAsNonRoot:nil,ReadOnlyRootFilesystem:nil,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod nova-operator-controller-manager-697bc559fc-gcj5d_openstack-operators(3e6bc3f6-1f1a-4b99-8835-98252369d364): ErrImagePull: pull QPS exceeded" logger="UnhandledError" Dec 03 11:35:54 crc kubenswrapper[5052]: E1203 11:35:54.577540 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="[failed to \"StartContainer\" for \"manager\" with ErrImagePull: \"pull QPS exceeded\", failed to \"StartContainer\" for \"kube-rbac-proxy\" with ErrImagePull: \"pull QPS exceeded\"]" pod="openstack-operators/nova-operator-controller-manager-697bc559fc-gcj5d" podUID="3e6bc3f6-1f1a-4b99-8835-98252369d364" Dec 03 11:35:54 crc kubenswrapper[5052]: W1203 11:35:54.585743 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0e70c0d_10fe_43b7_a08c_0ce411f223a6.slice/crio-63b967be8542405d76ece84234f3c81710dfda97bca61633bc0f8bb8e114ed85 WatchSource:0}: Error finding container 63b967be8542405d76ece84234f3c81710dfda97bca61633bc0f8bb8e114ed85: Status 404 returned error can't find the container with id 63b967be8542405d76ece84234f3c81710dfda97bca61633bc0f8bb8e114ed85 Dec 03 11:35:54 crc kubenswrapper[5052]: I1203 11:35:54.586866 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-operators/telemetry-operator-controller-manager-76cc84c6bb-zl7px"] Dec 03 11:35:54 crc kubenswrapper[5052]: I1203 11:35:54.602773 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-operators/nova-operator-controller-manager-697bc559fc-gcj5d"] Dec 03 11:35:54 crc kubenswrapper[5052]: E1203 11:35:54.602878 5052 kuberuntime_manager.go:1274] "Unhandled Error" err="container &Container{Name:manager,Image:quay.io/openstack-k8s-operators/mariadb-operator@sha256:600ca007e493d3af0fcc2ebac92e8da5efd2afe812b62d7d3d4dd0115bdf05d7,Command:[/manager],Args:[--leader-elect --health-probe-bind-address=:8081 --metrics-bind-address=127.0.0.1:8080],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{EnvVar{Name:LEASE_DURATION,Value:30,ValueFrom:nil,},EnvVar{Name:RENEW_DEADLINE,Value:20,ValueFrom:nil,},EnvVar{Name:RETRY_PERIOD,Value:5,ValueFrom:nil,},EnvVar{Name:ENABLE_WEBHOOKS,Value:false,ValueFrom:nil,},EnvVar{Name:METRICS_CERTS,Value:false,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{500 -3} {} 500m DecimalSI},memory: {{536870912 0} {} BinarySI},},Requests:ResourceList{cpu: {{10 -3} {} 10m DecimalSI},memory: {{268435456 0} {} BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:kube-api-access-cpczm,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/healthz,Port:{0 8081 },Host:,Scheme:HTTP,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:15,TimeoutSeconds:1,PeriodSeconds:20,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/readyz,Port:{0 8081 },Host:,Scheme:HTTP,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:1,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[MKNOD],},Privileged:nil,SELinuxOptions:nil,RunAsUser:nil,RunAsNonRoot:nil,ReadOnlyRootFilesystem:nil,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod mariadb-operator-controller-manager-56bbcc9d85-sr7zw_openstack-operators(f0e70c0d-10fe-43b7-a08c-0ce411f223a6): ErrImagePull: pull QPS exceeded" logger="UnhandledError" Dec 03 11:35:54 crc kubenswrapper[5052]: I1203 11:35:54.604584 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-operators/mariadb-operator-controller-manager-56bbcc9d85-sr7zw"] Dec 03 11:35:54 crc kubenswrapper[5052]: E1203 11:35:54.605228 5052 kuberuntime_manager.go:1274] "Unhandled Error" err="container &Container{Name:kube-rbac-proxy,Image:quay.io/openstack-k8s-operators/kube-rbac-proxy:v0.16.0,Command:[],Args:[--secure-listen-address=0.0.0.0:8443 --upstream=http://127.0.0.1:8080/ --logtostderr=true --v=0],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{500 -3} {} 500m DecimalSI},memory: {{134217728 0} {} BinarySI},},Requests:ResourceList{cpu: {{5 -3} {} 5m DecimalSI},memory: {{67108864 0} {} BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:kube-api-access-cpczm,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[MKNOD],},Privileged:nil,SELinuxOptions:nil,RunAsUser:nil,RunAsNonRoot:nil,ReadOnlyRootFilesystem:nil,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod mariadb-operator-controller-manager-56bbcc9d85-sr7zw_openstack-operators(f0e70c0d-10fe-43b7-a08c-0ce411f223a6): ErrImagePull: pull QPS exceeded" logger="UnhandledError" Dec 03 11:35:54 crc kubenswrapper[5052]: E1203 11:35:54.606517 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="[failed to \"StartContainer\" for \"manager\" with ErrImagePull: \"pull QPS exceeded\", failed to \"StartContainer\" for \"kube-rbac-proxy\" with ErrImagePull: \"pull QPS exceeded\"]" pod="openstack-operators/mariadb-operator-controller-manager-56bbcc9d85-sr7zw" podUID="f0e70c0d-10fe-43b7-a08c-0ce411f223a6" Dec 03 11:35:54 crc kubenswrapper[5052]: I1203 11:35:54.663696 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-operators/ovn-operator-controller-manager-b6456fdb6-sggpb"] Dec 03 11:35:54 crc kubenswrapper[5052]: E1203 11:35:54.668190 5052 kuberuntime_manager.go:1274] "Unhandled Error" err="container &Container{Name:manager,Image:quay.io/openstack-k8s-operators/ovn-operator@sha256:635a4aef9d6f0b799e8ec91333dbb312160c001d05b3c63f614c124e0b67cb59,Command:[/manager],Args:[--leader-elect --health-probe-bind-address=:8081 --metrics-bind-address=127.0.0.1:8080],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{EnvVar{Name:LEASE_DURATION,Value:30,ValueFrom:nil,},EnvVar{Name:RENEW_DEADLINE,Value:20,ValueFrom:nil,},EnvVar{Name:RETRY_PERIOD,Value:5,ValueFrom:nil,},EnvVar{Name:ENABLE_WEBHOOKS,Value:false,ValueFrom:nil,},EnvVar{Name:METRICS_CERTS,Value:false,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{500 -3} {} 500m DecimalSI},memory: {{536870912 0} {} BinarySI},},Requests:ResourceList{cpu: {{10 -3} {} 10m DecimalSI},memory: {{268435456 0} {} BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:kube-api-access-6jkbw,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/healthz,Port:{0 8081 },Host:,Scheme:HTTP,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:15,TimeoutSeconds:1,PeriodSeconds:20,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/readyz,Port:{0 8081 },Host:,Scheme:HTTP,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:1,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[MKNOD],},Privileged:nil,SELinuxOptions:nil,RunAsUser:nil,RunAsNonRoot:nil,ReadOnlyRootFilesystem:nil,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod ovn-operator-controller-manager-b6456fdb6-sggpb_openstack-operators(a605d632-d9a5-4ce0-9449-e3d57105df1b): ErrImagePull: pull QPS exceeded" logger="UnhandledError" Dec 03 11:35:54 crc kubenswrapper[5052]: I1203 11:35:54.670557 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-operators/placement-operator-controller-manager-78f8948974-j62hd"] Dec 03 11:35:54 crc kubenswrapper[5052]: E1203 11:35:54.674052 5052 kuberuntime_manager.go:1274] "Unhandled Error" err="container &Container{Name:kube-rbac-proxy,Image:quay.io/openstack-k8s-operators/kube-rbac-proxy:v0.16.0,Command:[],Args:[--secure-listen-address=0.0.0.0:8443 --upstream=http://127.0.0.1:8080/ --logtostderr=true --v=0],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{500 -3} {} 500m DecimalSI},memory: {{134217728 0} {} BinarySI},},Requests:ResourceList{cpu: {{5 -3} {} 5m DecimalSI},memory: {{67108864 0} {} BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:kube-api-access-6jkbw,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[MKNOD],},Privileged:nil,SELinuxOptions:nil,RunAsUser:nil,RunAsNonRoot:nil,ReadOnlyRootFilesystem:nil,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod ovn-operator-controller-manager-b6456fdb6-sggpb_openstack-operators(a605d632-d9a5-4ce0-9449-e3d57105df1b): ErrImagePull: pull QPS exceeded" logger="UnhandledError" Dec 03 11:35:54 crc kubenswrapper[5052]: E1203 11:35:54.675198 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="[failed to \"StartContainer\" for \"manager\" with ErrImagePull: \"pull QPS exceeded\", failed to \"StartContainer\" for \"kube-rbac-proxy\" with ErrImagePull: \"pull QPS exceeded\"]" pod="openstack-operators/ovn-operator-controller-manager-b6456fdb6-sggpb" podUID="a605d632-d9a5-4ce0-9449-e3d57105df1b" Dec 03 11:35:54 crc kubenswrapper[5052]: E1203 11:35:54.679150 5052 kuberuntime_manager.go:1274] "Unhandled Error" err="container &Container{Name:manager,Image:quay.io/openstack-k8s-operators/placement-operator@sha256:d29650b006da97eb9178fcc58f2eb9fead8c2b414fac18f86a3c3a1507488c4f,Command:[/manager],Args:[--leader-elect --health-probe-bind-address=:8081 --metrics-bind-address=127.0.0.1:8080],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{EnvVar{Name:LEASE_DURATION,Value:30,ValueFrom:nil,},EnvVar{Name:RENEW_DEADLINE,Value:20,ValueFrom:nil,},EnvVar{Name:RETRY_PERIOD,Value:5,ValueFrom:nil,},EnvVar{Name:ENABLE_WEBHOOKS,Value:false,ValueFrom:nil,},EnvVar{Name:METRICS_CERTS,Value:false,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{500 -3} {} 500m DecimalSI},memory: {{536870912 0} {} BinarySI},},Requests:ResourceList{cpu: {{10 -3} {} 10m DecimalSI},memory: {{268435456 0} {} BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:kube-api-access-bpxzs,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/healthz,Port:{0 8081 },Host:,Scheme:HTTP,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:15,TimeoutSeconds:1,PeriodSeconds:20,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/readyz,Port:{0 8081 },Host:,Scheme:HTTP,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:1,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[MKNOD],},Privileged:nil,SELinuxOptions:nil,RunAsUser:nil,RunAsNonRoot:nil,ReadOnlyRootFilesystem:nil,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod placement-operator-controller-manager-78f8948974-j62hd_openstack-operators(37ab421b-0ad3-4d9d-bf47-8bf908c4aa57): ErrImagePull: pull QPS exceeded" logger="UnhandledError" Dec 03 11:35:54 crc kubenswrapper[5052]: E1203 11:35:54.680961 5052 kuberuntime_manager.go:1274] "Unhandled Error" err="container &Container{Name:kube-rbac-proxy,Image:quay.io/openstack-k8s-operators/kube-rbac-proxy:v0.16.0,Command:[],Args:[--secure-listen-address=0.0.0.0:8443 --upstream=http://127.0.0.1:8080/ --logtostderr=true --v=0],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{500 -3} {} 500m DecimalSI},memory: {{134217728 0} {} BinarySI},},Requests:ResourceList{cpu: {{5 -3} {} 5m DecimalSI},memory: {{67108864 0} {} BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:kube-api-access-bpxzs,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[MKNOD],},Privileged:nil,SELinuxOptions:nil,RunAsUser:nil,RunAsNonRoot:nil,ReadOnlyRootFilesystem:nil,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod placement-operator-controller-manager-78f8948974-j62hd_openstack-operators(37ab421b-0ad3-4d9d-bf47-8bf908c4aa57): ErrImagePull: pull QPS exceeded" logger="UnhandledError" Dec 03 11:35:54 crc kubenswrapper[5052]: E1203 11:35:54.682758 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="[failed to \"StartContainer\" for \"manager\" with ErrImagePull: \"pull QPS exceeded\", failed to \"StartContainer\" for \"kube-rbac-proxy\" with ErrImagePull: \"pull QPS exceeded\"]" pod="openstack-operators/placement-operator-controller-manager-78f8948974-j62hd" podUID="37ab421b-0ad3-4d9d-bf47-8bf908c4aa57" Dec 03 11:35:54 crc kubenswrapper[5052]: I1203 11:35:54.759225 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-operators/swift-operator-controller-manager-5f8c65bbfc-2cgdz"] Dec 03 11:35:54 crc kubenswrapper[5052]: I1203 11:35:54.771864 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-operators/test-operator-controller-manager-5854674fcc-jltb6"] Dec 03 11:35:54 crc kubenswrapper[5052]: I1203 11:35:54.793368 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-operators/watcher-operator-controller-manager-769dc69bc-7rhg6"] Dec 03 11:35:54 crc kubenswrapper[5052]: E1203 11:35:54.800680 5052 kuberuntime_manager.go:1274] "Unhandled Error" err="container &Container{Name:manager,Image:quay.io/openstack-k8s-operators/test-operator@sha256:101b3e007d8c9f2e183262d7712f986ad51256448099069bc14f1ea5f997ab94,Command:[/manager],Args:[--leader-elect --health-probe-bind-address=:8081 --metrics-bind-address=127.0.0.1:8080],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{EnvVar{Name:LEASE_DURATION,Value:30,ValueFrom:nil,},EnvVar{Name:RENEW_DEADLINE,Value:20,ValueFrom:nil,},EnvVar{Name:RETRY_PERIOD,Value:5,ValueFrom:nil,},EnvVar{Name:ENABLE_WEBHOOKS,Value:false,ValueFrom:nil,},EnvVar{Name:METRICS_CERTS,Value:false,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{500 -3} {} 500m DecimalSI},memory: {{536870912 0} {} BinarySI},},Requests:ResourceList{cpu: {{10 -3} {} 10m DecimalSI},memory: {{268435456 0} {} BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:kube-api-access-dsx9h,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/healthz,Port:{0 8081 },Host:,Scheme:HTTP,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:15,TimeoutSeconds:1,PeriodSeconds:20,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/readyz,Port:{0 8081 },Host:,Scheme:HTTP,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:1,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[MKNOD],},Privileged:nil,SELinuxOptions:nil,RunAsUser:nil,RunAsNonRoot:nil,ReadOnlyRootFilesystem:nil,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod test-operator-controller-manager-5854674fcc-jltb6_openstack-operators(1cb3d739-e4d6-4058-9456-a3c2e1c09130): ErrImagePull: pull QPS exceeded" logger="UnhandledError" Dec 03 11:35:54 crc kubenswrapper[5052]: E1203 11:35:54.807172 5052 kuberuntime_manager.go:1274] "Unhandled Error" err="container &Container{Name:kube-rbac-proxy,Image:quay.io/openstack-k8s-operators/kube-rbac-proxy:v0.16.0,Command:[],Args:[--secure-listen-address=0.0.0.0:8443 --upstream=http://127.0.0.1:8080/ --logtostderr=true --v=0],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{500 -3} {} 500m DecimalSI},memory: {{134217728 0} {} BinarySI},},Requests:ResourceList{cpu: {{5 -3} {} 5m DecimalSI},memory: {{67108864 0} {} BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:kube-api-access-dsx9h,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[MKNOD],},Privileged:nil,SELinuxOptions:nil,RunAsUser:nil,RunAsNonRoot:nil,ReadOnlyRootFilesystem:nil,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod test-operator-controller-manager-5854674fcc-jltb6_openstack-operators(1cb3d739-e4d6-4058-9456-a3c2e1c09130): ErrImagePull: pull QPS exceeded" logger="UnhandledError" Dec 03 11:35:54 crc kubenswrapper[5052]: E1203 11:35:54.808424 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="[failed to \"StartContainer\" for \"manager\" with ErrImagePull: \"pull QPS exceeded\", failed to \"StartContainer\" for \"kube-rbac-proxy\" with ErrImagePull: \"pull QPS exceeded\"]" pod="openstack-operators/test-operator-controller-manager-5854674fcc-jltb6" podUID="1cb3d739-e4d6-4058-9456-a3c2e1c09130" Dec 03 11:35:54 crc kubenswrapper[5052]: E1203 11:35:54.835650 5052 kuberuntime_manager.go:1274] "Unhandled Error" err="container &Container{Name:manager,Image:quay.io/openstack-k8s-operators/watcher-operator@sha256:9aa8c03633e4b934c57868c1660acf47e7d386ac86bcb344df262c9ad76b8621,Command:[/manager],Args:[--leader-elect --health-probe-bind-address=:8081 --metrics-bind-address=127.0.0.1:8080],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{EnvVar{Name:LEASE_DURATION,Value:30,ValueFrom:nil,},EnvVar{Name:RENEW_DEADLINE,Value:20,ValueFrom:nil,},EnvVar{Name:RETRY_PERIOD,Value:5,ValueFrom:nil,},EnvVar{Name:ENABLE_WEBHOOKS,Value:false,ValueFrom:nil,},EnvVar{Name:METRICS_CERTS,Value:false,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{500 -3} {} 500m DecimalSI},memory: {{536870912 0} {} BinarySI},},Requests:ResourceList{cpu: {{10 -3} {} 10m DecimalSI},memory: {{268435456 0} {} BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:kube-api-access-gxjqd,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/healthz,Port:{0 8081 },Host:,Scheme:HTTP,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:15,TimeoutSeconds:1,PeriodSeconds:20,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/readyz,Port:{0 8081 },Host:,Scheme:HTTP,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:1,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[MKNOD],},Privileged:nil,SELinuxOptions:nil,RunAsUser:nil,RunAsNonRoot:nil,ReadOnlyRootFilesystem:nil,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod watcher-operator-controller-manager-769dc69bc-7rhg6_openstack-operators(6bea5f42-82da-4148-9996-dc05121212f7): ErrImagePull: pull QPS exceeded" logger="UnhandledError" Dec 03 11:35:54 crc kubenswrapper[5052]: E1203 11:35:54.843358 5052 kuberuntime_manager.go:1274] "Unhandled Error" err="container &Container{Name:kube-rbac-proxy,Image:quay.io/openstack-k8s-operators/kube-rbac-proxy:v0.16.0,Command:[],Args:[--secure-listen-address=0.0.0.0:8443 --upstream=http://127.0.0.1:8080/ --logtostderr=true --v=0],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{500 -3} {} 500m DecimalSI},memory: {{134217728 0} {} BinarySI},},Requests:ResourceList{cpu: {{5 -3} {} 5m DecimalSI},memory: {{67108864 0} {} BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:kube-api-access-gxjqd,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[MKNOD],},Privileged:nil,SELinuxOptions:nil,RunAsUser:nil,RunAsNonRoot:nil,ReadOnlyRootFilesystem:nil,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod watcher-operator-controller-manager-769dc69bc-7rhg6_openstack-operators(6bea5f42-82da-4148-9996-dc05121212f7): ErrImagePull: pull QPS exceeded" logger="UnhandledError" Dec 03 11:35:54 crc kubenswrapper[5052]: E1203 11:35:54.845094 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="[failed to \"StartContainer\" for \"manager\" with ErrImagePull: \"pull QPS exceeded\", failed to \"StartContainer\" for \"kube-rbac-proxy\" with ErrImagePull: \"pull QPS exceeded\"]" pod="openstack-operators/watcher-operator-controller-manager-769dc69bc-7rhg6" podUID="6bea5f42-82da-4148-9996-dc05121212f7" Dec 03 11:35:54 crc kubenswrapper[5052]: I1203 11:35:54.939491 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-operators/rabbitmq-cluster-operator-manager-668c99d594-ksw2p"] Dec 03 11:35:54 crc kubenswrapper[5052]: I1203 11:35:54.940548 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/cinder-operator-controller-manager-859b6ccc6-vt9d2" event={"ID":"5968d273-33dd-45cc-aab0-e83628bcd15c","Type":"ContainerStarted","Data":"4767fb81ab0a91c75523ffdadd4d3c055b86bd200dce767ffc908e79f3bc07b6"} Dec 03 11:35:54 crc kubenswrapper[5052]: I1203 11:35:54.941985 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/barbican-operator-controller-manager-7d9dfd778-cqfdk" event={"ID":"8dbd25a1-9de9-431e-be37-d239e3f64728","Type":"ContainerStarted","Data":"1a2e5fff53f0ca0a4391f1595e6092df6dd19f5f6bd8eafc64d716bf92687cc8"} Dec 03 11:35:54 crc kubenswrapper[5052]: I1203 11:35:54.943257 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/test-operator-controller-manager-5854674fcc-jltb6" event={"ID":"1cb3d739-e4d6-4058-9456-a3c2e1c09130","Type":"ContainerStarted","Data":"b3f6f7a69f6afa8cf29e3a2449acca972f8066a53aa83808f7f32f9d8477b2e9"} Dec 03 11:35:54 crc kubenswrapper[5052]: I1203 11:35:54.945029 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e10da887-70df-43aa-aa79-4a14231e5ec9-cert\") pod \"openstack-baremetal-operator-controller-manager-55d86b6686f2mft\" (UID: \"e10da887-70df-43aa-aa79-4a14231e5ec9\") " pod="openstack-operators/openstack-baremetal-operator-controller-manager-55d86b6686f2mft" Dec 03 11:35:54 crc kubenswrapper[5052]: E1203 11:35:54.945254 5052 secret.go:188] Couldn't get secret openstack-operators/openstack-baremetal-operator-webhook-server-cert: secret "openstack-baremetal-operator-webhook-server-cert" not found Dec 03 11:35:54 crc kubenswrapper[5052]: E1203 11:35:54.945301 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e10da887-70df-43aa-aa79-4a14231e5ec9-cert podName:e10da887-70df-43aa-aa79-4a14231e5ec9 nodeName:}" failed. No retries permitted until 2025-12-03 11:35:56.945288226 +0000 UTC m=+947.533529990 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/e10da887-70df-43aa-aa79-4a14231e5ec9-cert") pod "openstack-baremetal-operator-controller-manager-55d86b6686f2mft" (UID: "e10da887-70df-43aa-aa79-4a14231e5ec9") : secret "openstack-baremetal-operator-webhook-server-cert" not found Dec 03 11:35:54 crc kubenswrapper[5052]: I1203 11:35:54.946101 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/ironic-operator-controller-manager-6c548fd776-pqfnd" event={"ID":"7017b3f7-1eb9-4dfe-8ef3-6db27cf3a1f1","Type":"ContainerStarted","Data":"30b3e66e0e29044a9783b97379a11cf5b95d8e7a61c3e8f54beb16a7be1e4d3b"} Dec 03 11:35:54 crc kubenswrapper[5052]: E1203 11:35:54.946413 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="[failed to \"StartContainer\" for \"manager\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openstack-k8s-operators/test-operator@sha256:101b3e007d8c9f2e183262d7712f986ad51256448099069bc14f1ea5f997ab94\\\"\", failed to \"StartContainer\" for \"kube-rbac-proxy\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openstack-k8s-operators/kube-rbac-proxy:v0.16.0\\\"\"]" pod="openstack-operators/test-operator-controller-manager-5854674fcc-jltb6" podUID="1cb3d739-e4d6-4058-9456-a3c2e1c09130" Dec 03 11:35:54 crc kubenswrapper[5052]: I1203 11:35:54.949186 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/telemetry-operator-controller-manager-76cc84c6bb-zl7px" event={"ID":"3894016b-8242-4b12-8407-bcf78af00851","Type":"ContainerStarted","Data":"0f99870dc82861c793dbad4149f80170c0c9dc2378102d4f80a980b605b8485f"} Dec 03 11:35:54 crc kubenswrapper[5052]: I1203 11:35:54.954009 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/manila-operator-controller-manager-7c79b5df47-ccbqq" event={"ID":"ea1e91de-7e7d-498a-a17b-551f2bee325a","Type":"ContainerStarted","Data":"de9266c7e68b4ad9a6760a4efe0699ed57cc7eaba3f772d4b1d0f34a3589e5a0"} Dec 03 11:35:54 crc kubenswrapper[5052]: I1203 11:35:54.954246 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/horizon-operator-controller-manager-68c6d99b8f-mjdvh" event={"ID":"ce3d34f6-e545-4eb2-bcc5-999377931601","Type":"ContainerStarted","Data":"c1394d759658941f9b2cdd49562466fdf93b8e45f0f6eb82dea4b803e222fd83"} Dec 03 11:35:54 crc kubenswrapper[5052]: I1203 11:35:54.963113 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/ovn-operator-controller-manager-b6456fdb6-sggpb" event={"ID":"a605d632-d9a5-4ce0-9449-e3d57105df1b","Type":"ContainerStarted","Data":"1a9cebed54caff6e9a56f1b8ace88fd42d7b461be12f6905f9f8e7708d196205"} Dec 03 11:35:54 crc kubenswrapper[5052]: I1203 11:35:54.965222 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/neutron-operator-controller-manager-5fdfd5b6b5-q48nd" event={"ID":"53c88ca3-39f7-4a81-b305-40047bcaeb88","Type":"ContainerStarted","Data":"40332f3aff533656bb056f489cc49291efeba64208111acbf8513a4f9ecbfd55"} Dec 03 11:35:54 crc kubenswrapper[5052]: E1203 11:35:54.965439 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="[failed to \"StartContainer\" for \"manager\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openstack-k8s-operators/ovn-operator@sha256:635a4aef9d6f0b799e8ec91333dbb312160c001d05b3c63f614c124e0b67cb59\\\"\", failed to \"StartContainer\" for \"kube-rbac-proxy\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openstack-k8s-operators/kube-rbac-proxy:v0.16.0\\\"\"]" pod="openstack-operators/ovn-operator-controller-manager-b6456fdb6-sggpb" podUID="a605d632-d9a5-4ce0-9449-e3d57105df1b" Dec 03 11:35:54 crc kubenswrapper[5052]: I1203 11:35:54.971180 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/nova-operator-controller-manager-697bc559fc-gcj5d" event={"ID":"3e6bc3f6-1f1a-4b99-8835-98252369d364","Type":"ContainerStarted","Data":"7045ff1c243a7bb725866b1b3e3247264c4aacef017a1c00ffd6fbcd148e25d4"} Dec 03 11:35:54 crc kubenswrapper[5052]: E1203 11:35:54.973311 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="[failed to \"StartContainer\" for \"manager\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openstack-k8s-operators/nova-operator@sha256:779f0cee6024d0fb8f259b036fe790e62aa5a3b0431ea9bf15a6e7d02e2e5670\\\"\", failed to \"StartContainer\" for \"kube-rbac-proxy\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openstack-k8s-operators/kube-rbac-proxy:v0.16.0\\\"\"]" pod="openstack-operators/nova-operator-controller-manager-697bc559fc-gcj5d" podUID="3e6bc3f6-1f1a-4b99-8835-98252369d364" Dec 03 11:35:54 crc kubenswrapper[5052]: I1203 11:35:54.976404 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/placement-operator-controller-manager-78f8948974-j62hd" event={"ID":"37ab421b-0ad3-4d9d-bf47-8bf908c4aa57","Type":"ContainerStarted","Data":"b30889a9adcb74865c1c9d647128b6159f40569f2b1f1e64751965e5079962e2"} Dec 03 11:35:54 crc kubenswrapper[5052]: I1203 11:35:54.977937 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/octavia-operator-controller-manager-998648c74-z6cw9" event={"ID":"50c6e918-7f1f-4d6f-ba5f-940e5f485825","Type":"ContainerStarted","Data":"3365351e01a48c03cdca7119bd0ecf92f9b4476a8f54c2cb8418e4e17d437293"} Dec 03 11:35:54 crc kubenswrapper[5052]: E1203 11:35:54.978021 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="[failed to \"StartContainer\" for \"manager\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openstack-k8s-operators/placement-operator@sha256:d29650b006da97eb9178fcc58f2eb9fead8c2b414fac18f86a3c3a1507488c4f\\\"\", failed to \"StartContainer\" for \"kube-rbac-proxy\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openstack-k8s-operators/kube-rbac-proxy:v0.16.0\\\"\"]" pod="openstack-operators/placement-operator-controller-manager-78f8948974-j62hd" podUID="37ab421b-0ad3-4d9d-bf47-8bf908c4aa57" Dec 03 11:35:54 crc kubenswrapper[5052]: I1203 11:35:54.978980 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/keystone-operator-controller-manager-7765d96ddf-nlrl9" event={"ID":"95b50980-fd5a-4a19-b1cd-13df98d257fe","Type":"ContainerStarted","Data":"4e2b8f936690d204398735706497822d2ac5c4c0d87f2b484844cdaa49396523"} Dec 03 11:35:54 crc kubenswrapper[5052]: I1203 11:35:54.982071 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/designate-operator-controller-manager-78b4bc895b-874g2" event={"ID":"2e2dc077-c02f-4f98-8fa6-9f72890faeaf","Type":"ContainerStarted","Data":"e122802b4e64823d6136a13ba845b5b5991723cebe321d42e23c65d78aa51ace"} Dec 03 11:35:54 crc kubenswrapper[5052]: I1203 11:35:54.987002 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/mariadb-operator-controller-manager-56bbcc9d85-sr7zw" event={"ID":"f0e70c0d-10fe-43b7-a08c-0ce411f223a6","Type":"ContainerStarted","Data":"63b967be8542405d76ece84234f3c81710dfda97bca61633bc0f8bb8e114ed85"} Dec 03 11:35:54 crc kubenswrapper[5052]: I1203 11:35:54.987824 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/heat-operator-controller-manager-5f64f6f8bb-48fnb" event={"ID":"0cbd234c-ecb3-4e54-af86-a2b15fdd94a3","Type":"ContainerStarted","Data":"64418c91a2ae67f0561b20b32b426ab61c7e64d86df77e00d7495ca6dec54ebb"} Dec 03 11:35:54 crc kubenswrapper[5052]: E1203 11:35:54.988931 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="[failed to \"StartContainer\" for \"manager\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openstack-k8s-operators/mariadb-operator@sha256:600ca007e493d3af0fcc2ebac92e8da5efd2afe812b62d7d3d4dd0115bdf05d7\\\"\", failed to \"StartContainer\" for \"kube-rbac-proxy\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openstack-k8s-operators/kube-rbac-proxy:v0.16.0\\\"\"]" pod="openstack-operators/mariadb-operator-controller-manager-56bbcc9d85-sr7zw" podUID="f0e70c0d-10fe-43b7-a08c-0ce411f223a6" Dec 03 11:35:54 crc kubenswrapper[5052]: I1203 11:35:54.992582 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/watcher-operator-controller-manager-769dc69bc-7rhg6" event={"ID":"6bea5f42-82da-4148-9996-dc05121212f7","Type":"ContainerStarted","Data":"1fd4785ca7f9f26faf2ff9815a8f586e287d376cfb561cb9cd6dc321b0639c61"} Dec 03 11:35:54 crc kubenswrapper[5052]: E1203 11:35:54.996614 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="[failed to \"StartContainer\" for \"manager\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openstack-k8s-operators/watcher-operator@sha256:9aa8c03633e4b934c57868c1660acf47e7d386ac86bcb344df262c9ad76b8621\\\"\", failed to \"StartContainer\" for \"kube-rbac-proxy\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openstack-k8s-operators/kube-rbac-proxy:v0.16.0\\\"\"]" pod="openstack-operators/watcher-operator-controller-manager-769dc69bc-7rhg6" podUID="6bea5f42-82da-4148-9996-dc05121212f7" Dec 03 11:35:55 crc kubenswrapper[5052]: I1203 11:35:54.999969 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/swift-operator-controller-manager-5f8c65bbfc-2cgdz" event={"ID":"8fd08931-7f0b-4d5b-bfb1-f27980648561","Type":"ContainerStarted","Data":"28d85cfd295d39693c46611fbf3d445b3491e1c5f0e900a55bc02fd874da005e"} Dec 03 11:35:55 crc kubenswrapper[5052]: I1203 11:35:55.001953 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/glance-operator-controller-manager-77987cd8cd-btn2l" event={"ID":"d429f7ab-9a88-4411-a7a5-88bc8ab76285","Type":"ContainerStarted","Data":"5a62234bd4ac90688320976093ff7b70fc1912755385a234ae0df8cfe617705e"} Dec 03 11:35:55 crc kubenswrapper[5052]: I1203 11:35:55.361937 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"webhook-certs\" (UniqueName: \"kubernetes.io/secret/527bcd50-6d8d-4d4f-bc11-5382eb9adfc2-webhook-certs\") pod \"openstack-operator-controller-manager-9f56fc979-b8jv8\" (UID: \"527bcd50-6d8d-4d4f-bc11-5382eb9adfc2\") " pod="openstack-operators/openstack-operator-controller-manager-9f56fc979-b8jv8" Dec 03 11:35:55 crc kubenswrapper[5052]: I1203 11:35:55.362085 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/527bcd50-6d8d-4d4f-bc11-5382eb9adfc2-metrics-certs\") pod \"openstack-operator-controller-manager-9f56fc979-b8jv8\" (UID: \"527bcd50-6d8d-4d4f-bc11-5382eb9adfc2\") " pod="openstack-operators/openstack-operator-controller-manager-9f56fc979-b8jv8" Dec 03 11:35:55 crc kubenswrapper[5052]: E1203 11:35:55.362111 5052 secret.go:188] Couldn't get secret openstack-operators/webhook-server-cert: secret "webhook-server-cert" not found Dec 03 11:35:55 crc kubenswrapper[5052]: E1203 11:35:55.362209 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/527bcd50-6d8d-4d4f-bc11-5382eb9adfc2-webhook-certs podName:527bcd50-6d8d-4d4f-bc11-5382eb9adfc2 nodeName:}" failed. No retries permitted until 2025-12-03 11:35:57.362187942 +0000 UTC m=+947.950429706 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "webhook-certs" (UniqueName: "kubernetes.io/secret/527bcd50-6d8d-4d4f-bc11-5382eb9adfc2-webhook-certs") pod "openstack-operator-controller-manager-9f56fc979-b8jv8" (UID: "527bcd50-6d8d-4d4f-bc11-5382eb9adfc2") : secret "webhook-server-cert" not found Dec 03 11:35:55 crc kubenswrapper[5052]: E1203 11:35:55.362294 5052 secret.go:188] Couldn't get secret openstack-operators/metrics-server-cert: secret "metrics-server-cert" not found Dec 03 11:35:55 crc kubenswrapper[5052]: E1203 11:35:55.362353 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/527bcd50-6d8d-4d4f-bc11-5382eb9adfc2-metrics-certs podName:527bcd50-6d8d-4d4f-bc11-5382eb9adfc2 nodeName:}" failed. No retries permitted until 2025-12-03 11:35:57.362335796 +0000 UTC m=+947.950577750 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/527bcd50-6d8d-4d4f-bc11-5382eb9adfc2-metrics-certs") pod "openstack-operator-controller-manager-9f56fc979-b8jv8" (UID: "527bcd50-6d8d-4d4f-bc11-5382eb9adfc2") : secret "metrics-server-cert" not found Dec 03 11:35:56 crc kubenswrapper[5052]: I1203 11:35:56.017442 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/rabbitmq-cluster-operator-manager-668c99d594-ksw2p" event={"ID":"459f5555-efae-4c19-84d0-2af917b9560f","Type":"ContainerStarted","Data":"07c1349b2f26f45310447aa0bc4ba69d47ba46de5abeb65a7707c18910c76842"} Dec 03 11:35:56 crc kubenswrapper[5052]: E1203 11:35:56.020095 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="[failed to \"StartContainer\" for \"manager\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openstack-k8s-operators/ovn-operator@sha256:635a4aef9d6f0b799e8ec91333dbb312160c001d05b3c63f614c124e0b67cb59\\\"\", failed to \"StartContainer\" for \"kube-rbac-proxy\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openstack-k8s-operators/kube-rbac-proxy:v0.16.0\\\"\"]" pod="openstack-operators/ovn-operator-controller-manager-b6456fdb6-sggpb" podUID="a605d632-d9a5-4ce0-9449-e3d57105df1b" Dec 03 11:35:56 crc kubenswrapper[5052]: E1203 11:35:56.020780 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="[failed to \"StartContainer\" for \"manager\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openstack-k8s-operators/placement-operator@sha256:d29650b006da97eb9178fcc58f2eb9fead8c2b414fac18f86a3c3a1507488c4f\\\"\", failed to \"StartContainer\" for \"kube-rbac-proxy\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openstack-k8s-operators/kube-rbac-proxy:v0.16.0\\\"\"]" pod="openstack-operators/placement-operator-controller-manager-78f8948974-j62hd" podUID="37ab421b-0ad3-4d9d-bf47-8bf908c4aa57" Dec 03 11:35:56 crc kubenswrapper[5052]: E1203 11:35:56.020864 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="[failed to \"StartContainer\" for \"manager\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openstack-k8s-operators/nova-operator@sha256:779f0cee6024d0fb8f259b036fe790e62aa5a3b0431ea9bf15a6e7d02e2e5670\\\"\", failed to \"StartContainer\" for \"kube-rbac-proxy\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openstack-k8s-operators/kube-rbac-proxy:v0.16.0\\\"\"]" pod="openstack-operators/nova-operator-controller-manager-697bc559fc-gcj5d" podUID="3e6bc3f6-1f1a-4b99-8835-98252369d364" Dec 03 11:35:56 crc kubenswrapper[5052]: E1203 11:35:56.020922 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="[failed to \"StartContainer\" for \"manager\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openstack-k8s-operators/test-operator@sha256:101b3e007d8c9f2e183262d7712f986ad51256448099069bc14f1ea5f997ab94\\\"\", failed to \"StartContainer\" for \"kube-rbac-proxy\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openstack-k8s-operators/kube-rbac-proxy:v0.16.0\\\"\"]" pod="openstack-operators/test-operator-controller-manager-5854674fcc-jltb6" podUID="1cb3d739-e4d6-4058-9456-a3c2e1c09130" Dec 03 11:35:56 crc kubenswrapper[5052]: E1203 11:35:56.020982 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="[failed to \"StartContainer\" for \"manager\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openstack-k8s-operators/mariadb-operator@sha256:600ca007e493d3af0fcc2ebac92e8da5efd2afe812b62d7d3d4dd0115bdf05d7\\\"\", failed to \"StartContainer\" for \"kube-rbac-proxy\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openstack-k8s-operators/kube-rbac-proxy:v0.16.0\\\"\"]" pod="openstack-operators/mariadb-operator-controller-manager-56bbcc9d85-sr7zw" podUID="f0e70c0d-10fe-43b7-a08c-0ce411f223a6" Dec 03 11:35:56 crc kubenswrapper[5052]: E1203 11:35:56.022389 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="[failed to \"StartContainer\" for \"manager\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openstack-k8s-operators/watcher-operator@sha256:9aa8c03633e4b934c57868c1660acf47e7d386ac86bcb344df262c9ad76b8621\\\"\", failed to \"StartContainer\" for \"kube-rbac-proxy\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openstack-k8s-operators/kube-rbac-proxy:v0.16.0\\\"\"]" pod="openstack-operators/watcher-operator-controller-manager-769dc69bc-7rhg6" podUID="6bea5f42-82da-4148-9996-dc05121212f7" Dec 03 11:35:56 crc kubenswrapper[5052]: I1203 11:35:56.463898 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/81f5f8e1-015a-4b45-83e6-13e88efb8e80-cert\") pod \"infra-operator-controller-manager-57548d458d-9fkll\" (UID: \"81f5f8e1-015a-4b45-83e6-13e88efb8e80\") " pod="openstack-operators/infra-operator-controller-manager-57548d458d-9fkll" Dec 03 11:35:56 crc kubenswrapper[5052]: E1203 11:35:56.464122 5052 secret.go:188] Couldn't get secret openstack-operators/infra-operator-webhook-server-cert: secret "infra-operator-webhook-server-cert" not found Dec 03 11:35:56 crc kubenswrapper[5052]: E1203 11:35:56.464182 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/81f5f8e1-015a-4b45-83e6-13e88efb8e80-cert podName:81f5f8e1-015a-4b45-83e6-13e88efb8e80 nodeName:}" failed. No retries permitted until 2025-12-03 11:36:00.464163504 +0000 UTC m=+951.052405268 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/81f5f8e1-015a-4b45-83e6-13e88efb8e80-cert") pod "infra-operator-controller-manager-57548d458d-9fkll" (UID: "81f5f8e1-015a-4b45-83e6-13e88efb8e80") : secret "infra-operator-webhook-server-cert" not found Dec 03 11:35:56 crc kubenswrapper[5052]: I1203 11:35:56.977722 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e10da887-70df-43aa-aa79-4a14231e5ec9-cert\") pod \"openstack-baremetal-operator-controller-manager-55d86b6686f2mft\" (UID: \"e10da887-70df-43aa-aa79-4a14231e5ec9\") " pod="openstack-operators/openstack-baremetal-operator-controller-manager-55d86b6686f2mft" Dec 03 11:35:56 crc kubenswrapper[5052]: E1203 11:35:56.978765 5052 secret.go:188] Couldn't get secret openstack-operators/openstack-baremetal-operator-webhook-server-cert: secret "openstack-baremetal-operator-webhook-server-cert" not found Dec 03 11:35:56 crc kubenswrapper[5052]: E1203 11:35:56.978852 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e10da887-70df-43aa-aa79-4a14231e5ec9-cert podName:e10da887-70df-43aa-aa79-4a14231e5ec9 nodeName:}" failed. No retries permitted until 2025-12-03 11:36:00.978827768 +0000 UTC m=+951.567069532 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/e10da887-70df-43aa-aa79-4a14231e5ec9-cert") pod "openstack-baremetal-operator-controller-manager-55d86b6686f2mft" (UID: "e10da887-70df-43aa-aa79-4a14231e5ec9") : secret "openstack-baremetal-operator-webhook-server-cert" not found Dec 03 11:35:57 crc kubenswrapper[5052]: I1203 11:35:57.386737 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/527bcd50-6d8d-4d4f-bc11-5382eb9adfc2-metrics-certs\") pod \"openstack-operator-controller-manager-9f56fc979-b8jv8\" (UID: \"527bcd50-6d8d-4d4f-bc11-5382eb9adfc2\") " pod="openstack-operators/openstack-operator-controller-manager-9f56fc979-b8jv8" Dec 03 11:35:57 crc kubenswrapper[5052]: E1203 11:35:57.386909 5052 secret.go:188] Couldn't get secret openstack-operators/metrics-server-cert: secret "metrics-server-cert" not found Dec 03 11:35:57 crc kubenswrapper[5052]: E1203 11:35:57.386985 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/527bcd50-6d8d-4d4f-bc11-5382eb9adfc2-metrics-certs podName:527bcd50-6d8d-4d4f-bc11-5382eb9adfc2 nodeName:}" failed. No retries permitted until 2025-12-03 11:36:01.386967762 +0000 UTC m=+951.975209526 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/527bcd50-6d8d-4d4f-bc11-5382eb9adfc2-metrics-certs") pod "openstack-operator-controller-manager-9f56fc979-b8jv8" (UID: "527bcd50-6d8d-4d4f-bc11-5382eb9adfc2") : secret "metrics-server-cert" not found Dec 03 11:35:57 crc kubenswrapper[5052]: I1203 11:35:57.388349 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"webhook-certs\" (UniqueName: \"kubernetes.io/secret/527bcd50-6d8d-4d4f-bc11-5382eb9adfc2-webhook-certs\") pod \"openstack-operator-controller-manager-9f56fc979-b8jv8\" (UID: \"527bcd50-6d8d-4d4f-bc11-5382eb9adfc2\") " pod="openstack-operators/openstack-operator-controller-manager-9f56fc979-b8jv8" Dec 03 11:35:57 crc kubenswrapper[5052]: E1203 11:35:57.388453 5052 secret.go:188] Couldn't get secret openstack-operators/webhook-server-cert: secret "webhook-server-cert" not found Dec 03 11:35:57 crc kubenswrapper[5052]: E1203 11:35:57.388484 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/527bcd50-6d8d-4d4f-bc11-5382eb9adfc2-webhook-certs podName:527bcd50-6d8d-4d4f-bc11-5382eb9adfc2 nodeName:}" failed. No retries permitted until 2025-12-03 11:36:01.388473946 +0000 UTC m=+951.976715710 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "webhook-certs" (UniqueName: "kubernetes.io/secret/527bcd50-6d8d-4d4f-bc11-5382eb9adfc2-webhook-certs") pod "openstack-operator-controller-manager-9f56fc979-b8jv8" (UID: "527bcd50-6d8d-4d4f-bc11-5382eb9adfc2") : secret "webhook-server-cert" not found Dec 03 11:35:59 crc kubenswrapper[5052]: I1203 11:35:59.379193 5052 patch_prober.go:28] interesting pod/machine-config-daemon-74bj6 container/machine-config-daemon namespace/openshift-machine-config-operator: Liveness probe status=failure output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" start-of-body= Dec 03 11:35:59 crc kubenswrapper[5052]: I1203 11:35:59.379263 5052 prober.go:107] "Probe failed" probeType="Liveness" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" containerName="machine-config-daemon" probeResult="failure" output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" Dec 03 11:36:00 crc kubenswrapper[5052]: I1203 11:36:00.544373 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/81f5f8e1-015a-4b45-83e6-13e88efb8e80-cert\") pod \"infra-operator-controller-manager-57548d458d-9fkll\" (UID: \"81f5f8e1-015a-4b45-83e6-13e88efb8e80\") " pod="openstack-operators/infra-operator-controller-manager-57548d458d-9fkll" Dec 03 11:36:00 crc kubenswrapper[5052]: E1203 11:36:00.545380 5052 secret.go:188] Couldn't get secret openstack-operators/infra-operator-webhook-server-cert: secret "infra-operator-webhook-server-cert" not found Dec 03 11:36:00 crc kubenswrapper[5052]: E1203 11:36:00.545446 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/81f5f8e1-015a-4b45-83e6-13e88efb8e80-cert podName:81f5f8e1-015a-4b45-83e6-13e88efb8e80 nodeName:}" failed. No retries permitted until 2025-12-03 11:36:08.545425628 +0000 UTC m=+959.133667392 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/81f5f8e1-015a-4b45-83e6-13e88efb8e80-cert") pod "infra-operator-controller-manager-57548d458d-9fkll" (UID: "81f5f8e1-015a-4b45-83e6-13e88efb8e80") : secret "infra-operator-webhook-server-cert" not found Dec 03 11:36:01 crc kubenswrapper[5052]: I1203 11:36:01.051673 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e10da887-70df-43aa-aa79-4a14231e5ec9-cert\") pod \"openstack-baremetal-operator-controller-manager-55d86b6686f2mft\" (UID: \"e10da887-70df-43aa-aa79-4a14231e5ec9\") " pod="openstack-operators/openstack-baremetal-operator-controller-manager-55d86b6686f2mft" Dec 03 11:36:01 crc kubenswrapper[5052]: E1203 11:36:01.051895 5052 secret.go:188] Couldn't get secret openstack-operators/openstack-baremetal-operator-webhook-server-cert: secret "openstack-baremetal-operator-webhook-server-cert" not found Dec 03 11:36:01 crc kubenswrapper[5052]: E1203 11:36:01.051946 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e10da887-70df-43aa-aa79-4a14231e5ec9-cert podName:e10da887-70df-43aa-aa79-4a14231e5ec9 nodeName:}" failed. No retries permitted until 2025-12-03 11:36:09.051933907 +0000 UTC m=+959.640175671 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/e10da887-70df-43aa-aa79-4a14231e5ec9-cert") pod "openstack-baremetal-operator-controller-manager-55d86b6686f2mft" (UID: "e10da887-70df-43aa-aa79-4a14231e5ec9") : secret "openstack-baremetal-operator-webhook-server-cert" not found Dec 03 11:36:01 crc kubenswrapper[5052]: I1203 11:36:01.460995 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"webhook-certs\" (UniqueName: \"kubernetes.io/secret/527bcd50-6d8d-4d4f-bc11-5382eb9adfc2-webhook-certs\") pod \"openstack-operator-controller-manager-9f56fc979-b8jv8\" (UID: \"527bcd50-6d8d-4d4f-bc11-5382eb9adfc2\") " pod="openstack-operators/openstack-operator-controller-manager-9f56fc979-b8jv8" Dec 03 11:36:01 crc kubenswrapper[5052]: I1203 11:36:01.461119 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/527bcd50-6d8d-4d4f-bc11-5382eb9adfc2-metrics-certs\") pod \"openstack-operator-controller-manager-9f56fc979-b8jv8\" (UID: \"527bcd50-6d8d-4d4f-bc11-5382eb9adfc2\") " pod="openstack-operators/openstack-operator-controller-manager-9f56fc979-b8jv8" Dec 03 11:36:01 crc kubenswrapper[5052]: E1203 11:36:01.461126 5052 secret.go:188] Couldn't get secret openstack-operators/webhook-server-cert: secret "webhook-server-cert" not found Dec 03 11:36:01 crc kubenswrapper[5052]: E1203 11:36:01.461219 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/527bcd50-6d8d-4d4f-bc11-5382eb9adfc2-webhook-certs podName:527bcd50-6d8d-4d4f-bc11-5382eb9adfc2 nodeName:}" failed. No retries permitted until 2025-12-03 11:36:09.461199922 +0000 UTC m=+960.049441686 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "webhook-certs" (UniqueName: "kubernetes.io/secret/527bcd50-6d8d-4d4f-bc11-5382eb9adfc2-webhook-certs") pod "openstack-operator-controller-manager-9f56fc979-b8jv8" (UID: "527bcd50-6d8d-4d4f-bc11-5382eb9adfc2") : secret "webhook-server-cert" not found Dec 03 11:36:01 crc kubenswrapper[5052]: E1203 11:36:01.461379 5052 secret.go:188] Couldn't get secret openstack-operators/metrics-server-cert: secret "metrics-server-cert" not found Dec 03 11:36:01 crc kubenswrapper[5052]: E1203 11:36:01.461531 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/527bcd50-6d8d-4d4f-bc11-5382eb9adfc2-metrics-certs podName:527bcd50-6d8d-4d4f-bc11-5382eb9adfc2 nodeName:}" failed. No retries permitted until 2025-12-03 11:36:09.46148796 +0000 UTC m=+960.049729724 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/527bcd50-6d8d-4d4f-bc11-5382eb9adfc2-metrics-certs") pod "openstack-operator-controller-manager-9f56fc979-b8jv8" (UID: "527bcd50-6d8d-4d4f-bc11-5382eb9adfc2") : secret "metrics-server-cert" not found Dec 03 11:36:07 crc kubenswrapper[5052]: E1203 11:36:07.984906 5052 log.go:32] "PullImage from image service failed" err="rpc error: code = Canceled desc = copying config: context canceled" image="quay.io/openstack-k8s-operators/neutron-operator@sha256:0b3fb69f35c151895d3dffd514974a9f9fe1c77c3bca69b78b81efb183cf4557" Dec 03 11:36:07 crc kubenswrapper[5052]: E1203 11:36:07.985696 5052 kuberuntime_manager.go:1274] "Unhandled Error" err="container &Container{Name:manager,Image:quay.io/openstack-k8s-operators/neutron-operator@sha256:0b3fb69f35c151895d3dffd514974a9f9fe1c77c3bca69b78b81efb183cf4557,Command:[/manager],Args:[--leader-elect --health-probe-bind-address=:8081 --metrics-bind-address=127.0.0.1:8080],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{EnvVar{Name:LEASE_DURATION,Value:30,ValueFrom:nil,},EnvVar{Name:RENEW_DEADLINE,Value:20,ValueFrom:nil,},EnvVar{Name:RETRY_PERIOD,Value:5,ValueFrom:nil,},EnvVar{Name:ENABLE_WEBHOOKS,Value:false,ValueFrom:nil,},EnvVar{Name:METRICS_CERTS,Value:false,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{500 -3} {} 500m DecimalSI},memory: {{536870912 0} {} BinarySI},},Requests:ResourceList{cpu: {{10 -3} {} 10m DecimalSI},memory: {{268435456 0} {} BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:kube-api-access-j2rjq,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/healthz,Port:{0 8081 },Host:,Scheme:HTTP,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:15,TimeoutSeconds:1,PeriodSeconds:20,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/readyz,Port:{0 8081 },Host:,Scheme:HTTP,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:1,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[MKNOD],},Privileged:nil,SELinuxOptions:nil,RunAsUser:nil,RunAsNonRoot:nil,ReadOnlyRootFilesystem:nil,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod neutron-operator-controller-manager-5fdfd5b6b5-q48nd_openstack-operators(53c88ca3-39f7-4a81-b305-40047bcaeb88): ErrImagePull: rpc error: code = Canceled desc = copying config: context canceled" logger="UnhandledError" Dec 03 11:36:08 crc kubenswrapper[5052]: I1203 11:36:08.561721 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/81f5f8e1-015a-4b45-83e6-13e88efb8e80-cert\") pod \"infra-operator-controller-manager-57548d458d-9fkll\" (UID: \"81f5f8e1-015a-4b45-83e6-13e88efb8e80\") " pod="openstack-operators/infra-operator-controller-manager-57548d458d-9fkll" Dec 03 11:36:08 crc kubenswrapper[5052]: E1203 11:36:08.561931 5052 secret.go:188] Couldn't get secret openstack-operators/infra-operator-webhook-server-cert: secret "infra-operator-webhook-server-cert" not found Dec 03 11:36:08 crc kubenswrapper[5052]: E1203 11:36:08.562444 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/81f5f8e1-015a-4b45-83e6-13e88efb8e80-cert podName:81f5f8e1-015a-4b45-83e6-13e88efb8e80 nodeName:}" failed. No retries permitted until 2025-12-03 11:36:24.562413077 +0000 UTC m=+975.150654841 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/81f5f8e1-015a-4b45-83e6-13e88efb8e80-cert") pod "infra-operator-controller-manager-57548d458d-9fkll" (UID: "81f5f8e1-015a-4b45-83e6-13e88efb8e80") : secret "infra-operator-webhook-server-cert" not found Dec 03 11:36:09 crc kubenswrapper[5052]: I1203 11:36:09.075108 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e10da887-70df-43aa-aa79-4a14231e5ec9-cert\") pod \"openstack-baremetal-operator-controller-manager-55d86b6686f2mft\" (UID: \"e10da887-70df-43aa-aa79-4a14231e5ec9\") " pod="openstack-operators/openstack-baremetal-operator-controller-manager-55d86b6686f2mft" Dec 03 11:36:09 crc kubenswrapper[5052]: I1203 11:36:09.080786 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e10da887-70df-43aa-aa79-4a14231e5ec9-cert\") pod \"openstack-baremetal-operator-controller-manager-55d86b6686f2mft\" (UID: \"e10da887-70df-43aa-aa79-4a14231e5ec9\") " pod="openstack-operators/openstack-baremetal-operator-controller-manager-55d86b6686f2mft" Dec 03 11:36:09 crc kubenswrapper[5052]: I1203 11:36:09.103844 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-operators/openstack-baremetal-operator-controller-manager-55d86b6686f2mft" Dec 03 11:36:09 crc kubenswrapper[5052]: I1203 11:36:09.481580 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"webhook-certs\" (UniqueName: \"kubernetes.io/secret/527bcd50-6d8d-4d4f-bc11-5382eb9adfc2-webhook-certs\") pod \"openstack-operator-controller-manager-9f56fc979-b8jv8\" (UID: \"527bcd50-6d8d-4d4f-bc11-5382eb9adfc2\") " pod="openstack-operators/openstack-operator-controller-manager-9f56fc979-b8jv8" Dec 03 11:36:09 crc kubenswrapper[5052]: I1203 11:36:09.481677 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/527bcd50-6d8d-4d4f-bc11-5382eb9adfc2-metrics-certs\") pod \"openstack-operator-controller-manager-9f56fc979-b8jv8\" (UID: \"527bcd50-6d8d-4d4f-bc11-5382eb9adfc2\") " pod="openstack-operators/openstack-operator-controller-manager-9f56fc979-b8jv8" Dec 03 11:36:09 crc kubenswrapper[5052]: I1203 11:36:09.487223 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"webhook-certs\" (UniqueName: \"kubernetes.io/secret/527bcd50-6d8d-4d4f-bc11-5382eb9adfc2-webhook-certs\") pod \"openstack-operator-controller-manager-9f56fc979-b8jv8\" (UID: \"527bcd50-6d8d-4d4f-bc11-5382eb9adfc2\") " pod="openstack-operators/openstack-operator-controller-manager-9f56fc979-b8jv8" Dec 03 11:36:09 crc kubenswrapper[5052]: I1203 11:36:09.487462 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/527bcd50-6d8d-4d4f-bc11-5382eb9adfc2-metrics-certs\") pod \"openstack-operator-controller-manager-9f56fc979-b8jv8\" (UID: \"527bcd50-6d8d-4d4f-bc11-5382eb9adfc2\") " pod="openstack-operators/openstack-operator-controller-manager-9f56fc979-b8jv8" Dec 03 11:36:09 crc kubenswrapper[5052]: I1203 11:36:09.552172 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-operators/openstack-operator-controller-manager-9f56fc979-b8jv8" Dec 03 11:36:14 crc kubenswrapper[5052]: E1203 11:36:14.047941 5052 log.go:32] "PullImage from image service failed" err="rpc error: code = Canceled desc = copying config: context canceled" image="quay.io/openstack-k8s-operators/telemetry-operator@sha256:7d66757c0af67104f0389e851a7cc0daa44443ad202d157417bd86bbb57cc385" Dec 03 11:36:14 crc kubenswrapper[5052]: E1203 11:36:14.048790 5052 kuberuntime_manager.go:1274] "Unhandled Error" err="container &Container{Name:manager,Image:quay.io/openstack-k8s-operators/telemetry-operator@sha256:7d66757c0af67104f0389e851a7cc0daa44443ad202d157417bd86bbb57cc385,Command:[/manager],Args:[--leader-elect --health-probe-bind-address=:8081 --metrics-bind-address=127.0.0.1:8080],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{EnvVar{Name:LEASE_DURATION,Value:30,ValueFrom:nil,},EnvVar{Name:RENEW_DEADLINE,Value:20,ValueFrom:nil,},EnvVar{Name:RETRY_PERIOD,Value:5,ValueFrom:nil,},EnvVar{Name:ENABLE_WEBHOOKS,Value:false,ValueFrom:nil,},EnvVar{Name:METRICS_CERTS,Value:false,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{500 -3} {} 500m DecimalSI},memory: {{536870912 0} {} BinarySI},},Requests:ResourceList{cpu: {{10 -3} {} 10m DecimalSI},memory: {{268435456 0} {} BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:kube-api-access-pmdm6,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/healthz,Port:{0 8081 },Host:,Scheme:HTTP,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:15,TimeoutSeconds:1,PeriodSeconds:20,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/readyz,Port:{0 8081 },Host:,Scheme:HTTP,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:1,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[MKNOD],},Privileged:nil,SELinuxOptions:nil,RunAsUser:nil,RunAsNonRoot:nil,ReadOnlyRootFilesystem:nil,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod telemetry-operator-controller-manager-76cc84c6bb-zl7px_openstack-operators(3894016b-8242-4b12-8407-bcf78af00851): ErrImagePull: rpc error: code = Canceled desc = copying config: context canceled" logger="UnhandledError" Dec 03 11:36:17 crc kubenswrapper[5052]: E1203 11:36:17.532491 5052 log.go:32] "PullImage from image service failed" err="rpc error: code = Canceled desc = copying config: context canceled" image="quay.io/openstack-k8s-operators/keystone-operator@sha256:72ad6517987f674af0d0ae092cbb874aeae909c8b8b60188099c311762ebc8f7" Dec 03 11:36:17 crc kubenswrapper[5052]: E1203 11:36:17.533154 5052 kuberuntime_manager.go:1274] "Unhandled Error" err="container &Container{Name:manager,Image:quay.io/openstack-k8s-operators/keystone-operator@sha256:72ad6517987f674af0d0ae092cbb874aeae909c8b8b60188099c311762ebc8f7,Command:[/manager],Args:[--leader-elect --health-probe-bind-address=:8081 --metrics-bind-address=127.0.0.1:8080],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{EnvVar{Name:LEASE_DURATION,Value:30,ValueFrom:nil,},EnvVar{Name:RENEW_DEADLINE,Value:20,ValueFrom:nil,},EnvVar{Name:RETRY_PERIOD,Value:5,ValueFrom:nil,},EnvVar{Name:ENABLE_WEBHOOKS,Value:false,ValueFrom:nil,},EnvVar{Name:METRICS_CERTS,Value:false,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{500 -3} {} 500m DecimalSI},memory: {{536870912 0} {} BinarySI},},Requests:ResourceList{cpu: {{10 -3} {} 10m DecimalSI},memory: {{268435456 0} {} BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:kube-api-access-sw44h,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/healthz,Port:{0 8081 },Host:,Scheme:HTTP,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:15,TimeoutSeconds:1,PeriodSeconds:20,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/readyz,Port:{0 8081 },Host:,Scheme:HTTP,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:1,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[MKNOD],},Privileged:nil,SELinuxOptions:nil,RunAsUser:nil,RunAsNonRoot:nil,ReadOnlyRootFilesystem:nil,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod keystone-operator-controller-manager-7765d96ddf-nlrl9_openstack-operators(95b50980-fd5a-4a19-b1cd-13df98d257fe): ErrImagePull: rpc error: code = Canceled desc = copying config: context canceled" logger="UnhandledError" Dec 03 11:36:17 crc kubenswrapper[5052]: E1203 11:36:17.905227 5052 log.go:32] "PullImage from image service failed" err="rpc error: code = Canceled desc = copying config: context canceled" image="quay.io/openstack-k8s-operators/rabbitmq-cluster-operator@sha256:893e66303c1b0bc1d00a299a3f0380bad55c8dc813c8a1c6a4aab379f5aa12a2" Dec 03 11:36:17 crc kubenswrapper[5052]: E1203 11:36:17.906966 5052 kuberuntime_manager.go:1274] "Unhandled Error" err="container &Container{Name:operator,Image:quay.io/openstack-k8s-operators/rabbitmq-cluster-operator@sha256:893e66303c1b0bc1d00a299a3f0380bad55c8dc813c8a1c6a4aab379f5aa12a2,Command:[/manager],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:metrics,HostPort:0,ContainerPort:9782,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:OPERATOR_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:LEASE_DURATION,Value:30,ValueFrom:nil,},EnvVar{Name:RENEW_DEADLINE,Value:20,ValueFrom:nil,},EnvVar{Name:RETRY_PERIOD,Value:5,ValueFrom:nil,},EnvVar{Name:ENABLE_WEBHOOKS,Value:false,ValueFrom:nil,},EnvVar{Name:METRICS_CERTS,Value:false,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {} 200m DecimalSI},memory: {{524288000 0} {} 500Mi BinarySI},},Requests:ResourceList{cpu: {{5 -3} {} 5m DecimalSI},memory: {{67108864 0} {} BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:kube-api-access-7dk26,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:nil,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod rabbitmq-cluster-operator-manager-668c99d594-ksw2p_openstack-operators(459f5555-efae-4c19-84d0-2af917b9560f): ErrImagePull: rpc error: code = Canceled desc = copying config: context canceled" logger="UnhandledError" Dec 03 11:36:17 crc kubenswrapper[5052]: E1203 11:36:17.908177 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"operator\" with ErrImagePull: \"rpc error: code = Canceled desc = copying config: context canceled\"" pod="openstack-operators/rabbitmq-cluster-operator-manager-668c99d594-ksw2p" podUID="459f5555-efae-4c19-84d0-2af917b9560f" Dec 03 11:36:18 crc kubenswrapper[5052]: E1203 11:36:18.207933 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"operator\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openstack-k8s-operators/rabbitmq-cluster-operator@sha256:893e66303c1b0bc1d00a299a3f0380bad55c8dc813c8a1c6a4aab379f5aa12a2\\\"\"" pod="openstack-operators/rabbitmq-cluster-operator-manager-668c99d594-ksw2p" podUID="459f5555-efae-4c19-84d0-2af917b9560f" Dec 03 11:36:20 crc kubenswrapper[5052]: I1203 11:36:20.109881 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-operators/openstack-operator-controller-manager-9f56fc979-b8jv8"] Dec 03 11:36:20 crc kubenswrapper[5052]: I1203 11:36:20.170034 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-operators/openstack-baremetal-operator-controller-manager-55d86b6686f2mft"] Dec 03 11:36:20 crc kubenswrapper[5052]: I1203 11:36:20.218441 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/horizon-operator-controller-manager-68c6d99b8f-mjdvh" event={"ID":"ce3d34f6-e545-4eb2-bcc5-999377931601","Type":"ContainerStarted","Data":"202df0b6bac6f793c5ac016ef55a754be22068b68cff50a0616b5de880db9a3c"} Dec 03 11:36:20 crc kubenswrapper[5052]: W1203 11:36:20.301261 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod527bcd50_6d8d_4d4f_bc11_5382eb9adfc2.slice/crio-539dfe13c0fa9889c9112a3f1df72e1767185682a73bbc446b193911a1924168 WatchSource:0}: Error finding container 539dfe13c0fa9889c9112a3f1df72e1767185682a73bbc446b193911a1924168: Status 404 returned error can't find the container with id 539dfe13c0fa9889c9112a3f1df72e1767185682a73bbc446b193911a1924168 Dec 03 11:36:20 crc kubenswrapper[5052]: W1203 11:36:20.302926 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode10da887_70df_43aa_aa79_4a14231e5ec9.slice/crio-7e31de0787f492c6adfc9a9e6a17a8682798e05059001ae1d2b2235f6c963d16 WatchSource:0}: Error finding container 7e31de0787f492c6adfc9a9e6a17a8682798e05059001ae1d2b2235f6c963d16: Status 404 returned error can't find the container with id 7e31de0787f492c6adfc9a9e6a17a8682798e05059001ae1d2b2235f6c963d16 Dec 03 11:36:21 crc kubenswrapper[5052]: I1203 11:36:21.248029 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/manila-operator-controller-manager-7c79b5df47-ccbqq" event={"ID":"ea1e91de-7e7d-498a-a17b-551f2bee325a","Type":"ContainerStarted","Data":"0add256c4e87d7b91cc63837e4958d34bfbf4128a08af08316deec77be2a75eb"} Dec 03 11:36:21 crc kubenswrapper[5052]: I1203 11:36:21.260234 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/cinder-operator-controller-manager-859b6ccc6-vt9d2" event={"ID":"5968d273-33dd-45cc-aab0-e83628bcd15c","Type":"ContainerStarted","Data":"e020c88c64940fc2399454d010fff5531ddc1320fccf47d9835db05c4424f1b2"} Dec 03 11:36:21 crc kubenswrapper[5052]: I1203 11:36:21.262814 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/heat-operator-controller-manager-5f64f6f8bb-48fnb" event={"ID":"0cbd234c-ecb3-4e54-af86-a2b15fdd94a3","Type":"ContainerStarted","Data":"05da1e86576897a06ee62b48ffa3acc05751be603b91af013f2fc3b98094a039"} Dec 03 11:36:21 crc kubenswrapper[5052]: I1203 11:36:21.265798 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/openstack-operator-controller-manager-9f56fc979-b8jv8" event={"ID":"527bcd50-6d8d-4d4f-bc11-5382eb9adfc2","Type":"ContainerStarted","Data":"539dfe13c0fa9889c9112a3f1df72e1767185682a73bbc446b193911a1924168"} Dec 03 11:36:21 crc kubenswrapper[5052]: I1203 11:36:21.267317 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/watcher-operator-controller-manager-769dc69bc-7rhg6" event={"ID":"6bea5f42-82da-4148-9996-dc05121212f7","Type":"ContainerStarted","Data":"af5bcb24f3ea207a68cadd9479e8bc16e424b9d9fac437c1e5ea1ef07af38d6d"} Dec 03 11:36:21 crc kubenswrapper[5052]: I1203 11:36:21.268755 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/swift-operator-controller-manager-5f8c65bbfc-2cgdz" event={"ID":"8fd08931-7f0b-4d5b-bfb1-f27980648561","Type":"ContainerStarted","Data":"a075d6f4a44381ae20a16b8fa5bb2d8c0561bce325ee8a9d99cee59abaad79c8"} Dec 03 11:36:21 crc kubenswrapper[5052]: I1203 11:36:21.271126 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/openstack-baremetal-operator-controller-manager-55d86b6686f2mft" event={"ID":"e10da887-70df-43aa-aa79-4a14231e5ec9","Type":"ContainerStarted","Data":"7e31de0787f492c6adfc9a9e6a17a8682798e05059001ae1d2b2235f6c963d16"} Dec 03 11:36:21 crc kubenswrapper[5052]: I1203 11:36:21.272423 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/barbican-operator-controller-manager-7d9dfd778-cqfdk" event={"ID":"8dbd25a1-9de9-431e-be37-d239e3f64728","Type":"ContainerStarted","Data":"5025f6eb197bd7384ff4dad6b52992f6489ac18de5c05c2535a9026ee37b6f24"} Dec 03 11:36:21 crc kubenswrapper[5052]: I1203 11:36:21.274938 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/ovn-operator-controller-manager-b6456fdb6-sggpb" event={"ID":"a605d632-d9a5-4ce0-9449-e3d57105df1b","Type":"ContainerStarted","Data":"2944fe91598c6c5f038d1de9c8551c8352a4c6b145016c6151bfc37b330edddd"} Dec 03 11:36:21 crc kubenswrapper[5052]: I1203 11:36:21.277827 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/designate-operator-controller-manager-78b4bc895b-874g2" event={"ID":"2e2dc077-c02f-4f98-8fa6-9f72890faeaf","Type":"ContainerStarted","Data":"bc9d68e8694514b249866a594a571232d780cb8f8315ced546035025828c5930"} Dec 03 11:36:21 crc kubenswrapper[5052]: I1203 11:36:21.283421 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/octavia-operator-controller-manager-998648c74-z6cw9" event={"ID":"50c6e918-7f1f-4d6f-ba5f-940e5f485825","Type":"ContainerStarted","Data":"829ed8e10d7aad4792ee147f6891ccdb7a056f605c04a4c0349d94bfa248bff9"} Dec 03 11:36:22 crc kubenswrapper[5052]: I1203 11:36:22.290802 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/test-operator-controller-manager-5854674fcc-jltb6" event={"ID":"1cb3d739-e4d6-4058-9456-a3c2e1c09130","Type":"ContainerStarted","Data":"7f760d28d8a9a27373a0d324cb077f28fc6b0310ae54ade63b76d7f19979ccad"} Dec 03 11:36:22 crc kubenswrapper[5052]: I1203 11:36:22.292500 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/mariadb-operator-controller-manager-56bbcc9d85-sr7zw" event={"ID":"f0e70c0d-10fe-43b7-a08c-0ce411f223a6","Type":"ContainerStarted","Data":"c16b4916d85027f28775414e1e9e061796b804441bf4aded0b2a1af11134a874"} Dec 03 11:36:22 crc kubenswrapper[5052]: I1203 11:36:22.294166 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/ironic-operator-controller-manager-6c548fd776-pqfnd" event={"ID":"7017b3f7-1eb9-4dfe-8ef3-6db27cf3a1f1","Type":"ContainerStarted","Data":"acbb799df7a125d48309f76c7ac211fd06dfd6580980ca517793b3d574553f2e"} Dec 03 11:36:23 crc kubenswrapper[5052]: I1203 11:36:23.301999 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/placement-operator-controller-manager-78f8948974-j62hd" event={"ID":"37ab421b-0ad3-4d9d-bf47-8bf908c4aa57","Type":"ContainerStarted","Data":"e3943739460d5bb48bf27023494d0f23c5ff3cd0f58f04e59d49d0dbc05a7830"} Dec 03 11:36:23 crc kubenswrapper[5052]: I1203 11:36:23.303341 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/openstack-operator-controller-manager-9f56fc979-b8jv8" event={"ID":"527bcd50-6d8d-4d4f-bc11-5382eb9adfc2","Type":"ContainerStarted","Data":"d1bb936db1c98c58f4e109008a5dc7d293c9a04b310fd4b2ab5bb2e70449c5aa"} Dec 03 11:36:23 crc kubenswrapper[5052]: I1203 11:36:23.303477 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-operators/openstack-operator-controller-manager-9f56fc979-b8jv8" Dec 03 11:36:23 crc kubenswrapper[5052]: I1203 11:36:23.306233 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/glance-operator-controller-manager-77987cd8cd-btn2l" event={"ID":"d429f7ab-9a88-4411-a7a5-88bc8ab76285","Type":"ContainerStarted","Data":"27a2c0f47320ac8e0e4f631d691ed2469ebf4f27f0dad96adcf0fcb2e224d156"} Dec 03 11:36:23 crc kubenswrapper[5052]: I1203 11:36:23.345224 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-operators/openstack-operator-controller-manager-9f56fc979-b8jv8" podStartSLOduration=30.34519943 podStartE2EDuration="30.34519943s" podCreationTimestamp="2025-12-03 11:35:53 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:36:23.339269319 +0000 UTC m=+973.927511113" watchObservedRunningTime="2025-12-03 11:36:23.34519943 +0000 UTC m=+973.933441204" Dec 03 11:36:23 crc kubenswrapper[5052]: E1203 11:36:23.903010 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with ErrImagePull: \"rpc error: code = Canceled desc = copying config: context canceled\"" pod="openstack-operators/telemetry-operator-controller-manager-76cc84c6bb-zl7px" podUID="3894016b-8242-4b12-8407-bcf78af00851" Dec 03 11:36:24 crc kubenswrapper[5052]: E1203 11:36:24.026654 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with ErrImagePull: \"rpc error: code = Canceled desc = copying config: context canceled\"" pod="openstack-operators/neutron-operator-controller-manager-5fdfd5b6b5-q48nd" podUID="53c88ca3-39f7-4a81-b305-40047bcaeb88" Dec 03 11:36:24 crc kubenswrapper[5052]: E1203 11:36:24.264482 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with ErrImagePull: \"rpc error: code = Canceled desc = copying config: context canceled\"" pod="openstack-operators/keystone-operator-controller-manager-7765d96ddf-nlrl9" podUID="95b50980-fd5a-4a19-b1cd-13df98d257fe" Dec 03 11:36:24 crc kubenswrapper[5052]: I1203 11:36:24.313973 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/neutron-operator-controller-manager-5fdfd5b6b5-q48nd" event={"ID":"53c88ca3-39f7-4a81-b305-40047bcaeb88","Type":"ContainerStarted","Data":"5216edc450cbbbdc9be723de0d23e9c4c7f79734c44ce55e5ee6afa4b2ca1bd2"} Dec 03 11:36:24 crc kubenswrapper[5052]: I1203 11:36:24.319363 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/keystone-operator-controller-manager-7765d96ddf-nlrl9" event={"ID":"95b50980-fd5a-4a19-b1cd-13df98d257fe","Type":"ContainerStarted","Data":"97782ec29d2e9e3193fca38c78d6d80a51ddf23167b6089a154cabfc32ab7307"} Dec 03 11:36:24 crc kubenswrapper[5052]: E1203 11:36:24.320138 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openstack-k8s-operators/keystone-operator@sha256:72ad6517987f674af0d0ae092cbb874aeae909c8b8b60188099c311762ebc8f7\\\"\"" pod="openstack-operators/keystone-operator-controller-manager-7765d96ddf-nlrl9" podUID="95b50980-fd5a-4a19-b1cd-13df98d257fe" Dec 03 11:36:24 crc kubenswrapper[5052]: I1203 11:36:24.320601 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/telemetry-operator-controller-manager-76cc84c6bb-zl7px" event={"ID":"3894016b-8242-4b12-8407-bcf78af00851","Type":"ContainerStarted","Data":"97920a5e6bb5c96fa08c020aae2055f6d6c6ddf72d91e1b35081548101fd59bc"} Dec 03 11:36:24 crc kubenswrapper[5052]: I1203 11:36:24.329115 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/nova-operator-controller-manager-697bc559fc-gcj5d" event={"ID":"3e6bc3f6-1f1a-4b99-8835-98252369d364","Type":"ContainerStarted","Data":"21bac89f97ac17aa04e4ad5d34fe06b056702b1896eabe8f69dc6ad5ebbc42bf"} Dec 03 11:36:24 crc kubenswrapper[5052]: I1203 11:36:24.329701 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-operators/nova-operator-controller-manager-697bc559fc-gcj5d" Dec 03 11:36:24 crc kubenswrapper[5052]: I1203 11:36:24.331222 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/openstack-baremetal-operator-controller-manager-55d86b6686f2mft" event={"ID":"e10da887-70df-43aa-aa79-4a14231e5ec9","Type":"ContainerStarted","Data":"0f961febcff89cbc4c5cd57e2535ba7fe24e5f74a9bf78788782f9c8e2c423d5"} Dec 03 11:36:24 crc kubenswrapper[5052]: I1203 11:36:24.408116 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-operators/nova-operator-controller-manager-697bc559fc-gcj5d" podStartSLOduration=7.028521952 podStartE2EDuration="32.408099369s" podCreationTimestamp="2025-12-03 11:35:52 +0000 UTC" firstStartedPulling="2025-12-03 11:35:54.568219997 +0000 UTC m=+945.156461761" lastFinishedPulling="2025-12-03 11:36:19.947797414 +0000 UTC m=+970.536039178" observedRunningTime="2025-12-03 11:36:24.404086313 +0000 UTC m=+974.992328087" watchObservedRunningTime="2025-12-03 11:36:24.408099369 +0000 UTC m=+974.996341133" Dec 03 11:36:24 crc kubenswrapper[5052]: I1203 11:36:24.631027 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/81f5f8e1-015a-4b45-83e6-13e88efb8e80-cert\") pod \"infra-operator-controller-manager-57548d458d-9fkll\" (UID: \"81f5f8e1-015a-4b45-83e6-13e88efb8e80\") " pod="openstack-operators/infra-operator-controller-manager-57548d458d-9fkll" Dec 03 11:36:24 crc kubenswrapper[5052]: I1203 11:36:24.645376 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/81f5f8e1-015a-4b45-83e6-13e88efb8e80-cert\") pod \"infra-operator-controller-manager-57548d458d-9fkll\" (UID: \"81f5f8e1-015a-4b45-83e6-13e88efb8e80\") " pod="openstack-operators/infra-operator-controller-manager-57548d458d-9fkll" Dec 03 11:36:24 crc kubenswrapper[5052]: I1203 11:36:24.854891 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-operators"/"infra-operator-controller-manager-dockercfg-xgv5g" Dec 03 11:36:24 crc kubenswrapper[5052]: I1203 11:36:24.864360 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-operators/infra-operator-controller-manager-57548d458d-9fkll" Dec 03 11:36:25 crc kubenswrapper[5052]: I1203 11:36:25.220121 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-operators/infra-operator-controller-manager-57548d458d-9fkll"] Dec 03 11:36:25 crc kubenswrapper[5052]: I1203 11:36:25.338425 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/swift-operator-controller-manager-5f8c65bbfc-2cgdz" event={"ID":"8fd08931-7f0b-4d5b-bfb1-f27980648561","Type":"ContainerStarted","Data":"0c828acdac4251182a50609e1c01b6a1bb38eb6449736b7bbd7aef20f9dc2f09"} Dec 03 11:36:25 crc kubenswrapper[5052]: I1203 11:36:25.341407 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-operators/swift-operator-controller-manager-5f8c65bbfc-2cgdz" Dec 03 11:36:25 crc kubenswrapper[5052]: I1203 11:36:25.342853 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/placement-operator-controller-manager-78f8948974-j62hd" event={"ID":"37ab421b-0ad3-4d9d-bf47-8bf908c4aa57","Type":"ContainerStarted","Data":"2dcc207b577f508a2a51aeed06eda37f7a37cdaa339bc4bd5b4f2712d43ec227"} Dec 03 11:36:25 crc kubenswrapper[5052]: I1203 11:36:25.343408 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-operators/placement-operator-controller-manager-78f8948974-j62hd" Dec 03 11:36:25 crc kubenswrapper[5052]: I1203 11:36:25.344295 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-operators/swift-operator-controller-manager-5f8c65bbfc-2cgdz" Dec 03 11:36:25 crc kubenswrapper[5052]: I1203 11:36:25.345793 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/ovn-operator-controller-manager-b6456fdb6-sggpb" event={"ID":"a605d632-d9a5-4ce0-9449-e3d57105df1b","Type":"ContainerStarted","Data":"6e810659ae074c724982667dc34e29af08df25d8aa94eea774ad9e2b2ba3b241"} Dec 03 11:36:25 crc kubenswrapper[5052]: I1203 11:36:25.346228 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-operators/ovn-operator-controller-manager-b6456fdb6-sggpb" Dec 03 11:36:25 crc kubenswrapper[5052]: I1203 11:36:25.347938 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/heat-operator-controller-manager-5f64f6f8bb-48fnb" event={"ID":"0cbd234c-ecb3-4e54-af86-a2b15fdd94a3","Type":"ContainerStarted","Data":"ac6e6a21862aaede93ccb524e481d0b5e6a4f1f2e70966ac0cb19106016bbeef"} Dec 03 11:36:25 crc kubenswrapper[5052]: I1203 11:36:25.349430 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-operators/heat-operator-controller-manager-5f64f6f8bb-48fnb" Dec 03 11:36:25 crc kubenswrapper[5052]: I1203 11:36:25.350839 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-operators/heat-operator-controller-manager-5f64f6f8bb-48fnb" Dec 03 11:36:25 crc kubenswrapper[5052]: I1203 11:36:25.351320 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/ironic-operator-controller-manager-6c548fd776-pqfnd" event={"ID":"7017b3f7-1eb9-4dfe-8ef3-6db27cf3a1f1","Type":"ContainerStarted","Data":"f4171f259274e4008a6e5b98254b34066d3fc61afa68cf052421761cd5affdfb"} Dec 03 11:36:25 crc kubenswrapper[5052]: I1203 11:36:25.351797 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-operators/ironic-operator-controller-manager-6c548fd776-pqfnd" Dec 03 11:36:25 crc kubenswrapper[5052]: I1203 11:36:25.354308 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/watcher-operator-controller-manager-769dc69bc-7rhg6" event={"ID":"6bea5f42-82da-4148-9996-dc05121212f7","Type":"ContainerStarted","Data":"7145c8d45908da236459c9313b6926c516ea98b7ba253967ef2007aca4397a66"} Dec 03 11:36:25 crc kubenswrapper[5052]: I1203 11:36:25.355699 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-operators/watcher-operator-controller-manager-769dc69bc-7rhg6" Dec 03 11:36:25 crc kubenswrapper[5052]: I1203 11:36:25.357167 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-operators/watcher-operator-controller-manager-769dc69bc-7rhg6" Dec 03 11:36:25 crc kubenswrapper[5052]: I1203 11:36:25.357367 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/nova-operator-controller-manager-697bc559fc-gcj5d" event={"ID":"3e6bc3f6-1f1a-4b99-8835-98252369d364","Type":"ContainerStarted","Data":"a7e939b334d0e92f65588cb0c22e7536f6716a1e32472cd5033b456da2b36a68"} Dec 03 11:36:25 crc kubenswrapper[5052]: I1203 11:36:25.361878 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/openstack-baremetal-operator-controller-manager-55d86b6686f2mft" event={"ID":"e10da887-70df-43aa-aa79-4a14231e5ec9","Type":"ContainerStarted","Data":"0f00b4f3db71336942e4ffb2969c9f402b3aa40103a5ee65081fd36ca18be57d"} Dec 03 11:36:25 crc kubenswrapper[5052]: I1203 11:36:25.362491 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-operators/openstack-baremetal-operator-controller-manager-55d86b6686f2mft" Dec 03 11:36:25 crc kubenswrapper[5052]: I1203 11:36:25.366683 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/barbican-operator-controller-manager-7d9dfd778-cqfdk" event={"ID":"8dbd25a1-9de9-431e-be37-d239e3f64728","Type":"ContainerStarted","Data":"754a05a1ad89a85b548ec6034299b8041b571878c20c15a1d29f8a87a0d19577"} Dec 03 11:36:25 crc kubenswrapper[5052]: I1203 11:36:25.366875 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-operators/barbican-operator-controller-manager-7d9dfd778-cqfdk" Dec 03 11:36:25 crc kubenswrapper[5052]: I1203 11:36:25.368548 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/test-operator-controller-manager-5854674fcc-jltb6" event={"ID":"1cb3d739-e4d6-4058-9456-a3c2e1c09130","Type":"ContainerStarted","Data":"dabaf7e10aa7d0b0ec06b6b8e0023ae509392bd5567d994a850b73988ff68ec4"} Dec 03 11:36:25 crc kubenswrapper[5052]: I1203 11:36:25.369041 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-operators/test-operator-controller-manager-5854674fcc-jltb6" Dec 03 11:36:25 crc kubenswrapper[5052]: I1203 11:36:25.371733 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-operators/barbican-operator-controller-manager-7d9dfd778-cqfdk" Dec 03 11:36:25 crc kubenswrapper[5052]: I1203 11:36:25.373377 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/mariadb-operator-controller-manager-56bbcc9d85-sr7zw" event={"ID":"f0e70c0d-10fe-43b7-a08c-0ce411f223a6","Type":"ContainerStarted","Data":"7a38fc46fa1de916c4dcdb66fb009b9e798bf1d5b7c4bb72dc6db4bde166741c"} Dec 03 11:36:25 crc kubenswrapper[5052]: I1203 11:36:25.373940 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-operators/mariadb-operator-controller-manager-56bbcc9d85-sr7zw" Dec 03 11:36:25 crc kubenswrapper[5052]: I1203 11:36:25.379838 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/designate-operator-controller-manager-78b4bc895b-874g2" event={"ID":"2e2dc077-c02f-4f98-8fa6-9f72890faeaf","Type":"ContainerStarted","Data":"c38ccb2125893d9107a9365db7e805b861125cfe34c07ee20dd8158df83fc99c"} Dec 03 11:36:25 crc kubenswrapper[5052]: I1203 11:36:25.380392 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-operators/designate-operator-controller-manager-78b4bc895b-874g2" Dec 03 11:36:25 crc kubenswrapper[5052]: I1203 11:36:25.381804 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-operators/designate-operator-controller-manager-78b4bc895b-874g2" Dec 03 11:36:25 crc kubenswrapper[5052]: I1203 11:36:25.381913 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/horizon-operator-controller-manager-68c6d99b8f-mjdvh" event={"ID":"ce3d34f6-e545-4eb2-bcc5-999377931601","Type":"ContainerStarted","Data":"6ba995fc9ab1c19d638faf3e62a10a01ba4220c74844ad63f599ace4affd6e49"} Dec 03 11:36:25 crc kubenswrapper[5052]: I1203 11:36:25.383764 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-operators/horizon-operator-controller-manager-68c6d99b8f-mjdvh" Dec 03 11:36:25 crc kubenswrapper[5052]: I1203 11:36:25.387773 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-operators/horizon-operator-controller-manager-68c6d99b8f-mjdvh" Dec 03 11:36:25 crc kubenswrapper[5052]: I1203 11:36:25.389270 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/telemetry-operator-controller-manager-76cc84c6bb-zl7px" event={"ID":"3894016b-8242-4b12-8407-bcf78af00851","Type":"ContainerStarted","Data":"3015e5486b47a528197db8ba47bf3513191f52877b648e0cf08565573ad9202b"} Dec 03 11:36:25 crc kubenswrapper[5052]: I1203 11:36:25.389449 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-operators/telemetry-operator-controller-manager-76cc84c6bb-zl7px" Dec 03 11:36:25 crc kubenswrapper[5052]: I1203 11:36:25.394032 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/infra-operator-controller-manager-57548d458d-9fkll" event={"ID":"81f5f8e1-015a-4b45-83e6-13e88efb8e80","Type":"ContainerStarted","Data":"9e9b4a8947d10fcefa968eaeff05f6115cfd21c0b57f83374c28eb0d6a26a01b"} Dec 03 11:36:25 crc kubenswrapper[5052]: I1203 11:36:25.395797 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-operators/swift-operator-controller-manager-5f8c65bbfc-2cgdz" podStartSLOduration=3.165887131 podStartE2EDuration="32.395780981s" podCreationTimestamp="2025-12-03 11:35:53 +0000 UTC" firstStartedPulling="2025-12-03 11:35:54.775859502 +0000 UTC m=+945.364101266" lastFinishedPulling="2025-12-03 11:36:24.005753352 +0000 UTC m=+974.593995116" observedRunningTime="2025-12-03 11:36:25.371716664 +0000 UTC m=+975.959958428" watchObservedRunningTime="2025-12-03 11:36:25.395780981 +0000 UTC m=+975.984022755" Dec 03 11:36:25 crc kubenswrapper[5052]: I1203 11:36:25.396200 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-operators/placement-operator-controller-manager-78f8948974-j62hd" podStartSLOduration=3.871878912 podStartE2EDuration="33.396194403s" podCreationTimestamp="2025-12-03 11:35:52 +0000 UTC" firstStartedPulling="2025-12-03 11:35:54.67898975 +0000 UTC m=+945.267231514" lastFinishedPulling="2025-12-03 11:36:24.203305241 +0000 UTC m=+974.791547005" observedRunningTime="2025-12-03 11:36:25.394879665 +0000 UTC m=+975.983121439" watchObservedRunningTime="2025-12-03 11:36:25.396194403 +0000 UTC m=+975.984436167" Dec 03 11:36:25 crc kubenswrapper[5052]: I1203 11:36:25.398896 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/manila-operator-controller-manager-7c79b5df47-ccbqq" event={"ID":"ea1e91de-7e7d-498a-a17b-551f2bee325a","Type":"ContainerStarted","Data":"c8a22e6613869e6f1219b539f8d24e9e232f41001d8d1d96f9c514c62fd7214c"} Dec 03 11:36:25 crc kubenswrapper[5052]: I1203 11:36:25.399177 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-operators/manila-operator-controller-manager-7c79b5df47-ccbqq" Dec 03 11:36:25 crc kubenswrapper[5052]: I1203 11:36:25.401073 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/cinder-operator-controller-manager-859b6ccc6-vt9d2" event={"ID":"5968d273-33dd-45cc-aab0-e83628bcd15c","Type":"ContainerStarted","Data":"cd51be9345abb5eafe858033c68a1ced7d4100915afaf161d165bcac5ef6661e"} Dec 03 11:36:25 crc kubenswrapper[5052]: I1203 11:36:25.401524 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-operators/cinder-operator-controller-manager-859b6ccc6-vt9d2" Dec 03 11:36:25 crc kubenswrapper[5052]: I1203 11:36:25.402236 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-operators/manila-operator-controller-manager-7c79b5df47-ccbqq" Dec 03 11:36:25 crc kubenswrapper[5052]: I1203 11:36:25.402800 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-operators/cinder-operator-controller-manager-859b6ccc6-vt9d2" Dec 03 11:36:25 crc kubenswrapper[5052]: I1203 11:36:25.403014 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/neutron-operator-controller-manager-5fdfd5b6b5-q48nd" event={"ID":"53c88ca3-39f7-4a81-b305-40047bcaeb88","Type":"ContainerStarted","Data":"efce01fd34c9aaa82b7beeb80cae30909d6a58a957f233a93543d43ab86d6552"} Dec 03 11:36:25 crc kubenswrapper[5052]: I1203 11:36:25.403733 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-operators/neutron-operator-controller-manager-5fdfd5b6b5-q48nd" Dec 03 11:36:25 crc kubenswrapper[5052]: I1203 11:36:25.405344 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/octavia-operator-controller-manager-998648c74-z6cw9" event={"ID":"50c6e918-7f1f-4d6f-ba5f-940e5f485825","Type":"ContainerStarted","Data":"c55bdca1c305eb9768bdfbe023291651a872e6e10fe50e1b32182c071f21b870"} Dec 03 11:36:25 crc kubenswrapper[5052]: I1203 11:36:25.405831 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-operators/octavia-operator-controller-manager-998648c74-z6cw9" Dec 03 11:36:25 crc kubenswrapper[5052]: I1203 11:36:25.407791 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-operators/octavia-operator-controller-manager-998648c74-z6cw9" Dec 03 11:36:25 crc kubenswrapper[5052]: I1203 11:36:25.408637 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/glance-operator-controller-manager-77987cd8cd-btn2l" event={"ID":"d429f7ab-9a88-4411-a7a5-88bc8ab76285","Type":"ContainerStarted","Data":"cd378be1b828198fb7e23f2fd50570c21a7bd0e958e5f4f92497cb7ebe420226"} Dec 03 11:36:25 crc kubenswrapper[5052]: I1203 11:36:25.408726 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-operators/glance-operator-controller-manager-77987cd8cd-btn2l" Dec 03 11:36:25 crc kubenswrapper[5052]: E1203 11:36:25.409129 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openstack-k8s-operators/keystone-operator@sha256:72ad6517987f674af0d0ae092cbb874aeae909c8b8b60188099c311762ebc8f7\\\"\"" pod="openstack-operators/keystone-operator-controller-manager-7765d96ddf-nlrl9" podUID="95b50980-fd5a-4a19-b1cd-13df98d257fe" Dec 03 11:36:25 crc kubenswrapper[5052]: I1203 11:36:25.430767 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-operators/heat-operator-controller-manager-5f64f6f8bb-48fnb" podStartSLOduration=3.4728166910000002 podStartE2EDuration="33.430743143s" podCreationTimestamp="2025-12-03 11:35:52 +0000 UTC" firstStartedPulling="2025-12-03 11:35:54.118815885 +0000 UTC m=+944.707057649" lastFinishedPulling="2025-12-03 11:36:24.076742337 +0000 UTC m=+974.664984101" observedRunningTime="2025-12-03 11:36:25.421814705 +0000 UTC m=+976.010056469" watchObservedRunningTime="2025-12-03 11:36:25.430743143 +0000 UTC m=+976.018984907" Dec 03 11:36:25 crc kubenswrapper[5052]: I1203 11:36:25.490217 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-operators/ironic-operator-controller-manager-6c548fd776-pqfnd" podStartSLOduration=3.687141651 podStartE2EDuration="33.490203174s" podCreationTimestamp="2025-12-03 11:35:52 +0000 UTC" firstStartedPulling="2025-12-03 11:35:54.384876283 +0000 UTC m=+944.973118047" lastFinishedPulling="2025-12-03 11:36:24.187937806 +0000 UTC m=+974.776179570" observedRunningTime="2025-12-03 11:36:25.490027539 +0000 UTC m=+976.078269303" watchObservedRunningTime="2025-12-03 11:36:25.490203174 +0000 UTC m=+976.078444938" Dec 03 11:36:25 crc kubenswrapper[5052]: I1203 11:36:25.518899 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-operators/watcher-operator-controller-manager-769dc69bc-7rhg6" podStartSLOduration=3.488972366 podStartE2EDuration="32.518877254s" podCreationTimestamp="2025-12-03 11:35:53 +0000 UTC" firstStartedPulling="2025-12-03 11:35:54.835536302 +0000 UTC m=+945.423778066" lastFinishedPulling="2025-12-03 11:36:23.86544119 +0000 UTC m=+974.453682954" observedRunningTime="2025-12-03 11:36:25.517828364 +0000 UTC m=+976.106070148" watchObservedRunningTime="2025-12-03 11:36:25.518877254 +0000 UTC m=+976.107119018" Dec 03 11:36:25 crc kubenswrapper[5052]: I1203 11:36:25.556636 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-operators/ovn-operator-controller-manager-b6456fdb6-sggpb" podStartSLOduration=4.146006159 podStartE2EDuration="33.556616677s" podCreationTimestamp="2025-12-03 11:35:52 +0000 UTC" firstStartedPulling="2025-12-03 11:35:54.668089136 +0000 UTC m=+945.256330900" lastFinishedPulling="2025-12-03 11:36:24.078699654 +0000 UTC m=+974.666941418" observedRunningTime="2025-12-03 11:36:25.550924452 +0000 UTC m=+976.139166216" watchObservedRunningTime="2025-12-03 11:36:25.556616677 +0000 UTC m=+976.144858441" Dec 03 11:36:25 crc kubenswrapper[5052]: I1203 11:36:25.581945 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-operators/barbican-operator-controller-manager-7d9dfd778-cqfdk" podStartSLOduration=3.686658752 podStartE2EDuration="33.581919029s" podCreationTimestamp="2025-12-03 11:35:52 +0000 UTC" firstStartedPulling="2025-12-03 11:35:54.190385188 +0000 UTC m=+944.778626952" lastFinishedPulling="2025-12-03 11:36:24.085645455 +0000 UTC m=+974.673887229" observedRunningTime="2025-12-03 11:36:25.574421652 +0000 UTC m=+976.162663436" watchObservedRunningTime="2025-12-03 11:36:25.581919029 +0000 UTC m=+976.170160793" Dec 03 11:36:25 crc kubenswrapper[5052]: I1203 11:36:25.622676 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-operators/openstack-baremetal-operator-controller-manager-55d86b6686f2mft" podStartSLOduration=30.14931475 podStartE2EDuration="33.622655029s" podCreationTimestamp="2025-12-03 11:35:52 +0000 UTC" firstStartedPulling="2025-12-03 11:36:20.305532955 +0000 UTC m=+970.893774719" lastFinishedPulling="2025-12-03 11:36:23.778873234 +0000 UTC m=+974.367114998" observedRunningTime="2025-12-03 11:36:25.622333399 +0000 UTC m=+976.210575183" watchObservedRunningTime="2025-12-03 11:36:25.622655029 +0000 UTC m=+976.210896793" Dec 03 11:36:25 crc kubenswrapper[5052]: I1203 11:36:25.675728 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-operators/test-operator-controller-manager-5854674fcc-jltb6" podStartSLOduration=3.399814241 podStartE2EDuration="32.675712665s" podCreationTimestamp="2025-12-03 11:35:53 +0000 UTC" firstStartedPulling="2025-12-03 11:35:54.800539304 +0000 UTC m=+945.388781068" lastFinishedPulling="2025-12-03 11:36:24.076437728 +0000 UTC m=+974.664679492" observedRunningTime="2025-12-03 11:36:25.672073979 +0000 UTC m=+976.260315743" watchObservedRunningTime="2025-12-03 11:36:25.675712665 +0000 UTC m=+976.263954429" Dec 03 11:36:25 crc kubenswrapper[5052]: I1203 11:36:25.723689 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-operators/glance-operator-controller-manager-77987cd8cd-btn2l" podStartSLOduration=4.018358981 podStartE2EDuration="33.723672763s" podCreationTimestamp="2025-12-03 11:35:52 +0000 UTC" firstStartedPulling="2025-12-03 11:35:54.369426107 +0000 UTC m=+944.957667871" lastFinishedPulling="2025-12-03 11:36:24.074739889 +0000 UTC m=+974.662981653" observedRunningTime="2025-12-03 11:36:25.702187831 +0000 UTC m=+976.290429585" watchObservedRunningTime="2025-12-03 11:36:25.723672763 +0000 UTC m=+976.311914527" Dec 03 11:36:25 crc kubenswrapper[5052]: I1203 11:36:25.725786 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-operators/manila-operator-controller-manager-7c79b5df47-ccbqq" podStartSLOduration=4.011190373 podStartE2EDuration="33.725780214s" podCreationTimestamp="2025-12-03 11:35:52 +0000 UTC" firstStartedPulling="2025-12-03 11:35:54.370741615 +0000 UTC m=+944.958983379" lastFinishedPulling="2025-12-03 11:36:24.085331456 +0000 UTC m=+974.673573220" observedRunningTime="2025-12-03 11:36:25.721389377 +0000 UTC m=+976.309631141" watchObservedRunningTime="2025-12-03 11:36:25.725780214 +0000 UTC m=+976.314021978" Dec 03 11:36:25 crc kubenswrapper[5052]: I1203 11:36:25.793038 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-operators/horizon-operator-controller-manager-68c6d99b8f-mjdvh" podStartSLOduration=3.7851387450000002 podStartE2EDuration="33.7930221s" podCreationTimestamp="2025-12-03 11:35:52 +0000 UTC" firstStartedPulling="2025-12-03 11:35:54.092084305 +0000 UTC m=+944.680326069" lastFinishedPulling="2025-12-03 11:36:24.09996766 +0000 UTC m=+974.688209424" observedRunningTime="2025-12-03 11:36:25.758858552 +0000 UTC m=+976.347100316" watchObservedRunningTime="2025-12-03 11:36:25.7930221 +0000 UTC m=+976.381263864" Dec 03 11:36:25 crc kubenswrapper[5052]: I1203 11:36:25.794366 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-operators/cinder-operator-controller-manager-859b6ccc6-vt9d2" podStartSLOduration=3.879798015 podStartE2EDuration="33.794360269s" podCreationTimestamp="2025-12-03 11:35:52 +0000 UTC" firstStartedPulling="2025-12-03 11:35:54.092451035 +0000 UTC m=+944.680692799" lastFinishedPulling="2025-12-03 11:36:24.007013289 +0000 UTC m=+974.595255053" observedRunningTime="2025-12-03 11:36:25.791999571 +0000 UTC m=+976.380241335" watchObservedRunningTime="2025-12-03 11:36:25.794360269 +0000 UTC m=+976.382602033" Dec 03 11:36:25 crc kubenswrapper[5052]: I1203 11:36:25.854112 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-operators/mariadb-operator-controller-manager-56bbcc9d85-sr7zw" podStartSLOduration=4.378146497 podStartE2EDuration="33.854089878s" podCreationTimestamp="2025-12-03 11:35:52 +0000 UTC" firstStartedPulling="2025-12-03 11:35:54.602293269 +0000 UTC m=+945.190535033" lastFinishedPulling="2025-12-03 11:36:24.07823665 +0000 UTC m=+974.666478414" observedRunningTime="2025-12-03 11:36:25.853649166 +0000 UTC m=+976.441890930" watchObservedRunningTime="2025-12-03 11:36:25.854089878 +0000 UTC m=+976.442331642" Dec 03 11:36:25 crc kubenswrapper[5052]: I1203 11:36:25.873405 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-operators/neutron-operator-controller-manager-5fdfd5b6b5-q48nd" podStartSLOduration=3.465674988 podStartE2EDuration="33.873389947s" podCreationTimestamp="2025-12-03 11:35:52 +0000 UTC" firstStartedPulling="2025-12-03 11:35:54.556980563 +0000 UTC m=+945.145222327" lastFinishedPulling="2025-12-03 11:36:24.964695522 +0000 UTC m=+975.552937286" observedRunningTime="2025-12-03 11:36:25.873039967 +0000 UTC m=+976.461281731" watchObservedRunningTime="2025-12-03 11:36:25.873389947 +0000 UTC m=+976.461631711" Dec 03 11:36:25 crc kubenswrapper[5052]: I1203 11:36:25.887460 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-operators/telemetry-operator-controller-manager-76cc84c6bb-zl7px" podStartSLOduration=2.6006336770000003 podStartE2EDuration="32.887441924s" podCreationTimestamp="2025-12-03 11:35:53 +0000 UTC" firstStartedPulling="2025-12-03 11:35:54.549461147 +0000 UTC m=+945.137702911" lastFinishedPulling="2025-12-03 11:36:24.836269394 +0000 UTC m=+975.424511158" observedRunningTime="2025-12-03 11:36:25.884440757 +0000 UTC m=+976.472682531" watchObservedRunningTime="2025-12-03 11:36:25.887441924 +0000 UTC m=+976.475683688" Dec 03 11:36:25 crc kubenswrapper[5052]: I1203 11:36:25.918749 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-operators/designate-operator-controller-manager-78b4bc895b-874g2" podStartSLOduration=4.171139989 podStartE2EDuration="33.918726009s" podCreationTimestamp="2025-12-03 11:35:52 +0000 UTC" firstStartedPulling="2025-12-03 11:35:54.096013608 +0000 UTC m=+944.684255372" lastFinishedPulling="2025-12-03 11:36:23.843599628 +0000 UTC m=+974.431841392" observedRunningTime="2025-12-03 11:36:25.904427745 +0000 UTC m=+976.492669509" watchObservedRunningTime="2025-12-03 11:36:25.918726009 +0000 UTC m=+976.506967773" Dec 03 11:36:25 crc kubenswrapper[5052]: I1203 11:36:25.918921 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-operators/octavia-operator-controller-manager-998648c74-z6cw9" podStartSLOduration=4.341812136 podStartE2EDuration="33.918915705s" podCreationTimestamp="2025-12-03 11:35:52 +0000 UTC" firstStartedPulling="2025-12-03 11:35:54.364889987 +0000 UTC m=+944.953131751" lastFinishedPulling="2025-12-03 11:36:23.941993546 +0000 UTC m=+974.530235320" observedRunningTime="2025-12-03 11:36:25.917225576 +0000 UTC m=+976.505467350" watchObservedRunningTime="2025-12-03 11:36:25.918915705 +0000 UTC m=+976.507157469" Dec 03 11:36:26 crc kubenswrapper[5052]: I1203 11:36:26.419841 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-operators/ironic-operator-controller-manager-6c548fd776-pqfnd" Dec 03 11:36:26 crc kubenswrapper[5052]: I1203 11:36:26.421167 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-operators/mariadb-operator-controller-manager-56bbcc9d85-sr7zw" Dec 03 11:36:26 crc kubenswrapper[5052]: I1203 11:36:26.421211 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-operators/ovn-operator-controller-manager-b6456fdb6-sggpb" Dec 03 11:36:26 crc kubenswrapper[5052]: I1203 11:36:26.422429 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-operators/test-operator-controller-manager-5854674fcc-jltb6" Dec 03 11:36:27 crc kubenswrapper[5052]: I1203 11:36:27.426471 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-operators/glance-operator-controller-manager-77987cd8cd-btn2l" Dec 03 11:36:27 crc kubenswrapper[5052]: I1203 11:36:27.426855 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-operators/placement-operator-controller-manager-78f8948974-j62hd" Dec 03 11:36:28 crc kubenswrapper[5052]: I1203 11:36:28.433562 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/infra-operator-controller-manager-57548d458d-9fkll" event={"ID":"81f5f8e1-015a-4b45-83e6-13e88efb8e80","Type":"ContainerStarted","Data":"01a4046ea27a459acec0b9a6a2a20c352ddcb26e0f79bddb80874932462702e9"} Dec 03 11:36:28 crc kubenswrapper[5052]: I1203 11:36:28.433594 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/infra-operator-controller-manager-57548d458d-9fkll" event={"ID":"81f5f8e1-015a-4b45-83e6-13e88efb8e80","Type":"ContainerStarted","Data":"43ae682827f3b8fe92dd61ac021bde418f76e0a34a72497c150c64083536410d"} Dec 03 11:36:28 crc kubenswrapper[5052]: I1203 11:36:28.433926 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-operators/infra-operator-controller-manager-57548d458d-9fkll" Dec 03 11:36:28 crc kubenswrapper[5052]: I1203 11:36:28.455117 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-operators/infra-operator-controller-manager-57548d458d-9fkll" podStartSLOduration=33.823592645 podStartE2EDuration="36.455099172s" podCreationTimestamp="2025-12-03 11:35:52 +0000 UTC" firstStartedPulling="2025-12-03 11:36:25.23098478 +0000 UTC m=+975.819226544" lastFinishedPulling="2025-12-03 11:36:27.862491307 +0000 UTC m=+978.450733071" observedRunningTime="2025-12-03 11:36:28.449407617 +0000 UTC m=+979.037649381" watchObservedRunningTime="2025-12-03 11:36:28.455099172 +0000 UTC m=+979.043340926" Dec 03 11:36:29 crc kubenswrapper[5052]: I1203 11:36:29.112878 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-operators/openstack-baremetal-operator-controller-manager-55d86b6686f2mft" Dec 03 11:36:29 crc kubenswrapper[5052]: I1203 11:36:29.379417 5052 patch_prober.go:28] interesting pod/machine-config-daemon-74bj6 container/machine-config-daemon namespace/openshift-machine-config-operator: Liveness probe status=failure output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" start-of-body= Dec 03 11:36:29 crc kubenswrapper[5052]: I1203 11:36:29.379476 5052 prober.go:107] "Probe failed" probeType="Liveness" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" containerName="machine-config-daemon" probeResult="failure" output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" Dec 03 11:36:29 crc kubenswrapper[5052]: I1203 11:36:29.379545 5052 kubelet.go:2542] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" Dec 03 11:36:29 crc kubenswrapper[5052]: I1203 11:36:29.380144 5052 kuberuntime_manager.go:1027] "Message for Container of pod" containerName="machine-config-daemon" containerStatusID={"Type":"cri-o","ID":"f37a0f397f498a9429565aa0bada2c6757e002bd2d8af169475a45c6c645d73e"} pod="openshift-machine-config-operator/machine-config-daemon-74bj6" containerMessage="Container machine-config-daemon failed liveness probe, will be restarted" Dec 03 11:36:29 crc kubenswrapper[5052]: I1203 11:36:29.380200 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" containerName="machine-config-daemon" containerID="cri-o://f37a0f397f498a9429565aa0bada2c6757e002bd2d8af169475a45c6c645d73e" gracePeriod=600 Dec 03 11:36:29 crc kubenswrapper[5052]: I1203 11:36:29.559045 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-operators/openstack-operator-controller-manager-9f56fc979-b8jv8" Dec 03 11:36:30 crc kubenswrapper[5052]: I1203 11:36:30.448862 5052 generic.go:334] "Generic (PLEG): container finished" podID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" containerID="f37a0f397f498a9429565aa0bada2c6757e002bd2d8af169475a45c6c645d73e" exitCode=0 Dec 03 11:36:30 crc kubenswrapper[5052]: I1203 11:36:30.448934 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" event={"ID":"10c237bc-72e7-4da3-9f2c-b1688395bf6e","Type":"ContainerDied","Data":"f37a0f397f498a9429565aa0bada2c6757e002bd2d8af169475a45c6c645d73e"} Dec 03 11:36:30 crc kubenswrapper[5052]: I1203 11:36:30.449303 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" event={"ID":"10c237bc-72e7-4da3-9f2c-b1688395bf6e","Type":"ContainerStarted","Data":"42f4b7304c54076bcbb388fc8df0720f8096981812e6c2b3d22518ea7235934e"} Dec 03 11:36:30 crc kubenswrapper[5052]: I1203 11:36:30.449361 5052 scope.go:117] "RemoveContainer" containerID="a8ea894737897a85494be3992d10d75dae39643d0a55aa5bfc62a4269324bdcd" Dec 03 11:36:33 crc kubenswrapper[5052]: I1203 11:36:33.431859 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-operators/neutron-operator-controller-manager-5fdfd5b6b5-q48nd" Dec 03 11:36:33 crc kubenswrapper[5052]: I1203 11:36:33.436948 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-operators/nova-operator-controller-manager-697bc559fc-gcj5d" Dec 03 11:36:33 crc kubenswrapper[5052]: I1203 11:36:33.842668 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-operators/telemetry-operator-controller-manager-76cc84c6bb-zl7px" Dec 03 11:36:34 crc kubenswrapper[5052]: I1203 11:36:34.871597 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-operators/infra-operator-controller-manager-57548d458d-9fkll" Dec 03 11:36:35 crc kubenswrapper[5052]: I1203 11:36:35.502184 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/rabbitmq-cluster-operator-manager-668c99d594-ksw2p" event={"ID":"459f5555-efae-4c19-84d0-2af917b9560f","Type":"ContainerStarted","Data":"9f7f47624da378a4c9e182afc6f0245cb94a2c07414a47ce8f104bcb13883f17"} Dec 03 11:36:35 crc kubenswrapper[5052]: I1203 11:36:35.524817 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-operators/rabbitmq-cluster-operator-manager-668c99d594-ksw2p" podStartSLOduration=2.290160967 podStartE2EDuration="42.524794406s" podCreationTimestamp="2025-12-03 11:35:53 +0000 UTC" firstStartedPulling="2025-12-03 11:35:54.962906573 +0000 UTC m=+945.551148337" lastFinishedPulling="2025-12-03 11:36:35.197540012 +0000 UTC m=+985.785781776" observedRunningTime="2025-12-03 11:36:35.519998957 +0000 UTC m=+986.108240721" watchObservedRunningTime="2025-12-03 11:36:35.524794406 +0000 UTC m=+986.113036170" Dec 03 11:36:38 crc kubenswrapper[5052]: I1203 11:36:38.536814 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/keystone-operator-controller-manager-7765d96ddf-nlrl9" event={"ID":"95b50980-fd5a-4a19-b1cd-13df98d257fe","Type":"ContainerStarted","Data":"61b7e037d8a6c427bda841efbbb994e108ee2daac74c44aca5542d608eed3910"} Dec 03 11:36:38 crc kubenswrapper[5052]: I1203 11:36:38.538402 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-operators/keystone-operator-controller-manager-7765d96ddf-nlrl9" Dec 03 11:36:38 crc kubenswrapper[5052]: I1203 11:36:38.564440 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-operators/keystone-operator-controller-manager-7765d96ddf-nlrl9" podStartSLOduration=3.214223773 podStartE2EDuration="46.564423786s" podCreationTimestamp="2025-12-03 11:35:52 +0000 UTC" firstStartedPulling="2025-12-03 11:35:54.369815009 +0000 UTC m=+944.958056773" lastFinishedPulling="2025-12-03 11:36:37.720015022 +0000 UTC m=+988.308256786" observedRunningTime="2025-12-03 11:36:38.561330687 +0000 UTC m=+989.149572461" watchObservedRunningTime="2025-12-03 11:36:38.564423786 +0000 UTC m=+989.152665550" Dec 03 11:36:45 crc kubenswrapper[5052]: I1203 11:36:43.302666 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-operators/keystone-operator-controller-manager-7765d96ddf-nlrl9" Dec 03 11:36:50 crc kubenswrapper[5052]: I1203 11:36:50.036970 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["crc-storage/crc-storage-crc-zjczd"] Dec 03 11:36:50 crc kubenswrapper[5052]: I1203 11:36:50.039491 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="crc-storage/crc-storage-crc-zjczd" Dec 03 11:36:50 crc kubenswrapper[5052]: I1203 11:36:50.041847 5052 reflector.go:368] Caches populated for *v1.Secret from object-"crc-storage"/"crc-storage-dockercfg-mf76g" Dec 03 11:36:50 crc kubenswrapper[5052]: I1203 11:36:50.042118 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"crc-storage"/"openshift-service-ca.crt" Dec 03 11:36:50 crc kubenswrapper[5052]: I1203 11:36:50.042254 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"crc-storage"/"crc-storage" Dec 03 11:36:50 crc kubenswrapper[5052]: I1203 11:36:50.050417 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"crc-storage"/"kube-root-ca.crt" Dec 03 11:36:50 crc kubenswrapper[5052]: I1203 11:36:50.050949 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["crc-storage/crc-storage-crc-zjczd"] Dec 03 11:36:50 crc kubenswrapper[5052]: I1203 11:36:50.256616 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/f7a857fe-facf-42aa-a4f0-7d5f923a5571-crc-storage\") pod \"crc-storage-crc-zjczd\" (UID: \"f7a857fe-facf-42aa-a4f0-7d5f923a5571\") " pod="crc-storage/crc-storage-crc-zjczd" Dec 03 11:36:50 crc kubenswrapper[5052]: I1203 11:36:50.256694 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5gh97\" (UniqueName: \"kubernetes.io/projected/f7a857fe-facf-42aa-a4f0-7d5f923a5571-kube-api-access-5gh97\") pod \"crc-storage-crc-zjczd\" (UID: \"f7a857fe-facf-42aa-a4f0-7d5f923a5571\") " pod="crc-storage/crc-storage-crc-zjczd" Dec 03 11:36:50 crc kubenswrapper[5052]: I1203 11:36:50.256716 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/f7a857fe-facf-42aa-a4f0-7d5f923a5571-node-mnt\") pod \"crc-storage-crc-zjczd\" (UID: \"f7a857fe-facf-42aa-a4f0-7d5f923a5571\") " pod="crc-storage/crc-storage-crc-zjczd" Dec 03 11:36:50 crc kubenswrapper[5052]: I1203 11:36:50.358037 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-5gh97\" (UniqueName: \"kubernetes.io/projected/f7a857fe-facf-42aa-a4f0-7d5f923a5571-kube-api-access-5gh97\") pod \"crc-storage-crc-zjczd\" (UID: \"f7a857fe-facf-42aa-a4f0-7d5f923a5571\") " pod="crc-storage/crc-storage-crc-zjczd" Dec 03 11:36:50 crc kubenswrapper[5052]: I1203 11:36:50.358084 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/f7a857fe-facf-42aa-a4f0-7d5f923a5571-node-mnt\") pod \"crc-storage-crc-zjczd\" (UID: \"f7a857fe-facf-42aa-a4f0-7d5f923a5571\") " pod="crc-storage/crc-storage-crc-zjczd" Dec 03 11:36:50 crc kubenswrapper[5052]: I1203 11:36:50.358177 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/f7a857fe-facf-42aa-a4f0-7d5f923a5571-crc-storage\") pod \"crc-storage-crc-zjczd\" (UID: \"f7a857fe-facf-42aa-a4f0-7d5f923a5571\") " pod="crc-storage/crc-storage-crc-zjczd" Dec 03 11:36:50 crc kubenswrapper[5052]: I1203 11:36:50.358570 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/f7a857fe-facf-42aa-a4f0-7d5f923a5571-node-mnt\") pod \"crc-storage-crc-zjczd\" (UID: \"f7a857fe-facf-42aa-a4f0-7d5f923a5571\") " pod="crc-storage/crc-storage-crc-zjczd" Dec 03 11:36:50 crc kubenswrapper[5052]: I1203 11:36:50.359032 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/f7a857fe-facf-42aa-a4f0-7d5f923a5571-crc-storage\") pod \"crc-storage-crc-zjczd\" (UID: \"f7a857fe-facf-42aa-a4f0-7d5f923a5571\") " pod="crc-storage/crc-storage-crc-zjczd" Dec 03 11:36:50 crc kubenswrapper[5052]: I1203 11:36:50.387544 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-5gh97\" (UniqueName: \"kubernetes.io/projected/f7a857fe-facf-42aa-a4f0-7d5f923a5571-kube-api-access-5gh97\") pod \"crc-storage-crc-zjczd\" (UID: \"f7a857fe-facf-42aa-a4f0-7d5f923a5571\") " pod="crc-storage/crc-storage-crc-zjczd" Dec 03 11:36:50 crc kubenswrapper[5052]: I1203 11:36:50.462729 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="crc-storage/crc-storage-crc-zjczd" Dec 03 11:36:50 crc kubenswrapper[5052]: I1203 11:36:50.899502 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["crc-storage/crc-storage-crc-zjczd"] Dec 03 11:36:51 crc kubenswrapper[5052]: I1203 11:36:51.649940 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="crc-storage/crc-storage-crc-zjczd" event={"ID":"f7a857fe-facf-42aa-a4f0-7d5f923a5571","Type":"ContainerStarted","Data":"6d471c917bc87063c6e1c0a2bdf056ec0c03ca805ae38c46899e9986376f78eb"} Dec 03 11:36:55 crc kubenswrapper[5052]: I1203 11:36:55.681245 5052 generic.go:334] "Generic (PLEG): container finished" podID="f7a857fe-facf-42aa-a4f0-7d5f923a5571" containerID="4371847f7354c50c0cc712325674ada84304f41aaf647cb3acc17b089cb235e7" exitCode=0 Dec 03 11:36:55 crc kubenswrapper[5052]: I1203 11:36:55.683915 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="crc-storage/crc-storage-crc-zjczd" event={"ID":"f7a857fe-facf-42aa-a4f0-7d5f923a5571","Type":"ContainerDied","Data":"4371847f7354c50c0cc712325674ada84304f41aaf647cb3acc17b089cb235e7"} Dec 03 11:36:56 crc kubenswrapper[5052]: I1203 11:36:56.960397 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="crc-storage/crc-storage-crc-zjczd" Dec 03 11:36:57 crc kubenswrapper[5052]: I1203 11:36:57.004320 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5gh97\" (UniqueName: \"kubernetes.io/projected/f7a857fe-facf-42aa-a4f0-7d5f923a5571-kube-api-access-5gh97\") pod \"f7a857fe-facf-42aa-a4f0-7d5f923a5571\" (UID: \"f7a857fe-facf-42aa-a4f0-7d5f923a5571\") " Dec 03 11:36:57 crc kubenswrapper[5052]: I1203 11:36:57.004373 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/f7a857fe-facf-42aa-a4f0-7d5f923a5571-crc-storage\") pod \"f7a857fe-facf-42aa-a4f0-7d5f923a5571\" (UID: \"f7a857fe-facf-42aa-a4f0-7d5f923a5571\") " Dec 03 11:36:57 crc kubenswrapper[5052]: I1203 11:36:57.004428 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/f7a857fe-facf-42aa-a4f0-7d5f923a5571-node-mnt\") pod \"f7a857fe-facf-42aa-a4f0-7d5f923a5571\" (UID: \"f7a857fe-facf-42aa-a4f0-7d5f923a5571\") " Dec 03 11:36:57 crc kubenswrapper[5052]: I1203 11:36:57.004754 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/f7a857fe-facf-42aa-a4f0-7d5f923a5571-node-mnt" (OuterVolumeSpecName: "node-mnt") pod "f7a857fe-facf-42aa-a4f0-7d5f923a5571" (UID: "f7a857fe-facf-42aa-a4f0-7d5f923a5571"). InnerVolumeSpecName "node-mnt". PluginName "kubernetes.io/host-path", VolumeGidValue "" Dec 03 11:36:57 crc kubenswrapper[5052]: I1203 11:36:57.010955 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f7a857fe-facf-42aa-a4f0-7d5f923a5571-kube-api-access-5gh97" (OuterVolumeSpecName: "kube-api-access-5gh97") pod "f7a857fe-facf-42aa-a4f0-7d5f923a5571" (UID: "f7a857fe-facf-42aa-a4f0-7d5f923a5571"). InnerVolumeSpecName "kube-api-access-5gh97". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:36:57 crc kubenswrapper[5052]: I1203 11:36:57.025299 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f7a857fe-facf-42aa-a4f0-7d5f923a5571-crc-storage" (OuterVolumeSpecName: "crc-storage") pod "f7a857fe-facf-42aa-a4f0-7d5f923a5571" (UID: "f7a857fe-facf-42aa-a4f0-7d5f923a5571"). InnerVolumeSpecName "crc-storage". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:36:57 crc kubenswrapper[5052]: I1203 11:36:57.105852 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-5gh97\" (UniqueName: \"kubernetes.io/projected/f7a857fe-facf-42aa-a4f0-7d5f923a5571-kube-api-access-5gh97\") on node \"crc\" DevicePath \"\"" Dec 03 11:36:57 crc kubenswrapper[5052]: I1203 11:36:57.105896 5052 reconciler_common.go:293] "Volume detached for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/f7a857fe-facf-42aa-a4f0-7d5f923a5571-crc-storage\") on node \"crc\" DevicePath \"\"" Dec 03 11:36:57 crc kubenswrapper[5052]: I1203 11:36:57.105906 5052 reconciler_common.go:293] "Volume detached for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/f7a857fe-facf-42aa-a4f0-7d5f923a5571-node-mnt\") on node \"crc\" DevicePath \"\"" Dec 03 11:36:57 crc kubenswrapper[5052]: I1203 11:36:57.697470 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="crc-storage/crc-storage-crc-zjczd" event={"ID":"f7a857fe-facf-42aa-a4f0-7d5f923a5571","Type":"ContainerDied","Data":"6d471c917bc87063c6e1c0a2bdf056ec0c03ca805ae38c46899e9986376f78eb"} Dec 03 11:36:57 crc kubenswrapper[5052]: I1203 11:36:57.697532 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6d471c917bc87063c6e1c0a2bdf056ec0c03ca805ae38c46899e9986376f78eb" Dec 03 11:36:57 crc kubenswrapper[5052]: I1203 11:36:57.697549 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="crc-storage/crc-storage-crc-zjczd" Dec 03 11:37:00 crc kubenswrapper[5052]: I1203 11:37:00.717223 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["crc-storage/crc-storage-crc-zjczd"] Dec 03 11:37:00 crc kubenswrapper[5052]: I1203 11:37:00.726046 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["crc-storage/crc-storage-crc-zjczd"] Dec 03 11:37:00 crc kubenswrapper[5052]: I1203 11:37:00.852041 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["crc-storage/crc-storage-crc-g5t8m"] Dec 03 11:37:00 crc kubenswrapper[5052]: E1203 11:37:00.852355 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="f7a857fe-facf-42aa-a4f0-7d5f923a5571" containerName="storage" Dec 03 11:37:00 crc kubenswrapper[5052]: I1203 11:37:00.852369 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7a857fe-facf-42aa-a4f0-7d5f923a5571" containerName="storage" Dec 03 11:37:00 crc kubenswrapper[5052]: I1203 11:37:00.853465 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="f7a857fe-facf-42aa-a4f0-7d5f923a5571" containerName="storage" Dec 03 11:37:00 crc kubenswrapper[5052]: I1203 11:37:00.854257 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="crc-storage/crc-storage-crc-g5t8m" Dec 03 11:37:00 crc kubenswrapper[5052]: I1203 11:37:00.856395 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"crc-storage"/"openshift-service-ca.crt" Dec 03 11:37:00 crc kubenswrapper[5052]: I1203 11:37:00.856429 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"crc-storage"/"crc-storage" Dec 03 11:37:00 crc kubenswrapper[5052]: I1203 11:37:00.856394 5052 reflector.go:368] Caches populated for *v1.Secret from object-"crc-storage"/"crc-storage-dockercfg-mf76g" Dec 03 11:37:00 crc kubenswrapper[5052]: I1203 11:37:00.858473 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"crc-storage"/"kube-root-ca.crt" Dec 03 11:37:00 crc kubenswrapper[5052]: I1203 11:37:00.860227 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["crc-storage/crc-storage-crc-g5t8m"] Dec 03 11:37:00 crc kubenswrapper[5052]: I1203 11:37:00.862380 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/f9310868-40a7-4ede-bfb4-4f97a35345d0-crc-storage\") pod \"crc-storage-crc-g5t8m\" (UID: \"f9310868-40a7-4ede-bfb4-4f97a35345d0\") " pod="crc-storage/crc-storage-crc-g5t8m" Dec 03 11:37:00 crc kubenswrapper[5052]: I1203 11:37:00.862440 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jtldm\" (UniqueName: \"kubernetes.io/projected/f9310868-40a7-4ede-bfb4-4f97a35345d0-kube-api-access-jtldm\") pod \"crc-storage-crc-g5t8m\" (UID: \"f9310868-40a7-4ede-bfb4-4f97a35345d0\") " pod="crc-storage/crc-storage-crc-g5t8m" Dec 03 11:37:00 crc kubenswrapper[5052]: I1203 11:37:00.862482 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/f9310868-40a7-4ede-bfb4-4f97a35345d0-node-mnt\") pod \"crc-storage-crc-g5t8m\" (UID: \"f9310868-40a7-4ede-bfb4-4f97a35345d0\") " pod="crc-storage/crc-storage-crc-g5t8m" Dec 03 11:37:00 crc kubenswrapper[5052]: I1203 11:37:00.964816 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-jtldm\" (UniqueName: \"kubernetes.io/projected/f9310868-40a7-4ede-bfb4-4f97a35345d0-kube-api-access-jtldm\") pod \"crc-storage-crc-g5t8m\" (UID: \"f9310868-40a7-4ede-bfb4-4f97a35345d0\") " pod="crc-storage/crc-storage-crc-g5t8m" Dec 03 11:37:00 crc kubenswrapper[5052]: I1203 11:37:00.964875 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/f9310868-40a7-4ede-bfb4-4f97a35345d0-node-mnt\") pod \"crc-storage-crc-g5t8m\" (UID: \"f9310868-40a7-4ede-bfb4-4f97a35345d0\") " pod="crc-storage/crc-storage-crc-g5t8m" Dec 03 11:37:00 crc kubenswrapper[5052]: I1203 11:37:00.965026 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/f9310868-40a7-4ede-bfb4-4f97a35345d0-crc-storage\") pod \"crc-storage-crc-g5t8m\" (UID: \"f9310868-40a7-4ede-bfb4-4f97a35345d0\") " pod="crc-storage/crc-storage-crc-g5t8m" Dec 03 11:37:00 crc kubenswrapper[5052]: I1203 11:37:00.965386 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/f9310868-40a7-4ede-bfb4-4f97a35345d0-node-mnt\") pod \"crc-storage-crc-g5t8m\" (UID: \"f9310868-40a7-4ede-bfb4-4f97a35345d0\") " pod="crc-storage/crc-storage-crc-g5t8m" Dec 03 11:37:00 crc kubenswrapper[5052]: I1203 11:37:00.965975 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/f9310868-40a7-4ede-bfb4-4f97a35345d0-crc-storage\") pod \"crc-storage-crc-g5t8m\" (UID: \"f9310868-40a7-4ede-bfb4-4f97a35345d0\") " pod="crc-storage/crc-storage-crc-g5t8m" Dec 03 11:37:00 crc kubenswrapper[5052]: I1203 11:37:00.984877 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-jtldm\" (UniqueName: \"kubernetes.io/projected/f9310868-40a7-4ede-bfb4-4f97a35345d0-kube-api-access-jtldm\") pod \"crc-storage-crc-g5t8m\" (UID: \"f9310868-40a7-4ede-bfb4-4f97a35345d0\") " pod="crc-storage/crc-storage-crc-g5t8m" Dec 03 11:37:01 crc kubenswrapper[5052]: I1203 11:37:01.196054 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="crc-storage/crc-storage-crc-g5t8m" Dec 03 11:37:01 crc kubenswrapper[5052]: I1203 11:37:01.605603 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["crc-storage/crc-storage-crc-g5t8m"] Dec 03 11:37:01 crc kubenswrapper[5052]: I1203 11:37:01.621272 5052 provider.go:102] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Dec 03 11:37:01 crc kubenswrapper[5052]: I1203 11:37:01.682138 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f7a857fe-facf-42aa-a4f0-7d5f923a5571" path="/var/lib/kubelet/pods/f7a857fe-facf-42aa-a4f0-7d5f923a5571/volumes" Dec 03 11:37:01 crc kubenswrapper[5052]: I1203 11:37:01.730076 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="crc-storage/crc-storage-crc-g5t8m" event={"ID":"f9310868-40a7-4ede-bfb4-4f97a35345d0","Type":"ContainerStarted","Data":"377c710f6d44acb37e611fb14ea7059473b2b77c32228c74d099785f0ca6d068"} Dec 03 11:37:05 crc kubenswrapper[5052]: I1203 11:37:05.767593 5052 generic.go:334] "Generic (PLEG): container finished" podID="f9310868-40a7-4ede-bfb4-4f97a35345d0" containerID="649f1e16c0c9e3ca2dc574767b85143b4bf02e3118f334d98a6e745e58991caa" exitCode=0 Dec 03 11:37:05 crc kubenswrapper[5052]: I1203 11:37:05.767671 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="crc-storage/crc-storage-crc-g5t8m" event={"ID":"f9310868-40a7-4ede-bfb4-4f97a35345d0","Type":"ContainerDied","Data":"649f1e16c0c9e3ca2dc574767b85143b4bf02e3118f334d98a6e745e58991caa"} Dec 03 11:37:07 crc kubenswrapper[5052]: I1203 11:37:07.042689 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="crc-storage/crc-storage-crc-g5t8m" Dec 03 11:37:07 crc kubenswrapper[5052]: I1203 11:37:07.048867 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jtldm\" (UniqueName: \"kubernetes.io/projected/f9310868-40a7-4ede-bfb4-4f97a35345d0-kube-api-access-jtldm\") pod \"f9310868-40a7-4ede-bfb4-4f97a35345d0\" (UID: \"f9310868-40a7-4ede-bfb4-4f97a35345d0\") " Dec 03 11:37:07 crc kubenswrapper[5052]: I1203 11:37:07.049029 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/f9310868-40a7-4ede-bfb4-4f97a35345d0-node-mnt\") pod \"f9310868-40a7-4ede-bfb4-4f97a35345d0\" (UID: \"f9310868-40a7-4ede-bfb4-4f97a35345d0\") " Dec 03 11:37:07 crc kubenswrapper[5052]: I1203 11:37:07.049074 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/f9310868-40a7-4ede-bfb4-4f97a35345d0-node-mnt" (OuterVolumeSpecName: "node-mnt") pod "f9310868-40a7-4ede-bfb4-4f97a35345d0" (UID: "f9310868-40a7-4ede-bfb4-4f97a35345d0"). InnerVolumeSpecName "node-mnt". PluginName "kubernetes.io/host-path", VolumeGidValue "" Dec 03 11:37:07 crc kubenswrapper[5052]: I1203 11:37:07.049675 5052 reconciler_common.go:293] "Volume detached for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/f9310868-40a7-4ede-bfb4-4f97a35345d0-node-mnt\") on node \"crc\" DevicePath \"\"" Dec 03 11:37:07 crc kubenswrapper[5052]: I1203 11:37:07.054910 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f9310868-40a7-4ede-bfb4-4f97a35345d0-kube-api-access-jtldm" (OuterVolumeSpecName: "kube-api-access-jtldm") pod "f9310868-40a7-4ede-bfb4-4f97a35345d0" (UID: "f9310868-40a7-4ede-bfb4-4f97a35345d0"). InnerVolumeSpecName "kube-api-access-jtldm". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:37:07 crc kubenswrapper[5052]: I1203 11:37:07.149939 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/f9310868-40a7-4ede-bfb4-4f97a35345d0-crc-storage\") pod \"f9310868-40a7-4ede-bfb4-4f97a35345d0\" (UID: \"f9310868-40a7-4ede-bfb4-4f97a35345d0\") " Dec 03 11:37:07 crc kubenswrapper[5052]: I1203 11:37:07.150545 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-jtldm\" (UniqueName: \"kubernetes.io/projected/f9310868-40a7-4ede-bfb4-4f97a35345d0-kube-api-access-jtldm\") on node \"crc\" DevicePath \"\"" Dec 03 11:37:07 crc kubenswrapper[5052]: I1203 11:37:07.164642 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f9310868-40a7-4ede-bfb4-4f97a35345d0-crc-storage" (OuterVolumeSpecName: "crc-storage") pod "f9310868-40a7-4ede-bfb4-4f97a35345d0" (UID: "f9310868-40a7-4ede-bfb4-4f97a35345d0"). InnerVolumeSpecName "crc-storage". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:37:07 crc kubenswrapper[5052]: I1203 11:37:07.251751 5052 reconciler_common.go:293] "Volume detached for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/f9310868-40a7-4ede-bfb4-4f97a35345d0-crc-storage\") on node \"crc\" DevicePath \"\"" Dec 03 11:37:07 crc kubenswrapper[5052]: I1203 11:37:07.785065 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="crc-storage/crc-storage-crc-g5t8m" event={"ID":"f9310868-40a7-4ede-bfb4-4f97a35345d0","Type":"ContainerDied","Data":"377c710f6d44acb37e611fb14ea7059473b2b77c32228c74d099785f0ca6d068"} Dec 03 11:37:07 crc kubenswrapper[5052]: I1203 11:37:07.785105 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="377c710f6d44acb37e611fb14ea7059473b2b77c32228c74d099785f0ca6d068" Dec 03 11:37:07 crc kubenswrapper[5052]: I1203 11:37:07.785137 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="crc-storage/crc-storage-crc-g5t8m" Dec 03 11:37:10 crc kubenswrapper[5052]: I1203 11:37:10.357586 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-f5849d7b9-nrdjn"] Dec 03 11:37:10 crc kubenswrapper[5052]: E1203 11:37:10.357914 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="f9310868-40a7-4ede-bfb4-4f97a35345d0" containerName="storage" Dec 03 11:37:10 crc kubenswrapper[5052]: I1203 11:37:10.357947 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="f9310868-40a7-4ede-bfb4-4f97a35345d0" containerName="storage" Dec 03 11:37:10 crc kubenswrapper[5052]: I1203 11:37:10.358092 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="f9310868-40a7-4ede-bfb4-4f97a35345d0" containerName="storage" Dec 03 11:37:10 crc kubenswrapper[5052]: I1203 11:37:10.358840 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-f5849d7b9-nrdjn" Dec 03 11:37:10 crc kubenswrapper[5052]: I1203 11:37:10.360489 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"dnsmasq-dnsmasq-dockercfg-jqmm8" Dec 03 11:37:10 crc kubenswrapper[5052]: I1203 11:37:10.361594 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"kube-root-ca.crt" Dec 03 11:37:10 crc kubenswrapper[5052]: I1203 11:37:10.361861 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"dnsmasq" Dec 03 11:37:10 crc kubenswrapper[5052]: I1203 11:37:10.374455 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"openshift-service-ca.crt" Dec 03 11:37:10 crc kubenswrapper[5052]: I1203 11:37:10.399562 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rd7b5\" (UniqueName: \"kubernetes.io/projected/897024db-e95e-4ab8-868f-ddb1ffadcbed-kube-api-access-rd7b5\") pod \"dnsmasq-dnsmasq-f5849d7b9-nrdjn\" (UID: \"897024db-e95e-4ab8-868f-ddb1ffadcbed\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-f5849d7b9-nrdjn" Dec 03 11:37:10 crc kubenswrapper[5052]: I1203 11:37:10.399614 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/897024db-e95e-4ab8-868f-ddb1ffadcbed-config\") pod \"dnsmasq-dnsmasq-f5849d7b9-nrdjn\" (UID: \"897024db-e95e-4ab8-868f-ddb1ffadcbed\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-f5849d7b9-nrdjn" Dec 03 11:37:10 crc kubenswrapper[5052]: I1203 11:37:10.405928 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-f5849d7b9-nrdjn"] Dec 03 11:37:10 crc kubenswrapper[5052]: I1203 11:37:10.426891 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-78bj2"] Dec 03 11:37:10 crc kubenswrapper[5052]: I1203 11:37:10.428153 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-78bj2" Dec 03 11:37:10 crc kubenswrapper[5052]: I1203 11:37:10.430427 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"dnsmasq-svc" Dec 03 11:37:10 crc kubenswrapper[5052]: I1203 11:37:10.442187 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-78bj2"] Dec 03 11:37:10 crc kubenswrapper[5052]: I1203 11:37:10.500701 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c5216bac-5393-4c1e-82e0-748b95282da0-config\") pod \"dnsmasq-dnsmasq-84b9f45d47-78bj2\" (UID: \"c5216bac-5393-4c1e-82e0-748b95282da0\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-78bj2" Dec 03 11:37:10 crc kubenswrapper[5052]: I1203 11:37:10.500780 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/c5216bac-5393-4c1e-82e0-748b95282da0-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-84b9f45d47-78bj2\" (UID: \"c5216bac-5393-4c1e-82e0-748b95282da0\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-78bj2" Dec 03 11:37:10 crc kubenswrapper[5052]: I1203 11:37:10.500818 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z855s\" (UniqueName: \"kubernetes.io/projected/c5216bac-5393-4c1e-82e0-748b95282da0-kube-api-access-z855s\") pod \"dnsmasq-dnsmasq-84b9f45d47-78bj2\" (UID: \"c5216bac-5393-4c1e-82e0-748b95282da0\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-78bj2" Dec 03 11:37:10 crc kubenswrapper[5052]: I1203 11:37:10.500871 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-rd7b5\" (UniqueName: \"kubernetes.io/projected/897024db-e95e-4ab8-868f-ddb1ffadcbed-kube-api-access-rd7b5\") pod \"dnsmasq-dnsmasq-f5849d7b9-nrdjn\" (UID: \"897024db-e95e-4ab8-868f-ddb1ffadcbed\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-f5849d7b9-nrdjn" Dec 03 11:37:10 crc kubenswrapper[5052]: I1203 11:37:10.500966 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/897024db-e95e-4ab8-868f-ddb1ffadcbed-config\") pod \"dnsmasq-dnsmasq-f5849d7b9-nrdjn\" (UID: \"897024db-e95e-4ab8-868f-ddb1ffadcbed\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-f5849d7b9-nrdjn" Dec 03 11:37:10 crc kubenswrapper[5052]: I1203 11:37:10.502159 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/897024db-e95e-4ab8-868f-ddb1ffadcbed-config\") pod \"dnsmasq-dnsmasq-f5849d7b9-nrdjn\" (UID: \"897024db-e95e-4ab8-868f-ddb1ffadcbed\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-f5849d7b9-nrdjn" Dec 03 11:37:10 crc kubenswrapper[5052]: I1203 11:37:10.522392 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-rd7b5\" (UniqueName: \"kubernetes.io/projected/897024db-e95e-4ab8-868f-ddb1ffadcbed-kube-api-access-rd7b5\") pod \"dnsmasq-dnsmasq-f5849d7b9-nrdjn\" (UID: \"897024db-e95e-4ab8-868f-ddb1ffadcbed\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-f5849d7b9-nrdjn" Dec 03 11:37:10 crc kubenswrapper[5052]: I1203 11:37:10.601932 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-z855s\" (UniqueName: \"kubernetes.io/projected/c5216bac-5393-4c1e-82e0-748b95282da0-kube-api-access-z855s\") pod \"dnsmasq-dnsmasq-84b9f45d47-78bj2\" (UID: \"c5216bac-5393-4c1e-82e0-748b95282da0\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-78bj2" Dec 03 11:37:10 crc kubenswrapper[5052]: I1203 11:37:10.602402 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c5216bac-5393-4c1e-82e0-748b95282da0-config\") pod \"dnsmasq-dnsmasq-84b9f45d47-78bj2\" (UID: \"c5216bac-5393-4c1e-82e0-748b95282da0\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-78bj2" Dec 03 11:37:10 crc kubenswrapper[5052]: I1203 11:37:10.602459 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/c5216bac-5393-4c1e-82e0-748b95282da0-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-84b9f45d47-78bj2\" (UID: \"c5216bac-5393-4c1e-82e0-748b95282da0\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-78bj2" Dec 03 11:37:10 crc kubenswrapper[5052]: I1203 11:37:10.603255 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/c5216bac-5393-4c1e-82e0-748b95282da0-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-84b9f45d47-78bj2\" (UID: \"c5216bac-5393-4c1e-82e0-748b95282da0\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-78bj2" Dec 03 11:37:10 crc kubenswrapper[5052]: I1203 11:37:10.603315 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c5216bac-5393-4c1e-82e0-748b95282da0-config\") pod \"dnsmasq-dnsmasq-84b9f45d47-78bj2\" (UID: \"c5216bac-5393-4c1e-82e0-748b95282da0\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-78bj2" Dec 03 11:37:10 crc kubenswrapper[5052]: I1203 11:37:10.620237 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-z855s\" (UniqueName: \"kubernetes.io/projected/c5216bac-5393-4c1e-82e0-748b95282da0-kube-api-access-z855s\") pod \"dnsmasq-dnsmasq-84b9f45d47-78bj2\" (UID: \"c5216bac-5393-4c1e-82e0-748b95282da0\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-78bj2" Dec 03 11:37:10 crc kubenswrapper[5052]: I1203 11:37:10.677942 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-f5849d7b9-nrdjn" Dec 03 11:37:10 crc kubenswrapper[5052]: I1203 11:37:10.751255 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-78bj2" Dec 03 11:37:11 crc kubenswrapper[5052]: W1203 11:37:11.086031 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod897024db_e95e_4ab8_868f_ddb1ffadcbed.slice/crio-d9c2ed6f55687734f078b1c884c95198b8314d5c9ebf5654249ad8fe90445f9f WatchSource:0}: Error finding container d9c2ed6f55687734f078b1c884c95198b8314d5c9ebf5654249ad8fe90445f9f: Status 404 returned error can't find the container with id d9c2ed6f55687734f078b1c884c95198b8314d5c9ebf5654249ad8fe90445f9f Dec 03 11:37:11 crc kubenswrapper[5052]: I1203 11:37:11.086762 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-f5849d7b9-nrdjn"] Dec 03 11:37:11 crc kubenswrapper[5052]: I1203 11:37:11.177023 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-78bj2"] Dec 03 11:37:11 crc kubenswrapper[5052]: W1203 11:37:11.183892 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc5216bac_5393_4c1e_82e0_748b95282da0.slice/crio-6e388fdd91614eb21038110ec0e7db48fd4e8c8dc0b9df0f589c57da9cc9add7 WatchSource:0}: Error finding container 6e388fdd91614eb21038110ec0e7db48fd4e8c8dc0b9df0f589c57da9cc9add7: Status 404 returned error can't find the container with id 6e388fdd91614eb21038110ec0e7db48fd4e8c8dc0b9df0f589c57da9cc9add7 Dec 03 11:37:11 crc kubenswrapper[5052]: I1203 11:37:11.837263 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-78bj2" event={"ID":"c5216bac-5393-4c1e-82e0-748b95282da0","Type":"ContainerStarted","Data":"6e388fdd91614eb21038110ec0e7db48fd4e8c8dc0b9df0f589c57da9cc9add7"} Dec 03 11:37:11 crc kubenswrapper[5052]: I1203 11:37:11.838933 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-f5849d7b9-nrdjn" event={"ID":"897024db-e95e-4ab8-868f-ddb1ffadcbed","Type":"ContainerStarted","Data":"d9c2ed6f55687734f078b1c884c95198b8314d5c9ebf5654249ad8fe90445f9f"} Dec 03 11:37:22 crc kubenswrapper[5052]: I1203 11:37:22.086048 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/rabbitmq-server-0"] Dec 03 11:37:22 crc kubenswrapper[5052]: I1203 11:37:22.106811 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/rabbitmq-server-0"] Dec 03 11:37:22 crc kubenswrapper[5052]: I1203 11:37:22.106928 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 11:37:22 crc kubenswrapper[5052]: I1203 11:37:22.111236 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"rabbitmq-config-data" Dec 03 11:37:22 crc kubenswrapper[5052]: I1203 11:37:22.112973 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"rabbitmq-plugins-conf" Dec 03 11:37:22 crc kubenswrapper[5052]: I1203 11:37:22.113322 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"rabbitmq-erlang-cookie" Dec 03 11:37:22 crc kubenswrapper[5052]: I1203 11:37:22.113672 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"rabbitmq-server-conf" Dec 03 11:37:22 crc kubenswrapper[5052]: I1203 11:37:22.113726 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"rabbitmq-default-user" Dec 03 11:37:22 crc kubenswrapper[5052]: I1203 11:37:22.113908 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-rabbitmq-svc" Dec 03 11:37:22 crc kubenswrapper[5052]: I1203 11:37:22.113961 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"rabbitmq-server-dockercfg-7fgq9" Dec 03 11:37:22 crc kubenswrapper[5052]: I1203 11:37:22.214957 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-conf\" (UniqueName: \"kubernetes.io/configmap/9944d9af-7a7b-4773-8279-72914d63ef5f-plugins-conf\") pod \"rabbitmq-server-0\" (UID: \"9944d9af-7a7b-4773-8279-72914d63ef5f\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 11:37:22 crc kubenswrapper[5052]: I1203 11:37:22.215005 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"rabbitmq-plugins\" (UniqueName: \"kubernetes.io/empty-dir/9944d9af-7a7b-4773-8279-72914d63ef5f-rabbitmq-plugins\") pod \"rabbitmq-server-0\" (UID: \"9944d9af-7a7b-4773-8279-72914d63ef5f\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 11:37:22 crc kubenswrapper[5052]: I1203 11:37:22.215025 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"pod-info\" (UniqueName: \"kubernetes.io/downward-api/9944d9af-7a7b-4773-8279-72914d63ef5f-pod-info\") pod \"rabbitmq-server-0\" (UID: \"9944d9af-7a7b-4773-8279-72914d63ef5f\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 11:37:22 crc kubenswrapper[5052]: I1203 11:37:22.215055 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage08-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage08-crc\") pod \"rabbitmq-server-0\" (UID: \"9944d9af-7a7b-4773-8279-72914d63ef5f\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 11:37:22 crc kubenswrapper[5052]: I1203 11:37:22.215137 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"erlang-cookie-secret\" (UniqueName: \"kubernetes.io/secret/9944d9af-7a7b-4773-8279-72914d63ef5f-erlang-cookie-secret\") pod \"rabbitmq-server-0\" (UID: \"9944d9af-7a7b-4773-8279-72914d63ef5f\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 11:37:22 crc kubenswrapper[5052]: I1203 11:37:22.215197 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5qrmz\" (UniqueName: \"kubernetes.io/projected/9944d9af-7a7b-4773-8279-72914d63ef5f-kube-api-access-5qrmz\") pod \"rabbitmq-server-0\" (UID: \"9944d9af-7a7b-4773-8279-72914d63ef5f\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 11:37:22 crc kubenswrapper[5052]: I1203 11:37:22.215251 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"rabbitmq-erlang-cookie\" (UniqueName: \"kubernetes.io/empty-dir/9944d9af-7a7b-4773-8279-72914d63ef5f-rabbitmq-erlang-cookie\") pod \"rabbitmq-server-0\" (UID: \"9944d9af-7a7b-4773-8279-72914d63ef5f\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 11:37:22 crc kubenswrapper[5052]: I1203 11:37:22.215280 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"server-conf\" (UniqueName: \"kubernetes.io/configmap/9944d9af-7a7b-4773-8279-72914d63ef5f-server-conf\") pod \"rabbitmq-server-0\" (UID: \"9944d9af-7a7b-4773-8279-72914d63ef5f\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 11:37:22 crc kubenswrapper[5052]: I1203 11:37:22.215435 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/9944d9af-7a7b-4773-8279-72914d63ef5f-config-data\") pod \"rabbitmq-server-0\" (UID: \"9944d9af-7a7b-4773-8279-72914d63ef5f\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 11:37:22 crc kubenswrapper[5052]: I1203 11:37:22.215492 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"rabbitmq-tls\" (UniqueName: \"kubernetes.io/projected/9944d9af-7a7b-4773-8279-72914d63ef5f-rabbitmq-tls\") pod \"rabbitmq-server-0\" (UID: \"9944d9af-7a7b-4773-8279-72914d63ef5f\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 11:37:22 crc kubenswrapper[5052]: I1203 11:37:22.215583 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"rabbitmq-confd\" (UniqueName: \"kubernetes.io/projected/9944d9af-7a7b-4773-8279-72914d63ef5f-rabbitmq-confd\") pod \"rabbitmq-server-0\" (UID: \"9944d9af-7a7b-4773-8279-72914d63ef5f\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 11:37:22 crc kubenswrapper[5052]: I1203 11:37:22.316600 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"pod-info\" (UniqueName: \"kubernetes.io/downward-api/9944d9af-7a7b-4773-8279-72914d63ef5f-pod-info\") pod \"rabbitmq-server-0\" (UID: \"9944d9af-7a7b-4773-8279-72914d63ef5f\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 11:37:22 crc kubenswrapper[5052]: I1203 11:37:22.316665 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage08-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage08-crc\") pod \"rabbitmq-server-0\" (UID: \"9944d9af-7a7b-4773-8279-72914d63ef5f\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 11:37:22 crc kubenswrapper[5052]: I1203 11:37:22.316697 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"erlang-cookie-secret\" (UniqueName: \"kubernetes.io/secret/9944d9af-7a7b-4773-8279-72914d63ef5f-erlang-cookie-secret\") pod \"rabbitmq-server-0\" (UID: \"9944d9af-7a7b-4773-8279-72914d63ef5f\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 11:37:22 crc kubenswrapper[5052]: I1203 11:37:22.316724 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-5qrmz\" (UniqueName: \"kubernetes.io/projected/9944d9af-7a7b-4773-8279-72914d63ef5f-kube-api-access-5qrmz\") pod \"rabbitmq-server-0\" (UID: \"9944d9af-7a7b-4773-8279-72914d63ef5f\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 11:37:22 crc kubenswrapper[5052]: I1203 11:37:22.316770 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"rabbitmq-erlang-cookie\" (UniqueName: \"kubernetes.io/empty-dir/9944d9af-7a7b-4773-8279-72914d63ef5f-rabbitmq-erlang-cookie\") pod \"rabbitmq-server-0\" (UID: \"9944d9af-7a7b-4773-8279-72914d63ef5f\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 11:37:22 crc kubenswrapper[5052]: I1203 11:37:22.316798 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"server-conf\" (UniqueName: \"kubernetes.io/configmap/9944d9af-7a7b-4773-8279-72914d63ef5f-server-conf\") pod \"rabbitmq-server-0\" (UID: \"9944d9af-7a7b-4773-8279-72914d63ef5f\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 11:37:22 crc kubenswrapper[5052]: I1203 11:37:22.316845 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/9944d9af-7a7b-4773-8279-72914d63ef5f-config-data\") pod \"rabbitmq-server-0\" (UID: \"9944d9af-7a7b-4773-8279-72914d63ef5f\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 11:37:22 crc kubenswrapper[5052]: I1203 11:37:22.316871 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"rabbitmq-tls\" (UniqueName: \"kubernetes.io/projected/9944d9af-7a7b-4773-8279-72914d63ef5f-rabbitmq-tls\") pod \"rabbitmq-server-0\" (UID: \"9944d9af-7a7b-4773-8279-72914d63ef5f\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 11:37:22 crc kubenswrapper[5052]: I1203 11:37:22.316902 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"rabbitmq-confd\" (UniqueName: \"kubernetes.io/projected/9944d9af-7a7b-4773-8279-72914d63ef5f-rabbitmq-confd\") pod \"rabbitmq-server-0\" (UID: \"9944d9af-7a7b-4773-8279-72914d63ef5f\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 11:37:22 crc kubenswrapper[5052]: I1203 11:37:22.316922 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"plugins-conf\" (UniqueName: \"kubernetes.io/configmap/9944d9af-7a7b-4773-8279-72914d63ef5f-plugins-conf\") pod \"rabbitmq-server-0\" (UID: \"9944d9af-7a7b-4773-8279-72914d63ef5f\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 11:37:22 crc kubenswrapper[5052]: I1203 11:37:22.316944 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"rabbitmq-plugins\" (UniqueName: \"kubernetes.io/empty-dir/9944d9af-7a7b-4773-8279-72914d63ef5f-rabbitmq-plugins\") pod \"rabbitmq-server-0\" (UID: \"9944d9af-7a7b-4773-8279-72914d63ef5f\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 11:37:22 crc kubenswrapper[5052]: I1203 11:37:22.317125 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage08-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage08-crc\") pod \"rabbitmq-server-0\" (UID: \"9944d9af-7a7b-4773-8279-72914d63ef5f\") device mount path \"/mnt/openstack/pv08\"" pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 11:37:22 crc kubenswrapper[5052]: I1203 11:37:22.317490 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"rabbitmq-plugins\" (UniqueName: \"kubernetes.io/empty-dir/9944d9af-7a7b-4773-8279-72914d63ef5f-rabbitmq-plugins\") pod \"rabbitmq-server-0\" (UID: \"9944d9af-7a7b-4773-8279-72914d63ef5f\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 11:37:22 crc kubenswrapper[5052]: I1203 11:37:22.318645 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/9944d9af-7a7b-4773-8279-72914d63ef5f-config-data\") pod \"rabbitmq-server-0\" (UID: \"9944d9af-7a7b-4773-8279-72914d63ef5f\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 11:37:22 crc kubenswrapper[5052]: I1203 11:37:22.318955 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"rabbitmq-erlang-cookie\" (UniqueName: \"kubernetes.io/empty-dir/9944d9af-7a7b-4773-8279-72914d63ef5f-rabbitmq-erlang-cookie\") pod \"rabbitmq-server-0\" (UID: \"9944d9af-7a7b-4773-8279-72914d63ef5f\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 11:37:22 crc kubenswrapper[5052]: I1203 11:37:22.319808 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"server-conf\" (UniqueName: \"kubernetes.io/configmap/9944d9af-7a7b-4773-8279-72914d63ef5f-server-conf\") pod \"rabbitmq-server-0\" (UID: \"9944d9af-7a7b-4773-8279-72914d63ef5f\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 11:37:22 crc kubenswrapper[5052]: I1203 11:37:22.322536 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"pod-info\" (UniqueName: \"kubernetes.io/downward-api/9944d9af-7a7b-4773-8279-72914d63ef5f-pod-info\") pod \"rabbitmq-server-0\" (UID: \"9944d9af-7a7b-4773-8279-72914d63ef5f\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 11:37:22 crc kubenswrapper[5052]: I1203 11:37:22.322777 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"rabbitmq-confd\" (UniqueName: \"kubernetes.io/projected/9944d9af-7a7b-4773-8279-72914d63ef5f-rabbitmq-confd\") pod \"rabbitmq-server-0\" (UID: \"9944d9af-7a7b-4773-8279-72914d63ef5f\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 11:37:22 crc kubenswrapper[5052]: I1203 11:37:22.326912 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"erlang-cookie-secret\" (UniqueName: \"kubernetes.io/secret/9944d9af-7a7b-4773-8279-72914d63ef5f-erlang-cookie-secret\") pod \"rabbitmq-server-0\" (UID: \"9944d9af-7a7b-4773-8279-72914d63ef5f\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 11:37:22 crc kubenswrapper[5052]: I1203 11:37:22.327930 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"plugins-conf\" (UniqueName: \"kubernetes.io/configmap/9944d9af-7a7b-4773-8279-72914d63ef5f-plugins-conf\") pod \"rabbitmq-server-0\" (UID: \"9944d9af-7a7b-4773-8279-72914d63ef5f\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 11:37:22 crc kubenswrapper[5052]: I1203 11:37:22.332816 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"rabbitmq-tls\" (UniqueName: \"kubernetes.io/projected/9944d9af-7a7b-4773-8279-72914d63ef5f-rabbitmq-tls\") pod \"rabbitmq-server-0\" (UID: \"9944d9af-7a7b-4773-8279-72914d63ef5f\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 11:37:22 crc kubenswrapper[5052]: I1203 11:37:22.340843 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-5qrmz\" (UniqueName: \"kubernetes.io/projected/9944d9af-7a7b-4773-8279-72914d63ef5f-kube-api-access-5qrmz\") pod \"rabbitmq-server-0\" (UID: \"9944d9af-7a7b-4773-8279-72914d63ef5f\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 11:37:22 crc kubenswrapper[5052]: I1203 11:37:22.352041 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage08-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage08-crc\") pod \"rabbitmq-server-0\" (UID: \"9944d9af-7a7b-4773-8279-72914d63ef5f\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 11:37:22 crc kubenswrapper[5052]: I1203 11:37:22.354795 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/rabbitmq-cell1-server-0"] Dec 03 11:37:22 crc kubenswrapper[5052]: I1203 11:37:22.355928 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 11:37:22 crc kubenswrapper[5052]: I1203 11:37:22.358740 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"rabbitmq-cell1-config-data" Dec 03 11:37:22 crc kubenswrapper[5052]: I1203 11:37:22.359160 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"rabbitmq-cell1-default-user" Dec 03 11:37:22 crc kubenswrapper[5052]: I1203 11:37:22.359316 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"rabbitmq-cell1-erlang-cookie" Dec 03 11:37:22 crc kubenswrapper[5052]: I1203 11:37:22.359529 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"rabbitmq-cell1-server-conf" Dec 03 11:37:22 crc kubenswrapper[5052]: I1203 11:37:22.359702 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-rabbitmq-cell1-svc" Dec 03 11:37:22 crc kubenswrapper[5052]: I1203 11:37:22.359863 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"rabbitmq-cell1-plugins-conf" Dec 03 11:37:22 crc kubenswrapper[5052]: I1203 11:37:22.365503 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"rabbitmq-cell1-server-dockercfg-qhnht" Dec 03 11:37:22 crc kubenswrapper[5052]: I1203 11:37:22.380281 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/rabbitmq-cell1-server-0"] Dec 03 11:37:22 crc kubenswrapper[5052]: I1203 11:37:22.431933 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 11:37:22 crc kubenswrapper[5052]: I1203 11:37:22.520359 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"server-conf\" (UniqueName: \"kubernetes.io/configmap/daa521fc-7e1d-44d2-b7d8-6aab003cfb66-server-conf\") pod \"rabbitmq-cell1-server-0\" (UID: \"daa521fc-7e1d-44d2-b7d8-6aab003cfb66\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 11:37:22 crc kubenswrapper[5052]: I1203 11:37:22.520431 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"rabbitmq-plugins\" (UniqueName: \"kubernetes.io/empty-dir/daa521fc-7e1d-44d2-b7d8-6aab003cfb66-rabbitmq-plugins\") pod \"rabbitmq-cell1-server-0\" (UID: \"daa521fc-7e1d-44d2-b7d8-6aab003cfb66\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 11:37:22 crc kubenswrapper[5052]: I1203 11:37:22.520661 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage09-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage09-crc\") pod \"rabbitmq-cell1-server-0\" (UID: \"daa521fc-7e1d-44d2-b7d8-6aab003cfb66\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 11:37:22 crc kubenswrapper[5052]: I1203 11:37:22.520682 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/daa521fc-7e1d-44d2-b7d8-6aab003cfb66-config-data\") pod \"rabbitmq-cell1-server-0\" (UID: \"daa521fc-7e1d-44d2-b7d8-6aab003cfb66\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 11:37:22 crc kubenswrapper[5052]: I1203 11:37:22.520714 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"pod-info\" (UniqueName: \"kubernetes.io/downward-api/daa521fc-7e1d-44d2-b7d8-6aab003cfb66-pod-info\") pod \"rabbitmq-cell1-server-0\" (UID: \"daa521fc-7e1d-44d2-b7d8-6aab003cfb66\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 11:37:22 crc kubenswrapper[5052]: I1203 11:37:22.520736 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"rabbitmq-tls\" (UniqueName: \"kubernetes.io/projected/daa521fc-7e1d-44d2-b7d8-6aab003cfb66-rabbitmq-tls\") pod \"rabbitmq-cell1-server-0\" (UID: \"daa521fc-7e1d-44d2-b7d8-6aab003cfb66\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 11:37:22 crc kubenswrapper[5052]: I1203 11:37:22.520778 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"erlang-cookie-secret\" (UniqueName: \"kubernetes.io/secret/daa521fc-7e1d-44d2-b7d8-6aab003cfb66-erlang-cookie-secret\") pod \"rabbitmq-cell1-server-0\" (UID: \"daa521fc-7e1d-44d2-b7d8-6aab003cfb66\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 11:37:22 crc kubenswrapper[5052]: I1203 11:37:22.520813 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-conf\" (UniqueName: \"kubernetes.io/configmap/daa521fc-7e1d-44d2-b7d8-6aab003cfb66-plugins-conf\") pod \"rabbitmq-cell1-server-0\" (UID: \"daa521fc-7e1d-44d2-b7d8-6aab003cfb66\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 11:37:22 crc kubenswrapper[5052]: I1203 11:37:22.520894 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"rabbitmq-confd\" (UniqueName: \"kubernetes.io/projected/daa521fc-7e1d-44d2-b7d8-6aab003cfb66-rabbitmq-confd\") pod \"rabbitmq-cell1-server-0\" (UID: \"daa521fc-7e1d-44d2-b7d8-6aab003cfb66\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 11:37:22 crc kubenswrapper[5052]: I1203 11:37:22.520942 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-st9gb\" (UniqueName: \"kubernetes.io/projected/daa521fc-7e1d-44d2-b7d8-6aab003cfb66-kube-api-access-st9gb\") pod \"rabbitmq-cell1-server-0\" (UID: \"daa521fc-7e1d-44d2-b7d8-6aab003cfb66\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 11:37:22 crc kubenswrapper[5052]: I1203 11:37:22.520975 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"rabbitmq-erlang-cookie\" (UniqueName: \"kubernetes.io/empty-dir/daa521fc-7e1d-44d2-b7d8-6aab003cfb66-rabbitmq-erlang-cookie\") pod \"rabbitmq-cell1-server-0\" (UID: \"daa521fc-7e1d-44d2-b7d8-6aab003cfb66\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 11:37:22 crc kubenswrapper[5052]: I1203 11:37:22.622703 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"rabbitmq-plugins\" (UniqueName: \"kubernetes.io/empty-dir/daa521fc-7e1d-44d2-b7d8-6aab003cfb66-rabbitmq-plugins\") pod \"rabbitmq-cell1-server-0\" (UID: \"daa521fc-7e1d-44d2-b7d8-6aab003cfb66\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 11:37:22 crc kubenswrapper[5052]: I1203 11:37:22.622758 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/daa521fc-7e1d-44d2-b7d8-6aab003cfb66-config-data\") pod \"rabbitmq-cell1-server-0\" (UID: \"daa521fc-7e1d-44d2-b7d8-6aab003cfb66\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 11:37:22 crc kubenswrapper[5052]: I1203 11:37:22.622783 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage09-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage09-crc\") pod \"rabbitmq-cell1-server-0\" (UID: \"daa521fc-7e1d-44d2-b7d8-6aab003cfb66\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 11:37:22 crc kubenswrapper[5052]: I1203 11:37:22.622812 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"pod-info\" (UniqueName: \"kubernetes.io/downward-api/daa521fc-7e1d-44d2-b7d8-6aab003cfb66-pod-info\") pod \"rabbitmq-cell1-server-0\" (UID: \"daa521fc-7e1d-44d2-b7d8-6aab003cfb66\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 11:37:22 crc kubenswrapper[5052]: I1203 11:37:22.622839 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"rabbitmq-tls\" (UniqueName: \"kubernetes.io/projected/daa521fc-7e1d-44d2-b7d8-6aab003cfb66-rabbitmq-tls\") pod \"rabbitmq-cell1-server-0\" (UID: \"daa521fc-7e1d-44d2-b7d8-6aab003cfb66\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 11:37:22 crc kubenswrapper[5052]: I1203 11:37:22.622881 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"erlang-cookie-secret\" (UniqueName: \"kubernetes.io/secret/daa521fc-7e1d-44d2-b7d8-6aab003cfb66-erlang-cookie-secret\") pod \"rabbitmq-cell1-server-0\" (UID: \"daa521fc-7e1d-44d2-b7d8-6aab003cfb66\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 11:37:22 crc kubenswrapper[5052]: I1203 11:37:22.622914 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"plugins-conf\" (UniqueName: \"kubernetes.io/configmap/daa521fc-7e1d-44d2-b7d8-6aab003cfb66-plugins-conf\") pod \"rabbitmq-cell1-server-0\" (UID: \"daa521fc-7e1d-44d2-b7d8-6aab003cfb66\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 11:37:22 crc kubenswrapper[5052]: I1203 11:37:22.622945 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"rabbitmq-confd\" (UniqueName: \"kubernetes.io/projected/daa521fc-7e1d-44d2-b7d8-6aab003cfb66-rabbitmq-confd\") pod \"rabbitmq-cell1-server-0\" (UID: \"daa521fc-7e1d-44d2-b7d8-6aab003cfb66\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 11:37:22 crc kubenswrapper[5052]: I1203 11:37:22.622980 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-st9gb\" (UniqueName: \"kubernetes.io/projected/daa521fc-7e1d-44d2-b7d8-6aab003cfb66-kube-api-access-st9gb\") pod \"rabbitmq-cell1-server-0\" (UID: \"daa521fc-7e1d-44d2-b7d8-6aab003cfb66\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 11:37:22 crc kubenswrapper[5052]: I1203 11:37:22.623013 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"rabbitmq-erlang-cookie\" (UniqueName: \"kubernetes.io/empty-dir/daa521fc-7e1d-44d2-b7d8-6aab003cfb66-rabbitmq-erlang-cookie\") pod \"rabbitmq-cell1-server-0\" (UID: \"daa521fc-7e1d-44d2-b7d8-6aab003cfb66\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 11:37:22 crc kubenswrapper[5052]: I1203 11:37:22.623042 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"server-conf\" (UniqueName: \"kubernetes.io/configmap/daa521fc-7e1d-44d2-b7d8-6aab003cfb66-server-conf\") pod \"rabbitmq-cell1-server-0\" (UID: \"daa521fc-7e1d-44d2-b7d8-6aab003cfb66\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 11:37:22 crc kubenswrapper[5052]: I1203 11:37:22.623409 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"rabbitmq-plugins\" (UniqueName: \"kubernetes.io/empty-dir/daa521fc-7e1d-44d2-b7d8-6aab003cfb66-rabbitmq-plugins\") pod \"rabbitmq-cell1-server-0\" (UID: \"daa521fc-7e1d-44d2-b7d8-6aab003cfb66\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 11:37:22 crc kubenswrapper[5052]: I1203 11:37:22.624213 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage09-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage09-crc\") pod \"rabbitmq-cell1-server-0\" (UID: \"daa521fc-7e1d-44d2-b7d8-6aab003cfb66\") device mount path \"/mnt/openstack/pv09\"" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 11:37:22 crc kubenswrapper[5052]: I1203 11:37:22.624534 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"rabbitmq-erlang-cookie\" (UniqueName: \"kubernetes.io/empty-dir/daa521fc-7e1d-44d2-b7d8-6aab003cfb66-rabbitmq-erlang-cookie\") pod \"rabbitmq-cell1-server-0\" (UID: \"daa521fc-7e1d-44d2-b7d8-6aab003cfb66\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 11:37:22 crc kubenswrapper[5052]: I1203 11:37:22.625360 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"server-conf\" (UniqueName: \"kubernetes.io/configmap/daa521fc-7e1d-44d2-b7d8-6aab003cfb66-server-conf\") pod \"rabbitmq-cell1-server-0\" (UID: \"daa521fc-7e1d-44d2-b7d8-6aab003cfb66\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 11:37:22 crc kubenswrapper[5052]: I1203 11:37:22.625610 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"plugins-conf\" (UniqueName: \"kubernetes.io/configmap/daa521fc-7e1d-44d2-b7d8-6aab003cfb66-plugins-conf\") pod \"rabbitmq-cell1-server-0\" (UID: \"daa521fc-7e1d-44d2-b7d8-6aab003cfb66\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 11:37:22 crc kubenswrapper[5052]: I1203 11:37:22.628126 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/daa521fc-7e1d-44d2-b7d8-6aab003cfb66-config-data\") pod \"rabbitmq-cell1-server-0\" (UID: \"daa521fc-7e1d-44d2-b7d8-6aab003cfb66\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 11:37:22 crc kubenswrapper[5052]: I1203 11:37:22.629024 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"erlang-cookie-secret\" (UniqueName: \"kubernetes.io/secret/daa521fc-7e1d-44d2-b7d8-6aab003cfb66-erlang-cookie-secret\") pod \"rabbitmq-cell1-server-0\" (UID: \"daa521fc-7e1d-44d2-b7d8-6aab003cfb66\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 11:37:22 crc kubenswrapper[5052]: I1203 11:37:22.645244 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"rabbitmq-confd\" (UniqueName: \"kubernetes.io/projected/daa521fc-7e1d-44d2-b7d8-6aab003cfb66-rabbitmq-confd\") pod \"rabbitmq-cell1-server-0\" (UID: \"daa521fc-7e1d-44d2-b7d8-6aab003cfb66\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 11:37:22 crc kubenswrapper[5052]: I1203 11:37:22.653813 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"pod-info\" (UniqueName: \"kubernetes.io/downward-api/daa521fc-7e1d-44d2-b7d8-6aab003cfb66-pod-info\") pod \"rabbitmq-cell1-server-0\" (UID: \"daa521fc-7e1d-44d2-b7d8-6aab003cfb66\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 11:37:22 crc kubenswrapper[5052]: I1203 11:37:22.662209 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"rabbitmq-tls\" (UniqueName: \"kubernetes.io/projected/daa521fc-7e1d-44d2-b7d8-6aab003cfb66-rabbitmq-tls\") pod \"rabbitmq-cell1-server-0\" (UID: \"daa521fc-7e1d-44d2-b7d8-6aab003cfb66\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 11:37:22 crc kubenswrapper[5052]: I1203 11:37:22.670207 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-st9gb\" (UniqueName: \"kubernetes.io/projected/daa521fc-7e1d-44d2-b7d8-6aab003cfb66-kube-api-access-st9gb\") pod \"rabbitmq-cell1-server-0\" (UID: \"daa521fc-7e1d-44d2-b7d8-6aab003cfb66\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 11:37:22 crc kubenswrapper[5052]: I1203 11:37:22.679084 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage09-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage09-crc\") pod \"rabbitmq-cell1-server-0\" (UID: \"daa521fc-7e1d-44d2-b7d8-6aab003cfb66\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 11:37:22 crc kubenswrapper[5052]: I1203 11:37:22.775871 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 11:37:23 crc kubenswrapper[5052]: I1203 11:37:23.720084 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/openstack-galera-0"] Dec 03 11:37:23 crc kubenswrapper[5052]: I1203 11:37:23.722244 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 11:37:23 crc kubenswrapper[5052]: I1203 11:37:23.730586 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"openstack-config-data" Dec 03 11:37:23 crc kubenswrapper[5052]: I1203 11:37:23.730868 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"openstack-scripts" Dec 03 11:37:23 crc kubenswrapper[5052]: I1203 11:37:23.730950 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"galera-openstack-dockercfg-dfbmj" Dec 03 11:37:23 crc kubenswrapper[5052]: I1203 11:37:23.736918 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-galera-openstack-svc" Dec 03 11:37:23 crc kubenswrapper[5052]: I1203 11:37:23.744662 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"combined-ca-bundle" Dec 03 11:37:23 crc kubenswrapper[5052]: I1203 11:37:23.762469 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/openstack-galera-0"] Dec 03 11:37:23 crc kubenswrapper[5052]: I1203 11:37:23.845892 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-scx8f\" (UniqueName: \"kubernetes.io/projected/251248ba-578c-49cf-b77e-fc3d5c6cb6fd-kube-api-access-scx8f\") pod \"openstack-galera-0\" (UID: \"251248ba-578c-49cf-b77e-fc3d5c6cb6fd\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 11:37:23 crc kubenswrapper[5052]: I1203 11:37:23.846014 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/251248ba-578c-49cf-b77e-fc3d5c6cb6fd-combined-ca-bundle\") pod \"openstack-galera-0\" (UID: \"251248ba-578c-49cf-b77e-fc3d5c6cb6fd\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 11:37:23 crc kubenswrapper[5052]: I1203 11:37:23.846066 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage20-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage20-crc\") pod \"openstack-galera-0\" (UID: \"251248ba-578c-49cf-b77e-fc3d5c6cb6fd\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 11:37:23 crc kubenswrapper[5052]: I1203 11:37:23.846098 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/251248ba-578c-49cf-b77e-fc3d5c6cb6fd-config-data-generated\") pod \"openstack-galera-0\" (UID: \"251248ba-578c-49cf-b77e-fc3d5c6cb6fd\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 11:37:23 crc kubenswrapper[5052]: I1203 11:37:23.846122 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/251248ba-578c-49cf-b77e-fc3d5c6cb6fd-galera-tls-certs\") pod \"openstack-galera-0\" (UID: \"251248ba-578c-49cf-b77e-fc3d5c6cb6fd\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 11:37:23 crc kubenswrapper[5052]: I1203 11:37:23.846152 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/251248ba-578c-49cf-b77e-fc3d5c6cb6fd-kolla-config\") pod \"openstack-galera-0\" (UID: \"251248ba-578c-49cf-b77e-fc3d5c6cb6fd\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 11:37:23 crc kubenswrapper[5052]: I1203 11:37:23.846178 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/251248ba-578c-49cf-b77e-fc3d5c6cb6fd-config-data-default\") pod \"openstack-galera-0\" (UID: \"251248ba-578c-49cf-b77e-fc3d5c6cb6fd\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 11:37:23 crc kubenswrapper[5052]: I1203 11:37:23.846205 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/251248ba-578c-49cf-b77e-fc3d5c6cb6fd-operator-scripts\") pod \"openstack-galera-0\" (UID: \"251248ba-578c-49cf-b77e-fc3d5c6cb6fd\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 11:37:23 crc kubenswrapper[5052]: I1203 11:37:23.947288 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/251248ba-578c-49cf-b77e-fc3d5c6cb6fd-combined-ca-bundle\") pod \"openstack-galera-0\" (UID: \"251248ba-578c-49cf-b77e-fc3d5c6cb6fd\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 11:37:23 crc kubenswrapper[5052]: I1203 11:37:23.947370 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage20-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage20-crc\") pod \"openstack-galera-0\" (UID: \"251248ba-578c-49cf-b77e-fc3d5c6cb6fd\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 11:37:23 crc kubenswrapper[5052]: I1203 11:37:23.947403 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/251248ba-578c-49cf-b77e-fc3d5c6cb6fd-config-data-generated\") pod \"openstack-galera-0\" (UID: \"251248ba-578c-49cf-b77e-fc3d5c6cb6fd\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 11:37:23 crc kubenswrapper[5052]: I1203 11:37:23.947439 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/251248ba-578c-49cf-b77e-fc3d5c6cb6fd-galera-tls-certs\") pod \"openstack-galera-0\" (UID: \"251248ba-578c-49cf-b77e-fc3d5c6cb6fd\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 11:37:23 crc kubenswrapper[5052]: I1203 11:37:23.947474 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/251248ba-578c-49cf-b77e-fc3d5c6cb6fd-kolla-config\") pod \"openstack-galera-0\" (UID: \"251248ba-578c-49cf-b77e-fc3d5c6cb6fd\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 11:37:23 crc kubenswrapper[5052]: I1203 11:37:23.947498 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/251248ba-578c-49cf-b77e-fc3d5c6cb6fd-config-data-default\") pod \"openstack-galera-0\" (UID: \"251248ba-578c-49cf-b77e-fc3d5c6cb6fd\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 11:37:23 crc kubenswrapper[5052]: I1203 11:37:23.947544 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/251248ba-578c-49cf-b77e-fc3d5c6cb6fd-operator-scripts\") pod \"openstack-galera-0\" (UID: \"251248ba-578c-49cf-b77e-fc3d5c6cb6fd\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 11:37:23 crc kubenswrapper[5052]: I1203 11:37:23.947578 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-scx8f\" (UniqueName: \"kubernetes.io/projected/251248ba-578c-49cf-b77e-fc3d5c6cb6fd-kube-api-access-scx8f\") pod \"openstack-galera-0\" (UID: \"251248ba-578c-49cf-b77e-fc3d5c6cb6fd\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 11:37:23 crc kubenswrapper[5052]: I1203 11:37:23.950378 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage20-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage20-crc\") pod \"openstack-galera-0\" (UID: \"251248ba-578c-49cf-b77e-fc3d5c6cb6fd\") device mount path \"/mnt/openstack/pv20\"" pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 11:37:23 crc kubenswrapper[5052]: I1203 11:37:23.951776 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/251248ba-578c-49cf-b77e-fc3d5c6cb6fd-combined-ca-bundle\") pod \"openstack-galera-0\" (UID: \"251248ba-578c-49cf-b77e-fc3d5c6cb6fd\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 11:37:23 crc kubenswrapper[5052]: I1203 11:37:23.952332 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/251248ba-578c-49cf-b77e-fc3d5c6cb6fd-kolla-config\") pod \"openstack-galera-0\" (UID: \"251248ba-578c-49cf-b77e-fc3d5c6cb6fd\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 11:37:23 crc kubenswrapper[5052]: I1203 11:37:23.953031 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/251248ba-578c-49cf-b77e-fc3d5c6cb6fd-config-data-default\") pod \"openstack-galera-0\" (UID: \"251248ba-578c-49cf-b77e-fc3d5c6cb6fd\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 11:37:23 crc kubenswrapper[5052]: I1203 11:37:23.954641 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/251248ba-578c-49cf-b77e-fc3d5c6cb6fd-operator-scripts\") pod \"openstack-galera-0\" (UID: \"251248ba-578c-49cf-b77e-fc3d5c6cb6fd\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 11:37:23 crc kubenswrapper[5052]: I1203 11:37:23.954896 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/251248ba-578c-49cf-b77e-fc3d5c6cb6fd-config-data-generated\") pod \"openstack-galera-0\" (UID: \"251248ba-578c-49cf-b77e-fc3d5c6cb6fd\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 11:37:23 crc kubenswrapper[5052]: I1203 11:37:23.959600 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/251248ba-578c-49cf-b77e-fc3d5c6cb6fd-galera-tls-certs\") pod \"openstack-galera-0\" (UID: \"251248ba-578c-49cf-b77e-fc3d5c6cb6fd\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 11:37:23 crc kubenswrapper[5052]: I1203 11:37:23.971300 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-scx8f\" (UniqueName: \"kubernetes.io/projected/251248ba-578c-49cf-b77e-fc3d5c6cb6fd-kube-api-access-scx8f\") pod \"openstack-galera-0\" (UID: \"251248ba-578c-49cf-b77e-fc3d5c6cb6fd\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 11:37:23 crc kubenswrapper[5052]: I1203 11:37:23.985405 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage20-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage20-crc\") pod \"openstack-galera-0\" (UID: \"251248ba-578c-49cf-b77e-fc3d5c6cb6fd\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 11:37:24 crc kubenswrapper[5052]: I1203 11:37:24.056046 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 11:37:24 crc kubenswrapper[5052]: I1203 11:37:24.871621 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/openstack-cell1-galera-0"] Dec 03 11:37:24 crc kubenswrapper[5052]: I1203 11:37:24.872928 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 11:37:24 crc kubenswrapper[5052]: I1203 11:37:24.876203 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-galera-openstack-cell1-svc" Dec 03 11:37:24 crc kubenswrapper[5052]: I1203 11:37:24.876631 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"openstack-cell1-config-data" Dec 03 11:37:24 crc kubenswrapper[5052]: I1203 11:37:24.877045 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"openstack-cell1-scripts" Dec 03 11:37:24 crc kubenswrapper[5052]: I1203 11:37:24.877160 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"galera-openstack-cell1-dockercfg-fq4wh" Dec 03 11:37:24 crc kubenswrapper[5052]: I1203 11:37:24.888465 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/openstack-cell1-galera-0"] Dec 03 11:37:24 crc kubenswrapper[5052]: I1203 11:37:24.963665 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k49vf\" (UniqueName: \"kubernetes.io/projected/ecbf8b8e-cdb4-45c3-9455-d0dbc551356e-kube-api-access-k49vf\") pod \"openstack-cell1-galera-0\" (UID: \"ecbf8b8e-cdb4-45c3-9455-d0dbc551356e\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 11:37:24 crc kubenswrapper[5052]: I1203 11:37:24.963760 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/ecbf8b8e-cdb4-45c3-9455-d0dbc551356e-kolla-config\") pod \"openstack-cell1-galera-0\" (UID: \"ecbf8b8e-cdb4-45c3-9455-d0dbc551356e\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 11:37:24 crc kubenswrapper[5052]: I1203 11:37:24.963785 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/ecbf8b8e-cdb4-45c3-9455-d0dbc551356e-galera-tls-certs\") pod \"openstack-cell1-galera-0\" (UID: \"ecbf8b8e-cdb4-45c3-9455-d0dbc551356e\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 11:37:24 crc kubenswrapper[5052]: I1203 11:37:24.963807 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage12-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage12-crc\") pod \"openstack-cell1-galera-0\" (UID: \"ecbf8b8e-cdb4-45c3-9455-d0dbc551356e\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 11:37:24 crc kubenswrapper[5052]: I1203 11:37:24.963856 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ecbf8b8e-cdb4-45c3-9455-d0dbc551356e-combined-ca-bundle\") pod \"openstack-cell1-galera-0\" (UID: \"ecbf8b8e-cdb4-45c3-9455-d0dbc551356e\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 11:37:24 crc kubenswrapper[5052]: I1203 11:37:24.963901 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/ecbf8b8e-cdb4-45c3-9455-d0dbc551356e-operator-scripts\") pod \"openstack-cell1-galera-0\" (UID: \"ecbf8b8e-cdb4-45c3-9455-d0dbc551356e\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 11:37:24 crc kubenswrapper[5052]: I1203 11:37:24.963942 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/ecbf8b8e-cdb4-45c3-9455-d0dbc551356e-config-data-generated\") pod \"openstack-cell1-galera-0\" (UID: \"ecbf8b8e-cdb4-45c3-9455-d0dbc551356e\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 11:37:24 crc kubenswrapper[5052]: I1203 11:37:24.963968 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/ecbf8b8e-cdb4-45c3-9455-d0dbc551356e-config-data-default\") pod \"openstack-cell1-galera-0\" (UID: \"ecbf8b8e-cdb4-45c3-9455-d0dbc551356e\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 11:37:25 crc kubenswrapper[5052]: I1203 11:37:25.045463 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/memcached-0"] Dec 03 11:37:25 crc kubenswrapper[5052]: I1203 11:37:25.046990 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/memcached-0" Dec 03 11:37:25 crc kubenswrapper[5052]: I1203 11:37:25.049858 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-memcached-svc" Dec 03 11:37:25 crc kubenswrapper[5052]: I1203 11:37:25.050139 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"memcached-config-data" Dec 03 11:37:25 crc kubenswrapper[5052]: I1203 11:37:25.057363 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/memcached-0"] Dec 03 11:37:25 crc kubenswrapper[5052]: I1203 11:37:25.060815 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"memcached-memcached-dockercfg-jcxhp" Dec 03 11:37:25 crc kubenswrapper[5052]: I1203 11:37:25.064997 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ecbf8b8e-cdb4-45c3-9455-d0dbc551356e-combined-ca-bundle\") pod \"openstack-cell1-galera-0\" (UID: \"ecbf8b8e-cdb4-45c3-9455-d0dbc551356e\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 11:37:25 crc kubenswrapper[5052]: I1203 11:37:25.065048 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/ecbf8b8e-cdb4-45c3-9455-d0dbc551356e-operator-scripts\") pod \"openstack-cell1-galera-0\" (UID: \"ecbf8b8e-cdb4-45c3-9455-d0dbc551356e\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 11:37:25 crc kubenswrapper[5052]: I1203 11:37:25.065098 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/ecbf8b8e-cdb4-45c3-9455-d0dbc551356e-config-data-generated\") pod \"openstack-cell1-galera-0\" (UID: \"ecbf8b8e-cdb4-45c3-9455-d0dbc551356e\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 11:37:25 crc kubenswrapper[5052]: I1203 11:37:25.065121 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/ecbf8b8e-cdb4-45c3-9455-d0dbc551356e-config-data-default\") pod \"openstack-cell1-galera-0\" (UID: \"ecbf8b8e-cdb4-45c3-9455-d0dbc551356e\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 11:37:25 crc kubenswrapper[5052]: I1203 11:37:25.065169 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-k49vf\" (UniqueName: \"kubernetes.io/projected/ecbf8b8e-cdb4-45c3-9455-d0dbc551356e-kube-api-access-k49vf\") pod \"openstack-cell1-galera-0\" (UID: \"ecbf8b8e-cdb4-45c3-9455-d0dbc551356e\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 11:37:25 crc kubenswrapper[5052]: I1203 11:37:25.065231 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/ecbf8b8e-cdb4-45c3-9455-d0dbc551356e-kolla-config\") pod \"openstack-cell1-galera-0\" (UID: \"ecbf8b8e-cdb4-45c3-9455-d0dbc551356e\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 11:37:25 crc kubenswrapper[5052]: I1203 11:37:25.065255 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/ecbf8b8e-cdb4-45c3-9455-d0dbc551356e-galera-tls-certs\") pod \"openstack-cell1-galera-0\" (UID: \"ecbf8b8e-cdb4-45c3-9455-d0dbc551356e\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 11:37:25 crc kubenswrapper[5052]: I1203 11:37:25.065277 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage12-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage12-crc\") pod \"openstack-cell1-galera-0\" (UID: \"ecbf8b8e-cdb4-45c3-9455-d0dbc551356e\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 11:37:25 crc kubenswrapper[5052]: I1203 11:37:25.065474 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage12-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage12-crc\") pod \"openstack-cell1-galera-0\" (UID: \"ecbf8b8e-cdb4-45c3-9455-d0dbc551356e\") device mount path \"/mnt/openstack/pv12\"" pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 11:37:25 crc kubenswrapper[5052]: I1203 11:37:25.066910 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/ecbf8b8e-cdb4-45c3-9455-d0dbc551356e-operator-scripts\") pod \"openstack-cell1-galera-0\" (UID: \"ecbf8b8e-cdb4-45c3-9455-d0dbc551356e\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 11:37:25 crc kubenswrapper[5052]: I1203 11:37:25.067153 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/ecbf8b8e-cdb4-45c3-9455-d0dbc551356e-config-data-generated\") pod \"openstack-cell1-galera-0\" (UID: \"ecbf8b8e-cdb4-45c3-9455-d0dbc551356e\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 11:37:25 crc kubenswrapper[5052]: I1203 11:37:25.088983 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ecbf8b8e-cdb4-45c3-9455-d0dbc551356e-combined-ca-bundle\") pod \"openstack-cell1-galera-0\" (UID: \"ecbf8b8e-cdb4-45c3-9455-d0dbc551356e\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 11:37:25 crc kubenswrapper[5052]: I1203 11:37:25.091083 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/ecbf8b8e-cdb4-45c3-9455-d0dbc551356e-kolla-config\") pod \"openstack-cell1-galera-0\" (UID: \"ecbf8b8e-cdb4-45c3-9455-d0dbc551356e\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 11:37:25 crc kubenswrapper[5052]: I1203 11:37:25.091914 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/ecbf8b8e-cdb4-45c3-9455-d0dbc551356e-config-data-default\") pod \"openstack-cell1-galera-0\" (UID: \"ecbf8b8e-cdb4-45c3-9455-d0dbc551356e\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 11:37:25 crc kubenswrapper[5052]: I1203 11:37:25.104792 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/ecbf8b8e-cdb4-45c3-9455-d0dbc551356e-galera-tls-certs\") pod \"openstack-cell1-galera-0\" (UID: \"ecbf8b8e-cdb4-45c3-9455-d0dbc551356e\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 11:37:25 crc kubenswrapper[5052]: I1203 11:37:25.109365 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage12-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage12-crc\") pod \"openstack-cell1-galera-0\" (UID: \"ecbf8b8e-cdb4-45c3-9455-d0dbc551356e\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 11:37:25 crc kubenswrapper[5052]: I1203 11:37:25.112433 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-k49vf\" (UniqueName: \"kubernetes.io/projected/ecbf8b8e-cdb4-45c3-9455-d0dbc551356e-kube-api-access-k49vf\") pod \"openstack-cell1-galera-0\" (UID: \"ecbf8b8e-cdb4-45c3-9455-d0dbc551356e\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 11:37:25 crc kubenswrapper[5052]: I1203 11:37:25.166398 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/1609a5b5-6666-47e3-b02e-f796d972d331-kolla-config\") pod \"memcached-0\" (UID: \"1609a5b5-6666-47e3-b02e-f796d972d331\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 11:37:25 crc kubenswrapper[5052]: I1203 11:37:25.166466 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1609a5b5-6666-47e3-b02e-f796d972d331-combined-ca-bundle\") pod \"memcached-0\" (UID: \"1609a5b5-6666-47e3-b02e-f796d972d331\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 11:37:25 crc kubenswrapper[5052]: I1203 11:37:25.166556 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"memcached-tls-certs\" (UniqueName: \"kubernetes.io/secret/1609a5b5-6666-47e3-b02e-f796d972d331-memcached-tls-certs\") pod \"memcached-0\" (UID: \"1609a5b5-6666-47e3-b02e-f796d972d331\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 11:37:25 crc kubenswrapper[5052]: I1203 11:37:25.166578 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k9bfs\" (UniqueName: \"kubernetes.io/projected/1609a5b5-6666-47e3-b02e-f796d972d331-kube-api-access-k9bfs\") pod \"memcached-0\" (UID: \"1609a5b5-6666-47e3-b02e-f796d972d331\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 11:37:25 crc kubenswrapper[5052]: I1203 11:37:25.166642 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/1609a5b5-6666-47e3-b02e-f796d972d331-config-data\") pod \"memcached-0\" (UID: \"1609a5b5-6666-47e3-b02e-f796d972d331\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 11:37:25 crc kubenswrapper[5052]: I1203 11:37:25.192791 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 11:37:25 crc kubenswrapper[5052]: I1203 11:37:25.267775 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/1609a5b5-6666-47e3-b02e-f796d972d331-config-data\") pod \"memcached-0\" (UID: \"1609a5b5-6666-47e3-b02e-f796d972d331\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 11:37:25 crc kubenswrapper[5052]: I1203 11:37:25.267853 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/1609a5b5-6666-47e3-b02e-f796d972d331-kolla-config\") pod \"memcached-0\" (UID: \"1609a5b5-6666-47e3-b02e-f796d972d331\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 11:37:25 crc kubenswrapper[5052]: I1203 11:37:25.267872 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1609a5b5-6666-47e3-b02e-f796d972d331-combined-ca-bundle\") pod \"memcached-0\" (UID: \"1609a5b5-6666-47e3-b02e-f796d972d331\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 11:37:25 crc kubenswrapper[5052]: I1203 11:37:25.267924 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"memcached-tls-certs\" (UniqueName: \"kubernetes.io/secret/1609a5b5-6666-47e3-b02e-f796d972d331-memcached-tls-certs\") pod \"memcached-0\" (UID: \"1609a5b5-6666-47e3-b02e-f796d972d331\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 11:37:25 crc kubenswrapper[5052]: I1203 11:37:25.267941 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-k9bfs\" (UniqueName: \"kubernetes.io/projected/1609a5b5-6666-47e3-b02e-f796d972d331-kube-api-access-k9bfs\") pod \"memcached-0\" (UID: \"1609a5b5-6666-47e3-b02e-f796d972d331\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 11:37:25 crc kubenswrapper[5052]: I1203 11:37:25.268817 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/1609a5b5-6666-47e3-b02e-f796d972d331-kolla-config\") pod \"memcached-0\" (UID: \"1609a5b5-6666-47e3-b02e-f796d972d331\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 11:37:25 crc kubenswrapper[5052]: I1203 11:37:25.268817 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/1609a5b5-6666-47e3-b02e-f796d972d331-config-data\") pod \"memcached-0\" (UID: \"1609a5b5-6666-47e3-b02e-f796d972d331\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 11:37:25 crc kubenswrapper[5052]: I1203 11:37:25.273049 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1609a5b5-6666-47e3-b02e-f796d972d331-combined-ca-bundle\") pod \"memcached-0\" (UID: \"1609a5b5-6666-47e3-b02e-f796d972d331\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 11:37:25 crc kubenswrapper[5052]: I1203 11:37:25.275015 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"memcached-tls-certs\" (UniqueName: \"kubernetes.io/secret/1609a5b5-6666-47e3-b02e-f796d972d331-memcached-tls-certs\") pod \"memcached-0\" (UID: \"1609a5b5-6666-47e3-b02e-f796d972d331\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 11:37:25 crc kubenswrapper[5052]: I1203 11:37:25.285890 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-k9bfs\" (UniqueName: \"kubernetes.io/projected/1609a5b5-6666-47e3-b02e-f796d972d331-kube-api-access-k9bfs\") pod \"memcached-0\" (UID: \"1609a5b5-6666-47e3-b02e-f796d972d331\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 11:37:25 crc kubenswrapper[5052]: I1203 11:37:25.365004 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/memcached-0" Dec 03 11:37:26 crc kubenswrapper[5052]: E1203 11:37:26.200241 5052 log.go:32] "PullImage from image service failed" err="rpc error: code = Canceled desc = copying config: context canceled" image="quay.io/podified-antelope-centos9/openstack-neutron-server:current-podified" Dec 03 11:37:26 crc kubenswrapper[5052]: E1203 11:37:26.203758 5052 kuberuntime_manager.go:1274] "Unhandled Error" err="init container &Container{Name:init,Image:quay.io/podified-antelope-centos9/openstack-neutron-server:current-podified,Command:[/bin/bash],Args:[-c dnsmasq --interface=* --conf-dir=/etc/dnsmasq.d --hostsdir=/etc/dnsmasq.d/hosts --keep-in-foreground --log-debug --bind-interfaces --listen-address=$(POD_IP) --port 5353 --log-facility=- --no-hosts --domain-needed --no-resolv --bogus-priv --log-queries --test],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{EnvVar{Name:CONFIG_HASH,Value:n56bh585hb8hf5h5f4h7fh658hc6h549h8fh595h58bh4h5d6h66ch544h5c6hdh586h7dh59hb7h67h675h648h547h557h576h59bh5bdh5c4h5cbq,ValueFrom:nil,},EnvVar{Name:POD_IP,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:status.podIP,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:config,ReadOnly:true,MountPath:/etc/dnsmasq.d/config.cfg,SubPath:dnsmasq,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-rd7b5,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000680000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:nil,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:&SeccompProfile{Type:RuntimeDefault,LocalhostProfile:nil,},AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod dnsmasq-dnsmasq-f5849d7b9-nrdjn_openstack-kuttl-tests(897024db-e95e-4ab8-868f-ddb1ffadcbed): ErrImagePull: rpc error: code = Canceled desc = copying config: context canceled" logger="UnhandledError" Dec 03 11:37:26 crc kubenswrapper[5052]: E1203 11:37:26.204906 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"init\" with ErrImagePull: \"rpc error: code = Canceled desc = copying config: context canceled\"" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-f5849d7b9-nrdjn" podUID="897024db-e95e-4ab8-868f-ddb1ffadcbed" Dec 03 11:37:26 crc kubenswrapper[5052]: E1203 11:37:26.232033 5052 log.go:32] "PullImage from image service failed" err="rpc error: code = Canceled desc = copying config: context canceled" image="quay.io/podified-antelope-centos9/openstack-neutron-server:current-podified" Dec 03 11:37:26 crc kubenswrapper[5052]: E1203 11:37:26.232311 5052 kuberuntime_manager.go:1274] "Unhandled Error" err="init container &Container{Name:init,Image:quay.io/podified-antelope-centos9/openstack-neutron-server:current-podified,Command:[/bin/bash],Args:[-c dnsmasq --interface=* --conf-dir=/etc/dnsmasq.d --hostsdir=/etc/dnsmasq.d/hosts --keep-in-foreground --log-debug --bind-interfaces --listen-address=$(POD_IP) --port 5353 --log-facility=- --no-hosts --domain-needed --no-resolv --bogus-priv --log-queries --test],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{EnvVar{Name:CONFIG_HASH,Value:n689h697h5cbh9dh679h79hc8hb8h688h68h54fh5cbh88h67dh565h5cfh5ddh688h677h68ch64h67ch566h79h5h5b9h58h59ch568h68fh596h65dq,ValueFrom:nil,},EnvVar{Name:POD_IP,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:status.podIP,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:config,ReadOnly:true,MountPath:/etc/dnsmasq.d/config.cfg,SubPath:dnsmasq,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:dnsmasq-svc,ReadOnly:true,MountPath:/etc/dnsmasq.d/hosts/dnsmasq-svc,SubPath:dnsmasq-svc,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-z855s,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000680000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:nil,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:&SeccompProfile{Type:RuntimeDefault,LocalhostProfile:nil,},AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod dnsmasq-dnsmasq-84b9f45d47-78bj2_openstack-kuttl-tests(c5216bac-5393-4c1e-82e0-748b95282da0): ErrImagePull: rpc error: code = Canceled desc = copying config: context canceled" logger="UnhandledError" Dec 03 11:37:26 crc kubenswrapper[5052]: E1203 11:37:26.233849 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"init\" with ErrImagePull: \"rpc error: code = Canceled desc = copying config: context canceled\"" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-78bj2" podUID="c5216bac-5393-4c1e-82e0-748b95282da0" Dec 03 11:37:26 crc kubenswrapper[5052]: I1203 11:37:26.695221 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/rabbitmq-cell1-server-0"] Dec 03 11:37:26 crc kubenswrapper[5052]: W1203 11:37:26.760083 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9944d9af_7a7b_4773_8279_72914d63ef5f.slice/crio-4714213df4291409dd8c9248d4558e17471f3d7a3a646b8ada0b4e519793e1cb WatchSource:0}: Error finding container 4714213df4291409dd8c9248d4558e17471f3d7a3a646b8ada0b4e519793e1cb: Status 404 returned error can't find the container with id 4714213df4291409dd8c9248d4558e17471f3d7a3a646b8ada0b4e519793e1cb Dec 03 11:37:26 crc kubenswrapper[5052]: I1203 11:37:26.764969 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/rabbitmq-server-0"] Dec 03 11:37:26 crc kubenswrapper[5052]: I1203 11:37:26.794167 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/kube-state-metrics-0"] Dec 03 11:37:26 crc kubenswrapper[5052]: I1203 11:37:26.796925 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 11:37:26 crc kubenswrapper[5052]: I1203 11:37:26.806687 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/openstack-galera-0"] Dec 03 11:37:26 crc kubenswrapper[5052]: I1203 11:37:26.811776 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"telemetry-ceilometer-dockercfg-dvl69" Dec 03 11:37:26 crc kubenswrapper[5052]: I1203 11:37:26.822529 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/memcached-0"] Dec 03 11:37:26 crc kubenswrapper[5052]: I1203 11:37:26.831338 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/kube-state-metrics-0"] Dec 03 11:37:26 crc kubenswrapper[5052]: I1203 11:37:26.839568 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/openstack-cell1-galera-0"] Dec 03 11:37:26 crc kubenswrapper[5052]: I1203 11:37:26.895166 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9dxw6\" (UniqueName: \"kubernetes.io/projected/7535f906-b13f-4b91-b026-bac20ecb4867-kube-api-access-9dxw6\") pod \"kube-state-metrics-0\" (UID: \"7535f906-b13f-4b91-b026-bac20ecb4867\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 11:37:26 crc kubenswrapper[5052]: I1203 11:37:26.966330 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" event={"ID":"daa521fc-7e1d-44d2-b7d8-6aab003cfb66","Type":"ContainerStarted","Data":"21dffb5505e77fa5f1310e4358351755d36c795e92230fe26413268ed7da8b3f"} Dec 03 11:37:26 crc kubenswrapper[5052]: I1203 11:37:26.967543 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-server-0" event={"ID":"9944d9af-7a7b-4773-8279-72914d63ef5f","Type":"ContainerStarted","Data":"4714213df4291409dd8c9248d4558e17471f3d7a3a646b8ada0b4e519793e1cb"} Dec 03 11:37:26 crc kubenswrapper[5052]: I1203 11:37:26.968647 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-cell1-galera-0" event={"ID":"ecbf8b8e-cdb4-45c3-9455-d0dbc551356e","Type":"ContainerStarted","Data":"89cc373939988478670e3838e2f18f4c330ddc76d766fa4fb981a0b443876b76"} Dec 03 11:37:26 crc kubenswrapper[5052]: I1203 11:37:26.969719 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-galera-0" event={"ID":"251248ba-578c-49cf-b77e-fc3d5c6cb6fd","Type":"ContainerStarted","Data":"099cdfaa9cfd4abce46e13e4ab35647f5b1187b327bbce29cce00cc459aed7b4"} Dec 03 11:37:26 crc kubenswrapper[5052]: I1203 11:37:26.972502 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/memcached-0" event={"ID":"1609a5b5-6666-47e3-b02e-f796d972d331","Type":"ContainerStarted","Data":"b86af5c4131b572796dba50ca4f6e60a107928f2bf9062d24ad9be05669cd74f"} Dec 03 11:37:26 crc kubenswrapper[5052]: E1203 11:37:26.974038 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"init\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/podified-antelope-centos9/openstack-neutron-server:current-podified\\\"\"" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-78bj2" podUID="c5216bac-5393-4c1e-82e0-748b95282da0" Dec 03 11:37:26 crc kubenswrapper[5052]: E1203 11:37:26.974316 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"init\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/podified-antelope-centos9/openstack-neutron-server:current-podified\\\"\"" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-f5849d7b9-nrdjn" podUID="897024db-e95e-4ab8-868f-ddb1ffadcbed" Dec 03 11:37:26 crc kubenswrapper[5052]: I1203 11:37:26.996118 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-9dxw6\" (UniqueName: \"kubernetes.io/projected/7535f906-b13f-4b91-b026-bac20ecb4867-kube-api-access-9dxw6\") pod \"kube-state-metrics-0\" (UID: \"7535f906-b13f-4b91-b026-bac20ecb4867\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 11:37:27 crc kubenswrapper[5052]: I1203 11:37:27.020597 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-9dxw6\" (UniqueName: \"kubernetes.io/projected/7535f906-b13f-4b91-b026-bac20ecb4867-kube-api-access-9dxw6\") pod \"kube-state-metrics-0\" (UID: \"7535f906-b13f-4b91-b026-bac20ecb4867\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 11:37:27 crc kubenswrapper[5052]: I1203 11:37:27.117910 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 11:37:27 crc kubenswrapper[5052]: I1203 11:37:27.367289 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/kube-state-metrics-0"] Dec 03 11:37:27 crc kubenswrapper[5052]: I1203 11:37:27.986940 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/kube-state-metrics-0" event={"ID":"7535f906-b13f-4b91-b026-bac20ecb4867","Type":"ContainerStarted","Data":"5dc4f23ca754180c196168e997e200ebb2d925083ff1ecc22f6984be58982edc"} Dec 03 11:37:32 crc kubenswrapper[5052]: I1203 11:37:32.193012 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ovsdbserver-nb-0"] Dec 03 11:37:32 crc kubenswrapper[5052]: I1203 11:37:32.194884 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 11:37:32 crc kubenswrapper[5052]: I1203 11:37:32.198933 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"ovndbcluster-nb-config" Dec 03 11:37:32 crc kubenswrapper[5052]: I1203 11:37:32.199194 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-ovn-metrics" Dec 03 11:37:32 crc kubenswrapper[5052]: I1203 11:37:32.200065 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-ovndbcluster-nb-ovndbs" Dec 03 11:37:32 crc kubenswrapper[5052]: I1203 11:37:32.200275 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"ovndbcluster-nb-scripts" Dec 03 11:37:32 crc kubenswrapper[5052]: I1203 11:37:32.201045 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ovncluster-ovndbcluster-nb-dockercfg-gbwpl" Dec 03 11:37:32 crc kubenswrapper[5052]: I1203 11:37:32.211861 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-nb-0"] Dec 03 11:37:32 crc kubenswrapper[5052]: I1203 11:37:32.306646 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d7bd0c57-728c-46a4-bbec-2e209d9f4049-config\") pod \"ovsdbserver-nb-0\" (UID: \"d7bd0c57-728c-46a4-bbec-2e209d9f4049\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 11:37:32 crc kubenswrapper[5052]: I1203 11:37:32.306708 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8n24z\" (UniqueName: \"kubernetes.io/projected/d7bd0c57-728c-46a4-bbec-2e209d9f4049-kube-api-access-8n24z\") pod \"ovsdbserver-nb-0\" (UID: \"d7bd0c57-728c-46a4-bbec-2e209d9f4049\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 11:37:32 crc kubenswrapper[5052]: I1203 11:37:32.306738 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/d7bd0c57-728c-46a4-bbec-2e209d9f4049-scripts\") pod \"ovsdbserver-nb-0\" (UID: \"d7bd0c57-728c-46a4-bbec-2e209d9f4049\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 11:37:32 crc kubenswrapper[5052]: I1203 11:37:32.306763 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovsdbserver-nb-tls-certs\" (UniqueName: \"kubernetes.io/secret/d7bd0c57-728c-46a4-bbec-2e209d9f4049-ovsdbserver-nb-tls-certs\") pod \"ovsdbserver-nb-0\" (UID: \"d7bd0c57-728c-46a4-bbec-2e209d9f4049\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 11:37:32 crc kubenswrapper[5052]: I1203 11:37:32.306790 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/d7bd0c57-728c-46a4-bbec-2e209d9f4049-metrics-certs-tls-certs\") pod \"ovsdbserver-nb-0\" (UID: \"d7bd0c57-728c-46a4-bbec-2e209d9f4049\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 11:37:32 crc kubenswrapper[5052]: I1203 11:37:32.306828 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage14-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage14-crc\") pod \"ovsdbserver-nb-0\" (UID: \"d7bd0c57-728c-46a4-bbec-2e209d9f4049\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 11:37:32 crc kubenswrapper[5052]: I1203 11:37:32.306853 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d7bd0c57-728c-46a4-bbec-2e209d9f4049-combined-ca-bundle\") pod \"ovsdbserver-nb-0\" (UID: \"d7bd0c57-728c-46a4-bbec-2e209d9f4049\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 11:37:32 crc kubenswrapper[5052]: I1203 11:37:32.306869 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/d7bd0c57-728c-46a4-bbec-2e209d9f4049-ovsdb-rundir\") pod \"ovsdbserver-nb-0\" (UID: \"d7bd0c57-728c-46a4-bbec-2e209d9f4049\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 11:37:32 crc kubenswrapper[5052]: I1203 11:37:32.409309 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/d7bd0c57-728c-46a4-bbec-2e209d9f4049-metrics-certs-tls-certs\") pod \"ovsdbserver-nb-0\" (UID: \"d7bd0c57-728c-46a4-bbec-2e209d9f4049\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 11:37:32 crc kubenswrapper[5052]: I1203 11:37:32.409385 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage14-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage14-crc\") pod \"ovsdbserver-nb-0\" (UID: \"d7bd0c57-728c-46a4-bbec-2e209d9f4049\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 11:37:32 crc kubenswrapper[5052]: I1203 11:37:32.409491 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d7bd0c57-728c-46a4-bbec-2e209d9f4049-combined-ca-bundle\") pod \"ovsdbserver-nb-0\" (UID: \"d7bd0c57-728c-46a4-bbec-2e209d9f4049\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 11:37:32 crc kubenswrapper[5052]: I1203 11:37:32.409540 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/d7bd0c57-728c-46a4-bbec-2e209d9f4049-ovsdb-rundir\") pod \"ovsdbserver-nb-0\" (UID: \"d7bd0c57-728c-46a4-bbec-2e209d9f4049\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 11:37:32 crc kubenswrapper[5052]: I1203 11:37:32.409604 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d7bd0c57-728c-46a4-bbec-2e209d9f4049-config\") pod \"ovsdbserver-nb-0\" (UID: \"d7bd0c57-728c-46a4-bbec-2e209d9f4049\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 11:37:32 crc kubenswrapper[5052]: I1203 11:37:32.409728 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-8n24z\" (UniqueName: \"kubernetes.io/projected/d7bd0c57-728c-46a4-bbec-2e209d9f4049-kube-api-access-8n24z\") pod \"ovsdbserver-nb-0\" (UID: \"d7bd0c57-728c-46a4-bbec-2e209d9f4049\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 11:37:32 crc kubenswrapper[5052]: I1203 11:37:32.409770 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/d7bd0c57-728c-46a4-bbec-2e209d9f4049-scripts\") pod \"ovsdbserver-nb-0\" (UID: \"d7bd0c57-728c-46a4-bbec-2e209d9f4049\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 11:37:32 crc kubenswrapper[5052]: I1203 11:37:32.409804 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovsdbserver-nb-tls-certs\" (UniqueName: \"kubernetes.io/secret/d7bd0c57-728c-46a4-bbec-2e209d9f4049-ovsdbserver-nb-tls-certs\") pod \"ovsdbserver-nb-0\" (UID: \"d7bd0c57-728c-46a4-bbec-2e209d9f4049\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 11:37:32 crc kubenswrapper[5052]: I1203 11:37:32.410497 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d7bd0c57-728c-46a4-bbec-2e209d9f4049-config\") pod \"ovsdbserver-nb-0\" (UID: \"d7bd0c57-728c-46a4-bbec-2e209d9f4049\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 11:37:32 crc kubenswrapper[5052]: I1203 11:37:32.411385 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/d7bd0c57-728c-46a4-bbec-2e209d9f4049-scripts\") pod \"ovsdbserver-nb-0\" (UID: \"d7bd0c57-728c-46a4-bbec-2e209d9f4049\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 11:37:32 crc kubenswrapper[5052]: I1203 11:37:32.411917 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/d7bd0c57-728c-46a4-bbec-2e209d9f4049-ovsdb-rundir\") pod \"ovsdbserver-nb-0\" (UID: \"d7bd0c57-728c-46a4-bbec-2e209d9f4049\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 11:37:32 crc kubenswrapper[5052]: I1203 11:37:32.412133 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage14-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage14-crc\") pod \"ovsdbserver-nb-0\" (UID: \"d7bd0c57-728c-46a4-bbec-2e209d9f4049\") device mount path \"/mnt/openstack/pv14\"" pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 11:37:32 crc kubenswrapper[5052]: I1203 11:37:32.416753 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovsdbserver-nb-tls-certs\" (UniqueName: \"kubernetes.io/secret/d7bd0c57-728c-46a4-bbec-2e209d9f4049-ovsdbserver-nb-tls-certs\") pod \"ovsdbserver-nb-0\" (UID: \"d7bd0c57-728c-46a4-bbec-2e209d9f4049\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 11:37:32 crc kubenswrapper[5052]: I1203 11:37:32.424005 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d7bd0c57-728c-46a4-bbec-2e209d9f4049-combined-ca-bundle\") pod \"ovsdbserver-nb-0\" (UID: \"d7bd0c57-728c-46a4-bbec-2e209d9f4049\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 11:37:32 crc kubenswrapper[5052]: I1203 11:37:32.426710 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/d7bd0c57-728c-46a4-bbec-2e209d9f4049-metrics-certs-tls-certs\") pod \"ovsdbserver-nb-0\" (UID: \"d7bd0c57-728c-46a4-bbec-2e209d9f4049\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 11:37:32 crc kubenswrapper[5052]: I1203 11:37:32.428410 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-8n24z\" (UniqueName: \"kubernetes.io/projected/d7bd0c57-728c-46a4-bbec-2e209d9f4049-kube-api-access-8n24z\") pod \"ovsdbserver-nb-0\" (UID: \"d7bd0c57-728c-46a4-bbec-2e209d9f4049\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 11:37:32 crc kubenswrapper[5052]: I1203 11:37:32.464812 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage14-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage14-crc\") pod \"ovsdbserver-nb-0\" (UID: \"d7bd0c57-728c-46a4-bbec-2e209d9f4049\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 11:37:32 crc kubenswrapper[5052]: I1203 11:37:32.519064 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 11:37:33 crc kubenswrapper[5052]: I1203 11:37:33.185316 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ovsdbserver-sb-0"] Dec 03 11:37:33 crc kubenswrapper[5052]: I1203 11:37:33.188531 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 11:37:33 crc kubenswrapper[5052]: I1203 11:37:33.190936 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"ovndbcluster-sb-config" Dec 03 11:37:33 crc kubenswrapper[5052]: I1203 11:37:33.191143 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-ovndbcluster-sb-ovndbs" Dec 03 11:37:33 crc kubenswrapper[5052]: I1203 11:37:33.191294 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"ovndbcluster-sb-scripts" Dec 03 11:37:33 crc kubenswrapper[5052]: I1203 11:37:33.191773 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ovncluster-ovndbcluster-sb-dockercfg-ngh58" Dec 03 11:37:33 crc kubenswrapper[5052]: I1203 11:37:33.197473 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-sb-0"] Dec 03 11:37:33 crc kubenswrapper[5052]: I1203 11:37:33.229760 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/e14d926b-5dc2-46a4-ab27-dfde118cba89-metrics-certs-tls-certs\") pod \"ovsdbserver-sb-0\" (UID: \"e14d926b-5dc2-46a4-ab27-dfde118cba89\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 11:37:33 crc kubenswrapper[5052]: I1203 11:37:33.229829 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-crnf7\" (UniqueName: \"kubernetes.io/projected/e14d926b-5dc2-46a4-ab27-dfde118cba89-kube-api-access-crnf7\") pod \"ovsdbserver-sb-0\" (UID: \"e14d926b-5dc2-46a4-ab27-dfde118cba89\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 11:37:33 crc kubenswrapper[5052]: I1203 11:37:33.229902 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage01-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage01-crc\") pod \"ovsdbserver-sb-0\" (UID: \"e14d926b-5dc2-46a4-ab27-dfde118cba89\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 11:37:33 crc kubenswrapper[5052]: I1203 11:37:33.229941 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/e14d926b-5dc2-46a4-ab27-dfde118cba89-ovsdb-rundir\") pod \"ovsdbserver-sb-0\" (UID: \"e14d926b-5dc2-46a4-ab27-dfde118cba89\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 11:37:33 crc kubenswrapper[5052]: I1203 11:37:33.229984 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e14d926b-5dc2-46a4-ab27-dfde118cba89-config\") pod \"ovsdbserver-sb-0\" (UID: \"e14d926b-5dc2-46a4-ab27-dfde118cba89\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 11:37:33 crc kubenswrapper[5052]: I1203 11:37:33.230038 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/e14d926b-5dc2-46a4-ab27-dfde118cba89-scripts\") pod \"ovsdbserver-sb-0\" (UID: \"e14d926b-5dc2-46a4-ab27-dfde118cba89\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 11:37:33 crc kubenswrapper[5052]: I1203 11:37:33.230080 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovsdbserver-sb-tls-certs\" (UniqueName: \"kubernetes.io/secret/e14d926b-5dc2-46a4-ab27-dfde118cba89-ovsdbserver-sb-tls-certs\") pod \"ovsdbserver-sb-0\" (UID: \"e14d926b-5dc2-46a4-ab27-dfde118cba89\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 11:37:33 crc kubenswrapper[5052]: I1203 11:37:33.230118 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e14d926b-5dc2-46a4-ab27-dfde118cba89-combined-ca-bundle\") pod \"ovsdbserver-sb-0\" (UID: \"e14d926b-5dc2-46a4-ab27-dfde118cba89\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 11:37:33 crc kubenswrapper[5052]: I1203 11:37:33.331527 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/e14d926b-5dc2-46a4-ab27-dfde118cba89-ovsdb-rundir\") pod \"ovsdbserver-sb-0\" (UID: \"e14d926b-5dc2-46a4-ab27-dfde118cba89\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 11:37:33 crc kubenswrapper[5052]: I1203 11:37:33.331595 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e14d926b-5dc2-46a4-ab27-dfde118cba89-config\") pod \"ovsdbserver-sb-0\" (UID: \"e14d926b-5dc2-46a4-ab27-dfde118cba89\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 11:37:33 crc kubenswrapper[5052]: I1203 11:37:33.331652 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/e14d926b-5dc2-46a4-ab27-dfde118cba89-scripts\") pod \"ovsdbserver-sb-0\" (UID: \"e14d926b-5dc2-46a4-ab27-dfde118cba89\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 11:37:33 crc kubenswrapper[5052]: I1203 11:37:33.331686 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovsdbserver-sb-tls-certs\" (UniqueName: \"kubernetes.io/secret/e14d926b-5dc2-46a4-ab27-dfde118cba89-ovsdbserver-sb-tls-certs\") pod \"ovsdbserver-sb-0\" (UID: \"e14d926b-5dc2-46a4-ab27-dfde118cba89\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 11:37:33 crc kubenswrapper[5052]: I1203 11:37:33.331717 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e14d926b-5dc2-46a4-ab27-dfde118cba89-combined-ca-bundle\") pod \"ovsdbserver-sb-0\" (UID: \"e14d926b-5dc2-46a4-ab27-dfde118cba89\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 11:37:33 crc kubenswrapper[5052]: I1203 11:37:33.331754 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/e14d926b-5dc2-46a4-ab27-dfde118cba89-metrics-certs-tls-certs\") pod \"ovsdbserver-sb-0\" (UID: \"e14d926b-5dc2-46a4-ab27-dfde118cba89\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 11:37:33 crc kubenswrapper[5052]: I1203 11:37:33.331780 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-crnf7\" (UniqueName: \"kubernetes.io/projected/e14d926b-5dc2-46a4-ab27-dfde118cba89-kube-api-access-crnf7\") pod \"ovsdbserver-sb-0\" (UID: \"e14d926b-5dc2-46a4-ab27-dfde118cba89\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 11:37:33 crc kubenswrapper[5052]: I1203 11:37:33.331810 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage01-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage01-crc\") pod \"ovsdbserver-sb-0\" (UID: \"e14d926b-5dc2-46a4-ab27-dfde118cba89\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 11:37:33 crc kubenswrapper[5052]: I1203 11:37:33.332071 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage01-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage01-crc\") pod \"ovsdbserver-sb-0\" (UID: \"e14d926b-5dc2-46a4-ab27-dfde118cba89\") device mount path \"/mnt/openstack/pv01\"" pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 11:37:33 crc kubenswrapper[5052]: I1203 11:37:33.332489 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/e14d926b-5dc2-46a4-ab27-dfde118cba89-ovsdb-rundir\") pod \"ovsdbserver-sb-0\" (UID: \"e14d926b-5dc2-46a4-ab27-dfde118cba89\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 11:37:33 crc kubenswrapper[5052]: I1203 11:37:33.333241 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e14d926b-5dc2-46a4-ab27-dfde118cba89-config\") pod \"ovsdbserver-sb-0\" (UID: \"e14d926b-5dc2-46a4-ab27-dfde118cba89\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 11:37:33 crc kubenswrapper[5052]: I1203 11:37:33.334355 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/e14d926b-5dc2-46a4-ab27-dfde118cba89-scripts\") pod \"ovsdbserver-sb-0\" (UID: \"e14d926b-5dc2-46a4-ab27-dfde118cba89\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 11:37:33 crc kubenswrapper[5052]: I1203 11:37:33.344578 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/e14d926b-5dc2-46a4-ab27-dfde118cba89-metrics-certs-tls-certs\") pod \"ovsdbserver-sb-0\" (UID: \"e14d926b-5dc2-46a4-ab27-dfde118cba89\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 11:37:33 crc kubenswrapper[5052]: I1203 11:37:33.345630 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovsdbserver-sb-tls-certs\" (UniqueName: \"kubernetes.io/secret/e14d926b-5dc2-46a4-ab27-dfde118cba89-ovsdbserver-sb-tls-certs\") pod \"ovsdbserver-sb-0\" (UID: \"e14d926b-5dc2-46a4-ab27-dfde118cba89\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 11:37:33 crc kubenswrapper[5052]: I1203 11:37:33.350419 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e14d926b-5dc2-46a4-ab27-dfde118cba89-combined-ca-bundle\") pod \"ovsdbserver-sb-0\" (UID: \"e14d926b-5dc2-46a4-ab27-dfde118cba89\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 11:37:33 crc kubenswrapper[5052]: I1203 11:37:33.353185 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-crnf7\" (UniqueName: \"kubernetes.io/projected/e14d926b-5dc2-46a4-ab27-dfde118cba89-kube-api-access-crnf7\") pod \"ovsdbserver-sb-0\" (UID: \"e14d926b-5dc2-46a4-ab27-dfde118cba89\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 11:37:33 crc kubenswrapper[5052]: I1203 11:37:33.365275 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage01-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage01-crc\") pod \"ovsdbserver-sb-0\" (UID: \"e14d926b-5dc2-46a4-ab27-dfde118cba89\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 11:37:33 crc kubenswrapper[5052]: I1203 11:37:33.530911 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 11:37:34 crc kubenswrapper[5052]: I1203 11:37:34.212326 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-nb-0"] Dec 03 11:37:34 crc kubenswrapper[5052]: I1203 11:37:34.377598 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-sb-0"] Dec 03 11:37:34 crc kubenswrapper[5052]: W1203 11:37:34.424494 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode14d926b_5dc2_46a4_ab27_dfde118cba89.slice/crio-7b70c188cc2577da2ec80ec8d366b69c4acc8260edefe42f53b57aa50f107f62 WatchSource:0}: Error finding container 7b70c188cc2577da2ec80ec8d366b69c4acc8260edefe42f53b57aa50f107f62: Status 404 returned error can't find the container with id 7b70c188cc2577da2ec80ec8d366b69c4acc8260edefe42f53b57aa50f107f62 Dec 03 11:37:35 crc kubenswrapper[5052]: I1203 11:37:35.067362 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-galera-0" event={"ID":"251248ba-578c-49cf-b77e-fc3d5c6cb6fd","Type":"ContainerStarted","Data":"e0bd842931dfa2118f2a524f12f2a960d4ae5d1b4551cb7036c004b424844c69"} Dec 03 11:37:35 crc kubenswrapper[5052]: I1203 11:37:35.069193 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-nb-0" event={"ID":"d7bd0c57-728c-46a4-bbec-2e209d9f4049","Type":"ContainerStarted","Data":"97afd4cd6ab520f3b9f6350f09167b84e4a6e1ed7ab37540b3f4244b47290b0f"} Dec 03 11:37:35 crc kubenswrapper[5052]: I1203 11:37:35.074863 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-sb-0" event={"ID":"e14d926b-5dc2-46a4-ab27-dfde118cba89","Type":"ContainerStarted","Data":"7b70c188cc2577da2ec80ec8d366b69c4acc8260edefe42f53b57aa50f107f62"} Dec 03 11:37:35 crc kubenswrapper[5052]: I1203 11:37:35.078867 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/memcached-0" event={"ID":"1609a5b5-6666-47e3-b02e-f796d972d331","Type":"ContainerStarted","Data":"352c7041b2ebcbf03c6b6dbad2b08bc20c546043c65916bf3b506556deb9182f"} Dec 03 11:37:35 crc kubenswrapper[5052]: I1203 11:37:35.078936 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/memcached-0" Dec 03 11:37:35 crc kubenswrapper[5052]: I1203 11:37:35.081327 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/kube-state-metrics-0" event={"ID":"7535f906-b13f-4b91-b026-bac20ecb4867","Type":"ContainerStarted","Data":"c5eb397a2824a206861b0c68d056b30eaa38918807f14f653143e1839a6979a7"} Dec 03 11:37:35 crc kubenswrapper[5052]: I1203 11:37:35.081458 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 11:37:35 crc kubenswrapper[5052]: I1203 11:37:35.083214 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-cell1-galera-0" event={"ID":"ecbf8b8e-cdb4-45c3-9455-d0dbc551356e","Type":"ContainerStarted","Data":"5d159c33361809fb42fe366ed481f946a656d0a56465673cfd5f86a09775940d"} Dec 03 11:37:35 crc kubenswrapper[5052]: I1203 11:37:35.135491 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/kube-state-metrics-0" podStartSLOduration=2.066350951 podStartE2EDuration="9.135473188s" podCreationTimestamp="2025-12-03 11:37:26 +0000 UTC" firstStartedPulling="2025-12-03 11:37:27.375614646 +0000 UTC m=+1037.963856410" lastFinishedPulling="2025-12-03 11:37:34.444736883 +0000 UTC m=+1045.032978647" observedRunningTime="2025-12-03 11:37:35.135063917 +0000 UTC m=+1045.723305691" watchObservedRunningTime="2025-12-03 11:37:35.135473188 +0000 UTC m=+1045.723714952" Dec 03 11:37:36 crc kubenswrapper[5052]: I1203 11:37:36.093065 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-server-0" event={"ID":"9944d9af-7a7b-4773-8279-72914d63ef5f","Type":"ContainerStarted","Data":"a4c0b116720eadc0b68b284395f846e2b236870d208137226f3a533a48d6959e"} Dec 03 11:37:36 crc kubenswrapper[5052]: I1203 11:37:36.095449 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" event={"ID":"daa521fc-7e1d-44d2-b7d8-6aab003cfb66","Type":"ContainerStarted","Data":"0fbd3a4d4383d62bcd48e87a34d3dbb4ccd0417936dd06ed21f799d8c04dd391"} Dec 03 11:37:36 crc kubenswrapper[5052]: I1203 11:37:36.121911 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/memcached-0" podStartSLOduration=4.568164387 podStartE2EDuration="11.121885933s" podCreationTimestamp="2025-12-03 11:37:25 +0000 UTC" firstStartedPulling="2025-12-03 11:37:26.819806848 +0000 UTC m=+1037.408048612" lastFinishedPulling="2025-12-03 11:37:33.373528394 +0000 UTC m=+1043.961770158" observedRunningTime="2025-12-03 11:37:35.158637409 +0000 UTC m=+1045.746879173" watchObservedRunningTime="2025-12-03 11:37:36.121885933 +0000 UTC m=+1046.710127697" Dec 03 11:37:39 crc kubenswrapper[5052]: I1203 11:37:39.122694 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-nb-0" event={"ID":"d7bd0c57-728c-46a4-bbec-2e209d9f4049","Type":"ContainerStarted","Data":"d8d8be4b59c92e57d45d986195fe45d17c86d578df37f5f3af30a57358eda608"} Dec 03 11:37:39 crc kubenswrapper[5052]: I1203 11:37:39.124867 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-sb-0" event={"ID":"e14d926b-5dc2-46a4-ab27-dfde118cba89","Type":"ContainerStarted","Data":"ef90774de4b83b9fc123ba664ece34787dcdb1ad9fd0d230dacf72b8eee1a1e1"} Dec 03 11:37:39 crc kubenswrapper[5052]: I1203 11:37:39.127049 5052 generic.go:334] "Generic (PLEG): container finished" podID="ecbf8b8e-cdb4-45c3-9455-d0dbc551356e" containerID="5d159c33361809fb42fe366ed481f946a656d0a56465673cfd5f86a09775940d" exitCode=0 Dec 03 11:37:39 crc kubenswrapper[5052]: I1203 11:37:39.127111 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-cell1-galera-0" event={"ID":"ecbf8b8e-cdb4-45c3-9455-d0dbc551356e","Type":"ContainerDied","Data":"5d159c33361809fb42fe366ed481f946a656d0a56465673cfd5f86a09775940d"} Dec 03 11:37:39 crc kubenswrapper[5052]: I1203 11:37:39.129988 5052 generic.go:334] "Generic (PLEG): container finished" podID="251248ba-578c-49cf-b77e-fc3d5c6cb6fd" containerID="e0bd842931dfa2118f2a524f12f2a960d4ae5d1b4551cb7036c004b424844c69" exitCode=0 Dec 03 11:37:39 crc kubenswrapper[5052]: I1203 11:37:39.130010 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-galera-0" event={"ID":"251248ba-578c-49cf-b77e-fc3d5c6cb6fd","Type":"ContainerDied","Data":"e0bd842931dfa2118f2a524f12f2a960d4ae5d1b4551cb7036c004b424844c69"} Dec 03 11:37:40 crc kubenswrapper[5052]: I1203 11:37:40.159655 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-cell1-galera-0" event={"ID":"ecbf8b8e-cdb4-45c3-9455-d0dbc551356e","Type":"ContainerStarted","Data":"d9bef6e726b038ae865fa13770a700b7a8625ffee06c08933704155c1695d94f"} Dec 03 11:37:40 crc kubenswrapper[5052]: I1203 11:37:40.163606 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-galera-0" event={"ID":"251248ba-578c-49cf-b77e-fc3d5c6cb6fd","Type":"ContainerStarted","Data":"5e0a4e3bdf4887c1a95e0d9815e6d8a1fc6c0dccf8359c3a1a7a1505e8cc9053"} Dec 03 11:37:40 crc kubenswrapper[5052]: I1203 11:37:40.173923 5052 generic.go:334] "Generic (PLEG): container finished" podID="c5216bac-5393-4c1e-82e0-748b95282da0" containerID="97ac574ed6407f65761e184a22ba07e4cf13f3422f77c42f0080ecc0878e410e" exitCode=0 Dec 03 11:37:40 crc kubenswrapper[5052]: I1203 11:37:40.173965 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-78bj2" event={"ID":"c5216bac-5393-4c1e-82e0-748b95282da0","Type":"ContainerDied","Data":"97ac574ed6407f65761e184a22ba07e4cf13f3422f77c42f0080ecc0878e410e"} Dec 03 11:37:40 crc kubenswrapper[5052]: I1203 11:37:40.186392 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/openstack-cell1-galera-0" podStartSLOduration=10.172433423 podStartE2EDuration="17.186371112s" podCreationTimestamp="2025-12-03 11:37:23 +0000 UTC" firstStartedPulling="2025-12-03 11:37:26.849178798 +0000 UTC m=+1037.437420562" lastFinishedPulling="2025-12-03 11:37:33.863116487 +0000 UTC m=+1044.451358251" observedRunningTime="2025-12-03 11:37:40.179891145 +0000 UTC m=+1050.768132909" watchObservedRunningTime="2025-12-03 11:37:40.186371112 +0000 UTC m=+1050.774612876" Dec 03 11:37:40 crc kubenswrapper[5052]: I1203 11:37:40.212444 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/openstack-galera-0" podStartSLOduration=11.226336723 podStartE2EDuration="18.212417466s" podCreationTimestamp="2025-12-03 11:37:22 +0000 UTC" firstStartedPulling="2025-12-03 11:37:26.81989492 +0000 UTC m=+1037.408136674" lastFinishedPulling="2025-12-03 11:37:33.805975653 +0000 UTC m=+1044.394217417" observedRunningTime="2025-12-03 11:37:40.209765819 +0000 UTC m=+1050.798007593" watchObservedRunningTime="2025-12-03 11:37:40.212417466 +0000 UTC m=+1050.800659230" Dec 03 11:37:40 crc kubenswrapper[5052]: I1203 11:37:40.366737 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/memcached-0" Dec 03 11:37:42 crc kubenswrapper[5052]: I1203 11:37:42.195590 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-78bj2" event={"ID":"c5216bac-5393-4c1e-82e0-748b95282da0","Type":"ContainerStarted","Data":"45c8cb70e0e388973a2dee58d830290c0d639943ca82589ae999cad7fe891a84"} Dec 03 11:37:42 crc kubenswrapper[5052]: I1203 11:37:42.197905 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-f5849d7b9-nrdjn" event={"ID":"897024db-e95e-4ab8-868f-ddb1ffadcbed","Type":"ContainerStarted","Data":"552bd1c850f4094b35e2c2b531421785b4fc7d4b1201f8e7f2e52d15939f7a71"} Dec 03 11:37:42 crc kubenswrapper[5052]: I1203 11:37:42.200539 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-nb-0" event={"ID":"d7bd0c57-728c-46a4-bbec-2e209d9f4049","Type":"ContainerStarted","Data":"959614b917450812745a85b82761fed66dc1a03b7fd08eb36921870d079f6214"} Dec 03 11:37:42 crc kubenswrapper[5052]: I1203 11:37:42.204338 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-sb-0" event={"ID":"e14d926b-5dc2-46a4-ab27-dfde118cba89","Type":"ContainerStarted","Data":"8da90eb5a7870f9bbe93a949f459805faa0302426245aa70e016cd57e8992d74"} Dec 03 11:37:42 crc kubenswrapper[5052]: I1203 11:37:42.225048 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/ovsdbserver-nb-0" podStartSLOduration=3.957628451 podStartE2EDuration="11.225028847s" podCreationTimestamp="2025-12-03 11:37:31 +0000 UTC" firstStartedPulling="2025-12-03 11:37:34.414625452 +0000 UTC m=+1045.002867226" lastFinishedPulling="2025-12-03 11:37:41.682025858 +0000 UTC m=+1052.270267622" observedRunningTime="2025-12-03 11:37:42.222285017 +0000 UTC m=+1052.810526781" watchObservedRunningTime="2025-12-03 11:37:42.225028847 +0000 UTC m=+1052.813270611" Dec 03 11:37:42 crc kubenswrapper[5052]: I1203 11:37:42.519966 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 11:37:43 crc kubenswrapper[5052]: I1203 11:37:43.222334 5052 generic.go:334] "Generic (PLEG): container finished" podID="897024db-e95e-4ab8-868f-ddb1ffadcbed" containerID="552bd1c850f4094b35e2c2b531421785b4fc7d4b1201f8e7f2e52d15939f7a71" exitCode=0 Dec 03 11:37:43 crc kubenswrapper[5052]: I1203 11:37:43.232694 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-f5849d7b9-nrdjn" event={"ID":"897024db-e95e-4ab8-868f-ddb1ffadcbed","Type":"ContainerDied","Data":"552bd1c850f4094b35e2c2b531421785b4fc7d4b1201f8e7f2e52d15939f7a71"} Dec 03 11:37:43 crc kubenswrapper[5052]: I1203 11:37:43.233719 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-78bj2" Dec 03 11:37:43 crc kubenswrapper[5052]: I1203 11:37:43.273560 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-78bj2" podStartSLOduration=5.188219876 podStartE2EDuration="33.273531579s" podCreationTimestamp="2025-12-03 11:37:10 +0000 UTC" firstStartedPulling="2025-12-03 11:37:11.18634463 +0000 UTC m=+1021.774586404" lastFinishedPulling="2025-12-03 11:37:39.271656343 +0000 UTC m=+1049.859898107" observedRunningTime="2025-12-03 11:37:43.249535214 +0000 UTC m=+1053.837776978" watchObservedRunningTime="2025-12-03 11:37:43.273531579 +0000 UTC m=+1053.861773353" Dec 03 11:37:43 crc kubenswrapper[5052]: I1203 11:37:43.307633 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/ovsdbserver-sb-0" podStartSLOduration=4.041489936 podStartE2EDuration="11.307612326s" podCreationTimestamp="2025-12-03 11:37:32 +0000 UTC" firstStartedPulling="2025-12-03 11:37:34.438489082 +0000 UTC m=+1045.026730846" lastFinishedPulling="2025-12-03 11:37:41.704611472 +0000 UTC m=+1052.292853236" observedRunningTime="2025-12-03 11:37:43.301149809 +0000 UTC m=+1053.889391593" watchObservedRunningTime="2025-12-03 11:37:43.307612326 +0000 UTC m=+1053.895854090" Dec 03 11:37:43 crc kubenswrapper[5052]: I1203 11:37:43.532237 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 11:37:44 crc kubenswrapper[5052]: I1203 11:37:44.057177 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 11:37:44 crc kubenswrapper[5052]: I1203 11:37:44.057227 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 11:37:44 crc kubenswrapper[5052]: I1203 11:37:44.520384 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 11:37:44 crc kubenswrapper[5052]: I1203 11:37:44.567952 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 11:37:45 crc kubenswrapper[5052]: I1203 11:37:45.194032 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 11:37:45 crc kubenswrapper[5052]: I1203 11:37:45.194092 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 11:37:45 crc kubenswrapper[5052]: I1203 11:37:45.283983 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 11:37:45 crc kubenswrapper[5052]: I1203 11:37:45.532573 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 11:37:45 crc kubenswrapper[5052]: I1203 11:37:45.582316 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 11:37:46 crc kubenswrapper[5052]: I1203 11:37:46.285805 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 11:37:46 crc kubenswrapper[5052]: I1203 11:37:46.444844 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ovn-northd-0"] Dec 03 11:37:46 crc kubenswrapper[5052]: I1203 11:37:46.450617 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 11:37:46 crc kubenswrapper[5052]: I1203 11:37:46.498433 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"ovnnorthd-scripts" Dec 03 11:37:46 crc kubenswrapper[5052]: I1203 11:37:46.499287 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-ovnnorthd-ovndbs" Dec 03 11:37:46 crc kubenswrapper[5052]: I1203 11:37:46.499419 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"ovnnorthd-config" Dec 03 11:37:46 crc kubenswrapper[5052]: I1203 11:37:46.499541 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ovnnorthd-ovnnorthd-dockercfg-f99zf" Dec 03 11:37:46 crc kubenswrapper[5052]: I1203 11:37:46.506779 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ovn-northd-0"] Dec 03 11:37:46 crc kubenswrapper[5052]: I1203 11:37:46.600292 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d5lbp\" (UniqueName: \"kubernetes.io/projected/a30eaf15-65ff-426b-8ac2-1ed5a369886a-kube-api-access-d5lbp\") pod \"ovn-northd-0\" (UID: \"a30eaf15-65ff-426b-8ac2-1ed5a369886a\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 11:37:46 crc kubenswrapper[5052]: I1203 11:37:46.600332 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/a30eaf15-65ff-426b-8ac2-1ed5a369886a-scripts\") pod \"ovn-northd-0\" (UID: \"a30eaf15-65ff-426b-8ac2-1ed5a369886a\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 11:37:46 crc kubenswrapper[5052]: I1203 11:37:46.600361 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-rundir\" (UniqueName: \"kubernetes.io/empty-dir/a30eaf15-65ff-426b-8ac2-1ed5a369886a-ovn-rundir\") pod \"ovn-northd-0\" (UID: \"a30eaf15-65ff-426b-8ac2-1ed5a369886a\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 11:37:46 crc kubenswrapper[5052]: I1203 11:37:46.600377 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-northd-tls-certs\" (UniqueName: \"kubernetes.io/secret/a30eaf15-65ff-426b-8ac2-1ed5a369886a-ovn-northd-tls-certs\") pod \"ovn-northd-0\" (UID: \"a30eaf15-65ff-426b-8ac2-1ed5a369886a\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 11:37:46 crc kubenswrapper[5052]: I1203 11:37:46.600726 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/a30eaf15-65ff-426b-8ac2-1ed5a369886a-metrics-certs-tls-certs\") pod \"ovn-northd-0\" (UID: \"a30eaf15-65ff-426b-8ac2-1ed5a369886a\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 11:37:46 crc kubenswrapper[5052]: I1203 11:37:46.600781 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a30eaf15-65ff-426b-8ac2-1ed5a369886a-config\") pod \"ovn-northd-0\" (UID: \"a30eaf15-65ff-426b-8ac2-1ed5a369886a\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 11:37:46 crc kubenswrapper[5052]: I1203 11:37:46.600822 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a30eaf15-65ff-426b-8ac2-1ed5a369886a-combined-ca-bundle\") pod \"ovn-northd-0\" (UID: \"a30eaf15-65ff-426b-8ac2-1ed5a369886a\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 11:37:46 crc kubenswrapper[5052]: I1203 11:37:46.702015 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-d5lbp\" (UniqueName: \"kubernetes.io/projected/a30eaf15-65ff-426b-8ac2-1ed5a369886a-kube-api-access-d5lbp\") pod \"ovn-northd-0\" (UID: \"a30eaf15-65ff-426b-8ac2-1ed5a369886a\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 11:37:46 crc kubenswrapper[5052]: I1203 11:37:46.702065 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/a30eaf15-65ff-426b-8ac2-1ed5a369886a-scripts\") pod \"ovn-northd-0\" (UID: \"a30eaf15-65ff-426b-8ac2-1ed5a369886a\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 11:37:46 crc kubenswrapper[5052]: I1203 11:37:46.702085 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovn-rundir\" (UniqueName: \"kubernetes.io/empty-dir/a30eaf15-65ff-426b-8ac2-1ed5a369886a-ovn-rundir\") pod \"ovn-northd-0\" (UID: \"a30eaf15-65ff-426b-8ac2-1ed5a369886a\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 11:37:46 crc kubenswrapper[5052]: I1203 11:37:46.702102 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovn-northd-tls-certs\" (UniqueName: \"kubernetes.io/secret/a30eaf15-65ff-426b-8ac2-1ed5a369886a-ovn-northd-tls-certs\") pod \"ovn-northd-0\" (UID: \"a30eaf15-65ff-426b-8ac2-1ed5a369886a\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 11:37:46 crc kubenswrapper[5052]: I1203 11:37:46.702175 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/a30eaf15-65ff-426b-8ac2-1ed5a369886a-metrics-certs-tls-certs\") pod \"ovn-northd-0\" (UID: \"a30eaf15-65ff-426b-8ac2-1ed5a369886a\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 11:37:46 crc kubenswrapper[5052]: I1203 11:37:46.702193 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a30eaf15-65ff-426b-8ac2-1ed5a369886a-config\") pod \"ovn-northd-0\" (UID: \"a30eaf15-65ff-426b-8ac2-1ed5a369886a\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 11:37:46 crc kubenswrapper[5052]: I1203 11:37:46.702211 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a30eaf15-65ff-426b-8ac2-1ed5a369886a-combined-ca-bundle\") pod \"ovn-northd-0\" (UID: \"a30eaf15-65ff-426b-8ac2-1ed5a369886a\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 11:37:46 crc kubenswrapper[5052]: I1203 11:37:46.702764 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovn-rundir\" (UniqueName: \"kubernetes.io/empty-dir/a30eaf15-65ff-426b-8ac2-1ed5a369886a-ovn-rundir\") pod \"ovn-northd-0\" (UID: \"a30eaf15-65ff-426b-8ac2-1ed5a369886a\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 11:37:46 crc kubenswrapper[5052]: I1203 11:37:46.703380 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/a30eaf15-65ff-426b-8ac2-1ed5a369886a-scripts\") pod \"ovn-northd-0\" (UID: \"a30eaf15-65ff-426b-8ac2-1ed5a369886a\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 11:37:46 crc kubenswrapper[5052]: I1203 11:37:46.703390 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a30eaf15-65ff-426b-8ac2-1ed5a369886a-config\") pod \"ovn-northd-0\" (UID: \"a30eaf15-65ff-426b-8ac2-1ed5a369886a\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 11:37:46 crc kubenswrapper[5052]: I1203 11:37:46.711244 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a30eaf15-65ff-426b-8ac2-1ed5a369886a-combined-ca-bundle\") pod \"ovn-northd-0\" (UID: \"a30eaf15-65ff-426b-8ac2-1ed5a369886a\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 11:37:46 crc kubenswrapper[5052]: I1203 11:37:46.711293 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovn-northd-tls-certs\" (UniqueName: \"kubernetes.io/secret/a30eaf15-65ff-426b-8ac2-1ed5a369886a-ovn-northd-tls-certs\") pod \"ovn-northd-0\" (UID: \"a30eaf15-65ff-426b-8ac2-1ed5a369886a\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 11:37:46 crc kubenswrapper[5052]: I1203 11:37:46.712404 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/a30eaf15-65ff-426b-8ac2-1ed5a369886a-metrics-certs-tls-certs\") pod \"ovn-northd-0\" (UID: \"a30eaf15-65ff-426b-8ac2-1ed5a369886a\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 11:37:46 crc kubenswrapper[5052]: I1203 11:37:46.719613 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-d5lbp\" (UniqueName: \"kubernetes.io/projected/a30eaf15-65ff-426b-8ac2-1ed5a369886a-kube-api-access-d5lbp\") pod \"ovn-northd-0\" (UID: \"a30eaf15-65ff-426b-8ac2-1ed5a369886a\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 11:37:46 crc kubenswrapper[5052]: I1203 11:37:46.823961 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 11:37:47 crc kubenswrapper[5052]: I1203 11:37:47.121958 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 11:37:47 crc kubenswrapper[5052]: W1203 11:37:47.270000 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda30eaf15_65ff_426b_8ac2_1ed5a369886a.slice/crio-551c2741749652575c73fe7f2d226f491f86182f4b8934b424925c7a1cca852e WatchSource:0}: Error finding container 551c2741749652575c73fe7f2d226f491f86182f4b8934b424925c7a1cca852e: Status 404 returned error can't find the container with id 551c2741749652575c73fe7f2d226f491f86182f4b8934b424925c7a1cca852e Dec 03 11:37:47 crc kubenswrapper[5052]: I1203 11:37:47.275302 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ovn-northd-0"] Dec 03 11:37:48 crc kubenswrapper[5052]: I1203 11:37:48.209733 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/swift-storage-0"] Dec 03 11:37:48 crc kubenswrapper[5052]: I1203 11:37:48.220605 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-storage-0" Dec 03 11:37:48 crc kubenswrapper[5052]: I1203 11:37:48.224938 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"swift-conf" Dec 03 11:37:48 crc kubenswrapper[5052]: I1203 11:37:48.225245 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"swift-swift-dockercfg-rc2b7" Dec 03 11:37:48 crc kubenswrapper[5052]: I1203 11:37:48.225482 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"swift-ring-files" Dec 03 11:37:48 crc kubenswrapper[5052]: I1203 11:37:48.225771 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"swift-storage-config-data" Dec 03 11:37:48 crc kubenswrapper[5052]: I1203 11:37:48.244906 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/swift-storage-0"] Dec 03 11:37:48 crc kubenswrapper[5052]: I1203 11:37:48.269225 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovn-northd-0" event={"ID":"a30eaf15-65ff-426b-8ac2-1ed5a369886a","Type":"ContainerStarted","Data":"551c2741749652575c73fe7f2d226f491f86182f4b8934b424925c7a1cca852e"} Dec 03 11:37:48 crc kubenswrapper[5052]: I1203 11:37:48.331547 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/d19f0c21-bb49-40a1-87fa-e55188308729-etc-swift\") pod \"swift-storage-0\" (UID: \"d19f0c21-bb49-40a1-87fa-e55188308729\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 11:37:48 crc kubenswrapper[5052]: I1203 11:37:48.331616 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zk5pc\" (UniqueName: \"kubernetes.io/projected/d19f0c21-bb49-40a1-87fa-e55188308729-kube-api-access-zk5pc\") pod \"swift-storage-0\" (UID: \"d19f0c21-bb49-40a1-87fa-e55188308729\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 11:37:48 crc kubenswrapper[5052]: I1203 11:37:48.331669 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage02-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage02-crc\") pod \"swift-storage-0\" (UID: \"d19f0c21-bb49-40a1-87fa-e55188308729\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 11:37:48 crc kubenswrapper[5052]: I1203 11:37:48.331722 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cache\" (UniqueName: \"kubernetes.io/empty-dir/d19f0c21-bb49-40a1-87fa-e55188308729-cache\") pod \"swift-storage-0\" (UID: \"d19f0c21-bb49-40a1-87fa-e55188308729\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 11:37:48 crc kubenswrapper[5052]: I1203 11:37:48.331904 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lock\" (UniqueName: \"kubernetes.io/empty-dir/d19f0c21-bb49-40a1-87fa-e55188308729-lock\") pod \"swift-storage-0\" (UID: \"d19f0c21-bb49-40a1-87fa-e55188308729\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 11:37:48 crc kubenswrapper[5052]: I1203 11:37:48.433291 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/d19f0c21-bb49-40a1-87fa-e55188308729-etc-swift\") pod \"swift-storage-0\" (UID: \"d19f0c21-bb49-40a1-87fa-e55188308729\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 11:37:48 crc kubenswrapper[5052]: I1203 11:37:48.433348 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-zk5pc\" (UniqueName: \"kubernetes.io/projected/d19f0c21-bb49-40a1-87fa-e55188308729-kube-api-access-zk5pc\") pod \"swift-storage-0\" (UID: \"d19f0c21-bb49-40a1-87fa-e55188308729\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 11:37:48 crc kubenswrapper[5052]: I1203 11:37:48.433388 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage02-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage02-crc\") pod \"swift-storage-0\" (UID: \"d19f0c21-bb49-40a1-87fa-e55188308729\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 11:37:48 crc kubenswrapper[5052]: I1203 11:37:48.433410 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"cache\" (UniqueName: \"kubernetes.io/empty-dir/d19f0c21-bb49-40a1-87fa-e55188308729-cache\") pod \"swift-storage-0\" (UID: \"d19f0c21-bb49-40a1-87fa-e55188308729\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 11:37:48 crc kubenswrapper[5052]: I1203 11:37:48.433468 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"lock\" (UniqueName: \"kubernetes.io/empty-dir/d19f0c21-bb49-40a1-87fa-e55188308729-lock\") pod \"swift-storage-0\" (UID: \"d19f0c21-bb49-40a1-87fa-e55188308729\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 11:37:48 crc kubenswrapper[5052]: E1203 11:37:48.433506 5052 projected.go:288] Couldn't get configMap openstack-kuttl-tests/swift-ring-files: configmap "swift-ring-files" not found Dec 03 11:37:48 crc kubenswrapper[5052]: E1203 11:37:48.433560 5052 projected.go:194] Error preparing data for projected volume etc-swift for pod openstack-kuttl-tests/swift-storage-0: configmap "swift-ring-files" not found Dec 03 11:37:48 crc kubenswrapper[5052]: E1203 11:37:48.433629 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d19f0c21-bb49-40a1-87fa-e55188308729-etc-swift podName:d19f0c21-bb49-40a1-87fa-e55188308729 nodeName:}" failed. No retries permitted until 2025-12-03 11:37:48.933605023 +0000 UTC m=+1059.521846877 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "etc-swift" (UniqueName: "kubernetes.io/projected/d19f0c21-bb49-40a1-87fa-e55188308729-etc-swift") pod "swift-storage-0" (UID: "d19f0c21-bb49-40a1-87fa-e55188308729") : configmap "swift-ring-files" not found Dec 03 11:37:48 crc kubenswrapper[5052]: I1203 11:37:48.433975 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"lock\" (UniqueName: \"kubernetes.io/empty-dir/d19f0c21-bb49-40a1-87fa-e55188308729-lock\") pod \"swift-storage-0\" (UID: \"d19f0c21-bb49-40a1-87fa-e55188308729\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 11:37:48 crc kubenswrapper[5052]: I1203 11:37:48.434217 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage02-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage02-crc\") pod \"swift-storage-0\" (UID: \"d19f0c21-bb49-40a1-87fa-e55188308729\") device mount path \"/mnt/openstack/pv02\"" pod="openstack-kuttl-tests/swift-storage-0" Dec 03 11:37:48 crc kubenswrapper[5052]: I1203 11:37:48.434283 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"cache\" (UniqueName: \"kubernetes.io/empty-dir/d19f0c21-bb49-40a1-87fa-e55188308729-cache\") pod \"swift-storage-0\" (UID: \"d19f0c21-bb49-40a1-87fa-e55188308729\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 11:37:48 crc kubenswrapper[5052]: I1203 11:37:48.468940 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-zk5pc\" (UniqueName: \"kubernetes.io/projected/d19f0c21-bb49-40a1-87fa-e55188308729-kube-api-access-zk5pc\") pod \"swift-storage-0\" (UID: \"d19f0c21-bb49-40a1-87fa-e55188308729\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 11:37:48 crc kubenswrapper[5052]: I1203 11:37:48.471832 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage02-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage02-crc\") pod \"swift-storage-0\" (UID: \"d19f0c21-bb49-40a1-87fa-e55188308729\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 11:37:48 crc kubenswrapper[5052]: I1203 11:37:48.719394 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/swift-ring-rebalance-krb94"] Dec 03 11:37:48 crc kubenswrapper[5052]: I1203 11:37:48.721112 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-ring-rebalance-krb94" Dec 03 11:37:48 crc kubenswrapper[5052]: I1203 11:37:48.723864 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"swift-proxy-config-data" Dec 03 11:37:48 crc kubenswrapper[5052]: I1203 11:37:48.724453 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"swift-ring-config-data" Dec 03 11:37:48 crc kubenswrapper[5052]: I1203 11:37:48.724612 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"swift-ring-scripts" Dec 03 11:37:48 crc kubenswrapper[5052]: I1203 11:37:48.728936 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/swift-ring-rebalance-krb94"] Dec 03 11:37:48 crc kubenswrapper[5052]: I1203 11:37:48.845956 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/empty-dir/ff7c4fed-d9f0-4220-9c04-33fae89bacd2-etc-swift\") pod \"swift-ring-rebalance-krb94\" (UID: \"ff7c4fed-d9f0-4220-9c04-33fae89bacd2\") " pod="openstack-kuttl-tests/swift-ring-rebalance-krb94" Dec 03 11:37:48 crc kubenswrapper[5052]: I1203 11:37:48.846597 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ff7c4fed-d9f0-4220-9c04-33fae89bacd2-combined-ca-bundle\") pod \"swift-ring-rebalance-krb94\" (UID: \"ff7c4fed-d9f0-4220-9c04-33fae89bacd2\") " pod="openstack-kuttl-tests/swift-ring-rebalance-krb94" Dec 03 11:37:48 crc kubenswrapper[5052]: I1203 11:37:48.846650 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lbw7m\" (UniqueName: \"kubernetes.io/projected/ff7c4fed-d9f0-4220-9c04-33fae89bacd2-kube-api-access-lbw7m\") pod \"swift-ring-rebalance-krb94\" (UID: \"ff7c4fed-d9f0-4220-9c04-33fae89bacd2\") " pod="openstack-kuttl-tests/swift-ring-rebalance-krb94" Dec 03 11:37:48 crc kubenswrapper[5052]: I1203 11:37:48.846852 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"swiftconf\" (UniqueName: \"kubernetes.io/secret/ff7c4fed-d9f0-4220-9c04-33fae89bacd2-swiftconf\") pod \"swift-ring-rebalance-krb94\" (UID: \"ff7c4fed-d9f0-4220-9c04-33fae89bacd2\") " pod="openstack-kuttl-tests/swift-ring-rebalance-krb94" Dec 03 11:37:48 crc kubenswrapper[5052]: I1203 11:37:48.846959 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ring-data-devices\" (UniqueName: \"kubernetes.io/configmap/ff7c4fed-d9f0-4220-9c04-33fae89bacd2-ring-data-devices\") pod \"swift-ring-rebalance-krb94\" (UID: \"ff7c4fed-d9f0-4220-9c04-33fae89bacd2\") " pod="openstack-kuttl-tests/swift-ring-rebalance-krb94" Dec 03 11:37:48 crc kubenswrapper[5052]: I1203 11:37:48.847010 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dispersionconf\" (UniqueName: \"kubernetes.io/secret/ff7c4fed-d9f0-4220-9c04-33fae89bacd2-dispersionconf\") pod \"swift-ring-rebalance-krb94\" (UID: \"ff7c4fed-d9f0-4220-9c04-33fae89bacd2\") " pod="openstack-kuttl-tests/swift-ring-rebalance-krb94" Dec 03 11:37:48 crc kubenswrapper[5052]: I1203 11:37:48.847050 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/ff7c4fed-d9f0-4220-9c04-33fae89bacd2-scripts\") pod \"swift-ring-rebalance-krb94\" (UID: \"ff7c4fed-d9f0-4220-9c04-33fae89bacd2\") " pod="openstack-kuttl-tests/swift-ring-rebalance-krb94" Dec 03 11:37:48 crc kubenswrapper[5052]: I1203 11:37:48.948852 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/empty-dir/ff7c4fed-d9f0-4220-9c04-33fae89bacd2-etc-swift\") pod \"swift-ring-rebalance-krb94\" (UID: \"ff7c4fed-d9f0-4220-9c04-33fae89bacd2\") " pod="openstack-kuttl-tests/swift-ring-rebalance-krb94" Dec 03 11:37:48 crc kubenswrapper[5052]: I1203 11:37:48.948917 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ff7c4fed-d9f0-4220-9c04-33fae89bacd2-combined-ca-bundle\") pod \"swift-ring-rebalance-krb94\" (UID: \"ff7c4fed-d9f0-4220-9c04-33fae89bacd2\") " pod="openstack-kuttl-tests/swift-ring-rebalance-krb94" Dec 03 11:37:48 crc kubenswrapper[5052]: I1203 11:37:48.948959 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-lbw7m\" (UniqueName: \"kubernetes.io/projected/ff7c4fed-d9f0-4220-9c04-33fae89bacd2-kube-api-access-lbw7m\") pod \"swift-ring-rebalance-krb94\" (UID: \"ff7c4fed-d9f0-4220-9c04-33fae89bacd2\") " pod="openstack-kuttl-tests/swift-ring-rebalance-krb94" Dec 03 11:37:48 crc kubenswrapper[5052]: I1203 11:37:48.948986 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"swiftconf\" (UniqueName: \"kubernetes.io/secret/ff7c4fed-d9f0-4220-9c04-33fae89bacd2-swiftconf\") pod \"swift-ring-rebalance-krb94\" (UID: \"ff7c4fed-d9f0-4220-9c04-33fae89bacd2\") " pod="openstack-kuttl-tests/swift-ring-rebalance-krb94" Dec 03 11:37:48 crc kubenswrapper[5052]: I1203 11:37:48.949024 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ring-data-devices\" (UniqueName: \"kubernetes.io/configmap/ff7c4fed-d9f0-4220-9c04-33fae89bacd2-ring-data-devices\") pod \"swift-ring-rebalance-krb94\" (UID: \"ff7c4fed-d9f0-4220-9c04-33fae89bacd2\") " pod="openstack-kuttl-tests/swift-ring-rebalance-krb94" Dec 03 11:37:48 crc kubenswrapper[5052]: I1203 11:37:48.949050 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"dispersionconf\" (UniqueName: \"kubernetes.io/secret/ff7c4fed-d9f0-4220-9c04-33fae89bacd2-dispersionconf\") pod \"swift-ring-rebalance-krb94\" (UID: \"ff7c4fed-d9f0-4220-9c04-33fae89bacd2\") " pod="openstack-kuttl-tests/swift-ring-rebalance-krb94" Dec 03 11:37:48 crc kubenswrapper[5052]: I1203 11:37:48.949076 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/ff7c4fed-d9f0-4220-9c04-33fae89bacd2-scripts\") pod \"swift-ring-rebalance-krb94\" (UID: \"ff7c4fed-d9f0-4220-9c04-33fae89bacd2\") " pod="openstack-kuttl-tests/swift-ring-rebalance-krb94" Dec 03 11:37:48 crc kubenswrapper[5052]: I1203 11:37:48.949129 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/d19f0c21-bb49-40a1-87fa-e55188308729-etc-swift\") pod \"swift-storage-0\" (UID: \"d19f0c21-bb49-40a1-87fa-e55188308729\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 11:37:48 crc kubenswrapper[5052]: E1203 11:37:48.949307 5052 projected.go:288] Couldn't get configMap openstack-kuttl-tests/swift-ring-files: configmap "swift-ring-files" not found Dec 03 11:37:48 crc kubenswrapper[5052]: E1203 11:37:48.949326 5052 projected.go:194] Error preparing data for projected volume etc-swift for pod openstack-kuttl-tests/swift-storage-0: configmap "swift-ring-files" not found Dec 03 11:37:48 crc kubenswrapper[5052]: E1203 11:37:48.949385 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d19f0c21-bb49-40a1-87fa-e55188308729-etc-swift podName:d19f0c21-bb49-40a1-87fa-e55188308729 nodeName:}" failed. No retries permitted until 2025-12-03 11:37:49.949368653 +0000 UTC m=+1060.537610437 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "etc-swift" (UniqueName: "kubernetes.io/projected/d19f0c21-bb49-40a1-87fa-e55188308729-etc-swift") pod "swift-storage-0" (UID: "d19f0c21-bb49-40a1-87fa-e55188308729") : configmap "swift-ring-files" not found Dec 03 11:37:48 crc kubenswrapper[5052]: I1203 11:37:48.949814 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"etc-swift\" (UniqueName: \"kubernetes.io/empty-dir/ff7c4fed-d9f0-4220-9c04-33fae89bacd2-etc-swift\") pod \"swift-ring-rebalance-krb94\" (UID: \"ff7c4fed-d9f0-4220-9c04-33fae89bacd2\") " pod="openstack-kuttl-tests/swift-ring-rebalance-krb94" Dec 03 11:37:48 crc kubenswrapper[5052]: I1203 11:37:48.950862 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ring-data-devices\" (UniqueName: \"kubernetes.io/configmap/ff7c4fed-d9f0-4220-9c04-33fae89bacd2-ring-data-devices\") pod \"swift-ring-rebalance-krb94\" (UID: \"ff7c4fed-d9f0-4220-9c04-33fae89bacd2\") " pod="openstack-kuttl-tests/swift-ring-rebalance-krb94" Dec 03 11:37:48 crc kubenswrapper[5052]: I1203 11:37:48.953878 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ff7c4fed-d9f0-4220-9c04-33fae89bacd2-combined-ca-bundle\") pod \"swift-ring-rebalance-krb94\" (UID: \"ff7c4fed-d9f0-4220-9c04-33fae89bacd2\") " pod="openstack-kuttl-tests/swift-ring-rebalance-krb94" Dec 03 11:37:48 crc kubenswrapper[5052]: I1203 11:37:48.954016 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/ff7c4fed-d9f0-4220-9c04-33fae89bacd2-scripts\") pod \"swift-ring-rebalance-krb94\" (UID: \"ff7c4fed-d9f0-4220-9c04-33fae89bacd2\") " pod="openstack-kuttl-tests/swift-ring-rebalance-krb94" Dec 03 11:37:48 crc kubenswrapper[5052]: I1203 11:37:48.956610 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"swiftconf\" (UniqueName: \"kubernetes.io/secret/ff7c4fed-d9f0-4220-9c04-33fae89bacd2-swiftconf\") pod \"swift-ring-rebalance-krb94\" (UID: \"ff7c4fed-d9f0-4220-9c04-33fae89bacd2\") " pod="openstack-kuttl-tests/swift-ring-rebalance-krb94" Dec 03 11:37:48 crc kubenswrapper[5052]: I1203 11:37:48.962165 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"dispersionconf\" (UniqueName: \"kubernetes.io/secret/ff7c4fed-d9f0-4220-9c04-33fae89bacd2-dispersionconf\") pod \"swift-ring-rebalance-krb94\" (UID: \"ff7c4fed-d9f0-4220-9c04-33fae89bacd2\") " pod="openstack-kuttl-tests/swift-ring-rebalance-krb94" Dec 03 11:37:48 crc kubenswrapper[5052]: I1203 11:37:48.980436 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-lbw7m\" (UniqueName: \"kubernetes.io/projected/ff7c4fed-d9f0-4220-9c04-33fae89bacd2-kube-api-access-lbw7m\") pod \"swift-ring-rebalance-krb94\" (UID: \"ff7c4fed-d9f0-4220-9c04-33fae89bacd2\") " pod="openstack-kuttl-tests/swift-ring-rebalance-krb94" Dec 03 11:37:49 crc kubenswrapper[5052]: I1203 11:37:49.056884 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-ring-rebalance-krb94" Dec 03 11:37:49 crc kubenswrapper[5052]: I1203 11:37:49.545623 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/swift-ring-rebalance-krb94"] Dec 03 11:37:49 crc kubenswrapper[5052]: I1203 11:37:49.966197 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/d19f0c21-bb49-40a1-87fa-e55188308729-etc-swift\") pod \"swift-storage-0\" (UID: \"d19f0c21-bb49-40a1-87fa-e55188308729\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 11:37:49 crc kubenswrapper[5052]: E1203 11:37:49.966383 5052 projected.go:288] Couldn't get configMap openstack-kuttl-tests/swift-ring-files: configmap "swift-ring-files" not found Dec 03 11:37:49 crc kubenswrapper[5052]: E1203 11:37:49.966400 5052 projected.go:194] Error preparing data for projected volume etc-swift for pod openstack-kuttl-tests/swift-storage-0: configmap "swift-ring-files" not found Dec 03 11:37:49 crc kubenswrapper[5052]: E1203 11:37:49.966449 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d19f0c21-bb49-40a1-87fa-e55188308729-etc-swift podName:d19f0c21-bb49-40a1-87fa-e55188308729 nodeName:}" failed. No retries permitted until 2025-12-03 11:37:51.966433475 +0000 UTC m=+1062.554675239 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "etc-swift" (UniqueName: "kubernetes.io/projected/d19f0c21-bb49-40a1-87fa-e55188308729-etc-swift") pod "swift-storage-0" (UID: "d19f0c21-bb49-40a1-87fa-e55188308729") : configmap "swift-ring-files" not found Dec 03 11:37:50 crc kubenswrapper[5052]: I1203 11:37:50.297756 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-ring-rebalance-krb94" event={"ID":"ff7c4fed-d9f0-4220-9c04-33fae89bacd2","Type":"ContainerStarted","Data":"cc107403ddb2466ff8b9de6b126c9612db8e2115d5f9ee2b4673f7fec3ada22e"} Dec 03 11:37:50 crc kubenswrapper[5052]: I1203 11:37:50.753578 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-78bj2" Dec 03 11:37:50 crc kubenswrapper[5052]: I1203 11:37:50.811135 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-f5849d7b9-nrdjn"] Dec 03 11:37:52 crc kubenswrapper[5052]: I1203 11:37:52.006674 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/d19f0c21-bb49-40a1-87fa-e55188308729-etc-swift\") pod \"swift-storage-0\" (UID: \"d19f0c21-bb49-40a1-87fa-e55188308729\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 11:37:52 crc kubenswrapper[5052]: E1203 11:37:52.006842 5052 projected.go:288] Couldn't get configMap openstack-kuttl-tests/swift-ring-files: configmap "swift-ring-files" not found Dec 03 11:37:52 crc kubenswrapper[5052]: E1203 11:37:52.007033 5052 projected.go:194] Error preparing data for projected volume etc-swift for pod openstack-kuttl-tests/swift-storage-0: configmap "swift-ring-files" not found Dec 03 11:37:52 crc kubenswrapper[5052]: E1203 11:37:52.007092 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d19f0c21-bb49-40a1-87fa-e55188308729-etc-swift podName:d19f0c21-bb49-40a1-87fa-e55188308729 nodeName:}" failed. No retries permitted until 2025-12-03 11:37:56.007074608 +0000 UTC m=+1066.595316372 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "etc-swift" (UniqueName: "kubernetes.io/projected/d19f0c21-bb49-40a1-87fa-e55188308729-etc-swift") pod "swift-storage-0" (UID: "d19f0c21-bb49-40a1-87fa-e55188308729") : configmap "swift-ring-files" not found Dec 03 11:37:52 crc kubenswrapper[5052]: I1203 11:37:52.316329 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-f5849d7b9-nrdjn" event={"ID":"897024db-e95e-4ab8-868f-ddb1ffadcbed","Type":"ContainerStarted","Data":"f73f570696421fcb689ea2984aaffd72ca565a981af3a331d44c08a90a5466e5"} Dec 03 11:37:52 crc kubenswrapper[5052]: I1203 11:37:52.316630 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-f5849d7b9-nrdjn" podUID="897024db-e95e-4ab8-868f-ddb1ffadcbed" containerName="dnsmasq-dns" containerID="cri-o://f73f570696421fcb689ea2984aaffd72ca565a981af3a331d44c08a90a5466e5" gracePeriod=10 Dec 03 11:37:52 crc kubenswrapper[5052]: I1203 11:37:52.316746 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-f5849d7b9-nrdjn" Dec 03 11:37:52 crc kubenswrapper[5052]: I1203 11:37:52.342870 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-f5849d7b9-nrdjn" podStartSLOduration=-9223371994.511951 podStartE2EDuration="42.342824886s" podCreationTimestamp="2025-12-03 11:37:10 +0000 UTC" firstStartedPulling="2025-12-03 11:37:11.088862968 +0000 UTC m=+1021.677104732" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:37:52.337906773 +0000 UTC m=+1062.926148557" watchObservedRunningTime="2025-12-03 11:37:52.342824886 +0000 UTC m=+1062.931066650" Dec 03 11:37:53 crc kubenswrapper[5052]: I1203 11:37:53.194324 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 11:37:53 crc kubenswrapper[5052]: I1203 11:37:53.264149 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 11:37:53 crc kubenswrapper[5052]: I1203 11:37:53.325974 5052 generic.go:334] "Generic (PLEG): container finished" podID="897024db-e95e-4ab8-868f-ddb1ffadcbed" containerID="f73f570696421fcb689ea2984aaffd72ca565a981af3a331d44c08a90a5466e5" exitCode=0 Dec 03 11:37:53 crc kubenswrapper[5052]: I1203 11:37:53.326095 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-f5849d7b9-nrdjn" event={"ID":"897024db-e95e-4ab8-868f-ddb1ffadcbed","Type":"ContainerDied","Data":"f73f570696421fcb689ea2984aaffd72ca565a981af3a331d44c08a90a5466e5"} Dec 03 11:37:54 crc kubenswrapper[5052]: I1203 11:37:54.904550 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-f5849d7b9-nrdjn" Dec 03 11:37:55 crc kubenswrapper[5052]: I1203 11:37:55.015055 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/keystone-c8af-account-create-update-fcv24"] Dec 03 11:37:55 crc kubenswrapper[5052]: E1203 11:37:55.015495 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="897024db-e95e-4ab8-868f-ddb1ffadcbed" containerName="dnsmasq-dns" Dec 03 11:37:55 crc kubenswrapper[5052]: I1203 11:37:55.015533 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="897024db-e95e-4ab8-868f-ddb1ffadcbed" containerName="dnsmasq-dns" Dec 03 11:37:55 crc kubenswrapper[5052]: E1203 11:37:55.015579 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="897024db-e95e-4ab8-868f-ddb1ffadcbed" containerName="init" Dec 03 11:37:55 crc kubenswrapper[5052]: I1203 11:37:55.015588 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="897024db-e95e-4ab8-868f-ddb1ffadcbed" containerName="init" Dec 03 11:37:55 crc kubenswrapper[5052]: I1203 11:37:55.015797 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="897024db-e95e-4ab8-868f-ddb1ffadcbed" containerName="dnsmasq-dns" Dec 03 11:37:55 crc kubenswrapper[5052]: I1203 11:37:55.016411 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-c8af-account-create-update-fcv24" Dec 03 11:37:55 crc kubenswrapper[5052]: I1203 11:37:55.018710 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone-db-secret" Dec 03 11:37:55 crc kubenswrapper[5052]: I1203 11:37:55.022123 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-c8af-account-create-update-fcv24"] Dec 03 11:37:55 crc kubenswrapper[5052]: I1203 11:37:55.062283 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rd7b5\" (UniqueName: \"kubernetes.io/projected/897024db-e95e-4ab8-868f-ddb1ffadcbed-kube-api-access-rd7b5\") pod \"897024db-e95e-4ab8-868f-ddb1ffadcbed\" (UID: \"897024db-e95e-4ab8-868f-ddb1ffadcbed\") " Dec 03 11:37:55 crc kubenswrapper[5052]: I1203 11:37:55.063122 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/897024db-e95e-4ab8-868f-ddb1ffadcbed-config\") pod \"897024db-e95e-4ab8-868f-ddb1ffadcbed\" (UID: \"897024db-e95e-4ab8-868f-ddb1ffadcbed\") " Dec 03 11:37:55 crc kubenswrapper[5052]: I1203 11:37:55.066138 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/897024db-e95e-4ab8-868f-ddb1ffadcbed-kube-api-access-rd7b5" (OuterVolumeSpecName: "kube-api-access-rd7b5") pod "897024db-e95e-4ab8-868f-ddb1ffadcbed" (UID: "897024db-e95e-4ab8-868f-ddb1ffadcbed"). InnerVolumeSpecName "kube-api-access-rd7b5". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:37:55 crc kubenswrapper[5052]: I1203 11:37:55.113383 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/897024db-e95e-4ab8-868f-ddb1ffadcbed-config" (OuterVolumeSpecName: "config") pod "897024db-e95e-4ab8-868f-ddb1ffadcbed" (UID: "897024db-e95e-4ab8-868f-ddb1ffadcbed"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:37:55 crc kubenswrapper[5052]: I1203 11:37:55.121584 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/keystone-db-create-bfh9z"] Dec 03 11:37:55 crc kubenswrapper[5052]: I1203 11:37:55.122834 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-db-create-bfh9z" Dec 03 11:37:55 crc kubenswrapper[5052]: I1203 11:37:55.148654 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-db-create-bfh9z"] Dec 03 11:37:55 crc kubenswrapper[5052]: I1203 11:37:55.165803 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/23eed8d3-3d02-4f02-9cde-7e5dd4363b9b-operator-scripts\") pod \"keystone-c8af-account-create-update-fcv24\" (UID: \"23eed8d3-3d02-4f02-9cde-7e5dd4363b9b\") " pod="openstack-kuttl-tests/keystone-c8af-account-create-update-fcv24" Dec 03 11:37:55 crc kubenswrapper[5052]: I1203 11:37:55.165881 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xrxgf\" (UniqueName: \"kubernetes.io/projected/23eed8d3-3d02-4f02-9cde-7e5dd4363b9b-kube-api-access-xrxgf\") pod \"keystone-c8af-account-create-update-fcv24\" (UID: \"23eed8d3-3d02-4f02-9cde-7e5dd4363b9b\") " pod="openstack-kuttl-tests/keystone-c8af-account-create-update-fcv24" Dec 03 11:37:55 crc kubenswrapper[5052]: I1203 11:37:55.166220 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-rd7b5\" (UniqueName: \"kubernetes.io/projected/897024db-e95e-4ab8-868f-ddb1ffadcbed-kube-api-access-rd7b5\") on node \"crc\" DevicePath \"\"" Dec 03 11:37:55 crc kubenswrapper[5052]: I1203 11:37:55.166240 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/897024db-e95e-4ab8-868f-ddb1ffadcbed-config\") on node \"crc\" DevicePath \"\"" Dec 03 11:37:55 crc kubenswrapper[5052]: I1203 11:37:55.267497 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/69fe8db0-25e9-4f48-b9a2-77dc31224a34-operator-scripts\") pod \"keystone-db-create-bfh9z\" (UID: \"69fe8db0-25e9-4f48-b9a2-77dc31224a34\") " pod="openstack-kuttl-tests/keystone-db-create-bfh9z" Dec 03 11:37:55 crc kubenswrapper[5052]: I1203 11:37:55.267632 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/23eed8d3-3d02-4f02-9cde-7e5dd4363b9b-operator-scripts\") pod \"keystone-c8af-account-create-update-fcv24\" (UID: \"23eed8d3-3d02-4f02-9cde-7e5dd4363b9b\") " pod="openstack-kuttl-tests/keystone-c8af-account-create-update-fcv24" Dec 03 11:37:55 crc kubenswrapper[5052]: I1203 11:37:55.267697 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-xrxgf\" (UniqueName: \"kubernetes.io/projected/23eed8d3-3d02-4f02-9cde-7e5dd4363b9b-kube-api-access-xrxgf\") pod \"keystone-c8af-account-create-update-fcv24\" (UID: \"23eed8d3-3d02-4f02-9cde-7e5dd4363b9b\") " pod="openstack-kuttl-tests/keystone-c8af-account-create-update-fcv24" Dec 03 11:37:55 crc kubenswrapper[5052]: I1203 11:37:55.267752 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b58g8\" (UniqueName: \"kubernetes.io/projected/69fe8db0-25e9-4f48-b9a2-77dc31224a34-kube-api-access-b58g8\") pod \"keystone-db-create-bfh9z\" (UID: \"69fe8db0-25e9-4f48-b9a2-77dc31224a34\") " pod="openstack-kuttl-tests/keystone-db-create-bfh9z" Dec 03 11:37:55 crc kubenswrapper[5052]: I1203 11:37:55.268416 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/23eed8d3-3d02-4f02-9cde-7e5dd4363b9b-operator-scripts\") pod \"keystone-c8af-account-create-update-fcv24\" (UID: \"23eed8d3-3d02-4f02-9cde-7e5dd4363b9b\") " pod="openstack-kuttl-tests/keystone-c8af-account-create-update-fcv24" Dec 03 11:37:55 crc kubenswrapper[5052]: I1203 11:37:55.284965 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-xrxgf\" (UniqueName: \"kubernetes.io/projected/23eed8d3-3d02-4f02-9cde-7e5dd4363b9b-kube-api-access-xrxgf\") pod \"keystone-c8af-account-create-update-fcv24\" (UID: \"23eed8d3-3d02-4f02-9cde-7e5dd4363b9b\") " pod="openstack-kuttl-tests/keystone-c8af-account-create-update-fcv24" Dec 03 11:37:55 crc kubenswrapper[5052]: I1203 11:37:55.301727 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 11:37:55 crc kubenswrapper[5052]: I1203 11:37:55.306397 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/placement-db-create-zvq9n"] Dec 03 11:37:55 crc kubenswrapper[5052]: I1203 11:37:55.307616 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-db-create-zvq9n" Dec 03 11:37:55 crc kubenswrapper[5052]: I1203 11:37:55.343738 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-c8af-account-create-update-fcv24" Dec 03 11:37:55 crc kubenswrapper[5052]: I1203 11:37:55.344815 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/placement-db-create-zvq9n"] Dec 03 11:37:55 crc kubenswrapper[5052]: I1203 11:37:55.358143 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovn-northd-0" event={"ID":"a30eaf15-65ff-426b-8ac2-1ed5a369886a","Type":"ContainerStarted","Data":"abd32ba7a2fea1f9280ace25d3fe52938520b763a092b3e8d9e1b4a4f4e6e2be"} Dec 03 11:37:55 crc kubenswrapper[5052]: I1203 11:37:55.358181 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovn-northd-0" event={"ID":"a30eaf15-65ff-426b-8ac2-1ed5a369886a","Type":"ContainerStarted","Data":"455e8792612e9ef5fbedefc9390ff87ea262cbd9365b333801dd2dc51224737e"} Dec 03 11:37:55 crc kubenswrapper[5052]: I1203 11:37:55.358287 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 11:37:55 crc kubenswrapper[5052]: I1203 11:37:55.360424 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-f5849d7b9-nrdjn" event={"ID":"897024db-e95e-4ab8-868f-ddb1ffadcbed","Type":"ContainerDied","Data":"d9c2ed6f55687734f078b1c884c95198b8314d5c9ebf5654249ad8fe90445f9f"} Dec 03 11:37:55 crc kubenswrapper[5052]: I1203 11:37:55.360461 5052 scope.go:117] "RemoveContainer" containerID="f73f570696421fcb689ea2984aaffd72ca565a981af3a331d44c08a90a5466e5" Dec 03 11:37:55 crc kubenswrapper[5052]: I1203 11:37:55.360581 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-f5849d7b9-nrdjn" Dec 03 11:37:55 crc kubenswrapper[5052]: I1203 11:37:55.364198 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-ring-rebalance-krb94" event={"ID":"ff7c4fed-d9f0-4220-9c04-33fae89bacd2","Type":"ContainerStarted","Data":"f60cd400b6652773ef4753bb550741040c4ae2eb9c6c8a5ebbe3a84a430ead37"} Dec 03 11:37:55 crc kubenswrapper[5052]: I1203 11:37:55.369211 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-b58g8\" (UniqueName: \"kubernetes.io/projected/69fe8db0-25e9-4f48-b9a2-77dc31224a34-kube-api-access-b58g8\") pod \"keystone-db-create-bfh9z\" (UID: \"69fe8db0-25e9-4f48-b9a2-77dc31224a34\") " pod="openstack-kuttl-tests/keystone-db-create-bfh9z" Dec 03 11:37:55 crc kubenswrapper[5052]: I1203 11:37:55.369295 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/69fe8db0-25e9-4f48-b9a2-77dc31224a34-operator-scripts\") pod \"keystone-db-create-bfh9z\" (UID: \"69fe8db0-25e9-4f48-b9a2-77dc31224a34\") " pod="openstack-kuttl-tests/keystone-db-create-bfh9z" Dec 03 11:37:55 crc kubenswrapper[5052]: I1203 11:37:55.370151 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/69fe8db0-25e9-4f48-b9a2-77dc31224a34-operator-scripts\") pod \"keystone-db-create-bfh9z\" (UID: \"69fe8db0-25e9-4f48-b9a2-77dc31224a34\") " pod="openstack-kuttl-tests/keystone-db-create-bfh9z" Dec 03 11:37:55 crc kubenswrapper[5052]: I1203 11:37:55.378468 5052 scope.go:117] "RemoveContainer" containerID="552bd1c850f4094b35e2c2b531421785b4fc7d4b1201f8e7f2e52d15939f7a71" Dec 03 11:37:55 crc kubenswrapper[5052]: I1203 11:37:55.388249 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-b58g8\" (UniqueName: \"kubernetes.io/projected/69fe8db0-25e9-4f48-b9a2-77dc31224a34-kube-api-access-b58g8\") pod \"keystone-db-create-bfh9z\" (UID: \"69fe8db0-25e9-4f48-b9a2-77dc31224a34\") " pod="openstack-kuttl-tests/keystone-db-create-bfh9z" Dec 03 11:37:55 crc kubenswrapper[5052]: I1203 11:37:55.402091 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 11:37:55 crc kubenswrapper[5052]: I1203 11:37:55.404964 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/ovn-northd-0" podStartSLOduration=1.960635891 podStartE2EDuration="9.404942219s" podCreationTimestamp="2025-12-03 11:37:46 +0000 UTC" firstStartedPulling="2025-12-03 11:37:47.271959535 +0000 UTC m=+1057.860201289" lastFinishedPulling="2025-12-03 11:37:54.716265853 +0000 UTC m=+1065.304507617" observedRunningTime="2025-12-03 11:37:55.393995222 +0000 UTC m=+1065.982236986" watchObservedRunningTime="2025-12-03 11:37:55.404942219 +0000 UTC m=+1065.993183993" Dec 03 11:37:55 crc kubenswrapper[5052]: I1203 11:37:55.492037 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/fbef0d42-0ae8-4e90-bacb-081a4b1e8d47-operator-scripts\") pod \"placement-db-create-zvq9n\" (UID: \"fbef0d42-0ae8-4e90-bacb-081a4b1e8d47\") " pod="openstack-kuttl-tests/placement-db-create-zvq9n" Dec 03 11:37:55 crc kubenswrapper[5052]: I1203 11:37:55.492128 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4xgq7\" (UniqueName: \"kubernetes.io/projected/fbef0d42-0ae8-4e90-bacb-081a4b1e8d47-kube-api-access-4xgq7\") pod \"placement-db-create-zvq9n\" (UID: \"fbef0d42-0ae8-4e90-bacb-081a4b1e8d47\") " pod="openstack-kuttl-tests/placement-db-create-zvq9n" Dec 03 11:37:55 crc kubenswrapper[5052]: I1203 11:37:55.492601 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-db-create-bfh9z" Dec 03 11:37:55 crc kubenswrapper[5052]: I1203 11:37:55.496157 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/placement-79ca-account-create-update-xhl8v"] Dec 03 11:37:55 crc kubenswrapper[5052]: I1203 11:37:55.497411 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-79ca-account-create-update-xhl8v" Dec 03 11:37:55 crc kubenswrapper[5052]: I1203 11:37:55.506348 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"placement-db-secret" Dec 03 11:37:55 crc kubenswrapper[5052]: I1203 11:37:55.507817 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-f5849d7b9-nrdjn"] Dec 03 11:37:55 crc kubenswrapper[5052]: I1203 11:37:55.533862 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-f5849d7b9-nrdjn"] Dec 03 11:37:55 crc kubenswrapper[5052]: I1203 11:37:55.545033 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/placement-79ca-account-create-update-xhl8v"] Dec 03 11:37:55 crc kubenswrapper[5052]: I1203 11:37:55.559840 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/swift-ring-rebalance-krb94" podStartSLOduration=2.345512609 podStartE2EDuration="7.559807772s" podCreationTimestamp="2025-12-03 11:37:48 +0000 UTC" firstStartedPulling="2025-12-03 11:37:49.558458055 +0000 UTC m=+1060.146699819" lastFinishedPulling="2025-12-03 11:37:54.772753208 +0000 UTC m=+1065.360994982" observedRunningTime="2025-12-03 11:37:55.433356791 +0000 UTC m=+1066.021598555" watchObservedRunningTime="2025-12-03 11:37:55.559807772 +0000 UTC m=+1066.148049536" Dec 03 11:37:55 crc kubenswrapper[5052]: I1203 11:37:55.594322 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/fbef0d42-0ae8-4e90-bacb-081a4b1e8d47-operator-scripts\") pod \"placement-db-create-zvq9n\" (UID: \"fbef0d42-0ae8-4e90-bacb-081a4b1e8d47\") " pod="openstack-kuttl-tests/placement-db-create-zvq9n" Dec 03 11:37:55 crc kubenswrapper[5052]: I1203 11:37:55.594407 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s2kd2\" (UniqueName: \"kubernetes.io/projected/c3452ae3-e803-4020-b5db-3ddbdf4cba75-kube-api-access-s2kd2\") pod \"placement-79ca-account-create-update-xhl8v\" (UID: \"c3452ae3-e803-4020-b5db-3ddbdf4cba75\") " pod="openstack-kuttl-tests/placement-79ca-account-create-update-xhl8v" Dec 03 11:37:55 crc kubenswrapper[5052]: I1203 11:37:55.594444 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/c3452ae3-e803-4020-b5db-3ddbdf4cba75-operator-scripts\") pod \"placement-79ca-account-create-update-xhl8v\" (UID: \"c3452ae3-e803-4020-b5db-3ddbdf4cba75\") " pod="openstack-kuttl-tests/placement-79ca-account-create-update-xhl8v" Dec 03 11:37:55 crc kubenswrapper[5052]: I1203 11:37:55.594615 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-4xgq7\" (UniqueName: \"kubernetes.io/projected/fbef0d42-0ae8-4e90-bacb-081a4b1e8d47-kube-api-access-4xgq7\") pod \"placement-db-create-zvq9n\" (UID: \"fbef0d42-0ae8-4e90-bacb-081a4b1e8d47\") " pod="openstack-kuttl-tests/placement-db-create-zvq9n" Dec 03 11:37:55 crc kubenswrapper[5052]: I1203 11:37:55.595425 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/fbef0d42-0ae8-4e90-bacb-081a4b1e8d47-operator-scripts\") pod \"placement-db-create-zvq9n\" (UID: \"fbef0d42-0ae8-4e90-bacb-081a4b1e8d47\") " pod="openstack-kuttl-tests/placement-db-create-zvq9n" Dec 03 11:37:55 crc kubenswrapper[5052]: I1203 11:37:55.618267 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-4xgq7\" (UniqueName: \"kubernetes.io/projected/fbef0d42-0ae8-4e90-bacb-081a4b1e8d47-kube-api-access-4xgq7\") pod \"placement-db-create-zvq9n\" (UID: \"fbef0d42-0ae8-4e90-bacb-081a4b1e8d47\") " pod="openstack-kuttl-tests/placement-db-create-zvq9n" Dec 03 11:37:55 crc kubenswrapper[5052]: I1203 11:37:55.625379 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-db-create-zvq9n" Dec 03 11:37:55 crc kubenswrapper[5052]: I1203 11:37:55.690440 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="897024db-e95e-4ab8-868f-ddb1ffadcbed" path="/var/lib/kubelet/pods/897024db-e95e-4ab8-868f-ddb1ffadcbed/volumes" Dec 03 11:37:55 crc kubenswrapper[5052]: I1203 11:37:55.696797 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-s2kd2\" (UniqueName: \"kubernetes.io/projected/c3452ae3-e803-4020-b5db-3ddbdf4cba75-kube-api-access-s2kd2\") pod \"placement-79ca-account-create-update-xhl8v\" (UID: \"c3452ae3-e803-4020-b5db-3ddbdf4cba75\") " pod="openstack-kuttl-tests/placement-79ca-account-create-update-xhl8v" Dec 03 11:37:55 crc kubenswrapper[5052]: I1203 11:37:55.696837 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/c3452ae3-e803-4020-b5db-3ddbdf4cba75-operator-scripts\") pod \"placement-79ca-account-create-update-xhl8v\" (UID: \"c3452ae3-e803-4020-b5db-3ddbdf4cba75\") " pod="openstack-kuttl-tests/placement-79ca-account-create-update-xhl8v" Dec 03 11:37:55 crc kubenswrapper[5052]: I1203 11:37:55.697709 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/c3452ae3-e803-4020-b5db-3ddbdf4cba75-operator-scripts\") pod \"placement-79ca-account-create-update-xhl8v\" (UID: \"c3452ae3-e803-4020-b5db-3ddbdf4cba75\") " pod="openstack-kuttl-tests/placement-79ca-account-create-update-xhl8v" Dec 03 11:37:55 crc kubenswrapper[5052]: I1203 11:37:55.722359 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-s2kd2\" (UniqueName: \"kubernetes.io/projected/c3452ae3-e803-4020-b5db-3ddbdf4cba75-kube-api-access-s2kd2\") pod \"placement-79ca-account-create-update-xhl8v\" (UID: \"c3452ae3-e803-4020-b5db-3ddbdf4cba75\") " pod="openstack-kuttl-tests/placement-79ca-account-create-update-xhl8v" Dec 03 11:37:55 crc kubenswrapper[5052]: I1203 11:37:55.834075 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-79ca-account-create-update-xhl8v" Dec 03 11:37:55 crc kubenswrapper[5052]: I1203 11:37:55.881779 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-c8af-account-create-update-fcv24"] Dec 03 11:37:56 crc kubenswrapper[5052]: I1203 11:37:56.012169 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/d19f0c21-bb49-40a1-87fa-e55188308729-etc-swift\") pod \"swift-storage-0\" (UID: \"d19f0c21-bb49-40a1-87fa-e55188308729\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 11:37:56 crc kubenswrapper[5052]: E1203 11:37:56.012388 5052 projected.go:288] Couldn't get configMap openstack-kuttl-tests/swift-ring-files: configmap "swift-ring-files" not found Dec 03 11:37:56 crc kubenswrapper[5052]: E1203 11:37:56.013135 5052 projected.go:194] Error preparing data for projected volume etc-swift for pod openstack-kuttl-tests/swift-storage-0: configmap "swift-ring-files" not found Dec 03 11:37:56 crc kubenswrapper[5052]: E1203 11:37:56.013200 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d19f0c21-bb49-40a1-87fa-e55188308729-etc-swift podName:d19f0c21-bb49-40a1-87fa-e55188308729 nodeName:}" failed. No retries permitted until 2025-12-03 11:38:04.013177035 +0000 UTC m=+1074.601418799 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "etc-swift" (UniqueName: "kubernetes.io/projected/d19f0c21-bb49-40a1-87fa-e55188308729-etc-swift") pod "swift-storage-0" (UID: "d19f0c21-bb49-40a1-87fa-e55188308729") : configmap "swift-ring-files" not found Dec 03 11:37:56 crc kubenswrapper[5052]: I1203 11:37:56.027676 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-db-create-bfh9z"] Dec 03 11:37:56 crc kubenswrapper[5052]: W1203 11:37:56.055074 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod69fe8db0_25e9_4f48_b9a2_77dc31224a34.slice/crio-1268573c1030df2e6e2bea83cb86bedf16f335320a5b19fe20a9ce04fc357080 WatchSource:0}: Error finding container 1268573c1030df2e6e2bea83cb86bedf16f335320a5b19fe20a9ce04fc357080: Status 404 returned error can't find the container with id 1268573c1030df2e6e2bea83cb86bedf16f335320a5b19fe20a9ce04fc357080 Dec 03 11:37:56 crc kubenswrapper[5052]: I1203 11:37:56.096346 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/placement-db-create-zvq9n"] Dec 03 11:37:56 crc kubenswrapper[5052]: W1203 11:37:56.105971 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfbef0d42_0ae8_4e90_bacb_081a4b1e8d47.slice/crio-10ba1e01541849ab298290759edc46053781685a4229f3aa27b653b6861f8f65 WatchSource:0}: Error finding container 10ba1e01541849ab298290759edc46053781685a4229f3aa27b653b6861f8f65: Status 404 returned error can't find the container with id 10ba1e01541849ab298290759edc46053781685a4229f3aa27b653b6861f8f65 Dec 03 11:37:56 crc kubenswrapper[5052]: I1203 11:37:56.299800 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/placement-79ca-account-create-update-xhl8v"] Dec 03 11:37:56 crc kubenswrapper[5052]: W1203 11:37:56.357325 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc3452ae3_e803_4020_b5db_3ddbdf4cba75.slice/crio-44ff0b6775181abf6e7cb2d5bdb02d8ded80e32a5ecd9371181beab3128dad0b WatchSource:0}: Error finding container 44ff0b6775181abf6e7cb2d5bdb02d8ded80e32a5ecd9371181beab3128dad0b: Status 404 returned error can't find the container with id 44ff0b6775181abf6e7cb2d5bdb02d8ded80e32a5ecd9371181beab3128dad0b Dec 03 11:37:56 crc kubenswrapper[5052]: I1203 11:37:56.371892 5052 generic.go:334] "Generic (PLEG): container finished" podID="23eed8d3-3d02-4f02-9cde-7e5dd4363b9b" containerID="92a6a8836b87d1315cfd2ea55a4b4571531256aa00150f268dbfca3e6eabb207" exitCode=0 Dec 03 11:37:56 crc kubenswrapper[5052]: I1203 11:37:56.371969 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-c8af-account-create-update-fcv24" event={"ID":"23eed8d3-3d02-4f02-9cde-7e5dd4363b9b","Type":"ContainerDied","Data":"92a6a8836b87d1315cfd2ea55a4b4571531256aa00150f268dbfca3e6eabb207"} Dec 03 11:37:56 crc kubenswrapper[5052]: I1203 11:37:56.371998 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-c8af-account-create-update-fcv24" event={"ID":"23eed8d3-3d02-4f02-9cde-7e5dd4363b9b","Type":"ContainerStarted","Data":"70491ae436faa3e034c7e6c1e0b4b447d205b78c41a44a2d994c81ce5dde91b6"} Dec 03 11:37:56 crc kubenswrapper[5052]: I1203 11:37:56.373400 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-db-create-zvq9n" event={"ID":"fbef0d42-0ae8-4e90-bacb-081a4b1e8d47","Type":"ContainerStarted","Data":"b6fc837cf63e99e9b463372d833ef8414220a2bac6210bbf692afd78142eaaf5"} Dec 03 11:37:56 crc kubenswrapper[5052]: I1203 11:37:56.373449 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-db-create-zvq9n" event={"ID":"fbef0d42-0ae8-4e90-bacb-081a4b1e8d47","Type":"ContainerStarted","Data":"10ba1e01541849ab298290759edc46053781685a4229f3aa27b653b6861f8f65"} Dec 03 11:37:56 crc kubenswrapper[5052]: I1203 11:37:56.375670 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-79ca-account-create-update-xhl8v" event={"ID":"c3452ae3-e803-4020-b5db-3ddbdf4cba75","Type":"ContainerStarted","Data":"44ff0b6775181abf6e7cb2d5bdb02d8ded80e32a5ecd9371181beab3128dad0b"} Dec 03 11:37:56 crc kubenswrapper[5052]: I1203 11:37:56.376755 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-db-create-bfh9z" event={"ID":"69fe8db0-25e9-4f48-b9a2-77dc31224a34","Type":"ContainerStarted","Data":"82e865c60792e8a1db857646f7e7a31829df3081386f9952e8ded31712c937e7"} Dec 03 11:37:56 crc kubenswrapper[5052]: I1203 11:37:56.376785 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-db-create-bfh9z" event={"ID":"69fe8db0-25e9-4f48-b9a2-77dc31224a34","Type":"ContainerStarted","Data":"1268573c1030df2e6e2bea83cb86bedf16f335320a5b19fe20a9ce04fc357080"} Dec 03 11:37:56 crc kubenswrapper[5052]: I1203 11:37:56.418654 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/placement-db-create-zvq9n" podStartSLOduration=1.418629102 podStartE2EDuration="1.418629102s" podCreationTimestamp="2025-12-03 11:37:55 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:37:56.409988932 +0000 UTC m=+1066.998230696" watchObservedRunningTime="2025-12-03 11:37:56.418629102 +0000 UTC m=+1067.006870866" Dec 03 11:37:56 crc kubenswrapper[5052]: I1203 11:37:56.432772 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/keystone-db-create-bfh9z" podStartSLOduration=1.432752281 podStartE2EDuration="1.432752281s" podCreationTimestamp="2025-12-03 11:37:55 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:37:56.427474038 +0000 UTC m=+1067.015715812" watchObservedRunningTime="2025-12-03 11:37:56.432752281 +0000 UTC m=+1067.020994045" Dec 03 11:37:57 crc kubenswrapper[5052]: I1203 11:37:57.391008 5052 generic.go:334] "Generic (PLEG): container finished" podID="fbef0d42-0ae8-4e90-bacb-081a4b1e8d47" containerID="b6fc837cf63e99e9b463372d833ef8414220a2bac6210bbf692afd78142eaaf5" exitCode=0 Dec 03 11:37:57 crc kubenswrapper[5052]: I1203 11:37:57.391061 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-db-create-zvq9n" event={"ID":"fbef0d42-0ae8-4e90-bacb-081a4b1e8d47","Type":"ContainerDied","Data":"b6fc837cf63e99e9b463372d833ef8414220a2bac6210bbf692afd78142eaaf5"} Dec 03 11:37:57 crc kubenswrapper[5052]: I1203 11:37:57.393142 5052 generic.go:334] "Generic (PLEG): container finished" podID="c3452ae3-e803-4020-b5db-3ddbdf4cba75" containerID="285f0301df3076e5804e7d1881c2e5541ac3790946cb4841c7f7ed169e3bdd4e" exitCode=0 Dec 03 11:37:57 crc kubenswrapper[5052]: I1203 11:37:57.393250 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-79ca-account-create-update-xhl8v" event={"ID":"c3452ae3-e803-4020-b5db-3ddbdf4cba75","Type":"ContainerDied","Data":"285f0301df3076e5804e7d1881c2e5541ac3790946cb4841c7f7ed169e3bdd4e"} Dec 03 11:37:57 crc kubenswrapper[5052]: I1203 11:37:57.394604 5052 generic.go:334] "Generic (PLEG): container finished" podID="69fe8db0-25e9-4f48-b9a2-77dc31224a34" containerID="82e865c60792e8a1db857646f7e7a31829df3081386f9952e8ded31712c937e7" exitCode=0 Dec 03 11:37:57 crc kubenswrapper[5052]: I1203 11:37:57.394828 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-db-create-bfh9z" event={"ID":"69fe8db0-25e9-4f48-b9a2-77dc31224a34","Type":"ContainerDied","Data":"82e865c60792e8a1db857646f7e7a31829df3081386f9952e8ded31712c937e7"} Dec 03 11:37:57 crc kubenswrapper[5052]: I1203 11:37:57.742049 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-c8af-account-create-update-fcv24" Dec 03 11:37:57 crc kubenswrapper[5052]: I1203 11:37:57.961724 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/23eed8d3-3d02-4f02-9cde-7e5dd4363b9b-operator-scripts\") pod \"23eed8d3-3d02-4f02-9cde-7e5dd4363b9b\" (UID: \"23eed8d3-3d02-4f02-9cde-7e5dd4363b9b\") " Dec 03 11:37:57 crc kubenswrapper[5052]: I1203 11:37:57.962233 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/23eed8d3-3d02-4f02-9cde-7e5dd4363b9b-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "23eed8d3-3d02-4f02-9cde-7e5dd4363b9b" (UID: "23eed8d3-3d02-4f02-9cde-7e5dd4363b9b"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:37:57 crc kubenswrapper[5052]: I1203 11:37:57.962249 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xrxgf\" (UniqueName: \"kubernetes.io/projected/23eed8d3-3d02-4f02-9cde-7e5dd4363b9b-kube-api-access-xrxgf\") pod \"23eed8d3-3d02-4f02-9cde-7e5dd4363b9b\" (UID: \"23eed8d3-3d02-4f02-9cde-7e5dd4363b9b\") " Dec 03 11:37:57 crc kubenswrapper[5052]: I1203 11:37:57.962690 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/23eed8d3-3d02-4f02-9cde-7e5dd4363b9b-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:37:57 crc kubenswrapper[5052]: I1203 11:37:57.976813 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/23eed8d3-3d02-4f02-9cde-7e5dd4363b9b-kube-api-access-xrxgf" (OuterVolumeSpecName: "kube-api-access-xrxgf") pod "23eed8d3-3d02-4f02-9cde-7e5dd4363b9b" (UID: "23eed8d3-3d02-4f02-9cde-7e5dd4363b9b"). InnerVolumeSpecName "kube-api-access-xrxgf". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:37:58 crc kubenswrapper[5052]: I1203 11:37:58.064541 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-xrxgf\" (UniqueName: \"kubernetes.io/projected/23eed8d3-3d02-4f02-9cde-7e5dd4363b9b-kube-api-access-xrxgf\") on node \"crc\" DevicePath \"\"" Dec 03 11:37:58 crc kubenswrapper[5052]: I1203 11:37:58.404033 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-c8af-account-create-update-fcv24" event={"ID":"23eed8d3-3d02-4f02-9cde-7e5dd4363b9b","Type":"ContainerDied","Data":"70491ae436faa3e034c7e6c1e0b4b447d205b78c41a44a2d994c81ce5dde91b6"} Dec 03 11:37:58 crc kubenswrapper[5052]: I1203 11:37:58.404087 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="70491ae436faa3e034c7e6c1e0b4b447d205b78c41a44a2d994c81ce5dde91b6" Dec 03 11:37:58 crc kubenswrapper[5052]: I1203 11:37:58.404122 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-c8af-account-create-update-fcv24" Dec 03 11:37:58 crc kubenswrapper[5052]: I1203 11:37:58.793266 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-79ca-account-create-update-xhl8v" Dec 03 11:37:58 crc kubenswrapper[5052]: I1203 11:37:58.911661 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-db-create-zvq9n" Dec 03 11:37:58 crc kubenswrapper[5052]: I1203 11:37:58.925668 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-db-create-bfh9z" Dec 03 11:37:58 crc kubenswrapper[5052]: I1203 11:37:58.983524 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/c3452ae3-e803-4020-b5db-3ddbdf4cba75-operator-scripts\") pod \"c3452ae3-e803-4020-b5db-3ddbdf4cba75\" (UID: \"c3452ae3-e803-4020-b5db-3ddbdf4cba75\") " Dec 03 11:37:58 crc kubenswrapper[5052]: I1203 11:37:58.984170 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s2kd2\" (UniqueName: \"kubernetes.io/projected/c3452ae3-e803-4020-b5db-3ddbdf4cba75-kube-api-access-s2kd2\") pod \"c3452ae3-e803-4020-b5db-3ddbdf4cba75\" (UID: \"c3452ae3-e803-4020-b5db-3ddbdf4cba75\") " Dec 03 11:37:58 crc kubenswrapper[5052]: I1203 11:37:58.984218 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c3452ae3-e803-4020-b5db-3ddbdf4cba75-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "c3452ae3-e803-4020-b5db-3ddbdf4cba75" (UID: "c3452ae3-e803-4020-b5db-3ddbdf4cba75"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:37:58 crc kubenswrapper[5052]: I1203 11:37:58.984842 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/c3452ae3-e803-4020-b5db-3ddbdf4cba75-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:37:58 crc kubenswrapper[5052]: I1203 11:37:58.989384 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c3452ae3-e803-4020-b5db-3ddbdf4cba75-kube-api-access-s2kd2" (OuterVolumeSpecName: "kube-api-access-s2kd2") pod "c3452ae3-e803-4020-b5db-3ddbdf4cba75" (UID: "c3452ae3-e803-4020-b5db-3ddbdf4cba75"). InnerVolumeSpecName "kube-api-access-s2kd2". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:37:59 crc kubenswrapper[5052]: I1203 11:37:59.086272 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b58g8\" (UniqueName: \"kubernetes.io/projected/69fe8db0-25e9-4f48-b9a2-77dc31224a34-kube-api-access-b58g8\") pod \"69fe8db0-25e9-4f48-b9a2-77dc31224a34\" (UID: \"69fe8db0-25e9-4f48-b9a2-77dc31224a34\") " Dec 03 11:37:59 crc kubenswrapper[5052]: I1203 11:37:59.086339 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/fbef0d42-0ae8-4e90-bacb-081a4b1e8d47-operator-scripts\") pod \"fbef0d42-0ae8-4e90-bacb-081a4b1e8d47\" (UID: \"fbef0d42-0ae8-4e90-bacb-081a4b1e8d47\") " Dec 03 11:37:59 crc kubenswrapper[5052]: I1203 11:37:59.086372 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4xgq7\" (UniqueName: \"kubernetes.io/projected/fbef0d42-0ae8-4e90-bacb-081a4b1e8d47-kube-api-access-4xgq7\") pod \"fbef0d42-0ae8-4e90-bacb-081a4b1e8d47\" (UID: \"fbef0d42-0ae8-4e90-bacb-081a4b1e8d47\") " Dec 03 11:37:59 crc kubenswrapper[5052]: I1203 11:37:59.086475 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/69fe8db0-25e9-4f48-b9a2-77dc31224a34-operator-scripts\") pod \"69fe8db0-25e9-4f48-b9a2-77dc31224a34\" (UID: \"69fe8db0-25e9-4f48-b9a2-77dc31224a34\") " Dec 03 11:37:59 crc kubenswrapper[5052]: I1203 11:37:59.087169 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/69fe8db0-25e9-4f48-b9a2-77dc31224a34-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "69fe8db0-25e9-4f48-b9a2-77dc31224a34" (UID: "69fe8db0-25e9-4f48-b9a2-77dc31224a34"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:37:59 crc kubenswrapper[5052]: I1203 11:37:59.087193 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fbef0d42-0ae8-4e90-bacb-081a4b1e8d47-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "fbef0d42-0ae8-4e90-bacb-081a4b1e8d47" (UID: "fbef0d42-0ae8-4e90-bacb-081a4b1e8d47"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:37:59 crc kubenswrapper[5052]: I1203 11:37:59.087810 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/69fe8db0-25e9-4f48-b9a2-77dc31224a34-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:37:59 crc kubenswrapper[5052]: I1203 11:37:59.087833 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-s2kd2\" (UniqueName: \"kubernetes.io/projected/c3452ae3-e803-4020-b5db-3ddbdf4cba75-kube-api-access-s2kd2\") on node \"crc\" DevicePath \"\"" Dec 03 11:37:59 crc kubenswrapper[5052]: I1203 11:37:59.087844 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/fbef0d42-0ae8-4e90-bacb-081a4b1e8d47-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:37:59 crc kubenswrapper[5052]: I1203 11:37:59.089702 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fbef0d42-0ae8-4e90-bacb-081a4b1e8d47-kube-api-access-4xgq7" (OuterVolumeSpecName: "kube-api-access-4xgq7") pod "fbef0d42-0ae8-4e90-bacb-081a4b1e8d47" (UID: "fbef0d42-0ae8-4e90-bacb-081a4b1e8d47"). InnerVolumeSpecName "kube-api-access-4xgq7". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:37:59 crc kubenswrapper[5052]: I1203 11:37:59.090351 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/69fe8db0-25e9-4f48-b9a2-77dc31224a34-kube-api-access-b58g8" (OuterVolumeSpecName: "kube-api-access-b58g8") pod "69fe8db0-25e9-4f48-b9a2-77dc31224a34" (UID: "69fe8db0-25e9-4f48-b9a2-77dc31224a34"). InnerVolumeSpecName "kube-api-access-b58g8". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:37:59 crc kubenswrapper[5052]: I1203 11:37:59.189363 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-b58g8\" (UniqueName: \"kubernetes.io/projected/69fe8db0-25e9-4f48-b9a2-77dc31224a34-kube-api-access-b58g8\") on node \"crc\" DevicePath \"\"" Dec 03 11:37:59 crc kubenswrapper[5052]: I1203 11:37:59.189408 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-4xgq7\" (UniqueName: \"kubernetes.io/projected/fbef0d42-0ae8-4e90-bacb-081a4b1e8d47-kube-api-access-4xgq7\") on node \"crc\" DevicePath \"\"" Dec 03 11:37:59 crc kubenswrapper[5052]: I1203 11:37:59.413104 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-db-create-zvq9n" Dec 03 11:37:59 crc kubenswrapper[5052]: I1203 11:37:59.413107 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-db-create-zvq9n" event={"ID":"fbef0d42-0ae8-4e90-bacb-081a4b1e8d47","Type":"ContainerDied","Data":"10ba1e01541849ab298290759edc46053781685a4229f3aa27b653b6861f8f65"} Dec 03 11:37:59 crc kubenswrapper[5052]: I1203 11:37:59.413275 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="10ba1e01541849ab298290759edc46053781685a4229f3aa27b653b6861f8f65" Dec 03 11:37:59 crc kubenswrapper[5052]: I1203 11:37:59.414259 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-79ca-account-create-update-xhl8v" event={"ID":"c3452ae3-e803-4020-b5db-3ddbdf4cba75","Type":"ContainerDied","Data":"44ff0b6775181abf6e7cb2d5bdb02d8ded80e32a5ecd9371181beab3128dad0b"} Dec 03 11:37:59 crc kubenswrapper[5052]: I1203 11:37:59.414299 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="44ff0b6775181abf6e7cb2d5bdb02d8ded80e32a5ecd9371181beab3128dad0b" Dec 03 11:37:59 crc kubenswrapper[5052]: I1203 11:37:59.414325 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-79ca-account-create-update-xhl8v" Dec 03 11:37:59 crc kubenswrapper[5052]: I1203 11:37:59.415460 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-db-create-bfh9z" event={"ID":"69fe8db0-25e9-4f48-b9a2-77dc31224a34","Type":"ContainerDied","Data":"1268573c1030df2e6e2bea83cb86bedf16f335320a5b19fe20a9ce04fc357080"} Dec 03 11:37:59 crc kubenswrapper[5052]: I1203 11:37:59.415493 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1268573c1030df2e6e2bea83cb86bedf16f335320a5b19fe20a9ce04fc357080" Dec 03 11:37:59 crc kubenswrapper[5052]: I1203 11:37:59.415568 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-db-create-bfh9z" Dec 03 11:38:00 crc kubenswrapper[5052]: I1203 11:38:00.538813 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/glance-db-create-ckh5l"] Dec 03 11:38:00 crc kubenswrapper[5052]: E1203 11:38:00.539636 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="c3452ae3-e803-4020-b5db-3ddbdf4cba75" containerName="mariadb-account-create-update" Dec 03 11:38:00 crc kubenswrapper[5052]: I1203 11:38:00.539653 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="c3452ae3-e803-4020-b5db-3ddbdf4cba75" containerName="mariadb-account-create-update" Dec 03 11:38:00 crc kubenswrapper[5052]: E1203 11:38:00.539685 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="23eed8d3-3d02-4f02-9cde-7e5dd4363b9b" containerName="mariadb-account-create-update" Dec 03 11:38:00 crc kubenswrapper[5052]: I1203 11:38:00.539692 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="23eed8d3-3d02-4f02-9cde-7e5dd4363b9b" containerName="mariadb-account-create-update" Dec 03 11:38:00 crc kubenswrapper[5052]: E1203 11:38:00.539705 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="69fe8db0-25e9-4f48-b9a2-77dc31224a34" containerName="mariadb-database-create" Dec 03 11:38:00 crc kubenswrapper[5052]: I1203 11:38:00.539712 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="69fe8db0-25e9-4f48-b9a2-77dc31224a34" containerName="mariadb-database-create" Dec 03 11:38:00 crc kubenswrapper[5052]: E1203 11:38:00.539729 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="fbef0d42-0ae8-4e90-bacb-081a4b1e8d47" containerName="mariadb-database-create" Dec 03 11:38:00 crc kubenswrapper[5052]: I1203 11:38:00.539768 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="fbef0d42-0ae8-4e90-bacb-081a4b1e8d47" containerName="mariadb-database-create" Dec 03 11:38:00 crc kubenswrapper[5052]: I1203 11:38:00.539937 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="c3452ae3-e803-4020-b5db-3ddbdf4cba75" containerName="mariadb-account-create-update" Dec 03 11:38:00 crc kubenswrapper[5052]: I1203 11:38:00.539955 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="69fe8db0-25e9-4f48-b9a2-77dc31224a34" containerName="mariadb-database-create" Dec 03 11:38:00 crc kubenswrapper[5052]: I1203 11:38:00.539979 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="fbef0d42-0ae8-4e90-bacb-081a4b1e8d47" containerName="mariadb-database-create" Dec 03 11:38:00 crc kubenswrapper[5052]: I1203 11:38:00.539994 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="23eed8d3-3d02-4f02-9cde-7e5dd4363b9b" containerName="mariadb-account-create-update" Dec 03 11:38:00 crc kubenswrapper[5052]: I1203 11:38:00.540687 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-db-create-ckh5l" Dec 03 11:38:00 crc kubenswrapper[5052]: I1203 11:38:00.563201 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-db-create-ckh5l"] Dec 03 11:38:00 crc kubenswrapper[5052]: I1203 11:38:00.646057 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/glance-78ad-account-create-update-7wlzh"] Dec 03 11:38:00 crc kubenswrapper[5052]: I1203 11:38:00.647077 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-78ad-account-create-update-7wlzh" Dec 03 11:38:00 crc kubenswrapper[5052]: I1203 11:38:00.652760 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-db-secret" Dec 03 11:38:00 crc kubenswrapper[5052]: I1203 11:38:00.676074 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-78ad-account-create-update-7wlzh"] Dec 03 11:38:00 crc kubenswrapper[5052]: I1203 11:38:00.722431 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/33a7c4f9-6f64-4bf9-b0fc-419117a03f9d-operator-scripts\") pod \"glance-db-create-ckh5l\" (UID: \"33a7c4f9-6f64-4bf9-b0fc-419117a03f9d\") " pod="openstack-kuttl-tests/glance-db-create-ckh5l" Dec 03 11:38:00 crc kubenswrapper[5052]: I1203 11:38:00.722535 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rnnk8\" (UniqueName: \"kubernetes.io/projected/33a7c4f9-6f64-4bf9-b0fc-419117a03f9d-kube-api-access-rnnk8\") pod \"glance-db-create-ckh5l\" (UID: \"33a7c4f9-6f64-4bf9-b0fc-419117a03f9d\") " pod="openstack-kuttl-tests/glance-db-create-ckh5l" Dec 03 11:38:00 crc kubenswrapper[5052]: I1203 11:38:00.823742 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/801b412e-0431-4a95-9639-93eeb8809c75-operator-scripts\") pod \"glance-78ad-account-create-update-7wlzh\" (UID: \"801b412e-0431-4a95-9639-93eeb8809c75\") " pod="openstack-kuttl-tests/glance-78ad-account-create-update-7wlzh" Dec 03 11:38:00 crc kubenswrapper[5052]: I1203 11:38:00.823820 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-rnnk8\" (UniqueName: \"kubernetes.io/projected/33a7c4f9-6f64-4bf9-b0fc-419117a03f9d-kube-api-access-rnnk8\") pod \"glance-db-create-ckh5l\" (UID: \"33a7c4f9-6f64-4bf9-b0fc-419117a03f9d\") " pod="openstack-kuttl-tests/glance-db-create-ckh5l" Dec 03 11:38:00 crc kubenswrapper[5052]: I1203 11:38:00.824240 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hlcsq\" (UniqueName: \"kubernetes.io/projected/801b412e-0431-4a95-9639-93eeb8809c75-kube-api-access-hlcsq\") pod \"glance-78ad-account-create-update-7wlzh\" (UID: \"801b412e-0431-4a95-9639-93eeb8809c75\") " pod="openstack-kuttl-tests/glance-78ad-account-create-update-7wlzh" Dec 03 11:38:00 crc kubenswrapper[5052]: I1203 11:38:00.824292 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/33a7c4f9-6f64-4bf9-b0fc-419117a03f9d-operator-scripts\") pod \"glance-db-create-ckh5l\" (UID: \"33a7c4f9-6f64-4bf9-b0fc-419117a03f9d\") " pod="openstack-kuttl-tests/glance-db-create-ckh5l" Dec 03 11:38:00 crc kubenswrapper[5052]: I1203 11:38:00.825075 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/33a7c4f9-6f64-4bf9-b0fc-419117a03f9d-operator-scripts\") pod \"glance-db-create-ckh5l\" (UID: \"33a7c4f9-6f64-4bf9-b0fc-419117a03f9d\") " pod="openstack-kuttl-tests/glance-db-create-ckh5l" Dec 03 11:38:00 crc kubenswrapper[5052]: I1203 11:38:00.851375 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-rnnk8\" (UniqueName: \"kubernetes.io/projected/33a7c4f9-6f64-4bf9-b0fc-419117a03f9d-kube-api-access-rnnk8\") pod \"glance-db-create-ckh5l\" (UID: \"33a7c4f9-6f64-4bf9-b0fc-419117a03f9d\") " pod="openstack-kuttl-tests/glance-db-create-ckh5l" Dec 03 11:38:00 crc kubenswrapper[5052]: I1203 11:38:00.863838 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-db-create-ckh5l" Dec 03 11:38:00 crc kubenswrapper[5052]: I1203 11:38:00.925391 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-hlcsq\" (UniqueName: \"kubernetes.io/projected/801b412e-0431-4a95-9639-93eeb8809c75-kube-api-access-hlcsq\") pod \"glance-78ad-account-create-update-7wlzh\" (UID: \"801b412e-0431-4a95-9639-93eeb8809c75\") " pod="openstack-kuttl-tests/glance-78ad-account-create-update-7wlzh" Dec 03 11:38:00 crc kubenswrapper[5052]: I1203 11:38:00.925448 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/801b412e-0431-4a95-9639-93eeb8809c75-operator-scripts\") pod \"glance-78ad-account-create-update-7wlzh\" (UID: \"801b412e-0431-4a95-9639-93eeb8809c75\") " pod="openstack-kuttl-tests/glance-78ad-account-create-update-7wlzh" Dec 03 11:38:00 crc kubenswrapper[5052]: I1203 11:38:00.926232 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/801b412e-0431-4a95-9639-93eeb8809c75-operator-scripts\") pod \"glance-78ad-account-create-update-7wlzh\" (UID: \"801b412e-0431-4a95-9639-93eeb8809c75\") " pod="openstack-kuttl-tests/glance-78ad-account-create-update-7wlzh" Dec 03 11:38:00 crc kubenswrapper[5052]: I1203 11:38:00.943390 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-hlcsq\" (UniqueName: \"kubernetes.io/projected/801b412e-0431-4a95-9639-93eeb8809c75-kube-api-access-hlcsq\") pod \"glance-78ad-account-create-update-7wlzh\" (UID: \"801b412e-0431-4a95-9639-93eeb8809c75\") " pod="openstack-kuttl-tests/glance-78ad-account-create-update-7wlzh" Dec 03 11:38:00 crc kubenswrapper[5052]: I1203 11:38:00.973054 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-78ad-account-create-update-7wlzh" Dec 03 11:38:01 crc kubenswrapper[5052]: I1203 11:38:01.337291 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-db-create-ckh5l"] Dec 03 11:38:01 crc kubenswrapper[5052]: I1203 11:38:01.431273 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-db-create-ckh5l" event={"ID":"33a7c4f9-6f64-4bf9-b0fc-419117a03f9d","Type":"ContainerStarted","Data":"9db2d403061d0085d0d2fba9ab19d2dc8df7a4d220eab42cc3e1546c60d0edf9"} Dec 03 11:38:01 crc kubenswrapper[5052]: I1203 11:38:01.449295 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-78ad-account-create-update-7wlzh"] Dec 03 11:38:01 crc kubenswrapper[5052]: W1203 11:38:01.458649 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod801b412e_0431_4a95_9639_93eeb8809c75.slice/crio-59bbd43f3e133ed1ae4c11f16b239b84f909edf35c24d753ec517bf5e7e46eba WatchSource:0}: Error finding container 59bbd43f3e133ed1ae4c11f16b239b84f909edf35c24d753ec517bf5e7e46eba: Status 404 returned error can't find the container with id 59bbd43f3e133ed1ae4c11f16b239b84f909edf35c24d753ec517bf5e7e46eba Dec 03 11:38:02 crc kubenswrapper[5052]: I1203 11:38:02.441160 5052 generic.go:334] "Generic (PLEG): container finished" podID="33a7c4f9-6f64-4bf9-b0fc-419117a03f9d" containerID="82bdd60bb6c2b4f3232ce11412a9103ff8cdf6022d10e2a0fc8d882e6a7cbde9" exitCode=0 Dec 03 11:38:02 crc kubenswrapper[5052]: I1203 11:38:02.441217 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-db-create-ckh5l" event={"ID":"33a7c4f9-6f64-4bf9-b0fc-419117a03f9d","Type":"ContainerDied","Data":"82bdd60bb6c2b4f3232ce11412a9103ff8cdf6022d10e2a0fc8d882e6a7cbde9"} Dec 03 11:38:02 crc kubenswrapper[5052]: I1203 11:38:02.443107 5052 generic.go:334] "Generic (PLEG): container finished" podID="801b412e-0431-4a95-9639-93eeb8809c75" containerID="6d71e00901cb2b18bfa9c3d46e251e99210b915413b7d7d03d74cb72b37c2e29" exitCode=0 Dec 03 11:38:02 crc kubenswrapper[5052]: I1203 11:38:02.443184 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-78ad-account-create-update-7wlzh" event={"ID":"801b412e-0431-4a95-9639-93eeb8809c75","Type":"ContainerDied","Data":"6d71e00901cb2b18bfa9c3d46e251e99210b915413b7d7d03d74cb72b37c2e29"} Dec 03 11:38:02 crc kubenswrapper[5052]: I1203 11:38:02.443226 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-78ad-account-create-update-7wlzh" event={"ID":"801b412e-0431-4a95-9639-93eeb8809c75","Type":"ContainerStarted","Data":"59bbd43f3e133ed1ae4c11f16b239b84f909edf35c24d753ec517bf5e7e46eba"} Dec 03 11:38:02 crc kubenswrapper[5052]: I1203 11:38:02.444894 5052 generic.go:334] "Generic (PLEG): container finished" podID="ff7c4fed-d9f0-4220-9c04-33fae89bacd2" containerID="f60cd400b6652773ef4753bb550741040c4ae2eb9c6c8a5ebbe3a84a430ead37" exitCode=0 Dec 03 11:38:02 crc kubenswrapper[5052]: I1203 11:38:02.444985 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-ring-rebalance-krb94" event={"ID":"ff7c4fed-d9f0-4220-9c04-33fae89bacd2","Type":"ContainerDied","Data":"f60cd400b6652773ef4753bb550741040c4ae2eb9c6c8a5ebbe3a84a430ead37"} Dec 03 11:38:03 crc kubenswrapper[5052]: I1203 11:38:03.825373 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-78ad-account-create-update-7wlzh" Dec 03 11:38:03 crc kubenswrapper[5052]: I1203 11:38:03.870288 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/801b412e-0431-4a95-9639-93eeb8809c75-operator-scripts\") pod \"801b412e-0431-4a95-9639-93eeb8809c75\" (UID: \"801b412e-0431-4a95-9639-93eeb8809c75\") " Dec 03 11:38:03 crc kubenswrapper[5052]: I1203 11:38:03.870332 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hlcsq\" (UniqueName: \"kubernetes.io/projected/801b412e-0431-4a95-9639-93eeb8809c75-kube-api-access-hlcsq\") pod \"801b412e-0431-4a95-9639-93eeb8809c75\" (UID: \"801b412e-0431-4a95-9639-93eeb8809c75\") " Dec 03 11:38:03 crc kubenswrapper[5052]: I1203 11:38:03.871046 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/801b412e-0431-4a95-9639-93eeb8809c75-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "801b412e-0431-4a95-9639-93eeb8809c75" (UID: "801b412e-0431-4a95-9639-93eeb8809c75"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:38:03 crc kubenswrapper[5052]: I1203 11:38:03.878813 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/801b412e-0431-4a95-9639-93eeb8809c75-kube-api-access-hlcsq" (OuterVolumeSpecName: "kube-api-access-hlcsq") pod "801b412e-0431-4a95-9639-93eeb8809c75" (UID: "801b412e-0431-4a95-9639-93eeb8809c75"). InnerVolumeSpecName "kube-api-access-hlcsq". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:38:03 crc kubenswrapper[5052]: I1203 11:38:03.938778 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-db-create-ckh5l" Dec 03 11:38:03 crc kubenswrapper[5052]: I1203 11:38:03.947277 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-ring-rebalance-krb94" Dec 03 11:38:03 crc kubenswrapper[5052]: I1203 11:38:03.971650 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/ff7c4fed-d9f0-4220-9c04-33fae89bacd2-scripts\") pod \"ff7c4fed-d9f0-4220-9c04-33fae89bacd2\" (UID: \"ff7c4fed-d9f0-4220-9c04-33fae89bacd2\") " Dec 03 11:38:03 crc kubenswrapper[5052]: I1203 11:38:03.971703 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ring-data-devices\" (UniqueName: \"kubernetes.io/configmap/ff7c4fed-d9f0-4220-9c04-33fae89bacd2-ring-data-devices\") pod \"ff7c4fed-d9f0-4220-9c04-33fae89bacd2\" (UID: \"ff7c4fed-d9f0-4220-9c04-33fae89bacd2\") " Dec 03 11:38:03 crc kubenswrapper[5052]: I1203 11:38:03.971744 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"dispersionconf\" (UniqueName: \"kubernetes.io/secret/ff7c4fed-d9f0-4220-9c04-33fae89bacd2-dispersionconf\") pod \"ff7c4fed-d9f0-4220-9c04-33fae89bacd2\" (UID: \"ff7c4fed-d9f0-4220-9c04-33fae89bacd2\") " Dec 03 11:38:03 crc kubenswrapper[5052]: I1203 11:38:03.971764 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lbw7m\" (UniqueName: \"kubernetes.io/projected/ff7c4fed-d9f0-4220-9c04-33fae89bacd2-kube-api-access-lbw7m\") pod \"ff7c4fed-d9f0-4220-9c04-33fae89bacd2\" (UID: \"ff7c4fed-d9f0-4220-9c04-33fae89bacd2\") " Dec 03 11:38:03 crc kubenswrapper[5052]: I1203 11:38:03.971787 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"swiftconf\" (UniqueName: \"kubernetes.io/secret/ff7c4fed-d9f0-4220-9c04-33fae89bacd2-swiftconf\") pod \"ff7c4fed-d9f0-4220-9c04-33fae89bacd2\" (UID: \"ff7c4fed-d9f0-4220-9c04-33fae89bacd2\") " Dec 03 11:38:03 crc kubenswrapper[5052]: I1203 11:38:03.971810 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/empty-dir/ff7c4fed-d9f0-4220-9c04-33fae89bacd2-etc-swift\") pod \"ff7c4fed-d9f0-4220-9c04-33fae89bacd2\" (UID: \"ff7c4fed-d9f0-4220-9c04-33fae89bacd2\") " Dec 03 11:38:03 crc kubenswrapper[5052]: I1203 11:38:03.971825 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/33a7c4f9-6f64-4bf9-b0fc-419117a03f9d-operator-scripts\") pod \"33a7c4f9-6f64-4bf9-b0fc-419117a03f9d\" (UID: \"33a7c4f9-6f64-4bf9-b0fc-419117a03f9d\") " Dec 03 11:38:03 crc kubenswrapper[5052]: I1203 11:38:03.971857 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ff7c4fed-d9f0-4220-9c04-33fae89bacd2-combined-ca-bundle\") pod \"ff7c4fed-d9f0-4220-9c04-33fae89bacd2\" (UID: \"ff7c4fed-d9f0-4220-9c04-33fae89bacd2\") " Dec 03 11:38:03 crc kubenswrapper[5052]: I1203 11:38:03.971883 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rnnk8\" (UniqueName: \"kubernetes.io/projected/33a7c4f9-6f64-4bf9-b0fc-419117a03f9d-kube-api-access-rnnk8\") pod \"33a7c4f9-6f64-4bf9-b0fc-419117a03f9d\" (UID: \"33a7c4f9-6f64-4bf9-b0fc-419117a03f9d\") " Dec 03 11:38:03 crc kubenswrapper[5052]: I1203 11:38:03.972082 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/801b412e-0431-4a95-9639-93eeb8809c75-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:38:03 crc kubenswrapper[5052]: I1203 11:38:03.972095 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-hlcsq\" (UniqueName: \"kubernetes.io/projected/801b412e-0431-4a95-9639-93eeb8809c75-kube-api-access-hlcsq\") on node \"crc\" DevicePath \"\"" Dec 03 11:38:03 crc kubenswrapper[5052]: I1203 11:38:03.972136 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ff7c4fed-d9f0-4220-9c04-33fae89bacd2-ring-data-devices" (OuterVolumeSpecName: "ring-data-devices") pod "ff7c4fed-d9f0-4220-9c04-33fae89bacd2" (UID: "ff7c4fed-d9f0-4220-9c04-33fae89bacd2"). InnerVolumeSpecName "ring-data-devices". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:38:03 crc kubenswrapper[5052]: I1203 11:38:03.972529 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/33a7c4f9-6f64-4bf9-b0fc-419117a03f9d-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "33a7c4f9-6f64-4bf9-b0fc-419117a03f9d" (UID: "33a7c4f9-6f64-4bf9-b0fc-419117a03f9d"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:38:03 crc kubenswrapper[5052]: I1203 11:38:03.974787 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff7c4fed-d9f0-4220-9c04-33fae89bacd2-etc-swift" (OuterVolumeSpecName: "etc-swift") pod "ff7c4fed-d9f0-4220-9c04-33fae89bacd2" (UID: "ff7c4fed-d9f0-4220-9c04-33fae89bacd2"). InnerVolumeSpecName "etc-swift". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:38:03 crc kubenswrapper[5052]: I1203 11:38:03.978956 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ff7c4fed-d9f0-4220-9c04-33fae89bacd2-kube-api-access-lbw7m" (OuterVolumeSpecName: "kube-api-access-lbw7m") pod "ff7c4fed-d9f0-4220-9c04-33fae89bacd2" (UID: "ff7c4fed-d9f0-4220-9c04-33fae89bacd2"). InnerVolumeSpecName "kube-api-access-lbw7m". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:38:03 crc kubenswrapper[5052]: I1203 11:38:03.983825 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ff7c4fed-d9f0-4220-9c04-33fae89bacd2-dispersionconf" (OuterVolumeSpecName: "dispersionconf") pod "ff7c4fed-d9f0-4220-9c04-33fae89bacd2" (UID: "ff7c4fed-d9f0-4220-9c04-33fae89bacd2"). InnerVolumeSpecName "dispersionconf". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:38:03 crc kubenswrapper[5052]: I1203 11:38:03.987329 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/33a7c4f9-6f64-4bf9-b0fc-419117a03f9d-kube-api-access-rnnk8" (OuterVolumeSpecName: "kube-api-access-rnnk8") pod "33a7c4f9-6f64-4bf9-b0fc-419117a03f9d" (UID: "33a7c4f9-6f64-4bf9-b0fc-419117a03f9d"). InnerVolumeSpecName "kube-api-access-rnnk8". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:38:03 crc kubenswrapper[5052]: I1203 11:38:03.992431 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ff7c4fed-d9f0-4220-9c04-33fae89bacd2-scripts" (OuterVolumeSpecName: "scripts") pod "ff7c4fed-d9f0-4220-9c04-33fae89bacd2" (UID: "ff7c4fed-d9f0-4220-9c04-33fae89bacd2"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:38:03 crc kubenswrapper[5052]: I1203 11:38:03.993235 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ff7c4fed-d9f0-4220-9c04-33fae89bacd2-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "ff7c4fed-d9f0-4220-9c04-33fae89bacd2" (UID: "ff7c4fed-d9f0-4220-9c04-33fae89bacd2"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:38:04 crc kubenswrapper[5052]: I1203 11:38:04.005478 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ff7c4fed-d9f0-4220-9c04-33fae89bacd2-swiftconf" (OuterVolumeSpecName: "swiftconf") pod "ff7c4fed-d9f0-4220-9c04-33fae89bacd2" (UID: "ff7c4fed-d9f0-4220-9c04-33fae89bacd2"). InnerVolumeSpecName "swiftconf". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:38:04 crc kubenswrapper[5052]: I1203 11:38:04.073027 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/d19f0c21-bb49-40a1-87fa-e55188308729-etc-swift\") pod \"swift-storage-0\" (UID: \"d19f0c21-bb49-40a1-87fa-e55188308729\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 11:38:04 crc kubenswrapper[5052]: I1203 11:38:04.073463 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/ff7c4fed-d9f0-4220-9c04-33fae89bacd2-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:38:04 crc kubenswrapper[5052]: I1203 11:38:04.073501 5052 reconciler_common.go:293] "Volume detached for volume \"ring-data-devices\" (UniqueName: \"kubernetes.io/configmap/ff7c4fed-d9f0-4220-9c04-33fae89bacd2-ring-data-devices\") on node \"crc\" DevicePath \"\"" Dec 03 11:38:04 crc kubenswrapper[5052]: I1203 11:38:04.073532 5052 reconciler_common.go:293] "Volume detached for volume \"dispersionconf\" (UniqueName: \"kubernetes.io/secret/ff7c4fed-d9f0-4220-9c04-33fae89bacd2-dispersionconf\") on node \"crc\" DevicePath \"\"" Dec 03 11:38:04 crc kubenswrapper[5052]: I1203 11:38:04.073543 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-lbw7m\" (UniqueName: \"kubernetes.io/projected/ff7c4fed-d9f0-4220-9c04-33fae89bacd2-kube-api-access-lbw7m\") on node \"crc\" DevicePath \"\"" Dec 03 11:38:04 crc kubenswrapper[5052]: I1203 11:38:04.073554 5052 reconciler_common.go:293] "Volume detached for volume \"swiftconf\" (UniqueName: \"kubernetes.io/secret/ff7c4fed-d9f0-4220-9c04-33fae89bacd2-swiftconf\") on node \"crc\" DevicePath \"\"" Dec 03 11:38:04 crc kubenswrapper[5052]: I1203 11:38:04.073563 5052 reconciler_common.go:293] "Volume detached for volume \"etc-swift\" (UniqueName: \"kubernetes.io/empty-dir/ff7c4fed-d9f0-4220-9c04-33fae89bacd2-etc-swift\") on node \"crc\" DevicePath \"\"" Dec 03 11:38:04 crc kubenswrapper[5052]: I1203 11:38:04.073575 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/33a7c4f9-6f64-4bf9-b0fc-419117a03f9d-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:38:04 crc kubenswrapper[5052]: I1203 11:38:04.073585 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ff7c4fed-d9f0-4220-9c04-33fae89bacd2-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:38:04 crc kubenswrapper[5052]: I1203 11:38:04.073596 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-rnnk8\" (UniqueName: \"kubernetes.io/projected/33a7c4f9-6f64-4bf9-b0fc-419117a03f9d-kube-api-access-rnnk8\") on node \"crc\" DevicePath \"\"" Dec 03 11:38:04 crc kubenswrapper[5052]: I1203 11:38:04.080130 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/d19f0c21-bb49-40a1-87fa-e55188308729-etc-swift\") pod \"swift-storage-0\" (UID: \"d19f0c21-bb49-40a1-87fa-e55188308729\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 11:38:04 crc kubenswrapper[5052]: I1203 11:38:04.145023 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-storage-0" Dec 03 11:38:04 crc kubenswrapper[5052]: I1203 11:38:04.460946 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-ring-rebalance-krb94" Dec 03 11:38:04 crc kubenswrapper[5052]: I1203 11:38:04.460959 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-ring-rebalance-krb94" event={"ID":"ff7c4fed-d9f0-4220-9c04-33fae89bacd2","Type":"ContainerDied","Data":"cc107403ddb2466ff8b9de6b126c9612db8e2115d5f9ee2b4673f7fec3ada22e"} Dec 03 11:38:04 crc kubenswrapper[5052]: I1203 11:38:04.461007 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cc107403ddb2466ff8b9de6b126c9612db8e2115d5f9ee2b4673f7fec3ada22e" Dec 03 11:38:04 crc kubenswrapper[5052]: I1203 11:38:04.463035 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-db-create-ckh5l" Dec 03 11:38:04 crc kubenswrapper[5052]: I1203 11:38:04.463048 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-db-create-ckh5l" event={"ID":"33a7c4f9-6f64-4bf9-b0fc-419117a03f9d","Type":"ContainerDied","Data":"9db2d403061d0085d0d2fba9ab19d2dc8df7a4d220eab42cc3e1546c60d0edf9"} Dec 03 11:38:04 crc kubenswrapper[5052]: I1203 11:38:04.463097 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9db2d403061d0085d0d2fba9ab19d2dc8df7a4d220eab42cc3e1546c60d0edf9" Dec 03 11:38:04 crc kubenswrapper[5052]: I1203 11:38:04.464447 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-78ad-account-create-update-7wlzh" event={"ID":"801b412e-0431-4a95-9639-93eeb8809c75","Type":"ContainerDied","Data":"59bbd43f3e133ed1ae4c11f16b239b84f909edf35c24d753ec517bf5e7e46eba"} Dec 03 11:38:04 crc kubenswrapper[5052]: I1203 11:38:04.464493 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="59bbd43f3e133ed1ae4c11f16b239b84f909edf35c24d753ec517bf5e7e46eba" Dec 03 11:38:04 crc kubenswrapper[5052]: I1203 11:38:04.464566 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-78ad-account-create-update-7wlzh" Dec 03 11:38:04 crc kubenswrapper[5052]: I1203 11:38:04.606571 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/swift-storage-0"] Dec 03 11:38:04 crc kubenswrapper[5052]: W1203 11:38:04.610634 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd19f0c21_bb49_40a1_87fa_e55188308729.slice/crio-78fdcedb4dfb761391bae389e6c4cba41223c19c5bee28700240078692e02f22 WatchSource:0}: Error finding container 78fdcedb4dfb761391bae389e6c4cba41223c19c5bee28700240078692e02f22: Status 404 returned error can't find the container with id 78fdcedb4dfb761391bae389e6c4cba41223c19c5bee28700240078692e02f22 Dec 03 11:38:05 crc kubenswrapper[5052]: I1203 11:38:05.473673 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"d19f0c21-bb49-40a1-87fa-e55188308729","Type":"ContainerStarted","Data":"78fdcedb4dfb761391bae389e6c4cba41223c19c5bee28700240078692e02f22"} Dec 03 11:38:05 crc kubenswrapper[5052]: I1203 11:38:05.869115 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/glance-db-sync-k6vlv"] Dec 03 11:38:05 crc kubenswrapper[5052]: E1203 11:38:05.869445 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="33a7c4f9-6f64-4bf9-b0fc-419117a03f9d" containerName="mariadb-database-create" Dec 03 11:38:05 crc kubenswrapper[5052]: I1203 11:38:05.869465 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="33a7c4f9-6f64-4bf9-b0fc-419117a03f9d" containerName="mariadb-database-create" Dec 03 11:38:05 crc kubenswrapper[5052]: E1203 11:38:05.869493 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="ff7c4fed-d9f0-4220-9c04-33fae89bacd2" containerName="swift-ring-rebalance" Dec 03 11:38:05 crc kubenswrapper[5052]: I1203 11:38:05.869499 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="ff7c4fed-d9f0-4220-9c04-33fae89bacd2" containerName="swift-ring-rebalance" Dec 03 11:38:05 crc kubenswrapper[5052]: E1203 11:38:05.869613 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="801b412e-0431-4a95-9639-93eeb8809c75" containerName="mariadb-account-create-update" Dec 03 11:38:05 crc kubenswrapper[5052]: I1203 11:38:05.869621 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="801b412e-0431-4a95-9639-93eeb8809c75" containerName="mariadb-account-create-update" Dec 03 11:38:05 crc kubenswrapper[5052]: I1203 11:38:05.869801 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="801b412e-0431-4a95-9639-93eeb8809c75" containerName="mariadb-account-create-update" Dec 03 11:38:05 crc kubenswrapper[5052]: I1203 11:38:05.869823 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="ff7c4fed-d9f0-4220-9c04-33fae89bacd2" containerName="swift-ring-rebalance" Dec 03 11:38:05 crc kubenswrapper[5052]: I1203 11:38:05.869845 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="33a7c4f9-6f64-4bf9-b0fc-419117a03f9d" containerName="mariadb-database-create" Dec 03 11:38:05 crc kubenswrapper[5052]: I1203 11:38:05.870330 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-db-sync-k6vlv" Dec 03 11:38:05 crc kubenswrapper[5052]: I1203 11:38:05.872712 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-config-data" Dec 03 11:38:05 crc kubenswrapper[5052]: I1203 11:38:05.872798 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-glance-dockercfg-kt8wz" Dec 03 11:38:05 crc kubenswrapper[5052]: I1203 11:38:05.879571 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-db-sync-k6vlv"] Dec 03 11:38:05 crc kubenswrapper[5052]: I1203 11:38:05.901380 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b2b64\" (UniqueName: \"kubernetes.io/projected/063c5d2e-68a5-497a-b7bd-f95d45804848-kube-api-access-b2b64\") pod \"glance-db-sync-k6vlv\" (UID: \"063c5d2e-68a5-497a-b7bd-f95d45804848\") " pod="openstack-kuttl-tests/glance-db-sync-k6vlv" Dec 03 11:38:05 crc kubenswrapper[5052]: I1203 11:38:05.901424 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/063c5d2e-68a5-497a-b7bd-f95d45804848-combined-ca-bundle\") pod \"glance-db-sync-k6vlv\" (UID: \"063c5d2e-68a5-497a-b7bd-f95d45804848\") " pod="openstack-kuttl-tests/glance-db-sync-k6vlv" Dec 03 11:38:05 crc kubenswrapper[5052]: I1203 11:38:05.901587 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/063c5d2e-68a5-497a-b7bd-f95d45804848-config-data\") pod \"glance-db-sync-k6vlv\" (UID: \"063c5d2e-68a5-497a-b7bd-f95d45804848\") " pod="openstack-kuttl-tests/glance-db-sync-k6vlv" Dec 03 11:38:05 crc kubenswrapper[5052]: I1203 11:38:05.901718 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/063c5d2e-68a5-497a-b7bd-f95d45804848-db-sync-config-data\") pod \"glance-db-sync-k6vlv\" (UID: \"063c5d2e-68a5-497a-b7bd-f95d45804848\") " pod="openstack-kuttl-tests/glance-db-sync-k6vlv" Dec 03 11:38:06 crc kubenswrapper[5052]: I1203 11:38:06.002689 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-b2b64\" (UniqueName: \"kubernetes.io/projected/063c5d2e-68a5-497a-b7bd-f95d45804848-kube-api-access-b2b64\") pod \"glance-db-sync-k6vlv\" (UID: \"063c5d2e-68a5-497a-b7bd-f95d45804848\") " pod="openstack-kuttl-tests/glance-db-sync-k6vlv" Dec 03 11:38:06 crc kubenswrapper[5052]: I1203 11:38:06.002742 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/063c5d2e-68a5-497a-b7bd-f95d45804848-combined-ca-bundle\") pod \"glance-db-sync-k6vlv\" (UID: \"063c5d2e-68a5-497a-b7bd-f95d45804848\") " pod="openstack-kuttl-tests/glance-db-sync-k6vlv" Dec 03 11:38:06 crc kubenswrapper[5052]: I1203 11:38:06.002984 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/063c5d2e-68a5-497a-b7bd-f95d45804848-config-data\") pod \"glance-db-sync-k6vlv\" (UID: \"063c5d2e-68a5-497a-b7bd-f95d45804848\") " pod="openstack-kuttl-tests/glance-db-sync-k6vlv" Dec 03 11:38:06 crc kubenswrapper[5052]: I1203 11:38:06.003028 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/063c5d2e-68a5-497a-b7bd-f95d45804848-db-sync-config-data\") pod \"glance-db-sync-k6vlv\" (UID: \"063c5d2e-68a5-497a-b7bd-f95d45804848\") " pod="openstack-kuttl-tests/glance-db-sync-k6vlv" Dec 03 11:38:06 crc kubenswrapper[5052]: I1203 11:38:06.009095 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/063c5d2e-68a5-497a-b7bd-f95d45804848-combined-ca-bundle\") pod \"glance-db-sync-k6vlv\" (UID: \"063c5d2e-68a5-497a-b7bd-f95d45804848\") " pod="openstack-kuttl-tests/glance-db-sync-k6vlv" Dec 03 11:38:06 crc kubenswrapper[5052]: I1203 11:38:06.009309 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/063c5d2e-68a5-497a-b7bd-f95d45804848-config-data\") pod \"glance-db-sync-k6vlv\" (UID: \"063c5d2e-68a5-497a-b7bd-f95d45804848\") " pod="openstack-kuttl-tests/glance-db-sync-k6vlv" Dec 03 11:38:06 crc kubenswrapper[5052]: I1203 11:38:06.009629 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/063c5d2e-68a5-497a-b7bd-f95d45804848-db-sync-config-data\") pod \"glance-db-sync-k6vlv\" (UID: \"063c5d2e-68a5-497a-b7bd-f95d45804848\") " pod="openstack-kuttl-tests/glance-db-sync-k6vlv" Dec 03 11:38:06 crc kubenswrapper[5052]: I1203 11:38:06.023852 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-b2b64\" (UniqueName: \"kubernetes.io/projected/063c5d2e-68a5-497a-b7bd-f95d45804848-kube-api-access-b2b64\") pod \"glance-db-sync-k6vlv\" (UID: \"063c5d2e-68a5-497a-b7bd-f95d45804848\") " pod="openstack-kuttl-tests/glance-db-sync-k6vlv" Dec 03 11:38:06 crc kubenswrapper[5052]: I1203 11:38:06.215880 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-db-sync-k6vlv" Dec 03 11:38:06 crc kubenswrapper[5052]: I1203 11:38:06.483191 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"d19f0c21-bb49-40a1-87fa-e55188308729","Type":"ContainerStarted","Data":"38627e147a9f39cc2d052969a6aa30eeb14f6d31573e540afa5859023f9656d8"} Dec 03 11:38:06 crc kubenswrapper[5052]: I1203 11:38:06.483549 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"d19f0c21-bb49-40a1-87fa-e55188308729","Type":"ContainerStarted","Data":"e776b8b08a7136d5453955d041fce850605188fd0e17c51f2024905c9aa86aec"} Dec 03 11:38:06 crc kubenswrapper[5052]: I1203 11:38:06.483560 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"d19f0c21-bb49-40a1-87fa-e55188308729","Type":"ContainerStarted","Data":"375e74dc879bff82efc908e9893ad30ab3c979218f143e6a3e6fcd9936d34248"} Dec 03 11:38:06 crc kubenswrapper[5052]: I1203 11:38:06.483569 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"d19f0c21-bb49-40a1-87fa-e55188308729","Type":"ContainerStarted","Data":"3a64f478f235a4fb6291efc13b9a7d651e52032e9126c5a8ee70cbaa6f7bc0c4"} Dec 03 11:38:06 crc kubenswrapper[5052]: I1203 11:38:06.697489 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-db-sync-k6vlv"] Dec 03 11:38:06 crc kubenswrapper[5052]: I1203 11:38:06.892608 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 11:38:07 crc kubenswrapper[5052]: I1203 11:38:07.491249 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-db-sync-k6vlv" event={"ID":"063c5d2e-68a5-497a-b7bd-f95d45804848","Type":"ContainerStarted","Data":"f240f6af4fd3ce44721e4cc39bb4c69c2cfd30b0fa77f7dee789e7b31083f7ee"} Dec 03 11:38:08 crc kubenswrapper[5052]: I1203 11:38:08.500469 5052 generic.go:334] "Generic (PLEG): container finished" podID="9944d9af-7a7b-4773-8279-72914d63ef5f" containerID="a4c0b116720eadc0b68b284395f846e2b236870d208137226f3a533a48d6959e" exitCode=0 Dec 03 11:38:08 crc kubenswrapper[5052]: I1203 11:38:08.500529 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-server-0" event={"ID":"9944d9af-7a7b-4773-8279-72914d63ef5f","Type":"ContainerDied","Data":"a4c0b116720eadc0b68b284395f846e2b236870d208137226f3a533a48d6959e"} Dec 03 11:38:08 crc kubenswrapper[5052]: I1203 11:38:08.505156 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"d19f0c21-bb49-40a1-87fa-e55188308729","Type":"ContainerStarted","Data":"e36b32b951337695419521a1f3271a1e8e2cac46f10174c506c5e0e369ad8704"} Dec 03 11:38:08 crc kubenswrapper[5052]: I1203 11:38:08.505192 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"d19f0c21-bb49-40a1-87fa-e55188308729","Type":"ContainerStarted","Data":"66b82b9a9602e546754f3199323f445795063d7d84d38d42c9e7bdead8e2d119"} Dec 03 11:38:08 crc kubenswrapper[5052]: I1203 11:38:08.505201 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"d19f0c21-bb49-40a1-87fa-e55188308729","Type":"ContainerStarted","Data":"8b82c3eafedcad816bdd8f848dfa51930b6fcd64c66cf454cbb3af2debd04b86"} Dec 03 11:38:08 crc kubenswrapper[5052]: I1203 11:38:08.505211 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"d19f0c21-bb49-40a1-87fa-e55188308729","Type":"ContainerStarted","Data":"cc5f444de12548f88c1caa6a61bd1efc6e2fd9da4c672919931aac6ba15d3f15"} Dec 03 11:38:08 crc kubenswrapper[5052]: I1203 11:38:08.506836 5052 generic.go:334] "Generic (PLEG): container finished" podID="daa521fc-7e1d-44d2-b7d8-6aab003cfb66" containerID="0fbd3a4d4383d62bcd48e87a34d3dbb4ccd0417936dd06ed21f799d8c04dd391" exitCode=0 Dec 03 11:38:08 crc kubenswrapper[5052]: I1203 11:38:08.506871 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" event={"ID":"daa521fc-7e1d-44d2-b7d8-6aab003cfb66","Type":"ContainerDied","Data":"0fbd3a4d4383d62bcd48e87a34d3dbb4ccd0417936dd06ed21f799d8c04dd391"} Dec 03 11:38:09 crc kubenswrapper[5052]: I1203 11:38:09.531263 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" event={"ID":"daa521fc-7e1d-44d2-b7d8-6aab003cfb66","Type":"ContainerStarted","Data":"f084100fa637b545df332be6f942cc3ba527b4e7f8898dd5c51db5555433c0fd"} Dec 03 11:38:09 crc kubenswrapper[5052]: I1203 11:38:09.531826 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 11:38:09 crc kubenswrapper[5052]: I1203 11:38:09.533854 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-server-0" event={"ID":"9944d9af-7a7b-4773-8279-72914d63ef5f","Type":"ContainerStarted","Data":"c7a780b226312d9ca6d3ee18cc8115287601a5ebf3182143b2ccad2a68a3f43a"} Dec 03 11:38:09 crc kubenswrapper[5052]: I1203 11:38:09.534363 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 11:38:09 crc kubenswrapper[5052]: I1203 11:38:09.560361 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" podStartSLOduration=41.789914618 podStartE2EDuration="48.560345018s" podCreationTimestamp="2025-12-03 11:37:21 +0000 UTC" firstStartedPulling="2025-12-03 11:37:26.70209896 +0000 UTC m=+1037.290340724" lastFinishedPulling="2025-12-03 11:37:33.47252936 +0000 UTC m=+1044.060771124" observedRunningTime="2025-12-03 11:38:09.554818678 +0000 UTC m=+1080.143060442" watchObservedRunningTime="2025-12-03 11:38:09.560345018 +0000 UTC m=+1080.148586772" Dec 03 11:38:09 crc kubenswrapper[5052]: I1203 11:38:09.586469 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/rabbitmq-server-0" podStartSLOduration=41.954555134 podStartE2EDuration="48.586452194s" podCreationTimestamp="2025-12-03 11:37:21 +0000 UTC" firstStartedPulling="2025-12-03 11:37:26.762521679 +0000 UTC m=+1037.350763443" lastFinishedPulling="2025-12-03 11:37:33.394418739 +0000 UTC m=+1043.982660503" observedRunningTime="2025-12-03 11:38:09.583651663 +0000 UTC m=+1080.171893437" watchObservedRunningTime="2025-12-03 11:38:09.586452194 +0000 UTC m=+1080.174693958" Dec 03 11:38:10 crc kubenswrapper[5052]: I1203 11:38:10.548403 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"d19f0c21-bb49-40a1-87fa-e55188308729","Type":"ContainerStarted","Data":"e0f3046cd638bd8d68fc0858b30e3c71c02254e1579ac5fdf0d14362c56787ae"} Dec 03 11:38:10 crc kubenswrapper[5052]: I1203 11:38:10.548743 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"d19f0c21-bb49-40a1-87fa-e55188308729","Type":"ContainerStarted","Data":"842157c852d9af69e655dafb6e96753c92d5a174fac182e141f97a78bcaf8577"} Dec 03 11:38:10 crc kubenswrapper[5052]: I1203 11:38:10.548756 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"d19f0c21-bb49-40a1-87fa-e55188308729","Type":"ContainerStarted","Data":"480a4695021f8caec2d8b5d9b753bac979ecb6a2f7643abdd8ce149daf4a8b4a"} Dec 03 11:38:10 crc kubenswrapper[5052]: I1203 11:38:10.548765 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"d19f0c21-bb49-40a1-87fa-e55188308729","Type":"ContainerStarted","Data":"6f7206455d66da31b1ca0c6033a701e9ef605c548a9309c2b0967a2b9726a603"} Dec 03 11:38:11 crc kubenswrapper[5052]: I1203 11:38:11.562086 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"d19f0c21-bb49-40a1-87fa-e55188308729","Type":"ContainerStarted","Data":"907a694952444699abbc7ea53d63f1c5b7ce75550d76d99fb7d54e27cbbb8c66"} Dec 03 11:38:11 crc kubenswrapper[5052]: I1203 11:38:11.563031 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"d19f0c21-bb49-40a1-87fa-e55188308729","Type":"ContainerStarted","Data":"198e5b3e40307e3ae065a4823c3cadc6edfe24c354d3dc8885bdeb1b65e4a57a"} Dec 03 11:38:11 crc kubenswrapper[5052]: I1203 11:38:11.563051 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"d19f0c21-bb49-40a1-87fa-e55188308729","Type":"ContainerStarted","Data":"1feeef96663c3d2882692e8837c421c0dab4a5a5be167acf6810b788e07033be"} Dec 03 11:38:11 crc kubenswrapper[5052]: I1203 11:38:11.599201 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/swift-storage-0" podStartSLOduration=19.479529644 podStartE2EDuration="24.599183668s" podCreationTimestamp="2025-12-03 11:37:47 +0000 UTC" firstStartedPulling="2025-12-03 11:38:04.613302411 +0000 UTC m=+1075.201544175" lastFinishedPulling="2025-12-03 11:38:09.732956445 +0000 UTC m=+1080.321198199" observedRunningTime="2025-12-03 11:38:11.595861772 +0000 UTC m=+1082.184103536" watchObservedRunningTime="2025-12-03 11:38:11.599183668 +0000 UTC m=+1082.187425432" Dec 03 11:38:11 crc kubenswrapper[5052]: I1203 11:38:11.851788 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-58b8ddd7c-jtml2"] Dec 03 11:38:11 crc kubenswrapper[5052]: I1203 11:38:11.853932 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-58b8ddd7c-jtml2" Dec 03 11:38:11 crc kubenswrapper[5052]: I1203 11:38:11.855929 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"dns-swift-storage-0" Dec 03 11:38:11 crc kubenswrapper[5052]: I1203 11:38:11.884281 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-58b8ddd7c-jtml2"] Dec 03 11:38:12 crc kubenswrapper[5052]: I1203 11:38:12.040387 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k6nvd\" (UniqueName: \"kubernetes.io/projected/1e003f91-c464-4200-94c1-18c2bea8ee8a-kube-api-access-k6nvd\") pod \"dnsmasq-dnsmasq-58b8ddd7c-jtml2\" (UID: \"1e003f91-c464-4200-94c1-18c2bea8ee8a\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-58b8ddd7c-jtml2" Dec 03 11:38:12 crc kubenswrapper[5052]: I1203 11:38:12.040426 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1e003f91-c464-4200-94c1-18c2bea8ee8a-config\") pod \"dnsmasq-dnsmasq-58b8ddd7c-jtml2\" (UID: \"1e003f91-c464-4200-94c1-18c2bea8ee8a\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-58b8ddd7c-jtml2" Dec 03 11:38:12 crc kubenswrapper[5052]: I1203 11:38:12.040463 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/1e003f91-c464-4200-94c1-18c2bea8ee8a-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-58b8ddd7c-jtml2\" (UID: \"1e003f91-c464-4200-94c1-18c2bea8ee8a\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-58b8ddd7c-jtml2" Dec 03 11:38:12 crc kubenswrapper[5052]: I1203 11:38:12.040997 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dns-swift-storage-0\" (UniqueName: \"kubernetes.io/configmap/1e003f91-c464-4200-94c1-18c2bea8ee8a-dns-swift-storage-0\") pod \"dnsmasq-dnsmasq-58b8ddd7c-jtml2\" (UID: \"1e003f91-c464-4200-94c1-18c2bea8ee8a\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-58b8ddd7c-jtml2" Dec 03 11:38:12 crc kubenswrapper[5052]: I1203 11:38:12.142531 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-k6nvd\" (UniqueName: \"kubernetes.io/projected/1e003f91-c464-4200-94c1-18c2bea8ee8a-kube-api-access-k6nvd\") pod \"dnsmasq-dnsmasq-58b8ddd7c-jtml2\" (UID: \"1e003f91-c464-4200-94c1-18c2bea8ee8a\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-58b8ddd7c-jtml2" Dec 03 11:38:12 crc kubenswrapper[5052]: I1203 11:38:12.142579 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1e003f91-c464-4200-94c1-18c2bea8ee8a-config\") pod \"dnsmasq-dnsmasq-58b8ddd7c-jtml2\" (UID: \"1e003f91-c464-4200-94c1-18c2bea8ee8a\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-58b8ddd7c-jtml2" Dec 03 11:38:12 crc kubenswrapper[5052]: I1203 11:38:12.142620 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/1e003f91-c464-4200-94c1-18c2bea8ee8a-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-58b8ddd7c-jtml2\" (UID: \"1e003f91-c464-4200-94c1-18c2bea8ee8a\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-58b8ddd7c-jtml2" Dec 03 11:38:12 crc kubenswrapper[5052]: I1203 11:38:12.142691 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"dns-swift-storage-0\" (UniqueName: \"kubernetes.io/configmap/1e003f91-c464-4200-94c1-18c2bea8ee8a-dns-swift-storage-0\") pod \"dnsmasq-dnsmasq-58b8ddd7c-jtml2\" (UID: \"1e003f91-c464-4200-94c1-18c2bea8ee8a\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-58b8ddd7c-jtml2" Dec 03 11:38:12 crc kubenswrapper[5052]: I1203 11:38:12.143823 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/1e003f91-c464-4200-94c1-18c2bea8ee8a-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-58b8ddd7c-jtml2\" (UID: \"1e003f91-c464-4200-94c1-18c2bea8ee8a\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-58b8ddd7c-jtml2" Dec 03 11:38:12 crc kubenswrapper[5052]: I1203 11:38:12.143854 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"dns-swift-storage-0\" (UniqueName: \"kubernetes.io/configmap/1e003f91-c464-4200-94c1-18c2bea8ee8a-dns-swift-storage-0\") pod \"dnsmasq-dnsmasq-58b8ddd7c-jtml2\" (UID: \"1e003f91-c464-4200-94c1-18c2bea8ee8a\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-58b8ddd7c-jtml2" Dec 03 11:38:12 crc kubenswrapper[5052]: I1203 11:38:12.143908 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1e003f91-c464-4200-94c1-18c2bea8ee8a-config\") pod \"dnsmasq-dnsmasq-58b8ddd7c-jtml2\" (UID: \"1e003f91-c464-4200-94c1-18c2bea8ee8a\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-58b8ddd7c-jtml2" Dec 03 11:38:12 crc kubenswrapper[5052]: I1203 11:38:12.162395 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-k6nvd\" (UniqueName: \"kubernetes.io/projected/1e003f91-c464-4200-94c1-18c2bea8ee8a-kube-api-access-k6nvd\") pod \"dnsmasq-dnsmasq-58b8ddd7c-jtml2\" (UID: \"1e003f91-c464-4200-94c1-18c2bea8ee8a\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-58b8ddd7c-jtml2" Dec 03 11:38:12 crc kubenswrapper[5052]: I1203 11:38:12.171276 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-58b8ddd7c-jtml2" Dec 03 11:38:12 crc kubenswrapper[5052]: I1203 11:38:12.780094 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-58b8ddd7c-jtml2"] Dec 03 11:38:21 crc kubenswrapper[5052]: W1203 11:38:21.505753 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1e003f91_c464_4200_94c1_18c2bea8ee8a.slice/crio-63a143edc85f8a66a5a6b506dfb03193e5395d27f22fe5cc397c54fea46d8f52 WatchSource:0}: Error finding container 63a143edc85f8a66a5a6b506dfb03193e5395d27f22fe5cc397c54fea46d8f52: Status 404 returned error can't find the container with id 63a143edc85f8a66a5a6b506dfb03193e5395d27f22fe5cc397c54fea46d8f52 Dec 03 11:38:21 crc kubenswrapper[5052]: E1203 11:38:21.540495 5052 log.go:32] "PullImage from image service failed" err="rpc error: code = Canceled desc = copying config: context canceled" image="quay.io/podified-antelope-centos9/openstack-glance-api@sha256:481073ac9deefb38bbd989aaa8dd7aedb4e0af26017f4883f85fce433380bf63" Dec 03 11:38:21 crc kubenswrapper[5052]: E1203 11:38:21.541077 5052 kuberuntime_manager.go:1274] "Unhandled Error" err="container &Container{Name:glance-db-sync,Image:quay.io/podified-antelope-centos9/openstack-glance-api@sha256:481073ac9deefb38bbd989aaa8dd7aedb4e0af26017f4883f85fce433380bf63,Command:[/bin/bash],Args:[-c /usr/local/bin/kolla_start],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{EnvVar{Name:KOLLA_BOOTSTRAP,Value:true,ValueFrom:nil,},EnvVar{Name:KOLLA_CONFIG_STRATEGY,Value:COPY_ALWAYS,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:db-sync-config-data,ReadOnly:true,MountPath:/etc/glance/glance.conf.d,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:config-data,ReadOnly:true,MountPath:/etc/my.cnf,SubPath:my.cnf,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:config-data,ReadOnly:true,MountPath:/var/lib/kolla/config_files/config.json,SubPath:db-sync-config.json,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:combined-ca-bundle,ReadOnly:true,MountPath:/etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem,SubPath:tls-ca-bundle.pem,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-b2b64,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[MKNOD],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*42415,RunAsNonRoot:*true,ReadOnlyRootFilesystem:nil,AllowPrivilegeEscalation:nil,RunAsGroup:*42415,ProcMount:nil,WindowsOptions:nil,SeccompProfile:&SeccompProfile{Type:RuntimeDefault,LocalhostProfile:nil,},AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod glance-db-sync-k6vlv_openstack-kuttl-tests(063c5d2e-68a5-497a-b7bd-f95d45804848): ErrImagePull: rpc error: code = Canceled desc = copying config: context canceled" logger="UnhandledError" Dec 03 11:38:21 crc kubenswrapper[5052]: E1203 11:38:21.542297 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"glance-db-sync\" with ErrImagePull: \"rpc error: code = Canceled desc = copying config: context canceled\"" pod="openstack-kuttl-tests/glance-db-sync-k6vlv" podUID="063c5d2e-68a5-497a-b7bd-f95d45804848" Dec 03 11:38:21 crc kubenswrapper[5052]: I1203 11:38:21.656290 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-58b8ddd7c-jtml2" event={"ID":"1e003f91-c464-4200-94c1-18c2bea8ee8a","Type":"ContainerStarted","Data":"63a143edc85f8a66a5a6b506dfb03193e5395d27f22fe5cc397c54fea46d8f52"} Dec 03 11:38:21 crc kubenswrapper[5052]: E1203 11:38:21.657356 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"glance-db-sync\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/podified-antelope-centos9/openstack-glance-api@sha256:481073ac9deefb38bbd989aaa8dd7aedb4e0af26017f4883f85fce433380bf63\\\"\"" pod="openstack-kuttl-tests/glance-db-sync-k6vlv" podUID="063c5d2e-68a5-497a-b7bd-f95d45804848" Dec 03 11:38:22 crc kubenswrapper[5052]: I1203 11:38:22.435671 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 11:38:22 crc kubenswrapper[5052]: I1203 11:38:22.663930 5052 generic.go:334] "Generic (PLEG): container finished" podID="1e003f91-c464-4200-94c1-18c2bea8ee8a" containerID="d5282e088e97a4376d50474919f0ef21c7f76187e137d2647b7d27033f930758" exitCode=0 Dec 03 11:38:22 crc kubenswrapper[5052]: I1203 11:38:22.663968 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-58b8ddd7c-jtml2" event={"ID":"1e003f91-c464-4200-94c1-18c2bea8ee8a","Type":"ContainerDied","Data":"d5282e088e97a4376d50474919f0ef21c7f76187e137d2647b7d27033f930758"} Dec 03 11:38:22 crc kubenswrapper[5052]: I1203 11:38:22.779692 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 11:38:23 crc kubenswrapper[5052]: I1203 11:38:23.729792 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/barbican-db-create-nrwg6"] Dec 03 11:38:23 crc kubenswrapper[5052]: I1203 11:38:23.731392 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-db-create-nrwg6" Dec 03 11:38:23 crc kubenswrapper[5052]: I1203 11:38:23.737812 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/neutron-db-create-9kfp5"] Dec 03 11:38:23 crc kubenswrapper[5052]: I1203 11:38:23.738851 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-db-create-9kfp5" Dec 03 11:38:23 crc kubenswrapper[5052]: I1203 11:38:23.752753 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/barbican-c57b-account-create-update-nmdhk"] Dec 03 11:38:23 crc kubenswrapper[5052]: I1203 11:38:23.753796 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-c57b-account-create-update-nmdhk" Dec 03 11:38:23 crc kubenswrapper[5052]: I1203 11:38:23.756292 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"barbican-db-secret" Dec 03 11:38:23 crc kubenswrapper[5052]: I1203 11:38:23.765481 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/cinder-db-create-c922p"] Dec 03 11:38:23 crc kubenswrapper[5052]: I1203 11:38:23.766912 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-db-create-c922p" Dec 03 11:38:23 crc kubenswrapper[5052]: I1203 11:38:23.775559 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/neutron-bfb7-account-create-update-2684r"] Dec 03 11:38:23 crc kubenswrapper[5052]: I1203 11:38:23.777224 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-bfb7-account-create-update-2684r" Dec 03 11:38:23 crc kubenswrapper[5052]: I1203 11:38:23.780841 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"neutron-db-secret" Dec 03 11:38:23 crc kubenswrapper[5052]: I1203 11:38:23.796965 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/keystone-db-sync-6s975"] Dec 03 11:38:23 crc kubenswrapper[5052]: I1203 11:38:23.798112 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-db-sync-6s975" Dec 03 11:38:23 crc kubenswrapper[5052]: I1203 11:38:23.801720 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone-scripts" Dec 03 11:38:23 crc kubenswrapper[5052]: I1203 11:38:23.801878 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone-config-data" Dec 03 11:38:23 crc kubenswrapper[5052]: I1203 11:38:23.801944 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone" Dec 03 11:38:23 crc kubenswrapper[5052]: I1203 11:38:23.802203 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone-keystone-dockercfg-tt6lh" Dec 03 11:38:23 crc kubenswrapper[5052]: I1203 11:38:23.805556 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-c57b-account-create-update-nmdhk"] Dec 03 11:38:23 crc kubenswrapper[5052]: I1203 11:38:23.815004 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-bfb7-account-create-update-2684r"] Dec 03 11:38:23 crc kubenswrapper[5052]: I1203 11:38:23.828930 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-db-create-c922p"] Dec 03 11:38:23 crc kubenswrapper[5052]: I1203 11:38:23.834160 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/594d60cb-8133-46b7-8cf9-960b1b12ffa3-combined-ca-bundle\") pod \"keystone-db-sync-6s975\" (UID: \"594d60cb-8133-46b7-8cf9-960b1b12ffa3\") " pod="openstack-kuttl-tests/keystone-db-sync-6s975" Dec 03 11:38:23 crc kubenswrapper[5052]: I1203 11:38:23.834338 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/594d60cb-8133-46b7-8cf9-960b1b12ffa3-config-data\") pod \"keystone-db-sync-6s975\" (UID: \"594d60cb-8133-46b7-8cf9-960b1b12ffa3\") " pod="openstack-kuttl-tests/keystone-db-sync-6s975" Dec 03 11:38:23 crc kubenswrapper[5052]: I1203 11:38:23.834458 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d4fdx\" (UniqueName: \"kubernetes.io/projected/c0ef8371-9171-45c5-9202-4d16f774ecf4-kube-api-access-d4fdx\") pod \"barbican-db-create-nrwg6\" (UID: \"c0ef8371-9171-45c5-9202-4d16f774ecf4\") " pod="openstack-kuttl-tests/barbican-db-create-nrwg6" Dec 03 11:38:23 crc kubenswrapper[5052]: I1203 11:38:23.834596 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-krjq9\" (UniqueName: \"kubernetes.io/projected/0cbd53a9-fe89-4b73-b56f-d54b8fec2230-kube-api-access-krjq9\") pod \"barbican-c57b-account-create-update-nmdhk\" (UID: \"0cbd53a9-fe89-4b73-b56f-d54b8fec2230\") " pod="openstack-kuttl-tests/barbican-c57b-account-create-update-nmdhk" Dec 03 11:38:23 crc kubenswrapper[5052]: I1203 11:38:23.834724 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/2ff66059-68e1-4c42-8bde-6c599a549ad5-operator-scripts\") pod \"neutron-bfb7-account-create-update-2684r\" (UID: \"2ff66059-68e1-4c42-8bde-6c599a549ad5\") " pod="openstack-kuttl-tests/neutron-bfb7-account-create-update-2684r" Dec 03 11:38:23 crc kubenswrapper[5052]: I1203 11:38:23.834838 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/ec205491-fbcd-43b5-88d3-d98ae0160f15-operator-scripts\") pod \"cinder-db-create-c922p\" (UID: \"ec205491-fbcd-43b5-88d3-d98ae0160f15\") " pod="openstack-kuttl-tests/cinder-db-create-c922p" Dec 03 11:38:23 crc kubenswrapper[5052]: I1203 11:38:23.834986 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4qjl9\" (UniqueName: \"kubernetes.io/projected/2ff66059-68e1-4c42-8bde-6c599a549ad5-kube-api-access-4qjl9\") pod \"neutron-bfb7-account-create-update-2684r\" (UID: \"2ff66059-68e1-4c42-8bde-6c599a549ad5\") " pod="openstack-kuttl-tests/neutron-bfb7-account-create-update-2684r" Dec 03 11:38:23 crc kubenswrapper[5052]: I1203 11:38:23.835139 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hvq22\" (UniqueName: \"kubernetes.io/projected/ec205491-fbcd-43b5-88d3-d98ae0160f15-kube-api-access-hvq22\") pod \"cinder-db-create-c922p\" (UID: \"ec205491-fbcd-43b5-88d3-d98ae0160f15\") " pod="openstack-kuttl-tests/cinder-db-create-c922p" Dec 03 11:38:23 crc kubenswrapper[5052]: I1203 11:38:23.835259 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/0cbd53a9-fe89-4b73-b56f-d54b8fec2230-operator-scripts\") pod \"barbican-c57b-account-create-update-nmdhk\" (UID: \"0cbd53a9-fe89-4b73-b56f-d54b8fec2230\") " pod="openstack-kuttl-tests/barbican-c57b-account-create-update-nmdhk" Dec 03 11:38:23 crc kubenswrapper[5052]: I1203 11:38:23.835374 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mjr8d\" (UniqueName: \"kubernetes.io/projected/594d60cb-8133-46b7-8cf9-960b1b12ffa3-kube-api-access-mjr8d\") pod \"keystone-db-sync-6s975\" (UID: \"594d60cb-8133-46b7-8cf9-960b1b12ffa3\") " pod="openstack-kuttl-tests/keystone-db-sync-6s975" Dec 03 11:38:23 crc kubenswrapper[5052]: I1203 11:38:23.835479 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/f4e66dc5-2979-4ac3-b5a1-959f36db75da-operator-scripts\") pod \"neutron-db-create-9kfp5\" (UID: \"f4e66dc5-2979-4ac3-b5a1-959f36db75da\") " pod="openstack-kuttl-tests/neutron-db-create-9kfp5" Dec 03 11:38:23 crc kubenswrapper[5052]: I1203 11:38:23.835627 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/c0ef8371-9171-45c5-9202-4d16f774ecf4-operator-scripts\") pod \"barbican-db-create-nrwg6\" (UID: \"c0ef8371-9171-45c5-9202-4d16f774ecf4\") " pod="openstack-kuttl-tests/barbican-db-create-nrwg6" Dec 03 11:38:23 crc kubenswrapper[5052]: I1203 11:38:23.835739 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wvcng\" (UniqueName: \"kubernetes.io/projected/f4e66dc5-2979-4ac3-b5a1-959f36db75da-kube-api-access-wvcng\") pod \"neutron-db-create-9kfp5\" (UID: \"f4e66dc5-2979-4ac3-b5a1-959f36db75da\") " pod="openstack-kuttl-tests/neutron-db-create-9kfp5" Dec 03 11:38:23 crc kubenswrapper[5052]: I1203 11:38:23.839816 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-db-create-9kfp5"] Dec 03 11:38:23 crc kubenswrapper[5052]: I1203 11:38:23.846590 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-db-create-nrwg6"] Dec 03 11:38:23 crc kubenswrapper[5052]: I1203 11:38:23.855602 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-db-sync-6s975"] Dec 03 11:38:23 crc kubenswrapper[5052]: I1203 11:38:23.874374 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/cinder-5365-account-create-update-w5fnk"] Dec 03 11:38:23 crc kubenswrapper[5052]: I1203 11:38:23.875581 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-5365-account-create-update-w5fnk" Dec 03 11:38:23 crc kubenswrapper[5052]: I1203 11:38:23.878428 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cinder-db-secret" Dec 03 11:38:23 crc kubenswrapper[5052]: I1203 11:38:23.913807 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-5365-account-create-update-w5fnk"] Dec 03 11:38:23 crc kubenswrapper[5052]: I1203 11:38:23.936928 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/2ff66059-68e1-4c42-8bde-6c599a549ad5-operator-scripts\") pod \"neutron-bfb7-account-create-update-2684r\" (UID: \"2ff66059-68e1-4c42-8bde-6c599a549ad5\") " pod="openstack-kuttl-tests/neutron-bfb7-account-create-update-2684r" Dec 03 11:38:23 crc kubenswrapper[5052]: I1203 11:38:23.936970 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/ec205491-fbcd-43b5-88d3-d98ae0160f15-operator-scripts\") pod \"cinder-db-create-c922p\" (UID: \"ec205491-fbcd-43b5-88d3-d98ae0160f15\") " pod="openstack-kuttl-tests/cinder-db-create-c922p" Dec 03 11:38:23 crc kubenswrapper[5052]: I1203 11:38:23.937004 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-4qjl9\" (UniqueName: \"kubernetes.io/projected/2ff66059-68e1-4c42-8bde-6c599a549ad5-kube-api-access-4qjl9\") pod \"neutron-bfb7-account-create-update-2684r\" (UID: \"2ff66059-68e1-4c42-8bde-6c599a549ad5\") " pod="openstack-kuttl-tests/neutron-bfb7-account-create-update-2684r" Dec 03 11:38:23 crc kubenswrapper[5052]: I1203 11:38:23.937048 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/63477217-d60a-4fd8-a58c-5b37cd2cd23f-operator-scripts\") pod \"cinder-5365-account-create-update-w5fnk\" (UID: \"63477217-d60a-4fd8-a58c-5b37cd2cd23f\") " pod="openstack-kuttl-tests/cinder-5365-account-create-update-w5fnk" Dec 03 11:38:23 crc kubenswrapper[5052]: I1203 11:38:23.937070 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ccwsh\" (UniqueName: \"kubernetes.io/projected/63477217-d60a-4fd8-a58c-5b37cd2cd23f-kube-api-access-ccwsh\") pod \"cinder-5365-account-create-update-w5fnk\" (UID: \"63477217-d60a-4fd8-a58c-5b37cd2cd23f\") " pod="openstack-kuttl-tests/cinder-5365-account-create-update-w5fnk" Dec 03 11:38:23 crc kubenswrapper[5052]: I1203 11:38:23.937102 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-hvq22\" (UniqueName: \"kubernetes.io/projected/ec205491-fbcd-43b5-88d3-d98ae0160f15-kube-api-access-hvq22\") pod \"cinder-db-create-c922p\" (UID: \"ec205491-fbcd-43b5-88d3-d98ae0160f15\") " pod="openstack-kuttl-tests/cinder-db-create-c922p" Dec 03 11:38:23 crc kubenswrapper[5052]: I1203 11:38:23.937128 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/0cbd53a9-fe89-4b73-b56f-d54b8fec2230-operator-scripts\") pod \"barbican-c57b-account-create-update-nmdhk\" (UID: \"0cbd53a9-fe89-4b73-b56f-d54b8fec2230\") " pod="openstack-kuttl-tests/barbican-c57b-account-create-update-nmdhk" Dec 03 11:38:23 crc kubenswrapper[5052]: I1203 11:38:23.937147 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-mjr8d\" (UniqueName: \"kubernetes.io/projected/594d60cb-8133-46b7-8cf9-960b1b12ffa3-kube-api-access-mjr8d\") pod \"keystone-db-sync-6s975\" (UID: \"594d60cb-8133-46b7-8cf9-960b1b12ffa3\") " pod="openstack-kuttl-tests/keystone-db-sync-6s975" Dec 03 11:38:23 crc kubenswrapper[5052]: I1203 11:38:23.937170 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/f4e66dc5-2979-4ac3-b5a1-959f36db75da-operator-scripts\") pod \"neutron-db-create-9kfp5\" (UID: \"f4e66dc5-2979-4ac3-b5a1-959f36db75da\") " pod="openstack-kuttl-tests/neutron-db-create-9kfp5" Dec 03 11:38:23 crc kubenswrapper[5052]: I1203 11:38:23.937190 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/c0ef8371-9171-45c5-9202-4d16f774ecf4-operator-scripts\") pod \"barbican-db-create-nrwg6\" (UID: \"c0ef8371-9171-45c5-9202-4d16f774ecf4\") " pod="openstack-kuttl-tests/barbican-db-create-nrwg6" Dec 03 11:38:23 crc kubenswrapper[5052]: I1203 11:38:23.937207 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-wvcng\" (UniqueName: \"kubernetes.io/projected/f4e66dc5-2979-4ac3-b5a1-959f36db75da-kube-api-access-wvcng\") pod \"neutron-db-create-9kfp5\" (UID: \"f4e66dc5-2979-4ac3-b5a1-959f36db75da\") " pod="openstack-kuttl-tests/neutron-db-create-9kfp5" Dec 03 11:38:23 crc kubenswrapper[5052]: I1203 11:38:23.937232 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/594d60cb-8133-46b7-8cf9-960b1b12ffa3-combined-ca-bundle\") pod \"keystone-db-sync-6s975\" (UID: \"594d60cb-8133-46b7-8cf9-960b1b12ffa3\") " pod="openstack-kuttl-tests/keystone-db-sync-6s975" Dec 03 11:38:23 crc kubenswrapper[5052]: I1203 11:38:23.937251 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/594d60cb-8133-46b7-8cf9-960b1b12ffa3-config-data\") pod \"keystone-db-sync-6s975\" (UID: \"594d60cb-8133-46b7-8cf9-960b1b12ffa3\") " pod="openstack-kuttl-tests/keystone-db-sync-6s975" Dec 03 11:38:23 crc kubenswrapper[5052]: I1203 11:38:23.937267 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-d4fdx\" (UniqueName: \"kubernetes.io/projected/c0ef8371-9171-45c5-9202-4d16f774ecf4-kube-api-access-d4fdx\") pod \"barbican-db-create-nrwg6\" (UID: \"c0ef8371-9171-45c5-9202-4d16f774ecf4\") " pod="openstack-kuttl-tests/barbican-db-create-nrwg6" Dec 03 11:38:23 crc kubenswrapper[5052]: I1203 11:38:23.937291 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-krjq9\" (UniqueName: \"kubernetes.io/projected/0cbd53a9-fe89-4b73-b56f-d54b8fec2230-kube-api-access-krjq9\") pod \"barbican-c57b-account-create-update-nmdhk\" (UID: \"0cbd53a9-fe89-4b73-b56f-d54b8fec2230\") " pod="openstack-kuttl-tests/barbican-c57b-account-create-update-nmdhk" Dec 03 11:38:23 crc kubenswrapper[5052]: I1203 11:38:23.937723 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/ec205491-fbcd-43b5-88d3-d98ae0160f15-operator-scripts\") pod \"cinder-db-create-c922p\" (UID: \"ec205491-fbcd-43b5-88d3-d98ae0160f15\") " pod="openstack-kuttl-tests/cinder-db-create-c922p" Dec 03 11:38:23 crc kubenswrapper[5052]: I1203 11:38:23.938556 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/f4e66dc5-2979-4ac3-b5a1-959f36db75da-operator-scripts\") pod \"neutron-db-create-9kfp5\" (UID: \"f4e66dc5-2979-4ac3-b5a1-959f36db75da\") " pod="openstack-kuttl-tests/neutron-db-create-9kfp5" Dec 03 11:38:23 crc kubenswrapper[5052]: I1203 11:38:23.938666 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/0cbd53a9-fe89-4b73-b56f-d54b8fec2230-operator-scripts\") pod \"barbican-c57b-account-create-update-nmdhk\" (UID: \"0cbd53a9-fe89-4b73-b56f-d54b8fec2230\") " pod="openstack-kuttl-tests/barbican-c57b-account-create-update-nmdhk" Dec 03 11:38:23 crc kubenswrapper[5052]: I1203 11:38:23.938749 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/c0ef8371-9171-45c5-9202-4d16f774ecf4-operator-scripts\") pod \"barbican-db-create-nrwg6\" (UID: \"c0ef8371-9171-45c5-9202-4d16f774ecf4\") " pod="openstack-kuttl-tests/barbican-db-create-nrwg6" Dec 03 11:38:23 crc kubenswrapper[5052]: I1203 11:38:23.952887 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/594d60cb-8133-46b7-8cf9-960b1b12ffa3-combined-ca-bundle\") pod \"keystone-db-sync-6s975\" (UID: \"594d60cb-8133-46b7-8cf9-960b1b12ffa3\") " pod="openstack-kuttl-tests/keystone-db-sync-6s975" Dec 03 11:38:23 crc kubenswrapper[5052]: I1203 11:38:23.956262 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/594d60cb-8133-46b7-8cf9-960b1b12ffa3-config-data\") pod \"keystone-db-sync-6s975\" (UID: \"594d60cb-8133-46b7-8cf9-960b1b12ffa3\") " pod="openstack-kuttl-tests/keystone-db-sync-6s975" Dec 03 11:38:23 crc kubenswrapper[5052]: I1203 11:38:23.956779 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-krjq9\" (UniqueName: \"kubernetes.io/projected/0cbd53a9-fe89-4b73-b56f-d54b8fec2230-kube-api-access-krjq9\") pod \"barbican-c57b-account-create-update-nmdhk\" (UID: \"0cbd53a9-fe89-4b73-b56f-d54b8fec2230\") " pod="openstack-kuttl-tests/barbican-c57b-account-create-update-nmdhk" Dec 03 11:38:23 crc kubenswrapper[5052]: I1203 11:38:23.957259 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-wvcng\" (UniqueName: \"kubernetes.io/projected/f4e66dc5-2979-4ac3-b5a1-959f36db75da-kube-api-access-wvcng\") pod \"neutron-db-create-9kfp5\" (UID: \"f4e66dc5-2979-4ac3-b5a1-959f36db75da\") " pod="openstack-kuttl-tests/neutron-db-create-9kfp5" Dec 03 11:38:23 crc kubenswrapper[5052]: I1203 11:38:23.964453 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-hvq22\" (UniqueName: \"kubernetes.io/projected/ec205491-fbcd-43b5-88d3-d98ae0160f15-kube-api-access-hvq22\") pod \"cinder-db-create-c922p\" (UID: \"ec205491-fbcd-43b5-88d3-d98ae0160f15\") " pod="openstack-kuttl-tests/cinder-db-create-c922p" Dec 03 11:38:23 crc kubenswrapper[5052]: I1203 11:38:23.964918 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-mjr8d\" (UniqueName: \"kubernetes.io/projected/594d60cb-8133-46b7-8cf9-960b1b12ffa3-kube-api-access-mjr8d\") pod \"keystone-db-sync-6s975\" (UID: \"594d60cb-8133-46b7-8cf9-960b1b12ffa3\") " pod="openstack-kuttl-tests/keystone-db-sync-6s975" Dec 03 11:38:23 crc kubenswrapper[5052]: I1203 11:38:23.968163 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-d4fdx\" (UniqueName: \"kubernetes.io/projected/c0ef8371-9171-45c5-9202-4d16f774ecf4-kube-api-access-d4fdx\") pod \"barbican-db-create-nrwg6\" (UID: \"c0ef8371-9171-45c5-9202-4d16f774ecf4\") " pod="openstack-kuttl-tests/barbican-db-create-nrwg6" Dec 03 11:38:24 crc kubenswrapper[5052]: I1203 11:38:24.018866 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/2ff66059-68e1-4c42-8bde-6c599a549ad5-operator-scripts\") pod \"neutron-bfb7-account-create-update-2684r\" (UID: \"2ff66059-68e1-4c42-8bde-6c599a549ad5\") " pod="openstack-kuttl-tests/neutron-bfb7-account-create-update-2684r" Dec 03 11:38:24 crc kubenswrapper[5052]: I1203 11:38:24.022039 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-4qjl9\" (UniqueName: \"kubernetes.io/projected/2ff66059-68e1-4c42-8bde-6c599a549ad5-kube-api-access-4qjl9\") pod \"neutron-bfb7-account-create-update-2684r\" (UID: \"2ff66059-68e1-4c42-8bde-6c599a549ad5\") " pod="openstack-kuttl-tests/neutron-bfb7-account-create-update-2684r" Dec 03 11:38:24 crc kubenswrapper[5052]: I1203 11:38:24.041244 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/63477217-d60a-4fd8-a58c-5b37cd2cd23f-operator-scripts\") pod \"cinder-5365-account-create-update-w5fnk\" (UID: \"63477217-d60a-4fd8-a58c-5b37cd2cd23f\") " pod="openstack-kuttl-tests/cinder-5365-account-create-update-w5fnk" Dec 03 11:38:24 crc kubenswrapper[5052]: I1203 11:38:24.041298 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-ccwsh\" (UniqueName: \"kubernetes.io/projected/63477217-d60a-4fd8-a58c-5b37cd2cd23f-kube-api-access-ccwsh\") pod \"cinder-5365-account-create-update-w5fnk\" (UID: \"63477217-d60a-4fd8-a58c-5b37cd2cd23f\") " pod="openstack-kuttl-tests/cinder-5365-account-create-update-w5fnk" Dec 03 11:38:24 crc kubenswrapper[5052]: I1203 11:38:24.042920 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/63477217-d60a-4fd8-a58c-5b37cd2cd23f-operator-scripts\") pod \"cinder-5365-account-create-update-w5fnk\" (UID: \"63477217-d60a-4fd8-a58c-5b37cd2cd23f\") " pod="openstack-kuttl-tests/cinder-5365-account-create-update-w5fnk" Dec 03 11:38:24 crc kubenswrapper[5052]: I1203 11:38:24.062676 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-ccwsh\" (UniqueName: \"kubernetes.io/projected/63477217-d60a-4fd8-a58c-5b37cd2cd23f-kube-api-access-ccwsh\") pod \"cinder-5365-account-create-update-w5fnk\" (UID: \"63477217-d60a-4fd8-a58c-5b37cd2cd23f\") " pod="openstack-kuttl-tests/cinder-5365-account-create-update-w5fnk" Dec 03 11:38:24 crc kubenswrapper[5052]: I1203 11:38:24.124781 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-db-create-nrwg6" Dec 03 11:38:24 crc kubenswrapper[5052]: I1203 11:38:24.142621 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-db-create-9kfp5" Dec 03 11:38:24 crc kubenswrapper[5052]: I1203 11:38:24.160459 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-c57b-account-create-update-nmdhk" Dec 03 11:38:24 crc kubenswrapper[5052]: I1203 11:38:24.175792 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-db-create-c922p" Dec 03 11:38:24 crc kubenswrapper[5052]: I1203 11:38:24.201552 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-bfb7-account-create-update-2684r" Dec 03 11:38:24 crc kubenswrapper[5052]: I1203 11:38:24.203081 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-db-sync-6s975" Dec 03 11:38:24 crc kubenswrapper[5052]: I1203 11:38:24.212559 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-5365-account-create-update-w5fnk" Dec 03 11:38:24 crc kubenswrapper[5052]: I1203 11:38:24.685661 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-58b8ddd7c-jtml2" event={"ID":"1e003f91-c464-4200-94c1-18c2bea8ee8a","Type":"ContainerStarted","Data":"c4284d7a16341b2a8e09ff5d310dc289d1b1e7b19a227679126b07d26626f9f9"} Dec 03 11:38:24 crc kubenswrapper[5052]: I1203 11:38:24.686098 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-58b8ddd7c-jtml2" Dec 03 11:38:24 crc kubenswrapper[5052]: I1203 11:38:24.713061 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-58b8ddd7c-jtml2" podStartSLOduration=13.713039567 podStartE2EDuration="13.713039567s" podCreationTimestamp="2025-12-03 11:38:11 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:38:24.706463257 +0000 UTC m=+1095.294705021" watchObservedRunningTime="2025-12-03 11:38:24.713039567 +0000 UTC m=+1095.301281331" Dec 03 11:38:24 crc kubenswrapper[5052]: I1203 11:38:24.728625 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-db-create-nrwg6"] Dec 03 11:38:24 crc kubenswrapper[5052]: W1203 11:38:24.729972 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc0ef8371_9171_45c5_9202_4d16f774ecf4.slice/crio-311c6a8c6bc0938e83db57b690fe737a811c070cc31b5e9d38fd96aa6d7aabca WatchSource:0}: Error finding container 311c6a8c6bc0938e83db57b690fe737a811c070cc31b5e9d38fd96aa6d7aabca: Status 404 returned error can't find the container with id 311c6a8c6bc0938e83db57b690fe737a811c070cc31b5e9d38fd96aa6d7aabca Dec 03 11:38:24 crc kubenswrapper[5052]: I1203 11:38:24.734279 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-bfb7-account-create-update-2684r"] Dec 03 11:38:24 crc kubenswrapper[5052]: I1203 11:38:24.753169 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-db-sync-6s975"] Dec 03 11:38:24 crc kubenswrapper[5052]: I1203 11:38:24.819387 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-db-create-c922p"] Dec 03 11:38:24 crc kubenswrapper[5052]: I1203 11:38:24.833107 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-c57b-account-create-update-nmdhk"] Dec 03 11:38:24 crc kubenswrapper[5052]: I1203 11:38:24.848481 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-db-create-9kfp5"] Dec 03 11:38:24 crc kubenswrapper[5052]: I1203 11:38:24.955944 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-5365-account-create-update-w5fnk"] Dec 03 11:38:25 crc kubenswrapper[5052]: I1203 11:38:25.699748 5052 generic.go:334] "Generic (PLEG): container finished" podID="c0ef8371-9171-45c5-9202-4d16f774ecf4" containerID="aab7d5710cba4d408281d37d8a730119c212b83f1f048e7073978d81874cfc2c" exitCode=0 Dec 03 11:38:25 crc kubenswrapper[5052]: I1203 11:38:25.699978 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-db-create-nrwg6" event={"ID":"c0ef8371-9171-45c5-9202-4d16f774ecf4","Type":"ContainerDied","Data":"aab7d5710cba4d408281d37d8a730119c212b83f1f048e7073978d81874cfc2c"} Dec 03 11:38:25 crc kubenswrapper[5052]: I1203 11:38:25.700215 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-db-create-nrwg6" event={"ID":"c0ef8371-9171-45c5-9202-4d16f774ecf4","Type":"ContainerStarted","Data":"311c6a8c6bc0938e83db57b690fe737a811c070cc31b5e9d38fd96aa6d7aabca"} Dec 03 11:38:25 crc kubenswrapper[5052]: I1203 11:38:25.702837 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-db-sync-6s975" event={"ID":"594d60cb-8133-46b7-8cf9-960b1b12ffa3","Type":"ContainerStarted","Data":"d305593ee6c9da811739989ca0dbb4a1d57d008db4b4b0e8e28399fc9d166895"} Dec 03 11:38:25 crc kubenswrapper[5052]: I1203 11:38:25.707258 5052 generic.go:334] "Generic (PLEG): container finished" podID="0cbd53a9-fe89-4b73-b56f-d54b8fec2230" containerID="f4ccab5332191ea20d35a5a0230ee3ca43860ab19341b4d767df3f139c66e61f" exitCode=0 Dec 03 11:38:25 crc kubenswrapper[5052]: I1203 11:38:25.707309 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-c57b-account-create-update-nmdhk" event={"ID":"0cbd53a9-fe89-4b73-b56f-d54b8fec2230","Type":"ContainerDied","Data":"f4ccab5332191ea20d35a5a0230ee3ca43860ab19341b4d767df3f139c66e61f"} Dec 03 11:38:25 crc kubenswrapper[5052]: I1203 11:38:25.707326 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-c57b-account-create-update-nmdhk" event={"ID":"0cbd53a9-fe89-4b73-b56f-d54b8fec2230","Type":"ContainerStarted","Data":"a061f9ab55f4138c8b95afcd9206ee9680064ed67bb3fd21d992349a0ae29b5c"} Dec 03 11:38:25 crc kubenswrapper[5052]: I1203 11:38:25.709775 5052 generic.go:334] "Generic (PLEG): container finished" podID="63477217-d60a-4fd8-a58c-5b37cd2cd23f" containerID="baa1660ab693e586b550adda4472e315a3f29bc7b42ec3764f341536de2ddf22" exitCode=0 Dec 03 11:38:25 crc kubenswrapper[5052]: I1203 11:38:25.709843 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-5365-account-create-update-w5fnk" event={"ID":"63477217-d60a-4fd8-a58c-5b37cd2cd23f","Type":"ContainerDied","Data":"baa1660ab693e586b550adda4472e315a3f29bc7b42ec3764f341536de2ddf22"} Dec 03 11:38:25 crc kubenswrapper[5052]: I1203 11:38:25.709872 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-5365-account-create-update-w5fnk" event={"ID":"63477217-d60a-4fd8-a58c-5b37cd2cd23f","Type":"ContainerStarted","Data":"5d967e7a85921c49a1977256fcd76958f0bd6914baf6ffb8652e7c9be1d2c5e4"} Dec 03 11:38:25 crc kubenswrapper[5052]: I1203 11:38:25.729579 5052 generic.go:334] "Generic (PLEG): container finished" podID="f4e66dc5-2979-4ac3-b5a1-959f36db75da" containerID="5f4f7fdc94d870e8565fff27865a12a867b3e689e5b0d716cbfdbcc13cfb6c9f" exitCode=0 Dec 03 11:38:25 crc kubenswrapper[5052]: I1203 11:38:25.729922 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-db-create-9kfp5" event={"ID":"f4e66dc5-2979-4ac3-b5a1-959f36db75da","Type":"ContainerDied","Data":"5f4f7fdc94d870e8565fff27865a12a867b3e689e5b0d716cbfdbcc13cfb6c9f"} Dec 03 11:38:25 crc kubenswrapper[5052]: I1203 11:38:25.729972 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-db-create-9kfp5" event={"ID":"f4e66dc5-2979-4ac3-b5a1-959f36db75da","Type":"ContainerStarted","Data":"bea4c48d90c90cc5e5b1a2a71e4e94a365661bda100d5d748a7ff09635f28522"} Dec 03 11:38:25 crc kubenswrapper[5052]: I1203 11:38:25.743830 5052 generic.go:334] "Generic (PLEG): container finished" podID="2ff66059-68e1-4c42-8bde-6c599a549ad5" containerID="d95a648c64f8ec110244fef86abf1e3ac6b622589ba5777470f9f86092e7a4de" exitCode=0 Dec 03 11:38:25 crc kubenswrapper[5052]: I1203 11:38:25.743945 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-bfb7-account-create-update-2684r" event={"ID":"2ff66059-68e1-4c42-8bde-6c599a549ad5","Type":"ContainerDied","Data":"d95a648c64f8ec110244fef86abf1e3ac6b622589ba5777470f9f86092e7a4de"} Dec 03 11:38:25 crc kubenswrapper[5052]: I1203 11:38:25.743986 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-bfb7-account-create-update-2684r" event={"ID":"2ff66059-68e1-4c42-8bde-6c599a549ad5","Type":"ContainerStarted","Data":"48a77cf80c3171d106b108460b3a1c93dc7185742952a298dbbed37ef4bd0c26"} Dec 03 11:38:25 crc kubenswrapper[5052]: I1203 11:38:25.753142 5052 generic.go:334] "Generic (PLEG): container finished" podID="ec205491-fbcd-43b5-88d3-d98ae0160f15" containerID="022f484f466350e668aa8ee358ecb5870270b3faf6ed2e427124749b11985787" exitCode=0 Dec 03 11:38:25 crc kubenswrapper[5052]: I1203 11:38:25.753429 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-db-create-c922p" event={"ID":"ec205491-fbcd-43b5-88d3-d98ae0160f15","Type":"ContainerDied","Data":"022f484f466350e668aa8ee358ecb5870270b3faf6ed2e427124749b11985787"} Dec 03 11:38:25 crc kubenswrapper[5052]: I1203 11:38:25.753582 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-db-create-c922p" event={"ID":"ec205491-fbcd-43b5-88d3-d98ae0160f15","Type":"ContainerStarted","Data":"0e11ab988447237de4653bbca083436e062557d77a236f05dadf703af5c0644b"} Dec 03 11:38:29 crc kubenswrapper[5052]: I1203 11:38:29.379147 5052 patch_prober.go:28] interesting pod/machine-config-daemon-74bj6 container/machine-config-daemon namespace/openshift-machine-config-operator: Liveness probe status=failure output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" start-of-body= Dec 03 11:38:29 crc kubenswrapper[5052]: I1203 11:38:29.379504 5052 prober.go:107] "Probe failed" probeType="Liveness" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" containerName="machine-config-daemon" probeResult="failure" output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" Dec 03 11:38:32 crc kubenswrapper[5052]: I1203 11:38:32.172669 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-58b8ddd7c-jtml2" Dec 03 11:38:32 crc kubenswrapper[5052]: I1203 11:38:32.231045 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-78bj2"] Dec 03 11:38:32 crc kubenswrapper[5052]: I1203 11:38:32.231321 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-78bj2" podUID="c5216bac-5393-4c1e-82e0-748b95282da0" containerName="dnsmasq-dns" containerID="cri-o://45c8cb70e0e388973a2dee58d830290c0d639943ca82589ae999cad7fe891a84" gracePeriod=10 Dec 03 11:38:33 crc kubenswrapper[5052]: I1203 11:38:33.395067 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-db-create-c922p" Dec 03 11:38:33 crc kubenswrapper[5052]: I1203 11:38:33.404369 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-db-create-9kfp5" Dec 03 11:38:33 crc kubenswrapper[5052]: I1203 11:38:33.421717 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-c57b-account-create-update-nmdhk" Dec 03 11:38:33 crc kubenswrapper[5052]: I1203 11:38:33.438494 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-bfb7-account-create-update-2684r" Dec 03 11:38:33 crc kubenswrapper[5052]: I1203 11:38:33.454684 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-db-create-nrwg6" Dec 03 11:38:33 crc kubenswrapper[5052]: I1203 11:38:33.474267 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-5365-account-create-update-w5fnk" Dec 03 11:38:33 crc kubenswrapper[5052]: I1203 11:38:33.502943 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4qjl9\" (UniqueName: \"kubernetes.io/projected/2ff66059-68e1-4c42-8bde-6c599a549ad5-kube-api-access-4qjl9\") pod \"2ff66059-68e1-4c42-8bde-6c599a549ad5\" (UID: \"2ff66059-68e1-4c42-8bde-6c599a549ad5\") " Dec 03 11:38:33 crc kubenswrapper[5052]: I1203 11:38:33.502988 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/2ff66059-68e1-4c42-8bde-6c599a549ad5-operator-scripts\") pod \"2ff66059-68e1-4c42-8bde-6c599a549ad5\" (UID: \"2ff66059-68e1-4c42-8bde-6c599a549ad5\") " Dec 03 11:38:33 crc kubenswrapper[5052]: I1203 11:38:33.503022 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/ec205491-fbcd-43b5-88d3-d98ae0160f15-operator-scripts\") pod \"ec205491-fbcd-43b5-88d3-d98ae0160f15\" (UID: \"ec205491-fbcd-43b5-88d3-d98ae0160f15\") " Dec 03 11:38:33 crc kubenswrapper[5052]: I1203 11:38:33.503096 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wvcng\" (UniqueName: \"kubernetes.io/projected/f4e66dc5-2979-4ac3-b5a1-959f36db75da-kube-api-access-wvcng\") pod \"f4e66dc5-2979-4ac3-b5a1-959f36db75da\" (UID: \"f4e66dc5-2979-4ac3-b5a1-959f36db75da\") " Dec 03 11:38:33 crc kubenswrapper[5052]: I1203 11:38:33.503123 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-krjq9\" (UniqueName: \"kubernetes.io/projected/0cbd53a9-fe89-4b73-b56f-d54b8fec2230-kube-api-access-krjq9\") pod \"0cbd53a9-fe89-4b73-b56f-d54b8fec2230\" (UID: \"0cbd53a9-fe89-4b73-b56f-d54b8fec2230\") " Dec 03 11:38:33 crc kubenswrapper[5052]: I1203 11:38:33.503145 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/f4e66dc5-2979-4ac3-b5a1-959f36db75da-operator-scripts\") pod \"f4e66dc5-2979-4ac3-b5a1-959f36db75da\" (UID: \"f4e66dc5-2979-4ac3-b5a1-959f36db75da\") " Dec 03 11:38:33 crc kubenswrapper[5052]: I1203 11:38:33.503185 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/c0ef8371-9171-45c5-9202-4d16f774ecf4-operator-scripts\") pod \"c0ef8371-9171-45c5-9202-4d16f774ecf4\" (UID: \"c0ef8371-9171-45c5-9202-4d16f774ecf4\") " Dec 03 11:38:33 crc kubenswrapper[5052]: I1203 11:38:33.503207 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hvq22\" (UniqueName: \"kubernetes.io/projected/ec205491-fbcd-43b5-88d3-d98ae0160f15-kube-api-access-hvq22\") pod \"ec205491-fbcd-43b5-88d3-d98ae0160f15\" (UID: \"ec205491-fbcd-43b5-88d3-d98ae0160f15\") " Dec 03 11:38:33 crc kubenswrapper[5052]: I1203 11:38:33.503244 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d4fdx\" (UniqueName: \"kubernetes.io/projected/c0ef8371-9171-45c5-9202-4d16f774ecf4-kube-api-access-d4fdx\") pod \"c0ef8371-9171-45c5-9202-4d16f774ecf4\" (UID: \"c0ef8371-9171-45c5-9202-4d16f774ecf4\") " Dec 03 11:38:33 crc kubenswrapper[5052]: I1203 11:38:33.503272 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/0cbd53a9-fe89-4b73-b56f-d54b8fec2230-operator-scripts\") pod \"0cbd53a9-fe89-4b73-b56f-d54b8fec2230\" (UID: \"0cbd53a9-fe89-4b73-b56f-d54b8fec2230\") " Dec 03 11:38:33 crc kubenswrapper[5052]: I1203 11:38:33.504012 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ec205491-fbcd-43b5-88d3-d98ae0160f15-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "ec205491-fbcd-43b5-88d3-d98ae0160f15" (UID: "ec205491-fbcd-43b5-88d3-d98ae0160f15"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:38:33 crc kubenswrapper[5052]: I1203 11:38:33.504012 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2ff66059-68e1-4c42-8bde-6c599a549ad5-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "2ff66059-68e1-4c42-8bde-6c599a549ad5" (UID: "2ff66059-68e1-4c42-8bde-6c599a549ad5"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:38:33 crc kubenswrapper[5052]: I1203 11:38:33.504055 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0cbd53a9-fe89-4b73-b56f-d54b8fec2230-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "0cbd53a9-fe89-4b73-b56f-d54b8fec2230" (UID: "0cbd53a9-fe89-4b73-b56f-d54b8fec2230"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:38:33 crc kubenswrapper[5052]: I1203 11:38:33.504557 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f4e66dc5-2979-4ac3-b5a1-959f36db75da-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "f4e66dc5-2979-4ac3-b5a1-959f36db75da" (UID: "f4e66dc5-2979-4ac3-b5a1-959f36db75da"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:38:33 crc kubenswrapper[5052]: I1203 11:38:33.510438 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f4e66dc5-2979-4ac3-b5a1-959f36db75da-kube-api-access-wvcng" (OuterVolumeSpecName: "kube-api-access-wvcng") pod "f4e66dc5-2979-4ac3-b5a1-959f36db75da" (UID: "f4e66dc5-2979-4ac3-b5a1-959f36db75da"). InnerVolumeSpecName "kube-api-access-wvcng". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:38:33 crc kubenswrapper[5052]: I1203 11:38:33.510813 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c0ef8371-9171-45c5-9202-4d16f774ecf4-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "c0ef8371-9171-45c5-9202-4d16f774ecf4" (UID: "c0ef8371-9171-45c5-9202-4d16f774ecf4"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:38:33 crc kubenswrapper[5052]: I1203 11:38:33.511268 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c0ef8371-9171-45c5-9202-4d16f774ecf4-kube-api-access-d4fdx" (OuterVolumeSpecName: "kube-api-access-d4fdx") pod "c0ef8371-9171-45c5-9202-4d16f774ecf4" (UID: "c0ef8371-9171-45c5-9202-4d16f774ecf4"). InnerVolumeSpecName "kube-api-access-d4fdx". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:38:33 crc kubenswrapper[5052]: I1203 11:38:33.511410 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ec205491-fbcd-43b5-88d3-d98ae0160f15-kube-api-access-hvq22" (OuterVolumeSpecName: "kube-api-access-hvq22") pod "ec205491-fbcd-43b5-88d3-d98ae0160f15" (UID: "ec205491-fbcd-43b5-88d3-d98ae0160f15"). InnerVolumeSpecName "kube-api-access-hvq22". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:38:33 crc kubenswrapper[5052]: I1203 11:38:33.512095 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2ff66059-68e1-4c42-8bde-6c599a549ad5-kube-api-access-4qjl9" (OuterVolumeSpecName: "kube-api-access-4qjl9") pod "2ff66059-68e1-4c42-8bde-6c599a549ad5" (UID: "2ff66059-68e1-4c42-8bde-6c599a549ad5"). InnerVolumeSpecName "kube-api-access-4qjl9". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:38:33 crc kubenswrapper[5052]: I1203 11:38:33.525628 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0cbd53a9-fe89-4b73-b56f-d54b8fec2230-kube-api-access-krjq9" (OuterVolumeSpecName: "kube-api-access-krjq9") pod "0cbd53a9-fe89-4b73-b56f-d54b8fec2230" (UID: "0cbd53a9-fe89-4b73-b56f-d54b8fec2230"). InnerVolumeSpecName "kube-api-access-krjq9". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:38:33 crc kubenswrapper[5052]: I1203 11:38:33.605103 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ccwsh\" (UniqueName: \"kubernetes.io/projected/63477217-d60a-4fd8-a58c-5b37cd2cd23f-kube-api-access-ccwsh\") pod \"63477217-d60a-4fd8-a58c-5b37cd2cd23f\" (UID: \"63477217-d60a-4fd8-a58c-5b37cd2cd23f\") " Dec 03 11:38:33 crc kubenswrapper[5052]: I1203 11:38:33.605288 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/63477217-d60a-4fd8-a58c-5b37cd2cd23f-operator-scripts\") pod \"63477217-d60a-4fd8-a58c-5b37cd2cd23f\" (UID: \"63477217-d60a-4fd8-a58c-5b37cd2cd23f\") " Dec 03 11:38:33 crc kubenswrapper[5052]: I1203 11:38:33.605617 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/c0ef8371-9171-45c5-9202-4d16f774ecf4-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:38:33 crc kubenswrapper[5052]: I1203 11:38:33.605634 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-hvq22\" (UniqueName: \"kubernetes.io/projected/ec205491-fbcd-43b5-88d3-d98ae0160f15-kube-api-access-hvq22\") on node \"crc\" DevicePath \"\"" Dec 03 11:38:33 crc kubenswrapper[5052]: I1203 11:38:33.605647 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-d4fdx\" (UniqueName: \"kubernetes.io/projected/c0ef8371-9171-45c5-9202-4d16f774ecf4-kube-api-access-d4fdx\") on node \"crc\" DevicePath \"\"" Dec 03 11:38:33 crc kubenswrapper[5052]: I1203 11:38:33.605656 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/0cbd53a9-fe89-4b73-b56f-d54b8fec2230-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:38:33 crc kubenswrapper[5052]: I1203 11:38:33.605665 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-4qjl9\" (UniqueName: \"kubernetes.io/projected/2ff66059-68e1-4c42-8bde-6c599a549ad5-kube-api-access-4qjl9\") on node \"crc\" DevicePath \"\"" Dec 03 11:38:33 crc kubenswrapper[5052]: I1203 11:38:33.605673 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/2ff66059-68e1-4c42-8bde-6c599a549ad5-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:38:33 crc kubenswrapper[5052]: I1203 11:38:33.605681 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/ec205491-fbcd-43b5-88d3-d98ae0160f15-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:38:33 crc kubenswrapper[5052]: I1203 11:38:33.605693 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-wvcng\" (UniqueName: \"kubernetes.io/projected/f4e66dc5-2979-4ac3-b5a1-959f36db75da-kube-api-access-wvcng\") on node \"crc\" DevicePath \"\"" Dec 03 11:38:33 crc kubenswrapper[5052]: I1203 11:38:33.605704 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-krjq9\" (UniqueName: \"kubernetes.io/projected/0cbd53a9-fe89-4b73-b56f-d54b8fec2230-kube-api-access-krjq9\") on node \"crc\" DevicePath \"\"" Dec 03 11:38:33 crc kubenswrapper[5052]: I1203 11:38:33.605708 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/63477217-d60a-4fd8-a58c-5b37cd2cd23f-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "63477217-d60a-4fd8-a58c-5b37cd2cd23f" (UID: "63477217-d60a-4fd8-a58c-5b37cd2cd23f"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:38:33 crc kubenswrapper[5052]: I1203 11:38:33.605714 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/f4e66dc5-2979-4ac3-b5a1-959f36db75da-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:38:33 crc kubenswrapper[5052]: I1203 11:38:33.607952 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/63477217-d60a-4fd8-a58c-5b37cd2cd23f-kube-api-access-ccwsh" (OuterVolumeSpecName: "kube-api-access-ccwsh") pod "63477217-d60a-4fd8-a58c-5b37cd2cd23f" (UID: "63477217-d60a-4fd8-a58c-5b37cd2cd23f"). InnerVolumeSpecName "kube-api-access-ccwsh". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:38:33 crc kubenswrapper[5052]: I1203 11:38:33.699213 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-78bj2" Dec 03 11:38:33 crc kubenswrapper[5052]: I1203 11:38:33.713539 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-ccwsh\" (UniqueName: \"kubernetes.io/projected/63477217-d60a-4fd8-a58c-5b37cd2cd23f-kube-api-access-ccwsh\") on node \"crc\" DevicePath \"\"" Dec 03 11:38:33 crc kubenswrapper[5052]: I1203 11:38:33.713580 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/63477217-d60a-4fd8-a58c-5b37cd2cd23f-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:38:33 crc kubenswrapper[5052]: I1203 11:38:33.814611 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/c5216bac-5393-4c1e-82e0-748b95282da0-dnsmasq-svc\") pod \"c5216bac-5393-4c1e-82e0-748b95282da0\" (UID: \"c5216bac-5393-4c1e-82e0-748b95282da0\") " Dec 03 11:38:33 crc kubenswrapper[5052]: I1203 11:38:33.814658 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z855s\" (UniqueName: \"kubernetes.io/projected/c5216bac-5393-4c1e-82e0-748b95282da0-kube-api-access-z855s\") pod \"c5216bac-5393-4c1e-82e0-748b95282da0\" (UID: \"c5216bac-5393-4c1e-82e0-748b95282da0\") " Dec 03 11:38:33 crc kubenswrapper[5052]: I1203 11:38:33.814830 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c5216bac-5393-4c1e-82e0-748b95282da0-config\") pod \"c5216bac-5393-4c1e-82e0-748b95282da0\" (UID: \"c5216bac-5393-4c1e-82e0-748b95282da0\") " Dec 03 11:38:33 crc kubenswrapper[5052]: I1203 11:38:33.818638 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c5216bac-5393-4c1e-82e0-748b95282da0-kube-api-access-z855s" (OuterVolumeSpecName: "kube-api-access-z855s") pod "c5216bac-5393-4c1e-82e0-748b95282da0" (UID: "c5216bac-5393-4c1e-82e0-748b95282da0"). InnerVolumeSpecName "kube-api-access-z855s". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:38:33 crc kubenswrapper[5052]: I1203 11:38:33.835344 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-db-create-nrwg6" Dec 03 11:38:33 crc kubenswrapper[5052]: I1203 11:38:33.835343 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-db-create-nrwg6" event={"ID":"c0ef8371-9171-45c5-9202-4d16f774ecf4","Type":"ContainerDied","Data":"311c6a8c6bc0938e83db57b690fe737a811c070cc31b5e9d38fd96aa6d7aabca"} Dec 03 11:38:33 crc kubenswrapper[5052]: I1203 11:38:33.836349 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="311c6a8c6bc0938e83db57b690fe737a811c070cc31b5e9d38fd96aa6d7aabca" Dec 03 11:38:33 crc kubenswrapper[5052]: I1203 11:38:33.837207 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-c57b-account-create-update-nmdhk" Dec 03 11:38:33 crc kubenswrapper[5052]: I1203 11:38:33.837247 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-c57b-account-create-update-nmdhk" event={"ID":"0cbd53a9-fe89-4b73-b56f-d54b8fec2230","Type":"ContainerDied","Data":"a061f9ab55f4138c8b95afcd9206ee9680064ed67bb3fd21d992349a0ae29b5c"} Dec 03 11:38:33 crc kubenswrapper[5052]: I1203 11:38:33.837363 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a061f9ab55f4138c8b95afcd9206ee9680064ed67bb3fd21d992349a0ae29b5c" Dec 03 11:38:33 crc kubenswrapper[5052]: I1203 11:38:33.839965 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-5365-account-create-update-w5fnk" event={"ID":"63477217-d60a-4fd8-a58c-5b37cd2cd23f","Type":"ContainerDied","Data":"5d967e7a85921c49a1977256fcd76958f0bd6914baf6ffb8652e7c9be1d2c5e4"} Dec 03 11:38:33 crc kubenswrapper[5052]: I1203 11:38:33.840082 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5d967e7a85921c49a1977256fcd76958f0bd6914baf6ffb8652e7c9be1d2c5e4" Dec 03 11:38:33 crc kubenswrapper[5052]: I1203 11:38:33.841165 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-5365-account-create-update-w5fnk" Dec 03 11:38:33 crc kubenswrapper[5052]: I1203 11:38:33.843121 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-bfb7-account-create-update-2684r" event={"ID":"2ff66059-68e1-4c42-8bde-6c599a549ad5","Type":"ContainerDied","Data":"48a77cf80c3171d106b108460b3a1c93dc7185742952a298dbbed37ef4bd0c26"} Dec 03 11:38:33 crc kubenswrapper[5052]: I1203 11:38:33.843253 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="48a77cf80c3171d106b108460b3a1c93dc7185742952a298dbbed37ef4bd0c26" Dec 03 11:38:33 crc kubenswrapper[5052]: I1203 11:38:33.843349 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-bfb7-account-create-update-2684r" Dec 03 11:38:33 crc kubenswrapper[5052]: I1203 11:38:33.844618 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-db-create-9kfp5" event={"ID":"f4e66dc5-2979-4ac3-b5a1-959f36db75da","Type":"ContainerDied","Data":"bea4c48d90c90cc5e5b1a2a71e4e94a365661bda100d5d748a7ff09635f28522"} Dec 03 11:38:33 crc kubenswrapper[5052]: I1203 11:38:33.844643 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bea4c48d90c90cc5e5b1a2a71e4e94a365661bda100d5d748a7ff09635f28522" Dec 03 11:38:33 crc kubenswrapper[5052]: I1203 11:38:33.844693 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-db-create-9kfp5" Dec 03 11:38:33 crc kubenswrapper[5052]: I1203 11:38:33.850291 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-db-create-c922p" Dec 03 11:38:33 crc kubenswrapper[5052]: I1203 11:38:33.850286 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-db-create-c922p" event={"ID":"ec205491-fbcd-43b5-88d3-d98ae0160f15","Type":"ContainerDied","Data":"0e11ab988447237de4653bbca083436e062557d77a236f05dadf703af5c0644b"} Dec 03 11:38:33 crc kubenswrapper[5052]: I1203 11:38:33.850568 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0e11ab988447237de4653bbca083436e062557d77a236f05dadf703af5c0644b" Dec 03 11:38:33 crc kubenswrapper[5052]: I1203 11:38:33.854387 5052 generic.go:334] "Generic (PLEG): container finished" podID="c5216bac-5393-4c1e-82e0-748b95282da0" containerID="45c8cb70e0e388973a2dee58d830290c0d639943ca82589ae999cad7fe891a84" exitCode=0 Dec 03 11:38:33 crc kubenswrapper[5052]: I1203 11:38:33.854431 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-78bj2" event={"ID":"c5216bac-5393-4c1e-82e0-748b95282da0","Type":"ContainerDied","Data":"45c8cb70e0e388973a2dee58d830290c0d639943ca82589ae999cad7fe891a84"} Dec 03 11:38:33 crc kubenswrapper[5052]: I1203 11:38:33.854458 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-78bj2" event={"ID":"c5216bac-5393-4c1e-82e0-748b95282da0","Type":"ContainerDied","Data":"6e388fdd91614eb21038110ec0e7db48fd4e8c8dc0b9df0f589c57da9cc9add7"} Dec 03 11:38:33 crc kubenswrapper[5052]: I1203 11:38:33.854479 5052 scope.go:117] "RemoveContainer" containerID="45c8cb70e0e388973a2dee58d830290c0d639943ca82589ae999cad7fe891a84" Dec 03 11:38:33 crc kubenswrapper[5052]: I1203 11:38:33.854495 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-78bj2" Dec 03 11:38:33 crc kubenswrapper[5052]: I1203 11:38:33.859819 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c5216bac-5393-4c1e-82e0-748b95282da0-config" (OuterVolumeSpecName: "config") pod "c5216bac-5393-4c1e-82e0-748b95282da0" (UID: "c5216bac-5393-4c1e-82e0-748b95282da0"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:38:33 crc kubenswrapper[5052]: I1203 11:38:33.868366 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c5216bac-5393-4c1e-82e0-748b95282da0-dnsmasq-svc" (OuterVolumeSpecName: "dnsmasq-svc") pod "c5216bac-5393-4c1e-82e0-748b95282da0" (UID: "c5216bac-5393-4c1e-82e0-748b95282da0"). InnerVolumeSpecName "dnsmasq-svc". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:38:33 crc kubenswrapper[5052]: I1203 11:38:33.886637 5052 scope.go:117] "RemoveContainer" containerID="97ac574ed6407f65761e184a22ba07e4cf13f3422f77c42f0080ecc0878e410e" Dec 03 11:38:33 crc kubenswrapper[5052]: I1203 11:38:33.905812 5052 scope.go:117] "RemoveContainer" containerID="45c8cb70e0e388973a2dee58d830290c0d639943ca82589ae999cad7fe891a84" Dec 03 11:38:33 crc kubenswrapper[5052]: E1203 11:38:33.906260 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"45c8cb70e0e388973a2dee58d830290c0d639943ca82589ae999cad7fe891a84\": container with ID starting with 45c8cb70e0e388973a2dee58d830290c0d639943ca82589ae999cad7fe891a84 not found: ID does not exist" containerID="45c8cb70e0e388973a2dee58d830290c0d639943ca82589ae999cad7fe891a84" Dec 03 11:38:33 crc kubenswrapper[5052]: I1203 11:38:33.906301 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"45c8cb70e0e388973a2dee58d830290c0d639943ca82589ae999cad7fe891a84"} err="failed to get container status \"45c8cb70e0e388973a2dee58d830290c0d639943ca82589ae999cad7fe891a84\": rpc error: code = NotFound desc = could not find container \"45c8cb70e0e388973a2dee58d830290c0d639943ca82589ae999cad7fe891a84\": container with ID starting with 45c8cb70e0e388973a2dee58d830290c0d639943ca82589ae999cad7fe891a84 not found: ID does not exist" Dec 03 11:38:33 crc kubenswrapper[5052]: I1203 11:38:33.906326 5052 scope.go:117] "RemoveContainer" containerID="97ac574ed6407f65761e184a22ba07e4cf13f3422f77c42f0080ecc0878e410e" Dec 03 11:38:33 crc kubenswrapper[5052]: E1203 11:38:33.906640 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"97ac574ed6407f65761e184a22ba07e4cf13f3422f77c42f0080ecc0878e410e\": container with ID starting with 97ac574ed6407f65761e184a22ba07e4cf13f3422f77c42f0080ecc0878e410e not found: ID does not exist" containerID="97ac574ed6407f65761e184a22ba07e4cf13f3422f77c42f0080ecc0878e410e" Dec 03 11:38:33 crc kubenswrapper[5052]: I1203 11:38:33.906674 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"97ac574ed6407f65761e184a22ba07e4cf13f3422f77c42f0080ecc0878e410e"} err="failed to get container status \"97ac574ed6407f65761e184a22ba07e4cf13f3422f77c42f0080ecc0878e410e\": rpc error: code = NotFound desc = could not find container \"97ac574ed6407f65761e184a22ba07e4cf13f3422f77c42f0080ecc0878e410e\": container with ID starting with 97ac574ed6407f65761e184a22ba07e4cf13f3422f77c42f0080ecc0878e410e not found: ID does not exist" Dec 03 11:38:33 crc kubenswrapper[5052]: I1203 11:38:33.916936 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c5216bac-5393-4c1e-82e0-748b95282da0-config\") on node \"crc\" DevicePath \"\"" Dec 03 11:38:33 crc kubenswrapper[5052]: I1203 11:38:33.916968 5052 reconciler_common.go:293] "Volume detached for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/c5216bac-5393-4c1e-82e0-748b95282da0-dnsmasq-svc\") on node \"crc\" DevicePath \"\"" Dec 03 11:38:33 crc kubenswrapper[5052]: I1203 11:38:33.916982 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-z855s\" (UniqueName: \"kubernetes.io/projected/c5216bac-5393-4c1e-82e0-748b95282da0-kube-api-access-z855s\") on node \"crc\" DevicePath \"\"" Dec 03 11:38:34 crc kubenswrapper[5052]: I1203 11:38:34.189874 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-78bj2"] Dec 03 11:38:34 crc kubenswrapper[5052]: I1203 11:38:34.198502 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-78bj2"] Dec 03 11:38:34 crc kubenswrapper[5052]: I1203 11:38:34.865020 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-db-sync-6s975" event={"ID":"594d60cb-8133-46b7-8cf9-960b1b12ffa3","Type":"ContainerStarted","Data":"4dd837c600b78e6aad667f7eb01ad1f8a043677942a003c3fb1219733ddb1f0e"} Dec 03 11:38:34 crc kubenswrapper[5052]: I1203 11:38:34.867267 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-db-sync-k6vlv" event={"ID":"063c5d2e-68a5-497a-b7bd-f95d45804848","Type":"ContainerStarted","Data":"831ae3f9746c506b87d5990844ed13f7bb77d7fc3b6415de96a98290e3869a4c"} Dec 03 11:38:34 crc kubenswrapper[5052]: I1203 11:38:34.893122 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/keystone-db-sync-6s975" podStartSLOduration=2.80545869 podStartE2EDuration="11.89310381s" podCreationTimestamp="2025-12-03 11:38:23 +0000 UTC" firstStartedPulling="2025-12-03 11:38:24.800053096 +0000 UTC m=+1095.388294860" lastFinishedPulling="2025-12-03 11:38:33.887698216 +0000 UTC m=+1104.475939980" observedRunningTime="2025-12-03 11:38:34.886469458 +0000 UTC m=+1105.474711232" watchObservedRunningTime="2025-12-03 11:38:34.89310381 +0000 UTC m=+1105.481345584" Dec 03 11:38:34 crc kubenswrapper[5052]: I1203 11:38:34.907554 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/glance-db-sync-k6vlv" podStartSLOduration=2.722689849 podStartE2EDuration="29.907502495s" podCreationTimestamp="2025-12-03 11:38:05 +0000 UTC" firstStartedPulling="2025-12-03 11:38:06.703211229 +0000 UTC m=+1077.291452993" lastFinishedPulling="2025-12-03 11:38:33.888023865 +0000 UTC m=+1104.476265639" observedRunningTime="2025-12-03 11:38:34.905860968 +0000 UTC m=+1105.494102732" watchObservedRunningTime="2025-12-03 11:38:34.907502495 +0000 UTC m=+1105.495744299" Dec 03 11:38:35 crc kubenswrapper[5052]: I1203 11:38:35.693610 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c5216bac-5393-4c1e-82e0-748b95282da0" path="/var/lib/kubelet/pods/c5216bac-5393-4c1e-82e0-748b95282da0/volumes" Dec 03 11:38:39 crc kubenswrapper[5052]: I1203 11:38:39.910661 5052 generic.go:334] "Generic (PLEG): container finished" podID="594d60cb-8133-46b7-8cf9-960b1b12ffa3" containerID="4dd837c600b78e6aad667f7eb01ad1f8a043677942a003c3fb1219733ddb1f0e" exitCode=0 Dec 03 11:38:39 crc kubenswrapper[5052]: I1203 11:38:39.910701 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-db-sync-6s975" event={"ID":"594d60cb-8133-46b7-8cf9-960b1b12ffa3","Type":"ContainerDied","Data":"4dd837c600b78e6aad667f7eb01ad1f8a043677942a003c3fb1219733ddb1f0e"} Dec 03 11:38:41 crc kubenswrapper[5052]: I1203 11:38:41.286284 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-db-sync-6s975" Dec 03 11:38:41 crc kubenswrapper[5052]: I1203 11:38:41.346201 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/594d60cb-8133-46b7-8cf9-960b1b12ffa3-config-data\") pod \"594d60cb-8133-46b7-8cf9-960b1b12ffa3\" (UID: \"594d60cb-8133-46b7-8cf9-960b1b12ffa3\") " Dec 03 11:38:41 crc kubenswrapper[5052]: I1203 11:38:41.346245 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/594d60cb-8133-46b7-8cf9-960b1b12ffa3-combined-ca-bundle\") pod \"594d60cb-8133-46b7-8cf9-960b1b12ffa3\" (UID: \"594d60cb-8133-46b7-8cf9-960b1b12ffa3\") " Dec 03 11:38:41 crc kubenswrapper[5052]: I1203 11:38:41.346329 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mjr8d\" (UniqueName: \"kubernetes.io/projected/594d60cb-8133-46b7-8cf9-960b1b12ffa3-kube-api-access-mjr8d\") pod \"594d60cb-8133-46b7-8cf9-960b1b12ffa3\" (UID: \"594d60cb-8133-46b7-8cf9-960b1b12ffa3\") " Dec 03 11:38:41 crc kubenswrapper[5052]: I1203 11:38:41.352090 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/594d60cb-8133-46b7-8cf9-960b1b12ffa3-kube-api-access-mjr8d" (OuterVolumeSpecName: "kube-api-access-mjr8d") pod "594d60cb-8133-46b7-8cf9-960b1b12ffa3" (UID: "594d60cb-8133-46b7-8cf9-960b1b12ffa3"). InnerVolumeSpecName "kube-api-access-mjr8d". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:38:41 crc kubenswrapper[5052]: I1203 11:38:41.373095 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/594d60cb-8133-46b7-8cf9-960b1b12ffa3-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "594d60cb-8133-46b7-8cf9-960b1b12ffa3" (UID: "594d60cb-8133-46b7-8cf9-960b1b12ffa3"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:38:41 crc kubenswrapper[5052]: I1203 11:38:41.390035 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/594d60cb-8133-46b7-8cf9-960b1b12ffa3-config-data" (OuterVolumeSpecName: "config-data") pod "594d60cb-8133-46b7-8cf9-960b1b12ffa3" (UID: "594d60cb-8133-46b7-8cf9-960b1b12ffa3"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:38:41 crc kubenswrapper[5052]: I1203 11:38:41.448002 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/594d60cb-8133-46b7-8cf9-960b1b12ffa3-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:38:41 crc kubenswrapper[5052]: I1203 11:38:41.448051 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/594d60cb-8133-46b7-8cf9-960b1b12ffa3-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:38:41 crc kubenswrapper[5052]: I1203 11:38:41.448066 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-mjr8d\" (UniqueName: \"kubernetes.io/projected/594d60cb-8133-46b7-8cf9-960b1b12ffa3-kube-api-access-mjr8d\") on node \"crc\" DevicePath \"\"" Dec 03 11:38:41 crc kubenswrapper[5052]: I1203 11:38:41.930985 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-db-sync-6s975" event={"ID":"594d60cb-8133-46b7-8cf9-960b1b12ffa3","Type":"ContainerDied","Data":"d305593ee6c9da811739989ca0dbb4a1d57d008db4b4b0e8e28399fc9d166895"} Dec 03 11:38:41 crc kubenswrapper[5052]: I1203 11:38:41.931034 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-db-sync-6s975" Dec 03 11:38:41 crc kubenswrapper[5052]: I1203 11:38:41.931043 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d305593ee6c9da811739989ca0dbb4a1d57d008db4b4b0e8e28399fc9d166895" Dec 03 11:38:42 crc kubenswrapper[5052]: I1203 11:38:42.106286 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/keystone-bootstrap-fstdw"] Dec 03 11:38:42 crc kubenswrapper[5052]: E1203 11:38:42.107140 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="2ff66059-68e1-4c42-8bde-6c599a549ad5" containerName="mariadb-account-create-update" Dec 03 11:38:42 crc kubenswrapper[5052]: I1203 11:38:42.107245 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="2ff66059-68e1-4c42-8bde-6c599a549ad5" containerName="mariadb-account-create-update" Dec 03 11:38:42 crc kubenswrapper[5052]: E1203 11:38:42.107315 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="c5216bac-5393-4c1e-82e0-748b95282da0" containerName="dnsmasq-dns" Dec 03 11:38:42 crc kubenswrapper[5052]: I1203 11:38:42.107367 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="c5216bac-5393-4c1e-82e0-748b95282da0" containerName="dnsmasq-dns" Dec 03 11:38:42 crc kubenswrapper[5052]: E1203 11:38:42.107429 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="f4e66dc5-2979-4ac3-b5a1-959f36db75da" containerName="mariadb-database-create" Dec 03 11:38:42 crc kubenswrapper[5052]: I1203 11:38:42.107526 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="f4e66dc5-2979-4ac3-b5a1-959f36db75da" containerName="mariadb-database-create" Dec 03 11:38:42 crc kubenswrapper[5052]: E1203 11:38:42.107617 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="0cbd53a9-fe89-4b73-b56f-d54b8fec2230" containerName="mariadb-account-create-update" Dec 03 11:38:42 crc kubenswrapper[5052]: I1203 11:38:42.107690 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="0cbd53a9-fe89-4b73-b56f-d54b8fec2230" containerName="mariadb-account-create-update" Dec 03 11:38:42 crc kubenswrapper[5052]: E1203 11:38:42.107770 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="594d60cb-8133-46b7-8cf9-960b1b12ffa3" containerName="keystone-db-sync" Dec 03 11:38:42 crc kubenswrapper[5052]: I1203 11:38:42.107846 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="594d60cb-8133-46b7-8cf9-960b1b12ffa3" containerName="keystone-db-sync" Dec 03 11:38:42 crc kubenswrapper[5052]: E1203 11:38:42.107931 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="c5216bac-5393-4c1e-82e0-748b95282da0" containerName="init" Dec 03 11:38:42 crc kubenswrapper[5052]: I1203 11:38:42.108011 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="c5216bac-5393-4c1e-82e0-748b95282da0" containerName="init" Dec 03 11:38:42 crc kubenswrapper[5052]: E1203 11:38:42.108117 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="ec205491-fbcd-43b5-88d3-d98ae0160f15" containerName="mariadb-database-create" Dec 03 11:38:42 crc kubenswrapper[5052]: I1203 11:38:42.108186 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="ec205491-fbcd-43b5-88d3-d98ae0160f15" containerName="mariadb-database-create" Dec 03 11:38:42 crc kubenswrapper[5052]: E1203 11:38:42.108254 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="63477217-d60a-4fd8-a58c-5b37cd2cd23f" containerName="mariadb-account-create-update" Dec 03 11:38:42 crc kubenswrapper[5052]: I1203 11:38:42.108333 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="63477217-d60a-4fd8-a58c-5b37cd2cd23f" containerName="mariadb-account-create-update" Dec 03 11:38:42 crc kubenswrapper[5052]: E1203 11:38:42.108417 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="c0ef8371-9171-45c5-9202-4d16f774ecf4" containerName="mariadb-database-create" Dec 03 11:38:42 crc kubenswrapper[5052]: I1203 11:38:42.108492 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="c0ef8371-9171-45c5-9202-4d16f774ecf4" containerName="mariadb-database-create" Dec 03 11:38:42 crc kubenswrapper[5052]: I1203 11:38:42.108761 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="63477217-d60a-4fd8-a58c-5b37cd2cd23f" containerName="mariadb-account-create-update" Dec 03 11:38:42 crc kubenswrapper[5052]: I1203 11:38:42.108867 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="c5216bac-5393-4c1e-82e0-748b95282da0" containerName="dnsmasq-dns" Dec 03 11:38:42 crc kubenswrapper[5052]: I1203 11:38:42.108956 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="f4e66dc5-2979-4ac3-b5a1-959f36db75da" containerName="mariadb-database-create" Dec 03 11:38:42 crc kubenswrapper[5052]: I1203 11:38:42.109038 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="594d60cb-8133-46b7-8cf9-960b1b12ffa3" containerName="keystone-db-sync" Dec 03 11:38:42 crc kubenswrapper[5052]: I1203 11:38:42.109129 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="0cbd53a9-fe89-4b73-b56f-d54b8fec2230" containerName="mariadb-account-create-update" Dec 03 11:38:42 crc kubenswrapper[5052]: I1203 11:38:42.109199 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="2ff66059-68e1-4c42-8bde-6c599a549ad5" containerName="mariadb-account-create-update" Dec 03 11:38:42 crc kubenswrapper[5052]: I1203 11:38:42.109260 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="ec205491-fbcd-43b5-88d3-d98ae0160f15" containerName="mariadb-database-create" Dec 03 11:38:42 crc kubenswrapper[5052]: I1203 11:38:42.109329 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="c0ef8371-9171-45c5-9202-4d16f774ecf4" containerName="mariadb-database-create" Dec 03 11:38:42 crc kubenswrapper[5052]: I1203 11:38:42.109911 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-bootstrap-fstdw" Dec 03 11:38:42 crc kubenswrapper[5052]: I1203 11:38:42.112241 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone-keystone-dockercfg-tt6lh" Dec 03 11:38:42 crc kubenswrapper[5052]: I1203 11:38:42.113179 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone" Dec 03 11:38:42 crc kubenswrapper[5052]: I1203 11:38:42.113179 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone-scripts" Dec 03 11:38:42 crc kubenswrapper[5052]: I1203 11:38:42.114577 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone-config-data" Dec 03 11:38:42 crc kubenswrapper[5052]: I1203 11:38:42.115692 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"osp-secret" Dec 03 11:38:42 crc kubenswrapper[5052]: I1203 11:38:42.124839 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-bootstrap-fstdw"] Dec 03 11:38:42 crc kubenswrapper[5052]: I1203 11:38:42.159625 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e1e49fa1-2ef1-4f43-a0b2-f98c1364bdcb-combined-ca-bundle\") pod \"keystone-bootstrap-fstdw\" (UID: \"e1e49fa1-2ef1-4f43-a0b2-f98c1364bdcb\") " pod="openstack-kuttl-tests/keystone-bootstrap-fstdw" Dec 03 11:38:42 crc kubenswrapper[5052]: I1203 11:38:42.159702 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/e1e49fa1-2ef1-4f43-a0b2-f98c1364bdcb-credential-keys\") pod \"keystone-bootstrap-fstdw\" (UID: \"e1e49fa1-2ef1-4f43-a0b2-f98c1364bdcb\") " pod="openstack-kuttl-tests/keystone-bootstrap-fstdw" Dec 03 11:38:42 crc kubenswrapper[5052]: I1203 11:38:42.159744 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/e1e49fa1-2ef1-4f43-a0b2-f98c1364bdcb-fernet-keys\") pod \"keystone-bootstrap-fstdw\" (UID: \"e1e49fa1-2ef1-4f43-a0b2-f98c1364bdcb\") " pod="openstack-kuttl-tests/keystone-bootstrap-fstdw" Dec 03 11:38:42 crc kubenswrapper[5052]: I1203 11:38:42.159808 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/e1e49fa1-2ef1-4f43-a0b2-f98c1364bdcb-config-data\") pod \"keystone-bootstrap-fstdw\" (UID: \"e1e49fa1-2ef1-4f43-a0b2-f98c1364bdcb\") " pod="openstack-kuttl-tests/keystone-bootstrap-fstdw" Dec 03 11:38:42 crc kubenswrapper[5052]: I1203 11:38:42.159864 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/e1e49fa1-2ef1-4f43-a0b2-f98c1364bdcb-scripts\") pod \"keystone-bootstrap-fstdw\" (UID: \"e1e49fa1-2ef1-4f43-a0b2-f98c1364bdcb\") " pod="openstack-kuttl-tests/keystone-bootstrap-fstdw" Dec 03 11:38:42 crc kubenswrapper[5052]: I1203 11:38:42.159905 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sg8wv\" (UniqueName: \"kubernetes.io/projected/e1e49fa1-2ef1-4f43-a0b2-f98c1364bdcb-kube-api-access-sg8wv\") pod \"keystone-bootstrap-fstdw\" (UID: \"e1e49fa1-2ef1-4f43-a0b2-f98c1364bdcb\") " pod="openstack-kuttl-tests/keystone-bootstrap-fstdw" Dec 03 11:38:42 crc kubenswrapper[5052]: I1203 11:38:42.261232 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e1e49fa1-2ef1-4f43-a0b2-f98c1364bdcb-combined-ca-bundle\") pod \"keystone-bootstrap-fstdw\" (UID: \"e1e49fa1-2ef1-4f43-a0b2-f98c1364bdcb\") " pod="openstack-kuttl-tests/keystone-bootstrap-fstdw" Dec 03 11:38:42 crc kubenswrapper[5052]: I1203 11:38:42.261487 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/e1e49fa1-2ef1-4f43-a0b2-f98c1364bdcb-credential-keys\") pod \"keystone-bootstrap-fstdw\" (UID: \"e1e49fa1-2ef1-4f43-a0b2-f98c1364bdcb\") " pod="openstack-kuttl-tests/keystone-bootstrap-fstdw" Dec 03 11:38:42 crc kubenswrapper[5052]: I1203 11:38:42.261671 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/e1e49fa1-2ef1-4f43-a0b2-f98c1364bdcb-fernet-keys\") pod \"keystone-bootstrap-fstdw\" (UID: \"e1e49fa1-2ef1-4f43-a0b2-f98c1364bdcb\") " pod="openstack-kuttl-tests/keystone-bootstrap-fstdw" Dec 03 11:38:42 crc kubenswrapper[5052]: I1203 11:38:42.261794 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/e1e49fa1-2ef1-4f43-a0b2-f98c1364bdcb-config-data\") pod \"keystone-bootstrap-fstdw\" (UID: \"e1e49fa1-2ef1-4f43-a0b2-f98c1364bdcb\") " pod="openstack-kuttl-tests/keystone-bootstrap-fstdw" Dec 03 11:38:42 crc kubenswrapper[5052]: I1203 11:38:42.261926 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/e1e49fa1-2ef1-4f43-a0b2-f98c1364bdcb-scripts\") pod \"keystone-bootstrap-fstdw\" (UID: \"e1e49fa1-2ef1-4f43-a0b2-f98c1364bdcb\") " pod="openstack-kuttl-tests/keystone-bootstrap-fstdw" Dec 03 11:38:42 crc kubenswrapper[5052]: I1203 11:38:42.262055 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-sg8wv\" (UniqueName: \"kubernetes.io/projected/e1e49fa1-2ef1-4f43-a0b2-f98c1364bdcb-kube-api-access-sg8wv\") pod \"keystone-bootstrap-fstdw\" (UID: \"e1e49fa1-2ef1-4f43-a0b2-f98c1364bdcb\") " pod="openstack-kuttl-tests/keystone-bootstrap-fstdw" Dec 03 11:38:42 crc kubenswrapper[5052]: I1203 11:38:42.267401 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e1e49fa1-2ef1-4f43-a0b2-f98c1364bdcb-combined-ca-bundle\") pod \"keystone-bootstrap-fstdw\" (UID: \"e1e49fa1-2ef1-4f43-a0b2-f98c1364bdcb\") " pod="openstack-kuttl-tests/keystone-bootstrap-fstdw" Dec 03 11:38:42 crc kubenswrapper[5052]: I1203 11:38:42.267403 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/e1e49fa1-2ef1-4f43-a0b2-f98c1364bdcb-credential-keys\") pod \"keystone-bootstrap-fstdw\" (UID: \"e1e49fa1-2ef1-4f43-a0b2-f98c1364bdcb\") " pod="openstack-kuttl-tests/keystone-bootstrap-fstdw" Dec 03 11:38:42 crc kubenswrapper[5052]: I1203 11:38:42.267769 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/e1e49fa1-2ef1-4f43-a0b2-f98c1364bdcb-fernet-keys\") pod \"keystone-bootstrap-fstdw\" (UID: \"e1e49fa1-2ef1-4f43-a0b2-f98c1364bdcb\") " pod="openstack-kuttl-tests/keystone-bootstrap-fstdw" Dec 03 11:38:42 crc kubenswrapper[5052]: I1203 11:38:42.267921 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/e1e49fa1-2ef1-4f43-a0b2-f98c1364bdcb-scripts\") pod \"keystone-bootstrap-fstdw\" (UID: \"e1e49fa1-2ef1-4f43-a0b2-f98c1364bdcb\") " pod="openstack-kuttl-tests/keystone-bootstrap-fstdw" Dec 03 11:38:42 crc kubenswrapper[5052]: I1203 11:38:42.272170 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/e1e49fa1-2ef1-4f43-a0b2-f98c1364bdcb-config-data\") pod \"keystone-bootstrap-fstdw\" (UID: \"e1e49fa1-2ef1-4f43-a0b2-f98c1364bdcb\") " pod="openstack-kuttl-tests/keystone-bootstrap-fstdw" Dec 03 11:38:42 crc kubenswrapper[5052]: I1203 11:38:42.329637 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-sg8wv\" (UniqueName: \"kubernetes.io/projected/e1e49fa1-2ef1-4f43-a0b2-f98c1364bdcb-kube-api-access-sg8wv\") pod \"keystone-bootstrap-fstdw\" (UID: \"e1e49fa1-2ef1-4f43-a0b2-f98c1364bdcb\") " pod="openstack-kuttl-tests/keystone-bootstrap-fstdw" Dec 03 11:38:42 crc kubenswrapper[5052]: I1203 11:38:42.344575 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:38:42 crc kubenswrapper[5052]: I1203 11:38:42.346424 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:38:42 crc kubenswrapper[5052]: I1203 11:38:42.350183 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-config-data" Dec 03 11:38:42 crc kubenswrapper[5052]: I1203 11:38:42.350345 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-scripts" Dec 03 11:38:42 crc kubenswrapper[5052]: I1203 11:38:42.369584 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/cinder-db-sync-pbw4k"] Dec 03 11:38:42 crc kubenswrapper[5052]: I1203 11:38:42.370733 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-db-sync-pbw4k" Dec 03 11:38:42 crc kubenswrapper[5052]: I1203 11:38:42.379479 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cinder-scripts" Dec 03 11:38:42 crc kubenswrapper[5052]: I1203 11:38:42.379877 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cinder-config-data" Dec 03 11:38:42 crc kubenswrapper[5052]: I1203 11:38:42.380007 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cinder-cinder-dockercfg-m58ql" Dec 03 11:38:42 crc kubenswrapper[5052]: I1203 11:38:42.380111 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-db-sync-pbw4k"] Dec 03 11:38:42 crc kubenswrapper[5052]: I1203 11:38:42.407727 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/placement-db-sync-qp7zq"] Dec 03 11:38:42 crc kubenswrapper[5052]: I1203 11:38:42.408890 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-db-sync-qp7zq" Dec 03 11:38:42 crc kubenswrapper[5052]: I1203 11:38:42.416977 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"placement-config-data" Dec 03 11:38:42 crc kubenswrapper[5052]: I1203 11:38:42.417196 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"placement-placement-dockercfg-sgp5f" Dec 03 11:38:42 crc kubenswrapper[5052]: I1203 11:38:42.417308 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"placement-scripts" Dec 03 11:38:42 crc kubenswrapper[5052]: I1203 11:38:42.418534 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:38:42 crc kubenswrapper[5052]: I1203 11:38:42.435122 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/placement-db-sync-qp7zq"] Dec 03 11:38:42 crc kubenswrapper[5052]: I1203 11:38:42.435922 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-bootstrap-fstdw" Dec 03 11:38:42 crc kubenswrapper[5052]: I1203 11:38:42.440584 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/neutron-db-sync-j7t4l"] Dec 03 11:38:42 crc kubenswrapper[5052]: I1203 11:38:42.441729 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-db-sync-j7t4l" Dec 03 11:38:42 crc kubenswrapper[5052]: I1203 11:38:42.445623 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"neutron-httpd-config" Dec 03 11:38:42 crc kubenswrapper[5052]: I1203 11:38:42.445821 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"neutron-neutron-dockercfg-6xp5c" Dec 03 11:38:42 crc kubenswrapper[5052]: I1203 11:38:42.446032 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"neutron-config" Dec 03 11:38:42 crc kubenswrapper[5052]: I1203 11:38:42.466643 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d1676789-356e-4853-a302-f2c5bd3f446f-combined-ca-bundle\") pod \"cinder-db-sync-pbw4k\" (UID: \"d1676789-356e-4853-a302-f2c5bd3f446f\") " pod="openstack-kuttl-tests/cinder-db-sync-pbw4k" Dec 03 11:38:42 crc kubenswrapper[5052]: I1203 11:38:42.466731 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5ff261a9-6d56-45fe-88fa-7184d8a3b90b-combined-ca-bundle\") pod \"placement-db-sync-qp7zq\" (UID: \"5ff261a9-6d56-45fe-88fa-7184d8a3b90b\") " pod="openstack-kuttl-tests/placement-db-sync-qp7zq" Dec 03 11:38:42 crc kubenswrapper[5052]: I1203 11:38:42.466777 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8lqcr\" (UniqueName: \"kubernetes.io/projected/d1676789-356e-4853-a302-f2c5bd3f446f-kube-api-access-8lqcr\") pod \"cinder-db-sync-pbw4k\" (UID: \"d1676789-356e-4853-a302-f2c5bd3f446f\") " pod="openstack-kuttl-tests/cinder-db-sync-pbw4k" Dec 03 11:38:42 crc kubenswrapper[5052]: I1203 11:38:42.466822 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c43f59f2-deb7-46ee-8e07-54e3ff9f59aa-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"c43f59f2-deb7-46ee-8e07-54e3ff9f59aa\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:38:42 crc kubenswrapper[5052]: I1203 11:38:42.466845 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/c43f59f2-deb7-46ee-8e07-54e3ff9f59aa-log-httpd\") pod \"ceilometer-0\" (UID: \"c43f59f2-deb7-46ee-8e07-54e3ff9f59aa\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:38:42 crc kubenswrapper[5052]: I1203 11:38:42.466885 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/c43f59f2-deb7-46ee-8e07-54e3ff9f59aa-scripts\") pod \"ceilometer-0\" (UID: \"c43f59f2-deb7-46ee-8e07-54e3ff9f59aa\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:38:42 crc kubenswrapper[5052]: I1203 11:38:42.466934 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/d1676789-356e-4853-a302-f2c5bd3f446f-config-data\") pod \"cinder-db-sync-pbw4k\" (UID: \"d1676789-356e-4853-a302-f2c5bd3f446f\") " pod="openstack-kuttl-tests/cinder-db-sync-pbw4k" Dec 03 11:38:42 crc kubenswrapper[5052]: I1203 11:38:42.466958 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/c43f59f2-deb7-46ee-8e07-54e3ff9f59aa-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"c43f59f2-deb7-46ee-8e07-54e3ff9f59aa\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:38:42 crc kubenswrapper[5052]: I1203 11:38:42.466980 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/5ff261a9-6d56-45fe-88fa-7184d8a3b90b-scripts\") pod \"placement-db-sync-qp7zq\" (UID: \"5ff261a9-6d56-45fe-88fa-7184d8a3b90b\") " pod="openstack-kuttl-tests/placement-db-sync-qp7zq" Dec 03 11:38:42 crc kubenswrapper[5052]: I1203 11:38:42.467004 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/d1676789-356e-4853-a302-f2c5bd3f446f-db-sync-config-data\") pod \"cinder-db-sync-pbw4k\" (UID: \"d1676789-356e-4853-a302-f2c5bd3f446f\") " pod="openstack-kuttl-tests/cinder-db-sync-pbw4k" Dec 03 11:38:42 crc kubenswrapper[5052]: I1203 11:38:42.467033 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/5ff261a9-6d56-45fe-88fa-7184d8a3b90b-config-data\") pod \"placement-db-sync-qp7zq\" (UID: \"5ff261a9-6d56-45fe-88fa-7184d8a3b90b\") " pod="openstack-kuttl-tests/placement-db-sync-qp7zq" Dec 03 11:38:42 crc kubenswrapper[5052]: I1203 11:38:42.467054 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/c43f59f2-deb7-46ee-8e07-54e3ff9f59aa-run-httpd\") pod \"ceilometer-0\" (UID: \"c43f59f2-deb7-46ee-8e07-54e3ff9f59aa\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:38:42 crc kubenswrapper[5052]: I1203 11:38:42.467093 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h8zbz\" (UniqueName: \"kubernetes.io/projected/c43f59f2-deb7-46ee-8e07-54e3ff9f59aa-kube-api-access-h8zbz\") pod \"ceilometer-0\" (UID: \"c43f59f2-deb7-46ee-8e07-54e3ff9f59aa\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:38:42 crc kubenswrapper[5052]: I1203 11:38:42.467119 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/d1676789-356e-4853-a302-f2c5bd3f446f-scripts\") pod \"cinder-db-sync-pbw4k\" (UID: \"d1676789-356e-4853-a302-f2c5bd3f446f\") " pod="openstack-kuttl-tests/cinder-db-sync-pbw4k" Dec 03 11:38:42 crc kubenswrapper[5052]: I1203 11:38:42.467162 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/d1676789-356e-4853-a302-f2c5bd3f446f-etc-machine-id\") pod \"cinder-db-sync-pbw4k\" (UID: \"d1676789-356e-4853-a302-f2c5bd3f446f\") " pod="openstack-kuttl-tests/cinder-db-sync-pbw4k" Dec 03 11:38:42 crc kubenswrapper[5052]: I1203 11:38:42.467187 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/c43f59f2-deb7-46ee-8e07-54e3ff9f59aa-config-data\") pod \"ceilometer-0\" (UID: \"c43f59f2-deb7-46ee-8e07-54e3ff9f59aa\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:38:42 crc kubenswrapper[5052]: I1203 11:38:42.467216 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/5ff261a9-6d56-45fe-88fa-7184d8a3b90b-logs\") pod \"placement-db-sync-qp7zq\" (UID: \"5ff261a9-6d56-45fe-88fa-7184d8a3b90b\") " pod="openstack-kuttl-tests/placement-db-sync-qp7zq" Dec 03 11:38:42 crc kubenswrapper[5052]: I1203 11:38:42.467239 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l6jzd\" (UniqueName: \"kubernetes.io/projected/5ff261a9-6d56-45fe-88fa-7184d8a3b90b-kube-api-access-l6jzd\") pod \"placement-db-sync-qp7zq\" (UID: \"5ff261a9-6d56-45fe-88fa-7184d8a3b90b\") " pod="openstack-kuttl-tests/placement-db-sync-qp7zq" Dec 03 11:38:42 crc kubenswrapper[5052]: I1203 11:38:42.475695 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-db-sync-j7t4l"] Dec 03 11:38:42 crc kubenswrapper[5052]: I1203 11:38:42.492579 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/barbican-db-sync-bfv74"] Dec 03 11:38:42 crc kubenswrapper[5052]: I1203 11:38:42.493694 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-db-sync-bfv74" Dec 03 11:38:42 crc kubenswrapper[5052]: I1203 11:38:42.498486 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"barbican-barbican-dockercfg-bq7sh" Dec 03 11:38:42 crc kubenswrapper[5052]: I1203 11:38:42.498793 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"barbican-config-data" Dec 03 11:38:42 crc kubenswrapper[5052]: I1203 11:38:42.523044 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-db-sync-bfv74"] Dec 03 11:38:42 crc kubenswrapper[5052]: I1203 11:38:42.570010 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/aae8de3e-7ad6-4a25-bdd5-3f98730ae227-combined-ca-bundle\") pod \"neutron-db-sync-j7t4l\" (UID: \"aae8de3e-7ad6-4a25-bdd5-3f98730ae227\") " pod="openstack-kuttl-tests/neutron-db-sync-j7t4l" Dec 03 11:38:42 crc kubenswrapper[5052]: I1203 11:38:42.570063 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/c43f59f2-deb7-46ee-8e07-54e3ff9f59aa-scripts\") pod \"ceilometer-0\" (UID: \"c43f59f2-deb7-46ee-8e07-54e3ff9f59aa\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:38:42 crc kubenswrapper[5052]: I1203 11:38:42.570108 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/d1676789-356e-4853-a302-f2c5bd3f446f-config-data\") pod \"cinder-db-sync-pbw4k\" (UID: \"d1676789-356e-4853-a302-f2c5bd3f446f\") " pod="openstack-kuttl-tests/cinder-db-sync-pbw4k" Dec 03 11:38:42 crc kubenswrapper[5052]: I1203 11:38:42.570130 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/c43f59f2-deb7-46ee-8e07-54e3ff9f59aa-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"c43f59f2-deb7-46ee-8e07-54e3ff9f59aa\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:38:42 crc kubenswrapper[5052]: I1203 11:38:42.570151 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/5ff261a9-6d56-45fe-88fa-7184d8a3b90b-scripts\") pod \"placement-db-sync-qp7zq\" (UID: \"5ff261a9-6d56-45fe-88fa-7184d8a3b90b\") " pod="openstack-kuttl-tests/placement-db-sync-qp7zq" Dec 03 11:38:42 crc kubenswrapper[5052]: I1203 11:38:42.570172 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/d1676789-356e-4853-a302-f2c5bd3f446f-db-sync-config-data\") pod \"cinder-db-sync-pbw4k\" (UID: \"d1676789-356e-4853-a302-f2c5bd3f446f\") " pod="openstack-kuttl-tests/cinder-db-sync-pbw4k" Dec 03 11:38:42 crc kubenswrapper[5052]: I1203 11:38:42.570197 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/5ff261a9-6d56-45fe-88fa-7184d8a3b90b-config-data\") pod \"placement-db-sync-qp7zq\" (UID: \"5ff261a9-6d56-45fe-88fa-7184d8a3b90b\") " pod="openstack-kuttl-tests/placement-db-sync-qp7zq" Dec 03 11:38:42 crc kubenswrapper[5052]: I1203 11:38:42.570221 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/c43f59f2-deb7-46ee-8e07-54e3ff9f59aa-run-httpd\") pod \"ceilometer-0\" (UID: \"c43f59f2-deb7-46ee-8e07-54e3ff9f59aa\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:38:42 crc kubenswrapper[5052]: I1203 11:38:42.570258 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-h8zbz\" (UniqueName: \"kubernetes.io/projected/c43f59f2-deb7-46ee-8e07-54e3ff9f59aa-kube-api-access-h8zbz\") pod \"ceilometer-0\" (UID: \"c43f59f2-deb7-46ee-8e07-54e3ff9f59aa\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:38:42 crc kubenswrapper[5052]: I1203 11:38:42.570281 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/d1676789-356e-4853-a302-f2c5bd3f446f-scripts\") pod \"cinder-db-sync-pbw4k\" (UID: \"d1676789-356e-4853-a302-f2c5bd3f446f\") " pod="openstack-kuttl-tests/cinder-db-sync-pbw4k" Dec 03 11:38:42 crc kubenswrapper[5052]: I1203 11:38:42.570348 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/d1676789-356e-4853-a302-f2c5bd3f446f-etc-machine-id\") pod \"cinder-db-sync-pbw4k\" (UID: \"d1676789-356e-4853-a302-f2c5bd3f446f\") " pod="openstack-kuttl-tests/cinder-db-sync-pbw4k" Dec 03 11:38:42 crc kubenswrapper[5052]: I1203 11:38:42.570377 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/c43f59f2-deb7-46ee-8e07-54e3ff9f59aa-config-data\") pod \"ceilometer-0\" (UID: \"c43f59f2-deb7-46ee-8e07-54e3ff9f59aa\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:38:42 crc kubenswrapper[5052]: I1203 11:38:42.570426 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1dc19382-24a8-4a4f-b735-d794619492bf-combined-ca-bundle\") pod \"barbican-db-sync-bfv74\" (UID: \"1dc19382-24a8-4a4f-b735-d794619492bf\") " pod="openstack-kuttl-tests/barbican-db-sync-bfv74" Dec 03 11:38:42 crc kubenswrapper[5052]: I1203 11:38:42.570462 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/5ff261a9-6d56-45fe-88fa-7184d8a3b90b-logs\") pod \"placement-db-sync-qp7zq\" (UID: \"5ff261a9-6d56-45fe-88fa-7184d8a3b90b\") " pod="openstack-kuttl-tests/placement-db-sync-qp7zq" Dec 03 11:38:42 crc kubenswrapper[5052]: I1203 11:38:42.570483 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-l6jzd\" (UniqueName: \"kubernetes.io/projected/5ff261a9-6d56-45fe-88fa-7184d8a3b90b-kube-api-access-l6jzd\") pod \"placement-db-sync-qp7zq\" (UID: \"5ff261a9-6d56-45fe-88fa-7184d8a3b90b\") " pod="openstack-kuttl-tests/placement-db-sync-qp7zq" Dec 03 11:38:42 crc kubenswrapper[5052]: I1203 11:38:42.570536 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xrvc5\" (UniqueName: \"kubernetes.io/projected/aae8de3e-7ad6-4a25-bdd5-3f98730ae227-kube-api-access-xrvc5\") pod \"neutron-db-sync-j7t4l\" (UID: \"aae8de3e-7ad6-4a25-bdd5-3f98730ae227\") " pod="openstack-kuttl-tests/neutron-db-sync-j7t4l" Dec 03 11:38:42 crc kubenswrapper[5052]: I1203 11:38:42.570559 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d1676789-356e-4853-a302-f2c5bd3f446f-combined-ca-bundle\") pod \"cinder-db-sync-pbw4k\" (UID: \"d1676789-356e-4853-a302-f2c5bd3f446f\") " pod="openstack-kuttl-tests/cinder-db-sync-pbw4k" Dec 03 11:38:42 crc kubenswrapper[5052]: I1203 11:38:42.570586 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/1dc19382-24a8-4a4f-b735-d794619492bf-db-sync-config-data\") pod \"barbican-db-sync-bfv74\" (UID: \"1dc19382-24a8-4a4f-b735-d794619492bf\") " pod="openstack-kuttl-tests/barbican-db-sync-bfv74" Dec 03 11:38:42 crc kubenswrapper[5052]: I1203 11:38:42.570608 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5ff261a9-6d56-45fe-88fa-7184d8a3b90b-combined-ca-bundle\") pod \"placement-db-sync-qp7zq\" (UID: \"5ff261a9-6d56-45fe-88fa-7184d8a3b90b\") " pod="openstack-kuttl-tests/placement-db-sync-qp7zq" Dec 03 11:38:42 crc kubenswrapper[5052]: I1203 11:38:42.570632 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/aae8de3e-7ad6-4a25-bdd5-3f98730ae227-config\") pod \"neutron-db-sync-j7t4l\" (UID: \"aae8de3e-7ad6-4a25-bdd5-3f98730ae227\") " pod="openstack-kuttl-tests/neutron-db-sync-j7t4l" Dec 03 11:38:42 crc kubenswrapper[5052]: I1203 11:38:42.570662 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zp4pf\" (UniqueName: \"kubernetes.io/projected/1dc19382-24a8-4a4f-b735-d794619492bf-kube-api-access-zp4pf\") pod \"barbican-db-sync-bfv74\" (UID: \"1dc19382-24a8-4a4f-b735-d794619492bf\") " pod="openstack-kuttl-tests/barbican-db-sync-bfv74" Dec 03 11:38:42 crc kubenswrapper[5052]: I1203 11:38:42.570684 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-8lqcr\" (UniqueName: \"kubernetes.io/projected/d1676789-356e-4853-a302-f2c5bd3f446f-kube-api-access-8lqcr\") pod \"cinder-db-sync-pbw4k\" (UID: \"d1676789-356e-4853-a302-f2c5bd3f446f\") " pod="openstack-kuttl-tests/cinder-db-sync-pbw4k" Dec 03 11:38:42 crc kubenswrapper[5052]: I1203 11:38:42.570715 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c43f59f2-deb7-46ee-8e07-54e3ff9f59aa-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"c43f59f2-deb7-46ee-8e07-54e3ff9f59aa\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:38:42 crc kubenswrapper[5052]: I1203 11:38:42.570737 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/c43f59f2-deb7-46ee-8e07-54e3ff9f59aa-log-httpd\") pod \"ceilometer-0\" (UID: \"c43f59f2-deb7-46ee-8e07-54e3ff9f59aa\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:38:42 crc kubenswrapper[5052]: I1203 11:38:42.572425 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/5ff261a9-6d56-45fe-88fa-7184d8a3b90b-logs\") pod \"placement-db-sync-qp7zq\" (UID: \"5ff261a9-6d56-45fe-88fa-7184d8a3b90b\") " pod="openstack-kuttl-tests/placement-db-sync-qp7zq" Dec 03 11:38:42 crc kubenswrapper[5052]: I1203 11:38:42.576229 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/c43f59f2-deb7-46ee-8e07-54e3ff9f59aa-run-httpd\") pod \"ceilometer-0\" (UID: \"c43f59f2-deb7-46ee-8e07-54e3ff9f59aa\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:38:42 crc kubenswrapper[5052]: I1203 11:38:42.576814 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/d1676789-356e-4853-a302-f2c5bd3f446f-etc-machine-id\") pod \"cinder-db-sync-pbw4k\" (UID: \"d1676789-356e-4853-a302-f2c5bd3f446f\") " pod="openstack-kuttl-tests/cinder-db-sync-pbw4k" Dec 03 11:38:42 crc kubenswrapper[5052]: I1203 11:38:42.578790 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/c43f59f2-deb7-46ee-8e07-54e3ff9f59aa-config-data\") pod \"ceilometer-0\" (UID: \"c43f59f2-deb7-46ee-8e07-54e3ff9f59aa\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:38:42 crc kubenswrapper[5052]: I1203 11:38:42.579493 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d1676789-356e-4853-a302-f2c5bd3f446f-combined-ca-bundle\") pod \"cinder-db-sync-pbw4k\" (UID: \"d1676789-356e-4853-a302-f2c5bd3f446f\") " pod="openstack-kuttl-tests/cinder-db-sync-pbw4k" Dec 03 11:38:42 crc kubenswrapper[5052]: I1203 11:38:42.579614 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/5ff261a9-6d56-45fe-88fa-7184d8a3b90b-config-data\") pod \"placement-db-sync-qp7zq\" (UID: \"5ff261a9-6d56-45fe-88fa-7184d8a3b90b\") " pod="openstack-kuttl-tests/placement-db-sync-qp7zq" Dec 03 11:38:42 crc kubenswrapper[5052]: I1203 11:38:42.580250 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/5ff261a9-6d56-45fe-88fa-7184d8a3b90b-scripts\") pod \"placement-db-sync-qp7zq\" (UID: \"5ff261a9-6d56-45fe-88fa-7184d8a3b90b\") " pod="openstack-kuttl-tests/placement-db-sync-qp7zq" Dec 03 11:38:42 crc kubenswrapper[5052]: I1203 11:38:42.582617 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/d1676789-356e-4853-a302-f2c5bd3f446f-scripts\") pod \"cinder-db-sync-pbw4k\" (UID: \"d1676789-356e-4853-a302-f2c5bd3f446f\") " pod="openstack-kuttl-tests/cinder-db-sync-pbw4k" Dec 03 11:38:42 crc kubenswrapper[5052]: I1203 11:38:42.583933 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/c43f59f2-deb7-46ee-8e07-54e3ff9f59aa-scripts\") pod \"ceilometer-0\" (UID: \"c43f59f2-deb7-46ee-8e07-54e3ff9f59aa\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:38:42 crc kubenswrapper[5052]: I1203 11:38:42.592749 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5ff261a9-6d56-45fe-88fa-7184d8a3b90b-combined-ca-bundle\") pod \"placement-db-sync-qp7zq\" (UID: \"5ff261a9-6d56-45fe-88fa-7184d8a3b90b\") " pod="openstack-kuttl-tests/placement-db-sync-qp7zq" Dec 03 11:38:42 crc kubenswrapper[5052]: I1203 11:38:42.593251 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/d1676789-356e-4853-a302-f2c5bd3f446f-db-sync-config-data\") pod \"cinder-db-sync-pbw4k\" (UID: \"d1676789-356e-4853-a302-f2c5bd3f446f\") " pod="openstack-kuttl-tests/cinder-db-sync-pbw4k" Dec 03 11:38:42 crc kubenswrapper[5052]: I1203 11:38:42.594167 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/c43f59f2-deb7-46ee-8e07-54e3ff9f59aa-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"c43f59f2-deb7-46ee-8e07-54e3ff9f59aa\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:38:42 crc kubenswrapper[5052]: I1203 11:38:42.594301 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-l6jzd\" (UniqueName: \"kubernetes.io/projected/5ff261a9-6d56-45fe-88fa-7184d8a3b90b-kube-api-access-l6jzd\") pod \"placement-db-sync-qp7zq\" (UID: \"5ff261a9-6d56-45fe-88fa-7184d8a3b90b\") " pod="openstack-kuttl-tests/placement-db-sync-qp7zq" Dec 03 11:38:42 crc kubenswrapper[5052]: I1203 11:38:42.571183 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/c43f59f2-deb7-46ee-8e07-54e3ff9f59aa-log-httpd\") pod \"ceilometer-0\" (UID: \"c43f59f2-deb7-46ee-8e07-54e3ff9f59aa\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:38:42 crc kubenswrapper[5052]: I1203 11:38:42.594917 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c43f59f2-deb7-46ee-8e07-54e3ff9f59aa-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"c43f59f2-deb7-46ee-8e07-54e3ff9f59aa\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:38:42 crc kubenswrapper[5052]: I1203 11:38:42.597380 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-h8zbz\" (UniqueName: \"kubernetes.io/projected/c43f59f2-deb7-46ee-8e07-54e3ff9f59aa-kube-api-access-h8zbz\") pod \"ceilometer-0\" (UID: \"c43f59f2-deb7-46ee-8e07-54e3ff9f59aa\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:38:42 crc kubenswrapper[5052]: I1203 11:38:42.600302 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-8lqcr\" (UniqueName: \"kubernetes.io/projected/d1676789-356e-4853-a302-f2c5bd3f446f-kube-api-access-8lqcr\") pod \"cinder-db-sync-pbw4k\" (UID: \"d1676789-356e-4853-a302-f2c5bd3f446f\") " pod="openstack-kuttl-tests/cinder-db-sync-pbw4k" Dec 03 11:38:42 crc kubenswrapper[5052]: I1203 11:38:42.615711 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/d1676789-356e-4853-a302-f2c5bd3f446f-config-data\") pod \"cinder-db-sync-pbw4k\" (UID: \"d1676789-356e-4853-a302-f2c5bd3f446f\") " pod="openstack-kuttl-tests/cinder-db-sync-pbw4k" Dec 03 11:38:42 crc kubenswrapper[5052]: I1203 11:38:42.672986 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1dc19382-24a8-4a4f-b735-d794619492bf-combined-ca-bundle\") pod \"barbican-db-sync-bfv74\" (UID: \"1dc19382-24a8-4a4f-b735-d794619492bf\") " pod="openstack-kuttl-tests/barbican-db-sync-bfv74" Dec 03 11:38:42 crc kubenswrapper[5052]: I1203 11:38:42.673077 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-xrvc5\" (UniqueName: \"kubernetes.io/projected/aae8de3e-7ad6-4a25-bdd5-3f98730ae227-kube-api-access-xrvc5\") pod \"neutron-db-sync-j7t4l\" (UID: \"aae8de3e-7ad6-4a25-bdd5-3f98730ae227\") " pod="openstack-kuttl-tests/neutron-db-sync-j7t4l" Dec 03 11:38:42 crc kubenswrapper[5052]: I1203 11:38:42.673113 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/1dc19382-24a8-4a4f-b735-d794619492bf-db-sync-config-data\") pod \"barbican-db-sync-bfv74\" (UID: \"1dc19382-24a8-4a4f-b735-d794619492bf\") " pod="openstack-kuttl-tests/barbican-db-sync-bfv74" Dec 03 11:38:42 crc kubenswrapper[5052]: I1203 11:38:42.673150 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/aae8de3e-7ad6-4a25-bdd5-3f98730ae227-config\") pod \"neutron-db-sync-j7t4l\" (UID: \"aae8de3e-7ad6-4a25-bdd5-3f98730ae227\") " pod="openstack-kuttl-tests/neutron-db-sync-j7t4l" Dec 03 11:38:42 crc kubenswrapper[5052]: I1203 11:38:42.673190 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-zp4pf\" (UniqueName: \"kubernetes.io/projected/1dc19382-24a8-4a4f-b735-d794619492bf-kube-api-access-zp4pf\") pod \"barbican-db-sync-bfv74\" (UID: \"1dc19382-24a8-4a4f-b735-d794619492bf\") " pod="openstack-kuttl-tests/barbican-db-sync-bfv74" Dec 03 11:38:42 crc kubenswrapper[5052]: I1203 11:38:42.673255 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/aae8de3e-7ad6-4a25-bdd5-3f98730ae227-combined-ca-bundle\") pod \"neutron-db-sync-j7t4l\" (UID: \"aae8de3e-7ad6-4a25-bdd5-3f98730ae227\") " pod="openstack-kuttl-tests/neutron-db-sync-j7t4l" Dec 03 11:38:42 crc kubenswrapper[5052]: I1203 11:38:42.679155 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/aae8de3e-7ad6-4a25-bdd5-3f98730ae227-combined-ca-bundle\") pod \"neutron-db-sync-j7t4l\" (UID: \"aae8de3e-7ad6-4a25-bdd5-3f98730ae227\") " pod="openstack-kuttl-tests/neutron-db-sync-j7t4l" Dec 03 11:38:42 crc kubenswrapper[5052]: I1203 11:38:42.679160 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1dc19382-24a8-4a4f-b735-d794619492bf-combined-ca-bundle\") pod \"barbican-db-sync-bfv74\" (UID: \"1dc19382-24a8-4a4f-b735-d794619492bf\") " pod="openstack-kuttl-tests/barbican-db-sync-bfv74" Dec 03 11:38:42 crc kubenswrapper[5052]: I1203 11:38:42.681012 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/aae8de3e-7ad6-4a25-bdd5-3f98730ae227-config\") pod \"neutron-db-sync-j7t4l\" (UID: \"aae8de3e-7ad6-4a25-bdd5-3f98730ae227\") " pod="openstack-kuttl-tests/neutron-db-sync-j7t4l" Dec 03 11:38:42 crc kubenswrapper[5052]: I1203 11:38:42.686128 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/1dc19382-24a8-4a4f-b735-d794619492bf-db-sync-config-data\") pod \"barbican-db-sync-bfv74\" (UID: \"1dc19382-24a8-4a4f-b735-d794619492bf\") " pod="openstack-kuttl-tests/barbican-db-sync-bfv74" Dec 03 11:38:42 crc kubenswrapper[5052]: I1203 11:38:42.689747 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-zp4pf\" (UniqueName: \"kubernetes.io/projected/1dc19382-24a8-4a4f-b735-d794619492bf-kube-api-access-zp4pf\") pod \"barbican-db-sync-bfv74\" (UID: \"1dc19382-24a8-4a4f-b735-d794619492bf\") " pod="openstack-kuttl-tests/barbican-db-sync-bfv74" Dec 03 11:38:42 crc kubenswrapper[5052]: I1203 11:38:42.691183 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-xrvc5\" (UniqueName: \"kubernetes.io/projected/aae8de3e-7ad6-4a25-bdd5-3f98730ae227-kube-api-access-xrvc5\") pod \"neutron-db-sync-j7t4l\" (UID: \"aae8de3e-7ad6-4a25-bdd5-3f98730ae227\") " pod="openstack-kuttl-tests/neutron-db-sync-j7t4l" Dec 03 11:38:42 crc kubenswrapper[5052]: I1203 11:38:42.725881 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:38:42 crc kubenswrapper[5052]: I1203 11:38:42.752256 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-db-sync-pbw4k" Dec 03 11:38:42 crc kubenswrapper[5052]: I1203 11:38:42.870350 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-db-sync-qp7zq" Dec 03 11:38:42 crc kubenswrapper[5052]: I1203 11:38:42.892085 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-db-sync-j7t4l" Dec 03 11:38:42 crc kubenswrapper[5052]: I1203 11:38:42.903806 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-db-sync-bfv74" Dec 03 11:38:42 crc kubenswrapper[5052]: I1203 11:38:42.933982 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-bootstrap-fstdw"] Dec 03 11:38:43 crc kubenswrapper[5052]: I1203 11:38:43.115387 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-db-sync-pbw4k"] Dec 03 11:38:43 crc kubenswrapper[5052]: I1203 11:38:43.232331 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:38:43 crc kubenswrapper[5052]: I1203 11:38:43.531960 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-db-sync-bfv74"] Dec 03 11:38:43 crc kubenswrapper[5052]: I1203 11:38:43.544215 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/placement-db-sync-qp7zq"] Dec 03 11:38:43 crc kubenswrapper[5052]: I1203 11:38:43.554790 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-db-sync-j7t4l"] Dec 03 11:38:43 crc kubenswrapper[5052]: I1203 11:38:43.961407 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-db-sync-qp7zq" event={"ID":"5ff261a9-6d56-45fe-88fa-7184d8a3b90b","Type":"ContainerStarted","Data":"ff691e377bc65754a8824271a09fbe6041732211d46c452b77e403ba10dcf603"} Dec 03 11:38:43 crc kubenswrapper[5052]: I1203 11:38:43.963208 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"c43f59f2-deb7-46ee-8e07-54e3ff9f59aa","Type":"ContainerStarted","Data":"465b7c2dbcbc8a63d5b3400d3cacd0b29170f695e867e401f0b9fbb2dd63587f"} Dec 03 11:38:43 crc kubenswrapper[5052]: I1203 11:38:43.965756 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-db-sync-pbw4k" event={"ID":"d1676789-356e-4853-a302-f2c5bd3f446f","Type":"ContainerStarted","Data":"9c3a23d24e9f19c47daa0c2a455117a451af85c686194e2b184c1f25a5ac6f52"} Dec 03 11:38:43 crc kubenswrapper[5052]: I1203 11:38:43.967808 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-db-sync-bfv74" event={"ID":"1dc19382-24a8-4a4f-b735-d794619492bf","Type":"ContainerStarted","Data":"f5e0f6eb722d532f0bd3399d5aa3cad9c6468d78aabaf922d936fc2eddd2804b"} Dec 03 11:38:43 crc kubenswrapper[5052]: I1203 11:38:43.976941 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-db-sync-j7t4l" event={"ID":"aae8de3e-7ad6-4a25-bdd5-3f98730ae227","Type":"ContainerStarted","Data":"b976664871cebff06450133c9273e24bbc1c92f6e86458b6364c4f2e4f2e4d6f"} Dec 03 11:38:43 crc kubenswrapper[5052]: I1203 11:38:43.977099 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-db-sync-j7t4l" event={"ID":"aae8de3e-7ad6-4a25-bdd5-3f98730ae227","Type":"ContainerStarted","Data":"8ef02fc07bf5d93df6b03f41902ac447aba7de6e23dd011ce30455235ca91854"} Dec 03 11:38:43 crc kubenswrapper[5052]: I1203 11:38:43.979681 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-bootstrap-fstdw" event={"ID":"e1e49fa1-2ef1-4f43-a0b2-f98c1364bdcb","Type":"ContainerStarted","Data":"52ef4f5dde2a47529b2ebf6b602ec3711d01b687b6e696dd4f128ab5239e064a"} Dec 03 11:38:43 crc kubenswrapper[5052]: I1203 11:38:43.979724 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-bootstrap-fstdw" event={"ID":"e1e49fa1-2ef1-4f43-a0b2-f98c1364bdcb","Type":"ContainerStarted","Data":"66bd9df9253dda0cb16759f9020973dc876018932d166cda76b8d5df39a0e91c"} Dec 03 11:38:44 crc kubenswrapper[5052]: I1203 11:38:44.006673 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/neutron-db-sync-j7t4l" podStartSLOduration=2.006657205 podStartE2EDuration="2.006657205s" podCreationTimestamp="2025-12-03 11:38:42 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:38:43.999449917 +0000 UTC m=+1114.587691681" watchObservedRunningTime="2025-12-03 11:38:44.006657205 +0000 UTC m=+1114.594898969" Dec 03 11:38:44 crc kubenswrapper[5052]: I1203 11:38:44.025953 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/keystone-bootstrap-fstdw" podStartSLOduration=2.025929941 podStartE2EDuration="2.025929941s" podCreationTimestamp="2025-12-03 11:38:42 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:38:44.018465686 +0000 UTC m=+1114.606707460" watchObservedRunningTime="2025-12-03 11:38:44.025929941 +0000 UTC m=+1114.614171705" Dec 03 11:38:44 crc kubenswrapper[5052]: I1203 11:38:44.337420 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:38:44 crc kubenswrapper[5052]: I1203 11:38:44.993409 5052 generic.go:334] "Generic (PLEG): container finished" podID="063c5d2e-68a5-497a-b7bd-f95d45804848" containerID="831ae3f9746c506b87d5990844ed13f7bb77d7fc3b6415de96a98290e3869a4c" exitCode=0 Dec 03 11:38:44 crc kubenswrapper[5052]: I1203 11:38:44.994420 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-db-sync-k6vlv" event={"ID":"063c5d2e-68a5-497a-b7bd-f95d45804848","Type":"ContainerDied","Data":"831ae3f9746c506b87d5990844ed13f7bb77d7fc3b6415de96a98290e3869a4c"} Dec 03 11:38:48 crc kubenswrapper[5052]: I1203 11:38:48.028269 5052 generic.go:334] "Generic (PLEG): container finished" podID="e1e49fa1-2ef1-4f43-a0b2-f98c1364bdcb" containerID="52ef4f5dde2a47529b2ebf6b602ec3711d01b687b6e696dd4f128ab5239e064a" exitCode=0 Dec 03 11:38:48 crc kubenswrapper[5052]: I1203 11:38:48.028333 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-bootstrap-fstdw" event={"ID":"e1e49fa1-2ef1-4f43-a0b2-f98c1364bdcb","Type":"ContainerDied","Data":"52ef4f5dde2a47529b2ebf6b602ec3711d01b687b6e696dd4f128ab5239e064a"} Dec 03 11:38:51 crc kubenswrapper[5052]: I1203 11:38:51.252084 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-db-sync-k6vlv" Dec 03 11:38:51 crc kubenswrapper[5052]: I1203 11:38:51.351152 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/063c5d2e-68a5-497a-b7bd-f95d45804848-combined-ca-bundle\") pod \"063c5d2e-68a5-497a-b7bd-f95d45804848\" (UID: \"063c5d2e-68a5-497a-b7bd-f95d45804848\") " Dec 03 11:38:51 crc kubenswrapper[5052]: I1203 11:38:51.351240 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/063c5d2e-68a5-497a-b7bd-f95d45804848-db-sync-config-data\") pod \"063c5d2e-68a5-497a-b7bd-f95d45804848\" (UID: \"063c5d2e-68a5-497a-b7bd-f95d45804848\") " Dec 03 11:38:51 crc kubenswrapper[5052]: I1203 11:38:51.351290 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b2b64\" (UniqueName: \"kubernetes.io/projected/063c5d2e-68a5-497a-b7bd-f95d45804848-kube-api-access-b2b64\") pod \"063c5d2e-68a5-497a-b7bd-f95d45804848\" (UID: \"063c5d2e-68a5-497a-b7bd-f95d45804848\") " Dec 03 11:38:51 crc kubenswrapper[5052]: I1203 11:38:51.351308 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/063c5d2e-68a5-497a-b7bd-f95d45804848-config-data\") pod \"063c5d2e-68a5-497a-b7bd-f95d45804848\" (UID: \"063c5d2e-68a5-497a-b7bd-f95d45804848\") " Dec 03 11:38:51 crc kubenswrapper[5052]: I1203 11:38:51.356837 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/063c5d2e-68a5-497a-b7bd-f95d45804848-db-sync-config-data" (OuterVolumeSpecName: "db-sync-config-data") pod "063c5d2e-68a5-497a-b7bd-f95d45804848" (UID: "063c5d2e-68a5-497a-b7bd-f95d45804848"). InnerVolumeSpecName "db-sync-config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:38:51 crc kubenswrapper[5052]: I1203 11:38:51.358653 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/063c5d2e-68a5-497a-b7bd-f95d45804848-kube-api-access-b2b64" (OuterVolumeSpecName: "kube-api-access-b2b64") pod "063c5d2e-68a5-497a-b7bd-f95d45804848" (UID: "063c5d2e-68a5-497a-b7bd-f95d45804848"). InnerVolumeSpecName "kube-api-access-b2b64". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:38:51 crc kubenswrapper[5052]: I1203 11:38:51.378010 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/063c5d2e-68a5-497a-b7bd-f95d45804848-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "063c5d2e-68a5-497a-b7bd-f95d45804848" (UID: "063c5d2e-68a5-497a-b7bd-f95d45804848"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:38:51 crc kubenswrapper[5052]: I1203 11:38:51.404488 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/063c5d2e-68a5-497a-b7bd-f95d45804848-config-data" (OuterVolumeSpecName: "config-data") pod "063c5d2e-68a5-497a-b7bd-f95d45804848" (UID: "063c5d2e-68a5-497a-b7bd-f95d45804848"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:38:51 crc kubenswrapper[5052]: I1203 11:38:51.452904 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-b2b64\" (UniqueName: \"kubernetes.io/projected/063c5d2e-68a5-497a-b7bd-f95d45804848-kube-api-access-b2b64\") on node \"crc\" DevicePath \"\"" Dec 03 11:38:51 crc kubenswrapper[5052]: I1203 11:38:51.452940 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/063c5d2e-68a5-497a-b7bd-f95d45804848-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:38:51 crc kubenswrapper[5052]: I1203 11:38:51.452950 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/063c5d2e-68a5-497a-b7bd-f95d45804848-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:38:51 crc kubenswrapper[5052]: I1203 11:38:51.452959 5052 reconciler_common.go:293] "Volume detached for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/063c5d2e-68a5-497a-b7bd-f95d45804848-db-sync-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:38:52 crc kubenswrapper[5052]: I1203 11:38:52.069864 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-db-sync-k6vlv" event={"ID":"063c5d2e-68a5-497a-b7bd-f95d45804848","Type":"ContainerDied","Data":"f240f6af4fd3ce44721e4cc39bb4c69c2cfd30b0fa77f7dee789e7b31083f7ee"} Dec 03 11:38:52 crc kubenswrapper[5052]: I1203 11:38:52.069905 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f240f6af4fd3ce44721e4cc39bb4c69c2cfd30b0fa77f7dee789e7b31083f7ee" Dec 03 11:38:52 crc kubenswrapper[5052]: I1203 11:38:52.069914 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-db-sync-k6vlv" Dec 03 11:38:53 crc kubenswrapper[5052]: I1203 11:38:53.638420 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 11:38:53 crc kubenswrapper[5052]: E1203 11:38:53.639169 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="063c5d2e-68a5-497a-b7bd-f95d45804848" containerName="glance-db-sync" Dec 03 11:38:53 crc kubenswrapper[5052]: I1203 11:38:53.639186 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="063c5d2e-68a5-497a-b7bd-f95d45804848" containerName="glance-db-sync" Dec 03 11:38:53 crc kubenswrapper[5052]: I1203 11:38:53.639416 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="063c5d2e-68a5-497a-b7bd-f95d45804848" containerName="glance-db-sync" Dec 03 11:38:53 crc kubenswrapper[5052]: I1203 11:38:53.640547 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:38:53 crc kubenswrapper[5052]: I1203 11:38:53.642279 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-scripts" Dec 03 11:38:53 crc kubenswrapper[5052]: I1203 11:38:53.643885 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-default-internal-config-data" Dec 03 11:38:53 crc kubenswrapper[5052]: I1203 11:38:53.644105 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-glance-dockercfg-kt8wz" Dec 03 11:38:53 crc kubenswrapper[5052]: I1203 11:38:53.652722 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 11:38:53 crc kubenswrapper[5052]: I1203 11:38:53.713608 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 11:38:53 crc kubenswrapper[5052]: I1203 11:38:53.715068 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:38:53 crc kubenswrapper[5052]: I1203 11:38:53.717670 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-default-external-config-data" Dec 03 11:38:53 crc kubenswrapper[5052]: I1203 11:38:53.722526 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 11:38:53 crc kubenswrapper[5052]: I1203 11:38:53.790817 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/3a98a307-4671-43f7-86af-3b86591c8d7e-config-data\") pod \"glance-default-internal-api-0\" (UID: \"3a98a307-4671-43f7-86af-3b86591c8d7e\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:38:53 crc kubenswrapper[5052]: I1203 11:38:53.790889 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/3a98a307-4671-43f7-86af-3b86591c8d7e-httpd-run\") pod \"glance-default-internal-api-0\" (UID: \"3a98a307-4671-43f7-86af-3b86591c8d7e\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:38:53 crc kubenswrapper[5052]: I1203 11:38:53.790943 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/3a98a307-4671-43f7-86af-3b86591c8d7e-scripts\") pod \"glance-default-internal-api-0\" (UID: \"3a98a307-4671-43f7-86af-3b86591c8d7e\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:38:53 crc kubenswrapper[5052]: I1203 11:38:53.790973 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/3a98a307-4671-43f7-86af-3b86591c8d7e-logs\") pod \"glance-default-internal-api-0\" (UID: \"3a98a307-4671-43f7-86af-3b86591c8d7e\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:38:53 crc kubenswrapper[5052]: I1203 11:38:53.791345 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3a98a307-4671-43f7-86af-3b86591c8d7e-combined-ca-bundle\") pod \"glance-default-internal-api-0\" (UID: \"3a98a307-4671-43f7-86af-3b86591c8d7e\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:38:53 crc kubenswrapper[5052]: I1203 11:38:53.791538 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jhsf4\" (UniqueName: \"kubernetes.io/projected/3a98a307-4671-43f7-86af-3b86591c8d7e-kube-api-access-jhsf4\") pod \"glance-default-internal-api-0\" (UID: \"3a98a307-4671-43f7-86af-3b86591c8d7e\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:38:53 crc kubenswrapper[5052]: I1203 11:38:53.791591 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage04-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage04-crc\") pod \"glance-default-internal-api-0\" (UID: \"3a98a307-4671-43f7-86af-3b86591c8d7e\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:38:53 crc kubenswrapper[5052]: I1203 11:38:53.892963 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage04-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage04-crc\") pod \"glance-default-internal-api-0\" (UID: \"3a98a307-4671-43f7-86af-3b86591c8d7e\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:38:53 crc kubenswrapper[5052]: I1203 11:38:53.893019 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/3a98a307-4671-43f7-86af-3b86591c8d7e-config-data\") pod \"glance-default-internal-api-0\" (UID: \"3a98a307-4671-43f7-86af-3b86591c8d7e\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:38:53 crc kubenswrapper[5052]: I1203 11:38:53.893046 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/de570c88-14c0-4761-be7f-da7a6d97715a-scripts\") pod \"glance-default-external-api-0\" (UID: \"de570c88-14c0-4761-be7f-da7a6d97715a\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:38:53 crc kubenswrapper[5052]: I1203 11:38:53.893067 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/de570c88-14c0-4761-be7f-da7a6d97715a-httpd-run\") pod \"glance-default-external-api-0\" (UID: \"de570c88-14c0-4761-be7f-da7a6d97715a\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:38:53 crc kubenswrapper[5052]: I1203 11:38:53.893093 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/3a98a307-4671-43f7-86af-3b86591c8d7e-httpd-run\") pod \"glance-default-internal-api-0\" (UID: \"3a98a307-4671-43f7-86af-3b86591c8d7e\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:38:53 crc kubenswrapper[5052]: I1203 11:38:53.893131 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/3a98a307-4671-43f7-86af-3b86591c8d7e-scripts\") pod \"glance-default-internal-api-0\" (UID: \"3a98a307-4671-43f7-86af-3b86591c8d7e\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:38:53 crc kubenswrapper[5052]: I1203 11:38:53.893151 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/3a98a307-4671-43f7-86af-3b86591c8d7e-logs\") pod \"glance-default-internal-api-0\" (UID: \"3a98a307-4671-43f7-86af-3b86591c8d7e\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:38:53 crc kubenswrapper[5052]: I1203 11:38:53.893192 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/de570c88-14c0-4761-be7f-da7a6d97715a-logs\") pod \"glance-default-external-api-0\" (UID: \"de570c88-14c0-4761-be7f-da7a6d97715a\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:38:53 crc kubenswrapper[5052]: I1203 11:38:53.893211 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/de570c88-14c0-4761-be7f-da7a6d97715a-combined-ca-bundle\") pod \"glance-default-external-api-0\" (UID: \"de570c88-14c0-4761-be7f-da7a6d97715a\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:38:53 crc kubenswrapper[5052]: I1203 11:38:53.893240 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/de570c88-14c0-4761-be7f-da7a6d97715a-config-data\") pod \"glance-default-external-api-0\" (UID: \"de570c88-14c0-4761-be7f-da7a6d97715a\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:38:53 crc kubenswrapper[5052]: I1203 11:38:53.893258 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6bqgp\" (UniqueName: \"kubernetes.io/projected/de570c88-14c0-4761-be7f-da7a6d97715a-kube-api-access-6bqgp\") pod \"glance-default-external-api-0\" (UID: \"de570c88-14c0-4761-be7f-da7a6d97715a\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:38:53 crc kubenswrapper[5052]: I1203 11:38:53.893279 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3a98a307-4671-43f7-86af-3b86591c8d7e-combined-ca-bundle\") pod \"glance-default-internal-api-0\" (UID: \"3a98a307-4671-43f7-86af-3b86591c8d7e\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:38:53 crc kubenswrapper[5052]: I1203 11:38:53.893307 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage15-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage15-crc\") pod \"glance-default-external-api-0\" (UID: \"de570c88-14c0-4761-be7f-da7a6d97715a\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:38:53 crc kubenswrapper[5052]: I1203 11:38:53.893327 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-jhsf4\" (UniqueName: \"kubernetes.io/projected/3a98a307-4671-43f7-86af-3b86591c8d7e-kube-api-access-jhsf4\") pod \"glance-default-internal-api-0\" (UID: \"3a98a307-4671-43f7-86af-3b86591c8d7e\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:38:53 crc kubenswrapper[5052]: I1203 11:38:53.893923 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage04-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage04-crc\") pod \"glance-default-internal-api-0\" (UID: \"3a98a307-4671-43f7-86af-3b86591c8d7e\") device mount path \"/mnt/openstack/pv04\"" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:38:53 crc kubenswrapper[5052]: I1203 11:38:53.895399 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/3a98a307-4671-43f7-86af-3b86591c8d7e-httpd-run\") pod \"glance-default-internal-api-0\" (UID: \"3a98a307-4671-43f7-86af-3b86591c8d7e\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:38:53 crc kubenswrapper[5052]: I1203 11:38:53.895905 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/3a98a307-4671-43f7-86af-3b86591c8d7e-logs\") pod \"glance-default-internal-api-0\" (UID: \"3a98a307-4671-43f7-86af-3b86591c8d7e\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:38:53 crc kubenswrapper[5052]: I1203 11:38:53.901284 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/3a98a307-4671-43f7-86af-3b86591c8d7e-config-data\") pod \"glance-default-internal-api-0\" (UID: \"3a98a307-4671-43f7-86af-3b86591c8d7e\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:38:53 crc kubenswrapper[5052]: I1203 11:38:53.901441 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/3a98a307-4671-43f7-86af-3b86591c8d7e-scripts\") pod \"glance-default-internal-api-0\" (UID: \"3a98a307-4671-43f7-86af-3b86591c8d7e\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:38:53 crc kubenswrapper[5052]: I1203 11:38:53.902420 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3a98a307-4671-43f7-86af-3b86591c8d7e-combined-ca-bundle\") pod \"glance-default-internal-api-0\" (UID: \"3a98a307-4671-43f7-86af-3b86591c8d7e\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:38:53 crc kubenswrapper[5052]: I1203 11:38:53.916519 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage04-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage04-crc\") pod \"glance-default-internal-api-0\" (UID: \"3a98a307-4671-43f7-86af-3b86591c8d7e\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:38:53 crc kubenswrapper[5052]: I1203 11:38:53.923168 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-jhsf4\" (UniqueName: \"kubernetes.io/projected/3a98a307-4671-43f7-86af-3b86591c8d7e-kube-api-access-jhsf4\") pod \"glance-default-internal-api-0\" (UID: \"3a98a307-4671-43f7-86af-3b86591c8d7e\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:38:53 crc kubenswrapper[5052]: I1203 11:38:53.961977 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:38:53 crc kubenswrapper[5052]: I1203 11:38:53.994470 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/de570c88-14c0-4761-be7f-da7a6d97715a-logs\") pod \"glance-default-external-api-0\" (UID: \"de570c88-14c0-4761-be7f-da7a6d97715a\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:38:53 crc kubenswrapper[5052]: I1203 11:38:53.994588 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/de570c88-14c0-4761-be7f-da7a6d97715a-combined-ca-bundle\") pod \"glance-default-external-api-0\" (UID: \"de570c88-14c0-4761-be7f-da7a6d97715a\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:38:53 crc kubenswrapper[5052]: I1203 11:38:53.994623 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/de570c88-14c0-4761-be7f-da7a6d97715a-config-data\") pod \"glance-default-external-api-0\" (UID: \"de570c88-14c0-4761-be7f-da7a6d97715a\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:38:53 crc kubenswrapper[5052]: I1203 11:38:53.995085 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-6bqgp\" (UniqueName: \"kubernetes.io/projected/de570c88-14c0-4761-be7f-da7a6d97715a-kube-api-access-6bqgp\") pod \"glance-default-external-api-0\" (UID: \"de570c88-14c0-4761-be7f-da7a6d97715a\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:38:53 crc kubenswrapper[5052]: I1203 11:38:53.995121 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage15-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage15-crc\") pod \"glance-default-external-api-0\" (UID: \"de570c88-14c0-4761-be7f-da7a6d97715a\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:38:53 crc kubenswrapper[5052]: I1203 11:38:53.995155 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/de570c88-14c0-4761-be7f-da7a6d97715a-scripts\") pod \"glance-default-external-api-0\" (UID: \"de570c88-14c0-4761-be7f-da7a6d97715a\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:38:53 crc kubenswrapper[5052]: I1203 11:38:53.995174 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/de570c88-14c0-4761-be7f-da7a6d97715a-httpd-run\") pod \"glance-default-external-api-0\" (UID: \"de570c88-14c0-4761-be7f-da7a6d97715a\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:38:53 crc kubenswrapper[5052]: I1203 11:38:53.995411 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage15-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage15-crc\") pod \"glance-default-external-api-0\" (UID: \"de570c88-14c0-4761-be7f-da7a6d97715a\") device mount path \"/mnt/openstack/pv15\"" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:38:53 crc kubenswrapper[5052]: I1203 11:38:53.995471 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/de570c88-14c0-4761-be7f-da7a6d97715a-httpd-run\") pod \"glance-default-external-api-0\" (UID: \"de570c88-14c0-4761-be7f-da7a6d97715a\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:38:53 crc kubenswrapper[5052]: I1203 11:38:53.995729 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/de570c88-14c0-4761-be7f-da7a6d97715a-logs\") pod \"glance-default-external-api-0\" (UID: \"de570c88-14c0-4761-be7f-da7a6d97715a\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:38:54 crc kubenswrapper[5052]: I1203 11:38:54.009725 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/de570c88-14c0-4761-be7f-da7a6d97715a-scripts\") pod \"glance-default-external-api-0\" (UID: \"de570c88-14c0-4761-be7f-da7a6d97715a\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:38:54 crc kubenswrapper[5052]: I1203 11:38:54.017431 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/de570c88-14c0-4761-be7f-da7a6d97715a-combined-ca-bundle\") pod \"glance-default-external-api-0\" (UID: \"de570c88-14c0-4761-be7f-da7a6d97715a\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:38:54 crc kubenswrapper[5052]: I1203 11:38:54.017953 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/de570c88-14c0-4761-be7f-da7a6d97715a-config-data\") pod \"glance-default-external-api-0\" (UID: \"de570c88-14c0-4761-be7f-da7a6d97715a\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:38:54 crc kubenswrapper[5052]: I1203 11:38:54.030943 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-6bqgp\" (UniqueName: \"kubernetes.io/projected/de570c88-14c0-4761-be7f-da7a6d97715a-kube-api-access-6bqgp\") pod \"glance-default-external-api-0\" (UID: \"de570c88-14c0-4761-be7f-da7a6d97715a\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:38:54 crc kubenswrapper[5052]: I1203 11:38:54.077934 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage15-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage15-crc\") pod \"glance-default-external-api-0\" (UID: \"de570c88-14c0-4761-be7f-da7a6d97715a\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:38:54 crc kubenswrapper[5052]: I1203 11:38:54.328053 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:38:54 crc kubenswrapper[5052]: I1203 11:38:54.881967 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 11:38:54 crc kubenswrapper[5052]: I1203 11:38:54.967563 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 11:38:55 crc kubenswrapper[5052]: E1203 11:38:55.710214 5052 log.go:32] "PullImage from image service failed" err="rpc error: code = Canceled desc = copying config: context canceled" image="quay.io/podified-antelope-centos9/openstack-ceilometer-central@sha256:43a24796dabde68270dbfefa107205e173fdd6a0dc701502858cadbede69da31" Dec 03 11:38:55 crc kubenswrapper[5052]: E1203 11:38:55.710795 5052 kuberuntime_manager.go:1274] "Unhandled Error" err="container &Container{Name:ceilometer-central-agent,Image:quay.io/podified-antelope-centos9/openstack-ceilometer-central@sha256:43a24796dabde68270dbfefa107205e173fdd6a0dc701502858cadbede69da31,Command:[/bin/bash],Args:[-c /usr/local/bin/kolla_start],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{EnvVar{Name:CONFIG_HASH,Value:n5d8h8bh584h569hfch649h8ch647h5hc8h5fh6h688h5b5h5d5h597h558hch5dh54ch654h649h5c7h5ffh69h566h655hb9h9dhd9h98h66fq,ValueFrom:nil,},EnvVar{Name:KOLLA_CONFIG_STRATEGY,Value:COPY_ALWAYS,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:scripts,ReadOnly:true,MountPath:/var/lib/openstack/bin,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:config-data,ReadOnly:true,MountPath:/var/lib/openstack/config,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:config-data,ReadOnly:true,MountPath:/var/lib/kolla/config_files/config.json,SubPath:ceilometer-central-config.json,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:combined-ca-bundle,ReadOnly:true,MountPath:/etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem,SubPath:tls-ca-bundle.pem,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-h8zbz,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:&ExecAction{Command:[/usr/bin/python3 /var/lib/openstack/bin/centralhealth.py],},HTTPGet:nil,TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:300,TimeoutSeconds:5,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[MKNOD],},Privileged:nil,SELinuxOptions:nil,RunAsUser:nil,RunAsNonRoot:nil,ReadOnlyRootFilesystem:nil,AllowPrivilegeEscalation:nil,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod ceilometer-0_openstack-kuttl-tests(c43f59f2-deb7-46ee-8e07-54e3ff9f59aa): ErrImagePull: rpc error: code = Canceled desc = copying config: context canceled" logger="UnhandledError" Dec 03 11:38:59 crc kubenswrapper[5052]: I1203 11:38:59.379198 5052 patch_prober.go:28] interesting pod/machine-config-daemon-74bj6 container/machine-config-daemon namespace/openshift-machine-config-operator: Liveness probe status=failure output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" start-of-body= Dec 03 11:38:59 crc kubenswrapper[5052]: I1203 11:38:59.380110 5052 prober.go:107] "Probe failed" probeType="Liveness" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" containerName="machine-config-daemon" probeResult="failure" output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" Dec 03 11:39:03 crc kubenswrapper[5052]: E1203 11:39:03.900109 5052 log.go:32] "PullImage from image service failed" err="rpc error: code = Canceled desc = copying config: context canceled" image="quay.io/podified-antelope-centos9/openstack-cinder-api@sha256:7a2056615520e272bae43ec3f34e2ba7a92c1d364b8d9106b53bd694619fc9c2" Dec 03 11:39:03 crc kubenswrapper[5052]: E1203 11:39:03.900659 5052 kuberuntime_manager.go:1274] "Unhandled Error" err="container &Container{Name:cinder-db-sync,Image:quay.io/podified-antelope-centos9/openstack-cinder-api@sha256:7a2056615520e272bae43ec3f34e2ba7a92c1d364b8d9106b53bd694619fc9c2,Command:[/bin/bash],Args:[-c /usr/local/bin/kolla_set_configs && /usr/local/bin/kolla_start],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{EnvVar{Name:KOLLA_BOOTSTRAP,Value:TRUE,ValueFrom:nil,},EnvVar{Name:KOLLA_CONFIG_STRATEGY,Value:COPY_ALWAYS,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:etc-machine-id,ReadOnly:true,MountPath:/etc/machine-id,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:scripts,ReadOnly:true,MountPath:/usr/local/bin/container-scripts,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:config-data,ReadOnly:true,MountPath:/var/lib/config-data/merged,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:config-data,ReadOnly:true,MountPath:/etc/my.cnf,SubPath:my.cnf,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:db-sync-config-data,ReadOnly:true,MountPath:/etc/cinder/cinder.conf.d,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:config-data,ReadOnly:true,MountPath:/var/lib/kolla/config_files/config.json,SubPath:db-sync-config.json,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:combined-ca-bundle,ReadOnly:true,MountPath:/etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem,SubPath:tls-ca-bundle.pem,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-8lqcr,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:&SecurityContext{Capabilities:nil,Privileged:nil,SELinuxOptions:nil,RunAsUser:*0,RunAsNonRoot:nil,ReadOnlyRootFilesystem:nil,AllowPrivilegeEscalation:nil,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod cinder-db-sync-pbw4k_openstack-kuttl-tests(d1676789-356e-4853-a302-f2c5bd3f446f): ErrImagePull: rpc error: code = Canceled desc = copying config: context canceled" logger="UnhandledError" Dec 03 11:39:03 crc kubenswrapper[5052]: E1203 11:39:03.901894 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cinder-db-sync\" with ErrImagePull: \"rpc error: code = Canceled desc = copying config: context canceled\"" pod="openstack-kuttl-tests/cinder-db-sync-pbw4k" podUID="d1676789-356e-4853-a302-f2c5bd3f446f" Dec 03 11:39:03 crc kubenswrapper[5052]: I1203 11:39:03.930163 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-bootstrap-fstdw" Dec 03 11:39:04 crc kubenswrapper[5052]: I1203 11:39:04.075571 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/e1e49fa1-2ef1-4f43-a0b2-f98c1364bdcb-fernet-keys\") pod \"e1e49fa1-2ef1-4f43-a0b2-f98c1364bdcb\" (UID: \"e1e49fa1-2ef1-4f43-a0b2-f98c1364bdcb\") " Dec 03 11:39:04 crc kubenswrapper[5052]: I1203 11:39:04.076087 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e1e49fa1-2ef1-4f43-a0b2-f98c1364bdcb-combined-ca-bundle\") pod \"e1e49fa1-2ef1-4f43-a0b2-f98c1364bdcb\" (UID: \"e1e49fa1-2ef1-4f43-a0b2-f98c1364bdcb\") " Dec 03 11:39:04 crc kubenswrapper[5052]: I1203 11:39:04.076123 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/e1e49fa1-2ef1-4f43-a0b2-f98c1364bdcb-credential-keys\") pod \"e1e49fa1-2ef1-4f43-a0b2-f98c1364bdcb\" (UID: \"e1e49fa1-2ef1-4f43-a0b2-f98c1364bdcb\") " Dec 03 11:39:04 crc kubenswrapper[5052]: I1203 11:39:04.076341 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sg8wv\" (UniqueName: \"kubernetes.io/projected/e1e49fa1-2ef1-4f43-a0b2-f98c1364bdcb-kube-api-access-sg8wv\") pod \"e1e49fa1-2ef1-4f43-a0b2-f98c1364bdcb\" (UID: \"e1e49fa1-2ef1-4f43-a0b2-f98c1364bdcb\") " Dec 03 11:39:04 crc kubenswrapper[5052]: I1203 11:39:04.076620 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/e1e49fa1-2ef1-4f43-a0b2-f98c1364bdcb-scripts\") pod \"e1e49fa1-2ef1-4f43-a0b2-f98c1364bdcb\" (UID: \"e1e49fa1-2ef1-4f43-a0b2-f98c1364bdcb\") " Dec 03 11:39:04 crc kubenswrapper[5052]: I1203 11:39:04.076681 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/e1e49fa1-2ef1-4f43-a0b2-f98c1364bdcb-config-data\") pod \"e1e49fa1-2ef1-4f43-a0b2-f98c1364bdcb\" (UID: \"e1e49fa1-2ef1-4f43-a0b2-f98c1364bdcb\") " Dec 03 11:39:04 crc kubenswrapper[5052]: I1203 11:39:04.081656 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e1e49fa1-2ef1-4f43-a0b2-f98c1364bdcb-fernet-keys" (OuterVolumeSpecName: "fernet-keys") pod "e1e49fa1-2ef1-4f43-a0b2-f98c1364bdcb" (UID: "e1e49fa1-2ef1-4f43-a0b2-f98c1364bdcb"). InnerVolumeSpecName "fernet-keys". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:39:04 crc kubenswrapper[5052]: I1203 11:39:04.081986 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e1e49fa1-2ef1-4f43-a0b2-f98c1364bdcb-scripts" (OuterVolumeSpecName: "scripts") pod "e1e49fa1-2ef1-4f43-a0b2-f98c1364bdcb" (UID: "e1e49fa1-2ef1-4f43-a0b2-f98c1364bdcb"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:39:04 crc kubenswrapper[5052]: I1203 11:39:04.082203 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e1e49fa1-2ef1-4f43-a0b2-f98c1364bdcb-kube-api-access-sg8wv" (OuterVolumeSpecName: "kube-api-access-sg8wv") pod "e1e49fa1-2ef1-4f43-a0b2-f98c1364bdcb" (UID: "e1e49fa1-2ef1-4f43-a0b2-f98c1364bdcb"). InnerVolumeSpecName "kube-api-access-sg8wv". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:39:04 crc kubenswrapper[5052]: I1203 11:39:04.104695 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e1e49fa1-2ef1-4f43-a0b2-f98c1364bdcb-config-data" (OuterVolumeSpecName: "config-data") pod "e1e49fa1-2ef1-4f43-a0b2-f98c1364bdcb" (UID: "e1e49fa1-2ef1-4f43-a0b2-f98c1364bdcb"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:39:04 crc kubenswrapper[5052]: I1203 11:39:04.106938 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e1e49fa1-2ef1-4f43-a0b2-f98c1364bdcb-credential-keys" (OuterVolumeSpecName: "credential-keys") pod "e1e49fa1-2ef1-4f43-a0b2-f98c1364bdcb" (UID: "e1e49fa1-2ef1-4f43-a0b2-f98c1364bdcb"). InnerVolumeSpecName "credential-keys". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:39:04 crc kubenswrapper[5052]: I1203 11:39:04.109214 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e1e49fa1-2ef1-4f43-a0b2-f98c1364bdcb-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "e1e49fa1-2ef1-4f43-a0b2-f98c1364bdcb" (UID: "e1e49fa1-2ef1-4f43-a0b2-f98c1364bdcb"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:39:04 crc kubenswrapper[5052]: I1203 11:39:04.167416 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-bootstrap-fstdw" Dec 03 11:39:04 crc kubenswrapper[5052]: I1203 11:39:04.167405 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-bootstrap-fstdw" event={"ID":"e1e49fa1-2ef1-4f43-a0b2-f98c1364bdcb","Type":"ContainerDied","Data":"66bd9df9253dda0cb16759f9020973dc876018932d166cda76b8d5df39a0e91c"} Dec 03 11:39:04 crc kubenswrapper[5052]: I1203 11:39:04.167620 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="66bd9df9253dda0cb16759f9020973dc876018932d166cda76b8d5df39a0e91c" Dec 03 11:39:04 crc kubenswrapper[5052]: E1203 11:39:04.169295 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cinder-db-sync\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/podified-antelope-centos9/openstack-cinder-api@sha256:7a2056615520e272bae43ec3f34e2ba7a92c1d364b8d9106b53bd694619fc9c2\\\"\"" pod="openstack-kuttl-tests/cinder-db-sync-pbw4k" podUID="d1676789-356e-4853-a302-f2c5bd3f446f" Dec 03 11:39:04 crc kubenswrapper[5052]: I1203 11:39:04.178846 5052 reconciler_common.go:293] "Volume detached for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/e1e49fa1-2ef1-4f43-a0b2-f98c1364bdcb-fernet-keys\") on node \"crc\" DevicePath \"\"" Dec 03 11:39:04 crc kubenswrapper[5052]: I1203 11:39:04.179107 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e1e49fa1-2ef1-4f43-a0b2-f98c1364bdcb-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:39:04 crc kubenswrapper[5052]: I1203 11:39:04.179219 5052 reconciler_common.go:293] "Volume detached for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/e1e49fa1-2ef1-4f43-a0b2-f98c1364bdcb-credential-keys\") on node \"crc\" DevicePath \"\"" Dec 03 11:39:04 crc kubenswrapper[5052]: I1203 11:39:04.179359 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-sg8wv\" (UniqueName: \"kubernetes.io/projected/e1e49fa1-2ef1-4f43-a0b2-f98c1364bdcb-kube-api-access-sg8wv\") on node \"crc\" DevicePath \"\"" Dec 03 11:39:04 crc kubenswrapper[5052]: I1203 11:39:04.179449 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/e1e49fa1-2ef1-4f43-a0b2-f98c1364bdcb-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:39:04 crc kubenswrapper[5052]: I1203 11:39:04.179555 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/e1e49fa1-2ef1-4f43-a0b2-f98c1364bdcb-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:39:04 crc kubenswrapper[5052]: E1203 11:39:04.416520 5052 log.go:32] "PullImage from image service failed" err="rpc error: code = Canceled desc = copying config: context canceled" image="quay.io/podified-antelope-centos9/openstack-barbican-api@sha256:82006b9c64d4c5f80483cda262d960ce6be4813665158ef1a53ea7734bbe431f" Dec 03 11:39:04 crc kubenswrapper[5052]: E1203 11:39:04.416698 5052 kuberuntime_manager.go:1274] "Unhandled Error" err="container &Container{Name:barbican-db-sync,Image:quay.io/podified-antelope-centos9/openstack-barbican-api@sha256:82006b9c64d4c5f80483cda262d960ce6be4813665158ef1a53ea7734bbe431f,Command:[/bin/bash],Args:[-c barbican-manage db upgrade],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{EnvVar{Name:KOLLA_BOOTSTRAP,Value:TRUE,ValueFrom:nil,},EnvVar{Name:KOLLA_CONFIG_STRATEGY,Value:COPY_ALWAYS,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:db-sync-config-data,ReadOnly:true,MountPath:/etc/barbican/barbican.conf.d,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:combined-ca-bundle,ReadOnly:true,MountPath:/etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem,SubPath:tls-ca-bundle.pem,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-zp4pf,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[MKNOD],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*42403,RunAsNonRoot:*true,ReadOnlyRootFilesystem:nil,AllowPrivilegeEscalation:nil,RunAsGroup:*42403,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod barbican-db-sync-bfv74_openstack-kuttl-tests(1dc19382-24a8-4a4f-b735-d794619492bf): ErrImagePull: rpc error: code = Canceled desc = copying config: context canceled" logger="UnhandledError" Dec 03 11:39:04 crc kubenswrapper[5052]: E1203 11:39:04.418103 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"barbican-db-sync\" with ErrImagePull: \"rpc error: code = Canceled desc = copying config: context canceled\"" pod="openstack-kuttl-tests/barbican-db-sync-bfv74" podUID="1dc19382-24a8-4a4f-b735-d794619492bf" Dec 03 11:39:04 crc kubenswrapper[5052]: I1203 11:39:04.870406 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 11:39:05 crc kubenswrapper[5052]: I1203 11:39:05.015724 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/keystone-bootstrap-fstdw"] Dec 03 11:39:05 crc kubenswrapper[5052]: I1203 11:39:05.031697 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/keystone-bootstrap-fstdw"] Dec 03 11:39:05 crc kubenswrapper[5052]: I1203 11:39:05.129364 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/keystone-bootstrap-v8g6k"] Dec 03 11:39:05 crc kubenswrapper[5052]: E1203 11:39:05.129897 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="e1e49fa1-2ef1-4f43-a0b2-f98c1364bdcb" containerName="keystone-bootstrap" Dec 03 11:39:05 crc kubenswrapper[5052]: I1203 11:39:05.129921 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="e1e49fa1-2ef1-4f43-a0b2-f98c1364bdcb" containerName="keystone-bootstrap" Dec 03 11:39:05 crc kubenswrapper[5052]: I1203 11:39:05.130162 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="e1e49fa1-2ef1-4f43-a0b2-f98c1364bdcb" containerName="keystone-bootstrap" Dec 03 11:39:05 crc kubenswrapper[5052]: I1203 11:39:05.130943 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-bootstrap-v8g6k" Dec 03 11:39:05 crc kubenswrapper[5052]: I1203 11:39:05.135965 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"osp-secret" Dec 03 11:39:05 crc kubenswrapper[5052]: I1203 11:39:05.136059 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone" Dec 03 11:39:05 crc kubenswrapper[5052]: I1203 11:39:05.135965 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone-scripts" Dec 03 11:39:05 crc kubenswrapper[5052]: I1203 11:39:05.136339 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone-config-data" Dec 03 11:39:05 crc kubenswrapper[5052]: I1203 11:39:05.136455 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone-keystone-dockercfg-tt6lh" Dec 03 11:39:05 crc kubenswrapper[5052]: I1203 11:39:05.165820 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-bootstrap-v8g6k"] Dec 03 11:39:05 crc kubenswrapper[5052]: I1203 11:39:05.191159 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-db-sync-qp7zq" event={"ID":"5ff261a9-6d56-45fe-88fa-7184d8a3b90b","Type":"ContainerStarted","Data":"68ef4a664deede083c7a3421bfaeed869fe2a0d81c8e476b2b9f92399d181fa1"} Dec 03 11:39:05 crc kubenswrapper[5052]: I1203 11:39:05.195493 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"c43f59f2-deb7-46ee-8e07-54e3ff9f59aa","Type":"ContainerStarted","Data":"740b963fb25833c1ae839f5909691da25e2203dbabc6e50ecaeefd885b130f22"} Dec 03 11:39:05 crc kubenswrapper[5052]: I1203 11:39:05.200708 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/2204f9d2-fb11-429c-808f-8fd46881102d-config-data\") pod \"keystone-bootstrap-v8g6k\" (UID: \"2204f9d2-fb11-429c-808f-8fd46881102d\") " pod="openstack-kuttl-tests/keystone-bootstrap-v8g6k" Dec 03 11:39:05 crc kubenswrapper[5052]: I1203 11:39:05.200791 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/2204f9d2-fb11-429c-808f-8fd46881102d-scripts\") pod \"keystone-bootstrap-v8g6k\" (UID: \"2204f9d2-fb11-429c-808f-8fd46881102d\") " pod="openstack-kuttl-tests/keystone-bootstrap-v8g6k" Dec 03 11:39:05 crc kubenswrapper[5052]: I1203 11:39:05.200850 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/2204f9d2-fb11-429c-808f-8fd46881102d-credential-keys\") pod \"keystone-bootstrap-v8g6k\" (UID: \"2204f9d2-fb11-429c-808f-8fd46881102d\") " pod="openstack-kuttl-tests/keystone-bootstrap-v8g6k" Dec 03 11:39:05 crc kubenswrapper[5052]: I1203 11:39:05.200872 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/2204f9d2-fb11-429c-808f-8fd46881102d-fernet-keys\") pod \"keystone-bootstrap-v8g6k\" (UID: \"2204f9d2-fb11-429c-808f-8fd46881102d\") " pod="openstack-kuttl-tests/keystone-bootstrap-v8g6k" Dec 03 11:39:05 crc kubenswrapper[5052]: I1203 11:39:05.200923 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cxjgj\" (UniqueName: \"kubernetes.io/projected/2204f9d2-fb11-429c-808f-8fd46881102d-kube-api-access-cxjgj\") pod \"keystone-bootstrap-v8g6k\" (UID: \"2204f9d2-fb11-429c-808f-8fd46881102d\") " pod="openstack-kuttl-tests/keystone-bootstrap-v8g6k" Dec 03 11:39:05 crc kubenswrapper[5052]: I1203 11:39:05.200959 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2204f9d2-fb11-429c-808f-8fd46881102d-combined-ca-bundle\") pod \"keystone-bootstrap-v8g6k\" (UID: \"2204f9d2-fb11-429c-808f-8fd46881102d\") " pod="openstack-kuttl-tests/keystone-bootstrap-v8g6k" Dec 03 11:39:05 crc kubenswrapper[5052]: I1203 11:39:05.202213 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"de570c88-14c0-4761-be7f-da7a6d97715a","Type":"ContainerStarted","Data":"6ed2ca0ce999fbd725336cbb1b7fb4853d5da9819e56d0268e19686ba90e7c0a"} Dec 03 11:39:05 crc kubenswrapper[5052]: I1203 11:39:05.202268 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 11:39:05 crc kubenswrapper[5052]: E1203 11:39:05.205354 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"barbican-db-sync\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/podified-antelope-centos9/openstack-barbican-api@sha256:82006b9c64d4c5f80483cda262d960ce6be4813665158ef1a53ea7734bbe431f\\\"\"" pod="openstack-kuttl-tests/barbican-db-sync-bfv74" podUID="1dc19382-24a8-4a4f-b735-d794619492bf" Dec 03 11:39:05 crc kubenswrapper[5052]: I1203 11:39:05.220856 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/placement-db-sync-qp7zq" podStartSLOduration=2.336195238 podStartE2EDuration="23.220839376s" podCreationTimestamp="2025-12-03 11:38:42 +0000 UTC" firstStartedPulling="2025-12-03 11:38:43.553135173 +0000 UTC m=+1114.141376937" lastFinishedPulling="2025-12-03 11:39:04.437779311 +0000 UTC m=+1135.026021075" observedRunningTime="2025-12-03 11:39:05.214766911 +0000 UTC m=+1135.803008675" watchObservedRunningTime="2025-12-03 11:39:05.220839376 +0000 UTC m=+1135.809081140" Dec 03 11:39:05 crc kubenswrapper[5052]: W1203 11:39:05.221176 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3a98a307_4671_43f7_86af_3b86591c8d7e.slice/crio-fddeb452fffb834f16afc576695523b4c5db19742e70223d4d376caa66f04f6a WatchSource:0}: Error finding container fddeb452fffb834f16afc576695523b4c5db19742e70223d4d376caa66f04f6a: Status 404 returned error can't find the container with id fddeb452fffb834f16afc576695523b4c5db19742e70223d4d376caa66f04f6a Dec 03 11:39:05 crc kubenswrapper[5052]: I1203 11:39:05.302926 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/2204f9d2-fb11-429c-808f-8fd46881102d-credential-keys\") pod \"keystone-bootstrap-v8g6k\" (UID: \"2204f9d2-fb11-429c-808f-8fd46881102d\") " pod="openstack-kuttl-tests/keystone-bootstrap-v8g6k" Dec 03 11:39:05 crc kubenswrapper[5052]: I1203 11:39:05.302975 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/2204f9d2-fb11-429c-808f-8fd46881102d-fernet-keys\") pod \"keystone-bootstrap-v8g6k\" (UID: \"2204f9d2-fb11-429c-808f-8fd46881102d\") " pod="openstack-kuttl-tests/keystone-bootstrap-v8g6k" Dec 03 11:39:05 crc kubenswrapper[5052]: I1203 11:39:05.303038 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-cxjgj\" (UniqueName: \"kubernetes.io/projected/2204f9d2-fb11-429c-808f-8fd46881102d-kube-api-access-cxjgj\") pod \"keystone-bootstrap-v8g6k\" (UID: \"2204f9d2-fb11-429c-808f-8fd46881102d\") " pod="openstack-kuttl-tests/keystone-bootstrap-v8g6k" Dec 03 11:39:05 crc kubenswrapper[5052]: I1203 11:39:05.303063 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2204f9d2-fb11-429c-808f-8fd46881102d-combined-ca-bundle\") pod \"keystone-bootstrap-v8g6k\" (UID: \"2204f9d2-fb11-429c-808f-8fd46881102d\") " pod="openstack-kuttl-tests/keystone-bootstrap-v8g6k" Dec 03 11:39:05 crc kubenswrapper[5052]: I1203 11:39:05.303126 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/2204f9d2-fb11-429c-808f-8fd46881102d-config-data\") pod \"keystone-bootstrap-v8g6k\" (UID: \"2204f9d2-fb11-429c-808f-8fd46881102d\") " pod="openstack-kuttl-tests/keystone-bootstrap-v8g6k" Dec 03 11:39:05 crc kubenswrapper[5052]: I1203 11:39:05.303168 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/2204f9d2-fb11-429c-808f-8fd46881102d-scripts\") pod \"keystone-bootstrap-v8g6k\" (UID: \"2204f9d2-fb11-429c-808f-8fd46881102d\") " pod="openstack-kuttl-tests/keystone-bootstrap-v8g6k" Dec 03 11:39:05 crc kubenswrapper[5052]: I1203 11:39:05.308997 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2204f9d2-fb11-429c-808f-8fd46881102d-combined-ca-bundle\") pod \"keystone-bootstrap-v8g6k\" (UID: \"2204f9d2-fb11-429c-808f-8fd46881102d\") " pod="openstack-kuttl-tests/keystone-bootstrap-v8g6k" Dec 03 11:39:05 crc kubenswrapper[5052]: I1203 11:39:05.310633 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/2204f9d2-fb11-429c-808f-8fd46881102d-scripts\") pod \"keystone-bootstrap-v8g6k\" (UID: \"2204f9d2-fb11-429c-808f-8fd46881102d\") " pod="openstack-kuttl-tests/keystone-bootstrap-v8g6k" Dec 03 11:39:05 crc kubenswrapper[5052]: I1203 11:39:05.311055 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/2204f9d2-fb11-429c-808f-8fd46881102d-credential-keys\") pod \"keystone-bootstrap-v8g6k\" (UID: \"2204f9d2-fb11-429c-808f-8fd46881102d\") " pod="openstack-kuttl-tests/keystone-bootstrap-v8g6k" Dec 03 11:39:05 crc kubenswrapper[5052]: I1203 11:39:05.313303 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/2204f9d2-fb11-429c-808f-8fd46881102d-config-data\") pod \"keystone-bootstrap-v8g6k\" (UID: \"2204f9d2-fb11-429c-808f-8fd46881102d\") " pod="openstack-kuttl-tests/keystone-bootstrap-v8g6k" Dec 03 11:39:05 crc kubenswrapper[5052]: I1203 11:39:05.314081 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/2204f9d2-fb11-429c-808f-8fd46881102d-fernet-keys\") pod \"keystone-bootstrap-v8g6k\" (UID: \"2204f9d2-fb11-429c-808f-8fd46881102d\") " pod="openstack-kuttl-tests/keystone-bootstrap-v8g6k" Dec 03 11:39:05 crc kubenswrapper[5052]: I1203 11:39:05.329137 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-cxjgj\" (UniqueName: \"kubernetes.io/projected/2204f9d2-fb11-429c-808f-8fd46881102d-kube-api-access-cxjgj\") pod \"keystone-bootstrap-v8g6k\" (UID: \"2204f9d2-fb11-429c-808f-8fd46881102d\") " pod="openstack-kuttl-tests/keystone-bootstrap-v8g6k" Dec 03 11:39:05 crc kubenswrapper[5052]: I1203 11:39:05.464075 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-bootstrap-v8g6k" Dec 03 11:39:05 crc kubenswrapper[5052]: I1203 11:39:05.698376 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e1e49fa1-2ef1-4f43-a0b2-f98c1364bdcb" path="/var/lib/kubelet/pods/e1e49fa1-2ef1-4f43-a0b2-f98c1364bdcb/volumes" Dec 03 11:39:06 crc kubenswrapper[5052]: I1203 11:39:06.013337 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-bootstrap-v8g6k"] Dec 03 11:39:06 crc kubenswrapper[5052]: I1203 11:39:06.257973 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-bootstrap-v8g6k" event={"ID":"2204f9d2-fb11-429c-808f-8fd46881102d","Type":"ContainerStarted","Data":"17810cee57aa4fdda97fb15709dcf7901c9df4796fb99645a79039b518a51059"} Dec 03 11:39:06 crc kubenswrapper[5052]: I1203 11:39:06.288377 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"3a98a307-4671-43f7-86af-3b86591c8d7e","Type":"ContainerStarted","Data":"d028cc806e6ca374c34c8d04e35b3b1f8f7c0519bd4bbb5bebc699c00aaa1e53"} Dec 03 11:39:06 crc kubenswrapper[5052]: I1203 11:39:06.288431 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"3a98a307-4671-43f7-86af-3b86591c8d7e","Type":"ContainerStarted","Data":"fddeb452fffb834f16afc576695523b4c5db19742e70223d4d376caa66f04f6a"} Dec 03 11:39:06 crc kubenswrapper[5052]: I1203 11:39:06.292867 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"de570c88-14c0-4761-be7f-da7a6d97715a","Type":"ContainerStarted","Data":"1e36542bcba777c859a2d3c7c005a13275c8b488c65a6401477cc95235b53c39"} Dec 03 11:39:07 crc kubenswrapper[5052]: I1203 11:39:07.307140 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"3a98a307-4671-43f7-86af-3b86591c8d7e","Type":"ContainerStarted","Data":"e63b1e0b518c228170b13685174d8fa52d0df2ab0d5dc71d4ad114ea64d71ae4"} Dec 03 11:39:07 crc kubenswrapper[5052]: I1203 11:39:07.307180 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-internal-api-0" podUID="3a98a307-4671-43f7-86af-3b86591c8d7e" containerName="glance-log" containerID="cri-o://d028cc806e6ca374c34c8d04e35b3b1f8f7c0519bd4bbb5bebc699c00aaa1e53" gracePeriod=30 Dec 03 11:39:07 crc kubenswrapper[5052]: I1203 11:39:07.307260 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-internal-api-0" podUID="3a98a307-4671-43f7-86af-3b86591c8d7e" containerName="glance-httpd" containerID="cri-o://e63b1e0b518c228170b13685174d8fa52d0df2ab0d5dc71d4ad114ea64d71ae4" gracePeriod=30 Dec 03 11:39:07 crc kubenswrapper[5052]: I1203 11:39:07.313283 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"de570c88-14c0-4761-be7f-da7a6d97715a","Type":"ContainerStarted","Data":"c9111aa5f09ee7dfc464783e5e12845d19057e42ac17492647a8779a1d7018ce"} Dec 03 11:39:07 crc kubenswrapper[5052]: I1203 11:39:07.313638 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-external-api-0" podUID="de570c88-14c0-4761-be7f-da7a6d97715a" containerName="glance-log" containerID="cri-o://1e36542bcba777c859a2d3c7c005a13275c8b488c65a6401477cc95235b53c39" gracePeriod=30 Dec 03 11:39:07 crc kubenswrapper[5052]: I1203 11:39:07.313721 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-external-api-0" podUID="de570c88-14c0-4761-be7f-da7a6d97715a" containerName="glance-httpd" containerID="cri-o://c9111aa5f09ee7dfc464783e5e12845d19057e42ac17492647a8779a1d7018ce" gracePeriod=30 Dec 03 11:39:07 crc kubenswrapper[5052]: I1203 11:39:07.315663 5052 generic.go:334] "Generic (PLEG): container finished" podID="aae8de3e-7ad6-4a25-bdd5-3f98730ae227" containerID="b976664871cebff06450133c9273e24bbc1c92f6e86458b6364c4f2e4f2e4d6f" exitCode=0 Dec 03 11:39:07 crc kubenswrapper[5052]: I1203 11:39:07.315832 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-db-sync-j7t4l" event={"ID":"aae8de3e-7ad6-4a25-bdd5-3f98730ae227","Type":"ContainerDied","Data":"b976664871cebff06450133c9273e24bbc1c92f6e86458b6364c4f2e4f2e4d6f"} Dec 03 11:39:07 crc kubenswrapper[5052]: I1203 11:39:07.318871 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-bootstrap-v8g6k" event={"ID":"2204f9d2-fb11-429c-808f-8fd46881102d","Type":"ContainerStarted","Data":"c6d817a30b2e7094442dd66f61af96f6badd467a3b61dd19427f47351547a72d"} Dec 03 11:39:07 crc kubenswrapper[5052]: I1203 11:39:07.342575 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/glance-default-internal-api-0" podStartSLOduration=15.342529554 podStartE2EDuration="15.342529554s" podCreationTimestamp="2025-12-03 11:38:52 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:39:07.334549834 +0000 UTC m=+1137.922791598" watchObservedRunningTime="2025-12-03 11:39:07.342529554 +0000 UTC m=+1137.930771318" Dec 03 11:39:07 crc kubenswrapper[5052]: I1203 11:39:07.367913 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/keystone-bootstrap-v8g6k" podStartSLOduration=2.367894787 podStartE2EDuration="2.367894787s" podCreationTimestamp="2025-12-03 11:39:05 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:39:07.366389163 +0000 UTC m=+1137.954630937" watchObservedRunningTime="2025-12-03 11:39:07.367894787 +0000 UTC m=+1137.956136551" Dec 03 11:39:07 crc kubenswrapper[5052]: I1203 11:39:07.402357 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/glance-default-external-api-0" podStartSLOduration=15.402331971 podStartE2EDuration="15.402331971s" podCreationTimestamp="2025-12-03 11:38:52 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:39:07.393213738 +0000 UTC m=+1137.981455502" watchObservedRunningTime="2025-12-03 11:39:07.402331971 +0000 UTC m=+1137.990573735" Dec 03 11:39:08 crc kubenswrapper[5052]: I1203 11:39:08.328230 5052 generic.go:334] "Generic (PLEG): container finished" podID="3a98a307-4671-43f7-86af-3b86591c8d7e" containerID="e63b1e0b518c228170b13685174d8fa52d0df2ab0d5dc71d4ad114ea64d71ae4" exitCode=0 Dec 03 11:39:08 crc kubenswrapper[5052]: I1203 11:39:08.328271 5052 generic.go:334] "Generic (PLEG): container finished" podID="3a98a307-4671-43f7-86af-3b86591c8d7e" containerID="d028cc806e6ca374c34c8d04e35b3b1f8f7c0519bd4bbb5bebc699c00aaa1e53" exitCode=143 Dec 03 11:39:08 crc kubenswrapper[5052]: I1203 11:39:08.328304 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"3a98a307-4671-43f7-86af-3b86591c8d7e","Type":"ContainerDied","Data":"e63b1e0b518c228170b13685174d8fa52d0df2ab0d5dc71d4ad114ea64d71ae4"} Dec 03 11:39:08 crc kubenswrapper[5052]: I1203 11:39:08.328349 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"3a98a307-4671-43f7-86af-3b86591c8d7e","Type":"ContainerDied","Data":"d028cc806e6ca374c34c8d04e35b3b1f8f7c0519bd4bbb5bebc699c00aaa1e53"} Dec 03 11:39:08 crc kubenswrapper[5052]: I1203 11:39:08.329899 5052 generic.go:334] "Generic (PLEG): container finished" podID="de570c88-14c0-4761-be7f-da7a6d97715a" containerID="c9111aa5f09ee7dfc464783e5e12845d19057e42ac17492647a8779a1d7018ce" exitCode=0 Dec 03 11:39:08 crc kubenswrapper[5052]: I1203 11:39:08.329920 5052 generic.go:334] "Generic (PLEG): container finished" podID="de570c88-14c0-4761-be7f-da7a6d97715a" containerID="1e36542bcba777c859a2d3c7c005a13275c8b488c65a6401477cc95235b53c39" exitCode=143 Dec 03 11:39:08 crc kubenswrapper[5052]: I1203 11:39:08.329952 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"de570c88-14c0-4761-be7f-da7a6d97715a","Type":"ContainerDied","Data":"c9111aa5f09ee7dfc464783e5e12845d19057e42ac17492647a8779a1d7018ce"} Dec 03 11:39:08 crc kubenswrapper[5052]: I1203 11:39:08.329966 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"de570c88-14c0-4761-be7f-da7a6d97715a","Type":"ContainerDied","Data":"1e36542bcba777c859a2d3c7c005a13275c8b488c65a6401477cc95235b53c39"} Dec 03 11:39:08 crc kubenswrapper[5052]: I1203 11:39:08.331910 5052 generic.go:334] "Generic (PLEG): container finished" podID="5ff261a9-6d56-45fe-88fa-7184d8a3b90b" containerID="68ef4a664deede083c7a3421bfaeed869fe2a0d81c8e476b2b9f92399d181fa1" exitCode=0 Dec 03 11:39:08 crc kubenswrapper[5052]: I1203 11:39:08.332677 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-db-sync-qp7zq" event={"ID":"5ff261a9-6d56-45fe-88fa-7184d8a3b90b","Type":"ContainerDied","Data":"68ef4a664deede083c7a3421bfaeed869fe2a0d81c8e476b2b9f92399d181fa1"} Dec 03 11:39:10 crc kubenswrapper[5052]: I1203 11:39:10.011113 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-db-sync-qp7zq" Dec 03 11:39:10 crc kubenswrapper[5052]: I1203 11:39:10.085813 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-db-sync-j7t4l" Dec 03 11:39:10 crc kubenswrapper[5052]: I1203 11:39:10.110129 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/5ff261a9-6d56-45fe-88fa-7184d8a3b90b-config-data\") pod \"5ff261a9-6d56-45fe-88fa-7184d8a3b90b\" (UID: \"5ff261a9-6d56-45fe-88fa-7184d8a3b90b\") " Dec 03 11:39:10 crc kubenswrapper[5052]: I1203 11:39:10.110178 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/5ff261a9-6d56-45fe-88fa-7184d8a3b90b-logs\") pod \"5ff261a9-6d56-45fe-88fa-7184d8a3b90b\" (UID: \"5ff261a9-6d56-45fe-88fa-7184d8a3b90b\") " Dec 03 11:39:10 crc kubenswrapper[5052]: I1203 11:39:10.110313 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/5ff261a9-6d56-45fe-88fa-7184d8a3b90b-scripts\") pod \"5ff261a9-6d56-45fe-88fa-7184d8a3b90b\" (UID: \"5ff261a9-6d56-45fe-88fa-7184d8a3b90b\") " Dec 03 11:39:10 crc kubenswrapper[5052]: I1203 11:39:10.110361 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5ff261a9-6d56-45fe-88fa-7184d8a3b90b-combined-ca-bundle\") pod \"5ff261a9-6d56-45fe-88fa-7184d8a3b90b\" (UID: \"5ff261a9-6d56-45fe-88fa-7184d8a3b90b\") " Dec 03 11:39:10 crc kubenswrapper[5052]: I1203 11:39:10.110387 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l6jzd\" (UniqueName: \"kubernetes.io/projected/5ff261a9-6d56-45fe-88fa-7184d8a3b90b-kube-api-access-l6jzd\") pod \"5ff261a9-6d56-45fe-88fa-7184d8a3b90b\" (UID: \"5ff261a9-6d56-45fe-88fa-7184d8a3b90b\") " Dec 03 11:39:10 crc kubenswrapper[5052]: I1203 11:39:10.110895 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5ff261a9-6d56-45fe-88fa-7184d8a3b90b-logs" (OuterVolumeSpecName: "logs") pod "5ff261a9-6d56-45fe-88fa-7184d8a3b90b" (UID: "5ff261a9-6d56-45fe-88fa-7184d8a3b90b"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:39:10 crc kubenswrapper[5052]: I1203 11:39:10.111597 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/5ff261a9-6d56-45fe-88fa-7184d8a3b90b-logs\") on node \"crc\" DevicePath \"\"" Dec 03 11:39:10 crc kubenswrapper[5052]: I1203 11:39:10.117533 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5ff261a9-6d56-45fe-88fa-7184d8a3b90b-scripts" (OuterVolumeSpecName: "scripts") pod "5ff261a9-6d56-45fe-88fa-7184d8a3b90b" (UID: "5ff261a9-6d56-45fe-88fa-7184d8a3b90b"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:39:10 crc kubenswrapper[5052]: I1203 11:39:10.117734 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5ff261a9-6d56-45fe-88fa-7184d8a3b90b-kube-api-access-l6jzd" (OuterVolumeSpecName: "kube-api-access-l6jzd") pod "5ff261a9-6d56-45fe-88fa-7184d8a3b90b" (UID: "5ff261a9-6d56-45fe-88fa-7184d8a3b90b"). InnerVolumeSpecName "kube-api-access-l6jzd". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:39:10 crc kubenswrapper[5052]: I1203 11:39:10.138992 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5ff261a9-6d56-45fe-88fa-7184d8a3b90b-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "5ff261a9-6d56-45fe-88fa-7184d8a3b90b" (UID: "5ff261a9-6d56-45fe-88fa-7184d8a3b90b"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:39:10 crc kubenswrapper[5052]: I1203 11:39:10.162686 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5ff261a9-6d56-45fe-88fa-7184d8a3b90b-config-data" (OuterVolumeSpecName: "config-data") pod "5ff261a9-6d56-45fe-88fa-7184d8a3b90b" (UID: "5ff261a9-6d56-45fe-88fa-7184d8a3b90b"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:39:10 crc kubenswrapper[5052]: I1203 11:39:10.193583 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:39:10 crc kubenswrapper[5052]: I1203 11:39:10.213007 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/aae8de3e-7ad6-4a25-bdd5-3f98730ae227-combined-ca-bundle\") pod \"aae8de3e-7ad6-4a25-bdd5-3f98730ae227\" (UID: \"aae8de3e-7ad6-4a25-bdd5-3f98730ae227\") " Dec 03 11:39:10 crc kubenswrapper[5052]: I1203 11:39:10.213075 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xrvc5\" (UniqueName: \"kubernetes.io/projected/aae8de3e-7ad6-4a25-bdd5-3f98730ae227-kube-api-access-xrvc5\") pod \"aae8de3e-7ad6-4a25-bdd5-3f98730ae227\" (UID: \"aae8de3e-7ad6-4a25-bdd5-3f98730ae227\") " Dec 03 11:39:10 crc kubenswrapper[5052]: I1203 11:39:10.213167 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/aae8de3e-7ad6-4a25-bdd5-3f98730ae227-config\") pod \"aae8de3e-7ad6-4a25-bdd5-3f98730ae227\" (UID: \"aae8de3e-7ad6-4a25-bdd5-3f98730ae227\") " Dec 03 11:39:10 crc kubenswrapper[5052]: I1203 11:39:10.213741 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/5ff261a9-6d56-45fe-88fa-7184d8a3b90b-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:39:10 crc kubenswrapper[5052]: I1203 11:39:10.213761 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/5ff261a9-6d56-45fe-88fa-7184d8a3b90b-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:39:10 crc kubenswrapper[5052]: I1203 11:39:10.213772 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5ff261a9-6d56-45fe-88fa-7184d8a3b90b-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:39:10 crc kubenswrapper[5052]: I1203 11:39:10.213781 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-l6jzd\" (UniqueName: \"kubernetes.io/projected/5ff261a9-6d56-45fe-88fa-7184d8a3b90b-kube-api-access-l6jzd\") on node \"crc\" DevicePath \"\"" Dec 03 11:39:10 crc kubenswrapper[5052]: I1203 11:39:10.217608 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aae8de3e-7ad6-4a25-bdd5-3f98730ae227-kube-api-access-xrvc5" (OuterVolumeSpecName: "kube-api-access-xrvc5") pod "aae8de3e-7ad6-4a25-bdd5-3f98730ae227" (UID: "aae8de3e-7ad6-4a25-bdd5-3f98730ae227"). InnerVolumeSpecName "kube-api-access-xrvc5". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:39:10 crc kubenswrapper[5052]: I1203 11:39:10.237484 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aae8de3e-7ad6-4a25-bdd5-3f98730ae227-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "aae8de3e-7ad6-4a25-bdd5-3f98730ae227" (UID: "aae8de3e-7ad6-4a25-bdd5-3f98730ae227"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:39:10 crc kubenswrapper[5052]: I1203 11:39:10.255316 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aae8de3e-7ad6-4a25-bdd5-3f98730ae227-config" (OuterVolumeSpecName: "config") pod "aae8de3e-7ad6-4a25-bdd5-3f98730ae227" (UID: "aae8de3e-7ad6-4a25-bdd5-3f98730ae227"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:39:10 crc kubenswrapper[5052]: I1203 11:39:10.314860 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/3a98a307-4671-43f7-86af-3b86591c8d7e-logs\") pod \"3a98a307-4671-43f7-86af-3b86591c8d7e\" (UID: \"3a98a307-4671-43f7-86af-3b86591c8d7e\") " Dec 03 11:39:10 crc kubenswrapper[5052]: I1203 11:39:10.314910 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/3a98a307-4671-43f7-86af-3b86591c8d7e-config-data\") pod \"3a98a307-4671-43f7-86af-3b86591c8d7e\" (UID: \"3a98a307-4671-43f7-86af-3b86591c8d7e\") " Dec 03 11:39:10 crc kubenswrapper[5052]: I1203 11:39:10.314947 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3a98a307-4671-43f7-86af-3b86591c8d7e-combined-ca-bundle\") pod \"3a98a307-4671-43f7-86af-3b86591c8d7e\" (UID: \"3a98a307-4671-43f7-86af-3b86591c8d7e\") " Dec 03 11:39:10 crc kubenswrapper[5052]: I1203 11:39:10.314977 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/3a98a307-4671-43f7-86af-3b86591c8d7e-scripts\") pod \"3a98a307-4671-43f7-86af-3b86591c8d7e\" (UID: \"3a98a307-4671-43f7-86af-3b86591c8d7e\") " Dec 03 11:39:10 crc kubenswrapper[5052]: I1203 11:39:10.315006 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"glance\" (UniqueName: \"kubernetes.io/local-volume/local-storage04-crc\") pod \"3a98a307-4671-43f7-86af-3b86591c8d7e\" (UID: \"3a98a307-4671-43f7-86af-3b86591c8d7e\") " Dec 03 11:39:10 crc kubenswrapper[5052]: I1203 11:39:10.315055 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/3a98a307-4671-43f7-86af-3b86591c8d7e-httpd-run\") pod \"3a98a307-4671-43f7-86af-3b86591c8d7e\" (UID: \"3a98a307-4671-43f7-86af-3b86591c8d7e\") " Dec 03 11:39:10 crc kubenswrapper[5052]: I1203 11:39:10.315090 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jhsf4\" (UniqueName: \"kubernetes.io/projected/3a98a307-4671-43f7-86af-3b86591c8d7e-kube-api-access-jhsf4\") pod \"3a98a307-4671-43f7-86af-3b86591c8d7e\" (UID: \"3a98a307-4671-43f7-86af-3b86591c8d7e\") " Dec 03 11:39:10 crc kubenswrapper[5052]: I1203 11:39:10.315414 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/aae8de3e-7ad6-4a25-bdd5-3f98730ae227-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:39:10 crc kubenswrapper[5052]: I1203 11:39:10.315426 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-xrvc5\" (UniqueName: \"kubernetes.io/projected/aae8de3e-7ad6-4a25-bdd5-3f98730ae227-kube-api-access-xrvc5\") on node \"crc\" DevicePath \"\"" Dec 03 11:39:10 crc kubenswrapper[5052]: I1203 11:39:10.315438 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/aae8de3e-7ad6-4a25-bdd5-3f98730ae227-config\") on node \"crc\" DevicePath \"\"" Dec 03 11:39:10 crc kubenswrapper[5052]: I1203 11:39:10.315430 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3a98a307-4671-43f7-86af-3b86591c8d7e-logs" (OuterVolumeSpecName: "logs") pod "3a98a307-4671-43f7-86af-3b86591c8d7e" (UID: "3a98a307-4671-43f7-86af-3b86591c8d7e"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:39:10 crc kubenswrapper[5052]: I1203 11:39:10.315844 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3a98a307-4671-43f7-86af-3b86591c8d7e-httpd-run" (OuterVolumeSpecName: "httpd-run") pod "3a98a307-4671-43f7-86af-3b86591c8d7e" (UID: "3a98a307-4671-43f7-86af-3b86591c8d7e"). InnerVolumeSpecName "httpd-run". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:39:10 crc kubenswrapper[5052]: I1203 11:39:10.318854 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3a98a307-4671-43f7-86af-3b86591c8d7e-scripts" (OuterVolumeSpecName: "scripts") pod "3a98a307-4671-43f7-86af-3b86591c8d7e" (UID: "3a98a307-4671-43f7-86af-3b86591c8d7e"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:39:10 crc kubenswrapper[5052]: I1203 11:39:10.319033 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3a98a307-4671-43f7-86af-3b86591c8d7e-kube-api-access-jhsf4" (OuterVolumeSpecName: "kube-api-access-jhsf4") pod "3a98a307-4671-43f7-86af-3b86591c8d7e" (UID: "3a98a307-4671-43f7-86af-3b86591c8d7e"). InnerVolumeSpecName "kube-api-access-jhsf4". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:39:10 crc kubenswrapper[5052]: I1203 11:39:10.320697 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage04-crc" (OuterVolumeSpecName: "glance") pod "3a98a307-4671-43f7-86af-3b86591c8d7e" (UID: "3a98a307-4671-43f7-86af-3b86591c8d7e"). InnerVolumeSpecName "local-storage04-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 11:39:10 crc kubenswrapper[5052]: I1203 11:39:10.346766 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3a98a307-4671-43f7-86af-3b86591c8d7e-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "3a98a307-4671-43f7-86af-3b86591c8d7e" (UID: "3a98a307-4671-43f7-86af-3b86591c8d7e"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:39:10 crc kubenswrapper[5052]: I1203 11:39:10.349478 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"c43f59f2-deb7-46ee-8e07-54e3ff9f59aa","Type":"ContainerStarted","Data":"2eba4f00d60fae5fc479684e63a7bbddd03d0a038f2ae23cb78da9235c80d7d6"} Dec 03 11:39:10 crc kubenswrapper[5052]: I1203 11:39:10.350710 5052 generic.go:334] "Generic (PLEG): container finished" podID="2204f9d2-fb11-429c-808f-8fd46881102d" containerID="c6d817a30b2e7094442dd66f61af96f6badd467a3b61dd19427f47351547a72d" exitCode=0 Dec 03 11:39:10 crc kubenswrapper[5052]: I1203 11:39:10.350727 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-bootstrap-v8g6k" event={"ID":"2204f9d2-fb11-429c-808f-8fd46881102d","Type":"ContainerDied","Data":"c6d817a30b2e7094442dd66f61af96f6badd467a3b61dd19427f47351547a72d"} Dec 03 11:39:10 crc kubenswrapper[5052]: I1203 11:39:10.352494 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"3a98a307-4671-43f7-86af-3b86591c8d7e","Type":"ContainerDied","Data":"fddeb452fffb834f16afc576695523b4c5db19742e70223d4d376caa66f04f6a"} Dec 03 11:39:10 crc kubenswrapper[5052]: I1203 11:39:10.352588 5052 scope.go:117] "RemoveContainer" containerID="e63b1e0b518c228170b13685174d8fa52d0df2ab0d5dc71d4ad114ea64d71ae4" Dec 03 11:39:10 crc kubenswrapper[5052]: I1203 11:39:10.352810 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:39:10 crc kubenswrapper[5052]: I1203 11:39:10.355936 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-db-sync-j7t4l" event={"ID":"aae8de3e-7ad6-4a25-bdd5-3f98730ae227","Type":"ContainerDied","Data":"8ef02fc07bf5d93df6b03f41902ac447aba7de6e23dd011ce30455235ca91854"} Dec 03 11:39:10 crc kubenswrapper[5052]: I1203 11:39:10.355965 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8ef02fc07bf5d93df6b03f41902ac447aba7de6e23dd011ce30455235ca91854" Dec 03 11:39:10 crc kubenswrapper[5052]: I1203 11:39:10.356019 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-db-sync-j7t4l" Dec 03 11:39:10 crc kubenswrapper[5052]: I1203 11:39:10.364393 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-db-sync-qp7zq" event={"ID":"5ff261a9-6d56-45fe-88fa-7184d8a3b90b","Type":"ContainerDied","Data":"ff691e377bc65754a8824271a09fbe6041732211d46c452b77e403ba10dcf603"} Dec 03 11:39:10 crc kubenswrapper[5052]: I1203 11:39:10.364475 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ff691e377bc65754a8824271a09fbe6041732211d46c452b77e403ba10dcf603" Dec 03 11:39:10 crc kubenswrapper[5052]: I1203 11:39:10.364577 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-db-sync-qp7zq" Dec 03 11:39:10 crc kubenswrapper[5052]: I1203 11:39:10.424303 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/3a98a307-4671-43f7-86af-3b86591c8d7e-logs\") on node \"crc\" DevicePath \"\"" Dec 03 11:39:10 crc kubenswrapper[5052]: I1203 11:39:10.424342 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3a98a307-4671-43f7-86af-3b86591c8d7e-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:39:10 crc kubenswrapper[5052]: I1203 11:39:10.424355 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/3a98a307-4671-43f7-86af-3b86591c8d7e-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:39:10 crc kubenswrapper[5052]: I1203 11:39:10.424381 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage04-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage04-crc\") on node \"crc\" " Dec 03 11:39:10 crc kubenswrapper[5052]: I1203 11:39:10.424394 5052 reconciler_common.go:293] "Volume detached for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/3a98a307-4671-43f7-86af-3b86591c8d7e-httpd-run\") on node \"crc\" DevicePath \"\"" Dec 03 11:39:10 crc kubenswrapper[5052]: I1203 11:39:10.424405 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-jhsf4\" (UniqueName: \"kubernetes.io/projected/3a98a307-4671-43f7-86af-3b86591c8d7e-kube-api-access-jhsf4\") on node \"crc\" DevicePath \"\"" Dec 03 11:39:10 crc kubenswrapper[5052]: I1203 11:39:10.427519 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3a98a307-4671-43f7-86af-3b86591c8d7e-config-data" (OuterVolumeSpecName: "config-data") pod "3a98a307-4671-43f7-86af-3b86591c8d7e" (UID: "3a98a307-4671-43f7-86af-3b86591c8d7e"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:39:10 crc kubenswrapper[5052]: I1203 11:39:10.429860 5052 scope.go:117] "RemoveContainer" containerID="d028cc806e6ca374c34c8d04e35b3b1f8f7c0519bd4bbb5bebc699c00aaa1e53" Dec 03 11:39:10 crc kubenswrapper[5052]: I1203 11:39:10.457037 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage04-crc" (UniqueName: "kubernetes.io/local-volume/local-storage04-crc") on node "crc" Dec 03 11:39:10 crc kubenswrapper[5052]: I1203 11:39:10.525817 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage04-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage04-crc\") on node \"crc\" DevicePath \"\"" Dec 03 11:39:10 crc kubenswrapper[5052]: I1203 11:39:10.525849 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/3a98a307-4671-43f7-86af-3b86591c8d7e-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:39:10 crc kubenswrapper[5052]: I1203 11:39:10.559945 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/placement-699ddb8646-rfm4p"] Dec 03 11:39:10 crc kubenswrapper[5052]: E1203 11:39:10.560382 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="3a98a307-4671-43f7-86af-3b86591c8d7e" containerName="glance-httpd" Dec 03 11:39:10 crc kubenswrapper[5052]: I1203 11:39:10.560397 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="3a98a307-4671-43f7-86af-3b86591c8d7e" containerName="glance-httpd" Dec 03 11:39:10 crc kubenswrapper[5052]: E1203 11:39:10.560423 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="3a98a307-4671-43f7-86af-3b86591c8d7e" containerName="glance-log" Dec 03 11:39:10 crc kubenswrapper[5052]: I1203 11:39:10.560434 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="3a98a307-4671-43f7-86af-3b86591c8d7e" containerName="glance-log" Dec 03 11:39:10 crc kubenswrapper[5052]: E1203 11:39:10.560451 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="aae8de3e-7ad6-4a25-bdd5-3f98730ae227" containerName="neutron-db-sync" Dec 03 11:39:10 crc kubenswrapper[5052]: I1203 11:39:10.560460 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="aae8de3e-7ad6-4a25-bdd5-3f98730ae227" containerName="neutron-db-sync" Dec 03 11:39:10 crc kubenswrapper[5052]: E1203 11:39:10.560471 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="5ff261a9-6d56-45fe-88fa-7184d8a3b90b" containerName="placement-db-sync" Dec 03 11:39:10 crc kubenswrapper[5052]: I1203 11:39:10.560479 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ff261a9-6d56-45fe-88fa-7184d8a3b90b" containerName="placement-db-sync" Dec 03 11:39:10 crc kubenswrapper[5052]: I1203 11:39:10.560733 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="aae8de3e-7ad6-4a25-bdd5-3f98730ae227" containerName="neutron-db-sync" Dec 03 11:39:10 crc kubenswrapper[5052]: I1203 11:39:10.560750 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="5ff261a9-6d56-45fe-88fa-7184d8a3b90b" containerName="placement-db-sync" Dec 03 11:39:10 crc kubenswrapper[5052]: I1203 11:39:10.560768 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="3a98a307-4671-43f7-86af-3b86591c8d7e" containerName="glance-log" Dec 03 11:39:10 crc kubenswrapper[5052]: I1203 11:39:10.560778 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="3a98a307-4671-43f7-86af-3b86591c8d7e" containerName="glance-httpd" Dec 03 11:39:10 crc kubenswrapper[5052]: I1203 11:39:10.561848 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-699ddb8646-rfm4p" Dec 03 11:39:10 crc kubenswrapper[5052]: I1203 11:39:10.565848 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"placement-placement-dockercfg-sgp5f" Dec 03 11:39:10 crc kubenswrapper[5052]: I1203 11:39:10.566099 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-placement-public-svc" Dec 03 11:39:10 crc kubenswrapper[5052]: I1203 11:39:10.566268 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-placement-internal-svc" Dec 03 11:39:10 crc kubenswrapper[5052]: I1203 11:39:10.566395 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"placement-scripts" Dec 03 11:39:10 crc kubenswrapper[5052]: I1203 11:39:10.566552 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"placement-config-data" Dec 03 11:39:10 crc kubenswrapper[5052]: I1203 11:39:10.614578 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/placement-699ddb8646-rfm4p"] Dec 03 11:39:10 crc kubenswrapper[5052]: I1203 11:39:10.728829 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2l2kf\" (UniqueName: \"kubernetes.io/projected/0b4b31e5-e82d-4d44-9c8f-d001484293b6-kube-api-access-2l2kf\") pod \"placement-699ddb8646-rfm4p\" (UID: \"0b4b31e5-e82d-4d44-9c8f-d001484293b6\") " pod="openstack-kuttl-tests/placement-699ddb8646-rfm4p" Dec 03 11:39:10 crc kubenswrapper[5052]: I1203 11:39:10.729148 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/0b4b31e5-e82d-4d44-9c8f-d001484293b6-scripts\") pod \"placement-699ddb8646-rfm4p\" (UID: \"0b4b31e5-e82d-4d44-9c8f-d001484293b6\") " pod="openstack-kuttl-tests/placement-699ddb8646-rfm4p" Dec 03 11:39:10 crc kubenswrapper[5052]: I1203 11:39:10.729210 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/0b4b31e5-e82d-4d44-9c8f-d001484293b6-logs\") pod \"placement-699ddb8646-rfm4p\" (UID: \"0b4b31e5-e82d-4d44-9c8f-d001484293b6\") " pod="openstack-kuttl-tests/placement-699ddb8646-rfm4p" Dec 03 11:39:10 crc kubenswrapper[5052]: I1203 11:39:10.729253 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/0b4b31e5-e82d-4d44-9c8f-d001484293b6-config-data\") pod \"placement-699ddb8646-rfm4p\" (UID: \"0b4b31e5-e82d-4d44-9c8f-d001484293b6\") " pod="openstack-kuttl-tests/placement-699ddb8646-rfm4p" Dec 03 11:39:10 crc kubenswrapper[5052]: I1203 11:39:10.729280 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/0b4b31e5-e82d-4d44-9c8f-d001484293b6-internal-tls-certs\") pod \"placement-699ddb8646-rfm4p\" (UID: \"0b4b31e5-e82d-4d44-9c8f-d001484293b6\") " pod="openstack-kuttl-tests/placement-699ddb8646-rfm4p" Dec 03 11:39:10 crc kubenswrapper[5052]: I1203 11:39:10.729321 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0b4b31e5-e82d-4d44-9c8f-d001484293b6-combined-ca-bundle\") pod \"placement-699ddb8646-rfm4p\" (UID: \"0b4b31e5-e82d-4d44-9c8f-d001484293b6\") " pod="openstack-kuttl-tests/placement-699ddb8646-rfm4p" Dec 03 11:39:10 crc kubenswrapper[5052]: I1203 11:39:10.729368 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/0b4b31e5-e82d-4d44-9c8f-d001484293b6-public-tls-certs\") pod \"placement-699ddb8646-rfm4p\" (UID: \"0b4b31e5-e82d-4d44-9c8f-d001484293b6\") " pod="openstack-kuttl-tests/placement-699ddb8646-rfm4p" Dec 03 11:39:10 crc kubenswrapper[5052]: I1203 11:39:10.753610 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 11:39:10 crc kubenswrapper[5052]: I1203 11:39:10.761939 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 11:39:10 crc kubenswrapper[5052]: I1203 11:39:10.788818 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 11:39:10 crc kubenswrapper[5052]: I1203 11:39:10.790130 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:39:10 crc kubenswrapper[5052]: I1203 11:39:10.796163 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-default-internal-config-data" Dec 03 11:39:10 crc kubenswrapper[5052]: I1203 11:39:10.796389 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-glance-default-internal-svc" Dec 03 11:39:10 crc kubenswrapper[5052]: I1203 11:39:10.811199 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 11:39:10 crc kubenswrapper[5052]: I1203 11:39:10.838526 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-2l2kf\" (UniqueName: \"kubernetes.io/projected/0b4b31e5-e82d-4d44-9c8f-d001484293b6-kube-api-access-2l2kf\") pod \"placement-699ddb8646-rfm4p\" (UID: \"0b4b31e5-e82d-4d44-9c8f-d001484293b6\") " pod="openstack-kuttl-tests/placement-699ddb8646-rfm4p" Dec 03 11:39:10 crc kubenswrapper[5052]: I1203 11:39:10.838604 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/0b4b31e5-e82d-4d44-9c8f-d001484293b6-scripts\") pod \"placement-699ddb8646-rfm4p\" (UID: \"0b4b31e5-e82d-4d44-9c8f-d001484293b6\") " pod="openstack-kuttl-tests/placement-699ddb8646-rfm4p" Dec 03 11:39:10 crc kubenswrapper[5052]: I1203 11:39:10.838677 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/0b4b31e5-e82d-4d44-9c8f-d001484293b6-logs\") pod \"placement-699ddb8646-rfm4p\" (UID: \"0b4b31e5-e82d-4d44-9c8f-d001484293b6\") " pod="openstack-kuttl-tests/placement-699ddb8646-rfm4p" Dec 03 11:39:10 crc kubenswrapper[5052]: I1203 11:39:10.838705 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/0b4b31e5-e82d-4d44-9c8f-d001484293b6-config-data\") pod \"placement-699ddb8646-rfm4p\" (UID: \"0b4b31e5-e82d-4d44-9c8f-d001484293b6\") " pod="openstack-kuttl-tests/placement-699ddb8646-rfm4p" Dec 03 11:39:10 crc kubenswrapper[5052]: I1203 11:39:10.838723 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/0b4b31e5-e82d-4d44-9c8f-d001484293b6-internal-tls-certs\") pod \"placement-699ddb8646-rfm4p\" (UID: \"0b4b31e5-e82d-4d44-9c8f-d001484293b6\") " pod="openstack-kuttl-tests/placement-699ddb8646-rfm4p" Dec 03 11:39:10 crc kubenswrapper[5052]: I1203 11:39:10.838760 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0b4b31e5-e82d-4d44-9c8f-d001484293b6-combined-ca-bundle\") pod \"placement-699ddb8646-rfm4p\" (UID: \"0b4b31e5-e82d-4d44-9c8f-d001484293b6\") " pod="openstack-kuttl-tests/placement-699ddb8646-rfm4p" Dec 03 11:39:10 crc kubenswrapper[5052]: I1203 11:39:10.838788 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/0b4b31e5-e82d-4d44-9c8f-d001484293b6-public-tls-certs\") pod \"placement-699ddb8646-rfm4p\" (UID: \"0b4b31e5-e82d-4d44-9c8f-d001484293b6\") " pod="openstack-kuttl-tests/placement-699ddb8646-rfm4p" Dec 03 11:39:10 crc kubenswrapper[5052]: I1203 11:39:10.839927 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/0b4b31e5-e82d-4d44-9c8f-d001484293b6-logs\") pod \"placement-699ddb8646-rfm4p\" (UID: \"0b4b31e5-e82d-4d44-9c8f-d001484293b6\") " pod="openstack-kuttl-tests/placement-699ddb8646-rfm4p" Dec 03 11:39:10 crc kubenswrapper[5052]: I1203 11:39:10.845142 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/0b4b31e5-e82d-4d44-9c8f-d001484293b6-config-data\") pod \"placement-699ddb8646-rfm4p\" (UID: \"0b4b31e5-e82d-4d44-9c8f-d001484293b6\") " pod="openstack-kuttl-tests/placement-699ddb8646-rfm4p" Dec 03 11:39:10 crc kubenswrapper[5052]: I1203 11:39:10.849411 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0b4b31e5-e82d-4d44-9c8f-d001484293b6-combined-ca-bundle\") pod \"placement-699ddb8646-rfm4p\" (UID: \"0b4b31e5-e82d-4d44-9c8f-d001484293b6\") " pod="openstack-kuttl-tests/placement-699ddb8646-rfm4p" Dec 03 11:39:10 crc kubenswrapper[5052]: I1203 11:39:10.850669 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/0b4b31e5-e82d-4d44-9c8f-d001484293b6-scripts\") pod \"placement-699ddb8646-rfm4p\" (UID: \"0b4b31e5-e82d-4d44-9c8f-d001484293b6\") " pod="openstack-kuttl-tests/placement-699ddb8646-rfm4p" Dec 03 11:39:10 crc kubenswrapper[5052]: I1203 11:39:10.850995 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/0b4b31e5-e82d-4d44-9c8f-d001484293b6-internal-tls-certs\") pod \"placement-699ddb8646-rfm4p\" (UID: \"0b4b31e5-e82d-4d44-9c8f-d001484293b6\") " pod="openstack-kuttl-tests/placement-699ddb8646-rfm4p" Dec 03 11:39:10 crc kubenswrapper[5052]: I1203 11:39:10.851235 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/0b4b31e5-e82d-4d44-9c8f-d001484293b6-public-tls-certs\") pod \"placement-699ddb8646-rfm4p\" (UID: \"0b4b31e5-e82d-4d44-9c8f-d001484293b6\") " pod="openstack-kuttl-tests/placement-699ddb8646-rfm4p" Dec 03 11:39:10 crc kubenswrapper[5052]: I1203 11:39:10.858109 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-2l2kf\" (UniqueName: \"kubernetes.io/projected/0b4b31e5-e82d-4d44-9c8f-d001484293b6-kube-api-access-2l2kf\") pod \"placement-699ddb8646-rfm4p\" (UID: \"0b4b31e5-e82d-4d44-9c8f-d001484293b6\") " pod="openstack-kuttl-tests/placement-699ddb8646-rfm4p" Dec 03 11:39:10 crc kubenswrapper[5052]: I1203 11:39:10.929382 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-699ddb8646-rfm4p" Dec 03 11:39:10 crc kubenswrapper[5052]: I1203 11:39:10.939995 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/502cb03c-d1a5-4dc7-a040-0571a4695379-config-data\") pod \"glance-default-internal-api-0\" (UID: \"502cb03c-d1a5-4dc7-a040-0571a4695379\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:39:10 crc kubenswrapper[5052]: I1203 11:39:10.940052 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/502cb03c-d1a5-4dc7-a040-0571a4695379-httpd-run\") pod \"glance-default-internal-api-0\" (UID: \"502cb03c-d1a5-4dc7-a040-0571a4695379\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:39:10 crc kubenswrapper[5052]: I1203 11:39:10.940088 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/502cb03c-d1a5-4dc7-a040-0571a4695379-combined-ca-bundle\") pod \"glance-default-internal-api-0\" (UID: \"502cb03c-d1a5-4dc7-a040-0571a4695379\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:39:10 crc kubenswrapper[5052]: I1203 11:39:10.940148 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6zvv5\" (UniqueName: \"kubernetes.io/projected/502cb03c-d1a5-4dc7-a040-0571a4695379-kube-api-access-6zvv5\") pod \"glance-default-internal-api-0\" (UID: \"502cb03c-d1a5-4dc7-a040-0571a4695379\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:39:10 crc kubenswrapper[5052]: I1203 11:39:10.940184 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage04-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage04-crc\") pod \"glance-default-internal-api-0\" (UID: \"502cb03c-d1a5-4dc7-a040-0571a4695379\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:39:10 crc kubenswrapper[5052]: I1203 11:39:10.940234 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/502cb03c-d1a5-4dc7-a040-0571a4695379-scripts\") pod \"glance-default-internal-api-0\" (UID: \"502cb03c-d1a5-4dc7-a040-0571a4695379\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:39:10 crc kubenswrapper[5052]: I1203 11:39:10.940268 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/502cb03c-d1a5-4dc7-a040-0571a4695379-logs\") pod \"glance-default-internal-api-0\" (UID: \"502cb03c-d1a5-4dc7-a040-0571a4695379\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:39:10 crc kubenswrapper[5052]: I1203 11:39:10.940323 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/502cb03c-d1a5-4dc7-a040-0571a4695379-internal-tls-certs\") pod \"glance-default-internal-api-0\" (UID: \"502cb03c-d1a5-4dc7-a040-0571a4695379\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:39:10 crc kubenswrapper[5052]: I1203 11:39:10.991647 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:39:11 crc kubenswrapper[5052]: I1203 11:39:11.041575 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-6zvv5\" (UniqueName: \"kubernetes.io/projected/502cb03c-d1a5-4dc7-a040-0571a4695379-kube-api-access-6zvv5\") pod \"glance-default-internal-api-0\" (UID: \"502cb03c-d1a5-4dc7-a040-0571a4695379\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:39:11 crc kubenswrapper[5052]: I1203 11:39:11.041895 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage04-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage04-crc\") pod \"glance-default-internal-api-0\" (UID: \"502cb03c-d1a5-4dc7-a040-0571a4695379\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:39:11 crc kubenswrapper[5052]: I1203 11:39:11.041932 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/502cb03c-d1a5-4dc7-a040-0571a4695379-scripts\") pod \"glance-default-internal-api-0\" (UID: \"502cb03c-d1a5-4dc7-a040-0571a4695379\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:39:11 crc kubenswrapper[5052]: I1203 11:39:11.041958 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/502cb03c-d1a5-4dc7-a040-0571a4695379-logs\") pod \"glance-default-internal-api-0\" (UID: \"502cb03c-d1a5-4dc7-a040-0571a4695379\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:39:11 crc kubenswrapper[5052]: I1203 11:39:11.041998 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/502cb03c-d1a5-4dc7-a040-0571a4695379-internal-tls-certs\") pod \"glance-default-internal-api-0\" (UID: \"502cb03c-d1a5-4dc7-a040-0571a4695379\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:39:11 crc kubenswrapper[5052]: I1203 11:39:11.042029 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/502cb03c-d1a5-4dc7-a040-0571a4695379-config-data\") pod \"glance-default-internal-api-0\" (UID: \"502cb03c-d1a5-4dc7-a040-0571a4695379\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:39:11 crc kubenswrapper[5052]: I1203 11:39:11.042053 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/502cb03c-d1a5-4dc7-a040-0571a4695379-httpd-run\") pod \"glance-default-internal-api-0\" (UID: \"502cb03c-d1a5-4dc7-a040-0571a4695379\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:39:11 crc kubenswrapper[5052]: I1203 11:39:11.042089 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/502cb03c-d1a5-4dc7-a040-0571a4695379-combined-ca-bundle\") pod \"glance-default-internal-api-0\" (UID: \"502cb03c-d1a5-4dc7-a040-0571a4695379\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:39:11 crc kubenswrapper[5052]: I1203 11:39:11.043918 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/502cb03c-d1a5-4dc7-a040-0571a4695379-logs\") pod \"glance-default-internal-api-0\" (UID: \"502cb03c-d1a5-4dc7-a040-0571a4695379\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:39:11 crc kubenswrapper[5052]: I1203 11:39:11.044534 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage04-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage04-crc\") pod \"glance-default-internal-api-0\" (UID: \"502cb03c-d1a5-4dc7-a040-0571a4695379\") device mount path \"/mnt/openstack/pv04\"" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:39:11 crc kubenswrapper[5052]: I1203 11:39:11.044984 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/502cb03c-d1a5-4dc7-a040-0571a4695379-httpd-run\") pod \"glance-default-internal-api-0\" (UID: \"502cb03c-d1a5-4dc7-a040-0571a4695379\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:39:11 crc kubenswrapper[5052]: I1203 11:39:11.050079 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/502cb03c-d1a5-4dc7-a040-0571a4695379-scripts\") pod \"glance-default-internal-api-0\" (UID: \"502cb03c-d1a5-4dc7-a040-0571a4695379\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:39:11 crc kubenswrapper[5052]: I1203 11:39:11.053907 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/502cb03c-d1a5-4dc7-a040-0571a4695379-combined-ca-bundle\") pod \"glance-default-internal-api-0\" (UID: \"502cb03c-d1a5-4dc7-a040-0571a4695379\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:39:11 crc kubenswrapper[5052]: I1203 11:39:11.054478 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/502cb03c-d1a5-4dc7-a040-0571a4695379-internal-tls-certs\") pod \"glance-default-internal-api-0\" (UID: \"502cb03c-d1a5-4dc7-a040-0571a4695379\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:39:11 crc kubenswrapper[5052]: I1203 11:39:11.061500 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/502cb03c-d1a5-4dc7-a040-0571a4695379-config-data\") pod \"glance-default-internal-api-0\" (UID: \"502cb03c-d1a5-4dc7-a040-0571a4695379\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:39:11 crc kubenswrapper[5052]: I1203 11:39:11.069452 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-6zvv5\" (UniqueName: \"kubernetes.io/projected/502cb03c-d1a5-4dc7-a040-0571a4695379-kube-api-access-6zvv5\") pod \"glance-default-internal-api-0\" (UID: \"502cb03c-d1a5-4dc7-a040-0571a4695379\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:39:11 crc kubenswrapper[5052]: I1203 11:39:11.077439 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage04-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage04-crc\") pod \"glance-default-internal-api-0\" (UID: \"502cb03c-d1a5-4dc7-a040-0571a4695379\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:39:11 crc kubenswrapper[5052]: I1203 11:39:11.116203 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:39:11 crc kubenswrapper[5052]: I1203 11:39:11.143769 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/de570c88-14c0-4761-be7f-da7a6d97715a-httpd-run\") pod \"de570c88-14c0-4761-be7f-da7a6d97715a\" (UID: \"de570c88-14c0-4761-be7f-da7a6d97715a\") " Dec 03 11:39:11 crc kubenswrapper[5052]: I1203 11:39:11.143852 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"glance\" (UniqueName: \"kubernetes.io/local-volume/local-storage15-crc\") pod \"de570c88-14c0-4761-be7f-da7a6d97715a\" (UID: \"de570c88-14c0-4761-be7f-da7a6d97715a\") " Dec 03 11:39:11 crc kubenswrapper[5052]: I1203 11:39:11.143901 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/de570c88-14c0-4761-be7f-da7a6d97715a-logs\") pod \"de570c88-14c0-4761-be7f-da7a6d97715a\" (UID: \"de570c88-14c0-4761-be7f-da7a6d97715a\") " Dec 03 11:39:11 crc kubenswrapper[5052]: I1203 11:39:11.143951 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/de570c88-14c0-4761-be7f-da7a6d97715a-combined-ca-bundle\") pod \"de570c88-14c0-4761-be7f-da7a6d97715a\" (UID: \"de570c88-14c0-4761-be7f-da7a6d97715a\") " Dec 03 11:39:11 crc kubenswrapper[5052]: I1203 11:39:11.143977 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/de570c88-14c0-4761-be7f-da7a6d97715a-config-data\") pod \"de570c88-14c0-4761-be7f-da7a6d97715a\" (UID: \"de570c88-14c0-4761-be7f-da7a6d97715a\") " Dec 03 11:39:11 crc kubenswrapper[5052]: I1203 11:39:11.144007 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6bqgp\" (UniqueName: \"kubernetes.io/projected/de570c88-14c0-4761-be7f-da7a6d97715a-kube-api-access-6bqgp\") pod \"de570c88-14c0-4761-be7f-da7a6d97715a\" (UID: \"de570c88-14c0-4761-be7f-da7a6d97715a\") " Dec 03 11:39:11 crc kubenswrapper[5052]: I1203 11:39:11.144058 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/de570c88-14c0-4761-be7f-da7a6d97715a-scripts\") pod \"de570c88-14c0-4761-be7f-da7a6d97715a\" (UID: \"de570c88-14c0-4761-be7f-da7a6d97715a\") " Dec 03 11:39:11 crc kubenswrapper[5052]: I1203 11:39:11.147144 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de570c88-14c0-4761-be7f-da7a6d97715a-httpd-run" (OuterVolumeSpecName: "httpd-run") pod "de570c88-14c0-4761-be7f-da7a6d97715a" (UID: "de570c88-14c0-4761-be7f-da7a6d97715a"). InnerVolumeSpecName "httpd-run". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:39:11 crc kubenswrapper[5052]: I1203 11:39:11.147214 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de570c88-14c0-4761-be7f-da7a6d97715a-logs" (OuterVolumeSpecName: "logs") pod "de570c88-14c0-4761-be7f-da7a6d97715a" (UID: "de570c88-14c0-4761-be7f-da7a6d97715a"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:39:11 crc kubenswrapper[5052]: I1203 11:39:11.147387 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/de570c88-14c0-4761-be7f-da7a6d97715a-logs\") on node \"crc\" DevicePath \"\"" Dec 03 11:39:11 crc kubenswrapper[5052]: I1203 11:39:11.147420 5052 reconciler_common.go:293] "Volume detached for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/de570c88-14c0-4761-be7f-da7a6d97715a-httpd-run\") on node \"crc\" DevicePath \"\"" Dec 03 11:39:11 crc kubenswrapper[5052]: I1203 11:39:11.151145 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/de570c88-14c0-4761-be7f-da7a6d97715a-kube-api-access-6bqgp" (OuterVolumeSpecName: "kube-api-access-6bqgp") pod "de570c88-14c0-4761-be7f-da7a6d97715a" (UID: "de570c88-14c0-4761-be7f-da7a6d97715a"). InnerVolumeSpecName "kube-api-access-6bqgp". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:39:11 crc kubenswrapper[5052]: I1203 11:39:11.152424 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage15-crc" (OuterVolumeSpecName: "glance") pod "de570c88-14c0-4761-be7f-da7a6d97715a" (UID: "de570c88-14c0-4761-be7f-da7a6d97715a"). InnerVolumeSpecName "local-storage15-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 11:39:11 crc kubenswrapper[5052]: I1203 11:39:11.152646 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/de570c88-14c0-4761-be7f-da7a6d97715a-scripts" (OuterVolumeSpecName: "scripts") pod "de570c88-14c0-4761-be7f-da7a6d97715a" (UID: "de570c88-14c0-4761-be7f-da7a6d97715a"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:39:11 crc kubenswrapper[5052]: I1203 11:39:11.193742 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/de570c88-14c0-4761-be7f-da7a6d97715a-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "de570c88-14c0-4761-be7f-da7a6d97715a" (UID: "de570c88-14c0-4761-be7f-da7a6d97715a"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:39:11 crc kubenswrapper[5052]: I1203 11:39:11.245659 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/de570c88-14c0-4761-be7f-da7a6d97715a-config-data" (OuterVolumeSpecName: "config-data") pod "de570c88-14c0-4761-be7f-da7a6d97715a" (UID: "de570c88-14c0-4761-be7f-da7a6d97715a"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:39:11 crc kubenswrapper[5052]: I1203 11:39:11.248201 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/neutron-58dc59ff57-9dsf9"] Dec 03 11:39:11 crc kubenswrapper[5052]: I1203 11:39:11.248725 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage15-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage15-crc\") on node \"crc\" " Dec 03 11:39:11 crc kubenswrapper[5052]: I1203 11:39:11.248772 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/de570c88-14c0-4761-be7f-da7a6d97715a-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:39:11 crc kubenswrapper[5052]: I1203 11:39:11.248789 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/de570c88-14c0-4761-be7f-da7a6d97715a-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:39:11 crc kubenswrapper[5052]: I1203 11:39:11.248802 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-6bqgp\" (UniqueName: \"kubernetes.io/projected/de570c88-14c0-4761-be7f-da7a6d97715a-kube-api-access-6bqgp\") on node \"crc\" DevicePath \"\"" Dec 03 11:39:11 crc kubenswrapper[5052]: I1203 11:39:11.248816 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/de570c88-14c0-4761-be7f-da7a6d97715a-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:39:11 crc kubenswrapper[5052]: E1203 11:39:11.250848 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="de570c88-14c0-4761-be7f-da7a6d97715a" containerName="glance-log" Dec 03 11:39:11 crc kubenswrapper[5052]: I1203 11:39:11.250882 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="de570c88-14c0-4761-be7f-da7a6d97715a" containerName="glance-log" Dec 03 11:39:11 crc kubenswrapper[5052]: E1203 11:39:11.250911 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="de570c88-14c0-4761-be7f-da7a6d97715a" containerName="glance-httpd" Dec 03 11:39:11 crc kubenswrapper[5052]: I1203 11:39:11.250918 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="de570c88-14c0-4761-be7f-da7a6d97715a" containerName="glance-httpd" Dec 03 11:39:11 crc kubenswrapper[5052]: I1203 11:39:11.251192 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="de570c88-14c0-4761-be7f-da7a6d97715a" containerName="glance-log" Dec 03 11:39:11 crc kubenswrapper[5052]: I1203 11:39:11.251214 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="de570c88-14c0-4761-be7f-da7a6d97715a" containerName="glance-httpd" Dec 03 11:39:11 crc kubenswrapper[5052]: I1203 11:39:11.254003 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-58dc59ff57-9dsf9" Dec 03 11:39:11 crc kubenswrapper[5052]: I1203 11:39:11.257119 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"neutron-config" Dec 03 11:39:11 crc kubenswrapper[5052]: I1203 11:39:11.257266 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"neutron-httpd-config" Dec 03 11:39:11 crc kubenswrapper[5052]: I1203 11:39:11.257412 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"neutron-neutron-dockercfg-6xp5c" Dec 03 11:39:11 crc kubenswrapper[5052]: I1203 11:39:11.257542 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-neutron-ovndbs" Dec 03 11:39:11 crc kubenswrapper[5052]: I1203 11:39:11.276295 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-58dc59ff57-9dsf9"] Dec 03 11:39:11 crc kubenswrapper[5052]: I1203 11:39:11.306202 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage15-crc" (UniqueName: "kubernetes.io/local-volume/local-storage15-crc") on node "crc" Dec 03 11:39:11 crc kubenswrapper[5052]: I1203 11:39:11.350583 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/577c99af-c225-41f3-9d97-e96326a8c1c4-ovndb-tls-certs\") pod \"neutron-58dc59ff57-9dsf9\" (UID: \"577c99af-c225-41f3-9d97-e96326a8c1c4\") " pod="openstack-kuttl-tests/neutron-58dc59ff57-9dsf9" Dec 03 11:39:11 crc kubenswrapper[5052]: I1203 11:39:11.350657 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/577c99af-c225-41f3-9d97-e96326a8c1c4-combined-ca-bundle\") pod \"neutron-58dc59ff57-9dsf9\" (UID: \"577c99af-c225-41f3-9d97-e96326a8c1c4\") " pod="openstack-kuttl-tests/neutron-58dc59ff57-9dsf9" Dec 03 11:39:11 crc kubenswrapper[5052]: I1203 11:39:11.350682 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q6nff\" (UniqueName: \"kubernetes.io/projected/577c99af-c225-41f3-9d97-e96326a8c1c4-kube-api-access-q6nff\") pod \"neutron-58dc59ff57-9dsf9\" (UID: \"577c99af-c225-41f3-9d97-e96326a8c1c4\") " pod="openstack-kuttl-tests/neutron-58dc59ff57-9dsf9" Dec 03 11:39:11 crc kubenswrapper[5052]: I1203 11:39:11.350786 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/577c99af-c225-41f3-9d97-e96326a8c1c4-httpd-config\") pod \"neutron-58dc59ff57-9dsf9\" (UID: \"577c99af-c225-41f3-9d97-e96326a8c1c4\") " pod="openstack-kuttl-tests/neutron-58dc59ff57-9dsf9" Dec 03 11:39:11 crc kubenswrapper[5052]: I1203 11:39:11.350829 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/577c99af-c225-41f3-9d97-e96326a8c1c4-config\") pod \"neutron-58dc59ff57-9dsf9\" (UID: \"577c99af-c225-41f3-9d97-e96326a8c1c4\") " pod="openstack-kuttl-tests/neutron-58dc59ff57-9dsf9" Dec 03 11:39:11 crc kubenswrapper[5052]: I1203 11:39:11.350874 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage15-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage15-crc\") on node \"crc\" DevicePath \"\"" Dec 03 11:39:11 crc kubenswrapper[5052]: I1203 11:39:11.394214 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:39:11 crc kubenswrapper[5052]: I1203 11:39:11.395067 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"de570c88-14c0-4761-be7f-da7a6d97715a","Type":"ContainerDied","Data":"6ed2ca0ce999fbd725336cbb1b7fb4853d5da9819e56d0268e19686ba90e7c0a"} Dec 03 11:39:11 crc kubenswrapper[5052]: I1203 11:39:11.395129 5052 scope.go:117] "RemoveContainer" containerID="c9111aa5f09ee7dfc464783e5e12845d19057e42ac17492647a8779a1d7018ce" Dec 03 11:39:11 crc kubenswrapper[5052]: I1203 11:39:11.412672 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/placement-699ddb8646-rfm4p"] Dec 03 11:39:11 crc kubenswrapper[5052]: W1203 11:39:11.417445 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0b4b31e5_e82d_4d44_9c8f_d001484293b6.slice/crio-e192664a9c8e1f0804fec134de74214459efa8fd9b22de65ed888db40bd3cb99 WatchSource:0}: Error finding container e192664a9c8e1f0804fec134de74214459efa8fd9b22de65ed888db40bd3cb99: Status 404 returned error can't find the container with id e192664a9c8e1f0804fec134de74214459efa8fd9b22de65ed888db40bd3cb99 Dec 03 11:39:11 crc kubenswrapper[5052]: I1203 11:39:11.452554 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/577c99af-c225-41f3-9d97-e96326a8c1c4-ovndb-tls-certs\") pod \"neutron-58dc59ff57-9dsf9\" (UID: \"577c99af-c225-41f3-9d97-e96326a8c1c4\") " pod="openstack-kuttl-tests/neutron-58dc59ff57-9dsf9" Dec 03 11:39:11 crc kubenswrapper[5052]: I1203 11:39:11.452952 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/577c99af-c225-41f3-9d97-e96326a8c1c4-combined-ca-bundle\") pod \"neutron-58dc59ff57-9dsf9\" (UID: \"577c99af-c225-41f3-9d97-e96326a8c1c4\") " pod="openstack-kuttl-tests/neutron-58dc59ff57-9dsf9" Dec 03 11:39:11 crc kubenswrapper[5052]: I1203 11:39:11.452983 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-q6nff\" (UniqueName: \"kubernetes.io/projected/577c99af-c225-41f3-9d97-e96326a8c1c4-kube-api-access-q6nff\") pod \"neutron-58dc59ff57-9dsf9\" (UID: \"577c99af-c225-41f3-9d97-e96326a8c1c4\") " pod="openstack-kuttl-tests/neutron-58dc59ff57-9dsf9" Dec 03 11:39:11 crc kubenswrapper[5052]: I1203 11:39:11.453118 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/577c99af-c225-41f3-9d97-e96326a8c1c4-httpd-config\") pod \"neutron-58dc59ff57-9dsf9\" (UID: \"577c99af-c225-41f3-9d97-e96326a8c1c4\") " pod="openstack-kuttl-tests/neutron-58dc59ff57-9dsf9" Dec 03 11:39:11 crc kubenswrapper[5052]: I1203 11:39:11.453171 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/577c99af-c225-41f3-9d97-e96326a8c1c4-config\") pod \"neutron-58dc59ff57-9dsf9\" (UID: \"577c99af-c225-41f3-9d97-e96326a8c1c4\") " pod="openstack-kuttl-tests/neutron-58dc59ff57-9dsf9" Dec 03 11:39:11 crc kubenswrapper[5052]: I1203 11:39:11.463907 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/577c99af-c225-41f3-9d97-e96326a8c1c4-httpd-config\") pod \"neutron-58dc59ff57-9dsf9\" (UID: \"577c99af-c225-41f3-9d97-e96326a8c1c4\") " pod="openstack-kuttl-tests/neutron-58dc59ff57-9dsf9" Dec 03 11:39:11 crc kubenswrapper[5052]: I1203 11:39:11.463919 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/577c99af-c225-41f3-9d97-e96326a8c1c4-ovndb-tls-certs\") pod \"neutron-58dc59ff57-9dsf9\" (UID: \"577c99af-c225-41f3-9d97-e96326a8c1c4\") " pod="openstack-kuttl-tests/neutron-58dc59ff57-9dsf9" Dec 03 11:39:11 crc kubenswrapper[5052]: I1203 11:39:11.468452 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/577c99af-c225-41f3-9d97-e96326a8c1c4-combined-ca-bundle\") pod \"neutron-58dc59ff57-9dsf9\" (UID: \"577c99af-c225-41f3-9d97-e96326a8c1c4\") " pod="openstack-kuttl-tests/neutron-58dc59ff57-9dsf9" Dec 03 11:39:11 crc kubenswrapper[5052]: I1203 11:39:11.469449 5052 scope.go:117] "RemoveContainer" containerID="1e36542bcba777c859a2d3c7c005a13275c8b488c65a6401477cc95235b53c39" Dec 03 11:39:11 crc kubenswrapper[5052]: I1203 11:39:11.480619 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-q6nff\" (UniqueName: \"kubernetes.io/projected/577c99af-c225-41f3-9d97-e96326a8c1c4-kube-api-access-q6nff\") pod \"neutron-58dc59ff57-9dsf9\" (UID: \"577c99af-c225-41f3-9d97-e96326a8c1c4\") " pod="openstack-kuttl-tests/neutron-58dc59ff57-9dsf9" Dec 03 11:39:11 crc kubenswrapper[5052]: I1203 11:39:11.490961 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 11:39:11 crc kubenswrapper[5052]: I1203 11:39:11.492339 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/577c99af-c225-41f3-9d97-e96326a8c1c4-config\") pod \"neutron-58dc59ff57-9dsf9\" (UID: \"577c99af-c225-41f3-9d97-e96326a8c1c4\") " pod="openstack-kuttl-tests/neutron-58dc59ff57-9dsf9" Dec 03 11:39:11 crc kubenswrapper[5052]: I1203 11:39:11.506914 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 11:39:11 crc kubenswrapper[5052]: I1203 11:39:11.519484 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 11:39:11 crc kubenswrapper[5052]: I1203 11:39:11.522869 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:39:11 crc kubenswrapper[5052]: I1203 11:39:11.531685 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 11:39:11 crc kubenswrapper[5052]: I1203 11:39:11.535947 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-default-external-config-data" Dec 03 11:39:11 crc kubenswrapper[5052]: I1203 11:39:11.537182 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-glance-default-public-svc" Dec 03 11:39:11 crc kubenswrapper[5052]: I1203 11:39:11.589986 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-58dc59ff57-9dsf9" Dec 03 11:39:11 crc kubenswrapper[5052]: I1203 11:39:11.663386 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 11:39:11 crc kubenswrapper[5052]: I1203 11:39:11.669354 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/8bf67244-3ae2-4c00-a15d-4c4180d5f835-httpd-run\") pod \"glance-default-external-api-0\" (UID: \"8bf67244-3ae2-4c00-a15d-4c4180d5f835\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:39:11 crc kubenswrapper[5052]: I1203 11:39:11.669868 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage15-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage15-crc\") pod \"glance-default-external-api-0\" (UID: \"8bf67244-3ae2-4c00-a15d-4c4180d5f835\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:39:11 crc kubenswrapper[5052]: I1203 11:39:11.669903 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/8bf67244-3ae2-4c00-a15d-4c4180d5f835-public-tls-certs\") pod \"glance-default-external-api-0\" (UID: \"8bf67244-3ae2-4c00-a15d-4c4180d5f835\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:39:11 crc kubenswrapper[5052]: I1203 11:39:11.670153 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8bf67244-3ae2-4c00-a15d-4c4180d5f835-combined-ca-bundle\") pod \"glance-default-external-api-0\" (UID: \"8bf67244-3ae2-4c00-a15d-4c4180d5f835\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:39:11 crc kubenswrapper[5052]: I1203 11:39:11.670384 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/8bf67244-3ae2-4c00-a15d-4c4180d5f835-scripts\") pod \"glance-default-external-api-0\" (UID: \"8bf67244-3ae2-4c00-a15d-4c4180d5f835\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:39:11 crc kubenswrapper[5052]: I1203 11:39:11.670615 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/8bf67244-3ae2-4c00-a15d-4c4180d5f835-logs\") pod \"glance-default-external-api-0\" (UID: \"8bf67244-3ae2-4c00-a15d-4c4180d5f835\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:39:11 crc kubenswrapper[5052]: I1203 11:39:11.670648 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cp4l6\" (UniqueName: \"kubernetes.io/projected/8bf67244-3ae2-4c00-a15d-4c4180d5f835-kube-api-access-cp4l6\") pod \"glance-default-external-api-0\" (UID: \"8bf67244-3ae2-4c00-a15d-4c4180d5f835\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:39:11 crc kubenswrapper[5052]: I1203 11:39:11.670874 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/8bf67244-3ae2-4c00-a15d-4c4180d5f835-config-data\") pod \"glance-default-external-api-0\" (UID: \"8bf67244-3ae2-4c00-a15d-4c4180d5f835\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:39:11 crc kubenswrapper[5052]: I1203 11:39:11.719384 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3a98a307-4671-43f7-86af-3b86591c8d7e" path="/var/lib/kubelet/pods/3a98a307-4671-43f7-86af-3b86591c8d7e/volumes" Dec 03 11:39:11 crc kubenswrapper[5052]: I1203 11:39:11.720897 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="de570c88-14c0-4761-be7f-da7a6d97715a" path="/var/lib/kubelet/pods/de570c88-14c0-4761-be7f-da7a6d97715a/volumes" Dec 03 11:39:11 crc kubenswrapper[5052]: I1203 11:39:11.772773 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/8bf67244-3ae2-4c00-a15d-4c4180d5f835-logs\") pod \"glance-default-external-api-0\" (UID: \"8bf67244-3ae2-4c00-a15d-4c4180d5f835\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:39:11 crc kubenswrapper[5052]: I1203 11:39:11.772836 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-cp4l6\" (UniqueName: \"kubernetes.io/projected/8bf67244-3ae2-4c00-a15d-4c4180d5f835-kube-api-access-cp4l6\") pod \"glance-default-external-api-0\" (UID: \"8bf67244-3ae2-4c00-a15d-4c4180d5f835\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:39:11 crc kubenswrapper[5052]: I1203 11:39:11.772891 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/8bf67244-3ae2-4c00-a15d-4c4180d5f835-config-data\") pod \"glance-default-external-api-0\" (UID: \"8bf67244-3ae2-4c00-a15d-4c4180d5f835\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:39:11 crc kubenswrapper[5052]: I1203 11:39:11.772947 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/8bf67244-3ae2-4c00-a15d-4c4180d5f835-httpd-run\") pod \"glance-default-external-api-0\" (UID: \"8bf67244-3ae2-4c00-a15d-4c4180d5f835\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:39:11 crc kubenswrapper[5052]: I1203 11:39:11.772990 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage15-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage15-crc\") pod \"glance-default-external-api-0\" (UID: \"8bf67244-3ae2-4c00-a15d-4c4180d5f835\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:39:11 crc kubenswrapper[5052]: I1203 11:39:11.773011 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/8bf67244-3ae2-4c00-a15d-4c4180d5f835-public-tls-certs\") pod \"glance-default-external-api-0\" (UID: \"8bf67244-3ae2-4c00-a15d-4c4180d5f835\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:39:11 crc kubenswrapper[5052]: I1203 11:39:11.773095 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8bf67244-3ae2-4c00-a15d-4c4180d5f835-combined-ca-bundle\") pod \"glance-default-external-api-0\" (UID: \"8bf67244-3ae2-4c00-a15d-4c4180d5f835\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:39:11 crc kubenswrapper[5052]: I1203 11:39:11.773161 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/8bf67244-3ae2-4c00-a15d-4c4180d5f835-scripts\") pod \"glance-default-external-api-0\" (UID: \"8bf67244-3ae2-4c00-a15d-4c4180d5f835\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:39:11 crc kubenswrapper[5052]: I1203 11:39:11.774173 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/8bf67244-3ae2-4c00-a15d-4c4180d5f835-logs\") pod \"glance-default-external-api-0\" (UID: \"8bf67244-3ae2-4c00-a15d-4c4180d5f835\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:39:11 crc kubenswrapper[5052]: I1203 11:39:11.775169 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage15-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage15-crc\") pod \"glance-default-external-api-0\" (UID: \"8bf67244-3ae2-4c00-a15d-4c4180d5f835\") device mount path \"/mnt/openstack/pv15\"" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:39:11 crc kubenswrapper[5052]: I1203 11:39:11.775215 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/8bf67244-3ae2-4c00-a15d-4c4180d5f835-httpd-run\") pod \"glance-default-external-api-0\" (UID: \"8bf67244-3ae2-4c00-a15d-4c4180d5f835\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:39:11 crc kubenswrapper[5052]: I1203 11:39:11.783186 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8bf67244-3ae2-4c00-a15d-4c4180d5f835-combined-ca-bundle\") pod \"glance-default-external-api-0\" (UID: \"8bf67244-3ae2-4c00-a15d-4c4180d5f835\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:39:11 crc kubenswrapper[5052]: I1203 11:39:11.783229 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/8bf67244-3ae2-4c00-a15d-4c4180d5f835-scripts\") pod \"glance-default-external-api-0\" (UID: \"8bf67244-3ae2-4c00-a15d-4c4180d5f835\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:39:11 crc kubenswrapper[5052]: I1203 11:39:11.784574 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/8bf67244-3ae2-4c00-a15d-4c4180d5f835-config-data\") pod \"glance-default-external-api-0\" (UID: \"8bf67244-3ae2-4c00-a15d-4c4180d5f835\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:39:11 crc kubenswrapper[5052]: I1203 11:39:11.786251 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/8bf67244-3ae2-4c00-a15d-4c4180d5f835-public-tls-certs\") pod \"glance-default-external-api-0\" (UID: \"8bf67244-3ae2-4c00-a15d-4c4180d5f835\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:39:11 crc kubenswrapper[5052]: I1203 11:39:11.800496 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-cp4l6\" (UniqueName: \"kubernetes.io/projected/8bf67244-3ae2-4c00-a15d-4c4180d5f835-kube-api-access-cp4l6\") pod \"glance-default-external-api-0\" (UID: \"8bf67244-3ae2-4c00-a15d-4c4180d5f835\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:39:11 crc kubenswrapper[5052]: I1203 11:39:11.946902 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage15-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage15-crc\") pod \"glance-default-external-api-0\" (UID: \"8bf67244-3ae2-4c00-a15d-4c4180d5f835\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:39:12 crc kubenswrapper[5052]: I1203 11:39:12.137059 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-bootstrap-v8g6k" Dec 03 11:39:12 crc kubenswrapper[5052]: I1203 11:39:12.170335 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:39:12 crc kubenswrapper[5052]: I1203 11:39:12.197616 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-58dc59ff57-9dsf9"] Dec 03 11:39:12 crc kubenswrapper[5052]: W1203 11:39:12.251714 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod577c99af_c225_41f3_9d97_e96326a8c1c4.slice/crio-4973fa976f43a54ac2651aef1f0013167f193a85272c9d8480cdb9ff98c3fe29 WatchSource:0}: Error finding container 4973fa976f43a54ac2651aef1f0013167f193a85272c9d8480cdb9ff98c3fe29: Status 404 returned error can't find the container with id 4973fa976f43a54ac2651aef1f0013167f193a85272c9d8480cdb9ff98c3fe29 Dec 03 11:39:12 crc kubenswrapper[5052]: I1203 11:39:12.287321 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/2204f9d2-fb11-429c-808f-8fd46881102d-fernet-keys\") pod \"2204f9d2-fb11-429c-808f-8fd46881102d\" (UID: \"2204f9d2-fb11-429c-808f-8fd46881102d\") " Dec 03 11:39:12 crc kubenswrapper[5052]: I1203 11:39:12.287480 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2204f9d2-fb11-429c-808f-8fd46881102d-combined-ca-bundle\") pod \"2204f9d2-fb11-429c-808f-8fd46881102d\" (UID: \"2204f9d2-fb11-429c-808f-8fd46881102d\") " Dec 03 11:39:12 crc kubenswrapper[5052]: I1203 11:39:12.287587 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/2204f9d2-fb11-429c-808f-8fd46881102d-credential-keys\") pod \"2204f9d2-fb11-429c-808f-8fd46881102d\" (UID: \"2204f9d2-fb11-429c-808f-8fd46881102d\") " Dec 03 11:39:12 crc kubenswrapper[5052]: I1203 11:39:12.287639 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cxjgj\" (UniqueName: \"kubernetes.io/projected/2204f9d2-fb11-429c-808f-8fd46881102d-kube-api-access-cxjgj\") pod \"2204f9d2-fb11-429c-808f-8fd46881102d\" (UID: \"2204f9d2-fb11-429c-808f-8fd46881102d\") " Dec 03 11:39:12 crc kubenswrapper[5052]: I1203 11:39:12.287694 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/2204f9d2-fb11-429c-808f-8fd46881102d-config-data\") pod \"2204f9d2-fb11-429c-808f-8fd46881102d\" (UID: \"2204f9d2-fb11-429c-808f-8fd46881102d\") " Dec 03 11:39:12 crc kubenswrapper[5052]: I1203 11:39:12.287750 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/2204f9d2-fb11-429c-808f-8fd46881102d-scripts\") pod \"2204f9d2-fb11-429c-808f-8fd46881102d\" (UID: \"2204f9d2-fb11-429c-808f-8fd46881102d\") " Dec 03 11:39:12 crc kubenswrapper[5052]: I1203 11:39:12.291628 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2204f9d2-fb11-429c-808f-8fd46881102d-scripts" (OuterVolumeSpecName: "scripts") pod "2204f9d2-fb11-429c-808f-8fd46881102d" (UID: "2204f9d2-fb11-429c-808f-8fd46881102d"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:39:12 crc kubenswrapper[5052]: I1203 11:39:12.294490 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2204f9d2-fb11-429c-808f-8fd46881102d-credential-keys" (OuterVolumeSpecName: "credential-keys") pod "2204f9d2-fb11-429c-808f-8fd46881102d" (UID: "2204f9d2-fb11-429c-808f-8fd46881102d"). InnerVolumeSpecName "credential-keys". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:39:12 crc kubenswrapper[5052]: I1203 11:39:12.304028 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2204f9d2-fb11-429c-808f-8fd46881102d-fernet-keys" (OuterVolumeSpecName: "fernet-keys") pod "2204f9d2-fb11-429c-808f-8fd46881102d" (UID: "2204f9d2-fb11-429c-808f-8fd46881102d"). InnerVolumeSpecName "fernet-keys". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:39:12 crc kubenswrapper[5052]: I1203 11:39:12.305790 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2204f9d2-fb11-429c-808f-8fd46881102d-kube-api-access-cxjgj" (OuterVolumeSpecName: "kube-api-access-cxjgj") pod "2204f9d2-fb11-429c-808f-8fd46881102d" (UID: "2204f9d2-fb11-429c-808f-8fd46881102d"). InnerVolumeSpecName "kube-api-access-cxjgj". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:39:12 crc kubenswrapper[5052]: I1203 11:39:12.388307 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2204f9d2-fb11-429c-808f-8fd46881102d-config-data" (OuterVolumeSpecName: "config-data") pod "2204f9d2-fb11-429c-808f-8fd46881102d" (UID: "2204f9d2-fb11-429c-808f-8fd46881102d"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:39:12 crc kubenswrapper[5052]: I1203 11:39:12.389635 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/2204f9d2-fb11-429c-808f-8fd46881102d-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:39:12 crc kubenswrapper[5052]: I1203 11:39:12.389656 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/2204f9d2-fb11-429c-808f-8fd46881102d-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:39:12 crc kubenswrapper[5052]: I1203 11:39:12.389665 5052 reconciler_common.go:293] "Volume detached for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/2204f9d2-fb11-429c-808f-8fd46881102d-fernet-keys\") on node \"crc\" DevicePath \"\"" Dec 03 11:39:12 crc kubenswrapper[5052]: I1203 11:39:12.389673 5052 reconciler_common.go:293] "Volume detached for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/2204f9d2-fb11-429c-808f-8fd46881102d-credential-keys\") on node \"crc\" DevicePath \"\"" Dec 03 11:39:12 crc kubenswrapper[5052]: I1203 11:39:12.389682 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-cxjgj\" (UniqueName: \"kubernetes.io/projected/2204f9d2-fb11-429c-808f-8fd46881102d-kube-api-access-cxjgj\") on node \"crc\" DevicePath \"\"" Dec 03 11:39:12 crc kubenswrapper[5052]: I1203 11:39:12.392680 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2204f9d2-fb11-429c-808f-8fd46881102d-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "2204f9d2-fb11-429c-808f-8fd46881102d" (UID: "2204f9d2-fb11-429c-808f-8fd46881102d"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:39:12 crc kubenswrapper[5052]: I1203 11:39:12.449291 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/keystone-d6bbfcfb4-xdw2x"] Dec 03 11:39:12 crc kubenswrapper[5052]: E1203 11:39:12.449680 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="2204f9d2-fb11-429c-808f-8fd46881102d" containerName="keystone-bootstrap" Dec 03 11:39:12 crc kubenswrapper[5052]: I1203 11:39:12.449692 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="2204f9d2-fb11-429c-808f-8fd46881102d" containerName="keystone-bootstrap" Dec 03 11:39:12 crc kubenswrapper[5052]: I1203 11:39:12.449867 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="2204f9d2-fb11-429c-808f-8fd46881102d" containerName="keystone-bootstrap" Dec 03 11:39:12 crc kubenswrapper[5052]: I1203 11:39:12.450616 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-d6bbfcfb4-xdw2x" Dec 03 11:39:12 crc kubenswrapper[5052]: I1203 11:39:12.459786 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-bootstrap-v8g6k" Dec 03 11:39:12 crc kubenswrapper[5052]: I1203 11:39:12.459912 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-keystone-internal-svc" Dec 03 11:39:12 crc kubenswrapper[5052]: I1203 11:39:12.459925 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-keystone-public-svc" Dec 03 11:39:12 crc kubenswrapper[5052]: I1203 11:39:12.459781 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-bootstrap-v8g6k" event={"ID":"2204f9d2-fb11-429c-808f-8fd46881102d","Type":"ContainerDied","Data":"17810cee57aa4fdda97fb15709dcf7901c9df4796fb99645a79039b518a51059"} Dec 03 11:39:12 crc kubenswrapper[5052]: I1203 11:39:12.460392 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="17810cee57aa4fdda97fb15709dcf7901c9df4796fb99645a79039b518a51059" Dec 03 11:39:12 crc kubenswrapper[5052]: I1203 11:39:12.477684 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-d6bbfcfb4-xdw2x"] Dec 03 11:39:12 crc kubenswrapper[5052]: I1203 11:39:12.491377 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2204f9d2-fb11-429c-808f-8fd46881102d-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:39:12 crc kubenswrapper[5052]: I1203 11:39:12.506337 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-699ddb8646-rfm4p" event={"ID":"0b4b31e5-e82d-4d44-9c8f-d001484293b6","Type":"ContainerStarted","Data":"2289f14f2059fc517e80d4db86044454e7b7fc71a8ff28682e6b78283502038f"} Dec 03 11:39:12 crc kubenswrapper[5052]: I1203 11:39:12.506406 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-699ddb8646-rfm4p" event={"ID":"0b4b31e5-e82d-4d44-9c8f-d001484293b6","Type":"ContainerStarted","Data":"e192664a9c8e1f0804fec134de74214459efa8fd9b22de65ed888db40bd3cb99"} Dec 03 11:39:12 crc kubenswrapper[5052]: I1203 11:39:12.579123 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"502cb03c-d1a5-4dc7-a040-0571a4695379","Type":"ContainerStarted","Data":"a1fa89c15194a6d777274ed0f8572100b4a875ebf55e89ab2de18462f43cc500"} Dec 03 11:39:12 crc kubenswrapper[5052]: I1203 11:39:12.593701 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ksfgw\" (UniqueName: \"kubernetes.io/projected/5737462d-2e5c-4453-9d6c-7d67503555f7-kube-api-access-ksfgw\") pod \"keystone-d6bbfcfb4-xdw2x\" (UID: \"5737462d-2e5c-4453-9d6c-7d67503555f7\") " pod="openstack-kuttl-tests/keystone-d6bbfcfb4-xdw2x" Dec 03 11:39:12 crc kubenswrapper[5052]: I1203 11:39:12.593766 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/5737462d-2e5c-4453-9d6c-7d67503555f7-public-tls-certs\") pod \"keystone-d6bbfcfb4-xdw2x\" (UID: \"5737462d-2e5c-4453-9d6c-7d67503555f7\") " pod="openstack-kuttl-tests/keystone-d6bbfcfb4-xdw2x" Dec 03 11:39:12 crc kubenswrapper[5052]: I1203 11:39:12.593785 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/5737462d-2e5c-4453-9d6c-7d67503555f7-internal-tls-certs\") pod \"keystone-d6bbfcfb4-xdw2x\" (UID: \"5737462d-2e5c-4453-9d6c-7d67503555f7\") " pod="openstack-kuttl-tests/keystone-d6bbfcfb4-xdw2x" Dec 03 11:39:12 crc kubenswrapper[5052]: I1203 11:39:12.593830 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5737462d-2e5c-4453-9d6c-7d67503555f7-combined-ca-bundle\") pod \"keystone-d6bbfcfb4-xdw2x\" (UID: \"5737462d-2e5c-4453-9d6c-7d67503555f7\") " pod="openstack-kuttl-tests/keystone-d6bbfcfb4-xdw2x" Dec 03 11:39:12 crc kubenswrapper[5052]: I1203 11:39:12.593848 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/5737462d-2e5c-4453-9d6c-7d67503555f7-fernet-keys\") pod \"keystone-d6bbfcfb4-xdw2x\" (UID: \"5737462d-2e5c-4453-9d6c-7d67503555f7\") " pod="openstack-kuttl-tests/keystone-d6bbfcfb4-xdw2x" Dec 03 11:39:12 crc kubenswrapper[5052]: I1203 11:39:12.593870 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/5737462d-2e5c-4453-9d6c-7d67503555f7-scripts\") pod \"keystone-d6bbfcfb4-xdw2x\" (UID: \"5737462d-2e5c-4453-9d6c-7d67503555f7\") " pod="openstack-kuttl-tests/keystone-d6bbfcfb4-xdw2x" Dec 03 11:39:12 crc kubenswrapper[5052]: I1203 11:39:12.593890 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/5737462d-2e5c-4453-9d6c-7d67503555f7-credential-keys\") pod \"keystone-d6bbfcfb4-xdw2x\" (UID: \"5737462d-2e5c-4453-9d6c-7d67503555f7\") " pod="openstack-kuttl-tests/keystone-d6bbfcfb4-xdw2x" Dec 03 11:39:12 crc kubenswrapper[5052]: I1203 11:39:12.593921 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/5737462d-2e5c-4453-9d6c-7d67503555f7-config-data\") pod \"keystone-d6bbfcfb4-xdw2x\" (UID: \"5737462d-2e5c-4453-9d6c-7d67503555f7\") " pod="openstack-kuttl-tests/keystone-d6bbfcfb4-xdw2x" Dec 03 11:39:12 crc kubenswrapper[5052]: I1203 11:39:12.619465 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-58dc59ff57-9dsf9" event={"ID":"577c99af-c225-41f3-9d97-e96326a8c1c4","Type":"ContainerStarted","Data":"4973fa976f43a54ac2651aef1f0013167f193a85272c9d8480cdb9ff98c3fe29"} Dec 03 11:39:12 crc kubenswrapper[5052]: I1203 11:39:12.695468 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-ksfgw\" (UniqueName: \"kubernetes.io/projected/5737462d-2e5c-4453-9d6c-7d67503555f7-kube-api-access-ksfgw\") pod \"keystone-d6bbfcfb4-xdw2x\" (UID: \"5737462d-2e5c-4453-9d6c-7d67503555f7\") " pod="openstack-kuttl-tests/keystone-d6bbfcfb4-xdw2x" Dec 03 11:39:12 crc kubenswrapper[5052]: I1203 11:39:12.695628 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/5737462d-2e5c-4453-9d6c-7d67503555f7-public-tls-certs\") pod \"keystone-d6bbfcfb4-xdw2x\" (UID: \"5737462d-2e5c-4453-9d6c-7d67503555f7\") " pod="openstack-kuttl-tests/keystone-d6bbfcfb4-xdw2x" Dec 03 11:39:12 crc kubenswrapper[5052]: I1203 11:39:12.695652 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/5737462d-2e5c-4453-9d6c-7d67503555f7-internal-tls-certs\") pod \"keystone-d6bbfcfb4-xdw2x\" (UID: \"5737462d-2e5c-4453-9d6c-7d67503555f7\") " pod="openstack-kuttl-tests/keystone-d6bbfcfb4-xdw2x" Dec 03 11:39:12 crc kubenswrapper[5052]: I1203 11:39:12.695711 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5737462d-2e5c-4453-9d6c-7d67503555f7-combined-ca-bundle\") pod \"keystone-d6bbfcfb4-xdw2x\" (UID: \"5737462d-2e5c-4453-9d6c-7d67503555f7\") " pod="openstack-kuttl-tests/keystone-d6bbfcfb4-xdw2x" Dec 03 11:39:12 crc kubenswrapper[5052]: I1203 11:39:12.695733 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/5737462d-2e5c-4453-9d6c-7d67503555f7-fernet-keys\") pod \"keystone-d6bbfcfb4-xdw2x\" (UID: \"5737462d-2e5c-4453-9d6c-7d67503555f7\") " pod="openstack-kuttl-tests/keystone-d6bbfcfb4-xdw2x" Dec 03 11:39:12 crc kubenswrapper[5052]: I1203 11:39:12.695761 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/5737462d-2e5c-4453-9d6c-7d67503555f7-scripts\") pod \"keystone-d6bbfcfb4-xdw2x\" (UID: \"5737462d-2e5c-4453-9d6c-7d67503555f7\") " pod="openstack-kuttl-tests/keystone-d6bbfcfb4-xdw2x" Dec 03 11:39:12 crc kubenswrapper[5052]: I1203 11:39:12.695785 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/5737462d-2e5c-4453-9d6c-7d67503555f7-credential-keys\") pod \"keystone-d6bbfcfb4-xdw2x\" (UID: \"5737462d-2e5c-4453-9d6c-7d67503555f7\") " pod="openstack-kuttl-tests/keystone-d6bbfcfb4-xdw2x" Dec 03 11:39:12 crc kubenswrapper[5052]: I1203 11:39:12.695830 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/5737462d-2e5c-4453-9d6c-7d67503555f7-config-data\") pod \"keystone-d6bbfcfb4-xdw2x\" (UID: \"5737462d-2e5c-4453-9d6c-7d67503555f7\") " pod="openstack-kuttl-tests/keystone-d6bbfcfb4-xdw2x" Dec 03 11:39:12 crc kubenswrapper[5052]: I1203 11:39:12.705864 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/5737462d-2e5c-4453-9d6c-7d67503555f7-scripts\") pod \"keystone-d6bbfcfb4-xdw2x\" (UID: \"5737462d-2e5c-4453-9d6c-7d67503555f7\") " pod="openstack-kuttl-tests/keystone-d6bbfcfb4-xdw2x" Dec 03 11:39:12 crc kubenswrapper[5052]: I1203 11:39:12.706427 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5737462d-2e5c-4453-9d6c-7d67503555f7-combined-ca-bundle\") pod \"keystone-d6bbfcfb4-xdw2x\" (UID: \"5737462d-2e5c-4453-9d6c-7d67503555f7\") " pod="openstack-kuttl-tests/keystone-d6bbfcfb4-xdw2x" Dec 03 11:39:12 crc kubenswrapper[5052]: I1203 11:39:12.710835 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/5737462d-2e5c-4453-9d6c-7d67503555f7-credential-keys\") pod \"keystone-d6bbfcfb4-xdw2x\" (UID: \"5737462d-2e5c-4453-9d6c-7d67503555f7\") " pod="openstack-kuttl-tests/keystone-d6bbfcfb4-xdw2x" Dec 03 11:39:12 crc kubenswrapper[5052]: I1203 11:39:12.711280 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/5737462d-2e5c-4453-9d6c-7d67503555f7-public-tls-certs\") pod \"keystone-d6bbfcfb4-xdw2x\" (UID: \"5737462d-2e5c-4453-9d6c-7d67503555f7\") " pod="openstack-kuttl-tests/keystone-d6bbfcfb4-xdw2x" Dec 03 11:39:12 crc kubenswrapper[5052]: I1203 11:39:12.711500 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/5737462d-2e5c-4453-9d6c-7d67503555f7-internal-tls-certs\") pod \"keystone-d6bbfcfb4-xdw2x\" (UID: \"5737462d-2e5c-4453-9d6c-7d67503555f7\") " pod="openstack-kuttl-tests/keystone-d6bbfcfb4-xdw2x" Dec 03 11:39:12 crc kubenswrapper[5052]: I1203 11:39:12.715013 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/5737462d-2e5c-4453-9d6c-7d67503555f7-fernet-keys\") pod \"keystone-d6bbfcfb4-xdw2x\" (UID: \"5737462d-2e5c-4453-9d6c-7d67503555f7\") " pod="openstack-kuttl-tests/keystone-d6bbfcfb4-xdw2x" Dec 03 11:39:12 crc kubenswrapper[5052]: I1203 11:39:12.716274 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/5737462d-2e5c-4453-9d6c-7d67503555f7-config-data\") pod \"keystone-d6bbfcfb4-xdw2x\" (UID: \"5737462d-2e5c-4453-9d6c-7d67503555f7\") " pod="openstack-kuttl-tests/keystone-d6bbfcfb4-xdw2x" Dec 03 11:39:12 crc kubenswrapper[5052]: I1203 11:39:12.721090 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-ksfgw\" (UniqueName: \"kubernetes.io/projected/5737462d-2e5c-4453-9d6c-7d67503555f7-kube-api-access-ksfgw\") pod \"keystone-d6bbfcfb4-xdw2x\" (UID: \"5737462d-2e5c-4453-9d6c-7d67503555f7\") " pod="openstack-kuttl-tests/keystone-d6bbfcfb4-xdw2x" Dec 03 11:39:12 crc kubenswrapper[5052]: I1203 11:39:12.786717 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 11:39:12 crc kubenswrapper[5052]: I1203 11:39:12.798868 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-d6bbfcfb4-xdw2x" Dec 03 11:39:13 crc kubenswrapper[5052]: I1203 11:39:13.368216 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-d6bbfcfb4-xdw2x"] Dec 03 11:39:13 crc kubenswrapper[5052]: I1203 11:39:13.636152 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"502cb03c-d1a5-4dc7-a040-0571a4695379","Type":"ContainerStarted","Data":"33a49358f7fcc1bb28274587ea7ed79d3fcea032502f0292b9e672835cb852c4"} Dec 03 11:39:13 crc kubenswrapper[5052]: I1203 11:39:13.657674 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-58dc59ff57-9dsf9" event={"ID":"577c99af-c225-41f3-9d97-e96326a8c1c4","Type":"ContainerStarted","Data":"a55c9e545d9db9e0eaff80f2fad72700c0a38b5a3dcd4bb8357e991a4e2f25c1"} Dec 03 11:39:13 crc kubenswrapper[5052]: I1203 11:39:13.657757 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-58dc59ff57-9dsf9" event={"ID":"577c99af-c225-41f3-9d97-e96326a8c1c4","Type":"ContainerStarted","Data":"f198dbe5aa735908998d7dc99e4c1016ec4ce99693fe8cdc9b5c3d6597777a05"} Dec 03 11:39:13 crc kubenswrapper[5052]: I1203 11:39:13.657820 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/neutron-58dc59ff57-9dsf9" Dec 03 11:39:13 crc kubenswrapper[5052]: I1203 11:39:13.660202 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"8bf67244-3ae2-4c00-a15d-4c4180d5f835","Type":"ContainerStarted","Data":"b524ed8cfb003b48dcd655e889d08edfc66e061ccb7e6123c9c7505efd3ae39b"} Dec 03 11:39:13 crc kubenswrapper[5052]: I1203 11:39:13.664155 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-699ddb8646-rfm4p" event={"ID":"0b4b31e5-e82d-4d44-9c8f-d001484293b6","Type":"ContainerStarted","Data":"77dd24bffc9318ae0172000a2a92f5dd4f72dcfd986df49b78a78df630d16741"} Dec 03 11:39:13 crc kubenswrapper[5052]: I1203 11:39:13.665071 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/placement-699ddb8646-rfm4p" Dec 03 11:39:13 crc kubenswrapper[5052]: I1203 11:39:13.665108 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/placement-699ddb8646-rfm4p" Dec 03 11:39:13 crc kubenswrapper[5052]: I1203 11:39:13.666624 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-d6bbfcfb4-xdw2x" event={"ID":"5737462d-2e5c-4453-9d6c-7d67503555f7","Type":"ContainerStarted","Data":"70de95a1cec1fd36a398a2fe13eb11bec2874f8487ab6c53add1cdcc12338f59"} Dec 03 11:39:13 crc kubenswrapper[5052]: I1203 11:39:13.699479 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/neutron-58dc59ff57-9dsf9" podStartSLOduration=2.699461253 podStartE2EDuration="2.699461253s" podCreationTimestamp="2025-12-03 11:39:11 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:39:13.692929454 +0000 UTC m=+1144.281171228" watchObservedRunningTime="2025-12-03 11:39:13.699461253 +0000 UTC m=+1144.287703017" Dec 03 11:39:13 crc kubenswrapper[5052]: I1203 11:39:13.721036 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/placement-699ddb8646-rfm4p" podStartSLOduration=3.7210132849999997 podStartE2EDuration="3.721013285s" podCreationTimestamp="2025-12-03 11:39:10 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:39:13.711889392 +0000 UTC m=+1144.300131156" watchObservedRunningTime="2025-12-03 11:39:13.721013285 +0000 UTC m=+1144.309255049" Dec 03 11:39:14 crc kubenswrapper[5052]: I1203 11:39:14.195410 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/neutron-75475cc596-bfbzz"] Dec 03 11:39:14 crc kubenswrapper[5052]: I1203 11:39:14.198257 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-75475cc596-bfbzz" Dec 03 11:39:14 crc kubenswrapper[5052]: I1203 11:39:14.212854 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-neutron-internal-svc" Dec 03 11:39:14 crc kubenswrapper[5052]: I1203 11:39:14.213170 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-neutron-public-svc" Dec 03 11:39:14 crc kubenswrapper[5052]: I1203 11:39:14.253442 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-75475cc596-bfbzz"] Dec 03 11:39:14 crc kubenswrapper[5052]: I1203 11:39:14.346236 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4a23886b-73c6-4b21-8464-07a2dd2e06f6-combined-ca-bundle\") pod \"neutron-75475cc596-bfbzz\" (UID: \"4a23886b-73c6-4b21-8464-07a2dd2e06f6\") " pod="openstack-kuttl-tests/neutron-75475cc596-bfbzz" Dec 03 11:39:14 crc kubenswrapper[5052]: I1203 11:39:14.346347 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/4a23886b-73c6-4b21-8464-07a2dd2e06f6-ovndb-tls-certs\") pod \"neutron-75475cc596-bfbzz\" (UID: \"4a23886b-73c6-4b21-8464-07a2dd2e06f6\") " pod="openstack-kuttl-tests/neutron-75475cc596-bfbzz" Dec 03 11:39:14 crc kubenswrapper[5052]: I1203 11:39:14.346407 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/4a23886b-73c6-4b21-8464-07a2dd2e06f6-config\") pod \"neutron-75475cc596-bfbzz\" (UID: \"4a23886b-73c6-4b21-8464-07a2dd2e06f6\") " pod="openstack-kuttl-tests/neutron-75475cc596-bfbzz" Dec 03 11:39:14 crc kubenswrapper[5052]: I1203 11:39:14.346472 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/4a23886b-73c6-4b21-8464-07a2dd2e06f6-internal-tls-certs\") pod \"neutron-75475cc596-bfbzz\" (UID: \"4a23886b-73c6-4b21-8464-07a2dd2e06f6\") " pod="openstack-kuttl-tests/neutron-75475cc596-bfbzz" Dec 03 11:39:14 crc kubenswrapper[5052]: I1203 11:39:14.346541 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/4a23886b-73c6-4b21-8464-07a2dd2e06f6-httpd-config\") pod \"neutron-75475cc596-bfbzz\" (UID: \"4a23886b-73c6-4b21-8464-07a2dd2e06f6\") " pod="openstack-kuttl-tests/neutron-75475cc596-bfbzz" Dec 03 11:39:14 crc kubenswrapper[5052]: I1203 11:39:14.346749 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/4a23886b-73c6-4b21-8464-07a2dd2e06f6-public-tls-certs\") pod \"neutron-75475cc596-bfbzz\" (UID: \"4a23886b-73c6-4b21-8464-07a2dd2e06f6\") " pod="openstack-kuttl-tests/neutron-75475cc596-bfbzz" Dec 03 11:39:14 crc kubenswrapper[5052]: I1203 11:39:14.346835 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zrlxq\" (UniqueName: \"kubernetes.io/projected/4a23886b-73c6-4b21-8464-07a2dd2e06f6-kube-api-access-zrlxq\") pod \"neutron-75475cc596-bfbzz\" (UID: \"4a23886b-73c6-4b21-8464-07a2dd2e06f6\") " pod="openstack-kuttl-tests/neutron-75475cc596-bfbzz" Dec 03 11:39:14 crc kubenswrapper[5052]: I1203 11:39:14.448618 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4a23886b-73c6-4b21-8464-07a2dd2e06f6-combined-ca-bundle\") pod \"neutron-75475cc596-bfbzz\" (UID: \"4a23886b-73c6-4b21-8464-07a2dd2e06f6\") " pod="openstack-kuttl-tests/neutron-75475cc596-bfbzz" Dec 03 11:39:14 crc kubenswrapper[5052]: I1203 11:39:14.448672 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/4a23886b-73c6-4b21-8464-07a2dd2e06f6-ovndb-tls-certs\") pod \"neutron-75475cc596-bfbzz\" (UID: \"4a23886b-73c6-4b21-8464-07a2dd2e06f6\") " pod="openstack-kuttl-tests/neutron-75475cc596-bfbzz" Dec 03 11:39:14 crc kubenswrapper[5052]: I1203 11:39:14.448691 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/4a23886b-73c6-4b21-8464-07a2dd2e06f6-config\") pod \"neutron-75475cc596-bfbzz\" (UID: \"4a23886b-73c6-4b21-8464-07a2dd2e06f6\") " pod="openstack-kuttl-tests/neutron-75475cc596-bfbzz" Dec 03 11:39:14 crc kubenswrapper[5052]: I1203 11:39:14.448717 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/4a23886b-73c6-4b21-8464-07a2dd2e06f6-internal-tls-certs\") pod \"neutron-75475cc596-bfbzz\" (UID: \"4a23886b-73c6-4b21-8464-07a2dd2e06f6\") " pod="openstack-kuttl-tests/neutron-75475cc596-bfbzz" Dec 03 11:39:14 crc kubenswrapper[5052]: I1203 11:39:14.448738 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/4a23886b-73c6-4b21-8464-07a2dd2e06f6-httpd-config\") pod \"neutron-75475cc596-bfbzz\" (UID: \"4a23886b-73c6-4b21-8464-07a2dd2e06f6\") " pod="openstack-kuttl-tests/neutron-75475cc596-bfbzz" Dec 03 11:39:14 crc kubenswrapper[5052]: I1203 11:39:14.448755 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/4a23886b-73c6-4b21-8464-07a2dd2e06f6-public-tls-certs\") pod \"neutron-75475cc596-bfbzz\" (UID: \"4a23886b-73c6-4b21-8464-07a2dd2e06f6\") " pod="openstack-kuttl-tests/neutron-75475cc596-bfbzz" Dec 03 11:39:14 crc kubenswrapper[5052]: I1203 11:39:14.448786 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-zrlxq\" (UniqueName: \"kubernetes.io/projected/4a23886b-73c6-4b21-8464-07a2dd2e06f6-kube-api-access-zrlxq\") pod \"neutron-75475cc596-bfbzz\" (UID: \"4a23886b-73c6-4b21-8464-07a2dd2e06f6\") " pod="openstack-kuttl-tests/neutron-75475cc596-bfbzz" Dec 03 11:39:14 crc kubenswrapper[5052]: I1203 11:39:14.454417 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4a23886b-73c6-4b21-8464-07a2dd2e06f6-combined-ca-bundle\") pod \"neutron-75475cc596-bfbzz\" (UID: \"4a23886b-73c6-4b21-8464-07a2dd2e06f6\") " pod="openstack-kuttl-tests/neutron-75475cc596-bfbzz" Dec 03 11:39:14 crc kubenswrapper[5052]: I1203 11:39:14.455282 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/4a23886b-73c6-4b21-8464-07a2dd2e06f6-internal-tls-certs\") pod \"neutron-75475cc596-bfbzz\" (UID: \"4a23886b-73c6-4b21-8464-07a2dd2e06f6\") " pod="openstack-kuttl-tests/neutron-75475cc596-bfbzz" Dec 03 11:39:14 crc kubenswrapper[5052]: I1203 11:39:14.456172 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/4a23886b-73c6-4b21-8464-07a2dd2e06f6-ovndb-tls-certs\") pod \"neutron-75475cc596-bfbzz\" (UID: \"4a23886b-73c6-4b21-8464-07a2dd2e06f6\") " pod="openstack-kuttl-tests/neutron-75475cc596-bfbzz" Dec 03 11:39:14 crc kubenswrapper[5052]: I1203 11:39:14.457293 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/4a23886b-73c6-4b21-8464-07a2dd2e06f6-public-tls-certs\") pod \"neutron-75475cc596-bfbzz\" (UID: \"4a23886b-73c6-4b21-8464-07a2dd2e06f6\") " pod="openstack-kuttl-tests/neutron-75475cc596-bfbzz" Dec 03 11:39:14 crc kubenswrapper[5052]: I1203 11:39:14.457817 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/4a23886b-73c6-4b21-8464-07a2dd2e06f6-config\") pod \"neutron-75475cc596-bfbzz\" (UID: \"4a23886b-73c6-4b21-8464-07a2dd2e06f6\") " pod="openstack-kuttl-tests/neutron-75475cc596-bfbzz" Dec 03 11:39:14 crc kubenswrapper[5052]: I1203 11:39:14.466206 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/4a23886b-73c6-4b21-8464-07a2dd2e06f6-httpd-config\") pod \"neutron-75475cc596-bfbzz\" (UID: \"4a23886b-73c6-4b21-8464-07a2dd2e06f6\") " pod="openstack-kuttl-tests/neutron-75475cc596-bfbzz" Dec 03 11:39:14 crc kubenswrapper[5052]: I1203 11:39:14.466760 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-zrlxq\" (UniqueName: \"kubernetes.io/projected/4a23886b-73c6-4b21-8464-07a2dd2e06f6-kube-api-access-zrlxq\") pod \"neutron-75475cc596-bfbzz\" (UID: \"4a23886b-73c6-4b21-8464-07a2dd2e06f6\") " pod="openstack-kuttl-tests/neutron-75475cc596-bfbzz" Dec 03 11:39:14 crc kubenswrapper[5052]: I1203 11:39:14.579899 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-75475cc596-bfbzz" Dec 03 11:39:14 crc kubenswrapper[5052]: I1203 11:39:14.680978 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"8bf67244-3ae2-4c00-a15d-4c4180d5f835","Type":"ContainerStarted","Data":"26f247c9c6d8664d070a0e01318f861102e07448fbac21823c5f2aca4b6f5da2"} Dec 03 11:39:14 crc kubenswrapper[5052]: I1203 11:39:14.683540 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-d6bbfcfb4-xdw2x" event={"ID":"5737462d-2e5c-4453-9d6c-7d67503555f7","Type":"ContainerStarted","Data":"85486efe955265533bc0eee17583150138fcae470d553a61d68bb5477eb454dd"} Dec 03 11:39:14 crc kubenswrapper[5052]: I1203 11:39:14.684340 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/keystone-d6bbfcfb4-xdw2x" Dec 03 11:39:14 crc kubenswrapper[5052]: I1203 11:39:14.687272 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"502cb03c-d1a5-4dc7-a040-0571a4695379","Type":"ContainerStarted","Data":"3c6d3ddfe8759dec5d3030b3c1b64bcbc9f4228dfeaac36309751c177de0b102"} Dec 03 11:39:14 crc kubenswrapper[5052]: I1203 11:39:14.710810 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/keystone-d6bbfcfb4-xdw2x" podStartSLOduration=2.710792427 podStartE2EDuration="2.710792427s" podCreationTimestamp="2025-12-03 11:39:12 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:39:14.707690447 +0000 UTC m=+1145.295932211" watchObservedRunningTime="2025-12-03 11:39:14.710792427 +0000 UTC m=+1145.299034191" Dec 03 11:39:14 crc kubenswrapper[5052]: I1203 11:39:14.729662 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/glance-default-internal-api-0" podStartSLOduration=4.729639411 podStartE2EDuration="4.729639411s" podCreationTimestamp="2025-12-03 11:39:10 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:39:14.727259462 +0000 UTC m=+1145.315501236" watchObservedRunningTime="2025-12-03 11:39:14.729639411 +0000 UTC m=+1145.317881195" Dec 03 11:39:19 crc kubenswrapper[5052]: I1203 11:39:19.108584 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-75475cc596-bfbzz"] Dec 03 11:39:19 crc kubenswrapper[5052]: I1203 11:39:19.739325 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"8bf67244-3ae2-4c00-a15d-4c4180d5f835","Type":"ContainerStarted","Data":"dcd58ef3b45066b82652446bcbd875e2abf4758e96cd73b62fededb0469b531e"} Dec 03 11:39:19 crc kubenswrapper[5052]: I1203 11:39:19.769461 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/glance-default-external-api-0" podStartSLOduration=8.769437427 podStartE2EDuration="8.769437427s" podCreationTimestamp="2025-12-03 11:39:11 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:39:19.760360725 +0000 UTC m=+1150.348602489" watchObservedRunningTime="2025-12-03 11:39:19.769437427 +0000 UTC m=+1150.357679191" Dec 03 11:39:21 crc kubenswrapper[5052]: I1203 11:39:21.124414 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:39:21 crc kubenswrapper[5052]: I1203 11:39:21.124757 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:39:21 crc kubenswrapper[5052]: I1203 11:39:21.164698 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:39:21 crc kubenswrapper[5052]: I1203 11:39:21.170447 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:39:22 crc kubenswrapper[5052]: I1203 11:39:22.069901 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:39:22 crc kubenswrapper[5052]: I1203 11:39:22.069946 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:39:22 crc kubenswrapper[5052]: I1203 11:39:22.171788 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:39:22 crc kubenswrapper[5052]: I1203 11:39:22.171833 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:39:22 crc kubenswrapper[5052]: I1203 11:39:22.240845 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:39:22 crc kubenswrapper[5052]: I1203 11:39:22.255359 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:39:23 crc kubenswrapper[5052]: I1203 11:39:23.076873 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:39:23 crc kubenswrapper[5052]: I1203 11:39:23.077244 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:39:23 crc kubenswrapper[5052]: W1203 11:39:23.837749 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4a23886b_73c6_4b21_8464_07a2dd2e06f6.slice/crio-018553d3b157f7ef0dd01fa9a27025af3476387607cfe61f6fb65382fd109a20 WatchSource:0}: Error finding container 018553d3b157f7ef0dd01fa9a27025af3476387607cfe61f6fb65382fd109a20: Status 404 returned error can't find the container with id 018553d3b157f7ef0dd01fa9a27025af3476387607cfe61f6fb65382fd109a20 Dec 03 11:39:24 crc kubenswrapper[5052]: I1203 11:39:24.090557 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-75475cc596-bfbzz" event={"ID":"4a23886b-73c6-4b21-8464-07a2dd2e06f6","Type":"ContainerStarted","Data":"018553d3b157f7ef0dd01fa9a27025af3476387607cfe61f6fb65382fd109a20"} Dec 03 11:39:24 crc kubenswrapper[5052]: I1203 11:39:24.104830 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:39:24 crc kubenswrapper[5052]: I1203 11:39:24.104917 5052 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Dec 03 11:39:24 crc kubenswrapper[5052]: I1203 11:39:24.128086 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:39:25 crc kubenswrapper[5052]: E1203 11:39:25.220717 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"ceilometer-central-agent\" with ErrImagePull: \"rpc error: code = Canceled desc = copying config: context canceled\"" pod="openstack-kuttl-tests/ceilometer-0" podUID="c43f59f2-deb7-46ee-8e07-54e3ff9f59aa" Dec 03 11:39:25 crc kubenswrapper[5052]: I1203 11:39:25.407492 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:39:25 crc kubenswrapper[5052]: I1203 11:39:25.407881 5052 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Dec 03 11:39:25 crc kubenswrapper[5052]: I1203 11:39:25.660162 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:39:26 crc kubenswrapper[5052]: I1203 11:39:26.108872 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-75475cc596-bfbzz" event={"ID":"4a23886b-73c6-4b21-8464-07a2dd2e06f6","Type":"ContainerStarted","Data":"f284580bf78ee1c66dcbbafaa358e1788c2f65a743bb73ce1b43008a67f4e28c"} Dec 03 11:39:26 crc kubenswrapper[5052]: I1203 11:39:26.108915 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-75475cc596-bfbzz" event={"ID":"4a23886b-73c6-4b21-8464-07a2dd2e06f6","Type":"ContainerStarted","Data":"0aea4f7a00ba159f8d2bf6070cfb3c8fd85b362b1295e3d63d6e0afddf0002dd"} Dec 03 11:39:26 crc kubenswrapper[5052]: I1203 11:39:26.109007 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/neutron-75475cc596-bfbzz" Dec 03 11:39:26 crc kubenswrapper[5052]: I1203 11:39:26.111229 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"c43f59f2-deb7-46ee-8e07-54e3ff9f59aa","Type":"ContainerStarted","Data":"427ddc970b17ff9ab4b4ac2cd086a50fd2a7fb9c51e4f969ae3d4b5bdf13c23a"} Dec 03 11:39:26 crc kubenswrapper[5052]: I1203 11:39:26.111378 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:39:26 crc kubenswrapper[5052]: I1203 11:39:26.111394 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="c43f59f2-deb7-46ee-8e07-54e3ff9f59aa" containerName="ceilometer-notification-agent" containerID="cri-o://740b963fb25833c1ae839f5909691da25e2203dbabc6e50ecaeefd885b130f22" gracePeriod=30 Dec 03 11:39:26 crc kubenswrapper[5052]: I1203 11:39:26.111459 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="c43f59f2-deb7-46ee-8e07-54e3ff9f59aa" containerName="sg-core" containerID="cri-o://2eba4f00d60fae5fc479684e63a7bbddd03d0a038f2ae23cb78da9235c80d7d6" gracePeriod=30 Dec 03 11:39:26 crc kubenswrapper[5052]: I1203 11:39:26.111440 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="c43f59f2-deb7-46ee-8e07-54e3ff9f59aa" containerName="proxy-httpd" containerID="cri-o://427ddc970b17ff9ab4b4ac2cd086a50fd2a7fb9c51e4f969ae3d4b5bdf13c23a" gracePeriod=30 Dec 03 11:39:26 crc kubenswrapper[5052]: I1203 11:39:26.115092 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-db-sync-pbw4k" event={"ID":"d1676789-356e-4853-a302-f2c5bd3f446f","Type":"ContainerStarted","Data":"1820ff11c6142cf8027dde668e11d985d56d48b18de14c51c0337c5af52c8909"} Dec 03 11:39:26 crc kubenswrapper[5052]: I1203 11:39:26.117981 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-db-sync-bfv74" event={"ID":"1dc19382-24a8-4a4f-b735-d794619492bf","Type":"ContainerStarted","Data":"36337ab63eb8262535e7e16ba49f89375f566b548bb4c5bf36fb9e7e0605e1c7"} Dec 03 11:39:26 crc kubenswrapper[5052]: I1203 11:39:26.136280 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/neutron-75475cc596-bfbzz" podStartSLOduration=12.136255791 podStartE2EDuration="12.136255791s" podCreationTimestamp="2025-12-03 11:39:14 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:39:26.12685609 +0000 UTC m=+1156.715097864" watchObservedRunningTime="2025-12-03 11:39:26.136255791 +0000 UTC m=+1156.724497555" Dec 03 11:39:26 crc kubenswrapper[5052]: I1203 11:39:26.171950 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/barbican-db-sync-bfv74" podStartSLOduration=2.820534799 podStartE2EDuration="44.171936281s" podCreationTimestamp="2025-12-03 11:38:42 +0000 UTC" firstStartedPulling="2025-12-03 11:38:43.535941397 +0000 UTC m=+1114.124183161" lastFinishedPulling="2025-12-03 11:39:24.887342879 +0000 UTC m=+1155.475584643" observedRunningTime="2025-12-03 11:39:26.171095207 +0000 UTC m=+1156.759336981" watchObservedRunningTime="2025-12-03 11:39:26.171936281 +0000 UTC m=+1156.760178045" Dec 03 11:39:26 crc kubenswrapper[5052]: I1203 11:39:26.196967 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/cinder-db-sync-pbw4k" podStartSLOduration=2.49185408 podStartE2EDuration="44.196949083s" podCreationTimestamp="2025-12-03 11:38:42 +0000 UTC" firstStartedPulling="2025-12-03 11:38:43.181739961 +0000 UTC m=+1113.769981725" lastFinishedPulling="2025-12-03 11:39:24.886834944 +0000 UTC m=+1155.475076728" observedRunningTime="2025-12-03 11:39:26.193465123 +0000 UTC m=+1156.781706907" watchObservedRunningTime="2025-12-03 11:39:26.196949083 +0000 UTC m=+1156.785190847" Dec 03 11:39:27 crc kubenswrapper[5052]: I1203 11:39:27.126348 5052 generic.go:334] "Generic (PLEG): container finished" podID="c43f59f2-deb7-46ee-8e07-54e3ff9f59aa" containerID="427ddc970b17ff9ab4b4ac2cd086a50fd2a7fb9c51e4f969ae3d4b5bdf13c23a" exitCode=0 Dec 03 11:39:27 crc kubenswrapper[5052]: I1203 11:39:27.126694 5052 generic.go:334] "Generic (PLEG): container finished" podID="c43f59f2-deb7-46ee-8e07-54e3ff9f59aa" containerID="2eba4f00d60fae5fc479684e63a7bbddd03d0a038f2ae23cb78da9235c80d7d6" exitCode=2 Dec 03 11:39:27 crc kubenswrapper[5052]: I1203 11:39:27.126419 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"c43f59f2-deb7-46ee-8e07-54e3ff9f59aa","Type":"ContainerDied","Data":"427ddc970b17ff9ab4b4ac2cd086a50fd2a7fb9c51e4f969ae3d4b5bdf13c23a"} Dec 03 11:39:27 crc kubenswrapper[5052]: I1203 11:39:27.126819 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"c43f59f2-deb7-46ee-8e07-54e3ff9f59aa","Type":"ContainerDied","Data":"2eba4f00d60fae5fc479684e63a7bbddd03d0a038f2ae23cb78da9235c80d7d6"} Dec 03 11:39:28 crc kubenswrapper[5052]: I1203 11:39:28.136340 5052 generic.go:334] "Generic (PLEG): container finished" podID="1dc19382-24a8-4a4f-b735-d794619492bf" containerID="36337ab63eb8262535e7e16ba49f89375f566b548bb4c5bf36fb9e7e0605e1c7" exitCode=0 Dec 03 11:39:28 crc kubenswrapper[5052]: I1203 11:39:28.136388 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-db-sync-bfv74" event={"ID":"1dc19382-24a8-4a4f-b735-d794619492bf","Type":"ContainerDied","Data":"36337ab63eb8262535e7e16ba49f89375f566b548bb4c5bf36fb9e7e0605e1c7"} Dec 03 11:39:29 crc kubenswrapper[5052]: I1203 11:39:29.128102 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:39:29 crc kubenswrapper[5052]: I1203 11:39:29.150946 5052 generic.go:334] "Generic (PLEG): container finished" podID="c43f59f2-deb7-46ee-8e07-54e3ff9f59aa" containerID="740b963fb25833c1ae839f5909691da25e2203dbabc6e50ecaeefd885b130f22" exitCode=0 Dec 03 11:39:29 crc kubenswrapper[5052]: I1203 11:39:29.151005 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:39:29 crc kubenswrapper[5052]: I1203 11:39:29.151041 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"c43f59f2-deb7-46ee-8e07-54e3ff9f59aa","Type":"ContainerDied","Data":"740b963fb25833c1ae839f5909691da25e2203dbabc6e50ecaeefd885b130f22"} Dec 03 11:39:29 crc kubenswrapper[5052]: I1203 11:39:29.151102 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"c43f59f2-deb7-46ee-8e07-54e3ff9f59aa","Type":"ContainerDied","Data":"465b7c2dbcbc8a63d5b3400d3cacd0b29170f695e867e401f0b9fbb2dd63587f"} Dec 03 11:39:29 crc kubenswrapper[5052]: I1203 11:39:29.151130 5052 scope.go:117] "RemoveContainer" containerID="427ddc970b17ff9ab4b4ac2cd086a50fd2a7fb9c51e4f969ae3d4b5bdf13c23a" Dec 03 11:39:29 crc kubenswrapper[5052]: I1203 11:39:29.177752 5052 scope.go:117] "RemoveContainer" containerID="2eba4f00d60fae5fc479684e63a7bbddd03d0a038f2ae23cb78da9235c80d7d6" Dec 03 11:39:29 crc kubenswrapper[5052]: I1203 11:39:29.195661 5052 scope.go:117] "RemoveContainer" containerID="740b963fb25833c1ae839f5909691da25e2203dbabc6e50ecaeefd885b130f22" Dec 03 11:39:29 crc kubenswrapper[5052]: I1203 11:39:29.216569 5052 scope.go:117] "RemoveContainer" containerID="427ddc970b17ff9ab4b4ac2cd086a50fd2a7fb9c51e4f969ae3d4b5bdf13c23a" Dec 03 11:39:29 crc kubenswrapper[5052]: E1203 11:39:29.216922 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"427ddc970b17ff9ab4b4ac2cd086a50fd2a7fb9c51e4f969ae3d4b5bdf13c23a\": container with ID starting with 427ddc970b17ff9ab4b4ac2cd086a50fd2a7fb9c51e4f969ae3d4b5bdf13c23a not found: ID does not exist" containerID="427ddc970b17ff9ab4b4ac2cd086a50fd2a7fb9c51e4f969ae3d4b5bdf13c23a" Dec 03 11:39:29 crc kubenswrapper[5052]: I1203 11:39:29.216952 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"427ddc970b17ff9ab4b4ac2cd086a50fd2a7fb9c51e4f969ae3d4b5bdf13c23a"} err="failed to get container status \"427ddc970b17ff9ab4b4ac2cd086a50fd2a7fb9c51e4f969ae3d4b5bdf13c23a\": rpc error: code = NotFound desc = could not find container \"427ddc970b17ff9ab4b4ac2cd086a50fd2a7fb9c51e4f969ae3d4b5bdf13c23a\": container with ID starting with 427ddc970b17ff9ab4b4ac2cd086a50fd2a7fb9c51e4f969ae3d4b5bdf13c23a not found: ID does not exist" Dec 03 11:39:29 crc kubenswrapper[5052]: I1203 11:39:29.216972 5052 scope.go:117] "RemoveContainer" containerID="2eba4f00d60fae5fc479684e63a7bbddd03d0a038f2ae23cb78da9235c80d7d6" Dec 03 11:39:29 crc kubenswrapper[5052]: E1203 11:39:29.217263 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2eba4f00d60fae5fc479684e63a7bbddd03d0a038f2ae23cb78da9235c80d7d6\": container with ID starting with 2eba4f00d60fae5fc479684e63a7bbddd03d0a038f2ae23cb78da9235c80d7d6 not found: ID does not exist" containerID="2eba4f00d60fae5fc479684e63a7bbddd03d0a038f2ae23cb78da9235c80d7d6" Dec 03 11:39:29 crc kubenswrapper[5052]: I1203 11:39:29.217281 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2eba4f00d60fae5fc479684e63a7bbddd03d0a038f2ae23cb78da9235c80d7d6"} err="failed to get container status \"2eba4f00d60fae5fc479684e63a7bbddd03d0a038f2ae23cb78da9235c80d7d6\": rpc error: code = NotFound desc = could not find container \"2eba4f00d60fae5fc479684e63a7bbddd03d0a038f2ae23cb78da9235c80d7d6\": container with ID starting with 2eba4f00d60fae5fc479684e63a7bbddd03d0a038f2ae23cb78da9235c80d7d6 not found: ID does not exist" Dec 03 11:39:29 crc kubenswrapper[5052]: I1203 11:39:29.217294 5052 scope.go:117] "RemoveContainer" containerID="740b963fb25833c1ae839f5909691da25e2203dbabc6e50ecaeefd885b130f22" Dec 03 11:39:29 crc kubenswrapper[5052]: E1203 11:39:29.217482 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"740b963fb25833c1ae839f5909691da25e2203dbabc6e50ecaeefd885b130f22\": container with ID starting with 740b963fb25833c1ae839f5909691da25e2203dbabc6e50ecaeefd885b130f22 not found: ID does not exist" containerID="740b963fb25833c1ae839f5909691da25e2203dbabc6e50ecaeefd885b130f22" Dec 03 11:39:29 crc kubenswrapper[5052]: I1203 11:39:29.217503 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"740b963fb25833c1ae839f5909691da25e2203dbabc6e50ecaeefd885b130f22"} err="failed to get container status \"740b963fb25833c1ae839f5909691da25e2203dbabc6e50ecaeefd885b130f22\": rpc error: code = NotFound desc = could not find container \"740b963fb25833c1ae839f5909691da25e2203dbabc6e50ecaeefd885b130f22\": container with ID starting with 740b963fb25833c1ae839f5909691da25e2203dbabc6e50ecaeefd885b130f22 not found: ID does not exist" Dec 03 11:39:29 crc kubenswrapper[5052]: I1203 11:39:29.276197 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/c43f59f2-deb7-46ee-8e07-54e3ff9f59aa-sg-core-conf-yaml\") pod \"c43f59f2-deb7-46ee-8e07-54e3ff9f59aa\" (UID: \"c43f59f2-deb7-46ee-8e07-54e3ff9f59aa\") " Dec 03 11:39:29 crc kubenswrapper[5052]: I1203 11:39:29.276315 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h8zbz\" (UniqueName: \"kubernetes.io/projected/c43f59f2-deb7-46ee-8e07-54e3ff9f59aa-kube-api-access-h8zbz\") pod \"c43f59f2-deb7-46ee-8e07-54e3ff9f59aa\" (UID: \"c43f59f2-deb7-46ee-8e07-54e3ff9f59aa\") " Dec 03 11:39:29 crc kubenswrapper[5052]: I1203 11:39:29.276370 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/c43f59f2-deb7-46ee-8e07-54e3ff9f59aa-scripts\") pod \"c43f59f2-deb7-46ee-8e07-54e3ff9f59aa\" (UID: \"c43f59f2-deb7-46ee-8e07-54e3ff9f59aa\") " Dec 03 11:39:29 crc kubenswrapper[5052]: I1203 11:39:29.276408 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/c43f59f2-deb7-46ee-8e07-54e3ff9f59aa-run-httpd\") pod \"c43f59f2-deb7-46ee-8e07-54e3ff9f59aa\" (UID: \"c43f59f2-deb7-46ee-8e07-54e3ff9f59aa\") " Dec 03 11:39:29 crc kubenswrapper[5052]: I1203 11:39:29.276458 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c43f59f2-deb7-46ee-8e07-54e3ff9f59aa-combined-ca-bundle\") pod \"c43f59f2-deb7-46ee-8e07-54e3ff9f59aa\" (UID: \"c43f59f2-deb7-46ee-8e07-54e3ff9f59aa\") " Dec 03 11:39:29 crc kubenswrapper[5052]: I1203 11:39:29.276530 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/c43f59f2-deb7-46ee-8e07-54e3ff9f59aa-config-data\") pod \"c43f59f2-deb7-46ee-8e07-54e3ff9f59aa\" (UID: \"c43f59f2-deb7-46ee-8e07-54e3ff9f59aa\") " Dec 03 11:39:29 crc kubenswrapper[5052]: I1203 11:39:29.276563 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/c43f59f2-deb7-46ee-8e07-54e3ff9f59aa-log-httpd\") pod \"c43f59f2-deb7-46ee-8e07-54e3ff9f59aa\" (UID: \"c43f59f2-deb7-46ee-8e07-54e3ff9f59aa\") " Dec 03 11:39:29 crc kubenswrapper[5052]: I1203 11:39:29.277732 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c43f59f2-deb7-46ee-8e07-54e3ff9f59aa-log-httpd" (OuterVolumeSpecName: "log-httpd") pod "c43f59f2-deb7-46ee-8e07-54e3ff9f59aa" (UID: "c43f59f2-deb7-46ee-8e07-54e3ff9f59aa"). InnerVolumeSpecName "log-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:39:29 crc kubenswrapper[5052]: I1203 11:39:29.278643 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c43f59f2-deb7-46ee-8e07-54e3ff9f59aa-run-httpd" (OuterVolumeSpecName: "run-httpd") pod "c43f59f2-deb7-46ee-8e07-54e3ff9f59aa" (UID: "c43f59f2-deb7-46ee-8e07-54e3ff9f59aa"). InnerVolumeSpecName "run-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:39:29 crc kubenswrapper[5052]: I1203 11:39:29.283541 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c43f59f2-deb7-46ee-8e07-54e3ff9f59aa-scripts" (OuterVolumeSpecName: "scripts") pod "c43f59f2-deb7-46ee-8e07-54e3ff9f59aa" (UID: "c43f59f2-deb7-46ee-8e07-54e3ff9f59aa"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:39:29 crc kubenswrapper[5052]: I1203 11:39:29.297212 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c43f59f2-deb7-46ee-8e07-54e3ff9f59aa-kube-api-access-h8zbz" (OuterVolumeSpecName: "kube-api-access-h8zbz") pod "c43f59f2-deb7-46ee-8e07-54e3ff9f59aa" (UID: "c43f59f2-deb7-46ee-8e07-54e3ff9f59aa"). InnerVolumeSpecName "kube-api-access-h8zbz". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:39:29 crc kubenswrapper[5052]: I1203 11:39:29.313765 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c43f59f2-deb7-46ee-8e07-54e3ff9f59aa-sg-core-conf-yaml" (OuterVolumeSpecName: "sg-core-conf-yaml") pod "c43f59f2-deb7-46ee-8e07-54e3ff9f59aa" (UID: "c43f59f2-deb7-46ee-8e07-54e3ff9f59aa"). InnerVolumeSpecName "sg-core-conf-yaml". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:39:29 crc kubenswrapper[5052]: I1203 11:39:29.365667 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c43f59f2-deb7-46ee-8e07-54e3ff9f59aa-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "c43f59f2-deb7-46ee-8e07-54e3ff9f59aa" (UID: "c43f59f2-deb7-46ee-8e07-54e3ff9f59aa"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:39:29 crc kubenswrapper[5052]: I1203 11:39:29.377992 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/c43f59f2-deb7-46ee-8e07-54e3ff9f59aa-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:39:29 crc kubenswrapper[5052]: I1203 11:39:29.378021 5052 reconciler_common.go:293] "Volume detached for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/c43f59f2-deb7-46ee-8e07-54e3ff9f59aa-run-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 11:39:29 crc kubenswrapper[5052]: I1203 11:39:29.378031 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c43f59f2-deb7-46ee-8e07-54e3ff9f59aa-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:39:29 crc kubenswrapper[5052]: I1203 11:39:29.378041 5052 reconciler_common.go:293] "Volume detached for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/c43f59f2-deb7-46ee-8e07-54e3ff9f59aa-log-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 11:39:29 crc kubenswrapper[5052]: I1203 11:39:29.378050 5052 reconciler_common.go:293] "Volume detached for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/c43f59f2-deb7-46ee-8e07-54e3ff9f59aa-sg-core-conf-yaml\") on node \"crc\" DevicePath \"\"" Dec 03 11:39:29 crc kubenswrapper[5052]: I1203 11:39:29.378059 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-h8zbz\" (UniqueName: \"kubernetes.io/projected/c43f59f2-deb7-46ee-8e07-54e3ff9f59aa-kube-api-access-h8zbz\") on node \"crc\" DevicePath \"\"" Dec 03 11:39:29 crc kubenswrapper[5052]: I1203 11:39:29.379436 5052 patch_prober.go:28] interesting pod/machine-config-daemon-74bj6 container/machine-config-daemon namespace/openshift-machine-config-operator: Liveness probe status=failure output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" start-of-body= Dec 03 11:39:29 crc kubenswrapper[5052]: I1203 11:39:29.379483 5052 prober.go:107] "Probe failed" probeType="Liveness" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" containerName="machine-config-daemon" probeResult="failure" output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" Dec 03 11:39:29 crc kubenswrapper[5052]: I1203 11:39:29.379542 5052 kubelet.go:2542] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" Dec 03 11:39:29 crc kubenswrapper[5052]: I1203 11:39:29.380025 5052 kuberuntime_manager.go:1027] "Message for Container of pod" containerName="machine-config-daemon" containerStatusID={"Type":"cri-o","ID":"42f4b7304c54076bcbb388fc8df0720f8096981812e6c2b3d22518ea7235934e"} pod="openshift-machine-config-operator/machine-config-daemon-74bj6" containerMessage="Container machine-config-daemon failed liveness probe, will be restarted" Dec 03 11:39:29 crc kubenswrapper[5052]: I1203 11:39:29.380077 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" containerName="machine-config-daemon" containerID="cri-o://42f4b7304c54076bcbb388fc8df0720f8096981812e6c2b3d22518ea7235934e" gracePeriod=600 Dec 03 11:39:29 crc kubenswrapper[5052]: I1203 11:39:29.392897 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c43f59f2-deb7-46ee-8e07-54e3ff9f59aa-config-data" (OuterVolumeSpecName: "config-data") pod "c43f59f2-deb7-46ee-8e07-54e3ff9f59aa" (UID: "c43f59f2-deb7-46ee-8e07-54e3ff9f59aa"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:39:29 crc kubenswrapper[5052]: I1203 11:39:29.444138 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-db-sync-bfv74" Dec 03 11:39:29 crc kubenswrapper[5052]: I1203 11:39:29.484616 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/c43f59f2-deb7-46ee-8e07-54e3ff9f59aa-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:39:29 crc kubenswrapper[5052]: I1203 11:39:29.552603 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:39:29 crc kubenswrapper[5052]: I1203 11:39:29.571199 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:39:29 crc kubenswrapper[5052]: I1203 11:39:29.584904 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:39:29 crc kubenswrapper[5052]: E1203 11:39:29.585356 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="c43f59f2-deb7-46ee-8e07-54e3ff9f59aa" containerName="sg-core" Dec 03 11:39:29 crc kubenswrapper[5052]: I1203 11:39:29.585377 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="c43f59f2-deb7-46ee-8e07-54e3ff9f59aa" containerName="sg-core" Dec 03 11:39:29 crc kubenswrapper[5052]: E1203 11:39:29.585401 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="1dc19382-24a8-4a4f-b735-d794619492bf" containerName="barbican-db-sync" Dec 03 11:39:29 crc kubenswrapper[5052]: I1203 11:39:29.585410 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="1dc19382-24a8-4a4f-b735-d794619492bf" containerName="barbican-db-sync" Dec 03 11:39:29 crc kubenswrapper[5052]: E1203 11:39:29.585424 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="c43f59f2-deb7-46ee-8e07-54e3ff9f59aa" containerName="ceilometer-notification-agent" Dec 03 11:39:29 crc kubenswrapper[5052]: I1203 11:39:29.585431 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="c43f59f2-deb7-46ee-8e07-54e3ff9f59aa" containerName="ceilometer-notification-agent" Dec 03 11:39:29 crc kubenswrapper[5052]: E1203 11:39:29.585447 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="c43f59f2-deb7-46ee-8e07-54e3ff9f59aa" containerName="proxy-httpd" Dec 03 11:39:29 crc kubenswrapper[5052]: I1203 11:39:29.585454 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="c43f59f2-deb7-46ee-8e07-54e3ff9f59aa" containerName="proxy-httpd" Dec 03 11:39:29 crc kubenswrapper[5052]: I1203 11:39:29.585973 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="c43f59f2-deb7-46ee-8e07-54e3ff9f59aa" containerName="proxy-httpd" Dec 03 11:39:29 crc kubenswrapper[5052]: I1203 11:39:29.585993 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1dc19382-24a8-4a4f-b735-d794619492bf-combined-ca-bundle\") pod \"1dc19382-24a8-4a4f-b735-d794619492bf\" (UID: \"1dc19382-24a8-4a4f-b735-d794619492bf\") " Dec 03 11:39:29 crc kubenswrapper[5052]: I1203 11:39:29.586097 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zp4pf\" (UniqueName: \"kubernetes.io/projected/1dc19382-24a8-4a4f-b735-d794619492bf-kube-api-access-zp4pf\") pod \"1dc19382-24a8-4a4f-b735-d794619492bf\" (UID: \"1dc19382-24a8-4a4f-b735-d794619492bf\") " Dec 03 11:39:29 crc kubenswrapper[5052]: I1203 11:39:29.586007 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="c43f59f2-deb7-46ee-8e07-54e3ff9f59aa" containerName="ceilometer-notification-agent" Dec 03 11:39:29 crc kubenswrapper[5052]: I1203 11:39:29.586285 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="1dc19382-24a8-4a4f-b735-d794619492bf" containerName="barbican-db-sync" Dec 03 11:39:29 crc kubenswrapper[5052]: I1203 11:39:29.586329 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="c43f59f2-deb7-46ee-8e07-54e3ff9f59aa" containerName="sg-core" Dec 03 11:39:29 crc kubenswrapper[5052]: I1203 11:39:29.588024 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:39:29 crc kubenswrapper[5052]: I1203 11:39:29.586250 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/1dc19382-24a8-4a4f-b735-d794619492bf-db-sync-config-data\") pod \"1dc19382-24a8-4a4f-b735-d794619492bf\" (UID: \"1dc19382-24a8-4a4f-b735-d794619492bf\") " Dec 03 11:39:29 crc kubenswrapper[5052]: I1203 11:39:29.590998 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1dc19382-24a8-4a4f-b735-d794619492bf-db-sync-config-data" (OuterVolumeSpecName: "db-sync-config-data") pod "1dc19382-24a8-4a4f-b735-d794619492bf" (UID: "1dc19382-24a8-4a4f-b735-d794619492bf"). InnerVolumeSpecName "db-sync-config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:39:29 crc kubenswrapper[5052]: I1203 11:39:29.592690 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-scripts" Dec 03 11:39:29 crc kubenswrapper[5052]: I1203 11:39:29.593161 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1dc19382-24a8-4a4f-b735-d794619492bf-kube-api-access-zp4pf" (OuterVolumeSpecName: "kube-api-access-zp4pf") pod "1dc19382-24a8-4a4f-b735-d794619492bf" (UID: "1dc19382-24a8-4a4f-b735-d794619492bf"). InnerVolumeSpecName "kube-api-access-zp4pf". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:39:29 crc kubenswrapper[5052]: I1203 11:39:29.593577 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-config-data" Dec 03 11:39:29 crc kubenswrapper[5052]: I1203 11:39:29.593932 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:39:29 crc kubenswrapper[5052]: I1203 11:39:29.628932 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1dc19382-24a8-4a4f-b735-d794619492bf-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "1dc19382-24a8-4a4f-b735-d794619492bf" (UID: "1dc19382-24a8-4a4f-b735-d794619492bf"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:39:29 crc kubenswrapper[5052]: I1203 11:39:29.689853 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c43f59f2-deb7-46ee-8e07-54e3ff9f59aa" path="/var/lib/kubelet/pods/c43f59f2-deb7-46ee-8e07-54e3ff9f59aa/volumes" Dec 03 11:39:29 crc kubenswrapper[5052]: I1203 11:39:29.691009 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/70e2b274-dd4d-409a-8d45-a09af330f7a3-log-httpd\") pod \"ceilometer-0\" (UID: \"70e2b274-dd4d-409a-8d45-a09af330f7a3\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:39:29 crc kubenswrapper[5052]: I1203 11:39:29.691052 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/70e2b274-dd4d-409a-8d45-a09af330f7a3-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"70e2b274-dd4d-409a-8d45-a09af330f7a3\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:39:29 crc kubenswrapper[5052]: I1203 11:39:29.691072 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/70e2b274-dd4d-409a-8d45-a09af330f7a3-config-data\") pod \"ceilometer-0\" (UID: \"70e2b274-dd4d-409a-8d45-a09af330f7a3\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:39:29 crc kubenswrapper[5052]: I1203 11:39:29.691335 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/70e2b274-dd4d-409a-8d45-a09af330f7a3-run-httpd\") pod \"ceilometer-0\" (UID: \"70e2b274-dd4d-409a-8d45-a09af330f7a3\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:39:29 crc kubenswrapper[5052]: I1203 11:39:29.691479 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/70e2b274-dd4d-409a-8d45-a09af330f7a3-scripts\") pod \"ceilometer-0\" (UID: \"70e2b274-dd4d-409a-8d45-a09af330f7a3\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:39:29 crc kubenswrapper[5052]: I1203 11:39:29.691544 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-728bk\" (UniqueName: \"kubernetes.io/projected/70e2b274-dd4d-409a-8d45-a09af330f7a3-kube-api-access-728bk\") pod \"ceilometer-0\" (UID: \"70e2b274-dd4d-409a-8d45-a09af330f7a3\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:39:29 crc kubenswrapper[5052]: I1203 11:39:29.691568 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/70e2b274-dd4d-409a-8d45-a09af330f7a3-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"70e2b274-dd4d-409a-8d45-a09af330f7a3\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:39:29 crc kubenswrapper[5052]: I1203 11:39:29.691651 5052 reconciler_common.go:293] "Volume detached for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/1dc19382-24a8-4a4f-b735-d794619492bf-db-sync-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:39:29 crc kubenswrapper[5052]: I1203 11:39:29.691662 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1dc19382-24a8-4a4f-b735-d794619492bf-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:39:29 crc kubenswrapper[5052]: I1203 11:39:29.691672 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-zp4pf\" (UniqueName: \"kubernetes.io/projected/1dc19382-24a8-4a4f-b735-d794619492bf-kube-api-access-zp4pf\") on node \"crc\" DevicePath \"\"" Dec 03 11:39:29 crc kubenswrapper[5052]: I1203 11:39:29.792766 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/70e2b274-dd4d-409a-8d45-a09af330f7a3-scripts\") pod \"ceilometer-0\" (UID: \"70e2b274-dd4d-409a-8d45-a09af330f7a3\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:39:29 crc kubenswrapper[5052]: I1203 11:39:29.792812 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-728bk\" (UniqueName: \"kubernetes.io/projected/70e2b274-dd4d-409a-8d45-a09af330f7a3-kube-api-access-728bk\") pod \"ceilometer-0\" (UID: \"70e2b274-dd4d-409a-8d45-a09af330f7a3\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:39:29 crc kubenswrapper[5052]: I1203 11:39:29.792835 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/70e2b274-dd4d-409a-8d45-a09af330f7a3-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"70e2b274-dd4d-409a-8d45-a09af330f7a3\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:39:29 crc kubenswrapper[5052]: I1203 11:39:29.792922 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/70e2b274-dd4d-409a-8d45-a09af330f7a3-log-httpd\") pod \"ceilometer-0\" (UID: \"70e2b274-dd4d-409a-8d45-a09af330f7a3\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:39:29 crc kubenswrapper[5052]: I1203 11:39:29.792965 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/70e2b274-dd4d-409a-8d45-a09af330f7a3-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"70e2b274-dd4d-409a-8d45-a09af330f7a3\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:39:29 crc kubenswrapper[5052]: I1203 11:39:29.793002 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/70e2b274-dd4d-409a-8d45-a09af330f7a3-config-data\") pod \"ceilometer-0\" (UID: \"70e2b274-dd4d-409a-8d45-a09af330f7a3\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:39:29 crc kubenswrapper[5052]: I1203 11:39:29.793080 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/70e2b274-dd4d-409a-8d45-a09af330f7a3-run-httpd\") pod \"ceilometer-0\" (UID: \"70e2b274-dd4d-409a-8d45-a09af330f7a3\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:39:29 crc kubenswrapper[5052]: I1203 11:39:29.793557 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/70e2b274-dd4d-409a-8d45-a09af330f7a3-run-httpd\") pod \"ceilometer-0\" (UID: \"70e2b274-dd4d-409a-8d45-a09af330f7a3\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:39:29 crc kubenswrapper[5052]: I1203 11:39:29.794172 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/70e2b274-dd4d-409a-8d45-a09af330f7a3-log-httpd\") pod \"ceilometer-0\" (UID: \"70e2b274-dd4d-409a-8d45-a09af330f7a3\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:39:29 crc kubenswrapper[5052]: I1203 11:39:29.796867 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/70e2b274-dd4d-409a-8d45-a09af330f7a3-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"70e2b274-dd4d-409a-8d45-a09af330f7a3\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:39:29 crc kubenswrapper[5052]: I1203 11:39:29.797237 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/70e2b274-dd4d-409a-8d45-a09af330f7a3-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"70e2b274-dd4d-409a-8d45-a09af330f7a3\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:39:29 crc kubenswrapper[5052]: I1203 11:39:29.797349 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/70e2b274-dd4d-409a-8d45-a09af330f7a3-scripts\") pod \"ceilometer-0\" (UID: \"70e2b274-dd4d-409a-8d45-a09af330f7a3\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:39:29 crc kubenswrapper[5052]: I1203 11:39:29.797971 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/70e2b274-dd4d-409a-8d45-a09af330f7a3-config-data\") pod \"ceilometer-0\" (UID: \"70e2b274-dd4d-409a-8d45-a09af330f7a3\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:39:29 crc kubenswrapper[5052]: I1203 11:39:29.808299 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-728bk\" (UniqueName: \"kubernetes.io/projected/70e2b274-dd4d-409a-8d45-a09af330f7a3-kube-api-access-728bk\") pod \"ceilometer-0\" (UID: \"70e2b274-dd4d-409a-8d45-a09af330f7a3\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:39:29 crc kubenswrapper[5052]: I1203 11:39:29.949547 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:39:30 crc kubenswrapper[5052]: I1203 11:39:30.161673 5052 generic.go:334] "Generic (PLEG): container finished" podID="d1676789-356e-4853-a302-f2c5bd3f446f" containerID="1820ff11c6142cf8027dde668e11d985d56d48b18de14c51c0337c5af52c8909" exitCode=0 Dec 03 11:39:30 crc kubenswrapper[5052]: I1203 11:39:30.161753 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-db-sync-pbw4k" event={"ID":"d1676789-356e-4853-a302-f2c5bd3f446f","Type":"ContainerDied","Data":"1820ff11c6142cf8027dde668e11d985d56d48b18de14c51c0337c5af52c8909"} Dec 03 11:39:30 crc kubenswrapper[5052]: I1203 11:39:30.163844 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-db-sync-bfv74" Dec 03 11:39:30 crc kubenswrapper[5052]: I1203 11:39:30.163870 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-db-sync-bfv74" event={"ID":"1dc19382-24a8-4a4f-b735-d794619492bf","Type":"ContainerDied","Data":"f5e0f6eb722d532f0bd3399d5aa3cad9c6468d78aabaf922d936fc2eddd2804b"} Dec 03 11:39:30 crc kubenswrapper[5052]: I1203 11:39:30.163955 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f5e0f6eb722d532f0bd3399d5aa3cad9c6468d78aabaf922d936fc2eddd2804b" Dec 03 11:39:30 crc kubenswrapper[5052]: I1203 11:39:30.169719 5052 generic.go:334] "Generic (PLEG): container finished" podID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" containerID="42f4b7304c54076bcbb388fc8df0720f8096981812e6c2b3d22518ea7235934e" exitCode=0 Dec 03 11:39:30 crc kubenswrapper[5052]: I1203 11:39:30.169791 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" event={"ID":"10c237bc-72e7-4da3-9f2c-b1688395bf6e","Type":"ContainerDied","Data":"42f4b7304c54076bcbb388fc8df0720f8096981812e6c2b3d22518ea7235934e"} Dec 03 11:39:30 crc kubenswrapper[5052]: I1203 11:39:30.169826 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" event={"ID":"10c237bc-72e7-4da3-9f2c-b1688395bf6e","Type":"ContainerStarted","Data":"9a1bbf3b5805844b6adac48684e34e2c0f409388266eaea05dc4a45f4951ccdf"} Dec 03 11:39:30 crc kubenswrapper[5052]: I1203 11:39:30.169851 5052 scope.go:117] "RemoveContainer" containerID="f37a0f397f498a9429565aa0bada2c6757e002bd2d8af169475a45c6c645d73e" Dec 03 11:39:30 crc kubenswrapper[5052]: I1203 11:39:30.380840 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:39:30 crc kubenswrapper[5052]: W1203 11:39:30.392670 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod70e2b274_dd4d_409a_8d45_a09af330f7a3.slice/crio-f6395474b3ead890bff3ec7ef4a1992397a65b07d36b291cb39d7a172ca1da72 WatchSource:0}: Error finding container f6395474b3ead890bff3ec7ef4a1992397a65b07d36b291cb39d7a172ca1da72: Status 404 returned error can't find the container with id f6395474b3ead890bff3ec7ef4a1992397a65b07d36b291cb39d7a172ca1da72 Dec 03 11:39:30 crc kubenswrapper[5052]: I1203 11:39:30.726708 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/barbican-keystone-listener-649457694-r7gzw"] Dec 03 11:39:30 crc kubenswrapper[5052]: I1203 11:39:30.728074 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-keystone-listener-649457694-r7gzw" Dec 03 11:39:30 crc kubenswrapper[5052]: I1203 11:39:30.732395 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"barbican-barbican-dockercfg-bq7sh" Dec 03 11:39:30 crc kubenswrapper[5052]: I1203 11:39:30.732541 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"barbican-keystone-listener-config-data" Dec 03 11:39:30 crc kubenswrapper[5052]: I1203 11:39:30.741169 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/barbican-worker-79cc7656c5-896b7"] Dec 03 11:39:30 crc kubenswrapper[5052]: I1203 11:39:30.742608 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-worker-79cc7656c5-896b7" Dec 03 11:39:30 crc kubenswrapper[5052]: I1203 11:39:30.744614 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"barbican-worker-config-data" Dec 03 11:39:30 crc kubenswrapper[5052]: I1203 11:39:30.763133 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-keystone-listener-649457694-r7gzw"] Dec 03 11:39:30 crc kubenswrapper[5052]: I1203 11:39:30.765486 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"barbican-config-data" Dec 03 11:39:30 crc kubenswrapper[5052]: I1203 11:39:30.784986 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-worker-79cc7656c5-896b7"] Dec 03 11:39:30 crc kubenswrapper[5052]: I1203 11:39:30.816495 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8c4732f9-0f9e-4600-aed9-6a230cbf7b16-combined-ca-bundle\") pod \"barbican-keystone-listener-649457694-r7gzw\" (UID: \"8c4732f9-0f9e-4600-aed9-6a230cbf7b16\") " pod="openstack-kuttl-tests/barbican-keystone-listener-649457694-r7gzw" Dec 03 11:39:30 crc kubenswrapper[5052]: I1203 11:39:30.816550 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fd4cm\" (UniqueName: \"kubernetes.io/projected/32190156-44ee-4a16-a86c-2584f6a94866-kube-api-access-fd4cm\") pod \"barbican-worker-79cc7656c5-896b7\" (UID: \"32190156-44ee-4a16-a86c-2584f6a94866\") " pod="openstack-kuttl-tests/barbican-worker-79cc7656c5-896b7" Dec 03 11:39:30 crc kubenswrapper[5052]: I1203 11:39:30.816603 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/32190156-44ee-4a16-a86c-2584f6a94866-config-data\") pod \"barbican-worker-79cc7656c5-896b7\" (UID: \"32190156-44ee-4a16-a86c-2584f6a94866\") " pod="openstack-kuttl-tests/barbican-worker-79cc7656c5-896b7" Dec 03 11:39:30 crc kubenswrapper[5052]: I1203 11:39:30.816637 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/32190156-44ee-4a16-a86c-2584f6a94866-logs\") pod \"barbican-worker-79cc7656c5-896b7\" (UID: \"32190156-44ee-4a16-a86c-2584f6a94866\") " pod="openstack-kuttl-tests/barbican-worker-79cc7656c5-896b7" Dec 03 11:39:30 crc kubenswrapper[5052]: I1203 11:39:30.816669 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/8c4732f9-0f9e-4600-aed9-6a230cbf7b16-config-data-custom\") pod \"barbican-keystone-listener-649457694-r7gzw\" (UID: \"8c4732f9-0f9e-4600-aed9-6a230cbf7b16\") " pod="openstack-kuttl-tests/barbican-keystone-listener-649457694-r7gzw" Dec 03 11:39:30 crc kubenswrapper[5052]: I1203 11:39:30.816689 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/32190156-44ee-4a16-a86c-2584f6a94866-config-data-custom\") pod \"barbican-worker-79cc7656c5-896b7\" (UID: \"32190156-44ee-4a16-a86c-2584f6a94866\") " pod="openstack-kuttl-tests/barbican-worker-79cc7656c5-896b7" Dec 03 11:39:30 crc kubenswrapper[5052]: I1203 11:39:30.816723 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/8c4732f9-0f9e-4600-aed9-6a230cbf7b16-logs\") pod \"barbican-keystone-listener-649457694-r7gzw\" (UID: \"8c4732f9-0f9e-4600-aed9-6a230cbf7b16\") " pod="openstack-kuttl-tests/barbican-keystone-listener-649457694-r7gzw" Dec 03 11:39:30 crc kubenswrapper[5052]: I1203 11:39:30.816743 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/32190156-44ee-4a16-a86c-2584f6a94866-combined-ca-bundle\") pod \"barbican-worker-79cc7656c5-896b7\" (UID: \"32190156-44ee-4a16-a86c-2584f6a94866\") " pod="openstack-kuttl-tests/barbican-worker-79cc7656c5-896b7" Dec 03 11:39:30 crc kubenswrapper[5052]: I1203 11:39:30.816800 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/8c4732f9-0f9e-4600-aed9-6a230cbf7b16-config-data\") pod \"barbican-keystone-listener-649457694-r7gzw\" (UID: \"8c4732f9-0f9e-4600-aed9-6a230cbf7b16\") " pod="openstack-kuttl-tests/barbican-keystone-listener-649457694-r7gzw" Dec 03 11:39:30 crc kubenswrapper[5052]: I1203 11:39:30.816819 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l8572\" (UniqueName: \"kubernetes.io/projected/8c4732f9-0f9e-4600-aed9-6a230cbf7b16-kube-api-access-l8572\") pod \"barbican-keystone-listener-649457694-r7gzw\" (UID: \"8c4732f9-0f9e-4600-aed9-6a230cbf7b16\") " pod="openstack-kuttl-tests/barbican-keystone-listener-649457694-r7gzw" Dec 03 11:39:30 crc kubenswrapper[5052]: I1203 11:39:30.902001 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/barbican-api-65968d9848-mntb6"] Dec 03 11:39:30 crc kubenswrapper[5052]: I1203 11:39:30.904547 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-api-65968d9848-mntb6" Dec 03 11:39:30 crc kubenswrapper[5052]: I1203 11:39:30.907396 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"barbican-api-config-data" Dec 03 11:39:30 crc kubenswrapper[5052]: I1203 11:39:30.913083 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-api-65968d9848-mntb6"] Dec 03 11:39:30 crc kubenswrapper[5052]: I1203 11:39:30.932266 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-fd4cm\" (UniqueName: \"kubernetes.io/projected/32190156-44ee-4a16-a86c-2584f6a94866-kube-api-access-fd4cm\") pod \"barbican-worker-79cc7656c5-896b7\" (UID: \"32190156-44ee-4a16-a86c-2584f6a94866\") " pod="openstack-kuttl-tests/barbican-worker-79cc7656c5-896b7" Dec 03 11:39:30 crc kubenswrapper[5052]: I1203 11:39:30.932430 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/32190156-44ee-4a16-a86c-2584f6a94866-config-data\") pod \"barbican-worker-79cc7656c5-896b7\" (UID: \"32190156-44ee-4a16-a86c-2584f6a94866\") " pod="openstack-kuttl-tests/barbican-worker-79cc7656c5-896b7" Dec 03 11:39:30 crc kubenswrapper[5052]: I1203 11:39:30.932529 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/32190156-44ee-4a16-a86c-2584f6a94866-logs\") pod \"barbican-worker-79cc7656c5-896b7\" (UID: \"32190156-44ee-4a16-a86c-2584f6a94866\") " pod="openstack-kuttl-tests/barbican-worker-79cc7656c5-896b7" Dec 03 11:39:30 crc kubenswrapper[5052]: I1203 11:39:30.932609 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/8c4732f9-0f9e-4600-aed9-6a230cbf7b16-config-data-custom\") pod \"barbican-keystone-listener-649457694-r7gzw\" (UID: \"8c4732f9-0f9e-4600-aed9-6a230cbf7b16\") " pod="openstack-kuttl-tests/barbican-keystone-listener-649457694-r7gzw" Dec 03 11:39:30 crc kubenswrapper[5052]: I1203 11:39:30.932657 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/32190156-44ee-4a16-a86c-2584f6a94866-config-data-custom\") pod \"barbican-worker-79cc7656c5-896b7\" (UID: \"32190156-44ee-4a16-a86c-2584f6a94866\") " pod="openstack-kuttl-tests/barbican-worker-79cc7656c5-896b7" Dec 03 11:39:30 crc kubenswrapper[5052]: I1203 11:39:30.932705 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/8c4732f9-0f9e-4600-aed9-6a230cbf7b16-logs\") pod \"barbican-keystone-listener-649457694-r7gzw\" (UID: \"8c4732f9-0f9e-4600-aed9-6a230cbf7b16\") " pod="openstack-kuttl-tests/barbican-keystone-listener-649457694-r7gzw" Dec 03 11:39:30 crc kubenswrapper[5052]: I1203 11:39:30.932752 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/32190156-44ee-4a16-a86c-2584f6a94866-combined-ca-bundle\") pod \"barbican-worker-79cc7656c5-896b7\" (UID: \"32190156-44ee-4a16-a86c-2584f6a94866\") " pod="openstack-kuttl-tests/barbican-worker-79cc7656c5-896b7" Dec 03 11:39:30 crc kubenswrapper[5052]: I1203 11:39:30.932857 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/8c4732f9-0f9e-4600-aed9-6a230cbf7b16-config-data\") pod \"barbican-keystone-listener-649457694-r7gzw\" (UID: \"8c4732f9-0f9e-4600-aed9-6a230cbf7b16\") " pod="openstack-kuttl-tests/barbican-keystone-listener-649457694-r7gzw" Dec 03 11:39:30 crc kubenswrapper[5052]: I1203 11:39:30.932901 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-l8572\" (UniqueName: \"kubernetes.io/projected/8c4732f9-0f9e-4600-aed9-6a230cbf7b16-kube-api-access-l8572\") pod \"barbican-keystone-listener-649457694-r7gzw\" (UID: \"8c4732f9-0f9e-4600-aed9-6a230cbf7b16\") " pod="openstack-kuttl-tests/barbican-keystone-listener-649457694-r7gzw" Dec 03 11:39:30 crc kubenswrapper[5052]: I1203 11:39:30.932926 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8c4732f9-0f9e-4600-aed9-6a230cbf7b16-combined-ca-bundle\") pod \"barbican-keystone-listener-649457694-r7gzw\" (UID: \"8c4732f9-0f9e-4600-aed9-6a230cbf7b16\") " pod="openstack-kuttl-tests/barbican-keystone-listener-649457694-r7gzw" Dec 03 11:39:30 crc kubenswrapper[5052]: I1203 11:39:30.934721 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/32190156-44ee-4a16-a86c-2584f6a94866-logs\") pod \"barbican-worker-79cc7656c5-896b7\" (UID: \"32190156-44ee-4a16-a86c-2584f6a94866\") " pod="openstack-kuttl-tests/barbican-worker-79cc7656c5-896b7" Dec 03 11:39:30 crc kubenswrapper[5052]: I1203 11:39:30.934894 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/8c4732f9-0f9e-4600-aed9-6a230cbf7b16-logs\") pod \"barbican-keystone-listener-649457694-r7gzw\" (UID: \"8c4732f9-0f9e-4600-aed9-6a230cbf7b16\") " pod="openstack-kuttl-tests/barbican-keystone-listener-649457694-r7gzw" Dec 03 11:39:30 crc kubenswrapper[5052]: I1203 11:39:30.939044 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/8c4732f9-0f9e-4600-aed9-6a230cbf7b16-config-data-custom\") pod \"barbican-keystone-listener-649457694-r7gzw\" (UID: \"8c4732f9-0f9e-4600-aed9-6a230cbf7b16\") " pod="openstack-kuttl-tests/barbican-keystone-listener-649457694-r7gzw" Dec 03 11:39:30 crc kubenswrapper[5052]: I1203 11:39:30.939100 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8c4732f9-0f9e-4600-aed9-6a230cbf7b16-combined-ca-bundle\") pod \"barbican-keystone-listener-649457694-r7gzw\" (UID: \"8c4732f9-0f9e-4600-aed9-6a230cbf7b16\") " pod="openstack-kuttl-tests/barbican-keystone-listener-649457694-r7gzw" Dec 03 11:39:30 crc kubenswrapper[5052]: I1203 11:39:30.939578 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/32190156-44ee-4a16-a86c-2584f6a94866-combined-ca-bundle\") pod \"barbican-worker-79cc7656c5-896b7\" (UID: \"32190156-44ee-4a16-a86c-2584f6a94866\") " pod="openstack-kuttl-tests/barbican-worker-79cc7656c5-896b7" Dec 03 11:39:30 crc kubenswrapper[5052]: I1203 11:39:30.940720 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/8c4732f9-0f9e-4600-aed9-6a230cbf7b16-config-data\") pod \"barbican-keystone-listener-649457694-r7gzw\" (UID: \"8c4732f9-0f9e-4600-aed9-6a230cbf7b16\") " pod="openstack-kuttl-tests/barbican-keystone-listener-649457694-r7gzw" Dec 03 11:39:30 crc kubenswrapper[5052]: I1203 11:39:30.949249 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-l8572\" (UniqueName: \"kubernetes.io/projected/8c4732f9-0f9e-4600-aed9-6a230cbf7b16-kube-api-access-l8572\") pod \"barbican-keystone-listener-649457694-r7gzw\" (UID: \"8c4732f9-0f9e-4600-aed9-6a230cbf7b16\") " pod="openstack-kuttl-tests/barbican-keystone-listener-649457694-r7gzw" Dec 03 11:39:30 crc kubenswrapper[5052]: I1203 11:39:30.951228 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/32190156-44ee-4a16-a86c-2584f6a94866-config-data\") pod \"barbican-worker-79cc7656c5-896b7\" (UID: \"32190156-44ee-4a16-a86c-2584f6a94866\") " pod="openstack-kuttl-tests/barbican-worker-79cc7656c5-896b7" Dec 03 11:39:30 crc kubenswrapper[5052]: I1203 11:39:30.953766 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-fd4cm\" (UniqueName: \"kubernetes.io/projected/32190156-44ee-4a16-a86c-2584f6a94866-kube-api-access-fd4cm\") pod \"barbican-worker-79cc7656c5-896b7\" (UID: \"32190156-44ee-4a16-a86c-2584f6a94866\") " pod="openstack-kuttl-tests/barbican-worker-79cc7656c5-896b7" Dec 03 11:39:30 crc kubenswrapper[5052]: I1203 11:39:30.954575 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/32190156-44ee-4a16-a86c-2584f6a94866-config-data-custom\") pod \"barbican-worker-79cc7656c5-896b7\" (UID: \"32190156-44ee-4a16-a86c-2584f6a94866\") " pod="openstack-kuttl-tests/barbican-worker-79cc7656c5-896b7" Dec 03 11:39:31 crc kubenswrapper[5052]: I1203 11:39:31.034720 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w2dbz\" (UniqueName: \"kubernetes.io/projected/ef47ba52-772d-4732-bd15-e7513e626be9-kube-api-access-w2dbz\") pod \"barbican-api-65968d9848-mntb6\" (UID: \"ef47ba52-772d-4732-bd15-e7513e626be9\") " pod="openstack-kuttl-tests/barbican-api-65968d9848-mntb6" Dec 03 11:39:31 crc kubenswrapper[5052]: I1203 11:39:31.034784 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/ef47ba52-772d-4732-bd15-e7513e626be9-logs\") pod \"barbican-api-65968d9848-mntb6\" (UID: \"ef47ba52-772d-4732-bd15-e7513e626be9\") " pod="openstack-kuttl-tests/barbican-api-65968d9848-mntb6" Dec 03 11:39:31 crc kubenswrapper[5052]: I1203 11:39:31.035296 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/ef47ba52-772d-4732-bd15-e7513e626be9-config-data\") pod \"barbican-api-65968d9848-mntb6\" (UID: \"ef47ba52-772d-4732-bd15-e7513e626be9\") " pod="openstack-kuttl-tests/barbican-api-65968d9848-mntb6" Dec 03 11:39:31 crc kubenswrapper[5052]: I1203 11:39:31.035384 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ef47ba52-772d-4732-bd15-e7513e626be9-combined-ca-bundle\") pod \"barbican-api-65968d9848-mntb6\" (UID: \"ef47ba52-772d-4732-bd15-e7513e626be9\") " pod="openstack-kuttl-tests/barbican-api-65968d9848-mntb6" Dec 03 11:39:31 crc kubenswrapper[5052]: I1203 11:39:31.035469 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/ef47ba52-772d-4732-bd15-e7513e626be9-config-data-custom\") pod \"barbican-api-65968d9848-mntb6\" (UID: \"ef47ba52-772d-4732-bd15-e7513e626be9\") " pod="openstack-kuttl-tests/barbican-api-65968d9848-mntb6" Dec 03 11:39:31 crc kubenswrapper[5052]: I1203 11:39:31.086795 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-keystone-listener-649457694-r7gzw" Dec 03 11:39:31 crc kubenswrapper[5052]: I1203 11:39:31.099366 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-worker-79cc7656c5-896b7" Dec 03 11:39:31 crc kubenswrapper[5052]: I1203 11:39:31.136986 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-w2dbz\" (UniqueName: \"kubernetes.io/projected/ef47ba52-772d-4732-bd15-e7513e626be9-kube-api-access-w2dbz\") pod \"barbican-api-65968d9848-mntb6\" (UID: \"ef47ba52-772d-4732-bd15-e7513e626be9\") " pod="openstack-kuttl-tests/barbican-api-65968d9848-mntb6" Dec 03 11:39:31 crc kubenswrapper[5052]: I1203 11:39:31.137060 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/ef47ba52-772d-4732-bd15-e7513e626be9-logs\") pod \"barbican-api-65968d9848-mntb6\" (UID: \"ef47ba52-772d-4732-bd15-e7513e626be9\") " pod="openstack-kuttl-tests/barbican-api-65968d9848-mntb6" Dec 03 11:39:31 crc kubenswrapper[5052]: I1203 11:39:31.137137 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/ef47ba52-772d-4732-bd15-e7513e626be9-config-data\") pod \"barbican-api-65968d9848-mntb6\" (UID: \"ef47ba52-772d-4732-bd15-e7513e626be9\") " pod="openstack-kuttl-tests/barbican-api-65968d9848-mntb6" Dec 03 11:39:31 crc kubenswrapper[5052]: I1203 11:39:31.137158 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ef47ba52-772d-4732-bd15-e7513e626be9-combined-ca-bundle\") pod \"barbican-api-65968d9848-mntb6\" (UID: \"ef47ba52-772d-4732-bd15-e7513e626be9\") " pod="openstack-kuttl-tests/barbican-api-65968d9848-mntb6" Dec 03 11:39:31 crc kubenswrapper[5052]: I1203 11:39:31.137182 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/ef47ba52-772d-4732-bd15-e7513e626be9-config-data-custom\") pod \"barbican-api-65968d9848-mntb6\" (UID: \"ef47ba52-772d-4732-bd15-e7513e626be9\") " pod="openstack-kuttl-tests/barbican-api-65968d9848-mntb6" Dec 03 11:39:31 crc kubenswrapper[5052]: I1203 11:39:31.137864 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/ef47ba52-772d-4732-bd15-e7513e626be9-logs\") pod \"barbican-api-65968d9848-mntb6\" (UID: \"ef47ba52-772d-4732-bd15-e7513e626be9\") " pod="openstack-kuttl-tests/barbican-api-65968d9848-mntb6" Dec 03 11:39:31 crc kubenswrapper[5052]: I1203 11:39:31.142300 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/ef47ba52-772d-4732-bd15-e7513e626be9-config-data-custom\") pod \"barbican-api-65968d9848-mntb6\" (UID: \"ef47ba52-772d-4732-bd15-e7513e626be9\") " pod="openstack-kuttl-tests/barbican-api-65968d9848-mntb6" Dec 03 11:39:31 crc kubenswrapper[5052]: I1203 11:39:31.143459 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/ef47ba52-772d-4732-bd15-e7513e626be9-config-data\") pod \"barbican-api-65968d9848-mntb6\" (UID: \"ef47ba52-772d-4732-bd15-e7513e626be9\") " pod="openstack-kuttl-tests/barbican-api-65968d9848-mntb6" Dec 03 11:39:31 crc kubenswrapper[5052]: I1203 11:39:31.153736 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ef47ba52-772d-4732-bd15-e7513e626be9-combined-ca-bundle\") pod \"barbican-api-65968d9848-mntb6\" (UID: \"ef47ba52-772d-4732-bd15-e7513e626be9\") " pod="openstack-kuttl-tests/barbican-api-65968d9848-mntb6" Dec 03 11:39:31 crc kubenswrapper[5052]: I1203 11:39:31.156769 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-w2dbz\" (UniqueName: \"kubernetes.io/projected/ef47ba52-772d-4732-bd15-e7513e626be9-kube-api-access-w2dbz\") pod \"barbican-api-65968d9848-mntb6\" (UID: \"ef47ba52-772d-4732-bd15-e7513e626be9\") " pod="openstack-kuttl-tests/barbican-api-65968d9848-mntb6" Dec 03 11:39:31 crc kubenswrapper[5052]: I1203 11:39:31.188934 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"70e2b274-dd4d-409a-8d45-a09af330f7a3","Type":"ContainerStarted","Data":"f6395474b3ead890bff3ec7ef4a1992397a65b07d36b291cb39d7a172ca1da72"} Dec 03 11:39:31 crc kubenswrapper[5052]: I1203 11:39:31.325525 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-api-65968d9848-mntb6" Dec 03 11:39:31 crc kubenswrapper[5052]: I1203 11:39:31.575316 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-db-sync-pbw4k" Dec 03 11:39:31 crc kubenswrapper[5052]: I1203 11:39:31.606725 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-keystone-listener-649457694-r7gzw"] Dec 03 11:39:31 crc kubenswrapper[5052]: I1203 11:39:31.645754 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8lqcr\" (UniqueName: \"kubernetes.io/projected/d1676789-356e-4853-a302-f2c5bd3f446f-kube-api-access-8lqcr\") pod \"d1676789-356e-4853-a302-f2c5bd3f446f\" (UID: \"d1676789-356e-4853-a302-f2c5bd3f446f\") " Dec 03 11:39:31 crc kubenswrapper[5052]: I1203 11:39:31.646951 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d1676789-356e-4853-a302-f2c5bd3f446f-combined-ca-bundle\") pod \"d1676789-356e-4853-a302-f2c5bd3f446f\" (UID: \"d1676789-356e-4853-a302-f2c5bd3f446f\") " Dec 03 11:39:31 crc kubenswrapper[5052]: I1203 11:39:31.647070 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/d1676789-356e-4853-a302-f2c5bd3f446f-etc-machine-id\") pod \"d1676789-356e-4853-a302-f2c5bd3f446f\" (UID: \"d1676789-356e-4853-a302-f2c5bd3f446f\") " Dec 03 11:39:31 crc kubenswrapper[5052]: I1203 11:39:31.647167 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/d1676789-356e-4853-a302-f2c5bd3f446f-config-data\") pod \"d1676789-356e-4853-a302-f2c5bd3f446f\" (UID: \"d1676789-356e-4853-a302-f2c5bd3f446f\") " Dec 03 11:39:31 crc kubenswrapper[5052]: I1203 11:39:31.647249 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/d1676789-356e-4853-a302-f2c5bd3f446f-scripts\") pod \"d1676789-356e-4853-a302-f2c5bd3f446f\" (UID: \"d1676789-356e-4853-a302-f2c5bd3f446f\") " Dec 03 11:39:31 crc kubenswrapper[5052]: I1203 11:39:31.647346 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/d1676789-356e-4853-a302-f2c5bd3f446f-db-sync-config-data\") pod \"d1676789-356e-4853-a302-f2c5bd3f446f\" (UID: \"d1676789-356e-4853-a302-f2c5bd3f446f\") " Dec 03 11:39:31 crc kubenswrapper[5052]: I1203 11:39:31.647579 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/d1676789-356e-4853-a302-f2c5bd3f446f-etc-machine-id" (OuterVolumeSpecName: "etc-machine-id") pod "d1676789-356e-4853-a302-f2c5bd3f446f" (UID: "d1676789-356e-4853-a302-f2c5bd3f446f"). InnerVolumeSpecName "etc-machine-id". PluginName "kubernetes.io/host-path", VolumeGidValue "" Dec 03 11:39:31 crc kubenswrapper[5052]: I1203 11:39:31.648159 5052 reconciler_common.go:293] "Volume detached for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/d1676789-356e-4853-a302-f2c5bd3f446f-etc-machine-id\") on node \"crc\" DevicePath \"\"" Dec 03 11:39:31 crc kubenswrapper[5052]: I1203 11:39:31.650499 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d1676789-356e-4853-a302-f2c5bd3f446f-kube-api-access-8lqcr" (OuterVolumeSpecName: "kube-api-access-8lqcr") pod "d1676789-356e-4853-a302-f2c5bd3f446f" (UID: "d1676789-356e-4853-a302-f2c5bd3f446f"). InnerVolumeSpecName "kube-api-access-8lqcr". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:39:31 crc kubenswrapper[5052]: I1203 11:39:31.653765 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d1676789-356e-4853-a302-f2c5bd3f446f-scripts" (OuterVolumeSpecName: "scripts") pod "d1676789-356e-4853-a302-f2c5bd3f446f" (UID: "d1676789-356e-4853-a302-f2c5bd3f446f"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:39:31 crc kubenswrapper[5052]: I1203 11:39:31.666812 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d1676789-356e-4853-a302-f2c5bd3f446f-db-sync-config-data" (OuterVolumeSpecName: "db-sync-config-data") pod "d1676789-356e-4853-a302-f2c5bd3f446f" (UID: "d1676789-356e-4853-a302-f2c5bd3f446f"). InnerVolumeSpecName "db-sync-config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:39:31 crc kubenswrapper[5052]: I1203 11:39:31.689123 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d1676789-356e-4853-a302-f2c5bd3f446f-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "d1676789-356e-4853-a302-f2c5bd3f446f" (UID: "d1676789-356e-4853-a302-f2c5bd3f446f"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:39:31 crc kubenswrapper[5052]: W1203 11:39:31.713125 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod32190156_44ee_4a16_a86c_2584f6a94866.slice/crio-a002a7e6855ea6b8eaa15396fc2fd13adb56ddeb2fa4a0cca2f3154d7e4ea303 WatchSource:0}: Error finding container a002a7e6855ea6b8eaa15396fc2fd13adb56ddeb2fa4a0cca2f3154d7e4ea303: Status 404 returned error can't find the container with id a002a7e6855ea6b8eaa15396fc2fd13adb56ddeb2fa4a0cca2f3154d7e4ea303 Dec 03 11:39:31 crc kubenswrapper[5052]: I1203 11:39:31.716284 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-worker-79cc7656c5-896b7"] Dec 03 11:39:31 crc kubenswrapper[5052]: I1203 11:39:31.737754 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d1676789-356e-4853-a302-f2c5bd3f446f-config-data" (OuterVolumeSpecName: "config-data") pod "d1676789-356e-4853-a302-f2c5bd3f446f" (UID: "d1676789-356e-4853-a302-f2c5bd3f446f"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:39:31 crc kubenswrapper[5052]: I1203 11:39:31.750286 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-8lqcr\" (UniqueName: \"kubernetes.io/projected/d1676789-356e-4853-a302-f2c5bd3f446f-kube-api-access-8lqcr\") on node \"crc\" DevicePath \"\"" Dec 03 11:39:31 crc kubenswrapper[5052]: I1203 11:39:31.750321 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d1676789-356e-4853-a302-f2c5bd3f446f-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:39:31 crc kubenswrapper[5052]: I1203 11:39:31.750331 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/d1676789-356e-4853-a302-f2c5bd3f446f-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:39:31 crc kubenswrapper[5052]: I1203 11:39:31.750340 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/d1676789-356e-4853-a302-f2c5bd3f446f-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:39:31 crc kubenswrapper[5052]: I1203 11:39:31.750350 5052 reconciler_common.go:293] "Volume detached for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/d1676789-356e-4853-a302-f2c5bd3f446f-db-sync-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:39:31 crc kubenswrapper[5052]: I1203 11:39:31.828117 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-api-65968d9848-mntb6"] Dec 03 11:39:32 crc kubenswrapper[5052]: I1203 11:39:32.198048 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-65968d9848-mntb6" event={"ID":"ef47ba52-772d-4732-bd15-e7513e626be9","Type":"ContainerStarted","Data":"a00718351b51b3492d282952f384864b842002d371827646c87c60e86863c87e"} Dec 03 11:39:32 crc kubenswrapper[5052]: I1203 11:39:32.198370 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-65968d9848-mntb6" event={"ID":"ef47ba52-772d-4732-bd15-e7513e626be9","Type":"ContainerStarted","Data":"2339c61a18e31e2883d1240a7db9fbdee44126d5621c2d1d1866f7ef3df0220b"} Dec 03 11:39:32 crc kubenswrapper[5052]: I1203 11:39:32.199972 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"70e2b274-dd4d-409a-8d45-a09af330f7a3","Type":"ContainerStarted","Data":"a976462aed958683aefdf6044f7b3b06bddc3fc3b6d77f4841493e30c067161c"} Dec 03 11:39:32 crc kubenswrapper[5052]: I1203 11:39:32.202021 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-db-sync-pbw4k" event={"ID":"d1676789-356e-4853-a302-f2c5bd3f446f","Type":"ContainerDied","Data":"9c3a23d24e9f19c47daa0c2a455117a451af85c686194e2b184c1f25a5ac6f52"} Dec 03 11:39:32 crc kubenswrapper[5052]: I1203 11:39:32.202052 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9c3a23d24e9f19c47daa0c2a455117a451af85c686194e2b184c1f25a5ac6f52" Dec 03 11:39:32 crc kubenswrapper[5052]: I1203 11:39:32.202057 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-db-sync-pbw4k" Dec 03 11:39:32 crc kubenswrapper[5052]: I1203 11:39:32.203170 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-keystone-listener-649457694-r7gzw" event={"ID":"8c4732f9-0f9e-4600-aed9-6a230cbf7b16","Type":"ContainerStarted","Data":"622135ca2a0b77a1c80989556a4247dde77e57653be6c1afecc46f0f6c35f2bc"} Dec 03 11:39:32 crc kubenswrapper[5052]: I1203 11:39:32.204116 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-worker-79cc7656c5-896b7" event={"ID":"32190156-44ee-4a16-a86c-2584f6a94866","Type":"ContainerStarted","Data":"a002a7e6855ea6b8eaa15396fc2fd13adb56ddeb2fa4a0cca2f3154d7e4ea303"} Dec 03 11:39:32 crc kubenswrapper[5052]: I1203 11:39:32.422309 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 11:39:32 crc kubenswrapper[5052]: E1203 11:39:32.422790 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="d1676789-356e-4853-a302-f2c5bd3f446f" containerName="cinder-db-sync" Dec 03 11:39:32 crc kubenswrapper[5052]: I1203 11:39:32.422815 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="d1676789-356e-4853-a302-f2c5bd3f446f" containerName="cinder-db-sync" Dec 03 11:39:32 crc kubenswrapper[5052]: I1203 11:39:32.423023 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="d1676789-356e-4853-a302-f2c5bd3f446f" containerName="cinder-db-sync" Dec 03 11:39:32 crc kubenswrapper[5052]: I1203 11:39:32.433240 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:39:32 crc kubenswrapper[5052]: I1203 11:39:32.438992 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 11:39:32 crc kubenswrapper[5052]: I1203 11:39:32.439140 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cinder-cinder-dockercfg-m58ql" Dec 03 11:39:32 crc kubenswrapper[5052]: I1203 11:39:32.439290 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cinder-config-data" Dec 03 11:39:32 crc kubenswrapper[5052]: I1203 11:39:32.439330 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cinder-scheduler-config-data" Dec 03 11:39:32 crc kubenswrapper[5052]: I1203 11:39:32.441863 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cinder-scripts" Dec 03 11:39:32 crc kubenswrapper[5052]: I1203 11:39:32.568409 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tk5lf\" (UniqueName: \"kubernetes.io/projected/a8040985-8ab8-47ee-b1e8-ad692191fe4d-kube-api-access-tk5lf\") pod \"cinder-scheduler-0\" (UID: \"a8040985-8ab8-47ee-b1e8-ad692191fe4d\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:39:32 crc kubenswrapper[5052]: I1203 11:39:32.568824 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/a8040985-8ab8-47ee-b1e8-ad692191fe4d-config-data-custom\") pod \"cinder-scheduler-0\" (UID: \"a8040985-8ab8-47ee-b1e8-ad692191fe4d\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:39:32 crc kubenswrapper[5052]: I1203 11:39:32.568864 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/a8040985-8ab8-47ee-b1e8-ad692191fe4d-etc-machine-id\") pod \"cinder-scheduler-0\" (UID: \"a8040985-8ab8-47ee-b1e8-ad692191fe4d\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:39:32 crc kubenswrapper[5052]: I1203 11:39:32.568984 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a8040985-8ab8-47ee-b1e8-ad692191fe4d-config-data\") pod \"cinder-scheduler-0\" (UID: \"a8040985-8ab8-47ee-b1e8-ad692191fe4d\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:39:32 crc kubenswrapper[5052]: I1203 11:39:32.569026 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/a8040985-8ab8-47ee-b1e8-ad692191fe4d-scripts\") pod \"cinder-scheduler-0\" (UID: \"a8040985-8ab8-47ee-b1e8-ad692191fe4d\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:39:32 crc kubenswrapper[5052]: I1203 11:39:32.569146 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a8040985-8ab8-47ee-b1e8-ad692191fe4d-combined-ca-bundle\") pod \"cinder-scheduler-0\" (UID: \"a8040985-8ab8-47ee-b1e8-ad692191fe4d\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:39:32 crc kubenswrapper[5052]: I1203 11:39:32.587389 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 11:39:32 crc kubenswrapper[5052]: I1203 11:39:32.589308 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:39:32 crc kubenswrapper[5052]: I1203 11:39:32.595068 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cinder-api-config-data" Dec 03 11:39:32 crc kubenswrapper[5052]: I1203 11:39:32.595537 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 11:39:32 crc kubenswrapper[5052]: I1203 11:39:32.679662 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a8040985-8ab8-47ee-b1e8-ad692191fe4d-config-data\") pod \"cinder-scheduler-0\" (UID: \"a8040985-8ab8-47ee-b1e8-ad692191fe4d\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:39:32 crc kubenswrapper[5052]: I1203 11:39:32.679708 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/a8040985-8ab8-47ee-b1e8-ad692191fe4d-scripts\") pod \"cinder-scheduler-0\" (UID: \"a8040985-8ab8-47ee-b1e8-ad692191fe4d\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:39:32 crc kubenswrapper[5052]: I1203 11:39:32.679737 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/e72e68d4-8dd6-488b-ad4f-bf276c8eb066-logs\") pod \"cinder-api-0\" (UID: \"e72e68d4-8dd6-488b-ad4f-bf276c8eb066\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:39:32 crc kubenswrapper[5052]: I1203 11:39:32.679774 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e72e68d4-8dd6-488b-ad4f-bf276c8eb066-combined-ca-bundle\") pod \"cinder-api-0\" (UID: \"e72e68d4-8dd6-488b-ad4f-bf276c8eb066\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:39:32 crc kubenswrapper[5052]: I1203 11:39:32.679811 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a8040985-8ab8-47ee-b1e8-ad692191fe4d-combined-ca-bundle\") pod \"cinder-scheduler-0\" (UID: \"a8040985-8ab8-47ee-b1e8-ad692191fe4d\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:39:32 crc kubenswrapper[5052]: I1203 11:39:32.679853 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-tk5lf\" (UniqueName: \"kubernetes.io/projected/a8040985-8ab8-47ee-b1e8-ad692191fe4d-kube-api-access-tk5lf\") pod \"cinder-scheduler-0\" (UID: \"a8040985-8ab8-47ee-b1e8-ad692191fe4d\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:39:32 crc kubenswrapper[5052]: I1203 11:39:32.679882 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/e72e68d4-8dd6-488b-ad4f-bf276c8eb066-scripts\") pod \"cinder-api-0\" (UID: \"e72e68d4-8dd6-488b-ad4f-bf276c8eb066\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:39:32 crc kubenswrapper[5052]: I1203 11:39:32.679900 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/e72e68d4-8dd6-488b-ad4f-bf276c8eb066-config-data\") pod \"cinder-api-0\" (UID: \"e72e68d4-8dd6-488b-ad4f-bf276c8eb066\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:39:32 crc kubenswrapper[5052]: I1203 11:39:32.679921 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jhlwp\" (UniqueName: \"kubernetes.io/projected/e72e68d4-8dd6-488b-ad4f-bf276c8eb066-kube-api-access-jhlwp\") pod \"cinder-api-0\" (UID: \"e72e68d4-8dd6-488b-ad4f-bf276c8eb066\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:39:32 crc kubenswrapper[5052]: I1203 11:39:32.679942 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/a8040985-8ab8-47ee-b1e8-ad692191fe4d-config-data-custom\") pod \"cinder-scheduler-0\" (UID: \"a8040985-8ab8-47ee-b1e8-ad692191fe4d\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:39:32 crc kubenswrapper[5052]: I1203 11:39:32.679963 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/e72e68d4-8dd6-488b-ad4f-bf276c8eb066-config-data-custom\") pod \"cinder-api-0\" (UID: \"e72e68d4-8dd6-488b-ad4f-bf276c8eb066\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:39:32 crc kubenswrapper[5052]: I1203 11:39:32.679981 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/e72e68d4-8dd6-488b-ad4f-bf276c8eb066-etc-machine-id\") pod \"cinder-api-0\" (UID: \"e72e68d4-8dd6-488b-ad4f-bf276c8eb066\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:39:32 crc kubenswrapper[5052]: I1203 11:39:32.680006 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/a8040985-8ab8-47ee-b1e8-ad692191fe4d-etc-machine-id\") pod \"cinder-scheduler-0\" (UID: \"a8040985-8ab8-47ee-b1e8-ad692191fe4d\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:39:32 crc kubenswrapper[5052]: I1203 11:39:32.680103 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/a8040985-8ab8-47ee-b1e8-ad692191fe4d-etc-machine-id\") pod \"cinder-scheduler-0\" (UID: \"a8040985-8ab8-47ee-b1e8-ad692191fe4d\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:39:32 crc kubenswrapper[5052]: I1203 11:39:32.694963 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a8040985-8ab8-47ee-b1e8-ad692191fe4d-config-data\") pod \"cinder-scheduler-0\" (UID: \"a8040985-8ab8-47ee-b1e8-ad692191fe4d\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:39:32 crc kubenswrapper[5052]: I1203 11:39:32.696772 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/a8040985-8ab8-47ee-b1e8-ad692191fe4d-scripts\") pod \"cinder-scheduler-0\" (UID: \"a8040985-8ab8-47ee-b1e8-ad692191fe4d\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:39:32 crc kubenswrapper[5052]: I1203 11:39:32.704125 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a8040985-8ab8-47ee-b1e8-ad692191fe4d-combined-ca-bundle\") pod \"cinder-scheduler-0\" (UID: \"a8040985-8ab8-47ee-b1e8-ad692191fe4d\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:39:32 crc kubenswrapper[5052]: I1203 11:39:32.705350 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/a8040985-8ab8-47ee-b1e8-ad692191fe4d-config-data-custom\") pod \"cinder-scheduler-0\" (UID: \"a8040985-8ab8-47ee-b1e8-ad692191fe4d\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:39:32 crc kubenswrapper[5052]: I1203 11:39:32.717120 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-tk5lf\" (UniqueName: \"kubernetes.io/projected/a8040985-8ab8-47ee-b1e8-ad692191fe4d-kube-api-access-tk5lf\") pod \"cinder-scheduler-0\" (UID: \"a8040985-8ab8-47ee-b1e8-ad692191fe4d\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:39:32 crc kubenswrapper[5052]: I1203 11:39:32.765967 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:39:32 crc kubenswrapper[5052]: I1203 11:39:32.790539 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/e72e68d4-8dd6-488b-ad4f-bf276c8eb066-logs\") pod \"cinder-api-0\" (UID: \"e72e68d4-8dd6-488b-ad4f-bf276c8eb066\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:39:32 crc kubenswrapper[5052]: I1203 11:39:32.790615 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e72e68d4-8dd6-488b-ad4f-bf276c8eb066-combined-ca-bundle\") pod \"cinder-api-0\" (UID: \"e72e68d4-8dd6-488b-ad4f-bf276c8eb066\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:39:32 crc kubenswrapper[5052]: I1203 11:39:32.790711 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/e72e68d4-8dd6-488b-ad4f-bf276c8eb066-scripts\") pod \"cinder-api-0\" (UID: \"e72e68d4-8dd6-488b-ad4f-bf276c8eb066\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:39:32 crc kubenswrapper[5052]: I1203 11:39:32.790733 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/e72e68d4-8dd6-488b-ad4f-bf276c8eb066-config-data\") pod \"cinder-api-0\" (UID: \"e72e68d4-8dd6-488b-ad4f-bf276c8eb066\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:39:32 crc kubenswrapper[5052]: I1203 11:39:32.790761 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-jhlwp\" (UniqueName: \"kubernetes.io/projected/e72e68d4-8dd6-488b-ad4f-bf276c8eb066-kube-api-access-jhlwp\") pod \"cinder-api-0\" (UID: \"e72e68d4-8dd6-488b-ad4f-bf276c8eb066\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:39:32 crc kubenswrapper[5052]: I1203 11:39:32.790794 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/e72e68d4-8dd6-488b-ad4f-bf276c8eb066-config-data-custom\") pod \"cinder-api-0\" (UID: \"e72e68d4-8dd6-488b-ad4f-bf276c8eb066\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:39:32 crc kubenswrapper[5052]: I1203 11:39:32.790816 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/e72e68d4-8dd6-488b-ad4f-bf276c8eb066-etc-machine-id\") pod \"cinder-api-0\" (UID: \"e72e68d4-8dd6-488b-ad4f-bf276c8eb066\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:39:32 crc kubenswrapper[5052]: I1203 11:39:32.790928 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/e72e68d4-8dd6-488b-ad4f-bf276c8eb066-etc-machine-id\") pod \"cinder-api-0\" (UID: \"e72e68d4-8dd6-488b-ad4f-bf276c8eb066\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:39:32 crc kubenswrapper[5052]: I1203 11:39:32.791319 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/e72e68d4-8dd6-488b-ad4f-bf276c8eb066-logs\") pod \"cinder-api-0\" (UID: \"e72e68d4-8dd6-488b-ad4f-bf276c8eb066\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:39:32 crc kubenswrapper[5052]: I1203 11:39:32.814017 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/e72e68d4-8dd6-488b-ad4f-bf276c8eb066-scripts\") pod \"cinder-api-0\" (UID: \"e72e68d4-8dd6-488b-ad4f-bf276c8eb066\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:39:32 crc kubenswrapper[5052]: I1203 11:39:32.815012 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/e72e68d4-8dd6-488b-ad4f-bf276c8eb066-config-data\") pod \"cinder-api-0\" (UID: \"e72e68d4-8dd6-488b-ad4f-bf276c8eb066\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:39:32 crc kubenswrapper[5052]: I1203 11:39:32.824278 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/e72e68d4-8dd6-488b-ad4f-bf276c8eb066-config-data-custom\") pod \"cinder-api-0\" (UID: \"e72e68d4-8dd6-488b-ad4f-bf276c8eb066\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:39:32 crc kubenswrapper[5052]: I1203 11:39:32.824982 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e72e68d4-8dd6-488b-ad4f-bf276c8eb066-combined-ca-bundle\") pod \"cinder-api-0\" (UID: \"e72e68d4-8dd6-488b-ad4f-bf276c8eb066\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:39:32 crc kubenswrapper[5052]: I1203 11:39:32.854112 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-jhlwp\" (UniqueName: \"kubernetes.io/projected/e72e68d4-8dd6-488b-ad4f-bf276c8eb066-kube-api-access-jhlwp\") pod \"cinder-api-0\" (UID: \"e72e68d4-8dd6-488b-ad4f-bf276c8eb066\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:39:32 crc kubenswrapper[5052]: I1203 11:39:32.920305 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:39:33 crc kubenswrapper[5052]: I1203 11:39:33.214784 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-65968d9848-mntb6" event={"ID":"ef47ba52-772d-4732-bd15-e7513e626be9","Type":"ContainerStarted","Data":"f531b9f35a146882772e5bb658e2970143bf9db9263912e975397ff80f1c2cda"} Dec 03 11:39:33 crc kubenswrapper[5052]: I1203 11:39:33.216112 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/barbican-api-65968d9848-mntb6" Dec 03 11:39:33 crc kubenswrapper[5052]: I1203 11:39:33.216135 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/barbican-api-65968d9848-mntb6" Dec 03 11:39:33 crc kubenswrapper[5052]: I1203 11:39:33.220364 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"70e2b274-dd4d-409a-8d45-a09af330f7a3","Type":"ContainerStarted","Data":"466f8e75a4f4613fdb58e65fc3b11d64b99b313618c5c45ebe14444fc70dbacf"} Dec 03 11:39:33 crc kubenswrapper[5052]: I1203 11:39:33.256641 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/barbican-api-65968d9848-mntb6" podStartSLOduration=3.256599768 podStartE2EDuration="3.256599768s" podCreationTimestamp="2025-12-03 11:39:30 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:39:33.243290033 +0000 UTC m=+1163.831531797" watchObservedRunningTime="2025-12-03 11:39:33.256599768 +0000 UTC m=+1163.844841532" Dec 03 11:39:33 crc kubenswrapper[5052]: I1203 11:39:33.439242 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 11:39:33 crc kubenswrapper[5052]: I1203 11:39:33.459091 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 11:39:33 crc kubenswrapper[5052]: W1203 11:39:33.918969 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode72e68d4_8dd6_488b_ad4f_bf276c8eb066.slice/crio-306198b04aa722a5baaa142bff8bdd0f20a5700dcaf668e9320954ca69635fea WatchSource:0}: Error finding container 306198b04aa722a5baaa142bff8bdd0f20a5700dcaf668e9320954ca69635fea: Status 404 returned error can't find the container with id 306198b04aa722a5baaa142bff8bdd0f20a5700dcaf668e9320954ca69635fea Dec 03 11:39:34 crc kubenswrapper[5052]: I1203 11:39:34.237171 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"70e2b274-dd4d-409a-8d45-a09af330f7a3","Type":"ContainerStarted","Data":"89b928d0bee5a1b120bb1a4de7f35f330ce3fb961abc80c74f6017fd2b9b6ae4"} Dec 03 11:39:34 crc kubenswrapper[5052]: I1203 11:39:34.242162 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"e72e68d4-8dd6-488b-ad4f-bf276c8eb066","Type":"ContainerStarted","Data":"306198b04aa722a5baaa142bff8bdd0f20a5700dcaf668e9320954ca69635fea"} Dec 03 11:39:34 crc kubenswrapper[5052]: I1203 11:39:34.245013 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"a8040985-8ab8-47ee-b1e8-ad692191fe4d","Type":"ContainerStarted","Data":"c935cfb9dda4b09af01053f4f22591764d5ec573c22e1071aea00996d9103899"} Dec 03 11:39:35 crc kubenswrapper[5052]: I1203 11:39:35.267786 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-keystone-listener-649457694-r7gzw" event={"ID":"8c4732f9-0f9e-4600-aed9-6a230cbf7b16","Type":"ContainerStarted","Data":"cc72d236c40c2e7f87295c5f31e63f10a061fd78a90f6c26b95db3911d675ff7"} Dec 03 11:39:35 crc kubenswrapper[5052]: I1203 11:39:35.268388 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-keystone-listener-649457694-r7gzw" event={"ID":"8c4732f9-0f9e-4600-aed9-6a230cbf7b16","Type":"ContainerStarted","Data":"61b746fb82156eb75de6a2356739407db21615dda656157379f9510b62f96730"} Dec 03 11:39:35 crc kubenswrapper[5052]: I1203 11:39:35.273584 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-worker-79cc7656c5-896b7" event={"ID":"32190156-44ee-4a16-a86c-2584f6a94866","Type":"ContainerStarted","Data":"63e884f6db70c59723e3f8be480e8d365efc639a9133cf1e01bf9eb55912f224"} Dec 03 11:39:35 crc kubenswrapper[5052]: I1203 11:39:35.273638 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-worker-79cc7656c5-896b7" event={"ID":"32190156-44ee-4a16-a86c-2584f6a94866","Type":"ContainerStarted","Data":"6e34f31b2be164f36807f4119b800ae1a884c550a462f7a9c586c37f82893863"} Dec 03 11:39:35 crc kubenswrapper[5052]: I1203 11:39:35.277243 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"70e2b274-dd4d-409a-8d45-a09af330f7a3","Type":"ContainerStarted","Data":"33c7c237d3624c744edaa6397912e20b3e447503cda667a0d60162d16ad547d5"} Dec 03 11:39:35 crc kubenswrapper[5052]: I1203 11:39:35.278047 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:39:35 crc kubenswrapper[5052]: I1203 11:39:35.285026 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"e72e68d4-8dd6-488b-ad4f-bf276c8eb066","Type":"ContainerStarted","Data":"d9e8edf0a3614e6406ed081e41d90643bdb16b77548d6948dc47e3a4064b333e"} Dec 03 11:39:35 crc kubenswrapper[5052]: I1203 11:39:35.292582 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/barbican-keystone-listener-649457694-r7gzw" podStartSLOduration=2.9587031980000003 podStartE2EDuration="5.292559201s" podCreationTimestamp="2025-12-03 11:39:30 +0000 UTC" firstStartedPulling="2025-12-03 11:39:31.654358055 +0000 UTC m=+1162.242599819" lastFinishedPulling="2025-12-03 11:39:33.988214068 +0000 UTC m=+1164.576455822" observedRunningTime="2025-12-03 11:39:35.288735621 +0000 UTC m=+1165.876977395" watchObservedRunningTime="2025-12-03 11:39:35.292559201 +0000 UTC m=+1165.880800975" Dec 03 11:39:35 crc kubenswrapper[5052]: I1203 11:39:35.317149 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/ceilometer-0" podStartSLOduration=2.066095825 podStartE2EDuration="6.317123741s" podCreationTimestamp="2025-12-03 11:39:29 +0000 UTC" firstStartedPulling="2025-12-03 11:39:30.395077013 +0000 UTC m=+1160.983318777" lastFinishedPulling="2025-12-03 11:39:34.646104929 +0000 UTC m=+1165.234346693" observedRunningTime="2025-12-03 11:39:35.315522274 +0000 UTC m=+1165.903764058" watchObservedRunningTime="2025-12-03 11:39:35.317123741 +0000 UTC m=+1165.905365525" Dec 03 11:39:35 crc kubenswrapper[5052]: I1203 11:39:35.332673 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/barbican-worker-79cc7656c5-896b7" podStartSLOduration=3.05361674 podStartE2EDuration="5.332659499s" podCreationTimestamp="2025-12-03 11:39:30 +0000 UTC" firstStartedPulling="2025-12-03 11:39:31.719208858 +0000 UTC m=+1162.307450622" lastFinishedPulling="2025-12-03 11:39:33.998251617 +0000 UTC m=+1164.586493381" observedRunningTime="2025-12-03 11:39:35.330114836 +0000 UTC m=+1165.918356600" watchObservedRunningTime="2025-12-03 11:39:35.332659499 +0000 UTC m=+1165.920901263" Dec 03 11:39:36 crc kubenswrapper[5052]: I1203 11:39:36.260828 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 11:39:36 crc kubenswrapper[5052]: I1203 11:39:36.294146 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"a8040985-8ab8-47ee-b1e8-ad692191fe4d","Type":"ContainerStarted","Data":"27a49abcaf4c4c47a542806b2d85ebfba88c348a219c09be22e5c5a52566b889"} Dec 03 11:39:36 crc kubenswrapper[5052]: I1203 11:39:36.294249 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"a8040985-8ab8-47ee-b1e8-ad692191fe4d","Type":"ContainerStarted","Data":"6920a886819e6444771d9e24d02bda1230c57f492aad5dfb84f1dd9fab7de559"} Dec 03 11:39:36 crc kubenswrapper[5052]: I1203 11:39:36.296784 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"e72e68d4-8dd6-488b-ad4f-bf276c8eb066","Type":"ContainerStarted","Data":"64c8b84a240fd182cefc93f400d13b9d3ff52a744deab5121afb6b516b12861b"} Dec 03 11:39:36 crc kubenswrapper[5052]: I1203 11:39:36.338876 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/cinder-api-0" podStartSLOduration=4.338855845 podStartE2EDuration="4.338855845s" podCreationTimestamp="2025-12-03 11:39:32 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:39:36.334411206 +0000 UTC m=+1166.922652990" watchObservedRunningTime="2025-12-03 11:39:36.338855845 +0000 UTC m=+1166.927097609" Dec 03 11:39:36 crc kubenswrapper[5052]: I1203 11:39:36.342601 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/cinder-scheduler-0" podStartSLOduration=3.414814879 podStartE2EDuration="4.342581892s" podCreationTimestamp="2025-12-03 11:39:32 +0000 UTC" firstStartedPulling="2025-12-03 11:39:33.969733204 +0000 UTC m=+1164.557974968" lastFinishedPulling="2025-12-03 11:39:34.897500207 +0000 UTC m=+1165.485741981" observedRunningTime="2025-12-03 11:39:36.318071835 +0000 UTC m=+1166.906313599" watchObservedRunningTime="2025-12-03 11:39:36.342581892 +0000 UTC m=+1166.930823656" Dec 03 11:39:37 crc kubenswrapper[5052]: I1203 11:39:37.306323 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:39:37 crc kubenswrapper[5052]: I1203 11:39:37.306803 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/cinder-api-0" podUID="e72e68d4-8dd6-488b-ad4f-bf276c8eb066" containerName="cinder-api-log" containerID="cri-o://d9e8edf0a3614e6406ed081e41d90643bdb16b77548d6948dc47e3a4064b333e" gracePeriod=30 Dec 03 11:39:37 crc kubenswrapper[5052]: I1203 11:39:37.307053 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/cinder-api-0" podUID="e72e68d4-8dd6-488b-ad4f-bf276c8eb066" containerName="cinder-api" containerID="cri-o://64c8b84a240fd182cefc93f400d13b9d3ff52a744deab5121afb6b516b12861b" gracePeriod=30 Dec 03 11:39:37 crc kubenswrapper[5052]: I1203 11:39:37.767690 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:39:37 crc kubenswrapper[5052]: I1203 11:39:37.920477 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:39:38 crc kubenswrapper[5052]: I1203 11:39:38.014563 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/barbican-api-66f4bb9676-j6vzc"] Dec 03 11:39:38 crc kubenswrapper[5052]: E1203 11:39:38.015111 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="e72e68d4-8dd6-488b-ad4f-bf276c8eb066" containerName="cinder-api" Dec 03 11:39:38 crc kubenswrapper[5052]: I1203 11:39:38.015132 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="e72e68d4-8dd6-488b-ad4f-bf276c8eb066" containerName="cinder-api" Dec 03 11:39:38 crc kubenswrapper[5052]: E1203 11:39:38.015161 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="e72e68d4-8dd6-488b-ad4f-bf276c8eb066" containerName="cinder-api-log" Dec 03 11:39:38 crc kubenswrapper[5052]: I1203 11:39:38.015169 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="e72e68d4-8dd6-488b-ad4f-bf276c8eb066" containerName="cinder-api-log" Dec 03 11:39:38 crc kubenswrapper[5052]: I1203 11:39:38.015384 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="e72e68d4-8dd6-488b-ad4f-bf276c8eb066" containerName="cinder-api-log" Dec 03 11:39:38 crc kubenswrapper[5052]: I1203 11:39:38.015412 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="e72e68d4-8dd6-488b-ad4f-bf276c8eb066" containerName="cinder-api" Dec 03 11:39:38 crc kubenswrapper[5052]: I1203 11:39:38.016735 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-api-66f4bb9676-j6vzc" Dec 03 11:39:38 crc kubenswrapper[5052]: I1203 11:39:38.017196 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jhlwp\" (UniqueName: \"kubernetes.io/projected/e72e68d4-8dd6-488b-ad4f-bf276c8eb066-kube-api-access-jhlwp\") pod \"e72e68d4-8dd6-488b-ad4f-bf276c8eb066\" (UID: \"e72e68d4-8dd6-488b-ad4f-bf276c8eb066\") " Dec 03 11:39:38 crc kubenswrapper[5052]: I1203 11:39:38.017243 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/e72e68d4-8dd6-488b-ad4f-bf276c8eb066-scripts\") pod \"e72e68d4-8dd6-488b-ad4f-bf276c8eb066\" (UID: \"e72e68d4-8dd6-488b-ad4f-bf276c8eb066\") " Dec 03 11:39:38 crc kubenswrapper[5052]: I1203 11:39:38.017412 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/e72e68d4-8dd6-488b-ad4f-bf276c8eb066-logs\") pod \"e72e68d4-8dd6-488b-ad4f-bf276c8eb066\" (UID: \"e72e68d4-8dd6-488b-ad4f-bf276c8eb066\") " Dec 03 11:39:38 crc kubenswrapper[5052]: I1203 11:39:38.017469 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/e72e68d4-8dd6-488b-ad4f-bf276c8eb066-config-data-custom\") pod \"e72e68d4-8dd6-488b-ad4f-bf276c8eb066\" (UID: \"e72e68d4-8dd6-488b-ad4f-bf276c8eb066\") " Dec 03 11:39:38 crc kubenswrapper[5052]: I1203 11:39:38.017503 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/e72e68d4-8dd6-488b-ad4f-bf276c8eb066-etc-machine-id\") pod \"e72e68d4-8dd6-488b-ad4f-bf276c8eb066\" (UID: \"e72e68d4-8dd6-488b-ad4f-bf276c8eb066\") " Dec 03 11:39:38 crc kubenswrapper[5052]: I1203 11:39:38.017605 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e72e68d4-8dd6-488b-ad4f-bf276c8eb066-combined-ca-bundle\") pod \"e72e68d4-8dd6-488b-ad4f-bf276c8eb066\" (UID: \"e72e68d4-8dd6-488b-ad4f-bf276c8eb066\") " Dec 03 11:39:38 crc kubenswrapper[5052]: I1203 11:39:38.017659 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/e72e68d4-8dd6-488b-ad4f-bf276c8eb066-config-data\") pod \"e72e68d4-8dd6-488b-ad4f-bf276c8eb066\" (UID: \"e72e68d4-8dd6-488b-ad4f-bf276c8eb066\") " Dec 03 11:39:38 crc kubenswrapper[5052]: I1203 11:39:38.019229 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/e72e68d4-8dd6-488b-ad4f-bf276c8eb066-etc-machine-id" (OuterVolumeSpecName: "etc-machine-id") pod "e72e68d4-8dd6-488b-ad4f-bf276c8eb066" (UID: "e72e68d4-8dd6-488b-ad4f-bf276c8eb066"). InnerVolumeSpecName "etc-machine-id". PluginName "kubernetes.io/host-path", VolumeGidValue "" Dec 03 11:39:38 crc kubenswrapper[5052]: I1203 11:39:38.022386 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e72e68d4-8dd6-488b-ad4f-bf276c8eb066-logs" (OuterVolumeSpecName: "logs") pod "e72e68d4-8dd6-488b-ad4f-bf276c8eb066" (UID: "e72e68d4-8dd6-488b-ad4f-bf276c8eb066"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:39:38 crc kubenswrapper[5052]: I1203 11:39:38.026495 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e72e68d4-8dd6-488b-ad4f-bf276c8eb066-kube-api-access-jhlwp" (OuterVolumeSpecName: "kube-api-access-jhlwp") pod "e72e68d4-8dd6-488b-ad4f-bf276c8eb066" (UID: "e72e68d4-8dd6-488b-ad4f-bf276c8eb066"). InnerVolumeSpecName "kube-api-access-jhlwp". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:39:38 crc kubenswrapper[5052]: I1203 11:39:38.026862 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e72e68d4-8dd6-488b-ad4f-bf276c8eb066-scripts" (OuterVolumeSpecName: "scripts") pod "e72e68d4-8dd6-488b-ad4f-bf276c8eb066" (UID: "e72e68d4-8dd6-488b-ad4f-bf276c8eb066"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:39:38 crc kubenswrapper[5052]: I1203 11:39:38.026942 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-barbican-internal-svc" Dec 03 11:39:38 crc kubenswrapper[5052]: I1203 11:39:38.027438 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-barbican-public-svc" Dec 03 11:39:38 crc kubenswrapper[5052]: I1203 11:39:38.037467 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-api-66f4bb9676-j6vzc"] Dec 03 11:39:38 crc kubenswrapper[5052]: I1203 11:39:38.070219 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e72e68d4-8dd6-488b-ad4f-bf276c8eb066-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "e72e68d4-8dd6-488b-ad4f-bf276c8eb066" (UID: "e72e68d4-8dd6-488b-ad4f-bf276c8eb066"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:39:38 crc kubenswrapper[5052]: I1203 11:39:38.071698 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e72e68d4-8dd6-488b-ad4f-bf276c8eb066-config-data-custom" (OuterVolumeSpecName: "config-data-custom") pod "e72e68d4-8dd6-488b-ad4f-bf276c8eb066" (UID: "e72e68d4-8dd6-488b-ad4f-bf276c8eb066"). InnerVolumeSpecName "config-data-custom". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:39:38 crc kubenswrapper[5052]: I1203 11:39:38.092913 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e72e68d4-8dd6-488b-ad4f-bf276c8eb066-config-data" (OuterVolumeSpecName: "config-data") pod "e72e68d4-8dd6-488b-ad4f-bf276c8eb066" (UID: "e72e68d4-8dd6-488b-ad4f-bf276c8eb066"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:39:38 crc kubenswrapper[5052]: I1203 11:39:38.120085 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/fe69e54e-ae29-49e2-8a21-0ff81cc0100f-combined-ca-bundle\") pod \"barbican-api-66f4bb9676-j6vzc\" (UID: \"fe69e54e-ae29-49e2-8a21-0ff81cc0100f\") " pod="openstack-kuttl-tests/barbican-api-66f4bb9676-j6vzc" Dec 03 11:39:38 crc kubenswrapper[5052]: I1203 11:39:38.120181 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/fe69e54e-ae29-49e2-8a21-0ff81cc0100f-internal-tls-certs\") pod \"barbican-api-66f4bb9676-j6vzc\" (UID: \"fe69e54e-ae29-49e2-8a21-0ff81cc0100f\") " pod="openstack-kuttl-tests/barbican-api-66f4bb9676-j6vzc" Dec 03 11:39:38 crc kubenswrapper[5052]: I1203 11:39:38.120283 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/fe69e54e-ae29-49e2-8a21-0ff81cc0100f-public-tls-certs\") pod \"barbican-api-66f4bb9676-j6vzc\" (UID: \"fe69e54e-ae29-49e2-8a21-0ff81cc0100f\") " pod="openstack-kuttl-tests/barbican-api-66f4bb9676-j6vzc" Dec 03 11:39:38 crc kubenswrapper[5052]: I1203 11:39:38.120324 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/fe69e54e-ae29-49e2-8a21-0ff81cc0100f-config-data-custom\") pod \"barbican-api-66f4bb9676-j6vzc\" (UID: \"fe69e54e-ae29-49e2-8a21-0ff81cc0100f\") " pod="openstack-kuttl-tests/barbican-api-66f4bb9676-j6vzc" Dec 03 11:39:38 crc kubenswrapper[5052]: I1203 11:39:38.120425 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/fe69e54e-ae29-49e2-8a21-0ff81cc0100f-config-data\") pod \"barbican-api-66f4bb9676-j6vzc\" (UID: \"fe69e54e-ae29-49e2-8a21-0ff81cc0100f\") " pod="openstack-kuttl-tests/barbican-api-66f4bb9676-j6vzc" Dec 03 11:39:38 crc kubenswrapper[5052]: I1203 11:39:38.120480 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9sj25\" (UniqueName: \"kubernetes.io/projected/fe69e54e-ae29-49e2-8a21-0ff81cc0100f-kube-api-access-9sj25\") pod \"barbican-api-66f4bb9676-j6vzc\" (UID: \"fe69e54e-ae29-49e2-8a21-0ff81cc0100f\") " pod="openstack-kuttl-tests/barbican-api-66f4bb9676-j6vzc" Dec 03 11:39:38 crc kubenswrapper[5052]: I1203 11:39:38.120552 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/fe69e54e-ae29-49e2-8a21-0ff81cc0100f-logs\") pod \"barbican-api-66f4bb9676-j6vzc\" (UID: \"fe69e54e-ae29-49e2-8a21-0ff81cc0100f\") " pod="openstack-kuttl-tests/barbican-api-66f4bb9676-j6vzc" Dec 03 11:39:38 crc kubenswrapper[5052]: I1203 11:39:38.120639 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/e72e68d4-8dd6-488b-ad4f-bf276c8eb066-logs\") on node \"crc\" DevicePath \"\"" Dec 03 11:39:38 crc kubenswrapper[5052]: I1203 11:39:38.120664 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/e72e68d4-8dd6-488b-ad4f-bf276c8eb066-config-data-custom\") on node \"crc\" DevicePath \"\"" Dec 03 11:39:38 crc kubenswrapper[5052]: I1203 11:39:38.120680 5052 reconciler_common.go:293] "Volume detached for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/e72e68d4-8dd6-488b-ad4f-bf276c8eb066-etc-machine-id\") on node \"crc\" DevicePath \"\"" Dec 03 11:39:38 crc kubenswrapper[5052]: I1203 11:39:38.120693 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e72e68d4-8dd6-488b-ad4f-bf276c8eb066-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:39:38 crc kubenswrapper[5052]: I1203 11:39:38.120706 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/e72e68d4-8dd6-488b-ad4f-bf276c8eb066-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:39:38 crc kubenswrapper[5052]: I1203 11:39:38.120719 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-jhlwp\" (UniqueName: \"kubernetes.io/projected/e72e68d4-8dd6-488b-ad4f-bf276c8eb066-kube-api-access-jhlwp\") on node \"crc\" DevicePath \"\"" Dec 03 11:39:38 crc kubenswrapper[5052]: I1203 11:39:38.120730 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/e72e68d4-8dd6-488b-ad4f-bf276c8eb066-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:39:38 crc kubenswrapper[5052]: I1203 11:39:38.224608 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/fe69e54e-ae29-49e2-8a21-0ff81cc0100f-config-data-custom\") pod \"barbican-api-66f4bb9676-j6vzc\" (UID: \"fe69e54e-ae29-49e2-8a21-0ff81cc0100f\") " pod="openstack-kuttl-tests/barbican-api-66f4bb9676-j6vzc" Dec 03 11:39:38 crc kubenswrapper[5052]: I1203 11:39:38.224785 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/fe69e54e-ae29-49e2-8a21-0ff81cc0100f-config-data\") pod \"barbican-api-66f4bb9676-j6vzc\" (UID: \"fe69e54e-ae29-49e2-8a21-0ff81cc0100f\") " pod="openstack-kuttl-tests/barbican-api-66f4bb9676-j6vzc" Dec 03 11:39:38 crc kubenswrapper[5052]: I1203 11:39:38.224810 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-9sj25\" (UniqueName: \"kubernetes.io/projected/fe69e54e-ae29-49e2-8a21-0ff81cc0100f-kube-api-access-9sj25\") pod \"barbican-api-66f4bb9676-j6vzc\" (UID: \"fe69e54e-ae29-49e2-8a21-0ff81cc0100f\") " pod="openstack-kuttl-tests/barbican-api-66f4bb9676-j6vzc" Dec 03 11:39:38 crc kubenswrapper[5052]: I1203 11:39:38.224841 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/fe69e54e-ae29-49e2-8a21-0ff81cc0100f-logs\") pod \"barbican-api-66f4bb9676-j6vzc\" (UID: \"fe69e54e-ae29-49e2-8a21-0ff81cc0100f\") " pod="openstack-kuttl-tests/barbican-api-66f4bb9676-j6vzc" Dec 03 11:39:38 crc kubenswrapper[5052]: I1203 11:39:38.224887 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/fe69e54e-ae29-49e2-8a21-0ff81cc0100f-combined-ca-bundle\") pod \"barbican-api-66f4bb9676-j6vzc\" (UID: \"fe69e54e-ae29-49e2-8a21-0ff81cc0100f\") " pod="openstack-kuttl-tests/barbican-api-66f4bb9676-j6vzc" Dec 03 11:39:38 crc kubenswrapper[5052]: I1203 11:39:38.224930 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/fe69e54e-ae29-49e2-8a21-0ff81cc0100f-internal-tls-certs\") pod \"barbican-api-66f4bb9676-j6vzc\" (UID: \"fe69e54e-ae29-49e2-8a21-0ff81cc0100f\") " pod="openstack-kuttl-tests/barbican-api-66f4bb9676-j6vzc" Dec 03 11:39:38 crc kubenswrapper[5052]: I1203 11:39:38.224979 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/fe69e54e-ae29-49e2-8a21-0ff81cc0100f-public-tls-certs\") pod \"barbican-api-66f4bb9676-j6vzc\" (UID: \"fe69e54e-ae29-49e2-8a21-0ff81cc0100f\") " pod="openstack-kuttl-tests/barbican-api-66f4bb9676-j6vzc" Dec 03 11:39:38 crc kubenswrapper[5052]: I1203 11:39:38.225289 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/fe69e54e-ae29-49e2-8a21-0ff81cc0100f-logs\") pod \"barbican-api-66f4bb9676-j6vzc\" (UID: \"fe69e54e-ae29-49e2-8a21-0ff81cc0100f\") " pod="openstack-kuttl-tests/barbican-api-66f4bb9676-j6vzc" Dec 03 11:39:38 crc kubenswrapper[5052]: I1203 11:39:38.228602 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/fe69e54e-ae29-49e2-8a21-0ff81cc0100f-combined-ca-bundle\") pod \"barbican-api-66f4bb9676-j6vzc\" (UID: \"fe69e54e-ae29-49e2-8a21-0ff81cc0100f\") " pod="openstack-kuttl-tests/barbican-api-66f4bb9676-j6vzc" Dec 03 11:39:38 crc kubenswrapper[5052]: I1203 11:39:38.229039 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/fe69e54e-ae29-49e2-8a21-0ff81cc0100f-public-tls-certs\") pod \"barbican-api-66f4bb9676-j6vzc\" (UID: \"fe69e54e-ae29-49e2-8a21-0ff81cc0100f\") " pod="openstack-kuttl-tests/barbican-api-66f4bb9676-j6vzc" Dec 03 11:39:38 crc kubenswrapper[5052]: I1203 11:39:38.230030 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/fe69e54e-ae29-49e2-8a21-0ff81cc0100f-config-data\") pod \"barbican-api-66f4bb9676-j6vzc\" (UID: \"fe69e54e-ae29-49e2-8a21-0ff81cc0100f\") " pod="openstack-kuttl-tests/barbican-api-66f4bb9676-j6vzc" Dec 03 11:39:38 crc kubenswrapper[5052]: I1203 11:39:38.230151 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/fe69e54e-ae29-49e2-8a21-0ff81cc0100f-config-data-custom\") pod \"barbican-api-66f4bb9676-j6vzc\" (UID: \"fe69e54e-ae29-49e2-8a21-0ff81cc0100f\") " pod="openstack-kuttl-tests/barbican-api-66f4bb9676-j6vzc" Dec 03 11:39:38 crc kubenswrapper[5052]: I1203 11:39:38.232168 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/fe69e54e-ae29-49e2-8a21-0ff81cc0100f-internal-tls-certs\") pod \"barbican-api-66f4bb9676-j6vzc\" (UID: \"fe69e54e-ae29-49e2-8a21-0ff81cc0100f\") " pod="openstack-kuttl-tests/barbican-api-66f4bb9676-j6vzc" Dec 03 11:39:38 crc kubenswrapper[5052]: I1203 11:39:38.253902 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-9sj25\" (UniqueName: \"kubernetes.io/projected/fe69e54e-ae29-49e2-8a21-0ff81cc0100f-kube-api-access-9sj25\") pod \"barbican-api-66f4bb9676-j6vzc\" (UID: \"fe69e54e-ae29-49e2-8a21-0ff81cc0100f\") " pod="openstack-kuttl-tests/barbican-api-66f4bb9676-j6vzc" Dec 03 11:39:38 crc kubenswrapper[5052]: I1203 11:39:38.324029 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:39:38 crc kubenswrapper[5052]: I1203 11:39:38.324051 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"e72e68d4-8dd6-488b-ad4f-bf276c8eb066","Type":"ContainerDied","Data":"64c8b84a240fd182cefc93f400d13b9d3ff52a744deab5121afb6b516b12861b"} Dec 03 11:39:38 crc kubenswrapper[5052]: I1203 11:39:38.324088 5052 scope.go:117] "RemoveContainer" containerID="64c8b84a240fd182cefc93f400d13b9d3ff52a744deab5121afb6b516b12861b" Dec 03 11:39:38 crc kubenswrapper[5052]: I1203 11:39:38.324007 5052 generic.go:334] "Generic (PLEG): container finished" podID="e72e68d4-8dd6-488b-ad4f-bf276c8eb066" containerID="64c8b84a240fd182cefc93f400d13b9d3ff52a744deab5121afb6b516b12861b" exitCode=0 Dec 03 11:39:38 crc kubenswrapper[5052]: I1203 11:39:38.324191 5052 generic.go:334] "Generic (PLEG): container finished" podID="e72e68d4-8dd6-488b-ad4f-bf276c8eb066" containerID="d9e8edf0a3614e6406ed081e41d90643bdb16b77548d6948dc47e3a4064b333e" exitCode=143 Dec 03 11:39:38 crc kubenswrapper[5052]: I1203 11:39:38.324301 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"e72e68d4-8dd6-488b-ad4f-bf276c8eb066","Type":"ContainerDied","Data":"d9e8edf0a3614e6406ed081e41d90643bdb16b77548d6948dc47e3a4064b333e"} Dec 03 11:39:38 crc kubenswrapper[5052]: I1203 11:39:38.324336 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"e72e68d4-8dd6-488b-ad4f-bf276c8eb066","Type":"ContainerDied","Data":"306198b04aa722a5baaa142bff8bdd0f20a5700dcaf668e9320954ca69635fea"} Dec 03 11:39:38 crc kubenswrapper[5052]: I1203 11:39:38.357102 5052 scope.go:117] "RemoveContainer" containerID="d9e8edf0a3614e6406ed081e41d90643bdb16b77548d6948dc47e3a4064b333e" Dec 03 11:39:38 crc kubenswrapper[5052]: I1203 11:39:38.368123 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 11:39:38 crc kubenswrapper[5052]: I1203 11:39:38.377878 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 11:39:38 crc kubenswrapper[5052]: I1203 11:39:38.386915 5052 scope.go:117] "RemoveContainer" containerID="64c8b84a240fd182cefc93f400d13b9d3ff52a744deab5121afb6b516b12861b" Dec 03 11:39:38 crc kubenswrapper[5052]: I1203 11:39:38.387216 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 11:39:38 crc kubenswrapper[5052]: I1203 11:39:38.389064 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:39:38 crc kubenswrapper[5052]: I1203 11:39:38.390537 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-api-66f4bb9676-j6vzc" Dec 03 11:39:38 crc kubenswrapper[5052]: E1203 11:39:38.391128 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"64c8b84a240fd182cefc93f400d13b9d3ff52a744deab5121afb6b516b12861b\": container with ID starting with 64c8b84a240fd182cefc93f400d13b9d3ff52a744deab5121afb6b516b12861b not found: ID does not exist" containerID="64c8b84a240fd182cefc93f400d13b9d3ff52a744deab5121afb6b516b12861b" Dec 03 11:39:38 crc kubenswrapper[5052]: I1203 11:39:38.391464 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"64c8b84a240fd182cefc93f400d13b9d3ff52a744deab5121afb6b516b12861b"} err="failed to get container status \"64c8b84a240fd182cefc93f400d13b9d3ff52a744deab5121afb6b516b12861b\": rpc error: code = NotFound desc = could not find container \"64c8b84a240fd182cefc93f400d13b9d3ff52a744deab5121afb6b516b12861b\": container with ID starting with 64c8b84a240fd182cefc93f400d13b9d3ff52a744deab5121afb6b516b12861b not found: ID does not exist" Dec 03 11:39:38 crc kubenswrapper[5052]: I1203 11:39:38.391496 5052 scope.go:117] "RemoveContainer" containerID="d9e8edf0a3614e6406ed081e41d90643bdb16b77548d6948dc47e3a4064b333e" Dec 03 11:39:38 crc kubenswrapper[5052]: E1203 11:39:38.391920 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d9e8edf0a3614e6406ed081e41d90643bdb16b77548d6948dc47e3a4064b333e\": container with ID starting with d9e8edf0a3614e6406ed081e41d90643bdb16b77548d6948dc47e3a4064b333e not found: ID does not exist" containerID="d9e8edf0a3614e6406ed081e41d90643bdb16b77548d6948dc47e3a4064b333e" Dec 03 11:39:38 crc kubenswrapper[5052]: I1203 11:39:38.391952 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d9e8edf0a3614e6406ed081e41d90643bdb16b77548d6948dc47e3a4064b333e"} err="failed to get container status \"d9e8edf0a3614e6406ed081e41d90643bdb16b77548d6948dc47e3a4064b333e\": rpc error: code = NotFound desc = could not find container \"d9e8edf0a3614e6406ed081e41d90643bdb16b77548d6948dc47e3a4064b333e\": container with ID starting with d9e8edf0a3614e6406ed081e41d90643bdb16b77548d6948dc47e3a4064b333e not found: ID does not exist" Dec 03 11:39:38 crc kubenswrapper[5052]: I1203 11:39:38.391976 5052 scope.go:117] "RemoveContainer" containerID="64c8b84a240fd182cefc93f400d13b9d3ff52a744deab5121afb6b516b12861b" Dec 03 11:39:38 crc kubenswrapper[5052]: I1203 11:39:38.392251 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"64c8b84a240fd182cefc93f400d13b9d3ff52a744deab5121afb6b516b12861b"} err="failed to get container status \"64c8b84a240fd182cefc93f400d13b9d3ff52a744deab5121afb6b516b12861b\": rpc error: code = NotFound desc = could not find container \"64c8b84a240fd182cefc93f400d13b9d3ff52a744deab5121afb6b516b12861b\": container with ID starting with 64c8b84a240fd182cefc93f400d13b9d3ff52a744deab5121afb6b516b12861b not found: ID does not exist" Dec 03 11:39:38 crc kubenswrapper[5052]: I1203 11:39:38.392279 5052 scope.go:117] "RemoveContainer" containerID="d9e8edf0a3614e6406ed081e41d90643bdb16b77548d6948dc47e3a4064b333e" Dec 03 11:39:38 crc kubenswrapper[5052]: I1203 11:39:38.392483 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d9e8edf0a3614e6406ed081e41d90643bdb16b77548d6948dc47e3a4064b333e"} err="failed to get container status \"d9e8edf0a3614e6406ed081e41d90643bdb16b77548d6948dc47e3a4064b333e\": rpc error: code = NotFound desc = could not find container \"d9e8edf0a3614e6406ed081e41d90643bdb16b77548d6948dc47e3a4064b333e\": container with ID starting with d9e8edf0a3614e6406ed081e41d90643bdb16b77548d6948dc47e3a4064b333e not found: ID does not exist" Dec 03 11:39:38 crc kubenswrapper[5052]: I1203 11:39:38.394347 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cinder-api-config-data" Dec 03 11:39:38 crc kubenswrapper[5052]: I1203 11:39:38.394370 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 11:39:38 crc kubenswrapper[5052]: I1203 11:39:38.394625 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-cinder-public-svc" Dec 03 11:39:38 crc kubenswrapper[5052]: I1203 11:39:38.394631 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-cinder-internal-svc" Dec 03 11:39:38 crc kubenswrapper[5052]: I1203 11:39:38.530381 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lzhb7\" (UniqueName: \"kubernetes.io/projected/ee46c3a4-5e21-4f6b-a944-8cf96d87506e-kube-api-access-lzhb7\") pod \"cinder-api-0\" (UID: \"ee46c3a4-5e21-4f6b-a944-8cf96d87506e\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:39:38 crc kubenswrapper[5052]: I1203 11:39:38.530439 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/ee46c3a4-5e21-4f6b-a944-8cf96d87506e-config-data-custom\") pod \"cinder-api-0\" (UID: \"ee46c3a4-5e21-4f6b-a944-8cf96d87506e\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:39:38 crc kubenswrapper[5052]: I1203 11:39:38.530463 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/ee46c3a4-5e21-4f6b-a944-8cf96d87506e-config-data\") pod \"cinder-api-0\" (UID: \"ee46c3a4-5e21-4f6b-a944-8cf96d87506e\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:39:38 crc kubenswrapper[5052]: I1203 11:39:38.530503 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ee46c3a4-5e21-4f6b-a944-8cf96d87506e-combined-ca-bundle\") pod \"cinder-api-0\" (UID: \"ee46c3a4-5e21-4f6b-a944-8cf96d87506e\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:39:38 crc kubenswrapper[5052]: I1203 11:39:38.530549 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/ee46c3a4-5e21-4f6b-a944-8cf96d87506e-internal-tls-certs\") pod \"cinder-api-0\" (UID: \"ee46c3a4-5e21-4f6b-a944-8cf96d87506e\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:39:38 crc kubenswrapper[5052]: I1203 11:39:38.530576 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/ee46c3a4-5e21-4f6b-a944-8cf96d87506e-public-tls-certs\") pod \"cinder-api-0\" (UID: \"ee46c3a4-5e21-4f6b-a944-8cf96d87506e\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:39:38 crc kubenswrapper[5052]: I1203 11:39:38.530649 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/ee46c3a4-5e21-4f6b-a944-8cf96d87506e-etc-machine-id\") pod \"cinder-api-0\" (UID: \"ee46c3a4-5e21-4f6b-a944-8cf96d87506e\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:39:38 crc kubenswrapper[5052]: I1203 11:39:38.530680 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/ee46c3a4-5e21-4f6b-a944-8cf96d87506e-logs\") pod \"cinder-api-0\" (UID: \"ee46c3a4-5e21-4f6b-a944-8cf96d87506e\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:39:38 crc kubenswrapper[5052]: I1203 11:39:38.530723 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/ee46c3a4-5e21-4f6b-a944-8cf96d87506e-scripts\") pod \"cinder-api-0\" (UID: \"ee46c3a4-5e21-4f6b-a944-8cf96d87506e\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:39:38 crc kubenswrapper[5052]: I1203 11:39:38.632066 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-lzhb7\" (UniqueName: \"kubernetes.io/projected/ee46c3a4-5e21-4f6b-a944-8cf96d87506e-kube-api-access-lzhb7\") pod \"cinder-api-0\" (UID: \"ee46c3a4-5e21-4f6b-a944-8cf96d87506e\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:39:38 crc kubenswrapper[5052]: I1203 11:39:38.632440 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/ee46c3a4-5e21-4f6b-a944-8cf96d87506e-config-data-custom\") pod \"cinder-api-0\" (UID: \"ee46c3a4-5e21-4f6b-a944-8cf96d87506e\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:39:38 crc kubenswrapper[5052]: I1203 11:39:38.632465 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/ee46c3a4-5e21-4f6b-a944-8cf96d87506e-config-data\") pod \"cinder-api-0\" (UID: \"ee46c3a4-5e21-4f6b-a944-8cf96d87506e\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:39:38 crc kubenswrapper[5052]: I1203 11:39:38.632526 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ee46c3a4-5e21-4f6b-a944-8cf96d87506e-combined-ca-bundle\") pod \"cinder-api-0\" (UID: \"ee46c3a4-5e21-4f6b-a944-8cf96d87506e\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:39:38 crc kubenswrapper[5052]: I1203 11:39:38.632561 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/ee46c3a4-5e21-4f6b-a944-8cf96d87506e-internal-tls-certs\") pod \"cinder-api-0\" (UID: \"ee46c3a4-5e21-4f6b-a944-8cf96d87506e\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:39:38 crc kubenswrapper[5052]: I1203 11:39:38.632584 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/ee46c3a4-5e21-4f6b-a944-8cf96d87506e-public-tls-certs\") pod \"cinder-api-0\" (UID: \"ee46c3a4-5e21-4f6b-a944-8cf96d87506e\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:39:38 crc kubenswrapper[5052]: I1203 11:39:38.632642 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/ee46c3a4-5e21-4f6b-a944-8cf96d87506e-etc-machine-id\") pod \"cinder-api-0\" (UID: \"ee46c3a4-5e21-4f6b-a944-8cf96d87506e\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:39:38 crc kubenswrapper[5052]: I1203 11:39:38.632673 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/ee46c3a4-5e21-4f6b-a944-8cf96d87506e-logs\") pod \"cinder-api-0\" (UID: \"ee46c3a4-5e21-4f6b-a944-8cf96d87506e\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:39:38 crc kubenswrapper[5052]: I1203 11:39:38.632716 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/ee46c3a4-5e21-4f6b-a944-8cf96d87506e-scripts\") pod \"cinder-api-0\" (UID: \"ee46c3a4-5e21-4f6b-a944-8cf96d87506e\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:39:38 crc kubenswrapper[5052]: I1203 11:39:38.633313 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/ee46c3a4-5e21-4f6b-a944-8cf96d87506e-etc-machine-id\") pod \"cinder-api-0\" (UID: \"ee46c3a4-5e21-4f6b-a944-8cf96d87506e\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:39:38 crc kubenswrapper[5052]: I1203 11:39:38.633987 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/ee46c3a4-5e21-4f6b-a944-8cf96d87506e-logs\") pod \"cinder-api-0\" (UID: \"ee46c3a4-5e21-4f6b-a944-8cf96d87506e\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:39:38 crc kubenswrapper[5052]: I1203 11:39:38.645705 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/ee46c3a4-5e21-4f6b-a944-8cf96d87506e-public-tls-certs\") pod \"cinder-api-0\" (UID: \"ee46c3a4-5e21-4f6b-a944-8cf96d87506e\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:39:38 crc kubenswrapper[5052]: I1203 11:39:38.645862 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/ee46c3a4-5e21-4f6b-a944-8cf96d87506e-config-data-custom\") pod \"cinder-api-0\" (UID: \"ee46c3a4-5e21-4f6b-a944-8cf96d87506e\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:39:38 crc kubenswrapper[5052]: I1203 11:39:38.648492 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ee46c3a4-5e21-4f6b-a944-8cf96d87506e-combined-ca-bundle\") pod \"cinder-api-0\" (UID: \"ee46c3a4-5e21-4f6b-a944-8cf96d87506e\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:39:38 crc kubenswrapper[5052]: I1203 11:39:38.649660 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/ee46c3a4-5e21-4f6b-a944-8cf96d87506e-config-data\") pod \"cinder-api-0\" (UID: \"ee46c3a4-5e21-4f6b-a944-8cf96d87506e\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:39:38 crc kubenswrapper[5052]: I1203 11:39:38.652285 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/ee46c3a4-5e21-4f6b-a944-8cf96d87506e-internal-tls-certs\") pod \"cinder-api-0\" (UID: \"ee46c3a4-5e21-4f6b-a944-8cf96d87506e\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:39:38 crc kubenswrapper[5052]: I1203 11:39:38.659626 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-lzhb7\" (UniqueName: \"kubernetes.io/projected/ee46c3a4-5e21-4f6b-a944-8cf96d87506e-kube-api-access-lzhb7\") pod \"cinder-api-0\" (UID: \"ee46c3a4-5e21-4f6b-a944-8cf96d87506e\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:39:38 crc kubenswrapper[5052]: I1203 11:39:38.659700 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/ee46c3a4-5e21-4f6b-a944-8cf96d87506e-scripts\") pod \"cinder-api-0\" (UID: \"ee46c3a4-5e21-4f6b-a944-8cf96d87506e\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:39:38 crc kubenswrapper[5052]: I1203 11:39:38.710324 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:39:38 crc kubenswrapper[5052]: I1203 11:39:38.925821 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-api-66f4bb9676-j6vzc"] Dec 03 11:39:39 crc kubenswrapper[5052]: I1203 11:39:39.212963 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 11:39:39 crc kubenswrapper[5052]: W1203 11:39:39.214911 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podee46c3a4_5e21_4f6b_a944_8cf96d87506e.slice/crio-a2f0d9ea0593c2ee959146b8662e9a0356d2ce9d308b5519e59c62c7aa9da179 WatchSource:0}: Error finding container a2f0d9ea0593c2ee959146b8662e9a0356d2ce9d308b5519e59c62c7aa9da179: Status 404 returned error can't find the container with id a2f0d9ea0593c2ee959146b8662e9a0356d2ce9d308b5519e59c62c7aa9da179 Dec 03 11:39:39 crc kubenswrapper[5052]: I1203 11:39:39.338992 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"ee46c3a4-5e21-4f6b-a944-8cf96d87506e","Type":"ContainerStarted","Data":"a2f0d9ea0593c2ee959146b8662e9a0356d2ce9d308b5519e59c62c7aa9da179"} Dec 03 11:39:39 crc kubenswrapper[5052]: I1203 11:39:39.342022 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-66f4bb9676-j6vzc" event={"ID":"fe69e54e-ae29-49e2-8a21-0ff81cc0100f","Type":"ContainerStarted","Data":"c4c73ffbd6756af9b104cfcef9220c7341172a6c5dadc45edcc42921bbd74050"} Dec 03 11:39:39 crc kubenswrapper[5052]: I1203 11:39:39.342069 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-66f4bb9676-j6vzc" event={"ID":"fe69e54e-ae29-49e2-8a21-0ff81cc0100f","Type":"ContainerStarted","Data":"076608a93a4dad7d699f90f3532f41e2ef1cae56396664204466a92cea8df0e7"} Dec 03 11:39:39 crc kubenswrapper[5052]: I1203 11:39:39.707358 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e72e68d4-8dd6-488b-ad4f-bf276c8eb066" path="/var/lib/kubelet/pods/e72e68d4-8dd6-488b-ad4f-bf276c8eb066/volumes" Dec 03 11:39:40 crc kubenswrapper[5052]: I1203 11:39:40.351433 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"ee46c3a4-5e21-4f6b-a944-8cf96d87506e","Type":"ContainerStarted","Data":"f924c6cd365dded654c74f9f506f8cb2317b25d77d3b15ae01a497a3800a80db"} Dec 03 11:39:40 crc kubenswrapper[5052]: I1203 11:39:40.353544 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-66f4bb9676-j6vzc" event={"ID":"fe69e54e-ae29-49e2-8a21-0ff81cc0100f","Type":"ContainerStarted","Data":"4c92b1739b5436291f4de41014a637fd37a00db71202d30b8ea1ae9006123ea8"} Dec 03 11:39:40 crc kubenswrapper[5052]: I1203 11:39:40.354066 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/barbican-api-66f4bb9676-j6vzc" Dec 03 11:39:40 crc kubenswrapper[5052]: I1203 11:39:40.354093 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/barbican-api-66f4bb9676-j6vzc" Dec 03 11:39:41 crc kubenswrapper[5052]: I1203 11:39:41.386533 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"ee46c3a4-5e21-4f6b-a944-8cf96d87506e","Type":"ContainerStarted","Data":"a5e3cd0978d13d8f638fcff947df03b89089415c5babbbe442713809453149ae"} Dec 03 11:39:41 crc kubenswrapper[5052]: I1203 11:39:41.408911 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/cinder-api-0" podStartSLOduration=3.408890734 podStartE2EDuration="3.408890734s" podCreationTimestamp="2025-12-03 11:39:38 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:39:41.404771295 +0000 UTC m=+1171.993013079" watchObservedRunningTime="2025-12-03 11:39:41.408890734 +0000 UTC m=+1171.997132498" Dec 03 11:39:41 crc kubenswrapper[5052]: I1203 11:39:41.416902 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/barbican-api-66f4bb9676-j6vzc" podStartSLOduration=4.416882715 podStartE2EDuration="4.416882715s" podCreationTimestamp="2025-12-03 11:39:37 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:39:40.381960959 +0000 UTC m=+1170.970202723" watchObservedRunningTime="2025-12-03 11:39:41.416882715 +0000 UTC m=+1172.005124479" Dec 03 11:39:41 crc kubenswrapper[5052]: I1203 11:39:41.604992 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/neutron-58dc59ff57-9dsf9" Dec 03 11:39:42 crc kubenswrapper[5052]: I1203 11:39:42.029781 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/placement-699ddb8646-rfm4p" Dec 03 11:39:42 crc kubenswrapper[5052]: I1203 11:39:42.065987 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/placement-699ddb8646-rfm4p" Dec 03 11:39:42 crc kubenswrapper[5052]: I1203 11:39:42.394997 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:39:43 crc kubenswrapper[5052]: I1203 11:39:43.049400 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:39:43 crc kubenswrapper[5052]: I1203 11:39:43.086288 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 11:39:43 crc kubenswrapper[5052]: I1203 11:39:43.294955 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/barbican-api-65968d9848-mntb6" Dec 03 11:39:43 crc kubenswrapper[5052]: I1203 11:39:43.368064 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/barbican-api-65968d9848-mntb6" Dec 03 11:39:43 crc kubenswrapper[5052]: I1203 11:39:43.411130 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/cinder-scheduler-0" podUID="a8040985-8ab8-47ee-b1e8-ad692191fe4d" containerName="cinder-scheduler" containerID="cri-o://6920a886819e6444771d9e24d02bda1230c57f492aad5dfb84f1dd9fab7de559" gracePeriod=30 Dec 03 11:39:43 crc kubenswrapper[5052]: I1203 11:39:43.411902 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/cinder-scheduler-0" podUID="a8040985-8ab8-47ee-b1e8-ad692191fe4d" containerName="probe" containerID="cri-o://27a49abcaf4c4c47a542806b2d85ebfba88c348a219c09be22e5c5a52566b889" gracePeriod=30 Dec 03 11:39:44 crc kubenswrapper[5052]: I1203 11:39:44.421374 5052 generic.go:334] "Generic (PLEG): container finished" podID="a8040985-8ab8-47ee-b1e8-ad692191fe4d" containerID="27a49abcaf4c4c47a542806b2d85ebfba88c348a219c09be22e5c5a52566b889" exitCode=0 Dec 03 11:39:44 crc kubenswrapper[5052]: I1203 11:39:44.421414 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"a8040985-8ab8-47ee-b1e8-ad692191fe4d","Type":"ContainerDied","Data":"27a49abcaf4c4c47a542806b2d85ebfba88c348a219c09be22e5c5a52566b889"} Dec 03 11:39:44 crc kubenswrapper[5052]: I1203 11:39:44.506350 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/keystone-d6bbfcfb4-xdw2x" Dec 03 11:39:44 crc kubenswrapper[5052]: I1203 11:39:44.619348 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/neutron-75475cc596-bfbzz" Dec 03 11:39:44 crc kubenswrapper[5052]: I1203 11:39:44.679419 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/neutron-58dc59ff57-9dsf9"] Dec 03 11:39:44 crc kubenswrapper[5052]: I1203 11:39:44.679655 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/neutron-58dc59ff57-9dsf9" podUID="577c99af-c225-41f3-9d97-e96326a8c1c4" containerName="neutron-api" containerID="cri-o://f198dbe5aa735908998d7dc99e4c1016ec4ce99693fe8cdc9b5c3d6597777a05" gracePeriod=30 Dec 03 11:39:44 crc kubenswrapper[5052]: I1203 11:39:44.679834 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/neutron-58dc59ff57-9dsf9" podUID="577c99af-c225-41f3-9d97-e96326a8c1c4" containerName="neutron-httpd" containerID="cri-o://a55c9e545d9db9e0eaff80f2fad72700c0a38b5a3dcd4bb8357e991a4e2f25c1" gracePeriod=30 Dec 03 11:39:45 crc kubenswrapper[5052]: I1203 11:39:45.435191 5052 generic.go:334] "Generic (PLEG): container finished" podID="577c99af-c225-41f3-9d97-e96326a8c1c4" containerID="a55c9e545d9db9e0eaff80f2fad72700c0a38b5a3dcd4bb8357e991a4e2f25c1" exitCode=0 Dec 03 11:39:45 crc kubenswrapper[5052]: I1203 11:39:45.435251 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-58dc59ff57-9dsf9" event={"ID":"577c99af-c225-41f3-9d97-e96326a8c1c4","Type":"ContainerDied","Data":"a55c9e545d9db9e0eaff80f2fad72700c0a38b5a3dcd4bb8357e991a4e2f25c1"} Dec 03 11:39:46 crc kubenswrapper[5052]: I1203 11:39:46.542967 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/openstackclient"] Dec 03 11:39:46 crc kubenswrapper[5052]: I1203 11:39:46.544153 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstackclient" Dec 03 11:39:46 crc kubenswrapper[5052]: I1203 11:39:46.546275 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"openstack-config-secret" Dec 03 11:39:46 crc kubenswrapper[5052]: I1203 11:39:46.546603 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"openstackclient-openstackclient-dockercfg-mpsbv" Dec 03 11:39:46 crc kubenswrapper[5052]: I1203 11:39:46.546841 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"openstack-config" Dec 03 11:39:46 crc kubenswrapper[5052]: I1203 11:39:46.552576 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/openstackclient"] Dec 03 11:39:46 crc kubenswrapper[5052]: I1203 11:39:46.634931 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openstack-config-secret\" (UniqueName: \"kubernetes.io/secret/25ad6214-e650-4b24-95b3-8459a622a28c-openstack-config-secret\") pod \"openstackclient\" (UID: \"25ad6214-e650-4b24-95b3-8459a622a28c\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 11:39:46 crc kubenswrapper[5052]: I1203 11:39:46.635323 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/25ad6214-e650-4b24-95b3-8459a622a28c-combined-ca-bundle\") pod \"openstackclient\" (UID: \"25ad6214-e650-4b24-95b3-8459a622a28c\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 11:39:46 crc kubenswrapper[5052]: I1203 11:39:46.635484 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openstack-config\" (UniqueName: \"kubernetes.io/configmap/25ad6214-e650-4b24-95b3-8459a622a28c-openstack-config\") pod \"openstackclient\" (UID: \"25ad6214-e650-4b24-95b3-8459a622a28c\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 11:39:46 crc kubenswrapper[5052]: I1203 11:39:46.635552 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f8m2n\" (UniqueName: \"kubernetes.io/projected/25ad6214-e650-4b24-95b3-8459a622a28c-kube-api-access-f8m2n\") pod \"openstackclient\" (UID: \"25ad6214-e650-4b24-95b3-8459a622a28c\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 11:39:46 crc kubenswrapper[5052]: I1203 11:39:46.736264 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"openstack-config\" (UniqueName: \"kubernetes.io/configmap/25ad6214-e650-4b24-95b3-8459a622a28c-openstack-config\") pod \"openstackclient\" (UID: \"25ad6214-e650-4b24-95b3-8459a622a28c\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 11:39:46 crc kubenswrapper[5052]: I1203 11:39:46.736322 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-f8m2n\" (UniqueName: \"kubernetes.io/projected/25ad6214-e650-4b24-95b3-8459a622a28c-kube-api-access-f8m2n\") pod \"openstackclient\" (UID: \"25ad6214-e650-4b24-95b3-8459a622a28c\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 11:39:46 crc kubenswrapper[5052]: I1203 11:39:46.736365 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"openstack-config-secret\" (UniqueName: \"kubernetes.io/secret/25ad6214-e650-4b24-95b3-8459a622a28c-openstack-config-secret\") pod \"openstackclient\" (UID: \"25ad6214-e650-4b24-95b3-8459a622a28c\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 11:39:46 crc kubenswrapper[5052]: I1203 11:39:46.736436 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/25ad6214-e650-4b24-95b3-8459a622a28c-combined-ca-bundle\") pod \"openstackclient\" (UID: \"25ad6214-e650-4b24-95b3-8459a622a28c\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 11:39:46 crc kubenswrapper[5052]: I1203 11:39:46.737116 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"openstack-config\" (UniqueName: \"kubernetes.io/configmap/25ad6214-e650-4b24-95b3-8459a622a28c-openstack-config\") pod \"openstackclient\" (UID: \"25ad6214-e650-4b24-95b3-8459a622a28c\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 11:39:46 crc kubenswrapper[5052]: I1203 11:39:46.749267 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"openstack-config-secret\" (UniqueName: \"kubernetes.io/secret/25ad6214-e650-4b24-95b3-8459a622a28c-openstack-config-secret\") pod \"openstackclient\" (UID: \"25ad6214-e650-4b24-95b3-8459a622a28c\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 11:39:46 crc kubenswrapper[5052]: I1203 11:39:46.749426 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/25ad6214-e650-4b24-95b3-8459a622a28c-combined-ca-bundle\") pod \"openstackclient\" (UID: \"25ad6214-e650-4b24-95b3-8459a622a28c\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 11:39:46 crc kubenswrapper[5052]: I1203 11:39:46.759841 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-f8m2n\" (UniqueName: \"kubernetes.io/projected/25ad6214-e650-4b24-95b3-8459a622a28c-kube-api-access-f8m2n\") pod \"openstackclient\" (UID: \"25ad6214-e650-4b24-95b3-8459a622a28c\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 11:39:46 crc kubenswrapper[5052]: I1203 11:39:46.861374 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/openstackclient"] Dec 03 11:39:46 crc kubenswrapper[5052]: I1203 11:39:46.862195 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstackclient" Dec 03 11:39:46 crc kubenswrapper[5052]: I1203 11:39:46.865280 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/openstackclient"] Dec 03 11:39:46 crc kubenswrapper[5052]: I1203 11:39:46.879883 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/openstackclient"] Dec 03 11:39:46 crc kubenswrapper[5052]: I1203 11:39:46.881078 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstackclient" Dec 03 11:39:46 crc kubenswrapper[5052]: I1203 11:39:46.893453 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/openstackclient"] Dec 03 11:39:47 crc kubenswrapper[5052]: I1203 11:39:47.046300 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9xmxf\" (UniqueName: \"kubernetes.io/projected/722e5457-ae47-40b8-816f-8eab5b7a4cb0-kube-api-access-9xmxf\") pod \"openstackclient\" (UID: \"722e5457-ae47-40b8-816f-8eab5b7a4cb0\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 11:39:47 crc kubenswrapper[5052]: I1203 11:39:47.046366 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openstack-config\" (UniqueName: \"kubernetes.io/configmap/722e5457-ae47-40b8-816f-8eab5b7a4cb0-openstack-config\") pod \"openstackclient\" (UID: \"722e5457-ae47-40b8-816f-8eab5b7a4cb0\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 11:39:47 crc kubenswrapper[5052]: I1203 11:39:47.046566 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/722e5457-ae47-40b8-816f-8eab5b7a4cb0-combined-ca-bundle\") pod \"openstackclient\" (UID: \"722e5457-ae47-40b8-816f-8eab5b7a4cb0\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 11:39:47 crc kubenswrapper[5052]: I1203 11:39:47.046699 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openstack-config-secret\" (UniqueName: \"kubernetes.io/secret/722e5457-ae47-40b8-816f-8eab5b7a4cb0-openstack-config-secret\") pod \"openstackclient\" (UID: \"722e5457-ae47-40b8-816f-8eab5b7a4cb0\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 11:39:47 crc kubenswrapper[5052]: E1203 11:39:47.048352 5052 log.go:32] "RunPodSandbox from runtime service failed" err=< Dec 03 11:39:47 crc kubenswrapper[5052]: rpc error: code = Unknown desc = failed to create pod network sandbox k8s_openstackclient_openstack-kuttl-tests_25ad6214-e650-4b24-95b3-8459a622a28c_0(fc93094e2863fc9cf9d53c7e5c98c2f200a58684995fa9e0cd90bf7ae99459ea): error adding pod openstack-kuttl-tests_openstackclient to CNI network "multus-cni-network": plugin type="multus-shim" name="multus-cni-network" failed (add): CmdAdd (shim): CNI request failed with status 400: 'ContainerID:"fc93094e2863fc9cf9d53c7e5c98c2f200a58684995fa9e0cd90bf7ae99459ea" Netns:"/var/run/netns/2a25a433-b029-4596-89b9-2dac7301125b" IfName:"eth0" Args:"IgnoreUnknown=1;K8S_POD_NAMESPACE=openstack-kuttl-tests;K8S_POD_NAME=openstackclient;K8S_POD_INFRA_CONTAINER_ID=fc93094e2863fc9cf9d53c7e5c98c2f200a58684995fa9e0cd90bf7ae99459ea;K8S_POD_UID=25ad6214-e650-4b24-95b3-8459a622a28c" Path:"" ERRORED: error configuring pod [openstack-kuttl-tests/openstackclient] networking: Multus: [openstack-kuttl-tests/openstackclient/25ad6214-e650-4b24-95b3-8459a622a28c]: expected pod UID "25ad6214-e650-4b24-95b3-8459a622a28c" but got "722e5457-ae47-40b8-816f-8eab5b7a4cb0" from Kube API Dec 03 11:39:47 crc kubenswrapper[5052]: ': StdinData: {"binDir":"/var/lib/cni/bin","clusterNetwork":"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf","cniVersion":"0.3.1","daemonSocketDir":"/run/multus/socket","globalNamespaces":"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv","logLevel":"verbose","logToStderr":true,"name":"multus-cni-network","namespaceIsolation":true,"type":"multus-shim"} Dec 03 11:39:47 crc kubenswrapper[5052]: > Dec 03 11:39:47 crc kubenswrapper[5052]: E1203 11:39:47.048441 5052 kuberuntime_sandbox.go:72] "Failed to create sandbox for pod" err=< Dec 03 11:39:47 crc kubenswrapper[5052]: rpc error: code = Unknown desc = failed to create pod network sandbox k8s_openstackclient_openstack-kuttl-tests_25ad6214-e650-4b24-95b3-8459a622a28c_0(fc93094e2863fc9cf9d53c7e5c98c2f200a58684995fa9e0cd90bf7ae99459ea): error adding pod openstack-kuttl-tests_openstackclient to CNI network "multus-cni-network": plugin type="multus-shim" name="multus-cni-network" failed (add): CmdAdd (shim): CNI request failed with status 400: 'ContainerID:"fc93094e2863fc9cf9d53c7e5c98c2f200a58684995fa9e0cd90bf7ae99459ea" Netns:"/var/run/netns/2a25a433-b029-4596-89b9-2dac7301125b" IfName:"eth0" Args:"IgnoreUnknown=1;K8S_POD_NAMESPACE=openstack-kuttl-tests;K8S_POD_NAME=openstackclient;K8S_POD_INFRA_CONTAINER_ID=fc93094e2863fc9cf9d53c7e5c98c2f200a58684995fa9e0cd90bf7ae99459ea;K8S_POD_UID=25ad6214-e650-4b24-95b3-8459a622a28c" Path:"" ERRORED: error configuring pod [openstack-kuttl-tests/openstackclient] networking: Multus: [openstack-kuttl-tests/openstackclient/25ad6214-e650-4b24-95b3-8459a622a28c]: expected pod UID "25ad6214-e650-4b24-95b3-8459a622a28c" but got "722e5457-ae47-40b8-816f-8eab5b7a4cb0" from Kube API Dec 03 11:39:47 crc kubenswrapper[5052]: ': StdinData: {"binDir":"/var/lib/cni/bin","clusterNetwork":"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf","cniVersion":"0.3.1","daemonSocketDir":"/run/multus/socket","globalNamespaces":"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv","logLevel":"verbose","logToStderr":true,"name":"multus-cni-network","namespaceIsolation":true,"type":"multus-shim"} Dec 03 11:39:47 crc kubenswrapper[5052]: > pod="openstack-kuttl-tests/openstackclient" Dec 03 11:39:47 crc kubenswrapper[5052]: I1203 11:39:47.153637 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/722e5457-ae47-40b8-816f-8eab5b7a4cb0-combined-ca-bundle\") pod \"openstackclient\" (UID: \"722e5457-ae47-40b8-816f-8eab5b7a4cb0\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 11:39:47 crc kubenswrapper[5052]: I1203 11:39:47.153778 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"openstack-config-secret\" (UniqueName: \"kubernetes.io/secret/722e5457-ae47-40b8-816f-8eab5b7a4cb0-openstack-config-secret\") pod \"openstackclient\" (UID: \"722e5457-ae47-40b8-816f-8eab5b7a4cb0\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 11:39:47 crc kubenswrapper[5052]: I1203 11:39:47.153839 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-9xmxf\" (UniqueName: \"kubernetes.io/projected/722e5457-ae47-40b8-816f-8eab5b7a4cb0-kube-api-access-9xmxf\") pod \"openstackclient\" (UID: \"722e5457-ae47-40b8-816f-8eab5b7a4cb0\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 11:39:47 crc kubenswrapper[5052]: I1203 11:39:47.153874 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"openstack-config\" (UniqueName: \"kubernetes.io/configmap/722e5457-ae47-40b8-816f-8eab5b7a4cb0-openstack-config\") pod \"openstackclient\" (UID: \"722e5457-ae47-40b8-816f-8eab5b7a4cb0\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 11:39:47 crc kubenswrapper[5052]: I1203 11:39:47.154776 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"openstack-config\" (UniqueName: \"kubernetes.io/configmap/722e5457-ae47-40b8-816f-8eab5b7a4cb0-openstack-config\") pod \"openstackclient\" (UID: \"722e5457-ae47-40b8-816f-8eab5b7a4cb0\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 11:39:47 crc kubenswrapper[5052]: I1203 11:39:47.159626 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/722e5457-ae47-40b8-816f-8eab5b7a4cb0-combined-ca-bundle\") pod \"openstackclient\" (UID: \"722e5457-ae47-40b8-816f-8eab5b7a4cb0\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 11:39:47 crc kubenswrapper[5052]: I1203 11:39:47.175050 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"openstack-config-secret\" (UniqueName: \"kubernetes.io/secret/722e5457-ae47-40b8-816f-8eab5b7a4cb0-openstack-config-secret\") pod \"openstackclient\" (UID: \"722e5457-ae47-40b8-816f-8eab5b7a4cb0\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 11:39:47 crc kubenswrapper[5052]: I1203 11:39:47.179231 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-9xmxf\" (UniqueName: \"kubernetes.io/projected/722e5457-ae47-40b8-816f-8eab5b7a4cb0-kube-api-access-9xmxf\") pod \"openstackclient\" (UID: \"722e5457-ae47-40b8-816f-8eab5b7a4cb0\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 11:39:47 crc kubenswrapper[5052]: I1203 11:39:47.231136 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:39:47 crc kubenswrapper[5052]: I1203 11:39:47.258298 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/a8040985-8ab8-47ee-b1e8-ad692191fe4d-etc-machine-id\") pod \"a8040985-8ab8-47ee-b1e8-ad692191fe4d\" (UID: \"a8040985-8ab8-47ee-b1e8-ad692191fe4d\") " Dec 03 11:39:47 crc kubenswrapper[5052]: I1203 11:39:47.258343 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a8040985-8ab8-47ee-b1e8-ad692191fe4d-config-data\") pod \"a8040985-8ab8-47ee-b1e8-ad692191fe4d\" (UID: \"a8040985-8ab8-47ee-b1e8-ad692191fe4d\") " Dec 03 11:39:47 crc kubenswrapper[5052]: I1203 11:39:47.258378 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a8040985-8ab8-47ee-b1e8-ad692191fe4d-combined-ca-bundle\") pod \"a8040985-8ab8-47ee-b1e8-ad692191fe4d\" (UID: \"a8040985-8ab8-47ee-b1e8-ad692191fe4d\") " Dec 03 11:39:47 crc kubenswrapper[5052]: I1203 11:39:47.258409 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tk5lf\" (UniqueName: \"kubernetes.io/projected/a8040985-8ab8-47ee-b1e8-ad692191fe4d-kube-api-access-tk5lf\") pod \"a8040985-8ab8-47ee-b1e8-ad692191fe4d\" (UID: \"a8040985-8ab8-47ee-b1e8-ad692191fe4d\") " Dec 03 11:39:47 crc kubenswrapper[5052]: I1203 11:39:47.258430 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/a8040985-8ab8-47ee-b1e8-ad692191fe4d-scripts\") pod \"a8040985-8ab8-47ee-b1e8-ad692191fe4d\" (UID: \"a8040985-8ab8-47ee-b1e8-ad692191fe4d\") " Dec 03 11:39:47 crc kubenswrapper[5052]: I1203 11:39:47.258490 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/a8040985-8ab8-47ee-b1e8-ad692191fe4d-config-data-custom\") pod \"a8040985-8ab8-47ee-b1e8-ad692191fe4d\" (UID: \"a8040985-8ab8-47ee-b1e8-ad692191fe4d\") " Dec 03 11:39:47 crc kubenswrapper[5052]: I1203 11:39:47.262751 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/a8040985-8ab8-47ee-b1e8-ad692191fe4d-etc-machine-id" (OuterVolumeSpecName: "etc-machine-id") pod "a8040985-8ab8-47ee-b1e8-ad692191fe4d" (UID: "a8040985-8ab8-47ee-b1e8-ad692191fe4d"). InnerVolumeSpecName "etc-machine-id". PluginName "kubernetes.io/host-path", VolumeGidValue "" Dec 03 11:39:47 crc kubenswrapper[5052]: I1203 11:39:47.276142 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a8040985-8ab8-47ee-b1e8-ad692191fe4d-kube-api-access-tk5lf" (OuterVolumeSpecName: "kube-api-access-tk5lf") pod "a8040985-8ab8-47ee-b1e8-ad692191fe4d" (UID: "a8040985-8ab8-47ee-b1e8-ad692191fe4d"). InnerVolumeSpecName "kube-api-access-tk5lf". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:39:47 crc kubenswrapper[5052]: I1203 11:39:47.276808 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a8040985-8ab8-47ee-b1e8-ad692191fe4d-scripts" (OuterVolumeSpecName: "scripts") pod "a8040985-8ab8-47ee-b1e8-ad692191fe4d" (UID: "a8040985-8ab8-47ee-b1e8-ad692191fe4d"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:39:47 crc kubenswrapper[5052]: I1203 11:39:47.277174 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a8040985-8ab8-47ee-b1e8-ad692191fe4d-config-data-custom" (OuterVolumeSpecName: "config-data-custom") pod "a8040985-8ab8-47ee-b1e8-ad692191fe4d" (UID: "a8040985-8ab8-47ee-b1e8-ad692191fe4d"). InnerVolumeSpecName "config-data-custom". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:39:47 crc kubenswrapper[5052]: I1203 11:39:47.291955 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstackclient" Dec 03 11:39:47 crc kubenswrapper[5052]: I1203 11:39:47.330660 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a8040985-8ab8-47ee-b1e8-ad692191fe4d-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "a8040985-8ab8-47ee-b1e8-ad692191fe4d" (UID: "a8040985-8ab8-47ee-b1e8-ad692191fe4d"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:39:47 crc kubenswrapper[5052]: I1203 11:39:47.359883 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/a8040985-8ab8-47ee-b1e8-ad692191fe4d-config-data-custom\") on node \"crc\" DevicePath \"\"" Dec 03 11:39:47 crc kubenswrapper[5052]: I1203 11:39:47.359917 5052 reconciler_common.go:293] "Volume detached for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/a8040985-8ab8-47ee-b1e8-ad692191fe4d-etc-machine-id\") on node \"crc\" DevicePath \"\"" Dec 03 11:39:47 crc kubenswrapper[5052]: I1203 11:39:47.359926 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a8040985-8ab8-47ee-b1e8-ad692191fe4d-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:39:47 crc kubenswrapper[5052]: I1203 11:39:47.359935 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-tk5lf\" (UniqueName: \"kubernetes.io/projected/a8040985-8ab8-47ee-b1e8-ad692191fe4d-kube-api-access-tk5lf\") on node \"crc\" DevicePath \"\"" Dec 03 11:39:47 crc kubenswrapper[5052]: I1203 11:39:47.359946 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/a8040985-8ab8-47ee-b1e8-ad692191fe4d-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:39:47 crc kubenswrapper[5052]: I1203 11:39:47.418119 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a8040985-8ab8-47ee-b1e8-ad692191fe4d-config-data" (OuterVolumeSpecName: "config-data") pod "a8040985-8ab8-47ee-b1e8-ad692191fe4d" (UID: "a8040985-8ab8-47ee-b1e8-ad692191fe4d"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:39:47 crc kubenswrapper[5052]: I1203 11:39:47.459829 5052 generic.go:334] "Generic (PLEG): container finished" podID="a8040985-8ab8-47ee-b1e8-ad692191fe4d" containerID="6920a886819e6444771d9e24d02bda1230c57f492aad5dfb84f1dd9fab7de559" exitCode=0 Dec 03 11:39:47 crc kubenswrapper[5052]: I1203 11:39:47.459904 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstackclient" Dec 03 11:39:47 crc kubenswrapper[5052]: I1203 11:39:47.460579 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:39:47 crc kubenswrapper[5052]: I1203 11:39:47.460602 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"a8040985-8ab8-47ee-b1e8-ad692191fe4d","Type":"ContainerDied","Data":"6920a886819e6444771d9e24d02bda1230c57f492aad5dfb84f1dd9fab7de559"} Dec 03 11:39:47 crc kubenswrapper[5052]: I1203 11:39:47.460636 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"a8040985-8ab8-47ee-b1e8-ad692191fe4d","Type":"ContainerDied","Data":"c935cfb9dda4b09af01053f4f22591764d5ec573c22e1071aea00996d9103899"} Dec 03 11:39:47 crc kubenswrapper[5052]: I1203 11:39:47.460661 5052 scope.go:117] "RemoveContainer" containerID="27a49abcaf4c4c47a542806b2d85ebfba88c348a219c09be22e5c5a52566b889" Dec 03 11:39:47 crc kubenswrapper[5052]: I1203 11:39:47.464428 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a8040985-8ab8-47ee-b1e8-ad692191fe4d-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:39:47 crc kubenswrapper[5052]: I1203 11:39:47.470050 5052 status_manager.go:861] "Pod was deleted and then recreated, skipping status update" pod="openstack-kuttl-tests/openstackclient" oldPodUID="25ad6214-e650-4b24-95b3-8459a622a28c" podUID="722e5457-ae47-40b8-816f-8eab5b7a4cb0" Dec 03 11:39:47 crc kubenswrapper[5052]: I1203 11:39:47.476658 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstackclient" Dec 03 11:39:47 crc kubenswrapper[5052]: I1203 11:39:47.520576 5052 scope.go:117] "RemoveContainer" containerID="6920a886819e6444771d9e24d02bda1230c57f492aad5dfb84f1dd9fab7de559" Dec 03 11:39:47 crc kubenswrapper[5052]: I1203 11:39:47.521052 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 11:39:47 crc kubenswrapper[5052]: I1203 11:39:47.553960 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 11:39:47 crc kubenswrapper[5052]: I1203 11:39:47.565305 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f8m2n\" (UniqueName: \"kubernetes.io/projected/25ad6214-e650-4b24-95b3-8459a622a28c-kube-api-access-f8m2n\") pod \"25ad6214-e650-4b24-95b3-8459a622a28c\" (UID: \"25ad6214-e650-4b24-95b3-8459a622a28c\") " Dec 03 11:39:47 crc kubenswrapper[5052]: I1203 11:39:47.565554 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/25ad6214-e650-4b24-95b3-8459a622a28c-combined-ca-bundle\") pod \"25ad6214-e650-4b24-95b3-8459a622a28c\" (UID: \"25ad6214-e650-4b24-95b3-8459a622a28c\") " Dec 03 11:39:47 crc kubenswrapper[5052]: I1203 11:39:47.568100 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"openstack-config-secret\" (UniqueName: \"kubernetes.io/secret/25ad6214-e650-4b24-95b3-8459a622a28c-openstack-config-secret\") pod \"25ad6214-e650-4b24-95b3-8459a622a28c\" (UID: \"25ad6214-e650-4b24-95b3-8459a622a28c\") " Dec 03 11:39:47 crc kubenswrapper[5052]: I1203 11:39:47.568132 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"openstack-config\" (UniqueName: \"kubernetes.io/configmap/25ad6214-e650-4b24-95b3-8459a622a28c-openstack-config\") pod \"25ad6214-e650-4b24-95b3-8459a622a28c\" (UID: \"25ad6214-e650-4b24-95b3-8459a622a28c\") " Dec 03 11:39:47 crc kubenswrapper[5052]: I1203 11:39:47.569287 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/25ad6214-e650-4b24-95b3-8459a622a28c-openstack-config" (OuterVolumeSpecName: "openstack-config") pod "25ad6214-e650-4b24-95b3-8459a622a28c" (UID: "25ad6214-e650-4b24-95b3-8459a622a28c"). InnerVolumeSpecName "openstack-config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:39:47 crc kubenswrapper[5052]: I1203 11:39:47.569397 5052 scope.go:117] "RemoveContainer" containerID="27a49abcaf4c4c47a542806b2d85ebfba88c348a219c09be22e5c5a52566b889" Dec 03 11:39:47 crc kubenswrapper[5052]: E1203 11:39:47.570691 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"27a49abcaf4c4c47a542806b2d85ebfba88c348a219c09be22e5c5a52566b889\": container with ID starting with 27a49abcaf4c4c47a542806b2d85ebfba88c348a219c09be22e5c5a52566b889 not found: ID does not exist" containerID="27a49abcaf4c4c47a542806b2d85ebfba88c348a219c09be22e5c5a52566b889" Dec 03 11:39:47 crc kubenswrapper[5052]: I1203 11:39:47.570732 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"27a49abcaf4c4c47a542806b2d85ebfba88c348a219c09be22e5c5a52566b889"} err="failed to get container status \"27a49abcaf4c4c47a542806b2d85ebfba88c348a219c09be22e5c5a52566b889\": rpc error: code = NotFound desc = could not find container \"27a49abcaf4c4c47a542806b2d85ebfba88c348a219c09be22e5c5a52566b889\": container with ID starting with 27a49abcaf4c4c47a542806b2d85ebfba88c348a219c09be22e5c5a52566b889 not found: ID does not exist" Dec 03 11:39:47 crc kubenswrapper[5052]: I1203 11:39:47.570758 5052 scope.go:117] "RemoveContainer" containerID="6920a886819e6444771d9e24d02bda1230c57f492aad5dfb84f1dd9fab7de559" Dec 03 11:39:47 crc kubenswrapper[5052]: E1203 11:39:47.571917 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6920a886819e6444771d9e24d02bda1230c57f492aad5dfb84f1dd9fab7de559\": container with ID starting with 6920a886819e6444771d9e24d02bda1230c57f492aad5dfb84f1dd9fab7de559 not found: ID does not exist" containerID="6920a886819e6444771d9e24d02bda1230c57f492aad5dfb84f1dd9fab7de559" Dec 03 11:39:47 crc kubenswrapper[5052]: I1203 11:39:47.571941 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6920a886819e6444771d9e24d02bda1230c57f492aad5dfb84f1dd9fab7de559"} err="failed to get container status \"6920a886819e6444771d9e24d02bda1230c57f492aad5dfb84f1dd9fab7de559\": rpc error: code = NotFound desc = could not find container \"6920a886819e6444771d9e24d02bda1230c57f492aad5dfb84f1dd9fab7de559\": container with ID starting with 6920a886819e6444771d9e24d02bda1230c57f492aad5dfb84f1dd9fab7de559 not found: ID does not exist" Dec 03 11:39:47 crc kubenswrapper[5052]: I1203 11:39:47.572327 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/25ad6214-e650-4b24-95b3-8459a622a28c-openstack-config-secret" (OuterVolumeSpecName: "openstack-config-secret") pod "25ad6214-e650-4b24-95b3-8459a622a28c" (UID: "25ad6214-e650-4b24-95b3-8459a622a28c"). InnerVolumeSpecName "openstack-config-secret". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:39:47 crc kubenswrapper[5052]: I1203 11:39:47.572864 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/25ad6214-e650-4b24-95b3-8459a622a28c-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "25ad6214-e650-4b24-95b3-8459a622a28c" (UID: "25ad6214-e650-4b24-95b3-8459a622a28c"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:39:47 crc kubenswrapper[5052]: I1203 11:39:47.576711 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/25ad6214-e650-4b24-95b3-8459a622a28c-kube-api-access-f8m2n" (OuterVolumeSpecName: "kube-api-access-f8m2n") pod "25ad6214-e650-4b24-95b3-8459a622a28c" (UID: "25ad6214-e650-4b24-95b3-8459a622a28c"). InnerVolumeSpecName "kube-api-access-f8m2n". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:39:47 crc kubenswrapper[5052]: I1203 11:39:47.583857 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 11:39:47 crc kubenswrapper[5052]: E1203 11:39:47.584257 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="a8040985-8ab8-47ee-b1e8-ad692191fe4d" containerName="cinder-scheduler" Dec 03 11:39:47 crc kubenswrapper[5052]: I1203 11:39:47.584272 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="a8040985-8ab8-47ee-b1e8-ad692191fe4d" containerName="cinder-scheduler" Dec 03 11:39:47 crc kubenswrapper[5052]: E1203 11:39:47.584298 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="a8040985-8ab8-47ee-b1e8-ad692191fe4d" containerName="probe" Dec 03 11:39:47 crc kubenswrapper[5052]: I1203 11:39:47.584305 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="a8040985-8ab8-47ee-b1e8-ad692191fe4d" containerName="probe" Dec 03 11:39:47 crc kubenswrapper[5052]: I1203 11:39:47.584457 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="a8040985-8ab8-47ee-b1e8-ad692191fe4d" containerName="cinder-scheduler" Dec 03 11:39:47 crc kubenswrapper[5052]: I1203 11:39:47.584481 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="a8040985-8ab8-47ee-b1e8-ad692191fe4d" containerName="probe" Dec 03 11:39:47 crc kubenswrapper[5052]: I1203 11:39:47.585368 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:39:47 crc kubenswrapper[5052]: I1203 11:39:47.588652 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cinder-scheduler-config-data" Dec 03 11:39:47 crc kubenswrapper[5052]: W1203 11:39:47.596469 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod722e5457_ae47_40b8_816f_8eab5b7a4cb0.slice/crio-c546fad2c7df08ae53296fd9af23e0254a60b21cb613124162dce41ad0a92662 WatchSource:0}: Error finding container c546fad2c7df08ae53296fd9af23e0254a60b21cb613124162dce41ad0a92662: Status 404 returned error can't find the container with id c546fad2c7df08ae53296fd9af23e0254a60b21cb613124162dce41ad0a92662 Dec 03 11:39:47 crc kubenswrapper[5052]: I1203 11:39:47.605872 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 11:39:47 crc kubenswrapper[5052]: I1203 11:39:47.617458 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/openstackclient"] Dec 03 11:39:47 crc kubenswrapper[5052]: I1203 11:39:47.670547 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/14e9bff5-66d5-4808-8446-cf180d759208-scripts\") pod \"cinder-scheduler-0\" (UID: \"14e9bff5-66d5-4808-8446-cf180d759208\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:39:47 crc kubenswrapper[5052]: I1203 11:39:47.670605 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/14e9bff5-66d5-4808-8446-cf180d759208-etc-machine-id\") pod \"cinder-scheduler-0\" (UID: \"14e9bff5-66d5-4808-8446-cf180d759208\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:39:47 crc kubenswrapper[5052]: I1203 11:39:47.670637 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wzf59\" (UniqueName: \"kubernetes.io/projected/14e9bff5-66d5-4808-8446-cf180d759208-kube-api-access-wzf59\") pod \"cinder-scheduler-0\" (UID: \"14e9bff5-66d5-4808-8446-cf180d759208\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:39:47 crc kubenswrapper[5052]: I1203 11:39:47.670669 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/14e9bff5-66d5-4808-8446-cf180d759208-combined-ca-bundle\") pod \"cinder-scheduler-0\" (UID: \"14e9bff5-66d5-4808-8446-cf180d759208\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:39:47 crc kubenswrapper[5052]: I1203 11:39:47.670695 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/14e9bff5-66d5-4808-8446-cf180d759208-config-data\") pod \"cinder-scheduler-0\" (UID: \"14e9bff5-66d5-4808-8446-cf180d759208\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:39:47 crc kubenswrapper[5052]: I1203 11:39:47.670757 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/14e9bff5-66d5-4808-8446-cf180d759208-config-data-custom\") pod \"cinder-scheduler-0\" (UID: \"14e9bff5-66d5-4808-8446-cf180d759208\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:39:47 crc kubenswrapper[5052]: I1203 11:39:47.670804 5052 reconciler_common.go:293] "Volume detached for volume \"openstack-config-secret\" (UniqueName: \"kubernetes.io/secret/25ad6214-e650-4b24-95b3-8459a622a28c-openstack-config-secret\") on node \"crc\" DevicePath \"\"" Dec 03 11:39:47 crc kubenswrapper[5052]: I1203 11:39:47.670815 5052 reconciler_common.go:293] "Volume detached for volume \"openstack-config\" (UniqueName: \"kubernetes.io/configmap/25ad6214-e650-4b24-95b3-8459a622a28c-openstack-config\") on node \"crc\" DevicePath \"\"" Dec 03 11:39:47 crc kubenswrapper[5052]: I1203 11:39:47.670824 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-f8m2n\" (UniqueName: \"kubernetes.io/projected/25ad6214-e650-4b24-95b3-8459a622a28c-kube-api-access-f8m2n\") on node \"crc\" DevicePath \"\"" Dec 03 11:39:47 crc kubenswrapper[5052]: I1203 11:39:47.670832 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/25ad6214-e650-4b24-95b3-8459a622a28c-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:39:47 crc kubenswrapper[5052]: I1203 11:39:47.714804 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="25ad6214-e650-4b24-95b3-8459a622a28c" path="/var/lib/kubelet/pods/25ad6214-e650-4b24-95b3-8459a622a28c/volumes" Dec 03 11:39:47 crc kubenswrapper[5052]: I1203 11:39:47.715412 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a8040985-8ab8-47ee-b1e8-ad692191fe4d" path="/var/lib/kubelet/pods/a8040985-8ab8-47ee-b1e8-ad692191fe4d/volumes" Dec 03 11:39:47 crc kubenswrapper[5052]: I1203 11:39:47.771709 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-wzf59\" (UniqueName: \"kubernetes.io/projected/14e9bff5-66d5-4808-8446-cf180d759208-kube-api-access-wzf59\") pod \"cinder-scheduler-0\" (UID: \"14e9bff5-66d5-4808-8446-cf180d759208\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:39:47 crc kubenswrapper[5052]: I1203 11:39:47.771774 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/14e9bff5-66d5-4808-8446-cf180d759208-combined-ca-bundle\") pod \"cinder-scheduler-0\" (UID: \"14e9bff5-66d5-4808-8446-cf180d759208\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:39:47 crc kubenswrapper[5052]: I1203 11:39:47.771797 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/14e9bff5-66d5-4808-8446-cf180d759208-config-data\") pod \"cinder-scheduler-0\" (UID: \"14e9bff5-66d5-4808-8446-cf180d759208\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:39:47 crc kubenswrapper[5052]: I1203 11:39:47.771884 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/14e9bff5-66d5-4808-8446-cf180d759208-config-data-custom\") pod \"cinder-scheduler-0\" (UID: \"14e9bff5-66d5-4808-8446-cf180d759208\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:39:47 crc kubenswrapper[5052]: I1203 11:39:47.771974 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/14e9bff5-66d5-4808-8446-cf180d759208-scripts\") pod \"cinder-scheduler-0\" (UID: \"14e9bff5-66d5-4808-8446-cf180d759208\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:39:47 crc kubenswrapper[5052]: I1203 11:39:47.772050 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/14e9bff5-66d5-4808-8446-cf180d759208-etc-machine-id\") pod \"cinder-scheduler-0\" (UID: \"14e9bff5-66d5-4808-8446-cf180d759208\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:39:47 crc kubenswrapper[5052]: I1203 11:39:47.772125 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/14e9bff5-66d5-4808-8446-cf180d759208-etc-machine-id\") pod \"cinder-scheduler-0\" (UID: \"14e9bff5-66d5-4808-8446-cf180d759208\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:39:47 crc kubenswrapper[5052]: I1203 11:39:47.780181 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/14e9bff5-66d5-4808-8446-cf180d759208-scripts\") pod \"cinder-scheduler-0\" (UID: \"14e9bff5-66d5-4808-8446-cf180d759208\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:39:47 crc kubenswrapper[5052]: I1203 11:39:47.780554 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/14e9bff5-66d5-4808-8446-cf180d759208-config-data\") pod \"cinder-scheduler-0\" (UID: \"14e9bff5-66d5-4808-8446-cf180d759208\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:39:47 crc kubenswrapper[5052]: I1203 11:39:47.781042 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/14e9bff5-66d5-4808-8446-cf180d759208-combined-ca-bundle\") pod \"cinder-scheduler-0\" (UID: \"14e9bff5-66d5-4808-8446-cf180d759208\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:39:47 crc kubenswrapper[5052]: I1203 11:39:47.784173 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/14e9bff5-66d5-4808-8446-cf180d759208-config-data-custom\") pod \"cinder-scheduler-0\" (UID: \"14e9bff5-66d5-4808-8446-cf180d759208\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:39:47 crc kubenswrapper[5052]: I1203 11:39:47.799104 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-wzf59\" (UniqueName: \"kubernetes.io/projected/14e9bff5-66d5-4808-8446-cf180d759208-kube-api-access-wzf59\") pod \"cinder-scheduler-0\" (UID: \"14e9bff5-66d5-4808-8446-cf180d759208\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:39:47 crc kubenswrapper[5052]: I1203 11:39:47.910795 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:39:48 crc kubenswrapper[5052]: W1203 11:39:48.421533 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod14e9bff5_66d5_4808_8446_cf180d759208.slice/crio-88c1ea502795f1e909ef74fbad2692105bbd678ea481be72b91edbbae6c098d6 WatchSource:0}: Error finding container 88c1ea502795f1e909ef74fbad2692105bbd678ea481be72b91edbbae6c098d6: Status 404 returned error can't find the container with id 88c1ea502795f1e909ef74fbad2692105bbd678ea481be72b91edbbae6c098d6 Dec 03 11:39:48 crc kubenswrapper[5052]: I1203 11:39:48.423475 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 11:39:48 crc kubenswrapper[5052]: I1203 11:39:48.490836 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstackclient" event={"ID":"722e5457-ae47-40b8-816f-8eab5b7a4cb0","Type":"ContainerStarted","Data":"c546fad2c7df08ae53296fd9af23e0254a60b21cb613124162dce41ad0a92662"} Dec 03 11:39:48 crc kubenswrapper[5052]: I1203 11:39:48.503815 5052 generic.go:334] "Generic (PLEG): container finished" podID="577c99af-c225-41f3-9d97-e96326a8c1c4" containerID="f198dbe5aa735908998d7dc99e4c1016ec4ce99693fe8cdc9b5c3d6597777a05" exitCode=0 Dec 03 11:39:48 crc kubenswrapper[5052]: I1203 11:39:48.503908 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-58dc59ff57-9dsf9" event={"ID":"577c99af-c225-41f3-9d97-e96326a8c1c4","Type":"ContainerDied","Data":"f198dbe5aa735908998d7dc99e4c1016ec4ce99693fe8cdc9b5c3d6597777a05"} Dec 03 11:39:48 crc kubenswrapper[5052]: I1203 11:39:48.506059 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstackclient" Dec 03 11:39:48 crc kubenswrapper[5052]: I1203 11:39:48.506076 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"14e9bff5-66d5-4808-8446-cf180d759208","Type":"ContainerStarted","Data":"88c1ea502795f1e909ef74fbad2692105bbd678ea481be72b91edbbae6c098d6"} Dec 03 11:39:48 crc kubenswrapper[5052]: I1203 11:39:48.513158 5052 status_manager.go:861] "Pod was deleted and then recreated, skipping status update" pod="openstack-kuttl-tests/openstackclient" oldPodUID="25ad6214-e650-4b24-95b3-8459a622a28c" podUID="722e5457-ae47-40b8-816f-8eab5b7a4cb0" Dec 03 11:39:49 crc kubenswrapper[5052]: I1203 11:39:49.317960 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-58dc59ff57-9dsf9" Dec 03 11:39:49 crc kubenswrapper[5052]: I1203 11:39:49.403432 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/577c99af-c225-41f3-9d97-e96326a8c1c4-config\") pod \"577c99af-c225-41f3-9d97-e96326a8c1c4\" (UID: \"577c99af-c225-41f3-9d97-e96326a8c1c4\") " Dec 03 11:39:49 crc kubenswrapper[5052]: I1203 11:39:49.403587 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/577c99af-c225-41f3-9d97-e96326a8c1c4-combined-ca-bundle\") pod \"577c99af-c225-41f3-9d97-e96326a8c1c4\" (UID: \"577c99af-c225-41f3-9d97-e96326a8c1c4\") " Dec 03 11:39:49 crc kubenswrapper[5052]: I1203 11:39:49.403620 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q6nff\" (UniqueName: \"kubernetes.io/projected/577c99af-c225-41f3-9d97-e96326a8c1c4-kube-api-access-q6nff\") pod \"577c99af-c225-41f3-9d97-e96326a8c1c4\" (UID: \"577c99af-c225-41f3-9d97-e96326a8c1c4\") " Dec 03 11:39:49 crc kubenswrapper[5052]: I1203 11:39:49.403699 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/577c99af-c225-41f3-9d97-e96326a8c1c4-ovndb-tls-certs\") pod \"577c99af-c225-41f3-9d97-e96326a8c1c4\" (UID: \"577c99af-c225-41f3-9d97-e96326a8c1c4\") " Dec 03 11:39:49 crc kubenswrapper[5052]: I1203 11:39:49.403718 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/577c99af-c225-41f3-9d97-e96326a8c1c4-httpd-config\") pod \"577c99af-c225-41f3-9d97-e96326a8c1c4\" (UID: \"577c99af-c225-41f3-9d97-e96326a8c1c4\") " Dec 03 11:39:49 crc kubenswrapper[5052]: I1203 11:39:49.409633 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/577c99af-c225-41f3-9d97-e96326a8c1c4-httpd-config" (OuterVolumeSpecName: "httpd-config") pod "577c99af-c225-41f3-9d97-e96326a8c1c4" (UID: "577c99af-c225-41f3-9d97-e96326a8c1c4"). InnerVolumeSpecName "httpd-config". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:39:49 crc kubenswrapper[5052]: I1203 11:39:49.425712 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/577c99af-c225-41f3-9d97-e96326a8c1c4-kube-api-access-q6nff" (OuterVolumeSpecName: "kube-api-access-q6nff") pod "577c99af-c225-41f3-9d97-e96326a8c1c4" (UID: "577c99af-c225-41f3-9d97-e96326a8c1c4"). InnerVolumeSpecName "kube-api-access-q6nff". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:39:49 crc kubenswrapper[5052]: I1203 11:39:49.490315 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/577c99af-c225-41f3-9d97-e96326a8c1c4-config" (OuterVolumeSpecName: "config") pod "577c99af-c225-41f3-9d97-e96326a8c1c4" (UID: "577c99af-c225-41f3-9d97-e96326a8c1c4"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:39:49 crc kubenswrapper[5052]: I1203 11:39:49.496181 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/577c99af-c225-41f3-9d97-e96326a8c1c4-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "577c99af-c225-41f3-9d97-e96326a8c1c4" (UID: "577c99af-c225-41f3-9d97-e96326a8c1c4"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:39:49 crc kubenswrapper[5052]: I1203 11:39:49.505036 5052 reconciler_common.go:293] "Volume detached for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/577c99af-c225-41f3-9d97-e96326a8c1c4-httpd-config\") on node \"crc\" DevicePath \"\"" Dec 03 11:39:49 crc kubenswrapper[5052]: I1203 11:39:49.505068 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/577c99af-c225-41f3-9d97-e96326a8c1c4-config\") on node \"crc\" DevicePath \"\"" Dec 03 11:39:49 crc kubenswrapper[5052]: I1203 11:39:49.505078 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/577c99af-c225-41f3-9d97-e96326a8c1c4-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:39:49 crc kubenswrapper[5052]: I1203 11:39:49.505088 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-q6nff\" (UniqueName: \"kubernetes.io/projected/577c99af-c225-41f3-9d97-e96326a8c1c4-kube-api-access-q6nff\") on node \"crc\" DevicePath \"\"" Dec 03 11:39:49 crc kubenswrapper[5052]: I1203 11:39:49.517643 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/577c99af-c225-41f3-9d97-e96326a8c1c4-ovndb-tls-certs" (OuterVolumeSpecName: "ovndb-tls-certs") pod "577c99af-c225-41f3-9d97-e96326a8c1c4" (UID: "577c99af-c225-41f3-9d97-e96326a8c1c4"). InnerVolumeSpecName "ovndb-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:39:49 crc kubenswrapper[5052]: I1203 11:39:49.543631 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-58dc59ff57-9dsf9" event={"ID":"577c99af-c225-41f3-9d97-e96326a8c1c4","Type":"ContainerDied","Data":"4973fa976f43a54ac2651aef1f0013167f193a85272c9d8480cdb9ff98c3fe29"} Dec 03 11:39:49 crc kubenswrapper[5052]: I1203 11:39:49.543678 5052 scope.go:117] "RemoveContainer" containerID="a55c9e545d9db9e0eaff80f2fad72700c0a38b5a3dcd4bb8357e991a4e2f25c1" Dec 03 11:39:49 crc kubenswrapper[5052]: I1203 11:39:49.543796 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-58dc59ff57-9dsf9" Dec 03 11:39:49 crc kubenswrapper[5052]: I1203 11:39:49.556054 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"14e9bff5-66d5-4808-8446-cf180d759208","Type":"ContainerStarted","Data":"b875595b2b281c94c97104d6ea786f06ea56deeebb4024873ed0f82dba8c7b52"} Dec 03 11:39:49 crc kubenswrapper[5052]: I1203 11:39:49.589041 5052 scope.go:117] "RemoveContainer" containerID="f198dbe5aa735908998d7dc99e4c1016ec4ce99693fe8cdc9b5c3d6597777a05" Dec 03 11:39:49 crc kubenswrapper[5052]: I1203 11:39:49.595947 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/neutron-58dc59ff57-9dsf9"] Dec 03 11:39:49 crc kubenswrapper[5052]: I1203 11:39:49.606570 5052 reconciler_common.go:293] "Volume detached for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/577c99af-c225-41f3-9d97-e96326a8c1c4-ovndb-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 11:39:49 crc kubenswrapper[5052]: I1203 11:39:49.609145 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/neutron-58dc59ff57-9dsf9"] Dec 03 11:39:49 crc kubenswrapper[5052]: I1203 11:39:49.701822 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="577c99af-c225-41f3-9d97-e96326a8c1c4" path="/var/lib/kubelet/pods/577c99af-c225-41f3-9d97-e96326a8c1c4/volumes" Dec 03 11:39:50 crc kubenswrapper[5052]: I1203 11:39:50.550502 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/barbican-api-66f4bb9676-j6vzc" Dec 03 11:39:50 crc kubenswrapper[5052]: I1203 11:39:50.576966 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"14e9bff5-66d5-4808-8446-cf180d759208","Type":"ContainerStarted","Data":"6c977dab718207e18c944877c552253d8c5723de1e4d496e75dea068be83b234"} Dec 03 11:39:50 crc kubenswrapper[5052]: I1203 11:39:50.596371 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/cinder-scheduler-0" podStartSLOduration=3.596351733 podStartE2EDuration="3.596351733s" podCreationTimestamp="2025-12-03 11:39:47 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:39:50.593935663 +0000 UTC m=+1181.182177427" watchObservedRunningTime="2025-12-03 11:39:50.596351733 +0000 UTC m=+1181.184593497" Dec 03 11:39:50 crc kubenswrapper[5052]: I1203 11:39:50.719002 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/barbican-api-66f4bb9676-j6vzc" Dec 03 11:39:50 crc kubenswrapper[5052]: I1203 11:39:50.809962 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-api-65968d9848-mntb6"] Dec 03 11:39:50 crc kubenswrapper[5052]: I1203 11:39:50.810391 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/barbican-api-65968d9848-mntb6" podUID="ef47ba52-772d-4732-bd15-e7513e626be9" containerName="barbican-api-log" containerID="cri-o://a00718351b51b3492d282952f384864b842002d371827646c87c60e86863c87e" gracePeriod=30 Dec 03 11:39:50 crc kubenswrapper[5052]: I1203 11:39:50.814231 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/barbican-api-65968d9848-mntb6" podUID="ef47ba52-772d-4732-bd15-e7513e626be9" containerName="barbican-api" containerID="cri-o://f531b9f35a146882772e5bb658e2970143bf9db9263912e975397ff80f1c2cda" gracePeriod=30 Dec 03 11:39:51 crc kubenswrapper[5052]: I1203 11:39:51.105330 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/swift-proxy-fd9678558-qx8wh"] Dec 03 11:39:51 crc kubenswrapper[5052]: E1203 11:39:51.105716 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="577c99af-c225-41f3-9d97-e96326a8c1c4" containerName="neutron-httpd" Dec 03 11:39:51 crc kubenswrapper[5052]: I1203 11:39:51.105730 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="577c99af-c225-41f3-9d97-e96326a8c1c4" containerName="neutron-httpd" Dec 03 11:39:51 crc kubenswrapper[5052]: E1203 11:39:51.105764 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="577c99af-c225-41f3-9d97-e96326a8c1c4" containerName="neutron-api" Dec 03 11:39:51 crc kubenswrapper[5052]: I1203 11:39:51.105771 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="577c99af-c225-41f3-9d97-e96326a8c1c4" containerName="neutron-api" Dec 03 11:39:51 crc kubenswrapper[5052]: I1203 11:39:51.105944 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="577c99af-c225-41f3-9d97-e96326a8c1c4" containerName="neutron-httpd" Dec 03 11:39:51 crc kubenswrapper[5052]: I1203 11:39:51.105960 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="577c99af-c225-41f3-9d97-e96326a8c1c4" containerName="neutron-api" Dec 03 11:39:51 crc kubenswrapper[5052]: I1203 11:39:51.106798 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-proxy-fd9678558-qx8wh" Dec 03 11:39:51 crc kubenswrapper[5052]: I1203 11:39:51.108864 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"swift-proxy-config-data" Dec 03 11:39:51 crc kubenswrapper[5052]: I1203 11:39:51.109065 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-swift-public-svc" Dec 03 11:39:51 crc kubenswrapper[5052]: I1203 11:39:51.109296 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-swift-internal-svc" Dec 03 11:39:51 crc kubenswrapper[5052]: I1203 11:39:51.122976 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/swift-proxy-fd9678558-qx8wh"] Dec 03 11:39:51 crc kubenswrapper[5052]: I1203 11:39:51.136702 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/fd14a5c0-3402-4891-a726-32d75b1b7857-internal-tls-certs\") pod \"swift-proxy-fd9678558-qx8wh\" (UID: \"fd14a5c0-3402-4891-a726-32d75b1b7857\") " pod="openstack-kuttl-tests/swift-proxy-fd9678558-qx8wh" Dec 03 11:39:51 crc kubenswrapper[5052]: I1203 11:39:51.136758 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hj52s\" (UniqueName: \"kubernetes.io/projected/fd14a5c0-3402-4891-a726-32d75b1b7857-kube-api-access-hj52s\") pod \"swift-proxy-fd9678558-qx8wh\" (UID: \"fd14a5c0-3402-4891-a726-32d75b1b7857\") " pod="openstack-kuttl-tests/swift-proxy-fd9678558-qx8wh" Dec 03 11:39:51 crc kubenswrapper[5052]: I1203 11:39:51.136780 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/fd14a5c0-3402-4891-a726-32d75b1b7857-log-httpd\") pod \"swift-proxy-fd9678558-qx8wh\" (UID: \"fd14a5c0-3402-4891-a726-32d75b1b7857\") " pod="openstack-kuttl-tests/swift-proxy-fd9678558-qx8wh" Dec 03 11:39:51 crc kubenswrapper[5052]: I1203 11:39:51.136852 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/fd14a5c0-3402-4891-a726-32d75b1b7857-combined-ca-bundle\") pod \"swift-proxy-fd9678558-qx8wh\" (UID: \"fd14a5c0-3402-4891-a726-32d75b1b7857\") " pod="openstack-kuttl-tests/swift-proxy-fd9678558-qx8wh" Dec 03 11:39:51 crc kubenswrapper[5052]: I1203 11:39:51.136893 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/fd14a5c0-3402-4891-a726-32d75b1b7857-run-httpd\") pod \"swift-proxy-fd9678558-qx8wh\" (UID: \"fd14a5c0-3402-4891-a726-32d75b1b7857\") " pod="openstack-kuttl-tests/swift-proxy-fd9678558-qx8wh" Dec 03 11:39:51 crc kubenswrapper[5052]: I1203 11:39:51.136917 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/fd14a5c0-3402-4891-a726-32d75b1b7857-public-tls-certs\") pod \"swift-proxy-fd9678558-qx8wh\" (UID: \"fd14a5c0-3402-4891-a726-32d75b1b7857\") " pod="openstack-kuttl-tests/swift-proxy-fd9678558-qx8wh" Dec 03 11:39:51 crc kubenswrapper[5052]: I1203 11:39:51.136945 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/fd14a5c0-3402-4891-a726-32d75b1b7857-config-data\") pod \"swift-proxy-fd9678558-qx8wh\" (UID: \"fd14a5c0-3402-4891-a726-32d75b1b7857\") " pod="openstack-kuttl-tests/swift-proxy-fd9678558-qx8wh" Dec 03 11:39:51 crc kubenswrapper[5052]: I1203 11:39:51.136962 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/fd14a5c0-3402-4891-a726-32d75b1b7857-etc-swift\") pod \"swift-proxy-fd9678558-qx8wh\" (UID: \"fd14a5c0-3402-4891-a726-32d75b1b7857\") " pod="openstack-kuttl-tests/swift-proxy-fd9678558-qx8wh" Dec 03 11:39:51 crc kubenswrapper[5052]: I1203 11:39:51.238129 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/fd14a5c0-3402-4891-a726-32d75b1b7857-run-httpd\") pod \"swift-proxy-fd9678558-qx8wh\" (UID: \"fd14a5c0-3402-4891-a726-32d75b1b7857\") " pod="openstack-kuttl-tests/swift-proxy-fd9678558-qx8wh" Dec 03 11:39:51 crc kubenswrapper[5052]: I1203 11:39:51.238180 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/fd14a5c0-3402-4891-a726-32d75b1b7857-public-tls-certs\") pod \"swift-proxy-fd9678558-qx8wh\" (UID: \"fd14a5c0-3402-4891-a726-32d75b1b7857\") " pod="openstack-kuttl-tests/swift-proxy-fd9678558-qx8wh" Dec 03 11:39:51 crc kubenswrapper[5052]: I1203 11:39:51.238212 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/fd14a5c0-3402-4891-a726-32d75b1b7857-config-data\") pod \"swift-proxy-fd9678558-qx8wh\" (UID: \"fd14a5c0-3402-4891-a726-32d75b1b7857\") " pod="openstack-kuttl-tests/swift-proxy-fd9678558-qx8wh" Dec 03 11:39:51 crc kubenswrapper[5052]: I1203 11:39:51.238227 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/fd14a5c0-3402-4891-a726-32d75b1b7857-etc-swift\") pod \"swift-proxy-fd9678558-qx8wh\" (UID: \"fd14a5c0-3402-4891-a726-32d75b1b7857\") " pod="openstack-kuttl-tests/swift-proxy-fd9678558-qx8wh" Dec 03 11:39:51 crc kubenswrapper[5052]: I1203 11:39:51.238271 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/fd14a5c0-3402-4891-a726-32d75b1b7857-internal-tls-certs\") pod \"swift-proxy-fd9678558-qx8wh\" (UID: \"fd14a5c0-3402-4891-a726-32d75b1b7857\") " pod="openstack-kuttl-tests/swift-proxy-fd9678558-qx8wh" Dec 03 11:39:51 crc kubenswrapper[5052]: I1203 11:39:51.238287 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-hj52s\" (UniqueName: \"kubernetes.io/projected/fd14a5c0-3402-4891-a726-32d75b1b7857-kube-api-access-hj52s\") pod \"swift-proxy-fd9678558-qx8wh\" (UID: \"fd14a5c0-3402-4891-a726-32d75b1b7857\") " pod="openstack-kuttl-tests/swift-proxy-fd9678558-qx8wh" Dec 03 11:39:51 crc kubenswrapper[5052]: I1203 11:39:51.238304 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/fd14a5c0-3402-4891-a726-32d75b1b7857-log-httpd\") pod \"swift-proxy-fd9678558-qx8wh\" (UID: \"fd14a5c0-3402-4891-a726-32d75b1b7857\") " pod="openstack-kuttl-tests/swift-proxy-fd9678558-qx8wh" Dec 03 11:39:51 crc kubenswrapper[5052]: I1203 11:39:51.238372 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/fd14a5c0-3402-4891-a726-32d75b1b7857-combined-ca-bundle\") pod \"swift-proxy-fd9678558-qx8wh\" (UID: \"fd14a5c0-3402-4891-a726-32d75b1b7857\") " pod="openstack-kuttl-tests/swift-proxy-fd9678558-qx8wh" Dec 03 11:39:51 crc kubenswrapper[5052]: I1203 11:39:51.240637 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/fd14a5c0-3402-4891-a726-32d75b1b7857-log-httpd\") pod \"swift-proxy-fd9678558-qx8wh\" (UID: \"fd14a5c0-3402-4891-a726-32d75b1b7857\") " pod="openstack-kuttl-tests/swift-proxy-fd9678558-qx8wh" Dec 03 11:39:51 crc kubenswrapper[5052]: I1203 11:39:51.241922 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/fd14a5c0-3402-4891-a726-32d75b1b7857-run-httpd\") pod \"swift-proxy-fd9678558-qx8wh\" (UID: \"fd14a5c0-3402-4891-a726-32d75b1b7857\") " pod="openstack-kuttl-tests/swift-proxy-fd9678558-qx8wh" Dec 03 11:39:51 crc kubenswrapper[5052]: I1203 11:39:51.245916 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/fd14a5c0-3402-4891-a726-32d75b1b7857-combined-ca-bundle\") pod \"swift-proxy-fd9678558-qx8wh\" (UID: \"fd14a5c0-3402-4891-a726-32d75b1b7857\") " pod="openstack-kuttl-tests/swift-proxy-fd9678558-qx8wh" Dec 03 11:39:51 crc kubenswrapper[5052]: I1203 11:39:51.248814 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/fd14a5c0-3402-4891-a726-32d75b1b7857-config-data\") pod \"swift-proxy-fd9678558-qx8wh\" (UID: \"fd14a5c0-3402-4891-a726-32d75b1b7857\") " pod="openstack-kuttl-tests/swift-proxy-fd9678558-qx8wh" Dec 03 11:39:51 crc kubenswrapper[5052]: I1203 11:39:51.250099 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/fd14a5c0-3402-4891-a726-32d75b1b7857-internal-tls-certs\") pod \"swift-proxy-fd9678558-qx8wh\" (UID: \"fd14a5c0-3402-4891-a726-32d75b1b7857\") " pod="openstack-kuttl-tests/swift-proxy-fd9678558-qx8wh" Dec 03 11:39:51 crc kubenswrapper[5052]: I1203 11:39:51.250828 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/fd14a5c0-3402-4891-a726-32d75b1b7857-etc-swift\") pod \"swift-proxy-fd9678558-qx8wh\" (UID: \"fd14a5c0-3402-4891-a726-32d75b1b7857\") " pod="openstack-kuttl-tests/swift-proxy-fd9678558-qx8wh" Dec 03 11:39:51 crc kubenswrapper[5052]: I1203 11:39:51.251067 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/fd14a5c0-3402-4891-a726-32d75b1b7857-public-tls-certs\") pod \"swift-proxy-fd9678558-qx8wh\" (UID: \"fd14a5c0-3402-4891-a726-32d75b1b7857\") " pod="openstack-kuttl-tests/swift-proxy-fd9678558-qx8wh" Dec 03 11:39:51 crc kubenswrapper[5052]: I1203 11:39:51.260199 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-hj52s\" (UniqueName: \"kubernetes.io/projected/fd14a5c0-3402-4891-a726-32d75b1b7857-kube-api-access-hj52s\") pod \"swift-proxy-fd9678558-qx8wh\" (UID: \"fd14a5c0-3402-4891-a726-32d75b1b7857\") " pod="openstack-kuttl-tests/swift-proxy-fd9678558-qx8wh" Dec 03 11:39:51 crc kubenswrapper[5052]: I1203 11:39:51.402714 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:39:51 crc kubenswrapper[5052]: I1203 11:39:51.431136 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-proxy-fd9678558-qx8wh" Dec 03 11:39:51 crc kubenswrapper[5052]: I1203 11:39:51.590641 5052 generic.go:334] "Generic (PLEG): container finished" podID="ef47ba52-772d-4732-bd15-e7513e626be9" containerID="a00718351b51b3492d282952f384864b842002d371827646c87c60e86863c87e" exitCode=143 Dec 03 11:39:51 crc kubenswrapper[5052]: I1203 11:39:51.590732 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-65968d9848-mntb6" event={"ID":"ef47ba52-772d-4732-bd15-e7513e626be9","Type":"ContainerDied","Data":"a00718351b51b3492d282952f384864b842002d371827646c87c60e86863c87e"} Dec 03 11:39:52 crc kubenswrapper[5052]: I1203 11:39:52.056880 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/swift-proxy-fd9678558-qx8wh"] Dec 03 11:39:52 crc kubenswrapper[5052]: W1203 11:39:52.070124 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfd14a5c0_3402_4891_a726_32d75b1b7857.slice/crio-568cbc12b65aa7d5ee89f3ddee562073c311967a018243e6d3cb69bb82ff3f4e WatchSource:0}: Error finding container 568cbc12b65aa7d5ee89f3ddee562073c311967a018243e6d3cb69bb82ff3f4e: Status 404 returned error can't find the container with id 568cbc12b65aa7d5ee89f3ddee562073c311967a018243e6d3cb69bb82ff3f4e Dec 03 11:39:52 crc kubenswrapper[5052]: I1203 11:39:52.600154 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-proxy-fd9678558-qx8wh" event={"ID":"fd14a5c0-3402-4891-a726-32d75b1b7857","Type":"ContainerStarted","Data":"568cbc12b65aa7d5ee89f3ddee562073c311967a018243e6d3cb69bb82ff3f4e"} Dec 03 11:39:52 crc kubenswrapper[5052]: I1203 11:39:52.911169 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:39:53 crc kubenswrapper[5052]: I1203 11:39:53.611970 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-proxy-fd9678558-qx8wh" event={"ID":"fd14a5c0-3402-4891-a726-32d75b1b7857","Type":"ContainerStarted","Data":"def28b7790031f83be9c79b54dbbc74332f5e5917d604dfb281ac32589501053"} Dec 03 11:39:53 crc kubenswrapper[5052]: I1203 11:39:53.612324 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-proxy-fd9678558-qx8wh" event={"ID":"fd14a5c0-3402-4891-a726-32d75b1b7857","Type":"ContainerStarted","Data":"3132bd07375ba7c18c7bbb9921d64b3f404ab0c85abce7e27085e92e0ba00bea"} Dec 03 11:39:53 crc kubenswrapper[5052]: I1203 11:39:53.612345 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/swift-proxy-fd9678558-qx8wh" Dec 03 11:39:53 crc kubenswrapper[5052]: I1203 11:39:53.640806 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/swift-proxy-fd9678558-qx8wh" podStartSLOduration=2.640787989 podStartE2EDuration="2.640787989s" podCreationTimestamp="2025-12-03 11:39:51 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:39:53.629679819 +0000 UTC m=+1184.217921583" watchObservedRunningTime="2025-12-03 11:39:53.640787989 +0000 UTC m=+1184.229029753" Dec 03 11:39:53 crc kubenswrapper[5052]: I1203 11:39:53.991816 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/barbican-api-65968d9848-mntb6" podUID="ef47ba52-772d-4732-bd15-e7513e626be9" containerName="barbican-api" probeResult="failure" output="Get \"http://10.217.0.144:9311/healthcheck\": read tcp 10.217.0.2:52110->10.217.0.144:9311: read: connection reset by peer" Dec 03 11:39:53 crc kubenswrapper[5052]: I1203 11:39:53.991878 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/barbican-api-65968d9848-mntb6" podUID="ef47ba52-772d-4732-bd15-e7513e626be9" containerName="barbican-api-log" probeResult="failure" output="Get \"http://10.217.0.144:9311/healthcheck\": read tcp 10.217.0.2:52112->10.217.0.144:9311: read: connection reset by peer" Dec 03 11:39:54 crc kubenswrapper[5052]: I1203 11:39:54.629015 5052 generic.go:334] "Generic (PLEG): container finished" podID="ef47ba52-772d-4732-bd15-e7513e626be9" containerID="f531b9f35a146882772e5bb658e2970143bf9db9263912e975397ff80f1c2cda" exitCode=0 Dec 03 11:39:54 crc kubenswrapper[5052]: I1203 11:39:54.629949 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-65968d9848-mntb6" event={"ID":"ef47ba52-772d-4732-bd15-e7513e626be9","Type":"ContainerDied","Data":"f531b9f35a146882772e5bb658e2970143bf9db9263912e975397ff80f1c2cda"} Dec 03 11:39:54 crc kubenswrapper[5052]: I1203 11:39:54.629985 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/swift-proxy-fd9678558-qx8wh" Dec 03 11:39:55 crc kubenswrapper[5052]: I1203 11:39:55.068675 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 11:39:55 crc kubenswrapper[5052]: I1203 11:39:55.069753 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-external-api-0" podUID="8bf67244-3ae2-4c00-a15d-4c4180d5f835" containerName="glance-log" containerID="cri-o://26f247c9c6d8664d070a0e01318f861102e07448fbac21823c5f2aca4b6f5da2" gracePeriod=30 Dec 03 11:39:55 crc kubenswrapper[5052]: I1203 11:39:55.070060 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-external-api-0" podUID="8bf67244-3ae2-4c00-a15d-4c4180d5f835" containerName="glance-httpd" containerID="cri-o://dcd58ef3b45066b82652446bcbd875e2abf4758e96cd73b62fededb0469b531e" gracePeriod=30 Dec 03 11:39:55 crc kubenswrapper[5052]: I1203 11:39:55.219667 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:39:55 crc kubenswrapper[5052]: I1203 11:39:55.219979 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="70e2b274-dd4d-409a-8d45-a09af330f7a3" containerName="ceilometer-central-agent" containerID="cri-o://a976462aed958683aefdf6044f7b3b06bddc3fc3b6d77f4841493e30c067161c" gracePeriod=30 Dec 03 11:39:55 crc kubenswrapper[5052]: I1203 11:39:55.220098 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="70e2b274-dd4d-409a-8d45-a09af330f7a3" containerName="proxy-httpd" containerID="cri-o://33c7c237d3624c744edaa6397912e20b3e447503cda667a0d60162d16ad547d5" gracePeriod=30 Dec 03 11:39:55 crc kubenswrapper[5052]: I1203 11:39:55.220129 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="70e2b274-dd4d-409a-8d45-a09af330f7a3" containerName="sg-core" containerID="cri-o://89b928d0bee5a1b120bb1a4de7f35f330ce3fb961abc80c74f6017fd2b9b6ae4" gracePeriod=30 Dec 03 11:39:55 crc kubenswrapper[5052]: I1203 11:39:55.220112 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="70e2b274-dd4d-409a-8d45-a09af330f7a3" containerName="ceilometer-notification-agent" containerID="cri-o://466f8e75a4f4613fdb58e65fc3b11d64b99b313618c5c45ebe14444fc70dbacf" gracePeriod=30 Dec 03 11:39:55 crc kubenswrapper[5052]: I1203 11:39:55.232108 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/ceilometer-0" podUID="70e2b274-dd4d-409a-8d45-a09af330f7a3" containerName="proxy-httpd" probeResult="failure" output="Get \"http://10.217.0.141:3000/\": EOF" Dec 03 11:39:55 crc kubenswrapper[5052]: I1203 11:39:55.646548 5052 generic.go:334] "Generic (PLEG): container finished" podID="70e2b274-dd4d-409a-8d45-a09af330f7a3" containerID="33c7c237d3624c744edaa6397912e20b3e447503cda667a0d60162d16ad547d5" exitCode=0 Dec 03 11:39:55 crc kubenswrapper[5052]: I1203 11:39:55.646574 5052 generic.go:334] "Generic (PLEG): container finished" podID="70e2b274-dd4d-409a-8d45-a09af330f7a3" containerID="89b928d0bee5a1b120bb1a4de7f35f330ce3fb961abc80c74f6017fd2b9b6ae4" exitCode=2 Dec 03 11:39:55 crc kubenswrapper[5052]: I1203 11:39:55.646613 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"70e2b274-dd4d-409a-8d45-a09af330f7a3","Type":"ContainerDied","Data":"33c7c237d3624c744edaa6397912e20b3e447503cda667a0d60162d16ad547d5"} Dec 03 11:39:55 crc kubenswrapper[5052]: I1203 11:39:55.646638 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"70e2b274-dd4d-409a-8d45-a09af330f7a3","Type":"ContainerDied","Data":"89b928d0bee5a1b120bb1a4de7f35f330ce3fb961abc80c74f6017fd2b9b6ae4"} Dec 03 11:39:55 crc kubenswrapper[5052]: I1203 11:39:55.649153 5052 generic.go:334] "Generic (PLEG): container finished" podID="8bf67244-3ae2-4c00-a15d-4c4180d5f835" containerID="26f247c9c6d8664d070a0e01318f861102e07448fbac21823c5f2aca4b6f5da2" exitCode=143 Dec 03 11:39:55 crc kubenswrapper[5052]: I1203 11:39:55.649234 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"8bf67244-3ae2-4c00-a15d-4c4180d5f835","Type":"ContainerDied","Data":"26f247c9c6d8664d070a0e01318f861102e07448fbac21823c5f2aca4b6f5da2"} Dec 03 11:39:56 crc kubenswrapper[5052]: I1203 11:39:56.064759 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 11:39:56 crc kubenswrapper[5052]: I1203 11:39:56.065318 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-internal-api-0" podUID="502cb03c-d1a5-4dc7-a040-0571a4695379" containerName="glance-log" containerID="cri-o://33a49358f7fcc1bb28274587ea7ed79d3fcea032502f0292b9e672835cb852c4" gracePeriod=30 Dec 03 11:39:56 crc kubenswrapper[5052]: I1203 11:39:56.065420 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-internal-api-0" podUID="502cb03c-d1a5-4dc7-a040-0571a4695379" containerName="glance-httpd" containerID="cri-o://3c6d3ddfe8759dec5d3030b3c1b64bcbc9f4228dfeaac36309751c177de0b102" gracePeriod=30 Dec 03 11:39:56 crc kubenswrapper[5052]: I1203 11:39:56.325937 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/barbican-api-65968d9848-mntb6" podUID="ef47ba52-772d-4732-bd15-e7513e626be9" containerName="barbican-api-log" probeResult="failure" output="Get \"http://10.217.0.144:9311/healthcheck\": dial tcp 10.217.0.144:9311: connect: connection refused" Dec 03 11:39:56 crc kubenswrapper[5052]: I1203 11:39:56.325937 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/barbican-api-65968d9848-mntb6" podUID="ef47ba52-772d-4732-bd15-e7513e626be9" containerName="barbican-api" probeResult="failure" output="Get \"http://10.217.0.144:9311/healthcheck\": dial tcp 10.217.0.144:9311: connect: connection refused" Dec 03 11:39:56 crc kubenswrapper[5052]: I1203 11:39:56.666850 5052 generic.go:334] "Generic (PLEG): container finished" podID="70e2b274-dd4d-409a-8d45-a09af330f7a3" containerID="a976462aed958683aefdf6044f7b3b06bddc3fc3b6d77f4841493e30c067161c" exitCode=0 Dec 03 11:39:56 crc kubenswrapper[5052]: I1203 11:39:56.666928 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"70e2b274-dd4d-409a-8d45-a09af330f7a3","Type":"ContainerDied","Data":"a976462aed958683aefdf6044f7b3b06bddc3fc3b6d77f4841493e30c067161c"} Dec 03 11:39:56 crc kubenswrapper[5052]: I1203 11:39:56.669808 5052 generic.go:334] "Generic (PLEG): container finished" podID="502cb03c-d1a5-4dc7-a040-0571a4695379" containerID="33a49358f7fcc1bb28274587ea7ed79d3fcea032502f0292b9e672835cb852c4" exitCode=143 Dec 03 11:39:56 crc kubenswrapper[5052]: I1203 11:39:56.669853 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"502cb03c-d1a5-4dc7-a040-0571a4695379","Type":"ContainerDied","Data":"33a49358f7fcc1bb28274587ea7ed79d3fcea032502f0292b9e672835cb852c4"} Dec 03 11:39:57 crc kubenswrapper[5052]: I1203 11:39:57.895556 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-api-db-create-wm8nm"] Dec 03 11:39:57 crc kubenswrapper[5052]: I1203 11:39:57.897097 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-db-create-wm8nm" Dec 03 11:39:57 crc kubenswrapper[5052]: I1203 11:39:57.912941 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-db-create-wm8nm"] Dec 03 11:39:57 crc kubenswrapper[5052]: I1203 11:39:57.994812 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell0-db-create-w672x"] Dec 03 11:39:57 crc kubenswrapper[5052]: I1203 11:39:57.998985 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-db-create-w672x" Dec 03 11:39:58 crc kubenswrapper[5052]: I1203 11:39:58.007217 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-api-73a4-account-create-update-kfmjt"] Dec 03 11:39:58 crc kubenswrapper[5052]: I1203 11:39:58.008746 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-73a4-account-create-update-kfmjt" Dec 03 11:39:58 crc kubenswrapper[5052]: I1203 11:39:58.013534 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-api-db-secret" Dec 03 11:39:58 crc kubenswrapper[5052]: I1203 11:39:58.018751 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell0-db-create-w672x"] Dec 03 11:39:58 crc kubenswrapper[5052]: I1203 11:39:58.030047 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-73a4-account-create-update-kfmjt"] Dec 03 11:39:58 crc kubenswrapper[5052]: I1203 11:39:58.089385 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xfcmg\" (UniqueName: \"kubernetes.io/projected/996bddd2-d361-4a2f-aa3c-773b37ad9b59-kube-api-access-xfcmg\") pod \"nova-api-db-create-wm8nm\" (UID: \"996bddd2-d361-4a2f-aa3c-773b37ad9b59\") " pod="openstack-kuttl-tests/nova-api-db-create-wm8nm" Dec 03 11:39:58 crc kubenswrapper[5052]: I1203 11:39:58.089475 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/996bddd2-d361-4a2f-aa3c-773b37ad9b59-operator-scripts\") pod \"nova-api-db-create-wm8nm\" (UID: \"996bddd2-d361-4a2f-aa3c-773b37ad9b59\") " pod="openstack-kuttl-tests/nova-api-db-create-wm8nm" Dec 03 11:39:58 crc kubenswrapper[5052]: I1203 11:39:58.192344 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/26a22bd0-90b3-4fe9-8029-a56c6c7bcdfd-operator-scripts\") pod \"nova-api-73a4-account-create-update-kfmjt\" (UID: \"26a22bd0-90b3-4fe9-8029-a56c6c7bcdfd\") " pod="openstack-kuttl-tests/nova-api-73a4-account-create-update-kfmjt" Dec 03 11:39:58 crc kubenswrapper[5052]: I1203 11:39:58.192404 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/6410c857-4e7d-486c-be0a-6734565014d4-operator-scripts\") pod \"nova-cell0-db-create-w672x\" (UID: \"6410c857-4e7d-486c-be0a-6734565014d4\") " pod="openstack-kuttl-tests/nova-cell0-db-create-w672x" Dec 03 11:39:58 crc kubenswrapper[5052]: I1203 11:39:58.192434 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xg85d\" (UniqueName: \"kubernetes.io/projected/6410c857-4e7d-486c-be0a-6734565014d4-kube-api-access-xg85d\") pod \"nova-cell0-db-create-w672x\" (UID: \"6410c857-4e7d-486c-be0a-6734565014d4\") " pod="openstack-kuttl-tests/nova-cell0-db-create-w672x" Dec 03 11:39:58 crc kubenswrapper[5052]: I1203 11:39:58.192455 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lwqbw\" (UniqueName: \"kubernetes.io/projected/26a22bd0-90b3-4fe9-8029-a56c6c7bcdfd-kube-api-access-lwqbw\") pod \"nova-api-73a4-account-create-update-kfmjt\" (UID: \"26a22bd0-90b3-4fe9-8029-a56c6c7bcdfd\") " pod="openstack-kuttl-tests/nova-api-73a4-account-create-update-kfmjt" Dec 03 11:39:58 crc kubenswrapper[5052]: I1203 11:39:58.192546 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-xfcmg\" (UniqueName: \"kubernetes.io/projected/996bddd2-d361-4a2f-aa3c-773b37ad9b59-kube-api-access-xfcmg\") pod \"nova-api-db-create-wm8nm\" (UID: \"996bddd2-d361-4a2f-aa3c-773b37ad9b59\") " pod="openstack-kuttl-tests/nova-api-db-create-wm8nm" Dec 03 11:39:58 crc kubenswrapper[5052]: I1203 11:39:58.192639 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/996bddd2-d361-4a2f-aa3c-773b37ad9b59-operator-scripts\") pod \"nova-api-db-create-wm8nm\" (UID: \"996bddd2-d361-4a2f-aa3c-773b37ad9b59\") " pod="openstack-kuttl-tests/nova-api-db-create-wm8nm" Dec 03 11:39:58 crc kubenswrapper[5052]: I1203 11:39:58.193503 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/996bddd2-d361-4a2f-aa3c-773b37ad9b59-operator-scripts\") pod \"nova-api-db-create-wm8nm\" (UID: \"996bddd2-d361-4a2f-aa3c-773b37ad9b59\") " pod="openstack-kuttl-tests/nova-api-db-create-wm8nm" Dec 03 11:39:58 crc kubenswrapper[5052]: I1203 11:39:58.209733 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell1-db-create-4mc7v"] Dec 03 11:39:58 crc kubenswrapper[5052]: I1203 11:39:58.211163 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-db-create-4mc7v" Dec 03 11:39:58 crc kubenswrapper[5052]: I1203 11:39:58.214736 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-xfcmg\" (UniqueName: \"kubernetes.io/projected/996bddd2-d361-4a2f-aa3c-773b37ad9b59-kube-api-access-xfcmg\") pod \"nova-api-db-create-wm8nm\" (UID: \"996bddd2-d361-4a2f-aa3c-773b37ad9b59\") " pod="openstack-kuttl-tests/nova-api-db-create-wm8nm" Dec 03 11:39:58 crc kubenswrapper[5052]: I1203 11:39:58.221136 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell0-3d99-account-create-update-kqb94"] Dec 03 11:39:58 crc kubenswrapper[5052]: I1203 11:39:58.223040 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-3d99-account-create-update-kqb94" Dec 03 11:39:58 crc kubenswrapper[5052]: I1203 11:39:58.223834 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-db-create-wm8nm" Dec 03 11:39:58 crc kubenswrapper[5052]: I1203 11:39:58.224933 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell0-db-secret" Dec 03 11:39:58 crc kubenswrapper[5052]: I1203 11:39:58.226651 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:39:58 crc kubenswrapper[5052]: I1203 11:39:58.231908 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-db-create-4mc7v"] Dec 03 11:39:58 crc kubenswrapper[5052]: I1203 11:39:58.241693 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell0-3d99-account-create-update-kqb94"] Dec 03 11:39:58 crc kubenswrapper[5052]: I1203 11:39:58.294659 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/a53c0da6-8b05-40fd-90c7-78cfb701780a-operator-scripts\") pod \"nova-cell0-3d99-account-create-update-kqb94\" (UID: \"a53c0da6-8b05-40fd-90c7-78cfb701780a\") " pod="openstack-kuttl-tests/nova-cell0-3d99-account-create-update-kqb94" Dec 03 11:39:58 crc kubenswrapper[5052]: I1203 11:39:58.294790 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/26a22bd0-90b3-4fe9-8029-a56c6c7bcdfd-operator-scripts\") pod \"nova-api-73a4-account-create-update-kfmjt\" (UID: \"26a22bd0-90b3-4fe9-8029-a56c6c7bcdfd\") " pod="openstack-kuttl-tests/nova-api-73a4-account-create-update-kfmjt" Dec 03 11:39:58 crc kubenswrapper[5052]: I1203 11:39:58.294824 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/6410c857-4e7d-486c-be0a-6734565014d4-operator-scripts\") pod \"nova-cell0-db-create-w672x\" (UID: \"6410c857-4e7d-486c-be0a-6734565014d4\") " pod="openstack-kuttl-tests/nova-cell0-db-create-w672x" Dec 03 11:39:58 crc kubenswrapper[5052]: I1203 11:39:58.294854 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-xg85d\" (UniqueName: \"kubernetes.io/projected/6410c857-4e7d-486c-be0a-6734565014d4-kube-api-access-xg85d\") pod \"nova-cell0-db-create-w672x\" (UID: \"6410c857-4e7d-486c-be0a-6734565014d4\") " pod="openstack-kuttl-tests/nova-cell0-db-create-w672x" Dec 03 11:39:58 crc kubenswrapper[5052]: I1203 11:39:58.294878 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-lwqbw\" (UniqueName: \"kubernetes.io/projected/26a22bd0-90b3-4fe9-8029-a56c6c7bcdfd-kube-api-access-lwqbw\") pod \"nova-api-73a4-account-create-update-kfmjt\" (UID: \"26a22bd0-90b3-4fe9-8029-a56c6c7bcdfd\") " pod="openstack-kuttl-tests/nova-api-73a4-account-create-update-kfmjt" Dec 03 11:39:58 crc kubenswrapper[5052]: I1203 11:39:58.294967 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/066f70dd-4aa5-4b30-b913-2285c642134e-operator-scripts\") pod \"nova-cell1-db-create-4mc7v\" (UID: \"066f70dd-4aa5-4b30-b913-2285c642134e\") " pod="openstack-kuttl-tests/nova-cell1-db-create-4mc7v" Dec 03 11:39:58 crc kubenswrapper[5052]: I1203 11:39:58.295092 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5qrnn\" (UniqueName: \"kubernetes.io/projected/a53c0da6-8b05-40fd-90c7-78cfb701780a-kube-api-access-5qrnn\") pod \"nova-cell0-3d99-account-create-update-kqb94\" (UID: \"a53c0da6-8b05-40fd-90c7-78cfb701780a\") " pod="openstack-kuttl-tests/nova-cell0-3d99-account-create-update-kqb94" Dec 03 11:39:58 crc kubenswrapper[5052]: I1203 11:39:58.295130 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6v4fc\" (UniqueName: \"kubernetes.io/projected/066f70dd-4aa5-4b30-b913-2285c642134e-kube-api-access-6v4fc\") pod \"nova-cell1-db-create-4mc7v\" (UID: \"066f70dd-4aa5-4b30-b913-2285c642134e\") " pod="openstack-kuttl-tests/nova-cell1-db-create-4mc7v" Dec 03 11:39:58 crc kubenswrapper[5052]: I1203 11:39:58.296964 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/26a22bd0-90b3-4fe9-8029-a56c6c7bcdfd-operator-scripts\") pod \"nova-api-73a4-account-create-update-kfmjt\" (UID: \"26a22bd0-90b3-4fe9-8029-a56c6c7bcdfd\") " pod="openstack-kuttl-tests/nova-api-73a4-account-create-update-kfmjt" Dec 03 11:39:58 crc kubenswrapper[5052]: I1203 11:39:58.297210 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/6410c857-4e7d-486c-be0a-6734565014d4-operator-scripts\") pod \"nova-cell0-db-create-w672x\" (UID: \"6410c857-4e7d-486c-be0a-6734565014d4\") " pod="openstack-kuttl-tests/nova-cell0-db-create-w672x" Dec 03 11:39:58 crc kubenswrapper[5052]: I1203 11:39:58.312449 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-xg85d\" (UniqueName: \"kubernetes.io/projected/6410c857-4e7d-486c-be0a-6734565014d4-kube-api-access-xg85d\") pod \"nova-cell0-db-create-w672x\" (UID: \"6410c857-4e7d-486c-be0a-6734565014d4\") " pod="openstack-kuttl-tests/nova-cell0-db-create-w672x" Dec 03 11:39:58 crc kubenswrapper[5052]: I1203 11:39:58.315192 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-db-create-w672x" Dec 03 11:39:58 crc kubenswrapper[5052]: I1203 11:39:58.316167 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-lwqbw\" (UniqueName: \"kubernetes.io/projected/26a22bd0-90b3-4fe9-8029-a56c6c7bcdfd-kube-api-access-lwqbw\") pod \"nova-api-73a4-account-create-update-kfmjt\" (UID: \"26a22bd0-90b3-4fe9-8029-a56c6c7bcdfd\") " pod="openstack-kuttl-tests/nova-api-73a4-account-create-update-kfmjt" Dec 03 11:39:58 crc kubenswrapper[5052]: I1203 11:39:58.327733 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-73a4-account-create-update-kfmjt" Dec 03 11:39:58 crc kubenswrapper[5052]: I1203 11:39:58.397473 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-5qrnn\" (UniqueName: \"kubernetes.io/projected/a53c0da6-8b05-40fd-90c7-78cfb701780a-kube-api-access-5qrnn\") pod \"nova-cell0-3d99-account-create-update-kqb94\" (UID: \"a53c0da6-8b05-40fd-90c7-78cfb701780a\") " pod="openstack-kuttl-tests/nova-cell0-3d99-account-create-update-kqb94" Dec 03 11:39:58 crc kubenswrapper[5052]: I1203 11:39:58.397810 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-6v4fc\" (UniqueName: \"kubernetes.io/projected/066f70dd-4aa5-4b30-b913-2285c642134e-kube-api-access-6v4fc\") pod \"nova-cell1-db-create-4mc7v\" (UID: \"066f70dd-4aa5-4b30-b913-2285c642134e\") " pod="openstack-kuttl-tests/nova-cell1-db-create-4mc7v" Dec 03 11:39:58 crc kubenswrapper[5052]: I1203 11:39:58.397961 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/a53c0da6-8b05-40fd-90c7-78cfb701780a-operator-scripts\") pod \"nova-cell0-3d99-account-create-update-kqb94\" (UID: \"a53c0da6-8b05-40fd-90c7-78cfb701780a\") " pod="openstack-kuttl-tests/nova-cell0-3d99-account-create-update-kqb94" Dec 03 11:39:58 crc kubenswrapper[5052]: I1203 11:39:58.398202 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/066f70dd-4aa5-4b30-b913-2285c642134e-operator-scripts\") pod \"nova-cell1-db-create-4mc7v\" (UID: \"066f70dd-4aa5-4b30-b913-2285c642134e\") " pod="openstack-kuttl-tests/nova-cell1-db-create-4mc7v" Dec 03 11:39:58 crc kubenswrapper[5052]: I1203 11:39:58.398952 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/a53c0da6-8b05-40fd-90c7-78cfb701780a-operator-scripts\") pod \"nova-cell0-3d99-account-create-update-kqb94\" (UID: \"a53c0da6-8b05-40fd-90c7-78cfb701780a\") " pod="openstack-kuttl-tests/nova-cell0-3d99-account-create-update-kqb94" Dec 03 11:39:58 crc kubenswrapper[5052]: I1203 11:39:58.399087 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/066f70dd-4aa5-4b30-b913-2285c642134e-operator-scripts\") pod \"nova-cell1-db-create-4mc7v\" (UID: \"066f70dd-4aa5-4b30-b913-2285c642134e\") " pod="openstack-kuttl-tests/nova-cell1-db-create-4mc7v" Dec 03 11:39:58 crc kubenswrapper[5052]: I1203 11:39:58.406685 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell1-cdd4-account-create-update-lbpqk"] Dec 03 11:39:58 crc kubenswrapper[5052]: I1203 11:39:58.408828 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-cdd4-account-create-update-lbpqk" Dec 03 11:39:58 crc kubenswrapper[5052]: I1203 11:39:58.412958 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell1-db-secret" Dec 03 11:39:58 crc kubenswrapper[5052]: I1203 11:39:58.420590 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-cdd4-account-create-update-lbpqk"] Dec 03 11:39:58 crc kubenswrapper[5052]: I1203 11:39:58.423281 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-5qrnn\" (UniqueName: \"kubernetes.io/projected/a53c0da6-8b05-40fd-90c7-78cfb701780a-kube-api-access-5qrnn\") pod \"nova-cell0-3d99-account-create-update-kqb94\" (UID: \"a53c0da6-8b05-40fd-90c7-78cfb701780a\") " pod="openstack-kuttl-tests/nova-cell0-3d99-account-create-update-kqb94" Dec 03 11:39:58 crc kubenswrapper[5052]: I1203 11:39:58.430046 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-6v4fc\" (UniqueName: \"kubernetes.io/projected/066f70dd-4aa5-4b30-b913-2285c642134e-kube-api-access-6v4fc\") pod \"nova-cell1-db-create-4mc7v\" (UID: \"066f70dd-4aa5-4b30-b913-2285c642134e\") " pod="openstack-kuttl-tests/nova-cell1-db-create-4mc7v" Dec 03 11:39:58 crc kubenswrapper[5052]: I1203 11:39:58.499119 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/5f9bdf8c-c97c-477e-bf48-73fe6ddae983-operator-scripts\") pod \"nova-cell1-cdd4-account-create-update-lbpqk\" (UID: \"5f9bdf8c-c97c-477e-bf48-73fe6ddae983\") " pod="openstack-kuttl-tests/nova-cell1-cdd4-account-create-update-lbpqk" Dec 03 11:39:58 crc kubenswrapper[5052]: I1203 11:39:58.499158 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fvwzw\" (UniqueName: \"kubernetes.io/projected/5f9bdf8c-c97c-477e-bf48-73fe6ddae983-kube-api-access-fvwzw\") pod \"nova-cell1-cdd4-account-create-update-lbpqk\" (UID: \"5f9bdf8c-c97c-477e-bf48-73fe6ddae983\") " pod="openstack-kuttl-tests/nova-cell1-cdd4-account-create-update-lbpqk" Dec 03 11:39:58 crc kubenswrapper[5052]: I1203 11:39:58.591266 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-db-create-4mc7v" Dec 03 11:39:58 crc kubenswrapper[5052]: I1203 11:39:58.601119 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/5f9bdf8c-c97c-477e-bf48-73fe6ddae983-operator-scripts\") pod \"nova-cell1-cdd4-account-create-update-lbpqk\" (UID: \"5f9bdf8c-c97c-477e-bf48-73fe6ddae983\") " pod="openstack-kuttl-tests/nova-cell1-cdd4-account-create-update-lbpqk" Dec 03 11:39:58 crc kubenswrapper[5052]: I1203 11:39:58.601158 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-fvwzw\" (UniqueName: \"kubernetes.io/projected/5f9bdf8c-c97c-477e-bf48-73fe6ddae983-kube-api-access-fvwzw\") pod \"nova-cell1-cdd4-account-create-update-lbpqk\" (UID: \"5f9bdf8c-c97c-477e-bf48-73fe6ddae983\") " pod="openstack-kuttl-tests/nova-cell1-cdd4-account-create-update-lbpqk" Dec 03 11:39:58 crc kubenswrapper[5052]: I1203 11:39:58.602048 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/5f9bdf8c-c97c-477e-bf48-73fe6ddae983-operator-scripts\") pod \"nova-cell1-cdd4-account-create-update-lbpqk\" (UID: \"5f9bdf8c-c97c-477e-bf48-73fe6ddae983\") " pod="openstack-kuttl-tests/nova-cell1-cdd4-account-create-update-lbpqk" Dec 03 11:39:58 crc kubenswrapper[5052]: I1203 11:39:58.604094 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-3d99-account-create-update-kqb94" Dec 03 11:39:58 crc kubenswrapper[5052]: I1203 11:39:58.617742 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-fvwzw\" (UniqueName: \"kubernetes.io/projected/5f9bdf8c-c97c-477e-bf48-73fe6ddae983-kube-api-access-fvwzw\") pod \"nova-cell1-cdd4-account-create-update-lbpqk\" (UID: \"5f9bdf8c-c97c-477e-bf48-73fe6ddae983\") " pod="openstack-kuttl-tests/nova-cell1-cdd4-account-create-update-lbpqk" Dec 03 11:39:58 crc kubenswrapper[5052]: I1203 11:39:58.696670 5052 generic.go:334] "Generic (PLEG): container finished" podID="8bf67244-3ae2-4c00-a15d-4c4180d5f835" containerID="dcd58ef3b45066b82652446bcbd875e2abf4758e96cd73b62fededb0469b531e" exitCode=0 Dec 03 11:39:58 crc kubenswrapper[5052]: I1203 11:39:58.696717 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"8bf67244-3ae2-4c00-a15d-4c4180d5f835","Type":"ContainerDied","Data":"dcd58ef3b45066b82652446bcbd875e2abf4758e96cd73b62fededb0469b531e"} Dec 03 11:39:58 crc kubenswrapper[5052]: I1203 11:39:58.872181 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-cdd4-account-create-update-lbpqk" Dec 03 11:39:59 crc kubenswrapper[5052]: I1203 11:39:59.716977 5052 generic.go:334] "Generic (PLEG): container finished" podID="70e2b274-dd4d-409a-8d45-a09af330f7a3" containerID="466f8e75a4f4613fdb58e65fc3b11d64b99b313618c5c45ebe14444fc70dbacf" exitCode=0 Dec 03 11:39:59 crc kubenswrapper[5052]: I1203 11:39:59.717054 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"70e2b274-dd4d-409a-8d45-a09af330f7a3","Type":"ContainerDied","Data":"466f8e75a4f4613fdb58e65fc3b11d64b99b313618c5c45ebe14444fc70dbacf"} Dec 03 11:39:59 crc kubenswrapper[5052]: I1203 11:39:59.720553 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"502cb03c-d1a5-4dc7-a040-0571a4695379","Type":"ContainerDied","Data":"3c6d3ddfe8759dec5d3030b3c1b64bcbc9f4228dfeaac36309751c177de0b102"} Dec 03 11:39:59 crc kubenswrapper[5052]: I1203 11:39:59.721362 5052 generic.go:334] "Generic (PLEG): container finished" podID="502cb03c-d1a5-4dc7-a040-0571a4695379" containerID="3c6d3ddfe8759dec5d3030b3c1b64bcbc9f4228dfeaac36309751c177de0b102" exitCode=0 Dec 03 11:39:59 crc kubenswrapper[5052]: I1203 11:39:59.950906 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/ceilometer-0" podUID="70e2b274-dd4d-409a-8d45-a09af330f7a3" containerName="proxy-httpd" probeResult="failure" output="Get \"http://10.217.0.141:3000/\": dial tcp 10.217.0.141:3000: connect: connection refused" Dec 03 11:40:00 crc kubenswrapper[5052]: I1203 11:40:00.556052 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-api-65968d9848-mntb6" Dec 03 11:40:00 crc kubenswrapper[5052]: I1203 11:40:00.643216 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w2dbz\" (UniqueName: \"kubernetes.io/projected/ef47ba52-772d-4732-bd15-e7513e626be9-kube-api-access-w2dbz\") pod \"ef47ba52-772d-4732-bd15-e7513e626be9\" (UID: \"ef47ba52-772d-4732-bd15-e7513e626be9\") " Dec 03 11:40:00 crc kubenswrapper[5052]: I1203 11:40:00.643278 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/ef47ba52-772d-4732-bd15-e7513e626be9-logs\") pod \"ef47ba52-772d-4732-bd15-e7513e626be9\" (UID: \"ef47ba52-772d-4732-bd15-e7513e626be9\") " Dec 03 11:40:00 crc kubenswrapper[5052]: I1203 11:40:00.643347 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/ef47ba52-772d-4732-bd15-e7513e626be9-config-data-custom\") pod \"ef47ba52-772d-4732-bd15-e7513e626be9\" (UID: \"ef47ba52-772d-4732-bd15-e7513e626be9\") " Dec 03 11:40:00 crc kubenswrapper[5052]: I1203 11:40:00.643378 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ef47ba52-772d-4732-bd15-e7513e626be9-combined-ca-bundle\") pod \"ef47ba52-772d-4732-bd15-e7513e626be9\" (UID: \"ef47ba52-772d-4732-bd15-e7513e626be9\") " Dec 03 11:40:00 crc kubenswrapper[5052]: I1203 11:40:00.643442 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/ef47ba52-772d-4732-bd15-e7513e626be9-config-data\") pod \"ef47ba52-772d-4732-bd15-e7513e626be9\" (UID: \"ef47ba52-772d-4732-bd15-e7513e626be9\") " Dec 03 11:40:00 crc kubenswrapper[5052]: I1203 11:40:00.645136 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ef47ba52-772d-4732-bd15-e7513e626be9-logs" (OuterVolumeSpecName: "logs") pod "ef47ba52-772d-4732-bd15-e7513e626be9" (UID: "ef47ba52-772d-4732-bd15-e7513e626be9"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:40:00 crc kubenswrapper[5052]: I1203 11:40:00.652869 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ef47ba52-772d-4732-bd15-e7513e626be9-kube-api-access-w2dbz" (OuterVolumeSpecName: "kube-api-access-w2dbz") pod "ef47ba52-772d-4732-bd15-e7513e626be9" (UID: "ef47ba52-772d-4732-bd15-e7513e626be9"). InnerVolumeSpecName "kube-api-access-w2dbz". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:40:00 crc kubenswrapper[5052]: I1203 11:40:00.657443 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ef47ba52-772d-4732-bd15-e7513e626be9-config-data-custom" (OuterVolumeSpecName: "config-data-custom") pod "ef47ba52-772d-4732-bd15-e7513e626be9" (UID: "ef47ba52-772d-4732-bd15-e7513e626be9"). InnerVolumeSpecName "config-data-custom". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:40:00 crc kubenswrapper[5052]: I1203 11:40:00.692920 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ef47ba52-772d-4732-bd15-e7513e626be9-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "ef47ba52-772d-4732-bd15-e7513e626be9" (UID: "ef47ba52-772d-4732-bd15-e7513e626be9"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:40:00 crc kubenswrapper[5052]: I1203 11:40:00.749828 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ef47ba52-772d-4732-bd15-e7513e626be9-config-data" (OuterVolumeSpecName: "config-data") pod "ef47ba52-772d-4732-bd15-e7513e626be9" (UID: "ef47ba52-772d-4732-bd15-e7513e626be9"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:40:00 crc kubenswrapper[5052]: I1203 11:40:00.749928 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-w2dbz\" (UniqueName: \"kubernetes.io/projected/ef47ba52-772d-4732-bd15-e7513e626be9-kube-api-access-w2dbz\") on node \"crc\" DevicePath \"\"" Dec 03 11:40:00 crc kubenswrapper[5052]: I1203 11:40:00.750195 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/ef47ba52-772d-4732-bd15-e7513e626be9-logs\") on node \"crc\" DevicePath \"\"" Dec 03 11:40:00 crc kubenswrapper[5052]: I1203 11:40:00.750207 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/ef47ba52-772d-4732-bd15-e7513e626be9-config-data-custom\") on node \"crc\" DevicePath \"\"" Dec 03 11:40:00 crc kubenswrapper[5052]: I1203 11:40:00.750221 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ef47ba52-772d-4732-bd15-e7513e626be9-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:40:00 crc kubenswrapper[5052]: I1203 11:40:00.768565 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstackclient" event={"ID":"722e5457-ae47-40b8-816f-8eab5b7a4cb0","Type":"ContainerStarted","Data":"7f7814a46f2c2a853ddc0d2142a5cfb0bf53aa603be45491209a66cd585a4f18"} Dec 03 11:40:00 crc kubenswrapper[5052]: I1203 11:40:00.783865 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-65968d9848-mntb6" event={"ID":"ef47ba52-772d-4732-bd15-e7513e626be9","Type":"ContainerDied","Data":"2339c61a18e31e2883d1240a7db9fbdee44126d5621c2d1d1866f7ef3df0220b"} Dec 03 11:40:00 crc kubenswrapper[5052]: I1203 11:40:00.783905 5052 scope.go:117] "RemoveContainer" containerID="f531b9f35a146882772e5bb658e2970143bf9db9263912e975397ff80f1c2cda" Dec 03 11:40:00 crc kubenswrapper[5052]: I1203 11:40:00.784026 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-api-65968d9848-mntb6" Dec 03 11:40:00 crc kubenswrapper[5052]: I1203 11:40:00.804974 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:40:00 crc kubenswrapper[5052]: I1203 11:40:00.808111 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/openstackclient" podStartSLOduration=2.094644355 podStartE2EDuration="14.807228266s" podCreationTimestamp="2025-12-03 11:39:46 +0000 UTC" firstStartedPulling="2025-12-03 11:39:47.598794671 +0000 UTC m=+1178.187036435" lastFinishedPulling="2025-12-03 11:40:00.311378582 +0000 UTC m=+1190.899620346" observedRunningTime="2025-12-03 11:40:00.803415686 +0000 UTC m=+1191.391657450" watchObservedRunningTime="2025-12-03 11:40:00.807228266 +0000 UTC m=+1191.395470030" Dec 03 11:40:00 crc kubenswrapper[5052]: I1203 11:40:00.841417 5052 scope.go:117] "RemoveContainer" containerID="a00718351b51b3492d282952f384864b842002d371827646c87c60e86863c87e" Dec 03 11:40:00 crc kubenswrapper[5052]: I1203 11:40:00.854670 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/502cb03c-d1a5-4dc7-a040-0571a4695379-httpd-run\") pod \"502cb03c-d1a5-4dc7-a040-0571a4695379\" (UID: \"502cb03c-d1a5-4dc7-a040-0571a4695379\") " Dec 03 11:40:00 crc kubenswrapper[5052]: I1203 11:40:00.856913 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/502cb03c-d1a5-4dc7-a040-0571a4695379-httpd-run" (OuterVolumeSpecName: "httpd-run") pod "502cb03c-d1a5-4dc7-a040-0571a4695379" (UID: "502cb03c-d1a5-4dc7-a040-0571a4695379"). InnerVolumeSpecName "httpd-run". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:40:00 crc kubenswrapper[5052]: I1203 11:40:00.857824 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/502cb03c-d1a5-4dc7-a040-0571a4695379-scripts\") pod \"502cb03c-d1a5-4dc7-a040-0571a4695379\" (UID: \"502cb03c-d1a5-4dc7-a040-0571a4695379\") " Dec 03 11:40:00 crc kubenswrapper[5052]: I1203 11:40:00.857978 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"glance\" (UniqueName: \"kubernetes.io/local-volume/local-storage04-crc\") pod \"502cb03c-d1a5-4dc7-a040-0571a4695379\" (UID: \"502cb03c-d1a5-4dc7-a040-0571a4695379\") " Dec 03 11:40:00 crc kubenswrapper[5052]: I1203 11:40:00.858064 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6zvv5\" (UniqueName: \"kubernetes.io/projected/502cb03c-d1a5-4dc7-a040-0571a4695379-kube-api-access-6zvv5\") pod \"502cb03c-d1a5-4dc7-a040-0571a4695379\" (UID: \"502cb03c-d1a5-4dc7-a040-0571a4695379\") " Dec 03 11:40:00 crc kubenswrapper[5052]: I1203 11:40:00.858099 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/502cb03c-d1a5-4dc7-a040-0571a4695379-logs\") pod \"502cb03c-d1a5-4dc7-a040-0571a4695379\" (UID: \"502cb03c-d1a5-4dc7-a040-0571a4695379\") " Dec 03 11:40:00 crc kubenswrapper[5052]: I1203 11:40:00.858225 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/502cb03c-d1a5-4dc7-a040-0571a4695379-config-data\") pod \"502cb03c-d1a5-4dc7-a040-0571a4695379\" (UID: \"502cb03c-d1a5-4dc7-a040-0571a4695379\") " Dec 03 11:40:00 crc kubenswrapper[5052]: I1203 11:40:00.858297 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/502cb03c-d1a5-4dc7-a040-0571a4695379-internal-tls-certs\") pod \"502cb03c-d1a5-4dc7-a040-0571a4695379\" (UID: \"502cb03c-d1a5-4dc7-a040-0571a4695379\") " Dec 03 11:40:00 crc kubenswrapper[5052]: I1203 11:40:00.861119 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/502cb03c-d1a5-4dc7-a040-0571a4695379-logs" (OuterVolumeSpecName: "logs") pod "502cb03c-d1a5-4dc7-a040-0571a4695379" (UID: "502cb03c-d1a5-4dc7-a040-0571a4695379"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:40:00 crc kubenswrapper[5052]: I1203 11:40:00.861190 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/502cb03c-d1a5-4dc7-a040-0571a4695379-combined-ca-bundle\") pod \"502cb03c-d1a5-4dc7-a040-0571a4695379\" (UID: \"502cb03c-d1a5-4dc7-a040-0571a4695379\") " Dec 03 11:40:00 crc kubenswrapper[5052]: I1203 11:40:00.863779 5052 reconciler_common.go:293] "Volume detached for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/502cb03c-d1a5-4dc7-a040-0571a4695379-httpd-run\") on node \"crc\" DevicePath \"\"" Dec 03 11:40:00 crc kubenswrapper[5052]: I1203 11:40:00.863815 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/502cb03c-d1a5-4dc7-a040-0571a4695379-logs\") on node \"crc\" DevicePath \"\"" Dec 03 11:40:00 crc kubenswrapper[5052]: I1203 11:40:00.863825 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/ef47ba52-772d-4732-bd15-e7513e626be9-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:40:00 crc kubenswrapper[5052]: I1203 11:40:00.864937 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:40:00 crc kubenswrapper[5052]: I1203 11:40:00.867181 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/502cb03c-d1a5-4dc7-a040-0571a4695379-scripts" (OuterVolumeSpecName: "scripts") pod "502cb03c-d1a5-4dc7-a040-0571a4695379" (UID: "502cb03c-d1a5-4dc7-a040-0571a4695379"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:40:00 crc kubenswrapper[5052]: I1203 11:40:00.868758 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/502cb03c-d1a5-4dc7-a040-0571a4695379-kube-api-access-6zvv5" (OuterVolumeSpecName: "kube-api-access-6zvv5") pod "502cb03c-d1a5-4dc7-a040-0571a4695379" (UID: "502cb03c-d1a5-4dc7-a040-0571a4695379"). InnerVolumeSpecName "kube-api-access-6zvv5". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:40:00 crc kubenswrapper[5052]: I1203 11:40:00.872153 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-api-65968d9848-mntb6"] Dec 03 11:40:00 crc kubenswrapper[5052]: I1203 11:40:00.877058 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:40:00 crc kubenswrapper[5052]: I1203 11:40:00.878945 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage04-crc" (OuterVolumeSpecName: "glance") pod "502cb03c-d1a5-4dc7-a040-0571a4695379" (UID: "502cb03c-d1a5-4dc7-a040-0571a4695379"). InnerVolumeSpecName "local-storage04-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 11:40:00 crc kubenswrapper[5052]: I1203 11:40:00.882773 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/barbican-api-65968d9848-mntb6"] Dec 03 11:40:00 crc kubenswrapper[5052]: I1203 11:40:00.913662 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/502cb03c-d1a5-4dc7-a040-0571a4695379-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "502cb03c-d1a5-4dc7-a040-0571a4695379" (UID: "502cb03c-d1a5-4dc7-a040-0571a4695379"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:40:00 crc kubenswrapper[5052]: I1203 11:40:00.940474 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/502cb03c-d1a5-4dc7-a040-0571a4695379-config-data" (OuterVolumeSpecName: "config-data") pod "502cb03c-d1a5-4dc7-a040-0571a4695379" (UID: "502cb03c-d1a5-4dc7-a040-0571a4695379"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:40:00 crc kubenswrapper[5052]: I1203 11:40:00.948785 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/502cb03c-d1a5-4dc7-a040-0571a4695379-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "502cb03c-d1a5-4dc7-a040-0571a4695379" (UID: "502cb03c-d1a5-4dc7-a040-0571a4695379"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:40:00 crc kubenswrapper[5052]: I1203 11:40:00.964401 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/8bf67244-3ae2-4c00-a15d-4c4180d5f835-public-tls-certs\") pod \"8bf67244-3ae2-4c00-a15d-4c4180d5f835\" (UID: \"8bf67244-3ae2-4c00-a15d-4c4180d5f835\") " Dec 03 11:40:00 crc kubenswrapper[5052]: I1203 11:40:00.964441 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/8bf67244-3ae2-4c00-a15d-4c4180d5f835-scripts\") pod \"8bf67244-3ae2-4c00-a15d-4c4180d5f835\" (UID: \"8bf67244-3ae2-4c00-a15d-4c4180d5f835\") " Dec 03 11:40:00 crc kubenswrapper[5052]: I1203 11:40:00.964464 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-728bk\" (UniqueName: \"kubernetes.io/projected/70e2b274-dd4d-409a-8d45-a09af330f7a3-kube-api-access-728bk\") pod \"70e2b274-dd4d-409a-8d45-a09af330f7a3\" (UID: \"70e2b274-dd4d-409a-8d45-a09af330f7a3\") " Dec 03 11:40:00 crc kubenswrapper[5052]: I1203 11:40:00.964490 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cp4l6\" (UniqueName: \"kubernetes.io/projected/8bf67244-3ae2-4c00-a15d-4c4180d5f835-kube-api-access-cp4l6\") pod \"8bf67244-3ae2-4c00-a15d-4c4180d5f835\" (UID: \"8bf67244-3ae2-4c00-a15d-4c4180d5f835\") " Dec 03 11:40:00 crc kubenswrapper[5052]: I1203 11:40:00.964606 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/70e2b274-dd4d-409a-8d45-a09af330f7a3-scripts\") pod \"70e2b274-dd4d-409a-8d45-a09af330f7a3\" (UID: \"70e2b274-dd4d-409a-8d45-a09af330f7a3\") " Dec 03 11:40:00 crc kubenswrapper[5052]: I1203 11:40:00.964637 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/70e2b274-dd4d-409a-8d45-a09af330f7a3-sg-core-conf-yaml\") pod \"70e2b274-dd4d-409a-8d45-a09af330f7a3\" (UID: \"70e2b274-dd4d-409a-8d45-a09af330f7a3\") " Dec 03 11:40:00 crc kubenswrapper[5052]: I1203 11:40:00.964660 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/8bf67244-3ae2-4c00-a15d-4c4180d5f835-config-data\") pod \"8bf67244-3ae2-4c00-a15d-4c4180d5f835\" (UID: \"8bf67244-3ae2-4c00-a15d-4c4180d5f835\") " Dec 03 11:40:00 crc kubenswrapper[5052]: I1203 11:40:00.964681 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/8bf67244-3ae2-4c00-a15d-4c4180d5f835-logs\") pod \"8bf67244-3ae2-4c00-a15d-4c4180d5f835\" (UID: \"8bf67244-3ae2-4c00-a15d-4c4180d5f835\") " Dec 03 11:40:00 crc kubenswrapper[5052]: I1203 11:40:00.964711 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/70e2b274-dd4d-409a-8d45-a09af330f7a3-combined-ca-bundle\") pod \"70e2b274-dd4d-409a-8d45-a09af330f7a3\" (UID: \"70e2b274-dd4d-409a-8d45-a09af330f7a3\") " Dec 03 11:40:00 crc kubenswrapper[5052]: I1203 11:40:00.964734 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/70e2b274-dd4d-409a-8d45-a09af330f7a3-config-data\") pod \"70e2b274-dd4d-409a-8d45-a09af330f7a3\" (UID: \"70e2b274-dd4d-409a-8d45-a09af330f7a3\") " Dec 03 11:40:00 crc kubenswrapper[5052]: I1203 11:40:00.964797 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/70e2b274-dd4d-409a-8d45-a09af330f7a3-log-httpd\") pod \"70e2b274-dd4d-409a-8d45-a09af330f7a3\" (UID: \"70e2b274-dd4d-409a-8d45-a09af330f7a3\") " Dec 03 11:40:00 crc kubenswrapper[5052]: I1203 11:40:00.964833 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/70e2b274-dd4d-409a-8d45-a09af330f7a3-run-httpd\") pod \"70e2b274-dd4d-409a-8d45-a09af330f7a3\" (UID: \"70e2b274-dd4d-409a-8d45-a09af330f7a3\") " Dec 03 11:40:00 crc kubenswrapper[5052]: I1203 11:40:00.964859 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"glance\" (UniqueName: \"kubernetes.io/local-volume/local-storage15-crc\") pod \"8bf67244-3ae2-4c00-a15d-4c4180d5f835\" (UID: \"8bf67244-3ae2-4c00-a15d-4c4180d5f835\") " Dec 03 11:40:00 crc kubenswrapper[5052]: I1203 11:40:00.964905 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/8bf67244-3ae2-4c00-a15d-4c4180d5f835-httpd-run\") pod \"8bf67244-3ae2-4c00-a15d-4c4180d5f835\" (UID: \"8bf67244-3ae2-4c00-a15d-4c4180d5f835\") " Dec 03 11:40:00 crc kubenswrapper[5052]: I1203 11:40:00.964920 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8bf67244-3ae2-4c00-a15d-4c4180d5f835-combined-ca-bundle\") pod \"8bf67244-3ae2-4c00-a15d-4c4180d5f835\" (UID: \"8bf67244-3ae2-4c00-a15d-4c4180d5f835\") " Dec 03 11:40:00 crc kubenswrapper[5052]: I1203 11:40:00.965306 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage04-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage04-crc\") on node \"crc\" " Dec 03 11:40:00 crc kubenswrapper[5052]: I1203 11:40:00.965322 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-6zvv5\" (UniqueName: \"kubernetes.io/projected/502cb03c-d1a5-4dc7-a040-0571a4695379-kube-api-access-6zvv5\") on node \"crc\" DevicePath \"\"" Dec 03 11:40:00 crc kubenswrapper[5052]: I1203 11:40:00.965333 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/502cb03c-d1a5-4dc7-a040-0571a4695379-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:40:00 crc kubenswrapper[5052]: I1203 11:40:00.965342 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/502cb03c-d1a5-4dc7-a040-0571a4695379-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 11:40:00 crc kubenswrapper[5052]: I1203 11:40:00.965352 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/502cb03c-d1a5-4dc7-a040-0571a4695379-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:40:00 crc kubenswrapper[5052]: I1203 11:40:00.965369 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/502cb03c-d1a5-4dc7-a040-0571a4695379-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:40:00 crc kubenswrapper[5052]: I1203 11:40:00.966162 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8bf67244-3ae2-4c00-a15d-4c4180d5f835-logs" (OuterVolumeSpecName: "logs") pod "8bf67244-3ae2-4c00-a15d-4c4180d5f835" (UID: "8bf67244-3ae2-4c00-a15d-4c4180d5f835"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:40:00 crc kubenswrapper[5052]: I1203 11:40:00.967863 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/70e2b274-dd4d-409a-8d45-a09af330f7a3-run-httpd" (OuterVolumeSpecName: "run-httpd") pod "70e2b274-dd4d-409a-8d45-a09af330f7a3" (UID: "70e2b274-dd4d-409a-8d45-a09af330f7a3"). InnerVolumeSpecName "run-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:40:00 crc kubenswrapper[5052]: I1203 11:40:00.968932 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8bf67244-3ae2-4c00-a15d-4c4180d5f835-httpd-run" (OuterVolumeSpecName: "httpd-run") pod "8bf67244-3ae2-4c00-a15d-4c4180d5f835" (UID: "8bf67244-3ae2-4c00-a15d-4c4180d5f835"). InnerVolumeSpecName "httpd-run". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:40:00 crc kubenswrapper[5052]: I1203 11:40:00.969439 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/70e2b274-dd4d-409a-8d45-a09af330f7a3-log-httpd" (OuterVolumeSpecName: "log-httpd") pod "70e2b274-dd4d-409a-8d45-a09af330f7a3" (UID: "70e2b274-dd4d-409a-8d45-a09af330f7a3"). InnerVolumeSpecName "log-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:40:00 crc kubenswrapper[5052]: I1203 11:40:00.972268 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8bf67244-3ae2-4c00-a15d-4c4180d5f835-kube-api-access-cp4l6" (OuterVolumeSpecName: "kube-api-access-cp4l6") pod "8bf67244-3ae2-4c00-a15d-4c4180d5f835" (UID: "8bf67244-3ae2-4c00-a15d-4c4180d5f835"). InnerVolumeSpecName "kube-api-access-cp4l6". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:40:00 crc kubenswrapper[5052]: I1203 11:40:00.973211 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/70e2b274-dd4d-409a-8d45-a09af330f7a3-kube-api-access-728bk" (OuterVolumeSpecName: "kube-api-access-728bk") pod "70e2b274-dd4d-409a-8d45-a09af330f7a3" (UID: "70e2b274-dd4d-409a-8d45-a09af330f7a3"). InnerVolumeSpecName "kube-api-access-728bk". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:40:00 crc kubenswrapper[5052]: I1203 11:40:00.973689 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/70e2b274-dd4d-409a-8d45-a09af330f7a3-scripts" (OuterVolumeSpecName: "scripts") pod "70e2b274-dd4d-409a-8d45-a09af330f7a3" (UID: "70e2b274-dd4d-409a-8d45-a09af330f7a3"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:40:00 crc kubenswrapper[5052]: I1203 11:40:00.975674 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8bf67244-3ae2-4c00-a15d-4c4180d5f835-scripts" (OuterVolumeSpecName: "scripts") pod "8bf67244-3ae2-4c00-a15d-4c4180d5f835" (UID: "8bf67244-3ae2-4c00-a15d-4c4180d5f835"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:40:00 crc kubenswrapper[5052]: I1203 11:40:00.975792 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage15-crc" (OuterVolumeSpecName: "glance") pod "8bf67244-3ae2-4c00-a15d-4c4180d5f835" (UID: "8bf67244-3ae2-4c00-a15d-4c4180d5f835"). InnerVolumeSpecName "local-storage15-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 11:40:00 crc kubenswrapper[5052]: I1203 11:40:00.995761 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage04-crc" (UniqueName: "kubernetes.io/local-volume/local-storage04-crc") on node "crc" Dec 03 11:40:00 crc kubenswrapper[5052]: I1203 11:40:00.999072 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/70e2b274-dd4d-409a-8d45-a09af330f7a3-sg-core-conf-yaml" (OuterVolumeSpecName: "sg-core-conf-yaml") pod "70e2b274-dd4d-409a-8d45-a09af330f7a3" (UID: "70e2b274-dd4d-409a-8d45-a09af330f7a3"). InnerVolumeSpecName "sg-core-conf-yaml". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:40:01 crc kubenswrapper[5052]: I1203 11:40:01.034965 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8bf67244-3ae2-4c00-a15d-4c4180d5f835-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "8bf67244-3ae2-4c00-a15d-4c4180d5f835" (UID: "8bf67244-3ae2-4c00-a15d-4c4180d5f835"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:40:01 crc kubenswrapper[5052]: I1203 11:40:01.060809 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8bf67244-3ae2-4c00-a15d-4c4180d5f835-config-data" (OuterVolumeSpecName: "config-data") pod "8bf67244-3ae2-4c00-a15d-4c4180d5f835" (UID: "8bf67244-3ae2-4c00-a15d-4c4180d5f835"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:40:01 crc kubenswrapper[5052]: I1203 11:40:01.067729 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/8bf67244-3ae2-4c00-a15d-4c4180d5f835-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:40:01 crc kubenswrapper[5052]: I1203 11:40:01.067766 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-728bk\" (UniqueName: \"kubernetes.io/projected/70e2b274-dd4d-409a-8d45-a09af330f7a3-kube-api-access-728bk\") on node \"crc\" DevicePath \"\"" Dec 03 11:40:01 crc kubenswrapper[5052]: I1203 11:40:01.067776 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage04-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage04-crc\") on node \"crc\" DevicePath \"\"" Dec 03 11:40:01 crc kubenswrapper[5052]: I1203 11:40:01.067788 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-cp4l6\" (UniqueName: \"kubernetes.io/projected/8bf67244-3ae2-4c00-a15d-4c4180d5f835-kube-api-access-cp4l6\") on node \"crc\" DevicePath \"\"" Dec 03 11:40:01 crc kubenswrapper[5052]: I1203 11:40:01.067797 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/70e2b274-dd4d-409a-8d45-a09af330f7a3-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:40:01 crc kubenswrapper[5052]: I1203 11:40:01.067805 5052 reconciler_common.go:293] "Volume detached for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/70e2b274-dd4d-409a-8d45-a09af330f7a3-sg-core-conf-yaml\") on node \"crc\" DevicePath \"\"" Dec 03 11:40:01 crc kubenswrapper[5052]: I1203 11:40:01.067813 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/8bf67244-3ae2-4c00-a15d-4c4180d5f835-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:40:01 crc kubenswrapper[5052]: I1203 11:40:01.067821 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/8bf67244-3ae2-4c00-a15d-4c4180d5f835-logs\") on node \"crc\" DevicePath \"\"" Dec 03 11:40:01 crc kubenswrapper[5052]: I1203 11:40:01.067829 5052 reconciler_common.go:293] "Volume detached for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/70e2b274-dd4d-409a-8d45-a09af330f7a3-log-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 11:40:01 crc kubenswrapper[5052]: I1203 11:40:01.067837 5052 reconciler_common.go:293] "Volume detached for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/70e2b274-dd4d-409a-8d45-a09af330f7a3-run-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 11:40:01 crc kubenswrapper[5052]: I1203 11:40:01.067864 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage15-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage15-crc\") on node \"crc\" " Dec 03 11:40:01 crc kubenswrapper[5052]: I1203 11:40:01.067873 5052 reconciler_common.go:293] "Volume detached for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/8bf67244-3ae2-4c00-a15d-4c4180d5f835-httpd-run\") on node \"crc\" DevicePath \"\"" Dec 03 11:40:01 crc kubenswrapper[5052]: I1203 11:40:01.067882 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8bf67244-3ae2-4c00-a15d-4c4180d5f835-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:40:01 crc kubenswrapper[5052]: I1203 11:40:01.070548 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8bf67244-3ae2-4c00-a15d-4c4180d5f835-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "8bf67244-3ae2-4c00-a15d-4c4180d5f835" (UID: "8bf67244-3ae2-4c00-a15d-4c4180d5f835"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:40:01 crc kubenswrapper[5052]: I1203 11:40:01.084045 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/70e2b274-dd4d-409a-8d45-a09af330f7a3-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "70e2b274-dd4d-409a-8d45-a09af330f7a3" (UID: "70e2b274-dd4d-409a-8d45-a09af330f7a3"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:40:01 crc kubenswrapper[5052]: I1203 11:40:01.097423 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage15-crc" (UniqueName: "kubernetes.io/local-volume/local-storage15-crc") on node "crc" Dec 03 11:40:01 crc kubenswrapper[5052]: I1203 11:40:01.134318 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/70e2b274-dd4d-409a-8d45-a09af330f7a3-config-data" (OuterVolumeSpecName: "config-data") pod "70e2b274-dd4d-409a-8d45-a09af330f7a3" (UID: "70e2b274-dd4d-409a-8d45-a09af330f7a3"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:40:01 crc kubenswrapper[5052]: I1203 11:40:01.169019 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/70e2b274-dd4d-409a-8d45-a09af330f7a3-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:40:01 crc kubenswrapper[5052]: I1203 11:40:01.169049 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/70e2b274-dd4d-409a-8d45-a09af330f7a3-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:40:01 crc kubenswrapper[5052]: I1203 11:40:01.169060 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage15-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage15-crc\") on node \"crc\" DevicePath \"\"" Dec 03 11:40:01 crc kubenswrapper[5052]: I1203 11:40:01.169068 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/8bf67244-3ae2-4c00-a15d-4c4180d5f835-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 11:40:01 crc kubenswrapper[5052]: I1203 11:40:01.261586 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-73a4-account-create-update-kfmjt"] Dec 03 11:40:01 crc kubenswrapper[5052]: W1203 11:40:01.266544 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod26a22bd0_90b3_4fe9_8029_a56c6c7bcdfd.slice/crio-ebef274a7eb3459db92d5d0c23812c088da58695c21ed98debb674f544bf4f98 WatchSource:0}: Error finding container ebef274a7eb3459db92d5d0c23812c088da58695c21ed98debb674f544bf4f98: Status 404 returned error can't find the container with id ebef274a7eb3459db92d5d0c23812c088da58695c21ed98debb674f544bf4f98 Dec 03 11:40:01 crc kubenswrapper[5052]: I1203 11:40:01.273232 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell0-3d99-account-create-update-kqb94"] Dec 03 11:40:01 crc kubenswrapper[5052]: I1203 11:40:01.285036 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-cdd4-account-create-update-lbpqk"] Dec 03 11:40:01 crc kubenswrapper[5052]: I1203 11:40:01.304805 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-db-create-4mc7v"] Dec 03 11:40:01 crc kubenswrapper[5052]: W1203 11:40:01.307138 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda53c0da6_8b05_40fd_90c7_78cfb701780a.slice/crio-0b6241e6c7698449e8c9cd94ed0f6d1ed0b4578cd7f683ed47b50d6e0a7aeb3e WatchSource:0}: Error finding container 0b6241e6c7698449e8c9cd94ed0f6d1ed0b4578cd7f683ed47b50d6e0a7aeb3e: Status 404 returned error can't find the container with id 0b6241e6c7698449e8c9cd94ed0f6d1ed0b4578cd7f683ed47b50d6e0a7aeb3e Dec 03 11:40:01 crc kubenswrapper[5052]: W1203 11:40:01.321161 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5f9bdf8c_c97c_477e_bf48_73fe6ddae983.slice/crio-909fac460000c720d2ca0d5f9584db78b78a09588b187e203e6c62f18a5bbc6c WatchSource:0}: Error finding container 909fac460000c720d2ca0d5f9584db78b78a09588b187e203e6c62f18a5bbc6c: Status 404 returned error can't find the container with id 909fac460000c720d2ca0d5f9584db78b78a09588b187e203e6c62f18a5bbc6c Dec 03 11:40:01 crc kubenswrapper[5052]: W1203 11:40:01.322067 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod066f70dd_4aa5_4b30_b913_2285c642134e.slice/crio-418e72dc8dd3adcadb952eb547dde16a4372698dbe49a1984725a32ec2a33632 WatchSource:0}: Error finding container 418e72dc8dd3adcadb952eb547dde16a4372698dbe49a1984725a32ec2a33632: Status 404 returned error can't find the container with id 418e72dc8dd3adcadb952eb547dde16a4372698dbe49a1984725a32ec2a33632 Dec 03 11:40:01 crc kubenswrapper[5052]: I1203 11:40:01.328654 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell0-db-create-w672x"] Dec 03 11:40:01 crc kubenswrapper[5052]: I1203 11:40:01.354912 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-db-create-wm8nm"] Dec 03 11:40:01 crc kubenswrapper[5052]: I1203 11:40:01.437858 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/swift-proxy-fd9678558-qx8wh" Dec 03 11:40:01 crc kubenswrapper[5052]: I1203 11:40:01.440633 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/swift-proxy-fd9678558-qx8wh" Dec 03 11:40:01 crc kubenswrapper[5052]: I1203 11:40:01.687400 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ef47ba52-772d-4732-bd15-e7513e626be9" path="/var/lib/kubelet/pods/ef47ba52-772d-4732-bd15-e7513e626be9/volumes" Dec 03 11:40:01 crc kubenswrapper[5052]: I1203 11:40:01.795734 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-db-create-w672x" event={"ID":"6410c857-4e7d-486c-be0a-6734565014d4","Type":"ContainerStarted","Data":"ac95353be0e40db2feb743167732689ef07e0a7b12dc8b53d9193046d46c4d89"} Dec 03 11:40:01 crc kubenswrapper[5052]: I1203 11:40:01.795770 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-db-create-w672x" event={"ID":"6410c857-4e7d-486c-be0a-6734565014d4","Type":"ContainerStarted","Data":"32906a5f17bed523a04efda052e098b62e93da02f83d13b806888cf190a583b9"} Dec 03 11:40:01 crc kubenswrapper[5052]: I1203 11:40:01.800594 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-3d99-account-create-update-kqb94" event={"ID":"a53c0da6-8b05-40fd-90c7-78cfb701780a","Type":"ContainerStarted","Data":"0b6241e6c7698449e8c9cd94ed0f6d1ed0b4578cd7f683ed47b50d6e0a7aeb3e"} Dec 03 11:40:01 crc kubenswrapper[5052]: I1203 11:40:01.802115 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-73a4-account-create-update-kfmjt" event={"ID":"26a22bd0-90b3-4fe9-8029-a56c6c7bcdfd","Type":"ContainerStarted","Data":"230b893b58887ce8075576ec97b7c1b839aac29a4c4a0c82f043dbd5a3fbe8eb"} Dec 03 11:40:01 crc kubenswrapper[5052]: I1203 11:40:01.802167 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-73a4-account-create-update-kfmjt" event={"ID":"26a22bd0-90b3-4fe9-8029-a56c6c7bcdfd","Type":"ContainerStarted","Data":"ebef274a7eb3459db92d5d0c23812c088da58695c21ed98debb674f544bf4f98"} Dec 03 11:40:01 crc kubenswrapper[5052]: I1203 11:40:01.804880 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"8bf67244-3ae2-4c00-a15d-4c4180d5f835","Type":"ContainerDied","Data":"b524ed8cfb003b48dcd655e889d08edfc66e061ccb7e6123c9c7505efd3ae39b"} Dec 03 11:40:01 crc kubenswrapper[5052]: I1203 11:40:01.804925 5052 scope.go:117] "RemoveContainer" containerID="dcd58ef3b45066b82652446bcbd875e2abf4758e96cd73b62fededb0469b531e" Dec 03 11:40:01 crc kubenswrapper[5052]: I1203 11:40:01.804937 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:40:01 crc kubenswrapper[5052]: I1203 11:40:01.810448 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-cdd4-account-create-update-lbpqk" event={"ID":"5f9bdf8c-c97c-477e-bf48-73fe6ddae983","Type":"ContainerStarted","Data":"8353d9ed68998103ae5183e6a229e34b9624453345905d1783ef8d0257b3e95b"} Dec 03 11:40:01 crc kubenswrapper[5052]: I1203 11:40:01.810501 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-cdd4-account-create-update-lbpqk" event={"ID":"5f9bdf8c-c97c-477e-bf48-73fe6ddae983","Type":"ContainerStarted","Data":"909fac460000c720d2ca0d5f9584db78b78a09588b187e203e6c62f18a5bbc6c"} Dec 03 11:40:01 crc kubenswrapper[5052]: I1203 11:40:01.817541 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-db-create-wm8nm" event={"ID":"996bddd2-d361-4a2f-aa3c-773b37ad9b59","Type":"ContainerStarted","Data":"7f0e3ee717dce1717e1541b66434fdf13121b35964b4328bce3b46e0ca09e4ca"} Dec 03 11:40:01 crc kubenswrapper[5052]: I1203 11:40:01.817586 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-db-create-wm8nm" event={"ID":"996bddd2-d361-4a2f-aa3c-773b37ad9b59","Type":"ContainerStarted","Data":"028954ede6805438c4e1592920671530b30ec68dab4b09dadd25e39649f4fa5b"} Dec 03 11:40:01 crc kubenswrapper[5052]: I1203 11:40:01.821880 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-cell0-db-create-w672x" podStartSLOduration=4.821862836 podStartE2EDuration="4.821862836s" podCreationTimestamp="2025-12-03 11:39:57 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:40:01.817316914 +0000 UTC m=+1192.405558678" watchObservedRunningTime="2025-12-03 11:40:01.821862836 +0000 UTC m=+1192.410104610" Dec 03 11:40:01 crc kubenswrapper[5052]: I1203 11:40:01.821990 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-db-create-4mc7v" event={"ID":"066f70dd-4aa5-4b30-b913-2285c642134e","Type":"ContainerStarted","Data":"735a470576dc74f7e9b95db518ca1e458cf10b51cd97c1cbdb4f9e95d02a1414"} Dec 03 11:40:01 crc kubenswrapper[5052]: I1203 11:40:01.822049 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-db-create-4mc7v" event={"ID":"066f70dd-4aa5-4b30-b913-2285c642134e","Type":"ContainerStarted","Data":"418e72dc8dd3adcadb952eb547dde16a4372698dbe49a1984725a32ec2a33632"} Dec 03 11:40:01 crc kubenswrapper[5052]: I1203 11:40:01.833630 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"70e2b274-dd4d-409a-8d45-a09af330f7a3","Type":"ContainerDied","Data":"f6395474b3ead890bff3ec7ef4a1992397a65b07d36b291cb39d7a172ca1da72"} Dec 03 11:40:01 crc kubenswrapper[5052]: I1203 11:40:01.833739 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:40:01 crc kubenswrapper[5052]: I1203 11:40:01.833904 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-api-73a4-account-create-update-kfmjt" podStartSLOduration=4.833885683 podStartE2EDuration="4.833885683s" podCreationTimestamp="2025-12-03 11:39:57 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:40:01.832918045 +0000 UTC m=+1192.421159809" watchObservedRunningTime="2025-12-03 11:40:01.833885683 +0000 UTC m=+1192.422127447" Dec 03 11:40:01 crc kubenswrapper[5052]: I1203 11:40:01.838721 5052 scope.go:117] "RemoveContainer" containerID="26f247c9c6d8664d070a0e01318f861102e07448fbac21823c5f2aca4b6f5da2" Dec 03 11:40:01 crc kubenswrapper[5052]: I1203 11:40:01.847309 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"502cb03c-d1a5-4dc7-a040-0571a4695379","Type":"ContainerDied","Data":"a1fa89c15194a6d777274ed0f8572100b4a875ebf55e89ab2de18462f43cc500"} Dec 03 11:40:01 crc kubenswrapper[5052]: I1203 11:40:01.847429 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:40:01 crc kubenswrapper[5052]: I1203 11:40:01.870644 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-cell1-cdd4-account-create-update-lbpqk" podStartSLOduration=3.870627223 podStartE2EDuration="3.870627223s" podCreationTimestamp="2025-12-03 11:39:58 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:40:01.84869232 +0000 UTC m=+1192.436934084" watchObservedRunningTime="2025-12-03 11:40:01.870627223 +0000 UTC m=+1192.458868987" Dec 03 11:40:01 crc kubenswrapper[5052]: I1203 11:40:01.907759 5052 scope.go:117] "RemoveContainer" containerID="33c7c237d3624c744edaa6397912e20b3e447503cda667a0d60162d16ad547d5" Dec 03 11:40:01 crc kubenswrapper[5052]: I1203 11:40:01.919905 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 11:40:01 crc kubenswrapper[5052]: I1203 11:40:01.932761 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 11:40:01 crc kubenswrapper[5052]: I1203 11:40:01.942792 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-cell0-3d99-account-create-update-kqb94" podStartSLOduration=3.9427674660000003 podStartE2EDuration="3.942767466s" podCreationTimestamp="2025-12-03 11:39:58 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:40:01.883966208 +0000 UTC m=+1192.472207972" watchObservedRunningTime="2025-12-03 11:40:01.942767466 +0000 UTC m=+1192.531009230" Dec 03 11:40:01 crc kubenswrapper[5052]: I1203 11:40:01.966901 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 11:40:01 crc kubenswrapper[5052]: E1203 11:40:01.967320 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="8bf67244-3ae2-4c00-a15d-4c4180d5f835" containerName="glance-log" Dec 03 11:40:01 crc kubenswrapper[5052]: I1203 11:40:01.967332 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="8bf67244-3ae2-4c00-a15d-4c4180d5f835" containerName="glance-log" Dec 03 11:40:01 crc kubenswrapper[5052]: E1203 11:40:01.967343 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="ef47ba52-772d-4732-bd15-e7513e626be9" containerName="barbican-api" Dec 03 11:40:01 crc kubenswrapper[5052]: I1203 11:40:01.967349 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="ef47ba52-772d-4732-bd15-e7513e626be9" containerName="barbican-api" Dec 03 11:40:01 crc kubenswrapper[5052]: E1203 11:40:01.967361 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="70e2b274-dd4d-409a-8d45-a09af330f7a3" containerName="sg-core" Dec 03 11:40:01 crc kubenswrapper[5052]: I1203 11:40:01.967371 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="70e2b274-dd4d-409a-8d45-a09af330f7a3" containerName="sg-core" Dec 03 11:40:01 crc kubenswrapper[5052]: E1203 11:40:01.967381 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="502cb03c-d1a5-4dc7-a040-0571a4695379" containerName="glance-httpd" Dec 03 11:40:01 crc kubenswrapper[5052]: I1203 11:40:01.967386 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="502cb03c-d1a5-4dc7-a040-0571a4695379" containerName="glance-httpd" Dec 03 11:40:01 crc kubenswrapper[5052]: E1203 11:40:01.967397 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="8bf67244-3ae2-4c00-a15d-4c4180d5f835" containerName="glance-httpd" Dec 03 11:40:01 crc kubenswrapper[5052]: I1203 11:40:01.967402 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="8bf67244-3ae2-4c00-a15d-4c4180d5f835" containerName="glance-httpd" Dec 03 11:40:01 crc kubenswrapper[5052]: E1203 11:40:01.967417 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="ef47ba52-772d-4732-bd15-e7513e626be9" containerName="barbican-api-log" Dec 03 11:40:01 crc kubenswrapper[5052]: I1203 11:40:01.967422 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="ef47ba52-772d-4732-bd15-e7513e626be9" containerName="barbican-api-log" Dec 03 11:40:01 crc kubenswrapper[5052]: E1203 11:40:01.967434 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="70e2b274-dd4d-409a-8d45-a09af330f7a3" containerName="proxy-httpd" Dec 03 11:40:01 crc kubenswrapper[5052]: I1203 11:40:01.967440 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="70e2b274-dd4d-409a-8d45-a09af330f7a3" containerName="proxy-httpd" Dec 03 11:40:01 crc kubenswrapper[5052]: E1203 11:40:01.967452 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="70e2b274-dd4d-409a-8d45-a09af330f7a3" containerName="ceilometer-notification-agent" Dec 03 11:40:01 crc kubenswrapper[5052]: I1203 11:40:01.967458 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="70e2b274-dd4d-409a-8d45-a09af330f7a3" containerName="ceilometer-notification-agent" Dec 03 11:40:01 crc kubenswrapper[5052]: E1203 11:40:01.967474 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="70e2b274-dd4d-409a-8d45-a09af330f7a3" containerName="ceilometer-central-agent" Dec 03 11:40:01 crc kubenswrapper[5052]: I1203 11:40:01.967479 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="70e2b274-dd4d-409a-8d45-a09af330f7a3" containerName="ceilometer-central-agent" Dec 03 11:40:01 crc kubenswrapper[5052]: E1203 11:40:01.967487 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="502cb03c-d1a5-4dc7-a040-0571a4695379" containerName="glance-log" Dec 03 11:40:01 crc kubenswrapper[5052]: I1203 11:40:01.967492 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="502cb03c-d1a5-4dc7-a040-0571a4695379" containerName="glance-log" Dec 03 11:40:01 crc kubenswrapper[5052]: I1203 11:40:01.967664 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="8bf67244-3ae2-4c00-a15d-4c4180d5f835" containerName="glance-log" Dec 03 11:40:01 crc kubenswrapper[5052]: I1203 11:40:01.967676 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="ef47ba52-772d-4732-bd15-e7513e626be9" containerName="barbican-api-log" Dec 03 11:40:01 crc kubenswrapper[5052]: I1203 11:40:01.967690 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="502cb03c-d1a5-4dc7-a040-0571a4695379" containerName="glance-httpd" Dec 03 11:40:01 crc kubenswrapper[5052]: I1203 11:40:01.967696 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="70e2b274-dd4d-409a-8d45-a09af330f7a3" containerName="ceilometer-central-agent" Dec 03 11:40:01 crc kubenswrapper[5052]: I1203 11:40:01.967703 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="70e2b274-dd4d-409a-8d45-a09af330f7a3" containerName="sg-core" Dec 03 11:40:01 crc kubenswrapper[5052]: I1203 11:40:01.967713 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="502cb03c-d1a5-4dc7-a040-0571a4695379" containerName="glance-log" Dec 03 11:40:01 crc kubenswrapper[5052]: I1203 11:40:01.967723 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="8bf67244-3ae2-4c00-a15d-4c4180d5f835" containerName="glance-httpd" Dec 03 11:40:01 crc kubenswrapper[5052]: I1203 11:40:01.967735 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="70e2b274-dd4d-409a-8d45-a09af330f7a3" containerName="ceilometer-notification-agent" Dec 03 11:40:01 crc kubenswrapper[5052]: I1203 11:40:01.967743 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="70e2b274-dd4d-409a-8d45-a09af330f7a3" containerName="proxy-httpd" Dec 03 11:40:01 crc kubenswrapper[5052]: I1203 11:40:01.967751 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="ef47ba52-772d-4732-bd15-e7513e626be9" containerName="barbican-api" Dec 03 11:40:01 crc kubenswrapper[5052]: I1203 11:40:01.968659 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:40:01 crc kubenswrapper[5052]: I1203 11:40:01.973072 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-scripts" Dec 03 11:40:01 crc kubenswrapper[5052]: I1203 11:40:01.973267 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-default-external-config-data" Dec 03 11:40:01 crc kubenswrapper[5052]: I1203 11:40:01.973328 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-glance-default-public-svc" Dec 03 11:40:01 crc kubenswrapper[5052]: I1203 11:40:01.986270 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-glance-dockercfg-kt8wz" Dec 03 11:40:01 crc kubenswrapper[5052]: I1203 11:40:01.989545 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 11:40:02 crc kubenswrapper[5052]: I1203 11:40:02.000386 5052 scope.go:117] "RemoveContainer" containerID="89b928d0bee5a1b120bb1a4de7f35f330ce3fb961abc80c74f6017fd2b9b6ae4" Dec 03 11:40:02 crc kubenswrapper[5052]: I1203 11:40:02.000846 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-cell1-db-create-4mc7v" podStartSLOduration=4.000827772 podStartE2EDuration="4.000827772s" podCreationTimestamp="2025-12-03 11:39:58 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:40:01.907396045 +0000 UTC m=+1192.495637809" watchObservedRunningTime="2025-12-03 11:40:02.000827772 +0000 UTC m=+1192.589069536" Dec 03 11:40:02 crc kubenswrapper[5052]: I1203 11:40:02.047999 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 11:40:02 crc kubenswrapper[5052]: I1203 11:40:02.082097 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 11:40:02 crc kubenswrapper[5052]: I1203 11:40:02.088058 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/75e4b34e-f7b8-4209-95a6-6cc45f6eb57c-httpd-run\") pod \"glance-default-external-api-0\" (UID: \"75e4b34e-f7b8-4209-95a6-6cc45f6eb57c\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:40:02 crc kubenswrapper[5052]: I1203 11:40:02.088113 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rf74q\" (UniqueName: \"kubernetes.io/projected/75e4b34e-f7b8-4209-95a6-6cc45f6eb57c-kube-api-access-rf74q\") pod \"glance-default-external-api-0\" (UID: \"75e4b34e-f7b8-4209-95a6-6cc45f6eb57c\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:40:02 crc kubenswrapper[5052]: I1203 11:40:02.088146 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/75e4b34e-f7b8-4209-95a6-6cc45f6eb57c-public-tls-certs\") pod \"glance-default-external-api-0\" (UID: \"75e4b34e-f7b8-4209-95a6-6cc45f6eb57c\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:40:02 crc kubenswrapper[5052]: I1203 11:40:02.088192 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/75e4b34e-f7b8-4209-95a6-6cc45f6eb57c-scripts\") pod \"glance-default-external-api-0\" (UID: \"75e4b34e-f7b8-4209-95a6-6cc45f6eb57c\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:40:02 crc kubenswrapper[5052]: I1203 11:40:02.088225 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage15-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage15-crc\") pod \"glance-default-external-api-0\" (UID: \"75e4b34e-f7b8-4209-95a6-6cc45f6eb57c\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:40:02 crc kubenswrapper[5052]: I1203 11:40:02.088285 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/75e4b34e-f7b8-4209-95a6-6cc45f6eb57c-logs\") pod \"glance-default-external-api-0\" (UID: \"75e4b34e-f7b8-4209-95a6-6cc45f6eb57c\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:40:02 crc kubenswrapper[5052]: I1203 11:40:02.088338 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/75e4b34e-f7b8-4209-95a6-6cc45f6eb57c-combined-ca-bundle\") pod \"glance-default-external-api-0\" (UID: \"75e4b34e-f7b8-4209-95a6-6cc45f6eb57c\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:40:02 crc kubenswrapper[5052]: I1203 11:40:02.088366 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/75e4b34e-f7b8-4209-95a6-6cc45f6eb57c-config-data\") pod \"glance-default-external-api-0\" (UID: \"75e4b34e-f7b8-4209-95a6-6cc45f6eb57c\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:40:02 crc kubenswrapper[5052]: I1203 11:40:02.126581 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:40:02 crc kubenswrapper[5052]: I1203 11:40:02.129690 5052 scope.go:117] "RemoveContainer" containerID="466f8e75a4f4613fdb58e65fc3b11d64b99b313618c5c45ebe14444fc70dbacf" Dec 03 11:40:02 crc kubenswrapper[5052]: I1203 11:40:02.146572 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:40:02 crc kubenswrapper[5052]: I1203 11:40:02.169349 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 11:40:02 crc kubenswrapper[5052]: I1203 11:40:02.171391 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:40:02 crc kubenswrapper[5052]: I1203 11:40:02.172665 5052 scope.go:117] "RemoveContainer" containerID="a976462aed958683aefdf6044f7b3b06bddc3fc3b6d77f4841493e30c067161c" Dec 03 11:40:02 crc kubenswrapper[5052]: I1203 11:40:02.178140 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-api-db-create-wm8nm" podStartSLOduration=5.1781191 podStartE2EDuration="5.1781191s" podCreationTimestamp="2025-12-03 11:39:57 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:40:01.999051881 +0000 UTC m=+1192.587293635" watchObservedRunningTime="2025-12-03 11:40:02.1781191 +0000 UTC m=+1192.766360864" Dec 03 11:40:02 crc kubenswrapper[5052]: I1203 11:40:02.180022 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-default-internal-config-data" Dec 03 11:40:02 crc kubenswrapper[5052]: I1203 11:40:02.180224 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-glance-default-internal-svc" Dec 03 11:40:02 crc kubenswrapper[5052]: I1203 11:40:02.189633 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/75e4b34e-f7b8-4209-95a6-6cc45f6eb57c-httpd-run\") pod \"glance-default-external-api-0\" (UID: \"75e4b34e-f7b8-4209-95a6-6cc45f6eb57c\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:40:02 crc kubenswrapper[5052]: I1203 11:40:02.189688 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-rf74q\" (UniqueName: \"kubernetes.io/projected/75e4b34e-f7b8-4209-95a6-6cc45f6eb57c-kube-api-access-rf74q\") pod \"glance-default-external-api-0\" (UID: \"75e4b34e-f7b8-4209-95a6-6cc45f6eb57c\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:40:02 crc kubenswrapper[5052]: I1203 11:40:02.189717 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/75e4b34e-f7b8-4209-95a6-6cc45f6eb57c-public-tls-certs\") pod \"glance-default-external-api-0\" (UID: \"75e4b34e-f7b8-4209-95a6-6cc45f6eb57c\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:40:02 crc kubenswrapper[5052]: I1203 11:40:02.189775 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/75e4b34e-f7b8-4209-95a6-6cc45f6eb57c-scripts\") pod \"glance-default-external-api-0\" (UID: \"75e4b34e-f7b8-4209-95a6-6cc45f6eb57c\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:40:02 crc kubenswrapper[5052]: I1203 11:40:02.189808 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage15-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage15-crc\") pod \"glance-default-external-api-0\" (UID: \"75e4b34e-f7b8-4209-95a6-6cc45f6eb57c\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:40:02 crc kubenswrapper[5052]: I1203 11:40:02.189900 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/75e4b34e-f7b8-4209-95a6-6cc45f6eb57c-logs\") pod \"glance-default-external-api-0\" (UID: \"75e4b34e-f7b8-4209-95a6-6cc45f6eb57c\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:40:02 crc kubenswrapper[5052]: I1203 11:40:02.190011 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/75e4b34e-f7b8-4209-95a6-6cc45f6eb57c-combined-ca-bundle\") pod \"glance-default-external-api-0\" (UID: \"75e4b34e-f7b8-4209-95a6-6cc45f6eb57c\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:40:02 crc kubenswrapper[5052]: I1203 11:40:02.190047 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/75e4b34e-f7b8-4209-95a6-6cc45f6eb57c-config-data\") pod \"glance-default-external-api-0\" (UID: \"75e4b34e-f7b8-4209-95a6-6cc45f6eb57c\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:40:02 crc kubenswrapper[5052]: I1203 11:40:02.194018 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/75e4b34e-f7b8-4209-95a6-6cc45f6eb57c-httpd-run\") pod \"glance-default-external-api-0\" (UID: \"75e4b34e-f7b8-4209-95a6-6cc45f6eb57c\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:40:02 crc kubenswrapper[5052]: I1203 11:40:02.194305 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/75e4b34e-f7b8-4209-95a6-6cc45f6eb57c-logs\") pod \"glance-default-external-api-0\" (UID: \"75e4b34e-f7b8-4209-95a6-6cc45f6eb57c\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:40:02 crc kubenswrapper[5052]: I1203 11:40:02.194625 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage15-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage15-crc\") pod \"glance-default-external-api-0\" (UID: \"75e4b34e-f7b8-4209-95a6-6cc45f6eb57c\") device mount path \"/mnt/openstack/pv15\"" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:40:02 crc kubenswrapper[5052]: I1203 11:40:02.197242 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:40:02 crc kubenswrapper[5052]: I1203 11:40:02.199650 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/75e4b34e-f7b8-4209-95a6-6cc45f6eb57c-public-tls-certs\") pod \"glance-default-external-api-0\" (UID: \"75e4b34e-f7b8-4209-95a6-6cc45f6eb57c\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:40:02 crc kubenswrapper[5052]: I1203 11:40:02.200905 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/75e4b34e-f7b8-4209-95a6-6cc45f6eb57c-config-data\") pod \"glance-default-external-api-0\" (UID: \"75e4b34e-f7b8-4209-95a6-6cc45f6eb57c\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:40:02 crc kubenswrapper[5052]: I1203 11:40:02.209690 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 11:40:02 crc kubenswrapper[5052]: I1203 11:40:02.209835 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:40:02 crc kubenswrapper[5052]: I1203 11:40:02.211297 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/75e4b34e-f7b8-4209-95a6-6cc45f6eb57c-combined-ca-bundle\") pod \"glance-default-external-api-0\" (UID: \"75e4b34e-f7b8-4209-95a6-6cc45f6eb57c\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:40:02 crc kubenswrapper[5052]: I1203 11:40:02.214892 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-scripts" Dec 03 11:40:02 crc kubenswrapper[5052]: I1203 11:40:02.227370 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:40:02 crc kubenswrapper[5052]: I1203 11:40:02.229254 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-config-data" Dec 03 11:40:02 crc kubenswrapper[5052]: I1203 11:40:02.231044 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/75e4b34e-f7b8-4209-95a6-6cc45f6eb57c-scripts\") pod \"glance-default-external-api-0\" (UID: \"75e4b34e-f7b8-4209-95a6-6cc45f6eb57c\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:40:02 crc kubenswrapper[5052]: I1203 11:40:02.237167 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-rf74q\" (UniqueName: \"kubernetes.io/projected/75e4b34e-f7b8-4209-95a6-6cc45f6eb57c-kube-api-access-rf74q\") pod \"glance-default-external-api-0\" (UID: \"75e4b34e-f7b8-4209-95a6-6cc45f6eb57c\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:40:02 crc kubenswrapper[5052]: I1203 11:40:02.237420 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage15-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage15-crc\") pod \"glance-default-external-api-0\" (UID: \"75e4b34e-f7b8-4209-95a6-6cc45f6eb57c\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:40:02 crc kubenswrapper[5052]: I1203 11:40:02.264846 5052 scope.go:117] "RemoveContainer" containerID="3c6d3ddfe8759dec5d3030b3c1b64bcbc9f4228dfeaac36309751c177de0b102" Dec 03 11:40:02 crc kubenswrapper[5052]: I1203 11:40:02.292533 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/58df8ad3-3e8f-41ae-88ab-f4960ee9a812-config-data\") pod \"ceilometer-0\" (UID: \"58df8ad3-3e8f-41ae-88ab-f4960ee9a812\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:40:02 crc kubenswrapper[5052]: I1203 11:40:02.292590 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lhh66\" (UniqueName: \"kubernetes.io/projected/665a736c-d550-4160-a8bb-619d1d5ea826-kube-api-access-lhh66\") pod \"glance-default-internal-api-0\" (UID: \"665a736c-d550-4160-a8bb-619d1d5ea826\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:40:02 crc kubenswrapper[5052]: I1203 11:40:02.292621 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ztj8g\" (UniqueName: \"kubernetes.io/projected/58df8ad3-3e8f-41ae-88ab-f4960ee9a812-kube-api-access-ztj8g\") pod \"ceilometer-0\" (UID: \"58df8ad3-3e8f-41ae-88ab-f4960ee9a812\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:40:02 crc kubenswrapper[5052]: I1203 11:40:02.292678 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/58df8ad3-3e8f-41ae-88ab-f4960ee9a812-run-httpd\") pod \"ceilometer-0\" (UID: \"58df8ad3-3e8f-41ae-88ab-f4960ee9a812\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:40:02 crc kubenswrapper[5052]: I1203 11:40:02.292711 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/58df8ad3-3e8f-41ae-88ab-f4960ee9a812-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"58df8ad3-3e8f-41ae-88ab-f4960ee9a812\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:40:02 crc kubenswrapper[5052]: I1203 11:40:02.292732 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/665a736c-d550-4160-a8bb-619d1d5ea826-combined-ca-bundle\") pod \"glance-default-internal-api-0\" (UID: \"665a736c-d550-4160-a8bb-619d1d5ea826\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:40:02 crc kubenswrapper[5052]: I1203 11:40:02.292757 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/665a736c-d550-4160-a8bb-619d1d5ea826-internal-tls-certs\") pod \"glance-default-internal-api-0\" (UID: \"665a736c-d550-4160-a8bb-619d1d5ea826\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:40:02 crc kubenswrapper[5052]: I1203 11:40:02.292820 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage04-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage04-crc\") pod \"glance-default-internal-api-0\" (UID: \"665a736c-d550-4160-a8bb-619d1d5ea826\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:40:02 crc kubenswrapper[5052]: I1203 11:40:02.292846 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/665a736c-d550-4160-a8bb-619d1d5ea826-scripts\") pod \"glance-default-internal-api-0\" (UID: \"665a736c-d550-4160-a8bb-619d1d5ea826\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:40:02 crc kubenswrapper[5052]: I1203 11:40:02.292894 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/665a736c-d550-4160-a8bb-619d1d5ea826-httpd-run\") pod \"glance-default-internal-api-0\" (UID: \"665a736c-d550-4160-a8bb-619d1d5ea826\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:40:02 crc kubenswrapper[5052]: I1203 11:40:02.292919 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/665a736c-d550-4160-a8bb-619d1d5ea826-config-data\") pod \"glance-default-internal-api-0\" (UID: \"665a736c-d550-4160-a8bb-619d1d5ea826\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:40:02 crc kubenswrapper[5052]: I1203 11:40:02.292943 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/58df8ad3-3e8f-41ae-88ab-f4960ee9a812-scripts\") pod \"ceilometer-0\" (UID: \"58df8ad3-3e8f-41ae-88ab-f4960ee9a812\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:40:02 crc kubenswrapper[5052]: I1203 11:40:02.292972 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/58df8ad3-3e8f-41ae-88ab-f4960ee9a812-log-httpd\") pod \"ceilometer-0\" (UID: \"58df8ad3-3e8f-41ae-88ab-f4960ee9a812\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:40:02 crc kubenswrapper[5052]: I1203 11:40:02.293001 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/58df8ad3-3e8f-41ae-88ab-f4960ee9a812-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"58df8ad3-3e8f-41ae-88ab-f4960ee9a812\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:40:02 crc kubenswrapper[5052]: I1203 11:40:02.293020 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/665a736c-d550-4160-a8bb-619d1d5ea826-logs\") pod \"glance-default-internal-api-0\" (UID: \"665a736c-d550-4160-a8bb-619d1d5ea826\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:40:02 crc kubenswrapper[5052]: I1203 11:40:02.347752 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:40:02 crc kubenswrapper[5052]: I1203 11:40:02.395274 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/58df8ad3-3e8f-41ae-88ab-f4960ee9a812-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"58df8ad3-3e8f-41ae-88ab-f4960ee9a812\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:40:02 crc kubenswrapper[5052]: I1203 11:40:02.395317 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/665a736c-d550-4160-a8bb-619d1d5ea826-logs\") pod \"glance-default-internal-api-0\" (UID: \"665a736c-d550-4160-a8bb-619d1d5ea826\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:40:02 crc kubenswrapper[5052]: I1203 11:40:02.395353 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/58df8ad3-3e8f-41ae-88ab-f4960ee9a812-config-data\") pod \"ceilometer-0\" (UID: \"58df8ad3-3e8f-41ae-88ab-f4960ee9a812\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:40:02 crc kubenswrapper[5052]: I1203 11:40:02.395379 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-lhh66\" (UniqueName: \"kubernetes.io/projected/665a736c-d550-4160-a8bb-619d1d5ea826-kube-api-access-lhh66\") pod \"glance-default-internal-api-0\" (UID: \"665a736c-d550-4160-a8bb-619d1d5ea826\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:40:02 crc kubenswrapper[5052]: I1203 11:40:02.395412 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-ztj8g\" (UniqueName: \"kubernetes.io/projected/58df8ad3-3e8f-41ae-88ab-f4960ee9a812-kube-api-access-ztj8g\") pod \"ceilometer-0\" (UID: \"58df8ad3-3e8f-41ae-88ab-f4960ee9a812\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:40:02 crc kubenswrapper[5052]: I1203 11:40:02.395480 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/58df8ad3-3e8f-41ae-88ab-f4960ee9a812-run-httpd\") pod \"ceilometer-0\" (UID: \"58df8ad3-3e8f-41ae-88ab-f4960ee9a812\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:40:02 crc kubenswrapper[5052]: I1203 11:40:02.395552 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/58df8ad3-3e8f-41ae-88ab-f4960ee9a812-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"58df8ad3-3e8f-41ae-88ab-f4960ee9a812\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:40:02 crc kubenswrapper[5052]: I1203 11:40:02.395581 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/665a736c-d550-4160-a8bb-619d1d5ea826-combined-ca-bundle\") pod \"glance-default-internal-api-0\" (UID: \"665a736c-d550-4160-a8bb-619d1d5ea826\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:40:02 crc kubenswrapper[5052]: I1203 11:40:02.395612 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/665a736c-d550-4160-a8bb-619d1d5ea826-internal-tls-certs\") pod \"glance-default-internal-api-0\" (UID: \"665a736c-d550-4160-a8bb-619d1d5ea826\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:40:02 crc kubenswrapper[5052]: I1203 11:40:02.395707 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage04-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage04-crc\") pod \"glance-default-internal-api-0\" (UID: \"665a736c-d550-4160-a8bb-619d1d5ea826\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:40:02 crc kubenswrapper[5052]: I1203 11:40:02.395733 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/665a736c-d550-4160-a8bb-619d1d5ea826-scripts\") pod \"glance-default-internal-api-0\" (UID: \"665a736c-d550-4160-a8bb-619d1d5ea826\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:40:02 crc kubenswrapper[5052]: I1203 11:40:02.395785 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/665a736c-d550-4160-a8bb-619d1d5ea826-httpd-run\") pod \"glance-default-internal-api-0\" (UID: \"665a736c-d550-4160-a8bb-619d1d5ea826\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:40:02 crc kubenswrapper[5052]: I1203 11:40:02.395810 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/665a736c-d550-4160-a8bb-619d1d5ea826-config-data\") pod \"glance-default-internal-api-0\" (UID: \"665a736c-d550-4160-a8bb-619d1d5ea826\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:40:02 crc kubenswrapper[5052]: I1203 11:40:02.395835 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/58df8ad3-3e8f-41ae-88ab-f4960ee9a812-scripts\") pod \"ceilometer-0\" (UID: \"58df8ad3-3e8f-41ae-88ab-f4960ee9a812\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:40:02 crc kubenswrapper[5052]: I1203 11:40:02.395856 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/58df8ad3-3e8f-41ae-88ab-f4960ee9a812-log-httpd\") pod \"ceilometer-0\" (UID: \"58df8ad3-3e8f-41ae-88ab-f4960ee9a812\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:40:02 crc kubenswrapper[5052]: I1203 11:40:02.397266 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/58df8ad3-3e8f-41ae-88ab-f4960ee9a812-log-httpd\") pod \"ceilometer-0\" (UID: \"58df8ad3-3e8f-41ae-88ab-f4960ee9a812\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:40:02 crc kubenswrapper[5052]: I1203 11:40:02.402245 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/665a736c-d550-4160-a8bb-619d1d5ea826-combined-ca-bundle\") pod \"glance-default-internal-api-0\" (UID: \"665a736c-d550-4160-a8bb-619d1d5ea826\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:40:02 crc kubenswrapper[5052]: I1203 11:40:02.402826 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/665a736c-d550-4160-a8bb-619d1d5ea826-logs\") pod \"glance-default-internal-api-0\" (UID: \"665a736c-d550-4160-a8bb-619d1d5ea826\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:40:02 crc kubenswrapper[5052]: I1203 11:40:02.403660 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/58df8ad3-3e8f-41ae-88ab-f4960ee9a812-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"58df8ad3-3e8f-41ae-88ab-f4960ee9a812\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:40:02 crc kubenswrapper[5052]: I1203 11:40:02.403967 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/665a736c-d550-4160-a8bb-619d1d5ea826-httpd-run\") pod \"glance-default-internal-api-0\" (UID: \"665a736c-d550-4160-a8bb-619d1d5ea826\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:40:02 crc kubenswrapper[5052]: I1203 11:40:02.405362 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/58df8ad3-3e8f-41ae-88ab-f4960ee9a812-run-httpd\") pod \"ceilometer-0\" (UID: \"58df8ad3-3e8f-41ae-88ab-f4960ee9a812\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:40:02 crc kubenswrapper[5052]: I1203 11:40:02.405665 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage04-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage04-crc\") pod \"glance-default-internal-api-0\" (UID: \"665a736c-d550-4160-a8bb-619d1d5ea826\") device mount path \"/mnt/openstack/pv04\"" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:40:02 crc kubenswrapper[5052]: I1203 11:40:02.406974 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/58df8ad3-3e8f-41ae-88ab-f4960ee9a812-config-data\") pod \"ceilometer-0\" (UID: \"58df8ad3-3e8f-41ae-88ab-f4960ee9a812\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:40:02 crc kubenswrapper[5052]: I1203 11:40:02.408313 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/665a736c-d550-4160-a8bb-619d1d5ea826-scripts\") pod \"glance-default-internal-api-0\" (UID: \"665a736c-d550-4160-a8bb-619d1d5ea826\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:40:02 crc kubenswrapper[5052]: I1203 11:40:02.423765 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/665a736c-d550-4160-a8bb-619d1d5ea826-internal-tls-certs\") pod \"glance-default-internal-api-0\" (UID: \"665a736c-d550-4160-a8bb-619d1d5ea826\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:40:02 crc kubenswrapper[5052]: I1203 11:40:02.424595 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/58df8ad3-3e8f-41ae-88ab-f4960ee9a812-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"58df8ad3-3e8f-41ae-88ab-f4960ee9a812\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:40:02 crc kubenswrapper[5052]: I1203 11:40:02.427598 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/58df8ad3-3e8f-41ae-88ab-f4960ee9a812-scripts\") pod \"ceilometer-0\" (UID: \"58df8ad3-3e8f-41ae-88ab-f4960ee9a812\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:40:02 crc kubenswrapper[5052]: I1203 11:40:02.428632 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/665a736c-d550-4160-a8bb-619d1d5ea826-config-data\") pod \"glance-default-internal-api-0\" (UID: \"665a736c-d550-4160-a8bb-619d1d5ea826\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:40:02 crc kubenswrapper[5052]: I1203 11:40:02.442730 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-lhh66\" (UniqueName: \"kubernetes.io/projected/665a736c-d550-4160-a8bb-619d1d5ea826-kube-api-access-lhh66\") pod \"glance-default-internal-api-0\" (UID: \"665a736c-d550-4160-a8bb-619d1d5ea826\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:40:02 crc kubenswrapper[5052]: I1203 11:40:02.447336 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-ztj8g\" (UniqueName: \"kubernetes.io/projected/58df8ad3-3e8f-41ae-88ab-f4960ee9a812-kube-api-access-ztj8g\") pod \"ceilometer-0\" (UID: \"58df8ad3-3e8f-41ae-88ab-f4960ee9a812\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:40:02 crc kubenswrapper[5052]: I1203 11:40:02.499139 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage04-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage04-crc\") pod \"glance-default-internal-api-0\" (UID: \"665a736c-d550-4160-a8bb-619d1d5ea826\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:40:02 crc kubenswrapper[5052]: I1203 11:40:02.523354 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:40:02 crc kubenswrapper[5052]: I1203 11:40:02.740769 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:40:02 crc kubenswrapper[5052]: I1203 11:40:02.755796 5052 scope.go:117] "RemoveContainer" containerID="33a49358f7fcc1bb28274587ea7ed79d3fcea032502f0292b9e672835cb852c4" Dec 03 11:40:02 crc kubenswrapper[5052]: I1203 11:40:02.898832 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 11:40:02 crc kubenswrapper[5052]: I1203 11:40:02.899651 5052 generic.go:334] "Generic (PLEG): container finished" podID="6410c857-4e7d-486c-be0a-6734565014d4" containerID="ac95353be0e40db2feb743167732689ef07e0a7b12dc8b53d9193046d46c4d89" exitCode=0 Dec 03 11:40:02 crc kubenswrapper[5052]: I1203 11:40:02.899734 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-db-create-w672x" event={"ID":"6410c857-4e7d-486c-be0a-6734565014d4","Type":"ContainerDied","Data":"ac95353be0e40db2feb743167732689ef07e0a7b12dc8b53d9193046d46c4d89"} Dec 03 11:40:02 crc kubenswrapper[5052]: W1203 11:40:02.906721 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod75e4b34e_f7b8_4209_95a6_6cc45f6eb57c.slice/crio-66330ee1ed6977ce001254e4d9a3a10bb2366368e36c994e7f2059d8938176af WatchSource:0}: Error finding container 66330ee1ed6977ce001254e4d9a3a10bb2366368e36c994e7f2059d8938176af: Status 404 returned error can't find the container with id 66330ee1ed6977ce001254e4d9a3a10bb2366368e36c994e7f2059d8938176af Dec 03 11:40:02 crc kubenswrapper[5052]: I1203 11:40:02.907148 5052 generic.go:334] "Generic (PLEG): container finished" podID="996bddd2-d361-4a2f-aa3c-773b37ad9b59" containerID="7f0e3ee717dce1717e1541b66434fdf13121b35964b4328bce3b46e0ca09e4ca" exitCode=0 Dec 03 11:40:02 crc kubenswrapper[5052]: I1203 11:40:02.907223 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-db-create-wm8nm" event={"ID":"996bddd2-d361-4a2f-aa3c-773b37ad9b59","Type":"ContainerDied","Data":"7f0e3ee717dce1717e1541b66434fdf13121b35964b4328bce3b46e0ca09e4ca"} Dec 03 11:40:02 crc kubenswrapper[5052]: I1203 11:40:02.908687 5052 generic.go:334] "Generic (PLEG): container finished" podID="066f70dd-4aa5-4b30-b913-2285c642134e" containerID="735a470576dc74f7e9b95db518ca1e458cf10b51cd97c1cbdb4f9e95d02a1414" exitCode=0 Dec 03 11:40:02 crc kubenswrapper[5052]: I1203 11:40:02.908734 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-db-create-4mc7v" event={"ID":"066f70dd-4aa5-4b30-b913-2285c642134e","Type":"ContainerDied","Data":"735a470576dc74f7e9b95db518ca1e458cf10b51cd97c1cbdb4f9e95d02a1414"} Dec 03 11:40:02 crc kubenswrapper[5052]: I1203 11:40:02.912229 5052 generic.go:334] "Generic (PLEG): container finished" podID="26a22bd0-90b3-4fe9-8029-a56c6c7bcdfd" containerID="230b893b58887ce8075576ec97b7c1b839aac29a4c4a0c82f043dbd5a3fbe8eb" exitCode=0 Dec 03 11:40:02 crc kubenswrapper[5052]: I1203 11:40:02.912340 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-73a4-account-create-update-kfmjt" event={"ID":"26a22bd0-90b3-4fe9-8029-a56c6c7bcdfd","Type":"ContainerDied","Data":"230b893b58887ce8075576ec97b7c1b839aac29a4c4a0c82f043dbd5a3fbe8eb"} Dec 03 11:40:02 crc kubenswrapper[5052]: I1203 11:40:02.916367 5052 generic.go:334] "Generic (PLEG): container finished" podID="5f9bdf8c-c97c-477e-bf48-73fe6ddae983" containerID="8353d9ed68998103ae5183e6a229e34b9624453345905d1783ef8d0257b3e95b" exitCode=0 Dec 03 11:40:02 crc kubenswrapper[5052]: I1203 11:40:02.916487 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-cdd4-account-create-update-lbpqk" event={"ID":"5f9bdf8c-c97c-477e-bf48-73fe6ddae983","Type":"ContainerDied","Data":"8353d9ed68998103ae5183e6a229e34b9624453345905d1783ef8d0257b3e95b"} Dec 03 11:40:02 crc kubenswrapper[5052]: I1203 11:40:02.921112 5052 generic.go:334] "Generic (PLEG): container finished" podID="a53c0da6-8b05-40fd-90c7-78cfb701780a" containerID="866dbba5fee7602bed5d8a92bd3bd35bca49efdc6fe77d6689fef78dc3ac2b60" exitCode=0 Dec 03 11:40:02 crc kubenswrapper[5052]: I1203 11:40:02.921146 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-3d99-account-create-update-kqb94" event={"ID":"a53c0da6-8b05-40fd-90c7-78cfb701780a","Type":"ContainerDied","Data":"866dbba5fee7602bed5d8a92bd3bd35bca49efdc6fe77d6689fef78dc3ac2b60"} Dec 03 11:40:03 crc kubenswrapper[5052]: I1203 11:40:03.047953 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 11:40:03 crc kubenswrapper[5052]: I1203 11:40:03.292663 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:40:03 crc kubenswrapper[5052]: I1203 11:40:03.721522 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="502cb03c-d1a5-4dc7-a040-0571a4695379" path="/var/lib/kubelet/pods/502cb03c-d1a5-4dc7-a040-0571a4695379/volumes" Dec 03 11:40:03 crc kubenswrapper[5052]: I1203 11:40:03.722907 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="70e2b274-dd4d-409a-8d45-a09af330f7a3" path="/var/lib/kubelet/pods/70e2b274-dd4d-409a-8d45-a09af330f7a3/volumes" Dec 03 11:40:03 crc kubenswrapper[5052]: I1203 11:40:03.724769 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8bf67244-3ae2-4c00-a15d-4c4180d5f835" path="/var/lib/kubelet/pods/8bf67244-3ae2-4c00-a15d-4c4180d5f835/volumes" Dec 03 11:40:03 crc kubenswrapper[5052]: I1203 11:40:03.943340 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"58df8ad3-3e8f-41ae-88ab-f4960ee9a812","Type":"ContainerStarted","Data":"405dfa56db01c161f963704991ad8eedadeff1077dbec16e14c48589f8e2b30e"} Dec 03 11:40:03 crc kubenswrapper[5052]: I1203 11:40:03.946602 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"665a736c-d550-4160-a8bb-619d1d5ea826","Type":"ContainerStarted","Data":"907dd58d6f36fcd52c737d7df15fd6614bc5eeddfe7082592f83e68d9f16190c"} Dec 03 11:40:03 crc kubenswrapper[5052]: I1203 11:40:03.946651 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"665a736c-d550-4160-a8bb-619d1d5ea826","Type":"ContainerStarted","Data":"3fc2133f2600b0034eb40472da075cfa1df1c9267bd498cfce382d79666ef928"} Dec 03 11:40:03 crc kubenswrapper[5052]: I1203 11:40:03.953040 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"75e4b34e-f7b8-4209-95a6-6cc45f6eb57c","Type":"ContainerStarted","Data":"99cb39696946884496d7905d0018b8b8e349cbda3cdb669d009adf92f668080b"} Dec 03 11:40:03 crc kubenswrapper[5052]: I1203 11:40:03.953082 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"75e4b34e-f7b8-4209-95a6-6cc45f6eb57c","Type":"ContainerStarted","Data":"66330ee1ed6977ce001254e4d9a3a10bb2366368e36c994e7f2059d8938176af"} Dec 03 11:40:04 crc kubenswrapper[5052]: I1203 11:40:04.400732 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-73a4-account-create-update-kfmjt" Dec 03 11:40:04 crc kubenswrapper[5052]: I1203 11:40:04.545161 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lwqbw\" (UniqueName: \"kubernetes.io/projected/26a22bd0-90b3-4fe9-8029-a56c6c7bcdfd-kube-api-access-lwqbw\") pod \"26a22bd0-90b3-4fe9-8029-a56c6c7bcdfd\" (UID: \"26a22bd0-90b3-4fe9-8029-a56c6c7bcdfd\") " Dec 03 11:40:04 crc kubenswrapper[5052]: I1203 11:40:04.545326 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/26a22bd0-90b3-4fe9-8029-a56c6c7bcdfd-operator-scripts\") pod \"26a22bd0-90b3-4fe9-8029-a56c6c7bcdfd\" (UID: \"26a22bd0-90b3-4fe9-8029-a56c6c7bcdfd\") " Dec 03 11:40:04 crc kubenswrapper[5052]: I1203 11:40:04.547932 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/26a22bd0-90b3-4fe9-8029-a56c6c7bcdfd-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "26a22bd0-90b3-4fe9-8029-a56c6c7bcdfd" (UID: "26a22bd0-90b3-4fe9-8029-a56c6c7bcdfd"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:40:04 crc kubenswrapper[5052]: I1203 11:40:04.554587 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/26a22bd0-90b3-4fe9-8029-a56c6c7bcdfd-kube-api-access-lwqbw" (OuterVolumeSpecName: "kube-api-access-lwqbw") pod "26a22bd0-90b3-4fe9-8029-a56c6c7bcdfd" (UID: "26a22bd0-90b3-4fe9-8029-a56c6c7bcdfd"). InnerVolumeSpecName "kube-api-access-lwqbw". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:40:04 crc kubenswrapper[5052]: I1203 11:40:04.556984 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-lwqbw\" (UniqueName: \"kubernetes.io/projected/26a22bd0-90b3-4fe9-8029-a56c6c7bcdfd-kube-api-access-lwqbw\") on node \"crc\" DevicePath \"\"" Dec 03 11:40:04 crc kubenswrapper[5052]: I1203 11:40:04.557031 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/26a22bd0-90b3-4fe9-8029-a56c6c7bcdfd-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:40:04 crc kubenswrapper[5052]: I1203 11:40:04.569489 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-3d99-account-create-update-kqb94" Dec 03 11:40:04 crc kubenswrapper[5052]: I1203 11:40:04.578961 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-db-create-4mc7v" Dec 03 11:40:04 crc kubenswrapper[5052]: I1203 11:40:04.601000 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-db-create-w672x" Dec 03 11:40:04 crc kubenswrapper[5052]: I1203 11:40:04.607224 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-db-create-wm8nm" Dec 03 11:40:04 crc kubenswrapper[5052]: I1203 11:40:04.631734 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-cdd4-account-create-update-lbpqk" Dec 03 11:40:04 crc kubenswrapper[5052]: I1203 11:40:04.660382 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6v4fc\" (UniqueName: \"kubernetes.io/projected/066f70dd-4aa5-4b30-b913-2285c642134e-kube-api-access-6v4fc\") pod \"066f70dd-4aa5-4b30-b913-2285c642134e\" (UID: \"066f70dd-4aa5-4b30-b913-2285c642134e\") " Dec 03 11:40:04 crc kubenswrapper[5052]: I1203 11:40:04.660720 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5qrnn\" (UniqueName: \"kubernetes.io/projected/a53c0da6-8b05-40fd-90c7-78cfb701780a-kube-api-access-5qrnn\") pod \"a53c0da6-8b05-40fd-90c7-78cfb701780a\" (UID: \"a53c0da6-8b05-40fd-90c7-78cfb701780a\") " Dec 03 11:40:04 crc kubenswrapper[5052]: I1203 11:40:04.660767 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/a53c0da6-8b05-40fd-90c7-78cfb701780a-operator-scripts\") pod \"a53c0da6-8b05-40fd-90c7-78cfb701780a\" (UID: \"a53c0da6-8b05-40fd-90c7-78cfb701780a\") " Dec 03 11:40:04 crc kubenswrapper[5052]: I1203 11:40:04.660889 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/066f70dd-4aa5-4b30-b913-2285c642134e-operator-scripts\") pod \"066f70dd-4aa5-4b30-b913-2285c642134e\" (UID: \"066f70dd-4aa5-4b30-b913-2285c642134e\") " Dec 03 11:40:04 crc kubenswrapper[5052]: I1203 11:40:04.664740 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/066f70dd-4aa5-4b30-b913-2285c642134e-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "066f70dd-4aa5-4b30-b913-2285c642134e" (UID: "066f70dd-4aa5-4b30-b913-2285c642134e"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:40:04 crc kubenswrapper[5052]: I1203 11:40:04.666157 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a53c0da6-8b05-40fd-90c7-78cfb701780a-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "a53c0da6-8b05-40fd-90c7-78cfb701780a" (UID: "a53c0da6-8b05-40fd-90c7-78cfb701780a"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:40:04 crc kubenswrapper[5052]: I1203 11:40:04.685877 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a53c0da6-8b05-40fd-90c7-78cfb701780a-kube-api-access-5qrnn" (OuterVolumeSpecName: "kube-api-access-5qrnn") pod "a53c0da6-8b05-40fd-90c7-78cfb701780a" (UID: "a53c0da6-8b05-40fd-90c7-78cfb701780a"). InnerVolumeSpecName "kube-api-access-5qrnn". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:40:04 crc kubenswrapper[5052]: I1203 11:40:04.686457 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/066f70dd-4aa5-4b30-b913-2285c642134e-kube-api-access-6v4fc" (OuterVolumeSpecName: "kube-api-access-6v4fc") pod "066f70dd-4aa5-4b30-b913-2285c642134e" (UID: "066f70dd-4aa5-4b30-b913-2285c642134e"). InnerVolumeSpecName "kube-api-access-6v4fc". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:40:04 crc kubenswrapper[5052]: I1203 11:40:04.767438 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/5f9bdf8c-c97c-477e-bf48-73fe6ddae983-operator-scripts\") pod \"5f9bdf8c-c97c-477e-bf48-73fe6ddae983\" (UID: \"5f9bdf8c-c97c-477e-bf48-73fe6ddae983\") " Dec 03 11:40:04 crc kubenswrapper[5052]: I1203 11:40:04.767548 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fvwzw\" (UniqueName: \"kubernetes.io/projected/5f9bdf8c-c97c-477e-bf48-73fe6ddae983-kube-api-access-fvwzw\") pod \"5f9bdf8c-c97c-477e-bf48-73fe6ddae983\" (UID: \"5f9bdf8c-c97c-477e-bf48-73fe6ddae983\") " Dec 03 11:40:04 crc kubenswrapper[5052]: I1203 11:40:04.767661 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/6410c857-4e7d-486c-be0a-6734565014d4-operator-scripts\") pod \"6410c857-4e7d-486c-be0a-6734565014d4\" (UID: \"6410c857-4e7d-486c-be0a-6734565014d4\") " Dec 03 11:40:04 crc kubenswrapper[5052]: I1203 11:40:04.767783 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xg85d\" (UniqueName: \"kubernetes.io/projected/6410c857-4e7d-486c-be0a-6734565014d4-kube-api-access-xg85d\") pod \"6410c857-4e7d-486c-be0a-6734565014d4\" (UID: \"6410c857-4e7d-486c-be0a-6734565014d4\") " Dec 03 11:40:04 crc kubenswrapper[5052]: I1203 11:40:04.767816 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xfcmg\" (UniqueName: \"kubernetes.io/projected/996bddd2-d361-4a2f-aa3c-773b37ad9b59-kube-api-access-xfcmg\") pod \"996bddd2-d361-4a2f-aa3c-773b37ad9b59\" (UID: \"996bddd2-d361-4a2f-aa3c-773b37ad9b59\") " Dec 03 11:40:04 crc kubenswrapper[5052]: I1203 11:40:04.767878 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/996bddd2-d361-4a2f-aa3c-773b37ad9b59-operator-scripts\") pod \"996bddd2-d361-4a2f-aa3c-773b37ad9b59\" (UID: \"996bddd2-d361-4a2f-aa3c-773b37ad9b59\") " Dec 03 11:40:04 crc kubenswrapper[5052]: I1203 11:40:04.767988 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5f9bdf8c-c97c-477e-bf48-73fe6ddae983-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "5f9bdf8c-c97c-477e-bf48-73fe6ddae983" (UID: "5f9bdf8c-c97c-477e-bf48-73fe6ddae983"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:40:04 crc kubenswrapper[5052]: I1203 11:40:04.768227 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6410c857-4e7d-486c-be0a-6734565014d4-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "6410c857-4e7d-486c-be0a-6734565014d4" (UID: "6410c857-4e7d-486c-be0a-6734565014d4"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:40:04 crc kubenswrapper[5052]: I1203 11:40:04.768825 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/996bddd2-d361-4a2f-aa3c-773b37ad9b59-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "996bddd2-d361-4a2f-aa3c-773b37ad9b59" (UID: "996bddd2-d361-4a2f-aa3c-773b37ad9b59"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:40:04 crc kubenswrapper[5052]: I1203 11:40:04.768922 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-6v4fc\" (UniqueName: \"kubernetes.io/projected/066f70dd-4aa5-4b30-b913-2285c642134e-kube-api-access-6v4fc\") on node \"crc\" DevicePath \"\"" Dec 03 11:40:04 crc kubenswrapper[5052]: I1203 11:40:04.768950 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-5qrnn\" (UniqueName: \"kubernetes.io/projected/a53c0da6-8b05-40fd-90c7-78cfb701780a-kube-api-access-5qrnn\") on node \"crc\" DevicePath \"\"" Dec 03 11:40:04 crc kubenswrapper[5052]: I1203 11:40:04.768968 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/a53c0da6-8b05-40fd-90c7-78cfb701780a-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:40:04 crc kubenswrapper[5052]: I1203 11:40:04.768979 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/5f9bdf8c-c97c-477e-bf48-73fe6ddae983-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:40:04 crc kubenswrapper[5052]: I1203 11:40:04.768990 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/066f70dd-4aa5-4b30-b913-2285c642134e-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:40:04 crc kubenswrapper[5052]: I1203 11:40:04.769000 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/6410c857-4e7d-486c-be0a-6734565014d4-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:40:04 crc kubenswrapper[5052]: I1203 11:40:04.771452 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6410c857-4e7d-486c-be0a-6734565014d4-kube-api-access-xg85d" (OuterVolumeSpecName: "kube-api-access-xg85d") pod "6410c857-4e7d-486c-be0a-6734565014d4" (UID: "6410c857-4e7d-486c-be0a-6734565014d4"). InnerVolumeSpecName "kube-api-access-xg85d". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:40:04 crc kubenswrapper[5052]: I1203 11:40:04.772492 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5f9bdf8c-c97c-477e-bf48-73fe6ddae983-kube-api-access-fvwzw" (OuterVolumeSpecName: "kube-api-access-fvwzw") pod "5f9bdf8c-c97c-477e-bf48-73fe6ddae983" (UID: "5f9bdf8c-c97c-477e-bf48-73fe6ddae983"). InnerVolumeSpecName "kube-api-access-fvwzw". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:40:04 crc kubenswrapper[5052]: I1203 11:40:04.772614 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/996bddd2-d361-4a2f-aa3c-773b37ad9b59-kube-api-access-xfcmg" (OuterVolumeSpecName: "kube-api-access-xfcmg") pod "996bddd2-d361-4a2f-aa3c-773b37ad9b59" (UID: "996bddd2-d361-4a2f-aa3c-773b37ad9b59"). InnerVolumeSpecName "kube-api-access-xfcmg". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:40:04 crc kubenswrapper[5052]: I1203 11:40:04.870659 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-xfcmg\" (UniqueName: \"kubernetes.io/projected/996bddd2-d361-4a2f-aa3c-773b37ad9b59-kube-api-access-xfcmg\") on node \"crc\" DevicePath \"\"" Dec 03 11:40:04 crc kubenswrapper[5052]: I1203 11:40:04.870692 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-xg85d\" (UniqueName: \"kubernetes.io/projected/6410c857-4e7d-486c-be0a-6734565014d4-kube-api-access-xg85d\") on node \"crc\" DevicePath \"\"" Dec 03 11:40:04 crc kubenswrapper[5052]: I1203 11:40:04.870705 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/996bddd2-d361-4a2f-aa3c-773b37ad9b59-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:40:04 crc kubenswrapper[5052]: I1203 11:40:04.870716 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-fvwzw\" (UniqueName: \"kubernetes.io/projected/5f9bdf8c-c97c-477e-bf48-73fe6ddae983-kube-api-access-fvwzw\") on node \"crc\" DevicePath \"\"" Dec 03 11:40:04 crc kubenswrapper[5052]: I1203 11:40:04.990251 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-73a4-account-create-update-kfmjt" Dec 03 11:40:04 crc kubenswrapper[5052]: I1203 11:40:04.990293 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-73a4-account-create-update-kfmjt" event={"ID":"26a22bd0-90b3-4fe9-8029-a56c6c7bcdfd","Type":"ContainerDied","Data":"ebef274a7eb3459db92d5d0c23812c088da58695c21ed98debb674f544bf4f98"} Dec 03 11:40:04 crc kubenswrapper[5052]: I1203 11:40:04.990355 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ebef274a7eb3459db92d5d0c23812c088da58695c21ed98debb674f544bf4f98" Dec 03 11:40:04 crc kubenswrapper[5052]: I1203 11:40:04.995863 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"58df8ad3-3e8f-41ae-88ab-f4960ee9a812","Type":"ContainerStarted","Data":"f3539dc032428625f39c3dbfe94289e5d3c8c337d9a1d057514e60cd543a0578"} Dec 03 11:40:04 crc kubenswrapper[5052]: I1203 11:40:04.995915 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"58df8ad3-3e8f-41ae-88ab-f4960ee9a812","Type":"ContainerStarted","Data":"5890f3dae14a30d1a826ef2b8aa5acf1257a4e89db16c5fb03adb8200f94f437"} Dec 03 11:40:04 crc kubenswrapper[5052]: I1203 11:40:04.998692 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-db-create-w672x" event={"ID":"6410c857-4e7d-486c-be0a-6734565014d4","Type":"ContainerDied","Data":"32906a5f17bed523a04efda052e098b62e93da02f83d13b806888cf190a583b9"} Dec 03 11:40:04 crc kubenswrapper[5052]: I1203 11:40:04.999689 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="32906a5f17bed523a04efda052e098b62e93da02f83d13b806888cf190a583b9" Dec 03 11:40:04 crc kubenswrapper[5052]: I1203 11:40:04.998815 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-db-create-w672x" Dec 03 11:40:05 crc kubenswrapper[5052]: I1203 11:40:05.001659 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"665a736c-d550-4160-a8bb-619d1d5ea826","Type":"ContainerStarted","Data":"25c8b36672104ddddb9cda8135c26753231310450ea8bff9f6700b865bc3d056"} Dec 03 11:40:05 crc kubenswrapper[5052]: I1203 11:40:05.005662 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"75e4b34e-f7b8-4209-95a6-6cc45f6eb57c","Type":"ContainerStarted","Data":"2b213199dec26418337b9d73443612a12a1ce69360675968db75a8f6aba268ae"} Dec 03 11:40:05 crc kubenswrapper[5052]: I1203 11:40:05.010792 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-cdd4-account-create-update-lbpqk" event={"ID":"5f9bdf8c-c97c-477e-bf48-73fe6ddae983","Type":"ContainerDied","Data":"909fac460000c720d2ca0d5f9584db78b78a09588b187e203e6c62f18a5bbc6c"} Dec 03 11:40:05 crc kubenswrapper[5052]: I1203 11:40:05.010849 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="909fac460000c720d2ca0d5f9584db78b78a09588b187e203e6c62f18a5bbc6c" Dec 03 11:40:05 crc kubenswrapper[5052]: I1203 11:40:05.010969 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-cdd4-account-create-update-lbpqk" Dec 03 11:40:05 crc kubenswrapper[5052]: I1203 11:40:05.013941 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-db-create-wm8nm" event={"ID":"996bddd2-d361-4a2f-aa3c-773b37ad9b59","Type":"ContainerDied","Data":"028954ede6805438c4e1592920671530b30ec68dab4b09dadd25e39649f4fa5b"} Dec 03 11:40:05 crc kubenswrapper[5052]: I1203 11:40:05.014018 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="028954ede6805438c4e1592920671530b30ec68dab4b09dadd25e39649f4fa5b" Dec 03 11:40:05 crc kubenswrapper[5052]: I1203 11:40:05.014143 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-db-create-wm8nm" Dec 03 11:40:05 crc kubenswrapper[5052]: I1203 11:40:05.018087 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-db-create-4mc7v" event={"ID":"066f70dd-4aa5-4b30-b913-2285c642134e","Type":"ContainerDied","Data":"418e72dc8dd3adcadb952eb547dde16a4372698dbe49a1984725a32ec2a33632"} Dec 03 11:40:05 crc kubenswrapper[5052]: I1203 11:40:05.018137 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="418e72dc8dd3adcadb952eb547dde16a4372698dbe49a1984725a32ec2a33632" Dec 03 11:40:05 crc kubenswrapper[5052]: I1203 11:40:05.018257 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-db-create-4mc7v" Dec 03 11:40:05 crc kubenswrapper[5052]: I1203 11:40:05.035806 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/glance-default-internal-api-0" podStartSLOduration=4.035787484 podStartE2EDuration="4.035787484s" podCreationTimestamp="2025-12-03 11:40:01 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:40:05.02631666 +0000 UTC m=+1195.614558414" watchObservedRunningTime="2025-12-03 11:40:05.035787484 +0000 UTC m=+1195.624029248" Dec 03 11:40:05 crc kubenswrapper[5052]: I1203 11:40:05.043921 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-3d99-account-create-update-kqb94" event={"ID":"a53c0da6-8b05-40fd-90c7-78cfb701780a","Type":"ContainerDied","Data":"0b6241e6c7698449e8c9cd94ed0f6d1ed0b4578cd7f683ed47b50d6e0a7aeb3e"} Dec 03 11:40:05 crc kubenswrapper[5052]: I1203 11:40:05.043968 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0b6241e6c7698449e8c9cd94ed0f6d1ed0b4578cd7f683ed47b50d6e0a7aeb3e" Dec 03 11:40:05 crc kubenswrapper[5052]: I1203 11:40:05.043997 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-3d99-account-create-update-kqb94" Dec 03 11:40:05 crc kubenswrapper[5052]: I1203 11:40:05.077938 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/glance-default-external-api-0" podStartSLOduration=4.07790948 podStartE2EDuration="4.07790948s" podCreationTimestamp="2025-12-03 11:40:01 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:40:05.065695897 +0000 UTC m=+1195.653937661" watchObservedRunningTime="2025-12-03 11:40:05.07790948 +0000 UTC m=+1195.666151254" Dec 03 11:40:07 crc kubenswrapper[5052]: I1203 11:40:07.072222 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"58df8ad3-3e8f-41ae-88ab-f4960ee9a812","Type":"ContainerStarted","Data":"c529dc6ad0ca5c24028d4709078b583b56b2d4a252dcffc6e21892403aea3b12"} Dec 03 11:40:08 crc kubenswrapper[5052]: I1203 11:40:08.529794 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-db-sync-t7mx5"] Dec 03 11:40:08 crc kubenswrapper[5052]: E1203 11:40:08.530729 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="996bddd2-d361-4a2f-aa3c-773b37ad9b59" containerName="mariadb-database-create" Dec 03 11:40:08 crc kubenswrapper[5052]: I1203 11:40:08.530742 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="996bddd2-d361-4a2f-aa3c-773b37ad9b59" containerName="mariadb-database-create" Dec 03 11:40:08 crc kubenswrapper[5052]: E1203 11:40:08.530752 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="a53c0da6-8b05-40fd-90c7-78cfb701780a" containerName="mariadb-account-create-update" Dec 03 11:40:08 crc kubenswrapper[5052]: I1203 11:40:08.530758 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="a53c0da6-8b05-40fd-90c7-78cfb701780a" containerName="mariadb-account-create-update" Dec 03 11:40:08 crc kubenswrapper[5052]: E1203 11:40:08.530773 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="5f9bdf8c-c97c-477e-bf48-73fe6ddae983" containerName="mariadb-account-create-update" Dec 03 11:40:08 crc kubenswrapper[5052]: I1203 11:40:08.530778 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f9bdf8c-c97c-477e-bf48-73fe6ddae983" containerName="mariadb-account-create-update" Dec 03 11:40:08 crc kubenswrapper[5052]: E1203 11:40:08.530801 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="6410c857-4e7d-486c-be0a-6734565014d4" containerName="mariadb-database-create" Dec 03 11:40:08 crc kubenswrapper[5052]: I1203 11:40:08.530807 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="6410c857-4e7d-486c-be0a-6734565014d4" containerName="mariadb-database-create" Dec 03 11:40:08 crc kubenswrapper[5052]: E1203 11:40:08.530824 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="066f70dd-4aa5-4b30-b913-2285c642134e" containerName="mariadb-database-create" Dec 03 11:40:08 crc kubenswrapper[5052]: I1203 11:40:08.530829 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="066f70dd-4aa5-4b30-b913-2285c642134e" containerName="mariadb-database-create" Dec 03 11:40:08 crc kubenswrapper[5052]: E1203 11:40:08.530838 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="26a22bd0-90b3-4fe9-8029-a56c6c7bcdfd" containerName="mariadb-account-create-update" Dec 03 11:40:08 crc kubenswrapper[5052]: I1203 11:40:08.530843 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="26a22bd0-90b3-4fe9-8029-a56c6c7bcdfd" containerName="mariadb-account-create-update" Dec 03 11:40:08 crc kubenswrapper[5052]: I1203 11:40:08.531047 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="066f70dd-4aa5-4b30-b913-2285c642134e" containerName="mariadb-database-create" Dec 03 11:40:08 crc kubenswrapper[5052]: I1203 11:40:08.531058 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="5f9bdf8c-c97c-477e-bf48-73fe6ddae983" containerName="mariadb-account-create-update" Dec 03 11:40:08 crc kubenswrapper[5052]: I1203 11:40:08.531067 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="a53c0da6-8b05-40fd-90c7-78cfb701780a" containerName="mariadb-account-create-update" Dec 03 11:40:08 crc kubenswrapper[5052]: I1203 11:40:08.531081 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="996bddd2-d361-4a2f-aa3c-773b37ad9b59" containerName="mariadb-database-create" Dec 03 11:40:08 crc kubenswrapper[5052]: I1203 11:40:08.531088 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="6410c857-4e7d-486c-be0a-6734565014d4" containerName="mariadb-database-create" Dec 03 11:40:08 crc kubenswrapper[5052]: I1203 11:40:08.531105 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="26a22bd0-90b3-4fe9-8029-a56c6c7bcdfd" containerName="mariadb-account-create-update" Dec 03 11:40:08 crc kubenswrapper[5052]: I1203 11:40:08.531783 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-t7mx5" Dec 03 11:40:08 crc kubenswrapper[5052]: I1203 11:40:08.537124 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell0-conductor-scripts" Dec 03 11:40:08 crc kubenswrapper[5052]: I1203 11:40:08.538457 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-nova-dockercfg-6l7ft" Dec 03 11:40:08 crc kubenswrapper[5052]: I1203 11:40:08.539014 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell0-conductor-config-data" Dec 03 11:40:08 crc kubenswrapper[5052]: I1203 11:40:08.541256 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-db-sync-t7mx5"] Dec 03 11:40:08 crc kubenswrapper[5052]: I1203 11:40:08.639931 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8sbn6\" (UniqueName: \"kubernetes.io/projected/0059df68-09bd-434b-b3a8-e2f56cc8bda0-kube-api-access-8sbn6\") pod \"nova-cell0-conductor-db-sync-t7mx5\" (UID: \"0059df68-09bd-434b-b3a8-e2f56cc8bda0\") " pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-t7mx5" Dec 03 11:40:08 crc kubenswrapper[5052]: I1203 11:40:08.640000 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0059df68-09bd-434b-b3a8-e2f56cc8bda0-combined-ca-bundle\") pod \"nova-cell0-conductor-db-sync-t7mx5\" (UID: \"0059df68-09bd-434b-b3a8-e2f56cc8bda0\") " pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-t7mx5" Dec 03 11:40:08 crc kubenswrapper[5052]: I1203 11:40:08.640029 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/0059df68-09bd-434b-b3a8-e2f56cc8bda0-config-data\") pod \"nova-cell0-conductor-db-sync-t7mx5\" (UID: \"0059df68-09bd-434b-b3a8-e2f56cc8bda0\") " pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-t7mx5" Dec 03 11:40:08 crc kubenswrapper[5052]: I1203 11:40:08.640075 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/0059df68-09bd-434b-b3a8-e2f56cc8bda0-scripts\") pod \"nova-cell0-conductor-db-sync-t7mx5\" (UID: \"0059df68-09bd-434b-b3a8-e2f56cc8bda0\") " pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-t7mx5" Dec 03 11:40:08 crc kubenswrapper[5052]: I1203 11:40:08.741837 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/0059df68-09bd-434b-b3a8-e2f56cc8bda0-scripts\") pod \"nova-cell0-conductor-db-sync-t7mx5\" (UID: \"0059df68-09bd-434b-b3a8-e2f56cc8bda0\") " pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-t7mx5" Dec 03 11:40:08 crc kubenswrapper[5052]: I1203 11:40:08.741999 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-8sbn6\" (UniqueName: \"kubernetes.io/projected/0059df68-09bd-434b-b3a8-e2f56cc8bda0-kube-api-access-8sbn6\") pod \"nova-cell0-conductor-db-sync-t7mx5\" (UID: \"0059df68-09bd-434b-b3a8-e2f56cc8bda0\") " pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-t7mx5" Dec 03 11:40:08 crc kubenswrapper[5052]: I1203 11:40:08.742057 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0059df68-09bd-434b-b3a8-e2f56cc8bda0-combined-ca-bundle\") pod \"nova-cell0-conductor-db-sync-t7mx5\" (UID: \"0059df68-09bd-434b-b3a8-e2f56cc8bda0\") " pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-t7mx5" Dec 03 11:40:08 crc kubenswrapper[5052]: I1203 11:40:08.742096 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/0059df68-09bd-434b-b3a8-e2f56cc8bda0-config-data\") pod \"nova-cell0-conductor-db-sync-t7mx5\" (UID: \"0059df68-09bd-434b-b3a8-e2f56cc8bda0\") " pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-t7mx5" Dec 03 11:40:08 crc kubenswrapper[5052]: I1203 11:40:08.747306 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/0059df68-09bd-434b-b3a8-e2f56cc8bda0-scripts\") pod \"nova-cell0-conductor-db-sync-t7mx5\" (UID: \"0059df68-09bd-434b-b3a8-e2f56cc8bda0\") " pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-t7mx5" Dec 03 11:40:08 crc kubenswrapper[5052]: I1203 11:40:08.747465 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0059df68-09bd-434b-b3a8-e2f56cc8bda0-combined-ca-bundle\") pod \"nova-cell0-conductor-db-sync-t7mx5\" (UID: \"0059df68-09bd-434b-b3a8-e2f56cc8bda0\") " pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-t7mx5" Dec 03 11:40:08 crc kubenswrapper[5052]: I1203 11:40:08.747677 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/0059df68-09bd-434b-b3a8-e2f56cc8bda0-config-data\") pod \"nova-cell0-conductor-db-sync-t7mx5\" (UID: \"0059df68-09bd-434b-b3a8-e2f56cc8bda0\") " pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-t7mx5" Dec 03 11:40:08 crc kubenswrapper[5052]: I1203 11:40:08.769307 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-8sbn6\" (UniqueName: \"kubernetes.io/projected/0059df68-09bd-434b-b3a8-e2f56cc8bda0-kube-api-access-8sbn6\") pod \"nova-cell0-conductor-db-sync-t7mx5\" (UID: \"0059df68-09bd-434b-b3a8-e2f56cc8bda0\") " pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-t7mx5" Dec 03 11:40:08 crc kubenswrapper[5052]: I1203 11:40:08.850013 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-t7mx5" Dec 03 11:40:09 crc kubenswrapper[5052]: I1203 11:40:09.096738 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"58df8ad3-3e8f-41ae-88ab-f4960ee9a812","Type":"ContainerStarted","Data":"8d311372f37bbc70148fe99fa5548e9d5fa34a191a3d5bbd473334629ce50efa"} Dec 03 11:40:09 crc kubenswrapper[5052]: I1203 11:40:09.098296 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:40:09 crc kubenswrapper[5052]: I1203 11:40:09.132252 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/ceilometer-0" podStartSLOduration=3.378099818 podStartE2EDuration="8.132224968s" podCreationTimestamp="2025-12-03 11:40:01 +0000 UTC" firstStartedPulling="2025-12-03 11:40:03.318000976 +0000 UTC m=+1193.906242740" lastFinishedPulling="2025-12-03 11:40:08.072126106 +0000 UTC m=+1198.660367890" observedRunningTime="2025-12-03 11:40:09.122907219 +0000 UTC m=+1199.711148983" watchObservedRunningTime="2025-12-03 11:40:09.132224968 +0000 UTC m=+1199.720466742" Dec 03 11:40:09 crc kubenswrapper[5052]: I1203 11:40:09.309252 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-db-sync-t7mx5"] Dec 03 11:40:10 crc kubenswrapper[5052]: I1203 11:40:10.108464 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-t7mx5" event={"ID":"0059df68-09bd-434b-b3a8-e2f56cc8bda0","Type":"ContainerStarted","Data":"96ae7c2a68fb7481f8f4c0d21461d082a7f7316a165efef28b5f72f6349b1368"} Dec 03 11:40:11 crc kubenswrapper[5052]: I1203 11:40:11.777794 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:40:12 crc kubenswrapper[5052]: I1203 11:40:12.125830 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="58df8ad3-3e8f-41ae-88ab-f4960ee9a812" containerName="ceilometer-central-agent" containerID="cri-o://5890f3dae14a30d1a826ef2b8aa5acf1257a4e89db16c5fb03adb8200f94f437" gracePeriod=30 Dec 03 11:40:12 crc kubenswrapper[5052]: I1203 11:40:12.125853 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="58df8ad3-3e8f-41ae-88ab-f4960ee9a812" containerName="proxy-httpd" containerID="cri-o://8d311372f37bbc70148fe99fa5548e9d5fa34a191a3d5bbd473334629ce50efa" gracePeriod=30 Dec 03 11:40:12 crc kubenswrapper[5052]: I1203 11:40:12.125907 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="58df8ad3-3e8f-41ae-88ab-f4960ee9a812" containerName="ceilometer-notification-agent" containerID="cri-o://f3539dc032428625f39c3dbfe94289e5d3c8c337d9a1d057514e60cd543a0578" gracePeriod=30 Dec 03 11:40:12 crc kubenswrapper[5052]: I1203 11:40:12.125911 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="58df8ad3-3e8f-41ae-88ab-f4960ee9a812" containerName="sg-core" containerID="cri-o://c529dc6ad0ca5c24028d4709078b583b56b2d4a252dcffc6e21892403aea3b12" gracePeriod=30 Dec 03 11:40:12 crc kubenswrapper[5052]: I1203 11:40:12.348745 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:40:12 crc kubenswrapper[5052]: I1203 11:40:12.349940 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:40:12 crc kubenswrapper[5052]: I1203 11:40:12.406229 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:40:12 crc kubenswrapper[5052]: I1203 11:40:12.421091 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:40:12 crc kubenswrapper[5052]: I1203 11:40:12.524823 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:40:12 crc kubenswrapper[5052]: I1203 11:40:12.524906 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:40:12 crc kubenswrapper[5052]: I1203 11:40:12.570774 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:40:12 crc kubenswrapper[5052]: I1203 11:40:12.585578 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:40:13 crc kubenswrapper[5052]: I1203 11:40:13.138018 5052 generic.go:334] "Generic (PLEG): container finished" podID="58df8ad3-3e8f-41ae-88ab-f4960ee9a812" containerID="8d311372f37bbc70148fe99fa5548e9d5fa34a191a3d5bbd473334629ce50efa" exitCode=0 Dec 03 11:40:13 crc kubenswrapper[5052]: I1203 11:40:13.138062 5052 generic.go:334] "Generic (PLEG): container finished" podID="58df8ad3-3e8f-41ae-88ab-f4960ee9a812" containerID="c529dc6ad0ca5c24028d4709078b583b56b2d4a252dcffc6e21892403aea3b12" exitCode=2 Dec 03 11:40:13 crc kubenswrapper[5052]: I1203 11:40:13.138072 5052 generic.go:334] "Generic (PLEG): container finished" podID="58df8ad3-3e8f-41ae-88ab-f4960ee9a812" containerID="f3539dc032428625f39c3dbfe94289e5d3c8c337d9a1d057514e60cd543a0578" exitCode=0 Dec 03 11:40:13 crc kubenswrapper[5052]: I1203 11:40:13.138082 5052 generic.go:334] "Generic (PLEG): container finished" podID="58df8ad3-3e8f-41ae-88ab-f4960ee9a812" containerID="5890f3dae14a30d1a826ef2b8aa5acf1257a4e89db16c5fb03adb8200f94f437" exitCode=0 Dec 03 11:40:13 crc kubenswrapper[5052]: I1203 11:40:13.138232 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"58df8ad3-3e8f-41ae-88ab-f4960ee9a812","Type":"ContainerDied","Data":"8d311372f37bbc70148fe99fa5548e9d5fa34a191a3d5bbd473334629ce50efa"} Dec 03 11:40:13 crc kubenswrapper[5052]: I1203 11:40:13.138263 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"58df8ad3-3e8f-41ae-88ab-f4960ee9a812","Type":"ContainerDied","Data":"c529dc6ad0ca5c24028d4709078b583b56b2d4a252dcffc6e21892403aea3b12"} Dec 03 11:40:13 crc kubenswrapper[5052]: I1203 11:40:13.138278 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"58df8ad3-3e8f-41ae-88ab-f4960ee9a812","Type":"ContainerDied","Data":"f3539dc032428625f39c3dbfe94289e5d3c8c337d9a1d057514e60cd543a0578"} Dec 03 11:40:13 crc kubenswrapper[5052]: I1203 11:40:13.138291 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"58df8ad3-3e8f-41ae-88ab-f4960ee9a812","Type":"ContainerDied","Data":"5890f3dae14a30d1a826ef2b8aa5acf1257a4e89db16c5fb03adb8200f94f437"} Dec 03 11:40:13 crc kubenswrapper[5052]: I1203 11:40:13.138350 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:40:13 crc kubenswrapper[5052]: I1203 11:40:13.138728 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:40:13 crc kubenswrapper[5052]: I1203 11:40:13.138774 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:40:13 crc kubenswrapper[5052]: I1203 11:40:13.138786 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:40:15 crc kubenswrapper[5052]: I1203 11:40:15.159297 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:40:15 crc kubenswrapper[5052]: I1203 11:40:15.159816 5052 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Dec 03 11:40:15 crc kubenswrapper[5052]: I1203 11:40:15.296959 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:40:15 crc kubenswrapper[5052]: I1203 11:40:15.297330 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:40:15 crc kubenswrapper[5052]: I1203 11:40:15.336860 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:40:17 crc kubenswrapper[5052]: I1203 11:40:17.889968 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:40:18 crc kubenswrapper[5052]: I1203 11:40:18.034463 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ztj8g\" (UniqueName: \"kubernetes.io/projected/58df8ad3-3e8f-41ae-88ab-f4960ee9a812-kube-api-access-ztj8g\") pod \"58df8ad3-3e8f-41ae-88ab-f4960ee9a812\" (UID: \"58df8ad3-3e8f-41ae-88ab-f4960ee9a812\") " Dec 03 11:40:18 crc kubenswrapper[5052]: I1203 11:40:18.034591 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/58df8ad3-3e8f-41ae-88ab-f4960ee9a812-sg-core-conf-yaml\") pod \"58df8ad3-3e8f-41ae-88ab-f4960ee9a812\" (UID: \"58df8ad3-3e8f-41ae-88ab-f4960ee9a812\") " Dec 03 11:40:18 crc kubenswrapper[5052]: I1203 11:40:18.034669 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/58df8ad3-3e8f-41ae-88ab-f4960ee9a812-combined-ca-bundle\") pod \"58df8ad3-3e8f-41ae-88ab-f4960ee9a812\" (UID: \"58df8ad3-3e8f-41ae-88ab-f4960ee9a812\") " Dec 03 11:40:18 crc kubenswrapper[5052]: I1203 11:40:18.034744 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/58df8ad3-3e8f-41ae-88ab-f4960ee9a812-config-data\") pod \"58df8ad3-3e8f-41ae-88ab-f4960ee9a812\" (UID: \"58df8ad3-3e8f-41ae-88ab-f4960ee9a812\") " Dec 03 11:40:18 crc kubenswrapper[5052]: I1203 11:40:18.034783 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/58df8ad3-3e8f-41ae-88ab-f4960ee9a812-scripts\") pod \"58df8ad3-3e8f-41ae-88ab-f4960ee9a812\" (UID: \"58df8ad3-3e8f-41ae-88ab-f4960ee9a812\") " Dec 03 11:40:18 crc kubenswrapper[5052]: I1203 11:40:18.034922 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/58df8ad3-3e8f-41ae-88ab-f4960ee9a812-run-httpd\") pod \"58df8ad3-3e8f-41ae-88ab-f4960ee9a812\" (UID: \"58df8ad3-3e8f-41ae-88ab-f4960ee9a812\") " Dec 03 11:40:18 crc kubenswrapper[5052]: I1203 11:40:18.034942 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/58df8ad3-3e8f-41ae-88ab-f4960ee9a812-log-httpd\") pod \"58df8ad3-3e8f-41ae-88ab-f4960ee9a812\" (UID: \"58df8ad3-3e8f-41ae-88ab-f4960ee9a812\") " Dec 03 11:40:18 crc kubenswrapper[5052]: I1203 11:40:18.035804 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/58df8ad3-3e8f-41ae-88ab-f4960ee9a812-log-httpd" (OuterVolumeSpecName: "log-httpd") pod "58df8ad3-3e8f-41ae-88ab-f4960ee9a812" (UID: "58df8ad3-3e8f-41ae-88ab-f4960ee9a812"). InnerVolumeSpecName "log-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:40:18 crc kubenswrapper[5052]: I1203 11:40:18.035964 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/58df8ad3-3e8f-41ae-88ab-f4960ee9a812-run-httpd" (OuterVolumeSpecName: "run-httpd") pod "58df8ad3-3e8f-41ae-88ab-f4960ee9a812" (UID: "58df8ad3-3e8f-41ae-88ab-f4960ee9a812"). InnerVolumeSpecName "run-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:40:18 crc kubenswrapper[5052]: I1203 11:40:18.040978 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/58df8ad3-3e8f-41ae-88ab-f4960ee9a812-kube-api-access-ztj8g" (OuterVolumeSpecName: "kube-api-access-ztj8g") pod "58df8ad3-3e8f-41ae-88ab-f4960ee9a812" (UID: "58df8ad3-3e8f-41ae-88ab-f4960ee9a812"). InnerVolumeSpecName "kube-api-access-ztj8g". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:40:18 crc kubenswrapper[5052]: I1203 11:40:18.040975 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/58df8ad3-3e8f-41ae-88ab-f4960ee9a812-scripts" (OuterVolumeSpecName: "scripts") pod "58df8ad3-3e8f-41ae-88ab-f4960ee9a812" (UID: "58df8ad3-3e8f-41ae-88ab-f4960ee9a812"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:40:18 crc kubenswrapper[5052]: I1203 11:40:18.073080 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/58df8ad3-3e8f-41ae-88ab-f4960ee9a812-sg-core-conf-yaml" (OuterVolumeSpecName: "sg-core-conf-yaml") pod "58df8ad3-3e8f-41ae-88ab-f4960ee9a812" (UID: "58df8ad3-3e8f-41ae-88ab-f4960ee9a812"). InnerVolumeSpecName "sg-core-conf-yaml". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:40:18 crc kubenswrapper[5052]: I1203 11:40:18.112144 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/58df8ad3-3e8f-41ae-88ab-f4960ee9a812-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "58df8ad3-3e8f-41ae-88ab-f4960ee9a812" (UID: "58df8ad3-3e8f-41ae-88ab-f4960ee9a812"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:40:18 crc kubenswrapper[5052]: I1203 11:40:18.136798 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/58df8ad3-3e8f-41ae-88ab-f4960ee9a812-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:40:18 crc kubenswrapper[5052]: I1203 11:40:18.136835 5052 reconciler_common.go:293] "Volume detached for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/58df8ad3-3e8f-41ae-88ab-f4960ee9a812-run-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 11:40:18 crc kubenswrapper[5052]: I1203 11:40:18.136848 5052 reconciler_common.go:293] "Volume detached for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/58df8ad3-3e8f-41ae-88ab-f4960ee9a812-log-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 11:40:18 crc kubenswrapper[5052]: I1203 11:40:18.136886 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-ztj8g\" (UniqueName: \"kubernetes.io/projected/58df8ad3-3e8f-41ae-88ab-f4960ee9a812-kube-api-access-ztj8g\") on node \"crc\" DevicePath \"\"" Dec 03 11:40:18 crc kubenswrapper[5052]: I1203 11:40:18.136904 5052 reconciler_common.go:293] "Volume detached for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/58df8ad3-3e8f-41ae-88ab-f4960ee9a812-sg-core-conf-yaml\") on node \"crc\" DevicePath \"\"" Dec 03 11:40:18 crc kubenswrapper[5052]: I1203 11:40:18.136915 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/58df8ad3-3e8f-41ae-88ab-f4960ee9a812-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:40:18 crc kubenswrapper[5052]: I1203 11:40:18.139296 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/58df8ad3-3e8f-41ae-88ab-f4960ee9a812-config-data" (OuterVolumeSpecName: "config-data") pod "58df8ad3-3e8f-41ae-88ab-f4960ee9a812" (UID: "58df8ad3-3e8f-41ae-88ab-f4960ee9a812"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:40:18 crc kubenswrapper[5052]: I1203 11:40:18.190197 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"58df8ad3-3e8f-41ae-88ab-f4960ee9a812","Type":"ContainerDied","Data":"405dfa56db01c161f963704991ad8eedadeff1077dbec16e14c48589f8e2b30e"} Dec 03 11:40:18 crc kubenswrapper[5052]: I1203 11:40:18.190235 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:40:18 crc kubenswrapper[5052]: I1203 11:40:18.190589 5052 scope.go:117] "RemoveContainer" containerID="8d311372f37bbc70148fe99fa5548e9d5fa34a191a3d5bbd473334629ce50efa" Dec 03 11:40:18 crc kubenswrapper[5052]: I1203 11:40:18.192789 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-t7mx5" event={"ID":"0059df68-09bd-434b-b3a8-e2f56cc8bda0","Type":"ContainerStarted","Data":"74b7843216af5eaa832491d05cc3f8cd094a9315e804a6f571a7f09fd5df0acf"} Dec 03 11:40:18 crc kubenswrapper[5052]: I1203 11:40:18.209720 5052 scope.go:117] "RemoveContainer" containerID="c529dc6ad0ca5c24028d4709078b583b56b2d4a252dcffc6e21892403aea3b12" Dec 03 11:40:18 crc kubenswrapper[5052]: I1203 11:40:18.217583 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-t7mx5" podStartSLOduration=1.862362145 podStartE2EDuration="10.21756425s" podCreationTimestamp="2025-12-03 11:40:08 +0000 UTC" firstStartedPulling="2025-12-03 11:40:09.317759804 +0000 UTC m=+1199.906001578" lastFinishedPulling="2025-12-03 11:40:17.672961919 +0000 UTC m=+1208.261203683" observedRunningTime="2025-12-03 11:40:18.212466773 +0000 UTC m=+1208.800708527" watchObservedRunningTime="2025-12-03 11:40:18.21756425 +0000 UTC m=+1208.805806014" Dec 03 11:40:18 crc kubenswrapper[5052]: I1203 11:40:18.228872 5052 scope.go:117] "RemoveContainer" containerID="f3539dc032428625f39c3dbfe94289e5d3c8c337d9a1d057514e60cd543a0578" Dec 03 11:40:18 crc kubenswrapper[5052]: I1203 11:40:18.238832 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/58df8ad3-3e8f-41ae-88ab-f4960ee9a812-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:40:18 crc kubenswrapper[5052]: I1203 11:40:18.247761 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:40:18 crc kubenswrapper[5052]: I1203 11:40:18.259140 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:40:18 crc kubenswrapper[5052]: I1203 11:40:18.259247 5052 scope.go:117] "RemoveContainer" containerID="5890f3dae14a30d1a826ef2b8aa5acf1257a4e89db16c5fb03adb8200f94f437" Dec 03 11:40:18 crc kubenswrapper[5052]: I1203 11:40:18.270269 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:40:18 crc kubenswrapper[5052]: E1203 11:40:18.270778 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="58df8ad3-3e8f-41ae-88ab-f4960ee9a812" containerName="ceilometer-central-agent" Dec 03 11:40:18 crc kubenswrapper[5052]: I1203 11:40:18.270804 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="58df8ad3-3e8f-41ae-88ab-f4960ee9a812" containerName="ceilometer-central-agent" Dec 03 11:40:18 crc kubenswrapper[5052]: E1203 11:40:18.270821 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="58df8ad3-3e8f-41ae-88ab-f4960ee9a812" containerName="ceilometer-notification-agent" Dec 03 11:40:18 crc kubenswrapper[5052]: I1203 11:40:18.270831 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="58df8ad3-3e8f-41ae-88ab-f4960ee9a812" containerName="ceilometer-notification-agent" Dec 03 11:40:18 crc kubenswrapper[5052]: E1203 11:40:18.270851 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="58df8ad3-3e8f-41ae-88ab-f4960ee9a812" containerName="proxy-httpd" Dec 03 11:40:18 crc kubenswrapper[5052]: I1203 11:40:18.270859 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="58df8ad3-3e8f-41ae-88ab-f4960ee9a812" containerName="proxy-httpd" Dec 03 11:40:18 crc kubenswrapper[5052]: E1203 11:40:18.270876 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="58df8ad3-3e8f-41ae-88ab-f4960ee9a812" containerName="sg-core" Dec 03 11:40:18 crc kubenswrapper[5052]: I1203 11:40:18.270885 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="58df8ad3-3e8f-41ae-88ab-f4960ee9a812" containerName="sg-core" Dec 03 11:40:18 crc kubenswrapper[5052]: I1203 11:40:18.271108 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="58df8ad3-3e8f-41ae-88ab-f4960ee9a812" containerName="proxy-httpd" Dec 03 11:40:18 crc kubenswrapper[5052]: I1203 11:40:18.271123 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="58df8ad3-3e8f-41ae-88ab-f4960ee9a812" containerName="ceilometer-notification-agent" Dec 03 11:40:18 crc kubenswrapper[5052]: I1203 11:40:18.271136 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="58df8ad3-3e8f-41ae-88ab-f4960ee9a812" containerName="ceilometer-central-agent" Dec 03 11:40:18 crc kubenswrapper[5052]: I1203 11:40:18.271148 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="58df8ad3-3e8f-41ae-88ab-f4960ee9a812" containerName="sg-core" Dec 03 11:40:18 crc kubenswrapper[5052]: I1203 11:40:18.273248 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:40:18 crc kubenswrapper[5052]: I1203 11:40:18.276459 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-scripts" Dec 03 11:40:18 crc kubenswrapper[5052]: I1203 11:40:18.278681 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-config-data" Dec 03 11:40:18 crc kubenswrapper[5052]: I1203 11:40:18.286554 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:40:18 crc kubenswrapper[5052]: I1203 11:40:18.341726 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/5deeda32-3ab5-4e5a-a41a-320996d2a162-run-httpd\") pod \"ceilometer-0\" (UID: \"5deeda32-3ab5-4e5a-a41a-320996d2a162\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:40:18 crc kubenswrapper[5052]: I1203 11:40:18.341814 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5deeda32-3ab5-4e5a-a41a-320996d2a162-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"5deeda32-3ab5-4e5a-a41a-320996d2a162\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:40:18 crc kubenswrapper[5052]: I1203 11:40:18.341862 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/5deeda32-3ab5-4e5a-a41a-320996d2a162-config-data\") pod \"ceilometer-0\" (UID: \"5deeda32-3ab5-4e5a-a41a-320996d2a162\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:40:18 crc kubenswrapper[5052]: I1203 11:40:18.342035 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/5deeda32-3ab5-4e5a-a41a-320996d2a162-log-httpd\") pod \"ceilometer-0\" (UID: \"5deeda32-3ab5-4e5a-a41a-320996d2a162\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:40:18 crc kubenswrapper[5052]: I1203 11:40:18.342200 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/5deeda32-3ab5-4e5a-a41a-320996d2a162-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"5deeda32-3ab5-4e5a-a41a-320996d2a162\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:40:18 crc kubenswrapper[5052]: I1203 11:40:18.342257 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/5deeda32-3ab5-4e5a-a41a-320996d2a162-scripts\") pod \"ceilometer-0\" (UID: \"5deeda32-3ab5-4e5a-a41a-320996d2a162\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:40:18 crc kubenswrapper[5052]: I1203 11:40:18.342291 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7smpm\" (UniqueName: \"kubernetes.io/projected/5deeda32-3ab5-4e5a-a41a-320996d2a162-kube-api-access-7smpm\") pod \"ceilometer-0\" (UID: \"5deeda32-3ab5-4e5a-a41a-320996d2a162\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:40:18 crc kubenswrapper[5052]: I1203 11:40:18.447685 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/5deeda32-3ab5-4e5a-a41a-320996d2a162-run-httpd\") pod \"ceilometer-0\" (UID: \"5deeda32-3ab5-4e5a-a41a-320996d2a162\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:40:18 crc kubenswrapper[5052]: I1203 11:40:18.447753 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5deeda32-3ab5-4e5a-a41a-320996d2a162-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"5deeda32-3ab5-4e5a-a41a-320996d2a162\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:40:18 crc kubenswrapper[5052]: I1203 11:40:18.447785 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/5deeda32-3ab5-4e5a-a41a-320996d2a162-config-data\") pod \"ceilometer-0\" (UID: \"5deeda32-3ab5-4e5a-a41a-320996d2a162\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:40:18 crc kubenswrapper[5052]: I1203 11:40:18.447847 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/5deeda32-3ab5-4e5a-a41a-320996d2a162-log-httpd\") pod \"ceilometer-0\" (UID: \"5deeda32-3ab5-4e5a-a41a-320996d2a162\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:40:18 crc kubenswrapper[5052]: I1203 11:40:18.447901 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/5deeda32-3ab5-4e5a-a41a-320996d2a162-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"5deeda32-3ab5-4e5a-a41a-320996d2a162\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:40:18 crc kubenswrapper[5052]: I1203 11:40:18.447932 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/5deeda32-3ab5-4e5a-a41a-320996d2a162-scripts\") pod \"ceilometer-0\" (UID: \"5deeda32-3ab5-4e5a-a41a-320996d2a162\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:40:18 crc kubenswrapper[5052]: I1203 11:40:18.447956 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-7smpm\" (UniqueName: \"kubernetes.io/projected/5deeda32-3ab5-4e5a-a41a-320996d2a162-kube-api-access-7smpm\") pod \"ceilometer-0\" (UID: \"5deeda32-3ab5-4e5a-a41a-320996d2a162\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:40:18 crc kubenswrapper[5052]: I1203 11:40:18.449523 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/5deeda32-3ab5-4e5a-a41a-320996d2a162-run-httpd\") pod \"ceilometer-0\" (UID: \"5deeda32-3ab5-4e5a-a41a-320996d2a162\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:40:18 crc kubenswrapper[5052]: I1203 11:40:18.454979 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/5deeda32-3ab5-4e5a-a41a-320996d2a162-log-httpd\") pod \"ceilometer-0\" (UID: \"5deeda32-3ab5-4e5a-a41a-320996d2a162\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:40:18 crc kubenswrapper[5052]: I1203 11:40:18.455082 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/5deeda32-3ab5-4e5a-a41a-320996d2a162-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"5deeda32-3ab5-4e5a-a41a-320996d2a162\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:40:18 crc kubenswrapper[5052]: I1203 11:40:18.455368 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/5deeda32-3ab5-4e5a-a41a-320996d2a162-scripts\") pod \"ceilometer-0\" (UID: \"5deeda32-3ab5-4e5a-a41a-320996d2a162\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:40:18 crc kubenswrapper[5052]: I1203 11:40:18.455682 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5deeda32-3ab5-4e5a-a41a-320996d2a162-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"5deeda32-3ab5-4e5a-a41a-320996d2a162\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:40:18 crc kubenswrapper[5052]: I1203 11:40:18.457928 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/5deeda32-3ab5-4e5a-a41a-320996d2a162-config-data\") pod \"ceilometer-0\" (UID: \"5deeda32-3ab5-4e5a-a41a-320996d2a162\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:40:18 crc kubenswrapper[5052]: I1203 11:40:18.465912 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-7smpm\" (UniqueName: \"kubernetes.io/projected/5deeda32-3ab5-4e5a-a41a-320996d2a162-kube-api-access-7smpm\") pod \"ceilometer-0\" (UID: \"5deeda32-3ab5-4e5a-a41a-320996d2a162\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:40:18 crc kubenswrapper[5052]: I1203 11:40:18.610120 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:40:18 crc kubenswrapper[5052]: I1203 11:40:18.899433 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:40:19 crc kubenswrapper[5052]: I1203 11:40:19.040333 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:40:19 crc kubenswrapper[5052]: I1203 11:40:19.203942 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"5deeda32-3ab5-4e5a-a41a-320996d2a162","Type":"ContainerStarted","Data":"7146a6129d7ab2ee25c0bc4cbe3168ca1c5584af37f6351da8d6fa3df818af1e"} Dec 03 11:40:19 crc kubenswrapper[5052]: I1203 11:40:19.685216 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="58df8ad3-3e8f-41ae-88ab-f4960ee9a812" path="/var/lib/kubelet/pods/58df8ad3-3e8f-41ae-88ab-f4960ee9a812/volumes" Dec 03 11:40:20 crc kubenswrapper[5052]: I1203 11:40:20.213769 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"5deeda32-3ab5-4e5a-a41a-320996d2a162","Type":"ContainerStarted","Data":"dd86a49370d10cb46359fe17d587fd45828c313fb14c9069b419453b88909a65"} Dec 03 11:40:23 crc kubenswrapper[5052]: I1203 11:40:23.246817 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"5deeda32-3ab5-4e5a-a41a-320996d2a162","Type":"ContainerStarted","Data":"1da162c0a015d19ca065ff14467a32ad1056d33565d61c8e83cecbfc7e351ce9"} Dec 03 11:40:24 crc kubenswrapper[5052]: I1203 11:40:24.262153 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"5deeda32-3ab5-4e5a-a41a-320996d2a162","Type":"ContainerStarted","Data":"c0518ba0cbf4a620ac95e4bd0300d3debf3bfea6cc7e4cd9e7dd9c663af2143f"} Dec 03 11:40:25 crc kubenswrapper[5052]: I1203 11:40:25.272145 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"5deeda32-3ab5-4e5a-a41a-320996d2a162","Type":"ContainerStarted","Data":"7abc54fcc37732cf92e685189f39cb5d0dd7bb107427d126fe6b14d462fa03bd"} Dec 03 11:40:25 crc kubenswrapper[5052]: I1203 11:40:25.272693 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:40:25 crc kubenswrapper[5052]: I1203 11:40:25.272454 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="5deeda32-3ab5-4e5a-a41a-320996d2a162" containerName="proxy-httpd" containerID="cri-o://7abc54fcc37732cf92e685189f39cb5d0dd7bb107427d126fe6b14d462fa03bd" gracePeriod=30 Dec 03 11:40:25 crc kubenswrapper[5052]: I1203 11:40:25.272310 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="5deeda32-3ab5-4e5a-a41a-320996d2a162" containerName="ceilometer-central-agent" containerID="cri-o://dd86a49370d10cb46359fe17d587fd45828c313fb14c9069b419453b88909a65" gracePeriod=30 Dec 03 11:40:25 crc kubenswrapper[5052]: I1203 11:40:25.272499 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="5deeda32-3ab5-4e5a-a41a-320996d2a162" containerName="sg-core" containerID="cri-o://c0518ba0cbf4a620ac95e4bd0300d3debf3bfea6cc7e4cd9e7dd9c663af2143f" gracePeriod=30 Dec 03 11:40:25 crc kubenswrapper[5052]: I1203 11:40:25.272556 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="5deeda32-3ab5-4e5a-a41a-320996d2a162" containerName="ceilometer-notification-agent" containerID="cri-o://1da162c0a015d19ca065ff14467a32ad1056d33565d61c8e83cecbfc7e351ce9" gracePeriod=30 Dec 03 11:40:25 crc kubenswrapper[5052]: I1203 11:40:25.304650 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/ceilometer-0" podStartSLOduration=1.428302051 podStartE2EDuration="7.304631836s" podCreationTimestamp="2025-12-03 11:40:18 +0000 UTC" firstStartedPulling="2025-12-03 11:40:19.04340257 +0000 UTC m=+1209.631644334" lastFinishedPulling="2025-12-03 11:40:24.919732355 +0000 UTC m=+1215.507974119" observedRunningTime="2025-12-03 11:40:25.294534485 +0000 UTC m=+1215.882776249" watchObservedRunningTime="2025-12-03 11:40:25.304631836 +0000 UTC m=+1215.892873600" Dec 03 11:40:26 crc kubenswrapper[5052]: I1203 11:40:26.285343 5052 generic.go:334] "Generic (PLEG): container finished" podID="5deeda32-3ab5-4e5a-a41a-320996d2a162" containerID="c0518ba0cbf4a620ac95e4bd0300d3debf3bfea6cc7e4cd9e7dd9c663af2143f" exitCode=2 Dec 03 11:40:26 crc kubenswrapper[5052]: I1203 11:40:26.285376 5052 generic.go:334] "Generic (PLEG): container finished" podID="5deeda32-3ab5-4e5a-a41a-320996d2a162" containerID="1da162c0a015d19ca065ff14467a32ad1056d33565d61c8e83cecbfc7e351ce9" exitCode=0 Dec 03 11:40:26 crc kubenswrapper[5052]: I1203 11:40:26.285390 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"5deeda32-3ab5-4e5a-a41a-320996d2a162","Type":"ContainerDied","Data":"c0518ba0cbf4a620ac95e4bd0300d3debf3bfea6cc7e4cd9e7dd9c663af2143f"} Dec 03 11:40:26 crc kubenswrapper[5052]: I1203 11:40:26.285451 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"5deeda32-3ab5-4e5a-a41a-320996d2a162","Type":"ContainerDied","Data":"1da162c0a015d19ca065ff14467a32ad1056d33565d61c8e83cecbfc7e351ce9"} Dec 03 11:40:31 crc kubenswrapper[5052]: I1203 11:40:31.334409 5052 generic.go:334] "Generic (PLEG): container finished" podID="5deeda32-3ab5-4e5a-a41a-320996d2a162" containerID="dd86a49370d10cb46359fe17d587fd45828c313fb14c9069b419453b88909a65" exitCode=0 Dec 03 11:40:31 crc kubenswrapper[5052]: I1203 11:40:31.334535 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"5deeda32-3ab5-4e5a-a41a-320996d2a162","Type":"ContainerDied","Data":"dd86a49370d10cb46359fe17d587fd45828c313fb14c9069b419453b88909a65"} Dec 03 11:40:31 crc kubenswrapper[5052]: I1203 11:40:31.336868 5052 generic.go:334] "Generic (PLEG): container finished" podID="0059df68-09bd-434b-b3a8-e2f56cc8bda0" containerID="74b7843216af5eaa832491d05cc3f8cd094a9315e804a6f571a7f09fd5df0acf" exitCode=0 Dec 03 11:40:31 crc kubenswrapper[5052]: I1203 11:40:31.336900 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-t7mx5" event={"ID":"0059df68-09bd-434b-b3a8-e2f56cc8bda0","Type":"ContainerDied","Data":"74b7843216af5eaa832491d05cc3f8cd094a9315e804a6f571a7f09fd5df0acf"} Dec 03 11:40:32 crc kubenswrapper[5052]: I1203 11:40:32.680766 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-t7mx5" Dec 03 11:40:32 crc kubenswrapper[5052]: I1203 11:40:32.701462 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/0059df68-09bd-434b-b3a8-e2f56cc8bda0-scripts\") pod \"0059df68-09bd-434b-b3a8-e2f56cc8bda0\" (UID: \"0059df68-09bd-434b-b3a8-e2f56cc8bda0\") " Dec 03 11:40:32 crc kubenswrapper[5052]: I1203 11:40:32.701559 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/0059df68-09bd-434b-b3a8-e2f56cc8bda0-config-data\") pod \"0059df68-09bd-434b-b3a8-e2f56cc8bda0\" (UID: \"0059df68-09bd-434b-b3a8-e2f56cc8bda0\") " Dec 03 11:40:32 crc kubenswrapper[5052]: I1203 11:40:32.701606 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0059df68-09bd-434b-b3a8-e2f56cc8bda0-combined-ca-bundle\") pod \"0059df68-09bd-434b-b3a8-e2f56cc8bda0\" (UID: \"0059df68-09bd-434b-b3a8-e2f56cc8bda0\") " Dec 03 11:40:32 crc kubenswrapper[5052]: I1203 11:40:32.711985 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0059df68-09bd-434b-b3a8-e2f56cc8bda0-scripts" (OuterVolumeSpecName: "scripts") pod "0059df68-09bd-434b-b3a8-e2f56cc8bda0" (UID: "0059df68-09bd-434b-b3a8-e2f56cc8bda0"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:40:32 crc kubenswrapper[5052]: I1203 11:40:32.740947 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0059df68-09bd-434b-b3a8-e2f56cc8bda0-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "0059df68-09bd-434b-b3a8-e2f56cc8bda0" (UID: "0059df68-09bd-434b-b3a8-e2f56cc8bda0"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:40:32 crc kubenswrapper[5052]: I1203 11:40:32.750474 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0059df68-09bd-434b-b3a8-e2f56cc8bda0-config-data" (OuterVolumeSpecName: "config-data") pod "0059df68-09bd-434b-b3a8-e2f56cc8bda0" (UID: "0059df68-09bd-434b-b3a8-e2f56cc8bda0"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:40:32 crc kubenswrapper[5052]: I1203 11:40:32.802849 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8sbn6\" (UniqueName: \"kubernetes.io/projected/0059df68-09bd-434b-b3a8-e2f56cc8bda0-kube-api-access-8sbn6\") pod \"0059df68-09bd-434b-b3a8-e2f56cc8bda0\" (UID: \"0059df68-09bd-434b-b3a8-e2f56cc8bda0\") " Dec 03 11:40:32 crc kubenswrapper[5052]: I1203 11:40:32.803389 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0059df68-09bd-434b-b3a8-e2f56cc8bda0-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:40:32 crc kubenswrapper[5052]: I1203 11:40:32.803415 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/0059df68-09bd-434b-b3a8-e2f56cc8bda0-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:40:32 crc kubenswrapper[5052]: I1203 11:40:32.803428 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/0059df68-09bd-434b-b3a8-e2f56cc8bda0-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:40:32 crc kubenswrapper[5052]: I1203 11:40:32.805438 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0059df68-09bd-434b-b3a8-e2f56cc8bda0-kube-api-access-8sbn6" (OuterVolumeSpecName: "kube-api-access-8sbn6") pod "0059df68-09bd-434b-b3a8-e2f56cc8bda0" (UID: "0059df68-09bd-434b-b3a8-e2f56cc8bda0"). InnerVolumeSpecName "kube-api-access-8sbn6". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:40:32 crc kubenswrapper[5052]: I1203 11:40:32.905295 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-8sbn6\" (UniqueName: \"kubernetes.io/projected/0059df68-09bd-434b-b3a8-e2f56cc8bda0-kube-api-access-8sbn6\") on node \"crc\" DevicePath \"\"" Dec 03 11:40:33 crc kubenswrapper[5052]: I1203 11:40:33.354411 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-t7mx5" event={"ID":"0059df68-09bd-434b-b3a8-e2f56cc8bda0","Type":"ContainerDied","Data":"96ae7c2a68fb7481f8f4c0d21461d082a7f7316a165efef28b5f72f6349b1368"} Dec 03 11:40:33 crc kubenswrapper[5052]: I1203 11:40:33.354446 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="96ae7c2a68fb7481f8f4c0d21461d082a7f7316a165efef28b5f72f6349b1368" Dec 03 11:40:33 crc kubenswrapper[5052]: I1203 11:40:33.354464 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-t7mx5" Dec 03 11:40:33 crc kubenswrapper[5052]: I1203 11:40:33.448176 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-0"] Dec 03 11:40:33 crc kubenswrapper[5052]: E1203 11:40:33.448878 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="0059df68-09bd-434b-b3a8-e2f56cc8bda0" containerName="nova-cell0-conductor-db-sync" Dec 03 11:40:33 crc kubenswrapper[5052]: I1203 11:40:33.448895 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="0059df68-09bd-434b-b3a8-e2f56cc8bda0" containerName="nova-cell0-conductor-db-sync" Dec 03 11:40:33 crc kubenswrapper[5052]: I1203 11:40:33.449056 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="0059df68-09bd-434b-b3a8-e2f56cc8bda0" containerName="nova-cell0-conductor-db-sync" Dec 03 11:40:33 crc kubenswrapper[5052]: I1203 11:40:33.449656 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 11:40:33 crc kubenswrapper[5052]: I1203 11:40:33.458116 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell0-conductor-config-data" Dec 03 11:40:33 crc kubenswrapper[5052]: I1203 11:40:33.458200 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-nova-dockercfg-6l7ft" Dec 03 11:40:33 crc kubenswrapper[5052]: I1203 11:40:33.461086 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-0"] Dec 03 11:40:33 crc kubenswrapper[5052]: I1203 11:40:33.615178 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kfzp9\" (UniqueName: \"kubernetes.io/projected/f8cdf05e-7081-4542-844d-f76378e8989f-kube-api-access-kfzp9\") pod \"nova-cell0-conductor-0\" (UID: \"f8cdf05e-7081-4542-844d-f76378e8989f\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 11:40:33 crc kubenswrapper[5052]: I1203 11:40:33.615260 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f8cdf05e-7081-4542-844d-f76378e8989f-combined-ca-bundle\") pod \"nova-cell0-conductor-0\" (UID: \"f8cdf05e-7081-4542-844d-f76378e8989f\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 11:40:33 crc kubenswrapper[5052]: I1203 11:40:33.615293 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/f8cdf05e-7081-4542-844d-f76378e8989f-config-data\") pod \"nova-cell0-conductor-0\" (UID: \"f8cdf05e-7081-4542-844d-f76378e8989f\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 11:40:33 crc kubenswrapper[5052]: I1203 11:40:33.716329 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-kfzp9\" (UniqueName: \"kubernetes.io/projected/f8cdf05e-7081-4542-844d-f76378e8989f-kube-api-access-kfzp9\") pod \"nova-cell0-conductor-0\" (UID: \"f8cdf05e-7081-4542-844d-f76378e8989f\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 11:40:33 crc kubenswrapper[5052]: I1203 11:40:33.716389 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f8cdf05e-7081-4542-844d-f76378e8989f-combined-ca-bundle\") pod \"nova-cell0-conductor-0\" (UID: \"f8cdf05e-7081-4542-844d-f76378e8989f\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 11:40:33 crc kubenswrapper[5052]: I1203 11:40:33.716426 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/f8cdf05e-7081-4542-844d-f76378e8989f-config-data\") pod \"nova-cell0-conductor-0\" (UID: \"f8cdf05e-7081-4542-844d-f76378e8989f\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 11:40:33 crc kubenswrapper[5052]: I1203 11:40:33.721440 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f8cdf05e-7081-4542-844d-f76378e8989f-combined-ca-bundle\") pod \"nova-cell0-conductor-0\" (UID: \"f8cdf05e-7081-4542-844d-f76378e8989f\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 11:40:33 crc kubenswrapper[5052]: I1203 11:40:33.737104 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/f8cdf05e-7081-4542-844d-f76378e8989f-config-data\") pod \"nova-cell0-conductor-0\" (UID: \"f8cdf05e-7081-4542-844d-f76378e8989f\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 11:40:33 crc kubenswrapper[5052]: I1203 11:40:33.737661 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-kfzp9\" (UniqueName: \"kubernetes.io/projected/f8cdf05e-7081-4542-844d-f76378e8989f-kube-api-access-kfzp9\") pod \"nova-cell0-conductor-0\" (UID: \"f8cdf05e-7081-4542-844d-f76378e8989f\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 11:40:33 crc kubenswrapper[5052]: I1203 11:40:33.765653 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 11:40:34 crc kubenswrapper[5052]: I1203 11:40:34.201724 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-0"] Dec 03 11:40:34 crc kubenswrapper[5052]: W1203 11:40:34.224795 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf8cdf05e_7081_4542_844d_f76378e8989f.slice/crio-ba20fb745c314713f7c6e16fe25ffbfb735832f224a085cb11f9713cbedbc771 WatchSource:0}: Error finding container ba20fb745c314713f7c6e16fe25ffbfb735832f224a085cb11f9713cbedbc771: Status 404 returned error can't find the container with id ba20fb745c314713f7c6e16fe25ffbfb735832f224a085cb11f9713cbedbc771 Dec 03 11:40:34 crc kubenswrapper[5052]: I1203 11:40:34.367406 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-conductor-0" event={"ID":"f8cdf05e-7081-4542-844d-f76378e8989f","Type":"ContainerStarted","Data":"ba20fb745c314713f7c6e16fe25ffbfb735832f224a085cb11f9713cbedbc771"} Dec 03 11:40:35 crc kubenswrapper[5052]: I1203 11:40:35.378849 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-conductor-0" event={"ID":"f8cdf05e-7081-4542-844d-f76378e8989f","Type":"ContainerStarted","Data":"9cfabfd91ca0525b54d3922c76017cb0defbc772bca498498fc287ac2175e2aa"} Dec 03 11:40:35 crc kubenswrapper[5052]: I1203 11:40:35.381604 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 11:40:35 crc kubenswrapper[5052]: I1203 11:40:35.401805 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-cell0-conductor-0" podStartSLOduration=2.401785486 podStartE2EDuration="2.401785486s" podCreationTimestamp="2025-12-03 11:40:33 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:40:35.395884506 +0000 UTC m=+1225.984126270" watchObservedRunningTime="2025-12-03 11:40:35.401785486 +0000 UTC m=+1225.990027250" Dec 03 11:40:43 crc kubenswrapper[5052]: I1203 11:40:43.794207 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 11:40:44 crc kubenswrapper[5052]: I1203 11:40:44.369576 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell0-cell-mapping-6lr8z"] Dec 03 11:40:44 crc kubenswrapper[5052]: I1203 11:40:44.370919 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-cell-mapping-6lr8z" Dec 03 11:40:44 crc kubenswrapper[5052]: I1203 11:40:44.372895 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell0-manage-scripts" Dec 03 11:40:44 crc kubenswrapper[5052]: I1203 11:40:44.373101 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell0-manage-config-data" Dec 03 11:40:44 crc kubenswrapper[5052]: I1203 11:40:44.387359 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell0-cell-mapping-6lr8z"] Dec 03 11:40:44 crc kubenswrapper[5052]: I1203 11:40:44.433417 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/8934c593-c918-48c9-b19b-e7c0e47474c2-config-data\") pod \"nova-cell0-cell-mapping-6lr8z\" (UID: \"8934c593-c918-48c9-b19b-e7c0e47474c2\") " pod="openstack-kuttl-tests/nova-cell0-cell-mapping-6lr8z" Dec 03 11:40:44 crc kubenswrapper[5052]: I1203 11:40:44.433472 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8934c593-c918-48c9-b19b-e7c0e47474c2-combined-ca-bundle\") pod \"nova-cell0-cell-mapping-6lr8z\" (UID: \"8934c593-c918-48c9-b19b-e7c0e47474c2\") " pod="openstack-kuttl-tests/nova-cell0-cell-mapping-6lr8z" Dec 03 11:40:44 crc kubenswrapper[5052]: I1203 11:40:44.433517 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/8934c593-c918-48c9-b19b-e7c0e47474c2-scripts\") pod \"nova-cell0-cell-mapping-6lr8z\" (UID: \"8934c593-c918-48c9-b19b-e7c0e47474c2\") " pod="openstack-kuttl-tests/nova-cell0-cell-mapping-6lr8z" Dec 03 11:40:44 crc kubenswrapper[5052]: I1203 11:40:44.433679 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-snc9z\" (UniqueName: \"kubernetes.io/projected/8934c593-c918-48c9-b19b-e7c0e47474c2-kube-api-access-snc9z\") pod \"nova-cell0-cell-mapping-6lr8z\" (UID: \"8934c593-c918-48c9-b19b-e7c0e47474c2\") " pod="openstack-kuttl-tests/nova-cell0-cell-mapping-6lr8z" Dec 03 11:40:44 crc kubenswrapper[5052]: I1203 11:40:44.496344 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 11:40:44 crc kubenswrapper[5052]: I1203 11:40:44.497872 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:40:44 crc kubenswrapper[5052]: I1203 11:40:44.500236 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-api-config-data" Dec 03 11:40:44 crc kubenswrapper[5052]: I1203 11:40:44.514075 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 11:40:44 crc kubenswrapper[5052]: I1203 11:40:44.515839 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:40:44 crc kubenswrapper[5052]: I1203 11:40:44.519712 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-metadata-config-data" Dec 03 11:40:44 crc kubenswrapper[5052]: I1203 11:40:44.522727 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 11:40:44 crc kubenswrapper[5052]: I1203 11:40:44.534513 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 11:40:44 crc kubenswrapper[5052]: I1203 11:40:44.535835 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-snc9z\" (UniqueName: \"kubernetes.io/projected/8934c593-c918-48c9-b19b-e7c0e47474c2-kube-api-access-snc9z\") pod \"nova-cell0-cell-mapping-6lr8z\" (UID: \"8934c593-c918-48c9-b19b-e7c0e47474c2\") " pod="openstack-kuttl-tests/nova-cell0-cell-mapping-6lr8z" Dec 03 11:40:44 crc kubenswrapper[5052]: I1203 11:40:44.535912 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/8934c593-c918-48c9-b19b-e7c0e47474c2-config-data\") pod \"nova-cell0-cell-mapping-6lr8z\" (UID: \"8934c593-c918-48c9-b19b-e7c0e47474c2\") " pod="openstack-kuttl-tests/nova-cell0-cell-mapping-6lr8z" Dec 03 11:40:44 crc kubenswrapper[5052]: I1203 11:40:44.535965 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8934c593-c918-48c9-b19b-e7c0e47474c2-combined-ca-bundle\") pod \"nova-cell0-cell-mapping-6lr8z\" (UID: \"8934c593-c918-48c9-b19b-e7c0e47474c2\") " pod="openstack-kuttl-tests/nova-cell0-cell-mapping-6lr8z" Dec 03 11:40:44 crc kubenswrapper[5052]: I1203 11:40:44.536003 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/8934c593-c918-48c9-b19b-e7c0e47474c2-scripts\") pod \"nova-cell0-cell-mapping-6lr8z\" (UID: \"8934c593-c918-48c9-b19b-e7c0e47474c2\") " pod="openstack-kuttl-tests/nova-cell0-cell-mapping-6lr8z" Dec 03 11:40:44 crc kubenswrapper[5052]: I1203 11:40:44.554563 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/8934c593-c918-48c9-b19b-e7c0e47474c2-scripts\") pod \"nova-cell0-cell-mapping-6lr8z\" (UID: \"8934c593-c918-48c9-b19b-e7c0e47474c2\") " pod="openstack-kuttl-tests/nova-cell0-cell-mapping-6lr8z" Dec 03 11:40:44 crc kubenswrapper[5052]: I1203 11:40:44.569189 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/8934c593-c918-48c9-b19b-e7c0e47474c2-config-data\") pod \"nova-cell0-cell-mapping-6lr8z\" (UID: \"8934c593-c918-48c9-b19b-e7c0e47474c2\") " pod="openstack-kuttl-tests/nova-cell0-cell-mapping-6lr8z" Dec 03 11:40:44 crc kubenswrapper[5052]: I1203 11:40:44.569682 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8934c593-c918-48c9-b19b-e7c0e47474c2-combined-ca-bundle\") pod \"nova-cell0-cell-mapping-6lr8z\" (UID: \"8934c593-c918-48c9-b19b-e7c0e47474c2\") " pod="openstack-kuttl-tests/nova-cell0-cell-mapping-6lr8z" Dec 03 11:40:44 crc kubenswrapper[5052]: I1203 11:40:44.602610 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-snc9z\" (UniqueName: \"kubernetes.io/projected/8934c593-c918-48c9-b19b-e7c0e47474c2-kube-api-access-snc9z\") pod \"nova-cell0-cell-mapping-6lr8z\" (UID: \"8934c593-c918-48c9-b19b-e7c0e47474c2\") " pod="openstack-kuttl-tests/nova-cell0-cell-mapping-6lr8z" Dec 03 11:40:44 crc kubenswrapper[5052]: I1203 11:40:44.620263 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 11:40:44 crc kubenswrapper[5052]: I1203 11:40:44.630130 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:40:44 crc kubenswrapper[5052]: I1203 11:40:44.643857 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-scheduler-config-data" Dec 03 11:40:44 crc kubenswrapper[5052]: I1203 11:40:44.645547 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/115e3fec-ca17-47d7-a079-6b9bb3a9dd81-config-data\") pod \"nova-metadata-0\" (UID: \"115e3fec-ca17-47d7-a079-6b9bb3a9dd81\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:40:44 crc kubenswrapper[5052]: I1203 11:40:44.645582 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/413c4d67-a09d-40ba-8ecb-e73a229b1ad6-combined-ca-bundle\") pod \"nova-scheduler-0\" (UID: \"413c4d67-a09d-40ba-8ecb-e73a229b1ad6\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:40:44 crc kubenswrapper[5052]: I1203 11:40:44.645670 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/07b13d48-fa3d-4274-b138-b1c54e05811b-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"07b13d48-fa3d-4274-b138-b1c54e05811b\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:40:44 crc kubenswrapper[5052]: I1203 11:40:44.645697 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fdjzg\" (UniqueName: \"kubernetes.io/projected/115e3fec-ca17-47d7-a079-6b9bb3a9dd81-kube-api-access-fdjzg\") pod \"nova-metadata-0\" (UID: \"115e3fec-ca17-47d7-a079-6b9bb3a9dd81\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:40:44 crc kubenswrapper[5052]: I1203 11:40:44.645735 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n6nbh\" (UniqueName: \"kubernetes.io/projected/07b13d48-fa3d-4274-b138-b1c54e05811b-kube-api-access-n6nbh\") pod \"nova-api-0\" (UID: \"07b13d48-fa3d-4274-b138-b1c54e05811b\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:40:44 crc kubenswrapper[5052]: I1203 11:40:44.645792 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z5qsv\" (UniqueName: \"kubernetes.io/projected/413c4d67-a09d-40ba-8ecb-e73a229b1ad6-kube-api-access-z5qsv\") pod \"nova-scheduler-0\" (UID: \"413c4d67-a09d-40ba-8ecb-e73a229b1ad6\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:40:44 crc kubenswrapper[5052]: I1203 11:40:44.672646 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/115e3fec-ca17-47d7-a079-6b9bb3a9dd81-logs\") pod \"nova-metadata-0\" (UID: \"115e3fec-ca17-47d7-a079-6b9bb3a9dd81\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:40:44 crc kubenswrapper[5052]: I1203 11:40:44.672739 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/07b13d48-fa3d-4274-b138-b1c54e05811b-logs\") pod \"nova-api-0\" (UID: \"07b13d48-fa3d-4274-b138-b1c54e05811b\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:40:44 crc kubenswrapper[5052]: I1203 11:40:44.672774 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/115e3fec-ca17-47d7-a079-6b9bb3a9dd81-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"115e3fec-ca17-47d7-a079-6b9bb3a9dd81\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:40:44 crc kubenswrapper[5052]: I1203 11:40:44.672850 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/413c4d67-a09d-40ba-8ecb-e73a229b1ad6-config-data\") pod \"nova-scheduler-0\" (UID: \"413c4d67-a09d-40ba-8ecb-e73a229b1ad6\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:40:44 crc kubenswrapper[5052]: I1203 11:40:44.672912 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/07b13d48-fa3d-4274-b138-b1c54e05811b-config-data\") pod \"nova-api-0\" (UID: \"07b13d48-fa3d-4274-b138-b1c54e05811b\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:40:44 crc kubenswrapper[5052]: I1203 11:40:44.674983 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 11:40:44 crc kubenswrapper[5052]: I1203 11:40:44.703079 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-cell-mapping-6lr8z" Dec 03 11:40:44 crc kubenswrapper[5052]: I1203 11:40:44.720856 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 11:40:44 crc kubenswrapper[5052]: I1203 11:40:44.722011 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 11:40:44 crc kubenswrapper[5052]: I1203 11:40:44.725595 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell1-novncproxy-config-data" Dec 03 11:40:44 crc kubenswrapper[5052]: I1203 11:40:44.744078 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 11:40:44 crc kubenswrapper[5052]: I1203 11:40:44.775274 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0b744776-104c-466a-be29-79efdd996a28-combined-ca-bundle\") pod \"nova-cell1-novncproxy-0\" (UID: \"0b744776-104c-466a-be29-79efdd996a28\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 11:40:44 crc kubenswrapper[5052]: I1203 11:40:44.775533 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9qccm\" (UniqueName: \"kubernetes.io/projected/0b744776-104c-466a-be29-79efdd996a28-kube-api-access-9qccm\") pod \"nova-cell1-novncproxy-0\" (UID: \"0b744776-104c-466a-be29-79efdd996a28\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 11:40:44 crc kubenswrapper[5052]: I1203 11:40:44.775631 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-fdjzg\" (UniqueName: \"kubernetes.io/projected/115e3fec-ca17-47d7-a079-6b9bb3a9dd81-kube-api-access-fdjzg\") pod \"nova-metadata-0\" (UID: \"115e3fec-ca17-47d7-a079-6b9bb3a9dd81\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:40:44 crc kubenswrapper[5052]: I1203 11:40:44.775701 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/07b13d48-fa3d-4274-b138-b1c54e05811b-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"07b13d48-fa3d-4274-b138-b1c54e05811b\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:40:44 crc kubenswrapper[5052]: I1203 11:40:44.775827 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-n6nbh\" (UniqueName: \"kubernetes.io/projected/07b13d48-fa3d-4274-b138-b1c54e05811b-kube-api-access-n6nbh\") pod \"nova-api-0\" (UID: \"07b13d48-fa3d-4274-b138-b1c54e05811b\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:40:44 crc kubenswrapper[5052]: I1203 11:40:44.775941 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-z5qsv\" (UniqueName: \"kubernetes.io/projected/413c4d67-a09d-40ba-8ecb-e73a229b1ad6-kube-api-access-z5qsv\") pod \"nova-scheduler-0\" (UID: \"413c4d67-a09d-40ba-8ecb-e73a229b1ad6\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:40:44 crc kubenswrapper[5052]: I1203 11:40:44.776016 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/115e3fec-ca17-47d7-a079-6b9bb3a9dd81-logs\") pod \"nova-metadata-0\" (UID: \"115e3fec-ca17-47d7-a079-6b9bb3a9dd81\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:40:44 crc kubenswrapper[5052]: I1203 11:40:44.776096 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/07b13d48-fa3d-4274-b138-b1c54e05811b-logs\") pod \"nova-api-0\" (UID: \"07b13d48-fa3d-4274-b138-b1c54e05811b\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:40:44 crc kubenswrapper[5052]: I1203 11:40:44.776166 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/115e3fec-ca17-47d7-a079-6b9bb3a9dd81-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"115e3fec-ca17-47d7-a079-6b9bb3a9dd81\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:40:44 crc kubenswrapper[5052]: I1203 11:40:44.776261 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/0b744776-104c-466a-be29-79efdd996a28-config-data\") pod \"nova-cell1-novncproxy-0\" (UID: \"0b744776-104c-466a-be29-79efdd996a28\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 11:40:44 crc kubenswrapper[5052]: I1203 11:40:44.776340 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/413c4d67-a09d-40ba-8ecb-e73a229b1ad6-config-data\") pod \"nova-scheduler-0\" (UID: \"413c4d67-a09d-40ba-8ecb-e73a229b1ad6\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:40:44 crc kubenswrapper[5052]: I1203 11:40:44.776418 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/07b13d48-fa3d-4274-b138-b1c54e05811b-config-data\") pod \"nova-api-0\" (UID: \"07b13d48-fa3d-4274-b138-b1c54e05811b\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:40:44 crc kubenswrapper[5052]: I1203 11:40:44.776503 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/115e3fec-ca17-47d7-a079-6b9bb3a9dd81-config-data\") pod \"nova-metadata-0\" (UID: \"115e3fec-ca17-47d7-a079-6b9bb3a9dd81\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:40:44 crc kubenswrapper[5052]: I1203 11:40:44.776601 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/413c4d67-a09d-40ba-8ecb-e73a229b1ad6-combined-ca-bundle\") pod \"nova-scheduler-0\" (UID: \"413c4d67-a09d-40ba-8ecb-e73a229b1ad6\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:40:44 crc kubenswrapper[5052]: I1203 11:40:44.776845 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/07b13d48-fa3d-4274-b138-b1c54e05811b-logs\") pod \"nova-api-0\" (UID: \"07b13d48-fa3d-4274-b138-b1c54e05811b\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:40:44 crc kubenswrapper[5052]: I1203 11:40:44.776558 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/115e3fec-ca17-47d7-a079-6b9bb3a9dd81-logs\") pod \"nova-metadata-0\" (UID: \"115e3fec-ca17-47d7-a079-6b9bb3a9dd81\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:40:44 crc kubenswrapper[5052]: I1203 11:40:44.780230 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/07b13d48-fa3d-4274-b138-b1c54e05811b-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"07b13d48-fa3d-4274-b138-b1c54e05811b\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:40:44 crc kubenswrapper[5052]: I1203 11:40:44.782913 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/07b13d48-fa3d-4274-b138-b1c54e05811b-config-data\") pod \"nova-api-0\" (UID: \"07b13d48-fa3d-4274-b138-b1c54e05811b\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:40:44 crc kubenswrapper[5052]: I1203 11:40:44.783684 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/115e3fec-ca17-47d7-a079-6b9bb3a9dd81-config-data\") pod \"nova-metadata-0\" (UID: \"115e3fec-ca17-47d7-a079-6b9bb3a9dd81\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:40:44 crc kubenswrapper[5052]: I1203 11:40:44.792121 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/413c4d67-a09d-40ba-8ecb-e73a229b1ad6-config-data\") pod \"nova-scheduler-0\" (UID: \"413c4d67-a09d-40ba-8ecb-e73a229b1ad6\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:40:44 crc kubenswrapper[5052]: I1203 11:40:44.794707 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/115e3fec-ca17-47d7-a079-6b9bb3a9dd81-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"115e3fec-ca17-47d7-a079-6b9bb3a9dd81\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:40:44 crc kubenswrapper[5052]: I1203 11:40:44.802857 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/413c4d67-a09d-40ba-8ecb-e73a229b1ad6-combined-ca-bundle\") pod \"nova-scheduler-0\" (UID: \"413c4d67-a09d-40ba-8ecb-e73a229b1ad6\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:40:44 crc kubenswrapper[5052]: I1203 11:40:44.805607 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-z5qsv\" (UniqueName: \"kubernetes.io/projected/413c4d67-a09d-40ba-8ecb-e73a229b1ad6-kube-api-access-z5qsv\") pod \"nova-scheduler-0\" (UID: \"413c4d67-a09d-40ba-8ecb-e73a229b1ad6\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:40:44 crc kubenswrapper[5052]: I1203 11:40:44.805789 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-n6nbh\" (UniqueName: \"kubernetes.io/projected/07b13d48-fa3d-4274-b138-b1c54e05811b-kube-api-access-n6nbh\") pod \"nova-api-0\" (UID: \"07b13d48-fa3d-4274-b138-b1c54e05811b\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:40:44 crc kubenswrapper[5052]: I1203 11:40:44.805907 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-fdjzg\" (UniqueName: \"kubernetes.io/projected/115e3fec-ca17-47d7-a079-6b9bb3a9dd81-kube-api-access-fdjzg\") pod \"nova-metadata-0\" (UID: \"115e3fec-ca17-47d7-a079-6b9bb3a9dd81\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:40:44 crc kubenswrapper[5052]: I1203 11:40:44.818348 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:40:44 crc kubenswrapper[5052]: I1203 11:40:44.851308 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:40:44 crc kubenswrapper[5052]: I1203 11:40:44.878627 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0b744776-104c-466a-be29-79efdd996a28-combined-ca-bundle\") pod \"nova-cell1-novncproxy-0\" (UID: \"0b744776-104c-466a-be29-79efdd996a28\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 11:40:44 crc kubenswrapper[5052]: I1203 11:40:44.878705 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-9qccm\" (UniqueName: \"kubernetes.io/projected/0b744776-104c-466a-be29-79efdd996a28-kube-api-access-9qccm\") pod \"nova-cell1-novncproxy-0\" (UID: \"0b744776-104c-466a-be29-79efdd996a28\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 11:40:44 crc kubenswrapper[5052]: I1203 11:40:44.878811 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/0b744776-104c-466a-be29-79efdd996a28-config-data\") pod \"nova-cell1-novncproxy-0\" (UID: \"0b744776-104c-466a-be29-79efdd996a28\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 11:40:44 crc kubenswrapper[5052]: I1203 11:40:44.885613 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0b744776-104c-466a-be29-79efdd996a28-combined-ca-bundle\") pod \"nova-cell1-novncproxy-0\" (UID: \"0b744776-104c-466a-be29-79efdd996a28\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 11:40:44 crc kubenswrapper[5052]: I1203 11:40:44.886059 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/0b744776-104c-466a-be29-79efdd996a28-config-data\") pod \"nova-cell1-novncproxy-0\" (UID: \"0b744776-104c-466a-be29-79efdd996a28\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 11:40:44 crc kubenswrapper[5052]: I1203 11:40:44.905174 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-9qccm\" (UniqueName: \"kubernetes.io/projected/0b744776-104c-466a-be29-79efdd996a28-kube-api-access-9qccm\") pod \"nova-cell1-novncproxy-0\" (UID: \"0b744776-104c-466a-be29-79efdd996a28\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 11:40:44 crc kubenswrapper[5052]: I1203 11:40:44.990740 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:40:45 crc kubenswrapper[5052]: I1203 11:40:45.167019 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 11:40:45 crc kubenswrapper[5052]: I1203 11:40:45.191508 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell0-cell-mapping-6lr8z"] Dec 03 11:40:45 crc kubenswrapper[5052]: W1203 11:40:45.206069 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8934c593_c918_48c9_b19b_e7c0e47474c2.slice/crio-58b0221599cb89c02b4a754ec37ae1fa6e44b7dcd97d27cc4b3cdc42de26295f WatchSource:0}: Error finding container 58b0221599cb89c02b4a754ec37ae1fa6e44b7dcd97d27cc4b3cdc42de26295f: Status 404 returned error can't find the container with id 58b0221599cb89c02b4a754ec37ae1fa6e44b7dcd97d27cc4b3cdc42de26295f Dec 03 11:40:45 crc kubenswrapper[5052]: I1203 11:40:45.371992 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 11:40:45 crc kubenswrapper[5052]: I1203 11:40:45.505139 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 11:40:45 crc kubenswrapper[5052]: I1203 11:40:45.506511 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"07b13d48-fa3d-4274-b138-b1c54e05811b","Type":"ContainerStarted","Data":"c96f0f48d5837f6b0957ba4a27b2d8a7033266452b899f8e44fdc6b5bc1a7671"} Dec 03 11:40:45 crc kubenswrapper[5052]: I1203 11:40:45.520372 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-db-sync-cw46s"] Dec 03 11:40:45 crc kubenswrapper[5052]: I1203 11:40:45.521800 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-cw46s" Dec 03 11:40:45 crc kubenswrapper[5052]: I1203 11:40:45.522216 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-cell-mapping-6lr8z" event={"ID":"8934c593-c918-48c9-b19b-e7c0e47474c2","Type":"ContainerStarted","Data":"1e380bc4c681225a08515748fc7e47b1e5ab3aca3455957a89e3055115aa44a9"} Dec 03 11:40:45 crc kubenswrapper[5052]: I1203 11:40:45.522264 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-cell-mapping-6lr8z" event={"ID":"8934c593-c918-48c9-b19b-e7c0e47474c2","Type":"ContainerStarted","Data":"58b0221599cb89c02b4a754ec37ae1fa6e44b7dcd97d27cc4b3cdc42de26295f"} Dec 03 11:40:45 crc kubenswrapper[5052]: I1203 11:40:45.524664 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell1-conductor-config-data" Dec 03 11:40:45 crc kubenswrapper[5052]: I1203 11:40:45.524827 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell1-conductor-scripts" Dec 03 11:40:45 crc kubenswrapper[5052]: I1203 11:40:45.532784 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-db-sync-cw46s"] Dec 03 11:40:45 crc kubenswrapper[5052]: I1203 11:40:45.557604 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-cell0-cell-mapping-6lr8z" podStartSLOduration=1.5575869629999999 podStartE2EDuration="1.557586963s" podCreationTimestamp="2025-12-03 11:40:44 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:40:45.554575675 +0000 UTC m=+1236.142817449" watchObservedRunningTime="2025-12-03 11:40:45.557586963 +0000 UTC m=+1236.145828727" Dec 03 11:40:45 crc kubenswrapper[5052]: I1203 11:40:45.601148 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/1285046f-826b-4a30-a414-46e598daf9ca-scripts\") pod \"nova-cell1-conductor-db-sync-cw46s\" (UID: \"1285046f-826b-4a30-a414-46e598daf9ca\") " pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-cw46s" Dec 03 11:40:45 crc kubenswrapper[5052]: I1203 11:40:45.601213 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g5p6x\" (UniqueName: \"kubernetes.io/projected/1285046f-826b-4a30-a414-46e598daf9ca-kube-api-access-g5p6x\") pod \"nova-cell1-conductor-db-sync-cw46s\" (UID: \"1285046f-826b-4a30-a414-46e598daf9ca\") " pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-cw46s" Dec 03 11:40:45 crc kubenswrapper[5052]: I1203 11:40:45.601235 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1285046f-826b-4a30-a414-46e598daf9ca-combined-ca-bundle\") pod \"nova-cell1-conductor-db-sync-cw46s\" (UID: \"1285046f-826b-4a30-a414-46e598daf9ca\") " pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-cw46s" Dec 03 11:40:45 crc kubenswrapper[5052]: I1203 11:40:45.601271 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/1285046f-826b-4a30-a414-46e598daf9ca-config-data\") pod \"nova-cell1-conductor-db-sync-cw46s\" (UID: \"1285046f-826b-4a30-a414-46e598daf9ca\") " pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-cw46s" Dec 03 11:40:45 crc kubenswrapper[5052]: I1203 11:40:45.618298 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 11:40:45 crc kubenswrapper[5052]: I1203 11:40:45.702931 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/1285046f-826b-4a30-a414-46e598daf9ca-config-data\") pod \"nova-cell1-conductor-db-sync-cw46s\" (UID: \"1285046f-826b-4a30-a414-46e598daf9ca\") " pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-cw46s" Dec 03 11:40:45 crc kubenswrapper[5052]: I1203 11:40:45.703080 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/1285046f-826b-4a30-a414-46e598daf9ca-scripts\") pod \"nova-cell1-conductor-db-sync-cw46s\" (UID: \"1285046f-826b-4a30-a414-46e598daf9ca\") " pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-cw46s" Dec 03 11:40:45 crc kubenswrapper[5052]: I1203 11:40:45.703139 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-g5p6x\" (UniqueName: \"kubernetes.io/projected/1285046f-826b-4a30-a414-46e598daf9ca-kube-api-access-g5p6x\") pod \"nova-cell1-conductor-db-sync-cw46s\" (UID: \"1285046f-826b-4a30-a414-46e598daf9ca\") " pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-cw46s" Dec 03 11:40:45 crc kubenswrapper[5052]: I1203 11:40:45.703157 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1285046f-826b-4a30-a414-46e598daf9ca-combined-ca-bundle\") pod \"nova-cell1-conductor-db-sync-cw46s\" (UID: \"1285046f-826b-4a30-a414-46e598daf9ca\") " pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-cw46s" Dec 03 11:40:45 crc kubenswrapper[5052]: I1203 11:40:45.708725 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/1285046f-826b-4a30-a414-46e598daf9ca-config-data\") pod \"nova-cell1-conductor-db-sync-cw46s\" (UID: \"1285046f-826b-4a30-a414-46e598daf9ca\") " pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-cw46s" Dec 03 11:40:45 crc kubenswrapper[5052]: I1203 11:40:45.709341 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/1285046f-826b-4a30-a414-46e598daf9ca-scripts\") pod \"nova-cell1-conductor-db-sync-cw46s\" (UID: \"1285046f-826b-4a30-a414-46e598daf9ca\") " pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-cw46s" Dec 03 11:40:45 crc kubenswrapper[5052]: I1203 11:40:45.720872 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-g5p6x\" (UniqueName: \"kubernetes.io/projected/1285046f-826b-4a30-a414-46e598daf9ca-kube-api-access-g5p6x\") pod \"nova-cell1-conductor-db-sync-cw46s\" (UID: \"1285046f-826b-4a30-a414-46e598daf9ca\") " pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-cw46s" Dec 03 11:40:45 crc kubenswrapper[5052]: I1203 11:40:45.721300 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1285046f-826b-4a30-a414-46e598daf9ca-combined-ca-bundle\") pod \"nova-cell1-conductor-db-sync-cw46s\" (UID: \"1285046f-826b-4a30-a414-46e598daf9ca\") " pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-cw46s" Dec 03 11:40:45 crc kubenswrapper[5052]: I1203 11:40:45.776452 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 11:40:45 crc kubenswrapper[5052]: W1203 11:40:45.777008 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0b744776_104c_466a_be29_79efdd996a28.slice/crio-1b465e582a74ac1f559f37138e65b1f1cf0e3836cc1e1e6a5046bd8b27f87f74 WatchSource:0}: Error finding container 1b465e582a74ac1f559f37138e65b1f1cf0e3836cc1e1e6a5046bd8b27f87f74: Status 404 returned error can't find the container with id 1b465e582a74ac1f559f37138e65b1f1cf0e3836cc1e1e6a5046bd8b27f87f74 Dec 03 11:40:45 crc kubenswrapper[5052]: I1203 11:40:45.953210 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-cw46s" Dec 03 11:40:46 crc kubenswrapper[5052]: I1203 11:40:46.432382 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-db-sync-cw46s"] Dec 03 11:40:46 crc kubenswrapper[5052]: W1203 11:40:46.445486 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1285046f_826b_4a30_a414_46e598daf9ca.slice/crio-0f2ff91c0b7e75c1a8b059bf7a45c2212c1135f700ac9c4c16ecc1865b7dd13a WatchSource:0}: Error finding container 0f2ff91c0b7e75c1a8b059bf7a45c2212c1135f700ac9c4c16ecc1865b7dd13a: Status 404 returned error can't find the container with id 0f2ff91c0b7e75c1a8b059bf7a45c2212c1135f700ac9c4c16ecc1865b7dd13a Dec 03 11:40:46 crc kubenswrapper[5052]: I1203 11:40:46.534997 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"115e3fec-ca17-47d7-a079-6b9bb3a9dd81","Type":"ContainerStarted","Data":"ea1d5d4f7de98e6702113fb9eebd0748ebae09ce3182752f991e80e1567a7894"} Dec 03 11:40:46 crc kubenswrapper[5052]: I1203 11:40:46.537082 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"413c4d67-a09d-40ba-8ecb-e73a229b1ad6","Type":"ContainerStarted","Data":"6338483b2b756c96dcf60aaa9a7bbe747ac6817f81a380c7037cfa0f53f03bb1"} Dec 03 11:40:46 crc kubenswrapper[5052]: I1203 11:40:46.538064 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-cw46s" event={"ID":"1285046f-826b-4a30-a414-46e598daf9ca","Type":"ContainerStarted","Data":"0f2ff91c0b7e75c1a8b059bf7a45c2212c1135f700ac9c4c16ecc1865b7dd13a"} Dec 03 11:40:46 crc kubenswrapper[5052]: I1203 11:40:46.540120 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" event={"ID":"0b744776-104c-466a-be29-79efdd996a28","Type":"ContainerStarted","Data":"1b465e582a74ac1f559f37138e65b1f1cf0e3836cc1e1e6a5046bd8b27f87f74"} Dec 03 11:40:47 crc kubenswrapper[5052]: I1203 11:40:47.549314 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-cw46s" event={"ID":"1285046f-826b-4a30-a414-46e598daf9ca","Type":"ContainerStarted","Data":"a47db30ff0a5b853393f81265de91d0780b0ae16051c26ce22422a53ba83dc46"} Dec 03 11:40:47 crc kubenswrapper[5052]: I1203 11:40:47.570555 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-cw46s" podStartSLOduration=2.570535005 podStartE2EDuration="2.570535005s" podCreationTimestamp="2025-12-03 11:40:45 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:40:47.566261841 +0000 UTC m=+1238.154503625" watchObservedRunningTime="2025-12-03 11:40:47.570535005 +0000 UTC m=+1238.158776769" Dec 03 11:40:48 crc kubenswrapper[5052]: I1203 11:40:48.059974 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 11:40:48 crc kubenswrapper[5052]: I1203 11:40:48.074308 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 11:40:48 crc kubenswrapper[5052]: I1203 11:40:48.619231 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/ceilometer-0" podUID="5deeda32-3ab5-4e5a-a41a-320996d2a162" containerName="proxy-httpd" probeResult="failure" output="HTTP probe failed with statuscode: 503" Dec 03 11:40:49 crc kubenswrapper[5052]: I1203 11:40:49.567556 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"413c4d67-a09d-40ba-8ecb-e73a229b1ad6","Type":"ContainerStarted","Data":"eec4b2ec6abbb7588fec20cd2b6a8279879b8912e1316fee7e71bd784bb5e05e"} Dec 03 11:40:49 crc kubenswrapper[5052]: I1203 11:40:49.569549 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"07b13d48-fa3d-4274-b138-b1c54e05811b","Type":"ContainerStarted","Data":"781f723d704c08a0bb05affca300ba883bf5c2febee85b35ef6e758a1b497a8e"} Dec 03 11:40:49 crc kubenswrapper[5052]: I1203 11:40:49.569676 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"07b13d48-fa3d-4274-b138-b1c54e05811b","Type":"ContainerStarted","Data":"d2f41ae028ba676e4abc4a6c3cef0c2a13e596f42f7e81f790e0406f1bd2f8ba"} Dec 03 11:40:49 crc kubenswrapper[5052]: I1203 11:40:49.571266 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" event={"ID":"0b744776-104c-466a-be29-79efdd996a28","Type":"ContainerStarted","Data":"08aececa6f47b8deb531a78d1231ed436b94566bc55dd432fa69a3f7a05d3ebf"} Dec 03 11:40:49 crc kubenswrapper[5052]: I1203 11:40:49.571477 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" podUID="0b744776-104c-466a-be29-79efdd996a28" containerName="nova-cell1-novncproxy-novncproxy" containerID="cri-o://08aececa6f47b8deb531a78d1231ed436b94566bc55dd432fa69a3f7a05d3ebf" gracePeriod=30 Dec 03 11:40:49 crc kubenswrapper[5052]: I1203 11:40:49.573359 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"115e3fec-ca17-47d7-a079-6b9bb3a9dd81","Type":"ContainerStarted","Data":"9336f42aa7f610b7c1c95f8f84b09607852f8edcc2bb4321efa580212ca5c69d"} Dec 03 11:40:49 crc kubenswrapper[5052]: I1203 11:40:49.573394 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"115e3fec-ca17-47d7-a079-6b9bb3a9dd81","Type":"ContainerStarted","Data":"c6354c146b16bffa1182e6875189002d155dbd94c6891dbd34541814d6f4ee08"} Dec 03 11:40:49 crc kubenswrapper[5052]: I1203 11:40:49.573525 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-metadata-0" podUID="115e3fec-ca17-47d7-a079-6b9bb3a9dd81" containerName="nova-metadata-log" containerID="cri-o://c6354c146b16bffa1182e6875189002d155dbd94c6891dbd34541814d6f4ee08" gracePeriod=30 Dec 03 11:40:49 crc kubenswrapper[5052]: I1203 11:40:49.573541 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-metadata-0" podUID="115e3fec-ca17-47d7-a079-6b9bb3a9dd81" containerName="nova-metadata-metadata" containerID="cri-o://9336f42aa7f610b7c1c95f8f84b09607852f8edcc2bb4321efa580212ca5c69d" gracePeriod=30 Dec 03 11:40:49 crc kubenswrapper[5052]: I1203 11:40:49.585829 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-scheduler-0" podStartSLOduration=2.587452944 podStartE2EDuration="5.585808364s" podCreationTimestamp="2025-12-03 11:40:44 +0000 UTC" firstStartedPulling="2025-12-03 11:40:45.618922277 +0000 UTC m=+1236.207164041" lastFinishedPulling="2025-12-03 11:40:48.617277677 +0000 UTC m=+1239.205519461" observedRunningTime="2025-12-03 11:40:49.583321572 +0000 UTC m=+1240.171563336" watchObservedRunningTime="2025-12-03 11:40:49.585808364 +0000 UTC m=+1240.174050128" Dec 03 11:40:49 crc kubenswrapper[5052]: I1203 11:40:49.615919 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-metadata-0" podStartSLOduration=2.52820061 podStartE2EDuration="5.61589957s" podCreationTimestamp="2025-12-03 11:40:44 +0000 UTC" firstStartedPulling="2025-12-03 11:40:45.527226149 +0000 UTC m=+1236.115467913" lastFinishedPulling="2025-12-03 11:40:48.614925109 +0000 UTC m=+1239.203166873" observedRunningTime="2025-12-03 11:40:49.607935628 +0000 UTC m=+1240.196177392" watchObservedRunningTime="2025-12-03 11:40:49.61589957 +0000 UTC m=+1240.204141334" Dec 03 11:40:49 crc kubenswrapper[5052]: I1203 11:40:49.630521 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-api-0" podStartSLOduration=2.418758857 podStartE2EDuration="5.630498665s" podCreationTimestamp="2025-12-03 11:40:44 +0000 UTC" firstStartedPulling="2025-12-03 11:40:45.403366086 +0000 UTC m=+1235.991607850" lastFinishedPulling="2025-12-03 11:40:48.615105904 +0000 UTC m=+1239.203347658" observedRunningTime="2025-12-03 11:40:49.62276186 +0000 UTC m=+1240.211003634" watchObservedRunningTime="2025-12-03 11:40:49.630498665 +0000 UTC m=+1240.218740429" Dec 03 11:40:49 crc kubenswrapper[5052]: I1203 11:40:49.637239 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" podStartSLOduration=2.797734382 podStartE2EDuration="5.6372137s" podCreationTimestamp="2025-12-03 11:40:44 +0000 UTC" firstStartedPulling="2025-12-03 11:40:45.779792697 +0000 UTC m=+1236.368034461" lastFinishedPulling="2025-12-03 11:40:48.619272015 +0000 UTC m=+1239.207513779" observedRunningTime="2025-12-03 11:40:49.636096987 +0000 UTC m=+1240.224338761" watchObservedRunningTime="2025-12-03 11:40:49.6372137 +0000 UTC m=+1240.225455464" Dec 03 11:40:49 crc kubenswrapper[5052]: I1203 11:40:49.852597 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:40:49 crc kubenswrapper[5052]: I1203 11:40:49.852866 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:40:49 crc kubenswrapper[5052]: I1203 11:40:49.991628 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:40:50 crc kubenswrapper[5052]: I1203 11:40:50.168533 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 11:40:50 crc kubenswrapper[5052]: I1203 11:40:50.174879 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:40:50 crc kubenswrapper[5052]: I1203 11:40:50.208437 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/115e3fec-ca17-47d7-a079-6b9bb3a9dd81-config-data\") pod \"115e3fec-ca17-47d7-a079-6b9bb3a9dd81\" (UID: \"115e3fec-ca17-47d7-a079-6b9bb3a9dd81\") " Dec 03 11:40:50 crc kubenswrapper[5052]: I1203 11:40:50.208559 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/115e3fec-ca17-47d7-a079-6b9bb3a9dd81-logs\") pod \"115e3fec-ca17-47d7-a079-6b9bb3a9dd81\" (UID: \"115e3fec-ca17-47d7-a079-6b9bb3a9dd81\") " Dec 03 11:40:50 crc kubenswrapper[5052]: I1203 11:40:50.208756 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fdjzg\" (UniqueName: \"kubernetes.io/projected/115e3fec-ca17-47d7-a079-6b9bb3a9dd81-kube-api-access-fdjzg\") pod \"115e3fec-ca17-47d7-a079-6b9bb3a9dd81\" (UID: \"115e3fec-ca17-47d7-a079-6b9bb3a9dd81\") " Dec 03 11:40:50 crc kubenswrapper[5052]: I1203 11:40:50.208829 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/115e3fec-ca17-47d7-a079-6b9bb3a9dd81-combined-ca-bundle\") pod \"115e3fec-ca17-47d7-a079-6b9bb3a9dd81\" (UID: \"115e3fec-ca17-47d7-a079-6b9bb3a9dd81\") " Dec 03 11:40:50 crc kubenswrapper[5052]: I1203 11:40:50.209116 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/115e3fec-ca17-47d7-a079-6b9bb3a9dd81-logs" (OuterVolumeSpecName: "logs") pod "115e3fec-ca17-47d7-a079-6b9bb3a9dd81" (UID: "115e3fec-ca17-47d7-a079-6b9bb3a9dd81"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:40:50 crc kubenswrapper[5052]: I1203 11:40:50.209482 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/115e3fec-ca17-47d7-a079-6b9bb3a9dd81-logs\") on node \"crc\" DevicePath \"\"" Dec 03 11:40:50 crc kubenswrapper[5052]: I1203 11:40:50.219230 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/115e3fec-ca17-47d7-a079-6b9bb3a9dd81-kube-api-access-fdjzg" (OuterVolumeSpecName: "kube-api-access-fdjzg") pod "115e3fec-ca17-47d7-a079-6b9bb3a9dd81" (UID: "115e3fec-ca17-47d7-a079-6b9bb3a9dd81"). InnerVolumeSpecName "kube-api-access-fdjzg". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:40:50 crc kubenswrapper[5052]: I1203 11:40:50.244548 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/115e3fec-ca17-47d7-a079-6b9bb3a9dd81-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "115e3fec-ca17-47d7-a079-6b9bb3a9dd81" (UID: "115e3fec-ca17-47d7-a079-6b9bb3a9dd81"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:40:50 crc kubenswrapper[5052]: I1203 11:40:50.244704 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/115e3fec-ca17-47d7-a079-6b9bb3a9dd81-config-data" (OuterVolumeSpecName: "config-data") pod "115e3fec-ca17-47d7-a079-6b9bb3a9dd81" (UID: "115e3fec-ca17-47d7-a079-6b9bb3a9dd81"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:40:50 crc kubenswrapper[5052]: I1203 11:40:50.311271 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/115e3fec-ca17-47d7-a079-6b9bb3a9dd81-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:40:50 crc kubenswrapper[5052]: I1203 11:40:50.311312 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/115e3fec-ca17-47d7-a079-6b9bb3a9dd81-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:40:50 crc kubenswrapper[5052]: I1203 11:40:50.311321 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-fdjzg\" (UniqueName: \"kubernetes.io/projected/115e3fec-ca17-47d7-a079-6b9bb3a9dd81-kube-api-access-fdjzg\") on node \"crc\" DevicePath \"\"" Dec 03 11:40:50 crc kubenswrapper[5052]: I1203 11:40:50.586905 5052 generic.go:334] "Generic (PLEG): container finished" podID="115e3fec-ca17-47d7-a079-6b9bb3a9dd81" containerID="9336f42aa7f610b7c1c95f8f84b09607852f8edcc2bb4321efa580212ca5c69d" exitCode=0 Dec 03 11:40:50 crc kubenswrapper[5052]: I1203 11:40:50.586964 5052 generic.go:334] "Generic (PLEG): container finished" podID="115e3fec-ca17-47d7-a079-6b9bb3a9dd81" containerID="c6354c146b16bffa1182e6875189002d155dbd94c6891dbd34541814d6f4ee08" exitCode=143 Dec 03 11:40:50 crc kubenswrapper[5052]: I1203 11:40:50.588600 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:40:50 crc kubenswrapper[5052]: I1203 11:40:50.588755 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"115e3fec-ca17-47d7-a079-6b9bb3a9dd81","Type":"ContainerDied","Data":"9336f42aa7f610b7c1c95f8f84b09607852f8edcc2bb4321efa580212ca5c69d"} Dec 03 11:40:50 crc kubenswrapper[5052]: I1203 11:40:50.588831 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"115e3fec-ca17-47d7-a079-6b9bb3a9dd81","Type":"ContainerDied","Data":"c6354c146b16bffa1182e6875189002d155dbd94c6891dbd34541814d6f4ee08"} Dec 03 11:40:50 crc kubenswrapper[5052]: I1203 11:40:50.588856 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"115e3fec-ca17-47d7-a079-6b9bb3a9dd81","Type":"ContainerDied","Data":"ea1d5d4f7de98e6702113fb9eebd0748ebae09ce3182752f991e80e1567a7894"} Dec 03 11:40:50 crc kubenswrapper[5052]: I1203 11:40:50.588896 5052 scope.go:117] "RemoveContainer" containerID="9336f42aa7f610b7c1c95f8f84b09607852f8edcc2bb4321efa580212ca5c69d" Dec 03 11:40:50 crc kubenswrapper[5052]: I1203 11:40:50.627302 5052 scope.go:117] "RemoveContainer" containerID="c6354c146b16bffa1182e6875189002d155dbd94c6891dbd34541814d6f4ee08" Dec 03 11:40:50 crc kubenswrapper[5052]: I1203 11:40:50.638625 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 11:40:50 crc kubenswrapper[5052]: I1203 11:40:50.654942 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 11:40:50 crc kubenswrapper[5052]: I1203 11:40:50.666084 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 11:40:50 crc kubenswrapper[5052]: E1203 11:40:50.666534 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="115e3fec-ca17-47d7-a079-6b9bb3a9dd81" containerName="nova-metadata-log" Dec 03 11:40:50 crc kubenswrapper[5052]: I1203 11:40:50.666553 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="115e3fec-ca17-47d7-a079-6b9bb3a9dd81" containerName="nova-metadata-log" Dec 03 11:40:50 crc kubenswrapper[5052]: E1203 11:40:50.666568 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="115e3fec-ca17-47d7-a079-6b9bb3a9dd81" containerName="nova-metadata-metadata" Dec 03 11:40:50 crc kubenswrapper[5052]: I1203 11:40:50.666575 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="115e3fec-ca17-47d7-a079-6b9bb3a9dd81" containerName="nova-metadata-metadata" Dec 03 11:40:50 crc kubenswrapper[5052]: I1203 11:40:50.666895 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="115e3fec-ca17-47d7-a079-6b9bb3a9dd81" containerName="nova-metadata-metadata" Dec 03 11:40:50 crc kubenswrapper[5052]: I1203 11:40:50.666933 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="115e3fec-ca17-47d7-a079-6b9bb3a9dd81" containerName="nova-metadata-log" Dec 03 11:40:50 crc kubenswrapper[5052]: I1203 11:40:50.671486 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:40:50 crc kubenswrapper[5052]: I1203 11:40:50.676140 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-metadata-config-data" Dec 03 11:40:50 crc kubenswrapper[5052]: I1203 11:40:50.676333 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-nova-metadata-internal-svc" Dec 03 11:40:50 crc kubenswrapper[5052]: I1203 11:40:50.698185 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 11:40:50 crc kubenswrapper[5052]: I1203 11:40:50.719075 5052 scope.go:117] "RemoveContainer" containerID="9336f42aa7f610b7c1c95f8f84b09607852f8edcc2bb4321efa580212ca5c69d" Dec 03 11:40:50 crc kubenswrapper[5052]: E1203 11:40:50.719544 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9336f42aa7f610b7c1c95f8f84b09607852f8edcc2bb4321efa580212ca5c69d\": container with ID starting with 9336f42aa7f610b7c1c95f8f84b09607852f8edcc2bb4321efa580212ca5c69d not found: ID does not exist" containerID="9336f42aa7f610b7c1c95f8f84b09607852f8edcc2bb4321efa580212ca5c69d" Dec 03 11:40:50 crc kubenswrapper[5052]: I1203 11:40:50.719578 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9336f42aa7f610b7c1c95f8f84b09607852f8edcc2bb4321efa580212ca5c69d"} err="failed to get container status \"9336f42aa7f610b7c1c95f8f84b09607852f8edcc2bb4321efa580212ca5c69d\": rpc error: code = NotFound desc = could not find container \"9336f42aa7f610b7c1c95f8f84b09607852f8edcc2bb4321efa580212ca5c69d\": container with ID starting with 9336f42aa7f610b7c1c95f8f84b09607852f8edcc2bb4321efa580212ca5c69d not found: ID does not exist" Dec 03 11:40:50 crc kubenswrapper[5052]: I1203 11:40:50.719606 5052 scope.go:117] "RemoveContainer" containerID="c6354c146b16bffa1182e6875189002d155dbd94c6891dbd34541814d6f4ee08" Dec 03 11:40:50 crc kubenswrapper[5052]: E1203 11:40:50.719886 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c6354c146b16bffa1182e6875189002d155dbd94c6891dbd34541814d6f4ee08\": container with ID starting with c6354c146b16bffa1182e6875189002d155dbd94c6891dbd34541814d6f4ee08 not found: ID does not exist" containerID="c6354c146b16bffa1182e6875189002d155dbd94c6891dbd34541814d6f4ee08" Dec 03 11:40:50 crc kubenswrapper[5052]: I1203 11:40:50.719933 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c6354c146b16bffa1182e6875189002d155dbd94c6891dbd34541814d6f4ee08"} err="failed to get container status \"c6354c146b16bffa1182e6875189002d155dbd94c6891dbd34541814d6f4ee08\": rpc error: code = NotFound desc = could not find container \"c6354c146b16bffa1182e6875189002d155dbd94c6891dbd34541814d6f4ee08\": container with ID starting with c6354c146b16bffa1182e6875189002d155dbd94c6891dbd34541814d6f4ee08 not found: ID does not exist" Dec 03 11:40:50 crc kubenswrapper[5052]: I1203 11:40:50.719961 5052 scope.go:117] "RemoveContainer" containerID="9336f42aa7f610b7c1c95f8f84b09607852f8edcc2bb4321efa580212ca5c69d" Dec 03 11:40:50 crc kubenswrapper[5052]: I1203 11:40:50.720234 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9336f42aa7f610b7c1c95f8f84b09607852f8edcc2bb4321efa580212ca5c69d"} err="failed to get container status \"9336f42aa7f610b7c1c95f8f84b09607852f8edcc2bb4321efa580212ca5c69d\": rpc error: code = NotFound desc = could not find container \"9336f42aa7f610b7c1c95f8f84b09607852f8edcc2bb4321efa580212ca5c69d\": container with ID starting with 9336f42aa7f610b7c1c95f8f84b09607852f8edcc2bb4321efa580212ca5c69d not found: ID does not exist" Dec 03 11:40:50 crc kubenswrapper[5052]: I1203 11:40:50.720263 5052 scope.go:117] "RemoveContainer" containerID="c6354c146b16bffa1182e6875189002d155dbd94c6891dbd34541814d6f4ee08" Dec 03 11:40:50 crc kubenswrapper[5052]: I1203 11:40:50.720434 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c6354c146b16bffa1182e6875189002d155dbd94c6891dbd34541814d6f4ee08"} err="failed to get container status \"c6354c146b16bffa1182e6875189002d155dbd94c6891dbd34541814d6f4ee08\": rpc error: code = NotFound desc = could not find container \"c6354c146b16bffa1182e6875189002d155dbd94c6891dbd34541814d6f4ee08\": container with ID starting with c6354c146b16bffa1182e6875189002d155dbd94c6891dbd34541814d6f4ee08 not found: ID does not exist" Dec 03 11:40:50 crc kubenswrapper[5052]: I1203 11:40:50.722574 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e0ad8648-db9b-46f7-b017-2cede96ad400-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"e0ad8648-db9b-46f7-b017-2cede96ad400\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:40:50 crc kubenswrapper[5052]: I1203 11:40:50.722755 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/e0ad8648-db9b-46f7-b017-2cede96ad400-logs\") pod \"nova-metadata-0\" (UID: \"e0ad8648-db9b-46f7-b017-2cede96ad400\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:40:50 crc kubenswrapper[5052]: I1203 11:40:50.722832 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/e0ad8648-db9b-46f7-b017-2cede96ad400-nova-metadata-tls-certs\") pod \"nova-metadata-0\" (UID: \"e0ad8648-db9b-46f7-b017-2cede96ad400\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:40:50 crc kubenswrapper[5052]: I1203 11:40:50.722898 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t2pmq\" (UniqueName: \"kubernetes.io/projected/e0ad8648-db9b-46f7-b017-2cede96ad400-kube-api-access-t2pmq\") pod \"nova-metadata-0\" (UID: \"e0ad8648-db9b-46f7-b017-2cede96ad400\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:40:50 crc kubenswrapper[5052]: I1203 11:40:50.723832 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/e0ad8648-db9b-46f7-b017-2cede96ad400-config-data\") pod \"nova-metadata-0\" (UID: \"e0ad8648-db9b-46f7-b017-2cede96ad400\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:40:50 crc kubenswrapper[5052]: I1203 11:40:50.825485 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/e0ad8648-db9b-46f7-b017-2cede96ad400-logs\") pod \"nova-metadata-0\" (UID: \"e0ad8648-db9b-46f7-b017-2cede96ad400\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:40:50 crc kubenswrapper[5052]: I1203 11:40:50.825631 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/e0ad8648-db9b-46f7-b017-2cede96ad400-nova-metadata-tls-certs\") pod \"nova-metadata-0\" (UID: \"e0ad8648-db9b-46f7-b017-2cede96ad400\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:40:50 crc kubenswrapper[5052]: I1203 11:40:50.825682 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-t2pmq\" (UniqueName: \"kubernetes.io/projected/e0ad8648-db9b-46f7-b017-2cede96ad400-kube-api-access-t2pmq\") pod \"nova-metadata-0\" (UID: \"e0ad8648-db9b-46f7-b017-2cede96ad400\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:40:50 crc kubenswrapper[5052]: I1203 11:40:50.825746 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/e0ad8648-db9b-46f7-b017-2cede96ad400-config-data\") pod \"nova-metadata-0\" (UID: \"e0ad8648-db9b-46f7-b017-2cede96ad400\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:40:50 crc kubenswrapper[5052]: I1203 11:40:50.825783 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e0ad8648-db9b-46f7-b017-2cede96ad400-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"e0ad8648-db9b-46f7-b017-2cede96ad400\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:40:50 crc kubenswrapper[5052]: I1203 11:40:50.825931 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/e0ad8648-db9b-46f7-b017-2cede96ad400-logs\") pod \"nova-metadata-0\" (UID: \"e0ad8648-db9b-46f7-b017-2cede96ad400\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:40:50 crc kubenswrapper[5052]: I1203 11:40:50.830682 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/e0ad8648-db9b-46f7-b017-2cede96ad400-nova-metadata-tls-certs\") pod \"nova-metadata-0\" (UID: \"e0ad8648-db9b-46f7-b017-2cede96ad400\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:40:50 crc kubenswrapper[5052]: I1203 11:40:50.831934 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e0ad8648-db9b-46f7-b017-2cede96ad400-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"e0ad8648-db9b-46f7-b017-2cede96ad400\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:40:50 crc kubenswrapper[5052]: I1203 11:40:50.836246 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/e0ad8648-db9b-46f7-b017-2cede96ad400-config-data\") pod \"nova-metadata-0\" (UID: \"e0ad8648-db9b-46f7-b017-2cede96ad400\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:40:50 crc kubenswrapper[5052]: I1203 11:40:50.842004 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-t2pmq\" (UniqueName: \"kubernetes.io/projected/e0ad8648-db9b-46f7-b017-2cede96ad400-kube-api-access-t2pmq\") pod \"nova-metadata-0\" (UID: \"e0ad8648-db9b-46f7-b017-2cede96ad400\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:40:51 crc kubenswrapper[5052]: I1203 11:40:51.008177 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:40:51 crc kubenswrapper[5052]: W1203 11:40:51.460498 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode0ad8648_db9b_46f7_b017_2cede96ad400.slice/crio-5157dc8c64cbe91a10a95b5f7bf1c749b75c15342793eab527d03069157c5c96 WatchSource:0}: Error finding container 5157dc8c64cbe91a10a95b5f7bf1c749b75c15342793eab527d03069157c5c96: Status 404 returned error can't find the container with id 5157dc8c64cbe91a10a95b5f7bf1c749b75c15342793eab527d03069157c5c96 Dec 03 11:40:51 crc kubenswrapper[5052]: I1203 11:40:51.465619 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 11:40:51 crc kubenswrapper[5052]: I1203 11:40:51.597815 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"e0ad8648-db9b-46f7-b017-2cede96ad400","Type":"ContainerStarted","Data":"5157dc8c64cbe91a10a95b5f7bf1c749b75c15342793eab527d03069157c5c96"} Dec 03 11:40:51 crc kubenswrapper[5052]: I1203 11:40:51.683637 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="115e3fec-ca17-47d7-a079-6b9bb3a9dd81" path="/var/lib/kubelet/pods/115e3fec-ca17-47d7-a079-6b9bb3a9dd81/volumes" Dec 03 11:40:52 crc kubenswrapper[5052]: I1203 11:40:52.608168 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"e0ad8648-db9b-46f7-b017-2cede96ad400","Type":"ContainerStarted","Data":"4ad54ed3f890bb5eda1753ebe8bcf0aabc5fee6943b81588d00f5a1498514553"} Dec 03 11:40:52 crc kubenswrapper[5052]: I1203 11:40:52.608216 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"e0ad8648-db9b-46f7-b017-2cede96ad400","Type":"ContainerStarted","Data":"85e249bc476c7d0391b07cdd63b9907b54f7f7f1f5b5d1eefd2857df626c5545"} Dec 03 11:40:52 crc kubenswrapper[5052]: I1203 11:40:52.638449 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-metadata-0" podStartSLOduration=2.638425093 podStartE2EDuration="2.638425093s" podCreationTimestamp="2025-12-03 11:40:50 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:40:52.631389348 +0000 UTC m=+1243.219631132" watchObservedRunningTime="2025-12-03 11:40:52.638425093 +0000 UTC m=+1243.226666907" Dec 03 11:40:53 crc kubenswrapper[5052]: I1203 11:40:53.619105 5052 generic.go:334] "Generic (PLEG): container finished" podID="1285046f-826b-4a30-a414-46e598daf9ca" containerID="a47db30ff0a5b853393f81265de91d0780b0ae16051c26ce22422a53ba83dc46" exitCode=0 Dec 03 11:40:53 crc kubenswrapper[5052]: I1203 11:40:53.619174 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-cw46s" event={"ID":"1285046f-826b-4a30-a414-46e598daf9ca","Type":"ContainerDied","Data":"a47db30ff0a5b853393f81265de91d0780b0ae16051c26ce22422a53ba83dc46"} Dec 03 11:40:53 crc kubenswrapper[5052]: I1203 11:40:53.621428 5052 generic.go:334] "Generic (PLEG): container finished" podID="8934c593-c918-48c9-b19b-e7c0e47474c2" containerID="1e380bc4c681225a08515748fc7e47b1e5ab3aca3455957a89e3055115aa44a9" exitCode=0 Dec 03 11:40:53 crc kubenswrapper[5052]: I1203 11:40:53.621531 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-cell-mapping-6lr8z" event={"ID":"8934c593-c918-48c9-b19b-e7c0e47474c2","Type":"ContainerDied","Data":"1e380bc4c681225a08515748fc7e47b1e5ab3aca3455957a89e3055115aa44a9"} Dec 03 11:40:54 crc kubenswrapper[5052]: I1203 11:40:54.819941 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:40:54 crc kubenswrapper[5052]: I1203 11:40:54.820269 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:40:54 crc kubenswrapper[5052]: I1203 11:40:54.992279 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:40:54 crc kubenswrapper[5052]: I1203 11:40:54.998132 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-cw46s" Dec 03 11:40:55 crc kubenswrapper[5052]: I1203 11:40:55.020165 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g5p6x\" (UniqueName: \"kubernetes.io/projected/1285046f-826b-4a30-a414-46e598daf9ca-kube-api-access-g5p6x\") pod \"1285046f-826b-4a30-a414-46e598daf9ca\" (UID: \"1285046f-826b-4a30-a414-46e598daf9ca\") " Dec 03 11:40:55 crc kubenswrapper[5052]: I1203 11:40:55.020284 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/1285046f-826b-4a30-a414-46e598daf9ca-config-data\") pod \"1285046f-826b-4a30-a414-46e598daf9ca\" (UID: \"1285046f-826b-4a30-a414-46e598daf9ca\") " Dec 03 11:40:55 crc kubenswrapper[5052]: I1203 11:40:55.020351 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/1285046f-826b-4a30-a414-46e598daf9ca-scripts\") pod \"1285046f-826b-4a30-a414-46e598daf9ca\" (UID: \"1285046f-826b-4a30-a414-46e598daf9ca\") " Dec 03 11:40:55 crc kubenswrapper[5052]: I1203 11:40:55.020395 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1285046f-826b-4a30-a414-46e598daf9ca-combined-ca-bundle\") pod \"1285046f-826b-4a30-a414-46e598daf9ca\" (UID: \"1285046f-826b-4a30-a414-46e598daf9ca\") " Dec 03 11:40:55 crc kubenswrapper[5052]: I1203 11:40:55.025740 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1285046f-826b-4a30-a414-46e598daf9ca-scripts" (OuterVolumeSpecName: "scripts") pod "1285046f-826b-4a30-a414-46e598daf9ca" (UID: "1285046f-826b-4a30-a414-46e598daf9ca"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:40:55 crc kubenswrapper[5052]: I1203 11:40:55.031700 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1285046f-826b-4a30-a414-46e598daf9ca-kube-api-access-g5p6x" (OuterVolumeSpecName: "kube-api-access-g5p6x") pod "1285046f-826b-4a30-a414-46e598daf9ca" (UID: "1285046f-826b-4a30-a414-46e598daf9ca"). InnerVolumeSpecName "kube-api-access-g5p6x". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:40:55 crc kubenswrapper[5052]: I1203 11:40:55.031977 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:40:55 crc kubenswrapper[5052]: I1203 11:40:55.053152 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1285046f-826b-4a30-a414-46e598daf9ca-config-data" (OuterVolumeSpecName: "config-data") pod "1285046f-826b-4a30-a414-46e598daf9ca" (UID: "1285046f-826b-4a30-a414-46e598daf9ca"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:40:55 crc kubenswrapper[5052]: I1203 11:40:55.070376 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1285046f-826b-4a30-a414-46e598daf9ca-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "1285046f-826b-4a30-a414-46e598daf9ca" (UID: "1285046f-826b-4a30-a414-46e598daf9ca"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:40:55 crc kubenswrapper[5052]: I1203 11:40:55.123916 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/1285046f-826b-4a30-a414-46e598daf9ca-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:40:55 crc kubenswrapper[5052]: I1203 11:40:55.123960 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1285046f-826b-4a30-a414-46e598daf9ca-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:40:55 crc kubenswrapper[5052]: I1203 11:40:55.123976 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-g5p6x\" (UniqueName: \"kubernetes.io/projected/1285046f-826b-4a30-a414-46e598daf9ca-kube-api-access-g5p6x\") on node \"crc\" DevicePath \"\"" Dec 03 11:40:55 crc kubenswrapper[5052]: I1203 11:40:55.123989 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/1285046f-826b-4a30-a414-46e598daf9ca-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:40:55 crc kubenswrapper[5052]: I1203 11:40:55.128673 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-cell-mapping-6lr8z" Dec 03 11:40:55 crc kubenswrapper[5052]: I1203 11:40:55.227346 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/8934c593-c918-48c9-b19b-e7c0e47474c2-config-data\") pod \"8934c593-c918-48c9-b19b-e7c0e47474c2\" (UID: \"8934c593-c918-48c9-b19b-e7c0e47474c2\") " Dec 03 11:40:55 crc kubenswrapper[5052]: I1203 11:40:55.227498 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8934c593-c918-48c9-b19b-e7c0e47474c2-combined-ca-bundle\") pod \"8934c593-c918-48c9-b19b-e7c0e47474c2\" (UID: \"8934c593-c918-48c9-b19b-e7c0e47474c2\") " Dec 03 11:40:55 crc kubenswrapper[5052]: I1203 11:40:55.227535 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-snc9z\" (UniqueName: \"kubernetes.io/projected/8934c593-c918-48c9-b19b-e7c0e47474c2-kube-api-access-snc9z\") pod \"8934c593-c918-48c9-b19b-e7c0e47474c2\" (UID: \"8934c593-c918-48c9-b19b-e7c0e47474c2\") " Dec 03 11:40:55 crc kubenswrapper[5052]: I1203 11:40:55.227646 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/8934c593-c918-48c9-b19b-e7c0e47474c2-scripts\") pod \"8934c593-c918-48c9-b19b-e7c0e47474c2\" (UID: \"8934c593-c918-48c9-b19b-e7c0e47474c2\") " Dec 03 11:40:55 crc kubenswrapper[5052]: I1203 11:40:55.231619 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8934c593-c918-48c9-b19b-e7c0e47474c2-scripts" (OuterVolumeSpecName: "scripts") pod "8934c593-c918-48c9-b19b-e7c0e47474c2" (UID: "8934c593-c918-48c9-b19b-e7c0e47474c2"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:40:55 crc kubenswrapper[5052]: I1203 11:40:55.235158 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8934c593-c918-48c9-b19b-e7c0e47474c2-kube-api-access-snc9z" (OuterVolumeSpecName: "kube-api-access-snc9z") pod "8934c593-c918-48c9-b19b-e7c0e47474c2" (UID: "8934c593-c918-48c9-b19b-e7c0e47474c2"). InnerVolumeSpecName "kube-api-access-snc9z". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:40:55 crc kubenswrapper[5052]: I1203 11:40:55.259822 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8934c593-c918-48c9-b19b-e7c0e47474c2-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "8934c593-c918-48c9-b19b-e7c0e47474c2" (UID: "8934c593-c918-48c9-b19b-e7c0e47474c2"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:40:55 crc kubenswrapper[5052]: I1203 11:40:55.260145 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8934c593-c918-48c9-b19b-e7c0e47474c2-config-data" (OuterVolumeSpecName: "config-data") pod "8934c593-c918-48c9-b19b-e7c0e47474c2" (UID: "8934c593-c918-48c9-b19b-e7c0e47474c2"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:40:55 crc kubenswrapper[5052]: I1203 11:40:55.329276 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/8934c593-c918-48c9-b19b-e7c0e47474c2-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:40:55 crc kubenswrapper[5052]: I1203 11:40:55.329315 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/8934c593-c918-48c9-b19b-e7c0e47474c2-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:40:55 crc kubenswrapper[5052]: I1203 11:40:55.329327 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8934c593-c918-48c9-b19b-e7c0e47474c2-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:40:55 crc kubenswrapper[5052]: I1203 11:40:55.329337 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-snc9z\" (UniqueName: \"kubernetes.io/projected/8934c593-c918-48c9-b19b-e7c0e47474c2-kube-api-access-snc9z\") on node \"crc\" DevicePath \"\"" Dec 03 11:40:55 crc kubenswrapper[5052]: I1203 11:40:55.616642 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:40:55 crc kubenswrapper[5052]: I1203 11:40:55.657064 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-cw46s" event={"ID":"1285046f-826b-4a30-a414-46e598daf9ca","Type":"ContainerDied","Data":"0f2ff91c0b7e75c1a8b059bf7a45c2212c1135f700ac9c4c16ecc1865b7dd13a"} Dec 03 11:40:55 crc kubenswrapper[5052]: I1203 11:40:55.657107 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0f2ff91c0b7e75c1a8b059bf7a45c2212c1135f700ac9c4c16ecc1865b7dd13a" Dec 03 11:40:55 crc kubenswrapper[5052]: I1203 11:40:55.657171 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-cw46s" Dec 03 11:40:55 crc kubenswrapper[5052]: I1203 11:40:55.662561 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-cell-mapping-6lr8z" event={"ID":"8934c593-c918-48c9-b19b-e7c0e47474c2","Type":"ContainerDied","Data":"58b0221599cb89c02b4a754ec37ae1fa6e44b7dcd97d27cc4b3cdc42de26295f"} Dec 03 11:40:55 crc kubenswrapper[5052]: I1203 11:40:55.662606 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="58b0221599cb89c02b4a754ec37ae1fa6e44b7dcd97d27cc4b3cdc42de26295f" Dec 03 11:40:55 crc kubenswrapper[5052]: I1203 11:40:55.662674 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-cell-mapping-6lr8z" Dec 03 11:40:55 crc kubenswrapper[5052]: I1203 11:40:55.674453 5052 generic.go:334] "Generic (PLEG): container finished" podID="5deeda32-3ab5-4e5a-a41a-320996d2a162" containerID="7abc54fcc37732cf92e685189f39cb5d0dd7bb107427d126fe6b14d462fa03bd" exitCode=137 Dec 03 11:40:55 crc kubenswrapper[5052]: I1203 11:40:55.675089 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:40:55 crc kubenswrapper[5052]: I1203 11:40:55.712951 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"5deeda32-3ab5-4e5a-a41a-320996d2a162","Type":"ContainerDied","Data":"7abc54fcc37732cf92e685189f39cb5d0dd7bb107427d126fe6b14d462fa03bd"} Dec 03 11:40:55 crc kubenswrapper[5052]: I1203 11:40:55.713008 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"5deeda32-3ab5-4e5a-a41a-320996d2a162","Type":"ContainerDied","Data":"7146a6129d7ab2ee25c0bc4cbe3168ca1c5584af37f6351da8d6fa3df818af1e"} Dec 03 11:40:55 crc kubenswrapper[5052]: I1203 11:40:55.713031 5052 scope.go:117] "RemoveContainer" containerID="7abc54fcc37732cf92e685189f39cb5d0dd7bb107427d126fe6b14d462fa03bd" Dec 03 11:40:55 crc kubenswrapper[5052]: I1203 11:40:55.733948 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:40:55 crc kubenswrapper[5052]: I1203 11:40:55.734893 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/5deeda32-3ab5-4e5a-a41a-320996d2a162-log-httpd\") pod \"5deeda32-3ab5-4e5a-a41a-320996d2a162\" (UID: \"5deeda32-3ab5-4e5a-a41a-320996d2a162\") " Dec 03 11:40:55 crc kubenswrapper[5052]: I1203 11:40:55.734945 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/5deeda32-3ab5-4e5a-a41a-320996d2a162-run-httpd\") pod \"5deeda32-3ab5-4e5a-a41a-320996d2a162\" (UID: \"5deeda32-3ab5-4e5a-a41a-320996d2a162\") " Dec 03 11:40:55 crc kubenswrapper[5052]: I1203 11:40:55.735002 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5deeda32-3ab5-4e5a-a41a-320996d2a162-combined-ca-bundle\") pod \"5deeda32-3ab5-4e5a-a41a-320996d2a162\" (UID: \"5deeda32-3ab5-4e5a-a41a-320996d2a162\") " Dec 03 11:40:55 crc kubenswrapper[5052]: I1203 11:40:55.735040 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/5deeda32-3ab5-4e5a-a41a-320996d2a162-sg-core-conf-yaml\") pod \"5deeda32-3ab5-4e5a-a41a-320996d2a162\" (UID: \"5deeda32-3ab5-4e5a-a41a-320996d2a162\") " Dec 03 11:40:55 crc kubenswrapper[5052]: I1203 11:40:55.735061 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/5deeda32-3ab5-4e5a-a41a-320996d2a162-config-data\") pod \"5deeda32-3ab5-4e5a-a41a-320996d2a162\" (UID: \"5deeda32-3ab5-4e5a-a41a-320996d2a162\") " Dec 03 11:40:55 crc kubenswrapper[5052]: I1203 11:40:55.735143 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/5deeda32-3ab5-4e5a-a41a-320996d2a162-scripts\") pod \"5deeda32-3ab5-4e5a-a41a-320996d2a162\" (UID: \"5deeda32-3ab5-4e5a-a41a-320996d2a162\") " Dec 03 11:40:55 crc kubenswrapper[5052]: I1203 11:40:55.735168 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7smpm\" (UniqueName: \"kubernetes.io/projected/5deeda32-3ab5-4e5a-a41a-320996d2a162-kube-api-access-7smpm\") pod \"5deeda32-3ab5-4e5a-a41a-320996d2a162\" (UID: \"5deeda32-3ab5-4e5a-a41a-320996d2a162\") " Dec 03 11:40:55 crc kubenswrapper[5052]: I1203 11:40:55.738611 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5deeda32-3ab5-4e5a-a41a-320996d2a162-run-httpd" (OuterVolumeSpecName: "run-httpd") pod "5deeda32-3ab5-4e5a-a41a-320996d2a162" (UID: "5deeda32-3ab5-4e5a-a41a-320996d2a162"). InnerVolumeSpecName "run-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:40:55 crc kubenswrapper[5052]: I1203 11:40:55.739389 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5deeda32-3ab5-4e5a-a41a-320996d2a162-log-httpd" (OuterVolumeSpecName: "log-httpd") pod "5deeda32-3ab5-4e5a-a41a-320996d2a162" (UID: "5deeda32-3ab5-4e5a-a41a-320996d2a162"). InnerVolumeSpecName "log-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:40:55 crc kubenswrapper[5052]: I1203 11:40:55.742908 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-0"] Dec 03 11:40:55 crc kubenswrapper[5052]: I1203 11:40:55.742997 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5deeda32-3ab5-4e5a-a41a-320996d2a162-scripts" (OuterVolumeSpecName: "scripts") pod "5deeda32-3ab5-4e5a-a41a-320996d2a162" (UID: "5deeda32-3ab5-4e5a-a41a-320996d2a162"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:40:55 crc kubenswrapper[5052]: E1203 11:40:55.743301 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="8934c593-c918-48c9-b19b-e7c0e47474c2" containerName="nova-manage" Dec 03 11:40:55 crc kubenswrapper[5052]: I1203 11:40:55.743319 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="8934c593-c918-48c9-b19b-e7c0e47474c2" containerName="nova-manage" Dec 03 11:40:55 crc kubenswrapper[5052]: E1203 11:40:55.743332 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="5deeda32-3ab5-4e5a-a41a-320996d2a162" containerName="sg-core" Dec 03 11:40:55 crc kubenswrapper[5052]: I1203 11:40:55.743337 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="5deeda32-3ab5-4e5a-a41a-320996d2a162" containerName="sg-core" Dec 03 11:40:55 crc kubenswrapper[5052]: E1203 11:40:55.743353 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="5deeda32-3ab5-4e5a-a41a-320996d2a162" containerName="ceilometer-notification-agent" Dec 03 11:40:55 crc kubenswrapper[5052]: I1203 11:40:55.743359 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="5deeda32-3ab5-4e5a-a41a-320996d2a162" containerName="ceilometer-notification-agent" Dec 03 11:40:55 crc kubenswrapper[5052]: E1203 11:40:55.743371 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="1285046f-826b-4a30-a414-46e598daf9ca" containerName="nova-cell1-conductor-db-sync" Dec 03 11:40:55 crc kubenswrapper[5052]: I1203 11:40:55.743376 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="1285046f-826b-4a30-a414-46e598daf9ca" containerName="nova-cell1-conductor-db-sync" Dec 03 11:40:55 crc kubenswrapper[5052]: E1203 11:40:55.743387 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="5deeda32-3ab5-4e5a-a41a-320996d2a162" containerName="ceilometer-central-agent" Dec 03 11:40:55 crc kubenswrapper[5052]: I1203 11:40:55.743393 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="5deeda32-3ab5-4e5a-a41a-320996d2a162" containerName="ceilometer-central-agent" Dec 03 11:40:55 crc kubenswrapper[5052]: E1203 11:40:55.743406 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="5deeda32-3ab5-4e5a-a41a-320996d2a162" containerName="proxy-httpd" Dec 03 11:40:55 crc kubenswrapper[5052]: I1203 11:40:55.743411 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="5deeda32-3ab5-4e5a-a41a-320996d2a162" containerName="proxy-httpd" Dec 03 11:40:55 crc kubenswrapper[5052]: I1203 11:40:55.743632 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="5deeda32-3ab5-4e5a-a41a-320996d2a162" containerName="ceilometer-central-agent" Dec 03 11:40:55 crc kubenswrapper[5052]: I1203 11:40:55.743648 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="8934c593-c918-48c9-b19b-e7c0e47474c2" containerName="nova-manage" Dec 03 11:40:55 crc kubenswrapper[5052]: I1203 11:40:55.743665 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="5deeda32-3ab5-4e5a-a41a-320996d2a162" containerName="ceilometer-notification-agent" Dec 03 11:40:55 crc kubenswrapper[5052]: I1203 11:40:55.743673 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="5deeda32-3ab5-4e5a-a41a-320996d2a162" containerName="proxy-httpd" Dec 03 11:40:55 crc kubenswrapper[5052]: I1203 11:40:55.743692 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="1285046f-826b-4a30-a414-46e598daf9ca" containerName="nova-cell1-conductor-db-sync" Dec 03 11:40:55 crc kubenswrapper[5052]: I1203 11:40:55.743699 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="5deeda32-3ab5-4e5a-a41a-320996d2a162" containerName="sg-core" Dec 03 11:40:55 crc kubenswrapper[5052]: I1203 11:40:55.744276 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 11:40:55 crc kubenswrapper[5052]: I1203 11:40:55.747502 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5deeda32-3ab5-4e5a-a41a-320996d2a162-kube-api-access-7smpm" (OuterVolumeSpecName: "kube-api-access-7smpm") pod "5deeda32-3ab5-4e5a-a41a-320996d2a162" (UID: "5deeda32-3ab5-4e5a-a41a-320996d2a162"). InnerVolumeSpecName "kube-api-access-7smpm". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:40:55 crc kubenswrapper[5052]: I1203 11:40:55.750914 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell1-conductor-config-data" Dec 03 11:40:55 crc kubenswrapper[5052]: I1203 11:40:55.756755 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-0"] Dec 03 11:40:55 crc kubenswrapper[5052]: I1203 11:40:55.778568 5052 scope.go:117] "RemoveContainer" containerID="c0518ba0cbf4a620ac95e4bd0300d3debf3bfea6cc7e4cd9e7dd9c663af2143f" Dec 03 11:40:55 crc kubenswrapper[5052]: I1203 11:40:55.830672 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5deeda32-3ab5-4e5a-a41a-320996d2a162-sg-core-conf-yaml" (OuterVolumeSpecName: "sg-core-conf-yaml") pod "5deeda32-3ab5-4e5a-a41a-320996d2a162" (UID: "5deeda32-3ab5-4e5a-a41a-320996d2a162"). InnerVolumeSpecName "sg-core-conf-yaml". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:40:55 crc kubenswrapper[5052]: I1203 11:40:55.844604 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/2501eda8-a78d-4609-a465-761e89bb291e-config-data\") pod \"nova-cell1-conductor-0\" (UID: \"2501eda8-a78d-4609-a465-761e89bb291e\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 11:40:55 crc kubenswrapper[5052]: I1203 11:40:55.844713 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rnsxg\" (UniqueName: \"kubernetes.io/projected/2501eda8-a78d-4609-a465-761e89bb291e-kube-api-access-rnsxg\") pod \"nova-cell1-conductor-0\" (UID: \"2501eda8-a78d-4609-a465-761e89bb291e\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 11:40:55 crc kubenswrapper[5052]: I1203 11:40:55.844808 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2501eda8-a78d-4609-a465-761e89bb291e-combined-ca-bundle\") pod \"nova-cell1-conductor-0\" (UID: \"2501eda8-a78d-4609-a465-761e89bb291e\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 11:40:55 crc kubenswrapper[5052]: I1203 11:40:55.844862 5052 reconciler_common.go:293] "Volume detached for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/5deeda32-3ab5-4e5a-a41a-320996d2a162-run-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 11:40:55 crc kubenswrapper[5052]: I1203 11:40:55.844876 5052 reconciler_common.go:293] "Volume detached for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/5deeda32-3ab5-4e5a-a41a-320996d2a162-sg-core-conf-yaml\") on node \"crc\" DevicePath \"\"" Dec 03 11:40:55 crc kubenswrapper[5052]: I1203 11:40:55.844889 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/5deeda32-3ab5-4e5a-a41a-320996d2a162-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:40:55 crc kubenswrapper[5052]: I1203 11:40:55.844901 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-7smpm\" (UniqueName: \"kubernetes.io/projected/5deeda32-3ab5-4e5a-a41a-320996d2a162-kube-api-access-7smpm\") on node \"crc\" DevicePath \"\"" Dec 03 11:40:55 crc kubenswrapper[5052]: I1203 11:40:55.844912 5052 reconciler_common.go:293] "Volume detached for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/5deeda32-3ab5-4e5a-a41a-320996d2a162-log-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 11:40:55 crc kubenswrapper[5052]: I1203 11:40:55.883730 5052 scope.go:117] "RemoveContainer" containerID="1da162c0a015d19ca065ff14467a32ad1056d33565d61c8e83cecbfc7e351ce9" Dec 03 11:40:55 crc kubenswrapper[5052]: I1203 11:40:55.891707 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5deeda32-3ab5-4e5a-a41a-320996d2a162-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "5deeda32-3ab5-4e5a-a41a-320996d2a162" (UID: "5deeda32-3ab5-4e5a-a41a-320996d2a162"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:40:55 crc kubenswrapper[5052]: I1203 11:40:55.904188 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-api-0" podUID="07b13d48-fa3d-4274-b138-b1c54e05811b" containerName="nova-api-log" probeResult="failure" output="Get \"http://10.217.0.166:8774/\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Dec 03 11:40:55 crc kubenswrapper[5052]: I1203 11:40:55.904641 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-api-0" podUID="07b13d48-fa3d-4274-b138-b1c54e05811b" containerName="nova-api-api" probeResult="failure" output="Get \"http://10.217.0.166:8774/\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Dec 03 11:40:55 crc kubenswrapper[5052]: I1203 11:40:55.953498 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2501eda8-a78d-4609-a465-761e89bb291e-combined-ca-bundle\") pod \"nova-cell1-conductor-0\" (UID: \"2501eda8-a78d-4609-a465-761e89bb291e\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 11:40:55 crc kubenswrapper[5052]: I1203 11:40:55.953599 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/2501eda8-a78d-4609-a465-761e89bb291e-config-data\") pod \"nova-cell1-conductor-0\" (UID: \"2501eda8-a78d-4609-a465-761e89bb291e\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 11:40:55 crc kubenswrapper[5052]: I1203 11:40:55.953658 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-rnsxg\" (UniqueName: \"kubernetes.io/projected/2501eda8-a78d-4609-a465-761e89bb291e-kube-api-access-rnsxg\") pod \"nova-cell1-conductor-0\" (UID: \"2501eda8-a78d-4609-a465-761e89bb291e\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 11:40:55 crc kubenswrapper[5052]: I1203 11:40:55.953728 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5deeda32-3ab5-4e5a-a41a-320996d2a162-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:40:55 crc kubenswrapper[5052]: I1203 11:40:55.962676 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5deeda32-3ab5-4e5a-a41a-320996d2a162-config-data" (OuterVolumeSpecName: "config-data") pod "5deeda32-3ab5-4e5a-a41a-320996d2a162" (UID: "5deeda32-3ab5-4e5a-a41a-320996d2a162"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:40:55 crc kubenswrapper[5052]: I1203 11:40:55.965415 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/2501eda8-a78d-4609-a465-761e89bb291e-config-data\") pod \"nova-cell1-conductor-0\" (UID: \"2501eda8-a78d-4609-a465-761e89bb291e\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 11:40:55 crc kubenswrapper[5052]: I1203 11:40:55.970267 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2501eda8-a78d-4609-a465-761e89bb291e-combined-ca-bundle\") pod \"nova-cell1-conductor-0\" (UID: \"2501eda8-a78d-4609-a465-761e89bb291e\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 11:40:55 crc kubenswrapper[5052]: I1203 11:40:55.997246 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-rnsxg\" (UniqueName: \"kubernetes.io/projected/2501eda8-a78d-4609-a465-761e89bb291e-kube-api-access-rnsxg\") pod \"nova-cell1-conductor-0\" (UID: \"2501eda8-a78d-4609-a465-761e89bb291e\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 11:40:56 crc kubenswrapper[5052]: I1203 11:40:56.009625 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:40:56 crc kubenswrapper[5052]: I1203 11:40:56.010649 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:40:56 crc kubenswrapper[5052]: I1203 11:40:56.056057 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/5deeda32-3ab5-4e5a-a41a-320996d2a162-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:40:56 crc kubenswrapper[5052]: I1203 11:40:56.061119 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 11:40:56 crc kubenswrapper[5052]: I1203 11:40:56.061334 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-api-0" podUID="07b13d48-fa3d-4274-b138-b1c54e05811b" containerName="nova-api-log" containerID="cri-o://d2f41ae028ba676e4abc4a6c3cef0c2a13e596f42f7e81f790e0406f1bd2f8ba" gracePeriod=30 Dec 03 11:40:56 crc kubenswrapper[5052]: I1203 11:40:56.061789 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-api-0" podUID="07b13d48-fa3d-4274-b138-b1c54e05811b" containerName="nova-api-api" containerID="cri-o://781f723d704c08a0bb05affca300ba883bf5c2febee85b35ef6e758a1b497a8e" gracePeriod=30 Dec 03 11:40:56 crc kubenswrapper[5052]: I1203 11:40:56.079667 5052 scope.go:117] "RemoveContainer" containerID="dd86a49370d10cb46359fe17d587fd45828c313fb14c9069b419453b88909a65" Dec 03 11:40:56 crc kubenswrapper[5052]: I1203 11:40:56.090630 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 11:40:56 crc kubenswrapper[5052]: I1203 11:40:56.154614 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:40:56 crc kubenswrapper[5052]: I1203 11:40:56.180163 5052 scope.go:117] "RemoveContainer" containerID="7abc54fcc37732cf92e685189f39cb5d0dd7bb107427d126fe6b14d462fa03bd" Dec 03 11:40:56 crc kubenswrapper[5052]: E1203 11:40:56.180536 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7abc54fcc37732cf92e685189f39cb5d0dd7bb107427d126fe6b14d462fa03bd\": container with ID starting with 7abc54fcc37732cf92e685189f39cb5d0dd7bb107427d126fe6b14d462fa03bd not found: ID does not exist" containerID="7abc54fcc37732cf92e685189f39cb5d0dd7bb107427d126fe6b14d462fa03bd" Dec 03 11:40:56 crc kubenswrapper[5052]: I1203 11:40:56.180563 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7abc54fcc37732cf92e685189f39cb5d0dd7bb107427d126fe6b14d462fa03bd"} err="failed to get container status \"7abc54fcc37732cf92e685189f39cb5d0dd7bb107427d126fe6b14d462fa03bd\": rpc error: code = NotFound desc = could not find container \"7abc54fcc37732cf92e685189f39cb5d0dd7bb107427d126fe6b14d462fa03bd\": container with ID starting with 7abc54fcc37732cf92e685189f39cb5d0dd7bb107427d126fe6b14d462fa03bd not found: ID does not exist" Dec 03 11:40:56 crc kubenswrapper[5052]: I1203 11:40:56.180583 5052 scope.go:117] "RemoveContainer" containerID="c0518ba0cbf4a620ac95e4bd0300d3debf3bfea6cc7e4cd9e7dd9c663af2143f" Dec 03 11:40:56 crc kubenswrapper[5052]: E1203 11:40:56.180749 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c0518ba0cbf4a620ac95e4bd0300d3debf3bfea6cc7e4cd9e7dd9c663af2143f\": container with ID starting with c0518ba0cbf4a620ac95e4bd0300d3debf3bfea6cc7e4cd9e7dd9c663af2143f not found: ID does not exist" containerID="c0518ba0cbf4a620ac95e4bd0300d3debf3bfea6cc7e4cd9e7dd9c663af2143f" Dec 03 11:40:56 crc kubenswrapper[5052]: I1203 11:40:56.180766 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c0518ba0cbf4a620ac95e4bd0300d3debf3bfea6cc7e4cd9e7dd9c663af2143f"} err="failed to get container status \"c0518ba0cbf4a620ac95e4bd0300d3debf3bfea6cc7e4cd9e7dd9c663af2143f\": rpc error: code = NotFound desc = could not find container \"c0518ba0cbf4a620ac95e4bd0300d3debf3bfea6cc7e4cd9e7dd9c663af2143f\": container with ID starting with c0518ba0cbf4a620ac95e4bd0300d3debf3bfea6cc7e4cd9e7dd9c663af2143f not found: ID does not exist" Dec 03 11:40:56 crc kubenswrapper[5052]: I1203 11:40:56.180777 5052 scope.go:117] "RemoveContainer" containerID="1da162c0a015d19ca065ff14467a32ad1056d33565d61c8e83cecbfc7e351ce9" Dec 03 11:40:56 crc kubenswrapper[5052]: E1203 11:40:56.180926 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1da162c0a015d19ca065ff14467a32ad1056d33565d61c8e83cecbfc7e351ce9\": container with ID starting with 1da162c0a015d19ca065ff14467a32ad1056d33565d61c8e83cecbfc7e351ce9 not found: ID does not exist" containerID="1da162c0a015d19ca065ff14467a32ad1056d33565d61c8e83cecbfc7e351ce9" Dec 03 11:40:56 crc kubenswrapper[5052]: I1203 11:40:56.180944 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1da162c0a015d19ca065ff14467a32ad1056d33565d61c8e83cecbfc7e351ce9"} err="failed to get container status \"1da162c0a015d19ca065ff14467a32ad1056d33565d61c8e83cecbfc7e351ce9\": rpc error: code = NotFound desc = could not find container \"1da162c0a015d19ca065ff14467a32ad1056d33565d61c8e83cecbfc7e351ce9\": container with ID starting with 1da162c0a015d19ca065ff14467a32ad1056d33565d61c8e83cecbfc7e351ce9 not found: ID does not exist" Dec 03 11:40:56 crc kubenswrapper[5052]: I1203 11:40:56.180957 5052 scope.go:117] "RemoveContainer" containerID="dd86a49370d10cb46359fe17d587fd45828c313fb14c9069b419453b88909a65" Dec 03 11:40:56 crc kubenswrapper[5052]: E1203 11:40:56.181320 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dd86a49370d10cb46359fe17d587fd45828c313fb14c9069b419453b88909a65\": container with ID starting with dd86a49370d10cb46359fe17d587fd45828c313fb14c9069b419453b88909a65 not found: ID does not exist" containerID="dd86a49370d10cb46359fe17d587fd45828c313fb14c9069b419453b88909a65" Dec 03 11:40:56 crc kubenswrapper[5052]: I1203 11:40:56.181340 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dd86a49370d10cb46359fe17d587fd45828c313fb14c9069b419453b88909a65"} err="failed to get container status \"dd86a49370d10cb46359fe17d587fd45828c313fb14c9069b419453b88909a65\": rpc error: code = NotFound desc = could not find container \"dd86a49370d10cb46359fe17d587fd45828c313fb14c9069b419453b88909a65\": container with ID starting with dd86a49370d10cb46359fe17d587fd45828c313fb14c9069b419453b88909a65 not found: ID does not exist" Dec 03 11:40:56 crc kubenswrapper[5052]: I1203 11:40:56.181407 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:40:56 crc kubenswrapper[5052]: I1203 11:40:56.204605 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 11:40:56 crc kubenswrapper[5052]: I1203 11:40:56.219581 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:40:56 crc kubenswrapper[5052]: I1203 11:40:56.221860 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:40:56 crc kubenswrapper[5052]: I1203 11:40:56.225531 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:40:56 crc kubenswrapper[5052]: I1203 11:40:56.226868 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-scripts" Dec 03 11:40:56 crc kubenswrapper[5052]: I1203 11:40:56.227071 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-config-data" Dec 03 11:40:56 crc kubenswrapper[5052]: I1203 11:40:56.258920 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/6b68de4d-a6d8-495d-becf-bd9f10da4a0b-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"6b68de4d-a6d8-495d-becf-bd9f10da4a0b\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:40:56 crc kubenswrapper[5052]: I1203 11:40:56.258984 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sx6c6\" (UniqueName: \"kubernetes.io/projected/6b68de4d-a6d8-495d-becf-bd9f10da4a0b-kube-api-access-sx6c6\") pod \"ceilometer-0\" (UID: \"6b68de4d-a6d8-495d-becf-bd9f10da4a0b\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:40:56 crc kubenswrapper[5052]: I1203 11:40:56.259022 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/6b68de4d-a6d8-495d-becf-bd9f10da4a0b-run-httpd\") pod \"ceilometer-0\" (UID: \"6b68de4d-a6d8-495d-becf-bd9f10da4a0b\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:40:56 crc kubenswrapper[5052]: I1203 11:40:56.259051 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/6b68de4d-a6d8-495d-becf-bd9f10da4a0b-scripts\") pod \"ceilometer-0\" (UID: \"6b68de4d-a6d8-495d-becf-bd9f10da4a0b\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:40:56 crc kubenswrapper[5052]: I1203 11:40:56.259069 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6b68de4d-a6d8-495d-becf-bd9f10da4a0b-config-data\") pod \"ceilometer-0\" (UID: \"6b68de4d-a6d8-495d-becf-bd9f10da4a0b\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:40:56 crc kubenswrapper[5052]: I1203 11:40:56.259084 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6b68de4d-a6d8-495d-becf-bd9f10da4a0b-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"6b68de4d-a6d8-495d-becf-bd9f10da4a0b\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:40:56 crc kubenswrapper[5052]: I1203 11:40:56.259105 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/6b68de4d-a6d8-495d-becf-bd9f10da4a0b-log-httpd\") pod \"ceilometer-0\" (UID: \"6b68de4d-a6d8-495d-becf-bd9f10da4a0b\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:40:56 crc kubenswrapper[5052]: I1203 11:40:56.361262 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/6b68de4d-a6d8-495d-becf-bd9f10da4a0b-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"6b68de4d-a6d8-495d-becf-bd9f10da4a0b\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:40:56 crc kubenswrapper[5052]: I1203 11:40:56.361347 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-sx6c6\" (UniqueName: \"kubernetes.io/projected/6b68de4d-a6d8-495d-becf-bd9f10da4a0b-kube-api-access-sx6c6\") pod \"ceilometer-0\" (UID: \"6b68de4d-a6d8-495d-becf-bd9f10da4a0b\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:40:56 crc kubenswrapper[5052]: I1203 11:40:56.361395 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/6b68de4d-a6d8-495d-becf-bd9f10da4a0b-run-httpd\") pod \"ceilometer-0\" (UID: \"6b68de4d-a6d8-495d-becf-bd9f10da4a0b\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:40:56 crc kubenswrapper[5052]: I1203 11:40:56.361427 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/6b68de4d-a6d8-495d-becf-bd9f10da4a0b-scripts\") pod \"ceilometer-0\" (UID: \"6b68de4d-a6d8-495d-becf-bd9f10da4a0b\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:40:56 crc kubenswrapper[5052]: I1203 11:40:56.361449 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6b68de4d-a6d8-495d-becf-bd9f10da4a0b-config-data\") pod \"ceilometer-0\" (UID: \"6b68de4d-a6d8-495d-becf-bd9f10da4a0b\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:40:56 crc kubenswrapper[5052]: I1203 11:40:56.361467 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6b68de4d-a6d8-495d-becf-bd9f10da4a0b-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"6b68de4d-a6d8-495d-becf-bd9f10da4a0b\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:40:56 crc kubenswrapper[5052]: I1203 11:40:56.361489 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/6b68de4d-a6d8-495d-becf-bd9f10da4a0b-log-httpd\") pod \"ceilometer-0\" (UID: \"6b68de4d-a6d8-495d-becf-bd9f10da4a0b\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:40:56 crc kubenswrapper[5052]: I1203 11:40:56.362207 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/6b68de4d-a6d8-495d-becf-bd9f10da4a0b-log-httpd\") pod \"ceilometer-0\" (UID: \"6b68de4d-a6d8-495d-becf-bd9f10da4a0b\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:40:56 crc kubenswrapper[5052]: I1203 11:40:56.362402 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/6b68de4d-a6d8-495d-becf-bd9f10da4a0b-run-httpd\") pod \"ceilometer-0\" (UID: \"6b68de4d-a6d8-495d-becf-bd9f10da4a0b\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:40:56 crc kubenswrapper[5052]: I1203 11:40:56.369435 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6b68de4d-a6d8-495d-becf-bd9f10da4a0b-config-data\") pod \"ceilometer-0\" (UID: \"6b68de4d-a6d8-495d-becf-bd9f10da4a0b\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:40:56 crc kubenswrapper[5052]: I1203 11:40:56.371974 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/6b68de4d-a6d8-495d-becf-bd9f10da4a0b-scripts\") pod \"ceilometer-0\" (UID: \"6b68de4d-a6d8-495d-becf-bd9f10da4a0b\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:40:56 crc kubenswrapper[5052]: I1203 11:40:56.373245 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/6b68de4d-a6d8-495d-becf-bd9f10da4a0b-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"6b68de4d-a6d8-495d-becf-bd9f10da4a0b\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:40:56 crc kubenswrapper[5052]: I1203 11:40:56.378401 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6b68de4d-a6d8-495d-becf-bd9f10da4a0b-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"6b68de4d-a6d8-495d-becf-bd9f10da4a0b\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:40:56 crc kubenswrapper[5052]: I1203 11:40:56.389334 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-sx6c6\" (UniqueName: \"kubernetes.io/projected/6b68de4d-a6d8-495d-becf-bd9f10da4a0b-kube-api-access-sx6c6\") pod \"ceilometer-0\" (UID: \"6b68de4d-a6d8-495d-becf-bd9f10da4a0b\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:40:56 crc kubenswrapper[5052]: I1203 11:40:56.573324 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:40:56 crc kubenswrapper[5052]: I1203 11:40:56.668555 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-0"] Dec 03 11:40:56 crc kubenswrapper[5052]: W1203 11:40:56.683032 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2501eda8_a78d_4609_a465_761e89bb291e.slice/crio-3f31cf67801442b58d6756229b602b05f44a185ad727f20aa451a39e5424be8b WatchSource:0}: Error finding container 3f31cf67801442b58d6756229b602b05f44a185ad727f20aa451a39e5424be8b: Status 404 returned error can't find the container with id 3f31cf67801442b58d6756229b602b05f44a185ad727f20aa451a39e5424be8b Dec 03 11:40:56 crc kubenswrapper[5052]: I1203 11:40:56.703211 5052 generic.go:334] "Generic (PLEG): container finished" podID="07b13d48-fa3d-4274-b138-b1c54e05811b" containerID="d2f41ae028ba676e4abc4a6c3cef0c2a13e596f42f7e81f790e0406f1bd2f8ba" exitCode=143 Dec 03 11:40:56 crc kubenswrapper[5052]: I1203 11:40:56.703477 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"07b13d48-fa3d-4274-b138-b1c54e05811b","Type":"ContainerDied","Data":"d2f41ae028ba676e4abc4a6c3cef0c2a13e596f42f7e81f790e0406f1bd2f8ba"} Dec 03 11:40:56 crc kubenswrapper[5052]: I1203 11:40:56.712886 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 11:40:57 crc kubenswrapper[5052]: I1203 11:40:57.135469 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:40:57 crc kubenswrapper[5052]: I1203 11:40:57.687105 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5deeda32-3ab5-4e5a-a41a-320996d2a162" path="/var/lib/kubelet/pods/5deeda32-3ab5-4e5a-a41a-320996d2a162/volumes" Dec 03 11:40:57 crc kubenswrapper[5052]: I1203 11:40:57.720661 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"6b68de4d-a6d8-495d-becf-bd9f10da4a0b","Type":"ContainerStarted","Data":"6aaf0623a38647268f722562653fe3da57df21b34502b4307b848205cef8463b"} Dec 03 11:40:57 crc kubenswrapper[5052]: I1203 11:40:57.722577 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-metadata-0" podUID="e0ad8648-db9b-46f7-b017-2cede96ad400" containerName="nova-metadata-log" containerID="cri-o://85e249bc476c7d0391b07cdd63b9907b54f7f7f1f5b5d1eefd2857df626c5545" gracePeriod=30 Dec 03 11:40:57 crc kubenswrapper[5052]: I1203 11:40:57.723808 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-conductor-0" event={"ID":"2501eda8-a78d-4609-a465-761e89bb291e","Type":"ContainerStarted","Data":"1714a77b6dd6ea9a3fe18b86a4ab32f16a2525fd31478012ffe19746b1260454"} Dec 03 11:40:57 crc kubenswrapper[5052]: I1203 11:40:57.723845 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 11:40:57 crc kubenswrapper[5052]: I1203 11:40:57.723858 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-conductor-0" event={"ID":"2501eda8-a78d-4609-a465-761e89bb291e","Type":"ContainerStarted","Data":"3f31cf67801442b58d6756229b602b05f44a185ad727f20aa451a39e5424be8b"} Dec 03 11:40:57 crc kubenswrapper[5052]: I1203 11:40:57.723965 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-scheduler-0" podUID="413c4d67-a09d-40ba-8ecb-e73a229b1ad6" containerName="nova-scheduler-scheduler" containerID="cri-o://eec4b2ec6abbb7588fec20cd2b6a8279879b8912e1316fee7e71bd784bb5e05e" gracePeriod=30 Dec 03 11:40:57 crc kubenswrapper[5052]: I1203 11:40:57.724062 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-metadata-0" podUID="e0ad8648-db9b-46f7-b017-2cede96ad400" containerName="nova-metadata-metadata" containerID="cri-o://4ad54ed3f890bb5eda1753ebe8bcf0aabc5fee6943b81588d00f5a1498514553" gracePeriod=30 Dec 03 11:40:57 crc kubenswrapper[5052]: I1203 11:40:57.756011 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-cell1-conductor-0" podStartSLOduration=2.7559866570000002 podStartE2EDuration="2.755986657s" podCreationTimestamp="2025-12-03 11:40:55 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:40:57.751751374 +0000 UTC m=+1248.339993138" watchObservedRunningTime="2025-12-03 11:40:57.755986657 +0000 UTC m=+1248.344228431" Dec 03 11:40:58 crc kubenswrapper[5052]: I1203 11:40:58.328374 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:40:58 crc kubenswrapper[5052]: I1203 11:40:58.413643 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/e0ad8648-db9b-46f7-b017-2cede96ad400-nova-metadata-tls-certs\") pod \"e0ad8648-db9b-46f7-b017-2cede96ad400\" (UID: \"e0ad8648-db9b-46f7-b017-2cede96ad400\") " Dec 03 11:40:58 crc kubenswrapper[5052]: I1203 11:40:58.413710 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/e0ad8648-db9b-46f7-b017-2cede96ad400-logs\") pod \"e0ad8648-db9b-46f7-b017-2cede96ad400\" (UID: \"e0ad8648-db9b-46f7-b017-2cede96ad400\") " Dec 03 11:40:58 crc kubenswrapper[5052]: I1203 11:40:58.413772 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e0ad8648-db9b-46f7-b017-2cede96ad400-combined-ca-bundle\") pod \"e0ad8648-db9b-46f7-b017-2cede96ad400\" (UID: \"e0ad8648-db9b-46f7-b017-2cede96ad400\") " Dec 03 11:40:58 crc kubenswrapper[5052]: I1203 11:40:58.413828 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/e0ad8648-db9b-46f7-b017-2cede96ad400-config-data\") pod \"e0ad8648-db9b-46f7-b017-2cede96ad400\" (UID: \"e0ad8648-db9b-46f7-b017-2cede96ad400\") " Dec 03 11:40:58 crc kubenswrapper[5052]: I1203 11:40:58.413963 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t2pmq\" (UniqueName: \"kubernetes.io/projected/e0ad8648-db9b-46f7-b017-2cede96ad400-kube-api-access-t2pmq\") pod \"e0ad8648-db9b-46f7-b017-2cede96ad400\" (UID: \"e0ad8648-db9b-46f7-b017-2cede96ad400\") " Dec 03 11:40:58 crc kubenswrapper[5052]: I1203 11:40:58.418024 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e0ad8648-db9b-46f7-b017-2cede96ad400-logs" (OuterVolumeSpecName: "logs") pod "e0ad8648-db9b-46f7-b017-2cede96ad400" (UID: "e0ad8648-db9b-46f7-b017-2cede96ad400"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:40:58 crc kubenswrapper[5052]: I1203 11:40:58.448577 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e0ad8648-db9b-46f7-b017-2cede96ad400-kube-api-access-t2pmq" (OuterVolumeSpecName: "kube-api-access-t2pmq") pod "e0ad8648-db9b-46f7-b017-2cede96ad400" (UID: "e0ad8648-db9b-46f7-b017-2cede96ad400"). InnerVolumeSpecName "kube-api-access-t2pmq". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:40:58 crc kubenswrapper[5052]: I1203 11:40:58.499592 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e0ad8648-db9b-46f7-b017-2cede96ad400-config-data" (OuterVolumeSpecName: "config-data") pod "e0ad8648-db9b-46f7-b017-2cede96ad400" (UID: "e0ad8648-db9b-46f7-b017-2cede96ad400"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:40:58 crc kubenswrapper[5052]: I1203 11:40:58.499767 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e0ad8648-db9b-46f7-b017-2cede96ad400-nova-metadata-tls-certs" (OuterVolumeSpecName: "nova-metadata-tls-certs") pod "e0ad8648-db9b-46f7-b017-2cede96ad400" (UID: "e0ad8648-db9b-46f7-b017-2cede96ad400"). InnerVolumeSpecName "nova-metadata-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:40:58 crc kubenswrapper[5052]: I1203 11:40:58.506765 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e0ad8648-db9b-46f7-b017-2cede96ad400-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "e0ad8648-db9b-46f7-b017-2cede96ad400" (UID: "e0ad8648-db9b-46f7-b017-2cede96ad400"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:40:58 crc kubenswrapper[5052]: I1203 11:40:58.515868 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/e0ad8648-db9b-46f7-b017-2cede96ad400-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:40:58 crc kubenswrapper[5052]: I1203 11:40:58.515896 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-t2pmq\" (UniqueName: \"kubernetes.io/projected/e0ad8648-db9b-46f7-b017-2cede96ad400-kube-api-access-t2pmq\") on node \"crc\" DevicePath \"\"" Dec 03 11:40:58 crc kubenswrapper[5052]: I1203 11:40:58.515907 5052 reconciler_common.go:293] "Volume detached for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/e0ad8648-db9b-46f7-b017-2cede96ad400-nova-metadata-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 11:40:58 crc kubenswrapper[5052]: I1203 11:40:58.515916 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/e0ad8648-db9b-46f7-b017-2cede96ad400-logs\") on node \"crc\" DevicePath \"\"" Dec 03 11:40:58 crc kubenswrapper[5052]: I1203 11:40:58.515925 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e0ad8648-db9b-46f7-b017-2cede96ad400-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:40:58 crc kubenswrapper[5052]: I1203 11:40:58.734411 5052 generic.go:334] "Generic (PLEG): container finished" podID="e0ad8648-db9b-46f7-b017-2cede96ad400" containerID="4ad54ed3f890bb5eda1753ebe8bcf0aabc5fee6943b81588d00f5a1498514553" exitCode=0 Dec 03 11:40:58 crc kubenswrapper[5052]: I1203 11:40:58.734716 5052 generic.go:334] "Generic (PLEG): container finished" podID="e0ad8648-db9b-46f7-b017-2cede96ad400" containerID="85e249bc476c7d0391b07cdd63b9907b54f7f7f1f5b5d1eefd2857df626c5545" exitCode=143 Dec 03 11:40:58 crc kubenswrapper[5052]: I1203 11:40:58.734484 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:40:58 crc kubenswrapper[5052]: I1203 11:40:58.734549 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"e0ad8648-db9b-46f7-b017-2cede96ad400","Type":"ContainerDied","Data":"4ad54ed3f890bb5eda1753ebe8bcf0aabc5fee6943b81588d00f5a1498514553"} Dec 03 11:40:58 crc kubenswrapper[5052]: I1203 11:40:58.734789 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"e0ad8648-db9b-46f7-b017-2cede96ad400","Type":"ContainerDied","Data":"85e249bc476c7d0391b07cdd63b9907b54f7f7f1f5b5d1eefd2857df626c5545"} Dec 03 11:40:58 crc kubenswrapper[5052]: I1203 11:40:58.734815 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"e0ad8648-db9b-46f7-b017-2cede96ad400","Type":"ContainerDied","Data":"5157dc8c64cbe91a10a95b5f7bf1c749b75c15342793eab527d03069157c5c96"} Dec 03 11:40:58 crc kubenswrapper[5052]: I1203 11:40:58.734834 5052 scope.go:117] "RemoveContainer" containerID="4ad54ed3f890bb5eda1753ebe8bcf0aabc5fee6943b81588d00f5a1498514553" Dec 03 11:40:58 crc kubenswrapper[5052]: I1203 11:40:58.737702 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"6b68de4d-a6d8-495d-becf-bd9f10da4a0b","Type":"ContainerStarted","Data":"ab919ccb3c04eb505bf1bce32f7913ab8034eced7677cfe0f55e36d797175613"} Dec 03 11:40:58 crc kubenswrapper[5052]: I1203 11:40:58.753617 5052 scope.go:117] "RemoveContainer" containerID="85e249bc476c7d0391b07cdd63b9907b54f7f7f1f5b5d1eefd2857df626c5545" Dec 03 11:40:58 crc kubenswrapper[5052]: I1203 11:40:58.774009 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 11:40:58 crc kubenswrapper[5052]: I1203 11:40:58.787864 5052 scope.go:117] "RemoveContainer" containerID="4ad54ed3f890bb5eda1753ebe8bcf0aabc5fee6943b81588d00f5a1498514553" Dec 03 11:40:58 crc kubenswrapper[5052]: E1203 11:40:58.789868 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4ad54ed3f890bb5eda1753ebe8bcf0aabc5fee6943b81588d00f5a1498514553\": container with ID starting with 4ad54ed3f890bb5eda1753ebe8bcf0aabc5fee6943b81588d00f5a1498514553 not found: ID does not exist" containerID="4ad54ed3f890bb5eda1753ebe8bcf0aabc5fee6943b81588d00f5a1498514553" Dec 03 11:40:58 crc kubenswrapper[5052]: I1203 11:40:58.790014 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4ad54ed3f890bb5eda1753ebe8bcf0aabc5fee6943b81588d00f5a1498514553"} err="failed to get container status \"4ad54ed3f890bb5eda1753ebe8bcf0aabc5fee6943b81588d00f5a1498514553\": rpc error: code = NotFound desc = could not find container \"4ad54ed3f890bb5eda1753ebe8bcf0aabc5fee6943b81588d00f5a1498514553\": container with ID starting with 4ad54ed3f890bb5eda1753ebe8bcf0aabc5fee6943b81588d00f5a1498514553 not found: ID does not exist" Dec 03 11:40:58 crc kubenswrapper[5052]: I1203 11:40:58.790134 5052 scope.go:117] "RemoveContainer" containerID="85e249bc476c7d0391b07cdd63b9907b54f7f7f1f5b5d1eefd2857df626c5545" Dec 03 11:40:58 crc kubenswrapper[5052]: E1203 11:40:58.792811 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"85e249bc476c7d0391b07cdd63b9907b54f7f7f1f5b5d1eefd2857df626c5545\": container with ID starting with 85e249bc476c7d0391b07cdd63b9907b54f7f7f1f5b5d1eefd2857df626c5545 not found: ID does not exist" containerID="85e249bc476c7d0391b07cdd63b9907b54f7f7f1f5b5d1eefd2857df626c5545" Dec 03 11:40:58 crc kubenswrapper[5052]: I1203 11:40:58.792948 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"85e249bc476c7d0391b07cdd63b9907b54f7f7f1f5b5d1eefd2857df626c5545"} err="failed to get container status \"85e249bc476c7d0391b07cdd63b9907b54f7f7f1f5b5d1eefd2857df626c5545\": rpc error: code = NotFound desc = could not find container \"85e249bc476c7d0391b07cdd63b9907b54f7f7f1f5b5d1eefd2857df626c5545\": container with ID starting with 85e249bc476c7d0391b07cdd63b9907b54f7f7f1f5b5d1eefd2857df626c5545 not found: ID does not exist" Dec 03 11:40:58 crc kubenswrapper[5052]: I1203 11:40:58.793059 5052 scope.go:117] "RemoveContainer" containerID="4ad54ed3f890bb5eda1753ebe8bcf0aabc5fee6943b81588d00f5a1498514553" Dec 03 11:40:58 crc kubenswrapper[5052]: I1203 11:40:58.797948 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4ad54ed3f890bb5eda1753ebe8bcf0aabc5fee6943b81588d00f5a1498514553"} err="failed to get container status \"4ad54ed3f890bb5eda1753ebe8bcf0aabc5fee6943b81588d00f5a1498514553\": rpc error: code = NotFound desc = could not find container \"4ad54ed3f890bb5eda1753ebe8bcf0aabc5fee6943b81588d00f5a1498514553\": container with ID starting with 4ad54ed3f890bb5eda1753ebe8bcf0aabc5fee6943b81588d00f5a1498514553 not found: ID does not exist" Dec 03 11:40:58 crc kubenswrapper[5052]: I1203 11:40:58.798630 5052 scope.go:117] "RemoveContainer" containerID="85e249bc476c7d0391b07cdd63b9907b54f7f7f1f5b5d1eefd2857df626c5545" Dec 03 11:40:58 crc kubenswrapper[5052]: I1203 11:40:58.799901 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"85e249bc476c7d0391b07cdd63b9907b54f7f7f1f5b5d1eefd2857df626c5545"} err="failed to get container status \"85e249bc476c7d0391b07cdd63b9907b54f7f7f1f5b5d1eefd2857df626c5545\": rpc error: code = NotFound desc = could not find container \"85e249bc476c7d0391b07cdd63b9907b54f7f7f1f5b5d1eefd2857df626c5545\": container with ID starting with 85e249bc476c7d0391b07cdd63b9907b54f7f7f1f5b5d1eefd2857df626c5545 not found: ID does not exist" Dec 03 11:40:58 crc kubenswrapper[5052]: I1203 11:40:58.805634 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 11:40:58 crc kubenswrapper[5052]: I1203 11:40:58.827182 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 11:40:58 crc kubenswrapper[5052]: E1203 11:40:58.828060 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="e0ad8648-db9b-46f7-b017-2cede96ad400" containerName="nova-metadata-log" Dec 03 11:40:58 crc kubenswrapper[5052]: I1203 11:40:58.828154 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="e0ad8648-db9b-46f7-b017-2cede96ad400" containerName="nova-metadata-log" Dec 03 11:40:58 crc kubenswrapper[5052]: E1203 11:40:58.828235 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="e0ad8648-db9b-46f7-b017-2cede96ad400" containerName="nova-metadata-metadata" Dec 03 11:40:58 crc kubenswrapper[5052]: I1203 11:40:58.828304 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="e0ad8648-db9b-46f7-b017-2cede96ad400" containerName="nova-metadata-metadata" Dec 03 11:40:58 crc kubenswrapper[5052]: I1203 11:40:58.828661 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="e0ad8648-db9b-46f7-b017-2cede96ad400" containerName="nova-metadata-log" Dec 03 11:40:58 crc kubenswrapper[5052]: I1203 11:40:58.828893 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="e0ad8648-db9b-46f7-b017-2cede96ad400" containerName="nova-metadata-metadata" Dec 03 11:40:58 crc kubenswrapper[5052]: I1203 11:40:58.830811 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:40:58 crc kubenswrapper[5052]: I1203 11:40:58.836253 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 11:40:58 crc kubenswrapper[5052]: I1203 11:40:58.836994 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-nova-metadata-internal-svc" Dec 03 11:40:58 crc kubenswrapper[5052]: I1203 11:40:58.837205 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-metadata-config-data" Dec 03 11:40:58 crc kubenswrapper[5052]: I1203 11:40:58.923309 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/77636371-bf0e-4fbd-b4da-d6c789f9421a-config-data\") pod \"nova-metadata-0\" (UID: \"77636371-bf0e-4fbd-b4da-d6c789f9421a\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:40:58 crc kubenswrapper[5052]: I1203 11:40:58.923361 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/77636371-bf0e-4fbd-b4da-d6c789f9421a-nova-metadata-tls-certs\") pod \"nova-metadata-0\" (UID: \"77636371-bf0e-4fbd-b4da-d6c789f9421a\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:40:58 crc kubenswrapper[5052]: I1203 11:40:58.923383 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-84ns4\" (UniqueName: \"kubernetes.io/projected/77636371-bf0e-4fbd-b4da-d6c789f9421a-kube-api-access-84ns4\") pod \"nova-metadata-0\" (UID: \"77636371-bf0e-4fbd-b4da-d6c789f9421a\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:40:58 crc kubenswrapper[5052]: I1203 11:40:58.923467 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/77636371-bf0e-4fbd-b4da-d6c789f9421a-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"77636371-bf0e-4fbd-b4da-d6c789f9421a\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:40:58 crc kubenswrapper[5052]: I1203 11:40:58.923493 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/77636371-bf0e-4fbd-b4da-d6c789f9421a-logs\") pod \"nova-metadata-0\" (UID: \"77636371-bf0e-4fbd-b4da-d6c789f9421a\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:40:59 crc kubenswrapper[5052]: I1203 11:40:59.024966 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/77636371-bf0e-4fbd-b4da-d6c789f9421a-config-data\") pod \"nova-metadata-0\" (UID: \"77636371-bf0e-4fbd-b4da-d6c789f9421a\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:40:59 crc kubenswrapper[5052]: I1203 11:40:59.025023 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/77636371-bf0e-4fbd-b4da-d6c789f9421a-nova-metadata-tls-certs\") pod \"nova-metadata-0\" (UID: \"77636371-bf0e-4fbd-b4da-d6c789f9421a\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:40:59 crc kubenswrapper[5052]: I1203 11:40:59.025042 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-84ns4\" (UniqueName: \"kubernetes.io/projected/77636371-bf0e-4fbd-b4da-d6c789f9421a-kube-api-access-84ns4\") pod \"nova-metadata-0\" (UID: \"77636371-bf0e-4fbd-b4da-d6c789f9421a\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:40:59 crc kubenswrapper[5052]: I1203 11:40:59.025124 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/77636371-bf0e-4fbd-b4da-d6c789f9421a-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"77636371-bf0e-4fbd-b4da-d6c789f9421a\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:40:59 crc kubenswrapper[5052]: I1203 11:40:59.025149 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/77636371-bf0e-4fbd-b4da-d6c789f9421a-logs\") pod \"nova-metadata-0\" (UID: \"77636371-bf0e-4fbd-b4da-d6c789f9421a\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:40:59 crc kubenswrapper[5052]: I1203 11:40:59.025577 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/77636371-bf0e-4fbd-b4da-d6c789f9421a-logs\") pod \"nova-metadata-0\" (UID: \"77636371-bf0e-4fbd-b4da-d6c789f9421a\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:40:59 crc kubenswrapper[5052]: I1203 11:40:59.029892 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/77636371-bf0e-4fbd-b4da-d6c789f9421a-config-data\") pod \"nova-metadata-0\" (UID: \"77636371-bf0e-4fbd-b4da-d6c789f9421a\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:40:59 crc kubenswrapper[5052]: I1203 11:40:59.044679 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/77636371-bf0e-4fbd-b4da-d6c789f9421a-nova-metadata-tls-certs\") pod \"nova-metadata-0\" (UID: \"77636371-bf0e-4fbd-b4da-d6c789f9421a\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:40:59 crc kubenswrapper[5052]: I1203 11:40:59.051359 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/77636371-bf0e-4fbd-b4da-d6c789f9421a-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"77636371-bf0e-4fbd-b4da-d6c789f9421a\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:40:59 crc kubenswrapper[5052]: I1203 11:40:59.051497 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-84ns4\" (UniqueName: \"kubernetes.io/projected/77636371-bf0e-4fbd-b4da-d6c789f9421a-kube-api-access-84ns4\") pod \"nova-metadata-0\" (UID: \"77636371-bf0e-4fbd-b4da-d6c789f9421a\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:40:59 crc kubenswrapper[5052]: I1203 11:40:59.150440 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:40:59 crc kubenswrapper[5052]: W1203 11:40:59.612296 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod77636371_bf0e_4fbd_b4da_d6c789f9421a.slice/crio-7284075c35ea1d02d4b745432658994f06cd82bedb5aafffb336c033b8bf09f5 WatchSource:0}: Error finding container 7284075c35ea1d02d4b745432658994f06cd82bedb5aafffb336c033b8bf09f5: Status 404 returned error can't find the container with id 7284075c35ea1d02d4b745432658994f06cd82bedb5aafffb336c033b8bf09f5 Dec 03 11:40:59 crc kubenswrapper[5052]: I1203 11:40:59.613688 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 11:40:59 crc kubenswrapper[5052]: I1203 11:40:59.687788 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e0ad8648-db9b-46f7-b017-2cede96ad400" path="/var/lib/kubelet/pods/e0ad8648-db9b-46f7-b017-2cede96ad400/volumes" Dec 03 11:40:59 crc kubenswrapper[5052]: I1203 11:40:59.751095 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"6b68de4d-a6d8-495d-becf-bd9f10da4a0b","Type":"ContainerStarted","Data":"e4055c48b95707806b72f0c0d8e98ad9447f6d9e2e1a3b1fe418cb3de4778816"} Dec 03 11:40:59 crc kubenswrapper[5052]: I1203 11:40:59.751145 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"6b68de4d-a6d8-495d-becf-bd9f10da4a0b","Type":"ContainerStarted","Data":"a4bea6910b59cfab257d2ff4d76a6cbd50648e0cfb6710c83724db04cf294850"} Dec 03 11:40:59 crc kubenswrapper[5052]: I1203 11:40:59.753551 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"77636371-bf0e-4fbd-b4da-d6c789f9421a","Type":"ContainerStarted","Data":"7284075c35ea1d02d4b745432658994f06cd82bedb5aafffb336c033b8bf09f5"} Dec 03 11:40:59 crc kubenswrapper[5052]: E1203 11:40:59.993638 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="eec4b2ec6abbb7588fec20cd2b6a8279879b8912e1316fee7e71bd784bb5e05e" cmd=["/usr/bin/pgrep","-r","DRST","nova-scheduler"] Dec 03 11:40:59 crc kubenswrapper[5052]: E1203 11:40:59.994809 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="eec4b2ec6abbb7588fec20cd2b6a8279879b8912e1316fee7e71bd784bb5e05e" cmd=["/usr/bin/pgrep","-r","DRST","nova-scheduler"] Dec 03 11:40:59 crc kubenswrapper[5052]: E1203 11:40:59.996068 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="eec4b2ec6abbb7588fec20cd2b6a8279879b8912e1316fee7e71bd784bb5e05e" cmd=["/usr/bin/pgrep","-r","DRST","nova-scheduler"] Dec 03 11:40:59 crc kubenswrapper[5052]: E1203 11:40:59.996139 5052 prober.go:104] "Probe errored" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" probeType="Readiness" pod="openstack-kuttl-tests/nova-scheduler-0" podUID="413c4d67-a09d-40ba-8ecb-e73a229b1ad6" containerName="nova-scheduler-scheduler" Dec 03 11:41:00 crc kubenswrapper[5052]: I1203 11:41:00.776929 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"77636371-bf0e-4fbd-b4da-d6c789f9421a","Type":"ContainerStarted","Data":"6f72511fa8fb2eabcc630482287a1ab0f5c038b6f10ade208a2c5f251276187a"} Dec 03 11:41:00 crc kubenswrapper[5052]: I1203 11:41:00.776980 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"77636371-bf0e-4fbd-b4da-d6c789f9421a","Type":"ContainerStarted","Data":"b1e10fd0abad06efc53bc41997abe362f796469d7e56b0403fc0f89c2913b6cd"} Dec 03 11:41:00 crc kubenswrapper[5052]: I1203 11:41:00.783526 5052 generic.go:334] "Generic (PLEG): container finished" podID="413c4d67-a09d-40ba-8ecb-e73a229b1ad6" containerID="eec4b2ec6abbb7588fec20cd2b6a8279879b8912e1316fee7e71bd784bb5e05e" exitCode=0 Dec 03 11:41:00 crc kubenswrapper[5052]: I1203 11:41:00.783580 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"413c4d67-a09d-40ba-8ecb-e73a229b1ad6","Type":"ContainerDied","Data":"eec4b2ec6abbb7588fec20cd2b6a8279879b8912e1316fee7e71bd784bb5e05e"} Dec 03 11:41:00 crc kubenswrapper[5052]: I1203 11:41:00.824297 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-metadata-0" podStartSLOduration=2.8242795320000003 podStartE2EDuration="2.824279532s" podCreationTimestamp="2025-12-03 11:40:58 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:41:00.803799506 +0000 UTC m=+1251.392041270" watchObservedRunningTime="2025-12-03 11:41:00.824279532 +0000 UTC m=+1251.412521286" Dec 03 11:41:01 crc kubenswrapper[5052]: I1203 11:41:01.025592 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:41:01 crc kubenswrapper[5052]: I1203 11:41:01.075655 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z5qsv\" (UniqueName: \"kubernetes.io/projected/413c4d67-a09d-40ba-8ecb-e73a229b1ad6-kube-api-access-z5qsv\") pod \"413c4d67-a09d-40ba-8ecb-e73a229b1ad6\" (UID: \"413c4d67-a09d-40ba-8ecb-e73a229b1ad6\") " Dec 03 11:41:01 crc kubenswrapper[5052]: I1203 11:41:01.075790 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/413c4d67-a09d-40ba-8ecb-e73a229b1ad6-combined-ca-bundle\") pod \"413c4d67-a09d-40ba-8ecb-e73a229b1ad6\" (UID: \"413c4d67-a09d-40ba-8ecb-e73a229b1ad6\") " Dec 03 11:41:01 crc kubenswrapper[5052]: I1203 11:41:01.075823 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/413c4d67-a09d-40ba-8ecb-e73a229b1ad6-config-data\") pod \"413c4d67-a09d-40ba-8ecb-e73a229b1ad6\" (UID: \"413c4d67-a09d-40ba-8ecb-e73a229b1ad6\") " Dec 03 11:41:01 crc kubenswrapper[5052]: I1203 11:41:01.094210 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/413c4d67-a09d-40ba-8ecb-e73a229b1ad6-kube-api-access-z5qsv" (OuterVolumeSpecName: "kube-api-access-z5qsv") pod "413c4d67-a09d-40ba-8ecb-e73a229b1ad6" (UID: "413c4d67-a09d-40ba-8ecb-e73a229b1ad6"). InnerVolumeSpecName "kube-api-access-z5qsv". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:41:01 crc kubenswrapper[5052]: I1203 11:41:01.114006 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/413c4d67-a09d-40ba-8ecb-e73a229b1ad6-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "413c4d67-a09d-40ba-8ecb-e73a229b1ad6" (UID: "413c4d67-a09d-40ba-8ecb-e73a229b1ad6"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:41:01 crc kubenswrapper[5052]: I1203 11:41:01.128950 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 11:41:01 crc kubenswrapper[5052]: I1203 11:41:01.133611 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/413c4d67-a09d-40ba-8ecb-e73a229b1ad6-config-data" (OuterVolumeSpecName: "config-data") pod "413c4d67-a09d-40ba-8ecb-e73a229b1ad6" (UID: "413c4d67-a09d-40ba-8ecb-e73a229b1ad6"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:41:01 crc kubenswrapper[5052]: I1203 11:41:01.178780 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-z5qsv\" (UniqueName: \"kubernetes.io/projected/413c4d67-a09d-40ba-8ecb-e73a229b1ad6-kube-api-access-z5qsv\") on node \"crc\" DevicePath \"\"" Dec 03 11:41:01 crc kubenswrapper[5052]: I1203 11:41:01.178823 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/413c4d67-a09d-40ba-8ecb-e73a229b1ad6-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:41:01 crc kubenswrapper[5052]: I1203 11:41:01.178837 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/413c4d67-a09d-40ba-8ecb-e73a229b1ad6-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:41:01 crc kubenswrapper[5052]: I1203 11:41:01.807924 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"413c4d67-a09d-40ba-8ecb-e73a229b1ad6","Type":"ContainerDied","Data":"6338483b2b756c96dcf60aaa9a7bbe747ac6817f81a380c7037cfa0f53f03bb1"} Dec 03 11:41:01 crc kubenswrapper[5052]: I1203 11:41:01.808082 5052 scope.go:117] "RemoveContainer" containerID="eec4b2ec6abbb7588fec20cd2b6a8279879b8912e1316fee7e71bd784bb5e05e" Dec 03 11:41:01 crc kubenswrapper[5052]: I1203 11:41:01.808049 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:41:01 crc kubenswrapper[5052]: I1203 11:41:01.818200 5052 generic.go:334] "Generic (PLEG): container finished" podID="07b13d48-fa3d-4274-b138-b1c54e05811b" containerID="781f723d704c08a0bb05affca300ba883bf5c2febee85b35ef6e758a1b497a8e" exitCode=0 Dec 03 11:41:01 crc kubenswrapper[5052]: I1203 11:41:01.818400 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"07b13d48-fa3d-4274-b138-b1c54e05811b","Type":"ContainerDied","Data":"781f723d704c08a0bb05affca300ba883bf5c2febee85b35ef6e758a1b497a8e"} Dec 03 11:41:01 crc kubenswrapper[5052]: I1203 11:41:01.823816 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"6b68de4d-a6d8-495d-becf-bd9f10da4a0b","Type":"ContainerStarted","Data":"ac07efb86dea0bf7769f94c38526b11b8aedace2c8ee55227222140c40e3d1b3"} Dec 03 11:41:01 crc kubenswrapper[5052]: I1203 11:41:01.823873 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:41:01 crc kubenswrapper[5052]: I1203 11:41:01.843633 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 11:41:01 crc kubenswrapper[5052]: I1203 11:41:01.851889 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 11:41:01 crc kubenswrapper[5052]: I1203 11:41:01.866599 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 11:41:01 crc kubenswrapper[5052]: E1203 11:41:01.871161 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="413c4d67-a09d-40ba-8ecb-e73a229b1ad6" containerName="nova-scheduler-scheduler" Dec 03 11:41:01 crc kubenswrapper[5052]: I1203 11:41:01.871191 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="413c4d67-a09d-40ba-8ecb-e73a229b1ad6" containerName="nova-scheduler-scheduler" Dec 03 11:41:01 crc kubenswrapper[5052]: I1203 11:41:01.871565 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="413c4d67-a09d-40ba-8ecb-e73a229b1ad6" containerName="nova-scheduler-scheduler" Dec 03 11:41:01 crc kubenswrapper[5052]: I1203 11:41:01.872168 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:41:01 crc kubenswrapper[5052]: I1203 11:41:01.875968 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/ceilometer-0" podStartSLOduration=2.508355576 podStartE2EDuration="5.875943138s" podCreationTimestamp="2025-12-03 11:40:56 +0000 UTC" firstStartedPulling="2025-12-03 11:40:57.142237341 +0000 UTC m=+1247.730479095" lastFinishedPulling="2025-12-03 11:41:00.509824893 +0000 UTC m=+1251.098066657" observedRunningTime="2025-12-03 11:41:01.854159244 +0000 UTC m=+1252.442401008" watchObservedRunningTime="2025-12-03 11:41:01.875943138 +0000 UTC m=+1252.464184912" Dec 03 11:41:01 crc kubenswrapper[5052]: I1203 11:41:01.881918 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-scheduler-config-data" Dec 03 11:41:01 crc kubenswrapper[5052]: I1203 11:41:01.899262 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qk4zq\" (UniqueName: \"kubernetes.io/projected/04dea2ba-82d8-4bab-a03c-74f94e34b760-kube-api-access-qk4zq\") pod \"nova-scheduler-0\" (UID: \"04dea2ba-82d8-4bab-a03c-74f94e34b760\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:41:01 crc kubenswrapper[5052]: I1203 11:41:01.899353 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/04dea2ba-82d8-4bab-a03c-74f94e34b760-combined-ca-bundle\") pod \"nova-scheduler-0\" (UID: \"04dea2ba-82d8-4bab-a03c-74f94e34b760\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:41:01 crc kubenswrapper[5052]: I1203 11:41:01.899417 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/04dea2ba-82d8-4bab-a03c-74f94e34b760-config-data\") pod \"nova-scheduler-0\" (UID: \"04dea2ba-82d8-4bab-a03c-74f94e34b760\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:41:01 crc kubenswrapper[5052]: I1203 11:41:01.923380 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 11:41:02 crc kubenswrapper[5052]: I1203 11:41:02.002378 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-qk4zq\" (UniqueName: \"kubernetes.io/projected/04dea2ba-82d8-4bab-a03c-74f94e34b760-kube-api-access-qk4zq\") pod \"nova-scheduler-0\" (UID: \"04dea2ba-82d8-4bab-a03c-74f94e34b760\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:41:02 crc kubenswrapper[5052]: I1203 11:41:02.002649 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/04dea2ba-82d8-4bab-a03c-74f94e34b760-combined-ca-bundle\") pod \"nova-scheduler-0\" (UID: \"04dea2ba-82d8-4bab-a03c-74f94e34b760\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:41:02 crc kubenswrapper[5052]: I1203 11:41:02.002729 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/04dea2ba-82d8-4bab-a03c-74f94e34b760-config-data\") pod \"nova-scheduler-0\" (UID: \"04dea2ba-82d8-4bab-a03c-74f94e34b760\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:41:02 crc kubenswrapper[5052]: I1203 11:41:02.028406 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/04dea2ba-82d8-4bab-a03c-74f94e34b760-combined-ca-bundle\") pod \"nova-scheduler-0\" (UID: \"04dea2ba-82d8-4bab-a03c-74f94e34b760\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:41:02 crc kubenswrapper[5052]: I1203 11:41:02.036470 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/04dea2ba-82d8-4bab-a03c-74f94e34b760-config-data\") pod \"nova-scheduler-0\" (UID: \"04dea2ba-82d8-4bab-a03c-74f94e34b760\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:41:02 crc kubenswrapper[5052]: I1203 11:41:02.100300 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-qk4zq\" (UniqueName: \"kubernetes.io/projected/04dea2ba-82d8-4bab-a03c-74f94e34b760-kube-api-access-qk4zq\") pod \"nova-scheduler-0\" (UID: \"04dea2ba-82d8-4bab-a03c-74f94e34b760\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:41:02 crc kubenswrapper[5052]: I1203 11:41:02.169203 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:41:02 crc kubenswrapper[5052]: I1203 11:41:02.212753 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/07b13d48-fa3d-4274-b138-b1c54e05811b-config-data\") pod \"07b13d48-fa3d-4274-b138-b1c54e05811b\" (UID: \"07b13d48-fa3d-4274-b138-b1c54e05811b\") " Dec 03 11:41:02 crc kubenswrapper[5052]: I1203 11:41:02.212950 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n6nbh\" (UniqueName: \"kubernetes.io/projected/07b13d48-fa3d-4274-b138-b1c54e05811b-kube-api-access-n6nbh\") pod \"07b13d48-fa3d-4274-b138-b1c54e05811b\" (UID: \"07b13d48-fa3d-4274-b138-b1c54e05811b\") " Dec 03 11:41:02 crc kubenswrapper[5052]: I1203 11:41:02.212982 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/07b13d48-fa3d-4274-b138-b1c54e05811b-logs\") pod \"07b13d48-fa3d-4274-b138-b1c54e05811b\" (UID: \"07b13d48-fa3d-4274-b138-b1c54e05811b\") " Dec 03 11:41:02 crc kubenswrapper[5052]: I1203 11:41:02.213016 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/07b13d48-fa3d-4274-b138-b1c54e05811b-combined-ca-bundle\") pod \"07b13d48-fa3d-4274-b138-b1c54e05811b\" (UID: \"07b13d48-fa3d-4274-b138-b1c54e05811b\") " Dec 03 11:41:02 crc kubenswrapper[5052]: I1203 11:41:02.213482 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07b13d48-fa3d-4274-b138-b1c54e05811b-logs" (OuterVolumeSpecName: "logs") pod "07b13d48-fa3d-4274-b138-b1c54e05811b" (UID: "07b13d48-fa3d-4274-b138-b1c54e05811b"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:41:02 crc kubenswrapper[5052]: I1203 11:41:02.213673 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:41:02 crc kubenswrapper[5052]: I1203 11:41:02.229746 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/07b13d48-fa3d-4274-b138-b1c54e05811b-kube-api-access-n6nbh" (OuterVolumeSpecName: "kube-api-access-n6nbh") pod "07b13d48-fa3d-4274-b138-b1c54e05811b" (UID: "07b13d48-fa3d-4274-b138-b1c54e05811b"). InnerVolumeSpecName "kube-api-access-n6nbh". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:41:02 crc kubenswrapper[5052]: I1203 11:41:02.255088 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/07b13d48-fa3d-4274-b138-b1c54e05811b-config-data" (OuterVolumeSpecName: "config-data") pod "07b13d48-fa3d-4274-b138-b1c54e05811b" (UID: "07b13d48-fa3d-4274-b138-b1c54e05811b"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:41:02 crc kubenswrapper[5052]: I1203 11:41:02.263985 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/07b13d48-fa3d-4274-b138-b1c54e05811b-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "07b13d48-fa3d-4274-b138-b1c54e05811b" (UID: "07b13d48-fa3d-4274-b138-b1c54e05811b"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:41:02 crc kubenswrapper[5052]: I1203 11:41:02.314959 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/07b13d48-fa3d-4274-b138-b1c54e05811b-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:41:02 crc kubenswrapper[5052]: I1203 11:41:02.315016 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-n6nbh\" (UniqueName: \"kubernetes.io/projected/07b13d48-fa3d-4274-b138-b1c54e05811b-kube-api-access-n6nbh\") on node \"crc\" DevicePath \"\"" Dec 03 11:41:02 crc kubenswrapper[5052]: I1203 11:41:02.315033 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/07b13d48-fa3d-4274-b138-b1c54e05811b-logs\") on node \"crc\" DevicePath \"\"" Dec 03 11:41:02 crc kubenswrapper[5052]: I1203 11:41:02.315045 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/07b13d48-fa3d-4274-b138-b1c54e05811b-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:41:02 crc kubenswrapper[5052]: I1203 11:41:02.709297 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 11:41:02 crc kubenswrapper[5052]: I1203 11:41:02.831652 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"04dea2ba-82d8-4bab-a03c-74f94e34b760","Type":"ContainerStarted","Data":"8dfd84b11033d329c7b8fc8ade6dd558fcfcfceb2759780298ecdb330cb8f0da"} Dec 03 11:41:02 crc kubenswrapper[5052]: I1203 11:41:02.836599 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"07b13d48-fa3d-4274-b138-b1c54e05811b","Type":"ContainerDied","Data":"c96f0f48d5837f6b0957ba4a27b2d8a7033266452b899f8e44fdc6b5bc1a7671"} Dec 03 11:41:02 crc kubenswrapper[5052]: I1203 11:41:02.836653 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:41:02 crc kubenswrapper[5052]: I1203 11:41:02.836668 5052 scope.go:117] "RemoveContainer" containerID="781f723d704c08a0bb05affca300ba883bf5c2febee85b35ef6e758a1b497a8e" Dec 03 11:41:02 crc kubenswrapper[5052]: I1203 11:41:02.881211 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 11:41:02 crc kubenswrapper[5052]: I1203 11:41:02.884661 5052 scope.go:117] "RemoveContainer" containerID="d2f41ae028ba676e4abc4a6c3cef0c2a13e596f42f7e81f790e0406f1bd2f8ba" Dec 03 11:41:02 crc kubenswrapper[5052]: I1203 11:41:02.896355 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 11:41:02 crc kubenswrapper[5052]: I1203 11:41:02.907769 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 11:41:02 crc kubenswrapper[5052]: E1203 11:41:02.908132 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="07b13d48-fa3d-4274-b138-b1c54e05811b" containerName="nova-api-api" Dec 03 11:41:02 crc kubenswrapper[5052]: I1203 11:41:02.908150 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="07b13d48-fa3d-4274-b138-b1c54e05811b" containerName="nova-api-api" Dec 03 11:41:02 crc kubenswrapper[5052]: E1203 11:41:02.908166 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="07b13d48-fa3d-4274-b138-b1c54e05811b" containerName="nova-api-log" Dec 03 11:41:02 crc kubenswrapper[5052]: I1203 11:41:02.908174 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="07b13d48-fa3d-4274-b138-b1c54e05811b" containerName="nova-api-log" Dec 03 11:41:02 crc kubenswrapper[5052]: I1203 11:41:02.908339 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="07b13d48-fa3d-4274-b138-b1c54e05811b" containerName="nova-api-log" Dec 03 11:41:02 crc kubenswrapper[5052]: I1203 11:41:02.908361 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="07b13d48-fa3d-4274-b138-b1c54e05811b" containerName="nova-api-api" Dec 03 11:41:02 crc kubenswrapper[5052]: I1203 11:41:02.909266 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:41:02 crc kubenswrapper[5052]: I1203 11:41:02.911451 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-api-config-data" Dec 03 11:41:02 crc kubenswrapper[5052]: I1203 11:41:02.918804 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 11:41:02 crc kubenswrapper[5052]: I1203 11:41:02.927133 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c1ca7808-d027-424d-9850-e0bcd12ada2a-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"c1ca7808-d027-424d-9850-e0bcd12ada2a\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:41:02 crc kubenswrapper[5052]: I1203 11:41:02.927228 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f46jc\" (UniqueName: \"kubernetes.io/projected/c1ca7808-d027-424d-9850-e0bcd12ada2a-kube-api-access-f46jc\") pod \"nova-api-0\" (UID: \"c1ca7808-d027-424d-9850-e0bcd12ada2a\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:41:02 crc kubenswrapper[5052]: I1203 11:41:02.927307 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/c1ca7808-d027-424d-9850-e0bcd12ada2a-logs\") pod \"nova-api-0\" (UID: \"c1ca7808-d027-424d-9850-e0bcd12ada2a\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:41:02 crc kubenswrapper[5052]: I1203 11:41:02.927333 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/c1ca7808-d027-424d-9850-e0bcd12ada2a-config-data\") pod \"nova-api-0\" (UID: \"c1ca7808-d027-424d-9850-e0bcd12ada2a\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:41:03 crc kubenswrapper[5052]: I1203 11:41:03.028459 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c1ca7808-d027-424d-9850-e0bcd12ada2a-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"c1ca7808-d027-424d-9850-e0bcd12ada2a\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:41:03 crc kubenswrapper[5052]: I1203 11:41:03.028587 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-f46jc\" (UniqueName: \"kubernetes.io/projected/c1ca7808-d027-424d-9850-e0bcd12ada2a-kube-api-access-f46jc\") pod \"nova-api-0\" (UID: \"c1ca7808-d027-424d-9850-e0bcd12ada2a\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:41:03 crc kubenswrapper[5052]: I1203 11:41:03.028653 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/c1ca7808-d027-424d-9850-e0bcd12ada2a-logs\") pod \"nova-api-0\" (UID: \"c1ca7808-d027-424d-9850-e0bcd12ada2a\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:41:03 crc kubenswrapper[5052]: I1203 11:41:03.028683 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/c1ca7808-d027-424d-9850-e0bcd12ada2a-config-data\") pod \"nova-api-0\" (UID: \"c1ca7808-d027-424d-9850-e0bcd12ada2a\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:41:03 crc kubenswrapper[5052]: I1203 11:41:03.029607 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/c1ca7808-d027-424d-9850-e0bcd12ada2a-logs\") pod \"nova-api-0\" (UID: \"c1ca7808-d027-424d-9850-e0bcd12ada2a\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:41:03 crc kubenswrapper[5052]: I1203 11:41:03.032816 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c1ca7808-d027-424d-9850-e0bcd12ada2a-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"c1ca7808-d027-424d-9850-e0bcd12ada2a\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:41:03 crc kubenswrapper[5052]: I1203 11:41:03.033624 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/c1ca7808-d027-424d-9850-e0bcd12ada2a-config-data\") pod \"nova-api-0\" (UID: \"c1ca7808-d027-424d-9850-e0bcd12ada2a\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:41:03 crc kubenswrapper[5052]: I1203 11:41:03.051910 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-f46jc\" (UniqueName: \"kubernetes.io/projected/c1ca7808-d027-424d-9850-e0bcd12ada2a-kube-api-access-f46jc\") pod \"nova-api-0\" (UID: \"c1ca7808-d027-424d-9850-e0bcd12ada2a\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:41:03 crc kubenswrapper[5052]: I1203 11:41:03.245092 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:41:03 crc kubenswrapper[5052]: I1203 11:41:03.683942 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="07b13d48-fa3d-4274-b138-b1c54e05811b" path="/var/lib/kubelet/pods/07b13d48-fa3d-4274-b138-b1c54e05811b/volumes" Dec 03 11:41:03 crc kubenswrapper[5052]: I1203 11:41:03.684848 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="413c4d67-a09d-40ba-8ecb-e73a229b1ad6" path="/var/lib/kubelet/pods/413c4d67-a09d-40ba-8ecb-e73a229b1ad6/volumes" Dec 03 11:41:03 crc kubenswrapper[5052]: I1203 11:41:03.719284 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 11:41:03 crc kubenswrapper[5052]: W1203 11:41:03.728817 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc1ca7808_d027_424d_9850_e0bcd12ada2a.slice/crio-685050ed96ac5a7d6d628198e239092ab69230718034ac6af0ca5135529ebbf6 WatchSource:0}: Error finding container 685050ed96ac5a7d6d628198e239092ab69230718034ac6af0ca5135529ebbf6: Status 404 returned error can't find the container with id 685050ed96ac5a7d6d628198e239092ab69230718034ac6af0ca5135529ebbf6 Dec 03 11:41:03 crc kubenswrapper[5052]: I1203 11:41:03.846571 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"04dea2ba-82d8-4bab-a03c-74f94e34b760","Type":"ContainerStarted","Data":"5464a7a42a3745c79d8459b75c25e64208243026ee81b165ab30a3bdba3b2f6f"} Dec 03 11:41:03 crc kubenswrapper[5052]: I1203 11:41:03.850043 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"c1ca7808-d027-424d-9850-e0bcd12ada2a","Type":"ContainerStarted","Data":"685050ed96ac5a7d6d628198e239092ab69230718034ac6af0ca5135529ebbf6"} Dec 03 11:41:03 crc kubenswrapper[5052]: I1203 11:41:03.870044 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-scheduler-0" podStartSLOduration=2.87002149 podStartE2EDuration="2.87002149s" podCreationTimestamp="2025-12-03 11:41:01 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:41:03.86106485 +0000 UTC m=+1254.449306604" watchObservedRunningTime="2025-12-03 11:41:03.87002149 +0000 UTC m=+1254.458263265" Dec 03 11:41:04 crc kubenswrapper[5052]: I1203 11:41:04.151916 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:41:04 crc kubenswrapper[5052]: I1203 11:41:04.152558 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:41:04 crc kubenswrapper[5052]: I1203 11:41:04.863030 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"c1ca7808-d027-424d-9850-e0bcd12ada2a","Type":"ContainerStarted","Data":"be7d93b2064ab92bb9933e97b2f2019f219a60958f433d063793dd8831b62f4f"} Dec 03 11:41:04 crc kubenswrapper[5052]: I1203 11:41:04.863087 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"c1ca7808-d027-424d-9850-e0bcd12ada2a","Type":"ContainerStarted","Data":"1c657dcd8b7ce371f9cd12b8cc944e9c2595bb1bb2d4e31785b5a4846ab0a22e"} Dec 03 11:41:04 crc kubenswrapper[5052]: I1203 11:41:04.883567 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-api-0" podStartSLOduration=2.883547857 podStartE2EDuration="2.883547857s" podCreationTimestamp="2025-12-03 11:41:02 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:41:04.881814437 +0000 UTC m=+1255.470056191" watchObservedRunningTime="2025-12-03 11:41:04.883547857 +0000 UTC m=+1255.471789641" Dec 03 11:41:07 crc kubenswrapper[5052]: I1203 11:41:07.214481 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:41:09 crc kubenswrapper[5052]: I1203 11:41:09.151310 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:41:09 crc kubenswrapper[5052]: I1203 11:41:09.151836 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:41:10 crc kubenswrapper[5052]: I1203 11:41:10.167679 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-metadata-0" podUID="77636371-bf0e-4fbd-b4da-d6c789f9421a" containerName="nova-metadata-log" probeResult="failure" output="Get \"https://10.217.0.174:8775/\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Dec 03 11:41:10 crc kubenswrapper[5052]: I1203 11:41:10.172641 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-metadata-0" podUID="77636371-bf0e-4fbd-b4da-d6c789f9421a" containerName="nova-metadata-metadata" probeResult="failure" output="Get \"https://10.217.0.174:8775/\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Dec 03 11:41:12 crc kubenswrapper[5052]: I1203 11:41:12.214080 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:41:12 crc kubenswrapper[5052]: I1203 11:41:12.245497 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:41:12 crc kubenswrapper[5052]: I1203 11:41:12.967953 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:41:13 crc kubenswrapper[5052]: I1203 11:41:13.245818 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:41:13 crc kubenswrapper[5052]: I1203 11:41:13.245889 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:41:14 crc kubenswrapper[5052]: I1203 11:41:14.328858 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-api-0" podUID="c1ca7808-d027-424d-9850-e0bcd12ada2a" containerName="nova-api-api" probeResult="failure" output="Get \"http://10.217.0.176:8774/\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Dec 03 11:41:14 crc kubenswrapper[5052]: I1203 11:41:14.330790 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-api-0" podUID="c1ca7808-d027-424d-9850-e0bcd12ada2a" containerName="nova-api-log" probeResult="failure" output="Get \"http://10.217.0.176:8774/\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Dec 03 11:41:19 crc kubenswrapper[5052]: I1203 11:41:19.157323 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:41:19 crc kubenswrapper[5052]: I1203 11:41:19.160544 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:41:19 crc kubenswrapper[5052]: I1203 11:41:19.163172 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:41:19 crc kubenswrapper[5052]: I1203 11:41:19.991325 5052 generic.go:334] "Generic (PLEG): container finished" podID="0b744776-104c-466a-be29-79efdd996a28" containerID="08aececa6f47b8deb531a78d1231ed436b94566bc55dd432fa69a3f7a05d3ebf" exitCode=137 Dec 03 11:41:19 crc kubenswrapper[5052]: I1203 11:41:19.991374 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" event={"ID":"0b744776-104c-466a-be29-79efdd996a28","Type":"ContainerDied","Data":"08aececa6f47b8deb531a78d1231ed436b94566bc55dd432fa69a3f7a05d3ebf"} Dec 03 11:41:19 crc kubenswrapper[5052]: I1203 11:41:19.991903 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" event={"ID":"0b744776-104c-466a-be29-79efdd996a28","Type":"ContainerDied","Data":"1b465e582a74ac1f559f37138e65b1f1cf0e3836cc1e1e6a5046bd8b27f87f74"} Dec 03 11:41:19 crc kubenswrapper[5052]: I1203 11:41:19.991918 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1b465e582a74ac1f559f37138e65b1f1cf0e3836cc1e1e6a5046bd8b27f87f74" Dec 03 11:41:19 crc kubenswrapper[5052]: I1203 11:41:19.995956 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 11:41:20 crc kubenswrapper[5052]: I1203 11:41:20.006113 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:41:20 crc kubenswrapper[5052]: I1203 11:41:20.060373 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9qccm\" (UniqueName: \"kubernetes.io/projected/0b744776-104c-466a-be29-79efdd996a28-kube-api-access-9qccm\") pod \"0b744776-104c-466a-be29-79efdd996a28\" (UID: \"0b744776-104c-466a-be29-79efdd996a28\") " Dec 03 11:41:20 crc kubenswrapper[5052]: I1203 11:41:20.060457 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/0b744776-104c-466a-be29-79efdd996a28-config-data\") pod \"0b744776-104c-466a-be29-79efdd996a28\" (UID: \"0b744776-104c-466a-be29-79efdd996a28\") " Dec 03 11:41:20 crc kubenswrapper[5052]: I1203 11:41:20.060604 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0b744776-104c-466a-be29-79efdd996a28-combined-ca-bundle\") pod \"0b744776-104c-466a-be29-79efdd996a28\" (UID: \"0b744776-104c-466a-be29-79efdd996a28\") " Dec 03 11:41:20 crc kubenswrapper[5052]: I1203 11:41:20.066273 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0b744776-104c-466a-be29-79efdd996a28-kube-api-access-9qccm" (OuterVolumeSpecName: "kube-api-access-9qccm") pod "0b744776-104c-466a-be29-79efdd996a28" (UID: "0b744776-104c-466a-be29-79efdd996a28"). InnerVolumeSpecName "kube-api-access-9qccm". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:41:20 crc kubenswrapper[5052]: I1203 11:41:20.090548 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0b744776-104c-466a-be29-79efdd996a28-config-data" (OuterVolumeSpecName: "config-data") pod "0b744776-104c-466a-be29-79efdd996a28" (UID: "0b744776-104c-466a-be29-79efdd996a28"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:41:20 crc kubenswrapper[5052]: I1203 11:41:20.124701 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0b744776-104c-466a-be29-79efdd996a28-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "0b744776-104c-466a-be29-79efdd996a28" (UID: "0b744776-104c-466a-be29-79efdd996a28"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:41:20 crc kubenswrapper[5052]: I1203 11:41:20.169806 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-9qccm\" (UniqueName: \"kubernetes.io/projected/0b744776-104c-466a-be29-79efdd996a28-kube-api-access-9qccm\") on node \"crc\" DevicePath \"\"" Dec 03 11:41:20 crc kubenswrapper[5052]: I1203 11:41:20.171050 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/0b744776-104c-466a-be29-79efdd996a28-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:41:20 crc kubenswrapper[5052]: I1203 11:41:20.171156 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0b744776-104c-466a-be29-79efdd996a28-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:41:20 crc kubenswrapper[5052]: I1203 11:41:20.999159 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 11:41:21 crc kubenswrapper[5052]: I1203 11:41:21.028259 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 11:41:21 crc kubenswrapper[5052]: I1203 11:41:21.037973 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 11:41:21 crc kubenswrapper[5052]: I1203 11:41:21.059376 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 11:41:21 crc kubenswrapper[5052]: E1203 11:41:21.059800 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="0b744776-104c-466a-be29-79efdd996a28" containerName="nova-cell1-novncproxy-novncproxy" Dec 03 11:41:21 crc kubenswrapper[5052]: I1203 11:41:21.059815 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="0b744776-104c-466a-be29-79efdd996a28" containerName="nova-cell1-novncproxy-novncproxy" Dec 03 11:41:21 crc kubenswrapper[5052]: I1203 11:41:21.059992 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="0b744776-104c-466a-be29-79efdd996a28" containerName="nova-cell1-novncproxy-novncproxy" Dec 03 11:41:21 crc kubenswrapper[5052]: I1203 11:41:21.060604 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 11:41:21 crc kubenswrapper[5052]: I1203 11:41:21.062383 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell1-novncproxy-config-data" Dec 03 11:41:21 crc kubenswrapper[5052]: I1203 11:41:21.063106 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-nova-novncproxy-cell1-public-svc" Dec 03 11:41:21 crc kubenswrapper[5052]: I1203 11:41:21.063173 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-nova-novncproxy-cell1-vencrypt" Dec 03 11:41:21 crc kubenswrapper[5052]: I1203 11:41:21.066861 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 11:41:21 crc kubenswrapper[5052]: I1203 11:41:21.189861 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nova-novncproxy-tls-certs\" (UniqueName: \"kubernetes.io/secret/049153cf-b5da-4a61-8a2d-3f694625642b-nova-novncproxy-tls-certs\") pod \"nova-cell1-novncproxy-0\" (UID: \"049153cf-b5da-4a61-8a2d-3f694625642b\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 11:41:21 crc kubenswrapper[5052]: I1203 11:41:21.189915 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/049153cf-b5da-4a61-8a2d-3f694625642b-config-data\") pod \"nova-cell1-novncproxy-0\" (UID: \"049153cf-b5da-4a61-8a2d-3f694625642b\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 11:41:21 crc kubenswrapper[5052]: I1203 11:41:21.189975 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/049153cf-b5da-4a61-8a2d-3f694625642b-combined-ca-bundle\") pod \"nova-cell1-novncproxy-0\" (UID: \"049153cf-b5da-4a61-8a2d-3f694625642b\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 11:41:21 crc kubenswrapper[5052]: I1203 11:41:21.190030 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"vencrypt-tls-certs\" (UniqueName: \"kubernetes.io/secret/049153cf-b5da-4a61-8a2d-3f694625642b-vencrypt-tls-certs\") pod \"nova-cell1-novncproxy-0\" (UID: \"049153cf-b5da-4a61-8a2d-3f694625642b\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 11:41:21 crc kubenswrapper[5052]: I1203 11:41:21.190066 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ctjfw\" (UniqueName: \"kubernetes.io/projected/049153cf-b5da-4a61-8a2d-3f694625642b-kube-api-access-ctjfw\") pod \"nova-cell1-novncproxy-0\" (UID: \"049153cf-b5da-4a61-8a2d-3f694625642b\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 11:41:21 crc kubenswrapper[5052]: I1203 11:41:21.291774 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"nova-novncproxy-tls-certs\" (UniqueName: \"kubernetes.io/secret/049153cf-b5da-4a61-8a2d-3f694625642b-nova-novncproxy-tls-certs\") pod \"nova-cell1-novncproxy-0\" (UID: \"049153cf-b5da-4a61-8a2d-3f694625642b\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 11:41:21 crc kubenswrapper[5052]: I1203 11:41:21.292132 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/049153cf-b5da-4a61-8a2d-3f694625642b-config-data\") pod \"nova-cell1-novncproxy-0\" (UID: \"049153cf-b5da-4a61-8a2d-3f694625642b\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 11:41:21 crc kubenswrapper[5052]: I1203 11:41:21.292265 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/049153cf-b5da-4a61-8a2d-3f694625642b-combined-ca-bundle\") pod \"nova-cell1-novncproxy-0\" (UID: \"049153cf-b5da-4a61-8a2d-3f694625642b\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 11:41:21 crc kubenswrapper[5052]: I1203 11:41:21.292393 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"vencrypt-tls-certs\" (UniqueName: \"kubernetes.io/secret/049153cf-b5da-4a61-8a2d-3f694625642b-vencrypt-tls-certs\") pod \"nova-cell1-novncproxy-0\" (UID: \"049153cf-b5da-4a61-8a2d-3f694625642b\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 11:41:21 crc kubenswrapper[5052]: I1203 11:41:21.292522 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-ctjfw\" (UniqueName: \"kubernetes.io/projected/049153cf-b5da-4a61-8a2d-3f694625642b-kube-api-access-ctjfw\") pod \"nova-cell1-novncproxy-0\" (UID: \"049153cf-b5da-4a61-8a2d-3f694625642b\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 11:41:21 crc kubenswrapper[5052]: I1203 11:41:21.296999 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"nova-novncproxy-tls-certs\" (UniqueName: \"kubernetes.io/secret/049153cf-b5da-4a61-8a2d-3f694625642b-nova-novncproxy-tls-certs\") pod \"nova-cell1-novncproxy-0\" (UID: \"049153cf-b5da-4a61-8a2d-3f694625642b\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 11:41:21 crc kubenswrapper[5052]: I1203 11:41:21.297133 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/049153cf-b5da-4a61-8a2d-3f694625642b-combined-ca-bundle\") pod \"nova-cell1-novncproxy-0\" (UID: \"049153cf-b5da-4a61-8a2d-3f694625642b\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 11:41:21 crc kubenswrapper[5052]: I1203 11:41:21.297401 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/049153cf-b5da-4a61-8a2d-3f694625642b-config-data\") pod \"nova-cell1-novncproxy-0\" (UID: \"049153cf-b5da-4a61-8a2d-3f694625642b\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 11:41:21 crc kubenswrapper[5052]: I1203 11:41:21.297790 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"vencrypt-tls-certs\" (UniqueName: \"kubernetes.io/secret/049153cf-b5da-4a61-8a2d-3f694625642b-vencrypt-tls-certs\") pod \"nova-cell1-novncproxy-0\" (UID: \"049153cf-b5da-4a61-8a2d-3f694625642b\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 11:41:21 crc kubenswrapper[5052]: I1203 11:41:21.602355 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-ctjfw\" (UniqueName: \"kubernetes.io/projected/049153cf-b5da-4a61-8a2d-3f694625642b-kube-api-access-ctjfw\") pod \"nova-cell1-novncproxy-0\" (UID: \"049153cf-b5da-4a61-8a2d-3f694625642b\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 11:41:21 crc kubenswrapper[5052]: I1203 11:41:21.683053 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0b744776-104c-466a-be29-79efdd996a28" path="/var/lib/kubelet/pods/0b744776-104c-466a-be29-79efdd996a28/volumes" Dec 03 11:41:21 crc kubenswrapper[5052]: I1203 11:41:21.684065 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 11:41:22 crc kubenswrapper[5052]: I1203 11:41:22.149709 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 11:41:23 crc kubenswrapper[5052]: I1203 11:41:23.023760 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" event={"ID":"049153cf-b5da-4a61-8a2d-3f694625642b","Type":"ContainerStarted","Data":"10f606fe8fa7b691b8917f929fb59e6261d389781afc79a34d36abb996d45333"} Dec 03 11:41:23 crc kubenswrapper[5052]: I1203 11:41:23.023838 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" event={"ID":"049153cf-b5da-4a61-8a2d-3f694625642b","Type":"ContainerStarted","Data":"ded9cddd3ba921f6b429e0b802f59fa4795b1425e1728696575c769f420fc6f1"} Dec 03 11:41:23 crc kubenswrapper[5052]: I1203 11:41:23.061200 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" podStartSLOduration=2.061160763 podStartE2EDuration="2.061160763s" podCreationTimestamp="2025-12-03 11:41:21 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:41:23.044074056 +0000 UTC m=+1273.632315850" watchObservedRunningTime="2025-12-03 11:41:23.061160763 +0000 UTC m=+1273.649402527" Dec 03 11:41:23 crc kubenswrapper[5052]: I1203 11:41:23.250373 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:41:23 crc kubenswrapper[5052]: I1203 11:41:23.250760 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:41:23 crc kubenswrapper[5052]: I1203 11:41:23.258239 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:41:23 crc kubenswrapper[5052]: I1203 11:41:23.262615 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:41:24 crc kubenswrapper[5052]: I1203 11:41:24.036542 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:41:24 crc kubenswrapper[5052]: I1203 11:41:24.051072 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:41:26 crc kubenswrapper[5052]: I1203 11:41:26.304198 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:41:26 crc kubenswrapper[5052]: I1203 11:41:26.304897 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="6b68de4d-a6d8-495d-becf-bd9f10da4a0b" containerName="ceilometer-central-agent" containerID="cri-o://ab919ccb3c04eb505bf1bce32f7913ab8034eced7677cfe0f55e36d797175613" gracePeriod=30 Dec 03 11:41:26 crc kubenswrapper[5052]: I1203 11:41:26.305057 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="6b68de4d-a6d8-495d-becf-bd9f10da4a0b" containerName="proxy-httpd" containerID="cri-o://ac07efb86dea0bf7769f94c38526b11b8aedace2c8ee55227222140c40e3d1b3" gracePeriod=30 Dec 03 11:41:26 crc kubenswrapper[5052]: I1203 11:41:26.305115 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="6b68de4d-a6d8-495d-becf-bd9f10da4a0b" containerName="sg-core" containerID="cri-o://e4055c48b95707806b72f0c0d8e98ad9447f6d9e2e1a3b1fe418cb3de4778816" gracePeriod=30 Dec 03 11:41:26 crc kubenswrapper[5052]: I1203 11:41:26.305158 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="6b68de4d-a6d8-495d-becf-bd9f10da4a0b" containerName="ceilometer-notification-agent" containerID="cri-o://a4bea6910b59cfab257d2ff4d76a6cbd50648e0cfb6710c83724db04cf294850" gracePeriod=30 Dec 03 11:41:26 crc kubenswrapper[5052]: I1203 11:41:26.311600 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:41:26 crc kubenswrapper[5052]: I1203 11:41:26.575607 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/ceilometer-0" podUID="6b68de4d-a6d8-495d-becf-bd9f10da4a0b" containerName="proxy-httpd" probeResult="failure" output="Get \"http://10.217.0.173:3000/\": dial tcp 10.217.0.173:3000: connect: connection refused" Dec 03 11:41:26 crc kubenswrapper[5052]: I1203 11:41:26.684250 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 11:41:26 crc kubenswrapper[5052]: I1203 11:41:26.948092 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 11:41:27 crc kubenswrapper[5052]: I1203 11:41:27.067879 5052 generic.go:334] "Generic (PLEG): container finished" podID="6b68de4d-a6d8-495d-becf-bd9f10da4a0b" containerID="ac07efb86dea0bf7769f94c38526b11b8aedace2c8ee55227222140c40e3d1b3" exitCode=0 Dec 03 11:41:27 crc kubenswrapper[5052]: I1203 11:41:27.068211 5052 generic.go:334] "Generic (PLEG): container finished" podID="6b68de4d-a6d8-495d-becf-bd9f10da4a0b" containerID="e4055c48b95707806b72f0c0d8e98ad9447f6d9e2e1a3b1fe418cb3de4778816" exitCode=2 Dec 03 11:41:27 crc kubenswrapper[5052]: I1203 11:41:27.068220 5052 generic.go:334] "Generic (PLEG): container finished" podID="6b68de4d-a6d8-495d-becf-bd9f10da4a0b" containerID="ab919ccb3c04eb505bf1bce32f7913ab8034eced7677cfe0f55e36d797175613" exitCode=0 Dec 03 11:41:27 crc kubenswrapper[5052]: I1203 11:41:27.068372 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-api-0" podUID="c1ca7808-d027-424d-9850-e0bcd12ada2a" containerName="nova-api-log" containerID="cri-o://1c657dcd8b7ce371f9cd12b8cc944e9c2595bb1bb2d4e31785b5a4846ab0a22e" gracePeriod=30 Dec 03 11:41:27 crc kubenswrapper[5052]: I1203 11:41:27.068645 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"6b68de4d-a6d8-495d-becf-bd9f10da4a0b","Type":"ContainerDied","Data":"ac07efb86dea0bf7769f94c38526b11b8aedace2c8ee55227222140c40e3d1b3"} Dec 03 11:41:27 crc kubenswrapper[5052]: I1203 11:41:27.068670 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"6b68de4d-a6d8-495d-becf-bd9f10da4a0b","Type":"ContainerDied","Data":"e4055c48b95707806b72f0c0d8e98ad9447f6d9e2e1a3b1fe418cb3de4778816"} Dec 03 11:41:27 crc kubenswrapper[5052]: I1203 11:41:27.068681 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"6b68de4d-a6d8-495d-becf-bd9f10da4a0b","Type":"ContainerDied","Data":"ab919ccb3c04eb505bf1bce32f7913ab8034eced7677cfe0f55e36d797175613"} Dec 03 11:41:27 crc kubenswrapper[5052]: I1203 11:41:27.068984 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-api-0" podUID="c1ca7808-d027-424d-9850-e0bcd12ada2a" containerName="nova-api-api" containerID="cri-o://be7d93b2064ab92bb9933e97b2f2019f219a60958f433d063793dd8831b62f4f" gracePeriod=30 Dec 03 11:41:28 crc kubenswrapper[5052]: I1203 11:41:28.078826 5052 generic.go:334] "Generic (PLEG): container finished" podID="c1ca7808-d027-424d-9850-e0bcd12ada2a" containerID="1c657dcd8b7ce371f9cd12b8cc944e9c2595bb1bb2d4e31785b5a4846ab0a22e" exitCode=143 Dec 03 11:41:28 crc kubenswrapper[5052]: I1203 11:41:28.078881 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"c1ca7808-d027-424d-9850-e0bcd12ada2a","Type":"ContainerDied","Data":"1c657dcd8b7ce371f9cd12b8cc944e9c2595bb1bb2d4e31785b5a4846ab0a22e"} Dec 03 11:41:29 crc kubenswrapper[5052]: I1203 11:41:29.379664 5052 patch_prober.go:28] interesting pod/machine-config-daemon-74bj6 container/machine-config-daemon namespace/openshift-machine-config-operator: Liveness probe status=failure output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" start-of-body= Dec 03 11:41:29 crc kubenswrapper[5052]: I1203 11:41:29.379974 5052 prober.go:107] "Probe failed" probeType="Liveness" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" containerName="machine-config-daemon" probeResult="failure" output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" Dec 03 11:41:30 crc kubenswrapper[5052]: I1203 11:41:30.751633 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:41:30 crc kubenswrapper[5052]: I1203 11:41:30.947932 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/c1ca7808-d027-424d-9850-e0bcd12ada2a-config-data\") pod \"c1ca7808-d027-424d-9850-e0bcd12ada2a\" (UID: \"c1ca7808-d027-424d-9850-e0bcd12ada2a\") " Dec 03 11:41:30 crc kubenswrapper[5052]: I1203 11:41:30.948008 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/c1ca7808-d027-424d-9850-e0bcd12ada2a-logs\") pod \"c1ca7808-d027-424d-9850-e0bcd12ada2a\" (UID: \"c1ca7808-d027-424d-9850-e0bcd12ada2a\") " Dec 03 11:41:30 crc kubenswrapper[5052]: I1203 11:41:30.948108 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f46jc\" (UniqueName: \"kubernetes.io/projected/c1ca7808-d027-424d-9850-e0bcd12ada2a-kube-api-access-f46jc\") pod \"c1ca7808-d027-424d-9850-e0bcd12ada2a\" (UID: \"c1ca7808-d027-424d-9850-e0bcd12ada2a\") " Dec 03 11:41:30 crc kubenswrapper[5052]: I1203 11:41:30.948207 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c1ca7808-d027-424d-9850-e0bcd12ada2a-combined-ca-bundle\") pod \"c1ca7808-d027-424d-9850-e0bcd12ada2a\" (UID: \"c1ca7808-d027-424d-9850-e0bcd12ada2a\") " Dec 03 11:41:30 crc kubenswrapper[5052]: I1203 11:41:30.951803 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c1ca7808-d027-424d-9850-e0bcd12ada2a-logs" (OuterVolumeSpecName: "logs") pod "c1ca7808-d027-424d-9850-e0bcd12ada2a" (UID: "c1ca7808-d027-424d-9850-e0bcd12ada2a"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:41:30 crc kubenswrapper[5052]: I1203 11:41:30.974399 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c1ca7808-d027-424d-9850-e0bcd12ada2a-kube-api-access-f46jc" (OuterVolumeSpecName: "kube-api-access-f46jc") pod "c1ca7808-d027-424d-9850-e0bcd12ada2a" (UID: "c1ca7808-d027-424d-9850-e0bcd12ada2a"). InnerVolumeSpecName "kube-api-access-f46jc". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:41:30 crc kubenswrapper[5052]: I1203 11:41:30.977503 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c1ca7808-d027-424d-9850-e0bcd12ada2a-config-data" (OuterVolumeSpecName: "config-data") pod "c1ca7808-d027-424d-9850-e0bcd12ada2a" (UID: "c1ca7808-d027-424d-9850-e0bcd12ada2a"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:41:30 crc kubenswrapper[5052]: I1203 11:41:30.982465 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c1ca7808-d027-424d-9850-e0bcd12ada2a-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "c1ca7808-d027-424d-9850-e0bcd12ada2a" (UID: "c1ca7808-d027-424d-9850-e0bcd12ada2a"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:41:31 crc kubenswrapper[5052]: I1203 11:41:31.050385 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c1ca7808-d027-424d-9850-e0bcd12ada2a-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:41:31 crc kubenswrapper[5052]: I1203 11:41:31.050434 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/c1ca7808-d027-424d-9850-e0bcd12ada2a-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:41:31 crc kubenswrapper[5052]: I1203 11:41:31.050443 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/c1ca7808-d027-424d-9850-e0bcd12ada2a-logs\") on node \"crc\" DevicePath \"\"" Dec 03 11:41:31 crc kubenswrapper[5052]: I1203 11:41:31.050452 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-f46jc\" (UniqueName: \"kubernetes.io/projected/c1ca7808-d027-424d-9850-e0bcd12ada2a-kube-api-access-f46jc\") on node \"crc\" DevicePath \"\"" Dec 03 11:41:31 crc kubenswrapper[5052]: I1203 11:41:31.072694 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:41:31 crc kubenswrapper[5052]: I1203 11:41:31.125975 5052 generic.go:334] "Generic (PLEG): container finished" podID="c1ca7808-d027-424d-9850-e0bcd12ada2a" containerID="be7d93b2064ab92bb9933e97b2f2019f219a60958f433d063793dd8831b62f4f" exitCode=0 Dec 03 11:41:31 crc kubenswrapper[5052]: I1203 11:41:31.126394 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"c1ca7808-d027-424d-9850-e0bcd12ada2a","Type":"ContainerDied","Data":"be7d93b2064ab92bb9933e97b2f2019f219a60958f433d063793dd8831b62f4f"} Dec 03 11:41:31 crc kubenswrapper[5052]: I1203 11:41:31.126434 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"c1ca7808-d027-424d-9850-e0bcd12ada2a","Type":"ContainerDied","Data":"685050ed96ac5a7d6d628198e239092ab69230718034ac6af0ca5135529ebbf6"} Dec 03 11:41:31 crc kubenswrapper[5052]: I1203 11:41:31.126454 5052 scope.go:117] "RemoveContainer" containerID="be7d93b2064ab92bb9933e97b2f2019f219a60958f433d063793dd8831b62f4f" Dec 03 11:41:31 crc kubenswrapper[5052]: I1203 11:41:31.126634 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:41:31 crc kubenswrapper[5052]: I1203 11:41:31.177277 5052 generic.go:334] "Generic (PLEG): container finished" podID="6b68de4d-a6d8-495d-becf-bd9f10da4a0b" containerID="a4bea6910b59cfab257d2ff4d76a6cbd50648e0cfb6710c83724db04cf294850" exitCode=0 Dec 03 11:41:31 crc kubenswrapper[5052]: I1203 11:41:31.177319 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"6b68de4d-a6d8-495d-becf-bd9f10da4a0b","Type":"ContainerDied","Data":"a4bea6910b59cfab257d2ff4d76a6cbd50648e0cfb6710c83724db04cf294850"} Dec 03 11:41:31 crc kubenswrapper[5052]: I1203 11:41:31.177344 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"6b68de4d-a6d8-495d-becf-bd9f10da4a0b","Type":"ContainerDied","Data":"6aaf0623a38647268f722562653fe3da57df21b34502b4307b848205cef8463b"} Dec 03 11:41:31 crc kubenswrapper[5052]: I1203 11:41:31.177413 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:41:31 crc kubenswrapper[5052]: I1203 11:41:31.190725 5052 scope.go:117] "RemoveContainer" containerID="1c657dcd8b7ce371f9cd12b8cc944e9c2595bb1bb2d4e31785b5a4846ab0a22e" Dec 03 11:41:31 crc kubenswrapper[5052]: I1203 11:41:31.197777 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 11:41:31 crc kubenswrapper[5052]: I1203 11:41:31.215883 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 11:41:31 crc kubenswrapper[5052]: I1203 11:41:31.224967 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 11:41:31 crc kubenswrapper[5052]: E1203 11:41:31.225389 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="c1ca7808-d027-424d-9850-e0bcd12ada2a" containerName="nova-api-log" Dec 03 11:41:31 crc kubenswrapper[5052]: I1203 11:41:31.225401 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="c1ca7808-d027-424d-9850-e0bcd12ada2a" containerName="nova-api-log" Dec 03 11:41:31 crc kubenswrapper[5052]: E1203 11:41:31.225421 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="6b68de4d-a6d8-495d-becf-bd9f10da4a0b" containerName="sg-core" Dec 03 11:41:31 crc kubenswrapper[5052]: I1203 11:41:31.225428 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="6b68de4d-a6d8-495d-becf-bd9f10da4a0b" containerName="sg-core" Dec 03 11:41:31 crc kubenswrapper[5052]: E1203 11:41:31.225439 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="6b68de4d-a6d8-495d-becf-bd9f10da4a0b" containerName="ceilometer-notification-agent" Dec 03 11:41:31 crc kubenswrapper[5052]: I1203 11:41:31.225445 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="6b68de4d-a6d8-495d-becf-bd9f10da4a0b" containerName="ceilometer-notification-agent" Dec 03 11:41:31 crc kubenswrapper[5052]: E1203 11:41:31.225460 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="6b68de4d-a6d8-495d-becf-bd9f10da4a0b" containerName="proxy-httpd" Dec 03 11:41:31 crc kubenswrapper[5052]: I1203 11:41:31.225466 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="6b68de4d-a6d8-495d-becf-bd9f10da4a0b" containerName="proxy-httpd" Dec 03 11:41:31 crc kubenswrapper[5052]: E1203 11:41:31.225479 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="c1ca7808-d027-424d-9850-e0bcd12ada2a" containerName="nova-api-api" Dec 03 11:41:31 crc kubenswrapper[5052]: I1203 11:41:31.225485 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="c1ca7808-d027-424d-9850-e0bcd12ada2a" containerName="nova-api-api" Dec 03 11:41:31 crc kubenswrapper[5052]: E1203 11:41:31.225500 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="6b68de4d-a6d8-495d-becf-bd9f10da4a0b" containerName="ceilometer-central-agent" Dec 03 11:41:31 crc kubenswrapper[5052]: I1203 11:41:31.225506 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="6b68de4d-a6d8-495d-becf-bd9f10da4a0b" containerName="ceilometer-central-agent" Dec 03 11:41:31 crc kubenswrapper[5052]: I1203 11:41:31.225691 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="c1ca7808-d027-424d-9850-e0bcd12ada2a" containerName="nova-api-log" Dec 03 11:41:31 crc kubenswrapper[5052]: I1203 11:41:31.225707 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="c1ca7808-d027-424d-9850-e0bcd12ada2a" containerName="nova-api-api" Dec 03 11:41:31 crc kubenswrapper[5052]: I1203 11:41:31.225728 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="6b68de4d-a6d8-495d-becf-bd9f10da4a0b" containerName="ceilometer-central-agent" Dec 03 11:41:31 crc kubenswrapper[5052]: I1203 11:41:31.225740 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="6b68de4d-a6d8-495d-becf-bd9f10da4a0b" containerName="ceilometer-notification-agent" Dec 03 11:41:31 crc kubenswrapper[5052]: I1203 11:41:31.225750 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="6b68de4d-a6d8-495d-becf-bd9f10da4a0b" containerName="sg-core" Dec 03 11:41:31 crc kubenswrapper[5052]: I1203 11:41:31.225758 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="6b68de4d-a6d8-495d-becf-bd9f10da4a0b" containerName="proxy-httpd" Dec 03 11:41:31 crc kubenswrapper[5052]: I1203 11:41:31.226666 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:41:31 crc kubenswrapper[5052]: I1203 11:41:31.232201 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-nova-public-svc" Dec 03 11:41:31 crc kubenswrapper[5052]: I1203 11:41:31.232312 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-nova-internal-svc" Dec 03 11:41:31 crc kubenswrapper[5052]: I1203 11:41:31.232453 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-api-config-data" Dec 03 11:41:31 crc kubenswrapper[5052]: I1203 11:41:31.236278 5052 scope.go:117] "RemoveContainer" containerID="be7d93b2064ab92bb9933e97b2f2019f219a60958f433d063793dd8831b62f4f" Dec 03 11:41:31 crc kubenswrapper[5052]: I1203 11:41:31.237571 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 11:41:31 crc kubenswrapper[5052]: E1203 11:41:31.240801 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"be7d93b2064ab92bb9933e97b2f2019f219a60958f433d063793dd8831b62f4f\": container with ID starting with be7d93b2064ab92bb9933e97b2f2019f219a60958f433d063793dd8831b62f4f not found: ID does not exist" containerID="be7d93b2064ab92bb9933e97b2f2019f219a60958f433d063793dd8831b62f4f" Dec 03 11:41:31 crc kubenswrapper[5052]: I1203 11:41:31.240847 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"be7d93b2064ab92bb9933e97b2f2019f219a60958f433d063793dd8831b62f4f"} err="failed to get container status \"be7d93b2064ab92bb9933e97b2f2019f219a60958f433d063793dd8831b62f4f\": rpc error: code = NotFound desc = could not find container \"be7d93b2064ab92bb9933e97b2f2019f219a60958f433d063793dd8831b62f4f\": container with ID starting with be7d93b2064ab92bb9933e97b2f2019f219a60958f433d063793dd8831b62f4f not found: ID does not exist" Dec 03 11:41:31 crc kubenswrapper[5052]: I1203 11:41:31.240878 5052 scope.go:117] "RemoveContainer" containerID="1c657dcd8b7ce371f9cd12b8cc944e9c2595bb1bb2d4e31785b5a4846ab0a22e" Dec 03 11:41:31 crc kubenswrapper[5052]: E1203 11:41:31.241491 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1c657dcd8b7ce371f9cd12b8cc944e9c2595bb1bb2d4e31785b5a4846ab0a22e\": container with ID starting with 1c657dcd8b7ce371f9cd12b8cc944e9c2595bb1bb2d4e31785b5a4846ab0a22e not found: ID does not exist" containerID="1c657dcd8b7ce371f9cd12b8cc944e9c2595bb1bb2d4e31785b5a4846ab0a22e" Dec 03 11:41:31 crc kubenswrapper[5052]: I1203 11:41:31.241532 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1c657dcd8b7ce371f9cd12b8cc944e9c2595bb1bb2d4e31785b5a4846ab0a22e"} err="failed to get container status \"1c657dcd8b7ce371f9cd12b8cc944e9c2595bb1bb2d4e31785b5a4846ab0a22e\": rpc error: code = NotFound desc = could not find container \"1c657dcd8b7ce371f9cd12b8cc944e9c2595bb1bb2d4e31785b5a4846ab0a22e\": container with ID starting with 1c657dcd8b7ce371f9cd12b8cc944e9c2595bb1bb2d4e31785b5a4846ab0a22e not found: ID does not exist" Dec 03 11:41:31 crc kubenswrapper[5052]: I1203 11:41:31.241550 5052 scope.go:117] "RemoveContainer" containerID="ac07efb86dea0bf7769f94c38526b11b8aedace2c8ee55227222140c40e3d1b3" Dec 03 11:41:31 crc kubenswrapper[5052]: I1203 11:41:31.254249 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/6b68de4d-a6d8-495d-becf-bd9f10da4a0b-log-httpd\") pod \"6b68de4d-a6d8-495d-becf-bd9f10da4a0b\" (UID: \"6b68de4d-a6d8-495d-becf-bd9f10da4a0b\") " Dec 03 11:41:31 crc kubenswrapper[5052]: I1203 11:41:31.255760 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b68de4d-a6d8-495d-becf-bd9f10da4a0b-log-httpd" (OuterVolumeSpecName: "log-httpd") pod "6b68de4d-a6d8-495d-becf-bd9f10da4a0b" (UID: "6b68de4d-a6d8-495d-becf-bd9f10da4a0b"). InnerVolumeSpecName "log-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:41:31 crc kubenswrapper[5052]: I1203 11:41:31.256812 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6b68de4d-a6d8-495d-becf-bd9f10da4a0b-config-data\") pod \"6b68de4d-a6d8-495d-becf-bd9f10da4a0b\" (UID: \"6b68de4d-a6d8-495d-becf-bd9f10da4a0b\") " Dec 03 11:41:31 crc kubenswrapper[5052]: I1203 11:41:31.256904 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/6b68de4d-a6d8-495d-becf-bd9f10da4a0b-run-httpd\") pod \"6b68de4d-a6d8-495d-becf-bd9f10da4a0b\" (UID: \"6b68de4d-a6d8-495d-becf-bd9f10da4a0b\") " Dec 03 11:41:31 crc kubenswrapper[5052]: I1203 11:41:31.256983 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/6b68de4d-a6d8-495d-becf-bd9f10da4a0b-sg-core-conf-yaml\") pod \"6b68de4d-a6d8-495d-becf-bd9f10da4a0b\" (UID: \"6b68de4d-a6d8-495d-becf-bd9f10da4a0b\") " Dec 03 11:41:31 crc kubenswrapper[5052]: I1203 11:41:31.257285 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/6b68de4d-a6d8-495d-becf-bd9f10da4a0b-scripts\") pod \"6b68de4d-a6d8-495d-becf-bd9f10da4a0b\" (UID: \"6b68de4d-a6d8-495d-becf-bd9f10da4a0b\") " Dec 03 11:41:31 crc kubenswrapper[5052]: I1203 11:41:31.257340 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6b68de4d-a6d8-495d-becf-bd9f10da4a0b-combined-ca-bundle\") pod \"6b68de4d-a6d8-495d-becf-bd9f10da4a0b\" (UID: \"6b68de4d-a6d8-495d-becf-bd9f10da4a0b\") " Dec 03 11:41:31 crc kubenswrapper[5052]: I1203 11:41:31.257383 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sx6c6\" (UniqueName: \"kubernetes.io/projected/6b68de4d-a6d8-495d-becf-bd9f10da4a0b-kube-api-access-sx6c6\") pod \"6b68de4d-a6d8-495d-becf-bd9f10da4a0b\" (UID: \"6b68de4d-a6d8-495d-becf-bd9f10da4a0b\") " Dec 03 11:41:31 crc kubenswrapper[5052]: I1203 11:41:31.257853 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/a4d316f2-ae6d-4b1c-b553-bb81b1cb7f61-internal-tls-certs\") pod \"nova-api-0\" (UID: \"a4d316f2-ae6d-4b1c-b553-bb81b1cb7f61\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:41:31 crc kubenswrapper[5052]: I1203 11:41:31.257918 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/a4d316f2-ae6d-4b1c-b553-bb81b1cb7f61-logs\") pod \"nova-api-0\" (UID: \"a4d316f2-ae6d-4b1c-b553-bb81b1cb7f61\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:41:31 crc kubenswrapper[5052]: I1203 11:41:31.257959 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6m8n6\" (UniqueName: \"kubernetes.io/projected/a4d316f2-ae6d-4b1c-b553-bb81b1cb7f61-kube-api-access-6m8n6\") pod \"nova-api-0\" (UID: \"a4d316f2-ae6d-4b1c-b553-bb81b1cb7f61\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:41:31 crc kubenswrapper[5052]: I1203 11:41:31.257988 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a4d316f2-ae6d-4b1c-b553-bb81b1cb7f61-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"a4d316f2-ae6d-4b1c-b553-bb81b1cb7f61\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:41:31 crc kubenswrapper[5052]: I1203 11:41:31.258008 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a4d316f2-ae6d-4b1c-b553-bb81b1cb7f61-config-data\") pod \"nova-api-0\" (UID: \"a4d316f2-ae6d-4b1c-b553-bb81b1cb7f61\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:41:31 crc kubenswrapper[5052]: I1203 11:41:31.258174 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/a4d316f2-ae6d-4b1c-b553-bb81b1cb7f61-public-tls-certs\") pod \"nova-api-0\" (UID: \"a4d316f2-ae6d-4b1c-b553-bb81b1cb7f61\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:41:31 crc kubenswrapper[5052]: I1203 11:41:31.258258 5052 reconciler_common.go:293] "Volume detached for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/6b68de4d-a6d8-495d-becf-bd9f10da4a0b-log-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 11:41:31 crc kubenswrapper[5052]: I1203 11:41:31.258755 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b68de4d-a6d8-495d-becf-bd9f10da4a0b-run-httpd" (OuterVolumeSpecName: "run-httpd") pod "6b68de4d-a6d8-495d-becf-bd9f10da4a0b" (UID: "6b68de4d-a6d8-495d-becf-bd9f10da4a0b"). InnerVolumeSpecName "run-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:41:31 crc kubenswrapper[5052]: I1203 11:41:31.264733 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6b68de4d-a6d8-495d-becf-bd9f10da4a0b-kube-api-access-sx6c6" (OuterVolumeSpecName: "kube-api-access-sx6c6") pod "6b68de4d-a6d8-495d-becf-bd9f10da4a0b" (UID: "6b68de4d-a6d8-495d-becf-bd9f10da4a0b"). InnerVolumeSpecName "kube-api-access-sx6c6". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:41:31 crc kubenswrapper[5052]: I1203 11:41:31.267438 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6b68de4d-a6d8-495d-becf-bd9f10da4a0b-scripts" (OuterVolumeSpecName: "scripts") pod "6b68de4d-a6d8-495d-becf-bd9f10da4a0b" (UID: "6b68de4d-a6d8-495d-becf-bd9f10da4a0b"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:41:31 crc kubenswrapper[5052]: I1203 11:41:31.279349 5052 scope.go:117] "RemoveContainer" containerID="e4055c48b95707806b72f0c0d8e98ad9447f6d9e2e1a3b1fe418cb3de4778816" Dec 03 11:41:31 crc kubenswrapper[5052]: I1203 11:41:31.302013 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6b68de4d-a6d8-495d-becf-bd9f10da4a0b-sg-core-conf-yaml" (OuterVolumeSpecName: "sg-core-conf-yaml") pod "6b68de4d-a6d8-495d-becf-bd9f10da4a0b" (UID: "6b68de4d-a6d8-495d-becf-bd9f10da4a0b"). InnerVolumeSpecName "sg-core-conf-yaml". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:41:31 crc kubenswrapper[5052]: I1203 11:41:31.359772 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/a4d316f2-ae6d-4b1c-b553-bb81b1cb7f61-internal-tls-certs\") pod \"nova-api-0\" (UID: \"a4d316f2-ae6d-4b1c-b553-bb81b1cb7f61\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:41:31 crc kubenswrapper[5052]: I1203 11:41:31.359845 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/a4d316f2-ae6d-4b1c-b553-bb81b1cb7f61-logs\") pod \"nova-api-0\" (UID: \"a4d316f2-ae6d-4b1c-b553-bb81b1cb7f61\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:41:31 crc kubenswrapper[5052]: I1203 11:41:31.359889 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-6m8n6\" (UniqueName: \"kubernetes.io/projected/a4d316f2-ae6d-4b1c-b553-bb81b1cb7f61-kube-api-access-6m8n6\") pod \"nova-api-0\" (UID: \"a4d316f2-ae6d-4b1c-b553-bb81b1cb7f61\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:41:31 crc kubenswrapper[5052]: I1203 11:41:31.359917 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a4d316f2-ae6d-4b1c-b553-bb81b1cb7f61-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"a4d316f2-ae6d-4b1c-b553-bb81b1cb7f61\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:41:31 crc kubenswrapper[5052]: I1203 11:41:31.359939 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a4d316f2-ae6d-4b1c-b553-bb81b1cb7f61-config-data\") pod \"nova-api-0\" (UID: \"a4d316f2-ae6d-4b1c-b553-bb81b1cb7f61\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:41:31 crc kubenswrapper[5052]: I1203 11:41:31.360063 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/a4d316f2-ae6d-4b1c-b553-bb81b1cb7f61-public-tls-certs\") pod \"nova-api-0\" (UID: \"a4d316f2-ae6d-4b1c-b553-bb81b1cb7f61\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:41:31 crc kubenswrapper[5052]: I1203 11:41:31.360158 5052 reconciler_common.go:293] "Volume detached for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/6b68de4d-a6d8-495d-becf-bd9f10da4a0b-run-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 11:41:31 crc kubenswrapper[5052]: I1203 11:41:31.360173 5052 reconciler_common.go:293] "Volume detached for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/6b68de4d-a6d8-495d-becf-bd9f10da4a0b-sg-core-conf-yaml\") on node \"crc\" DevicePath \"\"" Dec 03 11:41:31 crc kubenswrapper[5052]: I1203 11:41:31.360186 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/6b68de4d-a6d8-495d-becf-bd9f10da4a0b-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:41:31 crc kubenswrapper[5052]: I1203 11:41:31.360197 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-sx6c6\" (UniqueName: \"kubernetes.io/projected/6b68de4d-a6d8-495d-becf-bd9f10da4a0b-kube-api-access-sx6c6\") on node \"crc\" DevicePath \"\"" Dec 03 11:41:31 crc kubenswrapper[5052]: I1203 11:41:31.361217 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/a4d316f2-ae6d-4b1c-b553-bb81b1cb7f61-logs\") pod \"nova-api-0\" (UID: \"a4d316f2-ae6d-4b1c-b553-bb81b1cb7f61\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:41:31 crc kubenswrapper[5052]: I1203 11:41:31.364119 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a4d316f2-ae6d-4b1c-b553-bb81b1cb7f61-config-data\") pod \"nova-api-0\" (UID: \"a4d316f2-ae6d-4b1c-b553-bb81b1cb7f61\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:41:31 crc kubenswrapper[5052]: I1203 11:41:31.365418 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6b68de4d-a6d8-495d-becf-bd9f10da4a0b-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "6b68de4d-a6d8-495d-becf-bd9f10da4a0b" (UID: "6b68de4d-a6d8-495d-becf-bd9f10da4a0b"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:41:31 crc kubenswrapper[5052]: I1203 11:41:31.366745 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/a4d316f2-ae6d-4b1c-b553-bb81b1cb7f61-internal-tls-certs\") pod \"nova-api-0\" (UID: \"a4d316f2-ae6d-4b1c-b553-bb81b1cb7f61\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:41:31 crc kubenswrapper[5052]: I1203 11:41:31.369488 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/a4d316f2-ae6d-4b1c-b553-bb81b1cb7f61-public-tls-certs\") pod \"nova-api-0\" (UID: \"a4d316f2-ae6d-4b1c-b553-bb81b1cb7f61\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:41:31 crc kubenswrapper[5052]: I1203 11:41:31.379428 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a4d316f2-ae6d-4b1c-b553-bb81b1cb7f61-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"a4d316f2-ae6d-4b1c-b553-bb81b1cb7f61\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:41:31 crc kubenswrapper[5052]: I1203 11:41:31.380273 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-6m8n6\" (UniqueName: \"kubernetes.io/projected/a4d316f2-ae6d-4b1c-b553-bb81b1cb7f61-kube-api-access-6m8n6\") pod \"nova-api-0\" (UID: \"a4d316f2-ae6d-4b1c-b553-bb81b1cb7f61\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:41:31 crc kubenswrapper[5052]: I1203 11:41:31.381704 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6b68de4d-a6d8-495d-becf-bd9f10da4a0b-config-data" (OuterVolumeSpecName: "config-data") pod "6b68de4d-a6d8-495d-becf-bd9f10da4a0b" (UID: "6b68de4d-a6d8-495d-becf-bd9f10da4a0b"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:41:31 crc kubenswrapper[5052]: I1203 11:41:31.459167 5052 scope.go:117] "RemoveContainer" containerID="a4bea6910b59cfab257d2ff4d76a6cbd50648e0cfb6710c83724db04cf294850" Dec 03 11:41:31 crc kubenswrapper[5052]: I1203 11:41:31.461384 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6b68de4d-a6d8-495d-becf-bd9f10da4a0b-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:41:31 crc kubenswrapper[5052]: I1203 11:41:31.461414 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6b68de4d-a6d8-495d-becf-bd9f10da4a0b-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:41:31 crc kubenswrapper[5052]: I1203 11:41:31.479936 5052 scope.go:117] "RemoveContainer" containerID="ab919ccb3c04eb505bf1bce32f7913ab8034eced7677cfe0f55e36d797175613" Dec 03 11:41:31 crc kubenswrapper[5052]: I1203 11:41:31.556135 5052 scope.go:117] "RemoveContainer" containerID="ac07efb86dea0bf7769f94c38526b11b8aedace2c8ee55227222140c40e3d1b3" Dec 03 11:41:31 crc kubenswrapper[5052]: E1203 11:41:31.556650 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ac07efb86dea0bf7769f94c38526b11b8aedace2c8ee55227222140c40e3d1b3\": container with ID starting with ac07efb86dea0bf7769f94c38526b11b8aedace2c8ee55227222140c40e3d1b3 not found: ID does not exist" containerID="ac07efb86dea0bf7769f94c38526b11b8aedace2c8ee55227222140c40e3d1b3" Dec 03 11:41:31 crc kubenswrapper[5052]: I1203 11:41:31.556699 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ac07efb86dea0bf7769f94c38526b11b8aedace2c8ee55227222140c40e3d1b3"} err="failed to get container status \"ac07efb86dea0bf7769f94c38526b11b8aedace2c8ee55227222140c40e3d1b3\": rpc error: code = NotFound desc = could not find container \"ac07efb86dea0bf7769f94c38526b11b8aedace2c8ee55227222140c40e3d1b3\": container with ID starting with ac07efb86dea0bf7769f94c38526b11b8aedace2c8ee55227222140c40e3d1b3 not found: ID does not exist" Dec 03 11:41:31 crc kubenswrapper[5052]: I1203 11:41:31.556717 5052 scope.go:117] "RemoveContainer" containerID="e4055c48b95707806b72f0c0d8e98ad9447f6d9e2e1a3b1fe418cb3de4778816" Dec 03 11:41:31 crc kubenswrapper[5052]: I1203 11:41:31.556950 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:41:31 crc kubenswrapper[5052]: E1203 11:41:31.557824 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e4055c48b95707806b72f0c0d8e98ad9447f6d9e2e1a3b1fe418cb3de4778816\": container with ID starting with e4055c48b95707806b72f0c0d8e98ad9447f6d9e2e1a3b1fe418cb3de4778816 not found: ID does not exist" containerID="e4055c48b95707806b72f0c0d8e98ad9447f6d9e2e1a3b1fe418cb3de4778816" Dec 03 11:41:31 crc kubenswrapper[5052]: I1203 11:41:31.557855 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e4055c48b95707806b72f0c0d8e98ad9447f6d9e2e1a3b1fe418cb3de4778816"} err="failed to get container status \"e4055c48b95707806b72f0c0d8e98ad9447f6d9e2e1a3b1fe418cb3de4778816\": rpc error: code = NotFound desc = could not find container \"e4055c48b95707806b72f0c0d8e98ad9447f6d9e2e1a3b1fe418cb3de4778816\": container with ID starting with e4055c48b95707806b72f0c0d8e98ad9447f6d9e2e1a3b1fe418cb3de4778816 not found: ID does not exist" Dec 03 11:41:31 crc kubenswrapper[5052]: I1203 11:41:31.557873 5052 scope.go:117] "RemoveContainer" containerID="a4bea6910b59cfab257d2ff4d76a6cbd50648e0cfb6710c83724db04cf294850" Dec 03 11:41:31 crc kubenswrapper[5052]: E1203 11:41:31.558435 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a4bea6910b59cfab257d2ff4d76a6cbd50648e0cfb6710c83724db04cf294850\": container with ID starting with a4bea6910b59cfab257d2ff4d76a6cbd50648e0cfb6710c83724db04cf294850 not found: ID does not exist" containerID="a4bea6910b59cfab257d2ff4d76a6cbd50648e0cfb6710c83724db04cf294850" Dec 03 11:41:31 crc kubenswrapper[5052]: I1203 11:41:31.558498 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a4bea6910b59cfab257d2ff4d76a6cbd50648e0cfb6710c83724db04cf294850"} err="failed to get container status \"a4bea6910b59cfab257d2ff4d76a6cbd50648e0cfb6710c83724db04cf294850\": rpc error: code = NotFound desc = could not find container \"a4bea6910b59cfab257d2ff4d76a6cbd50648e0cfb6710c83724db04cf294850\": container with ID starting with a4bea6910b59cfab257d2ff4d76a6cbd50648e0cfb6710c83724db04cf294850 not found: ID does not exist" Dec 03 11:41:31 crc kubenswrapper[5052]: I1203 11:41:31.558550 5052 scope.go:117] "RemoveContainer" containerID="ab919ccb3c04eb505bf1bce32f7913ab8034eced7677cfe0f55e36d797175613" Dec 03 11:41:31 crc kubenswrapper[5052]: E1203 11:41:31.566201 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ab919ccb3c04eb505bf1bce32f7913ab8034eced7677cfe0f55e36d797175613\": container with ID starting with ab919ccb3c04eb505bf1bce32f7913ab8034eced7677cfe0f55e36d797175613 not found: ID does not exist" containerID="ab919ccb3c04eb505bf1bce32f7913ab8034eced7677cfe0f55e36d797175613" Dec 03 11:41:31 crc kubenswrapper[5052]: I1203 11:41:31.566264 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ab919ccb3c04eb505bf1bce32f7913ab8034eced7677cfe0f55e36d797175613"} err="failed to get container status \"ab919ccb3c04eb505bf1bce32f7913ab8034eced7677cfe0f55e36d797175613\": rpc error: code = NotFound desc = could not find container \"ab919ccb3c04eb505bf1bce32f7913ab8034eced7677cfe0f55e36d797175613\": container with ID starting with ab919ccb3c04eb505bf1bce32f7913ab8034eced7677cfe0f55e36d797175613 not found: ID does not exist" Dec 03 11:41:31 crc kubenswrapper[5052]: I1203 11:41:31.580067 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:41:31 crc kubenswrapper[5052]: I1203 11:41:31.607378 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:41:31 crc kubenswrapper[5052]: I1203 11:41:31.638835 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:41:31 crc kubenswrapper[5052]: I1203 11:41:31.641226 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:41:31 crc kubenswrapper[5052]: I1203 11:41:31.643790 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-config-data" Dec 03 11:41:31 crc kubenswrapper[5052]: I1203 11:41:31.648924 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-scripts" Dec 03 11:41:31 crc kubenswrapper[5052]: I1203 11:41:31.650075 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:41:31 crc kubenswrapper[5052]: I1203 11:41:31.698611 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6b68de4d-a6d8-495d-becf-bd9f10da4a0b" path="/var/lib/kubelet/pods/6b68de4d-a6d8-495d-becf-bd9f10da4a0b/volumes" Dec 03 11:41:31 crc kubenswrapper[5052]: I1203 11:41:31.699646 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c1ca7808-d027-424d-9850-e0bcd12ada2a" path="/var/lib/kubelet/pods/c1ca7808-d027-424d-9850-e0bcd12ada2a/volumes" Dec 03 11:41:31 crc kubenswrapper[5052]: I1203 11:41:31.700168 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 11:41:31 crc kubenswrapper[5052]: I1203 11:41:31.703709 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/d13d0e77-9a6e-4e51-98c4-39828e320c6a-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"d13d0e77-9a6e-4e51-98c4-39828e320c6a\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:41:31 crc kubenswrapper[5052]: I1203 11:41:31.703754 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/d13d0e77-9a6e-4e51-98c4-39828e320c6a-run-httpd\") pod \"ceilometer-0\" (UID: \"d13d0e77-9a6e-4e51-98c4-39828e320c6a\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:41:31 crc kubenswrapper[5052]: I1203 11:41:31.704079 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zn8nv\" (UniqueName: \"kubernetes.io/projected/d13d0e77-9a6e-4e51-98c4-39828e320c6a-kube-api-access-zn8nv\") pod \"ceilometer-0\" (UID: \"d13d0e77-9a6e-4e51-98c4-39828e320c6a\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:41:31 crc kubenswrapper[5052]: I1203 11:41:31.704106 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/d13d0e77-9a6e-4e51-98c4-39828e320c6a-log-httpd\") pod \"ceilometer-0\" (UID: \"d13d0e77-9a6e-4e51-98c4-39828e320c6a\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:41:31 crc kubenswrapper[5052]: I1203 11:41:31.704153 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/d13d0e77-9a6e-4e51-98c4-39828e320c6a-config-data\") pod \"ceilometer-0\" (UID: \"d13d0e77-9a6e-4e51-98c4-39828e320c6a\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:41:31 crc kubenswrapper[5052]: I1203 11:41:31.704196 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d13d0e77-9a6e-4e51-98c4-39828e320c6a-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"d13d0e77-9a6e-4e51-98c4-39828e320c6a\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:41:31 crc kubenswrapper[5052]: I1203 11:41:31.704266 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/d13d0e77-9a6e-4e51-98c4-39828e320c6a-scripts\") pod \"ceilometer-0\" (UID: \"d13d0e77-9a6e-4e51-98c4-39828e320c6a\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:41:31 crc kubenswrapper[5052]: I1203 11:41:31.725940 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 11:41:31 crc kubenswrapper[5052]: I1203 11:41:31.806446 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/d13d0e77-9a6e-4e51-98c4-39828e320c6a-log-httpd\") pod \"ceilometer-0\" (UID: \"d13d0e77-9a6e-4e51-98c4-39828e320c6a\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:41:31 crc kubenswrapper[5052]: I1203 11:41:31.806486 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-zn8nv\" (UniqueName: \"kubernetes.io/projected/d13d0e77-9a6e-4e51-98c4-39828e320c6a-kube-api-access-zn8nv\") pod \"ceilometer-0\" (UID: \"d13d0e77-9a6e-4e51-98c4-39828e320c6a\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:41:31 crc kubenswrapper[5052]: I1203 11:41:31.806550 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/d13d0e77-9a6e-4e51-98c4-39828e320c6a-config-data\") pod \"ceilometer-0\" (UID: \"d13d0e77-9a6e-4e51-98c4-39828e320c6a\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:41:31 crc kubenswrapper[5052]: I1203 11:41:31.806583 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d13d0e77-9a6e-4e51-98c4-39828e320c6a-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"d13d0e77-9a6e-4e51-98c4-39828e320c6a\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:41:31 crc kubenswrapper[5052]: I1203 11:41:31.806664 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/d13d0e77-9a6e-4e51-98c4-39828e320c6a-scripts\") pod \"ceilometer-0\" (UID: \"d13d0e77-9a6e-4e51-98c4-39828e320c6a\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:41:31 crc kubenswrapper[5052]: I1203 11:41:31.806712 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/d13d0e77-9a6e-4e51-98c4-39828e320c6a-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"d13d0e77-9a6e-4e51-98c4-39828e320c6a\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:41:31 crc kubenswrapper[5052]: I1203 11:41:31.806735 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/d13d0e77-9a6e-4e51-98c4-39828e320c6a-run-httpd\") pod \"ceilometer-0\" (UID: \"d13d0e77-9a6e-4e51-98c4-39828e320c6a\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:41:31 crc kubenswrapper[5052]: I1203 11:41:31.807169 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/d13d0e77-9a6e-4e51-98c4-39828e320c6a-run-httpd\") pod \"ceilometer-0\" (UID: \"d13d0e77-9a6e-4e51-98c4-39828e320c6a\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:41:31 crc kubenswrapper[5052]: I1203 11:41:31.807385 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/d13d0e77-9a6e-4e51-98c4-39828e320c6a-log-httpd\") pod \"ceilometer-0\" (UID: \"d13d0e77-9a6e-4e51-98c4-39828e320c6a\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:41:31 crc kubenswrapper[5052]: I1203 11:41:31.814038 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d13d0e77-9a6e-4e51-98c4-39828e320c6a-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"d13d0e77-9a6e-4e51-98c4-39828e320c6a\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:41:31 crc kubenswrapper[5052]: I1203 11:41:31.814699 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/d13d0e77-9a6e-4e51-98c4-39828e320c6a-scripts\") pod \"ceilometer-0\" (UID: \"d13d0e77-9a6e-4e51-98c4-39828e320c6a\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:41:31 crc kubenswrapper[5052]: I1203 11:41:31.815034 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/d13d0e77-9a6e-4e51-98c4-39828e320c6a-config-data\") pod \"ceilometer-0\" (UID: \"d13d0e77-9a6e-4e51-98c4-39828e320c6a\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:41:31 crc kubenswrapper[5052]: I1203 11:41:31.816495 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/d13d0e77-9a6e-4e51-98c4-39828e320c6a-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"d13d0e77-9a6e-4e51-98c4-39828e320c6a\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:41:31 crc kubenswrapper[5052]: I1203 11:41:31.843480 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-zn8nv\" (UniqueName: \"kubernetes.io/projected/d13d0e77-9a6e-4e51-98c4-39828e320c6a-kube-api-access-zn8nv\") pod \"ceilometer-0\" (UID: \"d13d0e77-9a6e-4e51-98c4-39828e320c6a\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:41:31 crc kubenswrapper[5052]: I1203 11:41:31.887279 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 11:41:31 crc kubenswrapper[5052]: W1203 11:41:31.895724 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda4d316f2_ae6d_4b1c_b553_bb81b1cb7f61.slice/crio-b76c70ca5e0b575b822e5b4012bed974504cc893d54ceab8b9d02f2bbece9af4 WatchSource:0}: Error finding container b76c70ca5e0b575b822e5b4012bed974504cc893d54ceab8b9d02f2bbece9af4: Status 404 returned error can't find the container with id b76c70ca5e0b575b822e5b4012bed974504cc893d54ceab8b9d02f2bbece9af4 Dec 03 11:41:31 crc kubenswrapper[5052]: I1203 11:41:31.992495 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:41:32 crc kubenswrapper[5052]: I1203 11:41:32.205052 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"a4d316f2-ae6d-4b1c-b553-bb81b1cb7f61","Type":"ContainerStarted","Data":"10df7a9640ada9b3c5a1aafcbff355b09e089f0926f1993c36cba1a424029fe5"} Dec 03 11:41:32 crc kubenswrapper[5052]: I1203 11:41:32.205096 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"a4d316f2-ae6d-4b1c-b553-bb81b1cb7f61","Type":"ContainerStarted","Data":"b76c70ca5e0b575b822e5b4012bed974504cc893d54ceab8b9d02f2bbece9af4"} Dec 03 11:41:32 crc kubenswrapper[5052]: I1203 11:41:32.229016 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 11:41:32 crc kubenswrapper[5052]: I1203 11:41:32.488342 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:41:32 crc kubenswrapper[5052]: I1203 11:41:32.563318 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell1-cell-mapping-wpd4q"] Dec 03 11:41:32 crc kubenswrapper[5052]: I1203 11:41:32.564693 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-cell-mapping-wpd4q" Dec 03 11:41:32 crc kubenswrapper[5052]: I1203 11:41:32.567677 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell1-manage-config-data" Dec 03 11:41:32 crc kubenswrapper[5052]: I1203 11:41:32.569926 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell1-manage-scripts" Dec 03 11:41:32 crc kubenswrapper[5052]: I1203 11:41:32.575172 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-cell-mapping-wpd4q"] Dec 03 11:41:32 crc kubenswrapper[5052]: I1203 11:41:32.624503 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/c0af0947-fd26-43a5-8b64-bb9bae861cf2-config-data\") pod \"nova-cell1-cell-mapping-wpd4q\" (UID: \"c0af0947-fd26-43a5-8b64-bb9bae861cf2\") " pod="openstack-kuttl-tests/nova-cell1-cell-mapping-wpd4q" Dec 03 11:41:32 crc kubenswrapper[5052]: I1203 11:41:32.624596 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-drrdl\" (UniqueName: \"kubernetes.io/projected/c0af0947-fd26-43a5-8b64-bb9bae861cf2-kube-api-access-drrdl\") pod \"nova-cell1-cell-mapping-wpd4q\" (UID: \"c0af0947-fd26-43a5-8b64-bb9bae861cf2\") " pod="openstack-kuttl-tests/nova-cell1-cell-mapping-wpd4q" Dec 03 11:41:32 crc kubenswrapper[5052]: I1203 11:41:32.624631 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c0af0947-fd26-43a5-8b64-bb9bae861cf2-combined-ca-bundle\") pod \"nova-cell1-cell-mapping-wpd4q\" (UID: \"c0af0947-fd26-43a5-8b64-bb9bae861cf2\") " pod="openstack-kuttl-tests/nova-cell1-cell-mapping-wpd4q" Dec 03 11:41:32 crc kubenswrapper[5052]: I1203 11:41:32.624703 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/c0af0947-fd26-43a5-8b64-bb9bae861cf2-scripts\") pod \"nova-cell1-cell-mapping-wpd4q\" (UID: \"c0af0947-fd26-43a5-8b64-bb9bae861cf2\") " pod="openstack-kuttl-tests/nova-cell1-cell-mapping-wpd4q" Dec 03 11:41:32 crc kubenswrapper[5052]: I1203 11:41:32.726289 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/c0af0947-fd26-43a5-8b64-bb9bae861cf2-config-data\") pod \"nova-cell1-cell-mapping-wpd4q\" (UID: \"c0af0947-fd26-43a5-8b64-bb9bae861cf2\") " pod="openstack-kuttl-tests/nova-cell1-cell-mapping-wpd4q" Dec 03 11:41:32 crc kubenswrapper[5052]: I1203 11:41:32.726358 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-drrdl\" (UniqueName: \"kubernetes.io/projected/c0af0947-fd26-43a5-8b64-bb9bae861cf2-kube-api-access-drrdl\") pod \"nova-cell1-cell-mapping-wpd4q\" (UID: \"c0af0947-fd26-43a5-8b64-bb9bae861cf2\") " pod="openstack-kuttl-tests/nova-cell1-cell-mapping-wpd4q" Dec 03 11:41:32 crc kubenswrapper[5052]: I1203 11:41:32.726404 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c0af0947-fd26-43a5-8b64-bb9bae861cf2-combined-ca-bundle\") pod \"nova-cell1-cell-mapping-wpd4q\" (UID: \"c0af0947-fd26-43a5-8b64-bb9bae861cf2\") " pod="openstack-kuttl-tests/nova-cell1-cell-mapping-wpd4q" Dec 03 11:41:32 crc kubenswrapper[5052]: I1203 11:41:32.726484 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/c0af0947-fd26-43a5-8b64-bb9bae861cf2-scripts\") pod \"nova-cell1-cell-mapping-wpd4q\" (UID: \"c0af0947-fd26-43a5-8b64-bb9bae861cf2\") " pod="openstack-kuttl-tests/nova-cell1-cell-mapping-wpd4q" Dec 03 11:41:32 crc kubenswrapper[5052]: I1203 11:41:32.731920 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c0af0947-fd26-43a5-8b64-bb9bae861cf2-combined-ca-bundle\") pod \"nova-cell1-cell-mapping-wpd4q\" (UID: \"c0af0947-fd26-43a5-8b64-bb9bae861cf2\") " pod="openstack-kuttl-tests/nova-cell1-cell-mapping-wpd4q" Dec 03 11:41:32 crc kubenswrapper[5052]: I1203 11:41:32.732023 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/c0af0947-fd26-43a5-8b64-bb9bae861cf2-scripts\") pod \"nova-cell1-cell-mapping-wpd4q\" (UID: \"c0af0947-fd26-43a5-8b64-bb9bae861cf2\") " pod="openstack-kuttl-tests/nova-cell1-cell-mapping-wpd4q" Dec 03 11:41:32 crc kubenswrapper[5052]: I1203 11:41:32.732196 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/c0af0947-fd26-43a5-8b64-bb9bae861cf2-config-data\") pod \"nova-cell1-cell-mapping-wpd4q\" (UID: \"c0af0947-fd26-43a5-8b64-bb9bae861cf2\") " pod="openstack-kuttl-tests/nova-cell1-cell-mapping-wpd4q" Dec 03 11:41:32 crc kubenswrapper[5052]: I1203 11:41:32.749826 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-drrdl\" (UniqueName: \"kubernetes.io/projected/c0af0947-fd26-43a5-8b64-bb9bae861cf2-kube-api-access-drrdl\") pod \"nova-cell1-cell-mapping-wpd4q\" (UID: \"c0af0947-fd26-43a5-8b64-bb9bae861cf2\") " pod="openstack-kuttl-tests/nova-cell1-cell-mapping-wpd4q" Dec 03 11:41:32 crc kubenswrapper[5052]: I1203 11:41:32.882808 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-cell-mapping-wpd4q" Dec 03 11:41:33 crc kubenswrapper[5052]: I1203 11:41:33.223973 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"d13d0e77-9a6e-4e51-98c4-39828e320c6a","Type":"ContainerStarted","Data":"636c294322bc7f27ae4cb1193bbeba3606afb3192fd2dc2b169b44566e0d038e"} Dec 03 11:41:33 crc kubenswrapper[5052]: I1203 11:41:33.226948 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"a4d316f2-ae6d-4b1c-b553-bb81b1cb7f61","Type":"ContainerStarted","Data":"16d8bd223b40ce74ff57080feb69709f07470b17bf4c54151f36087948f9585e"} Dec 03 11:41:33 crc kubenswrapper[5052]: I1203 11:41:33.248233 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-api-0" podStartSLOduration=2.248210662 podStartE2EDuration="2.248210662s" podCreationTimestamp="2025-12-03 11:41:31 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:41:33.24713406 +0000 UTC m=+1283.835375834" watchObservedRunningTime="2025-12-03 11:41:33.248210662 +0000 UTC m=+1283.836452426" Dec 03 11:41:33 crc kubenswrapper[5052]: I1203 11:41:33.383718 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-cell-mapping-wpd4q"] Dec 03 11:41:33 crc kubenswrapper[5052]: W1203 11:41:33.387756 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc0af0947_fd26_43a5_8b64_bb9bae861cf2.slice/crio-b9edc84962678742ba632d212a07e0e6202608075eb8e0bb0c932138c61b44c3 WatchSource:0}: Error finding container b9edc84962678742ba632d212a07e0e6202608075eb8e0bb0c932138c61b44c3: Status 404 returned error can't find the container with id b9edc84962678742ba632d212a07e0e6202608075eb8e0bb0c932138c61b44c3 Dec 03 11:41:34 crc kubenswrapper[5052]: I1203 11:41:34.236441 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-cell-mapping-wpd4q" event={"ID":"c0af0947-fd26-43a5-8b64-bb9bae861cf2","Type":"ContainerStarted","Data":"98a5d3b986b2516d0ac8f36fbeebe27e3364347af72f582505d55d4a22ea7445"} Dec 03 11:41:34 crc kubenswrapper[5052]: I1203 11:41:34.236989 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-cell-mapping-wpd4q" event={"ID":"c0af0947-fd26-43a5-8b64-bb9bae861cf2","Type":"ContainerStarted","Data":"b9edc84962678742ba632d212a07e0e6202608075eb8e0bb0c932138c61b44c3"} Dec 03 11:41:34 crc kubenswrapper[5052]: I1203 11:41:34.238563 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"d13d0e77-9a6e-4e51-98c4-39828e320c6a","Type":"ContainerStarted","Data":"2be85ceb7d3cd57a9befa38f17da587b414e83a180cebc28727fc4a5996939d1"} Dec 03 11:41:34 crc kubenswrapper[5052]: I1203 11:41:34.238591 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"d13d0e77-9a6e-4e51-98c4-39828e320c6a","Type":"ContainerStarted","Data":"1b0b626fe0bfe23d2e33c4a429245669c17a1a07a4936903e8ddb5ec47f1b434"} Dec 03 11:41:34 crc kubenswrapper[5052]: I1203 11:41:34.255853 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-cell1-cell-mapping-wpd4q" podStartSLOduration=2.255833125 podStartE2EDuration="2.255833125s" podCreationTimestamp="2025-12-03 11:41:32 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:41:34.252138708 +0000 UTC m=+1284.840380482" watchObservedRunningTime="2025-12-03 11:41:34.255833125 +0000 UTC m=+1284.844074889" Dec 03 11:41:34 crc kubenswrapper[5052]: I1203 11:41:34.613830 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/kube-state-metrics-0"] Dec 03 11:41:34 crc kubenswrapper[5052]: I1203 11:41:34.614255 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/kube-state-metrics-0" podUID="7535f906-b13f-4b91-b026-bac20ecb4867" containerName="kube-state-metrics" containerID="cri-o://c5eb397a2824a206861b0c68d056b30eaa38918807f14f653143e1839a6979a7" gracePeriod=30 Dec 03 11:41:35 crc kubenswrapper[5052]: I1203 11:41:35.030282 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 11:41:35 crc kubenswrapper[5052]: I1203 11:41:35.171841 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9dxw6\" (UniqueName: \"kubernetes.io/projected/7535f906-b13f-4b91-b026-bac20ecb4867-kube-api-access-9dxw6\") pod \"7535f906-b13f-4b91-b026-bac20ecb4867\" (UID: \"7535f906-b13f-4b91-b026-bac20ecb4867\") " Dec 03 11:41:35 crc kubenswrapper[5052]: I1203 11:41:35.176570 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7535f906-b13f-4b91-b026-bac20ecb4867-kube-api-access-9dxw6" (OuterVolumeSpecName: "kube-api-access-9dxw6") pod "7535f906-b13f-4b91-b026-bac20ecb4867" (UID: "7535f906-b13f-4b91-b026-bac20ecb4867"). InnerVolumeSpecName "kube-api-access-9dxw6". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:41:35 crc kubenswrapper[5052]: I1203 11:41:35.250062 5052 generic.go:334] "Generic (PLEG): container finished" podID="7535f906-b13f-4b91-b026-bac20ecb4867" containerID="c5eb397a2824a206861b0c68d056b30eaa38918807f14f653143e1839a6979a7" exitCode=2 Dec 03 11:41:35 crc kubenswrapper[5052]: I1203 11:41:35.250117 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/kube-state-metrics-0" event={"ID":"7535f906-b13f-4b91-b026-bac20ecb4867","Type":"ContainerDied","Data":"c5eb397a2824a206861b0c68d056b30eaa38918807f14f653143e1839a6979a7"} Dec 03 11:41:35 crc kubenswrapper[5052]: I1203 11:41:35.250558 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/kube-state-metrics-0" event={"ID":"7535f906-b13f-4b91-b026-bac20ecb4867","Type":"ContainerDied","Data":"5dc4f23ca754180c196168e997e200ebb2d925083ff1ecc22f6984be58982edc"} Dec 03 11:41:35 crc kubenswrapper[5052]: I1203 11:41:35.250139 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 11:41:35 crc kubenswrapper[5052]: I1203 11:41:35.250583 5052 scope.go:117] "RemoveContainer" containerID="c5eb397a2824a206861b0c68d056b30eaa38918807f14f653143e1839a6979a7" Dec 03 11:41:35 crc kubenswrapper[5052]: I1203 11:41:35.254314 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"d13d0e77-9a6e-4e51-98c4-39828e320c6a","Type":"ContainerStarted","Data":"9de6d546e32a6ebadd1f211b6d25ef1622cc49f940869c042b2e0e10bb7d3cb9"} Dec 03 11:41:35 crc kubenswrapper[5052]: I1203 11:41:35.274297 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-9dxw6\" (UniqueName: \"kubernetes.io/projected/7535f906-b13f-4b91-b026-bac20ecb4867-kube-api-access-9dxw6\") on node \"crc\" DevicePath \"\"" Dec 03 11:41:35 crc kubenswrapper[5052]: I1203 11:41:35.282570 5052 scope.go:117] "RemoveContainer" containerID="c5eb397a2824a206861b0c68d056b30eaa38918807f14f653143e1839a6979a7" Dec 03 11:41:35 crc kubenswrapper[5052]: E1203 11:41:35.283095 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c5eb397a2824a206861b0c68d056b30eaa38918807f14f653143e1839a6979a7\": container with ID starting with c5eb397a2824a206861b0c68d056b30eaa38918807f14f653143e1839a6979a7 not found: ID does not exist" containerID="c5eb397a2824a206861b0c68d056b30eaa38918807f14f653143e1839a6979a7" Dec 03 11:41:35 crc kubenswrapper[5052]: I1203 11:41:35.283136 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c5eb397a2824a206861b0c68d056b30eaa38918807f14f653143e1839a6979a7"} err="failed to get container status \"c5eb397a2824a206861b0c68d056b30eaa38918807f14f653143e1839a6979a7\": rpc error: code = NotFound desc = could not find container \"c5eb397a2824a206861b0c68d056b30eaa38918807f14f653143e1839a6979a7\": container with ID starting with c5eb397a2824a206861b0c68d056b30eaa38918807f14f653143e1839a6979a7 not found: ID does not exist" Dec 03 11:41:35 crc kubenswrapper[5052]: I1203 11:41:35.285051 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/kube-state-metrics-0"] Dec 03 11:41:35 crc kubenswrapper[5052]: I1203 11:41:35.295841 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/kube-state-metrics-0"] Dec 03 11:41:35 crc kubenswrapper[5052]: I1203 11:41:35.315712 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/kube-state-metrics-0"] Dec 03 11:41:35 crc kubenswrapper[5052]: E1203 11:41:35.316310 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="7535f906-b13f-4b91-b026-bac20ecb4867" containerName="kube-state-metrics" Dec 03 11:41:35 crc kubenswrapper[5052]: I1203 11:41:35.316847 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="7535f906-b13f-4b91-b026-bac20ecb4867" containerName="kube-state-metrics" Dec 03 11:41:35 crc kubenswrapper[5052]: I1203 11:41:35.317151 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="7535f906-b13f-4b91-b026-bac20ecb4867" containerName="kube-state-metrics" Dec 03 11:41:35 crc kubenswrapper[5052]: I1203 11:41:35.318362 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 11:41:35 crc kubenswrapper[5052]: I1203 11:41:35.326303 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"kube-state-metrics-tls-config" Dec 03 11:41:35 crc kubenswrapper[5052]: I1203 11:41:35.326590 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-kube-state-metrics-svc" Dec 03 11:41:35 crc kubenswrapper[5052]: I1203 11:41:35.327950 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/kube-state-metrics-0"] Dec 03 11:41:35 crc kubenswrapper[5052]: I1203 11:41:35.389443 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls-config\" (UniqueName: \"kubernetes.io/secret/8b465e38-7dbb-4d02-bce0-28ae0145520c-kube-state-metrics-tls-config\") pod \"kube-state-metrics-0\" (UID: \"8b465e38-7dbb-4d02-bce0-28ae0145520c\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 11:41:35 crc kubenswrapper[5052]: I1203 11:41:35.389707 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls-certs\" (UniqueName: \"kubernetes.io/secret/8b465e38-7dbb-4d02-bce0-28ae0145520c-kube-state-metrics-tls-certs\") pod \"kube-state-metrics-0\" (UID: \"8b465e38-7dbb-4d02-bce0-28ae0145520c\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 11:41:35 crc kubenswrapper[5052]: I1203 11:41:35.391089 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wwv7d\" (UniqueName: \"kubernetes.io/projected/8b465e38-7dbb-4d02-bce0-28ae0145520c-kube-api-access-wwv7d\") pod \"kube-state-metrics-0\" (UID: \"8b465e38-7dbb-4d02-bce0-28ae0145520c\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 11:41:35 crc kubenswrapper[5052]: I1203 11:41:35.391643 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8b465e38-7dbb-4d02-bce0-28ae0145520c-combined-ca-bundle\") pod \"kube-state-metrics-0\" (UID: \"8b465e38-7dbb-4d02-bce0-28ae0145520c\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 11:41:35 crc kubenswrapper[5052]: I1203 11:41:35.492965 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-wwv7d\" (UniqueName: \"kubernetes.io/projected/8b465e38-7dbb-4d02-bce0-28ae0145520c-kube-api-access-wwv7d\") pod \"kube-state-metrics-0\" (UID: \"8b465e38-7dbb-4d02-bce0-28ae0145520c\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 11:41:35 crc kubenswrapper[5052]: I1203 11:41:35.493275 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8b465e38-7dbb-4d02-bce0-28ae0145520c-combined-ca-bundle\") pod \"kube-state-metrics-0\" (UID: \"8b465e38-7dbb-4d02-bce0-28ae0145520c\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 11:41:35 crc kubenswrapper[5052]: I1203 11:41:35.493379 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls-config\" (UniqueName: \"kubernetes.io/secret/8b465e38-7dbb-4d02-bce0-28ae0145520c-kube-state-metrics-tls-config\") pod \"kube-state-metrics-0\" (UID: \"8b465e38-7dbb-4d02-bce0-28ae0145520c\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 11:41:35 crc kubenswrapper[5052]: I1203 11:41:35.493471 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls-certs\" (UniqueName: \"kubernetes.io/secret/8b465e38-7dbb-4d02-bce0-28ae0145520c-kube-state-metrics-tls-certs\") pod \"kube-state-metrics-0\" (UID: \"8b465e38-7dbb-4d02-bce0-28ae0145520c\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 11:41:35 crc kubenswrapper[5052]: I1203 11:41:35.497803 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls-certs\" (UniqueName: \"kubernetes.io/secret/8b465e38-7dbb-4d02-bce0-28ae0145520c-kube-state-metrics-tls-certs\") pod \"kube-state-metrics-0\" (UID: \"8b465e38-7dbb-4d02-bce0-28ae0145520c\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 11:41:35 crc kubenswrapper[5052]: I1203 11:41:35.498009 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls-config\" (UniqueName: \"kubernetes.io/secret/8b465e38-7dbb-4d02-bce0-28ae0145520c-kube-state-metrics-tls-config\") pod \"kube-state-metrics-0\" (UID: \"8b465e38-7dbb-4d02-bce0-28ae0145520c\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 11:41:35 crc kubenswrapper[5052]: I1203 11:41:35.498484 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8b465e38-7dbb-4d02-bce0-28ae0145520c-combined-ca-bundle\") pod \"kube-state-metrics-0\" (UID: \"8b465e38-7dbb-4d02-bce0-28ae0145520c\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 11:41:35 crc kubenswrapper[5052]: I1203 11:41:35.522344 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-wwv7d\" (UniqueName: \"kubernetes.io/projected/8b465e38-7dbb-4d02-bce0-28ae0145520c-kube-api-access-wwv7d\") pod \"kube-state-metrics-0\" (UID: \"8b465e38-7dbb-4d02-bce0-28ae0145520c\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 11:41:35 crc kubenswrapper[5052]: I1203 11:41:35.684899 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7535f906-b13f-4b91-b026-bac20ecb4867" path="/var/lib/kubelet/pods/7535f906-b13f-4b91-b026-bac20ecb4867/volumes" Dec 03 11:41:35 crc kubenswrapper[5052]: I1203 11:41:35.692647 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 11:41:36 crc kubenswrapper[5052]: I1203 11:41:36.319876 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/kube-state-metrics-0"] Dec 03 11:41:36 crc kubenswrapper[5052]: I1203 11:41:36.892956 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:41:37 crc kubenswrapper[5052]: I1203 11:41:37.277116 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="d13d0e77-9a6e-4e51-98c4-39828e320c6a" containerName="ceilometer-central-agent" containerID="cri-o://1b0b626fe0bfe23d2e33c4a429245669c17a1a07a4936903e8ddb5ec47f1b434" gracePeriod=30 Dec 03 11:41:37 crc kubenswrapper[5052]: I1203 11:41:37.277220 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="d13d0e77-9a6e-4e51-98c4-39828e320c6a" containerName="ceilometer-notification-agent" containerID="cri-o://2be85ceb7d3cd57a9befa38f17da587b414e83a180cebc28727fc4a5996939d1" gracePeriod=30 Dec 03 11:41:37 crc kubenswrapper[5052]: I1203 11:41:37.277231 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="d13d0e77-9a6e-4e51-98c4-39828e320c6a" containerName="proxy-httpd" containerID="cri-o://018800c14e3ce4d36361a2c7660a63547b7a9c5239ad164f8b420f98bb52cd55" gracePeriod=30 Dec 03 11:41:37 crc kubenswrapper[5052]: I1203 11:41:37.277306 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="d13d0e77-9a6e-4e51-98c4-39828e320c6a" containerName="sg-core" containerID="cri-o://9de6d546e32a6ebadd1f211b6d25ef1622cc49f940869c042b2e0e10bb7d3cb9" gracePeriod=30 Dec 03 11:41:37 crc kubenswrapper[5052]: I1203 11:41:37.277366 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"d13d0e77-9a6e-4e51-98c4-39828e320c6a","Type":"ContainerStarted","Data":"018800c14e3ce4d36361a2c7660a63547b7a9c5239ad164f8b420f98bb52cd55"} Dec 03 11:41:37 crc kubenswrapper[5052]: I1203 11:41:37.277586 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:41:37 crc kubenswrapper[5052]: I1203 11:41:37.283730 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/kube-state-metrics-0" event={"ID":"8b465e38-7dbb-4d02-bce0-28ae0145520c","Type":"ContainerStarted","Data":"cb0825ceeb51bae53da489a08c6c35ac9d9db71eece046983569823ad7e6cd20"} Dec 03 11:41:37 crc kubenswrapper[5052]: I1203 11:41:37.283777 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/kube-state-metrics-0" event={"ID":"8b465e38-7dbb-4d02-bce0-28ae0145520c","Type":"ContainerStarted","Data":"c6fb336a131731e9dc70f681f53cac866c4c33e5f464349113ef876b024ecd59"} Dec 03 11:41:37 crc kubenswrapper[5052]: I1203 11:41:37.284118 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 11:41:37 crc kubenswrapper[5052]: I1203 11:41:37.309458 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/ceilometer-0" podStartSLOduration=2.2373402860000002 podStartE2EDuration="6.309441844s" podCreationTimestamp="2025-12-03 11:41:31 +0000 UTC" firstStartedPulling="2025-12-03 11:41:32.495797742 +0000 UTC m=+1283.084039506" lastFinishedPulling="2025-12-03 11:41:36.5678993 +0000 UTC m=+1287.156141064" observedRunningTime="2025-12-03 11:41:37.30070912 +0000 UTC m=+1287.888950884" watchObservedRunningTime="2025-12-03 11:41:37.309441844 +0000 UTC m=+1287.897683608" Dec 03 11:41:38 crc kubenswrapper[5052]: I1203 11:41:38.294912 5052 generic.go:334] "Generic (PLEG): container finished" podID="d13d0e77-9a6e-4e51-98c4-39828e320c6a" containerID="018800c14e3ce4d36361a2c7660a63547b7a9c5239ad164f8b420f98bb52cd55" exitCode=0 Dec 03 11:41:38 crc kubenswrapper[5052]: I1203 11:41:38.296004 5052 generic.go:334] "Generic (PLEG): container finished" podID="d13d0e77-9a6e-4e51-98c4-39828e320c6a" containerID="9de6d546e32a6ebadd1f211b6d25ef1622cc49f940869c042b2e0e10bb7d3cb9" exitCode=2 Dec 03 11:41:38 crc kubenswrapper[5052]: I1203 11:41:38.296069 5052 generic.go:334] "Generic (PLEG): container finished" podID="d13d0e77-9a6e-4e51-98c4-39828e320c6a" containerID="2be85ceb7d3cd57a9befa38f17da587b414e83a180cebc28727fc4a5996939d1" exitCode=0 Dec 03 11:41:38 crc kubenswrapper[5052]: I1203 11:41:38.297067 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"d13d0e77-9a6e-4e51-98c4-39828e320c6a","Type":"ContainerDied","Data":"018800c14e3ce4d36361a2c7660a63547b7a9c5239ad164f8b420f98bb52cd55"} Dec 03 11:41:38 crc kubenswrapper[5052]: I1203 11:41:38.297176 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"d13d0e77-9a6e-4e51-98c4-39828e320c6a","Type":"ContainerDied","Data":"9de6d546e32a6ebadd1f211b6d25ef1622cc49f940869c042b2e0e10bb7d3cb9"} Dec 03 11:41:38 crc kubenswrapper[5052]: I1203 11:41:38.297297 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"d13d0e77-9a6e-4e51-98c4-39828e320c6a","Type":"ContainerDied","Data":"2be85ceb7d3cd57a9befa38f17da587b414e83a180cebc28727fc4a5996939d1"} Dec 03 11:41:41 crc kubenswrapper[5052]: I1203 11:41:41.580601 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:41:41 crc kubenswrapper[5052]: I1203 11:41:41.581560 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:41:42 crc kubenswrapper[5052]: I1203 11:41:42.331312 5052 generic.go:334] "Generic (PLEG): container finished" podID="c0af0947-fd26-43a5-8b64-bb9bae861cf2" containerID="98a5d3b986b2516d0ac8f36fbeebe27e3364347af72f582505d55d4a22ea7445" exitCode=0 Dec 03 11:41:42 crc kubenswrapper[5052]: I1203 11:41:42.331358 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-cell-mapping-wpd4q" event={"ID":"c0af0947-fd26-43a5-8b64-bb9bae861cf2","Type":"ContainerDied","Data":"98a5d3b986b2516d0ac8f36fbeebe27e3364347af72f582505d55d4a22ea7445"} Dec 03 11:41:42 crc kubenswrapper[5052]: I1203 11:41:42.349488 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/kube-state-metrics-0" podStartSLOduration=6.87476755 podStartE2EDuration="7.349472191s" podCreationTimestamp="2025-12-03 11:41:35 +0000 UTC" firstStartedPulling="2025-12-03 11:41:36.332587834 +0000 UTC m=+1286.920829588" lastFinishedPulling="2025-12-03 11:41:36.807292465 +0000 UTC m=+1287.395534229" observedRunningTime="2025-12-03 11:41:37.321308269 +0000 UTC m=+1287.909550033" watchObservedRunningTime="2025-12-03 11:41:42.349472191 +0000 UTC m=+1292.937713955" Dec 03 11:41:42 crc kubenswrapper[5052]: I1203 11:41:42.593678 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-api-0" podUID="a4d316f2-ae6d-4b1c-b553-bb81b1cb7f61" containerName="nova-api-log" probeResult="failure" output="Get \"https://10.217.0.178:8774/\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Dec 03 11:41:42 crc kubenswrapper[5052]: I1203 11:41:42.593673 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-api-0" podUID="a4d316f2-ae6d-4b1c-b553-bb81b1cb7f61" containerName="nova-api-api" probeResult="failure" output="Get \"https://10.217.0.178:8774/\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Dec 03 11:41:43 crc kubenswrapper[5052]: I1203 11:41:43.345456 5052 generic.go:334] "Generic (PLEG): container finished" podID="d13d0e77-9a6e-4e51-98c4-39828e320c6a" containerID="1b0b626fe0bfe23d2e33c4a429245669c17a1a07a4936903e8ddb5ec47f1b434" exitCode=0 Dec 03 11:41:43 crc kubenswrapper[5052]: I1203 11:41:43.345538 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"d13d0e77-9a6e-4e51-98c4-39828e320c6a","Type":"ContainerDied","Data":"1b0b626fe0bfe23d2e33c4a429245669c17a1a07a4936903e8ddb5ec47f1b434"} Dec 03 11:41:43 crc kubenswrapper[5052]: I1203 11:41:43.345863 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"d13d0e77-9a6e-4e51-98c4-39828e320c6a","Type":"ContainerDied","Data":"636c294322bc7f27ae4cb1193bbeba3606afb3192fd2dc2b169b44566e0d038e"} Dec 03 11:41:43 crc kubenswrapper[5052]: I1203 11:41:43.345878 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="636c294322bc7f27ae4cb1193bbeba3606afb3192fd2dc2b169b44566e0d038e" Dec 03 11:41:43 crc kubenswrapper[5052]: I1203 11:41:43.427306 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:41:43 crc kubenswrapper[5052]: I1203 11:41:43.554034 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/d13d0e77-9a6e-4e51-98c4-39828e320c6a-sg-core-conf-yaml\") pod \"d13d0e77-9a6e-4e51-98c4-39828e320c6a\" (UID: \"d13d0e77-9a6e-4e51-98c4-39828e320c6a\") " Dec 03 11:41:43 crc kubenswrapper[5052]: I1203 11:41:43.554085 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/d13d0e77-9a6e-4e51-98c4-39828e320c6a-config-data\") pod \"d13d0e77-9a6e-4e51-98c4-39828e320c6a\" (UID: \"d13d0e77-9a6e-4e51-98c4-39828e320c6a\") " Dec 03 11:41:43 crc kubenswrapper[5052]: I1203 11:41:43.554142 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/d13d0e77-9a6e-4e51-98c4-39828e320c6a-run-httpd\") pod \"d13d0e77-9a6e-4e51-98c4-39828e320c6a\" (UID: \"d13d0e77-9a6e-4e51-98c4-39828e320c6a\") " Dec 03 11:41:43 crc kubenswrapper[5052]: I1203 11:41:43.554284 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zn8nv\" (UniqueName: \"kubernetes.io/projected/d13d0e77-9a6e-4e51-98c4-39828e320c6a-kube-api-access-zn8nv\") pod \"d13d0e77-9a6e-4e51-98c4-39828e320c6a\" (UID: \"d13d0e77-9a6e-4e51-98c4-39828e320c6a\") " Dec 03 11:41:43 crc kubenswrapper[5052]: I1203 11:41:43.554358 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d13d0e77-9a6e-4e51-98c4-39828e320c6a-combined-ca-bundle\") pod \"d13d0e77-9a6e-4e51-98c4-39828e320c6a\" (UID: \"d13d0e77-9a6e-4e51-98c4-39828e320c6a\") " Dec 03 11:41:43 crc kubenswrapper[5052]: I1203 11:41:43.554451 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/d13d0e77-9a6e-4e51-98c4-39828e320c6a-scripts\") pod \"d13d0e77-9a6e-4e51-98c4-39828e320c6a\" (UID: \"d13d0e77-9a6e-4e51-98c4-39828e320c6a\") " Dec 03 11:41:43 crc kubenswrapper[5052]: I1203 11:41:43.554488 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/d13d0e77-9a6e-4e51-98c4-39828e320c6a-log-httpd\") pod \"d13d0e77-9a6e-4e51-98c4-39828e320c6a\" (UID: \"d13d0e77-9a6e-4e51-98c4-39828e320c6a\") " Dec 03 11:41:43 crc kubenswrapper[5052]: I1203 11:41:43.555860 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d13d0e77-9a6e-4e51-98c4-39828e320c6a-log-httpd" (OuterVolumeSpecName: "log-httpd") pod "d13d0e77-9a6e-4e51-98c4-39828e320c6a" (UID: "d13d0e77-9a6e-4e51-98c4-39828e320c6a"). InnerVolumeSpecName "log-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:41:43 crc kubenswrapper[5052]: I1203 11:41:43.556915 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d13d0e77-9a6e-4e51-98c4-39828e320c6a-run-httpd" (OuterVolumeSpecName: "run-httpd") pod "d13d0e77-9a6e-4e51-98c4-39828e320c6a" (UID: "d13d0e77-9a6e-4e51-98c4-39828e320c6a"). InnerVolumeSpecName "run-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:41:43 crc kubenswrapper[5052]: I1203 11:41:43.562147 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d13d0e77-9a6e-4e51-98c4-39828e320c6a-scripts" (OuterVolumeSpecName: "scripts") pod "d13d0e77-9a6e-4e51-98c4-39828e320c6a" (UID: "d13d0e77-9a6e-4e51-98c4-39828e320c6a"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:41:43 crc kubenswrapper[5052]: I1203 11:41:43.573801 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d13d0e77-9a6e-4e51-98c4-39828e320c6a-kube-api-access-zn8nv" (OuterVolumeSpecName: "kube-api-access-zn8nv") pod "d13d0e77-9a6e-4e51-98c4-39828e320c6a" (UID: "d13d0e77-9a6e-4e51-98c4-39828e320c6a"). InnerVolumeSpecName "kube-api-access-zn8nv". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:41:43 crc kubenswrapper[5052]: I1203 11:41:43.622809 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d13d0e77-9a6e-4e51-98c4-39828e320c6a-sg-core-conf-yaml" (OuterVolumeSpecName: "sg-core-conf-yaml") pod "d13d0e77-9a6e-4e51-98c4-39828e320c6a" (UID: "d13d0e77-9a6e-4e51-98c4-39828e320c6a"). InnerVolumeSpecName "sg-core-conf-yaml". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:41:43 crc kubenswrapper[5052]: I1203 11:41:43.656690 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-zn8nv\" (UniqueName: \"kubernetes.io/projected/d13d0e77-9a6e-4e51-98c4-39828e320c6a-kube-api-access-zn8nv\") on node \"crc\" DevicePath \"\"" Dec 03 11:41:43 crc kubenswrapper[5052]: I1203 11:41:43.656732 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/d13d0e77-9a6e-4e51-98c4-39828e320c6a-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:41:43 crc kubenswrapper[5052]: I1203 11:41:43.656745 5052 reconciler_common.go:293] "Volume detached for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/d13d0e77-9a6e-4e51-98c4-39828e320c6a-log-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 11:41:43 crc kubenswrapper[5052]: I1203 11:41:43.656759 5052 reconciler_common.go:293] "Volume detached for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/d13d0e77-9a6e-4e51-98c4-39828e320c6a-sg-core-conf-yaml\") on node \"crc\" DevicePath \"\"" Dec 03 11:41:43 crc kubenswrapper[5052]: I1203 11:41:43.656771 5052 reconciler_common.go:293] "Volume detached for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/d13d0e77-9a6e-4e51-98c4-39828e320c6a-run-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 11:41:43 crc kubenswrapper[5052]: I1203 11:41:43.672061 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-cell-mapping-wpd4q" Dec 03 11:41:43 crc kubenswrapper[5052]: I1203 11:41:43.691455 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d13d0e77-9a6e-4e51-98c4-39828e320c6a-config-data" (OuterVolumeSpecName: "config-data") pod "d13d0e77-9a6e-4e51-98c4-39828e320c6a" (UID: "d13d0e77-9a6e-4e51-98c4-39828e320c6a"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:41:43 crc kubenswrapper[5052]: I1203 11:41:43.711780 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d13d0e77-9a6e-4e51-98c4-39828e320c6a-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "d13d0e77-9a6e-4e51-98c4-39828e320c6a" (UID: "d13d0e77-9a6e-4e51-98c4-39828e320c6a"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:41:43 crc kubenswrapper[5052]: I1203 11:41:43.758669 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/d13d0e77-9a6e-4e51-98c4-39828e320c6a-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:41:43 crc kubenswrapper[5052]: I1203 11:41:43.758706 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d13d0e77-9a6e-4e51-98c4-39828e320c6a-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:41:43 crc kubenswrapper[5052]: I1203 11:41:43.859634 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/c0af0947-fd26-43a5-8b64-bb9bae861cf2-config-data\") pod \"c0af0947-fd26-43a5-8b64-bb9bae861cf2\" (UID: \"c0af0947-fd26-43a5-8b64-bb9bae861cf2\") " Dec 03 11:41:43 crc kubenswrapper[5052]: I1203 11:41:43.859681 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-drrdl\" (UniqueName: \"kubernetes.io/projected/c0af0947-fd26-43a5-8b64-bb9bae861cf2-kube-api-access-drrdl\") pod \"c0af0947-fd26-43a5-8b64-bb9bae861cf2\" (UID: \"c0af0947-fd26-43a5-8b64-bb9bae861cf2\") " Dec 03 11:41:43 crc kubenswrapper[5052]: I1203 11:41:43.859739 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/c0af0947-fd26-43a5-8b64-bb9bae861cf2-scripts\") pod \"c0af0947-fd26-43a5-8b64-bb9bae861cf2\" (UID: \"c0af0947-fd26-43a5-8b64-bb9bae861cf2\") " Dec 03 11:41:43 crc kubenswrapper[5052]: I1203 11:41:43.859820 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c0af0947-fd26-43a5-8b64-bb9bae861cf2-combined-ca-bundle\") pod \"c0af0947-fd26-43a5-8b64-bb9bae861cf2\" (UID: \"c0af0947-fd26-43a5-8b64-bb9bae861cf2\") " Dec 03 11:41:43 crc kubenswrapper[5052]: I1203 11:41:43.871756 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c0af0947-fd26-43a5-8b64-bb9bae861cf2-scripts" (OuterVolumeSpecName: "scripts") pod "c0af0947-fd26-43a5-8b64-bb9bae861cf2" (UID: "c0af0947-fd26-43a5-8b64-bb9bae861cf2"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:41:43 crc kubenswrapper[5052]: I1203 11:41:43.882830 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c0af0947-fd26-43a5-8b64-bb9bae861cf2-kube-api-access-drrdl" (OuterVolumeSpecName: "kube-api-access-drrdl") pod "c0af0947-fd26-43a5-8b64-bb9bae861cf2" (UID: "c0af0947-fd26-43a5-8b64-bb9bae861cf2"). InnerVolumeSpecName "kube-api-access-drrdl". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:41:43 crc kubenswrapper[5052]: I1203 11:41:43.895762 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c0af0947-fd26-43a5-8b64-bb9bae861cf2-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "c0af0947-fd26-43a5-8b64-bb9bae861cf2" (UID: "c0af0947-fd26-43a5-8b64-bb9bae861cf2"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:41:43 crc kubenswrapper[5052]: I1203 11:41:43.921679 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c0af0947-fd26-43a5-8b64-bb9bae861cf2-config-data" (OuterVolumeSpecName: "config-data") pod "c0af0947-fd26-43a5-8b64-bb9bae861cf2" (UID: "c0af0947-fd26-43a5-8b64-bb9bae861cf2"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:41:43 crc kubenswrapper[5052]: I1203 11:41:43.962018 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/c0af0947-fd26-43a5-8b64-bb9bae861cf2-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:41:43 crc kubenswrapper[5052]: I1203 11:41:43.962059 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-drrdl\" (UniqueName: \"kubernetes.io/projected/c0af0947-fd26-43a5-8b64-bb9bae861cf2-kube-api-access-drrdl\") on node \"crc\" DevicePath \"\"" Dec 03 11:41:43 crc kubenswrapper[5052]: I1203 11:41:43.962072 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/c0af0947-fd26-43a5-8b64-bb9bae861cf2-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:41:43 crc kubenswrapper[5052]: I1203 11:41:43.962080 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c0af0947-fd26-43a5-8b64-bb9bae861cf2-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:41:44 crc kubenswrapper[5052]: I1203 11:41:44.358305 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:41:44 crc kubenswrapper[5052]: I1203 11:41:44.361889 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-cell-mapping-wpd4q" Dec 03 11:41:44 crc kubenswrapper[5052]: I1203 11:41:44.362012 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-cell-mapping-wpd4q" event={"ID":"c0af0947-fd26-43a5-8b64-bb9bae861cf2","Type":"ContainerDied","Data":"b9edc84962678742ba632d212a07e0e6202608075eb8e0bb0c932138c61b44c3"} Dec 03 11:41:44 crc kubenswrapper[5052]: I1203 11:41:44.381794 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b9edc84962678742ba632d212a07e0e6202608075eb8e0bb0c932138c61b44c3" Dec 03 11:41:44 crc kubenswrapper[5052]: I1203 11:41:44.425623 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:41:44 crc kubenswrapper[5052]: I1203 11:41:44.440558 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:41:44 crc kubenswrapper[5052]: I1203 11:41:44.453595 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:41:44 crc kubenswrapper[5052]: E1203 11:41:44.454077 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="d13d0e77-9a6e-4e51-98c4-39828e320c6a" containerName="proxy-httpd" Dec 03 11:41:44 crc kubenswrapper[5052]: I1203 11:41:44.454099 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="d13d0e77-9a6e-4e51-98c4-39828e320c6a" containerName="proxy-httpd" Dec 03 11:41:44 crc kubenswrapper[5052]: E1203 11:41:44.454117 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="d13d0e77-9a6e-4e51-98c4-39828e320c6a" containerName="ceilometer-central-agent" Dec 03 11:41:44 crc kubenswrapper[5052]: I1203 11:41:44.454127 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="d13d0e77-9a6e-4e51-98c4-39828e320c6a" containerName="ceilometer-central-agent" Dec 03 11:41:44 crc kubenswrapper[5052]: E1203 11:41:44.454159 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="c0af0947-fd26-43a5-8b64-bb9bae861cf2" containerName="nova-manage" Dec 03 11:41:44 crc kubenswrapper[5052]: I1203 11:41:44.454167 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="c0af0947-fd26-43a5-8b64-bb9bae861cf2" containerName="nova-manage" Dec 03 11:41:44 crc kubenswrapper[5052]: E1203 11:41:44.454216 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="d13d0e77-9a6e-4e51-98c4-39828e320c6a" containerName="ceilometer-notification-agent" Dec 03 11:41:44 crc kubenswrapper[5052]: I1203 11:41:44.454226 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="d13d0e77-9a6e-4e51-98c4-39828e320c6a" containerName="ceilometer-notification-agent" Dec 03 11:41:44 crc kubenswrapper[5052]: E1203 11:41:44.454243 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="d13d0e77-9a6e-4e51-98c4-39828e320c6a" containerName="sg-core" Dec 03 11:41:44 crc kubenswrapper[5052]: I1203 11:41:44.454252 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="d13d0e77-9a6e-4e51-98c4-39828e320c6a" containerName="sg-core" Dec 03 11:41:44 crc kubenswrapper[5052]: I1203 11:41:44.454503 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="d13d0e77-9a6e-4e51-98c4-39828e320c6a" containerName="proxy-httpd" Dec 03 11:41:44 crc kubenswrapper[5052]: I1203 11:41:44.454552 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="c0af0947-fd26-43a5-8b64-bb9bae861cf2" containerName="nova-manage" Dec 03 11:41:44 crc kubenswrapper[5052]: I1203 11:41:44.454566 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="d13d0e77-9a6e-4e51-98c4-39828e320c6a" containerName="ceilometer-central-agent" Dec 03 11:41:44 crc kubenswrapper[5052]: I1203 11:41:44.454587 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="d13d0e77-9a6e-4e51-98c4-39828e320c6a" containerName="sg-core" Dec 03 11:41:44 crc kubenswrapper[5052]: I1203 11:41:44.454597 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="d13d0e77-9a6e-4e51-98c4-39828e320c6a" containerName="ceilometer-notification-agent" Dec 03 11:41:44 crc kubenswrapper[5052]: I1203 11:41:44.457401 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:41:44 crc kubenswrapper[5052]: I1203 11:41:44.460995 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-scripts" Dec 03 11:41:44 crc kubenswrapper[5052]: I1203 11:41:44.461111 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-ceilometer-internal-svc" Dec 03 11:41:44 crc kubenswrapper[5052]: I1203 11:41:44.461400 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-config-data" Dec 03 11:41:44 crc kubenswrapper[5052]: I1203 11:41:44.482191 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:41:44 crc kubenswrapper[5052]: I1203 11:41:44.533249 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 11:41:44 crc kubenswrapper[5052]: I1203 11:41:44.533456 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-api-0" podUID="a4d316f2-ae6d-4b1c-b553-bb81b1cb7f61" containerName="nova-api-log" containerID="cri-o://10df7a9640ada9b3c5a1aafcbff355b09e089f0926f1993c36cba1a424029fe5" gracePeriod=30 Dec 03 11:41:44 crc kubenswrapper[5052]: I1203 11:41:44.534049 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-api-0" podUID="a4d316f2-ae6d-4b1c-b553-bb81b1cb7f61" containerName="nova-api-api" containerID="cri-o://16d8bd223b40ce74ff57080feb69709f07470b17bf4c54151f36087948f9585e" gracePeriod=30 Dec 03 11:41:44 crc kubenswrapper[5052]: I1203 11:41:44.562801 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 11:41:44 crc kubenswrapper[5052]: I1203 11:41:44.563020 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-scheduler-0" podUID="04dea2ba-82d8-4bab-a03c-74f94e34b760" containerName="nova-scheduler-scheduler" containerID="cri-o://5464a7a42a3745c79d8459b75c25e64208243026ee81b165ab30a3bdba3b2f6f" gracePeriod=30 Dec 03 11:41:44 crc kubenswrapper[5052]: I1203 11:41:44.579418 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/eae5b719-0017-4621-9cb4-73ed6353648e-config-data\") pod \"ceilometer-0\" (UID: \"eae5b719-0017-4621-9cb4-73ed6353648e\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:41:44 crc kubenswrapper[5052]: I1203 11:41:44.579574 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/eae5b719-0017-4621-9cb4-73ed6353648e-run-httpd\") pod \"ceilometer-0\" (UID: \"eae5b719-0017-4621-9cb4-73ed6353648e\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:41:44 crc kubenswrapper[5052]: I1203 11:41:44.579656 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/eae5b719-0017-4621-9cb4-73ed6353648e-log-httpd\") pod \"ceilometer-0\" (UID: \"eae5b719-0017-4621-9cb4-73ed6353648e\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:41:44 crc kubenswrapper[5052]: I1203 11:41:44.579709 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/eae5b719-0017-4621-9cb4-73ed6353648e-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"eae5b719-0017-4621-9cb4-73ed6353648e\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:41:44 crc kubenswrapper[5052]: I1203 11:41:44.579788 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/eae5b719-0017-4621-9cb4-73ed6353648e-ceilometer-tls-certs\") pod \"ceilometer-0\" (UID: \"eae5b719-0017-4621-9cb4-73ed6353648e\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:41:44 crc kubenswrapper[5052]: I1203 11:41:44.579851 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/eae5b719-0017-4621-9cb4-73ed6353648e-scripts\") pod \"ceilometer-0\" (UID: \"eae5b719-0017-4621-9cb4-73ed6353648e\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:41:44 crc kubenswrapper[5052]: I1203 11:41:44.579906 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/eae5b719-0017-4621-9cb4-73ed6353648e-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"eae5b719-0017-4621-9cb4-73ed6353648e\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:41:44 crc kubenswrapper[5052]: I1203 11:41:44.579960 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5hz2d\" (UniqueName: \"kubernetes.io/projected/eae5b719-0017-4621-9cb4-73ed6353648e-kube-api-access-5hz2d\") pod \"ceilometer-0\" (UID: \"eae5b719-0017-4621-9cb4-73ed6353648e\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:41:44 crc kubenswrapper[5052]: I1203 11:41:44.599651 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 11:41:44 crc kubenswrapper[5052]: I1203 11:41:44.600255 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-metadata-0" podUID="77636371-bf0e-4fbd-b4da-d6c789f9421a" containerName="nova-metadata-log" containerID="cri-o://b1e10fd0abad06efc53bc41997abe362f796469d7e56b0403fc0f89c2913b6cd" gracePeriod=30 Dec 03 11:41:44 crc kubenswrapper[5052]: I1203 11:41:44.600386 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-metadata-0" podUID="77636371-bf0e-4fbd-b4da-d6c789f9421a" containerName="nova-metadata-metadata" containerID="cri-o://6f72511fa8fb2eabcc630482287a1ab0f5c038b6f10ade208a2c5f251276187a" gracePeriod=30 Dec 03 11:41:44 crc kubenswrapper[5052]: I1203 11:41:44.681740 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/eae5b719-0017-4621-9cb4-73ed6353648e-ceilometer-tls-certs\") pod \"ceilometer-0\" (UID: \"eae5b719-0017-4621-9cb4-73ed6353648e\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:41:44 crc kubenswrapper[5052]: I1203 11:41:44.681814 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/eae5b719-0017-4621-9cb4-73ed6353648e-scripts\") pod \"ceilometer-0\" (UID: \"eae5b719-0017-4621-9cb4-73ed6353648e\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:41:44 crc kubenswrapper[5052]: I1203 11:41:44.681853 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/eae5b719-0017-4621-9cb4-73ed6353648e-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"eae5b719-0017-4621-9cb4-73ed6353648e\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:41:44 crc kubenswrapper[5052]: I1203 11:41:44.681894 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-5hz2d\" (UniqueName: \"kubernetes.io/projected/eae5b719-0017-4621-9cb4-73ed6353648e-kube-api-access-5hz2d\") pod \"ceilometer-0\" (UID: \"eae5b719-0017-4621-9cb4-73ed6353648e\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:41:44 crc kubenswrapper[5052]: I1203 11:41:44.681936 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/eae5b719-0017-4621-9cb4-73ed6353648e-config-data\") pod \"ceilometer-0\" (UID: \"eae5b719-0017-4621-9cb4-73ed6353648e\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:41:44 crc kubenswrapper[5052]: I1203 11:41:44.681983 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/eae5b719-0017-4621-9cb4-73ed6353648e-run-httpd\") pod \"ceilometer-0\" (UID: \"eae5b719-0017-4621-9cb4-73ed6353648e\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:41:44 crc kubenswrapper[5052]: I1203 11:41:44.682035 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/eae5b719-0017-4621-9cb4-73ed6353648e-log-httpd\") pod \"ceilometer-0\" (UID: \"eae5b719-0017-4621-9cb4-73ed6353648e\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:41:44 crc kubenswrapper[5052]: I1203 11:41:44.682073 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/eae5b719-0017-4621-9cb4-73ed6353648e-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"eae5b719-0017-4621-9cb4-73ed6353648e\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:41:44 crc kubenswrapper[5052]: I1203 11:41:44.682693 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/eae5b719-0017-4621-9cb4-73ed6353648e-run-httpd\") pod \"ceilometer-0\" (UID: \"eae5b719-0017-4621-9cb4-73ed6353648e\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:41:44 crc kubenswrapper[5052]: I1203 11:41:44.682731 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/eae5b719-0017-4621-9cb4-73ed6353648e-log-httpd\") pod \"ceilometer-0\" (UID: \"eae5b719-0017-4621-9cb4-73ed6353648e\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:41:44 crc kubenswrapper[5052]: I1203 11:41:44.686382 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/eae5b719-0017-4621-9cb4-73ed6353648e-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"eae5b719-0017-4621-9cb4-73ed6353648e\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:41:44 crc kubenswrapper[5052]: I1203 11:41:44.687615 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/eae5b719-0017-4621-9cb4-73ed6353648e-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"eae5b719-0017-4621-9cb4-73ed6353648e\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:41:44 crc kubenswrapper[5052]: I1203 11:41:44.688450 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/eae5b719-0017-4621-9cb4-73ed6353648e-config-data\") pod \"ceilometer-0\" (UID: \"eae5b719-0017-4621-9cb4-73ed6353648e\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:41:44 crc kubenswrapper[5052]: I1203 11:41:44.689062 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/eae5b719-0017-4621-9cb4-73ed6353648e-ceilometer-tls-certs\") pod \"ceilometer-0\" (UID: \"eae5b719-0017-4621-9cb4-73ed6353648e\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:41:44 crc kubenswrapper[5052]: I1203 11:41:44.702276 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/eae5b719-0017-4621-9cb4-73ed6353648e-scripts\") pod \"ceilometer-0\" (UID: \"eae5b719-0017-4621-9cb4-73ed6353648e\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:41:44 crc kubenswrapper[5052]: I1203 11:41:44.704425 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-5hz2d\" (UniqueName: \"kubernetes.io/projected/eae5b719-0017-4621-9cb4-73ed6353648e-kube-api-access-5hz2d\") pod \"ceilometer-0\" (UID: \"eae5b719-0017-4621-9cb4-73ed6353648e\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:41:44 crc kubenswrapper[5052]: I1203 11:41:44.783360 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:41:45 crc kubenswrapper[5052]: I1203 11:41:45.228878 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:41:45 crc kubenswrapper[5052]: I1203 11:41:45.369829 5052 generic.go:334] "Generic (PLEG): container finished" podID="a4d316f2-ae6d-4b1c-b553-bb81b1cb7f61" containerID="10df7a9640ada9b3c5a1aafcbff355b09e089f0926f1993c36cba1a424029fe5" exitCode=143 Dec 03 11:41:45 crc kubenswrapper[5052]: I1203 11:41:45.369921 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"a4d316f2-ae6d-4b1c-b553-bb81b1cb7f61","Type":"ContainerDied","Data":"10df7a9640ada9b3c5a1aafcbff355b09e089f0926f1993c36cba1a424029fe5"} Dec 03 11:41:45 crc kubenswrapper[5052]: I1203 11:41:45.372363 5052 generic.go:334] "Generic (PLEG): container finished" podID="77636371-bf0e-4fbd-b4da-d6c789f9421a" containerID="b1e10fd0abad06efc53bc41997abe362f796469d7e56b0403fc0f89c2913b6cd" exitCode=143 Dec 03 11:41:45 crc kubenswrapper[5052]: I1203 11:41:45.372426 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"77636371-bf0e-4fbd-b4da-d6c789f9421a","Type":"ContainerDied","Data":"b1e10fd0abad06efc53bc41997abe362f796469d7e56b0403fc0f89c2913b6cd"} Dec 03 11:41:45 crc kubenswrapper[5052]: I1203 11:41:45.373949 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"eae5b719-0017-4621-9cb4-73ed6353648e","Type":"ContainerStarted","Data":"b038a1489d4fb7dcf01398e95cf717b68ddf1cbe706098dc42c66f25700f0667"} Dec 03 11:41:45 crc kubenswrapper[5052]: I1203 11:41:45.690932 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d13d0e77-9a6e-4e51-98c4-39828e320c6a" path="/var/lib/kubelet/pods/d13d0e77-9a6e-4e51-98c4-39828e320c6a/volumes" Dec 03 11:41:45 crc kubenswrapper[5052]: I1203 11:41:45.706338 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 11:41:46 crc kubenswrapper[5052]: I1203 11:41:46.385159 5052 generic.go:334] "Generic (PLEG): container finished" podID="04dea2ba-82d8-4bab-a03c-74f94e34b760" containerID="5464a7a42a3745c79d8459b75c25e64208243026ee81b165ab30a3bdba3b2f6f" exitCode=0 Dec 03 11:41:46 crc kubenswrapper[5052]: I1203 11:41:46.385494 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"04dea2ba-82d8-4bab-a03c-74f94e34b760","Type":"ContainerDied","Data":"5464a7a42a3745c79d8459b75c25e64208243026ee81b165ab30a3bdba3b2f6f"} Dec 03 11:41:46 crc kubenswrapper[5052]: I1203 11:41:46.386709 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"eae5b719-0017-4621-9cb4-73ed6353648e","Type":"ContainerStarted","Data":"7d22ae141a66b8ccb952b7ed5aa56004799cd8db057a131cc24fa1331ea7e389"} Dec 03 11:41:46 crc kubenswrapper[5052]: I1203 11:41:46.529637 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:41:46 crc kubenswrapper[5052]: I1203 11:41:46.619379 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/04dea2ba-82d8-4bab-a03c-74f94e34b760-config-data\") pod \"04dea2ba-82d8-4bab-a03c-74f94e34b760\" (UID: \"04dea2ba-82d8-4bab-a03c-74f94e34b760\") " Dec 03 11:41:46 crc kubenswrapper[5052]: I1203 11:41:46.619499 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/04dea2ba-82d8-4bab-a03c-74f94e34b760-combined-ca-bundle\") pod \"04dea2ba-82d8-4bab-a03c-74f94e34b760\" (UID: \"04dea2ba-82d8-4bab-a03c-74f94e34b760\") " Dec 03 11:41:46 crc kubenswrapper[5052]: I1203 11:41:46.619719 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qk4zq\" (UniqueName: \"kubernetes.io/projected/04dea2ba-82d8-4bab-a03c-74f94e34b760-kube-api-access-qk4zq\") pod \"04dea2ba-82d8-4bab-a03c-74f94e34b760\" (UID: \"04dea2ba-82d8-4bab-a03c-74f94e34b760\") " Dec 03 11:41:46 crc kubenswrapper[5052]: I1203 11:41:46.624956 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/04dea2ba-82d8-4bab-a03c-74f94e34b760-kube-api-access-qk4zq" (OuterVolumeSpecName: "kube-api-access-qk4zq") pod "04dea2ba-82d8-4bab-a03c-74f94e34b760" (UID: "04dea2ba-82d8-4bab-a03c-74f94e34b760"). InnerVolumeSpecName "kube-api-access-qk4zq". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:41:46 crc kubenswrapper[5052]: I1203 11:41:46.658555 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/04dea2ba-82d8-4bab-a03c-74f94e34b760-config-data" (OuterVolumeSpecName: "config-data") pod "04dea2ba-82d8-4bab-a03c-74f94e34b760" (UID: "04dea2ba-82d8-4bab-a03c-74f94e34b760"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:41:46 crc kubenswrapper[5052]: I1203 11:41:46.709867 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/04dea2ba-82d8-4bab-a03c-74f94e34b760-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "04dea2ba-82d8-4bab-a03c-74f94e34b760" (UID: "04dea2ba-82d8-4bab-a03c-74f94e34b760"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:41:46 crc kubenswrapper[5052]: I1203 11:41:46.722300 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-qk4zq\" (UniqueName: \"kubernetes.io/projected/04dea2ba-82d8-4bab-a03c-74f94e34b760-kube-api-access-qk4zq\") on node \"crc\" DevicePath \"\"" Dec 03 11:41:46 crc kubenswrapper[5052]: I1203 11:41:46.722336 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/04dea2ba-82d8-4bab-a03c-74f94e34b760-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:41:46 crc kubenswrapper[5052]: I1203 11:41:46.722345 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/04dea2ba-82d8-4bab-a03c-74f94e34b760-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:41:47 crc kubenswrapper[5052]: I1203 11:41:47.398215 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"eae5b719-0017-4621-9cb4-73ed6353648e","Type":"ContainerStarted","Data":"0bcbc83b612aeb3a9d9d8d6b5d25c20cfdd3b0b8c6002297d10049035fb2b217"} Dec 03 11:41:47 crc kubenswrapper[5052]: I1203 11:41:47.398264 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"eae5b719-0017-4621-9cb4-73ed6353648e","Type":"ContainerStarted","Data":"7474492a6f12a4bba8719b2ffab92d689c3e76fc1c263f7a31eea5c422579780"} Dec 03 11:41:47 crc kubenswrapper[5052]: I1203 11:41:47.399847 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"04dea2ba-82d8-4bab-a03c-74f94e34b760","Type":"ContainerDied","Data":"8dfd84b11033d329c7b8fc8ade6dd558fcfcfceb2759780298ecdb330cb8f0da"} Dec 03 11:41:47 crc kubenswrapper[5052]: I1203 11:41:47.399881 5052 scope.go:117] "RemoveContainer" containerID="5464a7a42a3745c79d8459b75c25e64208243026ee81b165ab30a3bdba3b2f6f" Dec 03 11:41:47 crc kubenswrapper[5052]: I1203 11:41:47.399906 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:41:47 crc kubenswrapper[5052]: I1203 11:41:47.450026 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 11:41:47 crc kubenswrapper[5052]: I1203 11:41:47.466472 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 11:41:47 crc kubenswrapper[5052]: I1203 11:41:47.484899 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 11:41:47 crc kubenswrapper[5052]: E1203 11:41:47.485359 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="04dea2ba-82d8-4bab-a03c-74f94e34b760" containerName="nova-scheduler-scheduler" Dec 03 11:41:47 crc kubenswrapper[5052]: I1203 11:41:47.485381 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="04dea2ba-82d8-4bab-a03c-74f94e34b760" containerName="nova-scheduler-scheduler" Dec 03 11:41:47 crc kubenswrapper[5052]: I1203 11:41:47.485656 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="04dea2ba-82d8-4bab-a03c-74f94e34b760" containerName="nova-scheduler-scheduler" Dec 03 11:41:47 crc kubenswrapper[5052]: I1203 11:41:47.486382 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:41:47 crc kubenswrapper[5052]: I1203 11:41:47.488674 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-scheduler-config-data" Dec 03 11:41:47 crc kubenswrapper[5052]: I1203 11:41:47.502760 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 11:41:47 crc kubenswrapper[5052]: I1203 11:41:47.644107 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d80d1476-351a-40c6-9f76-ce66d623df74-combined-ca-bundle\") pod \"nova-scheduler-0\" (UID: \"d80d1476-351a-40c6-9f76-ce66d623df74\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:41:47 crc kubenswrapper[5052]: I1203 11:41:47.644563 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cpm9m\" (UniqueName: \"kubernetes.io/projected/d80d1476-351a-40c6-9f76-ce66d623df74-kube-api-access-cpm9m\") pod \"nova-scheduler-0\" (UID: \"d80d1476-351a-40c6-9f76-ce66d623df74\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:41:47 crc kubenswrapper[5052]: I1203 11:41:47.644609 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/d80d1476-351a-40c6-9f76-ce66d623df74-config-data\") pod \"nova-scheduler-0\" (UID: \"d80d1476-351a-40c6-9f76-ce66d623df74\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:41:47 crc kubenswrapper[5052]: I1203 11:41:47.683910 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="04dea2ba-82d8-4bab-a03c-74f94e34b760" path="/var/lib/kubelet/pods/04dea2ba-82d8-4bab-a03c-74f94e34b760/volumes" Dec 03 11:41:47 crc kubenswrapper[5052]: I1203 11:41:47.751936 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-cpm9m\" (UniqueName: \"kubernetes.io/projected/d80d1476-351a-40c6-9f76-ce66d623df74-kube-api-access-cpm9m\") pod \"nova-scheduler-0\" (UID: \"d80d1476-351a-40c6-9f76-ce66d623df74\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:41:47 crc kubenswrapper[5052]: I1203 11:41:47.751985 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/d80d1476-351a-40c6-9f76-ce66d623df74-config-data\") pod \"nova-scheduler-0\" (UID: \"d80d1476-351a-40c6-9f76-ce66d623df74\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:41:47 crc kubenswrapper[5052]: I1203 11:41:47.752031 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d80d1476-351a-40c6-9f76-ce66d623df74-combined-ca-bundle\") pod \"nova-scheduler-0\" (UID: \"d80d1476-351a-40c6-9f76-ce66d623df74\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:41:47 crc kubenswrapper[5052]: I1203 11:41:47.759342 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/d80d1476-351a-40c6-9f76-ce66d623df74-config-data\") pod \"nova-scheduler-0\" (UID: \"d80d1476-351a-40c6-9f76-ce66d623df74\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:41:47 crc kubenswrapper[5052]: I1203 11:41:47.761093 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d80d1476-351a-40c6-9f76-ce66d623df74-combined-ca-bundle\") pod \"nova-scheduler-0\" (UID: \"d80d1476-351a-40c6-9f76-ce66d623df74\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:41:47 crc kubenswrapper[5052]: I1203 11:41:47.770078 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-cpm9m\" (UniqueName: \"kubernetes.io/projected/d80d1476-351a-40c6-9f76-ce66d623df74-kube-api-access-cpm9m\") pod \"nova-scheduler-0\" (UID: \"d80d1476-351a-40c6-9f76-ce66d623df74\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:41:47 crc kubenswrapper[5052]: I1203 11:41:47.829037 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:41:48 crc kubenswrapper[5052]: I1203 11:41:48.092375 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:41:48 crc kubenswrapper[5052]: I1203 11:41:48.264415 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/a4d316f2-ae6d-4b1c-b553-bb81b1cb7f61-internal-tls-certs\") pod \"a4d316f2-ae6d-4b1c-b553-bb81b1cb7f61\" (UID: \"a4d316f2-ae6d-4b1c-b553-bb81b1cb7f61\") " Dec 03 11:41:48 crc kubenswrapper[5052]: I1203 11:41:48.264470 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a4d316f2-ae6d-4b1c-b553-bb81b1cb7f61-combined-ca-bundle\") pod \"a4d316f2-ae6d-4b1c-b553-bb81b1cb7f61\" (UID: \"a4d316f2-ae6d-4b1c-b553-bb81b1cb7f61\") " Dec 03 11:41:48 crc kubenswrapper[5052]: I1203 11:41:48.264681 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6m8n6\" (UniqueName: \"kubernetes.io/projected/a4d316f2-ae6d-4b1c-b553-bb81b1cb7f61-kube-api-access-6m8n6\") pod \"a4d316f2-ae6d-4b1c-b553-bb81b1cb7f61\" (UID: \"a4d316f2-ae6d-4b1c-b553-bb81b1cb7f61\") " Dec 03 11:41:48 crc kubenswrapper[5052]: I1203 11:41:48.264724 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a4d316f2-ae6d-4b1c-b553-bb81b1cb7f61-config-data\") pod \"a4d316f2-ae6d-4b1c-b553-bb81b1cb7f61\" (UID: \"a4d316f2-ae6d-4b1c-b553-bb81b1cb7f61\") " Dec 03 11:41:48 crc kubenswrapper[5052]: I1203 11:41:48.264751 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/a4d316f2-ae6d-4b1c-b553-bb81b1cb7f61-logs\") pod \"a4d316f2-ae6d-4b1c-b553-bb81b1cb7f61\" (UID: \"a4d316f2-ae6d-4b1c-b553-bb81b1cb7f61\") " Dec 03 11:41:48 crc kubenswrapper[5052]: I1203 11:41:48.264770 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/a4d316f2-ae6d-4b1c-b553-bb81b1cb7f61-public-tls-certs\") pod \"a4d316f2-ae6d-4b1c-b553-bb81b1cb7f61\" (UID: \"a4d316f2-ae6d-4b1c-b553-bb81b1cb7f61\") " Dec 03 11:41:48 crc kubenswrapper[5052]: I1203 11:41:48.265385 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4d316f2-ae6d-4b1c-b553-bb81b1cb7f61-logs" (OuterVolumeSpecName: "logs") pod "a4d316f2-ae6d-4b1c-b553-bb81b1cb7f61" (UID: "a4d316f2-ae6d-4b1c-b553-bb81b1cb7f61"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:41:48 crc kubenswrapper[5052]: I1203 11:41:48.269092 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a4d316f2-ae6d-4b1c-b553-bb81b1cb7f61-kube-api-access-6m8n6" (OuterVolumeSpecName: "kube-api-access-6m8n6") pod "a4d316f2-ae6d-4b1c-b553-bb81b1cb7f61" (UID: "a4d316f2-ae6d-4b1c-b553-bb81b1cb7f61"). InnerVolumeSpecName "kube-api-access-6m8n6". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:41:48 crc kubenswrapper[5052]: I1203 11:41:48.294462 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a4d316f2-ae6d-4b1c-b553-bb81b1cb7f61-config-data" (OuterVolumeSpecName: "config-data") pod "a4d316f2-ae6d-4b1c-b553-bb81b1cb7f61" (UID: "a4d316f2-ae6d-4b1c-b553-bb81b1cb7f61"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:41:48 crc kubenswrapper[5052]: I1203 11:41:48.300893 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a4d316f2-ae6d-4b1c-b553-bb81b1cb7f61-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "a4d316f2-ae6d-4b1c-b553-bb81b1cb7f61" (UID: "a4d316f2-ae6d-4b1c-b553-bb81b1cb7f61"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:41:48 crc kubenswrapper[5052]: I1203 11:41:48.322742 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a4d316f2-ae6d-4b1c-b553-bb81b1cb7f61-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "a4d316f2-ae6d-4b1c-b553-bb81b1cb7f61" (UID: "a4d316f2-ae6d-4b1c-b553-bb81b1cb7f61"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:41:48 crc kubenswrapper[5052]: I1203 11:41:48.324027 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a4d316f2-ae6d-4b1c-b553-bb81b1cb7f61-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "a4d316f2-ae6d-4b1c-b553-bb81b1cb7f61" (UID: "a4d316f2-ae6d-4b1c-b553-bb81b1cb7f61"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:41:48 crc kubenswrapper[5052]: I1203 11:41:48.366403 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-6m8n6\" (UniqueName: \"kubernetes.io/projected/a4d316f2-ae6d-4b1c-b553-bb81b1cb7f61-kube-api-access-6m8n6\") on node \"crc\" DevicePath \"\"" Dec 03 11:41:48 crc kubenswrapper[5052]: I1203 11:41:48.366467 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a4d316f2-ae6d-4b1c-b553-bb81b1cb7f61-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:41:48 crc kubenswrapper[5052]: I1203 11:41:48.366479 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/a4d316f2-ae6d-4b1c-b553-bb81b1cb7f61-logs\") on node \"crc\" DevicePath \"\"" Dec 03 11:41:48 crc kubenswrapper[5052]: I1203 11:41:48.366488 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/a4d316f2-ae6d-4b1c-b553-bb81b1cb7f61-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 11:41:48 crc kubenswrapper[5052]: I1203 11:41:48.366496 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/a4d316f2-ae6d-4b1c-b553-bb81b1cb7f61-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 11:41:48 crc kubenswrapper[5052]: I1203 11:41:48.366506 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a4d316f2-ae6d-4b1c-b553-bb81b1cb7f61-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:41:48 crc kubenswrapper[5052]: I1203 11:41:48.396366 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:41:48 crc kubenswrapper[5052]: I1203 11:41:48.437066 5052 generic.go:334] "Generic (PLEG): container finished" podID="a4d316f2-ae6d-4b1c-b553-bb81b1cb7f61" containerID="16d8bd223b40ce74ff57080feb69709f07470b17bf4c54151f36087948f9585e" exitCode=0 Dec 03 11:41:48 crc kubenswrapper[5052]: I1203 11:41:48.437148 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"a4d316f2-ae6d-4b1c-b553-bb81b1cb7f61","Type":"ContainerDied","Data":"16d8bd223b40ce74ff57080feb69709f07470b17bf4c54151f36087948f9585e"} Dec 03 11:41:48 crc kubenswrapper[5052]: I1203 11:41:48.437174 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"a4d316f2-ae6d-4b1c-b553-bb81b1cb7f61","Type":"ContainerDied","Data":"b76c70ca5e0b575b822e5b4012bed974504cc893d54ceab8b9d02f2bbece9af4"} Dec 03 11:41:48 crc kubenswrapper[5052]: I1203 11:41:48.437190 5052 scope.go:117] "RemoveContainer" containerID="16d8bd223b40ce74ff57080feb69709f07470b17bf4c54151f36087948f9585e" Dec 03 11:41:48 crc kubenswrapper[5052]: I1203 11:41:48.437339 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:41:48 crc kubenswrapper[5052]: I1203 11:41:48.446339 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 11:41:48 crc kubenswrapper[5052]: I1203 11:41:48.457205 5052 generic.go:334] "Generic (PLEG): container finished" podID="77636371-bf0e-4fbd-b4da-d6c789f9421a" containerID="6f72511fa8fb2eabcc630482287a1ab0f5c038b6f10ade208a2c5f251276187a" exitCode=0 Dec 03 11:41:48 crc kubenswrapper[5052]: I1203 11:41:48.457305 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"77636371-bf0e-4fbd-b4da-d6c789f9421a","Type":"ContainerDied","Data":"6f72511fa8fb2eabcc630482287a1ab0f5c038b6f10ade208a2c5f251276187a"} Dec 03 11:41:48 crc kubenswrapper[5052]: I1203 11:41:48.457336 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"77636371-bf0e-4fbd-b4da-d6c789f9421a","Type":"ContainerDied","Data":"7284075c35ea1d02d4b745432658994f06cd82bedb5aafffb336c033b8bf09f5"} Dec 03 11:41:48 crc kubenswrapper[5052]: I1203 11:41:48.457431 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:41:48 crc kubenswrapper[5052]: I1203 11:41:48.502663 5052 scope.go:117] "RemoveContainer" containerID="10df7a9640ada9b3c5a1aafcbff355b09e089f0926f1993c36cba1a424029fe5" Dec 03 11:41:48 crc kubenswrapper[5052]: I1203 11:41:48.528500 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 11:41:48 crc kubenswrapper[5052]: I1203 11:41:48.537259 5052 scope.go:117] "RemoveContainer" containerID="16d8bd223b40ce74ff57080feb69709f07470b17bf4c54151f36087948f9585e" Dec 03 11:41:48 crc kubenswrapper[5052]: E1203 11:41:48.539381 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"16d8bd223b40ce74ff57080feb69709f07470b17bf4c54151f36087948f9585e\": container with ID starting with 16d8bd223b40ce74ff57080feb69709f07470b17bf4c54151f36087948f9585e not found: ID does not exist" containerID="16d8bd223b40ce74ff57080feb69709f07470b17bf4c54151f36087948f9585e" Dec 03 11:41:48 crc kubenswrapper[5052]: I1203 11:41:48.539422 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"16d8bd223b40ce74ff57080feb69709f07470b17bf4c54151f36087948f9585e"} err="failed to get container status \"16d8bd223b40ce74ff57080feb69709f07470b17bf4c54151f36087948f9585e\": rpc error: code = NotFound desc = could not find container \"16d8bd223b40ce74ff57080feb69709f07470b17bf4c54151f36087948f9585e\": container with ID starting with 16d8bd223b40ce74ff57080feb69709f07470b17bf4c54151f36087948f9585e not found: ID does not exist" Dec 03 11:41:48 crc kubenswrapper[5052]: I1203 11:41:48.539444 5052 scope.go:117] "RemoveContainer" containerID="10df7a9640ada9b3c5a1aafcbff355b09e089f0926f1993c36cba1a424029fe5" Dec 03 11:41:48 crc kubenswrapper[5052]: E1203 11:41:48.540120 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"10df7a9640ada9b3c5a1aafcbff355b09e089f0926f1993c36cba1a424029fe5\": container with ID starting with 10df7a9640ada9b3c5a1aafcbff355b09e089f0926f1993c36cba1a424029fe5 not found: ID does not exist" containerID="10df7a9640ada9b3c5a1aafcbff355b09e089f0926f1993c36cba1a424029fe5" Dec 03 11:41:48 crc kubenswrapper[5052]: I1203 11:41:48.540139 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"10df7a9640ada9b3c5a1aafcbff355b09e089f0926f1993c36cba1a424029fe5"} err="failed to get container status \"10df7a9640ada9b3c5a1aafcbff355b09e089f0926f1993c36cba1a424029fe5\": rpc error: code = NotFound desc = could not find container \"10df7a9640ada9b3c5a1aafcbff355b09e089f0926f1993c36cba1a424029fe5\": container with ID starting with 10df7a9640ada9b3c5a1aafcbff355b09e089f0926f1993c36cba1a424029fe5 not found: ID does not exist" Dec 03 11:41:48 crc kubenswrapper[5052]: I1203 11:41:48.540158 5052 scope.go:117] "RemoveContainer" containerID="6f72511fa8fb2eabcc630482287a1ab0f5c038b6f10ade208a2c5f251276187a" Dec 03 11:41:48 crc kubenswrapper[5052]: I1203 11:41:48.543977 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 11:41:48 crc kubenswrapper[5052]: I1203 11:41:48.558639 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 11:41:48 crc kubenswrapper[5052]: E1203 11:41:48.559182 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="77636371-bf0e-4fbd-b4da-d6c789f9421a" containerName="nova-metadata-metadata" Dec 03 11:41:48 crc kubenswrapper[5052]: I1203 11:41:48.559213 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="77636371-bf0e-4fbd-b4da-d6c789f9421a" containerName="nova-metadata-metadata" Dec 03 11:41:48 crc kubenswrapper[5052]: E1203 11:41:48.559248 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="77636371-bf0e-4fbd-b4da-d6c789f9421a" containerName="nova-metadata-log" Dec 03 11:41:48 crc kubenswrapper[5052]: I1203 11:41:48.559259 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="77636371-bf0e-4fbd-b4da-d6c789f9421a" containerName="nova-metadata-log" Dec 03 11:41:48 crc kubenswrapper[5052]: E1203 11:41:48.559275 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="a4d316f2-ae6d-4b1c-b553-bb81b1cb7f61" containerName="nova-api-log" Dec 03 11:41:48 crc kubenswrapper[5052]: I1203 11:41:48.559284 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="a4d316f2-ae6d-4b1c-b553-bb81b1cb7f61" containerName="nova-api-log" Dec 03 11:41:48 crc kubenswrapper[5052]: E1203 11:41:48.559294 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="a4d316f2-ae6d-4b1c-b553-bb81b1cb7f61" containerName="nova-api-api" Dec 03 11:41:48 crc kubenswrapper[5052]: I1203 11:41:48.559302 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="a4d316f2-ae6d-4b1c-b553-bb81b1cb7f61" containerName="nova-api-api" Dec 03 11:41:48 crc kubenswrapper[5052]: I1203 11:41:48.559567 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="a4d316f2-ae6d-4b1c-b553-bb81b1cb7f61" containerName="nova-api-api" Dec 03 11:41:48 crc kubenswrapper[5052]: I1203 11:41:48.559590 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="a4d316f2-ae6d-4b1c-b553-bb81b1cb7f61" containerName="nova-api-log" Dec 03 11:41:48 crc kubenswrapper[5052]: I1203 11:41:48.559605 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="77636371-bf0e-4fbd-b4da-d6c789f9421a" containerName="nova-metadata-metadata" Dec 03 11:41:48 crc kubenswrapper[5052]: I1203 11:41:48.559624 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="77636371-bf0e-4fbd-b4da-d6c789f9421a" containerName="nova-metadata-log" Dec 03 11:41:48 crc kubenswrapper[5052]: I1203 11:41:48.565050 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:41:48 crc kubenswrapper[5052]: I1203 11:41:48.567502 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 11:41:48 crc kubenswrapper[5052]: I1203 11:41:48.569068 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-nova-public-svc" Dec 03 11:41:48 crc kubenswrapper[5052]: I1203 11:41:48.569262 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-nova-internal-svc" Dec 03 11:41:48 crc kubenswrapper[5052]: I1203 11:41:48.569480 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-api-config-data" Dec 03 11:41:48 crc kubenswrapper[5052]: I1203 11:41:48.571052 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/77636371-bf0e-4fbd-b4da-d6c789f9421a-combined-ca-bundle\") pod \"77636371-bf0e-4fbd-b4da-d6c789f9421a\" (UID: \"77636371-bf0e-4fbd-b4da-d6c789f9421a\") " Dec 03 11:41:48 crc kubenswrapper[5052]: I1203 11:41:48.571188 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/77636371-bf0e-4fbd-b4da-d6c789f9421a-nova-metadata-tls-certs\") pod \"77636371-bf0e-4fbd-b4da-d6c789f9421a\" (UID: \"77636371-bf0e-4fbd-b4da-d6c789f9421a\") " Dec 03 11:41:48 crc kubenswrapper[5052]: I1203 11:41:48.571259 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-84ns4\" (UniqueName: \"kubernetes.io/projected/77636371-bf0e-4fbd-b4da-d6c789f9421a-kube-api-access-84ns4\") pod \"77636371-bf0e-4fbd-b4da-d6c789f9421a\" (UID: \"77636371-bf0e-4fbd-b4da-d6c789f9421a\") " Dec 03 11:41:48 crc kubenswrapper[5052]: I1203 11:41:48.571393 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/77636371-bf0e-4fbd-b4da-d6c789f9421a-logs\") pod \"77636371-bf0e-4fbd-b4da-d6c789f9421a\" (UID: \"77636371-bf0e-4fbd-b4da-d6c789f9421a\") " Dec 03 11:41:48 crc kubenswrapper[5052]: I1203 11:41:48.571431 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/77636371-bf0e-4fbd-b4da-d6c789f9421a-config-data\") pod \"77636371-bf0e-4fbd-b4da-d6c789f9421a\" (UID: \"77636371-bf0e-4fbd-b4da-d6c789f9421a\") " Dec 03 11:41:48 crc kubenswrapper[5052]: I1203 11:41:48.576339 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/77636371-bf0e-4fbd-b4da-d6c789f9421a-kube-api-access-84ns4" (OuterVolumeSpecName: "kube-api-access-84ns4") pod "77636371-bf0e-4fbd-b4da-d6c789f9421a" (UID: "77636371-bf0e-4fbd-b4da-d6c789f9421a"). InnerVolumeSpecName "kube-api-access-84ns4". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:41:48 crc kubenswrapper[5052]: I1203 11:41:48.576435 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77636371-bf0e-4fbd-b4da-d6c789f9421a-logs" (OuterVolumeSpecName: "logs") pod "77636371-bf0e-4fbd-b4da-d6c789f9421a" (UID: "77636371-bf0e-4fbd-b4da-d6c789f9421a"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:41:48 crc kubenswrapper[5052]: I1203 11:41:48.589934 5052 scope.go:117] "RemoveContainer" containerID="b1e10fd0abad06efc53bc41997abe362f796469d7e56b0403fc0f89c2913b6cd" Dec 03 11:41:48 crc kubenswrapper[5052]: I1203 11:41:48.615313 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/77636371-bf0e-4fbd-b4da-d6c789f9421a-config-data" (OuterVolumeSpecName: "config-data") pod "77636371-bf0e-4fbd-b4da-d6c789f9421a" (UID: "77636371-bf0e-4fbd-b4da-d6c789f9421a"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:41:48 crc kubenswrapper[5052]: I1203 11:41:48.621663 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/77636371-bf0e-4fbd-b4da-d6c789f9421a-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "77636371-bf0e-4fbd-b4da-d6c789f9421a" (UID: "77636371-bf0e-4fbd-b4da-d6c789f9421a"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:41:48 crc kubenswrapper[5052]: I1203 11:41:48.632042 5052 scope.go:117] "RemoveContainer" containerID="6f72511fa8fb2eabcc630482287a1ab0f5c038b6f10ade208a2c5f251276187a" Dec 03 11:41:48 crc kubenswrapper[5052]: E1203 11:41:48.637142 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6f72511fa8fb2eabcc630482287a1ab0f5c038b6f10ade208a2c5f251276187a\": container with ID starting with 6f72511fa8fb2eabcc630482287a1ab0f5c038b6f10ade208a2c5f251276187a not found: ID does not exist" containerID="6f72511fa8fb2eabcc630482287a1ab0f5c038b6f10ade208a2c5f251276187a" Dec 03 11:41:48 crc kubenswrapper[5052]: I1203 11:41:48.637196 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f72511fa8fb2eabcc630482287a1ab0f5c038b6f10ade208a2c5f251276187a"} err="failed to get container status \"6f72511fa8fb2eabcc630482287a1ab0f5c038b6f10ade208a2c5f251276187a\": rpc error: code = NotFound desc = could not find container \"6f72511fa8fb2eabcc630482287a1ab0f5c038b6f10ade208a2c5f251276187a\": container with ID starting with 6f72511fa8fb2eabcc630482287a1ab0f5c038b6f10ade208a2c5f251276187a not found: ID does not exist" Dec 03 11:41:48 crc kubenswrapper[5052]: I1203 11:41:48.637229 5052 scope.go:117] "RemoveContainer" containerID="b1e10fd0abad06efc53bc41997abe362f796469d7e56b0403fc0f89c2913b6cd" Dec 03 11:41:48 crc kubenswrapper[5052]: E1203 11:41:48.638041 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b1e10fd0abad06efc53bc41997abe362f796469d7e56b0403fc0f89c2913b6cd\": container with ID starting with b1e10fd0abad06efc53bc41997abe362f796469d7e56b0403fc0f89c2913b6cd not found: ID does not exist" containerID="b1e10fd0abad06efc53bc41997abe362f796469d7e56b0403fc0f89c2913b6cd" Dec 03 11:41:48 crc kubenswrapper[5052]: I1203 11:41:48.638069 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b1e10fd0abad06efc53bc41997abe362f796469d7e56b0403fc0f89c2913b6cd"} err="failed to get container status \"b1e10fd0abad06efc53bc41997abe362f796469d7e56b0403fc0f89c2913b6cd\": rpc error: code = NotFound desc = could not find container \"b1e10fd0abad06efc53bc41997abe362f796469d7e56b0403fc0f89c2913b6cd\": container with ID starting with b1e10fd0abad06efc53bc41997abe362f796469d7e56b0403fc0f89c2913b6cd not found: ID does not exist" Dec 03 11:41:48 crc kubenswrapper[5052]: I1203 11:41:48.653162 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/77636371-bf0e-4fbd-b4da-d6c789f9421a-nova-metadata-tls-certs" (OuterVolumeSpecName: "nova-metadata-tls-certs") pod "77636371-bf0e-4fbd-b4da-d6c789f9421a" (UID: "77636371-bf0e-4fbd-b4da-d6c789f9421a"). InnerVolumeSpecName "nova-metadata-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:41:48 crc kubenswrapper[5052]: I1203 11:41:48.677655 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/82dce330-8f65-4c14-80d4-958eb8ca5212-logs\") pod \"nova-api-0\" (UID: \"82dce330-8f65-4c14-80d4-958eb8ca5212\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:41:48 crc kubenswrapper[5052]: I1203 11:41:48.677714 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/82dce330-8f65-4c14-80d4-958eb8ca5212-internal-tls-certs\") pod \"nova-api-0\" (UID: \"82dce330-8f65-4c14-80d4-958eb8ca5212\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:41:48 crc kubenswrapper[5052]: I1203 11:41:48.677809 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/82dce330-8f65-4c14-80d4-958eb8ca5212-config-data\") pod \"nova-api-0\" (UID: \"82dce330-8f65-4c14-80d4-958eb8ca5212\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:41:48 crc kubenswrapper[5052]: I1203 11:41:48.677916 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/82dce330-8f65-4c14-80d4-958eb8ca5212-public-tls-certs\") pod \"nova-api-0\" (UID: \"82dce330-8f65-4c14-80d4-958eb8ca5212\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:41:48 crc kubenswrapper[5052]: I1203 11:41:48.677961 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hczfd\" (UniqueName: \"kubernetes.io/projected/82dce330-8f65-4c14-80d4-958eb8ca5212-kube-api-access-hczfd\") pod \"nova-api-0\" (UID: \"82dce330-8f65-4c14-80d4-958eb8ca5212\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:41:48 crc kubenswrapper[5052]: I1203 11:41:48.678021 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/82dce330-8f65-4c14-80d4-958eb8ca5212-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"82dce330-8f65-4c14-80d4-958eb8ca5212\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:41:48 crc kubenswrapper[5052]: I1203 11:41:48.678188 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/77636371-bf0e-4fbd-b4da-d6c789f9421a-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:41:48 crc kubenswrapper[5052]: I1203 11:41:48.678216 5052 reconciler_common.go:293] "Volume detached for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/77636371-bf0e-4fbd-b4da-d6c789f9421a-nova-metadata-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 11:41:48 crc kubenswrapper[5052]: I1203 11:41:48.678229 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-84ns4\" (UniqueName: \"kubernetes.io/projected/77636371-bf0e-4fbd-b4da-d6c789f9421a-kube-api-access-84ns4\") on node \"crc\" DevicePath \"\"" Dec 03 11:41:48 crc kubenswrapper[5052]: I1203 11:41:48.678241 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/77636371-bf0e-4fbd-b4da-d6c789f9421a-logs\") on node \"crc\" DevicePath \"\"" Dec 03 11:41:48 crc kubenswrapper[5052]: I1203 11:41:48.678252 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/77636371-bf0e-4fbd-b4da-d6c789f9421a-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:41:48 crc kubenswrapper[5052]: I1203 11:41:48.779585 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/82dce330-8f65-4c14-80d4-958eb8ca5212-logs\") pod \"nova-api-0\" (UID: \"82dce330-8f65-4c14-80d4-958eb8ca5212\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:41:48 crc kubenswrapper[5052]: I1203 11:41:48.779626 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/82dce330-8f65-4c14-80d4-958eb8ca5212-internal-tls-certs\") pod \"nova-api-0\" (UID: \"82dce330-8f65-4c14-80d4-958eb8ca5212\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:41:48 crc kubenswrapper[5052]: I1203 11:41:48.779667 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/82dce330-8f65-4c14-80d4-958eb8ca5212-config-data\") pod \"nova-api-0\" (UID: \"82dce330-8f65-4c14-80d4-958eb8ca5212\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:41:48 crc kubenswrapper[5052]: I1203 11:41:48.779710 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/82dce330-8f65-4c14-80d4-958eb8ca5212-public-tls-certs\") pod \"nova-api-0\" (UID: \"82dce330-8f65-4c14-80d4-958eb8ca5212\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:41:48 crc kubenswrapper[5052]: I1203 11:41:48.779731 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-hczfd\" (UniqueName: \"kubernetes.io/projected/82dce330-8f65-4c14-80d4-958eb8ca5212-kube-api-access-hczfd\") pod \"nova-api-0\" (UID: \"82dce330-8f65-4c14-80d4-958eb8ca5212\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:41:48 crc kubenswrapper[5052]: I1203 11:41:48.779753 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/82dce330-8f65-4c14-80d4-958eb8ca5212-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"82dce330-8f65-4c14-80d4-958eb8ca5212\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:41:48 crc kubenswrapper[5052]: I1203 11:41:48.781166 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/82dce330-8f65-4c14-80d4-958eb8ca5212-logs\") pod \"nova-api-0\" (UID: \"82dce330-8f65-4c14-80d4-958eb8ca5212\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:41:48 crc kubenswrapper[5052]: I1203 11:41:48.783316 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/82dce330-8f65-4c14-80d4-958eb8ca5212-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"82dce330-8f65-4c14-80d4-958eb8ca5212\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:41:48 crc kubenswrapper[5052]: I1203 11:41:48.786134 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/82dce330-8f65-4c14-80d4-958eb8ca5212-internal-tls-certs\") pod \"nova-api-0\" (UID: \"82dce330-8f65-4c14-80d4-958eb8ca5212\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:41:48 crc kubenswrapper[5052]: I1203 11:41:48.786754 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/82dce330-8f65-4c14-80d4-958eb8ca5212-public-tls-certs\") pod \"nova-api-0\" (UID: \"82dce330-8f65-4c14-80d4-958eb8ca5212\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:41:48 crc kubenswrapper[5052]: I1203 11:41:48.787733 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/82dce330-8f65-4c14-80d4-958eb8ca5212-config-data\") pod \"nova-api-0\" (UID: \"82dce330-8f65-4c14-80d4-958eb8ca5212\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:41:48 crc kubenswrapper[5052]: I1203 11:41:48.800895 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 11:41:48 crc kubenswrapper[5052]: I1203 11:41:48.807734 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-hczfd\" (UniqueName: \"kubernetes.io/projected/82dce330-8f65-4c14-80d4-958eb8ca5212-kube-api-access-hczfd\") pod \"nova-api-0\" (UID: \"82dce330-8f65-4c14-80d4-958eb8ca5212\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:41:48 crc kubenswrapper[5052]: I1203 11:41:48.811255 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 11:41:48 crc kubenswrapper[5052]: I1203 11:41:48.826080 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 11:41:48 crc kubenswrapper[5052]: I1203 11:41:48.828024 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:41:48 crc kubenswrapper[5052]: I1203 11:41:48.833454 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-nova-metadata-internal-svc" Dec 03 11:41:48 crc kubenswrapper[5052]: I1203 11:41:48.833724 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-metadata-config-data" Dec 03 11:41:48 crc kubenswrapper[5052]: I1203 11:41:48.835139 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 11:41:48 crc kubenswrapper[5052]: I1203 11:41:48.885107 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:41:48 crc kubenswrapper[5052]: I1203 11:41:48.982241 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/f98f227a-397b-4665-bf18-e00b105ce058-logs\") pod \"nova-metadata-0\" (UID: \"f98f227a-397b-4665-bf18-e00b105ce058\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:41:48 crc kubenswrapper[5052]: I1203 11:41:48.982450 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hxcnf\" (UniqueName: \"kubernetes.io/projected/f98f227a-397b-4665-bf18-e00b105ce058-kube-api-access-hxcnf\") pod \"nova-metadata-0\" (UID: \"f98f227a-397b-4665-bf18-e00b105ce058\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:41:48 crc kubenswrapper[5052]: I1203 11:41:48.982520 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/f98f227a-397b-4665-bf18-e00b105ce058-config-data\") pod \"nova-metadata-0\" (UID: \"f98f227a-397b-4665-bf18-e00b105ce058\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:41:48 crc kubenswrapper[5052]: I1203 11:41:48.982576 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f98f227a-397b-4665-bf18-e00b105ce058-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"f98f227a-397b-4665-bf18-e00b105ce058\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:41:48 crc kubenswrapper[5052]: I1203 11:41:48.982605 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/f98f227a-397b-4665-bf18-e00b105ce058-nova-metadata-tls-certs\") pod \"nova-metadata-0\" (UID: \"f98f227a-397b-4665-bf18-e00b105ce058\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:41:49 crc kubenswrapper[5052]: I1203 11:41:49.090446 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f98f227a-397b-4665-bf18-e00b105ce058-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"f98f227a-397b-4665-bf18-e00b105ce058\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:41:49 crc kubenswrapper[5052]: I1203 11:41:49.090496 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/f98f227a-397b-4665-bf18-e00b105ce058-nova-metadata-tls-certs\") pod \"nova-metadata-0\" (UID: \"f98f227a-397b-4665-bf18-e00b105ce058\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:41:49 crc kubenswrapper[5052]: I1203 11:41:49.090557 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/f98f227a-397b-4665-bf18-e00b105ce058-logs\") pod \"nova-metadata-0\" (UID: \"f98f227a-397b-4665-bf18-e00b105ce058\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:41:49 crc kubenswrapper[5052]: I1203 11:41:49.090600 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-hxcnf\" (UniqueName: \"kubernetes.io/projected/f98f227a-397b-4665-bf18-e00b105ce058-kube-api-access-hxcnf\") pod \"nova-metadata-0\" (UID: \"f98f227a-397b-4665-bf18-e00b105ce058\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:41:49 crc kubenswrapper[5052]: I1203 11:41:49.090648 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/f98f227a-397b-4665-bf18-e00b105ce058-config-data\") pod \"nova-metadata-0\" (UID: \"f98f227a-397b-4665-bf18-e00b105ce058\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:41:49 crc kubenswrapper[5052]: I1203 11:41:49.095741 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/f98f227a-397b-4665-bf18-e00b105ce058-logs\") pod \"nova-metadata-0\" (UID: \"f98f227a-397b-4665-bf18-e00b105ce058\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:41:49 crc kubenswrapper[5052]: I1203 11:41:49.102860 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/f98f227a-397b-4665-bf18-e00b105ce058-nova-metadata-tls-certs\") pod \"nova-metadata-0\" (UID: \"f98f227a-397b-4665-bf18-e00b105ce058\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:41:49 crc kubenswrapper[5052]: I1203 11:41:49.105402 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/f98f227a-397b-4665-bf18-e00b105ce058-config-data\") pod \"nova-metadata-0\" (UID: \"f98f227a-397b-4665-bf18-e00b105ce058\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:41:49 crc kubenswrapper[5052]: I1203 11:41:49.115019 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-hxcnf\" (UniqueName: \"kubernetes.io/projected/f98f227a-397b-4665-bf18-e00b105ce058-kube-api-access-hxcnf\") pod \"nova-metadata-0\" (UID: \"f98f227a-397b-4665-bf18-e00b105ce058\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:41:49 crc kubenswrapper[5052]: I1203 11:41:49.122436 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f98f227a-397b-4665-bf18-e00b105ce058-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"f98f227a-397b-4665-bf18-e00b105ce058\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:41:49 crc kubenswrapper[5052]: I1203 11:41:49.178250 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:41:49 crc kubenswrapper[5052]: I1203 11:41:49.398944 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 11:41:49 crc kubenswrapper[5052]: I1203 11:41:49.475880 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"82dce330-8f65-4c14-80d4-958eb8ca5212","Type":"ContainerStarted","Data":"a925b2a38721f9f89df454d11e7db79c799898fc4b17f0e225ad34b672efb017"} Dec 03 11:41:49 crc kubenswrapper[5052]: I1203 11:41:49.477798 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"d80d1476-351a-40c6-9f76-ce66d623df74","Type":"ContainerStarted","Data":"2acdb0814bfda1b6c48a4fde354fd532f86ae71dcaac188c16bee44a794b3713"} Dec 03 11:41:49 crc kubenswrapper[5052]: I1203 11:41:49.477819 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"d80d1476-351a-40c6-9f76-ce66d623df74","Type":"ContainerStarted","Data":"8ecda4d41d18206a128e089bbc4ef5a55fa2288abf2a12148734dafbc276a6bb"} Dec 03 11:41:49 crc kubenswrapper[5052]: I1203 11:41:49.504536 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"eae5b719-0017-4621-9cb4-73ed6353648e","Type":"ContainerStarted","Data":"aa0bc44913bc443884760e2a18faf9da4a9e4a859e8c121cd05e1cd62282505e"} Dec 03 11:41:49 crc kubenswrapper[5052]: I1203 11:41:49.504737 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:41:49 crc kubenswrapper[5052]: I1203 11:41:49.537905 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-scheduler-0" podStartSLOduration=2.5378839109999998 podStartE2EDuration="2.537883911s" podCreationTimestamp="2025-12-03 11:41:47 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:41:49.498192367 +0000 UTC m=+1300.086434131" watchObservedRunningTime="2025-12-03 11:41:49.537883911 +0000 UTC m=+1300.126125675" Dec 03 11:41:49 crc kubenswrapper[5052]: I1203 11:41:49.540318 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/ceilometer-0" podStartSLOduration=2.238691148 podStartE2EDuration="5.540308132s" podCreationTimestamp="2025-12-03 11:41:44 +0000 UTC" firstStartedPulling="2025-12-03 11:41:45.236542594 +0000 UTC m=+1295.824784358" lastFinishedPulling="2025-12-03 11:41:48.538159578 +0000 UTC m=+1299.126401342" observedRunningTime="2025-12-03 11:41:49.53025785 +0000 UTC m=+1300.118499634" watchObservedRunningTime="2025-12-03 11:41:49.540308132 +0000 UTC m=+1300.128549896" Dec 03 11:41:49 crc kubenswrapper[5052]: I1203 11:41:49.684392 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="77636371-bf0e-4fbd-b4da-d6c789f9421a" path="/var/lib/kubelet/pods/77636371-bf0e-4fbd-b4da-d6c789f9421a/volumes" Dec 03 11:41:49 crc kubenswrapper[5052]: I1203 11:41:49.685462 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a4d316f2-ae6d-4b1c-b553-bb81b1cb7f61" path="/var/lib/kubelet/pods/a4d316f2-ae6d-4b1c-b553-bb81b1cb7f61/volumes" Dec 03 11:41:49 crc kubenswrapper[5052]: I1203 11:41:49.711188 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 11:41:50 crc kubenswrapper[5052]: I1203 11:41:50.536603 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"f98f227a-397b-4665-bf18-e00b105ce058","Type":"ContainerStarted","Data":"b2697ef9ab442e1740e9992a4ae8a2ad183a400760ac5eea2d853dc944ed7fa8"} Dec 03 11:41:50 crc kubenswrapper[5052]: I1203 11:41:50.536928 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"f98f227a-397b-4665-bf18-e00b105ce058","Type":"ContainerStarted","Data":"9a73f69a43329d67c0d372934e250290b21f5530d0b0079f8192b4709695cb6a"} Dec 03 11:41:50 crc kubenswrapper[5052]: I1203 11:41:50.536937 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"f98f227a-397b-4665-bf18-e00b105ce058","Type":"ContainerStarted","Data":"114a172183480c19154e1e08e67868f0385b850efd333a1ba31774310890d146"} Dec 03 11:41:50 crc kubenswrapper[5052]: I1203 11:41:50.541668 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"82dce330-8f65-4c14-80d4-958eb8ca5212","Type":"ContainerStarted","Data":"d6cb51da86b5f965cccc6e7f9d1241d74c446a9276dbf42e057cec9147384cb3"} Dec 03 11:41:50 crc kubenswrapper[5052]: I1203 11:41:50.541715 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"82dce330-8f65-4c14-80d4-958eb8ca5212","Type":"ContainerStarted","Data":"b939d536bc53d88cf0c53f3e90f1a9bbe204dc2f74470b14127832658e7f3898"} Dec 03 11:41:50 crc kubenswrapper[5052]: I1203 11:41:50.567206 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-metadata-0" podStartSLOduration=2.567184937 podStartE2EDuration="2.567184937s" podCreationTimestamp="2025-12-03 11:41:48 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:41:50.557199016 +0000 UTC m=+1301.145440800" watchObservedRunningTime="2025-12-03 11:41:50.567184937 +0000 UTC m=+1301.155426711" Dec 03 11:41:50 crc kubenswrapper[5052]: I1203 11:41:50.587182 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-api-0" podStartSLOduration=2.587164928 podStartE2EDuration="2.587164928s" podCreationTimestamp="2025-12-03 11:41:48 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:41:50.57899361 +0000 UTC m=+1301.167235374" watchObservedRunningTime="2025-12-03 11:41:50.587164928 +0000 UTC m=+1301.175406692" Dec 03 11:41:52 crc kubenswrapper[5052]: I1203 11:41:52.829773 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:41:54 crc kubenswrapper[5052]: I1203 11:41:54.179313 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:41:54 crc kubenswrapper[5052]: I1203 11:41:54.179651 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:41:57 crc kubenswrapper[5052]: I1203 11:41:57.830004 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:41:57 crc kubenswrapper[5052]: I1203 11:41:57.912315 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:41:58 crc kubenswrapper[5052]: I1203 11:41:58.646448 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:41:58 crc kubenswrapper[5052]: I1203 11:41:58.885619 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:41:58 crc kubenswrapper[5052]: I1203 11:41:58.885663 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:41:59 crc kubenswrapper[5052]: I1203 11:41:59.179372 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:41:59 crc kubenswrapper[5052]: I1203 11:41:59.179625 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:41:59 crc kubenswrapper[5052]: I1203 11:41:59.379120 5052 patch_prober.go:28] interesting pod/machine-config-daemon-74bj6 container/machine-config-daemon namespace/openshift-machine-config-operator: Liveness probe status=failure output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" start-of-body= Dec 03 11:41:59 crc kubenswrapper[5052]: I1203 11:41:59.379177 5052 prober.go:107] "Probe failed" probeType="Liveness" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" containerName="machine-config-daemon" probeResult="failure" output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" Dec 03 11:41:59 crc kubenswrapper[5052]: I1203 11:41:59.897904 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-api-0" podUID="82dce330-8f65-4c14-80d4-958eb8ca5212" containerName="nova-api-log" probeResult="failure" output="Get \"https://10.217.0.184:8774/\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Dec 03 11:41:59 crc kubenswrapper[5052]: I1203 11:41:59.897968 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-api-0" podUID="82dce330-8f65-4c14-80d4-958eb8ca5212" containerName="nova-api-api" probeResult="failure" output="Get \"https://10.217.0.184:8774/\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Dec 03 11:42:00 crc kubenswrapper[5052]: I1203 11:42:00.193787 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-metadata-0" podUID="f98f227a-397b-4665-bf18-e00b105ce058" containerName="nova-metadata-metadata" probeResult="failure" output="Get \"https://10.217.0.185:8775/\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Dec 03 11:42:00 crc kubenswrapper[5052]: I1203 11:42:00.193858 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-metadata-0" podUID="f98f227a-397b-4665-bf18-e00b105ce058" containerName="nova-metadata-log" probeResult="failure" output="Get \"https://10.217.0.185:8775/\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Dec 03 11:42:06 crc kubenswrapper[5052]: I1203 11:42:06.761160 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openshift-marketplace/redhat-operators-bvp4v"] Dec 03 11:42:06 crc kubenswrapper[5052]: I1203 11:42:06.763554 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/redhat-operators-bvp4v" Dec 03 11:42:06 crc kubenswrapper[5052]: I1203 11:42:06.786468 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8cvmt\" (UniqueName: \"kubernetes.io/projected/18c1ce3d-6d64-433f-a115-9977227455be-kube-api-access-8cvmt\") pod \"redhat-operators-bvp4v\" (UID: \"18c1ce3d-6d64-433f-a115-9977227455be\") " pod="openshift-marketplace/redhat-operators-bvp4v" Dec 03 11:42:06 crc kubenswrapper[5052]: I1203 11:42:06.786797 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/18c1ce3d-6d64-433f-a115-9977227455be-catalog-content\") pod \"redhat-operators-bvp4v\" (UID: \"18c1ce3d-6d64-433f-a115-9977227455be\") " pod="openshift-marketplace/redhat-operators-bvp4v" Dec 03 11:42:06 crc kubenswrapper[5052]: I1203 11:42:06.786942 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/18c1ce3d-6d64-433f-a115-9977227455be-utilities\") pod \"redhat-operators-bvp4v\" (UID: \"18c1ce3d-6d64-433f-a115-9977227455be\") " pod="openshift-marketplace/redhat-operators-bvp4v" Dec 03 11:42:06 crc kubenswrapper[5052]: I1203 11:42:06.868376 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/redhat-operators-bvp4v"] Dec 03 11:42:06 crc kubenswrapper[5052]: I1203 11:42:06.889278 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/18c1ce3d-6d64-433f-a115-9977227455be-utilities\") pod \"redhat-operators-bvp4v\" (UID: \"18c1ce3d-6d64-433f-a115-9977227455be\") " pod="openshift-marketplace/redhat-operators-bvp4v" Dec 03 11:42:06 crc kubenswrapper[5052]: I1203 11:42:06.889424 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-8cvmt\" (UniqueName: \"kubernetes.io/projected/18c1ce3d-6d64-433f-a115-9977227455be-kube-api-access-8cvmt\") pod \"redhat-operators-bvp4v\" (UID: \"18c1ce3d-6d64-433f-a115-9977227455be\") " pod="openshift-marketplace/redhat-operators-bvp4v" Dec 03 11:42:06 crc kubenswrapper[5052]: I1203 11:42:06.889475 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/18c1ce3d-6d64-433f-a115-9977227455be-catalog-content\") pod \"redhat-operators-bvp4v\" (UID: \"18c1ce3d-6d64-433f-a115-9977227455be\") " pod="openshift-marketplace/redhat-operators-bvp4v" Dec 03 11:42:06 crc kubenswrapper[5052]: I1203 11:42:06.889836 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/18c1ce3d-6d64-433f-a115-9977227455be-utilities\") pod \"redhat-operators-bvp4v\" (UID: \"18c1ce3d-6d64-433f-a115-9977227455be\") " pod="openshift-marketplace/redhat-operators-bvp4v" Dec 03 11:42:06 crc kubenswrapper[5052]: I1203 11:42:06.889963 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/18c1ce3d-6d64-433f-a115-9977227455be-catalog-content\") pod \"redhat-operators-bvp4v\" (UID: \"18c1ce3d-6d64-433f-a115-9977227455be\") " pod="openshift-marketplace/redhat-operators-bvp4v" Dec 03 11:42:06 crc kubenswrapper[5052]: I1203 11:42:06.912711 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-8cvmt\" (UniqueName: \"kubernetes.io/projected/18c1ce3d-6d64-433f-a115-9977227455be-kube-api-access-8cvmt\") pod \"redhat-operators-bvp4v\" (UID: \"18c1ce3d-6d64-433f-a115-9977227455be\") " pod="openshift-marketplace/redhat-operators-bvp4v" Dec 03 11:42:07 crc kubenswrapper[5052]: I1203 11:42:07.146255 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/redhat-operators-bvp4v" Dec 03 11:42:07 crc kubenswrapper[5052]: W1203 11:42:07.625614 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18c1ce3d_6d64_433f_a115_9977227455be.slice/crio-3abd0b1c0f5c12b60a51715e9fd0f3bc9393f48ca48a455c099f922894cdf07e WatchSource:0}: Error finding container 3abd0b1c0f5c12b60a51715e9fd0f3bc9393f48ca48a455c099f922894cdf07e: Status 404 returned error can't find the container with id 3abd0b1c0f5c12b60a51715e9fd0f3bc9393f48ca48a455c099f922894cdf07e Dec 03 11:42:07 crc kubenswrapper[5052]: I1203 11:42:07.627495 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/redhat-operators-bvp4v"] Dec 03 11:42:07 crc kubenswrapper[5052]: I1203 11:42:07.722261 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-operators-bvp4v" event={"ID":"18c1ce3d-6d64-433f-a115-9977227455be","Type":"ContainerStarted","Data":"3abd0b1c0f5c12b60a51715e9fd0f3bc9393f48ca48a455c099f922894cdf07e"} Dec 03 11:42:08 crc kubenswrapper[5052]: I1203 11:42:08.747314 5052 generic.go:334] "Generic (PLEG): container finished" podID="18c1ce3d-6d64-433f-a115-9977227455be" containerID="910971ac3056ce793483491994f03b4796ccbd03d513c9fc09a55fb5a371b881" exitCode=0 Dec 03 11:42:08 crc kubenswrapper[5052]: I1203 11:42:08.747444 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-operators-bvp4v" event={"ID":"18c1ce3d-6d64-433f-a115-9977227455be","Type":"ContainerDied","Data":"910971ac3056ce793483491994f03b4796ccbd03d513c9fc09a55fb5a371b881"} Dec 03 11:42:08 crc kubenswrapper[5052]: I1203 11:42:08.750203 5052 provider.go:102] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Dec 03 11:42:08 crc kubenswrapper[5052]: I1203 11:42:08.891742 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:42:08 crc kubenswrapper[5052]: I1203 11:42:08.893004 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:42:08 crc kubenswrapper[5052]: I1203 11:42:08.894403 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:42:08 crc kubenswrapper[5052]: I1203 11:42:08.901786 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:42:09 crc kubenswrapper[5052]: I1203 11:42:09.184031 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:42:09 crc kubenswrapper[5052]: I1203 11:42:09.184993 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:42:09 crc kubenswrapper[5052]: I1203 11:42:09.189157 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:42:09 crc kubenswrapper[5052]: I1203 11:42:09.761222 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:42:09 crc kubenswrapper[5052]: I1203 11:42:09.765782 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:42:09 crc kubenswrapper[5052]: I1203 11:42:09.769688 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:42:10 crc kubenswrapper[5052]: I1203 11:42:10.771135 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-operators-bvp4v" event={"ID":"18c1ce3d-6d64-433f-a115-9977227455be","Type":"ContainerStarted","Data":"ed4175dc9ade01ddd2b17bd932170a14f62de0010dc8527f7fa2c37dde74bfe6"} Dec 03 11:42:12 crc kubenswrapper[5052]: I1203 11:42:12.809148 5052 generic.go:334] "Generic (PLEG): container finished" podID="18c1ce3d-6d64-433f-a115-9977227455be" containerID="ed4175dc9ade01ddd2b17bd932170a14f62de0010dc8527f7fa2c37dde74bfe6" exitCode=0 Dec 03 11:42:12 crc kubenswrapper[5052]: I1203 11:42:12.809237 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-operators-bvp4v" event={"ID":"18c1ce3d-6d64-433f-a115-9977227455be","Type":"ContainerDied","Data":"ed4175dc9ade01ddd2b17bd932170a14f62de0010dc8527f7fa2c37dde74bfe6"} Dec 03 11:42:13 crc kubenswrapper[5052]: I1203 11:42:13.819703 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-operators-bvp4v" event={"ID":"18c1ce3d-6d64-433f-a115-9977227455be","Type":"ContainerStarted","Data":"1fddb059a8118b5a724a5d5a74bfd9dcb6fdac1b88a0a1f2bd904834de863a32"} Dec 03 11:42:13 crc kubenswrapper[5052]: I1203 11:42:13.851845 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-marketplace/redhat-operators-bvp4v" podStartSLOduration=3.175474771 podStartE2EDuration="7.851824779s" podCreationTimestamp="2025-12-03 11:42:06 +0000 UTC" firstStartedPulling="2025-12-03 11:42:08.749927682 +0000 UTC m=+1319.338169446" lastFinishedPulling="2025-12-03 11:42:13.42627769 +0000 UTC m=+1324.014519454" observedRunningTime="2025-12-03 11:42:13.847149423 +0000 UTC m=+1324.435391197" watchObservedRunningTime="2025-12-03 11:42:13.851824779 +0000 UTC m=+1324.440066543" Dec 03 11:42:14 crc kubenswrapper[5052]: I1203 11:42:14.792348 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:42:17 crc kubenswrapper[5052]: I1203 11:42:17.147839 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-marketplace/redhat-operators-bvp4v" Dec 03 11:42:17 crc kubenswrapper[5052]: I1203 11:42:17.148144 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openshift-marketplace/redhat-operators-bvp4v" Dec 03 11:42:18 crc kubenswrapper[5052]: I1203 11:42:18.197879 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openshift-marketplace/redhat-operators-bvp4v" podUID="18c1ce3d-6d64-433f-a115-9977227455be" containerName="registry-server" probeResult="failure" output=< Dec 03 11:42:18 crc kubenswrapper[5052]: timeout: failed to connect service ":50051" within 1s Dec 03 11:42:18 crc kubenswrapper[5052]: > Dec 03 11:42:21 crc kubenswrapper[5052]: I1203 11:42:21.980230 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-0"] Dec 03 11:42:21 crc kubenswrapper[5052]: I1203 11:42:21.980984 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-cell0-conductor-0" podUID="f8cdf05e-7081-4542-844d-f76378e8989f" containerName="nova-cell0-conductor-conductor" containerID="cri-o://9cfabfd91ca0525b54d3922c76017cb0defbc772bca498498fc287ac2175e2aa" gracePeriod=30 Dec 03 11:42:22 crc kubenswrapper[5052]: I1203 11:42:22.001099 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/openstackclient"] Dec 03 11:42:22 crc kubenswrapper[5052]: I1203 11:42:22.001272 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/openstackclient" podUID="722e5457-ae47-40b8-816f-8eab5b7a4cb0" containerName="openstackclient" containerID="cri-o://7f7814a46f2c2a853ddc0d2142a5cfb0bf53aa603be45491209a66cd585a4f18" gracePeriod=2 Dec 03 11:42:22 crc kubenswrapper[5052]: I1203 11:42:22.044591 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 11:42:22 crc kubenswrapper[5052]: I1203 11:42:22.044900 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-metadata-0" podUID="f98f227a-397b-4665-bf18-e00b105ce058" containerName="nova-metadata-log" containerID="cri-o://9a73f69a43329d67c0d372934e250290b21f5530d0b0079f8192b4709695cb6a" gracePeriod=30 Dec 03 11:42:22 crc kubenswrapper[5052]: I1203 11:42:22.045327 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-metadata-0" podUID="f98f227a-397b-4665-bf18-e00b105ce058" containerName="nova-metadata-metadata" containerID="cri-o://b2697ef9ab442e1740e9992a4ae8a2ad183a400760ac5eea2d853dc944ed7fa8" gracePeriod=30 Dec 03 11:42:22 crc kubenswrapper[5052]: I1203 11:42:22.051286 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 11:42:22 crc kubenswrapper[5052]: I1203 11:42:22.051619 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-scheduler-0" podUID="d80d1476-351a-40c6-9f76-ce66d623df74" containerName="nova-scheduler-scheduler" containerID="cri-o://2acdb0814bfda1b6c48a4fde354fd532f86ae71dcaac188c16bee44a794b3713" gracePeriod=30 Dec 03 11:42:22 crc kubenswrapper[5052]: I1203 11:42:22.063883 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ovn-northd-0"] Dec 03 11:42:22 crc kubenswrapper[5052]: I1203 11:42:22.064101 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ovn-northd-0" podUID="a30eaf15-65ff-426b-8ac2-1ed5a369886a" containerName="ovn-northd" containerID="cri-o://455e8792612e9ef5fbedefc9390ff87ea262cbd9365b333801dd2dc51224737e" gracePeriod=30 Dec 03 11:42:22 crc kubenswrapper[5052]: I1203 11:42:22.064208 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ovn-northd-0" podUID="a30eaf15-65ff-426b-8ac2-1ed5a369886a" containerName="openstack-network-exporter" containerID="cri-o://abd32ba7a2fea1f9280ace25d3fe52938520b763a092b3e8d9e1b4a4f4e6e2be" gracePeriod=30 Dec 03 11:42:22 crc kubenswrapper[5052]: I1203 11:42:22.076052 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/openstackclient"] Dec 03 11:42:22 crc kubenswrapper[5052]: I1203 11:42:22.083023 5052 status_manager.go:861] "Pod was deleted and then recreated, skipping status update" pod="openstack-kuttl-tests/openstackclient" oldPodUID="722e5457-ae47-40b8-816f-8eab5b7a4cb0" podUID="24f61ea3-1916-46dd-a764-2211a9735d42" Dec 03 11:42:22 crc kubenswrapper[5052]: I1203 11:42:22.086588 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/barbican-keystone-listener-5dd6f4bfd-glsv7"] Dec 03 11:42:22 crc kubenswrapper[5052]: E1203 11:42:22.086989 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="722e5457-ae47-40b8-816f-8eab5b7a4cb0" containerName="openstackclient" Dec 03 11:42:22 crc kubenswrapper[5052]: I1203 11:42:22.087004 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="722e5457-ae47-40b8-816f-8eab5b7a4cb0" containerName="openstackclient" Dec 03 11:42:22 crc kubenswrapper[5052]: I1203 11:42:22.087219 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="722e5457-ae47-40b8-816f-8eab5b7a4cb0" containerName="openstackclient" Dec 03 11:42:22 crc kubenswrapper[5052]: I1203 11:42:22.088221 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-keystone-listener-5dd6f4bfd-glsv7" Dec 03 11:42:22 crc kubenswrapper[5052]: I1203 11:42:22.097055 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 11:42:22 crc kubenswrapper[5052]: I1203 11:42:22.097241 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" podUID="049153cf-b5da-4a61-8a2d-3f694625642b" containerName="nova-cell1-novncproxy-novncproxy" containerID="cri-o://10f606fe8fa7b691b8917f929fb59e6261d389781afc79a34d36abb996d45333" gracePeriod=30 Dec 03 11:42:22 crc kubenswrapper[5052]: I1203 11:42:22.102681 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/ovn-northd-0" podUID="a30eaf15-65ff-426b-8ac2-1ed5a369886a" containerName="ovn-northd" probeResult="failure" output=< Dec 03 11:42:22 crc kubenswrapper[5052]: 2025-12-03T11:42:22Z|00001|unixctl|WARN|failed to connect to /tmp/ovn-northd.1.ctl Dec 03 11:42:22 crc kubenswrapper[5052]: ovn-appctl: cannot connect to "/tmp/ovn-northd.1.ctl" (No such file or directory) Dec 03 11:42:22 crc kubenswrapper[5052]: > Dec 03 11:42:22 crc kubenswrapper[5052]: I1203 11:42:22.107742 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-keystone-listener-5dd6f4bfd-glsv7"] Dec 03 11:42:22 crc kubenswrapper[5052]: I1203 11:42:22.132541 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/barbican-worker-67c775d6bc-kwxzt"] Dec 03 11:42:22 crc kubenswrapper[5052]: I1203 11:42:22.134130 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-worker-67c775d6bc-kwxzt" Dec 03 11:42:22 crc kubenswrapper[5052]: I1203 11:42:22.157562 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 11:42:22 crc kubenswrapper[5052]: I1203 11:42:22.157844 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-external-api-0" podUID="75e4b34e-f7b8-4209-95a6-6cc45f6eb57c" containerName="glance-log" containerID="cri-o://99cb39696946884496d7905d0018b8b8e349cbda3cdb669d009adf92f668080b" gracePeriod=30 Dec 03 11:42:22 crc kubenswrapper[5052]: I1203 11:42:22.157932 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-external-api-0" podUID="75e4b34e-f7b8-4209-95a6-6cc45f6eb57c" containerName="glance-httpd" containerID="cri-o://2b213199dec26418337b9d73443612a12a1ce69360675968db75a8f6aba268ae" gracePeriod=30 Dec 03 11:42:22 crc kubenswrapper[5052]: I1203 11:42:22.206772 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-worker-67c775d6bc-kwxzt"] Dec 03 11:42:22 crc kubenswrapper[5052]: I1203 11:42:22.221984 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zqmkd\" (UniqueName: \"kubernetes.io/projected/222f033c-3660-4347-b621-7122c4c94929-kube-api-access-zqmkd\") pod \"barbican-worker-67c775d6bc-kwxzt\" (UID: \"222f033c-3660-4347-b621-7122c4c94929\") " pod="openstack-kuttl-tests/barbican-worker-67c775d6bc-kwxzt" Dec 03 11:42:22 crc kubenswrapper[5052]: I1203 11:42:22.222033 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/3004edac-eef4-4ddc-a539-4d4294029774-config-data\") pod \"barbican-keystone-listener-5dd6f4bfd-glsv7\" (UID: \"3004edac-eef4-4ddc-a539-4d4294029774\") " pod="openstack-kuttl-tests/barbican-keystone-listener-5dd6f4bfd-glsv7" Dec 03 11:42:22 crc kubenswrapper[5052]: I1203 11:42:22.222058 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/222f033c-3660-4347-b621-7122c4c94929-config-data\") pod \"barbican-worker-67c775d6bc-kwxzt\" (UID: \"222f033c-3660-4347-b621-7122c4c94929\") " pod="openstack-kuttl-tests/barbican-worker-67c775d6bc-kwxzt" Dec 03 11:42:22 crc kubenswrapper[5052]: I1203 11:42:22.222123 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3004edac-eef4-4ddc-a539-4d4294029774-combined-ca-bundle\") pod \"barbican-keystone-listener-5dd6f4bfd-glsv7\" (UID: \"3004edac-eef4-4ddc-a539-4d4294029774\") " pod="openstack-kuttl-tests/barbican-keystone-listener-5dd6f4bfd-glsv7" Dec 03 11:42:22 crc kubenswrapper[5052]: I1203 11:42:22.222174 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/222f033c-3660-4347-b621-7122c4c94929-combined-ca-bundle\") pod \"barbican-worker-67c775d6bc-kwxzt\" (UID: \"222f033c-3660-4347-b621-7122c4c94929\") " pod="openstack-kuttl-tests/barbican-worker-67c775d6bc-kwxzt" Dec 03 11:42:22 crc kubenswrapper[5052]: I1203 11:42:22.222250 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wzn2j\" (UniqueName: \"kubernetes.io/projected/3004edac-eef4-4ddc-a539-4d4294029774-kube-api-access-wzn2j\") pod \"barbican-keystone-listener-5dd6f4bfd-glsv7\" (UID: \"3004edac-eef4-4ddc-a539-4d4294029774\") " pod="openstack-kuttl-tests/barbican-keystone-listener-5dd6f4bfd-glsv7" Dec 03 11:42:22 crc kubenswrapper[5052]: I1203 11:42:22.222387 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/222f033c-3660-4347-b621-7122c4c94929-config-data-custom\") pod \"barbican-worker-67c775d6bc-kwxzt\" (UID: \"222f033c-3660-4347-b621-7122c4c94929\") " pod="openstack-kuttl-tests/barbican-worker-67c775d6bc-kwxzt" Dec 03 11:42:22 crc kubenswrapper[5052]: I1203 11:42:22.222419 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/3004edac-eef4-4ddc-a539-4d4294029774-config-data-custom\") pod \"barbican-keystone-listener-5dd6f4bfd-glsv7\" (UID: \"3004edac-eef4-4ddc-a539-4d4294029774\") " pod="openstack-kuttl-tests/barbican-keystone-listener-5dd6f4bfd-glsv7" Dec 03 11:42:22 crc kubenswrapper[5052]: I1203 11:42:22.222576 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/222f033c-3660-4347-b621-7122c4c94929-logs\") pod \"barbican-worker-67c775d6bc-kwxzt\" (UID: \"222f033c-3660-4347-b621-7122c4c94929\") " pod="openstack-kuttl-tests/barbican-worker-67c775d6bc-kwxzt" Dec 03 11:42:22 crc kubenswrapper[5052]: I1203 11:42:22.222604 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/3004edac-eef4-4ddc-a539-4d4294029774-logs\") pod \"barbican-keystone-listener-5dd6f4bfd-glsv7\" (UID: \"3004edac-eef4-4ddc-a539-4d4294029774\") " pod="openstack-kuttl-tests/barbican-keystone-listener-5dd6f4bfd-glsv7" Dec 03 11:42:22 crc kubenswrapper[5052]: I1203 11:42:22.224862 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/memcached-0"] Dec 03 11:42:22 crc kubenswrapper[5052]: I1203 11:42:22.225084 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/memcached-0" podUID="1609a5b5-6666-47e3-b02e-f796d972d331" containerName="memcached" containerID="cri-o://352c7041b2ebcbf03c6b6dbad2b08bc20c546043c65916bf3b506556deb9182f" gracePeriod=30 Dec 03 11:42:22 crc kubenswrapper[5052]: I1203 11:42:22.238917 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 11:42:22 crc kubenswrapper[5052]: I1203 11:42:22.239358 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-api-0" podUID="82dce330-8f65-4c14-80d4-958eb8ca5212" containerName="nova-api-log" containerID="cri-o://b939d536bc53d88cf0c53f3e90f1a9bbe204dc2f74470b14127832658e7f3898" gracePeriod=30 Dec 03 11:42:22 crc kubenswrapper[5052]: I1203 11:42:22.239454 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-api-0" podUID="82dce330-8f65-4c14-80d4-958eb8ca5212" containerName="nova-api-api" containerID="cri-o://d6cb51da86b5f965cccc6e7f9d1241d74c446a9276dbf42e057cec9147384cb3" gracePeriod=30 Dec 03 11:42:22 crc kubenswrapper[5052]: I1203 11:42:22.267912 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/openstackclient"] Dec 03 11:42:22 crc kubenswrapper[5052]: I1203 11:42:22.269523 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstackclient" Dec 03 11:42:22 crc kubenswrapper[5052]: I1203 11:42:22.307612 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-sb-0"] Dec 03 11:42:22 crc kubenswrapper[5052]: I1203 11:42:22.307989 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ovsdbserver-sb-0" podUID="e14d926b-5dc2-46a4-ab27-dfde118cba89" containerName="openstack-network-exporter" containerID="cri-o://8da90eb5a7870f9bbe93a949f459805faa0302426245aa70e016cd57e8992d74" gracePeriod=300 Dec 03 11:42:22 crc kubenswrapper[5052]: I1203 11:42:22.327369 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-wzn2j\" (UniqueName: \"kubernetes.io/projected/3004edac-eef4-4ddc-a539-4d4294029774-kube-api-access-wzn2j\") pod \"barbican-keystone-listener-5dd6f4bfd-glsv7\" (UID: \"3004edac-eef4-4ddc-a539-4d4294029774\") " pod="openstack-kuttl-tests/barbican-keystone-listener-5dd6f4bfd-glsv7" Dec 03 11:42:22 crc kubenswrapper[5052]: I1203 11:42:22.327418 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/24f61ea3-1916-46dd-a764-2211a9735d42-combined-ca-bundle\") pod \"openstackclient\" (UID: \"24f61ea3-1916-46dd-a764-2211a9735d42\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 11:42:22 crc kubenswrapper[5052]: I1203 11:42:22.327460 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c2d2h\" (UniqueName: \"kubernetes.io/projected/24f61ea3-1916-46dd-a764-2211a9735d42-kube-api-access-c2d2h\") pod \"openstackclient\" (UID: \"24f61ea3-1916-46dd-a764-2211a9735d42\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 11:42:22 crc kubenswrapper[5052]: I1203 11:42:22.327497 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/222f033c-3660-4347-b621-7122c4c94929-config-data-custom\") pod \"barbican-worker-67c775d6bc-kwxzt\" (UID: \"222f033c-3660-4347-b621-7122c4c94929\") " pod="openstack-kuttl-tests/barbican-worker-67c775d6bc-kwxzt" Dec 03 11:42:22 crc kubenswrapper[5052]: I1203 11:42:22.327545 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/3004edac-eef4-4ddc-a539-4d4294029774-config-data-custom\") pod \"barbican-keystone-listener-5dd6f4bfd-glsv7\" (UID: \"3004edac-eef4-4ddc-a539-4d4294029774\") " pod="openstack-kuttl-tests/barbican-keystone-listener-5dd6f4bfd-glsv7" Dec 03 11:42:22 crc kubenswrapper[5052]: I1203 11:42:22.327581 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openstack-config-secret\" (UniqueName: \"kubernetes.io/secret/24f61ea3-1916-46dd-a764-2211a9735d42-openstack-config-secret\") pod \"openstackclient\" (UID: \"24f61ea3-1916-46dd-a764-2211a9735d42\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 11:42:22 crc kubenswrapper[5052]: I1203 11:42:22.327652 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/222f033c-3660-4347-b621-7122c4c94929-logs\") pod \"barbican-worker-67c775d6bc-kwxzt\" (UID: \"222f033c-3660-4347-b621-7122c4c94929\") " pod="openstack-kuttl-tests/barbican-worker-67c775d6bc-kwxzt" Dec 03 11:42:22 crc kubenswrapper[5052]: I1203 11:42:22.327682 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/3004edac-eef4-4ddc-a539-4d4294029774-logs\") pod \"barbican-keystone-listener-5dd6f4bfd-glsv7\" (UID: \"3004edac-eef4-4ddc-a539-4d4294029774\") " pod="openstack-kuttl-tests/barbican-keystone-listener-5dd6f4bfd-glsv7" Dec 03 11:42:22 crc kubenswrapper[5052]: I1203 11:42:22.327710 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-zqmkd\" (UniqueName: \"kubernetes.io/projected/222f033c-3660-4347-b621-7122c4c94929-kube-api-access-zqmkd\") pod \"barbican-worker-67c775d6bc-kwxzt\" (UID: \"222f033c-3660-4347-b621-7122c4c94929\") " pod="openstack-kuttl-tests/barbican-worker-67c775d6bc-kwxzt" Dec 03 11:42:22 crc kubenswrapper[5052]: I1203 11:42:22.327748 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/3004edac-eef4-4ddc-a539-4d4294029774-config-data\") pod \"barbican-keystone-listener-5dd6f4bfd-glsv7\" (UID: \"3004edac-eef4-4ddc-a539-4d4294029774\") " pod="openstack-kuttl-tests/barbican-keystone-listener-5dd6f4bfd-glsv7" Dec 03 11:42:22 crc kubenswrapper[5052]: I1203 11:42:22.327771 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/222f033c-3660-4347-b621-7122c4c94929-config-data\") pod \"barbican-worker-67c775d6bc-kwxzt\" (UID: \"222f033c-3660-4347-b621-7122c4c94929\") " pod="openstack-kuttl-tests/barbican-worker-67c775d6bc-kwxzt" Dec 03 11:42:22 crc kubenswrapper[5052]: I1203 11:42:22.327804 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3004edac-eef4-4ddc-a539-4d4294029774-combined-ca-bundle\") pod \"barbican-keystone-listener-5dd6f4bfd-glsv7\" (UID: \"3004edac-eef4-4ddc-a539-4d4294029774\") " pod="openstack-kuttl-tests/barbican-keystone-listener-5dd6f4bfd-glsv7" Dec 03 11:42:22 crc kubenswrapper[5052]: I1203 11:42:22.327825 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openstack-config\" (UniqueName: \"kubernetes.io/configmap/24f61ea3-1916-46dd-a764-2211a9735d42-openstack-config\") pod \"openstackclient\" (UID: \"24f61ea3-1916-46dd-a764-2211a9735d42\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 11:42:22 crc kubenswrapper[5052]: I1203 11:42:22.327853 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/222f033c-3660-4347-b621-7122c4c94929-combined-ca-bundle\") pod \"barbican-worker-67c775d6bc-kwxzt\" (UID: \"222f033c-3660-4347-b621-7122c4c94929\") " pod="openstack-kuttl-tests/barbican-worker-67c775d6bc-kwxzt" Dec 03 11:42:22 crc kubenswrapper[5052]: I1203 11:42:22.338922 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/222f033c-3660-4347-b621-7122c4c94929-logs\") pod \"barbican-worker-67c775d6bc-kwxzt\" (UID: \"222f033c-3660-4347-b621-7122c4c94929\") " pod="openstack-kuttl-tests/barbican-worker-67c775d6bc-kwxzt" Dec 03 11:42:22 crc kubenswrapper[5052]: I1203 11:42:22.342370 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/3004edac-eef4-4ddc-a539-4d4294029774-logs\") pod \"barbican-keystone-listener-5dd6f4bfd-glsv7\" (UID: \"3004edac-eef4-4ddc-a539-4d4294029774\") " pod="openstack-kuttl-tests/barbican-keystone-listener-5dd6f4bfd-glsv7" Dec 03 11:42:22 crc kubenswrapper[5052]: I1203 11:42:22.345495 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/222f033c-3660-4347-b621-7122c4c94929-combined-ca-bundle\") pod \"barbican-worker-67c775d6bc-kwxzt\" (UID: \"222f033c-3660-4347-b621-7122c4c94929\") " pod="openstack-kuttl-tests/barbican-worker-67c775d6bc-kwxzt" Dec 03 11:42:22 crc kubenswrapper[5052]: I1203 11:42:22.345683 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/openstackclient"] Dec 03 11:42:22 crc kubenswrapper[5052]: I1203 11:42:22.355162 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3004edac-eef4-4ddc-a539-4d4294029774-combined-ca-bundle\") pod \"barbican-keystone-listener-5dd6f4bfd-glsv7\" (UID: \"3004edac-eef4-4ddc-a539-4d4294029774\") " pod="openstack-kuttl-tests/barbican-keystone-listener-5dd6f4bfd-glsv7" Dec 03 11:42:22 crc kubenswrapper[5052]: I1203 11:42:22.359943 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/3004edac-eef4-4ddc-a539-4d4294029774-config-data\") pod \"barbican-keystone-listener-5dd6f4bfd-glsv7\" (UID: \"3004edac-eef4-4ddc-a539-4d4294029774\") " pod="openstack-kuttl-tests/barbican-keystone-listener-5dd6f4bfd-glsv7" Dec 03 11:42:22 crc kubenswrapper[5052]: I1203 11:42:22.363556 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/3004edac-eef4-4ddc-a539-4d4294029774-config-data-custom\") pod \"barbican-keystone-listener-5dd6f4bfd-glsv7\" (UID: \"3004edac-eef4-4ddc-a539-4d4294029774\") " pod="openstack-kuttl-tests/barbican-keystone-listener-5dd6f4bfd-glsv7" Dec 03 11:42:22 crc kubenswrapper[5052]: I1203 11:42:22.365720 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/222f033c-3660-4347-b621-7122c4c94929-config-data-custom\") pod \"barbican-worker-67c775d6bc-kwxzt\" (UID: \"222f033c-3660-4347-b621-7122c4c94929\") " pod="openstack-kuttl-tests/barbican-worker-67c775d6bc-kwxzt" Dec 03 11:42:22 crc kubenswrapper[5052]: I1203 11:42:22.379756 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/222f033c-3660-4347-b621-7122c4c94929-config-data\") pod \"barbican-worker-67c775d6bc-kwxzt\" (UID: \"222f033c-3660-4347-b621-7122c4c94929\") " pod="openstack-kuttl-tests/barbican-worker-67c775d6bc-kwxzt" Dec 03 11:42:22 crc kubenswrapper[5052]: I1203 11:42:22.396345 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 11:42:22 crc kubenswrapper[5052]: I1203 11:42:22.396639 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/cinder-scheduler-0" podUID="14e9bff5-66d5-4808-8446-cf180d759208" containerName="cinder-scheduler" containerID="cri-o://b875595b2b281c94c97104d6ea786f06ea56deeebb4024873ed0f82dba8c7b52" gracePeriod=30 Dec 03 11:42:22 crc kubenswrapper[5052]: I1203 11:42:22.396785 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/cinder-scheduler-0" podUID="14e9bff5-66d5-4808-8446-cf180d759208" containerName="probe" containerID="cri-o://6c977dab718207e18c944877c552253d8c5723de1e4d496e75dea068be83b234" gracePeriod=30 Dec 03 11:42:22 crc kubenswrapper[5052]: I1203 11:42:22.441635 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/24f61ea3-1916-46dd-a764-2211a9735d42-combined-ca-bundle\") pod \"openstackclient\" (UID: \"24f61ea3-1916-46dd-a764-2211a9735d42\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 11:42:22 crc kubenswrapper[5052]: I1203 11:42:22.441707 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-c2d2h\" (UniqueName: \"kubernetes.io/projected/24f61ea3-1916-46dd-a764-2211a9735d42-kube-api-access-c2d2h\") pod \"openstackclient\" (UID: \"24f61ea3-1916-46dd-a764-2211a9735d42\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 11:42:22 crc kubenswrapper[5052]: I1203 11:42:22.441770 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"openstack-config-secret\" (UniqueName: \"kubernetes.io/secret/24f61ea3-1916-46dd-a764-2211a9735d42-openstack-config-secret\") pod \"openstackclient\" (UID: \"24f61ea3-1916-46dd-a764-2211a9735d42\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 11:42:22 crc kubenswrapper[5052]: I1203 11:42:22.441886 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"openstack-config\" (UniqueName: \"kubernetes.io/configmap/24f61ea3-1916-46dd-a764-2211a9735d42-openstack-config\") pod \"openstackclient\" (UID: \"24f61ea3-1916-46dd-a764-2211a9735d42\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 11:42:22 crc kubenswrapper[5052]: I1203 11:42:22.442941 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"openstack-config\" (UniqueName: \"kubernetes.io/configmap/24f61ea3-1916-46dd-a764-2211a9735d42-openstack-config\") pod \"openstackclient\" (UID: \"24f61ea3-1916-46dd-a764-2211a9735d42\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 11:42:22 crc kubenswrapper[5052]: I1203 11:42:22.455810 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/24f61ea3-1916-46dd-a764-2211a9735d42-combined-ca-bundle\") pod \"openstackclient\" (UID: \"24f61ea3-1916-46dd-a764-2211a9735d42\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 11:42:22 crc kubenswrapper[5052]: I1203 11:42:22.467283 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-zqmkd\" (UniqueName: \"kubernetes.io/projected/222f033c-3660-4347-b621-7122c4c94929-kube-api-access-zqmkd\") pod \"barbican-worker-67c775d6bc-kwxzt\" (UID: \"222f033c-3660-4347-b621-7122c4c94929\") " pod="openstack-kuttl-tests/barbican-worker-67c775d6bc-kwxzt" Dec 03 11:42:22 crc kubenswrapper[5052]: I1203 11:42:22.467615 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"openstack-config-secret\" (UniqueName: \"kubernetes.io/secret/24f61ea3-1916-46dd-a764-2211a9735d42-openstack-config-secret\") pod \"openstackclient\" (UID: \"24f61ea3-1916-46dd-a764-2211a9735d42\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 11:42:22 crc kubenswrapper[5052]: I1203 11:42:22.474566 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/placement-6d84cdf894-dqcg9"] Dec 03 11:42:22 crc kubenswrapper[5052]: I1203 11:42:22.476354 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-6d84cdf894-dqcg9" Dec 03 11:42:22 crc kubenswrapper[5052]: I1203 11:42:22.497096 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-c2d2h\" (UniqueName: \"kubernetes.io/projected/24f61ea3-1916-46dd-a764-2211a9735d42-kube-api-access-c2d2h\") pod \"openstackclient\" (UID: \"24f61ea3-1916-46dd-a764-2211a9735d42\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 11:42:22 crc kubenswrapper[5052]: I1203 11:42:22.504604 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ovsdbserver-sb-0" podUID="e14d926b-5dc2-46a4-ab27-dfde118cba89" containerName="ovsdbserver-sb" containerID="cri-o://ef90774de4b83b9fc123ba664ece34787dcdb1ad9fd0d230dacf72b8eee1a1e1" gracePeriod=300 Dec 03 11:42:22 crc kubenswrapper[5052]: I1203 11:42:22.507729 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/openstack-galera-0"] Dec 03 11:42:22 crc kubenswrapper[5052]: I1203 11:42:22.545718 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/placement-6d84cdf894-dqcg9"] Dec 03 11:42:22 crc kubenswrapper[5052]: I1203 11:42:22.550995 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstackclient" Dec 03 11:42:22 crc kubenswrapper[5052]: I1203 11:42:22.554454 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/0c7d3107-6dd9-4aa2-a392-68a213013508-config-data\") pod \"placement-6d84cdf894-dqcg9\" (UID: \"0c7d3107-6dd9-4aa2-a392-68a213013508\") " pod="openstack-kuttl-tests/placement-6d84cdf894-dqcg9" Dec 03 11:42:22 crc kubenswrapper[5052]: I1203 11:42:22.554624 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/0c7d3107-6dd9-4aa2-a392-68a213013508-logs\") pod \"placement-6d84cdf894-dqcg9\" (UID: \"0c7d3107-6dd9-4aa2-a392-68a213013508\") " pod="openstack-kuttl-tests/placement-6d84cdf894-dqcg9" Dec 03 11:42:22 crc kubenswrapper[5052]: I1203 11:42:22.554722 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/0c7d3107-6dd9-4aa2-a392-68a213013508-scripts\") pod \"placement-6d84cdf894-dqcg9\" (UID: \"0c7d3107-6dd9-4aa2-a392-68a213013508\") " pod="openstack-kuttl-tests/placement-6d84cdf894-dqcg9" Dec 03 11:42:22 crc kubenswrapper[5052]: I1203 11:42:22.554799 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0c7d3107-6dd9-4aa2-a392-68a213013508-combined-ca-bundle\") pod \"placement-6d84cdf894-dqcg9\" (UID: \"0c7d3107-6dd9-4aa2-a392-68a213013508\") " pod="openstack-kuttl-tests/placement-6d84cdf894-dqcg9" Dec 03 11:42:22 crc kubenswrapper[5052]: I1203 11:42:22.554814 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/0c7d3107-6dd9-4aa2-a392-68a213013508-public-tls-certs\") pod \"placement-6d84cdf894-dqcg9\" (UID: \"0c7d3107-6dd9-4aa2-a392-68a213013508\") " pod="openstack-kuttl-tests/placement-6d84cdf894-dqcg9" Dec 03 11:42:22 crc kubenswrapper[5052]: I1203 11:42:22.554882 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wrjxc\" (UniqueName: \"kubernetes.io/projected/0c7d3107-6dd9-4aa2-a392-68a213013508-kube-api-access-wrjxc\") pod \"placement-6d84cdf894-dqcg9\" (UID: \"0c7d3107-6dd9-4aa2-a392-68a213013508\") " pod="openstack-kuttl-tests/placement-6d84cdf894-dqcg9" Dec 03 11:42:22 crc kubenswrapper[5052]: I1203 11:42:22.554946 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/0c7d3107-6dd9-4aa2-a392-68a213013508-internal-tls-certs\") pod \"placement-6d84cdf894-dqcg9\" (UID: \"0c7d3107-6dd9-4aa2-a392-68a213013508\") " pod="openstack-kuttl-tests/placement-6d84cdf894-dqcg9" Dec 03 11:42:22 crc kubenswrapper[5052]: I1203 11:42:22.583731 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-wzn2j\" (UniqueName: \"kubernetes.io/projected/3004edac-eef4-4ddc-a539-4d4294029774-kube-api-access-wzn2j\") pod \"barbican-keystone-listener-5dd6f4bfd-glsv7\" (UID: \"3004edac-eef4-4ddc-a539-4d4294029774\") " pod="openstack-kuttl-tests/barbican-keystone-listener-5dd6f4bfd-glsv7" Dec 03 11:42:22 crc kubenswrapper[5052]: I1203 11:42:22.692695 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-worker-67c775d6bc-kwxzt" Dec 03 11:42:22 crc kubenswrapper[5052]: I1203 11:42:22.697163 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/0c7d3107-6dd9-4aa2-a392-68a213013508-internal-tls-certs\") pod \"placement-6d84cdf894-dqcg9\" (UID: \"0c7d3107-6dd9-4aa2-a392-68a213013508\") " pod="openstack-kuttl-tests/placement-6d84cdf894-dqcg9" Dec 03 11:42:22 crc kubenswrapper[5052]: I1203 11:42:22.697225 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/0c7d3107-6dd9-4aa2-a392-68a213013508-config-data\") pod \"placement-6d84cdf894-dqcg9\" (UID: \"0c7d3107-6dd9-4aa2-a392-68a213013508\") " pod="openstack-kuttl-tests/placement-6d84cdf894-dqcg9" Dec 03 11:42:22 crc kubenswrapper[5052]: I1203 11:42:22.697366 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/0c7d3107-6dd9-4aa2-a392-68a213013508-logs\") pod \"placement-6d84cdf894-dqcg9\" (UID: \"0c7d3107-6dd9-4aa2-a392-68a213013508\") " pod="openstack-kuttl-tests/placement-6d84cdf894-dqcg9" Dec 03 11:42:22 crc kubenswrapper[5052]: I1203 11:42:22.697452 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/0c7d3107-6dd9-4aa2-a392-68a213013508-scripts\") pod \"placement-6d84cdf894-dqcg9\" (UID: \"0c7d3107-6dd9-4aa2-a392-68a213013508\") " pod="openstack-kuttl-tests/placement-6d84cdf894-dqcg9" Dec 03 11:42:22 crc kubenswrapper[5052]: I1203 11:42:22.697552 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0c7d3107-6dd9-4aa2-a392-68a213013508-combined-ca-bundle\") pod \"placement-6d84cdf894-dqcg9\" (UID: \"0c7d3107-6dd9-4aa2-a392-68a213013508\") " pod="openstack-kuttl-tests/placement-6d84cdf894-dqcg9" Dec 03 11:42:22 crc kubenswrapper[5052]: I1203 11:42:22.697573 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/0c7d3107-6dd9-4aa2-a392-68a213013508-public-tls-certs\") pod \"placement-6d84cdf894-dqcg9\" (UID: \"0c7d3107-6dd9-4aa2-a392-68a213013508\") " pod="openstack-kuttl-tests/placement-6d84cdf894-dqcg9" Dec 03 11:42:22 crc kubenswrapper[5052]: I1203 11:42:22.697634 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-wrjxc\" (UniqueName: \"kubernetes.io/projected/0c7d3107-6dd9-4aa2-a392-68a213013508-kube-api-access-wrjxc\") pod \"placement-6d84cdf894-dqcg9\" (UID: \"0c7d3107-6dd9-4aa2-a392-68a213013508\") " pod="openstack-kuttl-tests/placement-6d84cdf894-dqcg9" Dec 03 11:42:22 crc kubenswrapper[5052]: I1203 11:42:22.706431 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/0c7d3107-6dd9-4aa2-a392-68a213013508-logs\") pod \"placement-6d84cdf894-dqcg9\" (UID: \"0c7d3107-6dd9-4aa2-a392-68a213013508\") " pod="openstack-kuttl-tests/placement-6d84cdf894-dqcg9" Dec 03 11:42:22 crc kubenswrapper[5052]: I1203 11:42:22.711222 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/0c7d3107-6dd9-4aa2-a392-68a213013508-config-data\") pod \"placement-6d84cdf894-dqcg9\" (UID: \"0c7d3107-6dd9-4aa2-a392-68a213013508\") " pod="openstack-kuttl-tests/placement-6d84cdf894-dqcg9" Dec 03 11:42:22 crc kubenswrapper[5052]: I1203 11:42:22.713079 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/0c7d3107-6dd9-4aa2-a392-68a213013508-scripts\") pod \"placement-6d84cdf894-dqcg9\" (UID: \"0c7d3107-6dd9-4aa2-a392-68a213013508\") " pod="openstack-kuttl-tests/placement-6d84cdf894-dqcg9" Dec 03 11:42:22 crc kubenswrapper[5052]: I1203 11:42:22.713693 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/0c7d3107-6dd9-4aa2-a392-68a213013508-internal-tls-certs\") pod \"placement-6d84cdf894-dqcg9\" (UID: \"0c7d3107-6dd9-4aa2-a392-68a213013508\") " pod="openstack-kuttl-tests/placement-6d84cdf894-dqcg9" Dec 03 11:42:22 crc kubenswrapper[5052]: I1203 11:42:22.716080 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0c7d3107-6dd9-4aa2-a392-68a213013508-combined-ca-bundle\") pod \"placement-6d84cdf894-dqcg9\" (UID: \"0c7d3107-6dd9-4aa2-a392-68a213013508\") " pod="openstack-kuttl-tests/placement-6d84cdf894-dqcg9" Dec 03 11:42:22 crc kubenswrapper[5052]: I1203 11:42:22.724040 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-keystone-listener-5dd6f4bfd-glsv7" Dec 03 11:42:22 crc kubenswrapper[5052]: I1203 11:42:22.738023 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/0c7d3107-6dd9-4aa2-a392-68a213013508-public-tls-certs\") pod \"placement-6d84cdf894-dqcg9\" (UID: \"0c7d3107-6dd9-4aa2-a392-68a213013508\") " pod="openstack-kuttl-tests/placement-6d84cdf894-dqcg9" Dec 03 11:42:22 crc kubenswrapper[5052]: I1203 11:42:22.771623 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/barbican-api-df67d6cb4-8sllj"] Dec 03 11:42:22 crc kubenswrapper[5052]: I1203 11:42:22.785090 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-api-df67d6cb4-8sllj" Dec 03 11:42:22 crc kubenswrapper[5052]: I1203 11:42:22.812432 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-wrjxc\" (UniqueName: \"kubernetes.io/projected/0c7d3107-6dd9-4aa2-a392-68a213013508-kube-api-access-wrjxc\") pod \"placement-6d84cdf894-dqcg9\" (UID: \"0c7d3107-6dd9-4aa2-a392-68a213013508\") " pod="openstack-kuttl-tests/placement-6d84cdf894-dqcg9" Dec 03 11:42:22 crc kubenswrapper[5052]: I1203 11:42:22.819179 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-api-df67d6cb4-8sllj"] Dec 03 11:42:22 crc kubenswrapper[5052]: E1203 11:42:22.860437 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="2acdb0814bfda1b6c48a4fde354fd532f86ae71dcaac188c16bee44a794b3713" cmd=["/usr/bin/pgrep","-r","DRST","nova-scheduler"] Dec 03 11:42:22 crc kubenswrapper[5052]: I1203 11:42:22.860583 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 11:42:22 crc kubenswrapper[5052]: I1203 11:42:22.860769 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/cinder-api-0" podUID="ee46c3a4-5e21-4f6b-a944-8cf96d87506e" containerName="cinder-api-log" containerID="cri-o://f924c6cd365dded654c74f9f506f8cb2317b25d77d3b15ae01a497a3800a80db" gracePeriod=30 Dec 03 11:42:22 crc kubenswrapper[5052]: I1203 11:42:22.862153 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/cinder-api-0" podUID="ee46c3a4-5e21-4f6b-a944-8cf96d87506e" containerName="cinder-api" containerID="cri-o://a5e3cd0978d13d8f638fcff947df03b89089415c5babbbe442713809453149ae" gracePeriod=30 Dec 03 11:42:22 crc kubenswrapper[5052]: I1203 11:42:22.875691 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-nb-0"] Dec 03 11:42:22 crc kubenswrapper[5052]: I1203 11:42:22.876354 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ovsdbserver-nb-0" podUID="d7bd0c57-728c-46a4-bbec-2e209d9f4049" containerName="openstack-network-exporter" containerID="cri-o://959614b917450812745a85b82761fed66dc1a03b7fd08eb36921870d079f6214" gracePeriod=300 Dec 03 11:42:22 crc kubenswrapper[5052]: I1203 11:42:22.889618 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/openstack-cell1-galera-0"] Dec 03 11:42:22 crc kubenswrapper[5052]: I1203 11:42:22.917581 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/b88af976-65f2-4d29-8c14-d5c43e23a459-config-data\") pod \"barbican-api-df67d6cb4-8sllj\" (UID: \"b88af976-65f2-4d29-8c14-d5c43e23a459\") " pod="openstack-kuttl-tests/barbican-api-df67d6cb4-8sllj" Dec 03 11:42:22 crc kubenswrapper[5052]: I1203 11:42:22.917648 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/b88af976-65f2-4d29-8c14-d5c43e23a459-logs\") pod \"barbican-api-df67d6cb4-8sllj\" (UID: \"b88af976-65f2-4d29-8c14-d5c43e23a459\") " pod="openstack-kuttl-tests/barbican-api-df67d6cb4-8sllj" Dec 03 11:42:22 crc kubenswrapper[5052]: I1203 11:42:22.917668 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/b88af976-65f2-4d29-8c14-d5c43e23a459-public-tls-certs\") pod \"barbican-api-df67d6cb4-8sllj\" (UID: \"b88af976-65f2-4d29-8c14-d5c43e23a459\") " pod="openstack-kuttl-tests/barbican-api-df67d6cb4-8sllj" Dec 03 11:42:22 crc kubenswrapper[5052]: I1203 11:42:22.917759 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/b88af976-65f2-4d29-8c14-d5c43e23a459-config-data-custom\") pod \"barbican-api-df67d6cb4-8sllj\" (UID: \"b88af976-65f2-4d29-8c14-d5c43e23a459\") " pod="openstack-kuttl-tests/barbican-api-df67d6cb4-8sllj" Dec 03 11:42:22 crc kubenswrapper[5052]: I1203 11:42:22.917781 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vr472\" (UniqueName: \"kubernetes.io/projected/b88af976-65f2-4d29-8c14-d5c43e23a459-kube-api-access-vr472\") pod \"barbican-api-df67d6cb4-8sllj\" (UID: \"b88af976-65f2-4d29-8c14-d5c43e23a459\") " pod="openstack-kuttl-tests/barbican-api-df67d6cb4-8sllj" Dec 03 11:42:22 crc kubenswrapper[5052]: I1203 11:42:22.917849 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/b88af976-65f2-4d29-8c14-d5c43e23a459-internal-tls-certs\") pod \"barbican-api-df67d6cb4-8sllj\" (UID: \"b88af976-65f2-4d29-8c14-d5c43e23a459\") " pod="openstack-kuttl-tests/barbican-api-df67d6cb4-8sllj" Dec 03 11:42:22 crc kubenswrapper[5052]: I1203 11:42:22.917873 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b88af976-65f2-4d29-8c14-d5c43e23a459-combined-ca-bundle\") pod \"barbican-api-df67d6cb4-8sllj\" (UID: \"b88af976-65f2-4d29-8c14-d5c43e23a459\") " pod="openstack-kuttl-tests/barbican-api-df67d6cb4-8sllj" Dec 03 11:42:22 crc kubenswrapper[5052]: E1203 11:42:22.919465 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="2acdb0814bfda1b6c48a4fde354fd532f86ae71dcaac188c16bee44a794b3713" cmd=["/usr/bin/pgrep","-r","DRST","nova-scheduler"] Dec 03 11:42:22 crc kubenswrapper[5052]: I1203 11:42:22.929823 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-6d84cdf894-dqcg9" Dec 03 11:42:22 crc kubenswrapper[5052]: I1203 11:42:22.982572 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 11:42:22 crc kubenswrapper[5052]: I1203 11:42:22.982852 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-internal-api-0" podUID="665a736c-d550-4160-a8bb-619d1d5ea826" containerName="glance-log" containerID="cri-o://907dd58d6f36fcd52c737d7df15fd6614bc5eeddfe7082592f83e68d9f16190c" gracePeriod=30 Dec 03 11:42:22 crc kubenswrapper[5052]: I1203 11:42:22.982950 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-internal-api-0" podUID="665a736c-d550-4160-a8bb-619d1d5ea826" containerName="glance-httpd" containerID="cri-o://25c8b36672104ddddb9cda8135c26753231310450ea8bff9f6700b865bc3d056" gracePeriod=30 Dec 03 11:42:22 crc kubenswrapper[5052]: I1203 11:42:22.991427 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/neutron-69cf8cb7d6-jfb57"] Dec 03 11:42:22 crc kubenswrapper[5052]: I1203 11:42:22.993432 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-69cf8cb7d6-jfb57" Dec 03 11:42:23 crc kubenswrapper[5052]: I1203 11:42:23.001223 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-69cf8cb7d6-jfb57"] Dec 03 11:42:23 crc kubenswrapper[5052]: I1203 11:42:23.013302 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/keystone-6858c48994-zq757"] Dec 03 11:42:23 crc kubenswrapper[5052]: I1203 11:42:23.014914 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-6858c48994-zq757" Dec 03 11:42:23 crc kubenswrapper[5052]: I1203 11:42:23.023944 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/b88af976-65f2-4d29-8c14-d5c43e23a459-config-data-custom\") pod \"barbican-api-df67d6cb4-8sllj\" (UID: \"b88af976-65f2-4d29-8c14-d5c43e23a459\") " pod="openstack-kuttl-tests/barbican-api-df67d6cb4-8sllj" Dec 03 11:42:23 crc kubenswrapper[5052]: I1203 11:42:23.024004 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-vr472\" (UniqueName: \"kubernetes.io/projected/b88af976-65f2-4d29-8c14-d5c43e23a459-kube-api-access-vr472\") pod \"barbican-api-df67d6cb4-8sllj\" (UID: \"b88af976-65f2-4d29-8c14-d5c43e23a459\") " pod="openstack-kuttl-tests/barbican-api-df67d6cb4-8sllj" Dec 03 11:42:23 crc kubenswrapper[5052]: I1203 11:42:23.024091 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/b88af976-65f2-4d29-8c14-d5c43e23a459-internal-tls-certs\") pod \"barbican-api-df67d6cb4-8sllj\" (UID: \"b88af976-65f2-4d29-8c14-d5c43e23a459\") " pod="openstack-kuttl-tests/barbican-api-df67d6cb4-8sllj" Dec 03 11:42:23 crc kubenswrapper[5052]: I1203 11:42:23.024116 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b88af976-65f2-4d29-8c14-d5c43e23a459-combined-ca-bundle\") pod \"barbican-api-df67d6cb4-8sllj\" (UID: \"b88af976-65f2-4d29-8c14-d5c43e23a459\") " pod="openstack-kuttl-tests/barbican-api-df67d6cb4-8sllj" Dec 03 11:42:23 crc kubenswrapper[5052]: I1203 11:42:23.024170 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/b88af976-65f2-4d29-8c14-d5c43e23a459-config-data\") pod \"barbican-api-df67d6cb4-8sllj\" (UID: \"b88af976-65f2-4d29-8c14-d5c43e23a459\") " pod="openstack-kuttl-tests/barbican-api-df67d6cb4-8sllj" Dec 03 11:42:23 crc kubenswrapper[5052]: I1203 11:42:23.024229 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/b88af976-65f2-4d29-8c14-d5c43e23a459-logs\") pod \"barbican-api-df67d6cb4-8sllj\" (UID: \"b88af976-65f2-4d29-8c14-d5c43e23a459\") " pod="openstack-kuttl-tests/barbican-api-df67d6cb4-8sllj" Dec 03 11:42:23 crc kubenswrapper[5052]: I1203 11:42:23.024251 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/b88af976-65f2-4d29-8c14-d5c43e23a459-public-tls-certs\") pod \"barbican-api-df67d6cb4-8sllj\" (UID: \"b88af976-65f2-4d29-8c14-d5c43e23a459\") " pod="openstack-kuttl-tests/barbican-api-df67d6cb4-8sllj" Dec 03 11:42:23 crc kubenswrapper[5052]: E1203 11:42:23.037575 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="2acdb0814bfda1b6c48a4fde354fd532f86ae71dcaac188c16bee44a794b3713" cmd=["/usr/bin/pgrep","-r","DRST","nova-scheduler"] Dec 03 11:42:23 crc kubenswrapper[5052]: E1203 11:42:23.037637 5052 prober.go:104] "Probe errored" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" probeType="Readiness" pod="openstack-kuttl-tests/nova-scheduler-0" podUID="d80d1476-351a-40c6-9f76-ce66d623df74" containerName="nova-scheduler-scheduler" Dec 03 11:42:23 crc kubenswrapper[5052]: I1203 11:42:23.040949 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-6858c48994-zq757"] Dec 03 11:42:23 crc kubenswrapper[5052]: I1203 11:42:23.041342 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/b88af976-65f2-4d29-8c14-d5c43e23a459-logs\") pod \"barbican-api-df67d6cb4-8sllj\" (UID: \"b88af976-65f2-4d29-8c14-d5c43e23a459\") " pod="openstack-kuttl-tests/barbican-api-df67d6cb4-8sllj" Dec 03 11:42:23 crc kubenswrapper[5052]: I1203 11:42:23.042257 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/b88af976-65f2-4d29-8c14-d5c43e23a459-config-data\") pod \"barbican-api-df67d6cb4-8sllj\" (UID: \"b88af976-65f2-4d29-8c14-d5c43e23a459\") " pod="openstack-kuttl-tests/barbican-api-df67d6cb4-8sllj" Dec 03 11:42:23 crc kubenswrapper[5052]: I1203 11:42:23.052718 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/b88af976-65f2-4d29-8c14-d5c43e23a459-public-tls-certs\") pod \"barbican-api-df67d6cb4-8sllj\" (UID: \"b88af976-65f2-4d29-8c14-d5c43e23a459\") " pod="openstack-kuttl-tests/barbican-api-df67d6cb4-8sllj" Dec 03 11:42:23 crc kubenswrapper[5052]: I1203 11:42:23.059626 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/b88af976-65f2-4d29-8c14-d5c43e23a459-config-data-custom\") pod \"barbican-api-df67d6cb4-8sllj\" (UID: \"b88af976-65f2-4d29-8c14-d5c43e23a459\") " pod="openstack-kuttl-tests/barbican-api-df67d6cb4-8sllj" Dec 03 11:42:23 crc kubenswrapper[5052]: I1203 11:42:23.066892 5052 generic.go:334] "Generic (PLEG): container finished" podID="82dce330-8f65-4c14-80d4-958eb8ca5212" containerID="b939d536bc53d88cf0c53f3e90f1a9bbe204dc2f74470b14127832658e7f3898" exitCode=143 Dec 03 11:42:23 crc kubenswrapper[5052]: I1203 11:42:23.066986 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"82dce330-8f65-4c14-80d4-958eb8ca5212","Type":"ContainerDied","Data":"b939d536bc53d88cf0c53f3e90f1a9bbe204dc2f74470b14127832658e7f3898"} Dec 03 11:42:23 crc kubenswrapper[5052]: I1203 11:42:23.067580 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b88af976-65f2-4d29-8c14-d5c43e23a459-combined-ca-bundle\") pod \"barbican-api-df67d6cb4-8sllj\" (UID: \"b88af976-65f2-4d29-8c14-d5c43e23a459\") " pod="openstack-kuttl-tests/barbican-api-df67d6cb4-8sllj" Dec 03 11:42:23 crc kubenswrapper[5052]: I1203 11:42:23.069057 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/b88af976-65f2-4d29-8c14-d5c43e23a459-internal-tls-certs\") pod \"barbican-api-df67d6cb4-8sllj\" (UID: \"b88af976-65f2-4d29-8c14-d5c43e23a459\") " pod="openstack-kuttl-tests/barbican-api-df67d6cb4-8sllj" Dec 03 11:42:23 crc kubenswrapper[5052]: I1203 11:42:23.090097 5052 generic.go:334] "Generic (PLEG): container finished" podID="e14d926b-5dc2-46a4-ab27-dfde118cba89" containerID="8da90eb5a7870f9bbe93a949f459805faa0302426245aa70e016cd57e8992d74" exitCode=2 Dec 03 11:42:23 crc kubenswrapper[5052]: I1203 11:42:23.090194 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-sb-0" event={"ID":"e14d926b-5dc2-46a4-ab27-dfde118cba89","Type":"ContainerDied","Data":"8da90eb5a7870f9bbe93a949f459805faa0302426245aa70e016cd57e8992d74"} Dec 03 11:42:23 crc kubenswrapper[5052]: I1203 11:42:23.121949 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ovsdbserver-nb-0" podUID="d7bd0c57-728c-46a4-bbec-2e209d9f4049" containerName="ovsdbserver-nb" containerID="cri-o://d8d8be4b59c92e57d45d986195fe45d17c86d578df37f5f3af30a57358eda608" gracePeriod=300 Dec 03 11:42:23 crc kubenswrapper[5052]: I1203 11:42:23.125622 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/8f1442a2-a28b-4784-934a-34b417dbcce9-ovndb-tls-certs\") pod \"neutron-69cf8cb7d6-jfb57\" (UID: \"8f1442a2-a28b-4784-934a-34b417dbcce9\") " pod="openstack-kuttl-tests/neutron-69cf8cb7d6-jfb57" Dec 03 11:42:23 crc kubenswrapper[5052]: I1203 11:42:23.125680 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8f1442a2-a28b-4784-934a-34b417dbcce9-combined-ca-bundle\") pod \"neutron-69cf8cb7d6-jfb57\" (UID: \"8f1442a2-a28b-4784-934a-34b417dbcce9\") " pod="openstack-kuttl-tests/neutron-69cf8cb7d6-jfb57" Dec 03 11:42:23 crc kubenswrapper[5052]: I1203 11:42:23.125705 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/8f1442a2-a28b-4784-934a-34b417dbcce9-internal-tls-certs\") pod \"neutron-69cf8cb7d6-jfb57\" (UID: \"8f1442a2-a28b-4784-934a-34b417dbcce9\") " pod="openstack-kuttl-tests/neutron-69cf8cb7d6-jfb57" Dec 03 11:42:23 crc kubenswrapper[5052]: I1203 11:42:23.125750 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/48409c36-734b-4b1d-adee-0e9981d635c9-config-data\") pod \"keystone-6858c48994-zq757\" (UID: \"48409c36-734b-4b1d-adee-0e9981d635c9\") " pod="openstack-kuttl-tests/keystone-6858c48994-zq757" Dec 03 11:42:23 crc kubenswrapper[5052]: I1203 11:42:23.125774 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8r4bn\" (UniqueName: \"kubernetes.io/projected/48409c36-734b-4b1d-adee-0e9981d635c9-kube-api-access-8r4bn\") pod \"keystone-6858c48994-zq757\" (UID: \"48409c36-734b-4b1d-adee-0e9981d635c9\") " pod="openstack-kuttl-tests/keystone-6858c48994-zq757" Dec 03 11:42:23 crc kubenswrapper[5052]: I1203 11:42:23.125812 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/48409c36-734b-4b1d-adee-0e9981d635c9-public-tls-certs\") pod \"keystone-6858c48994-zq757\" (UID: \"48409c36-734b-4b1d-adee-0e9981d635c9\") " pod="openstack-kuttl-tests/keystone-6858c48994-zq757" Dec 03 11:42:23 crc kubenswrapper[5052]: I1203 11:42:23.125858 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/8f1442a2-a28b-4784-934a-34b417dbcce9-public-tls-certs\") pod \"neutron-69cf8cb7d6-jfb57\" (UID: \"8f1442a2-a28b-4784-934a-34b417dbcce9\") " pod="openstack-kuttl-tests/neutron-69cf8cb7d6-jfb57" Dec 03 11:42:23 crc kubenswrapper[5052]: I1203 11:42:23.125878 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/8f1442a2-a28b-4784-934a-34b417dbcce9-config\") pod \"neutron-69cf8cb7d6-jfb57\" (UID: \"8f1442a2-a28b-4784-934a-34b417dbcce9\") " pod="openstack-kuttl-tests/neutron-69cf8cb7d6-jfb57" Dec 03 11:42:23 crc kubenswrapper[5052]: I1203 11:42:23.125902 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/48409c36-734b-4b1d-adee-0e9981d635c9-fernet-keys\") pod \"keystone-6858c48994-zq757\" (UID: \"48409c36-734b-4b1d-adee-0e9981d635c9\") " pod="openstack-kuttl-tests/keystone-6858c48994-zq757" Dec 03 11:42:23 crc kubenswrapper[5052]: I1203 11:42:23.125929 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cbfzg\" (UniqueName: \"kubernetes.io/projected/8f1442a2-a28b-4784-934a-34b417dbcce9-kube-api-access-cbfzg\") pod \"neutron-69cf8cb7d6-jfb57\" (UID: \"8f1442a2-a28b-4784-934a-34b417dbcce9\") " pod="openstack-kuttl-tests/neutron-69cf8cb7d6-jfb57" Dec 03 11:42:23 crc kubenswrapper[5052]: I1203 11:42:23.125943 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/8f1442a2-a28b-4784-934a-34b417dbcce9-httpd-config\") pod \"neutron-69cf8cb7d6-jfb57\" (UID: \"8f1442a2-a28b-4784-934a-34b417dbcce9\") " pod="openstack-kuttl-tests/neutron-69cf8cb7d6-jfb57" Dec 03 11:42:23 crc kubenswrapper[5052]: I1203 11:42:23.125962 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/48409c36-734b-4b1d-adee-0e9981d635c9-scripts\") pod \"keystone-6858c48994-zq757\" (UID: \"48409c36-734b-4b1d-adee-0e9981d635c9\") " pod="openstack-kuttl-tests/keystone-6858c48994-zq757" Dec 03 11:42:23 crc kubenswrapper[5052]: I1203 11:42:23.125979 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/48409c36-734b-4b1d-adee-0e9981d635c9-internal-tls-certs\") pod \"keystone-6858c48994-zq757\" (UID: \"48409c36-734b-4b1d-adee-0e9981d635c9\") " pod="openstack-kuttl-tests/keystone-6858c48994-zq757" Dec 03 11:42:23 crc kubenswrapper[5052]: I1203 11:42:23.126034 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/48409c36-734b-4b1d-adee-0e9981d635c9-credential-keys\") pod \"keystone-6858c48994-zq757\" (UID: \"48409c36-734b-4b1d-adee-0e9981d635c9\") " pod="openstack-kuttl-tests/keystone-6858c48994-zq757" Dec 03 11:42:23 crc kubenswrapper[5052]: I1203 11:42:23.126058 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/48409c36-734b-4b1d-adee-0e9981d635c9-combined-ca-bundle\") pod \"keystone-6858c48994-zq757\" (UID: \"48409c36-734b-4b1d-adee-0e9981d635c9\") " pod="openstack-kuttl-tests/keystone-6858c48994-zq757" Dec 03 11:42:23 crc kubenswrapper[5052]: I1203 11:42:23.130722 5052 generic.go:334] "Generic (PLEG): container finished" podID="a30eaf15-65ff-426b-8ac2-1ed5a369886a" containerID="abd32ba7a2fea1f9280ace25d3fe52938520b763a092b3e8d9e1b4a4f4e6e2be" exitCode=2 Dec 03 11:42:23 crc kubenswrapper[5052]: I1203 11:42:23.130782 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovn-northd-0" event={"ID":"a30eaf15-65ff-426b-8ac2-1ed5a369886a","Type":"ContainerDied","Data":"abd32ba7a2fea1f9280ace25d3fe52938520b763a092b3e8d9e1b4a4f4e6e2be"} Dec 03 11:42:23 crc kubenswrapper[5052]: I1203 11:42:23.148155 5052 generic.go:334] "Generic (PLEG): container finished" podID="f98f227a-397b-4665-bf18-e00b105ce058" containerID="9a73f69a43329d67c0d372934e250290b21f5530d0b0079f8192b4709695cb6a" exitCode=143 Dec 03 11:42:23 crc kubenswrapper[5052]: I1203 11:42:23.148484 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"f98f227a-397b-4665-bf18-e00b105ce058","Type":"ContainerDied","Data":"9a73f69a43329d67c0d372934e250290b21f5530d0b0079f8192b4709695cb6a"} Dec 03 11:42:23 crc kubenswrapper[5052]: I1203 11:42:23.164466 5052 generic.go:334] "Generic (PLEG): container finished" podID="75e4b34e-f7b8-4209-95a6-6cc45f6eb57c" containerID="99cb39696946884496d7905d0018b8b8e349cbda3cdb669d009adf92f668080b" exitCode=143 Dec 03 11:42:23 crc kubenswrapper[5052]: I1203 11:42:23.164503 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"75e4b34e-f7b8-4209-95a6-6cc45f6eb57c","Type":"ContainerDied","Data":"99cb39696946884496d7905d0018b8b8e349cbda3cdb669d009adf92f668080b"} Dec 03 11:42:23 crc kubenswrapper[5052]: I1203 11:42:23.186339 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-vr472\" (UniqueName: \"kubernetes.io/projected/b88af976-65f2-4d29-8c14-d5c43e23a459-kube-api-access-vr472\") pod \"barbican-api-df67d6cb4-8sllj\" (UID: \"b88af976-65f2-4d29-8c14-d5c43e23a459\") " pod="openstack-kuttl-tests/barbican-api-df67d6cb4-8sllj" Dec 03 11:42:23 crc kubenswrapper[5052]: I1203 11:42:23.246948 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/48409c36-734b-4b1d-adee-0e9981d635c9-credential-keys\") pod \"keystone-6858c48994-zq757\" (UID: \"48409c36-734b-4b1d-adee-0e9981d635c9\") " pod="openstack-kuttl-tests/keystone-6858c48994-zq757" Dec 03 11:42:23 crc kubenswrapper[5052]: I1203 11:42:23.247000 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/48409c36-734b-4b1d-adee-0e9981d635c9-combined-ca-bundle\") pod \"keystone-6858c48994-zq757\" (UID: \"48409c36-734b-4b1d-adee-0e9981d635c9\") " pod="openstack-kuttl-tests/keystone-6858c48994-zq757" Dec 03 11:42:23 crc kubenswrapper[5052]: I1203 11:42:23.247054 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/8f1442a2-a28b-4784-934a-34b417dbcce9-ovndb-tls-certs\") pod \"neutron-69cf8cb7d6-jfb57\" (UID: \"8f1442a2-a28b-4784-934a-34b417dbcce9\") " pod="openstack-kuttl-tests/neutron-69cf8cb7d6-jfb57" Dec 03 11:42:23 crc kubenswrapper[5052]: I1203 11:42:23.247077 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8f1442a2-a28b-4784-934a-34b417dbcce9-combined-ca-bundle\") pod \"neutron-69cf8cb7d6-jfb57\" (UID: \"8f1442a2-a28b-4784-934a-34b417dbcce9\") " pod="openstack-kuttl-tests/neutron-69cf8cb7d6-jfb57" Dec 03 11:42:23 crc kubenswrapper[5052]: I1203 11:42:23.247101 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/8f1442a2-a28b-4784-934a-34b417dbcce9-internal-tls-certs\") pod \"neutron-69cf8cb7d6-jfb57\" (UID: \"8f1442a2-a28b-4784-934a-34b417dbcce9\") " pod="openstack-kuttl-tests/neutron-69cf8cb7d6-jfb57" Dec 03 11:42:23 crc kubenswrapper[5052]: I1203 11:42:23.247146 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/48409c36-734b-4b1d-adee-0e9981d635c9-config-data\") pod \"keystone-6858c48994-zq757\" (UID: \"48409c36-734b-4b1d-adee-0e9981d635c9\") " pod="openstack-kuttl-tests/keystone-6858c48994-zq757" Dec 03 11:42:23 crc kubenswrapper[5052]: I1203 11:42:23.247169 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-8r4bn\" (UniqueName: \"kubernetes.io/projected/48409c36-734b-4b1d-adee-0e9981d635c9-kube-api-access-8r4bn\") pod \"keystone-6858c48994-zq757\" (UID: \"48409c36-734b-4b1d-adee-0e9981d635c9\") " pod="openstack-kuttl-tests/keystone-6858c48994-zq757" Dec 03 11:42:23 crc kubenswrapper[5052]: I1203 11:42:23.247206 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/48409c36-734b-4b1d-adee-0e9981d635c9-public-tls-certs\") pod \"keystone-6858c48994-zq757\" (UID: \"48409c36-734b-4b1d-adee-0e9981d635c9\") " pod="openstack-kuttl-tests/keystone-6858c48994-zq757" Dec 03 11:42:23 crc kubenswrapper[5052]: I1203 11:42:23.247265 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/8f1442a2-a28b-4784-934a-34b417dbcce9-public-tls-certs\") pod \"neutron-69cf8cb7d6-jfb57\" (UID: \"8f1442a2-a28b-4784-934a-34b417dbcce9\") " pod="openstack-kuttl-tests/neutron-69cf8cb7d6-jfb57" Dec 03 11:42:23 crc kubenswrapper[5052]: I1203 11:42:23.247291 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/8f1442a2-a28b-4784-934a-34b417dbcce9-config\") pod \"neutron-69cf8cb7d6-jfb57\" (UID: \"8f1442a2-a28b-4784-934a-34b417dbcce9\") " pod="openstack-kuttl-tests/neutron-69cf8cb7d6-jfb57" Dec 03 11:42:23 crc kubenswrapper[5052]: I1203 11:42:23.247320 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/48409c36-734b-4b1d-adee-0e9981d635c9-fernet-keys\") pod \"keystone-6858c48994-zq757\" (UID: \"48409c36-734b-4b1d-adee-0e9981d635c9\") " pod="openstack-kuttl-tests/keystone-6858c48994-zq757" Dec 03 11:42:23 crc kubenswrapper[5052]: I1203 11:42:23.247351 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-cbfzg\" (UniqueName: \"kubernetes.io/projected/8f1442a2-a28b-4784-934a-34b417dbcce9-kube-api-access-cbfzg\") pod \"neutron-69cf8cb7d6-jfb57\" (UID: \"8f1442a2-a28b-4784-934a-34b417dbcce9\") " pod="openstack-kuttl-tests/neutron-69cf8cb7d6-jfb57" Dec 03 11:42:23 crc kubenswrapper[5052]: I1203 11:42:23.247372 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/8f1442a2-a28b-4784-934a-34b417dbcce9-httpd-config\") pod \"neutron-69cf8cb7d6-jfb57\" (UID: \"8f1442a2-a28b-4784-934a-34b417dbcce9\") " pod="openstack-kuttl-tests/neutron-69cf8cb7d6-jfb57" Dec 03 11:42:23 crc kubenswrapper[5052]: I1203 11:42:23.247395 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/48409c36-734b-4b1d-adee-0e9981d635c9-scripts\") pod \"keystone-6858c48994-zq757\" (UID: \"48409c36-734b-4b1d-adee-0e9981d635c9\") " pod="openstack-kuttl-tests/keystone-6858c48994-zq757" Dec 03 11:42:23 crc kubenswrapper[5052]: I1203 11:42:23.247417 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/48409c36-734b-4b1d-adee-0e9981d635c9-internal-tls-certs\") pod \"keystone-6858c48994-zq757\" (UID: \"48409c36-734b-4b1d-adee-0e9981d635c9\") " pod="openstack-kuttl-tests/keystone-6858c48994-zq757" Dec 03 11:42:23 crc kubenswrapper[5052]: I1203 11:42:23.253162 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8f1442a2-a28b-4784-934a-34b417dbcce9-combined-ca-bundle\") pod \"neutron-69cf8cb7d6-jfb57\" (UID: \"8f1442a2-a28b-4784-934a-34b417dbcce9\") " pod="openstack-kuttl-tests/neutron-69cf8cb7d6-jfb57" Dec 03 11:42:23 crc kubenswrapper[5052]: I1203 11:42:23.254834 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/48409c36-734b-4b1d-adee-0e9981d635c9-public-tls-certs\") pod \"keystone-6858c48994-zq757\" (UID: \"48409c36-734b-4b1d-adee-0e9981d635c9\") " pod="openstack-kuttl-tests/keystone-6858c48994-zq757" Dec 03 11:42:23 crc kubenswrapper[5052]: I1203 11:42:23.265296 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/48409c36-734b-4b1d-adee-0e9981d635c9-internal-tls-certs\") pod \"keystone-6858c48994-zq757\" (UID: \"48409c36-734b-4b1d-adee-0e9981d635c9\") " pod="openstack-kuttl-tests/keystone-6858c48994-zq757" Dec 03 11:42:23 crc kubenswrapper[5052]: I1203 11:42:23.270682 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/8f1442a2-a28b-4784-934a-34b417dbcce9-config\") pod \"neutron-69cf8cb7d6-jfb57\" (UID: \"8f1442a2-a28b-4784-934a-34b417dbcce9\") " pod="openstack-kuttl-tests/neutron-69cf8cb7d6-jfb57" Dec 03 11:42:23 crc kubenswrapper[5052]: I1203 11:42:23.272057 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/48409c36-734b-4b1d-adee-0e9981d635c9-combined-ca-bundle\") pod \"keystone-6858c48994-zq757\" (UID: \"48409c36-734b-4b1d-adee-0e9981d635c9\") " pod="openstack-kuttl-tests/keystone-6858c48994-zq757" Dec 03 11:42:23 crc kubenswrapper[5052]: I1203 11:42:23.272709 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/8f1442a2-a28b-4784-934a-34b417dbcce9-public-tls-certs\") pod \"neutron-69cf8cb7d6-jfb57\" (UID: \"8f1442a2-a28b-4784-934a-34b417dbcce9\") " pod="openstack-kuttl-tests/neutron-69cf8cb7d6-jfb57" Dec 03 11:42:23 crc kubenswrapper[5052]: I1203 11:42:23.274236 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/48409c36-734b-4b1d-adee-0e9981d635c9-credential-keys\") pod \"keystone-6858c48994-zq757\" (UID: \"48409c36-734b-4b1d-adee-0e9981d635c9\") " pod="openstack-kuttl-tests/keystone-6858c48994-zq757" Dec 03 11:42:23 crc kubenswrapper[5052]: I1203 11:42:23.295482 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/48409c36-734b-4b1d-adee-0e9981d635c9-fernet-keys\") pod \"keystone-6858c48994-zq757\" (UID: \"48409c36-734b-4b1d-adee-0e9981d635c9\") " pod="openstack-kuttl-tests/keystone-6858c48994-zq757" Dec 03 11:42:23 crc kubenswrapper[5052]: I1203 11:42:23.297209 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/48409c36-734b-4b1d-adee-0e9981d635c9-config-data\") pod \"keystone-6858c48994-zq757\" (UID: \"48409c36-734b-4b1d-adee-0e9981d635c9\") " pod="openstack-kuttl-tests/keystone-6858c48994-zq757" Dec 03 11:42:23 crc kubenswrapper[5052]: I1203 11:42:23.298025 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/8f1442a2-a28b-4784-934a-34b417dbcce9-httpd-config\") pod \"neutron-69cf8cb7d6-jfb57\" (UID: \"8f1442a2-a28b-4784-934a-34b417dbcce9\") " pod="openstack-kuttl-tests/neutron-69cf8cb7d6-jfb57" Dec 03 11:42:23 crc kubenswrapper[5052]: I1203 11:42:23.306164 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/8f1442a2-a28b-4784-934a-34b417dbcce9-ovndb-tls-certs\") pod \"neutron-69cf8cb7d6-jfb57\" (UID: \"8f1442a2-a28b-4784-934a-34b417dbcce9\") " pod="openstack-kuttl-tests/neutron-69cf8cb7d6-jfb57" Dec 03 11:42:23 crc kubenswrapper[5052]: I1203 11:42:23.308088 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-8r4bn\" (UniqueName: \"kubernetes.io/projected/48409c36-734b-4b1d-adee-0e9981d635c9-kube-api-access-8r4bn\") pod \"keystone-6858c48994-zq757\" (UID: \"48409c36-734b-4b1d-adee-0e9981d635c9\") " pod="openstack-kuttl-tests/keystone-6858c48994-zq757" Dec 03 11:42:23 crc kubenswrapper[5052]: I1203 11:42:23.310255 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/8f1442a2-a28b-4784-934a-34b417dbcce9-internal-tls-certs\") pod \"neutron-69cf8cb7d6-jfb57\" (UID: \"8f1442a2-a28b-4784-934a-34b417dbcce9\") " pod="openstack-kuttl-tests/neutron-69cf8cb7d6-jfb57" Dec 03 11:42:23 crc kubenswrapper[5052]: I1203 11:42:23.324748 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/48409c36-734b-4b1d-adee-0e9981d635c9-scripts\") pod \"keystone-6858c48994-zq757\" (UID: \"48409c36-734b-4b1d-adee-0e9981d635c9\") " pod="openstack-kuttl-tests/keystone-6858c48994-zq757" Dec 03 11:42:23 crc kubenswrapper[5052]: I1203 11:42:23.325792 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-cbfzg\" (UniqueName: \"kubernetes.io/projected/8f1442a2-a28b-4784-934a-34b417dbcce9-kube-api-access-cbfzg\") pod \"neutron-69cf8cb7d6-jfb57\" (UID: \"8f1442a2-a28b-4784-934a-34b417dbcce9\") " pod="openstack-kuttl-tests/neutron-69cf8cb7d6-jfb57" Dec 03 11:42:23 crc kubenswrapper[5052]: I1203 11:42:23.379192 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/openstack-galera-0" podUID="251248ba-578c-49cf-b77e-fc3d5c6cb6fd" containerName="galera" containerID="cri-o://5e0a4e3bdf4887c1a95e0d9815e6d8a1fc6c0dccf8359c3a1a7a1505e8cc9053" gracePeriod=30 Dec 03 11:42:23 crc kubenswrapper[5052]: I1203 11:42:23.400048 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-69cf8cb7d6-jfb57" Dec 03 11:42:23 crc kubenswrapper[5052]: I1203 11:42:23.448393 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-6858c48994-zq757" Dec 03 11:42:23 crc kubenswrapper[5052]: I1203 11:42:23.468153 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-0"] Dec 03 11:42:23 crc kubenswrapper[5052]: I1203 11:42:23.468386 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-cell1-conductor-0" podUID="2501eda8-a78d-4609-a465-761e89bb291e" containerName="nova-cell1-conductor-conductor" containerID="cri-o://1714a77b6dd6ea9a3fe18b86a4ab32f16a2525fd31478012ffe19746b1260454" gracePeriod=30 Dec 03 11:42:23 crc kubenswrapper[5052]: I1203 11:42:23.468770 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-api-df67d6cb4-8sllj" Dec 03 11:42:23 crc kubenswrapper[5052]: I1203 11:42:23.532312 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/openstack-cell1-galera-0" podUID="ecbf8b8e-cdb4-45c3-9455-d0dbc551356e" containerName="galera" containerID="cri-o://d9bef6e726b038ae865fa13770a700b7a8625ffee06c08933704155c1695d94f" gracePeriod=30 Dec 03 11:42:23 crc kubenswrapper[5052]: E1203 11:42:23.532624 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of ef90774de4b83b9fc123ba664ece34787dcdb1ad9fd0d230dacf72b8eee1a1e1 is running failed: container process not found" containerID="ef90774de4b83b9fc123ba664ece34787dcdb1ad9fd0d230dacf72b8eee1a1e1" cmd=["/usr/bin/pidof","ovsdb-server"] Dec 03 11:42:23 crc kubenswrapper[5052]: E1203 11:42:23.533046 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of ef90774de4b83b9fc123ba664ece34787dcdb1ad9fd0d230dacf72b8eee1a1e1 is running failed: container process not found" containerID="ef90774de4b83b9fc123ba664ece34787dcdb1ad9fd0d230dacf72b8eee1a1e1" cmd=["/usr/bin/pidof","ovsdb-server"] Dec 03 11:42:23 crc kubenswrapper[5052]: E1203 11:42:23.533456 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of ef90774de4b83b9fc123ba664ece34787dcdb1ad9fd0d230dacf72b8eee1a1e1 is running failed: container process not found" containerID="ef90774de4b83b9fc123ba664ece34787dcdb1ad9fd0d230dacf72b8eee1a1e1" cmd=["/usr/bin/pidof","ovsdb-server"] Dec 03 11:42:23 crc kubenswrapper[5052]: E1203 11:42:23.533484 5052 prober.go:104] "Probe errored" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of ef90774de4b83b9fc123ba664ece34787dcdb1ad9fd0d230dacf72b8eee1a1e1 is running failed: container process not found" probeType="Readiness" pod="openstack-kuttl-tests/ovsdbserver-sb-0" podUID="e14d926b-5dc2-46a4-ab27-dfde118cba89" containerName="ovsdbserver-sb" Dec 03 11:42:23 crc kubenswrapper[5052]: I1203 11:42:23.617414 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-worker-67c775d6bc-kwxzt"] Dec 03 11:42:23 crc kubenswrapper[5052]: W1203 11:42:23.770665 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod222f033c_3660_4347_b621_7122c4c94929.slice/crio-a5ac6cc1055df402445233edeff6fd6db77ec35aa50f1a126fefc71ff474afe2 WatchSource:0}: Error finding container a5ac6cc1055df402445233edeff6fd6db77ec35aa50f1a126fefc71ff474afe2: Status 404 returned error can't find the container with id a5ac6cc1055df402445233edeff6fd6db77ec35aa50f1a126fefc71ff474afe2 Dec 03 11:42:23 crc kubenswrapper[5052]: E1203 11:42:23.775056 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="9cfabfd91ca0525b54d3922c76017cb0defbc772bca498498fc287ac2175e2aa" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 11:42:23 crc kubenswrapper[5052]: E1203 11:42:23.794394 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="9cfabfd91ca0525b54d3922c76017cb0defbc772bca498498fc287ac2175e2aa" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 11:42:23 crc kubenswrapper[5052]: E1203 11:42:23.797688 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="9cfabfd91ca0525b54d3922c76017cb0defbc772bca498498fc287ac2175e2aa" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 11:42:23 crc kubenswrapper[5052]: E1203 11:42:23.797767 5052 prober.go:104] "Probe errored" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" probeType="Readiness" pod="openstack-kuttl-tests/nova-cell0-conductor-0" podUID="f8cdf05e-7081-4542-844d-f76378e8989f" containerName="nova-cell0-conductor-conductor" Dec 03 11:42:23 crc kubenswrapper[5052]: I1203 11:42:23.914991 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_ovsdbserver-nb-0_d7bd0c57-728c-46a4-bbec-2e209d9f4049/ovsdbserver-nb/0.log" Dec 03 11:42:23 crc kubenswrapper[5052]: I1203 11:42:23.915328 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 11:42:24 crc kubenswrapper[5052]: E1203 11:42:24.085894 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="5e0a4e3bdf4887c1a95e0d9815e6d8a1fc6c0dccf8359c3a1a7a1505e8cc9053" cmd=["/bin/bash","/var/lib/operator-scripts/mysql_probe.sh","readiness"] Dec 03 11:42:24 crc kubenswrapper[5052]: I1203 11:42:24.086359 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d7bd0c57-728c-46a4-bbec-2e209d9f4049-config\") pod \"d7bd0c57-728c-46a4-bbec-2e209d9f4049\" (UID: \"d7bd0c57-728c-46a4-bbec-2e209d9f4049\") " Dec 03 11:42:24 crc kubenswrapper[5052]: I1203 11:42:24.086481 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/d7bd0c57-728c-46a4-bbec-2e209d9f4049-ovsdb-rundir\") pod \"d7bd0c57-728c-46a4-bbec-2e209d9f4049\" (UID: \"d7bd0c57-728c-46a4-bbec-2e209d9f4049\") " Dec 03 11:42:24 crc kubenswrapper[5052]: I1203 11:42:24.086586 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovndbcluster-nb-etc-ovn\" (UniqueName: \"kubernetes.io/local-volume/local-storage14-crc\") pod \"d7bd0c57-728c-46a4-bbec-2e209d9f4049\" (UID: \"d7bd0c57-728c-46a4-bbec-2e209d9f4049\") " Dec 03 11:42:24 crc kubenswrapper[5052]: I1203 11:42:24.086655 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8n24z\" (UniqueName: \"kubernetes.io/projected/d7bd0c57-728c-46a4-bbec-2e209d9f4049-kube-api-access-8n24z\") pod \"d7bd0c57-728c-46a4-bbec-2e209d9f4049\" (UID: \"d7bd0c57-728c-46a4-bbec-2e209d9f4049\") " Dec 03 11:42:24 crc kubenswrapper[5052]: I1203 11:42:24.089198 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovsdbserver-nb-tls-certs\" (UniqueName: \"kubernetes.io/secret/d7bd0c57-728c-46a4-bbec-2e209d9f4049-ovsdbserver-nb-tls-certs\") pod \"d7bd0c57-728c-46a4-bbec-2e209d9f4049\" (UID: \"d7bd0c57-728c-46a4-bbec-2e209d9f4049\") " Dec 03 11:42:24 crc kubenswrapper[5052]: I1203 11:42:24.089278 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/d7bd0c57-728c-46a4-bbec-2e209d9f4049-scripts\") pod \"d7bd0c57-728c-46a4-bbec-2e209d9f4049\" (UID: \"d7bd0c57-728c-46a4-bbec-2e209d9f4049\") " Dec 03 11:42:24 crc kubenswrapper[5052]: I1203 11:42:24.089328 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/d7bd0c57-728c-46a4-bbec-2e209d9f4049-metrics-certs-tls-certs\") pod \"d7bd0c57-728c-46a4-bbec-2e209d9f4049\" (UID: \"d7bd0c57-728c-46a4-bbec-2e209d9f4049\") " Dec 03 11:42:24 crc kubenswrapper[5052]: I1203 11:42:24.089363 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d7bd0c57-728c-46a4-bbec-2e209d9f4049-combined-ca-bundle\") pod \"d7bd0c57-728c-46a4-bbec-2e209d9f4049\" (UID: \"d7bd0c57-728c-46a4-bbec-2e209d9f4049\") " Dec 03 11:42:24 crc kubenswrapper[5052]: I1203 11:42:24.094816 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:42:24 crc kubenswrapper[5052]: I1203 11:42:24.095088 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="eae5b719-0017-4621-9cb4-73ed6353648e" containerName="ceilometer-central-agent" containerID="cri-o://7d22ae141a66b8ccb952b7ed5aa56004799cd8db057a131cc24fa1331ea7e389" gracePeriod=30 Dec 03 11:42:24 crc kubenswrapper[5052]: I1203 11:42:24.095539 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="eae5b719-0017-4621-9cb4-73ed6353648e" containerName="proxy-httpd" containerID="cri-o://aa0bc44913bc443884760e2a18faf9da4a9e4a859e8c121cd05e1cd62282505e" gracePeriod=30 Dec 03 11:42:24 crc kubenswrapper[5052]: I1203 11:42:24.095585 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="eae5b719-0017-4621-9cb4-73ed6353648e" containerName="sg-core" containerID="cri-o://0bcbc83b612aeb3a9d9d8d6b5d25c20cfdd3b0b8c6002297d10049035fb2b217" gracePeriod=30 Dec 03 11:42:24 crc kubenswrapper[5052]: I1203 11:42:24.095620 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="eae5b719-0017-4621-9cb4-73ed6353648e" containerName="ceilometer-notification-agent" containerID="cri-o://7474492a6f12a4bba8719b2ffab92d689c3e76fc1c263f7a31eea5c422579780" gracePeriod=30 Dec 03 11:42:24 crc kubenswrapper[5052]: E1203 11:42:24.095715 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="5e0a4e3bdf4887c1a95e0d9815e6d8a1fc6c0dccf8359c3a1a7a1505e8cc9053" cmd=["/bin/bash","/var/lib/operator-scripts/mysql_probe.sh","readiness"] Dec 03 11:42:24 crc kubenswrapper[5052]: E1203 11:42:24.125622 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="5e0a4e3bdf4887c1a95e0d9815e6d8a1fc6c0dccf8359c3a1a7a1505e8cc9053" cmd=["/bin/bash","/var/lib/operator-scripts/mysql_probe.sh","readiness"] Dec 03 11:42:24 crc kubenswrapper[5052]: E1203 11:42:24.125678 5052 prober.go:104] "Probe errored" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" probeType="Readiness" pod="openstack-kuttl-tests/openstack-galera-0" podUID="251248ba-578c-49cf-b77e-fc3d5c6cb6fd" containerName="galera" Dec 03 11:42:24 crc kubenswrapper[5052]: I1203 11:42:24.155570 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7bd0c57-728c-46a4-bbec-2e209d9f4049-ovsdb-rundir" (OuterVolumeSpecName: "ovsdb-rundir") pod "d7bd0c57-728c-46a4-bbec-2e209d9f4049" (UID: "d7bd0c57-728c-46a4-bbec-2e209d9f4049"). InnerVolumeSpecName "ovsdb-rundir". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:42:24 crc kubenswrapper[5052]: I1203 11:42:24.156169 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d7bd0c57-728c-46a4-bbec-2e209d9f4049-config" (OuterVolumeSpecName: "config") pod "d7bd0c57-728c-46a4-bbec-2e209d9f4049" (UID: "d7bd0c57-728c-46a4-bbec-2e209d9f4049"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:42:24 crc kubenswrapper[5052]: I1203 11:42:24.158955 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d7bd0c57-728c-46a4-bbec-2e209d9f4049-scripts" (OuterVolumeSpecName: "scripts") pod "d7bd0c57-728c-46a4-bbec-2e209d9f4049" (UID: "d7bd0c57-728c-46a4-bbec-2e209d9f4049"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:42:24 crc kubenswrapper[5052]: I1203 11:42:24.193062 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d7bd0c57-728c-46a4-bbec-2e209d9f4049-config\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:24 crc kubenswrapper[5052]: I1203 11:42:24.193481 5052 reconciler_common.go:293] "Volume detached for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/d7bd0c57-728c-46a4-bbec-2e209d9f4049-ovsdb-rundir\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:24 crc kubenswrapper[5052]: I1203 11:42:24.193498 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/d7bd0c57-728c-46a4-bbec-2e209d9f4049-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:24 crc kubenswrapper[5052]: I1203 11:42:24.239896 5052 generic.go:334] "Generic (PLEG): container finished" podID="665a736c-d550-4160-a8bb-619d1d5ea826" containerID="907dd58d6f36fcd52c737d7df15fd6614bc5eeddfe7082592f83e68d9f16190c" exitCode=143 Dec 03 11:42:24 crc kubenswrapper[5052]: I1203 11:42:24.239979 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"665a736c-d550-4160-a8bb-619d1d5ea826","Type":"ContainerDied","Data":"907dd58d6f36fcd52c737d7df15fd6614bc5eeddfe7082592f83e68d9f16190c"} Dec 03 11:42:24 crc kubenswrapper[5052]: I1203 11:42:24.242981 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d7bd0c57-728c-46a4-bbec-2e209d9f4049-kube-api-access-8n24z" (OuterVolumeSpecName: "kube-api-access-8n24z") pod "d7bd0c57-728c-46a4-bbec-2e209d9f4049" (UID: "d7bd0c57-728c-46a4-bbec-2e209d9f4049"). InnerVolumeSpecName "kube-api-access-8n24z". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:42:24 crc kubenswrapper[5052]: I1203 11:42:24.258915 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage14-crc" (OuterVolumeSpecName: "ovndbcluster-nb-etc-ovn") pod "d7bd0c57-728c-46a4-bbec-2e209d9f4049" (UID: "d7bd0c57-728c-46a4-bbec-2e209d9f4049"). InnerVolumeSpecName "local-storage14-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 11:42:24 crc kubenswrapper[5052]: I1203 11:42:24.291613 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_ovsdbserver-sb-0_e14d926b-5dc2-46a4-ab27-dfde118cba89/ovsdbserver-sb/0.log" Dec 03 11:42:24 crc kubenswrapper[5052]: I1203 11:42:24.291658 5052 generic.go:334] "Generic (PLEG): container finished" podID="e14d926b-5dc2-46a4-ab27-dfde118cba89" containerID="ef90774de4b83b9fc123ba664ece34787dcdb1ad9fd0d230dacf72b8eee1a1e1" exitCode=143 Dec 03 11:42:24 crc kubenswrapper[5052]: I1203 11:42:24.291727 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-sb-0" event={"ID":"e14d926b-5dc2-46a4-ab27-dfde118cba89","Type":"ContainerDied","Data":"ef90774de4b83b9fc123ba664ece34787dcdb1ad9fd0d230dacf72b8eee1a1e1"} Dec 03 11:42:24 crc kubenswrapper[5052]: I1203 11:42:24.295987 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage14-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage14-crc\") on node \"crc\" " Dec 03 11:42:24 crc kubenswrapper[5052]: I1203 11:42:24.296045 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-8n24z\" (UniqueName: \"kubernetes.io/projected/d7bd0c57-728c-46a4-bbec-2e209d9f4049-kube-api-access-8n24z\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:24 crc kubenswrapper[5052]: I1203 11:42:24.298807 5052 generic.go:334] "Generic (PLEG): container finished" podID="1609a5b5-6666-47e3-b02e-f796d972d331" containerID="352c7041b2ebcbf03c6b6dbad2b08bc20c546043c65916bf3b506556deb9182f" exitCode=0 Dec 03 11:42:24 crc kubenswrapper[5052]: I1203 11:42:24.298924 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/memcached-0" event={"ID":"1609a5b5-6666-47e3-b02e-f796d972d331","Type":"ContainerDied","Data":"352c7041b2ebcbf03c6b6dbad2b08bc20c546043c65916bf3b506556deb9182f"} Dec 03 11:42:24 crc kubenswrapper[5052]: I1203 11:42:24.329172 5052 generic.go:334] "Generic (PLEG): container finished" podID="14e9bff5-66d5-4808-8446-cf180d759208" containerID="6c977dab718207e18c944877c552253d8c5723de1e4d496e75dea068be83b234" exitCode=0 Dec 03 11:42:24 crc kubenswrapper[5052]: I1203 11:42:24.329310 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"14e9bff5-66d5-4808-8446-cf180d759208","Type":"ContainerDied","Data":"6c977dab718207e18c944877c552253d8c5723de1e4d496e75dea068be83b234"} Dec 03 11:42:24 crc kubenswrapper[5052]: I1203 11:42:24.348982 5052 generic.go:334] "Generic (PLEG): container finished" podID="049153cf-b5da-4a61-8a2d-3f694625642b" containerID="10f606fe8fa7b691b8917f929fb59e6261d389781afc79a34d36abb996d45333" exitCode=0 Dec 03 11:42:24 crc kubenswrapper[5052]: I1203 11:42:24.349083 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" event={"ID":"049153cf-b5da-4a61-8a2d-3f694625642b","Type":"ContainerDied","Data":"10f606fe8fa7b691b8917f929fb59e6261d389781afc79a34d36abb996d45333"} Dec 03 11:42:24 crc kubenswrapper[5052]: I1203 11:42:24.359078 5052 generic.go:334] "Generic (PLEG): container finished" podID="ee46c3a4-5e21-4f6b-a944-8cf96d87506e" containerID="f924c6cd365dded654c74f9f506f8cb2317b25d77d3b15ae01a497a3800a80db" exitCode=143 Dec 03 11:42:24 crc kubenswrapper[5052]: I1203 11:42:24.359162 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"ee46c3a4-5e21-4f6b-a944-8cf96d87506e","Type":"ContainerDied","Data":"f924c6cd365dded654c74f9f506f8cb2317b25d77d3b15ae01a497a3800a80db"} Dec 03 11:42:24 crc kubenswrapper[5052]: I1203 11:42:24.385118 5052 generic.go:334] "Generic (PLEG): container finished" podID="722e5457-ae47-40b8-816f-8eab5b7a4cb0" containerID="7f7814a46f2c2a853ddc0d2142a5cfb0bf53aa603be45491209a66cd585a4f18" exitCode=137 Dec 03 11:42:24 crc kubenswrapper[5052]: I1203 11:42:24.418319 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_ovsdbserver-nb-0_d7bd0c57-728c-46a4-bbec-2e209d9f4049/ovsdbserver-nb/0.log" Dec 03 11:42:24 crc kubenswrapper[5052]: I1203 11:42:24.418373 5052 generic.go:334] "Generic (PLEG): container finished" podID="d7bd0c57-728c-46a4-bbec-2e209d9f4049" containerID="959614b917450812745a85b82761fed66dc1a03b7fd08eb36921870d079f6214" exitCode=2 Dec 03 11:42:24 crc kubenswrapper[5052]: I1203 11:42:24.418394 5052 generic.go:334] "Generic (PLEG): container finished" podID="d7bd0c57-728c-46a4-bbec-2e209d9f4049" containerID="d8d8be4b59c92e57d45d986195fe45d17c86d578df37f5f3af30a57358eda608" exitCode=143 Dec 03 11:42:24 crc kubenswrapper[5052]: I1203 11:42:24.418469 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-nb-0" event={"ID":"d7bd0c57-728c-46a4-bbec-2e209d9f4049","Type":"ContainerDied","Data":"959614b917450812745a85b82761fed66dc1a03b7fd08eb36921870d079f6214"} Dec 03 11:42:24 crc kubenswrapper[5052]: I1203 11:42:24.418502 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-nb-0" event={"ID":"d7bd0c57-728c-46a4-bbec-2e209d9f4049","Type":"ContainerDied","Data":"d8d8be4b59c92e57d45d986195fe45d17c86d578df37f5f3af30a57358eda608"} Dec 03 11:42:24 crc kubenswrapper[5052]: I1203 11:42:24.418538 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-nb-0" event={"ID":"d7bd0c57-728c-46a4-bbec-2e209d9f4049","Type":"ContainerDied","Data":"97afd4cd6ab520f3b9f6350f09167b84e4a6e1ed7ab37540b3f4244b47290b0f"} Dec 03 11:42:24 crc kubenswrapper[5052]: I1203 11:42:24.418560 5052 scope.go:117] "RemoveContainer" containerID="959614b917450812745a85b82761fed66dc1a03b7fd08eb36921870d079f6214" Dec 03 11:42:24 crc kubenswrapper[5052]: I1203 11:42:24.418718 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 11:42:24 crc kubenswrapper[5052]: I1203 11:42:24.431108 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-worker-67c775d6bc-kwxzt" event={"ID":"222f033c-3660-4347-b621-7122c4c94929","Type":"ContainerStarted","Data":"a5ac6cc1055df402445233edeff6fd6db77ec35aa50f1a126fefc71ff474afe2"} Dec 03 11:42:24 crc kubenswrapper[5052]: I1203 11:42:24.443722 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/openstackclient"] Dec 03 11:42:24 crc kubenswrapper[5052]: I1203 11:42:24.539811 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/openstackclient"] Dec 03 11:42:24 crc kubenswrapper[5052]: I1203 11:42:24.753900 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d7bd0c57-728c-46a4-bbec-2e209d9f4049-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "d7bd0c57-728c-46a4-bbec-2e209d9f4049" (UID: "d7bd0c57-728c-46a4-bbec-2e209d9f4049"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:42:24 crc kubenswrapper[5052]: I1203 11:42:24.844861 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d7bd0c57-728c-46a4-bbec-2e209d9f4049-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:24 crc kubenswrapper[5052]: I1203 11:42:24.862952 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/rabbitmq-cell1-server-0"] Dec 03 11:42:24 crc kubenswrapper[5052]: I1203 11:42:24.913825 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage14-crc" (UniqueName: "kubernetes.io/local-volume/local-storage14-crc") on node "crc" Dec 03 11:42:25 crc kubenswrapper[5052]: I1203 11:42:25.013561 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage14-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage14-crc\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:25 crc kubenswrapper[5052]: E1203 11:42:25.013976 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/rabbitmq-cell1-config-data: configmap "rabbitmq-cell1-config-data" not found Dec 03 11:42:25 crc kubenswrapper[5052]: E1203 11:42:25.014040 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/daa521fc-7e1d-44d2-b7d8-6aab003cfb66-config-data podName:daa521fc-7e1d-44d2-b7d8-6aab003cfb66 nodeName:}" failed. No retries permitted until 2025-12-03 11:42:25.514019708 +0000 UTC m=+1336.102261472 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "config-data" (UniqueName: "kubernetes.io/configmap/daa521fc-7e1d-44d2-b7d8-6aab003cfb66-config-data") pod "rabbitmq-cell1-server-0" (UID: "daa521fc-7e1d-44d2-b7d8-6aab003cfb66") : configmap "rabbitmq-cell1-config-data" not found Dec 03 11:42:25 crc kubenswrapper[5052]: I1203 11:42:25.019708 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/neutron-db-sync-j7t4l"] Dec 03 11:42:25 crc kubenswrapper[5052]: I1203 11:42:25.068943 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/neutron-db-sync-j7t4l"] Dec 03 11:42:25 crc kubenswrapper[5052]: I1203 11:42:25.088440 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/cinder5365-account-delete-shtcv"] Dec 03 11:42:25 crc kubenswrapper[5052]: E1203 11:42:25.089956 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="d7bd0c57-728c-46a4-bbec-2e209d9f4049" containerName="openstack-network-exporter" Dec 03 11:42:25 crc kubenswrapper[5052]: I1203 11:42:25.089982 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="d7bd0c57-728c-46a4-bbec-2e209d9f4049" containerName="openstack-network-exporter" Dec 03 11:42:25 crc kubenswrapper[5052]: E1203 11:42:25.090013 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="d7bd0c57-728c-46a4-bbec-2e209d9f4049" containerName="ovsdbserver-nb" Dec 03 11:42:25 crc kubenswrapper[5052]: I1203 11:42:25.090019 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="d7bd0c57-728c-46a4-bbec-2e209d9f4049" containerName="ovsdbserver-nb" Dec 03 11:42:25 crc kubenswrapper[5052]: I1203 11:42:25.090436 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="d7bd0c57-728c-46a4-bbec-2e209d9f4049" containerName="openstack-network-exporter" Dec 03 11:42:25 crc kubenswrapper[5052]: I1203 11:42:25.090458 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="d7bd0c57-728c-46a4-bbec-2e209d9f4049" containerName="ovsdbserver-nb" Dec 03 11:42:25 crc kubenswrapper[5052]: I1203 11:42:25.091385 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder5365-account-delete-shtcv" Dec 03 11:42:25 crc kubenswrapper[5052]: I1203 11:42:25.109966 5052 scope.go:117] "RemoveContainer" containerID="d8d8be4b59c92e57d45d986195fe45d17c86d578df37f5f3af30a57358eda608" Dec 03 11:42:25 crc kubenswrapper[5052]: I1203 11:42:25.120731 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b65mb\" (UniqueName: \"kubernetes.io/projected/d3e34248-e7e5-4158-8add-165d17bee9ab-kube-api-access-b65mb\") pod \"cinder5365-account-delete-shtcv\" (UID: \"d3e34248-e7e5-4158-8add-165d17bee9ab\") " pod="openstack-kuttl-tests/cinder5365-account-delete-shtcv" Dec 03 11:42:25 crc kubenswrapper[5052]: I1203 11:42:25.120806 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/d3e34248-e7e5-4158-8add-165d17bee9ab-operator-scripts\") pod \"cinder5365-account-delete-shtcv\" (UID: \"d3e34248-e7e5-4158-8add-165d17bee9ab\") " pod="openstack-kuttl-tests/cinder5365-account-delete-shtcv" Dec 03 11:42:25 crc kubenswrapper[5052]: I1203 11:42:25.169618 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d7bd0c57-728c-46a4-bbec-2e209d9f4049-metrics-certs-tls-certs" (OuterVolumeSpecName: "metrics-certs-tls-certs") pod "d7bd0c57-728c-46a4-bbec-2e209d9f4049" (UID: "d7bd0c57-728c-46a4-bbec-2e209d9f4049"). InnerVolumeSpecName "metrics-certs-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:42:25 crc kubenswrapper[5052]: I1203 11:42:25.202673 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder-db-sync-pbw4k"] Dec 03 11:42:25 crc kubenswrapper[5052]: I1203 11:42:25.225858 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-b65mb\" (UniqueName: \"kubernetes.io/projected/d3e34248-e7e5-4158-8add-165d17bee9ab-kube-api-access-b65mb\") pod \"cinder5365-account-delete-shtcv\" (UID: \"d3e34248-e7e5-4158-8add-165d17bee9ab\") " pod="openstack-kuttl-tests/cinder5365-account-delete-shtcv" Dec 03 11:42:25 crc kubenswrapper[5052]: I1203 11:42:25.225998 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/d3e34248-e7e5-4158-8add-165d17bee9ab-operator-scripts\") pod \"cinder5365-account-delete-shtcv\" (UID: \"d3e34248-e7e5-4158-8add-165d17bee9ab\") " pod="openstack-kuttl-tests/cinder5365-account-delete-shtcv" Dec 03 11:42:25 crc kubenswrapper[5052]: I1203 11:42:25.228196 5052 reconciler_common.go:293] "Volume detached for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/d7bd0c57-728c-46a4-bbec-2e209d9f4049-metrics-certs-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:25 crc kubenswrapper[5052]: E1203 11:42:25.238848 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="d9bef6e726b038ae865fa13770a700b7a8625ffee06c08933704155c1695d94f" cmd=["/bin/bash","/var/lib/operator-scripts/mysql_probe.sh","readiness"] Dec 03 11:42:25 crc kubenswrapper[5052]: I1203 11:42:25.239050 5052 scope.go:117] "RemoveContainer" containerID="959614b917450812745a85b82761fed66dc1a03b7fd08eb36921870d079f6214" Dec 03 11:42:25 crc kubenswrapper[5052]: I1203 11:42:25.240413 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/d3e34248-e7e5-4158-8add-165d17bee9ab-operator-scripts\") pod \"cinder5365-account-delete-shtcv\" (UID: \"d3e34248-e7e5-4158-8add-165d17bee9ab\") " pod="openstack-kuttl-tests/cinder5365-account-delete-shtcv" Dec 03 11:42:25 crc kubenswrapper[5052]: E1203 11:42:25.246113 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"959614b917450812745a85b82761fed66dc1a03b7fd08eb36921870d079f6214\": container with ID starting with 959614b917450812745a85b82761fed66dc1a03b7fd08eb36921870d079f6214 not found: ID does not exist" containerID="959614b917450812745a85b82761fed66dc1a03b7fd08eb36921870d079f6214" Dec 03 11:42:25 crc kubenswrapper[5052]: I1203 11:42:25.246190 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"959614b917450812745a85b82761fed66dc1a03b7fd08eb36921870d079f6214"} err="failed to get container status \"959614b917450812745a85b82761fed66dc1a03b7fd08eb36921870d079f6214\": rpc error: code = NotFound desc = could not find container \"959614b917450812745a85b82761fed66dc1a03b7fd08eb36921870d079f6214\": container with ID starting with 959614b917450812745a85b82761fed66dc1a03b7fd08eb36921870d079f6214 not found: ID does not exist" Dec 03 11:42:25 crc kubenswrapper[5052]: I1203 11:42:25.246235 5052 scope.go:117] "RemoveContainer" containerID="d8d8be4b59c92e57d45d986195fe45d17c86d578df37f5f3af30a57358eda608" Dec 03 11:42:25 crc kubenswrapper[5052]: E1203 11:42:25.281271 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="d9bef6e726b038ae865fa13770a700b7a8625ffee06c08933704155c1695d94f" cmd=["/bin/bash","/var/lib/operator-scripts/mysql_probe.sh","readiness"] Dec 03 11:42:25 crc kubenswrapper[5052]: E1203 11:42:25.281873 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d8d8be4b59c92e57d45d986195fe45d17c86d578df37f5f3af30a57358eda608\": container with ID starting with d8d8be4b59c92e57d45d986195fe45d17c86d578df37f5f3af30a57358eda608 not found: ID does not exist" containerID="d8d8be4b59c92e57d45d986195fe45d17c86d578df37f5f3af30a57358eda608" Dec 03 11:42:25 crc kubenswrapper[5052]: I1203 11:42:25.292427 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d8d8be4b59c92e57d45d986195fe45d17c86d578df37f5f3af30a57358eda608"} err="failed to get container status \"d8d8be4b59c92e57d45d986195fe45d17c86d578df37f5f3af30a57358eda608\": rpc error: code = NotFound desc = could not find container \"d8d8be4b59c92e57d45d986195fe45d17c86d578df37f5f3af30a57358eda608\": container with ID starting with d8d8be4b59c92e57d45d986195fe45d17c86d578df37f5f3af30a57358eda608 not found: ID does not exist" Dec 03 11:42:25 crc kubenswrapper[5052]: I1203 11:42:25.292542 5052 scope.go:117] "RemoveContainer" containerID="959614b917450812745a85b82761fed66dc1a03b7fd08eb36921870d079f6214" Dec 03 11:42:25 crc kubenswrapper[5052]: I1203 11:42:25.290458 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/memcached-0" Dec 03 11:42:25 crc kubenswrapper[5052]: I1203 11:42:25.293353 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"959614b917450812745a85b82761fed66dc1a03b7fd08eb36921870d079f6214"} err="failed to get container status \"959614b917450812745a85b82761fed66dc1a03b7fd08eb36921870d079f6214\": rpc error: code = NotFound desc = could not find container \"959614b917450812745a85b82761fed66dc1a03b7fd08eb36921870d079f6214\": container with ID starting with 959614b917450812745a85b82761fed66dc1a03b7fd08eb36921870d079f6214 not found: ID does not exist" Dec 03 11:42:25 crc kubenswrapper[5052]: I1203 11:42:25.293392 5052 scope.go:117] "RemoveContainer" containerID="d8d8be4b59c92e57d45d986195fe45d17c86d578df37f5f3af30a57358eda608" Dec 03 11:42:25 crc kubenswrapper[5052]: I1203 11:42:25.293618 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d8d8be4b59c92e57d45d986195fe45d17c86d578df37f5f3af30a57358eda608"} err="failed to get container status \"d8d8be4b59c92e57d45d986195fe45d17c86d578df37f5f3af30a57358eda608\": rpc error: code = NotFound desc = could not find container \"d8d8be4b59c92e57d45d986195fe45d17c86d578df37f5f3af30a57358eda608\": container with ID starting with d8d8be4b59c92e57d45d986195fe45d17c86d578df37f5f3af30a57358eda608 not found: ID does not exist" Dec 03 11:42:25 crc kubenswrapper[5052]: E1203 11:42:25.295669 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="d9bef6e726b038ae865fa13770a700b7a8625ffee06c08933704155c1695d94f" cmd=["/bin/bash","/var/lib/operator-scripts/mysql_probe.sh","readiness"] Dec 03 11:42:25 crc kubenswrapper[5052]: E1203 11:42:25.295726 5052 prober.go:104] "Probe errored" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" probeType="Readiness" pod="openstack-kuttl-tests/openstack-cell1-galera-0" podUID="ecbf8b8e-cdb4-45c3-9455-d0dbc551356e" containerName="galera" Dec 03 11:42:25 crc kubenswrapper[5052]: I1203 11:42:25.302520 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-b65mb\" (UniqueName: \"kubernetes.io/projected/d3e34248-e7e5-4158-8add-165d17bee9ab-kube-api-access-b65mb\") pod \"cinder5365-account-delete-shtcv\" (UID: \"d3e34248-e7e5-4158-8add-165d17bee9ab\") " pod="openstack-kuttl-tests/cinder5365-account-delete-shtcv" Dec 03 11:42:25 crc kubenswrapper[5052]: I1203 11:42:25.347579 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder5365-account-delete-shtcv"] Dec 03 11:42:25 crc kubenswrapper[5052]: I1203 11:42:25.390046 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d7bd0c57-728c-46a4-bbec-2e209d9f4049-ovsdbserver-nb-tls-certs" (OuterVolumeSpecName: "ovsdbserver-nb-tls-certs") pod "d7bd0c57-728c-46a4-bbec-2e209d9f4049" (UID: "d7bd0c57-728c-46a4-bbec-2e209d9f4049"). InnerVolumeSpecName "ovsdbserver-nb-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:42:25 crc kubenswrapper[5052]: I1203 11:42:25.391201 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/1609a5b5-6666-47e3-b02e-f796d972d331-config-data\") pod \"1609a5b5-6666-47e3-b02e-f796d972d331\" (UID: \"1609a5b5-6666-47e3-b02e-f796d972d331\") " Dec 03 11:42:25 crc kubenswrapper[5052]: I1203 11:42:25.391333 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"memcached-tls-certs\" (UniqueName: \"kubernetes.io/secret/1609a5b5-6666-47e3-b02e-f796d972d331-memcached-tls-certs\") pod \"1609a5b5-6666-47e3-b02e-f796d972d331\" (UID: \"1609a5b5-6666-47e3-b02e-f796d972d331\") " Dec 03 11:42:25 crc kubenswrapper[5052]: I1203 11:42:25.391395 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1609a5b5-6666-47e3-b02e-f796d972d331-combined-ca-bundle\") pod \"1609a5b5-6666-47e3-b02e-f796d972d331\" (UID: \"1609a5b5-6666-47e3-b02e-f796d972d331\") " Dec 03 11:42:25 crc kubenswrapper[5052]: I1203 11:42:25.391430 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/1609a5b5-6666-47e3-b02e-f796d972d331-kolla-config\") pod \"1609a5b5-6666-47e3-b02e-f796d972d331\" (UID: \"1609a5b5-6666-47e3-b02e-f796d972d331\") " Dec 03 11:42:25 crc kubenswrapper[5052]: I1203 11:42:25.391532 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k9bfs\" (UniqueName: \"kubernetes.io/projected/1609a5b5-6666-47e3-b02e-f796d972d331-kube-api-access-k9bfs\") pod \"1609a5b5-6666-47e3-b02e-f796d972d331\" (UID: \"1609a5b5-6666-47e3-b02e-f796d972d331\") " Dec 03 11:42:25 crc kubenswrapper[5052]: I1203 11:42:25.391911 5052 reconciler_common.go:293] "Volume detached for volume \"ovsdbserver-nb-tls-certs\" (UniqueName: \"kubernetes.io/secret/d7bd0c57-728c-46a4-bbec-2e209d9f4049-ovsdbserver-nb-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:25 crc kubenswrapper[5052]: I1203 11:42:25.393134 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1609a5b5-6666-47e3-b02e-f796d972d331-config-data" (OuterVolumeSpecName: "config-data") pod "1609a5b5-6666-47e3-b02e-f796d972d331" (UID: "1609a5b5-6666-47e3-b02e-f796d972d331"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:42:25 crc kubenswrapper[5052]: I1203 11:42:25.395668 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1609a5b5-6666-47e3-b02e-f796d972d331-kolla-config" (OuterVolumeSpecName: "kolla-config") pod "1609a5b5-6666-47e3-b02e-f796d972d331" (UID: "1609a5b5-6666-47e3-b02e-f796d972d331"). InnerVolumeSpecName "kolla-config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:42:25 crc kubenswrapper[5052]: I1203 11:42:25.498814 5052 reconciler_common.go:293] "Volume detached for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/1609a5b5-6666-47e3-b02e-f796d972d331-kolla-config\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:25 crc kubenswrapper[5052]: I1203 11:42:25.499467 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/1609a5b5-6666-47e3-b02e-f796d972d331-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:25 crc kubenswrapper[5052]: I1203 11:42:25.506240 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/cinder-db-sync-pbw4k"] Dec 03 11:42:25 crc kubenswrapper[5052]: I1203 11:42:25.515179 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-keystone-listener-5dd6f4bfd-glsv7"] Dec 03 11:42:25 crc kubenswrapper[5052]: I1203 11:42:25.525600 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/neutronbfb7-account-delete-ff26t"] Dec 03 11:42:25 crc kubenswrapper[5052]: E1203 11:42:25.526334 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="1609a5b5-6666-47e3-b02e-f796d972d331" containerName="memcached" Dec 03 11:42:25 crc kubenswrapper[5052]: I1203 11:42:25.526359 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="1609a5b5-6666-47e3-b02e-f796d972d331" containerName="memcached" Dec 03 11:42:25 crc kubenswrapper[5052]: I1203 11:42:25.526682 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="1609a5b5-6666-47e3-b02e-f796d972d331" containerName="memcached" Dec 03 11:42:25 crc kubenswrapper[5052]: I1203 11:42:25.535416 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1609a5b5-6666-47e3-b02e-f796d972d331-kube-api-access-k9bfs" (OuterVolumeSpecName: "kube-api-access-k9bfs") pod "1609a5b5-6666-47e3-b02e-f796d972d331" (UID: "1609a5b5-6666-47e3-b02e-f796d972d331"). InnerVolumeSpecName "kube-api-access-k9bfs". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:42:25 crc kubenswrapper[5052]: I1203 11:42:25.544525 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutronbfb7-account-delete-ff26t" Dec 03 11:42:25 crc kubenswrapper[5052]: I1203 11:42:25.602858 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5fg2w\" (UniqueName: \"kubernetes.io/projected/bd2dc8ee-f6b9-4c8b-af5e-79e67a2d1ef0-kube-api-access-5fg2w\") pod \"neutronbfb7-account-delete-ff26t\" (UID: \"bd2dc8ee-f6b9-4c8b-af5e-79e67a2d1ef0\") " pod="openstack-kuttl-tests/neutronbfb7-account-delete-ff26t" Dec 03 11:42:25 crc kubenswrapper[5052]: I1203 11:42:25.644421 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/bd2dc8ee-f6b9-4c8b-af5e-79e67a2d1ef0-operator-scripts\") pod \"neutronbfb7-account-delete-ff26t\" (UID: \"bd2dc8ee-f6b9-4c8b-af5e-79e67a2d1ef0\") " pod="openstack-kuttl-tests/neutronbfb7-account-delete-ff26t" Dec 03 11:42:25 crc kubenswrapper[5052]: I1203 11:42:25.648737 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-k9bfs\" (UniqueName: \"kubernetes.io/projected/1609a5b5-6666-47e3-b02e-f796d972d331-kube-api-access-k9bfs\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:25 crc kubenswrapper[5052]: E1203 11:42:25.603953 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/rabbitmq-cell1-config-data: configmap "rabbitmq-cell1-config-data" not found Dec 03 11:42:25 crc kubenswrapper[5052]: E1203 11:42:25.648992 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/daa521fc-7e1d-44d2-b7d8-6aab003cfb66-config-data podName:daa521fc-7e1d-44d2-b7d8-6aab003cfb66 nodeName:}" failed. No retries permitted until 2025-12-03 11:42:26.6489724 +0000 UTC m=+1337.237214164 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "config-data" (UniqueName: "kubernetes.io/configmap/daa521fc-7e1d-44d2-b7d8-6aab003cfb66-config-data") pod "rabbitmq-cell1-server-0" (UID: "daa521fc-7e1d-44d2-b7d8-6aab003cfb66") : configmap "rabbitmq-cell1-config-data" not found Dec 03 11:42:25 crc kubenswrapper[5052]: I1203 11:42:25.628020 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-6d84cdf894-dqcg9" event={"ID":"0c7d3107-6dd9-4aa2-a392-68a213013508","Type":"ContainerStarted","Data":"55fe708364bcd032d50f6462f0711e8cc4abea2cb54eb4f7e91c5c2b63841a8e"} Dec 03 11:42:25 crc kubenswrapper[5052]: I1203 11:42:25.650989 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutronbfb7-account-delete-ff26t"] Dec 03 11:42:25 crc kubenswrapper[5052]: I1203 11:42:25.654638 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" event={"ID":"049153cf-b5da-4a61-8a2d-3f694625642b","Type":"ContainerDied","Data":"ded9cddd3ba921f6b429e0b802f59fa4795b1425e1728696575c769f420fc6f1"} Dec 03 11:42:25 crc kubenswrapper[5052]: I1203 11:42:25.654687 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ded9cddd3ba921f6b429e0b802f59fa4795b1425e1728696575c769f420fc6f1" Dec 03 11:42:25 crc kubenswrapper[5052]: I1203 11:42:25.654788 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 11:42:25 crc kubenswrapper[5052]: I1203 11:42:25.663740 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/placement79ca-account-delete-sqtcb"] Dec 03 11:42:25 crc kubenswrapper[5052]: E1203 11:42:25.664342 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="049153cf-b5da-4a61-8a2d-3f694625642b" containerName="nova-cell1-novncproxy-novncproxy" Dec 03 11:42:25 crc kubenswrapper[5052]: I1203 11:42:25.664363 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="049153cf-b5da-4a61-8a2d-3f694625642b" containerName="nova-cell1-novncproxy-novncproxy" Dec 03 11:42:25 crc kubenswrapper[5052]: I1203 11:42:25.664641 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="049153cf-b5da-4a61-8a2d-3f694625642b" containerName="nova-cell1-novncproxy-novncproxy" Dec 03 11:42:25 crc kubenswrapper[5052]: I1203 11:42:25.673800 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement79ca-account-delete-sqtcb" Dec 03 11:42:25 crc kubenswrapper[5052]: I1203 11:42:25.714975 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_ovsdbserver-sb-0_e14d926b-5dc2-46a4-ab27-dfde118cba89/ovsdbserver-sb/0.log" Dec 03 11:42:25 crc kubenswrapper[5052]: I1203 11:42:25.715072 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 11:42:25 crc kubenswrapper[5052]: I1203 11:42:25.748852 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aae8de3e-7ad6-4a25-bdd5-3f98730ae227" path="/var/lib/kubelet/pods/aae8de3e-7ad6-4a25-bdd5-3f98730ae227/volumes" Dec 03 11:42:25 crc kubenswrapper[5052]: I1203 11:42:25.750288 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ctjfw\" (UniqueName: \"kubernetes.io/projected/049153cf-b5da-4a61-8a2d-3f694625642b-kube-api-access-ctjfw\") pod \"049153cf-b5da-4a61-8a2d-3f694625642b\" (UID: \"049153cf-b5da-4a61-8a2d-3f694625642b\") " Dec 03 11:42:25 crc kubenswrapper[5052]: I1203 11:42:25.784218 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"nova-novncproxy-tls-certs\" (UniqueName: \"kubernetes.io/secret/049153cf-b5da-4a61-8a2d-3f694625642b-nova-novncproxy-tls-certs\") pod \"049153cf-b5da-4a61-8a2d-3f694625642b\" (UID: \"049153cf-b5da-4a61-8a2d-3f694625642b\") " Dec 03 11:42:25 crc kubenswrapper[5052]: I1203 11:42:25.784240 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovndbcluster-sb-etc-ovn\" (UniqueName: \"kubernetes.io/local-volume/local-storage01-crc\") pod \"e14d926b-5dc2-46a4-ab27-dfde118cba89\" (UID: \"e14d926b-5dc2-46a4-ab27-dfde118cba89\") " Dec 03 11:42:25 crc kubenswrapper[5052]: I1203 11:42:25.784286 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovsdbserver-sb-tls-certs\" (UniqueName: \"kubernetes.io/secret/e14d926b-5dc2-46a4-ab27-dfde118cba89-ovsdbserver-sb-tls-certs\") pod \"e14d926b-5dc2-46a4-ab27-dfde118cba89\" (UID: \"e14d926b-5dc2-46a4-ab27-dfde118cba89\") " Dec 03 11:42:25 crc kubenswrapper[5052]: I1203 11:42:25.784304 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/e14d926b-5dc2-46a4-ab27-dfde118cba89-scripts\") pod \"e14d926b-5dc2-46a4-ab27-dfde118cba89\" (UID: \"e14d926b-5dc2-46a4-ab27-dfde118cba89\") " Dec 03 11:42:25 crc kubenswrapper[5052]: I1203 11:42:25.784321 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/049153cf-b5da-4a61-8a2d-3f694625642b-combined-ca-bundle\") pod \"049153cf-b5da-4a61-8a2d-3f694625642b\" (UID: \"049153cf-b5da-4a61-8a2d-3f694625642b\") " Dec 03 11:42:25 crc kubenswrapper[5052]: I1203 11:42:25.784353 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e14d926b-5dc2-46a4-ab27-dfde118cba89-combined-ca-bundle\") pod \"e14d926b-5dc2-46a4-ab27-dfde118cba89\" (UID: \"e14d926b-5dc2-46a4-ab27-dfde118cba89\") " Dec 03 11:42:25 crc kubenswrapper[5052]: I1203 11:42:25.784372 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"vencrypt-tls-certs\" (UniqueName: \"kubernetes.io/secret/049153cf-b5da-4a61-8a2d-3f694625642b-vencrypt-tls-certs\") pod \"049153cf-b5da-4a61-8a2d-3f694625642b\" (UID: \"049153cf-b5da-4a61-8a2d-3f694625642b\") " Dec 03 11:42:25 crc kubenswrapper[5052]: I1203 11:42:25.784429 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/049153cf-b5da-4a61-8a2d-3f694625642b-config-data\") pod \"049153cf-b5da-4a61-8a2d-3f694625642b\" (UID: \"049153cf-b5da-4a61-8a2d-3f694625642b\") " Dec 03 11:42:25 crc kubenswrapper[5052]: I1203 11:42:25.784622 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-crnf7\" (UniqueName: \"kubernetes.io/projected/e14d926b-5dc2-46a4-ab27-dfde118cba89-kube-api-access-crnf7\") pod \"e14d926b-5dc2-46a4-ab27-dfde118cba89\" (UID: \"e14d926b-5dc2-46a4-ab27-dfde118cba89\") " Dec 03 11:42:25 crc kubenswrapper[5052]: I1203 11:42:25.784682 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/e14d926b-5dc2-46a4-ab27-dfde118cba89-metrics-certs-tls-certs\") pod \"e14d926b-5dc2-46a4-ab27-dfde118cba89\" (UID: \"e14d926b-5dc2-46a4-ab27-dfde118cba89\") " Dec 03 11:42:25 crc kubenswrapper[5052]: I1203 11:42:25.786998 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-5fg2w\" (UniqueName: \"kubernetes.io/projected/bd2dc8ee-f6b9-4c8b-af5e-79e67a2d1ef0-kube-api-access-5fg2w\") pod \"neutronbfb7-account-delete-ff26t\" (UID: \"bd2dc8ee-f6b9-4c8b-af5e-79e67a2d1ef0\") " pod="openstack-kuttl-tests/neutronbfb7-account-delete-ff26t" Dec 03 11:42:25 crc kubenswrapper[5052]: I1203 11:42:25.787152 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/32573220-587f-4703-a5e9-0fb67d2019ef-operator-scripts\") pod \"placement79ca-account-delete-sqtcb\" (UID: \"32573220-587f-4703-a5e9-0fb67d2019ef\") " pod="openstack-kuttl-tests/placement79ca-account-delete-sqtcb" Dec 03 11:42:25 crc kubenswrapper[5052]: I1203 11:42:25.787232 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/bd2dc8ee-f6b9-4c8b-af5e-79e67a2d1ef0-operator-scripts\") pod \"neutronbfb7-account-delete-ff26t\" (UID: \"bd2dc8ee-f6b9-4c8b-af5e-79e67a2d1ef0\") " pod="openstack-kuttl-tests/neutronbfb7-account-delete-ff26t" Dec 03 11:42:25 crc kubenswrapper[5052]: I1203 11:42:25.787372 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m8jvk\" (UniqueName: \"kubernetes.io/projected/32573220-587f-4703-a5e9-0fb67d2019ef-kube-api-access-m8jvk\") pod \"placement79ca-account-delete-sqtcb\" (UID: \"32573220-587f-4703-a5e9-0fb67d2019ef\") " pod="openstack-kuttl-tests/placement79ca-account-delete-sqtcb" Dec 03 11:42:25 crc kubenswrapper[5052]: I1203 11:42:25.756284 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_ovsdbserver-sb-0_e14d926b-5dc2-46a4-ab27-dfde118cba89/ovsdbserver-sb/0.log" Dec 03 11:42:25 crc kubenswrapper[5052]: I1203 11:42:25.788630 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e14d926b-5dc2-46a4-ab27-dfde118cba89-scripts" (OuterVolumeSpecName: "scripts") pod "e14d926b-5dc2-46a4-ab27-dfde118cba89" (UID: "e14d926b-5dc2-46a4-ab27-dfde118cba89"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:42:25 crc kubenswrapper[5052]: I1203 11:42:25.790560 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/049153cf-b5da-4a61-8a2d-3f694625642b-kube-api-access-ctjfw" (OuterVolumeSpecName: "kube-api-access-ctjfw") pod "049153cf-b5da-4a61-8a2d-3f694625642b" (UID: "049153cf-b5da-4a61-8a2d-3f694625642b"). InnerVolumeSpecName "kube-api-access-ctjfw". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:42:25 crc kubenswrapper[5052]: I1203 11:42:25.804208 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/memcached-0" Dec 03 11:42:25 crc kubenswrapper[5052]: I1203 11:42:25.807905 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e14d926b-5dc2-46a4-ab27-dfde118cba89-kube-api-access-crnf7" (OuterVolumeSpecName: "kube-api-access-crnf7") pod "e14d926b-5dc2-46a4-ab27-dfde118cba89" (UID: "e14d926b-5dc2-46a4-ab27-dfde118cba89"). InnerVolumeSpecName "kube-api-access-crnf7". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:42:25 crc kubenswrapper[5052]: I1203 11:42:25.831198 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1609a5b5-6666-47e3-b02e-f796d972d331-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "1609a5b5-6666-47e3-b02e-f796d972d331" (UID: "1609a5b5-6666-47e3-b02e-f796d972d331"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:42:25 crc kubenswrapper[5052]: I1203 11:42:25.852399 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder5365-account-delete-shtcv" Dec 03 11:42:25 crc kubenswrapper[5052]: I1203 11:42:25.864720 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/bd2dc8ee-f6b9-4c8b-af5e-79e67a2d1ef0-operator-scripts\") pod \"neutronbfb7-account-delete-ff26t\" (UID: \"bd2dc8ee-f6b9-4c8b-af5e-79e67a2d1ef0\") " pod="openstack-kuttl-tests/neutronbfb7-account-delete-ff26t" Dec 03 11:42:25 crc kubenswrapper[5052]: I1203 11:42:25.906226 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e14d926b-5dc2-46a4-ab27-dfde118cba89-config\") pod \"e14d926b-5dc2-46a4-ab27-dfde118cba89\" (UID: \"e14d926b-5dc2-46a4-ab27-dfde118cba89\") " Dec 03 11:42:25 crc kubenswrapper[5052]: I1203 11:42:25.906344 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/e14d926b-5dc2-46a4-ab27-dfde118cba89-ovsdb-rundir\") pod \"e14d926b-5dc2-46a4-ab27-dfde118cba89\" (UID: \"e14d926b-5dc2-46a4-ab27-dfde118cba89\") " Dec 03 11:42:25 crc kubenswrapper[5052]: I1203 11:42:25.908557 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e14d926b-5dc2-46a4-ab27-dfde118cba89-ovsdb-rundir" (OuterVolumeSpecName: "ovsdb-rundir") pod "e14d926b-5dc2-46a4-ab27-dfde118cba89" (UID: "e14d926b-5dc2-46a4-ab27-dfde118cba89"). InnerVolumeSpecName "ovsdb-rundir". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:42:25 crc kubenswrapper[5052]: I1203 11:42:25.913208 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/32573220-587f-4703-a5e9-0fb67d2019ef-operator-scripts\") pod \"placement79ca-account-delete-sqtcb\" (UID: \"32573220-587f-4703-a5e9-0fb67d2019ef\") " pod="openstack-kuttl-tests/placement79ca-account-delete-sqtcb" Dec 03 11:42:25 crc kubenswrapper[5052]: I1203 11:42:25.913382 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-m8jvk\" (UniqueName: \"kubernetes.io/projected/32573220-587f-4703-a5e9-0fb67d2019ef-kube-api-access-m8jvk\") pod \"placement79ca-account-delete-sqtcb\" (UID: \"32573220-587f-4703-a5e9-0fb67d2019ef\") " pod="openstack-kuttl-tests/placement79ca-account-delete-sqtcb" Dec 03 11:42:25 crc kubenswrapper[5052]: I1203 11:42:25.913663 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/e14d926b-5dc2-46a4-ab27-dfde118cba89-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:25 crc kubenswrapper[5052]: I1203 11:42:25.913676 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1609a5b5-6666-47e3-b02e-f796d972d331-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:25 crc kubenswrapper[5052]: I1203 11:42:25.913691 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-crnf7\" (UniqueName: \"kubernetes.io/projected/e14d926b-5dc2-46a4-ab27-dfde118cba89-kube-api-access-crnf7\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:25 crc kubenswrapper[5052]: I1203 11:42:25.913700 5052 reconciler_common.go:293] "Volume detached for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/e14d926b-5dc2-46a4-ab27-dfde118cba89-ovsdb-rundir\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:25 crc kubenswrapper[5052]: I1203 11:42:25.913709 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-ctjfw\" (UniqueName: \"kubernetes.io/projected/049153cf-b5da-4a61-8a2d-3f694625642b-kube-api-access-ctjfw\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:25 crc kubenswrapper[5052]: I1203 11:42:25.914149 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/32573220-587f-4703-a5e9-0fb67d2019ef-operator-scripts\") pod \"placement79ca-account-delete-sqtcb\" (UID: \"32573220-587f-4703-a5e9-0fb67d2019ef\") " pod="openstack-kuttl-tests/placement79ca-account-delete-sqtcb" Dec 03 11:42:25 crc kubenswrapper[5052]: I1203 11:42:25.925640 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage01-crc" (OuterVolumeSpecName: "ovndbcluster-sb-etc-ovn") pod "e14d926b-5dc2-46a4-ab27-dfde118cba89" (UID: "e14d926b-5dc2-46a4-ab27-dfde118cba89"). InnerVolumeSpecName "local-storage01-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 11:42:25 crc kubenswrapper[5052]: I1203 11:42:25.925769 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-5fg2w\" (UniqueName: \"kubernetes.io/projected/bd2dc8ee-f6b9-4c8b-af5e-79e67a2d1ef0-kube-api-access-5fg2w\") pod \"neutronbfb7-account-delete-ff26t\" (UID: \"bd2dc8ee-f6b9-4c8b-af5e-79e67a2d1ef0\") " pod="openstack-kuttl-tests/neutronbfb7-account-delete-ff26t" Dec 03 11:42:25 crc kubenswrapper[5052]: I1203 11:42:25.926681 5052 generic.go:334] "Generic (PLEG): container finished" podID="eae5b719-0017-4621-9cb4-73ed6353648e" containerID="aa0bc44913bc443884760e2a18faf9da4a9e4a859e8c121cd05e1cd62282505e" exitCode=0 Dec 03 11:42:25 crc kubenswrapper[5052]: I1203 11:42:25.926707 5052 generic.go:334] "Generic (PLEG): container finished" podID="eae5b719-0017-4621-9cb4-73ed6353648e" containerID="0bcbc83b612aeb3a9d9d8d6b5d25c20cfdd3b0b8c6002297d10049035fb2b217" exitCode=2 Dec 03 11:42:25 crc kubenswrapper[5052]: I1203 11:42:25.927817 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e14d926b-5dc2-46a4-ab27-dfde118cba89-config" (OuterVolumeSpecName: "config") pod "e14d926b-5dc2-46a4-ab27-dfde118cba89" (UID: "e14d926b-5dc2-46a4-ab27-dfde118cba89"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:42:25 crc kubenswrapper[5052]: I1203 11:42:25.943493 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d1676789-356e-4853-a302-f2c5bd3f446f" path="/var/lib/kubelet/pods/d1676789-356e-4853-a302-f2c5bd3f446f/volumes" Dec 03 11:42:25 crc kubenswrapper[5052]: I1203 11:42:25.982039 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-m8jvk\" (UniqueName: \"kubernetes.io/projected/32573220-587f-4703-a5e9-0fb67d2019ef-kube-api-access-m8jvk\") pod \"placement79ca-account-delete-sqtcb\" (UID: \"32573220-587f-4703-a5e9-0fb67d2019ef\") " pod="openstack-kuttl-tests/placement79ca-account-delete-sqtcb" Dec 03 11:42:26 crc kubenswrapper[5052]: I1203 11:42:26.018954 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage01-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage01-crc\") on node \"crc\" " Dec 03 11:42:26 crc kubenswrapper[5052]: I1203 11:42:26.018988 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e14d926b-5dc2-46a4-ab27-dfde118cba89-config\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:26 crc kubenswrapper[5052]: I1203 11:42:26.030090 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/placement79ca-account-delete-sqtcb"] Dec 03 11:42:26 crc kubenswrapper[5052]: I1203 11:42:26.030331 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/placement-db-sync-qp7zq"] Dec 03 11:42:26 crc kubenswrapper[5052]: I1203 11:42:26.030347 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/placement-db-sync-qp7zq"] Dec 03 11:42:26 crc kubenswrapper[5052]: I1203 11:42:26.030361 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-db-sync-bfv74"] Dec 03 11:42:26 crc kubenswrapper[5052]: I1203 11:42:26.030372 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/barbican-db-sync-bfv74"] Dec 03 11:42:26 crc kubenswrapper[5052]: I1203 11:42:26.030404 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/swift-ring-rebalance-krb94"] Dec 03 11:42:26 crc kubenswrapper[5052]: I1203 11:42:26.030419 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-worker-67c775d6bc-kwxzt" event={"ID":"222f033c-3660-4347-b621-7122c4c94929","Type":"ContainerStarted","Data":"9e7335aefbd832cdb40aea97b2f03dec68bab190e8c59ded23cf65ab50caa997"} Dec 03 11:42:26 crc kubenswrapper[5052]: I1203 11:42:26.030443 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-sb-0" event={"ID":"e14d926b-5dc2-46a4-ab27-dfde118cba89","Type":"ContainerDied","Data":"7b70c188cc2577da2ec80ec8d366b69c4acc8260edefe42f53b57aa50f107f62"} Dec 03 11:42:26 crc kubenswrapper[5052]: I1203 11:42:26.030458 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/memcached-0" event={"ID":"1609a5b5-6666-47e3-b02e-f796d972d331","Type":"ContainerDied","Data":"b86af5c4131b572796dba50ca4f6e60a107928f2bf9062d24ad9be05669cd74f"} Dec 03 11:42:26 crc kubenswrapper[5052]: I1203 11:42:26.030490 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-keystone-listener-5dd6f4bfd-glsv7" event={"ID":"3004edac-eef4-4ddc-a539-4d4294029774","Type":"ContainerStarted","Data":"155d312d1a668b19eb1a81517ac71e09163d5f2409e7b82778a1605373d5fc62"} Dec 03 11:42:26 crc kubenswrapper[5052]: I1203 11:42:26.030502 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"eae5b719-0017-4621-9cb4-73ed6353648e","Type":"ContainerDied","Data":"aa0bc44913bc443884760e2a18faf9da4a9e4a859e8c121cd05e1cd62282505e"} Dec 03 11:42:26 crc kubenswrapper[5052]: I1203 11:42:26.030546 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"eae5b719-0017-4621-9cb4-73ed6353648e","Type":"ContainerDied","Data":"0bcbc83b612aeb3a9d9d8d6b5d25c20cfdd3b0b8c6002297d10049035fb2b217"} Dec 03 11:42:26 crc kubenswrapper[5052]: I1203 11:42:26.030563 5052 scope.go:117] "RemoveContainer" containerID="8da90eb5a7870f9bbe93a949f459805faa0302426245aa70e016cd57e8992d74" Dec 03 11:42:26 crc kubenswrapper[5052]: I1203 11:42:26.069471 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/swift-ring-rebalance-krb94"] Dec 03 11:42:26 crc kubenswrapper[5052]: I1203 11:42:26.088008 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/placement-6d84cdf894-dqcg9"] Dec 03 11:42:26 crc kubenswrapper[5052]: E1203 11:42:26.096667 5052 log.go:32] "RunPodSandbox from runtime service failed" err=< Dec 03 11:42:26 crc kubenswrapper[5052]: rpc error: code = Unknown desc = failed to create pod network sandbox k8s_openstackclient_openstack-kuttl-tests_24f61ea3-1916-46dd-a764-2211a9735d42_0(8b88922e540f6070d7af6e25d3f65dc8376523b6f89a577699e4152ffed729ae): error adding pod openstack-kuttl-tests_openstackclient to CNI network "multus-cni-network": plugin type="multus-shim" name="multus-cni-network" failed (add): CmdAdd (shim): CNI request failed with status 400: 'ContainerID:"8b88922e540f6070d7af6e25d3f65dc8376523b6f89a577699e4152ffed729ae" Netns:"/var/run/netns/fb72d732-2a97-4e7d-ad45-4dde7cb75a6b" IfName:"eth0" Args:"IgnoreUnknown=1;K8S_POD_NAMESPACE=openstack-kuttl-tests;K8S_POD_NAME=openstackclient;K8S_POD_INFRA_CONTAINER_ID=8b88922e540f6070d7af6e25d3f65dc8376523b6f89a577699e4152ffed729ae;K8S_POD_UID=24f61ea3-1916-46dd-a764-2211a9735d42" Path:"" ERRORED: error configuring pod [openstack-kuttl-tests/openstackclient] networking: [openstack-kuttl-tests/openstackclient/24f61ea3-1916-46dd-a764-2211a9735d42:ovn-kubernetes]: error adding container to network "ovn-kubernetes": CNI request failed with status 400: '[openstack-kuttl-tests/openstackclient 8b88922e540f6070d7af6e25d3f65dc8376523b6f89a577699e4152ffed729ae network default NAD default] [openstack-kuttl-tests/openstackclient 8b88922e540f6070d7af6e25d3f65dc8376523b6f89a577699e4152ffed729ae network default NAD default] failed to configure pod interface: pod deleted waiting for OVS port binding for 0a:58:0a:d9:00:bd [10.217.0.189/23] Dec 03 11:42:26 crc kubenswrapper[5052]: ' Dec 03 11:42:26 crc kubenswrapper[5052]: ': StdinData: {"binDir":"/var/lib/cni/bin","clusterNetwork":"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf","cniVersion":"0.3.1","daemonSocketDir":"/run/multus/socket","globalNamespaces":"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv","logLevel":"verbose","logToStderr":true,"name":"multus-cni-network","namespaceIsolation":true,"type":"multus-shim"} Dec 03 11:42:26 crc kubenswrapper[5052]: > Dec 03 11:42:26 crc kubenswrapper[5052]: E1203 11:42:26.096738 5052 kuberuntime_sandbox.go:72] "Failed to create sandbox for pod" err=< Dec 03 11:42:26 crc kubenswrapper[5052]: rpc error: code = Unknown desc = failed to create pod network sandbox k8s_openstackclient_openstack-kuttl-tests_24f61ea3-1916-46dd-a764-2211a9735d42_0(8b88922e540f6070d7af6e25d3f65dc8376523b6f89a577699e4152ffed729ae): error adding pod openstack-kuttl-tests_openstackclient to CNI network "multus-cni-network": plugin type="multus-shim" name="multus-cni-network" failed (add): CmdAdd (shim): CNI request failed with status 400: 'ContainerID:"8b88922e540f6070d7af6e25d3f65dc8376523b6f89a577699e4152ffed729ae" Netns:"/var/run/netns/fb72d732-2a97-4e7d-ad45-4dde7cb75a6b" IfName:"eth0" Args:"IgnoreUnknown=1;K8S_POD_NAMESPACE=openstack-kuttl-tests;K8S_POD_NAME=openstackclient;K8S_POD_INFRA_CONTAINER_ID=8b88922e540f6070d7af6e25d3f65dc8376523b6f89a577699e4152ffed729ae;K8S_POD_UID=24f61ea3-1916-46dd-a764-2211a9735d42" Path:"" ERRORED: error configuring pod [openstack-kuttl-tests/openstackclient] networking: [openstack-kuttl-tests/openstackclient/24f61ea3-1916-46dd-a764-2211a9735d42:ovn-kubernetes]: error adding container to network "ovn-kubernetes": CNI request failed with status 400: '[openstack-kuttl-tests/openstackclient 8b88922e540f6070d7af6e25d3f65dc8376523b6f89a577699e4152ffed729ae network default NAD default] [openstack-kuttl-tests/openstackclient 8b88922e540f6070d7af6e25d3f65dc8376523b6f89a577699e4152ffed729ae network default NAD default] failed to configure pod interface: pod deleted waiting for OVS port binding for 0a:58:0a:d9:00:bd [10.217.0.189/23] Dec 03 11:42:26 crc kubenswrapper[5052]: ' Dec 03 11:42:26 crc kubenswrapper[5052]: ': StdinData: {"binDir":"/var/lib/cni/bin","clusterNetwork":"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf","cniVersion":"0.3.1","daemonSocketDir":"/run/multus/socket","globalNamespaces":"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv","logLevel":"verbose","logToStderr":true,"name":"multus-cni-network","namespaceIsolation":true,"type":"multus-shim"} Dec 03 11:42:26 crc kubenswrapper[5052]: > pod="openstack-kuttl-tests/openstackclient" Dec 03 11:42:26 crc kubenswrapper[5052]: E1203 11:42:26.116724 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="1714a77b6dd6ea9a3fe18b86a4ab32f16a2525fd31478012ffe19746b1260454" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 11:42:26 crc kubenswrapper[5052]: E1203 11:42:26.120708 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="1714a77b6dd6ea9a3fe18b86a4ab32f16a2525fd31478012ffe19746b1260454" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 11:42:26 crc kubenswrapper[5052]: I1203 11:42:26.120984 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/neutron-75475cc596-bfbzz"] Dec 03 11:42:26 crc kubenswrapper[5052]: I1203 11:42:26.121232 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/neutron-75475cc596-bfbzz" podUID="4a23886b-73c6-4b21-8464-07a2dd2e06f6" containerName="neutron-api" containerID="cri-o://0aea4f7a00ba159f8d2bf6070cfb3c8fd85b362b1295e3d63d6e0afddf0002dd" gracePeriod=30 Dec 03 11:42:26 crc kubenswrapper[5052]: I1203 11:42:26.121832 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/neutron-75475cc596-bfbzz" podUID="4a23886b-73c6-4b21-8464-07a2dd2e06f6" containerName="neutron-httpd" containerID="cri-o://f284580bf78ee1c66dcbbafaa358e1788c2f65a743bb73ce1b43008a67f4e28c" gracePeriod=30 Dec 03 11:42:26 crc kubenswrapper[5052]: I1203 11:42:26.127966 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutronbfb7-account-delete-ff26t" Dec 03 11:42:26 crc kubenswrapper[5052]: I1203 11:42:26.145047 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/barbicanc57b-account-delete-nspr5"] Dec 03 11:42:26 crc kubenswrapper[5052]: E1203 11:42:26.145656 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="e14d926b-5dc2-46a4-ab27-dfde118cba89" containerName="ovsdbserver-sb" Dec 03 11:42:26 crc kubenswrapper[5052]: I1203 11:42:26.145679 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="e14d926b-5dc2-46a4-ab27-dfde118cba89" containerName="ovsdbserver-sb" Dec 03 11:42:26 crc kubenswrapper[5052]: E1203 11:42:26.145712 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="e14d926b-5dc2-46a4-ab27-dfde118cba89" containerName="openstack-network-exporter" Dec 03 11:42:26 crc kubenswrapper[5052]: I1203 11:42:26.145720 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="e14d926b-5dc2-46a4-ab27-dfde118cba89" containerName="openstack-network-exporter" Dec 03 11:42:26 crc kubenswrapper[5052]: I1203 11:42:26.145980 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="e14d926b-5dc2-46a4-ab27-dfde118cba89" containerName="openstack-network-exporter" Dec 03 11:42:26 crc kubenswrapper[5052]: I1203 11:42:26.146021 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="e14d926b-5dc2-46a4-ab27-dfde118cba89" containerName="ovsdbserver-sb" Dec 03 11:42:26 crc kubenswrapper[5052]: I1203 11:42:26.146825 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbicanc57b-account-delete-nspr5" Dec 03 11:42:26 crc kubenswrapper[5052]: E1203 11:42:26.153695 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="1714a77b6dd6ea9a3fe18b86a4ab32f16a2525fd31478012ffe19746b1260454" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 11:42:26 crc kubenswrapper[5052]: E1203 11:42:26.155465 5052 prober.go:104] "Probe errored" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" probeType="Readiness" pod="openstack-kuttl-tests/nova-cell1-conductor-0" podUID="2501eda8-a78d-4609-a465-761e89bb291e" containerName="nova-cell1-conductor-conductor" Dec 03 11:42:26 crc kubenswrapper[5052]: I1203 11:42:26.177687 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-db-sync-k6vlv"] Dec 03 11:42:26 crc kubenswrapper[5052]: I1203 11:42:26.195716 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/neutron-69cf8cb7d6-jfb57"] Dec 03 11:42:26 crc kubenswrapper[5052]: I1203 11:42:26.206129 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/glance-db-sync-k6vlv"] Dec 03 11:42:26 crc kubenswrapper[5052]: I1203 11:42:26.219059 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/049153cf-b5da-4a61-8a2d-3f694625642b-config-data" (OuterVolumeSpecName: "config-data") pod "049153cf-b5da-4a61-8a2d-3f694625642b" (UID: "049153cf-b5da-4a61-8a2d-3f694625642b"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:42:26 crc kubenswrapper[5052]: I1203 11:42:26.240322 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement79ca-account-delete-sqtcb" Dec 03 11:42:26 crc kubenswrapper[5052]: I1203 11:42:26.263916 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/ce4d9d20-a900-4b3d-8ee0-46ea28a51262-operator-scripts\") pod \"barbicanc57b-account-delete-nspr5\" (UID: \"ce4d9d20-a900-4b3d-8ee0-46ea28a51262\") " pod="openstack-kuttl-tests/barbicanc57b-account-delete-nspr5" Dec 03 11:42:26 crc kubenswrapper[5052]: I1203 11:42:26.263995 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xhnp9\" (UniqueName: \"kubernetes.io/projected/ce4d9d20-a900-4b3d-8ee0-46ea28a51262-kube-api-access-xhnp9\") pod \"barbicanc57b-account-delete-nspr5\" (UID: \"ce4d9d20-a900-4b3d-8ee0-46ea28a51262\") " pod="openstack-kuttl-tests/barbicanc57b-account-delete-nspr5" Dec 03 11:42:26 crc kubenswrapper[5052]: I1203 11:42:26.264167 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/049153cf-b5da-4a61-8a2d-3f694625642b-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:26 crc kubenswrapper[5052]: I1203 11:42:26.250669 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbicanc57b-account-delete-nspr5"] Dec 03 11:42:26 crc kubenswrapper[5052]: I1203 11:42:26.270229 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/glance78ad-account-delete-7l64w"] Dec 03 11:42:26 crc kubenswrapper[5052]: I1203 11:42:26.274561 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance78ad-account-delete-7l64w" Dec 03 11:42:26 crc kubenswrapper[5052]: I1203 11:42:26.279991 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance78ad-account-delete-7l64w"] Dec 03 11:42:26 crc kubenswrapper[5052]: I1203 11:42:26.375145 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6rdlm\" (UniqueName: \"kubernetes.io/projected/8408f908-2fbf-4933-9361-68af849aad18-kube-api-access-6rdlm\") pod \"glance78ad-account-delete-7l64w\" (UID: \"8408f908-2fbf-4933-9361-68af849aad18\") " pod="openstack-kuttl-tests/glance78ad-account-delete-7l64w" Dec 03 11:42:26 crc kubenswrapper[5052]: I1203 11:42:26.375422 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/ce4d9d20-a900-4b3d-8ee0-46ea28a51262-operator-scripts\") pod \"barbicanc57b-account-delete-nspr5\" (UID: \"ce4d9d20-a900-4b3d-8ee0-46ea28a51262\") " pod="openstack-kuttl-tests/barbicanc57b-account-delete-nspr5" Dec 03 11:42:26 crc kubenswrapper[5052]: I1203 11:42:26.375452 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/8408f908-2fbf-4933-9361-68af849aad18-operator-scripts\") pod \"glance78ad-account-delete-7l64w\" (UID: \"8408f908-2fbf-4933-9361-68af849aad18\") " pod="openstack-kuttl-tests/glance78ad-account-delete-7l64w" Dec 03 11:42:26 crc kubenswrapper[5052]: I1203 11:42:26.375560 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-xhnp9\" (UniqueName: \"kubernetes.io/projected/ce4d9d20-a900-4b3d-8ee0-46ea28a51262-kube-api-access-xhnp9\") pod \"barbicanc57b-account-delete-nspr5\" (UID: \"ce4d9d20-a900-4b3d-8ee0-46ea28a51262\") " pod="openstack-kuttl-tests/barbicanc57b-account-delete-nspr5" Dec 03 11:42:26 crc kubenswrapper[5052]: I1203 11:42:26.384257 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/ce4d9d20-a900-4b3d-8ee0-46ea28a51262-operator-scripts\") pod \"barbicanc57b-account-delete-nspr5\" (UID: \"ce4d9d20-a900-4b3d-8ee0-46ea28a51262\") " pod="openstack-kuttl-tests/barbicanc57b-account-delete-nspr5" Dec 03 11:42:26 crc kubenswrapper[5052]: I1203 11:42:26.406375 5052 scope.go:117] "RemoveContainer" containerID="ef90774de4b83b9fc123ba664ece34787dcdb1ad9fd0d230dacf72b8eee1a1e1" Dec 03 11:42:26 crc kubenswrapper[5052]: I1203 11:42:26.434683 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage01-crc" (UniqueName: "kubernetes.io/local-volume/local-storage01-crc") on node "crc" Dec 03 11:42:26 crc kubenswrapper[5052]: I1203 11:42:26.441354 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-xhnp9\" (UniqueName: \"kubernetes.io/projected/ce4d9d20-a900-4b3d-8ee0-46ea28a51262-kube-api-access-xhnp9\") pod \"barbicanc57b-account-delete-nspr5\" (UID: \"ce4d9d20-a900-4b3d-8ee0-46ea28a51262\") " pod="openstack-kuttl-tests/barbicanc57b-account-delete-nspr5" Dec 03 11:42:26 crc kubenswrapper[5052]: I1203 11:42:26.445813 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/novaapi73a4-account-delete-lrzcr"] Dec 03 11:42:26 crc kubenswrapper[5052]: I1203 11:42:26.447370 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/novaapi73a4-account-delete-lrzcr" Dec 03 11:42:26 crc kubenswrapper[5052]: I1203 11:42:26.461204 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell0-cell-mapping-6lr8z"] Dec 03 11:42:26 crc kubenswrapper[5052]: I1203 11:42:26.479225 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/8408f908-2fbf-4933-9361-68af849aad18-operator-scripts\") pod \"glance78ad-account-delete-7l64w\" (UID: \"8408f908-2fbf-4933-9361-68af849aad18\") " pod="openstack-kuttl-tests/glance78ad-account-delete-7l64w" Dec 03 11:42:26 crc kubenswrapper[5052]: I1203 11:42:26.480386 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-6rdlm\" (UniqueName: \"kubernetes.io/projected/8408f908-2fbf-4933-9361-68af849aad18-kube-api-access-6rdlm\") pod \"glance78ad-account-delete-7l64w\" (UID: \"8408f908-2fbf-4933-9361-68af849aad18\") " pod="openstack-kuttl-tests/glance78ad-account-delete-7l64w" Dec 03 11:42:26 crc kubenswrapper[5052]: I1203 11:42:26.480671 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage01-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage01-crc\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:26 crc kubenswrapper[5052]: I1203 11:42:26.479945 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell0-cell-mapping-6lr8z"] Dec 03 11:42:26 crc kubenswrapper[5052]: I1203 11:42:26.487370 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/8408f908-2fbf-4933-9361-68af849aad18-operator-scripts\") pod \"glance78ad-account-delete-7l64w\" (UID: \"8408f908-2fbf-4933-9361-68af849aad18\") " pod="openstack-kuttl-tests/glance78ad-account-delete-7l64w" Dec 03 11:42:26 crc kubenswrapper[5052]: I1203 11:42:26.492638 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbicanc57b-account-delete-nspr5" Dec 03 11:42:26 crc kubenswrapper[5052]: I1203 11:42:26.501330 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/novaapi73a4-account-delete-lrzcr"] Dec 03 11:42:26 crc kubenswrapper[5052]: I1203 11:42:26.548349 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-6rdlm\" (UniqueName: \"kubernetes.io/projected/8408f908-2fbf-4933-9361-68af849aad18-kube-api-access-6rdlm\") pod \"glance78ad-account-delete-7l64w\" (UID: \"8408f908-2fbf-4933-9361-68af849aad18\") " pod="openstack-kuttl-tests/glance78ad-account-delete-7l64w" Dec 03 11:42:26 crc kubenswrapper[5052]: I1203 11:42:26.583347 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/965f5591-b293-4101-be4a-a5329d88fb69-operator-scripts\") pod \"novaapi73a4-account-delete-lrzcr\" (UID: \"965f5591-b293-4101-be4a-a5329d88fb69\") " pod="openstack-kuttl-tests/novaapi73a4-account-delete-lrzcr" Dec 03 11:42:26 crc kubenswrapper[5052]: I1203 11:42:26.583446 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xk6mh\" (UniqueName: \"kubernetes.io/projected/965f5591-b293-4101-be4a-a5329d88fb69-kube-api-access-xk6mh\") pod \"novaapi73a4-account-delete-lrzcr\" (UID: \"965f5591-b293-4101-be4a-a5329d88fb69\") " pod="openstack-kuttl-tests/novaapi73a4-account-delete-lrzcr" Dec 03 11:42:26 crc kubenswrapper[5052]: I1203 11:42:26.610553 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e14d926b-5dc2-46a4-ab27-dfde118cba89-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "e14d926b-5dc2-46a4-ab27-dfde118cba89" (UID: "e14d926b-5dc2-46a4-ab27-dfde118cba89"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:42:26 crc kubenswrapper[5052]: I1203 11:42:26.616521 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1609a5b5-6666-47e3-b02e-f796d972d331-memcached-tls-certs" (OuterVolumeSpecName: "memcached-tls-certs") pod "1609a5b5-6666-47e3-b02e-f796d972d331" (UID: "1609a5b5-6666-47e3-b02e-f796d972d331"). InnerVolumeSpecName "memcached-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:42:26 crc kubenswrapper[5052]: I1203 11:42:26.621597 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/rabbitmq-server-0"] Dec 03 11:42:26 crc kubenswrapper[5052]: I1203 11:42:26.649120 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/049153cf-b5da-4a61-8a2d-3f694625642b-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "049153cf-b5da-4a61-8a2d-3f694625642b" (UID: "049153cf-b5da-4a61-8a2d-3f694625642b"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:42:26 crc kubenswrapper[5052]: I1203 11:42:26.683251 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/049153cf-b5da-4a61-8a2d-3f694625642b-nova-novncproxy-tls-certs" (OuterVolumeSpecName: "nova-novncproxy-tls-certs") pod "049153cf-b5da-4a61-8a2d-3f694625642b" (UID: "049153cf-b5da-4a61-8a2d-3f694625642b"). InnerVolumeSpecName "nova-novncproxy-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:42:26 crc kubenswrapper[5052]: E1203 11:42:26.696565 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/rabbitmq-config-data: configmap "rabbitmq-config-data" not found Dec 03 11:42:26 crc kubenswrapper[5052]: E1203 11:42:26.696631 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/rabbitmq-cell1-config-data: configmap "rabbitmq-cell1-config-data" not found Dec 03 11:42:26 crc kubenswrapper[5052]: E1203 11:42:26.696667 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9944d9af-7a7b-4773-8279-72914d63ef5f-config-data podName:9944d9af-7a7b-4773-8279-72914d63ef5f nodeName:}" failed. No retries permitted until 2025-12-03 11:42:27.19664378 +0000 UTC m=+1337.784885614 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "config-data" (UniqueName: "kubernetes.io/configmap/9944d9af-7a7b-4773-8279-72914d63ef5f-config-data") pod "rabbitmq-server-0" (UID: "9944d9af-7a7b-4773-8279-72914d63ef5f") : configmap "rabbitmq-config-data" not found Dec 03 11:42:26 crc kubenswrapper[5052]: E1203 11:42:26.696687 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/daa521fc-7e1d-44d2-b7d8-6aab003cfb66-config-data podName:daa521fc-7e1d-44d2-b7d8-6aab003cfb66 nodeName:}" failed. No retries permitted until 2025-12-03 11:42:28.696680111 +0000 UTC m=+1339.284921945 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "config-data" (UniqueName: "kubernetes.io/configmap/daa521fc-7e1d-44d2-b7d8-6aab003cfb66-config-data") pod "rabbitmq-cell1-server-0" (UID: "daa521fc-7e1d-44d2-b7d8-6aab003cfb66") : configmap "rabbitmq-cell1-config-data" not found Dec 03 11:42:26 crc kubenswrapper[5052]: I1203 11:42:26.695794 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/965f5591-b293-4101-be4a-a5329d88fb69-operator-scripts\") pod \"novaapi73a4-account-delete-lrzcr\" (UID: \"965f5591-b293-4101-be4a-a5329d88fb69\") " pod="openstack-kuttl-tests/novaapi73a4-account-delete-lrzcr" Dec 03 11:42:26 crc kubenswrapper[5052]: I1203 11:42:26.696883 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-xk6mh\" (UniqueName: \"kubernetes.io/projected/965f5591-b293-4101-be4a-a5329d88fb69-kube-api-access-xk6mh\") pod \"novaapi73a4-account-delete-lrzcr\" (UID: \"965f5591-b293-4101-be4a-a5329d88fb69\") " pod="openstack-kuttl-tests/novaapi73a4-account-delete-lrzcr" Dec 03 11:42:26 crc kubenswrapper[5052]: I1203 11:42:26.697011 5052 reconciler_common.go:293] "Volume detached for volume \"nova-novncproxy-tls-certs\" (UniqueName: \"kubernetes.io/secret/049153cf-b5da-4a61-8a2d-3f694625642b-nova-novncproxy-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:26 crc kubenswrapper[5052]: I1203 11:42:26.697024 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/049153cf-b5da-4a61-8a2d-3f694625642b-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:26 crc kubenswrapper[5052]: I1203 11:42:26.697036 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e14d926b-5dc2-46a4-ab27-dfde118cba89-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:26 crc kubenswrapper[5052]: I1203 11:42:26.697046 5052 reconciler_common.go:293] "Volume detached for volume \"memcached-tls-certs\" (UniqueName: \"kubernetes.io/secret/1609a5b5-6666-47e3-b02e-f796d972d331-memcached-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:26 crc kubenswrapper[5052]: I1203 11:42:26.702804 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/965f5591-b293-4101-be4a-a5329d88fb69-operator-scripts\") pod \"novaapi73a4-account-delete-lrzcr\" (UID: \"965f5591-b293-4101-be4a-a5329d88fb69\") " pod="openstack-kuttl-tests/novaapi73a4-account-delete-lrzcr" Dec 03 11:42:26 crc kubenswrapper[5052]: I1203 11:42:26.752915 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/nova-metadata-0" podUID="f98f227a-397b-4665-bf18-e00b105ce058" containerName="nova-metadata-metadata" probeResult="failure" output="Get \"https://10.217.0.185:8775/\": read tcp 10.217.0.2:50408->10.217.0.185:8775: read: connection reset by peer" Dec 03 11:42:26 crc kubenswrapper[5052]: I1203 11:42:26.753188 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/nova-metadata-0" podUID="f98f227a-397b-4665-bf18-e00b105ce058" containerName="nova-metadata-log" probeResult="failure" output="Get \"https://10.217.0.185:8775/\": read tcp 10.217.0.2:50414->10.217.0.185:8775: read: connection reset by peer" Dec 03 11:42:26 crc kubenswrapper[5052]: I1203 11:42:26.797490 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/049153cf-b5da-4a61-8a2d-3f694625642b-vencrypt-tls-certs" (OuterVolumeSpecName: "vencrypt-tls-certs") pod "049153cf-b5da-4a61-8a2d-3f694625642b" (UID: "049153cf-b5da-4a61-8a2d-3f694625642b"). InnerVolumeSpecName "vencrypt-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:42:26 crc kubenswrapper[5052]: I1203 11:42:26.798807 5052 reconciler_common.go:293] "Volume detached for volume \"vencrypt-tls-certs\" (UniqueName: \"kubernetes.io/secret/049153cf-b5da-4a61-8a2d-3f694625642b-vencrypt-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:26 crc kubenswrapper[5052]: I1203 11:42:26.824443 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell1-cell-mapping-wpd4q"] Dec 03 11:42:26 crc kubenswrapper[5052]: I1203 11:42:26.839714 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell1-cell-mapping-wpd4q"] Dec 03 11:42:26 crc kubenswrapper[5052]: E1203 11:42:26.843569 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="455e8792612e9ef5fbedefc9390ff87ea262cbd9365b333801dd2dc51224737e" cmd=["/usr/local/bin/container-scripts/status_check.sh"] Dec 03 11:42:26 crc kubenswrapper[5052]: I1203 11:42:26.851326 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/novacell03d99-account-delete-5gvmz"] Dec 03 11:42:26 crc kubenswrapper[5052]: I1203 11:42:26.852787 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/novacell03d99-account-delete-5gvmz" Dec 03 11:42:26 crc kubenswrapper[5052]: I1203 11:42:26.865622 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/placement-699ddb8646-rfm4p"] Dec 03 11:42:26 crc kubenswrapper[5052]: I1203 11:42:26.865889 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/placement-699ddb8646-rfm4p" podUID="0b4b31e5-e82d-4d44-9c8f-d001484293b6" containerName="placement-log" containerID="cri-o://2289f14f2059fc517e80d4db86044454e7b7fc71a8ff28682e6b78283502038f" gracePeriod=30 Dec 03 11:42:26 crc kubenswrapper[5052]: I1203 11:42:26.866040 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/placement-699ddb8646-rfm4p" podUID="0b4b31e5-e82d-4d44-9c8f-d001484293b6" containerName="placement-api" containerID="cri-o://77dd24bffc9318ae0172000a2a92f5dd4f72dcfd986df49b78a78df630d16741" gracePeriod=30 Dec 03 11:42:26 crc kubenswrapper[5052]: I1203 11:42:26.890295 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-xk6mh\" (UniqueName: \"kubernetes.io/projected/965f5591-b293-4101-be4a-a5329d88fb69-kube-api-access-xk6mh\") pod \"novaapi73a4-account-delete-lrzcr\" (UID: \"965f5591-b293-4101-be4a-a5329d88fb69\") " pod="openstack-kuttl-tests/novaapi73a4-account-delete-lrzcr" Dec 03 11:42:26 crc kubenswrapper[5052]: E1203 11:42:26.890443 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="455e8792612e9ef5fbedefc9390ff87ea262cbd9365b333801dd2dc51224737e" cmd=["/usr/local/bin/container-scripts/status_check.sh"] Dec 03 11:42:26 crc kubenswrapper[5052]: E1203 11:42:26.901832 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="455e8792612e9ef5fbedefc9390ff87ea262cbd9365b333801dd2dc51224737e" cmd=["/usr/local/bin/container-scripts/status_check.sh"] Dec 03 11:42:26 crc kubenswrapper[5052]: E1203 11:42:26.901885 5052 prober.go:104] "Probe errored" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" probeType="Readiness" pod="openstack-kuttl-tests/ovn-northd-0" podUID="a30eaf15-65ff-426b-8ac2-1ed5a369886a" containerName="ovn-northd" Dec 03 11:42:26 crc kubenswrapper[5052]: I1203 11:42:26.994783 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/novacell03d99-account-delete-5gvmz"] Dec 03 11:42:27 crc kubenswrapper[5052]: I1203 11:42:27.007685 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zmn62\" (UniqueName: \"kubernetes.io/projected/f274168d-df51-4f50-b014-1ea4e85370d7-kube-api-access-zmn62\") pod \"novacell03d99-account-delete-5gvmz\" (UID: \"f274168d-df51-4f50-b014-1ea4e85370d7\") " pod="openstack-kuttl-tests/novacell03d99-account-delete-5gvmz" Dec 03 11:42:27 crc kubenswrapper[5052]: I1203 11:42:27.007822 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/f274168d-df51-4f50-b014-1ea4e85370d7-operator-scripts\") pod \"novacell03d99-account-delete-5gvmz\" (UID: \"f274168d-df51-4f50-b014-1ea4e85370d7\") " pod="openstack-kuttl-tests/novacell03d99-account-delete-5gvmz" Dec 03 11:42:27 crc kubenswrapper[5052]: I1203 11:42:27.067695 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e14d926b-5dc2-46a4-ab27-dfde118cba89-metrics-certs-tls-certs" (OuterVolumeSpecName: "metrics-certs-tls-certs") pod "e14d926b-5dc2-46a4-ab27-dfde118cba89" (UID: "e14d926b-5dc2-46a4-ab27-dfde118cba89"). InnerVolumeSpecName "metrics-certs-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:42:27 crc kubenswrapper[5052]: I1203 11:42:27.102837 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 11:42:27 crc kubenswrapper[5052]: I1203 11:42:27.111662 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-zmn62\" (UniqueName: \"kubernetes.io/projected/f274168d-df51-4f50-b014-1ea4e85370d7-kube-api-access-zmn62\") pod \"novacell03d99-account-delete-5gvmz\" (UID: \"f274168d-df51-4f50-b014-1ea4e85370d7\") " pod="openstack-kuttl-tests/novacell03d99-account-delete-5gvmz" Dec 03 11:42:27 crc kubenswrapper[5052]: I1203 11:42:27.111787 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/f274168d-df51-4f50-b014-1ea4e85370d7-operator-scripts\") pod \"novacell03d99-account-delete-5gvmz\" (UID: \"f274168d-df51-4f50-b014-1ea4e85370d7\") " pod="openstack-kuttl-tests/novacell03d99-account-delete-5gvmz" Dec 03 11:42:27 crc kubenswrapper[5052]: I1203 11:42:27.111920 5052 reconciler_common.go:293] "Volume detached for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/e14d926b-5dc2-46a4-ab27-dfde118cba89-metrics-certs-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:27 crc kubenswrapper[5052]: E1203 11:42:27.112003 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 11:42:27 crc kubenswrapper[5052]: E1203 11:42:27.112052 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f274168d-df51-4f50-b014-1ea4e85370d7-operator-scripts podName:f274168d-df51-4f50-b014-1ea4e85370d7 nodeName:}" failed. No retries permitted until 2025-12-03 11:42:27.612035845 +0000 UTC m=+1338.200277609 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/f274168d-df51-4f50-b014-1ea4e85370d7-operator-scripts") pod "novacell03d99-account-delete-5gvmz" (UID: "f274168d-df51-4f50-b014-1ea4e85370d7") : configmap "openstack-scripts" not found Dec 03 11:42:27 crc kubenswrapper[5052]: I1203 11:42:27.115931 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/swift-proxy-fd9678558-qx8wh"] Dec 03 11:42:27 crc kubenswrapper[5052]: I1203 11:42:27.116235 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-proxy-fd9678558-qx8wh" podUID="fd14a5c0-3402-4891-a726-32d75b1b7857" containerName="proxy-httpd" containerID="cri-o://3132bd07375ba7c18c7bbb9921d64b3f404ab0c85abce7e27085e92e0ba00bea" gracePeriod=30 Dec 03 11:42:27 crc kubenswrapper[5052]: I1203 11:42:27.116374 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-proxy-fd9678558-qx8wh" podUID="fd14a5c0-3402-4891-a726-32d75b1b7857" containerName="proxy-server" containerID="cri-o://def28b7790031f83be9c79b54dbbc74332f5e5917d604dfb281ac32589501053" gracePeriod=30 Dec 03 11:42:27 crc kubenswrapper[5052]: I1203 11:42:27.156703 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/cinder-api-0" podUID="ee46c3a4-5e21-4f6b-a944-8cf96d87506e" containerName="cinder-api" probeResult="failure" output="Get \"https://10.217.0.148:8776/healthcheck\": read tcp 10.217.0.2:37294->10.217.0.148:8776: read: connection reset by peer" Dec 03 11:42:27 crc kubenswrapper[5052]: E1203 11:42:27.156860 5052 projected.go:194] Error preparing data for projected volume kube-api-access-zmn62 for pod openstack-kuttl-tests/novacell03d99-account-delete-5gvmz: failed to fetch token: serviceaccounts "galera-openstack" not found Dec 03 11:42:27 crc kubenswrapper[5052]: E1203 11:42:27.156921 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f274168d-df51-4f50-b014-1ea4e85370d7-kube-api-access-zmn62 podName:f274168d-df51-4f50-b014-1ea4e85370d7 nodeName:}" failed. No retries permitted until 2025-12-03 11:42:27.65690441 +0000 UTC m=+1338.245146174 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-zmn62" (UniqueName: "kubernetes.io/projected/f274168d-df51-4f50-b014-1ea4e85370d7-kube-api-access-zmn62") pod "novacell03d99-account-delete-5gvmz" (UID: "f274168d-df51-4f50-b014-1ea4e85370d7") : failed to fetch token: serviceaccounts "galera-openstack" not found Dec 03 11:42:27 crc kubenswrapper[5052]: I1203 11:42:27.160489 5052 generic.go:334] "Generic (PLEG): container finished" podID="f98f227a-397b-4665-bf18-e00b105ce058" containerID="b2697ef9ab442e1740e9992a4ae8a2ad183a400760ac5eea2d853dc944ed7fa8" exitCode=0 Dec 03 11:42:27 crc kubenswrapper[5052]: I1203 11:42:27.160580 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"f98f227a-397b-4665-bf18-e00b105ce058","Type":"ContainerDied","Data":"b2697ef9ab442e1740e9992a4ae8a2ad183a400760ac5eea2d853dc944ed7fa8"} Dec 03 11:42:27 crc kubenswrapper[5052]: I1203 11:42:27.175956 5052 generic.go:334] "Generic (PLEG): container finished" podID="665a736c-d550-4160-a8bb-619d1d5ea826" containerID="25c8b36672104ddddb9cda8135c26753231310450ea8bff9f6700b865bc3d056" exitCode=0 Dec 03 11:42:27 crc kubenswrapper[5052]: I1203 11:42:27.176527 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"665a736c-d550-4160-a8bb-619d1d5ea826","Type":"ContainerDied","Data":"25c8b36672104ddddb9cda8135c26753231310450ea8bff9f6700b865bc3d056"} Dec 03 11:42:27 crc kubenswrapper[5052]: I1203 11:42:27.191628 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e14d926b-5dc2-46a4-ab27-dfde118cba89-ovsdbserver-sb-tls-certs" (OuterVolumeSpecName: "ovsdbserver-sb-tls-certs") pod "e14d926b-5dc2-46a4-ab27-dfde118cba89" (UID: "e14d926b-5dc2-46a4-ab27-dfde118cba89"). InnerVolumeSpecName "ovsdbserver-sb-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:42:27 crc kubenswrapper[5052]: I1203 11:42:27.202607 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/swift-storage-0"] Dec 03 11:42:27 crc kubenswrapper[5052]: I1203 11:42:27.203220 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="d19f0c21-bb49-40a1-87fa-e55188308729" containerName="account-server" containerID="cri-o://3a64f478f235a4fb6291efc13b9a7d651e52032e9126c5a8ee70cbaa6f7bc0c4" gracePeriod=30 Dec 03 11:42:27 crc kubenswrapper[5052]: I1203 11:42:27.203242 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="d19f0c21-bb49-40a1-87fa-e55188308729" containerName="swift-recon-cron" containerID="cri-o://907a694952444699abbc7ea53d63f1c5b7ce75550d76d99fb7d54e27cbbb8c66" gracePeriod=30 Dec 03 11:42:27 crc kubenswrapper[5052]: I1203 11:42:27.203377 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="d19f0c21-bb49-40a1-87fa-e55188308729" containerName="rsync" containerID="cri-o://198e5b3e40307e3ae065a4823c3cadc6edfe24c354d3dc8885bdeb1b65e4a57a" gracePeriod=30 Dec 03 11:42:27 crc kubenswrapper[5052]: I1203 11:42:27.203422 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="d19f0c21-bb49-40a1-87fa-e55188308729" containerName="container-auditor" containerID="cri-o://66b82b9a9602e546754f3199323f445795063d7d84d38d42c9e7bdead8e2d119" gracePeriod=30 Dec 03 11:42:27 crc kubenswrapper[5052]: I1203 11:42:27.203436 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="d19f0c21-bb49-40a1-87fa-e55188308729" containerName="container-replicator" containerID="cri-o://8b82c3eafedcad816bdd8f848dfa51930b6fcd64c66cf454cbb3af2debd04b86" gracePeriod=30 Dec 03 11:42:27 crc kubenswrapper[5052]: I1203 11:42:27.203482 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="d19f0c21-bb49-40a1-87fa-e55188308729" containerName="container-server" containerID="cri-o://cc5f444de12548f88c1caa6a61bd1efc6e2fd9da4c672919931aac6ba15d3f15" gracePeriod=30 Dec 03 11:42:27 crc kubenswrapper[5052]: I1203 11:42:27.203689 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="d19f0c21-bb49-40a1-87fa-e55188308729" containerName="object-expirer" containerID="cri-o://1feeef96663c3d2882692e8837c421c0dab4a5a5be167acf6810b788e07033be" gracePeriod=30 Dec 03 11:42:27 crc kubenswrapper[5052]: I1203 11:42:27.203754 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="d19f0c21-bb49-40a1-87fa-e55188308729" containerName="object-updater" containerID="cri-o://e0f3046cd638bd8d68fc0858b30e3c71c02254e1579ac5fdf0d14362c56787ae" gracePeriod=30 Dec 03 11:42:27 crc kubenswrapper[5052]: I1203 11:42:27.203793 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="d19f0c21-bb49-40a1-87fa-e55188308729" containerName="object-replicator" containerID="cri-o://480a4695021f8caec2d8b5d9b753bac979ecb6a2f7643abdd8ce149daf4a8b4a" gracePeriod=30 Dec 03 11:42:27 crc kubenswrapper[5052]: I1203 11:42:27.203823 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="d19f0c21-bb49-40a1-87fa-e55188308729" containerName="object-server" containerID="cri-o://6f7206455d66da31b1ca0c6033a701e9ef605c548a9309c2b0967a2b9726a603" gracePeriod=30 Dec 03 11:42:27 crc kubenswrapper[5052]: I1203 11:42:27.203879 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="d19f0c21-bb49-40a1-87fa-e55188308729" containerName="container-updater" containerID="cri-o://e36b32b951337695419521a1f3271a1e8e2cac46f10174c506c5e0e369ad8704" gracePeriod=30 Dec 03 11:42:27 crc kubenswrapper[5052]: I1203 11:42:27.203927 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="d19f0c21-bb49-40a1-87fa-e55188308729" containerName="object-auditor" containerID="cri-o://842157c852d9af69e655dafb6e96753c92d5a174fac182e141f97a78bcaf8577" gracePeriod=30 Dec 03 11:42:27 crc kubenswrapper[5052]: I1203 11:42:27.203957 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="d19f0c21-bb49-40a1-87fa-e55188308729" containerName="account-auditor" containerID="cri-o://e776b8b08a7136d5453955d041fce850605188fd0e17c51f2024905c9aa86aec" gracePeriod=30 Dec 03 11:42:27 crc kubenswrapper[5052]: I1203 11:42:27.204000 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="d19f0c21-bb49-40a1-87fa-e55188308729" containerName="account-reaper" containerID="cri-o://38627e147a9f39cc2d052969a6aa30eeb14f6d31573e540afa5859023f9656d8" gracePeriod=30 Dec 03 11:42:27 crc kubenswrapper[5052]: I1203 11:42:27.204030 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="d19f0c21-bb49-40a1-87fa-e55188308729" containerName="account-replicator" containerID="cri-o://375e74dc879bff82efc908e9893ad30ab3c979218f143e6a3e6fcd9936d34248" gracePeriod=30 Dec 03 11:42:27 crc kubenswrapper[5052]: I1203 11:42:27.214642 5052 reconciler_common.go:293] "Volume detached for volume \"ovsdbserver-sb-tls-certs\" (UniqueName: \"kubernetes.io/secret/e14d926b-5dc2-46a4-ab27-dfde118cba89-ovsdbserver-sb-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:27 crc kubenswrapper[5052]: E1203 11:42:27.214730 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/rabbitmq-config-data: configmap "rabbitmq-config-data" not found Dec 03 11:42:27 crc kubenswrapper[5052]: E1203 11:42:27.214787 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9944d9af-7a7b-4773-8279-72914d63ef5f-config-data podName:9944d9af-7a7b-4773-8279-72914d63ef5f nodeName:}" failed. No retries permitted until 2025-12-03 11:42:28.214769584 +0000 UTC m=+1338.803011348 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "config-data" (UniqueName: "kubernetes.io/configmap/9944d9af-7a7b-4773-8279-72914d63ef5f-config-data") pod "rabbitmq-server-0" (UID: "9944d9af-7a7b-4773-8279-72914d63ef5f") : configmap "rabbitmq-config-data" not found Dec 03 11:42:27 crc kubenswrapper[5052]: I1203 11:42:27.252011 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/placement-6d84cdf894-dqcg9"] Dec 03 11:42:27 crc kubenswrapper[5052]: I1203 11:42:27.257891 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-df67d6cb4-8sllj" event={"ID":"b88af976-65f2-4d29-8c14-d5c43e23a459","Type":"ContainerStarted","Data":"8af6c176afbf0f72f2a163a363b1066519544bbb730d4dbb58a5ea5479468f2f"} Dec 03 11:42:27 crc kubenswrapper[5052]: I1203 11:42:27.262252 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-keystone-listener-5dd6f4bfd-glsv7" event={"ID":"3004edac-eef4-4ddc-a539-4d4294029774","Type":"ContainerStarted","Data":"e7be74f05a4bcf8543f0c42a95e74e455c72cfb86b1351dc8a08f45d49fb16ab"} Dec 03 11:42:27 crc kubenswrapper[5052]: I1203 11:42:27.276858 5052 generic.go:334] "Generic (PLEG): container finished" podID="ecbf8b8e-cdb4-45c3-9455-d0dbc551356e" containerID="d9bef6e726b038ae865fa13770a700b7a8625ffee06c08933704155c1695d94f" exitCode=0 Dec 03 11:42:27 crc kubenswrapper[5052]: I1203 11:42:27.276960 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-cell1-galera-0" event={"ID":"ecbf8b8e-cdb4-45c3-9455-d0dbc551356e","Type":"ContainerDied","Data":"d9bef6e726b038ae865fa13770a700b7a8625ffee06c08933704155c1695d94f"} Dec 03 11:42:27 crc kubenswrapper[5052]: I1203 11:42:27.297392 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-nb-0"] Dec 03 11:42:27 crc kubenswrapper[5052]: I1203 11:42:27.302118 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-6858c48994-zq757" event={"ID":"48409c36-734b-4b1d-adee-0e9981d635c9","Type":"ContainerStarted","Data":"85e4be2eed800fd86dad67db18b568783e795640a6b911659efa1aaed0d360fb"} Dec 03 11:42:27 crc kubenswrapper[5052]: I1203 11:42:27.305361 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-6d84cdf894-dqcg9" event={"ID":"0c7d3107-6dd9-4aa2-a392-68a213013508","Type":"ContainerStarted","Data":"f9ff1557270cd7f8f08565a1b152ff021c2e83623d66dbe9dd1606df3722c53a"} Dec 03 11:42:27 crc kubenswrapper[5052]: I1203 11:42:27.306379 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-nb-0"] Dec 03 11:42:27 crc kubenswrapper[5052]: I1203 11:42:27.308368 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-69cf8cb7d6-jfb57" event={"ID":"8f1442a2-a28b-4784-934a-34b417dbcce9","Type":"ContainerStarted","Data":"3ccc56079fd2b625eab01098a7f96885b1126387a64abae713733304004b4545"} Dec 03 11:42:27 crc kubenswrapper[5052]: I1203 11:42:27.312843 5052 generic.go:334] "Generic (PLEG): container finished" podID="2501eda8-a78d-4609-a465-761e89bb291e" containerID="1714a77b6dd6ea9a3fe18b86a4ab32f16a2525fd31478012ffe19746b1260454" exitCode=0 Dec 03 11:42:27 crc kubenswrapper[5052]: I1203 11:42:27.312937 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-conductor-0" event={"ID":"2501eda8-a78d-4609-a465-761e89bb291e","Type":"ContainerDied","Data":"1714a77b6dd6ea9a3fe18b86a4ab32f16a2525fd31478012ffe19746b1260454"} Dec 03 11:42:27 crc kubenswrapper[5052]: I1203 11:42:27.316691 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-worker-67c775d6bc-kwxzt" event={"ID":"222f033c-3660-4347-b621-7122c4c94929","Type":"ContainerStarted","Data":"e34cd5241cdc0d7ab2c3e9a3005641eb15aa576865c7a50550043b4722e25959"} Dec 03 11:42:27 crc kubenswrapper[5052]: I1203 11:42:27.320273 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-54w7t"] Dec 03 11:42:27 crc kubenswrapper[5052]: I1203 11:42:27.321748 5052 generic.go:334] "Generic (PLEG): container finished" podID="4a23886b-73c6-4b21-8464-07a2dd2e06f6" containerID="f284580bf78ee1c66dcbbafaa358e1788c2f65a743bb73ce1b43008a67f4e28c" exitCode=0 Dec 03 11:42:27 crc kubenswrapper[5052]: I1203 11:42:27.326741 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-75475cc596-bfbzz" event={"ID":"4a23886b-73c6-4b21-8464-07a2dd2e06f6","Type":"ContainerDied","Data":"f284580bf78ee1c66dcbbafaa358e1788c2f65a743bb73ce1b43008a67f4e28c"} Dec 03 11:42:27 crc kubenswrapper[5052]: I1203 11:42:27.327048 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-54w7t" Dec 03 11:42:27 crc kubenswrapper[5052]: I1203 11:42:27.340955 5052 generic.go:334] "Generic (PLEG): container finished" podID="eae5b719-0017-4621-9cb4-73ed6353648e" containerID="7d22ae141a66b8ccb952b7ed5aa56004799cd8db057a131cc24fa1331ea7e389" exitCode=0 Dec 03 11:42:27 crc kubenswrapper[5052]: I1203 11:42:27.341691 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"eae5b719-0017-4621-9cb4-73ed6353648e","Type":"ContainerDied","Data":"7d22ae141a66b8ccb952b7ed5aa56004799cd8db057a131cc24fa1331ea7e389"} Dec 03 11:42:27 crc kubenswrapper[5052]: I1203 11:42:27.348371 5052 kubelet_pods.go:1007] "Unable to retrieve pull secret, the image pull may not succeed." pod="openstack-kuttl-tests/barbican-worker-67c775d6bc-kwxzt" secret="" err="secret \"barbican-barbican-dockercfg-bq7sh\" not found" Dec 03 11:42:27 crc kubenswrapper[5052]: I1203 11:42:27.401152 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-54w7t"] Dec 03 11:42:27 crc kubenswrapper[5052]: I1203 11:42:27.410825 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-api-66f4bb9676-j6vzc"] Dec 03 11:42:27 crc kubenswrapper[5052]: I1203 11:42:27.411119 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/barbican-api-66f4bb9676-j6vzc" podUID="fe69e54e-ae29-49e2-8a21-0ff81cc0100f" containerName="barbican-api-log" containerID="cri-o://c4c73ffbd6756af9b104cfcef9220c7341172a6c5dadc45edcc42921bbd74050" gracePeriod=30 Dec 03 11:42:27 crc kubenswrapper[5052]: I1203 11:42:27.416244 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/barbican-api-66f4bb9676-j6vzc" podUID="fe69e54e-ae29-49e2-8a21-0ff81cc0100f" containerName="barbican-api" containerID="cri-o://4c92b1739b5436291f4de41014a637fd37a00db71202d30b8ea1ae9006123ea8" gracePeriod=30 Dec 03 11:42:27 crc kubenswrapper[5052]: I1203 11:42:27.522024 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c546fad2c7df08ae53296fd9af23e0254a60b21cb613124162dce41ad0a92662" Dec 03 11:42:27 crc kubenswrapper[5052]: I1203 11:42:27.522476 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstackclient" Dec 03 11:42:27 crc kubenswrapper[5052]: I1203 11:42:27.642274 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/f274168d-df51-4f50-b014-1ea4e85370d7-operator-scripts\") pod \"novacell03d99-account-delete-5gvmz\" (UID: \"f274168d-df51-4f50-b014-1ea4e85370d7\") " pod="openstack-kuttl-tests/novacell03d99-account-delete-5gvmz" Dec 03 11:42:27 crc kubenswrapper[5052]: I1203 11:42:27.642558 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ws6lz\" (UniqueName: \"kubernetes.io/projected/fc5f71c9-f943-4ba6-a8eb-8e16ed558675-kube-api-access-ws6lz\") pod \"dnsmasq-dnsmasq-84b9f45d47-54w7t\" (UID: \"fc5f71c9-f943-4ba6-a8eb-8e16ed558675\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-54w7t" Dec 03 11:42:27 crc kubenswrapper[5052]: I1203 11:42:27.642596 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/fc5f71c9-f943-4ba6-a8eb-8e16ed558675-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-84b9f45d47-54w7t\" (UID: \"fc5f71c9-f943-4ba6-a8eb-8e16ed558675\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-54w7t" Dec 03 11:42:27 crc kubenswrapper[5052]: I1203 11:42:27.642624 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/fc5f71c9-f943-4ba6-a8eb-8e16ed558675-config\") pod \"dnsmasq-dnsmasq-84b9f45d47-54w7t\" (UID: \"fc5f71c9-f943-4ba6-a8eb-8e16ed558675\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-54w7t" Dec 03 11:42:27 crc kubenswrapper[5052]: E1203 11:42:27.643688 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/barbican-config-data: secret "barbican-config-data" not found Dec 03 11:42:27 crc kubenswrapper[5052]: E1203 11:42:27.643744 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/222f033c-3660-4347-b621-7122c4c94929-config-data podName:222f033c-3660-4347-b621-7122c4c94929 nodeName:}" failed. No retries permitted until 2025-12-03 11:42:28.143726344 +0000 UTC m=+1338.731968138 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "config-data" (UniqueName: "kubernetes.io/secret/222f033c-3660-4347-b621-7122c4c94929-config-data") pod "barbican-worker-67c775d6bc-kwxzt" (UID: "222f033c-3660-4347-b621-7122c4c94929") : secret "barbican-config-data" not found Dec 03 11:42:27 crc kubenswrapper[5052]: I1203 11:42:27.648281 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-marketplace/redhat-operators-bvp4v" Dec 03 11:42:27 crc kubenswrapper[5052]: E1203 11:42:27.649409 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/combined-ca-bundle: secret "combined-ca-bundle" not found Dec 03 11:42:27 crc kubenswrapper[5052]: E1203 11:42:27.649498 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/222f033c-3660-4347-b621-7122c4c94929-combined-ca-bundle podName:222f033c-3660-4347-b621-7122c4c94929 nodeName:}" failed. No retries permitted until 2025-12-03 11:42:28.149466171 +0000 UTC m=+1338.737707945 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "combined-ca-bundle" (UniqueName: "kubernetes.io/secret/222f033c-3660-4347-b621-7122c4c94929-combined-ca-bundle") pod "barbican-worker-67c775d6bc-kwxzt" (UID: "222f033c-3660-4347-b621-7122c4c94929") : secret "combined-ca-bundle" not found Dec 03 11:42:27 crc kubenswrapper[5052]: E1203 11:42:27.649578 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 11:42:27 crc kubenswrapper[5052]: E1203 11:42:27.649624 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f274168d-df51-4f50-b014-1ea4e85370d7-operator-scripts podName:f274168d-df51-4f50-b014-1ea4e85370d7 nodeName:}" failed. No retries permitted until 2025-12-03 11:42:28.649612595 +0000 UTC m=+1339.237854359 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/f274168d-df51-4f50-b014-1ea4e85370d7-operator-scripts") pod "novacell03d99-account-delete-5gvmz" (UID: "f274168d-df51-4f50-b014-1ea4e85370d7") : configmap "openstack-scripts" not found Dec 03 11:42:27 crc kubenswrapper[5052]: E1203 11:42:27.649694 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/barbican-worker-config-data: secret "barbican-worker-config-data" not found Dec 03 11:42:27 crc kubenswrapper[5052]: E1203 11:42:27.649757 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/222f033c-3660-4347-b621-7122c4c94929-config-data-custom podName:222f033c-3660-4347-b621-7122c4c94929 nodeName:}" failed. No retries permitted until 2025-12-03 11:42:28.149742789 +0000 UTC m=+1338.737984553 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "config-data-custom" (UniqueName: "kubernetes.io/secret/222f033c-3660-4347-b621-7122c4c94929-config-data-custom") pod "barbican-worker-67c775d6bc-kwxzt" (UID: "222f033c-3660-4347-b621-7122c4c94929") : secret "barbican-worker-config-data" not found Dec 03 11:42:27 crc kubenswrapper[5052]: I1203 11:42:27.656790 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 11:42:27 crc kubenswrapper[5052]: I1203 11:42:27.744763 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-zmn62\" (UniqueName: \"kubernetes.io/projected/f274168d-df51-4f50-b014-1ea4e85370d7-kube-api-access-zmn62\") pod \"novacell03d99-account-delete-5gvmz\" (UID: \"f274168d-df51-4f50-b014-1ea4e85370d7\") " pod="openstack-kuttl-tests/novacell03d99-account-delete-5gvmz" Dec 03 11:42:27 crc kubenswrapper[5052]: I1203 11:42:27.744828 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-ws6lz\" (UniqueName: \"kubernetes.io/projected/fc5f71c9-f943-4ba6-a8eb-8e16ed558675-kube-api-access-ws6lz\") pod \"dnsmasq-dnsmasq-84b9f45d47-54w7t\" (UID: \"fc5f71c9-f943-4ba6-a8eb-8e16ed558675\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-54w7t" Dec 03 11:42:27 crc kubenswrapper[5052]: I1203 11:42:27.744860 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/fc5f71c9-f943-4ba6-a8eb-8e16ed558675-config\") pod \"dnsmasq-dnsmasq-84b9f45d47-54w7t\" (UID: \"fc5f71c9-f943-4ba6-a8eb-8e16ed558675\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-54w7t" Dec 03 11:42:27 crc kubenswrapper[5052]: I1203 11:42:27.744879 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/fc5f71c9-f943-4ba6-a8eb-8e16ed558675-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-84b9f45d47-54w7t\" (UID: \"fc5f71c9-f943-4ba6-a8eb-8e16ed558675\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-54w7t" Dec 03 11:42:27 crc kubenswrapper[5052]: I1203 11:42:27.747345 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/fc5f71c9-f943-4ba6-a8eb-8e16ed558675-config\") pod \"dnsmasq-dnsmasq-84b9f45d47-54w7t\" (UID: \"fc5f71c9-f943-4ba6-a8eb-8e16ed558675\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-54w7t" Dec 03 11:42:27 crc kubenswrapper[5052]: I1203 11:42:27.749335 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/fc5f71c9-f943-4ba6-a8eb-8e16ed558675-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-84b9f45d47-54w7t\" (UID: \"fc5f71c9-f943-4ba6-a8eb-8e16ed558675\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-54w7t" Dec 03 11:42:27 crc kubenswrapper[5052]: E1203 11:42:27.752038 5052 projected.go:194] Error preparing data for projected volume kube-api-access-zmn62 for pod openstack-kuttl-tests/novacell03d99-account-delete-5gvmz: failed to fetch token: serviceaccounts "galera-openstack" not found Dec 03 11:42:27 crc kubenswrapper[5052]: E1203 11:42:27.752093 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f274168d-df51-4f50-b014-1ea4e85370d7-kube-api-access-zmn62 podName:f274168d-df51-4f50-b014-1ea4e85370d7 nodeName:}" failed. No retries permitted until 2025-12-03 11:42:28.752074096 +0000 UTC m=+1339.340315860 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-zmn62" (UniqueName: "kubernetes.io/projected/f274168d-df51-4f50-b014-1ea4e85370d7-kube-api-access-zmn62") pod "novacell03d99-account-delete-5gvmz" (UID: "f274168d-df51-4f50-b014-1ea4e85370d7") : failed to fetch token: serviceaccounts "galera-openstack" not found Dec 03 11:42:27 crc kubenswrapper[5052]: I1203 11:42:27.786285 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-ws6lz\" (UniqueName: \"kubernetes.io/projected/fc5f71c9-f943-4ba6-a8eb-8e16ed558675-kube-api-access-ws6lz\") pod \"dnsmasq-dnsmasq-84b9f45d47-54w7t\" (UID: \"fc5f71c9-f943-4ba6-a8eb-8e16ed558675\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-54w7t" Dec 03 11:42:27 crc kubenswrapper[5052]: I1203 11:42:27.804197 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="063c5d2e-68a5-497a-b7bd-f95d45804848" path="/var/lib/kubelet/pods/063c5d2e-68a5-497a-b7bd-f95d45804848/volumes" Dec 03 11:42:27 crc kubenswrapper[5052]: I1203 11:42:27.804954 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1dc19382-24a8-4a4f-b735-d794619492bf" path="/var/lib/kubelet/pods/1dc19382-24a8-4a4f-b735-d794619492bf/volumes" Dec 03 11:42:27 crc kubenswrapper[5052]: I1203 11:42:27.805618 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5ff261a9-6d56-45fe-88fa-7184d8a3b90b" path="/var/lib/kubelet/pods/5ff261a9-6d56-45fe-88fa-7184d8a3b90b/volumes" Dec 03 11:42:27 crc kubenswrapper[5052]: I1203 11:42:27.813576 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8934c593-c918-48c9-b19b-e7c0e47474c2" path="/var/lib/kubelet/pods/8934c593-c918-48c9-b19b-e7c0e47474c2/volumes" Dec 03 11:42:27 crc kubenswrapper[5052]: I1203 11:42:27.814137 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c0af0947-fd26-43a5-8b64-bb9bae861cf2" path="/var/lib/kubelet/pods/c0af0947-fd26-43a5-8b64-bb9bae861cf2/volumes" Dec 03 11:42:27 crc kubenswrapper[5052]: I1203 11:42:27.814748 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d7bd0c57-728c-46a4-bbec-2e209d9f4049" path="/var/lib/kubelet/pods/d7bd0c57-728c-46a4-bbec-2e209d9f4049/volumes" Dec 03 11:42:27 crc kubenswrapper[5052]: I1203 11:42:27.831670 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ff7c4fed-d9f0-4220-9c04-33fae89bacd2" path="/var/lib/kubelet/pods/ff7c4fed-d9f0-4220-9c04-33fae89bacd2/volumes" Dec 03 11:42:27 crc kubenswrapper[5052]: I1203 11:42:27.845348 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/neutron-69cf8cb7d6-jfb57"] Dec 03 11:42:27 crc kubenswrapper[5052]: I1203 11:42:27.845377 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-keystone-listener-5dd6f4bfd-glsv7"] Dec 03 11:42:27 crc kubenswrapper[5052]: I1203 11:42:27.845390 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-api-df67d6cb4-8sllj"] Dec 03 11:42:27 crc kubenswrapper[5052]: I1203 11:42:27.845402 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-keystone-listener-649457694-r7gzw"] Dec 03 11:42:27 crc kubenswrapper[5052]: I1203 11:42:27.845413 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-worker-79cc7656c5-896b7"] Dec 03 11:42:27 crc kubenswrapper[5052]: I1203 11:42:27.845635 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/barbican-worker-79cc7656c5-896b7" podUID="32190156-44ee-4a16-a86c-2584f6a94866" containerName="barbican-worker-log" containerID="cri-o://6e34f31b2be164f36807f4119b800ae1a884c550a462f7a9c586c37f82893863" gracePeriod=30 Dec 03 11:42:27 crc kubenswrapper[5052]: I1203 11:42:27.866987 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/barbican-worker-79cc7656c5-896b7" podUID="32190156-44ee-4a16-a86c-2584f6a94866" containerName="barbican-worker" containerID="cri-o://63e884f6db70c59723e3f8be480e8d365efc639a9133cf1e01bf9eb55912f224" gracePeriod=30 Dec 03 11:42:27 crc kubenswrapper[5052]: I1203 11:42:27.877040 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/barbican-keystone-listener-649457694-r7gzw" podUID="8c4732f9-0f9e-4600-aed9-6a230cbf7b16" containerName="barbican-keystone-listener-log" containerID="cri-o://61b746fb82156eb75de6a2356739407db21615dda656157379f9510b62f96730" gracePeriod=30 Dec 03 11:42:27 crc kubenswrapper[5052]: I1203 11:42:27.877080 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/barbican-keystone-listener-649457694-r7gzw" podUID="8c4732f9-0f9e-4600-aed9-6a230cbf7b16" containerName="barbican-keystone-listener" containerID="cri-o://cc72d236c40c2e7f87295c5f31e63f10a061fd78a90f6c26b95db3911d675ff7" gracePeriod=30 Dec 03 11:42:27 crc kubenswrapper[5052]: E1203 11:42:27.880250 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="2acdb0814bfda1b6c48a4fde354fd532f86ae71dcaac188c16bee44a794b3713" cmd=["/usr/bin/pgrep","-r","DRST","nova-scheduler"] Dec 03 11:42:27 crc kubenswrapper[5052]: E1203 11:42:27.914089 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="2acdb0814bfda1b6c48a4fde354fd532f86ae71dcaac188c16bee44a794b3713" cmd=["/usr/bin/pgrep","-r","DRST","nova-scheduler"] Dec 03 11:42:27 crc kubenswrapper[5052]: I1203 11:42:27.914535 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-marketplace/redhat-operators-bvp4v" Dec 03 11:42:27 crc kubenswrapper[5052]: E1203 11:42:27.931013 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="2acdb0814bfda1b6c48a4fde354fd532f86ae71dcaac188c16bee44a794b3713" cmd=["/usr/bin/pgrep","-r","DRST","nova-scheduler"] Dec 03 11:42:27 crc kubenswrapper[5052]: E1203 11:42:27.931059 5052 prober.go:104] "Probe errored" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" probeType="Readiness" pod="openstack-kuttl-tests/nova-scheduler-0" podUID="d80d1476-351a-40c6-9f76-ce66d623df74" containerName="nova-scheduler-scheduler" Dec 03 11:42:27 crc kubenswrapper[5052]: I1203 11:42:27.936882 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-worker-67c775d6bc-kwxzt"] Dec 03 11:42:27 crc kubenswrapper[5052]: I1203 11:42:27.981482 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-api-df67d6cb4-8sllj"] Dec 03 11:42:27 crc kubenswrapper[5052]: I1203 11:42:27.992040 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-6858c48994-zq757"] Dec 03 11:42:28 crc kubenswrapper[5052]: I1203 11:42:28.011609 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/kube-state-metrics-0"] Dec 03 11:42:28 crc kubenswrapper[5052]: I1203 11:42:28.011882 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/kube-state-metrics-0" podUID="8b465e38-7dbb-4d02-bce0-28ae0145520c" containerName="kube-state-metrics" containerID="cri-o://cb0825ceeb51bae53da489a08c6c35ac9d9db71eece046983569823ad7e6cd20" gracePeriod=30 Dec 03 11:42:28 crc kubenswrapper[5052]: I1203 11:42:28.030597 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/rabbitmq-cell1-server-0"] Dec 03 11:42:28 crc kubenswrapper[5052]: I1203 11:42:28.037326 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell1-db-create-4mc7v"] Dec 03 11:42:28 crc kubenswrapper[5052]: I1203 11:42:28.052691 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell1-db-create-4mc7v"] Dec 03 11:42:28 crc kubenswrapper[5052]: I1203 11:42:28.064407 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell1-cdd4-account-create-update-lbpqk"] Dec 03 11:42:28 crc kubenswrapper[5052]: I1203 11:42:28.088492 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell1-cdd4-account-create-update-lbpqk"] Dec 03 11:42:28 crc kubenswrapper[5052]: I1203 11:42:28.091739 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" podUID="daa521fc-7e1d-44d2-b7d8-6aab003cfb66" containerName="rabbitmq" containerID="cri-o://f084100fa637b545df332be6f942cc3ba527b4e7f8898dd5c51db5555433c0fd" gracePeriod=604800 Dec 03 11:42:28 crc kubenswrapper[5052]: I1203 11:42:28.106755 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-db-sync-cw46s"] Dec 03 11:42:28 crc kubenswrapper[5052]: I1203 11:42:28.120650 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-db-sync-cw46s"] Dec 03 11:42:28 crc kubenswrapper[5052]: I1203 11:42:28.132381 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-db-sync-t7mx5"] Dec 03 11:42:28 crc kubenswrapper[5052]: I1203 11:42:28.139169 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-db-sync-t7mx5"] Dec 03 11:42:28 crc kubenswrapper[5052]: I1203 11:42:28.152583 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/keystonec8af-account-delete-xhhcr"] Dec 03 11:42:28 crc kubenswrapper[5052]: I1203 11:42:28.154366 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystonec8af-account-delete-xhhcr" Dec 03 11:42:28 crc kubenswrapper[5052]: I1203 11:42:28.159107 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystonec8af-account-delete-xhhcr"] Dec 03 11:42:28 crc kubenswrapper[5052]: I1203 11:42:28.167699 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/keystone-bootstrap-v8g6k"] Dec 03 11:42:28 crc kubenswrapper[5052]: I1203 11:42:28.187998 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/keystone-bootstrap-v8g6k"] Dec 03 11:42:28 crc kubenswrapper[5052]: E1203 11:42:28.197237 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/barbican-worker-config-data: secret "barbican-worker-config-data" not found Dec 03 11:42:28 crc kubenswrapper[5052]: E1203 11:42:28.197312 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/222f033c-3660-4347-b621-7122c4c94929-config-data-custom podName:222f033c-3660-4347-b621-7122c4c94929 nodeName:}" failed. No retries permitted until 2025-12-03 11:42:29.197293627 +0000 UTC m=+1339.785535391 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "config-data-custom" (UniqueName: "kubernetes.io/secret/222f033c-3660-4347-b621-7122c4c94929-config-data-custom") pod "barbican-worker-67c775d6bc-kwxzt" (UID: "222f033c-3660-4347-b621-7122c4c94929") : secret "barbican-worker-config-data" not found Dec 03 11:42:28 crc kubenswrapper[5052]: E1203 11:42:28.198058 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/combined-ca-bundle: secret "combined-ca-bundle" not found Dec 03 11:42:28 crc kubenswrapper[5052]: E1203 11:42:28.198202 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/222f033c-3660-4347-b621-7122c4c94929-combined-ca-bundle podName:222f033c-3660-4347-b621-7122c4c94929 nodeName:}" failed. No retries permitted until 2025-12-03 11:42:29.198151182 +0000 UTC m=+1339.786393016 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "combined-ca-bundle" (UniqueName: "kubernetes.io/secret/222f033c-3660-4347-b621-7122c4c94929-combined-ca-bundle") pod "barbican-worker-67c775d6bc-kwxzt" (UID: "222f033c-3660-4347-b621-7122c4c94929") : secret "combined-ca-bundle" not found Dec 03 11:42:28 crc kubenswrapper[5052]: I1203 11:42:28.198317 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/keystone-db-sync-6s975"] Dec 03 11:42:28 crc kubenswrapper[5052]: E1203 11:42:28.198964 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/barbican-config-data: secret "barbican-config-data" not found Dec 03 11:42:28 crc kubenswrapper[5052]: E1203 11:42:28.199065 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/222f033c-3660-4347-b621-7122c4c94929-config-data podName:222f033c-3660-4347-b621-7122c4c94929 nodeName:}" failed. No retries permitted until 2025-12-03 11:42:29.199049378 +0000 UTC m=+1339.787291222 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "config-data" (UniqueName: "kubernetes.io/secret/222f033c-3660-4347-b621-7122c4c94929-config-data") pod "barbican-worker-67c775d6bc-kwxzt" (UID: "222f033c-3660-4347-b621-7122c4c94929") : secret "barbican-config-data" not found Dec 03 11:42:28 crc kubenswrapper[5052]: I1203 11:42:28.212345 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder5365-account-delete-shtcv"] Dec 03 11:42:28 crc kubenswrapper[5052]: I1203 11:42:28.221666 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/keystone-db-sync-6s975"] Dec 03 11:42:28 crc kubenswrapper[5052]: I1203 11:42:28.241265 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/keystone-6858c48994-zq757"] Dec 03 11:42:28 crc kubenswrapper[5052]: I1203 11:42:28.255958 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/keystone-d6bbfcfb4-xdw2x"] Dec 03 11:42:28 crc kubenswrapper[5052]: I1203 11:42:28.256268 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/keystone-d6bbfcfb4-xdw2x" podUID="5737462d-2e5c-4453-9d6c-7d67503555f7" containerName="keystone-api" containerID="cri-o://85486efe955265533bc0eee17583150138fcae470d553a61d68bb5477eb454dd" gracePeriod=30 Dec 03 11:42:28 crc kubenswrapper[5052]: I1203 11:42:28.300931 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/1fc99a78-bfd8-4fc9-9ff7-bdd5adeaff73-operator-scripts\") pod \"keystonec8af-account-delete-xhhcr\" (UID: \"1fc99a78-bfd8-4fc9-9ff7-bdd5adeaff73\") " pod="openstack-kuttl-tests/keystonec8af-account-delete-xhhcr" Dec 03 11:42:28 crc kubenswrapper[5052]: I1203 11:42:28.301409 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b6hww\" (UniqueName: \"kubernetes.io/projected/1fc99a78-bfd8-4fc9-9ff7-bdd5adeaff73-kube-api-access-b6hww\") pod \"keystonec8af-account-delete-xhhcr\" (UID: \"1fc99a78-bfd8-4fc9-9ff7-bdd5adeaff73\") " pod="openstack-kuttl-tests/keystonec8af-account-delete-xhhcr" Dec 03 11:42:28 crc kubenswrapper[5052]: E1203 11:42:28.301575 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/rabbitmq-config-data: configmap "rabbitmq-config-data" not found Dec 03 11:42:28 crc kubenswrapper[5052]: E1203 11:42:28.301626 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9944d9af-7a7b-4773-8279-72914d63ef5f-config-data podName:9944d9af-7a7b-4773-8279-72914d63ef5f nodeName:}" failed. No retries permitted until 2025-12-03 11:42:30.301609532 +0000 UTC m=+1340.889851296 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "config-data" (UniqueName: "kubernetes.io/configmap/9944d9af-7a7b-4773-8279-72914d63ef5f-config-data") pod "rabbitmq-server-0" (UID: "9944d9af-7a7b-4773-8279-72914d63ef5f") : configmap "rabbitmq-config-data" not found Dec 03 11:42:28 crc kubenswrapper[5052]: I1203 11:42:28.356046 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/barbican-worker-67c775d6bc-kwxzt" podStartSLOduration=6.356028195 podStartE2EDuration="6.356028195s" podCreationTimestamp="2025-12-03 11:42:22 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:42:27.342182511 +0000 UTC m=+1337.930424275" watchObservedRunningTime="2025-12-03 11:42:28.356028195 +0000 UTC m=+1338.944269959" Dec 03 11:42:28 crc kubenswrapper[5052]: I1203 11:42:28.371294 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/rabbitmq-server-0"] Dec 03 11:42:28 crc kubenswrapper[5052]: I1203 11:42:28.410582 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-b6hww\" (UniqueName: \"kubernetes.io/projected/1fc99a78-bfd8-4fc9-9ff7-bdd5adeaff73-kube-api-access-b6hww\") pod \"keystonec8af-account-delete-xhhcr\" (UID: \"1fc99a78-bfd8-4fc9-9ff7-bdd5adeaff73\") " pod="openstack-kuttl-tests/keystonec8af-account-delete-xhhcr" Dec 03 11:42:28 crc kubenswrapper[5052]: I1203 11:42:28.410846 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/1fc99a78-bfd8-4fc9-9ff7-bdd5adeaff73-operator-scripts\") pod \"keystonec8af-account-delete-xhhcr\" (UID: \"1fc99a78-bfd8-4fc9-9ff7-bdd5adeaff73\") " pod="openstack-kuttl-tests/keystonec8af-account-delete-xhhcr" Dec 03 11:42:28 crc kubenswrapper[5052]: E1203 11:42:28.411284 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 11:42:28 crc kubenswrapper[5052]: E1203 11:42:28.411372 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1fc99a78-bfd8-4fc9-9ff7-bdd5adeaff73-operator-scripts podName:1fc99a78-bfd8-4fc9-9ff7-bdd5adeaff73 nodeName:}" failed. No retries permitted until 2025-12-03 11:42:28.911355365 +0000 UTC m=+1339.499597129 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/1fc99a78-bfd8-4fc9-9ff7-bdd5adeaff73-operator-scripts") pod "keystonec8af-account-delete-xhhcr" (UID: "1fc99a78-bfd8-4fc9-9ff7-bdd5adeaff73") : configmap "openstack-scripts" not found Dec 03 11:42:28 crc kubenswrapper[5052]: E1203 11:42:28.437920 5052 projected.go:194] Error preparing data for projected volume kube-api-access-b6hww for pod openstack-kuttl-tests/keystonec8af-account-delete-xhhcr: failed to fetch token: serviceaccounts "galera-openstack" not found Dec 03 11:42:28 crc kubenswrapper[5052]: E1203 11:42:28.438011 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1fc99a78-bfd8-4fc9-9ff7-bdd5adeaff73-kube-api-access-b6hww podName:1fc99a78-bfd8-4fc9-9ff7-bdd5adeaff73 nodeName:}" failed. No retries permitted until 2025-12-03 11:42:28.93798771 +0000 UTC m=+1339.526229484 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-b6hww" (UniqueName: "kubernetes.io/projected/1fc99a78-bfd8-4fc9-9ff7-bdd5adeaff73-kube-api-access-b6hww") pod "keystonec8af-account-delete-xhhcr" (UID: "1fc99a78-bfd8-4fc9-9ff7-bdd5adeaff73") : failed to fetch token: serviceaccounts "galera-openstack" not found Dec 03 11:42:28 crc kubenswrapper[5052]: I1203 11:42:28.509810 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openshift-marketplace/redhat-operators-bvp4v"] Dec 03 11:42:28 crc kubenswrapper[5052]: I1203 11:42:28.539724 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/placement79ca-account-delete-sqtcb"] Dec 03 11:42:28 crc kubenswrapper[5052]: I1203 11:42:28.548903 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutronbfb7-account-delete-ff26t"] Dec 03 11:42:28 crc kubenswrapper[5052]: W1203 11:42:28.554056 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod32573220_587f_4703_a5e9_0fb67d2019ef.slice/crio-fc5b5266a2b318f1577aca38b5cd24531faf4b17a72d764f030738dce87191c8 WatchSource:0}: Error finding container fc5b5266a2b318f1577aca38b5cd24531faf4b17a72d764f030738dce87191c8: Status 404 returned error can't find the container with id fc5b5266a2b318f1577aca38b5cd24531faf4b17a72d764f030738dce87191c8 Dec 03 11:42:28 crc kubenswrapper[5052]: I1203 11:42:28.554600 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/rabbitmq-server-0" podUID="9944d9af-7a7b-4773-8279-72914d63ef5f" containerName="rabbitmq" containerID="cri-o://c7a780b226312d9ca6d3ee18cc8115287601a5ebf3182143b2ccad2a68a3f43a" gracePeriod=604800 Dec 03 11:42:28 crc kubenswrapper[5052]: I1203 11:42:28.608979 5052 generic.go:334] "Generic (PLEG): container finished" podID="ee46c3a4-5e21-4f6b-a944-8cf96d87506e" containerID="a5e3cd0978d13d8f638fcff947df03b89089415c5babbbe442713809453149ae" exitCode=0 Dec 03 11:42:28 crc kubenswrapper[5052]: I1203 11:42:28.609078 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"ee46c3a4-5e21-4f6b-a944-8cf96d87506e","Type":"ContainerDied","Data":"a5e3cd0978d13d8f638fcff947df03b89089415c5babbbe442713809453149ae"} Dec 03 11:42:28 crc kubenswrapper[5052]: I1203 11:42:28.628447 5052 scope.go:117] "RemoveContainer" containerID="352c7041b2ebcbf03c6b6dbad2b08bc20c546043c65916bf3b506556deb9182f" Dec 03 11:42:28 crc kubenswrapper[5052]: I1203 11:42:28.665831 5052 generic.go:334] "Generic (PLEG): container finished" podID="d19f0c21-bb49-40a1-87fa-e55188308729" containerID="198e5b3e40307e3ae065a4823c3cadc6edfe24c354d3dc8885bdeb1b65e4a57a" exitCode=0 Dec 03 11:42:28 crc kubenswrapper[5052]: I1203 11:42:28.665865 5052 generic.go:334] "Generic (PLEG): container finished" podID="d19f0c21-bb49-40a1-87fa-e55188308729" containerID="1feeef96663c3d2882692e8837c421c0dab4a5a5be167acf6810b788e07033be" exitCode=0 Dec 03 11:42:28 crc kubenswrapper[5052]: I1203 11:42:28.665877 5052 generic.go:334] "Generic (PLEG): container finished" podID="d19f0c21-bb49-40a1-87fa-e55188308729" containerID="e0f3046cd638bd8d68fc0858b30e3c71c02254e1579ac5fdf0d14362c56787ae" exitCode=0 Dec 03 11:42:28 crc kubenswrapper[5052]: I1203 11:42:28.665886 5052 generic.go:334] "Generic (PLEG): container finished" podID="d19f0c21-bb49-40a1-87fa-e55188308729" containerID="842157c852d9af69e655dafb6e96753c92d5a174fac182e141f97a78bcaf8577" exitCode=0 Dec 03 11:42:28 crc kubenswrapper[5052]: I1203 11:42:28.665895 5052 generic.go:334] "Generic (PLEG): container finished" podID="d19f0c21-bb49-40a1-87fa-e55188308729" containerID="480a4695021f8caec2d8b5d9b753bac979ecb6a2f7643abdd8ce149daf4a8b4a" exitCode=0 Dec 03 11:42:28 crc kubenswrapper[5052]: I1203 11:42:28.665904 5052 generic.go:334] "Generic (PLEG): container finished" podID="d19f0c21-bb49-40a1-87fa-e55188308729" containerID="6f7206455d66da31b1ca0c6033a701e9ef605c548a9309c2b0967a2b9726a603" exitCode=0 Dec 03 11:42:28 crc kubenswrapper[5052]: I1203 11:42:28.665913 5052 generic.go:334] "Generic (PLEG): container finished" podID="d19f0c21-bb49-40a1-87fa-e55188308729" containerID="e36b32b951337695419521a1f3271a1e8e2cac46f10174c506c5e0e369ad8704" exitCode=0 Dec 03 11:42:28 crc kubenswrapper[5052]: I1203 11:42:28.665922 5052 generic.go:334] "Generic (PLEG): container finished" podID="d19f0c21-bb49-40a1-87fa-e55188308729" containerID="66b82b9a9602e546754f3199323f445795063d7d84d38d42c9e7bdead8e2d119" exitCode=0 Dec 03 11:42:28 crc kubenswrapper[5052]: I1203 11:42:28.665930 5052 generic.go:334] "Generic (PLEG): container finished" podID="d19f0c21-bb49-40a1-87fa-e55188308729" containerID="8b82c3eafedcad816bdd8f848dfa51930b6fcd64c66cf454cbb3af2debd04b86" exitCode=0 Dec 03 11:42:28 crc kubenswrapper[5052]: I1203 11:42:28.665938 5052 generic.go:334] "Generic (PLEG): container finished" podID="d19f0c21-bb49-40a1-87fa-e55188308729" containerID="cc5f444de12548f88c1caa6a61bd1efc6e2fd9da4c672919931aac6ba15d3f15" exitCode=0 Dec 03 11:42:28 crc kubenswrapper[5052]: I1203 11:42:28.665948 5052 generic.go:334] "Generic (PLEG): container finished" podID="d19f0c21-bb49-40a1-87fa-e55188308729" containerID="38627e147a9f39cc2d052969a6aa30eeb14f6d31573e540afa5859023f9656d8" exitCode=0 Dec 03 11:42:28 crc kubenswrapper[5052]: I1203 11:42:28.666013 5052 generic.go:334] "Generic (PLEG): container finished" podID="d19f0c21-bb49-40a1-87fa-e55188308729" containerID="e776b8b08a7136d5453955d041fce850605188fd0e17c51f2024905c9aa86aec" exitCode=0 Dec 03 11:42:28 crc kubenswrapper[5052]: I1203 11:42:28.666024 5052 generic.go:334] "Generic (PLEG): container finished" podID="d19f0c21-bb49-40a1-87fa-e55188308729" containerID="375e74dc879bff82efc908e9893ad30ab3c979218f143e6a3e6fcd9936d34248" exitCode=0 Dec 03 11:42:28 crc kubenswrapper[5052]: I1203 11:42:28.666032 5052 generic.go:334] "Generic (PLEG): container finished" podID="d19f0c21-bb49-40a1-87fa-e55188308729" containerID="3a64f478f235a4fb6291efc13b9a7d651e52032e9126c5a8ee70cbaa6f7bc0c4" exitCode=0 Dec 03 11:42:28 crc kubenswrapper[5052]: I1203 11:42:28.666105 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"d19f0c21-bb49-40a1-87fa-e55188308729","Type":"ContainerDied","Data":"198e5b3e40307e3ae065a4823c3cadc6edfe24c354d3dc8885bdeb1b65e4a57a"} Dec 03 11:42:28 crc kubenswrapper[5052]: I1203 11:42:28.666140 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"d19f0c21-bb49-40a1-87fa-e55188308729","Type":"ContainerDied","Data":"1feeef96663c3d2882692e8837c421c0dab4a5a5be167acf6810b788e07033be"} Dec 03 11:42:28 crc kubenswrapper[5052]: I1203 11:42:28.666178 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"d19f0c21-bb49-40a1-87fa-e55188308729","Type":"ContainerDied","Data":"e0f3046cd638bd8d68fc0858b30e3c71c02254e1579ac5fdf0d14362c56787ae"} Dec 03 11:42:28 crc kubenswrapper[5052]: I1203 11:42:28.666191 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"d19f0c21-bb49-40a1-87fa-e55188308729","Type":"ContainerDied","Data":"842157c852d9af69e655dafb6e96753c92d5a174fac182e141f97a78bcaf8577"} Dec 03 11:42:28 crc kubenswrapper[5052]: I1203 11:42:28.666202 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"d19f0c21-bb49-40a1-87fa-e55188308729","Type":"ContainerDied","Data":"480a4695021f8caec2d8b5d9b753bac979ecb6a2f7643abdd8ce149daf4a8b4a"} Dec 03 11:42:28 crc kubenswrapper[5052]: I1203 11:42:28.666213 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"d19f0c21-bb49-40a1-87fa-e55188308729","Type":"ContainerDied","Data":"6f7206455d66da31b1ca0c6033a701e9ef605c548a9309c2b0967a2b9726a603"} Dec 03 11:42:28 crc kubenswrapper[5052]: I1203 11:42:28.666225 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"d19f0c21-bb49-40a1-87fa-e55188308729","Type":"ContainerDied","Data":"e36b32b951337695419521a1f3271a1e8e2cac46f10174c506c5e0e369ad8704"} Dec 03 11:42:28 crc kubenswrapper[5052]: I1203 11:42:28.666261 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"d19f0c21-bb49-40a1-87fa-e55188308729","Type":"ContainerDied","Data":"66b82b9a9602e546754f3199323f445795063d7d84d38d42c9e7bdead8e2d119"} Dec 03 11:42:28 crc kubenswrapper[5052]: I1203 11:42:28.666272 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"d19f0c21-bb49-40a1-87fa-e55188308729","Type":"ContainerDied","Data":"8b82c3eafedcad816bdd8f848dfa51930b6fcd64c66cf454cbb3af2debd04b86"} Dec 03 11:42:28 crc kubenswrapper[5052]: I1203 11:42:28.666284 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"d19f0c21-bb49-40a1-87fa-e55188308729","Type":"ContainerDied","Data":"cc5f444de12548f88c1caa6a61bd1efc6e2fd9da4c672919931aac6ba15d3f15"} Dec 03 11:42:28 crc kubenswrapper[5052]: I1203 11:42:28.666296 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"d19f0c21-bb49-40a1-87fa-e55188308729","Type":"ContainerDied","Data":"38627e147a9f39cc2d052969a6aa30eeb14f6d31573e540afa5859023f9656d8"} Dec 03 11:42:28 crc kubenswrapper[5052]: I1203 11:42:28.666329 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"d19f0c21-bb49-40a1-87fa-e55188308729","Type":"ContainerDied","Data":"e776b8b08a7136d5453955d041fce850605188fd0e17c51f2024905c9aa86aec"} Dec 03 11:42:28 crc kubenswrapper[5052]: I1203 11:42:28.666345 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"d19f0c21-bb49-40a1-87fa-e55188308729","Type":"ContainerDied","Data":"375e74dc879bff82efc908e9893ad30ab3c979218f143e6a3e6fcd9936d34248"} Dec 03 11:42:28 crc kubenswrapper[5052]: I1203 11:42:28.666357 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"d19f0c21-bb49-40a1-87fa-e55188308729","Type":"ContainerDied","Data":"3a64f478f235a4fb6291efc13b9a7d651e52032e9126c5a8ee70cbaa6f7bc0c4"} Dec 03 11:42:28 crc kubenswrapper[5052]: I1203 11:42:28.670927 5052 generic.go:334] "Generic (PLEG): container finished" podID="32190156-44ee-4a16-a86c-2584f6a94866" containerID="6e34f31b2be164f36807f4119b800ae1a884c550a462f7a9c586c37f82893863" exitCode=143 Dec 03 11:42:28 crc kubenswrapper[5052]: I1203 11:42:28.671022 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-worker-79cc7656c5-896b7" event={"ID":"32190156-44ee-4a16-a86c-2584f6a94866","Type":"ContainerDied","Data":"6e34f31b2be164f36807f4119b800ae1a884c550a462f7a9c586c37f82893863"} Dec 03 11:42:28 crc kubenswrapper[5052]: I1203 11:42:28.675080 5052 generic.go:334] "Generic (PLEG): container finished" podID="f8cdf05e-7081-4542-844d-f76378e8989f" containerID="9cfabfd91ca0525b54d3922c76017cb0defbc772bca498498fc287ac2175e2aa" exitCode=0 Dec 03 11:42:28 crc kubenswrapper[5052]: I1203 11:42:28.675155 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-conductor-0" event={"ID":"f8cdf05e-7081-4542-844d-f76378e8989f","Type":"ContainerDied","Data":"9cfabfd91ca0525b54d3922c76017cb0defbc772bca498498fc287ac2175e2aa"} Dec 03 11:42:28 crc kubenswrapper[5052]: I1203 11:42:28.704948 5052 generic.go:334] "Generic (PLEG): container finished" podID="fd14a5c0-3402-4891-a726-32d75b1b7857" containerID="def28b7790031f83be9c79b54dbbc74332f5e5917d604dfb281ac32589501053" exitCode=0 Dec 03 11:42:28 crc kubenswrapper[5052]: I1203 11:42:28.704982 5052 generic.go:334] "Generic (PLEG): container finished" podID="fd14a5c0-3402-4891-a726-32d75b1b7857" containerID="3132bd07375ba7c18c7bbb9921d64b3f404ab0c85abce7e27085e92e0ba00bea" exitCode=0 Dec 03 11:42:28 crc kubenswrapper[5052]: I1203 11:42:28.705045 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-proxy-fd9678558-qx8wh" event={"ID":"fd14a5c0-3402-4891-a726-32d75b1b7857","Type":"ContainerDied","Data":"def28b7790031f83be9c79b54dbbc74332f5e5917d604dfb281ac32589501053"} Dec 03 11:42:28 crc kubenswrapper[5052]: I1203 11:42:28.705078 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-proxy-fd9678558-qx8wh" event={"ID":"fd14a5c0-3402-4891-a726-32d75b1b7857","Type":"ContainerDied","Data":"3132bd07375ba7c18c7bbb9921d64b3f404ab0c85abce7e27085e92e0ba00bea"} Dec 03 11:42:28 crc kubenswrapper[5052]: I1203 11:42:28.716594 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"f98f227a-397b-4665-bf18-e00b105ce058","Type":"ContainerDied","Data":"114a172183480c19154e1e08e67868f0385b850efd333a1ba31774310890d146"} Dec 03 11:42:28 crc kubenswrapper[5052]: I1203 11:42:28.716633 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="114a172183480c19154e1e08e67868f0385b850efd333a1ba31774310890d146" Dec 03 11:42:28 crc kubenswrapper[5052]: I1203 11:42:28.736165 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/f274168d-df51-4f50-b014-1ea4e85370d7-operator-scripts\") pod \"novacell03d99-account-delete-5gvmz\" (UID: \"f274168d-df51-4f50-b014-1ea4e85370d7\") " pod="openstack-kuttl-tests/novacell03d99-account-delete-5gvmz" Dec 03 11:42:28 crc kubenswrapper[5052]: E1203 11:42:28.736265 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 11:42:28 crc kubenswrapper[5052]: E1203 11:42:28.736312 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f274168d-df51-4f50-b014-1ea4e85370d7-operator-scripts podName:f274168d-df51-4f50-b014-1ea4e85370d7 nodeName:}" failed. No retries permitted until 2025-12-03 11:42:30.736296188 +0000 UTC m=+1341.324537952 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/f274168d-df51-4f50-b014-1ea4e85370d7-operator-scripts") pod "novacell03d99-account-delete-5gvmz" (UID: "f274168d-df51-4f50-b014-1ea4e85370d7") : configmap "openstack-scripts" not found Dec 03 11:42:28 crc kubenswrapper[5052]: E1203 11:42:28.736795 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/rabbitmq-cell1-config-data: configmap "rabbitmq-cell1-config-data" not found Dec 03 11:42:28 crc kubenswrapper[5052]: E1203 11:42:28.736827 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/daa521fc-7e1d-44d2-b7d8-6aab003cfb66-config-data podName:daa521fc-7e1d-44d2-b7d8-6aab003cfb66 nodeName:}" failed. No retries permitted until 2025-12-03 11:42:32.736815444 +0000 UTC m=+1343.325057208 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "config-data" (UniqueName: "kubernetes.io/configmap/daa521fc-7e1d-44d2-b7d8-6aab003cfb66-config-data") pod "rabbitmq-cell1-server-0" (UID: "daa521fc-7e1d-44d2-b7d8-6aab003cfb66") : configmap "rabbitmq-cell1-config-data" not found Dec 03 11:42:28 crc kubenswrapper[5052]: I1203 11:42:28.741923 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-conductor-0" event={"ID":"2501eda8-a78d-4609-a465-761e89bb291e","Type":"ContainerDied","Data":"3f31cf67801442b58d6756229b602b05f44a185ad727f20aa451a39e5424be8b"} Dec 03 11:42:28 crc kubenswrapper[5052]: I1203 11:42:28.741967 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3f31cf67801442b58d6756229b602b05f44a185ad727f20aa451a39e5424be8b" Dec 03 11:42:28 crc kubenswrapper[5052]: I1203 11:42:28.749970 5052 generic.go:334] "Generic (PLEG): container finished" podID="fe69e54e-ae29-49e2-8a21-0ff81cc0100f" containerID="c4c73ffbd6756af9b104cfcef9220c7341172a6c5dadc45edcc42921bbd74050" exitCode=143 Dec 03 11:42:28 crc kubenswrapper[5052]: I1203 11:42:28.750050 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-66f4bb9676-j6vzc" event={"ID":"fe69e54e-ae29-49e2-8a21-0ff81cc0100f","Type":"ContainerDied","Data":"c4c73ffbd6756af9b104cfcef9220c7341172a6c5dadc45edcc42921bbd74050"} Dec 03 11:42:28 crc kubenswrapper[5052]: I1203 11:42:28.752824 5052 generic.go:334] "Generic (PLEG): container finished" podID="75e4b34e-f7b8-4209-95a6-6cc45f6eb57c" containerID="2b213199dec26418337b9d73443612a12a1ce69360675968db75a8f6aba268ae" exitCode=0 Dec 03 11:42:28 crc kubenswrapper[5052]: I1203 11:42:28.752978 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"75e4b34e-f7b8-4209-95a6-6cc45f6eb57c","Type":"ContainerDied","Data":"2b213199dec26418337b9d73443612a12a1ce69360675968db75a8f6aba268ae"} Dec 03 11:42:28 crc kubenswrapper[5052]: I1203 11:42:28.755496 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"665a736c-d550-4160-a8bb-619d1d5ea826","Type":"ContainerDied","Data":"3fc2133f2600b0034eb40472da075cfa1df1c9267bd498cfce382d79666ef928"} Dec 03 11:42:28 crc kubenswrapper[5052]: I1203 11:42:28.755691 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3fc2133f2600b0034eb40472da075cfa1df1c9267bd498cfce382d79666ef928" Dec 03 11:42:28 crc kubenswrapper[5052]: I1203 11:42:28.759311 5052 generic.go:334] "Generic (PLEG): container finished" podID="82dce330-8f65-4c14-80d4-958eb8ca5212" containerID="d6cb51da86b5f965cccc6e7f9d1241d74c446a9276dbf42e057cec9147384cb3" exitCode=0 Dec 03 11:42:28 crc kubenswrapper[5052]: I1203 11:42:28.759967 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"82dce330-8f65-4c14-80d4-958eb8ca5212","Type":"ContainerDied","Data":"d6cb51da86b5f965cccc6e7f9d1241d74c446a9276dbf42e057cec9147384cb3"} Dec 03 11:42:28 crc kubenswrapper[5052]: I1203 11:42:28.761961 5052 generic.go:334] "Generic (PLEG): container finished" podID="d80d1476-351a-40c6-9f76-ce66d623df74" containerID="2acdb0814bfda1b6c48a4fde354fd532f86ae71dcaac188c16bee44a794b3713" exitCode=0 Dec 03 11:42:28 crc kubenswrapper[5052]: I1203 11:42:28.762091 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"d80d1476-351a-40c6-9f76-ce66d623df74","Type":"ContainerDied","Data":"2acdb0814bfda1b6c48a4fde354fd532f86ae71dcaac188c16bee44a794b3713"} Dec 03 11:42:28 crc kubenswrapper[5052]: I1203 11:42:28.763541 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder5365-account-delete-shtcv" event={"ID":"d3e34248-e7e5-4158-8add-165d17bee9ab","Type":"ContainerStarted","Data":"97ece835d9a4c804e6f6b9d476829694aa6078bfb88602c735413883ae97d229"} Dec 03 11:42:28 crc kubenswrapper[5052]: E1203 11:42:28.767699 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of 9cfabfd91ca0525b54d3922c76017cb0defbc772bca498498fc287ac2175e2aa is running failed: container process not found" containerID="9cfabfd91ca0525b54d3922c76017cb0defbc772bca498498fc287ac2175e2aa" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 11:42:28 crc kubenswrapper[5052]: E1203 11:42:28.771654 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of 9cfabfd91ca0525b54d3922c76017cb0defbc772bca498498fc287ac2175e2aa is running failed: container process not found" containerID="9cfabfd91ca0525b54d3922c76017cb0defbc772bca498498fc287ac2175e2aa" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 11:42:28 crc kubenswrapper[5052]: E1203 11:42:28.775741 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of 9cfabfd91ca0525b54d3922c76017cb0defbc772bca498498fc287ac2175e2aa is running failed: container process not found" containerID="9cfabfd91ca0525b54d3922c76017cb0defbc772bca498498fc287ac2175e2aa" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 11:42:28 crc kubenswrapper[5052]: E1203 11:42:28.775801 5052 prober.go:104] "Probe errored" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of 9cfabfd91ca0525b54d3922c76017cb0defbc772bca498498fc287ac2175e2aa is running failed: container process not found" probeType="Readiness" pod="openstack-kuttl-tests/nova-cell0-conductor-0" podUID="f8cdf05e-7081-4542-844d-f76378e8989f" containerName="nova-cell0-conductor-conductor" Dec 03 11:42:28 crc kubenswrapper[5052]: I1203 11:42:28.777912 5052 generic.go:334] "Generic (PLEG): container finished" podID="8b465e38-7dbb-4d02-bce0-28ae0145520c" containerID="cb0825ceeb51bae53da489a08c6c35ac9d9db71eece046983569823ad7e6cd20" exitCode=2 Dec 03 11:42:28 crc kubenswrapper[5052]: I1203 11:42:28.778721 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/kube-state-metrics-0" event={"ID":"8b465e38-7dbb-4d02-bce0-28ae0145520c","Type":"ContainerDied","Data":"cb0825ceeb51bae53da489a08c6c35ac9d9db71eece046983569823ad7e6cd20"} Dec 03 11:42:28 crc kubenswrapper[5052]: I1203 11:42:28.806239 5052 generic.go:334] "Generic (PLEG): container finished" podID="0b4b31e5-e82d-4d44-9c8f-d001484293b6" containerID="2289f14f2059fc517e80d4db86044454e7b7fc71a8ff28682e6b78283502038f" exitCode=143 Dec 03 11:42:28 crc kubenswrapper[5052]: I1203 11:42:28.806349 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-699ddb8646-rfm4p" event={"ID":"0b4b31e5-e82d-4d44-9c8f-d001484293b6","Type":"ContainerDied","Data":"2289f14f2059fc517e80d4db86044454e7b7fc71a8ff28682e6b78283502038f"} Dec 03 11:42:28 crc kubenswrapper[5052]: I1203 11:42:28.816401 5052 generic.go:334] "Generic (PLEG): container finished" podID="14e9bff5-66d5-4808-8446-cf180d759208" containerID="b875595b2b281c94c97104d6ea786f06ea56deeebb4024873ed0f82dba8c7b52" exitCode=0 Dec 03 11:42:28 crc kubenswrapper[5052]: I1203 11:42:28.816467 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"14e9bff5-66d5-4808-8446-cf180d759208","Type":"ContainerDied","Data":"b875595b2b281c94c97104d6ea786f06ea56deeebb4024873ed0f82dba8c7b52"} Dec 03 11:42:28 crc kubenswrapper[5052]: I1203 11:42:28.819891 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-cell1-galera-0" event={"ID":"ecbf8b8e-cdb4-45c3-9455-d0dbc551356e","Type":"ContainerDied","Data":"89cc373939988478670e3838e2f18f4c330ddc76d766fa4fb981a0b443876b76"} Dec 03 11:42:28 crc kubenswrapper[5052]: I1203 11:42:28.819941 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="89cc373939988478670e3838e2f18f4c330ddc76d766fa4fb981a0b443876b76" Dec 03 11:42:28 crc kubenswrapper[5052]: I1203 11:42:28.837998 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-zmn62\" (UniqueName: \"kubernetes.io/projected/f274168d-df51-4f50-b014-1ea4e85370d7-kube-api-access-zmn62\") pod \"novacell03d99-account-delete-5gvmz\" (UID: \"f274168d-df51-4f50-b014-1ea4e85370d7\") " pod="openstack-kuttl-tests/novacell03d99-account-delete-5gvmz" Dec 03 11:42:28 crc kubenswrapper[5052]: E1203 11:42:28.844252 5052 projected.go:194] Error preparing data for projected volume kube-api-access-zmn62 for pod openstack-kuttl-tests/novacell03d99-account-delete-5gvmz: failed to fetch token: serviceaccounts "galera-openstack" not found Dec 03 11:42:28 crc kubenswrapper[5052]: E1203 11:42:28.844327 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f274168d-df51-4f50-b014-1ea4e85370d7-kube-api-access-zmn62 podName:f274168d-df51-4f50-b014-1ea4e85370d7 nodeName:}" failed. No retries permitted until 2025-12-03 11:42:30.844306211 +0000 UTC m=+1341.432548045 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-zmn62" (UniqueName: "kubernetes.io/projected/f274168d-df51-4f50-b014-1ea4e85370d7-kube-api-access-zmn62") pod "novacell03d99-account-delete-5gvmz" (UID: "f274168d-df51-4f50-b014-1ea4e85370d7") : failed to fetch token: serviceaccounts "galera-openstack" not found Dec 03 11:42:28 crc kubenswrapper[5052]: I1203 11:42:28.849876 5052 generic.go:334] "Generic (PLEG): container finished" podID="8c4732f9-0f9e-4600-aed9-6a230cbf7b16" containerID="61b746fb82156eb75de6a2356739407db21615dda656157379f9510b62f96730" exitCode=143 Dec 03 11:42:28 crc kubenswrapper[5052]: I1203 11:42:28.849975 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-keystone-listener-649457694-r7gzw" event={"ID":"8c4732f9-0f9e-4600-aed9-6a230cbf7b16","Type":"ContainerDied","Data":"61b746fb82156eb75de6a2356739407db21615dda656157379f9510b62f96730"} Dec 03 11:42:28 crc kubenswrapper[5052]: I1203 11:42:28.850697 5052 kubelet_pods.go:1007] "Unable to retrieve pull secret, the image pull may not succeed." pod="openstack-kuttl-tests/barbican-worker-67c775d6bc-kwxzt" secret="" err="secret \"barbican-barbican-dockercfg-bq7sh\" not found" Dec 03 11:42:28 crc kubenswrapper[5052]: I1203 11:42:28.865583 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-sb-0"] Dec 03 11:42:28 crc kubenswrapper[5052]: I1203 11:42:28.887577 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-sb-0"] Dec 03 11:42:28 crc kubenswrapper[5052]: I1203 11:42:28.918533 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/memcached-0"] Dec 03 11:42:28 crc kubenswrapper[5052]: I1203 11:42:28.924299 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstackclient" Dec 03 11:42:28 crc kubenswrapper[5052]: I1203 11:42:28.939314 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/memcached-0"] Dec 03 11:42:28 crc kubenswrapper[5052]: I1203 11:42:28.947812 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"openstack-config-secret\" (UniqueName: \"kubernetes.io/secret/722e5457-ae47-40b8-816f-8eab5b7a4cb0-openstack-config-secret\") pod \"722e5457-ae47-40b8-816f-8eab5b7a4cb0\" (UID: \"722e5457-ae47-40b8-816f-8eab5b7a4cb0\") " Dec 03 11:42:28 crc kubenswrapper[5052]: I1203 11:42:28.947959 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9xmxf\" (UniqueName: \"kubernetes.io/projected/722e5457-ae47-40b8-816f-8eab5b7a4cb0-kube-api-access-9xmxf\") pod \"722e5457-ae47-40b8-816f-8eab5b7a4cb0\" (UID: \"722e5457-ae47-40b8-816f-8eab5b7a4cb0\") " Dec 03 11:42:28 crc kubenswrapper[5052]: I1203 11:42:28.948071 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"openstack-config\" (UniqueName: \"kubernetes.io/configmap/722e5457-ae47-40b8-816f-8eab5b7a4cb0-openstack-config\") pod \"722e5457-ae47-40b8-816f-8eab5b7a4cb0\" (UID: \"722e5457-ae47-40b8-816f-8eab5b7a4cb0\") " Dec 03 11:42:28 crc kubenswrapper[5052]: I1203 11:42:28.948551 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/1fc99a78-bfd8-4fc9-9ff7-bdd5adeaff73-operator-scripts\") pod \"keystonec8af-account-delete-xhhcr\" (UID: \"1fc99a78-bfd8-4fc9-9ff7-bdd5adeaff73\") " pod="openstack-kuttl-tests/keystonec8af-account-delete-xhhcr" Dec 03 11:42:28 crc kubenswrapper[5052]: I1203 11:42:28.948763 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-b6hww\" (UniqueName: \"kubernetes.io/projected/1fc99a78-bfd8-4fc9-9ff7-bdd5adeaff73-kube-api-access-b6hww\") pod \"keystonec8af-account-delete-xhhcr\" (UID: \"1fc99a78-bfd8-4fc9-9ff7-bdd5adeaff73\") " pod="openstack-kuttl-tests/keystonec8af-account-delete-xhhcr" Dec 03 11:42:28 crc kubenswrapper[5052]: E1203 11:42:28.950631 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 11:42:28 crc kubenswrapper[5052]: E1203 11:42:28.950697 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1fc99a78-bfd8-4fc9-9ff7-bdd5adeaff73-operator-scripts podName:1fc99a78-bfd8-4fc9-9ff7-bdd5adeaff73 nodeName:}" failed. No retries permitted until 2025-12-03 11:42:29.950678825 +0000 UTC m=+1340.538920589 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/1fc99a78-bfd8-4fc9-9ff7-bdd5adeaff73-operator-scripts") pod "keystonec8af-account-delete-xhhcr" (UID: "1fc99a78-bfd8-4fc9-9ff7-bdd5adeaff73") : configmap "openstack-scripts" not found Dec 03 11:42:28 crc kubenswrapper[5052]: E1203 11:42:28.952432 5052 projected.go:194] Error preparing data for projected volume kube-api-access-b6hww for pod openstack-kuttl-tests/keystonec8af-account-delete-xhhcr: failed to fetch token: serviceaccounts "galera-openstack" not found Dec 03 11:42:28 crc kubenswrapper[5052]: E1203 11:42:28.952673 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1fc99a78-bfd8-4fc9-9ff7-bdd5adeaff73-kube-api-access-b6hww podName:1fc99a78-bfd8-4fc9-9ff7-bdd5adeaff73 nodeName:}" failed. No retries permitted until 2025-12-03 11:42:29.952656943 +0000 UTC m=+1340.540898807 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-b6hww" (UniqueName: "kubernetes.io/projected/1fc99a78-bfd8-4fc9-9ff7-bdd5adeaff73-kube-api-access-b6hww") pod "keystonec8af-account-delete-xhhcr" (UID: "1fc99a78-bfd8-4fc9-9ff7-bdd5adeaff73") : failed to fetch token: serviceaccounts "galera-openstack" not found Dec 03 11:42:28 crc kubenswrapper[5052]: I1203 11:42:28.985658 5052 kubelet_pods.go:1007] "Unable to retrieve pull secret, the image pull may not succeed." pod="openstack-kuttl-tests/glance78ad-account-delete-7l64w" secret="" err="secret \"galera-openstack-dockercfg-dfbmj\" not found" Dec 03 11:42:28 crc kubenswrapper[5052]: I1203 11:42:28.985718 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance78ad-account-delete-7l64w" Dec 03 11:42:28 crc kubenswrapper[5052]: I1203 11:42:28.990098 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/722e5457-ae47-40b8-816f-8eab5b7a4cb0-kube-api-access-9xmxf" (OuterVolumeSpecName: "kube-api-access-9xmxf") pod "722e5457-ae47-40b8-816f-8eab5b7a4cb0" (UID: "722e5457-ae47-40b8-816f-8eab5b7a4cb0"). InnerVolumeSpecName "kube-api-access-9xmxf". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:42:28 crc kubenswrapper[5052]: I1203 11:42:28.990845 5052 kubelet_pods.go:1007] "Unable to retrieve pull secret, the image pull may not succeed." pod="openstack-kuttl-tests/novaapi73a4-account-delete-lrzcr" secret="" err="secret \"galera-openstack-dockercfg-dfbmj\" not found" Dec 03 11:42:28 crc kubenswrapper[5052]: I1203 11:42:28.990893 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/novaapi73a4-account-delete-lrzcr" Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.049443 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/722e5457-ae47-40b8-816f-8eab5b7a4cb0-combined-ca-bundle\") pod \"722e5457-ae47-40b8-816f-8eab5b7a4cb0\" (UID: \"722e5457-ae47-40b8-816f-8eab5b7a4cb0\") " Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.049991 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-9xmxf\" (UniqueName: \"kubernetes.io/projected/722e5457-ae47-40b8-816f-8eab5b7a4cb0-kube-api-access-9xmxf\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:29 crc kubenswrapper[5052]: E1203 11:42:29.050006 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 11:42:29 crc kubenswrapper[5052]: E1203 11:42:29.050080 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/965f5591-b293-4101-be4a-a5329d88fb69-operator-scripts podName:965f5591-b293-4101-be4a-a5329d88fb69 nodeName:}" failed. No retries permitted until 2025-12-03 11:42:29.550060807 +0000 UTC m=+1340.138302641 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/965f5591-b293-4101-be4a-a5329d88fb69-operator-scripts") pod "novaapi73a4-account-delete-lrzcr" (UID: "965f5591-b293-4101-be4a-a5329d88fb69") : configmap "openstack-scripts" not found Dec 03 11:42:29 crc kubenswrapper[5052]: E1203 11:42:29.050145 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 11:42:29 crc kubenswrapper[5052]: E1203 11:42:29.050168 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8408f908-2fbf-4933-9361-68af849aad18-operator-scripts podName:8408f908-2fbf-4933-9361-68af849aad18 nodeName:}" failed. No retries permitted until 2025-12-03 11:42:29.55016108 +0000 UTC m=+1340.138402954 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/8408f908-2fbf-4933-9361-68af849aad18-operator-scripts") pod "glance78ad-account-delete-7l64w" (UID: "8408f908-2fbf-4933-9361-68af849aad18") : configmap "openstack-scripts" not found Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.053065 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.096367 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-54w7t" Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.123631 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstackclient" Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.151479 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/ecbf8b8e-cdb4-45c3-9455-d0dbc551356e-config-data-generated\") pod \"ecbf8b8e-cdb4-45c3-9455-d0dbc551356e\" (UID: \"ecbf8b8e-cdb4-45c3-9455-d0dbc551356e\") " Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.151715 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c2d2h\" (UniqueName: \"kubernetes.io/projected/24f61ea3-1916-46dd-a764-2211a9735d42-kube-api-access-c2d2h\") pod \"24f61ea3-1916-46dd-a764-2211a9735d42\" (UID: \"24f61ea3-1916-46dd-a764-2211a9735d42\") " Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.151743 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/ecbf8b8e-cdb4-45c3-9455-d0dbc551356e-config-data-default\") pod \"ecbf8b8e-cdb4-45c3-9455-d0dbc551356e\" (UID: \"ecbf8b8e-cdb4-45c3-9455-d0dbc551356e\") " Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.151826 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/ecbf8b8e-cdb4-45c3-9455-d0dbc551356e-operator-scripts\") pod \"ecbf8b8e-cdb4-45c3-9455-d0dbc551356e\" (UID: \"ecbf8b8e-cdb4-45c3-9455-d0dbc551356e\") " Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.151877 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ecbf8b8e-cdb4-45c3-9455-d0dbc551356e-combined-ca-bundle\") pod \"ecbf8b8e-cdb4-45c3-9455-d0dbc551356e\" (UID: \"ecbf8b8e-cdb4-45c3-9455-d0dbc551356e\") " Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.151909 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/24f61ea3-1916-46dd-a764-2211a9735d42-combined-ca-bundle\") pod \"24f61ea3-1916-46dd-a764-2211a9735d42\" (UID: \"24f61ea3-1916-46dd-a764-2211a9735d42\") " Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.151943 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"openstack-config\" (UniqueName: \"kubernetes.io/configmap/24f61ea3-1916-46dd-a764-2211a9735d42-openstack-config\") pod \"24f61ea3-1916-46dd-a764-2211a9735d42\" (UID: \"24f61ea3-1916-46dd-a764-2211a9735d42\") " Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.151975 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/ecbf8b8e-cdb4-45c3-9455-d0dbc551356e-galera-tls-certs\") pod \"ecbf8b8e-cdb4-45c3-9455-d0dbc551356e\" (UID: \"ecbf8b8e-cdb4-45c3-9455-d0dbc551356e\") " Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.152018 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k49vf\" (UniqueName: \"kubernetes.io/projected/ecbf8b8e-cdb4-45c3-9455-d0dbc551356e-kube-api-access-k49vf\") pod \"ecbf8b8e-cdb4-45c3-9455-d0dbc551356e\" (UID: \"ecbf8b8e-cdb4-45c3-9455-d0dbc551356e\") " Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.152051 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"openstack-config-secret\" (UniqueName: \"kubernetes.io/secret/24f61ea3-1916-46dd-a764-2211a9735d42-openstack-config-secret\") pod \"24f61ea3-1916-46dd-a764-2211a9735d42\" (UID: \"24f61ea3-1916-46dd-a764-2211a9735d42\") " Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.152079 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/ecbf8b8e-cdb4-45c3-9455-d0dbc551356e-kolla-config\") pod \"ecbf8b8e-cdb4-45c3-9455-d0dbc551356e\" (UID: \"ecbf8b8e-cdb4-45c3-9455-d0dbc551356e\") " Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.152393 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ecbf8b8e-cdb4-45c3-9455-d0dbc551356e-config-data-generated" (OuterVolumeSpecName: "config-data-generated") pod "ecbf8b8e-cdb4-45c3-9455-d0dbc551356e" (UID: "ecbf8b8e-cdb4-45c3-9455-d0dbc551356e"). InnerVolumeSpecName "config-data-generated". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.152955 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/ecbf8b8e-cdb4-45c3-9455-d0dbc551356e-config-data-generated\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.153112 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ecbf8b8e-cdb4-45c3-9455-d0dbc551356e-kolla-config" (OuterVolumeSpecName: "kolla-config") pod "ecbf8b8e-cdb4-45c3-9455-d0dbc551356e" (UID: "ecbf8b8e-cdb4-45c3-9455-d0dbc551356e"). InnerVolumeSpecName "kolla-config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.155249 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ecbf8b8e-cdb4-45c3-9455-d0dbc551356e-config-data-default" (OuterVolumeSpecName: "config-data-default") pod "ecbf8b8e-cdb4-45c3-9455-d0dbc551356e" (UID: "ecbf8b8e-cdb4-45c3-9455-d0dbc551356e"). InnerVolumeSpecName "config-data-default". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.157165 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ecbf8b8e-cdb4-45c3-9455-d0dbc551356e-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "ecbf8b8e-cdb4-45c3-9455-d0dbc551356e" (UID: "ecbf8b8e-cdb4-45c3-9455-d0dbc551356e"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.167817 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/24f61ea3-1916-46dd-a764-2211a9735d42-openstack-config" (OuterVolumeSpecName: "openstack-config") pod "24f61ea3-1916-46dd-a764-2211a9735d42" (UID: "24f61ea3-1916-46dd-a764-2211a9735d42"). InnerVolumeSpecName "openstack-config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.175714 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.183706 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ecbf8b8e-cdb4-45c3-9455-d0dbc551356e-kube-api-access-k49vf" (OuterVolumeSpecName: "kube-api-access-k49vf") pod "ecbf8b8e-cdb4-45c3-9455-d0dbc551356e" (UID: "ecbf8b8e-cdb4-45c3-9455-d0dbc551356e"). InnerVolumeSpecName "kube-api-access-k49vf". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.185304 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.187564 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/24f61ea3-1916-46dd-a764-2211a9735d42-kube-api-access-c2d2h" (OuterVolumeSpecName: "kube-api-access-c2d2h") pod "24f61ea3-1916-46dd-a764-2211a9735d42" (UID: "24f61ea3-1916-46dd-a764-2211a9735d42"). InnerVolumeSpecName "kube-api-access-c2d2h". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.187616 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/24f61ea3-1916-46dd-a764-2211a9735d42-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "24f61ea3-1916-46dd-a764-2211a9735d42" (UID: "24f61ea3-1916-46dd-a764-2211a9735d42"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.235333 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/24f61ea3-1916-46dd-a764-2211a9735d42-openstack-config-secret" (OuterVolumeSpecName: "openstack-config-secret") pod "24f61ea3-1916-46dd-a764-2211a9735d42" (UID: "24f61ea3-1916-46dd-a764-2211a9735d42"). InnerVolumeSpecName "openstack-config-secret". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.242730 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.250023 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.253697 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"mysql-db\" (UniqueName: \"kubernetes.io/local-volume/local-storage12-crc\") pod \"ecbf8b8e-cdb4-45c3-9455-d0dbc551356e\" (UID: \"ecbf8b8e-cdb4-45c3-9455-d0dbc551356e\") " Dec 03 11:42:29 crc kubenswrapper[5052]: E1203 11:42:29.254926 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/combined-ca-bundle: secret "combined-ca-bundle" not found Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.254957 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-c2d2h\" (UniqueName: \"kubernetes.io/projected/24f61ea3-1916-46dd-a764-2211a9735d42-kube-api-access-c2d2h\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:29 crc kubenswrapper[5052]: E1203 11:42:29.255004 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/222f033c-3660-4347-b621-7122c4c94929-combined-ca-bundle podName:222f033c-3660-4347-b621-7122c4c94929 nodeName:}" failed. No retries permitted until 2025-12-03 11:42:31.254990119 +0000 UTC m=+1341.843231883 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "combined-ca-bundle" (UniqueName: "kubernetes.io/secret/222f033c-3660-4347-b621-7122c4c94929-combined-ca-bundle") pod "barbican-worker-67c775d6bc-kwxzt" (UID: "222f033c-3660-4347-b621-7122c4c94929") : secret "combined-ca-bundle" not found Dec 03 11:42:29 crc kubenswrapper[5052]: E1203 11:42:29.255195 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/barbican-worker-config-data: secret "barbican-worker-config-data" not found Dec 03 11:42:29 crc kubenswrapper[5052]: E1203 11:42:29.255370 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/222f033c-3660-4347-b621-7122c4c94929-config-data-custom podName:222f033c-3660-4347-b621-7122c4c94929 nodeName:}" failed. No retries permitted until 2025-12-03 11:42:31.255348949 +0000 UTC m=+1341.843590783 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "config-data-custom" (UniqueName: "kubernetes.io/secret/222f033c-3660-4347-b621-7122c4c94929-config-data-custom") pod "barbican-worker-67c775d6bc-kwxzt" (UID: "222f033c-3660-4347-b621-7122c4c94929") : secret "barbican-worker-config-data" not found Dec 03 11:42:29 crc kubenswrapper[5052]: E1203 11:42:29.255383 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/barbican-config-data: secret "barbican-config-data" not found Dec 03 11:42:29 crc kubenswrapper[5052]: E1203 11:42:29.255629 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/222f033c-3660-4347-b621-7122c4c94929-config-data podName:222f033c-3660-4347-b621-7122c4c94929 nodeName:}" failed. No retries permitted until 2025-12-03 11:42:31.255618217 +0000 UTC m=+1341.843859981 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "config-data" (UniqueName: "kubernetes.io/secret/222f033c-3660-4347-b621-7122c4c94929-config-data") pod "barbican-worker-67c775d6bc-kwxzt" (UID: "222f033c-3660-4347-b621-7122c4c94929") : secret "barbican-config-data" not found Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.255396 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/ecbf8b8e-cdb4-45c3-9455-d0dbc551356e-config-data-default\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.255786 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/ecbf8b8e-cdb4-45c3-9455-d0dbc551356e-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.255898 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/24f61ea3-1916-46dd-a764-2211a9735d42-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.255978 5052 reconciler_common.go:293] "Volume detached for volume \"openstack-config\" (UniqueName: \"kubernetes.io/configmap/24f61ea3-1916-46dd-a764-2211a9735d42-openstack-config\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.256045 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-k49vf\" (UniqueName: \"kubernetes.io/projected/ecbf8b8e-cdb4-45c3-9455-d0dbc551356e-kube-api-access-k49vf\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.256127 5052 reconciler_common.go:293] "Volume detached for volume \"openstack-config-secret\" (UniqueName: \"kubernetes.io/secret/24f61ea3-1916-46dd-a764-2211a9735d42-openstack-config-secret\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.256199 5052 reconciler_common.go:293] "Volume detached for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/ecbf8b8e-cdb4-45c3-9455-d0dbc551356e-kolla-config\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.263773 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.275377 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-proxy-fd9678558-qx8wh" Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.286484 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.318215 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.320576 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.323154 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.357819 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/ee46c3a4-5e21-4f6b-a944-8cf96d87506e-logs\") pod \"ee46c3a4-5e21-4f6b-a944-8cf96d87506e\" (UID: \"ee46c3a4-5e21-4f6b-a944-8cf96d87506e\") " Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.358392 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/82dce330-8f65-4c14-80d4-958eb8ca5212-combined-ca-bundle\") pod \"82dce330-8f65-4c14-80d4-958eb8ca5212\" (UID: \"82dce330-8f65-4c14-80d4-958eb8ca5212\") " Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.361546 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8b465e38-7dbb-4d02-bce0-28ae0145520c-combined-ca-bundle\") pod \"8b465e38-7dbb-4d02-bce0-28ae0145520c\" (UID: \"8b465e38-7dbb-4d02-bce0-28ae0145520c\") " Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.361669 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ee46c3a4-5e21-4f6b-a944-8cf96d87506e-logs" (OuterVolumeSpecName: "logs") pod "ee46c3a4-5e21-4f6b-a944-8cf96d87506e" (UID: "ee46c3a4-5e21-4f6b-a944-8cf96d87506e"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.361853 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/14e9bff5-66d5-4808-8446-cf180d759208-config-data-custom\") pod \"14e9bff5-66d5-4808-8446-cf180d759208\" (UID: \"14e9bff5-66d5-4808-8446-cf180d759208\") " Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.361884 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/fd14a5c0-3402-4891-a726-32d75b1b7857-public-tls-certs\") pod \"fd14a5c0-3402-4891-a726-32d75b1b7857\" (UID: \"fd14a5c0-3402-4891-a726-32d75b1b7857\") " Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.361913 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/665a736c-d550-4160-a8bb-619d1d5ea826-config-data\") pod \"665a736c-d550-4160-a8bb-619d1d5ea826\" (UID: \"665a736c-d550-4160-a8bb-619d1d5ea826\") " Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.361948 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-state-metrics-tls-config\" (UniqueName: \"kubernetes.io/secret/8b465e38-7dbb-4d02-bce0-28ae0145520c-kube-state-metrics-tls-config\") pod \"8b465e38-7dbb-4d02-bce0-28ae0145520c\" (UID: \"8b465e38-7dbb-4d02-bce0-28ae0145520c\") " Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.361977 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/665a736c-d550-4160-a8bb-619d1d5ea826-scripts\") pod \"665a736c-d550-4160-a8bb-619d1d5ea826\" (UID: \"665a736c-d550-4160-a8bb-619d1d5ea826\") " Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.362007 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lhh66\" (UniqueName: \"kubernetes.io/projected/665a736c-d550-4160-a8bb-619d1d5ea826-kube-api-access-lhh66\") pod \"665a736c-d550-4160-a8bb-619d1d5ea826\" (UID: \"665a736c-d550-4160-a8bb-619d1d5ea826\") " Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.362051 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lzhb7\" (UniqueName: \"kubernetes.io/projected/ee46c3a4-5e21-4f6b-a944-8cf96d87506e-kube-api-access-lzhb7\") pod \"ee46c3a4-5e21-4f6b-a944-8cf96d87506e\" (UID: \"ee46c3a4-5e21-4f6b-a944-8cf96d87506e\") " Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.362074 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/14e9bff5-66d5-4808-8446-cf180d759208-combined-ca-bundle\") pod \"14e9bff5-66d5-4808-8446-cf180d759208\" (UID: \"14e9bff5-66d5-4808-8446-cf180d759208\") " Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.362107 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"glance\" (UniqueName: \"kubernetes.io/local-volume/local-storage04-crc\") pod \"665a736c-d550-4160-a8bb-619d1d5ea826\" (UID: \"665a736c-d550-4160-a8bb-619d1d5ea826\") " Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.362136 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/75e4b34e-f7b8-4209-95a6-6cc45f6eb57c-httpd-run\") pod \"75e4b34e-f7b8-4209-95a6-6cc45f6eb57c\" (UID: \"75e4b34e-f7b8-4209-95a6-6cc45f6eb57c\") " Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.362209 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rf74q\" (UniqueName: \"kubernetes.io/projected/75e4b34e-f7b8-4209-95a6-6cc45f6eb57c-kube-api-access-rf74q\") pod \"75e4b34e-f7b8-4209-95a6-6cc45f6eb57c\" (UID: \"75e4b34e-f7b8-4209-95a6-6cc45f6eb57c\") " Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.362266 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/82dce330-8f65-4c14-80d4-958eb8ca5212-internal-tls-certs\") pod \"82dce330-8f65-4c14-80d4-958eb8ca5212\" (UID: \"82dce330-8f65-4c14-80d4-958eb8ca5212\") " Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.362298 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/fd14a5c0-3402-4891-a726-32d75b1b7857-run-httpd\") pod \"fd14a5c0-3402-4891-a726-32d75b1b7857\" (UID: \"fd14a5c0-3402-4891-a726-32d75b1b7857\") " Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.362322 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rnsxg\" (UniqueName: \"kubernetes.io/projected/2501eda8-a78d-4609-a465-761e89bb291e-kube-api-access-rnsxg\") pod \"2501eda8-a78d-4609-a465-761e89bb291e\" (UID: \"2501eda8-a78d-4609-a465-761e89bb291e\") " Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.362357 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/665a736c-d550-4160-a8bb-619d1d5ea826-combined-ca-bundle\") pod \"665a736c-d550-4160-a8bb-619d1d5ea826\" (UID: \"665a736c-d550-4160-a8bb-619d1d5ea826\") " Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.362386 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/82dce330-8f65-4c14-80d4-958eb8ca5212-public-tls-certs\") pod \"82dce330-8f65-4c14-80d4-958eb8ca5212\" (UID: \"82dce330-8f65-4c14-80d4-958eb8ca5212\") " Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.362414 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/75e4b34e-f7b8-4209-95a6-6cc45f6eb57c-combined-ca-bundle\") pod \"75e4b34e-f7b8-4209-95a6-6cc45f6eb57c\" (UID: \"75e4b34e-f7b8-4209-95a6-6cc45f6eb57c\") " Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.362482 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/ee46c3a4-5e21-4f6b-a944-8cf96d87506e-scripts\") pod \"ee46c3a4-5e21-4f6b-a944-8cf96d87506e\" (UID: \"ee46c3a4-5e21-4f6b-a944-8cf96d87506e\") " Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.362532 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/75e4b34e-f7b8-4209-95a6-6cc45f6eb57c-logs\") pod \"75e4b34e-f7b8-4209-95a6-6cc45f6eb57c\" (UID: \"75e4b34e-f7b8-4209-95a6-6cc45f6eb57c\") " Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.362563 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/665a736c-d550-4160-a8bb-619d1d5ea826-internal-tls-certs\") pod \"665a736c-d550-4160-a8bb-619d1d5ea826\" (UID: \"665a736c-d550-4160-a8bb-619d1d5ea826\") " Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.362590 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/fd14a5c0-3402-4891-a726-32d75b1b7857-etc-swift\") pod \"fd14a5c0-3402-4891-a726-32d75b1b7857\" (UID: \"fd14a5c0-3402-4891-a726-32d75b1b7857\") " Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.362615 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/f98f227a-397b-4665-bf18-e00b105ce058-logs\") pod \"f98f227a-397b-4665-bf18-e00b105ce058\" (UID: \"f98f227a-397b-4665-bf18-e00b105ce058\") " Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.362640 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/2501eda8-a78d-4609-a465-761e89bb291e-config-data\") pod \"2501eda8-a78d-4609-a465-761e89bb291e\" (UID: \"2501eda8-a78d-4609-a465-761e89bb291e\") " Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.362667 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hxcnf\" (UniqueName: \"kubernetes.io/projected/f98f227a-397b-4665-bf18-e00b105ce058-kube-api-access-hxcnf\") pod \"f98f227a-397b-4665-bf18-e00b105ce058\" (UID: \"f98f227a-397b-4665-bf18-e00b105ce058\") " Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.362691 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/fd14a5c0-3402-4891-a726-32d75b1b7857-internal-tls-certs\") pod \"fd14a5c0-3402-4891-a726-32d75b1b7857\" (UID: \"fd14a5c0-3402-4891-a726-32d75b1b7857\") " Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.362727 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/665a736c-d550-4160-a8bb-619d1d5ea826-logs\") pod \"665a736c-d550-4160-a8bb-619d1d5ea826\" (UID: \"665a736c-d550-4160-a8bb-619d1d5ea826\") " Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.362753 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/fd14a5c0-3402-4891-a726-32d75b1b7857-config-data\") pod \"fd14a5c0-3402-4891-a726-32d75b1b7857\" (UID: \"fd14a5c0-3402-4891-a726-32d75b1b7857\") " Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.362783 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2501eda8-a78d-4609-a465-761e89bb291e-combined-ca-bundle\") pod \"2501eda8-a78d-4609-a465-761e89bb291e\" (UID: \"2501eda8-a78d-4609-a465-761e89bb291e\") " Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.362804 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/ee46c3a4-5e21-4f6b-a944-8cf96d87506e-public-tls-certs\") pod \"ee46c3a4-5e21-4f6b-a944-8cf96d87506e\" (UID: \"ee46c3a4-5e21-4f6b-a944-8cf96d87506e\") " Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.362826 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/fd14a5c0-3402-4891-a726-32d75b1b7857-log-httpd\") pod \"fd14a5c0-3402-4891-a726-32d75b1b7857\" (UID: \"fd14a5c0-3402-4891-a726-32d75b1b7857\") " Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.362865 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/ee46c3a4-5e21-4f6b-a944-8cf96d87506e-config-data-custom\") pod \"ee46c3a4-5e21-4f6b-a944-8cf96d87506e\" (UID: \"ee46c3a4-5e21-4f6b-a944-8cf96d87506e\") " Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.362881 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f98f227a-397b-4665-bf18-e00b105ce058-combined-ca-bundle\") pod \"f98f227a-397b-4665-bf18-e00b105ce058\" (UID: \"f98f227a-397b-4665-bf18-e00b105ce058\") " Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.363648 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/d80d1476-351a-40c6-9f76-ce66d623df74-config-data\") pod \"d80d1476-351a-40c6-9f76-ce66d623df74\" (UID: \"d80d1476-351a-40c6-9f76-ce66d623df74\") " Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.363682 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/14e9bff5-66d5-4808-8446-cf180d759208-scripts\") pod \"14e9bff5-66d5-4808-8446-cf180d759208\" (UID: \"14e9bff5-66d5-4808-8446-cf180d759208\") " Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.363701 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"glance\" (UniqueName: \"kubernetes.io/local-volume/local-storage15-crc\") pod \"75e4b34e-f7b8-4209-95a6-6cc45f6eb57c\" (UID: \"75e4b34e-f7b8-4209-95a6-6cc45f6eb57c\") " Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.363720 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ee46c3a4-5e21-4f6b-a944-8cf96d87506e-combined-ca-bundle\") pod \"ee46c3a4-5e21-4f6b-a944-8cf96d87506e\" (UID: \"ee46c3a4-5e21-4f6b-a944-8cf96d87506e\") " Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.363735 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d80d1476-351a-40c6-9f76-ce66d623df74-combined-ca-bundle\") pod \"d80d1476-351a-40c6-9f76-ce66d623df74\" (UID: \"d80d1476-351a-40c6-9f76-ce66d623df74\") " Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.363751 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/82dce330-8f65-4c14-80d4-958eb8ca5212-logs\") pod \"82dce330-8f65-4c14-80d4-958eb8ca5212\" (UID: \"82dce330-8f65-4c14-80d4-958eb8ca5212\") " Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.363768 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hczfd\" (UniqueName: \"kubernetes.io/projected/82dce330-8f65-4c14-80d4-958eb8ca5212-kube-api-access-hczfd\") pod \"82dce330-8f65-4c14-80d4-958eb8ca5212\" (UID: \"82dce330-8f65-4c14-80d4-958eb8ca5212\") " Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.363783 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/ee46c3a4-5e21-4f6b-a944-8cf96d87506e-config-data\") pod \"ee46c3a4-5e21-4f6b-a944-8cf96d87506e\" (UID: \"ee46c3a4-5e21-4f6b-a944-8cf96d87506e\") " Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.363809 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/f98f227a-397b-4665-bf18-e00b105ce058-nova-metadata-tls-certs\") pod \"f98f227a-397b-4665-bf18-e00b105ce058\" (UID: \"f98f227a-397b-4665-bf18-e00b105ce058\") " Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.363833 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/ee46c3a4-5e21-4f6b-a944-8cf96d87506e-etc-machine-id\") pod \"ee46c3a4-5e21-4f6b-a944-8cf96d87506e\" (UID: \"ee46c3a4-5e21-4f6b-a944-8cf96d87506e\") " Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.363861 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/14e9bff5-66d5-4808-8446-cf180d759208-etc-machine-id\") pod \"14e9bff5-66d5-4808-8446-cf180d759208\" (UID: \"14e9bff5-66d5-4808-8446-cf180d759208\") " Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.363877 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/665a736c-d550-4160-a8bb-619d1d5ea826-httpd-run\") pod \"665a736c-d550-4160-a8bb-619d1d5ea826\" (UID: \"665a736c-d550-4160-a8bb-619d1d5ea826\") " Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.363892 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wwv7d\" (UniqueName: \"kubernetes.io/projected/8b465e38-7dbb-4d02-bce0-28ae0145520c-kube-api-access-wwv7d\") pod \"8b465e38-7dbb-4d02-bce0-28ae0145520c\" (UID: \"8b465e38-7dbb-4d02-bce0-28ae0145520c\") " Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.363919 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/82dce330-8f65-4c14-80d4-958eb8ca5212-config-data\") pod \"82dce330-8f65-4c14-80d4-958eb8ca5212\" (UID: \"82dce330-8f65-4c14-80d4-958eb8ca5212\") " Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.363934 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/75e4b34e-f7b8-4209-95a6-6cc45f6eb57c-scripts\") pod \"75e4b34e-f7b8-4209-95a6-6cc45f6eb57c\" (UID: \"75e4b34e-f7b8-4209-95a6-6cc45f6eb57c\") " Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.363949 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/fd14a5c0-3402-4891-a726-32d75b1b7857-combined-ca-bundle\") pod \"fd14a5c0-3402-4891-a726-32d75b1b7857\" (UID: \"fd14a5c0-3402-4891-a726-32d75b1b7857\") " Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.363971 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cpm9m\" (UniqueName: \"kubernetes.io/projected/d80d1476-351a-40c6-9f76-ce66d623df74-kube-api-access-cpm9m\") pod \"d80d1476-351a-40c6-9f76-ce66d623df74\" (UID: \"d80d1476-351a-40c6-9f76-ce66d623df74\") " Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.364008 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/75e4b34e-f7b8-4209-95a6-6cc45f6eb57c-public-tls-certs\") pod \"75e4b34e-f7b8-4209-95a6-6cc45f6eb57c\" (UID: \"75e4b34e-f7b8-4209-95a6-6cc45f6eb57c\") " Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.364024 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wzf59\" (UniqueName: \"kubernetes.io/projected/14e9bff5-66d5-4808-8446-cf180d759208-kube-api-access-wzf59\") pod \"14e9bff5-66d5-4808-8446-cf180d759208\" (UID: \"14e9bff5-66d5-4808-8446-cf180d759208\") " Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.364039 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/14e9bff5-66d5-4808-8446-cf180d759208-config-data\") pod \"14e9bff5-66d5-4808-8446-cf180d759208\" (UID: \"14e9bff5-66d5-4808-8446-cf180d759208\") " Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.364055 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/f98f227a-397b-4665-bf18-e00b105ce058-config-data\") pod \"f98f227a-397b-4665-bf18-e00b105ce058\" (UID: \"f98f227a-397b-4665-bf18-e00b105ce058\") " Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.364079 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/ee46c3a4-5e21-4f6b-a944-8cf96d87506e-internal-tls-certs\") pod \"ee46c3a4-5e21-4f6b-a944-8cf96d87506e\" (UID: \"ee46c3a4-5e21-4f6b-a944-8cf96d87506e\") " Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.364102 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/75e4b34e-f7b8-4209-95a6-6cc45f6eb57c-config-data\") pod \"75e4b34e-f7b8-4209-95a6-6cc45f6eb57c\" (UID: \"75e4b34e-f7b8-4209-95a6-6cc45f6eb57c\") " Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.364838 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/ee46c3a4-5e21-4f6b-a944-8cf96d87506e-logs\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.375891 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fd14a5c0-3402-4891-a726-32d75b1b7857-run-httpd" (OuterVolumeSpecName: "run-httpd") pod "fd14a5c0-3402-4891-a726-32d75b1b7857" (UID: "fd14a5c0-3402-4891-a726-32d75b1b7857"). InnerVolumeSpecName "run-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.376348 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/75e4b34e-f7b8-4209-95a6-6cc45f6eb57c-httpd-run" (OuterVolumeSpecName: "httpd-run") pod "75e4b34e-f7b8-4209-95a6-6cc45f6eb57c" (UID: "75e4b34e-f7b8-4209-95a6-6cc45f6eb57c"). InnerVolumeSpecName "httpd-run". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.378890 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fd14a5c0-3402-4891-a726-32d75b1b7857-log-httpd" (OuterVolumeSpecName: "log-httpd") pod "fd14a5c0-3402-4891-a726-32d75b1b7857" (UID: "fd14a5c0-3402-4891-a726-32d75b1b7857"). InnerVolumeSpecName "log-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.379422 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/75e4b34e-f7b8-4209-95a6-6cc45f6eb57c-logs" (OuterVolumeSpecName: "logs") pod "75e4b34e-f7b8-4209-95a6-6cc45f6eb57c" (UID: "75e4b34e-f7b8-4209-95a6-6cc45f6eb57c"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.380166 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f98f227a-397b-4665-bf18-e00b105ce058-logs" (OuterVolumeSpecName: "logs") pod "f98f227a-397b-4665-bf18-e00b105ce058" (UID: "f98f227a-397b-4665-bf18-e00b105ce058"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.397066 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/82dce330-8f65-4c14-80d4-958eb8ca5212-logs" (OuterVolumeSpecName: "logs") pod "82dce330-8f65-4c14-80d4-958eb8ca5212" (UID: "82dce330-8f65-4c14-80d4-958eb8ca5212"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.403697 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/ee46c3a4-5e21-4f6b-a944-8cf96d87506e-etc-machine-id" (OuterVolumeSpecName: "etc-machine-id") pod "ee46c3a4-5e21-4f6b-a944-8cf96d87506e" (UID: "ee46c3a4-5e21-4f6b-a944-8cf96d87506e"). InnerVolumeSpecName "etc-machine-id". PluginName "kubernetes.io/host-path", VolumeGidValue "" Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.403827 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/14e9bff5-66d5-4808-8446-cf180d759208-etc-machine-id" (OuterVolumeSpecName: "etc-machine-id") pod "14e9bff5-66d5-4808-8446-cf180d759208" (UID: "14e9bff5-66d5-4808-8446-cf180d759208"). InnerVolumeSpecName "etc-machine-id". PluginName "kubernetes.io/host-path", VolumeGidValue "" Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.407666 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/665a736c-d550-4160-a8bb-619d1d5ea826-logs" (OuterVolumeSpecName: "logs") pod "665a736c-d550-4160-a8bb-619d1d5ea826" (UID: "665a736c-d550-4160-a8bb-619d1d5ea826"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.407941 5052 patch_prober.go:28] interesting pod/machine-config-daemon-74bj6 container/machine-config-daemon namespace/openshift-machine-config-operator: Liveness probe status=failure output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" start-of-body= Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.408031 5052 prober.go:107] "Probe failed" probeType="Liveness" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" containerName="machine-config-daemon" probeResult="failure" output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.408079 5052 kubelet.go:2542] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.409207 5052 kuberuntime_manager.go:1027] "Message for Container of pod" containerName="machine-config-daemon" containerStatusID={"Type":"cri-o","ID":"9a1bbf3b5805844b6adac48684e34e2c0f409388266eaea05dc4a45f4951ccdf"} pod="openshift-machine-config-operator/machine-config-daemon-74bj6" containerMessage="Container machine-config-daemon failed liveness probe, will be restarted" Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.409263 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" containerName="machine-config-daemon" containerID="cri-o://9a1bbf3b5805844b6adac48684e34e2c0f409388266eaea05dc4a45f4951ccdf" gracePeriod=600 Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.421916 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/665a736c-d550-4160-a8bb-619d1d5ea826-httpd-run" (OuterVolumeSpecName: "httpd-run") pod "665a736c-d550-4160-a8bb-619d1d5ea826" (UID: "665a736c-d550-4160-a8bb-619d1d5ea826"). InnerVolumeSpecName "httpd-run". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.451464 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/665a736c-d550-4160-a8bb-619d1d5ea826-scripts" (OuterVolumeSpecName: "scripts") pod "665a736c-d550-4160-a8bb-619d1d5ea826" (UID: "665a736c-d550-4160-a8bb-619d1d5ea826"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.458302 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/665a736c-d550-4160-a8bb-619d1d5ea826-kube-api-access-lhh66" (OuterVolumeSpecName: "kube-api-access-lhh66") pod "665a736c-d550-4160-a8bb-619d1d5ea826" (UID: "665a736c-d550-4160-a8bb-619d1d5ea826"). InnerVolumeSpecName "kube-api-access-lhh66". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.458403 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2501eda8-a78d-4609-a465-761e89bb291e-kube-api-access-rnsxg" (OuterVolumeSpecName: "kube-api-access-rnsxg") pod "2501eda8-a78d-4609-a465-761e89bb291e" (UID: "2501eda8-a78d-4609-a465-761e89bb291e"). InnerVolumeSpecName "kube-api-access-rnsxg". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.462099 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ee46c3a4-5e21-4f6b-a944-8cf96d87506e-config-data-custom" (OuterVolumeSpecName: "config-data-custom") pod "ee46c3a4-5e21-4f6b-a944-8cf96d87506e" (UID: "ee46c3a4-5e21-4f6b-a944-8cf96d87506e"). InnerVolumeSpecName "config-data-custom". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.466168 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-state-metrics-tls-certs\" (UniqueName: \"kubernetes.io/secret/8b465e38-7dbb-4d02-bce0-28ae0145520c-kube-state-metrics-tls-certs\") pod \"8b465e38-7dbb-4d02-bce0-28ae0145520c\" (UID: \"8b465e38-7dbb-4d02-bce0-28ae0145520c\") " Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.466343 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hj52s\" (UniqueName: \"kubernetes.io/projected/fd14a5c0-3402-4891-a726-32d75b1b7857-kube-api-access-hj52s\") pod \"fd14a5c0-3402-4891-a726-32d75b1b7857\" (UID: \"fd14a5c0-3402-4891-a726-32d75b1b7857\") " Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.466854 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage12-crc" (OuterVolumeSpecName: "mysql-db") pod "ecbf8b8e-cdb4-45c3-9455-d0dbc551356e" (UID: "ecbf8b8e-cdb4-45c3-9455-d0dbc551356e"). InnerVolumeSpecName "local-storage12-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.469116 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"mysql-db\" (UniqueName: \"kubernetes.io/local-volume/local-storage12-crc\") pod \"ecbf8b8e-cdb4-45c3-9455-d0dbc551356e\" (UID: \"ecbf8b8e-cdb4-45c3-9455-d0dbc551356e\") " Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.470001 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/665a736c-d550-4160-a8bb-619d1d5ea826-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.470015 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-lhh66\" (UniqueName: \"kubernetes.io/projected/665a736c-d550-4160-a8bb-619d1d5ea826-kube-api-access-lhh66\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.470026 5052 reconciler_common.go:293] "Volume detached for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/75e4b34e-f7b8-4209-95a6-6cc45f6eb57c-httpd-run\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.470035 5052 reconciler_common.go:293] "Volume detached for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/fd14a5c0-3402-4891-a726-32d75b1b7857-run-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.470045 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-rnsxg\" (UniqueName: \"kubernetes.io/projected/2501eda8-a78d-4609-a465-761e89bb291e-kube-api-access-rnsxg\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.470054 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/75e4b34e-f7b8-4209-95a6-6cc45f6eb57c-logs\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.470061 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/f98f227a-397b-4665-bf18-e00b105ce058-logs\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.470072 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/665a736c-d550-4160-a8bb-619d1d5ea826-logs\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.470083 5052 reconciler_common.go:293] "Volume detached for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/fd14a5c0-3402-4891-a726-32d75b1b7857-log-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.470094 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/ee46c3a4-5e21-4f6b-a944-8cf96d87506e-config-data-custom\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.470104 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/82dce330-8f65-4c14-80d4-958eb8ca5212-logs\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.470114 5052 reconciler_common.go:293] "Volume detached for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/ee46c3a4-5e21-4f6b-a944-8cf96d87506e-etc-machine-id\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.470124 5052 reconciler_common.go:293] "Volume detached for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/14e9bff5-66d5-4808-8446-cf180d759208-etc-machine-id\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.470133 5052 reconciler_common.go:293] "Volume detached for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/665a736c-d550-4160-a8bb-619d1d5ea826-httpd-run\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:29 crc kubenswrapper[5052]: W1203 11:42:29.470616 5052 mount_helper_common.go:34] Warning: mount cleanup skipped because path does not exist: /var/lib/kubelet/pods/ecbf8b8e-cdb4-45c3-9455-d0dbc551356e/volumes/kubernetes.io~local-volume/local-storage12-crc Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.470647 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage12-crc" (OuterVolumeSpecName: "mysql-db") pod "ecbf8b8e-cdb4-45c3-9455-d0dbc551356e" (UID: "ecbf8b8e-cdb4-45c3-9455-d0dbc551356e"). InnerVolumeSpecName "local-storage12-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.487917 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/82dce330-8f65-4c14-80d4-958eb8ca5212-kube-api-access-hczfd" (OuterVolumeSpecName: "kube-api-access-hczfd") pod "82dce330-8f65-4c14-80d4-958eb8ca5212" (UID: "82dce330-8f65-4c14-80d4-958eb8ca5212"). InnerVolumeSpecName "kube-api-access-hczfd". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.487987 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8b465e38-7dbb-4d02-bce0-28ae0145520c-kube-api-access-wwv7d" (OuterVolumeSpecName: "kube-api-access-wwv7d") pod "8b465e38-7dbb-4d02-bce0-28ae0145520c" (UID: "8b465e38-7dbb-4d02-bce0-28ae0145520c"). InnerVolumeSpecName "kube-api-access-wwv7d". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.490808 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/75e4b34e-f7b8-4209-95a6-6cc45f6eb57c-scripts" (OuterVolumeSpecName: "scripts") pod "75e4b34e-f7b8-4209-95a6-6cc45f6eb57c" (UID: "75e4b34e-f7b8-4209-95a6-6cc45f6eb57c"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.493130 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fd14a5c0-3402-4891-a726-32d75b1b7857-kube-api-access-hj52s" (OuterVolumeSpecName: "kube-api-access-hj52s") pod "fd14a5c0-3402-4891-a726-32d75b1b7857" (UID: "fd14a5c0-3402-4891-a726-32d75b1b7857"). InnerVolumeSpecName "kube-api-access-hj52s". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.495560 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d80d1476-351a-40c6-9f76-ce66d623df74-kube-api-access-cpm9m" (OuterVolumeSpecName: "kube-api-access-cpm9m") pod "d80d1476-351a-40c6-9f76-ce66d623df74" (UID: "d80d1476-351a-40c6-9f76-ce66d623df74"). InnerVolumeSpecName "kube-api-access-cpm9m". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.499685 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ee46c3a4-5e21-4f6b-a944-8cf96d87506e-scripts" (OuterVolumeSpecName: "scripts") pod "ee46c3a4-5e21-4f6b-a944-8cf96d87506e" (UID: "ee46c3a4-5e21-4f6b-a944-8cf96d87506e"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.510117 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/75e4b34e-f7b8-4209-95a6-6cc45f6eb57c-kube-api-access-rf74q" (OuterVolumeSpecName: "kube-api-access-rf74q") pod "75e4b34e-f7b8-4209-95a6-6cc45f6eb57c" (UID: "75e4b34e-f7b8-4209-95a6-6cc45f6eb57c"). InnerVolumeSpecName "kube-api-access-rf74q". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.519537 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage04-crc" (OuterVolumeSpecName: "glance") pod "665a736c-d550-4160-a8bb-619d1d5ea826" (UID: "665a736c-d550-4160-a8bb-619d1d5ea826"). InnerVolumeSpecName "local-storage04-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.520153 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f98f227a-397b-4665-bf18-e00b105ce058-kube-api-access-hxcnf" (OuterVolumeSpecName: "kube-api-access-hxcnf") pod "f98f227a-397b-4665-bf18-e00b105ce058" (UID: "f98f227a-397b-4665-bf18-e00b105ce058"). InnerVolumeSpecName "kube-api-access-hxcnf". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.520434 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage15-crc" (OuterVolumeSpecName: "glance") pod "75e4b34e-f7b8-4209-95a6-6cc45f6eb57c" (UID: "75e4b34e-f7b8-4209-95a6-6cc45f6eb57c"). InnerVolumeSpecName "local-storage15-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.520419 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/14e9bff5-66d5-4808-8446-cf180d759208-config-data-custom" (OuterVolumeSpecName: "config-data-custom") pod "14e9bff5-66d5-4808-8446-cf180d759208" (UID: "14e9bff5-66d5-4808-8446-cf180d759208"). InnerVolumeSpecName "config-data-custom". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.524127 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ee46c3a4-5e21-4f6b-a944-8cf96d87506e-kube-api-access-lzhb7" (OuterVolumeSpecName: "kube-api-access-lzhb7") pod "ee46c3a4-5e21-4f6b-a944-8cf96d87506e" (UID: "ee46c3a4-5e21-4f6b-a944-8cf96d87506e"). InnerVolumeSpecName "kube-api-access-lzhb7". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.526867 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/14e9bff5-66d5-4808-8446-cf180d759208-kube-api-access-wzf59" (OuterVolumeSpecName: "kube-api-access-wzf59") pod "14e9bff5-66d5-4808-8446-cf180d759208" (UID: "14e9bff5-66d5-4808-8446-cf180d759208"). InnerVolumeSpecName "kube-api-access-wzf59". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.556855 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fd14a5c0-3402-4891-a726-32d75b1b7857-etc-swift" (OuterVolumeSpecName: "etc-swift") pod "fd14a5c0-3402-4891-a726-32d75b1b7857" (UID: "fd14a5c0-3402-4891-a726-32d75b1b7857"). InnerVolumeSpecName "etc-swift". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.557038 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/14e9bff5-66d5-4808-8446-cf180d759208-scripts" (OuterVolumeSpecName: "scripts") pod "14e9bff5-66d5-4808-8446-cf180d759208" (UID: "14e9bff5-66d5-4808-8446-cf180d759208"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.572607 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-lzhb7\" (UniqueName: \"kubernetes.io/projected/ee46c3a4-5e21-4f6b-a944-8cf96d87506e-kube-api-access-lzhb7\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.572651 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage04-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage04-crc\") on node \"crc\" " Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.572662 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-rf74q\" (UniqueName: \"kubernetes.io/projected/75e4b34e-f7b8-4209-95a6-6cc45f6eb57c-kube-api-access-rf74q\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.572672 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/ee46c3a4-5e21-4f6b-a944-8cf96d87506e-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.572680 5052 reconciler_common.go:293] "Volume detached for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/fd14a5c0-3402-4891-a726-32d75b1b7857-etc-swift\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.572689 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-hxcnf\" (UniqueName: \"kubernetes.io/projected/f98f227a-397b-4665-bf18-e00b105ce058-kube-api-access-hxcnf\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:29 crc kubenswrapper[5052]: E1203 11:42:29.572686 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 11:42:29 crc kubenswrapper[5052]: E1203 11:42:29.572762 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8408f908-2fbf-4933-9361-68af849aad18-operator-scripts podName:8408f908-2fbf-4933-9361-68af849aad18 nodeName:}" failed. No retries permitted until 2025-12-03 11:42:30.572744143 +0000 UTC m=+1341.160985907 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/8408f908-2fbf-4933-9361-68af849aad18-operator-scripts") pod "glance78ad-account-delete-7l64w" (UID: "8408f908-2fbf-4933-9361-68af849aad18") : configmap "openstack-scripts" not found Dec 03 11:42:29 crc kubenswrapper[5052]: E1203 11:42:29.573003 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 11:42:29 crc kubenswrapper[5052]: E1203 11:42:29.573054 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/965f5591-b293-4101-be4a-a5329d88fb69-operator-scripts podName:965f5591-b293-4101-be4a-a5329d88fb69 nodeName:}" failed. No retries permitted until 2025-12-03 11:42:30.573038332 +0000 UTC m=+1341.161280186 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/965f5591-b293-4101-be4a-a5329d88fb69-operator-scripts") pod "novaapi73a4-account-delete-lrzcr" (UID: "965f5591-b293-4101-be4a-a5329d88fb69") : configmap "openstack-scripts" not found Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.572705 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage12-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage12-crc\") on node \"crc\" " Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.573982 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/14e9bff5-66d5-4808-8446-cf180d759208-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.574019 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage15-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage15-crc\") on node \"crc\" " Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.574034 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-hczfd\" (UniqueName: \"kubernetes.io/projected/82dce330-8f65-4c14-80d4-958eb8ca5212-kube-api-access-hczfd\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.574046 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-wwv7d\" (UniqueName: \"kubernetes.io/projected/8b465e38-7dbb-4d02-bce0-28ae0145520c-kube-api-access-wwv7d\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.574058 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/75e4b34e-f7b8-4209-95a6-6cc45f6eb57c-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.574068 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-cpm9m\" (UniqueName: \"kubernetes.io/projected/d80d1476-351a-40c6-9f76-ce66d623df74-kube-api-access-cpm9m\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.574079 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-wzf59\" (UniqueName: \"kubernetes.io/projected/14e9bff5-66d5-4808-8446-cf180d759208-kube-api-access-wzf59\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.574090 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-hj52s\" (UniqueName: \"kubernetes.io/projected/fd14a5c0-3402-4891-a726-32d75b1b7857-kube-api-access-hj52s\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.574102 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/14e9bff5-66d5-4808-8446-cf180d759208-config-data-custom\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.670224 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbicanc57b-account-delete-nspr5"] Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.697492 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0059df68-09bd-434b-b3a8-e2f56cc8bda0" path="/var/lib/kubelet/pods/0059df68-09bd-434b-b3a8-e2f56cc8bda0/volumes" Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.698245 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="066f70dd-4aa5-4b30-b913-2285c642134e" path="/var/lib/kubelet/pods/066f70dd-4aa5-4b30-b913-2285c642134e/volumes" Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.698808 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1285046f-826b-4a30-a414-46e598daf9ca" path="/var/lib/kubelet/pods/1285046f-826b-4a30-a414-46e598daf9ca/volumes" Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.699925 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1609a5b5-6666-47e3-b02e-f796d972d331" path="/var/lib/kubelet/pods/1609a5b5-6666-47e3-b02e-f796d972d331/volumes" Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.700575 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2204f9d2-fb11-429c-808f-8fd46881102d" path="/var/lib/kubelet/pods/2204f9d2-fb11-429c-808f-8fd46881102d/volumes" Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.701178 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="24f61ea3-1916-46dd-a764-2211a9735d42" path="/var/lib/kubelet/pods/24f61ea3-1916-46dd-a764-2211a9735d42/volumes" Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.701637 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="594d60cb-8133-46b7-8cf9-960b1b12ffa3" path="/var/lib/kubelet/pods/594d60cb-8133-46b7-8cf9-960b1b12ffa3/volumes" Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.702618 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5f9bdf8c-c97c-477e-bf48-73fe6ddae983" path="/var/lib/kubelet/pods/5f9bdf8c-c97c-477e-bf48-73fe6ddae983/volumes" Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.703363 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e14d926b-5dc2-46a4-ab27-dfde118cba89" path="/var/lib/kubelet/pods/e14d926b-5dc2-46a4-ab27-dfde118cba89/volumes" Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.884095 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/keystone-6858c48994-zq757" podUID="48409c36-734b-4b1d-adee-0e9981d635c9" containerName="keystone-api" containerID="cri-o://a8c3f8ba64e0fed8deeb5029931d97733022d2a5ae3a2c13c54ed87abfa82462" gracePeriod=30 Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.893804 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.935985 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/barbican-keystone-listener-5dd6f4bfd-glsv7" podUID="3004edac-eef4-4ddc-a539-4d4294029774" containerName="barbican-keystone-listener-log" containerID="cri-o://e7be74f05a4bcf8543f0c42a95e74e455c72cfb86b1351dc8a08f45d49fb16ab" gracePeriod=30 Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.938124 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/barbican-keystone-listener-5dd6f4bfd-glsv7" podUID="3004edac-eef4-4ddc-a539-4d4294029774" containerName="barbican-keystone-listener" containerID="cri-o://4bf71fc6979a9fd31b45238a27ca6a41469062f0e760d70053f9813a19c47d57" gracePeriod=30 Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.948669 5052 generic.go:334] "Generic (PLEG): container finished" podID="eae5b719-0017-4621-9cb4-73ed6353648e" containerID="7474492a6f12a4bba8719b2ffab92d689c3e76fc1c263f7a31eea5c422579780" exitCode=0 Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.999550 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/1fc99a78-bfd8-4fc9-9ff7-bdd5adeaff73-operator-scripts\") pod \"keystonec8af-account-delete-xhhcr\" (UID: \"1fc99a78-bfd8-4fc9-9ff7-bdd5adeaff73\") " pod="openstack-kuttl-tests/keystonec8af-account-delete-xhhcr" Dec 03 11:42:29 crc kubenswrapper[5052]: I1203 11:42:29.999806 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-b6hww\" (UniqueName: \"kubernetes.io/projected/1fc99a78-bfd8-4fc9-9ff7-bdd5adeaff73-kube-api-access-b6hww\") pod \"keystonec8af-account-delete-xhhcr\" (UID: \"1fc99a78-bfd8-4fc9-9ff7-bdd5adeaff73\") " pod="openstack-kuttl-tests/keystonec8af-account-delete-xhhcr" Dec 03 11:42:30 crc kubenswrapper[5052]: E1203 11:42:30.000022 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 11:42:30 crc kubenswrapper[5052]: E1203 11:42:30.000556 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1fc99a78-bfd8-4fc9-9ff7-bdd5adeaff73-operator-scripts podName:1fc99a78-bfd8-4fc9-9ff7-bdd5adeaff73 nodeName:}" failed. No retries permitted until 2025-12-03 11:42:32.000493288 +0000 UTC m=+1342.588735062 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/1fc99a78-bfd8-4fc9-9ff7-bdd5adeaff73-operator-scripts") pod "keystonec8af-account-delete-xhhcr" (UID: "1fc99a78-bfd8-4fc9-9ff7-bdd5adeaff73") : configmap "openstack-scripts" not found Dec 03 11:42:30 crc kubenswrapper[5052]: E1203 11:42:30.018570 5052 projected.go:194] Error preparing data for projected volume kube-api-access-b6hww for pod openstack-kuttl-tests/keystonec8af-account-delete-xhhcr: failed to fetch token: serviceaccounts "galera-openstack" not found Dec 03 11:42:30 crc kubenswrapper[5052]: E1203 11:42:30.018641 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1fc99a78-bfd8-4fc9-9ff7-bdd5adeaff73-kube-api-access-b6hww podName:1fc99a78-bfd8-4fc9-9ff7-bdd5adeaff73 nodeName:}" failed. No retries permitted until 2025-12-03 11:42:32.018620785 +0000 UTC m=+1342.606862549 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-b6hww" (UniqueName: "kubernetes.io/projected/1fc99a78-bfd8-4fc9-9ff7-bdd5adeaff73-kube-api-access-b6hww") pod "keystonec8af-account-delete-xhhcr" (UID: "1fc99a78-bfd8-4fc9-9ff7-bdd5adeaff73") : failed to fetch token: serviceaccounts "galera-openstack" not found Dec 03 11:42:30 crc kubenswrapper[5052]: I1203 11:42:30.019486 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:42:30 crc kubenswrapper[5052]: I1203 11:42:30.035309 5052 generic.go:334] "Generic (PLEG): container finished" podID="251248ba-578c-49cf-b77e-fc3d5c6cb6fd" containerID="5e0a4e3bdf4887c1a95e0d9815e6d8a1fc6c0dccf8359c3a1a7a1505e8cc9053" exitCode=0 Dec 03 11:42:30 crc kubenswrapper[5052]: I1203 11:42:30.073757 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/keystone-6858c48994-zq757" podStartSLOduration=8.073736569 podStartE2EDuration="8.073736569s" podCreationTimestamp="2025-12-03 11:42:22 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:42:29.955073956 +0000 UTC m=+1340.543315730" watchObservedRunningTime="2025-12-03 11:42:30.073736569 +0000 UTC m=+1340.661978323" Dec 03 11:42:30 crc kubenswrapper[5052]: I1203 11:42:30.092419 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:42:30 crc kubenswrapper[5052]: I1203 11:42:30.102837 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:42:30 crc kubenswrapper[5052]: I1203 11:42:30.113393 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 11:42:30 crc kubenswrapper[5052]: I1203 11:42:30.121569 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-proxy-fd9678558-qx8wh" Dec 03 11:42:30 crc kubenswrapper[5052]: I1203 11:42:30.134779 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:42:30 crc kubenswrapper[5052]: I1203 11:42:30.137822 5052 generic.go:334] "Generic (PLEG): container finished" podID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" containerID="9a1bbf3b5805844b6adac48684e34e2c0f409388266eaea05dc4a45f4951ccdf" exitCode=0 Dec 03 11:42:30 crc kubenswrapper[5052]: I1203 11:42:30.145890 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 11:42:30 crc kubenswrapper[5052]: I1203 11:42:30.146219 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/barbican-worker-67c775d6bc-kwxzt" podUID="222f033c-3660-4347-b621-7122c4c94929" containerName="barbican-worker" containerID="cri-o://e34cd5241cdc0d7ab2c3e9a3005641eb15aa576865c7a50550043b4722e25959" gracePeriod=30 Dec 03 11:42:30 crc kubenswrapper[5052]: I1203 11:42:30.146931 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:42:30 crc kubenswrapper[5052]: I1203 11:42:30.145882 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/barbican-worker-67c775d6bc-kwxzt" podUID="222f033c-3660-4347-b621-7122c4c94929" containerName="barbican-worker-log" containerID="cri-o://9e7335aefbd832cdb40aea97b2f03dec68bab190e8c59ded23cf65ab50caa997" gracePeriod=30 Dec 03 11:42:30 crc kubenswrapper[5052]: I1203 11:42:30.148214 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openshift-marketplace/redhat-operators-bvp4v" podUID="18c1ce3d-6d64-433f-a115-9977227455be" containerName="registry-server" containerID="cri-o://1fddb059a8118b5a724a5d5a74bfd9dcb6fdac1b88a0a1f2bd904834de863a32" gracePeriod=2 Dec 03 11:42:30 crc kubenswrapper[5052]: I1203 11:42:30.148353 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 11:42:30 crc kubenswrapper[5052]: I1203 11:42:30.149332 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:42:30 crc kubenswrapper[5052]: I1203 11:42:30.150284 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstackclient" Dec 03 11:42:30 crc kubenswrapper[5052]: I1203 11:42:30.150338 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstackclient" Dec 03 11:42:30 crc kubenswrapper[5052]: I1203 11:42:30.233573 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/barbican-keystone-listener-5dd6f4bfd-glsv7" podStartSLOduration=9.233549888 podStartE2EDuration="9.233549888s" podCreationTimestamp="2025-12-03 11:42:21 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:42:29.994701329 +0000 UTC m=+1340.582943103" watchObservedRunningTime="2025-12-03 11:42:30.233549888 +0000 UTC m=+1340.821791652" Dec 03 11:42:30 crc kubenswrapper[5052]: E1203 11:42:30.312399 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/rabbitmq-config-data: configmap "rabbitmq-config-data" not found Dec 03 11:42:30 crc kubenswrapper[5052]: E1203 11:42:30.312459 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9944d9af-7a7b-4773-8279-72914d63ef5f-config-data podName:9944d9af-7a7b-4773-8279-72914d63ef5f nodeName:}" failed. No retries permitted until 2025-12-03 11:42:34.312444383 +0000 UTC m=+1344.900686147 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "config-data" (UniqueName: "kubernetes.io/configmap/9944d9af-7a7b-4773-8279-72914d63ef5f-config-data") pod "rabbitmq-server-0" (UID: "9944d9af-7a7b-4773-8279-72914d63ef5f") : configmap "rabbitmq-config-data" not found Dec 03 11:42:30 crc kubenswrapper[5052]: I1203 11:42:30.369794 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage12-crc" (UniqueName: "kubernetes.io/local-volume/local-storage12-crc") on node "crc" Dec 03 11:42:30 crc kubenswrapper[5052]: I1203 11:42:30.471208 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage12-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage12-crc\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:30 crc kubenswrapper[5052]: E1203 11:42:30.582842 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 11:42:30 crc kubenswrapper[5052]: E1203 11:42:30.583382 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/965f5591-b293-4101-be4a-a5329d88fb69-operator-scripts podName:965f5591-b293-4101-be4a-a5329d88fb69 nodeName:}" failed. No retries permitted until 2025-12-03 11:42:32.583339815 +0000 UTC m=+1343.171581579 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/965f5591-b293-4101-be4a-a5329d88fb69-operator-scripts") pod "novaapi73a4-account-delete-lrzcr" (UID: "965f5591-b293-4101-be4a-a5329d88fb69") : configmap "openstack-scripts" not found Dec 03 11:42:30 crc kubenswrapper[5052]: E1203 11:42:30.584050 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 11:42:30 crc kubenswrapper[5052]: E1203 11:42:30.584108 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8408f908-2fbf-4933-9361-68af849aad18-operator-scripts podName:8408f908-2fbf-4933-9361-68af849aad18 nodeName:}" failed. No retries permitted until 2025-12-03 11:42:32.584099067 +0000 UTC m=+1343.172340831 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/8408f908-2fbf-4933-9361-68af849aad18-operator-scripts") pod "glance78ad-account-delete-7l64w" (UID: "8408f908-2fbf-4933-9361-68af849aad18") : configmap "openstack-scripts" not found Dec 03 11:42:30 crc kubenswrapper[5052]: I1203 11:42:30.632838 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage04-crc" (UniqueName: "kubernetes.io/local-volume/local-storage04-crc") on node "crc" Dec 03 11:42:30 crc kubenswrapper[5052]: I1203 11:42:30.698820 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage04-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage04-crc\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:30 crc kubenswrapper[5052]: I1203 11:42:30.710846 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/722e5457-ae47-40b8-816f-8eab5b7a4cb0-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "722e5457-ae47-40b8-816f-8eab5b7a4cb0" (UID: "722e5457-ae47-40b8-816f-8eab5b7a4cb0"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:42:30 crc kubenswrapper[5052]: I1203 11:42:30.816675 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/f274168d-df51-4f50-b014-1ea4e85370d7-operator-scripts\") pod \"novacell03d99-account-delete-5gvmz\" (UID: \"f274168d-df51-4f50-b014-1ea4e85370d7\") " pod="openstack-kuttl-tests/novacell03d99-account-delete-5gvmz" Dec 03 11:42:30 crc kubenswrapper[5052]: I1203 11:42:30.816863 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/722e5457-ae47-40b8-816f-8eab5b7a4cb0-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:30 crc kubenswrapper[5052]: E1203 11:42:30.816934 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 11:42:30 crc kubenswrapper[5052]: E1203 11:42:30.816987 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f274168d-df51-4f50-b014-1ea4e85370d7-operator-scripts podName:f274168d-df51-4f50-b014-1ea4e85370d7 nodeName:}" failed. No retries permitted until 2025-12-03 11:42:34.816971542 +0000 UTC m=+1345.405213306 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/f274168d-df51-4f50-b014-1ea4e85370d7-operator-scripts") pod "novacell03d99-account-delete-5gvmz" (UID: "f274168d-df51-4f50-b014-1ea4e85370d7") : configmap "openstack-scripts" not found Dec 03 11:42:30 crc kubenswrapper[5052]: I1203 11:42:30.826658 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/82dce330-8f65-4c14-80d4-958eb8ca5212-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "82dce330-8f65-4c14-80d4-958eb8ca5212" (UID: "82dce330-8f65-4c14-80d4-958eb8ca5212"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:42:30 crc kubenswrapper[5052]: I1203 11:42:30.827226 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/14e9bff5-66d5-4808-8446-cf180d759208-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "14e9bff5-66d5-4808-8446-cf180d759208" (UID: "14e9bff5-66d5-4808-8446-cf180d759208"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:42:30 crc kubenswrapper[5052]: I1203 11:42:30.828619 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/barbican-api-66f4bb9676-j6vzc" podUID="fe69e54e-ae29-49e2-8a21-0ff81cc0100f" containerName="barbican-api" probeResult="failure" output="Get \"https://10.217.0.147:9311/healthcheck\": read tcp 10.217.0.2:39570->10.217.0.147:9311: read: connection reset by peer" Dec 03 11:42:30 crc kubenswrapper[5052]: I1203 11:42:30.828859 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/barbican-api-66f4bb9676-j6vzc" podUID="fe69e54e-ae29-49e2-8a21-0ff81cc0100f" containerName="barbican-api-log" probeResult="failure" output="Get \"https://10.217.0.147:9311/healthcheck\": read tcp 10.217.0.2:39568->10.217.0.147:9311: read: connection reset by peer" Dec 03 11:42:30 crc kubenswrapper[5052]: I1203 11:42:30.839014 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d80d1476-351a-40c6-9f76-ce66d623df74-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "d80d1476-351a-40c6-9f76-ce66d623df74" (UID: "d80d1476-351a-40c6-9f76-ce66d623df74"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:42:30 crc kubenswrapper[5052]: I1203 11:42:30.871871 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage15-crc" (UniqueName: "kubernetes.io/local-volume/local-storage15-crc") on node "crc" Dec 03 11:42:30 crc kubenswrapper[5052]: I1203 11:42:30.884056 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8b465e38-7dbb-4d02-bce0-28ae0145520c-kube-state-metrics-tls-certs" (OuterVolumeSpecName: "kube-state-metrics-tls-certs") pod "8b465e38-7dbb-4d02-bce0-28ae0145520c" (UID: "8b465e38-7dbb-4d02-bce0-28ae0145520c"). InnerVolumeSpecName "kube-state-metrics-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:42:30 crc kubenswrapper[5052]: I1203 11:42:30.894216 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/722e5457-ae47-40b8-816f-8eab5b7a4cb0-openstack-config" (OuterVolumeSpecName: "openstack-config") pod "722e5457-ae47-40b8-816f-8eab5b7a4cb0" (UID: "722e5457-ae47-40b8-816f-8eab5b7a4cb0"). InnerVolumeSpecName "openstack-config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:42:30 crc kubenswrapper[5052]: I1203 11:42:30.918353 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-zmn62\" (UniqueName: \"kubernetes.io/projected/f274168d-df51-4f50-b014-1ea4e85370d7-kube-api-access-zmn62\") pod \"novacell03d99-account-delete-5gvmz\" (UID: \"f274168d-df51-4f50-b014-1ea4e85370d7\") " pod="openstack-kuttl-tests/novacell03d99-account-delete-5gvmz" Dec 03 11:42:30 crc kubenswrapper[5052]: I1203 11:42:30.919547 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage15-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage15-crc\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:30 crc kubenswrapper[5052]: I1203 11:42:30.919632 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d80d1476-351a-40c6-9f76-ce66d623df74-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:30 crc kubenswrapper[5052]: I1203 11:42:30.919706 5052 reconciler_common.go:293] "Volume detached for volume \"openstack-config\" (UniqueName: \"kubernetes.io/configmap/722e5457-ae47-40b8-816f-8eab5b7a4cb0-openstack-config\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:30 crc kubenswrapper[5052]: I1203 11:42:30.919772 5052 reconciler_common.go:293] "Volume detached for volume \"kube-state-metrics-tls-certs\" (UniqueName: \"kubernetes.io/secret/8b465e38-7dbb-4d02-bce0-28ae0145520c-kube-state-metrics-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:30 crc kubenswrapper[5052]: I1203 11:42:30.920220 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/82dce330-8f65-4c14-80d4-958eb8ca5212-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:30 crc kubenswrapper[5052]: I1203 11:42:30.920256 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/14e9bff5-66d5-4808-8446-cf180d759208-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:30 crc kubenswrapper[5052]: E1203 11:42:30.922013 5052 projected.go:194] Error preparing data for projected volume kube-api-access-zmn62 for pod openstack-kuttl-tests/novacell03d99-account-delete-5gvmz: failed to fetch token: serviceaccounts "galera-openstack" not found Dec 03 11:42:30 crc kubenswrapper[5052]: E1203 11:42:30.922082 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f274168d-df51-4f50-b014-1ea4e85370d7-kube-api-access-zmn62 podName:f274168d-df51-4f50-b014-1ea4e85370d7 nodeName:}" failed. No retries permitted until 2025-12-03 11:42:34.922063879 +0000 UTC m=+1345.510305643 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-zmn62" (UniqueName: "kubernetes.io/projected/f274168d-df51-4f50-b014-1ea4e85370d7-kube-api-access-zmn62") pod "novacell03d99-account-delete-5gvmz" (UID: "f274168d-df51-4f50-b014-1ea4e85370d7") : failed to fetch token: serviceaccounts "galera-openstack" not found Dec 03 11:42:30 crc kubenswrapper[5052]: I1203 11:42:30.938044 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8b465e38-7dbb-4d02-bce0-28ae0145520c-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "8b465e38-7dbb-4d02-bce0-28ae0145520c" (UID: "8b465e38-7dbb-4d02-bce0-28ae0145520c"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:42:31 crc kubenswrapper[5052]: I1203 11:42:30.999638 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/75e4b34e-f7b8-4209-95a6-6cc45f6eb57c-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "75e4b34e-f7b8-4209-95a6-6cc45f6eb57c" (UID: "75e4b34e-f7b8-4209-95a6-6cc45f6eb57c"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:42:31 crc kubenswrapper[5052]: I1203 11:42:31.023380 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8b465e38-7dbb-4d02-bce0-28ae0145520c-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:31 crc kubenswrapper[5052]: I1203 11:42:31.023420 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/75e4b34e-f7b8-4209-95a6-6cc45f6eb57c-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:31 crc kubenswrapper[5052]: I1203 11:42:31.048094 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/665a736c-d550-4160-a8bb-619d1d5ea826-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "665a736c-d550-4160-a8bb-619d1d5ea826" (UID: "665a736c-d550-4160-a8bb-619d1d5ea826"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:42:31 crc kubenswrapper[5052]: I1203 11:42:31.050654 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fd14a5c0-3402-4891-a726-32d75b1b7857-config-data" (OuterVolumeSpecName: "config-data") pod "fd14a5c0-3402-4891-a726-32d75b1b7857" (UID: "fd14a5c0-3402-4891-a726-32d75b1b7857"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:42:31 crc kubenswrapper[5052]: I1203 11:42:31.053230 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d80d1476-351a-40c6-9f76-ce66d623df74-config-data" (OuterVolumeSpecName: "config-data") pod "d80d1476-351a-40c6-9f76-ce66d623df74" (UID: "d80d1476-351a-40c6-9f76-ce66d623df74"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:42:31 crc kubenswrapper[5052]: I1203 11:42:31.057424 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8b465e38-7dbb-4d02-bce0-28ae0145520c-kube-state-metrics-tls-config" (OuterVolumeSpecName: "kube-state-metrics-tls-config") pod "8b465e38-7dbb-4d02-bce0-28ae0145520c" (UID: "8b465e38-7dbb-4d02-bce0-28ae0145520c"). InnerVolumeSpecName "kube-state-metrics-tls-config". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:42:31 crc kubenswrapper[5052]: I1203 11:42:31.075482 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f98f227a-397b-4665-bf18-e00b105ce058-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "f98f227a-397b-4665-bf18-e00b105ce058" (UID: "f98f227a-397b-4665-bf18-e00b105ce058"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:42:31 crc kubenswrapper[5052]: I1203 11:42:31.125354 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/d80d1476-351a-40c6-9f76-ce66d623df74-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:31 crc kubenswrapper[5052]: I1203 11:42:31.125385 5052 reconciler_common.go:293] "Volume detached for volume \"kube-state-metrics-tls-config\" (UniqueName: \"kubernetes.io/secret/8b465e38-7dbb-4d02-bce0-28ae0145520c-kube-state-metrics-tls-config\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:31 crc kubenswrapper[5052]: I1203 11:42:31.125397 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/665a736c-d550-4160-a8bb-619d1d5ea826-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:31 crc kubenswrapper[5052]: I1203 11:42:31.125408 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/fd14a5c0-3402-4891-a726-32d75b1b7857-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:31 crc kubenswrapper[5052]: I1203 11:42:31.125416 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f98f227a-397b-4665-bf18-e00b105ce058-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:31 crc kubenswrapper[5052]: I1203 11:42:31.150299 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/82dce330-8f65-4c14-80d4-958eb8ca5212-config-data" (OuterVolumeSpecName: "config-data") pod "82dce330-8f65-4c14-80d4-958eb8ca5212" (UID: "82dce330-8f65-4c14-80d4-958eb8ca5212"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:42:31 crc kubenswrapper[5052]: I1203 11:42:31.154195 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f98f227a-397b-4665-bf18-e00b105ce058-config-data" (OuterVolumeSpecName: "config-data") pod "f98f227a-397b-4665-bf18-e00b105ce058" (UID: "f98f227a-397b-4665-bf18-e00b105ce058"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:42:31 crc kubenswrapper[5052]: I1203 11:42:31.154688 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ecbf8b8e-cdb4-45c3-9455-d0dbc551356e-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "ecbf8b8e-cdb4-45c3-9455-d0dbc551356e" (UID: "ecbf8b8e-cdb4-45c3-9455-d0dbc551356e"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:42:31 crc kubenswrapper[5052]: I1203 11:42:31.158238 5052 kubelet_pods.go:1007] "Unable to retrieve pull secret, the image pull may not succeed." pod="openstack-kuttl-tests/neutronbfb7-account-delete-ff26t" secret="" err="secret \"galera-openstack-dockercfg-dfbmj\" not found" Dec 03 11:42:31 crc kubenswrapper[5052]: I1203 11:42:31.164951 5052 generic.go:334] "Generic (PLEG): container finished" podID="222f033c-3660-4347-b621-7122c4c94929" containerID="9e7335aefbd832cdb40aea97b2f03dec68bab190e8c59ded23cf65ab50caa997" exitCode=143 Dec 03 11:42:31 crc kubenswrapper[5052]: I1203 11:42:31.166769 5052 generic.go:334] "Generic (PLEG): container finished" podID="3004edac-eef4-4ddc-a539-4d4294029774" containerID="e7be74f05a4bcf8543f0c42a95e74e455c72cfb86b1351dc8a08f45d49fb16ab" exitCode=143 Dec 03 11:42:31 crc kubenswrapper[5052]: I1203 11:42:31.185012 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/neutronbfb7-account-delete-ff26t" podStartSLOduration=7.184993088 podStartE2EDuration="7.184993088s" podCreationTimestamp="2025-12-03 11:42:24 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:42:31.177414608 +0000 UTC m=+1341.765656372" watchObservedRunningTime="2025-12-03 11:42:31.184993088 +0000 UTC m=+1341.773234852" Dec 03 11:42:31 crc kubenswrapper[5052]: I1203 11:42:31.187530 5052 kubelet_pods.go:1007] "Unable to retrieve pull secret, the image pull may not succeed." pod="openstack-kuttl-tests/placement79ca-account-delete-sqtcb" secret="" err="secret \"galera-openstack-dockercfg-dfbmj\" not found" Dec 03 11:42:31 crc kubenswrapper[5052]: I1203 11:42:31.204590 5052 generic.go:334] "Generic (PLEG): container finished" podID="d3e34248-e7e5-4158-8add-165d17bee9ab" containerID="a5cf0c996d18409bb7578d077566cdc4ce54bb2d70d008fc3eb57b7017b7c5cb" exitCode=1 Dec 03 11:42:31 crc kubenswrapper[5052]: I1203 11:42:31.205425 5052 kubelet_pods.go:1007] "Unable to retrieve pull secret, the image pull may not succeed." pod="openstack-kuttl-tests/cinder5365-account-delete-shtcv" secret="" err="secret \"galera-openstack-dockercfg-dfbmj\" not found" Dec 03 11:42:31 crc kubenswrapper[5052]: I1203 11:42:31.205460 5052 scope.go:117] "RemoveContainer" containerID="a5cf0c996d18409bb7578d077566cdc4ce54bb2d70d008fc3eb57b7017b7c5cb" Dec 03 11:42:31 crc kubenswrapper[5052]: I1203 11:42:31.212726 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ee46c3a4-5e21-4f6b-a944-8cf96d87506e-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "ee46c3a4-5e21-4f6b-a944-8cf96d87506e" (UID: "ee46c3a4-5e21-4f6b-a944-8cf96d87506e"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:42:31 crc kubenswrapper[5052]: I1203 11:42:31.228494 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/placement-6d84cdf894-dqcg9" podUID="0c7d3107-6dd9-4aa2-a392-68a213013508" containerName="placement-log" containerID="cri-o://f9ff1557270cd7f8f08565a1b152ff021c2e83623d66dbe9dd1606df3722c53a" gracePeriod=30 Dec 03 11:42:31 crc kubenswrapper[5052]: I1203 11:42:31.228644 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/placement-6d84cdf894-dqcg9" podUID="0c7d3107-6dd9-4aa2-a392-68a213013508" containerName="placement-api" containerID="cri-o://c807496164764c8d605198ee634b8fbee67717205c542334178d901158c10093" gracePeriod=30 Dec 03 11:42:31 crc kubenswrapper[5052]: I1203 11:42:31.229054 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/f98f227a-397b-4665-bf18-e00b105ce058-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:31 crc kubenswrapper[5052]: E1203 11:42:31.229070 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 11:42:31 crc kubenswrapper[5052]: E1203 11:42:31.229129 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/bd2dc8ee-f6b9-4c8b-af5e-79e67a2d1ef0-operator-scripts podName:bd2dc8ee-f6b9-4c8b-af5e-79e67a2d1ef0 nodeName:}" failed. No retries permitted until 2025-12-03 11:42:31.729108502 +0000 UTC m=+1342.317350266 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/bd2dc8ee-f6b9-4c8b-af5e-79e67a2d1ef0-operator-scripts") pod "neutronbfb7-account-delete-ff26t" (UID: "bd2dc8ee-f6b9-4c8b-af5e-79e67a2d1ef0") : configmap "openstack-scripts" not found Dec 03 11:42:31 crc kubenswrapper[5052]: I1203 11:42:31.229075 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ecbf8b8e-cdb4-45c3-9455-d0dbc551356e-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:31 crc kubenswrapper[5052]: I1203 11:42:31.229165 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ee46c3a4-5e21-4f6b-a944-8cf96d87506e-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:31 crc kubenswrapper[5052]: I1203 11:42:31.229179 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/82dce330-8f65-4c14-80d4-958eb8ca5212-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:31 crc kubenswrapper[5052]: I1203 11:42:31.245037 5052 generic.go:334] "Generic (PLEG): container finished" podID="0b4b31e5-e82d-4d44-9c8f-d001484293b6" containerID="77dd24bffc9318ae0172000a2a92f5dd4f72dcfd986df49b78a78df630d16741" exitCode=0 Dec 03 11:42:31 crc kubenswrapper[5052]: I1203 11:42:31.247744 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/placement79ca-account-delete-sqtcb" podStartSLOduration=7.247725004 podStartE2EDuration="7.247725004s" podCreationTimestamp="2025-12-03 11:42:24 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:42:31.208825422 +0000 UTC m=+1341.797067196" watchObservedRunningTime="2025-12-03 11:42:31.247725004 +0000 UTC m=+1341.835966768" Dec 03 11:42:31 crc kubenswrapper[5052]: I1203 11:42:31.273842 5052 generic.go:334] "Generic (PLEG): container finished" podID="fe69e54e-ae29-49e2-8a21-0ff81cc0100f" containerID="4c92b1739b5436291f4de41014a637fd37a00db71202d30b8ea1ae9006123ea8" exitCode=0 Dec 03 11:42:31 crc kubenswrapper[5052]: I1203 11:42:31.285894 5052 generic.go:334] "Generic (PLEG): container finished" podID="18c1ce3d-6d64-433f-a115-9977227455be" containerID="1fddb059a8118b5a724a5d5a74bfd9dcb6fdac1b88a0a1f2bd904834de863a32" exitCode=0 Dec 03 11:42:31 crc kubenswrapper[5052]: E1203 11:42:31.332633 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/barbican-config-data: secret "barbican-config-data" not found Dec 03 11:42:31 crc kubenswrapper[5052]: E1203 11:42:31.332725 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/222f033c-3660-4347-b621-7122c4c94929-config-data podName:222f033c-3660-4347-b621-7122c4c94929 nodeName:}" failed. No retries permitted until 2025-12-03 11:42:35.332700896 +0000 UTC m=+1345.920942730 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "config-data" (UniqueName: "kubernetes.io/secret/222f033c-3660-4347-b621-7122c4c94929-config-data") pod "barbican-worker-67c775d6bc-kwxzt" (UID: "222f033c-3660-4347-b621-7122c4c94929") : secret "barbican-config-data" not found Dec 03 11:42:31 crc kubenswrapper[5052]: E1203 11:42:31.333098 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 11:42:31 crc kubenswrapper[5052]: E1203 11:42:31.333133 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/32573220-587f-4703-a5e9-0fb67d2019ef-operator-scripts podName:32573220-587f-4703-a5e9-0fb67d2019ef nodeName:}" failed. No retries permitted until 2025-12-03 11:42:31.833126218 +0000 UTC m=+1342.421367982 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/32573220-587f-4703-a5e9-0fb67d2019ef-operator-scripts") pod "placement79ca-account-delete-sqtcb" (UID: "32573220-587f-4703-a5e9-0fb67d2019ef") : configmap "openstack-scripts" not found Dec 03 11:42:31 crc kubenswrapper[5052]: E1203 11:42:31.333182 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/combined-ca-bundle: secret "combined-ca-bundle" not found Dec 03 11:42:31 crc kubenswrapper[5052]: E1203 11:42:31.333209 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/222f033c-3660-4347-b621-7122c4c94929-combined-ca-bundle podName:222f033c-3660-4347-b621-7122c4c94929 nodeName:}" failed. No retries permitted until 2025-12-03 11:42:35.3332014 +0000 UTC m=+1345.921443244 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "combined-ca-bundle" (UniqueName: "kubernetes.io/secret/222f033c-3660-4347-b621-7122c4c94929-combined-ca-bundle") pod "barbican-worker-67c775d6bc-kwxzt" (UID: "222f033c-3660-4347-b621-7122c4c94929") : secret "combined-ca-bundle" not found Dec 03 11:42:31 crc kubenswrapper[5052]: E1203 11:42:31.334697 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/barbican-worker-config-data: secret "barbican-worker-config-data" not found Dec 03 11:42:31 crc kubenswrapper[5052]: E1203 11:42:31.334777 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/222f033c-3660-4347-b621-7122c4c94929-config-data-custom podName:222f033c-3660-4347-b621-7122c4c94929 nodeName:}" failed. No retries permitted until 2025-12-03 11:42:35.334755326 +0000 UTC m=+1345.922997150 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "config-data-custom" (UniqueName: "kubernetes.io/secret/222f033c-3660-4347-b621-7122c4c94929-config-data-custom") pod "barbican-worker-67c775d6bc-kwxzt" (UID: "222f033c-3660-4347-b621-7122c4c94929") : secret "barbican-worker-config-data" not found Dec 03 11:42:31 crc kubenswrapper[5052]: E1203 11:42:31.335784 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 11:42:31 crc kubenswrapper[5052]: E1203 11:42:31.335843 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d3e34248-e7e5-4158-8add-165d17bee9ab-operator-scripts podName:d3e34248-e7e5-4158-8add-165d17bee9ab nodeName:}" failed. No retries permitted until 2025-12-03 11:42:31.835828357 +0000 UTC m=+1342.424070121 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/d3e34248-e7e5-4158-8add-165d17bee9ab-operator-scripts") pod "cinder5365-account-delete-shtcv" (UID: "d3e34248-e7e5-4158-8add-165d17bee9ab") : configmap "openstack-scripts" not found Dec 03 11:42:31 crc kubenswrapper[5052]: I1203 11:42:31.337918 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2501eda8-a78d-4609-a465-761e89bb291e-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "2501eda8-a78d-4609-a465-761e89bb291e" (UID: "2501eda8-a78d-4609-a465-761e89bb291e"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:42:31 crc kubenswrapper[5052]: I1203 11:42:31.338324 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/722e5457-ae47-40b8-816f-8eab5b7a4cb0-openstack-config-secret" (OuterVolumeSpecName: "openstack-config-secret") pod "722e5457-ae47-40b8-816f-8eab5b7a4cb0" (UID: "722e5457-ae47-40b8-816f-8eab5b7a4cb0"). InnerVolumeSpecName "openstack-config-secret". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:42:31 crc kubenswrapper[5052]: I1203 11:42:31.376745 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2501eda8-a78d-4609-a465-761e89bb291e-config-data" (OuterVolumeSpecName: "config-data") pod "2501eda8-a78d-4609-a465-761e89bb291e" (UID: "2501eda8-a78d-4609-a465-761e89bb291e"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:42:31 crc kubenswrapper[5052]: I1203 11:42:31.377264 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f98f227a-397b-4665-bf18-e00b105ce058-nova-metadata-tls-certs" (OuterVolumeSpecName: "nova-metadata-tls-certs") pod "f98f227a-397b-4665-bf18-e00b105ce058" (UID: "f98f227a-397b-4665-bf18-e00b105ce058"). InnerVolumeSpecName "nova-metadata-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:42:31 crc kubenswrapper[5052]: I1203 11:42:31.401659 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/75e4b34e-f7b8-4209-95a6-6cc45f6eb57c-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "75e4b34e-f7b8-4209-95a6-6cc45f6eb57c" (UID: "75e4b34e-f7b8-4209-95a6-6cc45f6eb57c"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:42:31 crc kubenswrapper[5052]: I1203 11:42:31.432672 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/665a736c-d550-4160-a8bb-619d1d5ea826-config-data" (OuterVolumeSpecName: "config-data") pod "665a736c-d550-4160-a8bb-619d1d5ea826" (UID: "665a736c-d550-4160-a8bb-619d1d5ea826"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:42:31 crc kubenswrapper[5052]: I1203 11:42:31.434139 5052 reconciler_common.go:293] "Volume detached for volume \"openstack-config-secret\" (UniqueName: \"kubernetes.io/secret/722e5457-ae47-40b8-816f-8eab5b7a4cb0-openstack-config-secret\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:31 crc kubenswrapper[5052]: I1203 11:42:31.434174 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/2501eda8-a78d-4609-a465-761e89bb291e-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:31 crc kubenswrapper[5052]: I1203 11:42:31.434186 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2501eda8-a78d-4609-a465-761e89bb291e-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:31 crc kubenswrapper[5052]: I1203 11:42:31.434196 5052 reconciler_common.go:293] "Volume detached for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/f98f227a-397b-4665-bf18-e00b105ce058-nova-metadata-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:31 crc kubenswrapper[5052]: I1203 11:42:31.434207 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/75e4b34e-f7b8-4209-95a6-6cc45f6eb57c-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:31 crc kubenswrapper[5052]: I1203 11:42:31.434221 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/665a736c-d550-4160-a8bb-619d1d5ea826-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:31 crc kubenswrapper[5052]: I1203 11:42:31.464051 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/placement-6d84cdf894-dqcg9" podUID="0c7d3107-6dd9-4aa2-a392-68a213013508" containerName="placement-log" probeResult="failure" output="Get \"https://10.217.0.190:8778/\": read tcp 10.217.0.2:40884->10.217.0.190:8778: read: connection reset by peer" Dec 03 11:42:31 crc kubenswrapper[5052]: I1203 11:42:31.493883 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fd14a5c0-3402-4891-a726-32d75b1b7857-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "fd14a5c0-3402-4891-a726-32d75b1b7857" (UID: "fd14a5c0-3402-4891-a726-32d75b1b7857"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:42:31 crc kubenswrapper[5052]: I1203 11:42:31.499772 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/665a736c-d550-4160-a8bb-619d1d5ea826-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "665a736c-d550-4160-a8bb-619d1d5ea826" (UID: "665a736c-d550-4160-a8bb-619d1d5ea826"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:42:31 crc kubenswrapper[5052]: I1203 11:42:31.536339 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/665a736c-d550-4160-a8bb-619d1d5ea826-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:31 crc kubenswrapper[5052]: I1203 11:42:31.536385 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/fd14a5c0-3402-4891-a726-32d75b1b7857-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:31 crc kubenswrapper[5052]: I1203 11:42:31.540663 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ecbf8b8e-cdb4-45c3-9455-d0dbc551356e-galera-tls-certs" (OuterVolumeSpecName: "galera-tls-certs") pod "ecbf8b8e-cdb4-45c3-9455-d0dbc551356e" (UID: "ecbf8b8e-cdb4-45c3-9455-d0dbc551356e"). InnerVolumeSpecName "galera-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:42:31 crc kubenswrapper[5052]: I1203 11:42:31.540710 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fd14a5c0-3402-4891-a726-32d75b1b7857-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "fd14a5c0-3402-4891-a726-32d75b1b7857" (UID: "fd14a5c0-3402-4891-a726-32d75b1b7857"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:42:31 crc kubenswrapper[5052]: I1203 11:42:31.541014 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/75e4b34e-f7b8-4209-95a6-6cc45f6eb57c-config-data" (OuterVolumeSpecName: "config-data") pod "75e4b34e-f7b8-4209-95a6-6cc45f6eb57c" (UID: "75e4b34e-f7b8-4209-95a6-6cc45f6eb57c"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:42:31 crc kubenswrapper[5052]: I1203 11:42:31.551726 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ee46c3a4-5e21-4f6b-a944-8cf96d87506e-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "ee46c3a4-5e21-4f6b-a944-8cf96d87506e" (UID: "ee46c3a4-5e21-4f6b-a944-8cf96d87506e"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:42:31 crc kubenswrapper[5052]: I1203 11:42:31.557339 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fd14a5c0-3402-4891-a726-32d75b1b7857-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "fd14a5c0-3402-4891-a726-32d75b1b7857" (UID: "fd14a5c0-3402-4891-a726-32d75b1b7857"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:42:31 crc kubenswrapper[5052]: I1203 11:42:31.566553 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/82dce330-8f65-4c14-80d4-958eb8ca5212-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "82dce330-8f65-4c14-80d4-958eb8ca5212" (UID: "82dce330-8f65-4c14-80d4-958eb8ca5212"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:42:31 crc kubenswrapper[5052]: I1203 11:42:31.586142 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ee46c3a4-5e21-4f6b-a944-8cf96d87506e-config-data" (OuterVolumeSpecName: "config-data") pod "ee46c3a4-5e21-4f6b-a944-8cf96d87506e" (UID: "ee46c3a4-5e21-4f6b-a944-8cf96d87506e"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:42:31 crc kubenswrapper[5052]: I1203 11:42:31.588397 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/82dce330-8f65-4c14-80d4-958eb8ca5212-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "82dce330-8f65-4c14-80d4-958eb8ca5212" (UID: "82dce330-8f65-4c14-80d4-958eb8ca5212"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:42:31 crc kubenswrapper[5052]: I1203 11:42:31.598594 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ee46c3a4-5e21-4f6b-a944-8cf96d87506e-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "ee46c3a4-5e21-4f6b-a944-8cf96d87506e" (UID: "ee46c3a4-5e21-4f6b-a944-8cf96d87506e"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:42:31 crc kubenswrapper[5052]: I1203 11:42:31.602470 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/14e9bff5-66d5-4808-8446-cf180d759208-config-data" (OuterVolumeSpecName: "config-data") pod "14e9bff5-66d5-4808-8446-cf180d759208" (UID: "14e9bff5-66d5-4808-8446-cf180d759208"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:42:31 crc kubenswrapper[5052]: I1203 11:42:31.639107 5052 reconciler_common.go:293] "Volume detached for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/ecbf8b8e-cdb4-45c3-9455-d0dbc551356e-galera-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:31 crc kubenswrapper[5052]: I1203 11:42:31.639147 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/fd14a5c0-3402-4891-a726-32d75b1b7857-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:31 crc kubenswrapper[5052]: I1203 11:42:31.639160 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/ee46c3a4-5e21-4f6b-a944-8cf96d87506e-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:31 crc kubenswrapper[5052]: I1203 11:42:31.639170 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/ee46c3a4-5e21-4f6b-a944-8cf96d87506e-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:31 crc kubenswrapper[5052]: I1203 11:42:31.639181 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/14e9bff5-66d5-4808-8446-cf180d759208-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:31 crc kubenswrapper[5052]: I1203 11:42:31.639191 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/ee46c3a4-5e21-4f6b-a944-8cf96d87506e-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:31 crc kubenswrapper[5052]: I1203 11:42:31.639202 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/75e4b34e-f7b8-4209-95a6-6cc45f6eb57c-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:31 crc kubenswrapper[5052]: I1203 11:42:31.639213 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/fd14a5c0-3402-4891-a726-32d75b1b7857-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:31 crc kubenswrapper[5052]: I1203 11:42:31.639223 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/82dce330-8f65-4c14-80d4-958eb8ca5212-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:31 crc kubenswrapper[5052]: I1203 11:42:31.639235 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/82dce330-8f65-4c14-80d4-958eb8ca5212-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:31 crc kubenswrapper[5052]: E1203 11:42:31.740944 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 11:42:31 crc kubenswrapper[5052]: E1203 11:42:31.741031 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/bd2dc8ee-f6b9-4c8b-af5e-79e67a2d1ef0-operator-scripts podName:bd2dc8ee-f6b9-4c8b-af5e-79e67a2d1ef0 nodeName:}" failed. No retries permitted until 2025-12-03 11:42:32.741011674 +0000 UTC m=+1343.329253438 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/bd2dc8ee-f6b9-4c8b-af5e-79e67a2d1ef0-operator-scripts") pod "neutronbfb7-account-delete-ff26t" (UID: "bd2dc8ee-f6b9-4c8b-af5e-79e67a2d1ef0") : configmap "openstack-scripts" not found Dec 03 11:42:31 crc kubenswrapper[5052]: E1203 11:42:31.745632 5052 cadvisor_stats_provider.go:516] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod32573220_587f_4703_a5e9_0fb67d2019ef.slice/crio-conmon-c04cb50606e320ee3fa81bcf57983f377ac852528a4865911ec99a80eba0fc63.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0c7d3107_6dd9_4aa2_a392_68a213013508.slice/crio-c807496164764c8d605198ee634b8fbee67717205c542334178d901158c10093.scope\": RecentStats: unable to find data in memory cache]" Dec 03 11:42:31 crc kubenswrapper[5052]: E1203 11:42:31.826208 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="455e8792612e9ef5fbedefc9390ff87ea262cbd9365b333801dd2dc51224737e" cmd=["/usr/local/bin/container-scripts/status_check.sh"] Dec 03 11:42:31 crc kubenswrapper[5052]: E1203 11:42:31.828323 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="455e8792612e9ef5fbedefc9390ff87ea262cbd9365b333801dd2dc51224737e" cmd=["/usr/local/bin/container-scripts/status_check.sh"] Dec 03 11:42:31 crc kubenswrapper[5052]: E1203 11:42:31.830033 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="455e8792612e9ef5fbedefc9390ff87ea262cbd9365b333801dd2dc51224737e" cmd=["/usr/local/bin/container-scripts/status_check.sh"] Dec 03 11:42:31 crc kubenswrapper[5052]: E1203 11:42:31.830106 5052 prober.go:104] "Probe errored" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" probeType="Readiness" pod="openstack-kuttl-tests/ovn-northd-0" podUID="a30eaf15-65ff-426b-8ac2-1ed5a369886a" containerName="ovn-northd" Dec 03 11:42:31 crc kubenswrapper[5052]: E1203 11:42:31.843191 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 11:42:31 crc kubenswrapper[5052]: E1203 11:42:31.843283 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d3e34248-e7e5-4158-8add-165d17bee9ab-operator-scripts podName:d3e34248-e7e5-4158-8add-165d17bee9ab nodeName:}" failed. No retries permitted until 2025-12-03 11:42:32.843258098 +0000 UTC m=+1343.431499872 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/d3e34248-e7e5-4158-8add-165d17bee9ab-operator-scripts") pod "cinder5365-account-delete-shtcv" (UID: "d3e34248-e7e5-4158-8add-165d17bee9ab") : configmap "openstack-scripts" not found Dec 03 11:42:31 crc kubenswrapper[5052]: E1203 11:42:31.843363 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 11:42:31 crc kubenswrapper[5052]: E1203 11:42:31.843387 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/32573220-587f-4703-a5e9-0fb67d2019ef-operator-scripts podName:32573220-587f-4703-a5e9-0fb67d2019ef nodeName:}" failed. No retries permitted until 2025-12-03 11:42:32.843380742 +0000 UTC m=+1343.431622506 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/32573220-587f-4703-a5e9-0fb67d2019ef-operator-scripts") pod "placement79ca-account-delete-sqtcb" (UID: "32573220-587f-4703-a5e9-0fb67d2019ef") : configmap "openstack-scripts" not found Dec 03 11:42:31 crc kubenswrapper[5052]: E1203 11:42:31.987288 5052 kubelet.go:2526] "Housekeeping took longer than expected" err="housekeeping took too long" expected="1s" actual="2.315s" Dec 03 11:42:31 crc kubenswrapper[5052]: I1203 11:42:31.987335 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-df67d6cb4-8sllj" event={"ID":"b88af976-65f2-4d29-8c14-d5c43e23a459","Type":"ContainerStarted","Data":"07660c848ebf1a69fb37d115276c6d328ca3e8bccca35abc98bb76e21788b417"} Dec 03 11:42:31 crc kubenswrapper[5052]: I1203 11:42:31.987420 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutronbfb7-account-delete-ff26t" event={"ID":"bd2dc8ee-f6b9-4c8b-af5e-79e67a2d1ef0","Type":"ContainerStarted","Data":"e22e037c01d56a9771da6e8bbb240cecd8ba3025908ef45d7f860cce8ecd6d9a"} Dec 03 11:42:31 crc kubenswrapper[5052]: I1203 11:42:31.987444 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-6858c48994-zq757" event={"ID":"48409c36-734b-4b1d-adee-0e9981d635c9","Type":"ContainerStarted","Data":"a8c3f8ba64e0fed8deeb5029931d97733022d2a5ae3a2c13c54ed87abfa82462"} Dec 03 11:42:31 crc kubenswrapper[5052]: I1203 11:42:31.987457 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"75e4b34e-f7b8-4209-95a6-6cc45f6eb57c","Type":"ContainerDied","Data":"66330ee1ed6977ce001254e4d9a3a10bb2366368e36c994e7f2059d8938176af"} Dec 03 11:42:31 crc kubenswrapper[5052]: I1203 11:42:31.998850 5052 scope.go:117] "RemoveContainer" containerID="2b213199dec26418337b9d73443612a12a1ce69360675968db75a8f6aba268ae" Dec 03 11:42:32 crc kubenswrapper[5052]: I1203 11:42:32.025681 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 11:42:32 crc kubenswrapper[5052]: I1203 11:42:32.032570 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="722e5457-ae47-40b8-816f-8eab5b7a4cb0" path="/var/lib/kubelet/pods/722e5457-ae47-40b8-816f-8eab5b7a4cb0/volumes" Dec 03 11:42:32 crc kubenswrapper[5052]: I1203 11:42:32.035806 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance78ad-account-delete-7l64w"] Dec 03 11:42:32 crc kubenswrapper[5052]: I1203 11:42:32.035844 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder-db-create-c922p"] Dec 03 11:42:32 crc kubenswrapper[5052]: I1203 11:42:32.035867 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/keystone-6858c48994-zq757" Dec 03 11:42:32 crc kubenswrapper[5052]: I1203 11:42:32.035884 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/cinder-db-create-c922p"] Dec 03 11:42:32 crc kubenswrapper[5052]: I1203 11:42:32.035903 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/placement-6d84cdf894-dqcg9" Dec 03 11:42:32 crc kubenswrapper[5052]: I1203 11:42:32.035916 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance78ad-account-delete-7l64w" event={"ID":"8408f908-2fbf-4933-9361-68af849aad18","Type":"ContainerStarted","Data":"95f80e3d46b311c5878997c907876d12b66a84948e0743050f0551ed84d32a11"} Dec 03 11:42:32 crc kubenswrapper[5052]: I1203 11:42:32.035937 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-keystone-listener-5dd6f4bfd-glsv7" event={"ID":"3004edac-eef4-4ddc-a539-4d4294029774","Type":"ContainerStarted","Data":"4bf71fc6979a9fd31b45238a27ca6a41469062f0e760d70053f9813a19c47d57"} Dec 03 11:42:32 crc kubenswrapper[5052]: I1203 11:42:32.035954 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder-5365-account-create-update-w5fnk"] Dec 03 11:42:32 crc kubenswrapper[5052]: I1203 11:42:32.035968 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"eae5b719-0017-4621-9cb4-73ed6353648e","Type":"ContainerDied","Data":"7474492a6f12a4bba8719b2ffab92d689c3e76fc1c263f7a31eea5c422579780"} Dec 03 11:42:32 crc kubenswrapper[5052]: I1203 11:42:32.035987 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/placement-6d84cdf894-dqcg9" Dec 03 11:42:32 crc kubenswrapper[5052]: I1203 11:42:32.036002 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"eae5b719-0017-4621-9cb4-73ed6353648e","Type":"ContainerDied","Data":"b038a1489d4fb7dcf01398e95cf717b68ddf1cbe706098dc42c66f25700f0667"} Dec 03 11:42:32 crc kubenswrapper[5052]: I1203 11:42:32.036013 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b038a1489d4fb7dcf01398e95cf717b68ddf1cbe706098dc42c66f25700f0667" Dec 03 11:42:32 crc kubenswrapper[5052]: I1203 11:42:32.036023 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-conductor-0" event={"ID":"f8cdf05e-7081-4542-844d-f76378e8989f","Type":"ContainerDied","Data":"ba20fb745c314713f7c6e16fe25ffbfb735832f224a085cb11f9713cbedbc771"} Dec 03 11:42:32 crc kubenswrapper[5052]: I1203 11:42:32.036048 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/cinder-5365-account-create-update-w5fnk"] Dec 03 11:42:32 crc kubenswrapper[5052]: I1203 11:42:32.036067 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"82dce330-8f65-4c14-80d4-958eb8ca5212","Type":"ContainerDied","Data":"a925b2a38721f9f89df454d11e7db79c799898fc4b17f0e225ad34b672efb017"} Dec 03 11:42:32 crc kubenswrapper[5052]: I1203 11:42:32.036083 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder5365-account-delete-shtcv"] Dec 03 11:42:32 crc kubenswrapper[5052]: I1203 11:42:32.036100 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-galera-0" event={"ID":"251248ba-578c-49cf-b77e-fc3d5c6cb6fd","Type":"ContainerDied","Data":"5e0a4e3bdf4887c1a95e0d9815e6d8a1fc6c0dccf8359c3a1a7a1505e8cc9053"} Dec 03 11:42:32 crc kubenswrapper[5052]: I1203 11:42:32.036119 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-galera-0" event={"ID":"251248ba-578c-49cf-b77e-fc3d5c6cb6fd","Type":"ContainerDied","Data":"099cdfaa9cfd4abce46e13e4ab35647f5b1187b327bbce29cce00cc459aed7b4"} Dec 03 11:42:32 crc kubenswrapper[5052]: I1203 11:42:32.036130 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="099cdfaa9cfd4abce46e13e4ab35647f5b1187b327bbce29cce00cc459aed7b4" Dec 03 11:42:32 crc kubenswrapper[5052]: I1203 11:42:32.036142 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"ee46c3a4-5e21-4f6b-a944-8cf96d87506e","Type":"ContainerDied","Data":"a2f0d9ea0593c2ee959146b8662e9a0356d2ce9d308b5519e59c62c7aa9da179"} Dec 03 11:42:32 crc kubenswrapper[5052]: I1203 11:42:32.036156 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"d80d1476-351a-40c6-9f76-ce66d623df74","Type":"ContainerDied","Data":"8ecda4d41d18206a128e089bbc4ef5a55fa2288abf2a12148734dafbc276a6bb"} Dec 03 11:42:32 crc kubenswrapper[5052]: I1203 11:42:32.049987 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-b6hww\" (UniqueName: \"kubernetes.io/projected/1fc99a78-bfd8-4fc9-9ff7-bdd5adeaff73-kube-api-access-b6hww\") pod \"keystonec8af-account-delete-xhhcr\" (UID: \"1fc99a78-bfd8-4fc9-9ff7-bdd5adeaff73\") " pod="openstack-kuttl-tests/keystonec8af-account-delete-xhhcr" Dec 03 11:42:32 crc kubenswrapper[5052]: I1203 11:42:32.050246 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/1fc99a78-bfd8-4fc9-9ff7-bdd5adeaff73-operator-scripts\") pod \"keystonec8af-account-delete-xhhcr\" (UID: \"1fc99a78-bfd8-4fc9-9ff7-bdd5adeaff73\") " pod="openstack-kuttl-tests/keystonec8af-account-delete-xhhcr" Dec 03 11:42:32 crc kubenswrapper[5052]: E1203 11:42:32.050453 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 11:42:32 crc kubenswrapper[5052]: E1203 11:42:32.050530 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1fc99a78-bfd8-4fc9-9ff7-bdd5adeaff73-operator-scripts podName:1fc99a78-bfd8-4fc9-9ff7-bdd5adeaff73 nodeName:}" failed. No retries permitted until 2025-12-03 11:42:36.050495917 +0000 UTC m=+1346.638737691 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/1fc99a78-bfd8-4fc9-9ff7-bdd5adeaff73-operator-scripts") pod "keystonec8af-account-delete-xhhcr" (UID: "1fc99a78-bfd8-4fc9-9ff7-bdd5adeaff73") : configmap "openstack-scripts" not found Dec 03 11:42:32 crc kubenswrapper[5052]: I1203 11:42:32.056783 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/novaapi73a4-account-delete-lrzcr"] Dec 03 11:42:32 crc kubenswrapper[5052]: I1203 11:42:32.056817 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/kube-state-metrics-0" event={"ID":"8b465e38-7dbb-4d02-bce0-28ae0145520c","Type":"ContainerDied","Data":"c6fb336a131731e9dc70f681f53cac866c4c33e5f464349113ef876b024ecd59"} Dec 03 11:42:32 crc kubenswrapper[5052]: I1203 11:42:32.056836 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-proxy-fd9678558-qx8wh" event={"ID":"fd14a5c0-3402-4891-a726-32d75b1b7857","Type":"ContainerDied","Data":"568cbc12b65aa7d5ee89f3ddee562073c311967a018243e6d3cb69bb82ff3f4e"} Dec 03 11:42:32 crc kubenswrapper[5052]: I1203 11:42:32.056855 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/neutron-db-create-9kfp5"] Dec 03 11:42:32 crc kubenswrapper[5052]: I1203 11:42:32.056880 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/neutron-db-create-9kfp5"] Dec 03 11:42:32 crc kubenswrapper[5052]: I1203 11:42:32.056895 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-69cf8cb7d6-jfb57" event={"ID":"8f1442a2-a28b-4784-934a-34b417dbcce9","Type":"ContainerStarted","Data":"a5f8b0c46530b7fa02bdfaef171411cf9a70dd517b05545d95c2ab0253afcf85"} Dec 03 11:42:32 crc kubenswrapper[5052]: I1203 11:42:32.056910 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"14e9bff5-66d5-4808-8446-cf180d759208","Type":"ContainerDied","Data":"88c1ea502795f1e909ef74fbad2692105bbd678ea481be72b91edbbae6c098d6"} Dec 03 11:42:32 crc kubenswrapper[5052]: I1203 11:42:32.056921 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement79ca-account-delete-sqtcb" event={"ID":"32573220-587f-4703-a5e9-0fb67d2019ef","Type":"ContainerStarted","Data":"fc5b5266a2b318f1577aca38b5cd24531faf4b17a72d764f030738dce87191c8"} Dec 03 11:42:32 crc kubenswrapper[5052]: I1203 11:42:32.056932 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/neutron-bfb7-account-create-update-2684r"] Dec 03 11:42:32 crc kubenswrapper[5052]: I1203 11:42:32.056944 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/neutronbfb7-account-delete-ff26t"] Dec 03 11:42:32 crc kubenswrapper[5052]: I1203 11:42:32.056958 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/neutron-bfb7-account-create-update-2684r"] Dec 03 11:42:32 crc kubenswrapper[5052]: I1203 11:42:32.056969 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" event={"ID":"10c237bc-72e7-4da3-9f2c-b1688395bf6e","Type":"ContainerDied","Data":"9a1bbf3b5805844b6adac48684e34e2c0f409388266eaea05dc4a45f4951ccdf"} Dec 03 11:42:32 crc kubenswrapper[5052]: I1203 11:42:32.056983 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbicanc57b-account-delete-nspr5" event={"ID":"ce4d9d20-a900-4b3d-8ee0-46ea28a51262","Type":"ContainerStarted","Data":"29683920a6e7674e5e256703c5b75a547484dd141b18d2cae6708264d8942984"} Dec 03 11:42:32 crc kubenswrapper[5052]: I1203 11:42:32.056993 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/placement-db-create-zvq9n"] Dec 03 11:42:32 crc kubenswrapper[5052]: I1203 11:42:32.057004 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutronbfb7-account-delete-ff26t" event={"ID":"bd2dc8ee-f6b9-4c8b-af5e-79e67a2d1ef0","Type":"ContainerStarted","Data":"4514b8aae4d859abb11dc204bd75a6fb855f5e1b4b91f3cb73f993ef8101b439"} Dec 03 11:42:32 crc kubenswrapper[5052]: I1203 11:42:32.057011 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-54w7t" event={"ID":"fc5f71c9-f943-4ba6-a8eb-8e16ed558675","Type":"ContainerStarted","Data":"be1b0eeec7747986ad2608047680e1e8bbe1cc60aedb46608548d387ea104074"} Dec 03 11:42:32 crc kubenswrapper[5052]: I1203 11:42:32.057025 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/placement-db-create-zvq9n"] Dec 03 11:42:32 crc kubenswrapper[5052]: I1203 11:42:32.057037 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-worker-67c775d6bc-kwxzt" event={"ID":"222f033c-3660-4347-b621-7122c4c94929","Type":"ContainerDied","Data":"9e7335aefbd832cdb40aea97b2f03dec68bab190e8c59ded23cf65ab50caa997"} Dec 03 11:42:32 crc kubenswrapper[5052]: I1203 11:42:32.057049 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-keystone-listener-5dd6f4bfd-glsv7" event={"ID":"3004edac-eef4-4ddc-a539-4d4294029774","Type":"ContainerDied","Data":"e7be74f05a4bcf8543f0c42a95e74e455c72cfb86b1351dc8a08f45d49fb16ab"} Dec 03 11:42:32 crc kubenswrapper[5052]: I1203 11:42:32.057059 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/placement-79ca-account-create-update-xhl8v"] Dec 03 11:42:32 crc kubenswrapper[5052]: I1203 11:42:32.057068 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/placement79ca-account-delete-sqtcb"] Dec 03 11:42:32 crc kubenswrapper[5052]: I1203 11:42:32.057076 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/novaapi73a4-account-delete-lrzcr" event={"ID":"965f5591-b293-4101-be4a-a5329d88fb69","Type":"ContainerStarted","Data":"e29abc27dd77b54bfd434a6b172dad89359bf72d4901f1de70c3f2cfd6f60c9b"} Dec 03 11:42:32 crc kubenswrapper[5052]: I1203 11:42:32.057084 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement79ca-account-delete-sqtcb" event={"ID":"32573220-587f-4703-a5e9-0fb67d2019ef","Type":"ContainerStarted","Data":"c04cb50606e320ee3fa81bcf57983f377ac852528a4865911ec99a80eba0fc63"} Dec 03 11:42:32 crc kubenswrapper[5052]: I1203 11:42:32.057094 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/placement-79ca-account-create-update-xhl8v"] Dec 03 11:42:32 crc kubenswrapper[5052]: I1203 11:42:32.057104 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder5365-account-delete-shtcv" event={"ID":"d3e34248-e7e5-4158-8add-165d17bee9ab","Type":"ContainerDied","Data":"a5cf0c996d18409bb7578d077566cdc4ce54bb2d70d008fc3eb57b7017b7c5cb"} Dec 03 11:42:32 crc kubenswrapper[5052]: I1203 11:42:32.057115 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-6d84cdf894-dqcg9" event={"ID":"0c7d3107-6dd9-4aa2-a392-68a213013508","Type":"ContainerStarted","Data":"c807496164764c8d605198ee634b8fbee67717205c542334178d901158c10093"} Dec 03 11:42:32 crc kubenswrapper[5052]: I1203 11:42:32.057125 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-699ddb8646-rfm4p" event={"ID":"0b4b31e5-e82d-4d44-9c8f-d001484293b6","Type":"ContainerDied","Data":"77dd24bffc9318ae0172000a2a92f5dd4f72dcfd986df49b78a78df630d16741"} Dec 03 11:42:32 crc kubenswrapper[5052]: I1203 11:42:32.057150 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-66f4bb9676-j6vzc" event={"ID":"fe69e54e-ae29-49e2-8a21-0ff81cc0100f","Type":"ContainerDied","Data":"4c92b1739b5436291f4de41014a637fd37a00db71202d30b8ea1ae9006123ea8"} Dec 03 11:42:32 crc kubenswrapper[5052]: I1203 11:42:32.057163 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-54w7t"] Dec 03 11:42:32 crc kubenswrapper[5052]: I1203 11:42:32.057172 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-operators-bvp4v" event={"ID":"18c1ce3d-6d64-433f-a115-9977227455be","Type":"ContainerDied","Data":"1fddb059a8118b5a724a5d5a74bfd9dcb6fdac1b88a0a1f2bd904834de863a32"} Dec 03 11:42:32 crc kubenswrapper[5052]: I1203 11:42:32.057183 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-operators-bvp4v" event={"ID":"18c1ce3d-6d64-433f-a115-9977227455be","Type":"ContainerDied","Data":"3abd0b1c0f5c12b60a51715e9fd0f3bc9393f48ca48a455c099f922894cdf07e"} Dec 03 11:42:32 crc kubenswrapper[5052]: I1203 11:42:32.057193 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3abd0b1c0f5c12b60a51715e9fd0f3bc9393f48ca48a455c099f922894cdf07e" Dec 03 11:42:32 crc kubenswrapper[5052]: I1203 11:42:32.057202 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-db-create-nrwg6"] Dec 03 11:42:32 crc kubenswrapper[5052]: I1203 11:42:32.057213 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/barbican-db-create-nrwg6"] Dec 03 11:42:32 crc kubenswrapper[5052]: I1203 11:42:32.057223 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-c57b-account-create-update-nmdhk"] Dec 03 11:42:32 crc kubenswrapper[5052]: I1203 11:42:32.057234 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/barbican-c57b-account-create-update-nmdhk"] Dec 03 11:42:32 crc kubenswrapper[5052]: I1203 11:42:32.057243 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbicanc57b-account-delete-nspr5"] Dec 03 11:42:32 crc kubenswrapper[5052]: I1203 11:42:32.057252 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-db-create-ckh5l"] Dec 03 11:42:32 crc kubenswrapper[5052]: I1203 11:42:32.057261 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/glance-db-create-ckh5l"] Dec 03 11:42:32 crc kubenswrapper[5052]: I1203 11:42:32.057271 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance78ad-account-delete-7l64w"] Dec 03 11:42:32 crc kubenswrapper[5052]: I1203 11:42:32.057281 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-78ad-account-create-update-7wlzh"] Dec 03 11:42:32 crc kubenswrapper[5052]: I1203 11:42:32.057289 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/glance-78ad-account-create-update-7wlzh"] Dec 03 11:42:32 crc kubenswrapper[5052]: I1203 11:42:32.057298 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-api-db-create-wm8nm"] Dec 03 11:42:32 crc kubenswrapper[5052]: I1203 11:42:32.057309 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-api-db-create-wm8nm"] Dec 03 11:42:32 crc kubenswrapper[5052]: I1203 11:42:32.057319 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/novaapi73a4-account-delete-lrzcr"] Dec 03 11:42:32 crc kubenswrapper[5052]: I1203 11:42:32.057328 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-api-73a4-account-create-update-kfmjt"] Dec 03 11:42:32 crc kubenswrapper[5052]: I1203 11:42:32.057337 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-api-73a4-account-create-update-kfmjt"] Dec 03 11:42:32 crc kubenswrapper[5052]: I1203 11:42:32.057346 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell0-db-create-w672x"] Dec 03 11:42:32 crc kubenswrapper[5052]: I1203 11:42:32.057356 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell0-db-create-w672x"] Dec 03 11:42:32 crc kubenswrapper[5052]: I1203 11:42:32.057365 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell0-3d99-account-create-update-kqb94"] Dec 03 11:42:32 crc kubenswrapper[5052]: I1203 11:42:32.057384 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/novacell03d99-account-delete-5gvmz"] Dec 03 11:42:32 crc kubenswrapper[5052]: I1203 11:42:32.057400 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell0-3d99-account-create-update-kqb94"] Dec 03 11:42:32 crc kubenswrapper[5052]: E1203 11:42:32.058384 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-zmn62 operator-scripts], unattached volumes=[], failed to process volumes=[]: context canceled" pod="openstack-kuttl-tests/novacell03d99-account-delete-5gvmz" podUID="f274168d-df51-4f50-b014-1ea4e85370d7" Dec 03 11:42:32 crc kubenswrapper[5052]: E1203 11:42:32.062930 5052 projected.go:194] Error preparing data for projected volume kube-api-access-b6hww for pod openstack-kuttl-tests/keystonec8af-account-delete-xhhcr: failed to fetch token: serviceaccounts "galera-openstack" not found Dec 03 11:42:32 crc kubenswrapper[5052]: E1203 11:42:32.063008 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1fc99a78-bfd8-4fc9-9ff7-bdd5adeaff73-kube-api-access-b6hww podName:1fc99a78-bfd8-4fc9-9ff7-bdd5adeaff73 nodeName:}" failed. No retries permitted until 2025-12-03 11:42:36.062985141 +0000 UTC m=+1346.651226905 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-b6hww" (UniqueName: "kubernetes.io/projected/1fc99a78-bfd8-4fc9-9ff7-bdd5adeaff73-kube-api-access-b6hww") pod "keystonec8af-account-delete-xhhcr" (UID: "1fc99a78-bfd8-4fc9-9ff7-bdd5adeaff73") : failed to fetch token: serviceaccounts "galera-openstack" not found Dec 03 11:42:32 crc kubenswrapper[5052]: I1203 11:42:32.078879 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/placement-6d84cdf894-dqcg9" podStartSLOduration=10.078856863 podStartE2EDuration="10.078856863s" podCreationTimestamp="2025-12-03 11:42:22 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:42:31.282487065 +0000 UTC m=+1341.870728829" watchObservedRunningTime="2025-12-03 11:42:32.078856863 +0000 UTC m=+1342.667098637" Dec 03 11:42:32 crc kubenswrapper[5052]: I1203 11:42:32.122012 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/keystone-db-create-bfh9z"] Dec 03 11:42:32 crc kubenswrapper[5052]: I1203 11:42:32.151906 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kfzp9\" (UniqueName: \"kubernetes.io/projected/f8cdf05e-7081-4542-844d-f76378e8989f-kube-api-access-kfzp9\") pod \"f8cdf05e-7081-4542-844d-f76378e8989f\" (UID: \"f8cdf05e-7081-4542-844d-f76378e8989f\") " Dec 03 11:42:32 crc kubenswrapper[5052]: I1203 11:42:32.152959 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f8cdf05e-7081-4542-844d-f76378e8989f-combined-ca-bundle\") pod \"f8cdf05e-7081-4542-844d-f76378e8989f\" (UID: \"f8cdf05e-7081-4542-844d-f76378e8989f\") " Dec 03 11:42:32 crc kubenswrapper[5052]: I1203 11:42:32.152997 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/f8cdf05e-7081-4542-844d-f76378e8989f-config-data\") pod \"f8cdf05e-7081-4542-844d-f76378e8989f\" (UID: \"f8cdf05e-7081-4542-844d-f76378e8989f\") " Dec 03 11:42:32 crc kubenswrapper[5052]: I1203 11:42:32.157326 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f8cdf05e-7081-4542-844d-f76378e8989f-kube-api-access-kfzp9" (OuterVolumeSpecName: "kube-api-access-kfzp9") pod "f8cdf05e-7081-4542-844d-f76378e8989f" (UID: "f8cdf05e-7081-4542-844d-f76378e8989f"). InnerVolumeSpecName "kube-api-access-kfzp9". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:42:32 crc kubenswrapper[5052]: I1203 11:42:32.166389 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/keystone-db-create-bfh9z"] Dec 03 11:42:32 crc kubenswrapper[5052]: I1203 11:42:32.194347 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/keystone-c8af-account-create-update-fcv24"] Dec 03 11:42:32 crc kubenswrapper[5052]: I1203 11:42:32.200348 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f8cdf05e-7081-4542-844d-f76378e8989f-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "f8cdf05e-7081-4542-844d-f76378e8989f" (UID: "f8cdf05e-7081-4542-844d-f76378e8989f"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:42:32 crc kubenswrapper[5052]: I1203 11:42:32.202653 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/keystone-c8af-account-create-update-fcv24"] Dec 03 11:42:32 crc kubenswrapper[5052]: I1203 11:42:32.210168 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/keystonec8af-account-delete-xhhcr"] Dec 03 11:42:32 crc kubenswrapper[5052]: E1203 11:42:32.210808 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-b6hww operator-scripts], unattached volumes=[], failed to process volumes=[]: context canceled" pod="openstack-kuttl-tests/keystonec8af-account-delete-xhhcr" podUID="1fc99a78-bfd8-4fc9-9ff7-bdd5adeaff73" Dec 03 11:42:32 crc kubenswrapper[5052]: I1203 11:42:32.239075 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f8cdf05e-7081-4542-844d-f76378e8989f-config-data" (OuterVolumeSpecName: "config-data") pod "f8cdf05e-7081-4542-844d-f76378e8989f" (UID: "f8cdf05e-7081-4542-844d-f76378e8989f"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:42:32 crc kubenswrapper[5052]: I1203 11:42:32.257174 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-kfzp9\" (UniqueName: \"kubernetes.io/projected/f8cdf05e-7081-4542-844d-f76378e8989f-kube-api-access-kfzp9\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:32 crc kubenswrapper[5052]: I1203 11:42:32.257213 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f8cdf05e-7081-4542-844d-f76378e8989f-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:32 crc kubenswrapper[5052]: I1203 11:42:32.257233 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/f8cdf05e-7081-4542-844d-f76378e8989f-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:32 crc kubenswrapper[5052]: I1203 11:42:32.322587 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-66f4bb9676-j6vzc" event={"ID":"fe69e54e-ae29-49e2-8a21-0ff81cc0100f","Type":"ContainerDied","Data":"076608a93a4dad7d699f90f3532f41e2ef1cae56396664204466a92cea8df0e7"} Dec 03 11:42:32 crc kubenswrapper[5052]: I1203 11:42:32.322655 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="076608a93a4dad7d699f90f3532f41e2ef1cae56396664204466a92cea8df0e7" Dec 03 11:42:32 crc kubenswrapper[5052]: I1203 11:42:32.324498 5052 generic.go:334] "Generic (PLEG): container finished" podID="32573220-587f-4703-a5e9-0fb67d2019ef" containerID="c04cb50606e320ee3fa81bcf57983f377ac852528a4865911ec99a80eba0fc63" exitCode=1 Dec 03 11:42:32 crc kubenswrapper[5052]: I1203 11:42:32.324566 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement79ca-account-delete-sqtcb" event={"ID":"32573220-587f-4703-a5e9-0fb67d2019ef","Type":"ContainerDied","Data":"c04cb50606e320ee3fa81bcf57983f377ac852528a4865911ec99a80eba0fc63"} Dec 03 11:42:32 crc kubenswrapper[5052]: I1203 11:42:32.326420 5052 generic.go:334] "Generic (PLEG): container finished" podID="0c7d3107-6dd9-4aa2-a392-68a213013508" containerID="c807496164764c8d605198ee634b8fbee67717205c542334178d901158c10093" exitCode=0 Dec 03 11:42:32 crc kubenswrapper[5052]: I1203 11:42:32.326576 5052 generic.go:334] "Generic (PLEG): container finished" podID="0c7d3107-6dd9-4aa2-a392-68a213013508" containerID="f9ff1557270cd7f8f08565a1b152ff021c2e83623d66dbe9dd1606df3722c53a" exitCode=143 Dec 03 11:42:32 crc kubenswrapper[5052]: I1203 11:42:32.326539 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-6d84cdf894-dqcg9" event={"ID":"0c7d3107-6dd9-4aa2-a392-68a213013508","Type":"ContainerDied","Data":"c807496164764c8d605198ee634b8fbee67717205c542334178d901158c10093"} Dec 03 11:42:32 crc kubenswrapper[5052]: I1203 11:42:32.327831 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-6d84cdf894-dqcg9" event={"ID":"0c7d3107-6dd9-4aa2-a392-68a213013508","Type":"ContainerDied","Data":"f9ff1557270cd7f8f08565a1b152ff021c2e83623d66dbe9dd1606df3722c53a"} Dec 03 11:42:32 crc kubenswrapper[5052]: I1203 11:42:32.330523 5052 generic.go:334] "Generic (PLEG): container finished" podID="bd2dc8ee-f6b9-4c8b-af5e-79e67a2d1ef0" containerID="4514b8aae4d859abb11dc204bd75a6fb855f5e1b4b91f3cb73f993ef8101b439" exitCode=1 Dec 03 11:42:32 crc kubenswrapper[5052]: I1203 11:42:32.330789 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutronbfb7-account-delete-ff26t" event={"ID":"bd2dc8ee-f6b9-4c8b-af5e-79e67a2d1ef0","Type":"ContainerDied","Data":"4514b8aae4d859abb11dc204bd75a6fb855f5e1b4b91f3cb73f993ef8101b439"} Dec 03 11:42:32 crc kubenswrapper[5052]: I1203 11:42:32.331332 5052 kubelet_pods.go:1007] "Unable to retrieve pull secret, the image pull may not succeed." pod="openstack-kuttl-tests/neutronbfb7-account-delete-ff26t" secret="" err="secret \"galera-openstack-dockercfg-dfbmj\" not found" Dec 03 11:42:32 crc kubenswrapper[5052]: I1203 11:42:32.331438 5052 scope.go:117] "RemoveContainer" containerID="4514b8aae4d859abb11dc204bd75a6fb855f5e1b4b91f3cb73f993ef8101b439" Dec 03 11:42:32 crc kubenswrapper[5052]: I1203 11:42:32.335367 5052 generic.go:334] "Generic (PLEG): container finished" podID="8c4732f9-0f9e-4600-aed9-6a230cbf7b16" containerID="cc72d236c40c2e7f87295c5f31e63f10a061fd78a90f6c26b95db3911d675ff7" exitCode=0 Dec 03 11:42:32 crc kubenswrapper[5052]: I1203 11:42:32.335441 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-keystone-listener-649457694-r7gzw" event={"ID":"8c4732f9-0f9e-4600-aed9-6a230cbf7b16","Type":"ContainerDied","Data":"cc72d236c40c2e7f87295c5f31e63f10a061fd78a90f6c26b95db3911d675ff7"} Dec 03 11:42:32 crc kubenswrapper[5052]: I1203 11:42:32.343546 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-699ddb8646-rfm4p" event={"ID":"0b4b31e5-e82d-4d44-9c8f-d001484293b6","Type":"ContainerDied","Data":"e192664a9c8e1f0804fec134de74214459efa8fd9b22de65ed888db40bd3cb99"} Dec 03 11:42:32 crc kubenswrapper[5052]: I1203 11:42:32.343754 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e192664a9c8e1f0804fec134de74214459efa8fd9b22de65ed888db40bd3cb99" Dec 03 11:42:32 crc kubenswrapper[5052]: I1203 11:42:32.345617 5052 generic.go:334] "Generic (PLEG): container finished" podID="5737462d-2e5c-4453-9d6c-7d67503555f7" containerID="85486efe955265533bc0eee17583150138fcae470d553a61d68bb5477eb454dd" exitCode=0 Dec 03 11:42:32 crc kubenswrapper[5052]: I1203 11:42:32.345756 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystonec8af-account-delete-xhhcr" Dec 03 11:42:32 crc kubenswrapper[5052]: I1203 11:42:32.345888 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-d6bbfcfb4-xdw2x" event={"ID":"5737462d-2e5c-4453-9d6c-7d67503555f7","Type":"ContainerDied","Data":"85486efe955265533bc0eee17583150138fcae470d553a61d68bb5477eb454dd"} Dec 03 11:42:32 crc kubenswrapper[5052]: I1203 11:42:32.345984 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 11:42:32 crc kubenswrapper[5052]: I1203 11:42:32.351737 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/novacell03d99-account-delete-5gvmz" Dec 03 11:42:32 crc kubenswrapper[5052]: I1203 11:42:32.433606 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/rabbitmq-server-0" podUID="9944d9af-7a7b-4773-8279-72914d63ef5f" containerName="rabbitmq" probeResult="failure" output="dial tcp 10.217.0.100:5671: connect: connection refused" Dec 03 11:42:32 crc kubenswrapper[5052]: E1203 11:42:32.672472 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 11:42:32 crc kubenswrapper[5052]: E1203 11:42:32.672626 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 11:42:32 crc kubenswrapper[5052]: E1203 11:42:32.673171 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8408f908-2fbf-4933-9361-68af849aad18-operator-scripts podName:8408f908-2fbf-4933-9361-68af849aad18 nodeName:}" failed. No retries permitted until 2025-12-03 11:42:36.672596966 +0000 UTC m=+1347.260838780 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/8408f908-2fbf-4933-9361-68af849aad18-operator-scripts") pod "glance78ad-account-delete-7l64w" (UID: "8408f908-2fbf-4933-9361-68af849aad18") : configmap "openstack-scripts" not found Dec 03 11:42:32 crc kubenswrapper[5052]: E1203 11:42:32.673192 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/965f5591-b293-4101-be4a-a5329d88fb69-operator-scripts podName:965f5591-b293-4101-be4a-a5329d88fb69 nodeName:}" failed. No retries permitted until 2025-12-03 11:42:36.673184803 +0000 UTC m=+1347.261426567 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/965f5591-b293-4101-be4a-a5329d88fb69-operator-scripts") pod "novaapi73a4-account-delete-lrzcr" (UID: "965f5591-b293-4101-be4a-a5329d88fb69") : configmap "openstack-scripts" not found Dec 03 11:42:32 crc kubenswrapper[5052]: I1203 11:42:32.735398 5052 scope.go:117] "RemoveContainer" containerID="99cb39696946884496d7905d0018b8b8e349cbda3cdb669d009adf92f668080b" Dec 03 11:42:32 crc kubenswrapper[5052]: E1203 11:42:32.774277 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/rabbitmq-cell1-config-data: configmap "rabbitmq-cell1-config-data" not found Dec 03 11:42:32 crc kubenswrapper[5052]: E1203 11:42:32.774621 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/daa521fc-7e1d-44d2-b7d8-6aab003cfb66-config-data podName:daa521fc-7e1d-44d2-b7d8-6aab003cfb66 nodeName:}" failed. No retries permitted until 2025-12-03 11:42:40.774597834 +0000 UTC m=+1351.362839598 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "config-data" (UniqueName: "kubernetes.io/configmap/daa521fc-7e1d-44d2-b7d8-6aab003cfb66-config-data") pod "rabbitmq-cell1-server-0" (UID: "daa521fc-7e1d-44d2-b7d8-6aab003cfb66") : configmap "rabbitmq-cell1-config-data" not found Dec 03 11:42:32 crc kubenswrapper[5052]: E1203 11:42:32.774322 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 11:42:32 crc kubenswrapper[5052]: E1203 11:42:32.775210 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/bd2dc8ee-f6b9-4c8b-af5e-79e67a2d1ef0-operator-scripts podName:bd2dc8ee-f6b9-4c8b-af5e-79e67a2d1ef0 nodeName:}" failed. No retries permitted until 2025-12-03 11:42:34.775196371 +0000 UTC m=+1345.363438165 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/bd2dc8ee-f6b9-4c8b-af5e-79e67a2d1ef0-operator-scripts") pod "neutronbfb7-account-delete-ff26t" (UID: "bd2dc8ee-f6b9-4c8b-af5e-79e67a2d1ef0") : configmap "openstack-scripts" not found Dec 03 11:42:32 crc kubenswrapper[5052]: I1203 11:42:32.781249 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" podUID="daa521fc-7e1d-44d2-b7d8-6aab003cfb66" containerName="rabbitmq" probeResult="failure" output="dial tcp 10.217.0.101:5671: connect: connection refused" Dec 03 11:42:32 crc kubenswrapper[5052]: I1203 11:42:32.792752 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:42:32 crc kubenswrapper[5052]: I1203 11:42:32.808144 5052 scope.go:117] "RemoveContainer" containerID="9cfabfd91ca0525b54d3922c76017cb0defbc772bca498498fc287ac2175e2aa" Dec 03 11:42:32 crc kubenswrapper[5052]: I1203 11:42:32.875347 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/eae5b719-0017-4621-9cb4-73ed6353648e-config-data\") pod \"eae5b719-0017-4621-9cb4-73ed6353648e\" (UID: \"eae5b719-0017-4621-9cb4-73ed6353648e\") " Dec 03 11:42:32 crc kubenswrapper[5052]: I1203 11:42:32.875395 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/eae5b719-0017-4621-9cb4-73ed6353648e-scripts\") pod \"eae5b719-0017-4621-9cb4-73ed6353648e\" (UID: \"eae5b719-0017-4621-9cb4-73ed6353648e\") " Dec 03 11:42:32 crc kubenswrapper[5052]: I1203 11:42:32.875522 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/eae5b719-0017-4621-9cb4-73ed6353648e-sg-core-conf-yaml\") pod \"eae5b719-0017-4621-9cb4-73ed6353648e\" (UID: \"eae5b719-0017-4621-9cb4-73ed6353648e\") " Dec 03 11:42:32 crc kubenswrapper[5052]: I1203 11:42:32.875613 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/eae5b719-0017-4621-9cb4-73ed6353648e-run-httpd\") pod \"eae5b719-0017-4621-9cb4-73ed6353648e\" (UID: \"eae5b719-0017-4621-9cb4-73ed6353648e\") " Dec 03 11:42:32 crc kubenswrapper[5052]: I1203 11:42:32.875661 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5hz2d\" (UniqueName: \"kubernetes.io/projected/eae5b719-0017-4621-9cb4-73ed6353648e-kube-api-access-5hz2d\") pod \"eae5b719-0017-4621-9cb4-73ed6353648e\" (UID: \"eae5b719-0017-4621-9cb4-73ed6353648e\") " Dec 03 11:42:32 crc kubenswrapper[5052]: I1203 11:42:32.875691 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/eae5b719-0017-4621-9cb4-73ed6353648e-ceilometer-tls-certs\") pod \"eae5b719-0017-4621-9cb4-73ed6353648e\" (UID: \"eae5b719-0017-4621-9cb4-73ed6353648e\") " Dec 03 11:42:32 crc kubenswrapper[5052]: I1203 11:42:32.875724 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/eae5b719-0017-4621-9cb4-73ed6353648e-log-httpd\") pod \"eae5b719-0017-4621-9cb4-73ed6353648e\" (UID: \"eae5b719-0017-4621-9cb4-73ed6353648e\") " Dec 03 11:42:32 crc kubenswrapper[5052]: I1203 11:42:32.875789 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/eae5b719-0017-4621-9cb4-73ed6353648e-combined-ca-bundle\") pod \"eae5b719-0017-4621-9cb4-73ed6353648e\" (UID: \"eae5b719-0017-4621-9cb4-73ed6353648e\") " Dec 03 11:42:32 crc kubenswrapper[5052]: I1203 11:42:32.876183 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eae5b719-0017-4621-9cb4-73ed6353648e-run-httpd" (OuterVolumeSpecName: "run-httpd") pod "eae5b719-0017-4621-9cb4-73ed6353648e" (UID: "eae5b719-0017-4621-9cb4-73ed6353648e"). InnerVolumeSpecName "run-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:42:32 crc kubenswrapper[5052]: E1203 11:42:32.876266 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 11:42:32 crc kubenswrapper[5052]: E1203 11:42:32.876326 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d3e34248-e7e5-4158-8add-165d17bee9ab-operator-scripts podName:d3e34248-e7e5-4158-8add-165d17bee9ab nodeName:}" failed. No retries permitted until 2025-12-03 11:42:34.876308973 +0000 UTC m=+1345.464550737 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/d3e34248-e7e5-4158-8add-165d17bee9ab-operator-scripts") pod "cinder5365-account-delete-shtcv" (UID: "d3e34248-e7e5-4158-8add-165d17bee9ab") : configmap "openstack-scripts" not found Dec 03 11:42:32 crc kubenswrapper[5052]: I1203 11:42:32.876358 5052 reconciler_common.go:293] "Volume detached for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/eae5b719-0017-4621-9cb4-73ed6353648e-run-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:32 crc kubenswrapper[5052]: E1203 11:42:32.876402 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 11:42:32 crc kubenswrapper[5052]: E1203 11:42:32.876435 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/32573220-587f-4703-a5e9-0fb67d2019ef-operator-scripts podName:32573220-587f-4703-a5e9-0fb67d2019ef nodeName:}" failed. No retries permitted until 2025-12-03 11:42:34.876424346 +0000 UTC m=+1345.464666110 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/32573220-587f-4703-a5e9-0fb67d2019ef-operator-scripts") pod "placement79ca-account-delete-sqtcb" (UID: "32573220-587f-4703-a5e9-0fb67d2019ef") : configmap "openstack-scripts" not found Dec 03 11:42:32 crc kubenswrapper[5052]: I1203 11:42:32.877097 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eae5b719-0017-4621-9cb4-73ed6353648e-log-httpd" (OuterVolumeSpecName: "log-httpd") pod "eae5b719-0017-4621-9cb4-73ed6353648e" (UID: "eae5b719-0017-4621-9cb4-73ed6353648e"). InnerVolumeSpecName "log-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:42:32 crc kubenswrapper[5052]: I1203 11:42:32.879372 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eae5b719-0017-4621-9cb4-73ed6353648e-scripts" (OuterVolumeSpecName: "scripts") pod "eae5b719-0017-4621-9cb4-73ed6353648e" (UID: "eae5b719-0017-4621-9cb4-73ed6353648e"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:42:32 crc kubenswrapper[5052]: I1203 11:42:32.880251 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eae5b719-0017-4621-9cb4-73ed6353648e-kube-api-access-5hz2d" (OuterVolumeSpecName: "kube-api-access-5hz2d") pod "eae5b719-0017-4621-9cb4-73ed6353648e" (UID: "eae5b719-0017-4621-9cb4-73ed6353648e"). InnerVolumeSpecName "kube-api-access-5hz2d". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:42:32 crc kubenswrapper[5052]: I1203 11:42:32.909409 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eae5b719-0017-4621-9cb4-73ed6353648e-sg-core-conf-yaml" (OuterVolumeSpecName: "sg-core-conf-yaml") pod "eae5b719-0017-4621-9cb4-73ed6353648e" (UID: "eae5b719-0017-4621-9cb4-73ed6353648e"). InnerVolumeSpecName "sg-core-conf-yaml". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:42:32 crc kubenswrapper[5052]: I1203 11:42:32.918043 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 11:42:32 crc kubenswrapper[5052]: I1203 11:42:32.924113 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eae5b719-0017-4621-9cb4-73ed6353648e-ceilometer-tls-certs" (OuterVolumeSpecName: "ceilometer-tls-certs") pod "eae5b719-0017-4621-9cb4-73ed6353648e" (UID: "eae5b719-0017-4621-9cb4-73ed6353648e"). InnerVolumeSpecName "ceilometer-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:42:32 crc kubenswrapper[5052]: I1203 11:42:32.977252 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/251248ba-578c-49cf-b77e-fc3d5c6cb6fd-galera-tls-certs\") pod \"251248ba-578c-49cf-b77e-fc3d5c6cb6fd\" (UID: \"251248ba-578c-49cf-b77e-fc3d5c6cb6fd\") " Dec 03 11:42:32 crc kubenswrapper[5052]: I1203 11:42:32.977669 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/251248ba-578c-49cf-b77e-fc3d5c6cb6fd-config-data-default\") pod \"251248ba-578c-49cf-b77e-fc3d5c6cb6fd\" (UID: \"251248ba-578c-49cf-b77e-fc3d5c6cb6fd\") " Dec 03 11:42:32 crc kubenswrapper[5052]: I1203 11:42:32.977708 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-scx8f\" (UniqueName: \"kubernetes.io/projected/251248ba-578c-49cf-b77e-fc3d5c6cb6fd-kube-api-access-scx8f\") pod \"251248ba-578c-49cf-b77e-fc3d5c6cb6fd\" (UID: \"251248ba-578c-49cf-b77e-fc3d5c6cb6fd\") " Dec 03 11:42:32 crc kubenswrapper[5052]: I1203 11:42:32.977741 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/251248ba-578c-49cf-b77e-fc3d5c6cb6fd-operator-scripts\") pod \"251248ba-578c-49cf-b77e-fc3d5c6cb6fd\" (UID: \"251248ba-578c-49cf-b77e-fc3d5c6cb6fd\") " Dec 03 11:42:32 crc kubenswrapper[5052]: I1203 11:42:32.977796 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/251248ba-578c-49cf-b77e-fc3d5c6cb6fd-combined-ca-bundle\") pod \"251248ba-578c-49cf-b77e-fc3d5c6cb6fd\" (UID: \"251248ba-578c-49cf-b77e-fc3d5c6cb6fd\") " Dec 03 11:42:32 crc kubenswrapper[5052]: I1203 11:42:32.977895 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"mysql-db\" (UniqueName: \"kubernetes.io/local-volume/local-storage20-crc\") pod \"251248ba-578c-49cf-b77e-fc3d5c6cb6fd\" (UID: \"251248ba-578c-49cf-b77e-fc3d5c6cb6fd\") " Dec 03 11:42:32 crc kubenswrapper[5052]: I1203 11:42:32.978776 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/251248ba-578c-49cf-b77e-fc3d5c6cb6fd-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "251248ba-578c-49cf-b77e-fc3d5c6cb6fd" (UID: "251248ba-578c-49cf-b77e-fc3d5c6cb6fd"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:42:32 crc kubenswrapper[5052]: I1203 11:42:32.979987 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/251248ba-578c-49cf-b77e-fc3d5c6cb6fd-config-data-default" (OuterVolumeSpecName: "config-data-default") pod "251248ba-578c-49cf-b77e-fc3d5c6cb6fd" (UID: "251248ba-578c-49cf-b77e-fc3d5c6cb6fd"). InnerVolumeSpecName "config-data-default". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:42:32 crc kubenswrapper[5052]: I1203 11:42:32.978000 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/251248ba-578c-49cf-b77e-fc3d5c6cb6fd-config-data-generated\") pod \"251248ba-578c-49cf-b77e-fc3d5c6cb6fd\" (UID: \"251248ba-578c-49cf-b77e-fc3d5c6cb6fd\") " Dec 03 11:42:32 crc kubenswrapper[5052]: I1203 11:42:32.980279 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/251248ba-578c-49cf-b77e-fc3d5c6cb6fd-kolla-config\") pod \"251248ba-578c-49cf-b77e-fc3d5c6cb6fd\" (UID: \"251248ba-578c-49cf-b77e-fc3d5c6cb6fd\") " Dec 03 11:42:32 crc kubenswrapper[5052]: I1203 11:42:32.981078 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-5hz2d\" (UniqueName: \"kubernetes.io/projected/eae5b719-0017-4621-9cb4-73ed6353648e-kube-api-access-5hz2d\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:32 crc kubenswrapper[5052]: I1203 11:42:32.981103 5052 reconciler_common.go:293] "Volume detached for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/eae5b719-0017-4621-9cb4-73ed6353648e-ceilometer-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:32 crc kubenswrapper[5052]: I1203 11:42:32.981114 5052 reconciler_common.go:293] "Volume detached for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/eae5b719-0017-4621-9cb4-73ed6353648e-log-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:32 crc kubenswrapper[5052]: I1203 11:42:32.981127 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/251248ba-578c-49cf-b77e-fc3d5c6cb6fd-config-data-default\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:32 crc kubenswrapper[5052]: I1203 11:42:32.981139 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/251248ba-578c-49cf-b77e-fc3d5c6cb6fd-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:32 crc kubenswrapper[5052]: I1203 11:42:32.981151 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/eae5b719-0017-4621-9cb4-73ed6353648e-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:32 crc kubenswrapper[5052]: I1203 11:42:32.981161 5052 reconciler_common.go:293] "Volume detached for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/eae5b719-0017-4621-9cb4-73ed6353648e-sg-core-conf-yaml\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:32 crc kubenswrapper[5052]: I1203 11:42:32.981645 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/251248ba-578c-49cf-b77e-fc3d5c6cb6fd-kolla-config" (OuterVolumeSpecName: "kolla-config") pod "251248ba-578c-49cf-b77e-fc3d5c6cb6fd" (UID: "251248ba-578c-49cf-b77e-fc3d5c6cb6fd"). InnerVolumeSpecName "kolla-config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:42:32 crc kubenswrapper[5052]: I1203 11:42:32.982084 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/251248ba-578c-49cf-b77e-fc3d5c6cb6fd-config-data-generated" (OuterVolumeSpecName: "config-data-generated") pod "251248ba-578c-49cf-b77e-fc3d5c6cb6fd" (UID: "251248ba-578c-49cf-b77e-fc3d5c6cb6fd"). InnerVolumeSpecName "config-data-generated". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:42:32 crc kubenswrapper[5052]: I1203 11:42:32.985899 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/251248ba-578c-49cf-b77e-fc3d5c6cb6fd-kube-api-access-scx8f" (OuterVolumeSpecName: "kube-api-access-scx8f") pod "251248ba-578c-49cf-b77e-fc3d5c6cb6fd" (UID: "251248ba-578c-49cf-b77e-fc3d5c6cb6fd"). InnerVolumeSpecName "kube-api-access-scx8f". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:42:32 crc kubenswrapper[5052]: I1203 11:42:32.991337 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage20-crc" (OuterVolumeSpecName: "mysql-db") pod "251248ba-578c-49cf-b77e-fc3d5c6cb6fd" (UID: "251248ba-578c-49cf-b77e-fc3d5c6cb6fd"). InnerVolumeSpecName "local-storage20-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 11:42:32 crc kubenswrapper[5052]: I1203 11:42:32.993886 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eae5b719-0017-4621-9cb4-73ed6353648e-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "eae5b719-0017-4621-9cb4-73ed6353648e" (UID: "eae5b719-0017-4621-9cb4-73ed6353648e"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.012843 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eae5b719-0017-4621-9cb4-73ed6353648e-config-data" (OuterVolumeSpecName: "config-data") pod "eae5b719-0017-4621-9cb4-73ed6353648e" (UID: "eae5b719-0017-4621-9cb4-73ed6353648e"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.027299 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/251248ba-578c-49cf-b77e-fc3d5c6cb6fd-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "251248ba-578c-49cf-b77e-fc3d5c6cb6fd" (UID: "251248ba-578c-49cf-b77e-fc3d5c6cb6fd"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.042556 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/251248ba-578c-49cf-b77e-fc3d5c6cb6fd-galera-tls-certs" (OuterVolumeSpecName: "galera-tls-certs") pod "251248ba-578c-49cf-b77e-fc3d5c6cb6fd" (UID: "251248ba-578c-49cf-b77e-fc3d5c6cb6fd"). InnerVolumeSpecName "galera-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.088944 5052 reconciler_common.go:293] "Volume detached for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/251248ba-578c-49cf-b77e-fc3d5c6cb6fd-galera-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.088991 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/eae5b719-0017-4621-9cb4-73ed6353648e-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.089005 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-scx8f\" (UniqueName: \"kubernetes.io/projected/251248ba-578c-49cf-b77e-fc3d5c6cb6fd-kube-api-access-scx8f\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.089020 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/eae5b719-0017-4621-9cb4-73ed6353648e-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.089031 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/251248ba-578c-49cf-b77e-fc3d5c6cb6fd-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.089065 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage20-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage20-crc\") on node \"crc\" " Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.089079 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/251248ba-578c-49cf-b77e-fc3d5c6cb6fd-config-data-generated\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.089092 5052 reconciler_common.go:293] "Volume detached for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/251248ba-578c-49cf-b77e-fc3d5c6cb6fd-kolla-config\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.112637 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage20-crc" (UniqueName: "kubernetes.io/local-volume/local-storage20-crc") on node "crc" Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.124917 5052 scope.go:117] "RemoveContainer" containerID="d6cb51da86b5f965cccc6e7f9d1241d74c446a9276dbf42e057cec9147384cb3" Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.189612 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/redhat-operators-bvp4v" Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.193021 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage20-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage20-crc\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.217377 5052 scope.go:117] "RemoveContainer" containerID="b939d536bc53d88cf0c53f3e90f1a9bbe204dc2f74470b14127832658e7f3898" Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.238475 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/swift-proxy-fd9678558-qx8wh"] Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.244956 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-api-66f4bb9676-j6vzc" Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.250478 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/swift-proxy-fd9678558-qx8wh"] Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.259116 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.267872 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.293216 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-699ddb8646-rfm4p" Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.294939 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/18c1ce3d-6d64-433f-a115-9977227455be-utilities\") pod \"18c1ce3d-6d64-433f-a115-9977227455be\" (UID: \"18c1ce3d-6d64-433f-a115-9977227455be\") " Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.296337 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8cvmt\" (UniqueName: \"kubernetes.io/projected/18c1ce3d-6d64-433f-a115-9977227455be-kube-api-access-8cvmt\") pod \"18c1ce3d-6d64-433f-a115-9977227455be\" (UID: \"18c1ce3d-6d64-433f-a115-9977227455be\") " Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.297720 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/18c1ce3d-6d64-433f-a115-9977227455be-utilities" (OuterVolumeSpecName: "utilities") pod "18c1ce3d-6d64-433f-a115-9977227455be" (UID: "18c1ce3d-6d64-433f-a115-9977227455be"). InnerVolumeSpecName "utilities". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.300729 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/18c1ce3d-6d64-433f-a115-9977227455be-catalog-content\") pod \"18c1ce3d-6d64-433f-a115-9977227455be\" (UID: \"18c1ce3d-6d64-433f-a115-9977227455be\") " Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.302009 5052 reconciler_common.go:293] "Volume detached for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/18c1ce3d-6d64-433f-a115-9977227455be-utilities\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.311401 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.313455 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/18c1ce3d-6d64-433f-a115-9977227455be-kube-api-access-8cvmt" (OuterVolumeSpecName: "kube-api-access-8cvmt") pod "18c1ce3d-6d64-433f-a115-9977227455be" (UID: "18c1ce3d-6d64-433f-a115-9977227455be"). InnerVolumeSpecName "kube-api-access-8cvmt". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.314078 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-d6bbfcfb4-xdw2x" Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.320747 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystonec8af-account-delete-xhhcr" Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.321084 5052 scope.go:117] "RemoveContainer" containerID="a5e3cd0978d13d8f638fcff947df03b89089415c5babbbe442713809453149ae" Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.338054 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/novacell03d99-account-delete-5gvmz" Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.338217 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.357914 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-6d84cdf894-dqcg9" Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.366587 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.367988 5052 scope.go:117] "RemoveContainer" containerID="f924c6cd365dded654c74f9f506f8cb2317b25d77d3b15ae01a497a3800a80db" Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.368722 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement79ca-account-delete-sqtcb" Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.376214 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-d6bbfcfb4-xdw2x" event={"ID":"5737462d-2e5c-4453-9d6c-7d67503555f7","Type":"ContainerDied","Data":"70de95a1cec1fd36a398a2fe13eb11bec2874f8487ab6c53add1cdcc12338f59"} Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.376426 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-d6bbfcfb4-xdw2x" Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.385027 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.385450 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-keystone-listener-649457694-r7gzw" Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.387978 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement79ca-account-delete-sqtcb" event={"ID":"32573220-587f-4703-a5e9-0fb67d2019ef","Type":"ContainerDied","Data":"fc5b5266a2b318f1577aca38b5cd24531faf4b17a72d764f030738dce87191c8"} Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.388218 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement79ca-account-delete-sqtcb" Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.398603 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.400368 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.402987 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/5737462d-2e5c-4453-9d6c-7d67503555f7-scripts\") pod \"5737462d-2e5c-4453-9d6c-7d67503555f7\" (UID: \"5737462d-2e5c-4453-9d6c-7d67503555f7\") " Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.403021 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/fe69e54e-ae29-49e2-8a21-0ff81cc0100f-config-data\") pod \"fe69e54e-ae29-49e2-8a21-0ff81cc0100f\" (UID: \"fe69e54e-ae29-49e2-8a21-0ff81cc0100f\") " Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.403080 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/5737462d-2e5c-4453-9d6c-7d67503555f7-credential-keys\") pod \"5737462d-2e5c-4453-9d6c-7d67503555f7\" (UID: \"5737462d-2e5c-4453-9d6c-7d67503555f7\") " Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.403153 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2l2kf\" (UniqueName: \"kubernetes.io/projected/0b4b31e5-e82d-4d44-9c8f-d001484293b6-kube-api-access-2l2kf\") pod \"0b4b31e5-e82d-4d44-9c8f-d001484293b6\" (UID: \"0b4b31e5-e82d-4d44-9c8f-d001484293b6\") " Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.403188 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ksfgw\" (UniqueName: \"kubernetes.io/projected/5737462d-2e5c-4453-9d6c-7d67503555f7-kube-api-access-ksfgw\") pod \"5737462d-2e5c-4453-9d6c-7d67503555f7\" (UID: \"5737462d-2e5c-4453-9d6c-7d67503555f7\") " Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.403212 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/5737462d-2e5c-4453-9d6c-7d67503555f7-internal-tls-certs\") pod \"5737462d-2e5c-4453-9d6c-7d67503555f7\" (UID: \"5737462d-2e5c-4453-9d6c-7d67503555f7\") " Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.403233 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/fe69e54e-ae29-49e2-8a21-0ff81cc0100f-public-tls-certs\") pod \"fe69e54e-ae29-49e2-8a21-0ff81cc0100f\" (UID: \"fe69e54e-ae29-49e2-8a21-0ff81cc0100f\") " Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.403332 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/5737462d-2e5c-4453-9d6c-7d67503555f7-config-data\") pod \"5737462d-2e5c-4453-9d6c-7d67503555f7\" (UID: \"5737462d-2e5c-4453-9d6c-7d67503555f7\") " Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.403395 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9sj25\" (UniqueName: \"kubernetes.io/projected/fe69e54e-ae29-49e2-8a21-0ff81cc0100f-kube-api-access-9sj25\") pod \"fe69e54e-ae29-49e2-8a21-0ff81cc0100f\" (UID: \"fe69e54e-ae29-49e2-8a21-0ff81cc0100f\") " Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.403419 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/0b4b31e5-e82d-4d44-9c8f-d001484293b6-config-data\") pod \"0b4b31e5-e82d-4d44-9c8f-d001484293b6\" (UID: \"0b4b31e5-e82d-4d44-9c8f-d001484293b6\") " Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.403451 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/fe69e54e-ae29-49e2-8a21-0ff81cc0100f-config-data-custom\") pod \"fe69e54e-ae29-49e2-8a21-0ff81cc0100f\" (UID: \"fe69e54e-ae29-49e2-8a21-0ff81cc0100f\") " Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.403502 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/5737462d-2e5c-4453-9d6c-7d67503555f7-public-tls-certs\") pod \"5737462d-2e5c-4453-9d6c-7d67503555f7\" (UID: \"5737462d-2e5c-4453-9d6c-7d67503555f7\") " Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.403554 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/0b4b31e5-e82d-4d44-9c8f-d001484293b6-logs\") pod \"0b4b31e5-e82d-4d44-9c8f-d001484293b6\" (UID: \"0b4b31e5-e82d-4d44-9c8f-d001484293b6\") " Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.403590 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/fe69e54e-ae29-49e2-8a21-0ff81cc0100f-logs\") pod \"fe69e54e-ae29-49e2-8a21-0ff81cc0100f\" (UID: \"fe69e54e-ae29-49e2-8a21-0ff81cc0100f\") " Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.403628 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5737462d-2e5c-4453-9d6c-7d67503555f7-combined-ca-bundle\") pod \"5737462d-2e5c-4453-9d6c-7d67503555f7\" (UID: \"5737462d-2e5c-4453-9d6c-7d67503555f7\") " Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.403657 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/fe69e54e-ae29-49e2-8a21-0ff81cc0100f-internal-tls-certs\") pod \"fe69e54e-ae29-49e2-8a21-0ff81cc0100f\" (UID: \"fe69e54e-ae29-49e2-8a21-0ff81cc0100f\") " Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.403713 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/fe69e54e-ae29-49e2-8a21-0ff81cc0100f-combined-ca-bundle\") pod \"fe69e54e-ae29-49e2-8a21-0ff81cc0100f\" (UID: \"fe69e54e-ae29-49e2-8a21-0ff81cc0100f\") " Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.403747 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/0b4b31e5-e82d-4d44-9c8f-d001484293b6-public-tls-certs\") pod \"0b4b31e5-e82d-4d44-9c8f-d001484293b6\" (UID: \"0b4b31e5-e82d-4d44-9c8f-d001484293b6\") " Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.403795 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/5737462d-2e5c-4453-9d6c-7d67503555f7-fernet-keys\") pod \"5737462d-2e5c-4453-9d6c-7d67503555f7\" (UID: \"5737462d-2e5c-4453-9d6c-7d67503555f7\") " Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.403826 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0b4b31e5-e82d-4d44-9c8f-d001484293b6-combined-ca-bundle\") pod \"0b4b31e5-e82d-4d44-9c8f-d001484293b6\" (UID: \"0b4b31e5-e82d-4d44-9c8f-d001484293b6\") " Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.403857 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/0b4b31e5-e82d-4d44-9c8f-d001484293b6-scripts\") pod \"0b4b31e5-e82d-4d44-9c8f-d001484293b6\" (UID: \"0b4b31e5-e82d-4d44-9c8f-d001484293b6\") " Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.403879 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/0b4b31e5-e82d-4d44-9c8f-d001484293b6-internal-tls-certs\") pod \"0b4b31e5-e82d-4d44-9c8f-d001484293b6\" (UID: \"0b4b31e5-e82d-4d44-9c8f-d001484293b6\") " Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.404497 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-8cvmt\" (UniqueName: \"kubernetes.io/projected/18c1ce3d-6d64-433f-a115-9977227455be-kube-api-access-8cvmt\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.406398 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0b4b31e5-e82d-4d44-9c8f-d001484293b6-logs" (OuterVolumeSpecName: "logs") pod "0b4b31e5-e82d-4d44-9c8f-d001484293b6" (UID: "0b4b31e5-e82d-4d44-9c8f-d001484293b6"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.406409 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe69e54e-ae29-49e2-8a21-0ff81cc0100f-logs" (OuterVolumeSpecName: "logs") pod "fe69e54e-ae29-49e2-8a21-0ff81cc0100f" (UID: "fe69e54e-ae29-49e2-8a21-0ff81cc0100f"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.406567 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-6d84cdf894-dqcg9" event={"ID":"0c7d3107-6dd9-4aa2-a392-68a213013508","Type":"ContainerDied","Data":"55fe708364bcd032d50f6462f0711e8cc4abea2cb54eb4f7e91c5c2b63841a8e"} Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.406644 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-6d84cdf894-dqcg9" Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.412439 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/openstack-cell1-galera-0"] Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.422749 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-keystone-listener-649457694-r7gzw" event={"ID":"8c4732f9-0f9e-4600-aed9-6a230cbf7b16","Type":"ContainerDied","Data":"622135ca2a0b77a1c80989556a4247dde77e57653be6c1afecc46f0f6c35f2bc"} Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.422860 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-keystone-listener-649457694-r7gzw" Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.423574 5052 scope.go:117] "RemoveContainer" containerID="2acdb0814bfda1b6c48a4fde354fd532f86ae71dcaac188c16bee44a794b3713" Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.430887 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystonec8af-account-delete-xhhcr" Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.431785 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/redhat-operators-bvp4v" Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.431955 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-api-66f4bb9676-j6vzc" Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.432099 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/novacell03d99-account-delete-5gvmz" Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.432713 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.434196 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-699ddb8646-rfm4p" Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.435039 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.436644 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/openstack-cell1-galera-0"] Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.449194 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5737462d-2e5c-4453-9d6c-7d67503555f7-kube-api-access-ksfgw" (OuterVolumeSpecName: "kube-api-access-ksfgw") pod "5737462d-2e5c-4453-9d6c-7d67503555f7" (UID: "5737462d-2e5c-4453-9d6c-7d67503555f7"). InnerVolumeSpecName "kube-api-access-ksfgw". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.458550 5052 scope.go:117] "RemoveContainer" containerID="cb0825ceeb51bae53da489a08c6c35ac9d9db71eece046983569823ad7e6cd20" Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.458733 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0b4b31e5-e82d-4d44-9c8f-d001484293b6-scripts" (OuterVolumeSpecName: "scripts") pod "0b4b31e5-e82d-4d44-9c8f-d001484293b6" (UID: "0b4b31e5-e82d-4d44-9c8f-d001484293b6"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.463728 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5737462d-2e5c-4453-9d6c-7d67503555f7-scripts" (OuterVolumeSpecName: "scripts") pod "5737462d-2e5c-4453-9d6c-7d67503555f7" (UID: "5737462d-2e5c-4453-9d6c-7d67503555f7"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.477724 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/kube-state-metrics-0"] Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.484999 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5737462d-2e5c-4453-9d6c-7d67503555f7-credential-keys" (OuterVolumeSpecName: "credential-keys") pod "5737462d-2e5c-4453-9d6c-7d67503555f7" (UID: "5737462d-2e5c-4453-9d6c-7d67503555f7"). InnerVolumeSpecName "credential-keys". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.496551 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0b4b31e5-e82d-4d44-9c8f-d001484293b6-kube-api-access-2l2kf" (OuterVolumeSpecName: "kube-api-access-2l2kf") pod "0b4b31e5-e82d-4d44-9c8f-d001484293b6" (UID: "0b4b31e5-e82d-4d44-9c8f-d001484293b6"). InnerVolumeSpecName "kube-api-access-2l2kf". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.496828 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/kube-state-metrics-0"] Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.501221 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fe69e54e-ae29-49e2-8a21-0ff81cc0100f-kube-api-access-9sj25" (OuterVolumeSpecName: "kube-api-access-9sj25") pod "fe69e54e-ae29-49e2-8a21-0ff81cc0100f" (UID: "fe69e54e-ae29-49e2-8a21-0ff81cc0100f"). InnerVolumeSpecName "kube-api-access-9sj25". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.503083 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5737462d-2e5c-4453-9d6c-7d67503555f7-fernet-keys" (OuterVolumeSpecName: "fernet-keys") pod "5737462d-2e5c-4453-9d6c-7d67503555f7" (UID: "5737462d-2e5c-4453-9d6c-7d67503555f7"). InnerVolumeSpecName "fernet-keys". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.505943 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8c4732f9-0f9e-4600-aed9-6a230cbf7b16-combined-ca-bundle\") pod \"8c4732f9-0f9e-4600-aed9-6a230cbf7b16\" (UID: \"8c4732f9-0f9e-4600-aed9-6a230cbf7b16\") " Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.506038 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/8c4732f9-0f9e-4600-aed9-6a230cbf7b16-config-data\") pod \"8c4732f9-0f9e-4600-aed9-6a230cbf7b16\" (UID: \"8c4732f9-0f9e-4600-aed9-6a230cbf7b16\") " Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.506216 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0c7d3107-6dd9-4aa2-a392-68a213013508-combined-ca-bundle\") pod \"0c7d3107-6dd9-4aa2-a392-68a213013508\" (UID: \"0c7d3107-6dd9-4aa2-a392-68a213013508\") " Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.506277 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m8jvk\" (UniqueName: \"kubernetes.io/projected/32573220-587f-4703-a5e9-0fb67d2019ef-kube-api-access-m8jvk\") pod \"32573220-587f-4703-a5e9-0fb67d2019ef\" (UID: \"32573220-587f-4703-a5e9-0fb67d2019ef\") " Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.506298 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/0c7d3107-6dd9-4aa2-a392-68a213013508-config-data\") pod \"0c7d3107-6dd9-4aa2-a392-68a213013508\" (UID: \"0c7d3107-6dd9-4aa2-a392-68a213013508\") " Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.506318 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/32573220-587f-4703-a5e9-0fb67d2019ef-operator-scripts\") pod \"32573220-587f-4703-a5e9-0fb67d2019ef\" (UID: \"32573220-587f-4703-a5e9-0fb67d2019ef\") " Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.506345 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wrjxc\" (UniqueName: \"kubernetes.io/projected/0c7d3107-6dd9-4aa2-a392-68a213013508-kube-api-access-wrjxc\") pod \"0c7d3107-6dd9-4aa2-a392-68a213013508\" (UID: \"0c7d3107-6dd9-4aa2-a392-68a213013508\") " Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.506394 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/0c7d3107-6dd9-4aa2-a392-68a213013508-scripts\") pod \"0c7d3107-6dd9-4aa2-a392-68a213013508\" (UID: \"0c7d3107-6dd9-4aa2-a392-68a213013508\") " Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.506437 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/0c7d3107-6dd9-4aa2-a392-68a213013508-public-tls-certs\") pod \"0c7d3107-6dd9-4aa2-a392-68a213013508\" (UID: \"0c7d3107-6dd9-4aa2-a392-68a213013508\") " Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.506704 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/8c4732f9-0f9e-4600-aed9-6a230cbf7b16-logs\") pod \"8c4732f9-0f9e-4600-aed9-6a230cbf7b16\" (UID: \"8c4732f9-0f9e-4600-aed9-6a230cbf7b16\") " Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.506753 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/0c7d3107-6dd9-4aa2-a392-68a213013508-internal-tls-certs\") pod \"0c7d3107-6dd9-4aa2-a392-68a213013508\" (UID: \"0c7d3107-6dd9-4aa2-a392-68a213013508\") " Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.506795 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l8572\" (UniqueName: \"kubernetes.io/projected/8c4732f9-0f9e-4600-aed9-6a230cbf7b16-kube-api-access-l8572\") pod \"8c4732f9-0f9e-4600-aed9-6a230cbf7b16\" (UID: \"8c4732f9-0f9e-4600-aed9-6a230cbf7b16\") " Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.506835 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/0c7d3107-6dd9-4aa2-a392-68a213013508-logs\") pod \"0c7d3107-6dd9-4aa2-a392-68a213013508\" (UID: \"0c7d3107-6dd9-4aa2-a392-68a213013508\") " Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.506861 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/8c4732f9-0f9e-4600-aed9-6a230cbf7b16-config-data-custom\") pod \"8c4732f9-0f9e-4600-aed9-6a230cbf7b16\" (UID: \"8c4732f9-0f9e-4600-aed9-6a230cbf7b16\") " Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.508533 5052 reconciler_common.go:293] "Volume detached for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/5737462d-2e5c-4453-9d6c-7d67503555f7-fernet-keys\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.508548 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/0b4b31e5-e82d-4d44-9c8f-d001484293b6-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.508558 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/5737462d-2e5c-4453-9d6c-7d67503555f7-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.508568 5052 reconciler_common.go:293] "Volume detached for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/5737462d-2e5c-4453-9d6c-7d67503555f7-credential-keys\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.508578 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-2l2kf\" (UniqueName: \"kubernetes.io/projected/0b4b31e5-e82d-4d44-9c8f-d001484293b6-kube-api-access-2l2kf\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.508590 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-ksfgw\" (UniqueName: \"kubernetes.io/projected/5737462d-2e5c-4453-9d6c-7d67503555f7-kube-api-access-ksfgw\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.508602 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-9sj25\" (UniqueName: \"kubernetes.io/projected/fe69e54e-ae29-49e2-8a21-0ff81cc0100f-kube-api-access-9sj25\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.508612 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/0b4b31e5-e82d-4d44-9c8f-d001484293b6-logs\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.508625 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/fe69e54e-ae29-49e2-8a21-0ff81cc0100f-logs\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.512818 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c7d3107-6dd9-4aa2-a392-68a213013508-logs" (OuterVolumeSpecName: "logs") pod "0c7d3107-6dd9-4aa2-a392-68a213013508" (UID: "0c7d3107-6dd9-4aa2-a392-68a213013508"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.513765 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/32573220-587f-4703-a5e9-0fb67d2019ef-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "32573220-587f-4703-a5e9-0fb67d2019ef" (UID: "32573220-587f-4703-a5e9-0fb67d2019ef"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.516305 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8c4732f9-0f9e-4600-aed9-6a230cbf7b16-logs" (OuterVolumeSpecName: "logs") pod "8c4732f9-0f9e-4600-aed9-6a230cbf7b16" (UID: "8c4732f9-0f9e-4600-aed9-6a230cbf7b16"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.531719 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fe69e54e-ae29-49e2-8a21-0ff81cc0100f-config-data-custom" (OuterVolumeSpecName: "config-data-custom") pod "fe69e54e-ae29-49e2-8a21-0ff81cc0100f" (UID: "fe69e54e-ae29-49e2-8a21-0ff81cc0100f"). InnerVolumeSpecName "config-data-custom". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.535955 5052 scope.go:117] "RemoveContainer" containerID="def28b7790031f83be9c79b54dbbc74332f5e5917d604dfb281ac32589501053" Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.539147 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.554181 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.583856 5052 scope.go:117] "RemoveContainer" containerID="3132bd07375ba7c18c7bbb9921d64b3f404ab0c85abce7e27085e92e0ba00bea" Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.603867 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-0"] Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.605296 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8c4732f9-0f9e-4600-aed9-6a230cbf7b16-kube-api-access-l8572" (OuterVolumeSpecName: "kube-api-access-l8572") pod "8c4732f9-0f9e-4600-aed9-6a230cbf7b16" (UID: "8c4732f9-0f9e-4600-aed9-6a230cbf7b16"). InnerVolumeSpecName "kube-api-access-l8572". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.608812 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0c7d3107-6dd9-4aa2-a392-68a213013508-scripts" (OuterVolumeSpecName: "scripts") pod "0c7d3107-6dd9-4aa2-a392-68a213013508" (UID: "0c7d3107-6dd9-4aa2-a392-68a213013508"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.609208 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/32573220-587f-4703-a5e9-0fb67d2019ef-kube-api-access-m8jvk" (OuterVolumeSpecName: "kube-api-access-m8jvk") pod "32573220-587f-4703-a5e9-0fb67d2019ef" (UID: "32573220-587f-4703-a5e9-0fb67d2019ef"). InnerVolumeSpecName "kube-api-access-m8jvk". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.609566 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m8jvk\" (UniqueName: \"kubernetes.io/projected/32573220-587f-4703-a5e9-0fb67d2019ef-kube-api-access-m8jvk\") pod \"32573220-587f-4703-a5e9-0fb67d2019ef\" (UID: \"32573220-587f-4703-a5e9-0fb67d2019ef\") " Dec 03 11:42:33 crc kubenswrapper[5052]: W1203 11:42:33.609806 5052 empty_dir.go:500] Warning: Unmount skipped because path does not exist: /var/lib/kubelet/pods/32573220-587f-4703-a5e9-0fb67d2019ef/volumes/kubernetes.io~projected/kube-api-access-m8jvk Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.609890 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/32573220-587f-4703-a5e9-0fb67d2019ef-kube-api-access-m8jvk" (OuterVolumeSpecName: "kube-api-access-m8jvk") pod "32573220-587f-4703-a5e9-0fb67d2019ef" (UID: "32573220-587f-4703-a5e9-0fb67d2019ef"). InnerVolumeSpecName "kube-api-access-m8jvk". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.610498 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/fe69e54e-ae29-49e2-8a21-0ff81cc0100f-config-data-custom\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.610554 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/8c4732f9-0f9e-4600-aed9-6a230cbf7b16-logs\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.610567 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-l8572\" (UniqueName: \"kubernetes.io/projected/8c4732f9-0f9e-4600-aed9-6a230cbf7b16-kube-api-access-l8572\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.610855 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/0c7d3107-6dd9-4aa2-a392-68a213013508-logs\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.610870 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-m8jvk\" (UniqueName: \"kubernetes.io/projected/32573220-587f-4703-a5e9-0fb67d2019ef-kube-api-access-m8jvk\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.610883 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/32573220-587f-4703-a5e9-0fb67d2019ef-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.610894 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/0c7d3107-6dd9-4aa2-a392-68a213013508-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.610995 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0c7d3107-6dd9-4aa2-a392-68a213013508-kube-api-access-wrjxc" (OuterVolumeSpecName: "kube-api-access-wrjxc") pod "0c7d3107-6dd9-4aa2-a392-68a213013508" (UID: "0c7d3107-6dd9-4aa2-a392-68a213013508"). InnerVolumeSpecName "kube-api-access-wrjxc". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.616040 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8c4732f9-0f9e-4600-aed9-6a230cbf7b16-config-data-custom" (OuterVolumeSpecName: "config-data-custom") pod "8c4732f9-0f9e-4600-aed9-6a230cbf7b16" (UID: "8c4732f9-0f9e-4600-aed9-6a230cbf7b16"). InnerVolumeSpecName "config-data-custom". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.618919 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-0"] Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.637088 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-0"] Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.637564 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-0"] Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.659693 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/18c1ce3d-6d64-433f-a115-9977227455be-catalog-content" (OuterVolumeSpecName: "catalog-content") pod "18c1ce3d-6d64-433f-a115-9977227455be" (UID: "18c1ce3d-6d64-433f-a115-9977227455be"). InnerVolumeSpecName "catalog-content". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.670151 5052 scope.go:117] "RemoveContainer" containerID="6c977dab718207e18c944877c552253d8c5723de1e4d496e75dea068be83b234" Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.700478 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0cbd53a9-fe89-4b73-b56f-d54b8fec2230" path="/var/lib/kubelet/pods/0cbd53a9-fe89-4b73-b56f-d54b8fec2230/volumes" Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.700990 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="14e9bff5-66d5-4808-8446-cf180d759208" path="/var/lib/kubelet/pods/14e9bff5-66d5-4808-8446-cf180d759208/volumes" Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.701579 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="23eed8d3-3d02-4f02-9cde-7e5dd4363b9b" path="/var/lib/kubelet/pods/23eed8d3-3d02-4f02-9cde-7e5dd4363b9b/volumes" Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.702762 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2501eda8-a78d-4609-a465-761e89bb291e" path="/var/lib/kubelet/pods/2501eda8-a78d-4609-a465-761e89bb291e/volumes" Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.703320 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="26a22bd0-90b3-4fe9-8029-a56c6c7bcdfd" path="/var/lib/kubelet/pods/26a22bd0-90b3-4fe9-8029-a56c6c7bcdfd/volumes" Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.703825 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2ff66059-68e1-4c42-8bde-6c599a549ad5" path="/var/lib/kubelet/pods/2ff66059-68e1-4c42-8bde-6c599a549ad5/volumes" Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.712557 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="33a7c4f9-6f64-4bf9-b0fc-419117a03f9d" path="/var/lib/kubelet/pods/33a7c4f9-6f64-4bf9-b0fc-419117a03f9d/volumes" Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.712696 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/cinder-api-0" podUID="ee46c3a4-5e21-4f6b-a944-8cf96d87506e" containerName="cinder-api" probeResult="failure" output="Get \"https://10.217.0.148:8776/healthcheck\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.713671 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="63477217-d60a-4fd8-a58c-5b37cd2cd23f" path="/var/lib/kubelet/pods/63477217-d60a-4fd8-a58c-5b37cd2cd23f/volumes" Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.714314 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6410c857-4e7d-486c-be0a-6734565014d4" path="/var/lib/kubelet/pods/6410c857-4e7d-486c-be0a-6734565014d4/volumes" Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.714674 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/8c4732f9-0f9e-4600-aed9-6a230cbf7b16-config-data-custom\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.715243 5052 reconciler_common.go:293] "Volume detached for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/18c1ce3d-6d64-433f-a115-9977227455be-catalog-content\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.715280 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-wrjxc\" (UniqueName: \"kubernetes.io/projected/0c7d3107-6dd9-4aa2-a392-68a213013508-kube-api-access-wrjxc\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.715611 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="665a736c-d550-4160-a8bb-619d1d5ea826" path="/var/lib/kubelet/pods/665a736c-d550-4160-a8bb-619d1d5ea826/volumes" Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.716569 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="69fe8db0-25e9-4f48-b9a2-77dc31224a34" path="/var/lib/kubelet/pods/69fe8db0-25e9-4f48-b9a2-77dc31224a34/volumes" Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.717233 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="75e4b34e-f7b8-4209-95a6-6cc45f6eb57c" path="/var/lib/kubelet/pods/75e4b34e-f7b8-4209-95a6-6cc45f6eb57c/volumes" Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.718397 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="801b412e-0431-4a95-9639-93eeb8809c75" path="/var/lib/kubelet/pods/801b412e-0431-4a95-9639-93eeb8809c75/volumes" Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.718963 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8b465e38-7dbb-4d02-bce0-28ae0145520c" path="/var/lib/kubelet/pods/8b465e38-7dbb-4d02-bce0-28ae0145520c/volumes" Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.719431 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="996bddd2-d361-4a2f-aa3c-773b37ad9b59" path="/var/lib/kubelet/pods/996bddd2-d361-4a2f-aa3c-773b37ad9b59/volumes" Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.720002 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a53c0da6-8b05-40fd-90c7-78cfb701780a" path="/var/lib/kubelet/pods/a53c0da6-8b05-40fd-90c7-78cfb701780a/volumes" Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.722294 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c0ef8371-9171-45c5-9202-4d16f774ecf4" path="/var/lib/kubelet/pods/c0ef8371-9171-45c5-9202-4d16f774ecf4/volumes" Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.722953 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c3452ae3-e803-4020-b5db-3ddbdf4cba75" path="/var/lib/kubelet/pods/c3452ae3-e803-4020-b5db-3ddbdf4cba75/volumes" Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.725072 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d80d1476-351a-40c6-9f76-ce66d623df74" path="/var/lib/kubelet/pods/d80d1476-351a-40c6-9f76-ce66d623df74/volumes" Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.725994 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ec205491-fbcd-43b5-88d3-d98ae0160f15" path="/var/lib/kubelet/pods/ec205491-fbcd-43b5-88d3-d98ae0160f15/volumes" Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.727733 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ecbf8b8e-cdb4-45c3-9455-d0dbc551356e" path="/var/lib/kubelet/pods/ecbf8b8e-cdb4-45c3-9455-d0dbc551356e/volumes" Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.728626 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f4e66dc5-2979-4ac3-b5a1-959f36db75da" path="/var/lib/kubelet/pods/f4e66dc5-2979-4ac3-b5a1-959f36db75da/volumes" Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.729133 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f8cdf05e-7081-4542-844d-f76378e8989f" path="/var/lib/kubelet/pods/f8cdf05e-7081-4542-844d-f76378e8989f/volumes" Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.730225 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f98f227a-397b-4665-bf18-e00b105ce058" path="/var/lib/kubelet/pods/f98f227a-397b-4665-bf18-e00b105ce058/volumes" Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.731083 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fbef0d42-0ae8-4e90-bacb-081a4b1e8d47" path="/var/lib/kubelet/pods/fbef0d42-0ae8-4e90-bacb-081a4b1e8d47/volumes" Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.731676 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fd14a5c0-3402-4891-a726-32d75b1b7857" path="/var/lib/kubelet/pods/fd14a5c0-3402-4891-a726-32d75b1b7857/volumes" Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.786280 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5737462d-2e5c-4453-9d6c-7d67503555f7-config-data" (OuterVolumeSpecName: "config-data") pod "5737462d-2e5c-4453-9d6c-7d67503555f7" (UID: "5737462d-2e5c-4453-9d6c-7d67503555f7"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.792860 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fe69e54e-ae29-49e2-8a21-0ff81cc0100f-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "fe69e54e-ae29-49e2-8a21-0ff81cc0100f" (UID: "fe69e54e-ae29-49e2-8a21-0ff81cc0100f"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.815199 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fe69e54e-ae29-49e2-8a21-0ff81cc0100f-config-data" (OuterVolumeSpecName: "config-data") pod "fe69e54e-ae29-49e2-8a21-0ff81cc0100f" (UID: "fe69e54e-ae29-49e2-8a21-0ff81cc0100f"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.816598 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/fe69e54e-ae29-49e2-8a21-0ff81cc0100f-config-data\") pod \"fe69e54e-ae29-49e2-8a21-0ff81cc0100f\" (UID: \"fe69e54e-ae29-49e2-8a21-0ff81cc0100f\") " Dec 03 11:42:33 crc kubenswrapper[5052]: W1203 11:42:33.816831 5052 empty_dir.go:500] Warning: Unmount skipped because path does not exist: /var/lib/kubelet/pods/fe69e54e-ae29-49e2-8a21-0ff81cc0100f/volumes/kubernetes.io~secret/config-data Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.816913 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fe69e54e-ae29-49e2-8a21-0ff81cc0100f-config-data" (OuterVolumeSpecName: "config-data") pod "fe69e54e-ae29-49e2-8a21-0ff81cc0100f" (UID: "fe69e54e-ae29-49e2-8a21-0ff81cc0100f"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.820316 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/fe69e54e-ae29-49e2-8a21-0ff81cc0100f-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.820345 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/fe69e54e-ae29-49e2-8a21-0ff81cc0100f-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.820357 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/5737462d-2e5c-4453-9d6c-7d67503555f7-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.829503 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5737462d-2e5c-4453-9d6c-7d67503555f7-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "5737462d-2e5c-4453-9d6c-7d67503555f7" (UID: "5737462d-2e5c-4453-9d6c-7d67503555f7"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.829709 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8c4732f9-0f9e-4600-aed9-6a230cbf7b16-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "8c4732f9-0f9e-4600-aed9-6a230cbf7b16" (UID: "8c4732f9-0f9e-4600-aed9-6a230cbf7b16"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.842117 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0c7d3107-6dd9-4aa2-a392-68a213013508-config-data" (OuterVolumeSpecName: "config-data") pod "0c7d3107-6dd9-4aa2-a392-68a213013508" (UID: "0c7d3107-6dd9-4aa2-a392-68a213013508"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.873304 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fe69e54e-ae29-49e2-8a21-0ff81cc0100f-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "fe69e54e-ae29-49e2-8a21-0ff81cc0100f" (UID: "fe69e54e-ae29-49e2-8a21-0ff81cc0100f"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.880291 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fe69e54e-ae29-49e2-8a21-0ff81cc0100f-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "fe69e54e-ae29-49e2-8a21-0ff81cc0100f" (UID: "fe69e54e-ae29-49e2-8a21-0ff81cc0100f"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.892934 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0c7d3107-6dd9-4aa2-a392-68a213013508-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "0c7d3107-6dd9-4aa2-a392-68a213013508" (UID: "0c7d3107-6dd9-4aa2-a392-68a213013508"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.897824 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0b4b31e5-e82d-4d44-9c8f-d001484293b6-config-data" (OuterVolumeSpecName: "config-data") pod "0b4b31e5-e82d-4d44-9c8f-d001484293b6" (UID: "0b4b31e5-e82d-4d44-9c8f-d001484293b6"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.898647 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5737462d-2e5c-4453-9d6c-7d67503555f7-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "5737462d-2e5c-4453-9d6c-7d67503555f7" (UID: "5737462d-2e5c-4453-9d6c-7d67503555f7"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.908617 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8c4732f9-0f9e-4600-aed9-6a230cbf7b16-config-data" (OuterVolumeSpecName: "config-data") pod "8c4732f9-0f9e-4600-aed9-6a230cbf7b16" (UID: "8c4732f9-0f9e-4600-aed9-6a230cbf7b16"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.914771 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5737462d-2e5c-4453-9d6c-7d67503555f7-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "5737462d-2e5c-4453-9d6c-7d67503555f7" (UID: "5737462d-2e5c-4453-9d6c-7d67503555f7"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.915667 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0b4b31e5-e82d-4d44-9c8f-d001484293b6-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "0b4b31e5-e82d-4d44-9c8f-d001484293b6" (UID: "0b4b31e5-e82d-4d44-9c8f-d001484293b6"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.925001 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0b4b31e5-e82d-4d44-9c8f-d001484293b6-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.925296 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8c4732f9-0f9e-4600-aed9-6a230cbf7b16-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.925382 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/8c4732f9-0f9e-4600-aed9-6a230cbf7b16-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.925456 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0c7d3107-6dd9-4aa2-a392-68a213013508-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.925548 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/5737462d-2e5c-4453-9d6c-7d67503555f7-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.925634 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/fe69e54e-ae29-49e2-8a21-0ff81cc0100f-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.925720 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/0c7d3107-6dd9-4aa2-a392-68a213013508-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.925922 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/0b4b31e5-e82d-4d44-9c8f-d001484293b6-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.925999 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/5737462d-2e5c-4453-9d6c-7d67503555f7-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.926080 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5737462d-2e5c-4453-9d6c-7d67503555f7-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.926157 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/fe69e54e-ae29-49e2-8a21-0ff81cc0100f-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.934777 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0b4b31e5-e82d-4d44-9c8f-d001484293b6-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "0b4b31e5-e82d-4d44-9c8f-d001484293b6" (UID: "0b4b31e5-e82d-4d44-9c8f-d001484293b6"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.936606 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0c7d3107-6dd9-4aa2-a392-68a213013508-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "0c7d3107-6dd9-4aa2-a392-68a213013508" (UID: "0c7d3107-6dd9-4aa2-a392-68a213013508"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.967100 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0c7d3107-6dd9-4aa2-a392-68a213013508-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "0c7d3107-6dd9-4aa2-a392-68a213013508" (UID: "0c7d3107-6dd9-4aa2-a392-68a213013508"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.973828 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0b4b31e5-e82d-4d44-9c8f-d001484293b6-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "0b4b31e5-e82d-4d44-9c8f-d001484293b6" (UID: "0b4b31e5-e82d-4d44-9c8f-d001484293b6"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.993135 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.993505 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.993546 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.994646 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.994686 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/novacell03d99-account-delete-5gvmz"] Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.994699 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/novacell03d99-account-delete-5gvmz"] Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.994720 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/keystonec8af-account-delete-xhhcr"] Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.994734 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/keystonec8af-account-delete-xhhcr"] Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.994748 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/openstack-galera-0"] Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.994762 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/openstack-galera-0"] Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.994774 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:42:33 crc kubenswrapper[5052]: I1203 11:42:33.994787 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:42:34 crc kubenswrapper[5052]: I1203 11:42:34.025727 5052 scope.go:117] "RemoveContainer" containerID="b875595b2b281c94c97104d6ea786f06ea56deeebb4024873ed0f82dba8c7b52" Dec 03 11:42:34 crc kubenswrapper[5052]: I1203 11:42:34.035069 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/0b4b31e5-e82d-4d44-9c8f-d001484293b6-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:34 crc kubenswrapper[5052]: I1203 11:42:34.035099 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/0c7d3107-6dd9-4aa2-a392-68a213013508-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:34 crc kubenswrapper[5052]: I1203 11:42:34.035111 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/0c7d3107-6dd9-4aa2-a392-68a213013508-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:34 crc kubenswrapper[5052]: I1203 11:42:34.035124 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/0b4b31e5-e82d-4d44-9c8f-d001484293b6-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:34 crc kubenswrapper[5052]: I1203 11:42:34.136832 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/f274168d-df51-4f50-b014-1ea4e85370d7-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:34 crc kubenswrapper[5052]: I1203 11:42:34.137109 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-b6hww\" (UniqueName: \"kubernetes.io/projected/1fc99a78-bfd8-4fc9-9ff7-bdd5adeaff73-kube-api-access-b6hww\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:34 crc kubenswrapper[5052]: I1203 11:42:34.137181 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-zmn62\" (UniqueName: \"kubernetes.io/projected/f274168d-df51-4f50-b014-1ea4e85370d7-kube-api-access-zmn62\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:34 crc kubenswrapper[5052]: I1203 11:42:34.137244 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/1fc99a78-bfd8-4fc9-9ff7-bdd5adeaff73-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:34 crc kubenswrapper[5052]: I1203 11:42:34.180428 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/nova-metadata-0" podUID="f98f227a-397b-4665-bf18-e00b105ce058" containerName="nova-metadata-log" probeResult="failure" output="Get \"https://10.217.0.185:8775/\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Dec 03 11:42:34 crc kubenswrapper[5052]: I1203 11:42:34.180439 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/nova-metadata-0" podUID="f98f227a-397b-4665-bf18-e00b105ce058" containerName="nova-metadata-metadata" probeResult="failure" output="Get \"https://10.217.0.185:8775/\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Dec 03 11:42:34 crc kubenswrapper[5052]: I1203 11:42:34.218916 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/placement79ca-account-delete-sqtcb"] Dec 03 11:42:34 crc kubenswrapper[5052]: I1203 11:42:34.236741 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/placement79ca-account-delete-sqtcb"] Dec 03 11:42:34 crc kubenswrapper[5052]: I1203 11:42:34.236834 5052 scope.go:117] "RemoveContainer" containerID="42f4b7304c54076bcbb388fc8df0720f8096981812e6c2b3d22518ea7235934e" Dec 03 11:42:34 crc kubenswrapper[5052]: I1203 11:42:34.249195 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openshift-marketplace/redhat-operators-bvp4v"] Dec 03 11:42:34 crc kubenswrapper[5052]: I1203 11:42:34.259980 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openshift-marketplace/redhat-operators-bvp4v"] Dec 03 11:42:34 crc kubenswrapper[5052]: I1203 11:42:34.277585 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/keystone-d6bbfcfb4-xdw2x"] Dec 03 11:42:34 crc kubenswrapper[5052]: I1203 11:42:34.286737 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/keystone-d6bbfcfb4-xdw2x"] Dec 03 11:42:34 crc kubenswrapper[5052]: I1203 11:42:34.296006 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-keystone-listener-649457694-r7gzw"] Dec 03 11:42:34 crc kubenswrapper[5052]: I1203 11:42:34.301481 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/barbican-keystone-listener-649457694-r7gzw"] Dec 03 11:42:34 crc kubenswrapper[5052]: I1203 11:42:34.317765 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/placement-6d84cdf894-dqcg9"] Dec 03 11:42:34 crc kubenswrapper[5052]: E1203 11:42:34.343392 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/rabbitmq-config-data: configmap "rabbitmq-config-data" not found Dec 03 11:42:34 crc kubenswrapper[5052]: E1203 11:42:34.343472 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9944d9af-7a7b-4773-8279-72914d63ef5f-config-data podName:9944d9af-7a7b-4773-8279-72914d63ef5f nodeName:}" failed. No retries permitted until 2025-12-03 11:42:42.343448446 +0000 UTC m=+1352.931690210 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "config-data" (UniqueName: "kubernetes.io/configmap/9944d9af-7a7b-4773-8279-72914d63ef5f-config-data") pod "rabbitmq-server-0" (UID: "9944d9af-7a7b-4773-8279-72914d63ef5f") : configmap "rabbitmq-config-data" not found Dec 03 11:42:34 crc kubenswrapper[5052]: I1203 11:42:34.349040 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/placement-6d84cdf894-dqcg9"] Dec 03 11:42:34 crc kubenswrapper[5052]: I1203 11:42:34.364825 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-api-66f4bb9676-j6vzc"] Dec 03 11:42:34 crc kubenswrapper[5052]: I1203 11:42:34.374592 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/barbican-api-66f4bb9676-j6vzc"] Dec 03 11:42:34 crc kubenswrapper[5052]: I1203 11:42:34.379478 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/placement-699ddb8646-rfm4p"] Dec 03 11:42:34 crc kubenswrapper[5052]: I1203 11:42:34.386642 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/placement-699ddb8646-rfm4p"] Dec 03 11:42:34 crc kubenswrapper[5052]: I1203 11:42:34.398755 5052 scope.go:117] "RemoveContainer" containerID="85486efe955265533bc0eee17583150138fcae470d553a61d68bb5477eb454dd" Dec 03 11:42:34 crc kubenswrapper[5052]: I1203 11:42:34.422036 5052 scope.go:117] "RemoveContainer" containerID="c04cb50606e320ee3fa81bcf57983f377ac852528a4865911ec99a80eba0fc63" Dec 03 11:42:34 crc kubenswrapper[5052]: I1203 11:42:34.443610 5052 generic.go:334] "Generic (PLEG): container finished" podID="8408f908-2fbf-4933-9361-68af849aad18" containerID="aeb430a10cf9a53281e8c5bdd1de92d30dc2d95fdde0ad7b668d5ced9af8b243" exitCode=1 Dec 03 11:42:34 crc kubenswrapper[5052]: I1203 11:42:34.443678 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance78ad-account-delete-7l64w" event={"ID":"8408f908-2fbf-4933-9361-68af849aad18","Type":"ContainerDied","Data":"aeb430a10cf9a53281e8c5bdd1de92d30dc2d95fdde0ad7b668d5ced9af8b243"} Dec 03 11:42:34 crc kubenswrapper[5052]: I1203 11:42:34.448366 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/novaapi73a4-account-delete-lrzcr" event={"ID":"965f5591-b293-4101-be4a-a5329d88fb69","Type":"ContainerStarted","Data":"9e46a517bfb791f3665b76fc9e3c2c9c89f643eb87f5ee717db97d31c9a3d66a"} Dec 03 11:42:34 crc kubenswrapper[5052]: I1203 11:42:34.448549 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/novaapi73a4-account-delete-lrzcr" podUID="965f5591-b293-4101-be4a-a5329d88fb69" containerName="mariadb-account-delete" containerID="cri-o://9e46a517bfb791f3665b76fc9e3c2c9c89f643eb87f5ee717db97d31c9a3d66a" gracePeriod=30 Dec 03 11:42:34 crc kubenswrapper[5052]: I1203 11:42:34.448739 5052 scope.go:117] "RemoveContainer" containerID="c807496164764c8d605198ee634b8fbee67717205c542334178d901158c10093" Dec 03 11:42:34 crc kubenswrapper[5052]: I1203 11:42:34.453160 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbicanc57b-account-delete-nspr5" event={"ID":"ce4d9d20-a900-4b3d-8ee0-46ea28a51262","Type":"ContainerStarted","Data":"3a2ed15c3908872f81c10e9936876dee8eccd12aaf314daab66426a9b8eb5eb9"} Dec 03 11:42:34 crc kubenswrapper[5052]: I1203 11:42:34.456946 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder5365-account-delete-shtcv" event={"ID":"d3e34248-e7e5-4158-8add-165d17bee9ab","Type":"ContainerStarted","Data":"4fa9a4b1a92d9ef54877f81389ac87bef6c378f99bd28cf880465cab830fd8ec"} Dec 03 11:42:34 crc kubenswrapper[5052]: I1203 11:42:34.460019 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutronbfb7-account-delete-ff26t" event={"ID":"bd2dc8ee-f6b9-4c8b-af5e-79e67a2d1ef0","Type":"ContainerStarted","Data":"a0bc7468f2ccb2d1256696de392bc437be4375fc79aa743aa97e16b0fca6384c"} Dec 03 11:42:34 crc kubenswrapper[5052]: I1203 11:42:34.464182 5052 generic.go:334] "Generic (PLEG): container finished" podID="fc5f71c9-f943-4ba6-a8eb-8e16ed558675" containerID="9a7110a1e99d880ed3c4e50b9718ac4ef71d8575d2f793cdc84e9e47fb1598f0" exitCode=0 Dec 03 11:42:34 crc kubenswrapper[5052]: I1203 11:42:34.464247 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-54w7t" event={"ID":"fc5f71c9-f943-4ba6-a8eb-8e16ed558675","Type":"ContainerDied","Data":"9a7110a1e99d880ed3c4e50b9718ac4ef71d8575d2f793cdc84e9e47fb1598f0"} Dec 03 11:42:34 crc kubenswrapper[5052]: I1203 11:42:34.471455 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-df67d6cb4-8sllj" event={"ID":"b88af976-65f2-4d29-8c14-d5c43e23a459","Type":"ContainerStarted","Data":"293d2df95b90ded20b61e85a4c44a57977574cbb7ac21e2e8a6ba24672fee70b"} Dec 03 11:42:34 crc kubenswrapper[5052]: I1203 11:42:34.471633 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/barbican-api-df67d6cb4-8sllj" podUID="b88af976-65f2-4d29-8c14-d5c43e23a459" containerName="barbican-api-log" containerID="cri-o://07660c848ebf1a69fb37d115276c6d328ca3e8bccca35abc98bb76e21788b417" gracePeriod=30 Dec 03 11:42:34 crc kubenswrapper[5052]: I1203 11:42:34.471830 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/barbican-api-df67d6cb4-8sllj" Dec 03 11:42:34 crc kubenswrapper[5052]: I1203 11:42:34.471779 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/barbican-api-df67d6cb4-8sllj" podUID="b88af976-65f2-4d29-8c14-d5c43e23a459" containerName="barbican-api" containerID="cri-o://293d2df95b90ded20b61e85a4c44a57977574cbb7ac21e2e8a6ba24672fee70b" gracePeriod=30 Dec 03 11:42:34 crc kubenswrapper[5052]: I1203 11:42:34.489587 5052 scope.go:117] "RemoveContainer" containerID="f9ff1557270cd7f8f08565a1b152ff021c2e83623d66dbe9dd1606df3722c53a" Dec 03 11:42:34 crc kubenswrapper[5052]: I1203 11:42:34.495260 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/barbican-api-df67d6cb4-8sllj" podUID="b88af976-65f2-4d29-8c14-d5c43e23a459" containerName="barbican-api-log" probeResult="failure" output="Get \"https://10.217.0.191:9311/healthcheck\": EOF" Dec 03 11:42:34 crc kubenswrapper[5052]: I1203 11:42:34.495497 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-69cf8cb7d6-jfb57" event={"ID":"8f1442a2-a28b-4784-934a-34b417dbcce9","Type":"ContainerStarted","Data":"cdf56942522f4bc644bc6410245d93699130cab711f1b2e7b726b013a6148db5"} Dec 03 11:42:34 crc kubenswrapper[5052]: I1203 11:42:34.495714 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/neutron-69cf8cb7d6-jfb57" podUID="8f1442a2-a28b-4784-934a-34b417dbcce9" containerName="neutron-api" containerID="cri-o://a5f8b0c46530b7fa02bdfaef171411cf9a70dd517b05545d95c2ab0253afcf85" gracePeriod=30 Dec 03 11:42:34 crc kubenswrapper[5052]: I1203 11:42:34.495774 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/neutron-69cf8cb7d6-jfb57" Dec 03 11:42:34 crc kubenswrapper[5052]: I1203 11:42:34.495804 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/neutron-69cf8cb7d6-jfb57" podUID="8f1442a2-a28b-4784-934a-34b417dbcce9" containerName="neutron-httpd" containerID="cri-o://cdf56942522f4bc644bc6410245d93699130cab711f1b2e7b726b013a6148db5" gracePeriod=30 Dec 03 11:42:34 crc kubenswrapper[5052]: I1203 11:42:34.502905 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/barbican-api-df67d6cb4-8sllj" podStartSLOduration=12.502889135 podStartE2EDuration="12.502889135s" podCreationTimestamp="2025-12-03 11:42:22 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:42:34.500843505 +0000 UTC m=+1345.089085279" watchObservedRunningTime="2025-12-03 11:42:34.502889135 +0000 UTC m=+1345.091130899" Dec 03 11:42:34 crc kubenswrapper[5052]: I1203 11:42:34.510408 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" event={"ID":"10c237bc-72e7-4da3-9f2c-b1688395bf6e","Type":"ContainerStarted","Data":"25fe88f8d102f09983f8884e65f0c6c1575ade67921f10a630e6dcc3ea8b506e"} Dec 03 11:42:34 crc kubenswrapper[5052]: I1203 11:42:34.519618 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/novaapi73a4-account-delete-lrzcr" podStartSLOduration=9.519595841 podStartE2EDuration="9.519595841s" podCreationTimestamp="2025-12-03 11:42:25 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:42:34.516921823 +0000 UTC m=+1345.105163587" watchObservedRunningTime="2025-12-03 11:42:34.519595841 +0000 UTC m=+1345.107837615" Dec 03 11:42:34 crc kubenswrapper[5052]: I1203 11:42:34.580964 5052 scope.go:117] "RemoveContainer" containerID="cc72d236c40c2e7f87295c5f31e63f10a061fd78a90f6c26b95db3911d675ff7" Dec 03 11:42:34 crc kubenswrapper[5052]: I1203 11:42:34.591551 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/neutron-69cf8cb7d6-jfb57" podStartSLOduration=12.591534444 podStartE2EDuration="12.591534444s" podCreationTimestamp="2025-12-03 11:42:22 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:42:34.56460486 +0000 UTC m=+1345.152846624" watchObservedRunningTime="2025-12-03 11:42:34.591534444 +0000 UTC m=+1345.179776208" Dec 03 11:42:34 crc kubenswrapper[5052]: I1203 11:42:34.677999 5052 scope.go:117] "RemoveContainer" containerID="61b746fb82156eb75de6a2356739407db21615dda656157379f9510b62f96730" Dec 03 11:42:34 crc kubenswrapper[5052]: I1203 11:42:34.829322 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance78ad-account-delete-7l64w" Dec 03 11:42:34 crc kubenswrapper[5052]: E1203 11:42:34.854983 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 11:42:34 crc kubenswrapper[5052]: E1203 11:42:34.855056 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/bd2dc8ee-f6b9-4c8b-af5e-79e67a2d1ef0-operator-scripts podName:bd2dc8ee-f6b9-4c8b-af5e-79e67a2d1ef0 nodeName:}" failed. No retries permitted until 2025-12-03 11:42:38.85503844 +0000 UTC m=+1349.443280204 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/bd2dc8ee-f6b9-4c8b-af5e-79e67a2d1ef0-operator-scripts") pod "neutronbfb7-account-delete-ff26t" (UID: "bd2dc8ee-f6b9-4c8b-af5e-79e67a2d1ef0") : configmap "openstack-scripts" not found Dec 03 11:42:35 crc kubenswrapper[5052]: I1203 11:42:34.955652 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6rdlm\" (UniqueName: \"kubernetes.io/projected/8408f908-2fbf-4933-9361-68af849aad18-kube-api-access-6rdlm\") pod \"8408f908-2fbf-4933-9361-68af849aad18\" (UID: \"8408f908-2fbf-4933-9361-68af849aad18\") " Dec 03 11:42:35 crc kubenswrapper[5052]: I1203 11:42:34.955863 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/8408f908-2fbf-4933-9361-68af849aad18-operator-scripts\") pod \"8408f908-2fbf-4933-9361-68af849aad18\" (UID: \"8408f908-2fbf-4933-9361-68af849aad18\") " Dec 03 11:42:35 crc kubenswrapper[5052]: E1203 11:42:34.956335 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 11:42:35 crc kubenswrapper[5052]: E1203 11:42:34.956383 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d3e34248-e7e5-4158-8add-165d17bee9ab-operator-scripts podName:d3e34248-e7e5-4158-8add-165d17bee9ab nodeName:}" failed. No retries permitted until 2025-12-03 11:42:38.956368988 +0000 UTC m=+1349.544610742 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/d3e34248-e7e5-4158-8add-165d17bee9ab-operator-scripts") pod "cinder5365-account-delete-shtcv" (UID: "d3e34248-e7e5-4158-8add-165d17bee9ab") : configmap "openstack-scripts" not found Dec 03 11:42:35 crc kubenswrapper[5052]: I1203 11:42:34.957457 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8408f908-2fbf-4933-9361-68af849aad18-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "8408f908-2fbf-4933-9361-68af849aad18" (UID: "8408f908-2fbf-4933-9361-68af849aad18"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:42:35 crc kubenswrapper[5052]: I1203 11:42:34.964445 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8408f908-2fbf-4933-9361-68af849aad18-kube-api-access-6rdlm" (OuterVolumeSpecName: "kube-api-access-6rdlm") pod "8408f908-2fbf-4933-9361-68af849aad18" (UID: "8408f908-2fbf-4933-9361-68af849aad18"). InnerVolumeSpecName "kube-api-access-6rdlm". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:42:35 crc kubenswrapper[5052]: I1203 11:42:35.057649 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/8408f908-2fbf-4933-9361-68af849aad18-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:35 crc kubenswrapper[5052]: I1203 11:42:35.057964 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-6rdlm\" (UniqueName: \"kubernetes.io/projected/8408f908-2fbf-4933-9361-68af849aad18-kube-api-access-6rdlm\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:35 crc kubenswrapper[5052]: E1203 11:42:35.363491 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/barbican-worker-config-data: secret "barbican-worker-config-data" not found Dec 03 11:42:35 crc kubenswrapper[5052]: E1203 11:42:35.363497 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/combined-ca-bundle: secret "combined-ca-bundle" not found Dec 03 11:42:35 crc kubenswrapper[5052]: E1203 11:42:35.363497 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/barbican-config-data: secret "barbican-config-data" not found Dec 03 11:42:35 crc kubenswrapper[5052]: E1203 11:42:35.363579 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/222f033c-3660-4347-b621-7122c4c94929-config-data-custom podName:222f033c-3660-4347-b621-7122c4c94929 nodeName:}" failed. No retries permitted until 2025-12-03 11:42:43.363558433 +0000 UTC m=+1353.951800197 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "config-data-custom" (UniqueName: "kubernetes.io/secret/222f033c-3660-4347-b621-7122c4c94929-config-data-custom") pod "barbican-worker-67c775d6bc-kwxzt" (UID: "222f033c-3660-4347-b621-7122c4c94929") : secret "barbican-worker-config-data" not found Dec 03 11:42:35 crc kubenswrapper[5052]: E1203 11:42:35.363597 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/222f033c-3660-4347-b621-7122c4c94929-combined-ca-bundle podName:222f033c-3660-4347-b621-7122c4c94929 nodeName:}" failed. No retries permitted until 2025-12-03 11:42:43.363589454 +0000 UTC m=+1353.951831238 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "combined-ca-bundle" (UniqueName: "kubernetes.io/secret/222f033c-3660-4347-b621-7122c4c94929-combined-ca-bundle") pod "barbican-worker-67c775d6bc-kwxzt" (UID: "222f033c-3660-4347-b621-7122c4c94929") : secret "combined-ca-bundle" not found Dec 03 11:42:35 crc kubenswrapper[5052]: E1203 11:42:35.363610 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/222f033c-3660-4347-b621-7122c4c94929-config-data podName:222f033c-3660-4347-b621-7122c4c94929 nodeName:}" failed. No retries permitted until 2025-12-03 11:42:43.363604165 +0000 UTC m=+1353.951845929 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "config-data" (UniqueName: "kubernetes.io/secret/222f033c-3660-4347-b621-7122c4c94929-config-data") pod "barbican-worker-67c775d6bc-kwxzt" (UID: "222f033c-3660-4347-b621-7122c4c94929") : secret "barbican-config-data" not found Dec 03 11:42:35 crc kubenswrapper[5052]: I1203 11:42:35.522996 5052 generic.go:334] "Generic (PLEG): container finished" podID="ce4d9d20-a900-4b3d-8ee0-46ea28a51262" containerID="3a2ed15c3908872f81c10e9936876dee8eccd12aaf314daab66426a9b8eb5eb9" exitCode=1 Dec 03 11:42:35 crc kubenswrapper[5052]: I1203 11:42:35.523156 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbicanc57b-account-delete-nspr5" event={"ID":"ce4d9d20-a900-4b3d-8ee0-46ea28a51262","Type":"ContainerDied","Data":"3a2ed15c3908872f81c10e9936876dee8eccd12aaf314daab66426a9b8eb5eb9"} Dec 03 11:42:35 crc kubenswrapper[5052]: I1203 11:42:35.526013 5052 generic.go:334] "Generic (PLEG): container finished" podID="bd2dc8ee-f6b9-4c8b-af5e-79e67a2d1ef0" containerID="a0bc7468f2ccb2d1256696de392bc437be4375fc79aa743aa97e16b0fca6384c" exitCode=1 Dec 03 11:42:35 crc kubenswrapper[5052]: I1203 11:42:35.526070 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutronbfb7-account-delete-ff26t" event={"ID":"bd2dc8ee-f6b9-4c8b-af5e-79e67a2d1ef0","Type":"ContainerDied","Data":"a0bc7468f2ccb2d1256696de392bc437be4375fc79aa743aa97e16b0fca6384c"} Dec 03 11:42:35 crc kubenswrapper[5052]: I1203 11:42:35.526107 5052 scope.go:117] "RemoveContainer" containerID="4514b8aae4d859abb11dc204bd75a6fb855f5e1b4b91f3cb73f993ef8101b439" Dec 03 11:42:35 crc kubenswrapper[5052]: I1203 11:42:35.533665 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-54w7t" event={"ID":"fc5f71c9-f943-4ba6-a8eb-8e16ed558675","Type":"ContainerStarted","Data":"932d879f7c954a746e1dd51fe1d60d6090129a6831b82e9b480ae97cb1975957"} Dec 03 11:42:35 crc kubenswrapper[5052]: I1203 11:42:35.533903 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-54w7t" Dec 03 11:42:35 crc kubenswrapper[5052]: I1203 11:42:35.537543 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance78ad-account-delete-7l64w" event={"ID":"8408f908-2fbf-4933-9361-68af849aad18","Type":"ContainerDied","Data":"95f80e3d46b311c5878997c907876d12b66a84948e0743050f0551ed84d32a11"} Dec 03 11:42:35 crc kubenswrapper[5052]: I1203 11:42:35.537552 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance78ad-account-delete-7l64w" Dec 03 11:42:35 crc kubenswrapper[5052]: I1203 11:42:35.540363 5052 generic.go:334] "Generic (PLEG): container finished" podID="965f5591-b293-4101-be4a-a5329d88fb69" containerID="9e46a517bfb791f3665b76fc9e3c2c9c89f643eb87f5ee717db97d31c9a3d66a" exitCode=1 Dec 03 11:42:35 crc kubenswrapper[5052]: I1203 11:42:35.540436 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/novaapi73a4-account-delete-lrzcr" event={"ID":"965f5591-b293-4101-be4a-a5329d88fb69","Type":"ContainerDied","Data":"9e46a517bfb791f3665b76fc9e3c2c9c89f643eb87f5ee717db97d31c9a3d66a"} Dec 03 11:42:35 crc kubenswrapper[5052]: I1203 11:42:35.545850 5052 generic.go:334] "Generic (PLEG): container finished" podID="8f1442a2-a28b-4784-934a-34b417dbcce9" containerID="cdf56942522f4bc644bc6410245d93699130cab711f1b2e7b726b013a6148db5" exitCode=0 Dec 03 11:42:35 crc kubenswrapper[5052]: I1203 11:42:35.545923 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-69cf8cb7d6-jfb57" event={"ID":"8f1442a2-a28b-4784-934a-34b417dbcce9","Type":"ContainerDied","Data":"cdf56942522f4bc644bc6410245d93699130cab711f1b2e7b726b013a6148db5"} Dec 03 11:42:35 crc kubenswrapper[5052]: I1203 11:42:35.551526 5052 generic.go:334] "Generic (PLEG): container finished" podID="daa521fc-7e1d-44d2-b7d8-6aab003cfb66" containerID="f084100fa637b545df332be6f942cc3ba527b4e7f8898dd5c51db5555433c0fd" exitCode=0 Dec 03 11:42:35 crc kubenswrapper[5052]: I1203 11:42:35.551608 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" event={"ID":"daa521fc-7e1d-44d2-b7d8-6aab003cfb66","Type":"ContainerDied","Data":"f084100fa637b545df332be6f942cc3ba527b4e7f8898dd5c51db5555433c0fd"} Dec 03 11:42:35 crc kubenswrapper[5052]: I1203 11:42:35.559864 5052 generic.go:334] "Generic (PLEG): container finished" podID="b88af976-65f2-4d29-8c14-d5c43e23a459" containerID="07660c848ebf1a69fb37d115276c6d328ca3e8bccca35abc98bb76e21788b417" exitCode=143 Dec 03 11:42:35 crc kubenswrapper[5052]: I1203 11:42:35.559929 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-df67d6cb4-8sllj" event={"ID":"b88af976-65f2-4d29-8c14-d5c43e23a459","Type":"ContainerDied","Data":"07660c848ebf1a69fb37d115276c6d328ca3e8bccca35abc98bb76e21788b417"} Dec 03 11:42:35 crc kubenswrapper[5052]: I1203 11:42:35.563745 5052 generic.go:334] "Generic (PLEG): container finished" podID="d3e34248-e7e5-4158-8add-165d17bee9ab" containerID="4fa9a4b1a92d9ef54877f81389ac87bef6c378f99bd28cf880465cab830fd8ec" exitCode=1 Dec 03 11:42:35 crc kubenswrapper[5052]: I1203 11:42:35.563824 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder5365-account-delete-shtcv" event={"ID":"d3e34248-e7e5-4158-8add-165d17bee9ab","Type":"ContainerDied","Data":"4fa9a4b1a92d9ef54877f81389ac87bef6c378f99bd28cf880465cab830fd8ec"} Dec 03 11:42:35 crc kubenswrapper[5052]: I1203 11:42:35.573407 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_ovn-northd-0_a30eaf15-65ff-426b-8ac2-1ed5a369886a/ovn-northd/0.log" Dec 03 11:42:35 crc kubenswrapper[5052]: I1203 11:42:35.573456 5052 generic.go:334] "Generic (PLEG): container finished" podID="a30eaf15-65ff-426b-8ac2-1ed5a369886a" containerID="455e8792612e9ef5fbedefc9390ff87ea262cbd9365b333801dd2dc51224737e" exitCode=139 Dec 03 11:42:35 crc kubenswrapper[5052]: I1203 11:42:35.573566 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovn-northd-0" event={"ID":"a30eaf15-65ff-426b-8ac2-1ed5a369886a","Type":"ContainerDied","Data":"455e8792612e9ef5fbedefc9390ff87ea262cbd9365b333801dd2dc51224737e"} Dec 03 11:42:35 crc kubenswrapper[5052]: I1203 11:42:35.596957 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-54w7t" podStartSLOduration=10.596938053 podStartE2EDuration="10.596938053s" podCreationTimestamp="2025-12-03 11:42:25 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:42:35.590029852 +0000 UTC m=+1346.178271616" watchObservedRunningTime="2025-12-03 11:42:35.596938053 +0000 UTC m=+1346.185179817" Dec 03 11:42:35 crc kubenswrapper[5052]: I1203 11:42:35.620267 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance78ad-account-delete-7l64w"] Dec 03 11:42:35 crc kubenswrapper[5052]: I1203 11:42:35.634805 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/glance78ad-account-delete-7l64w"] Dec 03 11:42:35 crc kubenswrapper[5052]: I1203 11:42:35.706930 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0b4b31e5-e82d-4d44-9c8f-d001484293b6" path="/var/lib/kubelet/pods/0b4b31e5-e82d-4d44-9c8f-d001484293b6/volumes" Dec 03 11:42:35 crc kubenswrapper[5052]: I1203 11:42:35.707742 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0c7d3107-6dd9-4aa2-a392-68a213013508" path="/var/lib/kubelet/pods/0c7d3107-6dd9-4aa2-a392-68a213013508/volumes" Dec 03 11:42:35 crc kubenswrapper[5052]: I1203 11:42:35.708428 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="18c1ce3d-6d64-433f-a115-9977227455be" path="/var/lib/kubelet/pods/18c1ce3d-6d64-433f-a115-9977227455be/volumes" Dec 03 11:42:35 crc kubenswrapper[5052]: I1203 11:42:35.710098 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1fc99a78-bfd8-4fc9-9ff7-bdd5adeaff73" path="/var/lib/kubelet/pods/1fc99a78-bfd8-4fc9-9ff7-bdd5adeaff73/volumes" Dec 03 11:42:35 crc kubenswrapper[5052]: I1203 11:42:35.710632 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="251248ba-578c-49cf-b77e-fc3d5c6cb6fd" path="/var/lib/kubelet/pods/251248ba-578c-49cf-b77e-fc3d5c6cb6fd/volumes" Dec 03 11:42:35 crc kubenswrapper[5052]: I1203 11:42:35.711313 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="32573220-587f-4703-a5e9-0fb67d2019ef" path="/var/lib/kubelet/pods/32573220-587f-4703-a5e9-0fb67d2019ef/volumes" Dec 03 11:42:35 crc kubenswrapper[5052]: I1203 11:42:35.712447 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5737462d-2e5c-4453-9d6c-7d67503555f7" path="/var/lib/kubelet/pods/5737462d-2e5c-4453-9d6c-7d67503555f7/volumes" Dec 03 11:42:35 crc kubenswrapper[5052]: I1203 11:42:35.713299 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="82dce330-8f65-4c14-80d4-958eb8ca5212" path="/var/lib/kubelet/pods/82dce330-8f65-4c14-80d4-958eb8ca5212/volumes" Dec 03 11:42:35 crc kubenswrapper[5052]: I1203 11:42:35.714200 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8408f908-2fbf-4933-9361-68af849aad18" path="/var/lib/kubelet/pods/8408f908-2fbf-4933-9361-68af849aad18/volumes" Dec 03 11:42:35 crc kubenswrapper[5052]: I1203 11:42:35.715486 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8c4732f9-0f9e-4600-aed9-6a230cbf7b16" path="/var/lib/kubelet/pods/8c4732f9-0f9e-4600-aed9-6a230cbf7b16/volumes" Dec 03 11:42:35 crc kubenswrapper[5052]: I1203 11:42:35.716273 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eae5b719-0017-4621-9cb4-73ed6353648e" path="/var/lib/kubelet/pods/eae5b719-0017-4621-9cb4-73ed6353648e/volumes" Dec 03 11:42:35 crc kubenswrapper[5052]: I1203 11:42:35.717093 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ee46c3a4-5e21-4f6b-a944-8cf96d87506e" path="/var/lib/kubelet/pods/ee46c3a4-5e21-4f6b-a944-8cf96d87506e/volumes" Dec 03 11:42:35 crc kubenswrapper[5052]: I1203 11:42:35.718430 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f274168d-df51-4f50-b014-1ea4e85370d7" path="/var/lib/kubelet/pods/f274168d-df51-4f50-b014-1ea4e85370d7/volumes" Dec 03 11:42:35 crc kubenswrapper[5052]: I1203 11:42:35.718866 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fe69e54e-ae29-49e2-8a21-0ff81cc0100f" path="/var/lib/kubelet/pods/fe69e54e-ae29-49e2-8a21-0ff81cc0100f/volumes" Dec 03 11:42:35 crc kubenswrapper[5052]: I1203 11:42:35.760799 5052 scope.go:117] "RemoveContainer" containerID="aeb430a10cf9a53281e8c5bdd1de92d30dc2d95fdde0ad7b668d5ced9af8b243" Dec 03 11:42:35 crc kubenswrapper[5052]: I1203 11:42:35.927075 5052 scope.go:117] "RemoveContainer" containerID="a5cf0c996d18409bb7578d077566cdc4ce54bb2d70d008fc3eb57b7017b7c5cb" Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.128393 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/novaapi73a4-account-delete-lrzcr" Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.202823 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.209736 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutronbfb7-account-delete-ff26t" Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.214043 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder5365-account-delete-shtcv" Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.218882 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbicanc57b-account-delete-nspr5" Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.231065 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_ovn-northd-0_a30eaf15-65ff-426b-8ac2-1ed5a369886a/ovn-northd/0.log" Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.231132 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.286493 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/d3e34248-e7e5-4158-8add-165d17bee9ab-operator-scripts\") pod \"d3e34248-e7e5-4158-8add-165d17bee9ab\" (UID: \"d3e34248-e7e5-4158-8add-165d17bee9ab\") " Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.286571 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"persistence\" (UniqueName: \"kubernetes.io/local-volume/local-storage09-crc\") pod \"daa521fc-7e1d-44d2-b7d8-6aab003cfb66\" (UID: \"daa521fc-7e1d-44d2-b7d8-6aab003cfb66\") " Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.286621 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b65mb\" (UniqueName: \"kubernetes.io/projected/d3e34248-e7e5-4158-8add-165d17bee9ab-kube-api-access-b65mb\") pod \"d3e34248-e7e5-4158-8add-165d17bee9ab\" (UID: \"d3e34248-e7e5-4158-8add-165d17bee9ab\") " Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.286662 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"rabbitmq-erlang-cookie\" (UniqueName: \"kubernetes.io/empty-dir/daa521fc-7e1d-44d2-b7d8-6aab003cfb66-rabbitmq-erlang-cookie\") pod \"daa521fc-7e1d-44d2-b7d8-6aab003cfb66\" (UID: \"daa521fc-7e1d-44d2-b7d8-6aab003cfb66\") " Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.286684 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"server-conf\" (UniqueName: \"kubernetes.io/configmap/daa521fc-7e1d-44d2-b7d8-6aab003cfb66-server-conf\") pod \"daa521fc-7e1d-44d2-b7d8-6aab003cfb66\" (UID: \"daa521fc-7e1d-44d2-b7d8-6aab003cfb66\") " Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.286717 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"erlang-cookie-secret\" (UniqueName: \"kubernetes.io/secret/daa521fc-7e1d-44d2-b7d8-6aab003cfb66-erlang-cookie-secret\") pod \"daa521fc-7e1d-44d2-b7d8-6aab003cfb66\" (UID: \"daa521fc-7e1d-44d2-b7d8-6aab003cfb66\") " Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.286735 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xk6mh\" (UniqueName: \"kubernetes.io/projected/965f5591-b293-4101-be4a-a5329d88fb69-kube-api-access-xk6mh\") pod \"965f5591-b293-4101-be4a-a5329d88fb69\" (UID: \"965f5591-b293-4101-be4a-a5329d88fb69\") " Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.286754 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"pod-info\" (UniqueName: \"kubernetes.io/downward-api/daa521fc-7e1d-44d2-b7d8-6aab003cfb66-pod-info\") pod \"daa521fc-7e1d-44d2-b7d8-6aab003cfb66\" (UID: \"daa521fc-7e1d-44d2-b7d8-6aab003cfb66\") " Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.286780 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5fg2w\" (UniqueName: \"kubernetes.io/projected/bd2dc8ee-f6b9-4c8b-af5e-79e67a2d1ef0-kube-api-access-5fg2w\") pod \"bd2dc8ee-f6b9-4c8b-af5e-79e67a2d1ef0\" (UID: \"bd2dc8ee-f6b9-4c8b-af5e-79e67a2d1ef0\") " Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.286855 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"rabbitmq-plugins\" (UniqueName: \"kubernetes.io/empty-dir/daa521fc-7e1d-44d2-b7d8-6aab003cfb66-rabbitmq-plugins\") pod \"daa521fc-7e1d-44d2-b7d8-6aab003cfb66\" (UID: \"daa521fc-7e1d-44d2-b7d8-6aab003cfb66\") " Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.286880 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-st9gb\" (UniqueName: \"kubernetes.io/projected/daa521fc-7e1d-44d2-b7d8-6aab003cfb66-kube-api-access-st9gb\") pod \"daa521fc-7e1d-44d2-b7d8-6aab003cfb66\" (UID: \"daa521fc-7e1d-44d2-b7d8-6aab003cfb66\") " Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.286909 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/daa521fc-7e1d-44d2-b7d8-6aab003cfb66-config-data\") pod \"daa521fc-7e1d-44d2-b7d8-6aab003cfb66\" (UID: \"daa521fc-7e1d-44d2-b7d8-6aab003cfb66\") " Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.286929 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/965f5591-b293-4101-be4a-a5329d88fb69-operator-scripts\") pod \"965f5591-b293-4101-be4a-a5329d88fb69\" (UID: \"965f5591-b293-4101-be4a-a5329d88fb69\") " Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.286972 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"rabbitmq-tls\" (UniqueName: \"kubernetes.io/projected/daa521fc-7e1d-44d2-b7d8-6aab003cfb66-rabbitmq-tls\") pod \"daa521fc-7e1d-44d2-b7d8-6aab003cfb66\" (UID: \"daa521fc-7e1d-44d2-b7d8-6aab003cfb66\") " Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.287000 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/bd2dc8ee-f6b9-4c8b-af5e-79e67a2d1ef0-operator-scripts\") pod \"bd2dc8ee-f6b9-4c8b-af5e-79e67a2d1ef0\" (UID: \"bd2dc8ee-f6b9-4c8b-af5e-79e67a2d1ef0\") " Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.287037 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"plugins-conf\" (UniqueName: \"kubernetes.io/configmap/daa521fc-7e1d-44d2-b7d8-6aab003cfb66-plugins-conf\") pod \"daa521fc-7e1d-44d2-b7d8-6aab003cfb66\" (UID: \"daa521fc-7e1d-44d2-b7d8-6aab003cfb66\") " Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.287059 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"rabbitmq-confd\" (UniqueName: \"kubernetes.io/projected/daa521fc-7e1d-44d2-b7d8-6aab003cfb66-rabbitmq-confd\") pod \"daa521fc-7e1d-44d2-b7d8-6aab003cfb66\" (UID: \"daa521fc-7e1d-44d2-b7d8-6aab003cfb66\") " Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.288342 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/daa521fc-7e1d-44d2-b7d8-6aab003cfb66-rabbitmq-erlang-cookie" (OuterVolumeSpecName: "rabbitmq-erlang-cookie") pod "daa521fc-7e1d-44d2-b7d8-6aab003cfb66" (UID: "daa521fc-7e1d-44d2-b7d8-6aab003cfb66"). InnerVolumeSpecName "rabbitmq-erlang-cookie". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.288391 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d3e34248-e7e5-4158-8add-165d17bee9ab-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "d3e34248-e7e5-4158-8add-165d17bee9ab" (UID: "d3e34248-e7e5-4158-8add-165d17bee9ab"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.290128 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/daa521fc-7e1d-44d2-b7d8-6aab003cfb66-plugins-conf" (OuterVolumeSpecName: "plugins-conf") pod "daa521fc-7e1d-44d2-b7d8-6aab003cfb66" (UID: "daa521fc-7e1d-44d2-b7d8-6aab003cfb66"). InnerVolumeSpecName "plugins-conf". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.290171 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bd2dc8ee-f6b9-4c8b-af5e-79e67a2d1ef0-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "bd2dc8ee-f6b9-4c8b-af5e-79e67a2d1ef0" (UID: "bd2dc8ee-f6b9-4c8b-af5e-79e67a2d1ef0"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.310655 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/daa521fc-7e1d-44d2-b7d8-6aab003cfb66-pod-info" (OuterVolumeSpecName: "pod-info") pod "daa521fc-7e1d-44d2-b7d8-6aab003cfb66" (UID: "daa521fc-7e1d-44d2-b7d8-6aab003cfb66"). InnerVolumeSpecName "pod-info". PluginName "kubernetes.io/downward-api", VolumeGidValue "" Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.310848 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/daa521fc-7e1d-44d2-b7d8-6aab003cfb66-kube-api-access-st9gb" (OuterVolumeSpecName: "kube-api-access-st9gb") pod "daa521fc-7e1d-44d2-b7d8-6aab003cfb66" (UID: "daa521fc-7e1d-44d2-b7d8-6aab003cfb66"). InnerVolumeSpecName "kube-api-access-st9gb". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.311829 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/965f5591-b293-4101-be4a-a5329d88fb69-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "965f5591-b293-4101-be4a-a5329d88fb69" (UID: "965f5591-b293-4101-be4a-a5329d88fb69"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.311923 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d3e34248-e7e5-4158-8add-165d17bee9ab-kube-api-access-b65mb" (OuterVolumeSpecName: "kube-api-access-b65mb") pod "d3e34248-e7e5-4158-8add-165d17bee9ab" (UID: "d3e34248-e7e5-4158-8add-165d17bee9ab"). InnerVolumeSpecName "kube-api-access-b65mb". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.312332 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage09-crc" (OuterVolumeSpecName: "persistence") pod "daa521fc-7e1d-44d2-b7d8-6aab003cfb66" (UID: "daa521fc-7e1d-44d2-b7d8-6aab003cfb66"). InnerVolumeSpecName "local-storage09-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.312408 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/daa521fc-7e1d-44d2-b7d8-6aab003cfb66-rabbitmq-plugins" (OuterVolumeSpecName: "rabbitmq-plugins") pod "daa521fc-7e1d-44d2-b7d8-6aab003cfb66" (UID: "daa521fc-7e1d-44d2-b7d8-6aab003cfb66"). InnerVolumeSpecName "rabbitmq-plugins". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.315471 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/965f5591-b293-4101-be4a-a5329d88fb69-kube-api-access-xk6mh" (OuterVolumeSpecName: "kube-api-access-xk6mh") pod "965f5591-b293-4101-be4a-a5329d88fb69" (UID: "965f5591-b293-4101-be4a-a5329d88fb69"). InnerVolumeSpecName "kube-api-access-xk6mh". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.316826 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/daa521fc-7e1d-44d2-b7d8-6aab003cfb66-rabbitmq-tls" (OuterVolumeSpecName: "rabbitmq-tls") pod "daa521fc-7e1d-44d2-b7d8-6aab003cfb66" (UID: "daa521fc-7e1d-44d2-b7d8-6aab003cfb66"). InnerVolumeSpecName "rabbitmq-tls". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.316887 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bd2dc8ee-f6b9-4c8b-af5e-79e67a2d1ef0-kube-api-access-5fg2w" (OuterVolumeSpecName: "kube-api-access-5fg2w") pod "bd2dc8ee-f6b9-4c8b-af5e-79e67a2d1ef0" (UID: "bd2dc8ee-f6b9-4c8b-af5e-79e67a2d1ef0"). InnerVolumeSpecName "kube-api-access-5fg2w". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.343613 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/daa521fc-7e1d-44d2-b7d8-6aab003cfb66-erlang-cookie-secret" (OuterVolumeSpecName: "erlang-cookie-secret") pod "daa521fc-7e1d-44d2-b7d8-6aab003cfb66" (UID: "daa521fc-7e1d-44d2-b7d8-6aab003cfb66"). InnerVolumeSpecName "erlang-cookie-secret". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.362167 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/daa521fc-7e1d-44d2-b7d8-6aab003cfb66-config-data" (OuterVolumeSpecName: "config-data") pod "daa521fc-7e1d-44d2-b7d8-6aab003cfb66" (UID: "daa521fc-7e1d-44d2-b7d8-6aab003cfb66"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.388601 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/ce4d9d20-a900-4b3d-8ee0-46ea28a51262-operator-scripts\") pod \"ce4d9d20-a900-4b3d-8ee0-46ea28a51262\" (UID: \"ce4d9d20-a900-4b3d-8ee0-46ea28a51262\") " Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.388692 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/a30eaf15-65ff-426b-8ac2-1ed5a369886a-scripts\") pod \"a30eaf15-65ff-426b-8ac2-1ed5a369886a\" (UID: \"a30eaf15-65ff-426b-8ac2-1ed5a369886a\") " Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.388745 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovn-rundir\" (UniqueName: \"kubernetes.io/empty-dir/a30eaf15-65ff-426b-8ac2-1ed5a369886a-ovn-rundir\") pod \"a30eaf15-65ff-426b-8ac2-1ed5a369886a\" (UID: \"a30eaf15-65ff-426b-8ac2-1ed5a369886a\") " Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.388811 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xhnp9\" (UniqueName: \"kubernetes.io/projected/ce4d9d20-a900-4b3d-8ee0-46ea28a51262-kube-api-access-xhnp9\") pod \"ce4d9d20-a900-4b3d-8ee0-46ea28a51262\" (UID: \"ce4d9d20-a900-4b3d-8ee0-46ea28a51262\") " Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.388868 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a30eaf15-65ff-426b-8ac2-1ed5a369886a-combined-ca-bundle\") pod \"a30eaf15-65ff-426b-8ac2-1ed5a369886a\" (UID: \"a30eaf15-65ff-426b-8ac2-1ed5a369886a\") " Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.388886 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovn-northd-tls-certs\" (UniqueName: \"kubernetes.io/secret/a30eaf15-65ff-426b-8ac2-1ed5a369886a-ovn-northd-tls-certs\") pod \"a30eaf15-65ff-426b-8ac2-1ed5a369886a\" (UID: \"a30eaf15-65ff-426b-8ac2-1ed5a369886a\") " Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.388922 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d5lbp\" (UniqueName: \"kubernetes.io/projected/a30eaf15-65ff-426b-8ac2-1ed5a369886a-kube-api-access-d5lbp\") pod \"a30eaf15-65ff-426b-8ac2-1ed5a369886a\" (UID: \"a30eaf15-65ff-426b-8ac2-1ed5a369886a\") " Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.388945 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a30eaf15-65ff-426b-8ac2-1ed5a369886a-config\") pod \"a30eaf15-65ff-426b-8ac2-1ed5a369886a\" (UID: \"a30eaf15-65ff-426b-8ac2-1ed5a369886a\") " Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.388977 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/a30eaf15-65ff-426b-8ac2-1ed5a369886a-metrics-certs-tls-certs\") pod \"a30eaf15-65ff-426b-8ac2-1ed5a369886a\" (UID: \"a30eaf15-65ff-426b-8ac2-1ed5a369886a\") " Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.389377 5052 reconciler_common.go:293] "Volume detached for volume \"plugins-conf\" (UniqueName: \"kubernetes.io/configmap/daa521fc-7e1d-44d2-b7d8-6aab003cfb66-plugins-conf\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.389402 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/d3e34248-e7e5-4158-8add-165d17bee9ab-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.389427 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage09-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage09-crc\") on node \"crc\" " Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.389441 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-b65mb\" (UniqueName: \"kubernetes.io/projected/d3e34248-e7e5-4158-8add-165d17bee9ab-kube-api-access-b65mb\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.389456 5052 reconciler_common.go:293] "Volume detached for volume \"rabbitmq-erlang-cookie\" (UniqueName: \"kubernetes.io/empty-dir/daa521fc-7e1d-44d2-b7d8-6aab003cfb66-rabbitmq-erlang-cookie\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.389469 5052 reconciler_common.go:293] "Volume detached for volume \"erlang-cookie-secret\" (UniqueName: \"kubernetes.io/secret/daa521fc-7e1d-44d2-b7d8-6aab003cfb66-erlang-cookie-secret\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.389483 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-xk6mh\" (UniqueName: \"kubernetes.io/projected/965f5591-b293-4101-be4a-a5329d88fb69-kube-api-access-xk6mh\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.389494 5052 reconciler_common.go:293] "Volume detached for volume \"pod-info\" (UniqueName: \"kubernetes.io/downward-api/daa521fc-7e1d-44d2-b7d8-6aab003cfb66-pod-info\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.389526 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-5fg2w\" (UniqueName: \"kubernetes.io/projected/bd2dc8ee-f6b9-4c8b-af5e-79e67a2d1ef0-kube-api-access-5fg2w\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.389539 5052 reconciler_common.go:293] "Volume detached for volume \"rabbitmq-plugins\" (UniqueName: \"kubernetes.io/empty-dir/daa521fc-7e1d-44d2-b7d8-6aab003cfb66-rabbitmq-plugins\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.389551 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-st9gb\" (UniqueName: \"kubernetes.io/projected/daa521fc-7e1d-44d2-b7d8-6aab003cfb66-kube-api-access-st9gb\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.389562 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/daa521fc-7e1d-44d2-b7d8-6aab003cfb66-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.389573 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/965f5591-b293-4101-be4a-a5329d88fb69-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.389583 5052 reconciler_common.go:293] "Volume detached for volume \"rabbitmq-tls\" (UniqueName: \"kubernetes.io/projected/daa521fc-7e1d-44d2-b7d8-6aab003cfb66-rabbitmq-tls\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.389594 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/bd2dc8ee-f6b9-4c8b-af5e-79e67a2d1ef0-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.391329 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a30eaf15-65ff-426b-8ac2-1ed5a369886a-scripts" (OuterVolumeSpecName: "scripts") pod "a30eaf15-65ff-426b-8ac2-1ed5a369886a" (UID: "a30eaf15-65ff-426b-8ac2-1ed5a369886a"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.391854 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ce4d9d20-a900-4b3d-8ee0-46ea28a51262-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "ce4d9d20-a900-4b3d-8ee0-46ea28a51262" (UID: "ce4d9d20-a900-4b3d-8ee0-46ea28a51262"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.392125 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a30eaf15-65ff-426b-8ac2-1ed5a369886a-ovn-rundir" (OuterVolumeSpecName: "ovn-rundir") pod "a30eaf15-65ff-426b-8ac2-1ed5a369886a" (UID: "a30eaf15-65ff-426b-8ac2-1ed5a369886a"). InnerVolumeSpecName "ovn-rundir". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.393865 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a30eaf15-65ff-426b-8ac2-1ed5a369886a-config" (OuterVolumeSpecName: "config") pod "a30eaf15-65ff-426b-8ac2-1ed5a369886a" (UID: "a30eaf15-65ff-426b-8ac2-1ed5a369886a"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.399249 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a30eaf15-65ff-426b-8ac2-1ed5a369886a-kube-api-access-d5lbp" (OuterVolumeSpecName: "kube-api-access-d5lbp") pod "a30eaf15-65ff-426b-8ac2-1ed5a369886a" (UID: "a30eaf15-65ff-426b-8ac2-1ed5a369886a"). InnerVolumeSpecName "kube-api-access-d5lbp". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.401819 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/daa521fc-7e1d-44d2-b7d8-6aab003cfb66-server-conf" (OuterVolumeSpecName: "server-conf") pod "daa521fc-7e1d-44d2-b7d8-6aab003cfb66" (UID: "daa521fc-7e1d-44d2-b7d8-6aab003cfb66"). InnerVolumeSpecName "server-conf". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.401863 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ce4d9d20-a900-4b3d-8ee0-46ea28a51262-kube-api-access-xhnp9" (OuterVolumeSpecName: "kube-api-access-xhnp9") pod "ce4d9d20-a900-4b3d-8ee0-46ea28a51262" (UID: "ce4d9d20-a900-4b3d-8ee0-46ea28a51262"). InnerVolumeSpecName "kube-api-access-xhnp9". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.417648 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage09-crc" (UniqueName: "kubernetes.io/local-volume/local-storage09-crc") on node "crc" Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.421027 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.478205 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a30eaf15-65ff-426b-8ac2-1ed5a369886a-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "a30eaf15-65ff-426b-8ac2-1ed5a369886a" (UID: "a30eaf15-65ff-426b-8ac2-1ed5a369886a"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.481322 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/daa521fc-7e1d-44d2-b7d8-6aab003cfb66-rabbitmq-confd" (OuterVolumeSpecName: "rabbitmq-confd") pod "daa521fc-7e1d-44d2-b7d8-6aab003cfb66" (UID: "daa521fc-7e1d-44d2-b7d8-6aab003cfb66"). InnerVolumeSpecName "rabbitmq-confd". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.489238 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a30eaf15-65ff-426b-8ac2-1ed5a369886a-metrics-certs-tls-certs" (OuterVolumeSpecName: "metrics-certs-tls-certs") pod "a30eaf15-65ff-426b-8ac2-1ed5a369886a" (UID: "a30eaf15-65ff-426b-8ac2-1ed5a369886a"). InnerVolumeSpecName "metrics-certs-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.491404 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a30eaf15-65ff-426b-8ac2-1ed5a369886a-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.491427 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage09-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage09-crc\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.491441 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-d5lbp\" (UniqueName: \"kubernetes.io/projected/a30eaf15-65ff-426b-8ac2-1ed5a369886a-kube-api-access-d5lbp\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.491454 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a30eaf15-65ff-426b-8ac2-1ed5a369886a-config\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.491466 5052 reconciler_common.go:293] "Volume detached for volume \"server-conf\" (UniqueName: \"kubernetes.io/configmap/daa521fc-7e1d-44d2-b7d8-6aab003cfb66-server-conf\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.491478 5052 reconciler_common.go:293] "Volume detached for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/a30eaf15-65ff-426b-8ac2-1ed5a369886a-metrics-certs-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.491488 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/ce4d9d20-a900-4b3d-8ee0-46ea28a51262-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.491498 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/a30eaf15-65ff-426b-8ac2-1ed5a369886a-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.491560 5052 reconciler_common.go:293] "Volume detached for volume \"ovn-rundir\" (UniqueName: \"kubernetes.io/empty-dir/a30eaf15-65ff-426b-8ac2-1ed5a369886a-ovn-rundir\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.491572 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-xhnp9\" (UniqueName: \"kubernetes.io/projected/ce4d9d20-a900-4b3d-8ee0-46ea28a51262-kube-api-access-xhnp9\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.491585 5052 reconciler_common.go:293] "Volume detached for volume \"rabbitmq-confd\" (UniqueName: \"kubernetes.io/projected/daa521fc-7e1d-44d2-b7d8-6aab003cfb66-rabbitmq-confd\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.493591 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a30eaf15-65ff-426b-8ac2-1ed5a369886a-ovn-northd-tls-certs" (OuterVolumeSpecName: "ovn-northd-tls-certs") pod "a30eaf15-65ff-426b-8ac2-1ed5a369886a" (UID: "a30eaf15-65ff-426b-8ac2-1ed5a369886a"). InnerVolumeSpecName "ovn-northd-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.517414 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-worker-79cc7656c5-896b7" Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.587255 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutronbfb7-account-delete-ff26t" Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.587260 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutronbfb7-account-delete-ff26t" event={"ID":"bd2dc8ee-f6b9-4c8b-af5e-79e67a2d1ef0","Type":"ContainerDied","Data":"e22e037c01d56a9771da6e8bbb240cecd8ba3025908ef45d7f860cce8ecd6d9a"} Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.587311 5052 scope.go:117] "RemoveContainer" containerID="a0bc7468f2ccb2d1256696de392bc437be4375fc79aa743aa97e16b0fca6384c" Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.591868 5052 generic.go:334] "Generic (PLEG): container finished" podID="32190156-44ee-4a16-a86c-2584f6a94866" containerID="63e884f6db70c59723e3f8be480e8d365efc639a9133cf1e01bf9eb55912f224" exitCode=0 Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.591850 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"persistence\" (UniqueName: \"kubernetes.io/local-volume/local-storage08-crc\") pod \"9944d9af-7a7b-4773-8279-72914d63ef5f\" (UID: \"9944d9af-7a7b-4773-8279-72914d63ef5f\") " Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.591925 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-worker-79cc7656c5-896b7" event={"ID":"32190156-44ee-4a16-a86c-2584f6a94866","Type":"ContainerDied","Data":"63e884f6db70c59723e3f8be480e8d365efc639a9133cf1e01bf9eb55912f224"} Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.591951 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-worker-79cc7656c5-896b7" event={"ID":"32190156-44ee-4a16-a86c-2584f6a94866","Type":"ContainerDied","Data":"a002a7e6855ea6b8eaa15396fc2fd13adb56ddeb2fa4a0cca2f3154d7e4ea303"} Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.591959 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"rabbitmq-plugins\" (UniqueName: \"kubernetes.io/empty-dir/9944d9af-7a7b-4773-8279-72914d63ef5f-rabbitmq-plugins\") pod \"9944d9af-7a7b-4773-8279-72914d63ef5f\" (UID: \"9944d9af-7a7b-4773-8279-72914d63ef5f\") " Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.592005 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"rabbitmq-tls\" (UniqueName: \"kubernetes.io/projected/9944d9af-7a7b-4773-8279-72914d63ef5f-rabbitmq-tls\") pod \"9944d9af-7a7b-4773-8279-72914d63ef5f\" (UID: \"9944d9af-7a7b-4773-8279-72914d63ef5f\") " Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.592014 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-worker-79cc7656c5-896b7" Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.592063 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"erlang-cookie-secret\" (UniqueName: \"kubernetes.io/secret/9944d9af-7a7b-4773-8279-72914d63ef5f-erlang-cookie-secret\") pod \"9944d9af-7a7b-4773-8279-72914d63ef5f\" (UID: \"9944d9af-7a7b-4773-8279-72914d63ef5f\") " Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.592102 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/32190156-44ee-4a16-a86c-2584f6a94866-config-data\") pod \"32190156-44ee-4a16-a86c-2584f6a94866\" (UID: \"32190156-44ee-4a16-a86c-2584f6a94866\") " Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.592124 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/32190156-44ee-4a16-a86c-2584f6a94866-config-data-custom\") pod \"32190156-44ee-4a16-a86c-2584f6a94866\" (UID: \"32190156-44ee-4a16-a86c-2584f6a94866\") " Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.592189 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"pod-info\" (UniqueName: \"kubernetes.io/downward-api/9944d9af-7a7b-4773-8279-72914d63ef5f-pod-info\") pod \"9944d9af-7a7b-4773-8279-72914d63ef5f\" (UID: \"9944d9af-7a7b-4773-8279-72914d63ef5f\") " Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.592234 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/9944d9af-7a7b-4773-8279-72914d63ef5f-config-data\") pod \"9944d9af-7a7b-4773-8279-72914d63ef5f\" (UID: \"9944d9af-7a7b-4773-8279-72914d63ef5f\") " Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.592251 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/32190156-44ee-4a16-a86c-2584f6a94866-logs\") pod \"32190156-44ee-4a16-a86c-2584f6a94866\" (UID: \"32190156-44ee-4a16-a86c-2584f6a94866\") " Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.592281 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fd4cm\" (UniqueName: \"kubernetes.io/projected/32190156-44ee-4a16-a86c-2584f6a94866-kube-api-access-fd4cm\") pod \"32190156-44ee-4a16-a86c-2584f6a94866\" (UID: \"32190156-44ee-4a16-a86c-2584f6a94866\") " Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.592312 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"rabbitmq-erlang-cookie\" (UniqueName: \"kubernetes.io/empty-dir/9944d9af-7a7b-4773-8279-72914d63ef5f-rabbitmq-erlang-cookie\") pod \"9944d9af-7a7b-4773-8279-72914d63ef5f\" (UID: \"9944d9af-7a7b-4773-8279-72914d63ef5f\") " Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.592344 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5qrmz\" (UniqueName: \"kubernetes.io/projected/9944d9af-7a7b-4773-8279-72914d63ef5f-kube-api-access-5qrmz\") pod \"9944d9af-7a7b-4773-8279-72914d63ef5f\" (UID: \"9944d9af-7a7b-4773-8279-72914d63ef5f\") " Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.592368 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"rabbitmq-confd\" (UniqueName: \"kubernetes.io/projected/9944d9af-7a7b-4773-8279-72914d63ef5f-rabbitmq-confd\") pod \"9944d9af-7a7b-4773-8279-72914d63ef5f\" (UID: \"9944d9af-7a7b-4773-8279-72914d63ef5f\") " Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.592396 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"plugins-conf\" (UniqueName: \"kubernetes.io/configmap/9944d9af-7a7b-4773-8279-72914d63ef5f-plugins-conf\") pod \"9944d9af-7a7b-4773-8279-72914d63ef5f\" (UID: \"9944d9af-7a7b-4773-8279-72914d63ef5f\") " Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.592426 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"server-conf\" (UniqueName: \"kubernetes.io/configmap/9944d9af-7a7b-4773-8279-72914d63ef5f-server-conf\") pod \"9944d9af-7a7b-4773-8279-72914d63ef5f\" (UID: \"9944d9af-7a7b-4773-8279-72914d63ef5f\") " Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.592458 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/32190156-44ee-4a16-a86c-2584f6a94866-combined-ca-bundle\") pod \"32190156-44ee-4a16-a86c-2584f6a94866\" (UID: \"32190156-44ee-4a16-a86c-2584f6a94866\") " Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.592817 5052 reconciler_common.go:293] "Volume detached for volume \"ovn-northd-tls-certs\" (UniqueName: \"kubernetes.io/secret/a30eaf15-65ff-426b-8ac2-1ed5a369886a-ovn-northd-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.594751 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9944d9af-7a7b-4773-8279-72914d63ef5f-plugins-conf" (OuterVolumeSpecName: "plugins-conf") pod "9944d9af-7a7b-4773-8279-72914d63ef5f" (UID: "9944d9af-7a7b-4773-8279-72914d63ef5f"). InnerVolumeSpecName "plugins-conf". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.595186 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9944d9af-7a7b-4773-8279-72914d63ef5f-rabbitmq-plugins" (OuterVolumeSpecName: "rabbitmq-plugins") pod "9944d9af-7a7b-4773-8279-72914d63ef5f" (UID: "9944d9af-7a7b-4773-8279-72914d63ef5f"). InnerVolumeSpecName "rabbitmq-plugins". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.596842 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9944d9af-7a7b-4773-8279-72914d63ef5f-erlang-cookie-secret" (OuterVolumeSpecName: "erlang-cookie-secret") pod "9944d9af-7a7b-4773-8279-72914d63ef5f" (UID: "9944d9af-7a7b-4773-8279-72914d63ef5f"). InnerVolumeSpecName "erlang-cookie-secret". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.597346 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/32190156-44ee-4a16-a86c-2584f6a94866-config-data-custom" (OuterVolumeSpecName: "config-data-custom") pod "32190156-44ee-4a16-a86c-2584f6a94866" (UID: "32190156-44ee-4a16-a86c-2584f6a94866"). InnerVolumeSpecName "config-data-custom". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.598610 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/32190156-44ee-4a16-a86c-2584f6a94866-logs" (OuterVolumeSpecName: "logs") pod "32190156-44ee-4a16-a86c-2584f6a94866" (UID: "32190156-44ee-4a16-a86c-2584f6a94866"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.598811 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage08-crc" (OuterVolumeSpecName: "persistence") pod "9944d9af-7a7b-4773-8279-72914d63ef5f" (UID: "9944d9af-7a7b-4773-8279-72914d63ef5f"). InnerVolumeSpecName "local-storage08-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.599141 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9944d9af-7a7b-4773-8279-72914d63ef5f-rabbitmq-erlang-cookie" (OuterVolumeSpecName: "rabbitmq-erlang-cookie") pod "9944d9af-7a7b-4773-8279-72914d63ef5f" (UID: "9944d9af-7a7b-4773-8279-72914d63ef5f"). InnerVolumeSpecName "rabbitmq-erlang-cookie". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.599601 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" event={"ID":"daa521fc-7e1d-44d2-b7d8-6aab003cfb66","Type":"ContainerDied","Data":"21dffb5505e77fa5f1310e4358351755d36c795e92230fe26413268ed7da8b3f"} Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.599815 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.599847 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9944d9af-7a7b-4773-8279-72914d63ef5f-kube-api-access-5qrmz" (OuterVolumeSpecName: "kube-api-access-5qrmz") pod "9944d9af-7a7b-4773-8279-72914d63ef5f" (UID: "9944d9af-7a7b-4773-8279-72914d63ef5f"). InnerVolumeSpecName "kube-api-access-5qrmz". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.599946 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/32190156-44ee-4a16-a86c-2584f6a94866-kube-api-access-fd4cm" (OuterVolumeSpecName: "kube-api-access-fd4cm") pod "32190156-44ee-4a16-a86c-2584f6a94866" (UID: "32190156-44ee-4a16-a86c-2584f6a94866"). InnerVolumeSpecName "kube-api-access-fd4cm". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.601912 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9944d9af-7a7b-4773-8279-72914d63ef5f-rabbitmq-tls" (OuterVolumeSpecName: "rabbitmq-tls") pod "9944d9af-7a7b-4773-8279-72914d63ef5f" (UID: "9944d9af-7a7b-4773-8279-72914d63ef5f"). InnerVolumeSpecName "rabbitmq-tls". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.603317 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbicanc57b-account-delete-nspr5" event={"ID":"ce4d9d20-a900-4b3d-8ee0-46ea28a51262","Type":"ContainerDied","Data":"29683920a6e7674e5e256703c5b75a547484dd141b18d2cae6708264d8942984"} Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.603403 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbicanc57b-account-delete-nspr5" Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.606850 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder5365-account-delete-shtcv" event={"ID":"d3e34248-e7e5-4158-8add-165d17bee9ab","Type":"ContainerDied","Data":"97ece835d9a4c804e6f6b9d476829694aa6078bfb88602c735413883ae97d229"} Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.606939 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder5365-account-delete-shtcv" Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.611812 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/novaapi73a4-account-delete-lrzcr" Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.611812 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/novaapi73a4-account-delete-lrzcr" event={"ID":"965f5591-b293-4101-be4a-a5329d88fb69","Type":"ContainerDied","Data":"e29abc27dd77b54bfd434a6b172dad89359bf72d4901f1de70c3f2cfd6f60c9b"} Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.613671 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_ovn-northd-0_a30eaf15-65ff-426b-8ac2-1ed5a369886a/ovn-northd/0.log" Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.613727 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovn-northd-0" event={"ID":"a30eaf15-65ff-426b-8ac2-1ed5a369886a","Type":"ContainerDied","Data":"551c2741749652575c73fe7f2d226f491f86182f4b8934b424925c7a1cca852e"} Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.613796 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.614157 5052 scope.go:117] "RemoveContainer" containerID="63e884f6db70c59723e3f8be480e8d365efc639a9133cf1e01bf9eb55912f224" Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.621200 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9944d9af-7a7b-4773-8279-72914d63ef5f-pod-info" (OuterVolumeSpecName: "pod-info") pod "9944d9af-7a7b-4773-8279-72914d63ef5f" (UID: "9944d9af-7a7b-4773-8279-72914d63ef5f"). InnerVolumeSpecName "pod-info". PluginName "kubernetes.io/downward-api", VolumeGidValue "" Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.623040 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/neutronbfb7-account-delete-ff26t"] Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.625700 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9944d9af-7a7b-4773-8279-72914d63ef5f-config-data" (OuterVolumeSpecName: "config-data") pod "9944d9af-7a7b-4773-8279-72914d63ef5f" (UID: "9944d9af-7a7b-4773-8279-72914d63ef5f"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.625926 5052 generic.go:334] "Generic (PLEG): container finished" podID="9944d9af-7a7b-4773-8279-72914d63ef5f" containerID="c7a780b226312d9ca6d3ee18cc8115287601a5ebf3182143b2ccad2a68a3f43a" exitCode=0 Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.626103 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.626402 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-server-0" event={"ID":"9944d9af-7a7b-4773-8279-72914d63ef5f","Type":"ContainerDied","Data":"c7a780b226312d9ca6d3ee18cc8115287601a5ebf3182143b2ccad2a68a3f43a"} Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.626438 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-server-0" event={"ID":"9944d9af-7a7b-4773-8279-72914d63ef5f","Type":"ContainerDied","Data":"4714213df4291409dd8c9248d4558e17471f3d7a3a646b8ada0b4e519793e1cb"} Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.632448 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/32190156-44ee-4a16-a86c-2584f6a94866-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "32190156-44ee-4a16-a86c-2584f6a94866" (UID: "32190156-44ee-4a16-a86c-2584f6a94866"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.641025 5052 scope.go:117] "RemoveContainer" containerID="6e34f31b2be164f36807f4119b800ae1a884c550a462f7a9c586c37f82893863" Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.643455 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/neutronbfb7-account-delete-ff26t"] Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.657764 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/rabbitmq-cell1-server-0"] Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.668873 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/rabbitmq-cell1-server-0"] Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.680341 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbicanc57b-account-delete-nspr5"] Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.680539 5052 scope.go:117] "RemoveContainer" containerID="63e884f6db70c59723e3f8be480e8d365efc639a9133cf1e01bf9eb55912f224" Dec 03 11:42:36 crc kubenswrapper[5052]: E1203 11:42:36.682218 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"63e884f6db70c59723e3f8be480e8d365efc639a9133cf1e01bf9eb55912f224\": container with ID starting with 63e884f6db70c59723e3f8be480e8d365efc639a9133cf1e01bf9eb55912f224 not found: ID does not exist" containerID="63e884f6db70c59723e3f8be480e8d365efc639a9133cf1e01bf9eb55912f224" Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.682275 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"63e884f6db70c59723e3f8be480e8d365efc639a9133cf1e01bf9eb55912f224"} err="failed to get container status \"63e884f6db70c59723e3f8be480e8d365efc639a9133cf1e01bf9eb55912f224\": rpc error: code = NotFound desc = could not find container \"63e884f6db70c59723e3f8be480e8d365efc639a9133cf1e01bf9eb55912f224\": container with ID starting with 63e884f6db70c59723e3f8be480e8d365efc639a9133cf1e01bf9eb55912f224 not found: ID does not exist" Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.682316 5052 scope.go:117] "RemoveContainer" containerID="6e34f31b2be164f36807f4119b800ae1a884c550a462f7a9c586c37f82893863" Dec 03 11:42:36 crc kubenswrapper[5052]: E1203 11:42:36.684592 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6e34f31b2be164f36807f4119b800ae1a884c550a462f7a9c586c37f82893863\": container with ID starting with 6e34f31b2be164f36807f4119b800ae1a884c550a462f7a9c586c37f82893863 not found: ID does not exist" containerID="6e34f31b2be164f36807f4119b800ae1a884c550a462f7a9c586c37f82893863" Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.684689 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6e34f31b2be164f36807f4119b800ae1a884c550a462f7a9c586c37f82893863"} err="failed to get container status \"6e34f31b2be164f36807f4119b800ae1a884c550a462f7a9c586c37f82893863\": rpc error: code = NotFound desc = could not find container \"6e34f31b2be164f36807f4119b800ae1a884c550a462f7a9c586c37f82893863\": container with ID starting with 6e34f31b2be164f36807f4119b800ae1a884c550a462f7a9c586c37f82893863 not found: ID does not exist" Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.684846 5052 scope.go:117] "RemoveContainer" containerID="f084100fa637b545df332be6f942cc3ba527b4e7f8898dd5c51db5555433c0fd" Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.693379 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/barbicanc57b-account-delete-nspr5"] Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.697459 5052 reconciler_common.go:293] "Volume detached for volume \"rabbitmq-plugins\" (UniqueName: \"kubernetes.io/empty-dir/9944d9af-7a7b-4773-8279-72914d63ef5f-rabbitmq-plugins\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.697569 5052 reconciler_common.go:293] "Volume detached for volume \"rabbitmq-tls\" (UniqueName: \"kubernetes.io/projected/9944d9af-7a7b-4773-8279-72914d63ef5f-rabbitmq-tls\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.697585 5052 reconciler_common.go:293] "Volume detached for volume \"erlang-cookie-secret\" (UniqueName: \"kubernetes.io/secret/9944d9af-7a7b-4773-8279-72914d63ef5f-erlang-cookie-secret\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.697600 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/32190156-44ee-4a16-a86c-2584f6a94866-config-data-custom\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.697614 5052 reconciler_common.go:293] "Volume detached for volume \"pod-info\" (UniqueName: \"kubernetes.io/downward-api/9944d9af-7a7b-4773-8279-72914d63ef5f-pod-info\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.697628 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/9944d9af-7a7b-4773-8279-72914d63ef5f-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.697643 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/32190156-44ee-4a16-a86c-2584f6a94866-logs\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.697654 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-fd4cm\" (UniqueName: \"kubernetes.io/projected/32190156-44ee-4a16-a86c-2584f6a94866-kube-api-access-fd4cm\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.697668 5052 reconciler_common.go:293] "Volume detached for volume \"rabbitmq-erlang-cookie\" (UniqueName: \"kubernetes.io/empty-dir/9944d9af-7a7b-4773-8279-72914d63ef5f-rabbitmq-erlang-cookie\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.697679 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-5qrmz\" (UniqueName: \"kubernetes.io/projected/9944d9af-7a7b-4773-8279-72914d63ef5f-kube-api-access-5qrmz\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.697693 5052 reconciler_common.go:293] "Volume detached for volume \"plugins-conf\" (UniqueName: \"kubernetes.io/configmap/9944d9af-7a7b-4773-8279-72914d63ef5f-plugins-conf\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.697705 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/32190156-44ee-4a16-a86c-2584f6a94866-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.697737 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage08-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage08-crc\") on node \"crc\" " Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.703161 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder5365-account-delete-shtcv"] Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.706826 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9944d9af-7a7b-4773-8279-72914d63ef5f-server-conf" (OuterVolumeSpecName: "server-conf") pod "9944d9af-7a7b-4773-8279-72914d63ef5f" (UID: "9944d9af-7a7b-4773-8279-72914d63ef5f"). InnerVolumeSpecName "server-conf". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.713563 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/cinder5365-account-delete-shtcv"] Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.720329 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/novaapi73a4-account-delete-lrzcr"] Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.721000 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/32190156-44ee-4a16-a86c-2584f6a94866-config-data" (OuterVolumeSpecName: "config-data") pod "32190156-44ee-4a16-a86c-2584f6a94866" (UID: "32190156-44ee-4a16-a86c-2584f6a94866"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.732588 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/novaapi73a4-account-delete-lrzcr"] Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.739753 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ovn-northd-0"] Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.740457 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9944d9af-7a7b-4773-8279-72914d63ef5f-rabbitmq-confd" (OuterVolumeSpecName: "rabbitmq-confd") pod "9944d9af-7a7b-4773-8279-72914d63ef5f" (UID: "9944d9af-7a7b-4773-8279-72914d63ef5f"). InnerVolumeSpecName "rabbitmq-confd". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.741043 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage08-crc" (UniqueName: "kubernetes.io/local-volume/local-storage08-crc") on node "crc" Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.748618 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ovn-northd-0"] Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.799407 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/32190156-44ee-4a16-a86c-2584f6a94866-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.799452 5052 reconciler_common.go:293] "Volume detached for volume \"rabbitmq-confd\" (UniqueName: \"kubernetes.io/projected/9944d9af-7a7b-4773-8279-72914d63ef5f-rabbitmq-confd\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.799464 5052 reconciler_common.go:293] "Volume detached for volume \"server-conf\" (UniqueName: \"kubernetes.io/configmap/9944d9af-7a7b-4773-8279-72914d63ef5f-server-conf\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.799476 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage08-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage08-crc\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.844372 5052 scope.go:117] "RemoveContainer" containerID="0fbd3a4d4383d62bcd48e87a34d3dbb4ccd0417936dd06ed21f799d8c04dd391" Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.864836 5052 scope.go:117] "RemoveContainer" containerID="3a2ed15c3908872f81c10e9936876dee8eccd12aaf314daab66426a9b8eb5eb9" Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.897138 5052 scope.go:117] "RemoveContainer" containerID="4fa9a4b1a92d9ef54877f81389ac87bef6c378f99bd28cf880465cab830fd8ec" Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.922920 5052 scope.go:117] "RemoveContainer" containerID="9e46a517bfb791f3665b76fc9e3c2c9c89f643eb87f5ee717db97d31c9a3d66a" Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.961126 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-worker-79cc7656c5-896b7"] Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.965353 5052 scope.go:117] "RemoveContainer" containerID="abd32ba7a2fea1f9280ace25d3fe52938520b763a092b3e8d9e1b4a4f4e6e2be" Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.972147 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/barbican-worker-79cc7656c5-896b7"] Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.991683 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/rabbitmq-server-0"] Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.993400 5052 scope.go:117] "RemoveContainer" containerID="455e8792612e9ef5fbedefc9390ff87ea262cbd9365b333801dd2dc51224737e" Dec 03 11:42:36 crc kubenswrapper[5052]: I1203 11:42:36.997774 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/rabbitmq-server-0"] Dec 03 11:42:37 crc kubenswrapper[5052]: I1203 11:42:37.011222 5052 scope.go:117] "RemoveContainer" containerID="c7a780b226312d9ca6d3ee18cc8115287601a5ebf3182143b2ccad2a68a3f43a" Dec 03 11:42:37 crc kubenswrapper[5052]: I1203 11:42:37.028959 5052 scope.go:117] "RemoveContainer" containerID="a4c0b116720eadc0b68b284395f846e2b236870d208137226f3a533a48d6959e" Dec 03 11:42:37 crc kubenswrapper[5052]: I1203 11:42:37.047204 5052 scope.go:117] "RemoveContainer" containerID="c7a780b226312d9ca6d3ee18cc8115287601a5ebf3182143b2ccad2a68a3f43a" Dec 03 11:42:37 crc kubenswrapper[5052]: E1203 11:42:37.047656 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c7a780b226312d9ca6d3ee18cc8115287601a5ebf3182143b2ccad2a68a3f43a\": container with ID starting with c7a780b226312d9ca6d3ee18cc8115287601a5ebf3182143b2ccad2a68a3f43a not found: ID does not exist" containerID="c7a780b226312d9ca6d3ee18cc8115287601a5ebf3182143b2ccad2a68a3f43a" Dec 03 11:42:37 crc kubenswrapper[5052]: I1203 11:42:37.047686 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c7a780b226312d9ca6d3ee18cc8115287601a5ebf3182143b2ccad2a68a3f43a"} err="failed to get container status \"c7a780b226312d9ca6d3ee18cc8115287601a5ebf3182143b2ccad2a68a3f43a\": rpc error: code = NotFound desc = could not find container \"c7a780b226312d9ca6d3ee18cc8115287601a5ebf3182143b2ccad2a68a3f43a\": container with ID starting with c7a780b226312d9ca6d3ee18cc8115287601a5ebf3182143b2ccad2a68a3f43a not found: ID does not exist" Dec 03 11:42:37 crc kubenswrapper[5052]: I1203 11:42:37.047709 5052 scope.go:117] "RemoveContainer" containerID="a4c0b116720eadc0b68b284395f846e2b236870d208137226f3a533a48d6959e" Dec 03 11:42:37 crc kubenswrapper[5052]: E1203 11:42:37.047963 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a4c0b116720eadc0b68b284395f846e2b236870d208137226f3a533a48d6959e\": container with ID starting with a4c0b116720eadc0b68b284395f846e2b236870d208137226f3a533a48d6959e not found: ID does not exist" containerID="a4c0b116720eadc0b68b284395f846e2b236870d208137226f3a533a48d6959e" Dec 03 11:42:37 crc kubenswrapper[5052]: I1203 11:42:37.048014 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a4c0b116720eadc0b68b284395f846e2b236870d208137226f3a533a48d6959e"} err="failed to get container status \"a4c0b116720eadc0b68b284395f846e2b236870d208137226f3a533a48d6959e\": rpc error: code = NotFound desc = could not find container \"a4c0b116720eadc0b68b284395f846e2b236870d208137226f3a533a48d6959e\": container with ID starting with a4c0b116720eadc0b68b284395f846e2b236870d208137226f3a533a48d6959e not found: ID does not exist" Dec 03 11:42:37 crc kubenswrapper[5052]: I1203 11:42:37.690399 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="32190156-44ee-4a16-a86c-2584f6a94866" path="/var/lib/kubelet/pods/32190156-44ee-4a16-a86c-2584f6a94866/volumes" Dec 03 11:42:37 crc kubenswrapper[5052]: I1203 11:42:37.691765 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="965f5591-b293-4101-be4a-a5329d88fb69" path="/var/lib/kubelet/pods/965f5591-b293-4101-be4a-a5329d88fb69/volumes" Dec 03 11:42:37 crc kubenswrapper[5052]: I1203 11:42:37.692404 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9944d9af-7a7b-4773-8279-72914d63ef5f" path="/var/lib/kubelet/pods/9944d9af-7a7b-4773-8279-72914d63ef5f/volumes" Dec 03 11:42:37 crc kubenswrapper[5052]: I1203 11:42:37.693925 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a30eaf15-65ff-426b-8ac2-1ed5a369886a" path="/var/lib/kubelet/pods/a30eaf15-65ff-426b-8ac2-1ed5a369886a/volumes" Dec 03 11:42:37 crc kubenswrapper[5052]: I1203 11:42:37.694568 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bd2dc8ee-f6b9-4c8b-af5e-79e67a2d1ef0" path="/var/lib/kubelet/pods/bd2dc8ee-f6b9-4c8b-af5e-79e67a2d1ef0/volumes" Dec 03 11:42:37 crc kubenswrapper[5052]: I1203 11:42:37.695079 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ce4d9d20-a900-4b3d-8ee0-46ea28a51262" path="/var/lib/kubelet/pods/ce4d9d20-a900-4b3d-8ee0-46ea28a51262/volumes" Dec 03 11:42:37 crc kubenswrapper[5052]: I1203 11:42:37.696144 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d3e34248-e7e5-4158-8add-165d17bee9ab" path="/var/lib/kubelet/pods/d3e34248-e7e5-4158-8add-165d17bee9ab/volumes" Dec 03 11:42:37 crc kubenswrapper[5052]: I1203 11:42:37.696917 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="daa521fc-7e1d-44d2-b7d8-6aab003cfb66" path="/var/lib/kubelet/pods/daa521fc-7e1d-44d2-b7d8-6aab003cfb66/volumes" Dec 03 11:42:38 crc kubenswrapper[5052]: I1203 11:42:38.469170 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/barbican-api-df67d6cb4-8sllj" Dec 03 11:42:39 crc kubenswrapper[5052]: I1203 11:42:39.098725 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-54w7t" Dec 03 11:42:39 crc kubenswrapper[5052]: I1203 11:42:39.165732 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-58b8ddd7c-jtml2"] Dec 03 11:42:39 crc kubenswrapper[5052]: I1203 11:42:39.165998 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-58b8ddd7c-jtml2" podUID="1e003f91-c464-4200-94c1-18c2bea8ee8a" containerName="dnsmasq-dns" containerID="cri-o://c4284d7a16341b2a8e09ff5d310dc289d1b1e7b19a227679126b07d26626f9f9" gracePeriod=10 Dec 03 11:42:39 crc kubenswrapper[5052]: I1203 11:42:39.670938 5052 generic.go:334] "Generic (PLEG): container finished" podID="1e003f91-c464-4200-94c1-18c2bea8ee8a" containerID="c4284d7a16341b2a8e09ff5d310dc289d1b1e7b19a227679126b07d26626f9f9" exitCode=0 Dec 03 11:42:39 crc kubenswrapper[5052]: I1203 11:42:39.670987 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-58b8ddd7c-jtml2" event={"ID":"1e003f91-c464-4200-94c1-18c2bea8ee8a","Type":"ContainerDied","Data":"c4284d7a16341b2a8e09ff5d310dc289d1b1e7b19a227679126b07d26626f9f9"} Dec 03 11:42:40 crc kubenswrapper[5052]: I1203 11:42:40.217468 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-58b8ddd7c-jtml2" Dec 03 11:42:40 crc kubenswrapper[5052]: I1203 11:42:40.367116 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1e003f91-c464-4200-94c1-18c2bea8ee8a-config\") pod \"1e003f91-c464-4200-94c1-18c2bea8ee8a\" (UID: \"1e003f91-c464-4200-94c1-18c2bea8ee8a\") " Dec 03 11:42:40 crc kubenswrapper[5052]: I1203 11:42:40.367183 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k6nvd\" (UniqueName: \"kubernetes.io/projected/1e003f91-c464-4200-94c1-18c2bea8ee8a-kube-api-access-k6nvd\") pod \"1e003f91-c464-4200-94c1-18c2bea8ee8a\" (UID: \"1e003f91-c464-4200-94c1-18c2bea8ee8a\") " Dec 03 11:42:40 crc kubenswrapper[5052]: I1203 11:42:40.367250 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"dns-swift-storage-0\" (UniqueName: \"kubernetes.io/configmap/1e003f91-c464-4200-94c1-18c2bea8ee8a-dns-swift-storage-0\") pod \"1e003f91-c464-4200-94c1-18c2bea8ee8a\" (UID: \"1e003f91-c464-4200-94c1-18c2bea8ee8a\") " Dec 03 11:42:40 crc kubenswrapper[5052]: I1203 11:42:40.367389 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/1e003f91-c464-4200-94c1-18c2bea8ee8a-dnsmasq-svc\") pod \"1e003f91-c464-4200-94c1-18c2bea8ee8a\" (UID: \"1e003f91-c464-4200-94c1-18c2bea8ee8a\") " Dec 03 11:42:40 crc kubenswrapper[5052]: I1203 11:42:40.372778 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1e003f91-c464-4200-94c1-18c2bea8ee8a-kube-api-access-k6nvd" (OuterVolumeSpecName: "kube-api-access-k6nvd") pod "1e003f91-c464-4200-94c1-18c2bea8ee8a" (UID: "1e003f91-c464-4200-94c1-18c2bea8ee8a"). InnerVolumeSpecName "kube-api-access-k6nvd". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:42:40 crc kubenswrapper[5052]: I1203 11:42:40.424274 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1e003f91-c464-4200-94c1-18c2bea8ee8a-dnsmasq-svc" (OuterVolumeSpecName: "dnsmasq-svc") pod "1e003f91-c464-4200-94c1-18c2bea8ee8a" (UID: "1e003f91-c464-4200-94c1-18c2bea8ee8a"). InnerVolumeSpecName "dnsmasq-svc". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:42:40 crc kubenswrapper[5052]: I1203 11:42:40.425969 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1e003f91-c464-4200-94c1-18c2bea8ee8a-config" (OuterVolumeSpecName: "config") pod "1e003f91-c464-4200-94c1-18c2bea8ee8a" (UID: "1e003f91-c464-4200-94c1-18c2bea8ee8a"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:42:40 crc kubenswrapper[5052]: I1203 11:42:40.442832 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1e003f91-c464-4200-94c1-18c2bea8ee8a-dns-swift-storage-0" (OuterVolumeSpecName: "dns-swift-storage-0") pod "1e003f91-c464-4200-94c1-18c2bea8ee8a" (UID: "1e003f91-c464-4200-94c1-18c2bea8ee8a"). InnerVolumeSpecName "dns-swift-storage-0". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:42:40 crc kubenswrapper[5052]: I1203 11:42:40.471412 5052 reconciler_common.go:293] "Volume detached for volume \"dns-swift-storage-0\" (UniqueName: \"kubernetes.io/configmap/1e003f91-c464-4200-94c1-18c2bea8ee8a-dns-swift-storage-0\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:40 crc kubenswrapper[5052]: I1203 11:42:40.471447 5052 reconciler_common.go:293] "Volume detached for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/1e003f91-c464-4200-94c1-18c2bea8ee8a-dnsmasq-svc\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:40 crc kubenswrapper[5052]: I1203 11:42:40.471457 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1e003f91-c464-4200-94c1-18c2bea8ee8a-config\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:40 crc kubenswrapper[5052]: I1203 11:42:40.471467 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-k6nvd\" (UniqueName: \"kubernetes.io/projected/1e003f91-c464-4200-94c1-18c2bea8ee8a-kube-api-access-k6nvd\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:40 crc kubenswrapper[5052]: I1203 11:42:40.682130 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-58b8ddd7c-jtml2" event={"ID":"1e003f91-c464-4200-94c1-18c2bea8ee8a","Type":"ContainerDied","Data":"63a143edc85f8a66a5a6b506dfb03193e5395d27f22fe5cc397c54fea46d8f52"} Dec 03 11:42:40 crc kubenswrapper[5052]: I1203 11:42:40.682437 5052 scope.go:117] "RemoveContainer" containerID="c4284d7a16341b2a8e09ff5d310dc289d1b1e7b19a227679126b07d26626f9f9" Dec 03 11:42:40 crc kubenswrapper[5052]: I1203 11:42:40.682388 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-58b8ddd7c-jtml2" Dec 03 11:42:40 crc kubenswrapper[5052]: I1203 11:42:40.708134 5052 scope.go:117] "RemoveContainer" containerID="d5282e088e97a4376d50474919f0ef21c7f76187e137d2647b7d27033f930758" Dec 03 11:42:40 crc kubenswrapper[5052]: I1203 11:42:40.716297 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-58b8ddd7c-jtml2"] Dec 03 11:42:40 crc kubenswrapper[5052]: I1203 11:42:40.722992 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-58b8ddd7c-jtml2"] Dec 03 11:42:41 crc kubenswrapper[5052]: I1203 11:42:41.684999 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1e003f91-c464-4200-94c1-18c2bea8ee8a" path="/var/lib/kubelet/pods/1e003f91-c464-4200-94c1-18c2bea8ee8a/volumes" Dec 03 11:42:43 crc kubenswrapper[5052]: E1203 11:42:43.422533 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/combined-ca-bundle: secret "combined-ca-bundle" not found Dec 03 11:42:43 crc kubenswrapper[5052]: E1203 11:42:43.422834 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/222f033c-3660-4347-b621-7122c4c94929-combined-ca-bundle podName:222f033c-3660-4347-b621-7122c4c94929 nodeName:}" failed. No retries permitted until 2025-12-03 11:42:59.422813939 +0000 UTC m=+1370.011055703 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "combined-ca-bundle" (UniqueName: "kubernetes.io/secret/222f033c-3660-4347-b621-7122c4c94929-combined-ca-bundle") pod "barbican-worker-67c775d6bc-kwxzt" (UID: "222f033c-3660-4347-b621-7122c4c94929") : secret "combined-ca-bundle" not found Dec 03 11:42:43 crc kubenswrapper[5052]: E1203 11:42:43.423211 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/barbican-worker-config-data: secret "barbican-worker-config-data" not found Dec 03 11:42:43 crc kubenswrapper[5052]: E1203 11:42:43.423243 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/222f033c-3660-4347-b621-7122c4c94929-config-data-custom podName:222f033c-3660-4347-b621-7122c4c94929 nodeName:}" failed. No retries permitted until 2025-12-03 11:42:59.423235781 +0000 UTC m=+1370.011477545 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "config-data-custom" (UniqueName: "kubernetes.io/secret/222f033c-3660-4347-b621-7122c4c94929-config-data-custom") pod "barbican-worker-67c775d6bc-kwxzt" (UID: "222f033c-3660-4347-b621-7122c4c94929") : secret "barbican-worker-config-data" not found Dec 03 11:42:43 crc kubenswrapper[5052]: E1203 11:42:43.423568 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/barbican-config-data: secret "barbican-config-data" not found Dec 03 11:42:43 crc kubenswrapper[5052]: E1203 11:42:43.423614 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/222f033c-3660-4347-b621-7122c4c94929-config-data podName:222f033c-3660-4347-b621-7122c4c94929 nodeName:}" failed. No retries permitted until 2025-12-03 11:42:59.423602802 +0000 UTC m=+1370.011844566 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "config-data" (UniqueName: "kubernetes.io/secret/222f033c-3660-4347-b621-7122c4c94929-config-data") pod "barbican-worker-67c775d6bc-kwxzt" (UID: "222f033c-3660-4347-b621-7122c4c94929") : secret "barbican-config-data" not found Dec 03 11:42:43 crc kubenswrapper[5052]: I1203 11:42:43.469265 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/barbican-api-df67d6cb4-8sllj" podUID="b88af976-65f2-4d29-8c14-d5c43e23a459" containerName="barbican-api" probeResult="failure" output="Get \"https://10.217.0.191:9311/healthcheck\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Dec 03 11:42:43 crc kubenswrapper[5052]: I1203 11:42:43.469336 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/barbican-api-df67d6cb4-8sllj" podUID="b88af976-65f2-4d29-8c14-d5c43e23a459" containerName="barbican-api-log" probeResult="failure" output="Get \"https://10.217.0.191:9311/healthcheck\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Dec 03 11:42:44 crc kubenswrapper[5052]: I1203 11:42:44.582098 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/neutron-75475cc596-bfbzz" podUID="4a23886b-73c6-4b21-8464-07a2dd2e06f6" containerName="neutron-httpd" probeResult="failure" output="Get \"https://10.217.0.140:9696/\": dial tcp 10.217.0.140:9696: connect: connection refused" Dec 03 11:42:47 crc kubenswrapper[5052]: I1203 11:42:47.893045 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/barbican-api-df67d6cb4-8sllj" podUID="b88af976-65f2-4d29-8c14-d5c43e23a459" containerName="barbican-api-log" probeResult="failure" output="Get \"https://10.217.0.191:9311/healthcheck\": read tcp 10.217.0.2:38914->10.217.0.191:9311: read: connection reset by peer" Dec 03 11:42:47 crc kubenswrapper[5052]: I1203 11:42:47.893067 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/barbican-api-df67d6cb4-8sllj" podUID="b88af976-65f2-4d29-8c14-d5c43e23a459" containerName="barbican-api" probeResult="failure" output="Get \"https://10.217.0.191:9311/healthcheck\": read tcp 10.217.0.2:38922->10.217.0.191:9311: read: connection reset by peer" Dec 03 11:42:48 crc kubenswrapper[5052]: I1203 11:42:48.299520 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-api-df67d6cb4-8sllj" Dec 03 11:42:48 crc kubenswrapper[5052]: I1203 11:42:48.322667 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/b88af976-65f2-4d29-8c14-d5c43e23a459-config-data\") pod \"b88af976-65f2-4d29-8c14-d5c43e23a459\" (UID: \"b88af976-65f2-4d29-8c14-d5c43e23a459\") " Dec 03 11:42:48 crc kubenswrapper[5052]: I1203 11:42:48.324574 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/b88af976-65f2-4d29-8c14-d5c43e23a459-logs\") pod \"b88af976-65f2-4d29-8c14-d5c43e23a459\" (UID: \"b88af976-65f2-4d29-8c14-d5c43e23a459\") " Dec 03 11:42:48 crc kubenswrapper[5052]: I1203 11:42:48.324667 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vr472\" (UniqueName: \"kubernetes.io/projected/b88af976-65f2-4d29-8c14-d5c43e23a459-kube-api-access-vr472\") pod \"b88af976-65f2-4d29-8c14-d5c43e23a459\" (UID: \"b88af976-65f2-4d29-8c14-d5c43e23a459\") " Dec 03 11:42:48 crc kubenswrapper[5052]: I1203 11:42:48.324767 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/b88af976-65f2-4d29-8c14-d5c43e23a459-internal-tls-certs\") pod \"b88af976-65f2-4d29-8c14-d5c43e23a459\" (UID: \"b88af976-65f2-4d29-8c14-d5c43e23a459\") " Dec 03 11:42:48 crc kubenswrapper[5052]: I1203 11:42:48.324826 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b88af976-65f2-4d29-8c14-d5c43e23a459-combined-ca-bundle\") pod \"b88af976-65f2-4d29-8c14-d5c43e23a459\" (UID: \"b88af976-65f2-4d29-8c14-d5c43e23a459\") " Dec 03 11:42:48 crc kubenswrapper[5052]: I1203 11:42:48.324881 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/b88af976-65f2-4d29-8c14-d5c43e23a459-config-data-custom\") pod \"b88af976-65f2-4d29-8c14-d5c43e23a459\" (UID: \"b88af976-65f2-4d29-8c14-d5c43e23a459\") " Dec 03 11:42:48 crc kubenswrapper[5052]: I1203 11:42:48.324902 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/b88af976-65f2-4d29-8c14-d5c43e23a459-public-tls-certs\") pod \"b88af976-65f2-4d29-8c14-d5c43e23a459\" (UID: \"b88af976-65f2-4d29-8c14-d5c43e23a459\") " Dec 03 11:42:48 crc kubenswrapper[5052]: I1203 11:42:48.325418 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b88af976-65f2-4d29-8c14-d5c43e23a459-logs" (OuterVolumeSpecName: "logs") pod "b88af976-65f2-4d29-8c14-d5c43e23a459" (UID: "b88af976-65f2-4d29-8c14-d5c43e23a459"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:42:48 crc kubenswrapper[5052]: I1203 11:42:48.325968 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/b88af976-65f2-4d29-8c14-d5c43e23a459-logs\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:48 crc kubenswrapper[5052]: I1203 11:42:48.342565 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b88af976-65f2-4d29-8c14-d5c43e23a459-config-data-custom" (OuterVolumeSpecName: "config-data-custom") pod "b88af976-65f2-4d29-8c14-d5c43e23a459" (UID: "b88af976-65f2-4d29-8c14-d5c43e23a459"). InnerVolumeSpecName "config-data-custom". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:42:48 crc kubenswrapper[5052]: I1203 11:42:48.344156 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b88af976-65f2-4d29-8c14-d5c43e23a459-kube-api-access-vr472" (OuterVolumeSpecName: "kube-api-access-vr472") pod "b88af976-65f2-4d29-8c14-d5c43e23a459" (UID: "b88af976-65f2-4d29-8c14-d5c43e23a459"). InnerVolumeSpecName "kube-api-access-vr472". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:42:48 crc kubenswrapper[5052]: I1203 11:42:48.367410 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b88af976-65f2-4d29-8c14-d5c43e23a459-config-data" (OuterVolumeSpecName: "config-data") pod "b88af976-65f2-4d29-8c14-d5c43e23a459" (UID: "b88af976-65f2-4d29-8c14-d5c43e23a459"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:42:48 crc kubenswrapper[5052]: I1203 11:42:48.368471 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b88af976-65f2-4d29-8c14-d5c43e23a459-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "b88af976-65f2-4d29-8c14-d5c43e23a459" (UID: "b88af976-65f2-4d29-8c14-d5c43e23a459"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:42:48 crc kubenswrapper[5052]: I1203 11:42:48.369780 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b88af976-65f2-4d29-8c14-d5c43e23a459-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "b88af976-65f2-4d29-8c14-d5c43e23a459" (UID: "b88af976-65f2-4d29-8c14-d5c43e23a459"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:42:48 crc kubenswrapper[5052]: I1203 11:42:48.388657 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b88af976-65f2-4d29-8c14-d5c43e23a459-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "b88af976-65f2-4d29-8c14-d5c43e23a459" (UID: "b88af976-65f2-4d29-8c14-d5c43e23a459"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:42:48 crc kubenswrapper[5052]: I1203 11:42:48.427984 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/b88af976-65f2-4d29-8c14-d5c43e23a459-config-data-custom\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:48 crc kubenswrapper[5052]: I1203 11:42:48.428033 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/b88af976-65f2-4d29-8c14-d5c43e23a459-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:48 crc kubenswrapper[5052]: I1203 11:42:48.428046 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/b88af976-65f2-4d29-8c14-d5c43e23a459-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:48 crc kubenswrapper[5052]: I1203 11:42:48.428058 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-vr472\" (UniqueName: \"kubernetes.io/projected/b88af976-65f2-4d29-8c14-d5c43e23a459-kube-api-access-vr472\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:48 crc kubenswrapper[5052]: I1203 11:42:48.428092 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/b88af976-65f2-4d29-8c14-d5c43e23a459-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:48 crc kubenswrapper[5052]: I1203 11:42:48.428103 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b88af976-65f2-4d29-8c14-d5c43e23a459-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:48 crc kubenswrapper[5052]: I1203 11:42:48.761157 5052 generic.go:334] "Generic (PLEG): container finished" podID="b88af976-65f2-4d29-8c14-d5c43e23a459" containerID="293d2df95b90ded20b61e85a4c44a57977574cbb7ac21e2e8a6ba24672fee70b" exitCode=0 Dec 03 11:42:48 crc kubenswrapper[5052]: I1203 11:42:48.761204 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-df67d6cb4-8sllj" event={"ID":"b88af976-65f2-4d29-8c14-d5c43e23a459","Type":"ContainerDied","Data":"293d2df95b90ded20b61e85a4c44a57977574cbb7ac21e2e8a6ba24672fee70b"} Dec 03 11:42:48 crc kubenswrapper[5052]: I1203 11:42:48.761240 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-df67d6cb4-8sllj" event={"ID":"b88af976-65f2-4d29-8c14-d5c43e23a459","Type":"ContainerDied","Data":"8af6c176afbf0f72f2a163a363b1066519544bbb730d4dbb58a5ea5479468f2f"} Dec 03 11:42:48 crc kubenswrapper[5052]: I1203 11:42:48.761240 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-api-df67d6cb4-8sllj" Dec 03 11:42:48 crc kubenswrapper[5052]: I1203 11:42:48.761257 5052 scope.go:117] "RemoveContainer" containerID="293d2df95b90ded20b61e85a4c44a57977574cbb7ac21e2e8a6ba24672fee70b" Dec 03 11:42:48 crc kubenswrapper[5052]: I1203 11:42:48.787857 5052 scope.go:117] "RemoveContainer" containerID="07660c848ebf1a69fb37d115276c6d328ca3e8bccca35abc98bb76e21788b417" Dec 03 11:42:48 crc kubenswrapper[5052]: I1203 11:42:48.795858 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-api-df67d6cb4-8sllj"] Dec 03 11:42:48 crc kubenswrapper[5052]: I1203 11:42:48.803542 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/barbican-api-df67d6cb4-8sllj"] Dec 03 11:42:48 crc kubenswrapper[5052]: I1203 11:42:48.808671 5052 scope.go:117] "RemoveContainer" containerID="293d2df95b90ded20b61e85a4c44a57977574cbb7ac21e2e8a6ba24672fee70b" Dec 03 11:42:48 crc kubenswrapper[5052]: E1203 11:42:48.809060 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"293d2df95b90ded20b61e85a4c44a57977574cbb7ac21e2e8a6ba24672fee70b\": container with ID starting with 293d2df95b90ded20b61e85a4c44a57977574cbb7ac21e2e8a6ba24672fee70b not found: ID does not exist" containerID="293d2df95b90ded20b61e85a4c44a57977574cbb7ac21e2e8a6ba24672fee70b" Dec 03 11:42:48 crc kubenswrapper[5052]: I1203 11:42:48.809100 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"293d2df95b90ded20b61e85a4c44a57977574cbb7ac21e2e8a6ba24672fee70b"} err="failed to get container status \"293d2df95b90ded20b61e85a4c44a57977574cbb7ac21e2e8a6ba24672fee70b\": rpc error: code = NotFound desc = could not find container \"293d2df95b90ded20b61e85a4c44a57977574cbb7ac21e2e8a6ba24672fee70b\": container with ID starting with 293d2df95b90ded20b61e85a4c44a57977574cbb7ac21e2e8a6ba24672fee70b not found: ID does not exist" Dec 03 11:42:48 crc kubenswrapper[5052]: I1203 11:42:48.809133 5052 scope.go:117] "RemoveContainer" containerID="07660c848ebf1a69fb37d115276c6d328ca3e8bccca35abc98bb76e21788b417" Dec 03 11:42:48 crc kubenswrapper[5052]: E1203 11:42:48.809349 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"07660c848ebf1a69fb37d115276c6d328ca3e8bccca35abc98bb76e21788b417\": container with ID starting with 07660c848ebf1a69fb37d115276c6d328ca3e8bccca35abc98bb76e21788b417 not found: ID does not exist" containerID="07660c848ebf1a69fb37d115276c6d328ca3e8bccca35abc98bb76e21788b417" Dec 03 11:42:48 crc kubenswrapper[5052]: I1203 11:42:48.809370 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"07660c848ebf1a69fb37d115276c6d328ca3e8bccca35abc98bb76e21788b417"} err="failed to get container status \"07660c848ebf1a69fb37d115276c6d328ca3e8bccca35abc98bb76e21788b417\": rpc error: code = NotFound desc = could not find container \"07660c848ebf1a69fb37d115276c6d328ca3e8bccca35abc98bb76e21788b417\": container with ID starting with 07660c848ebf1a69fb37d115276c6d328ca3e8bccca35abc98bb76e21788b417 not found: ID does not exist" Dec 03 11:42:49 crc kubenswrapper[5052]: I1203 11:42:49.682062 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b88af976-65f2-4d29-8c14-d5c43e23a459" path="/var/lib/kubelet/pods/b88af976-65f2-4d29-8c14-d5c43e23a459/volumes" Dec 03 11:42:53 crc kubenswrapper[5052]: I1203 11:42:53.403700 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/neutron-69cf8cb7d6-jfb57" podUID="8f1442a2-a28b-4784-934a-34b417dbcce9" containerName="neutron-httpd" probeResult="failure" output="Get \"https://10.217.0.192:9696/\": dial tcp 10.217.0.192:9696: connect: connection refused" Dec 03 11:42:55 crc kubenswrapper[5052]: I1203 11:42:55.090396 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/keystone-6858c48994-zq757" Dec 03 11:42:56 crc kubenswrapper[5052]: I1203 11:42:56.520028 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_neutron-75475cc596-bfbzz_4a23886b-73c6-4b21-8464-07a2dd2e06f6/neutron-api/0.log" Dec 03 11:42:56 crc kubenswrapper[5052]: I1203 11:42:56.520628 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-75475cc596-bfbzz" Dec 03 11:42:56 crc kubenswrapper[5052]: I1203 11:42:56.570719 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4a23886b-73c6-4b21-8464-07a2dd2e06f6-combined-ca-bundle\") pod \"4a23886b-73c6-4b21-8464-07a2dd2e06f6\" (UID: \"4a23886b-73c6-4b21-8464-07a2dd2e06f6\") " Dec 03 11:42:56 crc kubenswrapper[5052]: I1203 11:42:56.570875 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/4a23886b-73c6-4b21-8464-07a2dd2e06f6-internal-tls-certs\") pod \"4a23886b-73c6-4b21-8464-07a2dd2e06f6\" (UID: \"4a23886b-73c6-4b21-8464-07a2dd2e06f6\") " Dec 03 11:42:56 crc kubenswrapper[5052]: I1203 11:42:56.570946 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zrlxq\" (UniqueName: \"kubernetes.io/projected/4a23886b-73c6-4b21-8464-07a2dd2e06f6-kube-api-access-zrlxq\") pod \"4a23886b-73c6-4b21-8464-07a2dd2e06f6\" (UID: \"4a23886b-73c6-4b21-8464-07a2dd2e06f6\") " Dec 03 11:42:56 crc kubenswrapper[5052]: I1203 11:42:56.570992 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/4a23886b-73c6-4b21-8464-07a2dd2e06f6-config\") pod \"4a23886b-73c6-4b21-8464-07a2dd2e06f6\" (UID: \"4a23886b-73c6-4b21-8464-07a2dd2e06f6\") " Dec 03 11:42:56 crc kubenswrapper[5052]: I1203 11:42:56.571021 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/4a23886b-73c6-4b21-8464-07a2dd2e06f6-httpd-config\") pod \"4a23886b-73c6-4b21-8464-07a2dd2e06f6\" (UID: \"4a23886b-73c6-4b21-8464-07a2dd2e06f6\") " Dec 03 11:42:56 crc kubenswrapper[5052]: I1203 11:42:56.571042 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/4a23886b-73c6-4b21-8464-07a2dd2e06f6-ovndb-tls-certs\") pod \"4a23886b-73c6-4b21-8464-07a2dd2e06f6\" (UID: \"4a23886b-73c6-4b21-8464-07a2dd2e06f6\") " Dec 03 11:42:56 crc kubenswrapper[5052]: I1203 11:42:56.571067 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/4a23886b-73c6-4b21-8464-07a2dd2e06f6-public-tls-certs\") pod \"4a23886b-73c6-4b21-8464-07a2dd2e06f6\" (UID: \"4a23886b-73c6-4b21-8464-07a2dd2e06f6\") " Dec 03 11:42:56 crc kubenswrapper[5052]: I1203 11:42:56.577596 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4a23886b-73c6-4b21-8464-07a2dd2e06f6-httpd-config" (OuterVolumeSpecName: "httpd-config") pod "4a23886b-73c6-4b21-8464-07a2dd2e06f6" (UID: "4a23886b-73c6-4b21-8464-07a2dd2e06f6"). InnerVolumeSpecName "httpd-config". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:42:56 crc kubenswrapper[5052]: I1203 11:42:56.585577 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4a23886b-73c6-4b21-8464-07a2dd2e06f6-kube-api-access-zrlxq" (OuterVolumeSpecName: "kube-api-access-zrlxq") pod "4a23886b-73c6-4b21-8464-07a2dd2e06f6" (UID: "4a23886b-73c6-4b21-8464-07a2dd2e06f6"). InnerVolumeSpecName "kube-api-access-zrlxq". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:42:56 crc kubenswrapper[5052]: I1203 11:42:56.616914 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4a23886b-73c6-4b21-8464-07a2dd2e06f6-config" (OuterVolumeSpecName: "config") pod "4a23886b-73c6-4b21-8464-07a2dd2e06f6" (UID: "4a23886b-73c6-4b21-8464-07a2dd2e06f6"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:42:56 crc kubenswrapper[5052]: I1203 11:42:56.620485 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4a23886b-73c6-4b21-8464-07a2dd2e06f6-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "4a23886b-73c6-4b21-8464-07a2dd2e06f6" (UID: "4a23886b-73c6-4b21-8464-07a2dd2e06f6"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:42:56 crc kubenswrapper[5052]: I1203 11:42:56.620804 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4a23886b-73c6-4b21-8464-07a2dd2e06f6-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "4a23886b-73c6-4b21-8464-07a2dd2e06f6" (UID: "4a23886b-73c6-4b21-8464-07a2dd2e06f6"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:42:56 crc kubenswrapper[5052]: I1203 11:42:56.631861 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4a23886b-73c6-4b21-8464-07a2dd2e06f6-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "4a23886b-73c6-4b21-8464-07a2dd2e06f6" (UID: "4a23886b-73c6-4b21-8464-07a2dd2e06f6"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:42:56 crc kubenswrapper[5052]: I1203 11:42:56.662589 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4a23886b-73c6-4b21-8464-07a2dd2e06f6-ovndb-tls-certs" (OuterVolumeSpecName: "ovndb-tls-certs") pod "4a23886b-73c6-4b21-8464-07a2dd2e06f6" (UID: "4a23886b-73c6-4b21-8464-07a2dd2e06f6"). InnerVolumeSpecName "ovndb-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:42:56 crc kubenswrapper[5052]: I1203 11:42:56.672623 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-zrlxq\" (UniqueName: \"kubernetes.io/projected/4a23886b-73c6-4b21-8464-07a2dd2e06f6-kube-api-access-zrlxq\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:56 crc kubenswrapper[5052]: I1203 11:42:56.672668 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/4a23886b-73c6-4b21-8464-07a2dd2e06f6-config\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:56 crc kubenswrapper[5052]: I1203 11:42:56.672685 5052 reconciler_common.go:293] "Volume detached for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/4a23886b-73c6-4b21-8464-07a2dd2e06f6-httpd-config\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:56 crc kubenswrapper[5052]: I1203 11:42:56.672699 5052 reconciler_common.go:293] "Volume detached for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/4a23886b-73c6-4b21-8464-07a2dd2e06f6-ovndb-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:56 crc kubenswrapper[5052]: I1203 11:42:56.672711 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/4a23886b-73c6-4b21-8464-07a2dd2e06f6-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:56 crc kubenswrapper[5052]: I1203 11:42:56.672723 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4a23886b-73c6-4b21-8464-07a2dd2e06f6-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:56 crc kubenswrapper[5052]: I1203 11:42:56.672735 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/4a23886b-73c6-4b21-8464-07a2dd2e06f6-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:56 crc kubenswrapper[5052]: I1203 11:42:56.828177 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_neutron-75475cc596-bfbzz_4a23886b-73c6-4b21-8464-07a2dd2e06f6/neutron-api/0.log" Dec 03 11:42:56 crc kubenswrapper[5052]: I1203 11:42:56.828747 5052 generic.go:334] "Generic (PLEG): container finished" podID="4a23886b-73c6-4b21-8464-07a2dd2e06f6" containerID="0aea4f7a00ba159f8d2bf6070cfb3c8fd85b362b1295e3d63d6e0afddf0002dd" exitCode=137 Dec 03 11:42:56 crc kubenswrapper[5052]: I1203 11:42:56.828792 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-75475cc596-bfbzz" event={"ID":"4a23886b-73c6-4b21-8464-07a2dd2e06f6","Type":"ContainerDied","Data":"0aea4f7a00ba159f8d2bf6070cfb3c8fd85b362b1295e3d63d6e0afddf0002dd"} Dec 03 11:42:56 crc kubenswrapper[5052]: I1203 11:42:56.828830 5052 scope.go:117] "RemoveContainer" containerID="f284580bf78ee1c66dcbbafaa358e1788c2f65a743bb73ce1b43008a67f4e28c" Dec 03 11:42:56 crc kubenswrapper[5052]: I1203 11:42:56.828868 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-75475cc596-bfbzz" event={"ID":"4a23886b-73c6-4b21-8464-07a2dd2e06f6","Type":"ContainerDied","Data":"018553d3b157f7ef0dd01fa9a27025af3476387607cfe61f6fb65382fd109a20"} Dec 03 11:42:56 crc kubenswrapper[5052]: I1203 11:42:56.829010 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-75475cc596-bfbzz" Dec 03 11:42:56 crc kubenswrapper[5052]: I1203 11:42:56.867787 5052 scope.go:117] "RemoveContainer" containerID="0aea4f7a00ba159f8d2bf6070cfb3c8fd85b362b1295e3d63d6e0afddf0002dd" Dec 03 11:42:56 crc kubenswrapper[5052]: I1203 11:42:56.872455 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/neutron-75475cc596-bfbzz"] Dec 03 11:42:56 crc kubenswrapper[5052]: I1203 11:42:56.878707 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/neutron-75475cc596-bfbzz"] Dec 03 11:42:56 crc kubenswrapper[5052]: I1203 11:42:56.899726 5052 scope.go:117] "RemoveContainer" containerID="f284580bf78ee1c66dcbbafaa358e1788c2f65a743bb73ce1b43008a67f4e28c" Dec 03 11:42:56 crc kubenswrapper[5052]: E1203 11:42:56.900408 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f284580bf78ee1c66dcbbafaa358e1788c2f65a743bb73ce1b43008a67f4e28c\": container with ID starting with f284580bf78ee1c66dcbbafaa358e1788c2f65a743bb73ce1b43008a67f4e28c not found: ID does not exist" containerID="f284580bf78ee1c66dcbbafaa358e1788c2f65a743bb73ce1b43008a67f4e28c" Dec 03 11:42:56 crc kubenswrapper[5052]: I1203 11:42:56.900444 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f284580bf78ee1c66dcbbafaa358e1788c2f65a743bb73ce1b43008a67f4e28c"} err="failed to get container status \"f284580bf78ee1c66dcbbafaa358e1788c2f65a743bb73ce1b43008a67f4e28c\": rpc error: code = NotFound desc = could not find container \"f284580bf78ee1c66dcbbafaa358e1788c2f65a743bb73ce1b43008a67f4e28c\": container with ID starting with f284580bf78ee1c66dcbbafaa358e1788c2f65a743bb73ce1b43008a67f4e28c not found: ID does not exist" Dec 03 11:42:56 crc kubenswrapper[5052]: I1203 11:42:56.900467 5052 scope.go:117] "RemoveContainer" containerID="0aea4f7a00ba159f8d2bf6070cfb3c8fd85b362b1295e3d63d6e0afddf0002dd" Dec 03 11:42:56 crc kubenswrapper[5052]: E1203 11:42:56.900758 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0aea4f7a00ba159f8d2bf6070cfb3c8fd85b362b1295e3d63d6e0afddf0002dd\": container with ID starting with 0aea4f7a00ba159f8d2bf6070cfb3c8fd85b362b1295e3d63d6e0afddf0002dd not found: ID does not exist" containerID="0aea4f7a00ba159f8d2bf6070cfb3c8fd85b362b1295e3d63d6e0afddf0002dd" Dec 03 11:42:56 crc kubenswrapper[5052]: I1203 11:42:56.900784 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0aea4f7a00ba159f8d2bf6070cfb3c8fd85b362b1295e3d63d6e0afddf0002dd"} err="failed to get container status \"0aea4f7a00ba159f8d2bf6070cfb3c8fd85b362b1295e3d63d6e0afddf0002dd\": rpc error: code = NotFound desc = could not find container \"0aea4f7a00ba159f8d2bf6070cfb3c8fd85b362b1295e3d63d6e0afddf0002dd\": container with ID starting with 0aea4f7a00ba159f8d2bf6070cfb3c8fd85b362b1295e3d63d6e0afddf0002dd not found: ID does not exist" Dec 03 11:42:57 crc kubenswrapper[5052]: I1203 11:42:57.626843 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-storage-0" Dec 03 11:42:57 crc kubenswrapper[5052]: I1203 11:42:57.685796 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4a23886b-73c6-4b21-8464-07a2dd2e06f6" path="/var/lib/kubelet/pods/4a23886b-73c6-4b21-8464-07a2dd2e06f6/volumes" Dec 03 11:42:57 crc kubenswrapper[5052]: I1203 11:42:57.688051 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"cache\" (UniqueName: \"kubernetes.io/empty-dir/d19f0c21-bb49-40a1-87fa-e55188308729-cache\") pod \"d19f0c21-bb49-40a1-87fa-e55188308729\" (UID: \"d19f0c21-bb49-40a1-87fa-e55188308729\") " Dec 03 11:42:57 crc kubenswrapper[5052]: I1203 11:42:57.688114 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"swift\" (UniqueName: \"kubernetes.io/local-volume/local-storage02-crc\") pod \"d19f0c21-bb49-40a1-87fa-e55188308729\" (UID: \"d19f0c21-bb49-40a1-87fa-e55188308729\") " Dec 03 11:42:57 crc kubenswrapper[5052]: I1203 11:42:57.688172 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zk5pc\" (UniqueName: \"kubernetes.io/projected/d19f0c21-bb49-40a1-87fa-e55188308729-kube-api-access-zk5pc\") pod \"d19f0c21-bb49-40a1-87fa-e55188308729\" (UID: \"d19f0c21-bb49-40a1-87fa-e55188308729\") " Dec 03 11:42:57 crc kubenswrapper[5052]: I1203 11:42:57.688245 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/d19f0c21-bb49-40a1-87fa-e55188308729-etc-swift\") pod \"d19f0c21-bb49-40a1-87fa-e55188308729\" (UID: \"d19f0c21-bb49-40a1-87fa-e55188308729\") " Dec 03 11:42:57 crc kubenswrapper[5052]: I1203 11:42:57.688286 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"lock\" (UniqueName: \"kubernetes.io/empty-dir/d19f0c21-bb49-40a1-87fa-e55188308729-lock\") pod \"d19f0c21-bb49-40a1-87fa-e55188308729\" (UID: \"d19f0c21-bb49-40a1-87fa-e55188308729\") " Dec 03 11:42:57 crc kubenswrapper[5052]: I1203 11:42:57.688809 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d19f0c21-bb49-40a1-87fa-e55188308729-cache" (OuterVolumeSpecName: "cache") pod "d19f0c21-bb49-40a1-87fa-e55188308729" (UID: "d19f0c21-bb49-40a1-87fa-e55188308729"). InnerVolumeSpecName "cache". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:42:57 crc kubenswrapper[5052]: I1203 11:42:57.689593 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d19f0c21-bb49-40a1-87fa-e55188308729-lock" (OuterVolumeSpecName: "lock") pod "d19f0c21-bb49-40a1-87fa-e55188308729" (UID: "d19f0c21-bb49-40a1-87fa-e55188308729"). InnerVolumeSpecName "lock". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:42:57 crc kubenswrapper[5052]: I1203 11:42:57.691993 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d19f0c21-bb49-40a1-87fa-e55188308729-etc-swift" (OuterVolumeSpecName: "etc-swift") pod "d19f0c21-bb49-40a1-87fa-e55188308729" (UID: "d19f0c21-bb49-40a1-87fa-e55188308729"). InnerVolumeSpecName "etc-swift". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:42:57 crc kubenswrapper[5052]: I1203 11:42:57.692090 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d19f0c21-bb49-40a1-87fa-e55188308729-kube-api-access-zk5pc" (OuterVolumeSpecName: "kube-api-access-zk5pc") pod "d19f0c21-bb49-40a1-87fa-e55188308729" (UID: "d19f0c21-bb49-40a1-87fa-e55188308729"). InnerVolumeSpecName "kube-api-access-zk5pc". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:42:57 crc kubenswrapper[5052]: I1203 11:42:57.693339 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage02-crc" (OuterVolumeSpecName: "swift") pod "d19f0c21-bb49-40a1-87fa-e55188308729" (UID: "d19f0c21-bb49-40a1-87fa-e55188308729"). InnerVolumeSpecName "local-storage02-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 11:42:57 crc kubenswrapper[5052]: I1203 11:42:57.790140 5052 reconciler_common.go:293] "Volume detached for volume \"cache\" (UniqueName: \"kubernetes.io/empty-dir/d19f0c21-bb49-40a1-87fa-e55188308729-cache\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:57 crc kubenswrapper[5052]: I1203 11:42:57.790173 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage02-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage02-crc\") on node \"crc\" " Dec 03 11:42:57 crc kubenswrapper[5052]: I1203 11:42:57.790184 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-zk5pc\" (UniqueName: \"kubernetes.io/projected/d19f0c21-bb49-40a1-87fa-e55188308729-kube-api-access-zk5pc\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:57 crc kubenswrapper[5052]: I1203 11:42:57.790194 5052 reconciler_common.go:293] "Volume detached for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/d19f0c21-bb49-40a1-87fa-e55188308729-etc-swift\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:57 crc kubenswrapper[5052]: I1203 11:42:57.790201 5052 reconciler_common.go:293] "Volume detached for volume \"lock\" (UniqueName: \"kubernetes.io/empty-dir/d19f0c21-bb49-40a1-87fa-e55188308729-lock\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:57 crc kubenswrapper[5052]: I1203 11:42:57.807860 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage02-crc" (UniqueName: "kubernetes.io/local-volume/local-storage02-crc") on node "crc" Dec 03 11:42:57 crc kubenswrapper[5052]: I1203 11:42:57.847940 5052 generic.go:334] "Generic (PLEG): container finished" podID="d19f0c21-bb49-40a1-87fa-e55188308729" containerID="907a694952444699abbc7ea53d63f1c5b7ce75550d76d99fb7d54e27cbbb8c66" exitCode=137 Dec 03 11:42:57 crc kubenswrapper[5052]: I1203 11:42:57.847991 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"d19f0c21-bb49-40a1-87fa-e55188308729","Type":"ContainerDied","Data":"907a694952444699abbc7ea53d63f1c5b7ce75550d76d99fb7d54e27cbbb8c66"} Dec 03 11:42:57 crc kubenswrapper[5052]: I1203 11:42:57.848026 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"d19f0c21-bb49-40a1-87fa-e55188308729","Type":"ContainerDied","Data":"78fdcedb4dfb761391bae389e6c4cba41223c19c5bee28700240078692e02f22"} Dec 03 11:42:57 crc kubenswrapper[5052]: I1203 11:42:57.848049 5052 scope.go:117] "RemoveContainer" containerID="907a694952444699abbc7ea53d63f1c5b7ce75550d76d99fb7d54e27cbbb8c66" Dec 03 11:42:57 crc kubenswrapper[5052]: I1203 11:42:57.848050 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-storage-0" Dec 03 11:42:57 crc kubenswrapper[5052]: I1203 11:42:57.875615 5052 scope.go:117] "RemoveContainer" containerID="198e5b3e40307e3ae065a4823c3cadc6edfe24c354d3dc8885bdeb1b65e4a57a" Dec 03 11:42:57 crc kubenswrapper[5052]: I1203 11:42:57.887806 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/swift-storage-0"] Dec 03 11:42:57 crc kubenswrapper[5052]: I1203 11:42:57.891746 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage02-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage02-crc\") on node \"crc\" DevicePath \"\"" Dec 03 11:42:57 crc kubenswrapper[5052]: I1203 11:42:57.894551 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/swift-storage-0"] Dec 03 11:42:57 crc kubenswrapper[5052]: I1203 11:42:57.903684 5052 scope.go:117] "RemoveContainer" containerID="1feeef96663c3d2882692e8837c421c0dab4a5a5be167acf6810b788e07033be" Dec 03 11:42:57 crc kubenswrapper[5052]: I1203 11:42:57.923553 5052 scope.go:117] "RemoveContainer" containerID="e0f3046cd638bd8d68fc0858b30e3c71c02254e1579ac5fdf0d14362c56787ae" Dec 03 11:42:57 crc kubenswrapper[5052]: I1203 11:42:57.941457 5052 scope.go:117] "RemoveContainer" containerID="842157c852d9af69e655dafb6e96753c92d5a174fac182e141f97a78bcaf8577" Dec 03 11:42:57 crc kubenswrapper[5052]: I1203 11:42:57.958496 5052 scope.go:117] "RemoveContainer" containerID="480a4695021f8caec2d8b5d9b753bac979ecb6a2f7643abdd8ce149daf4a8b4a" Dec 03 11:42:57 crc kubenswrapper[5052]: I1203 11:42:57.976945 5052 scope.go:117] "RemoveContainer" containerID="6f7206455d66da31b1ca0c6033a701e9ef605c548a9309c2b0967a2b9726a603" Dec 03 11:42:57 crc kubenswrapper[5052]: I1203 11:42:57.998572 5052 scope.go:117] "RemoveContainer" containerID="e36b32b951337695419521a1f3271a1e8e2cac46f10174c506c5e0e369ad8704" Dec 03 11:42:58 crc kubenswrapper[5052]: I1203 11:42:58.019338 5052 scope.go:117] "RemoveContainer" containerID="66b82b9a9602e546754f3199323f445795063d7d84d38d42c9e7bdead8e2d119" Dec 03 11:42:58 crc kubenswrapper[5052]: I1203 11:42:58.051027 5052 scope.go:117] "RemoveContainer" containerID="8b82c3eafedcad816bdd8f848dfa51930b6fcd64c66cf454cbb3af2debd04b86" Dec 03 11:42:58 crc kubenswrapper[5052]: I1203 11:42:58.070277 5052 scope.go:117] "RemoveContainer" containerID="cc5f444de12548f88c1caa6a61bd1efc6e2fd9da4c672919931aac6ba15d3f15" Dec 03 11:42:58 crc kubenswrapper[5052]: I1203 11:42:58.087228 5052 scope.go:117] "RemoveContainer" containerID="38627e147a9f39cc2d052969a6aa30eeb14f6d31573e540afa5859023f9656d8" Dec 03 11:42:58 crc kubenswrapper[5052]: I1203 11:42:58.104373 5052 scope.go:117] "RemoveContainer" containerID="e776b8b08a7136d5453955d041fce850605188fd0e17c51f2024905c9aa86aec" Dec 03 11:42:58 crc kubenswrapper[5052]: I1203 11:42:58.121272 5052 scope.go:117] "RemoveContainer" containerID="375e74dc879bff82efc908e9893ad30ab3c979218f143e6a3e6fcd9936d34248" Dec 03 11:42:58 crc kubenswrapper[5052]: I1203 11:42:58.137159 5052 scope.go:117] "RemoveContainer" containerID="3a64f478f235a4fb6291efc13b9a7d651e52032e9126c5a8ee70cbaa6f7bc0c4" Dec 03 11:42:58 crc kubenswrapper[5052]: I1203 11:42:58.155901 5052 scope.go:117] "RemoveContainer" containerID="907a694952444699abbc7ea53d63f1c5b7ce75550d76d99fb7d54e27cbbb8c66" Dec 03 11:42:58 crc kubenswrapper[5052]: E1203 11:42:58.156303 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"907a694952444699abbc7ea53d63f1c5b7ce75550d76d99fb7d54e27cbbb8c66\": container with ID starting with 907a694952444699abbc7ea53d63f1c5b7ce75550d76d99fb7d54e27cbbb8c66 not found: ID does not exist" containerID="907a694952444699abbc7ea53d63f1c5b7ce75550d76d99fb7d54e27cbbb8c66" Dec 03 11:42:58 crc kubenswrapper[5052]: I1203 11:42:58.156344 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"907a694952444699abbc7ea53d63f1c5b7ce75550d76d99fb7d54e27cbbb8c66"} err="failed to get container status \"907a694952444699abbc7ea53d63f1c5b7ce75550d76d99fb7d54e27cbbb8c66\": rpc error: code = NotFound desc = could not find container \"907a694952444699abbc7ea53d63f1c5b7ce75550d76d99fb7d54e27cbbb8c66\": container with ID starting with 907a694952444699abbc7ea53d63f1c5b7ce75550d76d99fb7d54e27cbbb8c66 not found: ID does not exist" Dec 03 11:42:58 crc kubenswrapper[5052]: I1203 11:42:58.156367 5052 scope.go:117] "RemoveContainer" containerID="198e5b3e40307e3ae065a4823c3cadc6edfe24c354d3dc8885bdeb1b65e4a57a" Dec 03 11:42:58 crc kubenswrapper[5052]: E1203 11:42:58.156793 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"198e5b3e40307e3ae065a4823c3cadc6edfe24c354d3dc8885bdeb1b65e4a57a\": container with ID starting with 198e5b3e40307e3ae065a4823c3cadc6edfe24c354d3dc8885bdeb1b65e4a57a not found: ID does not exist" containerID="198e5b3e40307e3ae065a4823c3cadc6edfe24c354d3dc8885bdeb1b65e4a57a" Dec 03 11:42:58 crc kubenswrapper[5052]: I1203 11:42:58.156848 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"198e5b3e40307e3ae065a4823c3cadc6edfe24c354d3dc8885bdeb1b65e4a57a"} err="failed to get container status \"198e5b3e40307e3ae065a4823c3cadc6edfe24c354d3dc8885bdeb1b65e4a57a\": rpc error: code = NotFound desc = could not find container \"198e5b3e40307e3ae065a4823c3cadc6edfe24c354d3dc8885bdeb1b65e4a57a\": container with ID starting with 198e5b3e40307e3ae065a4823c3cadc6edfe24c354d3dc8885bdeb1b65e4a57a not found: ID does not exist" Dec 03 11:42:58 crc kubenswrapper[5052]: I1203 11:42:58.156863 5052 scope.go:117] "RemoveContainer" containerID="1feeef96663c3d2882692e8837c421c0dab4a5a5be167acf6810b788e07033be" Dec 03 11:42:58 crc kubenswrapper[5052]: E1203 11:42:58.157235 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1feeef96663c3d2882692e8837c421c0dab4a5a5be167acf6810b788e07033be\": container with ID starting with 1feeef96663c3d2882692e8837c421c0dab4a5a5be167acf6810b788e07033be not found: ID does not exist" containerID="1feeef96663c3d2882692e8837c421c0dab4a5a5be167acf6810b788e07033be" Dec 03 11:42:58 crc kubenswrapper[5052]: I1203 11:42:58.157291 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1feeef96663c3d2882692e8837c421c0dab4a5a5be167acf6810b788e07033be"} err="failed to get container status \"1feeef96663c3d2882692e8837c421c0dab4a5a5be167acf6810b788e07033be\": rpc error: code = NotFound desc = could not find container \"1feeef96663c3d2882692e8837c421c0dab4a5a5be167acf6810b788e07033be\": container with ID starting with 1feeef96663c3d2882692e8837c421c0dab4a5a5be167acf6810b788e07033be not found: ID does not exist" Dec 03 11:42:58 crc kubenswrapper[5052]: I1203 11:42:58.157305 5052 scope.go:117] "RemoveContainer" containerID="e0f3046cd638bd8d68fc0858b30e3c71c02254e1579ac5fdf0d14362c56787ae" Dec 03 11:42:58 crc kubenswrapper[5052]: E1203 11:42:58.157658 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e0f3046cd638bd8d68fc0858b30e3c71c02254e1579ac5fdf0d14362c56787ae\": container with ID starting with e0f3046cd638bd8d68fc0858b30e3c71c02254e1579ac5fdf0d14362c56787ae not found: ID does not exist" containerID="e0f3046cd638bd8d68fc0858b30e3c71c02254e1579ac5fdf0d14362c56787ae" Dec 03 11:42:58 crc kubenswrapper[5052]: I1203 11:42:58.157682 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e0f3046cd638bd8d68fc0858b30e3c71c02254e1579ac5fdf0d14362c56787ae"} err="failed to get container status \"e0f3046cd638bd8d68fc0858b30e3c71c02254e1579ac5fdf0d14362c56787ae\": rpc error: code = NotFound desc = could not find container \"e0f3046cd638bd8d68fc0858b30e3c71c02254e1579ac5fdf0d14362c56787ae\": container with ID starting with e0f3046cd638bd8d68fc0858b30e3c71c02254e1579ac5fdf0d14362c56787ae not found: ID does not exist" Dec 03 11:42:58 crc kubenswrapper[5052]: I1203 11:42:58.157699 5052 scope.go:117] "RemoveContainer" containerID="842157c852d9af69e655dafb6e96753c92d5a174fac182e141f97a78bcaf8577" Dec 03 11:42:58 crc kubenswrapper[5052]: E1203 11:42:58.157981 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"842157c852d9af69e655dafb6e96753c92d5a174fac182e141f97a78bcaf8577\": container with ID starting with 842157c852d9af69e655dafb6e96753c92d5a174fac182e141f97a78bcaf8577 not found: ID does not exist" containerID="842157c852d9af69e655dafb6e96753c92d5a174fac182e141f97a78bcaf8577" Dec 03 11:42:58 crc kubenswrapper[5052]: I1203 11:42:58.158008 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"842157c852d9af69e655dafb6e96753c92d5a174fac182e141f97a78bcaf8577"} err="failed to get container status \"842157c852d9af69e655dafb6e96753c92d5a174fac182e141f97a78bcaf8577\": rpc error: code = NotFound desc = could not find container \"842157c852d9af69e655dafb6e96753c92d5a174fac182e141f97a78bcaf8577\": container with ID starting with 842157c852d9af69e655dafb6e96753c92d5a174fac182e141f97a78bcaf8577 not found: ID does not exist" Dec 03 11:42:58 crc kubenswrapper[5052]: I1203 11:42:58.158021 5052 scope.go:117] "RemoveContainer" containerID="480a4695021f8caec2d8b5d9b753bac979ecb6a2f7643abdd8ce149daf4a8b4a" Dec 03 11:42:58 crc kubenswrapper[5052]: E1203 11:42:58.158362 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"480a4695021f8caec2d8b5d9b753bac979ecb6a2f7643abdd8ce149daf4a8b4a\": container with ID starting with 480a4695021f8caec2d8b5d9b753bac979ecb6a2f7643abdd8ce149daf4a8b4a not found: ID does not exist" containerID="480a4695021f8caec2d8b5d9b753bac979ecb6a2f7643abdd8ce149daf4a8b4a" Dec 03 11:42:58 crc kubenswrapper[5052]: I1203 11:42:58.158389 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"480a4695021f8caec2d8b5d9b753bac979ecb6a2f7643abdd8ce149daf4a8b4a"} err="failed to get container status \"480a4695021f8caec2d8b5d9b753bac979ecb6a2f7643abdd8ce149daf4a8b4a\": rpc error: code = NotFound desc = could not find container \"480a4695021f8caec2d8b5d9b753bac979ecb6a2f7643abdd8ce149daf4a8b4a\": container with ID starting with 480a4695021f8caec2d8b5d9b753bac979ecb6a2f7643abdd8ce149daf4a8b4a not found: ID does not exist" Dec 03 11:42:58 crc kubenswrapper[5052]: I1203 11:42:58.158413 5052 scope.go:117] "RemoveContainer" containerID="6f7206455d66da31b1ca0c6033a701e9ef605c548a9309c2b0967a2b9726a603" Dec 03 11:42:58 crc kubenswrapper[5052]: E1203 11:42:58.158766 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6f7206455d66da31b1ca0c6033a701e9ef605c548a9309c2b0967a2b9726a603\": container with ID starting with 6f7206455d66da31b1ca0c6033a701e9ef605c548a9309c2b0967a2b9726a603 not found: ID does not exist" containerID="6f7206455d66da31b1ca0c6033a701e9ef605c548a9309c2b0967a2b9726a603" Dec 03 11:42:58 crc kubenswrapper[5052]: I1203 11:42:58.158807 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f7206455d66da31b1ca0c6033a701e9ef605c548a9309c2b0967a2b9726a603"} err="failed to get container status \"6f7206455d66da31b1ca0c6033a701e9ef605c548a9309c2b0967a2b9726a603\": rpc error: code = NotFound desc = could not find container \"6f7206455d66da31b1ca0c6033a701e9ef605c548a9309c2b0967a2b9726a603\": container with ID starting with 6f7206455d66da31b1ca0c6033a701e9ef605c548a9309c2b0967a2b9726a603 not found: ID does not exist" Dec 03 11:42:58 crc kubenswrapper[5052]: I1203 11:42:58.158823 5052 scope.go:117] "RemoveContainer" containerID="e36b32b951337695419521a1f3271a1e8e2cac46f10174c506c5e0e369ad8704" Dec 03 11:42:58 crc kubenswrapper[5052]: E1203 11:42:58.159240 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e36b32b951337695419521a1f3271a1e8e2cac46f10174c506c5e0e369ad8704\": container with ID starting with e36b32b951337695419521a1f3271a1e8e2cac46f10174c506c5e0e369ad8704 not found: ID does not exist" containerID="e36b32b951337695419521a1f3271a1e8e2cac46f10174c506c5e0e369ad8704" Dec 03 11:42:58 crc kubenswrapper[5052]: I1203 11:42:58.159269 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e36b32b951337695419521a1f3271a1e8e2cac46f10174c506c5e0e369ad8704"} err="failed to get container status \"e36b32b951337695419521a1f3271a1e8e2cac46f10174c506c5e0e369ad8704\": rpc error: code = NotFound desc = could not find container \"e36b32b951337695419521a1f3271a1e8e2cac46f10174c506c5e0e369ad8704\": container with ID starting with e36b32b951337695419521a1f3271a1e8e2cac46f10174c506c5e0e369ad8704 not found: ID does not exist" Dec 03 11:42:58 crc kubenswrapper[5052]: I1203 11:42:58.159288 5052 scope.go:117] "RemoveContainer" containerID="66b82b9a9602e546754f3199323f445795063d7d84d38d42c9e7bdead8e2d119" Dec 03 11:42:58 crc kubenswrapper[5052]: E1203 11:42:58.159567 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"66b82b9a9602e546754f3199323f445795063d7d84d38d42c9e7bdead8e2d119\": container with ID starting with 66b82b9a9602e546754f3199323f445795063d7d84d38d42c9e7bdead8e2d119 not found: ID does not exist" containerID="66b82b9a9602e546754f3199323f445795063d7d84d38d42c9e7bdead8e2d119" Dec 03 11:42:58 crc kubenswrapper[5052]: I1203 11:42:58.159595 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"66b82b9a9602e546754f3199323f445795063d7d84d38d42c9e7bdead8e2d119"} err="failed to get container status \"66b82b9a9602e546754f3199323f445795063d7d84d38d42c9e7bdead8e2d119\": rpc error: code = NotFound desc = could not find container \"66b82b9a9602e546754f3199323f445795063d7d84d38d42c9e7bdead8e2d119\": container with ID starting with 66b82b9a9602e546754f3199323f445795063d7d84d38d42c9e7bdead8e2d119 not found: ID does not exist" Dec 03 11:42:58 crc kubenswrapper[5052]: I1203 11:42:58.159611 5052 scope.go:117] "RemoveContainer" containerID="8b82c3eafedcad816bdd8f848dfa51930b6fcd64c66cf454cbb3af2debd04b86" Dec 03 11:42:58 crc kubenswrapper[5052]: E1203 11:42:58.159856 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8b82c3eafedcad816bdd8f848dfa51930b6fcd64c66cf454cbb3af2debd04b86\": container with ID starting with 8b82c3eafedcad816bdd8f848dfa51930b6fcd64c66cf454cbb3af2debd04b86 not found: ID does not exist" containerID="8b82c3eafedcad816bdd8f848dfa51930b6fcd64c66cf454cbb3af2debd04b86" Dec 03 11:42:58 crc kubenswrapper[5052]: I1203 11:42:58.159879 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8b82c3eafedcad816bdd8f848dfa51930b6fcd64c66cf454cbb3af2debd04b86"} err="failed to get container status \"8b82c3eafedcad816bdd8f848dfa51930b6fcd64c66cf454cbb3af2debd04b86\": rpc error: code = NotFound desc = could not find container \"8b82c3eafedcad816bdd8f848dfa51930b6fcd64c66cf454cbb3af2debd04b86\": container with ID starting with 8b82c3eafedcad816bdd8f848dfa51930b6fcd64c66cf454cbb3af2debd04b86 not found: ID does not exist" Dec 03 11:42:58 crc kubenswrapper[5052]: I1203 11:42:58.159895 5052 scope.go:117] "RemoveContainer" containerID="cc5f444de12548f88c1caa6a61bd1efc6e2fd9da4c672919931aac6ba15d3f15" Dec 03 11:42:58 crc kubenswrapper[5052]: E1203 11:42:58.160294 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cc5f444de12548f88c1caa6a61bd1efc6e2fd9da4c672919931aac6ba15d3f15\": container with ID starting with cc5f444de12548f88c1caa6a61bd1efc6e2fd9da4c672919931aac6ba15d3f15 not found: ID does not exist" containerID="cc5f444de12548f88c1caa6a61bd1efc6e2fd9da4c672919931aac6ba15d3f15" Dec 03 11:42:58 crc kubenswrapper[5052]: I1203 11:42:58.160324 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cc5f444de12548f88c1caa6a61bd1efc6e2fd9da4c672919931aac6ba15d3f15"} err="failed to get container status \"cc5f444de12548f88c1caa6a61bd1efc6e2fd9da4c672919931aac6ba15d3f15\": rpc error: code = NotFound desc = could not find container \"cc5f444de12548f88c1caa6a61bd1efc6e2fd9da4c672919931aac6ba15d3f15\": container with ID starting with cc5f444de12548f88c1caa6a61bd1efc6e2fd9da4c672919931aac6ba15d3f15 not found: ID does not exist" Dec 03 11:42:58 crc kubenswrapper[5052]: I1203 11:42:58.160342 5052 scope.go:117] "RemoveContainer" containerID="38627e147a9f39cc2d052969a6aa30eeb14f6d31573e540afa5859023f9656d8" Dec 03 11:42:58 crc kubenswrapper[5052]: E1203 11:42:58.160639 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"38627e147a9f39cc2d052969a6aa30eeb14f6d31573e540afa5859023f9656d8\": container with ID starting with 38627e147a9f39cc2d052969a6aa30eeb14f6d31573e540afa5859023f9656d8 not found: ID does not exist" containerID="38627e147a9f39cc2d052969a6aa30eeb14f6d31573e540afa5859023f9656d8" Dec 03 11:42:58 crc kubenswrapper[5052]: I1203 11:42:58.160673 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"38627e147a9f39cc2d052969a6aa30eeb14f6d31573e540afa5859023f9656d8"} err="failed to get container status \"38627e147a9f39cc2d052969a6aa30eeb14f6d31573e540afa5859023f9656d8\": rpc error: code = NotFound desc = could not find container \"38627e147a9f39cc2d052969a6aa30eeb14f6d31573e540afa5859023f9656d8\": container with ID starting with 38627e147a9f39cc2d052969a6aa30eeb14f6d31573e540afa5859023f9656d8 not found: ID does not exist" Dec 03 11:42:58 crc kubenswrapper[5052]: I1203 11:42:58.160687 5052 scope.go:117] "RemoveContainer" containerID="e776b8b08a7136d5453955d041fce850605188fd0e17c51f2024905c9aa86aec" Dec 03 11:42:58 crc kubenswrapper[5052]: E1203 11:42:58.160949 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e776b8b08a7136d5453955d041fce850605188fd0e17c51f2024905c9aa86aec\": container with ID starting with e776b8b08a7136d5453955d041fce850605188fd0e17c51f2024905c9aa86aec not found: ID does not exist" containerID="e776b8b08a7136d5453955d041fce850605188fd0e17c51f2024905c9aa86aec" Dec 03 11:42:58 crc kubenswrapper[5052]: I1203 11:42:58.160974 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e776b8b08a7136d5453955d041fce850605188fd0e17c51f2024905c9aa86aec"} err="failed to get container status \"e776b8b08a7136d5453955d041fce850605188fd0e17c51f2024905c9aa86aec\": rpc error: code = NotFound desc = could not find container \"e776b8b08a7136d5453955d041fce850605188fd0e17c51f2024905c9aa86aec\": container with ID starting with e776b8b08a7136d5453955d041fce850605188fd0e17c51f2024905c9aa86aec not found: ID does not exist" Dec 03 11:42:58 crc kubenswrapper[5052]: I1203 11:42:58.160992 5052 scope.go:117] "RemoveContainer" containerID="375e74dc879bff82efc908e9893ad30ab3c979218f143e6a3e6fcd9936d34248" Dec 03 11:42:58 crc kubenswrapper[5052]: E1203 11:42:58.161329 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"375e74dc879bff82efc908e9893ad30ab3c979218f143e6a3e6fcd9936d34248\": container with ID starting with 375e74dc879bff82efc908e9893ad30ab3c979218f143e6a3e6fcd9936d34248 not found: ID does not exist" containerID="375e74dc879bff82efc908e9893ad30ab3c979218f143e6a3e6fcd9936d34248" Dec 03 11:42:58 crc kubenswrapper[5052]: I1203 11:42:58.161351 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"375e74dc879bff82efc908e9893ad30ab3c979218f143e6a3e6fcd9936d34248"} err="failed to get container status \"375e74dc879bff82efc908e9893ad30ab3c979218f143e6a3e6fcd9936d34248\": rpc error: code = NotFound desc = could not find container \"375e74dc879bff82efc908e9893ad30ab3c979218f143e6a3e6fcd9936d34248\": container with ID starting with 375e74dc879bff82efc908e9893ad30ab3c979218f143e6a3e6fcd9936d34248 not found: ID does not exist" Dec 03 11:42:58 crc kubenswrapper[5052]: I1203 11:42:58.161363 5052 scope.go:117] "RemoveContainer" containerID="3a64f478f235a4fb6291efc13b9a7d651e52032e9126c5a8ee70cbaa6f7bc0c4" Dec 03 11:42:58 crc kubenswrapper[5052]: E1203 11:42:58.161603 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3a64f478f235a4fb6291efc13b9a7d651e52032e9126c5a8ee70cbaa6f7bc0c4\": container with ID starting with 3a64f478f235a4fb6291efc13b9a7d651e52032e9126c5a8ee70cbaa6f7bc0c4 not found: ID does not exist" containerID="3a64f478f235a4fb6291efc13b9a7d651e52032e9126c5a8ee70cbaa6f7bc0c4" Dec 03 11:42:58 crc kubenswrapper[5052]: I1203 11:42:58.161633 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3a64f478f235a4fb6291efc13b9a7d651e52032e9126c5a8ee70cbaa6f7bc0c4"} err="failed to get container status \"3a64f478f235a4fb6291efc13b9a7d651e52032e9126c5a8ee70cbaa6f7bc0c4\": rpc error: code = NotFound desc = could not find container \"3a64f478f235a4fb6291efc13b9a7d651e52032e9126c5a8ee70cbaa6f7bc0c4\": container with ID starting with 3a64f478f235a4fb6291efc13b9a7d651e52032e9126c5a8ee70cbaa6f7bc0c4 not found: ID does not exist" Dec 03 11:42:58 crc kubenswrapper[5052]: I1203 11:42:58.328388 5052 pod_container_manager_linux.go:210] "Failed to delete cgroup paths" cgroupName=["kubepods","besteffort","pod049153cf-b5da-4a61-8a2d-3f694625642b"] err="unable to destroy cgroup paths for cgroup [kubepods besteffort pod049153cf-b5da-4a61-8a2d-3f694625642b] : Timed out while waiting for systemd to remove kubepods-besteffort-pod049153cf_b5da_4a61_8a2d_3f694625642b.slice" Dec 03 11:42:58 crc kubenswrapper[5052]: E1203 11:42:58.328455 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to delete cgroup paths for [kubepods besteffort pod049153cf-b5da-4a61-8a2d-3f694625642b] : unable to destroy cgroup paths for cgroup [kubepods besteffort pod049153cf-b5da-4a61-8a2d-3f694625642b] : Timed out while waiting for systemd to remove kubepods-besteffort-pod049153cf_b5da_4a61_8a2d_3f694625642b.slice" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" podUID="049153cf-b5da-4a61-8a2d-3f694625642b" Dec 03 11:42:58 crc kubenswrapper[5052]: I1203 11:42:58.859417 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 11:42:58 crc kubenswrapper[5052]: I1203 11:42:58.877931 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 11:42:58 crc kubenswrapper[5052]: I1203 11:42:58.883537 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 11:42:59 crc kubenswrapper[5052]: E1203 11:42:59.516853 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/combined-ca-bundle: secret "combined-ca-bundle" not found Dec 03 11:42:59 crc kubenswrapper[5052]: E1203 11:42:59.517357 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/222f033c-3660-4347-b621-7122c4c94929-combined-ca-bundle podName:222f033c-3660-4347-b621-7122c4c94929 nodeName:}" failed. No retries permitted until 2025-12-03 11:43:31.517331616 +0000 UTC m=+1402.105573390 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "combined-ca-bundle" (UniqueName: "kubernetes.io/secret/222f033c-3660-4347-b621-7122c4c94929-combined-ca-bundle") pod "barbican-worker-67c775d6bc-kwxzt" (UID: "222f033c-3660-4347-b621-7122c4c94929") : secret "combined-ca-bundle" not found Dec 03 11:42:59 crc kubenswrapper[5052]: E1203 11:42:59.516966 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/barbican-config-data: secret "barbican-config-data" not found Dec 03 11:42:59 crc kubenswrapper[5052]: E1203 11:42:59.516972 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/barbican-worker-config-data: secret "barbican-worker-config-data" not found Dec 03 11:42:59 crc kubenswrapper[5052]: E1203 11:42:59.517409 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/222f033c-3660-4347-b621-7122c4c94929-config-data podName:222f033c-3660-4347-b621-7122c4c94929 nodeName:}" failed. No retries permitted until 2025-12-03 11:43:31.517401468 +0000 UTC m=+1402.105643292 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "config-data" (UniqueName: "kubernetes.io/secret/222f033c-3660-4347-b621-7122c4c94929-config-data") pod "barbican-worker-67c775d6bc-kwxzt" (UID: "222f033c-3660-4347-b621-7122c4c94929") : secret "barbican-config-data" not found Dec 03 11:42:59 crc kubenswrapper[5052]: E1203 11:42:59.517502 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/222f033c-3660-4347-b621-7122c4c94929-config-data-custom podName:222f033c-3660-4347-b621-7122c4c94929 nodeName:}" failed. No retries permitted until 2025-12-03 11:43:31.517483981 +0000 UTC m=+1402.105725735 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "config-data-custom" (UniqueName: "kubernetes.io/secret/222f033c-3660-4347-b621-7122c4c94929-config-data-custom") pod "barbican-worker-67c775d6bc-kwxzt" (UID: "222f033c-3660-4347-b621-7122c4c94929") : secret "barbican-worker-config-data" not found Dec 03 11:42:59 crc kubenswrapper[5052]: I1203 11:42:59.689668 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="049153cf-b5da-4a61-8a2d-3f694625642b" path="/var/lib/kubelet/pods/049153cf-b5da-4a61-8a2d-3f694625642b/volumes" Dec 03 11:42:59 crc kubenswrapper[5052]: I1203 11:42:59.690642 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d19f0c21-bb49-40a1-87fa-e55188308729" path="/var/lib/kubelet/pods/d19f0c21-bb49-40a1-87fa-e55188308729/volumes" Dec 03 11:43:00 crc kubenswrapper[5052]: I1203 11:43:00.325506 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-6858c48994-zq757" Dec 03 11:43:00 crc kubenswrapper[5052]: I1203 11:43:00.331651 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-keystone-listener-5dd6f4bfd-glsv7" Dec 03 11:43:00 crc kubenswrapper[5052]: I1203 11:43:00.429844 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/48409c36-734b-4b1d-adee-0e9981d635c9-public-tls-certs\") pod \"48409c36-734b-4b1d-adee-0e9981d635c9\" (UID: \"48409c36-734b-4b1d-adee-0e9981d635c9\") " Dec 03 11:43:00 crc kubenswrapper[5052]: I1203 11:43:00.429895 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/3004edac-eef4-4ddc-a539-4d4294029774-config-data\") pod \"3004edac-eef4-4ddc-a539-4d4294029774\" (UID: \"3004edac-eef4-4ddc-a539-4d4294029774\") " Dec 03 11:43:00 crc kubenswrapper[5052]: I1203 11:43:00.429942 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/3004edac-eef4-4ddc-a539-4d4294029774-logs\") pod \"3004edac-eef4-4ddc-a539-4d4294029774\" (UID: \"3004edac-eef4-4ddc-a539-4d4294029774\") " Dec 03 11:43:00 crc kubenswrapper[5052]: I1203 11:43:00.429998 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/48409c36-734b-4b1d-adee-0e9981d635c9-scripts\") pod \"48409c36-734b-4b1d-adee-0e9981d635c9\" (UID: \"48409c36-734b-4b1d-adee-0e9981d635c9\") " Dec 03 11:43:00 crc kubenswrapper[5052]: I1203 11:43:00.430029 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wzn2j\" (UniqueName: \"kubernetes.io/projected/3004edac-eef4-4ddc-a539-4d4294029774-kube-api-access-wzn2j\") pod \"3004edac-eef4-4ddc-a539-4d4294029774\" (UID: \"3004edac-eef4-4ddc-a539-4d4294029774\") " Dec 03 11:43:00 crc kubenswrapper[5052]: I1203 11:43:00.430056 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8r4bn\" (UniqueName: \"kubernetes.io/projected/48409c36-734b-4b1d-adee-0e9981d635c9-kube-api-access-8r4bn\") pod \"48409c36-734b-4b1d-adee-0e9981d635c9\" (UID: \"48409c36-734b-4b1d-adee-0e9981d635c9\") " Dec 03 11:43:00 crc kubenswrapper[5052]: I1203 11:43:00.430087 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3004edac-eef4-4ddc-a539-4d4294029774-combined-ca-bundle\") pod \"3004edac-eef4-4ddc-a539-4d4294029774\" (UID: \"3004edac-eef4-4ddc-a539-4d4294029774\") " Dec 03 11:43:00 crc kubenswrapper[5052]: I1203 11:43:00.430154 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/48409c36-734b-4b1d-adee-0e9981d635c9-credential-keys\") pod \"48409c36-734b-4b1d-adee-0e9981d635c9\" (UID: \"48409c36-734b-4b1d-adee-0e9981d635c9\") " Dec 03 11:43:00 crc kubenswrapper[5052]: I1203 11:43:00.430177 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/48409c36-734b-4b1d-adee-0e9981d635c9-config-data\") pod \"48409c36-734b-4b1d-adee-0e9981d635c9\" (UID: \"48409c36-734b-4b1d-adee-0e9981d635c9\") " Dec 03 11:43:00 crc kubenswrapper[5052]: I1203 11:43:00.430223 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/3004edac-eef4-4ddc-a539-4d4294029774-config-data-custom\") pod \"3004edac-eef4-4ddc-a539-4d4294029774\" (UID: \"3004edac-eef4-4ddc-a539-4d4294029774\") " Dec 03 11:43:00 crc kubenswrapper[5052]: I1203 11:43:00.430238 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/48409c36-734b-4b1d-adee-0e9981d635c9-combined-ca-bundle\") pod \"48409c36-734b-4b1d-adee-0e9981d635c9\" (UID: \"48409c36-734b-4b1d-adee-0e9981d635c9\") " Dec 03 11:43:00 crc kubenswrapper[5052]: I1203 11:43:00.430262 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/48409c36-734b-4b1d-adee-0e9981d635c9-internal-tls-certs\") pod \"48409c36-734b-4b1d-adee-0e9981d635c9\" (UID: \"48409c36-734b-4b1d-adee-0e9981d635c9\") " Dec 03 11:43:00 crc kubenswrapper[5052]: I1203 11:43:00.430282 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/48409c36-734b-4b1d-adee-0e9981d635c9-fernet-keys\") pod \"48409c36-734b-4b1d-adee-0e9981d635c9\" (UID: \"48409c36-734b-4b1d-adee-0e9981d635c9\") " Dec 03 11:43:00 crc kubenswrapper[5052]: I1203 11:43:00.432953 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3004edac-eef4-4ddc-a539-4d4294029774-logs" (OuterVolumeSpecName: "logs") pod "3004edac-eef4-4ddc-a539-4d4294029774" (UID: "3004edac-eef4-4ddc-a539-4d4294029774"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:43:00 crc kubenswrapper[5052]: I1203 11:43:00.443989 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/48409c36-734b-4b1d-adee-0e9981d635c9-kube-api-access-8r4bn" (OuterVolumeSpecName: "kube-api-access-8r4bn") pod "48409c36-734b-4b1d-adee-0e9981d635c9" (UID: "48409c36-734b-4b1d-adee-0e9981d635c9"). InnerVolumeSpecName "kube-api-access-8r4bn". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:43:00 crc kubenswrapper[5052]: I1203 11:43:00.443985 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/48409c36-734b-4b1d-adee-0e9981d635c9-credential-keys" (OuterVolumeSpecName: "credential-keys") pod "48409c36-734b-4b1d-adee-0e9981d635c9" (UID: "48409c36-734b-4b1d-adee-0e9981d635c9"). InnerVolumeSpecName "credential-keys". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:43:00 crc kubenswrapper[5052]: I1203 11:43:00.446774 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3004edac-eef4-4ddc-a539-4d4294029774-config-data-custom" (OuterVolumeSpecName: "config-data-custom") pod "3004edac-eef4-4ddc-a539-4d4294029774" (UID: "3004edac-eef4-4ddc-a539-4d4294029774"). InnerVolumeSpecName "config-data-custom". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:43:00 crc kubenswrapper[5052]: I1203 11:43:00.446791 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3004edac-eef4-4ddc-a539-4d4294029774-kube-api-access-wzn2j" (OuterVolumeSpecName: "kube-api-access-wzn2j") pod "3004edac-eef4-4ddc-a539-4d4294029774" (UID: "3004edac-eef4-4ddc-a539-4d4294029774"). InnerVolumeSpecName "kube-api-access-wzn2j". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:43:00 crc kubenswrapper[5052]: I1203 11:43:00.463321 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/48409c36-734b-4b1d-adee-0e9981d635c9-fernet-keys" (OuterVolumeSpecName: "fernet-keys") pod "48409c36-734b-4b1d-adee-0e9981d635c9" (UID: "48409c36-734b-4b1d-adee-0e9981d635c9"). InnerVolumeSpecName "fernet-keys". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:43:00 crc kubenswrapper[5052]: I1203 11:43:00.467539 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/48409c36-734b-4b1d-adee-0e9981d635c9-scripts" (OuterVolumeSpecName: "scripts") pod "48409c36-734b-4b1d-adee-0e9981d635c9" (UID: "48409c36-734b-4b1d-adee-0e9981d635c9"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:43:00 crc kubenswrapper[5052]: I1203 11:43:00.483936 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/48409c36-734b-4b1d-adee-0e9981d635c9-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "48409c36-734b-4b1d-adee-0e9981d635c9" (UID: "48409c36-734b-4b1d-adee-0e9981d635c9"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:43:00 crc kubenswrapper[5052]: I1203 11:43:00.488406 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3004edac-eef4-4ddc-a539-4d4294029774-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "3004edac-eef4-4ddc-a539-4d4294029774" (UID: "3004edac-eef4-4ddc-a539-4d4294029774"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:43:00 crc kubenswrapper[5052]: I1203 11:43:00.492502 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/48409c36-734b-4b1d-adee-0e9981d635c9-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "48409c36-734b-4b1d-adee-0e9981d635c9" (UID: "48409c36-734b-4b1d-adee-0e9981d635c9"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:43:00 crc kubenswrapper[5052]: I1203 11:43:00.495381 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/48409c36-734b-4b1d-adee-0e9981d635c9-config-data" (OuterVolumeSpecName: "config-data") pod "48409c36-734b-4b1d-adee-0e9981d635c9" (UID: "48409c36-734b-4b1d-adee-0e9981d635c9"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:43:00 crc kubenswrapper[5052]: I1203 11:43:00.505441 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3004edac-eef4-4ddc-a539-4d4294029774-config-data" (OuterVolumeSpecName: "config-data") pod "3004edac-eef4-4ddc-a539-4d4294029774" (UID: "3004edac-eef4-4ddc-a539-4d4294029774"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:43:00 crc kubenswrapper[5052]: I1203 11:43:00.505982 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/48409c36-734b-4b1d-adee-0e9981d635c9-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "48409c36-734b-4b1d-adee-0e9981d635c9" (UID: "48409c36-734b-4b1d-adee-0e9981d635c9"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:43:00 crc kubenswrapper[5052]: I1203 11:43:00.540436 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3004edac-eef4-4ddc-a539-4d4294029774-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:43:00 crc kubenswrapper[5052]: I1203 11:43:00.540475 5052 reconciler_common.go:293] "Volume detached for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/48409c36-734b-4b1d-adee-0e9981d635c9-credential-keys\") on node \"crc\" DevicePath \"\"" Dec 03 11:43:00 crc kubenswrapper[5052]: I1203 11:43:00.540485 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/48409c36-734b-4b1d-adee-0e9981d635c9-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:43:00 crc kubenswrapper[5052]: I1203 11:43:00.540495 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/3004edac-eef4-4ddc-a539-4d4294029774-config-data-custom\") on node \"crc\" DevicePath \"\"" Dec 03 11:43:00 crc kubenswrapper[5052]: I1203 11:43:00.540507 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/48409c36-734b-4b1d-adee-0e9981d635c9-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:43:00 crc kubenswrapper[5052]: I1203 11:43:00.540530 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/48409c36-734b-4b1d-adee-0e9981d635c9-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 11:43:00 crc kubenswrapper[5052]: I1203 11:43:00.540539 5052 reconciler_common.go:293] "Volume detached for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/48409c36-734b-4b1d-adee-0e9981d635c9-fernet-keys\") on node \"crc\" DevicePath \"\"" Dec 03 11:43:00 crc kubenswrapper[5052]: I1203 11:43:00.540548 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/48409c36-734b-4b1d-adee-0e9981d635c9-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 11:43:00 crc kubenswrapper[5052]: I1203 11:43:00.540557 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/3004edac-eef4-4ddc-a539-4d4294029774-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:43:00 crc kubenswrapper[5052]: I1203 11:43:00.540568 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/3004edac-eef4-4ddc-a539-4d4294029774-logs\") on node \"crc\" DevicePath \"\"" Dec 03 11:43:00 crc kubenswrapper[5052]: I1203 11:43:00.540577 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/48409c36-734b-4b1d-adee-0e9981d635c9-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:43:00 crc kubenswrapper[5052]: I1203 11:43:00.540587 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-wzn2j\" (UniqueName: \"kubernetes.io/projected/3004edac-eef4-4ddc-a539-4d4294029774-kube-api-access-wzn2j\") on node \"crc\" DevicePath \"\"" Dec 03 11:43:00 crc kubenswrapper[5052]: I1203 11:43:00.540601 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-8r4bn\" (UniqueName: \"kubernetes.io/projected/48409c36-734b-4b1d-adee-0e9981d635c9-kube-api-access-8r4bn\") on node \"crc\" DevicePath \"\"" Dec 03 11:43:00 crc kubenswrapper[5052]: I1203 11:43:00.562492 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-worker-67c775d6bc-kwxzt" Dec 03 11:43:00 crc kubenswrapper[5052]: I1203 11:43:00.641915 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zqmkd\" (UniqueName: \"kubernetes.io/projected/222f033c-3660-4347-b621-7122c4c94929-kube-api-access-zqmkd\") pod \"222f033c-3660-4347-b621-7122c4c94929\" (UID: \"222f033c-3660-4347-b621-7122c4c94929\") " Dec 03 11:43:00 crc kubenswrapper[5052]: I1203 11:43:00.641973 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/222f033c-3660-4347-b621-7122c4c94929-logs\") pod \"222f033c-3660-4347-b621-7122c4c94929\" (UID: \"222f033c-3660-4347-b621-7122c4c94929\") " Dec 03 11:43:00 crc kubenswrapper[5052]: I1203 11:43:00.642038 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/222f033c-3660-4347-b621-7122c4c94929-combined-ca-bundle\") pod \"222f033c-3660-4347-b621-7122c4c94929\" (UID: \"222f033c-3660-4347-b621-7122c4c94929\") " Dec 03 11:43:00 crc kubenswrapper[5052]: I1203 11:43:00.642133 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/222f033c-3660-4347-b621-7122c4c94929-config-data-custom\") pod \"222f033c-3660-4347-b621-7122c4c94929\" (UID: \"222f033c-3660-4347-b621-7122c4c94929\") " Dec 03 11:43:00 crc kubenswrapper[5052]: I1203 11:43:00.642187 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/222f033c-3660-4347-b621-7122c4c94929-config-data\") pod \"222f033c-3660-4347-b621-7122c4c94929\" (UID: \"222f033c-3660-4347-b621-7122c4c94929\") " Dec 03 11:43:00 crc kubenswrapper[5052]: I1203 11:43:00.642881 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/222f033c-3660-4347-b621-7122c4c94929-logs" (OuterVolumeSpecName: "logs") pod "222f033c-3660-4347-b621-7122c4c94929" (UID: "222f033c-3660-4347-b621-7122c4c94929"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:43:00 crc kubenswrapper[5052]: I1203 11:43:00.645189 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/222f033c-3660-4347-b621-7122c4c94929-config-data-custom" (OuterVolumeSpecName: "config-data-custom") pod "222f033c-3660-4347-b621-7122c4c94929" (UID: "222f033c-3660-4347-b621-7122c4c94929"). InnerVolumeSpecName "config-data-custom". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:43:00 crc kubenswrapper[5052]: I1203 11:43:00.645399 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/222f033c-3660-4347-b621-7122c4c94929-kube-api-access-zqmkd" (OuterVolumeSpecName: "kube-api-access-zqmkd") pod "222f033c-3660-4347-b621-7122c4c94929" (UID: "222f033c-3660-4347-b621-7122c4c94929"). InnerVolumeSpecName "kube-api-access-zqmkd". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:43:00 crc kubenswrapper[5052]: I1203 11:43:00.660132 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/222f033c-3660-4347-b621-7122c4c94929-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "222f033c-3660-4347-b621-7122c4c94929" (UID: "222f033c-3660-4347-b621-7122c4c94929"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:43:00 crc kubenswrapper[5052]: I1203 11:43:00.678786 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/222f033c-3660-4347-b621-7122c4c94929-config-data" (OuterVolumeSpecName: "config-data") pod "222f033c-3660-4347-b621-7122c4c94929" (UID: "222f033c-3660-4347-b621-7122c4c94929"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:43:00 crc kubenswrapper[5052]: I1203 11:43:00.743994 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/222f033c-3660-4347-b621-7122c4c94929-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:43:00 crc kubenswrapper[5052]: I1203 11:43:00.744055 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/222f033c-3660-4347-b621-7122c4c94929-config-data-custom\") on node \"crc\" DevicePath \"\"" Dec 03 11:43:00 crc kubenswrapper[5052]: I1203 11:43:00.744074 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/222f033c-3660-4347-b621-7122c4c94929-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:43:00 crc kubenswrapper[5052]: I1203 11:43:00.744092 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-zqmkd\" (UniqueName: \"kubernetes.io/projected/222f033c-3660-4347-b621-7122c4c94929-kube-api-access-zqmkd\") on node \"crc\" DevicePath \"\"" Dec 03 11:43:00 crc kubenswrapper[5052]: I1203 11:43:00.744106 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/222f033c-3660-4347-b621-7122c4c94929-logs\") on node \"crc\" DevicePath \"\"" Dec 03 11:43:00 crc kubenswrapper[5052]: I1203 11:43:00.889908 5052 generic.go:334] "Generic (PLEG): container finished" podID="222f033c-3660-4347-b621-7122c4c94929" containerID="e34cd5241cdc0d7ab2c3e9a3005641eb15aa576865c7a50550043b4722e25959" exitCode=137 Dec 03 11:43:00 crc kubenswrapper[5052]: I1203 11:43:00.889997 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-worker-67c775d6bc-kwxzt" Dec 03 11:43:00 crc kubenswrapper[5052]: I1203 11:43:00.890001 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-worker-67c775d6bc-kwxzt" event={"ID":"222f033c-3660-4347-b621-7122c4c94929","Type":"ContainerDied","Data":"e34cd5241cdc0d7ab2c3e9a3005641eb15aa576865c7a50550043b4722e25959"} Dec 03 11:43:00 crc kubenswrapper[5052]: I1203 11:43:00.890128 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-worker-67c775d6bc-kwxzt" event={"ID":"222f033c-3660-4347-b621-7122c4c94929","Type":"ContainerDied","Data":"a5ac6cc1055df402445233edeff6fd6db77ec35aa50f1a126fefc71ff474afe2"} Dec 03 11:43:00 crc kubenswrapper[5052]: I1203 11:43:00.890152 5052 scope.go:117] "RemoveContainer" containerID="e34cd5241cdc0d7ab2c3e9a3005641eb15aa576865c7a50550043b4722e25959" Dec 03 11:43:00 crc kubenswrapper[5052]: I1203 11:43:00.893499 5052 generic.go:334] "Generic (PLEG): container finished" podID="3004edac-eef4-4ddc-a539-4d4294029774" containerID="4bf71fc6979a9fd31b45238a27ca6a41469062f0e760d70053f9813a19c47d57" exitCode=137 Dec 03 11:43:00 crc kubenswrapper[5052]: I1203 11:43:00.893565 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-keystone-listener-5dd6f4bfd-glsv7" event={"ID":"3004edac-eef4-4ddc-a539-4d4294029774","Type":"ContainerDied","Data":"4bf71fc6979a9fd31b45238a27ca6a41469062f0e760d70053f9813a19c47d57"} Dec 03 11:43:00 crc kubenswrapper[5052]: I1203 11:43:00.893585 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-keystone-listener-5dd6f4bfd-glsv7" event={"ID":"3004edac-eef4-4ddc-a539-4d4294029774","Type":"ContainerDied","Data":"155d312d1a668b19eb1a81517ac71e09163d5f2409e7b82778a1605373d5fc62"} Dec 03 11:43:00 crc kubenswrapper[5052]: I1203 11:43:00.893590 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-keystone-listener-5dd6f4bfd-glsv7" Dec 03 11:43:00 crc kubenswrapper[5052]: I1203 11:43:00.896736 5052 generic.go:334] "Generic (PLEG): container finished" podID="48409c36-734b-4b1d-adee-0e9981d635c9" containerID="a8c3f8ba64e0fed8deeb5029931d97733022d2a5ae3a2c13c54ed87abfa82462" exitCode=137 Dec 03 11:43:00 crc kubenswrapper[5052]: I1203 11:43:00.896788 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-6858c48994-zq757" event={"ID":"48409c36-734b-4b1d-adee-0e9981d635c9","Type":"ContainerDied","Data":"a8c3f8ba64e0fed8deeb5029931d97733022d2a5ae3a2c13c54ed87abfa82462"} Dec 03 11:43:00 crc kubenswrapper[5052]: I1203 11:43:00.896814 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-6858c48994-zq757" Dec 03 11:43:00 crc kubenswrapper[5052]: I1203 11:43:00.896819 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-6858c48994-zq757" event={"ID":"48409c36-734b-4b1d-adee-0e9981d635c9","Type":"ContainerDied","Data":"85e4be2eed800fd86dad67db18b568783e795640a6b911659efa1aaed0d360fb"} Dec 03 11:43:00 crc kubenswrapper[5052]: I1203 11:43:00.921712 5052 scope.go:117] "RemoveContainer" containerID="9e7335aefbd832cdb40aea97b2f03dec68bab190e8c59ded23cf65ab50caa997" Dec 03 11:43:00 crc kubenswrapper[5052]: I1203 11:43:00.940166 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-worker-67c775d6bc-kwxzt"] Dec 03 11:43:00 crc kubenswrapper[5052]: I1203 11:43:00.948844 5052 scope.go:117] "RemoveContainer" containerID="e34cd5241cdc0d7ab2c3e9a3005641eb15aa576865c7a50550043b4722e25959" Dec 03 11:43:00 crc kubenswrapper[5052]: E1203 11:43:00.949898 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e34cd5241cdc0d7ab2c3e9a3005641eb15aa576865c7a50550043b4722e25959\": container with ID starting with e34cd5241cdc0d7ab2c3e9a3005641eb15aa576865c7a50550043b4722e25959 not found: ID does not exist" containerID="e34cd5241cdc0d7ab2c3e9a3005641eb15aa576865c7a50550043b4722e25959" Dec 03 11:43:00 crc kubenswrapper[5052]: I1203 11:43:00.949929 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e34cd5241cdc0d7ab2c3e9a3005641eb15aa576865c7a50550043b4722e25959"} err="failed to get container status \"e34cd5241cdc0d7ab2c3e9a3005641eb15aa576865c7a50550043b4722e25959\": rpc error: code = NotFound desc = could not find container \"e34cd5241cdc0d7ab2c3e9a3005641eb15aa576865c7a50550043b4722e25959\": container with ID starting with e34cd5241cdc0d7ab2c3e9a3005641eb15aa576865c7a50550043b4722e25959 not found: ID does not exist" Dec 03 11:43:00 crc kubenswrapper[5052]: I1203 11:43:00.949950 5052 scope.go:117] "RemoveContainer" containerID="9e7335aefbd832cdb40aea97b2f03dec68bab190e8c59ded23cf65ab50caa997" Dec 03 11:43:00 crc kubenswrapper[5052]: E1203 11:43:00.950134 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9e7335aefbd832cdb40aea97b2f03dec68bab190e8c59ded23cf65ab50caa997\": container with ID starting with 9e7335aefbd832cdb40aea97b2f03dec68bab190e8c59ded23cf65ab50caa997 not found: ID does not exist" containerID="9e7335aefbd832cdb40aea97b2f03dec68bab190e8c59ded23cf65ab50caa997" Dec 03 11:43:00 crc kubenswrapper[5052]: I1203 11:43:00.950151 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9e7335aefbd832cdb40aea97b2f03dec68bab190e8c59ded23cf65ab50caa997"} err="failed to get container status \"9e7335aefbd832cdb40aea97b2f03dec68bab190e8c59ded23cf65ab50caa997\": rpc error: code = NotFound desc = could not find container \"9e7335aefbd832cdb40aea97b2f03dec68bab190e8c59ded23cf65ab50caa997\": container with ID starting with 9e7335aefbd832cdb40aea97b2f03dec68bab190e8c59ded23cf65ab50caa997 not found: ID does not exist" Dec 03 11:43:00 crc kubenswrapper[5052]: I1203 11:43:00.950162 5052 scope.go:117] "RemoveContainer" containerID="4bf71fc6979a9fd31b45238a27ca6a41469062f0e760d70053f9813a19c47d57" Dec 03 11:43:00 crc kubenswrapper[5052]: I1203 11:43:00.954613 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/barbican-worker-67c775d6bc-kwxzt"] Dec 03 11:43:00 crc kubenswrapper[5052]: I1203 11:43:00.963483 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/keystone-6858c48994-zq757"] Dec 03 11:43:00 crc kubenswrapper[5052]: I1203 11:43:00.970153 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/keystone-6858c48994-zq757"] Dec 03 11:43:00 crc kubenswrapper[5052]: I1203 11:43:00.976422 5052 scope.go:117] "RemoveContainer" containerID="e7be74f05a4bcf8543f0c42a95e74e455c72cfb86b1351dc8a08f45d49fb16ab" Dec 03 11:43:00 crc kubenswrapper[5052]: I1203 11:43:00.976570 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-keystone-listener-5dd6f4bfd-glsv7"] Dec 03 11:43:00 crc kubenswrapper[5052]: I1203 11:43:00.984011 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/barbican-keystone-listener-5dd6f4bfd-glsv7"] Dec 03 11:43:00 crc kubenswrapper[5052]: I1203 11:43:00.999384 5052 scope.go:117] "RemoveContainer" containerID="4bf71fc6979a9fd31b45238a27ca6a41469062f0e760d70053f9813a19c47d57" Dec 03 11:43:01 crc kubenswrapper[5052]: E1203 11:43:01.000745 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4bf71fc6979a9fd31b45238a27ca6a41469062f0e760d70053f9813a19c47d57\": container with ID starting with 4bf71fc6979a9fd31b45238a27ca6a41469062f0e760d70053f9813a19c47d57 not found: ID does not exist" containerID="4bf71fc6979a9fd31b45238a27ca6a41469062f0e760d70053f9813a19c47d57" Dec 03 11:43:01 crc kubenswrapper[5052]: I1203 11:43:01.000793 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4bf71fc6979a9fd31b45238a27ca6a41469062f0e760d70053f9813a19c47d57"} err="failed to get container status \"4bf71fc6979a9fd31b45238a27ca6a41469062f0e760d70053f9813a19c47d57\": rpc error: code = NotFound desc = could not find container \"4bf71fc6979a9fd31b45238a27ca6a41469062f0e760d70053f9813a19c47d57\": container with ID starting with 4bf71fc6979a9fd31b45238a27ca6a41469062f0e760d70053f9813a19c47d57 not found: ID does not exist" Dec 03 11:43:01 crc kubenswrapper[5052]: I1203 11:43:01.000864 5052 scope.go:117] "RemoveContainer" containerID="e7be74f05a4bcf8543f0c42a95e74e455c72cfb86b1351dc8a08f45d49fb16ab" Dec 03 11:43:01 crc kubenswrapper[5052]: E1203 11:43:01.001817 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e7be74f05a4bcf8543f0c42a95e74e455c72cfb86b1351dc8a08f45d49fb16ab\": container with ID starting with e7be74f05a4bcf8543f0c42a95e74e455c72cfb86b1351dc8a08f45d49fb16ab not found: ID does not exist" containerID="e7be74f05a4bcf8543f0c42a95e74e455c72cfb86b1351dc8a08f45d49fb16ab" Dec 03 11:43:01 crc kubenswrapper[5052]: I1203 11:43:01.001842 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e7be74f05a4bcf8543f0c42a95e74e455c72cfb86b1351dc8a08f45d49fb16ab"} err="failed to get container status \"e7be74f05a4bcf8543f0c42a95e74e455c72cfb86b1351dc8a08f45d49fb16ab\": rpc error: code = NotFound desc = could not find container \"e7be74f05a4bcf8543f0c42a95e74e455c72cfb86b1351dc8a08f45d49fb16ab\": container with ID starting with e7be74f05a4bcf8543f0c42a95e74e455c72cfb86b1351dc8a08f45d49fb16ab not found: ID does not exist" Dec 03 11:43:01 crc kubenswrapper[5052]: I1203 11:43:01.001857 5052 scope.go:117] "RemoveContainer" containerID="a8c3f8ba64e0fed8deeb5029931d97733022d2a5ae3a2c13c54ed87abfa82462" Dec 03 11:43:01 crc kubenswrapper[5052]: I1203 11:43:01.025653 5052 scope.go:117] "RemoveContainer" containerID="a8c3f8ba64e0fed8deeb5029931d97733022d2a5ae3a2c13c54ed87abfa82462" Dec 03 11:43:01 crc kubenswrapper[5052]: E1203 11:43:01.026267 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a8c3f8ba64e0fed8deeb5029931d97733022d2a5ae3a2c13c54ed87abfa82462\": container with ID starting with a8c3f8ba64e0fed8deeb5029931d97733022d2a5ae3a2c13c54ed87abfa82462 not found: ID does not exist" containerID="a8c3f8ba64e0fed8deeb5029931d97733022d2a5ae3a2c13c54ed87abfa82462" Dec 03 11:43:01 crc kubenswrapper[5052]: I1203 11:43:01.026310 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a8c3f8ba64e0fed8deeb5029931d97733022d2a5ae3a2c13c54ed87abfa82462"} err="failed to get container status \"a8c3f8ba64e0fed8deeb5029931d97733022d2a5ae3a2c13c54ed87abfa82462\": rpc error: code = NotFound desc = could not find container \"a8c3f8ba64e0fed8deeb5029931d97733022d2a5ae3a2c13c54ed87abfa82462\": container with ID starting with a8c3f8ba64e0fed8deeb5029931d97733022d2a5ae3a2c13c54ed87abfa82462 not found: ID does not exist" Dec 03 11:43:01 crc kubenswrapper[5052]: I1203 11:43:01.683424 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="222f033c-3660-4347-b621-7122c4c94929" path="/var/lib/kubelet/pods/222f033c-3660-4347-b621-7122c4c94929/volumes" Dec 03 11:43:01 crc kubenswrapper[5052]: I1203 11:43:01.685044 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3004edac-eef4-4ddc-a539-4d4294029774" path="/var/lib/kubelet/pods/3004edac-eef4-4ddc-a539-4d4294029774/volumes" Dec 03 11:43:01 crc kubenswrapper[5052]: I1203 11:43:01.685812 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="48409c36-734b-4b1d-adee-0e9981d635c9" path="/var/lib/kubelet/pods/48409c36-734b-4b1d-adee-0e9981d635c9/volumes" Dec 03 11:43:01 crc kubenswrapper[5052]: I1203 11:43:01.984948 5052 pod_container_manager_linux.go:210] "Failed to delete cgroup paths" cgroupName=["kubepods","besteffort","pod24f61ea3-1916-46dd-a764-2211a9735d42"] err="unable to destroy cgroup paths for cgroup [kubepods besteffort pod24f61ea3-1916-46dd-a764-2211a9735d42] : Timed out while waiting for systemd to remove kubepods-besteffort-pod24f61ea3_1916_46dd_a764_2211a9735d42.slice" Dec 03 11:43:03 crc kubenswrapper[5052]: I1203 11:43:03.347699 5052 pod_container_manager_linux.go:210] "Failed to delete cgroup paths" cgroupName=["kubepods","besteffort","pod665a736c-d550-4160-a8bb-619d1d5ea826"] err="unable to destroy cgroup paths for cgroup [kubepods besteffort pod665a736c-d550-4160-a8bb-619d1d5ea826] : Timed out while waiting for systemd to remove kubepods-besteffort-pod665a736c_d550_4160_a8bb_619d1d5ea826.slice" Dec 03 11:43:04 crc kubenswrapper[5052]: I1203 11:43:04.853997 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_neutron-69cf8cb7d6-jfb57_8f1442a2-a28b-4784-934a-34b417dbcce9/neutron-api/0.log" Dec 03 11:43:04 crc kubenswrapper[5052]: I1203 11:43:04.854070 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-69cf8cb7d6-jfb57" Dec 03 11:43:04 crc kubenswrapper[5052]: I1203 11:43:04.902765 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/8f1442a2-a28b-4784-934a-34b417dbcce9-ovndb-tls-certs\") pod \"8f1442a2-a28b-4784-934a-34b417dbcce9\" (UID: \"8f1442a2-a28b-4784-934a-34b417dbcce9\") " Dec 03 11:43:04 crc kubenswrapper[5052]: I1203 11:43:04.902818 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/8f1442a2-a28b-4784-934a-34b417dbcce9-public-tls-certs\") pod \"8f1442a2-a28b-4784-934a-34b417dbcce9\" (UID: \"8f1442a2-a28b-4784-934a-34b417dbcce9\") " Dec 03 11:43:04 crc kubenswrapper[5052]: I1203 11:43:04.902880 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/8f1442a2-a28b-4784-934a-34b417dbcce9-httpd-config\") pod \"8f1442a2-a28b-4784-934a-34b417dbcce9\" (UID: \"8f1442a2-a28b-4784-934a-34b417dbcce9\") " Dec 03 11:43:04 crc kubenswrapper[5052]: I1203 11:43:04.903690 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cbfzg\" (UniqueName: \"kubernetes.io/projected/8f1442a2-a28b-4784-934a-34b417dbcce9-kube-api-access-cbfzg\") pod \"8f1442a2-a28b-4784-934a-34b417dbcce9\" (UID: \"8f1442a2-a28b-4784-934a-34b417dbcce9\") " Dec 03 11:43:04 crc kubenswrapper[5052]: I1203 11:43:04.903905 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/8f1442a2-a28b-4784-934a-34b417dbcce9-internal-tls-certs\") pod \"8f1442a2-a28b-4784-934a-34b417dbcce9\" (UID: \"8f1442a2-a28b-4784-934a-34b417dbcce9\") " Dec 03 11:43:04 crc kubenswrapper[5052]: I1203 11:43:04.904024 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8f1442a2-a28b-4784-934a-34b417dbcce9-combined-ca-bundle\") pod \"8f1442a2-a28b-4784-934a-34b417dbcce9\" (UID: \"8f1442a2-a28b-4784-934a-34b417dbcce9\") " Dec 03 11:43:04 crc kubenswrapper[5052]: I1203 11:43:04.904198 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/8f1442a2-a28b-4784-934a-34b417dbcce9-config\") pod \"8f1442a2-a28b-4784-934a-34b417dbcce9\" (UID: \"8f1442a2-a28b-4784-934a-34b417dbcce9\") " Dec 03 11:43:04 crc kubenswrapper[5052]: I1203 11:43:04.908363 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8f1442a2-a28b-4784-934a-34b417dbcce9-kube-api-access-cbfzg" (OuterVolumeSpecName: "kube-api-access-cbfzg") pod "8f1442a2-a28b-4784-934a-34b417dbcce9" (UID: "8f1442a2-a28b-4784-934a-34b417dbcce9"). InnerVolumeSpecName "kube-api-access-cbfzg". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:43:04 crc kubenswrapper[5052]: I1203 11:43:04.908537 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8f1442a2-a28b-4784-934a-34b417dbcce9-httpd-config" (OuterVolumeSpecName: "httpd-config") pod "8f1442a2-a28b-4784-934a-34b417dbcce9" (UID: "8f1442a2-a28b-4784-934a-34b417dbcce9"). InnerVolumeSpecName "httpd-config". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:43:04 crc kubenswrapper[5052]: I1203 11:43:04.941328 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8f1442a2-a28b-4784-934a-34b417dbcce9-config" (OuterVolumeSpecName: "config") pod "8f1442a2-a28b-4784-934a-34b417dbcce9" (UID: "8f1442a2-a28b-4784-934a-34b417dbcce9"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:43:04 crc kubenswrapper[5052]: I1203 11:43:04.942101 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8f1442a2-a28b-4784-934a-34b417dbcce9-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "8f1442a2-a28b-4784-934a-34b417dbcce9" (UID: "8f1442a2-a28b-4784-934a-34b417dbcce9"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:43:04 crc kubenswrapper[5052]: I1203 11:43:04.943089 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8f1442a2-a28b-4784-934a-34b417dbcce9-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "8f1442a2-a28b-4784-934a-34b417dbcce9" (UID: "8f1442a2-a28b-4784-934a-34b417dbcce9"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:43:04 crc kubenswrapper[5052]: I1203 11:43:04.947569 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_neutron-69cf8cb7d6-jfb57_8f1442a2-a28b-4784-934a-34b417dbcce9/neutron-api/0.log" Dec 03 11:43:04 crc kubenswrapper[5052]: I1203 11:43:04.947622 5052 generic.go:334] "Generic (PLEG): container finished" podID="8f1442a2-a28b-4784-934a-34b417dbcce9" containerID="a5f8b0c46530b7fa02bdfaef171411cf9a70dd517b05545d95c2ab0253afcf85" exitCode=137 Dec 03 11:43:04 crc kubenswrapper[5052]: I1203 11:43:04.947652 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-69cf8cb7d6-jfb57" event={"ID":"8f1442a2-a28b-4784-934a-34b417dbcce9","Type":"ContainerDied","Data":"a5f8b0c46530b7fa02bdfaef171411cf9a70dd517b05545d95c2ab0253afcf85"} Dec 03 11:43:04 crc kubenswrapper[5052]: I1203 11:43:04.947678 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-69cf8cb7d6-jfb57" event={"ID":"8f1442a2-a28b-4784-934a-34b417dbcce9","Type":"ContainerDied","Data":"3ccc56079fd2b625eab01098a7f96885b1126387a64abae713733304004b4545"} Dec 03 11:43:04 crc kubenswrapper[5052]: I1203 11:43:04.947695 5052 scope.go:117] "RemoveContainer" containerID="cdf56942522f4bc644bc6410245d93699130cab711f1b2e7b726b013a6148db5" Dec 03 11:43:04 crc kubenswrapper[5052]: I1203 11:43:04.947814 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-69cf8cb7d6-jfb57" Dec 03 11:43:04 crc kubenswrapper[5052]: I1203 11:43:04.948723 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8f1442a2-a28b-4784-934a-34b417dbcce9-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "8f1442a2-a28b-4784-934a-34b417dbcce9" (UID: "8f1442a2-a28b-4784-934a-34b417dbcce9"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:43:04 crc kubenswrapper[5052]: I1203 11:43:04.963729 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8f1442a2-a28b-4784-934a-34b417dbcce9-ovndb-tls-certs" (OuterVolumeSpecName: "ovndb-tls-certs") pod "8f1442a2-a28b-4784-934a-34b417dbcce9" (UID: "8f1442a2-a28b-4784-934a-34b417dbcce9"). InnerVolumeSpecName "ovndb-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:43:04 crc kubenswrapper[5052]: I1203 11:43:04.987365 5052 scope.go:117] "RemoveContainer" containerID="a5f8b0c46530b7fa02bdfaef171411cf9a70dd517b05545d95c2ab0253afcf85" Dec 03 11:43:05 crc kubenswrapper[5052]: I1203 11:43:05.004573 5052 scope.go:117] "RemoveContainer" containerID="cdf56942522f4bc644bc6410245d93699130cab711f1b2e7b726b013a6148db5" Dec 03 11:43:05 crc kubenswrapper[5052]: E1203 11:43:05.004951 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cdf56942522f4bc644bc6410245d93699130cab711f1b2e7b726b013a6148db5\": container with ID starting with cdf56942522f4bc644bc6410245d93699130cab711f1b2e7b726b013a6148db5 not found: ID does not exist" containerID="cdf56942522f4bc644bc6410245d93699130cab711f1b2e7b726b013a6148db5" Dec 03 11:43:05 crc kubenswrapper[5052]: I1203 11:43:05.004986 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cdf56942522f4bc644bc6410245d93699130cab711f1b2e7b726b013a6148db5"} err="failed to get container status \"cdf56942522f4bc644bc6410245d93699130cab711f1b2e7b726b013a6148db5\": rpc error: code = NotFound desc = could not find container \"cdf56942522f4bc644bc6410245d93699130cab711f1b2e7b726b013a6148db5\": container with ID starting with cdf56942522f4bc644bc6410245d93699130cab711f1b2e7b726b013a6148db5 not found: ID does not exist" Dec 03 11:43:05 crc kubenswrapper[5052]: I1203 11:43:05.005011 5052 scope.go:117] "RemoveContainer" containerID="a5f8b0c46530b7fa02bdfaef171411cf9a70dd517b05545d95c2ab0253afcf85" Dec 03 11:43:05 crc kubenswrapper[5052]: E1203 11:43:05.005295 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a5f8b0c46530b7fa02bdfaef171411cf9a70dd517b05545d95c2ab0253afcf85\": container with ID starting with a5f8b0c46530b7fa02bdfaef171411cf9a70dd517b05545d95c2ab0253afcf85 not found: ID does not exist" containerID="a5f8b0c46530b7fa02bdfaef171411cf9a70dd517b05545d95c2ab0253afcf85" Dec 03 11:43:05 crc kubenswrapper[5052]: I1203 11:43:05.005321 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a5f8b0c46530b7fa02bdfaef171411cf9a70dd517b05545d95c2ab0253afcf85"} err="failed to get container status \"a5f8b0c46530b7fa02bdfaef171411cf9a70dd517b05545d95c2ab0253afcf85\": rpc error: code = NotFound desc = could not find container \"a5f8b0c46530b7fa02bdfaef171411cf9a70dd517b05545d95c2ab0253afcf85\": container with ID starting with a5f8b0c46530b7fa02bdfaef171411cf9a70dd517b05545d95c2ab0253afcf85 not found: ID does not exist" Dec 03 11:43:05 crc kubenswrapper[5052]: I1203 11:43:05.005722 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8f1442a2-a28b-4784-934a-34b417dbcce9-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:43:05 crc kubenswrapper[5052]: I1203 11:43:05.005749 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/8f1442a2-a28b-4784-934a-34b417dbcce9-config\") on node \"crc\" DevicePath \"\"" Dec 03 11:43:05 crc kubenswrapper[5052]: I1203 11:43:05.005760 5052 reconciler_common.go:293] "Volume detached for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/8f1442a2-a28b-4784-934a-34b417dbcce9-ovndb-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 11:43:05 crc kubenswrapper[5052]: I1203 11:43:05.005770 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/8f1442a2-a28b-4784-934a-34b417dbcce9-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 11:43:05 crc kubenswrapper[5052]: I1203 11:43:05.005778 5052 reconciler_common.go:293] "Volume detached for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/8f1442a2-a28b-4784-934a-34b417dbcce9-httpd-config\") on node \"crc\" DevicePath \"\"" Dec 03 11:43:05 crc kubenswrapper[5052]: I1203 11:43:05.005788 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-cbfzg\" (UniqueName: \"kubernetes.io/projected/8f1442a2-a28b-4784-934a-34b417dbcce9-kube-api-access-cbfzg\") on node \"crc\" DevicePath \"\"" Dec 03 11:43:05 crc kubenswrapper[5052]: I1203 11:43:05.005796 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/8f1442a2-a28b-4784-934a-34b417dbcce9-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 11:43:05 crc kubenswrapper[5052]: I1203 11:43:05.285088 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/neutron-69cf8cb7d6-jfb57"] Dec 03 11:43:05 crc kubenswrapper[5052]: I1203 11:43:05.291579 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/neutron-69cf8cb7d6-jfb57"] Dec 03 11:43:05 crc kubenswrapper[5052]: I1203 11:43:05.688422 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8f1442a2-a28b-4784-934a-34b417dbcce9" path="/var/lib/kubelet/pods/8f1442a2-a28b-4784-934a-34b417dbcce9/volumes" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.468101 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/rabbitmq-cell1-server-0"] Dec 03 11:43:14 crc kubenswrapper[5052]: E1203 11:43:14.468914 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="8408f908-2fbf-4933-9361-68af849aad18" containerName="mariadb-account-delete" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.468928 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="8408f908-2fbf-4933-9361-68af849aad18" containerName="mariadb-account-delete" Dec 03 11:43:14 crc kubenswrapper[5052]: E1203 11:43:14.468943 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="251248ba-578c-49cf-b77e-fc3d5c6cb6fd" containerName="mysql-bootstrap" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.468949 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="251248ba-578c-49cf-b77e-fc3d5c6cb6fd" containerName="mysql-bootstrap" Dec 03 11:43:14 crc kubenswrapper[5052]: E1203 11:43:14.468956 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="d19f0c21-bb49-40a1-87fa-e55188308729" containerName="account-server" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.468963 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="d19f0c21-bb49-40a1-87fa-e55188308729" containerName="account-server" Dec 03 11:43:14 crc kubenswrapper[5052]: E1203 11:43:14.468970 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="1e003f91-c464-4200-94c1-18c2bea8ee8a" containerName="init" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.468976 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="1e003f91-c464-4200-94c1-18c2bea8ee8a" containerName="init" Dec 03 11:43:14 crc kubenswrapper[5052]: E1203 11:43:14.468988 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="3004edac-eef4-4ddc-a539-4d4294029774" containerName="barbican-keystone-listener-log" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.468994 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="3004edac-eef4-4ddc-a539-4d4294029774" containerName="barbican-keystone-listener-log" Dec 03 11:43:14 crc kubenswrapper[5052]: E1203 11:43:14.469001 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="b88af976-65f2-4d29-8c14-d5c43e23a459" containerName="barbican-api-log" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.469007 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="b88af976-65f2-4d29-8c14-d5c43e23a459" containerName="barbican-api-log" Dec 03 11:43:14 crc kubenswrapper[5052]: E1203 11:43:14.469016 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="d19f0c21-bb49-40a1-87fa-e55188308729" containerName="container-updater" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.469022 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="d19f0c21-bb49-40a1-87fa-e55188308729" containerName="container-updater" Dec 03 11:43:14 crc kubenswrapper[5052]: E1203 11:43:14.469032 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="75e4b34e-f7b8-4209-95a6-6cc45f6eb57c" containerName="glance-httpd" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.469039 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="75e4b34e-f7b8-4209-95a6-6cc45f6eb57c" containerName="glance-httpd" Dec 03 11:43:14 crc kubenswrapper[5052]: E1203 11:43:14.469046 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="eae5b719-0017-4621-9cb4-73ed6353648e" containerName="ceilometer-central-agent" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.469052 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="eae5b719-0017-4621-9cb4-73ed6353648e" containerName="ceilometer-central-agent" Dec 03 11:43:14 crc kubenswrapper[5052]: E1203 11:43:14.469064 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="32190156-44ee-4a16-a86c-2584f6a94866" containerName="barbican-worker" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.469069 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="32190156-44ee-4a16-a86c-2584f6a94866" containerName="barbican-worker" Dec 03 11:43:14 crc kubenswrapper[5052]: E1203 11:43:14.469076 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="1e003f91-c464-4200-94c1-18c2bea8ee8a" containerName="dnsmasq-dns" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.469082 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="1e003f91-c464-4200-94c1-18c2bea8ee8a" containerName="dnsmasq-dns" Dec 03 11:43:14 crc kubenswrapper[5052]: E1203 11:43:14.469093 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="0b4b31e5-e82d-4d44-9c8f-d001484293b6" containerName="placement-log" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.469099 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="0b4b31e5-e82d-4d44-9c8f-d001484293b6" containerName="placement-log" Dec 03 11:43:14 crc kubenswrapper[5052]: E1203 11:43:14.469111 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="d80d1476-351a-40c6-9f76-ce66d623df74" containerName="nova-scheduler-scheduler" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.469118 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="d80d1476-351a-40c6-9f76-ce66d623df74" containerName="nova-scheduler-scheduler" Dec 03 11:43:14 crc kubenswrapper[5052]: E1203 11:43:14.469126 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="8f1442a2-a28b-4784-934a-34b417dbcce9" containerName="neutron-api" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.469131 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="8f1442a2-a28b-4784-934a-34b417dbcce9" containerName="neutron-api" Dec 03 11:43:14 crc kubenswrapper[5052]: E1203 11:43:14.469139 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="b88af976-65f2-4d29-8c14-d5c43e23a459" containerName="barbican-api" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.469145 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="b88af976-65f2-4d29-8c14-d5c43e23a459" containerName="barbican-api" Dec 03 11:43:14 crc kubenswrapper[5052]: E1203 11:43:14.469154 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="0c7d3107-6dd9-4aa2-a392-68a213013508" containerName="placement-api" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.469160 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="0c7d3107-6dd9-4aa2-a392-68a213013508" containerName="placement-api" Dec 03 11:43:14 crc kubenswrapper[5052]: E1203 11:43:14.469171 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="9944d9af-7a7b-4773-8279-72914d63ef5f" containerName="setup-container" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.469176 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="9944d9af-7a7b-4773-8279-72914d63ef5f" containerName="setup-container" Dec 03 11:43:14 crc kubenswrapper[5052]: E1203 11:43:14.469187 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="d19f0c21-bb49-40a1-87fa-e55188308729" containerName="account-auditor" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.469192 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="d19f0c21-bb49-40a1-87fa-e55188308729" containerName="account-auditor" Dec 03 11:43:14 crc kubenswrapper[5052]: E1203 11:43:14.469198 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="ee46c3a4-5e21-4f6b-a944-8cf96d87506e" containerName="cinder-api-log" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.469203 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee46c3a4-5e21-4f6b-a944-8cf96d87506e" containerName="cinder-api-log" Dec 03 11:43:14 crc kubenswrapper[5052]: E1203 11:43:14.469209 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="d19f0c21-bb49-40a1-87fa-e55188308729" containerName="object-expirer" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.469215 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="d19f0c21-bb49-40a1-87fa-e55188308729" containerName="object-expirer" Dec 03 11:43:14 crc kubenswrapper[5052]: E1203 11:43:14.469225 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="eae5b719-0017-4621-9cb4-73ed6353648e" containerName="sg-core" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.469230 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="eae5b719-0017-4621-9cb4-73ed6353648e" containerName="sg-core" Dec 03 11:43:14 crc kubenswrapper[5052]: E1203 11:43:14.469244 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="d19f0c21-bb49-40a1-87fa-e55188308729" containerName="object-updater" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.469251 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="d19f0c21-bb49-40a1-87fa-e55188308729" containerName="object-updater" Dec 03 11:43:14 crc kubenswrapper[5052]: E1203 11:43:14.469266 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="222f033c-3660-4347-b621-7122c4c94929" containerName="barbican-worker-log" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.469275 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="222f033c-3660-4347-b621-7122c4c94929" containerName="barbican-worker-log" Dec 03 11:43:14 crc kubenswrapper[5052]: E1203 11:43:14.469287 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="ecbf8b8e-cdb4-45c3-9455-d0dbc551356e" containerName="galera" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.469294 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="ecbf8b8e-cdb4-45c3-9455-d0dbc551356e" containerName="galera" Dec 03 11:43:14 crc kubenswrapper[5052]: E1203 11:43:14.469304 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="daa521fc-7e1d-44d2-b7d8-6aab003cfb66" containerName="setup-container" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.469311 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="daa521fc-7e1d-44d2-b7d8-6aab003cfb66" containerName="setup-container" Dec 03 11:43:14 crc kubenswrapper[5052]: E1203 11:43:14.469319 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="d19f0c21-bb49-40a1-87fa-e55188308729" containerName="object-auditor" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.469325 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="d19f0c21-bb49-40a1-87fa-e55188308729" containerName="object-auditor" Dec 03 11:43:14 crc kubenswrapper[5052]: E1203 11:43:14.469334 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="d19f0c21-bb49-40a1-87fa-e55188308729" containerName="object-server" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.469340 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="d19f0c21-bb49-40a1-87fa-e55188308729" containerName="object-server" Dec 03 11:43:14 crc kubenswrapper[5052]: E1203 11:43:14.469348 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="d19f0c21-bb49-40a1-87fa-e55188308729" containerName="account-reaper" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.469354 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="d19f0c21-bb49-40a1-87fa-e55188308729" containerName="account-reaper" Dec 03 11:43:14 crc kubenswrapper[5052]: E1203 11:43:14.469362 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="9944d9af-7a7b-4773-8279-72914d63ef5f" containerName="rabbitmq" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.469367 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="9944d9af-7a7b-4773-8279-72914d63ef5f" containerName="rabbitmq" Dec 03 11:43:14 crc kubenswrapper[5052]: E1203 11:43:14.469378 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="f98f227a-397b-4665-bf18-e00b105ce058" containerName="nova-metadata-metadata" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.469383 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="f98f227a-397b-4665-bf18-e00b105ce058" containerName="nova-metadata-metadata" Dec 03 11:43:14 crc kubenswrapper[5052]: E1203 11:43:14.469392 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="bd2dc8ee-f6b9-4c8b-af5e-79e67a2d1ef0" containerName="mariadb-account-delete" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.469398 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd2dc8ee-f6b9-4c8b-af5e-79e67a2d1ef0" containerName="mariadb-account-delete" Dec 03 11:43:14 crc kubenswrapper[5052]: E1203 11:43:14.469407 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="daa521fc-7e1d-44d2-b7d8-6aab003cfb66" containerName="rabbitmq" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.469412 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="daa521fc-7e1d-44d2-b7d8-6aab003cfb66" containerName="rabbitmq" Dec 03 11:43:14 crc kubenswrapper[5052]: E1203 11:43:14.469423 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="fd14a5c0-3402-4891-a726-32d75b1b7857" containerName="proxy-server" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.469430 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="fd14a5c0-3402-4891-a726-32d75b1b7857" containerName="proxy-server" Dec 03 11:43:14 crc kubenswrapper[5052]: E1203 11:43:14.469440 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="14e9bff5-66d5-4808-8446-cf180d759208" containerName="cinder-scheduler" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.469446 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="14e9bff5-66d5-4808-8446-cf180d759208" containerName="cinder-scheduler" Dec 03 11:43:14 crc kubenswrapper[5052]: E1203 11:43:14.469456 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="665a736c-d550-4160-a8bb-619d1d5ea826" containerName="glance-httpd" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.469461 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="665a736c-d550-4160-a8bb-619d1d5ea826" containerName="glance-httpd" Dec 03 11:43:14 crc kubenswrapper[5052]: E1203 11:43:14.469469 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="14e9bff5-66d5-4808-8446-cf180d759208" containerName="probe" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.469475 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="14e9bff5-66d5-4808-8446-cf180d759208" containerName="probe" Dec 03 11:43:14 crc kubenswrapper[5052]: E1203 11:43:14.469481 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="f8cdf05e-7081-4542-844d-f76378e8989f" containerName="nova-cell0-conductor-conductor" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.469487 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="f8cdf05e-7081-4542-844d-f76378e8989f" containerName="nova-cell0-conductor-conductor" Dec 03 11:43:14 crc kubenswrapper[5052]: E1203 11:43:14.469495 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="48409c36-734b-4b1d-adee-0e9981d635c9" containerName="keystone-api" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.469501 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="48409c36-734b-4b1d-adee-0e9981d635c9" containerName="keystone-api" Dec 03 11:43:14 crc kubenswrapper[5052]: E1203 11:43:14.469522 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="d3e34248-e7e5-4158-8add-165d17bee9ab" containerName="mariadb-account-delete" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.469528 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="d3e34248-e7e5-4158-8add-165d17bee9ab" containerName="mariadb-account-delete" Dec 03 11:43:14 crc kubenswrapper[5052]: E1203 11:43:14.469537 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="a30eaf15-65ff-426b-8ac2-1ed5a369886a" containerName="openstack-network-exporter" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.469543 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="a30eaf15-65ff-426b-8ac2-1ed5a369886a" containerName="openstack-network-exporter" Dec 03 11:43:14 crc kubenswrapper[5052]: E1203 11:43:14.469551 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="d19f0c21-bb49-40a1-87fa-e55188308729" containerName="account-replicator" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.469557 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="d19f0c21-bb49-40a1-87fa-e55188308729" containerName="account-replicator" Dec 03 11:43:14 crc kubenswrapper[5052]: E1203 11:43:14.469567 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="ecbf8b8e-cdb4-45c3-9455-d0dbc551356e" containerName="mysql-bootstrap" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.469573 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="ecbf8b8e-cdb4-45c3-9455-d0dbc551356e" containerName="mysql-bootstrap" Dec 03 11:43:14 crc kubenswrapper[5052]: E1203 11:43:14.469581 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="d19f0c21-bb49-40a1-87fa-e55188308729" containerName="container-server" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.471652 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="d19f0c21-bb49-40a1-87fa-e55188308729" containerName="container-server" Dec 03 11:43:14 crc kubenswrapper[5052]: E1203 11:43:14.471678 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="222f033c-3660-4347-b621-7122c4c94929" containerName="barbican-worker" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.471685 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="222f033c-3660-4347-b621-7122c4c94929" containerName="barbican-worker" Dec 03 11:43:14 crc kubenswrapper[5052]: E1203 11:43:14.471701 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="d19f0c21-bb49-40a1-87fa-e55188308729" containerName="object-replicator" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.471708 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="d19f0c21-bb49-40a1-87fa-e55188308729" containerName="object-replicator" Dec 03 11:43:14 crc kubenswrapper[5052]: E1203 11:43:14.471716 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="8c4732f9-0f9e-4600-aed9-6a230cbf7b16" containerName="barbican-keystone-listener-log" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.471722 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="8c4732f9-0f9e-4600-aed9-6a230cbf7b16" containerName="barbican-keystone-listener-log" Dec 03 11:43:14 crc kubenswrapper[5052]: E1203 11:43:14.471732 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="d19f0c21-bb49-40a1-87fa-e55188308729" containerName="container-auditor" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.471739 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="d19f0c21-bb49-40a1-87fa-e55188308729" containerName="container-auditor" Dec 03 11:43:14 crc kubenswrapper[5052]: E1203 11:43:14.471751 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="8c4732f9-0f9e-4600-aed9-6a230cbf7b16" containerName="barbican-keystone-listener" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.471757 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="8c4732f9-0f9e-4600-aed9-6a230cbf7b16" containerName="barbican-keystone-listener" Dec 03 11:43:14 crc kubenswrapper[5052]: E1203 11:43:14.471768 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="d19f0c21-bb49-40a1-87fa-e55188308729" containerName="swift-recon-cron" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.471774 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="d19f0c21-bb49-40a1-87fa-e55188308729" containerName="swift-recon-cron" Dec 03 11:43:14 crc kubenswrapper[5052]: E1203 11:43:14.471783 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="18c1ce3d-6d64-433f-a115-9977227455be" containerName="registry-server" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.471789 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="18c1ce3d-6d64-433f-a115-9977227455be" containerName="registry-server" Dec 03 11:43:14 crc kubenswrapper[5052]: E1203 11:43:14.471809 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="965f5591-b293-4101-be4a-a5329d88fb69" containerName="mariadb-account-delete" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.471815 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="965f5591-b293-4101-be4a-a5329d88fb69" containerName="mariadb-account-delete" Dec 03 11:43:14 crc kubenswrapper[5052]: E1203 11:43:14.471825 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="82dce330-8f65-4c14-80d4-958eb8ca5212" containerName="nova-api-api" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.471831 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="82dce330-8f65-4c14-80d4-958eb8ca5212" containerName="nova-api-api" Dec 03 11:43:14 crc kubenswrapper[5052]: E1203 11:43:14.471839 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="a30eaf15-65ff-426b-8ac2-1ed5a369886a" containerName="ovn-northd" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.471846 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="a30eaf15-65ff-426b-8ac2-1ed5a369886a" containerName="ovn-northd" Dec 03 11:43:14 crc kubenswrapper[5052]: E1203 11:43:14.471856 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="eae5b719-0017-4621-9cb4-73ed6353648e" containerName="proxy-httpd" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.471861 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="eae5b719-0017-4621-9cb4-73ed6353648e" containerName="proxy-httpd" Dec 03 11:43:14 crc kubenswrapper[5052]: E1203 11:43:14.471871 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="18c1ce3d-6d64-433f-a115-9977227455be" containerName="extract-content" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.471877 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="18c1ce3d-6d64-433f-a115-9977227455be" containerName="extract-content" Dec 03 11:43:14 crc kubenswrapper[5052]: E1203 11:43:14.471884 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="d3e34248-e7e5-4158-8add-165d17bee9ab" containerName="mariadb-account-delete" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.471891 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="d3e34248-e7e5-4158-8add-165d17bee9ab" containerName="mariadb-account-delete" Dec 03 11:43:14 crc kubenswrapper[5052]: E1203 11:43:14.471904 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="665a736c-d550-4160-a8bb-619d1d5ea826" containerName="glance-log" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.471910 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="665a736c-d550-4160-a8bb-619d1d5ea826" containerName="glance-log" Dec 03 11:43:14 crc kubenswrapper[5052]: E1203 11:43:14.471922 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="fd14a5c0-3402-4891-a726-32d75b1b7857" containerName="proxy-httpd" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.471927 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="fd14a5c0-3402-4891-a726-32d75b1b7857" containerName="proxy-httpd" Dec 03 11:43:14 crc kubenswrapper[5052]: E1203 11:43:14.471935 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="bd2dc8ee-f6b9-4c8b-af5e-79e67a2d1ef0" containerName="mariadb-account-delete" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.471941 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd2dc8ee-f6b9-4c8b-af5e-79e67a2d1ef0" containerName="mariadb-account-delete" Dec 03 11:43:14 crc kubenswrapper[5052]: E1203 11:43:14.471952 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="4a23886b-73c6-4b21-8464-07a2dd2e06f6" containerName="neutron-httpd" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.471958 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="4a23886b-73c6-4b21-8464-07a2dd2e06f6" containerName="neutron-httpd" Dec 03 11:43:14 crc kubenswrapper[5052]: E1203 11:43:14.471964 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="ce4d9d20-a900-4b3d-8ee0-46ea28a51262" containerName="mariadb-account-delete" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.471970 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="ce4d9d20-a900-4b3d-8ee0-46ea28a51262" containerName="mariadb-account-delete" Dec 03 11:43:14 crc kubenswrapper[5052]: E1203 11:43:14.471978 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="75e4b34e-f7b8-4209-95a6-6cc45f6eb57c" containerName="glance-log" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.471983 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="75e4b34e-f7b8-4209-95a6-6cc45f6eb57c" containerName="glance-log" Dec 03 11:43:14 crc kubenswrapper[5052]: E1203 11:43:14.471995 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="5737462d-2e5c-4453-9d6c-7d67503555f7" containerName="keystone-api" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.472002 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="5737462d-2e5c-4453-9d6c-7d67503555f7" containerName="keystone-api" Dec 03 11:43:14 crc kubenswrapper[5052]: E1203 11:43:14.472008 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="eae5b719-0017-4621-9cb4-73ed6353648e" containerName="ceilometer-notification-agent" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.472014 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="eae5b719-0017-4621-9cb4-73ed6353648e" containerName="ceilometer-notification-agent" Dec 03 11:43:14 crc kubenswrapper[5052]: E1203 11:43:14.472022 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="251248ba-578c-49cf-b77e-fc3d5c6cb6fd" containerName="galera" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.472027 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="251248ba-578c-49cf-b77e-fc3d5c6cb6fd" containerName="galera" Dec 03 11:43:14 crc kubenswrapper[5052]: E1203 11:43:14.472040 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="d19f0c21-bb49-40a1-87fa-e55188308729" containerName="rsync" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.472046 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="d19f0c21-bb49-40a1-87fa-e55188308729" containerName="rsync" Dec 03 11:43:14 crc kubenswrapper[5052]: E1203 11:43:14.472056 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="0c7d3107-6dd9-4aa2-a392-68a213013508" containerName="placement-log" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.472061 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="0c7d3107-6dd9-4aa2-a392-68a213013508" containerName="placement-log" Dec 03 11:43:14 crc kubenswrapper[5052]: E1203 11:43:14.472069 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="fe69e54e-ae29-49e2-8a21-0ff81cc0100f" containerName="barbican-api-log" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.472077 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="fe69e54e-ae29-49e2-8a21-0ff81cc0100f" containerName="barbican-api-log" Dec 03 11:43:14 crc kubenswrapper[5052]: E1203 11:43:14.472088 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="f98f227a-397b-4665-bf18-e00b105ce058" containerName="nova-metadata-log" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.472097 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="f98f227a-397b-4665-bf18-e00b105ce058" containerName="nova-metadata-log" Dec 03 11:43:14 crc kubenswrapper[5052]: E1203 11:43:14.472107 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="3004edac-eef4-4ddc-a539-4d4294029774" containerName="barbican-keystone-listener" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.472114 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="3004edac-eef4-4ddc-a539-4d4294029774" containerName="barbican-keystone-listener" Dec 03 11:43:14 crc kubenswrapper[5052]: E1203 11:43:14.472128 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="0b4b31e5-e82d-4d44-9c8f-d001484293b6" containerName="placement-api" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.472135 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="0b4b31e5-e82d-4d44-9c8f-d001484293b6" containerName="placement-api" Dec 03 11:43:14 crc kubenswrapper[5052]: E1203 11:43:14.472145 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="18c1ce3d-6d64-433f-a115-9977227455be" containerName="extract-utilities" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.472152 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="18c1ce3d-6d64-433f-a115-9977227455be" containerName="extract-utilities" Dec 03 11:43:14 crc kubenswrapper[5052]: E1203 11:43:14.472165 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="82dce330-8f65-4c14-80d4-958eb8ca5212" containerName="nova-api-log" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.472173 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="82dce330-8f65-4c14-80d4-958eb8ca5212" containerName="nova-api-log" Dec 03 11:43:14 crc kubenswrapper[5052]: E1203 11:43:14.472183 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="32573220-587f-4703-a5e9-0fb67d2019ef" containerName="mariadb-account-delete" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.472190 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="32573220-587f-4703-a5e9-0fb67d2019ef" containerName="mariadb-account-delete" Dec 03 11:43:14 crc kubenswrapper[5052]: E1203 11:43:14.472198 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="8f1442a2-a28b-4784-934a-34b417dbcce9" containerName="neutron-httpd" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.472205 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="8f1442a2-a28b-4784-934a-34b417dbcce9" containerName="neutron-httpd" Dec 03 11:43:14 crc kubenswrapper[5052]: E1203 11:43:14.472221 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="8b465e38-7dbb-4d02-bce0-28ae0145520c" containerName="kube-state-metrics" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.472228 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="8b465e38-7dbb-4d02-bce0-28ae0145520c" containerName="kube-state-metrics" Dec 03 11:43:14 crc kubenswrapper[5052]: E1203 11:43:14.472241 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="ee46c3a4-5e21-4f6b-a944-8cf96d87506e" containerName="cinder-api" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.472248 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee46c3a4-5e21-4f6b-a944-8cf96d87506e" containerName="cinder-api" Dec 03 11:43:14 crc kubenswrapper[5052]: E1203 11:43:14.472259 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="fe69e54e-ae29-49e2-8a21-0ff81cc0100f" containerName="barbican-api" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.472268 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="fe69e54e-ae29-49e2-8a21-0ff81cc0100f" containerName="barbican-api" Dec 03 11:43:14 crc kubenswrapper[5052]: E1203 11:43:14.472282 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="4a23886b-73c6-4b21-8464-07a2dd2e06f6" containerName="neutron-api" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.472289 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="4a23886b-73c6-4b21-8464-07a2dd2e06f6" containerName="neutron-api" Dec 03 11:43:14 crc kubenswrapper[5052]: E1203 11:43:14.472299 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="32190156-44ee-4a16-a86c-2584f6a94866" containerName="barbican-worker-log" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.472306 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="32190156-44ee-4a16-a86c-2584f6a94866" containerName="barbican-worker-log" Dec 03 11:43:14 crc kubenswrapper[5052]: E1203 11:43:14.472318 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="2501eda8-a78d-4609-a465-761e89bb291e" containerName="nova-cell1-conductor-conductor" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.472325 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="2501eda8-a78d-4609-a465-761e89bb291e" containerName="nova-cell1-conductor-conductor" Dec 03 11:43:14 crc kubenswrapper[5052]: E1203 11:43:14.472336 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="d19f0c21-bb49-40a1-87fa-e55188308729" containerName="container-replicator" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.472344 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="d19f0c21-bb49-40a1-87fa-e55188308729" containerName="container-replicator" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.472672 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="d19f0c21-bb49-40a1-87fa-e55188308729" containerName="object-server" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.472690 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="d19f0c21-bb49-40a1-87fa-e55188308729" containerName="rsync" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.472699 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="ce4d9d20-a900-4b3d-8ee0-46ea28a51262" containerName="mariadb-account-delete" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.472714 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="a30eaf15-65ff-426b-8ac2-1ed5a369886a" containerName="ovn-northd" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.472721 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="14e9bff5-66d5-4808-8446-cf180d759208" containerName="cinder-scheduler" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.472735 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="8b465e38-7dbb-4d02-bce0-28ae0145520c" containerName="kube-state-metrics" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.472741 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="f98f227a-397b-4665-bf18-e00b105ce058" containerName="nova-metadata-metadata" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.472754 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="bd2dc8ee-f6b9-4c8b-af5e-79e67a2d1ef0" containerName="mariadb-account-delete" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.472773 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="4a23886b-73c6-4b21-8464-07a2dd2e06f6" containerName="neutron-httpd" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.472783 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="ecbf8b8e-cdb4-45c3-9455-d0dbc551356e" containerName="galera" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.472791 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="222f033c-3660-4347-b621-7122c4c94929" containerName="barbican-worker" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.472798 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="d19f0c21-bb49-40a1-87fa-e55188308729" containerName="object-replicator" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.472806 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="d19f0c21-bb49-40a1-87fa-e55188308729" containerName="account-auditor" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.472816 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="ee46c3a4-5e21-4f6b-a944-8cf96d87506e" containerName="cinder-api" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.472826 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="8f1442a2-a28b-4784-934a-34b417dbcce9" containerName="neutron-httpd" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.472836 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="d19f0c21-bb49-40a1-87fa-e55188308729" containerName="account-reaper" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.472847 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="82dce330-8f65-4c14-80d4-958eb8ca5212" containerName="nova-api-api" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.472856 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="d3e34248-e7e5-4158-8add-165d17bee9ab" containerName="mariadb-account-delete" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.472864 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="48409c36-734b-4b1d-adee-0e9981d635c9" containerName="keystone-api" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.472877 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="eae5b719-0017-4621-9cb4-73ed6353648e" containerName="ceilometer-notification-agent" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.472887 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="eae5b719-0017-4621-9cb4-73ed6353648e" containerName="proxy-httpd" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.472898 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="d19f0c21-bb49-40a1-87fa-e55188308729" containerName="account-replicator" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.472907 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="75e4b34e-f7b8-4209-95a6-6cc45f6eb57c" containerName="glance-log" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.472918 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="d19f0c21-bb49-40a1-87fa-e55188308729" containerName="object-updater" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.472927 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="32573220-587f-4703-a5e9-0fb67d2019ef" containerName="mariadb-account-delete" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.472938 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="fe69e54e-ae29-49e2-8a21-0ff81cc0100f" containerName="barbican-api-log" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.472949 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="665a736c-d550-4160-a8bb-619d1d5ea826" containerName="glance-log" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.472957 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="d3e34248-e7e5-4158-8add-165d17bee9ab" containerName="mariadb-account-delete" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.472985 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="d19f0c21-bb49-40a1-87fa-e55188308729" containerName="object-auditor" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.472997 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="965f5591-b293-4101-be4a-a5329d88fb69" containerName="mariadb-account-delete" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.473009 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="9944d9af-7a7b-4773-8279-72914d63ef5f" containerName="rabbitmq" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.473020 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="82dce330-8f65-4c14-80d4-958eb8ca5212" containerName="nova-api-log" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.473032 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="b88af976-65f2-4d29-8c14-d5c43e23a459" containerName="barbican-api" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.473043 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="d19f0c21-bb49-40a1-87fa-e55188308729" containerName="account-server" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.473054 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="f8cdf05e-7081-4542-844d-f76378e8989f" containerName="nova-cell0-conductor-conductor" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.473065 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="18c1ce3d-6d64-433f-a115-9977227455be" containerName="registry-server" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.473077 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="32190156-44ee-4a16-a86c-2584f6a94866" containerName="barbican-worker" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.473086 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="0b4b31e5-e82d-4d44-9c8f-d001484293b6" containerName="placement-log" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.473094 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="222f033c-3660-4347-b621-7122c4c94929" containerName="barbican-worker-log" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.473103 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="75e4b34e-f7b8-4209-95a6-6cc45f6eb57c" containerName="glance-httpd" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.473115 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="d19f0c21-bb49-40a1-87fa-e55188308729" containerName="swift-recon-cron" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.473122 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="2501eda8-a78d-4609-a465-761e89bb291e" containerName="nova-cell1-conductor-conductor" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.473133 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="0b4b31e5-e82d-4d44-9c8f-d001484293b6" containerName="placement-api" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.473142 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="fe69e54e-ae29-49e2-8a21-0ff81cc0100f" containerName="barbican-api" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.473152 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="fd14a5c0-3402-4891-a726-32d75b1b7857" containerName="proxy-server" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.473164 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="4a23886b-73c6-4b21-8464-07a2dd2e06f6" containerName="neutron-api" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.473175 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="f98f227a-397b-4665-bf18-e00b105ce058" containerName="nova-metadata-log" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.473185 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="ee46c3a4-5e21-4f6b-a944-8cf96d87506e" containerName="cinder-api-log" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.473193 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="665a736c-d550-4160-a8bb-619d1d5ea826" containerName="glance-httpd" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.473203 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="d19f0c21-bb49-40a1-87fa-e55188308729" containerName="object-expirer" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.473211 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="fd14a5c0-3402-4891-a726-32d75b1b7857" containerName="proxy-httpd" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.473221 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="32190156-44ee-4a16-a86c-2584f6a94866" containerName="barbican-worker-log" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.473234 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="8c4732f9-0f9e-4600-aed9-6a230cbf7b16" containerName="barbican-keystone-listener" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.473242 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="5737462d-2e5c-4453-9d6c-7d67503555f7" containerName="keystone-api" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.473251 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="8c4732f9-0f9e-4600-aed9-6a230cbf7b16" containerName="barbican-keystone-listener-log" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.473262 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="3004edac-eef4-4ddc-a539-4d4294029774" containerName="barbican-keystone-listener" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.473274 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="14e9bff5-66d5-4808-8446-cf180d759208" containerName="probe" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.473285 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="a30eaf15-65ff-426b-8ac2-1ed5a369886a" containerName="openstack-network-exporter" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.473294 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="1e003f91-c464-4200-94c1-18c2bea8ee8a" containerName="dnsmasq-dns" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.473303 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="d80d1476-351a-40c6-9f76-ce66d623df74" containerName="nova-scheduler-scheduler" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.473313 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="d19f0c21-bb49-40a1-87fa-e55188308729" containerName="container-replicator" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.473321 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="3004edac-eef4-4ddc-a539-4d4294029774" containerName="barbican-keystone-listener-log" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.473329 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="bd2dc8ee-f6b9-4c8b-af5e-79e67a2d1ef0" containerName="mariadb-account-delete" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.473340 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="daa521fc-7e1d-44d2-b7d8-6aab003cfb66" containerName="rabbitmq" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.473353 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="d19f0c21-bb49-40a1-87fa-e55188308729" containerName="container-updater" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.473366 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="eae5b719-0017-4621-9cb4-73ed6353648e" containerName="sg-core" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.473374 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="b88af976-65f2-4d29-8c14-d5c43e23a459" containerName="barbican-api-log" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.473385 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="8408f908-2fbf-4933-9361-68af849aad18" containerName="mariadb-account-delete" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.473397 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="8f1442a2-a28b-4784-934a-34b417dbcce9" containerName="neutron-api" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.473403 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="0c7d3107-6dd9-4aa2-a392-68a213013508" containerName="placement-api" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.473417 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="0c7d3107-6dd9-4aa2-a392-68a213013508" containerName="placement-log" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.473427 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="d19f0c21-bb49-40a1-87fa-e55188308729" containerName="container-server" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.473435 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="251248ba-578c-49cf-b77e-fc3d5c6cb6fd" containerName="galera" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.473446 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="eae5b719-0017-4621-9cb4-73ed6353648e" containerName="ceilometer-central-agent" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.473453 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="d19f0c21-bb49-40a1-87fa-e55188308729" containerName="container-auditor" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.474430 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.480816 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"rabbitmq-cell1-server-dockercfg-gmcz5" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.480883 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"rabbitmq-cell1-plugins-conf" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.480957 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"rabbitmq-cell1-erlang-cookie" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.481007 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"rabbitmq-cell1-config-data" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.481077 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-rabbitmq-cell1-svc" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.481084 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"rabbitmq-cell1-default-user" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.482923 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"rabbitmq-cell1-server-conf" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.499182 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/rabbitmq-cell1-server-0"] Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.542084 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"rabbitmq-tls\" (UniqueName: \"kubernetes.io/projected/2766f489-9217-47ae-a595-869a26389587-rabbitmq-tls\") pod \"rabbitmq-cell1-server-0\" (UID: \"2766f489-9217-47ae-a595-869a26389587\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.542135 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-conf\" (UniqueName: \"kubernetes.io/configmap/2766f489-9217-47ae-a595-869a26389587-plugins-conf\") pod \"rabbitmq-cell1-server-0\" (UID: \"2766f489-9217-47ae-a595-869a26389587\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.542161 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/2766f489-9217-47ae-a595-869a26389587-config-data\") pod \"rabbitmq-cell1-server-0\" (UID: \"2766f489-9217-47ae-a595-869a26389587\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.542280 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"pod-info\" (UniqueName: \"kubernetes.io/downward-api/2766f489-9217-47ae-a595-869a26389587-pod-info\") pod \"rabbitmq-cell1-server-0\" (UID: \"2766f489-9217-47ae-a595-869a26389587\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.542324 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"rabbitmq-plugins\" (UniqueName: \"kubernetes.io/empty-dir/2766f489-9217-47ae-a595-869a26389587-rabbitmq-plugins\") pod \"rabbitmq-cell1-server-0\" (UID: \"2766f489-9217-47ae-a595-869a26389587\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.542359 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"erlang-cookie-secret\" (UniqueName: \"kubernetes.io/secret/2766f489-9217-47ae-a595-869a26389587-erlang-cookie-secret\") pod \"rabbitmq-cell1-server-0\" (UID: \"2766f489-9217-47ae-a595-869a26389587\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.542397 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage18-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage18-crc\") pod \"rabbitmq-cell1-server-0\" (UID: \"2766f489-9217-47ae-a595-869a26389587\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.542419 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"rabbitmq-erlang-cookie\" (UniqueName: \"kubernetes.io/empty-dir/2766f489-9217-47ae-a595-869a26389587-rabbitmq-erlang-cookie\") pod \"rabbitmq-cell1-server-0\" (UID: \"2766f489-9217-47ae-a595-869a26389587\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.542446 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"rabbitmq-confd\" (UniqueName: \"kubernetes.io/projected/2766f489-9217-47ae-a595-869a26389587-rabbitmq-confd\") pod \"rabbitmq-cell1-server-0\" (UID: \"2766f489-9217-47ae-a595-869a26389587\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.542470 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nkgvw\" (UniqueName: \"kubernetes.io/projected/2766f489-9217-47ae-a595-869a26389587-kube-api-access-nkgvw\") pod \"rabbitmq-cell1-server-0\" (UID: \"2766f489-9217-47ae-a595-869a26389587\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.542524 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"server-conf\" (UniqueName: \"kubernetes.io/configmap/2766f489-9217-47ae-a595-869a26389587-server-conf\") pod \"rabbitmq-cell1-server-0\" (UID: \"2766f489-9217-47ae-a595-869a26389587\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.677502 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage18-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage18-crc\") pod \"rabbitmq-cell1-server-0\" (UID: \"2766f489-9217-47ae-a595-869a26389587\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.677559 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"rabbitmq-erlang-cookie\" (UniqueName: \"kubernetes.io/empty-dir/2766f489-9217-47ae-a595-869a26389587-rabbitmq-erlang-cookie\") pod \"rabbitmq-cell1-server-0\" (UID: \"2766f489-9217-47ae-a595-869a26389587\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.677588 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"rabbitmq-confd\" (UniqueName: \"kubernetes.io/projected/2766f489-9217-47ae-a595-869a26389587-rabbitmq-confd\") pod \"rabbitmq-cell1-server-0\" (UID: \"2766f489-9217-47ae-a595-869a26389587\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.677609 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-nkgvw\" (UniqueName: \"kubernetes.io/projected/2766f489-9217-47ae-a595-869a26389587-kube-api-access-nkgvw\") pod \"rabbitmq-cell1-server-0\" (UID: \"2766f489-9217-47ae-a595-869a26389587\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.677632 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"server-conf\" (UniqueName: \"kubernetes.io/configmap/2766f489-9217-47ae-a595-869a26389587-server-conf\") pod \"rabbitmq-cell1-server-0\" (UID: \"2766f489-9217-47ae-a595-869a26389587\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.677657 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"rabbitmq-tls\" (UniqueName: \"kubernetes.io/projected/2766f489-9217-47ae-a595-869a26389587-rabbitmq-tls\") pod \"rabbitmq-cell1-server-0\" (UID: \"2766f489-9217-47ae-a595-869a26389587\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.677686 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"plugins-conf\" (UniqueName: \"kubernetes.io/configmap/2766f489-9217-47ae-a595-869a26389587-plugins-conf\") pod \"rabbitmq-cell1-server-0\" (UID: \"2766f489-9217-47ae-a595-869a26389587\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.677707 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/2766f489-9217-47ae-a595-869a26389587-config-data\") pod \"rabbitmq-cell1-server-0\" (UID: \"2766f489-9217-47ae-a595-869a26389587\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.677733 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"pod-info\" (UniqueName: \"kubernetes.io/downward-api/2766f489-9217-47ae-a595-869a26389587-pod-info\") pod \"rabbitmq-cell1-server-0\" (UID: \"2766f489-9217-47ae-a595-869a26389587\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.677759 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"rabbitmq-plugins\" (UniqueName: \"kubernetes.io/empty-dir/2766f489-9217-47ae-a595-869a26389587-rabbitmq-plugins\") pod \"rabbitmq-cell1-server-0\" (UID: \"2766f489-9217-47ae-a595-869a26389587\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.677790 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"erlang-cookie-secret\" (UniqueName: \"kubernetes.io/secret/2766f489-9217-47ae-a595-869a26389587-erlang-cookie-secret\") pod \"rabbitmq-cell1-server-0\" (UID: \"2766f489-9217-47ae-a595-869a26389587\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.678183 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"rabbitmq-erlang-cookie\" (UniqueName: \"kubernetes.io/empty-dir/2766f489-9217-47ae-a595-869a26389587-rabbitmq-erlang-cookie\") pod \"rabbitmq-cell1-server-0\" (UID: \"2766f489-9217-47ae-a595-869a26389587\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.678540 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage18-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage18-crc\") pod \"rabbitmq-cell1-server-0\" (UID: \"2766f489-9217-47ae-a595-869a26389587\") device mount path \"/mnt/openstack/pv18\"" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.680085 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/2766f489-9217-47ae-a595-869a26389587-config-data\") pod \"rabbitmq-cell1-server-0\" (UID: \"2766f489-9217-47ae-a595-869a26389587\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.680699 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"rabbitmq-plugins\" (UniqueName: \"kubernetes.io/empty-dir/2766f489-9217-47ae-a595-869a26389587-rabbitmq-plugins\") pod \"rabbitmq-cell1-server-0\" (UID: \"2766f489-9217-47ae-a595-869a26389587\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.681374 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"plugins-conf\" (UniqueName: \"kubernetes.io/configmap/2766f489-9217-47ae-a595-869a26389587-plugins-conf\") pod \"rabbitmq-cell1-server-0\" (UID: \"2766f489-9217-47ae-a595-869a26389587\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.681718 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"server-conf\" (UniqueName: \"kubernetes.io/configmap/2766f489-9217-47ae-a595-869a26389587-server-conf\") pod \"rabbitmq-cell1-server-0\" (UID: \"2766f489-9217-47ae-a595-869a26389587\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.685661 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"erlang-cookie-secret\" (UniqueName: \"kubernetes.io/secret/2766f489-9217-47ae-a595-869a26389587-erlang-cookie-secret\") pod \"rabbitmq-cell1-server-0\" (UID: \"2766f489-9217-47ae-a595-869a26389587\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.688093 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"pod-info\" (UniqueName: \"kubernetes.io/downward-api/2766f489-9217-47ae-a595-869a26389587-pod-info\") pod \"rabbitmq-cell1-server-0\" (UID: \"2766f489-9217-47ae-a595-869a26389587\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.688658 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"rabbitmq-tls\" (UniqueName: \"kubernetes.io/projected/2766f489-9217-47ae-a595-869a26389587-rabbitmq-tls\") pod \"rabbitmq-cell1-server-0\" (UID: \"2766f489-9217-47ae-a595-869a26389587\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.699912 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"rabbitmq-confd\" (UniqueName: \"kubernetes.io/projected/2766f489-9217-47ae-a595-869a26389587-rabbitmq-confd\") pod \"rabbitmq-cell1-server-0\" (UID: \"2766f489-9217-47ae-a595-869a26389587\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.702793 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-nkgvw\" (UniqueName: \"kubernetes.io/projected/2766f489-9217-47ae-a595-869a26389587-kube-api-access-nkgvw\") pod \"rabbitmq-cell1-server-0\" (UID: \"2766f489-9217-47ae-a595-869a26389587\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.719062 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage18-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage18-crc\") pod \"rabbitmq-cell1-server-0\" (UID: \"2766f489-9217-47ae-a595-869a26389587\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 11:43:14 crc kubenswrapper[5052]: I1203 11:43:14.792351 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 11:43:15 crc kubenswrapper[5052]: I1203 11:43:15.228977 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/rabbitmq-cell1-server-0"] Dec 03 11:43:15 crc kubenswrapper[5052]: I1203 11:43:15.647180 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/rabbitmq-server-0"] Dec 03 11:43:15 crc kubenswrapper[5052]: I1203 11:43:15.651048 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 11:43:15 crc kubenswrapper[5052]: I1203 11:43:15.654392 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"rabbitmq-erlang-cookie" Dec 03 11:43:15 crc kubenswrapper[5052]: I1203 11:43:15.655237 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-rabbitmq-svc" Dec 03 11:43:15 crc kubenswrapper[5052]: I1203 11:43:15.655311 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"rabbitmq-server-conf" Dec 03 11:43:15 crc kubenswrapper[5052]: I1203 11:43:15.655329 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"rabbitmq-config-data" Dec 03 11:43:15 crc kubenswrapper[5052]: I1203 11:43:15.655446 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"rabbitmq-default-user" Dec 03 11:43:15 crc kubenswrapper[5052]: I1203 11:43:15.660702 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"rabbitmq-plugins-conf" Dec 03 11:43:15 crc kubenswrapper[5052]: I1203 11:43:15.664039 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"rabbitmq-server-dockercfg-lhhpw" Dec 03 11:43:15 crc kubenswrapper[5052]: I1203 11:43:15.670127 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/rabbitmq-server-0"] Dec 03 11:43:15 crc kubenswrapper[5052]: I1203 11:43:15.800314 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pz65r\" (UniqueName: \"kubernetes.io/projected/d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0-kube-api-access-pz65r\") pod \"rabbitmq-server-0\" (UID: \"d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 11:43:15 crc kubenswrapper[5052]: I1203 11:43:15.800681 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"pod-info\" (UniqueName: \"kubernetes.io/downward-api/d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0-pod-info\") pod \"rabbitmq-server-0\" (UID: \"d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 11:43:15 crc kubenswrapper[5052]: I1203 11:43:15.800811 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"server-conf\" (UniqueName: \"kubernetes.io/configmap/d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0-server-conf\") pod \"rabbitmq-server-0\" (UID: \"d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 11:43:15 crc kubenswrapper[5052]: I1203 11:43:15.800924 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"rabbitmq-plugins\" (UniqueName: \"kubernetes.io/empty-dir/d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0-rabbitmq-plugins\") pod \"rabbitmq-server-0\" (UID: \"d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 11:43:15 crc kubenswrapper[5052]: I1203 11:43:15.801019 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"erlang-cookie-secret\" (UniqueName: \"kubernetes.io/secret/d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0-erlang-cookie-secret\") pod \"rabbitmq-server-0\" (UID: \"d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 11:43:15 crc kubenswrapper[5052]: I1203 11:43:15.801131 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0-config-data\") pod \"rabbitmq-server-0\" (UID: \"d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 11:43:15 crc kubenswrapper[5052]: I1203 11:43:15.801279 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-conf\" (UniqueName: \"kubernetes.io/configmap/d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0-plugins-conf\") pod \"rabbitmq-server-0\" (UID: \"d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 11:43:15 crc kubenswrapper[5052]: I1203 11:43:15.801414 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage07-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage07-crc\") pod \"rabbitmq-server-0\" (UID: \"d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 11:43:15 crc kubenswrapper[5052]: I1203 11:43:15.801550 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"rabbitmq-erlang-cookie\" (UniqueName: \"kubernetes.io/empty-dir/d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0-rabbitmq-erlang-cookie\") pod \"rabbitmq-server-0\" (UID: \"d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 11:43:15 crc kubenswrapper[5052]: I1203 11:43:15.801667 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"rabbitmq-tls\" (UniqueName: \"kubernetes.io/projected/d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0-rabbitmq-tls\") pod \"rabbitmq-server-0\" (UID: \"d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 11:43:15 crc kubenswrapper[5052]: I1203 11:43:15.801786 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"rabbitmq-confd\" (UniqueName: \"kubernetes.io/projected/d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0-rabbitmq-confd\") pod \"rabbitmq-server-0\" (UID: \"d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 11:43:15 crc kubenswrapper[5052]: I1203 11:43:15.903463 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"pod-info\" (UniqueName: \"kubernetes.io/downward-api/d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0-pod-info\") pod \"rabbitmq-server-0\" (UID: \"d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 11:43:15 crc kubenswrapper[5052]: I1203 11:43:15.903554 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"server-conf\" (UniqueName: \"kubernetes.io/configmap/d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0-server-conf\") pod \"rabbitmq-server-0\" (UID: \"d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 11:43:15 crc kubenswrapper[5052]: I1203 11:43:15.903585 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"rabbitmq-plugins\" (UniqueName: \"kubernetes.io/empty-dir/d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0-rabbitmq-plugins\") pod \"rabbitmq-server-0\" (UID: \"d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 11:43:15 crc kubenswrapper[5052]: I1203 11:43:15.903613 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"erlang-cookie-secret\" (UniqueName: \"kubernetes.io/secret/d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0-erlang-cookie-secret\") pod \"rabbitmq-server-0\" (UID: \"d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 11:43:15 crc kubenswrapper[5052]: I1203 11:43:15.903630 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0-config-data\") pod \"rabbitmq-server-0\" (UID: \"d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 11:43:15 crc kubenswrapper[5052]: I1203 11:43:15.903653 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"plugins-conf\" (UniqueName: \"kubernetes.io/configmap/d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0-plugins-conf\") pod \"rabbitmq-server-0\" (UID: \"d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 11:43:15 crc kubenswrapper[5052]: I1203 11:43:15.903683 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage07-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage07-crc\") pod \"rabbitmq-server-0\" (UID: \"d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 11:43:15 crc kubenswrapper[5052]: I1203 11:43:15.903702 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"rabbitmq-erlang-cookie\" (UniqueName: \"kubernetes.io/empty-dir/d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0-rabbitmq-erlang-cookie\") pod \"rabbitmq-server-0\" (UID: \"d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 11:43:15 crc kubenswrapper[5052]: I1203 11:43:15.903735 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"rabbitmq-tls\" (UniqueName: \"kubernetes.io/projected/d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0-rabbitmq-tls\") pod \"rabbitmq-server-0\" (UID: \"d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 11:43:15 crc kubenswrapper[5052]: I1203 11:43:15.903758 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"rabbitmq-confd\" (UniqueName: \"kubernetes.io/projected/d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0-rabbitmq-confd\") pod \"rabbitmq-server-0\" (UID: \"d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 11:43:15 crc kubenswrapper[5052]: I1203 11:43:15.903795 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-pz65r\" (UniqueName: \"kubernetes.io/projected/d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0-kube-api-access-pz65r\") pod \"rabbitmq-server-0\" (UID: \"d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 11:43:15 crc kubenswrapper[5052]: I1203 11:43:15.905231 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage07-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage07-crc\") pod \"rabbitmq-server-0\" (UID: \"d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0\") device mount path \"/mnt/openstack/pv07\"" pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 11:43:15 crc kubenswrapper[5052]: I1203 11:43:15.905299 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"rabbitmq-plugins\" (UniqueName: \"kubernetes.io/empty-dir/d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0-rabbitmq-plugins\") pod \"rabbitmq-server-0\" (UID: \"d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 11:43:15 crc kubenswrapper[5052]: I1203 11:43:15.905362 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"rabbitmq-erlang-cookie\" (UniqueName: \"kubernetes.io/empty-dir/d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0-rabbitmq-erlang-cookie\") pod \"rabbitmq-server-0\" (UID: \"d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 11:43:15 crc kubenswrapper[5052]: I1203 11:43:15.905435 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"server-conf\" (UniqueName: \"kubernetes.io/configmap/d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0-server-conf\") pod \"rabbitmq-server-0\" (UID: \"d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 11:43:15 crc kubenswrapper[5052]: I1203 11:43:15.905551 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"plugins-conf\" (UniqueName: \"kubernetes.io/configmap/d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0-plugins-conf\") pod \"rabbitmq-server-0\" (UID: \"d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 11:43:15 crc kubenswrapper[5052]: I1203 11:43:15.905654 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0-config-data\") pod \"rabbitmq-server-0\" (UID: \"d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 11:43:15 crc kubenswrapper[5052]: I1203 11:43:15.909301 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"pod-info\" (UniqueName: \"kubernetes.io/downward-api/d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0-pod-info\") pod \"rabbitmq-server-0\" (UID: \"d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 11:43:15 crc kubenswrapper[5052]: I1203 11:43:15.909577 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"erlang-cookie-secret\" (UniqueName: \"kubernetes.io/secret/d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0-erlang-cookie-secret\") pod \"rabbitmq-server-0\" (UID: \"d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 11:43:15 crc kubenswrapper[5052]: I1203 11:43:15.909882 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"rabbitmq-confd\" (UniqueName: \"kubernetes.io/projected/d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0-rabbitmq-confd\") pod \"rabbitmq-server-0\" (UID: \"d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 11:43:15 crc kubenswrapper[5052]: I1203 11:43:15.913371 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"rabbitmq-tls\" (UniqueName: \"kubernetes.io/projected/d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0-rabbitmq-tls\") pod \"rabbitmq-server-0\" (UID: \"d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 11:43:15 crc kubenswrapper[5052]: I1203 11:43:15.920652 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-pz65r\" (UniqueName: \"kubernetes.io/projected/d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0-kube-api-access-pz65r\") pod \"rabbitmq-server-0\" (UID: \"d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 11:43:15 crc kubenswrapper[5052]: I1203 11:43:15.927808 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage07-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage07-crc\") pod \"rabbitmq-server-0\" (UID: \"d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 11:43:15 crc kubenswrapper[5052]: I1203 11:43:15.971710 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 11:43:16 crc kubenswrapper[5052]: I1203 11:43:16.052550 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" event={"ID":"2766f489-9217-47ae-a595-869a26389587","Type":"ContainerStarted","Data":"01a30dbd657eec98b6030db3277bea3757b501e6d8f09d2fab2597c1ed8d6dfb"} Dec 03 11:43:16 crc kubenswrapper[5052]: I1203 11:43:16.060218 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/openstack-galera-0"] Dec 03 11:43:16 crc kubenswrapper[5052]: I1203 11:43:16.061536 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 11:43:16 crc kubenswrapper[5052]: I1203 11:43:16.065991 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"openstack-config-data" Dec 03 11:43:16 crc kubenswrapper[5052]: I1203 11:43:16.066645 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-galera-openstack-svc" Dec 03 11:43:16 crc kubenswrapper[5052]: I1203 11:43:16.067086 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"openstack-scripts" Dec 03 11:43:16 crc kubenswrapper[5052]: I1203 11:43:16.067257 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"galera-openstack-dockercfg-cpskp" Dec 03 11:43:16 crc kubenswrapper[5052]: I1203 11:43:16.077102 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"combined-ca-bundle" Dec 03 11:43:16 crc kubenswrapper[5052]: I1203 11:43:16.083456 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/openstack-galera-0"] Dec 03 11:43:16 crc kubenswrapper[5052]: I1203 11:43:16.208042 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/e741a60a-1a4f-45a1-b315-064fb7458e8f-config-data-default\") pod \"openstack-galera-0\" (UID: \"e741a60a-1a4f-45a1-b315-064fb7458e8f\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 11:43:16 crc kubenswrapper[5052]: I1203 11:43:16.208126 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage20-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage20-crc\") pod \"openstack-galera-0\" (UID: \"e741a60a-1a4f-45a1-b315-064fb7458e8f\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 11:43:16 crc kubenswrapper[5052]: I1203 11:43:16.208153 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/e741a60a-1a4f-45a1-b315-064fb7458e8f-kolla-config\") pod \"openstack-galera-0\" (UID: \"e741a60a-1a4f-45a1-b315-064fb7458e8f\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 11:43:16 crc kubenswrapper[5052]: I1203 11:43:16.208267 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/e741a60a-1a4f-45a1-b315-064fb7458e8f-operator-scripts\") pod \"openstack-galera-0\" (UID: \"e741a60a-1a4f-45a1-b315-064fb7458e8f\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 11:43:16 crc kubenswrapper[5052]: I1203 11:43:16.208399 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/e741a60a-1a4f-45a1-b315-064fb7458e8f-config-data-generated\") pod \"openstack-galera-0\" (UID: \"e741a60a-1a4f-45a1-b315-064fb7458e8f\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 11:43:16 crc kubenswrapper[5052]: I1203 11:43:16.208433 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e741a60a-1a4f-45a1-b315-064fb7458e8f-combined-ca-bundle\") pod \"openstack-galera-0\" (UID: \"e741a60a-1a4f-45a1-b315-064fb7458e8f\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 11:43:16 crc kubenswrapper[5052]: I1203 11:43:16.208480 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n29tl\" (UniqueName: \"kubernetes.io/projected/e741a60a-1a4f-45a1-b315-064fb7458e8f-kube-api-access-n29tl\") pod \"openstack-galera-0\" (UID: \"e741a60a-1a4f-45a1-b315-064fb7458e8f\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 11:43:16 crc kubenswrapper[5052]: I1203 11:43:16.208769 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/e741a60a-1a4f-45a1-b315-064fb7458e8f-galera-tls-certs\") pod \"openstack-galera-0\" (UID: \"e741a60a-1a4f-45a1-b315-064fb7458e8f\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 11:43:16 crc kubenswrapper[5052]: I1203 11:43:16.269991 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/rabbitmq-server-0"] Dec 03 11:43:16 crc kubenswrapper[5052]: I1203 11:43:16.310047 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/e741a60a-1a4f-45a1-b315-064fb7458e8f-config-data-generated\") pod \"openstack-galera-0\" (UID: \"e741a60a-1a4f-45a1-b315-064fb7458e8f\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 11:43:16 crc kubenswrapper[5052]: I1203 11:43:16.310095 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e741a60a-1a4f-45a1-b315-064fb7458e8f-combined-ca-bundle\") pod \"openstack-galera-0\" (UID: \"e741a60a-1a4f-45a1-b315-064fb7458e8f\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 11:43:16 crc kubenswrapper[5052]: I1203 11:43:16.310132 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-n29tl\" (UniqueName: \"kubernetes.io/projected/e741a60a-1a4f-45a1-b315-064fb7458e8f-kube-api-access-n29tl\") pod \"openstack-galera-0\" (UID: \"e741a60a-1a4f-45a1-b315-064fb7458e8f\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 11:43:16 crc kubenswrapper[5052]: I1203 11:43:16.310176 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/e741a60a-1a4f-45a1-b315-064fb7458e8f-galera-tls-certs\") pod \"openstack-galera-0\" (UID: \"e741a60a-1a4f-45a1-b315-064fb7458e8f\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 11:43:16 crc kubenswrapper[5052]: I1203 11:43:16.310233 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/e741a60a-1a4f-45a1-b315-064fb7458e8f-config-data-default\") pod \"openstack-galera-0\" (UID: \"e741a60a-1a4f-45a1-b315-064fb7458e8f\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 11:43:16 crc kubenswrapper[5052]: I1203 11:43:16.310275 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage20-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage20-crc\") pod \"openstack-galera-0\" (UID: \"e741a60a-1a4f-45a1-b315-064fb7458e8f\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 11:43:16 crc kubenswrapper[5052]: I1203 11:43:16.310305 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/e741a60a-1a4f-45a1-b315-064fb7458e8f-kolla-config\") pod \"openstack-galera-0\" (UID: \"e741a60a-1a4f-45a1-b315-064fb7458e8f\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 11:43:16 crc kubenswrapper[5052]: I1203 11:43:16.310328 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/e741a60a-1a4f-45a1-b315-064fb7458e8f-operator-scripts\") pod \"openstack-galera-0\" (UID: \"e741a60a-1a4f-45a1-b315-064fb7458e8f\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 11:43:16 crc kubenswrapper[5052]: I1203 11:43:16.310914 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage20-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage20-crc\") pod \"openstack-galera-0\" (UID: \"e741a60a-1a4f-45a1-b315-064fb7458e8f\") device mount path \"/mnt/openstack/pv20\"" pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 11:43:16 crc kubenswrapper[5052]: I1203 11:43:16.311223 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/e741a60a-1a4f-45a1-b315-064fb7458e8f-config-data-generated\") pod \"openstack-galera-0\" (UID: \"e741a60a-1a4f-45a1-b315-064fb7458e8f\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 11:43:16 crc kubenswrapper[5052]: I1203 11:43:16.311458 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/e741a60a-1a4f-45a1-b315-064fb7458e8f-kolla-config\") pod \"openstack-galera-0\" (UID: \"e741a60a-1a4f-45a1-b315-064fb7458e8f\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 11:43:16 crc kubenswrapper[5052]: I1203 11:43:16.311707 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/e741a60a-1a4f-45a1-b315-064fb7458e8f-config-data-default\") pod \"openstack-galera-0\" (UID: \"e741a60a-1a4f-45a1-b315-064fb7458e8f\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 11:43:16 crc kubenswrapper[5052]: I1203 11:43:16.312257 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/e741a60a-1a4f-45a1-b315-064fb7458e8f-operator-scripts\") pod \"openstack-galera-0\" (UID: \"e741a60a-1a4f-45a1-b315-064fb7458e8f\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 11:43:16 crc kubenswrapper[5052]: I1203 11:43:16.316340 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e741a60a-1a4f-45a1-b315-064fb7458e8f-combined-ca-bundle\") pod \"openstack-galera-0\" (UID: \"e741a60a-1a4f-45a1-b315-064fb7458e8f\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 11:43:16 crc kubenswrapper[5052]: I1203 11:43:16.316949 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/e741a60a-1a4f-45a1-b315-064fb7458e8f-galera-tls-certs\") pod \"openstack-galera-0\" (UID: \"e741a60a-1a4f-45a1-b315-064fb7458e8f\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 11:43:16 crc kubenswrapper[5052]: I1203 11:43:16.326658 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-n29tl\" (UniqueName: \"kubernetes.io/projected/e741a60a-1a4f-45a1-b315-064fb7458e8f-kube-api-access-n29tl\") pod \"openstack-galera-0\" (UID: \"e741a60a-1a4f-45a1-b315-064fb7458e8f\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 11:43:16 crc kubenswrapper[5052]: W1203 11:43:16.330409 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd40fbd0a_3169_4fd1_bb23_5c5344d9a0f0.slice/crio-7711b51a7ed58585774b502006aeff8840e74cc12b62130717e231f47f598cac WatchSource:0}: Error finding container 7711b51a7ed58585774b502006aeff8840e74cc12b62130717e231f47f598cac: Status 404 returned error can't find the container with id 7711b51a7ed58585774b502006aeff8840e74cc12b62130717e231f47f598cac Dec 03 11:43:16 crc kubenswrapper[5052]: I1203 11:43:16.336973 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage20-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage20-crc\") pod \"openstack-galera-0\" (UID: \"e741a60a-1a4f-45a1-b315-064fb7458e8f\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 11:43:16 crc kubenswrapper[5052]: I1203 11:43:16.405225 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 11:43:16 crc kubenswrapper[5052]: I1203 11:43:16.822243 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/openstack-galera-0"] Dec 03 11:43:16 crc kubenswrapper[5052]: W1203 11:43:16.825577 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode741a60a_1a4f_45a1_b315_064fb7458e8f.slice/crio-3d6cb1b0ef473b63f88a4fa412741f5320a40ad57aad90117ee4b90c945ffd83 WatchSource:0}: Error finding container 3d6cb1b0ef473b63f88a4fa412741f5320a40ad57aad90117ee4b90c945ffd83: Status 404 returned error can't find the container with id 3d6cb1b0ef473b63f88a4fa412741f5320a40ad57aad90117ee4b90c945ffd83 Dec 03 11:43:17 crc kubenswrapper[5052]: I1203 11:43:17.062391 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-galera-0" event={"ID":"e741a60a-1a4f-45a1-b315-064fb7458e8f","Type":"ContainerStarted","Data":"0b842a895f06933ddfc5ac479287d4141d4c61e4c5d4906486c088a12e96d8e7"} Dec 03 11:43:17 crc kubenswrapper[5052]: I1203 11:43:17.063063 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-galera-0" event={"ID":"e741a60a-1a4f-45a1-b315-064fb7458e8f","Type":"ContainerStarted","Data":"3d6cb1b0ef473b63f88a4fa412741f5320a40ad57aad90117ee4b90c945ffd83"} Dec 03 11:43:17 crc kubenswrapper[5052]: I1203 11:43:17.064072 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-server-0" event={"ID":"d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0","Type":"ContainerStarted","Data":"7711b51a7ed58585774b502006aeff8840e74cc12b62130717e231f47f598cac"} Dec 03 11:43:17 crc kubenswrapper[5052]: I1203 11:43:17.066419 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" event={"ID":"2766f489-9217-47ae-a595-869a26389587","Type":"ContainerStarted","Data":"3b75a48ddef04dba6740dfd97e3821f7c14d8f167622e54e606f6aae843251a9"} Dec 03 11:43:17 crc kubenswrapper[5052]: I1203 11:43:17.357301 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/openstack-cell1-galera-0"] Dec 03 11:43:17 crc kubenswrapper[5052]: I1203 11:43:17.359602 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 11:43:17 crc kubenswrapper[5052]: I1203 11:43:17.362393 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"galera-openstack-cell1-dockercfg-pbqt9" Dec 03 11:43:17 crc kubenswrapper[5052]: I1203 11:43:17.362777 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"openstack-cell1-config-data" Dec 03 11:43:17 crc kubenswrapper[5052]: I1203 11:43:17.362937 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-galera-openstack-cell1-svc" Dec 03 11:43:17 crc kubenswrapper[5052]: I1203 11:43:17.368024 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"openstack-cell1-scripts" Dec 03 11:43:17 crc kubenswrapper[5052]: I1203 11:43:17.368672 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/openstack-cell1-galera-0"] Dec 03 11:43:17 crc kubenswrapper[5052]: I1203 11:43:17.428909 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage12-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage12-crc\") pod \"openstack-cell1-galera-0\" (UID: \"83be8c2e-57f8-4b45-b3ca-096f72a1defd\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 11:43:17 crc kubenswrapper[5052]: I1203 11:43:17.428983 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/83be8c2e-57f8-4b45-b3ca-096f72a1defd-config-data-default\") pod \"openstack-cell1-galera-0\" (UID: \"83be8c2e-57f8-4b45-b3ca-096f72a1defd\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 11:43:17 crc kubenswrapper[5052]: I1203 11:43:17.429014 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/83be8c2e-57f8-4b45-b3ca-096f72a1defd-kolla-config\") pod \"openstack-cell1-galera-0\" (UID: \"83be8c2e-57f8-4b45-b3ca-096f72a1defd\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 11:43:17 crc kubenswrapper[5052]: I1203 11:43:17.429036 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/83be8c2e-57f8-4b45-b3ca-096f72a1defd-galera-tls-certs\") pod \"openstack-cell1-galera-0\" (UID: \"83be8c2e-57f8-4b45-b3ca-096f72a1defd\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 11:43:17 crc kubenswrapper[5052]: I1203 11:43:17.429071 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/83be8c2e-57f8-4b45-b3ca-096f72a1defd-operator-scripts\") pod \"openstack-cell1-galera-0\" (UID: \"83be8c2e-57f8-4b45-b3ca-096f72a1defd\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 11:43:17 crc kubenswrapper[5052]: I1203 11:43:17.429250 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/83be8c2e-57f8-4b45-b3ca-096f72a1defd-config-data-generated\") pod \"openstack-cell1-galera-0\" (UID: \"83be8c2e-57f8-4b45-b3ca-096f72a1defd\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 11:43:17 crc kubenswrapper[5052]: I1203 11:43:17.429275 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fks6s\" (UniqueName: \"kubernetes.io/projected/83be8c2e-57f8-4b45-b3ca-096f72a1defd-kube-api-access-fks6s\") pod \"openstack-cell1-galera-0\" (UID: \"83be8c2e-57f8-4b45-b3ca-096f72a1defd\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 11:43:17 crc kubenswrapper[5052]: I1203 11:43:17.429295 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/83be8c2e-57f8-4b45-b3ca-096f72a1defd-combined-ca-bundle\") pod \"openstack-cell1-galera-0\" (UID: \"83be8c2e-57f8-4b45-b3ca-096f72a1defd\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 11:43:17 crc kubenswrapper[5052]: I1203 11:43:17.530471 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/83be8c2e-57f8-4b45-b3ca-096f72a1defd-config-data-default\") pod \"openstack-cell1-galera-0\" (UID: \"83be8c2e-57f8-4b45-b3ca-096f72a1defd\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 11:43:17 crc kubenswrapper[5052]: I1203 11:43:17.530756 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/83be8c2e-57f8-4b45-b3ca-096f72a1defd-kolla-config\") pod \"openstack-cell1-galera-0\" (UID: \"83be8c2e-57f8-4b45-b3ca-096f72a1defd\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 11:43:17 crc kubenswrapper[5052]: I1203 11:43:17.530867 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/83be8c2e-57f8-4b45-b3ca-096f72a1defd-galera-tls-certs\") pod \"openstack-cell1-galera-0\" (UID: \"83be8c2e-57f8-4b45-b3ca-096f72a1defd\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 11:43:17 crc kubenswrapper[5052]: I1203 11:43:17.530983 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/83be8c2e-57f8-4b45-b3ca-096f72a1defd-operator-scripts\") pod \"openstack-cell1-galera-0\" (UID: \"83be8c2e-57f8-4b45-b3ca-096f72a1defd\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 11:43:17 crc kubenswrapper[5052]: I1203 11:43:17.531123 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/83be8c2e-57f8-4b45-b3ca-096f72a1defd-config-data-generated\") pod \"openstack-cell1-galera-0\" (UID: \"83be8c2e-57f8-4b45-b3ca-096f72a1defd\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 11:43:17 crc kubenswrapper[5052]: I1203 11:43:17.531208 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-fks6s\" (UniqueName: \"kubernetes.io/projected/83be8c2e-57f8-4b45-b3ca-096f72a1defd-kube-api-access-fks6s\") pod \"openstack-cell1-galera-0\" (UID: \"83be8c2e-57f8-4b45-b3ca-096f72a1defd\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 11:43:17 crc kubenswrapper[5052]: I1203 11:43:17.531331 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/83be8c2e-57f8-4b45-b3ca-096f72a1defd-combined-ca-bundle\") pod \"openstack-cell1-galera-0\" (UID: \"83be8c2e-57f8-4b45-b3ca-096f72a1defd\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 11:43:17 crc kubenswrapper[5052]: I1203 11:43:17.531451 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage12-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage12-crc\") pod \"openstack-cell1-galera-0\" (UID: \"83be8c2e-57f8-4b45-b3ca-096f72a1defd\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 11:43:17 crc kubenswrapper[5052]: I1203 11:43:17.531798 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/83be8c2e-57f8-4b45-b3ca-096f72a1defd-kolla-config\") pod \"openstack-cell1-galera-0\" (UID: \"83be8c2e-57f8-4b45-b3ca-096f72a1defd\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 11:43:17 crc kubenswrapper[5052]: I1203 11:43:17.531901 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage12-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage12-crc\") pod \"openstack-cell1-galera-0\" (UID: \"83be8c2e-57f8-4b45-b3ca-096f72a1defd\") device mount path \"/mnt/openstack/pv12\"" pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 11:43:17 crc kubenswrapper[5052]: I1203 11:43:17.532113 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/83be8c2e-57f8-4b45-b3ca-096f72a1defd-config-data-generated\") pod \"openstack-cell1-galera-0\" (UID: \"83be8c2e-57f8-4b45-b3ca-096f72a1defd\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 11:43:17 crc kubenswrapper[5052]: I1203 11:43:17.532157 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/83be8c2e-57f8-4b45-b3ca-096f72a1defd-config-data-default\") pod \"openstack-cell1-galera-0\" (UID: \"83be8c2e-57f8-4b45-b3ca-096f72a1defd\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 11:43:17 crc kubenswrapper[5052]: I1203 11:43:17.533709 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/83be8c2e-57f8-4b45-b3ca-096f72a1defd-operator-scripts\") pod \"openstack-cell1-galera-0\" (UID: \"83be8c2e-57f8-4b45-b3ca-096f72a1defd\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 11:43:17 crc kubenswrapper[5052]: I1203 11:43:17.537088 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/83be8c2e-57f8-4b45-b3ca-096f72a1defd-galera-tls-certs\") pod \"openstack-cell1-galera-0\" (UID: \"83be8c2e-57f8-4b45-b3ca-096f72a1defd\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 11:43:17 crc kubenswrapper[5052]: I1203 11:43:17.542993 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/83be8c2e-57f8-4b45-b3ca-096f72a1defd-combined-ca-bundle\") pod \"openstack-cell1-galera-0\" (UID: \"83be8c2e-57f8-4b45-b3ca-096f72a1defd\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 11:43:17 crc kubenswrapper[5052]: I1203 11:43:17.551755 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-fks6s\" (UniqueName: \"kubernetes.io/projected/83be8c2e-57f8-4b45-b3ca-096f72a1defd-kube-api-access-fks6s\") pod \"openstack-cell1-galera-0\" (UID: \"83be8c2e-57f8-4b45-b3ca-096f72a1defd\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 11:43:17 crc kubenswrapper[5052]: I1203 11:43:17.562789 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage12-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage12-crc\") pod \"openstack-cell1-galera-0\" (UID: \"83be8c2e-57f8-4b45-b3ca-096f72a1defd\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 11:43:17 crc kubenswrapper[5052]: I1203 11:43:17.753919 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 11:43:18 crc kubenswrapper[5052]: I1203 11:43:18.079712 5052 generic.go:334] "Generic (PLEG): container finished" podID="e741a60a-1a4f-45a1-b315-064fb7458e8f" containerID="0b842a895f06933ddfc5ac479287d4141d4c61e4c5d4906486c088a12e96d8e7" exitCode=0 Dec 03 11:43:18 crc kubenswrapper[5052]: I1203 11:43:18.079786 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-galera-0" event={"ID":"e741a60a-1a4f-45a1-b315-064fb7458e8f","Type":"ContainerDied","Data":"0b842a895f06933ddfc5ac479287d4141d4c61e4c5d4906486c088a12e96d8e7"} Dec 03 11:43:18 crc kubenswrapper[5052]: I1203 11:43:18.084219 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-server-0" event={"ID":"d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0","Type":"ContainerStarted","Data":"843eb248ababc1108cc5ab5df2a9259468619c4e57c04fb2e96d893def21be0a"} Dec 03 11:43:18 crc kubenswrapper[5052]: I1203 11:43:18.233330 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/openstack-cell1-galera-0"] Dec 03 11:43:18 crc kubenswrapper[5052]: W1203 11:43:18.235918 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod83be8c2e_57f8_4b45_b3ca_096f72a1defd.slice/crio-0b060bce7e5421791fd92cbd97a19e5add5e20a00c96af5cc36dae6699841fbe WatchSource:0}: Error finding container 0b060bce7e5421791fd92cbd97a19e5add5e20a00c96af5cc36dae6699841fbe: Status 404 returned error can't find the container with id 0b060bce7e5421791fd92cbd97a19e5add5e20a00c96af5cc36dae6699841fbe Dec 03 11:43:18 crc kubenswrapper[5052]: I1203 11:43:18.758907 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/memcached-0"] Dec 03 11:43:18 crc kubenswrapper[5052]: I1203 11:43:18.760068 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/memcached-0" Dec 03 11:43:18 crc kubenswrapper[5052]: I1203 11:43:18.761799 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"memcached-config-data" Dec 03 11:43:18 crc kubenswrapper[5052]: I1203 11:43:18.762441 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"memcached-memcached-dockercfg-gn79n" Dec 03 11:43:18 crc kubenswrapper[5052]: I1203 11:43:18.762608 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-memcached-svc" Dec 03 11:43:18 crc kubenswrapper[5052]: I1203 11:43:18.767677 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/memcached-0"] Dec 03 11:43:18 crc kubenswrapper[5052]: I1203 11:43:18.952803 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"memcached-tls-certs\" (UniqueName: \"kubernetes.io/secret/6f9b208b-086f-4671-8c3f-b3a275740ad7-memcached-tls-certs\") pod \"memcached-0\" (UID: \"6f9b208b-086f-4671-8c3f-b3a275740ad7\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 11:43:18 crc kubenswrapper[5052]: I1203 11:43:18.953167 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6f9b208b-086f-4671-8c3f-b3a275740ad7-combined-ca-bundle\") pod \"memcached-0\" (UID: \"6f9b208b-086f-4671-8c3f-b3a275740ad7\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 11:43:18 crc kubenswrapper[5052]: I1203 11:43:18.953204 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/6f9b208b-086f-4671-8c3f-b3a275740ad7-config-data\") pod \"memcached-0\" (UID: \"6f9b208b-086f-4671-8c3f-b3a275740ad7\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 11:43:18 crc kubenswrapper[5052]: I1203 11:43:18.953249 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nn9xv\" (UniqueName: \"kubernetes.io/projected/6f9b208b-086f-4671-8c3f-b3a275740ad7-kube-api-access-nn9xv\") pod \"memcached-0\" (UID: \"6f9b208b-086f-4671-8c3f-b3a275740ad7\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 11:43:18 crc kubenswrapper[5052]: I1203 11:43:18.953485 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/6f9b208b-086f-4671-8c3f-b3a275740ad7-kolla-config\") pod \"memcached-0\" (UID: \"6f9b208b-086f-4671-8c3f-b3a275740ad7\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 11:43:19 crc kubenswrapper[5052]: I1203 11:43:19.054466 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/6f9b208b-086f-4671-8c3f-b3a275740ad7-kolla-config\") pod \"memcached-0\" (UID: \"6f9b208b-086f-4671-8c3f-b3a275740ad7\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 11:43:19 crc kubenswrapper[5052]: I1203 11:43:19.054564 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"memcached-tls-certs\" (UniqueName: \"kubernetes.io/secret/6f9b208b-086f-4671-8c3f-b3a275740ad7-memcached-tls-certs\") pod \"memcached-0\" (UID: \"6f9b208b-086f-4671-8c3f-b3a275740ad7\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 11:43:19 crc kubenswrapper[5052]: I1203 11:43:19.054587 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6f9b208b-086f-4671-8c3f-b3a275740ad7-combined-ca-bundle\") pod \"memcached-0\" (UID: \"6f9b208b-086f-4671-8c3f-b3a275740ad7\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 11:43:19 crc kubenswrapper[5052]: I1203 11:43:19.054607 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/6f9b208b-086f-4671-8c3f-b3a275740ad7-config-data\") pod \"memcached-0\" (UID: \"6f9b208b-086f-4671-8c3f-b3a275740ad7\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 11:43:19 crc kubenswrapper[5052]: I1203 11:43:19.054638 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-nn9xv\" (UniqueName: \"kubernetes.io/projected/6f9b208b-086f-4671-8c3f-b3a275740ad7-kube-api-access-nn9xv\") pod \"memcached-0\" (UID: \"6f9b208b-086f-4671-8c3f-b3a275740ad7\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 11:43:19 crc kubenswrapper[5052]: I1203 11:43:19.055226 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/6f9b208b-086f-4671-8c3f-b3a275740ad7-kolla-config\") pod \"memcached-0\" (UID: \"6f9b208b-086f-4671-8c3f-b3a275740ad7\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 11:43:19 crc kubenswrapper[5052]: I1203 11:43:19.056039 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/6f9b208b-086f-4671-8c3f-b3a275740ad7-config-data\") pod \"memcached-0\" (UID: \"6f9b208b-086f-4671-8c3f-b3a275740ad7\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 11:43:19 crc kubenswrapper[5052]: I1203 11:43:19.060034 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6f9b208b-086f-4671-8c3f-b3a275740ad7-combined-ca-bundle\") pod \"memcached-0\" (UID: \"6f9b208b-086f-4671-8c3f-b3a275740ad7\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 11:43:19 crc kubenswrapper[5052]: I1203 11:43:19.060217 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"memcached-tls-certs\" (UniqueName: \"kubernetes.io/secret/6f9b208b-086f-4671-8c3f-b3a275740ad7-memcached-tls-certs\") pod \"memcached-0\" (UID: \"6f9b208b-086f-4671-8c3f-b3a275740ad7\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 11:43:19 crc kubenswrapper[5052]: I1203 11:43:19.097029 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-nn9xv\" (UniqueName: \"kubernetes.io/projected/6f9b208b-086f-4671-8c3f-b3a275740ad7-kube-api-access-nn9xv\") pod \"memcached-0\" (UID: \"6f9b208b-086f-4671-8c3f-b3a275740ad7\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 11:43:19 crc kubenswrapper[5052]: I1203 11:43:19.099371 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-galera-0" event={"ID":"e741a60a-1a4f-45a1-b315-064fb7458e8f","Type":"ContainerStarted","Data":"a7e16a1558edb19db29f41cde5903f2a8a903af9d50cea81f332f651715f1903"} Dec 03 11:43:19 crc kubenswrapper[5052]: I1203 11:43:19.101474 5052 generic.go:334] "Generic (PLEG): container finished" podID="83be8c2e-57f8-4b45-b3ca-096f72a1defd" containerID="4d446fd67e4ff7b7abd8b762a4e09b8cacfc730c8985d62af75216baa471e69c" exitCode=0 Dec 03 11:43:19 crc kubenswrapper[5052]: I1203 11:43:19.102402 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-cell1-galera-0" event={"ID":"83be8c2e-57f8-4b45-b3ca-096f72a1defd","Type":"ContainerDied","Data":"4d446fd67e4ff7b7abd8b762a4e09b8cacfc730c8985d62af75216baa471e69c"} Dec 03 11:43:19 crc kubenswrapper[5052]: I1203 11:43:19.102430 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-cell1-galera-0" event={"ID":"83be8c2e-57f8-4b45-b3ca-096f72a1defd","Type":"ContainerStarted","Data":"0b060bce7e5421791fd92cbd97a19e5add5e20a00c96af5cc36dae6699841fbe"} Dec 03 11:43:19 crc kubenswrapper[5052]: I1203 11:43:19.201420 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/openstack-galera-0" podStartSLOduration=3.201399211 podStartE2EDuration="3.201399211s" podCreationTimestamp="2025-12-03 11:43:16 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:43:19.148077783 +0000 UTC m=+1389.736319537" watchObservedRunningTime="2025-12-03 11:43:19.201399211 +0000 UTC m=+1389.789640975" Dec 03 11:43:19 crc kubenswrapper[5052]: I1203 11:43:19.383308 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/memcached-0" Dec 03 11:43:19 crc kubenswrapper[5052]: I1203 11:43:19.831795 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/memcached-0"] Dec 03 11:43:19 crc kubenswrapper[5052]: W1203 11:43:19.837790 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6f9b208b_086f_4671_8c3f_b3a275740ad7.slice/crio-f836832155313a0381cab85f527e844cd45abd16cd9cc0a0b17fe34f70b740cf WatchSource:0}: Error finding container f836832155313a0381cab85f527e844cd45abd16cd9cc0a0b17fe34f70b740cf: Status 404 returned error can't find the container with id f836832155313a0381cab85f527e844cd45abd16cd9cc0a0b17fe34f70b740cf Dec 03 11:43:20 crc kubenswrapper[5052]: I1203 11:43:20.113876 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-cell1-galera-0" event={"ID":"83be8c2e-57f8-4b45-b3ca-096f72a1defd","Type":"ContainerStarted","Data":"e091f633e39fe79c62c14d716851b357c6ec86a4712a030e1839dbe830fb60a3"} Dec 03 11:43:20 crc kubenswrapper[5052]: I1203 11:43:20.120442 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/memcached-0" event={"ID":"6f9b208b-086f-4671-8c3f-b3a275740ad7","Type":"ContainerStarted","Data":"af1f689f6ec28c7e342622784d72e045524b9ad80e36d71f89fc38ea43408004"} Dec 03 11:43:20 crc kubenswrapper[5052]: I1203 11:43:20.120484 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/memcached-0" event={"ID":"6f9b208b-086f-4671-8c3f-b3a275740ad7","Type":"ContainerStarted","Data":"f836832155313a0381cab85f527e844cd45abd16cd9cc0a0b17fe34f70b740cf"} Dec 03 11:43:20 crc kubenswrapper[5052]: I1203 11:43:20.120502 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/memcached-0" Dec 03 11:43:20 crc kubenswrapper[5052]: I1203 11:43:20.149452 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/openstack-cell1-galera-0" podStartSLOduration=3.149434519 podStartE2EDuration="3.149434519s" podCreationTimestamp="2025-12-03 11:43:17 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:43:20.145667571 +0000 UTC m=+1390.733909335" watchObservedRunningTime="2025-12-03 11:43:20.149434519 +0000 UTC m=+1390.737676283" Dec 03 11:43:20 crc kubenswrapper[5052]: I1203 11:43:20.169138 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/memcached-0" podStartSLOduration=2.169114657 podStartE2EDuration="2.169114657s" podCreationTimestamp="2025-12-03 11:43:18 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:43:20.166399868 +0000 UTC m=+1390.754641652" watchObservedRunningTime="2025-12-03 11:43:20.169114657 +0000 UTC m=+1390.757356431" Dec 03 11:43:20 crc kubenswrapper[5052]: I1203 11:43:20.396149 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/kube-state-metrics-0"] Dec 03 11:43:20 crc kubenswrapper[5052]: I1203 11:43:20.397168 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 11:43:20 crc kubenswrapper[5052]: I1203 11:43:20.406593 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/kube-state-metrics-0"] Dec 03 11:43:20 crc kubenswrapper[5052]: I1203 11:43:20.410413 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"telemetry-ceilometer-dockercfg-6sbzv" Dec 03 11:43:20 crc kubenswrapper[5052]: I1203 11:43:20.479296 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tnf7p\" (UniqueName: \"kubernetes.io/projected/78388bce-bd9d-426d-b6b6-e0fba8b3d8f9-kube-api-access-tnf7p\") pod \"kube-state-metrics-0\" (UID: \"78388bce-bd9d-426d-b6b6-e0fba8b3d8f9\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 11:43:20 crc kubenswrapper[5052]: I1203 11:43:20.581137 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-tnf7p\" (UniqueName: \"kubernetes.io/projected/78388bce-bd9d-426d-b6b6-e0fba8b3d8f9-kube-api-access-tnf7p\") pod \"kube-state-metrics-0\" (UID: \"78388bce-bd9d-426d-b6b6-e0fba8b3d8f9\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 11:43:20 crc kubenswrapper[5052]: I1203 11:43:20.603593 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-tnf7p\" (UniqueName: \"kubernetes.io/projected/78388bce-bd9d-426d-b6b6-e0fba8b3d8f9-kube-api-access-tnf7p\") pod \"kube-state-metrics-0\" (UID: \"78388bce-bd9d-426d-b6b6-e0fba8b3d8f9\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 11:43:20 crc kubenswrapper[5052]: I1203 11:43:20.718538 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 11:43:21 crc kubenswrapper[5052]: I1203 11:43:21.130678 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/kube-state-metrics-0"] Dec 03 11:43:22 crc kubenswrapper[5052]: I1203 11:43:22.139644 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/kube-state-metrics-0" event={"ID":"78388bce-bd9d-426d-b6b6-e0fba8b3d8f9","Type":"ContainerStarted","Data":"e284c3d794c0b8aa02b660d2a83e53e3db40486717ae4fa7ff4311373d5644eb"} Dec 03 11:43:22 crc kubenswrapper[5052]: I1203 11:43:22.140039 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/kube-state-metrics-0" event={"ID":"78388bce-bd9d-426d-b6b6-e0fba8b3d8f9","Type":"ContainerStarted","Data":"3777b5fc38dd6dccf7902b068ada7954bec5d1b4b67fbe026aa85e7b5f1bf0d3"} Dec 03 11:43:22 crc kubenswrapper[5052]: I1203 11:43:22.140064 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 11:43:22 crc kubenswrapper[5052]: I1203 11:43:22.156003 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/kube-state-metrics-0" podStartSLOduration=1.811306642 podStartE2EDuration="2.155986161s" podCreationTimestamp="2025-12-03 11:43:20 +0000 UTC" firstStartedPulling="2025-12-03 11:43:21.141818767 +0000 UTC m=+1391.730060531" lastFinishedPulling="2025-12-03 11:43:21.486498286 +0000 UTC m=+1392.074740050" observedRunningTime="2025-12-03 11:43:22.154140148 +0000 UTC m=+1392.742381912" watchObservedRunningTime="2025-12-03 11:43:22.155986161 +0000 UTC m=+1392.744227925" Dec 03 11:43:22 crc kubenswrapper[5052]: I1203 11:43:22.923196 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ovsdbserver-nb-0"] Dec 03 11:43:22 crc kubenswrapper[5052]: I1203 11:43:22.924483 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 11:43:22 crc kubenswrapper[5052]: I1203 11:43:22.927146 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ovncluster-ovndbcluster-nb-dockercfg-t9tcx" Dec 03 11:43:22 crc kubenswrapper[5052]: I1203 11:43:22.927708 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-ovn-metrics" Dec 03 11:43:22 crc kubenswrapper[5052]: I1203 11:43:22.928544 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"ovndbcluster-nb-config" Dec 03 11:43:22 crc kubenswrapper[5052]: I1203 11:43:22.928932 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"ovndbcluster-nb-scripts" Dec 03 11:43:22 crc kubenswrapper[5052]: I1203 11:43:22.931896 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-ovndbcluster-nb-ovndbs" Dec 03 11:43:22 crc kubenswrapper[5052]: I1203 11:43:22.940969 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-nb-0"] Dec 03 11:43:23 crc kubenswrapper[5052]: I1203 11:43:23.019751 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/68f829f3-e972-4168-8b75-27c086a91142-metrics-certs-tls-certs\") pod \"ovsdbserver-nb-0\" (UID: \"68f829f3-e972-4168-8b75-27c086a91142\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 11:43:23 crc kubenswrapper[5052]: I1203 11:43:23.019993 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovsdbserver-nb-tls-certs\" (UniqueName: \"kubernetes.io/secret/68f829f3-e972-4168-8b75-27c086a91142-ovsdbserver-nb-tls-certs\") pod \"ovsdbserver-nb-0\" (UID: \"68f829f3-e972-4168-8b75-27c086a91142\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 11:43:23 crc kubenswrapper[5052]: I1203 11:43:23.020126 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/68f829f3-e972-4168-8b75-27c086a91142-config\") pod \"ovsdbserver-nb-0\" (UID: \"68f829f3-e972-4168-8b75-27c086a91142\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 11:43:23 crc kubenswrapper[5052]: I1203 11:43:23.020176 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage14-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage14-crc\") pod \"ovsdbserver-nb-0\" (UID: \"68f829f3-e972-4168-8b75-27c086a91142\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 11:43:23 crc kubenswrapper[5052]: I1203 11:43:23.020217 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/68f829f3-e972-4168-8b75-27c086a91142-scripts\") pod \"ovsdbserver-nb-0\" (UID: \"68f829f3-e972-4168-8b75-27c086a91142\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 11:43:23 crc kubenswrapper[5052]: I1203 11:43:23.020320 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gb5sd\" (UniqueName: \"kubernetes.io/projected/68f829f3-e972-4168-8b75-27c086a91142-kube-api-access-gb5sd\") pod \"ovsdbserver-nb-0\" (UID: \"68f829f3-e972-4168-8b75-27c086a91142\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 11:43:23 crc kubenswrapper[5052]: I1203 11:43:23.020371 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/68f829f3-e972-4168-8b75-27c086a91142-ovsdb-rundir\") pod \"ovsdbserver-nb-0\" (UID: \"68f829f3-e972-4168-8b75-27c086a91142\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 11:43:23 crc kubenswrapper[5052]: I1203 11:43:23.020396 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/68f829f3-e972-4168-8b75-27c086a91142-combined-ca-bundle\") pod \"ovsdbserver-nb-0\" (UID: \"68f829f3-e972-4168-8b75-27c086a91142\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 11:43:23 crc kubenswrapper[5052]: I1203 11:43:23.122227 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovsdbserver-nb-tls-certs\" (UniqueName: \"kubernetes.io/secret/68f829f3-e972-4168-8b75-27c086a91142-ovsdbserver-nb-tls-certs\") pod \"ovsdbserver-nb-0\" (UID: \"68f829f3-e972-4168-8b75-27c086a91142\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 11:43:23 crc kubenswrapper[5052]: I1203 11:43:23.122312 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/68f829f3-e972-4168-8b75-27c086a91142-config\") pod \"ovsdbserver-nb-0\" (UID: \"68f829f3-e972-4168-8b75-27c086a91142\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 11:43:23 crc kubenswrapper[5052]: I1203 11:43:23.122343 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage14-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage14-crc\") pod \"ovsdbserver-nb-0\" (UID: \"68f829f3-e972-4168-8b75-27c086a91142\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 11:43:23 crc kubenswrapper[5052]: I1203 11:43:23.122368 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/68f829f3-e972-4168-8b75-27c086a91142-scripts\") pod \"ovsdbserver-nb-0\" (UID: \"68f829f3-e972-4168-8b75-27c086a91142\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 11:43:23 crc kubenswrapper[5052]: I1203 11:43:23.122415 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-gb5sd\" (UniqueName: \"kubernetes.io/projected/68f829f3-e972-4168-8b75-27c086a91142-kube-api-access-gb5sd\") pod \"ovsdbserver-nb-0\" (UID: \"68f829f3-e972-4168-8b75-27c086a91142\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 11:43:23 crc kubenswrapper[5052]: I1203 11:43:23.122439 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/68f829f3-e972-4168-8b75-27c086a91142-ovsdb-rundir\") pod \"ovsdbserver-nb-0\" (UID: \"68f829f3-e972-4168-8b75-27c086a91142\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 11:43:23 crc kubenswrapper[5052]: I1203 11:43:23.122458 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/68f829f3-e972-4168-8b75-27c086a91142-combined-ca-bundle\") pod \"ovsdbserver-nb-0\" (UID: \"68f829f3-e972-4168-8b75-27c086a91142\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 11:43:23 crc kubenswrapper[5052]: I1203 11:43:23.122499 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/68f829f3-e972-4168-8b75-27c086a91142-metrics-certs-tls-certs\") pod \"ovsdbserver-nb-0\" (UID: \"68f829f3-e972-4168-8b75-27c086a91142\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 11:43:23 crc kubenswrapper[5052]: I1203 11:43:23.122999 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage14-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage14-crc\") pod \"ovsdbserver-nb-0\" (UID: \"68f829f3-e972-4168-8b75-27c086a91142\") device mount path \"/mnt/openstack/pv14\"" pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 11:43:23 crc kubenswrapper[5052]: I1203 11:43:23.123622 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/68f829f3-e972-4168-8b75-27c086a91142-ovsdb-rundir\") pod \"ovsdbserver-nb-0\" (UID: \"68f829f3-e972-4168-8b75-27c086a91142\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 11:43:23 crc kubenswrapper[5052]: I1203 11:43:23.123770 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/68f829f3-e972-4168-8b75-27c086a91142-config\") pod \"ovsdbserver-nb-0\" (UID: \"68f829f3-e972-4168-8b75-27c086a91142\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 11:43:23 crc kubenswrapper[5052]: I1203 11:43:23.124040 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/68f829f3-e972-4168-8b75-27c086a91142-scripts\") pod \"ovsdbserver-nb-0\" (UID: \"68f829f3-e972-4168-8b75-27c086a91142\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 11:43:23 crc kubenswrapper[5052]: I1203 11:43:23.129087 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovsdbserver-nb-tls-certs\" (UniqueName: \"kubernetes.io/secret/68f829f3-e972-4168-8b75-27c086a91142-ovsdbserver-nb-tls-certs\") pod \"ovsdbserver-nb-0\" (UID: \"68f829f3-e972-4168-8b75-27c086a91142\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 11:43:23 crc kubenswrapper[5052]: I1203 11:43:23.131311 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/68f829f3-e972-4168-8b75-27c086a91142-combined-ca-bundle\") pod \"ovsdbserver-nb-0\" (UID: \"68f829f3-e972-4168-8b75-27c086a91142\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 11:43:23 crc kubenswrapper[5052]: I1203 11:43:23.136269 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/68f829f3-e972-4168-8b75-27c086a91142-metrics-certs-tls-certs\") pod \"ovsdbserver-nb-0\" (UID: \"68f829f3-e972-4168-8b75-27c086a91142\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 11:43:23 crc kubenswrapper[5052]: I1203 11:43:23.141064 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-gb5sd\" (UniqueName: \"kubernetes.io/projected/68f829f3-e972-4168-8b75-27c086a91142-kube-api-access-gb5sd\") pod \"ovsdbserver-nb-0\" (UID: \"68f829f3-e972-4168-8b75-27c086a91142\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 11:43:23 crc kubenswrapper[5052]: I1203 11:43:23.149802 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage14-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage14-crc\") pod \"ovsdbserver-nb-0\" (UID: \"68f829f3-e972-4168-8b75-27c086a91142\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 11:43:23 crc kubenswrapper[5052]: I1203 11:43:23.241973 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 11:43:23 crc kubenswrapper[5052]: I1203 11:43:23.766329 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-nb-0"] Dec 03 11:43:24 crc kubenswrapper[5052]: I1203 11:43:24.163572 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-nb-0" event={"ID":"68f829f3-e972-4168-8b75-27c086a91142","Type":"ContainerStarted","Data":"630e406fa52125d27b16eb3de57c1b6dc5a497b24481482e2ba05d01fcd553fc"} Dec 03 11:43:24 crc kubenswrapper[5052]: I1203 11:43:24.163927 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-nb-0" event={"ID":"68f829f3-e972-4168-8b75-27c086a91142","Type":"ContainerStarted","Data":"57f10ab9d3a1c544593f058c6724e34ab08ea61ce5e3535d93586cc3d5834467"} Dec 03 11:43:24 crc kubenswrapper[5052]: I1203 11:43:24.163944 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-nb-0" event={"ID":"68f829f3-e972-4168-8b75-27c086a91142","Type":"ContainerStarted","Data":"b0f88b9592cc31cff4aa379dab071b70e918d627b3c3673ec2d6280a722488c7"} Dec 03 11:43:24 crc kubenswrapper[5052]: I1203 11:43:24.195078 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/ovsdbserver-nb-0" podStartSLOduration=2.195055132 podStartE2EDuration="2.195055132s" podCreationTimestamp="2025-12-03 11:43:22 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:43:24.187288668 +0000 UTC m=+1394.775530432" watchObservedRunningTime="2025-12-03 11:43:24.195055132 +0000 UTC m=+1394.783296896" Dec 03 11:43:25 crc kubenswrapper[5052]: I1203 11:43:25.736055 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ovsdbserver-sb-0"] Dec 03 11:43:25 crc kubenswrapper[5052]: I1203 11:43:25.738177 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 11:43:25 crc kubenswrapper[5052]: I1203 11:43:25.740065 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ovncluster-ovndbcluster-sb-dockercfg-8q2jq" Dec 03 11:43:25 crc kubenswrapper[5052]: I1203 11:43:25.740269 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-ovndbcluster-sb-ovndbs" Dec 03 11:43:25 crc kubenswrapper[5052]: I1203 11:43:25.740489 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"ovndbcluster-sb-config" Dec 03 11:43:25 crc kubenswrapper[5052]: I1203 11:43:25.746138 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"ovndbcluster-sb-scripts" Dec 03 11:43:25 crc kubenswrapper[5052]: I1203 11:43:25.749924 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-sb-0"] Dec 03 11:43:25 crc kubenswrapper[5052]: I1203 11:43:25.778073 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/9bdd2626-de50-4513-8313-3b25347ff641-scripts\") pod \"ovsdbserver-sb-0\" (UID: \"9bdd2626-de50-4513-8313-3b25347ff641\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 11:43:25 crc kubenswrapper[5052]: I1203 11:43:25.778295 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/9bdd2626-de50-4513-8313-3b25347ff641-config\") pod \"ovsdbserver-sb-0\" (UID: \"9bdd2626-de50-4513-8313-3b25347ff641\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 11:43:25 crc kubenswrapper[5052]: I1203 11:43:25.778386 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovsdbserver-sb-tls-certs\" (UniqueName: \"kubernetes.io/secret/9bdd2626-de50-4513-8313-3b25347ff641-ovsdbserver-sb-tls-certs\") pod \"ovsdbserver-sb-0\" (UID: \"9bdd2626-de50-4513-8313-3b25347ff641\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 11:43:25 crc kubenswrapper[5052]: I1203 11:43:25.778496 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage01-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage01-crc\") pod \"ovsdbserver-sb-0\" (UID: \"9bdd2626-de50-4513-8313-3b25347ff641\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 11:43:25 crc kubenswrapper[5052]: I1203 11:43:25.778606 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9bdd2626-de50-4513-8313-3b25347ff641-combined-ca-bundle\") pod \"ovsdbserver-sb-0\" (UID: \"9bdd2626-de50-4513-8313-3b25347ff641\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 11:43:25 crc kubenswrapper[5052]: I1203 11:43:25.778673 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/9bdd2626-de50-4513-8313-3b25347ff641-metrics-certs-tls-certs\") pod \"ovsdbserver-sb-0\" (UID: \"9bdd2626-de50-4513-8313-3b25347ff641\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 11:43:25 crc kubenswrapper[5052]: I1203 11:43:25.778746 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/9bdd2626-de50-4513-8313-3b25347ff641-ovsdb-rundir\") pod \"ovsdbserver-sb-0\" (UID: \"9bdd2626-de50-4513-8313-3b25347ff641\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 11:43:25 crc kubenswrapper[5052]: I1203 11:43:25.778809 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nxpnw\" (UniqueName: \"kubernetes.io/projected/9bdd2626-de50-4513-8313-3b25347ff641-kube-api-access-nxpnw\") pod \"ovsdbserver-sb-0\" (UID: \"9bdd2626-de50-4513-8313-3b25347ff641\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 11:43:25 crc kubenswrapper[5052]: I1203 11:43:25.880809 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/9bdd2626-de50-4513-8313-3b25347ff641-scripts\") pod \"ovsdbserver-sb-0\" (UID: \"9bdd2626-de50-4513-8313-3b25347ff641\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 11:43:25 crc kubenswrapper[5052]: I1203 11:43:25.880889 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/9bdd2626-de50-4513-8313-3b25347ff641-config\") pod \"ovsdbserver-sb-0\" (UID: \"9bdd2626-de50-4513-8313-3b25347ff641\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 11:43:25 crc kubenswrapper[5052]: I1203 11:43:25.880918 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovsdbserver-sb-tls-certs\" (UniqueName: \"kubernetes.io/secret/9bdd2626-de50-4513-8313-3b25347ff641-ovsdbserver-sb-tls-certs\") pod \"ovsdbserver-sb-0\" (UID: \"9bdd2626-de50-4513-8313-3b25347ff641\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 11:43:25 crc kubenswrapper[5052]: I1203 11:43:25.880965 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage01-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage01-crc\") pod \"ovsdbserver-sb-0\" (UID: \"9bdd2626-de50-4513-8313-3b25347ff641\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 11:43:25 crc kubenswrapper[5052]: I1203 11:43:25.880980 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9bdd2626-de50-4513-8313-3b25347ff641-combined-ca-bundle\") pod \"ovsdbserver-sb-0\" (UID: \"9bdd2626-de50-4513-8313-3b25347ff641\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 11:43:25 crc kubenswrapper[5052]: I1203 11:43:25.880997 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/9bdd2626-de50-4513-8313-3b25347ff641-metrics-certs-tls-certs\") pod \"ovsdbserver-sb-0\" (UID: \"9bdd2626-de50-4513-8313-3b25347ff641\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 11:43:25 crc kubenswrapper[5052]: I1203 11:43:25.881022 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/9bdd2626-de50-4513-8313-3b25347ff641-ovsdb-rundir\") pod \"ovsdbserver-sb-0\" (UID: \"9bdd2626-de50-4513-8313-3b25347ff641\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 11:43:25 crc kubenswrapper[5052]: I1203 11:43:25.881037 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-nxpnw\" (UniqueName: \"kubernetes.io/projected/9bdd2626-de50-4513-8313-3b25347ff641-kube-api-access-nxpnw\") pod \"ovsdbserver-sb-0\" (UID: \"9bdd2626-de50-4513-8313-3b25347ff641\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 11:43:25 crc kubenswrapper[5052]: I1203 11:43:25.881769 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage01-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage01-crc\") pod \"ovsdbserver-sb-0\" (UID: \"9bdd2626-de50-4513-8313-3b25347ff641\") device mount path \"/mnt/openstack/pv01\"" pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 11:43:25 crc kubenswrapper[5052]: I1203 11:43:25.881852 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/9bdd2626-de50-4513-8313-3b25347ff641-ovsdb-rundir\") pod \"ovsdbserver-sb-0\" (UID: \"9bdd2626-de50-4513-8313-3b25347ff641\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 11:43:25 crc kubenswrapper[5052]: I1203 11:43:25.882651 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/9bdd2626-de50-4513-8313-3b25347ff641-scripts\") pod \"ovsdbserver-sb-0\" (UID: \"9bdd2626-de50-4513-8313-3b25347ff641\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 11:43:25 crc kubenswrapper[5052]: I1203 11:43:25.882650 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/9bdd2626-de50-4513-8313-3b25347ff641-config\") pod \"ovsdbserver-sb-0\" (UID: \"9bdd2626-de50-4513-8313-3b25347ff641\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 11:43:25 crc kubenswrapper[5052]: I1203 11:43:25.891366 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovsdbserver-sb-tls-certs\" (UniqueName: \"kubernetes.io/secret/9bdd2626-de50-4513-8313-3b25347ff641-ovsdbserver-sb-tls-certs\") pod \"ovsdbserver-sb-0\" (UID: \"9bdd2626-de50-4513-8313-3b25347ff641\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 11:43:25 crc kubenswrapper[5052]: I1203 11:43:25.892284 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9bdd2626-de50-4513-8313-3b25347ff641-combined-ca-bundle\") pod \"ovsdbserver-sb-0\" (UID: \"9bdd2626-de50-4513-8313-3b25347ff641\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 11:43:25 crc kubenswrapper[5052]: I1203 11:43:25.894170 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/9bdd2626-de50-4513-8313-3b25347ff641-metrics-certs-tls-certs\") pod \"ovsdbserver-sb-0\" (UID: \"9bdd2626-de50-4513-8313-3b25347ff641\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 11:43:25 crc kubenswrapper[5052]: I1203 11:43:25.901394 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-nxpnw\" (UniqueName: \"kubernetes.io/projected/9bdd2626-de50-4513-8313-3b25347ff641-kube-api-access-nxpnw\") pod \"ovsdbserver-sb-0\" (UID: \"9bdd2626-de50-4513-8313-3b25347ff641\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 11:43:25 crc kubenswrapper[5052]: I1203 11:43:25.903716 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage01-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage01-crc\") pod \"ovsdbserver-sb-0\" (UID: \"9bdd2626-de50-4513-8313-3b25347ff641\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 11:43:26 crc kubenswrapper[5052]: I1203 11:43:26.076364 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 11:43:26 crc kubenswrapper[5052]: I1203 11:43:26.141628 5052 scope.go:117] "RemoveContainer" containerID="4371847f7354c50c0cc712325674ada84304f41aaf647cb3acc17b089cb235e7" Dec 03 11:43:26 crc kubenswrapper[5052]: I1203 11:43:26.242947 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 11:43:26 crc kubenswrapper[5052]: I1203 11:43:26.298899 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 11:43:26 crc kubenswrapper[5052]: I1203 11:43:26.406535 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 11:43:26 crc kubenswrapper[5052]: I1203 11:43:26.406583 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 11:43:26 crc kubenswrapper[5052]: I1203 11:43:26.482672 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 11:43:26 crc kubenswrapper[5052]: I1203 11:43:26.520494 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-sb-0"] Dec 03 11:43:26 crc kubenswrapper[5052]: W1203 11:43:26.521467 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9bdd2626_de50_4513_8313_3b25347ff641.slice/crio-307909d852a32b4c9508700a79e3efd7ffe02fd3cf77902dfa87a114307367df WatchSource:0}: Error finding container 307909d852a32b4c9508700a79e3efd7ffe02fd3cf77902dfa87a114307367df: Status 404 returned error can't find the container with id 307909d852a32b4c9508700a79e3efd7ffe02fd3cf77902dfa87a114307367df Dec 03 11:43:27 crc kubenswrapper[5052]: I1203 11:43:27.205173 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-sb-0" event={"ID":"9bdd2626-de50-4513-8313-3b25347ff641","Type":"ContainerStarted","Data":"d6c0926e352f7c2fd9039729545261adf7fb3d6ef95c52edff8cf47f36d45db5"} Dec 03 11:43:27 crc kubenswrapper[5052]: I1203 11:43:27.205472 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 11:43:27 crc kubenswrapper[5052]: I1203 11:43:27.205488 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-sb-0" event={"ID":"9bdd2626-de50-4513-8313-3b25347ff641","Type":"ContainerStarted","Data":"5e28858cdce596489bc28d8ac4d74cd0163a29ba280b7ffd53c1e8c3d713fddb"} Dec 03 11:43:27 crc kubenswrapper[5052]: I1203 11:43:27.205502 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-sb-0" event={"ID":"9bdd2626-de50-4513-8313-3b25347ff641","Type":"ContainerStarted","Data":"307909d852a32b4c9508700a79e3efd7ffe02fd3cf77902dfa87a114307367df"} Dec 03 11:43:27 crc kubenswrapper[5052]: I1203 11:43:27.232744 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/ovsdbserver-sb-0" podStartSLOduration=2.232719048 podStartE2EDuration="2.232719048s" podCreationTimestamp="2025-12-03 11:43:25 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:43:27.225577402 +0000 UTC m=+1397.813819186" watchObservedRunningTime="2025-12-03 11:43:27.232719048 +0000 UTC m=+1397.820960812" Dec 03 11:43:27 crc kubenswrapper[5052]: I1203 11:43:27.280136 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 11:43:27 crc kubenswrapper[5052]: I1203 11:43:27.754707 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 11:43:27 crc kubenswrapper[5052]: I1203 11:43:27.772459 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 11:43:27 crc kubenswrapper[5052]: I1203 11:43:27.840605 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 11:43:28 crc kubenswrapper[5052]: I1203 11:43:28.267746 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 11:43:28 crc kubenswrapper[5052]: I1203 11:43:28.314321 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 11:43:28 crc kubenswrapper[5052]: I1203 11:43:28.685544 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/keystone-db-create-h55rj"] Dec 03 11:43:28 crc kubenswrapper[5052]: I1203 11:43:28.687013 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-db-create-h55rj" Dec 03 11:43:28 crc kubenswrapper[5052]: I1203 11:43:28.691798 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-db-create-h55rj"] Dec 03 11:43:28 crc kubenswrapper[5052]: I1203 11:43:28.789885 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/keystone-c226-account-create-update-4scft"] Dec 03 11:43:28 crc kubenswrapper[5052]: I1203 11:43:28.791944 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-c226-account-create-update-4scft" Dec 03 11:43:28 crc kubenswrapper[5052]: I1203 11:43:28.794881 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone-db-secret" Dec 03 11:43:28 crc kubenswrapper[5052]: I1203 11:43:28.798369 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-c226-account-create-update-4scft"] Dec 03 11:43:28 crc kubenswrapper[5052]: I1203 11:43:28.839645 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/679ac416-0e30-4016-bb4a-59944d03a695-operator-scripts\") pod \"keystone-db-create-h55rj\" (UID: \"679ac416-0e30-4016-bb4a-59944d03a695\") " pod="openstack-kuttl-tests/keystone-db-create-h55rj" Dec 03 11:43:28 crc kubenswrapper[5052]: I1203 11:43:28.839760 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zs78g\" (UniqueName: \"kubernetes.io/projected/679ac416-0e30-4016-bb4a-59944d03a695-kube-api-access-zs78g\") pod \"keystone-db-create-h55rj\" (UID: \"679ac416-0e30-4016-bb4a-59944d03a695\") " pod="openstack-kuttl-tests/keystone-db-create-h55rj" Dec 03 11:43:28 crc kubenswrapper[5052]: I1203 11:43:28.940977 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/516527bf-2b7b-4bd1-b1a1-8d2ea94fbf16-operator-scripts\") pod \"keystone-c226-account-create-update-4scft\" (UID: \"516527bf-2b7b-4bd1-b1a1-8d2ea94fbf16\") " pod="openstack-kuttl-tests/keystone-c226-account-create-update-4scft" Dec 03 11:43:28 crc kubenswrapper[5052]: I1203 11:43:28.941054 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/679ac416-0e30-4016-bb4a-59944d03a695-operator-scripts\") pod \"keystone-db-create-h55rj\" (UID: \"679ac416-0e30-4016-bb4a-59944d03a695\") " pod="openstack-kuttl-tests/keystone-db-create-h55rj" Dec 03 11:43:28 crc kubenswrapper[5052]: I1203 11:43:28.941127 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-zs78g\" (UniqueName: \"kubernetes.io/projected/679ac416-0e30-4016-bb4a-59944d03a695-kube-api-access-zs78g\") pod \"keystone-db-create-h55rj\" (UID: \"679ac416-0e30-4016-bb4a-59944d03a695\") " pod="openstack-kuttl-tests/keystone-db-create-h55rj" Dec 03 11:43:28 crc kubenswrapper[5052]: I1203 11:43:28.941156 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mvqvq\" (UniqueName: \"kubernetes.io/projected/516527bf-2b7b-4bd1-b1a1-8d2ea94fbf16-kube-api-access-mvqvq\") pod \"keystone-c226-account-create-update-4scft\" (UID: \"516527bf-2b7b-4bd1-b1a1-8d2ea94fbf16\") " pod="openstack-kuttl-tests/keystone-c226-account-create-update-4scft" Dec 03 11:43:28 crc kubenswrapper[5052]: I1203 11:43:28.941911 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/679ac416-0e30-4016-bb4a-59944d03a695-operator-scripts\") pod \"keystone-db-create-h55rj\" (UID: \"679ac416-0e30-4016-bb4a-59944d03a695\") " pod="openstack-kuttl-tests/keystone-db-create-h55rj" Dec 03 11:43:28 crc kubenswrapper[5052]: I1203 11:43:28.973383 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-zs78g\" (UniqueName: \"kubernetes.io/projected/679ac416-0e30-4016-bb4a-59944d03a695-kube-api-access-zs78g\") pod \"keystone-db-create-h55rj\" (UID: \"679ac416-0e30-4016-bb4a-59944d03a695\") " pod="openstack-kuttl-tests/keystone-db-create-h55rj" Dec 03 11:43:28 crc kubenswrapper[5052]: I1203 11:43:28.987551 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/placement-db-create-8tl8n"] Dec 03 11:43:28 crc kubenswrapper[5052]: I1203 11:43:28.988550 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-db-create-8tl8n" Dec 03 11:43:29 crc kubenswrapper[5052]: I1203 11:43:29.003024 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/placement-db-create-8tl8n"] Dec 03 11:43:29 crc kubenswrapper[5052]: I1203 11:43:29.007264 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-db-create-h55rj" Dec 03 11:43:29 crc kubenswrapper[5052]: I1203 11:43:29.042978 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-mvqvq\" (UniqueName: \"kubernetes.io/projected/516527bf-2b7b-4bd1-b1a1-8d2ea94fbf16-kube-api-access-mvqvq\") pod \"keystone-c226-account-create-update-4scft\" (UID: \"516527bf-2b7b-4bd1-b1a1-8d2ea94fbf16\") " pod="openstack-kuttl-tests/keystone-c226-account-create-update-4scft" Dec 03 11:43:29 crc kubenswrapper[5052]: I1203 11:43:29.043458 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/516527bf-2b7b-4bd1-b1a1-8d2ea94fbf16-operator-scripts\") pod \"keystone-c226-account-create-update-4scft\" (UID: \"516527bf-2b7b-4bd1-b1a1-8d2ea94fbf16\") " pod="openstack-kuttl-tests/keystone-c226-account-create-update-4scft" Dec 03 11:43:29 crc kubenswrapper[5052]: I1203 11:43:29.044182 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/516527bf-2b7b-4bd1-b1a1-8d2ea94fbf16-operator-scripts\") pod \"keystone-c226-account-create-update-4scft\" (UID: \"516527bf-2b7b-4bd1-b1a1-8d2ea94fbf16\") " pod="openstack-kuttl-tests/keystone-c226-account-create-update-4scft" Dec 03 11:43:29 crc kubenswrapper[5052]: I1203 11:43:29.080380 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 11:43:29 crc kubenswrapper[5052]: I1203 11:43:29.081242 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-mvqvq\" (UniqueName: \"kubernetes.io/projected/516527bf-2b7b-4bd1-b1a1-8d2ea94fbf16-kube-api-access-mvqvq\") pod \"keystone-c226-account-create-update-4scft\" (UID: \"516527bf-2b7b-4bd1-b1a1-8d2ea94fbf16\") " pod="openstack-kuttl-tests/keystone-c226-account-create-update-4scft" Dec 03 11:43:29 crc kubenswrapper[5052]: I1203 11:43:29.097919 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/placement-9a4f-account-create-update-gs4xv"] Dec 03 11:43:29 crc kubenswrapper[5052]: I1203 11:43:29.099010 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-9a4f-account-create-update-gs4xv" Dec 03 11:43:29 crc kubenswrapper[5052]: I1203 11:43:29.101377 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"placement-db-secret" Dec 03 11:43:29 crc kubenswrapper[5052]: I1203 11:43:29.106353 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/placement-9a4f-account-create-update-gs4xv"] Dec 03 11:43:29 crc kubenswrapper[5052]: I1203 11:43:29.119498 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-c226-account-create-update-4scft" Dec 03 11:43:29 crc kubenswrapper[5052]: I1203 11:43:29.148204 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/a0a527da-143e-4d58-a118-0a1afea55b1f-operator-scripts\") pod \"placement-db-create-8tl8n\" (UID: \"a0a527da-143e-4d58-a118-0a1afea55b1f\") " pod="openstack-kuttl-tests/placement-db-create-8tl8n" Dec 03 11:43:29 crc kubenswrapper[5052]: I1203 11:43:29.148620 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9m5fq\" (UniqueName: \"kubernetes.io/projected/a0a527da-143e-4d58-a118-0a1afea55b1f-kube-api-access-9m5fq\") pod \"placement-db-create-8tl8n\" (UID: \"a0a527da-143e-4d58-a118-0a1afea55b1f\") " pod="openstack-kuttl-tests/placement-db-create-8tl8n" Dec 03 11:43:29 crc kubenswrapper[5052]: I1203 11:43:29.164533 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 11:43:29 crc kubenswrapper[5052]: I1203 11:43:29.228499 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 11:43:29 crc kubenswrapper[5052]: I1203 11:43:29.250810 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rnwmx\" (UniqueName: \"kubernetes.io/projected/75e340f6-2225-485f-8c78-53d05538a9c6-kube-api-access-rnwmx\") pod \"placement-9a4f-account-create-update-gs4xv\" (UID: \"75e340f6-2225-485f-8c78-53d05538a9c6\") " pod="openstack-kuttl-tests/placement-9a4f-account-create-update-gs4xv" Dec 03 11:43:29 crc kubenswrapper[5052]: I1203 11:43:29.250958 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-9m5fq\" (UniqueName: \"kubernetes.io/projected/a0a527da-143e-4d58-a118-0a1afea55b1f-kube-api-access-9m5fq\") pod \"placement-db-create-8tl8n\" (UID: \"a0a527da-143e-4d58-a118-0a1afea55b1f\") " pod="openstack-kuttl-tests/placement-db-create-8tl8n" Dec 03 11:43:29 crc kubenswrapper[5052]: I1203 11:43:29.251079 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/75e340f6-2225-485f-8c78-53d05538a9c6-operator-scripts\") pod \"placement-9a4f-account-create-update-gs4xv\" (UID: \"75e340f6-2225-485f-8c78-53d05538a9c6\") " pod="openstack-kuttl-tests/placement-9a4f-account-create-update-gs4xv" Dec 03 11:43:29 crc kubenswrapper[5052]: I1203 11:43:29.251133 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/a0a527da-143e-4d58-a118-0a1afea55b1f-operator-scripts\") pod \"placement-db-create-8tl8n\" (UID: \"a0a527da-143e-4d58-a118-0a1afea55b1f\") " pod="openstack-kuttl-tests/placement-db-create-8tl8n" Dec 03 11:43:29 crc kubenswrapper[5052]: I1203 11:43:29.252252 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/a0a527da-143e-4d58-a118-0a1afea55b1f-operator-scripts\") pod \"placement-db-create-8tl8n\" (UID: \"a0a527da-143e-4d58-a118-0a1afea55b1f\") " pod="openstack-kuttl-tests/placement-db-create-8tl8n" Dec 03 11:43:29 crc kubenswrapper[5052]: I1203 11:43:29.275968 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-9m5fq\" (UniqueName: \"kubernetes.io/projected/a0a527da-143e-4d58-a118-0a1afea55b1f-kube-api-access-9m5fq\") pod \"placement-db-create-8tl8n\" (UID: \"a0a527da-143e-4d58-a118-0a1afea55b1f\") " pod="openstack-kuttl-tests/placement-db-create-8tl8n" Dec 03 11:43:29 crc kubenswrapper[5052]: I1203 11:43:29.353031 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-rnwmx\" (UniqueName: \"kubernetes.io/projected/75e340f6-2225-485f-8c78-53d05538a9c6-kube-api-access-rnwmx\") pod \"placement-9a4f-account-create-update-gs4xv\" (UID: \"75e340f6-2225-485f-8c78-53d05538a9c6\") " pod="openstack-kuttl-tests/placement-9a4f-account-create-update-gs4xv" Dec 03 11:43:29 crc kubenswrapper[5052]: I1203 11:43:29.353249 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/75e340f6-2225-485f-8c78-53d05538a9c6-operator-scripts\") pod \"placement-9a4f-account-create-update-gs4xv\" (UID: \"75e340f6-2225-485f-8c78-53d05538a9c6\") " pod="openstack-kuttl-tests/placement-9a4f-account-create-update-gs4xv" Dec 03 11:43:29 crc kubenswrapper[5052]: I1203 11:43:29.355895 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/75e340f6-2225-485f-8c78-53d05538a9c6-operator-scripts\") pod \"placement-9a4f-account-create-update-gs4xv\" (UID: \"75e340f6-2225-485f-8c78-53d05538a9c6\") " pod="openstack-kuttl-tests/placement-9a4f-account-create-update-gs4xv" Dec 03 11:43:29 crc kubenswrapper[5052]: I1203 11:43:29.372175 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-rnwmx\" (UniqueName: \"kubernetes.io/projected/75e340f6-2225-485f-8c78-53d05538a9c6-kube-api-access-rnwmx\") pod \"placement-9a4f-account-create-update-gs4xv\" (UID: \"75e340f6-2225-485f-8c78-53d05538a9c6\") " pod="openstack-kuttl-tests/placement-9a4f-account-create-update-gs4xv" Dec 03 11:43:29 crc kubenswrapper[5052]: I1203 11:43:29.384944 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/memcached-0" Dec 03 11:43:29 crc kubenswrapper[5052]: I1203 11:43:29.432783 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-db-create-8tl8n" Dec 03 11:43:29 crc kubenswrapper[5052]: I1203 11:43:29.466898 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-db-create-h55rj"] Dec 03 11:43:29 crc kubenswrapper[5052]: I1203 11:43:29.521614 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-9a4f-account-create-update-gs4xv" Dec 03 11:43:29 crc kubenswrapper[5052]: I1203 11:43:29.534405 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/glance-db-create-l4qfz"] Dec 03 11:43:29 crc kubenswrapper[5052]: I1203 11:43:29.535614 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-db-create-l4qfz" Dec 03 11:43:29 crc kubenswrapper[5052]: I1203 11:43:29.547352 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-db-create-l4qfz"] Dec 03 11:43:29 crc kubenswrapper[5052]: I1203 11:43:29.584276 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-c226-account-create-update-4scft"] Dec 03 11:43:29 crc kubenswrapper[5052]: W1203 11:43:29.593263 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod516527bf_2b7b_4bd1_b1a1_8d2ea94fbf16.slice/crio-f09ccaac6f5e7efb7a6ac48cb6e2a819a03b40517e21c6fe9f8ce6c74b50b6cc WatchSource:0}: Error finding container f09ccaac6f5e7efb7a6ac48cb6e2a819a03b40517e21c6fe9f8ce6c74b50b6cc: Status 404 returned error can't find the container with id f09ccaac6f5e7efb7a6ac48cb6e2a819a03b40517e21c6fe9f8ce6c74b50b6cc Dec 03 11:43:29 crc kubenswrapper[5052]: I1203 11:43:29.643291 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/glance-3554-account-create-update-n48gm"] Dec 03 11:43:29 crc kubenswrapper[5052]: I1203 11:43:29.659639 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/d4682522-2ae7-4e71-95ff-c4453f145537-operator-scripts\") pod \"glance-db-create-l4qfz\" (UID: \"d4682522-2ae7-4e71-95ff-c4453f145537\") " pod="openstack-kuttl-tests/glance-db-create-l4qfz" Dec 03 11:43:29 crc kubenswrapper[5052]: I1203 11:43:29.659696 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v4g5x\" (UniqueName: \"kubernetes.io/projected/d4682522-2ae7-4e71-95ff-c4453f145537-kube-api-access-v4g5x\") pod \"glance-db-create-l4qfz\" (UID: \"d4682522-2ae7-4e71-95ff-c4453f145537\") " pod="openstack-kuttl-tests/glance-db-create-l4qfz" Dec 03 11:43:29 crc kubenswrapper[5052]: I1203 11:43:29.660659 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-3554-account-create-update-n48gm" Dec 03 11:43:29 crc kubenswrapper[5052]: I1203 11:43:29.661429 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-3554-account-create-update-n48gm"] Dec 03 11:43:29 crc kubenswrapper[5052]: I1203 11:43:29.662986 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-db-secret" Dec 03 11:43:29 crc kubenswrapper[5052]: I1203 11:43:29.763802 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2858h\" (UniqueName: \"kubernetes.io/projected/bd7f2307-3030-4f83-b90c-debe14f2da96-kube-api-access-2858h\") pod \"glance-3554-account-create-update-n48gm\" (UID: \"bd7f2307-3030-4f83-b90c-debe14f2da96\") " pod="openstack-kuttl-tests/glance-3554-account-create-update-n48gm" Dec 03 11:43:29 crc kubenswrapper[5052]: I1203 11:43:29.764784 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/bd7f2307-3030-4f83-b90c-debe14f2da96-operator-scripts\") pod \"glance-3554-account-create-update-n48gm\" (UID: \"bd7f2307-3030-4f83-b90c-debe14f2da96\") " pod="openstack-kuttl-tests/glance-3554-account-create-update-n48gm" Dec 03 11:43:29 crc kubenswrapper[5052]: I1203 11:43:29.764839 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/d4682522-2ae7-4e71-95ff-c4453f145537-operator-scripts\") pod \"glance-db-create-l4qfz\" (UID: \"d4682522-2ae7-4e71-95ff-c4453f145537\") " pod="openstack-kuttl-tests/glance-db-create-l4qfz" Dec 03 11:43:29 crc kubenswrapper[5052]: I1203 11:43:29.764866 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-v4g5x\" (UniqueName: \"kubernetes.io/projected/d4682522-2ae7-4e71-95ff-c4453f145537-kube-api-access-v4g5x\") pod \"glance-db-create-l4qfz\" (UID: \"d4682522-2ae7-4e71-95ff-c4453f145537\") " pod="openstack-kuttl-tests/glance-db-create-l4qfz" Dec 03 11:43:29 crc kubenswrapper[5052]: I1203 11:43:29.765791 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/d4682522-2ae7-4e71-95ff-c4453f145537-operator-scripts\") pod \"glance-db-create-l4qfz\" (UID: \"d4682522-2ae7-4e71-95ff-c4453f145537\") " pod="openstack-kuttl-tests/glance-db-create-l4qfz" Dec 03 11:43:29 crc kubenswrapper[5052]: I1203 11:43:29.784977 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-v4g5x\" (UniqueName: \"kubernetes.io/projected/d4682522-2ae7-4e71-95ff-c4453f145537-kube-api-access-v4g5x\") pod \"glance-db-create-l4qfz\" (UID: \"d4682522-2ae7-4e71-95ff-c4453f145537\") " pod="openstack-kuttl-tests/glance-db-create-l4qfz" Dec 03 11:43:29 crc kubenswrapper[5052]: I1203 11:43:29.858292 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-db-create-l4qfz" Dec 03 11:43:29 crc kubenswrapper[5052]: I1203 11:43:29.870951 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/bd7f2307-3030-4f83-b90c-debe14f2da96-operator-scripts\") pod \"glance-3554-account-create-update-n48gm\" (UID: \"bd7f2307-3030-4f83-b90c-debe14f2da96\") " pod="openstack-kuttl-tests/glance-3554-account-create-update-n48gm" Dec 03 11:43:29 crc kubenswrapper[5052]: I1203 11:43:29.872061 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/bd7f2307-3030-4f83-b90c-debe14f2da96-operator-scripts\") pod \"glance-3554-account-create-update-n48gm\" (UID: \"bd7f2307-3030-4f83-b90c-debe14f2da96\") " pod="openstack-kuttl-tests/glance-3554-account-create-update-n48gm" Dec 03 11:43:29 crc kubenswrapper[5052]: I1203 11:43:29.872355 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-2858h\" (UniqueName: \"kubernetes.io/projected/bd7f2307-3030-4f83-b90c-debe14f2da96-kube-api-access-2858h\") pod \"glance-3554-account-create-update-n48gm\" (UID: \"bd7f2307-3030-4f83-b90c-debe14f2da96\") " pod="openstack-kuttl-tests/glance-3554-account-create-update-n48gm" Dec 03 11:43:29 crc kubenswrapper[5052]: I1203 11:43:29.889157 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-2858h\" (UniqueName: \"kubernetes.io/projected/bd7f2307-3030-4f83-b90c-debe14f2da96-kube-api-access-2858h\") pod \"glance-3554-account-create-update-n48gm\" (UID: \"bd7f2307-3030-4f83-b90c-debe14f2da96\") " pod="openstack-kuttl-tests/glance-3554-account-create-update-n48gm" Dec 03 11:43:29 crc kubenswrapper[5052]: I1203 11:43:29.933247 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/placement-db-create-8tl8n"] Dec 03 11:43:29 crc kubenswrapper[5052]: I1203 11:43:29.991213 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-3554-account-create-update-n48gm" Dec 03 11:43:30 crc kubenswrapper[5052]: I1203 11:43:30.055840 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/placement-9a4f-account-create-update-gs4xv"] Dec 03 11:43:30 crc kubenswrapper[5052]: W1203 11:43:30.073216 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod75e340f6_2225_485f_8c78_53d05538a9c6.slice/crio-6098ce23f74ba061fce4b9c98f821511be0ecff3bbc0cf194364feba321081a4 WatchSource:0}: Error finding container 6098ce23f74ba061fce4b9c98f821511be0ecff3bbc0cf194364feba321081a4: Status 404 returned error can't find the container with id 6098ce23f74ba061fce4b9c98f821511be0ecff3bbc0cf194364feba321081a4 Dec 03 11:43:30 crc kubenswrapper[5052]: I1203 11:43:30.244926 5052 generic.go:334] "Generic (PLEG): container finished" podID="679ac416-0e30-4016-bb4a-59944d03a695" containerID="b4718facd796f54b8c673f27e81c4cebe41d2467bb0f69d919998113712b9cce" exitCode=0 Dec 03 11:43:30 crc kubenswrapper[5052]: I1203 11:43:30.245045 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-db-create-h55rj" event={"ID":"679ac416-0e30-4016-bb4a-59944d03a695","Type":"ContainerDied","Data":"b4718facd796f54b8c673f27e81c4cebe41d2467bb0f69d919998113712b9cce"} Dec 03 11:43:30 crc kubenswrapper[5052]: I1203 11:43:30.245137 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-db-create-h55rj" event={"ID":"679ac416-0e30-4016-bb4a-59944d03a695","Type":"ContainerStarted","Data":"a1df51a6766eff161ec9234793fd2e460970a7899ca153691ae5e4b113a482be"} Dec 03 11:43:30 crc kubenswrapper[5052]: I1203 11:43:30.249052 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-db-create-8tl8n" event={"ID":"a0a527da-143e-4d58-a118-0a1afea55b1f","Type":"ContainerStarted","Data":"69907479e7bbfe021243aeedcaf4a5b347779ce00ebbdb792d2b8dba78f212c0"} Dec 03 11:43:30 crc kubenswrapper[5052]: I1203 11:43:30.253915 5052 generic.go:334] "Generic (PLEG): container finished" podID="516527bf-2b7b-4bd1-b1a1-8d2ea94fbf16" containerID="4a26b9bf2f464394dfe8329500db73ffc7c0cc73ad437ed732610bb4cd958539" exitCode=0 Dec 03 11:43:30 crc kubenswrapper[5052]: I1203 11:43:30.254017 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-c226-account-create-update-4scft" event={"ID":"516527bf-2b7b-4bd1-b1a1-8d2ea94fbf16","Type":"ContainerDied","Data":"4a26b9bf2f464394dfe8329500db73ffc7c0cc73ad437ed732610bb4cd958539"} Dec 03 11:43:30 crc kubenswrapper[5052]: I1203 11:43:30.254068 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-c226-account-create-update-4scft" event={"ID":"516527bf-2b7b-4bd1-b1a1-8d2ea94fbf16","Type":"ContainerStarted","Data":"f09ccaac6f5e7efb7a6ac48cb6e2a819a03b40517e21c6fe9f8ce6c74b50b6cc"} Dec 03 11:43:30 crc kubenswrapper[5052]: I1203 11:43:30.255586 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-9a4f-account-create-update-gs4xv" event={"ID":"75e340f6-2225-485f-8c78-53d05538a9c6","Type":"ContainerStarted","Data":"6098ce23f74ba061fce4b9c98f821511be0ecff3bbc0cf194364feba321081a4"} Dec 03 11:43:30 crc kubenswrapper[5052]: I1203 11:43:30.311949 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-db-create-l4qfz"] Dec 03 11:43:30 crc kubenswrapper[5052]: W1203 11:43:30.314257 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd4682522_2ae7_4e71_95ff_c4453f145537.slice/crio-69be8b14b679ad394c8838dfabac4771b8c84511de760083fed5d3495f219b03 WatchSource:0}: Error finding container 69be8b14b679ad394c8838dfabac4771b8c84511de760083fed5d3495f219b03: Status 404 returned error can't find the container with id 69be8b14b679ad394c8838dfabac4771b8c84511de760083fed5d3495f219b03 Dec 03 11:43:30 crc kubenswrapper[5052]: I1203 11:43:30.451337 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-3554-account-create-update-n48gm"] Dec 03 11:43:30 crc kubenswrapper[5052]: I1203 11:43:30.585135 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/swift-storage-0"] Dec 03 11:43:30 crc kubenswrapper[5052]: I1203 11:43:30.637622 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/swift-storage-0"] Dec 03 11:43:30 crc kubenswrapper[5052]: I1203 11:43:30.637893 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-storage-0" Dec 03 11:43:30 crc kubenswrapper[5052]: I1203 11:43:30.642594 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"swift-ring-files" Dec 03 11:43:30 crc kubenswrapper[5052]: I1203 11:43:30.646624 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"swift-conf" Dec 03 11:43:30 crc kubenswrapper[5052]: I1203 11:43:30.647625 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"swift-storage-config-data" Dec 03 11:43:30 crc kubenswrapper[5052]: I1203 11:43:30.647764 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"swift-swift-dockercfg-s89mn" Dec 03 11:43:30 crc kubenswrapper[5052]: I1203 11:43:30.684513 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/93ca3376-6916-4fc9-8b76-7cd9a930865e-etc-swift\") pod \"swift-storage-0\" (UID: \"93ca3376-6916-4fc9-8b76-7cd9a930865e\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 11:43:30 crc kubenswrapper[5052]: I1203 11:43:30.684608 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage02-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage02-crc\") pod \"swift-storage-0\" (UID: \"93ca3376-6916-4fc9-8b76-7cd9a930865e\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 11:43:30 crc kubenswrapper[5052]: I1203 11:43:30.684642 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lock\" (UniqueName: \"kubernetes.io/empty-dir/93ca3376-6916-4fc9-8b76-7cd9a930865e-lock\") pod \"swift-storage-0\" (UID: \"93ca3376-6916-4fc9-8b76-7cd9a930865e\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 11:43:30 crc kubenswrapper[5052]: I1203 11:43:30.684705 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6qxz4\" (UniqueName: \"kubernetes.io/projected/93ca3376-6916-4fc9-8b76-7cd9a930865e-kube-api-access-6qxz4\") pod \"swift-storage-0\" (UID: \"93ca3376-6916-4fc9-8b76-7cd9a930865e\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 11:43:30 crc kubenswrapper[5052]: I1203 11:43:30.684736 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cache\" (UniqueName: \"kubernetes.io/empty-dir/93ca3376-6916-4fc9-8b76-7cd9a930865e-cache\") pod \"swift-storage-0\" (UID: \"93ca3376-6916-4fc9-8b76-7cd9a930865e\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 11:43:30 crc kubenswrapper[5052]: I1203 11:43:30.725909 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 11:43:30 crc kubenswrapper[5052]: I1203 11:43:30.728217 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/swift-ring-rebalance-lc79q"] Dec 03 11:43:30 crc kubenswrapper[5052]: I1203 11:43:30.730009 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-ring-rebalance-lc79q" Dec 03 11:43:30 crc kubenswrapper[5052]: I1203 11:43:30.737778 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"swift-proxy-config-data" Dec 03 11:43:30 crc kubenswrapper[5052]: I1203 11:43:30.738054 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"swift-ring-config-data" Dec 03 11:43:30 crc kubenswrapper[5052]: I1203 11:43:30.738217 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"swift-ring-scripts" Dec 03 11:43:30 crc kubenswrapper[5052]: I1203 11:43:30.743575 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/swift-ring-rebalance-lc79q"] Dec 03 11:43:30 crc kubenswrapper[5052]: I1203 11:43:30.771692 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/swift-ring-rebalance-lc79q"] Dec 03 11:43:30 crc kubenswrapper[5052]: I1203 11:43:30.780741 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/swift-ring-rebalance-r8qtm"] Dec 03 11:43:30 crc kubenswrapper[5052]: I1203 11:43:30.782151 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-ring-rebalance-r8qtm" Dec 03 11:43:30 crc kubenswrapper[5052]: E1203 11:43:30.784980 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[combined-ca-bundle dispersionconf etc-swift kube-api-access-2cvbm ring-data-devices scripts swiftconf], unattached volumes=[], failed to process volumes=[]: context canceled" pod="openstack-kuttl-tests/swift-ring-rebalance-lc79q" podUID="71b35848-69b6-4456-a0d1-8fee11ae3938" Dec 03 11:43:30 crc kubenswrapper[5052]: I1203 11:43:30.785409 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"swiftconf\" (UniqueName: \"kubernetes.io/secret/71b35848-69b6-4456-a0d1-8fee11ae3938-swiftconf\") pod \"swift-ring-rebalance-lc79q\" (UID: \"71b35848-69b6-4456-a0d1-8fee11ae3938\") " pod="openstack-kuttl-tests/swift-ring-rebalance-lc79q" Dec 03 11:43:30 crc kubenswrapper[5052]: I1203 11:43:30.785456 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/93ca3376-6916-4fc9-8b76-7cd9a930865e-etc-swift\") pod \"swift-storage-0\" (UID: \"93ca3376-6916-4fc9-8b76-7cd9a930865e\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 11:43:30 crc kubenswrapper[5052]: I1203 11:43:30.785476 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage02-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage02-crc\") pod \"swift-storage-0\" (UID: \"93ca3376-6916-4fc9-8b76-7cd9a930865e\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 11:43:30 crc kubenswrapper[5052]: I1203 11:43:30.785496 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ring-data-devices\" (UniqueName: \"kubernetes.io/configmap/71b35848-69b6-4456-a0d1-8fee11ae3938-ring-data-devices\") pod \"swift-ring-rebalance-lc79q\" (UID: \"71b35848-69b6-4456-a0d1-8fee11ae3938\") " pod="openstack-kuttl-tests/swift-ring-rebalance-lc79q" Dec 03 11:43:30 crc kubenswrapper[5052]: I1203 11:43:30.785551 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"lock\" (UniqueName: \"kubernetes.io/empty-dir/93ca3376-6916-4fc9-8b76-7cd9a930865e-lock\") pod \"swift-storage-0\" (UID: \"93ca3376-6916-4fc9-8b76-7cd9a930865e\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 11:43:30 crc kubenswrapper[5052]: I1203 11:43:30.785578 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dispersionconf\" (UniqueName: \"kubernetes.io/secret/71b35848-69b6-4456-a0d1-8fee11ae3938-dispersionconf\") pod \"swift-ring-rebalance-lc79q\" (UID: \"71b35848-69b6-4456-a0d1-8fee11ae3938\") " pod="openstack-kuttl-tests/swift-ring-rebalance-lc79q" Dec 03 11:43:30 crc kubenswrapper[5052]: I1203 11:43:30.785603 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/71b35848-69b6-4456-a0d1-8fee11ae3938-scripts\") pod \"swift-ring-rebalance-lc79q\" (UID: \"71b35848-69b6-4456-a0d1-8fee11ae3938\") " pod="openstack-kuttl-tests/swift-ring-rebalance-lc79q" Dec 03 11:43:30 crc kubenswrapper[5052]: I1203 11:43:30.785659 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-6qxz4\" (UniqueName: \"kubernetes.io/projected/93ca3376-6916-4fc9-8b76-7cd9a930865e-kube-api-access-6qxz4\") pod \"swift-storage-0\" (UID: \"93ca3376-6916-4fc9-8b76-7cd9a930865e\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 11:43:30 crc kubenswrapper[5052]: I1203 11:43:30.785684 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2cvbm\" (UniqueName: \"kubernetes.io/projected/71b35848-69b6-4456-a0d1-8fee11ae3938-kube-api-access-2cvbm\") pod \"swift-ring-rebalance-lc79q\" (UID: \"71b35848-69b6-4456-a0d1-8fee11ae3938\") " pod="openstack-kuttl-tests/swift-ring-rebalance-lc79q" Dec 03 11:43:30 crc kubenswrapper[5052]: E1203 11:43:30.785701 5052 projected.go:288] Couldn't get configMap openstack-kuttl-tests/swift-ring-files: configmap "swift-ring-files" not found Dec 03 11:43:30 crc kubenswrapper[5052]: I1203 11:43:30.785717 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"cache\" (UniqueName: \"kubernetes.io/empty-dir/93ca3376-6916-4fc9-8b76-7cd9a930865e-cache\") pod \"swift-storage-0\" (UID: \"93ca3376-6916-4fc9-8b76-7cd9a930865e\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 11:43:30 crc kubenswrapper[5052]: I1203 11:43:30.785787 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/empty-dir/71b35848-69b6-4456-a0d1-8fee11ae3938-etc-swift\") pod \"swift-ring-rebalance-lc79q\" (UID: \"71b35848-69b6-4456-a0d1-8fee11ae3938\") " pod="openstack-kuttl-tests/swift-ring-rebalance-lc79q" Dec 03 11:43:30 crc kubenswrapper[5052]: I1203 11:43:30.785814 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/71b35848-69b6-4456-a0d1-8fee11ae3938-combined-ca-bundle\") pod \"swift-ring-rebalance-lc79q\" (UID: \"71b35848-69b6-4456-a0d1-8fee11ae3938\") " pod="openstack-kuttl-tests/swift-ring-rebalance-lc79q" Dec 03 11:43:30 crc kubenswrapper[5052]: I1203 11:43:30.786161 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage02-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage02-crc\") pod \"swift-storage-0\" (UID: \"93ca3376-6916-4fc9-8b76-7cd9a930865e\") device mount path \"/mnt/openstack/pv02\"" pod="openstack-kuttl-tests/swift-storage-0" Dec 03 11:43:30 crc kubenswrapper[5052]: E1203 11:43:30.785721 5052 projected.go:194] Error preparing data for projected volume etc-swift for pod openstack-kuttl-tests/swift-storage-0: configmap "swift-ring-files" not found Dec 03 11:43:30 crc kubenswrapper[5052]: E1203 11:43:30.788335 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/93ca3376-6916-4fc9-8b76-7cd9a930865e-etc-swift podName:93ca3376-6916-4fc9-8b76-7cd9a930865e nodeName:}" failed. No retries permitted until 2025-12-03 11:43:31.28831422 +0000 UTC m=+1401.876556084 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "etc-swift" (UniqueName: "kubernetes.io/projected/93ca3376-6916-4fc9-8b76-7cd9a930865e-etc-swift") pod "swift-storage-0" (UID: "93ca3376-6916-4fc9-8b76-7cd9a930865e") : configmap "swift-ring-files" not found Dec 03 11:43:30 crc kubenswrapper[5052]: I1203 11:43:30.788462 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"lock\" (UniqueName: \"kubernetes.io/empty-dir/93ca3376-6916-4fc9-8b76-7cd9a930865e-lock\") pod \"swift-storage-0\" (UID: \"93ca3376-6916-4fc9-8b76-7cd9a930865e\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 11:43:30 crc kubenswrapper[5052]: I1203 11:43:30.788841 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"cache\" (UniqueName: \"kubernetes.io/empty-dir/93ca3376-6916-4fc9-8b76-7cd9a930865e-cache\") pod \"swift-storage-0\" (UID: \"93ca3376-6916-4fc9-8b76-7cd9a930865e\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 11:43:30 crc kubenswrapper[5052]: I1203 11:43:30.810572 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/swift-ring-rebalance-r8qtm"] Dec 03 11:43:30 crc kubenswrapper[5052]: I1203 11:43:30.812864 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-6qxz4\" (UniqueName: \"kubernetes.io/projected/93ca3376-6916-4fc9-8b76-7cd9a930865e-kube-api-access-6qxz4\") pod \"swift-storage-0\" (UID: \"93ca3376-6916-4fc9-8b76-7cd9a930865e\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 11:43:30 crc kubenswrapper[5052]: I1203 11:43:30.818445 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage02-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage02-crc\") pod \"swift-storage-0\" (UID: \"93ca3376-6916-4fc9-8b76-7cd9a930865e\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 11:43:30 crc kubenswrapper[5052]: I1203 11:43:30.886868 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"swiftconf\" (UniqueName: \"kubernetes.io/secret/c3a670b4-73f2-4c90-a3af-ba5d660ec1f7-swiftconf\") pod \"swift-ring-rebalance-r8qtm\" (UID: \"c3a670b4-73f2-4c90-a3af-ba5d660ec1f7\") " pod="openstack-kuttl-tests/swift-ring-rebalance-r8qtm" Dec 03 11:43:30 crc kubenswrapper[5052]: I1203 11:43:30.886924 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/empty-dir/c3a670b4-73f2-4c90-a3af-ba5d660ec1f7-etc-swift\") pod \"swift-ring-rebalance-r8qtm\" (UID: \"c3a670b4-73f2-4c90-a3af-ba5d660ec1f7\") " pod="openstack-kuttl-tests/swift-ring-rebalance-r8qtm" Dec 03 11:43:30 crc kubenswrapper[5052]: I1203 11:43:30.886950 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/c3a670b4-73f2-4c90-a3af-ba5d660ec1f7-scripts\") pod \"swift-ring-rebalance-r8qtm\" (UID: \"c3a670b4-73f2-4c90-a3af-ba5d660ec1f7\") " pod="openstack-kuttl-tests/swift-ring-rebalance-r8qtm" Dec 03 11:43:30 crc kubenswrapper[5052]: I1203 11:43:30.886973 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/empty-dir/71b35848-69b6-4456-a0d1-8fee11ae3938-etc-swift\") pod \"swift-ring-rebalance-lc79q\" (UID: \"71b35848-69b6-4456-a0d1-8fee11ae3938\") " pod="openstack-kuttl-tests/swift-ring-rebalance-lc79q" Dec 03 11:43:30 crc kubenswrapper[5052]: I1203 11:43:30.886991 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/71b35848-69b6-4456-a0d1-8fee11ae3938-combined-ca-bundle\") pod \"swift-ring-rebalance-lc79q\" (UID: \"71b35848-69b6-4456-a0d1-8fee11ae3938\") " pod="openstack-kuttl-tests/swift-ring-rebalance-lc79q" Dec 03 11:43:30 crc kubenswrapper[5052]: I1203 11:43:30.887026 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"swiftconf\" (UniqueName: \"kubernetes.io/secret/71b35848-69b6-4456-a0d1-8fee11ae3938-swiftconf\") pod \"swift-ring-rebalance-lc79q\" (UID: \"71b35848-69b6-4456-a0d1-8fee11ae3938\") " pod="openstack-kuttl-tests/swift-ring-rebalance-lc79q" Dec 03 11:43:30 crc kubenswrapper[5052]: I1203 11:43:30.887082 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-84ktv\" (UniqueName: \"kubernetes.io/projected/c3a670b4-73f2-4c90-a3af-ba5d660ec1f7-kube-api-access-84ktv\") pod \"swift-ring-rebalance-r8qtm\" (UID: \"c3a670b4-73f2-4c90-a3af-ba5d660ec1f7\") " pod="openstack-kuttl-tests/swift-ring-rebalance-r8qtm" Dec 03 11:43:30 crc kubenswrapper[5052]: I1203 11:43:30.887105 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ring-data-devices\" (UniqueName: \"kubernetes.io/configmap/71b35848-69b6-4456-a0d1-8fee11ae3938-ring-data-devices\") pod \"swift-ring-rebalance-lc79q\" (UID: \"71b35848-69b6-4456-a0d1-8fee11ae3938\") " pod="openstack-kuttl-tests/swift-ring-rebalance-lc79q" Dec 03 11:43:30 crc kubenswrapper[5052]: I1203 11:43:30.887122 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dispersionconf\" (UniqueName: \"kubernetes.io/secret/c3a670b4-73f2-4c90-a3af-ba5d660ec1f7-dispersionconf\") pod \"swift-ring-rebalance-r8qtm\" (UID: \"c3a670b4-73f2-4c90-a3af-ba5d660ec1f7\") " pod="openstack-kuttl-tests/swift-ring-rebalance-r8qtm" Dec 03 11:43:30 crc kubenswrapper[5052]: I1203 11:43:30.887143 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ring-data-devices\" (UniqueName: \"kubernetes.io/configmap/c3a670b4-73f2-4c90-a3af-ba5d660ec1f7-ring-data-devices\") pod \"swift-ring-rebalance-r8qtm\" (UID: \"c3a670b4-73f2-4c90-a3af-ba5d660ec1f7\") " pod="openstack-kuttl-tests/swift-ring-rebalance-r8qtm" Dec 03 11:43:30 crc kubenswrapper[5052]: I1203 11:43:30.887159 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"dispersionconf\" (UniqueName: \"kubernetes.io/secret/71b35848-69b6-4456-a0d1-8fee11ae3938-dispersionconf\") pod \"swift-ring-rebalance-lc79q\" (UID: \"71b35848-69b6-4456-a0d1-8fee11ae3938\") " pod="openstack-kuttl-tests/swift-ring-rebalance-lc79q" Dec 03 11:43:30 crc kubenswrapper[5052]: I1203 11:43:30.887177 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/71b35848-69b6-4456-a0d1-8fee11ae3938-scripts\") pod \"swift-ring-rebalance-lc79q\" (UID: \"71b35848-69b6-4456-a0d1-8fee11ae3938\") " pod="openstack-kuttl-tests/swift-ring-rebalance-lc79q" Dec 03 11:43:30 crc kubenswrapper[5052]: I1203 11:43:30.887202 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c3a670b4-73f2-4c90-a3af-ba5d660ec1f7-combined-ca-bundle\") pod \"swift-ring-rebalance-r8qtm\" (UID: \"c3a670b4-73f2-4c90-a3af-ba5d660ec1f7\") " pod="openstack-kuttl-tests/swift-ring-rebalance-r8qtm" Dec 03 11:43:30 crc kubenswrapper[5052]: I1203 11:43:30.887230 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-2cvbm\" (UniqueName: \"kubernetes.io/projected/71b35848-69b6-4456-a0d1-8fee11ae3938-kube-api-access-2cvbm\") pod \"swift-ring-rebalance-lc79q\" (UID: \"71b35848-69b6-4456-a0d1-8fee11ae3938\") " pod="openstack-kuttl-tests/swift-ring-rebalance-lc79q" Dec 03 11:43:30 crc kubenswrapper[5052]: I1203 11:43:30.887999 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"etc-swift\" (UniqueName: \"kubernetes.io/empty-dir/71b35848-69b6-4456-a0d1-8fee11ae3938-etc-swift\") pod \"swift-ring-rebalance-lc79q\" (UID: \"71b35848-69b6-4456-a0d1-8fee11ae3938\") " pod="openstack-kuttl-tests/swift-ring-rebalance-lc79q" Dec 03 11:43:30 crc kubenswrapper[5052]: I1203 11:43:30.888312 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/71b35848-69b6-4456-a0d1-8fee11ae3938-scripts\") pod \"swift-ring-rebalance-lc79q\" (UID: \"71b35848-69b6-4456-a0d1-8fee11ae3938\") " pod="openstack-kuttl-tests/swift-ring-rebalance-lc79q" Dec 03 11:43:30 crc kubenswrapper[5052]: I1203 11:43:30.888805 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ring-data-devices\" (UniqueName: \"kubernetes.io/configmap/71b35848-69b6-4456-a0d1-8fee11ae3938-ring-data-devices\") pod \"swift-ring-rebalance-lc79q\" (UID: \"71b35848-69b6-4456-a0d1-8fee11ae3938\") " pod="openstack-kuttl-tests/swift-ring-rebalance-lc79q" Dec 03 11:43:30 crc kubenswrapper[5052]: I1203 11:43:30.890467 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"swiftconf\" (UniqueName: \"kubernetes.io/secret/71b35848-69b6-4456-a0d1-8fee11ae3938-swiftconf\") pod \"swift-ring-rebalance-lc79q\" (UID: \"71b35848-69b6-4456-a0d1-8fee11ae3938\") " pod="openstack-kuttl-tests/swift-ring-rebalance-lc79q" Dec 03 11:43:30 crc kubenswrapper[5052]: I1203 11:43:30.891954 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"dispersionconf\" (UniqueName: \"kubernetes.io/secret/71b35848-69b6-4456-a0d1-8fee11ae3938-dispersionconf\") pod \"swift-ring-rebalance-lc79q\" (UID: \"71b35848-69b6-4456-a0d1-8fee11ae3938\") " pod="openstack-kuttl-tests/swift-ring-rebalance-lc79q" Dec 03 11:43:30 crc kubenswrapper[5052]: I1203 11:43:30.894403 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/71b35848-69b6-4456-a0d1-8fee11ae3938-combined-ca-bundle\") pod \"swift-ring-rebalance-lc79q\" (UID: \"71b35848-69b6-4456-a0d1-8fee11ae3938\") " pod="openstack-kuttl-tests/swift-ring-rebalance-lc79q" Dec 03 11:43:30 crc kubenswrapper[5052]: I1203 11:43:30.906015 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-2cvbm\" (UniqueName: \"kubernetes.io/projected/71b35848-69b6-4456-a0d1-8fee11ae3938-kube-api-access-2cvbm\") pod \"swift-ring-rebalance-lc79q\" (UID: \"71b35848-69b6-4456-a0d1-8fee11ae3938\") " pod="openstack-kuttl-tests/swift-ring-rebalance-lc79q" Dec 03 11:43:30 crc kubenswrapper[5052]: I1203 11:43:30.988996 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"swiftconf\" (UniqueName: \"kubernetes.io/secret/c3a670b4-73f2-4c90-a3af-ba5d660ec1f7-swiftconf\") pod \"swift-ring-rebalance-r8qtm\" (UID: \"c3a670b4-73f2-4c90-a3af-ba5d660ec1f7\") " pod="openstack-kuttl-tests/swift-ring-rebalance-r8qtm" Dec 03 11:43:30 crc kubenswrapper[5052]: I1203 11:43:30.989057 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/empty-dir/c3a670b4-73f2-4c90-a3af-ba5d660ec1f7-etc-swift\") pod \"swift-ring-rebalance-r8qtm\" (UID: \"c3a670b4-73f2-4c90-a3af-ba5d660ec1f7\") " pod="openstack-kuttl-tests/swift-ring-rebalance-r8qtm" Dec 03 11:43:30 crc kubenswrapper[5052]: I1203 11:43:30.989081 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/c3a670b4-73f2-4c90-a3af-ba5d660ec1f7-scripts\") pod \"swift-ring-rebalance-r8qtm\" (UID: \"c3a670b4-73f2-4c90-a3af-ba5d660ec1f7\") " pod="openstack-kuttl-tests/swift-ring-rebalance-r8qtm" Dec 03 11:43:30 crc kubenswrapper[5052]: I1203 11:43:30.989155 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-84ktv\" (UniqueName: \"kubernetes.io/projected/c3a670b4-73f2-4c90-a3af-ba5d660ec1f7-kube-api-access-84ktv\") pod \"swift-ring-rebalance-r8qtm\" (UID: \"c3a670b4-73f2-4c90-a3af-ba5d660ec1f7\") " pod="openstack-kuttl-tests/swift-ring-rebalance-r8qtm" Dec 03 11:43:30 crc kubenswrapper[5052]: I1203 11:43:30.989178 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"dispersionconf\" (UniqueName: \"kubernetes.io/secret/c3a670b4-73f2-4c90-a3af-ba5d660ec1f7-dispersionconf\") pod \"swift-ring-rebalance-r8qtm\" (UID: \"c3a670b4-73f2-4c90-a3af-ba5d660ec1f7\") " pod="openstack-kuttl-tests/swift-ring-rebalance-r8qtm" Dec 03 11:43:30 crc kubenswrapper[5052]: I1203 11:43:30.989202 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ring-data-devices\" (UniqueName: \"kubernetes.io/configmap/c3a670b4-73f2-4c90-a3af-ba5d660ec1f7-ring-data-devices\") pod \"swift-ring-rebalance-r8qtm\" (UID: \"c3a670b4-73f2-4c90-a3af-ba5d660ec1f7\") " pod="openstack-kuttl-tests/swift-ring-rebalance-r8qtm" Dec 03 11:43:30 crc kubenswrapper[5052]: I1203 11:43:30.989231 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c3a670b4-73f2-4c90-a3af-ba5d660ec1f7-combined-ca-bundle\") pod \"swift-ring-rebalance-r8qtm\" (UID: \"c3a670b4-73f2-4c90-a3af-ba5d660ec1f7\") " pod="openstack-kuttl-tests/swift-ring-rebalance-r8qtm" Dec 03 11:43:30 crc kubenswrapper[5052]: I1203 11:43:30.989499 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"etc-swift\" (UniqueName: \"kubernetes.io/empty-dir/c3a670b4-73f2-4c90-a3af-ba5d660ec1f7-etc-swift\") pod \"swift-ring-rebalance-r8qtm\" (UID: \"c3a670b4-73f2-4c90-a3af-ba5d660ec1f7\") " pod="openstack-kuttl-tests/swift-ring-rebalance-r8qtm" Dec 03 11:43:30 crc kubenswrapper[5052]: I1203 11:43:30.990022 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/c3a670b4-73f2-4c90-a3af-ba5d660ec1f7-scripts\") pod \"swift-ring-rebalance-r8qtm\" (UID: \"c3a670b4-73f2-4c90-a3af-ba5d660ec1f7\") " pod="openstack-kuttl-tests/swift-ring-rebalance-r8qtm" Dec 03 11:43:30 crc kubenswrapper[5052]: I1203 11:43:30.990104 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ring-data-devices\" (UniqueName: \"kubernetes.io/configmap/c3a670b4-73f2-4c90-a3af-ba5d660ec1f7-ring-data-devices\") pod \"swift-ring-rebalance-r8qtm\" (UID: \"c3a670b4-73f2-4c90-a3af-ba5d660ec1f7\") " pod="openstack-kuttl-tests/swift-ring-rebalance-r8qtm" Dec 03 11:43:30 crc kubenswrapper[5052]: I1203 11:43:30.992094 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"swiftconf\" (UniqueName: \"kubernetes.io/secret/c3a670b4-73f2-4c90-a3af-ba5d660ec1f7-swiftconf\") pod \"swift-ring-rebalance-r8qtm\" (UID: \"c3a670b4-73f2-4c90-a3af-ba5d660ec1f7\") " pod="openstack-kuttl-tests/swift-ring-rebalance-r8qtm" Dec 03 11:43:30 crc kubenswrapper[5052]: I1203 11:43:30.992102 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"dispersionconf\" (UniqueName: \"kubernetes.io/secret/c3a670b4-73f2-4c90-a3af-ba5d660ec1f7-dispersionconf\") pod \"swift-ring-rebalance-r8qtm\" (UID: \"c3a670b4-73f2-4c90-a3af-ba5d660ec1f7\") " pod="openstack-kuttl-tests/swift-ring-rebalance-r8qtm" Dec 03 11:43:30 crc kubenswrapper[5052]: I1203 11:43:30.992634 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c3a670b4-73f2-4c90-a3af-ba5d660ec1f7-combined-ca-bundle\") pod \"swift-ring-rebalance-r8qtm\" (UID: \"c3a670b4-73f2-4c90-a3af-ba5d660ec1f7\") " pod="openstack-kuttl-tests/swift-ring-rebalance-r8qtm" Dec 03 11:43:31 crc kubenswrapper[5052]: I1203 11:43:31.005339 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-84ktv\" (UniqueName: \"kubernetes.io/projected/c3a670b4-73f2-4c90-a3af-ba5d660ec1f7-kube-api-access-84ktv\") pod \"swift-ring-rebalance-r8qtm\" (UID: \"c3a670b4-73f2-4c90-a3af-ba5d660ec1f7\") " pod="openstack-kuttl-tests/swift-ring-rebalance-r8qtm" Dec 03 11:43:31 crc kubenswrapper[5052]: I1203 11:43:31.105289 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-ring-rebalance-r8qtm" Dec 03 11:43:31 crc kubenswrapper[5052]: I1203 11:43:31.119805 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 11:43:31 crc kubenswrapper[5052]: I1203 11:43:31.267839 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-db-create-8tl8n" event={"ID":"a0a527da-143e-4d58-a118-0a1afea55b1f","Type":"ContainerStarted","Data":"b61c583a8237d6942c8885a37c89d5361107f0bd48e80748feca1601cb46e570"} Dec 03 11:43:31 crc kubenswrapper[5052]: I1203 11:43:31.269688 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-3554-account-create-update-n48gm" event={"ID":"bd7f2307-3030-4f83-b90c-debe14f2da96","Type":"ContainerStarted","Data":"5bb72fbb2016062939d5ea85849b5d6c4e299c96778da77c8f957ba54d086e4c"} Dec 03 11:43:31 crc kubenswrapper[5052]: I1203 11:43:31.269736 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-3554-account-create-update-n48gm" event={"ID":"bd7f2307-3030-4f83-b90c-debe14f2da96","Type":"ContainerStarted","Data":"83469bb3fd7fb7d0f3dcc7d065983f7cbbb203f53ff781f09b93e0d3c05c9761"} Dec 03 11:43:31 crc kubenswrapper[5052]: I1203 11:43:31.272434 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-db-create-l4qfz" event={"ID":"d4682522-2ae7-4e71-95ff-c4453f145537","Type":"ContainerStarted","Data":"1b505b87190dc754cde56718805d0189a89e23308cc1e51b861d4ddf4b6bfc89"} Dec 03 11:43:31 crc kubenswrapper[5052]: I1203 11:43:31.272480 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-db-create-l4qfz" event={"ID":"d4682522-2ae7-4e71-95ff-c4453f145537","Type":"ContainerStarted","Data":"69be8b14b679ad394c8838dfabac4771b8c84511de760083fed5d3495f219b03"} Dec 03 11:43:31 crc kubenswrapper[5052]: I1203 11:43:31.274342 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-9a4f-account-create-update-gs4xv" event={"ID":"75e340f6-2225-485f-8c78-53d05538a9c6","Type":"ContainerStarted","Data":"553635edc0dee6e3978dee3481740c8629f4746318a9834f801f334db1a16c3d"} Dec 03 11:43:31 crc kubenswrapper[5052]: I1203 11:43:31.274920 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-ring-rebalance-lc79q" Dec 03 11:43:31 crc kubenswrapper[5052]: I1203 11:43:31.294037 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/placement-db-create-8tl8n" podStartSLOduration=3.294012543 podStartE2EDuration="3.294012543s" podCreationTimestamp="2025-12-03 11:43:28 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:43:31.289102441 +0000 UTC m=+1401.877344205" watchObservedRunningTime="2025-12-03 11:43:31.294012543 +0000 UTC m=+1401.882254317" Dec 03 11:43:31 crc kubenswrapper[5052]: I1203 11:43:31.296740 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/93ca3376-6916-4fc9-8b76-7cd9a930865e-etc-swift\") pod \"swift-storage-0\" (UID: \"93ca3376-6916-4fc9-8b76-7cd9a930865e\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 11:43:31 crc kubenswrapper[5052]: E1203 11:43:31.298166 5052 projected.go:288] Couldn't get configMap openstack-kuttl-tests/swift-ring-files: configmap "swift-ring-files" not found Dec 03 11:43:31 crc kubenswrapper[5052]: E1203 11:43:31.298210 5052 projected.go:194] Error preparing data for projected volume etc-swift for pod openstack-kuttl-tests/swift-storage-0: configmap "swift-ring-files" not found Dec 03 11:43:31 crc kubenswrapper[5052]: E1203 11:43:31.298277 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/93ca3376-6916-4fc9-8b76-7cd9a930865e-etc-swift podName:93ca3376-6916-4fc9-8b76-7cd9a930865e nodeName:}" failed. No retries permitted until 2025-12-03 11:43:32.298254105 +0000 UTC m=+1402.886495949 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "etc-swift" (UniqueName: "kubernetes.io/projected/93ca3376-6916-4fc9-8b76-7cd9a930865e-etc-swift") pod "swift-storage-0" (UID: "93ca3376-6916-4fc9-8b76-7cd9a930865e") : configmap "swift-ring-files" not found Dec 03 11:43:31 crc kubenswrapper[5052]: I1203 11:43:31.305697 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-ring-rebalance-lc79q" Dec 03 11:43:31 crc kubenswrapper[5052]: I1203 11:43:31.321895 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/glance-db-create-l4qfz" podStartSLOduration=2.321874966 podStartE2EDuration="2.321874966s" podCreationTimestamp="2025-12-03 11:43:29 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:43:31.317760618 +0000 UTC m=+1401.906002392" watchObservedRunningTime="2025-12-03 11:43:31.321874966 +0000 UTC m=+1401.910116730" Dec 03 11:43:31 crc kubenswrapper[5052]: I1203 11:43:31.344684 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ovn-northd-0"] Dec 03 11:43:31 crc kubenswrapper[5052]: I1203 11:43:31.346088 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 11:43:31 crc kubenswrapper[5052]: I1203 11:43:31.351101 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/glance-3554-account-create-update-n48gm" podStartSLOduration=2.351079779 podStartE2EDuration="2.351079779s" podCreationTimestamp="2025-12-03 11:43:29 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:43:31.333412189 +0000 UTC m=+1401.921653953" watchObservedRunningTime="2025-12-03 11:43:31.351079779 +0000 UTC m=+1401.939321563" Dec 03 11:43:31 crc kubenswrapper[5052]: I1203 11:43:31.351657 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ovnnorthd-ovnnorthd-dockercfg-psffh" Dec 03 11:43:31 crc kubenswrapper[5052]: I1203 11:43:31.351931 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"ovnnorthd-scripts" Dec 03 11:43:31 crc kubenswrapper[5052]: I1203 11:43:31.351953 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"ovnnorthd-config" Dec 03 11:43:31 crc kubenswrapper[5052]: I1203 11:43:31.353660 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-ovnnorthd-ovndbs" Dec 03 11:43:31 crc kubenswrapper[5052]: I1203 11:43:31.362479 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ovn-northd-0"] Dec 03 11:43:31 crc kubenswrapper[5052]: I1203 11:43:31.367635 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/placement-9a4f-account-create-update-gs4xv" podStartSLOduration=2.367618075 podStartE2EDuration="2.367618075s" podCreationTimestamp="2025-12-03 11:43:29 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:43:31.356412662 +0000 UTC m=+1401.944654426" watchObservedRunningTime="2025-12-03 11:43:31.367618075 +0000 UTC m=+1401.955859839" Dec 03 11:43:31 crc kubenswrapper[5052]: I1203 11:43:31.500887 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"dispersionconf\" (UniqueName: \"kubernetes.io/secret/71b35848-69b6-4456-a0d1-8fee11ae3938-dispersionconf\") pod \"71b35848-69b6-4456-a0d1-8fee11ae3938\" (UID: \"71b35848-69b6-4456-a0d1-8fee11ae3938\") " Dec 03 11:43:31 crc kubenswrapper[5052]: I1203 11:43:31.500966 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2cvbm\" (UniqueName: \"kubernetes.io/projected/71b35848-69b6-4456-a0d1-8fee11ae3938-kube-api-access-2cvbm\") pod \"71b35848-69b6-4456-a0d1-8fee11ae3938\" (UID: \"71b35848-69b6-4456-a0d1-8fee11ae3938\") " Dec 03 11:43:31 crc kubenswrapper[5052]: I1203 11:43:31.501038 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/empty-dir/71b35848-69b6-4456-a0d1-8fee11ae3938-etc-swift\") pod \"71b35848-69b6-4456-a0d1-8fee11ae3938\" (UID: \"71b35848-69b6-4456-a0d1-8fee11ae3938\") " Dec 03 11:43:31 crc kubenswrapper[5052]: I1203 11:43:31.501069 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ring-data-devices\" (UniqueName: \"kubernetes.io/configmap/71b35848-69b6-4456-a0d1-8fee11ae3938-ring-data-devices\") pod \"71b35848-69b6-4456-a0d1-8fee11ae3938\" (UID: \"71b35848-69b6-4456-a0d1-8fee11ae3938\") " Dec 03 11:43:31 crc kubenswrapper[5052]: I1203 11:43:31.501116 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/71b35848-69b6-4456-a0d1-8fee11ae3938-combined-ca-bundle\") pod \"71b35848-69b6-4456-a0d1-8fee11ae3938\" (UID: \"71b35848-69b6-4456-a0d1-8fee11ae3938\") " Dec 03 11:43:31 crc kubenswrapper[5052]: I1203 11:43:31.501138 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"swiftconf\" (UniqueName: \"kubernetes.io/secret/71b35848-69b6-4456-a0d1-8fee11ae3938-swiftconf\") pod \"71b35848-69b6-4456-a0d1-8fee11ae3938\" (UID: \"71b35848-69b6-4456-a0d1-8fee11ae3938\") " Dec 03 11:43:31 crc kubenswrapper[5052]: I1203 11:43:31.501168 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/71b35848-69b6-4456-a0d1-8fee11ae3938-scripts\") pod \"71b35848-69b6-4456-a0d1-8fee11ae3938\" (UID: \"71b35848-69b6-4456-a0d1-8fee11ae3938\") " Dec 03 11:43:31 crc kubenswrapper[5052]: I1203 11:43:31.501420 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/83a10afe-30d8-443d-a4f7-28784a2ea068-scripts\") pod \"ovn-northd-0\" (UID: \"83a10afe-30d8-443d-a4f7-28784a2ea068\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 11:43:31 crc kubenswrapper[5052]: I1203 11:43:31.501451 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/83a10afe-30d8-443d-a4f7-28784a2ea068-combined-ca-bundle\") pod \"ovn-northd-0\" (UID: \"83a10afe-30d8-443d-a4f7-28784a2ea068\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 11:43:31 crc kubenswrapper[5052]: I1203 11:43:31.501479 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-northd-tls-certs\" (UniqueName: \"kubernetes.io/secret/83a10afe-30d8-443d-a4f7-28784a2ea068-ovn-northd-tls-certs\") pod \"ovn-northd-0\" (UID: \"83a10afe-30d8-443d-a4f7-28784a2ea068\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 11:43:31 crc kubenswrapper[5052]: I1203 11:43:31.501511 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/83a10afe-30d8-443d-a4f7-28784a2ea068-config\") pod \"ovn-northd-0\" (UID: \"83a10afe-30d8-443d-a4f7-28784a2ea068\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 11:43:31 crc kubenswrapper[5052]: I1203 11:43:31.501630 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/83a10afe-30d8-443d-a4f7-28784a2ea068-metrics-certs-tls-certs\") pod \"ovn-northd-0\" (UID: \"83a10afe-30d8-443d-a4f7-28784a2ea068\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 11:43:31 crc kubenswrapper[5052]: I1203 11:43:31.501670 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-rundir\" (UniqueName: \"kubernetes.io/empty-dir/83a10afe-30d8-443d-a4f7-28784a2ea068-ovn-rundir\") pod \"ovn-northd-0\" (UID: \"83a10afe-30d8-443d-a4f7-28784a2ea068\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 11:43:31 crc kubenswrapper[5052]: I1203 11:43:31.501701 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xh5s8\" (UniqueName: \"kubernetes.io/projected/83a10afe-30d8-443d-a4f7-28784a2ea068-kube-api-access-xh5s8\") pod \"ovn-northd-0\" (UID: \"83a10afe-30d8-443d-a4f7-28784a2ea068\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 11:43:31 crc kubenswrapper[5052]: I1203 11:43:31.502076 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71b35848-69b6-4456-a0d1-8fee11ae3938-etc-swift" (OuterVolumeSpecName: "etc-swift") pod "71b35848-69b6-4456-a0d1-8fee11ae3938" (UID: "71b35848-69b6-4456-a0d1-8fee11ae3938"). InnerVolumeSpecName "etc-swift". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:43:31 crc kubenswrapper[5052]: I1203 11:43:31.502220 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/71b35848-69b6-4456-a0d1-8fee11ae3938-ring-data-devices" (OuterVolumeSpecName: "ring-data-devices") pod "71b35848-69b6-4456-a0d1-8fee11ae3938" (UID: "71b35848-69b6-4456-a0d1-8fee11ae3938"). InnerVolumeSpecName "ring-data-devices". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:43:31 crc kubenswrapper[5052]: I1203 11:43:31.502327 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/71b35848-69b6-4456-a0d1-8fee11ae3938-scripts" (OuterVolumeSpecName: "scripts") pod "71b35848-69b6-4456-a0d1-8fee11ae3938" (UID: "71b35848-69b6-4456-a0d1-8fee11ae3938"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:43:31 crc kubenswrapper[5052]: I1203 11:43:31.502791 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/71b35848-69b6-4456-a0d1-8fee11ae3938-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:43:31 crc kubenswrapper[5052]: I1203 11:43:31.502811 5052 reconciler_common.go:293] "Volume detached for volume \"etc-swift\" (UniqueName: \"kubernetes.io/empty-dir/71b35848-69b6-4456-a0d1-8fee11ae3938-etc-swift\") on node \"crc\" DevicePath \"\"" Dec 03 11:43:31 crc kubenswrapper[5052]: I1203 11:43:31.502821 5052 reconciler_common.go:293] "Volume detached for volume \"ring-data-devices\" (UniqueName: \"kubernetes.io/configmap/71b35848-69b6-4456-a0d1-8fee11ae3938-ring-data-devices\") on node \"crc\" DevicePath \"\"" Dec 03 11:43:31 crc kubenswrapper[5052]: I1203 11:43:31.507402 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/71b35848-69b6-4456-a0d1-8fee11ae3938-dispersionconf" (OuterVolumeSpecName: "dispersionconf") pod "71b35848-69b6-4456-a0d1-8fee11ae3938" (UID: "71b35848-69b6-4456-a0d1-8fee11ae3938"). InnerVolumeSpecName "dispersionconf". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:43:31 crc kubenswrapper[5052]: I1203 11:43:31.508595 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/71b35848-69b6-4456-a0d1-8fee11ae3938-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "71b35848-69b6-4456-a0d1-8fee11ae3938" (UID: "71b35848-69b6-4456-a0d1-8fee11ae3938"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:43:31 crc kubenswrapper[5052]: I1203 11:43:31.510711 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/71b35848-69b6-4456-a0d1-8fee11ae3938-kube-api-access-2cvbm" (OuterVolumeSpecName: "kube-api-access-2cvbm") pod "71b35848-69b6-4456-a0d1-8fee11ae3938" (UID: "71b35848-69b6-4456-a0d1-8fee11ae3938"). InnerVolumeSpecName "kube-api-access-2cvbm". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:43:31 crc kubenswrapper[5052]: I1203 11:43:31.511053 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/71b35848-69b6-4456-a0d1-8fee11ae3938-swiftconf" (OuterVolumeSpecName: "swiftconf") pod "71b35848-69b6-4456-a0d1-8fee11ae3938" (UID: "71b35848-69b6-4456-a0d1-8fee11ae3938"). InnerVolumeSpecName "swiftconf". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:43:31 crc kubenswrapper[5052]: I1203 11:43:31.604285 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-xh5s8\" (UniqueName: \"kubernetes.io/projected/83a10afe-30d8-443d-a4f7-28784a2ea068-kube-api-access-xh5s8\") pod \"ovn-northd-0\" (UID: \"83a10afe-30d8-443d-a4f7-28784a2ea068\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 11:43:31 crc kubenswrapper[5052]: I1203 11:43:31.604721 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/83a10afe-30d8-443d-a4f7-28784a2ea068-scripts\") pod \"ovn-northd-0\" (UID: \"83a10afe-30d8-443d-a4f7-28784a2ea068\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 11:43:31 crc kubenswrapper[5052]: I1203 11:43:31.604743 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/83a10afe-30d8-443d-a4f7-28784a2ea068-combined-ca-bundle\") pod \"ovn-northd-0\" (UID: \"83a10afe-30d8-443d-a4f7-28784a2ea068\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 11:43:31 crc kubenswrapper[5052]: I1203 11:43:31.604771 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovn-northd-tls-certs\" (UniqueName: \"kubernetes.io/secret/83a10afe-30d8-443d-a4f7-28784a2ea068-ovn-northd-tls-certs\") pod \"ovn-northd-0\" (UID: \"83a10afe-30d8-443d-a4f7-28784a2ea068\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 11:43:31 crc kubenswrapper[5052]: I1203 11:43:31.604805 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/83a10afe-30d8-443d-a4f7-28784a2ea068-config\") pod \"ovn-northd-0\" (UID: \"83a10afe-30d8-443d-a4f7-28784a2ea068\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 11:43:31 crc kubenswrapper[5052]: I1203 11:43:31.604823 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/83a10afe-30d8-443d-a4f7-28784a2ea068-metrics-certs-tls-certs\") pod \"ovn-northd-0\" (UID: \"83a10afe-30d8-443d-a4f7-28784a2ea068\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 11:43:31 crc kubenswrapper[5052]: I1203 11:43:31.604859 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovn-rundir\" (UniqueName: \"kubernetes.io/empty-dir/83a10afe-30d8-443d-a4f7-28784a2ea068-ovn-rundir\") pod \"ovn-northd-0\" (UID: \"83a10afe-30d8-443d-a4f7-28784a2ea068\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 11:43:31 crc kubenswrapper[5052]: I1203 11:43:31.604903 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/71b35848-69b6-4456-a0d1-8fee11ae3938-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:43:31 crc kubenswrapper[5052]: I1203 11:43:31.604914 5052 reconciler_common.go:293] "Volume detached for volume \"swiftconf\" (UniqueName: \"kubernetes.io/secret/71b35848-69b6-4456-a0d1-8fee11ae3938-swiftconf\") on node \"crc\" DevicePath \"\"" Dec 03 11:43:31 crc kubenswrapper[5052]: I1203 11:43:31.604924 5052 reconciler_common.go:293] "Volume detached for volume \"dispersionconf\" (UniqueName: \"kubernetes.io/secret/71b35848-69b6-4456-a0d1-8fee11ae3938-dispersionconf\") on node \"crc\" DevicePath \"\"" Dec 03 11:43:31 crc kubenswrapper[5052]: I1203 11:43:31.604932 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-2cvbm\" (UniqueName: \"kubernetes.io/projected/71b35848-69b6-4456-a0d1-8fee11ae3938-kube-api-access-2cvbm\") on node \"crc\" DevicePath \"\"" Dec 03 11:43:31 crc kubenswrapper[5052]: I1203 11:43:31.605336 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovn-rundir\" (UniqueName: \"kubernetes.io/empty-dir/83a10afe-30d8-443d-a4f7-28784a2ea068-ovn-rundir\") pod \"ovn-northd-0\" (UID: \"83a10afe-30d8-443d-a4f7-28784a2ea068\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 11:43:31 crc kubenswrapper[5052]: I1203 11:43:31.608918 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/83a10afe-30d8-443d-a4f7-28784a2ea068-combined-ca-bundle\") pod \"ovn-northd-0\" (UID: \"83a10afe-30d8-443d-a4f7-28784a2ea068\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 11:43:31 crc kubenswrapper[5052]: I1203 11:43:31.609606 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovn-northd-tls-certs\" (UniqueName: \"kubernetes.io/secret/83a10afe-30d8-443d-a4f7-28784a2ea068-ovn-northd-tls-certs\") pod \"ovn-northd-0\" (UID: \"83a10afe-30d8-443d-a4f7-28784a2ea068\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 11:43:31 crc kubenswrapper[5052]: I1203 11:43:31.609842 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/83a10afe-30d8-443d-a4f7-28784a2ea068-scripts\") pod \"ovn-northd-0\" (UID: \"83a10afe-30d8-443d-a4f7-28784a2ea068\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 11:43:31 crc kubenswrapper[5052]: I1203 11:43:31.610171 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/83a10afe-30d8-443d-a4f7-28784a2ea068-config\") pod \"ovn-northd-0\" (UID: \"83a10afe-30d8-443d-a4f7-28784a2ea068\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 11:43:31 crc kubenswrapper[5052]: I1203 11:43:31.618330 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/83a10afe-30d8-443d-a4f7-28784a2ea068-metrics-certs-tls-certs\") pod \"ovn-northd-0\" (UID: \"83a10afe-30d8-443d-a4f7-28784a2ea068\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 11:43:31 crc kubenswrapper[5052]: I1203 11:43:31.634233 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-xh5s8\" (UniqueName: \"kubernetes.io/projected/83a10afe-30d8-443d-a4f7-28784a2ea068-kube-api-access-xh5s8\") pod \"ovn-northd-0\" (UID: \"83a10afe-30d8-443d-a4f7-28784a2ea068\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 11:43:31 crc kubenswrapper[5052]: I1203 11:43:31.669533 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 11:43:31 crc kubenswrapper[5052]: I1203 11:43:31.683479 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/swift-ring-rebalance-r8qtm"] Dec 03 11:43:32 crc kubenswrapper[5052]: I1203 11:43:32.092133 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-db-create-h55rj" Dec 03 11:43:32 crc kubenswrapper[5052]: I1203 11:43:32.103224 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-c226-account-create-update-4scft" Dec 03 11:43:32 crc kubenswrapper[5052]: I1203 11:43:32.116126 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zs78g\" (UniqueName: \"kubernetes.io/projected/679ac416-0e30-4016-bb4a-59944d03a695-kube-api-access-zs78g\") pod \"679ac416-0e30-4016-bb4a-59944d03a695\" (UID: \"679ac416-0e30-4016-bb4a-59944d03a695\") " Dec 03 11:43:32 crc kubenswrapper[5052]: I1203 11:43:32.116165 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/679ac416-0e30-4016-bb4a-59944d03a695-operator-scripts\") pod \"679ac416-0e30-4016-bb4a-59944d03a695\" (UID: \"679ac416-0e30-4016-bb4a-59944d03a695\") " Dec 03 11:43:32 crc kubenswrapper[5052]: I1203 11:43:32.116209 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mvqvq\" (UniqueName: \"kubernetes.io/projected/516527bf-2b7b-4bd1-b1a1-8d2ea94fbf16-kube-api-access-mvqvq\") pod \"516527bf-2b7b-4bd1-b1a1-8d2ea94fbf16\" (UID: \"516527bf-2b7b-4bd1-b1a1-8d2ea94fbf16\") " Dec 03 11:43:32 crc kubenswrapper[5052]: I1203 11:43:32.116260 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/516527bf-2b7b-4bd1-b1a1-8d2ea94fbf16-operator-scripts\") pod \"516527bf-2b7b-4bd1-b1a1-8d2ea94fbf16\" (UID: \"516527bf-2b7b-4bd1-b1a1-8d2ea94fbf16\") " Dec 03 11:43:32 crc kubenswrapper[5052]: I1203 11:43:32.119134 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/679ac416-0e30-4016-bb4a-59944d03a695-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "679ac416-0e30-4016-bb4a-59944d03a695" (UID: "679ac416-0e30-4016-bb4a-59944d03a695"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:43:32 crc kubenswrapper[5052]: I1203 11:43:32.119396 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/516527bf-2b7b-4bd1-b1a1-8d2ea94fbf16-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "516527bf-2b7b-4bd1-b1a1-8d2ea94fbf16" (UID: "516527bf-2b7b-4bd1-b1a1-8d2ea94fbf16"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:43:32 crc kubenswrapper[5052]: I1203 11:43:32.122760 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/516527bf-2b7b-4bd1-b1a1-8d2ea94fbf16-kube-api-access-mvqvq" (OuterVolumeSpecName: "kube-api-access-mvqvq") pod "516527bf-2b7b-4bd1-b1a1-8d2ea94fbf16" (UID: "516527bf-2b7b-4bd1-b1a1-8d2ea94fbf16"). InnerVolumeSpecName "kube-api-access-mvqvq". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:43:32 crc kubenswrapper[5052]: I1203 11:43:32.127024 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/679ac416-0e30-4016-bb4a-59944d03a695-kube-api-access-zs78g" (OuterVolumeSpecName: "kube-api-access-zs78g") pod "679ac416-0e30-4016-bb4a-59944d03a695" (UID: "679ac416-0e30-4016-bb4a-59944d03a695"). InnerVolumeSpecName "kube-api-access-zs78g". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:43:32 crc kubenswrapper[5052]: I1203 11:43:32.190648 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ovn-northd-0"] Dec 03 11:43:32 crc kubenswrapper[5052]: W1203 11:43:32.197023 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod83a10afe_30d8_443d_a4f7_28784a2ea068.slice/crio-4eb65027991ca77618d30f61f575326274e9222f67ae812b47b2bc67b410437c WatchSource:0}: Error finding container 4eb65027991ca77618d30f61f575326274e9222f67ae812b47b2bc67b410437c: Status 404 returned error can't find the container with id 4eb65027991ca77618d30f61f575326274e9222f67ae812b47b2bc67b410437c Dec 03 11:43:32 crc kubenswrapper[5052]: I1203 11:43:32.217478 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/516527bf-2b7b-4bd1-b1a1-8d2ea94fbf16-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:43:32 crc kubenswrapper[5052]: I1203 11:43:32.217503 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-zs78g\" (UniqueName: \"kubernetes.io/projected/679ac416-0e30-4016-bb4a-59944d03a695-kube-api-access-zs78g\") on node \"crc\" DevicePath \"\"" Dec 03 11:43:32 crc kubenswrapper[5052]: I1203 11:43:32.217526 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/679ac416-0e30-4016-bb4a-59944d03a695-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:43:32 crc kubenswrapper[5052]: I1203 11:43:32.217535 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-mvqvq\" (UniqueName: \"kubernetes.io/projected/516527bf-2b7b-4bd1-b1a1-8d2ea94fbf16-kube-api-access-mvqvq\") on node \"crc\" DevicePath \"\"" Dec 03 11:43:32 crc kubenswrapper[5052]: I1203 11:43:32.284737 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-ring-rebalance-r8qtm" event={"ID":"c3a670b4-73f2-4c90-a3af-ba5d660ec1f7","Type":"ContainerStarted","Data":"623e31d09d8b8851a9b72fa865622454543354624b6913a00d781d3a39db046c"} Dec 03 11:43:32 crc kubenswrapper[5052]: I1203 11:43:32.286669 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-c226-account-create-update-4scft" event={"ID":"516527bf-2b7b-4bd1-b1a1-8d2ea94fbf16","Type":"ContainerDied","Data":"f09ccaac6f5e7efb7a6ac48cb6e2a819a03b40517e21c6fe9f8ce6c74b50b6cc"} Dec 03 11:43:32 crc kubenswrapper[5052]: I1203 11:43:32.286695 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f09ccaac6f5e7efb7a6ac48cb6e2a819a03b40517e21c6fe9f8ce6c74b50b6cc" Dec 03 11:43:32 crc kubenswrapper[5052]: I1203 11:43:32.286741 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-c226-account-create-update-4scft" Dec 03 11:43:32 crc kubenswrapper[5052]: I1203 11:43:32.293205 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovn-northd-0" event={"ID":"83a10afe-30d8-443d-a4f7-28784a2ea068","Type":"ContainerStarted","Data":"4eb65027991ca77618d30f61f575326274e9222f67ae812b47b2bc67b410437c"} Dec 03 11:43:32 crc kubenswrapper[5052]: I1203 11:43:32.296446 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-db-create-h55rj" event={"ID":"679ac416-0e30-4016-bb4a-59944d03a695","Type":"ContainerDied","Data":"a1df51a6766eff161ec9234793fd2e460970a7899ca153691ae5e4b113a482be"} Dec 03 11:43:32 crc kubenswrapper[5052]: I1203 11:43:32.296456 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-ring-rebalance-lc79q" Dec 03 11:43:32 crc kubenswrapper[5052]: I1203 11:43:32.296476 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a1df51a6766eff161ec9234793fd2e460970a7899ca153691ae5e4b113a482be" Dec 03 11:43:32 crc kubenswrapper[5052]: I1203 11:43:32.296560 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-db-create-h55rj" Dec 03 11:43:32 crc kubenswrapper[5052]: I1203 11:43:32.351679 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/93ca3376-6916-4fc9-8b76-7cd9a930865e-etc-swift\") pod \"swift-storage-0\" (UID: \"93ca3376-6916-4fc9-8b76-7cd9a930865e\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 11:43:32 crc kubenswrapper[5052]: E1203 11:43:32.352340 5052 projected.go:288] Couldn't get configMap openstack-kuttl-tests/swift-ring-files: configmap "swift-ring-files" not found Dec 03 11:43:32 crc kubenswrapper[5052]: E1203 11:43:32.352369 5052 projected.go:194] Error preparing data for projected volume etc-swift for pod openstack-kuttl-tests/swift-storage-0: configmap "swift-ring-files" not found Dec 03 11:43:32 crc kubenswrapper[5052]: E1203 11:43:32.352428 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/93ca3376-6916-4fc9-8b76-7cd9a930865e-etc-swift podName:93ca3376-6916-4fc9-8b76-7cd9a930865e nodeName:}" failed. No retries permitted until 2025-12-03 11:43:34.352407074 +0000 UTC m=+1404.940648828 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "etc-swift" (UniqueName: "kubernetes.io/projected/93ca3376-6916-4fc9-8b76-7cd9a930865e-etc-swift") pod "swift-storage-0" (UID: "93ca3376-6916-4fc9-8b76-7cd9a930865e") : configmap "swift-ring-files" not found Dec 03 11:43:32 crc kubenswrapper[5052]: I1203 11:43:32.380205 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/swift-ring-rebalance-lc79q"] Dec 03 11:43:32 crc kubenswrapper[5052]: I1203 11:43:32.389320 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/swift-ring-rebalance-lc79q"] Dec 03 11:43:33 crc kubenswrapper[5052]: I1203 11:43:33.684130 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="71b35848-69b6-4456-a0d1-8fee11ae3938" path="/var/lib/kubelet/pods/71b35848-69b6-4456-a0d1-8fee11ae3938/volumes" Dec 03 11:43:34 crc kubenswrapper[5052]: I1203 11:43:34.316309 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-ring-rebalance-r8qtm" event={"ID":"c3a670b4-73f2-4c90-a3af-ba5d660ec1f7","Type":"ContainerStarted","Data":"3164682c6c7ed301d0d1fe1c0872340c3ccf49ae8dd5ae7e5c42e69a4c52010a"} Dec 03 11:43:34 crc kubenswrapper[5052]: I1203 11:43:34.338797 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/swift-ring-rebalance-r8qtm" podStartSLOduration=4.338773474 podStartE2EDuration="4.338773474s" podCreationTimestamp="2025-12-03 11:43:30 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:43:34.333041079 +0000 UTC m=+1404.921282863" watchObservedRunningTime="2025-12-03 11:43:34.338773474 +0000 UTC m=+1404.927015238" Dec 03 11:43:34 crc kubenswrapper[5052]: I1203 11:43:34.382166 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/93ca3376-6916-4fc9-8b76-7cd9a930865e-etc-swift\") pod \"swift-storage-0\" (UID: \"93ca3376-6916-4fc9-8b76-7cd9a930865e\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 11:43:34 crc kubenswrapper[5052]: E1203 11:43:34.382363 5052 projected.go:288] Couldn't get configMap openstack-kuttl-tests/swift-ring-files: configmap "swift-ring-files" not found Dec 03 11:43:34 crc kubenswrapper[5052]: E1203 11:43:34.382384 5052 projected.go:194] Error preparing data for projected volume etc-swift for pod openstack-kuttl-tests/swift-storage-0: configmap "swift-ring-files" not found Dec 03 11:43:34 crc kubenswrapper[5052]: E1203 11:43:34.382437 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/93ca3376-6916-4fc9-8b76-7cd9a930865e-etc-swift podName:93ca3376-6916-4fc9-8b76-7cd9a930865e nodeName:}" failed. No retries permitted until 2025-12-03 11:43:38.382421052 +0000 UTC m=+1408.970662816 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "etc-swift" (UniqueName: "kubernetes.io/projected/93ca3376-6916-4fc9-8b76-7cd9a930865e-etc-swift") pod "swift-storage-0" (UID: "93ca3376-6916-4fc9-8b76-7cd9a930865e") : configmap "swift-ring-files" not found Dec 03 11:43:35 crc kubenswrapper[5052]: I1203 11:43:35.325479 5052 generic.go:334] "Generic (PLEG): container finished" podID="a0a527da-143e-4d58-a118-0a1afea55b1f" containerID="b61c583a8237d6942c8885a37c89d5361107f0bd48e80748feca1601cb46e570" exitCode=0 Dec 03 11:43:35 crc kubenswrapper[5052]: I1203 11:43:35.325558 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-db-create-8tl8n" event={"ID":"a0a527da-143e-4d58-a118-0a1afea55b1f","Type":"ContainerDied","Data":"b61c583a8237d6942c8885a37c89d5361107f0bd48e80748feca1601cb46e570"} Dec 03 11:43:35 crc kubenswrapper[5052]: I1203 11:43:35.328551 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovn-northd-0" event={"ID":"83a10afe-30d8-443d-a4f7-28784a2ea068","Type":"ContainerStarted","Data":"af748c115d51d513b98829b5044d30038362360180988763c14bac6e6886f163"} Dec 03 11:43:35 crc kubenswrapper[5052]: I1203 11:43:35.328582 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovn-northd-0" event={"ID":"83a10afe-30d8-443d-a4f7-28784a2ea068","Type":"ContainerStarted","Data":"3b43b29211f1c83741ff8d9a266e5e11a2e9560cc3168a7ef1db16f7622c61ba"} Dec 03 11:43:35 crc kubenswrapper[5052]: I1203 11:43:35.328697 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 11:43:35 crc kubenswrapper[5052]: I1203 11:43:35.329882 5052 generic.go:334] "Generic (PLEG): container finished" podID="bd7f2307-3030-4f83-b90c-debe14f2da96" containerID="5bb72fbb2016062939d5ea85849b5d6c4e299c96778da77c8f957ba54d086e4c" exitCode=0 Dec 03 11:43:35 crc kubenswrapper[5052]: I1203 11:43:35.329954 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-3554-account-create-update-n48gm" event={"ID":"bd7f2307-3030-4f83-b90c-debe14f2da96","Type":"ContainerDied","Data":"5bb72fbb2016062939d5ea85849b5d6c4e299c96778da77c8f957ba54d086e4c"} Dec 03 11:43:35 crc kubenswrapper[5052]: I1203 11:43:35.331376 5052 generic.go:334] "Generic (PLEG): container finished" podID="d4682522-2ae7-4e71-95ff-c4453f145537" containerID="1b505b87190dc754cde56718805d0189a89e23308cc1e51b861d4ddf4b6bfc89" exitCode=0 Dec 03 11:43:35 crc kubenswrapper[5052]: I1203 11:43:35.331435 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-db-create-l4qfz" event={"ID":"d4682522-2ae7-4e71-95ff-c4453f145537","Type":"ContainerDied","Data":"1b505b87190dc754cde56718805d0189a89e23308cc1e51b861d4ddf4b6bfc89"} Dec 03 11:43:35 crc kubenswrapper[5052]: I1203 11:43:35.332874 5052 generic.go:334] "Generic (PLEG): container finished" podID="75e340f6-2225-485f-8c78-53d05538a9c6" containerID="553635edc0dee6e3978dee3481740c8629f4746318a9834f801f334db1a16c3d" exitCode=0 Dec 03 11:43:35 crc kubenswrapper[5052]: I1203 11:43:35.332942 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-9a4f-account-create-update-gs4xv" event={"ID":"75e340f6-2225-485f-8c78-53d05538a9c6","Type":"ContainerDied","Data":"553635edc0dee6e3978dee3481740c8629f4746318a9834f801f334db1a16c3d"} Dec 03 11:43:35 crc kubenswrapper[5052]: I1203 11:43:35.393937 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/ovn-northd-0" podStartSLOduration=4.393918891 podStartE2EDuration="4.393918891s" podCreationTimestamp="2025-12-03 11:43:31 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:43:35.387803235 +0000 UTC m=+1405.976045009" watchObservedRunningTime="2025-12-03 11:43:35.393918891 +0000 UTC m=+1405.982160655" Dec 03 11:43:36 crc kubenswrapper[5052]: I1203 11:43:36.764989 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-db-create-8tl8n" Dec 03 11:43:36 crc kubenswrapper[5052]: I1203 11:43:36.926347 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/a0a527da-143e-4d58-a118-0a1afea55b1f-operator-scripts\") pod \"a0a527da-143e-4d58-a118-0a1afea55b1f\" (UID: \"a0a527da-143e-4d58-a118-0a1afea55b1f\") " Dec 03 11:43:36 crc kubenswrapper[5052]: I1203 11:43:36.926710 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9m5fq\" (UniqueName: \"kubernetes.io/projected/a0a527da-143e-4d58-a118-0a1afea55b1f-kube-api-access-9m5fq\") pod \"a0a527da-143e-4d58-a118-0a1afea55b1f\" (UID: \"a0a527da-143e-4d58-a118-0a1afea55b1f\") " Dec 03 11:43:36 crc kubenswrapper[5052]: I1203 11:43:36.927237 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a0a527da-143e-4d58-a118-0a1afea55b1f-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "a0a527da-143e-4d58-a118-0a1afea55b1f" (UID: "a0a527da-143e-4d58-a118-0a1afea55b1f"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:43:36 crc kubenswrapper[5052]: I1203 11:43:36.927428 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/a0a527da-143e-4d58-a118-0a1afea55b1f-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:43:36 crc kubenswrapper[5052]: I1203 11:43:36.932725 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a0a527da-143e-4d58-a118-0a1afea55b1f-kube-api-access-9m5fq" (OuterVolumeSpecName: "kube-api-access-9m5fq") pod "a0a527da-143e-4d58-a118-0a1afea55b1f" (UID: "a0a527da-143e-4d58-a118-0a1afea55b1f"). InnerVolumeSpecName "kube-api-access-9m5fq". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:43:36 crc kubenswrapper[5052]: I1203 11:43:36.969966 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-3554-account-create-update-n48gm" Dec 03 11:43:36 crc kubenswrapper[5052]: I1203 11:43:36.984964 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-db-create-l4qfz" Dec 03 11:43:36 crc kubenswrapper[5052]: I1203 11:43:36.993804 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-9a4f-account-create-update-gs4xv" Dec 03 11:43:37 crc kubenswrapper[5052]: I1203 11:43:37.028990 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-9m5fq\" (UniqueName: \"kubernetes.io/projected/a0a527da-143e-4d58-a118-0a1afea55b1f-kube-api-access-9m5fq\") on node \"crc\" DevicePath \"\"" Dec 03 11:43:37 crc kubenswrapper[5052]: I1203 11:43:37.130314 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rnwmx\" (UniqueName: \"kubernetes.io/projected/75e340f6-2225-485f-8c78-53d05538a9c6-kube-api-access-rnwmx\") pod \"75e340f6-2225-485f-8c78-53d05538a9c6\" (UID: \"75e340f6-2225-485f-8c78-53d05538a9c6\") " Dec 03 11:43:37 crc kubenswrapper[5052]: I1203 11:43:37.130677 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/bd7f2307-3030-4f83-b90c-debe14f2da96-operator-scripts\") pod \"bd7f2307-3030-4f83-b90c-debe14f2da96\" (UID: \"bd7f2307-3030-4f83-b90c-debe14f2da96\") " Dec 03 11:43:37 crc kubenswrapper[5052]: I1203 11:43:37.130734 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v4g5x\" (UniqueName: \"kubernetes.io/projected/d4682522-2ae7-4e71-95ff-c4453f145537-kube-api-access-v4g5x\") pod \"d4682522-2ae7-4e71-95ff-c4453f145537\" (UID: \"d4682522-2ae7-4e71-95ff-c4453f145537\") " Dec 03 11:43:37 crc kubenswrapper[5052]: I1203 11:43:37.130785 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/75e340f6-2225-485f-8c78-53d05538a9c6-operator-scripts\") pod \"75e340f6-2225-485f-8c78-53d05538a9c6\" (UID: \"75e340f6-2225-485f-8c78-53d05538a9c6\") " Dec 03 11:43:37 crc kubenswrapper[5052]: I1203 11:43:37.130858 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2858h\" (UniqueName: \"kubernetes.io/projected/bd7f2307-3030-4f83-b90c-debe14f2da96-kube-api-access-2858h\") pod \"bd7f2307-3030-4f83-b90c-debe14f2da96\" (UID: \"bd7f2307-3030-4f83-b90c-debe14f2da96\") " Dec 03 11:43:37 crc kubenswrapper[5052]: I1203 11:43:37.130913 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/d4682522-2ae7-4e71-95ff-c4453f145537-operator-scripts\") pod \"d4682522-2ae7-4e71-95ff-c4453f145537\" (UID: \"d4682522-2ae7-4e71-95ff-c4453f145537\") " Dec 03 11:43:37 crc kubenswrapper[5052]: I1203 11:43:37.131305 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bd7f2307-3030-4f83-b90c-debe14f2da96-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "bd7f2307-3030-4f83-b90c-debe14f2da96" (UID: "bd7f2307-3030-4f83-b90c-debe14f2da96"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:43:37 crc kubenswrapper[5052]: I1203 11:43:37.131645 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/75e340f6-2225-485f-8c78-53d05538a9c6-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "75e340f6-2225-485f-8c78-53d05538a9c6" (UID: "75e340f6-2225-485f-8c78-53d05538a9c6"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:43:37 crc kubenswrapper[5052]: I1203 11:43:37.131675 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d4682522-2ae7-4e71-95ff-c4453f145537-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "d4682522-2ae7-4e71-95ff-c4453f145537" (UID: "d4682522-2ae7-4e71-95ff-c4453f145537"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:43:37 crc kubenswrapper[5052]: I1203 11:43:37.134717 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d4682522-2ae7-4e71-95ff-c4453f145537-kube-api-access-v4g5x" (OuterVolumeSpecName: "kube-api-access-v4g5x") pod "d4682522-2ae7-4e71-95ff-c4453f145537" (UID: "d4682522-2ae7-4e71-95ff-c4453f145537"). InnerVolumeSpecName "kube-api-access-v4g5x". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:43:37 crc kubenswrapper[5052]: I1203 11:43:37.135171 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bd7f2307-3030-4f83-b90c-debe14f2da96-kube-api-access-2858h" (OuterVolumeSpecName: "kube-api-access-2858h") pod "bd7f2307-3030-4f83-b90c-debe14f2da96" (UID: "bd7f2307-3030-4f83-b90c-debe14f2da96"). InnerVolumeSpecName "kube-api-access-2858h". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:43:37 crc kubenswrapper[5052]: I1203 11:43:37.135693 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/75e340f6-2225-485f-8c78-53d05538a9c6-kube-api-access-rnwmx" (OuterVolumeSpecName: "kube-api-access-rnwmx") pod "75e340f6-2225-485f-8c78-53d05538a9c6" (UID: "75e340f6-2225-485f-8c78-53d05538a9c6"). InnerVolumeSpecName "kube-api-access-rnwmx". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:43:37 crc kubenswrapper[5052]: I1203 11:43:37.233336 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-rnwmx\" (UniqueName: \"kubernetes.io/projected/75e340f6-2225-485f-8c78-53d05538a9c6-kube-api-access-rnwmx\") on node \"crc\" DevicePath \"\"" Dec 03 11:43:37 crc kubenswrapper[5052]: I1203 11:43:37.233380 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/bd7f2307-3030-4f83-b90c-debe14f2da96-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:43:37 crc kubenswrapper[5052]: I1203 11:43:37.233393 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-v4g5x\" (UniqueName: \"kubernetes.io/projected/d4682522-2ae7-4e71-95ff-c4453f145537-kube-api-access-v4g5x\") on node \"crc\" DevicePath \"\"" Dec 03 11:43:37 crc kubenswrapper[5052]: I1203 11:43:37.233408 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/75e340f6-2225-485f-8c78-53d05538a9c6-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:43:37 crc kubenswrapper[5052]: I1203 11:43:37.233420 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-2858h\" (UniqueName: \"kubernetes.io/projected/bd7f2307-3030-4f83-b90c-debe14f2da96-kube-api-access-2858h\") on node \"crc\" DevicePath \"\"" Dec 03 11:43:37 crc kubenswrapper[5052]: I1203 11:43:37.233432 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/d4682522-2ae7-4e71-95ff-c4453f145537-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:43:37 crc kubenswrapper[5052]: I1203 11:43:37.364143 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-db-create-8tl8n" event={"ID":"a0a527da-143e-4d58-a118-0a1afea55b1f","Type":"ContainerDied","Data":"69907479e7bbfe021243aeedcaf4a5b347779ce00ebbdb792d2b8dba78f212c0"} Dec 03 11:43:37 crc kubenswrapper[5052]: I1203 11:43:37.364201 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="69907479e7bbfe021243aeedcaf4a5b347779ce00ebbdb792d2b8dba78f212c0" Dec 03 11:43:37 crc kubenswrapper[5052]: I1203 11:43:37.364277 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-db-create-8tl8n" Dec 03 11:43:37 crc kubenswrapper[5052]: I1203 11:43:37.370203 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-3554-account-create-update-n48gm" Dec 03 11:43:37 crc kubenswrapper[5052]: I1203 11:43:37.370181 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-3554-account-create-update-n48gm" event={"ID":"bd7f2307-3030-4f83-b90c-debe14f2da96","Type":"ContainerDied","Data":"83469bb3fd7fb7d0f3dcc7d065983f7cbbb203f53ff781f09b93e0d3c05c9761"} Dec 03 11:43:37 crc kubenswrapper[5052]: I1203 11:43:37.370646 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="83469bb3fd7fb7d0f3dcc7d065983f7cbbb203f53ff781f09b93e0d3c05c9761" Dec 03 11:43:37 crc kubenswrapper[5052]: I1203 11:43:37.373075 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-db-create-l4qfz" event={"ID":"d4682522-2ae7-4e71-95ff-c4453f145537","Type":"ContainerDied","Data":"69be8b14b679ad394c8838dfabac4771b8c84511de760083fed5d3495f219b03"} Dec 03 11:43:37 crc kubenswrapper[5052]: I1203 11:43:37.373135 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="69be8b14b679ad394c8838dfabac4771b8c84511de760083fed5d3495f219b03" Dec 03 11:43:37 crc kubenswrapper[5052]: I1203 11:43:37.373222 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-db-create-l4qfz" Dec 03 11:43:37 crc kubenswrapper[5052]: I1203 11:43:37.396405 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-9a4f-account-create-update-gs4xv" event={"ID":"75e340f6-2225-485f-8c78-53d05538a9c6","Type":"ContainerDied","Data":"6098ce23f74ba061fce4b9c98f821511be0ecff3bbc0cf194364feba321081a4"} Dec 03 11:43:37 crc kubenswrapper[5052]: I1203 11:43:37.396458 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6098ce23f74ba061fce4b9c98f821511be0ecff3bbc0cf194364feba321081a4" Dec 03 11:43:37 crc kubenswrapper[5052]: I1203 11:43:37.396543 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-9a4f-account-create-update-gs4xv" Dec 03 11:43:38 crc kubenswrapper[5052]: I1203 11:43:38.455299 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/93ca3376-6916-4fc9-8b76-7cd9a930865e-etc-swift\") pod \"swift-storage-0\" (UID: \"93ca3376-6916-4fc9-8b76-7cd9a930865e\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 11:43:38 crc kubenswrapper[5052]: E1203 11:43:38.455665 5052 projected.go:288] Couldn't get configMap openstack-kuttl-tests/swift-ring-files: configmap "swift-ring-files" not found Dec 03 11:43:38 crc kubenswrapper[5052]: E1203 11:43:38.455705 5052 projected.go:194] Error preparing data for projected volume etc-swift for pod openstack-kuttl-tests/swift-storage-0: configmap "swift-ring-files" not found Dec 03 11:43:38 crc kubenswrapper[5052]: E1203 11:43:38.455783 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/93ca3376-6916-4fc9-8b76-7cd9a930865e-etc-swift podName:93ca3376-6916-4fc9-8b76-7cd9a930865e nodeName:}" failed. No retries permitted until 2025-12-03 11:43:46.455759444 +0000 UTC m=+1417.044001258 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "etc-swift" (UniqueName: "kubernetes.io/projected/93ca3376-6916-4fc9-8b76-7cd9a930865e-etc-swift") pod "swift-storage-0" (UID: "93ca3376-6916-4fc9-8b76-7cd9a930865e") : configmap "swift-ring-files" not found Dec 03 11:43:39 crc kubenswrapper[5052]: I1203 11:43:39.740525 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/glance-db-sync-g22fp"] Dec 03 11:43:39 crc kubenswrapper[5052]: E1203 11:43:39.740915 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="d4682522-2ae7-4e71-95ff-c4453f145537" containerName="mariadb-database-create" Dec 03 11:43:39 crc kubenswrapper[5052]: I1203 11:43:39.740929 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="d4682522-2ae7-4e71-95ff-c4453f145537" containerName="mariadb-database-create" Dec 03 11:43:39 crc kubenswrapper[5052]: E1203 11:43:39.740942 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="679ac416-0e30-4016-bb4a-59944d03a695" containerName="mariadb-database-create" Dec 03 11:43:39 crc kubenswrapper[5052]: I1203 11:43:39.740948 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="679ac416-0e30-4016-bb4a-59944d03a695" containerName="mariadb-database-create" Dec 03 11:43:39 crc kubenswrapper[5052]: E1203 11:43:39.740961 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="bd7f2307-3030-4f83-b90c-debe14f2da96" containerName="mariadb-account-create-update" Dec 03 11:43:39 crc kubenswrapper[5052]: I1203 11:43:39.740967 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd7f2307-3030-4f83-b90c-debe14f2da96" containerName="mariadb-account-create-update" Dec 03 11:43:39 crc kubenswrapper[5052]: E1203 11:43:39.740982 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="516527bf-2b7b-4bd1-b1a1-8d2ea94fbf16" containerName="mariadb-account-create-update" Dec 03 11:43:39 crc kubenswrapper[5052]: I1203 11:43:39.740987 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="516527bf-2b7b-4bd1-b1a1-8d2ea94fbf16" containerName="mariadb-account-create-update" Dec 03 11:43:39 crc kubenswrapper[5052]: E1203 11:43:39.741009 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="a0a527da-143e-4d58-a118-0a1afea55b1f" containerName="mariadb-database-create" Dec 03 11:43:39 crc kubenswrapper[5052]: I1203 11:43:39.741015 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="a0a527da-143e-4d58-a118-0a1afea55b1f" containerName="mariadb-database-create" Dec 03 11:43:39 crc kubenswrapper[5052]: E1203 11:43:39.741030 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="75e340f6-2225-485f-8c78-53d05538a9c6" containerName="mariadb-account-create-update" Dec 03 11:43:39 crc kubenswrapper[5052]: I1203 11:43:39.741036 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="75e340f6-2225-485f-8c78-53d05538a9c6" containerName="mariadb-account-create-update" Dec 03 11:43:39 crc kubenswrapper[5052]: I1203 11:43:39.741212 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="516527bf-2b7b-4bd1-b1a1-8d2ea94fbf16" containerName="mariadb-account-create-update" Dec 03 11:43:39 crc kubenswrapper[5052]: I1203 11:43:39.741224 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="75e340f6-2225-485f-8c78-53d05538a9c6" containerName="mariadb-account-create-update" Dec 03 11:43:39 crc kubenswrapper[5052]: I1203 11:43:39.741234 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="a0a527da-143e-4d58-a118-0a1afea55b1f" containerName="mariadb-database-create" Dec 03 11:43:39 crc kubenswrapper[5052]: I1203 11:43:39.741247 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="d4682522-2ae7-4e71-95ff-c4453f145537" containerName="mariadb-database-create" Dec 03 11:43:39 crc kubenswrapper[5052]: I1203 11:43:39.741258 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="bd7f2307-3030-4f83-b90c-debe14f2da96" containerName="mariadb-account-create-update" Dec 03 11:43:39 crc kubenswrapper[5052]: I1203 11:43:39.741264 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="679ac416-0e30-4016-bb4a-59944d03a695" containerName="mariadb-database-create" Dec 03 11:43:39 crc kubenswrapper[5052]: I1203 11:43:39.741784 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-db-sync-g22fp" Dec 03 11:43:39 crc kubenswrapper[5052]: I1203 11:43:39.743732 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-glance-dockercfg-tlrkx" Dec 03 11:43:39 crc kubenswrapper[5052]: I1203 11:43:39.743916 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-config-data" Dec 03 11:43:39 crc kubenswrapper[5052]: I1203 11:43:39.756651 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-db-sync-g22fp"] Dec 03 11:43:39 crc kubenswrapper[5052]: I1203 11:43:39.779894 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3c063992-bf41-4c35-a5ad-cdc4b512bcee-combined-ca-bundle\") pod \"glance-db-sync-g22fp\" (UID: \"3c063992-bf41-4c35-a5ad-cdc4b512bcee\") " pod="openstack-kuttl-tests/glance-db-sync-g22fp" Dec 03 11:43:39 crc kubenswrapper[5052]: I1203 11:43:39.779940 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/3c063992-bf41-4c35-a5ad-cdc4b512bcee-config-data\") pod \"glance-db-sync-g22fp\" (UID: \"3c063992-bf41-4c35-a5ad-cdc4b512bcee\") " pod="openstack-kuttl-tests/glance-db-sync-g22fp" Dec 03 11:43:39 crc kubenswrapper[5052]: I1203 11:43:39.779963 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/3c063992-bf41-4c35-a5ad-cdc4b512bcee-db-sync-config-data\") pod \"glance-db-sync-g22fp\" (UID: \"3c063992-bf41-4c35-a5ad-cdc4b512bcee\") " pod="openstack-kuttl-tests/glance-db-sync-g22fp" Dec 03 11:43:39 crc kubenswrapper[5052]: I1203 11:43:39.780032 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w8v6f\" (UniqueName: \"kubernetes.io/projected/3c063992-bf41-4c35-a5ad-cdc4b512bcee-kube-api-access-w8v6f\") pod \"glance-db-sync-g22fp\" (UID: \"3c063992-bf41-4c35-a5ad-cdc4b512bcee\") " pod="openstack-kuttl-tests/glance-db-sync-g22fp" Dec 03 11:43:39 crc kubenswrapper[5052]: I1203 11:43:39.881672 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3c063992-bf41-4c35-a5ad-cdc4b512bcee-combined-ca-bundle\") pod \"glance-db-sync-g22fp\" (UID: \"3c063992-bf41-4c35-a5ad-cdc4b512bcee\") " pod="openstack-kuttl-tests/glance-db-sync-g22fp" Dec 03 11:43:39 crc kubenswrapper[5052]: I1203 11:43:39.881748 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/3c063992-bf41-4c35-a5ad-cdc4b512bcee-config-data\") pod \"glance-db-sync-g22fp\" (UID: \"3c063992-bf41-4c35-a5ad-cdc4b512bcee\") " pod="openstack-kuttl-tests/glance-db-sync-g22fp" Dec 03 11:43:39 crc kubenswrapper[5052]: I1203 11:43:39.881782 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/3c063992-bf41-4c35-a5ad-cdc4b512bcee-db-sync-config-data\") pod \"glance-db-sync-g22fp\" (UID: \"3c063992-bf41-4c35-a5ad-cdc4b512bcee\") " pod="openstack-kuttl-tests/glance-db-sync-g22fp" Dec 03 11:43:39 crc kubenswrapper[5052]: I1203 11:43:39.881839 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-w8v6f\" (UniqueName: \"kubernetes.io/projected/3c063992-bf41-4c35-a5ad-cdc4b512bcee-kube-api-access-w8v6f\") pod \"glance-db-sync-g22fp\" (UID: \"3c063992-bf41-4c35-a5ad-cdc4b512bcee\") " pod="openstack-kuttl-tests/glance-db-sync-g22fp" Dec 03 11:43:39 crc kubenswrapper[5052]: I1203 11:43:39.887276 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/3c063992-bf41-4c35-a5ad-cdc4b512bcee-config-data\") pod \"glance-db-sync-g22fp\" (UID: \"3c063992-bf41-4c35-a5ad-cdc4b512bcee\") " pod="openstack-kuttl-tests/glance-db-sync-g22fp" Dec 03 11:43:39 crc kubenswrapper[5052]: I1203 11:43:39.887289 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3c063992-bf41-4c35-a5ad-cdc4b512bcee-combined-ca-bundle\") pod \"glance-db-sync-g22fp\" (UID: \"3c063992-bf41-4c35-a5ad-cdc4b512bcee\") " pod="openstack-kuttl-tests/glance-db-sync-g22fp" Dec 03 11:43:39 crc kubenswrapper[5052]: I1203 11:43:39.897293 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/3c063992-bf41-4c35-a5ad-cdc4b512bcee-db-sync-config-data\") pod \"glance-db-sync-g22fp\" (UID: \"3c063992-bf41-4c35-a5ad-cdc4b512bcee\") " pod="openstack-kuttl-tests/glance-db-sync-g22fp" Dec 03 11:43:39 crc kubenswrapper[5052]: I1203 11:43:39.899506 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-w8v6f\" (UniqueName: \"kubernetes.io/projected/3c063992-bf41-4c35-a5ad-cdc4b512bcee-kube-api-access-w8v6f\") pod \"glance-db-sync-g22fp\" (UID: \"3c063992-bf41-4c35-a5ad-cdc4b512bcee\") " pod="openstack-kuttl-tests/glance-db-sync-g22fp" Dec 03 11:43:40 crc kubenswrapper[5052]: I1203 11:43:40.064883 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-db-sync-g22fp" Dec 03 11:43:40 crc kubenswrapper[5052]: I1203 11:43:40.499474 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-db-sync-g22fp"] Dec 03 11:43:41 crc kubenswrapper[5052]: I1203 11:43:41.438537 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-db-sync-g22fp" event={"ID":"3c063992-bf41-4c35-a5ad-cdc4b512bcee","Type":"ContainerStarted","Data":"d611ec42cb4376e27185390a976f403ae2efb08d4970d4d342ad87bfe46b3941"} Dec 03 11:43:41 crc kubenswrapper[5052]: I1203 11:43:41.438866 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-db-sync-g22fp" event={"ID":"3c063992-bf41-4c35-a5ad-cdc4b512bcee","Type":"ContainerStarted","Data":"26f4f09583bbf7632b9bdaf7e7510fd09044cb4fe5fa34de71c59e2f4a65855d"} Dec 03 11:43:41 crc kubenswrapper[5052]: I1203 11:43:41.443161 5052 generic.go:334] "Generic (PLEG): container finished" podID="c3a670b4-73f2-4c90-a3af-ba5d660ec1f7" containerID="3164682c6c7ed301d0d1fe1c0872340c3ccf49ae8dd5ae7e5c42e69a4c52010a" exitCode=0 Dec 03 11:43:41 crc kubenswrapper[5052]: I1203 11:43:41.443189 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-ring-rebalance-r8qtm" event={"ID":"c3a670b4-73f2-4c90-a3af-ba5d660ec1f7","Type":"ContainerDied","Data":"3164682c6c7ed301d0d1fe1c0872340c3ccf49ae8dd5ae7e5c42e69a4c52010a"} Dec 03 11:43:41 crc kubenswrapper[5052]: I1203 11:43:41.489313 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/glance-db-sync-g22fp" podStartSLOduration=2.489292981 podStartE2EDuration="2.489292981s" podCreationTimestamp="2025-12-03 11:43:39 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:43:41.464863787 +0000 UTC m=+1412.053105551" watchObservedRunningTime="2025-12-03 11:43:41.489292981 +0000 UTC m=+1412.077534745" Dec 03 11:43:42 crc kubenswrapper[5052]: I1203 11:43:42.860903 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-ring-rebalance-r8qtm" Dec 03 11:43:43 crc kubenswrapper[5052]: I1203 11:43:43.044341 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-84ktv\" (UniqueName: \"kubernetes.io/projected/c3a670b4-73f2-4c90-a3af-ba5d660ec1f7-kube-api-access-84ktv\") pod \"c3a670b4-73f2-4c90-a3af-ba5d660ec1f7\" (UID: \"c3a670b4-73f2-4c90-a3af-ba5d660ec1f7\") " Dec 03 11:43:43 crc kubenswrapper[5052]: I1203 11:43:43.044419 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/empty-dir/c3a670b4-73f2-4c90-a3af-ba5d660ec1f7-etc-swift\") pod \"c3a670b4-73f2-4c90-a3af-ba5d660ec1f7\" (UID: \"c3a670b4-73f2-4c90-a3af-ba5d660ec1f7\") " Dec 03 11:43:43 crc kubenswrapper[5052]: I1203 11:43:43.044476 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c3a670b4-73f2-4c90-a3af-ba5d660ec1f7-combined-ca-bundle\") pod \"c3a670b4-73f2-4c90-a3af-ba5d660ec1f7\" (UID: \"c3a670b4-73f2-4c90-a3af-ba5d660ec1f7\") " Dec 03 11:43:43 crc kubenswrapper[5052]: I1203 11:43:43.044565 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ring-data-devices\" (UniqueName: \"kubernetes.io/configmap/c3a670b4-73f2-4c90-a3af-ba5d660ec1f7-ring-data-devices\") pod \"c3a670b4-73f2-4c90-a3af-ba5d660ec1f7\" (UID: \"c3a670b4-73f2-4c90-a3af-ba5d660ec1f7\") " Dec 03 11:43:43 crc kubenswrapper[5052]: I1203 11:43:43.044598 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"dispersionconf\" (UniqueName: \"kubernetes.io/secret/c3a670b4-73f2-4c90-a3af-ba5d660ec1f7-dispersionconf\") pod \"c3a670b4-73f2-4c90-a3af-ba5d660ec1f7\" (UID: \"c3a670b4-73f2-4c90-a3af-ba5d660ec1f7\") " Dec 03 11:43:43 crc kubenswrapper[5052]: I1203 11:43:43.044627 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/c3a670b4-73f2-4c90-a3af-ba5d660ec1f7-scripts\") pod \"c3a670b4-73f2-4c90-a3af-ba5d660ec1f7\" (UID: \"c3a670b4-73f2-4c90-a3af-ba5d660ec1f7\") " Dec 03 11:43:43 crc kubenswrapper[5052]: I1203 11:43:43.044664 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"swiftconf\" (UniqueName: \"kubernetes.io/secret/c3a670b4-73f2-4c90-a3af-ba5d660ec1f7-swiftconf\") pod \"c3a670b4-73f2-4c90-a3af-ba5d660ec1f7\" (UID: \"c3a670b4-73f2-4c90-a3af-ba5d660ec1f7\") " Dec 03 11:43:43 crc kubenswrapper[5052]: I1203 11:43:43.045736 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c3a670b4-73f2-4c90-a3af-ba5d660ec1f7-ring-data-devices" (OuterVolumeSpecName: "ring-data-devices") pod "c3a670b4-73f2-4c90-a3af-ba5d660ec1f7" (UID: "c3a670b4-73f2-4c90-a3af-ba5d660ec1f7"). InnerVolumeSpecName "ring-data-devices". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:43:43 crc kubenswrapper[5052]: I1203 11:43:43.047059 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c3a670b4-73f2-4c90-a3af-ba5d660ec1f7-etc-swift" (OuterVolumeSpecName: "etc-swift") pod "c3a670b4-73f2-4c90-a3af-ba5d660ec1f7" (UID: "c3a670b4-73f2-4c90-a3af-ba5d660ec1f7"). InnerVolumeSpecName "etc-swift". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:43:43 crc kubenswrapper[5052]: I1203 11:43:43.051750 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c3a670b4-73f2-4c90-a3af-ba5d660ec1f7-kube-api-access-84ktv" (OuterVolumeSpecName: "kube-api-access-84ktv") pod "c3a670b4-73f2-4c90-a3af-ba5d660ec1f7" (UID: "c3a670b4-73f2-4c90-a3af-ba5d660ec1f7"). InnerVolumeSpecName "kube-api-access-84ktv". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:43:43 crc kubenswrapper[5052]: I1203 11:43:43.055693 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c3a670b4-73f2-4c90-a3af-ba5d660ec1f7-dispersionconf" (OuterVolumeSpecName: "dispersionconf") pod "c3a670b4-73f2-4c90-a3af-ba5d660ec1f7" (UID: "c3a670b4-73f2-4c90-a3af-ba5d660ec1f7"). InnerVolumeSpecName "dispersionconf". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:43:43 crc kubenswrapper[5052]: I1203 11:43:43.068294 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c3a670b4-73f2-4c90-a3af-ba5d660ec1f7-swiftconf" (OuterVolumeSpecName: "swiftconf") pod "c3a670b4-73f2-4c90-a3af-ba5d660ec1f7" (UID: "c3a670b4-73f2-4c90-a3af-ba5d660ec1f7"). InnerVolumeSpecName "swiftconf". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:43:43 crc kubenswrapper[5052]: I1203 11:43:43.071621 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c3a670b4-73f2-4c90-a3af-ba5d660ec1f7-scripts" (OuterVolumeSpecName: "scripts") pod "c3a670b4-73f2-4c90-a3af-ba5d660ec1f7" (UID: "c3a670b4-73f2-4c90-a3af-ba5d660ec1f7"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:43:43 crc kubenswrapper[5052]: I1203 11:43:43.074271 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c3a670b4-73f2-4c90-a3af-ba5d660ec1f7-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "c3a670b4-73f2-4c90-a3af-ba5d660ec1f7" (UID: "c3a670b4-73f2-4c90-a3af-ba5d660ec1f7"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:43:43 crc kubenswrapper[5052]: I1203 11:43:43.146227 5052 reconciler_common.go:293] "Volume detached for volume \"etc-swift\" (UniqueName: \"kubernetes.io/empty-dir/c3a670b4-73f2-4c90-a3af-ba5d660ec1f7-etc-swift\") on node \"crc\" DevicePath \"\"" Dec 03 11:43:43 crc kubenswrapper[5052]: I1203 11:43:43.147217 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c3a670b4-73f2-4c90-a3af-ba5d660ec1f7-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:43:43 crc kubenswrapper[5052]: I1203 11:43:43.147257 5052 reconciler_common.go:293] "Volume detached for volume \"ring-data-devices\" (UniqueName: \"kubernetes.io/configmap/c3a670b4-73f2-4c90-a3af-ba5d660ec1f7-ring-data-devices\") on node \"crc\" DevicePath \"\"" Dec 03 11:43:43 crc kubenswrapper[5052]: I1203 11:43:43.147269 5052 reconciler_common.go:293] "Volume detached for volume \"dispersionconf\" (UniqueName: \"kubernetes.io/secret/c3a670b4-73f2-4c90-a3af-ba5d660ec1f7-dispersionconf\") on node \"crc\" DevicePath \"\"" Dec 03 11:43:43 crc kubenswrapper[5052]: I1203 11:43:43.147278 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/c3a670b4-73f2-4c90-a3af-ba5d660ec1f7-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:43:43 crc kubenswrapper[5052]: I1203 11:43:43.147288 5052 reconciler_common.go:293] "Volume detached for volume \"swiftconf\" (UniqueName: \"kubernetes.io/secret/c3a670b4-73f2-4c90-a3af-ba5d660ec1f7-swiftconf\") on node \"crc\" DevicePath \"\"" Dec 03 11:43:43 crc kubenswrapper[5052]: I1203 11:43:43.147298 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-84ktv\" (UniqueName: \"kubernetes.io/projected/c3a670b4-73f2-4c90-a3af-ba5d660ec1f7-kube-api-access-84ktv\") on node \"crc\" DevicePath \"\"" Dec 03 11:43:43 crc kubenswrapper[5052]: I1203 11:43:43.479063 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-ring-rebalance-r8qtm" event={"ID":"c3a670b4-73f2-4c90-a3af-ba5d660ec1f7","Type":"ContainerDied","Data":"623e31d09d8b8851a9b72fa865622454543354624b6913a00d781d3a39db046c"} Dec 03 11:43:43 crc kubenswrapper[5052]: I1203 11:43:43.479386 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="623e31d09d8b8851a9b72fa865622454543354624b6913a00d781d3a39db046c" Dec 03 11:43:43 crc kubenswrapper[5052]: I1203 11:43:43.479331 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-ring-rebalance-r8qtm" Dec 03 11:43:44 crc kubenswrapper[5052]: I1203 11:43:44.492544 5052 generic.go:334] "Generic (PLEG): container finished" podID="3c063992-bf41-4c35-a5ad-cdc4b512bcee" containerID="d611ec42cb4376e27185390a976f403ae2efb08d4970d4d342ad87bfe46b3941" exitCode=0 Dec 03 11:43:44 crc kubenswrapper[5052]: I1203 11:43:44.492633 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-db-sync-g22fp" event={"ID":"3c063992-bf41-4c35-a5ad-cdc4b512bcee","Type":"ContainerDied","Data":"d611ec42cb4376e27185390a976f403ae2efb08d4970d4d342ad87bfe46b3941"} Dec 03 11:43:45 crc kubenswrapper[5052]: I1203 11:43:45.871293 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-db-sync-g22fp" Dec 03 11:43:45 crc kubenswrapper[5052]: I1203 11:43:45.896559 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/3c063992-bf41-4c35-a5ad-cdc4b512bcee-db-sync-config-data\") pod \"3c063992-bf41-4c35-a5ad-cdc4b512bcee\" (UID: \"3c063992-bf41-4c35-a5ad-cdc4b512bcee\") " Dec 03 11:43:45 crc kubenswrapper[5052]: I1203 11:43:45.896707 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w8v6f\" (UniqueName: \"kubernetes.io/projected/3c063992-bf41-4c35-a5ad-cdc4b512bcee-kube-api-access-w8v6f\") pod \"3c063992-bf41-4c35-a5ad-cdc4b512bcee\" (UID: \"3c063992-bf41-4c35-a5ad-cdc4b512bcee\") " Dec 03 11:43:45 crc kubenswrapper[5052]: I1203 11:43:45.897750 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3c063992-bf41-4c35-a5ad-cdc4b512bcee-combined-ca-bundle\") pod \"3c063992-bf41-4c35-a5ad-cdc4b512bcee\" (UID: \"3c063992-bf41-4c35-a5ad-cdc4b512bcee\") " Dec 03 11:43:45 crc kubenswrapper[5052]: I1203 11:43:45.898035 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/3c063992-bf41-4c35-a5ad-cdc4b512bcee-config-data\") pod \"3c063992-bf41-4c35-a5ad-cdc4b512bcee\" (UID: \"3c063992-bf41-4c35-a5ad-cdc4b512bcee\") " Dec 03 11:43:45 crc kubenswrapper[5052]: I1203 11:43:45.903859 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3c063992-bf41-4c35-a5ad-cdc4b512bcee-db-sync-config-data" (OuterVolumeSpecName: "db-sync-config-data") pod "3c063992-bf41-4c35-a5ad-cdc4b512bcee" (UID: "3c063992-bf41-4c35-a5ad-cdc4b512bcee"). InnerVolumeSpecName "db-sync-config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:43:45 crc kubenswrapper[5052]: I1203 11:43:45.905377 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3c063992-bf41-4c35-a5ad-cdc4b512bcee-kube-api-access-w8v6f" (OuterVolumeSpecName: "kube-api-access-w8v6f") pod "3c063992-bf41-4c35-a5ad-cdc4b512bcee" (UID: "3c063992-bf41-4c35-a5ad-cdc4b512bcee"). InnerVolumeSpecName "kube-api-access-w8v6f". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:43:45 crc kubenswrapper[5052]: I1203 11:43:45.929082 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3c063992-bf41-4c35-a5ad-cdc4b512bcee-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "3c063992-bf41-4c35-a5ad-cdc4b512bcee" (UID: "3c063992-bf41-4c35-a5ad-cdc4b512bcee"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:43:45 crc kubenswrapper[5052]: I1203 11:43:45.946138 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3c063992-bf41-4c35-a5ad-cdc4b512bcee-config-data" (OuterVolumeSpecName: "config-data") pod "3c063992-bf41-4c35-a5ad-cdc4b512bcee" (UID: "3c063992-bf41-4c35-a5ad-cdc4b512bcee"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:43:46 crc kubenswrapper[5052]: I1203 11:43:46.001424 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-w8v6f\" (UniqueName: \"kubernetes.io/projected/3c063992-bf41-4c35-a5ad-cdc4b512bcee-kube-api-access-w8v6f\") on node \"crc\" DevicePath \"\"" Dec 03 11:43:46 crc kubenswrapper[5052]: I1203 11:43:46.001475 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3c063992-bf41-4c35-a5ad-cdc4b512bcee-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:43:46 crc kubenswrapper[5052]: I1203 11:43:46.001488 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/3c063992-bf41-4c35-a5ad-cdc4b512bcee-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:43:46 crc kubenswrapper[5052]: I1203 11:43:46.001500 5052 reconciler_common.go:293] "Volume detached for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/3c063992-bf41-4c35-a5ad-cdc4b512bcee-db-sync-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:43:46 crc kubenswrapper[5052]: I1203 11:43:46.510282 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/93ca3376-6916-4fc9-8b76-7cd9a930865e-etc-swift\") pod \"swift-storage-0\" (UID: \"93ca3376-6916-4fc9-8b76-7cd9a930865e\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 11:43:46 crc kubenswrapper[5052]: I1203 11:43:46.514746 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-db-sync-g22fp" event={"ID":"3c063992-bf41-4c35-a5ad-cdc4b512bcee","Type":"ContainerDied","Data":"26f4f09583bbf7632b9bdaf7e7510fd09044cb4fe5fa34de71c59e2f4a65855d"} Dec 03 11:43:46 crc kubenswrapper[5052]: I1203 11:43:46.514793 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="26f4f09583bbf7632b9bdaf7e7510fd09044cb4fe5fa34de71c59e2f4a65855d" Dec 03 11:43:46 crc kubenswrapper[5052]: I1203 11:43:46.514804 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-db-sync-g22fp" Dec 03 11:43:46 crc kubenswrapper[5052]: I1203 11:43:46.518427 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/93ca3376-6916-4fc9-8b76-7cd9a930865e-etc-swift\") pod \"swift-storage-0\" (UID: \"93ca3376-6916-4fc9-8b76-7cd9a930865e\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 11:43:46 crc kubenswrapper[5052]: I1203 11:43:46.562594 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-storage-0" Dec 03 11:43:46 crc kubenswrapper[5052]: I1203 11:43:46.857605 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 11:43:47 crc kubenswrapper[5052]: I1203 11:43:47.141677 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/swift-storage-0"] Dec 03 11:43:47 crc kubenswrapper[5052]: W1203 11:43:47.150647 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod93ca3376_6916_4fc9_8b76_7cd9a930865e.slice/crio-7ec32bb8331974e634a7411cf1d4536553c9750926536e8c626baddd97b10c7a WatchSource:0}: Error finding container 7ec32bb8331974e634a7411cf1d4536553c9750926536e8c626baddd97b10c7a: Status 404 returned error can't find the container with id 7ec32bb8331974e634a7411cf1d4536553c9750926536e8c626baddd97b10c7a Dec 03 11:43:47 crc kubenswrapper[5052]: I1203 11:43:47.524010 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"93ca3376-6916-4fc9-8b76-7cd9a930865e","Type":"ContainerStarted","Data":"15eb357802bb079afbd7911a3c2b816f45034279f8a2a7e70d89277c5699b052"} Dec 03 11:43:47 crc kubenswrapper[5052]: I1203 11:43:47.524052 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"93ca3376-6916-4fc9-8b76-7cd9a930865e","Type":"ContainerStarted","Data":"7ec32bb8331974e634a7411cf1d4536553c9750926536e8c626baddd97b10c7a"} Dec 03 11:43:48 crc kubenswrapper[5052]: I1203 11:43:48.566154 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"93ca3376-6916-4fc9-8b76-7cd9a930865e","Type":"ContainerStarted","Data":"767653c60f88687aa9e17c73cfb59cad0f979a47f7e0eb1bbebd1374e7223b77"} Dec 03 11:43:48 crc kubenswrapper[5052]: I1203 11:43:48.566584 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"93ca3376-6916-4fc9-8b76-7cd9a930865e","Type":"ContainerStarted","Data":"170a73bd9b5a2ea0ebc55d043e19812afef343218daa53630096c9db6698e952"} Dec 03 11:43:48 crc kubenswrapper[5052]: I1203 11:43:48.566604 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"93ca3376-6916-4fc9-8b76-7cd9a930865e","Type":"ContainerStarted","Data":"05a11312e8ec969e44a91b4857ff72727254fdf705f9356f00323d2cea9feb2f"} Dec 03 11:43:48 crc kubenswrapper[5052]: I1203 11:43:48.566616 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"93ca3376-6916-4fc9-8b76-7cd9a930865e","Type":"ContainerStarted","Data":"08d93d7b7a336f74cf614cfa15c40191bdfb66f13fc15d41dcf943b57d2e92db"} Dec 03 11:43:48 crc kubenswrapper[5052]: I1203 11:43:48.566649 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"93ca3376-6916-4fc9-8b76-7cd9a930865e","Type":"ContainerStarted","Data":"e94a57b11b76a48c619f83900e8a10f2bdcc2a6fc844e2f0bb06dd5042373965"} Dec 03 11:43:48 crc kubenswrapper[5052]: I1203 11:43:48.566661 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"93ca3376-6916-4fc9-8b76-7cd9a930865e","Type":"ContainerStarted","Data":"f13880839e746af36615e3cd0a68485f5974bfc02f15aacef8d8c58a4cd8e970"} Dec 03 11:43:49 crc kubenswrapper[5052]: I1203 11:43:49.582961 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"93ca3376-6916-4fc9-8b76-7cd9a930865e","Type":"ContainerStarted","Data":"11b133a6cbb654d7396b3e561b4ee9d5897ec4b55a4240fcf44eff0dc1c666b1"} Dec 03 11:43:49 crc kubenswrapper[5052]: I1203 11:43:49.584113 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"93ca3376-6916-4fc9-8b76-7cd9a930865e","Type":"ContainerStarted","Data":"7c70d2e1650a64f27e594fadf25a119e4c9f5f13e151324319b5d9429e343564"} Dec 03 11:43:49 crc kubenswrapper[5052]: I1203 11:43:49.584442 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"93ca3376-6916-4fc9-8b76-7cd9a930865e","Type":"ContainerStarted","Data":"fdfc3ff42d3ff2045f403d93c6d32c7111bdf6fba8059c6ba854561088de3a81"} Dec 03 11:43:49 crc kubenswrapper[5052]: I1203 11:43:49.584578 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"93ca3376-6916-4fc9-8b76-7cd9a930865e","Type":"ContainerStarted","Data":"f561321b0fdca1154a6977a337503c90f840416e5bc0fa39babc9ea06bd5b9bf"} Dec 03 11:43:49 crc kubenswrapper[5052]: I1203 11:43:49.584656 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"93ca3376-6916-4fc9-8b76-7cd9a930865e","Type":"ContainerStarted","Data":"0049fb7f1583b92a6e9922087915b30b055b3e38241dbdef833f7e931c597b18"} Dec 03 11:43:49 crc kubenswrapper[5052]: I1203 11:43:49.584850 5052 generic.go:334] "Generic (PLEG): container finished" podID="d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0" containerID="843eb248ababc1108cc5ab5df2a9259468619c4e57c04fb2e96d893def21be0a" exitCode=0 Dec 03 11:43:49 crc kubenswrapper[5052]: I1203 11:43:49.584929 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-server-0" event={"ID":"d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0","Type":"ContainerDied","Data":"843eb248ababc1108cc5ab5df2a9259468619c4e57c04fb2e96d893def21be0a"} Dec 03 11:43:49 crc kubenswrapper[5052]: I1203 11:43:49.589428 5052 generic.go:334] "Generic (PLEG): container finished" podID="2766f489-9217-47ae-a595-869a26389587" containerID="3b75a48ddef04dba6740dfd97e3821f7c14d8f167622e54e606f6aae843251a9" exitCode=0 Dec 03 11:43:49 crc kubenswrapper[5052]: I1203 11:43:49.589491 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" event={"ID":"2766f489-9217-47ae-a595-869a26389587","Type":"ContainerDied","Data":"3b75a48ddef04dba6740dfd97e3821f7c14d8f167622e54e606f6aae843251a9"} Dec 03 11:43:50 crc kubenswrapper[5052]: I1203 11:43:50.601245 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-server-0" event={"ID":"d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0","Type":"ContainerStarted","Data":"f1ccb0c2e8810415edac670e94d35e316eb9d118a2ee2cb2d84f782419550625"} Dec 03 11:43:50 crc kubenswrapper[5052]: I1203 11:43:50.601738 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 11:43:50 crc kubenswrapper[5052]: I1203 11:43:50.604987 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" event={"ID":"2766f489-9217-47ae-a595-869a26389587","Type":"ContainerStarted","Data":"9725070208b6721d2746b3710550f0679a73b1a1ff7fc62cba3c5b2a68f65659"} Dec 03 11:43:50 crc kubenswrapper[5052]: I1203 11:43:50.605179 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 11:43:50 crc kubenswrapper[5052]: I1203 11:43:50.620902 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"93ca3376-6916-4fc9-8b76-7cd9a930865e","Type":"ContainerStarted","Data":"dcf4a9a4ecfc80507c16a16774b51db2868255753c35cb58f980eb45993a0c29"} Dec 03 11:43:50 crc kubenswrapper[5052]: I1203 11:43:50.620947 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"93ca3376-6916-4fc9-8b76-7cd9a930865e","Type":"ContainerStarted","Data":"57a92592bc143e7e16a01ba9cb34e8bc93f5c7a21ccf4e346f8763a448a75b42"} Dec 03 11:43:50 crc kubenswrapper[5052]: I1203 11:43:50.620959 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"93ca3376-6916-4fc9-8b76-7cd9a930865e","Type":"ContainerStarted","Data":"aca4ad76bb32ce93c82a24cd5e11df447906446998393a1992802096707a00b7"} Dec 03 11:43:50 crc kubenswrapper[5052]: I1203 11:43:50.635301 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/rabbitmq-server-0" podStartSLOduration=36.635273412 podStartE2EDuration="36.635273412s" podCreationTimestamp="2025-12-03 11:43:14 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:43:50.62343498 +0000 UTC m=+1421.211676744" watchObservedRunningTime="2025-12-03 11:43:50.635273412 +0000 UTC m=+1421.223515176" Dec 03 11:43:50 crc kubenswrapper[5052]: I1203 11:43:50.653386 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" podStartSLOduration=37.653366693 podStartE2EDuration="37.653366693s" podCreationTimestamp="2025-12-03 11:43:13 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:43:50.648304717 +0000 UTC m=+1421.236546491" watchObservedRunningTime="2025-12-03 11:43:50.653366693 +0000 UTC m=+1421.241608457" Dec 03 11:43:50 crc kubenswrapper[5052]: I1203 11:43:50.683316 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/swift-storage-0" podStartSLOduration=20.683299466 podStartE2EDuration="20.683299466s" podCreationTimestamp="2025-12-03 11:43:30 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:43:50.678481298 +0000 UTC m=+1421.266723062" watchObservedRunningTime="2025-12-03 11:43:50.683299466 +0000 UTC m=+1421.271541230" Dec 03 11:43:50 crc kubenswrapper[5052]: I1203 11:43:50.817308 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-7d7b5bb58f-sgdms"] Dec 03 11:43:50 crc kubenswrapper[5052]: E1203 11:43:50.817793 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="c3a670b4-73f2-4c90-a3af-ba5d660ec1f7" containerName="swift-ring-rebalance" Dec 03 11:43:50 crc kubenswrapper[5052]: I1203 11:43:50.817826 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="c3a670b4-73f2-4c90-a3af-ba5d660ec1f7" containerName="swift-ring-rebalance" Dec 03 11:43:50 crc kubenswrapper[5052]: E1203 11:43:50.817847 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="3c063992-bf41-4c35-a5ad-cdc4b512bcee" containerName="glance-db-sync" Dec 03 11:43:50 crc kubenswrapper[5052]: I1203 11:43:50.817853 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="3c063992-bf41-4c35-a5ad-cdc4b512bcee" containerName="glance-db-sync" Dec 03 11:43:50 crc kubenswrapper[5052]: I1203 11:43:50.818050 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="3c063992-bf41-4c35-a5ad-cdc4b512bcee" containerName="glance-db-sync" Dec 03 11:43:50 crc kubenswrapper[5052]: I1203 11:43:50.818068 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="c3a670b4-73f2-4c90-a3af-ba5d660ec1f7" containerName="swift-ring-rebalance" Dec 03 11:43:50 crc kubenswrapper[5052]: I1203 11:43:50.818923 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-7d7b5bb58f-sgdms" Dec 03 11:43:50 crc kubenswrapper[5052]: I1203 11:43:50.820886 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"dns-swift-storage-0" Dec 03 11:43:50 crc kubenswrapper[5052]: I1203 11:43:50.845357 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-7d7b5bb58f-sgdms"] Dec 03 11:43:51 crc kubenswrapper[5052]: I1203 11:43:50.999895 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dns-swift-storage-0\" (UniqueName: \"kubernetes.io/configmap/bbbc503e-d33d-42a9-856a-9ecc1d059f6b-dns-swift-storage-0\") pod \"dnsmasq-dnsmasq-7d7b5bb58f-sgdms\" (UID: \"bbbc503e-d33d-42a9-856a-9ecc1d059f6b\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-7d7b5bb58f-sgdms" Dec 03 11:43:51 crc kubenswrapper[5052]: I1203 11:43:50.999970 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/bbbc503e-d33d-42a9-856a-9ecc1d059f6b-config\") pod \"dnsmasq-dnsmasq-7d7b5bb58f-sgdms\" (UID: \"bbbc503e-d33d-42a9-856a-9ecc1d059f6b\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-7d7b5bb58f-sgdms" Dec 03 11:43:51 crc kubenswrapper[5052]: I1203 11:43:51.000097 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nctjk\" (UniqueName: \"kubernetes.io/projected/bbbc503e-d33d-42a9-856a-9ecc1d059f6b-kube-api-access-nctjk\") pod \"dnsmasq-dnsmasq-7d7b5bb58f-sgdms\" (UID: \"bbbc503e-d33d-42a9-856a-9ecc1d059f6b\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-7d7b5bb58f-sgdms" Dec 03 11:43:51 crc kubenswrapper[5052]: I1203 11:43:51.000186 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/bbbc503e-d33d-42a9-856a-9ecc1d059f6b-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-7d7b5bb58f-sgdms\" (UID: \"bbbc503e-d33d-42a9-856a-9ecc1d059f6b\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-7d7b5bb58f-sgdms" Dec 03 11:43:51 crc kubenswrapper[5052]: I1203 11:43:51.101598 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"dns-swift-storage-0\" (UniqueName: \"kubernetes.io/configmap/bbbc503e-d33d-42a9-856a-9ecc1d059f6b-dns-swift-storage-0\") pod \"dnsmasq-dnsmasq-7d7b5bb58f-sgdms\" (UID: \"bbbc503e-d33d-42a9-856a-9ecc1d059f6b\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-7d7b5bb58f-sgdms" Dec 03 11:43:51 crc kubenswrapper[5052]: I1203 11:43:51.101661 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/bbbc503e-d33d-42a9-856a-9ecc1d059f6b-config\") pod \"dnsmasq-dnsmasq-7d7b5bb58f-sgdms\" (UID: \"bbbc503e-d33d-42a9-856a-9ecc1d059f6b\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-7d7b5bb58f-sgdms" Dec 03 11:43:51 crc kubenswrapper[5052]: I1203 11:43:51.101713 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-nctjk\" (UniqueName: \"kubernetes.io/projected/bbbc503e-d33d-42a9-856a-9ecc1d059f6b-kube-api-access-nctjk\") pod \"dnsmasq-dnsmasq-7d7b5bb58f-sgdms\" (UID: \"bbbc503e-d33d-42a9-856a-9ecc1d059f6b\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-7d7b5bb58f-sgdms" Dec 03 11:43:51 crc kubenswrapper[5052]: I1203 11:43:51.101746 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/bbbc503e-d33d-42a9-856a-9ecc1d059f6b-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-7d7b5bb58f-sgdms\" (UID: \"bbbc503e-d33d-42a9-856a-9ecc1d059f6b\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-7d7b5bb58f-sgdms" Dec 03 11:43:51 crc kubenswrapper[5052]: I1203 11:43:51.102819 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/bbbc503e-d33d-42a9-856a-9ecc1d059f6b-config\") pod \"dnsmasq-dnsmasq-7d7b5bb58f-sgdms\" (UID: \"bbbc503e-d33d-42a9-856a-9ecc1d059f6b\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-7d7b5bb58f-sgdms" Dec 03 11:43:51 crc kubenswrapper[5052]: I1203 11:43:51.102840 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/bbbc503e-d33d-42a9-856a-9ecc1d059f6b-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-7d7b5bb58f-sgdms\" (UID: \"bbbc503e-d33d-42a9-856a-9ecc1d059f6b\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-7d7b5bb58f-sgdms" Dec 03 11:43:51 crc kubenswrapper[5052]: I1203 11:43:51.102840 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"dns-swift-storage-0\" (UniqueName: \"kubernetes.io/configmap/bbbc503e-d33d-42a9-856a-9ecc1d059f6b-dns-swift-storage-0\") pod \"dnsmasq-dnsmasq-7d7b5bb58f-sgdms\" (UID: \"bbbc503e-d33d-42a9-856a-9ecc1d059f6b\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-7d7b5bb58f-sgdms" Dec 03 11:43:51 crc kubenswrapper[5052]: I1203 11:43:51.143437 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-nctjk\" (UniqueName: \"kubernetes.io/projected/bbbc503e-d33d-42a9-856a-9ecc1d059f6b-kube-api-access-nctjk\") pod \"dnsmasq-dnsmasq-7d7b5bb58f-sgdms\" (UID: \"bbbc503e-d33d-42a9-856a-9ecc1d059f6b\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-7d7b5bb58f-sgdms" Dec 03 11:43:51 crc kubenswrapper[5052]: I1203 11:43:51.435909 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-7d7b5bb58f-sgdms" Dec 03 11:43:51 crc kubenswrapper[5052]: I1203 11:43:51.917304 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-7d7b5bb58f-sgdms"] Dec 03 11:43:51 crc kubenswrapper[5052]: W1203 11:43:51.920967 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbbbc503e_d33d_42a9_856a_9ecc1d059f6b.slice/crio-1397a5677def85e35981116a319fe663882469aeb115727b8a3e6bc09c12342e WatchSource:0}: Error finding container 1397a5677def85e35981116a319fe663882469aeb115727b8a3e6bc09c12342e: Status 404 returned error can't find the container with id 1397a5677def85e35981116a319fe663882469aeb115727b8a3e6bc09c12342e Dec 03 11:43:52 crc kubenswrapper[5052]: I1203 11:43:52.674306 5052 generic.go:334] "Generic (PLEG): container finished" podID="bbbc503e-d33d-42a9-856a-9ecc1d059f6b" containerID="a76bef2305c43c6073eaeef6861eaec8928c009de434c14bc97bc064ff85d56e" exitCode=0 Dec 03 11:43:52 crc kubenswrapper[5052]: I1203 11:43:52.674364 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-7d7b5bb58f-sgdms" event={"ID":"bbbc503e-d33d-42a9-856a-9ecc1d059f6b","Type":"ContainerDied","Data":"a76bef2305c43c6073eaeef6861eaec8928c009de434c14bc97bc064ff85d56e"} Dec 03 11:43:52 crc kubenswrapper[5052]: I1203 11:43:52.674392 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-7d7b5bb58f-sgdms" event={"ID":"bbbc503e-d33d-42a9-856a-9ecc1d059f6b","Type":"ContainerStarted","Data":"1397a5677def85e35981116a319fe663882469aeb115727b8a3e6bc09c12342e"} Dec 03 11:43:53 crc kubenswrapper[5052]: I1203 11:43:53.689987 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-7d7b5bb58f-sgdms" Dec 03 11:43:53 crc kubenswrapper[5052]: I1203 11:43:53.691411 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-7d7b5bb58f-sgdms" event={"ID":"bbbc503e-d33d-42a9-856a-9ecc1d059f6b","Type":"ContainerStarted","Data":"d0ee0cef3f1acf1c1868c3a8e765eb1d1fe5a4af1fd316d302221be7039841f9"} Dec 03 11:43:53 crc kubenswrapper[5052]: I1203 11:43:53.719842 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-7d7b5bb58f-sgdms" podStartSLOduration=3.719817231 podStartE2EDuration="3.719817231s" podCreationTimestamp="2025-12-03 11:43:50 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:43:53.7139139 +0000 UTC m=+1424.302155684" watchObservedRunningTime="2025-12-03 11:43:53.719817231 +0000 UTC m=+1424.308059015" Dec 03 11:44:01 crc kubenswrapper[5052]: I1203 11:44:01.437700 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-7d7b5bb58f-sgdms" Dec 03 11:44:01 crc kubenswrapper[5052]: I1203 11:44:01.490959 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-54w7t"] Dec 03 11:44:01 crc kubenswrapper[5052]: I1203 11:44:01.491249 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-54w7t" podUID="fc5f71c9-f943-4ba6-a8eb-8e16ed558675" containerName="dnsmasq-dns" containerID="cri-o://932d879f7c954a746e1dd51fe1d60d6090129a6831b82e9b480ae97cb1975957" gracePeriod=10 Dec 03 11:44:01 crc kubenswrapper[5052]: I1203 11:44:01.792114 5052 generic.go:334] "Generic (PLEG): container finished" podID="fc5f71c9-f943-4ba6-a8eb-8e16ed558675" containerID="932d879f7c954a746e1dd51fe1d60d6090129a6831b82e9b480ae97cb1975957" exitCode=0 Dec 03 11:44:01 crc kubenswrapper[5052]: I1203 11:44:01.792362 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-54w7t" event={"ID":"fc5f71c9-f943-4ba6-a8eb-8e16ed558675","Type":"ContainerDied","Data":"932d879f7c954a746e1dd51fe1d60d6090129a6831b82e9b480ae97cb1975957"} Dec 03 11:44:01 crc kubenswrapper[5052]: I1203 11:44:01.946103 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-54w7t" Dec 03 11:44:01 crc kubenswrapper[5052]: I1203 11:44:01.965728 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ws6lz\" (UniqueName: \"kubernetes.io/projected/fc5f71c9-f943-4ba6-a8eb-8e16ed558675-kube-api-access-ws6lz\") pod \"fc5f71c9-f943-4ba6-a8eb-8e16ed558675\" (UID: \"fc5f71c9-f943-4ba6-a8eb-8e16ed558675\") " Dec 03 11:44:01 crc kubenswrapper[5052]: I1203 11:44:01.965817 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/fc5f71c9-f943-4ba6-a8eb-8e16ed558675-dnsmasq-svc\") pod \"fc5f71c9-f943-4ba6-a8eb-8e16ed558675\" (UID: \"fc5f71c9-f943-4ba6-a8eb-8e16ed558675\") " Dec 03 11:44:01 crc kubenswrapper[5052]: I1203 11:44:01.965896 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/fc5f71c9-f943-4ba6-a8eb-8e16ed558675-config\") pod \"fc5f71c9-f943-4ba6-a8eb-8e16ed558675\" (UID: \"fc5f71c9-f943-4ba6-a8eb-8e16ed558675\") " Dec 03 11:44:01 crc kubenswrapper[5052]: I1203 11:44:01.972162 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fc5f71c9-f943-4ba6-a8eb-8e16ed558675-kube-api-access-ws6lz" (OuterVolumeSpecName: "kube-api-access-ws6lz") pod "fc5f71c9-f943-4ba6-a8eb-8e16ed558675" (UID: "fc5f71c9-f943-4ba6-a8eb-8e16ed558675"). InnerVolumeSpecName "kube-api-access-ws6lz". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:44:02 crc kubenswrapper[5052]: I1203 11:44:02.069386 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-ws6lz\" (UniqueName: \"kubernetes.io/projected/fc5f71c9-f943-4ba6-a8eb-8e16ed558675-kube-api-access-ws6lz\") on node \"crc\" DevicePath \"\"" Dec 03 11:44:02 crc kubenswrapper[5052]: I1203 11:44:02.071165 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fc5f71c9-f943-4ba6-a8eb-8e16ed558675-config" (OuterVolumeSpecName: "config") pod "fc5f71c9-f943-4ba6-a8eb-8e16ed558675" (UID: "fc5f71c9-f943-4ba6-a8eb-8e16ed558675"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:44:02 crc kubenswrapper[5052]: I1203 11:44:02.076074 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fc5f71c9-f943-4ba6-a8eb-8e16ed558675-dnsmasq-svc" (OuterVolumeSpecName: "dnsmasq-svc") pod "fc5f71c9-f943-4ba6-a8eb-8e16ed558675" (UID: "fc5f71c9-f943-4ba6-a8eb-8e16ed558675"). InnerVolumeSpecName "dnsmasq-svc". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:44:02 crc kubenswrapper[5052]: I1203 11:44:02.172973 5052 reconciler_common.go:293] "Volume detached for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/fc5f71c9-f943-4ba6-a8eb-8e16ed558675-dnsmasq-svc\") on node \"crc\" DevicePath \"\"" Dec 03 11:44:02 crc kubenswrapper[5052]: I1203 11:44:02.173010 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/fc5f71c9-f943-4ba6-a8eb-8e16ed558675-config\") on node \"crc\" DevicePath \"\"" Dec 03 11:44:02 crc kubenswrapper[5052]: I1203 11:44:02.804306 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-54w7t" event={"ID":"fc5f71c9-f943-4ba6-a8eb-8e16ed558675","Type":"ContainerDied","Data":"be1b0eeec7747986ad2608047680e1e8bbe1cc60aedb46608548d387ea104074"} Dec 03 11:44:02 crc kubenswrapper[5052]: I1203 11:44:02.804382 5052 scope.go:117] "RemoveContainer" containerID="932d879f7c954a746e1dd51fe1d60d6090129a6831b82e9b480ae97cb1975957" Dec 03 11:44:02 crc kubenswrapper[5052]: I1203 11:44:02.804433 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-54w7t" Dec 03 11:44:02 crc kubenswrapper[5052]: I1203 11:44:02.833846 5052 scope.go:117] "RemoveContainer" containerID="9a7110a1e99d880ed3c4e50b9718ac4ef71d8575d2f793cdc84e9e47fb1598f0" Dec 03 11:44:02 crc kubenswrapper[5052]: I1203 11:44:02.859453 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-54w7t"] Dec 03 11:44:02 crc kubenswrapper[5052]: I1203 11:44:02.866927 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-54w7t"] Dec 03 11:44:03 crc kubenswrapper[5052]: I1203 11:44:03.683648 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fc5f71c9-f943-4ba6-a8eb-8e16ed558675" path="/var/lib/kubelet/pods/fc5f71c9-f943-4ba6-a8eb-8e16ed558675/volumes" Dec 03 11:44:04 crc kubenswrapper[5052]: I1203 11:44:04.795800 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 11:44:05 crc kubenswrapper[5052]: I1203 11:44:05.977493 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 11:44:06 crc kubenswrapper[5052]: I1203 11:44:06.740381 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/cinder-db-create-57t99"] Dec 03 11:44:06 crc kubenswrapper[5052]: E1203 11:44:06.742224 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="fc5f71c9-f943-4ba6-a8eb-8e16ed558675" containerName="dnsmasq-dns" Dec 03 11:44:06 crc kubenswrapper[5052]: I1203 11:44:06.742253 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="fc5f71c9-f943-4ba6-a8eb-8e16ed558675" containerName="dnsmasq-dns" Dec 03 11:44:06 crc kubenswrapper[5052]: E1203 11:44:06.742282 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="fc5f71c9-f943-4ba6-a8eb-8e16ed558675" containerName="init" Dec 03 11:44:06 crc kubenswrapper[5052]: I1203 11:44:06.742290 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="fc5f71c9-f943-4ba6-a8eb-8e16ed558675" containerName="init" Dec 03 11:44:06 crc kubenswrapper[5052]: I1203 11:44:06.742478 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="fc5f71c9-f943-4ba6-a8eb-8e16ed558675" containerName="dnsmasq-dns" Dec 03 11:44:06 crc kubenswrapper[5052]: I1203 11:44:06.743174 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-db-create-57t99" Dec 03 11:44:06 crc kubenswrapper[5052]: I1203 11:44:06.751879 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-db-create-57t99"] Dec 03 11:44:06 crc kubenswrapper[5052]: I1203 11:44:06.857612 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/d4c5312a-6993-40de-b9af-bbe3820ae84a-operator-scripts\") pod \"cinder-db-create-57t99\" (UID: \"d4c5312a-6993-40de-b9af-bbe3820ae84a\") " pod="openstack-kuttl-tests/cinder-db-create-57t99" Dec 03 11:44:06 crc kubenswrapper[5052]: I1203 11:44:06.857694 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7442s\" (UniqueName: \"kubernetes.io/projected/d4c5312a-6993-40de-b9af-bbe3820ae84a-kube-api-access-7442s\") pod \"cinder-db-create-57t99\" (UID: \"d4c5312a-6993-40de-b9af-bbe3820ae84a\") " pod="openstack-kuttl-tests/cinder-db-create-57t99" Dec 03 11:44:06 crc kubenswrapper[5052]: I1203 11:44:06.877952 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/barbican-db-create-hvzcc"] Dec 03 11:44:06 crc kubenswrapper[5052]: I1203 11:44:06.879254 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-db-create-hvzcc" Dec 03 11:44:06 crc kubenswrapper[5052]: I1203 11:44:06.887826 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/neutron-18a7-account-create-update-n4q4t"] Dec 03 11:44:06 crc kubenswrapper[5052]: I1203 11:44:06.891097 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-18a7-account-create-update-n4q4t" Dec 03 11:44:06 crc kubenswrapper[5052]: I1203 11:44:06.895386 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"neutron-db-secret" Dec 03 11:44:06 crc kubenswrapper[5052]: I1203 11:44:06.900950 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-db-create-hvzcc"] Dec 03 11:44:06 crc kubenswrapper[5052]: I1203 11:44:06.913204 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-18a7-account-create-update-n4q4t"] Dec 03 11:44:06 crc kubenswrapper[5052]: I1203 11:44:06.958882 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/d4c5312a-6993-40de-b9af-bbe3820ae84a-operator-scripts\") pod \"cinder-db-create-57t99\" (UID: \"d4c5312a-6993-40de-b9af-bbe3820ae84a\") " pod="openstack-kuttl-tests/cinder-db-create-57t99" Dec 03 11:44:06 crc kubenswrapper[5052]: I1203 11:44:06.958939 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-7442s\" (UniqueName: \"kubernetes.io/projected/d4c5312a-6993-40de-b9af-bbe3820ae84a-kube-api-access-7442s\") pod \"cinder-db-create-57t99\" (UID: \"d4c5312a-6993-40de-b9af-bbe3820ae84a\") " pod="openstack-kuttl-tests/cinder-db-create-57t99" Dec 03 11:44:06 crc kubenswrapper[5052]: I1203 11:44:06.959568 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/d4c5312a-6993-40de-b9af-bbe3820ae84a-operator-scripts\") pod \"cinder-db-create-57t99\" (UID: \"d4c5312a-6993-40de-b9af-bbe3820ae84a\") " pod="openstack-kuttl-tests/cinder-db-create-57t99" Dec 03 11:44:06 crc kubenswrapper[5052]: I1203 11:44:06.979910 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-7442s\" (UniqueName: \"kubernetes.io/projected/d4c5312a-6993-40de-b9af-bbe3820ae84a-kube-api-access-7442s\") pod \"cinder-db-create-57t99\" (UID: \"d4c5312a-6993-40de-b9af-bbe3820ae84a\") " pod="openstack-kuttl-tests/cinder-db-create-57t99" Dec 03 11:44:07 crc kubenswrapper[5052]: I1203 11:44:07.040407 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/barbican-5976-account-create-update-dqgqh"] Dec 03 11:44:07 crc kubenswrapper[5052]: I1203 11:44:07.041939 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-5976-account-create-update-dqgqh" Dec 03 11:44:07 crc kubenswrapper[5052]: I1203 11:44:07.043577 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"barbican-db-secret" Dec 03 11:44:07 crc kubenswrapper[5052]: I1203 11:44:07.055896 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-5976-account-create-update-dqgqh"] Dec 03 11:44:07 crc kubenswrapper[5052]: I1203 11:44:07.060214 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/0c6c4cfd-0973-4ca6-beca-147213271fe3-operator-scripts\") pod \"barbican-db-create-hvzcc\" (UID: \"0c6c4cfd-0973-4ca6-beca-147213271fe3\") " pod="openstack-kuttl-tests/barbican-db-create-hvzcc" Dec 03 11:44:07 crc kubenswrapper[5052]: I1203 11:44:07.060429 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m4dgn\" (UniqueName: \"kubernetes.io/projected/dc73f58f-4601-4b36-92b1-707a5779c99e-kube-api-access-m4dgn\") pod \"neutron-18a7-account-create-update-n4q4t\" (UID: \"dc73f58f-4601-4b36-92b1-707a5779c99e\") " pod="openstack-kuttl-tests/neutron-18a7-account-create-update-n4q4t" Dec 03 11:44:07 crc kubenswrapper[5052]: I1203 11:44:07.060597 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/dc73f58f-4601-4b36-92b1-707a5779c99e-operator-scripts\") pod \"neutron-18a7-account-create-update-n4q4t\" (UID: \"dc73f58f-4601-4b36-92b1-707a5779c99e\") " pod="openstack-kuttl-tests/neutron-18a7-account-create-update-n4q4t" Dec 03 11:44:07 crc kubenswrapper[5052]: I1203 11:44:07.060722 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fgbq2\" (UniqueName: \"kubernetes.io/projected/0c6c4cfd-0973-4ca6-beca-147213271fe3-kube-api-access-fgbq2\") pod \"barbican-db-create-hvzcc\" (UID: \"0c6c4cfd-0973-4ca6-beca-147213271fe3\") " pod="openstack-kuttl-tests/barbican-db-create-hvzcc" Dec 03 11:44:07 crc kubenswrapper[5052]: I1203 11:44:07.071965 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-db-create-57t99" Dec 03 11:44:07 crc kubenswrapper[5052]: I1203 11:44:07.091933 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/keystone-db-sync-plblg"] Dec 03 11:44:07 crc kubenswrapper[5052]: I1203 11:44:07.093225 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-db-sync-plblg" Dec 03 11:44:07 crc kubenswrapper[5052]: I1203 11:44:07.096899 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone-scripts" Dec 03 11:44:07 crc kubenswrapper[5052]: I1203 11:44:07.097128 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone" Dec 03 11:44:07 crc kubenswrapper[5052]: I1203 11:44:07.097860 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone-config-data" Dec 03 11:44:07 crc kubenswrapper[5052]: I1203 11:44:07.099343 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone-keystone-dockercfg-4hzgj" Dec 03 11:44:07 crc kubenswrapper[5052]: I1203 11:44:07.117845 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-db-sync-plblg"] Dec 03 11:44:07 crc kubenswrapper[5052]: I1203 11:44:07.164329 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/0c6c4cfd-0973-4ca6-beca-147213271fe3-operator-scripts\") pod \"barbican-db-create-hvzcc\" (UID: \"0c6c4cfd-0973-4ca6-beca-147213271fe3\") " pod="openstack-kuttl-tests/barbican-db-create-hvzcc" Dec 03 11:44:07 crc kubenswrapper[5052]: I1203 11:44:07.164378 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-m4dgn\" (UniqueName: \"kubernetes.io/projected/dc73f58f-4601-4b36-92b1-707a5779c99e-kube-api-access-m4dgn\") pod \"neutron-18a7-account-create-update-n4q4t\" (UID: \"dc73f58f-4601-4b36-92b1-707a5779c99e\") " pod="openstack-kuttl-tests/neutron-18a7-account-create-update-n4q4t" Dec 03 11:44:07 crc kubenswrapper[5052]: I1203 11:44:07.164416 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zr4fx\" (UniqueName: \"kubernetes.io/projected/c97cfd9e-1b42-494f-8eb5-206bf9791575-kube-api-access-zr4fx\") pod \"barbican-5976-account-create-update-dqgqh\" (UID: \"c97cfd9e-1b42-494f-8eb5-206bf9791575\") " pod="openstack-kuttl-tests/barbican-5976-account-create-update-dqgqh" Dec 03 11:44:07 crc kubenswrapper[5052]: I1203 11:44:07.164445 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/dc73f58f-4601-4b36-92b1-707a5779c99e-operator-scripts\") pod \"neutron-18a7-account-create-update-n4q4t\" (UID: \"dc73f58f-4601-4b36-92b1-707a5779c99e\") " pod="openstack-kuttl-tests/neutron-18a7-account-create-update-n4q4t" Dec 03 11:44:07 crc kubenswrapper[5052]: I1203 11:44:07.164474 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/c97cfd9e-1b42-494f-8eb5-206bf9791575-operator-scripts\") pod \"barbican-5976-account-create-update-dqgqh\" (UID: \"c97cfd9e-1b42-494f-8eb5-206bf9791575\") " pod="openstack-kuttl-tests/barbican-5976-account-create-update-dqgqh" Dec 03 11:44:07 crc kubenswrapper[5052]: I1203 11:44:07.164520 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-fgbq2\" (UniqueName: \"kubernetes.io/projected/0c6c4cfd-0973-4ca6-beca-147213271fe3-kube-api-access-fgbq2\") pod \"barbican-db-create-hvzcc\" (UID: \"0c6c4cfd-0973-4ca6-beca-147213271fe3\") " pod="openstack-kuttl-tests/barbican-db-create-hvzcc" Dec 03 11:44:07 crc kubenswrapper[5052]: I1203 11:44:07.165772 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/0c6c4cfd-0973-4ca6-beca-147213271fe3-operator-scripts\") pod \"barbican-db-create-hvzcc\" (UID: \"0c6c4cfd-0973-4ca6-beca-147213271fe3\") " pod="openstack-kuttl-tests/barbican-db-create-hvzcc" Dec 03 11:44:07 crc kubenswrapper[5052]: I1203 11:44:07.165811 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/dc73f58f-4601-4b36-92b1-707a5779c99e-operator-scripts\") pod \"neutron-18a7-account-create-update-n4q4t\" (UID: \"dc73f58f-4601-4b36-92b1-707a5779c99e\") " pod="openstack-kuttl-tests/neutron-18a7-account-create-update-n4q4t" Dec 03 11:44:07 crc kubenswrapper[5052]: I1203 11:44:07.167573 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/cinder-1bf0-account-create-update-pvvl8"] Dec 03 11:44:07 crc kubenswrapper[5052]: I1203 11:44:07.171223 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-1bf0-account-create-update-pvvl8" Dec 03 11:44:07 crc kubenswrapper[5052]: I1203 11:44:07.175147 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cinder-db-secret" Dec 03 11:44:07 crc kubenswrapper[5052]: I1203 11:44:07.178551 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-1bf0-account-create-update-pvvl8"] Dec 03 11:44:07 crc kubenswrapper[5052]: I1203 11:44:07.190186 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-m4dgn\" (UniqueName: \"kubernetes.io/projected/dc73f58f-4601-4b36-92b1-707a5779c99e-kube-api-access-m4dgn\") pod \"neutron-18a7-account-create-update-n4q4t\" (UID: \"dc73f58f-4601-4b36-92b1-707a5779c99e\") " pod="openstack-kuttl-tests/neutron-18a7-account-create-update-n4q4t" Dec 03 11:44:07 crc kubenswrapper[5052]: I1203 11:44:07.190825 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-fgbq2\" (UniqueName: \"kubernetes.io/projected/0c6c4cfd-0973-4ca6-beca-147213271fe3-kube-api-access-fgbq2\") pod \"barbican-db-create-hvzcc\" (UID: \"0c6c4cfd-0973-4ca6-beca-147213271fe3\") " pod="openstack-kuttl-tests/barbican-db-create-hvzcc" Dec 03 11:44:07 crc kubenswrapper[5052]: I1203 11:44:07.203231 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-db-create-hvzcc" Dec 03 11:44:07 crc kubenswrapper[5052]: I1203 11:44:07.221339 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-18a7-account-create-update-n4q4t" Dec 03 11:44:07 crc kubenswrapper[5052]: I1203 11:44:07.243850 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/neutron-db-create-n5v7p"] Dec 03 11:44:07 crc kubenswrapper[5052]: I1203 11:44:07.244840 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-db-create-n5v7p" Dec 03 11:44:07 crc kubenswrapper[5052]: I1203 11:44:07.259335 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-db-create-n5v7p"] Dec 03 11:44:07 crc kubenswrapper[5052]: I1203 11:44:07.267361 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1445732d-e68a-4d4c-90db-b1b05c270962-combined-ca-bundle\") pod \"keystone-db-sync-plblg\" (UID: \"1445732d-e68a-4d4c-90db-b1b05c270962\") " pod="openstack-kuttl-tests/keystone-db-sync-plblg" Dec 03 11:44:07 crc kubenswrapper[5052]: I1203 11:44:07.267401 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7nw2w\" (UniqueName: \"kubernetes.io/projected/23973edb-ef8e-4e30-a06c-308e9f48da5d-kube-api-access-7nw2w\") pod \"cinder-1bf0-account-create-update-pvvl8\" (UID: \"23973edb-ef8e-4e30-a06c-308e9f48da5d\") " pod="openstack-kuttl-tests/cinder-1bf0-account-create-update-pvvl8" Dec 03 11:44:07 crc kubenswrapper[5052]: I1203 11:44:07.267440 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-zr4fx\" (UniqueName: \"kubernetes.io/projected/c97cfd9e-1b42-494f-8eb5-206bf9791575-kube-api-access-zr4fx\") pod \"barbican-5976-account-create-update-dqgqh\" (UID: \"c97cfd9e-1b42-494f-8eb5-206bf9791575\") " pod="openstack-kuttl-tests/barbican-5976-account-create-update-dqgqh" Dec 03 11:44:07 crc kubenswrapper[5052]: I1203 11:44:07.267484 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/c97cfd9e-1b42-494f-8eb5-206bf9791575-operator-scripts\") pod \"barbican-5976-account-create-update-dqgqh\" (UID: \"c97cfd9e-1b42-494f-8eb5-206bf9791575\") " pod="openstack-kuttl-tests/barbican-5976-account-create-update-dqgqh" Dec 03 11:44:07 crc kubenswrapper[5052]: I1203 11:44:07.267530 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/1445732d-e68a-4d4c-90db-b1b05c270962-config-data\") pod \"keystone-db-sync-plblg\" (UID: \"1445732d-e68a-4d4c-90db-b1b05c270962\") " pod="openstack-kuttl-tests/keystone-db-sync-plblg" Dec 03 11:44:07 crc kubenswrapper[5052]: I1203 11:44:07.267661 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/23973edb-ef8e-4e30-a06c-308e9f48da5d-operator-scripts\") pod \"cinder-1bf0-account-create-update-pvvl8\" (UID: \"23973edb-ef8e-4e30-a06c-308e9f48da5d\") " pod="openstack-kuttl-tests/cinder-1bf0-account-create-update-pvvl8" Dec 03 11:44:07 crc kubenswrapper[5052]: I1203 11:44:07.267791 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c9dkd\" (UniqueName: \"kubernetes.io/projected/1445732d-e68a-4d4c-90db-b1b05c270962-kube-api-access-c9dkd\") pod \"keystone-db-sync-plblg\" (UID: \"1445732d-e68a-4d4c-90db-b1b05c270962\") " pod="openstack-kuttl-tests/keystone-db-sync-plblg" Dec 03 11:44:07 crc kubenswrapper[5052]: I1203 11:44:07.268757 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/c97cfd9e-1b42-494f-8eb5-206bf9791575-operator-scripts\") pod \"barbican-5976-account-create-update-dqgqh\" (UID: \"c97cfd9e-1b42-494f-8eb5-206bf9791575\") " pod="openstack-kuttl-tests/barbican-5976-account-create-update-dqgqh" Dec 03 11:44:07 crc kubenswrapper[5052]: I1203 11:44:07.294291 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-zr4fx\" (UniqueName: \"kubernetes.io/projected/c97cfd9e-1b42-494f-8eb5-206bf9791575-kube-api-access-zr4fx\") pod \"barbican-5976-account-create-update-dqgqh\" (UID: \"c97cfd9e-1b42-494f-8eb5-206bf9791575\") " pod="openstack-kuttl-tests/barbican-5976-account-create-update-dqgqh" Dec 03 11:44:07 crc kubenswrapper[5052]: I1203 11:44:07.366148 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-5976-account-create-update-dqgqh" Dec 03 11:44:07 crc kubenswrapper[5052]: I1203 11:44:07.368748 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1445732d-e68a-4d4c-90db-b1b05c270962-combined-ca-bundle\") pod \"keystone-db-sync-plblg\" (UID: \"1445732d-e68a-4d4c-90db-b1b05c270962\") " pod="openstack-kuttl-tests/keystone-db-sync-plblg" Dec 03 11:44:07 crc kubenswrapper[5052]: I1203 11:44:07.368791 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-7nw2w\" (UniqueName: \"kubernetes.io/projected/23973edb-ef8e-4e30-a06c-308e9f48da5d-kube-api-access-7nw2w\") pod \"cinder-1bf0-account-create-update-pvvl8\" (UID: \"23973edb-ef8e-4e30-a06c-308e9f48da5d\") " pod="openstack-kuttl-tests/cinder-1bf0-account-create-update-pvvl8" Dec 03 11:44:07 crc kubenswrapper[5052]: I1203 11:44:07.368835 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wmsm9\" (UniqueName: \"kubernetes.io/projected/7ebea5de-22d5-4950-a7f3-32d980eaba12-kube-api-access-wmsm9\") pod \"neutron-db-create-n5v7p\" (UID: \"7ebea5de-22d5-4950-a7f3-32d980eaba12\") " pod="openstack-kuttl-tests/neutron-db-create-n5v7p" Dec 03 11:44:07 crc kubenswrapper[5052]: I1203 11:44:07.368857 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/7ebea5de-22d5-4950-a7f3-32d980eaba12-operator-scripts\") pod \"neutron-db-create-n5v7p\" (UID: \"7ebea5de-22d5-4950-a7f3-32d980eaba12\") " pod="openstack-kuttl-tests/neutron-db-create-n5v7p" Dec 03 11:44:07 crc kubenswrapper[5052]: I1203 11:44:07.368876 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/1445732d-e68a-4d4c-90db-b1b05c270962-config-data\") pod \"keystone-db-sync-plblg\" (UID: \"1445732d-e68a-4d4c-90db-b1b05c270962\") " pod="openstack-kuttl-tests/keystone-db-sync-plblg" Dec 03 11:44:07 crc kubenswrapper[5052]: I1203 11:44:07.368909 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/23973edb-ef8e-4e30-a06c-308e9f48da5d-operator-scripts\") pod \"cinder-1bf0-account-create-update-pvvl8\" (UID: \"23973edb-ef8e-4e30-a06c-308e9f48da5d\") " pod="openstack-kuttl-tests/cinder-1bf0-account-create-update-pvvl8" Dec 03 11:44:07 crc kubenswrapper[5052]: I1203 11:44:07.368947 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-c9dkd\" (UniqueName: \"kubernetes.io/projected/1445732d-e68a-4d4c-90db-b1b05c270962-kube-api-access-c9dkd\") pod \"keystone-db-sync-plblg\" (UID: \"1445732d-e68a-4d4c-90db-b1b05c270962\") " pod="openstack-kuttl-tests/keystone-db-sync-plblg" Dec 03 11:44:07 crc kubenswrapper[5052]: I1203 11:44:07.370959 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/23973edb-ef8e-4e30-a06c-308e9f48da5d-operator-scripts\") pod \"cinder-1bf0-account-create-update-pvvl8\" (UID: \"23973edb-ef8e-4e30-a06c-308e9f48da5d\") " pod="openstack-kuttl-tests/cinder-1bf0-account-create-update-pvvl8" Dec 03 11:44:07 crc kubenswrapper[5052]: I1203 11:44:07.373378 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1445732d-e68a-4d4c-90db-b1b05c270962-combined-ca-bundle\") pod \"keystone-db-sync-plblg\" (UID: \"1445732d-e68a-4d4c-90db-b1b05c270962\") " pod="openstack-kuttl-tests/keystone-db-sync-plblg" Dec 03 11:44:07 crc kubenswrapper[5052]: I1203 11:44:07.373884 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/1445732d-e68a-4d4c-90db-b1b05c270962-config-data\") pod \"keystone-db-sync-plblg\" (UID: \"1445732d-e68a-4d4c-90db-b1b05c270962\") " pod="openstack-kuttl-tests/keystone-db-sync-plblg" Dec 03 11:44:07 crc kubenswrapper[5052]: I1203 11:44:07.391687 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-c9dkd\" (UniqueName: \"kubernetes.io/projected/1445732d-e68a-4d4c-90db-b1b05c270962-kube-api-access-c9dkd\") pod \"keystone-db-sync-plblg\" (UID: \"1445732d-e68a-4d4c-90db-b1b05c270962\") " pod="openstack-kuttl-tests/keystone-db-sync-plblg" Dec 03 11:44:07 crc kubenswrapper[5052]: I1203 11:44:07.394160 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-7nw2w\" (UniqueName: \"kubernetes.io/projected/23973edb-ef8e-4e30-a06c-308e9f48da5d-kube-api-access-7nw2w\") pod \"cinder-1bf0-account-create-update-pvvl8\" (UID: \"23973edb-ef8e-4e30-a06c-308e9f48da5d\") " pod="openstack-kuttl-tests/cinder-1bf0-account-create-update-pvvl8" Dec 03 11:44:07 crc kubenswrapper[5052]: I1203 11:44:07.470695 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-wmsm9\" (UniqueName: \"kubernetes.io/projected/7ebea5de-22d5-4950-a7f3-32d980eaba12-kube-api-access-wmsm9\") pod \"neutron-db-create-n5v7p\" (UID: \"7ebea5de-22d5-4950-a7f3-32d980eaba12\") " pod="openstack-kuttl-tests/neutron-db-create-n5v7p" Dec 03 11:44:07 crc kubenswrapper[5052]: I1203 11:44:07.470747 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/7ebea5de-22d5-4950-a7f3-32d980eaba12-operator-scripts\") pod \"neutron-db-create-n5v7p\" (UID: \"7ebea5de-22d5-4950-a7f3-32d980eaba12\") " pod="openstack-kuttl-tests/neutron-db-create-n5v7p" Dec 03 11:44:07 crc kubenswrapper[5052]: I1203 11:44:07.471439 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/7ebea5de-22d5-4950-a7f3-32d980eaba12-operator-scripts\") pod \"neutron-db-create-n5v7p\" (UID: \"7ebea5de-22d5-4950-a7f3-32d980eaba12\") " pod="openstack-kuttl-tests/neutron-db-create-n5v7p" Dec 03 11:44:07 crc kubenswrapper[5052]: I1203 11:44:07.486906 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-db-sync-plblg" Dec 03 11:44:07 crc kubenswrapper[5052]: I1203 11:44:07.493283 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-wmsm9\" (UniqueName: \"kubernetes.io/projected/7ebea5de-22d5-4950-a7f3-32d980eaba12-kube-api-access-wmsm9\") pod \"neutron-db-create-n5v7p\" (UID: \"7ebea5de-22d5-4950-a7f3-32d980eaba12\") " pod="openstack-kuttl-tests/neutron-db-create-n5v7p" Dec 03 11:44:07 crc kubenswrapper[5052]: I1203 11:44:07.562033 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-1bf0-account-create-update-pvvl8" Dec 03 11:44:07 crc kubenswrapper[5052]: I1203 11:44:07.580731 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-db-create-n5v7p" Dec 03 11:44:07 crc kubenswrapper[5052]: I1203 11:44:07.594349 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-18a7-account-create-update-n4q4t"] Dec 03 11:44:07 crc kubenswrapper[5052]: I1203 11:44:07.617257 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-db-create-57t99"] Dec 03 11:44:07 crc kubenswrapper[5052]: I1203 11:44:07.745324 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-db-create-hvzcc"] Dec 03 11:44:07 crc kubenswrapper[5052]: I1203 11:44:07.919122 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-5976-account-create-update-dqgqh"] Dec 03 11:44:07 crc kubenswrapper[5052]: W1203 11:44:07.933459 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc97cfd9e_1b42_494f_8eb5_206bf9791575.slice/crio-bf105d6b652ecdb8c6f2685ce95eb98cfea7b21f9b44120e0ac17a5d07360de4 WatchSource:0}: Error finding container bf105d6b652ecdb8c6f2685ce95eb98cfea7b21f9b44120e0ac17a5d07360de4: Status 404 returned error can't find the container with id bf105d6b652ecdb8c6f2685ce95eb98cfea7b21f9b44120e0ac17a5d07360de4 Dec 03 11:44:08 crc kubenswrapper[5052]: I1203 11:44:08.016302 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-db-create-hvzcc" event={"ID":"0c6c4cfd-0973-4ca6-beca-147213271fe3","Type":"ContainerStarted","Data":"5b80f1c14a626985c6347c0075cc979c802721e90d65a916d82a9684fb6d56e4"} Dec 03 11:44:08 crc kubenswrapper[5052]: I1203 11:44:08.019071 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-db-create-57t99" event={"ID":"d4c5312a-6993-40de-b9af-bbe3820ae84a","Type":"ContainerStarted","Data":"5e4ba37f3dc718fe1333c37c5dbab54711be06535a90da3c06591ab9c24f200c"} Dec 03 11:44:08 crc kubenswrapper[5052]: I1203 11:44:08.019108 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-db-create-57t99" event={"ID":"d4c5312a-6993-40de-b9af-bbe3820ae84a","Type":"ContainerStarted","Data":"ffbeefbf50ccbf9516fa6dd2eebb83a74446ed901a5cda36911d9f03796225c3"} Dec 03 11:44:08 crc kubenswrapper[5052]: I1203 11:44:08.032171 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-db-create-n5v7p"] Dec 03 11:44:08 crc kubenswrapper[5052]: I1203 11:44:08.034378 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-5976-account-create-update-dqgqh" event={"ID":"c97cfd9e-1b42-494f-8eb5-206bf9791575","Type":"ContainerStarted","Data":"bf105d6b652ecdb8c6f2685ce95eb98cfea7b21f9b44120e0ac17a5d07360de4"} Dec 03 11:44:08 crc kubenswrapper[5052]: I1203 11:44:08.042312 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-db-sync-plblg"] Dec 03 11:44:08 crc kubenswrapper[5052]: I1203 11:44:08.044894 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-18a7-account-create-update-n4q4t" event={"ID":"dc73f58f-4601-4b36-92b1-707a5779c99e","Type":"ContainerStarted","Data":"58159cdbdb384b445bbfc44b06917784c26a4f04d2b5e2146d782963eab0eab7"} Dec 03 11:44:08 crc kubenswrapper[5052]: I1203 11:44:08.044954 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-18a7-account-create-update-n4q4t" event={"ID":"dc73f58f-4601-4b36-92b1-707a5779c99e","Type":"ContainerStarted","Data":"86c2ed53a1ee63236dcf17673741c98216f7e5110552d45a27febb5182207d2b"} Dec 03 11:44:08 crc kubenswrapper[5052]: I1203 11:44:08.078070 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/cinder-db-create-57t99" podStartSLOduration=2.078046276 podStartE2EDuration="2.078046276s" podCreationTimestamp="2025-12-03 11:44:06 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:44:08.047913597 +0000 UTC m=+1438.636155361" watchObservedRunningTime="2025-12-03 11:44:08.078046276 +0000 UTC m=+1438.666288050" Dec 03 11:44:08 crc kubenswrapper[5052]: I1203 11:44:08.085619 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/neutron-18a7-account-create-update-n4q4t" podStartSLOduration=2.0855966439999998 podStartE2EDuration="2.085596644s" podCreationTimestamp="2025-12-03 11:44:06 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:44:08.074354779 +0000 UTC m=+1438.662596543" watchObservedRunningTime="2025-12-03 11:44:08.085596644 +0000 UTC m=+1438.673838418" Dec 03 11:44:08 crc kubenswrapper[5052]: I1203 11:44:08.190738 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-1bf0-account-create-update-pvvl8"] Dec 03 11:44:08 crc kubenswrapper[5052]: W1203 11:44:08.201479 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod23973edb_ef8e_4e30_a06c_308e9f48da5d.slice/crio-288237b95af31cc1d6b9759b01f73ad7a2b704c87105030b2b70a7d860741414 WatchSource:0}: Error finding container 288237b95af31cc1d6b9759b01f73ad7a2b704c87105030b2b70a7d860741414: Status 404 returned error can't find the container with id 288237b95af31cc1d6b9759b01f73ad7a2b704c87105030b2b70a7d860741414 Dec 03 11:44:09 crc kubenswrapper[5052]: I1203 11:44:09.056012 5052 generic.go:334] "Generic (PLEG): container finished" podID="23973edb-ef8e-4e30-a06c-308e9f48da5d" containerID="49cca97ca89a76c55abbbc2cf6873647427b19a37272e11075513a988efda003" exitCode=0 Dec 03 11:44:09 crc kubenswrapper[5052]: I1203 11:44:09.056124 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-1bf0-account-create-update-pvvl8" event={"ID":"23973edb-ef8e-4e30-a06c-308e9f48da5d","Type":"ContainerDied","Data":"49cca97ca89a76c55abbbc2cf6873647427b19a37272e11075513a988efda003"} Dec 03 11:44:09 crc kubenswrapper[5052]: I1203 11:44:09.056390 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-1bf0-account-create-update-pvvl8" event={"ID":"23973edb-ef8e-4e30-a06c-308e9f48da5d","Type":"ContainerStarted","Data":"288237b95af31cc1d6b9759b01f73ad7a2b704c87105030b2b70a7d860741414"} Dec 03 11:44:09 crc kubenswrapper[5052]: I1203 11:44:09.058242 5052 generic.go:334] "Generic (PLEG): container finished" podID="0c6c4cfd-0973-4ca6-beca-147213271fe3" containerID="57ac6bf32578c2c473e847077b4327901f57601c65faea43ff20034734be5d19" exitCode=0 Dec 03 11:44:09 crc kubenswrapper[5052]: I1203 11:44:09.058320 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-db-create-hvzcc" event={"ID":"0c6c4cfd-0973-4ca6-beca-147213271fe3","Type":"ContainerDied","Data":"57ac6bf32578c2c473e847077b4327901f57601c65faea43ff20034734be5d19"} Dec 03 11:44:09 crc kubenswrapper[5052]: I1203 11:44:09.059886 5052 generic.go:334] "Generic (PLEG): container finished" podID="d4c5312a-6993-40de-b9af-bbe3820ae84a" containerID="5e4ba37f3dc718fe1333c37c5dbab54711be06535a90da3c06591ab9c24f200c" exitCode=0 Dec 03 11:44:09 crc kubenswrapper[5052]: I1203 11:44:09.059951 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-db-create-57t99" event={"ID":"d4c5312a-6993-40de-b9af-bbe3820ae84a","Type":"ContainerDied","Data":"5e4ba37f3dc718fe1333c37c5dbab54711be06535a90da3c06591ab9c24f200c"} Dec 03 11:44:09 crc kubenswrapper[5052]: I1203 11:44:09.062108 5052 generic.go:334] "Generic (PLEG): container finished" podID="c97cfd9e-1b42-494f-8eb5-206bf9791575" containerID="decc6b94b55978b5d80c7e0c6d2f7fe37d1a02b9b0af94ffa4573365f43e6d12" exitCode=0 Dec 03 11:44:09 crc kubenswrapper[5052]: I1203 11:44:09.062166 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-5976-account-create-update-dqgqh" event={"ID":"c97cfd9e-1b42-494f-8eb5-206bf9791575","Type":"ContainerDied","Data":"decc6b94b55978b5d80c7e0c6d2f7fe37d1a02b9b0af94ffa4573365f43e6d12"} Dec 03 11:44:09 crc kubenswrapper[5052]: I1203 11:44:09.064055 5052 generic.go:334] "Generic (PLEG): container finished" podID="dc73f58f-4601-4b36-92b1-707a5779c99e" containerID="58159cdbdb384b445bbfc44b06917784c26a4f04d2b5e2146d782963eab0eab7" exitCode=0 Dec 03 11:44:09 crc kubenswrapper[5052]: I1203 11:44:09.064121 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-18a7-account-create-update-n4q4t" event={"ID":"dc73f58f-4601-4b36-92b1-707a5779c99e","Type":"ContainerDied","Data":"58159cdbdb384b445bbfc44b06917784c26a4f04d2b5e2146d782963eab0eab7"} Dec 03 11:44:09 crc kubenswrapper[5052]: I1203 11:44:09.065720 5052 generic.go:334] "Generic (PLEG): container finished" podID="7ebea5de-22d5-4950-a7f3-32d980eaba12" containerID="b6e79fea7226f005ad352e5ed4669dd04e2f833341d603135672d32a6a08851b" exitCode=0 Dec 03 11:44:09 crc kubenswrapper[5052]: I1203 11:44:09.065800 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-db-create-n5v7p" event={"ID":"7ebea5de-22d5-4950-a7f3-32d980eaba12","Type":"ContainerDied","Data":"b6e79fea7226f005ad352e5ed4669dd04e2f833341d603135672d32a6a08851b"} Dec 03 11:44:09 crc kubenswrapper[5052]: I1203 11:44:09.065837 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-db-create-n5v7p" event={"ID":"7ebea5de-22d5-4950-a7f3-32d980eaba12","Type":"ContainerStarted","Data":"5d46d3cd035d6c5d6737cae53018ce2ceb5b25b7c89ea9aed578bbfe741e033a"} Dec 03 11:44:09 crc kubenswrapper[5052]: I1203 11:44:09.067737 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-db-sync-plblg" event={"ID":"1445732d-e68a-4d4c-90db-b1b05c270962","Type":"ContainerStarted","Data":"0d7ccf70d7dc58060ea283ad2f6c964fb25e52537fe4fef671cdf5a0610a07dc"} Dec 03 11:44:09 crc kubenswrapper[5052]: I1203 11:44:09.067762 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-db-sync-plblg" event={"ID":"1445732d-e68a-4d4c-90db-b1b05c270962","Type":"ContainerStarted","Data":"1d8243d5a3db74160553ab58815bd4b8b36c219727d18a9d42b498078d9c9a6a"} Dec 03 11:44:09 crc kubenswrapper[5052]: I1203 11:44:09.185219 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/keystone-db-sync-plblg" podStartSLOduration=2.185196172 podStartE2EDuration="2.185196172s" podCreationTimestamp="2025-12-03 11:44:07 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:44:09.180087335 +0000 UTC m=+1439.768329099" watchObservedRunningTime="2025-12-03 11:44:09.185196172 +0000 UTC m=+1439.773437936" Dec 03 11:44:10 crc kubenswrapper[5052]: I1203 11:44:10.450437 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-db-create-hvzcc" Dec 03 11:44:10 crc kubenswrapper[5052]: I1203 11:44:10.645641 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/0c6c4cfd-0973-4ca6-beca-147213271fe3-operator-scripts\") pod \"0c6c4cfd-0973-4ca6-beca-147213271fe3\" (UID: \"0c6c4cfd-0973-4ca6-beca-147213271fe3\") " Dec 03 11:44:10 crc kubenswrapper[5052]: I1203 11:44:10.645850 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fgbq2\" (UniqueName: \"kubernetes.io/projected/0c6c4cfd-0973-4ca6-beca-147213271fe3-kube-api-access-fgbq2\") pod \"0c6c4cfd-0973-4ca6-beca-147213271fe3\" (UID: \"0c6c4cfd-0973-4ca6-beca-147213271fe3\") " Dec 03 11:44:10 crc kubenswrapper[5052]: I1203 11:44:10.652421 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0c6c4cfd-0973-4ca6-beca-147213271fe3-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "0c6c4cfd-0973-4ca6-beca-147213271fe3" (UID: "0c6c4cfd-0973-4ca6-beca-147213271fe3"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:44:10 crc kubenswrapper[5052]: I1203 11:44:10.658568 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0c6c4cfd-0973-4ca6-beca-147213271fe3-kube-api-access-fgbq2" (OuterVolumeSpecName: "kube-api-access-fgbq2") pod "0c6c4cfd-0973-4ca6-beca-147213271fe3" (UID: "0c6c4cfd-0973-4ca6-beca-147213271fe3"). InnerVolumeSpecName "kube-api-access-fgbq2". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:44:10 crc kubenswrapper[5052]: I1203 11:44:10.747264 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/0c6c4cfd-0973-4ca6-beca-147213271fe3-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:44:10 crc kubenswrapper[5052]: I1203 11:44:10.747629 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-fgbq2\" (UniqueName: \"kubernetes.io/projected/0c6c4cfd-0973-4ca6-beca-147213271fe3-kube-api-access-fgbq2\") on node \"crc\" DevicePath \"\"" Dec 03 11:44:10 crc kubenswrapper[5052]: I1203 11:44:10.782419 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-db-create-n5v7p" Dec 03 11:44:10 crc kubenswrapper[5052]: I1203 11:44:10.788524 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-5976-account-create-update-dqgqh" Dec 03 11:44:10 crc kubenswrapper[5052]: I1203 11:44:10.796459 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-18a7-account-create-update-n4q4t" Dec 03 11:44:10 crc kubenswrapper[5052]: I1203 11:44:10.812602 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-1bf0-account-create-update-pvvl8" Dec 03 11:44:10 crc kubenswrapper[5052]: I1203 11:44:10.833447 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-db-create-57t99" Dec 03 11:44:10 crc kubenswrapper[5052]: I1203 11:44:10.950411 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/7ebea5de-22d5-4950-a7f3-32d980eaba12-operator-scripts\") pod \"7ebea5de-22d5-4950-a7f3-32d980eaba12\" (UID: \"7ebea5de-22d5-4950-a7f3-32d980eaba12\") " Dec 03 11:44:10 crc kubenswrapper[5052]: I1203 11:44:10.950456 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m4dgn\" (UniqueName: \"kubernetes.io/projected/dc73f58f-4601-4b36-92b1-707a5779c99e-kube-api-access-m4dgn\") pod \"dc73f58f-4601-4b36-92b1-707a5779c99e\" (UID: \"dc73f58f-4601-4b36-92b1-707a5779c99e\") " Dec 03 11:44:10 crc kubenswrapper[5052]: I1203 11:44:10.950488 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7nw2w\" (UniqueName: \"kubernetes.io/projected/23973edb-ef8e-4e30-a06c-308e9f48da5d-kube-api-access-7nw2w\") pod \"23973edb-ef8e-4e30-a06c-308e9f48da5d\" (UID: \"23973edb-ef8e-4e30-a06c-308e9f48da5d\") " Dec 03 11:44:10 crc kubenswrapper[5052]: I1203 11:44:10.950553 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zr4fx\" (UniqueName: \"kubernetes.io/projected/c97cfd9e-1b42-494f-8eb5-206bf9791575-kube-api-access-zr4fx\") pod \"c97cfd9e-1b42-494f-8eb5-206bf9791575\" (UID: \"c97cfd9e-1b42-494f-8eb5-206bf9791575\") " Dec 03 11:44:10 crc kubenswrapper[5052]: I1203 11:44:10.950608 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/c97cfd9e-1b42-494f-8eb5-206bf9791575-operator-scripts\") pod \"c97cfd9e-1b42-494f-8eb5-206bf9791575\" (UID: \"c97cfd9e-1b42-494f-8eb5-206bf9791575\") " Dec 03 11:44:10 crc kubenswrapper[5052]: I1203 11:44:10.950645 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wmsm9\" (UniqueName: \"kubernetes.io/projected/7ebea5de-22d5-4950-a7f3-32d980eaba12-kube-api-access-wmsm9\") pod \"7ebea5de-22d5-4950-a7f3-32d980eaba12\" (UID: \"7ebea5de-22d5-4950-a7f3-32d980eaba12\") " Dec 03 11:44:10 crc kubenswrapper[5052]: I1203 11:44:10.950674 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/23973edb-ef8e-4e30-a06c-308e9f48da5d-operator-scripts\") pod \"23973edb-ef8e-4e30-a06c-308e9f48da5d\" (UID: \"23973edb-ef8e-4e30-a06c-308e9f48da5d\") " Dec 03 11:44:10 crc kubenswrapper[5052]: I1203 11:44:10.950737 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/d4c5312a-6993-40de-b9af-bbe3820ae84a-operator-scripts\") pod \"d4c5312a-6993-40de-b9af-bbe3820ae84a\" (UID: \"d4c5312a-6993-40de-b9af-bbe3820ae84a\") " Dec 03 11:44:10 crc kubenswrapper[5052]: I1203 11:44:10.950752 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/dc73f58f-4601-4b36-92b1-707a5779c99e-operator-scripts\") pod \"dc73f58f-4601-4b36-92b1-707a5779c99e\" (UID: \"dc73f58f-4601-4b36-92b1-707a5779c99e\") " Dec 03 11:44:10 crc kubenswrapper[5052]: I1203 11:44:10.950771 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7442s\" (UniqueName: \"kubernetes.io/projected/d4c5312a-6993-40de-b9af-bbe3820ae84a-kube-api-access-7442s\") pod \"d4c5312a-6993-40de-b9af-bbe3820ae84a\" (UID: \"d4c5312a-6993-40de-b9af-bbe3820ae84a\") " Dec 03 11:44:10 crc kubenswrapper[5052]: I1203 11:44:10.951034 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7ebea5de-22d5-4950-a7f3-32d980eaba12-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "7ebea5de-22d5-4950-a7f3-32d980eaba12" (UID: "7ebea5de-22d5-4950-a7f3-32d980eaba12"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:44:10 crc kubenswrapper[5052]: I1203 11:44:10.951207 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/7ebea5de-22d5-4950-a7f3-32d980eaba12-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:44:10 crc kubenswrapper[5052]: I1203 11:44:10.951787 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dc73f58f-4601-4b36-92b1-707a5779c99e-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "dc73f58f-4601-4b36-92b1-707a5779c99e" (UID: "dc73f58f-4601-4b36-92b1-707a5779c99e"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:44:10 crc kubenswrapper[5052]: I1203 11:44:10.951989 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c97cfd9e-1b42-494f-8eb5-206bf9791575-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "c97cfd9e-1b42-494f-8eb5-206bf9791575" (UID: "c97cfd9e-1b42-494f-8eb5-206bf9791575"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:44:10 crc kubenswrapper[5052]: I1203 11:44:10.952178 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/23973edb-ef8e-4e30-a06c-308e9f48da5d-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "23973edb-ef8e-4e30-a06c-308e9f48da5d" (UID: "23973edb-ef8e-4e30-a06c-308e9f48da5d"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:44:10 crc kubenswrapper[5052]: I1203 11:44:10.952959 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d4c5312a-6993-40de-b9af-bbe3820ae84a-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "d4c5312a-6993-40de-b9af-bbe3820ae84a" (UID: "d4c5312a-6993-40de-b9af-bbe3820ae84a"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:44:10 crc kubenswrapper[5052]: I1203 11:44:10.955400 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dc73f58f-4601-4b36-92b1-707a5779c99e-kube-api-access-m4dgn" (OuterVolumeSpecName: "kube-api-access-m4dgn") pod "dc73f58f-4601-4b36-92b1-707a5779c99e" (UID: "dc73f58f-4601-4b36-92b1-707a5779c99e"). InnerVolumeSpecName "kube-api-access-m4dgn". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:44:10 crc kubenswrapper[5052]: I1203 11:44:10.955506 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/23973edb-ef8e-4e30-a06c-308e9f48da5d-kube-api-access-7nw2w" (OuterVolumeSpecName: "kube-api-access-7nw2w") pod "23973edb-ef8e-4e30-a06c-308e9f48da5d" (UID: "23973edb-ef8e-4e30-a06c-308e9f48da5d"). InnerVolumeSpecName "kube-api-access-7nw2w". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:44:10 crc kubenswrapper[5052]: I1203 11:44:10.955617 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c97cfd9e-1b42-494f-8eb5-206bf9791575-kube-api-access-zr4fx" (OuterVolumeSpecName: "kube-api-access-zr4fx") pod "c97cfd9e-1b42-494f-8eb5-206bf9791575" (UID: "c97cfd9e-1b42-494f-8eb5-206bf9791575"). InnerVolumeSpecName "kube-api-access-zr4fx". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:44:10 crc kubenswrapper[5052]: I1203 11:44:10.956048 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7ebea5de-22d5-4950-a7f3-32d980eaba12-kube-api-access-wmsm9" (OuterVolumeSpecName: "kube-api-access-wmsm9") pod "7ebea5de-22d5-4950-a7f3-32d980eaba12" (UID: "7ebea5de-22d5-4950-a7f3-32d980eaba12"). InnerVolumeSpecName "kube-api-access-wmsm9". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:44:10 crc kubenswrapper[5052]: I1203 11:44:10.958672 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d4c5312a-6993-40de-b9af-bbe3820ae84a-kube-api-access-7442s" (OuterVolumeSpecName: "kube-api-access-7442s") pod "d4c5312a-6993-40de-b9af-bbe3820ae84a" (UID: "d4c5312a-6993-40de-b9af-bbe3820ae84a"). InnerVolumeSpecName "kube-api-access-7442s". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:44:11 crc kubenswrapper[5052]: I1203 11:44:11.052647 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-m4dgn\" (UniqueName: \"kubernetes.io/projected/dc73f58f-4601-4b36-92b1-707a5779c99e-kube-api-access-m4dgn\") on node \"crc\" DevicePath \"\"" Dec 03 11:44:11 crc kubenswrapper[5052]: I1203 11:44:11.052685 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-7nw2w\" (UniqueName: \"kubernetes.io/projected/23973edb-ef8e-4e30-a06c-308e9f48da5d-kube-api-access-7nw2w\") on node \"crc\" DevicePath \"\"" Dec 03 11:44:11 crc kubenswrapper[5052]: I1203 11:44:11.052696 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-zr4fx\" (UniqueName: \"kubernetes.io/projected/c97cfd9e-1b42-494f-8eb5-206bf9791575-kube-api-access-zr4fx\") on node \"crc\" DevicePath \"\"" Dec 03 11:44:11 crc kubenswrapper[5052]: I1203 11:44:11.052706 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/c97cfd9e-1b42-494f-8eb5-206bf9791575-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:44:11 crc kubenswrapper[5052]: I1203 11:44:11.052716 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-wmsm9\" (UniqueName: \"kubernetes.io/projected/7ebea5de-22d5-4950-a7f3-32d980eaba12-kube-api-access-wmsm9\") on node \"crc\" DevicePath \"\"" Dec 03 11:44:11 crc kubenswrapper[5052]: I1203 11:44:11.052724 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/23973edb-ef8e-4e30-a06c-308e9f48da5d-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:44:11 crc kubenswrapper[5052]: I1203 11:44:11.052733 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/d4c5312a-6993-40de-b9af-bbe3820ae84a-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:44:11 crc kubenswrapper[5052]: I1203 11:44:11.052742 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/dc73f58f-4601-4b36-92b1-707a5779c99e-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:44:11 crc kubenswrapper[5052]: I1203 11:44:11.052752 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-7442s\" (UniqueName: \"kubernetes.io/projected/d4c5312a-6993-40de-b9af-bbe3820ae84a-kube-api-access-7442s\") on node \"crc\" DevicePath \"\"" Dec 03 11:44:11 crc kubenswrapper[5052]: I1203 11:44:11.116915 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-db-create-57t99" event={"ID":"d4c5312a-6993-40de-b9af-bbe3820ae84a","Type":"ContainerDied","Data":"ffbeefbf50ccbf9516fa6dd2eebb83a74446ed901a5cda36911d9f03796225c3"} Dec 03 11:44:11 crc kubenswrapper[5052]: I1203 11:44:11.116964 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ffbeefbf50ccbf9516fa6dd2eebb83a74446ed901a5cda36911d9f03796225c3" Dec 03 11:44:11 crc kubenswrapper[5052]: I1203 11:44:11.117029 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-db-create-57t99" Dec 03 11:44:11 crc kubenswrapper[5052]: I1203 11:44:11.120264 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-5976-account-create-update-dqgqh" event={"ID":"c97cfd9e-1b42-494f-8eb5-206bf9791575","Type":"ContainerDied","Data":"bf105d6b652ecdb8c6f2685ce95eb98cfea7b21f9b44120e0ac17a5d07360de4"} Dec 03 11:44:11 crc kubenswrapper[5052]: I1203 11:44:11.120322 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bf105d6b652ecdb8c6f2685ce95eb98cfea7b21f9b44120e0ac17a5d07360de4" Dec 03 11:44:11 crc kubenswrapper[5052]: I1203 11:44:11.120278 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-5976-account-create-update-dqgqh" Dec 03 11:44:11 crc kubenswrapper[5052]: I1203 11:44:11.122061 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-18a7-account-create-update-n4q4t" Dec 03 11:44:11 crc kubenswrapper[5052]: I1203 11:44:11.122060 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-18a7-account-create-update-n4q4t" event={"ID":"dc73f58f-4601-4b36-92b1-707a5779c99e","Type":"ContainerDied","Data":"86c2ed53a1ee63236dcf17673741c98216f7e5110552d45a27febb5182207d2b"} Dec 03 11:44:11 crc kubenswrapper[5052]: I1203 11:44:11.122161 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="86c2ed53a1ee63236dcf17673741c98216f7e5110552d45a27febb5182207d2b" Dec 03 11:44:11 crc kubenswrapper[5052]: I1203 11:44:11.123975 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-db-create-n5v7p" event={"ID":"7ebea5de-22d5-4950-a7f3-32d980eaba12","Type":"ContainerDied","Data":"5d46d3cd035d6c5d6737cae53018ce2ceb5b25b7c89ea9aed578bbfe741e033a"} Dec 03 11:44:11 crc kubenswrapper[5052]: I1203 11:44:11.124012 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5d46d3cd035d6c5d6737cae53018ce2ceb5b25b7c89ea9aed578bbfe741e033a" Dec 03 11:44:11 crc kubenswrapper[5052]: I1203 11:44:11.124011 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-db-create-n5v7p" Dec 03 11:44:11 crc kubenswrapper[5052]: I1203 11:44:11.130747 5052 generic.go:334] "Generic (PLEG): container finished" podID="1445732d-e68a-4d4c-90db-b1b05c270962" containerID="0d7ccf70d7dc58060ea283ad2f6c964fb25e52537fe4fef671cdf5a0610a07dc" exitCode=0 Dec 03 11:44:11 crc kubenswrapper[5052]: I1203 11:44:11.130795 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-db-sync-plblg" event={"ID":"1445732d-e68a-4d4c-90db-b1b05c270962","Type":"ContainerDied","Data":"0d7ccf70d7dc58060ea283ad2f6c964fb25e52537fe4fef671cdf5a0610a07dc"} Dec 03 11:44:11 crc kubenswrapper[5052]: I1203 11:44:11.132984 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-1bf0-account-create-update-pvvl8" event={"ID":"23973edb-ef8e-4e30-a06c-308e9f48da5d","Type":"ContainerDied","Data":"288237b95af31cc1d6b9759b01f73ad7a2b704c87105030b2b70a7d860741414"} Dec 03 11:44:11 crc kubenswrapper[5052]: I1203 11:44:11.133030 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-1bf0-account-create-update-pvvl8" Dec 03 11:44:11 crc kubenswrapper[5052]: I1203 11:44:11.133056 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="288237b95af31cc1d6b9759b01f73ad7a2b704c87105030b2b70a7d860741414" Dec 03 11:44:11 crc kubenswrapper[5052]: I1203 11:44:11.139186 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-db-create-hvzcc" event={"ID":"0c6c4cfd-0973-4ca6-beca-147213271fe3","Type":"ContainerDied","Data":"5b80f1c14a626985c6347c0075cc979c802721e90d65a916d82a9684fb6d56e4"} Dec 03 11:44:11 crc kubenswrapper[5052]: I1203 11:44:11.139226 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5b80f1c14a626985c6347c0075cc979c802721e90d65a916d82a9684fb6d56e4" Dec 03 11:44:11 crc kubenswrapper[5052]: I1203 11:44:11.139309 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-db-create-hvzcc" Dec 03 11:44:12 crc kubenswrapper[5052]: I1203 11:44:12.465349 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-db-sync-plblg" Dec 03 11:44:12 crc kubenswrapper[5052]: I1203 11:44:12.574008 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c9dkd\" (UniqueName: \"kubernetes.io/projected/1445732d-e68a-4d4c-90db-b1b05c270962-kube-api-access-c9dkd\") pod \"1445732d-e68a-4d4c-90db-b1b05c270962\" (UID: \"1445732d-e68a-4d4c-90db-b1b05c270962\") " Dec 03 11:44:12 crc kubenswrapper[5052]: I1203 11:44:12.574148 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1445732d-e68a-4d4c-90db-b1b05c270962-combined-ca-bundle\") pod \"1445732d-e68a-4d4c-90db-b1b05c270962\" (UID: \"1445732d-e68a-4d4c-90db-b1b05c270962\") " Dec 03 11:44:12 crc kubenswrapper[5052]: I1203 11:44:12.574279 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/1445732d-e68a-4d4c-90db-b1b05c270962-config-data\") pod \"1445732d-e68a-4d4c-90db-b1b05c270962\" (UID: \"1445732d-e68a-4d4c-90db-b1b05c270962\") " Dec 03 11:44:12 crc kubenswrapper[5052]: I1203 11:44:12.580687 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1445732d-e68a-4d4c-90db-b1b05c270962-kube-api-access-c9dkd" (OuterVolumeSpecName: "kube-api-access-c9dkd") pod "1445732d-e68a-4d4c-90db-b1b05c270962" (UID: "1445732d-e68a-4d4c-90db-b1b05c270962"). InnerVolumeSpecName "kube-api-access-c9dkd". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:44:12 crc kubenswrapper[5052]: I1203 11:44:12.598944 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1445732d-e68a-4d4c-90db-b1b05c270962-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "1445732d-e68a-4d4c-90db-b1b05c270962" (UID: "1445732d-e68a-4d4c-90db-b1b05c270962"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:44:12 crc kubenswrapper[5052]: I1203 11:44:12.617237 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1445732d-e68a-4d4c-90db-b1b05c270962-config-data" (OuterVolumeSpecName: "config-data") pod "1445732d-e68a-4d4c-90db-b1b05c270962" (UID: "1445732d-e68a-4d4c-90db-b1b05c270962"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:44:12 crc kubenswrapper[5052]: I1203 11:44:12.676783 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-c9dkd\" (UniqueName: \"kubernetes.io/projected/1445732d-e68a-4d4c-90db-b1b05c270962-kube-api-access-c9dkd\") on node \"crc\" DevicePath \"\"" Dec 03 11:44:12 crc kubenswrapper[5052]: I1203 11:44:12.676829 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1445732d-e68a-4d4c-90db-b1b05c270962-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:44:12 crc kubenswrapper[5052]: I1203 11:44:12.676844 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/1445732d-e68a-4d4c-90db-b1b05c270962-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:44:13 crc kubenswrapper[5052]: I1203 11:44:13.177197 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-db-sync-plblg" event={"ID":"1445732d-e68a-4d4c-90db-b1b05c270962","Type":"ContainerDied","Data":"1d8243d5a3db74160553ab58815bd4b8b36c219727d18a9d42b498078d9c9a6a"} Dec 03 11:44:13 crc kubenswrapper[5052]: I1203 11:44:13.177773 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1d8243d5a3db74160553ab58815bd4b8b36c219727d18a9d42b498078d9c9a6a" Dec 03 11:44:13 crc kubenswrapper[5052]: I1203 11:44:13.177235 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-db-sync-plblg" Dec 03 11:44:13 crc kubenswrapper[5052]: I1203 11:44:13.664503 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/keystone-bootstrap-2hjbj"] Dec 03 11:44:13 crc kubenswrapper[5052]: E1203 11:44:13.670379 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="d4c5312a-6993-40de-b9af-bbe3820ae84a" containerName="mariadb-database-create" Dec 03 11:44:13 crc kubenswrapper[5052]: I1203 11:44:13.670601 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="d4c5312a-6993-40de-b9af-bbe3820ae84a" containerName="mariadb-database-create" Dec 03 11:44:13 crc kubenswrapper[5052]: E1203 11:44:13.670618 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="c97cfd9e-1b42-494f-8eb5-206bf9791575" containerName="mariadb-account-create-update" Dec 03 11:44:13 crc kubenswrapper[5052]: I1203 11:44:13.670625 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="c97cfd9e-1b42-494f-8eb5-206bf9791575" containerName="mariadb-account-create-update" Dec 03 11:44:13 crc kubenswrapper[5052]: E1203 11:44:13.670635 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="0c6c4cfd-0973-4ca6-beca-147213271fe3" containerName="mariadb-database-create" Dec 03 11:44:13 crc kubenswrapper[5052]: I1203 11:44:13.670640 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="0c6c4cfd-0973-4ca6-beca-147213271fe3" containerName="mariadb-database-create" Dec 03 11:44:13 crc kubenswrapper[5052]: E1203 11:44:13.670660 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="23973edb-ef8e-4e30-a06c-308e9f48da5d" containerName="mariadb-account-create-update" Dec 03 11:44:13 crc kubenswrapper[5052]: I1203 11:44:13.670666 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="23973edb-ef8e-4e30-a06c-308e9f48da5d" containerName="mariadb-account-create-update" Dec 03 11:44:13 crc kubenswrapper[5052]: E1203 11:44:13.670675 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="dc73f58f-4601-4b36-92b1-707a5779c99e" containerName="mariadb-account-create-update" Dec 03 11:44:13 crc kubenswrapper[5052]: I1203 11:44:13.670681 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="dc73f58f-4601-4b36-92b1-707a5779c99e" containerName="mariadb-account-create-update" Dec 03 11:44:13 crc kubenswrapper[5052]: E1203 11:44:13.670704 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="7ebea5de-22d5-4950-a7f3-32d980eaba12" containerName="mariadb-database-create" Dec 03 11:44:13 crc kubenswrapper[5052]: I1203 11:44:13.670709 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="7ebea5de-22d5-4950-a7f3-32d980eaba12" containerName="mariadb-database-create" Dec 03 11:44:13 crc kubenswrapper[5052]: E1203 11:44:13.670723 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="1445732d-e68a-4d4c-90db-b1b05c270962" containerName="keystone-db-sync" Dec 03 11:44:13 crc kubenswrapper[5052]: I1203 11:44:13.670730 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="1445732d-e68a-4d4c-90db-b1b05c270962" containerName="keystone-db-sync" Dec 03 11:44:13 crc kubenswrapper[5052]: I1203 11:44:13.670862 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="23973edb-ef8e-4e30-a06c-308e9f48da5d" containerName="mariadb-account-create-update" Dec 03 11:44:13 crc kubenswrapper[5052]: I1203 11:44:13.670875 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="7ebea5de-22d5-4950-a7f3-32d980eaba12" containerName="mariadb-database-create" Dec 03 11:44:13 crc kubenswrapper[5052]: I1203 11:44:13.670888 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="0c6c4cfd-0973-4ca6-beca-147213271fe3" containerName="mariadb-database-create" Dec 03 11:44:13 crc kubenswrapper[5052]: I1203 11:44:13.670898 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="1445732d-e68a-4d4c-90db-b1b05c270962" containerName="keystone-db-sync" Dec 03 11:44:13 crc kubenswrapper[5052]: I1203 11:44:13.670910 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="d4c5312a-6993-40de-b9af-bbe3820ae84a" containerName="mariadb-database-create" Dec 03 11:44:13 crc kubenswrapper[5052]: I1203 11:44:13.670921 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="c97cfd9e-1b42-494f-8eb5-206bf9791575" containerName="mariadb-account-create-update" Dec 03 11:44:13 crc kubenswrapper[5052]: I1203 11:44:13.670932 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="dc73f58f-4601-4b36-92b1-707a5779c99e" containerName="mariadb-account-create-update" Dec 03 11:44:13 crc kubenswrapper[5052]: I1203 11:44:13.671539 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-bootstrap-2hjbj" Dec 03 11:44:13 crc kubenswrapper[5052]: I1203 11:44:13.677533 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone" Dec 03 11:44:13 crc kubenswrapper[5052]: I1203 11:44:13.677618 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone-config-data" Dec 03 11:44:13 crc kubenswrapper[5052]: I1203 11:44:13.677690 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone-keystone-dockercfg-4hzgj" Dec 03 11:44:13 crc kubenswrapper[5052]: I1203 11:44:13.677804 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"osp-secret" Dec 03 11:44:13 crc kubenswrapper[5052]: I1203 11:44:13.680395 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone-scripts" Dec 03 11:44:13 crc kubenswrapper[5052]: I1203 11:44:13.692798 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/b53d06f2-28c8-4c18-b9ce-0300f8b8ef06-scripts\") pod \"keystone-bootstrap-2hjbj\" (UID: \"b53d06f2-28c8-4c18-b9ce-0300f8b8ef06\") " pod="openstack-kuttl-tests/keystone-bootstrap-2hjbj" Dec 03 11:44:13 crc kubenswrapper[5052]: I1203 11:44:13.692841 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/b53d06f2-28c8-4c18-b9ce-0300f8b8ef06-fernet-keys\") pod \"keystone-bootstrap-2hjbj\" (UID: \"b53d06f2-28c8-4c18-b9ce-0300f8b8ef06\") " pod="openstack-kuttl-tests/keystone-bootstrap-2hjbj" Dec 03 11:44:13 crc kubenswrapper[5052]: I1203 11:44:13.692873 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/b53d06f2-28c8-4c18-b9ce-0300f8b8ef06-credential-keys\") pod \"keystone-bootstrap-2hjbj\" (UID: \"b53d06f2-28c8-4c18-b9ce-0300f8b8ef06\") " pod="openstack-kuttl-tests/keystone-bootstrap-2hjbj" Dec 03 11:44:13 crc kubenswrapper[5052]: I1203 11:44:13.692954 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pknlf\" (UniqueName: \"kubernetes.io/projected/b53d06f2-28c8-4c18-b9ce-0300f8b8ef06-kube-api-access-pknlf\") pod \"keystone-bootstrap-2hjbj\" (UID: \"b53d06f2-28c8-4c18-b9ce-0300f8b8ef06\") " pod="openstack-kuttl-tests/keystone-bootstrap-2hjbj" Dec 03 11:44:13 crc kubenswrapper[5052]: I1203 11:44:13.692987 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b53d06f2-28c8-4c18-b9ce-0300f8b8ef06-combined-ca-bundle\") pod \"keystone-bootstrap-2hjbj\" (UID: \"b53d06f2-28c8-4c18-b9ce-0300f8b8ef06\") " pod="openstack-kuttl-tests/keystone-bootstrap-2hjbj" Dec 03 11:44:13 crc kubenswrapper[5052]: I1203 11:44:13.693087 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/b53d06f2-28c8-4c18-b9ce-0300f8b8ef06-config-data\") pod \"keystone-bootstrap-2hjbj\" (UID: \"b53d06f2-28c8-4c18-b9ce-0300f8b8ef06\") " pod="openstack-kuttl-tests/keystone-bootstrap-2hjbj" Dec 03 11:44:13 crc kubenswrapper[5052]: I1203 11:44:13.707169 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-bootstrap-2hjbj"] Dec 03 11:44:13 crc kubenswrapper[5052]: I1203 11:44:13.776162 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 11:44:13 crc kubenswrapper[5052]: I1203 11:44:13.781322 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:44:13 crc kubenswrapper[5052]: I1203 11:44:13.786327 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-glance-dockercfg-tlrkx" Dec 03 11:44:13 crc kubenswrapper[5052]: I1203 11:44:13.786394 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-scripts" Dec 03 11:44:13 crc kubenswrapper[5052]: I1203 11:44:13.786658 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-glance-default-internal-svc" Dec 03 11:44:13 crc kubenswrapper[5052]: I1203 11:44:13.786894 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-default-internal-config-data" Dec 03 11:44:13 crc kubenswrapper[5052]: I1203 11:44:13.797046 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-pknlf\" (UniqueName: \"kubernetes.io/projected/b53d06f2-28c8-4c18-b9ce-0300f8b8ef06-kube-api-access-pknlf\") pod \"keystone-bootstrap-2hjbj\" (UID: \"b53d06f2-28c8-4c18-b9ce-0300f8b8ef06\") " pod="openstack-kuttl-tests/keystone-bootstrap-2hjbj" Dec 03 11:44:13 crc kubenswrapper[5052]: I1203 11:44:13.797104 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b53d06f2-28c8-4c18-b9ce-0300f8b8ef06-combined-ca-bundle\") pod \"keystone-bootstrap-2hjbj\" (UID: \"b53d06f2-28c8-4c18-b9ce-0300f8b8ef06\") " pod="openstack-kuttl-tests/keystone-bootstrap-2hjbj" Dec 03 11:44:13 crc kubenswrapper[5052]: I1203 11:44:13.797154 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/b53d06f2-28c8-4c18-b9ce-0300f8b8ef06-config-data\") pod \"keystone-bootstrap-2hjbj\" (UID: \"b53d06f2-28c8-4c18-b9ce-0300f8b8ef06\") " pod="openstack-kuttl-tests/keystone-bootstrap-2hjbj" Dec 03 11:44:13 crc kubenswrapper[5052]: I1203 11:44:13.797233 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/b53d06f2-28c8-4c18-b9ce-0300f8b8ef06-scripts\") pod \"keystone-bootstrap-2hjbj\" (UID: \"b53d06f2-28c8-4c18-b9ce-0300f8b8ef06\") " pod="openstack-kuttl-tests/keystone-bootstrap-2hjbj" Dec 03 11:44:13 crc kubenswrapper[5052]: I1203 11:44:13.797260 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/b53d06f2-28c8-4c18-b9ce-0300f8b8ef06-fernet-keys\") pod \"keystone-bootstrap-2hjbj\" (UID: \"b53d06f2-28c8-4c18-b9ce-0300f8b8ef06\") " pod="openstack-kuttl-tests/keystone-bootstrap-2hjbj" Dec 03 11:44:13 crc kubenswrapper[5052]: I1203 11:44:13.797297 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/b53d06f2-28c8-4c18-b9ce-0300f8b8ef06-credential-keys\") pod \"keystone-bootstrap-2hjbj\" (UID: \"b53d06f2-28c8-4c18-b9ce-0300f8b8ef06\") " pod="openstack-kuttl-tests/keystone-bootstrap-2hjbj" Dec 03 11:44:13 crc kubenswrapper[5052]: I1203 11:44:13.798896 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 11:44:13 crc kubenswrapper[5052]: I1203 11:44:13.810557 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/b53d06f2-28c8-4c18-b9ce-0300f8b8ef06-fernet-keys\") pod \"keystone-bootstrap-2hjbj\" (UID: \"b53d06f2-28c8-4c18-b9ce-0300f8b8ef06\") " pod="openstack-kuttl-tests/keystone-bootstrap-2hjbj" Dec 03 11:44:13 crc kubenswrapper[5052]: I1203 11:44:13.811278 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/b53d06f2-28c8-4c18-b9ce-0300f8b8ef06-config-data\") pod \"keystone-bootstrap-2hjbj\" (UID: \"b53d06f2-28c8-4c18-b9ce-0300f8b8ef06\") " pod="openstack-kuttl-tests/keystone-bootstrap-2hjbj" Dec 03 11:44:13 crc kubenswrapper[5052]: I1203 11:44:13.820296 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b53d06f2-28c8-4c18-b9ce-0300f8b8ef06-combined-ca-bundle\") pod \"keystone-bootstrap-2hjbj\" (UID: \"b53d06f2-28c8-4c18-b9ce-0300f8b8ef06\") " pod="openstack-kuttl-tests/keystone-bootstrap-2hjbj" Dec 03 11:44:13 crc kubenswrapper[5052]: I1203 11:44:13.821739 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/b53d06f2-28c8-4c18-b9ce-0300f8b8ef06-scripts\") pod \"keystone-bootstrap-2hjbj\" (UID: \"b53d06f2-28c8-4c18-b9ce-0300f8b8ef06\") " pod="openstack-kuttl-tests/keystone-bootstrap-2hjbj" Dec 03 11:44:13 crc kubenswrapper[5052]: I1203 11:44:13.827055 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/b53d06f2-28c8-4c18-b9ce-0300f8b8ef06-credential-keys\") pod \"keystone-bootstrap-2hjbj\" (UID: \"b53d06f2-28c8-4c18-b9ce-0300f8b8ef06\") " pod="openstack-kuttl-tests/keystone-bootstrap-2hjbj" Dec 03 11:44:13 crc kubenswrapper[5052]: I1203 11:44:13.858574 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 11:44:13 crc kubenswrapper[5052]: I1203 11:44:13.859998 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:44:13 crc kubenswrapper[5052]: I1203 11:44:13.880463 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-glance-default-public-svc" Dec 03 11:44:13 crc kubenswrapper[5052]: I1203 11:44:13.880874 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-default-external-config-data" Dec 03 11:44:13 crc kubenswrapper[5052]: I1203 11:44:13.899485 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/93c3ba7d-27d6-47cd-86a2-30a19e9e3c21-scripts\") pod \"glance-default-internal-api-0\" (UID: \"93c3ba7d-27d6-47cd-86a2-30a19e9e3c21\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:44:13 crc kubenswrapper[5052]: I1203 11:44:13.899561 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage04-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage04-crc\") pod \"glance-default-internal-api-0\" (UID: \"93c3ba7d-27d6-47cd-86a2-30a19e9e3c21\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:44:13 crc kubenswrapper[5052]: I1203 11:44:13.899676 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cq5qz\" (UniqueName: \"kubernetes.io/projected/93c3ba7d-27d6-47cd-86a2-30a19e9e3c21-kube-api-access-cq5qz\") pod \"glance-default-internal-api-0\" (UID: \"93c3ba7d-27d6-47cd-86a2-30a19e9e3c21\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:44:13 crc kubenswrapper[5052]: I1203 11:44:13.899708 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/93c3ba7d-27d6-47cd-86a2-30a19e9e3c21-httpd-run\") pod \"glance-default-internal-api-0\" (UID: \"93c3ba7d-27d6-47cd-86a2-30a19e9e3c21\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:44:13 crc kubenswrapper[5052]: I1203 11:44:13.899761 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/93c3ba7d-27d6-47cd-86a2-30a19e9e3c21-internal-tls-certs\") pod \"glance-default-internal-api-0\" (UID: \"93c3ba7d-27d6-47cd-86a2-30a19e9e3c21\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:44:13 crc kubenswrapper[5052]: I1203 11:44:13.899811 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/93c3ba7d-27d6-47cd-86a2-30a19e9e3c21-logs\") pod \"glance-default-internal-api-0\" (UID: \"93c3ba7d-27d6-47cd-86a2-30a19e9e3c21\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:44:13 crc kubenswrapper[5052]: I1203 11:44:13.899831 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/93c3ba7d-27d6-47cd-86a2-30a19e9e3c21-combined-ca-bundle\") pod \"glance-default-internal-api-0\" (UID: \"93c3ba7d-27d6-47cd-86a2-30a19e9e3c21\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:44:13 crc kubenswrapper[5052]: I1203 11:44:13.899899 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/93c3ba7d-27d6-47cd-86a2-30a19e9e3c21-config-data\") pod \"glance-default-internal-api-0\" (UID: \"93c3ba7d-27d6-47cd-86a2-30a19e9e3c21\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:44:13 crc kubenswrapper[5052]: I1203 11:44:13.902936 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-pknlf\" (UniqueName: \"kubernetes.io/projected/b53d06f2-28c8-4c18-b9ce-0300f8b8ef06-kube-api-access-pknlf\") pod \"keystone-bootstrap-2hjbj\" (UID: \"b53d06f2-28c8-4c18-b9ce-0300f8b8ef06\") " pod="openstack-kuttl-tests/keystone-bootstrap-2hjbj" Dec 03 11:44:13 crc kubenswrapper[5052]: I1203 11:44:13.975486 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 11:44:14 crc kubenswrapper[5052]: I1203 11:44:14.001488 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/93c3ba7d-27d6-47cd-86a2-30a19e9e3c21-scripts\") pod \"glance-default-internal-api-0\" (UID: \"93c3ba7d-27d6-47cd-86a2-30a19e9e3c21\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:44:14 crc kubenswrapper[5052]: I1203 11:44:14.001550 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage04-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage04-crc\") pod \"glance-default-internal-api-0\" (UID: \"93c3ba7d-27d6-47cd-86a2-30a19e9e3c21\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:44:14 crc kubenswrapper[5052]: I1203 11:44:14.001614 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/70097c03-11be-4f1a-b8a8-8cef71b8e8ce-httpd-run\") pod \"glance-default-external-api-0\" (UID: \"70097c03-11be-4f1a-b8a8-8cef71b8e8ce\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:44:14 crc kubenswrapper[5052]: I1203 11:44:14.001652 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-cq5qz\" (UniqueName: \"kubernetes.io/projected/93c3ba7d-27d6-47cd-86a2-30a19e9e3c21-kube-api-access-cq5qz\") pod \"glance-default-internal-api-0\" (UID: \"93c3ba7d-27d6-47cd-86a2-30a19e9e3c21\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:44:14 crc kubenswrapper[5052]: I1203 11:44:14.001679 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/93c3ba7d-27d6-47cd-86a2-30a19e9e3c21-httpd-run\") pod \"glance-default-internal-api-0\" (UID: \"93c3ba7d-27d6-47cd-86a2-30a19e9e3c21\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:44:14 crc kubenswrapper[5052]: I1203 11:44:14.001715 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/70097c03-11be-4f1a-b8a8-8cef71b8e8ce-public-tls-certs\") pod \"glance-default-external-api-0\" (UID: \"70097c03-11be-4f1a-b8a8-8cef71b8e8ce\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:44:14 crc kubenswrapper[5052]: I1203 11:44:14.001747 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/93c3ba7d-27d6-47cd-86a2-30a19e9e3c21-internal-tls-certs\") pod \"glance-default-internal-api-0\" (UID: \"93c3ba7d-27d6-47cd-86a2-30a19e9e3c21\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:44:14 crc kubenswrapper[5052]: I1203 11:44:14.001772 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sqpj4\" (UniqueName: \"kubernetes.io/projected/70097c03-11be-4f1a-b8a8-8cef71b8e8ce-kube-api-access-sqpj4\") pod \"glance-default-external-api-0\" (UID: \"70097c03-11be-4f1a-b8a8-8cef71b8e8ce\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:44:14 crc kubenswrapper[5052]: I1203 11:44:14.001811 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/93c3ba7d-27d6-47cd-86a2-30a19e9e3c21-logs\") pod \"glance-default-internal-api-0\" (UID: \"93c3ba7d-27d6-47cd-86a2-30a19e9e3c21\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:44:14 crc kubenswrapper[5052]: I1203 11:44:14.001830 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/93c3ba7d-27d6-47cd-86a2-30a19e9e3c21-combined-ca-bundle\") pod \"glance-default-internal-api-0\" (UID: \"93c3ba7d-27d6-47cd-86a2-30a19e9e3c21\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:44:14 crc kubenswrapper[5052]: I1203 11:44:14.001854 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/70097c03-11be-4f1a-b8a8-8cef71b8e8ce-config-data\") pod \"glance-default-external-api-0\" (UID: \"70097c03-11be-4f1a-b8a8-8cef71b8e8ce\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:44:14 crc kubenswrapper[5052]: I1203 11:44:14.001896 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/70097c03-11be-4f1a-b8a8-8cef71b8e8ce-scripts\") pod \"glance-default-external-api-0\" (UID: \"70097c03-11be-4f1a-b8a8-8cef71b8e8ce\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:44:14 crc kubenswrapper[5052]: I1203 11:44:14.001923 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/70097c03-11be-4f1a-b8a8-8cef71b8e8ce-logs\") pod \"glance-default-external-api-0\" (UID: \"70097c03-11be-4f1a-b8a8-8cef71b8e8ce\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:44:14 crc kubenswrapper[5052]: I1203 11:44:14.001956 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/70097c03-11be-4f1a-b8a8-8cef71b8e8ce-combined-ca-bundle\") pod \"glance-default-external-api-0\" (UID: \"70097c03-11be-4f1a-b8a8-8cef71b8e8ce\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:44:14 crc kubenswrapper[5052]: I1203 11:44:14.001978 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/93c3ba7d-27d6-47cd-86a2-30a19e9e3c21-config-data\") pod \"glance-default-internal-api-0\" (UID: \"93c3ba7d-27d6-47cd-86a2-30a19e9e3c21\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:44:14 crc kubenswrapper[5052]: I1203 11:44:14.002003 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage15-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage15-crc\") pod \"glance-default-external-api-0\" (UID: \"70097c03-11be-4f1a-b8a8-8cef71b8e8ce\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:44:14 crc kubenswrapper[5052]: I1203 11:44:14.008109 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage04-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage04-crc\") pod \"glance-default-internal-api-0\" (UID: \"93c3ba7d-27d6-47cd-86a2-30a19e9e3c21\") device mount path \"/mnt/openstack/pv04\"" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:44:14 crc kubenswrapper[5052]: I1203 11:44:14.010292 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/93c3ba7d-27d6-47cd-86a2-30a19e9e3c21-httpd-run\") pod \"glance-default-internal-api-0\" (UID: \"93c3ba7d-27d6-47cd-86a2-30a19e9e3c21\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:44:14 crc kubenswrapper[5052]: I1203 11:44:14.010691 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/93c3ba7d-27d6-47cd-86a2-30a19e9e3c21-logs\") pod \"glance-default-internal-api-0\" (UID: \"93c3ba7d-27d6-47cd-86a2-30a19e9e3c21\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:44:14 crc kubenswrapper[5052]: I1203 11:44:14.013151 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/neutron-db-sync-5m4kd"] Dec 03 11:44:14 crc kubenswrapper[5052]: I1203 11:44:14.014439 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-db-sync-5m4kd" Dec 03 11:44:14 crc kubenswrapper[5052]: I1203 11:44:14.024382 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/93c3ba7d-27d6-47cd-86a2-30a19e9e3c21-internal-tls-certs\") pod \"glance-default-internal-api-0\" (UID: \"93c3ba7d-27d6-47cd-86a2-30a19e9e3c21\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:44:14 crc kubenswrapper[5052]: I1203 11:44:14.024693 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-bootstrap-2hjbj" Dec 03 11:44:14 crc kubenswrapper[5052]: I1203 11:44:14.038413 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"neutron-httpd-config" Dec 03 11:44:14 crc kubenswrapper[5052]: I1203 11:44:14.038457 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"neutron-config" Dec 03 11:44:14 crc kubenswrapper[5052]: I1203 11:44:14.038730 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"neutron-neutron-dockercfg-kfdxn" Dec 03 11:44:14 crc kubenswrapper[5052]: I1203 11:44:14.042751 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/93c3ba7d-27d6-47cd-86a2-30a19e9e3c21-scripts\") pod \"glance-default-internal-api-0\" (UID: \"93c3ba7d-27d6-47cd-86a2-30a19e9e3c21\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:44:14 crc kubenswrapper[5052]: I1203 11:44:14.044538 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/93c3ba7d-27d6-47cd-86a2-30a19e9e3c21-config-data\") pod \"glance-default-internal-api-0\" (UID: \"93c3ba7d-27d6-47cd-86a2-30a19e9e3c21\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:44:14 crc kubenswrapper[5052]: I1203 11:44:14.055779 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:44:14 crc kubenswrapper[5052]: I1203 11:44:14.060429 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-cq5qz\" (UniqueName: \"kubernetes.io/projected/93c3ba7d-27d6-47cd-86a2-30a19e9e3c21-kube-api-access-cq5qz\") pod \"glance-default-internal-api-0\" (UID: \"93c3ba7d-27d6-47cd-86a2-30a19e9e3c21\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:44:14 crc kubenswrapper[5052]: I1203 11:44:14.060617 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:44:14 crc kubenswrapper[5052]: I1203 11:44:14.069370 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-scripts" Dec 03 11:44:14 crc kubenswrapper[5052]: I1203 11:44:14.069931 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-config-data" Dec 03 11:44:14 crc kubenswrapper[5052]: I1203 11:44:14.089462 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/93c3ba7d-27d6-47cd-86a2-30a19e9e3c21-combined-ca-bundle\") pod \"glance-default-internal-api-0\" (UID: \"93c3ba7d-27d6-47cd-86a2-30a19e9e3c21\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:44:14 crc kubenswrapper[5052]: I1203 11:44:14.103630 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/70097c03-11be-4f1a-b8a8-8cef71b8e8ce-config-data\") pod \"glance-default-external-api-0\" (UID: \"70097c03-11be-4f1a-b8a8-8cef71b8e8ce\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:44:14 crc kubenswrapper[5052]: I1203 11:44:14.103685 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/70097c03-11be-4f1a-b8a8-8cef71b8e8ce-scripts\") pod \"glance-default-external-api-0\" (UID: \"70097c03-11be-4f1a-b8a8-8cef71b8e8ce\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:44:14 crc kubenswrapper[5052]: I1203 11:44:14.103708 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/70097c03-11be-4f1a-b8a8-8cef71b8e8ce-logs\") pod \"glance-default-external-api-0\" (UID: \"70097c03-11be-4f1a-b8a8-8cef71b8e8ce\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:44:14 crc kubenswrapper[5052]: I1203 11:44:14.103732 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/70097c03-11be-4f1a-b8a8-8cef71b8e8ce-combined-ca-bundle\") pod \"glance-default-external-api-0\" (UID: \"70097c03-11be-4f1a-b8a8-8cef71b8e8ce\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:44:14 crc kubenswrapper[5052]: I1203 11:44:14.103753 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage15-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage15-crc\") pod \"glance-default-external-api-0\" (UID: \"70097c03-11be-4f1a-b8a8-8cef71b8e8ce\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:44:14 crc kubenswrapper[5052]: I1203 11:44:14.103798 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/70097c03-11be-4f1a-b8a8-8cef71b8e8ce-httpd-run\") pod \"glance-default-external-api-0\" (UID: \"70097c03-11be-4f1a-b8a8-8cef71b8e8ce\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:44:14 crc kubenswrapper[5052]: I1203 11:44:14.103843 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/70097c03-11be-4f1a-b8a8-8cef71b8e8ce-public-tls-certs\") pod \"glance-default-external-api-0\" (UID: \"70097c03-11be-4f1a-b8a8-8cef71b8e8ce\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:44:14 crc kubenswrapper[5052]: I1203 11:44:14.103875 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-sqpj4\" (UniqueName: \"kubernetes.io/projected/70097c03-11be-4f1a-b8a8-8cef71b8e8ce-kube-api-access-sqpj4\") pod \"glance-default-external-api-0\" (UID: \"70097c03-11be-4f1a-b8a8-8cef71b8e8ce\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:44:14 crc kubenswrapper[5052]: I1203 11:44:14.114765 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage15-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage15-crc\") pod \"glance-default-external-api-0\" (UID: \"70097c03-11be-4f1a-b8a8-8cef71b8e8ce\") device mount path \"/mnt/openstack/pv15\"" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:44:14 crc kubenswrapper[5052]: I1203 11:44:14.115178 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/70097c03-11be-4f1a-b8a8-8cef71b8e8ce-logs\") pod \"glance-default-external-api-0\" (UID: \"70097c03-11be-4f1a-b8a8-8cef71b8e8ce\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:44:14 crc kubenswrapper[5052]: I1203 11:44:14.131354 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/cinder-db-sync-qv4bk"] Dec 03 11:44:14 crc kubenswrapper[5052]: I1203 11:44:14.132914 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-db-sync-qv4bk" Dec 03 11:44:14 crc kubenswrapper[5052]: I1203 11:44:14.133499 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/70097c03-11be-4f1a-b8a8-8cef71b8e8ce-scripts\") pod \"glance-default-external-api-0\" (UID: \"70097c03-11be-4f1a-b8a8-8cef71b8e8ce\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:44:14 crc kubenswrapper[5052]: I1203 11:44:14.137429 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/70097c03-11be-4f1a-b8a8-8cef71b8e8ce-config-data\") pod \"glance-default-external-api-0\" (UID: \"70097c03-11be-4f1a-b8a8-8cef71b8e8ce\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:44:14 crc kubenswrapper[5052]: I1203 11:44:14.137743 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/70097c03-11be-4f1a-b8a8-8cef71b8e8ce-httpd-run\") pod \"glance-default-external-api-0\" (UID: \"70097c03-11be-4f1a-b8a8-8cef71b8e8ce\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:44:14 crc kubenswrapper[5052]: I1203 11:44:14.143209 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cinder-config-data" Dec 03 11:44:14 crc kubenswrapper[5052]: I1203 11:44:14.155402 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/70097c03-11be-4f1a-b8a8-8cef71b8e8ce-public-tls-certs\") pod \"glance-default-external-api-0\" (UID: \"70097c03-11be-4f1a-b8a8-8cef71b8e8ce\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:44:14 crc kubenswrapper[5052]: I1203 11:44:14.157317 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-sqpj4\" (UniqueName: \"kubernetes.io/projected/70097c03-11be-4f1a-b8a8-8cef71b8e8ce-kube-api-access-sqpj4\") pod \"glance-default-external-api-0\" (UID: \"70097c03-11be-4f1a-b8a8-8cef71b8e8ce\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:44:14 crc kubenswrapper[5052]: I1203 11:44:14.143257 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cinder-cinder-dockercfg-v4hdv" Dec 03 11:44:14 crc kubenswrapper[5052]: I1203 11:44:14.146472 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cinder-scripts" Dec 03 11:44:14 crc kubenswrapper[5052]: I1203 11:44:14.166998 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:44:14 crc kubenswrapper[5052]: I1203 11:44:14.175456 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/70097c03-11be-4f1a-b8a8-8cef71b8e8ce-combined-ca-bundle\") pod \"glance-default-external-api-0\" (UID: \"70097c03-11be-4f1a-b8a8-8cef71b8e8ce\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:44:14 crc kubenswrapper[5052]: I1203 11:44:14.204825 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/fb3a323e-09aa-47ac-b593-e1c6e9b898e5-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"fb3a323e-09aa-47ac-b593-e1c6e9b898e5\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:44:14 crc kubenswrapper[5052]: I1203 11:44:14.204873 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/fb3a323e-09aa-47ac-b593-e1c6e9b898e5-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"fb3a323e-09aa-47ac-b593-e1c6e9b898e5\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:44:14 crc kubenswrapper[5052]: I1203 11:44:14.204901 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/fb3a323e-09aa-47ac-b593-e1c6e9b898e5-scripts\") pod \"ceilometer-0\" (UID: \"fb3a323e-09aa-47ac-b593-e1c6e9b898e5\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:44:14 crc kubenswrapper[5052]: I1203 11:44:14.204949 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/3d4a523e-547f-4d7a-920f-7a24ca45f183-config-data\") pod \"cinder-db-sync-qv4bk\" (UID: \"3d4a523e-547f-4d7a-920f-7a24ca45f183\") " pod="openstack-kuttl-tests/cinder-db-sync-qv4bk" Dec 03 11:44:14 crc kubenswrapper[5052]: I1203 11:44:14.204967 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j2cp9\" (UniqueName: \"kubernetes.io/projected/fb3a323e-09aa-47ac-b593-e1c6e9b898e5-kube-api-access-j2cp9\") pod \"ceilometer-0\" (UID: \"fb3a323e-09aa-47ac-b593-e1c6e9b898e5\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:44:14 crc kubenswrapper[5052]: I1203 11:44:14.204986 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-glpqq\" (UniqueName: \"kubernetes.io/projected/b384ab60-c8a3-488d-8753-22058200e84f-kube-api-access-glpqq\") pod \"neutron-db-sync-5m4kd\" (UID: \"b384ab60-c8a3-488d-8753-22058200e84f\") " pod="openstack-kuttl-tests/neutron-db-sync-5m4kd" Dec 03 11:44:14 crc kubenswrapper[5052]: I1203 11:44:14.205018 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/b384ab60-c8a3-488d-8753-22058200e84f-config\") pod \"neutron-db-sync-5m4kd\" (UID: \"b384ab60-c8a3-488d-8753-22058200e84f\") " pod="openstack-kuttl-tests/neutron-db-sync-5m4kd" Dec 03 11:44:14 crc kubenswrapper[5052]: I1203 11:44:14.205037 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/3d4a523e-547f-4d7a-920f-7a24ca45f183-scripts\") pod \"cinder-db-sync-qv4bk\" (UID: \"3d4a523e-547f-4d7a-920f-7a24ca45f183\") " pod="openstack-kuttl-tests/cinder-db-sync-qv4bk" Dec 03 11:44:14 crc kubenswrapper[5052]: I1203 11:44:14.205052 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/3d4a523e-547f-4d7a-920f-7a24ca45f183-etc-machine-id\") pod \"cinder-db-sync-qv4bk\" (UID: \"3d4a523e-547f-4d7a-920f-7a24ca45f183\") " pod="openstack-kuttl-tests/cinder-db-sync-qv4bk" Dec 03 11:44:14 crc kubenswrapper[5052]: I1203 11:44:14.205085 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b384ab60-c8a3-488d-8753-22058200e84f-combined-ca-bundle\") pod \"neutron-db-sync-5m4kd\" (UID: \"b384ab60-c8a3-488d-8753-22058200e84f\") " pod="openstack-kuttl-tests/neutron-db-sync-5m4kd" Dec 03 11:44:14 crc kubenswrapper[5052]: I1203 11:44:14.205106 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tp8rl\" (UniqueName: \"kubernetes.io/projected/3d4a523e-547f-4d7a-920f-7a24ca45f183-kube-api-access-tp8rl\") pod \"cinder-db-sync-qv4bk\" (UID: \"3d4a523e-547f-4d7a-920f-7a24ca45f183\") " pod="openstack-kuttl-tests/cinder-db-sync-qv4bk" Dec 03 11:44:14 crc kubenswrapper[5052]: I1203 11:44:14.205129 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3d4a523e-547f-4d7a-920f-7a24ca45f183-combined-ca-bundle\") pod \"cinder-db-sync-qv4bk\" (UID: \"3d4a523e-547f-4d7a-920f-7a24ca45f183\") " pod="openstack-kuttl-tests/cinder-db-sync-qv4bk" Dec 03 11:44:14 crc kubenswrapper[5052]: I1203 11:44:14.205151 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/3d4a523e-547f-4d7a-920f-7a24ca45f183-db-sync-config-data\") pod \"cinder-db-sync-qv4bk\" (UID: \"3d4a523e-547f-4d7a-920f-7a24ca45f183\") " pod="openstack-kuttl-tests/cinder-db-sync-qv4bk" Dec 03 11:44:14 crc kubenswrapper[5052]: I1203 11:44:14.205172 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/fb3a323e-09aa-47ac-b593-e1c6e9b898e5-run-httpd\") pod \"ceilometer-0\" (UID: \"fb3a323e-09aa-47ac-b593-e1c6e9b898e5\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:44:14 crc kubenswrapper[5052]: I1203 11:44:14.205186 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/fb3a323e-09aa-47ac-b593-e1c6e9b898e5-log-httpd\") pod \"ceilometer-0\" (UID: \"fb3a323e-09aa-47ac-b593-e1c6e9b898e5\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:44:14 crc kubenswrapper[5052]: I1203 11:44:14.205208 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/fb3a323e-09aa-47ac-b593-e1c6e9b898e5-config-data\") pod \"ceilometer-0\" (UID: \"fb3a323e-09aa-47ac-b593-e1c6e9b898e5\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:44:14 crc kubenswrapper[5052]: I1203 11:44:14.214802 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-db-sync-5m4kd"] Dec 03 11:44:14 crc kubenswrapper[5052]: I1203 11:44:14.283302 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/placement-db-sync-jlwkf"] Dec 03 11:44:14 crc kubenswrapper[5052]: I1203 11:44:14.285140 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-db-sync-jlwkf" Dec 03 11:44:14 crc kubenswrapper[5052]: I1203 11:44:14.289380 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage04-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage04-crc\") pod \"glance-default-internal-api-0\" (UID: \"93c3ba7d-27d6-47cd-86a2-30a19e9e3c21\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:44:14 crc kubenswrapper[5052]: I1203 11:44:14.289974 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"placement-scripts" Dec 03 11:44:14 crc kubenswrapper[5052]: I1203 11:44:14.289980 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"placement-placement-dockercfg-s6bsn" Dec 03 11:44:14 crc kubenswrapper[5052]: I1203 11:44:14.292070 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"placement-config-data" Dec 03 11:44:14 crc kubenswrapper[5052]: I1203 11:44:14.302145 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage15-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage15-crc\") pod \"glance-default-external-api-0\" (UID: \"70097c03-11be-4f1a-b8a8-8cef71b8e8ce\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:44:14 crc kubenswrapper[5052]: I1203 11:44:14.308423 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b384ab60-c8a3-488d-8753-22058200e84f-combined-ca-bundle\") pod \"neutron-db-sync-5m4kd\" (UID: \"b384ab60-c8a3-488d-8753-22058200e84f\") " pod="openstack-kuttl-tests/neutron-db-sync-5m4kd" Dec 03 11:44:14 crc kubenswrapper[5052]: I1203 11:44:14.308705 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-tp8rl\" (UniqueName: \"kubernetes.io/projected/3d4a523e-547f-4d7a-920f-7a24ca45f183-kube-api-access-tp8rl\") pod \"cinder-db-sync-qv4bk\" (UID: \"3d4a523e-547f-4d7a-920f-7a24ca45f183\") " pod="openstack-kuttl-tests/cinder-db-sync-qv4bk" Dec 03 11:44:14 crc kubenswrapper[5052]: I1203 11:44:14.308800 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3d4a523e-547f-4d7a-920f-7a24ca45f183-combined-ca-bundle\") pod \"cinder-db-sync-qv4bk\" (UID: \"3d4a523e-547f-4d7a-920f-7a24ca45f183\") " pod="openstack-kuttl-tests/cinder-db-sync-qv4bk" Dec 03 11:44:14 crc kubenswrapper[5052]: I1203 11:44:14.308834 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/3d4a523e-547f-4d7a-920f-7a24ca45f183-db-sync-config-data\") pod \"cinder-db-sync-qv4bk\" (UID: \"3d4a523e-547f-4d7a-920f-7a24ca45f183\") " pod="openstack-kuttl-tests/cinder-db-sync-qv4bk" Dec 03 11:44:14 crc kubenswrapper[5052]: I1203 11:44:14.308882 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/fb3a323e-09aa-47ac-b593-e1c6e9b898e5-run-httpd\") pod \"ceilometer-0\" (UID: \"fb3a323e-09aa-47ac-b593-e1c6e9b898e5\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:44:14 crc kubenswrapper[5052]: I1203 11:44:14.308897 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/fb3a323e-09aa-47ac-b593-e1c6e9b898e5-log-httpd\") pod \"ceilometer-0\" (UID: \"fb3a323e-09aa-47ac-b593-e1c6e9b898e5\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:44:14 crc kubenswrapper[5052]: I1203 11:44:14.308943 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/fb3a323e-09aa-47ac-b593-e1c6e9b898e5-config-data\") pod \"ceilometer-0\" (UID: \"fb3a323e-09aa-47ac-b593-e1c6e9b898e5\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:44:14 crc kubenswrapper[5052]: I1203 11:44:14.308977 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/fb3a323e-09aa-47ac-b593-e1c6e9b898e5-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"fb3a323e-09aa-47ac-b593-e1c6e9b898e5\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:44:14 crc kubenswrapper[5052]: I1203 11:44:14.309020 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/fb3a323e-09aa-47ac-b593-e1c6e9b898e5-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"fb3a323e-09aa-47ac-b593-e1c6e9b898e5\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:44:14 crc kubenswrapper[5052]: I1203 11:44:14.309059 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/fb3a323e-09aa-47ac-b593-e1c6e9b898e5-scripts\") pod \"ceilometer-0\" (UID: \"fb3a323e-09aa-47ac-b593-e1c6e9b898e5\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:44:14 crc kubenswrapper[5052]: I1203 11:44:14.309134 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/3d4a523e-547f-4d7a-920f-7a24ca45f183-config-data\") pod \"cinder-db-sync-qv4bk\" (UID: \"3d4a523e-547f-4d7a-920f-7a24ca45f183\") " pod="openstack-kuttl-tests/cinder-db-sync-qv4bk" Dec 03 11:44:14 crc kubenswrapper[5052]: I1203 11:44:14.309158 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-j2cp9\" (UniqueName: \"kubernetes.io/projected/fb3a323e-09aa-47ac-b593-e1c6e9b898e5-kube-api-access-j2cp9\") pod \"ceilometer-0\" (UID: \"fb3a323e-09aa-47ac-b593-e1c6e9b898e5\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:44:14 crc kubenswrapper[5052]: I1203 11:44:14.309187 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-glpqq\" (UniqueName: \"kubernetes.io/projected/b384ab60-c8a3-488d-8753-22058200e84f-kube-api-access-glpqq\") pod \"neutron-db-sync-5m4kd\" (UID: \"b384ab60-c8a3-488d-8753-22058200e84f\") " pod="openstack-kuttl-tests/neutron-db-sync-5m4kd" Dec 03 11:44:14 crc kubenswrapper[5052]: I1203 11:44:14.309245 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/b384ab60-c8a3-488d-8753-22058200e84f-config\") pod \"neutron-db-sync-5m4kd\" (UID: \"b384ab60-c8a3-488d-8753-22058200e84f\") " pod="openstack-kuttl-tests/neutron-db-sync-5m4kd" Dec 03 11:44:14 crc kubenswrapper[5052]: I1203 11:44:14.309273 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/3d4a523e-547f-4d7a-920f-7a24ca45f183-scripts\") pod \"cinder-db-sync-qv4bk\" (UID: \"3d4a523e-547f-4d7a-920f-7a24ca45f183\") " pod="openstack-kuttl-tests/cinder-db-sync-qv4bk" Dec 03 11:44:14 crc kubenswrapper[5052]: I1203 11:44:14.309298 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/3d4a523e-547f-4d7a-920f-7a24ca45f183-etc-machine-id\") pod \"cinder-db-sync-qv4bk\" (UID: \"3d4a523e-547f-4d7a-920f-7a24ca45f183\") " pod="openstack-kuttl-tests/cinder-db-sync-qv4bk" Dec 03 11:44:14 crc kubenswrapper[5052]: I1203 11:44:14.309471 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/3d4a523e-547f-4d7a-920f-7a24ca45f183-etc-machine-id\") pod \"cinder-db-sync-qv4bk\" (UID: \"3d4a523e-547f-4d7a-920f-7a24ca45f183\") " pod="openstack-kuttl-tests/cinder-db-sync-qv4bk" Dec 03 11:44:14 crc kubenswrapper[5052]: I1203 11:44:14.323208 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/b384ab60-c8a3-488d-8753-22058200e84f-config\") pod \"neutron-db-sync-5m4kd\" (UID: \"b384ab60-c8a3-488d-8753-22058200e84f\") " pod="openstack-kuttl-tests/neutron-db-sync-5m4kd" Dec 03 11:44:14 crc kubenswrapper[5052]: I1203 11:44:14.326986 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b384ab60-c8a3-488d-8753-22058200e84f-combined-ca-bundle\") pod \"neutron-db-sync-5m4kd\" (UID: \"b384ab60-c8a3-488d-8753-22058200e84f\") " pod="openstack-kuttl-tests/neutron-db-sync-5m4kd" Dec 03 11:44:14 crc kubenswrapper[5052]: I1203 11:44:14.331715 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-db-sync-qv4bk"] Dec 03 11:44:14 crc kubenswrapper[5052]: I1203 11:44:14.359155 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/fb3a323e-09aa-47ac-b593-e1c6e9b898e5-log-httpd\") pod \"ceilometer-0\" (UID: \"fb3a323e-09aa-47ac-b593-e1c6e9b898e5\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:44:14 crc kubenswrapper[5052]: I1203 11:44:14.366598 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-glpqq\" (UniqueName: \"kubernetes.io/projected/b384ab60-c8a3-488d-8753-22058200e84f-kube-api-access-glpqq\") pod \"neutron-db-sync-5m4kd\" (UID: \"b384ab60-c8a3-488d-8753-22058200e84f\") " pod="openstack-kuttl-tests/neutron-db-sync-5m4kd" Dec 03 11:44:14 crc kubenswrapper[5052]: I1203 11:44:14.368611 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/fb3a323e-09aa-47ac-b593-e1c6e9b898e5-scripts\") pod \"ceilometer-0\" (UID: \"fb3a323e-09aa-47ac-b593-e1c6e9b898e5\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:44:14 crc kubenswrapper[5052]: I1203 11:44:14.383187 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/fb3a323e-09aa-47ac-b593-e1c6e9b898e5-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"fb3a323e-09aa-47ac-b593-e1c6e9b898e5\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:44:14 crc kubenswrapper[5052]: I1203 11:44:14.385215 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/3d4a523e-547f-4d7a-920f-7a24ca45f183-scripts\") pod \"cinder-db-sync-qv4bk\" (UID: \"3d4a523e-547f-4d7a-920f-7a24ca45f183\") " pod="openstack-kuttl-tests/cinder-db-sync-qv4bk" Dec 03 11:44:14 crc kubenswrapper[5052]: I1203 11:44:14.389364 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/placement-db-sync-jlwkf"] Dec 03 11:44:14 crc kubenswrapper[5052]: I1203 11:44:14.390576 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/fb3a323e-09aa-47ac-b593-e1c6e9b898e5-run-httpd\") pod \"ceilometer-0\" (UID: \"fb3a323e-09aa-47ac-b593-e1c6e9b898e5\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:44:14 crc kubenswrapper[5052]: I1203 11:44:14.399286 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/fb3a323e-09aa-47ac-b593-e1c6e9b898e5-config-data\") pod \"ceilometer-0\" (UID: \"fb3a323e-09aa-47ac-b593-e1c6e9b898e5\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:44:14 crc kubenswrapper[5052]: I1203 11:44:14.399480 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/fb3a323e-09aa-47ac-b593-e1c6e9b898e5-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"fb3a323e-09aa-47ac-b593-e1c6e9b898e5\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:44:14 crc kubenswrapper[5052]: I1203 11:44:14.402975 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3d4a523e-547f-4d7a-920f-7a24ca45f183-combined-ca-bundle\") pod \"cinder-db-sync-qv4bk\" (UID: \"3d4a523e-547f-4d7a-920f-7a24ca45f183\") " pod="openstack-kuttl-tests/cinder-db-sync-qv4bk" Dec 03 11:44:14 crc kubenswrapper[5052]: I1203 11:44:14.402432 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/3d4a523e-547f-4d7a-920f-7a24ca45f183-config-data\") pod \"cinder-db-sync-qv4bk\" (UID: \"3d4a523e-547f-4d7a-920f-7a24ca45f183\") " pod="openstack-kuttl-tests/cinder-db-sync-qv4bk" Dec 03 11:44:14 crc kubenswrapper[5052]: I1203 11:44:14.406212 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/3d4a523e-547f-4d7a-920f-7a24ca45f183-db-sync-config-data\") pod \"cinder-db-sync-qv4bk\" (UID: \"3d4a523e-547f-4d7a-920f-7a24ca45f183\") " pod="openstack-kuttl-tests/cinder-db-sync-qv4bk" Dec 03 11:44:14 crc kubenswrapper[5052]: I1203 11:44:14.410948 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/35ce22c2-a534-49af-9ba3-3282dffbaf95-logs\") pod \"placement-db-sync-jlwkf\" (UID: \"35ce22c2-a534-49af-9ba3-3282dffbaf95\") " pod="openstack-kuttl-tests/placement-db-sync-jlwkf" Dec 03 11:44:14 crc kubenswrapper[5052]: I1203 11:44:14.410980 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/35ce22c2-a534-49af-9ba3-3282dffbaf95-scripts\") pod \"placement-db-sync-jlwkf\" (UID: \"35ce22c2-a534-49af-9ba3-3282dffbaf95\") " pod="openstack-kuttl-tests/placement-db-sync-jlwkf" Dec 03 11:44:14 crc kubenswrapper[5052]: I1203 11:44:14.411023 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/35ce22c2-a534-49af-9ba3-3282dffbaf95-combined-ca-bundle\") pod \"placement-db-sync-jlwkf\" (UID: \"35ce22c2-a534-49af-9ba3-3282dffbaf95\") " pod="openstack-kuttl-tests/placement-db-sync-jlwkf" Dec 03 11:44:14 crc kubenswrapper[5052]: I1203 11:44:14.411098 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v9tjx\" (UniqueName: \"kubernetes.io/projected/35ce22c2-a534-49af-9ba3-3282dffbaf95-kube-api-access-v9tjx\") pod \"placement-db-sync-jlwkf\" (UID: \"35ce22c2-a534-49af-9ba3-3282dffbaf95\") " pod="openstack-kuttl-tests/placement-db-sync-jlwkf" Dec 03 11:44:14 crc kubenswrapper[5052]: I1203 11:44:14.411147 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/35ce22c2-a534-49af-9ba3-3282dffbaf95-config-data\") pod \"placement-db-sync-jlwkf\" (UID: \"35ce22c2-a534-49af-9ba3-3282dffbaf95\") " pod="openstack-kuttl-tests/placement-db-sync-jlwkf" Dec 03 11:44:14 crc kubenswrapper[5052]: I1203 11:44:14.412611 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-tp8rl\" (UniqueName: \"kubernetes.io/projected/3d4a523e-547f-4d7a-920f-7a24ca45f183-kube-api-access-tp8rl\") pod \"cinder-db-sync-qv4bk\" (UID: \"3d4a523e-547f-4d7a-920f-7a24ca45f183\") " pod="openstack-kuttl-tests/cinder-db-sync-qv4bk" Dec 03 11:44:14 crc kubenswrapper[5052]: I1203 11:44:14.418809 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-j2cp9\" (UniqueName: \"kubernetes.io/projected/fb3a323e-09aa-47ac-b593-e1c6e9b898e5-kube-api-access-j2cp9\") pod \"ceilometer-0\" (UID: \"fb3a323e-09aa-47ac-b593-e1c6e9b898e5\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:44:14 crc kubenswrapper[5052]: I1203 11:44:14.423749 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/barbican-db-sync-gmcqn"] Dec 03 11:44:14 crc kubenswrapper[5052]: I1203 11:44:14.425343 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-db-sync-gmcqn" Dec 03 11:44:14 crc kubenswrapper[5052]: I1203 11:44:14.430673 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"barbican-config-data" Dec 03 11:44:14 crc kubenswrapper[5052]: I1203 11:44:14.430924 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"barbican-barbican-dockercfg-ft6vf" Dec 03 11:44:14 crc kubenswrapper[5052]: I1203 11:44:14.432174 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-db-sync-gmcqn"] Dec 03 11:44:14 crc kubenswrapper[5052]: I1203 11:44:14.512364 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/35ce22c2-a534-49af-9ba3-3282dffbaf95-scripts\") pod \"placement-db-sync-jlwkf\" (UID: \"35ce22c2-a534-49af-9ba3-3282dffbaf95\") " pod="openstack-kuttl-tests/placement-db-sync-jlwkf" Dec 03 11:44:14 crc kubenswrapper[5052]: I1203 11:44:14.512777 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/35ce22c2-a534-49af-9ba3-3282dffbaf95-combined-ca-bundle\") pod \"placement-db-sync-jlwkf\" (UID: \"35ce22c2-a534-49af-9ba3-3282dffbaf95\") " pod="openstack-kuttl-tests/placement-db-sync-jlwkf" Dec 03 11:44:14 crc kubenswrapper[5052]: I1203 11:44:14.512919 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/189bc36c-9345-4a45-92ce-1ba812535b54-combined-ca-bundle\") pod \"barbican-db-sync-gmcqn\" (UID: \"189bc36c-9345-4a45-92ce-1ba812535b54\") " pod="openstack-kuttl-tests/barbican-db-sync-gmcqn" Dec 03 11:44:14 crc kubenswrapper[5052]: I1203 11:44:14.512956 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vkwnl\" (UniqueName: \"kubernetes.io/projected/189bc36c-9345-4a45-92ce-1ba812535b54-kube-api-access-vkwnl\") pod \"barbican-db-sync-gmcqn\" (UID: \"189bc36c-9345-4a45-92ce-1ba812535b54\") " pod="openstack-kuttl-tests/barbican-db-sync-gmcqn" Dec 03 11:44:14 crc kubenswrapper[5052]: I1203 11:44:14.513011 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/189bc36c-9345-4a45-92ce-1ba812535b54-db-sync-config-data\") pod \"barbican-db-sync-gmcqn\" (UID: \"189bc36c-9345-4a45-92ce-1ba812535b54\") " pod="openstack-kuttl-tests/barbican-db-sync-gmcqn" Dec 03 11:44:14 crc kubenswrapper[5052]: I1203 11:44:14.513095 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-v9tjx\" (UniqueName: \"kubernetes.io/projected/35ce22c2-a534-49af-9ba3-3282dffbaf95-kube-api-access-v9tjx\") pod \"placement-db-sync-jlwkf\" (UID: \"35ce22c2-a534-49af-9ba3-3282dffbaf95\") " pod="openstack-kuttl-tests/placement-db-sync-jlwkf" Dec 03 11:44:14 crc kubenswrapper[5052]: I1203 11:44:14.513185 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/35ce22c2-a534-49af-9ba3-3282dffbaf95-config-data\") pod \"placement-db-sync-jlwkf\" (UID: \"35ce22c2-a534-49af-9ba3-3282dffbaf95\") " pod="openstack-kuttl-tests/placement-db-sync-jlwkf" Dec 03 11:44:14 crc kubenswrapper[5052]: I1203 11:44:14.513251 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/35ce22c2-a534-49af-9ba3-3282dffbaf95-logs\") pod \"placement-db-sync-jlwkf\" (UID: \"35ce22c2-a534-49af-9ba3-3282dffbaf95\") " pod="openstack-kuttl-tests/placement-db-sync-jlwkf" Dec 03 11:44:14 crc kubenswrapper[5052]: I1203 11:44:14.513754 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/35ce22c2-a534-49af-9ba3-3282dffbaf95-logs\") pod \"placement-db-sync-jlwkf\" (UID: \"35ce22c2-a534-49af-9ba3-3282dffbaf95\") " pod="openstack-kuttl-tests/placement-db-sync-jlwkf" Dec 03 11:44:14 crc kubenswrapper[5052]: I1203 11:44:14.515472 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/35ce22c2-a534-49af-9ba3-3282dffbaf95-scripts\") pod \"placement-db-sync-jlwkf\" (UID: \"35ce22c2-a534-49af-9ba3-3282dffbaf95\") " pod="openstack-kuttl-tests/placement-db-sync-jlwkf" Dec 03 11:44:14 crc kubenswrapper[5052]: I1203 11:44:14.516498 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/35ce22c2-a534-49af-9ba3-3282dffbaf95-combined-ca-bundle\") pod \"placement-db-sync-jlwkf\" (UID: \"35ce22c2-a534-49af-9ba3-3282dffbaf95\") " pod="openstack-kuttl-tests/placement-db-sync-jlwkf" Dec 03 11:44:14 crc kubenswrapper[5052]: I1203 11:44:14.516554 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/35ce22c2-a534-49af-9ba3-3282dffbaf95-config-data\") pod \"placement-db-sync-jlwkf\" (UID: \"35ce22c2-a534-49af-9ba3-3282dffbaf95\") " pod="openstack-kuttl-tests/placement-db-sync-jlwkf" Dec 03 11:44:14 crc kubenswrapper[5052]: I1203 11:44:14.524630 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:44:14 crc kubenswrapper[5052]: I1203 11:44:14.529345 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-v9tjx\" (UniqueName: \"kubernetes.io/projected/35ce22c2-a534-49af-9ba3-3282dffbaf95-kube-api-access-v9tjx\") pod \"placement-db-sync-jlwkf\" (UID: \"35ce22c2-a534-49af-9ba3-3282dffbaf95\") " pod="openstack-kuttl-tests/placement-db-sync-jlwkf" Dec 03 11:44:14 crc kubenswrapper[5052]: I1203 11:44:14.567577 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:44:14 crc kubenswrapper[5052]: I1203 11:44:14.573443 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-db-sync-5m4kd" Dec 03 11:44:14 crc kubenswrapper[5052]: I1203 11:44:14.620323 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-vkwnl\" (UniqueName: \"kubernetes.io/projected/189bc36c-9345-4a45-92ce-1ba812535b54-kube-api-access-vkwnl\") pod \"barbican-db-sync-gmcqn\" (UID: \"189bc36c-9345-4a45-92ce-1ba812535b54\") " pod="openstack-kuttl-tests/barbican-db-sync-gmcqn" Dec 03 11:44:14 crc kubenswrapper[5052]: I1203 11:44:14.620375 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/189bc36c-9345-4a45-92ce-1ba812535b54-db-sync-config-data\") pod \"barbican-db-sync-gmcqn\" (UID: \"189bc36c-9345-4a45-92ce-1ba812535b54\") " pod="openstack-kuttl-tests/barbican-db-sync-gmcqn" Dec 03 11:44:14 crc kubenswrapper[5052]: I1203 11:44:14.620537 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/189bc36c-9345-4a45-92ce-1ba812535b54-combined-ca-bundle\") pod \"barbican-db-sync-gmcqn\" (UID: \"189bc36c-9345-4a45-92ce-1ba812535b54\") " pod="openstack-kuttl-tests/barbican-db-sync-gmcqn" Dec 03 11:44:14 crc kubenswrapper[5052]: I1203 11:44:14.626564 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/189bc36c-9345-4a45-92ce-1ba812535b54-combined-ca-bundle\") pod \"barbican-db-sync-gmcqn\" (UID: \"189bc36c-9345-4a45-92ce-1ba812535b54\") " pod="openstack-kuttl-tests/barbican-db-sync-gmcqn" Dec 03 11:44:14 crc kubenswrapper[5052]: I1203 11:44:14.627068 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/189bc36c-9345-4a45-92ce-1ba812535b54-db-sync-config-data\") pod \"barbican-db-sync-gmcqn\" (UID: \"189bc36c-9345-4a45-92ce-1ba812535b54\") " pod="openstack-kuttl-tests/barbican-db-sync-gmcqn" Dec 03 11:44:14 crc kubenswrapper[5052]: I1203 11:44:14.648908 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-vkwnl\" (UniqueName: \"kubernetes.io/projected/189bc36c-9345-4a45-92ce-1ba812535b54-kube-api-access-vkwnl\") pod \"barbican-db-sync-gmcqn\" (UID: \"189bc36c-9345-4a45-92ce-1ba812535b54\") " pod="openstack-kuttl-tests/barbican-db-sync-gmcqn" Dec 03 11:44:14 crc kubenswrapper[5052]: I1203 11:44:14.695022 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:44:14 crc kubenswrapper[5052]: I1203 11:44:14.698013 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-db-sync-qv4bk" Dec 03 11:44:14 crc kubenswrapper[5052]: I1203 11:44:14.698636 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-db-sync-jlwkf" Dec 03 11:44:14 crc kubenswrapper[5052]: I1203 11:44:14.751879 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-db-sync-gmcqn" Dec 03 11:44:14 crc kubenswrapper[5052]: I1203 11:44:14.752175 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-bootstrap-2hjbj"] Dec 03 11:44:14 crc kubenswrapper[5052]: W1203 11:44:14.767772 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb53d06f2_28c8_4c18_b9ce_0300f8b8ef06.slice/crio-817629e22a8788100a0ceccb4a44aed9b4f417d1251d93488132a0e5b59075bc WatchSource:0}: Error finding container 817629e22a8788100a0ceccb4a44aed9b4f417d1251d93488132a0e5b59075bc: Status 404 returned error can't find the container with id 817629e22a8788100a0ceccb4a44aed9b4f417d1251d93488132a0e5b59075bc Dec 03 11:44:15 crc kubenswrapper[5052]: I1203 11:44:15.027799 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 11:44:15 crc kubenswrapper[5052]: W1203 11:44:15.040632 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod93c3ba7d_27d6_47cd_86a2_30a19e9e3c21.slice/crio-644e3f8433c6f88d0b05b42f1e18a222f48d7e71a183cb71b8171bc45ae8b802 WatchSource:0}: Error finding container 644e3f8433c6f88d0b05b42f1e18a222f48d7e71a183cb71b8171bc45ae8b802: Status 404 returned error can't find the container with id 644e3f8433c6f88d0b05b42f1e18a222f48d7e71a183cb71b8171bc45ae8b802 Dec 03 11:44:15 crc kubenswrapper[5052]: I1203 11:44:15.097148 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-db-sync-5m4kd"] Dec 03 11:44:15 crc kubenswrapper[5052]: W1203 11:44:15.127164 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb384ab60_c8a3_488d_8753_22058200e84f.slice/crio-cec0285ed8174eb9623ed9d54fc95d1c8dd5f4912ff13b4cd2500eef0911c323 WatchSource:0}: Error finding container cec0285ed8174eb9623ed9d54fc95d1c8dd5f4912ff13b4cd2500eef0911c323: Status 404 returned error can't find the container with id cec0285ed8174eb9623ed9d54fc95d1c8dd5f4912ff13b4cd2500eef0911c323 Dec 03 11:44:15 crc kubenswrapper[5052]: I1203 11:44:15.229142 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-bootstrap-2hjbj" event={"ID":"b53d06f2-28c8-4c18-b9ce-0300f8b8ef06","Type":"ContainerStarted","Data":"817629e22a8788100a0ceccb4a44aed9b4f417d1251d93488132a0e5b59075bc"} Dec 03 11:44:15 crc kubenswrapper[5052]: I1203 11:44:15.233896 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"93c3ba7d-27d6-47cd-86a2-30a19e9e3c21","Type":"ContainerStarted","Data":"644e3f8433c6f88d0b05b42f1e18a222f48d7e71a183cb71b8171bc45ae8b802"} Dec 03 11:44:15 crc kubenswrapper[5052]: I1203 11:44:15.235616 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-db-sync-5m4kd" event={"ID":"b384ab60-c8a3-488d-8753-22058200e84f","Type":"ContainerStarted","Data":"cec0285ed8174eb9623ed9d54fc95d1c8dd5f4912ff13b4cd2500eef0911c323"} Dec 03 11:44:15 crc kubenswrapper[5052]: I1203 11:44:15.335131 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 11:44:15 crc kubenswrapper[5052]: W1203 11:44:15.363660 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod70097c03_11be_4f1a_b8a8_8cef71b8e8ce.slice/crio-442dd4192a5767001fe5a9809e9a3ac3b884b549d8e8ce20b12b17737ca8dc51 WatchSource:0}: Error finding container 442dd4192a5767001fe5a9809e9a3ac3b884b549d8e8ce20b12b17737ca8dc51: Status 404 returned error can't find the container with id 442dd4192a5767001fe5a9809e9a3ac3b884b549d8e8ce20b12b17737ca8dc51 Dec 03 11:44:15 crc kubenswrapper[5052]: I1203 11:44:15.455161 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:44:15 crc kubenswrapper[5052]: I1203 11:44:15.462766 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-db-sync-gmcqn"] Dec 03 11:44:15 crc kubenswrapper[5052]: I1203 11:44:15.607792 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/placement-db-sync-jlwkf"] Dec 03 11:44:15 crc kubenswrapper[5052]: W1203 11:44:15.615625 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3d4a523e_547f_4d7a_920f_7a24ca45f183.slice/crio-8ca7a090a69a8425edf74a6ee41359493434fd61eed1fac3855a560aac2c1840 WatchSource:0}: Error finding container 8ca7a090a69a8425edf74a6ee41359493434fd61eed1fac3855a560aac2c1840: Status 404 returned error can't find the container with id 8ca7a090a69a8425edf74a6ee41359493434fd61eed1fac3855a560aac2c1840 Dec 03 11:44:15 crc kubenswrapper[5052]: I1203 11:44:15.618631 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-db-sync-qv4bk"] Dec 03 11:44:16 crc kubenswrapper[5052]: I1203 11:44:16.279082 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-db-sync-gmcqn" event={"ID":"189bc36c-9345-4a45-92ce-1ba812535b54","Type":"ContainerStarted","Data":"f64575a6937b334384672b5b5ceb671a3f4b664259bb329427fc96882137e948"} Dec 03 11:44:16 crc kubenswrapper[5052]: I1203 11:44:16.279407 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-db-sync-gmcqn" event={"ID":"189bc36c-9345-4a45-92ce-1ba812535b54","Type":"ContainerStarted","Data":"1bd706ca1dd3485e6cea24043865e394cc3ed11e7e47f39bddac8f804a7cc592"} Dec 03 11:44:16 crc kubenswrapper[5052]: I1203 11:44:16.282405 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-bootstrap-2hjbj" event={"ID":"b53d06f2-28c8-4c18-b9ce-0300f8b8ef06","Type":"ContainerStarted","Data":"52f339b3d910dbf9c4fb68846eb91d563721d72aa18e7b140f1a773dba41565c"} Dec 03 11:44:16 crc kubenswrapper[5052]: I1203 11:44:16.288445 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"70097c03-11be-4f1a-b8a8-8cef71b8e8ce","Type":"ContainerStarted","Data":"442dd4192a5767001fe5a9809e9a3ac3b884b549d8e8ce20b12b17737ca8dc51"} Dec 03 11:44:16 crc kubenswrapper[5052]: I1203 11:44:16.297113 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"93c3ba7d-27d6-47cd-86a2-30a19e9e3c21","Type":"ContainerStarted","Data":"aacc584f4b653c45815076ff6ec9755843746f3d7a69d83ffcfdca689b761b08"} Dec 03 11:44:16 crc kubenswrapper[5052]: I1203 11:44:16.312939 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/barbican-db-sync-gmcqn" podStartSLOduration=2.312914342 podStartE2EDuration="2.312914342s" podCreationTimestamp="2025-12-03 11:44:14 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:44:16.305469447 +0000 UTC m=+1446.893711211" watchObservedRunningTime="2025-12-03 11:44:16.312914342 +0000 UTC m=+1446.901156106" Dec 03 11:44:16 crc kubenswrapper[5052]: I1203 11:44:16.313732 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-db-sync-qv4bk" event={"ID":"3d4a523e-547f-4d7a-920f-7a24ca45f183","Type":"ContainerStarted","Data":"8ca7a090a69a8425edf74a6ee41359493434fd61eed1fac3855a560aac2c1840"} Dec 03 11:44:16 crc kubenswrapper[5052]: I1203 11:44:16.357383 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-db-sync-5m4kd" event={"ID":"b384ab60-c8a3-488d-8753-22058200e84f","Type":"ContainerStarted","Data":"c9c41a712ce14f494b913d7d4165d8d399a912c664179fc011621e5b9c476b1c"} Dec 03 11:44:16 crc kubenswrapper[5052]: I1203 11:44:16.360564 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-db-sync-jlwkf" event={"ID":"35ce22c2-a534-49af-9ba3-3282dffbaf95","Type":"ContainerStarted","Data":"e8f9201ec877119fe422d30c220b1d235ac9fd50b57cec45153ccb587583396d"} Dec 03 11:44:16 crc kubenswrapper[5052]: I1203 11:44:16.360621 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-db-sync-jlwkf" event={"ID":"35ce22c2-a534-49af-9ba3-3282dffbaf95","Type":"ContainerStarted","Data":"d86c867797f12e395d872e95bc38b91defdb453ba72063d4d65419a6b2398a05"} Dec 03 11:44:16 crc kubenswrapper[5052]: I1203 11:44:16.362968 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"fb3a323e-09aa-47ac-b593-e1c6e9b898e5","Type":"ContainerStarted","Data":"3c0b98c68b59cee11baed24f6b4ecfa16e55232bf35a0455f88eaa58c922a93d"} Dec 03 11:44:16 crc kubenswrapper[5052]: I1203 11:44:16.394535 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/neutron-db-sync-5m4kd" podStartSLOduration=3.394497284 podStartE2EDuration="3.394497284s" podCreationTimestamp="2025-12-03 11:44:13 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:44:16.393178346 +0000 UTC m=+1446.981420110" watchObservedRunningTime="2025-12-03 11:44:16.394497284 +0000 UTC m=+1446.982739048" Dec 03 11:44:16 crc kubenswrapper[5052]: I1203 11:44:16.399999 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/keystone-bootstrap-2hjbj" podStartSLOduration=3.399982902 podStartE2EDuration="3.399982902s" podCreationTimestamp="2025-12-03 11:44:13 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:44:16.345608204 +0000 UTC m=+1446.933849968" watchObservedRunningTime="2025-12-03 11:44:16.399982902 +0000 UTC m=+1446.988224666" Dec 03 11:44:16 crc kubenswrapper[5052]: I1203 11:44:16.413578 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/placement-db-sync-jlwkf" podStartSLOduration=2.413561164 podStartE2EDuration="2.413561164s" podCreationTimestamp="2025-12-03 11:44:14 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:44:16.411827984 +0000 UTC m=+1447.000069748" watchObservedRunningTime="2025-12-03 11:44:16.413561164 +0000 UTC m=+1447.001802928" Dec 03 11:44:16 crc kubenswrapper[5052]: I1203 11:44:16.876539 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 11:44:16 crc kubenswrapper[5052]: I1203 11:44:16.897673 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 11:44:16 crc kubenswrapper[5052]: I1203 11:44:16.928691 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:44:17 crc kubenswrapper[5052]: I1203 11:44:17.387105 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"93c3ba7d-27d6-47cd-86a2-30a19e9e3c21","Type":"ContainerStarted","Data":"8ca2c0a229a587915dd55157616554b37d810216ac8f67032c79c498803a7d99"} Dec 03 11:44:17 crc kubenswrapper[5052]: I1203 11:44:17.387443 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-internal-api-0" podUID="93c3ba7d-27d6-47cd-86a2-30a19e9e3c21" containerName="glance-log" containerID="cri-o://aacc584f4b653c45815076ff6ec9755843746f3d7a69d83ffcfdca689b761b08" gracePeriod=30 Dec 03 11:44:17 crc kubenswrapper[5052]: I1203 11:44:17.387610 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-internal-api-0" podUID="93c3ba7d-27d6-47cd-86a2-30a19e9e3c21" containerName="glance-httpd" containerID="cri-o://8ca2c0a229a587915dd55157616554b37d810216ac8f67032c79c498803a7d99" gracePeriod=30 Dec 03 11:44:17 crc kubenswrapper[5052]: I1203 11:44:17.402695 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"fb3a323e-09aa-47ac-b593-e1c6e9b898e5","Type":"ContainerStarted","Data":"7bff6b5804029ed658f2a1d5487e3bce63445ac4e6250251b735aee6dff232f5"} Dec 03 11:44:17 crc kubenswrapper[5052]: I1203 11:44:17.405769 5052 generic.go:334] "Generic (PLEG): container finished" podID="189bc36c-9345-4a45-92ce-1ba812535b54" containerID="f64575a6937b334384672b5b5ceb671a3f4b664259bb329427fc96882137e948" exitCode=0 Dec 03 11:44:17 crc kubenswrapper[5052]: I1203 11:44:17.405826 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-db-sync-gmcqn" event={"ID":"189bc36c-9345-4a45-92ce-1ba812535b54","Type":"ContainerDied","Data":"f64575a6937b334384672b5b5ceb671a3f4b664259bb329427fc96882137e948"} Dec 03 11:44:17 crc kubenswrapper[5052]: I1203 11:44:17.412777 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"70097c03-11be-4f1a-b8a8-8cef71b8e8ce","Type":"ContainerStarted","Data":"60e01155d35f8c06fef45b2743f3cfe9b7364e3fa8bf67f6e5502383b4632ffc"} Dec 03 11:44:17 crc kubenswrapper[5052]: I1203 11:44:17.435922 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/glance-default-internal-api-0" podStartSLOduration=4.435906225 podStartE2EDuration="4.435906225s" podCreationTimestamp="2025-12-03 11:44:13 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:44:17.434570327 +0000 UTC m=+1448.022812091" watchObservedRunningTime="2025-12-03 11:44:17.435906225 +0000 UTC m=+1448.024147989" Dec 03 11:44:18 crc kubenswrapper[5052]: I1203 11:44:18.087153 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:44:18 crc kubenswrapper[5052]: I1203 11:44:18.239418 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/93c3ba7d-27d6-47cd-86a2-30a19e9e3c21-logs\") pod \"93c3ba7d-27d6-47cd-86a2-30a19e9e3c21\" (UID: \"93c3ba7d-27d6-47cd-86a2-30a19e9e3c21\") " Dec 03 11:44:18 crc kubenswrapper[5052]: I1203 11:44:18.239771 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/93c3ba7d-27d6-47cd-86a2-30a19e9e3c21-scripts\") pod \"93c3ba7d-27d6-47cd-86a2-30a19e9e3c21\" (UID: \"93c3ba7d-27d6-47cd-86a2-30a19e9e3c21\") " Dec 03 11:44:18 crc kubenswrapper[5052]: I1203 11:44:18.239790 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"glance\" (UniqueName: \"kubernetes.io/local-volume/local-storage04-crc\") pod \"93c3ba7d-27d6-47cd-86a2-30a19e9e3c21\" (UID: \"93c3ba7d-27d6-47cd-86a2-30a19e9e3c21\") " Dec 03 11:44:18 crc kubenswrapper[5052]: I1203 11:44:18.239837 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/93c3ba7d-27d6-47cd-86a2-30a19e9e3c21-httpd-run\") pod \"93c3ba7d-27d6-47cd-86a2-30a19e9e3c21\" (UID: \"93c3ba7d-27d6-47cd-86a2-30a19e9e3c21\") " Dec 03 11:44:18 crc kubenswrapper[5052]: I1203 11:44:18.239854 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/93c3ba7d-27d6-47cd-86a2-30a19e9e3c21-internal-tls-certs\") pod \"93c3ba7d-27d6-47cd-86a2-30a19e9e3c21\" (UID: \"93c3ba7d-27d6-47cd-86a2-30a19e9e3c21\") " Dec 03 11:44:18 crc kubenswrapper[5052]: I1203 11:44:18.239899 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/93c3ba7d-27d6-47cd-86a2-30a19e9e3c21-config-data\") pod \"93c3ba7d-27d6-47cd-86a2-30a19e9e3c21\" (UID: \"93c3ba7d-27d6-47cd-86a2-30a19e9e3c21\") " Dec 03 11:44:18 crc kubenswrapper[5052]: I1203 11:44:18.239915 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93c3ba7d-27d6-47cd-86a2-30a19e9e3c21-logs" (OuterVolumeSpecName: "logs") pod "93c3ba7d-27d6-47cd-86a2-30a19e9e3c21" (UID: "93c3ba7d-27d6-47cd-86a2-30a19e9e3c21"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:44:18 crc kubenswrapper[5052]: I1203 11:44:18.239971 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/93c3ba7d-27d6-47cd-86a2-30a19e9e3c21-combined-ca-bundle\") pod \"93c3ba7d-27d6-47cd-86a2-30a19e9e3c21\" (UID: \"93c3ba7d-27d6-47cd-86a2-30a19e9e3c21\") " Dec 03 11:44:18 crc kubenswrapper[5052]: I1203 11:44:18.240012 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cq5qz\" (UniqueName: \"kubernetes.io/projected/93c3ba7d-27d6-47cd-86a2-30a19e9e3c21-kube-api-access-cq5qz\") pod \"93c3ba7d-27d6-47cd-86a2-30a19e9e3c21\" (UID: \"93c3ba7d-27d6-47cd-86a2-30a19e9e3c21\") " Dec 03 11:44:18 crc kubenswrapper[5052]: I1203 11:44:18.240201 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93c3ba7d-27d6-47cd-86a2-30a19e9e3c21-httpd-run" (OuterVolumeSpecName: "httpd-run") pod "93c3ba7d-27d6-47cd-86a2-30a19e9e3c21" (UID: "93c3ba7d-27d6-47cd-86a2-30a19e9e3c21"). InnerVolumeSpecName "httpd-run". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:44:18 crc kubenswrapper[5052]: I1203 11:44:18.240575 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/93c3ba7d-27d6-47cd-86a2-30a19e9e3c21-logs\") on node \"crc\" DevicePath \"\"" Dec 03 11:44:18 crc kubenswrapper[5052]: I1203 11:44:18.240589 5052 reconciler_common.go:293] "Volume detached for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/93c3ba7d-27d6-47cd-86a2-30a19e9e3c21-httpd-run\") on node \"crc\" DevicePath \"\"" Dec 03 11:44:18 crc kubenswrapper[5052]: I1203 11:44:18.248652 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage04-crc" (OuterVolumeSpecName: "glance") pod "93c3ba7d-27d6-47cd-86a2-30a19e9e3c21" (UID: "93c3ba7d-27d6-47cd-86a2-30a19e9e3c21"). InnerVolumeSpecName "local-storage04-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 11:44:18 crc kubenswrapper[5052]: I1203 11:44:18.248665 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/93c3ba7d-27d6-47cd-86a2-30a19e9e3c21-scripts" (OuterVolumeSpecName: "scripts") pod "93c3ba7d-27d6-47cd-86a2-30a19e9e3c21" (UID: "93c3ba7d-27d6-47cd-86a2-30a19e9e3c21"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:44:18 crc kubenswrapper[5052]: I1203 11:44:18.248725 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/93c3ba7d-27d6-47cd-86a2-30a19e9e3c21-kube-api-access-cq5qz" (OuterVolumeSpecName: "kube-api-access-cq5qz") pod "93c3ba7d-27d6-47cd-86a2-30a19e9e3c21" (UID: "93c3ba7d-27d6-47cd-86a2-30a19e9e3c21"). InnerVolumeSpecName "kube-api-access-cq5qz". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:44:18 crc kubenswrapper[5052]: I1203 11:44:18.281128 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/93c3ba7d-27d6-47cd-86a2-30a19e9e3c21-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "93c3ba7d-27d6-47cd-86a2-30a19e9e3c21" (UID: "93c3ba7d-27d6-47cd-86a2-30a19e9e3c21"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:44:18 crc kubenswrapper[5052]: I1203 11:44:18.296274 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/93c3ba7d-27d6-47cd-86a2-30a19e9e3c21-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "93c3ba7d-27d6-47cd-86a2-30a19e9e3c21" (UID: "93c3ba7d-27d6-47cd-86a2-30a19e9e3c21"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:44:18 crc kubenswrapper[5052]: I1203 11:44:18.300581 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/93c3ba7d-27d6-47cd-86a2-30a19e9e3c21-config-data" (OuterVolumeSpecName: "config-data") pod "93c3ba7d-27d6-47cd-86a2-30a19e9e3c21" (UID: "93c3ba7d-27d6-47cd-86a2-30a19e9e3c21"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:44:18 crc kubenswrapper[5052]: I1203 11:44:18.342404 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/93c3ba7d-27d6-47cd-86a2-30a19e9e3c21-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:44:18 crc kubenswrapper[5052]: I1203 11:44:18.342458 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage04-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage04-crc\") on node \"crc\" " Dec 03 11:44:18 crc kubenswrapper[5052]: I1203 11:44:18.342471 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/93c3ba7d-27d6-47cd-86a2-30a19e9e3c21-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 11:44:18 crc kubenswrapper[5052]: I1203 11:44:18.342481 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/93c3ba7d-27d6-47cd-86a2-30a19e9e3c21-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:44:18 crc kubenswrapper[5052]: I1203 11:44:18.342491 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/93c3ba7d-27d6-47cd-86a2-30a19e9e3c21-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:44:18 crc kubenswrapper[5052]: I1203 11:44:18.342500 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-cq5qz\" (UniqueName: \"kubernetes.io/projected/93c3ba7d-27d6-47cd-86a2-30a19e9e3c21-kube-api-access-cq5qz\") on node \"crc\" DevicePath \"\"" Dec 03 11:44:18 crc kubenswrapper[5052]: I1203 11:44:18.359161 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage04-crc" (UniqueName: "kubernetes.io/local-volume/local-storage04-crc") on node "crc" Dec 03 11:44:18 crc kubenswrapper[5052]: I1203 11:44:18.423168 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-db-sync-qv4bk" event={"ID":"3d4a523e-547f-4d7a-920f-7a24ca45f183","Type":"ContainerStarted","Data":"2beb36e19d46bc6b746913b09b005f085e7e4ab7f58b32b97740b3059d80cf36"} Dec 03 11:44:18 crc kubenswrapper[5052]: I1203 11:44:18.424758 5052 generic.go:334] "Generic (PLEG): container finished" podID="35ce22c2-a534-49af-9ba3-3282dffbaf95" containerID="e8f9201ec877119fe422d30c220b1d235ac9fd50b57cec45153ccb587583396d" exitCode=0 Dec 03 11:44:18 crc kubenswrapper[5052]: I1203 11:44:18.424831 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-db-sync-jlwkf" event={"ID":"35ce22c2-a534-49af-9ba3-3282dffbaf95","Type":"ContainerDied","Data":"e8f9201ec877119fe422d30c220b1d235ac9fd50b57cec45153ccb587583396d"} Dec 03 11:44:18 crc kubenswrapper[5052]: I1203 11:44:18.426480 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"fb3a323e-09aa-47ac-b593-e1c6e9b898e5","Type":"ContainerStarted","Data":"9bf88bfd0d0e6f8a144f303520d8196838052ec8c253b8b04069315b384e953d"} Dec 03 11:44:18 crc kubenswrapper[5052]: I1203 11:44:18.428535 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"70097c03-11be-4f1a-b8a8-8cef71b8e8ce","Type":"ContainerStarted","Data":"c8a3c987aee9927249cd04f5476c21d1e934f742023404541be0cf3f7dae0ef0"} Dec 03 11:44:18 crc kubenswrapper[5052]: I1203 11:44:18.428783 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-external-api-0" podUID="70097c03-11be-4f1a-b8a8-8cef71b8e8ce" containerName="glance-log" containerID="cri-o://60e01155d35f8c06fef45b2743f3cfe9b7364e3fa8bf67f6e5502383b4632ffc" gracePeriod=30 Dec 03 11:44:18 crc kubenswrapper[5052]: I1203 11:44:18.429023 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-external-api-0" podUID="70097c03-11be-4f1a-b8a8-8cef71b8e8ce" containerName="glance-httpd" containerID="cri-o://c8a3c987aee9927249cd04f5476c21d1e934f742023404541be0cf3f7dae0ef0" gracePeriod=30 Dec 03 11:44:18 crc kubenswrapper[5052]: I1203 11:44:18.432023 5052 generic.go:334] "Generic (PLEG): container finished" podID="93c3ba7d-27d6-47cd-86a2-30a19e9e3c21" containerID="8ca2c0a229a587915dd55157616554b37d810216ac8f67032c79c498803a7d99" exitCode=143 Dec 03 11:44:18 crc kubenswrapper[5052]: I1203 11:44:18.432043 5052 generic.go:334] "Generic (PLEG): container finished" podID="93c3ba7d-27d6-47cd-86a2-30a19e9e3c21" containerID="aacc584f4b653c45815076ff6ec9755843746f3d7a69d83ffcfdca689b761b08" exitCode=143 Dec 03 11:44:18 crc kubenswrapper[5052]: I1203 11:44:18.432195 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"93c3ba7d-27d6-47cd-86a2-30a19e9e3c21","Type":"ContainerDied","Data":"8ca2c0a229a587915dd55157616554b37d810216ac8f67032c79c498803a7d99"} Dec 03 11:44:18 crc kubenswrapper[5052]: I1203 11:44:18.432210 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:44:18 crc kubenswrapper[5052]: I1203 11:44:18.432228 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"93c3ba7d-27d6-47cd-86a2-30a19e9e3c21","Type":"ContainerDied","Data":"aacc584f4b653c45815076ff6ec9755843746f3d7a69d83ffcfdca689b761b08"} Dec 03 11:44:18 crc kubenswrapper[5052]: I1203 11:44:18.432239 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"93c3ba7d-27d6-47cd-86a2-30a19e9e3c21","Type":"ContainerDied","Data":"644e3f8433c6f88d0b05b42f1e18a222f48d7e71a183cb71b8171bc45ae8b802"} Dec 03 11:44:18 crc kubenswrapper[5052]: I1203 11:44:18.432254 5052 scope.go:117] "RemoveContainer" containerID="8ca2c0a229a587915dd55157616554b37d810216ac8f67032c79c498803a7d99" Dec 03 11:44:18 crc kubenswrapper[5052]: I1203 11:44:18.444830 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage04-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage04-crc\") on node \"crc\" DevicePath \"\"" Dec 03 11:44:18 crc kubenswrapper[5052]: I1203 11:44:18.454561 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/cinder-db-sync-qv4bk" podStartSLOduration=5.450492583 podStartE2EDuration="5.450492583s" podCreationTimestamp="2025-12-03 11:44:13 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:44:18.445043296 +0000 UTC m=+1449.033285060" watchObservedRunningTime="2025-12-03 11:44:18.450492583 +0000 UTC m=+1449.038734367" Dec 03 11:44:18 crc kubenswrapper[5052]: I1203 11:44:18.510876 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/glance-default-external-api-0" podStartSLOduration=5.510855083 podStartE2EDuration="5.510855083s" podCreationTimestamp="2025-12-03 11:44:13 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:44:18.494222634 +0000 UTC m=+1449.082464398" watchObservedRunningTime="2025-12-03 11:44:18.510855083 +0000 UTC m=+1449.099096847" Dec 03 11:44:18 crc kubenswrapper[5052]: I1203 11:44:18.514494 5052 scope.go:117] "RemoveContainer" containerID="aacc584f4b653c45815076ff6ec9755843746f3d7a69d83ffcfdca689b761b08" Dec 03 11:44:18 crc kubenswrapper[5052]: I1203 11:44:18.531929 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 11:44:18 crc kubenswrapper[5052]: I1203 11:44:18.538879 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 11:44:18 crc kubenswrapper[5052]: I1203 11:44:18.561493 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 11:44:18 crc kubenswrapper[5052]: E1203 11:44:18.561910 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="93c3ba7d-27d6-47cd-86a2-30a19e9e3c21" containerName="glance-httpd" Dec 03 11:44:18 crc kubenswrapper[5052]: I1203 11:44:18.561929 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="93c3ba7d-27d6-47cd-86a2-30a19e9e3c21" containerName="glance-httpd" Dec 03 11:44:18 crc kubenswrapper[5052]: E1203 11:44:18.561938 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="93c3ba7d-27d6-47cd-86a2-30a19e9e3c21" containerName="glance-log" Dec 03 11:44:18 crc kubenswrapper[5052]: I1203 11:44:18.561945 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="93c3ba7d-27d6-47cd-86a2-30a19e9e3c21" containerName="glance-log" Dec 03 11:44:18 crc kubenswrapper[5052]: I1203 11:44:18.562157 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="93c3ba7d-27d6-47cd-86a2-30a19e9e3c21" containerName="glance-log" Dec 03 11:44:18 crc kubenswrapper[5052]: I1203 11:44:18.562181 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="93c3ba7d-27d6-47cd-86a2-30a19e9e3c21" containerName="glance-httpd" Dec 03 11:44:18 crc kubenswrapper[5052]: I1203 11:44:18.563280 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:44:18 crc kubenswrapper[5052]: I1203 11:44:18.570866 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-glance-default-internal-svc" Dec 03 11:44:18 crc kubenswrapper[5052]: I1203 11:44:18.571057 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-default-internal-config-data" Dec 03 11:44:18 crc kubenswrapper[5052]: I1203 11:44:18.590971 5052 scope.go:117] "RemoveContainer" containerID="8ca2c0a229a587915dd55157616554b37d810216ac8f67032c79c498803a7d99" Dec 03 11:44:18 crc kubenswrapper[5052]: E1203 11:44:18.592152 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8ca2c0a229a587915dd55157616554b37d810216ac8f67032c79c498803a7d99\": container with ID starting with 8ca2c0a229a587915dd55157616554b37d810216ac8f67032c79c498803a7d99 not found: ID does not exist" containerID="8ca2c0a229a587915dd55157616554b37d810216ac8f67032c79c498803a7d99" Dec 03 11:44:18 crc kubenswrapper[5052]: I1203 11:44:18.592190 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8ca2c0a229a587915dd55157616554b37d810216ac8f67032c79c498803a7d99"} err="failed to get container status \"8ca2c0a229a587915dd55157616554b37d810216ac8f67032c79c498803a7d99\": rpc error: code = NotFound desc = could not find container \"8ca2c0a229a587915dd55157616554b37d810216ac8f67032c79c498803a7d99\": container with ID starting with 8ca2c0a229a587915dd55157616554b37d810216ac8f67032c79c498803a7d99 not found: ID does not exist" Dec 03 11:44:18 crc kubenswrapper[5052]: I1203 11:44:18.592216 5052 scope.go:117] "RemoveContainer" containerID="aacc584f4b653c45815076ff6ec9755843746f3d7a69d83ffcfdca689b761b08" Dec 03 11:44:18 crc kubenswrapper[5052]: I1203 11:44:18.596009 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 11:44:18 crc kubenswrapper[5052]: E1203 11:44:18.617019 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aacc584f4b653c45815076ff6ec9755843746f3d7a69d83ffcfdca689b761b08\": container with ID starting with aacc584f4b653c45815076ff6ec9755843746f3d7a69d83ffcfdca689b761b08 not found: ID does not exist" containerID="aacc584f4b653c45815076ff6ec9755843746f3d7a69d83ffcfdca689b761b08" Dec 03 11:44:18 crc kubenswrapper[5052]: I1203 11:44:18.617066 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aacc584f4b653c45815076ff6ec9755843746f3d7a69d83ffcfdca689b761b08"} err="failed to get container status \"aacc584f4b653c45815076ff6ec9755843746f3d7a69d83ffcfdca689b761b08\": rpc error: code = NotFound desc = could not find container \"aacc584f4b653c45815076ff6ec9755843746f3d7a69d83ffcfdca689b761b08\": container with ID starting with aacc584f4b653c45815076ff6ec9755843746f3d7a69d83ffcfdca689b761b08 not found: ID does not exist" Dec 03 11:44:18 crc kubenswrapper[5052]: I1203 11:44:18.617093 5052 scope.go:117] "RemoveContainer" containerID="8ca2c0a229a587915dd55157616554b37d810216ac8f67032c79c498803a7d99" Dec 03 11:44:18 crc kubenswrapper[5052]: I1203 11:44:18.617404 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8ca2c0a229a587915dd55157616554b37d810216ac8f67032c79c498803a7d99"} err="failed to get container status \"8ca2c0a229a587915dd55157616554b37d810216ac8f67032c79c498803a7d99\": rpc error: code = NotFound desc = could not find container \"8ca2c0a229a587915dd55157616554b37d810216ac8f67032c79c498803a7d99\": container with ID starting with 8ca2c0a229a587915dd55157616554b37d810216ac8f67032c79c498803a7d99 not found: ID does not exist" Dec 03 11:44:18 crc kubenswrapper[5052]: I1203 11:44:18.617438 5052 scope.go:117] "RemoveContainer" containerID="aacc584f4b653c45815076ff6ec9755843746f3d7a69d83ffcfdca689b761b08" Dec 03 11:44:18 crc kubenswrapper[5052]: I1203 11:44:18.617703 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aacc584f4b653c45815076ff6ec9755843746f3d7a69d83ffcfdca689b761b08"} err="failed to get container status \"aacc584f4b653c45815076ff6ec9755843746f3d7a69d83ffcfdca689b761b08\": rpc error: code = NotFound desc = could not find container \"aacc584f4b653c45815076ff6ec9755843746f3d7a69d83ffcfdca689b761b08\": container with ID starting with aacc584f4b653c45815076ff6ec9755843746f3d7a69d83ffcfdca689b761b08 not found: ID does not exist" Dec 03 11:44:18 crc kubenswrapper[5052]: I1203 11:44:18.652471 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage04-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage04-crc\") pod \"glance-default-internal-api-0\" (UID: \"d2650df9-f097-4bb1-84e1-72077a661399\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:44:18 crc kubenswrapper[5052]: I1203 11:44:18.652562 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/d2650df9-f097-4bb1-84e1-72077a661399-config-data\") pod \"glance-default-internal-api-0\" (UID: \"d2650df9-f097-4bb1-84e1-72077a661399\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:44:18 crc kubenswrapper[5052]: I1203 11:44:18.652631 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/d2650df9-f097-4bb1-84e1-72077a661399-httpd-run\") pod \"glance-default-internal-api-0\" (UID: \"d2650df9-f097-4bb1-84e1-72077a661399\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:44:18 crc kubenswrapper[5052]: I1203 11:44:18.652663 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/d2650df9-f097-4bb1-84e1-72077a661399-logs\") pod \"glance-default-internal-api-0\" (UID: \"d2650df9-f097-4bb1-84e1-72077a661399\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:44:18 crc kubenswrapper[5052]: I1203 11:44:18.652709 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/d2650df9-f097-4bb1-84e1-72077a661399-internal-tls-certs\") pod \"glance-default-internal-api-0\" (UID: \"d2650df9-f097-4bb1-84e1-72077a661399\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:44:18 crc kubenswrapper[5052]: I1203 11:44:18.652738 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d2650df9-f097-4bb1-84e1-72077a661399-combined-ca-bundle\") pod \"glance-default-internal-api-0\" (UID: \"d2650df9-f097-4bb1-84e1-72077a661399\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:44:18 crc kubenswrapper[5052]: I1203 11:44:18.652760 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hgqft\" (UniqueName: \"kubernetes.io/projected/d2650df9-f097-4bb1-84e1-72077a661399-kube-api-access-hgqft\") pod \"glance-default-internal-api-0\" (UID: \"d2650df9-f097-4bb1-84e1-72077a661399\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:44:18 crc kubenswrapper[5052]: I1203 11:44:18.652797 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/d2650df9-f097-4bb1-84e1-72077a661399-scripts\") pod \"glance-default-internal-api-0\" (UID: \"d2650df9-f097-4bb1-84e1-72077a661399\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:44:18 crc kubenswrapper[5052]: I1203 11:44:18.755498 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage04-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage04-crc\") pod \"glance-default-internal-api-0\" (UID: \"d2650df9-f097-4bb1-84e1-72077a661399\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:44:18 crc kubenswrapper[5052]: I1203 11:44:18.755574 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/d2650df9-f097-4bb1-84e1-72077a661399-config-data\") pod \"glance-default-internal-api-0\" (UID: \"d2650df9-f097-4bb1-84e1-72077a661399\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:44:18 crc kubenswrapper[5052]: I1203 11:44:18.755618 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/d2650df9-f097-4bb1-84e1-72077a661399-httpd-run\") pod \"glance-default-internal-api-0\" (UID: \"d2650df9-f097-4bb1-84e1-72077a661399\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:44:18 crc kubenswrapper[5052]: I1203 11:44:18.755664 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/d2650df9-f097-4bb1-84e1-72077a661399-logs\") pod \"glance-default-internal-api-0\" (UID: \"d2650df9-f097-4bb1-84e1-72077a661399\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:44:18 crc kubenswrapper[5052]: I1203 11:44:18.755715 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/d2650df9-f097-4bb1-84e1-72077a661399-internal-tls-certs\") pod \"glance-default-internal-api-0\" (UID: \"d2650df9-f097-4bb1-84e1-72077a661399\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:44:18 crc kubenswrapper[5052]: I1203 11:44:18.755748 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d2650df9-f097-4bb1-84e1-72077a661399-combined-ca-bundle\") pod \"glance-default-internal-api-0\" (UID: \"d2650df9-f097-4bb1-84e1-72077a661399\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:44:18 crc kubenswrapper[5052]: I1203 11:44:18.755777 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage04-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage04-crc\") pod \"glance-default-internal-api-0\" (UID: \"d2650df9-f097-4bb1-84e1-72077a661399\") device mount path \"/mnt/openstack/pv04\"" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:44:18 crc kubenswrapper[5052]: I1203 11:44:18.755793 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-hgqft\" (UniqueName: \"kubernetes.io/projected/d2650df9-f097-4bb1-84e1-72077a661399-kube-api-access-hgqft\") pod \"glance-default-internal-api-0\" (UID: \"d2650df9-f097-4bb1-84e1-72077a661399\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:44:18 crc kubenswrapper[5052]: I1203 11:44:18.755833 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/d2650df9-f097-4bb1-84e1-72077a661399-scripts\") pod \"glance-default-internal-api-0\" (UID: \"d2650df9-f097-4bb1-84e1-72077a661399\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:44:18 crc kubenswrapper[5052]: I1203 11:44:18.756406 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/d2650df9-f097-4bb1-84e1-72077a661399-httpd-run\") pod \"glance-default-internal-api-0\" (UID: \"d2650df9-f097-4bb1-84e1-72077a661399\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:44:18 crc kubenswrapper[5052]: I1203 11:44:18.756674 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/d2650df9-f097-4bb1-84e1-72077a661399-logs\") pod \"glance-default-internal-api-0\" (UID: \"d2650df9-f097-4bb1-84e1-72077a661399\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:44:18 crc kubenswrapper[5052]: I1203 11:44:18.761795 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d2650df9-f097-4bb1-84e1-72077a661399-combined-ca-bundle\") pod \"glance-default-internal-api-0\" (UID: \"d2650df9-f097-4bb1-84e1-72077a661399\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:44:18 crc kubenswrapper[5052]: I1203 11:44:18.763311 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/d2650df9-f097-4bb1-84e1-72077a661399-config-data\") pod \"glance-default-internal-api-0\" (UID: \"d2650df9-f097-4bb1-84e1-72077a661399\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:44:18 crc kubenswrapper[5052]: I1203 11:44:18.766737 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/d2650df9-f097-4bb1-84e1-72077a661399-scripts\") pod \"glance-default-internal-api-0\" (UID: \"d2650df9-f097-4bb1-84e1-72077a661399\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:44:18 crc kubenswrapper[5052]: I1203 11:44:18.768633 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/d2650df9-f097-4bb1-84e1-72077a661399-internal-tls-certs\") pod \"glance-default-internal-api-0\" (UID: \"d2650df9-f097-4bb1-84e1-72077a661399\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:44:18 crc kubenswrapper[5052]: I1203 11:44:18.772603 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-hgqft\" (UniqueName: \"kubernetes.io/projected/d2650df9-f097-4bb1-84e1-72077a661399-kube-api-access-hgqft\") pod \"glance-default-internal-api-0\" (UID: \"d2650df9-f097-4bb1-84e1-72077a661399\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:44:18 crc kubenswrapper[5052]: I1203 11:44:18.800078 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage04-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage04-crc\") pod \"glance-default-internal-api-0\" (UID: \"d2650df9-f097-4bb1-84e1-72077a661399\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:44:18 crc kubenswrapper[5052]: I1203 11:44:18.895018 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-db-sync-gmcqn" Dec 03 11:44:18 crc kubenswrapper[5052]: I1203 11:44:18.901367 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:44:18 crc kubenswrapper[5052]: I1203 11:44:18.960001 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/189bc36c-9345-4a45-92ce-1ba812535b54-combined-ca-bundle\") pod \"189bc36c-9345-4a45-92ce-1ba812535b54\" (UID: \"189bc36c-9345-4a45-92ce-1ba812535b54\") " Dec 03 11:44:18 crc kubenswrapper[5052]: I1203 11:44:18.960343 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vkwnl\" (UniqueName: \"kubernetes.io/projected/189bc36c-9345-4a45-92ce-1ba812535b54-kube-api-access-vkwnl\") pod \"189bc36c-9345-4a45-92ce-1ba812535b54\" (UID: \"189bc36c-9345-4a45-92ce-1ba812535b54\") " Dec 03 11:44:18 crc kubenswrapper[5052]: I1203 11:44:18.960472 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/189bc36c-9345-4a45-92ce-1ba812535b54-db-sync-config-data\") pod \"189bc36c-9345-4a45-92ce-1ba812535b54\" (UID: \"189bc36c-9345-4a45-92ce-1ba812535b54\") " Dec 03 11:44:19 crc kubenswrapper[5052]: I1203 11:44:19.017864 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/189bc36c-9345-4a45-92ce-1ba812535b54-db-sync-config-data" (OuterVolumeSpecName: "db-sync-config-data") pod "189bc36c-9345-4a45-92ce-1ba812535b54" (UID: "189bc36c-9345-4a45-92ce-1ba812535b54"). InnerVolumeSpecName "db-sync-config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:44:19 crc kubenswrapper[5052]: I1203 11:44:19.038421 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/189bc36c-9345-4a45-92ce-1ba812535b54-kube-api-access-vkwnl" (OuterVolumeSpecName: "kube-api-access-vkwnl") pod "189bc36c-9345-4a45-92ce-1ba812535b54" (UID: "189bc36c-9345-4a45-92ce-1ba812535b54"). InnerVolumeSpecName "kube-api-access-vkwnl". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:44:19 crc kubenswrapper[5052]: I1203 11:44:19.042630 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/189bc36c-9345-4a45-92ce-1ba812535b54-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "189bc36c-9345-4a45-92ce-1ba812535b54" (UID: "189bc36c-9345-4a45-92ce-1ba812535b54"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:44:19 crc kubenswrapper[5052]: I1203 11:44:19.068791 5052 reconciler_common.go:293] "Volume detached for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/189bc36c-9345-4a45-92ce-1ba812535b54-db-sync-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:44:19 crc kubenswrapper[5052]: I1203 11:44:19.068943 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/189bc36c-9345-4a45-92ce-1ba812535b54-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:44:19 crc kubenswrapper[5052]: I1203 11:44:19.069029 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-vkwnl\" (UniqueName: \"kubernetes.io/projected/189bc36c-9345-4a45-92ce-1ba812535b54-kube-api-access-vkwnl\") on node \"crc\" DevicePath \"\"" Dec 03 11:44:19 crc kubenswrapper[5052]: I1203 11:44:19.173217 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:44:19 crc kubenswrapper[5052]: I1203 11:44:19.274528 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/70097c03-11be-4f1a-b8a8-8cef71b8e8ce-scripts\") pod \"70097c03-11be-4f1a-b8a8-8cef71b8e8ce\" (UID: \"70097c03-11be-4f1a-b8a8-8cef71b8e8ce\") " Dec 03 11:44:19 crc kubenswrapper[5052]: I1203 11:44:19.274797 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/70097c03-11be-4f1a-b8a8-8cef71b8e8ce-config-data\") pod \"70097c03-11be-4f1a-b8a8-8cef71b8e8ce\" (UID: \"70097c03-11be-4f1a-b8a8-8cef71b8e8ce\") " Dec 03 11:44:19 crc kubenswrapper[5052]: I1203 11:44:19.274848 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/70097c03-11be-4f1a-b8a8-8cef71b8e8ce-logs\") pod \"70097c03-11be-4f1a-b8a8-8cef71b8e8ce\" (UID: \"70097c03-11be-4f1a-b8a8-8cef71b8e8ce\") " Dec 03 11:44:19 crc kubenswrapper[5052]: I1203 11:44:19.274881 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sqpj4\" (UniqueName: \"kubernetes.io/projected/70097c03-11be-4f1a-b8a8-8cef71b8e8ce-kube-api-access-sqpj4\") pod \"70097c03-11be-4f1a-b8a8-8cef71b8e8ce\" (UID: \"70097c03-11be-4f1a-b8a8-8cef71b8e8ce\") " Dec 03 11:44:19 crc kubenswrapper[5052]: I1203 11:44:19.274925 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/70097c03-11be-4f1a-b8a8-8cef71b8e8ce-httpd-run\") pod \"70097c03-11be-4f1a-b8a8-8cef71b8e8ce\" (UID: \"70097c03-11be-4f1a-b8a8-8cef71b8e8ce\") " Dec 03 11:44:19 crc kubenswrapper[5052]: I1203 11:44:19.274940 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/70097c03-11be-4f1a-b8a8-8cef71b8e8ce-public-tls-certs\") pod \"70097c03-11be-4f1a-b8a8-8cef71b8e8ce\" (UID: \"70097c03-11be-4f1a-b8a8-8cef71b8e8ce\") " Dec 03 11:44:19 crc kubenswrapper[5052]: I1203 11:44:19.275034 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"glance\" (UniqueName: \"kubernetes.io/local-volume/local-storage15-crc\") pod \"70097c03-11be-4f1a-b8a8-8cef71b8e8ce\" (UID: \"70097c03-11be-4f1a-b8a8-8cef71b8e8ce\") " Dec 03 11:44:19 crc kubenswrapper[5052]: I1203 11:44:19.275066 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/70097c03-11be-4f1a-b8a8-8cef71b8e8ce-combined-ca-bundle\") pod \"70097c03-11be-4f1a-b8a8-8cef71b8e8ce\" (UID: \"70097c03-11be-4f1a-b8a8-8cef71b8e8ce\") " Dec 03 11:44:19 crc kubenswrapper[5052]: I1203 11:44:19.285286 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/70097c03-11be-4f1a-b8a8-8cef71b8e8ce-httpd-run" (OuterVolumeSpecName: "httpd-run") pod "70097c03-11be-4f1a-b8a8-8cef71b8e8ce" (UID: "70097c03-11be-4f1a-b8a8-8cef71b8e8ce"). InnerVolumeSpecName "httpd-run". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:44:19 crc kubenswrapper[5052]: I1203 11:44:19.287104 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/70097c03-11be-4f1a-b8a8-8cef71b8e8ce-logs" (OuterVolumeSpecName: "logs") pod "70097c03-11be-4f1a-b8a8-8cef71b8e8ce" (UID: "70097c03-11be-4f1a-b8a8-8cef71b8e8ce"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:44:19 crc kubenswrapper[5052]: I1203 11:44:19.290270 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/70097c03-11be-4f1a-b8a8-8cef71b8e8ce-scripts" (OuterVolumeSpecName: "scripts") pod "70097c03-11be-4f1a-b8a8-8cef71b8e8ce" (UID: "70097c03-11be-4f1a-b8a8-8cef71b8e8ce"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:44:19 crc kubenswrapper[5052]: I1203 11:44:19.294755 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/70097c03-11be-4f1a-b8a8-8cef71b8e8ce-kube-api-access-sqpj4" (OuterVolumeSpecName: "kube-api-access-sqpj4") pod "70097c03-11be-4f1a-b8a8-8cef71b8e8ce" (UID: "70097c03-11be-4f1a-b8a8-8cef71b8e8ce"). InnerVolumeSpecName "kube-api-access-sqpj4". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:44:19 crc kubenswrapper[5052]: I1203 11:44:19.307445 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage15-crc" (OuterVolumeSpecName: "glance") pod "70097c03-11be-4f1a-b8a8-8cef71b8e8ce" (UID: "70097c03-11be-4f1a-b8a8-8cef71b8e8ce"). InnerVolumeSpecName "local-storage15-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 11:44:19 crc kubenswrapper[5052]: I1203 11:44:19.335442 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/70097c03-11be-4f1a-b8a8-8cef71b8e8ce-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "70097c03-11be-4f1a-b8a8-8cef71b8e8ce" (UID: "70097c03-11be-4f1a-b8a8-8cef71b8e8ce"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:44:19 crc kubenswrapper[5052]: I1203 11:44:19.367010 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/70097c03-11be-4f1a-b8a8-8cef71b8e8ce-config-data" (OuterVolumeSpecName: "config-data") pod "70097c03-11be-4f1a-b8a8-8cef71b8e8ce" (UID: "70097c03-11be-4f1a-b8a8-8cef71b8e8ce"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:44:19 crc kubenswrapper[5052]: I1203 11:44:19.376864 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/70097c03-11be-4f1a-b8a8-8cef71b8e8ce-logs\") on node \"crc\" DevicePath \"\"" Dec 03 11:44:19 crc kubenswrapper[5052]: I1203 11:44:19.376897 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-sqpj4\" (UniqueName: \"kubernetes.io/projected/70097c03-11be-4f1a-b8a8-8cef71b8e8ce-kube-api-access-sqpj4\") on node \"crc\" DevicePath \"\"" Dec 03 11:44:19 crc kubenswrapper[5052]: I1203 11:44:19.376908 5052 reconciler_common.go:293] "Volume detached for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/70097c03-11be-4f1a-b8a8-8cef71b8e8ce-httpd-run\") on node \"crc\" DevicePath \"\"" Dec 03 11:44:19 crc kubenswrapper[5052]: I1203 11:44:19.376936 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage15-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage15-crc\") on node \"crc\" " Dec 03 11:44:19 crc kubenswrapper[5052]: I1203 11:44:19.376949 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/70097c03-11be-4f1a-b8a8-8cef71b8e8ce-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:44:19 crc kubenswrapper[5052]: I1203 11:44:19.376960 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/70097c03-11be-4f1a-b8a8-8cef71b8e8ce-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:44:19 crc kubenswrapper[5052]: I1203 11:44:19.376971 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/70097c03-11be-4f1a-b8a8-8cef71b8e8ce-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:44:19 crc kubenswrapper[5052]: I1203 11:44:19.382930 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/70097c03-11be-4f1a-b8a8-8cef71b8e8ce-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "70097c03-11be-4f1a-b8a8-8cef71b8e8ce" (UID: "70097c03-11be-4f1a-b8a8-8cef71b8e8ce"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:44:19 crc kubenswrapper[5052]: I1203 11:44:19.423093 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage15-crc" (UniqueName: "kubernetes.io/local-volume/local-storage15-crc") on node "crc" Dec 03 11:44:19 crc kubenswrapper[5052]: I1203 11:44:19.449756 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"fb3a323e-09aa-47ac-b593-e1c6e9b898e5","Type":"ContainerStarted","Data":"1349f4ae2e79b5b6366e52f5c0162ec782e07b2a25ad7f647b70280b2223aa0f"} Dec 03 11:44:19 crc kubenswrapper[5052]: I1203 11:44:19.451167 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-db-sync-gmcqn" event={"ID":"189bc36c-9345-4a45-92ce-1ba812535b54","Type":"ContainerDied","Data":"1bd706ca1dd3485e6cea24043865e394cc3ed11e7e47f39bddac8f804a7cc592"} Dec 03 11:44:19 crc kubenswrapper[5052]: I1203 11:44:19.451196 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1bd706ca1dd3485e6cea24043865e394cc3ed11e7e47f39bddac8f804a7cc592" Dec 03 11:44:19 crc kubenswrapper[5052]: I1203 11:44:19.451270 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-db-sync-gmcqn" Dec 03 11:44:19 crc kubenswrapper[5052]: I1203 11:44:19.457477 5052 generic.go:334] "Generic (PLEG): container finished" podID="70097c03-11be-4f1a-b8a8-8cef71b8e8ce" containerID="c8a3c987aee9927249cd04f5476c21d1e934f742023404541be0cf3f7dae0ef0" exitCode=0 Dec 03 11:44:19 crc kubenswrapper[5052]: I1203 11:44:19.457594 5052 generic.go:334] "Generic (PLEG): container finished" podID="70097c03-11be-4f1a-b8a8-8cef71b8e8ce" containerID="60e01155d35f8c06fef45b2743f3cfe9b7364e3fa8bf67f6e5502383b4632ffc" exitCode=143 Dec 03 11:44:19 crc kubenswrapper[5052]: I1203 11:44:19.458783 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:44:19 crc kubenswrapper[5052]: I1203 11:44:19.460100 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"70097c03-11be-4f1a-b8a8-8cef71b8e8ce","Type":"ContainerDied","Data":"c8a3c987aee9927249cd04f5476c21d1e934f742023404541be0cf3f7dae0ef0"} Dec 03 11:44:19 crc kubenswrapper[5052]: I1203 11:44:19.460159 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"70097c03-11be-4f1a-b8a8-8cef71b8e8ce","Type":"ContainerDied","Data":"60e01155d35f8c06fef45b2743f3cfe9b7364e3fa8bf67f6e5502383b4632ffc"} Dec 03 11:44:19 crc kubenswrapper[5052]: I1203 11:44:19.460172 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"70097c03-11be-4f1a-b8a8-8cef71b8e8ce","Type":"ContainerDied","Data":"442dd4192a5767001fe5a9809e9a3ac3b884b549d8e8ce20b12b17737ca8dc51"} Dec 03 11:44:19 crc kubenswrapper[5052]: I1203 11:44:19.460191 5052 scope.go:117] "RemoveContainer" containerID="c8a3c987aee9927249cd04f5476c21d1e934f742023404541be0cf3f7dae0ef0" Dec 03 11:44:19 crc kubenswrapper[5052]: I1203 11:44:19.477237 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 11:44:19 crc kubenswrapper[5052]: I1203 11:44:19.481376 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage15-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage15-crc\") on node \"crc\" DevicePath \"\"" Dec 03 11:44:19 crc kubenswrapper[5052]: I1203 11:44:19.481395 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/70097c03-11be-4f1a-b8a8-8cef71b8e8ce-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 11:44:19 crc kubenswrapper[5052]: I1203 11:44:19.524146 5052 scope.go:117] "RemoveContainer" containerID="60e01155d35f8c06fef45b2743f3cfe9b7364e3fa8bf67f6e5502383b4632ffc" Dec 03 11:44:19 crc kubenswrapper[5052]: I1203 11:44:19.528491 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 11:44:19 crc kubenswrapper[5052]: I1203 11:44:19.545629 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 11:44:19 crc kubenswrapper[5052]: I1203 11:44:19.567055 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 11:44:19 crc kubenswrapper[5052]: E1203 11:44:19.567600 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="70097c03-11be-4f1a-b8a8-8cef71b8e8ce" containerName="glance-httpd" Dec 03 11:44:19 crc kubenswrapper[5052]: I1203 11:44:19.567617 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="70097c03-11be-4f1a-b8a8-8cef71b8e8ce" containerName="glance-httpd" Dec 03 11:44:19 crc kubenswrapper[5052]: E1203 11:44:19.567643 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="70097c03-11be-4f1a-b8a8-8cef71b8e8ce" containerName="glance-log" Dec 03 11:44:19 crc kubenswrapper[5052]: I1203 11:44:19.567651 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="70097c03-11be-4f1a-b8a8-8cef71b8e8ce" containerName="glance-log" Dec 03 11:44:19 crc kubenswrapper[5052]: E1203 11:44:19.567689 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="189bc36c-9345-4a45-92ce-1ba812535b54" containerName="barbican-db-sync" Dec 03 11:44:19 crc kubenswrapper[5052]: I1203 11:44:19.567697 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="189bc36c-9345-4a45-92ce-1ba812535b54" containerName="barbican-db-sync" Dec 03 11:44:19 crc kubenswrapper[5052]: I1203 11:44:19.567894 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="70097c03-11be-4f1a-b8a8-8cef71b8e8ce" containerName="glance-log" Dec 03 11:44:19 crc kubenswrapper[5052]: I1203 11:44:19.567912 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="189bc36c-9345-4a45-92ce-1ba812535b54" containerName="barbican-db-sync" Dec 03 11:44:19 crc kubenswrapper[5052]: I1203 11:44:19.567934 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="70097c03-11be-4f1a-b8a8-8cef71b8e8ce" containerName="glance-httpd" Dec 03 11:44:19 crc kubenswrapper[5052]: I1203 11:44:19.569041 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:44:19 crc kubenswrapper[5052]: I1203 11:44:19.576099 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-default-external-config-data" Dec 03 11:44:19 crc kubenswrapper[5052]: I1203 11:44:19.576274 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-glance-default-public-svc" Dec 03 11:44:19 crc kubenswrapper[5052]: I1203 11:44:19.578430 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 11:44:19 crc kubenswrapper[5052]: I1203 11:44:19.598657 5052 scope.go:117] "RemoveContainer" containerID="c8a3c987aee9927249cd04f5476c21d1e934f742023404541be0cf3f7dae0ef0" Dec 03 11:44:19 crc kubenswrapper[5052]: E1203 11:44:19.604336 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c8a3c987aee9927249cd04f5476c21d1e934f742023404541be0cf3f7dae0ef0\": container with ID starting with c8a3c987aee9927249cd04f5476c21d1e934f742023404541be0cf3f7dae0ef0 not found: ID does not exist" containerID="c8a3c987aee9927249cd04f5476c21d1e934f742023404541be0cf3f7dae0ef0" Dec 03 11:44:19 crc kubenswrapper[5052]: I1203 11:44:19.604400 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c8a3c987aee9927249cd04f5476c21d1e934f742023404541be0cf3f7dae0ef0"} err="failed to get container status \"c8a3c987aee9927249cd04f5476c21d1e934f742023404541be0cf3f7dae0ef0\": rpc error: code = NotFound desc = could not find container \"c8a3c987aee9927249cd04f5476c21d1e934f742023404541be0cf3f7dae0ef0\": container with ID starting with c8a3c987aee9927249cd04f5476c21d1e934f742023404541be0cf3f7dae0ef0 not found: ID does not exist" Dec 03 11:44:19 crc kubenswrapper[5052]: I1203 11:44:19.604427 5052 scope.go:117] "RemoveContainer" containerID="60e01155d35f8c06fef45b2743f3cfe9b7364e3fa8bf67f6e5502383b4632ffc" Dec 03 11:44:19 crc kubenswrapper[5052]: E1203 11:44:19.608358 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"60e01155d35f8c06fef45b2743f3cfe9b7364e3fa8bf67f6e5502383b4632ffc\": container with ID starting with 60e01155d35f8c06fef45b2743f3cfe9b7364e3fa8bf67f6e5502383b4632ffc not found: ID does not exist" containerID="60e01155d35f8c06fef45b2743f3cfe9b7364e3fa8bf67f6e5502383b4632ffc" Dec 03 11:44:19 crc kubenswrapper[5052]: I1203 11:44:19.608401 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"60e01155d35f8c06fef45b2743f3cfe9b7364e3fa8bf67f6e5502383b4632ffc"} err="failed to get container status \"60e01155d35f8c06fef45b2743f3cfe9b7364e3fa8bf67f6e5502383b4632ffc\": rpc error: code = NotFound desc = could not find container \"60e01155d35f8c06fef45b2743f3cfe9b7364e3fa8bf67f6e5502383b4632ffc\": container with ID starting with 60e01155d35f8c06fef45b2743f3cfe9b7364e3fa8bf67f6e5502383b4632ffc not found: ID does not exist" Dec 03 11:44:19 crc kubenswrapper[5052]: I1203 11:44:19.608448 5052 scope.go:117] "RemoveContainer" containerID="c8a3c987aee9927249cd04f5476c21d1e934f742023404541be0cf3f7dae0ef0" Dec 03 11:44:19 crc kubenswrapper[5052]: I1203 11:44:19.610269 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c8a3c987aee9927249cd04f5476c21d1e934f742023404541be0cf3f7dae0ef0"} err="failed to get container status \"c8a3c987aee9927249cd04f5476c21d1e934f742023404541be0cf3f7dae0ef0\": rpc error: code = NotFound desc = could not find container \"c8a3c987aee9927249cd04f5476c21d1e934f742023404541be0cf3f7dae0ef0\": container with ID starting with c8a3c987aee9927249cd04f5476c21d1e934f742023404541be0cf3f7dae0ef0 not found: ID does not exist" Dec 03 11:44:19 crc kubenswrapper[5052]: I1203 11:44:19.610313 5052 scope.go:117] "RemoveContainer" containerID="60e01155d35f8c06fef45b2743f3cfe9b7364e3fa8bf67f6e5502383b4632ffc" Dec 03 11:44:19 crc kubenswrapper[5052]: I1203 11:44:19.610649 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"60e01155d35f8c06fef45b2743f3cfe9b7364e3fa8bf67f6e5502383b4632ffc"} err="failed to get container status \"60e01155d35f8c06fef45b2743f3cfe9b7364e3fa8bf67f6e5502383b4632ffc\": rpc error: code = NotFound desc = could not find container \"60e01155d35f8c06fef45b2743f3cfe9b7364e3fa8bf67f6e5502383b4632ffc\": container with ID starting with 60e01155d35f8c06fef45b2743f3cfe9b7364e3fa8bf67f6e5502383b4632ffc not found: ID does not exist" Dec 03 11:44:19 crc kubenswrapper[5052]: I1203 11:44:19.693527 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pbh2k\" (UniqueName: \"kubernetes.io/projected/1303e563-d0c3-4f9c-84b7-3e4fdeba3c6f-kube-api-access-pbh2k\") pod \"glance-default-external-api-0\" (UID: \"1303e563-d0c3-4f9c-84b7-3e4fdeba3c6f\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:44:19 crc kubenswrapper[5052]: I1203 11:44:19.693607 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1303e563-d0c3-4f9c-84b7-3e4fdeba3c6f-combined-ca-bundle\") pod \"glance-default-external-api-0\" (UID: \"1303e563-d0c3-4f9c-84b7-3e4fdeba3c6f\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:44:19 crc kubenswrapper[5052]: I1203 11:44:19.693637 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/1303e563-d0c3-4f9c-84b7-3e4fdeba3c6f-config-data\") pod \"glance-default-external-api-0\" (UID: \"1303e563-d0c3-4f9c-84b7-3e4fdeba3c6f\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:44:19 crc kubenswrapper[5052]: I1203 11:44:19.693680 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/1303e563-d0c3-4f9c-84b7-3e4fdeba3c6f-public-tls-certs\") pod \"glance-default-external-api-0\" (UID: \"1303e563-d0c3-4f9c-84b7-3e4fdeba3c6f\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:44:19 crc kubenswrapper[5052]: I1203 11:44:19.693758 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/1303e563-d0c3-4f9c-84b7-3e4fdeba3c6f-scripts\") pod \"glance-default-external-api-0\" (UID: \"1303e563-d0c3-4f9c-84b7-3e4fdeba3c6f\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:44:19 crc kubenswrapper[5052]: I1203 11:44:19.693794 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/1303e563-d0c3-4f9c-84b7-3e4fdeba3c6f-httpd-run\") pod \"glance-default-external-api-0\" (UID: \"1303e563-d0c3-4f9c-84b7-3e4fdeba3c6f\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:44:19 crc kubenswrapper[5052]: I1203 11:44:19.693847 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage15-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage15-crc\") pod \"glance-default-external-api-0\" (UID: \"1303e563-d0c3-4f9c-84b7-3e4fdeba3c6f\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:44:19 crc kubenswrapper[5052]: I1203 11:44:19.693888 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/1303e563-d0c3-4f9c-84b7-3e4fdeba3c6f-logs\") pod \"glance-default-external-api-0\" (UID: \"1303e563-d0c3-4f9c-84b7-3e4fdeba3c6f\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:44:19 crc kubenswrapper[5052]: I1203 11:44:19.758560 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="70097c03-11be-4f1a-b8a8-8cef71b8e8ce" path="/var/lib/kubelet/pods/70097c03-11be-4f1a-b8a8-8cef71b8e8ce/volumes" Dec 03 11:44:19 crc kubenswrapper[5052]: I1203 11:44:19.762807 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="93c3ba7d-27d6-47cd-86a2-30a19e9e3c21" path="/var/lib/kubelet/pods/93c3ba7d-27d6-47cd-86a2-30a19e9e3c21/volumes" Dec 03 11:44:19 crc kubenswrapper[5052]: I1203 11:44:19.763619 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/barbican-worker-79689f8957-qmkdm"] Dec 03 11:44:19 crc kubenswrapper[5052]: I1203 11:44:19.767222 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-worker-79689f8957-qmkdm" Dec 03 11:44:19 crc kubenswrapper[5052]: I1203 11:44:19.773009 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"barbican-barbican-dockercfg-ft6vf" Dec 03 11:44:19 crc kubenswrapper[5052]: I1203 11:44:19.773247 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"barbican-config-data" Dec 03 11:44:19 crc kubenswrapper[5052]: I1203 11:44:19.777711 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"barbican-worker-config-data" Dec 03 11:44:19 crc kubenswrapper[5052]: I1203 11:44:19.783580 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/barbican-keystone-listener-6794d844d6-f4tr7"] Dec 03 11:44:19 crc kubenswrapper[5052]: I1203 11:44:19.785389 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-keystone-listener-6794d844d6-f4tr7" Dec 03 11:44:19 crc kubenswrapper[5052]: I1203 11:44:19.795089 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/1303e563-d0c3-4f9c-84b7-3e4fdeba3c6f-logs\") pod \"glance-default-external-api-0\" (UID: \"1303e563-d0c3-4f9c-84b7-3e4fdeba3c6f\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:44:19 crc kubenswrapper[5052]: I1203 11:44:19.795386 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/cadac9a6-1d82-403a-b7a0-692a424dd0e2-logs\") pod \"barbican-worker-79689f8957-qmkdm\" (UID: \"cadac9a6-1d82-403a-b7a0-692a424dd0e2\") " pod="openstack-kuttl-tests/barbican-worker-79689f8957-qmkdm" Dec 03 11:44:19 crc kubenswrapper[5052]: I1203 11:44:19.795422 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-pbh2k\" (UniqueName: \"kubernetes.io/projected/1303e563-d0c3-4f9c-84b7-3e4fdeba3c6f-kube-api-access-pbh2k\") pod \"glance-default-external-api-0\" (UID: \"1303e563-d0c3-4f9c-84b7-3e4fdeba3c6f\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:44:19 crc kubenswrapper[5052]: I1203 11:44:19.795459 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jwbmx\" (UniqueName: \"kubernetes.io/projected/cadac9a6-1d82-403a-b7a0-692a424dd0e2-kube-api-access-jwbmx\") pod \"barbican-worker-79689f8957-qmkdm\" (UID: \"cadac9a6-1d82-403a-b7a0-692a424dd0e2\") " pod="openstack-kuttl-tests/barbican-worker-79689f8957-qmkdm" Dec 03 11:44:19 crc kubenswrapper[5052]: I1203 11:44:19.795479 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1303e563-d0c3-4f9c-84b7-3e4fdeba3c6f-combined-ca-bundle\") pod \"glance-default-external-api-0\" (UID: \"1303e563-d0c3-4f9c-84b7-3e4fdeba3c6f\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:44:19 crc kubenswrapper[5052]: I1203 11:44:19.795518 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/1303e563-d0c3-4f9c-84b7-3e4fdeba3c6f-config-data\") pod \"glance-default-external-api-0\" (UID: \"1303e563-d0c3-4f9c-84b7-3e4fdeba3c6f\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:44:19 crc kubenswrapper[5052]: I1203 11:44:19.795543 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/1303e563-d0c3-4f9c-84b7-3e4fdeba3c6f-public-tls-certs\") pod \"glance-default-external-api-0\" (UID: \"1303e563-d0c3-4f9c-84b7-3e4fdeba3c6f\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:44:19 crc kubenswrapper[5052]: I1203 11:44:19.795596 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/1303e563-d0c3-4f9c-84b7-3e4fdeba3c6f-scripts\") pod \"glance-default-external-api-0\" (UID: \"1303e563-d0c3-4f9c-84b7-3e4fdeba3c6f\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:44:19 crc kubenswrapper[5052]: I1203 11:44:19.795631 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/1303e563-d0c3-4f9c-84b7-3e4fdeba3c6f-httpd-run\") pod \"glance-default-external-api-0\" (UID: \"1303e563-d0c3-4f9c-84b7-3e4fdeba3c6f\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:44:19 crc kubenswrapper[5052]: I1203 11:44:19.795651 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/cadac9a6-1d82-403a-b7a0-692a424dd0e2-combined-ca-bundle\") pod \"barbican-worker-79689f8957-qmkdm\" (UID: \"cadac9a6-1d82-403a-b7a0-692a424dd0e2\") " pod="openstack-kuttl-tests/barbican-worker-79689f8957-qmkdm" Dec 03 11:44:19 crc kubenswrapper[5052]: I1203 11:44:19.795699 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/cadac9a6-1d82-403a-b7a0-692a424dd0e2-config-data-custom\") pod \"barbican-worker-79689f8957-qmkdm\" (UID: \"cadac9a6-1d82-403a-b7a0-692a424dd0e2\") " pod="openstack-kuttl-tests/barbican-worker-79689f8957-qmkdm" Dec 03 11:44:19 crc kubenswrapper[5052]: I1203 11:44:19.795718 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/cadac9a6-1d82-403a-b7a0-692a424dd0e2-config-data\") pod \"barbican-worker-79689f8957-qmkdm\" (UID: \"cadac9a6-1d82-403a-b7a0-692a424dd0e2\") " pod="openstack-kuttl-tests/barbican-worker-79689f8957-qmkdm" Dec 03 11:44:19 crc kubenswrapper[5052]: I1203 11:44:19.795747 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage15-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage15-crc\") pod \"glance-default-external-api-0\" (UID: \"1303e563-d0c3-4f9c-84b7-3e4fdeba3c6f\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:44:19 crc kubenswrapper[5052]: I1203 11:44:19.796000 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage15-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage15-crc\") pod \"glance-default-external-api-0\" (UID: \"1303e563-d0c3-4f9c-84b7-3e4fdeba3c6f\") device mount path \"/mnt/openstack/pv15\"" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:44:19 crc kubenswrapper[5052]: I1203 11:44:19.798382 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/1303e563-d0c3-4f9c-84b7-3e4fdeba3c6f-logs\") pod \"glance-default-external-api-0\" (UID: \"1303e563-d0c3-4f9c-84b7-3e4fdeba3c6f\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:44:19 crc kubenswrapper[5052]: I1203 11:44:19.799142 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/1303e563-d0c3-4f9c-84b7-3e4fdeba3c6f-httpd-run\") pod \"glance-default-external-api-0\" (UID: \"1303e563-d0c3-4f9c-84b7-3e4fdeba3c6f\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:44:19 crc kubenswrapper[5052]: I1203 11:44:19.803241 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/1303e563-d0c3-4f9c-84b7-3e4fdeba3c6f-config-data\") pod \"glance-default-external-api-0\" (UID: \"1303e563-d0c3-4f9c-84b7-3e4fdeba3c6f\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:44:19 crc kubenswrapper[5052]: I1203 11:44:19.804441 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/1303e563-d0c3-4f9c-84b7-3e4fdeba3c6f-public-tls-certs\") pod \"glance-default-external-api-0\" (UID: \"1303e563-d0c3-4f9c-84b7-3e4fdeba3c6f\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:44:19 crc kubenswrapper[5052]: I1203 11:44:19.816883 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"barbican-keystone-listener-config-data" Dec 03 11:44:19 crc kubenswrapper[5052]: I1203 11:44:19.818584 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/1303e563-d0c3-4f9c-84b7-3e4fdeba3c6f-scripts\") pod \"glance-default-external-api-0\" (UID: \"1303e563-d0c3-4f9c-84b7-3e4fdeba3c6f\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:44:19 crc kubenswrapper[5052]: I1203 11:44:19.823032 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1303e563-d0c3-4f9c-84b7-3e4fdeba3c6f-combined-ca-bundle\") pod \"glance-default-external-api-0\" (UID: \"1303e563-d0c3-4f9c-84b7-3e4fdeba3c6f\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:44:19 crc kubenswrapper[5052]: I1203 11:44:19.829666 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-pbh2k\" (UniqueName: \"kubernetes.io/projected/1303e563-d0c3-4f9c-84b7-3e4fdeba3c6f-kube-api-access-pbh2k\") pod \"glance-default-external-api-0\" (UID: \"1303e563-d0c3-4f9c-84b7-3e4fdeba3c6f\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:44:19 crc kubenswrapper[5052]: I1203 11:44:19.838187 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-worker-79689f8957-qmkdm"] Dec 03 11:44:19 crc kubenswrapper[5052]: I1203 11:44:19.845748 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage15-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage15-crc\") pod \"glance-default-external-api-0\" (UID: \"1303e563-d0c3-4f9c-84b7-3e4fdeba3c6f\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:44:19 crc kubenswrapper[5052]: I1203 11:44:19.903867 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/cadac9a6-1d82-403a-b7a0-692a424dd0e2-logs\") pod \"barbican-worker-79689f8957-qmkdm\" (UID: \"cadac9a6-1d82-403a-b7a0-692a424dd0e2\") " pod="openstack-kuttl-tests/barbican-worker-79689f8957-qmkdm" Dec 03 11:44:19 crc kubenswrapper[5052]: I1203 11:44:19.903921 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bjktc\" (UniqueName: \"kubernetes.io/projected/d8de865a-0f6c-413d-8a37-3ec4d496c4bb-kube-api-access-bjktc\") pod \"barbican-keystone-listener-6794d844d6-f4tr7\" (UID: \"d8de865a-0f6c-413d-8a37-3ec4d496c4bb\") " pod="openstack-kuttl-tests/barbican-keystone-listener-6794d844d6-f4tr7" Dec 03 11:44:19 crc kubenswrapper[5052]: I1203 11:44:19.903954 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/d8de865a-0f6c-413d-8a37-3ec4d496c4bb-config-data\") pod \"barbican-keystone-listener-6794d844d6-f4tr7\" (UID: \"d8de865a-0f6c-413d-8a37-3ec4d496c4bb\") " pod="openstack-kuttl-tests/barbican-keystone-listener-6794d844d6-f4tr7" Dec 03 11:44:19 crc kubenswrapper[5052]: I1203 11:44:19.903989 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-jwbmx\" (UniqueName: \"kubernetes.io/projected/cadac9a6-1d82-403a-b7a0-692a424dd0e2-kube-api-access-jwbmx\") pod \"barbican-worker-79689f8957-qmkdm\" (UID: \"cadac9a6-1d82-403a-b7a0-692a424dd0e2\") " pod="openstack-kuttl-tests/barbican-worker-79689f8957-qmkdm" Dec 03 11:44:19 crc kubenswrapper[5052]: I1203 11:44:19.904060 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/d8de865a-0f6c-413d-8a37-3ec4d496c4bb-config-data-custom\") pod \"barbican-keystone-listener-6794d844d6-f4tr7\" (UID: \"d8de865a-0f6c-413d-8a37-3ec4d496c4bb\") " pod="openstack-kuttl-tests/barbican-keystone-listener-6794d844d6-f4tr7" Dec 03 11:44:19 crc kubenswrapper[5052]: I1203 11:44:19.904175 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/cadac9a6-1d82-403a-b7a0-692a424dd0e2-combined-ca-bundle\") pod \"barbican-worker-79689f8957-qmkdm\" (UID: \"cadac9a6-1d82-403a-b7a0-692a424dd0e2\") " pod="openstack-kuttl-tests/barbican-worker-79689f8957-qmkdm" Dec 03 11:44:19 crc kubenswrapper[5052]: I1203 11:44:19.904204 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/cadac9a6-1d82-403a-b7a0-692a424dd0e2-config-data-custom\") pod \"barbican-worker-79689f8957-qmkdm\" (UID: \"cadac9a6-1d82-403a-b7a0-692a424dd0e2\") " pod="openstack-kuttl-tests/barbican-worker-79689f8957-qmkdm" Dec 03 11:44:19 crc kubenswrapper[5052]: I1203 11:44:19.904226 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/cadac9a6-1d82-403a-b7a0-692a424dd0e2-config-data\") pod \"barbican-worker-79689f8957-qmkdm\" (UID: \"cadac9a6-1d82-403a-b7a0-692a424dd0e2\") " pod="openstack-kuttl-tests/barbican-worker-79689f8957-qmkdm" Dec 03 11:44:19 crc kubenswrapper[5052]: I1203 11:44:19.903910 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:44:19 crc kubenswrapper[5052]: I1203 11:44:19.905598 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d8de865a-0f6c-413d-8a37-3ec4d496c4bb-combined-ca-bundle\") pod \"barbican-keystone-listener-6794d844d6-f4tr7\" (UID: \"d8de865a-0f6c-413d-8a37-3ec4d496c4bb\") " pod="openstack-kuttl-tests/barbican-keystone-listener-6794d844d6-f4tr7" Dec 03 11:44:19 crc kubenswrapper[5052]: I1203 11:44:19.905640 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/d8de865a-0f6c-413d-8a37-3ec4d496c4bb-logs\") pod \"barbican-keystone-listener-6794d844d6-f4tr7\" (UID: \"d8de865a-0f6c-413d-8a37-3ec4d496c4bb\") " pod="openstack-kuttl-tests/barbican-keystone-listener-6794d844d6-f4tr7" Dec 03 11:44:19 crc kubenswrapper[5052]: I1203 11:44:19.906213 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/cadac9a6-1d82-403a-b7a0-692a424dd0e2-logs\") pod \"barbican-worker-79689f8957-qmkdm\" (UID: \"cadac9a6-1d82-403a-b7a0-692a424dd0e2\") " pod="openstack-kuttl-tests/barbican-worker-79689f8957-qmkdm" Dec 03 11:44:19 crc kubenswrapper[5052]: I1203 11:44:19.915855 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-keystone-listener-6794d844d6-f4tr7"] Dec 03 11:44:19 crc kubenswrapper[5052]: I1203 11:44:19.927591 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/cadac9a6-1d82-403a-b7a0-692a424dd0e2-combined-ca-bundle\") pod \"barbican-worker-79689f8957-qmkdm\" (UID: \"cadac9a6-1d82-403a-b7a0-692a424dd0e2\") " pod="openstack-kuttl-tests/barbican-worker-79689f8957-qmkdm" Dec 03 11:44:19 crc kubenswrapper[5052]: I1203 11:44:19.927685 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/cadac9a6-1d82-403a-b7a0-692a424dd0e2-config-data-custom\") pod \"barbican-worker-79689f8957-qmkdm\" (UID: \"cadac9a6-1d82-403a-b7a0-692a424dd0e2\") " pod="openstack-kuttl-tests/barbican-worker-79689f8957-qmkdm" Dec 03 11:44:19 crc kubenswrapper[5052]: I1203 11:44:19.931352 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/cadac9a6-1d82-403a-b7a0-692a424dd0e2-config-data\") pod \"barbican-worker-79689f8957-qmkdm\" (UID: \"cadac9a6-1d82-403a-b7a0-692a424dd0e2\") " pod="openstack-kuttl-tests/barbican-worker-79689f8957-qmkdm" Dec 03 11:44:19 crc kubenswrapper[5052]: I1203 11:44:19.944704 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-jwbmx\" (UniqueName: \"kubernetes.io/projected/cadac9a6-1d82-403a-b7a0-692a424dd0e2-kube-api-access-jwbmx\") pod \"barbican-worker-79689f8957-qmkdm\" (UID: \"cadac9a6-1d82-403a-b7a0-692a424dd0e2\") " pod="openstack-kuttl-tests/barbican-worker-79689f8957-qmkdm" Dec 03 11:44:19 crc kubenswrapper[5052]: I1203 11:44:19.975574 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/barbican-api-f76947bf6-27nv8"] Dec 03 11:44:19 crc kubenswrapper[5052]: I1203 11:44:19.978055 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-api-f76947bf6-27nv8" Dec 03 11:44:19 crc kubenswrapper[5052]: I1203 11:44:19.980271 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"barbican-api-config-data" Dec 03 11:44:20 crc kubenswrapper[5052]: I1203 11:44:20.014676 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/23b2a11f-0129-474b-9f97-f9401abd3f6b-logs\") pod \"barbican-api-f76947bf6-27nv8\" (UID: \"23b2a11f-0129-474b-9f97-f9401abd3f6b\") " pod="openstack-kuttl-tests/barbican-api-f76947bf6-27nv8" Dec 03 11:44:20 crc kubenswrapper[5052]: I1203 11:44:20.014981 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/23b2a11f-0129-474b-9f97-f9401abd3f6b-combined-ca-bundle\") pod \"barbican-api-f76947bf6-27nv8\" (UID: \"23b2a11f-0129-474b-9f97-f9401abd3f6b\") " pod="openstack-kuttl-tests/barbican-api-f76947bf6-27nv8" Dec 03 11:44:20 crc kubenswrapper[5052]: I1203 11:44:20.015093 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/23b2a11f-0129-474b-9f97-f9401abd3f6b-config-data\") pod \"barbican-api-f76947bf6-27nv8\" (UID: \"23b2a11f-0129-474b-9f97-f9401abd3f6b\") " pod="openstack-kuttl-tests/barbican-api-f76947bf6-27nv8" Dec 03 11:44:20 crc kubenswrapper[5052]: I1203 11:44:20.015192 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d8de865a-0f6c-413d-8a37-3ec4d496c4bb-combined-ca-bundle\") pod \"barbican-keystone-listener-6794d844d6-f4tr7\" (UID: \"d8de865a-0f6c-413d-8a37-3ec4d496c4bb\") " pod="openstack-kuttl-tests/barbican-keystone-listener-6794d844d6-f4tr7" Dec 03 11:44:20 crc kubenswrapper[5052]: I1203 11:44:20.015280 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/d8de865a-0f6c-413d-8a37-3ec4d496c4bb-logs\") pod \"barbican-keystone-listener-6794d844d6-f4tr7\" (UID: \"d8de865a-0f6c-413d-8a37-3ec4d496c4bb\") " pod="openstack-kuttl-tests/barbican-keystone-listener-6794d844d6-f4tr7" Dec 03 11:44:20 crc kubenswrapper[5052]: I1203 11:44:20.015390 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/23b2a11f-0129-474b-9f97-f9401abd3f6b-config-data-custom\") pod \"barbican-api-f76947bf6-27nv8\" (UID: \"23b2a11f-0129-474b-9f97-f9401abd3f6b\") " pod="openstack-kuttl-tests/barbican-api-f76947bf6-27nv8" Dec 03 11:44:20 crc kubenswrapper[5052]: I1203 11:44:20.015494 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mtm6h\" (UniqueName: \"kubernetes.io/projected/23b2a11f-0129-474b-9f97-f9401abd3f6b-kube-api-access-mtm6h\") pod \"barbican-api-f76947bf6-27nv8\" (UID: \"23b2a11f-0129-474b-9f97-f9401abd3f6b\") " pod="openstack-kuttl-tests/barbican-api-f76947bf6-27nv8" Dec 03 11:44:20 crc kubenswrapper[5052]: I1203 11:44:20.015629 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-bjktc\" (UniqueName: \"kubernetes.io/projected/d8de865a-0f6c-413d-8a37-3ec4d496c4bb-kube-api-access-bjktc\") pod \"barbican-keystone-listener-6794d844d6-f4tr7\" (UID: \"d8de865a-0f6c-413d-8a37-3ec4d496c4bb\") " pod="openstack-kuttl-tests/barbican-keystone-listener-6794d844d6-f4tr7" Dec 03 11:44:20 crc kubenswrapper[5052]: I1203 11:44:20.015742 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/d8de865a-0f6c-413d-8a37-3ec4d496c4bb-config-data\") pod \"barbican-keystone-listener-6794d844d6-f4tr7\" (UID: \"d8de865a-0f6c-413d-8a37-3ec4d496c4bb\") " pod="openstack-kuttl-tests/barbican-keystone-listener-6794d844d6-f4tr7" Dec 03 11:44:20 crc kubenswrapper[5052]: I1203 11:44:20.017750 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/d8de865a-0f6c-413d-8a37-3ec4d496c4bb-config-data-custom\") pod \"barbican-keystone-listener-6794d844d6-f4tr7\" (UID: \"d8de865a-0f6c-413d-8a37-3ec4d496c4bb\") " pod="openstack-kuttl-tests/barbican-keystone-listener-6794d844d6-f4tr7" Dec 03 11:44:20 crc kubenswrapper[5052]: I1203 11:44:20.020354 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d8de865a-0f6c-413d-8a37-3ec4d496c4bb-combined-ca-bundle\") pod \"barbican-keystone-listener-6794d844d6-f4tr7\" (UID: \"d8de865a-0f6c-413d-8a37-3ec4d496c4bb\") " pod="openstack-kuttl-tests/barbican-keystone-listener-6794d844d6-f4tr7" Dec 03 11:44:20 crc kubenswrapper[5052]: I1203 11:44:20.016654 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/d8de865a-0f6c-413d-8a37-3ec4d496c4bb-logs\") pod \"barbican-keystone-listener-6794d844d6-f4tr7\" (UID: \"d8de865a-0f6c-413d-8a37-3ec4d496c4bb\") " pod="openstack-kuttl-tests/barbican-keystone-listener-6794d844d6-f4tr7" Dec 03 11:44:20 crc kubenswrapper[5052]: I1203 11:44:20.021703 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/d8de865a-0f6c-413d-8a37-3ec4d496c4bb-config-data\") pod \"barbican-keystone-listener-6794d844d6-f4tr7\" (UID: \"d8de865a-0f6c-413d-8a37-3ec4d496c4bb\") " pod="openstack-kuttl-tests/barbican-keystone-listener-6794d844d6-f4tr7" Dec 03 11:44:20 crc kubenswrapper[5052]: I1203 11:44:20.022708 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/d8de865a-0f6c-413d-8a37-3ec4d496c4bb-config-data-custom\") pod \"barbican-keystone-listener-6794d844d6-f4tr7\" (UID: \"d8de865a-0f6c-413d-8a37-3ec4d496c4bb\") " pod="openstack-kuttl-tests/barbican-keystone-listener-6794d844d6-f4tr7" Dec 03 11:44:20 crc kubenswrapper[5052]: I1203 11:44:20.023265 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-api-f76947bf6-27nv8"] Dec 03 11:44:20 crc kubenswrapper[5052]: I1203 11:44:20.048173 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-bjktc\" (UniqueName: \"kubernetes.io/projected/d8de865a-0f6c-413d-8a37-3ec4d496c4bb-kube-api-access-bjktc\") pod \"barbican-keystone-listener-6794d844d6-f4tr7\" (UID: \"d8de865a-0f6c-413d-8a37-3ec4d496c4bb\") " pod="openstack-kuttl-tests/barbican-keystone-listener-6794d844d6-f4tr7" Dec 03 11:44:20 crc kubenswrapper[5052]: I1203 11:44:20.121470 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/23b2a11f-0129-474b-9f97-f9401abd3f6b-logs\") pod \"barbican-api-f76947bf6-27nv8\" (UID: \"23b2a11f-0129-474b-9f97-f9401abd3f6b\") " pod="openstack-kuttl-tests/barbican-api-f76947bf6-27nv8" Dec 03 11:44:20 crc kubenswrapper[5052]: I1203 11:44:20.121832 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/23b2a11f-0129-474b-9f97-f9401abd3f6b-combined-ca-bundle\") pod \"barbican-api-f76947bf6-27nv8\" (UID: \"23b2a11f-0129-474b-9f97-f9401abd3f6b\") " pod="openstack-kuttl-tests/barbican-api-f76947bf6-27nv8" Dec 03 11:44:20 crc kubenswrapper[5052]: I1203 11:44:20.121857 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/23b2a11f-0129-474b-9f97-f9401abd3f6b-config-data\") pod \"barbican-api-f76947bf6-27nv8\" (UID: \"23b2a11f-0129-474b-9f97-f9401abd3f6b\") " pod="openstack-kuttl-tests/barbican-api-f76947bf6-27nv8" Dec 03 11:44:20 crc kubenswrapper[5052]: I1203 11:44:20.121889 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/23b2a11f-0129-474b-9f97-f9401abd3f6b-config-data-custom\") pod \"barbican-api-f76947bf6-27nv8\" (UID: \"23b2a11f-0129-474b-9f97-f9401abd3f6b\") " pod="openstack-kuttl-tests/barbican-api-f76947bf6-27nv8" Dec 03 11:44:20 crc kubenswrapper[5052]: I1203 11:44:20.121909 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-mtm6h\" (UniqueName: \"kubernetes.io/projected/23b2a11f-0129-474b-9f97-f9401abd3f6b-kube-api-access-mtm6h\") pod \"barbican-api-f76947bf6-27nv8\" (UID: \"23b2a11f-0129-474b-9f97-f9401abd3f6b\") " pod="openstack-kuttl-tests/barbican-api-f76947bf6-27nv8" Dec 03 11:44:20 crc kubenswrapper[5052]: I1203 11:44:20.123052 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/23b2a11f-0129-474b-9f97-f9401abd3f6b-logs\") pod \"barbican-api-f76947bf6-27nv8\" (UID: \"23b2a11f-0129-474b-9f97-f9401abd3f6b\") " pod="openstack-kuttl-tests/barbican-api-f76947bf6-27nv8" Dec 03 11:44:20 crc kubenswrapper[5052]: I1203 11:44:20.127626 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/23b2a11f-0129-474b-9f97-f9401abd3f6b-config-data-custom\") pod \"barbican-api-f76947bf6-27nv8\" (UID: \"23b2a11f-0129-474b-9f97-f9401abd3f6b\") " pod="openstack-kuttl-tests/barbican-api-f76947bf6-27nv8" Dec 03 11:44:20 crc kubenswrapper[5052]: I1203 11:44:20.128103 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/23b2a11f-0129-474b-9f97-f9401abd3f6b-combined-ca-bundle\") pod \"barbican-api-f76947bf6-27nv8\" (UID: \"23b2a11f-0129-474b-9f97-f9401abd3f6b\") " pod="openstack-kuttl-tests/barbican-api-f76947bf6-27nv8" Dec 03 11:44:20 crc kubenswrapper[5052]: I1203 11:44:20.131872 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/23b2a11f-0129-474b-9f97-f9401abd3f6b-config-data\") pod \"barbican-api-f76947bf6-27nv8\" (UID: \"23b2a11f-0129-474b-9f97-f9401abd3f6b\") " pod="openstack-kuttl-tests/barbican-api-f76947bf6-27nv8" Dec 03 11:44:20 crc kubenswrapper[5052]: I1203 11:44:20.181569 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-mtm6h\" (UniqueName: \"kubernetes.io/projected/23b2a11f-0129-474b-9f97-f9401abd3f6b-kube-api-access-mtm6h\") pod \"barbican-api-f76947bf6-27nv8\" (UID: \"23b2a11f-0129-474b-9f97-f9401abd3f6b\") " pod="openstack-kuttl-tests/barbican-api-f76947bf6-27nv8" Dec 03 11:44:20 crc kubenswrapper[5052]: I1203 11:44:20.193328 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-worker-79689f8957-qmkdm" Dec 03 11:44:20 crc kubenswrapper[5052]: I1203 11:44:20.232358 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-keystone-listener-6794d844d6-f4tr7" Dec 03 11:44:20 crc kubenswrapper[5052]: I1203 11:44:20.278100 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-api-f76947bf6-27nv8" Dec 03 11:44:20 crc kubenswrapper[5052]: I1203 11:44:20.351246 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-db-sync-jlwkf" Dec 03 11:44:20 crc kubenswrapper[5052]: I1203 11:44:20.440441 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/35ce22c2-a534-49af-9ba3-3282dffbaf95-combined-ca-bundle\") pod \"35ce22c2-a534-49af-9ba3-3282dffbaf95\" (UID: \"35ce22c2-a534-49af-9ba3-3282dffbaf95\") " Dec 03 11:44:20 crc kubenswrapper[5052]: I1203 11:44:20.452813 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/35ce22c2-a534-49af-9ba3-3282dffbaf95-logs\") pod \"35ce22c2-a534-49af-9ba3-3282dffbaf95\" (UID: \"35ce22c2-a534-49af-9ba3-3282dffbaf95\") " Dec 03 11:44:20 crc kubenswrapper[5052]: I1203 11:44:20.452865 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/35ce22c2-a534-49af-9ba3-3282dffbaf95-scripts\") pod \"35ce22c2-a534-49af-9ba3-3282dffbaf95\" (UID: \"35ce22c2-a534-49af-9ba3-3282dffbaf95\") " Dec 03 11:44:20 crc kubenswrapper[5052]: I1203 11:44:20.452902 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v9tjx\" (UniqueName: \"kubernetes.io/projected/35ce22c2-a534-49af-9ba3-3282dffbaf95-kube-api-access-v9tjx\") pod \"35ce22c2-a534-49af-9ba3-3282dffbaf95\" (UID: \"35ce22c2-a534-49af-9ba3-3282dffbaf95\") " Dec 03 11:44:20 crc kubenswrapper[5052]: I1203 11:44:20.452958 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/35ce22c2-a534-49af-9ba3-3282dffbaf95-config-data\") pod \"35ce22c2-a534-49af-9ba3-3282dffbaf95\" (UID: \"35ce22c2-a534-49af-9ba3-3282dffbaf95\") " Dec 03 11:44:20 crc kubenswrapper[5052]: I1203 11:44:20.455443 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/35ce22c2-a534-49af-9ba3-3282dffbaf95-logs" (OuterVolumeSpecName: "logs") pod "35ce22c2-a534-49af-9ba3-3282dffbaf95" (UID: "35ce22c2-a534-49af-9ba3-3282dffbaf95"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:44:20 crc kubenswrapper[5052]: I1203 11:44:20.460438 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/35ce22c2-a534-49af-9ba3-3282dffbaf95-kube-api-access-v9tjx" (OuterVolumeSpecName: "kube-api-access-v9tjx") pod "35ce22c2-a534-49af-9ba3-3282dffbaf95" (UID: "35ce22c2-a534-49af-9ba3-3282dffbaf95"). InnerVolumeSpecName "kube-api-access-v9tjx". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:44:20 crc kubenswrapper[5052]: I1203 11:44:20.469045 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/35ce22c2-a534-49af-9ba3-3282dffbaf95-scripts" (OuterVolumeSpecName: "scripts") pod "35ce22c2-a534-49af-9ba3-3282dffbaf95" (UID: "35ce22c2-a534-49af-9ba3-3282dffbaf95"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:44:20 crc kubenswrapper[5052]: I1203 11:44:20.507717 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/35ce22c2-a534-49af-9ba3-3282dffbaf95-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "35ce22c2-a534-49af-9ba3-3282dffbaf95" (UID: "35ce22c2-a534-49af-9ba3-3282dffbaf95"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:44:20 crc kubenswrapper[5052]: I1203 11:44:20.517157 5052 generic.go:334] "Generic (PLEG): container finished" podID="b53d06f2-28c8-4c18-b9ce-0300f8b8ef06" containerID="52f339b3d910dbf9c4fb68846eb91d563721d72aa18e7b140f1a773dba41565c" exitCode=0 Dec 03 11:44:20 crc kubenswrapper[5052]: I1203 11:44:20.517234 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-bootstrap-2hjbj" event={"ID":"b53d06f2-28c8-4c18-b9ce-0300f8b8ef06","Type":"ContainerDied","Data":"52f339b3d910dbf9c4fb68846eb91d563721d72aa18e7b140f1a773dba41565c"} Dec 03 11:44:20 crc kubenswrapper[5052]: I1203 11:44:20.527241 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/35ce22c2-a534-49af-9ba3-3282dffbaf95-config-data" (OuterVolumeSpecName: "config-data") pod "35ce22c2-a534-49af-9ba3-3282dffbaf95" (UID: "35ce22c2-a534-49af-9ba3-3282dffbaf95"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:44:20 crc kubenswrapper[5052]: I1203 11:44:20.549617 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-db-sync-5m4kd" event={"ID":"b384ab60-c8a3-488d-8753-22058200e84f","Type":"ContainerDied","Data":"c9c41a712ce14f494b913d7d4165d8d399a912c664179fc011621e5b9c476b1c"} Dec 03 11:44:20 crc kubenswrapper[5052]: I1203 11:44:20.549579 5052 generic.go:334] "Generic (PLEG): container finished" podID="b384ab60-c8a3-488d-8753-22058200e84f" containerID="c9c41a712ce14f494b913d7d4165d8d399a912c664179fc011621e5b9c476b1c" exitCode=0 Dec 03 11:44:20 crc kubenswrapper[5052]: I1203 11:44:20.564965 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-v9tjx\" (UniqueName: \"kubernetes.io/projected/35ce22c2-a534-49af-9ba3-3282dffbaf95-kube-api-access-v9tjx\") on node \"crc\" DevicePath \"\"" Dec 03 11:44:20 crc kubenswrapper[5052]: I1203 11:44:20.564990 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/35ce22c2-a534-49af-9ba3-3282dffbaf95-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:44:20 crc kubenswrapper[5052]: I1203 11:44:20.565223 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-db-sync-jlwkf" event={"ID":"35ce22c2-a534-49af-9ba3-3282dffbaf95","Type":"ContainerDied","Data":"d86c867797f12e395d872e95bc38b91defdb453ba72063d4d65419a6b2398a05"} Dec 03 11:44:20 crc kubenswrapper[5052]: I1203 11:44:20.565269 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d86c867797f12e395d872e95bc38b91defdb453ba72063d4d65419a6b2398a05" Dec 03 11:44:20 crc kubenswrapper[5052]: I1203 11:44:20.565340 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-db-sync-jlwkf" Dec 03 11:44:20 crc kubenswrapper[5052]: I1203 11:44:20.565688 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/35ce22c2-a534-49af-9ba3-3282dffbaf95-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:44:20 crc kubenswrapper[5052]: I1203 11:44:20.565826 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/35ce22c2-a534-49af-9ba3-3282dffbaf95-logs\") on node \"crc\" DevicePath \"\"" Dec 03 11:44:20 crc kubenswrapper[5052]: I1203 11:44:20.568733 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/35ce22c2-a534-49af-9ba3-3282dffbaf95-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:44:20 crc kubenswrapper[5052]: I1203 11:44:20.576649 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"d2650df9-f097-4bb1-84e1-72077a661399","Type":"ContainerStarted","Data":"4db6086957b0da783be909253db938082757fc1b65fe07e81bce17db573e75c2"} Dec 03 11:44:20 crc kubenswrapper[5052]: I1203 11:44:20.637818 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 11:44:20 crc kubenswrapper[5052]: I1203 11:44:20.696990 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/placement-79d5b958cd-cq6hs"] Dec 03 11:44:20 crc kubenswrapper[5052]: E1203 11:44:20.697383 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="35ce22c2-a534-49af-9ba3-3282dffbaf95" containerName="placement-db-sync" Dec 03 11:44:20 crc kubenswrapper[5052]: I1203 11:44:20.697398 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="35ce22c2-a534-49af-9ba3-3282dffbaf95" containerName="placement-db-sync" Dec 03 11:44:20 crc kubenswrapper[5052]: I1203 11:44:20.697788 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="35ce22c2-a534-49af-9ba3-3282dffbaf95" containerName="placement-db-sync" Dec 03 11:44:20 crc kubenswrapper[5052]: I1203 11:44:20.698693 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-79d5b958cd-cq6hs" Dec 03 11:44:20 crc kubenswrapper[5052]: I1203 11:44:20.704905 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"placement-config-data" Dec 03 11:44:20 crc kubenswrapper[5052]: I1203 11:44:20.705011 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"placement-scripts" Dec 03 11:44:20 crc kubenswrapper[5052]: I1203 11:44:20.705068 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"placement-placement-dockercfg-s6bsn" Dec 03 11:44:20 crc kubenswrapper[5052]: I1203 11:44:20.713773 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/placement-79d5b958cd-cq6hs"] Dec 03 11:44:20 crc kubenswrapper[5052]: I1203 11:44:20.770901 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/f8e49a02-823b-4851-9ecf-f2d1696fa201-scripts\") pod \"placement-79d5b958cd-cq6hs\" (UID: \"f8e49a02-823b-4851-9ecf-f2d1696fa201\") " pod="openstack-kuttl-tests/placement-79d5b958cd-cq6hs" Dec 03 11:44:20 crc kubenswrapper[5052]: I1203 11:44:20.771234 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/f8e49a02-823b-4851-9ecf-f2d1696fa201-config-data\") pod \"placement-79d5b958cd-cq6hs\" (UID: \"f8e49a02-823b-4851-9ecf-f2d1696fa201\") " pod="openstack-kuttl-tests/placement-79d5b958cd-cq6hs" Dec 03 11:44:20 crc kubenswrapper[5052]: I1203 11:44:20.771273 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/f8e49a02-823b-4851-9ecf-f2d1696fa201-logs\") pod \"placement-79d5b958cd-cq6hs\" (UID: \"f8e49a02-823b-4851-9ecf-f2d1696fa201\") " pod="openstack-kuttl-tests/placement-79d5b958cd-cq6hs" Dec 03 11:44:20 crc kubenswrapper[5052]: I1203 11:44:20.771322 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9clnj\" (UniqueName: \"kubernetes.io/projected/f8e49a02-823b-4851-9ecf-f2d1696fa201-kube-api-access-9clnj\") pod \"placement-79d5b958cd-cq6hs\" (UID: \"f8e49a02-823b-4851-9ecf-f2d1696fa201\") " pod="openstack-kuttl-tests/placement-79d5b958cd-cq6hs" Dec 03 11:44:20 crc kubenswrapper[5052]: I1203 11:44:20.771349 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f8e49a02-823b-4851-9ecf-f2d1696fa201-combined-ca-bundle\") pod \"placement-79d5b958cd-cq6hs\" (UID: \"f8e49a02-823b-4851-9ecf-f2d1696fa201\") " pod="openstack-kuttl-tests/placement-79d5b958cd-cq6hs" Dec 03 11:44:20 crc kubenswrapper[5052]: I1203 11:44:20.873366 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/f8e49a02-823b-4851-9ecf-f2d1696fa201-logs\") pod \"placement-79d5b958cd-cq6hs\" (UID: \"f8e49a02-823b-4851-9ecf-f2d1696fa201\") " pod="openstack-kuttl-tests/placement-79d5b958cd-cq6hs" Dec 03 11:44:20 crc kubenswrapper[5052]: I1203 11:44:20.873762 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-9clnj\" (UniqueName: \"kubernetes.io/projected/f8e49a02-823b-4851-9ecf-f2d1696fa201-kube-api-access-9clnj\") pod \"placement-79d5b958cd-cq6hs\" (UID: \"f8e49a02-823b-4851-9ecf-f2d1696fa201\") " pod="openstack-kuttl-tests/placement-79d5b958cd-cq6hs" Dec 03 11:44:20 crc kubenswrapper[5052]: I1203 11:44:20.873791 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f8e49a02-823b-4851-9ecf-f2d1696fa201-combined-ca-bundle\") pod \"placement-79d5b958cd-cq6hs\" (UID: \"f8e49a02-823b-4851-9ecf-f2d1696fa201\") " pod="openstack-kuttl-tests/placement-79d5b958cd-cq6hs" Dec 03 11:44:20 crc kubenswrapper[5052]: I1203 11:44:20.873833 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/f8e49a02-823b-4851-9ecf-f2d1696fa201-scripts\") pod \"placement-79d5b958cd-cq6hs\" (UID: \"f8e49a02-823b-4851-9ecf-f2d1696fa201\") " pod="openstack-kuttl-tests/placement-79d5b958cd-cq6hs" Dec 03 11:44:20 crc kubenswrapper[5052]: I1203 11:44:20.873895 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/f8e49a02-823b-4851-9ecf-f2d1696fa201-config-data\") pod \"placement-79d5b958cd-cq6hs\" (UID: \"f8e49a02-823b-4851-9ecf-f2d1696fa201\") " pod="openstack-kuttl-tests/placement-79d5b958cd-cq6hs" Dec 03 11:44:20 crc kubenswrapper[5052]: I1203 11:44:20.876532 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/f8e49a02-823b-4851-9ecf-f2d1696fa201-logs\") pod \"placement-79d5b958cd-cq6hs\" (UID: \"f8e49a02-823b-4851-9ecf-f2d1696fa201\") " pod="openstack-kuttl-tests/placement-79d5b958cd-cq6hs" Dec 03 11:44:20 crc kubenswrapper[5052]: I1203 11:44:20.880975 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/f8e49a02-823b-4851-9ecf-f2d1696fa201-config-data\") pod \"placement-79d5b958cd-cq6hs\" (UID: \"f8e49a02-823b-4851-9ecf-f2d1696fa201\") " pod="openstack-kuttl-tests/placement-79d5b958cd-cq6hs" Dec 03 11:44:20 crc kubenswrapper[5052]: I1203 11:44:20.883559 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f8e49a02-823b-4851-9ecf-f2d1696fa201-combined-ca-bundle\") pod \"placement-79d5b958cd-cq6hs\" (UID: \"f8e49a02-823b-4851-9ecf-f2d1696fa201\") " pod="openstack-kuttl-tests/placement-79d5b958cd-cq6hs" Dec 03 11:44:20 crc kubenswrapper[5052]: I1203 11:44:20.884713 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/f8e49a02-823b-4851-9ecf-f2d1696fa201-scripts\") pod \"placement-79d5b958cd-cq6hs\" (UID: \"f8e49a02-823b-4851-9ecf-f2d1696fa201\") " pod="openstack-kuttl-tests/placement-79d5b958cd-cq6hs" Dec 03 11:44:20 crc kubenswrapper[5052]: I1203 11:44:20.932722 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-9clnj\" (UniqueName: \"kubernetes.io/projected/f8e49a02-823b-4851-9ecf-f2d1696fa201-kube-api-access-9clnj\") pod \"placement-79d5b958cd-cq6hs\" (UID: \"f8e49a02-823b-4851-9ecf-f2d1696fa201\") " pod="openstack-kuttl-tests/placement-79d5b958cd-cq6hs" Dec 03 11:44:21 crc kubenswrapper[5052]: I1203 11:44:21.012823 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/placement-689c6465d6-nsjrx"] Dec 03 11:44:21 crc kubenswrapper[5052]: I1203 11:44:21.014379 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-689c6465d6-nsjrx" Dec 03 11:44:21 crc kubenswrapper[5052]: I1203 11:44:21.022982 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-placement-internal-svc" Dec 03 11:44:21 crc kubenswrapper[5052]: I1203 11:44:21.023697 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-placement-public-svc" Dec 03 11:44:21 crc kubenswrapper[5052]: I1203 11:44:21.031966 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/placement-689c6465d6-nsjrx"] Dec 03 11:44:21 crc kubenswrapper[5052]: I1203 11:44:21.045697 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-79d5b958cd-cq6hs" Dec 03 11:44:21 crc kubenswrapper[5052]: I1203 11:44:21.068597 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-worker-79689f8957-qmkdm"] Dec 03 11:44:21 crc kubenswrapper[5052]: I1203 11:44:21.189311 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/0183ebb5-b91b-443a-9535-2008c9207ab1-internal-tls-certs\") pod \"placement-689c6465d6-nsjrx\" (UID: \"0183ebb5-b91b-443a-9535-2008c9207ab1\") " pod="openstack-kuttl-tests/placement-689c6465d6-nsjrx" Dec 03 11:44:21 crc kubenswrapper[5052]: I1203 11:44:21.189399 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xb5gb\" (UniqueName: \"kubernetes.io/projected/0183ebb5-b91b-443a-9535-2008c9207ab1-kube-api-access-xb5gb\") pod \"placement-689c6465d6-nsjrx\" (UID: \"0183ebb5-b91b-443a-9535-2008c9207ab1\") " pod="openstack-kuttl-tests/placement-689c6465d6-nsjrx" Dec 03 11:44:21 crc kubenswrapper[5052]: I1203 11:44:21.189431 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/0183ebb5-b91b-443a-9535-2008c9207ab1-public-tls-certs\") pod \"placement-689c6465d6-nsjrx\" (UID: \"0183ebb5-b91b-443a-9535-2008c9207ab1\") " pod="openstack-kuttl-tests/placement-689c6465d6-nsjrx" Dec 03 11:44:21 crc kubenswrapper[5052]: I1203 11:44:21.189449 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0183ebb5-b91b-443a-9535-2008c9207ab1-combined-ca-bundle\") pod \"placement-689c6465d6-nsjrx\" (UID: \"0183ebb5-b91b-443a-9535-2008c9207ab1\") " pod="openstack-kuttl-tests/placement-689c6465d6-nsjrx" Dec 03 11:44:21 crc kubenswrapper[5052]: I1203 11:44:21.189471 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/0183ebb5-b91b-443a-9535-2008c9207ab1-logs\") pod \"placement-689c6465d6-nsjrx\" (UID: \"0183ebb5-b91b-443a-9535-2008c9207ab1\") " pod="openstack-kuttl-tests/placement-689c6465d6-nsjrx" Dec 03 11:44:21 crc kubenswrapper[5052]: I1203 11:44:21.189493 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/0183ebb5-b91b-443a-9535-2008c9207ab1-config-data\") pod \"placement-689c6465d6-nsjrx\" (UID: \"0183ebb5-b91b-443a-9535-2008c9207ab1\") " pod="openstack-kuttl-tests/placement-689c6465d6-nsjrx" Dec 03 11:44:21 crc kubenswrapper[5052]: I1203 11:44:21.189543 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/0183ebb5-b91b-443a-9535-2008c9207ab1-scripts\") pod \"placement-689c6465d6-nsjrx\" (UID: \"0183ebb5-b91b-443a-9535-2008c9207ab1\") " pod="openstack-kuttl-tests/placement-689c6465d6-nsjrx" Dec 03 11:44:21 crc kubenswrapper[5052]: I1203 11:44:21.284233 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-api-f76947bf6-27nv8"] Dec 03 11:44:21 crc kubenswrapper[5052]: I1203 11:44:21.290858 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/0183ebb5-b91b-443a-9535-2008c9207ab1-config-data\") pod \"placement-689c6465d6-nsjrx\" (UID: \"0183ebb5-b91b-443a-9535-2008c9207ab1\") " pod="openstack-kuttl-tests/placement-689c6465d6-nsjrx" Dec 03 11:44:21 crc kubenswrapper[5052]: I1203 11:44:21.290928 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/0183ebb5-b91b-443a-9535-2008c9207ab1-scripts\") pod \"placement-689c6465d6-nsjrx\" (UID: \"0183ebb5-b91b-443a-9535-2008c9207ab1\") " pod="openstack-kuttl-tests/placement-689c6465d6-nsjrx" Dec 03 11:44:21 crc kubenswrapper[5052]: I1203 11:44:21.290978 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/0183ebb5-b91b-443a-9535-2008c9207ab1-internal-tls-certs\") pod \"placement-689c6465d6-nsjrx\" (UID: \"0183ebb5-b91b-443a-9535-2008c9207ab1\") " pod="openstack-kuttl-tests/placement-689c6465d6-nsjrx" Dec 03 11:44:21 crc kubenswrapper[5052]: I1203 11:44:21.291112 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-xb5gb\" (UniqueName: \"kubernetes.io/projected/0183ebb5-b91b-443a-9535-2008c9207ab1-kube-api-access-xb5gb\") pod \"placement-689c6465d6-nsjrx\" (UID: \"0183ebb5-b91b-443a-9535-2008c9207ab1\") " pod="openstack-kuttl-tests/placement-689c6465d6-nsjrx" Dec 03 11:44:21 crc kubenswrapper[5052]: I1203 11:44:21.291164 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/0183ebb5-b91b-443a-9535-2008c9207ab1-public-tls-certs\") pod \"placement-689c6465d6-nsjrx\" (UID: \"0183ebb5-b91b-443a-9535-2008c9207ab1\") " pod="openstack-kuttl-tests/placement-689c6465d6-nsjrx" Dec 03 11:44:21 crc kubenswrapper[5052]: I1203 11:44:21.291186 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0183ebb5-b91b-443a-9535-2008c9207ab1-combined-ca-bundle\") pod \"placement-689c6465d6-nsjrx\" (UID: \"0183ebb5-b91b-443a-9535-2008c9207ab1\") " pod="openstack-kuttl-tests/placement-689c6465d6-nsjrx" Dec 03 11:44:21 crc kubenswrapper[5052]: I1203 11:44:21.291204 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/0183ebb5-b91b-443a-9535-2008c9207ab1-logs\") pod \"placement-689c6465d6-nsjrx\" (UID: \"0183ebb5-b91b-443a-9535-2008c9207ab1\") " pod="openstack-kuttl-tests/placement-689c6465d6-nsjrx" Dec 03 11:44:21 crc kubenswrapper[5052]: I1203 11:44:21.291784 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/0183ebb5-b91b-443a-9535-2008c9207ab1-logs\") pod \"placement-689c6465d6-nsjrx\" (UID: \"0183ebb5-b91b-443a-9535-2008c9207ab1\") " pod="openstack-kuttl-tests/placement-689c6465d6-nsjrx" Dec 03 11:44:21 crc kubenswrapper[5052]: I1203 11:44:21.298287 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/0183ebb5-b91b-443a-9535-2008c9207ab1-internal-tls-certs\") pod \"placement-689c6465d6-nsjrx\" (UID: \"0183ebb5-b91b-443a-9535-2008c9207ab1\") " pod="openstack-kuttl-tests/placement-689c6465d6-nsjrx" Dec 03 11:44:21 crc kubenswrapper[5052]: I1203 11:44:21.298305 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/0183ebb5-b91b-443a-9535-2008c9207ab1-scripts\") pod \"placement-689c6465d6-nsjrx\" (UID: \"0183ebb5-b91b-443a-9535-2008c9207ab1\") " pod="openstack-kuttl-tests/placement-689c6465d6-nsjrx" Dec 03 11:44:21 crc kubenswrapper[5052]: I1203 11:44:21.300283 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0183ebb5-b91b-443a-9535-2008c9207ab1-combined-ca-bundle\") pod \"placement-689c6465d6-nsjrx\" (UID: \"0183ebb5-b91b-443a-9535-2008c9207ab1\") " pod="openstack-kuttl-tests/placement-689c6465d6-nsjrx" Dec 03 11:44:21 crc kubenswrapper[5052]: I1203 11:44:21.301083 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/0183ebb5-b91b-443a-9535-2008c9207ab1-config-data\") pod \"placement-689c6465d6-nsjrx\" (UID: \"0183ebb5-b91b-443a-9535-2008c9207ab1\") " pod="openstack-kuttl-tests/placement-689c6465d6-nsjrx" Dec 03 11:44:21 crc kubenswrapper[5052]: I1203 11:44:21.301996 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/0183ebb5-b91b-443a-9535-2008c9207ab1-public-tls-certs\") pod \"placement-689c6465d6-nsjrx\" (UID: \"0183ebb5-b91b-443a-9535-2008c9207ab1\") " pod="openstack-kuttl-tests/placement-689c6465d6-nsjrx" Dec 03 11:44:21 crc kubenswrapper[5052]: I1203 11:44:21.306860 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-keystone-listener-6794d844d6-f4tr7"] Dec 03 11:44:21 crc kubenswrapper[5052]: I1203 11:44:21.313960 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-xb5gb\" (UniqueName: \"kubernetes.io/projected/0183ebb5-b91b-443a-9535-2008c9207ab1-kube-api-access-xb5gb\") pod \"placement-689c6465d6-nsjrx\" (UID: \"0183ebb5-b91b-443a-9535-2008c9207ab1\") " pod="openstack-kuttl-tests/placement-689c6465d6-nsjrx" Dec 03 11:44:21 crc kubenswrapper[5052]: W1203 11:44:21.358953 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd8de865a_0f6c_413d_8a37_3ec4d496c4bb.slice/crio-3b79aa1079659ba91b2bfc402ce89d6f8dcb2f07c0b860dd97bccda698d668a8 WatchSource:0}: Error finding container 3b79aa1079659ba91b2bfc402ce89d6f8dcb2f07c0b860dd97bccda698d668a8: Status 404 returned error can't find the container with id 3b79aa1079659ba91b2bfc402ce89d6f8dcb2f07c0b860dd97bccda698d668a8 Dec 03 11:44:21 crc kubenswrapper[5052]: W1203 11:44:21.362624 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod23b2a11f_0129_474b_9f97_f9401abd3f6b.slice/crio-784f6e93c3ddc744790d11bc216e9351d404558dd5301abec92c2c027daedd51 WatchSource:0}: Error finding container 784f6e93c3ddc744790d11bc216e9351d404558dd5301abec92c2c027daedd51: Status 404 returned error can't find the container with id 784f6e93c3ddc744790d11bc216e9351d404558dd5301abec92c2c027daedd51 Dec 03 11:44:21 crc kubenswrapper[5052]: I1203 11:44:21.367121 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-689c6465d6-nsjrx" Dec 03 11:44:21 crc kubenswrapper[5052]: I1203 11:44:21.632039 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"d2650df9-f097-4bb1-84e1-72077a661399","Type":"ContainerStarted","Data":"0ceade6cc9a65e0e414c39b52e672b86796c8cf5e58f3884914e32309bc0607b"} Dec 03 11:44:21 crc kubenswrapper[5052]: I1203 11:44:21.634965 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-worker-79689f8957-qmkdm" event={"ID":"cadac9a6-1d82-403a-b7a0-692a424dd0e2","Type":"ContainerStarted","Data":"4d318f4a7ce116ebcf5a2ea62f382127bf1d126d796753d913aaafd4044c08be"} Dec 03 11:44:21 crc kubenswrapper[5052]: I1203 11:44:21.634993 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-worker-79689f8957-qmkdm" event={"ID":"cadac9a6-1d82-403a-b7a0-692a424dd0e2","Type":"ContainerStarted","Data":"de6975120318e384ab085701669f1c586c0650ffb743e7c4a4482d13b9ed77b0"} Dec 03 11:44:21 crc kubenswrapper[5052]: I1203 11:44:21.639276 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"fb3a323e-09aa-47ac-b593-e1c6e9b898e5","Type":"ContainerStarted","Data":"c19db15400d9a8203b4dde2972be1e62f22efa653277a18e83912df2627ac8d9"} Dec 03 11:44:21 crc kubenswrapper[5052]: I1203 11:44:21.639423 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="fb3a323e-09aa-47ac-b593-e1c6e9b898e5" containerName="ceilometer-central-agent" containerID="cri-o://7bff6b5804029ed658f2a1d5487e3bce63445ac4e6250251b735aee6dff232f5" gracePeriod=30 Dec 03 11:44:21 crc kubenswrapper[5052]: I1203 11:44:21.639697 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:44:21 crc kubenswrapper[5052]: I1203 11:44:21.639924 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="fb3a323e-09aa-47ac-b593-e1c6e9b898e5" containerName="proxy-httpd" containerID="cri-o://c19db15400d9a8203b4dde2972be1e62f22efa653277a18e83912df2627ac8d9" gracePeriod=30 Dec 03 11:44:21 crc kubenswrapper[5052]: I1203 11:44:21.639969 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="fb3a323e-09aa-47ac-b593-e1c6e9b898e5" containerName="sg-core" containerID="cri-o://1349f4ae2e79b5b6366e52f5c0162ec782e07b2a25ad7f647b70280b2223aa0f" gracePeriod=30 Dec 03 11:44:21 crc kubenswrapper[5052]: I1203 11:44:21.640001 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="fb3a323e-09aa-47ac-b593-e1c6e9b898e5" containerName="ceilometer-notification-agent" containerID="cri-o://9bf88bfd0d0e6f8a144f303520d8196838052ec8c253b8b04069315b384e953d" gracePeriod=30 Dec 03 11:44:21 crc kubenswrapper[5052]: I1203 11:44:21.644136 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-f76947bf6-27nv8" event={"ID":"23b2a11f-0129-474b-9f97-f9401abd3f6b","Type":"ContainerStarted","Data":"784f6e93c3ddc744790d11bc216e9351d404558dd5301abec92c2c027daedd51"} Dec 03 11:44:21 crc kubenswrapper[5052]: I1203 11:44:21.645626 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-keystone-listener-6794d844d6-f4tr7" event={"ID":"d8de865a-0f6c-413d-8a37-3ec4d496c4bb","Type":"ContainerStarted","Data":"3b79aa1079659ba91b2bfc402ce89d6f8dcb2f07c0b860dd97bccda698d668a8"} Dec 03 11:44:21 crc kubenswrapper[5052]: I1203 11:44:21.648351 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"1303e563-d0c3-4f9c-84b7-3e4fdeba3c6f","Type":"ContainerStarted","Data":"fc4933f5ef18703d3d02ad0d36b4c306bbc914872afd5d2a1944f06f2180aa37"} Dec 03 11:44:21 crc kubenswrapper[5052]: I1203 11:44:21.669745 5052 generic.go:334] "Generic (PLEG): container finished" podID="3d4a523e-547f-4d7a-920f-7a24ca45f183" containerID="2beb36e19d46bc6b746913b09b005f085e7e4ab7f58b32b97740b3059d80cf36" exitCode=0 Dec 03 11:44:21 crc kubenswrapper[5052]: I1203 11:44:21.669815 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-db-sync-qv4bk" event={"ID":"3d4a523e-547f-4d7a-920f-7a24ca45f183","Type":"ContainerDied","Data":"2beb36e19d46bc6b746913b09b005f085e7e4ab7f58b32b97740b3059d80cf36"} Dec 03 11:44:21 crc kubenswrapper[5052]: I1203 11:44:21.704441 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/ceilometer-0" podStartSLOduration=4.085882351 podStartE2EDuration="8.704421455s" podCreationTimestamp="2025-12-03 11:44:13 +0000 UTC" firstStartedPulling="2025-12-03 11:44:15.517330179 +0000 UTC m=+1446.105571953" lastFinishedPulling="2025-12-03 11:44:20.135869293 +0000 UTC m=+1450.724111057" observedRunningTime="2025-12-03 11:44:21.664740731 +0000 UTC m=+1452.252982505" watchObservedRunningTime="2025-12-03 11:44:21.704421455 +0000 UTC m=+1452.292663219" Dec 03 11:44:21 crc kubenswrapper[5052]: I1203 11:44:21.806018 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/placement-79d5b958cd-cq6hs"] Dec 03 11:44:22 crc kubenswrapper[5052]: I1203 11:44:22.059081 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/placement-689c6465d6-nsjrx"] Dec 03 11:44:22 crc kubenswrapper[5052]: I1203 11:44:22.679023 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-db-sync-5m4kd" Dec 03 11:44:22 crc kubenswrapper[5052]: I1203 11:44:22.681595 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-bootstrap-2hjbj" Dec 03 11:44:22 crc kubenswrapper[5052]: I1203 11:44:22.728006 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/b53d06f2-28c8-4c18-b9ce-0300f8b8ef06-config-data\") pod \"b53d06f2-28c8-4c18-b9ce-0300f8b8ef06\" (UID: \"b53d06f2-28c8-4c18-b9ce-0300f8b8ef06\") " Dec 03 11:44:22 crc kubenswrapper[5052]: I1203 11:44:22.728064 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/b53d06f2-28c8-4c18-b9ce-0300f8b8ef06-scripts\") pod \"b53d06f2-28c8-4c18-b9ce-0300f8b8ef06\" (UID: \"b53d06f2-28c8-4c18-b9ce-0300f8b8ef06\") " Dec 03 11:44:22 crc kubenswrapper[5052]: I1203 11:44:22.728102 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-glpqq\" (UniqueName: \"kubernetes.io/projected/b384ab60-c8a3-488d-8753-22058200e84f-kube-api-access-glpqq\") pod \"b384ab60-c8a3-488d-8753-22058200e84f\" (UID: \"b384ab60-c8a3-488d-8753-22058200e84f\") " Dec 03 11:44:22 crc kubenswrapper[5052]: I1203 11:44:22.728150 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pknlf\" (UniqueName: \"kubernetes.io/projected/b53d06f2-28c8-4c18-b9ce-0300f8b8ef06-kube-api-access-pknlf\") pod \"b53d06f2-28c8-4c18-b9ce-0300f8b8ef06\" (UID: \"b53d06f2-28c8-4c18-b9ce-0300f8b8ef06\") " Dec 03 11:44:22 crc kubenswrapper[5052]: I1203 11:44:22.728183 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b53d06f2-28c8-4c18-b9ce-0300f8b8ef06-combined-ca-bundle\") pod \"b53d06f2-28c8-4c18-b9ce-0300f8b8ef06\" (UID: \"b53d06f2-28c8-4c18-b9ce-0300f8b8ef06\") " Dec 03 11:44:22 crc kubenswrapper[5052]: I1203 11:44:22.728242 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/b53d06f2-28c8-4c18-b9ce-0300f8b8ef06-fernet-keys\") pod \"b53d06f2-28c8-4c18-b9ce-0300f8b8ef06\" (UID: \"b53d06f2-28c8-4c18-b9ce-0300f8b8ef06\") " Dec 03 11:44:22 crc kubenswrapper[5052]: I1203 11:44:22.728271 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/b384ab60-c8a3-488d-8753-22058200e84f-config\") pod \"b384ab60-c8a3-488d-8753-22058200e84f\" (UID: \"b384ab60-c8a3-488d-8753-22058200e84f\") " Dec 03 11:44:22 crc kubenswrapper[5052]: I1203 11:44:22.728309 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/b53d06f2-28c8-4c18-b9ce-0300f8b8ef06-credential-keys\") pod \"b53d06f2-28c8-4c18-b9ce-0300f8b8ef06\" (UID: \"b53d06f2-28c8-4c18-b9ce-0300f8b8ef06\") " Dec 03 11:44:22 crc kubenswrapper[5052]: I1203 11:44:22.728334 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b384ab60-c8a3-488d-8753-22058200e84f-combined-ca-bundle\") pod \"b384ab60-c8a3-488d-8753-22058200e84f\" (UID: \"b384ab60-c8a3-488d-8753-22058200e84f\") " Dec 03 11:44:22 crc kubenswrapper[5052]: I1203 11:44:22.736920 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b53d06f2-28c8-4c18-b9ce-0300f8b8ef06-scripts" (OuterVolumeSpecName: "scripts") pod "b53d06f2-28c8-4c18-b9ce-0300f8b8ef06" (UID: "b53d06f2-28c8-4c18-b9ce-0300f8b8ef06"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:44:22 crc kubenswrapper[5052]: I1203 11:44:22.737289 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b53d06f2-28c8-4c18-b9ce-0300f8b8ef06-kube-api-access-pknlf" (OuterVolumeSpecName: "kube-api-access-pknlf") pod "b53d06f2-28c8-4c18-b9ce-0300f8b8ef06" (UID: "b53d06f2-28c8-4c18-b9ce-0300f8b8ef06"). InnerVolumeSpecName "kube-api-access-pknlf". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:44:22 crc kubenswrapper[5052]: I1203 11:44:22.739316 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-f76947bf6-27nv8" event={"ID":"23b2a11f-0129-474b-9f97-f9401abd3f6b","Type":"ContainerStarted","Data":"562c233abb263737b9f5f1e4e7573903554a8770ec0a228f0ae6b1eaeed438cd"} Dec 03 11:44:22 crc kubenswrapper[5052]: I1203 11:44:22.743754 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b384ab60-c8a3-488d-8753-22058200e84f-kube-api-access-glpqq" (OuterVolumeSpecName: "kube-api-access-glpqq") pod "b384ab60-c8a3-488d-8753-22058200e84f" (UID: "b384ab60-c8a3-488d-8753-22058200e84f"). InnerVolumeSpecName "kube-api-access-glpqq". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:44:22 crc kubenswrapper[5052]: I1203 11:44:22.748346 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-bootstrap-2hjbj" Dec 03 11:44:22 crc kubenswrapper[5052]: I1203 11:44:22.748972 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-bootstrap-2hjbj" event={"ID":"b53d06f2-28c8-4c18-b9ce-0300f8b8ef06","Type":"ContainerDied","Data":"817629e22a8788100a0ceccb4a44aed9b4f417d1251d93488132a0e5b59075bc"} Dec 03 11:44:22 crc kubenswrapper[5052]: I1203 11:44:22.749026 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="817629e22a8788100a0ceccb4a44aed9b4f417d1251d93488132a0e5b59075bc" Dec 03 11:44:22 crc kubenswrapper[5052]: I1203 11:44:22.764599 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b53d06f2-28c8-4c18-b9ce-0300f8b8ef06-credential-keys" (OuterVolumeSpecName: "credential-keys") pod "b53d06f2-28c8-4c18-b9ce-0300f8b8ef06" (UID: "b53d06f2-28c8-4c18-b9ce-0300f8b8ef06"). InnerVolumeSpecName "credential-keys". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:44:22 crc kubenswrapper[5052]: I1203 11:44:22.765285 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b53d06f2-28c8-4c18-b9ce-0300f8b8ef06-fernet-keys" (OuterVolumeSpecName: "fernet-keys") pod "b53d06f2-28c8-4c18-b9ce-0300f8b8ef06" (UID: "b53d06f2-28c8-4c18-b9ce-0300f8b8ef06"). InnerVolumeSpecName "fernet-keys". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:44:22 crc kubenswrapper[5052]: I1203 11:44:22.802500 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"1303e563-d0c3-4f9c-84b7-3e4fdeba3c6f","Type":"ContainerStarted","Data":"877f33b3229c729de1c28b12bb78472817b121f3f37fd1b975fd4575ebc88b59"} Dec 03 11:44:22 crc kubenswrapper[5052]: I1203 11:44:22.811338 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-689c6465d6-nsjrx" event={"ID":"0183ebb5-b91b-443a-9535-2008c9207ab1","Type":"ContainerStarted","Data":"aa1335a1c15d6d9a335f1c0f519b93e0eb71b01df0b155d55fdd9d086a2a1a1a"} Dec 03 11:44:22 crc kubenswrapper[5052]: I1203 11:44:22.833250 5052 reconciler_common.go:293] "Volume detached for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/b53d06f2-28c8-4c18-b9ce-0300f8b8ef06-credential-keys\") on node \"crc\" DevicePath \"\"" Dec 03 11:44:22 crc kubenswrapper[5052]: I1203 11:44:22.833299 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/b53d06f2-28c8-4c18-b9ce-0300f8b8ef06-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:44:22 crc kubenswrapper[5052]: I1203 11:44:22.833311 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-glpqq\" (UniqueName: \"kubernetes.io/projected/b384ab60-c8a3-488d-8753-22058200e84f-kube-api-access-glpqq\") on node \"crc\" DevicePath \"\"" Dec 03 11:44:22 crc kubenswrapper[5052]: I1203 11:44:22.833321 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-pknlf\" (UniqueName: \"kubernetes.io/projected/b53d06f2-28c8-4c18-b9ce-0300f8b8ef06-kube-api-access-pknlf\") on node \"crc\" DevicePath \"\"" Dec 03 11:44:22 crc kubenswrapper[5052]: I1203 11:44:22.833330 5052 reconciler_common.go:293] "Volume detached for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/b53d06f2-28c8-4c18-b9ce-0300f8b8ef06-fernet-keys\") on node \"crc\" DevicePath \"\"" Dec 03 11:44:22 crc kubenswrapper[5052]: I1203 11:44:22.836219 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b384ab60-c8a3-488d-8753-22058200e84f-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "b384ab60-c8a3-488d-8753-22058200e84f" (UID: "b384ab60-c8a3-488d-8753-22058200e84f"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:44:22 crc kubenswrapper[5052]: I1203 11:44:22.842715 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-db-sync-5m4kd" Dec 03 11:44:22 crc kubenswrapper[5052]: I1203 11:44:22.842866 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-db-sync-5m4kd" event={"ID":"b384ab60-c8a3-488d-8753-22058200e84f","Type":"ContainerDied","Data":"cec0285ed8174eb9623ed9d54fc95d1c8dd5f4912ff13b4cd2500eef0911c323"} Dec 03 11:44:22 crc kubenswrapper[5052]: I1203 11:44:22.842912 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cec0285ed8174eb9623ed9d54fc95d1c8dd5f4912ff13b4cd2500eef0911c323" Dec 03 11:44:22 crc kubenswrapper[5052]: I1203 11:44:22.862074 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-79d5b958cd-cq6hs" event={"ID":"f8e49a02-823b-4851-9ecf-f2d1696fa201","Type":"ContainerStarted","Data":"d6396b2c9995d5e984317dd8c2f933f027bba5fedf7a60a68099ea46c8bf40ac"} Dec 03 11:44:22 crc kubenswrapper[5052]: I1203 11:44:22.866273 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b53d06f2-28c8-4c18-b9ce-0300f8b8ef06-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "b53d06f2-28c8-4c18-b9ce-0300f8b8ef06" (UID: "b53d06f2-28c8-4c18-b9ce-0300f8b8ef06"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:44:22 crc kubenswrapper[5052]: I1203 11:44:22.878482 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b384ab60-c8a3-488d-8753-22058200e84f-config" (OuterVolumeSpecName: "config") pod "b384ab60-c8a3-488d-8753-22058200e84f" (UID: "b384ab60-c8a3-488d-8753-22058200e84f"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:44:22 crc kubenswrapper[5052]: I1203 11:44:22.881228 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b53d06f2-28c8-4c18-b9ce-0300f8b8ef06-config-data" (OuterVolumeSpecName: "config-data") pod "b53d06f2-28c8-4c18-b9ce-0300f8b8ef06" (UID: "b53d06f2-28c8-4c18-b9ce-0300f8b8ef06"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:44:22 crc kubenswrapper[5052]: I1203 11:44:22.881250 5052 generic.go:334] "Generic (PLEG): container finished" podID="fb3a323e-09aa-47ac-b593-e1c6e9b898e5" containerID="c19db15400d9a8203b4dde2972be1e62f22efa653277a18e83912df2627ac8d9" exitCode=0 Dec 03 11:44:22 crc kubenswrapper[5052]: I1203 11:44:22.881275 5052 generic.go:334] "Generic (PLEG): container finished" podID="fb3a323e-09aa-47ac-b593-e1c6e9b898e5" containerID="1349f4ae2e79b5b6366e52f5c0162ec782e07b2a25ad7f647b70280b2223aa0f" exitCode=2 Dec 03 11:44:22 crc kubenswrapper[5052]: I1203 11:44:22.881319 5052 generic.go:334] "Generic (PLEG): container finished" podID="fb3a323e-09aa-47ac-b593-e1c6e9b898e5" containerID="9bf88bfd0d0e6f8a144f303520d8196838052ec8c253b8b04069315b384e953d" exitCode=0 Dec 03 11:44:22 crc kubenswrapper[5052]: I1203 11:44:22.881329 5052 generic.go:334] "Generic (PLEG): container finished" podID="fb3a323e-09aa-47ac-b593-e1c6e9b898e5" containerID="7bff6b5804029ed658f2a1d5487e3bce63445ac4e6250251b735aee6dff232f5" exitCode=0 Dec 03 11:44:22 crc kubenswrapper[5052]: I1203 11:44:22.881398 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"fb3a323e-09aa-47ac-b593-e1c6e9b898e5","Type":"ContainerDied","Data":"c19db15400d9a8203b4dde2972be1e62f22efa653277a18e83912df2627ac8d9"} Dec 03 11:44:22 crc kubenswrapper[5052]: I1203 11:44:22.881437 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"fb3a323e-09aa-47ac-b593-e1c6e9b898e5","Type":"ContainerDied","Data":"1349f4ae2e79b5b6366e52f5c0162ec782e07b2a25ad7f647b70280b2223aa0f"} Dec 03 11:44:22 crc kubenswrapper[5052]: I1203 11:44:22.881448 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"fb3a323e-09aa-47ac-b593-e1c6e9b898e5","Type":"ContainerDied","Data":"9bf88bfd0d0e6f8a144f303520d8196838052ec8c253b8b04069315b384e953d"} Dec 03 11:44:22 crc kubenswrapper[5052]: I1203 11:44:22.881459 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"fb3a323e-09aa-47ac-b593-e1c6e9b898e5","Type":"ContainerDied","Data":"7bff6b5804029ed658f2a1d5487e3bce63445ac4e6250251b735aee6dff232f5"} Dec 03 11:44:22 crc kubenswrapper[5052]: I1203 11:44:22.935122 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b53d06f2-28c8-4c18-b9ce-0300f8b8ef06-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:44:22 crc kubenswrapper[5052]: I1203 11:44:22.935144 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/b384ab60-c8a3-488d-8753-22058200e84f-config\") on node \"crc\" DevicePath \"\"" Dec 03 11:44:22 crc kubenswrapper[5052]: I1203 11:44:22.935156 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b384ab60-c8a3-488d-8753-22058200e84f-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:44:22 crc kubenswrapper[5052]: I1203 11:44:22.935164 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/b53d06f2-28c8-4c18-b9ce-0300f8b8ef06-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:44:23 crc kubenswrapper[5052]: I1203 11:44:23.454318 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:44:23 crc kubenswrapper[5052]: I1203 11:44:23.458016 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-db-sync-qv4bk" Dec 03 11:44:23 crc kubenswrapper[5052]: I1203 11:44:23.651731 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/fb3a323e-09aa-47ac-b593-e1c6e9b898e5-combined-ca-bundle\") pod \"fb3a323e-09aa-47ac-b593-e1c6e9b898e5\" (UID: \"fb3a323e-09aa-47ac-b593-e1c6e9b898e5\") " Dec 03 11:44:23 crc kubenswrapper[5052]: I1203 11:44:23.652502 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/3d4a523e-547f-4d7a-920f-7a24ca45f183-scripts\") pod \"3d4a523e-547f-4d7a-920f-7a24ca45f183\" (UID: \"3d4a523e-547f-4d7a-920f-7a24ca45f183\") " Dec 03 11:44:23 crc kubenswrapper[5052]: I1203 11:44:23.653289 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tp8rl\" (UniqueName: \"kubernetes.io/projected/3d4a523e-547f-4d7a-920f-7a24ca45f183-kube-api-access-tp8rl\") pod \"3d4a523e-547f-4d7a-920f-7a24ca45f183\" (UID: \"3d4a523e-547f-4d7a-920f-7a24ca45f183\") " Dec 03 11:44:23 crc kubenswrapper[5052]: I1203 11:44:23.653421 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/fb3a323e-09aa-47ac-b593-e1c6e9b898e5-run-httpd\") pod \"fb3a323e-09aa-47ac-b593-e1c6e9b898e5\" (UID: \"fb3a323e-09aa-47ac-b593-e1c6e9b898e5\") " Dec 03 11:44:23 crc kubenswrapper[5052]: I1203 11:44:23.653552 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/fb3a323e-09aa-47ac-b593-e1c6e9b898e5-sg-core-conf-yaml\") pod \"fb3a323e-09aa-47ac-b593-e1c6e9b898e5\" (UID: \"fb3a323e-09aa-47ac-b593-e1c6e9b898e5\") " Dec 03 11:44:23 crc kubenswrapper[5052]: I1203 11:44:23.653659 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/3d4a523e-547f-4d7a-920f-7a24ca45f183-etc-machine-id\") pod \"3d4a523e-547f-4d7a-920f-7a24ca45f183\" (UID: \"3d4a523e-547f-4d7a-920f-7a24ca45f183\") " Dec 03 11:44:23 crc kubenswrapper[5052]: I1203 11:44:23.653795 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/fb3a323e-09aa-47ac-b593-e1c6e9b898e5-log-httpd\") pod \"fb3a323e-09aa-47ac-b593-e1c6e9b898e5\" (UID: \"fb3a323e-09aa-47ac-b593-e1c6e9b898e5\") " Dec 03 11:44:23 crc kubenswrapper[5052]: I1203 11:44:23.653927 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/3d4a523e-547f-4d7a-920f-7a24ca45f183-config-data\") pod \"3d4a523e-547f-4d7a-920f-7a24ca45f183\" (UID: \"3d4a523e-547f-4d7a-920f-7a24ca45f183\") " Dec 03 11:44:23 crc kubenswrapper[5052]: I1203 11:44:23.654033 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/fb3a323e-09aa-47ac-b593-e1c6e9b898e5-scripts\") pod \"fb3a323e-09aa-47ac-b593-e1c6e9b898e5\" (UID: \"fb3a323e-09aa-47ac-b593-e1c6e9b898e5\") " Dec 03 11:44:23 crc kubenswrapper[5052]: I1203 11:44:23.654140 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/3d4a523e-547f-4d7a-920f-7a24ca45f183-db-sync-config-data\") pod \"3d4a523e-547f-4d7a-920f-7a24ca45f183\" (UID: \"3d4a523e-547f-4d7a-920f-7a24ca45f183\") " Dec 03 11:44:23 crc kubenswrapper[5052]: I1203 11:44:23.654251 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j2cp9\" (UniqueName: \"kubernetes.io/projected/fb3a323e-09aa-47ac-b593-e1c6e9b898e5-kube-api-access-j2cp9\") pod \"fb3a323e-09aa-47ac-b593-e1c6e9b898e5\" (UID: \"fb3a323e-09aa-47ac-b593-e1c6e9b898e5\") " Dec 03 11:44:23 crc kubenswrapper[5052]: I1203 11:44:23.654366 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3d4a523e-547f-4d7a-920f-7a24ca45f183-combined-ca-bundle\") pod \"3d4a523e-547f-4d7a-920f-7a24ca45f183\" (UID: \"3d4a523e-547f-4d7a-920f-7a24ca45f183\") " Dec 03 11:44:23 crc kubenswrapper[5052]: I1203 11:44:23.654484 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/fb3a323e-09aa-47ac-b593-e1c6e9b898e5-config-data\") pod \"fb3a323e-09aa-47ac-b593-e1c6e9b898e5\" (UID: \"fb3a323e-09aa-47ac-b593-e1c6e9b898e5\") " Dec 03 11:44:23 crc kubenswrapper[5052]: I1203 11:44:23.654905 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb3a323e-09aa-47ac-b593-e1c6e9b898e5-run-httpd" (OuterVolumeSpecName: "run-httpd") pod "fb3a323e-09aa-47ac-b593-e1c6e9b898e5" (UID: "fb3a323e-09aa-47ac-b593-e1c6e9b898e5"). InnerVolumeSpecName "run-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:44:23 crc kubenswrapper[5052]: I1203 11:44:23.655285 5052 reconciler_common.go:293] "Volume detached for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/fb3a323e-09aa-47ac-b593-e1c6e9b898e5-run-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 11:44:23 crc kubenswrapper[5052]: I1203 11:44:23.655821 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/3d4a523e-547f-4d7a-920f-7a24ca45f183-etc-machine-id" (OuterVolumeSpecName: "etc-machine-id") pod "3d4a523e-547f-4d7a-920f-7a24ca45f183" (UID: "3d4a523e-547f-4d7a-920f-7a24ca45f183"). InnerVolumeSpecName "etc-machine-id". PluginName "kubernetes.io/host-path", VolumeGidValue "" Dec 03 11:44:23 crc kubenswrapper[5052]: I1203 11:44:23.656327 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb3a323e-09aa-47ac-b593-e1c6e9b898e5-log-httpd" (OuterVolumeSpecName: "log-httpd") pod "fb3a323e-09aa-47ac-b593-e1c6e9b898e5" (UID: "fb3a323e-09aa-47ac-b593-e1c6e9b898e5"). InnerVolumeSpecName "log-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:44:23 crc kubenswrapper[5052]: I1203 11:44:23.719379 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3d4a523e-547f-4d7a-920f-7a24ca45f183-scripts" (OuterVolumeSpecName: "scripts") pod "3d4a523e-547f-4d7a-920f-7a24ca45f183" (UID: "3d4a523e-547f-4d7a-920f-7a24ca45f183"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:44:23 crc kubenswrapper[5052]: I1203 11:44:23.722920 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3d4a523e-547f-4d7a-920f-7a24ca45f183-kube-api-access-tp8rl" (OuterVolumeSpecName: "kube-api-access-tp8rl") pod "3d4a523e-547f-4d7a-920f-7a24ca45f183" (UID: "3d4a523e-547f-4d7a-920f-7a24ca45f183"). InnerVolumeSpecName "kube-api-access-tp8rl". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:44:23 crc kubenswrapper[5052]: I1203 11:44:23.723352 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fb3a323e-09aa-47ac-b593-e1c6e9b898e5-scripts" (OuterVolumeSpecName: "scripts") pod "fb3a323e-09aa-47ac-b593-e1c6e9b898e5" (UID: "fb3a323e-09aa-47ac-b593-e1c6e9b898e5"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:44:23 crc kubenswrapper[5052]: I1203 11:44:23.723693 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fb3a323e-09aa-47ac-b593-e1c6e9b898e5-kube-api-access-j2cp9" (OuterVolumeSpecName: "kube-api-access-j2cp9") pod "fb3a323e-09aa-47ac-b593-e1c6e9b898e5" (UID: "fb3a323e-09aa-47ac-b593-e1c6e9b898e5"). InnerVolumeSpecName "kube-api-access-j2cp9". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:44:23 crc kubenswrapper[5052]: I1203 11:44:23.745312 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3d4a523e-547f-4d7a-920f-7a24ca45f183-db-sync-config-data" (OuterVolumeSpecName: "db-sync-config-data") pod "3d4a523e-547f-4d7a-920f-7a24ca45f183" (UID: "3d4a523e-547f-4d7a-920f-7a24ca45f183"). InnerVolumeSpecName "db-sync-config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:44:23 crc kubenswrapper[5052]: I1203 11:44:23.765057 5052 reconciler_common.go:293] "Volume detached for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/3d4a523e-547f-4d7a-920f-7a24ca45f183-etc-machine-id\") on node \"crc\" DevicePath \"\"" Dec 03 11:44:23 crc kubenswrapper[5052]: I1203 11:44:23.765086 5052 reconciler_common.go:293] "Volume detached for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/fb3a323e-09aa-47ac-b593-e1c6e9b898e5-log-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 11:44:23 crc kubenswrapper[5052]: I1203 11:44:23.765094 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/fb3a323e-09aa-47ac-b593-e1c6e9b898e5-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:44:23 crc kubenswrapper[5052]: I1203 11:44:23.765102 5052 reconciler_common.go:293] "Volume detached for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/3d4a523e-547f-4d7a-920f-7a24ca45f183-db-sync-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:44:23 crc kubenswrapper[5052]: I1203 11:44:23.765110 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-j2cp9\" (UniqueName: \"kubernetes.io/projected/fb3a323e-09aa-47ac-b593-e1c6e9b898e5-kube-api-access-j2cp9\") on node \"crc\" DevicePath \"\"" Dec 03 11:44:23 crc kubenswrapper[5052]: I1203 11:44:23.765120 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/3d4a523e-547f-4d7a-920f-7a24ca45f183-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:44:23 crc kubenswrapper[5052]: I1203 11:44:23.765129 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-tp8rl\" (UniqueName: \"kubernetes.io/projected/3d4a523e-547f-4d7a-920f-7a24ca45f183-kube-api-access-tp8rl\") on node \"crc\" DevicePath \"\"" Dec 03 11:44:23 crc kubenswrapper[5052]: I1203 11:44:23.781690 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fb3a323e-09aa-47ac-b593-e1c6e9b898e5-sg-core-conf-yaml" (OuterVolumeSpecName: "sg-core-conf-yaml") pod "fb3a323e-09aa-47ac-b593-e1c6e9b898e5" (UID: "fb3a323e-09aa-47ac-b593-e1c6e9b898e5"). InnerVolumeSpecName "sg-core-conf-yaml". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:44:23 crc kubenswrapper[5052]: I1203 11:44:23.872221 5052 reconciler_common.go:293] "Volume detached for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/fb3a323e-09aa-47ac-b593-e1c6e9b898e5-sg-core-conf-yaml\") on node \"crc\" DevicePath \"\"" Dec 03 11:44:23 crc kubenswrapper[5052]: I1203 11:44:23.916716 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/keystone-bootstrap-2hjbj"] Dec 03 11:44:23 crc kubenswrapper[5052]: I1203 11:44:23.918771 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-f76947bf6-27nv8" event={"ID":"23b2a11f-0129-474b-9f97-f9401abd3f6b","Type":"ContainerStarted","Data":"b0260015d44c0e665a4e129dce5d0997d993c150242031889ee970b1dbfc89df"} Dec 03 11:44:23 crc kubenswrapper[5052]: I1203 11:44:23.919947 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/barbican-api-f76947bf6-27nv8" Dec 03 11:44:23 crc kubenswrapper[5052]: I1203 11:44:23.919977 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/barbican-api-f76947bf6-27nv8" Dec 03 11:44:23 crc kubenswrapper[5052]: I1203 11:44:23.939331 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/keystone-bootstrap-2hjbj"] Dec 03 11:44:23 crc kubenswrapper[5052]: I1203 11:44:23.940255 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-worker-79689f8957-qmkdm" event={"ID":"cadac9a6-1d82-403a-b7a0-692a424dd0e2","Type":"ContainerStarted","Data":"38aac1236f884983fd56130cde1141029030d5995e585474bc43e38342fcd566"} Dec 03 11:44:23 crc kubenswrapper[5052]: I1203 11:44:23.950998 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"d2650df9-f097-4bb1-84e1-72077a661399","Type":"ContainerStarted","Data":"725bc621142a24553c3e356f496feb688f1c45840f857a725ee2ca892589de8b"} Dec 03 11:44:23 crc kubenswrapper[5052]: I1203 11:44:23.954654 5052 generic.go:334] "Generic (PLEG): container finished" podID="f8e49a02-823b-4851-9ecf-f2d1696fa201" containerID="2da14680cf8eb402a210aa0b17d027537e969fa081339d2233cfa2f5c81dd7fd" exitCode=1 Dec 03 11:44:23 crc kubenswrapper[5052]: I1203 11:44:23.954725 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-79d5b958cd-cq6hs" event={"ID":"f8e49a02-823b-4851-9ecf-f2d1696fa201","Type":"ContainerDied","Data":"2da14680cf8eb402a210aa0b17d027537e969fa081339d2233cfa2f5c81dd7fd"} Dec 03 11:44:23 crc kubenswrapper[5052]: I1203 11:44:23.954759 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-79d5b958cd-cq6hs" event={"ID":"f8e49a02-823b-4851-9ecf-f2d1696fa201","Type":"ContainerStarted","Data":"4565a0244d0b774732f948581904402406dc7ecdcdc28c9c004dd467cedd108d"} Dec 03 11:44:23 crc kubenswrapper[5052]: I1203 11:44:23.955361 5052 scope.go:117] "RemoveContainer" containerID="2da14680cf8eb402a210aa0b17d027537e969fa081339d2233cfa2f5c81dd7fd" Dec 03 11:44:23 crc kubenswrapper[5052]: I1203 11:44:23.956034 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/placement-79d5b958cd-cq6hs" Dec 03 11:44:23 crc kubenswrapper[5052]: I1203 11:44:23.968526 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/neutron-7cc99c958f-bjn2l"] Dec 03 11:44:23 crc kubenswrapper[5052]: E1203 11:44:23.969758 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="b53d06f2-28c8-4c18-b9ce-0300f8b8ef06" containerName="keystone-bootstrap" Dec 03 11:44:23 crc kubenswrapper[5052]: I1203 11:44:23.969785 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="b53d06f2-28c8-4c18-b9ce-0300f8b8ef06" containerName="keystone-bootstrap" Dec 03 11:44:23 crc kubenswrapper[5052]: E1203 11:44:23.969801 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="fb3a323e-09aa-47ac-b593-e1c6e9b898e5" containerName="ceilometer-notification-agent" Dec 03 11:44:23 crc kubenswrapper[5052]: I1203 11:44:23.969808 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb3a323e-09aa-47ac-b593-e1c6e9b898e5" containerName="ceilometer-notification-agent" Dec 03 11:44:23 crc kubenswrapper[5052]: E1203 11:44:23.969829 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="fb3a323e-09aa-47ac-b593-e1c6e9b898e5" containerName="proxy-httpd" Dec 03 11:44:23 crc kubenswrapper[5052]: I1203 11:44:23.969836 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb3a323e-09aa-47ac-b593-e1c6e9b898e5" containerName="proxy-httpd" Dec 03 11:44:23 crc kubenswrapper[5052]: E1203 11:44:23.969852 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="3d4a523e-547f-4d7a-920f-7a24ca45f183" containerName="cinder-db-sync" Dec 03 11:44:23 crc kubenswrapper[5052]: I1203 11:44:23.969859 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="3d4a523e-547f-4d7a-920f-7a24ca45f183" containerName="cinder-db-sync" Dec 03 11:44:23 crc kubenswrapper[5052]: E1203 11:44:23.969898 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="fb3a323e-09aa-47ac-b593-e1c6e9b898e5" containerName="ceilometer-central-agent" Dec 03 11:44:23 crc kubenswrapper[5052]: I1203 11:44:23.969905 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb3a323e-09aa-47ac-b593-e1c6e9b898e5" containerName="ceilometer-central-agent" Dec 03 11:44:23 crc kubenswrapper[5052]: E1203 11:44:23.969931 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="fb3a323e-09aa-47ac-b593-e1c6e9b898e5" containerName="sg-core" Dec 03 11:44:23 crc kubenswrapper[5052]: I1203 11:44:23.969939 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb3a323e-09aa-47ac-b593-e1c6e9b898e5" containerName="sg-core" Dec 03 11:44:23 crc kubenswrapper[5052]: E1203 11:44:23.969951 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="b384ab60-c8a3-488d-8753-22058200e84f" containerName="neutron-db-sync" Dec 03 11:44:23 crc kubenswrapper[5052]: I1203 11:44:23.969958 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="b384ab60-c8a3-488d-8753-22058200e84f" containerName="neutron-db-sync" Dec 03 11:44:23 crc kubenswrapper[5052]: I1203 11:44:23.970188 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="fb3a323e-09aa-47ac-b593-e1c6e9b898e5" containerName="proxy-httpd" Dec 03 11:44:23 crc kubenswrapper[5052]: I1203 11:44:23.970204 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="fb3a323e-09aa-47ac-b593-e1c6e9b898e5" containerName="ceilometer-central-agent" Dec 03 11:44:23 crc kubenswrapper[5052]: I1203 11:44:23.970228 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="b53d06f2-28c8-4c18-b9ce-0300f8b8ef06" containerName="keystone-bootstrap" Dec 03 11:44:23 crc kubenswrapper[5052]: I1203 11:44:23.970242 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="fb3a323e-09aa-47ac-b593-e1c6e9b898e5" containerName="sg-core" Dec 03 11:44:23 crc kubenswrapper[5052]: I1203 11:44:23.970256 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="fb3a323e-09aa-47ac-b593-e1c6e9b898e5" containerName="ceilometer-notification-agent" Dec 03 11:44:23 crc kubenswrapper[5052]: I1203 11:44:23.970266 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="b384ab60-c8a3-488d-8753-22058200e84f" containerName="neutron-db-sync" Dec 03 11:44:23 crc kubenswrapper[5052]: I1203 11:44:23.970280 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="3d4a523e-547f-4d7a-920f-7a24ca45f183" containerName="cinder-db-sync" Dec 03 11:44:23 crc kubenswrapper[5052]: I1203 11:44:23.971426 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-7cc99c958f-bjn2l" Dec 03 11:44:23 crc kubenswrapper[5052]: I1203 11:44:23.974801 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"fb3a323e-09aa-47ac-b593-e1c6e9b898e5","Type":"ContainerDied","Data":"3c0b98c68b59cee11baed24f6b4ecfa16e55232bf35a0455f88eaa58c922a93d"} Dec 03 11:44:23 crc kubenswrapper[5052]: I1203 11:44:23.974851 5052 scope.go:117] "RemoveContainer" containerID="c19db15400d9a8203b4dde2972be1e62f22efa653277a18e83912df2627ac8d9" Dec 03 11:44:23 crc kubenswrapper[5052]: I1203 11:44:23.974985 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:44:23 crc kubenswrapper[5052]: I1203 11:44:23.976551 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-neutron-ovndbs" Dec 03 11:44:23 crc kubenswrapper[5052]: I1203 11:44:23.976655 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"neutron-neutron-dockercfg-kfdxn" Dec 03 11:44:23 crc kubenswrapper[5052]: I1203 11:44:23.987440 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3d4a523e-547f-4d7a-920f-7a24ca45f183-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "3d4a523e-547f-4d7a-920f-7a24ca45f183" (UID: "3d4a523e-547f-4d7a-920f-7a24ca45f183"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:44:23 crc kubenswrapper[5052]: I1203 11:44:23.989561 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"neutron-config" Dec 03 11:44:23 crc kubenswrapper[5052]: I1203 11:44:23.993863 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3d4a523e-547f-4d7a-920f-7a24ca45f183-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:44:23 crc kubenswrapper[5052]: I1203 11:44:23.998179 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"neutron-httpd-config" Dec 03 11:44:24 crc kubenswrapper[5052]: I1203 11:44:24.006785 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-7cc99c958f-bjn2l"] Dec 03 11:44:24 crc kubenswrapper[5052]: I1203 11:44:24.015565 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/barbican-api-f76947bf6-27nv8" podStartSLOduration=5.01554151 podStartE2EDuration="5.01554151s" podCreationTimestamp="2025-12-03 11:44:19 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:44:23.950151945 +0000 UTC m=+1454.538393709" watchObservedRunningTime="2025-12-03 11:44:24.01554151 +0000 UTC m=+1454.603783274" Dec 03 11:44:24 crc kubenswrapper[5052]: I1203 11:44:24.016707 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-keystone-listener-6794d844d6-f4tr7" event={"ID":"d8de865a-0f6c-413d-8a37-3ec4d496c4bb","Type":"ContainerStarted","Data":"775a6cb46d400df5368627d0a86f59d32f62f17aac77b3a75fe7d8c2928b8362"} Dec 03 11:44:24 crc kubenswrapper[5052]: I1203 11:44:24.016846 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-keystone-listener-6794d844d6-f4tr7" event={"ID":"d8de865a-0f6c-413d-8a37-3ec4d496c4bb","Type":"ContainerStarted","Data":"def922586d645f9987c75d4862d880ed855e05523f69e809e4d0637bfe42a9d8"} Dec 03 11:44:24 crc kubenswrapper[5052]: I1203 11:44:24.030548 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/barbican-worker-79689f8957-qmkdm" podStartSLOduration=5.030528043 podStartE2EDuration="5.030528043s" podCreationTimestamp="2025-12-03 11:44:19 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:44:23.985192105 +0000 UTC m=+1454.573433869" watchObservedRunningTime="2025-12-03 11:44:24.030528043 +0000 UTC m=+1454.618769807" Dec 03 11:44:24 crc kubenswrapper[5052]: I1203 11:44:24.047730 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"1303e563-d0c3-4f9c-84b7-3e4fdeba3c6f","Type":"ContainerStarted","Data":"8eb9f63d2e238cddda381da8dd95d8d4b6eb3913bb5cdce6127df175953100c7"} Dec 03 11:44:24 crc kubenswrapper[5052]: I1203 11:44:24.053730 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 11:44:24 crc kubenswrapper[5052]: I1203 11:44:24.055631 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:44:24 crc kubenswrapper[5052]: I1203 11:44:24.059153 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cinder-scheduler-config-data" Dec 03 11:44:24 crc kubenswrapper[5052]: I1203 11:44:24.063128 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-689c6465d6-nsjrx" event={"ID":"0183ebb5-b91b-443a-9535-2008c9207ab1","Type":"ContainerStarted","Data":"32cafa0afafbb84110820599defab0c52864ba6ace469cdde0c2e43031966639"} Dec 03 11:44:24 crc kubenswrapper[5052]: I1203 11:44:24.065952 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-db-sync-qv4bk" event={"ID":"3d4a523e-547f-4d7a-920f-7a24ca45f183","Type":"ContainerDied","Data":"8ca7a090a69a8425edf74a6ee41359493434fd61eed1fac3855a560aac2c1840"} Dec 03 11:44:24 crc kubenswrapper[5052]: I1203 11:44:24.065989 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8ca7a090a69a8425edf74a6ee41359493434fd61eed1fac3855a560aac2c1840" Dec 03 11:44:24 crc kubenswrapper[5052]: I1203 11:44:24.066047 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-db-sync-qv4bk" Dec 03 11:44:24 crc kubenswrapper[5052]: I1203 11:44:24.069297 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 11:44:24 crc kubenswrapper[5052]: I1203 11:44:24.082738 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/keystone-bootstrap-fmkmc"] Dec 03 11:44:24 crc kubenswrapper[5052]: I1203 11:44:24.083894 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-bootstrap-fmkmc" Dec 03 11:44:24 crc kubenswrapper[5052]: I1203 11:44:24.087709 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone-scripts" Dec 03 11:44:24 crc kubenswrapper[5052]: I1203 11:44:24.088095 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone" Dec 03 11:44:24 crc kubenswrapper[5052]: I1203 11:44:24.088333 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone-keystone-dockercfg-4hzgj" Dec 03 11:44:24 crc kubenswrapper[5052]: I1203 11:44:24.088495 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"osp-secret" Dec 03 11:44:24 crc kubenswrapper[5052]: I1203 11:44:24.088605 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone-config-data" Dec 03 11:44:24 crc kubenswrapper[5052]: I1203 11:44:24.143697 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/glance-default-internal-api-0" podStartSLOduration=6.143675875 podStartE2EDuration="6.143675875s" podCreationTimestamp="2025-12-03 11:44:18 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:44:24.088760272 +0000 UTC m=+1454.677002036" watchObservedRunningTime="2025-12-03 11:44:24.143675875 +0000 UTC m=+1454.731917639" Dec 03 11:44:24 crc kubenswrapper[5052]: I1203 11:44:24.144603 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-bootstrap-fmkmc"] Dec 03 11:44:24 crc kubenswrapper[5052]: I1203 11:44:24.182584 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 11:44:24 crc kubenswrapper[5052]: I1203 11:44:24.184185 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:44:24 crc kubenswrapper[5052]: I1203 11:44:24.203192 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cinder-api-config-data" Dec 03 11:44:24 crc kubenswrapper[5052]: I1203 11:44:24.206162 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ee9c2229-695a-4930-89b0-39f140a3f328-combined-ca-bundle\") pod \"keystone-bootstrap-fmkmc\" (UID: \"ee9c2229-695a-4930-89b0-39f140a3f328\") " pod="openstack-kuttl-tests/keystone-bootstrap-fmkmc" Dec 03 11:44:24 crc kubenswrapper[5052]: I1203 11:44:24.206244 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/2dbe7002-eb09-4a0f-b65b-fc0294c13f4f-ovndb-tls-certs\") pod \"neutron-7cc99c958f-bjn2l\" (UID: \"2dbe7002-eb09-4a0f-b65b-fc0294c13f4f\") " pod="openstack-kuttl-tests/neutron-7cc99c958f-bjn2l" Dec 03 11:44:24 crc kubenswrapper[5052]: I1203 11:44:24.206326 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/bf425311-41fb-48cf-b34a-4cc4853c68b3-combined-ca-bundle\") pod \"cinder-scheduler-0\" (UID: \"bf425311-41fb-48cf-b34a-4cc4853c68b3\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:44:24 crc kubenswrapper[5052]: I1203 11:44:24.206358 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/bf425311-41fb-48cf-b34a-4cc4853c68b3-scripts\") pod \"cinder-scheduler-0\" (UID: \"bf425311-41fb-48cf-b34a-4cc4853c68b3\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:44:24 crc kubenswrapper[5052]: I1203 11:44:24.206382 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/2dbe7002-eb09-4a0f-b65b-fc0294c13f4f-httpd-config\") pod \"neutron-7cc99c958f-bjn2l\" (UID: \"2dbe7002-eb09-4a0f-b65b-fc0294c13f4f\") " pod="openstack-kuttl-tests/neutron-7cc99c958f-bjn2l" Dec 03 11:44:24 crc kubenswrapper[5052]: I1203 11:44:24.206760 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-22j7r\" (UniqueName: \"kubernetes.io/projected/2dbe7002-eb09-4a0f-b65b-fc0294c13f4f-kube-api-access-22j7r\") pod \"neutron-7cc99c958f-bjn2l\" (UID: \"2dbe7002-eb09-4a0f-b65b-fc0294c13f4f\") " pod="openstack-kuttl-tests/neutron-7cc99c958f-bjn2l" Dec 03 11:44:24 crc kubenswrapper[5052]: I1203 11:44:24.206830 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/ee9c2229-695a-4930-89b0-39f140a3f328-scripts\") pod \"keystone-bootstrap-fmkmc\" (UID: \"ee9c2229-695a-4930-89b0-39f140a3f328\") " pod="openstack-kuttl-tests/keystone-bootstrap-fmkmc" Dec 03 11:44:24 crc kubenswrapper[5052]: I1203 11:44:24.206875 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/bf425311-41fb-48cf-b34a-4cc4853c68b3-etc-machine-id\") pod \"cinder-scheduler-0\" (UID: \"bf425311-41fb-48cf-b34a-4cc4853c68b3\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:44:24 crc kubenswrapper[5052]: I1203 11:44:24.207016 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/bf425311-41fb-48cf-b34a-4cc4853c68b3-config-data\") pod \"cinder-scheduler-0\" (UID: \"bf425311-41fb-48cf-b34a-4cc4853c68b3\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:44:24 crc kubenswrapper[5052]: I1203 11:44:24.207076 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2dbe7002-eb09-4a0f-b65b-fc0294c13f4f-combined-ca-bundle\") pod \"neutron-7cc99c958f-bjn2l\" (UID: \"2dbe7002-eb09-4a0f-b65b-fc0294c13f4f\") " pod="openstack-kuttl-tests/neutron-7cc99c958f-bjn2l" Dec 03 11:44:24 crc kubenswrapper[5052]: I1203 11:44:24.207172 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/ee9c2229-695a-4930-89b0-39f140a3f328-fernet-keys\") pod \"keystone-bootstrap-fmkmc\" (UID: \"ee9c2229-695a-4930-89b0-39f140a3f328\") " pod="openstack-kuttl-tests/keystone-bootstrap-fmkmc" Dec 03 11:44:24 crc kubenswrapper[5052]: I1203 11:44:24.207263 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-97qf8\" (UniqueName: \"kubernetes.io/projected/ee9c2229-695a-4930-89b0-39f140a3f328-kube-api-access-97qf8\") pod \"keystone-bootstrap-fmkmc\" (UID: \"ee9c2229-695a-4930-89b0-39f140a3f328\") " pod="openstack-kuttl-tests/keystone-bootstrap-fmkmc" Dec 03 11:44:24 crc kubenswrapper[5052]: I1203 11:44:24.207320 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/bf425311-41fb-48cf-b34a-4cc4853c68b3-config-data-custom\") pod \"cinder-scheduler-0\" (UID: \"bf425311-41fb-48cf-b34a-4cc4853c68b3\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:44:24 crc kubenswrapper[5052]: I1203 11:44:24.207643 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/2dbe7002-eb09-4a0f-b65b-fc0294c13f4f-config\") pod \"neutron-7cc99c958f-bjn2l\" (UID: \"2dbe7002-eb09-4a0f-b65b-fc0294c13f4f\") " pod="openstack-kuttl-tests/neutron-7cc99c958f-bjn2l" Dec 03 11:44:24 crc kubenswrapper[5052]: I1203 11:44:24.207694 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/ee9c2229-695a-4930-89b0-39f140a3f328-config-data\") pod \"keystone-bootstrap-fmkmc\" (UID: \"ee9c2229-695a-4930-89b0-39f140a3f328\") " pod="openstack-kuttl-tests/keystone-bootstrap-fmkmc" Dec 03 11:44:24 crc kubenswrapper[5052]: I1203 11:44:24.207779 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/ee9c2229-695a-4930-89b0-39f140a3f328-credential-keys\") pod \"keystone-bootstrap-fmkmc\" (UID: \"ee9c2229-695a-4930-89b0-39f140a3f328\") " pod="openstack-kuttl-tests/keystone-bootstrap-fmkmc" Dec 03 11:44:24 crc kubenswrapper[5052]: I1203 11:44:24.208302 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r9mgh\" (UniqueName: \"kubernetes.io/projected/bf425311-41fb-48cf-b34a-4cc4853c68b3-kube-api-access-r9mgh\") pod \"cinder-scheduler-0\" (UID: \"bf425311-41fb-48cf-b34a-4cc4853c68b3\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:44:24 crc kubenswrapper[5052]: I1203 11:44:24.233655 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/glance-default-external-api-0" podStartSLOduration=5.233628569 podStartE2EDuration="5.233628569s" podCreationTimestamp="2025-12-03 11:44:19 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:44:24.167033349 +0000 UTC m=+1454.755275113" watchObservedRunningTime="2025-12-03 11:44:24.233628569 +0000 UTC m=+1454.821870333" Dec 03 11:44:24 crc kubenswrapper[5052]: I1203 11:44:24.233659 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 11:44:24 crc kubenswrapper[5052]: I1203 11:44:24.269677 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3d4a523e-547f-4d7a-920f-7a24ca45f183-config-data" (OuterVolumeSpecName: "config-data") pod "3d4a523e-547f-4d7a-920f-7a24ca45f183" (UID: "3d4a523e-547f-4d7a-920f-7a24ca45f183"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:44:24 crc kubenswrapper[5052]: I1203 11:44:24.313164 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/bf425311-41fb-48cf-b34a-4cc4853c68b3-etc-machine-id\") pod \"cinder-scheduler-0\" (UID: \"bf425311-41fb-48cf-b34a-4cc4853c68b3\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:44:24 crc kubenswrapper[5052]: I1203 11:44:24.313886 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/bf425311-41fb-48cf-b34a-4cc4853c68b3-config-data\") pod \"cinder-scheduler-0\" (UID: \"bf425311-41fb-48cf-b34a-4cc4853c68b3\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:44:24 crc kubenswrapper[5052]: I1203 11:44:24.314012 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2dbe7002-eb09-4a0f-b65b-fc0294c13f4f-combined-ca-bundle\") pod \"neutron-7cc99c958f-bjn2l\" (UID: \"2dbe7002-eb09-4a0f-b65b-fc0294c13f4f\") " pod="openstack-kuttl-tests/neutron-7cc99c958f-bjn2l" Dec 03 11:44:24 crc kubenswrapper[5052]: I1203 11:44:24.314223 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/ee9c2229-695a-4930-89b0-39f140a3f328-fernet-keys\") pod \"keystone-bootstrap-fmkmc\" (UID: \"ee9c2229-695a-4930-89b0-39f140a3f328\") " pod="openstack-kuttl-tests/keystone-bootstrap-fmkmc" Dec 03 11:44:24 crc kubenswrapper[5052]: I1203 11:44:24.314371 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-97qf8\" (UniqueName: \"kubernetes.io/projected/ee9c2229-695a-4930-89b0-39f140a3f328-kube-api-access-97qf8\") pod \"keystone-bootstrap-fmkmc\" (UID: \"ee9c2229-695a-4930-89b0-39f140a3f328\") " pod="openstack-kuttl-tests/keystone-bootstrap-fmkmc" Dec 03 11:44:24 crc kubenswrapper[5052]: I1203 11:44:24.314753 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/bf425311-41fb-48cf-b34a-4cc4853c68b3-config-data-custom\") pod \"cinder-scheduler-0\" (UID: \"bf425311-41fb-48cf-b34a-4cc4853c68b3\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:44:24 crc kubenswrapper[5052]: I1203 11:44:24.314914 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/7f4f10f6-2b65-45b9-a77e-ce3136668e1d-config-data\") pod \"cinder-api-0\" (UID: \"7f4f10f6-2b65-45b9-a77e-ce3136668e1d\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:44:24 crc kubenswrapper[5052]: I1203 11:44:24.315029 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/2dbe7002-eb09-4a0f-b65b-fc0294c13f4f-config\") pod \"neutron-7cc99c958f-bjn2l\" (UID: \"2dbe7002-eb09-4a0f-b65b-fc0294c13f4f\") " pod="openstack-kuttl-tests/neutron-7cc99c958f-bjn2l" Dec 03 11:44:24 crc kubenswrapper[5052]: I1203 11:44:24.315136 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/ee9c2229-695a-4930-89b0-39f140a3f328-config-data\") pod \"keystone-bootstrap-fmkmc\" (UID: \"ee9c2229-695a-4930-89b0-39f140a3f328\") " pod="openstack-kuttl-tests/keystone-bootstrap-fmkmc" Dec 03 11:44:24 crc kubenswrapper[5052]: I1203 11:44:24.315260 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/7f4f10f6-2b65-45b9-a77e-ce3136668e1d-etc-machine-id\") pod \"cinder-api-0\" (UID: \"7f4f10f6-2b65-45b9-a77e-ce3136668e1d\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:44:24 crc kubenswrapper[5052]: I1203 11:44:24.315407 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/ee9c2229-695a-4930-89b0-39f140a3f328-credential-keys\") pod \"keystone-bootstrap-fmkmc\" (UID: \"ee9c2229-695a-4930-89b0-39f140a3f328\") " pod="openstack-kuttl-tests/keystone-bootstrap-fmkmc" Dec 03 11:44:24 crc kubenswrapper[5052]: I1203 11:44:24.315549 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-r9mgh\" (UniqueName: \"kubernetes.io/projected/bf425311-41fb-48cf-b34a-4cc4853c68b3-kube-api-access-r9mgh\") pod \"cinder-scheduler-0\" (UID: \"bf425311-41fb-48cf-b34a-4cc4853c68b3\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:44:24 crc kubenswrapper[5052]: I1203 11:44:24.315679 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/7f4f10f6-2b65-45b9-a77e-ce3136668e1d-combined-ca-bundle\") pod \"cinder-api-0\" (UID: \"7f4f10f6-2b65-45b9-a77e-ce3136668e1d\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:44:24 crc kubenswrapper[5052]: I1203 11:44:24.315839 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/7f4f10f6-2b65-45b9-a77e-ce3136668e1d-scripts\") pod \"cinder-api-0\" (UID: \"7f4f10f6-2b65-45b9-a77e-ce3136668e1d\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:44:24 crc kubenswrapper[5052]: I1203 11:44:24.316021 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-45pfr\" (UniqueName: \"kubernetes.io/projected/7f4f10f6-2b65-45b9-a77e-ce3136668e1d-kube-api-access-45pfr\") pod \"cinder-api-0\" (UID: \"7f4f10f6-2b65-45b9-a77e-ce3136668e1d\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:44:24 crc kubenswrapper[5052]: I1203 11:44:24.316224 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ee9c2229-695a-4930-89b0-39f140a3f328-combined-ca-bundle\") pod \"keystone-bootstrap-fmkmc\" (UID: \"ee9c2229-695a-4930-89b0-39f140a3f328\") " pod="openstack-kuttl-tests/keystone-bootstrap-fmkmc" Dec 03 11:44:24 crc kubenswrapper[5052]: I1203 11:44:24.316352 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/2dbe7002-eb09-4a0f-b65b-fc0294c13f4f-ovndb-tls-certs\") pod \"neutron-7cc99c958f-bjn2l\" (UID: \"2dbe7002-eb09-4a0f-b65b-fc0294c13f4f\") " pod="openstack-kuttl-tests/neutron-7cc99c958f-bjn2l" Dec 03 11:44:24 crc kubenswrapper[5052]: I1203 11:44:24.316541 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/7f4f10f6-2b65-45b9-a77e-ce3136668e1d-config-data-custom\") pod \"cinder-api-0\" (UID: \"7f4f10f6-2b65-45b9-a77e-ce3136668e1d\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:44:24 crc kubenswrapper[5052]: I1203 11:44:24.316680 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/bf425311-41fb-48cf-b34a-4cc4853c68b3-combined-ca-bundle\") pod \"cinder-scheduler-0\" (UID: \"bf425311-41fb-48cf-b34a-4cc4853c68b3\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:44:24 crc kubenswrapper[5052]: I1203 11:44:24.316795 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/bf425311-41fb-48cf-b34a-4cc4853c68b3-scripts\") pod \"cinder-scheduler-0\" (UID: \"bf425311-41fb-48cf-b34a-4cc4853c68b3\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:44:24 crc kubenswrapper[5052]: I1203 11:44:24.316910 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/2dbe7002-eb09-4a0f-b65b-fc0294c13f4f-httpd-config\") pod \"neutron-7cc99c958f-bjn2l\" (UID: \"2dbe7002-eb09-4a0f-b65b-fc0294c13f4f\") " pod="openstack-kuttl-tests/neutron-7cc99c958f-bjn2l" Dec 03 11:44:24 crc kubenswrapper[5052]: I1203 11:44:24.317055 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-22j7r\" (UniqueName: \"kubernetes.io/projected/2dbe7002-eb09-4a0f-b65b-fc0294c13f4f-kube-api-access-22j7r\") pod \"neutron-7cc99c958f-bjn2l\" (UID: \"2dbe7002-eb09-4a0f-b65b-fc0294c13f4f\") " pod="openstack-kuttl-tests/neutron-7cc99c958f-bjn2l" Dec 03 11:44:24 crc kubenswrapper[5052]: I1203 11:44:24.317184 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/ee9c2229-695a-4930-89b0-39f140a3f328-scripts\") pod \"keystone-bootstrap-fmkmc\" (UID: \"ee9c2229-695a-4930-89b0-39f140a3f328\") " pod="openstack-kuttl-tests/keystone-bootstrap-fmkmc" Dec 03 11:44:24 crc kubenswrapper[5052]: I1203 11:44:24.317308 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/7f4f10f6-2b65-45b9-a77e-ce3136668e1d-logs\") pod \"cinder-api-0\" (UID: \"7f4f10f6-2b65-45b9-a77e-ce3136668e1d\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:44:24 crc kubenswrapper[5052]: I1203 11:44:24.317546 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/3d4a523e-547f-4d7a-920f-7a24ca45f183-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:44:24 crc kubenswrapper[5052]: I1203 11:44:24.317862 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/bf425311-41fb-48cf-b34a-4cc4853c68b3-etc-machine-id\") pod \"cinder-scheduler-0\" (UID: \"bf425311-41fb-48cf-b34a-4cc4853c68b3\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:44:24 crc kubenswrapper[5052]: I1203 11:44:24.336014 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ee9c2229-695a-4930-89b0-39f140a3f328-combined-ca-bundle\") pod \"keystone-bootstrap-fmkmc\" (UID: \"ee9c2229-695a-4930-89b0-39f140a3f328\") " pod="openstack-kuttl-tests/keystone-bootstrap-fmkmc" Dec 03 11:44:24 crc kubenswrapper[5052]: I1203 11:44:24.336262 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/bf425311-41fb-48cf-b34a-4cc4853c68b3-scripts\") pod \"cinder-scheduler-0\" (UID: \"bf425311-41fb-48cf-b34a-4cc4853c68b3\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:44:24 crc kubenswrapper[5052]: I1203 11:44:24.336438 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/barbican-keystone-listener-6794d844d6-f4tr7" podStartSLOduration=5.336420444 podStartE2EDuration="5.336420444s" podCreationTimestamp="2025-12-03 11:44:19 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:44:24.228346547 +0000 UTC m=+1454.816588311" watchObservedRunningTime="2025-12-03 11:44:24.336420444 +0000 UTC m=+1454.924662208" Dec 03 11:44:24 crc kubenswrapper[5052]: I1203 11:44:24.336495 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/ee9c2229-695a-4930-89b0-39f140a3f328-fernet-keys\") pod \"keystone-bootstrap-fmkmc\" (UID: \"ee9c2229-695a-4930-89b0-39f140a3f328\") " pod="openstack-kuttl-tests/keystone-bootstrap-fmkmc" Dec 03 11:44:24 crc kubenswrapper[5052]: I1203 11:44:24.336702 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2dbe7002-eb09-4a0f-b65b-fc0294c13f4f-combined-ca-bundle\") pod \"neutron-7cc99c958f-bjn2l\" (UID: \"2dbe7002-eb09-4a0f-b65b-fc0294c13f4f\") " pod="openstack-kuttl-tests/neutron-7cc99c958f-bjn2l" Dec 03 11:44:24 crc kubenswrapper[5052]: I1203 11:44:24.336760 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/2dbe7002-eb09-4a0f-b65b-fc0294c13f4f-config\") pod \"neutron-7cc99c958f-bjn2l\" (UID: \"2dbe7002-eb09-4a0f-b65b-fc0294c13f4f\") " pod="openstack-kuttl-tests/neutron-7cc99c958f-bjn2l" Dec 03 11:44:24 crc kubenswrapper[5052]: I1203 11:44:24.338469 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/bf425311-41fb-48cf-b34a-4cc4853c68b3-config-data\") pod \"cinder-scheduler-0\" (UID: \"bf425311-41fb-48cf-b34a-4cc4853c68b3\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:44:24 crc kubenswrapper[5052]: I1203 11:44:24.339205 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/bf425311-41fb-48cf-b34a-4cc4853c68b3-combined-ca-bundle\") pod \"cinder-scheduler-0\" (UID: \"bf425311-41fb-48cf-b34a-4cc4853c68b3\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:44:24 crc kubenswrapper[5052]: I1203 11:44:24.340230 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/ee9c2229-695a-4930-89b0-39f140a3f328-config-data\") pod \"keystone-bootstrap-fmkmc\" (UID: \"ee9c2229-695a-4930-89b0-39f140a3f328\") " pod="openstack-kuttl-tests/keystone-bootstrap-fmkmc" Dec 03 11:44:24 crc kubenswrapper[5052]: I1203 11:44:24.343585 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/bf425311-41fb-48cf-b34a-4cc4853c68b3-config-data-custom\") pod \"cinder-scheduler-0\" (UID: \"bf425311-41fb-48cf-b34a-4cc4853c68b3\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:44:24 crc kubenswrapper[5052]: I1203 11:44:24.348958 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/ee9c2229-695a-4930-89b0-39f140a3f328-credential-keys\") pod \"keystone-bootstrap-fmkmc\" (UID: \"ee9c2229-695a-4930-89b0-39f140a3f328\") " pod="openstack-kuttl-tests/keystone-bootstrap-fmkmc" Dec 03 11:44:24 crc kubenswrapper[5052]: I1203 11:44:24.354905 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/2dbe7002-eb09-4a0f-b65b-fc0294c13f4f-httpd-config\") pod \"neutron-7cc99c958f-bjn2l\" (UID: \"2dbe7002-eb09-4a0f-b65b-fc0294c13f4f\") " pod="openstack-kuttl-tests/neutron-7cc99c958f-bjn2l" Dec 03 11:44:24 crc kubenswrapper[5052]: I1203 11:44:24.355379 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-97qf8\" (UniqueName: \"kubernetes.io/projected/ee9c2229-695a-4930-89b0-39f140a3f328-kube-api-access-97qf8\") pod \"keystone-bootstrap-fmkmc\" (UID: \"ee9c2229-695a-4930-89b0-39f140a3f328\") " pod="openstack-kuttl-tests/keystone-bootstrap-fmkmc" Dec 03 11:44:24 crc kubenswrapper[5052]: I1203 11:44:24.355491 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/2dbe7002-eb09-4a0f-b65b-fc0294c13f4f-ovndb-tls-certs\") pod \"neutron-7cc99c958f-bjn2l\" (UID: \"2dbe7002-eb09-4a0f-b65b-fc0294c13f4f\") " pod="openstack-kuttl-tests/neutron-7cc99c958f-bjn2l" Dec 03 11:44:24 crc kubenswrapper[5052]: I1203 11:44:24.355491 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-r9mgh\" (UniqueName: \"kubernetes.io/projected/bf425311-41fb-48cf-b34a-4cc4853c68b3-kube-api-access-r9mgh\") pod \"cinder-scheduler-0\" (UID: \"bf425311-41fb-48cf-b34a-4cc4853c68b3\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:44:24 crc kubenswrapper[5052]: I1203 11:44:24.355742 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/ee9c2229-695a-4930-89b0-39f140a3f328-scripts\") pod \"keystone-bootstrap-fmkmc\" (UID: \"ee9c2229-695a-4930-89b0-39f140a3f328\") " pod="openstack-kuttl-tests/keystone-bootstrap-fmkmc" Dec 03 11:44:24 crc kubenswrapper[5052]: I1203 11:44:24.358924 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fb3a323e-09aa-47ac-b593-e1c6e9b898e5-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "fb3a323e-09aa-47ac-b593-e1c6e9b898e5" (UID: "fb3a323e-09aa-47ac-b593-e1c6e9b898e5"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:44:24 crc kubenswrapper[5052]: I1203 11:44:24.370686 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fb3a323e-09aa-47ac-b593-e1c6e9b898e5-config-data" (OuterVolumeSpecName: "config-data") pod "fb3a323e-09aa-47ac-b593-e1c6e9b898e5" (UID: "fb3a323e-09aa-47ac-b593-e1c6e9b898e5"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:44:24 crc kubenswrapper[5052]: I1203 11:44:24.370754 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-22j7r\" (UniqueName: \"kubernetes.io/projected/2dbe7002-eb09-4a0f-b65b-fc0294c13f4f-kube-api-access-22j7r\") pod \"neutron-7cc99c958f-bjn2l\" (UID: \"2dbe7002-eb09-4a0f-b65b-fc0294c13f4f\") " pod="openstack-kuttl-tests/neutron-7cc99c958f-bjn2l" Dec 03 11:44:24 crc kubenswrapper[5052]: I1203 11:44:24.419005 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/7f4f10f6-2b65-45b9-a77e-ce3136668e1d-logs\") pod \"cinder-api-0\" (UID: \"7f4f10f6-2b65-45b9-a77e-ce3136668e1d\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:44:24 crc kubenswrapper[5052]: I1203 11:44:24.419404 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/7f4f10f6-2b65-45b9-a77e-ce3136668e1d-config-data\") pod \"cinder-api-0\" (UID: \"7f4f10f6-2b65-45b9-a77e-ce3136668e1d\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:44:24 crc kubenswrapper[5052]: I1203 11:44:24.419441 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/7f4f10f6-2b65-45b9-a77e-ce3136668e1d-etc-machine-id\") pod \"cinder-api-0\" (UID: \"7f4f10f6-2b65-45b9-a77e-ce3136668e1d\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:44:24 crc kubenswrapper[5052]: I1203 11:44:24.419494 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/7f4f10f6-2b65-45b9-a77e-ce3136668e1d-combined-ca-bundle\") pod \"cinder-api-0\" (UID: \"7f4f10f6-2b65-45b9-a77e-ce3136668e1d\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:44:24 crc kubenswrapper[5052]: I1203 11:44:24.419536 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/7f4f10f6-2b65-45b9-a77e-ce3136668e1d-scripts\") pod \"cinder-api-0\" (UID: \"7f4f10f6-2b65-45b9-a77e-ce3136668e1d\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:44:24 crc kubenswrapper[5052]: I1203 11:44:24.419574 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-45pfr\" (UniqueName: \"kubernetes.io/projected/7f4f10f6-2b65-45b9-a77e-ce3136668e1d-kube-api-access-45pfr\") pod \"cinder-api-0\" (UID: \"7f4f10f6-2b65-45b9-a77e-ce3136668e1d\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:44:24 crc kubenswrapper[5052]: I1203 11:44:24.419650 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/7f4f10f6-2b65-45b9-a77e-ce3136668e1d-config-data-custom\") pod \"cinder-api-0\" (UID: \"7f4f10f6-2b65-45b9-a77e-ce3136668e1d\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:44:24 crc kubenswrapper[5052]: I1203 11:44:24.419692 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/7f4f10f6-2b65-45b9-a77e-ce3136668e1d-etc-machine-id\") pod \"cinder-api-0\" (UID: \"7f4f10f6-2b65-45b9-a77e-ce3136668e1d\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:44:24 crc kubenswrapper[5052]: I1203 11:44:24.419716 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/fb3a323e-09aa-47ac-b593-e1c6e9b898e5-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:44:24 crc kubenswrapper[5052]: I1203 11:44:24.419753 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/fb3a323e-09aa-47ac-b593-e1c6e9b898e5-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:44:24 crc kubenswrapper[5052]: I1203 11:44:24.422797 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/7f4f10f6-2b65-45b9-a77e-ce3136668e1d-scripts\") pod \"cinder-api-0\" (UID: \"7f4f10f6-2b65-45b9-a77e-ce3136668e1d\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:44:24 crc kubenswrapper[5052]: I1203 11:44:24.424492 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/7f4f10f6-2b65-45b9-a77e-ce3136668e1d-config-data\") pod \"cinder-api-0\" (UID: \"7f4f10f6-2b65-45b9-a77e-ce3136668e1d\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:44:24 crc kubenswrapper[5052]: I1203 11:44:24.425020 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/7f4f10f6-2b65-45b9-a77e-ce3136668e1d-logs\") pod \"cinder-api-0\" (UID: \"7f4f10f6-2b65-45b9-a77e-ce3136668e1d\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:44:24 crc kubenswrapper[5052]: I1203 11:44:24.429094 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/7f4f10f6-2b65-45b9-a77e-ce3136668e1d-combined-ca-bundle\") pod \"cinder-api-0\" (UID: \"7f4f10f6-2b65-45b9-a77e-ce3136668e1d\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:44:24 crc kubenswrapper[5052]: I1203 11:44:24.432735 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/7f4f10f6-2b65-45b9-a77e-ce3136668e1d-config-data-custom\") pod \"cinder-api-0\" (UID: \"7f4f10f6-2b65-45b9-a77e-ce3136668e1d\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:44:24 crc kubenswrapper[5052]: I1203 11:44:24.444075 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-45pfr\" (UniqueName: \"kubernetes.io/projected/7f4f10f6-2b65-45b9-a77e-ce3136668e1d-kube-api-access-45pfr\") pod \"cinder-api-0\" (UID: \"7f4f10f6-2b65-45b9-a77e-ce3136668e1d\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:44:24 crc kubenswrapper[5052]: I1203 11:44:24.530035 5052 scope.go:117] "RemoveContainer" containerID="1349f4ae2e79b5b6366e52f5c0162ec782e07b2a25ad7f647b70280b2223aa0f" Dec 03 11:44:24 crc kubenswrapper[5052]: I1203 11:44:24.537297 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-7cc99c958f-bjn2l" Dec 03 11:44:24 crc kubenswrapper[5052]: I1203 11:44:24.547765 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:44:24 crc kubenswrapper[5052]: I1203 11:44:24.586918 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-bootstrap-fmkmc" Dec 03 11:44:24 crc kubenswrapper[5052]: I1203 11:44:24.591291 5052 scope.go:117] "RemoveContainer" containerID="9bf88bfd0d0e6f8a144f303520d8196838052ec8c253b8b04069315b384e953d" Dec 03 11:44:24 crc kubenswrapper[5052]: I1203 11:44:24.593831 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:44:24 crc kubenswrapper[5052]: I1203 11:44:24.660183 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:44:24 crc kubenswrapper[5052]: I1203 11:44:24.671153 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:44:24 crc kubenswrapper[5052]: E1203 11:44:24.675198 5052 cadvisor_stats_provider.go:516] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfb3a323e_09aa_47ac_b593_e1c6e9b898e5.slice\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3d4a523e_547f_4d7a_920f_7a24ca45f183.slice\": RecentStats: unable to find data in memory cache]" Dec 03 11:44:24 crc kubenswrapper[5052]: I1203 11:44:24.707751 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:44:24 crc kubenswrapper[5052]: I1203 11:44:24.710168 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:44:24 crc kubenswrapper[5052]: I1203 11:44:24.716122 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:44:24 crc kubenswrapper[5052]: I1203 11:44:24.716163 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-config-data" Dec 03 11:44:24 crc kubenswrapper[5052]: I1203 11:44:24.717024 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-scripts" Dec 03 11:44:24 crc kubenswrapper[5052]: I1203 11:44:24.732792 5052 scope.go:117] "RemoveContainer" containerID="7bff6b5804029ed658f2a1d5487e3bce63445ac4e6250251b735aee6dff232f5" Dec 03 11:44:24 crc kubenswrapper[5052]: I1203 11:44:24.827960 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/b6eeb188-0ea6-46de-8cbf-d803e75df5e6-scripts\") pod \"ceilometer-0\" (UID: \"b6eeb188-0ea6-46de-8cbf-d803e75df5e6\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:44:24 crc kubenswrapper[5052]: I1203 11:44:24.830627 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b6eeb188-0ea6-46de-8cbf-d803e75df5e6-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"b6eeb188-0ea6-46de-8cbf-d803e75df5e6\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:44:24 crc kubenswrapper[5052]: I1203 11:44:24.830988 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/b6eeb188-0ea6-46de-8cbf-d803e75df5e6-run-httpd\") pod \"ceilometer-0\" (UID: \"b6eeb188-0ea6-46de-8cbf-d803e75df5e6\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:44:24 crc kubenswrapper[5052]: I1203 11:44:24.831482 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/b6eeb188-0ea6-46de-8cbf-d803e75df5e6-config-data\") pod \"ceilometer-0\" (UID: \"b6eeb188-0ea6-46de-8cbf-d803e75df5e6\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:44:24 crc kubenswrapper[5052]: I1203 11:44:24.831874 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/b6eeb188-0ea6-46de-8cbf-d803e75df5e6-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"b6eeb188-0ea6-46de-8cbf-d803e75df5e6\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:44:24 crc kubenswrapper[5052]: I1203 11:44:24.832380 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/b6eeb188-0ea6-46de-8cbf-d803e75df5e6-log-httpd\") pod \"ceilometer-0\" (UID: \"b6eeb188-0ea6-46de-8cbf-d803e75df5e6\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:44:24 crc kubenswrapper[5052]: I1203 11:44:24.832781 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n8s4c\" (UniqueName: \"kubernetes.io/projected/b6eeb188-0ea6-46de-8cbf-d803e75df5e6-kube-api-access-n8s4c\") pod \"ceilometer-0\" (UID: \"b6eeb188-0ea6-46de-8cbf-d803e75df5e6\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:44:24 crc kubenswrapper[5052]: I1203 11:44:24.935370 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/b6eeb188-0ea6-46de-8cbf-d803e75df5e6-log-httpd\") pod \"ceilometer-0\" (UID: \"b6eeb188-0ea6-46de-8cbf-d803e75df5e6\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:44:24 crc kubenswrapper[5052]: I1203 11:44:24.935468 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-n8s4c\" (UniqueName: \"kubernetes.io/projected/b6eeb188-0ea6-46de-8cbf-d803e75df5e6-kube-api-access-n8s4c\") pod \"ceilometer-0\" (UID: \"b6eeb188-0ea6-46de-8cbf-d803e75df5e6\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:44:24 crc kubenswrapper[5052]: I1203 11:44:24.935563 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/b6eeb188-0ea6-46de-8cbf-d803e75df5e6-scripts\") pod \"ceilometer-0\" (UID: \"b6eeb188-0ea6-46de-8cbf-d803e75df5e6\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:44:24 crc kubenswrapper[5052]: I1203 11:44:24.935588 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b6eeb188-0ea6-46de-8cbf-d803e75df5e6-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"b6eeb188-0ea6-46de-8cbf-d803e75df5e6\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:44:24 crc kubenswrapper[5052]: I1203 11:44:24.935621 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/b6eeb188-0ea6-46de-8cbf-d803e75df5e6-run-httpd\") pod \"ceilometer-0\" (UID: \"b6eeb188-0ea6-46de-8cbf-d803e75df5e6\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:44:24 crc kubenswrapper[5052]: I1203 11:44:24.935656 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/b6eeb188-0ea6-46de-8cbf-d803e75df5e6-config-data\") pod \"ceilometer-0\" (UID: \"b6eeb188-0ea6-46de-8cbf-d803e75df5e6\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:44:24 crc kubenswrapper[5052]: I1203 11:44:24.935688 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/b6eeb188-0ea6-46de-8cbf-d803e75df5e6-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"b6eeb188-0ea6-46de-8cbf-d803e75df5e6\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:44:24 crc kubenswrapper[5052]: I1203 11:44:24.937358 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/b6eeb188-0ea6-46de-8cbf-d803e75df5e6-log-httpd\") pod \"ceilometer-0\" (UID: \"b6eeb188-0ea6-46de-8cbf-d803e75df5e6\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:44:24 crc kubenswrapper[5052]: I1203 11:44:24.937631 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/b6eeb188-0ea6-46de-8cbf-d803e75df5e6-run-httpd\") pod \"ceilometer-0\" (UID: \"b6eeb188-0ea6-46de-8cbf-d803e75df5e6\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:44:24 crc kubenswrapper[5052]: I1203 11:44:24.943492 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/b6eeb188-0ea6-46de-8cbf-d803e75df5e6-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"b6eeb188-0ea6-46de-8cbf-d803e75df5e6\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:44:24 crc kubenswrapper[5052]: I1203 11:44:24.957597 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/b6eeb188-0ea6-46de-8cbf-d803e75df5e6-scripts\") pod \"ceilometer-0\" (UID: \"b6eeb188-0ea6-46de-8cbf-d803e75df5e6\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:44:24 crc kubenswrapper[5052]: I1203 11:44:24.959105 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b6eeb188-0ea6-46de-8cbf-d803e75df5e6-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"b6eeb188-0ea6-46de-8cbf-d803e75df5e6\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:44:24 crc kubenswrapper[5052]: I1203 11:44:24.959502 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/b6eeb188-0ea6-46de-8cbf-d803e75df5e6-config-data\") pod \"ceilometer-0\" (UID: \"b6eeb188-0ea6-46de-8cbf-d803e75df5e6\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:44:24 crc kubenswrapper[5052]: I1203 11:44:24.968420 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-n8s4c\" (UniqueName: \"kubernetes.io/projected/b6eeb188-0ea6-46de-8cbf-d803e75df5e6-kube-api-access-n8s4c\") pod \"ceilometer-0\" (UID: \"b6eeb188-0ea6-46de-8cbf-d803e75df5e6\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:44:25 crc kubenswrapper[5052]: I1203 11:44:25.059412 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:44:25 crc kubenswrapper[5052]: I1203 11:44:25.100137 5052 generic.go:334] "Generic (PLEG): container finished" podID="f8e49a02-823b-4851-9ecf-f2d1696fa201" containerID="3b836afc76cec7f0340dacbace97852b77688d99cd5e579a6908a42d2753abf1" exitCode=1 Dec 03 11:44:25 crc kubenswrapper[5052]: I1203 11:44:25.100184 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-79d5b958cd-cq6hs" event={"ID":"f8e49a02-823b-4851-9ecf-f2d1696fa201","Type":"ContainerDied","Data":"3b836afc76cec7f0340dacbace97852b77688d99cd5e579a6908a42d2753abf1"} Dec 03 11:44:25 crc kubenswrapper[5052]: I1203 11:44:25.100253 5052 scope.go:117] "RemoveContainer" containerID="2da14680cf8eb402a210aa0b17d027537e969fa081339d2233cfa2f5c81dd7fd" Dec 03 11:44:25 crc kubenswrapper[5052]: I1203 11:44:25.100788 5052 scope.go:117] "RemoveContainer" containerID="3b836afc76cec7f0340dacbace97852b77688d99cd5e579a6908a42d2753abf1" Dec 03 11:44:25 crc kubenswrapper[5052]: E1203 11:44:25.101010 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"placement-api\" with CrashLoopBackOff: \"back-off 10s restarting failed container=placement-api pod=placement-79d5b958cd-cq6hs_openstack-kuttl-tests(f8e49a02-823b-4851-9ecf-f2d1696fa201)\"" pod="openstack-kuttl-tests/placement-79d5b958cd-cq6hs" podUID="f8e49a02-823b-4851-9ecf-f2d1696fa201" Dec 03 11:44:25 crc kubenswrapper[5052]: I1203 11:44:25.114643 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-689c6465d6-nsjrx" event={"ID":"0183ebb5-b91b-443a-9535-2008c9207ab1","Type":"ContainerStarted","Data":"81c366447116b43e07821471a9d38ae69fcdb20b1db58e745b91476f5acbbc2f"} Dec 03 11:44:25 crc kubenswrapper[5052]: I1203 11:44:25.160038 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/placement-689c6465d6-nsjrx" podStartSLOduration=5.160012974 podStartE2EDuration="5.160012974s" podCreationTimestamp="2025-12-03 11:44:20 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:44:25.153020422 +0000 UTC m=+1455.741262196" watchObservedRunningTime="2025-12-03 11:44:25.160012974 +0000 UTC m=+1455.748254738" Dec 03 11:44:25 crc kubenswrapper[5052]: I1203 11:44:25.211793 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-7cc99c958f-bjn2l"] Dec 03 11:44:25 crc kubenswrapper[5052]: W1203 11:44:25.221176 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2dbe7002_eb09_4a0f_b65b_fc0294c13f4f.slice/crio-0357ef8312fb1e8f4b30dd9b2c0a1cf44e8e5977c178167e5200a3bdf5bd3f86 WatchSource:0}: Error finding container 0357ef8312fb1e8f4b30dd9b2c0a1cf44e8e5977c178167e5200a3bdf5bd3f86: Status 404 returned error can't find the container with id 0357ef8312fb1e8f4b30dd9b2c0a1cf44e8e5977c178167e5200a3bdf5bd3f86 Dec 03 11:44:25 crc kubenswrapper[5052]: I1203 11:44:25.303374 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 11:44:25 crc kubenswrapper[5052]: I1203 11:44:25.312903 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-bootstrap-fmkmc"] Dec 03 11:44:25 crc kubenswrapper[5052]: I1203 11:44:25.324730 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 11:44:25 crc kubenswrapper[5052]: W1203 11:44:25.327621 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podee9c2229_695a_4930_89b0_39f140a3f328.slice/crio-4999b392a0945626cc73f9af4c88aab05d254ed4f9a3b4faf90f1f699d063189 WatchSource:0}: Error finding container 4999b392a0945626cc73f9af4c88aab05d254ed4f9a3b4faf90f1f699d063189: Status 404 returned error can't find the container with id 4999b392a0945626cc73f9af4c88aab05d254ed4f9a3b4faf90f1f699d063189 Dec 03 11:44:25 crc kubenswrapper[5052]: I1203 11:44:25.612272 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:44:25 crc kubenswrapper[5052]: I1203 11:44:25.696347 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b53d06f2-28c8-4c18-b9ce-0300f8b8ef06" path="/var/lib/kubelet/pods/b53d06f2-28c8-4c18-b9ce-0300f8b8ef06/volumes" Dec 03 11:44:25 crc kubenswrapper[5052]: I1203 11:44:25.697095 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fb3a323e-09aa-47ac-b593-e1c6e9b898e5" path="/var/lib/kubelet/pods/fb3a323e-09aa-47ac-b593-e1c6e9b898e5/volumes" Dec 03 11:44:26 crc kubenswrapper[5052]: I1203 11:44:26.146824 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-bootstrap-fmkmc" event={"ID":"ee9c2229-695a-4930-89b0-39f140a3f328","Type":"ContainerStarted","Data":"484767d2c94ae919e47f683d5bfab4a9ab2ee0aca466f6bef5f32634711c335c"} Dec 03 11:44:26 crc kubenswrapper[5052]: I1203 11:44:26.147282 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-bootstrap-fmkmc" event={"ID":"ee9c2229-695a-4930-89b0-39f140a3f328","Type":"ContainerStarted","Data":"4999b392a0945626cc73f9af4c88aab05d254ed4f9a3b4faf90f1f699d063189"} Dec 03 11:44:26 crc kubenswrapper[5052]: I1203 11:44:26.162005 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"bf425311-41fb-48cf-b34a-4cc4853c68b3","Type":"ContainerStarted","Data":"c17a1df51765ca364dfd03ed31414bd7d7177c8683e926b08de663253d1a4f0f"} Dec 03 11:44:26 crc kubenswrapper[5052]: I1203 11:44:26.162068 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"bf425311-41fb-48cf-b34a-4cc4853c68b3","Type":"ContainerStarted","Data":"c37381da810a9c885c1e6987cf95114e831ca98788e37fe70ce26ed8ce8fe27c"} Dec 03 11:44:26 crc kubenswrapper[5052]: I1203 11:44:26.172194 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"b6eeb188-0ea6-46de-8cbf-d803e75df5e6","Type":"ContainerStarted","Data":"d320b741f24a721bdb9881c1fc972da531a73e2249a75ed8d9f85e5a7dc52dbc"} Dec 03 11:44:26 crc kubenswrapper[5052]: I1203 11:44:26.177838 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/keystone-bootstrap-fmkmc" podStartSLOduration=2.177820363 podStartE2EDuration="2.177820363s" podCreationTimestamp="2025-12-03 11:44:24 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:44:26.166043594 +0000 UTC m=+1456.754285358" watchObservedRunningTime="2025-12-03 11:44:26.177820363 +0000 UTC m=+1456.766062127" Dec 03 11:44:26 crc kubenswrapper[5052]: I1203 11:44:26.180807 5052 scope.go:117] "RemoveContainer" containerID="3b836afc76cec7f0340dacbace97852b77688d99cd5e579a6908a42d2753abf1" Dec 03 11:44:26 crc kubenswrapper[5052]: E1203 11:44:26.181326 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"placement-api\" with CrashLoopBackOff: \"back-off 10s restarting failed container=placement-api pod=placement-79d5b958cd-cq6hs_openstack-kuttl-tests(f8e49a02-823b-4851-9ecf-f2d1696fa201)\"" pod="openstack-kuttl-tests/placement-79d5b958cd-cq6hs" podUID="f8e49a02-823b-4851-9ecf-f2d1696fa201" Dec 03 11:44:26 crc kubenswrapper[5052]: I1203 11:44:26.189829 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-7cc99c958f-bjn2l" event={"ID":"2dbe7002-eb09-4a0f-b65b-fc0294c13f4f","Type":"ContainerStarted","Data":"a60bd070c0bbd29b273ad38f5c6a3450100e8c13d43d5b00488a10b98645de23"} Dec 03 11:44:26 crc kubenswrapper[5052]: I1203 11:44:26.189893 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-7cc99c958f-bjn2l" event={"ID":"2dbe7002-eb09-4a0f-b65b-fc0294c13f4f","Type":"ContainerStarted","Data":"8c2a5717a95924a833c522bb45a7070432896bb7e0e5d55da6ef352206e5b20a"} Dec 03 11:44:26 crc kubenswrapper[5052]: I1203 11:44:26.189904 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-7cc99c958f-bjn2l" event={"ID":"2dbe7002-eb09-4a0f-b65b-fc0294c13f4f","Type":"ContainerStarted","Data":"0357ef8312fb1e8f4b30dd9b2c0a1cf44e8e5977c178167e5200a3bdf5bd3f86"} Dec 03 11:44:26 crc kubenswrapper[5052]: I1203 11:44:26.190172 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/neutron-7cc99c958f-bjn2l" Dec 03 11:44:26 crc kubenswrapper[5052]: I1203 11:44:26.202282 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"7f4f10f6-2b65-45b9-a77e-ce3136668e1d","Type":"ContainerStarted","Data":"6ea169f90116f631eb8741136bc27449af38f1ace846817d95ebb65f5ab46b30"} Dec 03 11:44:26 crc kubenswrapper[5052]: I1203 11:44:26.202352 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"7f4f10f6-2b65-45b9-a77e-ce3136668e1d","Type":"ContainerStarted","Data":"be5706533c73fa8a32f4fc8036cdaee26e5d8433ac23e6e1805423b20fca460c"} Dec 03 11:44:26 crc kubenswrapper[5052]: I1203 11:44:26.202442 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/placement-689c6465d6-nsjrx" Dec 03 11:44:26 crc kubenswrapper[5052]: I1203 11:44:26.202523 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/placement-689c6465d6-nsjrx" Dec 03 11:44:26 crc kubenswrapper[5052]: I1203 11:44:26.234435 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/neutron-7cc99c958f-bjn2l" podStartSLOduration=3.234413355 podStartE2EDuration="3.234413355s" podCreationTimestamp="2025-12-03 11:44:23 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:44:26.219114844 +0000 UTC m=+1456.807356608" watchObservedRunningTime="2025-12-03 11:44:26.234413355 +0000 UTC m=+1456.822655119" Dec 03 11:44:26 crc kubenswrapper[5052]: I1203 11:44:26.868910 5052 scope.go:117] "RemoveContainer" containerID="f4ccab5332191ea20d35a5a0230ee3ca43860ab19341b4d767df3f139c66e61f" Dec 03 11:44:26 crc kubenswrapper[5052]: I1203 11:44:26.899557 5052 scope.go:117] "RemoveContainer" containerID="b6fc837cf63e99e9b463372d833ef8414220a2bac6210bbf692afd78142eaaf5" Dec 03 11:44:26 crc kubenswrapper[5052]: I1203 11:44:26.945678 5052 scope.go:117] "RemoveContainer" containerID="92a6a8836b87d1315cfd2ea55a4b4571531256aa00150f268dbfca3e6eabb207" Dec 03 11:44:26 crc kubenswrapper[5052]: I1203 11:44:26.998319 5052 scope.go:117] "RemoveContainer" containerID="f60cd400b6652773ef4753bb550741040c4ae2eb9c6c8a5ebbe3a84a430ead37" Dec 03 11:44:27 crc kubenswrapper[5052]: I1203 11:44:27.051208 5052 scope.go:117] "RemoveContainer" containerID="d9bef6e726b038ae865fa13770a700b7a8625ffee06c08933704155c1695d94f" Dec 03 11:44:27 crc kubenswrapper[5052]: I1203 11:44:27.100915 5052 scope.go:117] "RemoveContainer" containerID="baa1660ab693e586b550adda4472e315a3f29bc7b42ec3764f341536de2ddf22" Dec 03 11:44:27 crc kubenswrapper[5052]: I1203 11:44:27.132781 5052 scope.go:117] "RemoveContainer" containerID="5d159c33361809fb42fe366ed481f946a656d0a56465673cfd5f86a09775940d" Dec 03 11:44:27 crc kubenswrapper[5052]: I1203 11:44:27.159653 5052 scope.go:117] "RemoveContainer" containerID="d95a648c64f8ec110244fef86abf1e3ac6b622589ba5777470f9f86092e7a4de" Dec 03 11:44:27 crc kubenswrapper[5052]: I1203 11:44:27.198605 5052 scope.go:117] "RemoveContainer" containerID="285f0301df3076e5804e7d1881c2e5541ac3790946cb4841c7f7ed169e3bdd4e" Dec 03 11:44:27 crc kubenswrapper[5052]: I1203 11:44:27.230837 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"7f4f10f6-2b65-45b9-a77e-ce3136668e1d","Type":"ContainerStarted","Data":"43c36870fd89ee414be871c98622b89a7cd23756349871cc418fe90efd9bd8ab"} Dec 03 11:44:27 crc kubenswrapper[5052]: I1203 11:44:27.232222 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:44:27 crc kubenswrapper[5052]: I1203 11:44:27.244153 5052 scope.go:117] "RemoveContainer" containerID="5f4f7fdc94d870e8565fff27865a12a867b3e689e5b0d716cbfdbcc13cfb6c9f" Dec 03 11:44:27 crc kubenswrapper[5052]: I1203 11:44:27.250834 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"bf425311-41fb-48cf-b34a-4cc4853c68b3","Type":"ContainerStarted","Data":"bb18e1ff35492d0cbb8cd1813c33935f9d2b132b5086ef0adb508337d0236453"} Dec 03 11:44:27 crc kubenswrapper[5052]: I1203 11:44:27.273644 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/cinder-api-0" podStartSLOduration=3.273620903 podStartE2EDuration="3.273620903s" podCreationTimestamp="2025-12-03 11:44:24 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:44:27.248824198 +0000 UTC m=+1457.837065982" watchObservedRunningTime="2025-12-03 11:44:27.273620903 +0000 UTC m=+1457.861862667" Dec 03 11:44:27 crc kubenswrapper[5052]: I1203 11:44:27.277788 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/cinder-scheduler-0" podStartSLOduration=4.277767773 podStartE2EDuration="4.277767773s" podCreationTimestamp="2025-12-03 11:44:23 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:44:27.273975333 +0000 UTC m=+1457.862217107" watchObservedRunningTime="2025-12-03 11:44:27.277767773 +0000 UTC m=+1457.866009537" Dec 03 11:44:27 crc kubenswrapper[5052]: I1203 11:44:27.287171 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"b6eeb188-0ea6-46de-8cbf-d803e75df5e6","Type":"ContainerStarted","Data":"df00125cee9eca43a7001e16f293bd9269f75ced8e07fe1609749fa6ec555248"} Dec 03 11:44:27 crc kubenswrapper[5052]: I1203 11:44:27.294822 5052 scope.go:117] "RemoveContainer" containerID="aab7d5710cba4d408281d37d8a730119c212b83f1f048e7073978d81874cfc2c" Dec 03 11:44:27 crc kubenswrapper[5052]: I1203 11:44:27.328954 5052 scope.go:117] "RemoveContainer" containerID="e0bd842931dfa2118f2a524f12f2a960d4ae5d1b4551cb7036c004b424844c69" Dec 03 11:44:27 crc kubenswrapper[5052]: I1203 11:44:27.357037 5052 scope.go:117] "RemoveContainer" containerID="6d71e00901cb2b18bfa9c3d46e251e99210b915413b7d7d03d74cb72b37c2e29" Dec 03 11:44:27 crc kubenswrapper[5052]: I1203 11:44:27.389323 5052 scope.go:117] "RemoveContainer" containerID="82e865c60792e8a1db857646f7e7a31829df3081386f9952e8ded31712c937e7" Dec 03 11:44:27 crc kubenswrapper[5052]: I1203 11:44:27.422059 5052 scope.go:117] "RemoveContainer" containerID="022f484f466350e668aa8ee358ecb5870270b3faf6ed2e427124749b11985787" Dec 03 11:44:27 crc kubenswrapper[5052]: I1203 11:44:27.448502 5052 scope.go:117] "RemoveContainer" containerID="5e0a4e3bdf4887c1a95e0d9815e6d8a1fc6c0dccf8359c3a1a7a1505e8cc9053" Dec 03 11:44:27 crc kubenswrapper[5052]: I1203 11:44:27.476001 5052 scope.go:117] "RemoveContainer" containerID="82bdd60bb6c2b4f3232ce11412a9103ff8cdf6022d10e2a0fc8d882e6a7cbde9" Dec 03 11:44:28 crc kubenswrapper[5052]: I1203 11:44:28.212265 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/barbican-api-f76947bf6-27nv8" Dec 03 11:44:28 crc kubenswrapper[5052]: I1203 11:44:28.323786 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"b6eeb188-0ea6-46de-8cbf-d803e75df5e6","Type":"ContainerStarted","Data":"ba76522ceb5c6ff71560ef00ae164eb9deadf017edb824ce2c749304302a9186"} Dec 03 11:44:28 crc kubenswrapper[5052]: I1203 11:44:28.323846 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"b6eeb188-0ea6-46de-8cbf-d803e75df5e6","Type":"ContainerStarted","Data":"a543e5d6e1f5b544fd17dc9d996401e2195186e8556773c54e76285b9c705f51"} Dec 03 11:44:28 crc kubenswrapper[5052]: I1203 11:44:28.901664 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:44:28 crc kubenswrapper[5052]: I1203 11:44:28.901706 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:44:28 crc kubenswrapper[5052]: I1203 11:44:28.938871 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:44:28 crc kubenswrapper[5052]: I1203 11:44:28.954216 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:44:29 crc kubenswrapper[5052]: I1203 11:44:29.067676 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 11:44:29 crc kubenswrapper[5052]: I1203 11:44:29.334604 5052 generic.go:334] "Generic (PLEG): container finished" podID="ee9c2229-695a-4930-89b0-39f140a3f328" containerID="484767d2c94ae919e47f683d5bfab4a9ab2ee0aca466f6bef5f32634711c335c" exitCode=0 Dec 03 11:44:29 crc kubenswrapper[5052]: I1203 11:44:29.334861 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-bootstrap-fmkmc" event={"ID":"ee9c2229-695a-4930-89b0-39f140a3f328","Type":"ContainerDied","Data":"484767d2c94ae919e47f683d5bfab4a9ab2ee0aca466f6bef5f32634711c335c"} Dec 03 11:44:29 crc kubenswrapper[5052]: I1203 11:44:29.335577 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:44:29 crc kubenswrapper[5052]: I1203 11:44:29.335998 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:44:29 crc kubenswrapper[5052]: I1203 11:44:29.553846 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:44:29 crc kubenswrapper[5052]: I1203 11:44:29.904852 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:44:29 crc kubenswrapper[5052]: I1203 11:44:29.905133 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:44:29 crc kubenswrapper[5052]: I1203 11:44:29.952216 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:44:29 crc kubenswrapper[5052]: I1203 11:44:29.955025 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:44:30 crc kubenswrapper[5052]: I1203 11:44:30.032701 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/barbican-api-f76947bf6-27nv8" Dec 03 11:44:30 crc kubenswrapper[5052]: I1203 11:44:30.347460 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"b6eeb188-0ea6-46de-8cbf-d803e75df5e6","Type":"ContainerStarted","Data":"4605df729b344cdc8aa272fc21b35ff9f030fbd19d68d7fbb10d033f9171882f"} Dec 03 11:44:30 crc kubenswrapper[5052]: I1203 11:44:30.348646 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/cinder-api-0" podUID="7f4f10f6-2b65-45b9-a77e-ce3136668e1d" containerName="cinder-api-log" containerID="cri-o://6ea169f90116f631eb8741136bc27449af38f1ace846817d95ebb65f5ab46b30" gracePeriod=30 Dec 03 11:44:30 crc kubenswrapper[5052]: I1203 11:44:30.348724 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:44:30 crc kubenswrapper[5052]: I1203 11:44:30.348760 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/cinder-api-0" podUID="7f4f10f6-2b65-45b9-a77e-ce3136668e1d" containerName="cinder-api" containerID="cri-o://43c36870fd89ee414be871c98622b89a7cd23756349871cc418fe90efd9bd8ab" gracePeriod=30 Dec 03 11:44:30 crc kubenswrapper[5052]: I1203 11:44:30.348768 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:44:30 crc kubenswrapper[5052]: I1203 11:44:30.381755 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/ceilometer-0" podStartSLOduration=2.564101622 podStartE2EDuration="6.381738251s" podCreationTimestamp="2025-12-03 11:44:24 +0000 UTC" firstStartedPulling="2025-12-03 11:44:25.641840908 +0000 UTC m=+1456.230082672" lastFinishedPulling="2025-12-03 11:44:29.459477537 +0000 UTC m=+1460.047719301" observedRunningTime="2025-12-03 11:44:30.379888418 +0000 UTC m=+1460.968130192" watchObservedRunningTime="2025-12-03 11:44:30.381738251 +0000 UTC m=+1460.969980015" Dec 03 11:44:30 crc kubenswrapper[5052]: I1203 11:44:30.991796 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-bootstrap-fmkmc" Dec 03 11:44:31 crc kubenswrapper[5052]: I1203 11:44:31.082068 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/ee9c2229-695a-4930-89b0-39f140a3f328-config-data\") pod \"ee9c2229-695a-4930-89b0-39f140a3f328\" (UID: \"ee9c2229-695a-4930-89b0-39f140a3f328\") " Dec 03 11:44:31 crc kubenswrapper[5052]: I1203 11:44:31.082217 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-97qf8\" (UniqueName: \"kubernetes.io/projected/ee9c2229-695a-4930-89b0-39f140a3f328-kube-api-access-97qf8\") pod \"ee9c2229-695a-4930-89b0-39f140a3f328\" (UID: \"ee9c2229-695a-4930-89b0-39f140a3f328\") " Dec 03 11:44:31 crc kubenswrapper[5052]: I1203 11:44:31.082267 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ee9c2229-695a-4930-89b0-39f140a3f328-combined-ca-bundle\") pod \"ee9c2229-695a-4930-89b0-39f140a3f328\" (UID: \"ee9c2229-695a-4930-89b0-39f140a3f328\") " Dec 03 11:44:31 crc kubenswrapper[5052]: I1203 11:44:31.082328 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/ee9c2229-695a-4930-89b0-39f140a3f328-scripts\") pod \"ee9c2229-695a-4930-89b0-39f140a3f328\" (UID: \"ee9c2229-695a-4930-89b0-39f140a3f328\") " Dec 03 11:44:31 crc kubenswrapper[5052]: I1203 11:44:31.082355 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/ee9c2229-695a-4930-89b0-39f140a3f328-credential-keys\") pod \"ee9c2229-695a-4930-89b0-39f140a3f328\" (UID: \"ee9c2229-695a-4930-89b0-39f140a3f328\") " Dec 03 11:44:31 crc kubenswrapper[5052]: I1203 11:44:31.082410 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/ee9c2229-695a-4930-89b0-39f140a3f328-fernet-keys\") pod \"ee9c2229-695a-4930-89b0-39f140a3f328\" (UID: \"ee9c2229-695a-4930-89b0-39f140a3f328\") " Dec 03 11:44:31 crc kubenswrapper[5052]: I1203 11:44:31.111112 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ee9c2229-695a-4930-89b0-39f140a3f328-credential-keys" (OuterVolumeSpecName: "credential-keys") pod "ee9c2229-695a-4930-89b0-39f140a3f328" (UID: "ee9c2229-695a-4930-89b0-39f140a3f328"). InnerVolumeSpecName "credential-keys". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:44:31 crc kubenswrapper[5052]: I1203 11:44:31.114677 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ee9c2229-695a-4930-89b0-39f140a3f328-fernet-keys" (OuterVolumeSpecName: "fernet-keys") pod "ee9c2229-695a-4930-89b0-39f140a3f328" (UID: "ee9c2229-695a-4930-89b0-39f140a3f328"). InnerVolumeSpecName "fernet-keys". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:44:31 crc kubenswrapper[5052]: I1203 11:44:31.114694 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ee9c2229-695a-4930-89b0-39f140a3f328-scripts" (OuterVolumeSpecName: "scripts") pod "ee9c2229-695a-4930-89b0-39f140a3f328" (UID: "ee9c2229-695a-4930-89b0-39f140a3f328"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:44:31 crc kubenswrapper[5052]: I1203 11:44:31.114826 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ee9c2229-695a-4930-89b0-39f140a3f328-kube-api-access-97qf8" (OuterVolumeSpecName: "kube-api-access-97qf8") pod "ee9c2229-695a-4930-89b0-39f140a3f328" (UID: "ee9c2229-695a-4930-89b0-39f140a3f328"). InnerVolumeSpecName "kube-api-access-97qf8". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:44:31 crc kubenswrapper[5052]: I1203 11:44:31.124983 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ee9c2229-695a-4930-89b0-39f140a3f328-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "ee9c2229-695a-4930-89b0-39f140a3f328" (UID: "ee9c2229-695a-4930-89b0-39f140a3f328"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:44:31 crc kubenswrapper[5052]: I1203 11:44:31.164457 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ee9c2229-695a-4930-89b0-39f140a3f328-config-data" (OuterVolumeSpecName: "config-data") pod "ee9c2229-695a-4930-89b0-39f140a3f328" (UID: "ee9c2229-695a-4930-89b0-39f140a3f328"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:44:31 crc kubenswrapper[5052]: I1203 11:44:31.186070 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-97qf8\" (UniqueName: \"kubernetes.io/projected/ee9c2229-695a-4930-89b0-39f140a3f328-kube-api-access-97qf8\") on node \"crc\" DevicePath \"\"" Dec 03 11:44:31 crc kubenswrapper[5052]: I1203 11:44:31.186117 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ee9c2229-695a-4930-89b0-39f140a3f328-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:44:31 crc kubenswrapper[5052]: I1203 11:44:31.186131 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/ee9c2229-695a-4930-89b0-39f140a3f328-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:44:31 crc kubenswrapper[5052]: I1203 11:44:31.186145 5052 reconciler_common.go:293] "Volume detached for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/ee9c2229-695a-4930-89b0-39f140a3f328-credential-keys\") on node \"crc\" DevicePath \"\"" Dec 03 11:44:31 crc kubenswrapper[5052]: I1203 11:44:31.186157 5052 reconciler_common.go:293] "Volume detached for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/ee9c2229-695a-4930-89b0-39f140a3f328-fernet-keys\") on node \"crc\" DevicePath \"\"" Dec 03 11:44:31 crc kubenswrapper[5052]: I1203 11:44:31.186169 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/ee9c2229-695a-4930-89b0-39f140a3f328-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:44:31 crc kubenswrapper[5052]: I1203 11:44:31.357269 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-bootstrap-fmkmc" event={"ID":"ee9c2229-695a-4930-89b0-39f140a3f328","Type":"ContainerDied","Data":"4999b392a0945626cc73f9af4c88aab05d254ed4f9a3b4faf90f1f699d063189"} Dec 03 11:44:31 crc kubenswrapper[5052]: I1203 11:44:31.357329 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4999b392a0945626cc73f9af4c88aab05d254ed4f9a3b4faf90f1f699d063189" Dec 03 11:44:31 crc kubenswrapper[5052]: I1203 11:44:31.357421 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-bootstrap-fmkmc" Dec 03 11:44:31 crc kubenswrapper[5052]: I1203 11:44:31.364361 5052 generic.go:334] "Generic (PLEG): container finished" podID="7f4f10f6-2b65-45b9-a77e-ce3136668e1d" containerID="6ea169f90116f631eb8741136bc27449af38f1ace846817d95ebb65f5ab46b30" exitCode=143 Dec 03 11:44:31 crc kubenswrapper[5052]: I1203 11:44:31.364525 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"7f4f10f6-2b65-45b9-a77e-ce3136668e1d","Type":"ContainerDied","Data":"6ea169f90116f631eb8741136bc27449af38f1ace846817d95ebb65f5ab46b30"} Dec 03 11:44:31 crc kubenswrapper[5052]: I1203 11:44:31.364564 5052 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Dec 03 11:44:31 crc kubenswrapper[5052]: I1203 11:44:31.364588 5052 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Dec 03 11:44:31 crc kubenswrapper[5052]: I1203 11:44:31.364639 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:44:31 crc kubenswrapper[5052]: I1203 11:44:31.547766 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/keystone-779b679dc7-nvtdm"] Dec 03 11:44:31 crc kubenswrapper[5052]: E1203 11:44:31.548240 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="ee9c2229-695a-4930-89b0-39f140a3f328" containerName="keystone-bootstrap" Dec 03 11:44:31 crc kubenswrapper[5052]: I1203 11:44:31.548259 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee9c2229-695a-4930-89b0-39f140a3f328" containerName="keystone-bootstrap" Dec 03 11:44:31 crc kubenswrapper[5052]: I1203 11:44:31.548492 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="ee9c2229-695a-4930-89b0-39f140a3f328" containerName="keystone-bootstrap" Dec 03 11:44:31 crc kubenswrapper[5052]: I1203 11:44:31.549247 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-779b679dc7-nvtdm" Dec 03 11:44:31 crc kubenswrapper[5052]: I1203 11:44:31.551627 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone-scripts" Dec 03 11:44:31 crc kubenswrapper[5052]: I1203 11:44:31.551713 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone" Dec 03 11:44:31 crc kubenswrapper[5052]: I1203 11:44:31.552327 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone-keystone-dockercfg-4hzgj" Dec 03 11:44:31 crc kubenswrapper[5052]: I1203 11:44:31.552630 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-keystone-internal-svc" Dec 03 11:44:31 crc kubenswrapper[5052]: I1203 11:44:31.553032 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone-config-data" Dec 03 11:44:31 crc kubenswrapper[5052]: I1203 11:44:31.553163 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-keystone-public-svc" Dec 03 11:44:31 crc kubenswrapper[5052]: I1203 11:44:31.599789 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-779b679dc7-nvtdm"] Dec 03 11:44:31 crc kubenswrapper[5052]: I1203 11:44:31.695670 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/65be66f1-c32d-40b5-81a4-a63258364d9d-config-data\") pod \"keystone-779b679dc7-nvtdm\" (UID: \"65be66f1-c32d-40b5-81a4-a63258364d9d\") " pod="openstack-kuttl-tests/keystone-779b679dc7-nvtdm" Dec 03 11:44:31 crc kubenswrapper[5052]: I1203 11:44:31.695706 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/65be66f1-c32d-40b5-81a4-a63258364d9d-internal-tls-certs\") pod \"keystone-779b679dc7-nvtdm\" (UID: \"65be66f1-c32d-40b5-81a4-a63258364d9d\") " pod="openstack-kuttl-tests/keystone-779b679dc7-nvtdm" Dec 03 11:44:31 crc kubenswrapper[5052]: I1203 11:44:31.695739 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/65be66f1-c32d-40b5-81a4-a63258364d9d-public-tls-certs\") pod \"keystone-779b679dc7-nvtdm\" (UID: \"65be66f1-c32d-40b5-81a4-a63258364d9d\") " pod="openstack-kuttl-tests/keystone-779b679dc7-nvtdm" Dec 03 11:44:31 crc kubenswrapper[5052]: I1203 11:44:31.695772 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/65be66f1-c32d-40b5-81a4-a63258364d9d-scripts\") pod \"keystone-779b679dc7-nvtdm\" (UID: \"65be66f1-c32d-40b5-81a4-a63258364d9d\") " pod="openstack-kuttl-tests/keystone-779b679dc7-nvtdm" Dec 03 11:44:31 crc kubenswrapper[5052]: I1203 11:44:31.695805 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/65be66f1-c32d-40b5-81a4-a63258364d9d-combined-ca-bundle\") pod \"keystone-779b679dc7-nvtdm\" (UID: \"65be66f1-c32d-40b5-81a4-a63258364d9d\") " pod="openstack-kuttl-tests/keystone-779b679dc7-nvtdm" Dec 03 11:44:31 crc kubenswrapper[5052]: I1203 11:44:31.695840 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/65be66f1-c32d-40b5-81a4-a63258364d9d-fernet-keys\") pod \"keystone-779b679dc7-nvtdm\" (UID: \"65be66f1-c32d-40b5-81a4-a63258364d9d\") " pod="openstack-kuttl-tests/keystone-779b679dc7-nvtdm" Dec 03 11:44:31 crc kubenswrapper[5052]: I1203 11:44:31.695880 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/65be66f1-c32d-40b5-81a4-a63258364d9d-credential-keys\") pod \"keystone-779b679dc7-nvtdm\" (UID: \"65be66f1-c32d-40b5-81a4-a63258364d9d\") " pod="openstack-kuttl-tests/keystone-779b679dc7-nvtdm" Dec 03 11:44:31 crc kubenswrapper[5052]: I1203 11:44:31.695909 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7h9bm\" (UniqueName: \"kubernetes.io/projected/65be66f1-c32d-40b5-81a4-a63258364d9d-kube-api-access-7h9bm\") pod \"keystone-779b679dc7-nvtdm\" (UID: \"65be66f1-c32d-40b5-81a4-a63258364d9d\") " pod="openstack-kuttl-tests/keystone-779b679dc7-nvtdm" Dec 03 11:44:31 crc kubenswrapper[5052]: I1203 11:44:31.798019 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/65be66f1-c32d-40b5-81a4-a63258364d9d-credential-keys\") pod \"keystone-779b679dc7-nvtdm\" (UID: \"65be66f1-c32d-40b5-81a4-a63258364d9d\") " pod="openstack-kuttl-tests/keystone-779b679dc7-nvtdm" Dec 03 11:44:31 crc kubenswrapper[5052]: I1203 11:44:31.798414 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-7h9bm\" (UniqueName: \"kubernetes.io/projected/65be66f1-c32d-40b5-81a4-a63258364d9d-kube-api-access-7h9bm\") pod \"keystone-779b679dc7-nvtdm\" (UID: \"65be66f1-c32d-40b5-81a4-a63258364d9d\") " pod="openstack-kuttl-tests/keystone-779b679dc7-nvtdm" Dec 03 11:44:31 crc kubenswrapper[5052]: I1203 11:44:31.798551 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/65be66f1-c32d-40b5-81a4-a63258364d9d-config-data\") pod \"keystone-779b679dc7-nvtdm\" (UID: \"65be66f1-c32d-40b5-81a4-a63258364d9d\") " pod="openstack-kuttl-tests/keystone-779b679dc7-nvtdm" Dec 03 11:44:31 crc kubenswrapper[5052]: I1203 11:44:31.798582 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/65be66f1-c32d-40b5-81a4-a63258364d9d-internal-tls-certs\") pod \"keystone-779b679dc7-nvtdm\" (UID: \"65be66f1-c32d-40b5-81a4-a63258364d9d\") " pod="openstack-kuttl-tests/keystone-779b679dc7-nvtdm" Dec 03 11:44:31 crc kubenswrapper[5052]: I1203 11:44:31.798623 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/65be66f1-c32d-40b5-81a4-a63258364d9d-public-tls-certs\") pod \"keystone-779b679dc7-nvtdm\" (UID: \"65be66f1-c32d-40b5-81a4-a63258364d9d\") " pod="openstack-kuttl-tests/keystone-779b679dc7-nvtdm" Dec 03 11:44:31 crc kubenswrapper[5052]: I1203 11:44:31.798670 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/65be66f1-c32d-40b5-81a4-a63258364d9d-scripts\") pod \"keystone-779b679dc7-nvtdm\" (UID: \"65be66f1-c32d-40b5-81a4-a63258364d9d\") " pod="openstack-kuttl-tests/keystone-779b679dc7-nvtdm" Dec 03 11:44:31 crc kubenswrapper[5052]: I1203 11:44:31.798731 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/65be66f1-c32d-40b5-81a4-a63258364d9d-combined-ca-bundle\") pod \"keystone-779b679dc7-nvtdm\" (UID: \"65be66f1-c32d-40b5-81a4-a63258364d9d\") " pod="openstack-kuttl-tests/keystone-779b679dc7-nvtdm" Dec 03 11:44:31 crc kubenswrapper[5052]: I1203 11:44:31.798773 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/65be66f1-c32d-40b5-81a4-a63258364d9d-fernet-keys\") pod \"keystone-779b679dc7-nvtdm\" (UID: \"65be66f1-c32d-40b5-81a4-a63258364d9d\") " pod="openstack-kuttl-tests/keystone-779b679dc7-nvtdm" Dec 03 11:44:31 crc kubenswrapper[5052]: I1203 11:44:31.809088 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/65be66f1-c32d-40b5-81a4-a63258364d9d-public-tls-certs\") pod \"keystone-779b679dc7-nvtdm\" (UID: \"65be66f1-c32d-40b5-81a4-a63258364d9d\") " pod="openstack-kuttl-tests/keystone-779b679dc7-nvtdm" Dec 03 11:44:31 crc kubenswrapper[5052]: I1203 11:44:31.810020 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/65be66f1-c32d-40b5-81a4-a63258364d9d-credential-keys\") pod \"keystone-779b679dc7-nvtdm\" (UID: \"65be66f1-c32d-40b5-81a4-a63258364d9d\") " pod="openstack-kuttl-tests/keystone-779b679dc7-nvtdm" Dec 03 11:44:31 crc kubenswrapper[5052]: I1203 11:44:31.816063 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/65be66f1-c32d-40b5-81a4-a63258364d9d-internal-tls-certs\") pod \"keystone-779b679dc7-nvtdm\" (UID: \"65be66f1-c32d-40b5-81a4-a63258364d9d\") " pod="openstack-kuttl-tests/keystone-779b679dc7-nvtdm" Dec 03 11:44:31 crc kubenswrapper[5052]: I1203 11:44:31.816570 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/65be66f1-c32d-40b5-81a4-a63258364d9d-scripts\") pod \"keystone-779b679dc7-nvtdm\" (UID: \"65be66f1-c32d-40b5-81a4-a63258364d9d\") " pod="openstack-kuttl-tests/keystone-779b679dc7-nvtdm" Dec 03 11:44:31 crc kubenswrapper[5052]: I1203 11:44:31.822077 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-7h9bm\" (UniqueName: \"kubernetes.io/projected/65be66f1-c32d-40b5-81a4-a63258364d9d-kube-api-access-7h9bm\") pod \"keystone-779b679dc7-nvtdm\" (UID: \"65be66f1-c32d-40b5-81a4-a63258364d9d\") " pod="openstack-kuttl-tests/keystone-779b679dc7-nvtdm" Dec 03 11:44:31 crc kubenswrapper[5052]: I1203 11:44:31.831966 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/65be66f1-c32d-40b5-81a4-a63258364d9d-config-data\") pod \"keystone-779b679dc7-nvtdm\" (UID: \"65be66f1-c32d-40b5-81a4-a63258364d9d\") " pod="openstack-kuttl-tests/keystone-779b679dc7-nvtdm" Dec 03 11:44:31 crc kubenswrapper[5052]: I1203 11:44:31.837691 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/65be66f1-c32d-40b5-81a4-a63258364d9d-combined-ca-bundle\") pod \"keystone-779b679dc7-nvtdm\" (UID: \"65be66f1-c32d-40b5-81a4-a63258364d9d\") " pod="openstack-kuttl-tests/keystone-779b679dc7-nvtdm" Dec 03 11:44:31 crc kubenswrapper[5052]: I1203 11:44:31.839281 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/65be66f1-c32d-40b5-81a4-a63258364d9d-fernet-keys\") pod \"keystone-779b679dc7-nvtdm\" (UID: \"65be66f1-c32d-40b5-81a4-a63258364d9d\") " pod="openstack-kuttl-tests/keystone-779b679dc7-nvtdm" Dec 03 11:44:31 crc kubenswrapper[5052]: I1203 11:44:31.930215 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-779b679dc7-nvtdm" Dec 03 11:44:32 crc kubenswrapper[5052]: I1203 11:44:32.396783 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:44:32 crc kubenswrapper[5052]: I1203 11:44:32.410019 5052 generic.go:334] "Generic (PLEG): container finished" podID="7f4f10f6-2b65-45b9-a77e-ce3136668e1d" containerID="43c36870fd89ee414be871c98622b89a7cd23756349871cc418fe90efd9bd8ab" exitCode=0 Dec 03 11:44:32 crc kubenswrapper[5052]: I1203 11:44:32.411355 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"7f4f10f6-2b65-45b9-a77e-ce3136668e1d","Type":"ContainerDied","Data":"43c36870fd89ee414be871c98622b89a7cd23756349871cc418fe90efd9bd8ab"} Dec 03 11:44:32 crc kubenswrapper[5052]: I1203 11:44:32.411477 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"7f4f10f6-2b65-45b9-a77e-ce3136668e1d","Type":"ContainerDied","Data":"be5706533c73fa8a32f4fc8036cdaee26e5d8433ac23e6e1805423b20fca460c"} Dec 03 11:44:32 crc kubenswrapper[5052]: I1203 11:44:32.411578 5052 scope.go:117] "RemoveContainer" containerID="43c36870fd89ee414be871c98622b89a7cd23756349871cc418fe90efd9bd8ab" Dec 03 11:44:32 crc kubenswrapper[5052]: I1203 11:44:32.487738 5052 scope.go:117] "RemoveContainer" containerID="6ea169f90116f631eb8741136bc27449af38f1ace846817d95ebb65f5ab46b30" Dec 03 11:44:32 crc kubenswrapper[5052]: I1203 11:44:32.512538 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/7f4f10f6-2b65-45b9-a77e-ce3136668e1d-etc-machine-id\") pod \"7f4f10f6-2b65-45b9-a77e-ce3136668e1d\" (UID: \"7f4f10f6-2b65-45b9-a77e-ce3136668e1d\") " Dec 03 11:44:32 crc kubenswrapper[5052]: I1203 11:44:32.512646 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/7f4f10f6-2b65-45b9-a77e-ce3136668e1d-logs\") pod \"7f4f10f6-2b65-45b9-a77e-ce3136668e1d\" (UID: \"7f4f10f6-2b65-45b9-a77e-ce3136668e1d\") " Dec 03 11:44:32 crc kubenswrapper[5052]: I1203 11:44:32.512714 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/7f4f10f6-2b65-45b9-a77e-ce3136668e1d-config-data-custom\") pod \"7f4f10f6-2b65-45b9-a77e-ce3136668e1d\" (UID: \"7f4f10f6-2b65-45b9-a77e-ce3136668e1d\") " Dec 03 11:44:32 crc kubenswrapper[5052]: I1203 11:44:32.512824 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/7f4f10f6-2b65-45b9-a77e-ce3136668e1d-config-data\") pod \"7f4f10f6-2b65-45b9-a77e-ce3136668e1d\" (UID: \"7f4f10f6-2b65-45b9-a77e-ce3136668e1d\") " Dec 03 11:44:32 crc kubenswrapper[5052]: I1203 11:44:32.512877 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-45pfr\" (UniqueName: \"kubernetes.io/projected/7f4f10f6-2b65-45b9-a77e-ce3136668e1d-kube-api-access-45pfr\") pod \"7f4f10f6-2b65-45b9-a77e-ce3136668e1d\" (UID: \"7f4f10f6-2b65-45b9-a77e-ce3136668e1d\") " Dec 03 11:44:32 crc kubenswrapper[5052]: I1203 11:44:32.512910 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/7f4f10f6-2b65-45b9-a77e-ce3136668e1d-scripts\") pod \"7f4f10f6-2b65-45b9-a77e-ce3136668e1d\" (UID: \"7f4f10f6-2b65-45b9-a77e-ce3136668e1d\") " Dec 03 11:44:32 crc kubenswrapper[5052]: I1203 11:44:32.513025 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/7f4f10f6-2b65-45b9-a77e-ce3136668e1d-combined-ca-bundle\") pod \"7f4f10f6-2b65-45b9-a77e-ce3136668e1d\" (UID: \"7f4f10f6-2b65-45b9-a77e-ce3136668e1d\") " Dec 03 11:44:32 crc kubenswrapper[5052]: I1203 11:44:32.513384 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/7f4f10f6-2b65-45b9-a77e-ce3136668e1d-etc-machine-id" (OuterVolumeSpecName: "etc-machine-id") pod "7f4f10f6-2b65-45b9-a77e-ce3136668e1d" (UID: "7f4f10f6-2b65-45b9-a77e-ce3136668e1d"). InnerVolumeSpecName "etc-machine-id". PluginName "kubernetes.io/host-path", VolumeGidValue "" Dec 03 11:44:32 crc kubenswrapper[5052]: I1203 11:44:32.514659 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f4f10f6-2b65-45b9-a77e-ce3136668e1d-logs" (OuterVolumeSpecName: "logs") pod "7f4f10f6-2b65-45b9-a77e-ce3136668e1d" (UID: "7f4f10f6-2b65-45b9-a77e-ce3136668e1d"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:44:32 crc kubenswrapper[5052]: I1203 11:44:32.516064 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/7f4f10f6-2b65-45b9-a77e-ce3136668e1d-logs\") on node \"crc\" DevicePath \"\"" Dec 03 11:44:32 crc kubenswrapper[5052]: I1203 11:44:32.516400 5052 reconciler_common.go:293] "Volume detached for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/7f4f10f6-2b65-45b9-a77e-ce3136668e1d-etc-machine-id\") on node \"crc\" DevicePath \"\"" Dec 03 11:44:32 crc kubenswrapper[5052]: I1203 11:44:32.525886 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7f4f10f6-2b65-45b9-a77e-ce3136668e1d-config-data-custom" (OuterVolumeSpecName: "config-data-custom") pod "7f4f10f6-2b65-45b9-a77e-ce3136668e1d" (UID: "7f4f10f6-2b65-45b9-a77e-ce3136668e1d"). InnerVolumeSpecName "config-data-custom". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:44:32 crc kubenswrapper[5052]: I1203 11:44:32.528723 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7f4f10f6-2b65-45b9-a77e-ce3136668e1d-scripts" (OuterVolumeSpecName: "scripts") pod "7f4f10f6-2b65-45b9-a77e-ce3136668e1d" (UID: "7f4f10f6-2b65-45b9-a77e-ce3136668e1d"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:44:32 crc kubenswrapper[5052]: I1203 11:44:32.538387 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7f4f10f6-2b65-45b9-a77e-ce3136668e1d-kube-api-access-45pfr" (OuterVolumeSpecName: "kube-api-access-45pfr") pod "7f4f10f6-2b65-45b9-a77e-ce3136668e1d" (UID: "7f4f10f6-2b65-45b9-a77e-ce3136668e1d"). InnerVolumeSpecName "kube-api-access-45pfr". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:44:32 crc kubenswrapper[5052]: I1203 11:44:32.540803 5052 scope.go:117] "RemoveContainer" containerID="43c36870fd89ee414be871c98622b89a7cd23756349871cc418fe90efd9bd8ab" Dec 03 11:44:32 crc kubenswrapper[5052]: E1203 11:44:32.541555 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"43c36870fd89ee414be871c98622b89a7cd23756349871cc418fe90efd9bd8ab\": container with ID starting with 43c36870fd89ee414be871c98622b89a7cd23756349871cc418fe90efd9bd8ab not found: ID does not exist" containerID="43c36870fd89ee414be871c98622b89a7cd23756349871cc418fe90efd9bd8ab" Dec 03 11:44:32 crc kubenswrapper[5052]: I1203 11:44:32.541629 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"43c36870fd89ee414be871c98622b89a7cd23756349871cc418fe90efd9bd8ab"} err="failed to get container status \"43c36870fd89ee414be871c98622b89a7cd23756349871cc418fe90efd9bd8ab\": rpc error: code = NotFound desc = could not find container \"43c36870fd89ee414be871c98622b89a7cd23756349871cc418fe90efd9bd8ab\": container with ID starting with 43c36870fd89ee414be871c98622b89a7cd23756349871cc418fe90efd9bd8ab not found: ID does not exist" Dec 03 11:44:32 crc kubenswrapper[5052]: I1203 11:44:32.541667 5052 scope.go:117] "RemoveContainer" containerID="6ea169f90116f631eb8741136bc27449af38f1ace846817d95ebb65f5ab46b30" Dec 03 11:44:32 crc kubenswrapper[5052]: E1203 11:44:32.542172 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6ea169f90116f631eb8741136bc27449af38f1ace846817d95ebb65f5ab46b30\": container with ID starting with 6ea169f90116f631eb8741136bc27449af38f1ace846817d95ebb65f5ab46b30 not found: ID does not exist" containerID="6ea169f90116f631eb8741136bc27449af38f1ace846817d95ebb65f5ab46b30" Dec 03 11:44:32 crc kubenswrapper[5052]: I1203 11:44:32.542224 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6ea169f90116f631eb8741136bc27449af38f1ace846817d95ebb65f5ab46b30"} err="failed to get container status \"6ea169f90116f631eb8741136bc27449af38f1ace846817d95ebb65f5ab46b30\": rpc error: code = NotFound desc = could not find container \"6ea169f90116f631eb8741136bc27449af38f1ace846817d95ebb65f5ab46b30\": container with ID starting with 6ea169f90116f631eb8741136bc27449af38f1ace846817d95ebb65f5ab46b30 not found: ID does not exist" Dec 03 11:44:32 crc kubenswrapper[5052]: I1203 11:44:32.583932 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7f4f10f6-2b65-45b9-a77e-ce3136668e1d-config-data" (OuterVolumeSpecName: "config-data") pod "7f4f10f6-2b65-45b9-a77e-ce3136668e1d" (UID: "7f4f10f6-2b65-45b9-a77e-ce3136668e1d"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:44:32 crc kubenswrapper[5052]: I1203 11:44:32.597914 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7f4f10f6-2b65-45b9-a77e-ce3136668e1d-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "7f4f10f6-2b65-45b9-a77e-ce3136668e1d" (UID: "7f4f10f6-2b65-45b9-a77e-ce3136668e1d"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:44:32 crc kubenswrapper[5052]: I1203 11:44:32.625103 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/7f4f10f6-2b65-45b9-a77e-ce3136668e1d-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:44:32 crc kubenswrapper[5052]: I1203 11:44:32.625204 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-45pfr\" (UniqueName: \"kubernetes.io/projected/7f4f10f6-2b65-45b9-a77e-ce3136668e1d-kube-api-access-45pfr\") on node \"crc\" DevicePath \"\"" Dec 03 11:44:32 crc kubenswrapper[5052]: I1203 11:44:32.625264 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/7f4f10f6-2b65-45b9-a77e-ce3136668e1d-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:44:32 crc kubenswrapper[5052]: I1203 11:44:32.625321 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/7f4f10f6-2b65-45b9-a77e-ce3136668e1d-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:44:32 crc kubenswrapper[5052]: I1203 11:44:32.625376 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/7f4f10f6-2b65-45b9-a77e-ce3136668e1d-config-data-custom\") on node \"crc\" DevicePath \"\"" Dec 03 11:44:32 crc kubenswrapper[5052]: I1203 11:44:32.630566 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:44:32 crc kubenswrapper[5052]: I1203 11:44:32.631003 5052 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Dec 03 11:44:32 crc kubenswrapper[5052]: I1203 11:44:32.632529 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:44:32 crc kubenswrapper[5052]: I1203 11:44:32.632754 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-779b679dc7-nvtdm"] Dec 03 11:44:32 crc kubenswrapper[5052]: W1203 11:44:32.633775 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod65be66f1_c32d_40b5_81a4_a63258364d9d.slice/crio-fa978c7c93e89e6852b87846dfa444775998fa7b28a1e32a276fd4189faabb8b WatchSource:0}: Error finding container fa978c7c93e89e6852b87846dfa444775998fa7b28a1e32a276fd4189faabb8b: Status 404 returned error can't find the container with id fa978c7c93e89e6852b87846dfa444775998fa7b28a1e32a276fd4189faabb8b Dec 03 11:44:33 crc kubenswrapper[5052]: I1203 11:44:33.197269 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:44:33 crc kubenswrapper[5052]: I1203 11:44:33.197846 5052 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Dec 03 11:44:33 crc kubenswrapper[5052]: I1203 11:44:33.383544 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/neutron-549dfc49dd-qtk5n"] Dec 03 11:44:33 crc kubenswrapper[5052]: E1203 11:44:33.384004 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="7f4f10f6-2b65-45b9-a77e-ce3136668e1d" containerName="cinder-api-log" Dec 03 11:44:33 crc kubenswrapper[5052]: I1203 11:44:33.384030 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f4f10f6-2b65-45b9-a77e-ce3136668e1d" containerName="cinder-api-log" Dec 03 11:44:33 crc kubenswrapper[5052]: E1203 11:44:33.384042 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="7f4f10f6-2b65-45b9-a77e-ce3136668e1d" containerName="cinder-api" Dec 03 11:44:33 crc kubenswrapper[5052]: I1203 11:44:33.384051 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f4f10f6-2b65-45b9-a77e-ce3136668e1d" containerName="cinder-api" Dec 03 11:44:33 crc kubenswrapper[5052]: I1203 11:44:33.384296 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="7f4f10f6-2b65-45b9-a77e-ce3136668e1d" containerName="cinder-api-log" Dec 03 11:44:33 crc kubenswrapper[5052]: I1203 11:44:33.384323 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="7f4f10f6-2b65-45b9-a77e-ce3136668e1d" containerName="cinder-api" Dec 03 11:44:33 crc kubenswrapper[5052]: I1203 11:44:33.386948 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-549dfc49dd-qtk5n" Dec 03 11:44:33 crc kubenswrapper[5052]: I1203 11:44:33.389436 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-neutron-public-svc" Dec 03 11:44:33 crc kubenswrapper[5052]: I1203 11:44:33.393488 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-neutron-internal-svc" Dec 03 11:44:33 crc kubenswrapper[5052]: I1203 11:44:33.403218 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-549dfc49dd-qtk5n"] Dec 03 11:44:33 crc kubenswrapper[5052]: I1203 11:44:33.426223 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:44:33 crc kubenswrapper[5052]: I1203 11:44:33.433637 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-779b679dc7-nvtdm" event={"ID":"65be66f1-c32d-40b5-81a4-a63258364d9d","Type":"ContainerStarted","Data":"2be919af42c62148fa0417519c4eea762d9886b79020bd92b5f10692d1f30f31"} Dec 03 11:44:33 crc kubenswrapper[5052]: I1203 11:44:33.433930 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/keystone-779b679dc7-nvtdm" Dec 03 11:44:33 crc kubenswrapper[5052]: I1203 11:44:33.434002 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-779b679dc7-nvtdm" event={"ID":"65be66f1-c32d-40b5-81a4-a63258364d9d","Type":"ContainerStarted","Data":"fa978c7c93e89e6852b87846dfa444775998fa7b28a1e32a276fd4189faabb8b"} Dec 03 11:44:33 crc kubenswrapper[5052]: I1203 11:44:33.465598 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/keystone-779b679dc7-nvtdm" podStartSLOduration=2.465576829 podStartE2EDuration="2.465576829s" podCreationTimestamp="2025-12-03 11:44:31 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:44:33.463852629 +0000 UTC m=+1464.052094403" watchObservedRunningTime="2025-12-03 11:44:33.465576829 +0000 UTC m=+1464.053818593" Dec 03 11:44:33 crc kubenswrapper[5052]: I1203 11:44:33.519775 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 11:44:33 crc kubenswrapper[5052]: I1203 11:44:33.538027 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 11:44:33 crc kubenswrapper[5052]: I1203 11:44:33.552901 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/a4fa0509-8f85-462f-b7c3-4384346252bc-public-tls-certs\") pod \"neutron-549dfc49dd-qtk5n\" (UID: \"a4fa0509-8f85-462f-b7c3-4384346252bc\") " pod="openstack-kuttl-tests/neutron-549dfc49dd-qtk5n" Dec 03 11:44:33 crc kubenswrapper[5052]: I1203 11:44:33.553002 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/a4fa0509-8f85-462f-b7c3-4384346252bc-ovndb-tls-certs\") pod \"neutron-549dfc49dd-qtk5n\" (UID: \"a4fa0509-8f85-462f-b7c3-4384346252bc\") " pod="openstack-kuttl-tests/neutron-549dfc49dd-qtk5n" Dec 03 11:44:33 crc kubenswrapper[5052]: I1203 11:44:33.553021 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a4fa0509-8f85-462f-b7c3-4384346252bc-combined-ca-bundle\") pod \"neutron-549dfc49dd-qtk5n\" (UID: \"a4fa0509-8f85-462f-b7c3-4384346252bc\") " pod="openstack-kuttl-tests/neutron-549dfc49dd-qtk5n" Dec 03 11:44:33 crc kubenswrapper[5052]: I1203 11:44:33.553092 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/a4fa0509-8f85-462f-b7c3-4384346252bc-config\") pod \"neutron-549dfc49dd-qtk5n\" (UID: \"a4fa0509-8f85-462f-b7c3-4384346252bc\") " pod="openstack-kuttl-tests/neutron-549dfc49dd-qtk5n" Dec 03 11:44:33 crc kubenswrapper[5052]: I1203 11:44:33.553171 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/a4fa0509-8f85-462f-b7c3-4384346252bc-httpd-config\") pod \"neutron-549dfc49dd-qtk5n\" (UID: \"a4fa0509-8f85-462f-b7c3-4384346252bc\") " pod="openstack-kuttl-tests/neutron-549dfc49dd-qtk5n" Dec 03 11:44:33 crc kubenswrapper[5052]: I1203 11:44:33.553203 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/a4fa0509-8f85-462f-b7c3-4384346252bc-internal-tls-certs\") pod \"neutron-549dfc49dd-qtk5n\" (UID: \"a4fa0509-8f85-462f-b7c3-4384346252bc\") " pod="openstack-kuttl-tests/neutron-549dfc49dd-qtk5n" Dec 03 11:44:33 crc kubenswrapper[5052]: I1203 11:44:33.553284 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4zslh\" (UniqueName: \"kubernetes.io/projected/a4fa0509-8f85-462f-b7c3-4384346252bc-kube-api-access-4zslh\") pod \"neutron-549dfc49dd-qtk5n\" (UID: \"a4fa0509-8f85-462f-b7c3-4384346252bc\") " pod="openstack-kuttl-tests/neutron-549dfc49dd-qtk5n" Dec 03 11:44:33 crc kubenswrapper[5052]: I1203 11:44:33.554372 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 11:44:33 crc kubenswrapper[5052]: I1203 11:44:33.558871 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 11:44:33 crc kubenswrapper[5052]: I1203 11:44:33.558985 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:44:33 crc kubenswrapper[5052]: I1203 11:44:33.561777 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-cinder-internal-svc" Dec 03 11:44:33 crc kubenswrapper[5052]: I1203 11:44:33.563214 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-cinder-public-svc" Dec 03 11:44:33 crc kubenswrapper[5052]: I1203 11:44:33.563371 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cinder-api-config-data" Dec 03 11:44:33 crc kubenswrapper[5052]: I1203 11:44:33.655333 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/a4fa0509-8f85-462f-b7c3-4384346252bc-internal-tls-certs\") pod \"neutron-549dfc49dd-qtk5n\" (UID: \"a4fa0509-8f85-462f-b7c3-4384346252bc\") " pod="openstack-kuttl-tests/neutron-549dfc49dd-qtk5n" Dec 03 11:44:33 crc kubenswrapper[5052]: I1203 11:44:33.655436 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-4zslh\" (UniqueName: \"kubernetes.io/projected/a4fa0509-8f85-462f-b7c3-4384346252bc-kube-api-access-4zslh\") pod \"neutron-549dfc49dd-qtk5n\" (UID: \"a4fa0509-8f85-462f-b7c3-4384346252bc\") " pod="openstack-kuttl-tests/neutron-549dfc49dd-qtk5n" Dec 03 11:44:33 crc kubenswrapper[5052]: I1203 11:44:33.655481 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/00b12739-1437-4af3-ab6d-ad830b67ad2e-internal-tls-certs\") pod \"cinder-api-0\" (UID: \"00b12739-1437-4af3-ab6d-ad830b67ad2e\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:44:33 crc kubenswrapper[5052]: I1203 11:44:33.655545 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/00b12739-1437-4af3-ab6d-ad830b67ad2e-config-data\") pod \"cinder-api-0\" (UID: \"00b12739-1437-4af3-ab6d-ad830b67ad2e\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:44:33 crc kubenswrapper[5052]: I1203 11:44:33.655571 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/00b12739-1437-4af3-ab6d-ad830b67ad2e-public-tls-certs\") pod \"cinder-api-0\" (UID: \"00b12739-1437-4af3-ab6d-ad830b67ad2e\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:44:33 crc kubenswrapper[5052]: I1203 11:44:33.655608 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/00b12739-1437-4af3-ab6d-ad830b67ad2e-logs\") pod \"cinder-api-0\" (UID: \"00b12739-1437-4af3-ab6d-ad830b67ad2e\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:44:33 crc kubenswrapper[5052]: I1203 11:44:33.655637 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/a4fa0509-8f85-462f-b7c3-4384346252bc-public-tls-certs\") pod \"neutron-549dfc49dd-qtk5n\" (UID: \"a4fa0509-8f85-462f-b7c3-4384346252bc\") " pod="openstack-kuttl-tests/neutron-549dfc49dd-qtk5n" Dec 03 11:44:33 crc kubenswrapper[5052]: I1203 11:44:33.655664 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/00b12739-1437-4af3-ab6d-ad830b67ad2e-config-data-custom\") pod \"cinder-api-0\" (UID: \"00b12739-1437-4af3-ab6d-ad830b67ad2e\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:44:33 crc kubenswrapper[5052]: I1203 11:44:33.655691 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8d24s\" (UniqueName: \"kubernetes.io/projected/00b12739-1437-4af3-ab6d-ad830b67ad2e-kube-api-access-8d24s\") pod \"cinder-api-0\" (UID: \"00b12739-1437-4af3-ab6d-ad830b67ad2e\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:44:33 crc kubenswrapper[5052]: I1203 11:44:33.655721 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/a4fa0509-8f85-462f-b7c3-4384346252bc-ovndb-tls-certs\") pod \"neutron-549dfc49dd-qtk5n\" (UID: \"a4fa0509-8f85-462f-b7c3-4384346252bc\") " pod="openstack-kuttl-tests/neutron-549dfc49dd-qtk5n" Dec 03 11:44:33 crc kubenswrapper[5052]: I1203 11:44:33.655743 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a4fa0509-8f85-462f-b7c3-4384346252bc-combined-ca-bundle\") pod \"neutron-549dfc49dd-qtk5n\" (UID: \"a4fa0509-8f85-462f-b7c3-4384346252bc\") " pod="openstack-kuttl-tests/neutron-549dfc49dd-qtk5n" Dec 03 11:44:33 crc kubenswrapper[5052]: I1203 11:44:33.655780 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/a4fa0509-8f85-462f-b7c3-4384346252bc-config\") pod \"neutron-549dfc49dd-qtk5n\" (UID: \"a4fa0509-8f85-462f-b7c3-4384346252bc\") " pod="openstack-kuttl-tests/neutron-549dfc49dd-qtk5n" Dec 03 11:44:33 crc kubenswrapper[5052]: I1203 11:44:33.655803 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/00b12739-1437-4af3-ab6d-ad830b67ad2e-etc-machine-id\") pod \"cinder-api-0\" (UID: \"00b12739-1437-4af3-ab6d-ad830b67ad2e\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:44:33 crc kubenswrapper[5052]: I1203 11:44:33.655853 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/00b12739-1437-4af3-ab6d-ad830b67ad2e-combined-ca-bundle\") pod \"cinder-api-0\" (UID: \"00b12739-1437-4af3-ab6d-ad830b67ad2e\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:44:33 crc kubenswrapper[5052]: I1203 11:44:33.655873 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/00b12739-1437-4af3-ab6d-ad830b67ad2e-scripts\") pod \"cinder-api-0\" (UID: \"00b12739-1437-4af3-ab6d-ad830b67ad2e\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:44:33 crc kubenswrapper[5052]: I1203 11:44:33.655902 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/a4fa0509-8f85-462f-b7c3-4384346252bc-httpd-config\") pod \"neutron-549dfc49dd-qtk5n\" (UID: \"a4fa0509-8f85-462f-b7c3-4384346252bc\") " pod="openstack-kuttl-tests/neutron-549dfc49dd-qtk5n" Dec 03 11:44:33 crc kubenswrapper[5052]: I1203 11:44:33.663354 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/a4fa0509-8f85-462f-b7c3-4384346252bc-httpd-config\") pod \"neutron-549dfc49dd-qtk5n\" (UID: \"a4fa0509-8f85-462f-b7c3-4384346252bc\") " pod="openstack-kuttl-tests/neutron-549dfc49dd-qtk5n" Dec 03 11:44:33 crc kubenswrapper[5052]: I1203 11:44:33.664192 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/a4fa0509-8f85-462f-b7c3-4384346252bc-ovndb-tls-certs\") pod \"neutron-549dfc49dd-qtk5n\" (UID: \"a4fa0509-8f85-462f-b7c3-4384346252bc\") " pod="openstack-kuttl-tests/neutron-549dfc49dd-qtk5n" Dec 03 11:44:33 crc kubenswrapper[5052]: I1203 11:44:33.671717 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/a4fa0509-8f85-462f-b7c3-4384346252bc-internal-tls-certs\") pod \"neutron-549dfc49dd-qtk5n\" (UID: \"a4fa0509-8f85-462f-b7c3-4384346252bc\") " pod="openstack-kuttl-tests/neutron-549dfc49dd-qtk5n" Dec 03 11:44:33 crc kubenswrapper[5052]: I1203 11:44:33.672761 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a4fa0509-8f85-462f-b7c3-4384346252bc-combined-ca-bundle\") pod \"neutron-549dfc49dd-qtk5n\" (UID: \"a4fa0509-8f85-462f-b7c3-4384346252bc\") " pod="openstack-kuttl-tests/neutron-549dfc49dd-qtk5n" Dec 03 11:44:33 crc kubenswrapper[5052]: I1203 11:44:33.672938 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-4zslh\" (UniqueName: \"kubernetes.io/projected/a4fa0509-8f85-462f-b7c3-4384346252bc-kube-api-access-4zslh\") pod \"neutron-549dfc49dd-qtk5n\" (UID: \"a4fa0509-8f85-462f-b7c3-4384346252bc\") " pod="openstack-kuttl-tests/neutron-549dfc49dd-qtk5n" Dec 03 11:44:33 crc kubenswrapper[5052]: I1203 11:44:33.673355 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/a4fa0509-8f85-462f-b7c3-4384346252bc-config\") pod \"neutron-549dfc49dd-qtk5n\" (UID: \"a4fa0509-8f85-462f-b7c3-4384346252bc\") " pod="openstack-kuttl-tests/neutron-549dfc49dd-qtk5n" Dec 03 11:44:33 crc kubenswrapper[5052]: I1203 11:44:33.674035 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/a4fa0509-8f85-462f-b7c3-4384346252bc-public-tls-certs\") pod \"neutron-549dfc49dd-qtk5n\" (UID: \"a4fa0509-8f85-462f-b7c3-4384346252bc\") " pod="openstack-kuttl-tests/neutron-549dfc49dd-qtk5n" Dec 03 11:44:33 crc kubenswrapper[5052]: I1203 11:44:33.683268 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7f4f10f6-2b65-45b9-a77e-ce3136668e1d" path="/var/lib/kubelet/pods/7f4f10f6-2b65-45b9-a77e-ce3136668e1d/volumes" Dec 03 11:44:33 crc kubenswrapper[5052]: I1203 11:44:33.709803 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-549dfc49dd-qtk5n" Dec 03 11:44:33 crc kubenswrapper[5052]: I1203 11:44:33.757719 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/00b12739-1437-4af3-ab6d-ad830b67ad2e-etc-machine-id\") pod \"cinder-api-0\" (UID: \"00b12739-1437-4af3-ab6d-ad830b67ad2e\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:44:33 crc kubenswrapper[5052]: I1203 11:44:33.757795 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/00b12739-1437-4af3-ab6d-ad830b67ad2e-combined-ca-bundle\") pod \"cinder-api-0\" (UID: \"00b12739-1437-4af3-ab6d-ad830b67ad2e\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:44:33 crc kubenswrapper[5052]: I1203 11:44:33.757818 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/00b12739-1437-4af3-ab6d-ad830b67ad2e-scripts\") pod \"cinder-api-0\" (UID: \"00b12739-1437-4af3-ab6d-ad830b67ad2e\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:44:33 crc kubenswrapper[5052]: I1203 11:44:33.757929 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/00b12739-1437-4af3-ab6d-ad830b67ad2e-internal-tls-certs\") pod \"cinder-api-0\" (UID: \"00b12739-1437-4af3-ab6d-ad830b67ad2e\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:44:33 crc kubenswrapper[5052]: I1203 11:44:33.757961 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/00b12739-1437-4af3-ab6d-ad830b67ad2e-config-data\") pod \"cinder-api-0\" (UID: \"00b12739-1437-4af3-ab6d-ad830b67ad2e\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:44:33 crc kubenswrapper[5052]: I1203 11:44:33.757997 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/00b12739-1437-4af3-ab6d-ad830b67ad2e-public-tls-certs\") pod \"cinder-api-0\" (UID: \"00b12739-1437-4af3-ab6d-ad830b67ad2e\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:44:33 crc kubenswrapper[5052]: I1203 11:44:33.758054 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/00b12739-1437-4af3-ab6d-ad830b67ad2e-logs\") pod \"cinder-api-0\" (UID: \"00b12739-1437-4af3-ab6d-ad830b67ad2e\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:44:33 crc kubenswrapper[5052]: I1203 11:44:33.758093 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/00b12739-1437-4af3-ab6d-ad830b67ad2e-config-data-custom\") pod \"cinder-api-0\" (UID: \"00b12739-1437-4af3-ab6d-ad830b67ad2e\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:44:33 crc kubenswrapper[5052]: I1203 11:44:33.758132 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-8d24s\" (UniqueName: \"kubernetes.io/projected/00b12739-1437-4af3-ab6d-ad830b67ad2e-kube-api-access-8d24s\") pod \"cinder-api-0\" (UID: \"00b12739-1437-4af3-ab6d-ad830b67ad2e\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:44:33 crc kubenswrapper[5052]: I1203 11:44:33.759226 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/00b12739-1437-4af3-ab6d-ad830b67ad2e-etc-machine-id\") pod \"cinder-api-0\" (UID: \"00b12739-1437-4af3-ab6d-ad830b67ad2e\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:44:33 crc kubenswrapper[5052]: I1203 11:44:33.764816 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/00b12739-1437-4af3-ab6d-ad830b67ad2e-logs\") pod \"cinder-api-0\" (UID: \"00b12739-1437-4af3-ab6d-ad830b67ad2e\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:44:33 crc kubenswrapper[5052]: I1203 11:44:33.769867 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/00b12739-1437-4af3-ab6d-ad830b67ad2e-scripts\") pod \"cinder-api-0\" (UID: \"00b12739-1437-4af3-ab6d-ad830b67ad2e\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:44:33 crc kubenswrapper[5052]: I1203 11:44:33.777010 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/00b12739-1437-4af3-ab6d-ad830b67ad2e-internal-tls-certs\") pod \"cinder-api-0\" (UID: \"00b12739-1437-4af3-ab6d-ad830b67ad2e\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:44:33 crc kubenswrapper[5052]: I1203 11:44:33.777664 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/00b12739-1437-4af3-ab6d-ad830b67ad2e-config-data\") pod \"cinder-api-0\" (UID: \"00b12739-1437-4af3-ab6d-ad830b67ad2e\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:44:33 crc kubenswrapper[5052]: I1203 11:44:33.778245 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/00b12739-1437-4af3-ab6d-ad830b67ad2e-combined-ca-bundle\") pod \"cinder-api-0\" (UID: \"00b12739-1437-4af3-ab6d-ad830b67ad2e\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:44:33 crc kubenswrapper[5052]: I1203 11:44:33.782120 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/00b12739-1437-4af3-ab6d-ad830b67ad2e-public-tls-certs\") pod \"cinder-api-0\" (UID: \"00b12739-1437-4af3-ab6d-ad830b67ad2e\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:44:33 crc kubenswrapper[5052]: I1203 11:44:33.782280 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/00b12739-1437-4af3-ab6d-ad830b67ad2e-config-data-custom\") pod \"cinder-api-0\" (UID: \"00b12739-1437-4af3-ab6d-ad830b67ad2e\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:44:33 crc kubenswrapper[5052]: I1203 11:44:33.787210 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-8d24s\" (UniqueName: \"kubernetes.io/projected/00b12739-1437-4af3-ab6d-ad830b67ad2e-kube-api-access-8d24s\") pod \"cinder-api-0\" (UID: \"00b12739-1437-4af3-ab6d-ad830b67ad2e\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:44:33 crc kubenswrapper[5052]: I1203 11:44:33.806408 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:44:33 crc kubenswrapper[5052]: I1203 11:44:33.880543 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:44:33 crc kubenswrapper[5052]: I1203 11:44:33.934218 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/barbican-api-7cfb847cb8-jn85t"] Dec 03 11:44:33 crc kubenswrapper[5052]: I1203 11:44:33.945360 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-api-7cfb847cb8-jn85t" Dec 03 11:44:33 crc kubenswrapper[5052]: I1203 11:44:33.962175 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-barbican-internal-svc" Dec 03 11:44:33 crc kubenswrapper[5052]: I1203 11:44:33.962771 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-barbican-public-svc" Dec 03 11:44:33 crc kubenswrapper[5052]: I1203 11:44:33.978159 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-api-7cfb847cb8-jn85t"] Dec 03 11:44:34 crc kubenswrapper[5052]: I1203 11:44:34.066544 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/99f96963-d835-4445-84af-0976f1ba0701-config-data\") pod \"barbican-api-7cfb847cb8-jn85t\" (UID: \"99f96963-d835-4445-84af-0976f1ba0701\") " pod="openstack-kuttl-tests/barbican-api-7cfb847cb8-jn85t" Dec 03 11:44:34 crc kubenswrapper[5052]: I1203 11:44:34.066620 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gbvkd\" (UniqueName: \"kubernetes.io/projected/99f96963-d835-4445-84af-0976f1ba0701-kube-api-access-gbvkd\") pod \"barbican-api-7cfb847cb8-jn85t\" (UID: \"99f96963-d835-4445-84af-0976f1ba0701\") " pod="openstack-kuttl-tests/barbican-api-7cfb847cb8-jn85t" Dec 03 11:44:34 crc kubenswrapper[5052]: I1203 11:44:34.066675 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/99f96963-d835-4445-84af-0976f1ba0701-internal-tls-certs\") pod \"barbican-api-7cfb847cb8-jn85t\" (UID: \"99f96963-d835-4445-84af-0976f1ba0701\") " pod="openstack-kuttl-tests/barbican-api-7cfb847cb8-jn85t" Dec 03 11:44:34 crc kubenswrapper[5052]: I1203 11:44:34.066695 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/99f96963-d835-4445-84af-0976f1ba0701-combined-ca-bundle\") pod \"barbican-api-7cfb847cb8-jn85t\" (UID: \"99f96963-d835-4445-84af-0976f1ba0701\") " pod="openstack-kuttl-tests/barbican-api-7cfb847cb8-jn85t" Dec 03 11:44:34 crc kubenswrapper[5052]: I1203 11:44:34.066736 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/99f96963-d835-4445-84af-0976f1ba0701-public-tls-certs\") pod \"barbican-api-7cfb847cb8-jn85t\" (UID: \"99f96963-d835-4445-84af-0976f1ba0701\") " pod="openstack-kuttl-tests/barbican-api-7cfb847cb8-jn85t" Dec 03 11:44:34 crc kubenswrapper[5052]: I1203 11:44:34.066803 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/99f96963-d835-4445-84af-0976f1ba0701-logs\") pod \"barbican-api-7cfb847cb8-jn85t\" (UID: \"99f96963-d835-4445-84af-0976f1ba0701\") " pod="openstack-kuttl-tests/barbican-api-7cfb847cb8-jn85t" Dec 03 11:44:34 crc kubenswrapper[5052]: I1203 11:44:34.066832 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/99f96963-d835-4445-84af-0976f1ba0701-config-data-custom\") pod \"barbican-api-7cfb847cb8-jn85t\" (UID: \"99f96963-d835-4445-84af-0976f1ba0701\") " pod="openstack-kuttl-tests/barbican-api-7cfb847cb8-jn85t" Dec 03 11:44:34 crc kubenswrapper[5052]: I1203 11:44:34.168689 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/99f96963-d835-4445-84af-0976f1ba0701-config-data\") pod \"barbican-api-7cfb847cb8-jn85t\" (UID: \"99f96963-d835-4445-84af-0976f1ba0701\") " pod="openstack-kuttl-tests/barbican-api-7cfb847cb8-jn85t" Dec 03 11:44:34 crc kubenswrapper[5052]: I1203 11:44:34.168802 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-gbvkd\" (UniqueName: \"kubernetes.io/projected/99f96963-d835-4445-84af-0976f1ba0701-kube-api-access-gbvkd\") pod \"barbican-api-7cfb847cb8-jn85t\" (UID: \"99f96963-d835-4445-84af-0976f1ba0701\") " pod="openstack-kuttl-tests/barbican-api-7cfb847cb8-jn85t" Dec 03 11:44:34 crc kubenswrapper[5052]: I1203 11:44:34.168840 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/99f96963-d835-4445-84af-0976f1ba0701-internal-tls-certs\") pod \"barbican-api-7cfb847cb8-jn85t\" (UID: \"99f96963-d835-4445-84af-0976f1ba0701\") " pod="openstack-kuttl-tests/barbican-api-7cfb847cb8-jn85t" Dec 03 11:44:34 crc kubenswrapper[5052]: I1203 11:44:34.168868 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/99f96963-d835-4445-84af-0976f1ba0701-combined-ca-bundle\") pod \"barbican-api-7cfb847cb8-jn85t\" (UID: \"99f96963-d835-4445-84af-0976f1ba0701\") " pod="openstack-kuttl-tests/barbican-api-7cfb847cb8-jn85t" Dec 03 11:44:34 crc kubenswrapper[5052]: I1203 11:44:34.168923 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/99f96963-d835-4445-84af-0976f1ba0701-public-tls-certs\") pod \"barbican-api-7cfb847cb8-jn85t\" (UID: \"99f96963-d835-4445-84af-0976f1ba0701\") " pod="openstack-kuttl-tests/barbican-api-7cfb847cb8-jn85t" Dec 03 11:44:34 crc kubenswrapper[5052]: I1203 11:44:34.168984 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/99f96963-d835-4445-84af-0976f1ba0701-logs\") pod \"barbican-api-7cfb847cb8-jn85t\" (UID: \"99f96963-d835-4445-84af-0976f1ba0701\") " pod="openstack-kuttl-tests/barbican-api-7cfb847cb8-jn85t" Dec 03 11:44:34 crc kubenswrapper[5052]: I1203 11:44:34.169032 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/99f96963-d835-4445-84af-0976f1ba0701-config-data-custom\") pod \"barbican-api-7cfb847cb8-jn85t\" (UID: \"99f96963-d835-4445-84af-0976f1ba0701\") " pod="openstack-kuttl-tests/barbican-api-7cfb847cb8-jn85t" Dec 03 11:44:34 crc kubenswrapper[5052]: I1203 11:44:34.169967 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/99f96963-d835-4445-84af-0976f1ba0701-logs\") pod \"barbican-api-7cfb847cb8-jn85t\" (UID: \"99f96963-d835-4445-84af-0976f1ba0701\") " pod="openstack-kuttl-tests/barbican-api-7cfb847cb8-jn85t" Dec 03 11:44:34 crc kubenswrapper[5052]: I1203 11:44:34.190308 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/99f96963-d835-4445-84af-0976f1ba0701-internal-tls-certs\") pod \"barbican-api-7cfb847cb8-jn85t\" (UID: \"99f96963-d835-4445-84af-0976f1ba0701\") " pod="openstack-kuttl-tests/barbican-api-7cfb847cb8-jn85t" Dec 03 11:44:34 crc kubenswrapper[5052]: I1203 11:44:34.191153 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/99f96963-d835-4445-84af-0976f1ba0701-combined-ca-bundle\") pod \"barbican-api-7cfb847cb8-jn85t\" (UID: \"99f96963-d835-4445-84af-0976f1ba0701\") " pod="openstack-kuttl-tests/barbican-api-7cfb847cb8-jn85t" Dec 03 11:44:34 crc kubenswrapper[5052]: I1203 11:44:34.194194 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/99f96963-d835-4445-84af-0976f1ba0701-config-data-custom\") pod \"barbican-api-7cfb847cb8-jn85t\" (UID: \"99f96963-d835-4445-84af-0976f1ba0701\") " pod="openstack-kuttl-tests/barbican-api-7cfb847cb8-jn85t" Dec 03 11:44:34 crc kubenswrapper[5052]: I1203 11:44:34.199747 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/99f96963-d835-4445-84af-0976f1ba0701-config-data\") pod \"barbican-api-7cfb847cb8-jn85t\" (UID: \"99f96963-d835-4445-84af-0976f1ba0701\") " pod="openstack-kuttl-tests/barbican-api-7cfb847cb8-jn85t" Dec 03 11:44:34 crc kubenswrapper[5052]: I1203 11:44:34.202382 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/99f96963-d835-4445-84af-0976f1ba0701-public-tls-certs\") pod \"barbican-api-7cfb847cb8-jn85t\" (UID: \"99f96963-d835-4445-84af-0976f1ba0701\") " pod="openstack-kuttl-tests/barbican-api-7cfb847cb8-jn85t" Dec 03 11:44:34 crc kubenswrapper[5052]: I1203 11:44:34.206386 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-gbvkd\" (UniqueName: \"kubernetes.io/projected/99f96963-d835-4445-84af-0976f1ba0701-kube-api-access-gbvkd\") pod \"barbican-api-7cfb847cb8-jn85t\" (UID: \"99f96963-d835-4445-84af-0976f1ba0701\") " pod="openstack-kuttl-tests/barbican-api-7cfb847cb8-jn85t" Dec 03 11:44:34 crc kubenswrapper[5052]: I1203 11:44:34.315117 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-549dfc49dd-qtk5n"] Dec 03 11:44:34 crc kubenswrapper[5052]: I1203 11:44:34.315537 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-api-7cfb847cb8-jn85t" Dec 03 11:44:34 crc kubenswrapper[5052]: I1203 11:44:34.443463 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-549dfc49dd-qtk5n" event={"ID":"a4fa0509-8f85-462f-b7c3-4384346252bc","Type":"ContainerStarted","Data":"d721295f825c25b28d3ee5f53df5d8daf720db66d5c9eb7c09a79bb694afc720"} Dec 03 11:44:34 crc kubenswrapper[5052]: I1203 11:44:34.590400 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 11:44:34 crc kubenswrapper[5052]: W1203 11:44:34.612722 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod00b12739_1437_4af3_ab6d_ad830b67ad2e.slice/crio-e1bd7d7121261e1a387aa5383584eca243918b291d77768c5cb2f1fea3f2450c WatchSource:0}: Error finding container e1bd7d7121261e1a387aa5383584eca243918b291d77768c5cb2f1fea3f2450c: Status 404 returned error can't find the container with id e1bd7d7121261e1a387aa5383584eca243918b291d77768c5cb2f1fea3f2450c Dec 03 11:44:34 crc kubenswrapper[5052]: I1203 11:44:34.834635 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:44:34 crc kubenswrapper[5052]: I1203 11:44:34.861557 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-api-7cfb847cb8-jn85t"] Dec 03 11:44:34 crc kubenswrapper[5052]: I1203 11:44:34.904390 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 11:44:35 crc kubenswrapper[5052]: I1203 11:44:35.454882 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"00b12739-1437-4af3-ab6d-ad830b67ad2e","Type":"ContainerStarted","Data":"ef839a7336e307d7216464809bb35eccbc476e002b4d08e14e48e0259172f844"} Dec 03 11:44:35 crc kubenswrapper[5052]: I1203 11:44:35.455146 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"00b12739-1437-4af3-ab6d-ad830b67ad2e","Type":"ContainerStarted","Data":"e1bd7d7121261e1a387aa5383584eca243918b291d77768c5cb2f1fea3f2450c"} Dec 03 11:44:35 crc kubenswrapper[5052]: I1203 11:44:35.460333 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-549dfc49dd-qtk5n" event={"ID":"a4fa0509-8f85-462f-b7c3-4384346252bc","Type":"ContainerStarted","Data":"9ca8e5c7ceff313c395a8a6c7655cdc02bfef848fc00d1d1281da272c3a6f083"} Dec 03 11:44:35 crc kubenswrapper[5052]: I1203 11:44:35.460383 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-549dfc49dd-qtk5n" event={"ID":"a4fa0509-8f85-462f-b7c3-4384346252bc","Type":"ContainerStarted","Data":"a10887f92ee55d6e2db028846bedc75386b0fc5b0814cc18643f424e258d8b3c"} Dec 03 11:44:35 crc kubenswrapper[5052]: I1203 11:44:35.460421 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/neutron-549dfc49dd-qtk5n" Dec 03 11:44:35 crc kubenswrapper[5052]: I1203 11:44:35.463609 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/cinder-scheduler-0" podUID="bf425311-41fb-48cf-b34a-4cc4853c68b3" containerName="cinder-scheduler" containerID="cri-o://c17a1df51765ca364dfd03ed31414bd7d7177c8683e926b08de663253d1a4f0f" gracePeriod=30 Dec 03 11:44:35 crc kubenswrapper[5052]: I1203 11:44:35.464305 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-7cfb847cb8-jn85t" event={"ID":"99f96963-d835-4445-84af-0976f1ba0701","Type":"ContainerStarted","Data":"95b8a0e92f2ae27cf80dcce2b2af098739e919963ebae7c22edd8d9c46c89ddc"} Dec 03 11:44:35 crc kubenswrapper[5052]: I1203 11:44:35.464334 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-7cfb847cb8-jn85t" event={"ID":"99f96963-d835-4445-84af-0976f1ba0701","Type":"ContainerStarted","Data":"5592513a2c756f332087769178ec0527d69dee3863e8ebc7da5e5d5ac9fefa4f"} Dec 03 11:44:35 crc kubenswrapper[5052]: I1203 11:44:35.464346 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-7cfb847cb8-jn85t" event={"ID":"99f96963-d835-4445-84af-0976f1ba0701","Type":"ContainerStarted","Data":"c82775e0c56adde0c20cd2407dc86e7083b4db213f4d1d7df2de7a43f3a5bcb1"} Dec 03 11:44:35 crc kubenswrapper[5052]: I1203 11:44:35.464360 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/barbican-api-7cfb847cb8-jn85t" Dec 03 11:44:35 crc kubenswrapper[5052]: I1203 11:44:35.464403 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/cinder-scheduler-0" podUID="bf425311-41fb-48cf-b34a-4cc4853c68b3" containerName="probe" containerID="cri-o://bb18e1ff35492d0cbb8cd1813c33935f9d2b132b5086ef0adb508337d0236453" gracePeriod=30 Dec 03 11:44:35 crc kubenswrapper[5052]: I1203 11:44:35.464460 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/barbican-api-7cfb847cb8-jn85t" Dec 03 11:44:35 crc kubenswrapper[5052]: I1203 11:44:35.488204 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/neutron-549dfc49dd-qtk5n" podStartSLOduration=2.488186785 podStartE2EDuration="2.488186785s" podCreationTimestamp="2025-12-03 11:44:33 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:44:35.484036145 +0000 UTC m=+1466.072277959" watchObservedRunningTime="2025-12-03 11:44:35.488186785 +0000 UTC m=+1466.076428539" Dec 03 11:44:35 crc kubenswrapper[5052]: I1203 11:44:35.520793 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/barbican-api-7cfb847cb8-jn85t" podStartSLOduration=2.520757264 podStartE2EDuration="2.520757264s" podCreationTimestamp="2025-12-03 11:44:33 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:44:35.508940073 +0000 UTC m=+1466.097181837" watchObservedRunningTime="2025-12-03 11:44:35.520757264 +0000 UTC m=+1466.108999028" Dec 03 11:44:36 crc kubenswrapper[5052]: I1203 11:44:36.473689 5052 generic.go:334] "Generic (PLEG): container finished" podID="bf425311-41fb-48cf-b34a-4cc4853c68b3" containerID="bb18e1ff35492d0cbb8cd1813c33935f9d2b132b5086ef0adb508337d0236453" exitCode=0 Dec 03 11:44:36 crc kubenswrapper[5052]: I1203 11:44:36.473770 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"bf425311-41fb-48cf-b34a-4cc4853c68b3","Type":"ContainerDied","Data":"bb18e1ff35492d0cbb8cd1813c33935f9d2b132b5086ef0adb508337d0236453"} Dec 03 11:44:36 crc kubenswrapper[5052]: I1203 11:44:36.476373 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"00b12739-1437-4af3-ab6d-ad830b67ad2e","Type":"ContainerStarted","Data":"988a528826a2862aed42fdaece9c254222b91bc11952d4cd54b0b18c778396b2"} Dec 03 11:44:36 crc kubenswrapper[5052]: I1203 11:44:36.505194 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/cinder-api-0" podStartSLOduration=3.505172012 podStartE2EDuration="3.505172012s" podCreationTimestamp="2025-12-03 11:44:33 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:44:36.494438462 +0000 UTC m=+1467.082680246" watchObservedRunningTime="2025-12-03 11:44:36.505172012 +0000 UTC m=+1467.093413796" Dec 03 11:44:37 crc kubenswrapper[5052]: I1203 11:44:37.490185 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:44:37 crc kubenswrapper[5052]: I1203 11:44:37.674605 5052 scope.go:117] "RemoveContainer" containerID="3b836afc76cec7f0340dacbace97852b77688d99cd5e579a6908a42d2753abf1" Dec 03 11:44:37 crc kubenswrapper[5052]: I1203 11:44:37.674683 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/placement-79d5b958cd-cq6hs" podUID="f8e49a02-823b-4851-9ecf-f2d1696fa201" containerName="placement-log" probeResult="failure" output="Get \"http://10.217.0.243:8778/\": dial tcp 10.217.0.243:8778: connect: connection refused" Dec 03 11:44:38 crc kubenswrapper[5052]: I1203 11:44:38.475341 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:44:38 crc kubenswrapper[5052]: I1203 11:44:38.510344 5052 generic.go:334] "Generic (PLEG): container finished" podID="f8e49a02-823b-4851-9ecf-f2d1696fa201" containerID="c940ea4730f56e003ea25bbdb066d6cd2bced8d502534858c4da22e1bec785f5" exitCode=1 Dec 03 11:44:38 crc kubenswrapper[5052]: I1203 11:44:38.510405 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-79d5b958cd-cq6hs" event={"ID":"f8e49a02-823b-4851-9ecf-f2d1696fa201","Type":"ContainerDied","Data":"c940ea4730f56e003ea25bbdb066d6cd2bced8d502534858c4da22e1bec785f5"} Dec 03 11:44:38 crc kubenswrapper[5052]: I1203 11:44:38.510480 5052 scope.go:117] "RemoveContainer" containerID="3b836afc76cec7f0340dacbace97852b77688d99cd5e579a6908a42d2753abf1" Dec 03 11:44:38 crc kubenswrapper[5052]: I1203 11:44:38.511548 5052 scope.go:117] "RemoveContainer" containerID="c940ea4730f56e003ea25bbdb066d6cd2bced8d502534858c4da22e1bec785f5" Dec 03 11:44:38 crc kubenswrapper[5052]: E1203 11:44:38.511931 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"placement-api\" with CrashLoopBackOff: \"back-off 20s restarting failed container=placement-api pod=placement-79d5b958cd-cq6hs_openstack-kuttl-tests(f8e49a02-823b-4851-9ecf-f2d1696fa201)\"" pod="openstack-kuttl-tests/placement-79d5b958cd-cq6hs" podUID="f8e49a02-823b-4851-9ecf-f2d1696fa201" Dec 03 11:44:38 crc kubenswrapper[5052]: I1203 11:44:38.512162 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/placement-79d5b958cd-cq6hs" podUID="f8e49a02-823b-4851-9ecf-f2d1696fa201" containerName="placement-log" probeResult="failure" output="Get \"http://10.217.0.243:8778/\": dial tcp 10.217.0.243:8778: connect: connection refused" Dec 03 11:44:38 crc kubenswrapper[5052]: I1203 11:44:38.527412 5052 generic.go:334] "Generic (PLEG): container finished" podID="bf425311-41fb-48cf-b34a-4cc4853c68b3" containerID="c17a1df51765ca364dfd03ed31414bd7d7177c8683e926b08de663253d1a4f0f" exitCode=0 Dec 03 11:44:38 crc kubenswrapper[5052]: I1203 11:44:38.527520 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:44:38 crc kubenswrapper[5052]: I1203 11:44:38.527632 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"bf425311-41fb-48cf-b34a-4cc4853c68b3","Type":"ContainerDied","Data":"c17a1df51765ca364dfd03ed31414bd7d7177c8683e926b08de663253d1a4f0f"} Dec 03 11:44:38 crc kubenswrapper[5052]: I1203 11:44:38.527768 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"bf425311-41fb-48cf-b34a-4cc4853c68b3","Type":"ContainerDied","Data":"c37381da810a9c885c1e6987cf95114e831ca98788e37fe70ce26ed8ce8fe27c"} Dec 03 11:44:38 crc kubenswrapper[5052]: I1203 11:44:38.585226 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r9mgh\" (UniqueName: \"kubernetes.io/projected/bf425311-41fb-48cf-b34a-4cc4853c68b3-kube-api-access-r9mgh\") pod \"bf425311-41fb-48cf-b34a-4cc4853c68b3\" (UID: \"bf425311-41fb-48cf-b34a-4cc4853c68b3\") " Dec 03 11:44:38 crc kubenswrapper[5052]: I1203 11:44:38.585368 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/bf425311-41fb-48cf-b34a-4cc4853c68b3-scripts\") pod \"bf425311-41fb-48cf-b34a-4cc4853c68b3\" (UID: \"bf425311-41fb-48cf-b34a-4cc4853c68b3\") " Dec 03 11:44:38 crc kubenswrapper[5052]: I1203 11:44:38.585393 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/bf425311-41fb-48cf-b34a-4cc4853c68b3-combined-ca-bundle\") pod \"bf425311-41fb-48cf-b34a-4cc4853c68b3\" (UID: \"bf425311-41fb-48cf-b34a-4cc4853c68b3\") " Dec 03 11:44:38 crc kubenswrapper[5052]: I1203 11:44:38.585460 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/bf425311-41fb-48cf-b34a-4cc4853c68b3-config-data-custom\") pod \"bf425311-41fb-48cf-b34a-4cc4853c68b3\" (UID: \"bf425311-41fb-48cf-b34a-4cc4853c68b3\") " Dec 03 11:44:38 crc kubenswrapper[5052]: I1203 11:44:38.585502 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/bf425311-41fb-48cf-b34a-4cc4853c68b3-config-data\") pod \"bf425311-41fb-48cf-b34a-4cc4853c68b3\" (UID: \"bf425311-41fb-48cf-b34a-4cc4853c68b3\") " Dec 03 11:44:38 crc kubenswrapper[5052]: I1203 11:44:38.585546 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/bf425311-41fb-48cf-b34a-4cc4853c68b3-etc-machine-id\") pod \"bf425311-41fb-48cf-b34a-4cc4853c68b3\" (UID: \"bf425311-41fb-48cf-b34a-4cc4853c68b3\") " Dec 03 11:44:38 crc kubenswrapper[5052]: I1203 11:44:38.585911 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/bf425311-41fb-48cf-b34a-4cc4853c68b3-etc-machine-id" (OuterVolumeSpecName: "etc-machine-id") pod "bf425311-41fb-48cf-b34a-4cc4853c68b3" (UID: "bf425311-41fb-48cf-b34a-4cc4853c68b3"). InnerVolumeSpecName "etc-machine-id". PluginName "kubernetes.io/host-path", VolumeGidValue "" Dec 03 11:44:38 crc kubenswrapper[5052]: I1203 11:44:38.587202 5052 reconciler_common.go:293] "Volume detached for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/bf425311-41fb-48cf-b34a-4cc4853c68b3-etc-machine-id\") on node \"crc\" DevicePath \"\"" Dec 03 11:44:38 crc kubenswrapper[5052]: I1203 11:44:38.590879 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bf425311-41fb-48cf-b34a-4cc4853c68b3-scripts" (OuterVolumeSpecName: "scripts") pod "bf425311-41fb-48cf-b34a-4cc4853c68b3" (UID: "bf425311-41fb-48cf-b34a-4cc4853c68b3"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:44:38 crc kubenswrapper[5052]: I1203 11:44:38.593734 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bf425311-41fb-48cf-b34a-4cc4853c68b3-config-data-custom" (OuterVolumeSpecName: "config-data-custom") pod "bf425311-41fb-48cf-b34a-4cc4853c68b3" (UID: "bf425311-41fb-48cf-b34a-4cc4853c68b3"). InnerVolumeSpecName "config-data-custom". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:44:38 crc kubenswrapper[5052]: I1203 11:44:38.596031 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bf425311-41fb-48cf-b34a-4cc4853c68b3-kube-api-access-r9mgh" (OuterVolumeSpecName: "kube-api-access-r9mgh") pod "bf425311-41fb-48cf-b34a-4cc4853c68b3" (UID: "bf425311-41fb-48cf-b34a-4cc4853c68b3"). InnerVolumeSpecName "kube-api-access-r9mgh". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:44:38 crc kubenswrapper[5052]: I1203 11:44:38.611969 5052 scope.go:117] "RemoveContainer" containerID="bb18e1ff35492d0cbb8cd1813c33935f9d2b132b5086ef0adb508337d0236453" Dec 03 11:44:38 crc kubenswrapper[5052]: I1203 11:44:38.642001 5052 scope.go:117] "RemoveContainer" containerID="c17a1df51765ca364dfd03ed31414bd7d7177c8683e926b08de663253d1a4f0f" Dec 03 11:44:38 crc kubenswrapper[5052]: I1203 11:44:38.643955 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bf425311-41fb-48cf-b34a-4cc4853c68b3-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "bf425311-41fb-48cf-b34a-4cc4853c68b3" (UID: "bf425311-41fb-48cf-b34a-4cc4853c68b3"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:44:38 crc kubenswrapper[5052]: I1203 11:44:38.669694 5052 scope.go:117] "RemoveContainer" containerID="bb18e1ff35492d0cbb8cd1813c33935f9d2b132b5086ef0adb508337d0236453" Dec 03 11:44:38 crc kubenswrapper[5052]: E1203 11:44:38.670141 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bb18e1ff35492d0cbb8cd1813c33935f9d2b132b5086ef0adb508337d0236453\": container with ID starting with bb18e1ff35492d0cbb8cd1813c33935f9d2b132b5086ef0adb508337d0236453 not found: ID does not exist" containerID="bb18e1ff35492d0cbb8cd1813c33935f9d2b132b5086ef0adb508337d0236453" Dec 03 11:44:38 crc kubenswrapper[5052]: I1203 11:44:38.670184 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bb18e1ff35492d0cbb8cd1813c33935f9d2b132b5086ef0adb508337d0236453"} err="failed to get container status \"bb18e1ff35492d0cbb8cd1813c33935f9d2b132b5086ef0adb508337d0236453\": rpc error: code = NotFound desc = could not find container \"bb18e1ff35492d0cbb8cd1813c33935f9d2b132b5086ef0adb508337d0236453\": container with ID starting with bb18e1ff35492d0cbb8cd1813c33935f9d2b132b5086ef0adb508337d0236453 not found: ID does not exist" Dec 03 11:44:38 crc kubenswrapper[5052]: I1203 11:44:38.670210 5052 scope.go:117] "RemoveContainer" containerID="c17a1df51765ca364dfd03ed31414bd7d7177c8683e926b08de663253d1a4f0f" Dec 03 11:44:38 crc kubenswrapper[5052]: E1203 11:44:38.672819 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c17a1df51765ca364dfd03ed31414bd7d7177c8683e926b08de663253d1a4f0f\": container with ID starting with c17a1df51765ca364dfd03ed31414bd7d7177c8683e926b08de663253d1a4f0f not found: ID does not exist" containerID="c17a1df51765ca364dfd03ed31414bd7d7177c8683e926b08de663253d1a4f0f" Dec 03 11:44:38 crc kubenswrapper[5052]: I1203 11:44:38.672864 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c17a1df51765ca364dfd03ed31414bd7d7177c8683e926b08de663253d1a4f0f"} err="failed to get container status \"c17a1df51765ca364dfd03ed31414bd7d7177c8683e926b08de663253d1a4f0f\": rpc error: code = NotFound desc = could not find container \"c17a1df51765ca364dfd03ed31414bd7d7177c8683e926b08de663253d1a4f0f\": container with ID starting with c17a1df51765ca364dfd03ed31414bd7d7177c8683e926b08de663253d1a4f0f not found: ID does not exist" Dec 03 11:44:38 crc kubenswrapper[5052]: I1203 11:44:38.689137 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-r9mgh\" (UniqueName: \"kubernetes.io/projected/bf425311-41fb-48cf-b34a-4cc4853c68b3-kube-api-access-r9mgh\") on node \"crc\" DevicePath \"\"" Dec 03 11:44:38 crc kubenswrapper[5052]: I1203 11:44:38.689169 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/bf425311-41fb-48cf-b34a-4cc4853c68b3-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:44:38 crc kubenswrapper[5052]: I1203 11:44:38.689179 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/bf425311-41fb-48cf-b34a-4cc4853c68b3-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:44:38 crc kubenswrapper[5052]: I1203 11:44:38.689187 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/bf425311-41fb-48cf-b34a-4cc4853c68b3-config-data-custom\") on node \"crc\" DevicePath \"\"" Dec 03 11:44:38 crc kubenswrapper[5052]: I1203 11:44:38.690773 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bf425311-41fb-48cf-b34a-4cc4853c68b3-config-data" (OuterVolumeSpecName: "config-data") pod "bf425311-41fb-48cf-b34a-4cc4853c68b3" (UID: "bf425311-41fb-48cf-b34a-4cc4853c68b3"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:44:38 crc kubenswrapper[5052]: I1203 11:44:38.792641 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/bf425311-41fb-48cf-b34a-4cc4853c68b3-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:44:38 crc kubenswrapper[5052]: I1203 11:44:38.864091 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 11:44:38 crc kubenswrapper[5052]: I1203 11:44:38.880635 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 11:44:38 crc kubenswrapper[5052]: I1203 11:44:38.884600 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 11:44:38 crc kubenswrapper[5052]: E1203 11:44:38.885281 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="bf425311-41fb-48cf-b34a-4cc4853c68b3" containerName="cinder-scheduler" Dec 03 11:44:38 crc kubenswrapper[5052]: I1203 11:44:38.885304 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="bf425311-41fb-48cf-b34a-4cc4853c68b3" containerName="cinder-scheduler" Dec 03 11:44:38 crc kubenswrapper[5052]: E1203 11:44:38.885317 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="bf425311-41fb-48cf-b34a-4cc4853c68b3" containerName="probe" Dec 03 11:44:38 crc kubenswrapper[5052]: I1203 11:44:38.885323 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="bf425311-41fb-48cf-b34a-4cc4853c68b3" containerName="probe" Dec 03 11:44:38 crc kubenswrapper[5052]: I1203 11:44:38.885481 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="bf425311-41fb-48cf-b34a-4cc4853c68b3" containerName="cinder-scheduler" Dec 03 11:44:38 crc kubenswrapper[5052]: I1203 11:44:38.885522 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="bf425311-41fb-48cf-b34a-4cc4853c68b3" containerName="probe" Dec 03 11:44:38 crc kubenswrapper[5052]: I1203 11:44:38.888396 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:44:38 crc kubenswrapper[5052]: I1203 11:44:38.895201 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cinder-scheduler-config-data" Dec 03 11:44:38 crc kubenswrapper[5052]: I1203 11:44:38.895437 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 11:44:38 crc kubenswrapper[5052]: I1203 11:44:38.934807 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openshift-marketplace/certified-operators-r2kbh"] Dec 03 11:44:38 crc kubenswrapper[5052]: I1203 11:44:38.936704 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/certified-operators-r2kbh" Dec 03 11:44:38 crc kubenswrapper[5052]: I1203 11:44:38.964129 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/certified-operators-r2kbh"] Dec 03 11:44:38 crc kubenswrapper[5052]: I1203 11:44:38.995691 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e947107f-104b-4699-a994-bc77285f0cf5-combined-ca-bundle\") pod \"cinder-scheduler-0\" (UID: \"e947107f-104b-4699-a994-bc77285f0cf5\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:44:38 crc kubenswrapper[5052]: I1203 11:44:38.995745 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/e947107f-104b-4699-a994-bc77285f0cf5-scripts\") pod \"cinder-scheduler-0\" (UID: \"e947107f-104b-4699-a994-bc77285f0cf5\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:44:38 crc kubenswrapper[5052]: I1203 11:44:38.995763 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/e947107f-104b-4699-a994-bc77285f0cf5-config-data-custom\") pod \"cinder-scheduler-0\" (UID: \"e947107f-104b-4699-a994-bc77285f0cf5\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:44:38 crc kubenswrapper[5052]: I1203 11:44:38.995786 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xjztz\" (UniqueName: \"kubernetes.io/projected/75e05b55-fc4c-4f5f-851a-b0ae877fa047-kube-api-access-xjztz\") pod \"certified-operators-r2kbh\" (UID: \"75e05b55-fc4c-4f5f-851a-b0ae877fa047\") " pod="openshift-marketplace/certified-operators-r2kbh" Dec 03 11:44:38 crc kubenswrapper[5052]: I1203 11:44:38.995804 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/75e05b55-fc4c-4f5f-851a-b0ae877fa047-utilities\") pod \"certified-operators-r2kbh\" (UID: \"75e05b55-fc4c-4f5f-851a-b0ae877fa047\") " pod="openshift-marketplace/certified-operators-r2kbh" Dec 03 11:44:38 crc kubenswrapper[5052]: I1203 11:44:38.995842 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/e947107f-104b-4699-a994-bc77285f0cf5-etc-machine-id\") pod \"cinder-scheduler-0\" (UID: \"e947107f-104b-4699-a994-bc77285f0cf5\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:44:38 crc kubenswrapper[5052]: I1203 11:44:38.995872 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/e947107f-104b-4699-a994-bc77285f0cf5-config-data\") pod \"cinder-scheduler-0\" (UID: \"e947107f-104b-4699-a994-bc77285f0cf5\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:44:38 crc kubenswrapper[5052]: I1203 11:44:38.995895 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/75e05b55-fc4c-4f5f-851a-b0ae877fa047-catalog-content\") pod \"certified-operators-r2kbh\" (UID: \"75e05b55-fc4c-4f5f-851a-b0ae877fa047\") " pod="openshift-marketplace/certified-operators-r2kbh" Dec 03 11:44:38 crc kubenswrapper[5052]: I1203 11:44:38.995925 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wnktp\" (UniqueName: \"kubernetes.io/projected/e947107f-104b-4699-a994-bc77285f0cf5-kube-api-access-wnktp\") pod \"cinder-scheduler-0\" (UID: \"e947107f-104b-4699-a994-bc77285f0cf5\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:44:39 crc kubenswrapper[5052]: I1203 11:44:39.097962 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-wnktp\" (UniqueName: \"kubernetes.io/projected/e947107f-104b-4699-a994-bc77285f0cf5-kube-api-access-wnktp\") pod \"cinder-scheduler-0\" (UID: \"e947107f-104b-4699-a994-bc77285f0cf5\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:44:39 crc kubenswrapper[5052]: I1203 11:44:39.098071 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e947107f-104b-4699-a994-bc77285f0cf5-combined-ca-bundle\") pod \"cinder-scheduler-0\" (UID: \"e947107f-104b-4699-a994-bc77285f0cf5\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:44:39 crc kubenswrapper[5052]: I1203 11:44:39.098100 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/e947107f-104b-4699-a994-bc77285f0cf5-scripts\") pod \"cinder-scheduler-0\" (UID: \"e947107f-104b-4699-a994-bc77285f0cf5\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:44:39 crc kubenswrapper[5052]: I1203 11:44:39.098121 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/e947107f-104b-4699-a994-bc77285f0cf5-config-data-custom\") pod \"cinder-scheduler-0\" (UID: \"e947107f-104b-4699-a994-bc77285f0cf5\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:44:39 crc kubenswrapper[5052]: I1203 11:44:39.098146 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-xjztz\" (UniqueName: \"kubernetes.io/projected/75e05b55-fc4c-4f5f-851a-b0ae877fa047-kube-api-access-xjztz\") pod \"certified-operators-r2kbh\" (UID: \"75e05b55-fc4c-4f5f-851a-b0ae877fa047\") " pod="openshift-marketplace/certified-operators-r2kbh" Dec 03 11:44:39 crc kubenswrapper[5052]: I1203 11:44:39.098175 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/75e05b55-fc4c-4f5f-851a-b0ae877fa047-utilities\") pod \"certified-operators-r2kbh\" (UID: \"75e05b55-fc4c-4f5f-851a-b0ae877fa047\") " pod="openshift-marketplace/certified-operators-r2kbh" Dec 03 11:44:39 crc kubenswrapper[5052]: I1203 11:44:39.098225 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/e947107f-104b-4699-a994-bc77285f0cf5-etc-machine-id\") pod \"cinder-scheduler-0\" (UID: \"e947107f-104b-4699-a994-bc77285f0cf5\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:44:39 crc kubenswrapper[5052]: I1203 11:44:39.098270 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/e947107f-104b-4699-a994-bc77285f0cf5-config-data\") pod \"cinder-scheduler-0\" (UID: \"e947107f-104b-4699-a994-bc77285f0cf5\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:44:39 crc kubenswrapper[5052]: I1203 11:44:39.098303 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/75e05b55-fc4c-4f5f-851a-b0ae877fa047-catalog-content\") pod \"certified-operators-r2kbh\" (UID: \"75e05b55-fc4c-4f5f-851a-b0ae877fa047\") " pod="openshift-marketplace/certified-operators-r2kbh" Dec 03 11:44:39 crc kubenswrapper[5052]: I1203 11:44:39.098883 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/75e05b55-fc4c-4f5f-851a-b0ae877fa047-catalog-content\") pod \"certified-operators-r2kbh\" (UID: \"75e05b55-fc4c-4f5f-851a-b0ae877fa047\") " pod="openshift-marketplace/certified-operators-r2kbh" Dec 03 11:44:39 crc kubenswrapper[5052]: I1203 11:44:39.100193 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/e947107f-104b-4699-a994-bc77285f0cf5-etc-machine-id\") pod \"cinder-scheduler-0\" (UID: \"e947107f-104b-4699-a994-bc77285f0cf5\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:44:39 crc kubenswrapper[5052]: I1203 11:44:39.100772 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/75e05b55-fc4c-4f5f-851a-b0ae877fa047-utilities\") pod \"certified-operators-r2kbh\" (UID: \"75e05b55-fc4c-4f5f-851a-b0ae877fa047\") " pod="openshift-marketplace/certified-operators-r2kbh" Dec 03 11:44:39 crc kubenswrapper[5052]: I1203 11:44:39.105721 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e947107f-104b-4699-a994-bc77285f0cf5-combined-ca-bundle\") pod \"cinder-scheduler-0\" (UID: \"e947107f-104b-4699-a994-bc77285f0cf5\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:44:39 crc kubenswrapper[5052]: I1203 11:44:39.107067 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/e947107f-104b-4699-a994-bc77285f0cf5-scripts\") pod \"cinder-scheduler-0\" (UID: \"e947107f-104b-4699-a994-bc77285f0cf5\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:44:39 crc kubenswrapper[5052]: I1203 11:44:39.116408 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/e947107f-104b-4699-a994-bc77285f0cf5-config-data\") pod \"cinder-scheduler-0\" (UID: \"e947107f-104b-4699-a994-bc77285f0cf5\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:44:39 crc kubenswrapper[5052]: I1203 11:44:39.120073 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-xjztz\" (UniqueName: \"kubernetes.io/projected/75e05b55-fc4c-4f5f-851a-b0ae877fa047-kube-api-access-xjztz\") pod \"certified-operators-r2kbh\" (UID: \"75e05b55-fc4c-4f5f-851a-b0ae877fa047\") " pod="openshift-marketplace/certified-operators-r2kbh" Dec 03 11:44:39 crc kubenswrapper[5052]: I1203 11:44:39.126734 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/e947107f-104b-4699-a994-bc77285f0cf5-config-data-custom\") pod \"cinder-scheduler-0\" (UID: \"e947107f-104b-4699-a994-bc77285f0cf5\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:44:39 crc kubenswrapper[5052]: I1203 11:44:39.191671 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-wnktp\" (UniqueName: \"kubernetes.io/projected/e947107f-104b-4699-a994-bc77285f0cf5-kube-api-access-wnktp\") pod \"cinder-scheduler-0\" (UID: \"e947107f-104b-4699-a994-bc77285f0cf5\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:44:39 crc kubenswrapper[5052]: I1203 11:44:39.214972 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:44:39 crc kubenswrapper[5052]: I1203 11:44:39.262342 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/certified-operators-r2kbh" Dec 03 11:44:39 crc kubenswrapper[5052]: I1203 11:44:39.692816 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bf425311-41fb-48cf-b34a-4cc4853c68b3" path="/var/lib/kubelet/pods/bf425311-41fb-48cf-b34a-4cc4853c68b3/volumes" Dec 03 11:44:39 crc kubenswrapper[5052]: I1203 11:44:39.868246 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/certified-operators-r2kbh"] Dec 03 11:44:39 crc kubenswrapper[5052]: I1203 11:44:39.956268 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 11:44:39 crc kubenswrapper[5052]: W1203 11:44:39.973623 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode947107f_104b_4699_a994_bc77285f0cf5.slice/crio-4603e5e66c2357efa797fc51b20cdd6bd29abce2aa0235abf027cb539fb8a8a3 WatchSource:0}: Error finding container 4603e5e66c2357efa797fc51b20cdd6bd29abce2aa0235abf027cb539fb8a8a3: Status 404 returned error can't find the container with id 4603e5e66c2357efa797fc51b20cdd6bd29abce2aa0235abf027cb539fb8a8a3 Dec 03 11:44:40 crc kubenswrapper[5052]: I1203 11:44:40.585087 5052 generic.go:334] "Generic (PLEG): container finished" podID="75e05b55-fc4c-4f5f-851a-b0ae877fa047" containerID="c7ea52f80b0dfc432233b0ea7a7abf1169eec941d91338addc4291e60b8a5fa0" exitCode=0 Dec 03 11:44:40 crc kubenswrapper[5052]: I1203 11:44:40.585162 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/certified-operators-r2kbh" event={"ID":"75e05b55-fc4c-4f5f-851a-b0ae877fa047","Type":"ContainerDied","Data":"c7ea52f80b0dfc432233b0ea7a7abf1169eec941d91338addc4291e60b8a5fa0"} Dec 03 11:44:40 crc kubenswrapper[5052]: I1203 11:44:40.585196 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/certified-operators-r2kbh" event={"ID":"75e05b55-fc4c-4f5f-851a-b0ae877fa047","Type":"ContainerStarted","Data":"387f5544760c6687105ae813693edb861cb041fb41faaad87e52a091b12a403b"} Dec 03 11:44:40 crc kubenswrapper[5052]: I1203 11:44:40.588341 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"e947107f-104b-4699-a994-bc77285f0cf5","Type":"ContainerStarted","Data":"4603e5e66c2357efa797fc51b20cdd6bd29abce2aa0235abf027cb539fb8a8a3"} Dec 03 11:44:41 crc kubenswrapper[5052]: I1203 11:44:41.640850 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"e947107f-104b-4699-a994-bc77285f0cf5","Type":"ContainerStarted","Data":"08b743014f4f3e11912f7da330bbe8c095df94c7cd01737b812783a0bd8af17b"} Dec 03 11:44:41 crc kubenswrapper[5052]: I1203 11:44:41.643129 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"e947107f-104b-4699-a994-bc77285f0cf5","Type":"ContainerStarted","Data":"30b3c8981cdec3a0ba91236f985bc49025e43effcb7cbf0b9e7cddcc3ba0a0d9"} Dec 03 11:44:41 crc kubenswrapper[5052]: I1203 11:44:41.673857 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/cinder-scheduler-0" podStartSLOduration=3.673835659 podStartE2EDuration="3.673835659s" podCreationTimestamp="2025-12-03 11:44:38 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:44:41.663276894 +0000 UTC m=+1472.251518658" watchObservedRunningTime="2025-12-03 11:44:41.673835659 +0000 UTC m=+1472.262077423" Dec 03 11:44:44 crc kubenswrapper[5052]: I1203 11:44:44.216376 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:44:45 crc kubenswrapper[5052]: I1203 11:44:45.941630 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/barbican-api-7cfb847cb8-jn85t" Dec 03 11:44:45 crc kubenswrapper[5052]: I1203 11:44:45.991109 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:44:46 crc kubenswrapper[5052]: I1203 11:44:46.077256 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/barbican-api-7cfb847cb8-jn85t" Dec 03 11:44:46 crc kubenswrapper[5052]: I1203 11:44:46.156205 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-api-f76947bf6-27nv8"] Dec 03 11:44:46 crc kubenswrapper[5052]: I1203 11:44:46.156498 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/barbican-api-f76947bf6-27nv8" podUID="23b2a11f-0129-474b-9f97-f9401abd3f6b" containerName="barbican-api-log" containerID="cri-o://562c233abb263737b9f5f1e4e7573903554a8770ec0a228f0ae6b1eaeed438cd" gracePeriod=30 Dec 03 11:44:46 crc kubenswrapper[5052]: I1203 11:44:46.156596 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/barbican-api-f76947bf6-27nv8" podUID="23b2a11f-0129-474b-9f97-f9401abd3f6b" containerName="barbican-api" containerID="cri-o://b0260015d44c0e665a4e129dce5d0997d993c150242031889ee970b1dbfc89df" gracePeriod=30 Dec 03 11:44:46 crc kubenswrapper[5052]: I1203 11:44:46.717795 5052 generic.go:334] "Generic (PLEG): container finished" podID="23b2a11f-0129-474b-9f97-f9401abd3f6b" containerID="562c233abb263737b9f5f1e4e7573903554a8770ec0a228f0ae6b1eaeed438cd" exitCode=143 Dec 03 11:44:46 crc kubenswrapper[5052]: I1203 11:44:46.717888 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-f76947bf6-27nv8" event={"ID":"23b2a11f-0129-474b-9f97-f9401abd3f6b","Type":"ContainerDied","Data":"562c233abb263737b9f5f1e4e7573903554a8770ec0a228f0ae6b1eaeed438cd"} Dec 03 11:44:46 crc kubenswrapper[5052]: I1203 11:44:46.721298 5052 generic.go:334] "Generic (PLEG): container finished" podID="75e05b55-fc4c-4f5f-851a-b0ae877fa047" containerID="ce6db8fb0a474621307c611e33ee1e3a5ffcfa84a71596595a3bef2957d21d9e" exitCode=0 Dec 03 11:44:46 crc kubenswrapper[5052]: I1203 11:44:46.721341 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/certified-operators-r2kbh" event={"ID":"75e05b55-fc4c-4f5f-851a-b0ae877fa047","Type":"ContainerDied","Data":"ce6db8fb0a474621307c611e33ee1e3a5ffcfa84a71596595a3bef2957d21d9e"} Dec 03 11:44:48 crc kubenswrapper[5052]: I1203 11:44:48.744021 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/certified-operators-r2kbh" event={"ID":"75e05b55-fc4c-4f5f-851a-b0ae877fa047","Type":"ContainerStarted","Data":"eef6d0530580389e628b73bdd0eb124ec3415525e27838abca266e7220561c37"} Dec 03 11:44:48 crc kubenswrapper[5052]: I1203 11:44:48.769110 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-marketplace/certified-operators-r2kbh" podStartSLOduration=3.78138848 podStartE2EDuration="10.769093903s" podCreationTimestamp="2025-12-03 11:44:38 +0000 UTC" firstStartedPulling="2025-12-03 11:44:40.587734999 +0000 UTC m=+1471.175976763" lastFinishedPulling="2025-12-03 11:44:47.575440422 +0000 UTC m=+1478.163682186" observedRunningTime="2025-12-03 11:44:48.763225244 +0000 UTC m=+1479.351467008" watchObservedRunningTime="2025-12-03 11:44:48.769093903 +0000 UTC m=+1479.357335657" Dec 03 11:44:49 crc kubenswrapper[5052]: I1203 11:44:49.263628 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openshift-marketplace/certified-operators-r2kbh" Dec 03 11:44:49 crc kubenswrapper[5052]: I1203 11:44:49.263711 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-marketplace/certified-operators-r2kbh" Dec 03 11:44:49 crc kubenswrapper[5052]: I1203 11:44:49.316716 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-marketplace/certified-operators-r2kbh" Dec 03 11:44:49 crc kubenswrapper[5052]: I1203 11:44:49.466794 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:44:49 crc kubenswrapper[5052]: I1203 11:44:49.757182 5052 generic.go:334] "Generic (PLEG): container finished" podID="23b2a11f-0129-474b-9f97-f9401abd3f6b" containerID="b0260015d44c0e665a4e129dce5d0997d993c150242031889ee970b1dbfc89df" exitCode=0 Dec 03 11:44:49 crc kubenswrapper[5052]: I1203 11:44:49.757273 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-f76947bf6-27nv8" event={"ID":"23b2a11f-0129-474b-9f97-f9401abd3f6b","Type":"ContainerDied","Data":"b0260015d44c0e665a4e129dce5d0997d993c150242031889ee970b1dbfc89df"} Dec 03 11:44:49 crc kubenswrapper[5052]: I1203 11:44:49.758421 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-f76947bf6-27nv8" event={"ID":"23b2a11f-0129-474b-9f97-f9401abd3f6b","Type":"ContainerDied","Data":"784f6e93c3ddc744790d11bc216e9351d404558dd5301abec92c2c027daedd51"} Dec 03 11:44:49 crc kubenswrapper[5052]: I1203 11:44:49.758438 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="784f6e93c3ddc744790d11bc216e9351d404558dd5301abec92c2c027daedd51" Dec 03 11:44:49 crc kubenswrapper[5052]: I1203 11:44:49.784965 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-api-f76947bf6-27nv8" Dec 03 11:44:49 crc kubenswrapper[5052]: I1203 11:44:49.926486 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/23b2a11f-0129-474b-9f97-f9401abd3f6b-config-data\") pod \"23b2a11f-0129-474b-9f97-f9401abd3f6b\" (UID: \"23b2a11f-0129-474b-9f97-f9401abd3f6b\") " Dec 03 11:44:49 crc kubenswrapper[5052]: I1203 11:44:49.926554 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/23b2a11f-0129-474b-9f97-f9401abd3f6b-logs\") pod \"23b2a11f-0129-474b-9f97-f9401abd3f6b\" (UID: \"23b2a11f-0129-474b-9f97-f9401abd3f6b\") " Dec 03 11:44:49 crc kubenswrapper[5052]: I1203 11:44:49.926624 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/23b2a11f-0129-474b-9f97-f9401abd3f6b-config-data-custom\") pod \"23b2a11f-0129-474b-9f97-f9401abd3f6b\" (UID: \"23b2a11f-0129-474b-9f97-f9401abd3f6b\") " Dec 03 11:44:49 crc kubenswrapper[5052]: I1203 11:44:49.926662 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mtm6h\" (UniqueName: \"kubernetes.io/projected/23b2a11f-0129-474b-9f97-f9401abd3f6b-kube-api-access-mtm6h\") pod \"23b2a11f-0129-474b-9f97-f9401abd3f6b\" (UID: \"23b2a11f-0129-474b-9f97-f9401abd3f6b\") " Dec 03 11:44:49 crc kubenswrapper[5052]: I1203 11:44:49.926761 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/23b2a11f-0129-474b-9f97-f9401abd3f6b-combined-ca-bundle\") pod \"23b2a11f-0129-474b-9f97-f9401abd3f6b\" (UID: \"23b2a11f-0129-474b-9f97-f9401abd3f6b\") " Dec 03 11:44:49 crc kubenswrapper[5052]: I1203 11:44:49.928027 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/23b2a11f-0129-474b-9f97-f9401abd3f6b-logs" (OuterVolumeSpecName: "logs") pod "23b2a11f-0129-474b-9f97-f9401abd3f6b" (UID: "23b2a11f-0129-474b-9f97-f9401abd3f6b"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:44:49 crc kubenswrapper[5052]: I1203 11:44:49.933018 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/23b2a11f-0129-474b-9f97-f9401abd3f6b-kube-api-access-mtm6h" (OuterVolumeSpecName: "kube-api-access-mtm6h") pod "23b2a11f-0129-474b-9f97-f9401abd3f6b" (UID: "23b2a11f-0129-474b-9f97-f9401abd3f6b"). InnerVolumeSpecName "kube-api-access-mtm6h". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:44:49 crc kubenswrapper[5052]: I1203 11:44:49.940193 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/23b2a11f-0129-474b-9f97-f9401abd3f6b-config-data-custom" (OuterVolumeSpecName: "config-data-custom") pod "23b2a11f-0129-474b-9f97-f9401abd3f6b" (UID: "23b2a11f-0129-474b-9f97-f9401abd3f6b"). InnerVolumeSpecName "config-data-custom". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:44:49 crc kubenswrapper[5052]: I1203 11:44:49.966325 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/23b2a11f-0129-474b-9f97-f9401abd3f6b-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "23b2a11f-0129-474b-9f97-f9401abd3f6b" (UID: "23b2a11f-0129-474b-9f97-f9401abd3f6b"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:44:49 crc kubenswrapper[5052]: I1203 11:44:49.984095 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/23b2a11f-0129-474b-9f97-f9401abd3f6b-config-data" (OuterVolumeSpecName: "config-data") pod "23b2a11f-0129-474b-9f97-f9401abd3f6b" (UID: "23b2a11f-0129-474b-9f97-f9401abd3f6b"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:44:50 crc kubenswrapper[5052]: I1203 11:44:50.028802 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/23b2a11f-0129-474b-9f97-f9401abd3f6b-logs\") on node \"crc\" DevicePath \"\"" Dec 03 11:44:50 crc kubenswrapper[5052]: I1203 11:44:50.028836 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/23b2a11f-0129-474b-9f97-f9401abd3f6b-config-data-custom\") on node \"crc\" DevicePath \"\"" Dec 03 11:44:50 crc kubenswrapper[5052]: I1203 11:44:50.028846 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-mtm6h\" (UniqueName: \"kubernetes.io/projected/23b2a11f-0129-474b-9f97-f9401abd3f6b-kube-api-access-mtm6h\") on node \"crc\" DevicePath \"\"" Dec 03 11:44:50 crc kubenswrapper[5052]: I1203 11:44:50.028855 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/23b2a11f-0129-474b-9f97-f9401abd3f6b-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:44:50 crc kubenswrapper[5052]: I1203 11:44:50.028863 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/23b2a11f-0129-474b-9f97-f9401abd3f6b-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:44:50 crc kubenswrapper[5052]: I1203 11:44:50.767733 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-api-f76947bf6-27nv8" Dec 03 11:44:50 crc kubenswrapper[5052]: I1203 11:44:50.799067 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-api-f76947bf6-27nv8"] Dec 03 11:44:50 crc kubenswrapper[5052]: I1203 11:44:50.808376 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/barbican-api-f76947bf6-27nv8"] Dec 03 11:44:51 crc kubenswrapper[5052]: I1203 11:44:51.046945 5052 kubelet.go:2542] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openstack-kuttl-tests/placement-79d5b958cd-cq6hs" Dec 03 11:44:51 crc kubenswrapper[5052]: I1203 11:44:51.047731 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/placement-79d5b958cd-cq6hs" Dec 03 11:44:51 crc kubenswrapper[5052]: I1203 11:44:51.048443 5052 prober.go:107] "Probe failed" probeType="Liveness" pod="openstack-kuttl-tests/placement-79d5b958cd-cq6hs" podUID="f8e49a02-823b-4851-9ecf-f2d1696fa201" containerName="placement-log" probeResult="failure" output="Get \"http://10.217.0.243:8778/\": dial tcp 10.217.0.243:8778: connect: connection refused" Dec 03 11:44:51 crc kubenswrapper[5052]: I1203 11:44:51.048505 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/placement-79d5b958cd-cq6hs" podUID="f8e49a02-823b-4851-9ecf-f2d1696fa201" containerName="placement-log" probeResult="failure" output="Get \"http://10.217.0.243:8778/\": dial tcp 10.217.0.243:8778: connect: connection refused" Dec 03 11:44:51 crc kubenswrapper[5052]: I1203 11:44:51.049141 5052 scope.go:117] "RemoveContainer" containerID="c940ea4730f56e003ea25bbdb066d6cd2bced8d502534858c4da22e1bec785f5" Dec 03 11:44:51 crc kubenswrapper[5052]: I1203 11:44:51.049179 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/placement-79d5b958cd-cq6hs" podUID="f8e49a02-823b-4851-9ecf-f2d1696fa201" containerName="placement-log" probeResult="failure" output="Get \"http://10.217.0.243:8778/\": dial tcp 10.217.0.243:8778: connect: connection refused" Dec 03 11:44:51 crc kubenswrapper[5052]: E1203 11:44:51.049393 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"placement-api\" with CrashLoopBackOff: \"back-off 20s restarting failed container=placement-api pod=placement-79d5b958cd-cq6hs_openstack-kuttl-tests(f8e49a02-823b-4851-9ecf-f2d1696fa201)\"" pod="openstack-kuttl-tests/placement-79d5b958cd-cq6hs" podUID="f8e49a02-823b-4851-9ecf-f2d1696fa201" Dec 03 11:44:51 crc kubenswrapper[5052]: I1203 11:44:51.686335 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="23b2a11f-0129-474b-9f97-f9401abd3f6b" path="/var/lib/kubelet/pods/23b2a11f-0129-474b-9f97-f9401abd3f6b/volumes" Dec 03 11:44:52 crc kubenswrapper[5052]: I1203 11:44:52.753358 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/placement-689c6465d6-nsjrx" Dec 03 11:44:52 crc kubenswrapper[5052]: I1203 11:44:52.757706 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/placement-689c6465d6-nsjrx" Dec 03 11:44:52 crc kubenswrapper[5052]: I1203 11:44:52.853469 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/placement-79d5b958cd-cq6hs"] Dec 03 11:44:52 crc kubenswrapper[5052]: I1203 11:44:52.853969 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/placement-79d5b958cd-cq6hs" podUID="f8e49a02-823b-4851-9ecf-f2d1696fa201" containerName="placement-log" containerID="cri-o://4565a0244d0b774732f948581904402406dc7ecdcdc28c9c004dd467cedd108d" gracePeriod=30 Dec 03 11:44:52 crc kubenswrapper[5052]: I1203 11:44:52.854484 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/placement-79d5b958cd-cq6hs" podUID="f8e49a02-823b-4851-9ecf-f2d1696fa201" containerName="placement-log" probeResult="failure" output="Get \"http://10.217.0.243:8778/\": dial tcp 10.217.0.243:8778: connect: connection refused" Dec 03 11:44:53 crc kubenswrapper[5052]: I1203 11:44:53.808843 5052 generic.go:334] "Generic (PLEG): container finished" podID="f8e49a02-823b-4851-9ecf-f2d1696fa201" containerID="4565a0244d0b774732f948581904402406dc7ecdcdc28c9c004dd467cedd108d" exitCode=143 Dec 03 11:44:53 crc kubenswrapper[5052]: I1203 11:44:53.808911 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-79d5b958cd-cq6hs" event={"ID":"f8e49a02-823b-4851-9ecf-f2d1696fa201","Type":"ContainerDied","Data":"4565a0244d0b774732f948581904402406dc7ecdcdc28c9c004dd467cedd108d"} Dec 03 11:44:53 crc kubenswrapper[5052]: I1203 11:44:53.907493 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-79d5b958cd-cq6hs" Dec 03 11:44:53 crc kubenswrapper[5052]: I1203 11:44:53.992159 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/f8e49a02-823b-4851-9ecf-f2d1696fa201-config-data\") pod \"f8e49a02-823b-4851-9ecf-f2d1696fa201\" (UID: \"f8e49a02-823b-4851-9ecf-f2d1696fa201\") " Dec 03 11:44:53 crc kubenswrapper[5052]: I1203 11:44:53.992321 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9clnj\" (UniqueName: \"kubernetes.io/projected/f8e49a02-823b-4851-9ecf-f2d1696fa201-kube-api-access-9clnj\") pod \"f8e49a02-823b-4851-9ecf-f2d1696fa201\" (UID: \"f8e49a02-823b-4851-9ecf-f2d1696fa201\") " Dec 03 11:44:53 crc kubenswrapper[5052]: I1203 11:44:53.992361 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/f8e49a02-823b-4851-9ecf-f2d1696fa201-scripts\") pod \"f8e49a02-823b-4851-9ecf-f2d1696fa201\" (UID: \"f8e49a02-823b-4851-9ecf-f2d1696fa201\") " Dec 03 11:44:53 crc kubenswrapper[5052]: I1203 11:44:53.992447 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/f8e49a02-823b-4851-9ecf-f2d1696fa201-logs\") pod \"f8e49a02-823b-4851-9ecf-f2d1696fa201\" (UID: \"f8e49a02-823b-4851-9ecf-f2d1696fa201\") " Dec 03 11:44:53 crc kubenswrapper[5052]: I1203 11:44:53.992580 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f8e49a02-823b-4851-9ecf-f2d1696fa201-combined-ca-bundle\") pod \"f8e49a02-823b-4851-9ecf-f2d1696fa201\" (UID: \"f8e49a02-823b-4851-9ecf-f2d1696fa201\") " Dec 03 11:44:53 crc kubenswrapper[5052]: I1203 11:44:53.993049 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f8e49a02-823b-4851-9ecf-f2d1696fa201-logs" (OuterVolumeSpecName: "logs") pod "f8e49a02-823b-4851-9ecf-f2d1696fa201" (UID: "f8e49a02-823b-4851-9ecf-f2d1696fa201"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:44:53 crc kubenswrapper[5052]: I1203 11:44:53.993920 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/f8e49a02-823b-4851-9ecf-f2d1696fa201-logs\") on node \"crc\" DevicePath \"\"" Dec 03 11:44:53 crc kubenswrapper[5052]: I1203 11:44:53.998597 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f8e49a02-823b-4851-9ecf-f2d1696fa201-kube-api-access-9clnj" (OuterVolumeSpecName: "kube-api-access-9clnj") pod "f8e49a02-823b-4851-9ecf-f2d1696fa201" (UID: "f8e49a02-823b-4851-9ecf-f2d1696fa201"). InnerVolumeSpecName "kube-api-access-9clnj". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:44:53 crc kubenswrapper[5052]: I1203 11:44:53.998611 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f8e49a02-823b-4851-9ecf-f2d1696fa201-scripts" (OuterVolumeSpecName: "scripts") pod "f8e49a02-823b-4851-9ecf-f2d1696fa201" (UID: "f8e49a02-823b-4851-9ecf-f2d1696fa201"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:44:54 crc kubenswrapper[5052]: I1203 11:44:54.041907 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f8e49a02-823b-4851-9ecf-f2d1696fa201-config-data" (OuterVolumeSpecName: "config-data") pod "f8e49a02-823b-4851-9ecf-f2d1696fa201" (UID: "f8e49a02-823b-4851-9ecf-f2d1696fa201"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:44:54 crc kubenswrapper[5052]: I1203 11:44:54.042733 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f8e49a02-823b-4851-9ecf-f2d1696fa201-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "f8e49a02-823b-4851-9ecf-f2d1696fa201" (UID: "f8e49a02-823b-4851-9ecf-f2d1696fa201"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:44:54 crc kubenswrapper[5052]: I1203 11:44:54.095775 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/f8e49a02-823b-4851-9ecf-f2d1696fa201-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:44:54 crc kubenswrapper[5052]: I1203 11:44:54.095816 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f8e49a02-823b-4851-9ecf-f2d1696fa201-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:44:54 crc kubenswrapper[5052]: I1203 11:44:54.095830 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/f8e49a02-823b-4851-9ecf-f2d1696fa201-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:44:54 crc kubenswrapper[5052]: I1203 11:44:54.095856 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-9clnj\" (UniqueName: \"kubernetes.io/projected/f8e49a02-823b-4851-9ecf-f2d1696fa201-kube-api-access-9clnj\") on node \"crc\" DevicePath \"\"" Dec 03 11:44:54 crc kubenswrapper[5052]: I1203 11:44:54.547995 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/neutron-7cc99c958f-bjn2l" Dec 03 11:44:54 crc kubenswrapper[5052]: I1203 11:44:54.821139 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-79d5b958cd-cq6hs" event={"ID":"f8e49a02-823b-4851-9ecf-f2d1696fa201","Type":"ContainerDied","Data":"d6396b2c9995d5e984317dd8c2f933f027bba5fedf7a60a68099ea46c8bf40ac"} Dec 03 11:44:54 crc kubenswrapper[5052]: I1203 11:44:54.821200 5052 scope.go:117] "RemoveContainer" containerID="c940ea4730f56e003ea25bbdb066d6cd2bced8d502534858c4da22e1bec785f5" Dec 03 11:44:54 crc kubenswrapper[5052]: I1203 11:44:54.821320 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-79d5b958cd-cq6hs" Dec 03 11:44:54 crc kubenswrapper[5052]: I1203 11:44:54.852942 5052 scope.go:117] "RemoveContainer" containerID="4565a0244d0b774732f948581904402406dc7ecdcdc28c9c004dd467cedd108d" Dec 03 11:44:54 crc kubenswrapper[5052]: I1203 11:44:54.895804 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/placement-79d5b958cd-cq6hs"] Dec 03 11:44:54 crc kubenswrapper[5052]: I1203 11:44:54.905887 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/placement-79d5b958cd-cq6hs"] Dec 03 11:44:55 crc kubenswrapper[5052]: I1203 11:44:55.064771 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:44:55 crc kubenswrapper[5052]: I1203 11:44:55.684703 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f8e49a02-823b-4851-9ecf-f2d1696fa201" path="/var/lib/kubelet/pods/f8e49a02-823b-4851-9ecf-f2d1696fa201/volumes" Dec 03 11:44:59 crc kubenswrapper[5052]: I1203 11:44:59.312053 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-marketplace/certified-operators-r2kbh" Dec 03 11:44:59 crc kubenswrapper[5052]: I1203 11:44:59.374676 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/certified-operators-r2kbh"] Dec 03 11:44:59 crc kubenswrapper[5052]: I1203 11:44:59.379894 5052 patch_prober.go:28] interesting pod/machine-config-daemon-74bj6 container/machine-config-daemon namespace/openshift-machine-config-operator: Liveness probe status=failure output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" start-of-body= Dec 03 11:44:59 crc kubenswrapper[5052]: I1203 11:44:59.380188 5052 prober.go:107] "Probe failed" probeType="Liveness" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" containerName="machine-config-daemon" probeResult="failure" output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" Dec 03 11:44:59 crc kubenswrapper[5052]: I1203 11:44:59.412957 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openshift-marketplace/certified-operators-7wd6v"] Dec 03 11:44:59 crc kubenswrapper[5052]: I1203 11:44:59.413959 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openshift-marketplace/certified-operators-7wd6v" podUID="fb0eab4c-4728-4b3f-9df0-70bd099cb8c9" containerName="registry-server" containerID="cri-o://e9e06f4cae7c528e5ad942a4d6fd8849b4e5b0d0000bfe3132ef7697b540b97b" gracePeriod=2 Dec 03 11:45:00 crc kubenswrapper[5052]: I1203 11:45:00.168748 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openshift-operator-lifecycle-manager/collect-profiles-29412705-7rvq9"] Dec 03 11:45:00 crc kubenswrapper[5052]: E1203 11:45:00.169480 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="f8e49a02-823b-4851-9ecf-f2d1696fa201" containerName="placement-log" Dec 03 11:45:00 crc kubenswrapper[5052]: I1203 11:45:00.169496 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="f8e49a02-823b-4851-9ecf-f2d1696fa201" containerName="placement-log" Dec 03 11:45:00 crc kubenswrapper[5052]: E1203 11:45:00.169594 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="23b2a11f-0129-474b-9f97-f9401abd3f6b" containerName="barbican-api-log" Dec 03 11:45:00 crc kubenswrapper[5052]: I1203 11:45:00.169604 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="23b2a11f-0129-474b-9f97-f9401abd3f6b" containerName="barbican-api-log" Dec 03 11:45:00 crc kubenswrapper[5052]: E1203 11:45:00.169613 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="f8e49a02-823b-4851-9ecf-f2d1696fa201" containerName="placement-api" Dec 03 11:45:00 crc kubenswrapper[5052]: I1203 11:45:00.169620 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="f8e49a02-823b-4851-9ecf-f2d1696fa201" containerName="placement-api" Dec 03 11:45:00 crc kubenswrapper[5052]: E1203 11:45:00.169661 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="f8e49a02-823b-4851-9ecf-f2d1696fa201" containerName="placement-api" Dec 03 11:45:00 crc kubenswrapper[5052]: I1203 11:45:00.169668 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="f8e49a02-823b-4851-9ecf-f2d1696fa201" containerName="placement-api" Dec 03 11:45:00 crc kubenswrapper[5052]: E1203 11:45:00.169676 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="23b2a11f-0129-474b-9f97-f9401abd3f6b" containerName="barbican-api" Dec 03 11:45:00 crc kubenswrapper[5052]: I1203 11:45:00.169683 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="23b2a11f-0129-474b-9f97-f9401abd3f6b" containerName="barbican-api" Dec 03 11:45:00 crc kubenswrapper[5052]: I1203 11:45:00.169908 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="f8e49a02-823b-4851-9ecf-f2d1696fa201" containerName="placement-api" Dec 03 11:45:00 crc kubenswrapper[5052]: I1203 11:45:00.169920 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="f8e49a02-823b-4851-9ecf-f2d1696fa201" containerName="placement-log" Dec 03 11:45:00 crc kubenswrapper[5052]: I1203 11:45:00.169934 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="23b2a11f-0129-474b-9f97-f9401abd3f6b" containerName="barbican-api" Dec 03 11:45:00 crc kubenswrapper[5052]: I1203 11:45:00.169951 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="23b2a11f-0129-474b-9f97-f9401abd3f6b" containerName="barbican-api-log" Dec 03 11:45:00 crc kubenswrapper[5052]: I1203 11:45:00.169959 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="f8e49a02-823b-4851-9ecf-f2d1696fa201" containerName="placement-api" Dec 03 11:45:00 crc kubenswrapper[5052]: I1203 11:45:00.170715 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-operator-lifecycle-manager/collect-profiles-29412705-7rvq9" Dec 03 11:45:00 crc kubenswrapper[5052]: I1203 11:45:00.174154 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-operator-lifecycle-manager"/"collect-profiles-config" Dec 03 11:45:00 crc kubenswrapper[5052]: I1203 11:45:00.175012 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-operator-lifecycle-manager"/"collect-profiles-dockercfg-kzf4t" Dec 03 11:45:00 crc kubenswrapper[5052]: I1203 11:45:00.178494 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-operator-lifecycle-manager/collect-profiles-29412705-7rvq9"] Dec 03 11:45:00 crc kubenswrapper[5052]: I1203 11:45:00.202954 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-volume\" (UniqueName: \"kubernetes.io/secret/323fd029-104c-4804-8829-3ed690c6c15b-secret-volume\") pod \"collect-profiles-29412705-7rvq9\" (UID: \"323fd029-104c-4804-8829-3ed690c6c15b\") " pod="openshift-operator-lifecycle-manager/collect-profiles-29412705-7rvq9" Dec 03 11:45:00 crc kubenswrapper[5052]: I1203 11:45:00.202998 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jrrxm\" (UniqueName: \"kubernetes.io/projected/323fd029-104c-4804-8829-3ed690c6c15b-kube-api-access-jrrxm\") pod \"collect-profiles-29412705-7rvq9\" (UID: \"323fd029-104c-4804-8829-3ed690c6c15b\") " pod="openshift-operator-lifecycle-manager/collect-profiles-29412705-7rvq9" Dec 03 11:45:00 crc kubenswrapper[5052]: I1203 11:45:00.203022 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/323fd029-104c-4804-8829-3ed690c6c15b-config-volume\") pod \"collect-profiles-29412705-7rvq9\" (UID: \"323fd029-104c-4804-8829-3ed690c6c15b\") " pod="openshift-operator-lifecycle-manager/collect-profiles-29412705-7rvq9" Dec 03 11:45:00 crc kubenswrapper[5052]: I1203 11:45:00.304994 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/323fd029-104c-4804-8829-3ed690c6c15b-config-volume\") pod \"collect-profiles-29412705-7rvq9\" (UID: \"323fd029-104c-4804-8829-3ed690c6c15b\") " pod="openshift-operator-lifecycle-manager/collect-profiles-29412705-7rvq9" Dec 03 11:45:00 crc kubenswrapper[5052]: I1203 11:45:00.305198 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"secret-volume\" (UniqueName: \"kubernetes.io/secret/323fd029-104c-4804-8829-3ed690c6c15b-secret-volume\") pod \"collect-profiles-29412705-7rvq9\" (UID: \"323fd029-104c-4804-8829-3ed690c6c15b\") " pod="openshift-operator-lifecycle-manager/collect-profiles-29412705-7rvq9" Dec 03 11:45:00 crc kubenswrapper[5052]: I1203 11:45:00.305232 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-jrrxm\" (UniqueName: \"kubernetes.io/projected/323fd029-104c-4804-8829-3ed690c6c15b-kube-api-access-jrrxm\") pod \"collect-profiles-29412705-7rvq9\" (UID: \"323fd029-104c-4804-8829-3ed690c6c15b\") " pod="openshift-operator-lifecycle-manager/collect-profiles-29412705-7rvq9" Dec 03 11:45:00 crc kubenswrapper[5052]: I1203 11:45:00.306568 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/323fd029-104c-4804-8829-3ed690c6c15b-config-volume\") pod \"collect-profiles-29412705-7rvq9\" (UID: \"323fd029-104c-4804-8829-3ed690c6c15b\") " pod="openshift-operator-lifecycle-manager/collect-profiles-29412705-7rvq9" Dec 03 11:45:00 crc kubenswrapper[5052]: I1203 11:45:00.314178 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"secret-volume\" (UniqueName: \"kubernetes.io/secret/323fd029-104c-4804-8829-3ed690c6c15b-secret-volume\") pod \"collect-profiles-29412705-7rvq9\" (UID: \"323fd029-104c-4804-8829-3ed690c6c15b\") " pod="openshift-operator-lifecycle-manager/collect-profiles-29412705-7rvq9" Dec 03 11:45:00 crc kubenswrapper[5052]: I1203 11:45:00.325880 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-jrrxm\" (UniqueName: \"kubernetes.io/projected/323fd029-104c-4804-8829-3ed690c6c15b-kube-api-access-jrrxm\") pod \"collect-profiles-29412705-7rvq9\" (UID: \"323fd029-104c-4804-8829-3ed690c6c15b\") " pod="openshift-operator-lifecycle-manager/collect-profiles-29412705-7rvq9" Dec 03 11:45:00 crc kubenswrapper[5052]: I1203 11:45:00.531902 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-operator-lifecycle-manager/collect-profiles-29412705-7rvq9" Dec 03 11:45:00 crc kubenswrapper[5052]: I1203 11:45:00.535689 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/certified-operators-7wd6v" Dec 03 11:45:00 crc kubenswrapper[5052]: I1203 11:45:00.608763 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/fb0eab4c-4728-4b3f-9df0-70bd099cb8c9-utilities\") pod \"fb0eab4c-4728-4b3f-9df0-70bd099cb8c9\" (UID: \"fb0eab4c-4728-4b3f-9df0-70bd099cb8c9\") " Dec 03 11:45:00 crc kubenswrapper[5052]: I1203 11:45:00.608920 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/fb0eab4c-4728-4b3f-9df0-70bd099cb8c9-catalog-content\") pod \"fb0eab4c-4728-4b3f-9df0-70bd099cb8c9\" (UID: \"fb0eab4c-4728-4b3f-9df0-70bd099cb8c9\") " Dec 03 11:45:00 crc kubenswrapper[5052]: I1203 11:45:00.608981 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6zcn4\" (UniqueName: \"kubernetes.io/projected/fb0eab4c-4728-4b3f-9df0-70bd099cb8c9-kube-api-access-6zcn4\") pod \"fb0eab4c-4728-4b3f-9df0-70bd099cb8c9\" (UID: \"fb0eab4c-4728-4b3f-9df0-70bd099cb8c9\") " Dec 03 11:45:00 crc kubenswrapper[5052]: I1203 11:45:00.613333 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb0eab4c-4728-4b3f-9df0-70bd099cb8c9-utilities" (OuterVolumeSpecName: "utilities") pod "fb0eab4c-4728-4b3f-9df0-70bd099cb8c9" (UID: "fb0eab4c-4728-4b3f-9df0-70bd099cb8c9"). InnerVolumeSpecName "utilities". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:45:00 crc kubenswrapper[5052]: I1203 11:45:00.631742 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fb0eab4c-4728-4b3f-9df0-70bd099cb8c9-kube-api-access-6zcn4" (OuterVolumeSpecName: "kube-api-access-6zcn4") pod "fb0eab4c-4728-4b3f-9df0-70bd099cb8c9" (UID: "fb0eab4c-4728-4b3f-9df0-70bd099cb8c9"). InnerVolumeSpecName "kube-api-access-6zcn4". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:45:00 crc kubenswrapper[5052]: I1203 11:45:00.694277 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb0eab4c-4728-4b3f-9df0-70bd099cb8c9-catalog-content" (OuterVolumeSpecName: "catalog-content") pod "fb0eab4c-4728-4b3f-9df0-70bd099cb8c9" (UID: "fb0eab4c-4728-4b3f-9df0-70bd099cb8c9"). InnerVolumeSpecName "catalog-content". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:45:00 crc kubenswrapper[5052]: I1203 11:45:00.713678 5052 reconciler_common.go:293] "Volume detached for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/fb0eab4c-4728-4b3f-9df0-70bd099cb8c9-catalog-content\") on node \"crc\" DevicePath \"\"" Dec 03 11:45:00 crc kubenswrapper[5052]: I1203 11:45:00.713720 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-6zcn4\" (UniqueName: \"kubernetes.io/projected/fb0eab4c-4728-4b3f-9df0-70bd099cb8c9-kube-api-access-6zcn4\") on node \"crc\" DevicePath \"\"" Dec 03 11:45:00 crc kubenswrapper[5052]: I1203 11:45:00.713733 5052 reconciler_common.go:293] "Volume detached for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/fb0eab4c-4728-4b3f-9df0-70bd099cb8c9-utilities\") on node \"crc\" DevicePath \"\"" Dec 03 11:45:00 crc kubenswrapper[5052]: I1203 11:45:00.882920 5052 generic.go:334] "Generic (PLEG): container finished" podID="fb0eab4c-4728-4b3f-9df0-70bd099cb8c9" containerID="e9e06f4cae7c528e5ad942a4d6fd8849b4e5b0d0000bfe3132ef7697b540b97b" exitCode=0 Dec 03 11:45:00 crc kubenswrapper[5052]: I1203 11:45:00.882963 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/certified-operators-7wd6v" event={"ID":"fb0eab4c-4728-4b3f-9df0-70bd099cb8c9","Type":"ContainerDied","Data":"e9e06f4cae7c528e5ad942a4d6fd8849b4e5b0d0000bfe3132ef7697b540b97b"} Dec 03 11:45:00 crc kubenswrapper[5052]: I1203 11:45:00.882995 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/certified-operators-7wd6v" event={"ID":"fb0eab4c-4728-4b3f-9df0-70bd099cb8c9","Type":"ContainerDied","Data":"8137e47f9747a51ee4e1a48fa4930b93a7adf74fa53a8f3a4f8fcb986e0447f0"} Dec 03 11:45:00 crc kubenswrapper[5052]: I1203 11:45:00.883015 5052 scope.go:117] "RemoveContainer" containerID="e9e06f4cae7c528e5ad942a4d6fd8849b4e5b0d0000bfe3132ef7697b540b97b" Dec 03 11:45:00 crc kubenswrapper[5052]: I1203 11:45:00.883186 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/certified-operators-7wd6v" Dec 03 11:45:00 crc kubenswrapper[5052]: I1203 11:45:00.925055 5052 scope.go:117] "RemoveContainer" containerID="19b85075f2eda47e6aec0c9055473935cf58cfcdbf1f8ce63045e089415617fe" Dec 03 11:45:00 crc kubenswrapper[5052]: I1203 11:45:00.929871 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openshift-marketplace/certified-operators-7wd6v"] Dec 03 11:45:00 crc kubenswrapper[5052]: I1203 11:45:00.939432 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openshift-marketplace/certified-operators-7wd6v"] Dec 03 11:45:00 crc kubenswrapper[5052]: I1203 11:45:00.946794 5052 scope.go:117] "RemoveContainer" containerID="205156a1a270a68d4c328e186a20aed900af4b6ef2b3b532a3cae8e5047b5110" Dec 03 11:45:00 crc kubenswrapper[5052]: I1203 11:45:00.984393 5052 scope.go:117] "RemoveContainer" containerID="e9e06f4cae7c528e5ad942a4d6fd8849b4e5b0d0000bfe3132ef7697b540b97b" Dec 03 11:45:00 crc kubenswrapper[5052]: E1203 11:45:00.985088 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e9e06f4cae7c528e5ad942a4d6fd8849b4e5b0d0000bfe3132ef7697b540b97b\": container with ID starting with e9e06f4cae7c528e5ad942a4d6fd8849b4e5b0d0000bfe3132ef7697b540b97b not found: ID does not exist" containerID="e9e06f4cae7c528e5ad942a4d6fd8849b4e5b0d0000bfe3132ef7697b540b97b" Dec 03 11:45:00 crc kubenswrapper[5052]: I1203 11:45:00.985119 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e9e06f4cae7c528e5ad942a4d6fd8849b4e5b0d0000bfe3132ef7697b540b97b"} err="failed to get container status \"e9e06f4cae7c528e5ad942a4d6fd8849b4e5b0d0000bfe3132ef7697b540b97b\": rpc error: code = NotFound desc = could not find container \"e9e06f4cae7c528e5ad942a4d6fd8849b4e5b0d0000bfe3132ef7697b540b97b\": container with ID starting with e9e06f4cae7c528e5ad942a4d6fd8849b4e5b0d0000bfe3132ef7697b540b97b not found: ID does not exist" Dec 03 11:45:00 crc kubenswrapper[5052]: I1203 11:45:00.985139 5052 scope.go:117] "RemoveContainer" containerID="19b85075f2eda47e6aec0c9055473935cf58cfcdbf1f8ce63045e089415617fe" Dec 03 11:45:00 crc kubenswrapper[5052]: E1203 11:45:00.985722 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"19b85075f2eda47e6aec0c9055473935cf58cfcdbf1f8ce63045e089415617fe\": container with ID starting with 19b85075f2eda47e6aec0c9055473935cf58cfcdbf1f8ce63045e089415617fe not found: ID does not exist" containerID="19b85075f2eda47e6aec0c9055473935cf58cfcdbf1f8ce63045e089415617fe" Dec 03 11:45:00 crc kubenswrapper[5052]: I1203 11:45:00.985771 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"19b85075f2eda47e6aec0c9055473935cf58cfcdbf1f8ce63045e089415617fe"} err="failed to get container status \"19b85075f2eda47e6aec0c9055473935cf58cfcdbf1f8ce63045e089415617fe\": rpc error: code = NotFound desc = could not find container \"19b85075f2eda47e6aec0c9055473935cf58cfcdbf1f8ce63045e089415617fe\": container with ID starting with 19b85075f2eda47e6aec0c9055473935cf58cfcdbf1f8ce63045e089415617fe not found: ID does not exist" Dec 03 11:45:00 crc kubenswrapper[5052]: I1203 11:45:00.985800 5052 scope.go:117] "RemoveContainer" containerID="205156a1a270a68d4c328e186a20aed900af4b6ef2b3b532a3cae8e5047b5110" Dec 03 11:45:00 crc kubenswrapper[5052]: E1203 11:45:00.986249 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"205156a1a270a68d4c328e186a20aed900af4b6ef2b3b532a3cae8e5047b5110\": container with ID starting with 205156a1a270a68d4c328e186a20aed900af4b6ef2b3b532a3cae8e5047b5110 not found: ID does not exist" containerID="205156a1a270a68d4c328e186a20aed900af4b6ef2b3b532a3cae8e5047b5110" Dec 03 11:45:00 crc kubenswrapper[5052]: I1203 11:45:00.986282 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"205156a1a270a68d4c328e186a20aed900af4b6ef2b3b532a3cae8e5047b5110"} err="failed to get container status \"205156a1a270a68d4c328e186a20aed900af4b6ef2b3b532a3cae8e5047b5110\": rpc error: code = NotFound desc = could not find container \"205156a1a270a68d4c328e186a20aed900af4b6ef2b3b532a3cae8e5047b5110\": container with ID starting with 205156a1a270a68d4c328e186a20aed900af4b6ef2b3b532a3cae8e5047b5110 not found: ID does not exist" Dec 03 11:45:01 crc kubenswrapper[5052]: I1203 11:45:01.092829 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-operator-lifecycle-manager/collect-profiles-29412705-7rvq9"] Dec 03 11:45:01 crc kubenswrapper[5052]: I1203 11:45:01.686072 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fb0eab4c-4728-4b3f-9df0-70bd099cb8c9" path="/var/lib/kubelet/pods/fb0eab4c-4728-4b3f-9df0-70bd099cb8c9/volumes" Dec 03 11:45:01 crc kubenswrapper[5052]: I1203 11:45:01.896478 5052 generic.go:334] "Generic (PLEG): container finished" podID="323fd029-104c-4804-8829-3ed690c6c15b" containerID="698051bad98a7216c5799f152665bdd1626301fe4b20746cd84d12b4443a0156" exitCode=0 Dec 03 11:45:01 crc kubenswrapper[5052]: I1203 11:45:01.896535 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-operator-lifecycle-manager/collect-profiles-29412705-7rvq9" event={"ID":"323fd029-104c-4804-8829-3ed690c6c15b","Type":"ContainerDied","Data":"698051bad98a7216c5799f152665bdd1626301fe4b20746cd84d12b4443a0156"} Dec 03 11:45:01 crc kubenswrapper[5052]: I1203 11:45:01.896558 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-operator-lifecycle-manager/collect-profiles-29412705-7rvq9" event={"ID":"323fd029-104c-4804-8829-3ed690c6c15b","Type":"ContainerStarted","Data":"7409bcb8a927b34aa31ed7c1caf7c38c8f7658e24e336f5c7a2fe7d2dbce5f89"} Dec 03 11:45:03 crc kubenswrapper[5052]: I1203 11:45:03.286558 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-operator-lifecycle-manager/collect-profiles-29412705-7rvq9" Dec 03 11:45:03 crc kubenswrapper[5052]: I1203 11:45:03.360058 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/323fd029-104c-4804-8829-3ed690c6c15b-config-volume\") pod \"323fd029-104c-4804-8829-3ed690c6c15b\" (UID: \"323fd029-104c-4804-8829-3ed690c6c15b\") " Dec 03 11:45:03 crc kubenswrapper[5052]: I1203 11:45:03.360105 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"secret-volume\" (UniqueName: \"kubernetes.io/secret/323fd029-104c-4804-8829-3ed690c6c15b-secret-volume\") pod \"323fd029-104c-4804-8829-3ed690c6c15b\" (UID: \"323fd029-104c-4804-8829-3ed690c6c15b\") " Dec 03 11:45:03 crc kubenswrapper[5052]: I1203 11:45:03.360129 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jrrxm\" (UniqueName: \"kubernetes.io/projected/323fd029-104c-4804-8829-3ed690c6c15b-kube-api-access-jrrxm\") pod \"323fd029-104c-4804-8829-3ed690c6c15b\" (UID: \"323fd029-104c-4804-8829-3ed690c6c15b\") " Dec 03 11:45:03 crc kubenswrapper[5052]: I1203 11:45:03.361083 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/323fd029-104c-4804-8829-3ed690c6c15b-config-volume" (OuterVolumeSpecName: "config-volume") pod "323fd029-104c-4804-8829-3ed690c6c15b" (UID: "323fd029-104c-4804-8829-3ed690c6c15b"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:45:03 crc kubenswrapper[5052]: I1203 11:45:03.366006 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/323fd029-104c-4804-8829-3ed690c6c15b-secret-volume" (OuterVolumeSpecName: "secret-volume") pod "323fd029-104c-4804-8829-3ed690c6c15b" (UID: "323fd029-104c-4804-8829-3ed690c6c15b"). InnerVolumeSpecName "secret-volume". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:45:03 crc kubenswrapper[5052]: I1203 11:45:03.379779 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/323fd029-104c-4804-8829-3ed690c6c15b-kube-api-access-jrrxm" (OuterVolumeSpecName: "kube-api-access-jrrxm") pod "323fd029-104c-4804-8829-3ed690c6c15b" (UID: "323fd029-104c-4804-8829-3ed690c6c15b"). InnerVolumeSpecName "kube-api-access-jrrxm". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:45:03 crc kubenswrapper[5052]: I1203 11:45:03.462479 5052 reconciler_common.go:293] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/323fd029-104c-4804-8829-3ed690c6c15b-config-volume\") on node \"crc\" DevicePath \"\"" Dec 03 11:45:03 crc kubenswrapper[5052]: I1203 11:45:03.462541 5052 reconciler_common.go:293] "Volume detached for volume \"secret-volume\" (UniqueName: \"kubernetes.io/secret/323fd029-104c-4804-8829-3ed690c6c15b-secret-volume\") on node \"crc\" DevicePath \"\"" Dec 03 11:45:03 crc kubenswrapper[5052]: I1203 11:45:03.462551 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-jrrxm\" (UniqueName: \"kubernetes.io/projected/323fd029-104c-4804-8829-3ed690c6c15b-kube-api-access-jrrxm\") on node \"crc\" DevicePath \"\"" Dec 03 11:45:03 crc kubenswrapper[5052]: I1203 11:45:03.649542 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/keystone-779b679dc7-nvtdm" Dec 03 11:45:03 crc kubenswrapper[5052]: I1203 11:45:03.737669 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/neutron-549dfc49dd-qtk5n" Dec 03 11:45:03 crc kubenswrapper[5052]: I1203 11:45:03.816238 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/neutron-7cc99c958f-bjn2l"] Dec 03 11:45:03 crc kubenswrapper[5052]: I1203 11:45:03.816549 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/neutron-7cc99c958f-bjn2l" podUID="2dbe7002-eb09-4a0f-b65b-fc0294c13f4f" containerName="neutron-api" containerID="cri-o://8c2a5717a95924a833c522bb45a7070432896bb7e0e5d55da6ef352206e5b20a" gracePeriod=30 Dec 03 11:45:03 crc kubenswrapper[5052]: I1203 11:45:03.816684 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/neutron-7cc99c958f-bjn2l" podUID="2dbe7002-eb09-4a0f-b65b-fc0294c13f4f" containerName="neutron-httpd" containerID="cri-o://a60bd070c0bbd29b273ad38f5c6a3450100e8c13d43d5b00488a10b98645de23" gracePeriod=30 Dec 03 11:45:03 crc kubenswrapper[5052]: I1203 11:45:03.916365 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-operator-lifecycle-manager/collect-profiles-29412705-7rvq9" event={"ID":"323fd029-104c-4804-8829-3ed690c6c15b","Type":"ContainerDied","Data":"7409bcb8a927b34aa31ed7c1caf7c38c8f7658e24e336f5c7a2fe7d2dbce5f89"} Dec 03 11:45:03 crc kubenswrapper[5052]: I1203 11:45:03.916417 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7409bcb8a927b34aa31ed7c1caf7c38c8f7658e24e336f5c7a2fe7d2dbce5f89" Dec 03 11:45:03 crc kubenswrapper[5052]: I1203 11:45:03.916483 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-operator-lifecycle-manager/collect-profiles-29412705-7rvq9" Dec 03 11:45:04 crc kubenswrapper[5052]: I1203 11:45:04.925205 5052 generic.go:334] "Generic (PLEG): container finished" podID="2dbe7002-eb09-4a0f-b65b-fc0294c13f4f" containerID="a60bd070c0bbd29b273ad38f5c6a3450100e8c13d43d5b00488a10b98645de23" exitCode=0 Dec 03 11:45:04 crc kubenswrapper[5052]: I1203 11:45:04.925297 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-7cc99c958f-bjn2l" event={"ID":"2dbe7002-eb09-4a0f-b65b-fc0294c13f4f","Type":"ContainerDied","Data":"a60bd070c0bbd29b273ad38f5c6a3450100e8c13d43d5b00488a10b98645de23"} Dec 03 11:45:05 crc kubenswrapper[5052]: I1203 11:45:05.026425 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/openstackclient"] Dec 03 11:45:05 crc kubenswrapper[5052]: E1203 11:45:05.026876 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="fb0eab4c-4728-4b3f-9df0-70bd099cb8c9" containerName="extract-utilities" Dec 03 11:45:05 crc kubenswrapper[5052]: I1203 11:45:05.026899 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb0eab4c-4728-4b3f-9df0-70bd099cb8c9" containerName="extract-utilities" Dec 03 11:45:05 crc kubenswrapper[5052]: E1203 11:45:05.026928 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="fb0eab4c-4728-4b3f-9df0-70bd099cb8c9" containerName="registry-server" Dec 03 11:45:05 crc kubenswrapper[5052]: I1203 11:45:05.026934 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb0eab4c-4728-4b3f-9df0-70bd099cb8c9" containerName="registry-server" Dec 03 11:45:05 crc kubenswrapper[5052]: E1203 11:45:05.026947 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="fb0eab4c-4728-4b3f-9df0-70bd099cb8c9" containerName="extract-content" Dec 03 11:45:05 crc kubenswrapper[5052]: I1203 11:45:05.026953 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb0eab4c-4728-4b3f-9df0-70bd099cb8c9" containerName="extract-content" Dec 03 11:45:05 crc kubenswrapper[5052]: E1203 11:45:05.026963 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="323fd029-104c-4804-8829-3ed690c6c15b" containerName="collect-profiles" Dec 03 11:45:05 crc kubenswrapper[5052]: I1203 11:45:05.026969 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="323fd029-104c-4804-8829-3ed690c6c15b" containerName="collect-profiles" Dec 03 11:45:05 crc kubenswrapper[5052]: E1203 11:45:05.026982 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="f8e49a02-823b-4851-9ecf-f2d1696fa201" containerName="placement-api" Dec 03 11:45:05 crc kubenswrapper[5052]: I1203 11:45:05.026988 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="f8e49a02-823b-4851-9ecf-f2d1696fa201" containerName="placement-api" Dec 03 11:45:05 crc kubenswrapper[5052]: I1203 11:45:05.027170 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="fb0eab4c-4728-4b3f-9df0-70bd099cb8c9" containerName="registry-server" Dec 03 11:45:05 crc kubenswrapper[5052]: I1203 11:45:05.027183 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="f8e49a02-823b-4851-9ecf-f2d1696fa201" containerName="placement-api" Dec 03 11:45:05 crc kubenswrapper[5052]: I1203 11:45:05.027195 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="323fd029-104c-4804-8829-3ed690c6c15b" containerName="collect-profiles" Dec 03 11:45:05 crc kubenswrapper[5052]: I1203 11:45:05.027870 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstackclient" Dec 03 11:45:05 crc kubenswrapper[5052]: I1203 11:45:05.036680 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/openstackclient"] Dec 03 11:45:05 crc kubenswrapper[5052]: I1203 11:45:05.040042 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"openstackclient-openstackclient-dockercfg-slh9g" Dec 03 11:45:05 crc kubenswrapper[5052]: I1203 11:45:05.040042 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"openstack-config" Dec 03 11:45:05 crc kubenswrapper[5052]: I1203 11:45:05.040053 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"openstack-config-secret" Dec 03 11:45:05 crc kubenswrapper[5052]: I1203 11:45:05.059743 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/openstackclient"] Dec 03 11:45:05 crc kubenswrapper[5052]: E1203 11:45:05.060572 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[combined-ca-bundle kube-api-access-b7pw6 openstack-config openstack-config-secret], unattached volumes=[], failed to process volumes=[combined-ca-bundle kube-api-access-b7pw6 openstack-config openstack-config-secret]: context canceled" pod="openstack-kuttl-tests/openstackclient" podUID="b2f646d7-a0d6-410d-a6bc-1d64593ed7ad" Dec 03 11:45:05 crc kubenswrapper[5052]: I1203 11:45:05.067085 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/openstackclient"] Dec 03 11:45:05 crc kubenswrapper[5052]: I1203 11:45:05.107630 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/openstackclient"] Dec 03 11:45:05 crc kubenswrapper[5052]: I1203 11:45:05.108774 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstackclient" Dec 03 11:45:05 crc kubenswrapper[5052]: I1203 11:45:05.116948 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/openstackclient"] Dec 03 11:45:05 crc kubenswrapper[5052]: I1203 11:45:05.196835 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a9f103a5-8b29-4152-a711-60891b9aeefc-combined-ca-bundle\") pod \"openstackclient\" (UID: \"a9f103a5-8b29-4152-a711-60891b9aeefc\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 11:45:05 crc kubenswrapper[5052]: I1203 11:45:05.196885 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gmd9s\" (UniqueName: \"kubernetes.io/projected/a9f103a5-8b29-4152-a711-60891b9aeefc-kube-api-access-gmd9s\") pod \"openstackclient\" (UID: \"a9f103a5-8b29-4152-a711-60891b9aeefc\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 11:45:05 crc kubenswrapper[5052]: I1203 11:45:05.197040 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openstack-config\" (UniqueName: \"kubernetes.io/configmap/a9f103a5-8b29-4152-a711-60891b9aeefc-openstack-config\") pod \"openstackclient\" (UID: \"a9f103a5-8b29-4152-a711-60891b9aeefc\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 11:45:05 crc kubenswrapper[5052]: I1203 11:45:05.197083 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openstack-config-secret\" (UniqueName: \"kubernetes.io/secret/a9f103a5-8b29-4152-a711-60891b9aeefc-openstack-config-secret\") pod \"openstackclient\" (UID: \"a9f103a5-8b29-4152-a711-60891b9aeefc\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 11:45:05 crc kubenswrapper[5052]: I1203 11:45:05.298207 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"openstack-config-secret\" (UniqueName: \"kubernetes.io/secret/a9f103a5-8b29-4152-a711-60891b9aeefc-openstack-config-secret\") pod \"openstackclient\" (UID: \"a9f103a5-8b29-4152-a711-60891b9aeefc\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 11:45:05 crc kubenswrapper[5052]: I1203 11:45:05.298313 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a9f103a5-8b29-4152-a711-60891b9aeefc-combined-ca-bundle\") pod \"openstackclient\" (UID: \"a9f103a5-8b29-4152-a711-60891b9aeefc\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 11:45:05 crc kubenswrapper[5052]: I1203 11:45:05.298348 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-gmd9s\" (UniqueName: \"kubernetes.io/projected/a9f103a5-8b29-4152-a711-60891b9aeefc-kube-api-access-gmd9s\") pod \"openstackclient\" (UID: \"a9f103a5-8b29-4152-a711-60891b9aeefc\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 11:45:05 crc kubenswrapper[5052]: I1203 11:45:05.298442 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"openstack-config\" (UniqueName: \"kubernetes.io/configmap/a9f103a5-8b29-4152-a711-60891b9aeefc-openstack-config\") pod \"openstackclient\" (UID: \"a9f103a5-8b29-4152-a711-60891b9aeefc\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 11:45:05 crc kubenswrapper[5052]: I1203 11:45:05.299468 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"openstack-config\" (UniqueName: \"kubernetes.io/configmap/a9f103a5-8b29-4152-a711-60891b9aeefc-openstack-config\") pod \"openstackclient\" (UID: \"a9f103a5-8b29-4152-a711-60891b9aeefc\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 11:45:05 crc kubenswrapper[5052]: I1203 11:45:05.304982 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"openstack-config-secret\" (UniqueName: \"kubernetes.io/secret/a9f103a5-8b29-4152-a711-60891b9aeefc-openstack-config-secret\") pod \"openstackclient\" (UID: \"a9f103a5-8b29-4152-a711-60891b9aeefc\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 11:45:05 crc kubenswrapper[5052]: I1203 11:45:05.305171 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a9f103a5-8b29-4152-a711-60891b9aeefc-combined-ca-bundle\") pod \"openstackclient\" (UID: \"a9f103a5-8b29-4152-a711-60891b9aeefc\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 11:45:05 crc kubenswrapper[5052]: I1203 11:45:05.317233 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-gmd9s\" (UniqueName: \"kubernetes.io/projected/a9f103a5-8b29-4152-a711-60891b9aeefc-kube-api-access-gmd9s\") pod \"openstackclient\" (UID: \"a9f103a5-8b29-4152-a711-60891b9aeefc\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 11:45:05 crc kubenswrapper[5052]: I1203 11:45:05.436068 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstackclient" Dec 03 11:45:05 crc kubenswrapper[5052]: I1203 11:45:05.687959 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b2f646d7-a0d6-410d-a6bc-1d64593ed7ad" path="/var/lib/kubelet/pods/b2f646d7-a0d6-410d-a6bc-1d64593ed7ad/volumes" Dec 03 11:45:05 crc kubenswrapper[5052]: I1203 11:45:05.725801 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/openstackclient"] Dec 03 11:45:05 crc kubenswrapper[5052]: I1203 11:45:05.935732 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstackclient" Dec 03 11:45:05 crc kubenswrapper[5052]: I1203 11:45:05.936728 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstackclient" event={"ID":"a9f103a5-8b29-4152-a711-60891b9aeefc","Type":"ContainerStarted","Data":"ea35f3fa8909508bae00d8c488263cf9c60deaa8b3ac80535fc0b4335a978888"} Dec 03 11:45:05 crc kubenswrapper[5052]: I1203 11:45:05.952295 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstackclient" Dec 03 11:45:05 crc kubenswrapper[5052]: I1203 11:45:05.955678 5052 status_manager.go:861] "Pod was deleted and then recreated, skipping status update" pod="openstack-kuttl-tests/openstackclient" oldPodUID="b2f646d7-a0d6-410d-a6bc-1d64593ed7ad" podUID="a9f103a5-8b29-4152-a711-60891b9aeefc" Dec 03 11:45:06 crc kubenswrapper[5052]: I1203 11:45:06.946336 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstackclient" Dec 03 11:45:06 crc kubenswrapper[5052]: I1203 11:45:06.947175 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstackclient" event={"ID":"a9f103a5-8b29-4152-a711-60891b9aeefc","Type":"ContainerStarted","Data":"6b2a059106f1fcc4b1e035c9cf505bb5bc72d3f6e38d77875ea0e33ccbc713f1"} Dec 03 11:45:06 crc kubenswrapper[5052]: I1203 11:45:06.968106 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/openstackclient" podStartSLOduration=1.968083663 podStartE2EDuration="1.968083663s" podCreationTimestamp="2025-12-03 11:45:05 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:45:06.962878193 +0000 UTC m=+1497.551119977" watchObservedRunningTime="2025-12-03 11:45:06.968083663 +0000 UTC m=+1497.556325427" Dec 03 11:45:06 crc kubenswrapper[5052]: I1203 11:45:06.980126 5052 status_manager.go:861] "Pod was deleted and then recreated, skipping status update" pod="openstack-kuttl-tests/openstackclient" oldPodUID="b2f646d7-a0d6-410d-a6bc-1d64593ed7ad" podUID="a9f103a5-8b29-4152-a711-60891b9aeefc" Dec 03 11:45:07 crc kubenswrapper[5052]: I1203 11:45:07.945134 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:45:07 crc kubenswrapper[5052]: I1203 11:45:07.945457 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="b6eeb188-0ea6-46de-8cbf-d803e75df5e6" containerName="ceilometer-central-agent" containerID="cri-o://df00125cee9eca43a7001e16f293bd9269f75ced8e07fe1609749fa6ec555248" gracePeriod=30 Dec 03 11:45:07 crc kubenswrapper[5052]: I1203 11:45:07.945640 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="b6eeb188-0ea6-46de-8cbf-d803e75df5e6" containerName="proxy-httpd" containerID="cri-o://4605df729b344cdc8aa272fc21b35ff9f030fbd19d68d7fbb10d033f9171882f" gracePeriod=30 Dec 03 11:45:07 crc kubenswrapper[5052]: I1203 11:45:07.945701 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="b6eeb188-0ea6-46de-8cbf-d803e75df5e6" containerName="sg-core" containerID="cri-o://ba76522ceb5c6ff71560ef00ae164eb9deadf017edb824ce2c749304302a9186" gracePeriod=30 Dec 03 11:45:07 crc kubenswrapper[5052]: I1203 11:45:07.945749 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="b6eeb188-0ea6-46de-8cbf-d803e75df5e6" containerName="ceilometer-notification-agent" containerID="cri-o://a543e5d6e1f5b544fd17dc9d996401e2195186e8556773c54e76285b9c705f51" gracePeriod=30 Dec 03 11:45:08 crc kubenswrapper[5052]: I1203 11:45:08.970859 5052 generic.go:334] "Generic (PLEG): container finished" podID="b6eeb188-0ea6-46de-8cbf-d803e75df5e6" containerID="4605df729b344cdc8aa272fc21b35ff9f030fbd19d68d7fbb10d033f9171882f" exitCode=0 Dec 03 11:45:08 crc kubenswrapper[5052]: I1203 11:45:08.970888 5052 generic.go:334] "Generic (PLEG): container finished" podID="b6eeb188-0ea6-46de-8cbf-d803e75df5e6" containerID="ba76522ceb5c6ff71560ef00ae164eb9deadf017edb824ce2c749304302a9186" exitCode=2 Dec 03 11:45:08 crc kubenswrapper[5052]: I1203 11:45:08.970898 5052 generic.go:334] "Generic (PLEG): container finished" podID="b6eeb188-0ea6-46de-8cbf-d803e75df5e6" containerID="df00125cee9eca43a7001e16f293bd9269f75ced8e07fe1609749fa6ec555248" exitCode=0 Dec 03 11:45:08 crc kubenswrapper[5052]: I1203 11:45:08.970916 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"b6eeb188-0ea6-46de-8cbf-d803e75df5e6","Type":"ContainerDied","Data":"4605df729b344cdc8aa272fc21b35ff9f030fbd19d68d7fbb10d033f9171882f"} Dec 03 11:45:08 crc kubenswrapper[5052]: I1203 11:45:08.970942 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"b6eeb188-0ea6-46de-8cbf-d803e75df5e6","Type":"ContainerDied","Data":"ba76522ceb5c6ff71560ef00ae164eb9deadf017edb824ce2c749304302a9186"} Dec 03 11:45:08 crc kubenswrapper[5052]: I1203 11:45:08.970951 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"b6eeb188-0ea6-46de-8cbf-d803e75df5e6","Type":"ContainerDied","Data":"df00125cee9eca43a7001e16f293bd9269f75ced8e07fe1609749fa6ec555248"} Dec 03 11:45:09 crc kubenswrapper[5052]: I1203 11:45:09.082543 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/swift-proxy-6fdf66df4-gsqlq"] Dec 03 11:45:09 crc kubenswrapper[5052]: I1203 11:45:09.083897 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-proxy-6fdf66df4-gsqlq" Dec 03 11:45:09 crc kubenswrapper[5052]: I1203 11:45:09.088369 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-swift-internal-svc" Dec 03 11:45:09 crc kubenswrapper[5052]: I1203 11:45:09.093077 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/swift-proxy-6fdf66df4-gsqlq"] Dec 03 11:45:09 crc kubenswrapper[5052]: I1203 11:45:09.094012 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-swift-public-svc" Dec 03 11:45:09 crc kubenswrapper[5052]: I1203 11:45:09.094137 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"swift-proxy-config-data" Dec 03 11:45:09 crc kubenswrapper[5052]: I1203 11:45:09.273132 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/84957e18-5747-4786-9c4f-6af023122726-etc-swift\") pod \"swift-proxy-6fdf66df4-gsqlq\" (UID: \"84957e18-5747-4786-9c4f-6af023122726\") " pod="openstack-kuttl-tests/swift-proxy-6fdf66df4-gsqlq" Dec 03 11:45:09 crc kubenswrapper[5052]: I1203 11:45:09.273520 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/84957e18-5747-4786-9c4f-6af023122726-log-httpd\") pod \"swift-proxy-6fdf66df4-gsqlq\" (UID: \"84957e18-5747-4786-9c4f-6af023122726\") " pod="openstack-kuttl-tests/swift-proxy-6fdf66df4-gsqlq" Dec 03 11:45:09 crc kubenswrapper[5052]: I1203 11:45:09.273782 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/84957e18-5747-4786-9c4f-6af023122726-public-tls-certs\") pod \"swift-proxy-6fdf66df4-gsqlq\" (UID: \"84957e18-5747-4786-9c4f-6af023122726\") " pod="openstack-kuttl-tests/swift-proxy-6fdf66df4-gsqlq" Dec 03 11:45:09 crc kubenswrapper[5052]: I1203 11:45:09.273813 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/84957e18-5747-4786-9c4f-6af023122726-combined-ca-bundle\") pod \"swift-proxy-6fdf66df4-gsqlq\" (UID: \"84957e18-5747-4786-9c4f-6af023122726\") " pod="openstack-kuttl-tests/swift-proxy-6fdf66df4-gsqlq" Dec 03 11:45:09 crc kubenswrapper[5052]: I1203 11:45:09.273898 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/84957e18-5747-4786-9c4f-6af023122726-config-data\") pod \"swift-proxy-6fdf66df4-gsqlq\" (UID: \"84957e18-5747-4786-9c4f-6af023122726\") " pod="openstack-kuttl-tests/swift-proxy-6fdf66df4-gsqlq" Dec 03 11:45:09 crc kubenswrapper[5052]: I1203 11:45:09.274020 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/84957e18-5747-4786-9c4f-6af023122726-internal-tls-certs\") pod \"swift-proxy-6fdf66df4-gsqlq\" (UID: \"84957e18-5747-4786-9c4f-6af023122726\") " pod="openstack-kuttl-tests/swift-proxy-6fdf66df4-gsqlq" Dec 03 11:45:09 crc kubenswrapper[5052]: I1203 11:45:09.274127 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/84957e18-5747-4786-9c4f-6af023122726-run-httpd\") pod \"swift-proxy-6fdf66df4-gsqlq\" (UID: \"84957e18-5747-4786-9c4f-6af023122726\") " pod="openstack-kuttl-tests/swift-proxy-6fdf66df4-gsqlq" Dec 03 11:45:09 crc kubenswrapper[5052]: I1203 11:45:09.274244 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fsmm9\" (UniqueName: \"kubernetes.io/projected/84957e18-5747-4786-9c4f-6af023122726-kube-api-access-fsmm9\") pod \"swift-proxy-6fdf66df4-gsqlq\" (UID: \"84957e18-5747-4786-9c4f-6af023122726\") " pod="openstack-kuttl-tests/swift-proxy-6fdf66df4-gsqlq" Dec 03 11:45:09 crc kubenswrapper[5052]: I1203 11:45:09.377231 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/84957e18-5747-4786-9c4f-6af023122726-run-httpd\") pod \"swift-proxy-6fdf66df4-gsqlq\" (UID: \"84957e18-5747-4786-9c4f-6af023122726\") " pod="openstack-kuttl-tests/swift-proxy-6fdf66df4-gsqlq" Dec 03 11:45:09 crc kubenswrapper[5052]: I1203 11:45:09.377652 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/84957e18-5747-4786-9c4f-6af023122726-run-httpd\") pod \"swift-proxy-6fdf66df4-gsqlq\" (UID: \"84957e18-5747-4786-9c4f-6af023122726\") " pod="openstack-kuttl-tests/swift-proxy-6fdf66df4-gsqlq" Dec 03 11:45:09 crc kubenswrapper[5052]: I1203 11:45:09.377826 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-fsmm9\" (UniqueName: \"kubernetes.io/projected/84957e18-5747-4786-9c4f-6af023122726-kube-api-access-fsmm9\") pod \"swift-proxy-6fdf66df4-gsqlq\" (UID: \"84957e18-5747-4786-9c4f-6af023122726\") " pod="openstack-kuttl-tests/swift-proxy-6fdf66df4-gsqlq" Dec 03 11:45:09 crc kubenswrapper[5052]: I1203 11:45:09.378314 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/84957e18-5747-4786-9c4f-6af023122726-etc-swift\") pod \"swift-proxy-6fdf66df4-gsqlq\" (UID: \"84957e18-5747-4786-9c4f-6af023122726\") " pod="openstack-kuttl-tests/swift-proxy-6fdf66df4-gsqlq" Dec 03 11:45:09 crc kubenswrapper[5052]: I1203 11:45:09.378771 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/84957e18-5747-4786-9c4f-6af023122726-log-httpd\") pod \"swift-proxy-6fdf66df4-gsqlq\" (UID: \"84957e18-5747-4786-9c4f-6af023122726\") " pod="openstack-kuttl-tests/swift-proxy-6fdf66df4-gsqlq" Dec 03 11:45:09 crc kubenswrapper[5052]: I1203 11:45:09.379254 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/84957e18-5747-4786-9c4f-6af023122726-log-httpd\") pod \"swift-proxy-6fdf66df4-gsqlq\" (UID: \"84957e18-5747-4786-9c4f-6af023122726\") " pod="openstack-kuttl-tests/swift-proxy-6fdf66df4-gsqlq" Dec 03 11:45:09 crc kubenswrapper[5052]: I1203 11:45:09.379316 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/84957e18-5747-4786-9c4f-6af023122726-public-tls-certs\") pod \"swift-proxy-6fdf66df4-gsqlq\" (UID: \"84957e18-5747-4786-9c4f-6af023122726\") " pod="openstack-kuttl-tests/swift-proxy-6fdf66df4-gsqlq" Dec 03 11:45:09 crc kubenswrapper[5052]: I1203 11:45:09.379354 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/84957e18-5747-4786-9c4f-6af023122726-combined-ca-bundle\") pod \"swift-proxy-6fdf66df4-gsqlq\" (UID: \"84957e18-5747-4786-9c4f-6af023122726\") " pod="openstack-kuttl-tests/swift-proxy-6fdf66df4-gsqlq" Dec 03 11:45:09 crc kubenswrapper[5052]: I1203 11:45:09.379406 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/84957e18-5747-4786-9c4f-6af023122726-config-data\") pod \"swift-proxy-6fdf66df4-gsqlq\" (UID: \"84957e18-5747-4786-9c4f-6af023122726\") " pod="openstack-kuttl-tests/swift-proxy-6fdf66df4-gsqlq" Dec 03 11:45:09 crc kubenswrapper[5052]: I1203 11:45:09.379486 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/84957e18-5747-4786-9c4f-6af023122726-internal-tls-certs\") pod \"swift-proxy-6fdf66df4-gsqlq\" (UID: \"84957e18-5747-4786-9c4f-6af023122726\") " pod="openstack-kuttl-tests/swift-proxy-6fdf66df4-gsqlq" Dec 03 11:45:09 crc kubenswrapper[5052]: I1203 11:45:09.385339 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/84957e18-5747-4786-9c4f-6af023122726-public-tls-certs\") pod \"swift-proxy-6fdf66df4-gsqlq\" (UID: \"84957e18-5747-4786-9c4f-6af023122726\") " pod="openstack-kuttl-tests/swift-proxy-6fdf66df4-gsqlq" Dec 03 11:45:09 crc kubenswrapper[5052]: I1203 11:45:09.386193 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/84957e18-5747-4786-9c4f-6af023122726-combined-ca-bundle\") pod \"swift-proxy-6fdf66df4-gsqlq\" (UID: \"84957e18-5747-4786-9c4f-6af023122726\") " pod="openstack-kuttl-tests/swift-proxy-6fdf66df4-gsqlq" Dec 03 11:45:09 crc kubenswrapper[5052]: I1203 11:45:09.386297 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/84957e18-5747-4786-9c4f-6af023122726-internal-tls-certs\") pod \"swift-proxy-6fdf66df4-gsqlq\" (UID: \"84957e18-5747-4786-9c4f-6af023122726\") " pod="openstack-kuttl-tests/swift-proxy-6fdf66df4-gsqlq" Dec 03 11:45:09 crc kubenswrapper[5052]: I1203 11:45:09.387373 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/84957e18-5747-4786-9c4f-6af023122726-etc-swift\") pod \"swift-proxy-6fdf66df4-gsqlq\" (UID: \"84957e18-5747-4786-9c4f-6af023122726\") " pod="openstack-kuttl-tests/swift-proxy-6fdf66df4-gsqlq" Dec 03 11:45:09 crc kubenswrapper[5052]: I1203 11:45:09.387460 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/84957e18-5747-4786-9c4f-6af023122726-config-data\") pod \"swift-proxy-6fdf66df4-gsqlq\" (UID: \"84957e18-5747-4786-9c4f-6af023122726\") " pod="openstack-kuttl-tests/swift-proxy-6fdf66df4-gsqlq" Dec 03 11:45:09 crc kubenswrapper[5052]: I1203 11:45:09.399571 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-fsmm9\" (UniqueName: \"kubernetes.io/projected/84957e18-5747-4786-9c4f-6af023122726-kube-api-access-fsmm9\") pod \"swift-proxy-6fdf66df4-gsqlq\" (UID: \"84957e18-5747-4786-9c4f-6af023122726\") " pod="openstack-kuttl-tests/swift-proxy-6fdf66df4-gsqlq" Dec 03 11:45:09 crc kubenswrapper[5052]: I1203 11:45:09.698781 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-proxy-6fdf66df4-gsqlq" Dec 03 11:45:10 crc kubenswrapper[5052]: I1203 11:45:10.142345 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/swift-proxy-6fdf66df4-gsqlq"] Dec 03 11:45:10 crc kubenswrapper[5052]: I1203 11:45:10.996878 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-proxy-6fdf66df4-gsqlq" event={"ID":"84957e18-5747-4786-9c4f-6af023122726","Type":"ContainerStarted","Data":"8e5f5287d62edf8983c313bb53a6f8da8365856de089efb37c1baa4e18ad8c3b"} Dec 03 11:45:12 crc kubenswrapper[5052]: I1203 11:45:12.011090 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-proxy-6fdf66df4-gsqlq" event={"ID":"84957e18-5747-4786-9c4f-6af023122726","Type":"ContainerStarted","Data":"07cccd5afb2592c0003b92ef2faa428ad9bc5bc71a62a78385e0f7a553335aac"} Dec 03 11:45:12 crc kubenswrapper[5052]: I1203 11:45:12.011411 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-proxy-6fdf66df4-gsqlq" event={"ID":"84957e18-5747-4786-9c4f-6af023122726","Type":"ContainerStarted","Data":"9a14693162272f7ccaeacad5515545206de3c0c0552776c0d5a91a5bb80088eb"} Dec 03 11:45:12 crc kubenswrapper[5052]: I1203 11:45:12.011433 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/swift-proxy-6fdf66df4-gsqlq" Dec 03 11:45:12 crc kubenswrapper[5052]: I1203 11:45:12.011446 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/swift-proxy-6fdf66df4-gsqlq" Dec 03 11:45:12 crc kubenswrapper[5052]: I1203 11:45:12.013310 5052 generic.go:334] "Generic (PLEG): container finished" podID="2dbe7002-eb09-4a0f-b65b-fc0294c13f4f" containerID="8c2a5717a95924a833c522bb45a7070432896bb7e0e5d55da6ef352206e5b20a" exitCode=0 Dec 03 11:45:12 crc kubenswrapper[5052]: I1203 11:45:12.013344 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-7cc99c958f-bjn2l" event={"ID":"2dbe7002-eb09-4a0f-b65b-fc0294c13f4f","Type":"ContainerDied","Data":"8c2a5717a95924a833c522bb45a7070432896bb7e0e5d55da6ef352206e5b20a"} Dec 03 11:45:12 crc kubenswrapper[5052]: I1203 11:45:12.052015 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/swift-proxy-6fdf66df4-gsqlq" podStartSLOduration=3.051998877 podStartE2EDuration="3.051998877s" podCreationTimestamp="2025-12-03 11:45:09 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:45:12.048657591 +0000 UTC m=+1502.636899355" watchObservedRunningTime="2025-12-03 11:45:12.051998877 +0000 UTC m=+1502.640240641" Dec 03 11:45:12 crc kubenswrapper[5052]: I1203 11:45:12.214263 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-7cc99c958f-bjn2l" Dec 03 11:45:12 crc kubenswrapper[5052]: I1203 11:45:12.340801 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/2dbe7002-eb09-4a0f-b65b-fc0294c13f4f-httpd-config\") pod \"2dbe7002-eb09-4a0f-b65b-fc0294c13f4f\" (UID: \"2dbe7002-eb09-4a0f-b65b-fc0294c13f4f\") " Dec 03 11:45:12 crc kubenswrapper[5052]: I1203 11:45:12.340914 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/2dbe7002-eb09-4a0f-b65b-fc0294c13f4f-ovndb-tls-certs\") pod \"2dbe7002-eb09-4a0f-b65b-fc0294c13f4f\" (UID: \"2dbe7002-eb09-4a0f-b65b-fc0294c13f4f\") " Dec 03 11:45:12 crc kubenswrapper[5052]: I1203 11:45:12.340956 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-22j7r\" (UniqueName: \"kubernetes.io/projected/2dbe7002-eb09-4a0f-b65b-fc0294c13f4f-kube-api-access-22j7r\") pod \"2dbe7002-eb09-4a0f-b65b-fc0294c13f4f\" (UID: \"2dbe7002-eb09-4a0f-b65b-fc0294c13f4f\") " Dec 03 11:45:12 crc kubenswrapper[5052]: I1203 11:45:12.341001 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2dbe7002-eb09-4a0f-b65b-fc0294c13f4f-combined-ca-bundle\") pod \"2dbe7002-eb09-4a0f-b65b-fc0294c13f4f\" (UID: \"2dbe7002-eb09-4a0f-b65b-fc0294c13f4f\") " Dec 03 11:45:12 crc kubenswrapper[5052]: I1203 11:45:12.341110 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/2dbe7002-eb09-4a0f-b65b-fc0294c13f4f-config\") pod \"2dbe7002-eb09-4a0f-b65b-fc0294c13f4f\" (UID: \"2dbe7002-eb09-4a0f-b65b-fc0294c13f4f\") " Dec 03 11:45:12 crc kubenswrapper[5052]: I1203 11:45:12.346809 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2dbe7002-eb09-4a0f-b65b-fc0294c13f4f-kube-api-access-22j7r" (OuterVolumeSpecName: "kube-api-access-22j7r") pod "2dbe7002-eb09-4a0f-b65b-fc0294c13f4f" (UID: "2dbe7002-eb09-4a0f-b65b-fc0294c13f4f"). InnerVolumeSpecName "kube-api-access-22j7r". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:45:12 crc kubenswrapper[5052]: I1203 11:45:12.346820 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2dbe7002-eb09-4a0f-b65b-fc0294c13f4f-httpd-config" (OuterVolumeSpecName: "httpd-config") pod "2dbe7002-eb09-4a0f-b65b-fc0294c13f4f" (UID: "2dbe7002-eb09-4a0f-b65b-fc0294c13f4f"). InnerVolumeSpecName "httpd-config". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:45:12 crc kubenswrapper[5052]: I1203 11:45:12.395944 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2dbe7002-eb09-4a0f-b65b-fc0294c13f4f-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "2dbe7002-eb09-4a0f-b65b-fc0294c13f4f" (UID: "2dbe7002-eb09-4a0f-b65b-fc0294c13f4f"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:45:12 crc kubenswrapper[5052]: I1203 11:45:12.420539 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2dbe7002-eb09-4a0f-b65b-fc0294c13f4f-config" (OuterVolumeSpecName: "config") pod "2dbe7002-eb09-4a0f-b65b-fc0294c13f4f" (UID: "2dbe7002-eb09-4a0f-b65b-fc0294c13f4f"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:45:12 crc kubenswrapper[5052]: I1203 11:45:12.441415 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2dbe7002-eb09-4a0f-b65b-fc0294c13f4f-ovndb-tls-certs" (OuterVolumeSpecName: "ovndb-tls-certs") pod "2dbe7002-eb09-4a0f-b65b-fc0294c13f4f" (UID: "2dbe7002-eb09-4a0f-b65b-fc0294c13f4f"). InnerVolumeSpecName "ovndb-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:45:12 crc kubenswrapper[5052]: I1203 11:45:12.443944 5052 reconciler_common.go:293] "Volume detached for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/2dbe7002-eb09-4a0f-b65b-fc0294c13f4f-ovndb-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 11:45:12 crc kubenswrapper[5052]: I1203 11:45:12.443971 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-22j7r\" (UniqueName: \"kubernetes.io/projected/2dbe7002-eb09-4a0f-b65b-fc0294c13f4f-kube-api-access-22j7r\") on node \"crc\" DevicePath \"\"" Dec 03 11:45:12 crc kubenswrapper[5052]: I1203 11:45:12.443983 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2dbe7002-eb09-4a0f-b65b-fc0294c13f4f-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:45:12 crc kubenswrapper[5052]: I1203 11:45:12.443993 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/2dbe7002-eb09-4a0f-b65b-fc0294c13f4f-config\") on node \"crc\" DevicePath \"\"" Dec 03 11:45:12 crc kubenswrapper[5052]: I1203 11:45:12.444003 5052 reconciler_common.go:293] "Volume detached for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/2dbe7002-eb09-4a0f-b65b-fc0294c13f4f-httpd-config\") on node \"crc\" DevicePath \"\"" Dec 03 11:45:12 crc kubenswrapper[5052]: I1203 11:45:12.645671 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openshift-marketplace/community-operators-fvb75"] Dec 03 11:45:12 crc kubenswrapper[5052]: E1203 11:45:12.646217 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="2dbe7002-eb09-4a0f-b65b-fc0294c13f4f" containerName="neutron-httpd" Dec 03 11:45:12 crc kubenswrapper[5052]: I1203 11:45:12.646264 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="2dbe7002-eb09-4a0f-b65b-fc0294c13f4f" containerName="neutron-httpd" Dec 03 11:45:12 crc kubenswrapper[5052]: E1203 11:45:12.646287 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="2dbe7002-eb09-4a0f-b65b-fc0294c13f4f" containerName="neutron-api" Dec 03 11:45:12 crc kubenswrapper[5052]: I1203 11:45:12.646295 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="2dbe7002-eb09-4a0f-b65b-fc0294c13f4f" containerName="neutron-api" Dec 03 11:45:12 crc kubenswrapper[5052]: I1203 11:45:12.646486 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="2dbe7002-eb09-4a0f-b65b-fc0294c13f4f" containerName="neutron-api" Dec 03 11:45:12 crc kubenswrapper[5052]: I1203 11:45:12.646531 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="2dbe7002-eb09-4a0f-b65b-fc0294c13f4f" containerName="neutron-httpd" Dec 03 11:45:12 crc kubenswrapper[5052]: I1203 11:45:12.647996 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/community-operators-fvb75" Dec 03 11:45:12 crc kubenswrapper[5052]: I1203 11:45:12.657983 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/community-operators-fvb75"] Dec 03 11:45:12 crc kubenswrapper[5052]: I1203 11:45:12.749683 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/f0cc2483-7e43-4536-8d99-a46d422051dd-utilities\") pod \"community-operators-fvb75\" (UID: \"f0cc2483-7e43-4536-8d99-a46d422051dd\") " pod="openshift-marketplace/community-operators-fvb75" Dec 03 11:45:12 crc kubenswrapper[5052]: I1203 11:45:12.749784 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ln58z\" (UniqueName: \"kubernetes.io/projected/f0cc2483-7e43-4536-8d99-a46d422051dd-kube-api-access-ln58z\") pod \"community-operators-fvb75\" (UID: \"f0cc2483-7e43-4536-8d99-a46d422051dd\") " pod="openshift-marketplace/community-operators-fvb75" Dec 03 11:45:12 crc kubenswrapper[5052]: I1203 11:45:12.749825 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/f0cc2483-7e43-4536-8d99-a46d422051dd-catalog-content\") pod \"community-operators-fvb75\" (UID: \"f0cc2483-7e43-4536-8d99-a46d422051dd\") " pod="openshift-marketplace/community-operators-fvb75" Dec 03 11:45:12 crc kubenswrapper[5052]: I1203 11:45:12.851930 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-ln58z\" (UniqueName: \"kubernetes.io/projected/f0cc2483-7e43-4536-8d99-a46d422051dd-kube-api-access-ln58z\") pod \"community-operators-fvb75\" (UID: \"f0cc2483-7e43-4536-8d99-a46d422051dd\") " pod="openshift-marketplace/community-operators-fvb75" Dec 03 11:45:12 crc kubenswrapper[5052]: I1203 11:45:12.851988 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/f0cc2483-7e43-4536-8d99-a46d422051dd-catalog-content\") pod \"community-operators-fvb75\" (UID: \"f0cc2483-7e43-4536-8d99-a46d422051dd\") " pod="openshift-marketplace/community-operators-fvb75" Dec 03 11:45:12 crc kubenswrapper[5052]: I1203 11:45:12.852107 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/f0cc2483-7e43-4536-8d99-a46d422051dd-utilities\") pod \"community-operators-fvb75\" (UID: \"f0cc2483-7e43-4536-8d99-a46d422051dd\") " pod="openshift-marketplace/community-operators-fvb75" Dec 03 11:45:12 crc kubenswrapper[5052]: I1203 11:45:12.852541 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/f0cc2483-7e43-4536-8d99-a46d422051dd-catalog-content\") pod \"community-operators-fvb75\" (UID: \"f0cc2483-7e43-4536-8d99-a46d422051dd\") " pod="openshift-marketplace/community-operators-fvb75" Dec 03 11:45:12 crc kubenswrapper[5052]: I1203 11:45:12.852564 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/f0cc2483-7e43-4536-8d99-a46d422051dd-utilities\") pod \"community-operators-fvb75\" (UID: \"f0cc2483-7e43-4536-8d99-a46d422051dd\") " pod="openshift-marketplace/community-operators-fvb75" Dec 03 11:45:12 crc kubenswrapper[5052]: I1203 11:45:12.875459 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-ln58z\" (UniqueName: \"kubernetes.io/projected/f0cc2483-7e43-4536-8d99-a46d422051dd-kube-api-access-ln58z\") pod \"community-operators-fvb75\" (UID: \"f0cc2483-7e43-4536-8d99-a46d422051dd\") " pod="openshift-marketplace/community-operators-fvb75" Dec 03 11:45:12 crc kubenswrapper[5052]: I1203 11:45:12.963442 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/community-operators-fvb75" Dec 03 11:45:13 crc kubenswrapper[5052]: I1203 11:45:13.032430 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-7cc99c958f-bjn2l" Dec 03 11:45:13 crc kubenswrapper[5052]: I1203 11:45:13.043257 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-7cc99c958f-bjn2l" event={"ID":"2dbe7002-eb09-4a0f-b65b-fc0294c13f4f","Type":"ContainerDied","Data":"0357ef8312fb1e8f4b30dd9b2c0a1cf44e8e5977c178167e5200a3bdf5bd3f86"} Dec 03 11:45:13 crc kubenswrapper[5052]: I1203 11:45:13.043346 5052 scope.go:117] "RemoveContainer" containerID="a60bd070c0bbd29b273ad38f5c6a3450100e8c13d43d5b00488a10b98645de23" Dec 03 11:45:13 crc kubenswrapper[5052]: I1203 11:45:13.103087 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/neutron-7cc99c958f-bjn2l"] Dec 03 11:45:13 crc kubenswrapper[5052]: I1203 11:45:13.120904 5052 scope.go:117] "RemoveContainer" containerID="8c2a5717a95924a833c522bb45a7070432896bb7e0e5d55da6ef352206e5b20a" Dec 03 11:45:13 crc kubenswrapper[5052]: I1203 11:45:13.149848 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/neutron-7cc99c958f-bjn2l"] Dec 03 11:45:13 crc kubenswrapper[5052]: I1203 11:45:13.511125 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/community-operators-fvb75"] Dec 03 11:45:13 crc kubenswrapper[5052]: I1203 11:45:13.682784 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2dbe7002-eb09-4a0f-b65b-fc0294c13f4f" path="/var/lib/kubelet/pods/2dbe7002-eb09-4a0f-b65b-fc0294c13f4f/volumes" Dec 03 11:45:14 crc kubenswrapper[5052]: I1203 11:45:14.045635 5052 generic.go:334] "Generic (PLEG): container finished" podID="b6eeb188-0ea6-46de-8cbf-d803e75df5e6" containerID="a543e5d6e1f5b544fd17dc9d996401e2195186e8556773c54e76285b9c705f51" exitCode=0 Dec 03 11:45:14 crc kubenswrapper[5052]: I1203 11:45:14.045722 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"b6eeb188-0ea6-46de-8cbf-d803e75df5e6","Type":"ContainerDied","Data":"a543e5d6e1f5b544fd17dc9d996401e2195186e8556773c54e76285b9c705f51"} Dec 03 11:45:14 crc kubenswrapper[5052]: I1203 11:45:14.051570 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/community-operators-fvb75" event={"ID":"f0cc2483-7e43-4536-8d99-a46d422051dd","Type":"ContainerStarted","Data":"eaca177a3fb82e4fe11b49247bed5bb82b8b79ba942bf9feee8cd7cf37c3c210"} Dec 03 11:45:14 crc kubenswrapper[5052]: I1203 11:45:14.645208 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:45:14 crc kubenswrapper[5052]: I1203 11:45:14.683981 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/b6eeb188-0ea6-46de-8cbf-d803e75df5e6-run-httpd\") pod \"b6eeb188-0ea6-46de-8cbf-d803e75df5e6\" (UID: \"b6eeb188-0ea6-46de-8cbf-d803e75df5e6\") " Dec 03 11:45:14 crc kubenswrapper[5052]: I1203 11:45:14.684317 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n8s4c\" (UniqueName: \"kubernetes.io/projected/b6eeb188-0ea6-46de-8cbf-d803e75df5e6-kube-api-access-n8s4c\") pod \"b6eeb188-0ea6-46de-8cbf-d803e75df5e6\" (UID: \"b6eeb188-0ea6-46de-8cbf-d803e75df5e6\") " Dec 03 11:45:14 crc kubenswrapper[5052]: I1203 11:45:14.684452 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/b6eeb188-0ea6-46de-8cbf-d803e75df5e6-log-httpd\") pod \"b6eeb188-0ea6-46de-8cbf-d803e75df5e6\" (UID: \"b6eeb188-0ea6-46de-8cbf-d803e75df5e6\") " Dec 03 11:45:14 crc kubenswrapper[5052]: I1203 11:45:14.684604 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/b6eeb188-0ea6-46de-8cbf-d803e75df5e6-config-data\") pod \"b6eeb188-0ea6-46de-8cbf-d803e75df5e6\" (UID: \"b6eeb188-0ea6-46de-8cbf-d803e75df5e6\") " Dec 03 11:45:14 crc kubenswrapper[5052]: I1203 11:45:14.684693 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/b6eeb188-0ea6-46de-8cbf-d803e75df5e6-scripts\") pod \"b6eeb188-0ea6-46de-8cbf-d803e75df5e6\" (UID: \"b6eeb188-0ea6-46de-8cbf-d803e75df5e6\") " Dec 03 11:45:14 crc kubenswrapper[5052]: I1203 11:45:14.684800 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/b6eeb188-0ea6-46de-8cbf-d803e75df5e6-sg-core-conf-yaml\") pod \"b6eeb188-0ea6-46de-8cbf-d803e75df5e6\" (UID: \"b6eeb188-0ea6-46de-8cbf-d803e75df5e6\") " Dec 03 11:45:14 crc kubenswrapper[5052]: I1203 11:45:14.684706 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b6eeb188-0ea6-46de-8cbf-d803e75df5e6-run-httpd" (OuterVolumeSpecName: "run-httpd") pod "b6eeb188-0ea6-46de-8cbf-d803e75df5e6" (UID: "b6eeb188-0ea6-46de-8cbf-d803e75df5e6"). InnerVolumeSpecName "run-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:45:14 crc kubenswrapper[5052]: I1203 11:45:14.684928 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b6eeb188-0ea6-46de-8cbf-d803e75df5e6-log-httpd" (OuterVolumeSpecName: "log-httpd") pod "b6eeb188-0ea6-46de-8cbf-d803e75df5e6" (UID: "b6eeb188-0ea6-46de-8cbf-d803e75df5e6"). InnerVolumeSpecName "log-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:45:14 crc kubenswrapper[5052]: I1203 11:45:14.684889 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b6eeb188-0ea6-46de-8cbf-d803e75df5e6-combined-ca-bundle\") pod \"b6eeb188-0ea6-46de-8cbf-d803e75df5e6\" (UID: \"b6eeb188-0ea6-46de-8cbf-d803e75df5e6\") " Dec 03 11:45:14 crc kubenswrapper[5052]: I1203 11:45:14.685739 5052 reconciler_common.go:293] "Volume detached for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/b6eeb188-0ea6-46de-8cbf-d803e75df5e6-log-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 11:45:14 crc kubenswrapper[5052]: I1203 11:45:14.685757 5052 reconciler_common.go:293] "Volume detached for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/b6eeb188-0ea6-46de-8cbf-d803e75df5e6-run-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 11:45:14 crc kubenswrapper[5052]: I1203 11:45:14.690990 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b6eeb188-0ea6-46de-8cbf-d803e75df5e6-scripts" (OuterVolumeSpecName: "scripts") pod "b6eeb188-0ea6-46de-8cbf-d803e75df5e6" (UID: "b6eeb188-0ea6-46de-8cbf-d803e75df5e6"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:45:14 crc kubenswrapper[5052]: I1203 11:45:14.691062 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b6eeb188-0ea6-46de-8cbf-d803e75df5e6-kube-api-access-n8s4c" (OuterVolumeSpecName: "kube-api-access-n8s4c") pod "b6eeb188-0ea6-46de-8cbf-d803e75df5e6" (UID: "b6eeb188-0ea6-46de-8cbf-d803e75df5e6"). InnerVolumeSpecName "kube-api-access-n8s4c". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:45:14 crc kubenswrapper[5052]: I1203 11:45:14.718158 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b6eeb188-0ea6-46de-8cbf-d803e75df5e6-sg-core-conf-yaml" (OuterVolumeSpecName: "sg-core-conf-yaml") pod "b6eeb188-0ea6-46de-8cbf-d803e75df5e6" (UID: "b6eeb188-0ea6-46de-8cbf-d803e75df5e6"). InnerVolumeSpecName "sg-core-conf-yaml". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:45:14 crc kubenswrapper[5052]: I1203 11:45:14.772945 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b6eeb188-0ea6-46de-8cbf-d803e75df5e6-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "b6eeb188-0ea6-46de-8cbf-d803e75df5e6" (UID: "b6eeb188-0ea6-46de-8cbf-d803e75df5e6"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:45:14 crc kubenswrapper[5052]: I1203 11:45:14.786697 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-n8s4c\" (UniqueName: \"kubernetes.io/projected/b6eeb188-0ea6-46de-8cbf-d803e75df5e6-kube-api-access-n8s4c\") on node \"crc\" DevicePath \"\"" Dec 03 11:45:14 crc kubenswrapper[5052]: I1203 11:45:14.786732 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/b6eeb188-0ea6-46de-8cbf-d803e75df5e6-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:45:14 crc kubenswrapper[5052]: I1203 11:45:14.786741 5052 reconciler_common.go:293] "Volume detached for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/b6eeb188-0ea6-46de-8cbf-d803e75df5e6-sg-core-conf-yaml\") on node \"crc\" DevicePath \"\"" Dec 03 11:45:14 crc kubenswrapper[5052]: I1203 11:45:14.786749 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b6eeb188-0ea6-46de-8cbf-d803e75df5e6-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:45:14 crc kubenswrapper[5052]: I1203 11:45:14.817009 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b6eeb188-0ea6-46de-8cbf-d803e75df5e6-config-data" (OuterVolumeSpecName: "config-data") pod "b6eeb188-0ea6-46de-8cbf-d803e75df5e6" (UID: "b6eeb188-0ea6-46de-8cbf-d803e75df5e6"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:45:14 crc kubenswrapper[5052]: I1203 11:45:14.843558 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/kube-state-metrics-0"] Dec 03 11:45:14 crc kubenswrapper[5052]: I1203 11:45:14.843814 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/kube-state-metrics-0" podUID="78388bce-bd9d-426d-b6b6-e0fba8b3d8f9" containerName="kube-state-metrics" containerID="cri-o://e284c3d794c0b8aa02b660d2a83e53e3db40486717ae4fa7ff4311373d5644eb" gracePeriod=30 Dec 03 11:45:14 crc kubenswrapper[5052]: I1203 11:45:14.888160 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/b6eeb188-0ea6-46de-8cbf-d803e75df5e6-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:45:15 crc kubenswrapper[5052]: I1203 11:45:15.065459 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"b6eeb188-0ea6-46de-8cbf-d803e75df5e6","Type":"ContainerDied","Data":"d320b741f24a721bdb9881c1fc972da531a73e2249a75ed8d9f85e5a7dc52dbc"} Dec 03 11:45:15 crc kubenswrapper[5052]: I1203 11:45:15.065532 5052 scope.go:117] "RemoveContainer" containerID="4605df729b344cdc8aa272fc21b35ff9f030fbd19d68d7fbb10d033f9171882f" Dec 03 11:45:15 crc kubenswrapper[5052]: I1203 11:45:15.065546 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:45:15 crc kubenswrapper[5052]: I1203 11:45:15.069660 5052 generic.go:334] "Generic (PLEG): container finished" podID="f0cc2483-7e43-4536-8d99-a46d422051dd" containerID="c672ad36f46e8a30f58b40807ef2de7b16aeeea1b0b227113bde460f0731c592" exitCode=0 Dec 03 11:45:15 crc kubenswrapper[5052]: I1203 11:45:15.069725 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/community-operators-fvb75" event={"ID":"f0cc2483-7e43-4536-8d99-a46d422051dd","Type":"ContainerDied","Data":"c672ad36f46e8a30f58b40807ef2de7b16aeeea1b0b227113bde460f0731c592"} Dec 03 11:45:15 crc kubenswrapper[5052]: I1203 11:45:15.073205 5052 generic.go:334] "Generic (PLEG): container finished" podID="78388bce-bd9d-426d-b6b6-e0fba8b3d8f9" containerID="e284c3d794c0b8aa02b660d2a83e53e3db40486717ae4fa7ff4311373d5644eb" exitCode=2 Dec 03 11:45:15 crc kubenswrapper[5052]: I1203 11:45:15.073251 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/kube-state-metrics-0" event={"ID":"78388bce-bd9d-426d-b6b6-e0fba8b3d8f9","Type":"ContainerDied","Data":"e284c3d794c0b8aa02b660d2a83e53e3db40486717ae4fa7ff4311373d5644eb"} Dec 03 11:45:15 crc kubenswrapper[5052]: I1203 11:45:15.101715 5052 scope.go:117] "RemoveContainer" containerID="ba76522ceb5c6ff71560ef00ae164eb9deadf017edb824ce2c749304302a9186" Dec 03 11:45:15 crc kubenswrapper[5052]: I1203 11:45:15.118961 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:45:15 crc kubenswrapper[5052]: I1203 11:45:15.134069 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:45:15 crc kubenswrapper[5052]: I1203 11:45:15.142358 5052 scope.go:117] "RemoveContainer" containerID="a543e5d6e1f5b544fd17dc9d996401e2195186e8556773c54e76285b9c705f51" Dec 03 11:45:15 crc kubenswrapper[5052]: I1203 11:45:15.152312 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:45:15 crc kubenswrapper[5052]: E1203 11:45:15.152757 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="b6eeb188-0ea6-46de-8cbf-d803e75df5e6" containerName="sg-core" Dec 03 11:45:15 crc kubenswrapper[5052]: I1203 11:45:15.152775 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="b6eeb188-0ea6-46de-8cbf-d803e75df5e6" containerName="sg-core" Dec 03 11:45:15 crc kubenswrapper[5052]: E1203 11:45:15.152792 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="b6eeb188-0ea6-46de-8cbf-d803e75df5e6" containerName="proxy-httpd" Dec 03 11:45:15 crc kubenswrapper[5052]: I1203 11:45:15.152800 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="b6eeb188-0ea6-46de-8cbf-d803e75df5e6" containerName="proxy-httpd" Dec 03 11:45:15 crc kubenswrapper[5052]: E1203 11:45:15.152813 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="b6eeb188-0ea6-46de-8cbf-d803e75df5e6" containerName="ceilometer-central-agent" Dec 03 11:45:15 crc kubenswrapper[5052]: I1203 11:45:15.152820 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="b6eeb188-0ea6-46de-8cbf-d803e75df5e6" containerName="ceilometer-central-agent" Dec 03 11:45:15 crc kubenswrapper[5052]: E1203 11:45:15.152843 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="b6eeb188-0ea6-46de-8cbf-d803e75df5e6" containerName="ceilometer-notification-agent" Dec 03 11:45:15 crc kubenswrapper[5052]: I1203 11:45:15.152851 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="b6eeb188-0ea6-46de-8cbf-d803e75df5e6" containerName="ceilometer-notification-agent" Dec 03 11:45:15 crc kubenswrapper[5052]: I1203 11:45:15.153089 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="b6eeb188-0ea6-46de-8cbf-d803e75df5e6" containerName="ceilometer-central-agent" Dec 03 11:45:15 crc kubenswrapper[5052]: I1203 11:45:15.153104 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="b6eeb188-0ea6-46de-8cbf-d803e75df5e6" containerName="sg-core" Dec 03 11:45:15 crc kubenswrapper[5052]: I1203 11:45:15.153117 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="b6eeb188-0ea6-46de-8cbf-d803e75df5e6" containerName="ceilometer-notification-agent" Dec 03 11:45:15 crc kubenswrapper[5052]: I1203 11:45:15.153145 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="b6eeb188-0ea6-46de-8cbf-d803e75df5e6" containerName="proxy-httpd" Dec 03 11:45:15 crc kubenswrapper[5052]: I1203 11:45:15.155191 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:45:15 crc kubenswrapper[5052]: I1203 11:45:15.159131 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-scripts" Dec 03 11:45:15 crc kubenswrapper[5052]: I1203 11:45:15.160314 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:45:15 crc kubenswrapper[5052]: I1203 11:45:15.161094 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-config-data" Dec 03 11:45:15 crc kubenswrapper[5052]: I1203 11:45:15.194456 5052 scope.go:117] "RemoveContainer" containerID="df00125cee9eca43a7001e16f293bd9269f75ced8e07fe1609749fa6ec555248" Dec 03 11:45:15 crc kubenswrapper[5052]: I1203 11:45:15.196402 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/73440cd7-672e-4ace-91bd-b380dbcf681f-config-data\") pod \"ceilometer-0\" (UID: \"73440cd7-672e-4ace-91bd-b380dbcf681f\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:45:15 crc kubenswrapper[5052]: I1203 11:45:15.196465 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/73440cd7-672e-4ace-91bd-b380dbcf681f-scripts\") pod \"ceilometer-0\" (UID: \"73440cd7-672e-4ace-91bd-b380dbcf681f\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:45:15 crc kubenswrapper[5052]: I1203 11:45:15.196539 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rr9ft\" (UniqueName: \"kubernetes.io/projected/73440cd7-672e-4ace-91bd-b380dbcf681f-kube-api-access-rr9ft\") pod \"ceilometer-0\" (UID: \"73440cd7-672e-4ace-91bd-b380dbcf681f\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:45:15 crc kubenswrapper[5052]: I1203 11:45:15.196566 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/73440cd7-672e-4ace-91bd-b380dbcf681f-log-httpd\") pod \"ceilometer-0\" (UID: \"73440cd7-672e-4ace-91bd-b380dbcf681f\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:45:15 crc kubenswrapper[5052]: I1203 11:45:15.196587 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/73440cd7-672e-4ace-91bd-b380dbcf681f-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"73440cd7-672e-4ace-91bd-b380dbcf681f\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:45:15 crc kubenswrapper[5052]: I1203 11:45:15.196615 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/73440cd7-672e-4ace-91bd-b380dbcf681f-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"73440cd7-672e-4ace-91bd-b380dbcf681f\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:45:15 crc kubenswrapper[5052]: I1203 11:45:15.196686 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/73440cd7-672e-4ace-91bd-b380dbcf681f-run-httpd\") pod \"ceilometer-0\" (UID: \"73440cd7-672e-4ace-91bd-b380dbcf681f\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:45:15 crc kubenswrapper[5052]: I1203 11:45:15.298002 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/73440cd7-672e-4ace-91bd-b380dbcf681f-run-httpd\") pod \"ceilometer-0\" (UID: \"73440cd7-672e-4ace-91bd-b380dbcf681f\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:45:15 crc kubenswrapper[5052]: I1203 11:45:15.298173 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/73440cd7-672e-4ace-91bd-b380dbcf681f-config-data\") pod \"ceilometer-0\" (UID: \"73440cd7-672e-4ace-91bd-b380dbcf681f\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:45:15 crc kubenswrapper[5052]: I1203 11:45:15.298215 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/73440cd7-672e-4ace-91bd-b380dbcf681f-scripts\") pod \"ceilometer-0\" (UID: \"73440cd7-672e-4ace-91bd-b380dbcf681f\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:45:15 crc kubenswrapper[5052]: I1203 11:45:15.298257 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-rr9ft\" (UniqueName: \"kubernetes.io/projected/73440cd7-672e-4ace-91bd-b380dbcf681f-kube-api-access-rr9ft\") pod \"ceilometer-0\" (UID: \"73440cd7-672e-4ace-91bd-b380dbcf681f\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:45:15 crc kubenswrapper[5052]: I1203 11:45:15.298277 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/73440cd7-672e-4ace-91bd-b380dbcf681f-log-httpd\") pod \"ceilometer-0\" (UID: \"73440cd7-672e-4ace-91bd-b380dbcf681f\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:45:15 crc kubenswrapper[5052]: I1203 11:45:15.298295 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/73440cd7-672e-4ace-91bd-b380dbcf681f-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"73440cd7-672e-4ace-91bd-b380dbcf681f\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:45:15 crc kubenswrapper[5052]: I1203 11:45:15.298345 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/73440cd7-672e-4ace-91bd-b380dbcf681f-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"73440cd7-672e-4ace-91bd-b380dbcf681f\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:45:15 crc kubenswrapper[5052]: I1203 11:45:15.298606 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/73440cd7-672e-4ace-91bd-b380dbcf681f-run-httpd\") pod \"ceilometer-0\" (UID: \"73440cd7-672e-4ace-91bd-b380dbcf681f\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:45:15 crc kubenswrapper[5052]: I1203 11:45:15.299222 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/73440cd7-672e-4ace-91bd-b380dbcf681f-log-httpd\") pod \"ceilometer-0\" (UID: \"73440cd7-672e-4ace-91bd-b380dbcf681f\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:45:15 crc kubenswrapper[5052]: I1203 11:45:15.303158 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/73440cd7-672e-4ace-91bd-b380dbcf681f-config-data\") pod \"ceilometer-0\" (UID: \"73440cd7-672e-4ace-91bd-b380dbcf681f\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:45:15 crc kubenswrapper[5052]: I1203 11:45:15.303265 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/73440cd7-672e-4ace-91bd-b380dbcf681f-scripts\") pod \"ceilometer-0\" (UID: \"73440cd7-672e-4ace-91bd-b380dbcf681f\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:45:15 crc kubenswrapper[5052]: I1203 11:45:15.303574 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/73440cd7-672e-4ace-91bd-b380dbcf681f-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"73440cd7-672e-4ace-91bd-b380dbcf681f\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:45:15 crc kubenswrapper[5052]: I1203 11:45:15.305234 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/73440cd7-672e-4ace-91bd-b380dbcf681f-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"73440cd7-672e-4ace-91bd-b380dbcf681f\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:45:15 crc kubenswrapper[5052]: I1203 11:45:15.315046 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-rr9ft\" (UniqueName: \"kubernetes.io/projected/73440cd7-672e-4ace-91bd-b380dbcf681f-kube-api-access-rr9ft\") pod \"ceilometer-0\" (UID: \"73440cd7-672e-4ace-91bd-b380dbcf681f\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:45:15 crc kubenswrapper[5052]: I1203 11:45:15.362927 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 11:45:15 crc kubenswrapper[5052]: I1203 11:45:15.491317 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:45:15 crc kubenswrapper[5052]: I1203 11:45:15.501034 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tnf7p\" (UniqueName: \"kubernetes.io/projected/78388bce-bd9d-426d-b6b6-e0fba8b3d8f9-kube-api-access-tnf7p\") pod \"78388bce-bd9d-426d-b6b6-e0fba8b3d8f9\" (UID: \"78388bce-bd9d-426d-b6b6-e0fba8b3d8f9\") " Dec 03 11:45:15 crc kubenswrapper[5052]: I1203 11:45:15.504086 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/78388bce-bd9d-426d-b6b6-e0fba8b3d8f9-kube-api-access-tnf7p" (OuterVolumeSpecName: "kube-api-access-tnf7p") pod "78388bce-bd9d-426d-b6b6-e0fba8b3d8f9" (UID: "78388bce-bd9d-426d-b6b6-e0fba8b3d8f9"). InnerVolumeSpecName "kube-api-access-tnf7p". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:45:15 crc kubenswrapper[5052]: I1203 11:45:15.574162 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-api-db-create-bmscg"] Dec 03 11:45:15 crc kubenswrapper[5052]: E1203 11:45:15.574593 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="78388bce-bd9d-426d-b6b6-e0fba8b3d8f9" containerName="kube-state-metrics" Dec 03 11:45:15 crc kubenswrapper[5052]: I1203 11:45:15.574611 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="78388bce-bd9d-426d-b6b6-e0fba8b3d8f9" containerName="kube-state-metrics" Dec 03 11:45:15 crc kubenswrapper[5052]: I1203 11:45:15.574771 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="78388bce-bd9d-426d-b6b6-e0fba8b3d8f9" containerName="kube-state-metrics" Dec 03 11:45:15 crc kubenswrapper[5052]: I1203 11:45:15.575444 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-db-create-bmscg" Dec 03 11:45:15 crc kubenswrapper[5052]: I1203 11:45:15.589264 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-db-create-bmscg"] Dec 03 11:45:15 crc kubenswrapper[5052]: I1203 11:45:15.603482 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/b85b18b0-5745-44b1-8add-a67d56d1d0c4-operator-scripts\") pod \"nova-api-db-create-bmscg\" (UID: \"b85b18b0-5745-44b1-8add-a67d56d1d0c4\") " pod="openstack-kuttl-tests/nova-api-db-create-bmscg" Dec 03 11:45:15 crc kubenswrapper[5052]: I1203 11:45:15.603629 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jsw6l\" (UniqueName: \"kubernetes.io/projected/b85b18b0-5745-44b1-8add-a67d56d1d0c4-kube-api-access-jsw6l\") pod \"nova-api-db-create-bmscg\" (UID: \"b85b18b0-5745-44b1-8add-a67d56d1d0c4\") " pod="openstack-kuttl-tests/nova-api-db-create-bmscg" Dec 03 11:45:15 crc kubenswrapper[5052]: I1203 11:45:15.603738 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-tnf7p\" (UniqueName: \"kubernetes.io/projected/78388bce-bd9d-426d-b6b6-e0fba8b3d8f9-kube-api-access-tnf7p\") on node \"crc\" DevicePath \"\"" Dec 03 11:45:15 crc kubenswrapper[5052]: I1203 11:45:15.695283 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b6eeb188-0ea6-46de-8cbf-d803e75df5e6" path="/var/lib/kubelet/pods/b6eeb188-0ea6-46de-8cbf-d803e75df5e6/volumes" Dec 03 11:45:15 crc kubenswrapper[5052]: I1203 11:45:15.698535 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell0-db-create-7mslc"] Dec 03 11:45:15 crc kubenswrapper[5052]: I1203 11:45:15.700085 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-db-create-7mslc" Dec 03 11:45:15 crc kubenswrapper[5052]: I1203 11:45:15.715731 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/b85b18b0-5745-44b1-8add-a67d56d1d0c4-operator-scripts\") pod \"nova-api-db-create-bmscg\" (UID: \"b85b18b0-5745-44b1-8add-a67d56d1d0c4\") " pod="openstack-kuttl-tests/nova-api-db-create-bmscg" Dec 03 11:45:15 crc kubenswrapper[5052]: I1203 11:45:15.715858 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-jsw6l\" (UniqueName: \"kubernetes.io/projected/b85b18b0-5745-44b1-8add-a67d56d1d0c4-kube-api-access-jsw6l\") pod \"nova-api-db-create-bmscg\" (UID: \"b85b18b0-5745-44b1-8add-a67d56d1d0c4\") " pod="openstack-kuttl-tests/nova-api-db-create-bmscg" Dec 03 11:45:15 crc kubenswrapper[5052]: I1203 11:45:15.722647 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-api-5919-account-create-update-wdllt"] Dec 03 11:45:15 crc kubenswrapper[5052]: I1203 11:45:15.723303 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/b85b18b0-5745-44b1-8add-a67d56d1d0c4-operator-scripts\") pod \"nova-api-db-create-bmscg\" (UID: \"b85b18b0-5745-44b1-8add-a67d56d1d0c4\") " pod="openstack-kuttl-tests/nova-api-db-create-bmscg" Dec 03 11:45:15 crc kubenswrapper[5052]: I1203 11:45:15.739529 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-5919-account-create-update-wdllt" Dec 03 11:45:15 crc kubenswrapper[5052]: I1203 11:45:15.741571 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-api-db-secret" Dec 03 11:45:15 crc kubenswrapper[5052]: I1203 11:45:15.774207 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-jsw6l\" (UniqueName: \"kubernetes.io/projected/b85b18b0-5745-44b1-8add-a67d56d1d0c4-kube-api-access-jsw6l\") pod \"nova-api-db-create-bmscg\" (UID: \"b85b18b0-5745-44b1-8add-a67d56d1d0c4\") " pod="openstack-kuttl-tests/nova-api-db-create-bmscg" Dec 03 11:45:15 crc kubenswrapper[5052]: I1203 11:45:15.785149 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell0-db-create-7mslc"] Dec 03 11:45:15 crc kubenswrapper[5052]: I1203 11:45:15.801540 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-5919-account-create-update-wdllt"] Dec 03 11:45:15 crc kubenswrapper[5052]: I1203 11:45:15.818077 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c4jgv\" (UniqueName: \"kubernetes.io/projected/6d924068-a887-4c83-89ad-9f333eb94799-kube-api-access-c4jgv\") pod \"nova-api-5919-account-create-update-wdllt\" (UID: \"6d924068-a887-4c83-89ad-9f333eb94799\") " pod="openstack-kuttl-tests/nova-api-5919-account-create-update-wdllt" Dec 03 11:45:15 crc kubenswrapper[5052]: I1203 11:45:15.818186 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/6d924068-a887-4c83-89ad-9f333eb94799-operator-scripts\") pod \"nova-api-5919-account-create-update-wdllt\" (UID: \"6d924068-a887-4c83-89ad-9f333eb94799\") " pod="openstack-kuttl-tests/nova-api-5919-account-create-update-wdllt" Dec 03 11:45:15 crc kubenswrapper[5052]: I1203 11:45:15.818229 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c6md6\" (UniqueName: \"kubernetes.io/projected/2744f411-6860-47c0-a8d5-f52e23e390f3-kube-api-access-c6md6\") pod \"nova-cell0-db-create-7mslc\" (UID: \"2744f411-6860-47c0-a8d5-f52e23e390f3\") " pod="openstack-kuttl-tests/nova-cell0-db-create-7mslc" Dec 03 11:45:15 crc kubenswrapper[5052]: I1203 11:45:15.818283 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/2744f411-6860-47c0-a8d5-f52e23e390f3-operator-scripts\") pod \"nova-cell0-db-create-7mslc\" (UID: \"2744f411-6860-47c0-a8d5-f52e23e390f3\") " pod="openstack-kuttl-tests/nova-cell0-db-create-7mslc" Dec 03 11:45:15 crc kubenswrapper[5052]: I1203 11:45:15.840603 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell1-db-create-t49dl"] Dec 03 11:45:15 crc kubenswrapper[5052]: I1203 11:45:15.849457 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-db-create-t49dl" Dec 03 11:45:15 crc kubenswrapper[5052]: I1203 11:45:15.869887 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-db-create-t49dl"] Dec 03 11:45:15 crc kubenswrapper[5052]: I1203 11:45:15.877944 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell0-f13b-account-create-update-p7l6b"] Dec 03 11:45:15 crc kubenswrapper[5052]: I1203 11:45:15.879036 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-f13b-account-create-update-p7l6b" Dec 03 11:45:15 crc kubenswrapper[5052]: I1203 11:45:15.881096 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell0-db-secret" Dec 03 11:45:15 crc kubenswrapper[5052]: I1203 11:45:15.895007 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell0-f13b-account-create-update-p7l6b"] Dec 03 11:45:15 crc kubenswrapper[5052]: I1203 11:45:15.919419 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-c4jgv\" (UniqueName: \"kubernetes.io/projected/6d924068-a887-4c83-89ad-9f333eb94799-kube-api-access-c4jgv\") pod \"nova-api-5919-account-create-update-wdllt\" (UID: \"6d924068-a887-4c83-89ad-9f333eb94799\") " pod="openstack-kuttl-tests/nova-api-5919-account-create-update-wdllt" Dec 03 11:45:15 crc kubenswrapper[5052]: I1203 11:45:15.919478 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8vdpf\" (UniqueName: \"kubernetes.io/projected/2bd8667f-0169-4f07-aefc-e19027cf99e6-kube-api-access-8vdpf\") pod \"nova-cell1-db-create-t49dl\" (UID: \"2bd8667f-0169-4f07-aefc-e19027cf99e6\") " pod="openstack-kuttl-tests/nova-cell1-db-create-t49dl" Dec 03 11:45:15 crc kubenswrapper[5052]: I1203 11:45:15.919501 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/6d924068-a887-4c83-89ad-9f333eb94799-operator-scripts\") pod \"nova-api-5919-account-create-update-wdllt\" (UID: \"6d924068-a887-4c83-89ad-9f333eb94799\") " pod="openstack-kuttl-tests/nova-api-5919-account-create-update-wdllt" Dec 03 11:45:15 crc kubenswrapper[5052]: I1203 11:45:15.919542 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-c6md6\" (UniqueName: \"kubernetes.io/projected/2744f411-6860-47c0-a8d5-f52e23e390f3-kube-api-access-c6md6\") pod \"nova-cell0-db-create-7mslc\" (UID: \"2744f411-6860-47c0-a8d5-f52e23e390f3\") " pod="openstack-kuttl-tests/nova-cell0-db-create-7mslc" Dec 03 11:45:15 crc kubenswrapper[5052]: I1203 11:45:15.919577 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/2bd8667f-0169-4f07-aefc-e19027cf99e6-operator-scripts\") pod \"nova-cell1-db-create-t49dl\" (UID: \"2bd8667f-0169-4f07-aefc-e19027cf99e6\") " pod="openstack-kuttl-tests/nova-cell1-db-create-t49dl" Dec 03 11:45:15 crc kubenswrapper[5052]: I1203 11:45:15.919618 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/ae7af34b-d086-4215-b568-1bf5b75bbe39-operator-scripts\") pod \"nova-cell0-f13b-account-create-update-p7l6b\" (UID: \"ae7af34b-d086-4215-b568-1bf5b75bbe39\") " pod="openstack-kuttl-tests/nova-cell0-f13b-account-create-update-p7l6b" Dec 03 11:45:15 crc kubenswrapper[5052]: I1203 11:45:15.919641 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/2744f411-6860-47c0-a8d5-f52e23e390f3-operator-scripts\") pod \"nova-cell0-db-create-7mslc\" (UID: \"2744f411-6860-47c0-a8d5-f52e23e390f3\") " pod="openstack-kuttl-tests/nova-cell0-db-create-7mslc" Dec 03 11:45:15 crc kubenswrapper[5052]: I1203 11:45:15.919685 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-57kzm\" (UniqueName: \"kubernetes.io/projected/ae7af34b-d086-4215-b568-1bf5b75bbe39-kube-api-access-57kzm\") pod \"nova-cell0-f13b-account-create-update-p7l6b\" (UID: \"ae7af34b-d086-4215-b568-1bf5b75bbe39\") " pod="openstack-kuttl-tests/nova-cell0-f13b-account-create-update-p7l6b" Dec 03 11:45:15 crc kubenswrapper[5052]: I1203 11:45:15.920551 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/2744f411-6860-47c0-a8d5-f52e23e390f3-operator-scripts\") pod \"nova-cell0-db-create-7mslc\" (UID: \"2744f411-6860-47c0-a8d5-f52e23e390f3\") " pod="openstack-kuttl-tests/nova-cell0-db-create-7mslc" Dec 03 11:45:15 crc kubenswrapper[5052]: I1203 11:45:15.920682 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/6d924068-a887-4c83-89ad-9f333eb94799-operator-scripts\") pod \"nova-api-5919-account-create-update-wdllt\" (UID: \"6d924068-a887-4c83-89ad-9f333eb94799\") " pod="openstack-kuttl-tests/nova-api-5919-account-create-update-wdllt" Dec 03 11:45:15 crc kubenswrapper[5052]: I1203 11:45:15.952141 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-c6md6\" (UniqueName: \"kubernetes.io/projected/2744f411-6860-47c0-a8d5-f52e23e390f3-kube-api-access-c6md6\") pod \"nova-cell0-db-create-7mslc\" (UID: \"2744f411-6860-47c0-a8d5-f52e23e390f3\") " pod="openstack-kuttl-tests/nova-cell0-db-create-7mslc" Dec 03 11:45:15 crc kubenswrapper[5052]: I1203 11:45:15.952305 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-c4jgv\" (UniqueName: \"kubernetes.io/projected/6d924068-a887-4c83-89ad-9f333eb94799-kube-api-access-c4jgv\") pod \"nova-api-5919-account-create-update-wdllt\" (UID: \"6d924068-a887-4c83-89ad-9f333eb94799\") " pod="openstack-kuttl-tests/nova-api-5919-account-create-update-wdllt" Dec 03 11:45:15 crc kubenswrapper[5052]: I1203 11:45:15.975611 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-db-create-bmscg" Dec 03 11:45:16 crc kubenswrapper[5052]: I1203 11:45:16.020686 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/ae7af34b-d086-4215-b568-1bf5b75bbe39-operator-scripts\") pod \"nova-cell0-f13b-account-create-update-p7l6b\" (UID: \"ae7af34b-d086-4215-b568-1bf5b75bbe39\") " pod="openstack-kuttl-tests/nova-cell0-f13b-account-create-update-p7l6b" Dec 03 11:45:16 crc kubenswrapper[5052]: I1203 11:45:16.020758 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-57kzm\" (UniqueName: \"kubernetes.io/projected/ae7af34b-d086-4215-b568-1bf5b75bbe39-kube-api-access-57kzm\") pod \"nova-cell0-f13b-account-create-update-p7l6b\" (UID: \"ae7af34b-d086-4215-b568-1bf5b75bbe39\") " pod="openstack-kuttl-tests/nova-cell0-f13b-account-create-update-p7l6b" Dec 03 11:45:16 crc kubenswrapper[5052]: I1203 11:45:16.020847 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-8vdpf\" (UniqueName: \"kubernetes.io/projected/2bd8667f-0169-4f07-aefc-e19027cf99e6-kube-api-access-8vdpf\") pod \"nova-cell1-db-create-t49dl\" (UID: \"2bd8667f-0169-4f07-aefc-e19027cf99e6\") " pod="openstack-kuttl-tests/nova-cell1-db-create-t49dl" Dec 03 11:45:16 crc kubenswrapper[5052]: I1203 11:45:16.020876 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/2bd8667f-0169-4f07-aefc-e19027cf99e6-operator-scripts\") pod \"nova-cell1-db-create-t49dl\" (UID: \"2bd8667f-0169-4f07-aefc-e19027cf99e6\") " pod="openstack-kuttl-tests/nova-cell1-db-create-t49dl" Dec 03 11:45:16 crc kubenswrapper[5052]: I1203 11:45:16.021506 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/2bd8667f-0169-4f07-aefc-e19027cf99e6-operator-scripts\") pod \"nova-cell1-db-create-t49dl\" (UID: \"2bd8667f-0169-4f07-aefc-e19027cf99e6\") " pod="openstack-kuttl-tests/nova-cell1-db-create-t49dl" Dec 03 11:45:16 crc kubenswrapper[5052]: I1203 11:45:16.022422 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/ae7af34b-d086-4215-b568-1bf5b75bbe39-operator-scripts\") pod \"nova-cell0-f13b-account-create-update-p7l6b\" (UID: \"ae7af34b-d086-4215-b568-1bf5b75bbe39\") " pod="openstack-kuttl-tests/nova-cell0-f13b-account-create-update-p7l6b" Dec 03 11:45:16 crc kubenswrapper[5052]: I1203 11:45:16.038952 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-db-create-7mslc" Dec 03 11:45:16 crc kubenswrapper[5052]: I1203 11:45:16.043748 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-8vdpf\" (UniqueName: \"kubernetes.io/projected/2bd8667f-0169-4f07-aefc-e19027cf99e6-kube-api-access-8vdpf\") pod \"nova-cell1-db-create-t49dl\" (UID: \"2bd8667f-0169-4f07-aefc-e19027cf99e6\") " pod="openstack-kuttl-tests/nova-cell1-db-create-t49dl" Dec 03 11:45:16 crc kubenswrapper[5052]: I1203 11:45:16.047944 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-57kzm\" (UniqueName: \"kubernetes.io/projected/ae7af34b-d086-4215-b568-1bf5b75bbe39-kube-api-access-57kzm\") pod \"nova-cell0-f13b-account-create-update-p7l6b\" (UID: \"ae7af34b-d086-4215-b568-1bf5b75bbe39\") " pod="openstack-kuttl-tests/nova-cell0-f13b-account-create-update-p7l6b" Dec 03 11:45:16 crc kubenswrapper[5052]: I1203 11:45:16.053289 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:45:16 crc kubenswrapper[5052]: I1203 11:45:16.096477 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell1-1ea3-account-create-update-4x6kr"] Dec 03 11:45:16 crc kubenswrapper[5052]: I1203 11:45:16.100672 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-1ea3-account-create-update-4x6kr" Dec 03 11:45:16 crc kubenswrapper[5052]: I1203 11:45:16.104565 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell1-db-secret" Dec 03 11:45:16 crc kubenswrapper[5052]: I1203 11:45:16.110657 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-1ea3-account-create-update-4x6kr"] Dec 03 11:45:16 crc kubenswrapper[5052]: I1203 11:45:16.126414 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-5919-account-create-update-wdllt" Dec 03 11:45:16 crc kubenswrapper[5052]: I1203 11:45:16.146632 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/kube-state-metrics-0" event={"ID":"78388bce-bd9d-426d-b6b6-e0fba8b3d8f9","Type":"ContainerDied","Data":"3777b5fc38dd6dccf7902b068ada7954bec5d1b4b67fbe026aa85e7b5f1bf0d3"} Dec 03 11:45:16 crc kubenswrapper[5052]: I1203 11:45:16.147491 5052 scope.go:117] "RemoveContainer" containerID="e284c3d794c0b8aa02b660d2a83e53e3db40486717ae4fa7ff4311373d5644eb" Dec 03 11:45:16 crc kubenswrapper[5052]: I1203 11:45:16.147023 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 11:45:16 crc kubenswrapper[5052]: I1203 11:45:16.174177 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-db-create-t49dl" Dec 03 11:45:16 crc kubenswrapper[5052]: I1203 11:45:16.194309 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/kube-state-metrics-0"] Dec 03 11:45:16 crc kubenswrapper[5052]: I1203 11:45:16.208717 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-f13b-account-create-update-p7l6b" Dec 03 11:45:16 crc kubenswrapper[5052]: I1203 11:45:16.211516 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/kube-state-metrics-0"] Dec 03 11:45:16 crc kubenswrapper[5052]: I1203 11:45:16.223389 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/kube-state-metrics-0"] Dec 03 11:45:16 crc kubenswrapper[5052]: I1203 11:45:16.228407 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tsbrz\" (UniqueName: \"kubernetes.io/projected/2df10f41-fb7a-4a14-94b4-ebdaf98d1d2c-kube-api-access-tsbrz\") pod \"nova-cell1-1ea3-account-create-update-4x6kr\" (UID: \"2df10f41-fb7a-4a14-94b4-ebdaf98d1d2c\") " pod="openstack-kuttl-tests/nova-cell1-1ea3-account-create-update-4x6kr" Dec 03 11:45:16 crc kubenswrapper[5052]: I1203 11:45:16.228458 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/2df10f41-fb7a-4a14-94b4-ebdaf98d1d2c-operator-scripts\") pod \"nova-cell1-1ea3-account-create-update-4x6kr\" (UID: \"2df10f41-fb7a-4a14-94b4-ebdaf98d1d2c\") " pod="openstack-kuttl-tests/nova-cell1-1ea3-account-create-update-4x6kr" Dec 03 11:45:16 crc kubenswrapper[5052]: I1203 11:45:16.232889 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 11:45:16 crc kubenswrapper[5052]: I1203 11:45:16.241427 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-kube-state-metrics-svc" Dec 03 11:45:16 crc kubenswrapper[5052]: I1203 11:45:16.241601 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"kube-state-metrics-tls-config" Dec 03 11:45:16 crc kubenswrapper[5052]: I1203 11:45:16.250403 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/kube-state-metrics-0"] Dec 03 11:45:16 crc kubenswrapper[5052]: I1203 11:45:16.332400 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dm4wk\" (UniqueName: \"kubernetes.io/projected/496a1006-5380-40d1-998f-129a87233300-kube-api-access-dm4wk\") pod \"kube-state-metrics-0\" (UID: \"496a1006-5380-40d1-998f-129a87233300\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 11:45:16 crc kubenswrapper[5052]: I1203 11:45:16.332768 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-tsbrz\" (UniqueName: \"kubernetes.io/projected/2df10f41-fb7a-4a14-94b4-ebdaf98d1d2c-kube-api-access-tsbrz\") pod \"nova-cell1-1ea3-account-create-update-4x6kr\" (UID: \"2df10f41-fb7a-4a14-94b4-ebdaf98d1d2c\") " pod="openstack-kuttl-tests/nova-cell1-1ea3-account-create-update-4x6kr" Dec 03 11:45:16 crc kubenswrapper[5052]: I1203 11:45:16.332829 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/2df10f41-fb7a-4a14-94b4-ebdaf98d1d2c-operator-scripts\") pod \"nova-cell1-1ea3-account-create-update-4x6kr\" (UID: \"2df10f41-fb7a-4a14-94b4-ebdaf98d1d2c\") " pod="openstack-kuttl-tests/nova-cell1-1ea3-account-create-update-4x6kr" Dec 03 11:45:16 crc kubenswrapper[5052]: I1203 11:45:16.332863 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/496a1006-5380-40d1-998f-129a87233300-combined-ca-bundle\") pod \"kube-state-metrics-0\" (UID: \"496a1006-5380-40d1-998f-129a87233300\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 11:45:16 crc kubenswrapper[5052]: I1203 11:45:16.332939 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls-certs\" (UniqueName: \"kubernetes.io/secret/496a1006-5380-40d1-998f-129a87233300-kube-state-metrics-tls-certs\") pod \"kube-state-metrics-0\" (UID: \"496a1006-5380-40d1-998f-129a87233300\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 11:45:16 crc kubenswrapper[5052]: I1203 11:45:16.333020 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls-config\" (UniqueName: \"kubernetes.io/secret/496a1006-5380-40d1-998f-129a87233300-kube-state-metrics-tls-config\") pod \"kube-state-metrics-0\" (UID: \"496a1006-5380-40d1-998f-129a87233300\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 11:45:16 crc kubenswrapper[5052]: I1203 11:45:16.334479 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/2df10f41-fb7a-4a14-94b4-ebdaf98d1d2c-operator-scripts\") pod \"nova-cell1-1ea3-account-create-update-4x6kr\" (UID: \"2df10f41-fb7a-4a14-94b4-ebdaf98d1d2c\") " pod="openstack-kuttl-tests/nova-cell1-1ea3-account-create-update-4x6kr" Dec 03 11:45:16 crc kubenswrapper[5052]: I1203 11:45:16.357543 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-tsbrz\" (UniqueName: \"kubernetes.io/projected/2df10f41-fb7a-4a14-94b4-ebdaf98d1d2c-kube-api-access-tsbrz\") pod \"nova-cell1-1ea3-account-create-update-4x6kr\" (UID: \"2df10f41-fb7a-4a14-94b4-ebdaf98d1d2c\") " pod="openstack-kuttl-tests/nova-cell1-1ea3-account-create-update-4x6kr" Dec 03 11:45:16 crc kubenswrapper[5052]: I1203 11:45:16.434670 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-dm4wk\" (UniqueName: \"kubernetes.io/projected/496a1006-5380-40d1-998f-129a87233300-kube-api-access-dm4wk\") pod \"kube-state-metrics-0\" (UID: \"496a1006-5380-40d1-998f-129a87233300\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 11:45:16 crc kubenswrapper[5052]: I1203 11:45:16.434734 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/496a1006-5380-40d1-998f-129a87233300-combined-ca-bundle\") pod \"kube-state-metrics-0\" (UID: \"496a1006-5380-40d1-998f-129a87233300\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 11:45:16 crc kubenswrapper[5052]: I1203 11:45:16.434774 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls-certs\" (UniqueName: \"kubernetes.io/secret/496a1006-5380-40d1-998f-129a87233300-kube-state-metrics-tls-certs\") pod \"kube-state-metrics-0\" (UID: \"496a1006-5380-40d1-998f-129a87233300\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 11:45:16 crc kubenswrapper[5052]: I1203 11:45:16.434879 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls-config\" (UniqueName: \"kubernetes.io/secret/496a1006-5380-40d1-998f-129a87233300-kube-state-metrics-tls-config\") pod \"kube-state-metrics-0\" (UID: \"496a1006-5380-40d1-998f-129a87233300\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 11:45:16 crc kubenswrapper[5052]: I1203 11:45:16.438525 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-1ea3-account-create-update-4x6kr" Dec 03 11:45:16 crc kubenswrapper[5052]: I1203 11:45:16.441803 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls-config\" (UniqueName: \"kubernetes.io/secret/496a1006-5380-40d1-998f-129a87233300-kube-state-metrics-tls-config\") pod \"kube-state-metrics-0\" (UID: \"496a1006-5380-40d1-998f-129a87233300\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 11:45:16 crc kubenswrapper[5052]: I1203 11:45:16.444232 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls-certs\" (UniqueName: \"kubernetes.io/secret/496a1006-5380-40d1-998f-129a87233300-kube-state-metrics-tls-certs\") pod \"kube-state-metrics-0\" (UID: \"496a1006-5380-40d1-998f-129a87233300\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 11:45:16 crc kubenswrapper[5052]: I1203 11:45:16.446424 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/496a1006-5380-40d1-998f-129a87233300-combined-ca-bundle\") pod \"kube-state-metrics-0\" (UID: \"496a1006-5380-40d1-998f-129a87233300\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 11:45:16 crc kubenswrapper[5052]: I1203 11:45:16.452472 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-db-create-bmscg"] Dec 03 11:45:16 crc kubenswrapper[5052]: I1203 11:45:16.453969 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-dm4wk\" (UniqueName: \"kubernetes.io/projected/496a1006-5380-40d1-998f-129a87233300-kube-api-access-dm4wk\") pod \"kube-state-metrics-0\" (UID: \"496a1006-5380-40d1-998f-129a87233300\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 11:45:16 crc kubenswrapper[5052]: W1203 11:45:16.542914 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6d924068_a887_4c83_89ad_9f333eb94799.slice/crio-bd231c99b7725dd1957882c92bf6936de6f810ec0403a3c836ae0c54adf8e5b6 WatchSource:0}: Error finding container bd231c99b7725dd1957882c92bf6936de6f810ec0403a3c836ae0c54adf8e5b6: Status 404 returned error can't find the container with id bd231c99b7725dd1957882c92bf6936de6f810ec0403a3c836ae0c54adf8e5b6 Dec 03 11:45:16 crc kubenswrapper[5052]: I1203 11:45:16.547488 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-5919-account-create-update-wdllt"] Dec 03 11:45:16 crc kubenswrapper[5052]: I1203 11:45:16.565016 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 11:45:16 crc kubenswrapper[5052]: I1203 11:45:16.664784 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell0-db-create-7mslc"] Dec 03 11:45:16 crc kubenswrapper[5052]: W1203 11:45:16.687965 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2744f411_6860_47c0_a8d5_f52e23e390f3.slice/crio-6cf2258e11c8ee09dd98cb5dc630e55b9f3de4affc68925007154190c532cb63 WatchSource:0}: Error finding container 6cf2258e11c8ee09dd98cb5dc630e55b9f3de4affc68925007154190c532cb63: Status 404 returned error can't find the container with id 6cf2258e11c8ee09dd98cb5dc630e55b9f3de4affc68925007154190c532cb63 Dec 03 11:45:16 crc kubenswrapper[5052]: I1203 11:45:16.819109 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-db-create-t49dl"] Dec 03 11:45:16 crc kubenswrapper[5052]: I1203 11:45:16.838570 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell0-f13b-account-create-update-p7l6b"] Dec 03 11:45:16 crc kubenswrapper[5052]: I1203 11:45:16.971253 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-1ea3-account-create-update-4x6kr"] Dec 03 11:45:16 crc kubenswrapper[5052]: W1203 11:45:16.975622 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2df10f41_fb7a_4a14_94b4_ebdaf98d1d2c.slice/crio-03128fb5ee7a436d672ee5d53ca6948cddd5f232bbb0194f23b19e73c691b45a WatchSource:0}: Error finding container 03128fb5ee7a436d672ee5d53ca6948cddd5f232bbb0194f23b19e73c691b45a: Status 404 returned error can't find the container with id 03128fb5ee7a436d672ee5d53ca6948cddd5f232bbb0194f23b19e73c691b45a Dec 03 11:45:17 crc kubenswrapper[5052]: I1203 11:45:17.091498 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/kube-state-metrics-0"] Dec 03 11:45:17 crc kubenswrapper[5052]: I1203 11:45:17.140012 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:45:17 crc kubenswrapper[5052]: I1203 11:45:17.158319 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-db-create-7mslc" event={"ID":"2744f411-6860-47c0-a8d5-f52e23e390f3","Type":"ContainerStarted","Data":"6cf2258e11c8ee09dd98cb5dc630e55b9f3de4affc68925007154190c532cb63"} Dec 03 11:45:17 crc kubenswrapper[5052]: I1203 11:45:17.159586 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-5919-account-create-update-wdllt" event={"ID":"6d924068-a887-4c83-89ad-9f333eb94799","Type":"ContainerStarted","Data":"bd231c99b7725dd1957882c92bf6936de6f810ec0403a3c836ae0c54adf8e5b6"} Dec 03 11:45:17 crc kubenswrapper[5052]: I1203 11:45:17.161838 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-db-create-t49dl" event={"ID":"2bd8667f-0169-4f07-aefc-e19027cf99e6","Type":"ContainerStarted","Data":"9695f4bf148b5fe7ee4fe04ffab868b43465b3977ef00b2fdb8bd183524afb28"} Dec 03 11:45:17 crc kubenswrapper[5052]: I1203 11:45:17.162911 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-db-create-bmscg" event={"ID":"b85b18b0-5745-44b1-8add-a67d56d1d0c4","Type":"ContainerStarted","Data":"a9d2d58456eee9ce3ffd5d2d41819e673a73d2d497ba5244c84d4d77b075ea55"} Dec 03 11:45:17 crc kubenswrapper[5052]: I1203 11:45:17.163750 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/kube-state-metrics-0" event={"ID":"496a1006-5380-40d1-998f-129a87233300","Type":"ContainerStarted","Data":"767dce3a52d6b21ef52be4b20913500d01a22618603394980337bbfd8eca0b52"} Dec 03 11:45:17 crc kubenswrapper[5052]: I1203 11:45:17.164791 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"73440cd7-672e-4ace-91bd-b380dbcf681f","Type":"ContainerStarted","Data":"aa0f551fe18f4ec003503654accfde30e69465c1239c1cd129ed63b0d7cff116"} Dec 03 11:45:17 crc kubenswrapper[5052]: I1203 11:45:17.167529 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-f13b-account-create-update-p7l6b" event={"ID":"ae7af34b-d086-4215-b568-1bf5b75bbe39","Type":"ContainerStarted","Data":"3d1920e9cee1782e1b6fc7eaff312d29eb5bb78d19606ef93c0dff484496ce23"} Dec 03 11:45:17 crc kubenswrapper[5052]: I1203 11:45:17.168426 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-1ea3-account-create-update-4x6kr" event={"ID":"2df10f41-fb7a-4a14-94b4-ebdaf98d1d2c","Type":"ContainerStarted","Data":"03128fb5ee7a436d672ee5d53ca6948cddd5f232bbb0194f23b19e73c691b45a"} Dec 03 11:45:17 crc kubenswrapper[5052]: I1203 11:45:17.684243 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="78388bce-bd9d-426d-b6b6-e0fba8b3d8f9" path="/var/lib/kubelet/pods/78388bce-bd9d-426d-b6b6-e0fba8b3d8f9/volumes" Dec 03 11:45:18 crc kubenswrapper[5052]: I1203 11:45:18.180604 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-db-create-bmscg" event={"ID":"b85b18b0-5745-44b1-8add-a67d56d1d0c4","Type":"ContainerStarted","Data":"4add9b48367f94bee74008ec0e801ddbd291eb262a94b430d5c2adfb689ac2b9"} Dec 03 11:45:18 crc kubenswrapper[5052]: I1203 11:45:18.184127 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-f13b-account-create-update-p7l6b" event={"ID":"ae7af34b-d086-4215-b568-1bf5b75bbe39","Type":"ContainerStarted","Data":"5bd53afa88afc24e4d88a692fa06eded59d58f9f609240bdbab89ff86313aa6d"} Dec 03 11:45:18 crc kubenswrapper[5052]: I1203 11:45:18.186523 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-1ea3-account-create-update-4x6kr" event={"ID":"2df10f41-fb7a-4a14-94b4-ebdaf98d1d2c","Type":"ContainerStarted","Data":"1b7ece73f44642bcedc96bdc0480c0038d10712c4f5cc1cf1e1c3d04d924e6f9"} Dec 03 11:45:18 crc kubenswrapper[5052]: I1203 11:45:18.189071 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-db-create-7mslc" event={"ID":"2744f411-6860-47c0-a8d5-f52e23e390f3","Type":"ContainerStarted","Data":"18fb8243e49f7c54143f6d0ab14bae1d97d986b3e3dbf996c8186febab430a50"} Dec 03 11:45:18 crc kubenswrapper[5052]: I1203 11:45:18.194478 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-5919-account-create-update-wdllt" event={"ID":"6d924068-a887-4c83-89ad-9f333eb94799","Type":"ContainerStarted","Data":"de182e92eb499a14ca6d41657fcf50d4c07d51d2e7062a139fcdbe3c0e220744"} Dec 03 11:45:18 crc kubenswrapper[5052]: I1203 11:45:18.196301 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-db-create-t49dl" event={"ID":"2bd8667f-0169-4f07-aefc-e19027cf99e6","Type":"ContainerStarted","Data":"d26bbcf3d9f20787b3d596993e0924eb679af95f833778f1f8ad33c49224f4d8"} Dec 03 11:45:18 crc kubenswrapper[5052]: I1203 11:45:18.207124 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-api-db-create-bmscg" podStartSLOduration=3.2070999000000002 podStartE2EDuration="3.2070999s" podCreationTimestamp="2025-12-03 11:45:15 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:45:18.191871221 +0000 UTC m=+1508.780112985" watchObservedRunningTime="2025-12-03 11:45:18.2070999 +0000 UTC m=+1508.795341684" Dec 03 11:45:18 crc kubenswrapper[5052]: I1203 11:45:18.229263 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-cell0-f13b-account-create-update-p7l6b" podStartSLOduration=3.229243279 podStartE2EDuration="3.229243279s" podCreationTimestamp="2025-12-03 11:45:15 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:45:18.225776309 +0000 UTC m=+1508.814018073" watchObservedRunningTime="2025-12-03 11:45:18.229243279 +0000 UTC m=+1508.817485043" Dec 03 11:45:18 crc kubenswrapper[5052]: I1203 11:45:18.248263 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-cell0-db-create-7mslc" podStartSLOduration=3.248243967 podStartE2EDuration="3.248243967s" podCreationTimestamp="2025-12-03 11:45:15 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:45:18.242093309 +0000 UTC m=+1508.830335073" watchObservedRunningTime="2025-12-03 11:45:18.248243967 +0000 UTC m=+1508.836485741" Dec 03 11:45:18 crc kubenswrapper[5052]: I1203 11:45:18.258339 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-cell1-1ea3-account-create-update-4x6kr" podStartSLOduration=2.258319447 podStartE2EDuration="2.258319447s" podCreationTimestamp="2025-12-03 11:45:16 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:45:18.253935161 +0000 UTC m=+1508.842176925" watchObservedRunningTime="2025-12-03 11:45:18.258319447 +0000 UTC m=+1508.846561211" Dec 03 11:45:18 crc kubenswrapper[5052]: I1203 11:45:18.324362 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-api-5919-account-create-update-wdllt" podStartSLOduration=3.324340061 podStartE2EDuration="3.324340061s" podCreationTimestamp="2025-12-03 11:45:15 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:45:18.281776184 +0000 UTC m=+1508.870017948" watchObservedRunningTime="2025-12-03 11:45:18.324340061 +0000 UTC m=+1508.912581825" Dec 03 11:45:19 crc kubenswrapper[5052]: I1203 11:45:19.208293 5052 generic.go:334] "Generic (PLEG): container finished" podID="2bd8667f-0169-4f07-aefc-e19027cf99e6" containerID="d26bbcf3d9f20787b3d596993e0924eb679af95f833778f1f8ad33c49224f4d8" exitCode=0 Dec 03 11:45:19 crc kubenswrapper[5052]: I1203 11:45:19.208389 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-db-create-t49dl" event={"ID":"2bd8667f-0169-4f07-aefc-e19027cf99e6","Type":"ContainerDied","Data":"d26bbcf3d9f20787b3d596993e0924eb679af95f833778f1f8ad33c49224f4d8"} Dec 03 11:45:19 crc kubenswrapper[5052]: I1203 11:45:19.212795 5052 generic.go:334] "Generic (PLEG): container finished" podID="b85b18b0-5745-44b1-8add-a67d56d1d0c4" containerID="4add9b48367f94bee74008ec0e801ddbd291eb262a94b430d5c2adfb689ac2b9" exitCode=0 Dec 03 11:45:19 crc kubenswrapper[5052]: I1203 11:45:19.212839 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-db-create-bmscg" event={"ID":"b85b18b0-5745-44b1-8add-a67d56d1d0c4","Type":"ContainerDied","Data":"4add9b48367f94bee74008ec0e801ddbd291eb262a94b430d5c2adfb689ac2b9"} Dec 03 11:45:19 crc kubenswrapper[5052]: I1203 11:45:19.222124 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"73440cd7-672e-4ace-91bd-b380dbcf681f","Type":"ContainerStarted","Data":"1acb2031482f91324fd2b811f3a7715d4cef822d78733b2935b50623edcb4c12"} Dec 03 11:45:19 crc kubenswrapper[5052]: I1203 11:45:19.224823 5052 generic.go:334] "Generic (PLEG): container finished" podID="2df10f41-fb7a-4a14-94b4-ebdaf98d1d2c" containerID="1b7ece73f44642bcedc96bdc0480c0038d10712c4f5cc1cf1e1c3d04d924e6f9" exitCode=0 Dec 03 11:45:19 crc kubenswrapper[5052]: I1203 11:45:19.224929 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-1ea3-account-create-update-4x6kr" event={"ID":"2df10f41-fb7a-4a14-94b4-ebdaf98d1d2c","Type":"ContainerDied","Data":"1b7ece73f44642bcedc96bdc0480c0038d10712c4f5cc1cf1e1c3d04d924e6f9"} Dec 03 11:45:19 crc kubenswrapper[5052]: I1203 11:45:19.226879 5052 generic.go:334] "Generic (PLEG): container finished" podID="2744f411-6860-47c0-a8d5-f52e23e390f3" containerID="18fb8243e49f7c54143f6d0ab14bae1d97d986b3e3dbf996c8186febab430a50" exitCode=0 Dec 03 11:45:19 crc kubenswrapper[5052]: I1203 11:45:19.226949 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-db-create-7mslc" event={"ID":"2744f411-6860-47c0-a8d5-f52e23e390f3","Type":"ContainerDied","Data":"18fb8243e49f7c54143f6d0ab14bae1d97d986b3e3dbf996c8186febab430a50"} Dec 03 11:45:19 crc kubenswrapper[5052]: I1203 11:45:19.242815 5052 generic.go:334] "Generic (PLEG): container finished" podID="f0cc2483-7e43-4536-8d99-a46d422051dd" containerID="5356857ea35fe536b478870e1971c96b2cba923334de13d426d220aa943ef72d" exitCode=0 Dec 03 11:45:19 crc kubenswrapper[5052]: I1203 11:45:19.243994 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/community-operators-fvb75" event={"ID":"f0cc2483-7e43-4536-8d99-a46d422051dd","Type":"ContainerDied","Data":"5356857ea35fe536b478870e1971c96b2cba923334de13d426d220aa943ef72d"} Dec 03 11:45:19 crc kubenswrapper[5052]: I1203 11:45:19.642299 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-db-create-t49dl" Dec 03 11:45:19 crc kubenswrapper[5052]: I1203 11:45:19.704986 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/swift-proxy-6fdf66df4-gsqlq" Dec 03 11:45:19 crc kubenswrapper[5052]: I1203 11:45:19.706676 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/swift-proxy-6fdf66df4-gsqlq" Dec 03 11:45:19 crc kubenswrapper[5052]: I1203 11:45:19.714875 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8vdpf\" (UniqueName: \"kubernetes.io/projected/2bd8667f-0169-4f07-aefc-e19027cf99e6-kube-api-access-8vdpf\") pod \"2bd8667f-0169-4f07-aefc-e19027cf99e6\" (UID: \"2bd8667f-0169-4f07-aefc-e19027cf99e6\") " Dec 03 11:45:19 crc kubenswrapper[5052]: I1203 11:45:19.714973 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/2bd8667f-0169-4f07-aefc-e19027cf99e6-operator-scripts\") pod \"2bd8667f-0169-4f07-aefc-e19027cf99e6\" (UID: \"2bd8667f-0169-4f07-aefc-e19027cf99e6\") " Dec 03 11:45:19 crc kubenswrapper[5052]: I1203 11:45:19.715686 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2bd8667f-0169-4f07-aefc-e19027cf99e6-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "2bd8667f-0169-4f07-aefc-e19027cf99e6" (UID: "2bd8667f-0169-4f07-aefc-e19027cf99e6"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:45:19 crc kubenswrapper[5052]: I1203 11:45:19.723725 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2bd8667f-0169-4f07-aefc-e19027cf99e6-kube-api-access-8vdpf" (OuterVolumeSpecName: "kube-api-access-8vdpf") pod "2bd8667f-0169-4f07-aefc-e19027cf99e6" (UID: "2bd8667f-0169-4f07-aefc-e19027cf99e6"). InnerVolumeSpecName "kube-api-access-8vdpf". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:45:19 crc kubenswrapper[5052]: I1203 11:45:19.825779 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-8vdpf\" (UniqueName: \"kubernetes.io/projected/2bd8667f-0169-4f07-aefc-e19027cf99e6-kube-api-access-8vdpf\") on node \"crc\" DevicePath \"\"" Dec 03 11:45:19 crc kubenswrapper[5052]: I1203 11:45:19.825815 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/2bd8667f-0169-4f07-aefc-e19027cf99e6-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:45:20 crc kubenswrapper[5052]: I1203 11:45:20.261749 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-db-create-t49dl" event={"ID":"2bd8667f-0169-4f07-aefc-e19027cf99e6","Type":"ContainerDied","Data":"9695f4bf148b5fe7ee4fe04ffab868b43465b3977ef00b2fdb8bd183524afb28"} Dec 03 11:45:20 crc kubenswrapper[5052]: I1203 11:45:20.262035 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9695f4bf148b5fe7ee4fe04ffab868b43465b3977ef00b2fdb8bd183524afb28" Dec 03 11:45:20 crc kubenswrapper[5052]: I1203 11:45:20.261784 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-db-create-t49dl" Dec 03 11:45:20 crc kubenswrapper[5052]: I1203 11:45:20.265233 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/kube-state-metrics-0" event={"ID":"496a1006-5380-40d1-998f-129a87233300","Type":"ContainerStarted","Data":"64d2ed8863bdee50690f9bd22bda4e036b2111ebb9734c4223ad439fd9d3b384"} Dec 03 11:45:20 crc kubenswrapper[5052]: I1203 11:45:20.265636 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 11:45:20 crc kubenswrapper[5052]: I1203 11:45:20.299479 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/kube-state-metrics-0" podStartSLOduration=2.602938254 podStartE2EDuration="4.299456107s" podCreationTimestamp="2025-12-03 11:45:16 +0000 UTC" firstStartedPulling="2025-12-03 11:45:17.098770159 +0000 UTC m=+1507.687011923" lastFinishedPulling="2025-12-03 11:45:18.795288012 +0000 UTC m=+1509.383529776" observedRunningTime="2025-12-03 11:45:20.290990663 +0000 UTC m=+1510.879232437" watchObservedRunningTime="2025-12-03 11:45:20.299456107 +0000 UTC m=+1510.887697871" Dec 03 11:45:20 crc kubenswrapper[5052]: I1203 11:45:20.680805 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-db-create-7mslc" Dec 03 11:45:20 crc kubenswrapper[5052]: I1203 11:45:20.846491 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/2744f411-6860-47c0-a8d5-f52e23e390f3-operator-scripts\") pod \"2744f411-6860-47c0-a8d5-f52e23e390f3\" (UID: \"2744f411-6860-47c0-a8d5-f52e23e390f3\") " Dec 03 11:45:20 crc kubenswrapper[5052]: I1203 11:45:20.846678 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c6md6\" (UniqueName: \"kubernetes.io/projected/2744f411-6860-47c0-a8d5-f52e23e390f3-kube-api-access-c6md6\") pod \"2744f411-6860-47c0-a8d5-f52e23e390f3\" (UID: \"2744f411-6860-47c0-a8d5-f52e23e390f3\") " Dec 03 11:45:20 crc kubenswrapper[5052]: I1203 11:45:20.847974 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2744f411-6860-47c0-a8d5-f52e23e390f3-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "2744f411-6860-47c0-a8d5-f52e23e390f3" (UID: "2744f411-6860-47c0-a8d5-f52e23e390f3"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:45:20 crc kubenswrapper[5052]: I1203 11:45:20.898684 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2744f411-6860-47c0-a8d5-f52e23e390f3-kube-api-access-c6md6" (OuterVolumeSpecName: "kube-api-access-c6md6") pod "2744f411-6860-47c0-a8d5-f52e23e390f3" (UID: "2744f411-6860-47c0-a8d5-f52e23e390f3"). InnerVolumeSpecName "kube-api-access-c6md6". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:45:20 crc kubenswrapper[5052]: I1203 11:45:20.948834 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-c6md6\" (UniqueName: \"kubernetes.io/projected/2744f411-6860-47c0-a8d5-f52e23e390f3-kube-api-access-c6md6\") on node \"crc\" DevicePath \"\"" Dec 03 11:45:20 crc kubenswrapper[5052]: I1203 11:45:20.948868 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/2744f411-6860-47c0-a8d5-f52e23e390f3-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:45:21 crc kubenswrapper[5052]: I1203 11:45:21.237174 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-1ea3-account-create-update-4x6kr" Dec 03 11:45:21 crc kubenswrapper[5052]: I1203 11:45:21.244046 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-db-create-bmscg" Dec 03 11:45:21 crc kubenswrapper[5052]: I1203 11:45:21.319271 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-db-create-bmscg" event={"ID":"b85b18b0-5745-44b1-8add-a67d56d1d0c4","Type":"ContainerDied","Data":"a9d2d58456eee9ce3ffd5d2d41819e673a73d2d497ba5244c84d4d77b075ea55"} Dec 03 11:45:21 crc kubenswrapper[5052]: I1203 11:45:21.319322 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a9d2d58456eee9ce3ffd5d2d41819e673a73d2d497ba5244c84d4d77b075ea55" Dec 03 11:45:21 crc kubenswrapper[5052]: I1203 11:45:21.319299 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-db-create-bmscg" Dec 03 11:45:21 crc kubenswrapper[5052]: I1203 11:45:21.321049 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"73440cd7-672e-4ace-91bd-b380dbcf681f","Type":"ContainerStarted","Data":"350bc5605a247a6dbdb1d22ed1c84ac14dd38e149676586bc392fb27c359ae86"} Dec 03 11:45:21 crc kubenswrapper[5052]: I1203 11:45:21.322391 5052 generic.go:334] "Generic (PLEG): container finished" podID="ae7af34b-d086-4215-b568-1bf5b75bbe39" containerID="5bd53afa88afc24e4d88a692fa06eded59d58f9f609240bdbab89ff86313aa6d" exitCode=0 Dec 03 11:45:21 crc kubenswrapper[5052]: I1203 11:45:21.322429 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-f13b-account-create-update-p7l6b" event={"ID":"ae7af34b-d086-4215-b568-1bf5b75bbe39","Type":"ContainerDied","Data":"5bd53afa88afc24e4d88a692fa06eded59d58f9f609240bdbab89ff86313aa6d"} Dec 03 11:45:21 crc kubenswrapper[5052]: I1203 11:45:21.323756 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-1ea3-account-create-update-4x6kr" event={"ID":"2df10f41-fb7a-4a14-94b4-ebdaf98d1d2c","Type":"ContainerDied","Data":"03128fb5ee7a436d672ee5d53ca6948cddd5f232bbb0194f23b19e73c691b45a"} Dec 03 11:45:21 crc kubenswrapper[5052]: I1203 11:45:21.323788 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="03128fb5ee7a436d672ee5d53ca6948cddd5f232bbb0194f23b19e73c691b45a" Dec 03 11:45:21 crc kubenswrapper[5052]: I1203 11:45:21.323845 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-1ea3-account-create-update-4x6kr" Dec 03 11:45:21 crc kubenswrapper[5052]: I1203 11:45:21.326319 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-db-create-7mslc" event={"ID":"2744f411-6860-47c0-a8d5-f52e23e390f3","Type":"ContainerDied","Data":"6cf2258e11c8ee09dd98cb5dc630e55b9f3de4affc68925007154190c532cb63"} Dec 03 11:45:21 crc kubenswrapper[5052]: I1203 11:45:21.326368 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6cf2258e11c8ee09dd98cb5dc630e55b9f3de4affc68925007154190c532cb63" Dec 03 11:45:21 crc kubenswrapper[5052]: I1203 11:45:21.326447 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-db-create-7mslc" Dec 03 11:45:21 crc kubenswrapper[5052]: I1203 11:45:21.327757 5052 generic.go:334] "Generic (PLEG): container finished" podID="6d924068-a887-4c83-89ad-9f333eb94799" containerID="de182e92eb499a14ca6d41657fcf50d4c07d51d2e7062a139fcdbe3c0e220744" exitCode=0 Dec 03 11:45:21 crc kubenswrapper[5052]: I1203 11:45:21.328791 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-5919-account-create-update-wdllt" event={"ID":"6d924068-a887-4c83-89ad-9f333eb94799","Type":"ContainerDied","Data":"de182e92eb499a14ca6d41657fcf50d4c07d51d2e7062a139fcdbe3c0e220744"} Dec 03 11:45:21 crc kubenswrapper[5052]: I1203 11:45:21.361171 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tsbrz\" (UniqueName: \"kubernetes.io/projected/2df10f41-fb7a-4a14-94b4-ebdaf98d1d2c-kube-api-access-tsbrz\") pod \"2df10f41-fb7a-4a14-94b4-ebdaf98d1d2c\" (UID: \"2df10f41-fb7a-4a14-94b4-ebdaf98d1d2c\") " Dec 03 11:45:21 crc kubenswrapper[5052]: I1203 11:45:21.361288 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/2df10f41-fb7a-4a14-94b4-ebdaf98d1d2c-operator-scripts\") pod \"2df10f41-fb7a-4a14-94b4-ebdaf98d1d2c\" (UID: \"2df10f41-fb7a-4a14-94b4-ebdaf98d1d2c\") " Dec 03 11:45:21 crc kubenswrapper[5052]: I1203 11:45:21.361419 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/b85b18b0-5745-44b1-8add-a67d56d1d0c4-operator-scripts\") pod \"b85b18b0-5745-44b1-8add-a67d56d1d0c4\" (UID: \"b85b18b0-5745-44b1-8add-a67d56d1d0c4\") " Dec 03 11:45:21 crc kubenswrapper[5052]: I1203 11:45:21.361449 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jsw6l\" (UniqueName: \"kubernetes.io/projected/b85b18b0-5745-44b1-8add-a67d56d1d0c4-kube-api-access-jsw6l\") pod \"b85b18b0-5745-44b1-8add-a67d56d1d0c4\" (UID: \"b85b18b0-5745-44b1-8add-a67d56d1d0c4\") " Dec 03 11:45:21 crc kubenswrapper[5052]: I1203 11:45:21.361875 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2df10f41-fb7a-4a14-94b4-ebdaf98d1d2c-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "2df10f41-fb7a-4a14-94b4-ebdaf98d1d2c" (UID: "2df10f41-fb7a-4a14-94b4-ebdaf98d1d2c"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:45:21 crc kubenswrapper[5052]: I1203 11:45:21.362120 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/2df10f41-fb7a-4a14-94b4-ebdaf98d1d2c-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:45:21 crc kubenswrapper[5052]: I1203 11:45:21.362163 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b85b18b0-5745-44b1-8add-a67d56d1d0c4-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "b85b18b0-5745-44b1-8add-a67d56d1d0c4" (UID: "b85b18b0-5745-44b1-8add-a67d56d1d0c4"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:45:21 crc kubenswrapper[5052]: I1203 11:45:21.366420 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2df10f41-fb7a-4a14-94b4-ebdaf98d1d2c-kube-api-access-tsbrz" (OuterVolumeSpecName: "kube-api-access-tsbrz") pod "2df10f41-fb7a-4a14-94b4-ebdaf98d1d2c" (UID: "2df10f41-fb7a-4a14-94b4-ebdaf98d1d2c"). InnerVolumeSpecName "kube-api-access-tsbrz". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:45:21 crc kubenswrapper[5052]: I1203 11:45:21.389704 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b85b18b0-5745-44b1-8add-a67d56d1d0c4-kube-api-access-jsw6l" (OuterVolumeSpecName: "kube-api-access-jsw6l") pod "b85b18b0-5745-44b1-8add-a67d56d1d0c4" (UID: "b85b18b0-5745-44b1-8add-a67d56d1d0c4"). InnerVolumeSpecName "kube-api-access-jsw6l". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:45:21 crc kubenswrapper[5052]: I1203 11:45:21.463789 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/b85b18b0-5745-44b1-8add-a67d56d1d0c4-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:45:21 crc kubenswrapper[5052]: I1203 11:45:21.463846 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-jsw6l\" (UniqueName: \"kubernetes.io/projected/b85b18b0-5745-44b1-8add-a67d56d1d0c4-kube-api-access-jsw6l\") on node \"crc\" DevicePath \"\"" Dec 03 11:45:21 crc kubenswrapper[5052]: I1203 11:45:21.463858 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-tsbrz\" (UniqueName: \"kubernetes.io/projected/2df10f41-fb7a-4a14-94b4-ebdaf98d1d2c-kube-api-access-tsbrz\") on node \"crc\" DevicePath \"\"" Dec 03 11:45:22 crc kubenswrapper[5052]: I1203 11:45:22.340097 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/community-operators-fvb75" event={"ID":"f0cc2483-7e43-4536-8d99-a46d422051dd","Type":"ContainerStarted","Data":"8cc655ae1ba586fa750bed18700ed581d9783b3196c69bb1263ff10c0a967e6f"} Dec 03 11:45:22 crc kubenswrapper[5052]: I1203 11:45:22.827280 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 11:45:22 crc kubenswrapper[5052]: I1203 11:45:22.827795 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-external-api-0" podUID="1303e563-d0c3-4f9c-84b7-3e4fdeba3c6f" containerName="glance-log" containerID="cri-o://877f33b3229c729de1c28b12bb78472817b121f3f37fd1b975fd4575ebc88b59" gracePeriod=30 Dec 03 11:45:22 crc kubenswrapper[5052]: I1203 11:45:22.828011 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-external-api-0" podUID="1303e563-d0c3-4f9c-84b7-3e4fdeba3c6f" containerName="glance-httpd" containerID="cri-o://8eb9f63d2e238cddda381da8dd95d8d4b6eb3913bb5cdce6127df175953100c7" gracePeriod=30 Dec 03 11:45:22 crc kubenswrapper[5052]: I1203 11:45:22.914388 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-f13b-account-create-update-p7l6b" Dec 03 11:45:22 crc kubenswrapper[5052]: I1203 11:45:22.920479 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-5919-account-create-update-wdllt" Dec 03 11:45:23 crc kubenswrapper[5052]: I1203 11:45:23.077052 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/6d924068-a887-4c83-89ad-9f333eb94799-operator-scripts\") pod \"6d924068-a887-4c83-89ad-9f333eb94799\" (UID: \"6d924068-a887-4c83-89ad-9f333eb94799\") " Dec 03 11:45:23 crc kubenswrapper[5052]: I1203 11:45:23.077147 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-57kzm\" (UniqueName: \"kubernetes.io/projected/ae7af34b-d086-4215-b568-1bf5b75bbe39-kube-api-access-57kzm\") pod \"ae7af34b-d086-4215-b568-1bf5b75bbe39\" (UID: \"ae7af34b-d086-4215-b568-1bf5b75bbe39\") " Dec 03 11:45:23 crc kubenswrapper[5052]: I1203 11:45:23.077216 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/ae7af34b-d086-4215-b568-1bf5b75bbe39-operator-scripts\") pod \"ae7af34b-d086-4215-b568-1bf5b75bbe39\" (UID: \"ae7af34b-d086-4215-b568-1bf5b75bbe39\") " Dec 03 11:45:23 crc kubenswrapper[5052]: I1203 11:45:23.077271 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c4jgv\" (UniqueName: \"kubernetes.io/projected/6d924068-a887-4c83-89ad-9f333eb94799-kube-api-access-c4jgv\") pod \"6d924068-a887-4c83-89ad-9f333eb94799\" (UID: \"6d924068-a887-4c83-89ad-9f333eb94799\") " Dec 03 11:45:23 crc kubenswrapper[5052]: I1203 11:45:23.077762 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6d924068-a887-4c83-89ad-9f333eb94799-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "6d924068-a887-4c83-89ad-9f333eb94799" (UID: "6d924068-a887-4c83-89ad-9f333eb94799"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:45:23 crc kubenswrapper[5052]: I1203 11:45:23.077848 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ae7af34b-d086-4215-b568-1bf5b75bbe39-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "ae7af34b-d086-4215-b568-1bf5b75bbe39" (UID: "ae7af34b-d086-4215-b568-1bf5b75bbe39"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:45:23 crc kubenswrapper[5052]: I1203 11:45:23.083752 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6d924068-a887-4c83-89ad-9f333eb94799-kube-api-access-c4jgv" (OuterVolumeSpecName: "kube-api-access-c4jgv") pod "6d924068-a887-4c83-89ad-9f333eb94799" (UID: "6d924068-a887-4c83-89ad-9f333eb94799"). InnerVolumeSpecName "kube-api-access-c4jgv". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:45:23 crc kubenswrapper[5052]: I1203 11:45:23.084773 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ae7af34b-d086-4215-b568-1bf5b75bbe39-kube-api-access-57kzm" (OuterVolumeSpecName: "kube-api-access-57kzm") pod "ae7af34b-d086-4215-b568-1bf5b75bbe39" (UID: "ae7af34b-d086-4215-b568-1bf5b75bbe39"). InnerVolumeSpecName "kube-api-access-57kzm". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:45:23 crc kubenswrapper[5052]: I1203 11:45:23.179526 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/6d924068-a887-4c83-89ad-9f333eb94799-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:45:23 crc kubenswrapper[5052]: I1203 11:45:23.179775 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-57kzm\" (UniqueName: \"kubernetes.io/projected/ae7af34b-d086-4215-b568-1bf5b75bbe39-kube-api-access-57kzm\") on node \"crc\" DevicePath \"\"" Dec 03 11:45:23 crc kubenswrapper[5052]: I1203 11:45:23.179855 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/ae7af34b-d086-4215-b568-1bf5b75bbe39-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:45:23 crc kubenswrapper[5052]: I1203 11:45:23.179936 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-c4jgv\" (UniqueName: \"kubernetes.io/projected/6d924068-a887-4c83-89ad-9f333eb94799-kube-api-access-c4jgv\") on node \"crc\" DevicePath \"\"" Dec 03 11:45:23 crc kubenswrapper[5052]: I1203 11:45:23.350603 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-f13b-account-create-update-p7l6b" event={"ID":"ae7af34b-d086-4215-b568-1bf5b75bbe39","Type":"ContainerDied","Data":"3d1920e9cee1782e1b6fc7eaff312d29eb5bb78d19606ef93c0dff484496ce23"} Dec 03 11:45:23 crc kubenswrapper[5052]: I1203 11:45:23.350650 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3d1920e9cee1782e1b6fc7eaff312d29eb5bb78d19606ef93c0dff484496ce23" Dec 03 11:45:23 crc kubenswrapper[5052]: I1203 11:45:23.350611 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-f13b-account-create-update-p7l6b" Dec 03 11:45:23 crc kubenswrapper[5052]: I1203 11:45:23.360403 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-5919-account-create-update-wdllt" event={"ID":"6d924068-a887-4c83-89ad-9f333eb94799","Type":"ContainerDied","Data":"bd231c99b7725dd1957882c92bf6936de6f810ec0403a3c836ae0c54adf8e5b6"} Dec 03 11:45:23 crc kubenswrapper[5052]: I1203 11:45:23.360453 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bd231c99b7725dd1957882c92bf6936de6f810ec0403a3c836ae0c54adf8e5b6" Dec 03 11:45:23 crc kubenswrapper[5052]: I1203 11:45:23.360541 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-5919-account-create-update-wdllt" Dec 03 11:45:23 crc kubenswrapper[5052]: I1203 11:45:23.714453 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 11:45:23 crc kubenswrapper[5052]: I1203 11:45:23.714750 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-internal-api-0" podUID="d2650df9-f097-4bb1-84e1-72077a661399" containerName="glance-log" containerID="cri-o://0ceade6cc9a65e0e414c39b52e672b86796c8cf5e58f3884914e32309bc0607b" gracePeriod=30 Dec 03 11:45:23 crc kubenswrapper[5052]: I1203 11:45:23.714870 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-internal-api-0" podUID="d2650df9-f097-4bb1-84e1-72077a661399" containerName="glance-httpd" containerID="cri-o://725bc621142a24553c3e356f496feb688f1c45840f857a725ee2ca892589de8b" gracePeriod=30 Dec 03 11:45:24 crc kubenswrapper[5052]: I1203 11:45:24.371907 5052 generic.go:334] "Generic (PLEG): container finished" podID="1303e563-d0c3-4f9c-84b7-3e4fdeba3c6f" containerID="877f33b3229c729de1c28b12bb78472817b121f3f37fd1b975fd4575ebc88b59" exitCode=143 Dec 03 11:45:24 crc kubenswrapper[5052]: I1203 11:45:24.371956 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"1303e563-d0c3-4f9c-84b7-3e4fdeba3c6f","Type":"ContainerDied","Data":"877f33b3229c729de1c28b12bb78472817b121f3f37fd1b975fd4575ebc88b59"} Dec 03 11:45:25 crc kubenswrapper[5052]: I1203 11:45:25.422716 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-marketplace/community-operators-fvb75" podStartSLOduration=7.500713994 podStartE2EDuration="13.422693925s" podCreationTimestamp="2025-12-03 11:45:12 +0000 UTC" firstStartedPulling="2025-12-03 11:45:15.071586862 +0000 UTC m=+1505.659828636" lastFinishedPulling="2025-12-03 11:45:20.993566803 +0000 UTC m=+1511.581808567" observedRunningTime="2025-12-03 11:45:25.409839944 +0000 UTC m=+1515.998081708" watchObservedRunningTime="2025-12-03 11:45:25.422693925 +0000 UTC m=+1516.010935689" Dec 03 11:45:26 crc kubenswrapper[5052]: I1203 11:45:26.142328 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-db-sync-42jqd"] Dec 03 11:45:26 crc kubenswrapper[5052]: E1203 11:45:26.143059 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="2df10f41-fb7a-4a14-94b4-ebdaf98d1d2c" containerName="mariadb-account-create-update" Dec 03 11:45:26 crc kubenswrapper[5052]: I1203 11:45:26.143078 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="2df10f41-fb7a-4a14-94b4-ebdaf98d1d2c" containerName="mariadb-account-create-update" Dec 03 11:45:26 crc kubenswrapper[5052]: E1203 11:45:26.143100 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="2744f411-6860-47c0-a8d5-f52e23e390f3" containerName="mariadb-database-create" Dec 03 11:45:26 crc kubenswrapper[5052]: I1203 11:45:26.143107 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="2744f411-6860-47c0-a8d5-f52e23e390f3" containerName="mariadb-database-create" Dec 03 11:45:26 crc kubenswrapper[5052]: E1203 11:45:26.143123 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="6d924068-a887-4c83-89ad-9f333eb94799" containerName="mariadb-account-create-update" Dec 03 11:45:26 crc kubenswrapper[5052]: I1203 11:45:26.143129 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="6d924068-a887-4c83-89ad-9f333eb94799" containerName="mariadb-account-create-update" Dec 03 11:45:26 crc kubenswrapper[5052]: E1203 11:45:26.143141 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="ae7af34b-d086-4215-b568-1bf5b75bbe39" containerName="mariadb-account-create-update" Dec 03 11:45:26 crc kubenswrapper[5052]: I1203 11:45:26.143146 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae7af34b-d086-4215-b568-1bf5b75bbe39" containerName="mariadb-account-create-update" Dec 03 11:45:26 crc kubenswrapper[5052]: E1203 11:45:26.143157 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="b85b18b0-5745-44b1-8add-a67d56d1d0c4" containerName="mariadb-database-create" Dec 03 11:45:26 crc kubenswrapper[5052]: I1203 11:45:26.143163 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="b85b18b0-5745-44b1-8add-a67d56d1d0c4" containerName="mariadb-database-create" Dec 03 11:45:26 crc kubenswrapper[5052]: E1203 11:45:26.143174 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="2bd8667f-0169-4f07-aefc-e19027cf99e6" containerName="mariadb-database-create" Dec 03 11:45:26 crc kubenswrapper[5052]: I1203 11:45:26.143183 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="2bd8667f-0169-4f07-aefc-e19027cf99e6" containerName="mariadb-database-create" Dec 03 11:45:26 crc kubenswrapper[5052]: I1203 11:45:26.143360 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="6d924068-a887-4c83-89ad-9f333eb94799" containerName="mariadb-account-create-update" Dec 03 11:45:26 crc kubenswrapper[5052]: I1203 11:45:26.143373 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="2df10f41-fb7a-4a14-94b4-ebdaf98d1d2c" containerName="mariadb-account-create-update" Dec 03 11:45:26 crc kubenswrapper[5052]: I1203 11:45:26.143385 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="2744f411-6860-47c0-a8d5-f52e23e390f3" containerName="mariadb-database-create" Dec 03 11:45:26 crc kubenswrapper[5052]: I1203 11:45:26.143396 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="ae7af34b-d086-4215-b568-1bf5b75bbe39" containerName="mariadb-account-create-update" Dec 03 11:45:26 crc kubenswrapper[5052]: I1203 11:45:26.143405 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="2bd8667f-0169-4f07-aefc-e19027cf99e6" containerName="mariadb-database-create" Dec 03 11:45:26 crc kubenswrapper[5052]: I1203 11:45:26.143417 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="b85b18b0-5745-44b1-8add-a67d56d1d0c4" containerName="mariadb-database-create" Dec 03 11:45:26 crc kubenswrapper[5052]: I1203 11:45:26.144096 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-42jqd" Dec 03 11:45:26 crc kubenswrapper[5052]: I1203 11:45:26.147401 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-nova-dockercfg-q4mw9" Dec 03 11:45:26 crc kubenswrapper[5052]: I1203 11:45:26.147727 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell0-conductor-scripts" Dec 03 11:45:26 crc kubenswrapper[5052]: I1203 11:45:26.147854 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell0-conductor-config-data" Dec 03 11:45:26 crc kubenswrapper[5052]: I1203 11:45:26.154949 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-db-sync-42jqd"] Dec 03 11:45:26 crc kubenswrapper[5052]: I1203 11:45:26.240450 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/8dcfd9c8-2ef8-44a2-ad54-5382d56c630d-config-data\") pod \"nova-cell0-conductor-db-sync-42jqd\" (UID: \"8dcfd9c8-2ef8-44a2-ad54-5382d56c630d\") " pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-42jqd" Dec 03 11:45:26 crc kubenswrapper[5052]: I1203 11:45:26.240549 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/8dcfd9c8-2ef8-44a2-ad54-5382d56c630d-scripts\") pod \"nova-cell0-conductor-db-sync-42jqd\" (UID: \"8dcfd9c8-2ef8-44a2-ad54-5382d56c630d\") " pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-42jqd" Dec 03 11:45:26 crc kubenswrapper[5052]: I1203 11:45:26.240797 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8dcfd9c8-2ef8-44a2-ad54-5382d56c630d-combined-ca-bundle\") pod \"nova-cell0-conductor-db-sync-42jqd\" (UID: \"8dcfd9c8-2ef8-44a2-ad54-5382d56c630d\") " pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-42jqd" Dec 03 11:45:26 crc kubenswrapper[5052]: I1203 11:45:26.240856 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zhrr7\" (UniqueName: \"kubernetes.io/projected/8dcfd9c8-2ef8-44a2-ad54-5382d56c630d-kube-api-access-zhrr7\") pod \"nova-cell0-conductor-db-sync-42jqd\" (UID: \"8dcfd9c8-2ef8-44a2-ad54-5382d56c630d\") " pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-42jqd" Dec 03 11:45:26 crc kubenswrapper[5052]: I1203 11:45:26.342194 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/8dcfd9c8-2ef8-44a2-ad54-5382d56c630d-config-data\") pod \"nova-cell0-conductor-db-sync-42jqd\" (UID: \"8dcfd9c8-2ef8-44a2-ad54-5382d56c630d\") " pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-42jqd" Dec 03 11:45:26 crc kubenswrapper[5052]: I1203 11:45:26.342272 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/8dcfd9c8-2ef8-44a2-ad54-5382d56c630d-scripts\") pod \"nova-cell0-conductor-db-sync-42jqd\" (UID: \"8dcfd9c8-2ef8-44a2-ad54-5382d56c630d\") " pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-42jqd" Dec 03 11:45:26 crc kubenswrapper[5052]: I1203 11:45:26.342406 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8dcfd9c8-2ef8-44a2-ad54-5382d56c630d-combined-ca-bundle\") pod \"nova-cell0-conductor-db-sync-42jqd\" (UID: \"8dcfd9c8-2ef8-44a2-ad54-5382d56c630d\") " pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-42jqd" Dec 03 11:45:26 crc kubenswrapper[5052]: I1203 11:45:26.342442 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-zhrr7\" (UniqueName: \"kubernetes.io/projected/8dcfd9c8-2ef8-44a2-ad54-5382d56c630d-kube-api-access-zhrr7\") pod \"nova-cell0-conductor-db-sync-42jqd\" (UID: \"8dcfd9c8-2ef8-44a2-ad54-5382d56c630d\") " pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-42jqd" Dec 03 11:45:26 crc kubenswrapper[5052]: I1203 11:45:26.347210 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8dcfd9c8-2ef8-44a2-ad54-5382d56c630d-combined-ca-bundle\") pod \"nova-cell0-conductor-db-sync-42jqd\" (UID: \"8dcfd9c8-2ef8-44a2-ad54-5382d56c630d\") " pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-42jqd" Dec 03 11:45:26 crc kubenswrapper[5052]: I1203 11:45:26.348269 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/8dcfd9c8-2ef8-44a2-ad54-5382d56c630d-config-data\") pod \"nova-cell0-conductor-db-sync-42jqd\" (UID: \"8dcfd9c8-2ef8-44a2-ad54-5382d56c630d\") " pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-42jqd" Dec 03 11:45:26 crc kubenswrapper[5052]: I1203 11:45:26.351732 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/8dcfd9c8-2ef8-44a2-ad54-5382d56c630d-scripts\") pod \"nova-cell0-conductor-db-sync-42jqd\" (UID: \"8dcfd9c8-2ef8-44a2-ad54-5382d56c630d\") " pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-42jqd" Dec 03 11:45:26 crc kubenswrapper[5052]: I1203 11:45:26.372209 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-zhrr7\" (UniqueName: \"kubernetes.io/projected/8dcfd9c8-2ef8-44a2-ad54-5382d56c630d-kube-api-access-zhrr7\") pod \"nova-cell0-conductor-db-sync-42jqd\" (UID: \"8dcfd9c8-2ef8-44a2-ad54-5382d56c630d\") " pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-42jqd" Dec 03 11:45:26 crc kubenswrapper[5052]: I1203 11:45:26.396977 5052 generic.go:334] "Generic (PLEG): container finished" podID="1303e563-d0c3-4f9c-84b7-3e4fdeba3c6f" containerID="8eb9f63d2e238cddda381da8dd95d8d4b6eb3913bb5cdce6127df175953100c7" exitCode=0 Dec 03 11:45:26 crc kubenswrapper[5052]: I1203 11:45:26.397054 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"1303e563-d0c3-4f9c-84b7-3e4fdeba3c6f","Type":"ContainerDied","Data":"8eb9f63d2e238cddda381da8dd95d8d4b6eb3913bb5cdce6127df175953100c7"} Dec 03 11:45:26 crc kubenswrapper[5052]: I1203 11:45:26.400200 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"73440cd7-672e-4ace-91bd-b380dbcf681f","Type":"ContainerStarted","Data":"ba929f81e5969286ece6c7a6ab4237c80ee919c92a176f3b3c90b7cc34fbb30a"} Dec 03 11:45:26 crc kubenswrapper[5052]: I1203 11:45:26.402867 5052 generic.go:334] "Generic (PLEG): container finished" podID="d2650df9-f097-4bb1-84e1-72077a661399" containerID="0ceade6cc9a65e0e414c39b52e672b86796c8cf5e58f3884914e32309bc0607b" exitCode=143 Dec 03 11:45:26 crc kubenswrapper[5052]: I1203 11:45:26.402900 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"d2650df9-f097-4bb1-84e1-72077a661399","Type":"ContainerDied","Data":"0ceade6cc9a65e0e414c39b52e672b86796c8cf5e58f3884914e32309bc0607b"} Dec 03 11:45:26 crc kubenswrapper[5052]: I1203 11:45:26.498265 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-42jqd" Dec 03 11:45:26 crc kubenswrapper[5052]: I1203 11:45:26.575131 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 11:45:26 crc kubenswrapper[5052]: I1203 11:45:26.993555 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-db-sync-42jqd"] Dec 03 11:45:27 crc kubenswrapper[5052]: I1203 11:45:27.412402 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-42jqd" event={"ID":"8dcfd9c8-2ef8-44a2-ad54-5382d56c630d","Type":"ContainerStarted","Data":"6b3e2ba2cca6d56649a860106c9bc4e57a58f08225072e910e1ef07582b6ff63"} Dec 03 11:45:27 crc kubenswrapper[5052]: I1203 11:45:27.986449 5052 scope.go:117] "RemoveContainer" containerID="831ae3f9746c506b87d5990844ed13f7bb77d7fc3b6415de96a98290e3869a4c" Dec 03 11:45:28 crc kubenswrapper[5052]: I1203 11:45:28.030105 5052 scope.go:117] "RemoveContainer" containerID="b976664871cebff06450133c9273e24bbc1c92f6e86458b6364c4f2e4f2e4d6f" Dec 03 11:45:28 crc kubenswrapper[5052]: I1203 11:45:28.080097 5052 scope.go:117] "RemoveContainer" containerID="2289f14f2059fc517e80d4db86044454e7b7fc71a8ff28682e6b78283502038f" Dec 03 11:45:28 crc kubenswrapper[5052]: I1203 11:45:28.105379 5052 scope.go:117] "RemoveContainer" containerID="77dd24bffc9318ae0172000a2a92f5dd4f72dcfd986df49b78a78df630d16741" Dec 03 11:45:28 crc kubenswrapper[5052]: I1203 11:45:28.130757 5052 scope.go:117] "RemoveContainer" containerID="52ef4f5dde2a47529b2ebf6b602ec3711d01b687b6e696dd4f128ab5239e064a" Dec 03 11:45:28 crc kubenswrapper[5052]: I1203 11:45:28.199405 5052 scope.go:117] "RemoveContainer" containerID="c6d817a30b2e7094442dd66f61af96f6badd467a3b61dd19427f47351547a72d" Dec 03 11:45:28 crc kubenswrapper[5052]: I1203 11:45:28.244236 5052 scope.go:117] "RemoveContainer" containerID="1820ff11c6142cf8027dde668e11d985d56d48b18de14c51c0337c5af52c8909" Dec 03 11:45:28 crc kubenswrapper[5052]: I1203 11:45:28.294321 5052 scope.go:117] "RemoveContainer" containerID="4dd837c600b78e6aad667f7eb01ad1f8a043677942a003c3fb1219733ddb1f0e" Dec 03 11:45:28 crc kubenswrapper[5052]: I1203 11:45:28.371010 5052 scope.go:117] "RemoveContainer" containerID="68ef4a664deede083c7a3421bfaeed869fe2a0d81c8e476b2b9f92399d181fa1" Dec 03 11:45:28 crc kubenswrapper[5052]: I1203 11:45:28.446791 5052 scope.go:117] "RemoveContainer" containerID="36337ab63eb8262535e7e16ba49f89375f566b548bb4c5bf36fb9e7e0605e1c7" Dec 03 11:45:29 crc kubenswrapper[5052]: I1203 11:45:29.361994 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:45:29 crc kubenswrapper[5052]: I1203 11:45:29.379375 5052 patch_prober.go:28] interesting pod/machine-config-daemon-74bj6 container/machine-config-daemon namespace/openshift-machine-config-operator: Liveness probe status=failure output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" start-of-body= Dec 03 11:45:29 crc kubenswrapper[5052]: I1203 11:45:29.379447 5052 prober.go:107] "Probe failed" probeType="Liveness" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" containerName="machine-config-daemon" probeResult="failure" output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" Dec 03 11:45:29 crc kubenswrapper[5052]: I1203 11:45:29.414487 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"glance\" (UniqueName: \"kubernetes.io/local-volume/local-storage15-crc\") pod \"1303e563-d0c3-4f9c-84b7-3e4fdeba3c6f\" (UID: \"1303e563-d0c3-4f9c-84b7-3e4fdeba3c6f\") " Dec 03 11:45:29 crc kubenswrapper[5052]: I1203 11:45:29.415523 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/1303e563-d0c3-4f9c-84b7-3e4fdeba3c6f-logs\") pod \"1303e563-d0c3-4f9c-84b7-3e4fdeba3c6f\" (UID: \"1303e563-d0c3-4f9c-84b7-3e4fdeba3c6f\") " Dec 03 11:45:29 crc kubenswrapper[5052]: I1203 11:45:29.415655 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/1303e563-d0c3-4f9c-84b7-3e4fdeba3c6f-config-data\") pod \"1303e563-d0c3-4f9c-84b7-3e4fdeba3c6f\" (UID: \"1303e563-d0c3-4f9c-84b7-3e4fdeba3c6f\") " Dec 03 11:45:29 crc kubenswrapper[5052]: I1203 11:45:29.415726 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/1303e563-d0c3-4f9c-84b7-3e4fdeba3c6f-scripts\") pod \"1303e563-d0c3-4f9c-84b7-3e4fdeba3c6f\" (UID: \"1303e563-d0c3-4f9c-84b7-3e4fdeba3c6f\") " Dec 03 11:45:29 crc kubenswrapper[5052]: I1203 11:45:29.415815 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pbh2k\" (UniqueName: \"kubernetes.io/projected/1303e563-d0c3-4f9c-84b7-3e4fdeba3c6f-kube-api-access-pbh2k\") pod \"1303e563-d0c3-4f9c-84b7-3e4fdeba3c6f\" (UID: \"1303e563-d0c3-4f9c-84b7-3e4fdeba3c6f\") " Dec 03 11:45:29 crc kubenswrapper[5052]: I1203 11:45:29.415968 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1303e563-d0c3-4f9c-84b7-3e4fdeba3c6f-logs" (OuterVolumeSpecName: "logs") pod "1303e563-d0c3-4f9c-84b7-3e4fdeba3c6f" (UID: "1303e563-d0c3-4f9c-84b7-3e4fdeba3c6f"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:45:29 crc kubenswrapper[5052]: I1203 11:45:29.416171 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/1303e563-d0c3-4f9c-84b7-3e4fdeba3c6f-httpd-run\") pod \"1303e563-d0c3-4f9c-84b7-3e4fdeba3c6f\" (UID: \"1303e563-d0c3-4f9c-84b7-3e4fdeba3c6f\") " Dec 03 11:45:29 crc kubenswrapper[5052]: I1203 11:45:29.416267 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1303e563-d0c3-4f9c-84b7-3e4fdeba3c6f-combined-ca-bundle\") pod \"1303e563-d0c3-4f9c-84b7-3e4fdeba3c6f\" (UID: \"1303e563-d0c3-4f9c-84b7-3e4fdeba3c6f\") " Dec 03 11:45:29 crc kubenswrapper[5052]: I1203 11:45:29.416334 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/1303e563-d0c3-4f9c-84b7-3e4fdeba3c6f-public-tls-certs\") pod \"1303e563-d0c3-4f9c-84b7-3e4fdeba3c6f\" (UID: \"1303e563-d0c3-4f9c-84b7-3e4fdeba3c6f\") " Dec 03 11:45:29 crc kubenswrapper[5052]: I1203 11:45:29.416860 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/1303e563-d0c3-4f9c-84b7-3e4fdeba3c6f-logs\") on node \"crc\" DevicePath \"\"" Dec 03 11:45:29 crc kubenswrapper[5052]: I1203 11:45:29.418457 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1303e563-d0c3-4f9c-84b7-3e4fdeba3c6f-httpd-run" (OuterVolumeSpecName: "httpd-run") pod "1303e563-d0c3-4f9c-84b7-3e4fdeba3c6f" (UID: "1303e563-d0c3-4f9c-84b7-3e4fdeba3c6f"). InnerVolumeSpecName "httpd-run". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:45:29 crc kubenswrapper[5052]: I1203 11:45:29.422668 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage15-crc" (OuterVolumeSpecName: "glance") pod "1303e563-d0c3-4f9c-84b7-3e4fdeba3c6f" (UID: "1303e563-d0c3-4f9c-84b7-3e4fdeba3c6f"). InnerVolumeSpecName "local-storage15-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 11:45:29 crc kubenswrapper[5052]: I1203 11:45:29.423147 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1303e563-d0c3-4f9c-84b7-3e4fdeba3c6f-kube-api-access-pbh2k" (OuterVolumeSpecName: "kube-api-access-pbh2k") pod "1303e563-d0c3-4f9c-84b7-3e4fdeba3c6f" (UID: "1303e563-d0c3-4f9c-84b7-3e4fdeba3c6f"). InnerVolumeSpecName "kube-api-access-pbh2k". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:45:29 crc kubenswrapper[5052]: I1203 11:45:29.431271 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1303e563-d0c3-4f9c-84b7-3e4fdeba3c6f-scripts" (OuterVolumeSpecName: "scripts") pod "1303e563-d0c3-4f9c-84b7-3e4fdeba3c6f" (UID: "1303e563-d0c3-4f9c-84b7-3e4fdeba3c6f"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:45:29 crc kubenswrapper[5052]: I1203 11:45:29.451600 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1303e563-d0c3-4f9c-84b7-3e4fdeba3c6f-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "1303e563-d0c3-4f9c-84b7-3e4fdeba3c6f" (UID: "1303e563-d0c3-4f9c-84b7-3e4fdeba3c6f"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:45:29 crc kubenswrapper[5052]: I1203 11:45:29.485412 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1303e563-d0c3-4f9c-84b7-3e4fdeba3c6f-config-data" (OuterVolumeSpecName: "config-data") pod "1303e563-d0c3-4f9c-84b7-3e4fdeba3c6f" (UID: "1303e563-d0c3-4f9c-84b7-3e4fdeba3c6f"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:45:29 crc kubenswrapper[5052]: I1203 11:45:29.510879 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"1303e563-d0c3-4f9c-84b7-3e4fdeba3c6f","Type":"ContainerDied","Data":"fc4933f5ef18703d3d02ad0d36b4c306bbc914872afd5d2a1944f06f2180aa37"} Dec 03 11:45:29 crc kubenswrapper[5052]: I1203 11:45:29.510957 5052 scope.go:117] "RemoveContainer" containerID="8eb9f63d2e238cddda381da8dd95d8d4b6eb3913bb5cdce6127df175953100c7" Dec 03 11:45:29 crc kubenswrapper[5052]: I1203 11:45:29.511148 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:45:29 crc kubenswrapper[5052]: I1203 11:45:29.518541 5052 reconciler_common.go:293] "Volume detached for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/1303e563-d0c3-4f9c-84b7-3e4fdeba3c6f-httpd-run\") on node \"crc\" DevicePath \"\"" Dec 03 11:45:29 crc kubenswrapper[5052]: I1203 11:45:29.518563 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1303e563-d0c3-4f9c-84b7-3e4fdeba3c6f-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:45:29 crc kubenswrapper[5052]: I1203 11:45:29.518585 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage15-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage15-crc\") on node \"crc\" " Dec 03 11:45:29 crc kubenswrapper[5052]: I1203 11:45:29.518596 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/1303e563-d0c3-4f9c-84b7-3e4fdeba3c6f-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:45:29 crc kubenswrapper[5052]: I1203 11:45:29.518605 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/1303e563-d0c3-4f9c-84b7-3e4fdeba3c6f-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:45:29 crc kubenswrapper[5052]: I1203 11:45:29.518614 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-pbh2k\" (UniqueName: \"kubernetes.io/projected/1303e563-d0c3-4f9c-84b7-3e4fdeba3c6f-kube-api-access-pbh2k\") on node \"crc\" DevicePath \"\"" Dec 03 11:45:29 crc kubenswrapper[5052]: I1203 11:45:29.520965 5052 generic.go:334] "Generic (PLEG): container finished" podID="d2650df9-f097-4bb1-84e1-72077a661399" containerID="725bc621142a24553c3e356f496feb688f1c45840f857a725ee2ca892589de8b" exitCode=0 Dec 03 11:45:29 crc kubenswrapper[5052]: I1203 11:45:29.521021 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"d2650df9-f097-4bb1-84e1-72077a661399","Type":"ContainerDied","Data":"725bc621142a24553c3e356f496feb688f1c45840f857a725ee2ca892589de8b"} Dec 03 11:45:29 crc kubenswrapper[5052]: I1203 11:45:29.521350 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1303e563-d0c3-4f9c-84b7-3e4fdeba3c6f-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "1303e563-d0c3-4f9c-84b7-3e4fdeba3c6f" (UID: "1303e563-d0c3-4f9c-84b7-3e4fdeba3c6f"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:45:29 crc kubenswrapper[5052]: I1203 11:45:29.523660 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-42jqd" event={"ID":"8dcfd9c8-2ef8-44a2-ad54-5382d56c630d","Type":"ContainerStarted","Data":"aaed7fd471350da406347ddf3e6da7df5351689cca457e45e0443f8f32533c5e"} Dec 03 11:45:29 crc kubenswrapper[5052]: I1203 11:45:29.549629 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-42jqd" podStartSLOduration=3.549604242 podStartE2EDuration="3.549604242s" podCreationTimestamp="2025-12-03 11:45:26 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:45:29.538296766 +0000 UTC m=+1520.126538530" watchObservedRunningTime="2025-12-03 11:45:29.549604242 +0000 UTC m=+1520.137846006" Dec 03 11:45:29 crc kubenswrapper[5052]: I1203 11:45:29.553954 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage15-crc" (UniqueName: "kubernetes.io/local-volume/local-storage15-crc") on node "crc" Dec 03 11:45:29 crc kubenswrapper[5052]: I1203 11:45:29.620434 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/1303e563-d0c3-4f9c-84b7-3e4fdeba3c6f-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 11:45:29 crc kubenswrapper[5052]: I1203 11:45:29.620478 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage15-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage15-crc\") on node \"crc\" DevicePath \"\"" Dec 03 11:45:29 crc kubenswrapper[5052]: I1203 11:45:29.835733 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 11:45:29 crc kubenswrapper[5052]: I1203 11:45:29.843543 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 11:45:29 crc kubenswrapper[5052]: I1203 11:45:29.861707 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 11:45:29 crc kubenswrapper[5052]: E1203 11:45:29.862163 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="1303e563-d0c3-4f9c-84b7-3e4fdeba3c6f" containerName="glance-log" Dec 03 11:45:29 crc kubenswrapper[5052]: I1203 11:45:29.862185 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="1303e563-d0c3-4f9c-84b7-3e4fdeba3c6f" containerName="glance-log" Dec 03 11:45:29 crc kubenswrapper[5052]: E1203 11:45:29.862217 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="1303e563-d0c3-4f9c-84b7-3e4fdeba3c6f" containerName="glance-httpd" Dec 03 11:45:29 crc kubenswrapper[5052]: I1203 11:45:29.862226 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="1303e563-d0c3-4f9c-84b7-3e4fdeba3c6f" containerName="glance-httpd" Dec 03 11:45:29 crc kubenswrapper[5052]: I1203 11:45:29.862461 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="1303e563-d0c3-4f9c-84b7-3e4fdeba3c6f" containerName="glance-httpd" Dec 03 11:45:29 crc kubenswrapper[5052]: I1203 11:45:29.862484 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="1303e563-d0c3-4f9c-84b7-3e4fdeba3c6f" containerName="glance-log" Dec 03 11:45:29 crc kubenswrapper[5052]: I1203 11:45:29.863649 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:45:29 crc kubenswrapper[5052]: I1203 11:45:29.867816 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-glance-default-public-svc" Dec 03 11:45:29 crc kubenswrapper[5052]: I1203 11:45:29.869878 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-default-external-config-data" Dec 03 11:45:29 crc kubenswrapper[5052]: I1203 11:45:29.881438 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 11:45:29 crc kubenswrapper[5052]: I1203 11:45:29.925452 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/acea154c-f0bb-4066-bd4d-2dda815f301c-scripts\") pod \"glance-default-external-api-0\" (UID: \"acea154c-f0bb-4066-bd4d-2dda815f301c\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:45:29 crc kubenswrapper[5052]: I1203 11:45:29.925528 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/acea154c-f0bb-4066-bd4d-2dda815f301c-config-data\") pod \"glance-default-external-api-0\" (UID: \"acea154c-f0bb-4066-bd4d-2dda815f301c\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:45:29 crc kubenswrapper[5052]: I1203 11:45:29.925557 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/acea154c-f0bb-4066-bd4d-2dda815f301c-public-tls-certs\") pod \"glance-default-external-api-0\" (UID: \"acea154c-f0bb-4066-bd4d-2dda815f301c\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:45:29 crc kubenswrapper[5052]: I1203 11:45:29.925589 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/acea154c-f0bb-4066-bd4d-2dda815f301c-httpd-run\") pod \"glance-default-external-api-0\" (UID: \"acea154c-f0bb-4066-bd4d-2dda815f301c\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:45:29 crc kubenswrapper[5052]: I1203 11:45:29.925620 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage15-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage15-crc\") pod \"glance-default-external-api-0\" (UID: \"acea154c-f0bb-4066-bd4d-2dda815f301c\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:45:29 crc kubenswrapper[5052]: I1203 11:45:29.925647 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t7ghr\" (UniqueName: \"kubernetes.io/projected/acea154c-f0bb-4066-bd4d-2dda815f301c-kube-api-access-t7ghr\") pod \"glance-default-external-api-0\" (UID: \"acea154c-f0bb-4066-bd4d-2dda815f301c\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:45:29 crc kubenswrapper[5052]: I1203 11:45:29.925708 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/acea154c-f0bb-4066-bd4d-2dda815f301c-combined-ca-bundle\") pod \"glance-default-external-api-0\" (UID: \"acea154c-f0bb-4066-bd4d-2dda815f301c\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:45:29 crc kubenswrapper[5052]: I1203 11:45:29.925753 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/acea154c-f0bb-4066-bd4d-2dda815f301c-logs\") pod \"glance-default-external-api-0\" (UID: \"acea154c-f0bb-4066-bd4d-2dda815f301c\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:45:29 crc kubenswrapper[5052]: I1203 11:45:29.930400 5052 scope.go:117] "RemoveContainer" containerID="877f33b3229c729de1c28b12bb78472817b121f3f37fd1b975fd4575ebc88b59" Dec 03 11:45:30 crc kubenswrapper[5052]: I1203 11:45:30.041461 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/acea154c-f0bb-4066-bd4d-2dda815f301c-combined-ca-bundle\") pod \"glance-default-external-api-0\" (UID: \"acea154c-f0bb-4066-bd4d-2dda815f301c\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:45:30 crc kubenswrapper[5052]: I1203 11:45:30.041833 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/acea154c-f0bb-4066-bd4d-2dda815f301c-logs\") pod \"glance-default-external-api-0\" (UID: \"acea154c-f0bb-4066-bd4d-2dda815f301c\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:45:30 crc kubenswrapper[5052]: I1203 11:45:30.041878 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/acea154c-f0bb-4066-bd4d-2dda815f301c-scripts\") pod \"glance-default-external-api-0\" (UID: \"acea154c-f0bb-4066-bd4d-2dda815f301c\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:45:30 crc kubenswrapper[5052]: I1203 11:45:30.041897 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/acea154c-f0bb-4066-bd4d-2dda815f301c-config-data\") pod \"glance-default-external-api-0\" (UID: \"acea154c-f0bb-4066-bd4d-2dda815f301c\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:45:30 crc kubenswrapper[5052]: I1203 11:45:30.041915 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/acea154c-f0bb-4066-bd4d-2dda815f301c-public-tls-certs\") pod \"glance-default-external-api-0\" (UID: \"acea154c-f0bb-4066-bd4d-2dda815f301c\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:45:30 crc kubenswrapper[5052]: I1203 11:45:30.041940 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/acea154c-f0bb-4066-bd4d-2dda815f301c-httpd-run\") pod \"glance-default-external-api-0\" (UID: \"acea154c-f0bb-4066-bd4d-2dda815f301c\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:45:30 crc kubenswrapper[5052]: I1203 11:45:30.041975 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage15-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage15-crc\") pod \"glance-default-external-api-0\" (UID: \"acea154c-f0bb-4066-bd4d-2dda815f301c\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:45:30 crc kubenswrapper[5052]: I1203 11:45:30.042006 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-t7ghr\" (UniqueName: \"kubernetes.io/projected/acea154c-f0bb-4066-bd4d-2dda815f301c-kube-api-access-t7ghr\") pod \"glance-default-external-api-0\" (UID: \"acea154c-f0bb-4066-bd4d-2dda815f301c\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:45:30 crc kubenswrapper[5052]: I1203 11:45:30.049029 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/acea154c-f0bb-4066-bd4d-2dda815f301c-combined-ca-bundle\") pod \"glance-default-external-api-0\" (UID: \"acea154c-f0bb-4066-bd4d-2dda815f301c\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:45:30 crc kubenswrapper[5052]: I1203 11:45:30.049365 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/acea154c-f0bb-4066-bd4d-2dda815f301c-logs\") pod \"glance-default-external-api-0\" (UID: \"acea154c-f0bb-4066-bd4d-2dda815f301c\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:45:30 crc kubenswrapper[5052]: I1203 11:45:30.060630 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage15-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage15-crc\") pod \"glance-default-external-api-0\" (UID: \"acea154c-f0bb-4066-bd4d-2dda815f301c\") device mount path \"/mnt/openstack/pv15\"" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:45:30 crc kubenswrapper[5052]: I1203 11:45:30.060714 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/acea154c-f0bb-4066-bd4d-2dda815f301c-httpd-run\") pod \"glance-default-external-api-0\" (UID: \"acea154c-f0bb-4066-bd4d-2dda815f301c\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:45:30 crc kubenswrapper[5052]: I1203 11:45:30.071101 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/acea154c-f0bb-4066-bd4d-2dda815f301c-scripts\") pod \"glance-default-external-api-0\" (UID: \"acea154c-f0bb-4066-bd4d-2dda815f301c\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:45:30 crc kubenswrapper[5052]: I1203 11:45:30.071314 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/acea154c-f0bb-4066-bd4d-2dda815f301c-public-tls-certs\") pod \"glance-default-external-api-0\" (UID: \"acea154c-f0bb-4066-bd4d-2dda815f301c\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:45:30 crc kubenswrapper[5052]: I1203 11:45:30.078303 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/acea154c-f0bb-4066-bd4d-2dda815f301c-config-data\") pod \"glance-default-external-api-0\" (UID: \"acea154c-f0bb-4066-bd4d-2dda815f301c\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:45:30 crc kubenswrapper[5052]: I1203 11:45:30.079187 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-t7ghr\" (UniqueName: \"kubernetes.io/projected/acea154c-f0bb-4066-bd4d-2dda815f301c-kube-api-access-t7ghr\") pod \"glance-default-external-api-0\" (UID: \"acea154c-f0bb-4066-bd4d-2dda815f301c\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:45:30 crc kubenswrapper[5052]: I1203 11:45:30.107781 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage15-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage15-crc\") pod \"glance-default-external-api-0\" (UID: \"acea154c-f0bb-4066-bd4d-2dda815f301c\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:45:30 crc kubenswrapper[5052]: I1203 11:45:30.181254 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:45:30 crc kubenswrapper[5052]: I1203 11:45:30.316111 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:45:30 crc kubenswrapper[5052]: I1203 11:45:30.346345 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d2650df9-f097-4bb1-84e1-72077a661399-combined-ca-bundle\") pod \"d2650df9-f097-4bb1-84e1-72077a661399\" (UID: \"d2650df9-f097-4bb1-84e1-72077a661399\") " Dec 03 11:45:30 crc kubenswrapper[5052]: I1203 11:45:30.346412 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"glance\" (UniqueName: \"kubernetes.io/local-volume/local-storage04-crc\") pod \"d2650df9-f097-4bb1-84e1-72077a661399\" (UID: \"d2650df9-f097-4bb1-84e1-72077a661399\") " Dec 03 11:45:30 crc kubenswrapper[5052]: I1203 11:45:30.346534 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hgqft\" (UniqueName: \"kubernetes.io/projected/d2650df9-f097-4bb1-84e1-72077a661399-kube-api-access-hgqft\") pod \"d2650df9-f097-4bb1-84e1-72077a661399\" (UID: \"d2650df9-f097-4bb1-84e1-72077a661399\") " Dec 03 11:45:30 crc kubenswrapper[5052]: I1203 11:45:30.346591 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/d2650df9-f097-4bb1-84e1-72077a661399-httpd-run\") pod \"d2650df9-f097-4bb1-84e1-72077a661399\" (UID: \"d2650df9-f097-4bb1-84e1-72077a661399\") " Dec 03 11:45:30 crc kubenswrapper[5052]: I1203 11:45:30.346631 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/d2650df9-f097-4bb1-84e1-72077a661399-internal-tls-certs\") pod \"d2650df9-f097-4bb1-84e1-72077a661399\" (UID: \"d2650df9-f097-4bb1-84e1-72077a661399\") " Dec 03 11:45:30 crc kubenswrapper[5052]: I1203 11:45:30.346663 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/d2650df9-f097-4bb1-84e1-72077a661399-config-data\") pod \"d2650df9-f097-4bb1-84e1-72077a661399\" (UID: \"d2650df9-f097-4bb1-84e1-72077a661399\") " Dec 03 11:45:30 crc kubenswrapper[5052]: I1203 11:45:30.346792 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/d2650df9-f097-4bb1-84e1-72077a661399-scripts\") pod \"d2650df9-f097-4bb1-84e1-72077a661399\" (UID: \"d2650df9-f097-4bb1-84e1-72077a661399\") " Dec 03 11:45:30 crc kubenswrapper[5052]: I1203 11:45:30.346843 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/d2650df9-f097-4bb1-84e1-72077a661399-logs\") pod \"d2650df9-f097-4bb1-84e1-72077a661399\" (UID: \"d2650df9-f097-4bb1-84e1-72077a661399\") " Dec 03 11:45:30 crc kubenswrapper[5052]: I1203 11:45:30.348663 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d2650df9-f097-4bb1-84e1-72077a661399-logs" (OuterVolumeSpecName: "logs") pod "d2650df9-f097-4bb1-84e1-72077a661399" (UID: "d2650df9-f097-4bb1-84e1-72077a661399"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:45:30 crc kubenswrapper[5052]: I1203 11:45:30.348822 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d2650df9-f097-4bb1-84e1-72077a661399-httpd-run" (OuterVolumeSpecName: "httpd-run") pod "d2650df9-f097-4bb1-84e1-72077a661399" (UID: "d2650df9-f097-4bb1-84e1-72077a661399"). InnerVolumeSpecName "httpd-run". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:45:30 crc kubenswrapper[5052]: I1203 11:45:30.359658 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d2650df9-f097-4bb1-84e1-72077a661399-kube-api-access-hgqft" (OuterVolumeSpecName: "kube-api-access-hgqft") pod "d2650df9-f097-4bb1-84e1-72077a661399" (UID: "d2650df9-f097-4bb1-84e1-72077a661399"). InnerVolumeSpecName "kube-api-access-hgqft". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:45:30 crc kubenswrapper[5052]: I1203 11:45:30.363654 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d2650df9-f097-4bb1-84e1-72077a661399-scripts" (OuterVolumeSpecName: "scripts") pod "d2650df9-f097-4bb1-84e1-72077a661399" (UID: "d2650df9-f097-4bb1-84e1-72077a661399"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:45:30 crc kubenswrapper[5052]: I1203 11:45:30.369646 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage04-crc" (OuterVolumeSpecName: "glance") pod "d2650df9-f097-4bb1-84e1-72077a661399" (UID: "d2650df9-f097-4bb1-84e1-72077a661399"). InnerVolumeSpecName "local-storage04-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 11:45:30 crc kubenswrapper[5052]: I1203 11:45:30.425613 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d2650df9-f097-4bb1-84e1-72077a661399-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "d2650df9-f097-4bb1-84e1-72077a661399" (UID: "d2650df9-f097-4bb1-84e1-72077a661399"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:45:30 crc kubenswrapper[5052]: I1203 11:45:30.426413 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d2650df9-f097-4bb1-84e1-72077a661399-config-data" (OuterVolumeSpecName: "config-data") pod "d2650df9-f097-4bb1-84e1-72077a661399" (UID: "d2650df9-f097-4bb1-84e1-72077a661399"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:45:30 crc kubenswrapper[5052]: I1203 11:45:30.447995 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/d2650df9-f097-4bb1-84e1-72077a661399-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:45:30 crc kubenswrapper[5052]: I1203 11:45:30.448018 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/d2650df9-f097-4bb1-84e1-72077a661399-logs\") on node \"crc\" DevicePath \"\"" Dec 03 11:45:30 crc kubenswrapper[5052]: I1203 11:45:30.448029 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d2650df9-f097-4bb1-84e1-72077a661399-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:45:30 crc kubenswrapper[5052]: I1203 11:45:30.448051 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage04-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage04-crc\") on node \"crc\" " Dec 03 11:45:30 crc kubenswrapper[5052]: I1203 11:45:30.448061 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-hgqft\" (UniqueName: \"kubernetes.io/projected/d2650df9-f097-4bb1-84e1-72077a661399-kube-api-access-hgqft\") on node \"crc\" DevicePath \"\"" Dec 03 11:45:30 crc kubenswrapper[5052]: I1203 11:45:30.448069 5052 reconciler_common.go:293] "Volume detached for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/d2650df9-f097-4bb1-84e1-72077a661399-httpd-run\") on node \"crc\" DevicePath \"\"" Dec 03 11:45:30 crc kubenswrapper[5052]: I1203 11:45:30.448077 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/d2650df9-f097-4bb1-84e1-72077a661399-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:45:30 crc kubenswrapper[5052]: I1203 11:45:30.451839 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d2650df9-f097-4bb1-84e1-72077a661399-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "d2650df9-f097-4bb1-84e1-72077a661399" (UID: "d2650df9-f097-4bb1-84e1-72077a661399"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:45:30 crc kubenswrapper[5052]: I1203 11:45:30.468706 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage04-crc" (UniqueName: "kubernetes.io/local-volume/local-storage04-crc") on node "crc" Dec 03 11:45:30 crc kubenswrapper[5052]: I1203 11:45:30.550761 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage04-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage04-crc\") on node \"crc\" DevicePath \"\"" Dec 03 11:45:30 crc kubenswrapper[5052]: I1203 11:45:30.550792 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/d2650df9-f097-4bb1-84e1-72077a661399-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 11:45:30 crc kubenswrapper[5052]: I1203 11:45:30.551547 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"73440cd7-672e-4ace-91bd-b380dbcf681f","Type":"ContainerStarted","Data":"a9e5d011b3cd95329fb1749a8a824a2688dc235cdb93bab7b0ae8ce9705d1d89"} Dec 03 11:45:30 crc kubenswrapper[5052]: I1203 11:45:30.551715 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="73440cd7-672e-4ace-91bd-b380dbcf681f" containerName="ceilometer-central-agent" containerID="cri-o://1acb2031482f91324fd2b811f3a7715d4cef822d78733b2935b50623edcb4c12" gracePeriod=30 Dec 03 11:45:30 crc kubenswrapper[5052]: I1203 11:45:30.551760 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="73440cd7-672e-4ace-91bd-b380dbcf681f" containerName="sg-core" containerID="cri-o://ba929f81e5969286ece6c7a6ab4237c80ee919c92a176f3b3c90b7cc34fbb30a" gracePeriod=30 Dec 03 11:45:30 crc kubenswrapper[5052]: I1203 11:45:30.551770 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="73440cd7-672e-4ace-91bd-b380dbcf681f" containerName="proxy-httpd" containerID="cri-o://a9e5d011b3cd95329fb1749a8a824a2688dc235cdb93bab7b0ae8ce9705d1d89" gracePeriod=30 Dec 03 11:45:30 crc kubenswrapper[5052]: I1203 11:45:30.551820 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="73440cd7-672e-4ace-91bd-b380dbcf681f" containerName="ceilometer-notification-agent" containerID="cri-o://350bc5605a247a6dbdb1d22ed1c84ac14dd38e149676586bc392fb27c359ae86" gracePeriod=30 Dec 03 11:45:30 crc kubenswrapper[5052]: I1203 11:45:30.552002 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:45:30 crc kubenswrapper[5052]: I1203 11:45:30.556613 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"d2650df9-f097-4bb1-84e1-72077a661399","Type":"ContainerDied","Data":"4db6086957b0da783be909253db938082757fc1b65fe07e81bce17db573e75c2"} Dec 03 11:45:30 crc kubenswrapper[5052]: I1203 11:45:30.556777 5052 scope.go:117] "RemoveContainer" containerID="725bc621142a24553c3e356f496feb688f1c45840f857a725ee2ca892589de8b" Dec 03 11:45:30 crc kubenswrapper[5052]: I1203 11:45:30.557088 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:45:30 crc kubenswrapper[5052]: I1203 11:45:30.584956 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/ceilometer-0" podStartSLOduration=1.8007523760000002 podStartE2EDuration="15.584937447s" podCreationTimestamp="2025-12-03 11:45:15 +0000 UTC" firstStartedPulling="2025-12-03 11:45:16.146393977 +0000 UTC m=+1506.734635741" lastFinishedPulling="2025-12-03 11:45:29.930579048 +0000 UTC m=+1520.518820812" observedRunningTime="2025-12-03 11:45:30.57534316 +0000 UTC m=+1521.163584934" watchObservedRunningTime="2025-12-03 11:45:30.584937447 +0000 UTC m=+1521.173179211" Dec 03 11:45:30 crc kubenswrapper[5052]: I1203 11:45:30.600912 5052 scope.go:117] "RemoveContainer" containerID="0ceade6cc9a65e0e414c39b52e672b86796c8cf5e58f3884914e32309bc0607b" Dec 03 11:45:30 crc kubenswrapper[5052]: I1203 11:45:30.615998 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 11:45:30 crc kubenswrapper[5052]: I1203 11:45:30.652701 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 11:45:30 crc kubenswrapper[5052]: I1203 11:45:30.673805 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 11:45:30 crc kubenswrapper[5052]: E1203 11:45:30.674701 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="d2650df9-f097-4bb1-84e1-72077a661399" containerName="glance-log" Dec 03 11:45:30 crc kubenswrapper[5052]: I1203 11:45:30.674724 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="d2650df9-f097-4bb1-84e1-72077a661399" containerName="glance-log" Dec 03 11:45:30 crc kubenswrapper[5052]: E1203 11:45:30.674759 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="d2650df9-f097-4bb1-84e1-72077a661399" containerName="glance-httpd" Dec 03 11:45:30 crc kubenswrapper[5052]: I1203 11:45:30.674769 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="d2650df9-f097-4bb1-84e1-72077a661399" containerName="glance-httpd" Dec 03 11:45:30 crc kubenswrapper[5052]: I1203 11:45:30.675173 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="d2650df9-f097-4bb1-84e1-72077a661399" containerName="glance-log" Dec 03 11:45:30 crc kubenswrapper[5052]: I1203 11:45:30.675232 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="d2650df9-f097-4bb1-84e1-72077a661399" containerName="glance-httpd" Dec 03 11:45:30 crc kubenswrapper[5052]: I1203 11:45:30.677799 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:45:30 crc kubenswrapper[5052]: I1203 11:45:30.679787 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-glance-default-internal-svc" Dec 03 11:45:30 crc kubenswrapper[5052]: I1203 11:45:30.680061 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-default-internal-config-data" Dec 03 11:45:30 crc kubenswrapper[5052]: I1203 11:45:30.689131 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 11:45:30 crc kubenswrapper[5052]: I1203 11:45:30.707492 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 11:45:30 crc kubenswrapper[5052]: W1203 11:45:30.711663 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podacea154c_f0bb_4066_bd4d_2dda815f301c.slice/crio-a3d5fdb3d20e2604d292f20ec735e9c7ddb3d4e1018dcbc0aeb11a3eca4293ac WatchSource:0}: Error finding container a3d5fdb3d20e2604d292f20ec735e9c7ddb3d4e1018dcbc0aeb11a3eca4293ac: Status 404 returned error can't find the container with id a3d5fdb3d20e2604d292f20ec735e9c7ddb3d4e1018dcbc0aeb11a3eca4293ac Dec 03 11:45:30 crc kubenswrapper[5052]: I1203 11:45:30.860921 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/0914ebfe-13c6-4309-bd10-a67850124e3f-internal-tls-certs\") pod \"glance-default-internal-api-0\" (UID: \"0914ebfe-13c6-4309-bd10-a67850124e3f\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:45:30 crc kubenswrapper[5052]: I1203 11:45:30.860967 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/0914ebfe-13c6-4309-bd10-a67850124e3f-scripts\") pod \"glance-default-internal-api-0\" (UID: \"0914ebfe-13c6-4309-bd10-a67850124e3f\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:45:30 crc kubenswrapper[5052]: I1203 11:45:30.861034 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/0914ebfe-13c6-4309-bd10-a67850124e3f-config-data\") pod \"glance-default-internal-api-0\" (UID: \"0914ebfe-13c6-4309-bd10-a67850124e3f\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:45:30 crc kubenswrapper[5052]: I1203 11:45:30.861174 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0914ebfe-13c6-4309-bd10-a67850124e3f-combined-ca-bundle\") pod \"glance-default-internal-api-0\" (UID: \"0914ebfe-13c6-4309-bd10-a67850124e3f\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:45:30 crc kubenswrapper[5052]: I1203 11:45:30.861257 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p8kzb\" (UniqueName: \"kubernetes.io/projected/0914ebfe-13c6-4309-bd10-a67850124e3f-kube-api-access-p8kzb\") pod \"glance-default-internal-api-0\" (UID: \"0914ebfe-13c6-4309-bd10-a67850124e3f\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:45:30 crc kubenswrapper[5052]: I1203 11:45:30.861308 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage04-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage04-crc\") pod \"glance-default-internal-api-0\" (UID: \"0914ebfe-13c6-4309-bd10-a67850124e3f\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:45:30 crc kubenswrapper[5052]: I1203 11:45:30.861374 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/0914ebfe-13c6-4309-bd10-a67850124e3f-httpd-run\") pod \"glance-default-internal-api-0\" (UID: \"0914ebfe-13c6-4309-bd10-a67850124e3f\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:45:30 crc kubenswrapper[5052]: I1203 11:45:30.861458 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/0914ebfe-13c6-4309-bd10-a67850124e3f-logs\") pod \"glance-default-internal-api-0\" (UID: \"0914ebfe-13c6-4309-bd10-a67850124e3f\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:45:30 crc kubenswrapper[5052]: I1203 11:45:30.962850 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/0914ebfe-13c6-4309-bd10-a67850124e3f-logs\") pod \"glance-default-internal-api-0\" (UID: \"0914ebfe-13c6-4309-bd10-a67850124e3f\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:45:30 crc kubenswrapper[5052]: I1203 11:45:30.962929 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/0914ebfe-13c6-4309-bd10-a67850124e3f-internal-tls-certs\") pod \"glance-default-internal-api-0\" (UID: \"0914ebfe-13c6-4309-bd10-a67850124e3f\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:45:30 crc kubenswrapper[5052]: I1203 11:45:30.962957 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/0914ebfe-13c6-4309-bd10-a67850124e3f-scripts\") pod \"glance-default-internal-api-0\" (UID: \"0914ebfe-13c6-4309-bd10-a67850124e3f\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:45:30 crc kubenswrapper[5052]: I1203 11:45:30.963013 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/0914ebfe-13c6-4309-bd10-a67850124e3f-config-data\") pod \"glance-default-internal-api-0\" (UID: \"0914ebfe-13c6-4309-bd10-a67850124e3f\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:45:30 crc kubenswrapper[5052]: I1203 11:45:30.963042 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0914ebfe-13c6-4309-bd10-a67850124e3f-combined-ca-bundle\") pod \"glance-default-internal-api-0\" (UID: \"0914ebfe-13c6-4309-bd10-a67850124e3f\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:45:30 crc kubenswrapper[5052]: I1203 11:45:30.963068 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-p8kzb\" (UniqueName: \"kubernetes.io/projected/0914ebfe-13c6-4309-bd10-a67850124e3f-kube-api-access-p8kzb\") pod \"glance-default-internal-api-0\" (UID: \"0914ebfe-13c6-4309-bd10-a67850124e3f\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:45:30 crc kubenswrapper[5052]: I1203 11:45:30.963091 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage04-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage04-crc\") pod \"glance-default-internal-api-0\" (UID: \"0914ebfe-13c6-4309-bd10-a67850124e3f\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:45:30 crc kubenswrapper[5052]: I1203 11:45:30.963119 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/0914ebfe-13c6-4309-bd10-a67850124e3f-httpd-run\") pod \"glance-default-internal-api-0\" (UID: \"0914ebfe-13c6-4309-bd10-a67850124e3f\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:45:30 crc kubenswrapper[5052]: I1203 11:45:30.963595 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/0914ebfe-13c6-4309-bd10-a67850124e3f-httpd-run\") pod \"glance-default-internal-api-0\" (UID: \"0914ebfe-13c6-4309-bd10-a67850124e3f\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:45:30 crc kubenswrapper[5052]: I1203 11:45:30.963850 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/0914ebfe-13c6-4309-bd10-a67850124e3f-logs\") pod \"glance-default-internal-api-0\" (UID: \"0914ebfe-13c6-4309-bd10-a67850124e3f\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:45:30 crc kubenswrapper[5052]: I1203 11:45:30.970152 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/0914ebfe-13c6-4309-bd10-a67850124e3f-internal-tls-certs\") pod \"glance-default-internal-api-0\" (UID: \"0914ebfe-13c6-4309-bd10-a67850124e3f\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:45:30 crc kubenswrapper[5052]: I1203 11:45:30.974170 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/0914ebfe-13c6-4309-bd10-a67850124e3f-scripts\") pod \"glance-default-internal-api-0\" (UID: \"0914ebfe-13c6-4309-bd10-a67850124e3f\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:45:30 crc kubenswrapper[5052]: I1203 11:45:30.979018 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/0914ebfe-13c6-4309-bd10-a67850124e3f-config-data\") pod \"glance-default-internal-api-0\" (UID: \"0914ebfe-13c6-4309-bd10-a67850124e3f\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:45:30 crc kubenswrapper[5052]: I1203 11:45:30.979959 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage04-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage04-crc\") pod \"glance-default-internal-api-0\" (UID: \"0914ebfe-13c6-4309-bd10-a67850124e3f\") device mount path \"/mnt/openstack/pv04\"" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:45:30 crc kubenswrapper[5052]: I1203 11:45:30.985263 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0914ebfe-13c6-4309-bd10-a67850124e3f-combined-ca-bundle\") pod \"glance-default-internal-api-0\" (UID: \"0914ebfe-13c6-4309-bd10-a67850124e3f\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:45:31 crc kubenswrapper[5052]: I1203 11:45:31.006861 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-p8kzb\" (UniqueName: \"kubernetes.io/projected/0914ebfe-13c6-4309-bd10-a67850124e3f-kube-api-access-p8kzb\") pod \"glance-default-internal-api-0\" (UID: \"0914ebfe-13c6-4309-bd10-a67850124e3f\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:45:31 crc kubenswrapper[5052]: I1203 11:45:31.009623 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage04-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage04-crc\") pod \"glance-default-internal-api-0\" (UID: \"0914ebfe-13c6-4309-bd10-a67850124e3f\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:45:31 crc kubenswrapper[5052]: I1203 11:45:31.303882 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:45:31 crc kubenswrapper[5052]: I1203 11:45:31.574671 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"acea154c-f0bb-4066-bd4d-2dda815f301c","Type":"ContainerStarted","Data":"51504add0a6672a10359e1a4fc407e6171b498dd50bf1382d23a8247c04ea2fe"} Dec 03 11:45:31 crc kubenswrapper[5052]: I1203 11:45:31.575008 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"acea154c-f0bb-4066-bd4d-2dda815f301c","Type":"ContainerStarted","Data":"a3d5fdb3d20e2604d292f20ec735e9c7ddb3d4e1018dcbc0aeb11a3eca4293ac"} Dec 03 11:45:31 crc kubenswrapper[5052]: I1203 11:45:31.578008 5052 generic.go:334] "Generic (PLEG): container finished" podID="73440cd7-672e-4ace-91bd-b380dbcf681f" containerID="a9e5d011b3cd95329fb1749a8a824a2688dc235cdb93bab7b0ae8ce9705d1d89" exitCode=0 Dec 03 11:45:31 crc kubenswrapper[5052]: I1203 11:45:31.578041 5052 generic.go:334] "Generic (PLEG): container finished" podID="73440cd7-672e-4ace-91bd-b380dbcf681f" containerID="ba929f81e5969286ece6c7a6ab4237c80ee919c92a176f3b3c90b7cc34fbb30a" exitCode=2 Dec 03 11:45:31 crc kubenswrapper[5052]: I1203 11:45:31.578051 5052 generic.go:334] "Generic (PLEG): container finished" podID="73440cd7-672e-4ace-91bd-b380dbcf681f" containerID="350bc5605a247a6dbdb1d22ed1c84ac14dd38e149676586bc392fb27c359ae86" exitCode=0 Dec 03 11:45:31 crc kubenswrapper[5052]: I1203 11:45:31.578102 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"73440cd7-672e-4ace-91bd-b380dbcf681f","Type":"ContainerDied","Data":"a9e5d011b3cd95329fb1749a8a824a2688dc235cdb93bab7b0ae8ce9705d1d89"} Dec 03 11:45:31 crc kubenswrapper[5052]: I1203 11:45:31.578144 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"73440cd7-672e-4ace-91bd-b380dbcf681f","Type":"ContainerDied","Data":"ba929f81e5969286ece6c7a6ab4237c80ee919c92a176f3b3c90b7cc34fbb30a"} Dec 03 11:45:31 crc kubenswrapper[5052]: I1203 11:45:31.578190 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"73440cd7-672e-4ace-91bd-b380dbcf681f","Type":"ContainerDied","Data":"350bc5605a247a6dbdb1d22ed1c84ac14dd38e149676586bc392fb27c359ae86"} Dec 03 11:45:31 crc kubenswrapper[5052]: I1203 11:45:31.688270 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1303e563-d0c3-4f9c-84b7-3e4fdeba3c6f" path="/var/lib/kubelet/pods/1303e563-d0c3-4f9c-84b7-3e4fdeba3c6f/volumes" Dec 03 11:45:31 crc kubenswrapper[5052]: I1203 11:45:31.689428 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d2650df9-f097-4bb1-84e1-72077a661399" path="/var/lib/kubelet/pods/d2650df9-f097-4bb1-84e1-72077a661399/volumes" Dec 03 11:45:31 crc kubenswrapper[5052]: I1203 11:45:31.743702 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 11:45:32 crc kubenswrapper[5052]: I1203 11:45:32.426009 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:45:32 crc kubenswrapper[5052]: I1203 11:45:32.495558 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/73440cd7-672e-4ace-91bd-b380dbcf681f-scripts\") pod \"73440cd7-672e-4ace-91bd-b380dbcf681f\" (UID: \"73440cd7-672e-4ace-91bd-b380dbcf681f\") " Dec 03 11:45:32 crc kubenswrapper[5052]: I1203 11:45:32.495806 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/73440cd7-672e-4ace-91bd-b380dbcf681f-sg-core-conf-yaml\") pod \"73440cd7-672e-4ace-91bd-b380dbcf681f\" (UID: \"73440cd7-672e-4ace-91bd-b380dbcf681f\") " Dec 03 11:45:32 crc kubenswrapper[5052]: I1203 11:45:32.495887 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/73440cd7-672e-4ace-91bd-b380dbcf681f-combined-ca-bundle\") pod \"73440cd7-672e-4ace-91bd-b380dbcf681f\" (UID: \"73440cd7-672e-4ace-91bd-b380dbcf681f\") " Dec 03 11:45:32 crc kubenswrapper[5052]: I1203 11:45:32.496114 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/73440cd7-672e-4ace-91bd-b380dbcf681f-run-httpd\") pod \"73440cd7-672e-4ace-91bd-b380dbcf681f\" (UID: \"73440cd7-672e-4ace-91bd-b380dbcf681f\") " Dec 03 11:45:32 crc kubenswrapper[5052]: I1203 11:45:32.496263 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rr9ft\" (UniqueName: \"kubernetes.io/projected/73440cd7-672e-4ace-91bd-b380dbcf681f-kube-api-access-rr9ft\") pod \"73440cd7-672e-4ace-91bd-b380dbcf681f\" (UID: \"73440cd7-672e-4ace-91bd-b380dbcf681f\") " Dec 03 11:45:32 crc kubenswrapper[5052]: I1203 11:45:32.496372 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/73440cd7-672e-4ace-91bd-b380dbcf681f-log-httpd\") pod \"73440cd7-672e-4ace-91bd-b380dbcf681f\" (UID: \"73440cd7-672e-4ace-91bd-b380dbcf681f\") " Dec 03 11:45:32 crc kubenswrapper[5052]: I1203 11:45:32.496462 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/73440cd7-672e-4ace-91bd-b380dbcf681f-config-data\") pod \"73440cd7-672e-4ace-91bd-b380dbcf681f\" (UID: \"73440cd7-672e-4ace-91bd-b380dbcf681f\") " Dec 03 11:45:32 crc kubenswrapper[5052]: I1203 11:45:32.497548 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73440cd7-672e-4ace-91bd-b380dbcf681f-run-httpd" (OuterVolumeSpecName: "run-httpd") pod "73440cd7-672e-4ace-91bd-b380dbcf681f" (UID: "73440cd7-672e-4ace-91bd-b380dbcf681f"). InnerVolumeSpecName "run-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:45:32 crc kubenswrapper[5052]: I1203 11:45:32.497907 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73440cd7-672e-4ace-91bd-b380dbcf681f-log-httpd" (OuterVolumeSpecName: "log-httpd") pod "73440cd7-672e-4ace-91bd-b380dbcf681f" (UID: "73440cd7-672e-4ace-91bd-b380dbcf681f"). InnerVolumeSpecName "log-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:45:32 crc kubenswrapper[5052]: I1203 11:45:32.498097 5052 reconciler_common.go:293] "Volume detached for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/73440cd7-672e-4ace-91bd-b380dbcf681f-run-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 11:45:32 crc kubenswrapper[5052]: I1203 11:45:32.501111 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/73440cd7-672e-4ace-91bd-b380dbcf681f-scripts" (OuterVolumeSpecName: "scripts") pod "73440cd7-672e-4ace-91bd-b380dbcf681f" (UID: "73440cd7-672e-4ace-91bd-b380dbcf681f"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:45:32 crc kubenswrapper[5052]: I1203 11:45:32.502738 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/73440cd7-672e-4ace-91bd-b380dbcf681f-kube-api-access-rr9ft" (OuterVolumeSpecName: "kube-api-access-rr9ft") pod "73440cd7-672e-4ace-91bd-b380dbcf681f" (UID: "73440cd7-672e-4ace-91bd-b380dbcf681f"). InnerVolumeSpecName "kube-api-access-rr9ft". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:45:32 crc kubenswrapper[5052]: I1203 11:45:32.536189 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/73440cd7-672e-4ace-91bd-b380dbcf681f-sg-core-conf-yaml" (OuterVolumeSpecName: "sg-core-conf-yaml") pod "73440cd7-672e-4ace-91bd-b380dbcf681f" (UID: "73440cd7-672e-4ace-91bd-b380dbcf681f"). InnerVolumeSpecName "sg-core-conf-yaml". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:45:32 crc kubenswrapper[5052]: I1203 11:45:32.582671 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/73440cd7-672e-4ace-91bd-b380dbcf681f-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "73440cd7-672e-4ace-91bd-b380dbcf681f" (UID: "73440cd7-672e-4ace-91bd-b380dbcf681f"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:45:32 crc kubenswrapper[5052]: I1203 11:45:32.592190 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"acea154c-f0bb-4066-bd4d-2dda815f301c","Type":"ContainerStarted","Data":"e054bf606eb8ede8b913bb4874e3d9c8bf93f40bc4a7246b741ee83f941471a3"} Dec 03 11:45:32 crc kubenswrapper[5052]: I1203 11:45:32.600356 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-rr9ft\" (UniqueName: \"kubernetes.io/projected/73440cd7-672e-4ace-91bd-b380dbcf681f-kube-api-access-rr9ft\") on node \"crc\" DevicePath \"\"" Dec 03 11:45:32 crc kubenswrapper[5052]: I1203 11:45:32.600397 5052 reconciler_common.go:293] "Volume detached for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/73440cd7-672e-4ace-91bd-b380dbcf681f-log-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 11:45:32 crc kubenswrapper[5052]: I1203 11:45:32.600410 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/73440cd7-672e-4ace-91bd-b380dbcf681f-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:45:32 crc kubenswrapper[5052]: I1203 11:45:32.600421 5052 reconciler_common.go:293] "Volume detached for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/73440cd7-672e-4ace-91bd-b380dbcf681f-sg-core-conf-yaml\") on node \"crc\" DevicePath \"\"" Dec 03 11:45:32 crc kubenswrapper[5052]: I1203 11:45:32.600432 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/73440cd7-672e-4ace-91bd-b380dbcf681f-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:45:32 crc kubenswrapper[5052]: I1203 11:45:32.609183 5052 generic.go:334] "Generic (PLEG): container finished" podID="73440cd7-672e-4ace-91bd-b380dbcf681f" containerID="1acb2031482f91324fd2b811f3a7715d4cef822d78733b2935b50623edcb4c12" exitCode=0 Dec 03 11:45:32 crc kubenswrapper[5052]: I1203 11:45:32.609275 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"73440cd7-672e-4ace-91bd-b380dbcf681f","Type":"ContainerDied","Data":"1acb2031482f91324fd2b811f3a7715d4cef822d78733b2935b50623edcb4c12"} Dec 03 11:45:32 crc kubenswrapper[5052]: I1203 11:45:32.609315 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"73440cd7-672e-4ace-91bd-b380dbcf681f","Type":"ContainerDied","Data":"aa0f551fe18f4ec003503654accfde30e69465c1239c1cd129ed63b0d7cff116"} Dec 03 11:45:32 crc kubenswrapper[5052]: I1203 11:45:32.609334 5052 scope.go:117] "RemoveContainer" containerID="a9e5d011b3cd95329fb1749a8a824a2688dc235cdb93bab7b0ae8ce9705d1d89" Dec 03 11:45:32 crc kubenswrapper[5052]: I1203 11:45:32.609455 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:45:32 crc kubenswrapper[5052]: I1203 11:45:32.620305 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"0914ebfe-13c6-4309-bd10-a67850124e3f","Type":"ContainerStarted","Data":"295a2704643ed1264e3a64ad18e13a582bf7cc635129de442187ed9d461c917a"} Dec 03 11:45:32 crc kubenswrapper[5052]: I1203 11:45:32.620361 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"0914ebfe-13c6-4309-bd10-a67850124e3f","Type":"ContainerStarted","Data":"e1e1d1a1a9474d3fdc758a17a3894a41f6c6c842db0b3603ae153366d60c47fa"} Dec 03 11:45:32 crc kubenswrapper[5052]: I1203 11:45:32.626206 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/glance-default-external-api-0" podStartSLOduration=3.62618866 podStartE2EDuration="3.62618866s" podCreationTimestamp="2025-12-03 11:45:29 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:45:32.611929909 +0000 UTC m=+1523.200171673" watchObservedRunningTime="2025-12-03 11:45:32.62618866 +0000 UTC m=+1523.214430424" Dec 03 11:45:32 crc kubenswrapper[5052]: I1203 11:45:32.626697 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/73440cd7-672e-4ace-91bd-b380dbcf681f-config-data" (OuterVolumeSpecName: "config-data") pod "73440cd7-672e-4ace-91bd-b380dbcf681f" (UID: "73440cd7-672e-4ace-91bd-b380dbcf681f"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:45:32 crc kubenswrapper[5052]: I1203 11:45:32.650999 5052 scope.go:117] "RemoveContainer" containerID="ba929f81e5969286ece6c7a6ab4237c80ee919c92a176f3b3c90b7cc34fbb30a" Dec 03 11:45:32 crc kubenswrapper[5052]: I1203 11:45:32.677436 5052 scope.go:117] "RemoveContainer" containerID="350bc5605a247a6dbdb1d22ed1c84ac14dd38e149676586bc392fb27c359ae86" Dec 03 11:45:32 crc kubenswrapper[5052]: I1203 11:45:32.702454 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/73440cd7-672e-4ace-91bd-b380dbcf681f-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:45:32 crc kubenswrapper[5052]: I1203 11:45:32.710155 5052 scope.go:117] "RemoveContainer" containerID="1acb2031482f91324fd2b811f3a7715d4cef822d78733b2935b50623edcb4c12" Dec 03 11:45:32 crc kubenswrapper[5052]: I1203 11:45:32.731242 5052 scope.go:117] "RemoveContainer" containerID="a9e5d011b3cd95329fb1749a8a824a2688dc235cdb93bab7b0ae8ce9705d1d89" Dec 03 11:45:32 crc kubenswrapper[5052]: E1203 11:45:32.732221 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a9e5d011b3cd95329fb1749a8a824a2688dc235cdb93bab7b0ae8ce9705d1d89\": container with ID starting with a9e5d011b3cd95329fb1749a8a824a2688dc235cdb93bab7b0ae8ce9705d1d89 not found: ID does not exist" containerID="a9e5d011b3cd95329fb1749a8a824a2688dc235cdb93bab7b0ae8ce9705d1d89" Dec 03 11:45:32 crc kubenswrapper[5052]: I1203 11:45:32.732273 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a9e5d011b3cd95329fb1749a8a824a2688dc235cdb93bab7b0ae8ce9705d1d89"} err="failed to get container status \"a9e5d011b3cd95329fb1749a8a824a2688dc235cdb93bab7b0ae8ce9705d1d89\": rpc error: code = NotFound desc = could not find container \"a9e5d011b3cd95329fb1749a8a824a2688dc235cdb93bab7b0ae8ce9705d1d89\": container with ID starting with a9e5d011b3cd95329fb1749a8a824a2688dc235cdb93bab7b0ae8ce9705d1d89 not found: ID does not exist" Dec 03 11:45:32 crc kubenswrapper[5052]: I1203 11:45:32.732306 5052 scope.go:117] "RemoveContainer" containerID="ba929f81e5969286ece6c7a6ab4237c80ee919c92a176f3b3c90b7cc34fbb30a" Dec 03 11:45:32 crc kubenswrapper[5052]: E1203 11:45:32.732710 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ba929f81e5969286ece6c7a6ab4237c80ee919c92a176f3b3c90b7cc34fbb30a\": container with ID starting with ba929f81e5969286ece6c7a6ab4237c80ee919c92a176f3b3c90b7cc34fbb30a not found: ID does not exist" containerID="ba929f81e5969286ece6c7a6ab4237c80ee919c92a176f3b3c90b7cc34fbb30a" Dec 03 11:45:32 crc kubenswrapper[5052]: I1203 11:45:32.732737 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ba929f81e5969286ece6c7a6ab4237c80ee919c92a176f3b3c90b7cc34fbb30a"} err="failed to get container status \"ba929f81e5969286ece6c7a6ab4237c80ee919c92a176f3b3c90b7cc34fbb30a\": rpc error: code = NotFound desc = could not find container \"ba929f81e5969286ece6c7a6ab4237c80ee919c92a176f3b3c90b7cc34fbb30a\": container with ID starting with ba929f81e5969286ece6c7a6ab4237c80ee919c92a176f3b3c90b7cc34fbb30a not found: ID does not exist" Dec 03 11:45:32 crc kubenswrapper[5052]: I1203 11:45:32.732751 5052 scope.go:117] "RemoveContainer" containerID="350bc5605a247a6dbdb1d22ed1c84ac14dd38e149676586bc392fb27c359ae86" Dec 03 11:45:32 crc kubenswrapper[5052]: E1203 11:45:32.733050 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"350bc5605a247a6dbdb1d22ed1c84ac14dd38e149676586bc392fb27c359ae86\": container with ID starting with 350bc5605a247a6dbdb1d22ed1c84ac14dd38e149676586bc392fb27c359ae86 not found: ID does not exist" containerID="350bc5605a247a6dbdb1d22ed1c84ac14dd38e149676586bc392fb27c359ae86" Dec 03 11:45:32 crc kubenswrapper[5052]: I1203 11:45:32.733069 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"350bc5605a247a6dbdb1d22ed1c84ac14dd38e149676586bc392fb27c359ae86"} err="failed to get container status \"350bc5605a247a6dbdb1d22ed1c84ac14dd38e149676586bc392fb27c359ae86\": rpc error: code = NotFound desc = could not find container \"350bc5605a247a6dbdb1d22ed1c84ac14dd38e149676586bc392fb27c359ae86\": container with ID starting with 350bc5605a247a6dbdb1d22ed1c84ac14dd38e149676586bc392fb27c359ae86 not found: ID does not exist" Dec 03 11:45:32 crc kubenswrapper[5052]: I1203 11:45:32.733081 5052 scope.go:117] "RemoveContainer" containerID="1acb2031482f91324fd2b811f3a7715d4cef822d78733b2935b50623edcb4c12" Dec 03 11:45:32 crc kubenswrapper[5052]: E1203 11:45:32.733360 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1acb2031482f91324fd2b811f3a7715d4cef822d78733b2935b50623edcb4c12\": container with ID starting with 1acb2031482f91324fd2b811f3a7715d4cef822d78733b2935b50623edcb4c12 not found: ID does not exist" containerID="1acb2031482f91324fd2b811f3a7715d4cef822d78733b2935b50623edcb4c12" Dec 03 11:45:32 crc kubenswrapper[5052]: I1203 11:45:32.733389 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1acb2031482f91324fd2b811f3a7715d4cef822d78733b2935b50623edcb4c12"} err="failed to get container status \"1acb2031482f91324fd2b811f3a7715d4cef822d78733b2935b50623edcb4c12\": rpc error: code = NotFound desc = could not find container \"1acb2031482f91324fd2b811f3a7715d4cef822d78733b2935b50623edcb4c12\": container with ID starting with 1acb2031482f91324fd2b811f3a7715d4cef822d78733b2935b50623edcb4c12 not found: ID does not exist" Dec 03 11:45:32 crc kubenswrapper[5052]: I1203 11:45:32.952296 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:45:32 crc kubenswrapper[5052]: I1203 11:45:32.962956 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:45:32 crc kubenswrapper[5052]: I1203 11:45:32.963950 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-marketplace/community-operators-fvb75" Dec 03 11:45:32 crc kubenswrapper[5052]: I1203 11:45:32.964891 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openshift-marketplace/community-operators-fvb75" Dec 03 11:45:32 crc kubenswrapper[5052]: I1203 11:45:32.981247 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:45:32 crc kubenswrapper[5052]: E1203 11:45:32.983111 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="73440cd7-672e-4ace-91bd-b380dbcf681f" containerName="ceilometer-central-agent" Dec 03 11:45:32 crc kubenswrapper[5052]: I1203 11:45:32.983138 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="73440cd7-672e-4ace-91bd-b380dbcf681f" containerName="ceilometer-central-agent" Dec 03 11:45:32 crc kubenswrapper[5052]: E1203 11:45:32.983150 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="73440cd7-672e-4ace-91bd-b380dbcf681f" containerName="sg-core" Dec 03 11:45:32 crc kubenswrapper[5052]: I1203 11:45:32.983157 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="73440cd7-672e-4ace-91bd-b380dbcf681f" containerName="sg-core" Dec 03 11:45:32 crc kubenswrapper[5052]: E1203 11:45:32.983186 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="73440cd7-672e-4ace-91bd-b380dbcf681f" containerName="ceilometer-notification-agent" Dec 03 11:45:32 crc kubenswrapper[5052]: I1203 11:45:32.983193 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="73440cd7-672e-4ace-91bd-b380dbcf681f" containerName="ceilometer-notification-agent" Dec 03 11:45:32 crc kubenswrapper[5052]: E1203 11:45:32.983211 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="73440cd7-672e-4ace-91bd-b380dbcf681f" containerName="proxy-httpd" Dec 03 11:45:32 crc kubenswrapper[5052]: I1203 11:45:32.983216 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="73440cd7-672e-4ace-91bd-b380dbcf681f" containerName="proxy-httpd" Dec 03 11:45:32 crc kubenswrapper[5052]: I1203 11:45:32.983394 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="73440cd7-672e-4ace-91bd-b380dbcf681f" containerName="ceilometer-central-agent" Dec 03 11:45:32 crc kubenswrapper[5052]: I1203 11:45:32.983622 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="73440cd7-672e-4ace-91bd-b380dbcf681f" containerName="ceilometer-notification-agent" Dec 03 11:45:32 crc kubenswrapper[5052]: I1203 11:45:32.983636 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="73440cd7-672e-4ace-91bd-b380dbcf681f" containerName="proxy-httpd" Dec 03 11:45:32 crc kubenswrapper[5052]: I1203 11:45:32.983650 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="73440cd7-672e-4ace-91bd-b380dbcf681f" containerName="sg-core" Dec 03 11:45:32 crc kubenswrapper[5052]: I1203 11:45:32.985870 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:45:32 crc kubenswrapper[5052]: I1203 11:45:32.991037 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-config-data" Dec 03 11:45:32 crc kubenswrapper[5052]: I1203 11:45:32.991343 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-scripts" Dec 03 11:45:32 crc kubenswrapper[5052]: I1203 11:45:32.991349 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-ceilometer-internal-svc" Dec 03 11:45:32 crc kubenswrapper[5052]: I1203 11:45:32.995945 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:45:33 crc kubenswrapper[5052]: I1203 11:45:33.027653 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-marketplace/community-operators-fvb75" Dec 03 11:45:33 crc kubenswrapper[5052]: I1203 11:45:33.112912 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/118e9ee6-78ab-460c-8f42-e7db0a8bf632-run-httpd\") pod \"ceilometer-0\" (UID: \"118e9ee6-78ab-460c-8f42-e7db0a8bf632\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:45:33 crc kubenswrapper[5052]: I1203 11:45:33.112968 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/118e9ee6-78ab-460c-8f42-e7db0a8bf632-ceilometer-tls-certs\") pod \"ceilometer-0\" (UID: \"118e9ee6-78ab-460c-8f42-e7db0a8bf632\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:45:33 crc kubenswrapper[5052]: I1203 11:45:33.112996 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/118e9ee6-78ab-460c-8f42-e7db0a8bf632-config-data\") pod \"ceilometer-0\" (UID: \"118e9ee6-78ab-460c-8f42-e7db0a8bf632\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:45:33 crc kubenswrapper[5052]: I1203 11:45:33.113019 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4mqz2\" (UniqueName: \"kubernetes.io/projected/118e9ee6-78ab-460c-8f42-e7db0a8bf632-kube-api-access-4mqz2\") pod \"ceilometer-0\" (UID: \"118e9ee6-78ab-460c-8f42-e7db0a8bf632\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:45:33 crc kubenswrapper[5052]: I1203 11:45:33.113053 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/118e9ee6-78ab-460c-8f42-e7db0a8bf632-scripts\") pod \"ceilometer-0\" (UID: \"118e9ee6-78ab-460c-8f42-e7db0a8bf632\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:45:33 crc kubenswrapper[5052]: I1203 11:45:33.113069 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/118e9ee6-78ab-460c-8f42-e7db0a8bf632-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"118e9ee6-78ab-460c-8f42-e7db0a8bf632\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:45:33 crc kubenswrapper[5052]: I1203 11:45:33.113161 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/118e9ee6-78ab-460c-8f42-e7db0a8bf632-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"118e9ee6-78ab-460c-8f42-e7db0a8bf632\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:45:33 crc kubenswrapper[5052]: I1203 11:45:33.113191 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/118e9ee6-78ab-460c-8f42-e7db0a8bf632-log-httpd\") pod \"ceilometer-0\" (UID: \"118e9ee6-78ab-460c-8f42-e7db0a8bf632\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:45:33 crc kubenswrapper[5052]: I1203 11:45:33.214494 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/118e9ee6-78ab-460c-8f42-e7db0a8bf632-run-httpd\") pod \"ceilometer-0\" (UID: \"118e9ee6-78ab-460c-8f42-e7db0a8bf632\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:45:33 crc kubenswrapper[5052]: I1203 11:45:33.214561 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/118e9ee6-78ab-460c-8f42-e7db0a8bf632-ceilometer-tls-certs\") pod \"ceilometer-0\" (UID: \"118e9ee6-78ab-460c-8f42-e7db0a8bf632\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:45:33 crc kubenswrapper[5052]: I1203 11:45:33.214589 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/118e9ee6-78ab-460c-8f42-e7db0a8bf632-config-data\") pod \"ceilometer-0\" (UID: \"118e9ee6-78ab-460c-8f42-e7db0a8bf632\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:45:33 crc kubenswrapper[5052]: I1203 11:45:33.214608 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-4mqz2\" (UniqueName: \"kubernetes.io/projected/118e9ee6-78ab-460c-8f42-e7db0a8bf632-kube-api-access-4mqz2\") pod \"ceilometer-0\" (UID: \"118e9ee6-78ab-460c-8f42-e7db0a8bf632\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:45:33 crc kubenswrapper[5052]: I1203 11:45:33.214643 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/118e9ee6-78ab-460c-8f42-e7db0a8bf632-scripts\") pod \"ceilometer-0\" (UID: \"118e9ee6-78ab-460c-8f42-e7db0a8bf632\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:45:33 crc kubenswrapper[5052]: I1203 11:45:33.214657 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/118e9ee6-78ab-460c-8f42-e7db0a8bf632-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"118e9ee6-78ab-460c-8f42-e7db0a8bf632\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:45:33 crc kubenswrapper[5052]: I1203 11:45:33.214684 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/118e9ee6-78ab-460c-8f42-e7db0a8bf632-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"118e9ee6-78ab-460c-8f42-e7db0a8bf632\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:45:33 crc kubenswrapper[5052]: I1203 11:45:33.214711 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/118e9ee6-78ab-460c-8f42-e7db0a8bf632-log-httpd\") pod \"ceilometer-0\" (UID: \"118e9ee6-78ab-460c-8f42-e7db0a8bf632\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:45:33 crc kubenswrapper[5052]: I1203 11:45:33.215014 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/118e9ee6-78ab-460c-8f42-e7db0a8bf632-run-httpd\") pod \"ceilometer-0\" (UID: \"118e9ee6-78ab-460c-8f42-e7db0a8bf632\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:45:33 crc kubenswrapper[5052]: I1203 11:45:33.215125 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/118e9ee6-78ab-460c-8f42-e7db0a8bf632-log-httpd\") pod \"ceilometer-0\" (UID: \"118e9ee6-78ab-460c-8f42-e7db0a8bf632\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:45:33 crc kubenswrapper[5052]: I1203 11:45:33.219012 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/118e9ee6-78ab-460c-8f42-e7db0a8bf632-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"118e9ee6-78ab-460c-8f42-e7db0a8bf632\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:45:33 crc kubenswrapper[5052]: I1203 11:45:33.219862 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/118e9ee6-78ab-460c-8f42-e7db0a8bf632-config-data\") pod \"ceilometer-0\" (UID: \"118e9ee6-78ab-460c-8f42-e7db0a8bf632\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:45:33 crc kubenswrapper[5052]: I1203 11:45:33.220546 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/118e9ee6-78ab-460c-8f42-e7db0a8bf632-ceilometer-tls-certs\") pod \"ceilometer-0\" (UID: \"118e9ee6-78ab-460c-8f42-e7db0a8bf632\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:45:33 crc kubenswrapper[5052]: I1203 11:45:33.220615 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/118e9ee6-78ab-460c-8f42-e7db0a8bf632-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"118e9ee6-78ab-460c-8f42-e7db0a8bf632\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:45:33 crc kubenswrapper[5052]: I1203 11:45:33.221087 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/118e9ee6-78ab-460c-8f42-e7db0a8bf632-scripts\") pod \"ceilometer-0\" (UID: \"118e9ee6-78ab-460c-8f42-e7db0a8bf632\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:45:33 crc kubenswrapper[5052]: I1203 11:45:33.233384 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-4mqz2\" (UniqueName: \"kubernetes.io/projected/118e9ee6-78ab-460c-8f42-e7db0a8bf632-kube-api-access-4mqz2\") pod \"ceilometer-0\" (UID: \"118e9ee6-78ab-460c-8f42-e7db0a8bf632\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:45:33 crc kubenswrapper[5052]: I1203 11:45:33.372455 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:45:33 crc kubenswrapper[5052]: I1203 11:45:33.636471 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"0914ebfe-13c6-4309-bd10-a67850124e3f","Type":"ContainerStarted","Data":"cc31e16d3323f33d812b30383dad0c83a6bb71816abde9316f28a1841980791c"} Dec 03 11:45:33 crc kubenswrapper[5052]: I1203 11:45:33.670802 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/glance-default-internal-api-0" podStartSLOduration=3.670777853 podStartE2EDuration="3.670777853s" podCreationTimestamp="2025-12-03 11:45:30 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:45:33.65854687 +0000 UTC m=+1524.246788654" watchObservedRunningTime="2025-12-03 11:45:33.670777853 +0000 UTC m=+1524.259019617" Dec 03 11:45:33 crc kubenswrapper[5052]: I1203 11:45:33.684931 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="73440cd7-672e-4ace-91bd-b380dbcf681f" path="/var/lib/kubelet/pods/73440cd7-672e-4ace-91bd-b380dbcf681f/volumes" Dec 03 11:45:33 crc kubenswrapper[5052]: I1203 11:45:33.692820 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-marketplace/community-operators-fvb75" Dec 03 11:45:33 crc kubenswrapper[5052]: I1203 11:45:33.744726 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openshift-marketplace/community-operators-fvb75"] Dec 03 11:45:33 crc kubenswrapper[5052]: I1203 11:45:33.831496 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:45:33 crc kubenswrapper[5052]: W1203 11:45:33.843377 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod118e9ee6_78ab_460c_8f42_e7db0a8bf632.slice/crio-6988ff57b302c305166ccabc243593f7e1f0cbc1fb14225d28664100bb2abdd7 WatchSource:0}: Error finding container 6988ff57b302c305166ccabc243593f7e1f0cbc1fb14225d28664100bb2abdd7: Status 404 returned error can't find the container with id 6988ff57b302c305166ccabc243593f7e1f0cbc1fb14225d28664100bb2abdd7 Dec 03 11:45:34 crc kubenswrapper[5052]: I1203 11:45:34.649908 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"118e9ee6-78ab-460c-8f42-e7db0a8bf632","Type":"ContainerStarted","Data":"0a5f58a28001495eec6556c898240aa664f99a537bf8177e79bd12695098917f"} Dec 03 11:45:34 crc kubenswrapper[5052]: I1203 11:45:34.650271 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"118e9ee6-78ab-460c-8f42-e7db0a8bf632","Type":"ContainerStarted","Data":"6988ff57b302c305166ccabc243593f7e1f0cbc1fb14225d28664100bb2abdd7"} Dec 03 11:45:34 crc kubenswrapper[5052]: I1203 11:45:34.652125 5052 generic.go:334] "Generic (PLEG): container finished" podID="8dcfd9c8-2ef8-44a2-ad54-5382d56c630d" containerID="aaed7fd471350da406347ddf3e6da7df5351689cca457e45e0443f8f32533c5e" exitCode=0 Dec 03 11:45:34 crc kubenswrapper[5052]: I1203 11:45:34.652181 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-42jqd" event={"ID":"8dcfd9c8-2ef8-44a2-ad54-5382d56c630d","Type":"ContainerDied","Data":"aaed7fd471350da406347ddf3e6da7df5351689cca457e45e0443f8f32533c5e"} Dec 03 11:45:35 crc kubenswrapper[5052]: I1203 11:45:35.661256 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"118e9ee6-78ab-460c-8f42-e7db0a8bf632","Type":"ContainerStarted","Data":"654a5e049ba0d38968af8f2a6181dd489b689a0b1ca0a64b88c4e5e9b30fc78e"} Dec 03 11:45:35 crc kubenswrapper[5052]: I1203 11:45:35.661576 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openshift-marketplace/community-operators-fvb75" podUID="f0cc2483-7e43-4536-8d99-a46d422051dd" containerName="registry-server" containerID="cri-o://8cc655ae1ba586fa750bed18700ed581d9783b3196c69bb1263ff10c0a967e6f" gracePeriod=2 Dec 03 11:45:35 crc kubenswrapper[5052]: I1203 11:45:35.945245 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-42jqd" Dec 03 11:45:36 crc kubenswrapper[5052]: I1203 11:45:36.066487 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/8dcfd9c8-2ef8-44a2-ad54-5382d56c630d-scripts\") pod \"8dcfd9c8-2ef8-44a2-ad54-5382d56c630d\" (UID: \"8dcfd9c8-2ef8-44a2-ad54-5382d56c630d\") " Dec 03 11:45:36 crc kubenswrapper[5052]: I1203 11:45:36.066672 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8dcfd9c8-2ef8-44a2-ad54-5382d56c630d-combined-ca-bundle\") pod \"8dcfd9c8-2ef8-44a2-ad54-5382d56c630d\" (UID: \"8dcfd9c8-2ef8-44a2-ad54-5382d56c630d\") " Dec 03 11:45:36 crc kubenswrapper[5052]: I1203 11:45:36.066710 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/8dcfd9c8-2ef8-44a2-ad54-5382d56c630d-config-data\") pod \"8dcfd9c8-2ef8-44a2-ad54-5382d56c630d\" (UID: \"8dcfd9c8-2ef8-44a2-ad54-5382d56c630d\") " Dec 03 11:45:36 crc kubenswrapper[5052]: I1203 11:45:36.066761 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zhrr7\" (UniqueName: \"kubernetes.io/projected/8dcfd9c8-2ef8-44a2-ad54-5382d56c630d-kube-api-access-zhrr7\") pod \"8dcfd9c8-2ef8-44a2-ad54-5382d56c630d\" (UID: \"8dcfd9c8-2ef8-44a2-ad54-5382d56c630d\") " Dec 03 11:45:36 crc kubenswrapper[5052]: I1203 11:45:36.072607 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8dcfd9c8-2ef8-44a2-ad54-5382d56c630d-kube-api-access-zhrr7" (OuterVolumeSpecName: "kube-api-access-zhrr7") pod "8dcfd9c8-2ef8-44a2-ad54-5382d56c630d" (UID: "8dcfd9c8-2ef8-44a2-ad54-5382d56c630d"). InnerVolumeSpecName "kube-api-access-zhrr7". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:45:36 crc kubenswrapper[5052]: I1203 11:45:36.076595 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8dcfd9c8-2ef8-44a2-ad54-5382d56c630d-scripts" (OuterVolumeSpecName: "scripts") pod "8dcfd9c8-2ef8-44a2-ad54-5382d56c630d" (UID: "8dcfd9c8-2ef8-44a2-ad54-5382d56c630d"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:45:36 crc kubenswrapper[5052]: I1203 11:45:36.096170 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8dcfd9c8-2ef8-44a2-ad54-5382d56c630d-config-data" (OuterVolumeSpecName: "config-data") pod "8dcfd9c8-2ef8-44a2-ad54-5382d56c630d" (UID: "8dcfd9c8-2ef8-44a2-ad54-5382d56c630d"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:45:36 crc kubenswrapper[5052]: I1203 11:45:36.102913 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8dcfd9c8-2ef8-44a2-ad54-5382d56c630d-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "8dcfd9c8-2ef8-44a2-ad54-5382d56c630d" (UID: "8dcfd9c8-2ef8-44a2-ad54-5382d56c630d"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:45:36 crc kubenswrapper[5052]: I1203 11:45:36.169399 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8dcfd9c8-2ef8-44a2-ad54-5382d56c630d-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:45:36 crc kubenswrapper[5052]: I1203 11:45:36.169431 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/8dcfd9c8-2ef8-44a2-ad54-5382d56c630d-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:45:36 crc kubenswrapper[5052]: I1203 11:45:36.169442 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-zhrr7\" (UniqueName: \"kubernetes.io/projected/8dcfd9c8-2ef8-44a2-ad54-5382d56c630d-kube-api-access-zhrr7\") on node \"crc\" DevicePath \"\"" Dec 03 11:45:36 crc kubenswrapper[5052]: I1203 11:45:36.169452 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/8dcfd9c8-2ef8-44a2-ad54-5382d56c630d-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:45:36 crc kubenswrapper[5052]: I1203 11:45:36.180091 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/community-operators-fvb75" Dec 03 11:45:36 crc kubenswrapper[5052]: I1203 11:45:36.269999 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ln58z\" (UniqueName: \"kubernetes.io/projected/f0cc2483-7e43-4536-8d99-a46d422051dd-kube-api-access-ln58z\") pod \"f0cc2483-7e43-4536-8d99-a46d422051dd\" (UID: \"f0cc2483-7e43-4536-8d99-a46d422051dd\") " Dec 03 11:45:36 crc kubenswrapper[5052]: I1203 11:45:36.270085 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/f0cc2483-7e43-4536-8d99-a46d422051dd-catalog-content\") pod \"f0cc2483-7e43-4536-8d99-a46d422051dd\" (UID: \"f0cc2483-7e43-4536-8d99-a46d422051dd\") " Dec 03 11:45:36 crc kubenswrapper[5052]: I1203 11:45:36.270126 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/f0cc2483-7e43-4536-8d99-a46d422051dd-utilities\") pod \"f0cc2483-7e43-4536-8d99-a46d422051dd\" (UID: \"f0cc2483-7e43-4536-8d99-a46d422051dd\") " Dec 03 11:45:36 crc kubenswrapper[5052]: I1203 11:45:36.271241 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0cc2483-7e43-4536-8d99-a46d422051dd-utilities" (OuterVolumeSpecName: "utilities") pod "f0cc2483-7e43-4536-8d99-a46d422051dd" (UID: "f0cc2483-7e43-4536-8d99-a46d422051dd"). InnerVolumeSpecName "utilities". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:45:36 crc kubenswrapper[5052]: I1203 11:45:36.273696 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f0cc2483-7e43-4536-8d99-a46d422051dd-kube-api-access-ln58z" (OuterVolumeSpecName: "kube-api-access-ln58z") pod "f0cc2483-7e43-4536-8d99-a46d422051dd" (UID: "f0cc2483-7e43-4536-8d99-a46d422051dd"). InnerVolumeSpecName "kube-api-access-ln58z". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:45:36 crc kubenswrapper[5052]: I1203 11:45:36.325709 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0cc2483-7e43-4536-8d99-a46d422051dd-catalog-content" (OuterVolumeSpecName: "catalog-content") pod "f0cc2483-7e43-4536-8d99-a46d422051dd" (UID: "f0cc2483-7e43-4536-8d99-a46d422051dd"). InnerVolumeSpecName "catalog-content". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:45:36 crc kubenswrapper[5052]: I1203 11:45:36.372601 5052 reconciler_common.go:293] "Volume detached for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/f0cc2483-7e43-4536-8d99-a46d422051dd-catalog-content\") on node \"crc\" DevicePath \"\"" Dec 03 11:45:36 crc kubenswrapper[5052]: I1203 11:45:36.372652 5052 reconciler_common.go:293] "Volume detached for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/f0cc2483-7e43-4536-8d99-a46d422051dd-utilities\") on node \"crc\" DevicePath \"\"" Dec 03 11:45:36 crc kubenswrapper[5052]: I1203 11:45:36.372666 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-ln58z\" (UniqueName: \"kubernetes.io/projected/f0cc2483-7e43-4536-8d99-a46d422051dd-kube-api-access-ln58z\") on node \"crc\" DevicePath \"\"" Dec 03 11:45:36 crc kubenswrapper[5052]: I1203 11:45:36.681654 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-42jqd" event={"ID":"8dcfd9c8-2ef8-44a2-ad54-5382d56c630d","Type":"ContainerDied","Data":"6b3e2ba2cca6d56649a860106c9bc4e57a58f08225072e910e1ef07582b6ff63"} Dec 03 11:45:36 crc kubenswrapper[5052]: I1203 11:45:36.681697 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6b3e2ba2cca6d56649a860106c9bc4e57a58f08225072e910e1ef07582b6ff63" Dec 03 11:45:36 crc kubenswrapper[5052]: I1203 11:45:36.681792 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-42jqd" Dec 03 11:45:36 crc kubenswrapper[5052]: I1203 11:45:36.686669 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"118e9ee6-78ab-460c-8f42-e7db0a8bf632","Type":"ContainerStarted","Data":"8aa26a6ca34ca18a8274f1330a7ce992bf80788902b95e4f72cd9c387528b7d2"} Dec 03 11:45:36 crc kubenswrapper[5052]: I1203 11:45:36.691404 5052 generic.go:334] "Generic (PLEG): container finished" podID="f0cc2483-7e43-4536-8d99-a46d422051dd" containerID="8cc655ae1ba586fa750bed18700ed581d9783b3196c69bb1263ff10c0a967e6f" exitCode=0 Dec 03 11:45:36 crc kubenswrapper[5052]: I1203 11:45:36.691479 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/community-operators-fvb75" event={"ID":"f0cc2483-7e43-4536-8d99-a46d422051dd","Type":"ContainerDied","Data":"8cc655ae1ba586fa750bed18700ed581d9783b3196c69bb1263ff10c0a967e6f"} Dec 03 11:45:36 crc kubenswrapper[5052]: I1203 11:45:36.691492 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/community-operators-fvb75" Dec 03 11:45:36 crc kubenswrapper[5052]: I1203 11:45:36.691548 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/community-operators-fvb75" event={"ID":"f0cc2483-7e43-4536-8d99-a46d422051dd","Type":"ContainerDied","Data":"eaca177a3fb82e4fe11b49247bed5bb82b8b79ba942bf9feee8cd7cf37c3c210"} Dec 03 11:45:36 crc kubenswrapper[5052]: I1203 11:45:36.691574 5052 scope.go:117] "RemoveContainer" containerID="8cc655ae1ba586fa750bed18700ed581d9783b3196c69bb1263ff10c0a967e6f" Dec 03 11:45:36 crc kubenswrapper[5052]: I1203 11:45:36.760492 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openshift-marketplace/community-operators-fvb75"] Dec 03 11:45:36 crc kubenswrapper[5052]: I1203 11:45:36.767326 5052 scope.go:117] "RemoveContainer" containerID="5356857ea35fe536b478870e1971c96b2cba923334de13d426d220aa943ef72d" Dec 03 11:45:36 crc kubenswrapper[5052]: I1203 11:45:36.781615 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openshift-marketplace/community-operators-fvb75"] Dec 03 11:45:36 crc kubenswrapper[5052]: I1203 11:45:36.790327 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-0"] Dec 03 11:45:36 crc kubenswrapper[5052]: E1203 11:45:36.790765 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="f0cc2483-7e43-4536-8d99-a46d422051dd" containerName="extract-utilities" Dec 03 11:45:36 crc kubenswrapper[5052]: I1203 11:45:36.790777 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="f0cc2483-7e43-4536-8d99-a46d422051dd" containerName="extract-utilities" Dec 03 11:45:36 crc kubenswrapper[5052]: E1203 11:45:36.790788 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="f0cc2483-7e43-4536-8d99-a46d422051dd" containerName="registry-server" Dec 03 11:45:36 crc kubenswrapper[5052]: I1203 11:45:36.790797 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="f0cc2483-7e43-4536-8d99-a46d422051dd" containerName="registry-server" Dec 03 11:45:36 crc kubenswrapper[5052]: E1203 11:45:36.790817 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="8dcfd9c8-2ef8-44a2-ad54-5382d56c630d" containerName="nova-cell0-conductor-db-sync" Dec 03 11:45:36 crc kubenswrapper[5052]: I1203 11:45:36.790823 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="8dcfd9c8-2ef8-44a2-ad54-5382d56c630d" containerName="nova-cell0-conductor-db-sync" Dec 03 11:45:36 crc kubenswrapper[5052]: E1203 11:45:36.790839 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="f0cc2483-7e43-4536-8d99-a46d422051dd" containerName="extract-content" Dec 03 11:45:36 crc kubenswrapper[5052]: I1203 11:45:36.790846 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="f0cc2483-7e43-4536-8d99-a46d422051dd" containerName="extract-content" Dec 03 11:45:36 crc kubenswrapper[5052]: I1203 11:45:36.791030 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="8dcfd9c8-2ef8-44a2-ad54-5382d56c630d" containerName="nova-cell0-conductor-db-sync" Dec 03 11:45:36 crc kubenswrapper[5052]: I1203 11:45:36.791061 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="f0cc2483-7e43-4536-8d99-a46d422051dd" containerName="registry-server" Dec 03 11:45:36 crc kubenswrapper[5052]: I1203 11:45:36.791839 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 11:45:36 crc kubenswrapper[5052]: I1203 11:45:36.797463 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-nova-dockercfg-q4mw9" Dec 03 11:45:36 crc kubenswrapper[5052]: I1203 11:45:36.797932 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-0"] Dec 03 11:45:36 crc kubenswrapper[5052]: I1203 11:45:36.802081 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell0-conductor-config-data" Dec 03 11:45:36 crc kubenswrapper[5052]: I1203 11:45:36.837008 5052 scope.go:117] "RemoveContainer" containerID="c672ad36f46e8a30f58b40807ef2de7b16aeeea1b0b227113bde460f0731c592" Dec 03 11:45:36 crc kubenswrapper[5052]: I1203 11:45:36.865768 5052 scope.go:117] "RemoveContainer" containerID="8cc655ae1ba586fa750bed18700ed581d9783b3196c69bb1263ff10c0a967e6f" Dec 03 11:45:36 crc kubenswrapper[5052]: E1203 11:45:36.866286 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8cc655ae1ba586fa750bed18700ed581d9783b3196c69bb1263ff10c0a967e6f\": container with ID starting with 8cc655ae1ba586fa750bed18700ed581d9783b3196c69bb1263ff10c0a967e6f not found: ID does not exist" containerID="8cc655ae1ba586fa750bed18700ed581d9783b3196c69bb1263ff10c0a967e6f" Dec 03 11:45:36 crc kubenswrapper[5052]: I1203 11:45:36.866323 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8cc655ae1ba586fa750bed18700ed581d9783b3196c69bb1263ff10c0a967e6f"} err="failed to get container status \"8cc655ae1ba586fa750bed18700ed581d9783b3196c69bb1263ff10c0a967e6f\": rpc error: code = NotFound desc = could not find container \"8cc655ae1ba586fa750bed18700ed581d9783b3196c69bb1263ff10c0a967e6f\": container with ID starting with 8cc655ae1ba586fa750bed18700ed581d9783b3196c69bb1263ff10c0a967e6f not found: ID does not exist" Dec 03 11:45:36 crc kubenswrapper[5052]: I1203 11:45:36.866347 5052 scope.go:117] "RemoveContainer" containerID="5356857ea35fe536b478870e1971c96b2cba923334de13d426d220aa943ef72d" Dec 03 11:45:36 crc kubenswrapper[5052]: E1203 11:45:36.866716 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5356857ea35fe536b478870e1971c96b2cba923334de13d426d220aa943ef72d\": container with ID starting with 5356857ea35fe536b478870e1971c96b2cba923334de13d426d220aa943ef72d not found: ID does not exist" containerID="5356857ea35fe536b478870e1971c96b2cba923334de13d426d220aa943ef72d" Dec 03 11:45:36 crc kubenswrapper[5052]: I1203 11:45:36.866741 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5356857ea35fe536b478870e1971c96b2cba923334de13d426d220aa943ef72d"} err="failed to get container status \"5356857ea35fe536b478870e1971c96b2cba923334de13d426d220aa943ef72d\": rpc error: code = NotFound desc = could not find container \"5356857ea35fe536b478870e1971c96b2cba923334de13d426d220aa943ef72d\": container with ID starting with 5356857ea35fe536b478870e1971c96b2cba923334de13d426d220aa943ef72d not found: ID does not exist" Dec 03 11:45:36 crc kubenswrapper[5052]: I1203 11:45:36.866759 5052 scope.go:117] "RemoveContainer" containerID="c672ad36f46e8a30f58b40807ef2de7b16aeeea1b0b227113bde460f0731c592" Dec 03 11:45:36 crc kubenswrapper[5052]: E1203 11:45:36.867000 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c672ad36f46e8a30f58b40807ef2de7b16aeeea1b0b227113bde460f0731c592\": container with ID starting with c672ad36f46e8a30f58b40807ef2de7b16aeeea1b0b227113bde460f0731c592 not found: ID does not exist" containerID="c672ad36f46e8a30f58b40807ef2de7b16aeeea1b0b227113bde460f0731c592" Dec 03 11:45:36 crc kubenswrapper[5052]: I1203 11:45:36.867025 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c672ad36f46e8a30f58b40807ef2de7b16aeeea1b0b227113bde460f0731c592"} err="failed to get container status \"c672ad36f46e8a30f58b40807ef2de7b16aeeea1b0b227113bde460f0731c592\": rpc error: code = NotFound desc = could not find container \"c672ad36f46e8a30f58b40807ef2de7b16aeeea1b0b227113bde460f0731c592\": container with ID starting with c672ad36f46e8a30f58b40807ef2de7b16aeeea1b0b227113bde460f0731c592 not found: ID does not exist" Dec 03 11:45:36 crc kubenswrapper[5052]: I1203 11:45:36.885109 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/d6d9150c-e5b6-407a-b20a-4114a60312f8-config-data\") pod \"nova-cell0-conductor-0\" (UID: \"d6d9150c-e5b6-407a-b20a-4114a60312f8\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 11:45:36 crc kubenswrapper[5052]: I1203 11:45:36.885186 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d6d9150c-e5b6-407a-b20a-4114a60312f8-combined-ca-bundle\") pod \"nova-cell0-conductor-0\" (UID: \"d6d9150c-e5b6-407a-b20a-4114a60312f8\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 11:45:36 crc kubenswrapper[5052]: I1203 11:45:36.885314 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qs6nr\" (UniqueName: \"kubernetes.io/projected/d6d9150c-e5b6-407a-b20a-4114a60312f8-kube-api-access-qs6nr\") pod \"nova-cell0-conductor-0\" (UID: \"d6d9150c-e5b6-407a-b20a-4114a60312f8\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 11:45:36 crc kubenswrapper[5052]: I1203 11:45:36.987151 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/d6d9150c-e5b6-407a-b20a-4114a60312f8-config-data\") pod \"nova-cell0-conductor-0\" (UID: \"d6d9150c-e5b6-407a-b20a-4114a60312f8\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 11:45:36 crc kubenswrapper[5052]: I1203 11:45:36.987213 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d6d9150c-e5b6-407a-b20a-4114a60312f8-combined-ca-bundle\") pod \"nova-cell0-conductor-0\" (UID: \"d6d9150c-e5b6-407a-b20a-4114a60312f8\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 11:45:36 crc kubenswrapper[5052]: I1203 11:45:36.987253 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-qs6nr\" (UniqueName: \"kubernetes.io/projected/d6d9150c-e5b6-407a-b20a-4114a60312f8-kube-api-access-qs6nr\") pod \"nova-cell0-conductor-0\" (UID: \"d6d9150c-e5b6-407a-b20a-4114a60312f8\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 11:45:36 crc kubenswrapper[5052]: I1203 11:45:36.992269 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/d6d9150c-e5b6-407a-b20a-4114a60312f8-config-data\") pod \"nova-cell0-conductor-0\" (UID: \"d6d9150c-e5b6-407a-b20a-4114a60312f8\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 11:45:36 crc kubenswrapper[5052]: I1203 11:45:36.992393 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d6d9150c-e5b6-407a-b20a-4114a60312f8-combined-ca-bundle\") pod \"nova-cell0-conductor-0\" (UID: \"d6d9150c-e5b6-407a-b20a-4114a60312f8\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 11:45:37 crc kubenswrapper[5052]: I1203 11:45:37.003324 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-qs6nr\" (UniqueName: \"kubernetes.io/projected/d6d9150c-e5b6-407a-b20a-4114a60312f8-kube-api-access-qs6nr\") pod \"nova-cell0-conductor-0\" (UID: \"d6d9150c-e5b6-407a-b20a-4114a60312f8\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 11:45:37 crc kubenswrapper[5052]: I1203 11:45:37.153689 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 11:45:37 crc kubenswrapper[5052]: I1203 11:45:37.618179 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-0"] Dec 03 11:45:37 crc kubenswrapper[5052]: I1203 11:45:37.686418 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f0cc2483-7e43-4536-8d99-a46d422051dd" path="/var/lib/kubelet/pods/f0cc2483-7e43-4536-8d99-a46d422051dd/volumes" Dec 03 11:45:37 crc kubenswrapper[5052]: I1203 11:45:37.706836 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-conductor-0" event={"ID":"d6d9150c-e5b6-407a-b20a-4114a60312f8","Type":"ContainerStarted","Data":"c8b0bf7df94802874710eff6ff62a8275187b610c63db71e41dd8ed590a58aab"} Dec 03 11:45:38 crc kubenswrapper[5052]: I1203 11:45:38.723344 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-conductor-0" event={"ID":"d6d9150c-e5b6-407a-b20a-4114a60312f8","Type":"ContainerStarted","Data":"bd5639bf60648b831ed05e44afedef6c3f8829e60604e5ebb80ecdd5b8960855"} Dec 03 11:45:38 crc kubenswrapper[5052]: I1203 11:45:38.723653 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 11:45:38 crc kubenswrapper[5052]: I1203 11:45:38.727394 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"118e9ee6-78ab-460c-8f42-e7db0a8bf632","Type":"ContainerStarted","Data":"85d3d8672f9a85a6e39aadfe82bcb0f9e330f484ac61213e268d33849beb7b60"} Dec 03 11:45:38 crc kubenswrapper[5052]: I1203 11:45:38.728224 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:45:38 crc kubenswrapper[5052]: I1203 11:45:38.741947 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-cell0-conductor-0" podStartSLOduration=2.741931308 podStartE2EDuration="2.741931308s" podCreationTimestamp="2025-12-03 11:45:36 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:45:38.741405363 +0000 UTC m=+1529.329647147" watchObservedRunningTime="2025-12-03 11:45:38.741931308 +0000 UTC m=+1529.330173072" Dec 03 11:45:38 crc kubenswrapper[5052]: I1203 11:45:38.771634 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/ceilometer-0" podStartSLOduration=2.708275441 podStartE2EDuration="6.771609404s" podCreationTimestamp="2025-12-03 11:45:32 +0000 UTC" firstStartedPulling="2025-12-03 11:45:33.845199793 +0000 UTC m=+1524.433441557" lastFinishedPulling="2025-12-03 11:45:37.908533766 +0000 UTC m=+1528.496775520" observedRunningTime="2025-12-03 11:45:38.759766153 +0000 UTC m=+1529.348007917" watchObservedRunningTime="2025-12-03 11:45:38.771609404 +0000 UTC m=+1529.359851168" Dec 03 11:45:40 crc kubenswrapper[5052]: I1203 11:45:40.182443 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:45:40 crc kubenswrapper[5052]: I1203 11:45:40.182495 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:45:40 crc kubenswrapper[5052]: I1203 11:45:40.219906 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:45:40 crc kubenswrapper[5052]: I1203 11:45:40.248051 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:45:40 crc kubenswrapper[5052]: I1203 11:45:40.764641 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:45:40 crc kubenswrapper[5052]: I1203 11:45:40.764918 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:45:41 crc kubenswrapper[5052]: I1203 11:45:41.304813 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:45:41 crc kubenswrapper[5052]: I1203 11:45:41.304902 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:45:41 crc kubenswrapper[5052]: I1203 11:45:41.333330 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:45:41 crc kubenswrapper[5052]: I1203 11:45:41.368262 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:45:41 crc kubenswrapper[5052]: I1203 11:45:41.775707 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:45:41 crc kubenswrapper[5052]: I1203 11:45:41.775755 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:45:42 crc kubenswrapper[5052]: I1203 11:45:42.190667 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 11:45:42 crc kubenswrapper[5052]: I1203 11:45:42.699977 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:45:42 crc kubenswrapper[5052]: I1203 11:45:42.784683 5052 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Dec 03 11:45:42 crc kubenswrapper[5052]: I1203 11:45:42.790408 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell0-cell-mapping-s49m7"] Dec 03 11:45:42 crc kubenswrapper[5052]: I1203 11:45:42.791683 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-cell-mapping-s49m7" Dec 03 11:45:42 crc kubenswrapper[5052]: I1203 11:45:42.801243 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell0-cell-mapping-s49m7"] Dec 03 11:45:42 crc kubenswrapper[5052]: I1203 11:45:42.824018 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:45:42 crc kubenswrapper[5052]: I1203 11:45:42.824418 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell0-manage-scripts" Dec 03 11:45:42 crc kubenswrapper[5052]: I1203 11:45:42.824730 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell0-manage-config-data" Dec 03 11:45:42 crc kubenswrapper[5052]: I1203 11:45:42.895659 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/8e87b835-163c-4f90-a10d-c35a8e5ec9dd-config-data\") pod \"nova-cell0-cell-mapping-s49m7\" (UID: \"8e87b835-163c-4f90-a10d-c35a8e5ec9dd\") " pod="openstack-kuttl-tests/nova-cell0-cell-mapping-s49m7" Dec 03 11:45:42 crc kubenswrapper[5052]: I1203 11:45:42.895796 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8e87b835-163c-4f90-a10d-c35a8e5ec9dd-combined-ca-bundle\") pod \"nova-cell0-cell-mapping-s49m7\" (UID: \"8e87b835-163c-4f90-a10d-c35a8e5ec9dd\") " pod="openstack-kuttl-tests/nova-cell0-cell-mapping-s49m7" Dec 03 11:45:42 crc kubenswrapper[5052]: I1203 11:45:42.895890 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/8e87b835-163c-4f90-a10d-c35a8e5ec9dd-scripts\") pod \"nova-cell0-cell-mapping-s49m7\" (UID: \"8e87b835-163c-4f90-a10d-c35a8e5ec9dd\") " pod="openstack-kuttl-tests/nova-cell0-cell-mapping-s49m7" Dec 03 11:45:42 crc kubenswrapper[5052]: I1203 11:45:42.895927 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-phds8\" (UniqueName: \"kubernetes.io/projected/8e87b835-163c-4f90-a10d-c35a8e5ec9dd-kube-api-access-phds8\") pod \"nova-cell0-cell-mapping-s49m7\" (UID: \"8e87b835-163c-4f90-a10d-c35a8e5ec9dd\") " pod="openstack-kuttl-tests/nova-cell0-cell-mapping-s49m7" Dec 03 11:45:42 crc kubenswrapper[5052]: I1203 11:45:42.931034 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 11:45:42 crc kubenswrapper[5052]: I1203 11:45:42.932726 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:45:42 crc kubenswrapper[5052]: I1203 11:45:42.947872 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-api-config-data" Dec 03 11:45:42 crc kubenswrapper[5052]: I1203 11:45:42.954048 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 11:45:42 crc kubenswrapper[5052]: I1203 11:45:42.998334 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/37015ee5-cb17-446c-8ebe-8481eb0b46fd-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"37015ee5-cb17-446c-8ebe-8481eb0b46fd\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:45:42 crc kubenswrapper[5052]: I1203 11:45:42.998368 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/37015ee5-cb17-446c-8ebe-8481eb0b46fd-config-data\") pod \"nova-api-0\" (UID: \"37015ee5-cb17-446c-8ebe-8481eb0b46fd\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:45:42 crc kubenswrapper[5052]: I1203 11:45:42.998398 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8e87b835-163c-4f90-a10d-c35a8e5ec9dd-combined-ca-bundle\") pod \"nova-cell0-cell-mapping-s49m7\" (UID: \"8e87b835-163c-4f90-a10d-c35a8e5ec9dd\") " pod="openstack-kuttl-tests/nova-cell0-cell-mapping-s49m7" Dec 03 11:45:42 crc kubenswrapper[5052]: I1203 11:45:42.998429 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tlb9r\" (UniqueName: \"kubernetes.io/projected/37015ee5-cb17-446c-8ebe-8481eb0b46fd-kube-api-access-tlb9r\") pod \"nova-api-0\" (UID: \"37015ee5-cb17-446c-8ebe-8481eb0b46fd\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:45:42 crc kubenswrapper[5052]: I1203 11:45:42.998459 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/37015ee5-cb17-446c-8ebe-8481eb0b46fd-logs\") pod \"nova-api-0\" (UID: \"37015ee5-cb17-446c-8ebe-8481eb0b46fd\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:45:42 crc kubenswrapper[5052]: I1203 11:45:42.998495 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/8e87b835-163c-4f90-a10d-c35a8e5ec9dd-scripts\") pod \"nova-cell0-cell-mapping-s49m7\" (UID: \"8e87b835-163c-4f90-a10d-c35a8e5ec9dd\") " pod="openstack-kuttl-tests/nova-cell0-cell-mapping-s49m7" Dec 03 11:45:42 crc kubenswrapper[5052]: I1203 11:45:42.998536 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-phds8\" (UniqueName: \"kubernetes.io/projected/8e87b835-163c-4f90-a10d-c35a8e5ec9dd-kube-api-access-phds8\") pod \"nova-cell0-cell-mapping-s49m7\" (UID: \"8e87b835-163c-4f90-a10d-c35a8e5ec9dd\") " pod="openstack-kuttl-tests/nova-cell0-cell-mapping-s49m7" Dec 03 11:45:42 crc kubenswrapper[5052]: I1203 11:45:42.998580 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/8e87b835-163c-4f90-a10d-c35a8e5ec9dd-config-data\") pod \"nova-cell0-cell-mapping-s49m7\" (UID: \"8e87b835-163c-4f90-a10d-c35a8e5ec9dd\") " pod="openstack-kuttl-tests/nova-cell0-cell-mapping-s49m7" Dec 03 11:45:43 crc kubenswrapper[5052]: I1203 11:45:43.010843 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/8e87b835-163c-4f90-a10d-c35a8e5ec9dd-config-data\") pod \"nova-cell0-cell-mapping-s49m7\" (UID: \"8e87b835-163c-4f90-a10d-c35a8e5ec9dd\") " pod="openstack-kuttl-tests/nova-cell0-cell-mapping-s49m7" Dec 03 11:45:43 crc kubenswrapper[5052]: I1203 11:45:43.010880 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/8e87b835-163c-4f90-a10d-c35a8e5ec9dd-scripts\") pod \"nova-cell0-cell-mapping-s49m7\" (UID: \"8e87b835-163c-4f90-a10d-c35a8e5ec9dd\") " pod="openstack-kuttl-tests/nova-cell0-cell-mapping-s49m7" Dec 03 11:45:43 crc kubenswrapper[5052]: I1203 11:45:43.031097 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8e87b835-163c-4f90-a10d-c35a8e5ec9dd-combined-ca-bundle\") pod \"nova-cell0-cell-mapping-s49m7\" (UID: \"8e87b835-163c-4f90-a10d-c35a8e5ec9dd\") " pod="openstack-kuttl-tests/nova-cell0-cell-mapping-s49m7" Dec 03 11:45:43 crc kubenswrapper[5052]: I1203 11:45:43.032710 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 11:45:43 crc kubenswrapper[5052]: I1203 11:45:43.034116 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:45:43 crc kubenswrapper[5052]: I1203 11:45:43.035078 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-phds8\" (UniqueName: \"kubernetes.io/projected/8e87b835-163c-4f90-a10d-c35a8e5ec9dd-kube-api-access-phds8\") pod \"nova-cell0-cell-mapping-s49m7\" (UID: \"8e87b835-163c-4f90-a10d-c35a8e5ec9dd\") " pod="openstack-kuttl-tests/nova-cell0-cell-mapping-s49m7" Dec 03 11:45:43 crc kubenswrapper[5052]: I1203 11:45:43.061295 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-scheduler-config-data" Dec 03 11:45:43 crc kubenswrapper[5052]: I1203 11:45:43.063384 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 11:45:43 crc kubenswrapper[5052]: I1203 11:45:43.101652 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-tlb9r\" (UniqueName: \"kubernetes.io/projected/37015ee5-cb17-446c-8ebe-8481eb0b46fd-kube-api-access-tlb9r\") pod \"nova-api-0\" (UID: \"37015ee5-cb17-446c-8ebe-8481eb0b46fd\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:45:43 crc kubenswrapper[5052]: I1203 11:45:43.101727 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/37015ee5-cb17-446c-8ebe-8481eb0b46fd-logs\") pod \"nova-api-0\" (UID: \"37015ee5-cb17-446c-8ebe-8481eb0b46fd\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:45:43 crc kubenswrapper[5052]: I1203 11:45:43.101857 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/3b3fbbf0-f272-4d30-8fe3-26f1cf7b04cf-config-data\") pod \"nova-scheduler-0\" (UID: \"3b3fbbf0-f272-4d30-8fe3-26f1cf7b04cf\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:45:43 crc kubenswrapper[5052]: I1203 11:45:43.101905 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3b3fbbf0-f272-4d30-8fe3-26f1cf7b04cf-combined-ca-bundle\") pod \"nova-scheduler-0\" (UID: \"3b3fbbf0-f272-4d30-8fe3-26f1cf7b04cf\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:45:43 crc kubenswrapper[5052]: I1203 11:45:43.101981 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/37015ee5-cb17-446c-8ebe-8481eb0b46fd-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"37015ee5-cb17-446c-8ebe-8481eb0b46fd\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:45:43 crc kubenswrapper[5052]: I1203 11:45:43.102007 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/37015ee5-cb17-446c-8ebe-8481eb0b46fd-config-data\") pod \"nova-api-0\" (UID: \"37015ee5-cb17-446c-8ebe-8481eb0b46fd\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:45:43 crc kubenswrapper[5052]: I1203 11:45:43.102027 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-244nn\" (UniqueName: \"kubernetes.io/projected/3b3fbbf0-f272-4d30-8fe3-26f1cf7b04cf-kube-api-access-244nn\") pod \"nova-scheduler-0\" (UID: \"3b3fbbf0-f272-4d30-8fe3-26f1cf7b04cf\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:45:43 crc kubenswrapper[5052]: I1203 11:45:43.110534 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/37015ee5-cb17-446c-8ebe-8481eb0b46fd-logs\") pod \"nova-api-0\" (UID: \"37015ee5-cb17-446c-8ebe-8481eb0b46fd\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:45:43 crc kubenswrapper[5052]: I1203 11:45:43.127388 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/37015ee5-cb17-446c-8ebe-8481eb0b46fd-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"37015ee5-cb17-446c-8ebe-8481eb0b46fd\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:45:43 crc kubenswrapper[5052]: I1203 11:45:43.132998 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/37015ee5-cb17-446c-8ebe-8481eb0b46fd-config-data\") pod \"nova-api-0\" (UID: \"37015ee5-cb17-446c-8ebe-8481eb0b46fd\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:45:43 crc kubenswrapper[5052]: I1203 11:45:43.140952 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-cell-mapping-s49m7" Dec 03 11:45:43 crc kubenswrapper[5052]: I1203 11:45:43.142127 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-tlb9r\" (UniqueName: \"kubernetes.io/projected/37015ee5-cb17-446c-8ebe-8481eb0b46fd-kube-api-access-tlb9r\") pod \"nova-api-0\" (UID: \"37015ee5-cb17-446c-8ebe-8481eb0b46fd\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:45:43 crc kubenswrapper[5052]: I1203 11:45:43.151747 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 11:45:43 crc kubenswrapper[5052]: I1203 11:45:43.167833 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:45:43 crc kubenswrapper[5052]: I1203 11:45:43.185997 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-metadata-config-data" Dec 03 11:45:43 crc kubenswrapper[5052]: I1203 11:45:43.189798 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 11:45:43 crc kubenswrapper[5052]: I1203 11:45:43.204213 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/3b3fbbf0-f272-4d30-8fe3-26f1cf7b04cf-config-data\") pod \"nova-scheduler-0\" (UID: \"3b3fbbf0-f272-4d30-8fe3-26f1cf7b04cf\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:45:43 crc kubenswrapper[5052]: I1203 11:45:43.204288 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3b3fbbf0-f272-4d30-8fe3-26f1cf7b04cf-combined-ca-bundle\") pod \"nova-scheduler-0\" (UID: \"3b3fbbf0-f272-4d30-8fe3-26f1cf7b04cf\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:45:43 crc kubenswrapper[5052]: I1203 11:45:43.204362 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-244nn\" (UniqueName: \"kubernetes.io/projected/3b3fbbf0-f272-4d30-8fe3-26f1cf7b04cf-kube-api-access-244nn\") pod \"nova-scheduler-0\" (UID: \"3b3fbbf0-f272-4d30-8fe3-26f1cf7b04cf\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:45:43 crc kubenswrapper[5052]: I1203 11:45:43.212661 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/3b3fbbf0-f272-4d30-8fe3-26f1cf7b04cf-config-data\") pod \"nova-scheduler-0\" (UID: \"3b3fbbf0-f272-4d30-8fe3-26f1cf7b04cf\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:45:43 crc kubenswrapper[5052]: I1203 11:45:43.214797 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3b3fbbf0-f272-4d30-8fe3-26f1cf7b04cf-combined-ca-bundle\") pod \"nova-scheduler-0\" (UID: \"3b3fbbf0-f272-4d30-8fe3-26f1cf7b04cf\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:45:43 crc kubenswrapper[5052]: I1203 11:45:43.231637 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 11:45:43 crc kubenswrapper[5052]: I1203 11:45:43.232799 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 11:45:43 crc kubenswrapper[5052]: I1203 11:45:43.239923 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell1-novncproxy-config-data" Dec 03 11:45:43 crc kubenswrapper[5052]: I1203 11:45:43.248914 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-244nn\" (UniqueName: \"kubernetes.io/projected/3b3fbbf0-f272-4d30-8fe3-26f1cf7b04cf-kube-api-access-244nn\") pod \"nova-scheduler-0\" (UID: \"3b3fbbf0-f272-4d30-8fe3-26f1cf7b04cf\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:45:43 crc kubenswrapper[5052]: I1203 11:45:43.259987 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:45:43 crc kubenswrapper[5052]: I1203 11:45:43.269166 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 11:45:43 crc kubenswrapper[5052]: I1203 11:45:43.269338 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:45:43 crc kubenswrapper[5052]: I1203 11:45:43.314712 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/5ab1b01a-4ba9-42c9-8982-553507457768-config-data\") pod \"nova-cell1-novncproxy-0\" (UID: \"5ab1b01a-4ba9-42c9-8982-553507457768\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 11:45:43 crc kubenswrapper[5052]: I1203 11:45:43.315016 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/83d58cd2-ac50-4a88-8f12-feb14dc8ca7a-config-data\") pod \"nova-metadata-0\" (UID: \"83d58cd2-ac50-4a88-8f12-feb14dc8ca7a\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:45:43 crc kubenswrapper[5052]: I1203 11:45:43.315049 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/83d58cd2-ac50-4a88-8f12-feb14dc8ca7a-logs\") pod \"nova-metadata-0\" (UID: \"83d58cd2-ac50-4a88-8f12-feb14dc8ca7a\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:45:43 crc kubenswrapper[5052]: I1203 11:45:43.315083 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5ab1b01a-4ba9-42c9-8982-553507457768-combined-ca-bundle\") pod \"nova-cell1-novncproxy-0\" (UID: \"5ab1b01a-4ba9-42c9-8982-553507457768\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 11:45:43 crc kubenswrapper[5052]: I1203 11:45:43.315210 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/83d58cd2-ac50-4a88-8f12-feb14dc8ca7a-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"83d58cd2-ac50-4a88-8f12-feb14dc8ca7a\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:45:43 crc kubenswrapper[5052]: I1203 11:45:43.315243 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-49knx\" (UniqueName: \"kubernetes.io/projected/83d58cd2-ac50-4a88-8f12-feb14dc8ca7a-kube-api-access-49knx\") pod \"nova-metadata-0\" (UID: \"83d58cd2-ac50-4a88-8f12-feb14dc8ca7a\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:45:43 crc kubenswrapper[5052]: I1203 11:45:43.315278 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zvg2c\" (UniqueName: \"kubernetes.io/projected/5ab1b01a-4ba9-42c9-8982-553507457768-kube-api-access-zvg2c\") pod \"nova-cell1-novncproxy-0\" (UID: \"5ab1b01a-4ba9-42c9-8982-553507457768\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 11:45:43 crc kubenswrapper[5052]: I1203 11:45:43.419014 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/83d58cd2-ac50-4a88-8f12-feb14dc8ca7a-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"83d58cd2-ac50-4a88-8f12-feb14dc8ca7a\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:45:43 crc kubenswrapper[5052]: I1203 11:45:43.419341 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-49knx\" (UniqueName: \"kubernetes.io/projected/83d58cd2-ac50-4a88-8f12-feb14dc8ca7a-kube-api-access-49knx\") pod \"nova-metadata-0\" (UID: \"83d58cd2-ac50-4a88-8f12-feb14dc8ca7a\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:45:43 crc kubenswrapper[5052]: I1203 11:45:43.419370 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-zvg2c\" (UniqueName: \"kubernetes.io/projected/5ab1b01a-4ba9-42c9-8982-553507457768-kube-api-access-zvg2c\") pod \"nova-cell1-novncproxy-0\" (UID: \"5ab1b01a-4ba9-42c9-8982-553507457768\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 11:45:43 crc kubenswrapper[5052]: I1203 11:45:43.419475 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/5ab1b01a-4ba9-42c9-8982-553507457768-config-data\") pod \"nova-cell1-novncproxy-0\" (UID: \"5ab1b01a-4ba9-42c9-8982-553507457768\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 11:45:43 crc kubenswrapper[5052]: I1203 11:45:43.419542 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/83d58cd2-ac50-4a88-8f12-feb14dc8ca7a-logs\") pod \"nova-metadata-0\" (UID: \"83d58cd2-ac50-4a88-8f12-feb14dc8ca7a\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:45:43 crc kubenswrapper[5052]: I1203 11:45:43.419561 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/83d58cd2-ac50-4a88-8f12-feb14dc8ca7a-config-data\") pod \"nova-metadata-0\" (UID: \"83d58cd2-ac50-4a88-8f12-feb14dc8ca7a\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:45:43 crc kubenswrapper[5052]: I1203 11:45:43.419580 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5ab1b01a-4ba9-42c9-8982-553507457768-combined-ca-bundle\") pod \"nova-cell1-novncproxy-0\" (UID: \"5ab1b01a-4ba9-42c9-8982-553507457768\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 11:45:43 crc kubenswrapper[5052]: I1203 11:45:43.420658 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/83d58cd2-ac50-4a88-8f12-feb14dc8ca7a-logs\") pod \"nova-metadata-0\" (UID: \"83d58cd2-ac50-4a88-8f12-feb14dc8ca7a\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:45:43 crc kubenswrapper[5052]: I1203 11:45:43.427225 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/5ab1b01a-4ba9-42c9-8982-553507457768-config-data\") pod \"nova-cell1-novncproxy-0\" (UID: \"5ab1b01a-4ba9-42c9-8982-553507457768\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 11:45:43 crc kubenswrapper[5052]: I1203 11:45:43.432985 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/83d58cd2-ac50-4a88-8f12-feb14dc8ca7a-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"83d58cd2-ac50-4a88-8f12-feb14dc8ca7a\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:45:43 crc kubenswrapper[5052]: I1203 11:45:43.433471 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5ab1b01a-4ba9-42c9-8982-553507457768-combined-ca-bundle\") pod \"nova-cell1-novncproxy-0\" (UID: \"5ab1b01a-4ba9-42c9-8982-553507457768\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 11:45:43 crc kubenswrapper[5052]: I1203 11:45:43.440361 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/83d58cd2-ac50-4a88-8f12-feb14dc8ca7a-config-data\") pod \"nova-metadata-0\" (UID: \"83d58cd2-ac50-4a88-8f12-feb14dc8ca7a\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:45:43 crc kubenswrapper[5052]: I1203 11:45:43.442228 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-zvg2c\" (UniqueName: \"kubernetes.io/projected/5ab1b01a-4ba9-42c9-8982-553507457768-kube-api-access-zvg2c\") pod \"nova-cell1-novncproxy-0\" (UID: \"5ab1b01a-4ba9-42c9-8982-553507457768\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 11:45:43 crc kubenswrapper[5052]: I1203 11:45:43.451694 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-49knx\" (UniqueName: \"kubernetes.io/projected/83d58cd2-ac50-4a88-8f12-feb14dc8ca7a-kube-api-access-49knx\") pod \"nova-metadata-0\" (UID: \"83d58cd2-ac50-4a88-8f12-feb14dc8ca7a\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:45:43 crc kubenswrapper[5052]: I1203 11:45:43.619192 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:45:43 crc kubenswrapper[5052]: I1203 11:45:43.627164 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 11:45:43 crc kubenswrapper[5052]: I1203 11:45:43.815859 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell0-cell-mapping-s49m7"] Dec 03 11:45:43 crc kubenswrapper[5052]: I1203 11:45:43.816277 5052 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Dec 03 11:45:43 crc kubenswrapper[5052]: I1203 11:45:43.816287 5052 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Dec 03 11:45:43 crc kubenswrapper[5052]: I1203 11:45:43.839827 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 11:45:43 crc kubenswrapper[5052]: I1203 11:45:43.862579 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-db-sync-mbzlf"] Dec 03 11:45:43 crc kubenswrapper[5052]: I1203 11:45:43.864038 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-mbzlf" Dec 03 11:45:43 crc kubenswrapper[5052]: I1203 11:45:43.871908 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell1-conductor-scripts" Dec 03 11:45:43 crc kubenswrapper[5052]: I1203 11:45:43.872135 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell1-conductor-config-data" Dec 03 11:45:43 crc kubenswrapper[5052]: W1203 11:45:43.874970 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8e87b835_163c_4f90_a10d_c35a8e5ec9dd.slice/crio-23bb4022a6e9b900d38ec97b273425392baa7012e0992c53de5f8f4c22be8cda WatchSource:0}: Error finding container 23bb4022a6e9b900d38ec97b273425392baa7012e0992c53de5f8f4c22be8cda: Status 404 returned error can't find the container with id 23bb4022a6e9b900d38ec97b273425392baa7012e0992c53de5f8f4c22be8cda Dec 03 11:45:43 crc kubenswrapper[5052]: W1203 11:45:43.879431 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3b3fbbf0_f272_4d30_8fe3_26f1cf7b04cf.slice/crio-515ed2f933f399091ab34f24a496fd365f87726840e3a333047ee07484c0a279 WatchSource:0}: Error finding container 515ed2f933f399091ab34f24a496fd365f87726840e3a333047ee07484c0a279: Status 404 returned error can't find the container with id 515ed2f933f399091ab34f24a496fd365f87726840e3a333047ee07484c0a279 Dec 03 11:45:43 crc kubenswrapper[5052]: I1203 11:45:43.883874 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-db-sync-mbzlf"] Dec 03 11:45:43 crc kubenswrapper[5052]: I1203 11:45:43.937469 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/b3524e58-0557-4bdd-9b3e-4584c5595b18-config-data\") pod \"nova-cell1-conductor-db-sync-mbzlf\" (UID: \"b3524e58-0557-4bdd-9b3e-4584c5595b18\") " pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-mbzlf" Dec 03 11:45:43 crc kubenswrapper[5052]: I1203 11:45:43.937875 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cvftp\" (UniqueName: \"kubernetes.io/projected/b3524e58-0557-4bdd-9b3e-4584c5595b18-kube-api-access-cvftp\") pod \"nova-cell1-conductor-db-sync-mbzlf\" (UID: \"b3524e58-0557-4bdd-9b3e-4584c5595b18\") " pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-mbzlf" Dec 03 11:45:43 crc kubenswrapper[5052]: I1203 11:45:43.937926 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b3524e58-0557-4bdd-9b3e-4584c5595b18-combined-ca-bundle\") pod \"nova-cell1-conductor-db-sync-mbzlf\" (UID: \"b3524e58-0557-4bdd-9b3e-4584c5595b18\") " pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-mbzlf" Dec 03 11:45:43 crc kubenswrapper[5052]: I1203 11:45:43.938019 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/b3524e58-0557-4bdd-9b3e-4584c5595b18-scripts\") pod \"nova-cell1-conductor-db-sync-mbzlf\" (UID: \"b3524e58-0557-4bdd-9b3e-4584c5595b18\") " pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-mbzlf" Dec 03 11:45:43 crc kubenswrapper[5052]: I1203 11:45:43.943289 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 11:45:44 crc kubenswrapper[5052]: I1203 11:45:44.040613 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/b3524e58-0557-4bdd-9b3e-4584c5595b18-config-data\") pod \"nova-cell1-conductor-db-sync-mbzlf\" (UID: \"b3524e58-0557-4bdd-9b3e-4584c5595b18\") " pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-mbzlf" Dec 03 11:45:44 crc kubenswrapper[5052]: I1203 11:45:44.041024 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-cvftp\" (UniqueName: \"kubernetes.io/projected/b3524e58-0557-4bdd-9b3e-4584c5595b18-kube-api-access-cvftp\") pod \"nova-cell1-conductor-db-sync-mbzlf\" (UID: \"b3524e58-0557-4bdd-9b3e-4584c5595b18\") " pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-mbzlf" Dec 03 11:45:44 crc kubenswrapper[5052]: I1203 11:45:44.041077 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b3524e58-0557-4bdd-9b3e-4584c5595b18-combined-ca-bundle\") pod \"nova-cell1-conductor-db-sync-mbzlf\" (UID: \"b3524e58-0557-4bdd-9b3e-4584c5595b18\") " pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-mbzlf" Dec 03 11:45:44 crc kubenswrapper[5052]: I1203 11:45:44.041139 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/b3524e58-0557-4bdd-9b3e-4584c5595b18-scripts\") pod \"nova-cell1-conductor-db-sync-mbzlf\" (UID: \"b3524e58-0557-4bdd-9b3e-4584c5595b18\") " pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-mbzlf" Dec 03 11:45:44 crc kubenswrapper[5052]: I1203 11:45:44.047846 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/b3524e58-0557-4bdd-9b3e-4584c5595b18-scripts\") pod \"nova-cell1-conductor-db-sync-mbzlf\" (UID: \"b3524e58-0557-4bdd-9b3e-4584c5595b18\") " pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-mbzlf" Dec 03 11:45:44 crc kubenswrapper[5052]: I1203 11:45:44.052824 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/b3524e58-0557-4bdd-9b3e-4584c5595b18-config-data\") pod \"nova-cell1-conductor-db-sync-mbzlf\" (UID: \"b3524e58-0557-4bdd-9b3e-4584c5595b18\") " pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-mbzlf" Dec 03 11:45:44 crc kubenswrapper[5052]: I1203 11:45:44.062246 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b3524e58-0557-4bdd-9b3e-4584c5595b18-combined-ca-bundle\") pod \"nova-cell1-conductor-db-sync-mbzlf\" (UID: \"b3524e58-0557-4bdd-9b3e-4584c5595b18\") " pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-mbzlf" Dec 03 11:45:44 crc kubenswrapper[5052]: I1203 11:45:44.076049 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-cvftp\" (UniqueName: \"kubernetes.io/projected/b3524e58-0557-4bdd-9b3e-4584c5595b18-kube-api-access-cvftp\") pod \"nova-cell1-conductor-db-sync-mbzlf\" (UID: \"b3524e58-0557-4bdd-9b3e-4584c5595b18\") " pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-mbzlf" Dec 03 11:45:44 crc kubenswrapper[5052]: I1203 11:45:44.107297 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 11:45:44 crc kubenswrapper[5052]: I1203 11:45:44.149729 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 11:45:44 crc kubenswrapper[5052]: I1203 11:45:44.190088 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-mbzlf" Dec 03 11:45:44 crc kubenswrapper[5052]: I1203 11:45:44.773442 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-db-sync-mbzlf"] Dec 03 11:45:44 crc kubenswrapper[5052]: I1203 11:45:44.804184 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:45:44 crc kubenswrapper[5052]: I1203 11:45:44.844772 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-cell-mapping-s49m7" event={"ID":"8e87b835-163c-4f90-a10d-c35a8e5ec9dd","Type":"ContainerStarted","Data":"f94cadcc0ee480f6891ba5c2d39b97896b89ebe3f6eee06d110b630861ea1a90"} Dec 03 11:45:44 crc kubenswrapper[5052]: I1203 11:45:44.844822 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-cell-mapping-s49m7" event={"ID":"8e87b835-163c-4f90-a10d-c35a8e5ec9dd","Type":"ContainerStarted","Data":"23bb4022a6e9b900d38ec97b273425392baa7012e0992c53de5f8f4c22be8cda"} Dec 03 11:45:44 crc kubenswrapper[5052]: I1203 11:45:44.861659 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"83d58cd2-ac50-4a88-8f12-feb14dc8ca7a","Type":"ContainerStarted","Data":"c77ace297cd0a8214c9bdbe1f1bde1c2d7462dcf996a97470362fe5797aadc0c"} Dec 03 11:45:44 crc kubenswrapper[5052]: I1203 11:45:44.861709 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"83d58cd2-ac50-4a88-8f12-feb14dc8ca7a","Type":"ContainerStarted","Data":"90ce2d64198cf9c425ebe35fe6b608005355881ebd7f35799f3f6bcd53c8119f"} Dec 03 11:45:44 crc kubenswrapper[5052]: I1203 11:45:44.871266 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-cell0-cell-mapping-s49m7" podStartSLOduration=2.871249188 podStartE2EDuration="2.871249188s" podCreationTimestamp="2025-12-03 11:45:42 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:45:44.868777047 +0000 UTC m=+1535.457018821" watchObservedRunningTime="2025-12-03 11:45:44.871249188 +0000 UTC m=+1535.459490952" Dec 03 11:45:44 crc kubenswrapper[5052]: I1203 11:45:44.873500 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" event={"ID":"5ab1b01a-4ba9-42c9-8982-553507457768","Type":"ContainerStarted","Data":"f294b3685147d7934602368935ffbb7bf5ac5e89387de165f3886dd96b60c75e"} Dec 03 11:45:44 crc kubenswrapper[5052]: I1203 11:45:44.873564 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" event={"ID":"5ab1b01a-4ba9-42c9-8982-553507457768","Type":"ContainerStarted","Data":"1044a26c8fb65bfe1b7d388a0ac2bca07546d63a3d5c728d196e5952de117c6a"} Dec 03 11:45:44 crc kubenswrapper[5052]: I1203 11:45:44.880882 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-mbzlf" event={"ID":"b3524e58-0557-4bdd-9b3e-4584c5595b18","Type":"ContainerStarted","Data":"95dd2b00c7aaf24bbaca4cc88e30b6011c07f1e3a7ae8b425968c72a1133406f"} Dec 03 11:45:44 crc kubenswrapper[5052]: I1203 11:45:44.898793 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"37015ee5-cb17-446c-8ebe-8481eb0b46fd","Type":"ContainerStarted","Data":"5bd027359df8453d45c83f04becf67005f205297847458e29f4ddfc2a2570f4d"} Dec 03 11:45:44 crc kubenswrapper[5052]: I1203 11:45:44.898847 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"37015ee5-cb17-446c-8ebe-8481eb0b46fd","Type":"ContainerStarted","Data":"27adc888ca29306887b48ac7bc5e6d9f3d9a010c24b3294f1a9b513a5334a5e9"} Dec 03 11:45:44 crc kubenswrapper[5052]: I1203 11:45:44.901934 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" podStartSLOduration=1.901909152 podStartE2EDuration="1.901909152s" podCreationTimestamp="2025-12-03 11:45:43 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:45:44.895820816 +0000 UTC m=+1535.484062580" watchObservedRunningTime="2025-12-03 11:45:44.901909152 +0000 UTC m=+1535.490150926" Dec 03 11:45:44 crc kubenswrapper[5052]: I1203 11:45:44.907846 5052 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Dec 03 11:45:44 crc kubenswrapper[5052]: I1203 11:45:44.909237 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"3b3fbbf0-f272-4d30-8fe3-26f1cf7b04cf","Type":"ContainerStarted","Data":"2673df9a013017aa8eda1361d017c390b3a6eca4392fbcc429ab747951199aca"} Dec 03 11:45:44 crc kubenswrapper[5052]: I1203 11:45:44.909268 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"3b3fbbf0-f272-4d30-8fe3-26f1cf7b04cf","Type":"ContainerStarted","Data":"515ed2f933f399091ab34f24a496fd365f87726840e3a333047ee07484c0a279"} Dec 03 11:45:44 crc kubenswrapper[5052]: I1203 11:45:44.937474 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-scheduler-0" podStartSLOduration=2.937451217 podStartE2EDuration="2.937451217s" podCreationTimestamp="2025-12-03 11:45:42 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:45:44.926899863 +0000 UTC m=+1535.515141627" watchObservedRunningTime="2025-12-03 11:45:44.937451217 +0000 UTC m=+1535.525692971" Dec 03 11:45:44 crc kubenswrapper[5052]: I1203 11:45:44.959935 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:45:45 crc kubenswrapper[5052]: I1203 11:45:45.918679 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"83d58cd2-ac50-4a88-8f12-feb14dc8ca7a","Type":"ContainerStarted","Data":"c0b4c2744ead71261c17c883d0708bc24a0da18160b493255dfeba73567ca991"} Dec 03 11:45:45 crc kubenswrapper[5052]: I1203 11:45:45.921068 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-mbzlf" event={"ID":"b3524e58-0557-4bdd-9b3e-4584c5595b18","Type":"ContainerStarted","Data":"90f0d1ff5147afe42adcfb385bda757769b246377b2cbf0ce7d7370e10db5e32"} Dec 03 11:45:45 crc kubenswrapper[5052]: I1203 11:45:45.923481 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"37015ee5-cb17-446c-8ebe-8481eb0b46fd","Type":"ContainerStarted","Data":"ec1beecdc1a4aa062f33038ef7661bb5fc9f12bc257825975c268a0f9659e3a7"} Dec 03 11:45:45 crc kubenswrapper[5052]: I1203 11:45:45.970820 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-metadata-0" podStartSLOduration=2.970797566 podStartE2EDuration="2.970797566s" podCreationTimestamp="2025-12-03 11:45:43 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:45:45.946564547 +0000 UTC m=+1536.534806331" watchObservedRunningTime="2025-12-03 11:45:45.970797566 +0000 UTC m=+1536.559039340" Dec 03 11:45:45 crc kubenswrapper[5052]: I1203 11:45:45.993864 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-mbzlf" podStartSLOduration=2.99384476 podStartE2EDuration="2.99384476s" podCreationTimestamp="2025-12-03 11:45:43 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:45:45.961287631 +0000 UTC m=+1536.549529405" watchObservedRunningTime="2025-12-03 11:45:45.99384476 +0000 UTC m=+1536.582086524" Dec 03 11:45:46 crc kubenswrapper[5052]: I1203 11:45:46.008276 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-api-0" podStartSLOduration=4.008250956 podStartE2EDuration="4.008250956s" podCreationTimestamp="2025-12-03 11:45:42 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:45:46.004625561 +0000 UTC m=+1536.592867335" watchObservedRunningTime="2025-12-03 11:45:46.008250956 +0000 UTC m=+1536.596492720" Dec 03 11:45:46 crc kubenswrapper[5052]: I1203 11:45:46.091614 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 11:45:46 crc kubenswrapper[5052]: I1203 11:45:46.104855 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 11:45:46 crc kubenswrapper[5052]: I1203 11:45:46.934537 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" podUID="5ab1b01a-4ba9-42c9-8982-553507457768" containerName="nova-cell1-novncproxy-novncproxy" containerID="cri-o://f294b3685147d7934602368935ffbb7bf5ac5e89387de165f3886dd96b60c75e" gracePeriod=30 Dec 03 11:45:47 crc kubenswrapper[5052]: I1203 11:45:47.970308 5052 generic.go:334] "Generic (PLEG): container finished" podID="5ab1b01a-4ba9-42c9-8982-553507457768" containerID="f294b3685147d7934602368935ffbb7bf5ac5e89387de165f3886dd96b60c75e" exitCode=0 Dec 03 11:45:47 crc kubenswrapper[5052]: I1203 11:45:47.970496 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-metadata-0" podUID="83d58cd2-ac50-4a88-8f12-feb14dc8ca7a" containerName="nova-metadata-log" containerID="cri-o://c77ace297cd0a8214c9bdbe1f1bde1c2d7462dcf996a97470362fe5797aadc0c" gracePeriod=30 Dec 03 11:45:47 crc kubenswrapper[5052]: I1203 11:45:47.970601 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" event={"ID":"5ab1b01a-4ba9-42c9-8982-553507457768","Type":"ContainerDied","Data":"f294b3685147d7934602368935ffbb7bf5ac5e89387de165f3886dd96b60c75e"} Dec 03 11:45:47 crc kubenswrapper[5052]: I1203 11:45:47.970959 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-metadata-0" podUID="83d58cd2-ac50-4a88-8f12-feb14dc8ca7a" containerName="nova-metadata-metadata" containerID="cri-o://c0b4c2744ead71261c17c883d0708bc24a0da18160b493255dfeba73567ca991" gracePeriod=30 Dec 03 11:45:48 crc kubenswrapper[5052]: I1203 11:45:48.261002 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:45:48 crc kubenswrapper[5052]: I1203 11:45:48.333798 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 11:45:48 crc kubenswrapper[5052]: I1203 11:45:48.451194 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zvg2c\" (UniqueName: \"kubernetes.io/projected/5ab1b01a-4ba9-42c9-8982-553507457768-kube-api-access-zvg2c\") pod \"5ab1b01a-4ba9-42c9-8982-553507457768\" (UID: \"5ab1b01a-4ba9-42c9-8982-553507457768\") " Dec 03 11:45:48 crc kubenswrapper[5052]: I1203 11:45:48.451319 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/5ab1b01a-4ba9-42c9-8982-553507457768-config-data\") pod \"5ab1b01a-4ba9-42c9-8982-553507457768\" (UID: \"5ab1b01a-4ba9-42c9-8982-553507457768\") " Dec 03 11:45:48 crc kubenswrapper[5052]: I1203 11:45:48.451402 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5ab1b01a-4ba9-42c9-8982-553507457768-combined-ca-bundle\") pod \"5ab1b01a-4ba9-42c9-8982-553507457768\" (UID: \"5ab1b01a-4ba9-42c9-8982-553507457768\") " Dec 03 11:45:48 crc kubenswrapper[5052]: I1203 11:45:48.457781 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5ab1b01a-4ba9-42c9-8982-553507457768-kube-api-access-zvg2c" (OuterVolumeSpecName: "kube-api-access-zvg2c") pod "5ab1b01a-4ba9-42c9-8982-553507457768" (UID: "5ab1b01a-4ba9-42c9-8982-553507457768"). InnerVolumeSpecName "kube-api-access-zvg2c". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:45:48 crc kubenswrapper[5052]: I1203 11:45:48.483173 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5ab1b01a-4ba9-42c9-8982-553507457768-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "5ab1b01a-4ba9-42c9-8982-553507457768" (UID: "5ab1b01a-4ba9-42c9-8982-553507457768"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:45:48 crc kubenswrapper[5052]: I1203 11:45:48.484868 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5ab1b01a-4ba9-42c9-8982-553507457768-config-data" (OuterVolumeSpecName: "config-data") pod "5ab1b01a-4ba9-42c9-8982-553507457768" (UID: "5ab1b01a-4ba9-42c9-8982-553507457768"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:45:48 crc kubenswrapper[5052]: I1203 11:45:48.554041 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/5ab1b01a-4ba9-42c9-8982-553507457768-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:45:48 crc kubenswrapper[5052]: I1203 11:45:48.554079 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5ab1b01a-4ba9-42c9-8982-553507457768-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:45:48 crc kubenswrapper[5052]: I1203 11:45:48.554090 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-zvg2c\" (UniqueName: \"kubernetes.io/projected/5ab1b01a-4ba9-42c9-8982-553507457768-kube-api-access-zvg2c\") on node \"crc\" DevicePath \"\"" Dec 03 11:45:48 crc kubenswrapper[5052]: I1203 11:45:48.584975 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:45:48 crc kubenswrapper[5052]: I1203 11:45:48.757166 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/83d58cd2-ac50-4a88-8f12-feb14dc8ca7a-config-data\") pod \"83d58cd2-ac50-4a88-8f12-feb14dc8ca7a\" (UID: \"83d58cd2-ac50-4a88-8f12-feb14dc8ca7a\") " Dec 03 11:45:48 crc kubenswrapper[5052]: I1203 11:45:48.757291 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/83d58cd2-ac50-4a88-8f12-feb14dc8ca7a-combined-ca-bundle\") pod \"83d58cd2-ac50-4a88-8f12-feb14dc8ca7a\" (UID: \"83d58cd2-ac50-4a88-8f12-feb14dc8ca7a\") " Dec 03 11:45:48 crc kubenswrapper[5052]: I1203 11:45:48.757345 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/83d58cd2-ac50-4a88-8f12-feb14dc8ca7a-logs\") pod \"83d58cd2-ac50-4a88-8f12-feb14dc8ca7a\" (UID: \"83d58cd2-ac50-4a88-8f12-feb14dc8ca7a\") " Dec 03 11:45:48 crc kubenswrapper[5052]: I1203 11:45:48.757379 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-49knx\" (UniqueName: \"kubernetes.io/projected/83d58cd2-ac50-4a88-8f12-feb14dc8ca7a-kube-api-access-49knx\") pod \"83d58cd2-ac50-4a88-8f12-feb14dc8ca7a\" (UID: \"83d58cd2-ac50-4a88-8f12-feb14dc8ca7a\") " Dec 03 11:45:48 crc kubenswrapper[5052]: I1203 11:45:48.757868 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83d58cd2-ac50-4a88-8f12-feb14dc8ca7a-logs" (OuterVolumeSpecName: "logs") pod "83d58cd2-ac50-4a88-8f12-feb14dc8ca7a" (UID: "83d58cd2-ac50-4a88-8f12-feb14dc8ca7a"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:45:48 crc kubenswrapper[5052]: I1203 11:45:48.758289 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/83d58cd2-ac50-4a88-8f12-feb14dc8ca7a-logs\") on node \"crc\" DevicePath \"\"" Dec 03 11:45:48 crc kubenswrapper[5052]: I1203 11:45:48.760211 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/83d58cd2-ac50-4a88-8f12-feb14dc8ca7a-kube-api-access-49knx" (OuterVolumeSpecName: "kube-api-access-49knx") pod "83d58cd2-ac50-4a88-8f12-feb14dc8ca7a" (UID: "83d58cd2-ac50-4a88-8f12-feb14dc8ca7a"). InnerVolumeSpecName "kube-api-access-49knx". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:45:48 crc kubenswrapper[5052]: I1203 11:45:48.781829 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/83d58cd2-ac50-4a88-8f12-feb14dc8ca7a-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "83d58cd2-ac50-4a88-8f12-feb14dc8ca7a" (UID: "83d58cd2-ac50-4a88-8f12-feb14dc8ca7a"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:45:48 crc kubenswrapper[5052]: I1203 11:45:48.808632 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/83d58cd2-ac50-4a88-8f12-feb14dc8ca7a-config-data" (OuterVolumeSpecName: "config-data") pod "83d58cd2-ac50-4a88-8f12-feb14dc8ca7a" (UID: "83d58cd2-ac50-4a88-8f12-feb14dc8ca7a"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:45:48 crc kubenswrapper[5052]: I1203 11:45:48.860335 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/83d58cd2-ac50-4a88-8f12-feb14dc8ca7a-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:45:48 crc kubenswrapper[5052]: I1203 11:45:48.860382 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/83d58cd2-ac50-4a88-8f12-feb14dc8ca7a-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:45:48 crc kubenswrapper[5052]: I1203 11:45:48.860400 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-49knx\" (UniqueName: \"kubernetes.io/projected/83d58cd2-ac50-4a88-8f12-feb14dc8ca7a-kube-api-access-49knx\") on node \"crc\" DevicePath \"\"" Dec 03 11:45:48 crc kubenswrapper[5052]: I1203 11:45:48.992294 5052 generic.go:334] "Generic (PLEG): container finished" podID="83d58cd2-ac50-4a88-8f12-feb14dc8ca7a" containerID="c0b4c2744ead71261c17c883d0708bc24a0da18160b493255dfeba73567ca991" exitCode=0 Dec 03 11:45:48 crc kubenswrapper[5052]: I1203 11:45:48.992325 5052 generic.go:334] "Generic (PLEG): container finished" podID="83d58cd2-ac50-4a88-8f12-feb14dc8ca7a" containerID="c77ace297cd0a8214c9bdbe1f1bde1c2d7462dcf996a97470362fe5797aadc0c" exitCode=143 Dec 03 11:45:48 crc kubenswrapper[5052]: I1203 11:45:48.992342 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"83d58cd2-ac50-4a88-8f12-feb14dc8ca7a","Type":"ContainerDied","Data":"c0b4c2744ead71261c17c883d0708bc24a0da18160b493255dfeba73567ca991"} Dec 03 11:45:48 crc kubenswrapper[5052]: I1203 11:45:48.992392 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"83d58cd2-ac50-4a88-8f12-feb14dc8ca7a","Type":"ContainerDied","Data":"c77ace297cd0a8214c9bdbe1f1bde1c2d7462dcf996a97470362fe5797aadc0c"} Dec 03 11:45:48 crc kubenswrapper[5052]: I1203 11:45:48.992407 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"83d58cd2-ac50-4a88-8f12-feb14dc8ca7a","Type":"ContainerDied","Data":"90ce2d64198cf9c425ebe35fe6b608005355881ebd7f35799f3f6bcd53c8119f"} Dec 03 11:45:48 crc kubenswrapper[5052]: I1203 11:45:48.992428 5052 scope.go:117] "RemoveContainer" containerID="c0b4c2744ead71261c17c883d0708bc24a0da18160b493255dfeba73567ca991" Dec 03 11:45:48 crc kubenswrapper[5052]: I1203 11:45:48.992435 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:45:48 crc kubenswrapper[5052]: I1203 11:45:48.993877 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" event={"ID":"5ab1b01a-4ba9-42c9-8982-553507457768","Type":"ContainerDied","Data":"1044a26c8fb65bfe1b7d388a0ac2bca07546d63a3d5c728d196e5952de117c6a"} Dec 03 11:45:48 crc kubenswrapper[5052]: I1203 11:45:48.993940 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 11:45:49 crc kubenswrapper[5052]: I1203 11:45:49.045757 5052 scope.go:117] "RemoveContainer" containerID="c77ace297cd0a8214c9bdbe1f1bde1c2d7462dcf996a97470362fe5797aadc0c" Dec 03 11:45:49 crc kubenswrapper[5052]: I1203 11:45:49.053082 5052 generic.go:334] "Generic (PLEG): container finished" podID="b3524e58-0557-4bdd-9b3e-4584c5595b18" containerID="90f0d1ff5147afe42adcfb385bda757769b246377b2cbf0ce7d7370e10db5e32" exitCode=0 Dec 03 11:45:49 crc kubenswrapper[5052]: I1203 11:45:49.053209 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-mbzlf" event={"ID":"b3524e58-0557-4bdd-9b3e-4584c5595b18","Type":"ContainerDied","Data":"90f0d1ff5147afe42adcfb385bda757769b246377b2cbf0ce7d7370e10db5e32"} Dec 03 11:45:49 crc kubenswrapper[5052]: I1203 11:45:49.125764 5052 scope.go:117] "RemoveContainer" containerID="c0b4c2744ead71261c17c883d0708bc24a0da18160b493255dfeba73567ca991" Dec 03 11:45:49 crc kubenswrapper[5052]: E1203 11:45:49.140480 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c0b4c2744ead71261c17c883d0708bc24a0da18160b493255dfeba73567ca991\": container with ID starting with c0b4c2744ead71261c17c883d0708bc24a0da18160b493255dfeba73567ca991 not found: ID does not exist" containerID="c0b4c2744ead71261c17c883d0708bc24a0da18160b493255dfeba73567ca991" Dec 03 11:45:49 crc kubenswrapper[5052]: I1203 11:45:49.141937 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c0b4c2744ead71261c17c883d0708bc24a0da18160b493255dfeba73567ca991"} err="failed to get container status \"c0b4c2744ead71261c17c883d0708bc24a0da18160b493255dfeba73567ca991\": rpc error: code = NotFound desc = could not find container \"c0b4c2744ead71261c17c883d0708bc24a0da18160b493255dfeba73567ca991\": container with ID starting with c0b4c2744ead71261c17c883d0708bc24a0da18160b493255dfeba73567ca991 not found: ID does not exist" Dec 03 11:45:49 crc kubenswrapper[5052]: I1203 11:45:49.142433 5052 scope.go:117] "RemoveContainer" containerID="c77ace297cd0a8214c9bdbe1f1bde1c2d7462dcf996a97470362fe5797aadc0c" Dec 03 11:45:49 crc kubenswrapper[5052]: I1203 11:45:49.142586 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 11:45:49 crc kubenswrapper[5052]: E1203 11:45:49.154656 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c77ace297cd0a8214c9bdbe1f1bde1c2d7462dcf996a97470362fe5797aadc0c\": container with ID starting with c77ace297cd0a8214c9bdbe1f1bde1c2d7462dcf996a97470362fe5797aadc0c not found: ID does not exist" containerID="c77ace297cd0a8214c9bdbe1f1bde1c2d7462dcf996a97470362fe5797aadc0c" Dec 03 11:45:49 crc kubenswrapper[5052]: I1203 11:45:49.154703 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c77ace297cd0a8214c9bdbe1f1bde1c2d7462dcf996a97470362fe5797aadc0c"} err="failed to get container status \"c77ace297cd0a8214c9bdbe1f1bde1c2d7462dcf996a97470362fe5797aadc0c\": rpc error: code = NotFound desc = could not find container \"c77ace297cd0a8214c9bdbe1f1bde1c2d7462dcf996a97470362fe5797aadc0c\": container with ID starting with c77ace297cd0a8214c9bdbe1f1bde1c2d7462dcf996a97470362fe5797aadc0c not found: ID does not exist" Dec 03 11:45:49 crc kubenswrapper[5052]: I1203 11:45:49.154731 5052 scope.go:117] "RemoveContainer" containerID="c0b4c2744ead71261c17c883d0708bc24a0da18160b493255dfeba73567ca991" Dec 03 11:45:49 crc kubenswrapper[5052]: I1203 11:45:49.154830 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 11:45:49 crc kubenswrapper[5052]: I1203 11:45:49.159806 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c0b4c2744ead71261c17c883d0708bc24a0da18160b493255dfeba73567ca991"} err="failed to get container status \"c0b4c2744ead71261c17c883d0708bc24a0da18160b493255dfeba73567ca991\": rpc error: code = NotFound desc = could not find container \"c0b4c2744ead71261c17c883d0708bc24a0da18160b493255dfeba73567ca991\": container with ID starting with c0b4c2744ead71261c17c883d0708bc24a0da18160b493255dfeba73567ca991 not found: ID does not exist" Dec 03 11:45:49 crc kubenswrapper[5052]: I1203 11:45:49.159852 5052 scope.go:117] "RemoveContainer" containerID="c77ace297cd0a8214c9bdbe1f1bde1c2d7462dcf996a97470362fe5797aadc0c" Dec 03 11:45:49 crc kubenswrapper[5052]: I1203 11:45:49.167361 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c77ace297cd0a8214c9bdbe1f1bde1c2d7462dcf996a97470362fe5797aadc0c"} err="failed to get container status \"c77ace297cd0a8214c9bdbe1f1bde1c2d7462dcf996a97470362fe5797aadc0c\": rpc error: code = NotFound desc = could not find container \"c77ace297cd0a8214c9bdbe1f1bde1c2d7462dcf996a97470362fe5797aadc0c\": container with ID starting with c77ace297cd0a8214c9bdbe1f1bde1c2d7462dcf996a97470362fe5797aadc0c not found: ID does not exist" Dec 03 11:45:49 crc kubenswrapper[5052]: I1203 11:45:49.167400 5052 scope.go:117] "RemoveContainer" containerID="f294b3685147d7934602368935ffbb7bf5ac5e89387de165f3886dd96b60c75e" Dec 03 11:45:49 crc kubenswrapper[5052]: I1203 11:45:49.185897 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 11:45:49 crc kubenswrapper[5052]: E1203 11:45:49.186306 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="83d58cd2-ac50-4a88-8f12-feb14dc8ca7a" containerName="nova-metadata-log" Dec 03 11:45:49 crc kubenswrapper[5052]: I1203 11:45:49.186324 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="83d58cd2-ac50-4a88-8f12-feb14dc8ca7a" containerName="nova-metadata-log" Dec 03 11:45:49 crc kubenswrapper[5052]: E1203 11:45:49.186337 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="83d58cd2-ac50-4a88-8f12-feb14dc8ca7a" containerName="nova-metadata-metadata" Dec 03 11:45:49 crc kubenswrapper[5052]: I1203 11:45:49.186344 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="83d58cd2-ac50-4a88-8f12-feb14dc8ca7a" containerName="nova-metadata-metadata" Dec 03 11:45:49 crc kubenswrapper[5052]: E1203 11:45:49.186369 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="5ab1b01a-4ba9-42c9-8982-553507457768" containerName="nova-cell1-novncproxy-novncproxy" Dec 03 11:45:49 crc kubenswrapper[5052]: I1203 11:45:49.186374 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ab1b01a-4ba9-42c9-8982-553507457768" containerName="nova-cell1-novncproxy-novncproxy" Dec 03 11:45:49 crc kubenswrapper[5052]: I1203 11:45:49.186570 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="5ab1b01a-4ba9-42c9-8982-553507457768" containerName="nova-cell1-novncproxy-novncproxy" Dec 03 11:45:49 crc kubenswrapper[5052]: I1203 11:45:49.186582 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="83d58cd2-ac50-4a88-8f12-feb14dc8ca7a" containerName="nova-metadata-metadata" Dec 03 11:45:49 crc kubenswrapper[5052]: I1203 11:45:49.186600 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="83d58cd2-ac50-4a88-8f12-feb14dc8ca7a" containerName="nova-metadata-log" Dec 03 11:45:49 crc kubenswrapper[5052]: I1203 11:45:49.187761 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:45:49 crc kubenswrapper[5052]: I1203 11:45:49.196804 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-nova-metadata-internal-svc" Dec 03 11:45:49 crc kubenswrapper[5052]: I1203 11:45:49.197076 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-metadata-config-data" Dec 03 11:45:49 crc kubenswrapper[5052]: I1203 11:45:49.246606 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 11:45:49 crc kubenswrapper[5052]: I1203 11:45:49.272542 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 11:45:49 crc kubenswrapper[5052]: I1203 11:45:49.276157 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/41c74e84-0fb9-4d22-9481-8e4ccda651ec-config-data\") pod \"nova-metadata-0\" (UID: \"41c74e84-0fb9-4d22-9481-8e4ccda651ec\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:45:49 crc kubenswrapper[5052]: I1203 11:45:49.276203 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/41c74e84-0fb9-4d22-9481-8e4ccda651ec-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"41c74e84-0fb9-4d22-9481-8e4ccda651ec\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:45:49 crc kubenswrapper[5052]: I1203 11:45:49.276256 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/41c74e84-0fb9-4d22-9481-8e4ccda651ec-nova-metadata-tls-certs\") pod \"nova-metadata-0\" (UID: \"41c74e84-0fb9-4d22-9481-8e4ccda651ec\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:45:49 crc kubenswrapper[5052]: I1203 11:45:49.276287 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/41c74e84-0fb9-4d22-9481-8e4ccda651ec-logs\") pod \"nova-metadata-0\" (UID: \"41c74e84-0fb9-4d22-9481-8e4ccda651ec\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:45:49 crc kubenswrapper[5052]: I1203 11:45:49.276358 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f7b4x\" (UniqueName: \"kubernetes.io/projected/41c74e84-0fb9-4d22-9481-8e4ccda651ec-kube-api-access-f7b4x\") pod \"nova-metadata-0\" (UID: \"41c74e84-0fb9-4d22-9481-8e4ccda651ec\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:45:49 crc kubenswrapper[5052]: I1203 11:45:49.281638 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 11:45:49 crc kubenswrapper[5052]: I1203 11:45:49.288937 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 11:45:49 crc kubenswrapper[5052]: I1203 11:45:49.290039 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 11:45:49 crc kubenswrapper[5052]: I1203 11:45:49.295171 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell1-novncproxy-config-data" Dec 03 11:45:49 crc kubenswrapper[5052]: I1203 11:45:49.295325 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-nova-novncproxy-cell1-public-svc" Dec 03 11:45:49 crc kubenswrapper[5052]: I1203 11:45:49.295456 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-nova-novncproxy-cell1-vencrypt" Dec 03 11:45:49 crc kubenswrapper[5052]: I1203 11:45:49.305153 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 11:45:49 crc kubenswrapper[5052]: I1203 11:45:49.378210 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nova-novncproxy-tls-certs\" (UniqueName: \"kubernetes.io/secret/84b8e975-30ee-42c6-8952-25aebf2c378b-nova-novncproxy-tls-certs\") pod \"nova-cell1-novncproxy-0\" (UID: \"84b8e975-30ee-42c6-8952-25aebf2c378b\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 11:45:49 crc kubenswrapper[5052]: I1203 11:45:49.378262 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-f7b4x\" (UniqueName: \"kubernetes.io/projected/41c74e84-0fb9-4d22-9481-8e4ccda651ec-kube-api-access-f7b4x\") pod \"nova-metadata-0\" (UID: \"41c74e84-0fb9-4d22-9481-8e4ccda651ec\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:45:49 crc kubenswrapper[5052]: I1203 11:45:49.378435 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/41c74e84-0fb9-4d22-9481-8e4ccda651ec-config-data\") pod \"nova-metadata-0\" (UID: \"41c74e84-0fb9-4d22-9481-8e4ccda651ec\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:45:49 crc kubenswrapper[5052]: I1203 11:45:49.378489 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/41c74e84-0fb9-4d22-9481-8e4ccda651ec-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"41c74e84-0fb9-4d22-9481-8e4ccda651ec\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:45:49 crc kubenswrapper[5052]: I1203 11:45:49.378555 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/84b8e975-30ee-42c6-8952-25aebf2c378b-config-data\") pod \"nova-cell1-novncproxy-0\" (UID: \"84b8e975-30ee-42c6-8952-25aebf2c378b\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 11:45:49 crc kubenswrapper[5052]: I1203 11:45:49.378639 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/41c74e84-0fb9-4d22-9481-8e4ccda651ec-nova-metadata-tls-certs\") pod \"nova-metadata-0\" (UID: \"41c74e84-0fb9-4d22-9481-8e4ccda651ec\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:45:49 crc kubenswrapper[5052]: I1203 11:45:49.378661 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8l45l\" (UniqueName: \"kubernetes.io/projected/84b8e975-30ee-42c6-8952-25aebf2c378b-kube-api-access-8l45l\") pod \"nova-cell1-novncproxy-0\" (UID: \"84b8e975-30ee-42c6-8952-25aebf2c378b\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 11:45:49 crc kubenswrapper[5052]: I1203 11:45:49.378704 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/41c74e84-0fb9-4d22-9481-8e4ccda651ec-logs\") pod \"nova-metadata-0\" (UID: \"41c74e84-0fb9-4d22-9481-8e4ccda651ec\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:45:49 crc kubenswrapper[5052]: I1203 11:45:49.378766 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/84b8e975-30ee-42c6-8952-25aebf2c378b-combined-ca-bundle\") pod \"nova-cell1-novncproxy-0\" (UID: \"84b8e975-30ee-42c6-8952-25aebf2c378b\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 11:45:49 crc kubenswrapper[5052]: I1203 11:45:49.378803 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"vencrypt-tls-certs\" (UniqueName: \"kubernetes.io/secret/84b8e975-30ee-42c6-8952-25aebf2c378b-vencrypt-tls-certs\") pod \"nova-cell1-novncproxy-0\" (UID: \"84b8e975-30ee-42c6-8952-25aebf2c378b\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 11:45:49 crc kubenswrapper[5052]: I1203 11:45:49.379664 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/41c74e84-0fb9-4d22-9481-8e4ccda651ec-logs\") pod \"nova-metadata-0\" (UID: \"41c74e84-0fb9-4d22-9481-8e4ccda651ec\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:45:49 crc kubenswrapper[5052]: I1203 11:45:49.383612 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/41c74e84-0fb9-4d22-9481-8e4ccda651ec-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"41c74e84-0fb9-4d22-9481-8e4ccda651ec\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:45:49 crc kubenswrapper[5052]: I1203 11:45:49.383645 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/41c74e84-0fb9-4d22-9481-8e4ccda651ec-nova-metadata-tls-certs\") pod \"nova-metadata-0\" (UID: \"41c74e84-0fb9-4d22-9481-8e4ccda651ec\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:45:49 crc kubenswrapper[5052]: I1203 11:45:49.383681 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/41c74e84-0fb9-4d22-9481-8e4ccda651ec-config-data\") pod \"nova-metadata-0\" (UID: \"41c74e84-0fb9-4d22-9481-8e4ccda651ec\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:45:49 crc kubenswrapper[5052]: I1203 11:45:49.393641 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-f7b4x\" (UniqueName: \"kubernetes.io/projected/41c74e84-0fb9-4d22-9481-8e4ccda651ec-kube-api-access-f7b4x\") pod \"nova-metadata-0\" (UID: \"41c74e84-0fb9-4d22-9481-8e4ccda651ec\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:45:49 crc kubenswrapper[5052]: I1203 11:45:49.480249 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/84b8e975-30ee-42c6-8952-25aebf2c378b-config-data\") pod \"nova-cell1-novncproxy-0\" (UID: \"84b8e975-30ee-42c6-8952-25aebf2c378b\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 11:45:49 crc kubenswrapper[5052]: I1203 11:45:49.480326 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-8l45l\" (UniqueName: \"kubernetes.io/projected/84b8e975-30ee-42c6-8952-25aebf2c378b-kube-api-access-8l45l\") pod \"nova-cell1-novncproxy-0\" (UID: \"84b8e975-30ee-42c6-8952-25aebf2c378b\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 11:45:49 crc kubenswrapper[5052]: I1203 11:45:49.480368 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/84b8e975-30ee-42c6-8952-25aebf2c378b-combined-ca-bundle\") pod \"nova-cell1-novncproxy-0\" (UID: \"84b8e975-30ee-42c6-8952-25aebf2c378b\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 11:45:49 crc kubenswrapper[5052]: I1203 11:45:49.480398 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"vencrypt-tls-certs\" (UniqueName: \"kubernetes.io/secret/84b8e975-30ee-42c6-8952-25aebf2c378b-vencrypt-tls-certs\") pod \"nova-cell1-novncproxy-0\" (UID: \"84b8e975-30ee-42c6-8952-25aebf2c378b\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 11:45:49 crc kubenswrapper[5052]: I1203 11:45:49.480441 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"nova-novncproxy-tls-certs\" (UniqueName: \"kubernetes.io/secret/84b8e975-30ee-42c6-8952-25aebf2c378b-nova-novncproxy-tls-certs\") pod \"nova-cell1-novncproxy-0\" (UID: \"84b8e975-30ee-42c6-8952-25aebf2c378b\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 11:45:49 crc kubenswrapper[5052]: I1203 11:45:49.485451 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"nova-novncproxy-tls-certs\" (UniqueName: \"kubernetes.io/secret/84b8e975-30ee-42c6-8952-25aebf2c378b-nova-novncproxy-tls-certs\") pod \"nova-cell1-novncproxy-0\" (UID: \"84b8e975-30ee-42c6-8952-25aebf2c378b\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 11:45:49 crc kubenswrapper[5052]: I1203 11:45:49.485607 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"vencrypt-tls-certs\" (UniqueName: \"kubernetes.io/secret/84b8e975-30ee-42c6-8952-25aebf2c378b-vencrypt-tls-certs\") pod \"nova-cell1-novncproxy-0\" (UID: \"84b8e975-30ee-42c6-8952-25aebf2c378b\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 11:45:49 crc kubenswrapper[5052]: I1203 11:45:49.485848 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/84b8e975-30ee-42c6-8952-25aebf2c378b-combined-ca-bundle\") pod \"nova-cell1-novncproxy-0\" (UID: \"84b8e975-30ee-42c6-8952-25aebf2c378b\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 11:45:49 crc kubenswrapper[5052]: I1203 11:45:49.485878 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/84b8e975-30ee-42c6-8952-25aebf2c378b-config-data\") pod \"nova-cell1-novncproxy-0\" (UID: \"84b8e975-30ee-42c6-8952-25aebf2c378b\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 11:45:49 crc kubenswrapper[5052]: I1203 11:45:49.505884 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-8l45l\" (UniqueName: \"kubernetes.io/projected/84b8e975-30ee-42c6-8952-25aebf2c378b-kube-api-access-8l45l\") pod \"nova-cell1-novncproxy-0\" (UID: \"84b8e975-30ee-42c6-8952-25aebf2c378b\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 11:45:49 crc kubenswrapper[5052]: I1203 11:45:49.546289 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:45:49 crc kubenswrapper[5052]: I1203 11:45:49.609373 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 11:45:49 crc kubenswrapper[5052]: I1203 11:45:49.687857 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5ab1b01a-4ba9-42c9-8982-553507457768" path="/var/lib/kubelet/pods/5ab1b01a-4ba9-42c9-8982-553507457768/volumes" Dec 03 11:45:49 crc kubenswrapper[5052]: I1203 11:45:49.688787 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="83d58cd2-ac50-4a88-8f12-feb14dc8ca7a" path="/var/lib/kubelet/pods/83d58cd2-ac50-4a88-8f12-feb14dc8ca7a/volumes" Dec 03 11:45:49 crc kubenswrapper[5052]: I1203 11:45:49.987236 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 11:45:50 crc kubenswrapper[5052]: I1203 11:45:50.072669 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"41c74e84-0fb9-4d22-9481-8e4ccda651ec","Type":"ContainerStarted","Data":"dfc04ea1be4ad187f03d43b4aa4dff576af83c4da154811416ffe2f139e6927a"} Dec 03 11:45:50 crc kubenswrapper[5052]: I1203 11:45:50.097372 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 11:45:50 crc kubenswrapper[5052]: W1203 11:45:50.106374 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod84b8e975_30ee_42c6_8952_25aebf2c378b.slice/crio-f9ede52d88dd7d8d7233a7c22830fb249ecc781d116e71bf41ca385977ba3b2f WatchSource:0}: Error finding container f9ede52d88dd7d8d7233a7c22830fb249ecc781d116e71bf41ca385977ba3b2f: Status 404 returned error can't find the container with id f9ede52d88dd7d8d7233a7c22830fb249ecc781d116e71bf41ca385977ba3b2f Dec 03 11:45:50 crc kubenswrapper[5052]: I1203 11:45:50.340396 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-mbzlf" Dec 03 11:45:50 crc kubenswrapper[5052]: I1203 11:45:50.504496 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cvftp\" (UniqueName: \"kubernetes.io/projected/b3524e58-0557-4bdd-9b3e-4584c5595b18-kube-api-access-cvftp\") pod \"b3524e58-0557-4bdd-9b3e-4584c5595b18\" (UID: \"b3524e58-0557-4bdd-9b3e-4584c5595b18\") " Dec 03 11:45:50 crc kubenswrapper[5052]: I1203 11:45:50.504909 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/b3524e58-0557-4bdd-9b3e-4584c5595b18-config-data\") pod \"b3524e58-0557-4bdd-9b3e-4584c5595b18\" (UID: \"b3524e58-0557-4bdd-9b3e-4584c5595b18\") " Dec 03 11:45:50 crc kubenswrapper[5052]: I1203 11:45:50.504986 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b3524e58-0557-4bdd-9b3e-4584c5595b18-combined-ca-bundle\") pod \"b3524e58-0557-4bdd-9b3e-4584c5595b18\" (UID: \"b3524e58-0557-4bdd-9b3e-4584c5595b18\") " Dec 03 11:45:50 crc kubenswrapper[5052]: I1203 11:45:50.505016 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/b3524e58-0557-4bdd-9b3e-4584c5595b18-scripts\") pod \"b3524e58-0557-4bdd-9b3e-4584c5595b18\" (UID: \"b3524e58-0557-4bdd-9b3e-4584c5595b18\") " Dec 03 11:45:50 crc kubenswrapper[5052]: I1203 11:45:50.509407 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b3524e58-0557-4bdd-9b3e-4584c5595b18-kube-api-access-cvftp" (OuterVolumeSpecName: "kube-api-access-cvftp") pod "b3524e58-0557-4bdd-9b3e-4584c5595b18" (UID: "b3524e58-0557-4bdd-9b3e-4584c5595b18"). InnerVolumeSpecName "kube-api-access-cvftp". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:45:50 crc kubenswrapper[5052]: I1203 11:45:50.510150 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b3524e58-0557-4bdd-9b3e-4584c5595b18-scripts" (OuterVolumeSpecName: "scripts") pod "b3524e58-0557-4bdd-9b3e-4584c5595b18" (UID: "b3524e58-0557-4bdd-9b3e-4584c5595b18"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:45:50 crc kubenswrapper[5052]: I1203 11:45:50.532498 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b3524e58-0557-4bdd-9b3e-4584c5595b18-config-data" (OuterVolumeSpecName: "config-data") pod "b3524e58-0557-4bdd-9b3e-4584c5595b18" (UID: "b3524e58-0557-4bdd-9b3e-4584c5595b18"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:45:50 crc kubenswrapper[5052]: I1203 11:45:50.539790 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b3524e58-0557-4bdd-9b3e-4584c5595b18-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "b3524e58-0557-4bdd-9b3e-4584c5595b18" (UID: "b3524e58-0557-4bdd-9b3e-4584c5595b18"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:45:50 crc kubenswrapper[5052]: I1203 11:45:50.606827 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-cvftp\" (UniqueName: \"kubernetes.io/projected/b3524e58-0557-4bdd-9b3e-4584c5595b18-kube-api-access-cvftp\") on node \"crc\" DevicePath \"\"" Dec 03 11:45:50 crc kubenswrapper[5052]: I1203 11:45:50.606867 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/b3524e58-0557-4bdd-9b3e-4584c5595b18-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:45:50 crc kubenswrapper[5052]: I1203 11:45:50.606881 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b3524e58-0557-4bdd-9b3e-4584c5595b18-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:45:50 crc kubenswrapper[5052]: I1203 11:45:50.606892 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/b3524e58-0557-4bdd-9b3e-4584c5595b18-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:45:51 crc kubenswrapper[5052]: I1203 11:45:51.088464 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"41c74e84-0fb9-4d22-9481-8e4ccda651ec","Type":"ContainerStarted","Data":"f59d83e856206ad1cbd4ee908eb2f61eb5cb1e7dfad5d358fa949077a1f21510"} Dec 03 11:45:51 crc kubenswrapper[5052]: I1203 11:45:51.088763 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"41c74e84-0fb9-4d22-9481-8e4ccda651ec","Type":"ContainerStarted","Data":"b172eee6045ee1208fd4390e391bc024cf39ea40bb186c8a63d70d066101c113"} Dec 03 11:45:51 crc kubenswrapper[5052]: I1203 11:45:51.089932 5052 generic.go:334] "Generic (PLEG): container finished" podID="8e87b835-163c-4f90-a10d-c35a8e5ec9dd" containerID="f94cadcc0ee480f6891ba5c2d39b97896b89ebe3f6eee06d110b630861ea1a90" exitCode=0 Dec 03 11:45:51 crc kubenswrapper[5052]: I1203 11:45:51.090020 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-cell-mapping-s49m7" event={"ID":"8e87b835-163c-4f90-a10d-c35a8e5ec9dd","Type":"ContainerDied","Data":"f94cadcc0ee480f6891ba5c2d39b97896b89ebe3f6eee06d110b630861ea1a90"} Dec 03 11:45:51 crc kubenswrapper[5052]: I1203 11:45:51.091862 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" event={"ID":"84b8e975-30ee-42c6-8952-25aebf2c378b","Type":"ContainerStarted","Data":"57aafbdbe2a86778650bc79b3ec5dc6502116268591f2f9f5ff1056d241c1f66"} Dec 03 11:45:51 crc kubenswrapper[5052]: I1203 11:45:51.091886 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" event={"ID":"84b8e975-30ee-42c6-8952-25aebf2c378b","Type":"ContainerStarted","Data":"f9ede52d88dd7d8d7233a7c22830fb249ecc781d116e71bf41ca385977ba3b2f"} Dec 03 11:45:51 crc kubenswrapper[5052]: I1203 11:45:51.093871 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-mbzlf" Dec 03 11:45:51 crc kubenswrapper[5052]: I1203 11:45:51.093870 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-mbzlf" event={"ID":"b3524e58-0557-4bdd-9b3e-4584c5595b18","Type":"ContainerDied","Data":"95dd2b00c7aaf24bbaca4cc88e30b6011c07f1e3a7ae8b425968c72a1133406f"} Dec 03 11:45:51 crc kubenswrapper[5052]: I1203 11:45:51.093930 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="95dd2b00c7aaf24bbaca4cc88e30b6011c07f1e3a7ae8b425968c72a1133406f" Dec 03 11:45:51 crc kubenswrapper[5052]: I1203 11:45:51.150994 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-metadata-0" podStartSLOduration=2.150969466 podStartE2EDuration="2.150969466s" podCreationTimestamp="2025-12-03 11:45:49 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:45:51.109999514 +0000 UTC m=+1541.698241278" watchObservedRunningTime="2025-12-03 11:45:51.150969466 +0000 UTC m=+1541.739211230" Dec 03 11:45:51 crc kubenswrapper[5052]: I1203 11:45:51.187451 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-0"] Dec 03 11:45:51 crc kubenswrapper[5052]: E1203 11:45:51.187920 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="b3524e58-0557-4bdd-9b3e-4584c5595b18" containerName="nova-cell1-conductor-db-sync" Dec 03 11:45:51 crc kubenswrapper[5052]: I1203 11:45:51.187944 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="b3524e58-0557-4bdd-9b3e-4584c5595b18" containerName="nova-cell1-conductor-db-sync" Dec 03 11:45:51 crc kubenswrapper[5052]: I1203 11:45:51.188213 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="b3524e58-0557-4bdd-9b3e-4584c5595b18" containerName="nova-cell1-conductor-db-sync" Dec 03 11:45:51 crc kubenswrapper[5052]: I1203 11:45:51.188894 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 11:45:51 crc kubenswrapper[5052]: I1203 11:45:51.190760 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell1-conductor-config-data" Dec 03 11:45:51 crc kubenswrapper[5052]: I1203 11:45:51.199583 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" podStartSLOduration=2.199562367 podStartE2EDuration="2.199562367s" podCreationTimestamp="2025-12-03 11:45:49 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:45:51.167963116 +0000 UTC m=+1541.756204880" watchObservedRunningTime="2025-12-03 11:45:51.199562367 +0000 UTC m=+1541.787804131" Dec 03 11:45:51 crc kubenswrapper[5052]: I1203 11:45:51.215440 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-0"] Dec 03 11:45:51 crc kubenswrapper[5052]: I1203 11:45:51.325246 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/507c60e5-056c-4661-a7a7-75c4d9bce7de-config-data\") pod \"nova-cell1-conductor-0\" (UID: \"507c60e5-056c-4661-a7a7-75c4d9bce7de\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 11:45:51 crc kubenswrapper[5052]: I1203 11:45:51.325331 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/507c60e5-056c-4661-a7a7-75c4d9bce7de-combined-ca-bundle\") pod \"nova-cell1-conductor-0\" (UID: \"507c60e5-056c-4661-a7a7-75c4d9bce7de\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 11:45:51 crc kubenswrapper[5052]: I1203 11:45:51.325420 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9cbzw\" (UniqueName: \"kubernetes.io/projected/507c60e5-056c-4661-a7a7-75c4d9bce7de-kube-api-access-9cbzw\") pod \"nova-cell1-conductor-0\" (UID: \"507c60e5-056c-4661-a7a7-75c4d9bce7de\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 11:45:51 crc kubenswrapper[5052]: I1203 11:45:51.426751 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/507c60e5-056c-4661-a7a7-75c4d9bce7de-config-data\") pod \"nova-cell1-conductor-0\" (UID: \"507c60e5-056c-4661-a7a7-75c4d9bce7de\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 11:45:51 crc kubenswrapper[5052]: I1203 11:45:51.426866 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/507c60e5-056c-4661-a7a7-75c4d9bce7de-combined-ca-bundle\") pod \"nova-cell1-conductor-0\" (UID: \"507c60e5-056c-4661-a7a7-75c4d9bce7de\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 11:45:51 crc kubenswrapper[5052]: I1203 11:45:51.426988 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-9cbzw\" (UniqueName: \"kubernetes.io/projected/507c60e5-056c-4661-a7a7-75c4d9bce7de-kube-api-access-9cbzw\") pod \"nova-cell1-conductor-0\" (UID: \"507c60e5-056c-4661-a7a7-75c4d9bce7de\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 11:45:51 crc kubenswrapper[5052]: I1203 11:45:51.432045 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/507c60e5-056c-4661-a7a7-75c4d9bce7de-combined-ca-bundle\") pod \"nova-cell1-conductor-0\" (UID: \"507c60e5-056c-4661-a7a7-75c4d9bce7de\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 11:45:51 crc kubenswrapper[5052]: I1203 11:45:51.433867 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/507c60e5-056c-4661-a7a7-75c4d9bce7de-config-data\") pod \"nova-cell1-conductor-0\" (UID: \"507c60e5-056c-4661-a7a7-75c4d9bce7de\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 11:45:51 crc kubenswrapper[5052]: I1203 11:45:51.453240 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-9cbzw\" (UniqueName: \"kubernetes.io/projected/507c60e5-056c-4661-a7a7-75c4d9bce7de-kube-api-access-9cbzw\") pod \"nova-cell1-conductor-0\" (UID: \"507c60e5-056c-4661-a7a7-75c4d9bce7de\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 11:45:51 crc kubenswrapper[5052]: I1203 11:45:51.510059 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 11:45:52 crc kubenswrapper[5052]: I1203 11:45:52.033648 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-0"] Dec 03 11:45:52 crc kubenswrapper[5052]: I1203 11:45:52.103938 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-conductor-0" event={"ID":"507c60e5-056c-4661-a7a7-75c4d9bce7de","Type":"ContainerStarted","Data":"231dd4c4a99d89479c2a56cf5b64bd4cfe99fd1b95806cca5cada9901e47dcc1"} Dec 03 11:45:52 crc kubenswrapper[5052]: I1203 11:45:52.349677 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-cell-mapping-s49m7" Dec 03 11:45:52 crc kubenswrapper[5052]: I1203 11:45:52.447575 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8e87b835-163c-4f90-a10d-c35a8e5ec9dd-combined-ca-bundle\") pod \"8e87b835-163c-4f90-a10d-c35a8e5ec9dd\" (UID: \"8e87b835-163c-4f90-a10d-c35a8e5ec9dd\") " Dec 03 11:45:52 crc kubenswrapper[5052]: I1203 11:45:52.447627 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/8e87b835-163c-4f90-a10d-c35a8e5ec9dd-scripts\") pod \"8e87b835-163c-4f90-a10d-c35a8e5ec9dd\" (UID: \"8e87b835-163c-4f90-a10d-c35a8e5ec9dd\") " Dec 03 11:45:52 crc kubenswrapper[5052]: I1203 11:45:52.447687 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-phds8\" (UniqueName: \"kubernetes.io/projected/8e87b835-163c-4f90-a10d-c35a8e5ec9dd-kube-api-access-phds8\") pod \"8e87b835-163c-4f90-a10d-c35a8e5ec9dd\" (UID: \"8e87b835-163c-4f90-a10d-c35a8e5ec9dd\") " Dec 03 11:45:52 crc kubenswrapper[5052]: I1203 11:45:52.447779 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/8e87b835-163c-4f90-a10d-c35a8e5ec9dd-config-data\") pod \"8e87b835-163c-4f90-a10d-c35a8e5ec9dd\" (UID: \"8e87b835-163c-4f90-a10d-c35a8e5ec9dd\") " Dec 03 11:45:52 crc kubenswrapper[5052]: I1203 11:45:52.452345 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8e87b835-163c-4f90-a10d-c35a8e5ec9dd-scripts" (OuterVolumeSpecName: "scripts") pod "8e87b835-163c-4f90-a10d-c35a8e5ec9dd" (UID: "8e87b835-163c-4f90-a10d-c35a8e5ec9dd"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:45:52 crc kubenswrapper[5052]: I1203 11:45:52.452741 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8e87b835-163c-4f90-a10d-c35a8e5ec9dd-kube-api-access-phds8" (OuterVolumeSpecName: "kube-api-access-phds8") pod "8e87b835-163c-4f90-a10d-c35a8e5ec9dd" (UID: "8e87b835-163c-4f90-a10d-c35a8e5ec9dd"). InnerVolumeSpecName "kube-api-access-phds8". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:45:52 crc kubenswrapper[5052]: I1203 11:45:52.476285 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8e87b835-163c-4f90-a10d-c35a8e5ec9dd-config-data" (OuterVolumeSpecName: "config-data") pod "8e87b835-163c-4f90-a10d-c35a8e5ec9dd" (UID: "8e87b835-163c-4f90-a10d-c35a8e5ec9dd"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:45:52 crc kubenswrapper[5052]: I1203 11:45:52.481643 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8e87b835-163c-4f90-a10d-c35a8e5ec9dd-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "8e87b835-163c-4f90-a10d-c35a8e5ec9dd" (UID: "8e87b835-163c-4f90-a10d-c35a8e5ec9dd"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:45:52 crc kubenswrapper[5052]: I1203 11:45:52.549598 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8e87b835-163c-4f90-a10d-c35a8e5ec9dd-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:45:52 crc kubenswrapper[5052]: I1203 11:45:52.549641 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/8e87b835-163c-4f90-a10d-c35a8e5ec9dd-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:45:52 crc kubenswrapper[5052]: I1203 11:45:52.549654 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-phds8\" (UniqueName: \"kubernetes.io/projected/8e87b835-163c-4f90-a10d-c35a8e5ec9dd-kube-api-access-phds8\") on node \"crc\" DevicePath \"\"" Dec 03 11:45:52 crc kubenswrapper[5052]: I1203 11:45:52.549668 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/8e87b835-163c-4f90-a10d-c35a8e5ec9dd-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:45:53 crc kubenswrapper[5052]: I1203 11:45:53.114781 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-cell-mapping-s49m7" event={"ID":"8e87b835-163c-4f90-a10d-c35a8e5ec9dd","Type":"ContainerDied","Data":"23bb4022a6e9b900d38ec97b273425392baa7012e0992c53de5f8f4c22be8cda"} Dec 03 11:45:53 crc kubenswrapper[5052]: I1203 11:45:53.114849 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="23bb4022a6e9b900d38ec97b273425392baa7012e0992c53de5f8f4c22be8cda" Dec 03 11:45:53 crc kubenswrapper[5052]: I1203 11:45:53.114939 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-cell-mapping-s49m7" Dec 03 11:45:53 crc kubenswrapper[5052]: I1203 11:45:53.117070 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-conductor-0" event={"ID":"507c60e5-056c-4661-a7a7-75c4d9bce7de","Type":"ContainerStarted","Data":"ad64b4dca94e6f61fc353d5aa054f7de7c10558b8c9671ffe67a4be04544faad"} Dec 03 11:45:53 crc kubenswrapper[5052]: I1203 11:45:53.117353 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 11:45:53 crc kubenswrapper[5052]: I1203 11:45:53.146811 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-cell1-conductor-0" podStartSLOduration=2.146790028 podStartE2EDuration="2.146790028s" podCreationTimestamp="2025-12-03 11:45:51 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:45:53.140188058 +0000 UTC m=+1543.728429832" watchObservedRunningTime="2025-12-03 11:45:53.146790028 +0000 UTC m=+1543.735031792" Dec 03 11:45:53 crc kubenswrapper[5052]: I1203 11:45:53.260634 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:45:53 crc kubenswrapper[5052]: I1203 11:45:53.269565 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:45:53 crc kubenswrapper[5052]: I1203 11:45:53.269616 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:45:53 crc kubenswrapper[5052]: I1203 11:45:53.294224 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:45:53 crc kubenswrapper[5052]: I1203 11:45:53.346634 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 11:45:53 crc kubenswrapper[5052]: I1203 11:45:53.360209 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 11:45:53 crc kubenswrapper[5052]: I1203 11:45:53.371363 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 11:45:53 crc kubenswrapper[5052]: I1203 11:45:53.371610 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-metadata-0" podUID="41c74e84-0fb9-4d22-9481-8e4ccda651ec" containerName="nova-metadata-log" containerID="cri-o://b172eee6045ee1208fd4390e391bc024cf39ea40bb186c8a63d70d066101c113" gracePeriod=30 Dec 03 11:45:53 crc kubenswrapper[5052]: I1203 11:45:53.371705 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-metadata-0" podUID="41c74e84-0fb9-4d22-9481-8e4ccda651ec" containerName="nova-metadata-metadata" containerID="cri-o://f59d83e856206ad1cbd4ee908eb2f61eb5cb1e7dfad5d358fa949077a1f21510" gracePeriod=30 Dec 03 11:45:54 crc kubenswrapper[5052]: I1203 11:45:54.124948 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-api-0" podUID="37015ee5-cb17-446c-8ebe-8481eb0b46fd" containerName="nova-api-log" containerID="cri-o://5bd027359df8453d45c83f04becf67005f205297847458e29f4ddfc2a2570f4d" gracePeriod=30 Dec 03 11:45:54 crc kubenswrapper[5052]: I1203 11:45:54.125035 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-api-0" podUID="37015ee5-cb17-446c-8ebe-8481eb0b46fd" containerName="nova-api-api" containerID="cri-o://ec1beecdc1a4aa062f33038ef7661bb5fc9f12bc257825975c268a0f9659e3a7" gracePeriod=30 Dec 03 11:45:54 crc kubenswrapper[5052]: I1203 11:45:54.161117 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-api-0" podUID="37015ee5-cb17-446c-8ebe-8481eb0b46fd" containerName="nova-api-log" probeResult="failure" output="Get \"http://10.217.1.19:8774/\": EOF" Dec 03 11:45:54 crc kubenswrapper[5052]: I1203 11:45:54.161301 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-api-0" podUID="37015ee5-cb17-446c-8ebe-8481eb0b46fd" containerName="nova-api-api" probeResult="failure" output="Get \"http://10.217.1.19:8774/\": EOF" Dec 03 11:45:54 crc kubenswrapper[5052]: I1203 11:45:54.172444 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:45:54 crc kubenswrapper[5052]: I1203 11:45:54.547197 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:45:54 crc kubenswrapper[5052]: I1203 11:45:54.547247 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:45:54 crc kubenswrapper[5052]: I1203 11:45:54.610373 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 11:45:55 crc kubenswrapper[5052]: I1203 11:45:55.133980 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-scheduler-0" podUID="3b3fbbf0-f272-4d30-8fe3-26f1cf7b04cf" containerName="nova-scheduler-scheduler" containerID="cri-o://2673df9a013017aa8eda1361d017c390b3a6eca4392fbcc429ab747951199aca" gracePeriod=30 Dec 03 11:45:56 crc kubenswrapper[5052]: I1203 11:45:56.145429 5052 generic.go:334] "Generic (PLEG): container finished" podID="37015ee5-cb17-446c-8ebe-8481eb0b46fd" containerID="5bd027359df8453d45c83f04becf67005f205297847458e29f4ddfc2a2570f4d" exitCode=143 Dec 03 11:45:56 crc kubenswrapper[5052]: I1203 11:45:56.145488 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"37015ee5-cb17-446c-8ebe-8481eb0b46fd","Type":"ContainerDied","Data":"5bd027359df8453d45c83f04becf67005f205297847458e29f4ddfc2a2570f4d"} Dec 03 11:45:56 crc kubenswrapper[5052]: I1203 11:45:56.147608 5052 generic.go:334] "Generic (PLEG): container finished" podID="41c74e84-0fb9-4d22-9481-8e4ccda651ec" containerID="f59d83e856206ad1cbd4ee908eb2f61eb5cb1e7dfad5d358fa949077a1f21510" exitCode=0 Dec 03 11:45:56 crc kubenswrapper[5052]: I1203 11:45:56.147626 5052 generic.go:334] "Generic (PLEG): container finished" podID="41c74e84-0fb9-4d22-9481-8e4ccda651ec" containerID="b172eee6045ee1208fd4390e391bc024cf39ea40bb186c8a63d70d066101c113" exitCode=143 Dec 03 11:45:56 crc kubenswrapper[5052]: I1203 11:45:56.147643 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"41c74e84-0fb9-4d22-9481-8e4ccda651ec","Type":"ContainerDied","Data":"f59d83e856206ad1cbd4ee908eb2f61eb5cb1e7dfad5d358fa949077a1f21510"} Dec 03 11:45:56 crc kubenswrapper[5052]: I1203 11:45:56.147661 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"41c74e84-0fb9-4d22-9481-8e4ccda651ec","Type":"ContainerDied","Data":"b172eee6045ee1208fd4390e391bc024cf39ea40bb186c8a63d70d066101c113"} Dec 03 11:45:56 crc kubenswrapper[5052]: I1203 11:45:56.650253 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:45:56 crc kubenswrapper[5052]: I1203 11:45:56.739331 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f7b4x\" (UniqueName: \"kubernetes.io/projected/41c74e84-0fb9-4d22-9481-8e4ccda651ec-kube-api-access-f7b4x\") pod \"41c74e84-0fb9-4d22-9481-8e4ccda651ec\" (UID: \"41c74e84-0fb9-4d22-9481-8e4ccda651ec\") " Dec 03 11:45:56 crc kubenswrapper[5052]: I1203 11:45:56.739375 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/41c74e84-0fb9-4d22-9481-8e4ccda651ec-config-data\") pod \"41c74e84-0fb9-4d22-9481-8e4ccda651ec\" (UID: \"41c74e84-0fb9-4d22-9481-8e4ccda651ec\") " Dec 03 11:45:56 crc kubenswrapper[5052]: I1203 11:45:56.739430 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/41c74e84-0fb9-4d22-9481-8e4ccda651ec-logs\") pod \"41c74e84-0fb9-4d22-9481-8e4ccda651ec\" (UID: \"41c74e84-0fb9-4d22-9481-8e4ccda651ec\") " Dec 03 11:45:56 crc kubenswrapper[5052]: I1203 11:45:56.739636 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/41c74e84-0fb9-4d22-9481-8e4ccda651ec-combined-ca-bundle\") pod \"41c74e84-0fb9-4d22-9481-8e4ccda651ec\" (UID: \"41c74e84-0fb9-4d22-9481-8e4ccda651ec\") " Dec 03 11:45:56 crc kubenswrapper[5052]: I1203 11:45:56.739664 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/41c74e84-0fb9-4d22-9481-8e4ccda651ec-nova-metadata-tls-certs\") pod \"41c74e84-0fb9-4d22-9481-8e4ccda651ec\" (UID: \"41c74e84-0fb9-4d22-9481-8e4ccda651ec\") " Dec 03 11:45:56 crc kubenswrapper[5052]: I1203 11:45:56.740177 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/41c74e84-0fb9-4d22-9481-8e4ccda651ec-logs" (OuterVolumeSpecName: "logs") pod "41c74e84-0fb9-4d22-9481-8e4ccda651ec" (UID: "41c74e84-0fb9-4d22-9481-8e4ccda651ec"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:45:56 crc kubenswrapper[5052]: I1203 11:45:56.747919 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/41c74e84-0fb9-4d22-9481-8e4ccda651ec-kube-api-access-f7b4x" (OuterVolumeSpecName: "kube-api-access-f7b4x") pod "41c74e84-0fb9-4d22-9481-8e4ccda651ec" (UID: "41c74e84-0fb9-4d22-9481-8e4ccda651ec"). InnerVolumeSpecName "kube-api-access-f7b4x". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:45:56 crc kubenswrapper[5052]: I1203 11:45:56.766294 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/41c74e84-0fb9-4d22-9481-8e4ccda651ec-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "41c74e84-0fb9-4d22-9481-8e4ccda651ec" (UID: "41c74e84-0fb9-4d22-9481-8e4ccda651ec"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:45:56 crc kubenswrapper[5052]: I1203 11:45:56.768884 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/41c74e84-0fb9-4d22-9481-8e4ccda651ec-config-data" (OuterVolumeSpecName: "config-data") pod "41c74e84-0fb9-4d22-9481-8e4ccda651ec" (UID: "41c74e84-0fb9-4d22-9481-8e4ccda651ec"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:45:56 crc kubenswrapper[5052]: I1203 11:45:56.797233 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/41c74e84-0fb9-4d22-9481-8e4ccda651ec-nova-metadata-tls-certs" (OuterVolumeSpecName: "nova-metadata-tls-certs") pod "41c74e84-0fb9-4d22-9481-8e4ccda651ec" (UID: "41c74e84-0fb9-4d22-9481-8e4ccda651ec"). InnerVolumeSpecName "nova-metadata-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:45:56 crc kubenswrapper[5052]: I1203 11:45:56.841277 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/41c74e84-0fb9-4d22-9481-8e4ccda651ec-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:45:56 crc kubenswrapper[5052]: I1203 11:45:56.841435 5052 reconciler_common.go:293] "Volume detached for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/41c74e84-0fb9-4d22-9481-8e4ccda651ec-nova-metadata-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 11:45:56 crc kubenswrapper[5052]: I1203 11:45:56.841546 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-f7b4x\" (UniqueName: \"kubernetes.io/projected/41c74e84-0fb9-4d22-9481-8e4ccda651ec-kube-api-access-f7b4x\") on node \"crc\" DevicePath \"\"" Dec 03 11:45:56 crc kubenswrapper[5052]: I1203 11:45:56.841632 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/41c74e84-0fb9-4d22-9481-8e4ccda651ec-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:45:56 crc kubenswrapper[5052]: I1203 11:45:56.841713 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/41c74e84-0fb9-4d22-9481-8e4ccda651ec-logs\") on node \"crc\" DevicePath \"\"" Dec 03 11:45:57 crc kubenswrapper[5052]: I1203 11:45:57.108204 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:45:57 crc kubenswrapper[5052]: I1203 11:45:57.184078 5052 generic.go:334] "Generic (PLEG): container finished" podID="3b3fbbf0-f272-4d30-8fe3-26f1cf7b04cf" containerID="2673df9a013017aa8eda1361d017c390b3a6eca4392fbcc429ab747951199aca" exitCode=0 Dec 03 11:45:57 crc kubenswrapper[5052]: I1203 11:45:57.184132 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"3b3fbbf0-f272-4d30-8fe3-26f1cf7b04cf","Type":"ContainerDied","Data":"2673df9a013017aa8eda1361d017c390b3a6eca4392fbcc429ab747951199aca"} Dec 03 11:45:57 crc kubenswrapper[5052]: I1203 11:45:57.184167 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:45:57 crc kubenswrapper[5052]: I1203 11:45:57.184186 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"3b3fbbf0-f272-4d30-8fe3-26f1cf7b04cf","Type":"ContainerDied","Data":"515ed2f933f399091ab34f24a496fd365f87726840e3a333047ee07484c0a279"} Dec 03 11:45:57 crc kubenswrapper[5052]: I1203 11:45:57.184206 5052 scope.go:117] "RemoveContainer" containerID="2673df9a013017aa8eda1361d017c390b3a6eca4392fbcc429ab747951199aca" Dec 03 11:45:57 crc kubenswrapper[5052]: I1203 11:45:57.189969 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"41c74e84-0fb9-4d22-9481-8e4ccda651ec","Type":"ContainerDied","Data":"dfc04ea1be4ad187f03d43b4aa4dff576af83c4da154811416ffe2f139e6927a"} Dec 03 11:45:57 crc kubenswrapper[5052]: I1203 11:45:57.190020 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:45:57 crc kubenswrapper[5052]: I1203 11:45:57.214745 5052 scope.go:117] "RemoveContainer" containerID="2673df9a013017aa8eda1361d017c390b3a6eca4392fbcc429ab747951199aca" Dec 03 11:45:57 crc kubenswrapper[5052]: E1203 11:45:57.218997 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2673df9a013017aa8eda1361d017c390b3a6eca4392fbcc429ab747951199aca\": container with ID starting with 2673df9a013017aa8eda1361d017c390b3a6eca4392fbcc429ab747951199aca not found: ID does not exist" containerID="2673df9a013017aa8eda1361d017c390b3a6eca4392fbcc429ab747951199aca" Dec 03 11:45:57 crc kubenswrapper[5052]: I1203 11:45:57.223974 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2673df9a013017aa8eda1361d017c390b3a6eca4392fbcc429ab747951199aca"} err="failed to get container status \"2673df9a013017aa8eda1361d017c390b3a6eca4392fbcc429ab747951199aca\": rpc error: code = NotFound desc = could not find container \"2673df9a013017aa8eda1361d017c390b3a6eca4392fbcc429ab747951199aca\": container with ID starting with 2673df9a013017aa8eda1361d017c390b3a6eca4392fbcc429ab747951199aca not found: ID does not exist" Dec 03 11:45:57 crc kubenswrapper[5052]: I1203 11:45:57.224017 5052 scope.go:117] "RemoveContainer" containerID="f59d83e856206ad1cbd4ee908eb2f61eb5cb1e7dfad5d358fa949077a1f21510" Dec 03 11:45:57 crc kubenswrapper[5052]: I1203 11:45:57.248121 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-244nn\" (UniqueName: \"kubernetes.io/projected/3b3fbbf0-f272-4d30-8fe3-26f1cf7b04cf-kube-api-access-244nn\") pod \"3b3fbbf0-f272-4d30-8fe3-26f1cf7b04cf\" (UID: \"3b3fbbf0-f272-4d30-8fe3-26f1cf7b04cf\") " Dec 03 11:45:57 crc kubenswrapper[5052]: I1203 11:45:57.248191 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3b3fbbf0-f272-4d30-8fe3-26f1cf7b04cf-combined-ca-bundle\") pod \"3b3fbbf0-f272-4d30-8fe3-26f1cf7b04cf\" (UID: \"3b3fbbf0-f272-4d30-8fe3-26f1cf7b04cf\") " Dec 03 11:45:57 crc kubenswrapper[5052]: I1203 11:45:57.248249 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/3b3fbbf0-f272-4d30-8fe3-26f1cf7b04cf-config-data\") pod \"3b3fbbf0-f272-4d30-8fe3-26f1cf7b04cf\" (UID: \"3b3fbbf0-f272-4d30-8fe3-26f1cf7b04cf\") " Dec 03 11:45:57 crc kubenswrapper[5052]: I1203 11:45:57.259638 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3b3fbbf0-f272-4d30-8fe3-26f1cf7b04cf-kube-api-access-244nn" (OuterVolumeSpecName: "kube-api-access-244nn") pod "3b3fbbf0-f272-4d30-8fe3-26f1cf7b04cf" (UID: "3b3fbbf0-f272-4d30-8fe3-26f1cf7b04cf"). InnerVolumeSpecName "kube-api-access-244nn". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:45:57 crc kubenswrapper[5052]: I1203 11:45:57.266106 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 11:45:57 crc kubenswrapper[5052]: E1203 11:45:57.285451 5052 cadvisor_stats_provider.go:516] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod41c74e84_0fb9_4d22_9481_8e4ccda651ec.slice\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod41c74e84_0fb9_4d22_9481_8e4ccda651ec.slice/crio-dfc04ea1be4ad187f03d43b4aa4dff576af83c4da154811416ffe2f139e6927a\": RecentStats: unable to find data in memory cache]" Dec 03 11:45:57 crc kubenswrapper[5052]: I1203 11:45:57.299155 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 11:45:57 crc kubenswrapper[5052]: I1203 11:45:57.305862 5052 scope.go:117] "RemoveContainer" containerID="b172eee6045ee1208fd4390e391bc024cf39ea40bb186c8a63d70d066101c113" Dec 03 11:45:57 crc kubenswrapper[5052]: I1203 11:45:57.307718 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3b3fbbf0-f272-4d30-8fe3-26f1cf7b04cf-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "3b3fbbf0-f272-4d30-8fe3-26f1cf7b04cf" (UID: "3b3fbbf0-f272-4d30-8fe3-26f1cf7b04cf"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:45:57 crc kubenswrapper[5052]: I1203 11:45:57.312519 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3b3fbbf0-f272-4d30-8fe3-26f1cf7b04cf-config-data" (OuterVolumeSpecName: "config-data") pod "3b3fbbf0-f272-4d30-8fe3-26f1cf7b04cf" (UID: "3b3fbbf0-f272-4d30-8fe3-26f1cf7b04cf"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:45:57 crc kubenswrapper[5052]: I1203 11:45:57.320874 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 11:45:57 crc kubenswrapper[5052]: E1203 11:45:57.321301 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="41c74e84-0fb9-4d22-9481-8e4ccda651ec" containerName="nova-metadata-log" Dec 03 11:45:57 crc kubenswrapper[5052]: I1203 11:45:57.321339 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="41c74e84-0fb9-4d22-9481-8e4ccda651ec" containerName="nova-metadata-log" Dec 03 11:45:57 crc kubenswrapper[5052]: E1203 11:45:57.321356 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="8e87b835-163c-4f90-a10d-c35a8e5ec9dd" containerName="nova-manage" Dec 03 11:45:57 crc kubenswrapper[5052]: I1203 11:45:57.321362 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="8e87b835-163c-4f90-a10d-c35a8e5ec9dd" containerName="nova-manage" Dec 03 11:45:57 crc kubenswrapper[5052]: E1203 11:45:57.321383 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="3b3fbbf0-f272-4d30-8fe3-26f1cf7b04cf" containerName="nova-scheduler-scheduler" Dec 03 11:45:57 crc kubenswrapper[5052]: I1203 11:45:57.321389 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b3fbbf0-f272-4d30-8fe3-26f1cf7b04cf" containerName="nova-scheduler-scheduler" Dec 03 11:45:57 crc kubenswrapper[5052]: E1203 11:45:57.321432 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="41c74e84-0fb9-4d22-9481-8e4ccda651ec" containerName="nova-metadata-metadata" Dec 03 11:45:57 crc kubenswrapper[5052]: I1203 11:45:57.321440 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="41c74e84-0fb9-4d22-9481-8e4ccda651ec" containerName="nova-metadata-metadata" Dec 03 11:45:57 crc kubenswrapper[5052]: I1203 11:45:57.321698 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="3b3fbbf0-f272-4d30-8fe3-26f1cf7b04cf" containerName="nova-scheduler-scheduler" Dec 03 11:45:57 crc kubenswrapper[5052]: I1203 11:45:57.321720 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="8e87b835-163c-4f90-a10d-c35a8e5ec9dd" containerName="nova-manage" Dec 03 11:45:57 crc kubenswrapper[5052]: I1203 11:45:57.321728 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="41c74e84-0fb9-4d22-9481-8e4ccda651ec" containerName="nova-metadata-log" Dec 03 11:45:57 crc kubenswrapper[5052]: I1203 11:45:57.321765 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="41c74e84-0fb9-4d22-9481-8e4ccda651ec" containerName="nova-metadata-metadata" Dec 03 11:45:57 crc kubenswrapper[5052]: I1203 11:45:57.323054 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:45:57 crc kubenswrapper[5052]: I1203 11:45:57.326207 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-metadata-config-data" Dec 03 11:45:57 crc kubenswrapper[5052]: I1203 11:45:57.326220 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-nova-metadata-internal-svc" Dec 03 11:45:57 crc kubenswrapper[5052]: I1203 11:45:57.335365 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 11:45:57 crc kubenswrapper[5052]: I1203 11:45:57.352342 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-244nn\" (UniqueName: \"kubernetes.io/projected/3b3fbbf0-f272-4d30-8fe3-26f1cf7b04cf-kube-api-access-244nn\") on node \"crc\" DevicePath \"\"" Dec 03 11:45:57 crc kubenswrapper[5052]: I1203 11:45:57.352380 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3b3fbbf0-f272-4d30-8fe3-26f1cf7b04cf-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:45:57 crc kubenswrapper[5052]: I1203 11:45:57.352391 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/3b3fbbf0-f272-4d30-8fe3-26f1cf7b04cf-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:45:57 crc kubenswrapper[5052]: I1203 11:45:57.454291 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bwpxl\" (UniqueName: \"kubernetes.io/projected/38a8db66-3767-4079-91a7-157bbe1a23f3-kube-api-access-bwpxl\") pod \"nova-metadata-0\" (UID: \"38a8db66-3767-4079-91a7-157bbe1a23f3\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:45:57 crc kubenswrapper[5052]: I1203 11:45:57.454648 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/38a8db66-3767-4079-91a7-157bbe1a23f3-nova-metadata-tls-certs\") pod \"nova-metadata-0\" (UID: \"38a8db66-3767-4079-91a7-157bbe1a23f3\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:45:57 crc kubenswrapper[5052]: I1203 11:45:57.454691 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/38a8db66-3767-4079-91a7-157bbe1a23f3-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"38a8db66-3767-4079-91a7-157bbe1a23f3\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:45:57 crc kubenswrapper[5052]: I1203 11:45:57.454725 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/38a8db66-3767-4079-91a7-157bbe1a23f3-config-data\") pod \"nova-metadata-0\" (UID: \"38a8db66-3767-4079-91a7-157bbe1a23f3\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:45:57 crc kubenswrapper[5052]: I1203 11:45:57.454858 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/38a8db66-3767-4079-91a7-157bbe1a23f3-logs\") pod \"nova-metadata-0\" (UID: \"38a8db66-3767-4079-91a7-157bbe1a23f3\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:45:57 crc kubenswrapper[5052]: I1203 11:45:57.524123 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 11:45:57 crc kubenswrapper[5052]: I1203 11:45:57.533894 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 11:45:57 crc kubenswrapper[5052]: I1203 11:45:57.545892 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 11:45:57 crc kubenswrapper[5052]: I1203 11:45:57.547267 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:45:57 crc kubenswrapper[5052]: I1203 11:45:57.551865 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-scheduler-config-data" Dec 03 11:45:57 crc kubenswrapper[5052]: I1203 11:45:57.554788 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 11:45:57 crc kubenswrapper[5052]: I1203 11:45:57.559538 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/38a8db66-3767-4079-91a7-157bbe1a23f3-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"38a8db66-3767-4079-91a7-157bbe1a23f3\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:45:57 crc kubenswrapper[5052]: I1203 11:45:57.559581 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/38a8db66-3767-4079-91a7-157bbe1a23f3-config-data\") pod \"nova-metadata-0\" (UID: \"38a8db66-3767-4079-91a7-157bbe1a23f3\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:45:57 crc kubenswrapper[5052]: I1203 11:45:57.559682 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/38a8db66-3767-4079-91a7-157bbe1a23f3-logs\") pod \"nova-metadata-0\" (UID: \"38a8db66-3767-4079-91a7-157bbe1a23f3\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:45:57 crc kubenswrapper[5052]: I1203 11:45:57.559709 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-bwpxl\" (UniqueName: \"kubernetes.io/projected/38a8db66-3767-4079-91a7-157bbe1a23f3-kube-api-access-bwpxl\") pod \"nova-metadata-0\" (UID: \"38a8db66-3767-4079-91a7-157bbe1a23f3\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:45:57 crc kubenswrapper[5052]: I1203 11:45:57.559756 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/38a8db66-3767-4079-91a7-157bbe1a23f3-nova-metadata-tls-certs\") pod \"nova-metadata-0\" (UID: \"38a8db66-3767-4079-91a7-157bbe1a23f3\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:45:57 crc kubenswrapper[5052]: I1203 11:45:57.560213 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/38a8db66-3767-4079-91a7-157bbe1a23f3-logs\") pod \"nova-metadata-0\" (UID: \"38a8db66-3767-4079-91a7-157bbe1a23f3\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:45:57 crc kubenswrapper[5052]: I1203 11:45:57.563908 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/38a8db66-3767-4079-91a7-157bbe1a23f3-nova-metadata-tls-certs\") pod \"nova-metadata-0\" (UID: \"38a8db66-3767-4079-91a7-157bbe1a23f3\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:45:57 crc kubenswrapper[5052]: I1203 11:45:57.565726 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/38a8db66-3767-4079-91a7-157bbe1a23f3-config-data\") pod \"nova-metadata-0\" (UID: \"38a8db66-3767-4079-91a7-157bbe1a23f3\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:45:57 crc kubenswrapper[5052]: I1203 11:45:57.573838 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/38a8db66-3767-4079-91a7-157bbe1a23f3-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"38a8db66-3767-4079-91a7-157bbe1a23f3\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:45:57 crc kubenswrapper[5052]: I1203 11:45:57.577768 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-bwpxl\" (UniqueName: \"kubernetes.io/projected/38a8db66-3767-4079-91a7-157bbe1a23f3-kube-api-access-bwpxl\") pod \"nova-metadata-0\" (UID: \"38a8db66-3767-4079-91a7-157bbe1a23f3\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:45:57 crc kubenswrapper[5052]: I1203 11:45:57.659402 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:45:57 crc kubenswrapper[5052]: I1203 11:45:57.660636 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-44zx5\" (UniqueName: \"kubernetes.io/projected/f51e7748-4baa-4b1c-a13d-84bb07444009-kube-api-access-44zx5\") pod \"nova-scheduler-0\" (UID: \"f51e7748-4baa-4b1c-a13d-84bb07444009\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:45:57 crc kubenswrapper[5052]: I1203 11:45:57.660665 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/f51e7748-4baa-4b1c-a13d-84bb07444009-config-data\") pod \"nova-scheduler-0\" (UID: \"f51e7748-4baa-4b1c-a13d-84bb07444009\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:45:57 crc kubenswrapper[5052]: I1203 11:45:57.660809 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f51e7748-4baa-4b1c-a13d-84bb07444009-combined-ca-bundle\") pod \"nova-scheduler-0\" (UID: \"f51e7748-4baa-4b1c-a13d-84bb07444009\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:45:57 crc kubenswrapper[5052]: I1203 11:45:57.683897 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3b3fbbf0-f272-4d30-8fe3-26f1cf7b04cf" path="/var/lib/kubelet/pods/3b3fbbf0-f272-4d30-8fe3-26f1cf7b04cf/volumes" Dec 03 11:45:57 crc kubenswrapper[5052]: I1203 11:45:57.684431 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="41c74e84-0fb9-4d22-9481-8e4ccda651ec" path="/var/lib/kubelet/pods/41c74e84-0fb9-4d22-9481-8e4ccda651ec/volumes" Dec 03 11:45:57 crc kubenswrapper[5052]: I1203 11:45:57.762499 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-44zx5\" (UniqueName: \"kubernetes.io/projected/f51e7748-4baa-4b1c-a13d-84bb07444009-kube-api-access-44zx5\") pod \"nova-scheduler-0\" (UID: \"f51e7748-4baa-4b1c-a13d-84bb07444009\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:45:57 crc kubenswrapper[5052]: I1203 11:45:57.762546 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/f51e7748-4baa-4b1c-a13d-84bb07444009-config-data\") pod \"nova-scheduler-0\" (UID: \"f51e7748-4baa-4b1c-a13d-84bb07444009\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:45:57 crc kubenswrapper[5052]: I1203 11:45:57.762689 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f51e7748-4baa-4b1c-a13d-84bb07444009-combined-ca-bundle\") pod \"nova-scheduler-0\" (UID: \"f51e7748-4baa-4b1c-a13d-84bb07444009\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:45:57 crc kubenswrapper[5052]: I1203 11:45:57.768555 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f51e7748-4baa-4b1c-a13d-84bb07444009-combined-ca-bundle\") pod \"nova-scheduler-0\" (UID: \"f51e7748-4baa-4b1c-a13d-84bb07444009\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:45:57 crc kubenswrapper[5052]: I1203 11:45:57.772067 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/f51e7748-4baa-4b1c-a13d-84bb07444009-config-data\") pod \"nova-scheduler-0\" (UID: \"f51e7748-4baa-4b1c-a13d-84bb07444009\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:45:57 crc kubenswrapper[5052]: I1203 11:45:57.780649 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-44zx5\" (UniqueName: \"kubernetes.io/projected/f51e7748-4baa-4b1c-a13d-84bb07444009-kube-api-access-44zx5\") pod \"nova-scheduler-0\" (UID: \"f51e7748-4baa-4b1c-a13d-84bb07444009\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:45:57 crc kubenswrapper[5052]: I1203 11:45:57.970347 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:46:00 crc kubenswrapper[5052]: I1203 11:45:58.138902 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 11:46:00 crc kubenswrapper[5052]: I1203 11:45:58.203023 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"38a8db66-3767-4079-91a7-157bbe1a23f3","Type":"ContainerStarted","Data":"60c98d3b8a5ab8167d68bd076a2415df8fb418b5d603a3a2e17dfadea67a206a"} Dec 03 11:46:00 crc kubenswrapper[5052]: I1203 11:45:58.421088 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 11:46:00 crc kubenswrapper[5052]: W1203 11:45:58.425061 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf51e7748_4baa_4b1c_a13d_84bb07444009.slice/crio-e546758641bd9bbab5a2d38d7e0a777df416fc377f63bf8112944dd884fd95d8 WatchSource:0}: Error finding container e546758641bd9bbab5a2d38d7e0a777df416fc377f63bf8112944dd884fd95d8: Status 404 returned error can't find the container with id e546758641bd9bbab5a2d38d7e0a777df416fc377f63bf8112944dd884fd95d8 Dec 03 11:46:00 crc kubenswrapper[5052]: I1203 11:45:59.231848 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"f51e7748-4baa-4b1c-a13d-84bb07444009","Type":"ContainerStarted","Data":"b7ce974e9594aa72254374252b2c28f90aea8e0ff5b969e81586db771992d287"} Dec 03 11:46:00 crc kubenswrapper[5052]: I1203 11:45:59.231893 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"f51e7748-4baa-4b1c-a13d-84bb07444009","Type":"ContainerStarted","Data":"e546758641bd9bbab5a2d38d7e0a777df416fc377f63bf8112944dd884fd95d8"} Dec 03 11:46:00 crc kubenswrapper[5052]: I1203 11:45:59.234703 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"38a8db66-3767-4079-91a7-157bbe1a23f3","Type":"ContainerStarted","Data":"04a49934bbdc1fc138326569202470b6329b0ea42ba7c6cf027ab3bad497aa8d"} Dec 03 11:46:00 crc kubenswrapper[5052]: I1203 11:45:59.263425 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-scheduler-0" podStartSLOduration=2.263400741 podStartE2EDuration="2.263400741s" podCreationTimestamp="2025-12-03 11:45:57 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:45:59.259061596 +0000 UTC m=+1549.847303370" watchObservedRunningTime="2025-12-03 11:45:59.263400741 +0000 UTC m=+1549.851642505" Dec 03 11:46:00 crc kubenswrapper[5052]: I1203 11:45:59.379762 5052 patch_prober.go:28] interesting pod/machine-config-daemon-74bj6 container/machine-config-daemon namespace/openshift-machine-config-operator: Liveness probe status=failure output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" start-of-body= Dec 03 11:46:00 crc kubenswrapper[5052]: I1203 11:45:59.380050 5052 prober.go:107] "Probe failed" probeType="Liveness" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" containerName="machine-config-daemon" probeResult="failure" output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" Dec 03 11:46:00 crc kubenswrapper[5052]: I1203 11:45:59.380086 5052 kubelet.go:2542] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" Dec 03 11:46:00 crc kubenswrapper[5052]: I1203 11:45:59.380833 5052 kuberuntime_manager.go:1027] "Message for Container of pod" containerName="machine-config-daemon" containerStatusID={"Type":"cri-o","ID":"25fe88f8d102f09983f8884e65f0c6c1575ade67921f10a630e6dcc3ea8b506e"} pod="openshift-machine-config-operator/machine-config-daemon-74bj6" containerMessage="Container machine-config-daemon failed liveness probe, will be restarted" Dec 03 11:46:00 crc kubenswrapper[5052]: I1203 11:45:59.380882 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" containerName="machine-config-daemon" containerID="cri-o://25fe88f8d102f09983f8884e65f0c6c1575ade67921f10a630e6dcc3ea8b506e" gracePeriod=600 Dec 03 11:46:00 crc kubenswrapper[5052]: E1203 11:45:59.528370 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 11:46:00 crc kubenswrapper[5052]: I1203 11:45:59.610241 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 11:46:00 crc kubenswrapper[5052]: I1203 11:45:59.633935 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 11:46:00 crc kubenswrapper[5052]: I1203 11:46:00.257166 5052 generic.go:334] "Generic (PLEG): container finished" podID="37015ee5-cb17-446c-8ebe-8481eb0b46fd" containerID="ec1beecdc1a4aa062f33038ef7661bb5fc9f12bc257825975c268a0f9659e3a7" exitCode=0 Dec 03 11:46:00 crc kubenswrapper[5052]: I1203 11:46:00.257280 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"37015ee5-cb17-446c-8ebe-8481eb0b46fd","Type":"ContainerDied","Data":"ec1beecdc1a4aa062f33038ef7661bb5fc9f12bc257825975c268a0f9659e3a7"} Dec 03 11:46:00 crc kubenswrapper[5052]: I1203 11:46:00.264212 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" event={"ID":"10c237bc-72e7-4da3-9f2c-b1688395bf6e","Type":"ContainerDied","Data":"25fe88f8d102f09983f8884e65f0c6c1575ade67921f10a630e6dcc3ea8b506e"} Dec 03 11:46:00 crc kubenswrapper[5052]: I1203 11:46:00.264202 5052 generic.go:334] "Generic (PLEG): container finished" podID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" containerID="25fe88f8d102f09983f8884e65f0c6c1575ade67921f10a630e6dcc3ea8b506e" exitCode=0 Dec 03 11:46:00 crc kubenswrapper[5052]: I1203 11:46:00.264270 5052 scope.go:117] "RemoveContainer" containerID="9a1bbf3b5805844b6adac48684e34e2c0f409388266eaea05dc4a45f4951ccdf" Dec 03 11:46:00 crc kubenswrapper[5052]: I1203 11:46:00.265064 5052 scope.go:117] "RemoveContainer" containerID="25fe88f8d102f09983f8884e65f0c6c1575ade67921f10a630e6dcc3ea8b506e" Dec 03 11:46:00 crc kubenswrapper[5052]: E1203 11:46:00.265458 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 11:46:00 crc kubenswrapper[5052]: I1203 11:46:00.275739 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"38a8db66-3767-4079-91a7-157bbe1a23f3","Type":"ContainerStarted","Data":"e81f2bd5b57c0299aa652c12fddb54ea42890e277feba0dbd59df5827750445f"} Dec 03 11:46:00 crc kubenswrapper[5052]: I1203 11:46:00.302630 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 11:46:00 crc kubenswrapper[5052]: I1203 11:46:00.313993 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-metadata-0" podStartSLOduration=3.313968816 podStartE2EDuration="3.313968816s" podCreationTimestamp="2025-12-03 11:45:57 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:46:00.306895082 +0000 UTC m=+1550.895136846" watchObservedRunningTime="2025-12-03 11:46:00.313968816 +0000 UTC m=+1550.902210600" Dec 03 11:46:00 crc kubenswrapper[5052]: I1203 11:46:00.474115 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:46:00 crc kubenswrapper[5052]: I1203 11:46:00.618007 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/37015ee5-cb17-446c-8ebe-8481eb0b46fd-logs\") pod \"37015ee5-cb17-446c-8ebe-8481eb0b46fd\" (UID: \"37015ee5-cb17-446c-8ebe-8481eb0b46fd\") " Dec 03 11:46:00 crc kubenswrapper[5052]: I1203 11:46:00.618199 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/37015ee5-cb17-446c-8ebe-8481eb0b46fd-combined-ca-bundle\") pod \"37015ee5-cb17-446c-8ebe-8481eb0b46fd\" (UID: \"37015ee5-cb17-446c-8ebe-8481eb0b46fd\") " Dec 03 11:46:00 crc kubenswrapper[5052]: I1203 11:46:00.618255 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/37015ee5-cb17-446c-8ebe-8481eb0b46fd-config-data\") pod \"37015ee5-cb17-446c-8ebe-8481eb0b46fd\" (UID: \"37015ee5-cb17-446c-8ebe-8481eb0b46fd\") " Dec 03 11:46:00 crc kubenswrapper[5052]: I1203 11:46:00.618343 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tlb9r\" (UniqueName: \"kubernetes.io/projected/37015ee5-cb17-446c-8ebe-8481eb0b46fd-kube-api-access-tlb9r\") pod \"37015ee5-cb17-446c-8ebe-8481eb0b46fd\" (UID: \"37015ee5-cb17-446c-8ebe-8481eb0b46fd\") " Dec 03 11:46:00 crc kubenswrapper[5052]: I1203 11:46:00.618452 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37015ee5-cb17-446c-8ebe-8481eb0b46fd-logs" (OuterVolumeSpecName: "logs") pod "37015ee5-cb17-446c-8ebe-8481eb0b46fd" (UID: "37015ee5-cb17-446c-8ebe-8481eb0b46fd"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:46:00 crc kubenswrapper[5052]: I1203 11:46:00.618865 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/37015ee5-cb17-446c-8ebe-8481eb0b46fd-logs\") on node \"crc\" DevicePath \"\"" Dec 03 11:46:00 crc kubenswrapper[5052]: I1203 11:46:00.627874 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/37015ee5-cb17-446c-8ebe-8481eb0b46fd-kube-api-access-tlb9r" (OuterVolumeSpecName: "kube-api-access-tlb9r") pod "37015ee5-cb17-446c-8ebe-8481eb0b46fd" (UID: "37015ee5-cb17-446c-8ebe-8481eb0b46fd"). InnerVolumeSpecName "kube-api-access-tlb9r". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:46:00 crc kubenswrapper[5052]: I1203 11:46:00.650463 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/37015ee5-cb17-446c-8ebe-8481eb0b46fd-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "37015ee5-cb17-446c-8ebe-8481eb0b46fd" (UID: "37015ee5-cb17-446c-8ebe-8481eb0b46fd"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:46:00 crc kubenswrapper[5052]: I1203 11:46:00.654251 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/37015ee5-cb17-446c-8ebe-8481eb0b46fd-config-data" (OuterVolumeSpecName: "config-data") pod "37015ee5-cb17-446c-8ebe-8481eb0b46fd" (UID: "37015ee5-cb17-446c-8ebe-8481eb0b46fd"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:46:00 crc kubenswrapper[5052]: I1203 11:46:00.721101 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/37015ee5-cb17-446c-8ebe-8481eb0b46fd-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:46:00 crc kubenswrapper[5052]: I1203 11:46:00.721144 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/37015ee5-cb17-446c-8ebe-8481eb0b46fd-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:46:00 crc kubenswrapper[5052]: I1203 11:46:00.721160 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-tlb9r\" (UniqueName: \"kubernetes.io/projected/37015ee5-cb17-446c-8ebe-8481eb0b46fd-kube-api-access-tlb9r\") on node \"crc\" DevicePath \"\"" Dec 03 11:46:01 crc kubenswrapper[5052]: I1203 11:46:01.289930 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"37015ee5-cb17-446c-8ebe-8481eb0b46fd","Type":"ContainerDied","Data":"27adc888ca29306887b48ac7bc5e6d9f3d9a010c24b3294f1a9b513a5334a5e9"} Dec 03 11:46:01 crc kubenswrapper[5052]: I1203 11:46:01.289996 5052 scope.go:117] "RemoveContainer" containerID="ec1beecdc1a4aa062f33038ef7661bb5fc9f12bc257825975c268a0f9659e3a7" Dec 03 11:46:01 crc kubenswrapper[5052]: I1203 11:46:01.291154 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:46:01 crc kubenswrapper[5052]: I1203 11:46:01.319653 5052 scope.go:117] "RemoveContainer" containerID="5bd027359df8453d45c83f04becf67005f205297847458e29f4ddfc2a2570f4d" Dec 03 11:46:01 crc kubenswrapper[5052]: I1203 11:46:01.330298 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 11:46:01 crc kubenswrapper[5052]: I1203 11:46:01.345801 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 11:46:01 crc kubenswrapper[5052]: I1203 11:46:01.362729 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 11:46:01 crc kubenswrapper[5052]: E1203 11:46:01.363136 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="37015ee5-cb17-446c-8ebe-8481eb0b46fd" containerName="nova-api-api" Dec 03 11:46:01 crc kubenswrapper[5052]: I1203 11:46:01.363154 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="37015ee5-cb17-446c-8ebe-8481eb0b46fd" containerName="nova-api-api" Dec 03 11:46:01 crc kubenswrapper[5052]: E1203 11:46:01.363164 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="37015ee5-cb17-446c-8ebe-8481eb0b46fd" containerName="nova-api-log" Dec 03 11:46:01 crc kubenswrapper[5052]: I1203 11:46:01.363170 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="37015ee5-cb17-446c-8ebe-8481eb0b46fd" containerName="nova-api-log" Dec 03 11:46:01 crc kubenswrapper[5052]: I1203 11:46:01.363360 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="37015ee5-cb17-446c-8ebe-8481eb0b46fd" containerName="nova-api-log" Dec 03 11:46:01 crc kubenswrapper[5052]: I1203 11:46:01.363384 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="37015ee5-cb17-446c-8ebe-8481eb0b46fd" containerName="nova-api-api" Dec 03 11:46:01 crc kubenswrapper[5052]: I1203 11:46:01.364539 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:46:01 crc kubenswrapper[5052]: I1203 11:46:01.366641 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-api-config-data" Dec 03 11:46:01 crc kubenswrapper[5052]: I1203 11:46:01.375745 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 11:46:01 crc kubenswrapper[5052]: I1203 11:46:01.441830 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z2tgl\" (UniqueName: \"kubernetes.io/projected/da3b67be-c113-4fbd-b529-8aa2fcee2832-kube-api-access-z2tgl\") pod \"nova-api-0\" (UID: \"da3b67be-c113-4fbd-b529-8aa2fcee2832\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:46:01 crc kubenswrapper[5052]: I1203 11:46:01.441921 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/da3b67be-c113-4fbd-b529-8aa2fcee2832-logs\") pod \"nova-api-0\" (UID: \"da3b67be-c113-4fbd-b529-8aa2fcee2832\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:46:01 crc kubenswrapper[5052]: I1203 11:46:01.441945 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/da3b67be-c113-4fbd-b529-8aa2fcee2832-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"da3b67be-c113-4fbd-b529-8aa2fcee2832\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:46:01 crc kubenswrapper[5052]: I1203 11:46:01.441966 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/da3b67be-c113-4fbd-b529-8aa2fcee2832-config-data\") pod \"nova-api-0\" (UID: \"da3b67be-c113-4fbd-b529-8aa2fcee2832\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:46:01 crc kubenswrapper[5052]: I1203 11:46:01.541089 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 11:46:01 crc kubenswrapper[5052]: I1203 11:46:01.542977 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-z2tgl\" (UniqueName: \"kubernetes.io/projected/da3b67be-c113-4fbd-b529-8aa2fcee2832-kube-api-access-z2tgl\") pod \"nova-api-0\" (UID: \"da3b67be-c113-4fbd-b529-8aa2fcee2832\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:46:01 crc kubenswrapper[5052]: I1203 11:46:01.543045 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/da3b67be-c113-4fbd-b529-8aa2fcee2832-logs\") pod \"nova-api-0\" (UID: \"da3b67be-c113-4fbd-b529-8aa2fcee2832\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:46:01 crc kubenswrapper[5052]: I1203 11:46:01.543067 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/da3b67be-c113-4fbd-b529-8aa2fcee2832-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"da3b67be-c113-4fbd-b529-8aa2fcee2832\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:46:01 crc kubenswrapper[5052]: I1203 11:46:01.543086 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/da3b67be-c113-4fbd-b529-8aa2fcee2832-config-data\") pod \"nova-api-0\" (UID: \"da3b67be-c113-4fbd-b529-8aa2fcee2832\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:46:01 crc kubenswrapper[5052]: I1203 11:46:01.543647 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/da3b67be-c113-4fbd-b529-8aa2fcee2832-logs\") pod \"nova-api-0\" (UID: \"da3b67be-c113-4fbd-b529-8aa2fcee2832\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:46:01 crc kubenswrapper[5052]: I1203 11:46:01.552456 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/da3b67be-c113-4fbd-b529-8aa2fcee2832-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"da3b67be-c113-4fbd-b529-8aa2fcee2832\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:46:01 crc kubenswrapper[5052]: I1203 11:46:01.560802 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/da3b67be-c113-4fbd-b529-8aa2fcee2832-config-data\") pod \"nova-api-0\" (UID: \"da3b67be-c113-4fbd-b529-8aa2fcee2832\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:46:01 crc kubenswrapper[5052]: I1203 11:46:01.564474 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-z2tgl\" (UniqueName: \"kubernetes.io/projected/da3b67be-c113-4fbd-b529-8aa2fcee2832-kube-api-access-z2tgl\") pod \"nova-api-0\" (UID: \"da3b67be-c113-4fbd-b529-8aa2fcee2832\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:46:01 crc kubenswrapper[5052]: I1203 11:46:01.687343 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="37015ee5-cb17-446c-8ebe-8481eb0b46fd" path="/var/lib/kubelet/pods/37015ee5-cb17-446c-8ebe-8481eb0b46fd/volumes" Dec 03 11:46:01 crc kubenswrapper[5052]: I1203 11:46:01.725205 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:46:02 crc kubenswrapper[5052]: I1203 11:46:02.090401 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell1-cell-mapping-jbzxc"] Dec 03 11:46:02 crc kubenswrapper[5052]: I1203 11:46:02.092312 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-cell-mapping-jbzxc" Dec 03 11:46:02 crc kubenswrapper[5052]: I1203 11:46:02.095741 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell1-manage-config-data" Dec 03 11:46:02 crc kubenswrapper[5052]: I1203 11:46:02.095852 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell1-manage-scripts" Dec 03 11:46:02 crc kubenswrapper[5052]: I1203 11:46:02.127582 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-cell-mapping-jbzxc"] Dec 03 11:46:02 crc kubenswrapper[5052]: I1203 11:46:02.260921 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kqfvs\" (UniqueName: \"kubernetes.io/projected/059efa84-74c6-4d18-a279-5b7de59bb1b3-kube-api-access-kqfvs\") pod \"nova-cell1-cell-mapping-jbzxc\" (UID: \"059efa84-74c6-4d18-a279-5b7de59bb1b3\") " pod="openstack-kuttl-tests/nova-cell1-cell-mapping-jbzxc" Dec 03 11:46:02 crc kubenswrapper[5052]: I1203 11:46:02.261283 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/059efa84-74c6-4d18-a279-5b7de59bb1b3-scripts\") pod \"nova-cell1-cell-mapping-jbzxc\" (UID: \"059efa84-74c6-4d18-a279-5b7de59bb1b3\") " pod="openstack-kuttl-tests/nova-cell1-cell-mapping-jbzxc" Dec 03 11:46:02 crc kubenswrapper[5052]: I1203 11:46:02.261316 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/059efa84-74c6-4d18-a279-5b7de59bb1b3-config-data\") pod \"nova-cell1-cell-mapping-jbzxc\" (UID: \"059efa84-74c6-4d18-a279-5b7de59bb1b3\") " pod="openstack-kuttl-tests/nova-cell1-cell-mapping-jbzxc" Dec 03 11:46:02 crc kubenswrapper[5052]: I1203 11:46:02.261407 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/059efa84-74c6-4d18-a279-5b7de59bb1b3-combined-ca-bundle\") pod \"nova-cell1-cell-mapping-jbzxc\" (UID: \"059efa84-74c6-4d18-a279-5b7de59bb1b3\") " pod="openstack-kuttl-tests/nova-cell1-cell-mapping-jbzxc" Dec 03 11:46:02 crc kubenswrapper[5052]: I1203 11:46:02.272035 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 11:46:02 crc kubenswrapper[5052]: W1203 11:46:02.281822 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podda3b67be_c113_4fbd_b529_8aa2fcee2832.slice/crio-25d198d84b307c8e22e75f767240181dfee60030caef8bedee458cb5075342bf WatchSource:0}: Error finding container 25d198d84b307c8e22e75f767240181dfee60030caef8bedee458cb5075342bf: Status 404 returned error can't find the container with id 25d198d84b307c8e22e75f767240181dfee60030caef8bedee458cb5075342bf Dec 03 11:46:02 crc kubenswrapper[5052]: I1203 11:46:02.311089 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"da3b67be-c113-4fbd-b529-8aa2fcee2832","Type":"ContainerStarted","Data":"25d198d84b307c8e22e75f767240181dfee60030caef8bedee458cb5075342bf"} Dec 03 11:46:02 crc kubenswrapper[5052]: I1203 11:46:02.362799 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/059efa84-74c6-4d18-a279-5b7de59bb1b3-combined-ca-bundle\") pod \"nova-cell1-cell-mapping-jbzxc\" (UID: \"059efa84-74c6-4d18-a279-5b7de59bb1b3\") " pod="openstack-kuttl-tests/nova-cell1-cell-mapping-jbzxc" Dec 03 11:46:02 crc kubenswrapper[5052]: I1203 11:46:02.362938 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-kqfvs\" (UniqueName: \"kubernetes.io/projected/059efa84-74c6-4d18-a279-5b7de59bb1b3-kube-api-access-kqfvs\") pod \"nova-cell1-cell-mapping-jbzxc\" (UID: \"059efa84-74c6-4d18-a279-5b7de59bb1b3\") " pod="openstack-kuttl-tests/nova-cell1-cell-mapping-jbzxc" Dec 03 11:46:02 crc kubenswrapper[5052]: I1203 11:46:02.362968 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/059efa84-74c6-4d18-a279-5b7de59bb1b3-scripts\") pod \"nova-cell1-cell-mapping-jbzxc\" (UID: \"059efa84-74c6-4d18-a279-5b7de59bb1b3\") " pod="openstack-kuttl-tests/nova-cell1-cell-mapping-jbzxc" Dec 03 11:46:02 crc kubenswrapper[5052]: I1203 11:46:02.362986 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/059efa84-74c6-4d18-a279-5b7de59bb1b3-config-data\") pod \"nova-cell1-cell-mapping-jbzxc\" (UID: \"059efa84-74c6-4d18-a279-5b7de59bb1b3\") " pod="openstack-kuttl-tests/nova-cell1-cell-mapping-jbzxc" Dec 03 11:46:02 crc kubenswrapper[5052]: I1203 11:46:02.370076 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/059efa84-74c6-4d18-a279-5b7de59bb1b3-scripts\") pod \"nova-cell1-cell-mapping-jbzxc\" (UID: \"059efa84-74c6-4d18-a279-5b7de59bb1b3\") " pod="openstack-kuttl-tests/nova-cell1-cell-mapping-jbzxc" Dec 03 11:46:02 crc kubenswrapper[5052]: I1203 11:46:02.370162 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/059efa84-74c6-4d18-a279-5b7de59bb1b3-config-data\") pod \"nova-cell1-cell-mapping-jbzxc\" (UID: \"059efa84-74c6-4d18-a279-5b7de59bb1b3\") " pod="openstack-kuttl-tests/nova-cell1-cell-mapping-jbzxc" Dec 03 11:46:02 crc kubenswrapper[5052]: I1203 11:46:02.371225 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/059efa84-74c6-4d18-a279-5b7de59bb1b3-combined-ca-bundle\") pod \"nova-cell1-cell-mapping-jbzxc\" (UID: \"059efa84-74c6-4d18-a279-5b7de59bb1b3\") " pod="openstack-kuttl-tests/nova-cell1-cell-mapping-jbzxc" Dec 03 11:46:02 crc kubenswrapper[5052]: I1203 11:46:02.381578 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-kqfvs\" (UniqueName: \"kubernetes.io/projected/059efa84-74c6-4d18-a279-5b7de59bb1b3-kube-api-access-kqfvs\") pod \"nova-cell1-cell-mapping-jbzxc\" (UID: \"059efa84-74c6-4d18-a279-5b7de59bb1b3\") " pod="openstack-kuttl-tests/nova-cell1-cell-mapping-jbzxc" Dec 03 11:46:02 crc kubenswrapper[5052]: I1203 11:46:02.447564 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-cell-mapping-jbzxc" Dec 03 11:46:02 crc kubenswrapper[5052]: I1203 11:46:02.660229 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:46:02 crc kubenswrapper[5052]: I1203 11:46:02.660582 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:46:02 crc kubenswrapper[5052]: I1203 11:46:02.971254 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:46:02 crc kubenswrapper[5052]: I1203 11:46:02.990851 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-cell-mapping-jbzxc"] Dec 03 11:46:03 crc kubenswrapper[5052]: I1203 11:46:03.326523 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-cell-mapping-jbzxc" event={"ID":"059efa84-74c6-4d18-a279-5b7de59bb1b3","Type":"ContainerStarted","Data":"07d16e4be45e57ca8673d74569cef708a62ef881467ba482995fd58391fb42d1"} Dec 03 11:46:03 crc kubenswrapper[5052]: I1203 11:46:03.329764 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"da3b67be-c113-4fbd-b529-8aa2fcee2832","Type":"ContainerStarted","Data":"b41b9d3d1c01190eb88dae38327e17fe28cbc505eecf87648682a5568b4c48f1"} Dec 03 11:46:03 crc kubenswrapper[5052]: I1203 11:46:03.382893 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:46:04 crc kubenswrapper[5052]: I1203 11:46:04.343573 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"da3b67be-c113-4fbd-b529-8aa2fcee2832","Type":"ContainerStarted","Data":"8a086eb63697de2b6db28ff2a4b879d37f47b25d47c3d61efbccd9f2bbb5d9d3"} Dec 03 11:46:04 crc kubenswrapper[5052]: I1203 11:46:04.347392 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-cell-mapping-jbzxc" event={"ID":"059efa84-74c6-4d18-a279-5b7de59bb1b3","Type":"ContainerStarted","Data":"f671b26620e48a89be3dad617c0f1f2474db5f99b8b9f90227b82a243b116141"} Dec 03 11:46:04 crc kubenswrapper[5052]: I1203 11:46:04.385869 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-cell1-cell-mapping-jbzxc" podStartSLOduration=2.385851616 podStartE2EDuration="2.385851616s" podCreationTimestamp="2025-12-03 11:46:02 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:46:04.383479568 +0000 UTC m=+1554.971721332" watchObservedRunningTime="2025-12-03 11:46:04.385851616 +0000 UTC m=+1554.974093380" Dec 03 11:46:04 crc kubenswrapper[5052]: I1203 11:46:04.391367 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-api-0" podStartSLOduration=3.391352875 podStartE2EDuration="3.391352875s" podCreationTimestamp="2025-12-03 11:46:01 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:46:04.366705434 +0000 UTC m=+1554.954947208" watchObservedRunningTime="2025-12-03 11:46:04.391352875 +0000 UTC m=+1554.979594639" Dec 03 11:46:07 crc kubenswrapper[5052]: I1203 11:46:07.660015 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:46:07 crc kubenswrapper[5052]: I1203 11:46:07.660656 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:46:07 crc kubenswrapper[5052]: I1203 11:46:07.971394 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:46:08 crc kubenswrapper[5052]: I1203 11:46:08.003279 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:46:08 crc kubenswrapper[5052]: I1203 11:46:08.442942 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:46:08 crc kubenswrapper[5052]: I1203 11:46:08.675731 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-metadata-0" podUID="38a8db66-3767-4079-91a7-157bbe1a23f3" containerName="nova-metadata-metadata" probeResult="failure" output="Get \"https://10.217.1.27:8775/\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Dec 03 11:46:08 crc kubenswrapper[5052]: I1203 11:46:08.675747 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-metadata-0" podUID="38a8db66-3767-4079-91a7-157bbe1a23f3" containerName="nova-metadata-log" probeResult="failure" output="Get \"https://10.217.1.27:8775/\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Dec 03 11:46:09 crc kubenswrapper[5052]: I1203 11:46:09.397946 5052 generic.go:334] "Generic (PLEG): container finished" podID="059efa84-74c6-4d18-a279-5b7de59bb1b3" containerID="f671b26620e48a89be3dad617c0f1f2474db5f99b8b9f90227b82a243b116141" exitCode=0 Dec 03 11:46:09 crc kubenswrapper[5052]: I1203 11:46:09.398050 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-cell-mapping-jbzxc" event={"ID":"059efa84-74c6-4d18-a279-5b7de59bb1b3","Type":"ContainerDied","Data":"f671b26620e48a89be3dad617c0f1f2474db5f99b8b9f90227b82a243b116141"} Dec 03 11:46:10 crc kubenswrapper[5052]: I1203 11:46:10.814486 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-cell-mapping-jbzxc" Dec 03 11:46:10 crc kubenswrapper[5052]: I1203 11:46:10.975416 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/059efa84-74c6-4d18-a279-5b7de59bb1b3-config-data\") pod \"059efa84-74c6-4d18-a279-5b7de59bb1b3\" (UID: \"059efa84-74c6-4d18-a279-5b7de59bb1b3\") " Dec 03 11:46:10 crc kubenswrapper[5052]: I1203 11:46:10.975485 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/059efa84-74c6-4d18-a279-5b7de59bb1b3-combined-ca-bundle\") pod \"059efa84-74c6-4d18-a279-5b7de59bb1b3\" (UID: \"059efa84-74c6-4d18-a279-5b7de59bb1b3\") " Dec 03 11:46:10 crc kubenswrapper[5052]: I1203 11:46:10.975540 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/059efa84-74c6-4d18-a279-5b7de59bb1b3-scripts\") pod \"059efa84-74c6-4d18-a279-5b7de59bb1b3\" (UID: \"059efa84-74c6-4d18-a279-5b7de59bb1b3\") " Dec 03 11:46:10 crc kubenswrapper[5052]: I1203 11:46:10.975633 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kqfvs\" (UniqueName: \"kubernetes.io/projected/059efa84-74c6-4d18-a279-5b7de59bb1b3-kube-api-access-kqfvs\") pod \"059efa84-74c6-4d18-a279-5b7de59bb1b3\" (UID: \"059efa84-74c6-4d18-a279-5b7de59bb1b3\") " Dec 03 11:46:10 crc kubenswrapper[5052]: I1203 11:46:10.981261 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/059efa84-74c6-4d18-a279-5b7de59bb1b3-kube-api-access-kqfvs" (OuterVolumeSpecName: "kube-api-access-kqfvs") pod "059efa84-74c6-4d18-a279-5b7de59bb1b3" (UID: "059efa84-74c6-4d18-a279-5b7de59bb1b3"). InnerVolumeSpecName "kube-api-access-kqfvs". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:46:10 crc kubenswrapper[5052]: I1203 11:46:10.981702 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/059efa84-74c6-4d18-a279-5b7de59bb1b3-scripts" (OuterVolumeSpecName: "scripts") pod "059efa84-74c6-4d18-a279-5b7de59bb1b3" (UID: "059efa84-74c6-4d18-a279-5b7de59bb1b3"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:46:11 crc kubenswrapper[5052]: I1203 11:46:11.006407 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/059efa84-74c6-4d18-a279-5b7de59bb1b3-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "059efa84-74c6-4d18-a279-5b7de59bb1b3" (UID: "059efa84-74c6-4d18-a279-5b7de59bb1b3"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:46:11 crc kubenswrapper[5052]: I1203 11:46:11.007256 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/059efa84-74c6-4d18-a279-5b7de59bb1b3-config-data" (OuterVolumeSpecName: "config-data") pod "059efa84-74c6-4d18-a279-5b7de59bb1b3" (UID: "059efa84-74c6-4d18-a279-5b7de59bb1b3"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:46:11 crc kubenswrapper[5052]: I1203 11:46:11.078189 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-kqfvs\" (UniqueName: \"kubernetes.io/projected/059efa84-74c6-4d18-a279-5b7de59bb1b3-kube-api-access-kqfvs\") on node \"crc\" DevicePath \"\"" Dec 03 11:46:11 crc kubenswrapper[5052]: I1203 11:46:11.078228 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/059efa84-74c6-4d18-a279-5b7de59bb1b3-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:46:11 crc kubenswrapper[5052]: I1203 11:46:11.078237 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/059efa84-74c6-4d18-a279-5b7de59bb1b3-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:46:11 crc kubenswrapper[5052]: I1203 11:46:11.078246 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/059efa84-74c6-4d18-a279-5b7de59bb1b3-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:46:11 crc kubenswrapper[5052]: I1203 11:46:11.420812 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-cell-mapping-jbzxc" event={"ID":"059efa84-74c6-4d18-a279-5b7de59bb1b3","Type":"ContainerDied","Data":"07d16e4be45e57ca8673d74569cef708a62ef881467ba482995fd58391fb42d1"} Dec 03 11:46:11 crc kubenswrapper[5052]: I1203 11:46:11.420856 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="07d16e4be45e57ca8673d74569cef708a62ef881467ba482995fd58391fb42d1" Dec 03 11:46:11 crc kubenswrapper[5052]: I1203 11:46:11.420949 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-cell-mapping-jbzxc" Dec 03 11:46:11 crc kubenswrapper[5052]: I1203 11:46:11.603515 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 11:46:11 crc kubenswrapper[5052]: I1203 11:46:11.603871 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-api-0" podUID="da3b67be-c113-4fbd-b529-8aa2fcee2832" containerName="nova-api-log" containerID="cri-o://b41b9d3d1c01190eb88dae38327e17fe28cbc505eecf87648682a5568b4c48f1" gracePeriod=30 Dec 03 11:46:11 crc kubenswrapper[5052]: I1203 11:46:11.604003 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-api-0" podUID="da3b67be-c113-4fbd-b529-8aa2fcee2832" containerName="nova-api-api" containerID="cri-o://8a086eb63697de2b6db28ff2a4b879d37f47b25d47c3d61efbccd9f2bbb5d9d3" gracePeriod=30 Dec 03 11:46:11 crc kubenswrapper[5052]: I1203 11:46:11.622623 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 11:46:11 crc kubenswrapper[5052]: I1203 11:46:11.622843 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-scheduler-0" podUID="f51e7748-4baa-4b1c-a13d-84bb07444009" containerName="nova-scheduler-scheduler" containerID="cri-o://b7ce974e9594aa72254374252b2c28f90aea8e0ff5b969e81586db771992d287" gracePeriod=30 Dec 03 11:46:11 crc kubenswrapper[5052]: I1203 11:46:11.662944 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 11:46:11 crc kubenswrapper[5052]: I1203 11:46:11.666154 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-metadata-0" podUID="38a8db66-3767-4079-91a7-157bbe1a23f3" containerName="nova-metadata-log" containerID="cri-o://04a49934bbdc1fc138326569202470b6329b0ea42ba7c6cf027ab3bad497aa8d" gracePeriod=30 Dec 03 11:46:11 crc kubenswrapper[5052]: I1203 11:46:11.666255 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-metadata-0" podUID="38a8db66-3767-4079-91a7-157bbe1a23f3" containerName="nova-metadata-metadata" containerID="cri-o://e81f2bd5b57c0299aa652c12fddb54ea42890e277feba0dbd59df5827750445f" gracePeriod=30 Dec 03 11:46:12 crc kubenswrapper[5052]: I1203 11:46:12.430239 5052 generic.go:334] "Generic (PLEG): container finished" podID="38a8db66-3767-4079-91a7-157bbe1a23f3" containerID="04a49934bbdc1fc138326569202470b6329b0ea42ba7c6cf027ab3bad497aa8d" exitCode=143 Dec 03 11:46:12 crc kubenswrapper[5052]: I1203 11:46:12.430303 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"38a8db66-3767-4079-91a7-157bbe1a23f3","Type":"ContainerDied","Data":"04a49934bbdc1fc138326569202470b6329b0ea42ba7c6cf027ab3bad497aa8d"} Dec 03 11:46:12 crc kubenswrapper[5052]: I1203 11:46:12.432571 5052 generic.go:334] "Generic (PLEG): container finished" podID="da3b67be-c113-4fbd-b529-8aa2fcee2832" containerID="8a086eb63697de2b6db28ff2a4b879d37f47b25d47c3d61efbccd9f2bbb5d9d3" exitCode=0 Dec 03 11:46:12 crc kubenswrapper[5052]: I1203 11:46:12.432604 5052 generic.go:334] "Generic (PLEG): container finished" podID="da3b67be-c113-4fbd-b529-8aa2fcee2832" containerID="b41b9d3d1c01190eb88dae38327e17fe28cbc505eecf87648682a5568b4c48f1" exitCode=143 Dec 03 11:46:12 crc kubenswrapper[5052]: I1203 11:46:12.432627 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"da3b67be-c113-4fbd-b529-8aa2fcee2832","Type":"ContainerDied","Data":"8a086eb63697de2b6db28ff2a4b879d37f47b25d47c3d61efbccd9f2bbb5d9d3"} Dec 03 11:46:12 crc kubenswrapper[5052]: I1203 11:46:12.432659 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"da3b67be-c113-4fbd-b529-8aa2fcee2832","Type":"ContainerDied","Data":"b41b9d3d1c01190eb88dae38327e17fe28cbc505eecf87648682a5568b4c48f1"} Dec 03 11:46:12 crc kubenswrapper[5052]: I1203 11:46:12.965636 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:46:12 crc kubenswrapper[5052]: E1203 11:46:12.974290 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="b7ce974e9594aa72254374252b2c28f90aea8e0ff5b969e81586db771992d287" cmd=["/usr/bin/pgrep","-r","DRST","nova-scheduler"] Dec 03 11:46:12 crc kubenswrapper[5052]: E1203 11:46:12.975680 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="b7ce974e9594aa72254374252b2c28f90aea8e0ff5b969e81586db771992d287" cmd=["/usr/bin/pgrep","-r","DRST","nova-scheduler"] Dec 03 11:46:12 crc kubenswrapper[5052]: E1203 11:46:12.976911 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="b7ce974e9594aa72254374252b2c28f90aea8e0ff5b969e81586db771992d287" cmd=["/usr/bin/pgrep","-r","DRST","nova-scheduler"] Dec 03 11:46:12 crc kubenswrapper[5052]: E1203 11:46:12.976970 5052 prober.go:104] "Probe errored" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" probeType="Readiness" pod="openstack-kuttl-tests/nova-scheduler-0" podUID="f51e7748-4baa-4b1c-a13d-84bb07444009" containerName="nova-scheduler-scheduler" Dec 03 11:46:13 crc kubenswrapper[5052]: I1203 11:46:13.118053 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z2tgl\" (UniqueName: \"kubernetes.io/projected/da3b67be-c113-4fbd-b529-8aa2fcee2832-kube-api-access-z2tgl\") pod \"da3b67be-c113-4fbd-b529-8aa2fcee2832\" (UID: \"da3b67be-c113-4fbd-b529-8aa2fcee2832\") " Dec 03 11:46:13 crc kubenswrapper[5052]: I1203 11:46:13.118172 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/da3b67be-c113-4fbd-b529-8aa2fcee2832-config-data\") pod \"da3b67be-c113-4fbd-b529-8aa2fcee2832\" (UID: \"da3b67be-c113-4fbd-b529-8aa2fcee2832\") " Dec 03 11:46:13 crc kubenswrapper[5052]: I1203 11:46:13.118231 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/da3b67be-c113-4fbd-b529-8aa2fcee2832-combined-ca-bundle\") pod \"da3b67be-c113-4fbd-b529-8aa2fcee2832\" (UID: \"da3b67be-c113-4fbd-b529-8aa2fcee2832\") " Dec 03 11:46:13 crc kubenswrapper[5052]: I1203 11:46:13.118255 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/da3b67be-c113-4fbd-b529-8aa2fcee2832-logs\") pod \"da3b67be-c113-4fbd-b529-8aa2fcee2832\" (UID: \"da3b67be-c113-4fbd-b529-8aa2fcee2832\") " Dec 03 11:46:13 crc kubenswrapper[5052]: I1203 11:46:13.118734 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/da3b67be-c113-4fbd-b529-8aa2fcee2832-logs" (OuterVolumeSpecName: "logs") pod "da3b67be-c113-4fbd-b529-8aa2fcee2832" (UID: "da3b67be-c113-4fbd-b529-8aa2fcee2832"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:46:13 crc kubenswrapper[5052]: I1203 11:46:13.125163 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/da3b67be-c113-4fbd-b529-8aa2fcee2832-kube-api-access-z2tgl" (OuterVolumeSpecName: "kube-api-access-z2tgl") pod "da3b67be-c113-4fbd-b529-8aa2fcee2832" (UID: "da3b67be-c113-4fbd-b529-8aa2fcee2832"). InnerVolumeSpecName "kube-api-access-z2tgl". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:46:13 crc kubenswrapper[5052]: E1203 11:46:13.144096 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/da3b67be-c113-4fbd-b529-8aa2fcee2832-combined-ca-bundle podName:da3b67be-c113-4fbd-b529-8aa2fcee2832 nodeName:}" failed. No retries permitted until 2025-12-03 11:46:13.644066586 +0000 UTC m=+1564.232308350 (durationBeforeRetry 500ms). Error: error cleaning subPath mounts for volume "combined-ca-bundle" (UniqueName: "kubernetes.io/secret/da3b67be-c113-4fbd-b529-8aa2fcee2832-combined-ca-bundle") pod "da3b67be-c113-4fbd-b529-8aa2fcee2832" (UID: "da3b67be-c113-4fbd-b529-8aa2fcee2832") : error deleting /var/lib/kubelet/pods/da3b67be-c113-4fbd-b529-8aa2fcee2832/volume-subpaths: remove /var/lib/kubelet/pods/da3b67be-c113-4fbd-b529-8aa2fcee2832/volume-subpaths: no such file or directory Dec 03 11:46:13 crc kubenswrapper[5052]: I1203 11:46:13.147489 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/da3b67be-c113-4fbd-b529-8aa2fcee2832-config-data" (OuterVolumeSpecName: "config-data") pod "da3b67be-c113-4fbd-b529-8aa2fcee2832" (UID: "da3b67be-c113-4fbd-b529-8aa2fcee2832"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:46:13 crc kubenswrapper[5052]: I1203 11:46:13.220572 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-z2tgl\" (UniqueName: \"kubernetes.io/projected/da3b67be-c113-4fbd-b529-8aa2fcee2832-kube-api-access-z2tgl\") on node \"crc\" DevicePath \"\"" Dec 03 11:46:13 crc kubenswrapper[5052]: I1203 11:46:13.220604 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/da3b67be-c113-4fbd-b529-8aa2fcee2832-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:46:13 crc kubenswrapper[5052]: I1203 11:46:13.220613 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/da3b67be-c113-4fbd-b529-8aa2fcee2832-logs\") on node \"crc\" DevicePath \"\"" Dec 03 11:46:13 crc kubenswrapper[5052]: I1203 11:46:13.445585 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"da3b67be-c113-4fbd-b529-8aa2fcee2832","Type":"ContainerDied","Data":"25d198d84b307c8e22e75f767240181dfee60030caef8bedee458cb5075342bf"} Dec 03 11:46:13 crc kubenswrapper[5052]: I1203 11:46:13.445666 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:46:13 crc kubenswrapper[5052]: I1203 11:46:13.445924 5052 scope.go:117] "RemoveContainer" containerID="8a086eb63697de2b6db28ff2a4b879d37f47b25d47c3d61efbccd9f2bbb5d9d3" Dec 03 11:46:13 crc kubenswrapper[5052]: I1203 11:46:13.468213 5052 scope.go:117] "RemoveContainer" containerID="b41b9d3d1c01190eb88dae38327e17fe28cbc505eecf87648682a5568b4c48f1" Dec 03 11:46:13 crc kubenswrapper[5052]: I1203 11:46:13.728358 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/da3b67be-c113-4fbd-b529-8aa2fcee2832-combined-ca-bundle\") pod \"da3b67be-c113-4fbd-b529-8aa2fcee2832\" (UID: \"da3b67be-c113-4fbd-b529-8aa2fcee2832\") " Dec 03 11:46:13 crc kubenswrapper[5052]: I1203 11:46:13.737679 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/da3b67be-c113-4fbd-b529-8aa2fcee2832-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "da3b67be-c113-4fbd-b529-8aa2fcee2832" (UID: "da3b67be-c113-4fbd-b529-8aa2fcee2832"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:46:13 crc kubenswrapper[5052]: I1203 11:46:13.809801 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 11:46:13 crc kubenswrapper[5052]: I1203 11:46:13.819823 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 11:46:13 crc kubenswrapper[5052]: I1203 11:46:13.831354 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/da3b67be-c113-4fbd-b529-8aa2fcee2832-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:46:13 crc kubenswrapper[5052]: I1203 11:46:13.839227 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 11:46:13 crc kubenswrapper[5052]: E1203 11:46:13.839899 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="059efa84-74c6-4d18-a279-5b7de59bb1b3" containerName="nova-manage" Dec 03 11:46:13 crc kubenswrapper[5052]: I1203 11:46:13.839924 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="059efa84-74c6-4d18-a279-5b7de59bb1b3" containerName="nova-manage" Dec 03 11:46:13 crc kubenswrapper[5052]: E1203 11:46:13.839948 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="da3b67be-c113-4fbd-b529-8aa2fcee2832" containerName="nova-api-api" Dec 03 11:46:13 crc kubenswrapper[5052]: I1203 11:46:13.839957 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="da3b67be-c113-4fbd-b529-8aa2fcee2832" containerName="nova-api-api" Dec 03 11:46:13 crc kubenswrapper[5052]: E1203 11:46:13.840005 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="da3b67be-c113-4fbd-b529-8aa2fcee2832" containerName="nova-api-log" Dec 03 11:46:13 crc kubenswrapper[5052]: I1203 11:46:13.840484 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="da3b67be-c113-4fbd-b529-8aa2fcee2832" containerName="nova-api-log" Dec 03 11:46:13 crc kubenswrapper[5052]: I1203 11:46:13.841582 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="da3b67be-c113-4fbd-b529-8aa2fcee2832" containerName="nova-api-api" Dec 03 11:46:13 crc kubenswrapper[5052]: I1203 11:46:13.841621 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="da3b67be-c113-4fbd-b529-8aa2fcee2832" containerName="nova-api-log" Dec 03 11:46:13 crc kubenswrapper[5052]: I1203 11:46:13.841639 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="059efa84-74c6-4d18-a279-5b7de59bb1b3" containerName="nova-manage" Dec 03 11:46:13 crc kubenswrapper[5052]: I1203 11:46:13.843101 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:46:13 crc kubenswrapper[5052]: I1203 11:46:13.847038 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-api-config-data" Dec 03 11:46:13 crc kubenswrapper[5052]: I1203 11:46:13.851711 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 11:46:14 crc kubenswrapper[5052]: I1203 11:46:14.037319 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/85eff54b-65b2-4877-ac8b-892194af103a-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"85eff54b-65b2-4877-ac8b-892194af103a\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:46:14 crc kubenswrapper[5052]: I1203 11:46:14.037416 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/85eff54b-65b2-4877-ac8b-892194af103a-logs\") pod \"nova-api-0\" (UID: \"85eff54b-65b2-4877-ac8b-892194af103a\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:46:14 crc kubenswrapper[5052]: I1203 11:46:14.037496 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9km47\" (UniqueName: \"kubernetes.io/projected/85eff54b-65b2-4877-ac8b-892194af103a-kube-api-access-9km47\") pod \"nova-api-0\" (UID: \"85eff54b-65b2-4877-ac8b-892194af103a\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:46:14 crc kubenswrapper[5052]: I1203 11:46:14.037583 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/85eff54b-65b2-4877-ac8b-892194af103a-config-data\") pod \"nova-api-0\" (UID: \"85eff54b-65b2-4877-ac8b-892194af103a\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:46:14 crc kubenswrapper[5052]: I1203 11:46:14.139142 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/85eff54b-65b2-4877-ac8b-892194af103a-config-data\") pod \"nova-api-0\" (UID: \"85eff54b-65b2-4877-ac8b-892194af103a\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:46:14 crc kubenswrapper[5052]: I1203 11:46:14.139219 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/85eff54b-65b2-4877-ac8b-892194af103a-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"85eff54b-65b2-4877-ac8b-892194af103a\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:46:14 crc kubenswrapper[5052]: I1203 11:46:14.139296 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/85eff54b-65b2-4877-ac8b-892194af103a-logs\") pod \"nova-api-0\" (UID: \"85eff54b-65b2-4877-ac8b-892194af103a\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:46:14 crc kubenswrapper[5052]: I1203 11:46:14.139351 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-9km47\" (UniqueName: \"kubernetes.io/projected/85eff54b-65b2-4877-ac8b-892194af103a-kube-api-access-9km47\") pod \"nova-api-0\" (UID: \"85eff54b-65b2-4877-ac8b-892194af103a\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:46:14 crc kubenswrapper[5052]: I1203 11:46:14.139887 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/85eff54b-65b2-4877-ac8b-892194af103a-logs\") pod \"nova-api-0\" (UID: \"85eff54b-65b2-4877-ac8b-892194af103a\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:46:14 crc kubenswrapper[5052]: I1203 11:46:14.146197 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/85eff54b-65b2-4877-ac8b-892194af103a-config-data\") pod \"nova-api-0\" (UID: \"85eff54b-65b2-4877-ac8b-892194af103a\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:46:14 crc kubenswrapper[5052]: I1203 11:46:14.152614 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/85eff54b-65b2-4877-ac8b-892194af103a-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"85eff54b-65b2-4877-ac8b-892194af103a\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:46:14 crc kubenswrapper[5052]: I1203 11:46:14.161478 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-9km47\" (UniqueName: \"kubernetes.io/projected/85eff54b-65b2-4877-ac8b-892194af103a-kube-api-access-9km47\") pod \"nova-api-0\" (UID: \"85eff54b-65b2-4877-ac8b-892194af103a\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:46:14 crc kubenswrapper[5052]: I1203 11:46:14.182446 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:46:14 crc kubenswrapper[5052]: I1203 11:46:14.639879 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 11:46:14 crc kubenswrapper[5052]: I1203 11:46:14.673531 5052 scope.go:117] "RemoveContainer" containerID="25fe88f8d102f09983f8884e65f0c6c1575ade67921f10a630e6dcc3ea8b506e" Dec 03 11:46:14 crc kubenswrapper[5052]: E1203 11:46:14.673805 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 11:46:15 crc kubenswrapper[5052]: I1203 11:46:15.472278 5052 generic.go:334] "Generic (PLEG): container finished" podID="38a8db66-3767-4079-91a7-157bbe1a23f3" containerID="e81f2bd5b57c0299aa652c12fddb54ea42890e277feba0dbd59df5827750445f" exitCode=0 Dec 03 11:46:15 crc kubenswrapper[5052]: I1203 11:46:15.472372 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"38a8db66-3767-4079-91a7-157bbe1a23f3","Type":"ContainerDied","Data":"e81f2bd5b57c0299aa652c12fddb54ea42890e277feba0dbd59df5827750445f"} Dec 03 11:46:15 crc kubenswrapper[5052]: I1203 11:46:15.474755 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"85eff54b-65b2-4877-ac8b-892194af103a","Type":"ContainerStarted","Data":"02b689dd60f694ea5d3055e53d21f12b849a3d9810d0d522c151ad30211151f8"} Dec 03 11:46:15 crc kubenswrapper[5052]: I1203 11:46:15.474794 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"85eff54b-65b2-4877-ac8b-892194af103a","Type":"ContainerStarted","Data":"82fa539a90a7f5b43c9c1d7d7f512d8c93e1b2d3b4899dd97f447497e2058aa2"} Dec 03 11:46:15 crc kubenswrapper[5052]: I1203 11:46:15.688752 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="da3b67be-c113-4fbd-b529-8aa2fcee2832" path="/var/lib/kubelet/pods/da3b67be-c113-4fbd-b529-8aa2fcee2832/volumes" Dec 03 11:46:15 crc kubenswrapper[5052]: I1203 11:46:15.733494 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:46:15 crc kubenswrapper[5052]: I1203 11:46:15.874411 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/38a8db66-3767-4079-91a7-157bbe1a23f3-nova-metadata-tls-certs\") pod \"38a8db66-3767-4079-91a7-157bbe1a23f3\" (UID: \"38a8db66-3767-4079-91a7-157bbe1a23f3\") " Dec 03 11:46:15 crc kubenswrapper[5052]: I1203 11:46:15.874497 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/38a8db66-3767-4079-91a7-157bbe1a23f3-config-data\") pod \"38a8db66-3767-4079-91a7-157bbe1a23f3\" (UID: \"38a8db66-3767-4079-91a7-157bbe1a23f3\") " Dec 03 11:46:15 crc kubenswrapper[5052]: I1203 11:46:15.874741 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bwpxl\" (UniqueName: \"kubernetes.io/projected/38a8db66-3767-4079-91a7-157bbe1a23f3-kube-api-access-bwpxl\") pod \"38a8db66-3767-4079-91a7-157bbe1a23f3\" (UID: \"38a8db66-3767-4079-91a7-157bbe1a23f3\") " Dec 03 11:46:15 crc kubenswrapper[5052]: I1203 11:46:15.875450 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/38a8db66-3767-4079-91a7-157bbe1a23f3-combined-ca-bundle\") pod \"38a8db66-3767-4079-91a7-157bbe1a23f3\" (UID: \"38a8db66-3767-4079-91a7-157bbe1a23f3\") " Dec 03 11:46:15 crc kubenswrapper[5052]: I1203 11:46:15.875482 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/38a8db66-3767-4079-91a7-157bbe1a23f3-logs\") pod \"38a8db66-3767-4079-91a7-157bbe1a23f3\" (UID: \"38a8db66-3767-4079-91a7-157bbe1a23f3\") " Dec 03 11:46:15 crc kubenswrapper[5052]: I1203 11:46:15.875953 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/38a8db66-3767-4079-91a7-157bbe1a23f3-logs" (OuterVolumeSpecName: "logs") pod "38a8db66-3767-4079-91a7-157bbe1a23f3" (UID: "38a8db66-3767-4079-91a7-157bbe1a23f3"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:46:15 crc kubenswrapper[5052]: I1203 11:46:15.876168 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/38a8db66-3767-4079-91a7-157bbe1a23f3-logs\") on node \"crc\" DevicePath \"\"" Dec 03 11:46:15 crc kubenswrapper[5052]: I1203 11:46:15.880464 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/38a8db66-3767-4079-91a7-157bbe1a23f3-kube-api-access-bwpxl" (OuterVolumeSpecName: "kube-api-access-bwpxl") pod "38a8db66-3767-4079-91a7-157bbe1a23f3" (UID: "38a8db66-3767-4079-91a7-157bbe1a23f3"). InnerVolumeSpecName "kube-api-access-bwpxl". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:46:15 crc kubenswrapper[5052]: I1203 11:46:15.903411 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/38a8db66-3767-4079-91a7-157bbe1a23f3-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "38a8db66-3767-4079-91a7-157bbe1a23f3" (UID: "38a8db66-3767-4079-91a7-157bbe1a23f3"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:46:15 crc kubenswrapper[5052]: I1203 11:46:15.904053 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/38a8db66-3767-4079-91a7-157bbe1a23f3-config-data" (OuterVolumeSpecName: "config-data") pod "38a8db66-3767-4079-91a7-157bbe1a23f3" (UID: "38a8db66-3767-4079-91a7-157bbe1a23f3"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:46:15 crc kubenswrapper[5052]: I1203 11:46:15.925360 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/38a8db66-3767-4079-91a7-157bbe1a23f3-nova-metadata-tls-certs" (OuterVolumeSpecName: "nova-metadata-tls-certs") pod "38a8db66-3767-4079-91a7-157bbe1a23f3" (UID: "38a8db66-3767-4079-91a7-157bbe1a23f3"). InnerVolumeSpecName "nova-metadata-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:46:15 crc kubenswrapper[5052]: I1203 11:46:15.978491 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-bwpxl\" (UniqueName: \"kubernetes.io/projected/38a8db66-3767-4079-91a7-157bbe1a23f3-kube-api-access-bwpxl\") on node \"crc\" DevicePath \"\"" Dec 03 11:46:15 crc kubenswrapper[5052]: I1203 11:46:15.978546 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/38a8db66-3767-4079-91a7-157bbe1a23f3-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:46:15 crc kubenswrapper[5052]: I1203 11:46:15.978557 5052 reconciler_common.go:293] "Volume detached for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/38a8db66-3767-4079-91a7-157bbe1a23f3-nova-metadata-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 11:46:15 crc kubenswrapper[5052]: I1203 11:46:15.978568 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/38a8db66-3767-4079-91a7-157bbe1a23f3-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:46:16 crc kubenswrapper[5052]: I1203 11:46:16.484960 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"38a8db66-3767-4079-91a7-157bbe1a23f3","Type":"ContainerDied","Data":"60c98d3b8a5ab8167d68bd076a2415df8fb418b5d603a3a2e17dfadea67a206a"} Dec 03 11:46:16 crc kubenswrapper[5052]: I1203 11:46:16.485322 5052 scope.go:117] "RemoveContainer" containerID="e81f2bd5b57c0299aa652c12fddb54ea42890e277feba0dbd59df5827750445f" Dec 03 11:46:16 crc kubenswrapper[5052]: I1203 11:46:16.485441 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:46:16 crc kubenswrapper[5052]: I1203 11:46:16.495089 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"85eff54b-65b2-4877-ac8b-892194af103a","Type":"ContainerStarted","Data":"7e4e52218f80457d114f0a54da0635f7033f1fcd126523398bfe258d9d538373"} Dec 03 11:46:16 crc kubenswrapper[5052]: I1203 11:46:16.497100 5052 generic.go:334] "Generic (PLEG): container finished" podID="f51e7748-4baa-4b1c-a13d-84bb07444009" containerID="b7ce974e9594aa72254374252b2c28f90aea8e0ff5b969e81586db771992d287" exitCode=0 Dec 03 11:46:16 crc kubenswrapper[5052]: I1203 11:46:16.497259 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"f51e7748-4baa-4b1c-a13d-84bb07444009","Type":"ContainerDied","Data":"b7ce974e9594aa72254374252b2c28f90aea8e0ff5b969e81586db771992d287"} Dec 03 11:46:16 crc kubenswrapper[5052]: I1203 11:46:16.525779 5052 scope.go:117] "RemoveContainer" containerID="04a49934bbdc1fc138326569202470b6329b0ea42ba7c6cf027ab3bad497aa8d" Dec 03 11:46:16 crc kubenswrapper[5052]: I1203 11:46:16.539895 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-api-0" podStartSLOduration=3.53987679 podStartE2EDuration="3.53987679s" podCreationTimestamp="2025-12-03 11:46:13 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:46:16.520943264 +0000 UTC m=+1567.109185048" watchObservedRunningTime="2025-12-03 11:46:16.53987679 +0000 UTC m=+1567.128118554" Dec 03 11:46:16 crc kubenswrapper[5052]: I1203 11:46:16.544794 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 11:46:16 crc kubenswrapper[5052]: I1203 11:46:16.561334 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 11:46:16 crc kubenswrapper[5052]: I1203 11:46:16.577054 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 11:46:16 crc kubenswrapper[5052]: E1203 11:46:16.577640 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="38a8db66-3767-4079-91a7-157bbe1a23f3" containerName="nova-metadata-log" Dec 03 11:46:16 crc kubenswrapper[5052]: I1203 11:46:16.577660 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="38a8db66-3767-4079-91a7-157bbe1a23f3" containerName="nova-metadata-log" Dec 03 11:46:16 crc kubenswrapper[5052]: E1203 11:46:16.577670 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="38a8db66-3767-4079-91a7-157bbe1a23f3" containerName="nova-metadata-metadata" Dec 03 11:46:16 crc kubenswrapper[5052]: I1203 11:46:16.577679 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="38a8db66-3767-4079-91a7-157bbe1a23f3" containerName="nova-metadata-metadata" Dec 03 11:46:16 crc kubenswrapper[5052]: I1203 11:46:16.577906 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="38a8db66-3767-4079-91a7-157bbe1a23f3" containerName="nova-metadata-log" Dec 03 11:46:16 crc kubenswrapper[5052]: I1203 11:46:16.577929 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="38a8db66-3767-4079-91a7-157bbe1a23f3" containerName="nova-metadata-metadata" Dec 03 11:46:16 crc kubenswrapper[5052]: I1203 11:46:16.579046 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:46:16 crc kubenswrapper[5052]: I1203 11:46:16.583361 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-nova-metadata-internal-svc" Dec 03 11:46:16 crc kubenswrapper[5052]: I1203 11:46:16.583597 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-metadata-config-data" Dec 03 11:46:16 crc kubenswrapper[5052]: I1203 11:46:16.588114 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 11:46:16 crc kubenswrapper[5052]: I1203 11:46:16.690913 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/bb51da3b-72cc-4008-aac7-737f6a3031cb-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"bb51da3b-72cc-4008-aac7-737f6a3031cb\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:46:16 crc kubenswrapper[5052]: I1203 11:46:16.690972 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/bb51da3b-72cc-4008-aac7-737f6a3031cb-config-data\") pod \"nova-metadata-0\" (UID: \"bb51da3b-72cc-4008-aac7-737f6a3031cb\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:46:16 crc kubenswrapper[5052]: I1203 11:46:16.691200 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/bb51da3b-72cc-4008-aac7-737f6a3031cb-nova-metadata-tls-certs\") pod \"nova-metadata-0\" (UID: \"bb51da3b-72cc-4008-aac7-737f6a3031cb\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:46:16 crc kubenswrapper[5052]: I1203 11:46:16.691254 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dbh4l\" (UniqueName: \"kubernetes.io/projected/bb51da3b-72cc-4008-aac7-737f6a3031cb-kube-api-access-dbh4l\") pod \"nova-metadata-0\" (UID: \"bb51da3b-72cc-4008-aac7-737f6a3031cb\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:46:16 crc kubenswrapper[5052]: I1203 11:46:16.691274 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/bb51da3b-72cc-4008-aac7-737f6a3031cb-logs\") pod \"nova-metadata-0\" (UID: \"bb51da3b-72cc-4008-aac7-737f6a3031cb\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:46:16 crc kubenswrapper[5052]: I1203 11:46:16.792819 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/bb51da3b-72cc-4008-aac7-737f6a3031cb-nova-metadata-tls-certs\") pod \"nova-metadata-0\" (UID: \"bb51da3b-72cc-4008-aac7-737f6a3031cb\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:46:16 crc kubenswrapper[5052]: I1203 11:46:16.792963 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-dbh4l\" (UniqueName: \"kubernetes.io/projected/bb51da3b-72cc-4008-aac7-737f6a3031cb-kube-api-access-dbh4l\") pod \"nova-metadata-0\" (UID: \"bb51da3b-72cc-4008-aac7-737f6a3031cb\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:46:16 crc kubenswrapper[5052]: I1203 11:46:16.792982 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/bb51da3b-72cc-4008-aac7-737f6a3031cb-logs\") pod \"nova-metadata-0\" (UID: \"bb51da3b-72cc-4008-aac7-737f6a3031cb\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:46:16 crc kubenswrapper[5052]: I1203 11:46:16.793050 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/bb51da3b-72cc-4008-aac7-737f6a3031cb-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"bb51da3b-72cc-4008-aac7-737f6a3031cb\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:46:16 crc kubenswrapper[5052]: I1203 11:46:16.793076 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/bb51da3b-72cc-4008-aac7-737f6a3031cb-config-data\") pod \"nova-metadata-0\" (UID: \"bb51da3b-72cc-4008-aac7-737f6a3031cb\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:46:16 crc kubenswrapper[5052]: I1203 11:46:16.793478 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/bb51da3b-72cc-4008-aac7-737f6a3031cb-logs\") pod \"nova-metadata-0\" (UID: \"bb51da3b-72cc-4008-aac7-737f6a3031cb\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:46:16 crc kubenswrapper[5052]: I1203 11:46:16.797315 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/bb51da3b-72cc-4008-aac7-737f6a3031cb-nova-metadata-tls-certs\") pod \"nova-metadata-0\" (UID: \"bb51da3b-72cc-4008-aac7-737f6a3031cb\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:46:16 crc kubenswrapper[5052]: I1203 11:46:16.797828 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/bb51da3b-72cc-4008-aac7-737f6a3031cb-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"bb51da3b-72cc-4008-aac7-737f6a3031cb\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:46:16 crc kubenswrapper[5052]: I1203 11:46:16.798244 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/bb51da3b-72cc-4008-aac7-737f6a3031cb-config-data\") pod \"nova-metadata-0\" (UID: \"bb51da3b-72cc-4008-aac7-737f6a3031cb\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:46:16 crc kubenswrapper[5052]: I1203 11:46:16.811452 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-dbh4l\" (UniqueName: \"kubernetes.io/projected/bb51da3b-72cc-4008-aac7-737f6a3031cb-kube-api-access-dbh4l\") pod \"nova-metadata-0\" (UID: \"bb51da3b-72cc-4008-aac7-737f6a3031cb\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:46:16 crc kubenswrapper[5052]: I1203 11:46:16.907637 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:46:16 crc kubenswrapper[5052]: I1203 11:46:16.916130 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:46:17 crc kubenswrapper[5052]: I1203 11:46:17.097074 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/f51e7748-4baa-4b1c-a13d-84bb07444009-config-data\") pod \"f51e7748-4baa-4b1c-a13d-84bb07444009\" (UID: \"f51e7748-4baa-4b1c-a13d-84bb07444009\") " Dec 03 11:46:17 crc kubenswrapper[5052]: I1203 11:46:17.097453 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-44zx5\" (UniqueName: \"kubernetes.io/projected/f51e7748-4baa-4b1c-a13d-84bb07444009-kube-api-access-44zx5\") pod \"f51e7748-4baa-4b1c-a13d-84bb07444009\" (UID: \"f51e7748-4baa-4b1c-a13d-84bb07444009\") " Dec 03 11:46:17 crc kubenswrapper[5052]: I1203 11:46:17.097506 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f51e7748-4baa-4b1c-a13d-84bb07444009-combined-ca-bundle\") pod \"f51e7748-4baa-4b1c-a13d-84bb07444009\" (UID: \"f51e7748-4baa-4b1c-a13d-84bb07444009\") " Dec 03 11:46:17 crc kubenswrapper[5052]: I1203 11:46:17.102659 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f51e7748-4baa-4b1c-a13d-84bb07444009-kube-api-access-44zx5" (OuterVolumeSpecName: "kube-api-access-44zx5") pod "f51e7748-4baa-4b1c-a13d-84bb07444009" (UID: "f51e7748-4baa-4b1c-a13d-84bb07444009"). InnerVolumeSpecName "kube-api-access-44zx5". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:46:17 crc kubenswrapper[5052]: I1203 11:46:17.127870 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f51e7748-4baa-4b1c-a13d-84bb07444009-config-data" (OuterVolumeSpecName: "config-data") pod "f51e7748-4baa-4b1c-a13d-84bb07444009" (UID: "f51e7748-4baa-4b1c-a13d-84bb07444009"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:46:17 crc kubenswrapper[5052]: I1203 11:46:17.129494 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f51e7748-4baa-4b1c-a13d-84bb07444009-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "f51e7748-4baa-4b1c-a13d-84bb07444009" (UID: "f51e7748-4baa-4b1c-a13d-84bb07444009"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:46:17 crc kubenswrapper[5052]: I1203 11:46:17.199014 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f51e7748-4baa-4b1c-a13d-84bb07444009-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:46:17 crc kubenswrapper[5052]: I1203 11:46:17.199053 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/f51e7748-4baa-4b1c-a13d-84bb07444009-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:46:17 crc kubenswrapper[5052]: I1203 11:46:17.199065 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-44zx5\" (UniqueName: \"kubernetes.io/projected/f51e7748-4baa-4b1c-a13d-84bb07444009-kube-api-access-44zx5\") on node \"crc\" DevicePath \"\"" Dec 03 11:46:17 crc kubenswrapper[5052]: I1203 11:46:17.400982 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 11:46:17 crc kubenswrapper[5052]: W1203 11:46:17.408280 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbb51da3b_72cc_4008_aac7_737f6a3031cb.slice/crio-ecf1ebfdd6784d04d2c4832c90b30e0d04f644e2d2252aa874e943438f064bf5 WatchSource:0}: Error finding container ecf1ebfdd6784d04d2c4832c90b30e0d04f644e2d2252aa874e943438f064bf5: Status 404 returned error can't find the container with id ecf1ebfdd6784d04d2c4832c90b30e0d04f644e2d2252aa874e943438f064bf5 Dec 03 11:46:17 crc kubenswrapper[5052]: I1203 11:46:17.507997 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:46:17 crc kubenswrapper[5052]: I1203 11:46:17.508010 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"f51e7748-4baa-4b1c-a13d-84bb07444009","Type":"ContainerDied","Data":"e546758641bd9bbab5a2d38d7e0a777df416fc377f63bf8112944dd884fd95d8"} Dec 03 11:46:17 crc kubenswrapper[5052]: I1203 11:46:17.508059 5052 scope.go:117] "RemoveContainer" containerID="b7ce974e9594aa72254374252b2c28f90aea8e0ff5b969e81586db771992d287" Dec 03 11:46:17 crc kubenswrapper[5052]: I1203 11:46:17.510051 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"bb51da3b-72cc-4008-aac7-737f6a3031cb","Type":"ContainerStarted","Data":"ecf1ebfdd6784d04d2c4832c90b30e0d04f644e2d2252aa874e943438f064bf5"} Dec 03 11:46:17 crc kubenswrapper[5052]: I1203 11:46:17.567701 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 11:46:17 crc kubenswrapper[5052]: I1203 11:46:17.609621 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 11:46:17 crc kubenswrapper[5052]: I1203 11:46:17.623120 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 11:46:17 crc kubenswrapper[5052]: E1203 11:46:17.623598 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="f51e7748-4baa-4b1c-a13d-84bb07444009" containerName="nova-scheduler-scheduler" Dec 03 11:46:17 crc kubenswrapper[5052]: I1203 11:46:17.623617 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="f51e7748-4baa-4b1c-a13d-84bb07444009" containerName="nova-scheduler-scheduler" Dec 03 11:46:17 crc kubenswrapper[5052]: I1203 11:46:17.623843 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="f51e7748-4baa-4b1c-a13d-84bb07444009" containerName="nova-scheduler-scheduler" Dec 03 11:46:17 crc kubenswrapper[5052]: I1203 11:46:17.624867 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:46:17 crc kubenswrapper[5052]: I1203 11:46:17.627404 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-scheduler-config-data" Dec 03 11:46:17 crc kubenswrapper[5052]: I1203 11:46:17.635037 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 11:46:17 crc kubenswrapper[5052]: I1203 11:46:17.687671 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="38a8db66-3767-4079-91a7-157bbe1a23f3" path="/var/lib/kubelet/pods/38a8db66-3767-4079-91a7-157bbe1a23f3/volumes" Dec 03 11:46:17 crc kubenswrapper[5052]: I1203 11:46:17.689192 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f51e7748-4baa-4b1c-a13d-84bb07444009" path="/var/lib/kubelet/pods/f51e7748-4baa-4b1c-a13d-84bb07444009/volumes" Dec 03 11:46:17 crc kubenswrapper[5052]: I1203 11:46:17.708619 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8pptj\" (UniqueName: \"kubernetes.io/projected/39b9198e-e3d7-4f85-b6a2-55960bdda82a-kube-api-access-8pptj\") pod \"nova-scheduler-0\" (UID: \"39b9198e-e3d7-4f85-b6a2-55960bdda82a\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:46:17 crc kubenswrapper[5052]: I1203 11:46:17.708692 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/39b9198e-e3d7-4f85-b6a2-55960bdda82a-config-data\") pod \"nova-scheduler-0\" (UID: \"39b9198e-e3d7-4f85-b6a2-55960bdda82a\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:46:17 crc kubenswrapper[5052]: I1203 11:46:17.708842 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/39b9198e-e3d7-4f85-b6a2-55960bdda82a-combined-ca-bundle\") pod \"nova-scheduler-0\" (UID: \"39b9198e-e3d7-4f85-b6a2-55960bdda82a\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:46:17 crc kubenswrapper[5052]: I1203 11:46:17.810976 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-8pptj\" (UniqueName: \"kubernetes.io/projected/39b9198e-e3d7-4f85-b6a2-55960bdda82a-kube-api-access-8pptj\") pod \"nova-scheduler-0\" (UID: \"39b9198e-e3d7-4f85-b6a2-55960bdda82a\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:46:17 crc kubenswrapper[5052]: I1203 11:46:17.811182 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/39b9198e-e3d7-4f85-b6a2-55960bdda82a-config-data\") pod \"nova-scheduler-0\" (UID: \"39b9198e-e3d7-4f85-b6a2-55960bdda82a\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:46:17 crc kubenswrapper[5052]: I1203 11:46:17.811296 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/39b9198e-e3d7-4f85-b6a2-55960bdda82a-combined-ca-bundle\") pod \"nova-scheduler-0\" (UID: \"39b9198e-e3d7-4f85-b6a2-55960bdda82a\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:46:17 crc kubenswrapper[5052]: I1203 11:46:17.815258 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/39b9198e-e3d7-4f85-b6a2-55960bdda82a-config-data\") pod \"nova-scheduler-0\" (UID: \"39b9198e-e3d7-4f85-b6a2-55960bdda82a\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:46:17 crc kubenswrapper[5052]: I1203 11:46:17.815948 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/39b9198e-e3d7-4f85-b6a2-55960bdda82a-combined-ca-bundle\") pod \"nova-scheduler-0\" (UID: \"39b9198e-e3d7-4f85-b6a2-55960bdda82a\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:46:17 crc kubenswrapper[5052]: I1203 11:46:17.836353 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-8pptj\" (UniqueName: \"kubernetes.io/projected/39b9198e-e3d7-4f85-b6a2-55960bdda82a-kube-api-access-8pptj\") pod \"nova-scheduler-0\" (UID: \"39b9198e-e3d7-4f85-b6a2-55960bdda82a\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:46:18 crc kubenswrapper[5052]: I1203 11:46:18.063557 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:46:18 crc kubenswrapper[5052]: I1203 11:46:18.526621 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"bb51da3b-72cc-4008-aac7-737f6a3031cb","Type":"ContainerStarted","Data":"b39f7b31c335958d6613651cea9f3ccbfee7c25b7c7641048321f3e460ee0193"} Dec 03 11:46:18 crc kubenswrapper[5052]: I1203 11:46:18.526974 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"bb51da3b-72cc-4008-aac7-737f6a3031cb","Type":"ContainerStarted","Data":"4568b2e2e54107935eb412f6e606a9e5b4fd360b7f1d0fae94b2b383fd9205d0"} Dec 03 11:46:18 crc kubenswrapper[5052]: I1203 11:46:18.549242 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 11:46:18 crc kubenswrapper[5052]: I1203 11:46:18.552296 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-metadata-0" podStartSLOduration=2.5522773 podStartE2EDuration="2.5522773s" podCreationTimestamp="2025-12-03 11:46:16 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:46:18.543069745 +0000 UTC m=+1569.131311519" watchObservedRunningTime="2025-12-03 11:46:18.5522773 +0000 UTC m=+1569.140519064" Dec 03 11:46:19 crc kubenswrapper[5052]: I1203 11:46:19.538314 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"39b9198e-e3d7-4f85-b6a2-55960bdda82a","Type":"ContainerStarted","Data":"1dbdda8905493662bc20bc18fcae5f54dd4929f65dec1b16df59f03dbfd76d8a"} Dec 03 11:46:19 crc kubenswrapper[5052]: I1203 11:46:19.538810 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"39b9198e-e3d7-4f85-b6a2-55960bdda82a","Type":"ContainerStarted","Data":"2de361109ed724167cdae870a30228df924104c25e4439c710241e2b780c2e64"} Dec 03 11:46:19 crc kubenswrapper[5052]: I1203 11:46:19.561192 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-scheduler-0" podStartSLOduration=2.561171974 podStartE2EDuration="2.561171974s" podCreationTimestamp="2025-12-03 11:46:17 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:46:19.555956563 +0000 UTC m=+1570.144198337" watchObservedRunningTime="2025-12-03 11:46:19.561171974 +0000 UTC m=+1570.149413748" Dec 03 11:46:21 crc kubenswrapper[5052]: I1203 11:46:21.907975 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:46:21 crc kubenswrapper[5052]: I1203 11:46:21.908489 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:46:23 crc kubenswrapper[5052]: I1203 11:46:23.064397 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:46:24 crc kubenswrapper[5052]: I1203 11:46:24.183404 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:46:24 crc kubenswrapper[5052]: I1203 11:46:24.183500 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:46:25 crc kubenswrapper[5052]: I1203 11:46:25.266712 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-api-0" podUID="85eff54b-65b2-4877-ac8b-892194af103a" containerName="nova-api-log" probeResult="failure" output="Get \"http://10.217.1.31:8774/\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Dec 03 11:46:25 crc kubenswrapper[5052]: I1203 11:46:25.266708 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-api-0" podUID="85eff54b-65b2-4877-ac8b-892194af103a" containerName="nova-api-api" probeResult="failure" output="Get \"http://10.217.1.31:8774/\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Dec 03 11:46:26 crc kubenswrapper[5052]: I1203 11:46:26.908306 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:46:26 crc kubenswrapper[5052]: I1203 11:46:26.908775 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:46:27 crc kubenswrapper[5052]: I1203 11:46:27.673384 5052 scope.go:117] "RemoveContainer" containerID="25fe88f8d102f09983f8884e65f0c6c1575ade67921f10a630e6dcc3ea8b506e" Dec 03 11:46:27 crc kubenswrapper[5052]: E1203 11:46:27.674089 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 11:46:27 crc kubenswrapper[5052]: I1203 11:46:27.922770 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-metadata-0" podUID="bb51da3b-72cc-4008-aac7-737f6a3031cb" containerName="nova-metadata-metadata" probeResult="failure" output="Get \"https://10.217.1.32:8775/\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Dec 03 11:46:27 crc kubenswrapper[5052]: I1203 11:46:27.922768 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-metadata-0" podUID="bb51da3b-72cc-4008-aac7-737f6a3031cb" containerName="nova-metadata-log" probeResult="failure" output="Get \"https://10.217.1.32:8775/\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Dec 03 11:46:28 crc kubenswrapper[5052]: I1203 11:46:28.063958 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:46:28 crc kubenswrapper[5052]: I1203 11:46:28.100822 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:46:28 crc kubenswrapper[5052]: I1203 11:46:28.666468 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:46:28 crc kubenswrapper[5052]: I1203 11:46:28.974557 5052 scope.go:117] "RemoveContainer" containerID="7f7814a46f2c2a853ddc0d2142a5cfb0bf53aa603be45491209a66cd585a4f18" Dec 03 11:46:29 crc kubenswrapper[5052]: I1203 11:46:29.011613 5052 scope.go:117] "RemoveContainer" containerID="8353d9ed68998103ae5183e6a229e34b9624453345905d1783ef8d0257b3e95b" Dec 03 11:46:29 crc kubenswrapper[5052]: I1203 11:46:29.039199 5052 scope.go:117] "RemoveContainer" containerID="230b893b58887ce8075576ec97b7c1b839aac29a4c4a0c82f043dbd5a3fbe8eb" Dec 03 11:46:29 crc kubenswrapper[5052]: I1203 11:46:29.090973 5052 scope.go:117] "RemoveContainer" containerID="74b7843216af5eaa832491d05cc3f8cd094a9315e804a6f571a7f09fd5df0acf" Dec 03 11:46:29 crc kubenswrapper[5052]: I1203 11:46:29.151671 5052 scope.go:117] "RemoveContainer" containerID="907dd58d6f36fcd52c737d7df15fd6614bc5eeddfe7082592f83e68d9f16190c" Dec 03 11:46:29 crc kubenswrapper[5052]: I1203 11:46:29.175743 5052 scope.go:117] "RemoveContainer" containerID="ac95353be0e40db2feb743167732689ef07e0a7b12dc8b53d9193046d46c4d89" Dec 03 11:46:29 crc kubenswrapper[5052]: I1203 11:46:29.213991 5052 scope.go:117] "RemoveContainer" containerID="7f0e3ee717dce1717e1541b66434fdf13121b35964b4328bce3b46e0ca09e4ca" Dec 03 11:46:29 crc kubenswrapper[5052]: I1203 11:46:29.263829 5052 scope.go:117] "RemoveContainer" containerID="735a470576dc74f7e9b95db518ca1e458cf10b51cd97c1cbdb4f9e95d02a1414" Dec 03 11:46:29 crc kubenswrapper[5052]: I1203 11:46:29.296490 5052 scope.go:117] "RemoveContainer" containerID="c4c73ffbd6756af9b104cfcef9220c7341172a6c5dadc45edcc42921bbd74050" Dec 03 11:46:29 crc kubenswrapper[5052]: I1203 11:46:29.336457 5052 scope.go:117] "RemoveContainer" containerID="4c92b1739b5436291f4de41014a637fd37a00db71202d30b8ea1ae9006123ea8" Dec 03 11:46:29 crc kubenswrapper[5052]: I1203 11:46:29.368990 5052 scope.go:117] "RemoveContainer" containerID="25c8b36672104ddddb9cda8135c26753231310450ea8bff9f6700b865bc3d056" Dec 03 11:46:29 crc kubenswrapper[5052]: I1203 11:46:29.410426 5052 scope.go:117] "RemoveContainer" containerID="866dbba5fee7602bed5d8a92bd3bd35bca49efdc6fe77d6689fef78dc3ac2b60" Dec 03 11:46:34 crc kubenswrapper[5052]: I1203 11:46:34.187269 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:46:34 crc kubenswrapper[5052]: I1203 11:46:34.187871 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:46:34 crc kubenswrapper[5052]: I1203 11:46:34.188150 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:46:34 crc kubenswrapper[5052]: I1203 11:46:34.188467 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:46:34 crc kubenswrapper[5052]: I1203 11:46:34.190579 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:46:34 crc kubenswrapper[5052]: I1203 11:46:34.191753 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:46:36 crc kubenswrapper[5052]: I1203 11:46:36.241825 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:46:36 crc kubenswrapper[5052]: I1203 11:46:36.242460 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="118e9ee6-78ab-460c-8f42-e7db0a8bf632" containerName="ceilometer-central-agent" containerID="cri-o://0a5f58a28001495eec6556c898240aa664f99a537bf8177e79bd12695098917f" gracePeriod=30 Dec 03 11:46:36 crc kubenswrapper[5052]: I1203 11:46:36.242882 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="118e9ee6-78ab-460c-8f42-e7db0a8bf632" containerName="proxy-httpd" containerID="cri-o://85d3d8672f9a85a6e39aadfe82bcb0f9e330f484ac61213e268d33849beb7b60" gracePeriod=30 Dec 03 11:46:36 crc kubenswrapper[5052]: I1203 11:46:36.242951 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="118e9ee6-78ab-460c-8f42-e7db0a8bf632" containerName="sg-core" containerID="cri-o://8aa26a6ca34ca18a8274f1330a7ce992bf80788902b95e4f72cd9c387528b7d2" gracePeriod=30 Dec 03 11:46:36 crc kubenswrapper[5052]: I1203 11:46:36.243000 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="118e9ee6-78ab-460c-8f42-e7db0a8bf632" containerName="ceilometer-notification-agent" containerID="cri-o://654a5e049ba0d38968af8f2a6181dd489b689a0b1ca0a64b88c4e5e9b30fc78e" gracePeriod=30 Dec 03 11:46:36 crc kubenswrapper[5052]: I1203 11:46:36.550015 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 11:46:36 crc kubenswrapper[5052]: I1203 11:46:36.719699 5052 generic.go:334] "Generic (PLEG): container finished" podID="118e9ee6-78ab-460c-8f42-e7db0a8bf632" containerID="85d3d8672f9a85a6e39aadfe82bcb0f9e330f484ac61213e268d33849beb7b60" exitCode=0 Dec 03 11:46:36 crc kubenswrapper[5052]: I1203 11:46:36.719734 5052 generic.go:334] "Generic (PLEG): container finished" podID="118e9ee6-78ab-460c-8f42-e7db0a8bf632" containerID="8aa26a6ca34ca18a8274f1330a7ce992bf80788902b95e4f72cd9c387528b7d2" exitCode=2 Dec 03 11:46:36 crc kubenswrapper[5052]: I1203 11:46:36.719742 5052 generic.go:334] "Generic (PLEG): container finished" podID="118e9ee6-78ab-460c-8f42-e7db0a8bf632" containerID="0a5f58a28001495eec6556c898240aa664f99a537bf8177e79bd12695098917f" exitCode=0 Dec 03 11:46:36 crc kubenswrapper[5052]: I1203 11:46:36.719927 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-api-0" podUID="85eff54b-65b2-4877-ac8b-892194af103a" containerName="nova-api-log" containerID="cri-o://02b689dd60f694ea5d3055e53d21f12b849a3d9810d0d522c151ad30211151f8" gracePeriod=30 Dec 03 11:46:36 crc kubenswrapper[5052]: I1203 11:46:36.720199 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"118e9ee6-78ab-460c-8f42-e7db0a8bf632","Type":"ContainerDied","Data":"85d3d8672f9a85a6e39aadfe82bcb0f9e330f484ac61213e268d33849beb7b60"} Dec 03 11:46:36 crc kubenswrapper[5052]: I1203 11:46:36.720232 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"118e9ee6-78ab-460c-8f42-e7db0a8bf632","Type":"ContainerDied","Data":"8aa26a6ca34ca18a8274f1330a7ce992bf80788902b95e4f72cd9c387528b7d2"} Dec 03 11:46:36 crc kubenswrapper[5052]: I1203 11:46:36.720243 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"118e9ee6-78ab-460c-8f42-e7db0a8bf632","Type":"ContainerDied","Data":"0a5f58a28001495eec6556c898240aa664f99a537bf8177e79bd12695098917f"} Dec 03 11:46:36 crc kubenswrapper[5052]: I1203 11:46:36.720570 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-api-0" podUID="85eff54b-65b2-4877-ac8b-892194af103a" containerName="nova-api-api" containerID="cri-o://7e4e52218f80457d114f0a54da0635f7033f1fcd126523398bfe258d9d538373" gracePeriod=30 Dec 03 11:46:36 crc kubenswrapper[5052]: I1203 11:46:36.912932 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:46:36 crc kubenswrapper[5052]: I1203 11:46:36.921613 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:46:36 crc kubenswrapper[5052]: I1203 11:46:36.925991 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:46:37 crc kubenswrapper[5052]: I1203 11:46:37.802253 5052 generic.go:334] "Generic (PLEG): container finished" podID="85eff54b-65b2-4877-ac8b-892194af103a" containerID="02b689dd60f694ea5d3055e53d21f12b849a3d9810d0d522c151ad30211151f8" exitCode=143 Dec 03 11:46:37 crc kubenswrapper[5052]: I1203 11:46:37.803720 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"85eff54b-65b2-4877-ac8b-892194af103a","Type":"ContainerDied","Data":"02b689dd60f694ea5d3055e53d21f12b849a3d9810d0d522c151ad30211151f8"} Dec 03 11:46:37 crc kubenswrapper[5052]: I1203 11:46:37.844099 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:46:38 crc kubenswrapper[5052]: I1203 11:46:38.520718 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:46:38 crc kubenswrapper[5052]: I1203 11:46:38.625723 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/118e9ee6-78ab-460c-8f42-e7db0a8bf632-sg-core-conf-yaml\") pod \"118e9ee6-78ab-460c-8f42-e7db0a8bf632\" (UID: \"118e9ee6-78ab-460c-8f42-e7db0a8bf632\") " Dec 03 11:46:38 crc kubenswrapper[5052]: I1203 11:46:38.625770 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/118e9ee6-78ab-460c-8f42-e7db0a8bf632-config-data\") pod \"118e9ee6-78ab-460c-8f42-e7db0a8bf632\" (UID: \"118e9ee6-78ab-460c-8f42-e7db0a8bf632\") " Dec 03 11:46:38 crc kubenswrapper[5052]: I1203 11:46:38.625804 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/118e9ee6-78ab-460c-8f42-e7db0a8bf632-ceilometer-tls-certs\") pod \"118e9ee6-78ab-460c-8f42-e7db0a8bf632\" (UID: \"118e9ee6-78ab-460c-8f42-e7db0a8bf632\") " Dec 03 11:46:38 crc kubenswrapper[5052]: I1203 11:46:38.625828 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/118e9ee6-78ab-460c-8f42-e7db0a8bf632-combined-ca-bundle\") pod \"118e9ee6-78ab-460c-8f42-e7db0a8bf632\" (UID: \"118e9ee6-78ab-460c-8f42-e7db0a8bf632\") " Dec 03 11:46:38 crc kubenswrapper[5052]: I1203 11:46:38.625929 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/118e9ee6-78ab-460c-8f42-e7db0a8bf632-run-httpd\") pod \"118e9ee6-78ab-460c-8f42-e7db0a8bf632\" (UID: \"118e9ee6-78ab-460c-8f42-e7db0a8bf632\") " Dec 03 11:46:38 crc kubenswrapper[5052]: I1203 11:46:38.625990 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4mqz2\" (UniqueName: \"kubernetes.io/projected/118e9ee6-78ab-460c-8f42-e7db0a8bf632-kube-api-access-4mqz2\") pod \"118e9ee6-78ab-460c-8f42-e7db0a8bf632\" (UID: \"118e9ee6-78ab-460c-8f42-e7db0a8bf632\") " Dec 03 11:46:38 crc kubenswrapper[5052]: I1203 11:46:38.626016 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/118e9ee6-78ab-460c-8f42-e7db0a8bf632-scripts\") pod \"118e9ee6-78ab-460c-8f42-e7db0a8bf632\" (UID: \"118e9ee6-78ab-460c-8f42-e7db0a8bf632\") " Dec 03 11:46:38 crc kubenswrapper[5052]: I1203 11:46:38.626080 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/118e9ee6-78ab-460c-8f42-e7db0a8bf632-log-httpd\") pod \"118e9ee6-78ab-460c-8f42-e7db0a8bf632\" (UID: \"118e9ee6-78ab-460c-8f42-e7db0a8bf632\") " Dec 03 11:46:38 crc kubenswrapper[5052]: I1203 11:46:38.626765 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/118e9ee6-78ab-460c-8f42-e7db0a8bf632-run-httpd" (OuterVolumeSpecName: "run-httpd") pod "118e9ee6-78ab-460c-8f42-e7db0a8bf632" (UID: "118e9ee6-78ab-460c-8f42-e7db0a8bf632"). InnerVolumeSpecName "run-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:46:38 crc kubenswrapper[5052]: I1203 11:46:38.626867 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/118e9ee6-78ab-460c-8f42-e7db0a8bf632-log-httpd" (OuterVolumeSpecName: "log-httpd") pod "118e9ee6-78ab-460c-8f42-e7db0a8bf632" (UID: "118e9ee6-78ab-460c-8f42-e7db0a8bf632"). InnerVolumeSpecName "log-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:46:38 crc kubenswrapper[5052]: I1203 11:46:38.632047 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/118e9ee6-78ab-460c-8f42-e7db0a8bf632-scripts" (OuterVolumeSpecName: "scripts") pod "118e9ee6-78ab-460c-8f42-e7db0a8bf632" (UID: "118e9ee6-78ab-460c-8f42-e7db0a8bf632"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:46:38 crc kubenswrapper[5052]: I1203 11:46:38.636868 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/118e9ee6-78ab-460c-8f42-e7db0a8bf632-kube-api-access-4mqz2" (OuterVolumeSpecName: "kube-api-access-4mqz2") pod "118e9ee6-78ab-460c-8f42-e7db0a8bf632" (UID: "118e9ee6-78ab-460c-8f42-e7db0a8bf632"). InnerVolumeSpecName "kube-api-access-4mqz2". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:46:38 crc kubenswrapper[5052]: I1203 11:46:38.674606 5052 scope.go:117] "RemoveContainer" containerID="25fe88f8d102f09983f8884e65f0c6c1575ade67921f10a630e6dcc3ea8b506e" Dec 03 11:46:38 crc kubenswrapper[5052]: E1203 11:46:38.675119 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 11:46:38 crc kubenswrapper[5052]: I1203 11:46:38.675154 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/118e9ee6-78ab-460c-8f42-e7db0a8bf632-sg-core-conf-yaml" (OuterVolumeSpecName: "sg-core-conf-yaml") pod "118e9ee6-78ab-460c-8f42-e7db0a8bf632" (UID: "118e9ee6-78ab-460c-8f42-e7db0a8bf632"). InnerVolumeSpecName "sg-core-conf-yaml". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:46:38 crc kubenswrapper[5052]: I1203 11:46:38.685160 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/118e9ee6-78ab-460c-8f42-e7db0a8bf632-ceilometer-tls-certs" (OuterVolumeSpecName: "ceilometer-tls-certs") pod "118e9ee6-78ab-460c-8f42-e7db0a8bf632" (UID: "118e9ee6-78ab-460c-8f42-e7db0a8bf632"). InnerVolumeSpecName "ceilometer-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:46:38 crc kubenswrapper[5052]: I1203 11:46:38.729966 5052 reconciler_common.go:293] "Volume detached for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/118e9ee6-78ab-460c-8f42-e7db0a8bf632-log-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 11:46:38 crc kubenswrapper[5052]: I1203 11:46:38.730008 5052 reconciler_common.go:293] "Volume detached for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/118e9ee6-78ab-460c-8f42-e7db0a8bf632-sg-core-conf-yaml\") on node \"crc\" DevicePath \"\"" Dec 03 11:46:38 crc kubenswrapper[5052]: I1203 11:46:38.730020 5052 reconciler_common.go:293] "Volume detached for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/118e9ee6-78ab-460c-8f42-e7db0a8bf632-ceilometer-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 11:46:38 crc kubenswrapper[5052]: I1203 11:46:38.730030 5052 reconciler_common.go:293] "Volume detached for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/118e9ee6-78ab-460c-8f42-e7db0a8bf632-run-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 11:46:38 crc kubenswrapper[5052]: I1203 11:46:38.730040 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-4mqz2\" (UniqueName: \"kubernetes.io/projected/118e9ee6-78ab-460c-8f42-e7db0a8bf632-kube-api-access-4mqz2\") on node \"crc\" DevicePath \"\"" Dec 03 11:46:38 crc kubenswrapper[5052]: I1203 11:46:38.730047 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/118e9ee6-78ab-460c-8f42-e7db0a8bf632-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:46:38 crc kubenswrapper[5052]: I1203 11:46:38.732376 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/118e9ee6-78ab-460c-8f42-e7db0a8bf632-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "118e9ee6-78ab-460c-8f42-e7db0a8bf632" (UID: "118e9ee6-78ab-460c-8f42-e7db0a8bf632"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:46:38 crc kubenswrapper[5052]: I1203 11:46:38.763489 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/118e9ee6-78ab-460c-8f42-e7db0a8bf632-config-data" (OuterVolumeSpecName: "config-data") pod "118e9ee6-78ab-460c-8f42-e7db0a8bf632" (UID: "118e9ee6-78ab-460c-8f42-e7db0a8bf632"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:46:38 crc kubenswrapper[5052]: I1203 11:46:38.812902 5052 generic.go:334] "Generic (PLEG): container finished" podID="118e9ee6-78ab-460c-8f42-e7db0a8bf632" containerID="654a5e049ba0d38968af8f2a6181dd489b689a0b1ca0a64b88c4e5e9b30fc78e" exitCode=0 Dec 03 11:46:38 crc kubenswrapper[5052]: I1203 11:46:38.813870 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:46:38 crc kubenswrapper[5052]: I1203 11:46:38.813861 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"118e9ee6-78ab-460c-8f42-e7db0a8bf632","Type":"ContainerDied","Data":"654a5e049ba0d38968af8f2a6181dd489b689a0b1ca0a64b88c4e5e9b30fc78e"} Dec 03 11:46:38 crc kubenswrapper[5052]: I1203 11:46:38.814016 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"118e9ee6-78ab-460c-8f42-e7db0a8bf632","Type":"ContainerDied","Data":"6988ff57b302c305166ccabc243593f7e1f0cbc1fb14225d28664100bb2abdd7"} Dec 03 11:46:38 crc kubenswrapper[5052]: I1203 11:46:38.814044 5052 scope.go:117] "RemoveContainer" containerID="85d3d8672f9a85a6e39aadfe82bcb0f9e330f484ac61213e268d33849beb7b60" Dec 03 11:46:38 crc kubenswrapper[5052]: I1203 11:46:38.831360 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/118e9ee6-78ab-460c-8f42-e7db0a8bf632-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:46:38 crc kubenswrapper[5052]: I1203 11:46:38.831576 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/118e9ee6-78ab-460c-8f42-e7db0a8bf632-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:46:38 crc kubenswrapper[5052]: I1203 11:46:38.858423 5052 scope.go:117] "RemoveContainer" containerID="8aa26a6ca34ca18a8274f1330a7ce992bf80788902b95e4f72cd9c387528b7d2" Dec 03 11:46:38 crc kubenswrapper[5052]: I1203 11:46:38.865281 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:46:38 crc kubenswrapper[5052]: I1203 11:46:38.887284 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:46:38 crc kubenswrapper[5052]: I1203 11:46:38.905277 5052 scope.go:117] "RemoveContainer" containerID="654a5e049ba0d38968af8f2a6181dd489b689a0b1ca0a64b88c4e5e9b30fc78e" Dec 03 11:46:38 crc kubenswrapper[5052]: I1203 11:46:38.905458 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:46:38 crc kubenswrapper[5052]: E1203 11:46:38.905968 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="118e9ee6-78ab-460c-8f42-e7db0a8bf632" containerName="ceilometer-notification-agent" Dec 03 11:46:38 crc kubenswrapper[5052]: I1203 11:46:38.905990 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="118e9ee6-78ab-460c-8f42-e7db0a8bf632" containerName="ceilometer-notification-agent" Dec 03 11:46:38 crc kubenswrapper[5052]: E1203 11:46:38.906016 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="118e9ee6-78ab-460c-8f42-e7db0a8bf632" containerName="sg-core" Dec 03 11:46:38 crc kubenswrapper[5052]: I1203 11:46:38.906025 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="118e9ee6-78ab-460c-8f42-e7db0a8bf632" containerName="sg-core" Dec 03 11:46:38 crc kubenswrapper[5052]: E1203 11:46:38.906037 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="118e9ee6-78ab-460c-8f42-e7db0a8bf632" containerName="proxy-httpd" Dec 03 11:46:38 crc kubenswrapper[5052]: I1203 11:46:38.906045 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="118e9ee6-78ab-460c-8f42-e7db0a8bf632" containerName="proxy-httpd" Dec 03 11:46:38 crc kubenswrapper[5052]: E1203 11:46:38.906065 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="118e9ee6-78ab-460c-8f42-e7db0a8bf632" containerName="ceilometer-central-agent" Dec 03 11:46:38 crc kubenswrapper[5052]: I1203 11:46:38.906073 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="118e9ee6-78ab-460c-8f42-e7db0a8bf632" containerName="ceilometer-central-agent" Dec 03 11:46:38 crc kubenswrapper[5052]: I1203 11:46:38.906304 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="118e9ee6-78ab-460c-8f42-e7db0a8bf632" containerName="proxy-httpd" Dec 03 11:46:38 crc kubenswrapper[5052]: I1203 11:46:38.906331 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="118e9ee6-78ab-460c-8f42-e7db0a8bf632" containerName="sg-core" Dec 03 11:46:38 crc kubenswrapper[5052]: I1203 11:46:38.906358 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="118e9ee6-78ab-460c-8f42-e7db0a8bf632" containerName="ceilometer-notification-agent" Dec 03 11:46:38 crc kubenswrapper[5052]: I1203 11:46:38.906372 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="118e9ee6-78ab-460c-8f42-e7db0a8bf632" containerName="ceilometer-central-agent" Dec 03 11:46:38 crc kubenswrapper[5052]: I1203 11:46:38.908373 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:46:38 crc kubenswrapper[5052]: I1203 11:46:38.912704 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-config-data" Dec 03 11:46:38 crc kubenswrapper[5052]: I1203 11:46:38.912777 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-scripts" Dec 03 11:46:38 crc kubenswrapper[5052]: I1203 11:46:38.912881 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-ceilometer-internal-svc" Dec 03 11:46:38 crc kubenswrapper[5052]: I1203 11:46:38.916207 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:46:38 crc kubenswrapper[5052]: I1203 11:46:38.940704 5052 scope.go:117] "RemoveContainer" containerID="0a5f58a28001495eec6556c898240aa664f99a537bf8177e79bd12695098917f" Dec 03 11:46:38 crc kubenswrapper[5052]: I1203 11:46:38.961732 5052 scope.go:117] "RemoveContainer" containerID="85d3d8672f9a85a6e39aadfe82bcb0f9e330f484ac61213e268d33849beb7b60" Dec 03 11:46:38 crc kubenswrapper[5052]: E1203 11:46:38.962364 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"85d3d8672f9a85a6e39aadfe82bcb0f9e330f484ac61213e268d33849beb7b60\": container with ID starting with 85d3d8672f9a85a6e39aadfe82bcb0f9e330f484ac61213e268d33849beb7b60 not found: ID does not exist" containerID="85d3d8672f9a85a6e39aadfe82bcb0f9e330f484ac61213e268d33849beb7b60" Dec 03 11:46:38 crc kubenswrapper[5052]: I1203 11:46:38.962486 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"85d3d8672f9a85a6e39aadfe82bcb0f9e330f484ac61213e268d33849beb7b60"} err="failed to get container status \"85d3d8672f9a85a6e39aadfe82bcb0f9e330f484ac61213e268d33849beb7b60\": rpc error: code = NotFound desc = could not find container \"85d3d8672f9a85a6e39aadfe82bcb0f9e330f484ac61213e268d33849beb7b60\": container with ID starting with 85d3d8672f9a85a6e39aadfe82bcb0f9e330f484ac61213e268d33849beb7b60 not found: ID does not exist" Dec 03 11:46:38 crc kubenswrapper[5052]: I1203 11:46:38.962634 5052 scope.go:117] "RemoveContainer" containerID="8aa26a6ca34ca18a8274f1330a7ce992bf80788902b95e4f72cd9c387528b7d2" Dec 03 11:46:38 crc kubenswrapper[5052]: E1203 11:46:38.963196 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8aa26a6ca34ca18a8274f1330a7ce992bf80788902b95e4f72cd9c387528b7d2\": container with ID starting with 8aa26a6ca34ca18a8274f1330a7ce992bf80788902b95e4f72cd9c387528b7d2 not found: ID does not exist" containerID="8aa26a6ca34ca18a8274f1330a7ce992bf80788902b95e4f72cd9c387528b7d2" Dec 03 11:46:38 crc kubenswrapper[5052]: I1203 11:46:38.963236 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8aa26a6ca34ca18a8274f1330a7ce992bf80788902b95e4f72cd9c387528b7d2"} err="failed to get container status \"8aa26a6ca34ca18a8274f1330a7ce992bf80788902b95e4f72cd9c387528b7d2\": rpc error: code = NotFound desc = could not find container \"8aa26a6ca34ca18a8274f1330a7ce992bf80788902b95e4f72cd9c387528b7d2\": container with ID starting with 8aa26a6ca34ca18a8274f1330a7ce992bf80788902b95e4f72cd9c387528b7d2 not found: ID does not exist" Dec 03 11:46:38 crc kubenswrapper[5052]: I1203 11:46:38.963286 5052 scope.go:117] "RemoveContainer" containerID="654a5e049ba0d38968af8f2a6181dd489b689a0b1ca0a64b88c4e5e9b30fc78e" Dec 03 11:46:38 crc kubenswrapper[5052]: E1203 11:46:38.963799 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"654a5e049ba0d38968af8f2a6181dd489b689a0b1ca0a64b88c4e5e9b30fc78e\": container with ID starting with 654a5e049ba0d38968af8f2a6181dd489b689a0b1ca0a64b88c4e5e9b30fc78e not found: ID does not exist" containerID="654a5e049ba0d38968af8f2a6181dd489b689a0b1ca0a64b88c4e5e9b30fc78e" Dec 03 11:46:38 crc kubenswrapper[5052]: I1203 11:46:38.963833 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"654a5e049ba0d38968af8f2a6181dd489b689a0b1ca0a64b88c4e5e9b30fc78e"} err="failed to get container status \"654a5e049ba0d38968af8f2a6181dd489b689a0b1ca0a64b88c4e5e9b30fc78e\": rpc error: code = NotFound desc = could not find container \"654a5e049ba0d38968af8f2a6181dd489b689a0b1ca0a64b88c4e5e9b30fc78e\": container with ID starting with 654a5e049ba0d38968af8f2a6181dd489b689a0b1ca0a64b88c4e5e9b30fc78e not found: ID does not exist" Dec 03 11:46:38 crc kubenswrapper[5052]: I1203 11:46:38.963860 5052 scope.go:117] "RemoveContainer" containerID="0a5f58a28001495eec6556c898240aa664f99a537bf8177e79bd12695098917f" Dec 03 11:46:38 crc kubenswrapper[5052]: E1203 11:46:38.964169 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0a5f58a28001495eec6556c898240aa664f99a537bf8177e79bd12695098917f\": container with ID starting with 0a5f58a28001495eec6556c898240aa664f99a537bf8177e79bd12695098917f not found: ID does not exist" containerID="0a5f58a28001495eec6556c898240aa664f99a537bf8177e79bd12695098917f" Dec 03 11:46:38 crc kubenswrapper[5052]: I1203 11:46:38.964197 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0a5f58a28001495eec6556c898240aa664f99a537bf8177e79bd12695098917f"} err="failed to get container status \"0a5f58a28001495eec6556c898240aa664f99a537bf8177e79bd12695098917f\": rpc error: code = NotFound desc = could not find container \"0a5f58a28001495eec6556c898240aa664f99a537bf8177e79bd12695098917f\": container with ID starting with 0a5f58a28001495eec6556c898240aa664f99a537bf8177e79bd12695098917f not found: ID does not exist" Dec 03 11:46:39 crc kubenswrapper[5052]: I1203 11:46:39.038555 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/1b45577a-92b7-4c67-941b-cfc2fbfa3eb2-config-data\") pod \"ceilometer-0\" (UID: \"1b45577a-92b7-4c67-941b-cfc2fbfa3eb2\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:46:39 crc kubenswrapper[5052]: I1203 11:46:39.038654 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sgqmd\" (UniqueName: \"kubernetes.io/projected/1b45577a-92b7-4c67-941b-cfc2fbfa3eb2-kube-api-access-sgqmd\") pod \"ceilometer-0\" (UID: \"1b45577a-92b7-4c67-941b-cfc2fbfa3eb2\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:46:39 crc kubenswrapper[5052]: I1203 11:46:39.038725 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/1b45577a-92b7-4c67-941b-cfc2fbfa3eb2-ceilometer-tls-certs\") pod \"ceilometer-0\" (UID: \"1b45577a-92b7-4c67-941b-cfc2fbfa3eb2\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:46:39 crc kubenswrapper[5052]: I1203 11:46:39.039007 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/1b45577a-92b7-4c67-941b-cfc2fbfa3eb2-log-httpd\") pod \"ceilometer-0\" (UID: \"1b45577a-92b7-4c67-941b-cfc2fbfa3eb2\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:46:39 crc kubenswrapper[5052]: I1203 11:46:39.039086 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/1b45577a-92b7-4c67-941b-cfc2fbfa3eb2-scripts\") pod \"ceilometer-0\" (UID: \"1b45577a-92b7-4c67-941b-cfc2fbfa3eb2\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:46:39 crc kubenswrapper[5052]: I1203 11:46:39.039166 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1b45577a-92b7-4c67-941b-cfc2fbfa3eb2-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"1b45577a-92b7-4c67-941b-cfc2fbfa3eb2\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:46:39 crc kubenswrapper[5052]: I1203 11:46:39.039322 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/1b45577a-92b7-4c67-941b-cfc2fbfa3eb2-run-httpd\") pod \"ceilometer-0\" (UID: \"1b45577a-92b7-4c67-941b-cfc2fbfa3eb2\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:46:39 crc kubenswrapper[5052]: I1203 11:46:39.039487 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/1b45577a-92b7-4c67-941b-cfc2fbfa3eb2-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"1b45577a-92b7-4c67-941b-cfc2fbfa3eb2\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:46:39 crc kubenswrapper[5052]: I1203 11:46:39.141359 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/1b45577a-92b7-4c67-941b-cfc2fbfa3eb2-scripts\") pod \"ceilometer-0\" (UID: \"1b45577a-92b7-4c67-941b-cfc2fbfa3eb2\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:46:39 crc kubenswrapper[5052]: I1203 11:46:39.141425 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1b45577a-92b7-4c67-941b-cfc2fbfa3eb2-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"1b45577a-92b7-4c67-941b-cfc2fbfa3eb2\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:46:39 crc kubenswrapper[5052]: I1203 11:46:39.141457 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/1b45577a-92b7-4c67-941b-cfc2fbfa3eb2-run-httpd\") pod \"ceilometer-0\" (UID: \"1b45577a-92b7-4c67-941b-cfc2fbfa3eb2\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:46:39 crc kubenswrapper[5052]: I1203 11:46:39.141483 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/1b45577a-92b7-4c67-941b-cfc2fbfa3eb2-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"1b45577a-92b7-4c67-941b-cfc2fbfa3eb2\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:46:39 crc kubenswrapper[5052]: I1203 11:46:39.141513 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/1b45577a-92b7-4c67-941b-cfc2fbfa3eb2-config-data\") pod \"ceilometer-0\" (UID: \"1b45577a-92b7-4c67-941b-cfc2fbfa3eb2\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:46:39 crc kubenswrapper[5052]: I1203 11:46:39.141568 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-sgqmd\" (UniqueName: \"kubernetes.io/projected/1b45577a-92b7-4c67-941b-cfc2fbfa3eb2-kube-api-access-sgqmd\") pod \"ceilometer-0\" (UID: \"1b45577a-92b7-4c67-941b-cfc2fbfa3eb2\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:46:39 crc kubenswrapper[5052]: I1203 11:46:39.141612 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/1b45577a-92b7-4c67-941b-cfc2fbfa3eb2-ceilometer-tls-certs\") pod \"ceilometer-0\" (UID: \"1b45577a-92b7-4c67-941b-cfc2fbfa3eb2\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:46:39 crc kubenswrapper[5052]: I1203 11:46:39.141665 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/1b45577a-92b7-4c67-941b-cfc2fbfa3eb2-log-httpd\") pod \"ceilometer-0\" (UID: \"1b45577a-92b7-4c67-941b-cfc2fbfa3eb2\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:46:39 crc kubenswrapper[5052]: I1203 11:46:39.142188 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/1b45577a-92b7-4c67-941b-cfc2fbfa3eb2-run-httpd\") pod \"ceilometer-0\" (UID: \"1b45577a-92b7-4c67-941b-cfc2fbfa3eb2\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:46:39 crc kubenswrapper[5052]: I1203 11:46:39.142202 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/1b45577a-92b7-4c67-941b-cfc2fbfa3eb2-log-httpd\") pod \"ceilometer-0\" (UID: \"1b45577a-92b7-4c67-941b-cfc2fbfa3eb2\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:46:39 crc kubenswrapper[5052]: I1203 11:46:39.145553 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/1b45577a-92b7-4c67-941b-cfc2fbfa3eb2-ceilometer-tls-certs\") pod \"ceilometer-0\" (UID: \"1b45577a-92b7-4c67-941b-cfc2fbfa3eb2\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:46:39 crc kubenswrapper[5052]: I1203 11:46:39.145737 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1b45577a-92b7-4c67-941b-cfc2fbfa3eb2-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"1b45577a-92b7-4c67-941b-cfc2fbfa3eb2\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:46:39 crc kubenswrapper[5052]: I1203 11:46:39.145994 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/1b45577a-92b7-4c67-941b-cfc2fbfa3eb2-scripts\") pod \"ceilometer-0\" (UID: \"1b45577a-92b7-4c67-941b-cfc2fbfa3eb2\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:46:39 crc kubenswrapper[5052]: I1203 11:46:39.146877 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/1b45577a-92b7-4c67-941b-cfc2fbfa3eb2-config-data\") pod \"ceilometer-0\" (UID: \"1b45577a-92b7-4c67-941b-cfc2fbfa3eb2\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:46:39 crc kubenswrapper[5052]: I1203 11:46:39.147776 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/1b45577a-92b7-4c67-941b-cfc2fbfa3eb2-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"1b45577a-92b7-4c67-941b-cfc2fbfa3eb2\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:46:39 crc kubenswrapper[5052]: I1203 11:46:39.163149 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-sgqmd\" (UniqueName: \"kubernetes.io/projected/1b45577a-92b7-4c67-941b-cfc2fbfa3eb2-kube-api-access-sgqmd\") pod \"ceilometer-0\" (UID: \"1b45577a-92b7-4c67-941b-cfc2fbfa3eb2\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:46:39 crc kubenswrapper[5052]: I1203 11:46:39.229941 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:46:39 crc kubenswrapper[5052]: I1203 11:46:39.683299 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="118e9ee6-78ab-460c-8f42-e7db0a8bf632" path="/var/lib/kubelet/pods/118e9ee6-78ab-460c-8f42-e7db0a8bf632/volumes" Dec 03 11:46:39 crc kubenswrapper[5052]: I1203 11:46:39.699337 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:46:39 crc kubenswrapper[5052]: I1203 11:46:39.822243 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"1b45577a-92b7-4c67-941b-cfc2fbfa3eb2","Type":"ContainerStarted","Data":"4b13986efc84461e4d35d070b9899c7aaeea80554e4845c1f45b26df10d18678"} Dec 03 11:46:40 crc kubenswrapper[5052]: I1203 11:46:40.537371 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ovn-controller-clb8s"] Dec 03 11:46:40 crc kubenswrapper[5052]: I1203 11:46:40.552784 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovn-controller-clb8s" Dec 03 11:46:40 crc kubenswrapper[5052]: I1203 11:46:40.558454 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ovncontroller-ovncontroller-dockercfg-sk4nt" Dec 03 11:46:40 crc kubenswrapper[5052]: I1203 11:46:40.558570 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-ovncontroller-ovndbs" Dec 03 11:46:40 crc kubenswrapper[5052]: I1203 11:46:40.564144 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"ovncontroller-scripts" Dec 03 11:46:40 crc kubenswrapper[5052]: I1203 11:46:40.569231 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ovn-controller-clb8s"] Dec 03 11:46:40 crc kubenswrapper[5052]: I1203 11:46:40.595611 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ovn-controller-ovs-pk2zp"] Dec 03 11:46:40 crc kubenswrapper[5052]: I1203 11:46:40.598821 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovn-controller-ovs-pk2zp" Dec 03 11:46:40 crc kubenswrapper[5052]: I1203 11:46:40.614958 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ovn-controller-ovs-pk2zp"] Dec 03 11:46:40 crc kubenswrapper[5052]: I1203 11:46:40.631730 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ovn-controller-metrics-h4r6s"] Dec 03 11:46:40 crc kubenswrapper[5052]: I1203 11:46:40.633397 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovn-controller-metrics-h4r6s" Dec 03 11:46:40 crc kubenswrapper[5052]: I1203 11:46:40.638040 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"ovncontroller-metrics-config" Dec 03 11:46:40 crc kubenswrapper[5052]: I1203 11:46:40.658716 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ovn-controller-metrics-h4r6s"] Dec 03 11:46:40 crc kubenswrapper[5052]: I1203 11:46:40.684435 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-controller-tls-certs\" (UniqueName: \"kubernetes.io/secret/0bc28d89-7210-4c40-a72c-3f3903a6ea15-ovn-controller-tls-certs\") pod \"ovn-controller-clb8s\" (UID: \"0bc28d89-7210-4c40-a72c-3f3903a6ea15\") " pod="openstack-kuttl-tests/ovn-controller-clb8s" Dec 03 11:46:40 crc kubenswrapper[5052]: I1203 11:46:40.684482 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sshdr\" (UniqueName: \"kubernetes.io/projected/0bc28d89-7210-4c40-a72c-3f3903a6ea15-kube-api-access-sshdr\") pod \"ovn-controller-clb8s\" (UID: \"0bc28d89-7210-4c40-a72c-3f3903a6ea15\") " pod="openstack-kuttl-tests/ovn-controller-clb8s" Dec 03 11:46:40 crc kubenswrapper[5052]: I1203 11:46:40.684613 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0bc28d89-7210-4c40-a72c-3f3903a6ea15-combined-ca-bundle\") pod \"ovn-controller-clb8s\" (UID: \"0bc28d89-7210-4c40-a72c-3f3903a6ea15\") " pod="openstack-kuttl-tests/ovn-controller-clb8s" Dec 03 11:46:40 crc kubenswrapper[5052]: I1203 11:46:40.684649 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-log-ovn\" (UniqueName: \"kubernetes.io/host-path/0bc28d89-7210-4c40-a72c-3f3903a6ea15-var-log-ovn\") pod \"ovn-controller-clb8s\" (UID: \"0bc28d89-7210-4c40-a72c-3f3903a6ea15\") " pod="openstack-kuttl-tests/ovn-controller-clb8s" Dec 03 11:46:40 crc kubenswrapper[5052]: I1203 11:46:40.684672 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-run\" (UniqueName: \"kubernetes.io/host-path/0bc28d89-7210-4c40-a72c-3f3903a6ea15-var-run\") pod \"ovn-controller-clb8s\" (UID: \"0bc28d89-7210-4c40-a72c-3f3903a6ea15\") " pod="openstack-kuttl-tests/ovn-controller-clb8s" Dec 03 11:46:40 crc kubenswrapper[5052]: I1203 11:46:40.684697 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/0bc28d89-7210-4c40-a72c-3f3903a6ea15-scripts\") pod \"ovn-controller-clb8s\" (UID: \"0bc28d89-7210-4c40-a72c-3f3903a6ea15\") " pod="openstack-kuttl-tests/ovn-controller-clb8s" Dec 03 11:46:40 crc kubenswrapper[5052]: I1203 11:46:40.684729 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-run-ovn\" (UniqueName: \"kubernetes.io/host-path/0bc28d89-7210-4c40-a72c-3f3903a6ea15-var-run-ovn\") pod \"ovn-controller-clb8s\" (UID: \"0bc28d89-7210-4c40-a72c-3f3903a6ea15\") " pod="openstack-kuttl-tests/ovn-controller-clb8s" Dec 03 11:46:40 crc kubenswrapper[5052]: I1203 11:46:40.786987 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dbcdf\" (UniqueName: \"kubernetes.io/projected/84e7baa6-438b-460f-bb8e-3abee427f12b-kube-api-access-dbcdf\") pod \"ovn-controller-metrics-h4r6s\" (UID: \"84e7baa6-438b-460f-bb8e-3abee427f12b\") " pod="openstack-kuttl-tests/ovn-controller-metrics-h4r6s" Dec 03 11:46:40 crc kubenswrapper[5052]: I1203 11:46:40.787027 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib\" (UniqueName: \"kubernetes.io/host-path/d07ce1c1-dd2a-4cb5-b18b-104d1d804fba-var-lib\") pod \"ovn-controller-ovs-pk2zp\" (UID: \"d07ce1c1-dd2a-4cb5-b18b-104d1d804fba\") " pod="openstack-kuttl-tests/ovn-controller-ovs-pk2zp" Dec 03 11:46:40 crc kubenswrapper[5052]: I1203 11:46:40.787048 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-ovs\" (UniqueName: \"kubernetes.io/host-path/d07ce1c1-dd2a-4cb5-b18b-104d1d804fba-etc-ovs\") pod \"ovn-controller-ovs-pk2zp\" (UID: \"d07ce1c1-dd2a-4cb5-b18b-104d1d804fba\") " pod="openstack-kuttl-tests/ovn-controller-ovs-pk2zp" Dec 03 11:46:40 crc kubenswrapper[5052]: I1203 11:46:40.787261 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0bc28d89-7210-4c40-a72c-3f3903a6ea15-combined-ca-bundle\") pod \"ovn-controller-clb8s\" (UID: \"0bc28d89-7210-4c40-a72c-3f3903a6ea15\") " pod="openstack-kuttl-tests/ovn-controller-clb8s" Dec 03 11:46:40 crc kubenswrapper[5052]: I1203 11:46:40.787314 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-run\" (UniqueName: \"kubernetes.io/host-path/d07ce1c1-dd2a-4cb5-b18b-104d1d804fba-var-run\") pod \"ovn-controller-ovs-pk2zp\" (UID: \"d07ce1c1-dd2a-4cb5-b18b-104d1d804fba\") " pod="openstack-kuttl-tests/ovn-controller-ovs-pk2zp" Dec 03 11:46:40 crc kubenswrapper[5052]: I1203 11:46:40.787335 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"var-log-ovn\" (UniqueName: \"kubernetes.io/host-path/0bc28d89-7210-4c40-a72c-3f3903a6ea15-var-log-ovn\") pod \"ovn-controller-clb8s\" (UID: \"0bc28d89-7210-4c40-a72c-3f3903a6ea15\") " pod="openstack-kuttl-tests/ovn-controller-clb8s" Dec 03 11:46:40 crc kubenswrapper[5052]: I1203 11:46:40.787350 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/84e7baa6-438b-460f-bb8e-3abee427f12b-metrics-certs-tls-certs\") pod \"ovn-controller-metrics-h4r6s\" (UID: \"84e7baa6-438b-460f-bb8e-3abee427f12b\") " pod="openstack-kuttl-tests/ovn-controller-metrics-h4r6s" Dec 03 11:46:40 crc kubenswrapper[5052]: I1203 11:46:40.787377 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"var-run\" (UniqueName: \"kubernetes.io/host-path/0bc28d89-7210-4c40-a72c-3f3903a6ea15-var-run\") pod \"ovn-controller-clb8s\" (UID: \"0bc28d89-7210-4c40-a72c-3f3903a6ea15\") " pod="openstack-kuttl-tests/ovn-controller-clb8s" Dec 03 11:46:40 crc kubenswrapper[5052]: I1203 11:46:40.787394 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/84e7baa6-438b-460f-bb8e-3abee427f12b-combined-ca-bundle\") pod \"ovn-controller-metrics-h4r6s\" (UID: \"84e7baa6-438b-460f-bb8e-3abee427f12b\") " pod="openstack-kuttl-tests/ovn-controller-metrics-h4r6s" Dec 03 11:46:40 crc kubenswrapper[5052]: I1203 11:46:40.787418 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovs-rundir\" (UniqueName: \"kubernetes.io/host-path/84e7baa6-438b-460f-bb8e-3abee427f12b-ovs-rundir\") pod \"ovn-controller-metrics-h4r6s\" (UID: \"84e7baa6-438b-460f-bb8e-3abee427f12b\") " pod="openstack-kuttl-tests/ovn-controller-metrics-h4r6s" Dec 03 11:46:40 crc kubenswrapper[5052]: I1203 11:46:40.787436 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/0bc28d89-7210-4c40-a72c-3f3903a6ea15-scripts\") pod \"ovn-controller-clb8s\" (UID: \"0bc28d89-7210-4c40-a72c-3f3903a6ea15\") " pod="openstack-kuttl-tests/ovn-controller-clb8s" Dec 03 11:46:40 crc kubenswrapper[5052]: I1203 11:46:40.787458 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/84e7baa6-438b-460f-bb8e-3abee427f12b-config\") pod \"ovn-controller-metrics-h4r6s\" (UID: \"84e7baa6-438b-460f-bb8e-3abee427f12b\") " pod="openstack-kuttl-tests/ovn-controller-metrics-h4r6s" Dec 03 11:46:40 crc kubenswrapper[5052]: I1203 11:46:40.787476 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/d07ce1c1-dd2a-4cb5-b18b-104d1d804fba-scripts\") pod \"ovn-controller-ovs-pk2zp\" (UID: \"d07ce1c1-dd2a-4cb5-b18b-104d1d804fba\") " pod="openstack-kuttl-tests/ovn-controller-ovs-pk2zp" Dec 03 11:46:40 crc kubenswrapper[5052]: I1203 11:46:40.787493 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-log\" (UniqueName: \"kubernetes.io/host-path/d07ce1c1-dd2a-4cb5-b18b-104d1d804fba-var-log\") pod \"ovn-controller-ovs-pk2zp\" (UID: \"d07ce1c1-dd2a-4cb5-b18b-104d1d804fba\") " pod="openstack-kuttl-tests/ovn-controller-ovs-pk2zp" Dec 03 11:46:40 crc kubenswrapper[5052]: I1203 11:46:40.787519 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"var-run-ovn\" (UniqueName: \"kubernetes.io/host-path/0bc28d89-7210-4c40-a72c-3f3903a6ea15-var-run-ovn\") pod \"ovn-controller-clb8s\" (UID: \"0bc28d89-7210-4c40-a72c-3f3903a6ea15\") " pod="openstack-kuttl-tests/ovn-controller-clb8s" Dec 03 11:46:40 crc kubenswrapper[5052]: I1203 11:46:40.787563 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-rundir\" (UniqueName: \"kubernetes.io/host-path/84e7baa6-438b-460f-bb8e-3abee427f12b-ovn-rundir\") pod \"ovn-controller-metrics-h4r6s\" (UID: \"84e7baa6-438b-460f-bb8e-3abee427f12b\") " pod="openstack-kuttl-tests/ovn-controller-metrics-h4r6s" Dec 03 11:46:40 crc kubenswrapper[5052]: I1203 11:46:40.787604 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovn-controller-tls-certs\" (UniqueName: \"kubernetes.io/secret/0bc28d89-7210-4c40-a72c-3f3903a6ea15-ovn-controller-tls-certs\") pod \"ovn-controller-clb8s\" (UID: \"0bc28d89-7210-4c40-a72c-3f3903a6ea15\") " pod="openstack-kuttl-tests/ovn-controller-clb8s" Dec 03 11:46:40 crc kubenswrapper[5052]: I1203 11:46:40.787620 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xnlsv\" (UniqueName: \"kubernetes.io/projected/d07ce1c1-dd2a-4cb5-b18b-104d1d804fba-kube-api-access-xnlsv\") pod \"ovn-controller-ovs-pk2zp\" (UID: \"d07ce1c1-dd2a-4cb5-b18b-104d1d804fba\") " pod="openstack-kuttl-tests/ovn-controller-ovs-pk2zp" Dec 03 11:46:40 crc kubenswrapper[5052]: I1203 11:46:40.787644 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-sshdr\" (UniqueName: \"kubernetes.io/projected/0bc28d89-7210-4c40-a72c-3f3903a6ea15-kube-api-access-sshdr\") pod \"ovn-controller-clb8s\" (UID: \"0bc28d89-7210-4c40-a72c-3f3903a6ea15\") " pod="openstack-kuttl-tests/ovn-controller-clb8s" Dec 03 11:46:40 crc kubenswrapper[5052]: I1203 11:46:40.789945 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"var-run-ovn\" (UniqueName: \"kubernetes.io/host-path/0bc28d89-7210-4c40-a72c-3f3903a6ea15-var-run-ovn\") pod \"ovn-controller-clb8s\" (UID: \"0bc28d89-7210-4c40-a72c-3f3903a6ea15\") " pod="openstack-kuttl-tests/ovn-controller-clb8s" Dec 03 11:46:40 crc kubenswrapper[5052]: I1203 11:46:40.790777 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/0bc28d89-7210-4c40-a72c-3f3903a6ea15-scripts\") pod \"ovn-controller-clb8s\" (UID: \"0bc28d89-7210-4c40-a72c-3f3903a6ea15\") " pod="openstack-kuttl-tests/ovn-controller-clb8s" Dec 03 11:46:40 crc kubenswrapper[5052]: I1203 11:46:40.792669 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"var-log-ovn\" (UniqueName: \"kubernetes.io/host-path/0bc28d89-7210-4c40-a72c-3f3903a6ea15-var-log-ovn\") pod \"ovn-controller-clb8s\" (UID: \"0bc28d89-7210-4c40-a72c-3f3903a6ea15\") " pod="openstack-kuttl-tests/ovn-controller-clb8s" Dec 03 11:46:40 crc kubenswrapper[5052]: I1203 11:46:40.792746 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"var-run\" (UniqueName: \"kubernetes.io/host-path/0bc28d89-7210-4c40-a72c-3f3903a6ea15-var-run\") pod \"ovn-controller-clb8s\" (UID: \"0bc28d89-7210-4c40-a72c-3f3903a6ea15\") " pod="openstack-kuttl-tests/ovn-controller-clb8s" Dec 03 11:46:40 crc kubenswrapper[5052]: I1203 11:46:40.798977 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovn-controller-tls-certs\" (UniqueName: \"kubernetes.io/secret/0bc28d89-7210-4c40-a72c-3f3903a6ea15-ovn-controller-tls-certs\") pod \"ovn-controller-clb8s\" (UID: \"0bc28d89-7210-4c40-a72c-3f3903a6ea15\") " pod="openstack-kuttl-tests/ovn-controller-clb8s" Dec 03 11:46:40 crc kubenswrapper[5052]: I1203 11:46:40.815469 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-sshdr\" (UniqueName: \"kubernetes.io/projected/0bc28d89-7210-4c40-a72c-3f3903a6ea15-kube-api-access-sshdr\") pod \"ovn-controller-clb8s\" (UID: \"0bc28d89-7210-4c40-a72c-3f3903a6ea15\") " pod="openstack-kuttl-tests/ovn-controller-clb8s" Dec 03 11:46:40 crc kubenswrapper[5052]: I1203 11:46:40.836351 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0bc28d89-7210-4c40-a72c-3f3903a6ea15-combined-ca-bundle\") pod \"ovn-controller-clb8s\" (UID: \"0bc28d89-7210-4c40-a72c-3f3903a6ea15\") " pod="openstack-kuttl-tests/ovn-controller-clb8s" Dec 03 11:46:40 crc kubenswrapper[5052]: I1203 11:46:40.853102 5052 generic.go:334] "Generic (PLEG): container finished" podID="85eff54b-65b2-4877-ac8b-892194af103a" containerID="7e4e52218f80457d114f0a54da0635f7033f1fcd126523398bfe258d9d538373" exitCode=0 Dec 03 11:46:40 crc kubenswrapper[5052]: I1203 11:46:40.853161 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"85eff54b-65b2-4877-ac8b-892194af103a","Type":"ContainerDied","Data":"7e4e52218f80457d114f0a54da0635f7033f1fcd126523398bfe258d9d538373"} Dec 03 11:46:40 crc kubenswrapper[5052]: I1203 11:46:40.892721 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/84e7baa6-438b-460f-bb8e-3abee427f12b-combined-ca-bundle\") pod \"ovn-controller-metrics-h4r6s\" (UID: \"84e7baa6-438b-460f-bb8e-3abee427f12b\") " pod="openstack-kuttl-tests/ovn-controller-metrics-h4r6s" Dec 03 11:46:40 crc kubenswrapper[5052]: I1203 11:46:40.893351 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovs-rundir\" (UniqueName: \"kubernetes.io/host-path/84e7baa6-438b-460f-bb8e-3abee427f12b-ovs-rundir\") pod \"ovn-controller-metrics-h4r6s\" (UID: \"84e7baa6-438b-460f-bb8e-3abee427f12b\") " pod="openstack-kuttl-tests/ovn-controller-metrics-h4r6s" Dec 03 11:46:40 crc kubenswrapper[5052]: I1203 11:46:40.893462 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/84e7baa6-438b-460f-bb8e-3abee427f12b-config\") pod \"ovn-controller-metrics-h4r6s\" (UID: \"84e7baa6-438b-460f-bb8e-3abee427f12b\") " pod="openstack-kuttl-tests/ovn-controller-metrics-h4r6s" Dec 03 11:46:40 crc kubenswrapper[5052]: I1203 11:46:40.893553 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/d07ce1c1-dd2a-4cb5-b18b-104d1d804fba-scripts\") pod \"ovn-controller-ovs-pk2zp\" (UID: \"d07ce1c1-dd2a-4cb5-b18b-104d1d804fba\") " pod="openstack-kuttl-tests/ovn-controller-ovs-pk2zp" Dec 03 11:46:40 crc kubenswrapper[5052]: I1203 11:46:40.893632 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"var-log\" (UniqueName: \"kubernetes.io/host-path/d07ce1c1-dd2a-4cb5-b18b-104d1d804fba-var-log\") pod \"ovn-controller-ovs-pk2zp\" (UID: \"d07ce1c1-dd2a-4cb5-b18b-104d1d804fba\") " pod="openstack-kuttl-tests/ovn-controller-ovs-pk2zp" Dec 03 11:46:40 crc kubenswrapper[5052]: I1203 11:46:40.893548 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovs-rundir\" (UniqueName: \"kubernetes.io/host-path/84e7baa6-438b-460f-bb8e-3abee427f12b-ovs-rundir\") pod \"ovn-controller-metrics-h4r6s\" (UID: \"84e7baa6-438b-460f-bb8e-3abee427f12b\") " pod="openstack-kuttl-tests/ovn-controller-metrics-h4r6s" Dec 03 11:46:40 crc kubenswrapper[5052]: I1203 11:46:40.893823 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovn-rundir\" (UniqueName: \"kubernetes.io/host-path/84e7baa6-438b-460f-bb8e-3abee427f12b-ovn-rundir\") pod \"ovn-controller-metrics-h4r6s\" (UID: \"84e7baa6-438b-460f-bb8e-3abee427f12b\") " pod="openstack-kuttl-tests/ovn-controller-metrics-h4r6s" Dec 03 11:46:40 crc kubenswrapper[5052]: I1203 11:46:40.893980 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-xnlsv\" (UniqueName: \"kubernetes.io/projected/d07ce1c1-dd2a-4cb5-b18b-104d1d804fba-kube-api-access-xnlsv\") pod \"ovn-controller-ovs-pk2zp\" (UID: \"d07ce1c1-dd2a-4cb5-b18b-104d1d804fba\") " pod="openstack-kuttl-tests/ovn-controller-ovs-pk2zp" Dec 03 11:46:40 crc kubenswrapper[5052]: I1203 11:46:40.894215 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/84e7baa6-438b-460f-bb8e-3abee427f12b-config\") pod \"ovn-controller-metrics-h4r6s\" (UID: \"84e7baa6-438b-460f-bb8e-3abee427f12b\") " pod="openstack-kuttl-tests/ovn-controller-metrics-h4r6s" Dec 03 11:46:40 crc kubenswrapper[5052]: I1203 11:46:40.893381 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovn-controller-clb8s" Dec 03 11:46:40 crc kubenswrapper[5052]: I1203 11:46:40.895043 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-dbcdf\" (UniqueName: \"kubernetes.io/projected/84e7baa6-438b-460f-bb8e-3abee427f12b-kube-api-access-dbcdf\") pod \"ovn-controller-metrics-h4r6s\" (UID: \"84e7baa6-438b-460f-bb8e-3abee427f12b\") " pod="openstack-kuttl-tests/ovn-controller-metrics-h4r6s" Dec 03 11:46:40 crc kubenswrapper[5052]: I1203 11:46:40.895138 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"var-lib\" (UniqueName: \"kubernetes.io/host-path/d07ce1c1-dd2a-4cb5-b18b-104d1d804fba-var-lib\") pod \"ovn-controller-ovs-pk2zp\" (UID: \"d07ce1c1-dd2a-4cb5-b18b-104d1d804fba\") " pod="openstack-kuttl-tests/ovn-controller-ovs-pk2zp" Dec 03 11:46:40 crc kubenswrapper[5052]: I1203 11:46:40.895205 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-ovs\" (UniqueName: \"kubernetes.io/host-path/d07ce1c1-dd2a-4cb5-b18b-104d1d804fba-etc-ovs\") pod \"ovn-controller-ovs-pk2zp\" (UID: \"d07ce1c1-dd2a-4cb5-b18b-104d1d804fba\") " pod="openstack-kuttl-tests/ovn-controller-ovs-pk2zp" Dec 03 11:46:40 crc kubenswrapper[5052]: I1203 11:46:40.895369 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"var-run\" (UniqueName: \"kubernetes.io/host-path/d07ce1c1-dd2a-4cb5-b18b-104d1d804fba-var-run\") pod \"ovn-controller-ovs-pk2zp\" (UID: \"d07ce1c1-dd2a-4cb5-b18b-104d1d804fba\") " pod="openstack-kuttl-tests/ovn-controller-ovs-pk2zp" Dec 03 11:46:40 crc kubenswrapper[5052]: I1203 11:46:40.895436 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/84e7baa6-438b-460f-bb8e-3abee427f12b-metrics-certs-tls-certs\") pod \"ovn-controller-metrics-h4r6s\" (UID: \"84e7baa6-438b-460f-bb8e-3abee427f12b\") " pod="openstack-kuttl-tests/ovn-controller-metrics-h4r6s" Dec 03 11:46:40 crc kubenswrapper[5052]: I1203 11:46:40.895657 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/d07ce1c1-dd2a-4cb5-b18b-104d1d804fba-scripts\") pod \"ovn-controller-ovs-pk2zp\" (UID: \"d07ce1c1-dd2a-4cb5-b18b-104d1d804fba\") " pod="openstack-kuttl-tests/ovn-controller-ovs-pk2zp" Dec 03 11:46:40 crc kubenswrapper[5052]: I1203 11:46:40.895750 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"var-log\" (UniqueName: \"kubernetes.io/host-path/d07ce1c1-dd2a-4cb5-b18b-104d1d804fba-var-log\") pod \"ovn-controller-ovs-pk2zp\" (UID: \"d07ce1c1-dd2a-4cb5-b18b-104d1d804fba\") " pod="openstack-kuttl-tests/ovn-controller-ovs-pk2zp" Dec 03 11:46:40 crc kubenswrapper[5052]: I1203 11:46:40.895836 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovn-rundir\" (UniqueName: \"kubernetes.io/host-path/84e7baa6-438b-460f-bb8e-3abee427f12b-ovn-rundir\") pod \"ovn-controller-metrics-h4r6s\" (UID: \"84e7baa6-438b-460f-bb8e-3abee427f12b\") " pod="openstack-kuttl-tests/ovn-controller-metrics-h4r6s" Dec 03 11:46:40 crc kubenswrapper[5052]: I1203 11:46:40.896113 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"var-lib\" (UniqueName: \"kubernetes.io/host-path/d07ce1c1-dd2a-4cb5-b18b-104d1d804fba-var-lib\") pod \"ovn-controller-ovs-pk2zp\" (UID: \"d07ce1c1-dd2a-4cb5-b18b-104d1d804fba\") " pod="openstack-kuttl-tests/ovn-controller-ovs-pk2zp" Dec 03 11:46:40 crc kubenswrapper[5052]: I1203 11:46:40.896396 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"etc-ovs\" (UniqueName: \"kubernetes.io/host-path/d07ce1c1-dd2a-4cb5-b18b-104d1d804fba-etc-ovs\") pod \"ovn-controller-ovs-pk2zp\" (UID: \"d07ce1c1-dd2a-4cb5-b18b-104d1d804fba\") " pod="openstack-kuttl-tests/ovn-controller-ovs-pk2zp" Dec 03 11:46:40 crc kubenswrapper[5052]: I1203 11:46:40.896445 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"var-run\" (UniqueName: \"kubernetes.io/host-path/d07ce1c1-dd2a-4cb5-b18b-104d1d804fba-var-run\") pod \"ovn-controller-ovs-pk2zp\" (UID: \"d07ce1c1-dd2a-4cb5-b18b-104d1d804fba\") " pod="openstack-kuttl-tests/ovn-controller-ovs-pk2zp" Dec 03 11:46:40 crc kubenswrapper[5052]: I1203 11:46:40.898630 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/84e7baa6-438b-460f-bb8e-3abee427f12b-combined-ca-bundle\") pod \"ovn-controller-metrics-h4r6s\" (UID: \"84e7baa6-438b-460f-bb8e-3abee427f12b\") " pod="openstack-kuttl-tests/ovn-controller-metrics-h4r6s" Dec 03 11:46:40 crc kubenswrapper[5052]: I1203 11:46:40.905325 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/84e7baa6-438b-460f-bb8e-3abee427f12b-metrics-certs-tls-certs\") pod \"ovn-controller-metrics-h4r6s\" (UID: \"84e7baa6-438b-460f-bb8e-3abee427f12b\") " pod="openstack-kuttl-tests/ovn-controller-metrics-h4r6s" Dec 03 11:46:40 crc kubenswrapper[5052]: I1203 11:46:40.918174 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-xnlsv\" (UniqueName: \"kubernetes.io/projected/d07ce1c1-dd2a-4cb5-b18b-104d1d804fba-kube-api-access-xnlsv\") pod \"ovn-controller-ovs-pk2zp\" (UID: \"d07ce1c1-dd2a-4cb5-b18b-104d1d804fba\") " pod="openstack-kuttl-tests/ovn-controller-ovs-pk2zp" Dec 03 11:46:40 crc kubenswrapper[5052]: I1203 11:46:40.935200 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-dbcdf\" (UniqueName: \"kubernetes.io/projected/84e7baa6-438b-460f-bb8e-3abee427f12b-kube-api-access-dbcdf\") pod \"ovn-controller-metrics-h4r6s\" (UID: \"84e7baa6-438b-460f-bb8e-3abee427f12b\") " pod="openstack-kuttl-tests/ovn-controller-metrics-h4r6s" Dec 03 11:46:40 crc kubenswrapper[5052]: I1203 11:46:40.962363 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovn-controller-ovs-pk2zp" Dec 03 11:46:40 crc kubenswrapper[5052]: I1203 11:46:40.975227 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovn-controller-metrics-h4r6s" Dec 03 11:46:41 crc kubenswrapper[5052]: I1203 11:46:41.069802 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:46:41 crc kubenswrapper[5052]: I1203 11:46:41.199734 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/85eff54b-65b2-4877-ac8b-892194af103a-config-data\") pod \"85eff54b-65b2-4877-ac8b-892194af103a\" (UID: \"85eff54b-65b2-4877-ac8b-892194af103a\") " Dec 03 11:46:41 crc kubenswrapper[5052]: I1203 11:46:41.200029 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/85eff54b-65b2-4877-ac8b-892194af103a-logs\") pod \"85eff54b-65b2-4877-ac8b-892194af103a\" (UID: \"85eff54b-65b2-4877-ac8b-892194af103a\") " Dec 03 11:46:41 crc kubenswrapper[5052]: I1203 11:46:41.200092 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/85eff54b-65b2-4877-ac8b-892194af103a-combined-ca-bundle\") pod \"85eff54b-65b2-4877-ac8b-892194af103a\" (UID: \"85eff54b-65b2-4877-ac8b-892194af103a\") " Dec 03 11:46:41 crc kubenswrapper[5052]: I1203 11:46:41.200252 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9km47\" (UniqueName: \"kubernetes.io/projected/85eff54b-65b2-4877-ac8b-892194af103a-kube-api-access-9km47\") pod \"85eff54b-65b2-4877-ac8b-892194af103a\" (UID: \"85eff54b-65b2-4877-ac8b-892194af103a\") " Dec 03 11:46:41 crc kubenswrapper[5052]: I1203 11:46:41.201054 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/85eff54b-65b2-4877-ac8b-892194af103a-logs" (OuterVolumeSpecName: "logs") pod "85eff54b-65b2-4877-ac8b-892194af103a" (UID: "85eff54b-65b2-4877-ac8b-892194af103a"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:46:41 crc kubenswrapper[5052]: I1203 11:46:41.204763 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/85eff54b-65b2-4877-ac8b-892194af103a-logs\") on node \"crc\" DevicePath \"\"" Dec 03 11:46:41 crc kubenswrapper[5052]: I1203 11:46:41.211085 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/85eff54b-65b2-4877-ac8b-892194af103a-kube-api-access-9km47" (OuterVolumeSpecName: "kube-api-access-9km47") pod "85eff54b-65b2-4877-ac8b-892194af103a" (UID: "85eff54b-65b2-4877-ac8b-892194af103a"). InnerVolumeSpecName "kube-api-access-9km47". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:46:41 crc kubenswrapper[5052]: I1203 11:46:41.315939 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-9km47\" (UniqueName: \"kubernetes.io/projected/85eff54b-65b2-4877-ac8b-892194af103a-kube-api-access-9km47\") on node \"crc\" DevicePath \"\"" Dec 03 11:46:41 crc kubenswrapper[5052]: I1203 11:46:41.462934 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/85eff54b-65b2-4877-ac8b-892194af103a-config-data" (OuterVolumeSpecName: "config-data") pod "85eff54b-65b2-4877-ac8b-892194af103a" (UID: "85eff54b-65b2-4877-ac8b-892194af103a"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:46:41 crc kubenswrapper[5052]: I1203 11:46:41.472196 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/85eff54b-65b2-4877-ac8b-892194af103a-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "85eff54b-65b2-4877-ac8b-892194af103a" (UID: "85eff54b-65b2-4877-ac8b-892194af103a"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:46:41 crc kubenswrapper[5052]: I1203 11:46:41.521411 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ovn-controller-clb8s"] Dec 03 11:46:41 crc kubenswrapper[5052]: I1203 11:46:41.523094 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/85eff54b-65b2-4877-ac8b-892194af103a-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:46:41 crc kubenswrapper[5052]: I1203 11:46:41.523115 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/85eff54b-65b2-4877-ac8b-892194af103a-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:46:41 crc kubenswrapper[5052]: W1203 11:46:41.619728 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod84e7baa6_438b_460f_bb8e_3abee427f12b.slice/crio-422c2ea063bccfcef61f53f05992e30032a6e3607f0057ede00e6062c1995c29 WatchSource:0}: Error finding container 422c2ea063bccfcef61f53f05992e30032a6e3607f0057ede00e6062c1995c29: Status 404 returned error can't find the container with id 422c2ea063bccfcef61f53f05992e30032a6e3607f0057ede00e6062c1995c29 Dec 03 11:46:41 crc kubenswrapper[5052]: I1203 11:46:41.622581 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ovn-controller-metrics-h4r6s"] Dec 03 11:46:41 crc kubenswrapper[5052]: I1203 11:46:41.861292 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ovn-controller-ovs-pk2zp"] Dec 03 11:46:41 crc kubenswrapper[5052]: I1203 11:46:41.869359 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"85eff54b-65b2-4877-ac8b-892194af103a","Type":"ContainerDied","Data":"82fa539a90a7f5b43c9c1d7d7f512d8c93e1b2d3b4899dd97f447497e2058aa2"} Dec 03 11:46:41 crc kubenswrapper[5052]: I1203 11:46:41.869396 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:46:41 crc kubenswrapper[5052]: I1203 11:46:41.869427 5052 scope.go:117] "RemoveContainer" containerID="7e4e52218f80457d114f0a54da0635f7033f1fcd126523398bfe258d9d538373" Dec 03 11:46:41 crc kubenswrapper[5052]: I1203 11:46:41.870993 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovn-controller-clb8s" event={"ID":"0bc28d89-7210-4c40-a72c-3f3903a6ea15","Type":"ContainerStarted","Data":"4d57527bd0a11c490f63cf209a23a6a0d1fbb245334f1d23b071f18122ac486a"} Dec 03 11:46:41 crc kubenswrapper[5052]: I1203 11:46:41.872235 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovn-controller-metrics-h4r6s" event={"ID":"84e7baa6-438b-460f-bb8e-3abee427f12b","Type":"ContainerStarted","Data":"422c2ea063bccfcef61f53f05992e30032a6e3607f0057ede00e6062c1995c29"} Dec 03 11:46:41 crc kubenswrapper[5052]: I1203 11:46:41.874225 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"1b45577a-92b7-4c67-941b-cfc2fbfa3eb2","Type":"ContainerStarted","Data":"76c303acd28c23268bb01ed30bc5fa462afd9585a2872db7a6ce77014ebbad00"} Dec 03 11:46:41 crc kubenswrapper[5052]: W1203 11:46:41.881170 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd07ce1c1_dd2a_4cb5_b18b_104d1d804fba.slice/crio-8f071e83f1d59f01789ce0b86f2fa626b85326459f4ba2b194b0a32aaccecfa2 WatchSource:0}: Error finding container 8f071e83f1d59f01789ce0b86f2fa626b85326459f4ba2b194b0a32aaccecfa2: Status 404 returned error can't find the container with id 8f071e83f1d59f01789ce0b86f2fa626b85326459f4ba2b194b0a32aaccecfa2 Dec 03 11:46:41 crc kubenswrapper[5052]: I1203 11:46:41.901932 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 11:46:41 crc kubenswrapper[5052]: I1203 11:46:41.906051 5052 scope.go:117] "RemoveContainer" containerID="02b689dd60f694ea5d3055e53d21f12b849a3d9810d0d522c151ad30211151f8" Dec 03 11:46:41 crc kubenswrapper[5052]: I1203 11:46:41.911474 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 11:46:41 crc kubenswrapper[5052]: I1203 11:46:41.928400 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 11:46:41 crc kubenswrapper[5052]: E1203 11:46:41.928882 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="85eff54b-65b2-4877-ac8b-892194af103a" containerName="nova-api-api" Dec 03 11:46:41 crc kubenswrapper[5052]: I1203 11:46:41.928899 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="85eff54b-65b2-4877-ac8b-892194af103a" containerName="nova-api-api" Dec 03 11:46:41 crc kubenswrapper[5052]: E1203 11:46:41.928929 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="85eff54b-65b2-4877-ac8b-892194af103a" containerName="nova-api-log" Dec 03 11:46:41 crc kubenswrapper[5052]: I1203 11:46:41.928936 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="85eff54b-65b2-4877-ac8b-892194af103a" containerName="nova-api-log" Dec 03 11:46:41 crc kubenswrapper[5052]: I1203 11:46:41.929169 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="85eff54b-65b2-4877-ac8b-892194af103a" containerName="nova-api-log" Dec 03 11:46:41 crc kubenswrapper[5052]: I1203 11:46:41.929193 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="85eff54b-65b2-4877-ac8b-892194af103a" containerName="nova-api-api" Dec 03 11:46:41 crc kubenswrapper[5052]: I1203 11:46:41.931699 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:46:41 crc kubenswrapper[5052]: I1203 11:46:41.937487 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-nova-internal-svc" Dec 03 11:46:41 crc kubenswrapper[5052]: I1203 11:46:41.937725 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-nova-public-svc" Dec 03 11:46:41 crc kubenswrapper[5052]: I1203 11:46:41.937860 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-api-config-data" Dec 03 11:46:41 crc kubenswrapper[5052]: I1203 11:46:41.950834 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 11:46:42 crc kubenswrapper[5052]: I1203 11:46:42.033523 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/f7123213-06e1-4b26-9892-c16c3d49c0ab-internal-tls-certs\") pod \"nova-api-0\" (UID: \"f7123213-06e1-4b26-9892-c16c3d49c0ab\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:46:42 crc kubenswrapper[5052]: I1203 11:46:42.033936 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/f7123213-06e1-4b26-9892-c16c3d49c0ab-public-tls-certs\") pod \"nova-api-0\" (UID: \"f7123213-06e1-4b26-9892-c16c3d49c0ab\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:46:42 crc kubenswrapper[5052]: I1203 11:46:42.034054 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/f7123213-06e1-4b26-9892-c16c3d49c0ab-logs\") pod \"nova-api-0\" (UID: \"f7123213-06e1-4b26-9892-c16c3d49c0ab\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:46:42 crc kubenswrapper[5052]: I1203 11:46:42.034074 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/f7123213-06e1-4b26-9892-c16c3d49c0ab-config-data\") pod \"nova-api-0\" (UID: \"f7123213-06e1-4b26-9892-c16c3d49c0ab\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:46:42 crc kubenswrapper[5052]: I1203 11:46:42.034116 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f7123213-06e1-4b26-9892-c16c3d49c0ab-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"f7123213-06e1-4b26-9892-c16c3d49c0ab\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:46:42 crc kubenswrapper[5052]: I1203 11:46:42.034145 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j2dnf\" (UniqueName: \"kubernetes.io/projected/f7123213-06e1-4b26-9892-c16c3d49c0ab-kube-api-access-j2dnf\") pod \"nova-api-0\" (UID: \"f7123213-06e1-4b26-9892-c16c3d49c0ab\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:46:42 crc kubenswrapper[5052]: I1203 11:46:42.136035 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/f7123213-06e1-4b26-9892-c16c3d49c0ab-internal-tls-certs\") pod \"nova-api-0\" (UID: \"f7123213-06e1-4b26-9892-c16c3d49c0ab\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:46:42 crc kubenswrapper[5052]: I1203 11:46:42.136347 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/f7123213-06e1-4b26-9892-c16c3d49c0ab-public-tls-certs\") pod \"nova-api-0\" (UID: \"f7123213-06e1-4b26-9892-c16c3d49c0ab\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:46:42 crc kubenswrapper[5052]: I1203 11:46:42.136452 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/f7123213-06e1-4b26-9892-c16c3d49c0ab-logs\") pod \"nova-api-0\" (UID: \"f7123213-06e1-4b26-9892-c16c3d49c0ab\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:46:42 crc kubenswrapper[5052]: I1203 11:46:42.136479 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/f7123213-06e1-4b26-9892-c16c3d49c0ab-config-data\") pod \"nova-api-0\" (UID: \"f7123213-06e1-4b26-9892-c16c3d49c0ab\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:46:42 crc kubenswrapper[5052]: I1203 11:46:42.136514 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f7123213-06e1-4b26-9892-c16c3d49c0ab-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"f7123213-06e1-4b26-9892-c16c3d49c0ab\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:46:42 crc kubenswrapper[5052]: I1203 11:46:42.136570 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-j2dnf\" (UniqueName: \"kubernetes.io/projected/f7123213-06e1-4b26-9892-c16c3d49c0ab-kube-api-access-j2dnf\") pod \"nova-api-0\" (UID: \"f7123213-06e1-4b26-9892-c16c3d49c0ab\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:46:42 crc kubenswrapper[5052]: I1203 11:46:42.137014 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/f7123213-06e1-4b26-9892-c16c3d49c0ab-logs\") pod \"nova-api-0\" (UID: \"f7123213-06e1-4b26-9892-c16c3d49c0ab\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:46:42 crc kubenswrapper[5052]: I1203 11:46:42.144187 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/f7123213-06e1-4b26-9892-c16c3d49c0ab-public-tls-certs\") pod \"nova-api-0\" (UID: \"f7123213-06e1-4b26-9892-c16c3d49c0ab\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:46:42 crc kubenswrapper[5052]: I1203 11:46:42.144220 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/f7123213-06e1-4b26-9892-c16c3d49c0ab-config-data\") pod \"nova-api-0\" (UID: \"f7123213-06e1-4b26-9892-c16c3d49c0ab\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:46:42 crc kubenswrapper[5052]: I1203 11:46:42.144458 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f7123213-06e1-4b26-9892-c16c3d49c0ab-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"f7123213-06e1-4b26-9892-c16c3d49c0ab\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:46:42 crc kubenswrapper[5052]: I1203 11:46:42.144875 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/f7123213-06e1-4b26-9892-c16c3d49c0ab-internal-tls-certs\") pod \"nova-api-0\" (UID: \"f7123213-06e1-4b26-9892-c16c3d49c0ab\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:46:42 crc kubenswrapper[5052]: I1203 11:46:42.166419 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-j2dnf\" (UniqueName: \"kubernetes.io/projected/f7123213-06e1-4b26-9892-c16c3d49c0ab-kube-api-access-j2dnf\") pod \"nova-api-0\" (UID: \"f7123213-06e1-4b26-9892-c16c3d49c0ab\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:46:42 crc kubenswrapper[5052]: I1203 11:46:42.252825 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:46:42 crc kubenswrapper[5052]: I1203 11:46:42.721344 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 11:46:42 crc kubenswrapper[5052]: I1203 11:46:42.947472 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"f7123213-06e1-4b26-9892-c16c3d49c0ab","Type":"ContainerStarted","Data":"f7781482f3081fcf1252f242cf1ba75ded9204d78cf6aeef6e5e17e22416a3e9"} Dec 03 11:46:42 crc kubenswrapper[5052]: I1203 11:46:42.954389 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovn-controller-metrics-h4r6s" event={"ID":"84e7baa6-438b-460f-bb8e-3abee427f12b","Type":"ContainerStarted","Data":"1f38a6300f951d2085edb95778b8ab8cdb906b0397358a84fbf4a41bc6d06309"} Dec 03 11:46:42 crc kubenswrapper[5052]: I1203 11:46:42.956317 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovn-controller-ovs-pk2zp" event={"ID":"d07ce1c1-dd2a-4cb5-b18b-104d1d804fba","Type":"ContainerStarted","Data":"8f071e83f1d59f01789ce0b86f2fa626b85326459f4ba2b194b0a32aaccecfa2"} Dec 03 11:46:42 crc kubenswrapper[5052]: I1203 11:46:42.988897 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/ovn-controller-metrics-h4r6s" podStartSLOduration=2.988875112 podStartE2EDuration="2.988875112s" podCreationTimestamp="2025-12-03 11:46:40 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:46:42.969940776 +0000 UTC m=+1593.558182560" watchObservedRunningTime="2025-12-03 11:46:42.988875112 +0000 UTC m=+1593.577116876" Dec 03 11:46:43 crc kubenswrapper[5052]: I1203 11:46:43.684709 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="85eff54b-65b2-4877-ac8b-892194af103a" path="/var/lib/kubelet/pods/85eff54b-65b2-4877-ac8b-892194af103a/volumes" Dec 03 11:46:43 crc kubenswrapper[5052]: I1203 11:46:43.966450 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"f7123213-06e1-4b26-9892-c16c3d49c0ab","Type":"ContainerStarted","Data":"fa0c2ba3f86bf1385c8a82da17792deaa7cbb1c9c5250a02c070da47b2664562"} Dec 03 11:46:43 crc kubenswrapper[5052]: I1203 11:46:43.969416 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"1b45577a-92b7-4c67-941b-cfc2fbfa3eb2","Type":"ContainerStarted","Data":"9e68066ce0ef8e9380353738e119a2afc7b38dcd3bd33c671f3778712cfb46e4"} Dec 03 11:46:43 crc kubenswrapper[5052]: I1203 11:46:43.971262 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovn-controller-ovs-pk2zp" event={"ID":"d07ce1c1-dd2a-4cb5-b18b-104d1d804fba","Type":"ContainerStarted","Data":"090548138e69b6166f412957493047690313507ebeaeb842aa53a4bde8c0d141"} Dec 03 11:46:44 crc kubenswrapper[5052]: I1203 11:46:44.982547 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"1b45577a-92b7-4c67-941b-cfc2fbfa3eb2","Type":"ContainerStarted","Data":"05471f2484133fbd796cb8f6c5d16314217c42ed6ab7d7d6511de8a201b11eb9"} Dec 03 11:46:44 crc kubenswrapper[5052]: I1203 11:46:44.984245 5052 generic.go:334] "Generic (PLEG): container finished" podID="d07ce1c1-dd2a-4cb5-b18b-104d1d804fba" containerID="090548138e69b6166f412957493047690313507ebeaeb842aa53a4bde8c0d141" exitCode=0 Dec 03 11:46:44 crc kubenswrapper[5052]: I1203 11:46:44.985192 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovn-controller-ovs-pk2zp" event={"ID":"d07ce1c1-dd2a-4cb5-b18b-104d1d804fba","Type":"ContainerDied","Data":"090548138e69b6166f412957493047690313507ebeaeb842aa53a4bde8c0d141"} Dec 03 11:46:44 crc kubenswrapper[5052]: I1203 11:46:44.988573 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"f7123213-06e1-4b26-9892-c16c3d49c0ab","Type":"ContainerStarted","Data":"25b77fa7d98d8777337c1b92110af89807efa838eb759959d62314939c420ccf"} Dec 03 11:46:44 crc kubenswrapper[5052]: I1203 11:46:44.990899 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovn-controller-clb8s" event={"ID":"0bc28d89-7210-4c40-a72c-3f3903a6ea15","Type":"ContainerStarted","Data":"c1f0fb7eccffe8697552aa5a47d51d47b7e88244cbd383a6c79b3d22fc46417d"} Dec 03 11:46:44 crc kubenswrapper[5052]: I1203 11:46:44.991045 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/ovn-controller-clb8s" Dec 03 11:46:45 crc kubenswrapper[5052]: I1203 11:46:45.046863 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-api-0" podStartSLOduration=4.046839967 podStartE2EDuration="4.046839967s" podCreationTimestamp="2025-12-03 11:46:41 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:46:45.046312992 +0000 UTC m=+1595.634554776" watchObservedRunningTime="2025-12-03 11:46:45.046839967 +0000 UTC m=+1595.635081731" Dec 03 11:46:45 crc kubenswrapper[5052]: I1203 11:46:45.064565 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/ovn-controller-clb8s" podStartSLOduration=3.22909257 podStartE2EDuration="5.064545248s" podCreationTimestamp="2025-12-03 11:46:40 +0000 UTC" firstStartedPulling="2025-12-03 11:46:41.538400195 +0000 UTC m=+1592.126641959" lastFinishedPulling="2025-12-03 11:46:43.373852873 +0000 UTC m=+1593.962094637" observedRunningTime="2025-12-03 11:46:45.062108898 +0000 UTC m=+1595.650350692" watchObservedRunningTime="2025-12-03 11:46:45.064545248 +0000 UTC m=+1595.652787012" Dec 03 11:46:46 crc kubenswrapper[5052]: I1203 11:46:46.014551 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovn-controller-ovs-pk2zp" event={"ID":"d07ce1c1-dd2a-4cb5-b18b-104d1d804fba","Type":"ContainerStarted","Data":"56fbc866f43e285800b8f191573fefd9bb26eac5669a06e69f010ab712844416"} Dec 03 11:46:46 crc kubenswrapper[5052]: I1203 11:46:46.015892 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovn-controller-ovs-pk2zp" event={"ID":"d07ce1c1-dd2a-4cb5-b18b-104d1d804fba","Type":"ContainerStarted","Data":"18fb02d514e2e0b27dc1ec397b2d8667f545277e140c9896331636baa57b550a"} Dec 03 11:46:46 crc kubenswrapper[5052]: I1203 11:46:46.015926 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/ovn-controller-ovs-pk2zp" Dec 03 11:46:46 crc kubenswrapper[5052]: I1203 11:46:46.015943 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/ovn-controller-ovs-pk2zp" Dec 03 11:46:46 crc kubenswrapper[5052]: I1203 11:46:46.045631 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/ovn-controller-ovs-pk2zp" podStartSLOduration=5.131436727 podStartE2EDuration="6.045604588s" podCreationTimestamp="2025-12-03 11:46:40 +0000 UTC" firstStartedPulling="2025-12-03 11:46:41.885231577 +0000 UTC m=+1592.473473341" lastFinishedPulling="2025-12-03 11:46:42.799399438 +0000 UTC m=+1593.387641202" observedRunningTime="2025-12-03 11:46:46.041382496 +0000 UTC m=+1596.629624260" watchObservedRunningTime="2025-12-03 11:46:46.045604588 +0000 UTC m=+1596.633846362" Dec 03 11:46:47 crc kubenswrapper[5052]: I1203 11:46:47.026769 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"1b45577a-92b7-4c67-941b-cfc2fbfa3eb2","Type":"ContainerStarted","Data":"8c71cc229cea36e3361a6c384c5386d5cc260814f0041348a439548161af1249"} Dec 03 11:46:47 crc kubenswrapper[5052]: I1203 11:46:47.027111 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:46:47 crc kubenswrapper[5052]: I1203 11:46:47.054601 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/ceilometer-0" podStartSLOduration=2.647691451 podStartE2EDuration="9.054581074s" podCreationTimestamp="2025-12-03 11:46:38 +0000 UTC" firstStartedPulling="2025-12-03 11:46:39.714717036 +0000 UTC m=+1590.302958810" lastFinishedPulling="2025-12-03 11:46:46.121606669 +0000 UTC m=+1596.709848433" observedRunningTime="2025-12-03 11:46:47.051860725 +0000 UTC m=+1597.640102489" watchObservedRunningTime="2025-12-03 11:46:47.054581074 +0000 UTC m=+1597.642822848" Dec 03 11:46:47 crc kubenswrapper[5052]: I1203 11:46:47.526678 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ovn-controller-ovs-pk2zp"] Dec 03 11:46:47 crc kubenswrapper[5052]: I1203 11:46:47.538127 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ovn-controller-clb8s"] Dec 03 11:46:47 crc kubenswrapper[5052]: I1203 11:46:47.541184 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ovn-controller-metrics-h4r6s"] Dec 03 11:46:47 crc kubenswrapper[5052]: I1203 11:46:47.541398 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ovn-controller-metrics-h4r6s" podUID="84e7baa6-438b-460f-bb8e-3abee427f12b" containerName="openstack-network-exporter" containerID="cri-o://1f38a6300f951d2085edb95778b8ab8cdb906b0397358a84fbf4a41bc6d06309" gracePeriod=30 Dec 03 11:46:47 crc kubenswrapper[5052]: E1203 11:46:47.640308 5052 handlers.go:78] "Exec lifecycle hook for Container in Pod failed" err="command '/usr/share/ovn/scripts/ovn-ctl stop_controller' exited with 137: " execCommand=["/usr/share/ovn/scripts/ovn-ctl","stop_controller"] containerName="ovn-controller" pod="openstack-kuttl-tests/ovn-controller-clb8s" message=< Dec 03 11:46:47 crc kubenswrapper[5052]: Exiting ovn-controller (1) [ OK ] Dec 03 11:46:47 crc kubenswrapper[5052]: > Dec 03 11:46:47 crc kubenswrapper[5052]: E1203 11:46:47.640368 5052 kuberuntime_container.go:691] "PreStop hook failed" err="command '/usr/share/ovn/scripts/ovn-ctl stop_controller' exited with 137: " pod="openstack-kuttl-tests/ovn-controller-clb8s" podUID="0bc28d89-7210-4c40-a72c-3f3903a6ea15" containerName="ovn-controller" containerID="cri-o://c1f0fb7eccffe8697552aa5a47d51d47b7e88244cbd383a6c79b3d22fc46417d" Dec 03 11:46:47 crc kubenswrapper[5052]: I1203 11:46:47.640407 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ovn-controller-clb8s" podUID="0bc28d89-7210-4c40-a72c-3f3903a6ea15" containerName="ovn-controller" containerID="cri-o://c1f0fb7eccffe8697552aa5a47d51d47b7e88244cbd383a6c79b3d22fc46417d" gracePeriod=30 Dec 03 11:46:48 crc kubenswrapper[5052]: I1203 11:46:48.047563 5052 generic.go:334] "Generic (PLEG): container finished" podID="0bc28d89-7210-4c40-a72c-3f3903a6ea15" containerID="c1f0fb7eccffe8697552aa5a47d51d47b7e88244cbd383a6c79b3d22fc46417d" exitCode=0 Dec 03 11:46:48 crc kubenswrapper[5052]: I1203 11:46:48.047893 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovn-controller-clb8s" event={"ID":"0bc28d89-7210-4c40-a72c-3f3903a6ea15","Type":"ContainerDied","Data":"c1f0fb7eccffe8697552aa5a47d51d47b7e88244cbd383a6c79b3d22fc46417d"} Dec 03 11:46:48 crc kubenswrapper[5052]: I1203 11:46:48.047926 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovn-controller-clb8s" event={"ID":"0bc28d89-7210-4c40-a72c-3f3903a6ea15","Type":"ContainerDied","Data":"4d57527bd0a11c490f63cf209a23a6a0d1fbb245334f1d23b071f18122ac486a"} Dec 03 11:46:48 crc kubenswrapper[5052]: I1203 11:46:48.047940 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4d57527bd0a11c490f63cf209a23a6a0d1fbb245334f1d23b071f18122ac486a" Dec 03 11:46:48 crc kubenswrapper[5052]: I1203 11:46:48.050852 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_ovn-controller-metrics-h4r6s_84e7baa6-438b-460f-bb8e-3abee427f12b/openstack-network-exporter/0.log" Dec 03 11:46:48 crc kubenswrapper[5052]: I1203 11:46:48.050890 5052 generic.go:334] "Generic (PLEG): container finished" podID="84e7baa6-438b-460f-bb8e-3abee427f12b" containerID="1f38a6300f951d2085edb95778b8ab8cdb906b0397358a84fbf4a41bc6d06309" exitCode=2 Dec 03 11:46:48 crc kubenswrapper[5052]: I1203 11:46:48.051190 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovn-controller-metrics-h4r6s" event={"ID":"84e7baa6-438b-460f-bb8e-3abee427f12b","Type":"ContainerDied","Data":"1f38a6300f951d2085edb95778b8ab8cdb906b0397358a84fbf4a41bc6d06309"} Dec 03 11:46:48 crc kubenswrapper[5052]: I1203 11:46:48.160258 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovn-controller-clb8s" Dec 03 11:46:48 crc kubenswrapper[5052]: I1203 11:46:48.168475 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_ovn-controller-metrics-h4r6s_84e7baa6-438b-460f-bb8e-3abee427f12b/openstack-network-exporter/0.log" Dec 03 11:46:48 crc kubenswrapper[5052]: I1203 11:46:48.168572 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovn-controller-metrics-h4r6s" Dec 03 11:46:48 crc kubenswrapper[5052]: I1203 11:46:48.258296 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0bc28d89-7210-4c40-a72c-3f3903a6ea15-combined-ca-bundle\") pod \"0bc28d89-7210-4c40-a72c-3f3903a6ea15\" (UID: \"0bc28d89-7210-4c40-a72c-3f3903a6ea15\") " Dec 03 11:46:48 crc kubenswrapper[5052]: I1203 11:46:48.258626 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dbcdf\" (UniqueName: \"kubernetes.io/projected/84e7baa6-438b-460f-bb8e-3abee427f12b-kube-api-access-dbcdf\") pod \"84e7baa6-438b-460f-bb8e-3abee427f12b\" (UID: \"84e7baa6-438b-460f-bb8e-3abee427f12b\") " Dec 03 11:46:48 crc kubenswrapper[5052]: I1203 11:46:48.258775 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovn-controller-tls-certs\" (UniqueName: \"kubernetes.io/secret/0bc28d89-7210-4c40-a72c-3f3903a6ea15-ovn-controller-tls-certs\") pod \"0bc28d89-7210-4c40-a72c-3f3903a6ea15\" (UID: \"0bc28d89-7210-4c40-a72c-3f3903a6ea15\") " Dec 03 11:46:48 crc kubenswrapper[5052]: I1203 11:46:48.258887 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/0bc28d89-7210-4c40-a72c-3f3903a6ea15-scripts\") pod \"0bc28d89-7210-4c40-a72c-3f3903a6ea15\" (UID: \"0bc28d89-7210-4c40-a72c-3f3903a6ea15\") " Dec 03 11:46:48 crc kubenswrapper[5052]: I1203 11:46:48.259011 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/84e7baa6-438b-460f-bb8e-3abee427f12b-metrics-certs-tls-certs\") pod \"84e7baa6-438b-460f-bb8e-3abee427f12b\" (UID: \"84e7baa6-438b-460f-bb8e-3abee427f12b\") " Dec 03 11:46:48 crc kubenswrapper[5052]: I1203 11:46:48.259201 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/84e7baa6-438b-460f-bb8e-3abee427f12b-combined-ca-bundle\") pod \"84e7baa6-438b-460f-bb8e-3abee427f12b\" (UID: \"84e7baa6-438b-460f-bb8e-3abee427f12b\") " Dec 03 11:46:48 crc kubenswrapper[5052]: I1203 11:46:48.259292 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovs-rundir\" (UniqueName: \"kubernetes.io/host-path/84e7baa6-438b-460f-bb8e-3abee427f12b-ovs-rundir\") pod \"84e7baa6-438b-460f-bb8e-3abee427f12b\" (UID: \"84e7baa6-438b-460f-bb8e-3abee427f12b\") " Dec 03 11:46:48 crc kubenswrapper[5052]: I1203 11:46:48.259399 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"var-log-ovn\" (UniqueName: \"kubernetes.io/host-path/0bc28d89-7210-4c40-a72c-3f3903a6ea15-var-log-ovn\") pod \"0bc28d89-7210-4c40-a72c-3f3903a6ea15\" (UID: \"0bc28d89-7210-4c40-a72c-3f3903a6ea15\") " Dec 03 11:46:48 crc kubenswrapper[5052]: I1203 11:46:48.259499 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/84e7baa6-438b-460f-bb8e-3abee427f12b-config\") pod \"84e7baa6-438b-460f-bb8e-3abee427f12b\" (UID: \"84e7baa6-438b-460f-bb8e-3abee427f12b\") " Dec 03 11:46:48 crc kubenswrapper[5052]: I1203 11:46:48.259634 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"var-run\" (UniqueName: \"kubernetes.io/host-path/0bc28d89-7210-4c40-a72c-3f3903a6ea15-var-run\") pod \"0bc28d89-7210-4c40-a72c-3f3903a6ea15\" (UID: \"0bc28d89-7210-4c40-a72c-3f3903a6ea15\") " Dec 03 11:46:48 crc kubenswrapper[5052]: I1203 11:46:48.259787 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"var-run-ovn\" (UniqueName: \"kubernetes.io/host-path/0bc28d89-7210-4c40-a72c-3f3903a6ea15-var-run-ovn\") pod \"0bc28d89-7210-4c40-a72c-3f3903a6ea15\" (UID: \"0bc28d89-7210-4c40-a72c-3f3903a6ea15\") " Dec 03 11:46:48 crc kubenswrapper[5052]: I1203 11:46:48.259914 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sshdr\" (UniqueName: \"kubernetes.io/projected/0bc28d89-7210-4c40-a72c-3f3903a6ea15-kube-api-access-sshdr\") pod \"0bc28d89-7210-4c40-a72c-3f3903a6ea15\" (UID: \"0bc28d89-7210-4c40-a72c-3f3903a6ea15\") " Dec 03 11:46:48 crc kubenswrapper[5052]: I1203 11:46:48.261273 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovn-rundir\" (UniqueName: \"kubernetes.io/host-path/84e7baa6-438b-460f-bb8e-3abee427f12b-ovn-rundir\") pod \"84e7baa6-438b-460f-bb8e-3abee427f12b\" (UID: \"84e7baa6-438b-460f-bb8e-3abee427f12b\") " Dec 03 11:46:48 crc kubenswrapper[5052]: I1203 11:46:48.260592 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/0bc28d89-7210-4c40-a72c-3f3903a6ea15-var-run" (OuterVolumeSpecName: "var-run") pod "0bc28d89-7210-4c40-a72c-3f3903a6ea15" (UID: "0bc28d89-7210-4c40-a72c-3f3903a6ea15"). InnerVolumeSpecName "var-run". PluginName "kubernetes.io/host-path", VolumeGidValue "" Dec 03 11:46:48 crc kubenswrapper[5052]: I1203 11:46:48.260652 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/0bc28d89-7210-4c40-a72c-3f3903a6ea15-var-run-ovn" (OuterVolumeSpecName: "var-run-ovn") pod "0bc28d89-7210-4c40-a72c-3f3903a6ea15" (UID: "0bc28d89-7210-4c40-a72c-3f3903a6ea15"). InnerVolumeSpecName "var-run-ovn". PluginName "kubernetes.io/host-path", VolumeGidValue "" Dec 03 11:46:48 crc kubenswrapper[5052]: I1203 11:46:48.260680 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/84e7baa6-438b-460f-bb8e-3abee427f12b-ovs-rundir" (OuterVolumeSpecName: "ovs-rundir") pod "84e7baa6-438b-460f-bb8e-3abee427f12b" (UID: "84e7baa6-438b-460f-bb8e-3abee427f12b"). InnerVolumeSpecName "ovs-rundir". PluginName "kubernetes.io/host-path", VolumeGidValue "" Dec 03 11:46:48 crc kubenswrapper[5052]: I1203 11:46:48.260706 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/0bc28d89-7210-4c40-a72c-3f3903a6ea15-var-log-ovn" (OuterVolumeSpecName: "var-log-ovn") pod "0bc28d89-7210-4c40-a72c-3f3903a6ea15" (UID: "0bc28d89-7210-4c40-a72c-3f3903a6ea15"). InnerVolumeSpecName "var-log-ovn". PluginName "kubernetes.io/host-path", VolumeGidValue "" Dec 03 11:46:48 crc kubenswrapper[5052]: I1203 11:46:48.262006 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/84e7baa6-438b-460f-bb8e-3abee427f12b-ovn-rundir" (OuterVolumeSpecName: "ovn-rundir") pod "84e7baa6-438b-460f-bb8e-3abee427f12b" (UID: "84e7baa6-438b-460f-bb8e-3abee427f12b"). InnerVolumeSpecName "ovn-rundir". PluginName "kubernetes.io/host-path", VolumeGidValue "" Dec 03 11:46:48 crc kubenswrapper[5052]: I1203 11:46:48.265008 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/84e7baa6-438b-460f-bb8e-3abee427f12b-config" (OuterVolumeSpecName: "config") pod "84e7baa6-438b-460f-bb8e-3abee427f12b" (UID: "84e7baa6-438b-460f-bb8e-3abee427f12b"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:46:48 crc kubenswrapper[5052]: I1203 11:46:48.264693 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0bc28d89-7210-4c40-a72c-3f3903a6ea15-scripts" (OuterVolumeSpecName: "scripts") pod "0bc28d89-7210-4c40-a72c-3f3903a6ea15" (UID: "0bc28d89-7210-4c40-a72c-3f3903a6ea15"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:46:48 crc kubenswrapper[5052]: I1203 11:46:48.267286 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/84e7baa6-438b-460f-bb8e-3abee427f12b-kube-api-access-dbcdf" (OuterVolumeSpecName: "kube-api-access-dbcdf") pod "84e7baa6-438b-460f-bb8e-3abee427f12b" (UID: "84e7baa6-438b-460f-bb8e-3abee427f12b"). InnerVolumeSpecName "kube-api-access-dbcdf". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:46:48 crc kubenswrapper[5052]: I1203 11:46:48.270169 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0bc28d89-7210-4c40-a72c-3f3903a6ea15-kube-api-access-sshdr" (OuterVolumeSpecName: "kube-api-access-sshdr") pod "0bc28d89-7210-4c40-a72c-3f3903a6ea15" (UID: "0bc28d89-7210-4c40-a72c-3f3903a6ea15"). InnerVolumeSpecName "kube-api-access-sshdr". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:46:48 crc kubenswrapper[5052]: I1203 11:46:48.305158 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0bc28d89-7210-4c40-a72c-3f3903a6ea15-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "0bc28d89-7210-4c40-a72c-3f3903a6ea15" (UID: "0bc28d89-7210-4c40-a72c-3f3903a6ea15"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:46:48 crc kubenswrapper[5052]: I1203 11:46:48.314673 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/84e7baa6-438b-460f-bb8e-3abee427f12b-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "84e7baa6-438b-460f-bb8e-3abee427f12b" (UID: "84e7baa6-438b-460f-bb8e-3abee427f12b"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:46:48 crc kubenswrapper[5052]: I1203 11:46:48.356244 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0bc28d89-7210-4c40-a72c-3f3903a6ea15-ovn-controller-tls-certs" (OuterVolumeSpecName: "ovn-controller-tls-certs") pod "0bc28d89-7210-4c40-a72c-3f3903a6ea15" (UID: "0bc28d89-7210-4c40-a72c-3f3903a6ea15"). InnerVolumeSpecName "ovn-controller-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:46:48 crc kubenswrapper[5052]: I1203 11:46:48.363653 5052 reconciler_common.go:293] "Volume detached for volume \"var-run\" (UniqueName: \"kubernetes.io/host-path/0bc28d89-7210-4c40-a72c-3f3903a6ea15-var-run\") on node \"crc\" DevicePath \"\"" Dec 03 11:46:48 crc kubenswrapper[5052]: I1203 11:46:48.363684 5052 reconciler_common.go:293] "Volume detached for volume \"var-run-ovn\" (UniqueName: \"kubernetes.io/host-path/0bc28d89-7210-4c40-a72c-3f3903a6ea15-var-run-ovn\") on node \"crc\" DevicePath \"\"" Dec 03 11:46:48 crc kubenswrapper[5052]: I1203 11:46:48.363696 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-sshdr\" (UniqueName: \"kubernetes.io/projected/0bc28d89-7210-4c40-a72c-3f3903a6ea15-kube-api-access-sshdr\") on node \"crc\" DevicePath \"\"" Dec 03 11:46:48 crc kubenswrapper[5052]: I1203 11:46:48.363706 5052 reconciler_common.go:293] "Volume detached for volume \"ovn-rundir\" (UniqueName: \"kubernetes.io/host-path/84e7baa6-438b-460f-bb8e-3abee427f12b-ovn-rundir\") on node \"crc\" DevicePath \"\"" Dec 03 11:46:48 crc kubenswrapper[5052]: I1203 11:46:48.363721 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0bc28d89-7210-4c40-a72c-3f3903a6ea15-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:46:48 crc kubenswrapper[5052]: I1203 11:46:48.363730 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-dbcdf\" (UniqueName: \"kubernetes.io/projected/84e7baa6-438b-460f-bb8e-3abee427f12b-kube-api-access-dbcdf\") on node \"crc\" DevicePath \"\"" Dec 03 11:46:48 crc kubenswrapper[5052]: I1203 11:46:48.363739 5052 reconciler_common.go:293] "Volume detached for volume \"ovn-controller-tls-certs\" (UniqueName: \"kubernetes.io/secret/0bc28d89-7210-4c40-a72c-3f3903a6ea15-ovn-controller-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 11:46:48 crc kubenswrapper[5052]: I1203 11:46:48.363747 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/0bc28d89-7210-4c40-a72c-3f3903a6ea15-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:46:48 crc kubenswrapper[5052]: I1203 11:46:48.363755 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/84e7baa6-438b-460f-bb8e-3abee427f12b-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:46:48 crc kubenswrapper[5052]: I1203 11:46:48.363764 5052 reconciler_common.go:293] "Volume detached for volume \"ovs-rundir\" (UniqueName: \"kubernetes.io/host-path/84e7baa6-438b-460f-bb8e-3abee427f12b-ovs-rundir\") on node \"crc\" DevicePath \"\"" Dec 03 11:46:48 crc kubenswrapper[5052]: I1203 11:46:48.363774 5052 reconciler_common.go:293] "Volume detached for volume \"var-log-ovn\" (UniqueName: \"kubernetes.io/host-path/0bc28d89-7210-4c40-a72c-3f3903a6ea15-var-log-ovn\") on node \"crc\" DevicePath \"\"" Dec 03 11:46:48 crc kubenswrapper[5052]: I1203 11:46:48.363782 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/84e7baa6-438b-460f-bb8e-3abee427f12b-config\") on node \"crc\" DevicePath \"\"" Dec 03 11:46:48 crc kubenswrapper[5052]: I1203 11:46:48.377015 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/84e7baa6-438b-460f-bb8e-3abee427f12b-metrics-certs-tls-certs" (OuterVolumeSpecName: "metrics-certs-tls-certs") pod "84e7baa6-438b-460f-bb8e-3abee427f12b" (UID: "84e7baa6-438b-460f-bb8e-3abee427f12b"). InnerVolumeSpecName "metrics-certs-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:46:48 crc kubenswrapper[5052]: I1203 11:46:48.444888 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ovn-controller-ovs-pk2zp" podUID="d07ce1c1-dd2a-4cb5-b18b-104d1d804fba" containerName="ovs-vswitchd" containerID="cri-o://56fbc866f43e285800b8f191573fefd9bb26eac5669a06e69f010ab712844416" gracePeriod=30 Dec 03 11:46:48 crc kubenswrapper[5052]: I1203 11:46:48.464889 5052 reconciler_common.go:293] "Volume detached for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/84e7baa6-438b-460f-bb8e-3abee427f12b-metrics-certs-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 11:46:48 crc kubenswrapper[5052]: E1203 11:46:48.674611 5052 handlers.go:78] "Exec lifecycle hook for Container in Pod failed" err=< Dec 03 11:46:48 crc kubenswrapper[5052]: command '/usr/local/bin/container-scripts/stop-ovsdb-server.sh' exited with 137: ++ dirname /usr/local/bin/container-scripts/stop-ovsdb-server.sh Dec 03 11:46:48 crc kubenswrapper[5052]: + source /usr/local/bin/container-scripts/functions Dec 03 11:46:48 crc kubenswrapper[5052]: ++ OVNBridge=br-int Dec 03 11:46:48 crc kubenswrapper[5052]: ++ OVNRemote=tcp:localhost:6642 Dec 03 11:46:48 crc kubenswrapper[5052]: ++ OVNEncapType=geneve Dec 03 11:46:48 crc kubenswrapper[5052]: ++ OVNAvailabilityZones= Dec 03 11:46:48 crc kubenswrapper[5052]: ++ EnableChassisAsGateway=true Dec 03 11:46:48 crc kubenswrapper[5052]: ++ PhysicalNetworks= Dec 03 11:46:48 crc kubenswrapper[5052]: ++ OVNHostName= Dec 03 11:46:48 crc kubenswrapper[5052]: ++ DB_FILE=/etc/openvswitch/conf.db Dec 03 11:46:48 crc kubenswrapper[5052]: ++ ovs_dir=/var/lib/openvswitch Dec 03 11:46:48 crc kubenswrapper[5052]: ++ FLOWS_RESTORE_SCRIPT=/var/lib/openvswitch/flows-script Dec 03 11:46:48 crc kubenswrapper[5052]: ++ FLOWS_RESTORE_DIR=/var/lib/openvswitch/saved-flows Dec 03 11:46:48 crc kubenswrapper[5052]: ++ SAFE_TO_STOP_OVSDB_SERVER_SEMAPHORE=/var/lib/openvswitch/is_safe_to_stop_ovsdb_server Dec 03 11:46:48 crc kubenswrapper[5052]: + '[' '!' -f /var/lib/openvswitch/is_safe_to_stop_ovsdb_server ']' Dec 03 11:46:48 crc kubenswrapper[5052]: + sleep 0.5 Dec 03 11:46:48 crc kubenswrapper[5052]: + '[' '!' -f /var/lib/openvswitch/is_safe_to_stop_ovsdb_server ']' Dec 03 11:46:48 crc kubenswrapper[5052]: + cleanup_ovsdb_server_semaphore Dec 03 11:46:48 crc kubenswrapper[5052]: + rm -f /var/lib/openvswitch/is_safe_to_stop_ovsdb_server Dec 03 11:46:48 crc kubenswrapper[5052]: + /usr/share/openvswitch/scripts/ovs-ctl stop --no-ovs-vswitchd Dec 03 11:46:48 crc kubenswrapper[5052]: > execCommand=["/usr/local/bin/container-scripts/stop-ovsdb-server.sh"] containerName="ovsdb-server" pod="openstack-kuttl-tests/ovn-controller-ovs-pk2zp" message=< Dec 03 11:46:48 crc kubenswrapper[5052]: Exiting ovsdb-server (5) [ OK ] Dec 03 11:46:48 crc kubenswrapper[5052]: ++ dirname /usr/local/bin/container-scripts/stop-ovsdb-server.sh Dec 03 11:46:48 crc kubenswrapper[5052]: + source /usr/local/bin/container-scripts/functions Dec 03 11:46:48 crc kubenswrapper[5052]: ++ OVNBridge=br-int Dec 03 11:46:48 crc kubenswrapper[5052]: ++ OVNRemote=tcp:localhost:6642 Dec 03 11:46:48 crc kubenswrapper[5052]: ++ OVNEncapType=geneve Dec 03 11:46:48 crc kubenswrapper[5052]: ++ OVNAvailabilityZones= Dec 03 11:46:48 crc kubenswrapper[5052]: ++ EnableChassisAsGateway=true Dec 03 11:46:48 crc kubenswrapper[5052]: ++ PhysicalNetworks= Dec 03 11:46:48 crc kubenswrapper[5052]: ++ OVNHostName= Dec 03 11:46:48 crc kubenswrapper[5052]: ++ DB_FILE=/etc/openvswitch/conf.db Dec 03 11:46:48 crc kubenswrapper[5052]: ++ ovs_dir=/var/lib/openvswitch Dec 03 11:46:48 crc kubenswrapper[5052]: ++ FLOWS_RESTORE_SCRIPT=/var/lib/openvswitch/flows-script Dec 03 11:46:48 crc kubenswrapper[5052]: ++ FLOWS_RESTORE_DIR=/var/lib/openvswitch/saved-flows Dec 03 11:46:48 crc kubenswrapper[5052]: ++ SAFE_TO_STOP_OVSDB_SERVER_SEMAPHORE=/var/lib/openvswitch/is_safe_to_stop_ovsdb_server Dec 03 11:46:48 crc kubenswrapper[5052]: + '[' '!' -f /var/lib/openvswitch/is_safe_to_stop_ovsdb_server ']' Dec 03 11:46:48 crc kubenswrapper[5052]: + sleep 0.5 Dec 03 11:46:48 crc kubenswrapper[5052]: + '[' '!' -f /var/lib/openvswitch/is_safe_to_stop_ovsdb_server ']' Dec 03 11:46:48 crc kubenswrapper[5052]: + cleanup_ovsdb_server_semaphore Dec 03 11:46:48 crc kubenswrapper[5052]: + rm -f /var/lib/openvswitch/is_safe_to_stop_ovsdb_server Dec 03 11:46:48 crc kubenswrapper[5052]: + /usr/share/openvswitch/scripts/ovs-ctl stop --no-ovs-vswitchd Dec 03 11:46:48 crc kubenswrapper[5052]: > Dec 03 11:46:48 crc kubenswrapper[5052]: E1203 11:46:48.674660 5052 kuberuntime_container.go:691] "PreStop hook failed" err=< Dec 03 11:46:48 crc kubenswrapper[5052]: command '/usr/local/bin/container-scripts/stop-ovsdb-server.sh' exited with 137: ++ dirname /usr/local/bin/container-scripts/stop-ovsdb-server.sh Dec 03 11:46:48 crc kubenswrapper[5052]: + source /usr/local/bin/container-scripts/functions Dec 03 11:46:48 crc kubenswrapper[5052]: ++ OVNBridge=br-int Dec 03 11:46:48 crc kubenswrapper[5052]: ++ OVNRemote=tcp:localhost:6642 Dec 03 11:46:48 crc kubenswrapper[5052]: ++ OVNEncapType=geneve Dec 03 11:46:48 crc kubenswrapper[5052]: ++ OVNAvailabilityZones= Dec 03 11:46:48 crc kubenswrapper[5052]: ++ EnableChassisAsGateway=true Dec 03 11:46:48 crc kubenswrapper[5052]: ++ PhysicalNetworks= Dec 03 11:46:48 crc kubenswrapper[5052]: ++ OVNHostName= Dec 03 11:46:48 crc kubenswrapper[5052]: ++ DB_FILE=/etc/openvswitch/conf.db Dec 03 11:46:48 crc kubenswrapper[5052]: ++ ovs_dir=/var/lib/openvswitch Dec 03 11:46:48 crc kubenswrapper[5052]: ++ FLOWS_RESTORE_SCRIPT=/var/lib/openvswitch/flows-script Dec 03 11:46:48 crc kubenswrapper[5052]: ++ FLOWS_RESTORE_DIR=/var/lib/openvswitch/saved-flows Dec 03 11:46:48 crc kubenswrapper[5052]: ++ SAFE_TO_STOP_OVSDB_SERVER_SEMAPHORE=/var/lib/openvswitch/is_safe_to_stop_ovsdb_server Dec 03 11:46:48 crc kubenswrapper[5052]: + '[' '!' -f /var/lib/openvswitch/is_safe_to_stop_ovsdb_server ']' Dec 03 11:46:48 crc kubenswrapper[5052]: + sleep 0.5 Dec 03 11:46:48 crc kubenswrapper[5052]: + '[' '!' -f /var/lib/openvswitch/is_safe_to_stop_ovsdb_server ']' Dec 03 11:46:48 crc kubenswrapper[5052]: + cleanup_ovsdb_server_semaphore Dec 03 11:46:48 crc kubenswrapper[5052]: + rm -f /var/lib/openvswitch/is_safe_to_stop_ovsdb_server Dec 03 11:46:48 crc kubenswrapper[5052]: + /usr/share/openvswitch/scripts/ovs-ctl stop --no-ovs-vswitchd Dec 03 11:46:48 crc kubenswrapper[5052]: > pod="openstack-kuttl-tests/ovn-controller-ovs-pk2zp" podUID="d07ce1c1-dd2a-4cb5-b18b-104d1d804fba" containerName="ovsdb-server" containerID="cri-o://18fb02d514e2e0b27dc1ec397b2d8667f545277e140c9896331636baa57b550a" Dec 03 11:46:48 crc kubenswrapper[5052]: I1203 11:46:48.674704 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ovn-controller-ovs-pk2zp" podUID="d07ce1c1-dd2a-4cb5-b18b-104d1d804fba" containerName="ovsdb-server" containerID="cri-o://18fb02d514e2e0b27dc1ec397b2d8667f545277e140c9896331636baa57b550a" gracePeriod=30 Dec 03 11:46:49 crc kubenswrapper[5052]: I1203 11:46:49.061811 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_ovn-controller-metrics-h4r6s_84e7baa6-438b-460f-bb8e-3abee427f12b/openstack-network-exporter/0.log" Dec 03 11:46:49 crc kubenswrapper[5052]: I1203 11:46:49.062160 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovn-controller-metrics-h4r6s" event={"ID":"84e7baa6-438b-460f-bb8e-3abee427f12b","Type":"ContainerDied","Data":"422c2ea063bccfcef61f53f05992e30032a6e3607f0057ede00e6062c1995c29"} Dec 03 11:46:49 crc kubenswrapper[5052]: I1203 11:46:49.062203 5052 scope.go:117] "RemoveContainer" containerID="1f38a6300f951d2085edb95778b8ab8cdb906b0397358a84fbf4a41bc6d06309" Dec 03 11:46:49 crc kubenswrapper[5052]: I1203 11:46:49.062216 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovn-controller-metrics-h4r6s" Dec 03 11:46:49 crc kubenswrapper[5052]: I1203 11:46:49.067295 5052 generic.go:334] "Generic (PLEG): container finished" podID="d07ce1c1-dd2a-4cb5-b18b-104d1d804fba" containerID="18fb02d514e2e0b27dc1ec397b2d8667f545277e140c9896331636baa57b550a" exitCode=0 Dec 03 11:46:49 crc kubenswrapper[5052]: I1203 11:46:49.067383 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovn-controller-clb8s" Dec 03 11:46:49 crc kubenswrapper[5052]: I1203 11:46:49.067386 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovn-controller-ovs-pk2zp" event={"ID":"d07ce1c1-dd2a-4cb5-b18b-104d1d804fba","Type":"ContainerDied","Data":"18fb02d514e2e0b27dc1ec397b2d8667f545277e140c9896331636baa57b550a"} Dec 03 11:46:49 crc kubenswrapper[5052]: I1203 11:46:49.101061 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ovn-controller-metrics-h4r6s"] Dec 03 11:46:49 crc kubenswrapper[5052]: I1203 11:46:49.125093 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ovn-controller-metrics-h4r6s"] Dec 03 11:46:49 crc kubenswrapper[5052]: I1203 11:46:49.137724 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ovn-controller-clb8s"] Dec 03 11:46:49 crc kubenswrapper[5052]: I1203 11:46:49.175944 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ovn-controller-clb8s"] Dec 03 11:46:49 crc kubenswrapper[5052]: I1203 11:46:49.683608 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0bc28d89-7210-4c40-a72c-3f3903a6ea15" path="/var/lib/kubelet/pods/0bc28d89-7210-4c40-a72c-3f3903a6ea15/volumes" Dec 03 11:46:49 crc kubenswrapper[5052]: I1203 11:46:49.684305 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="84e7baa6-438b-460f-bb8e-3abee427f12b" path="/var/lib/kubelet/pods/84e7baa6-438b-460f-bb8e-3abee427f12b/volumes" Dec 03 11:46:52 crc kubenswrapper[5052]: I1203 11:46:52.253666 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:46:52 crc kubenswrapper[5052]: I1203 11:46:52.254137 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:46:52 crc kubenswrapper[5052]: I1203 11:46:52.672985 5052 scope.go:117] "RemoveContainer" containerID="25fe88f8d102f09983f8884e65f0c6c1575ade67921f10a630e6dcc3ea8b506e" Dec 03 11:46:52 crc kubenswrapper[5052]: E1203 11:46:52.673512 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 11:46:53 crc kubenswrapper[5052]: I1203 11:46:53.274036 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-api-0" podUID="f7123213-06e1-4b26-9892-c16c3d49c0ab" containerName="nova-api-log" probeResult="failure" output="Get \"https://10.217.1.38:8774/\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Dec 03 11:46:53 crc kubenswrapper[5052]: I1203 11:46:53.274132 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-api-0" podUID="f7123213-06e1-4b26-9892-c16c3d49c0ab" containerName="nova-api-api" probeResult="failure" output="Get \"https://10.217.1.38:8774/\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Dec 03 11:47:02 crc kubenswrapper[5052]: I1203 11:47:02.261530 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:47:02 crc kubenswrapper[5052]: I1203 11:47:02.262032 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:47:02 crc kubenswrapper[5052]: I1203 11:47:02.263232 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:47:02 crc kubenswrapper[5052]: I1203 11:47:02.263317 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:47:02 crc kubenswrapper[5052]: I1203 11:47:02.271471 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:47:02 crc kubenswrapper[5052]: I1203 11:47:02.273424 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:47:05 crc kubenswrapper[5052]: I1203 11:47:05.672901 5052 scope.go:117] "RemoveContainer" containerID="25fe88f8d102f09983f8884e65f0c6c1575ade67921f10a630e6dcc3ea8b506e" Dec 03 11:47:05 crc kubenswrapper[5052]: E1203 11:47:05.673566 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 11:47:09 crc kubenswrapper[5052]: I1203 11:47:09.237421 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:47:15 crc kubenswrapper[5052]: E1203 11:47:15.969389 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of 18fb02d514e2e0b27dc1ec397b2d8667f545277e140c9896331636baa57b550a is running failed: container process not found" containerID="18fb02d514e2e0b27dc1ec397b2d8667f545277e140c9896331636baa57b550a" cmd=["/usr/local/bin/container-scripts/ovsdb_server_readiness.sh"] Dec 03 11:47:15 crc kubenswrapper[5052]: E1203 11:47:15.970620 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="56fbc866f43e285800b8f191573fefd9bb26eac5669a06e69f010ab712844416" cmd=["/usr/local/bin/container-scripts/vswitchd_readiness.sh"] Dec 03 11:47:15 crc kubenswrapper[5052]: E1203 11:47:15.970910 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of 18fb02d514e2e0b27dc1ec397b2d8667f545277e140c9896331636baa57b550a is running failed: container process not found" containerID="18fb02d514e2e0b27dc1ec397b2d8667f545277e140c9896331636baa57b550a" cmd=["/usr/local/bin/container-scripts/ovsdb_server_readiness.sh"] Dec 03 11:47:15 crc kubenswrapper[5052]: E1203 11:47:15.971485 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of 18fb02d514e2e0b27dc1ec397b2d8667f545277e140c9896331636baa57b550a is running failed: container process not found" containerID="18fb02d514e2e0b27dc1ec397b2d8667f545277e140c9896331636baa57b550a" cmd=["/usr/local/bin/container-scripts/ovsdb_server_readiness.sh"] Dec 03 11:47:15 crc kubenswrapper[5052]: E1203 11:47:15.971585 5052 prober.go:104] "Probe errored" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of 18fb02d514e2e0b27dc1ec397b2d8667f545277e140c9896331636baa57b550a is running failed: container process not found" probeType="Readiness" pod="openstack-kuttl-tests/ovn-controller-ovs-pk2zp" podUID="d07ce1c1-dd2a-4cb5-b18b-104d1d804fba" containerName="ovsdb-server" Dec 03 11:47:15 crc kubenswrapper[5052]: E1203 11:47:15.972229 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="56fbc866f43e285800b8f191573fefd9bb26eac5669a06e69f010ab712844416" cmd=["/usr/local/bin/container-scripts/vswitchd_readiness.sh"] Dec 03 11:47:15 crc kubenswrapper[5052]: E1203 11:47:15.974782 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="56fbc866f43e285800b8f191573fefd9bb26eac5669a06e69f010ab712844416" cmd=["/usr/local/bin/container-scripts/vswitchd_readiness.sh"] Dec 03 11:47:15 crc kubenswrapper[5052]: E1203 11:47:15.974813 5052 prober.go:104] "Probe errored" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" probeType="Readiness" pod="openstack-kuttl-tests/ovn-controller-ovs-pk2zp" podUID="d07ce1c1-dd2a-4cb5-b18b-104d1d804fba" containerName="ovs-vswitchd" Dec 03 11:47:18 crc kubenswrapper[5052]: I1203 11:47:18.868451 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_ovn-controller-ovs-pk2zp_d07ce1c1-dd2a-4cb5-b18b-104d1d804fba/ovs-vswitchd/0.log" Dec 03 11:47:18 crc kubenswrapper[5052]: I1203 11:47:18.869900 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovn-controller-ovs-pk2zp" Dec 03 11:47:18 crc kubenswrapper[5052]: I1203 11:47:18.955341 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xnlsv\" (UniqueName: \"kubernetes.io/projected/d07ce1c1-dd2a-4cb5-b18b-104d1d804fba-kube-api-access-xnlsv\") pod \"d07ce1c1-dd2a-4cb5-b18b-104d1d804fba\" (UID: \"d07ce1c1-dd2a-4cb5-b18b-104d1d804fba\") " Dec 03 11:47:18 crc kubenswrapper[5052]: I1203 11:47:18.955414 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"var-log\" (UniqueName: \"kubernetes.io/host-path/d07ce1c1-dd2a-4cb5-b18b-104d1d804fba-var-log\") pod \"d07ce1c1-dd2a-4cb5-b18b-104d1d804fba\" (UID: \"d07ce1c1-dd2a-4cb5-b18b-104d1d804fba\") " Dec 03 11:47:18 crc kubenswrapper[5052]: I1203 11:47:18.955479 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"var-lib\" (UniqueName: \"kubernetes.io/host-path/d07ce1c1-dd2a-4cb5-b18b-104d1d804fba-var-lib\") pod \"d07ce1c1-dd2a-4cb5-b18b-104d1d804fba\" (UID: \"d07ce1c1-dd2a-4cb5-b18b-104d1d804fba\") " Dec 03 11:47:18 crc kubenswrapper[5052]: I1203 11:47:18.955511 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"var-run\" (UniqueName: \"kubernetes.io/host-path/d07ce1c1-dd2a-4cb5-b18b-104d1d804fba-var-run\") pod \"d07ce1c1-dd2a-4cb5-b18b-104d1d804fba\" (UID: \"d07ce1c1-dd2a-4cb5-b18b-104d1d804fba\") " Dec 03 11:47:18 crc kubenswrapper[5052]: I1203 11:47:18.955549 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/d07ce1c1-dd2a-4cb5-b18b-104d1d804fba-var-log" (OuterVolumeSpecName: "var-log") pod "d07ce1c1-dd2a-4cb5-b18b-104d1d804fba" (UID: "d07ce1c1-dd2a-4cb5-b18b-104d1d804fba"). InnerVolumeSpecName "var-log". PluginName "kubernetes.io/host-path", VolumeGidValue "" Dec 03 11:47:18 crc kubenswrapper[5052]: I1203 11:47:18.955583 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"etc-ovs\" (UniqueName: \"kubernetes.io/host-path/d07ce1c1-dd2a-4cb5-b18b-104d1d804fba-etc-ovs\") pod \"d07ce1c1-dd2a-4cb5-b18b-104d1d804fba\" (UID: \"d07ce1c1-dd2a-4cb5-b18b-104d1d804fba\") " Dec 03 11:47:18 crc kubenswrapper[5052]: I1203 11:47:18.955613 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/d07ce1c1-dd2a-4cb5-b18b-104d1d804fba-var-lib" (OuterVolumeSpecName: "var-lib") pod "d07ce1c1-dd2a-4cb5-b18b-104d1d804fba" (UID: "d07ce1c1-dd2a-4cb5-b18b-104d1d804fba"). InnerVolumeSpecName "var-lib". PluginName "kubernetes.io/host-path", VolumeGidValue "" Dec 03 11:47:18 crc kubenswrapper[5052]: I1203 11:47:18.955628 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/d07ce1c1-dd2a-4cb5-b18b-104d1d804fba-var-run" (OuterVolumeSpecName: "var-run") pod "d07ce1c1-dd2a-4cb5-b18b-104d1d804fba" (UID: "d07ce1c1-dd2a-4cb5-b18b-104d1d804fba"). InnerVolumeSpecName "var-run". PluginName "kubernetes.io/host-path", VolumeGidValue "" Dec 03 11:47:18 crc kubenswrapper[5052]: I1203 11:47:18.955731 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/d07ce1c1-dd2a-4cb5-b18b-104d1d804fba-scripts\") pod \"d07ce1c1-dd2a-4cb5-b18b-104d1d804fba\" (UID: \"d07ce1c1-dd2a-4cb5-b18b-104d1d804fba\") " Dec 03 11:47:18 crc kubenswrapper[5052]: I1203 11:47:18.955721 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/d07ce1c1-dd2a-4cb5-b18b-104d1d804fba-etc-ovs" (OuterVolumeSpecName: "etc-ovs") pod "d07ce1c1-dd2a-4cb5-b18b-104d1d804fba" (UID: "d07ce1c1-dd2a-4cb5-b18b-104d1d804fba"). InnerVolumeSpecName "etc-ovs". PluginName "kubernetes.io/host-path", VolumeGidValue "" Dec 03 11:47:18 crc kubenswrapper[5052]: I1203 11:47:18.956293 5052 reconciler_common.go:293] "Volume detached for volume \"var-log\" (UniqueName: \"kubernetes.io/host-path/d07ce1c1-dd2a-4cb5-b18b-104d1d804fba-var-log\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:18 crc kubenswrapper[5052]: I1203 11:47:18.956321 5052 reconciler_common.go:293] "Volume detached for volume \"var-lib\" (UniqueName: \"kubernetes.io/host-path/d07ce1c1-dd2a-4cb5-b18b-104d1d804fba-var-lib\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:18 crc kubenswrapper[5052]: I1203 11:47:18.956332 5052 reconciler_common.go:293] "Volume detached for volume \"etc-ovs\" (UniqueName: \"kubernetes.io/host-path/d07ce1c1-dd2a-4cb5-b18b-104d1d804fba-etc-ovs\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:18 crc kubenswrapper[5052]: I1203 11:47:18.956344 5052 reconciler_common.go:293] "Volume detached for volume \"var-run\" (UniqueName: \"kubernetes.io/host-path/d07ce1c1-dd2a-4cb5-b18b-104d1d804fba-var-run\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:18 crc kubenswrapper[5052]: I1203 11:47:18.956753 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d07ce1c1-dd2a-4cb5-b18b-104d1d804fba-scripts" (OuterVolumeSpecName: "scripts") pod "d07ce1c1-dd2a-4cb5-b18b-104d1d804fba" (UID: "d07ce1c1-dd2a-4cb5-b18b-104d1d804fba"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:47:18 crc kubenswrapper[5052]: I1203 11:47:18.961045 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d07ce1c1-dd2a-4cb5-b18b-104d1d804fba-kube-api-access-xnlsv" (OuterVolumeSpecName: "kube-api-access-xnlsv") pod "d07ce1c1-dd2a-4cb5-b18b-104d1d804fba" (UID: "d07ce1c1-dd2a-4cb5-b18b-104d1d804fba"). InnerVolumeSpecName "kube-api-access-xnlsv". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:47:19 crc kubenswrapper[5052]: I1203 11:47:19.058059 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-xnlsv\" (UniqueName: \"kubernetes.io/projected/d07ce1c1-dd2a-4cb5-b18b-104d1d804fba-kube-api-access-xnlsv\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:19 crc kubenswrapper[5052]: I1203 11:47:19.058094 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/d07ce1c1-dd2a-4cb5-b18b-104d1d804fba-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:19 crc kubenswrapper[5052]: I1203 11:47:19.366990 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_ovn-controller-ovs-pk2zp_d07ce1c1-dd2a-4cb5-b18b-104d1d804fba/ovs-vswitchd/0.log" Dec 03 11:47:19 crc kubenswrapper[5052]: I1203 11:47:19.368269 5052 generic.go:334] "Generic (PLEG): container finished" podID="d07ce1c1-dd2a-4cb5-b18b-104d1d804fba" containerID="56fbc866f43e285800b8f191573fefd9bb26eac5669a06e69f010ab712844416" exitCode=137 Dec 03 11:47:19 crc kubenswrapper[5052]: I1203 11:47:19.368313 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovn-controller-ovs-pk2zp" event={"ID":"d07ce1c1-dd2a-4cb5-b18b-104d1d804fba","Type":"ContainerDied","Data":"56fbc866f43e285800b8f191573fefd9bb26eac5669a06e69f010ab712844416"} Dec 03 11:47:19 crc kubenswrapper[5052]: I1203 11:47:19.368352 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovn-controller-ovs-pk2zp" event={"ID":"d07ce1c1-dd2a-4cb5-b18b-104d1d804fba","Type":"ContainerDied","Data":"8f071e83f1d59f01789ce0b86f2fa626b85326459f4ba2b194b0a32aaccecfa2"} Dec 03 11:47:19 crc kubenswrapper[5052]: I1203 11:47:19.368394 5052 scope.go:117] "RemoveContainer" containerID="56fbc866f43e285800b8f191573fefd9bb26eac5669a06e69f010ab712844416" Dec 03 11:47:19 crc kubenswrapper[5052]: I1203 11:47:19.368609 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovn-controller-ovs-pk2zp" Dec 03 11:47:19 crc kubenswrapper[5052]: I1203 11:47:19.392292 5052 scope.go:117] "RemoveContainer" containerID="18fb02d514e2e0b27dc1ec397b2d8667f545277e140c9896331636baa57b550a" Dec 03 11:47:19 crc kubenswrapper[5052]: I1203 11:47:19.410784 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ovn-controller-ovs-pk2zp"] Dec 03 11:47:19 crc kubenswrapper[5052]: I1203 11:47:19.416158 5052 scope.go:117] "RemoveContainer" containerID="090548138e69b6166f412957493047690313507ebeaeb842aa53a4bde8c0d141" Dec 03 11:47:19 crc kubenswrapper[5052]: I1203 11:47:19.425621 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ovn-controller-ovs-pk2zp"] Dec 03 11:47:19 crc kubenswrapper[5052]: I1203 11:47:19.452986 5052 scope.go:117] "RemoveContainer" containerID="56fbc866f43e285800b8f191573fefd9bb26eac5669a06e69f010ab712844416" Dec 03 11:47:19 crc kubenswrapper[5052]: E1203 11:47:19.453483 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"56fbc866f43e285800b8f191573fefd9bb26eac5669a06e69f010ab712844416\": container with ID starting with 56fbc866f43e285800b8f191573fefd9bb26eac5669a06e69f010ab712844416 not found: ID does not exist" containerID="56fbc866f43e285800b8f191573fefd9bb26eac5669a06e69f010ab712844416" Dec 03 11:47:19 crc kubenswrapper[5052]: I1203 11:47:19.453538 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"56fbc866f43e285800b8f191573fefd9bb26eac5669a06e69f010ab712844416"} err="failed to get container status \"56fbc866f43e285800b8f191573fefd9bb26eac5669a06e69f010ab712844416\": rpc error: code = NotFound desc = could not find container \"56fbc866f43e285800b8f191573fefd9bb26eac5669a06e69f010ab712844416\": container with ID starting with 56fbc866f43e285800b8f191573fefd9bb26eac5669a06e69f010ab712844416 not found: ID does not exist" Dec 03 11:47:19 crc kubenswrapper[5052]: I1203 11:47:19.453568 5052 scope.go:117] "RemoveContainer" containerID="18fb02d514e2e0b27dc1ec397b2d8667f545277e140c9896331636baa57b550a" Dec 03 11:47:19 crc kubenswrapper[5052]: E1203 11:47:19.453962 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"18fb02d514e2e0b27dc1ec397b2d8667f545277e140c9896331636baa57b550a\": container with ID starting with 18fb02d514e2e0b27dc1ec397b2d8667f545277e140c9896331636baa57b550a not found: ID does not exist" containerID="18fb02d514e2e0b27dc1ec397b2d8667f545277e140c9896331636baa57b550a" Dec 03 11:47:19 crc kubenswrapper[5052]: I1203 11:47:19.453984 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"18fb02d514e2e0b27dc1ec397b2d8667f545277e140c9896331636baa57b550a"} err="failed to get container status \"18fb02d514e2e0b27dc1ec397b2d8667f545277e140c9896331636baa57b550a\": rpc error: code = NotFound desc = could not find container \"18fb02d514e2e0b27dc1ec397b2d8667f545277e140c9896331636baa57b550a\": container with ID starting with 18fb02d514e2e0b27dc1ec397b2d8667f545277e140c9896331636baa57b550a not found: ID does not exist" Dec 03 11:47:19 crc kubenswrapper[5052]: I1203 11:47:19.453998 5052 scope.go:117] "RemoveContainer" containerID="090548138e69b6166f412957493047690313507ebeaeb842aa53a4bde8c0d141" Dec 03 11:47:19 crc kubenswrapper[5052]: E1203 11:47:19.454727 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"090548138e69b6166f412957493047690313507ebeaeb842aa53a4bde8c0d141\": container with ID starting with 090548138e69b6166f412957493047690313507ebeaeb842aa53a4bde8c0d141 not found: ID does not exist" containerID="090548138e69b6166f412957493047690313507ebeaeb842aa53a4bde8c0d141" Dec 03 11:47:19 crc kubenswrapper[5052]: I1203 11:47:19.454745 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"090548138e69b6166f412957493047690313507ebeaeb842aa53a4bde8c0d141"} err="failed to get container status \"090548138e69b6166f412957493047690313507ebeaeb842aa53a4bde8c0d141\": rpc error: code = NotFound desc = could not find container \"090548138e69b6166f412957493047690313507ebeaeb842aa53a4bde8c0d141\": container with ID starting with 090548138e69b6166f412957493047690313507ebeaeb842aa53a4bde8c0d141 not found: ID does not exist" Dec 03 11:47:19 crc kubenswrapper[5052]: I1203 11:47:19.699281 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d07ce1c1-dd2a-4cb5-b18b-104d1d804fba" path="/var/lib/kubelet/pods/d07ce1c1-dd2a-4cb5-b18b-104d1d804fba/volumes" Dec 03 11:47:20 crc kubenswrapper[5052]: I1203 11:47:20.618128 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/openstackclient"] Dec 03 11:47:20 crc kubenswrapper[5052]: I1203 11:47:20.618373 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/openstackclient" podUID="a9f103a5-8b29-4152-a711-60891b9aeefc" containerName="openstackclient" containerID="cri-o://6b2a059106f1fcc4b1e035c9cf505bb5bc72d3f6e38d77875ea0e33ccbc713f1" gracePeriod=2 Dec 03 11:47:20 crc kubenswrapper[5052]: I1203 11:47:20.640319 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/openstackclient"] Dec 03 11:47:20 crc kubenswrapper[5052]: I1203 11:47:20.677941 5052 scope.go:117] "RemoveContainer" containerID="25fe88f8d102f09983f8884e65f0c6c1575ade67921f10a630e6dcc3ea8b506e" Dec 03 11:47:20 crc kubenswrapper[5052]: E1203 11:47:20.678252 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 11:47:20 crc kubenswrapper[5052]: I1203 11:47:20.686939 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/rabbitmq-cell1-server-0"] Dec 03 11:47:20 crc kubenswrapper[5052]: E1203 11:47:20.793392 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/rabbitmq-cell1-config-data: configmap "rabbitmq-cell1-config-data" not found Dec 03 11:47:20 crc kubenswrapper[5052]: E1203 11:47:20.793734 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2766f489-9217-47ae-a595-869a26389587-config-data podName:2766f489-9217-47ae-a595-869a26389587 nodeName:}" failed. No retries permitted until 2025-12-03 11:47:21.293715068 +0000 UTC m=+1631.881956832 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "config-data" (UniqueName: "kubernetes.io/configmap/2766f489-9217-47ae-a595-869a26389587-config-data") pod "rabbitmq-cell1-server-0" (UID: "2766f489-9217-47ae-a595-869a26389587") : configmap "rabbitmq-cell1-config-data" not found Dec 03 11:47:20 crc kubenswrapper[5052]: E1203 11:47:20.796594 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/combined-ca-bundle: secret "combined-ca-bundle" not found Dec 03 11:47:20 crc kubenswrapper[5052]: E1203 11:47:20.799365 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/507c60e5-056c-4661-a7a7-75c4d9bce7de-combined-ca-bundle podName:507c60e5-056c-4661-a7a7-75c4d9bce7de nodeName:}" failed. No retries permitted until 2025-12-03 11:47:21.29935454 +0000 UTC m=+1631.887596304 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "combined-ca-bundle" (UniqueName: "kubernetes.io/secret/507c60e5-056c-4661-a7a7-75c4d9bce7de-combined-ca-bundle") pod "nova-cell1-conductor-0" (UID: "507c60e5-056c-4661-a7a7-75c4d9bce7de") : secret "combined-ca-bundle" not found Dec 03 11:47:20 crc kubenswrapper[5052]: I1203 11:47:20.916360 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/glance3554-account-delete-8ghdx"] Dec 03 11:47:20 crc kubenswrapper[5052]: E1203 11:47:20.916784 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="0bc28d89-7210-4c40-a72c-3f3903a6ea15" containerName="ovn-controller" Dec 03 11:47:20 crc kubenswrapper[5052]: I1203 11:47:20.916801 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="0bc28d89-7210-4c40-a72c-3f3903a6ea15" containerName="ovn-controller" Dec 03 11:47:20 crc kubenswrapper[5052]: E1203 11:47:20.916814 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="d07ce1c1-dd2a-4cb5-b18b-104d1d804fba" containerName="ovsdb-server-init" Dec 03 11:47:20 crc kubenswrapper[5052]: I1203 11:47:20.916820 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="d07ce1c1-dd2a-4cb5-b18b-104d1d804fba" containerName="ovsdb-server-init" Dec 03 11:47:20 crc kubenswrapper[5052]: E1203 11:47:20.916831 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="84e7baa6-438b-460f-bb8e-3abee427f12b" containerName="openstack-network-exporter" Dec 03 11:47:20 crc kubenswrapper[5052]: I1203 11:47:20.916837 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="84e7baa6-438b-460f-bb8e-3abee427f12b" containerName="openstack-network-exporter" Dec 03 11:47:20 crc kubenswrapper[5052]: E1203 11:47:20.916845 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="d07ce1c1-dd2a-4cb5-b18b-104d1d804fba" containerName="ovs-vswitchd" Dec 03 11:47:20 crc kubenswrapper[5052]: I1203 11:47:20.916852 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="d07ce1c1-dd2a-4cb5-b18b-104d1d804fba" containerName="ovs-vswitchd" Dec 03 11:47:20 crc kubenswrapper[5052]: E1203 11:47:20.916869 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="a9f103a5-8b29-4152-a711-60891b9aeefc" containerName="openstackclient" Dec 03 11:47:20 crc kubenswrapper[5052]: I1203 11:47:20.916876 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="a9f103a5-8b29-4152-a711-60891b9aeefc" containerName="openstackclient" Dec 03 11:47:20 crc kubenswrapper[5052]: E1203 11:47:20.916893 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="d07ce1c1-dd2a-4cb5-b18b-104d1d804fba" containerName="ovsdb-server" Dec 03 11:47:20 crc kubenswrapper[5052]: I1203 11:47:20.916898 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="d07ce1c1-dd2a-4cb5-b18b-104d1d804fba" containerName="ovsdb-server" Dec 03 11:47:20 crc kubenswrapper[5052]: I1203 11:47:20.917078 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="d07ce1c1-dd2a-4cb5-b18b-104d1d804fba" containerName="ovsdb-server" Dec 03 11:47:20 crc kubenswrapper[5052]: I1203 11:47:20.917100 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="a9f103a5-8b29-4152-a711-60891b9aeefc" containerName="openstackclient" Dec 03 11:47:20 crc kubenswrapper[5052]: I1203 11:47:20.917107 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="0bc28d89-7210-4c40-a72c-3f3903a6ea15" containerName="ovn-controller" Dec 03 11:47:20 crc kubenswrapper[5052]: I1203 11:47:20.917117 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="d07ce1c1-dd2a-4cb5-b18b-104d1d804fba" containerName="ovs-vswitchd" Dec 03 11:47:20 crc kubenswrapper[5052]: I1203 11:47:20.917126 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="84e7baa6-438b-460f-bb8e-3abee427f12b" containerName="openstack-network-exporter" Dec 03 11:47:20 crc kubenswrapper[5052]: I1203 11:47:20.917790 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance3554-account-delete-8ghdx" Dec 03 11:47:20 crc kubenswrapper[5052]: I1203 11:47:20.946731 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-nb-0"] Dec 03 11:47:20 crc kubenswrapper[5052]: I1203 11:47:20.947041 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ovsdbserver-nb-0" podUID="68f829f3-e972-4168-8b75-27c086a91142" containerName="openstack-network-exporter" containerID="cri-o://630e406fa52125d27b16eb3de57c1b6dc5a497b24481482e2ba05d01fcd553fc" gracePeriod=300 Dec 03 11:47:20 crc kubenswrapper[5052]: I1203 11:47:20.965373 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance3554-account-delete-8ghdx"] Dec 03 11:47:21 crc kubenswrapper[5052]: I1203 11:47:21.073819 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/placement9a4f-account-delete-f6sjd"] Dec 03 11:47:21 crc kubenswrapper[5052]: I1203 11:47:21.075159 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement9a4f-account-delete-f6sjd" Dec 03 11:47:21 crc kubenswrapper[5052]: I1203 11:47:21.092713 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-db-sync-g22fp"] Dec 03 11:47:21 crc kubenswrapper[5052]: I1203 11:47:21.117933 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/placement9a4f-account-delete-f6sjd"] Dec 03 11:47:21 crc kubenswrapper[5052]: I1203 11:47:21.120005 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rvkbs\" (UniqueName: \"kubernetes.io/projected/950f0626-40d2-41c9-9701-cba50843c43e-kube-api-access-rvkbs\") pod \"glance3554-account-delete-8ghdx\" (UID: \"950f0626-40d2-41c9-9701-cba50843c43e\") " pod="openstack-kuttl-tests/glance3554-account-delete-8ghdx" Dec 03 11:47:21 crc kubenswrapper[5052]: I1203 11:47:21.120296 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/950f0626-40d2-41c9-9701-cba50843c43e-operator-scripts\") pod \"glance3554-account-delete-8ghdx\" (UID: \"950f0626-40d2-41c9-9701-cba50843c43e\") " pod="openstack-kuttl-tests/glance3554-account-delete-8ghdx" Dec 03 11:47:21 crc kubenswrapper[5052]: I1203 11:47:21.120333 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dpgmf\" (UniqueName: \"kubernetes.io/projected/8c2252c0-5e30-4e38-b1a1-af0242df6ac0-kube-api-access-dpgmf\") pod \"placement9a4f-account-delete-f6sjd\" (UID: \"8c2252c0-5e30-4e38-b1a1-af0242df6ac0\") " pod="openstack-kuttl-tests/placement9a4f-account-delete-f6sjd" Dec 03 11:47:21 crc kubenswrapper[5052]: I1203 11:47:21.120413 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/8c2252c0-5e30-4e38-b1a1-af0242df6ac0-operator-scripts\") pod \"placement9a4f-account-delete-f6sjd\" (UID: \"8c2252c0-5e30-4e38-b1a1-af0242df6ac0\") " pod="openstack-kuttl-tests/placement9a4f-account-delete-f6sjd" Dec 03 11:47:21 crc kubenswrapper[5052]: I1203 11:47:21.134877 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/glance-db-sync-g22fp"] Dec 03 11:47:21 crc kubenswrapper[5052]: I1203 11:47:21.148628 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ovn-northd-0"] Dec 03 11:47:21 crc kubenswrapper[5052]: I1203 11:47:21.149193 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ovn-northd-0" podUID="83a10afe-30d8-443d-a4f7-28784a2ea068" containerName="ovn-northd" containerID="cri-o://3b43b29211f1c83741ff8d9a266e5e11a2e9560cc3168a7ef1db16f7622c61ba" gracePeriod=30 Dec 03 11:47:21 crc kubenswrapper[5052]: I1203 11:47:21.149630 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ovn-northd-0" podUID="83a10afe-30d8-443d-a4f7-28784a2ea068" containerName="openstack-network-exporter" containerID="cri-o://af748c115d51d513b98829b5044d30038362360180988763c14bac6e6886f163" gracePeriod=30 Dec 03 11:47:21 crc kubenswrapper[5052]: I1203 11:47:21.226109 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-dpgmf\" (UniqueName: \"kubernetes.io/projected/8c2252c0-5e30-4e38-b1a1-af0242df6ac0-kube-api-access-dpgmf\") pod \"placement9a4f-account-delete-f6sjd\" (UID: \"8c2252c0-5e30-4e38-b1a1-af0242df6ac0\") " pod="openstack-kuttl-tests/placement9a4f-account-delete-f6sjd" Dec 03 11:47:21 crc kubenswrapper[5052]: I1203 11:47:21.226814 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/8c2252c0-5e30-4e38-b1a1-af0242df6ac0-operator-scripts\") pod \"placement9a4f-account-delete-f6sjd\" (UID: \"8c2252c0-5e30-4e38-b1a1-af0242df6ac0\") " pod="openstack-kuttl-tests/placement9a4f-account-delete-f6sjd" Dec 03 11:47:21 crc kubenswrapper[5052]: I1203 11:47:21.227037 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-rvkbs\" (UniqueName: \"kubernetes.io/projected/950f0626-40d2-41c9-9701-cba50843c43e-kube-api-access-rvkbs\") pod \"glance3554-account-delete-8ghdx\" (UID: \"950f0626-40d2-41c9-9701-cba50843c43e\") " pod="openstack-kuttl-tests/glance3554-account-delete-8ghdx" Dec 03 11:47:21 crc kubenswrapper[5052]: I1203 11:47:21.227366 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/950f0626-40d2-41c9-9701-cba50843c43e-operator-scripts\") pod \"glance3554-account-delete-8ghdx\" (UID: \"950f0626-40d2-41c9-9701-cba50843c43e\") " pod="openstack-kuttl-tests/glance3554-account-delete-8ghdx" Dec 03 11:47:21 crc kubenswrapper[5052]: I1203 11:47:21.227871 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/8c2252c0-5e30-4e38-b1a1-af0242df6ac0-operator-scripts\") pod \"placement9a4f-account-delete-f6sjd\" (UID: \"8c2252c0-5e30-4e38-b1a1-af0242df6ac0\") " pod="openstack-kuttl-tests/placement9a4f-account-delete-f6sjd" Dec 03 11:47:21 crc kubenswrapper[5052]: I1203 11:47:21.228253 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/950f0626-40d2-41c9-9701-cba50843c43e-operator-scripts\") pod \"glance3554-account-delete-8ghdx\" (UID: \"950f0626-40d2-41c9-9701-cba50843c43e\") " pod="openstack-kuttl-tests/glance3554-account-delete-8ghdx" Dec 03 11:47:21 crc kubenswrapper[5052]: I1203 11:47:21.259213 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/cinder1bf0-account-delete-hs8j6"] Dec 03 11:47:21 crc kubenswrapper[5052]: I1203 11:47:21.279122 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder1bf0-account-delete-hs8j6"] Dec 03 11:47:21 crc kubenswrapper[5052]: I1203 11:47:21.279262 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder1bf0-account-delete-hs8j6" Dec 03 11:47:21 crc kubenswrapper[5052]: I1203 11:47:21.297464 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-rvkbs\" (UniqueName: \"kubernetes.io/projected/950f0626-40d2-41c9-9701-cba50843c43e-kube-api-access-rvkbs\") pod \"glance3554-account-delete-8ghdx\" (UID: \"950f0626-40d2-41c9-9701-cba50843c43e\") " pod="openstack-kuttl-tests/glance3554-account-delete-8ghdx" Dec 03 11:47:21 crc kubenswrapper[5052]: I1203 11:47:21.309393 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-dpgmf\" (UniqueName: \"kubernetes.io/projected/8c2252c0-5e30-4e38-b1a1-af0242df6ac0-kube-api-access-dpgmf\") pod \"placement9a4f-account-delete-f6sjd\" (UID: \"8c2252c0-5e30-4e38-b1a1-af0242df6ac0\") " pod="openstack-kuttl-tests/placement9a4f-account-delete-f6sjd" Dec 03 11:47:21 crc kubenswrapper[5052]: I1203 11:47:21.332755 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/0b94b517-5f9b-485d-a109-0a11e15a3ad3-operator-scripts\") pod \"cinder1bf0-account-delete-hs8j6\" (UID: \"0b94b517-5f9b-485d-a109-0a11e15a3ad3\") " pod="openstack-kuttl-tests/cinder1bf0-account-delete-hs8j6" Dec 03 11:47:21 crc kubenswrapper[5052]: I1203 11:47:21.332938 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-brvb8\" (UniqueName: \"kubernetes.io/projected/0b94b517-5f9b-485d-a109-0a11e15a3ad3-kube-api-access-brvb8\") pod \"cinder1bf0-account-delete-hs8j6\" (UID: \"0b94b517-5f9b-485d-a109-0a11e15a3ad3\") " pod="openstack-kuttl-tests/cinder1bf0-account-delete-hs8j6" Dec 03 11:47:21 crc kubenswrapper[5052]: E1203 11:47:21.333081 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/combined-ca-bundle: secret "combined-ca-bundle" not found Dec 03 11:47:21 crc kubenswrapper[5052]: E1203 11:47:21.333121 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/507c60e5-056c-4661-a7a7-75c4d9bce7de-combined-ca-bundle podName:507c60e5-056c-4661-a7a7-75c4d9bce7de nodeName:}" failed. No retries permitted until 2025-12-03 11:47:22.333106118 +0000 UTC m=+1632.921347882 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "combined-ca-bundle" (UniqueName: "kubernetes.io/secret/507c60e5-056c-4661-a7a7-75c4d9bce7de-combined-ca-bundle") pod "nova-cell1-conductor-0" (UID: "507c60e5-056c-4661-a7a7-75c4d9bce7de") : secret "combined-ca-bundle" not found Dec 03 11:47:21 crc kubenswrapper[5052]: E1203 11:47:21.333413 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/rabbitmq-cell1-config-data: configmap "rabbitmq-cell1-config-data" not found Dec 03 11:47:21 crc kubenswrapper[5052]: E1203 11:47:21.333446 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2766f489-9217-47ae-a595-869a26389587-config-data podName:2766f489-9217-47ae-a595-869a26389587 nodeName:}" failed. No retries permitted until 2025-12-03 11:47:22.333437788 +0000 UTC m=+1632.921679552 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "config-data" (UniqueName: "kubernetes.io/configmap/2766f489-9217-47ae-a595-869a26389587-config-data") pod "rabbitmq-cell1-server-0" (UID: "2766f489-9217-47ae-a595-869a26389587") : configmap "rabbitmq-cell1-config-data" not found Dec 03 11:47:21 crc kubenswrapper[5052]: I1203 11:47:21.410065 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement9a4f-account-delete-f6sjd" Dec 03 11:47:21 crc kubenswrapper[5052]: I1203 11:47:21.417454 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/neutron18a7-account-delete-x9lqd"] Dec 03 11:47:21 crc kubenswrapper[5052]: I1203 11:47:21.423342 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron18a7-account-delete-x9lqd" Dec 03 11:47:21 crc kubenswrapper[5052]: I1203 11:47:21.438465 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/0b94b517-5f9b-485d-a109-0a11e15a3ad3-operator-scripts\") pod \"cinder1bf0-account-delete-hs8j6\" (UID: \"0b94b517-5f9b-485d-a109-0a11e15a3ad3\") " pod="openstack-kuttl-tests/cinder1bf0-account-delete-hs8j6" Dec 03 11:47:21 crc kubenswrapper[5052]: I1203 11:47:21.438614 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/aa9a649b-9229-4cdd-8e27-8aedd48f4bbd-operator-scripts\") pod \"neutron18a7-account-delete-x9lqd\" (UID: \"aa9a649b-9229-4cdd-8e27-8aedd48f4bbd\") " pod="openstack-kuttl-tests/neutron18a7-account-delete-x9lqd" Dec 03 11:47:21 crc kubenswrapper[5052]: I1203 11:47:21.438674 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-brvb8\" (UniqueName: \"kubernetes.io/projected/0b94b517-5f9b-485d-a109-0a11e15a3ad3-kube-api-access-brvb8\") pod \"cinder1bf0-account-delete-hs8j6\" (UID: \"0b94b517-5f9b-485d-a109-0a11e15a3ad3\") " pod="openstack-kuttl-tests/cinder1bf0-account-delete-hs8j6" Dec 03 11:47:21 crc kubenswrapper[5052]: I1203 11:47:21.438693 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l9m8g\" (UniqueName: \"kubernetes.io/projected/aa9a649b-9229-4cdd-8e27-8aedd48f4bbd-kube-api-access-l9m8g\") pod \"neutron18a7-account-delete-x9lqd\" (UID: \"aa9a649b-9229-4cdd-8e27-8aedd48f4bbd\") " pod="openstack-kuttl-tests/neutron18a7-account-delete-x9lqd" Dec 03 11:47:21 crc kubenswrapper[5052]: I1203 11:47:21.443691 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/0b94b517-5f9b-485d-a109-0a11e15a3ad3-operator-scripts\") pod \"cinder1bf0-account-delete-hs8j6\" (UID: \"0b94b517-5f9b-485d-a109-0a11e15a3ad3\") " pod="openstack-kuttl-tests/cinder1bf0-account-delete-hs8j6" Dec 03 11:47:21 crc kubenswrapper[5052]: I1203 11:47:21.455490 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ovsdbserver-nb-0" podUID="68f829f3-e972-4168-8b75-27c086a91142" containerName="ovsdbserver-nb" containerID="cri-o://57f10ab9d3a1c544593f058c6724e34ab08ea61ce5e3535d93586cc3d5834467" gracePeriod=300 Dec 03 11:47:21 crc kubenswrapper[5052]: I1203 11:47:21.473606 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron18a7-account-delete-x9lqd"] Dec 03 11:47:21 crc kubenswrapper[5052]: I1203 11:47:21.493924 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder-db-sync-qv4bk"] Dec 03 11:47:21 crc kubenswrapper[5052]: I1203 11:47:21.514043 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-brvb8\" (UniqueName: \"kubernetes.io/projected/0b94b517-5f9b-485d-a109-0a11e15a3ad3-kube-api-access-brvb8\") pod \"cinder1bf0-account-delete-hs8j6\" (UID: \"0b94b517-5f9b-485d-a109-0a11e15a3ad3\") " pod="openstack-kuttl-tests/cinder1bf0-account-delete-hs8j6" Dec 03 11:47:21 crc kubenswrapper[5052]: I1203 11:47:21.579411 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance3554-account-delete-8ghdx" Dec 03 11:47:21 crc kubenswrapper[5052]: I1203 11:47:21.615311 5052 generic.go:334] "Generic (PLEG): container finished" podID="83a10afe-30d8-443d-a4f7-28784a2ea068" containerID="af748c115d51d513b98829b5044d30038362360180988763c14bac6e6886f163" exitCode=2 Dec 03 11:47:21 crc kubenswrapper[5052]: I1203 11:47:21.615375 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovn-northd-0" event={"ID":"83a10afe-30d8-443d-a4f7-28784a2ea068","Type":"ContainerDied","Data":"af748c115d51d513b98829b5044d30038362360180988763c14bac6e6886f163"} Dec 03 11:47:21 crc kubenswrapper[5052]: I1203 11:47:21.615678 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-l9m8g\" (UniqueName: \"kubernetes.io/projected/aa9a649b-9229-4cdd-8e27-8aedd48f4bbd-kube-api-access-l9m8g\") pod \"neutron18a7-account-delete-x9lqd\" (UID: \"aa9a649b-9229-4cdd-8e27-8aedd48f4bbd\") " pod="openstack-kuttl-tests/neutron18a7-account-delete-x9lqd" Dec 03 11:47:21 crc kubenswrapper[5052]: I1203 11:47:21.616100 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/aa9a649b-9229-4cdd-8e27-8aedd48f4bbd-operator-scripts\") pod \"neutron18a7-account-delete-x9lqd\" (UID: \"aa9a649b-9229-4cdd-8e27-8aedd48f4bbd\") " pod="openstack-kuttl-tests/neutron18a7-account-delete-x9lqd" Dec 03 11:47:21 crc kubenswrapper[5052]: I1203 11:47:21.629299 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/barbican5976-account-delete-q6j5s"] Dec 03 11:47:21 crc kubenswrapper[5052]: I1203 11:47:21.691875 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/aa9a649b-9229-4cdd-8e27-8aedd48f4bbd-operator-scripts\") pod \"neutron18a7-account-delete-x9lqd\" (UID: \"aa9a649b-9229-4cdd-8e27-8aedd48f4bbd\") " pod="openstack-kuttl-tests/neutron18a7-account-delete-x9lqd" Dec 03 11:47:21 crc kubenswrapper[5052]: I1203 11:47:21.695004 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican5976-account-delete-q6j5s" Dec 03 11:47:21 crc kubenswrapper[5052]: I1203 11:47:21.695764 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-l9m8g\" (UniqueName: \"kubernetes.io/projected/aa9a649b-9229-4cdd-8e27-8aedd48f4bbd-kube-api-access-l9m8g\") pod \"neutron18a7-account-delete-x9lqd\" (UID: \"aa9a649b-9229-4cdd-8e27-8aedd48f4bbd\") " pod="openstack-kuttl-tests/neutron18a7-account-delete-x9lqd" Dec 03 11:47:21 crc kubenswrapper[5052]: E1203 11:47:21.708814 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="3b43b29211f1c83741ff8d9a266e5e11a2e9560cc3168a7ef1db16f7622c61ba" cmd=["/usr/local/bin/container-scripts/status_check.sh"] Dec 03 11:47:21 crc kubenswrapper[5052]: I1203 11:47:21.747497 5052 kubelet_pods.go:1007] "Unable to retrieve pull secret, the image pull may not succeed." pod="openstack-kuttl-tests/cinder-scheduler-0" secret="" err="secret \"cinder-cinder-dockercfg-v4hdv\" not found" Dec 03 11:47:21 crc kubenswrapper[5052]: I1203 11:47:21.749040 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lcjjc\" (UniqueName: \"kubernetes.io/projected/0aefd6f6-cbb3-4c45-9f69-7124e43255e3-kube-api-access-lcjjc\") pod \"barbican5976-account-delete-q6j5s\" (UID: \"0aefd6f6-cbb3-4c45-9f69-7124e43255e3\") " pod="openstack-kuttl-tests/barbican5976-account-delete-q6j5s" Dec 03 11:47:21 crc kubenswrapper[5052]: I1203 11:47:21.749361 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/0aefd6f6-cbb3-4c45-9f69-7124e43255e3-operator-scripts\") pod \"barbican5976-account-delete-q6j5s\" (UID: \"0aefd6f6-cbb3-4c45-9f69-7124e43255e3\") " pod="openstack-kuttl-tests/barbican5976-account-delete-q6j5s" Dec 03 11:47:21 crc kubenswrapper[5052]: I1203 11:47:21.769454 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder1bf0-account-delete-hs8j6" Dec 03 11:47:21 crc kubenswrapper[5052]: E1203 11:47:21.771928 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="3b43b29211f1c83741ff8d9a266e5e11a2e9560cc3168a7ef1db16f7622c61ba" cmd=["/usr/local/bin/container-scripts/status_check.sh"] Dec 03 11:47:21 crc kubenswrapper[5052]: I1203 11:47:21.813983 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron18a7-account-delete-x9lqd" Dec 03 11:47:21 crc kubenswrapper[5052]: E1203 11:47:21.826454 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="3b43b29211f1c83741ff8d9a266e5e11a2e9560cc3168a7ef1db16f7622c61ba" cmd=["/usr/local/bin/container-scripts/status_check.sh"] Dec 03 11:47:21 crc kubenswrapper[5052]: E1203 11:47:21.826556 5052 prober.go:104] "Probe errored" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" probeType="Readiness" pod="openstack-kuttl-tests/ovn-northd-0" podUID="83a10afe-30d8-443d-a4f7-28784a2ea068" containerName="ovn-northd" Dec 03 11:47:21 crc kubenswrapper[5052]: I1203 11:47:21.852607 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3c063992-bf41-4c35-a5ad-cdc4b512bcee" path="/var/lib/kubelet/pods/3c063992-bf41-4c35-a5ad-cdc4b512bcee/volumes" Dec 03 11:47:21 crc kubenswrapper[5052]: I1203 11:47:21.854441 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/placement-db-sync-jlwkf"] Dec 03 11:47:21 crc kubenswrapper[5052]: I1203 11:47:21.867209 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-lcjjc\" (UniqueName: \"kubernetes.io/projected/0aefd6f6-cbb3-4c45-9f69-7124e43255e3-kube-api-access-lcjjc\") pod \"barbican5976-account-delete-q6j5s\" (UID: \"0aefd6f6-cbb3-4c45-9f69-7124e43255e3\") " pod="openstack-kuttl-tests/barbican5976-account-delete-q6j5s" Dec 03 11:47:21 crc kubenswrapper[5052]: I1203 11:47:21.867501 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/0aefd6f6-cbb3-4c45-9f69-7124e43255e3-operator-scripts\") pod \"barbican5976-account-delete-q6j5s\" (UID: \"0aefd6f6-cbb3-4c45-9f69-7124e43255e3\") " pod="openstack-kuttl-tests/barbican5976-account-delete-q6j5s" Dec 03 11:47:21 crc kubenswrapper[5052]: E1203 11:47:21.870361 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/combined-ca-bundle: secret "combined-ca-bundle" not found Dec 03 11:47:21 crc kubenswrapper[5052]: E1203 11:47:21.870421 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/84b8e975-30ee-42c6-8952-25aebf2c378b-combined-ca-bundle podName:84b8e975-30ee-42c6-8952-25aebf2c378b nodeName:}" failed. No retries permitted until 2025-12-03 11:47:22.370404717 +0000 UTC m=+1632.958646481 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "combined-ca-bundle" (UniqueName: "kubernetes.io/secret/84b8e975-30ee-42c6-8952-25aebf2c378b-combined-ca-bundle") pod "nova-cell1-novncproxy-0" (UID: "84b8e975-30ee-42c6-8952-25aebf2c378b") : secret "combined-ca-bundle" not found Dec 03 11:47:21 crc kubenswrapper[5052]: E1203 11:47:21.871490 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-nova-novncproxy-cell1-public-svc: secret "cert-nova-novncproxy-cell1-public-svc" not found Dec 03 11:47:21 crc kubenswrapper[5052]: E1203 11:47:21.871583 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/84b8e975-30ee-42c6-8952-25aebf2c378b-nova-novncproxy-tls-certs podName:84b8e975-30ee-42c6-8952-25aebf2c378b nodeName:}" failed. No retries permitted until 2025-12-03 11:47:22.37156537 +0000 UTC m=+1632.959807134 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "nova-novncproxy-tls-certs" (UniqueName: "kubernetes.io/secret/84b8e975-30ee-42c6-8952-25aebf2c378b-nova-novncproxy-tls-certs") pod "nova-cell1-novncproxy-0" (UID: "84b8e975-30ee-42c6-8952-25aebf2c378b") : secret "cert-nova-novncproxy-cell1-public-svc" not found Dec 03 11:47:21 crc kubenswrapper[5052]: I1203 11:47:21.872011 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/cinder-db-sync-qv4bk"] Dec 03 11:47:21 crc kubenswrapper[5052]: E1203 11:47:21.872299 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cinder-config-data: secret "cinder-config-data" not found Dec 03 11:47:21 crc kubenswrapper[5052]: E1203 11:47:21.872328 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e947107f-104b-4699-a994-bc77285f0cf5-config-data podName:e947107f-104b-4699-a994-bc77285f0cf5 nodeName:}" failed. No retries permitted until 2025-12-03 11:47:22.372318802 +0000 UTC m=+1632.960560566 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "config-data" (UniqueName: "kubernetes.io/secret/e947107f-104b-4699-a994-bc77285f0cf5-config-data") pod "cinder-scheduler-0" (UID: "e947107f-104b-4699-a994-bc77285f0cf5") : secret "cinder-config-data" not found Dec 03 11:47:21 crc kubenswrapper[5052]: E1203 11:47:21.872395 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/combined-ca-bundle: secret "combined-ca-bundle" not found Dec 03 11:47:21 crc kubenswrapper[5052]: E1203 11:47:21.872542 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e947107f-104b-4699-a994-bc77285f0cf5-combined-ca-bundle podName:e947107f-104b-4699-a994-bc77285f0cf5 nodeName:}" failed. No retries permitted until 2025-12-03 11:47:22.372409864 +0000 UTC m=+1632.960651628 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "combined-ca-bundle" (UniqueName: "kubernetes.io/secret/e947107f-104b-4699-a994-bc77285f0cf5-combined-ca-bundle") pod "cinder-scheduler-0" (UID: "e947107f-104b-4699-a994-bc77285f0cf5") : secret "combined-ca-bundle" not found Dec 03 11:47:21 crc kubenswrapper[5052]: E1203 11:47:21.873748 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cinder-scheduler-config-data: secret "cinder-scheduler-config-data" not found Dec 03 11:47:21 crc kubenswrapper[5052]: E1203 11:47:21.873813 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e947107f-104b-4699-a994-bc77285f0cf5-config-data-custom podName:e947107f-104b-4699-a994-bc77285f0cf5 nodeName:}" failed. No retries permitted until 2025-12-03 11:47:22.373793014 +0000 UTC m=+1632.962034838 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "config-data-custom" (UniqueName: "kubernetes.io/secret/e947107f-104b-4699-a994-bc77285f0cf5-config-data-custom") pod "cinder-scheduler-0" (UID: "e947107f-104b-4699-a994-bc77285f0cf5") : secret "cinder-scheduler-config-data" not found Dec 03 11:47:21 crc kubenswrapper[5052]: E1203 11:47:21.873812 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-nova-novncproxy-cell1-vencrypt: secret "cert-nova-novncproxy-cell1-vencrypt" not found Dec 03 11:47:21 crc kubenswrapper[5052]: E1203 11:47:21.873855 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/84b8e975-30ee-42c6-8952-25aebf2c378b-vencrypt-tls-certs podName:84b8e975-30ee-42c6-8952-25aebf2c378b nodeName:}" failed. No retries permitted until 2025-12-03 11:47:22.373845905 +0000 UTC m=+1632.962087669 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "vencrypt-tls-certs" (UniqueName: "kubernetes.io/secret/84b8e975-30ee-42c6-8952-25aebf2c378b-vencrypt-tls-certs") pod "nova-cell1-novncproxy-0" (UID: "84b8e975-30ee-42c6-8952-25aebf2c378b") : secret "cert-nova-novncproxy-cell1-vencrypt" not found Dec 03 11:47:21 crc kubenswrapper[5052]: E1203 11:47:21.873964 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cinder-scripts: secret "cinder-scripts" not found Dec 03 11:47:21 crc kubenswrapper[5052]: E1203 11:47:21.874054 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e947107f-104b-4699-a994-bc77285f0cf5-scripts podName:e947107f-104b-4699-a994-bc77285f0cf5 nodeName:}" failed. No retries permitted until 2025-12-03 11:47:22.374041111 +0000 UTC m=+1632.962282935 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "scripts" (UniqueName: "kubernetes.io/secret/e947107f-104b-4699-a994-bc77285f0cf5-scripts") pod "cinder-scheduler-0" (UID: "e947107f-104b-4699-a994-bc77285f0cf5") : secret "cinder-scripts" not found Dec 03 11:47:21 crc kubenswrapper[5052]: I1203 11:47:21.876503 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/0aefd6f6-cbb3-4c45-9f69-7124e43255e3-operator-scripts\") pod \"barbican5976-account-delete-q6j5s\" (UID: \"0aefd6f6-cbb3-4c45-9f69-7124e43255e3\") " pod="openstack-kuttl-tests/barbican5976-account-delete-q6j5s" Dec 03 11:47:21 crc kubenswrapper[5052]: I1203 11:47:21.901494 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-lcjjc\" (UniqueName: \"kubernetes.io/projected/0aefd6f6-cbb3-4c45-9f69-7124e43255e3-kube-api-access-lcjjc\") pod \"barbican5976-account-delete-q6j5s\" (UID: \"0aefd6f6-cbb3-4c45-9f69-7124e43255e3\") " pod="openstack-kuttl-tests/barbican5976-account-delete-q6j5s" Dec 03 11:47:21 crc kubenswrapper[5052]: I1203 11:47:21.917506 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-sb-0"] Dec 03 11:47:21 crc kubenswrapper[5052]: I1203 11:47:21.918246 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ovsdbserver-sb-0" podUID="9bdd2626-de50-4513-8313-3b25347ff641" containerName="openstack-network-exporter" containerID="cri-o://d6c0926e352f7c2fd9039729545261adf7fb3d6ef95c52edff8cf47f36d45db5" gracePeriod=300 Dec 03 11:47:21 crc kubenswrapper[5052]: I1203 11:47:21.950644 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/placement-db-sync-jlwkf"] Dec 03 11:47:22 crc kubenswrapper[5052]: I1203 11:47:22.000274 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/neutron-db-sync-5m4kd"] Dec 03 11:47:22 crc kubenswrapper[5052]: I1203 11:47:22.020272 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ovsdbserver-sb-0" podUID="9bdd2626-de50-4513-8313-3b25347ff641" containerName="ovsdbserver-sb" containerID="cri-o://5e28858cdce596489bc28d8ac4d74cd0163a29ba280b7ffd53c1e8c3d713fddb" gracePeriod=300 Dec 03 11:47:22 crc kubenswrapper[5052]: I1203 11:47:22.059674 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/neutron-db-sync-5m4kd"] Dec 03 11:47:22 crc kubenswrapper[5052]: I1203 11:47:22.077827 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican5976-account-delete-q6j5s"] Dec 03 11:47:22 crc kubenswrapper[5052]: I1203 11:47:22.079627 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican5976-account-delete-q6j5s" Dec 03 11:47:22 crc kubenswrapper[5052]: I1203 11:47:22.106018 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-db-sync-gmcqn"] Dec 03 11:47:22 crc kubenswrapper[5052]: I1203 11:47:22.128909 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/barbican-db-sync-gmcqn"] Dec 03 11:47:22 crc kubenswrapper[5052]: I1203 11:47:22.151708 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 11:47:22 crc kubenswrapper[5052]: I1203 11:47:22.153669 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-internal-api-0" podUID="0914ebfe-13c6-4309-bd10-a67850124e3f" containerName="glance-log" containerID="cri-o://295a2704643ed1264e3a64ad18e13a582bf7cc635129de442187ed9d461c917a" gracePeriod=30 Dec 03 11:47:22 crc kubenswrapper[5052]: I1203 11:47:22.154208 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-internal-api-0" podUID="0914ebfe-13c6-4309-bd10-a67850124e3f" containerName="glance-httpd" containerID="cri-o://cc31e16d3323f33d812b30383dad0c83a6bb71816abde9316f28a1841980791c" gracePeriod=30 Dec 03 11:47:22 crc kubenswrapper[5052]: I1203 11:47:22.173007 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/novaapi5919-account-delete-qkvzd"] Dec 03 11:47:22 crc kubenswrapper[5052]: I1203 11:47:22.176421 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/novaapi5919-account-delete-qkvzd" Dec 03 11:47:22 crc kubenswrapper[5052]: I1203 11:47:22.200939 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/novaapi5919-account-delete-qkvzd"] Dec 03 11:47:22 crc kubenswrapper[5052]: I1203 11:47:22.218711 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/swift-storage-0"] Dec 03 11:47:22 crc kubenswrapper[5052]: I1203 11:47:22.219215 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="93ca3376-6916-4fc9-8b76-7cd9a930865e" containerName="account-server" containerID="cri-o://15eb357802bb079afbd7911a3c2b816f45034279f8a2a7e70d89277c5699b052" gracePeriod=30 Dec 03 11:47:22 crc kubenswrapper[5052]: I1203 11:47:22.219296 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="93ca3376-6916-4fc9-8b76-7cd9a930865e" containerName="object-server" containerID="cri-o://f561321b0fdca1154a6977a337503c90f840416e5bc0fa39babc9ea06bd5b9bf" gracePeriod=30 Dec 03 11:47:22 crc kubenswrapper[5052]: I1203 11:47:22.219337 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="93ca3376-6916-4fc9-8b76-7cd9a930865e" containerName="account-reaper" containerID="cri-o://08d93d7b7a336f74cf614cfa15c40191bdfb66f13fc15d41dcf943b57d2e92db" gracePeriod=30 Dec 03 11:47:22 crc kubenswrapper[5052]: I1203 11:47:22.219376 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="93ca3376-6916-4fc9-8b76-7cd9a930865e" containerName="account-auditor" containerID="cri-o://e94a57b11b76a48c619f83900e8a10f2bdcc2a6fc844e2f0bb06dd5042373965" gracePeriod=30 Dec 03 11:47:22 crc kubenswrapper[5052]: I1203 11:47:22.219339 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="93ca3376-6916-4fc9-8b76-7cd9a930865e" containerName="container-auditor" containerID="cri-o://767653c60f88687aa9e17c73cfb59cad0f979a47f7e0eb1bbebd1374e7223b77" gracePeriod=30 Dec 03 11:47:22 crc kubenswrapper[5052]: I1203 11:47:22.219408 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="93ca3376-6916-4fc9-8b76-7cd9a930865e" containerName="account-replicator" containerID="cri-o://f13880839e746af36615e3cd0a68485f5974bfc02f15aacef8d8c58a4cd8e970" gracePeriod=30 Dec 03 11:47:22 crc kubenswrapper[5052]: I1203 11:47:22.219486 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="93ca3376-6916-4fc9-8b76-7cd9a930865e" containerName="container-updater" containerID="cri-o://0049fb7f1583b92a6e9922087915b30b055b3e38241dbdef833f7e931c597b18" gracePeriod=30 Dec 03 11:47:22 crc kubenswrapper[5052]: I1203 11:47:22.219532 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="93ca3376-6916-4fc9-8b76-7cd9a930865e" containerName="object-expirer" containerID="cri-o://dcf4a9a4ecfc80507c16a16774b51db2868255753c35cb58f980eb45993a0c29" gracePeriod=30 Dec 03 11:47:22 crc kubenswrapper[5052]: I1203 11:47:22.219566 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="93ca3376-6916-4fc9-8b76-7cd9a930865e" containerName="swift-recon-cron" containerID="cri-o://57a92592bc143e7e16a01ba9cb34e8bc93f5c7a21ccf4e346f8763a448a75b42" gracePeriod=30 Dec 03 11:47:22 crc kubenswrapper[5052]: I1203 11:47:22.219604 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="93ca3376-6916-4fc9-8b76-7cd9a930865e" containerName="rsync" containerID="cri-o://aca4ad76bb32ce93c82a24cd5e11df447906446998393a1992802096707a00b7" gracePeriod=30 Dec 03 11:47:22 crc kubenswrapper[5052]: I1203 11:47:22.219642 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="93ca3376-6916-4fc9-8b76-7cd9a930865e" containerName="object-auditor" containerID="cri-o://7c70d2e1650a64f27e594fadf25a119e4c9f5f13e151324319b5d9429e343564" gracePeriod=30 Dec 03 11:47:22 crc kubenswrapper[5052]: I1203 11:47:22.219689 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="93ca3376-6916-4fc9-8b76-7cd9a930865e" containerName="object-updater" containerID="cri-o://11b133a6cbb654d7396b3e561b4ee9d5897ec4b55a4240fcf44eff0dc1c666b1" gracePeriod=30 Dec 03 11:47:22 crc kubenswrapper[5052]: I1203 11:47:22.219750 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="93ca3376-6916-4fc9-8b76-7cd9a930865e" containerName="object-replicator" containerID="cri-o://fdfc3ff42d3ff2045f403d93c6d32c7111bdf6fba8059c6ba854561088de3a81" gracePeriod=30 Dec 03 11:47:22 crc kubenswrapper[5052]: I1203 11:47:22.219787 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="93ca3376-6916-4fc9-8b76-7cd9a930865e" containerName="container-replicator" containerID="cri-o://170a73bd9b5a2ea0ebc55d043e19812afef343218daa53630096c9db6698e952" gracePeriod=30 Dec 03 11:47:22 crc kubenswrapper[5052]: I1203 11:47:22.219841 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="93ca3376-6916-4fc9-8b76-7cd9a930865e" containerName="container-server" containerID="cri-o://05a11312e8ec969e44a91b4857ff72727254fdf705f9356f00323d2cea9feb2f" gracePeriod=30 Dec 03 11:47:22 crc kubenswrapper[5052]: I1203 11:47:22.240057 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/novacell0f13b-account-delete-g7m79"] Dec 03 11:47:22 crc kubenswrapper[5052]: I1203 11:47:22.241892 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/novacell0f13b-account-delete-g7m79" Dec 03 11:47:22 crc kubenswrapper[5052]: I1203 11:47:22.300691 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z8p4p\" (UniqueName: \"kubernetes.io/projected/2d3a3054-6c8e-4808-bd5d-01413970167c-kube-api-access-z8p4p\") pod \"novaapi5919-account-delete-qkvzd\" (UID: \"2d3a3054-6c8e-4808-bd5d-01413970167c\") " pod="openstack-kuttl-tests/novaapi5919-account-delete-qkvzd" Dec 03 11:47:22 crc kubenswrapper[5052]: I1203 11:47:22.300789 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/2d3a3054-6c8e-4808-bd5d-01413970167c-operator-scripts\") pod \"novaapi5919-account-delete-qkvzd\" (UID: \"2d3a3054-6c8e-4808-bd5d-01413970167c\") " pod="openstack-kuttl-tests/novaapi5919-account-delete-qkvzd" Dec 03 11:47:22 crc kubenswrapper[5052]: I1203 11:47:22.315232 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/novacell0f13b-account-delete-g7m79"] Dec 03 11:47:22 crc kubenswrapper[5052]: I1203 11:47:22.373083 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/swift-ring-rebalance-r8qtm"] Dec 03 11:47:22 crc kubenswrapper[5052]: I1203 11:47:22.386710 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/swift-ring-rebalance-r8qtm"] Dec 03 11:47:22 crc kubenswrapper[5052]: I1203 11:47:22.397276 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 11:47:22 crc kubenswrapper[5052]: I1203 11:47:22.397847 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-external-api-0" podUID="acea154c-f0bb-4066-bd4d-2dda815f301c" containerName="glance-httpd" containerID="cri-o://e054bf606eb8ede8b913bb4874e3d9c8bf93f40bc4a7246b741ee83f941471a3" gracePeriod=30 Dec 03 11:47:22 crc kubenswrapper[5052]: I1203 11:47:22.398123 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-external-api-0" podUID="acea154c-f0bb-4066-bd4d-2dda815f301c" containerName="glance-log" containerID="cri-o://51504add0a6672a10359e1a4fc407e6171b498dd50bf1382d23a8247c04ea2fe" gracePeriod=30 Dec 03 11:47:22 crc kubenswrapper[5052]: I1203 11:47:22.404319 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/e5339997-33d5-4794-8fa6-042b63e9391e-operator-scripts\") pod \"novacell0f13b-account-delete-g7m79\" (UID: \"e5339997-33d5-4794-8fa6-042b63e9391e\") " pod="openstack-kuttl-tests/novacell0f13b-account-delete-g7m79" Dec 03 11:47:22 crc kubenswrapper[5052]: I1203 11:47:22.404445 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-z8p4p\" (UniqueName: \"kubernetes.io/projected/2d3a3054-6c8e-4808-bd5d-01413970167c-kube-api-access-z8p4p\") pod \"novaapi5919-account-delete-qkvzd\" (UID: \"2d3a3054-6c8e-4808-bd5d-01413970167c\") " pod="openstack-kuttl-tests/novaapi5919-account-delete-qkvzd" Dec 03 11:47:22 crc kubenswrapper[5052]: E1203 11:47:22.404536 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cinder-scripts: secret "cinder-scripts" not found Dec 03 11:47:22 crc kubenswrapper[5052]: I1203 11:47:22.404576 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/2d3a3054-6c8e-4808-bd5d-01413970167c-operator-scripts\") pod \"novaapi5919-account-delete-qkvzd\" (UID: \"2d3a3054-6c8e-4808-bd5d-01413970167c\") " pod="openstack-kuttl-tests/novaapi5919-account-delete-qkvzd" Dec 03 11:47:22 crc kubenswrapper[5052]: E1203 11:47:22.404599 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e947107f-104b-4699-a994-bc77285f0cf5-scripts podName:e947107f-104b-4699-a994-bc77285f0cf5 nodeName:}" failed. No retries permitted until 2025-12-03 11:47:23.404583207 +0000 UTC m=+1633.992825031 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "scripts" (UniqueName: "kubernetes.io/secret/e947107f-104b-4699-a994-bc77285f0cf5-scripts") pod "cinder-scheduler-0" (UID: "e947107f-104b-4699-a994-bc77285f0cf5") : secret "cinder-scripts" not found Dec 03 11:47:22 crc kubenswrapper[5052]: I1203 11:47:22.404634 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c9h9d\" (UniqueName: \"kubernetes.io/projected/e5339997-33d5-4794-8fa6-042b63e9391e-kube-api-access-c9h9d\") pod \"novacell0f13b-account-delete-g7m79\" (UID: \"e5339997-33d5-4794-8fa6-042b63e9391e\") " pod="openstack-kuttl-tests/novacell0f13b-account-delete-g7m79" Dec 03 11:47:22 crc kubenswrapper[5052]: E1203 11:47:22.404645 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/combined-ca-bundle: secret "combined-ca-bundle" not found Dec 03 11:47:22 crc kubenswrapper[5052]: E1203 11:47:22.404666 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/507c60e5-056c-4661-a7a7-75c4d9bce7de-combined-ca-bundle podName:507c60e5-056c-4661-a7a7-75c4d9bce7de nodeName:}" failed. No retries permitted until 2025-12-03 11:47:24.404660509 +0000 UTC m=+1634.992902273 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "combined-ca-bundle" (UniqueName: "kubernetes.io/secret/507c60e5-056c-4661-a7a7-75c4d9bce7de-combined-ca-bundle") pod "nova-cell1-conductor-0" (UID: "507c60e5-056c-4661-a7a7-75c4d9bce7de") : secret "combined-ca-bundle" not found Dec 03 11:47:22 crc kubenswrapper[5052]: E1203 11:47:22.404702 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/combined-ca-bundle: secret "combined-ca-bundle" not found Dec 03 11:47:22 crc kubenswrapper[5052]: E1203 11:47:22.404668 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-nova-novncproxy-cell1-public-svc: secret "cert-nova-novncproxy-cell1-public-svc" not found Dec 03 11:47:22 crc kubenswrapper[5052]: E1203 11:47:22.404724 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e947107f-104b-4699-a994-bc77285f0cf5-combined-ca-bundle podName:e947107f-104b-4699-a994-bc77285f0cf5 nodeName:}" failed. No retries permitted until 2025-12-03 11:47:23.404717571 +0000 UTC m=+1633.992959335 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "combined-ca-bundle" (UniqueName: "kubernetes.io/secret/e947107f-104b-4699-a994-bc77285f0cf5-combined-ca-bundle") pod "cinder-scheduler-0" (UID: "e947107f-104b-4699-a994-bc77285f0cf5") : secret "combined-ca-bundle" not found Dec 03 11:47:22 crc kubenswrapper[5052]: E1203 11:47:22.404758 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/84b8e975-30ee-42c6-8952-25aebf2c378b-nova-novncproxy-tls-certs podName:84b8e975-30ee-42c6-8952-25aebf2c378b nodeName:}" failed. No retries permitted until 2025-12-03 11:47:23.404742512 +0000 UTC m=+1633.992984276 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "nova-novncproxy-tls-certs" (UniqueName: "kubernetes.io/secret/84b8e975-30ee-42c6-8952-25aebf2c378b-nova-novncproxy-tls-certs") pod "nova-cell1-novncproxy-0" (UID: "84b8e975-30ee-42c6-8952-25aebf2c378b") : secret "cert-nova-novncproxy-cell1-public-svc" not found Dec 03 11:47:22 crc kubenswrapper[5052]: E1203 11:47:22.404762 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cinder-scheduler-config-data: secret "cinder-scheduler-config-data" not found Dec 03 11:47:22 crc kubenswrapper[5052]: E1203 11:47:22.404773 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/combined-ca-bundle: secret "combined-ca-bundle" not found Dec 03 11:47:22 crc kubenswrapper[5052]: E1203 11:47:22.404814 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/84b8e975-30ee-42c6-8952-25aebf2c378b-combined-ca-bundle podName:84b8e975-30ee-42c6-8952-25aebf2c378b nodeName:}" failed. No retries permitted until 2025-12-03 11:47:23.404802284 +0000 UTC m=+1633.993044138 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "combined-ca-bundle" (UniqueName: "kubernetes.io/secret/84b8e975-30ee-42c6-8952-25aebf2c378b-combined-ca-bundle") pod "nova-cell1-novncproxy-0" (UID: "84b8e975-30ee-42c6-8952-25aebf2c378b") : secret "combined-ca-bundle" not found Dec 03 11:47:22 crc kubenswrapper[5052]: E1203 11:47:22.404837 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cinder-config-data: secret "cinder-config-data" not found Dec 03 11:47:22 crc kubenswrapper[5052]: E1203 11:47:22.404871 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/rabbitmq-cell1-config-data: configmap "rabbitmq-cell1-config-data" not found Dec 03 11:47:22 crc kubenswrapper[5052]: E1203 11:47:22.404886 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e947107f-104b-4699-a994-bc77285f0cf5-config-data podName:e947107f-104b-4699-a994-bc77285f0cf5 nodeName:}" failed. No retries permitted until 2025-12-03 11:47:23.404878236 +0000 UTC m=+1633.993120000 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "config-data" (UniqueName: "kubernetes.io/secret/e947107f-104b-4699-a994-bc77285f0cf5-config-data") pod "cinder-scheduler-0" (UID: "e947107f-104b-4699-a994-bc77285f0cf5") : secret "cinder-config-data" not found Dec 03 11:47:22 crc kubenswrapper[5052]: E1203 11:47:22.404923 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2766f489-9217-47ae-a595-869a26389587-config-data podName:2766f489-9217-47ae-a595-869a26389587 nodeName:}" failed. No retries permitted until 2025-12-03 11:47:24.404908267 +0000 UTC m=+1634.993150031 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "config-data" (UniqueName: "kubernetes.io/configmap/2766f489-9217-47ae-a595-869a26389587-config-data") pod "rabbitmq-cell1-server-0" (UID: "2766f489-9217-47ae-a595-869a26389587") : configmap "rabbitmq-cell1-config-data" not found Dec 03 11:47:22 crc kubenswrapper[5052]: E1203 11:47:22.404960 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-nova-novncproxy-cell1-vencrypt: secret "cert-nova-novncproxy-cell1-vencrypt" not found Dec 03 11:47:22 crc kubenswrapper[5052]: E1203 11:47:22.404984 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/84b8e975-30ee-42c6-8952-25aebf2c378b-vencrypt-tls-certs podName:84b8e975-30ee-42c6-8952-25aebf2c378b nodeName:}" failed. No retries permitted until 2025-12-03 11:47:23.404976059 +0000 UTC m=+1633.993217823 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "vencrypt-tls-certs" (UniqueName: "kubernetes.io/secret/84b8e975-30ee-42c6-8952-25aebf2c378b-vencrypt-tls-certs") pod "nova-cell1-novncproxy-0" (UID: "84b8e975-30ee-42c6-8952-25aebf2c378b") : secret "cert-nova-novncproxy-cell1-vencrypt" not found Dec 03 11:47:22 crc kubenswrapper[5052]: I1203 11:47:22.406173 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/2d3a3054-6c8e-4808-bd5d-01413970167c-operator-scripts\") pod \"novaapi5919-account-delete-qkvzd\" (UID: \"2d3a3054-6c8e-4808-bd5d-01413970167c\") " pod="openstack-kuttl-tests/novaapi5919-account-delete-qkvzd" Dec 03 11:47:22 crc kubenswrapper[5052]: E1203 11:47:22.406466 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e947107f-104b-4699-a994-bc77285f0cf5-config-data-custom podName:e947107f-104b-4699-a994-bc77285f0cf5 nodeName:}" failed. No retries permitted until 2025-12-03 11:47:23.406450141 +0000 UTC m=+1633.994691935 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "config-data-custom" (UniqueName: "kubernetes.io/secret/e947107f-104b-4699-a994-bc77285f0cf5-config-data-custom") pod "cinder-scheduler-0" (UID: "e947107f-104b-4699-a994-bc77285f0cf5") : secret "cinder-scheduler-config-data" not found Dec 03 11:47:22 crc kubenswrapper[5052]: I1203 11:47:22.408619 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 11:47:22 crc kubenswrapper[5052]: I1203 11:47:22.430878 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/placement-689c6465d6-nsjrx"] Dec 03 11:47:22 crc kubenswrapper[5052]: I1203 11:47:22.431322 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/placement-689c6465d6-nsjrx" podUID="0183ebb5-b91b-443a-9535-2008c9207ab1" containerName="placement-log" containerID="cri-o://32cafa0afafbb84110820599defab0c52864ba6ace469cdde0c2e43031966639" gracePeriod=30 Dec 03 11:47:22 crc kubenswrapper[5052]: I1203 11:47:22.431706 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/placement-689c6465d6-nsjrx" podUID="0183ebb5-b91b-443a-9535-2008c9207ab1" containerName="placement-api" containerID="cri-o://81c366447116b43e07821471a9d38ae69fcdb20b1db58e745b91476f5acbbc2f" gracePeriod=30 Dec 03 11:47:22 crc kubenswrapper[5052]: I1203 11:47:22.438576 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-z8p4p\" (UniqueName: \"kubernetes.io/projected/2d3a3054-6c8e-4808-bd5d-01413970167c-kube-api-access-z8p4p\") pod \"novaapi5919-account-delete-qkvzd\" (UID: \"2d3a3054-6c8e-4808-bd5d-01413970167c\") " pod="openstack-kuttl-tests/novaapi5919-account-delete-qkvzd" Dec 03 11:47:22 crc kubenswrapper[5052]: I1203 11:47:22.442297 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-lfl6r"] Dec 03 11:47:22 crc kubenswrapper[5052]: I1203 11:47:22.444583 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-lfl6r" Dec 03 11:47:22 crc kubenswrapper[5052]: I1203 11:47:22.496728 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/neutron-549dfc49dd-qtk5n"] Dec 03 11:47:22 crc kubenswrapper[5052]: I1203 11:47:22.497181 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/neutron-549dfc49dd-qtk5n" podUID="a4fa0509-8f85-462f-b7c3-4384346252bc" containerName="neutron-api" containerID="cri-o://a10887f92ee55d6e2db028846bedc75386b0fc5b0814cc18643f424e258d8b3c" gracePeriod=30 Dec 03 11:47:22 crc kubenswrapper[5052]: I1203 11:47:22.503581 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/neutron-549dfc49dd-qtk5n" podUID="a4fa0509-8f85-462f-b7c3-4384346252bc" containerName="neutron-httpd" containerID="cri-o://9ca8e5c7ceff313c395a8a6c7655cdc02bfef848fc00d1d1281da272c3a6f083" gracePeriod=30 Dec 03 11:47:22 crc kubenswrapper[5052]: I1203 11:47:22.508043 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-c9h9d\" (UniqueName: \"kubernetes.io/projected/e5339997-33d5-4794-8fa6-042b63e9391e-kube-api-access-c9h9d\") pod \"novacell0f13b-account-delete-g7m79\" (UID: \"e5339997-33d5-4794-8fa6-042b63e9391e\") " pod="openstack-kuttl-tests/novacell0f13b-account-delete-g7m79" Dec 03 11:47:22 crc kubenswrapper[5052]: I1203 11:47:22.508155 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/93fb4fd0-aa18-4a2d-bfb4-b65bbc378ae5-config\") pod \"dnsmasq-dnsmasq-84b9f45d47-lfl6r\" (UID: \"93fb4fd0-aa18-4a2d-bfb4-b65bbc378ae5\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-lfl6r" Dec 03 11:47:22 crc kubenswrapper[5052]: I1203 11:47:22.508189 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/93fb4fd0-aa18-4a2d-bfb4-b65bbc378ae5-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-84b9f45d47-lfl6r\" (UID: \"93fb4fd0-aa18-4a2d-bfb4-b65bbc378ae5\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-lfl6r" Dec 03 11:47:22 crc kubenswrapper[5052]: I1203 11:47:22.508273 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/e5339997-33d5-4794-8fa6-042b63e9391e-operator-scripts\") pod \"novacell0f13b-account-delete-g7m79\" (UID: \"e5339997-33d5-4794-8fa6-042b63e9391e\") " pod="openstack-kuttl-tests/novacell0f13b-account-delete-g7m79" Dec 03 11:47:22 crc kubenswrapper[5052]: I1203 11:47:22.508298 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hht6w\" (UniqueName: \"kubernetes.io/projected/93fb4fd0-aa18-4a2d-bfb4-b65bbc378ae5-kube-api-access-hht6w\") pod \"dnsmasq-dnsmasq-84b9f45d47-lfl6r\" (UID: \"93fb4fd0-aa18-4a2d-bfb4-b65bbc378ae5\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-lfl6r" Dec 03 11:47:22 crc kubenswrapper[5052]: I1203 11:47:22.514956 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/e5339997-33d5-4794-8fa6-042b63e9391e-operator-scripts\") pod \"novacell0f13b-account-delete-g7m79\" (UID: \"e5339997-33d5-4794-8fa6-042b63e9391e\") " pod="openstack-kuttl-tests/novacell0f13b-account-delete-g7m79" Dec 03 11:47:22 crc kubenswrapper[5052]: I1203 11:47:22.515286 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/novaapi5919-account-delete-qkvzd" Dec 03 11:47:22 crc kubenswrapper[5052]: I1203 11:47:22.530608 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell0-cell-mapping-s49m7"] Dec 03 11:47:22 crc kubenswrapper[5052]: I1203 11:47:22.538377 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell0-cell-mapping-s49m7"] Dec 03 11:47:22 crc kubenswrapper[5052]: I1203 11:47:22.552700 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-c9h9d\" (UniqueName: \"kubernetes.io/projected/e5339997-33d5-4794-8fa6-042b63e9391e-kube-api-access-c9h9d\") pod \"novacell0f13b-account-delete-g7m79\" (UID: \"e5339997-33d5-4794-8fa6-042b63e9391e\") " pod="openstack-kuttl-tests/novacell0f13b-account-delete-g7m79" Dec 03 11:47:22 crc kubenswrapper[5052]: I1203 11:47:22.558397 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-lfl6r"] Dec 03 11:47:22 crc kubenswrapper[5052]: I1203 11:47:22.567301 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell1-cell-mapping-jbzxc"] Dec 03 11:47:22 crc kubenswrapper[5052]: I1203 11:47:22.579076 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell1-cell-mapping-jbzxc"] Dec 03 11:47:22 crc kubenswrapper[5052]: I1203 11:47:22.599336 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 11:47:22 crc kubenswrapper[5052]: I1203 11:47:22.599705 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/cinder-api-0" podUID="00b12739-1437-4af3-ab6d-ad830b67ad2e" containerName="cinder-api-log" containerID="cri-o://ef839a7336e307d7216464809bb35eccbc476e002b4d08e14e48e0259172f844" gracePeriod=30 Dec 03 11:47:22 crc kubenswrapper[5052]: I1203 11:47:22.600416 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/cinder-api-0" podUID="00b12739-1437-4af3-ab6d-ad830b67ad2e" containerName="cinder-api" containerID="cri-o://988a528826a2862aed42fdaece9c254222b91bc11952d4cd54b0b18c778396b2" gracePeriod=30 Dec 03 11:47:22 crc kubenswrapper[5052]: I1203 11:47:22.622684 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/93fb4fd0-aa18-4a2d-bfb4-b65bbc378ae5-config\") pod \"dnsmasq-dnsmasq-84b9f45d47-lfl6r\" (UID: \"93fb4fd0-aa18-4a2d-bfb4-b65bbc378ae5\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-lfl6r" Dec 03 11:47:22 crc kubenswrapper[5052]: I1203 11:47:22.622766 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/93fb4fd0-aa18-4a2d-bfb4-b65bbc378ae5-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-84b9f45d47-lfl6r\" (UID: \"93fb4fd0-aa18-4a2d-bfb4-b65bbc378ae5\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-lfl6r" Dec 03 11:47:22 crc kubenswrapper[5052]: I1203 11:47:22.622953 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-hht6w\" (UniqueName: \"kubernetes.io/projected/93fb4fd0-aa18-4a2d-bfb4-b65bbc378ae5-kube-api-access-hht6w\") pod \"dnsmasq-dnsmasq-84b9f45d47-lfl6r\" (UID: \"93fb4fd0-aa18-4a2d-bfb4-b65bbc378ae5\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-lfl6r" Dec 03 11:47:22 crc kubenswrapper[5052]: I1203 11:47:22.624633 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/93fb4fd0-aa18-4a2d-bfb4-b65bbc378ae5-config\") pod \"dnsmasq-dnsmasq-84b9f45d47-lfl6r\" (UID: \"93fb4fd0-aa18-4a2d-bfb4-b65bbc378ae5\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-lfl6r" Dec 03 11:47:22 crc kubenswrapper[5052]: I1203 11:47:22.625649 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/93fb4fd0-aa18-4a2d-bfb4-b65bbc378ae5-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-84b9f45d47-lfl6r\" (UID: \"93fb4fd0-aa18-4a2d-bfb4-b65bbc378ae5\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-lfl6r" Dec 03 11:47:22 crc kubenswrapper[5052]: I1203 11:47:22.643684 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/novacell0f13b-account-delete-g7m79" Dec 03 11:47:22 crc kubenswrapper[5052]: I1203 11:47:22.673914 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-hht6w\" (UniqueName: \"kubernetes.io/projected/93fb4fd0-aa18-4a2d-bfb4-b65bbc378ae5-kube-api-access-hht6w\") pod \"dnsmasq-dnsmasq-84b9f45d47-lfl6r\" (UID: \"93fb4fd0-aa18-4a2d-bfb4-b65bbc378ae5\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-lfl6r" Dec 03 11:47:22 crc kubenswrapper[5052]: I1203 11:47:22.687719 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/swift-proxy-6fdf66df4-gsqlq"] Dec 03 11:47:22 crc kubenswrapper[5052]: I1203 11:47:22.688069 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-proxy-6fdf66df4-gsqlq" podUID="84957e18-5747-4786-9c4f-6af023122726" containerName="proxy-httpd" containerID="cri-o://07cccd5afb2592c0003b92ef2faa428ad9bc5bc71a62a78385e0f7a553335aac" gracePeriod=30 Dec 03 11:47:22 crc kubenswrapper[5052]: I1203 11:47:22.688759 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-proxy-6fdf66df4-gsqlq" podUID="84957e18-5747-4786-9c4f-6af023122726" containerName="proxy-server" containerID="cri-o://9a14693162272f7ccaeacad5515545206de3c0c0552776c0d5a91a5bb80088eb" gracePeriod=30 Dec 03 11:47:22 crc kubenswrapper[5052]: I1203 11:47:22.725712 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-lfl6r" Dec 03 11:47:22 crc kubenswrapper[5052]: I1203 11:47:22.761900 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_ovsdbserver-nb-0_68f829f3-e972-4168-8b75-27c086a91142/ovsdbserver-nb/0.log" Dec 03 11:47:22 crc kubenswrapper[5052]: I1203 11:47:22.761989 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 11:47:22 crc kubenswrapper[5052]: I1203 11:47:22.805583 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 11:47:22 crc kubenswrapper[5052]: I1203 11:47:22.805876 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-api-0" podUID="f7123213-06e1-4b26-9892-c16c3d49c0ab" containerName="nova-api-log" containerID="cri-o://fa0c2ba3f86bf1385c8a82da17792deaa7cbb1c9c5250a02c070da47b2664562" gracePeriod=30 Dec 03 11:47:22 crc kubenswrapper[5052]: I1203 11:47:22.806080 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-api-0" podUID="f7123213-06e1-4b26-9892-c16c3d49c0ab" containerName="nova-api-api" containerID="cri-o://25b77fa7d98d8777337c1b92110af89807efa838eb759959d62314939c420ccf" gracePeriod=30 Dec 03 11:47:22 crc kubenswrapper[5052]: I1203 11:47:22.851614 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-worker-79689f8957-qmkdm"] Dec 03 11:47:22 crc kubenswrapper[5052]: I1203 11:47:22.854712 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/barbican-worker-79689f8957-qmkdm" podUID="cadac9a6-1d82-403a-b7a0-692a424dd0e2" containerName="barbican-worker-log" containerID="cri-o://4d318f4a7ce116ebcf5a2ea62f382127bf1d126d796753d913aaafd4044c08be" gracePeriod=30 Dec 03 11:47:22 crc kubenswrapper[5052]: I1203 11:47:22.856969 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/barbican-worker-79689f8957-qmkdm" podUID="cadac9a6-1d82-403a-b7a0-692a424dd0e2" containerName="barbican-worker" containerID="cri-o://38aac1236f884983fd56130cde1141029030d5995e585474bc43e38342fcd566" gracePeriod=30 Dec 03 11:47:22 crc kubenswrapper[5052]: I1203 11:47:22.885165 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/openstack-cell1-galera-0"] Dec 03 11:47:22 crc kubenswrapper[5052]: I1203 11:47:22.900252 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 11:47:22 crc kubenswrapper[5052]: I1203 11:47:22.900964 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-metadata-0" podUID="bb51da3b-72cc-4008-aac7-737f6a3031cb" containerName="nova-metadata-log" containerID="cri-o://4568b2e2e54107935eb412f6e606a9e5b4fd360b7f1d0fae94b2b383fd9205d0" gracePeriod=30 Dec 03 11:47:22 crc kubenswrapper[5052]: I1203 11:47:22.902195 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-metadata-0" podUID="bb51da3b-72cc-4008-aac7-737f6a3031cb" containerName="nova-metadata-metadata" containerID="cri-o://b39f7b31c335958d6613651cea9f3ccbfee7c25b7c7641048321f3e460ee0193" gracePeriod=30 Dec 03 11:47:22 crc kubenswrapper[5052]: I1203 11:47:22.918023 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-keystone-listener-6794d844d6-f4tr7"] Dec 03 11:47:22 crc kubenswrapper[5052]: I1203 11:47:22.918364 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/barbican-keystone-listener-6794d844d6-f4tr7" podUID="d8de865a-0f6c-413d-8a37-3ec4d496c4bb" containerName="barbican-keystone-listener-log" containerID="cri-o://775a6cb46d400df5368627d0a86f59d32f62f17aac77b3a75fe7d8c2928b8362" gracePeriod=30 Dec 03 11:47:22 crc kubenswrapper[5052]: I1203 11:47:22.918847 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/barbican-keystone-listener-6794d844d6-f4tr7" podUID="d8de865a-0f6c-413d-8a37-3ec4d496c4bb" containerName="barbican-keystone-listener" containerID="cri-o://def922586d645f9987c75d4862d880ed855e05523f69e809e4d0637bfe42a9d8" gracePeriod=30 Dec 03 11:47:22 crc kubenswrapper[5052]: I1203 11:47:22.934676 5052 generic.go:334] "Generic (PLEG): container finished" podID="93ca3376-6916-4fc9-8b76-7cd9a930865e" containerID="dcf4a9a4ecfc80507c16a16774b51db2868255753c35cb58f980eb45993a0c29" exitCode=0 Dec 03 11:47:22 crc kubenswrapper[5052]: I1203 11:47:22.934711 5052 generic.go:334] "Generic (PLEG): container finished" podID="93ca3376-6916-4fc9-8b76-7cd9a930865e" containerID="11b133a6cbb654d7396b3e561b4ee9d5897ec4b55a4240fcf44eff0dc1c666b1" exitCode=0 Dec 03 11:47:22 crc kubenswrapper[5052]: I1203 11:47:22.934720 5052 generic.go:334] "Generic (PLEG): container finished" podID="93ca3376-6916-4fc9-8b76-7cd9a930865e" containerID="7c70d2e1650a64f27e594fadf25a119e4c9f5f13e151324319b5d9429e343564" exitCode=0 Dec 03 11:47:22 crc kubenswrapper[5052]: I1203 11:47:22.934728 5052 generic.go:334] "Generic (PLEG): container finished" podID="93ca3376-6916-4fc9-8b76-7cd9a930865e" containerID="fdfc3ff42d3ff2045f403d93c6d32c7111bdf6fba8059c6ba854561088de3a81" exitCode=0 Dec 03 11:47:22 crc kubenswrapper[5052]: I1203 11:47:22.934734 5052 generic.go:334] "Generic (PLEG): container finished" podID="93ca3376-6916-4fc9-8b76-7cd9a930865e" containerID="0049fb7f1583b92a6e9922087915b30b055b3e38241dbdef833f7e931c597b18" exitCode=0 Dec 03 11:47:22 crc kubenswrapper[5052]: I1203 11:47:22.934740 5052 generic.go:334] "Generic (PLEG): container finished" podID="93ca3376-6916-4fc9-8b76-7cd9a930865e" containerID="767653c60f88687aa9e17c73cfb59cad0f979a47f7e0eb1bbebd1374e7223b77" exitCode=0 Dec 03 11:47:22 crc kubenswrapper[5052]: I1203 11:47:22.934747 5052 generic.go:334] "Generic (PLEG): container finished" podID="93ca3376-6916-4fc9-8b76-7cd9a930865e" containerID="170a73bd9b5a2ea0ebc55d043e19812afef343218daa53630096c9db6698e952" exitCode=0 Dec 03 11:47:22 crc kubenswrapper[5052]: I1203 11:47:22.934753 5052 generic.go:334] "Generic (PLEG): container finished" podID="93ca3376-6916-4fc9-8b76-7cd9a930865e" containerID="08d93d7b7a336f74cf614cfa15c40191bdfb66f13fc15d41dcf943b57d2e92db" exitCode=0 Dec 03 11:47:22 crc kubenswrapper[5052]: I1203 11:47:22.934759 5052 generic.go:334] "Generic (PLEG): container finished" podID="93ca3376-6916-4fc9-8b76-7cd9a930865e" containerID="e94a57b11b76a48c619f83900e8a10f2bdcc2a6fc844e2f0bb06dd5042373965" exitCode=0 Dec 03 11:47:22 crc kubenswrapper[5052]: I1203 11:47:22.934765 5052 generic.go:334] "Generic (PLEG): container finished" podID="93ca3376-6916-4fc9-8b76-7cd9a930865e" containerID="f13880839e746af36615e3cd0a68485f5974bfc02f15aacef8d8c58a4cd8e970" exitCode=0 Dec 03 11:47:22 crc kubenswrapper[5052]: I1203 11:47:22.934770 5052 generic.go:334] "Generic (PLEG): container finished" podID="93ca3376-6916-4fc9-8b76-7cd9a930865e" containerID="15eb357802bb079afbd7911a3c2b816f45034279f8a2a7e70d89277c5699b052" exitCode=0 Dec 03 11:47:22 crc kubenswrapper[5052]: I1203 11:47:22.934837 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"93ca3376-6916-4fc9-8b76-7cd9a930865e","Type":"ContainerDied","Data":"dcf4a9a4ecfc80507c16a16774b51db2868255753c35cb58f980eb45993a0c29"} Dec 03 11:47:22 crc kubenswrapper[5052]: I1203 11:47:22.934866 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"93ca3376-6916-4fc9-8b76-7cd9a930865e","Type":"ContainerDied","Data":"11b133a6cbb654d7396b3e561b4ee9d5897ec4b55a4240fcf44eff0dc1c666b1"} Dec 03 11:47:22 crc kubenswrapper[5052]: I1203 11:47:22.934876 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"93ca3376-6916-4fc9-8b76-7cd9a930865e","Type":"ContainerDied","Data":"7c70d2e1650a64f27e594fadf25a119e4c9f5f13e151324319b5d9429e343564"} Dec 03 11:47:22 crc kubenswrapper[5052]: I1203 11:47:22.934884 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"93ca3376-6916-4fc9-8b76-7cd9a930865e","Type":"ContainerDied","Data":"fdfc3ff42d3ff2045f403d93c6d32c7111bdf6fba8059c6ba854561088de3a81"} Dec 03 11:47:22 crc kubenswrapper[5052]: I1203 11:47:22.934893 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"93ca3376-6916-4fc9-8b76-7cd9a930865e","Type":"ContainerDied","Data":"0049fb7f1583b92a6e9922087915b30b055b3e38241dbdef833f7e931c597b18"} Dec 03 11:47:22 crc kubenswrapper[5052]: I1203 11:47:22.934902 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"93ca3376-6916-4fc9-8b76-7cd9a930865e","Type":"ContainerDied","Data":"767653c60f88687aa9e17c73cfb59cad0f979a47f7e0eb1bbebd1374e7223b77"} Dec 03 11:47:22 crc kubenswrapper[5052]: I1203 11:47:22.934910 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"93ca3376-6916-4fc9-8b76-7cd9a930865e","Type":"ContainerDied","Data":"170a73bd9b5a2ea0ebc55d043e19812afef343218daa53630096c9db6698e952"} Dec 03 11:47:22 crc kubenswrapper[5052]: I1203 11:47:22.934919 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"93ca3376-6916-4fc9-8b76-7cd9a930865e","Type":"ContainerDied","Data":"08d93d7b7a336f74cf614cfa15c40191bdfb66f13fc15d41dcf943b57d2e92db"} Dec 03 11:47:22 crc kubenswrapper[5052]: I1203 11:47:22.934929 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"93ca3376-6916-4fc9-8b76-7cd9a930865e","Type":"ContainerDied","Data":"e94a57b11b76a48c619f83900e8a10f2bdcc2a6fc844e2f0bb06dd5042373965"} Dec 03 11:47:22 crc kubenswrapper[5052]: I1203 11:47:22.934939 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"93ca3376-6916-4fc9-8b76-7cd9a930865e","Type":"ContainerDied","Data":"f13880839e746af36615e3cd0a68485f5974bfc02f15aacef8d8c58a4cd8e970"} Dec 03 11:47:22 crc kubenswrapper[5052]: I1203 11:47:22.934948 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"93ca3376-6916-4fc9-8b76-7cd9a930865e","Type":"ContainerDied","Data":"15eb357802bb079afbd7911a3c2b816f45034279f8a2a7e70d89277c5699b052"} Dec 03 11:47:22 crc kubenswrapper[5052]: I1203 11:47:22.939891 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/68f829f3-e972-4168-8b75-27c086a91142-config\") pod \"68f829f3-e972-4168-8b75-27c086a91142\" (UID: \"68f829f3-e972-4168-8b75-27c086a91142\") " Dec 03 11:47:22 crc kubenswrapper[5052]: I1203 11:47:22.940654 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/68f829f3-e972-4168-8b75-27c086a91142-metrics-certs-tls-certs\") pod \"68f829f3-e972-4168-8b75-27c086a91142\" (UID: \"68f829f3-e972-4168-8b75-27c086a91142\") " Dec 03 11:47:22 crc kubenswrapper[5052]: I1203 11:47:22.942904 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gb5sd\" (UniqueName: \"kubernetes.io/projected/68f829f3-e972-4168-8b75-27c086a91142-kube-api-access-gb5sd\") pod \"68f829f3-e972-4168-8b75-27c086a91142\" (UID: \"68f829f3-e972-4168-8b75-27c086a91142\") " Dec 03 11:47:22 crc kubenswrapper[5052]: I1203 11:47:22.942999 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovsdbserver-nb-tls-certs\" (UniqueName: \"kubernetes.io/secret/68f829f3-e972-4168-8b75-27c086a91142-ovsdbserver-nb-tls-certs\") pod \"68f829f3-e972-4168-8b75-27c086a91142\" (UID: \"68f829f3-e972-4168-8b75-27c086a91142\") " Dec 03 11:47:22 crc kubenswrapper[5052]: I1203 11:47:22.943109 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/68f829f3-e972-4168-8b75-27c086a91142-scripts\") pod \"68f829f3-e972-4168-8b75-27c086a91142\" (UID: \"68f829f3-e972-4168-8b75-27c086a91142\") " Dec 03 11:47:22 crc kubenswrapper[5052]: I1203 11:47:22.943146 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovndbcluster-nb-etc-ovn\" (UniqueName: \"kubernetes.io/local-volume/local-storage14-crc\") pod \"68f829f3-e972-4168-8b75-27c086a91142\" (UID: \"68f829f3-e972-4168-8b75-27c086a91142\") " Dec 03 11:47:22 crc kubenswrapper[5052]: I1203 11:47:22.943169 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/68f829f3-e972-4168-8b75-27c086a91142-ovsdb-rundir\") pod \"68f829f3-e972-4168-8b75-27c086a91142\" (UID: \"68f829f3-e972-4168-8b75-27c086a91142\") " Dec 03 11:47:22 crc kubenswrapper[5052]: I1203 11:47:22.943201 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/68f829f3-e972-4168-8b75-27c086a91142-combined-ca-bundle\") pod \"68f829f3-e972-4168-8b75-27c086a91142\" (UID: \"68f829f3-e972-4168-8b75-27c086a91142\") " Dec 03 11:47:22 crc kubenswrapper[5052]: I1203 11:47:22.953376 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell1-db-create-t49dl"] Dec 03 11:47:22 crc kubenswrapper[5052]: I1203 11:47:22.954467 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/68f829f3-e972-4168-8b75-27c086a91142-scripts" (OuterVolumeSpecName: "scripts") pod "68f829f3-e972-4168-8b75-27c086a91142" (UID: "68f829f3-e972-4168-8b75-27c086a91142"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:47:22 crc kubenswrapper[5052]: I1203 11:47:22.956320 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/68f829f3-e972-4168-8b75-27c086a91142-ovsdb-rundir" (OuterVolumeSpecName: "ovsdb-rundir") pod "68f829f3-e972-4168-8b75-27c086a91142" (UID: "68f829f3-e972-4168-8b75-27c086a91142"). InnerVolumeSpecName "ovsdb-rundir". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:47:22 crc kubenswrapper[5052]: I1203 11:47:22.960050 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-api-7cfb847cb8-jn85t"] Dec 03 11:47:22 crc kubenswrapper[5052]: I1203 11:47:22.960355 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/barbican-api-7cfb847cb8-jn85t" podUID="99f96963-d835-4445-84af-0976f1ba0701" containerName="barbican-api-log" containerID="cri-o://5592513a2c756f332087769178ec0527d69dee3863e8ebc7da5e5d5ac9fefa4f" gracePeriod=30 Dec 03 11:47:22 crc kubenswrapper[5052]: I1203 11:47:22.960914 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/barbican-api-7cfb847cb8-jn85t" podUID="99f96963-d835-4445-84af-0976f1ba0701" containerName="barbican-api" containerID="cri-o://95b8a0e92f2ae27cf80dcce2b2af098739e919963ebae7c22edd8d9c46c89ddc" gracePeriod=30 Dec 03 11:47:22 crc kubenswrapper[5052]: I1203 11:47:22.963016 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/68f829f3-e972-4168-8b75-27c086a91142-config" (OuterVolumeSpecName: "config") pod "68f829f3-e972-4168-8b75-27c086a91142" (UID: "68f829f3-e972-4168-8b75-27c086a91142"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:47:22 crc kubenswrapper[5052]: I1203 11:47:22.964991 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/68f829f3-e972-4168-8b75-27c086a91142-kube-api-access-gb5sd" (OuterVolumeSpecName: "kube-api-access-gb5sd") pod "68f829f3-e972-4168-8b75-27c086a91142" (UID: "68f829f3-e972-4168-8b75-27c086a91142"). InnerVolumeSpecName "kube-api-access-gb5sd". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:47:22 crc kubenswrapper[5052]: I1203 11:47:22.968552 5052 generic.go:334] "Generic (PLEG): container finished" podID="a9f103a5-8b29-4152-a711-60891b9aeefc" containerID="6b2a059106f1fcc4b1e035c9cf505bb5bc72d3f6e38d77875ea0e33ccbc713f1" exitCode=137 Dec 03 11:47:22 crc kubenswrapper[5052]: I1203 11:47:22.977737 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage14-crc" (OuterVolumeSpecName: "ovndbcluster-nb-etc-ovn") pod "68f829f3-e972-4168-8b75-27c086a91142" (UID: "68f829f3-e972-4168-8b75-27c086a91142"). InnerVolumeSpecName "local-storage14-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 11:47:23 crc kubenswrapper[5052]: I1203 11:47:22.983831 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell1-db-create-t49dl"] Dec 03 11:47:23 crc kubenswrapper[5052]: I1203 11:47:23.014342 5052 generic.go:334] "Generic (PLEG): container finished" podID="acea154c-f0bb-4066-bd4d-2dda815f301c" containerID="51504add0a6672a10359e1a4fc407e6171b498dd50bf1382d23a8247c04ea2fe" exitCode=143 Dec 03 11:47:23 crc kubenswrapper[5052]: I1203 11:47:23.014481 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"acea154c-f0bb-4066-bd4d-2dda815f301c","Type":"ContainerDied","Data":"51504add0a6672a10359e1a4fc407e6171b498dd50bf1382d23a8247c04ea2fe"} Dec 03 11:47:23 crc kubenswrapper[5052]: I1203 11:47:23.017042 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell1-1ea3-account-create-update-4x6kr"] Dec 03 11:47:23 crc kubenswrapper[5052]: I1203 11:47:23.023358 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_ovsdbserver-nb-0_68f829f3-e972-4168-8b75-27c086a91142/ovsdbserver-nb/0.log" Dec 03 11:47:23 crc kubenswrapper[5052]: I1203 11:47:23.023396 5052 generic.go:334] "Generic (PLEG): container finished" podID="68f829f3-e972-4168-8b75-27c086a91142" containerID="630e406fa52125d27b16eb3de57c1b6dc5a497b24481482e2ba05d01fcd553fc" exitCode=2 Dec 03 11:47:23 crc kubenswrapper[5052]: I1203 11:47:23.023413 5052 generic.go:334] "Generic (PLEG): container finished" podID="68f829f3-e972-4168-8b75-27c086a91142" containerID="57f10ab9d3a1c544593f058c6724e34ab08ea61ce5e3535d93586cc3d5834467" exitCode=143 Dec 03 11:47:23 crc kubenswrapper[5052]: I1203 11:47:23.023507 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 11:47:23 crc kubenswrapper[5052]: I1203 11:47:23.024088 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-nb-0" event={"ID":"68f829f3-e972-4168-8b75-27c086a91142","Type":"ContainerDied","Data":"630e406fa52125d27b16eb3de57c1b6dc5a497b24481482e2ba05d01fcd553fc"} Dec 03 11:47:23 crc kubenswrapper[5052]: I1203 11:47:23.024161 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-nb-0" event={"ID":"68f829f3-e972-4168-8b75-27c086a91142","Type":"ContainerDied","Data":"57f10ab9d3a1c544593f058c6724e34ab08ea61ce5e3535d93586cc3d5834467"} Dec 03 11:47:23 crc kubenswrapper[5052]: I1203 11:47:23.024185 5052 scope.go:117] "RemoveContainer" containerID="630e406fa52125d27b16eb3de57c1b6dc5a497b24481482e2ba05d01fcd553fc" Dec 03 11:47:23 crc kubenswrapper[5052]: I1203 11:47:23.055001 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/68f829f3-e972-4168-8b75-27c086a91142-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:23 crc kubenswrapper[5052]: I1203 11:47:23.055077 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage14-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage14-crc\") on node \"crc\" " Dec 03 11:47:23 crc kubenswrapper[5052]: I1203 11:47:23.055091 5052 reconciler_common.go:293] "Volume detached for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/68f829f3-e972-4168-8b75-27c086a91142-ovsdb-rundir\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:23 crc kubenswrapper[5052]: I1203 11:47:23.055102 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/68f829f3-e972-4168-8b75-27c086a91142-config\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:23 crc kubenswrapper[5052]: I1203 11:47:23.055117 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-gb5sd\" (UniqueName: \"kubernetes.io/projected/68f829f3-e972-4168-8b75-27c086a91142-kube-api-access-gb5sd\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:23 crc kubenswrapper[5052]: I1203 11:47:23.080184 5052 generic.go:334] "Generic (PLEG): container finished" podID="0183ebb5-b91b-443a-9535-2008c9207ab1" containerID="32cafa0afafbb84110820599defab0c52864ba6ace469cdde0c2e43031966639" exitCode=143 Dec 03 11:47:23 crc kubenswrapper[5052]: I1203 11:47:23.080380 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-689c6465d6-nsjrx" event={"ID":"0183ebb5-b91b-443a-9535-2008c9207ab1","Type":"ContainerDied","Data":"32cafa0afafbb84110820599defab0c52864ba6ace469cdde0c2e43031966639"} Dec 03 11:47:23 crc kubenswrapper[5052]: I1203 11:47:23.116436 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell1-1ea3-account-create-update-4x6kr"] Dec 03 11:47:23 crc kubenswrapper[5052]: I1203 11:47:23.158069 5052 generic.go:334] "Generic (PLEG): container finished" podID="0914ebfe-13c6-4309-bd10-a67850124e3f" containerID="295a2704643ed1264e3a64ad18e13a582bf7cc635129de442187ed9d461c917a" exitCode=143 Dec 03 11:47:23 crc kubenswrapper[5052]: I1203 11:47:23.158204 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 11:47:23 crc kubenswrapper[5052]: I1203 11:47:23.158241 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"0914ebfe-13c6-4309-bd10-a67850124e3f","Type":"ContainerDied","Data":"295a2704643ed1264e3a64ad18e13a582bf7cc635129de442187ed9d461c917a"} Dec 03 11:47:23 crc kubenswrapper[5052]: I1203 11:47:23.186351 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/68f829f3-e972-4168-8b75-27c086a91142-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "68f829f3-e972-4168-8b75-27c086a91142" (UID: "68f829f3-e972-4168-8b75-27c086a91142"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:47:23 crc kubenswrapper[5052]: I1203 11:47:23.189022 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_ovsdbserver-sb-0_9bdd2626-de50-4513-8313-3b25347ff641/ovsdbserver-sb/0.log" Dec 03 11:47:23 crc kubenswrapper[5052]: I1203 11:47:23.189058 5052 generic.go:334] "Generic (PLEG): container finished" podID="9bdd2626-de50-4513-8313-3b25347ff641" containerID="d6c0926e352f7c2fd9039729545261adf7fb3d6ef95c52edff8cf47f36d45db5" exitCode=2 Dec 03 11:47:23 crc kubenswrapper[5052]: I1203 11:47:23.189096 5052 generic.go:334] "Generic (PLEG): container finished" podID="9bdd2626-de50-4513-8313-3b25347ff641" containerID="5e28858cdce596489bc28d8ac4d74cd0163a29ba280b7ffd53c1e8c3d713fddb" exitCode=143 Dec 03 11:47:23 crc kubenswrapper[5052]: I1203 11:47:23.189417 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/cinder-scheduler-0" podUID="e947107f-104b-4699-a994-bc77285f0cf5" containerName="cinder-scheduler" containerID="cri-o://30b3c8981cdec3a0ba91236f985bc49025e43effcb7cbf0b9e7cddcc3ba0a0d9" gracePeriod=30 Dec 03 11:47:23 crc kubenswrapper[5052]: I1203 11:47:23.189552 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-sb-0" event={"ID":"9bdd2626-de50-4513-8313-3b25347ff641","Type":"ContainerDied","Data":"d6c0926e352f7c2fd9039729545261adf7fb3d6ef95c52edff8cf47f36d45db5"} Dec 03 11:47:23 crc kubenswrapper[5052]: I1203 11:47:23.189579 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-sb-0" event={"ID":"9bdd2626-de50-4513-8313-3b25347ff641","Type":"ContainerDied","Data":"5e28858cdce596489bc28d8ac4d74cd0163a29ba280b7ffd53c1e8c3d713fddb"} Dec 03 11:47:23 crc kubenswrapper[5052]: I1203 11:47:23.189745 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" podUID="84b8e975-30ee-42c6-8952-25aebf2c378b" containerName="nova-cell1-novncproxy-novncproxy" containerID="cri-o://57aafbdbe2a86778650bc79b3ec5dc6502116268591f2f9f5ff1056d241c1f66" gracePeriod=30 Dec 03 11:47:23 crc kubenswrapper[5052]: I1203 11:47:23.190996 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/cinder-scheduler-0" podUID="e947107f-104b-4699-a994-bc77285f0cf5" containerName="probe" containerID="cri-o://08b743014f4f3e11912f7da330bbe8c095df94c7cd01737b812783a0bd8af17b" gracePeriod=30 Dec 03 11:47:23 crc kubenswrapper[5052]: I1203 11:47:23.219131 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:47:23 crc kubenswrapper[5052]: I1203 11:47:23.219562 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="1b45577a-92b7-4c67-941b-cfc2fbfa3eb2" containerName="ceilometer-central-agent" containerID="cri-o://76c303acd28c23268bb01ed30bc5fa462afd9585a2872db7a6ce77014ebbad00" gracePeriod=30 Dec 03 11:47:23 crc kubenswrapper[5052]: I1203 11:47:23.219711 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="1b45577a-92b7-4c67-941b-cfc2fbfa3eb2" containerName="proxy-httpd" containerID="cri-o://8c71cc229cea36e3361a6c384c5386d5cc260814f0041348a439548161af1249" gracePeriod=30 Dec 03 11:47:23 crc kubenswrapper[5052]: I1203 11:47:23.219762 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="1b45577a-92b7-4c67-941b-cfc2fbfa3eb2" containerName="sg-core" containerID="cri-o://05471f2484133fbd796cb8f6c5d16314217c42ed6ab7d7d6511de8a201b11eb9" gracePeriod=30 Dec 03 11:47:23 crc kubenswrapper[5052]: I1203 11:47:23.219800 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="1b45577a-92b7-4c67-941b-cfc2fbfa3eb2" containerName="ceilometer-notification-agent" containerID="cri-o://9e68066ce0ef8e9380353738e119a2afc7b38dcd3bd33c671f3778712cfb46e4" gracePeriod=30 Dec 03 11:47:23 crc kubenswrapper[5052]: I1203 11:47:23.221783 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage14-crc" (UniqueName: "kubernetes.io/local-volume/local-storage14-crc") on node "crc" Dec 03 11:47:23 crc kubenswrapper[5052]: I1203 11:47:23.236752 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/kube-state-metrics-0"] Dec 03 11:47:23 crc kubenswrapper[5052]: I1203 11:47:23.236991 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/kube-state-metrics-0" podUID="496a1006-5380-40d1-998f-129a87233300" containerName="kube-state-metrics" containerID="cri-o://64d2ed8863bdee50690f9bd22bda4e036b2111ebb9734c4223ad439fd9d3b384" gracePeriod=30 Dec 03 11:47:23 crc kubenswrapper[5052]: I1203 11:47:23.275970 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 11:47:23 crc kubenswrapper[5052]: I1203 11:47:23.276222 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-scheduler-0" podUID="39b9198e-e3d7-4f85-b6a2-55960bdda82a" containerName="nova-scheduler-scheduler" containerID="cri-o://1dbdda8905493662bc20bc18fcae5f54dd4929f65dec1b16df59f03dbfd76d8a" gracePeriod=30 Dec 03 11:47:23 crc kubenswrapper[5052]: I1203 11:47:23.278887 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage14-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage14-crc\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:23 crc kubenswrapper[5052]: I1203 11:47:23.278929 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/68f829f3-e972-4168-8b75-27c086a91142-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:23 crc kubenswrapper[5052]: I1203 11:47:23.287686 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/keystone-db-sync-plblg"] Dec 03 11:47:23 crc kubenswrapper[5052]: I1203 11:47:23.303186 5052 scope.go:117] "RemoveContainer" containerID="57f10ab9d3a1c544593f058c6724e34ab08ea61ce5e3535d93586cc3d5834467" Dec 03 11:47:23 crc kubenswrapper[5052]: I1203 11:47:23.306032 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/keystone-db-sync-plblg"] Dec 03 11:47:23 crc kubenswrapper[5052]: I1203 11:47:23.335578 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/keystone-bootstrap-fmkmc"] Dec 03 11:47:23 crc kubenswrapper[5052]: I1203 11:47:23.354591 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/keystone-bootstrap-fmkmc"] Dec 03 11:47:23 crc kubenswrapper[5052]: I1203 11:47:23.371630 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/placement9a4f-account-delete-f6sjd"] Dec 03 11:47:23 crc kubenswrapper[5052]: I1203 11:47:23.381299 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron18a7-account-delete-x9lqd"] Dec 03 11:47:23 crc kubenswrapper[5052]: I1203 11:47:23.393659 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/keystone-779b679dc7-nvtdm"] Dec 03 11:47:23 crc kubenswrapper[5052]: I1203 11:47:23.394024 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/keystone-779b679dc7-nvtdm" podUID="65be66f1-c32d-40b5-81a4-a63258364d9d" containerName="keystone-api" containerID="cri-o://2be919af42c62148fa0417519c4eea762d9886b79020bd92b5f10692d1f30f31" gracePeriod=30 Dec 03 11:47:23 crc kubenswrapper[5052]: I1203 11:47:23.410604 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/openstack-galera-0"] Dec 03 11:47:23 crc kubenswrapper[5052]: I1203 11:47:23.414458 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance3554-account-delete-8ghdx"] Dec 03 11:47:23 crc kubenswrapper[5052]: I1203 11:47:23.426262 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/keystone-db-create-h55rj"] Dec 03 11:47:23 crc kubenswrapper[5052]: I1203 11:47:23.446717 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/keystone-db-create-h55rj"] Dec 03 11:47:23 crc kubenswrapper[5052]: I1203 11:47:23.480680 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/keystone-c226-account-create-update-4scft"] Dec 03 11:47:23 crc kubenswrapper[5052]: E1203 11:47:23.483480 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cinder-config-data: secret "cinder-config-data" not found Dec 03 11:47:23 crc kubenswrapper[5052]: E1203 11:47:23.529791 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e947107f-104b-4699-a994-bc77285f0cf5-config-data podName:e947107f-104b-4699-a994-bc77285f0cf5 nodeName:}" failed. No retries permitted until 2025-12-03 11:47:25.529741167 +0000 UTC m=+1636.117982931 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "config-data" (UniqueName: "kubernetes.io/secret/e947107f-104b-4699-a994-bc77285f0cf5-config-data") pod "cinder-scheduler-0" (UID: "e947107f-104b-4699-a994-bc77285f0cf5") : secret "cinder-config-data" not found Dec 03 11:47:23 crc kubenswrapper[5052]: I1203 11:47:23.503088 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/keystone-c226-account-create-update-4scft"] Dec 03 11:47:23 crc kubenswrapper[5052]: I1203 11:47:23.496897 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/openstack-cell1-galera-0" podUID="83be8c2e-57f8-4b45-b3ca-096f72a1defd" containerName="galera" containerID="cri-o://e091f633e39fe79c62c14d716851b357c6ec86a4712a030e1839dbe830fb60a3" gracePeriod=30 Dec 03 11:47:23 crc kubenswrapper[5052]: E1203 11:47:23.497016 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/combined-ca-bundle: secret "combined-ca-bundle" not found Dec 03 11:47:23 crc kubenswrapper[5052]: E1203 11:47:23.530111 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e947107f-104b-4699-a994-bc77285f0cf5-combined-ca-bundle podName:e947107f-104b-4699-a994-bc77285f0cf5 nodeName:}" failed. No retries permitted until 2025-12-03 11:47:25.530084757 +0000 UTC m=+1636.118326521 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "combined-ca-bundle" (UniqueName: "kubernetes.io/secret/e947107f-104b-4699-a994-bc77285f0cf5-combined-ca-bundle") pod "cinder-scheduler-0" (UID: "e947107f-104b-4699-a994-bc77285f0cf5") : secret "combined-ca-bundle" not found Dec 03 11:47:23 crc kubenswrapper[5052]: E1203 11:47:23.497054 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/combined-ca-bundle: secret "combined-ca-bundle" not found Dec 03 11:47:23 crc kubenswrapper[5052]: E1203 11:47:23.530170 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/84b8e975-30ee-42c6-8952-25aebf2c378b-combined-ca-bundle podName:84b8e975-30ee-42c6-8952-25aebf2c378b nodeName:}" failed. No retries permitted until 2025-12-03 11:47:25.530144328 +0000 UTC m=+1636.118386092 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "combined-ca-bundle" (UniqueName: "kubernetes.io/secret/84b8e975-30ee-42c6-8952-25aebf2c378b-combined-ca-bundle") pod "nova-cell1-novncproxy-0" (UID: "84b8e975-30ee-42c6-8952-25aebf2c378b") : secret "combined-ca-bundle" not found Dec 03 11:47:23 crc kubenswrapper[5052]: E1203 11:47:23.497075 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cinder-scheduler-config-data: secret "cinder-scheduler-config-data" not found Dec 03 11:47:23 crc kubenswrapper[5052]: E1203 11:47:23.530220 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e947107f-104b-4699-a994-bc77285f0cf5-config-data-custom podName:e947107f-104b-4699-a994-bc77285f0cf5 nodeName:}" failed. No retries permitted until 2025-12-03 11:47:25.53020052 +0000 UTC m=+1636.118442284 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "config-data-custom" (UniqueName: "kubernetes.io/secret/e947107f-104b-4699-a994-bc77285f0cf5-config-data-custom") pod "cinder-scheduler-0" (UID: "e947107f-104b-4699-a994-bc77285f0cf5") : secret "cinder-scheduler-config-data" not found Dec 03 11:47:23 crc kubenswrapper[5052]: E1203 11:47:23.497086 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-nova-novncproxy-cell1-vencrypt: secret "cert-nova-novncproxy-cell1-vencrypt" not found Dec 03 11:47:23 crc kubenswrapper[5052]: E1203 11:47:23.530269 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/84b8e975-30ee-42c6-8952-25aebf2c378b-vencrypt-tls-certs podName:84b8e975-30ee-42c6-8952-25aebf2c378b nodeName:}" failed. No retries permitted until 2025-12-03 11:47:25.530255722 +0000 UTC m=+1636.118497486 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "vencrypt-tls-certs" (UniqueName: "kubernetes.io/secret/84b8e975-30ee-42c6-8952-25aebf2c378b-vencrypt-tls-certs") pod "nova-cell1-novncproxy-0" (UID: "84b8e975-30ee-42c6-8952-25aebf2c378b") : secret "cert-nova-novncproxy-cell1-vencrypt" not found Dec 03 11:47:23 crc kubenswrapper[5052]: E1203 11:47:23.497121 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-nova-novncproxy-cell1-public-svc: secret "cert-nova-novncproxy-cell1-public-svc" not found Dec 03 11:47:23 crc kubenswrapper[5052]: E1203 11:47:23.530321 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/84b8e975-30ee-42c6-8952-25aebf2c378b-nova-novncproxy-tls-certs podName:84b8e975-30ee-42c6-8952-25aebf2c378b nodeName:}" failed. No retries permitted until 2025-12-03 11:47:25.530305813 +0000 UTC m=+1636.118547577 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "nova-novncproxy-tls-certs" (UniqueName: "kubernetes.io/secret/84b8e975-30ee-42c6-8952-25aebf2c378b-nova-novncproxy-tls-certs") pod "nova-cell1-novncproxy-0" (UID: "84b8e975-30ee-42c6-8952-25aebf2c378b") : secret "cert-nova-novncproxy-cell1-public-svc" not found Dec 03 11:47:23 crc kubenswrapper[5052]: E1203 11:47:23.497129 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cinder-scripts: secret "cinder-scripts" not found Dec 03 11:47:23 crc kubenswrapper[5052]: E1203 11:47:23.530364 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e947107f-104b-4699-a994-bc77285f0cf5-scripts podName:e947107f-104b-4699-a994-bc77285f0cf5 nodeName:}" failed. No retries permitted until 2025-12-03 11:47:25.530349924 +0000 UTC m=+1636.118591688 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "scripts" (UniqueName: "kubernetes.io/secret/e947107f-104b-4699-a994-bc77285f0cf5-scripts") pod "cinder-scheduler-0" (UID: "e947107f-104b-4699-a994-bc77285f0cf5") : secret "cinder-scripts" not found Dec 03 11:47:23 crc kubenswrapper[5052]: I1203 11:47:23.671630 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/68f829f3-e972-4168-8b75-27c086a91142-metrics-certs-tls-certs" (OuterVolumeSpecName: "metrics-certs-tls-certs") pod "68f829f3-e972-4168-8b75-27c086a91142" (UID: "68f829f3-e972-4168-8b75-27c086a91142"). InnerVolumeSpecName "metrics-certs-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:47:23 crc kubenswrapper[5052]: I1203 11:47:23.671883 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/68f829f3-e972-4168-8b75-27c086a91142-ovsdbserver-nb-tls-certs" (OuterVolumeSpecName: "ovsdbserver-nb-tls-certs") pod "68f829f3-e972-4168-8b75-27c086a91142" (UID: "68f829f3-e972-4168-8b75-27c086a91142"). InnerVolumeSpecName "ovsdbserver-nb-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:47:23 crc kubenswrapper[5052]: I1203 11:47:23.711036 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="059efa84-74c6-4d18-a279-5b7de59bb1b3" path="/var/lib/kubelet/pods/059efa84-74c6-4d18-a279-5b7de59bb1b3/volumes" Dec 03 11:47:23 crc kubenswrapper[5052]: I1203 11:47:23.729803 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1445732d-e68a-4d4c-90db-b1b05c270962" path="/var/lib/kubelet/pods/1445732d-e68a-4d4c-90db-b1b05c270962/volumes" Dec 03 11:47:23 crc kubenswrapper[5052]: I1203 11:47:23.731179 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="189bc36c-9345-4a45-92ce-1ba812535b54" path="/var/lib/kubelet/pods/189bc36c-9345-4a45-92ce-1ba812535b54/volumes" Dec 03 11:47:23 crc kubenswrapper[5052]: I1203 11:47:23.739658 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2bd8667f-0169-4f07-aefc-e19027cf99e6" path="/var/lib/kubelet/pods/2bd8667f-0169-4f07-aefc-e19027cf99e6/volumes" Dec 03 11:47:23 crc kubenswrapper[5052]: I1203 11:47:23.748061 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2df10f41-fb7a-4a14-94b4-ebdaf98d1d2c" path="/var/lib/kubelet/pods/2df10f41-fb7a-4a14-94b4-ebdaf98d1d2c/volumes" Dec 03 11:47:23 crc kubenswrapper[5052]: I1203 11:47:23.749404 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="35ce22c2-a534-49af-9ba3-3282dffbaf95" path="/var/lib/kubelet/pods/35ce22c2-a534-49af-9ba3-3282dffbaf95/volumes" Dec 03 11:47:23 crc kubenswrapper[5052]: I1203 11:47:23.821490 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3d4a523e-547f-4d7a-920f-7a24ca45f183" path="/var/lib/kubelet/pods/3d4a523e-547f-4d7a-920f-7a24ca45f183/volumes" Dec 03 11:47:23 crc kubenswrapper[5052]: I1203 11:47:23.773177 5052 reconciler_common.go:293] "Volume detached for volume \"ovsdbserver-nb-tls-certs\" (UniqueName: \"kubernetes.io/secret/68f829f3-e972-4168-8b75-27c086a91142-ovsdbserver-nb-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:23 crc kubenswrapper[5052]: I1203 11:47:23.822012 5052 reconciler_common.go:293] "Volume detached for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/68f829f3-e972-4168-8b75-27c086a91142-metrics-certs-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:23 crc kubenswrapper[5052]: I1203 11:47:23.822316 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="516527bf-2b7b-4bd1-b1a1-8d2ea94fbf16" path="/var/lib/kubelet/pods/516527bf-2b7b-4bd1-b1a1-8d2ea94fbf16/volumes" Dec 03 11:47:23 crc kubenswrapper[5052]: I1203 11:47:23.822914 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="679ac416-0e30-4016-bb4a-59944d03a695" path="/var/lib/kubelet/pods/679ac416-0e30-4016-bb4a-59944d03a695/volumes" Dec 03 11:47:23 crc kubenswrapper[5052]: I1203 11:47:23.824406 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8e87b835-163c-4f90-a10d-c35a8e5ec9dd" path="/var/lib/kubelet/pods/8e87b835-163c-4f90-a10d-c35a8e5ec9dd/volumes" Dec 03 11:47:23 crc kubenswrapper[5052]: I1203 11:47:23.834172 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_ovsdbserver-sb-0_9bdd2626-de50-4513-8313-3b25347ff641/ovsdbserver-sb/0.log" Dec 03 11:47:23 crc kubenswrapper[5052]: I1203 11:47:23.834257 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 11:47:23 crc kubenswrapper[5052]: I1203 11:47:23.839648 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b384ab60-c8a3-488d-8753-22058200e84f" path="/var/lib/kubelet/pods/b384ab60-c8a3-488d-8753-22058200e84f/volumes" Dec 03 11:47:23 crc kubenswrapper[5052]: I1203 11:47:23.841847 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c3a670b4-73f2-4c90-a3af-ba5d660ec1f7" path="/var/lib/kubelet/pods/c3a670b4-73f2-4c90-a3af-ba5d660ec1f7/volumes" Dec 03 11:47:23 crc kubenswrapper[5052]: I1203 11:47:23.842854 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ee9c2229-695a-4930-89b0-39f140a3f328" path="/var/lib/kubelet/pods/ee9c2229-695a-4930-89b0-39f140a3f328/volumes" Dec 03 11:47:23 crc kubenswrapper[5052]: I1203 11:47:23.846988 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder1bf0-account-delete-hs8j6"] Dec 03 11:47:23 crc kubenswrapper[5052]: I1203 11:47:23.847013 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican5976-account-delete-q6j5s"] Dec 03 11:47:23 crc kubenswrapper[5052]: I1203 11:47:23.847023 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/novaapi5919-account-delete-qkvzd"] Dec 03 11:47:23 crc kubenswrapper[5052]: I1203 11:47:23.847033 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/novacell0f13b-account-delete-g7m79"] Dec 03 11:47:23 crc kubenswrapper[5052]: I1203 11:47:23.847042 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-lfl6r"] Dec 03 11:47:23 crc kubenswrapper[5052]: I1203 11:47:23.847051 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/rabbitmq-server-0"] Dec 03 11:47:23 crc kubenswrapper[5052]: I1203 11:47:23.847061 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/rabbitmq-cell1-server-0"] Dec 03 11:47:23 crc kubenswrapper[5052]: I1203 11:47:23.851837 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/openstack-galera-0" podUID="e741a60a-1a4f-45a1-b315-064fb7458e8f" containerName="galera" containerID="cri-o://a7e16a1558edb19db29f41cde5903f2a8a903af9d50cea81f332f651715f1903" gracePeriod=30 Dec 03 11:47:23 crc kubenswrapper[5052]: I1203 11:47:23.871680 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstackclient" Dec 03 11:47:23 crc kubenswrapper[5052]: I1203 11:47:23.922679 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"openstack-config-secret\" (UniqueName: \"kubernetes.io/secret/a9f103a5-8b29-4152-a711-60891b9aeefc-openstack-config-secret\") pod \"a9f103a5-8b29-4152-a711-60891b9aeefc\" (UID: \"a9f103a5-8b29-4152-a711-60891b9aeefc\") " Dec 03 11:47:23 crc kubenswrapper[5052]: I1203 11:47:23.922739 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/9bdd2626-de50-4513-8313-3b25347ff641-config\") pod \"9bdd2626-de50-4513-8313-3b25347ff641\" (UID: \"9bdd2626-de50-4513-8313-3b25347ff641\") " Dec 03 11:47:23 crc kubenswrapper[5052]: I1203 11:47:23.922795 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gmd9s\" (UniqueName: \"kubernetes.io/projected/a9f103a5-8b29-4152-a711-60891b9aeefc-kube-api-access-gmd9s\") pod \"a9f103a5-8b29-4152-a711-60891b9aeefc\" (UID: \"a9f103a5-8b29-4152-a711-60891b9aeefc\") " Dec 03 11:47:23 crc kubenswrapper[5052]: I1203 11:47:23.922899 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9bdd2626-de50-4513-8313-3b25347ff641-combined-ca-bundle\") pod \"9bdd2626-de50-4513-8313-3b25347ff641\" (UID: \"9bdd2626-de50-4513-8313-3b25347ff641\") " Dec 03 11:47:23 crc kubenswrapper[5052]: I1203 11:47:23.922930 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/9bdd2626-de50-4513-8313-3b25347ff641-scripts\") pod \"9bdd2626-de50-4513-8313-3b25347ff641\" (UID: \"9bdd2626-de50-4513-8313-3b25347ff641\") " Dec 03 11:47:23 crc kubenswrapper[5052]: I1203 11:47:23.922979 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"openstack-config\" (UniqueName: \"kubernetes.io/configmap/a9f103a5-8b29-4152-a711-60891b9aeefc-openstack-config\") pod \"a9f103a5-8b29-4152-a711-60891b9aeefc\" (UID: \"a9f103a5-8b29-4152-a711-60891b9aeefc\") " Dec 03 11:47:23 crc kubenswrapper[5052]: I1203 11:47:23.923024 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/9bdd2626-de50-4513-8313-3b25347ff641-metrics-certs-tls-certs\") pod \"9bdd2626-de50-4513-8313-3b25347ff641\" (UID: \"9bdd2626-de50-4513-8313-3b25347ff641\") " Dec 03 11:47:23 crc kubenswrapper[5052]: I1203 11:47:23.923050 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovsdbserver-sb-tls-certs\" (UniqueName: \"kubernetes.io/secret/9bdd2626-de50-4513-8313-3b25347ff641-ovsdbserver-sb-tls-certs\") pod \"9bdd2626-de50-4513-8313-3b25347ff641\" (UID: \"9bdd2626-de50-4513-8313-3b25347ff641\") " Dec 03 11:47:23 crc kubenswrapper[5052]: I1203 11:47:23.923111 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a9f103a5-8b29-4152-a711-60891b9aeefc-combined-ca-bundle\") pod \"a9f103a5-8b29-4152-a711-60891b9aeefc\" (UID: \"a9f103a5-8b29-4152-a711-60891b9aeefc\") " Dec 03 11:47:23 crc kubenswrapper[5052]: I1203 11:47:23.923138 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nxpnw\" (UniqueName: \"kubernetes.io/projected/9bdd2626-de50-4513-8313-3b25347ff641-kube-api-access-nxpnw\") pod \"9bdd2626-de50-4513-8313-3b25347ff641\" (UID: \"9bdd2626-de50-4513-8313-3b25347ff641\") " Dec 03 11:47:23 crc kubenswrapper[5052]: I1203 11:47:23.923179 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/9bdd2626-de50-4513-8313-3b25347ff641-ovsdb-rundir\") pod \"9bdd2626-de50-4513-8313-3b25347ff641\" (UID: \"9bdd2626-de50-4513-8313-3b25347ff641\") " Dec 03 11:47:23 crc kubenswrapper[5052]: I1203 11:47:23.923204 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovndbcluster-sb-etc-ovn\" (UniqueName: \"kubernetes.io/local-volume/local-storage01-crc\") pod \"9bdd2626-de50-4513-8313-3b25347ff641\" (UID: \"9bdd2626-de50-4513-8313-3b25347ff641\") " Dec 03 11:47:23 crc kubenswrapper[5052]: E1203 11:47:23.933068 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/rabbitmq-config-data: configmap "rabbitmq-config-data" not found Dec 03 11:47:23 crc kubenswrapper[5052]: E1203 11:47:23.933121 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0-config-data podName:d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0 nodeName:}" failed. No retries permitted until 2025-12-03 11:47:24.433105925 +0000 UTC m=+1635.021347689 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "config-data" (UniqueName: "kubernetes.io/configmap/d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0-config-data") pod "rabbitmq-server-0" (UID: "d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0") : configmap "rabbitmq-config-data" not found Dec 03 11:47:23 crc kubenswrapper[5052]: I1203 11:47:23.933944 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9bdd2626-de50-4513-8313-3b25347ff641-config" (OuterVolumeSpecName: "config") pod "9bdd2626-de50-4513-8313-3b25347ff641" (UID: "9bdd2626-de50-4513-8313-3b25347ff641"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:47:23 crc kubenswrapper[5052]: I1203 11:47:23.933975 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9bdd2626-de50-4513-8313-3b25347ff641-scripts" (OuterVolumeSpecName: "scripts") pod "9bdd2626-de50-4513-8313-3b25347ff641" (UID: "9bdd2626-de50-4513-8313-3b25347ff641"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:47:23 crc kubenswrapper[5052]: I1203 11:47:23.937144 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9bdd2626-de50-4513-8313-3b25347ff641-ovsdb-rundir" (OuterVolumeSpecName: "ovsdb-rundir") pod "9bdd2626-de50-4513-8313-3b25347ff641" (UID: "9bdd2626-de50-4513-8313-3b25347ff641"). InnerVolumeSpecName "ovsdb-rundir". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:47:23 crc kubenswrapper[5052]: I1203 11:47:23.948553 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage01-crc" (OuterVolumeSpecName: "ovndbcluster-sb-etc-ovn") pod "9bdd2626-de50-4513-8313-3b25347ff641" (UID: "9bdd2626-de50-4513-8313-3b25347ff641"). InnerVolumeSpecName "local-storage01-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 11:47:23 crc kubenswrapper[5052]: I1203 11:47:23.952703 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a9f103a5-8b29-4152-a711-60891b9aeefc-kube-api-access-gmd9s" (OuterVolumeSpecName: "kube-api-access-gmd9s") pod "a9f103a5-8b29-4152-a711-60891b9aeefc" (UID: "a9f103a5-8b29-4152-a711-60891b9aeefc"). InnerVolumeSpecName "kube-api-access-gmd9s". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:47:23 crc kubenswrapper[5052]: I1203 11:47:23.959883 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9bdd2626-de50-4513-8313-3b25347ff641-kube-api-access-nxpnw" (OuterVolumeSpecName: "kube-api-access-nxpnw") pod "9bdd2626-de50-4513-8313-3b25347ff641" (UID: "9bdd2626-de50-4513-8313-3b25347ff641"). InnerVolumeSpecName "kube-api-access-nxpnw". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:47:24 crc kubenswrapper[5052]: I1203 11:47:24.002300 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-nb-0"] Dec 03 11:47:24 crc kubenswrapper[5052]: I1203 11:47:24.027926 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/9bdd2626-de50-4513-8313-3b25347ff641-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:24 crc kubenswrapper[5052]: I1203 11:47:24.027955 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-nxpnw\" (UniqueName: \"kubernetes.io/projected/9bdd2626-de50-4513-8313-3b25347ff641-kube-api-access-nxpnw\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:24 crc kubenswrapper[5052]: I1203 11:47:24.027965 5052 reconciler_common.go:293] "Volume detached for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/9bdd2626-de50-4513-8313-3b25347ff641-ovsdb-rundir\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:24 crc kubenswrapper[5052]: I1203 11:47:24.027984 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage01-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage01-crc\") on node \"crc\" " Dec 03 11:47:24 crc kubenswrapper[5052]: I1203 11:47:24.027994 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/9bdd2626-de50-4513-8313-3b25347ff641-config\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:24 crc kubenswrapper[5052]: I1203 11:47:24.028004 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-gmd9s\" (UniqueName: \"kubernetes.io/projected/a9f103a5-8b29-4152-a711-60891b9aeefc-kube-api-access-gmd9s\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:24 crc kubenswrapper[5052]: I1203 11:47:24.037757 5052 scope.go:117] "RemoveContainer" containerID="630e406fa52125d27b16eb3de57c1b6dc5a497b24481482e2ba05d01fcd553fc" Dec 03 11:47:24 crc kubenswrapper[5052]: E1203 11:47:24.038234 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"630e406fa52125d27b16eb3de57c1b6dc5a497b24481482e2ba05d01fcd553fc\": container with ID starting with 630e406fa52125d27b16eb3de57c1b6dc5a497b24481482e2ba05d01fcd553fc not found: ID does not exist" containerID="630e406fa52125d27b16eb3de57c1b6dc5a497b24481482e2ba05d01fcd553fc" Dec 03 11:47:24 crc kubenswrapper[5052]: I1203 11:47:24.038265 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"630e406fa52125d27b16eb3de57c1b6dc5a497b24481482e2ba05d01fcd553fc"} err="failed to get container status \"630e406fa52125d27b16eb3de57c1b6dc5a497b24481482e2ba05d01fcd553fc\": rpc error: code = NotFound desc = could not find container \"630e406fa52125d27b16eb3de57c1b6dc5a497b24481482e2ba05d01fcd553fc\": container with ID starting with 630e406fa52125d27b16eb3de57c1b6dc5a497b24481482e2ba05d01fcd553fc not found: ID does not exist" Dec 03 11:47:24 crc kubenswrapper[5052]: I1203 11:47:24.038289 5052 scope.go:117] "RemoveContainer" containerID="57f10ab9d3a1c544593f058c6724e34ab08ea61ce5e3535d93586cc3d5834467" Dec 03 11:47:24 crc kubenswrapper[5052]: E1203 11:47:24.038761 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"57f10ab9d3a1c544593f058c6724e34ab08ea61ce5e3535d93586cc3d5834467\": container with ID starting with 57f10ab9d3a1c544593f058c6724e34ab08ea61ce5e3535d93586cc3d5834467 not found: ID does not exist" containerID="57f10ab9d3a1c544593f058c6724e34ab08ea61ce5e3535d93586cc3d5834467" Dec 03 11:47:24 crc kubenswrapper[5052]: I1203 11:47:24.038803 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"57f10ab9d3a1c544593f058c6724e34ab08ea61ce5e3535d93586cc3d5834467"} err="failed to get container status \"57f10ab9d3a1c544593f058c6724e34ab08ea61ce5e3535d93586cc3d5834467\": rpc error: code = NotFound desc = could not find container \"57f10ab9d3a1c544593f058c6724e34ab08ea61ce5e3535d93586cc3d5834467\": container with ID starting with 57f10ab9d3a1c544593f058c6724e34ab08ea61ce5e3535d93586cc3d5834467 not found: ID does not exist" Dec 03 11:47:24 crc kubenswrapper[5052]: I1203 11:47:24.039564 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-nb-0"] Dec 03 11:47:24 crc kubenswrapper[5052]: I1203 11:47:24.178101 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 11:47:24 crc kubenswrapper[5052]: I1203 11:47:24.239030 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-state-metrics-tls-config\" (UniqueName: \"kubernetes.io/secret/496a1006-5380-40d1-998f-129a87233300-kube-state-metrics-tls-config\") pod \"496a1006-5380-40d1-998f-129a87233300\" (UID: \"496a1006-5380-40d1-998f-129a87233300\") " Dec 03 11:47:24 crc kubenswrapper[5052]: I1203 11:47:24.241741 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dm4wk\" (UniqueName: \"kubernetes.io/projected/496a1006-5380-40d1-998f-129a87233300-kube-api-access-dm4wk\") pod \"496a1006-5380-40d1-998f-129a87233300\" (UID: \"496a1006-5380-40d1-998f-129a87233300\") " Dec 03 11:47:24 crc kubenswrapper[5052]: I1203 11:47:24.241960 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-state-metrics-tls-certs\" (UniqueName: \"kubernetes.io/secret/496a1006-5380-40d1-998f-129a87233300-kube-state-metrics-tls-certs\") pod \"496a1006-5380-40d1-998f-129a87233300\" (UID: \"496a1006-5380-40d1-998f-129a87233300\") " Dec 03 11:47:24 crc kubenswrapper[5052]: I1203 11:47:24.242068 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/496a1006-5380-40d1-998f-129a87233300-combined-ca-bundle\") pod \"496a1006-5380-40d1-998f-129a87233300\" (UID: \"496a1006-5380-40d1-998f-129a87233300\") " Dec 03 11:47:24 crc kubenswrapper[5052]: I1203 11:47:24.283759 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage01-crc" (UniqueName: "kubernetes.io/local-volume/local-storage01-crc") on node "crc" Dec 03 11:47:24 crc kubenswrapper[5052]: I1203 11:47:24.284100 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/496a1006-5380-40d1-998f-129a87233300-kube-api-access-dm4wk" (OuterVolumeSpecName: "kube-api-access-dm4wk") pod "496a1006-5380-40d1-998f-129a87233300" (UID: "496a1006-5380-40d1-998f-129a87233300"). InnerVolumeSpecName "kube-api-access-dm4wk". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:47:24 crc kubenswrapper[5052]: I1203 11:47:24.290284 5052 generic.go:334] "Generic (PLEG): container finished" podID="93ca3376-6916-4fc9-8b76-7cd9a930865e" containerID="aca4ad76bb32ce93c82a24cd5e11df447906446998393a1992802096707a00b7" exitCode=0 Dec 03 11:47:24 crc kubenswrapper[5052]: I1203 11:47:24.290312 5052 generic.go:334] "Generic (PLEG): container finished" podID="93ca3376-6916-4fc9-8b76-7cd9a930865e" containerID="f561321b0fdca1154a6977a337503c90f840416e5bc0fa39babc9ea06bd5b9bf" exitCode=0 Dec 03 11:47:24 crc kubenswrapper[5052]: I1203 11:47:24.290321 5052 generic.go:334] "Generic (PLEG): container finished" podID="93ca3376-6916-4fc9-8b76-7cd9a930865e" containerID="05a11312e8ec969e44a91b4857ff72727254fdf705f9356f00323d2cea9feb2f" exitCode=0 Dec 03 11:47:24 crc kubenswrapper[5052]: I1203 11:47:24.290365 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"93ca3376-6916-4fc9-8b76-7cd9a930865e","Type":"ContainerDied","Data":"aca4ad76bb32ce93c82a24cd5e11df447906446998393a1992802096707a00b7"} Dec 03 11:47:24 crc kubenswrapper[5052]: I1203 11:47:24.290420 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"93ca3376-6916-4fc9-8b76-7cd9a930865e","Type":"ContainerDied","Data":"f561321b0fdca1154a6977a337503c90f840416e5bc0fa39babc9ea06bd5b9bf"} Dec 03 11:47:24 crc kubenswrapper[5052]: I1203 11:47:24.290434 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"93ca3376-6916-4fc9-8b76-7cd9a930865e","Type":"ContainerDied","Data":"05a11312e8ec969e44a91b4857ff72727254fdf705f9356f00323d2cea9feb2f"} Dec 03 11:47:24 crc kubenswrapper[5052]: I1203 11:47:24.296942 5052 generic.go:334] "Generic (PLEG): container finished" podID="83be8c2e-57f8-4b45-b3ca-096f72a1defd" containerID="e091f633e39fe79c62c14d716851b357c6ec86a4712a030e1839dbe830fb60a3" exitCode=0 Dec 03 11:47:24 crc kubenswrapper[5052]: I1203 11:47:24.297000 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-cell1-galera-0" event={"ID":"83be8c2e-57f8-4b45-b3ca-096f72a1defd","Type":"ContainerDied","Data":"e091f633e39fe79c62c14d716851b357c6ec86a4712a030e1839dbe830fb60a3"} Dec 03 11:47:24 crc kubenswrapper[5052]: I1203 11:47:24.305424 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican5976-account-delete-q6j5s" event={"ID":"0aefd6f6-cbb3-4c45-9f69-7124e43255e3","Type":"ContainerStarted","Data":"943d5e6f6f6cf3ad6ad0d520da8f2680a837f3c4b198e348c121ae97df4c9f94"} Dec 03 11:47:24 crc kubenswrapper[5052]: I1203 11:47:24.308263 5052 generic.go:334] "Generic (PLEG): container finished" podID="cadac9a6-1d82-403a-b7a0-692a424dd0e2" containerID="4d318f4a7ce116ebcf5a2ea62f382127bf1d126d796753d913aaafd4044c08be" exitCode=143 Dec 03 11:47:24 crc kubenswrapper[5052]: I1203 11:47:24.308291 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-worker-79689f8957-qmkdm" event={"ID":"cadac9a6-1d82-403a-b7a0-692a424dd0e2","Type":"ContainerDied","Data":"4d318f4a7ce116ebcf5a2ea62f382127bf1d126d796753d913aaafd4044c08be"} Dec 03 11:47:24 crc kubenswrapper[5052]: I1203 11:47:24.311207 5052 generic.go:334] "Generic (PLEG): container finished" podID="84957e18-5747-4786-9c4f-6af023122726" containerID="9a14693162272f7ccaeacad5515545206de3c0c0552776c0d5a91a5bb80088eb" exitCode=0 Dec 03 11:47:24 crc kubenswrapper[5052]: I1203 11:47:24.311231 5052 generic.go:334] "Generic (PLEG): container finished" podID="84957e18-5747-4786-9c4f-6af023122726" containerID="07cccd5afb2592c0003b92ef2faa428ad9bc5bc71a62a78385e0f7a553335aac" exitCode=0 Dec 03 11:47:24 crc kubenswrapper[5052]: I1203 11:47:24.311270 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-proxy-6fdf66df4-gsqlq" event={"ID":"84957e18-5747-4786-9c4f-6af023122726","Type":"ContainerDied","Data":"9a14693162272f7ccaeacad5515545206de3c0c0552776c0d5a91a5bb80088eb"} Dec 03 11:47:24 crc kubenswrapper[5052]: I1203 11:47:24.311293 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-proxy-6fdf66df4-gsqlq" event={"ID":"84957e18-5747-4786-9c4f-6af023122726","Type":"ContainerDied","Data":"07cccd5afb2592c0003b92ef2faa428ad9bc5bc71a62a78385e0f7a553335aac"} Dec 03 11:47:24 crc kubenswrapper[5052]: I1203 11:47:24.313126 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement9a4f-account-delete-f6sjd" event={"ID":"8c2252c0-5e30-4e38-b1a1-af0242df6ac0","Type":"ContainerStarted","Data":"608dc22687b11333fce9416d617bd57ae95cf87ae7a7d4d3a551d4a0344855fb"} Dec 03 11:47:24 crc kubenswrapper[5052]: I1203 11:47:24.315021 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstackclient" Dec 03 11:47:24 crc kubenswrapper[5052]: I1203 11:47:24.315066 5052 scope.go:117] "RemoveContainer" containerID="6b2a059106f1fcc4b1e035c9cf505bb5bc72d3f6e38d77875ea0e33ccbc713f1" Dec 03 11:47:24 crc kubenswrapper[5052]: I1203 11:47:24.319481 5052 generic.go:334] "Generic (PLEG): container finished" podID="99f96963-d835-4445-84af-0976f1ba0701" containerID="5592513a2c756f332087769178ec0527d69dee3863e8ebc7da5e5d5ac9fefa4f" exitCode=143 Dec 03 11:47:24 crc kubenswrapper[5052]: I1203 11:47:24.319563 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-7cfb847cb8-jn85t" event={"ID":"99f96963-d835-4445-84af-0976f1ba0701","Type":"ContainerDied","Data":"5592513a2c756f332087769178ec0527d69dee3863e8ebc7da5e5d5ac9fefa4f"} Dec 03 11:47:24 crc kubenswrapper[5052]: I1203 11:47:24.322400 5052 generic.go:334] "Generic (PLEG): container finished" podID="bb51da3b-72cc-4008-aac7-737f6a3031cb" containerID="4568b2e2e54107935eb412f6e606a9e5b4fd360b7f1d0fae94b2b383fd9205d0" exitCode=143 Dec 03 11:47:24 crc kubenswrapper[5052]: I1203 11:47:24.322471 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"bb51da3b-72cc-4008-aac7-737f6a3031cb","Type":"ContainerDied","Data":"4568b2e2e54107935eb412f6e606a9e5b4fd360b7f1d0fae94b2b383fd9205d0"} Dec 03 11:47:24 crc kubenswrapper[5052]: I1203 11:47:24.328611 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/novacell0f13b-account-delete-g7m79" event={"ID":"e5339997-33d5-4794-8fa6-042b63e9391e","Type":"ContainerStarted","Data":"6d74fedfd0678d095a656c2ce967b2160548b1dbc9117678c8bb6f3fb9a65c37"} Dec 03 11:47:24 crc kubenswrapper[5052]: I1203 11:47:24.337100 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-lfl6r" event={"ID":"93fb4fd0-aa18-4a2d-bfb4-b65bbc378ae5","Type":"ContainerStarted","Data":"395380eced40d52322ffade7efa7a58446308db3cecd621bc7773f272b9c84e0"} Dec 03 11:47:24 crc kubenswrapper[5052]: I1203 11:47:24.343406 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9bdd2626-de50-4513-8313-3b25347ff641-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "9bdd2626-de50-4513-8313-3b25347ff641" (UID: "9bdd2626-de50-4513-8313-3b25347ff641"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:47:24 crc kubenswrapper[5052]: I1203 11:47:24.344909 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9bdd2626-de50-4513-8313-3b25347ff641-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:24 crc kubenswrapper[5052]: I1203 11:47:24.344935 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-dm4wk\" (UniqueName: \"kubernetes.io/projected/496a1006-5380-40d1-998f-129a87233300-kube-api-access-dm4wk\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:24 crc kubenswrapper[5052]: I1203 11:47:24.344945 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage01-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage01-crc\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:24 crc kubenswrapper[5052]: I1203 11:47:24.369920 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_ovsdbserver-sb-0_9bdd2626-de50-4513-8313-3b25347ff641/ovsdbserver-sb/0.log" Dec 03 11:47:24 crc kubenswrapper[5052]: I1203 11:47:24.370006 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-sb-0" event={"ID":"9bdd2626-de50-4513-8313-3b25347ff641","Type":"ContainerDied","Data":"307909d852a32b4c9508700a79e3efd7ffe02fd3cf77902dfa87a114307367df"} Dec 03 11:47:24 crc kubenswrapper[5052]: I1203 11:47:24.370050 5052 scope.go:117] "RemoveContainer" containerID="d6c0926e352f7c2fd9039729545261adf7fb3d6ef95c52edff8cf47f36d45db5" Dec 03 11:47:24 crc kubenswrapper[5052]: I1203 11:47:24.370162 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 11:47:24 crc kubenswrapper[5052]: I1203 11:47:24.383916 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder1bf0-account-delete-hs8j6" event={"ID":"0b94b517-5f9b-485d-a109-0a11e15a3ad3","Type":"ContainerStarted","Data":"cefa20da1d4e1f488a1ea2bdf21bf298fbaf83848e745c3e1ca95b0e9818d28b"} Dec 03 11:47:24 crc kubenswrapper[5052]: I1203 11:47:24.391340 5052 generic.go:334] "Generic (PLEG): container finished" podID="d8de865a-0f6c-413d-8a37-3ec4d496c4bb" containerID="775a6cb46d400df5368627d0a86f59d32f62f17aac77b3a75fe7d8c2928b8362" exitCode=143 Dec 03 11:47:24 crc kubenswrapper[5052]: I1203 11:47:24.391412 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-keystone-listener-6794d844d6-f4tr7" event={"ID":"d8de865a-0f6c-413d-8a37-3ec4d496c4bb","Type":"ContainerDied","Data":"775a6cb46d400df5368627d0a86f59d32f62f17aac77b3a75fe7d8c2928b8362"} Dec 03 11:47:24 crc kubenswrapper[5052]: I1203 11:47:24.395606 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/novaapi5919-account-delete-qkvzd" event={"ID":"2d3a3054-6c8e-4808-bd5d-01413970167c","Type":"ContainerStarted","Data":"a32e8eecdf37a0533359f0cca96b134dd958c5ee69c6850d73b655fa60a18f8d"} Dec 03 11:47:24 crc kubenswrapper[5052]: I1203 11:47:24.403723 5052 generic.go:334] "Generic (PLEG): container finished" podID="e947107f-104b-4699-a994-bc77285f0cf5" containerID="08b743014f4f3e11912f7da330bbe8c095df94c7cd01737b812783a0bd8af17b" exitCode=0 Dec 03 11:47:24 crc kubenswrapper[5052]: I1203 11:47:24.403806 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"e947107f-104b-4699-a994-bc77285f0cf5","Type":"ContainerDied","Data":"08b743014f4f3e11912f7da330bbe8c095df94c7cd01737b812783a0bd8af17b"} Dec 03 11:47:24 crc kubenswrapper[5052]: I1203 11:47:24.409623 5052 generic.go:334] "Generic (PLEG): container finished" podID="84b8e975-30ee-42c6-8952-25aebf2c378b" containerID="57aafbdbe2a86778650bc79b3ec5dc6502116268591f2f9f5ff1056d241c1f66" exitCode=0 Dec 03 11:47:24 crc kubenswrapper[5052]: I1203 11:47:24.409750 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" event={"ID":"84b8e975-30ee-42c6-8952-25aebf2c378b","Type":"ContainerDied","Data":"57aafbdbe2a86778650bc79b3ec5dc6502116268591f2f9f5ff1056d241c1f66"} Dec 03 11:47:24 crc kubenswrapper[5052]: I1203 11:47:24.418611 5052 generic.go:334] "Generic (PLEG): container finished" podID="496a1006-5380-40d1-998f-129a87233300" containerID="64d2ed8863bdee50690f9bd22bda4e036b2111ebb9734c4223ad439fd9d3b384" exitCode=2 Dec 03 11:47:24 crc kubenswrapper[5052]: I1203 11:47:24.418681 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 11:47:24 crc kubenswrapper[5052]: I1203 11:47:24.418710 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/kube-state-metrics-0" event={"ID":"496a1006-5380-40d1-998f-129a87233300","Type":"ContainerDied","Data":"64d2ed8863bdee50690f9bd22bda4e036b2111ebb9734c4223ad439fd9d3b384"} Dec 03 11:47:24 crc kubenswrapper[5052]: I1203 11:47:24.418742 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/kube-state-metrics-0" event={"ID":"496a1006-5380-40d1-998f-129a87233300","Type":"ContainerDied","Data":"767dce3a52d6b21ef52be4b20913500d01a22618603394980337bbfd8eca0b52"} Dec 03 11:47:24 crc kubenswrapper[5052]: I1203 11:47:24.422985 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron18a7-account-delete-x9lqd" event={"ID":"aa9a649b-9229-4cdd-8e27-8aedd48f4bbd","Type":"ContainerStarted","Data":"ba0d9f752351adaf18aa306215d73b8cbfb89b54414b7a903a9d289209402478"} Dec 03 11:47:24 crc kubenswrapper[5052]: I1203 11:47:24.423491 5052 kubelet_pods.go:1007] "Unable to retrieve pull secret, the image pull may not succeed." pod="openstack-kuttl-tests/neutron18a7-account-delete-x9lqd" secret="" err="secret \"galera-openstack-dockercfg-cpskp\" not found" Dec 03 11:47:24 crc kubenswrapper[5052]: I1203 11:47:24.442351 5052 generic.go:334] "Generic (PLEG): container finished" podID="f7123213-06e1-4b26-9892-c16c3d49c0ab" containerID="fa0c2ba3f86bf1385c8a82da17792deaa7cbb1c9c5250a02c070da47b2664562" exitCode=143 Dec 03 11:47:24 crc kubenswrapper[5052]: I1203 11:47:24.442428 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"f7123213-06e1-4b26-9892-c16c3d49c0ab","Type":"ContainerDied","Data":"fa0c2ba3f86bf1385c8a82da17792deaa7cbb1c9c5250a02c070da47b2664562"} Dec 03 11:47:24 crc kubenswrapper[5052]: I1203 11:47:24.446278 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/neutron18a7-account-delete-x9lqd" podStartSLOduration=3.446263293 podStartE2EDuration="3.446263293s" podCreationTimestamp="2025-12-03 11:47:21 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:47:24.441049673 +0000 UTC m=+1635.029291447" watchObservedRunningTime="2025-12-03 11:47:24.446263293 +0000 UTC m=+1635.034505057" Dec 03 11:47:24 crc kubenswrapper[5052]: E1203 11:47:24.447824 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/rabbitmq-config-data: configmap "rabbitmq-config-data" not found Dec 03 11:47:24 crc kubenswrapper[5052]: E1203 11:47:24.447882 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0-config-data podName:d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0 nodeName:}" failed. No retries permitted until 2025-12-03 11:47:25.447868309 +0000 UTC m=+1636.036110073 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "config-data" (UniqueName: "kubernetes.io/configmap/d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0-config-data") pod "rabbitmq-server-0" (UID: "d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0") : configmap "rabbitmq-config-data" not found Dec 03 11:47:24 crc kubenswrapper[5052]: E1203 11:47:24.448179 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/rabbitmq-cell1-config-data: configmap "rabbitmq-cell1-config-data" not found Dec 03 11:47:24 crc kubenswrapper[5052]: E1203 11:47:24.448203 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2766f489-9217-47ae-a595-869a26389587-config-data podName:2766f489-9217-47ae-a595-869a26389587 nodeName:}" failed. No retries permitted until 2025-12-03 11:47:28.448194578 +0000 UTC m=+1639.036436342 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "config-data" (UniqueName: "kubernetes.io/configmap/2766f489-9217-47ae-a595-869a26389587-config-data") pod "rabbitmq-cell1-server-0" (UID: "2766f489-9217-47ae-a595-869a26389587") : configmap "rabbitmq-cell1-config-data" not found Dec 03 11:47:24 crc kubenswrapper[5052]: E1203 11:47:24.448269 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/combined-ca-bundle: secret "combined-ca-bundle" not found Dec 03 11:47:24 crc kubenswrapper[5052]: E1203 11:47:24.448290 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/507c60e5-056c-4661-a7a7-75c4d9bce7de-combined-ca-bundle podName:507c60e5-056c-4661-a7a7-75c4d9bce7de nodeName:}" failed. No retries permitted until 2025-12-03 11:47:28.448283441 +0000 UTC m=+1639.036525205 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "combined-ca-bundle" (UniqueName: "kubernetes.io/secret/507c60e5-056c-4661-a7a7-75c4d9bce7de-combined-ca-bundle") pod "nova-cell1-conductor-0" (UID: "507c60e5-056c-4661-a7a7-75c4d9bce7de") : secret "combined-ca-bundle" not found Dec 03 11:47:24 crc kubenswrapper[5052]: I1203 11:47:24.465876 5052 generic.go:334] "Generic (PLEG): container finished" podID="00b12739-1437-4af3-ab6d-ad830b67ad2e" containerID="ef839a7336e307d7216464809bb35eccbc476e002b4d08e14e48e0259172f844" exitCode=143 Dec 03 11:47:24 crc kubenswrapper[5052]: I1203 11:47:24.465956 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"00b12739-1437-4af3-ab6d-ad830b67ad2e","Type":"ContainerDied","Data":"ef839a7336e307d7216464809bb35eccbc476e002b4d08e14e48e0259172f844"} Dec 03 11:47:24 crc kubenswrapper[5052]: I1203 11:47:24.480019 5052 generic.go:334] "Generic (PLEG): container finished" podID="a4fa0509-8f85-462f-b7c3-4384346252bc" containerID="9ca8e5c7ceff313c395a8a6c7655cdc02bfef848fc00d1d1281da272c3a6f083" exitCode=0 Dec 03 11:47:24 crc kubenswrapper[5052]: I1203 11:47:24.480184 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-549dfc49dd-qtk5n" event={"ID":"a4fa0509-8f85-462f-b7c3-4384346252bc","Type":"ContainerDied","Data":"9ca8e5c7ceff313c395a8a6c7655cdc02bfef848fc00d1d1281da272c3a6f083"} Dec 03 11:47:24 crc kubenswrapper[5052]: I1203 11:47:24.482092 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance3554-account-delete-8ghdx" event={"ID":"950f0626-40d2-41c9-9701-cba50843c43e","Type":"ContainerStarted","Data":"3c7a993dd1496b0fec21a1c11de57571355d652145a1ab336ad9cb1fa2aa09bd"} Dec 03 11:47:24 crc kubenswrapper[5052]: I1203 11:47:24.484694 5052 generic.go:334] "Generic (PLEG): container finished" podID="1b45577a-92b7-4c67-941b-cfc2fbfa3eb2" containerID="8c71cc229cea36e3361a6c384c5386d5cc260814f0041348a439548161af1249" exitCode=0 Dec 03 11:47:24 crc kubenswrapper[5052]: I1203 11:47:24.484718 5052 generic.go:334] "Generic (PLEG): container finished" podID="1b45577a-92b7-4c67-941b-cfc2fbfa3eb2" containerID="05471f2484133fbd796cb8f6c5d16314217c42ed6ab7d7d6511de8a201b11eb9" exitCode=2 Dec 03 11:47:24 crc kubenswrapper[5052]: I1203 11:47:24.484728 5052 generic.go:334] "Generic (PLEG): container finished" podID="1b45577a-92b7-4c67-941b-cfc2fbfa3eb2" containerID="76c303acd28c23268bb01ed30bc5fa462afd9585a2872db7a6ce77014ebbad00" exitCode=0 Dec 03 11:47:24 crc kubenswrapper[5052]: I1203 11:47:24.484746 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"1b45577a-92b7-4c67-941b-cfc2fbfa3eb2","Type":"ContainerDied","Data":"8c71cc229cea36e3361a6c384c5386d5cc260814f0041348a439548161af1249"} Dec 03 11:47:24 crc kubenswrapper[5052]: I1203 11:47:24.484768 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"1b45577a-92b7-4c67-941b-cfc2fbfa3eb2","Type":"ContainerDied","Data":"05471f2484133fbd796cb8f6c5d16314217c42ed6ab7d7d6511de8a201b11eb9"} Dec 03 11:47:24 crc kubenswrapper[5052]: I1203 11:47:24.484781 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"1b45577a-92b7-4c67-941b-cfc2fbfa3eb2","Type":"ContainerDied","Data":"76c303acd28c23268bb01ed30bc5fa462afd9585a2872db7a6ce77014ebbad00"} Dec 03 11:47:24 crc kubenswrapper[5052]: E1203 11:47:24.550436 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 11:47:24 crc kubenswrapper[5052]: E1203 11:47:24.550505 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/aa9a649b-9229-4cdd-8e27-8aedd48f4bbd-operator-scripts podName:aa9a649b-9229-4cdd-8e27-8aedd48f4bbd nodeName:}" failed. No retries permitted until 2025-12-03 11:47:25.050488132 +0000 UTC m=+1635.638729896 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/aa9a649b-9229-4cdd-8e27-8aedd48f4bbd-operator-scripts") pod "neutron18a7-account-delete-x9lqd" (UID: "aa9a649b-9229-4cdd-8e27-8aedd48f4bbd") : configmap "openstack-scripts" not found Dec 03 11:47:24 crc kubenswrapper[5052]: I1203 11:47:24.621912 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a9f103a5-8b29-4152-a711-60891b9aeefc-openstack-config" (OuterVolumeSpecName: "openstack-config") pod "a9f103a5-8b29-4152-a711-60891b9aeefc" (UID: "a9f103a5-8b29-4152-a711-60891b9aeefc"). InnerVolumeSpecName "openstack-config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:47:24 crc kubenswrapper[5052]: I1203 11:47:24.659055 5052 reconciler_common.go:293] "Volume detached for volume \"openstack-config\" (UniqueName: \"kubernetes.io/configmap/a9f103a5-8b29-4152-a711-60891b9aeefc-openstack-config\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:24 crc kubenswrapper[5052]: I1203 11:47:24.700019 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a9f103a5-8b29-4152-a711-60891b9aeefc-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "a9f103a5-8b29-4152-a711-60891b9aeefc" (UID: "a9f103a5-8b29-4152-a711-60891b9aeefc"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:47:24 crc kubenswrapper[5052]: I1203 11:47:24.764805 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a9f103a5-8b29-4152-a711-60891b9aeefc-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:24 crc kubenswrapper[5052]: I1203 11:47:24.789085 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/rabbitmq-server-0"] Dec 03 11:47:24 crc kubenswrapper[5052]: I1203 11:47:24.814668 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/496a1006-5380-40d1-998f-129a87233300-kube-state-metrics-tls-config" (OuterVolumeSpecName: "kube-state-metrics-tls-config") pod "496a1006-5380-40d1-998f-129a87233300" (UID: "496a1006-5380-40d1-998f-129a87233300"). InnerVolumeSpecName "kube-state-metrics-tls-config". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:47:24 crc kubenswrapper[5052]: I1203 11:47:24.828473 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9bdd2626-de50-4513-8313-3b25347ff641-metrics-certs-tls-certs" (OuterVolumeSpecName: "metrics-certs-tls-certs") pod "9bdd2626-de50-4513-8313-3b25347ff641" (UID: "9bdd2626-de50-4513-8313-3b25347ff641"). InnerVolumeSpecName "metrics-certs-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:47:24 crc kubenswrapper[5052]: I1203 11:47:24.829211 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a9f103a5-8b29-4152-a711-60891b9aeefc-openstack-config-secret" (OuterVolumeSpecName: "openstack-config-secret") pod "a9f103a5-8b29-4152-a711-60891b9aeefc" (UID: "a9f103a5-8b29-4152-a711-60891b9aeefc"). InnerVolumeSpecName "openstack-config-secret". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:47:24 crc kubenswrapper[5052]: I1203 11:47:24.837773 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9bdd2626-de50-4513-8313-3b25347ff641-ovsdbserver-sb-tls-certs" (OuterVolumeSpecName: "ovsdbserver-sb-tls-certs") pod "9bdd2626-de50-4513-8313-3b25347ff641" (UID: "9bdd2626-de50-4513-8313-3b25347ff641"). InnerVolumeSpecName "ovsdbserver-sb-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:47:24 crc kubenswrapper[5052]: I1203 11:47:24.838709 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/496a1006-5380-40d1-998f-129a87233300-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "496a1006-5380-40d1-998f-129a87233300" (UID: "496a1006-5380-40d1-998f-129a87233300"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:47:24 crc kubenswrapper[5052]: I1203 11:47:24.848407 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/rabbitmq-server-0" podUID="d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0" containerName="rabbitmq" containerID="cri-o://f1ccb0c2e8810415edac670e94d35e316eb9d118a2ee2cb2d84f782419550625" gracePeriod=604800 Dec 03 11:47:24 crc kubenswrapper[5052]: I1203 11:47:24.859566 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/496a1006-5380-40d1-998f-129a87233300-kube-state-metrics-tls-certs" (OuterVolumeSpecName: "kube-state-metrics-tls-certs") pod "496a1006-5380-40d1-998f-129a87233300" (UID: "496a1006-5380-40d1-998f-129a87233300"). InnerVolumeSpecName "kube-state-metrics-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:47:24 crc kubenswrapper[5052]: I1203 11:47:24.867214 5052 reconciler_common.go:293] "Volume detached for volume \"openstack-config-secret\" (UniqueName: \"kubernetes.io/secret/a9f103a5-8b29-4152-a711-60891b9aeefc-openstack-config-secret\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:24 crc kubenswrapper[5052]: I1203 11:47:24.867246 5052 reconciler_common.go:293] "Volume detached for volume \"kube-state-metrics-tls-certs\" (UniqueName: \"kubernetes.io/secret/496a1006-5380-40d1-998f-129a87233300-kube-state-metrics-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:24 crc kubenswrapper[5052]: I1203 11:47:24.867256 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/496a1006-5380-40d1-998f-129a87233300-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:24 crc kubenswrapper[5052]: I1203 11:47:24.867264 5052 reconciler_common.go:293] "Volume detached for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/9bdd2626-de50-4513-8313-3b25347ff641-metrics-certs-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:24 crc kubenswrapper[5052]: I1203 11:47:24.867276 5052 reconciler_common.go:293] "Volume detached for volume \"kube-state-metrics-tls-config\" (UniqueName: \"kubernetes.io/secret/496a1006-5380-40d1-998f-129a87233300-kube-state-metrics-tls-config\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:24 crc kubenswrapper[5052]: I1203 11:47:24.867286 5052 reconciler_common.go:293] "Volume detached for volume \"ovsdbserver-sb-tls-certs\" (UniqueName: \"kubernetes.io/secret/9bdd2626-de50-4513-8313-3b25347ff641-ovsdbserver-sb-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:25 crc kubenswrapper[5052]: E1203 11:47:25.070977 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 11:47:25 crc kubenswrapper[5052]: E1203 11:47:25.071067 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/aa9a649b-9229-4cdd-8e27-8aedd48f4bbd-operator-scripts podName:aa9a649b-9229-4cdd-8e27-8aedd48f4bbd nodeName:}" failed. No retries permitted until 2025-12-03 11:47:26.071045021 +0000 UTC m=+1636.659286855 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/aa9a649b-9229-4cdd-8e27-8aedd48f4bbd-operator-scripts") pod "neutron18a7-account-delete-x9lqd" (UID: "aa9a649b-9229-4cdd-8e27-8aedd48f4bbd") : configmap "openstack-scripts" not found Dec 03 11:47:25 crc kubenswrapper[5052]: I1203 11:47:25.189739 5052 scope.go:117] "RemoveContainer" containerID="5e28858cdce596489bc28d8ac4d74cd0163a29ba280b7ffd53c1e8c3d713fddb" Dec 03 11:47:25 crc kubenswrapper[5052]: I1203 11:47:25.308481 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-proxy-6fdf66df4-gsqlq" Dec 03 11:47:25 crc kubenswrapper[5052]: I1203 11:47:25.336574 5052 scope.go:117] "RemoveContainer" containerID="64d2ed8863bdee50690f9bd22bda4e036b2111ebb9734c4223ad439fd9d3b384" Dec 03 11:47:25 crc kubenswrapper[5052]: I1203 11:47:25.380954 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/84957e18-5747-4786-9c4f-6af023122726-internal-tls-certs\") pod \"84957e18-5747-4786-9c4f-6af023122726\" (UID: \"84957e18-5747-4786-9c4f-6af023122726\") " Dec 03 11:47:25 crc kubenswrapper[5052]: I1203 11:47:25.381019 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fsmm9\" (UniqueName: \"kubernetes.io/projected/84957e18-5747-4786-9c4f-6af023122726-kube-api-access-fsmm9\") pod \"84957e18-5747-4786-9c4f-6af023122726\" (UID: \"84957e18-5747-4786-9c4f-6af023122726\") " Dec 03 11:47:25 crc kubenswrapper[5052]: I1203 11:47:25.381141 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/84957e18-5747-4786-9c4f-6af023122726-combined-ca-bundle\") pod \"84957e18-5747-4786-9c4f-6af023122726\" (UID: \"84957e18-5747-4786-9c4f-6af023122726\") " Dec 03 11:47:25 crc kubenswrapper[5052]: I1203 11:47:25.381234 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/84957e18-5747-4786-9c4f-6af023122726-public-tls-certs\") pod \"84957e18-5747-4786-9c4f-6af023122726\" (UID: \"84957e18-5747-4786-9c4f-6af023122726\") " Dec 03 11:47:25 crc kubenswrapper[5052]: I1203 11:47:25.381286 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/84957e18-5747-4786-9c4f-6af023122726-etc-swift\") pod \"84957e18-5747-4786-9c4f-6af023122726\" (UID: \"84957e18-5747-4786-9c4f-6af023122726\") " Dec 03 11:47:25 crc kubenswrapper[5052]: I1203 11:47:25.381955 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/84957e18-5747-4786-9c4f-6af023122726-config-data\") pod \"84957e18-5747-4786-9c4f-6af023122726\" (UID: \"84957e18-5747-4786-9c4f-6af023122726\") " Dec 03 11:47:25 crc kubenswrapper[5052]: I1203 11:47:25.382013 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/84957e18-5747-4786-9c4f-6af023122726-log-httpd\") pod \"84957e18-5747-4786-9c4f-6af023122726\" (UID: \"84957e18-5747-4786-9c4f-6af023122726\") " Dec 03 11:47:25 crc kubenswrapper[5052]: I1203 11:47:25.382066 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/84957e18-5747-4786-9c4f-6af023122726-run-httpd\") pod \"84957e18-5747-4786-9c4f-6af023122726\" (UID: \"84957e18-5747-4786-9c4f-6af023122726\") " Dec 03 11:47:25 crc kubenswrapper[5052]: I1203 11:47:25.382963 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/84957e18-5747-4786-9c4f-6af023122726-run-httpd" (OuterVolumeSpecName: "run-httpd") pod "84957e18-5747-4786-9c4f-6af023122726" (UID: "84957e18-5747-4786-9c4f-6af023122726"). InnerVolumeSpecName "run-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:47:25 crc kubenswrapper[5052]: I1203 11:47:25.383172 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/84957e18-5747-4786-9c4f-6af023122726-log-httpd" (OuterVolumeSpecName: "log-httpd") pod "84957e18-5747-4786-9c4f-6af023122726" (UID: "84957e18-5747-4786-9c4f-6af023122726"). InnerVolumeSpecName "log-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:47:25 crc kubenswrapper[5052]: I1203 11:47:25.383544 5052 reconciler_common.go:293] "Volume detached for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/84957e18-5747-4786-9c4f-6af023122726-log-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:25 crc kubenswrapper[5052]: I1203 11:47:25.383564 5052 reconciler_common.go:293] "Volume detached for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/84957e18-5747-4786-9c4f-6af023122726-run-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:25 crc kubenswrapper[5052]: I1203 11:47:25.399291 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/84957e18-5747-4786-9c4f-6af023122726-kube-api-access-fsmm9" (OuterVolumeSpecName: "kube-api-access-fsmm9") pod "84957e18-5747-4786-9c4f-6af023122726" (UID: "84957e18-5747-4786-9c4f-6af023122726"). InnerVolumeSpecName "kube-api-access-fsmm9". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:47:25 crc kubenswrapper[5052]: I1203 11:47:25.400885 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/84957e18-5747-4786-9c4f-6af023122726-etc-swift" (OuterVolumeSpecName: "etc-swift") pod "84957e18-5747-4786-9c4f-6af023122726" (UID: "84957e18-5747-4786-9c4f-6af023122726"). InnerVolumeSpecName "etc-swift". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:47:25 crc kubenswrapper[5052]: I1203 11:47:25.452040 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/84957e18-5747-4786-9c4f-6af023122726-config-data" (OuterVolumeSpecName: "config-data") pod "84957e18-5747-4786-9c4f-6af023122726" (UID: "84957e18-5747-4786-9c4f-6af023122726"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:47:25 crc kubenswrapper[5052]: I1203 11:47:25.466229 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/84957e18-5747-4786-9c4f-6af023122726-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "84957e18-5747-4786-9c4f-6af023122726" (UID: "84957e18-5747-4786-9c4f-6af023122726"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:47:25 crc kubenswrapper[5052]: I1203 11:47:25.467793 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/84957e18-5747-4786-9c4f-6af023122726-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "84957e18-5747-4786-9c4f-6af023122726" (UID: "84957e18-5747-4786-9c4f-6af023122726"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:47:25 crc kubenswrapper[5052]: I1203 11:47:25.468760 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/84957e18-5747-4786-9c4f-6af023122726-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "84957e18-5747-4786-9c4f-6af023122726" (UID: "84957e18-5747-4786-9c4f-6af023122726"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:47:25 crc kubenswrapper[5052]: I1203 11:47:25.485120 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/84957e18-5747-4786-9c4f-6af023122726-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:25 crc kubenswrapper[5052]: I1203 11:47:25.485153 5052 reconciler_common.go:293] "Volume detached for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/84957e18-5747-4786-9c4f-6af023122726-etc-swift\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:25 crc kubenswrapper[5052]: I1203 11:47:25.485163 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/84957e18-5747-4786-9c4f-6af023122726-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:25 crc kubenswrapper[5052]: I1203 11:47:25.485171 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/84957e18-5747-4786-9c4f-6af023122726-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:25 crc kubenswrapper[5052]: E1203 11:47:25.485173 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/rabbitmq-config-data: configmap "rabbitmq-config-data" not found Dec 03 11:47:25 crc kubenswrapper[5052]: I1203 11:47:25.485201 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-fsmm9\" (UniqueName: \"kubernetes.io/projected/84957e18-5747-4786-9c4f-6af023122726-kube-api-access-fsmm9\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:25 crc kubenswrapper[5052]: I1203 11:47:25.485216 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/84957e18-5747-4786-9c4f-6af023122726-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:25 crc kubenswrapper[5052]: E1203 11:47:25.485277 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0-config-data podName:d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0 nodeName:}" failed. No retries permitted until 2025-12-03 11:47:27.48522729 +0000 UTC m=+1638.073469054 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "config-data" (UniqueName: "kubernetes.io/configmap/d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0-config-data") pod "rabbitmq-server-0" (UID: "d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0") : configmap "rabbitmq-config-data" not found Dec 03 11:47:25 crc kubenswrapper[5052]: I1203 11:47:25.501702 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-proxy-6fdf66df4-gsqlq" Dec 03 11:47:25 crc kubenswrapper[5052]: I1203 11:47:25.501692 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-proxy-6fdf66df4-gsqlq" event={"ID":"84957e18-5747-4786-9c4f-6af023122726","Type":"ContainerDied","Data":"8e5f5287d62edf8983c313bb53a6f8da8365856de089efb37c1baa4e18ad8c3b"} Dec 03 11:47:25 crc kubenswrapper[5052]: I1203 11:47:25.506164 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-cell1-galera-0" event={"ID":"83be8c2e-57f8-4b45-b3ca-096f72a1defd","Type":"ContainerDied","Data":"0b060bce7e5421791fd92cbd97a19e5add5e20a00c96af5cc36dae6699841fbe"} Dec 03 11:47:25 crc kubenswrapper[5052]: I1203 11:47:25.506202 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0b060bce7e5421791fd92cbd97a19e5add5e20a00c96af5cc36dae6699841fbe" Dec 03 11:47:25 crc kubenswrapper[5052]: I1203 11:47:25.507906 5052 generic.go:334] "Generic (PLEG): container finished" podID="aa9a649b-9229-4cdd-8e27-8aedd48f4bbd" containerID="8ac2dc5e5a71365dcb309d32fb3d721ec328645681e822f6169600deaef3d18d" exitCode=1 Dec 03 11:47:25 crc kubenswrapper[5052]: I1203 11:47:25.507949 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron18a7-account-delete-x9lqd" event={"ID":"aa9a649b-9229-4cdd-8e27-8aedd48f4bbd","Type":"ContainerDied","Data":"8ac2dc5e5a71365dcb309d32fb3d721ec328645681e822f6169600deaef3d18d"} Dec 03 11:47:25 crc kubenswrapper[5052]: I1203 11:47:25.508769 5052 kubelet_pods.go:1007] "Unable to retrieve pull secret, the image pull may not succeed." pod="openstack-kuttl-tests/neutron18a7-account-delete-x9lqd" secret="" err="secret \"galera-openstack-dockercfg-cpskp\" not found" Dec 03 11:47:25 crc kubenswrapper[5052]: I1203 11:47:25.508912 5052 scope.go:117] "RemoveContainer" containerID="8ac2dc5e5a71365dcb309d32fb3d721ec328645681e822f6169600deaef3d18d" Dec 03 11:47:25 crc kubenswrapper[5052]: I1203 11:47:25.517416 5052 generic.go:334] "Generic (PLEG): container finished" podID="8c2252c0-5e30-4e38-b1a1-af0242df6ac0" containerID="0b9fcdc42fa6273f3826d09c4e10faa45a8df9ed00f3f08f07d74ddbc5eea2b1" exitCode=1 Dec 03 11:47:25 crc kubenswrapper[5052]: I1203 11:47:25.517528 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement9a4f-account-delete-f6sjd" event={"ID":"8c2252c0-5e30-4e38-b1a1-af0242df6ac0","Type":"ContainerDied","Data":"0b9fcdc42fa6273f3826d09c4e10faa45a8df9ed00f3f08f07d74ddbc5eea2b1"} Dec 03 11:47:25 crc kubenswrapper[5052]: I1203 11:47:25.517889 5052 kubelet_pods.go:1007] "Unable to retrieve pull secret, the image pull may not succeed." pod="openstack-kuttl-tests/placement9a4f-account-delete-f6sjd" secret="" err="secret \"galera-openstack-dockercfg-cpskp\" not found" Dec 03 11:47:25 crc kubenswrapper[5052]: I1203 11:47:25.517932 5052 scope.go:117] "RemoveContainer" containerID="0b9fcdc42fa6273f3826d09c4e10faa45a8df9ed00f3f08f07d74ddbc5eea2b1" Dec 03 11:47:25 crc kubenswrapper[5052]: I1203 11:47:25.520941 5052 generic.go:334] "Generic (PLEG): container finished" podID="950f0626-40d2-41c9-9701-cba50843c43e" containerID="6080d8e70e70e85fc4f9ffe234e02f7073f0c7d3e0ee2386f2621e8608af4e90" exitCode=1 Dec 03 11:47:25 crc kubenswrapper[5052]: I1203 11:47:25.521063 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance3554-account-delete-8ghdx" event={"ID":"950f0626-40d2-41c9-9701-cba50843c43e","Type":"ContainerDied","Data":"6080d8e70e70e85fc4f9ffe234e02f7073f0c7d3e0ee2386f2621e8608af4e90"} Dec 03 11:47:25 crc kubenswrapper[5052]: I1203 11:47:25.524809 5052 kubelet_pods.go:1007] "Unable to retrieve pull secret, the image pull may not succeed." pod="openstack-kuttl-tests/glance3554-account-delete-8ghdx" secret="" err="secret \"galera-openstack-dockercfg-cpskp\" not found" Dec 03 11:47:25 crc kubenswrapper[5052]: I1203 11:47:25.529695 5052 scope.go:117] "RemoveContainer" containerID="6080d8e70e70e85fc4f9ffe234e02f7073f0c7d3e0ee2386f2621e8608af4e90" Dec 03 11:47:25 crc kubenswrapper[5052]: I1203 11:47:25.529934 5052 generic.go:334] "Generic (PLEG): container finished" podID="e741a60a-1a4f-45a1-b315-064fb7458e8f" containerID="a7e16a1558edb19db29f41cde5903f2a8a903af9d50cea81f332f651715f1903" exitCode=0 Dec 03 11:47:25 crc kubenswrapper[5052]: I1203 11:47:25.530042 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-galera-0" event={"ID":"e741a60a-1a4f-45a1-b315-064fb7458e8f","Type":"ContainerDied","Data":"a7e16a1558edb19db29f41cde5903f2a8a903af9d50cea81f332f651715f1903"} Dec 03 11:47:25 crc kubenswrapper[5052]: I1203 11:47:25.530075 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-galera-0" event={"ID":"e741a60a-1a4f-45a1-b315-064fb7458e8f","Type":"ContainerDied","Data":"3d6cb1b0ef473b63f88a4fa412741f5320a40ad57aad90117ee4b90c945ffd83"} Dec 03 11:47:25 crc kubenswrapper[5052]: I1203 11:47:25.530086 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3d6cb1b0ef473b63f88a4fa412741f5320a40ad57aad90117ee4b90c945ffd83" Dec 03 11:47:25 crc kubenswrapper[5052]: I1203 11:47:25.538033 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" event={"ID":"84b8e975-30ee-42c6-8952-25aebf2c378b","Type":"ContainerDied","Data":"f9ede52d88dd7d8d7233a7c22830fb249ecc781d116e71bf41ca385977ba3b2f"} Dec 03 11:47:25 crc kubenswrapper[5052]: I1203 11:47:25.538093 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f9ede52d88dd7d8d7233a7c22830fb249ecc781d116e71bf41ca385977ba3b2f" Dec 03 11:47:25 crc kubenswrapper[5052]: E1203 11:47:25.588413 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 11:47:25 crc kubenswrapper[5052]: E1203 11:47:25.588480 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/950f0626-40d2-41c9-9701-cba50843c43e-operator-scripts podName:950f0626-40d2-41c9-9701-cba50843c43e nodeName:}" failed. No retries permitted until 2025-12-03 11:47:26.0884664 +0000 UTC m=+1636.676708164 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/950f0626-40d2-41c9-9701-cba50843c43e-operator-scripts") pod "glance3554-account-delete-8ghdx" (UID: "950f0626-40d2-41c9-9701-cba50843c43e") : configmap "openstack-scripts" not found Dec 03 11:47:25 crc kubenswrapper[5052]: E1203 11:47:25.589090 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 11:47:25 crc kubenswrapper[5052]: E1203 11:47:25.589125 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8c2252c0-5e30-4e38-b1a1-af0242df6ac0-operator-scripts podName:8c2252c0-5e30-4e38-b1a1-af0242df6ac0 nodeName:}" failed. No retries permitted until 2025-12-03 11:47:26.089113529 +0000 UTC m=+1636.677355293 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/8c2252c0-5e30-4e38-b1a1-af0242df6ac0-operator-scripts") pod "placement9a4f-account-delete-f6sjd" (UID: "8c2252c0-5e30-4e38-b1a1-af0242df6ac0") : configmap "openstack-scripts" not found Dec 03 11:47:25 crc kubenswrapper[5052]: E1203 11:47:25.589172 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/combined-ca-bundle: secret "combined-ca-bundle" not found Dec 03 11:47:25 crc kubenswrapper[5052]: E1203 11:47:25.589196 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/84b8e975-30ee-42c6-8952-25aebf2c378b-combined-ca-bundle podName:84b8e975-30ee-42c6-8952-25aebf2c378b nodeName:}" failed. No retries permitted until 2025-12-03 11:47:29.589190091 +0000 UTC m=+1640.177431855 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "combined-ca-bundle" (UniqueName: "kubernetes.io/secret/84b8e975-30ee-42c6-8952-25aebf2c378b-combined-ca-bundle") pod "nova-cell1-novncproxy-0" (UID: "84b8e975-30ee-42c6-8952-25aebf2c378b") : secret "combined-ca-bundle" not found Dec 03 11:47:25 crc kubenswrapper[5052]: E1203 11:47:25.589235 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-nova-novncproxy-cell1-vencrypt: secret "cert-nova-novncproxy-cell1-vencrypt" not found Dec 03 11:47:25 crc kubenswrapper[5052]: E1203 11:47:25.589259 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/84b8e975-30ee-42c6-8952-25aebf2c378b-vencrypt-tls-certs podName:84b8e975-30ee-42c6-8952-25aebf2c378b nodeName:}" failed. No retries permitted until 2025-12-03 11:47:29.589249223 +0000 UTC m=+1640.177490987 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "vencrypt-tls-certs" (UniqueName: "kubernetes.io/secret/84b8e975-30ee-42c6-8952-25aebf2c378b-vencrypt-tls-certs") pod "nova-cell1-novncproxy-0" (UID: "84b8e975-30ee-42c6-8952-25aebf2c378b") : secret "cert-nova-novncproxy-cell1-vencrypt" not found Dec 03 11:47:25 crc kubenswrapper[5052]: E1203 11:47:25.589293 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/combined-ca-bundle: secret "combined-ca-bundle" not found Dec 03 11:47:25 crc kubenswrapper[5052]: E1203 11:47:25.589316 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e947107f-104b-4699-a994-bc77285f0cf5-combined-ca-bundle podName:e947107f-104b-4699-a994-bc77285f0cf5 nodeName:}" failed. No retries permitted until 2025-12-03 11:47:29.589309905 +0000 UTC m=+1640.177551669 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "combined-ca-bundle" (UniqueName: "kubernetes.io/secret/e947107f-104b-4699-a994-bc77285f0cf5-combined-ca-bundle") pod "cinder-scheduler-0" (UID: "e947107f-104b-4699-a994-bc77285f0cf5") : secret "combined-ca-bundle" not found Dec 03 11:47:25 crc kubenswrapper[5052]: E1203 11:47:25.589347 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cinder-scripts: secret "cinder-scripts" not found Dec 03 11:47:25 crc kubenswrapper[5052]: E1203 11:47:25.589368 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e947107f-104b-4699-a994-bc77285f0cf5-scripts podName:e947107f-104b-4699-a994-bc77285f0cf5 nodeName:}" failed. No retries permitted until 2025-12-03 11:47:29.589362676 +0000 UTC m=+1640.177604440 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "scripts" (UniqueName: "kubernetes.io/secret/e947107f-104b-4699-a994-bc77285f0cf5-scripts") pod "cinder-scheduler-0" (UID: "e947107f-104b-4699-a994-bc77285f0cf5") : secret "cinder-scripts" not found Dec 03 11:47:25 crc kubenswrapper[5052]: E1203 11:47:25.589403 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cinder-scheduler-config-data: secret "cinder-scheduler-config-data" not found Dec 03 11:47:25 crc kubenswrapper[5052]: E1203 11:47:25.589424 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e947107f-104b-4699-a994-bc77285f0cf5-config-data-custom podName:e947107f-104b-4699-a994-bc77285f0cf5 nodeName:}" failed. No retries permitted until 2025-12-03 11:47:29.589418678 +0000 UTC m=+1640.177660442 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "config-data-custom" (UniqueName: "kubernetes.io/secret/e947107f-104b-4699-a994-bc77285f0cf5-config-data-custom") pod "cinder-scheduler-0" (UID: "e947107f-104b-4699-a994-bc77285f0cf5") : secret "cinder-scheduler-config-data" not found Dec 03 11:47:25 crc kubenswrapper[5052]: E1203 11:47:25.589456 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-nova-novncproxy-cell1-public-svc: secret "cert-nova-novncproxy-cell1-public-svc" not found Dec 03 11:47:25 crc kubenswrapper[5052]: E1203 11:47:25.589479 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/84b8e975-30ee-42c6-8952-25aebf2c378b-nova-novncproxy-tls-certs podName:84b8e975-30ee-42c6-8952-25aebf2c378b nodeName:}" failed. No retries permitted until 2025-12-03 11:47:29.589473989 +0000 UTC m=+1640.177715753 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "nova-novncproxy-tls-certs" (UniqueName: "kubernetes.io/secret/84b8e975-30ee-42c6-8952-25aebf2c378b-nova-novncproxy-tls-certs") pod "nova-cell1-novncproxy-0" (UID: "84b8e975-30ee-42c6-8952-25aebf2c378b") : secret "cert-nova-novncproxy-cell1-public-svc" not found Dec 03 11:47:25 crc kubenswrapper[5052]: E1203 11:47:25.589772 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cinder-config-data: secret "cinder-config-data" not found Dec 03 11:47:25 crc kubenswrapper[5052]: E1203 11:47:25.589799 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e947107f-104b-4699-a994-bc77285f0cf5-config-data podName:e947107f-104b-4699-a994-bc77285f0cf5 nodeName:}" failed. No retries permitted until 2025-12-03 11:47:29.589792559 +0000 UTC m=+1640.178034323 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "config-data" (UniqueName: "kubernetes.io/secret/e947107f-104b-4699-a994-bc77285f0cf5-config-data") pod "cinder-scheduler-0" (UID: "e947107f-104b-4699-a994-bc77285f0cf5") : secret "cinder-config-data" not found Dec 03 11:47:25 crc kubenswrapper[5052]: I1203 11:47:25.604943 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 11:47:25 crc kubenswrapper[5052]: I1203 11:47:25.645802 5052 scope.go:117] "RemoveContainer" containerID="64d2ed8863bdee50690f9bd22bda4e036b2111ebb9734c4223ad439fd9d3b384" Dec 03 11:47:25 crc kubenswrapper[5052]: E1203 11:47:25.647376 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"64d2ed8863bdee50690f9bd22bda4e036b2111ebb9734c4223ad439fd9d3b384\": container with ID starting with 64d2ed8863bdee50690f9bd22bda4e036b2111ebb9734c4223ad439fd9d3b384 not found: ID does not exist" containerID="64d2ed8863bdee50690f9bd22bda4e036b2111ebb9734c4223ad439fd9d3b384" Dec 03 11:47:25 crc kubenswrapper[5052]: I1203 11:47:25.647412 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"64d2ed8863bdee50690f9bd22bda4e036b2111ebb9734c4223ad439fd9d3b384"} err="failed to get container status \"64d2ed8863bdee50690f9bd22bda4e036b2111ebb9734c4223ad439fd9d3b384\": rpc error: code = NotFound desc = could not find container \"64d2ed8863bdee50690f9bd22bda4e036b2111ebb9734c4223ad439fd9d3b384\": container with ID starting with 64d2ed8863bdee50690f9bd22bda4e036b2111ebb9734c4223ad439fd9d3b384 not found: ID does not exist" Dec 03 11:47:25 crc kubenswrapper[5052]: I1203 11:47:25.647437 5052 scope.go:117] "RemoveContainer" containerID="9a14693162272f7ccaeacad5515545206de3c0c0552776c0d5a91a5bb80088eb" Dec 03 11:47:25 crc kubenswrapper[5052]: I1203 11:47:25.669291 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 11:47:25 crc kubenswrapper[5052]: I1203 11:47:25.699000 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"mysql-db\" (UniqueName: \"kubernetes.io/local-volume/local-storage12-crc\") pod \"83be8c2e-57f8-4b45-b3ca-096f72a1defd\" (UID: \"83be8c2e-57f8-4b45-b3ca-096f72a1defd\") " Dec 03 11:47:25 crc kubenswrapper[5052]: I1203 11:47:25.699820 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fks6s\" (UniqueName: \"kubernetes.io/projected/83be8c2e-57f8-4b45-b3ca-096f72a1defd-kube-api-access-fks6s\") pod \"83be8c2e-57f8-4b45-b3ca-096f72a1defd\" (UID: \"83be8c2e-57f8-4b45-b3ca-096f72a1defd\") " Dec 03 11:47:25 crc kubenswrapper[5052]: I1203 11:47:25.699994 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/83be8c2e-57f8-4b45-b3ca-096f72a1defd-config-data-default\") pod \"83be8c2e-57f8-4b45-b3ca-096f72a1defd\" (UID: \"83be8c2e-57f8-4b45-b3ca-096f72a1defd\") " Dec 03 11:47:25 crc kubenswrapper[5052]: I1203 11:47:25.700065 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/83be8c2e-57f8-4b45-b3ca-096f72a1defd-galera-tls-certs\") pod \"83be8c2e-57f8-4b45-b3ca-096f72a1defd\" (UID: \"83be8c2e-57f8-4b45-b3ca-096f72a1defd\") " Dec 03 11:47:25 crc kubenswrapper[5052]: I1203 11:47:25.700160 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"vencrypt-tls-certs\" (UniqueName: \"kubernetes.io/secret/84b8e975-30ee-42c6-8952-25aebf2c378b-vencrypt-tls-certs\") pod \"84b8e975-30ee-42c6-8952-25aebf2c378b\" (UID: \"84b8e975-30ee-42c6-8952-25aebf2c378b\") " Dec 03 11:47:25 crc kubenswrapper[5052]: I1203 11:47:25.700820 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/83be8c2e-57f8-4b45-b3ca-096f72a1defd-config-data-generated\") pod \"83be8c2e-57f8-4b45-b3ca-096f72a1defd\" (UID: \"83be8c2e-57f8-4b45-b3ca-096f72a1defd\") " Dec 03 11:47:25 crc kubenswrapper[5052]: I1203 11:47:25.703489 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/84b8e975-30ee-42c6-8952-25aebf2c378b-combined-ca-bundle\") pod \"84b8e975-30ee-42c6-8952-25aebf2c378b\" (UID: \"84b8e975-30ee-42c6-8952-25aebf2c378b\") " Dec 03 11:47:25 crc kubenswrapper[5052]: I1203 11:47:25.703630 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"nova-novncproxy-tls-certs\" (UniqueName: \"kubernetes.io/secret/84b8e975-30ee-42c6-8952-25aebf2c378b-nova-novncproxy-tls-certs\") pod \"84b8e975-30ee-42c6-8952-25aebf2c378b\" (UID: \"84b8e975-30ee-42c6-8952-25aebf2c378b\") " Dec 03 11:47:25 crc kubenswrapper[5052]: I1203 11:47:25.703712 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/83be8c2e-57f8-4b45-b3ca-096f72a1defd-operator-scripts\") pod \"83be8c2e-57f8-4b45-b3ca-096f72a1defd\" (UID: \"83be8c2e-57f8-4b45-b3ca-096f72a1defd\") " Dec 03 11:47:25 crc kubenswrapper[5052]: I1203 11:47:25.703811 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/83be8c2e-57f8-4b45-b3ca-096f72a1defd-kolla-config\") pod \"83be8c2e-57f8-4b45-b3ca-096f72a1defd\" (UID: \"83be8c2e-57f8-4b45-b3ca-096f72a1defd\") " Dec 03 11:47:25 crc kubenswrapper[5052]: I1203 11:47:25.704007 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/83be8c2e-57f8-4b45-b3ca-096f72a1defd-combined-ca-bundle\") pod \"83be8c2e-57f8-4b45-b3ca-096f72a1defd\" (UID: \"83be8c2e-57f8-4b45-b3ca-096f72a1defd\") " Dec 03 11:47:25 crc kubenswrapper[5052]: I1203 11:47:25.704276 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8l45l\" (UniqueName: \"kubernetes.io/projected/84b8e975-30ee-42c6-8952-25aebf2c378b-kube-api-access-8l45l\") pod \"84b8e975-30ee-42c6-8952-25aebf2c378b\" (UID: \"84b8e975-30ee-42c6-8952-25aebf2c378b\") " Dec 03 11:47:25 crc kubenswrapper[5052]: I1203 11:47:25.704383 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/84b8e975-30ee-42c6-8952-25aebf2c378b-config-data\") pod \"84b8e975-30ee-42c6-8952-25aebf2c378b\" (UID: \"84b8e975-30ee-42c6-8952-25aebf2c378b\") " Dec 03 11:47:25 crc kubenswrapper[5052]: I1203 11:47:25.712971 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/83be8c2e-57f8-4b45-b3ca-096f72a1defd-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "83be8c2e-57f8-4b45-b3ca-096f72a1defd" (UID: "83be8c2e-57f8-4b45-b3ca-096f72a1defd"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:47:25 crc kubenswrapper[5052]: I1203 11:47:25.713837 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/83be8c2e-57f8-4b45-b3ca-096f72a1defd-kolla-config" (OuterVolumeSpecName: "kolla-config") pod "83be8c2e-57f8-4b45-b3ca-096f72a1defd" (UID: "83be8c2e-57f8-4b45-b3ca-096f72a1defd"). InnerVolumeSpecName "kolla-config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:47:25 crc kubenswrapper[5052]: I1203 11:47:25.714331 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/83be8c2e-57f8-4b45-b3ca-096f72a1defd-config-data-default" (OuterVolumeSpecName: "config-data-default") pod "83be8c2e-57f8-4b45-b3ca-096f72a1defd" (UID: "83be8c2e-57f8-4b45-b3ca-096f72a1defd"). InnerVolumeSpecName "config-data-default". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:47:25 crc kubenswrapper[5052]: I1203 11:47:25.719267 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83be8c2e-57f8-4b45-b3ca-096f72a1defd-config-data-generated" (OuterVolumeSpecName: "config-data-generated") pod "83be8c2e-57f8-4b45-b3ca-096f72a1defd" (UID: "83be8c2e-57f8-4b45-b3ca-096f72a1defd"). InnerVolumeSpecName "config-data-generated". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:47:25 crc kubenswrapper[5052]: I1203 11:47:25.727729 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/83be8c2e-57f8-4b45-b3ca-096f72a1defd-kube-api-access-fks6s" (OuterVolumeSpecName: "kube-api-access-fks6s") pod "83be8c2e-57f8-4b45-b3ca-096f72a1defd" (UID: "83be8c2e-57f8-4b45-b3ca-096f72a1defd"). InnerVolumeSpecName "kube-api-access-fks6s". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:47:25 crc kubenswrapper[5052]: I1203 11:47:25.727822 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/84b8e975-30ee-42c6-8952-25aebf2c378b-kube-api-access-8l45l" (OuterVolumeSpecName: "kube-api-access-8l45l") pod "84b8e975-30ee-42c6-8952-25aebf2c378b" (UID: "84b8e975-30ee-42c6-8952-25aebf2c378b"). InnerVolumeSpecName "kube-api-access-8l45l". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:47:25 crc kubenswrapper[5052]: I1203 11:47:25.741648 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="68f829f3-e972-4168-8b75-27c086a91142" path="/var/lib/kubelet/pods/68f829f3-e972-4168-8b75-27c086a91142/volumes" Dec 03 11:47:25 crc kubenswrapper[5052]: I1203 11:47:25.742423 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a9f103a5-8b29-4152-a711-60891b9aeefc" path="/var/lib/kubelet/pods/a9f103a5-8b29-4152-a711-60891b9aeefc/volumes" Dec 03 11:47:25 crc kubenswrapper[5052]: I1203 11:47:25.759901 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage12-crc" (OuterVolumeSpecName: "mysql-db") pod "83be8c2e-57f8-4b45-b3ca-096f72a1defd" (UID: "83be8c2e-57f8-4b45-b3ca-096f72a1defd"). InnerVolumeSpecName "local-storage12-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 11:47:25 crc kubenswrapper[5052]: I1203 11:47:25.760692 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/84b8e975-30ee-42c6-8952-25aebf2c378b-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "84b8e975-30ee-42c6-8952-25aebf2c378b" (UID: "84b8e975-30ee-42c6-8952-25aebf2c378b"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:47:25 crc kubenswrapper[5052]: I1203 11:47:25.774305 5052 scope.go:117] "RemoveContainer" containerID="07cccd5afb2592c0003b92ef2faa428ad9bc5bc71a62a78385e0f7a553335aac" Dec 03 11:47:25 crc kubenswrapper[5052]: I1203 11:47:25.785962 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/84b8e975-30ee-42c6-8952-25aebf2c378b-config-data" (OuterVolumeSpecName: "config-data") pod "84b8e975-30ee-42c6-8952-25aebf2c378b" (UID: "84b8e975-30ee-42c6-8952-25aebf2c378b"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:47:25 crc kubenswrapper[5052]: I1203 11:47:25.801969 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/83be8c2e-57f8-4b45-b3ca-096f72a1defd-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "83be8c2e-57f8-4b45-b3ca-096f72a1defd" (UID: "83be8c2e-57f8-4b45-b3ca-096f72a1defd"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:47:25 crc kubenswrapper[5052]: I1203 11:47:25.809612 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/83be8c2e-57f8-4b45-b3ca-096f72a1defd-config-data-generated\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:25 crc kubenswrapper[5052]: I1203 11:47:25.809645 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/84b8e975-30ee-42c6-8952-25aebf2c378b-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:25 crc kubenswrapper[5052]: I1203 11:47:25.809657 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/83be8c2e-57f8-4b45-b3ca-096f72a1defd-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:25 crc kubenswrapper[5052]: I1203 11:47:25.809668 5052 reconciler_common.go:293] "Volume detached for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/83be8c2e-57f8-4b45-b3ca-096f72a1defd-kolla-config\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:25 crc kubenswrapper[5052]: I1203 11:47:25.809680 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/83be8c2e-57f8-4b45-b3ca-096f72a1defd-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:25 crc kubenswrapper[5052]: I1203 11:47:25.809694 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-8l45l\" (UniqueName: \"kubernetes.io/projected/84b8e975-30ee-42c6-8952-25aebf2c378b-kube-api-access-8l45l\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:25 crc kubenswrapper[5052]: I1203 11:47:25.809705 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/84b8e975-30ee-42c6-8952-25aebf2c378b-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:25 crc kubenswrapper[5052]: I1203 11:47:25.809728 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage12-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage12-crc\") on node \"crc\" " Dec 03 11:47:25 crc kubenswrapper[5052]: I1203 11:47:25.809739 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-fks6s\" (UniqueName: \"kubernetes.io/projected/83be8c2e-57f8-4b45-b3ca-096f72a1defd-kube-api-access-fks6s\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:25 crc kubenswrapper[5052]: I1203 11:47:25.809753 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/83be8c2e-57f8-4b45-b3ca-096f72a1defd-config-data-default\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:25 crc kubenswrapper[5052]: I1203 11:47:25.816464 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/kube-state-metrics-0"] Dec 03 11:47:25 crc kubenswrapper[5052]: I1203 11:47:25.816495 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/kube-state-metrics-0"] Dec 03 11:47:25 crc kubenswrapper[5052]: I1203 11:47:25.816535 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/swift-proxy-6fdf66df4-gsqlq"] Dec 03 11:47:25 crc kubenswrapper[5052]: I1203 11:47:25.816546 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/swift-proxy-6fdf66df4-gsqlq"] Dec 03 11:47:25 crc kubenswrapper[5052]: I1203 11:47:25.816557 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-sb-0"] Dec 03 11:47:25 crc kubenswrapper[5052]: I1203 11:47:25.816566 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-sb-0"] Dec 03 11:47:25 crc kubenswrapper[5052]: I1203 11:47:25.821007 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 11:47:25 crc kubenswrapper[5052]: I1203 11:47:25.837944 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/84b8e975-30ee-42c6-8952-25aebf2c378b-vencrypt-tls-certs" (OuterVolumeSpecName: "vencrypt-tls-certs") pod "84b8e975-30ee-42c6-8952-25aebf2c378b" (UID: "84b8e975-30ee-42c6-8952-25aebf2c378b"). InnerVolumeSpecName "vencrypt-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:47:25 crc kubenswrapper[5052]: I1203 11:47:25.855894 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/84b8e975-30ee-42c6-8952-25aebf2c378b-nova-novncproxy-tls-certs" (OuterVolumeSpecName: "nova-novncproxy-tls-certs") pod "84b8e975-30ee-42c6-8952-25aebf2c378b" (UID: "84b8e975-30ee-42c6-8952-25aebf2c378b"). InnerVolumeSpecName "nova-novncproxy-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:47:25 crc kubenswrapper[5052]: I1203 11:47:25.910608 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"mysql-db\" (UniqueName: \"kubernetes.io/local-volume/local-storage20-crc\") pod \"e741a60a-1a4f-45a1-b315-064fb7458e8f\" (UID: \"e741a60a-1a4f-45a1-b315-064fb7458e8f\") " Dec 03 11:47:25 crc kubenswrapper[5052]: I1203 11:47:25.910698 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/e741a60a-1a4f-45a1-b315-064fb7458e8f-operator-scripts\") pod \"e741a60a-1a4f-45a1-b315-064fb7458e8f\" (UID: \"e741a60a-1a4f-45a1-b315-064fb7458e8f\") " Dec 03 11:47:25 crc kubenswrapper[5052]: I1203 11:47:25.910808 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e741a60a-1a4f-45a1-b315-064fb7458e8f-combined-ca-bundle\") pod \"e741a60a-1a4f-45a1-b315-064fb7458e8f\" (UID: \"e741a60a-1a4f-45a1-b315-064fb7458e8f\") " Dec 03 11:47:25 crc kubenswrapper[5052]: I1203 11:47:25.910902 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/e741a60a-1a4f-45a1-b315-064fb7458e8f-galera-tls-certs\") pod \"e741a60a-1a4f-45a1-b315-064fb7458e8f\" (UID: \"e741a60a-1a4f-45a1-b315-064fb7458e8f\") " Dec 03 11:47:25 crc kubenswrapper[5052]: I1203 11:47:25.910965 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/e741a60a-1a4f-45a1-b315-064fb7458e8f-config-data-generated\") pod \"e741a60a-1a4f-45a1-b315-064fb7458e8f\" (UID: \"e741a60a-1a4f-45a1-b315-064fb7458e8f\") " Dec 03 11:47:25 crc kubenswrapper[5052]: I1203 11:47:25.910990 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/e741a60a-1a4f-45a1-b315-064fb7458e8f-kolla-config\") pod \"e741a60a-1a4f-45a1-b315-064fb7458e8f\" (UID: \"e741a60a-1a4f-45a1-b315-064fb7458e8f\") " Dec 03 11:47:25 crc kubenswrapper[5052]: I1203 11:47:25.911017 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n29tl\" (UniqueName: \"kubernetes.io/projected/e741a60a-1a4f-45a1-b315-064fb7458e8f-kube-api-access-n29tl\") pod \"e741a60a-1a4f-45a1-b315-064fb7458e8f\" (UID: \"e741a60a-1a4f-45a1-b315-064fb7458e8f\") " Dec 03 11:47:25 crc kubenswrapper[5052]: I1203 11:47:25.911045 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/e741a60a-1a4f-45a1-b315-064fb7458e8f-config-data-default\") pod \"e741a60a-1a4f-45a1-b315-064fb7458e8f\" (UID: \"e741a60a-1a4f-45a1-b315-064fb7458e8f\") " Dec 03 11:47:25 crc kubenswrapper[5052]: I1203 11:47:25.911461 5052 reconciler_common.go:293] "Volume detached for volume \"vencrypt-tls-certs\" (UniqueName: \"kubernetes.io/secret/84b8e975-30ee-42c6-8952-25aebf2c378b-vencrypt-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:25 crc kubenswrapper[5052]: I1203 11:47:25.911482 5052 reconciler_common.go:293] "Volume detached for volume \"nova-novncproxy-tls-certs\" (UniqueName: \"kubernetes.io/secret/84b8e975-30ee-42c6-8952-25aebf2c378b-nova-novncproxy-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:25 crc kubenswrapper[5052]: I1203 11:47:25.911937 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e741a60a-1a4f-45a1-b315-064fb7458e8f-config-data-default" (OuterVolumeSpecName: "config-data-default") pod "e741a60a-1a4f-45a1-b315-064fb7458e8f" (UID: "e741a60a-1a4f-45a1-b315-064fb7458e8f"). InnerVolumeSpecName "config-data-default". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:47:25 crc kubenswrapper[5052]: I1203 11:47:25.914313 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e741a60a-1a4f-45a1-b315-064fb7458e8f-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "e741a60a-1a4f-45a1-b315-064fb7458e8f" (UID: "e741a60a-1a4f-45a1-b315-064fb7458e8f"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:47:25 crc kubenswrapper[5052]: I1203 11:47:25.915033 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e741a60a-1a4f-45a1-b315-064fb7458e8f-kolla-config" (OuterVolumeSpecName: "kolla-config") pod "e741a60a-1a4f-45a1-b315-064fb7458e8f" (UID: "e741a60a-1a4f-45a1-b315-064fb7458e8f"). InnerVolumeSpecName "kolla-config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:47:25 crc kubenswrapper[5052]: I1203 11:47:25.915997 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e741a60a-1a4f-45a1-b315-064fb7458e8f-config-data-generated" (OuterVolumeSpecName: "config-data-generated") pod "e741a60a-1a4f-45a1-b315-064fb7458e8f" (UID: "e741a60a-1a4f-45a1-b315-064fb7458e8f"). InnerVolumeSpecName "config-data-generated". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:47:25 crc kubenswrapper[5052]: I1203 11:47:25.924976 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-db-create-l4qfz"] Dec 03 11:47:25 crc kubenswrapper[5052]: I1203 11:47:25.940201 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/cinder-api-0" podUID="00b12739-1437-4af3-ab6d-ad830b67ad2e" containerName="cinder-api" probeResult="failure" output="Get \"https://10.217.0.252:8776/healthcheck\": read tcp 10.217.0.2:52334->10.217.0.252:8776: read: connection reset by peer" Dec 03 11:47:25 crc kubenswrapper[5052]: I1203 11:47:25.945678 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e741a60a-1a4f-45a1-b315-064fb7458e8f-kube-api-access-n29tl" (OuterVolumeSpecName: "kube-api-access-n29tl") pod "e741a60a-1a4f-45a1-b315-064fb7458e8f" (UID: "e741a60a-1a4f-45a1-b315-064fb7458e8f"). InnerVolumeSpecName "kube-api-access-n29tl". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:47:25 crc kubenswrapper[5052]: I1203 11:47:25.947903 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/glance-db-create-l4qfz"] Dec 03 11:47:25 crc kubenswrapper[5052]: I1203 11:47:25.962587 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance3554-account-delete-8ghdx"] Dec 03 11:47:25 crc kubenswrapper[5052]: I1203 11:47:25.972596 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/rabbitmq-server-0" podUID="d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0" containerName="rabbitmq" probeResult="failure" output="dial tcp 10.217.0.204:5671: connect: connection refused" Dec 03 11:47:25 crc kubenswrapper[5052]: I1203 11:47:25.973019 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-3554-account-create-update-n48gm"] Dec 03 11:47:25 crc kubenswrapper[5052]: I1203 11:47:25.983533 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage12-crc" (UniqueName: "kubernetes.io/local-volume/local-storage12-crc") on node "crc" Dec 03 11:47:25 crc kubenswrapper[5052]: I1203 11:47:25.988169 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/glance-3554-account-create-update-n48gm"] Dec 03 11:47:25 crc kubenswrapper[5052]: I1203 11:47:25.992974 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage20-crc" (OuterVolumeSpecName: "mysql-db") pod "e741a60a-1a4f-45a1-b315-064fb7458e8f" (UID: "e741a60a-1a4f-45a1-b315-064fb7458e8f"). InnerVolumeSpecName "local-storage20-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 11:47:26 crc kubenswrapper[5052]: I1203 11:47:26.002124 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/83be8c2e-57f8-4b45-b3ca-096f72a1defd-galera-tls-certs" (OuterVolumeSpecName: "galera-tls-certs") pod "83be8c2e-57f8-4b45-b3ca-096f72a1defd" (UID: "83be8c2e-57f8-4b45-b3ca-096f72a1defd"). InnerVolumeSpecName "galera-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:47:26 crc kubenswrapper[5052]: I1203 11:47:26.013374 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage12-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage12-crc\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:26 crc kubenswrapper[5052]: I1203 11:47:26.013413 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/e741a60a-1a4f-45a1-b315-064fb7458e8f-config-data-generated\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:26 crc kubenswrapper[5052]: I1203 11:47:26.013425 5052 reconciler_common.go:293] "Volume detached for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/e741a60a-1a4f-45a1-b315-064fb7458e8f-kolla-config\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:26 crc kubenswrapper[5052]: I1203 11:47:26.013439 5052 reconciler_common.go:293] "Volume detached for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/83be8c2e-57f8-4b45-b3ca-096f72a1defd-galera-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:26 crc kubenswrapper[5052]: I1203 11:47:26.013450 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-n29tl\" (UniqueName: \"kubernetes.io/projected/e741a60a-1a4f-45a1-b315-064fb7458e8f-kube-api-access-n29tl\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:26 crc kubenswrapper[5052]: I1203 11:47:26.013462 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/e741a60a-1a4f-45a1-b315-064fb7458e8f-config-data-default\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:26 crc kubenswrapper[5052]: I1203 11:47:26.013492 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage20-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage20-crc\") on node \"crc\" " Dec 03 11:47:26 crc kubenswrapper[5052]: I1203 11:47:26.013504 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/e741a60a-1a4f-45a1-b315-064fb7458e8f-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:26 crc kubenswrapper[5052]: I1203 11:47:26.089568 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage20-crc" (UniqueName: "kubernetes.io/local-volume/local-storage20-crc") on node "crc" Dec 03 11:47:26 crc kubenswrapper[5052]: I1203 11:47:26.115850 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/placement-db-create-8tl8n"] Dec 03 11:47:26 crc kubenswrapper[5052]: E1203 11:47:26.116031 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 11:47:26 crc kubenswrapper[5052]: E1203 11:47:26.116083 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 11:47:26 crc kubenswrapper[5052]: E1203 11:47:26.116146 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/950f0626-40d2-41c9-9701-cba50843c43e-operator-scripts podName:950f0626-40d2-41c9-9701-cba50843c43e nodeName:}" failed. No retries permitted until 2025-12-03 11:47:27.116099243 +0000 UTC m=+1637.704341077 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/950f0626-40d2-41c9-9701-cba50843c43e-operator-scripts") pod "glance3554-account-delete-8ghdx" (UID: "950f0626-40d2-41c9-9701-cba50843c43e") : configmap "openstack-scripts" not found Dec 03 11:47:26 crc kubenswrapper[5052]: E1203 11:47:26.116151 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 11:47:26 crc kubenswrapper[5052]: E1203 11:47:26.116184 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8c2252c0-5e30-4e38-b1a1-af0242df6ac0-operator-scripts podName:8c2252c0-5e30-4e38-b1a1-af0242df6ac0 nodeName:}" failed. No retries permitted until 2025-12-03 11:47:27.116161595 +0000 UTC m=+1637.704403439 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/8c2252c0-5e30-4e38-b1a1-af0242df6ac0-operator-scripts") pod "placement9a4f-account-delete-f6sjd" (UID: "8c2252c0-5e30-4e38-b1a1-af0242df6ac0") : configmap "openstack-scripts" not found Dec 03 11:47:26 crc kubenswrapper[5052]: I1203 11:47:26.116048 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage20-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage20-crc\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:26 crc kubenswrapper[5052]: E1203 11:47:26.116227 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/aa9a649b-9229-4cdd-8e27-8aedd48f4bbd-operator-scripts podName:aa9a649b-9229-4cdd-8e27-8aedd48f4bbd nodeName:}" failed. No retries permitted until 2025-12-03 11:47:28.116208976 +0000 UTC m=+1638.704450810 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/aa9a649b-9229-4cdd-8e27-8aedd48f4bbd-operator-scripts") pod "neutron18a7-account-delete-x9lqd" (UID: "aa9a649b-9229-4cdd-8e27-8aedd48f4bbd") : configmap "openstack-scripts" not found Dec 03 11:47:26 crc kubenswrapper[5052]: I1203 11:47:26.125611 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/placement-db-create-8tl8n"] Dec 03 11:47:26 crc kubenswrapper[5052]: I1203 11:47:26.139483 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/placement-9a4f-account-create-update-gs4xv"] Dec 03 11:47:26 crc kubenswrapper[5052]: I1203 11:47:26.149673 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e741a60a-1a4f-45a1-b315-064fb7458e8f-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "e741a60a-1a4f-45a1-b315-064fb7458e8f" (UID: "e741a60a-1a4f-45a1-b315-064fb7458e8f"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:47:26 crc kubenswrapper[5052]: I1203 11:47:26.151686 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/placement9a4f-account-delete-f6sjd"] Dec 03 11:47:26 crc kubenswrapper[5052]: I1203 11:47:26.159016 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/placement-9a4f-account-create-update-gs4xv"] Dec 03 11:47:26 crc kubenswrapper[5052]: I1203 11:47:26.182818 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e741a60a-1a4f-45a1-b315-064fb7458e8f-galera-tls-certs" (OuterVolumeSpecName: "galera-tls-certs") pod "e741a60a-1a4f-45a1-b315-064fb7458e8f" (UID: "e741a60a-1a4f-45a1-b315-064fb7458e8f"). InnerVolumeSpecName "galera-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:47:26 crc kubenswrapper[5052]: I1203 11:47:26.213776 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder-db-create-57t99"] Dec 03 11:47:26 crc kubenswrapper[5052]: I1203 11:47:26.217450 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e741a60a-1a4f-45a1-b315-064fb7458e8f-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:26 crc kubenswrapper[5052]: I1203 11:47:26.217470 5052 reconciler_common.go:293] "Volume detached for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/e741a60a-1a4f-45a1-b315-064fb7458e8f-galera-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:26 crc kubenswrapper[5052]: I1203 11:47:26.221565 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/cinder-db-create-57t99"] Dec 03 11:47:26 crc kubenswrapper[5052]: I1203 11:47:26.235255 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder-1bf0-account-create-update-pvvl8"] Dec 03 11:47:26 crc kubenswrapper[5052]: I1203 11:47:26.242915 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder1bf0-account-delete-hs8j6"] Dec 03 11:47:26 crc kubenswrapper[5052]: I1203 11:47:26.252362 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/cinder-1bf0-account-create-update-pvvl8"] Dec 03 11:47:26 crc kubenswrapper[5052]: I1203 11:47:26.282567 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/neutron-db-create-n5v7p"] Dec 03 11:47:26 crc kubenswrapper[5052]: I1203 11:47:26.298042 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/neutron-db-create-n5v7p"] Dec 03 11:47:26 crc kubenswrapper[5052]: I1203 11:47:26.317506 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/neutron18a7-account-delete-x9lqd"] Dec 03 11:47:26 crc kubenswrapper[5052]: I1203 11:47:26.330123 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/neutron-18a7-account-create-update-n4q4t"] Dec 03 11:47:26 crc kubenswrapper[5052]: I1203 11:47:26.349696 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/neutron-18a7-account-create-update-n4q4t"] Dec 03 11:47:26 crc kubenswrapper[5052]: I1203 11:47:26.359029 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-db-create-hvzcc"] Dec 03 11:47:26 crc kubenswrapper[5052]: I1203 11:47:26.371788 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/barbican-db-create-hvzcc"] Dec 03 11:47:26 crc kubenswrapper[5052]: I1203 11:47:26.388138 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-5976-account-create-update-dqgqh"] Dec 03 11:47:26 crc kubenswrapper[5052]: I1203 11:47:26.401802 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican5976-account-delete-q6j5s"] Dec 03 11:47:26 crc kubenswrapper[5052]: I1203 11:47:26.406285 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/barbican-5976-account-create-update-dqgqh"] Dec 03 11:47:26 crc kubenswrapper[5052]: I1203 11:47:26.424378 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/barbican-api-7cfb847cb8-jn85t" podUID="99f96963-d835-4445-84af-0976f1ba0701" containerName="barbican-api" probeResult="failure" output="Get \"https://10.217.0.253:9311/healthcheck\": read tcp 10.217.0.2:43214->10.217.0.253:9311: read: connection reset by peer" Dec 03 11:47:26 crc kubenswrapper[5052]: I1203 11:47:26.424598 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/barbican-api-7cfb847cb8-jn85t" podUID="99f96963-d835-4445-84af-0976f1ba0701" containerName="barbican-api-log" probeResult="failure" output="Get \"https://10.217.0.253:9311/healthcheck\": read tcp 10.217.0.2:43226->10.217.0.253:9311: read: connection reset by peer" Dec 03 11:47:26 crc kubenswrapper[5052]: I1203 11:47:26.565060 5052 generic.go:334] "Generic (PLEG): container finished" podID="0183ebb5-b91b-443a-9535-2008c9207ab1" containerID="81c366447116b43e07821471a9d38ae69fcdb20b1db58e745b91476f5acbbc2f" exitCode=0 Dec 03 11:47:26 crc kubenswrapper[5052]: I1203 11:47:26.565137 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-689c6465d6-nsjrx" event={"ID":"0183ebb5-b91b-443a-9535-2008c9207ab1","Type":"ContainerDied","Data":"81c366447116b43e07821471a9d38ae69fcdb20b1db58e745b91476f5acbbc2f"} Dec 03 11:47:26 crc kubenswrapper[5052]: I1203 11:47:26.580270 5052 generic.go:334] "Generic (PLEG): container finished" podID="0914ebfe-13c6-4309-bd10-a67850124e3f" containerID="cc31e16d3323f33d812b30383dad0c83a6bb71816abde9316f28a1841980791c" exitCode=0 Dec 03 11:47:26 crc kubenswrapper[5052]: I1203 11:47:26.580437 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"0914ebfe-13c6-4309-bd10-a67850124e3f","Type":"ContainerDied","Data":"cc31e16d3323f33d812b30383dad0c83a6bb71816abde9316f28a1841980791c"} Dec 03 11:47:26 crc kubenswrapper[5052]: I1203 11:47:26.585478 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican5976-account-delete-q6j5s" event={"ID":"0aefd6f6-cbb3-4c45-9f69-7124e43255e3","Type":"ContainerStarted","Data":"12cedf118f41103f94919b8993790529cc71a18a79a829fcc52ee74432f113ee"} Dec 03 11:47:26 crc kubenswrapper[5052]: I1203 11:47:26.588354 5052 generic.go:334] "Generic (PLEG): container finished" podID="cadac9a6-1d82-403a-b7a0-692a424dd0e2" containerID="38aac1236f884983fd56130cde1141029030d5995e585474bc43e38342fcd566" exitCode=0 Dec 03 11:47:26 crc kubenswrapper[5052]: I1203 11:47:26.588428 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-worker-79689f8957-qmkdm" event={"ID":"cadac9a6-1d82-403a-b7a0-692a424dd0e2","Type":"ContainerDied","Data":"38aac1236f884983fd56130cde1141029030d5995e585474bc43e38342fcd566"} Dec 03 11:47:26 crc kubenswrapper[5052]: I1203 11:47:26.590772 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/novaapi5919-account-delete-qkvzd" event={"ID":"2d3a3054-6c8e-4808-bd5d-01413970167c","Type":"ContainerStarted","Data":"c9d59899fe21862fbe05cb4f9b40af488d9a06ef3ce0bca59d2b5c6edc68c2cd"} Dec 03 11:47:26 crc kubenswrapper[5052]: I1203 11:47:26.593325 5052 generic.go:334] "Generic (PLEG): container finished" podID="acea154c-f0bb-4066-bd4d-2dda815f301c" containerID="e054bf606eb8ede8b913bb4874e3d9c8bf93f40bc4a7246b741ee83f941471a3" exitCode=0 Dec 03 11:47:26 crc kubenswrapper[5052]: I1203 11:47:26.593380 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"acea154c-f0bb-4066-bd4d-2dda815f301c","Type":"ContainerDied","Data":"e054bf606eb8ede8b913bb4874e3d9c8bf93f40bc4a7246b741ee83f941471a3"} Dec 03 11:47:26 crc kubenswrapper[5052]: I1203 11:47:26.599352 5052 generic.go:334] "Generic (PLEG): container finished" podID="00b12739-1437-4af3-ab6d-ad830b67ad2e" containerID="988a528826a2862aed42fdaece9c254222b91bc11952d4cd54b0b18c778396b2" exitCode=0 Dec 03 11:47:26 crc kubenswrapper[5052]: I1203 11:47:26.599416 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"00b12739-1437-4af3-ab6d-ad830b67ad2e","Type":"ContainerDied","Data":"988a528826a2862aed42fdaece9c254222b91bc11952d4cd54b0b18c778396b2"} Dec 03 11:47:26 crc kubenswrapper[5052]: I1203 11:47:26.603008 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 11:47:26 crc kubenswrapper[5052]: I1203 11:47:26.604646 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 11:47:26 crc kubenswrapper[5052]: I1203 11:47:26.605970 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder1bf0-account-delete-hs8j6" event={"ID":"0b94b517-5f9b-485d-a109-0a11e15a3ad3","Type":"ContainerStarted","Data":"d29340e2b19190d615ecfc711e5c8befec183416c40af1d92bde9b4447a33954"} Dec 03 11:47:26 crc kubenswrapper[5052]: I1203 11:47:26.606043 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 11:47:26 crc kubenswrapper[5052]: I1203 11:47:26.647741 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/openstack-cell1-galera-0"] Dec 03 11:47:26 crc kubenswrapper[5052]: I1203 11:47:26.659469 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/openstack-cell1-galera-0"] Dec 03 11:47:26 crc kubenswrapper[5052]: I1203 11:47:26.669538 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 11:47:26 crc kubenswrapper[5052]: E1203 11:47:26.671291 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="3b43b29211f1c83741ff8d9a266e5e11a2e9560cc3168a7ef1db16f7622c61ba" cmd=["/usr/local/bin/container-scripts/status_check.sh"] Dec 03 11:47:26 crc kubenswrapper[5052]: E1203 11:47:26.673989 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="3b43b29211f1c83741ff8d9a266e5e11a2e9560cc3168a7ef1db16f7622c61ba" cmd=["/usr/local/bin/container-scripts/status_check.sh"] Dec 03 11:47:26 crc kubenswrapper[5052]: E1203 11:47:26.675182 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="3b43b29211f1c83741ff8d9a266e5e11a2e9560cc3168a7ef1db16f7622c61ba" cmd=["/usr/local/bin/container-scripts/status_check.sh"] Dec 03 11:47:26 crc kubenswrapper[5052]: E1203 11:47:26.675211 5052 prober.go:104] "Probe errored" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" probeType="Readiness" pod="openstack-kuttl-tests/ovn-northd-0" podUID="83a10afe-30d8-443d-a4f7-28784a2ea068" containerName="ovn-northd" Dec 03 11:47:26 crc kubenswrapper[5052]: I1203 11:47:26.679587 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 11:47:26 crc kubenswrapper[5052]: I1203 11:47:26.684714 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/openstack-galera-0"] Dec 03 11:47:26 crc kubenswrapper[5052]: I1203 11:47:26.692017 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/openstack-galera-0"] Dec 03 11:47:26 crc kubenswrapper[5052]: E1203 11:47:26.730411 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/combined-ca-bundle: secret "combined-ca-bundle" not found Dec 03 11:47:26 crc kubenswrapper[5052]: E1203 11:47:26.730479 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f9b208b-086f-4671-8c3f-b3a275740ad7-combined-ca-bundle podName:6f9b208b-086f-4671-8c3f-b3a275740ad7 nodeName:}" failed. No retries permitted until 2025-12-03 11:47:27.230462953 +0000 UTC m=+1637.818704707 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "combined-ca-bundle" (UniqueName: "kubernetes.io/secret/6f9b208b-086f-4671-8c3f-b3a275740ad7-combined-ca-bundle") pod "memcached-0" (UID: "6f9b208b-086f-4671-8c3f-b3a275740ad7") : secret "combined-ca-bundle" not found Dec 03 11:47:26 crc kubenswrapper[5052]: E1203 11:47:26.730627 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-memcached-svc: secret "cert-memcached-svc" not found Dec 03 11:47:26 crc kubenswrapper[5052]: E1203 11:47:26.730650 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f9b208b-086f-4671-8c3f-b3a275740ad7-memcached-tls-certs podName:6f9b208b-086f-4671-8c3f-b3a275740ad7 nodeName:}" failed. No retries permitted until 2025-12-03 11:47:27.230643478 +0000 UTC m=+1637.818885242 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "memcached-tls-certs" (UniqueName: "kubernetes.io/secret/6f9b208b-086f-4671-8c3f-b3a275740ad7-memcached-tls-certs") pod "memcached-0" (UID: "6f9b208b-086f-4671-8c3f-b3a275740ad7") : secret "cert-memcached-svc" not found Dec 03 11:47:26 crc kubenswrapper[5052]: I1203 11:47:26.779327 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-api-db-create-bmscg"] Dec 03 11:47:26 crc kubenswrapper[5052]: I1203 11:47:26.783396 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-api-db-create-bmscg"] Dec 03 11:47:26 crc kubenswrapper[5052]: I1203 11:47:26.840506 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-api-5919-account-create-update-wdllt"] Dec 03 11:47:26 crc kubenswrapper[5052]: I1203 11:47:26.859323 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/novaapi5919-account-delete-qkvzd"] Dec 03 11:47:26 crc kubenswrapper[5052]: I1203 11:47:26.880588 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-api-5919-account-create-update-wdllt"] Dec 03 11:47:26 crc kubenswrapper[5052]: I1203 11:47:26.900390 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell0-db-create-7mslc"] Dec 03 11:47:26 crc kubenswrapper[5052]: I1203 11:47:26.907492 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell0-db-create-7mslc"] Dec 03 11:47:26 crc kubenswrapper[5052]: I1203 11:47:26.908447 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/nova-metadata-0" podUID="bb51da3b-72cc-4008-aac7-737f6a3031cb" containerName="nova-metadata-metadata" probeResult="failure" output="Get \"https://10.217.1.32:8775/\": dial tcp 10.217.1.32:8775: connect: connection refused" Dec 03 11:47:26 crc kubenswrapper[5052]: I1203 11:47:26.908480 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/nova-metadata-0" podUID="bb51da3b-72cc-4008-aac7-737f6a3031cb" containerName="nova-metadata-log" probeResult="failure" output="Get \"https://10.217.1.32:8775/\": dial tcp 10.217.1.32:8775: connect: connection refused" Dec 03 11:47:26 crc kubenswrapper[5052]: I1203 11:47:26.945626 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/novacell0f13b-account-delete-g7m79"] Dec 03 11:47:26 crc kubenswrapper[5052]: I1203 11:47:26.969163 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell0-f13b-account-create-update-p7l6b"] Dec 03 11:47:26 crc kubenswrapper[5052]: I1203 11:47:26.976759 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell0-f13b-account-create-update-p7l6b"] Dec 03 11:47:27 crc kubenswrapper[5052]: E1203 11:47:27.140387 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 11:47:27 crc kubenswrapper[5052]: E1203 11:47:27.140840 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/950f0626-40d2-41c9-9701-cba50843c43e-operator-scripts podName:950f0626-40d2-41c9-9701-cba50843c43e nodeName:}" failed. No retries permitted until 2025-12-03 11:47:29.140813792 +0000 UTC m=+1639.729055626 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/950f0626-40d2-41c9-9701-cba50843c43e-operator-scripts") pod "glance3554-account-delete-8ghdx" (UID: "950f0626-40d2-41c9-9701-cba50843c43e") : configmap "openstack-scripts" not found Dec 03 11:47:27 crc kubenswrapper[5052]: E1203 11:47:27.140535 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 11:47:27 crc kubenswrapper[5052]: E1203 11:47:27.140957 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8c2252c0-5e30-4e38-b1a1-af0242df6ac0-operator-scripts podName:8c2252c0-5e30-4e38-b1a1-af0242df6ac0 nodeName:}" failed. No retries permitted until 2025-12-03 11:47:29.140936876 +0000 UTC m=+1639.729178730 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/8c2252c0-5e30-4e38-b1a1-af0242df6ac0-operator-scripts") pod "placement9a4f-account-delete-f6sjd" (UID: "8c2252c0-5e30-4e38-b1a1-af0242df6ac0") : configmap "openstack-scripts" not found Dec 03 11:47:27 crc kubenswrapper[5052]: E1203 11:47:27.242145 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-memcached-svc: secret "cert-memcached-svc" not found Dec 03 11:47:27 crc kubenswrapper[5052]: E1203 11:47:27.242201 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/combined-ca-bundle: secret "combined-ca-bundle" not found Dec 03 11:47:27 crc kubenswrapper[5052]: E1203 11:47:27.242261 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f9b208b-086f-4671-8c3f-b3a275740ad7-memcached-tls-certs podName:6f9b208b-086f-4671-8c3f-b3a275740ad7 nodeName:}" failed. No retries permitted until 2025-12-03 11:47:28.242239561 +0000 UTC m=+1638.830481345 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "memcached-tls-certs" (UniqueName: "kubernetes.io/secret/6f9b208b-086f-4671-8c3f-b3a275740ad7-memcached-tls-certs") pod "memcached-0" (UID: "6f9b208b-086f-4671-8c3f-b3a275740ad7") : secret "cert-memcached-svc" not found Dec 03 11:47:27 crc kubenswrapper[5052]: E1203 11:47:27.242283 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f9b208b-086f-4671-8c3f-b3a275740ad7-combined-ca-bundle podName:6f9b208b-086f-4671-8c3f-b3a275740ad7 nodeName:}" failed. No retries permitted until 2025-12-03 11:47:28.242274422 +0000 UTC m=+1638.830516206 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "combined-ca-bundle" (UniqueName: "kubernetes.io/secret/6f9b208b-086f-4671-8c3f-b3a275740ad7-combined-ca-bundle") pod "memcached-0" (UID: "6f9b208b-086f-4671-8c3f-b3a275740ad7") : secret "combined-ca-bundle" not found Dec 03 11:47:27 crc kubenswrapper[5052]: E1203 11:47:27.547935 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/rabbitmq-config-data: configmap "rabbitmq-config-data" not found Dec 03 11:47:27 crc kubenswrapper[5052]: E1203 11:47:27.548006 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0-config-data podName:d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0 nodeName:}" failed. No retries permitted until 2025-12-03 11:47:31.54798885 +0000 UTC m=+1642.136230614 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "config-data" (UniqueName: "kubernetes.io/configmap/d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0-config-data") pod "rabbitmq-server-0" (UID: "d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0") : configmap "rabbitmq-config-data" not found Dec 03 11:47:27 crc kubenswrapper[5052]: I1203 11:47:27.615114 5052 generic.go:334] "Generic (PLEG): container finished" podID="e5339997-33d5-4794-8fa6-042b63e9391e" containerID="e8bef7322f8817aeda29d79f6179dd161790705405191177d57b564a93045852" exitCode=1 Dec 03 11:47:27 crc kubenswrapper[5052]: I1203 11:47:27.615187 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/novacell0f13b-account-delete-g7m79" event={"ID":"e5339997-33d5-4794-8fa6-042b63e9391e","Type":"ContainerDied","Data":"e8bef7322f8817aeda29d79f6179dd161790705405191177d57b564a93045852"} Dec 03 11:47:27 crc kubenswrapper[5052]: I1203 11:47:27.615777 5052 kubelet_pods.go:1007] "Unable to retrieve pull secret, the image pull may not succeed." pod="openstack-kuttl-tests/novacell0f13b-account-delete-g7m79" secret="" err="secret \"galera-openstack-dockercfg-cpskp\" not found" Dec 03 11:47:27 crc kubenswrapper[5052]: I1203 11:47:27.615861 5052 scope.go:117] "RemoveContainer" containerID="e8bef7322f8817aeda29d79f6179dd161790705405191177d57b564a93045852" Dec 03 11:47:27 crc kubenswrapper[5052]: I1203 11:47:27.622828 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-worker-79689f8957-qmkdm" event={"ID":"cadac9a6-1d82-403a-b7a0-692a424dd0e2","Type":"ContainerDied","Data":"de6975120318e384ab085701669f1c586c0650ffb743e7c4a4482d13b9ed77b0"} Dec 03 11:47:27 crc kubenswrapper[5052]: I1203 11:47:27.622873 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="de6975120318e384ab085701669f1c586c0650ffb743e7c4a4482d13b9ed77b0" Dec 03 11:47:27 crc kubenswrapper[5052]: I1203 11:47:27.629487 5052 generic.go:334] "Generic (PLEG): container finished" podID="99f96963-d835-4445-84af-0976f1ba0701" containerID="95b8a0e92f2ae27cf80dcce2b2af098739e919963ebae7c22edd8d9c46c89ddc" exitCode=0 Dec 03 11:47:27 crc kubenswrapper[5052]: I1203 11:47:27.629537 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-7cfb847cb8-jn85t" event={"ID":"99f96963-d835-4445-84af-0976f1ba0701","Type":"ContainerDied","Data":"95b8a0e92f2ae27cf80dcce2b2af098739e919963ebae7c22edd8d9c46c89ddc"} Dec 03 11:47:27 crc kubenswrapper[5052]: E1203 11:47:27.650257 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 11:47:27 crc kubenswrapper[5052]: E1203 11:47:27.650329 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e5339997-33d5-4794-8fa6-042b63e9391e-operator-scripts podName:e5339997-33d5-4794-8fa6-042b63e9391e nodeName:}" failed. No retries permitted until 2025-12-03 11:47:28.150310665 +0000 UTC m=+1638.738552429 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/e5339997-33d5-4794-8fa6-042b63e9391e-operator-scripts") pod "novacell0f13b-account-delete-g7m79" (UID: "e5339997-33d5-4794-8fa6-042b63e9391e") : configmap "openstack-scripts" not found Dec 03 11:47:27 crc kubenswrapper[5052]: I1203 11:47:27.652913 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"00b12739-1437-4af3-ab6d-ad830b67ad2e","Type":"ContainerDied","Data":"e1bd7d7121261e1a387aa5383584eca243918b291d77768c5cb2f1fea3f2450c"} Dec 03 11:47:27 crc kubenswrapper[5052]: I1203 11:47:27.653018 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e1bd7d7121261e1a387aa5383584eca243918b291d77768c5cb2f1fea3f2450c" Dec 03 11:47:27 crc kubenswrapper[5052]: I1203 11:47:27.665627 5052 generic.go:334] "Generic (PLEG): container finished" podID="8c2252c0-5e30-4e38-b1a1-af0242df6ac0" containerID="19fe01d36f124c6e5b005cff9acb3ba773fdc1a82d383392d6ca61a3276e293b" exitCode=1 Dec 03 11:47:27 crc kubenswrapper[5052]: I1203 11:47:27.665701 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement9a4f-account-delete-f6sjd" event={"ID":"8c2252c0-5e30-4e38-b1a1-af0242df6ac0","Type":"ContainerDied","Data":"19fe01d36f124c6e5b005cff9acb3ba773fdc1a82d383392d6ca61a3276e293b"} Dec 03 11:47:27 crc kubenswrapper[5052]: I1203 11:47:27.665738 5052 scope.go:117] "RemoveContainer" containerID="0b9fcdc42fa6273f3826d09c4e10faa45a8df9ed00f3f08f07d74ddbc5eea2b1" Dec 03 11:47:27 crc kubenswrapper[5052]: I1203 11:47:27.675241 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-worker-79689f8957-qmkdm" Dec 03 11:47:27 crc kubenswrapper[5052]: I1203 11:47:27.680833 5052 generic.go:334] "Generic (PLEG): container finished" podID="bb51da3b-72cc-4008-aac7-737f6a3031cb" containerID="b39f7b31c335958d6613651cea9f3ccbfee7c25b7c7641048321f3e460ee0193" exitCode=0 Dec 03 11:47:27 crc kubenswrapper[5052]: I1203 11:47:27.708790 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:47:27 crc kubenswrapper[5052]: I1203 11:47:27.715870 5052 generic.go:334] "Generic (PLEG): container finished" podID="950f0626-40d2-41c9-9701-cba50843c43e" containerID="92c1a0e919d06f22f9f6723e6dddde204ff33caafc0c8594584022f38ab81e75" exitCode=1 Dec 03 11:47:27 crc kubenswrapper[5052]: I1203 11:47:27.727745 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:47:27 crc kubenswrapper[5052]: I1203 11:47:27.736915 5052 generic.go:334] "Generic (PLEG): container finished" podID="65be66f1-c32d-40b5-81a4-a63258364d9d" containerID="2be919af42c62148fa0417519c4eea762d9886b79020bd92b5f10692d1f30f31" exitCode=0 Dec 03 11:47:27 crc kubenswrapper[5052]: I1203 11:47:27.758244 5052 generic.go:334] "Generic (PLEG): container finished" podID="93fb4fd0-aa18-4a2d-bfb4-b65bbc378ae5" containerID="a278a573e396cd799f85833fa26f6bb09544244a8956130e15e5c8b299a9a039" exitCode=0 Dec 03 11:47:27 crc kubenswrapper[5052]: I1203 11:47:27.759566 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0c6c4cfd-0973-4ca6-beca-147213271fe3" path="/var/lib/kubelet/pods/0c6c4cfd-0973-4ca6-beca-147213271fe3/volumes" Dec 03 11:47:27 crc kubenswrapper[5052]: I1203 11:47:27.760336 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="23973edb-ef8e-4e30-a06c-308e9f48da5d" path="/var/lib/kubelet/pods/23973edb-ef8e-4e30-a06c-308e9f48da5d/volumes" Dec 03 11:47:27 crc kubenswrapper[5052]: I1203 11:47:27.761189 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2744f411-6860-47c0-a8d5-f52e23e390f3" path="/var/lib/kubelet/pods/2744f411-6860-47c0-a8d5-f52e23e390f3/volumes" Dec 03 11:47:27 crc kubenswrapper[5052]: I1203 11:47:27.765892 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="496a1006-5380-40d1-998f-129a87233300" path="/var/lib/kubelet/pods/496a1006-5380-40d1-998f-129a87233300/volumes" Dec 03 11:47:27 crc kubenswrapper[5052]: I1203 11:47:27.766788 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6d924068-a887-4c83-89ad-9f333eb94799" path="/var/lib/kubelet/pods/6d924068-a887-4c83-89ad-9f333eb94799/volumes" Dec 03 11:47:27 crc kubenswrapper[5052]: I1203 11:47:27.767372 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="75e340f6-2225-485f-8c78-53d05538a9c6" path="/var/lib/kubelet/pods/75e340f6-2225-485f-8c78-53d05538a9c6/volumes" Dec 03 11:47:27 crc kubenswrapper[5052]: I1203 11:47:27.768760 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7ebea5de-22d5-4950-a7f3-32d980eaba12" path="/var/lib/kubelet/pods/7ebea5de-22d5-4950-a7f3-32d980eaba12/volumes" Dec 03 11:47:27 crc kubenswrapper[5052]: I1203 11:47:27.774058 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="83be8c2e-57f8-4b45-b3ca-096f72a1defd" path="/var/lib/kubelet/pods/83be8c2e-57f8-4b45-b3ca-096f72a1defd/volumes" Dec 03 11:47:27 crc kubenswrapper[5052]: I1203 11:47:27.775053 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="84957e18-5747-4786-9c4f-6af023122726" path="/var/lib/kubelet/pods/84957e18-5747-4786-9c4f-6af023122726/volumes" Dec 03 11:47:27 crc kubenswrapper[5052]: I1203 11:47:27.776288 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="84b8e975-30ee-42c6-8952-25aebf2c378b" path="/var/lib/kubelet/pods/84b8e975-30ee-42c6-8952-25aebf2c378b/volumes" Dec 03 11:47:27 crc kubenswrapper[5052]: I1203 11:47:27.777492 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9bdd2626-de50-4513-8313-3b25347ff641" path="/var/lib/kubelet/pods/9bdd2626-de50-4513-8313-3b25347ff641/volumes" Dec 03 11:47:27 crc kubenswrapper[5052]: I1203 11:47:27.778107 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a0a527da-143e-4d58-a118-0a1afea55b1f" path="/var/lib/kubelet/pods/a0a527da-143e-4d58-a118-0a1afea55b1f/volumes" Dec 03 11:47:27 crc kubenswrapper[5052]: I1203 11:47:27.778585 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:47:27 crc kubenswrapper[5052]: I1203 11:47:27.783649 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-689c6465d6-nsjrx" Dec 03 11:47:27 crc kubenswrapper[5052]: I1203 11:47:27.783833 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:47:27 crc kubenswrapper[5052]: I1203 11:47:27.784789 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ae7af34b-d086-4215-b568-1bf5b75bbe39" path="/var/lib/kubelet/pods/ae7af34b-d086-4215-b568-1bf5b75bbe39/volumes" Dec 03 11:47:27 crc kubenswrapper[5052]: I1203 11:47:27.788640 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b85b18b0-5745-44b1-8add-a67d56d1d0c4" path="/var/lib/kubelet/pods/b85b18b0-5745-44b1-8add-a67d56d1d0c4/volumes" Dec 03 11:47:27 crc kubenswrapper[5052]: I1203 11:47:27.789202 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bd7f2307-3030-4f83-b90c-debe14f2da96" path="/var/lib/kubelet/pods/bd7f2307-3030-4f83-b90c-debe14f2da96/volumes" Dec 03 11:47:27 crc kubenswrapper[5052]: I1203 11:47:27.789699 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c97cfd9e-1b42-494f-8eb5-206bf9791575" path="/var/lib/kubelet/pods/c97cfd9e-1b42-494f-8eb5-206bf9791575/volumes" Dec 03 11:47:27 crc kubenswrapper[5052]: I1203 11:47:27.789870 5052 generic.go:334] "Generic (PLEG): container finished" podID="0aefd6f6-cbb3-4c45-9f69-7124e43255e3" containerID="12cedf118f41103f94919b8993790529cc71a18a79a829fcc52ee74432f113ee" exitCode=1 Dec 03 11:47:27 crc kubenswrapper[5052]: I1203 11:47:27.790132 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:47:27 crc kubenswrapper[5052]: I1203 11:47:27.790240 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d4682522-2ae7-4e71-95ff-c4453f145537" path="/var/lib/kubelet/pods/d4682522-2ae7-4e71-95ff-c4453f145537/volumes" Dec 03 11:47:27 crc kubenswrapper[5052]: I1203 11:47:27.793004 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d4c5312a-6993-40de-b9af-bbe3820ae84a" path="/var/lib/kubelet/pods/d4c5312a-6993-40de-b9af-bbe3820ae84a/volumes" Dec 03 11:47:27 crc kubenswrapper[5052]: I1203 11:47:27.793470 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dc73f58f-4601-4b36-92b1-707a5779c99e" path="/var/lib/kubelet/pods/dc73f58f-4601-4b36-92b1-707a5779c99e/volumes" Dec 03 11:47:27 crc kubenswrapper[5052]: I1203 11:47:27.794111 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e741a60a-1a4f-45a1-b315-064fb7458e8f" path="/var/lib/kubelet/pods/e741a60a-1a4f-45a1-b315-064fb7458e8f/volumes" Dec 03 11:47:27 crc kubenswrapper[5052]: I1203 11:47:27.795975 5052 generic.go:334] "Generic (PLEG): container finished" podID="0b94b517-5f9b-485d-a109-0a11e15a3ad3" containerID="d29340e2b19190d615ecfc711e5c8befec183416c40af1d92bde9b4447a33954" exitCode=1 Dec 03 11:47:27 crc kubenswrapper[5052]: I1203 11:47:27.814430 5052 kubelet_pods.go:1007] "Unable to retrieve pull secret, the image pull may not succeed." pod="openstack-kuttl-tests/novaapi5919-account-delete-qkvzd" secret="" err="secret \"galera-openstack-dockercfg-cpskp\" not found" Dec 03 11:47:27 crc kubenswrapper[5052]: I1203 11:47:27.814478 5052 scope.go:117] "RemoveContainer" containerID="c9d59899fe21862fbe05cb4f9b40af488d9a06ef3ce0bca59d2b5c6edc68c2cd" Dec 03 11:47:27 crc kubenswrapper[5052]: I1203 11:47:27.814691 5052 generic.go:334] "Generic (PLEG): container finished" podID="2d3a3054-6c8e-4808-bd5d-01413970167c" containerID="c9d59899fe21862fbe05cb4f9b40af488d9a06ef3ce0bca59d2b5c6edc68c2cd" exitCode=1 Dec 03 11:47:27 crc kubenswrapper[5052]: I1203 11:47:27.820673 5052 generic.go:334] "Generic (PLEG): container finished" podID="1b45577a-92b7-4c67-941b-cfc2fbfa3eb2" containerID="9e68066ce0ef8e9380353738e119a2afc7b38dcd3bd33c671f3778712cfb46e4" exitCode=0 Dec 03 11:47:27 crc kubenswrapper[5052]: I1203 11:47:27.820825 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:47:27 crc kubenswrapper[5052]: I1203 11:47:27.843899 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-689c6465d6-nsjrx" Dec 03 11:47:27 crc kubenswrapper[5052]: I1203 11:47:27.857323 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/1b45577a-92b7-4c67-941b-cfc2fbfa3eb2-run-httpd\") pod \"1b45577a-92b7-4c67-941b-cfc2fbfa3eb2\" (UID: \"1b45577a-92b7-4c67-941b-cfc2fbfa3eb2\") " Dec 03 11:47:27 crc kubenswrapper[5052]: I1203 11:47:27.857397 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sgqmd\" (UniqueName: \"kubernetes.io/projected/1b45577a-92b7-4c67-941b-cfc2fbfa3eb2-kube-api-access-sgqmd\") pod \"1b45577a-92b7-4c67-941b-cfc2fbfa3eb2\" (UID: \"1b45577a-92b7-4c67-941b-cfc2fbfa3eb2\") " Dec 03 11:47:27 crc kubenswrapper[5052]: I1203 11:47:27.857428 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/0914ebfe-13c6-4309-bd10-a67850124e3f-scripts\") pod \"0914ebfe-13c6-4309-bd10-a67850124e3f\" (UID: \"0914ebfe-13c6-4309-bd10-a67850124e3f\") " Dec 03 11:47:27 crc kubenswrapper[5052]: I1203 11:47:27.857467 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/1b45577a-92b7-4c67-941b-cfc2fbfa3eb2-config-data\") pod \"1b45577a-92b7-4c67-941b-cfc2fbfa3eb2\" (UID: \"1b45577a-92b7-4c67-941b-cfc2fbfa3eb2\") " Dec 03 11:47:27 crc kubenswrapper[5052]: I1203 11:47:27.857487 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/0914ebfe-13c6-4309-bd10-a67850124e3f-logs\") pod \"0914ebfe-13c6-4309-bd10-a67850124e3f\" (UID: \"0914ebfe-13c6-4309-bd10-a67850124e3f\") " Dec 03 11:47:27 crc kubenswrapper[5052]: I1203 11:47:27.857506 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p8kzb\" (UniqueName: \"kubernetes.io/projected/0914ebfe-13c6-4309-bd10-a67850124e3f-kube-api-access-p8kzb\") pod \"0914ebfe-13c6-4309-bd10-a67850124e3f\" (UID: \"0914ebfe-13c6-4309-bd10-a67850124e3f\") " Dec 03 11:47:27 crc kubenswrapper[5052]: I1203 11:47:27.857564 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"glance\" (UniqueName: \"kubernetes.io/local-volume/local-storage04-crc\") pod \"0914ebfe-13c6-4309-bd10-a67850124e3f\" (UID: \"0914ebfe-13c6-4309-bd10-a67850124e3f\") " Dec 03 11:47:27 crc kubenswrapper[5052]: I1203 11:47:27.857594 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1b45577a-92b7-4c67-941b-cfc2fbfa3eb2-combined-ca-bundle\") pod \"1b45577a-92b7-4c67-941b-cfc2fbfa3eb2\" (UID: \"1b45577a-92b7-4c67-941b-cfc2fbfa3eb2\") " Dec 03 11:47:27 crc kubenswrapper[5052]: I1203 11:47:27.857619 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/1b45577a-92b7-4c67-941b-cfc2fbfa3eb2-log-httpd\") pod \"1b45577a-92b7-4c67-941b-cfc2fbfa3eb2\" (UID: \"1b45577a-92b7-4c67-941b-cfc2fbfa3eb2\") " Dec 03 11:47:27 crc kubenswrapper[5052]: I1203 11:47:27.857650 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/1b45577a-92b7-4c67-941b-cfc2fbfa3eb2-ceilometer-tls-certs\") pod \"1b45577a-92b7-4c67-941b-cfc2fbfa3eb2\" (UID: \"1b45577a-92b7-4c67-941b-cfc2fbfa3eb2\") " Dec 03 11:47:27 crc kubenswrapper[5052]: I1203 11:47:27.857664 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/1b45577a-92b7-4c67-941b-cfc2fbfa3eb2-scripts\") pod \"1b45577a-92b7-4c67-941b-cfc2fbfa3eb2\" (UID: \"1b45577a-92b7-4c67-941b-cfc2fbfa3eb2\") " Dec 03 11:47:27 crc kubenswrapper[5052]: I1203 11:47:27.857681 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/cadac9a6-1d82-403a-b7a0-692a424dd0e2-config-data\") pod \"cadac9a6-1d82-403a-b7a0-692a424dd0e2\" (UID: \"cadac9a6-1d82-403a-b7a0-692a424dd0e2\") " Dec 03 11:47:27 crc kubenswrapper[5052]: I1203 11:47:27.857702 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jwbmx\" (UniqueName: \"kubernetes.io/projected/cadac9a6-1d82-403a-b7a0-692a424dd0e2-kube-api-access-jwbmx\") pod \"cadac9a6-1d82-403a-b7a0-692a424dd0e2\" (UID: \"cadac9a6-1d82-403a-b7a0-692a424dd0e2\") " Dec 03 11:47:27 crc kubenswrapper[5052]: I1203 11:47:27.857724 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/0914ebfe-13c6-4309-bd10-a67850124e3f-internal-tls-certs\") pod \"0914ebfe-13c6-4309-bd10-a67850124e3f\" (UID: \"0914ebfe-13c6-4309-bd10-a67850124e3f\") " Dec 03 11:47:27 crc kubenswrapper[5052]: I1203 11:47:27.857763 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/0914ebfe-13c6-4309-bd10-a67850124e3f-config-data\") pod \"0914ebfe-13c6-4309-bd10-a67850124e3f\" (UID: \"0914ebfe-13c6-4309-bd10-a67850124e3f\") " Dec 03 11:47:27 crc kubenswrapper[5052]: I1203 11:47:27.857783 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/0914ebfe-13c6-4309-bd10-a67850124e3f-httpd-run\") pod \"0914ebfe-13c6-4309-bd10-a67850124e3f\" (UID: \"0914ebfe-13c6-4309-bd10-a67850124e3f\") " Dec 03 11:47:27 crc kubenswrapper[5052]: I1203 11:47:27.857801 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/cadac9a6-1d82-403a-b7a0-692a424dd0e2-combined-ca-bundle\") pod \"cadac9a6-1d82-403a-b7a0-692a424dd0e2\" (UID: \"cadac9a6-1d82-403a-b7a0-692a424dd0e2\") " Dec 03 11:47:27 crc kubenswrapper[5052]: I1203 11:47:27.857820 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/cadac9a6-1d82-403a-b7a0-692a424dd0e2-logs\") pod \"cadac9a6-1d82-403a-b7a0-692a424dd0e2\" (UID: \"cadac9a6-1d82-403a-b7a0-692a424dd0e2\") " Dec 03 11:47:27 crc kubenswrapper[5052]: I1203 11:47:27.857870 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0914ebfe-13c6-4309-bd10-a67850124e3f-combined-ca-bundle\") pod \"0914ebfe-13c6-4309-bd10-a67850124e3f\" (UID: \"0914ebfe-13c6-4309-bd10-a67850124e3f\") " Dec 03 11:47:27 crc kubenswrapper[5052]: I1203 11:47:27.857918 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/cadac9a6-1d82-403a-b7a0-692a424dd0e2-config-data-custom\") pod \"cadac9a6-1d82-403a-b7a0-692a424dd0e2\" (UID: \"cadac9a6-1d82-403a-b7a0-692a424dd0e2\") " Dec 03 11:47:27 crc kubenswrapper[5052]: I1203 11:47:27.857952 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/1b45577a-92b7-4c67-941b-cfc2fbfa3eb2-sg-core-conf-yaml\") pod \"1b45577a-92b7-4c67-941b-cfc2fbfa3eb2\" (UID: \"1b45577a-92b7-4c67-941b-cfc2fbfa3eb2\") " Dec 03 11:47:27 crc kubenswrapper[5052]: I1203 11:47:27.859680 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1b45577a-92b7-4c67-941b-cfc2fbfa3eb2-run-httpd" (OuterVolumeSpecName: "run-httpd") pod "1b45577a-92b7-4c67-941b-cfc2fbfa3eb2" (UID: "1b45577a-92b7-4c67-941b-cfc2fbfa3eb2"). InnerVolumeSpecName "run-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:47:27 crc kubenswrapper[5052]: I1203 11:47:27.866714 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cadac9a6-1d82-403a-b7a0-692a424dd0e2-logs" (OuterVolumeSpecName: "logs") pod "cadac9a6-1d82-403a-b7a0-692a424dd0e2" (UID: "cadac9a6-1d82-403a-b7a0-692a424dd0e2"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:47:27 crc kubenswrapper[5052]: I1203 11:47:27.867318 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1b45577a-92b7-4c67-941b-cfc2fbfa3eb2-kube-api-access-sgqmd" (OuterVolumeSpecName: "kube-api-access-sgqmd") pod "1b45577a-92b7-4c67-941b-cfc2fbfa3eb2" (UID: "1b45577a-92b7-4c67-941b-cfc2fbfa3eb2"). InnerVolumeSpecName "kube-api-access-sgqmd". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:47:27 crc kubenswrapper[5052]: I1203 11:47:27.871944 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cadac9a6-1d82-403a-b7a0-692a424dd0e2-config-data-custom" (OuterVolumeSpecName: "config-data-custom") pod "cadac9a6-1d82-403a-b7a0-692a424dd0e2" (UID: "cadac9a6-1d82-403a-b7a0-692a424dd0e2"). InnerVolumeSpecName "config-data-custom". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:47:27 crc kubenswrapper[5052]: I1203 11:47:27.872495 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1b45577a-92b7-4c67-941b-cfc2fbfa3eb2-log-httpd" (OuterVolumeSpecName: "log-httpd") pod "1b45577a-92b7-4c67-941b-cfc2fbfa3eb2" (UID: "1b45577a-92b7-4c67-941b-cfc2fbfa3eb2"). InnerVolumeSpecName "log-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:47:27 crc kubenswrapper[5052]: I1203 11:47:27.872708 5052 generic.go:334] "Generic (PLEG): container finished" podID="aa9a649b-9229-4cdd-8e27-8aedd48f4bbd" containerID="92cc714d81001cf63bf9236d9304b62df8d8ba4cf9d19b8376165ae7a9509d8a" exitCode=1 Dec 03 11:47:27 crc kubenswrapper[5052]: I1203 11:47:27.885414 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0914ebfe-13c6-4309-bd10-a67850124e3f-scripts" (OuterVolumeSpecName: "scripts") pod "0914ebfe-13c6-4309-bd10-a67850124e3f" (UID: "0914ebfe-13c6-4309-bd10-a67850124e3f"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:47:27 crc kubenswrapper[5052]: I1203 11:47:27.886179 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0914ebfe-13c6-4309-bd10-a67850124e3f-logs" (OuterVolumeSpecName: "logs") pod "0914ebfe-13c6-4309-bd10-a67850124e3f" (UID: "0914ebfe-13c6-4309-bd10-a67850124e3f"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:47:27 crc kubenswrapper[5052]: I1203 11:47:27.887796 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0914ebfe-13c6-4309-bd10-a67850124e3f-httpd-run" (OuterVolumeSpecName: "httpd-run") pod "0914ebfe-13c6-4309-bd10-a67850124e3f" (UID: "0914ebfe-13c6-4309-bd10-a67850124e3f"). InnerVolumeSpecName "httpd-run". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:47:27 crc kubenswrapper[5052]: I1203 11:47:27.897309 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1b45577a-92b7-4c67-941b-cfc2fbfa3eb2-scripts" (OuterVolumeSpecName: "scripts") pod "1b45577a-92b7-4c67-941b-cfc2fbfa3eb2" (UID: "1b45577a-92b7-4c67-941b-cfc2fbfa3eb2"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:47:27 crc kubenswrapper[5052]: I1203 11:47:27.897301 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cadac9a6-1d82-403a-b7a0-692a424dd0e2-kube-api-access-jwbmx" (OuterVolumeSpecName: "kube-api-access-jwbmx") pod "cadac9a6-1d82-403a-b7a0-692a424dd0e2" (UID: "cadac9a6-1d82-403a-b7a0-692a424dd0e2"). InnerVolumeSpecName "kube-api-access-jwbmx". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:47:27 crc kubenswrapper[5052]: I1203 11:47:27.897393 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage04-crc" (OuterVolumeSpecName: "glance") pod "0914ebfe-13c6-4309-bd10-a67850124e3f" (UID: "0914ebfe-13c6-4309-bd10-a67850124e3f"). InnerVolumeSpecName "local-storage04-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 11:47:27 crc kubenswrapper[5052]: I1203 11:47:27.901914 5052 generic.go:334] "Generic (PLEG): container finished" podID="f7123213-06e1-4b26-9892-c16c3d49c0ab" containerID="25b77fa7d98d8777337c1b92110af89807efa838eb759959d62314939c420ccf" exitCode=0 Dec 03 11:47:27 crc kubenswrapper[5052]: I1203 11:47:27.905728 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0914ebfe-13c6-4309-bd10-a67850124e3f-kube-api-access-p8kzb" (OuterVolumeSpecName: "kube-api-access-p8kzb") pod "0914ebfe-13c6-4309-bd10-a67850124e3f" (UID: "0914ebfe-13c6-4309-bd10-a67850124e3f"). InnerVolumeSpecName "kube-api-access-p8kzb". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:47:27 crc kubenswrapper[5052]: I1203 11:47:27.906776 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"bb51da3b-72cc-4008-aac7-737f6a3031cb","Type":"ContainerDied","Data":"b39f7b31c335958d6613651cea9f3ccbfee7c25b7c7641048321f3e460ee0193"} Dec 03 11:47:27 crc kubenswrapper[5052]: I1203 11:47:27.906836 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance3554-account-delete-8ghdx" event={"ID":"950f0626-40d2-41c9-9701-cba50843c43e","Type":"ContainerDied","Data":"92c1a0e919d06f22f9f6723e6dddde204ff33caafc0c8594584022f38ab81e75"} Dec 03 11:47:27 crc kubenswrapper[5052]: I1203 11:47:27.906872 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"acea154c-f0bb-4066-bd4d-2dda815f301c","Type":"ContainerDied","Data":"a3d5fdb3d20e2604d292f20ec735e9c7ddb3d4e1018dcbc0aeb11a3eca4293ac"} Dec 03 11:47:27 crc kubenswrapper[5052]: I1203 11:47:27.906888 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-779b679dc7-nvtdm" event={"ID":"65be66f1-c32d-40b5-81a4-a63258364d9d","Type":"ContainerDied","Data":"2be919af42c62148fa0417519c4eea762d9886b79020bd92b5f10692d1f30f31"} Dec 03 11:47:27 crc kubenswrapper[5052]: I1203 11:47:27.906900 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-lfl6r" event={"ID":"93fb4fd0-aa18-4a2d-bfb4-b65bbc378ae5","Type":"ContainerDied","Data":"a278a573e396cd799f85833fa26f6bb09544244a8956130e15e5c8b299a9a039"} Dec 03 11:47:27 crc kubenswrapper[5052]: I1203 11:47:27.907113 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"0914ebfe-13c6-4309-bd10-a67850124e3f","Type":"ContainerDied","Data":"e1e1d1a1a9474d3fdc758a17a3894a41f6c6c842db0b3603ae153366d60c47fa"} Dec 03 11:47:27 crc kubenswrapper[5052]: I1203 11:47:27.907124 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican5976-account-delete-q6j5s" event={"ID":"0aefd6f6-cbb3-4c45-9f69-7124e43255e3","Type":"ContainerDied","Data":"12cedf118f41103f94919b8993790529cc71a18a79a829fcc52ee74432f113ee"} Dec 03 11:47:27 crc kubenswrapper[5052]: I1203 11:47:27.907138 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder1bf0-account-delete-hs8j6" event={"ID":"0b94b517-5f9b-485d-a109-0a11e15a3ad3","Type":"ContainerDied","Data":"d29340e2b19190d615ecfc711e5c8befec183416c40af1d92bde9b4447a33954"} Dec 03 11:47:27 crc kubenswrapper[5052]: I1203 11:47:27.907149 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/novaapi5919-account-delete-qkvzd" event={"ID":"2d3a3054-6c8e-4808-bd5d-01413970167c","Type":"ContainerDied","Data":"c9d59899fe21862fbe05cb4f9b40af488d9a06ef3ce0bca59d2b5c6edc68c2cd"} Dec 03 11:47:27 crc kubenswrapper[5052]: I1203 11:47:27.907164 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"1b45577a-92b7-4c67-941b-cfc2fbfa3eb2","Type":"ContainerDied","Data":"9e68066ce0ef8e9380353738e119a2afc7b38dcd3bd33c671f3778712cfb46e4"} Dec 03 11:47:27 crc kubenswrapper[5052]: I1203 11:47:27.907174 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"1b45577a-92b7-4c67-941b-cfc2fbfa3eb2","Type":"ContainerDied","Data":"4b13986efc84461e4d35d070b9899c7aaeea80554e4845c1f45b26df10d18678"} Dec 03 11:47:27 crc kubenswrapper[5052]: I1203 11:47:27.907185 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-689c6465d6-nsjrx" event={"ID":"0183ebb5-b91b-443a-9535-2008c9207ab1","Type":"ContainerDied","Data":"aa1335a1c15d6d9a335f1c0f519b93e0eb71b01df0b155d55fdd9d086a2a1a1a"} Dec 03 11:47:27 crc kubenswrapper[5052]: I1203 11:47:27.907196 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron18a7-account-delete-x9lqd" event={"ID":"aa9a649b-9229-4cdd-8e27-8aedd48f4bbd","Type":"ContainerDied","Data":"92cc714d81001cf63bf9236d9304b62df8d8ba4cf9d19b8376165ae7a9509d8a"} Dec 03 11:47:27 crc kubenswrapper[5052]: I1203 11:47:27.907209 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"f7123213-06e1-4b26-9892-c16c3d49c0ab","Type":"ContainerDied","Data":"25b77fa7d98d8777337c1b92110af89807efa838eb759959d62314939c420ccf"} Dec 03 11:47:27 crc kubenswrapper[5052]: I1203 11:47:27.907227 5052 scope.go:117] "RemoveContainer" containerID="6080d8e70e70e85fc4f9ffe234e02f7073f0c7d3e0ee2386f2621e8608af4e90" Dec 03 11:47:27 crc kubenswrapper[5052]: I1203 11:47:27.958975 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t7ghr\" (UniqueName: \"kubernetes.io/projected/acea154c-f0bb-4066-bd4d-2dda815f301c-kube-api-access-t7ghr\") pod \"acea154c-f0bb-4066-bd4d-2dda815f301c\" (UID: \"acea154c-f0bb-4066-bd4d-2dda815f301c\") " Dec 03 11:47:27 crc kubenswrapper[5052]: I1203 11:47:27.959015 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/00b12739-1437-4af3-ab6d-ad830b67ad2e-etc-machine-id\") pod \"00b12739-1437-4af3-ab6d-ad830b67ad2e\" (UID: \"00b12739-1437-4af3-ab6d-ad830b67ad2e\") " Dec 03 11:47:27 crc kubenswrapper[5052]: I1203 11:47:27.959043 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/acea154c-f0bb-4066-bd4d-2dda815f301c-logs\") pod \"acea154c-f0bb-4066-bd4d-2dda815f301c\" (UID: \"acea154c-f0bb-4066-bd4d-2dda815f301c\") " Dec 03 11:47:27 crc kubenswrapper[5052]: I1203 11:47:27.959079 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/00b12739-1437-4af3-ab6d-ad830b67ad2e-logs\") pod \"00b12739-1437-4af3-ab6d-ad830b67ad2e\" (UID: \"00b12739-1437-4af3-ab6d-ad830b67ad2e\") " Dec 03 11:47:27 crc kubenswrapper[5052]: I1203 11:47:27.959112 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0183ebb5-b91b-443a-9535-2008c9207ab1-combined-ca-bundle\") pod \"0183ebb5-b91b-443a-9535-2008c9207ab1\" (UID: \"0183ebb5-b91b-443a-9535-2008c9207ab1\") " Dec 03 11:47:27 crc kubenswrapper[5052]: I1203 11:47:27.959153 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/00b12739-1437-4af3-ab6d-ad830b67ad2e-internal-tls-certs\") pod \"00b12739-1437-4af3-ab6d-ad830b67ad2e\" (UID: \"00b12739-1437-4af3-ab6d-ad830b67ad2e\") " Dec 03 11:47:27 crc kubenswrapper[5052]: I1203 11:47:27.959209 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/00b12739-1437-4af3-ab6d-ad830b67ad2e-config-data\") pod \"00b12739-1437-4af3-ab6d-ad830b67ad2e\" (UID: \"00b12739-1437-4af3-ab6d-ad830b67ad2e\") " Dec 03 11:47:27 crc kubenswrapper[5052]: I1203 11:47:27.959232 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/acea154c-f0bb-4066-bd4d-2dda815f301c-scripts\") pod \"acea154c-f0bb-4066-bd4d-2dda815f301c\" (UID: \"acea154c-f0bb-4066-bd4d-2dda815f301c\") " Dec 03 11:47:27 crc kubenswrapper[5052]: I1203 11:47:27.959306 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/0183ebb5-b91b-443a-9535-2008c9207ab1-config-data\") pod \"0183ebb5-b91b-443a-9535-2008c9207ab1\" (UID: \"0183ebb5-b91b-443a-9535-2008c9207ab1\") " Dec 03 11:47:27 crc kubenswrapper[5052]: I1203 11:47:27.959336 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"glance\" (UniqueName: \"kubernetes.io/local-volume/local-storage15-crc\") pod \"acea154c-f0bb-4066-bd4d-2dda815f301c\" (UID: \"acea154c-f0bb-4066-bd4d-2dda815f301c\") " Dec 03 11:47:27 crc kubenswrapper[5052]: I1203 11:47:27.959351 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/00b12739-1437-4af3-ab6d-ad830b67ad2e-scripts\") pod \"00b12739-1437-4af3-ab6d-ad830b67ad2e\" (UID: \"00b12739-1437-4af3-ab6d-ad830b67ad2e\") " Dec 03 11:47:27 crc kubenswrapper[5052]: I1203 11:47:27.959368 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/0183ebb5-b91b-443a-9535-2008c9207ab1-scripts\") pod \"0183ebb5-b91b-443a-9535-2008c9207ab1\" (UID: \"0183ebb5-b91b-443a-9535-2008c9207ab1\") " Dec 03 11:47:27 crc kubenswrapper[5052]: I1203 11:47:27.959384 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/acea154c-f0bb-4066-bd4d-2dda815f301c-public-tls-certs\") pod \"acea154c-f0bb-4066-bd4d-2dda815f301c\" (UID: \"acea154c-f0bb-4066-bd4d-2dda815f301c\") " Dec 03 11:47:27 crc kubenswrapper[5052]: I1203 11:47:27.959402 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/0183ebb5-b91b-443a-9535-2008c9207ab1-logs\") pod \"0183ebb5-b91b-443a-9535-2008c9207ab1\" (UID: \"0183ebb5-b91b-443a-9535-2008c9207ab1\") " Dec 03 11:47:27 crc kubenswrapper[5052]: I1203 11:47:27.959423 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/0183ebb5-b91b-443a-9535-2008c9207ab1-internal-tls-certs\") pod \"0183ebb5-b91b-443a-9535-2008c9207ab1\" (UID: \"0183ebb5-b91b-443a-9535-2008c9207ab1\") " Dec 03 11:47:27 crc kubenswrapper[5052]: I1203 11:47:27.959437 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xb5gb\" (UniqueName: \"kubernetes.io/projected/0183ebb5-b91b-443a-9535-2008c9207ab1-kube-api-access-xb5gb\") pod \"0183ebb5-b91b-443a-9535-2008c9207ab1\" (UID: \"0183ebb5-b91b-443a-9535-2008c9207ab1\") " Dec 03 11:47:27 crc kubenswrapper[5052]: I1203 11:47:27.959452 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/acea154c-f0bb-4066-bd4d-2dda815f301c-combined-ca-bundle\") pod \"acea154c-f0bb-4066-bd4d-2dda815f301c\" (UID: \"acea154c-f0bb-4066-bd4d-2dda815f301c\") " Dec 03 11:47:27 crc kubenswrapper[5052]: I1203 11:47:27.959471 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/00b12739-1437-4af3-ab6d-ad830b67ad2e-config-data-custom\") pod \"00b12739-1437-4af3-ab6d-ad830b67ad2e\" (UID: \"00b12739-1437-4af3-ab6d-ad830b67ad2e\") " Dec 03 11:47:27 crc kubenswrapper[5052]: I1203 11:47:27.959505 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/acea154c-f0bb-4066-bd4d-2dda815f301c-httpd-run\") pod \"acea154c-f0bb-4066-bd4d-2dda815f301c\" (UID: \"acea154c-f0bb-4066-bd4d-2dda815f301c\") " Dec 03 11:47:27 crc kubenswrapper[5052]: I1203 11:47:27.959545 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/acea154c-f0bb-4066-bd4d-2dda815f301c-config-data\") pod \"acea154c-f0bb-4066-bd4d-2dda815f301c\" (UID: \"acea154c-f0bb-4066-bd4d-2dda815f301c\") " Dec 03 11:47:27 crc kubenswrapper[5052]: I1203 11:47:27.959571 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/00b12739-1437-4af3-ab6d-ad830b67ad2e-public-tls-certs\") pod \"00b12739-1437-4af3-ab6d-ad830b67ad2e\" (UID: \"00b12739-1437-4af3-ab6d-ad830b67ad2e\") " Dec 03 11:47:27 crc kubenswrapper[5052]: I1203 11:47:27.959593 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/0183ebb5-b91b-443a-9535-2008c9207ab1-public-tls-certs\") pod \"0183ebb5-b91b-443a-9535-2008c9207ab1\" (UID: \"0183ebb5-b91b-443a-9535-2008c9207ab1\") " Dec 03 11:47:27 crc kubenswrapper[5052]: I1203 11:47:27.959613 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/00b12739-1437-4af3-ab6d-ad830b67ad2e-combined-ca-bundle\") pod \"00b12739-1437-4af3-ab6d-ad830b67ad2e\" (UID: \"00b12739-1437-4af3-ab6d-ad830b67ad2e\") " Dec 03 11:47:27 crc kubenswrapper[5052]: I1203 11:47:27.959633 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8d24s\" (UniqueName: \"kubernetes.io/projected/00b12739-1437-4af3-ab6d-ad830b67ad2e-kube-api-access-8d24s\") pod \"00b12739-1437-4af3-ab6d-ad830b67ad2e\" (UID: \"00b12739-1437-4af3-ab6d-ad830b67ad2e\") " Dec 03 11:47:27 crc kubenswrapper[5052]: I1203 11:47:27.960051 5052 reconciler_common.go:293] "Volume detached for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/1b45577a-92b7-4c67-941b-cfc2fbfa3eb2-log-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:27 crc kubenswrapper[5052]: I1203 11:47:27.960068 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/1b45577a-92b7-4c67-941b-cfc2fbfa3eb2-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:27 crc kubenswrapper[5052]: I1203 11:47:27.960077 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-jwbmx\" (UniqueName: \"kubernetes.io/projected/cadac9a6-1d82-403a-b7a0-692a424dd0e2-kube-api-access-jwbmx\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:27 crc kubenswrapper[5052]: I1203 11:47:27.960087 5052 reconciler_common.go:293] "Volume detached for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/0914ebfe-13c6-4309-bd10-a67850124e3f-httpd-run\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:27 crc kubenswrapper[5052]: I1203 11:47:27.960097 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/cadac9a6-1d82-403a-b7a0-692a424dd0e2-logs\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:27 crc kubenswrapper[5052]: I1203 11:47:27.960105 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/cadac9a6-1d82-403a-b7a0-692a424dd0e2-config-data-custom\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:27 crc kubenswrapper[5052]: I1203 11:47:27.960114 5052 reconciler_common.go:293] "Volume detached for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/1b45577a-92b7-4c67-941b-cfc2fbfa3eb2-run-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:27 crc kubenswrapper[5052]: I1203 11:47:27.960122 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-sgqmd\" (UniqueName: \"kubernetes.io/projected/1b45577a-92b7-4c67-941b-cfc2fbfa3eb2-kube-api-access-sgqmd\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:27 crc kubenswrapper[5052]: I1203 11:47:27.960130 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/0914ebfe-13c6-4309-bd10-a67850124e3f-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:27 crc kubenswrapper[5052]: I1203 11:47:27.960138 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/0914ebfe-13c6-4309-bd10-a67850124e3f-logs\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:27 crc kubenswrapper[5052]: I1203 11:47:27.960146 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-p8kzb\" (UniqueName: \"kubernetes.io/projected/0914ebfe-13c6-4309-bd10-a67850124e3f-kube-api-access-p8kzb\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:27 crc kubenswrapper[5052]: I1203 11:47:27.960162 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage04-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage04-crc\") on node \"crc\" " Dec 03 11:47:27 crc kubenswrapper[5052]: I1203 11:47:27.976168 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/00b12739-1437-4af3-ab6d-ad830b67ad2e-etc-machine-id" (OuterVolumeSpecName: "etc-machine-id") pod "00b12739-1437-4af3-ab6d-ad830b67ad2e" (UID: "00b12739-1437-4af3-ab6d-ad830b67ad2e"). InnerVolumeSpecName "etc-machine-id". PluginName "kubernetes.io/host-path", VolumeGidValue "" Dec 03 11:47:27 crc kubenswrapper[5052]: I1203 11:47:27.977159 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/acea154c-f0bb-4066-bd4d-2dda815f301c-httpd-run" (OuterVolumeSpecName: "httpd-run") pod "acea154c-f0bb-4066-bd4d-2dda815f301c" (UID: "acea154c-f0bb-4066-bd4d-2dda815f301c"). InnerVolumeSpecName "httpd-run". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:47:27 crc kubenswrapper[5052]: I1203 11:47:27.977604 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0183ebb5-b91b-443a-9535-2008c9207ab1-logs" (OuterVolumeSpecName: "logs") pod "0183ebb5-b91b-443a-9535-2008c9207ab1" (UID: "0183ebb5-b91b-443a-9535-2008c9207ab1"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:47:27 crc kubenswrapper[5052]: I1203 11:47:27.988080 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/acea154c-f0bb-4066-bd4d-2dda815f301c-logs" (OuterVolumeSpecName: "logs") pod "acea154c-f0bb-4066-bd4d-2dda815f301c" (UID: "acea154c-f0bb-4066-bd4d-2dda815f301c"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:47:27 crc kubenswrapper[5052]: E1203 11:47:27.997481 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 11:47:27 crc kubenswrapper[5052]: E1203 11:47:27.997570 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2d3a3054-6c8e-4808-bd5d-01413970167c-operator-scripts podName:2d3a3054-6c8e-4808-bd5d-01413970167c nodeName:}" failed. No retries permitted until 2025-12-03 11:47:28.497552633 +0000 UTC m=+1639.085794397 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/2d3a3054-6c8e-4808-bd5d-01413970167c-operator-scripts") pod "novaapi5919-account-delete-qkvzd" (UID: "2d3a3054-6c8e-4808-bd5d-01413970167c") : configmap "openstack-scripts" not found Dec 03 11:47:28 crc kubenswrapper[5052]: I1203 11:47:28.032878 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0183ebb5-b91b-443a-9535-2008c9207ab1-scripts" (OuterVolumeSpecName: "scripts") pod "0183ebb5-b91b-443a-9535-2008c9207ab1" (UID: "0183ebb5-b91b-443a-9535-2008c9207ab1"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:47:28 crc kubenswrapper[5052]: I1203 11:47:28.036593 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/00b12739-1437-4af3-ab6d-ad830b67ad2e-logs" (OuterVolumeSpecName: "logs") pod "00b12739-1437-4af3-ab6d-ad830b67ad2e" (UID: "00b12739-1437-4af3-ab6d-ad830b67ad2e"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:47:28 crc kubenswrapper[5052]: I1203 11:47:28.042323 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/acea154c-f0bb-4066-bd4d-2dda815f301c-kube-api-access-t7ghr" (OuterVolumeSpecName: "kube-api-access-t7ghr") pod "acea154c-f0bb-4066-bd4d-2dda815f301c" (UID: "acea154c-f0bb-4066-bd4d-2dda815f301c"). InnerVolumeSpecName "kube-api-access-t7ghr". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:47:28 crc kubenswrapper[5052]: I1203 11:47:28.050890 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage15-crc" (OuterVolumeSpecName: "glance") pod "acea154c-f0bb-4066-bd4d-2dda815f301c" (UID: "acea154c-f0bb-4066-bd4d-2dda815f301c"). InnerVolumeSpecName "local-storage15-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 11:47:28 crc kubenswrapper[5052]: I1203 11:47:28.058726 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/00b12739-1437-4af3-ab6d-ad830b67ad2e-config-data-custom" (OuterVolumeSpecName: "config-data-custom") pod "00b12739-1437-4af3-ab6d-ad830b67ad2e" (UID: "00b12739-1437-4af3-ab6d-ad830b67ad2e"). InnerVolumeSpecName "config-data-custom". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:47:28 crc kubenswrapper[5052]: I1203 11:47:28.059137 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-779b679dc7-nvtdm" Dec 03 11:47:28 crc kubenswrapper[5052]: I1203 11:47:28.059296 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0183ebb5-b91b-443a-9535-2008c9207ab1-kube-api-access-xb5gb" (OuterVolumeSpecName: "kube-api-access-xb5gb") pod "0183ebb5-b91b-443a-9535-2008c9207ab1" (UID: "0183ebb5-b91b-443a-9535-2008c9207ab1"). InnerVolumeSpecName "kube-api-access-xb5gb". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:47:28 crc kubenswrapper[5052]: I1203 11:47:28.061832 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage15-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage15-crc\") on node \"crc\" " Dec 03 11:47:28 crc kubenswrapper[5052]: I1203 11:47:28.061858 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/0183ebb5-b91b-443a-9535-2008c9207ab1-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:28 crc kubenswrapper[5052]: I1203 11:47:28.061868 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/0183ebb5-b91b-443a-9535-2008c9207ab1-logs\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:28 crc kubenswrapper[5052]: I1203 11:47:28.061897 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-xb5gb\" (UniqueName: \"kubernetes.io/projected/0183ebb5-b91b-443a-9535-2008c9207ab1-kube-api-access-xb5gb\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:28 crc kubenswrapper[5052]: I1203 11:47:28.061906 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/00b12739-1437-4af3-ab6d-ad830b67ad2e-config-data-custom\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:28 crc kubenswrapper[5052]: I1203 11:47:28.061915 5052 reconciler_common.go:293] "Volume detached for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/acea154c-f0bb-4066-bd4d-2dda815f301c-httpd-run\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:28 crc kubenswrapper[5052]: I1203 11:47:28.061925 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-t7ghr\" (UniqueName: \"kubernetes.io/projected/acea154c-f0bb-4066-bd4d-2dda815f301c-kube-api-access-t7ghr\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:28 crc kubenswrapper[5052]: I1203 11:47:28.061934 5052 reconciler_common.go:293] "Volume detached for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/00b12739-1437-4af3-ab6d-ad830b67ad2e-etc-machine-id\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:28 crc kubenswrapper[5052]: I1203 11:47:28.061942 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/acea154c-f0bb-4066-bd4d-2dda815f301c-logs\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:28 crc kubenswrapper[5052]: I1203 11:47:28.061950 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/00b12739-1437-4af3-ab6d-ad830b67ad2e-logs\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:28 crc kubenswrapper[5052]: I1203 11:47:28.119194 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0914ebfe-13c6-4309-bd10-a67850124e3f-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "0914ebfe-13c6-4309-bd10-a67850124e3f" (UID: "0914ebfe-13c6-4309-bd10-a67850124e3f"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:47:28 crc kubenswrapper[5052]: I1203 11:47:28.119198 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cadac9a6-1d82-403a-b7a0-692a424dd0e2-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "cadac9a6-1d82-403a-b7a0-692a424dd0e2" (UID: "cadac9a6-1d82-403a-b7a0-692a424dd0e2"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:47:28 crc kubenswrapper[5052]: I1203 11:47:28.119330 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/00b12739-1437-4af3-ab6d-ad830b67ad2e-kube-api-access-8d24s" (OuterVolumeSpecName: "kube-api-access-8d24s") pod "00b12739-1437-4af3-ab6d-ad830b67ad2e" (UID: "00b12739-1437-4af3-ab6d-ad830b67ad2e"). InnerVolumeSpecName "kube-api-access-8d24s". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:47:28 crc kubenswrapper[5052]: I1203 11:47:28.121732 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/acea154c-f0bb-4066-bd4d-2dda815f301c-scripts" (OuterVolumeSpecName: "scripts") pod "acea154c-f0bb-4066-bd4d-2dda815f301c" (UID: "acea154c-f0bb-4066-bd4d-2dda815f301c"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:47:28 crc kubenswrapper[5052]: I1203 11:47:28.153782 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/00b12739-1437-4af3-ab6d-ad830b67ad2e-scripts" (OuterVolumeSpecName: "scripts") pod "00b12739-1437-4af3-ab6d-ad830b67ad2e" (UID: "00b12739-1437-4af3-ab6d-ad830b67ad2e"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:47:28 crc kubenswrapper[5052]: E1203 11:47:28.155135 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="1dbdda8905493662bc20bc18fcae5f54dd4929f65dec1b16df59f03dbfd76d8a" cmd=["/usr/bin/pgrep","-r","DRST","nova-scheduler"] Dec 03 11:47:28 crc kubenswrapper[5052]: I1203 11:47:28.164618 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/65be66f1-c32d-40b5-81a4-a63258364d9d-config-data\") pod \"65be66f1-c32d-40b5-81a4-a63258364d9d\" (UID: \"65be66f1-c32d-40b5-81a4-a63258364d9d\") " Dec 03 11:47:28 crc kubenswrapper[5052]: I1203 11:47:28.189764 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7h9bm\" (UniqueName: \"kubernetes.io/projected/65be66f1-c32d-40b5-81a4-a63258364d9d-kube-api-access-7h9bm\") pod \"65be66f1-c32d-40b5-81a4-a63258364d9d\" (UID: \"65be66f1-c32d-40b5-81a4-a63258364d9d\") " Dec 03 11:47:28 crc kubenswrapper[5052]: I1203 11:47:28.189858 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/65be66f1-c32d-40b5-81a4-a63258364d9d-credential-keys\") pod \"65be66f1-c32d-40b5-81a4-a63258364d9d\" (UID: \"65be66f1-c32d-40b5-81a4-a63258364d9d\") " Dec 03 11:47:28 crc kubenswrapper[5052]: I1203 11:47:28.189925 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/00b12739-1437-4af3-ab6d-ad830b67ad2e-scripts\") pod \"00b12739-1437-4af3-ab6d-ad830b67ad2e\" (UID: \"00b12739-1437-4af3-ab6d-ad830b67ad2e\") " Dec 03 11:47:28 crc kubenswrapper[5052]: I1203 11:47:28.189963 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/65be66f1-c32d-40b5-81a4-a63258364d9d-public-tls-certs\") pod \"65be66f1-c32d-40b5-81a4-a63258364d9d\" (UID: \"65be66f1-c32d-40b5-81a4-a63258364d9d\") " Dec 03 11:47:28 crc kubenswrapper[5052]: I1203 11:47:28.190013 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/65be66f1-c32d-40b5-81a4-a63258364d9d-internal-tls-certs\") pod \"65be66f1-c32d-40b5-81a4-a63258364d9d\" (UID: \"65be66f1-c32d-40b5-81a4-a63258364d9d\") " Dec 03 11:47:28 crc kubenswrapper[5052]: I1203 11:47:28.190047 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/65be66f1-c32d-40b5-81a4-a63258364d9d-fernet-keys\") pod \"65be66f1-c32d-40b5-81a4-a63258364d9d\" (UID: \"65be66f1-c32d-40b5-81a4-a63258364d9d\") " Dec 03 11:47:28 crc kubenswrapper[5052]: I1203 11:47:28.190091 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/65be66f1-c32d-40b5-81a4-a63258364d9d-scripts\") pod \"65be66f1-c32d-40b5-81a4-a63258364d9d\" (UID: \"65be66f1-c32d-40b5-81a4-a63258364d9d\") " Dec 03 11:47:28 crc kubenswrapper[5052]: I1203 11:47:28.190192 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/65be66f1-c32d-40b5-81a4-a63258364d9d-combined-ca-bundle\") pod \"65be66f1-c32d-40b5-81a4-a63258364d9d\" (UID: \"65be66f1-c32d-40b5-81a4-a63258364d9d\") " Dec 03 11:47:28 crc kubenswrapper[5052]: W1203 11:47:28.190666 5052 empty_dir.go:500] Warning: Unmount skipped because path does not exist: /var/lib/kubelet/pods/00b12739-1437-4af3-ab6d-ad830b67ad2e/volumes/kubernetes.io~secret/scripts Dec 03 11:47:28 crc kubenswrapper[5052]: I1203 11:47:28.190688 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/00b12739-1437-4af3-ab6d-ad830b67ad2e-scripts" (OuterVolumeSpecName: "scripts") pod "00b12739-1437-4af3-ab6d-ad830b67ad2e" (UID: "00b12739-1437-4af3-ab6d-ad830b67ad2e"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:47:28 crc kubenswrapper[5052]: I1203 11:47:28.165590 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:47:28 crc kubenswrapper[5052]: I1203 11:47:28.191036 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/acea154c-f0bb-4066-bd4d-2dda815f301c-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:28 crc kubenswrapper[5052]: I1203 11:47:28.191055 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/cadac9a6-1d82-403a-b7a0-692a424dd0e2-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:28 crc kubenswrapper[5052]: I1203 11:47:28.191070 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0914ebfe-13c6-4309-bd10-a67850124e3f-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:28 crc kubenswrapper[5052]: I1203 11:47:28.191081 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/00b12739-1437-4af3-ab6d-ad830b67ad2e-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:28 crc kubenswrapper[5052]: I1203 11:47:28.191092 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-8d24s\" (UniqueName: \"kubernetes.io/projected/00b12739-1437-4af3-ab6d-ad830b67ad2e-kube-api-access-8d24s\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:28 crc kubenswrapper[5052]: E1203 11:47:28.191153 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 11:47:28 crc kubenswrapper[5052]: E1203 11:47:28.191204 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/aa9a649b-9229-4cdd-8e27-8aedd48f4bbd-operator-scripts podName:aa9a649b-9229-4cdd-8e27-8aedd48f4bbd nodeName:}" failed. No retries permitted until 2025-12-03 11:47:32.191186597 +0000 UTC m=+1642.779428411 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/aa9a649b-9229-4cdd-8e27-8aedd48f4bbd-operator-scripts") pod "neutron18a7-account-delete-x9lqd" (UID: "aa9a649b-9229-4cdd-8e27-8aedd48f4bbd") : configmap "openstack-scripts" not found Dec 03 11:47:28 crc kubenswrapper[5052]: I1203 11:47:28.165686 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-api-7cfb847cb8-jn85t" Dec 03 11:47:28 crc kubenswrapper[5052]: E1203 11:47:28.195540 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 11:47:28 crc kubenswrapper[5052]: E1203 11:47:28.195616 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e5339997-33d5-4794-8fa6-042b63e9391e-operator-scripts podName:e5339997-33d5-4794-8fa6-042b63e9391e nodeName:}" failed. No retries permitted until 2025-12-03 11:47:29.195593293 +0000 UTC m=+1639.783835057 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/e5339997-33d5-4794-8fa6-042b63e9391e-operator-scripts") pod "novacell0f13b-account-delete-g7m79" (UID: "e5339997-33d5-4794-8fa6-042b63e9391e") : configmap "openstack-scripts" not found Dec 03 11:47:28 crc kubenswrapper[5052]: E1203 11:47:28.195531 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="1dbdda8905493662bc20bc18fcae5f54dd4929f65dec1b16df59f03dbfd76d8a" cmd=["/usr/bin/pgrep","-r","DRST","nova-scheduler"] Dec 03 11:47:28 crc kubenswrapper[5052]: E1203 11:47:28.218704 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="1dbdda8905493662bc20bc18fcae5f54dd4929f65dec1b16df59f03dbfd76d8a" cmd=["/usr/bin/pgrep","-r","DRST","nova-scheduler"] Dec 03 11:47:28 crc kubenswrapper[5052]: E1203 11:47:28.218782 5052 prober.go:104] "Probe errored" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" probeType="Readiness" pod="openstack-kuttl-tests/nova-scheduler-0" podUID="39b9198e-e3d7-4f85-b6a2-55960bdda82a" containerName="nova-scheduler-scheduler" Dec 03 11:47:28 crc kubenswrapper[5052]: I1203 11:47:28.218780 5052 scope.go:117] "RemoveContainer" containerID="e054bf606eb8ede8b913bb4874e3d9c8bf93f40bc4a7246b741ee83f941471a3" Dec 03 11:47:28 crc kubenswrapper[5052]: I1203 11:47:28.255115 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/65be66f1-c32d-40b5-81a4-a63258364d9d-fernet-keys" (OuterVolumeSpecName: "fernet-keys") pod "65be66f1-c32d-40b5-81a4-a63258364d9d" (UID: "65be66f1-c32d-40b5-81a4-a63258364d9d"). InnerVolumeSpecName "fernet-keys". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:47:28 crc kubenswrapper[5052]: I1203 11:47:28.257510 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/65be66f1-c32d-40b5-81a4-a63258364d9d-credential-keys" (OuterVolumeSpecName: "credential-keys") pod "65be66f1-c32d-40b5-81a4-a63258364d9d" (UID: "65be66f1-c32d-40b5-81a4-a63258364d9d"). InnerVolumeSpecName "credential-keys". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:47:28 crc kubenswrapper[5052]: I1203 11:47:28.258637 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/65be66f1-c32d-40b5-81a4-a63258364d9d-scripts" (OuterVolumeSpecName: "scripts") pod "65be66f1-c32d-40b5-81a4-a63258364d9d" (UID: "65be66f1-c32d-40b5-81a4-a63258364d9d"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:47:28 crc kubenswrapper[5052]: I1203 11:47:28.260106 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/65be66f1-c32d-40b5-81a4-a63258364d9d-kube-api-access-7h9bm" (OuterVolumeSpecName: "kube-api-access-7h9bm") pod "65be66f1-c32d-40b5-81a4-a63258364d9d" (UID: "65be66f1-c32d-40b5-81a4-a63258364d9d"). InnerVolumeSpecName "kube-api-access-7h9bm". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:47:28 crc kubenswrapper[5052]: I1203 11:47:28.288551 5052 scope.go:117] "RemoveContainer" containerID="51504add0a6672a10359e1a4fc407e6171b498dd50bf1382d23a8247c04ea2fe" Dec 03 11:47:28 crc kubenswrapper[5052]: I1203 11:47:28.290950 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1b45577a-92b7-4c67-941b-cfc2fbfa3eb2-sg-core-conf-yaml" (OuterVolumeSpecName: "sg-core-conf-yaml") pod "1b45577a-92b7-4c67-941b-cfc2fbfa3eb2" (UID: "1b45577a-92b7-4c67-941b-cfc2fbfa3eb2"). InnerVolumeSpecName "sg-core-conf-yaml". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:47:28 crc kubenswrapper[5052]: I1203 11:47:28.294181 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gbvkd\" (UniqueName: \"kubernetes.io/projected/99f96963-d835-4445-84af-0976f1ba0701-kube-api-access-gbvkd\") pod \"99f96963-d835-4445-84af-0976f1ba0701\" (UID: \"99f96963-d835-4445-84af-0976f1ba0701\") " Dec 03 11:47:28 crc kubenswrapper[5052]: I1203 11:47:28.294226 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dbh4l\" (UniqueName: \"kubernetes.io/projected/bb51da3b-72cc-4008-aac7-737f6a3031cb-kube-api-access-dbh4l\") pod \"bb51da3b-72cc-4008-aac7-737f6a3031cb\" (UID: \"bb51da3b-72cc-4008-aac7-737f6a3031cb\") " Dec 03 11:47:28 crc kubenswrapper[5052]: I1203 11:47:28.294258 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/99f96963-d835-4445-84af-0976f1ba0701-logs\") pod \"99f96963-d835-4445-84af-0976f1ba0701\" (UID: \"99f96963-d835-4445-84af-0976f1ba0701\") " Dec 03 11:47:28 crc kubenswrapper[5052]: I1203 11:47:28.294328 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/bb51da3b-72cc-4008-aac7-737f6a3031cb-config-data\") pod \"bb51da3b-72cc-4008-aac7-737f6a3031cb\" (UID: \"bb51da3b-72cc-4008-aac7-737f6a3031cb\") " Dec 03 11:47:28 crc kubenswrapper[5052]: I1203 11:47:28.294346 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/99f96963-d835-4445-84af-0976f1ba0701-combined-ca-bundle\") pod \"99f96963-d835-4445-84af-0976f1ba0701\" (UID: \"99f96963-d835-4445-84af-0976f1ba0701\") " Dec 03 11:47:28 crc kubenswrapper[5052]: I1203 11:47:28.294374 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/bb51da3b-72cc-4008-aac7-737f6a3031cb-nova-metadata-tls-certs\") pod \"bb51da3b-72cc-4008-aac7-737f6a3031cb\" (UID: \"bb51da3b-72cc-4008-aac7-737f6a3031cb\") " Dec 03 11:47:28 crc kubenswrapper[5052]: I1203 11:47:28.294400 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/99f96963-d835-4445-84af-0976f1ba0701-config-data-custom\") pod \"99f96963-d835-4445-84af-0976f1ba0701\" (UID: \"99f96963-d835-4445-84af-0976f1ba0701\") " Dec 03 11:47:28 crc kubenswrapper[5052]: I1203 11:47:28.294436 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/99f96963-d835-4445-84af-0976f1ba0701-public-tls-certs\") pod \"99f96963-d835-4445-84af-0976f1ba0701\" (UID: \"99f96963-d835-4445-84af-0976f1ba0701\") " Dec 03 11:47:28 crc kubenswrapper[5052]: I1203 11:47:28.294528 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/99f96963-d835-4445-84af-0976f1ba0701-internal-tls-certs\") pod \"99f96963-d835-4445-84af-0976f1ba0701\" (UID: \"99f96963-d835-4445-84af-0976f1ba0701\") " Dec 03 11:47:28 crc kubenswrapper[5052]: I1203 11:47:28.294554 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/bb51da3b-72cc-4008-aac7-737f6a3031cb-combined-ca-bundle\") pod \"bb51da3b-72cc-4008-aac7-737f6a3031cb\" (UID: \"bb51da3b-72cc-4008-aac7-737f6a3031cb\") " Dec 03 11:47:28 crc kubenswrapper[5052]: I1203 11:47:28.294603 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/bb51da3b-72cc-4008-aac7-737f6a3031cb-logs\") pod \"bb51da3b-72cc-4008-aac7-737f6a3031cb\" (UID: \"bb51da3b-72cc-4008-aac7-737f6a3031cb\") " Dec 03 11:47:28 crc kubenswrapper[5052]: I1203 11:47:28.294639 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/99f96963-d835-4445-84af-0976f1ba0701-config-data\") pod \"99f96963-d835-4445-84af-0976f1ba0701\" (UID: \"99f96963-d835-4445-84af-0976f1ba0701\") " Dec 03 11:47:28 crc kubenswrapper[5052]: I1203 11:47:28.295120 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-7h9bm\" (UniqueName: \"kubernetes.io/projected/65be66f1-c32d-40b5-81a4-a63258364d9d-kube-api-access-7h9bm\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:28 crc kubenswrapper[5052]: I1203 11:47:28.295135 5052 reconciler_common.go:293] "Volume detached for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/65be66f1-c32d-40b5-81a4-a63258364d9d-credential-keys\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:28 crc kubenswrapper[5052]: I1203 11:47:28.295145 5052 reconciler_common.go:293] "Volume detached for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/1b45577a-92b7-4c67-941b-cfc2fbfa3eb2-sg-core-conf-yaml\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:28 crc kubenswrapper[5052]: I1203 11:47:28.295155 5052 reconciler_common.go:293] "Volume detached for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/65be66f1-c32d-40b5-81a4-a63258364d9d-fernet-keys\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:28 crc kubenswrapper[5052]: I1203 11:47:28.295163 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/65be66f1-c32d-40b5-81a4-a63258364d9d-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:28 crc kubenswrapper[5052]: E1203 11:47:28.295241 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/combined-ca-bundle: secret "combined-ca-bundle" not found Dec 03 11:47:28 crc kubenswrapper[5052]: E1203 11:47:28.295385 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f9b208b-086f-4671-8c3f-b3a275740ad7-combined-ca-bundle podName:6f9b208b-086f-4671-8c3f-b3a275740ad7 nodeName:}" failed. No retries permitted until 2025-12-03 11:47:30.295367965 +0000 UTC m=+1640.883609729 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "combined-ca-bundle" (UniqueName: "kubernetes.io/secret/6f9b208b-086f-4671-8c3f-b3a275740ad7-combined-ca-bundle") pod "memcached-0" (UID: "6f9b208b-086f-4671-8c3f-b3a275740ad7") : secret "combined-ca-bundle" not found Dec 03 11:47:28 crc kubenswrapper[5052]: I1203 11:47:28.296006 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bb51da3b-72cc-4008-aac7-737f6a3031cb-logs" (OuterVolumeSpecName: "logs") pod "bb51da3b-72cc-4008-aac7-737f6a3031cb" (UID: "bb51da3b-72cc-4008-aac7-737f6a3031cb"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:47:28 crc kubenswrapper[5052]: E1203 11:47:28.299239 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-memcached-svc: secret "cert-memcached-svc" not found Dec 03 11:47:28 crc kubenswrapper[5052]: E1203 11:47:28.299318 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f9b208b-086f-4671-8c3f-b3a275740ad7-memcached-tls-certs podName:6f9b208b-086f-4671-8c3f-b3a275740ad7 nodeName:}" failed. No retries permitted until 2025-12-03 11:47:30.299299488 +0000 UTC m=+1640.887541252 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "memcached-tls-certs" (UniqueName: "kubernetes.io/secret/6f9b208b-086f-4671-8c3f-b3a275740ad7-memcached-tls-certs") pod "memcached-0" (UID: "6f9b208b-086f-4671-8c3f-b3a275740ad7") : secret "cert-memcached-svc" not found Dec 03 11:47:28 crc kubenswrapper[5052]: I1203 11:47:28.304986 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/99f96963-d835-4445-84af-0976f1ba0701-logs" (OuterVolumeSpecName: "logs") pod "99f96963-d835-4445-84af-0976f1ba0701" (UID: "99f96963-d835-4445-84af-0976f1ba0701"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:47:28 crc kubenswrapper[5052]: I1203 11:47:28.320452 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:47:28 crc kubenswrapper[5052]: I1203 11:47:28.351864 5052 scope.go:117] "RemoveContainer" containerID="cc31e16d3323f33d812b30383dad0c83a6bb71816abde9316f28a1841980791c" Dec 03 11:47:28 crc kubenswrapper[5052]: I1203 11:47:28.356744 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/99f96963-d835-4445-84af-0976f1ba0701-kube-api-access-gbvkd" (OuterVolumeSpecName: "kube-api-access-gbvkd") pod "99f96963-d835-4445-84af-0976f1ba0701" (UID: "99f96963-d835-4445-84af-0976f1ba0701"). InnerVolumeSpecName "kube-api-access-gbvkd". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:47:28 crc kubenswrapper[5052]: I1203 11:47:28.382981 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/99f96963-d835-4445-84af-0976f1ba0701-config-data-custom" (OuterVolumeSpecName: "config-data-custom") pod "99f96963-d835-4445-84af-0976f1ba0701" (UID: "99f96963-d835-4445-84af-0976f1ba0701"). InnerVolumeSpecName "config-data-custom". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:47:28 crc kubenswrapper[5052]: I1203 11:47:28.383185 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bb51da3b-72cc-4008-aac7-737f6a3031cb-kube-api-access-dbh4l" (OuterVolumeSpecName: "kube-api-access-dbh4l") pod "bb51da3b-72cc-4008-aac7-737f6a3031cb" (UID: "bb51da3b-72cc-4008-aac7-737f6a3031cb"). InnerVolumeSpecName "kube-api-access-dbh4l". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:47:28 crc kubenswrapper[5052]: I1203 11:47:28.395731 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage15-crc" (UniqueName: "kubernetes.io/local-volume/local-storage15-crc") on node "crc" Dec 03 11:47:28 crc kubenswrapper[5052]: I1203 11:47:28.396477 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j2dnf\" (UniqueName: \"kubernetes.io/projected/f7123213-06e1-4b26-9892-c16c3d49c0ab-kube-api-access-j2dnf\") pod \"f7123213-06e1-4b26-9892-c16c3d49c0ab\" (UID: \"f7123213-06e1-4b26-9892-c16c3d49c0ab\") " Dec 03 11:47:28 crc kubenswrapper[5052]: I1203 11:47:28.396977 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/f7123213-06e1-4b26-9892-c16c3d49c0ab-public-tls-certs\") pod \"f7123213-06e1-4b26-9892-c16c3d49c0ab\" (UID: \"f7123213-06e1-4b26-9892-c16c3d49c0ab\") " Dec 03 11:47:28 crc kubenswrapper[5052]: I1203 11:47:28.397005 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/f7123213-06e1-4b26-9892-c16c3d49c0ab-logs\") pod \"f7123213-06e1-4b26-9892-c16c3d49c0ab\" (UID: \"f7123213-06e1-4b26-9892-c16c3d49c0ab\") " Dec 03 11:47:28 crc kubenswrapper[5052]: I1203 11:47:28.397033 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f7123213-06e1-4b26-9892-c16c3d49c0ab-combined-ca-bundle\") pod \"f7123213-06e1-4b26-9892-c16c3d49c0ab\" (UID: \"f7123213-06e1-4b26-9892-c16c3d49c0ab\") " Dec 03 11:47:28 crc kubenswrapper[5052]: I1203 11:47:28.397203 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/f7123213-06e1-4b26-9892-c16c3d49c0ab-internal-tls-certs\") pod \"f7123213-06e1-4b26-9892-c16c3d49c0ab\" (UID: \"f7123213-06e1-4b26-9892-c16c3d49c0ab\") " Dec 03 11:47:28 crc kubenswrapper[5052]: I1203 11:47:28.397319 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/f7123213-06e1-4b26-9892-c16c3d49c0ab-config-data\") pod \"f7123213-06e1-4b26-9892-c16c3d49c0ab\" (UID: \"f7123213-06e1-4b26-9892-c16c3d49c0ab\") " Dec 03 11:47:28 crc kubenswrapper[5052]: I1203 11:47:28.397725 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/99f96963-d835-4445-84af-0976f1ba0701-config-data-custom\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:28 crc kubenswrapper[5052]: I1203 11:47:28.397740 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage15-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage15-crc\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:28 crc kubenswrapper[5052]: I1203 11:47:28.397750 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/bb51da3b-72cc-4008-aac7-737f6a3031cb-logs\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:28 crc kubenswrapper[5052]: I1203 11:47:28.397792 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-gbvkd\" (UniqueName: \"kubernetes.io/projected/99f96963-d835-4445-84af-0976f1ba0701-kube-api-access-gbvkd\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:28 crc kubenswrapper[5052]: I1203 11:47:28.397804 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-dbh4l\" (UniqueName: \"kubernetes.io/projected/bb51da3b-72cc-4008-aac7-737f6a3031cb-kube-api-access-dbh4l\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:28 crc kubenswrapper[5052]: I1203 11:47:28.397814 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/99f96963-d835-4445-84af-0976f1ba0701-logs\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:28 crc kubenswrapper[5052]: I1203 11:47:28.398413 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7123213-06e1-4b26-9892-c16c3d49c0ab-logs" (OuterVolumeSpecName: "logs") pod "f7123213-06e1-4b26-9892-c16c3d49c0ab" (UID: "f7123213-06e1-4b26-9892-c16c3d49c0ab"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:47:28 crc kubenswrapper[5052]: I1203 11:47:28.454583 5052 scope.go:117] "RemoveContainer" containerID="295a2704643ed1264e3a64ad18e13a582bf7cc635129de442187ed9d461c917a" Dec 03 11:47:28 crc kubenswrapper[5052]: I1203 11:47:28.458198 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f7123213-06e1-4b26-9892-c16c3d49c0ab-kube-api-access-j2dnf" (OuterVolumeSpecName: "kube-api-access-j2dnf") pod "f7123213-06e1-4b26-9892-c16c3d49c0ab" (UID: "f7123213-06e1-4b26-9892-c16c3d49c0ab"). InnerVolumeSpecName "kube-api-access-j2dnf". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:47:28 crc kubenswrapper[5052]: I1203 11:47:28.470084 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0914ebfe-13c6-4309-bd10-a67850124e3f-config-data" (OuterVolumeSpecName: "config-data") pod "0914ebfe-13c6-4309-bd10-a67850124e3f" (UID: "0914ebfe-13c6-4309-bd10-a67850124e3f"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:47:28 crc kubenswrapper[5052]: I1203 11:47:28.472227 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/65be66f1-c32d-40b5-81a4-a63258364d9d-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "65be66f1-c32d-40b5-81a4-a63258364d9d" (UID: "65be66f1-c32d-40b5-81a4-a63258364d9d"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:47:28 crc kubenswrapper[5052]: I1203 11:47:28.499752 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/65be66f1-c32d-40b5-81a4-a63258364d9d-config-data" (OuterVolumeSpecName: "config-data") pod "65be66f1-c32d-40b5-81a4-a63258364d9d" (UID: "65be66f1-c32d-40b5-81a4-a63258364d9d"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:47:28 crc kubenswrapper[5052]: E1203 11:47:28.499893 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/combined-ca-bundle: secret "combined-ca-bundle" not found Dec 03 11:47:28 crc kubenswrapper[5052]: E1203 11:47:28.499948 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/507c60e5-056c-4661-a7a7-75c4d9bce7de-combined-ca-bundle podName:507c60e5-056c-4661-a7a7-75c4d9bce7de nodeName:}" failed. No retries permitted until 2025-12-03 11:47:36.499932052 +0000 UTC m=+1647.088173896 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "combined-ca-bundle" (UniqueName: "kubernetes.io/secret/507c60e5-056c-4661-a7a7-75c4d9bce7de-combined-ca-bundle") pod "nova-cell1-conductor-0" (UID: "507c60e5-056c-4661-a7a7-75c4d9bce7de") : secret "combined-ca-bundle" not found Dec 03 11:47:28 crc kubenswrapper[5052]: E1203 11:47:28.499994 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/rabbitmq-cell1-config-data: configmap "rabbitmq-cell1-config-data" not found Dec 03 11:47:28 crc kubenswrapper[5052]: I1203 11:47:28.500009 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/65be66f1-c32d-40b5-81a4-a63258364d9d-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:28 crc kubenswrapper[5052]: I1203 11:47:28.500026 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/65be66f1-c32d-40b5-81a4-a63258364d9d-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:28 crc kubenswrapper[5052]: E1203 11:47:28.500035 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2766f489-9217-47ae-a595-869a26389587-config-data podName:2766f489-9217-47ae-a595-869a26389587 nodeName:}" failed. No retries permitted until 2025-12-03 11:47:36.500021324 +0000 UTC m=+1647.088263088 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "config-data" (UniqueName: "kubernetes.io/configmap/2766f489-9217-47ae-a595-869a26389587-config-data") pod "rabbitmq-cell1-server-0" (UID: "2766f489-9217-47ae-a595-869a26389587") : configmap "rabbitmq-cell1-config-data" not found Dec 03 11:47:28 crc kubenswrapper[5052]: E1203 11:47:28.500059 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 11:47:28 crc kubenswrapper[5052]: I1203 11:47:28.500060 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/0914ebfe-13c6-4309-bd10-a67850124e3f-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:28 crc kubenswrapper[5052]: I1203 11:47:28.500077 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-j2dnf\" (UniqueName: \"kubernetes.io/projected/f7123213-06e1-4b26-9892-c16c3d49c0ab-kube-api-access-j2dnf\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:28 crc kubenswrapper[5052]: E1203 11:47:28.500081 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2d3a3054-6c8e-4808-bd5d-01413970167c-operator-scripts podName:2d3a3054-6c8e-4808-bd5d-01413970167c nodeName:}" failed. No retries permitted until 2025-12-03 11:47:29.500075506 +0000 UTC m=+1640.088317270 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/2d3a3054-6c8e-4808-bd5d-01413970167c-operator-scripts") pod "novaapi5919-account-delete-qkvzd" (UID: "2d3a3054-6c8e-4808-bd5d-01413970167c") : configmap "openstack-scripts" not found Dec 03 11:47:28 crc kubenswrapper[5052]: I1203 11:47:28.500094 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/f7123213-06e1-4b26-9892-c16c3d49c0ab-logs\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:28 crc kubenswrapper[5052]: I1203 11:47:28.507035 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage04-crc" (UniqueName: "kubernetes.io/local-volume/local-storage04-crc") on node "crc" Dec 03 11:47:28 crc kubenswrapper[5052]: I1203 11:47:28.556581 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cadac9a6-1d82-403a-b7a0-692a424dd0e2-config-data" (OuterVolumeSpecName: "config-data") pod "cadac9a6-1d82-403a-b7a0-692a424dd0e2" (UID: "cadac9a6-1d82-403a-b7a0-692a424dd0e2"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:47:28 crc kubenswrapper[5052]: I1203 11:47:28.581133 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/acea154c-f0bb-4066-bd4d-2dda815f301c-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "acea154c-f0bb-4066-bd4d-2dda815f301c" (UID: "acea154c-f0bb-4066-bd4d-2dda815f301c"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:47:28 crc kubenswrapper[5052]: I1203 11:47:28.603902 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/acea154c-f0bb-4066-bd4d-2dda815f301c-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:28 crc kubenswrapper[5052]: I1203 11:47:28.604266 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage04-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage04-crc\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:28 crc kubenswrapper[5052]: I1203 11:47:28.604280 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/cadac9a6-1d82-403a-b7a0-692a424dd0e2-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:28 crc kubenswrapper[5052]: I1203 11:47:28.669620 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/00b12739-1437-4af3-ab6d-ad830b67ad2e-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "00b12739-1437-4af3-ab6d-ad830b67ad2e" (UID: "00b12739-1437-4af3-ab6d-ad830b67ad2e"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:47:28 crc kubenswrapper[5052]: I1203 11:47:28.707139 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/00b12739-1437-4af3-ab6d-ad830b67ad2e-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:28 crc kubenswrapper[5052]: I1203 11:47:28.910797 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bb51da3b-72cc-4008-aac7-737f6a3031cb-config-data" (OuterVolumeSpecName: "config-data") pod "bb51da3b-72cc-4008-aac7-737f6a3031cb" (UID: "bb51da3b-72cc-4008-aac7-737f6a3031cb"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:47:28 crc kubenswrapper[5052]: I1203 11:47:28.912746 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/bb51da3b-72cc-4008-aac7-737f6a3031cb-config-data\") pod \"bb51da3b-72cc-4008-aac7-737f6a3031cb\" (UID: \"bb51da3b-72cc-4008-aac7-737f6a3031cb\") " Dec 03 11:47:28 crc kubenswrapper[5052]: W1203 11:47:28.913434 5052 empty_dir.go:500] Warning: Unmount skipped because path does not exist: /var/lib/kubelet/pods/bb51da3b-72cc-4008-aac7-737f6a3031cb/volumes/kubernetes.io~secret/config-data Dec 03 11:47:28 crc kubenswrapper[5052]: I1203 11:47:28.913449 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bb51da3b-72cc-4008-aac7-737f6a3031cb-config-data" (OuterVolumeSpecName: "config-data") pod "bb51da3b-72cc-4008-aac7-737f6a3031cb" (UID: "bb51da3b-72cc-4008-aac7-737f6a3031cb"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:47:28 crc kubenswrapper[5052]: I1203 11:47:28.927217 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/novaapi5919-account-delete-qkvzd" event={"ID":"2d3a3054-6c8e-4808-bd5d-01413970167c","Type":"ContainerStarted","Data":"ae6315a718283526ad43a5d86df7f92e3baca5b6bd2a9ae8afdd33e574384a43"} Dec 03 11:47:28 crc kubenswrapper[5052]: I1203 11:47:28.927427 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/novaapi5919-account-delete-qkvzd" podUID="2d3a3054-6c8e-4808-bd5d-01413970167c" containerName="mariadb-account-delete" containerID="cri-o://ae6315a718283526ad43a5d86df7f92e3baca5b6bd2a9ae8afdd33e574384a43" gracePeriod=30 Dec 03 11:47:28 crc kubenswrapper[5052]: I1203 11:47:28.933762 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement9a4f-account-delete-f6sjd" event={"ID":"8c2252c0-5e30-4e38-b1a1-af0242df6ac0","Type":"ContainerDied","Data":"608dc22687b11333fce9416d617bd57ae95cf87ae7a7d4d3a551d4a0344855fb"} Dec 03 11:47:28 crc kubenswrapper[5052]: I1203 11:47:28.933802 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="608dc22687b11333fce9416d617bd57ae95cf87ae7a7d4d3a551d4a0344855fb" Dec 03 11:47:28 crc kubenswrapper[5052]: I1203 11:47:28.937340 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron18a7-account-delete-x9lqd" event={"ID":"aa9a649b-9229-4cdd-8e27-8aedd48f4bbd","Type":"ContainerDied","Data":"ba0d9f752351adaf18aa306215d73b8cbfb89b54414b7a903a9d289209402478"} Dec 03 11:47:28 crc kubenswrapper[5052]: I1203 11:47:28.937464 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ba0d9f752351adaf18aa306215d73b8cbfb89b54414b7a903a9d289209402478" Dec 03 11:47:28 crc kubenswrapper[5052]: I1203 11:47:28.939639 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-7cfb847cb8-jn85t" event={"ID":"99f96963-d835-4445-84af-0976f1ba0701","Type":"ContainerDied","Data":"c82775e0c56adde0c20cd2407dc86e7083b4db213f4d1d7df2de7a43f3a5bcb1"} Dec 03 11:47:28 crc kubenswrapper[5052]: I1203 11:47:28.939732 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-api-7cfb847cb8-jn85t" Dec 03 11:47:28 crc kubenswrapper[5052]: I1203 11:47:28.947589 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/99f96963-d835-4445-84af-0976f1ba0701-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "99f96963-d835-4445-84af-0976f1ba0701" (UID: "99f96963-d835-4445-84af-0976f1ba0701"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:47:28 crc kubenswrapper[5052]: I1203 11:47:28.948875 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-779b679dc7-nvtdm" Dec 03 11:47:28 crc kubenswrapper[5052]: I1203 11:47:28.949231 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-779b679dc7-nvtdm" event={"ID":"65be66f1-c32d-40b5-81a4-a63258364d9d","Type":"ContainerDied","Data":"fa978c7c93e89e6852b87846dfa444775998fa7b28a1e32a276fd4189faabb8b"} Dec 03 11:47:28 crc kubenswrapper[5052]: I1203 11:47:28.951596 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican5976-account-delete-q6j5s" event={"ID":"0aefd6f6-cbb3-4c45-9f69-7124e43255e3","Type":"ContainerDied","Data":"943d5e6f6f6cf3ad6ad0d520da8f2680a837f3c4b198e348c121ae97df4c9f94"} Dec 03 11:47:28 crc kubenswrapper[5052]: I1203 11:47:28.952977 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="943d5e6f6f6cf3ad6ad0d520da8f2680a837f3c4b198e348c121ae97df4c9f94" Dec 03 11:47:28 crc kubenswrapper[5052]: I1203 11:47:28.956586 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/novaapi5919-account-delete-qkvzd" podStartSLOduration=7.956546877 podStartE2EDuration="7.956546877s" podCreationTimestamp="2025-12-03 11:47:21 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:47:28.943975086 +0000 UTC m=+1639.532216860" watchObservedRunningTime="2025-12-03 11:47:28.956546877 +0000 UTC m=+1639.544788641" Dec 03 11:47:28 crc kubenswrapper[5052]: I1203 11:47:28.956617 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder1bf0-account-delete-hs8j6" event={"ID":"0b94b517-5f9b-485d-a109-0a11e15a3ad3","Type":"ContainerDied","Data":"cefa20da1d4e1f488a1ea2bdf21bf298fbaf83848e745c3e1ca95b0e9818d28b"} Dec 03 11:47:28 crc kubenswrapper[5052]: I1203 11:47:28.956651 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cefa20da1d4e1f488a1ea2bdf21bf298fbaf83848e745c3e1ca95b0e9818d28b" Dec 03 11:47:28 crc kubenswrapper[5052]: I1203 11:47:28.959864 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f7123213-06e1-4b26-9892-c16c3d49c0ab-config-data" (OuterVolumeSpecName: "config-data") pod "f7123213-06e1-4b26-9892-c16c3d49c0ab" (UID: "f7123213-06e1-4b26-9892-c16c3d49c0ab"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:47:28 crc kubenswrapper[5052]: I1203 11:47:28.960517 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"f7123213-06e1-4b26-9892-c16c3d49c0ab","Type":"ContainerDied","Data":"f7781482f3081fcf1252f242cf1ba75ded9204d78cf6aeef6e5e17e22416a3e9"} Dec 03 11:47:28 crc kubenswrapper[5052]: I1203 11:47:28.961718 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:47:28 crc kubenswrapper[5052]: I1203 11:47:28.962991 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance3554-account-delete-8ghdx" event={"ID":"950f0626-40d2-41c9-9701-cba50843c43e","Type":"ContainerDied","Data":"3c7a993dd1496b0fec21a1c11de57571355d652145a1ab336ad9cb1fa2aa09bd"} Dec 03 11:47:28 crc kubenswrapper[5052]: I1203 11:47:28.963026 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3c7a993dd1496b0fec21a1c11de57571355d652145a1ab336ad9cb1fa2aa09bd" Dec 03 11:47:28 crc kubenswrapper[5052]: I1203 11:47:28.970409 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:47:28 crc kubenswrapper[5052]: I1203 11:47:28.981291 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/novacell0f13b-account-delete-g7m79" event={"ID":"e5339997-33d5-4794-8fa6-042b63e9391e","Type":"ContainerStarted","Data":"3943c83b52b8dca3321af840235a5573bed2a0ef796bf876d3af6fcba17e64ff"} Dec 03 11:47:28 crc kubenswrapper[5052]: I1203 11:47:28.981432 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/novacell0f13b-account-delete-g7m79" podUID="e5339997-33d5-4794-8fa6-042b63e9391e" containerName="mariadb-account-delete" containerID="cri-o://3943c83b52b8dca3321af840235a5573bed2a0ef796bf876d3af6fcba17e64ff" gracePeriod=30 Dec 03 11:47:28 crc kubenswrapper[5052]: I1203 11:47:28.985210 5052 generic.go:334] "Generic (PLEG): container finished" podID="39b9198e-e3d7-4f85-b6a2-55960bdda82a" containerID="1dbdda8905493662bc20bc18fcae5f54dd4929f65dec1b16df59f03dbfd76d8a" exitCode=0 Dec 03 11:47:28 crc kubenswrapper[5052]: I1203 11:47:28.985256 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"39b9198e-e3d7-4f85-b6a2-55960bdda82a","Type":"ContainerDied","Data":"1dbdda8905493662bc20bc18fcae5f54dd4929f65dec1b16df59f03dbfd76d8a"} Dec 03 11:47:28 crc kubenswrapper[5052]: I1203 11:47:28.987023 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-lfl6r" event={"ID":"93fb4fd0-aa18-4a2d-bfb4-b65bbc378ae5","Type":"ContainerStarted","Data":"ea6a0525e4279380cf0ca84429fe6bc0012e5082feedb56ba078f355d1fafc6d"} Dec 03 11:47:28 crc kubenswrapper[5052]: I1203 11:47:28.987863 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-lfl6r" Dec 03 11:47:28 crc kubenswrapper[5052]: I1203 11:47:28.993145 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/99f96963-d835-4445-84af-0976f1ba0701-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "99f96963-d835-4445-84af-0976f1ba0701" (UID: "99f96963-d835-4445-84af-0976f1ba0701"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:47:28 crc kubenswrapper[5052]: I1203 11:47:28.995961 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/00b12739-1437-4af3-ab6d-ad830b67ad2e-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "00b12739-1437-4af3-ab6d-ad830b67ad2e" (UID: "00b12739-1437-4af3-ab6d-ad830b67ad2e"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:47:29 crc kubenswrapper[5052]: I1203 11:47:29.004357 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/novacell0f13b-account-delete-g7m79" podStartSLOduration=8.004332537 podStartE2EDuration="8.004332537s" podCreationTimestamp="2025-12-03 11:47:21 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:47:28.999943761 +0000 UTC m=+1639.588185525" watchObservedRunningTime="2025-12-03 11:47:29.004332537 +0000 UTC m=+1639.592574301" Dec 03 11:47:29 crc kubenswrapper[5052]: I1203 11:47:29.011968 5052 generic.go:334] "Generic (PLEG): container finished" podID="d8de865a-0f6c-413d-8a37-3ec4d496c4bb" containerID="def922586d645f9987c75d4862d880ed855e05523f69e809e4d0637bfe42a9d8" exitCode=0 Dec 03 11:47:29 crc kubenswrapper[5052]: I1203 11:47:29.012149 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-keystone-listener-6794d844d6-f4tr7" event={"ID":"d8de865a-0f6c-413d-8a37-3ec4d496c4bb","Type":"ContainerDied","Data":"def922586d645f9987c75d4862d880ed855e05523f69e809e4d0637bfe42a9d8"} Dec 03 11:47:29 crc kubenswrapper[5052]: I1203 11:47:29.014733 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"bb51da3b-72cc-4008-aac7-737f6a3031cb","Type":"ContainerDied","Data":"ecf1ebfdd6784d04d2c4832c90b30e0d04f644e2d2252aa874e943438f064bf5"} Dec 03 11:47:29 crc kubenswrapper[5052]: I1203 11:47:29.015324 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/f7123213-06e1-4b26-9892-c16c3d49c0ab-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:29 crc kubenswrapper[5052]: I1203 11:47:29.015376 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/00b12739-1437-4af3-ab6d-ad830b67ad2e-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:29 crc kubenswrapper[5052]: I1203 11:47:29.015391 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/99f96963-d835-4445-84af-0976f1ba0701-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:29 crc kubenswrapper[5052]: I1203 11:47:29.015421 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/bb51da3b-72cc-4008-aac7-737f6a3031cb-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:29 crc kubenswrapper[5052]: I1203 11:47:29.015449 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/99f96963-d835-4445-84af-0976f1ba0701-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:29 crc kubenswrapper[5052]: I1203 11:47:29.015819 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:47:29 crc kubenswrapper[5052]: I1203 11:47:29.018855 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:47:29 crc kubenswrapper[5052]: I1203 11:47:29.019289 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-worker-79689f8957-qmkdm" Dec 03 11:47:29 crc kubenswrapper[5052]: I1203 11:47:29.036008 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-lfl6r" podStartSLOduration=8.035984275 podStartE2EDuration="8.035984275s" podCreationTimestamp="2025-12-03 11:47:21 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:47:29.024083114 +0000 UTC m=+1639.612324888" watchObservedRunningTime="2025-12-03 11:47:29.035984275 +0000 UTC m=+1639.624226039" Dec 03 11:47:29 crc kubenswrapper[5052]: I1203 11:47:29.040813 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0914ebfe-13c6-4309-bd10-a67850124e3f-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "0914ebfe-13c6-4309-bd10-a67850124e3f" (UID: "0914ebfe-13c6-4309-bd10-a67850124e3f"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:47:29 crc kubenswrapper[5052]: I1203 11:47:29.069704 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0183ebb5-b91b-443a-9535-2008c9207ab1-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "0183ebb5-b91b-443a-9535-2008c9207ab1" (UID: "0183ebb5-b91b-443a-9535-2008c9207ab1"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:47:29 crc kubenswrapper[5052]: I1203 11:47:29.115828 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/99f96963-d835-4445-84af-0976f1ba0701-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "99f96963-d835-4445-84af-0976f1ba0701" (UID: "99f96963-d835-4445-84af-0976f1ba0701"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:47:29 crc kubenswrapper[5052]: I1203 11:47:29.117150 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/acea154c-f0bb-4066-bd4d-2dda815f301c-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "acea154c-f0bb-4066-bd4d-2dda815f301c" (UID: "acea154c-f0bb-4066-bd4d-2dda815f301c"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:47:29 crc kubenswrapper[5052]: I1203 11:47:29.117656 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0183ebb5-b91b-443a-9535-2008c9207ab1-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:29 crc kubenswrapper[5052]: I1203 11:47:29.117678 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/0914ebfe-13c6-4309-bd10-a67850124e3f-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:29 crc kubenswrapper[5052]: I1203 11:47:29.117691 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/99f96963-d835-4445-84af-0976f1ba0701-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:29 crc kubenswrapper[5052]: I1203 11:47:29.117702 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/acea154c-f0bb-4066-bd4d-2dda815f301c-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:29 crc kubenswrapper[5052]: I1203 11:47:29.129899 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/00b12739-1437-4af3-ab6d-ad830b67ad2e-config-data" (OuterVolumeSpecName: "config-data") pod "00b12739-1437-4af3-ab6d-ad830b67ad2e" (UID: "00b12739-1437-4af3-ab6d-ad830b67ad2e"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:47:29 crc kubenswrapper[5052]: I1203 11:47:29.133401 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f7123213-06e1-4b26-9892-c16c3d49c0ab-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "f7123213-06e1-4b26-9892-c16c3d49c0ab" (UID: "f7123213-06e1-4b26-9892-c16c3d49c0ab"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:47:29 crc kubenswrapper[5052]: I1203 11:47:29.138454 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1b45577a-92b7-4c67-941b-cfc2fbfa3eb2-config-data" (OuterVolumeSpecName: "config-data") pod "1b45577a-92b7-4c67-941b-cfc2fbfa3eb2" (UID: "1b45577a-92b7-4c67-941b-cfc2fbfa3eb2"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:47:29 crc kubenswrapper[5052]: I1203 11:47:29.140904 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/99f96963-d835-4445-84af-0976f1ba0701-config-data" (OuterVolumeSpecName: "config-data") pod "99f96963-d835-4445-84af-0976f1ba0701" (UID: "99f96963-d835-4445-84af-0976f1ba0701"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:47:29 crc kubenswrapper[5052]: I1203 11:47:29.154410 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0183ebb5-b91b-443a-9535-2008c9207ab1-config-data" (OuterVolumeSpecName: "config-data") pod "0183ebb5-b91b-443a-9535-2008c9207ab1" (UID: "0183ebb5-b91b-443a-9535-2008c9207ab1"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:47:29 crc kubenswrapper[5052]: I1203 11:47:29.156007 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f7123213-06e1-4b26-9892-c16c3d49c0ab-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "f7123213-06e1-4b26-9892-c16c3d49c0ab" (UID: "f7123213-06e1-4b26-9892-c16c3d49c0ab"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:47:29 crc kubenswrapper[5052]: I1203 11:47:29.163397 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bb51da3b-72cc-4008-aac7-737f6a3031cb-nova-metadata-tls-certs" (OuterVolumeSpecName: "nova-metadata-tls-certs") pod "bb51da3b-72cc-4008-aac7-737f6a3031cb" (UID: "bb51da3b-72cc-4008-aac7-737f6a3031cb"). InnerVolumeSpecName "nova-metadata-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:47:29 crc kubenswrapper[5052]: I1203 11:47:29.164043 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1b45577a-92b7-4c67-941b-cfc2fbfa3eb2-ceilometer-tls-certs" (OuterVolumeSpecName: "ceilometer-tls-certs") pod "1b45577a-92b7-4c67-941b-cfc2fbfa3eb2" (UID: "1b45577a-92b7-4c67-941b-cfc2fbfa3eb2"). InnerVolumeSpecName "ceilometer-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:47:29 crc kubenswrapper[5052]: I1203 11:47:29.177580 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/65be66f1-c32d-40b5-81a4-a63258364d9d-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "65be66f1-c32d-40b5-81a4-a63258364d9d" (UID: "65be66f1-c32d-40b5-81a4-a63258364d9d"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:47:29 crc kubenswrapper[5052]: I1203 11:47:29.185782 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bb51da3b-72cc-4008-aac7-737f6a3031cb-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "bb51da3b-72cc-4008-aac7-737f6a3031cb" (UID: "bb51da3b-72cc-4008-aac7-737f6a3031cb"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:47:29 crc kubenswrapper[5052]: I1203 11:47:29.185784 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/acea154c-f0bb-4066-bd4d-2dda815f301c-config-data" (OuterVolumeSpecName: "config-data") pod "acea154c-f0bb-4066-bd4d-2dda815f301c" (UID: "acea154c-f0bb-4066-bd4d-2dda815f301c"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:47:29 crc kubenswrapper[5052]: I1203 11:47:29.195623 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f7123213-06e1-4b26-9892-c16c3d49c0ab-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "f7123213-06e1-4b26-9892-c16c3d49c0ab" (UID: "f7123213-06e1-4b26-9892-c16c3d49c0ab"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:47:29 crc kubenswrapper[5052]: I1203 11:47:29.196652 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/00b12739-1437-4af3-ab6d-ad830b67ad2e-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "00b12739-1437-4af3-ab6d-ad830b67ad2e" (UID: "00b12739-1437-4af3-ab6d-ad830b67ad2e"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:47:29 crc kubenswrapper[5052]: I1203 11:47:29.203685 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/65be66f1-c32d-40b5-81a4-a63258364d9d-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "65be66f1-c32d-40b5-81a4-a63258364d9d" (UID: "65be66f1-c32d-40b5-81a4-a63258364d9d"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:47:29 crc kubenswrapper[5052]: I1203 11:47:29.204996 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1b45577a-92b7-4c67-941b-cfc2fbfa3eb2-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "1b45577a-92b7-4c67-941b-cfc2fbfa3eb2" (UID: "1b45577a-92b7-4c67-941b-cfc2fbfa3eb2"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:47:29 crc kubenswrapper[5052]: I1203 11:47:29.207406 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0183ebb5-b91b-443a-9535-2008c9207ab1-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "0183ebb5-b91b-443a-9535-2008c9207ab1" (UID: "0183ebb5-b91b-443a-9535-2008c9207ab1"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:47:29 crc kubenswrapper[5052]: E1203 11:47:29.219562 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 11:47:29 crc kubenswrapper[5052]: E1203 11:47:29.219662 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/950f0626-40d2-41c9-9701-cba50843c43e-operator-scripts podName:950f0626-40d2-41c9-9701-cba50843c43e nodeName:}" failed. No retries permitted until 2025-12-03 11:47:33.219643642 +0000 UTC m=+1643.807885406 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/950f0626-40d2-41c9-9701-cba50843c43e-operator-scripts") pod "glance3554-account-delete-8ghdx" (UID: "950f0626-40d2-41c9-9701-cba50843c43e") : configmap "openstack-scripts" not found Dec 03 11:47:29 crc kubenswrapper[5052]: E1203 11:47:29.219696 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 11:47:29 crc kubenswrapper[5052]: E1203 11:47:29.219776 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e5339997-33d5-4794-8fa6-042b63e9391e-operator-scripts podName:e5339997-33d5-4794-8fa6-042b63e9391e nodeName:}" failed. No retries permitted until 2025-12-03 11:47:31.219758155 +0000 UTC m=+1641.807999979 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/e5339997-33d5-4794-8fa6-042b63e9391e-operator-scripts") pod "novacell0f13b-account-delete-g7m79" (UID: "e5339997-33d5-4794-8fa6-042b63e9391e") : configmap "openstack-scripts" not found Dec 03 11:47:29 crc kubenswrapper[5052]: E1203 11:47:29.219586 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 11:47:29 crc kubenswrapper[5052]: E1203 11:47:29.219819 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8c2252c0-5e30-4e38-b1a1-af0242df6ac0-operator-scripts podName:8c2252c0-5e30-4e38-b1a1-af0242df6ac0 nodeName:}" failed. No retries permitted until 2025-12-03 11:47:33.219813357 +0000 UTC m=+1643.808055121 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/8c2252c0-5e30-4e38-b1a1-af0242df6ac0-operator-scripts") pod "placement9a4f-account-delete-f6sjd" (UID: "8c2252c0-5e30-4e38-b1a1-af0242df6ac0") : configmap "openstack-scripts" not found Dec 03 11:47:29 crc kubenswrapper[5052]: I1203 11:47:29.219585 5052 reconciler_common.go:293] "Volume detached for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/1b45577a-92b7-4c67-941b-cfc2fbfa3eb2-ceilometer-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:29 crc kubenswrapper[5052]: I1203 11:47:29.219858 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/00b12739-1437-4af3-ab6d-ad830b67ad2e-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:29 crc kubenswrapper[5052]: I1203 11:47:29.219873 5052 reconciler_common.go:293] "Volume detached for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/bb51da3b-72cc-4008-aac7-737f6a3031cb-nova-metadata-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:29 crc kubenswrapper[5052]: I1203 11:47:29.219885 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/0183ebb5-b91b-443a-9535-2008c9207ab1-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:29 crc kubenswrapper[5052]: I1203 11:47:29.219896 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/f7123213-06e1-4b26-9892-c16c3d49c0ab-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:29 crc kubenswrapper[5052]: I1203 11:47:29.219918 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f7123213-06e1-4b26-9892-c16c3d49c0ab-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:29 crc kubenswrapper[5052]: I1203 11:47:29.219928 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/65be66f1-c32d-40b5-81a4-a63258364d9d-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:29 crc kubenswrapper[5052]: I1203 11:47:29.219936 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/0183ebb5-b91b-443a-9535-2008c9207ab1-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:29 crc kubenswrapper[5052]: I1203 11:47:29.219944 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/bb51da3b-72cc-4008-aac7-737f6a3031cb-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:29 crc kubenswrapper[5052]: I1203 11:47:29.219953 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/65be66f1-c32d-40b5-81a4-a63258364d9d-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:29 crc kubenswrapper[5052]: I1203 11:47:29.219962 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/acea154c-f0bb-4066-bd4d-2dda815f301c-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:29 crc kubenswrapper[5052]: I1203 11:47:29.219970 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/99f96963-d835-4445-84af-0976f1ba0701-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:29 crc kubenswrapper[5052]: I1203 11:47:29.219980 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/00b12739-1437-4af3-ab6d-ad830b67ad2e-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:29 crc kubenswrapper[5052]: I1203 11:47:29.219989 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/f7123213-06e1-4b26-9892-c16c3d49c0ab-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:29 crc kubenswrapper[5052]: I1203 11:47:29.219998 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/1b45577a-92b7-4c67-941b-cfc2fbfa3eb2-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:29 crc kubenswrapper[5052]: I1203 11:47:29.220008 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1b45577a-92b7-4c67-941b-cfc2fbfa3eb2-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:29 crc kubenswrapper[5052]: I1203 11:47:29.247600 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0183ebb5-b91b-443a-9535-2008c9207ab1-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "0183ebb5-b91b-443a-9535-2008c9207ab1" (UID: "0183ebb5-b91b-443a-9535-2008c9207ab1"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:47:29 crc kubenswrapper[5052]: I1203 11:47:29.321271 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/0183ebb5-b91b-443a-9535-2008c9207ab1-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:29 crc kubenswrapper[5052]: I1203 11:47:29.460490 5052 scope.go:117] "RemoveContainer" containerID="8c71cc229cea36e3361a6c384c5386d5cc260814f0041348a439548161af1249" Dec 03 11:47:29 crc kubenswrapper[5052]: I1203 11:47:29.466938 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican5976-account-delete-q6j5s" Dec 03 11:47:29 crc kubenswrapper[5052]: E1203 11:47:29.533139 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 11:47:29 crc kubenswrapper[5052]: E1203 11:47:29.533393 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2d3a3054-6c8e-4808-bd5d-01413970167c-operator-scripts podName:2d3a3054-6c8e-4808-bd5d-01413970167c nodeName:}" failed. No retries permitted until 2025-12-03 11:47:31.53337685 +0000 UTC m=+1642.121618614 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/2d3a3054-6c8e-4808-bd5d-01413970167c-operator-scripts") pod "novaapi5919-account-delete-qkvzd" (UID: "2d3a3054-6c8e-4808-bd5d-01413970167c") : configmap "openstack-scripts" not found Dec 03 11:47:29 crc kubenswrapper[5052]: E1203 11:47:29.587423 5052 cadvisor_stats_provider.go:516] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcadac9a6_1d82_403a_b7a0_692a424dd0e2.slice\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod65be66f1_c32d_40b5_81a4_a63258364d9d.slice\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod99f96963_d835_4445_84af_0976f1ba0701.slice/crio-c82775e0c56adde0c20cd2407dc86e7083b4db213f4d1d7df2de7a43f3a5bcb1\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcadac9a6_1d82_403a_b7a0_692a424dd0e2.slice/crio-de6975120318e384ab085701669f1c586c0650ffb743e7c4a4482d13b9ed77b0\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod99f96963_d835_4445_84af_0976f1ba0701.slice\": RecentStats: unable to find data in memory cache]" Dec 03 11:47:29 crc kubenswrapper[5052]: I1203 11:47:29.610931 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement9a4f-account-delete-f6sjd" Dec 03 11:47:29 crc kubenswrapper[5052]: I1203 11:47:29.612248 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" podUID="84b8e975-30ee-42c6-8952-25aebf2c378b" containerName="nova-cell1-novncproxy-novncproxy" probeResult="failure" output="Get \"https://10.217.1.25:6080/vnc_lite.html\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Dec 03 11:47:29 crc kubenswrapper[5052]: I1203 11:47:29.617936 5052 scope.go:117] "RemoveContainer" containerID="05471f2484133fbd796cb8f6c5d16314217c42ed6ab7d7d6511de8a201b11eb9" Dec 03 11:47:29 crc kubenswrapper[5052]: I1203 11:47:29.623447 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance3554-account-delete-8ghdx" Dec 03 11:47:29 crc kubenswrapper[5052]: I1203 11:47:29.634744 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lcjjc\" (UniqueName: \"kubernetes.io/projected/0aefd6f6-cbb3-4c45-9f69-7124e43255e3-kube-api-access-lcjjc\") pod \"0aefd6f6-cbb3-4c45-9f69-7124e43255e3\" (UID: \"0aefd6f6-cbb3-4c45-9f69-7124e43255e3\") " Dec 03 11:47:29 crc kubenswrapper[5052]: I1203 11:47:29.634809 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/0aefd6f6-cbb3-4c45-9f69-7124e43255e3-operator-scripts\") pod \"0aefd6f6-cbb3-4c45-9f69-7124e43255e3\" (UID: \"0aefd6f6-cbb3-4c45-9f69-7124e43255e3\") " Dec 03 11:47:29 crc kubenswrapper[5052]: E1203 11:47:29.635296 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cinder-config-data: secret "cinder-config-data" not found Dec 03 11:47:29 crc kubenswrapper[5052]: E1203 11:47:29.635349 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e947107f-104b-4699-a994-bc77285f0cf5-config-data podName:e947107f-104b-4699-a994-bc77285f0cf5 nodeName:}" failed. No retries permitted until 2025-12-03 11:47:37.635336424 +0000 UTC m=+1648.223578188 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "config-data" (UniqueName: "kubernetes.io/secret/e947107f-104b-4699-a994-bc77285f0cf5-config-data") pod "cinder-scheduler-0" (UID: "e947107f-104b-4699-a994-bc77285f0cf5") : secret "cinder-config-data" not found Dec 03 11:47:29 crc kubenswrapper[5052]: I1203 11:47:29.636044 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0aefd6f6-cbb3-4c45-9f69-7124e43255e3-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "0aefd6f6-cbb3-4c45-9f69-7124e43255e3" (UID: "0aefd6f6-cbb3-4c45-9f69-7124e43255e3"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:47:29 crc kubenswrapper[5052]: E1203 11:47:29.636109 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/combined-ca-bundle: secret "combined-ca-bundle" not found Dec 03 11:47:29 crc kubenswrapper[5052]: E1203 11:47:29.636133 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e947107f-104b-4699-a994-bc77285f0cf5-combined-ca-bundle podName:e947107f-104b-4699-a994-bc77285f0cf5 nodeName:}" failed. No retries permitted until 2025-12-03 11:47:37.636125407 +0000 UTC m=+1648.224367171 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "combined-ca-bundle" (UniqueName: "kubernetes.io/secret/e947107f-104b-4699-a994-bc77285f0cf5-combined-ca-bundle") pod "cinder-scheduler-0" (UID: "e947107f-104b-4699-a994-bc77285f0cf5") : secret "combined-ca-bundle" not found Dec 03 11:47:29 crc kubenswrapper[5052]: E1203 11:47:29.636238 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cinder-scripts: secret "cinder-scripts" not found Dec 03 11:47:29 crc kubenswrapper[5052]: E1203 11:47:29.636290 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e947107f-104b-4699-a994-bc77285f0cf5-scripts podName:e947107f-104b-4699-a994-bc77285f0cf5 nodeName:}" failed. No retries permitted until 2025-12-03 11:47:37.636261321 +0000 UTC m=+1648.224503085 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "scripts" (UniqueName: "kubernetes.io/secret/e947107f-104b-4699-a994-bc77285f0cf5-scripts") pod "cinder-scheduler-0" (UID: "e947107f-104b-4699-a994-bc77285f0cf5") : secret "cinder-scripts" not found Dec 03 11:47:29 crc kubenswrapper[5052]: E1203 11:47:29.636397 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cinder-scheduler-config-data: secret "cinder-scheduler-config-data" not found Dec 03 11:47:29 crc kubenswrapper[5052]: E1203 11:47:29.636429 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e947107f-104b-4699-a994-bc77285f0cf5-config-data-custom podName:e947107f-104b-4699-a994-bc77285f0cf5 nodeName:}" failed. No retries permitted until 2025-12-03 11:47:37.636420045 +0000 UTC m=+1648.224661809 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "config-data-custom" (UniqueName: "kubernetes.io/secret/e947107f-104b-4699-a994-bc77285f0cf5-config-data-custom") pod "cinder-scheduler-0" (UID: "e947107f-104b-4699-a994-bc77285f0cf5") : secret "cinder-scheduler-config-data" not found Dec 03 11:47:29 crc kubenswrapper[5052]: I1203 11:47:29.661304 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0aefd6f6-cbb3-4c45-9f69-7124e43255e3-kube-api-access-lcjjc" (OuterVolumeSpecName: "kube-api-access-lcjjc") pod "0aefd6f6-cbb3-4c45-9f69-7124e43255e3" (UID: "0aefd6f6-cbb3-4c45-9f69-7124e43255e3"). InnerVolumeSpecName "kube-api-access-lcjjc". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:47:29 crc kubenswrapper[5052]: I1203 11:47:29.686090 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder1bf0-account-delete-hs8j6" Dec 03 11:47:29 crc kubenswrapper[5052]: I1203 11:47:29.690924 5052 scope.go:117] "RemoveContainer" containerID="9e68066ce0ef8e9380353738e119a2afc7b38dcd3bd33c671f3778712cfb46e4" Dec 03 11:47:29 crc kubenswrapper[5052]: I1203 11:47:29.701206 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/swift-proxy-6fdf66df4-gsqlq" podUID="84957e18-5747-4786-9c4f-6af023122726" containerName="proxy-httpd" probeResult="failure" output="Get \"https://10.217.1.3:8080/healthcheck\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Dec 03 11:47:29 crc kubenswrapper[5052]: I1203 11:47:29.701348 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/swift-proxy-6fdf66df4-gsqlq" podUID="84957e18-5747-4786-9c4f-6af023122726" containerName="proxy-server" probeResult="failure" output="Get \"https://10.217.1.3:8080/healthcheck\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Dec 03 11:47:29 crc kubenswrapper[5052]: I1203 11:47:29.704278 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron18a7-account-delete-x9lqd" Dec 03 11:47:29 crc kubenswrapper[5052]: I1203 11:47:29.726980 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-worker-79689f8957-qmkdm"] Dec 03 11:47:29 crc kubenswrapper[5052]: I1203 11:47:29.727029 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/barbican-worker-79689f8957-qmkdm"] Dec 03 11:47:29 crc kubenswrapper[5052]: I1203 11:47:29.727047 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/keystone-779b679dc7-nvtdm"] Dec 03 11:47:29 crc kubenswrapper[5052]: I1203 11:47:29.734148 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-keystone-listener-6794d844d6-f4tr7" Dec 03 11:47:29 crc kubenswrapper[5052]: I1203 11:47:29.736351 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/8c2252c0-5e30-4e38-b1a1-af0242df6ac0-operator-scripts\") pod \"8c2252c0-5e30-4e38-b1a1-af0242df6ac0\" (UID: \"8c2252c0-5e30-4e38-b1a1-af0242df6ac0\") " Dec 03 11:47:29 crc kubenswrapper[5052]: I1203 11:47:29.736504 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/950f0626-40d2-41c9-9701-cba50843c43e-operator-scripts\") pod \"950f0626-40d2-41c9-9701-cba50843c43e\" (UID: \"950f0626-40d2-41c9-9701-cba50843c43e\") " Dec 03 11:47:29 crc kubenswrapper[5052]: I1203 11:47:29.736599 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dpgmf\" (UniqueName: \"kubernetes.io/projected/8c2252c0-5e30-4e38-b1a1-af0242df6ac0-kube-api-access-dpgmf\") pod \"8c2252c0-5e30-4e38-b1a1-af0242df6ac0\" (UID: \"8c2252c0-5e30-4e38-b1a1-af0242df6ac0\") " Dec 03 11:47:29 crc kubenswrapper[5052]: I1203 11:47:29.736673 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rvkbs\" (UniqueName: \"kubernetes.io/projected/950f0626-40d2-41c9-9701-cba50843c43e-kube-api-access-rvkbs\") pod \"950f0626-40d2-41c9-9701-cba50843c43e\" (UID: \"950f0626-40d2-41c9-9701-cba50843c43e\") " Dec 03 11:47:29 crc kubenswrapper[5052]: I1203 11:47:29.737181 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-lcjjc\" (UniqueName: \"kubernetes.io/projected/0aefd6f6-cbb3-4c45-9f69-7124e43255e3-kube-api-access-lcjjc\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:29 crc kubenswrapper[5052]: I1203 11:47:29.737202 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/0aefd6f6-cbb3-4c45-9f69-7124e43255e3-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:29 crc kubenswrapper[5052]: I1203 11:47:29.738757 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/950f0626-40d2-41c9-9701-cba50843c43e-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "950f0626-40d2-41c9-9701-cba50843c43e" (UID: "950f0626-40d2-41c9-9701-cba50843c43e"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:47:29 crc kubenswrapper[5052]: I1203 11:47:29.738889 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8c2252c0-5e30-4e38-b1a1-af0242df6ac0-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "8c2252c0-5e30-4e38-b1a1-af0242df6ac0" (UID: "8c2252c0-5e30-4e38-b1a1-af0242df6ac0"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:47:29 crc kubenswrapper[5052]: I1203 11:47:29.742986 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/950f0626-40d2-41c9-9701-cba50843c43e-kube-api-access-rvkbs" (OuterVolumeSpecName: "kube-api-access-rvkbs") pod "950f0626-40d2-41c9-9701-cba50843c43e" (UID: "950f0626-40d2-41c9-9701-cba50843c43e"). InnerVolumeSpecName "kube-api-access-rvkbs". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:47:29 crc kubenswrapper[5052]: I1203 11:47:29.743082 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8c2252c0-5e30-4e38-b1a1-af0242df6ac0-kube-api-access-dpgmf" (OuterVolumeSpecName: "kube-api-access-dpgmf") pod "8c2252c0-5e30-4e38-b1a1-af0242df6ac0" (UID: "8c2252c0-5e30-4e38-b1a1-af0242df6ac0"). InnerVolumeSpecName "kube-api-access-dpgmf". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:47:29 crc kubenswrapper[5052]: I1203 11:47:29.756688 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/keystone-779b679dc7-nvtdm"] Dec 03 11:47:29 crc kubenswrapper[5052]: I1203 11:47:29.763390 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:47:29 crc kubenswrapper[5052]: I1203 11:47:29.773902 5052 scope.go:117] "RemoveContainer" containerID="76c303acd28c23268bb01ed30bc5fa462afd9585a2872db7a6ce77014ebbad00" Dec 03 11:47:29 crc kubenswrapper[5052]: I1203 11:47:29.795227 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 11:47:29 crc kubenswrapper[5052]: I1203 11:47:29.803426 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/novacell0f13b-account-delete-g7m79" Dec 03 11:47:29 crc kubenswrapper[5052]: I1203 11:47:29.817331 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/novaapi5919-account-delete-qkvzd" Dec 03 11:47:29 crc kubenswrapper[5052]: I1203 11:47:29.818750 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 11:47:29 crc kubenswrapper[5052]: I1203 11:47:29.829209 5052 scope.go:117] "RemoveContainer" containerID="8c71cc229cea36e3361a6c384c5386d5cc260814f0041348a439548161af1249" Dec 03 11:47:29 crc kubenswrapper[5052]: E1203 11:47:29.829689 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8c71cc229cea36e3361a6c384c5386d5cc260814f0041348a439548161af1249\": container with ID starting with 8c71cc229cea36e3361a6c384c5386d5cc260814f0041348a439548161af1249 not found: ID does not exist" containerID="8c71cc229cea36e3361a6c384c5386d5cc260814f0041348a439548161af1249" Dec 03 11:47:29 crc kubenswrapper[5052]: I1203 11:47:29.829714 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8c71cc229cea36e3361a6c384c5386d5cc260814f0041348a439548161af1249"} err="failed to get container status \"8c71cc229cea36e3361a6c384c5386d5cc260814f0041348a439548161af1249\": rpc error: code = NotFound desc = could not find container \"8c71cc229cea36e3361a6c384c5386d5cc260814f0041348a439548161af1249\": container with ID starting with 8c71cc229cea36e3361a6c384c5386d5cc260814f0041348a439548161af1249 not found: ID does not exist" Dec 03 11:47:29 crc kubenswrapper[5052]: I1203 11:47:29.829734 5052 scope.go:117] "RemoveContainer" containerID="05471f2484133fbd796cb8f6c5d16314217c42ed6ab7d7d6511de8a201b11eb9" Dec 03 11:47:29 crc kubenswrapper[5052]: E1203 11:47:29.830164 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"05471f2484133fbd796cb8f6c5d16314217c42ed6ab7d7d6511de8a201b11eb9\": container with ID starting with 05471f2484133fbd796cb8f6c5d16314217c42ed6ab7d7d6511de8a201b11eb9 not found: ID does not exist" containerID="05471f2484133fbd796cb8f6c5d16314217c42ed6ab7d7d6511de8a201b11eb9" Dec 03 11:47:29 crc kubenswrapper[5052]: I1203 11:47:29.830201 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"05471f2484133fbd796cb8f6c5d16314217c42ed6ab7d7d6511de8a201b11eb9"} err="failed to get container status \"05471f2484133fbd796cb8f6c5d16314217c42ed6ab7d7d6511de8a201b11eb9\": rpc error: code = NotFound desc = could not find container \"05471f2484133fbd796cb8f6c5d16314217c42ed6ab7d7d6511de8a201b11eb9\": container with ID starting with 05471f2484133fbd796cb8f6c5d16314217c42ed6ab7d7d6511de8a201b11eb9 not found: ID does not exist" Dec 03 11:47:29 crc kubenswrapper[5052]: I1203 11:47:29.830228 5052 scope.go:117] "RemoveContainer" containerID="9e68066ce0ef8e9380353738e119a2afc7b38dcd3bd33c671f3778712cfb46e4" Dec 03 11:47:29 crc kubenswrapper[5052]: I1203 11:47:29.830647 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 11:47:29 crc kubenswrapper[5052]: E1203 11:47:29.831122 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9e68066ce0ef8e9380353738e119a2afc7b38dcd3bd33c671f3778712cfb46e4\": container with ID starting with 9e68066ce0ef8e9380353738e119a2afc7b38dcd3bd33c671f3778712cfb46e4 not found: ID does not exist" containerID="9e68066ce0ef8e9380353738e119a2afc7b38dcd3bd33c671f3778712cfb46e4" Dec 03 11:47:29 crc kubenswrapper[5052]: I1203 11:47:29.831202 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9e68066ce0ef8e9380353738e119a2afc7b38dcd3bd33c671f3778712cfb46e4"} err="failed to get container status \"9e68066ce0ef8e9380353738e119a2afc7b38dcd3bd33c671f3778712cfb46e4\": rpc error: code = NotFound desc = could not find container \"9e68066ce0ef8e9380353738e119a2afc7b38dcd3bd33c671f3778712cfb46e4\": container with ID starting with 9e68066ce0ef8e9380353738e119a2afc7b38dcd3bd33c671f3778712cfb46e4 not found: ID does not exist" Dec 03 11:47:29 crc kubenswrapper[5052]: I1203 11:47:29.831274 5052 scope.go:117] "RemoveContainer" containerID="76c303acd28c23268bb01ed30bc5fa462afd9585a2872db7a6ce77014ebbad00" Dec 03 11:47:29 crc kubenswrapper[5052]: E1203 11:47:29.831684 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"76c303acd28c23268bb01ed30bc5fa462afd9585a2872db7a6ce77014ebbad00\": container with ID starting with 76c303acd28c23268bb01ed30bc5fa462afd9585a2872db7a6ce77014ebbad00 not found: ID does not exist" containerID="76c303acd28c23268bb01ed30bc5fa462afd9585a2872db7a6ce77014ebbad00" Dec 03 11:47:29 crc kubenswrapper[5052]: I1203 11:47:29.831771 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"76c303acd28c23268bb01ed30bc5fa462afd9585a2872db7a6ce77014ebbad00"} err="failed to get container status \"76c303acd28c23268bb01ed30bc5fa462afd9585a2872db7a6ce77014ebbad00\": rpc error: code = NotFound desc = could not find container \"76c303acd28c23268bb01ed30bc5fa462afd9585a2872db7a6ce77014ebbad00\": container with ID starting with 76c303acd28c23268bb01ed30bc5fa462afd9585a2872db7a6ce77014ebbad00 not found: ID does not exist" Dec 03 11:47:29 crc kubenswrapper[5052]: I1203 11:47:29.831830 5052 scope.go:117] "RemoveContainer" containerID="81c366447116b43e07821471a9d38ae69fcdb20b1db58e745b91476f5acbbc2f" Dec 03 11:47:29 crc kubenswrapper[5052]: I1203 11:47:29.838776 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/d8de865a-0f6c-413d-8a37-3ec4d496c4bb-config-data\") pod \"d8de865a-0f6c-413d-8a37-3ec4d496c4bb\" (UID: \"d8de865a-0f6c-413d-8a37-3ec4d496c4bb\") " Dec 03 11:47:29 crc kubenswrapper[5052]: I1203 11:47:29.838808 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d8de865a-0f6c-413d-8a37-3ec4d496c4bb-combined-ca-bundle\") pod \"d8de865a-0f6c-413d-8a37-3ec4d496c4bb\" (UID: \"d8de865a-0f6c-413d-8a37-3ec4d496c4bb\") " Dec 03 11:47:29 crc kubenswrapper[5052]: I1203 11:47:29.838847 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/aa9a649b-9229-4cdd-8e27-8aedd48f4bbd-operator-scripts\") pod \"aa9a649b-9229-4cdd-8e27-8aedd48f4bbd\" (UID: \"aa9a649b-9229-4cdd-8e27-8aedd48f4bbd\") " Dec 03 11:47:29 crc kubenswrapper[5052]: I1203 11:47:29.838866 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/d8de865a-0f6c-413d-8a37-3ec4d496c4bb-config-data-custom\") pod \"d8de865a-0f6c-413d-8a37-3ec4d496c4bb\" (UID: \"d8de865a-0f6c-413d-8a37-3ec4d496c4bb\") " Dec 03 11:47:29 crc kubenswrapper[5052]: I1203 11:47:29.838921 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/39b9198e-e3d7-4f85-b6a2-55960bdda82a-combined-ca-bundle\") pod \"39b9198e-e3d7-4f85-b6a2-55960bdda82a\" (UID: \"39b9198e-e3d7-4f85-b6a2-55960bdda82a\") " Dec 03 11:47:29 crc kubenswrapper[5052]: I1203 11:47:29.838940 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-brvb8\" (UniqueName: \"kubernetes.io/projected/0b94b517-5f9b-485d-a109-0a11e15a3ad3-kube-api-access-brvb8\") pod \"0b94b517-5f9b-485d-a109-0a11e15a3ad3\" (UID: \"0b94b517-5f9b-485d-a109-0a11e15a3ad3\") " Dec 03 11:47:29 crc kubenswrapper[5052]: I1203 11:47:29.838969 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/d8de865a-0f6c-413d-8a37-3ec4d496c4bb-logs\") pod \"d8de865a-0f6c-413d-8a37-3ec4d496c4bb\" (UID: \"d8de865a-0f6c-413d-8a37-3ec4d496c4bb\") " Dec 03 11:47:29 crc kubenswrapper[5052]: I1203 11:47:29.839002 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/0b94b517-5f9b-485d-a109-0a11e15a3ad3-operator-scripts\") pod \"0b94b517-5f9b-485d-a109-0a11e15a3ad3\" (UID: \"0b94b517-5f9b-485d-a109-0a11e15a3ad3\") " Dec 03 11:47:29 crc kubenswrapper[5052]: I1203 11:47:29.839020 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8pptj\" (UniqueName: \"kubernetes.io/projected/39b9198e-e3d7-4f85-b6a2-55960bdda82a-kube-api-access-8pptj\") pod \"39b9198e-e3d7-4f85-b6a2-55960bdda82a\" (UID: \"39b9198e-e3d7-4f85-b6a2-55960bdda82a\") " Dec 03 11:47:29 crc kubenswrapper[5052]: I1203 11:47:29.839044 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bjktc\" (UniqueName: \"kubernetes.io/projected/d8de865a-0f6c-413d-8a37-3ec4d496c4bb-kube-api-access-bjktc\") pod \"d8de865a-0f6c-413d-8a37-3ec4d496c4bb\" (UID: \"d8de865a-0f6c-413d-8a37-3ec4d496c4bb\") " Dec 03 11:47:29 crc kubenswrapper[5052]: I1203 11:47:29.839108 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/39b9198e-e3d7-4f85-b6a2-55960bdda82a-config-data\") pod \"39b9198e-e3d7-4f85-b6a2-55960bdda82a\" (UID: \"39b9198e-e3d7-4f85-b6a2-55960bdda82a\") " Dec 03 11:47:29 crc kubenswrapper[5052]: I1203 11:47:29.839128 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l9m8g\" (UniqueName: \"kubernetes.io/projected/aa9a649b-9229-4cdd-8e27-8aedd48f4bbd-kube-api-access-l9m8g\") pod \"aa9a649b-9229-4cdd-8e27-8aedd48f4bbd\" (UID: \"aa9a649b-9229-4cdd-8e27-8aedd48f4bbd\") " Dec 03 11:47:29 crc kubenswrapper[5052]: I1203 11:47:29.839550 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/950f0626-40d2-41c9-9701-cba50843c43e-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:29 crc kubenswrapper[5052]: I1203 11:47:29.839563 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-dpgmf\" (UniqueName: \"kubernetes.io/projected/8c2252c0-5e30-4e38-b1a1-af0242df6ac0-kube-api-access-dpgmf\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:29 crc kubenswrapper[5052]: I1203 11:47:29.839573 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-rvkbs\" (UniqueName: \"kubernetes.io/projected/950f0626-40d2-41c9-9701-cba50843c43e-kube-api-access-rvkbs\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:29 crc kubenswrapper[5052]: I1203 11:47:29.839584 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/8c2252c0-5e30-4e38-b1a1-af0242df6ac0-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:29 crc kubenswrapper[5052]: I1203 11:47:29.840300 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d8de865a-0f6c-413d-8a37-3ec4d496c4bb-logs" (OuterVolumeSpecName: "logs") pod "d8de865a-0f6c-413d-8a37-3ec4d496c4bb" (UID: "d8de865a-0f6c-413d-8a37-3ec4d496c4bb"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:47:29 crc kubenswrapper[5052]: I1203 11:47:29.840417 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0b94b517-5f9b-485d-a109-0a11e15a3ad3-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "0b94b517-5f9b-485d-a109-0a11e15a3ad3" (UID: "0b94b517-5f9b-485d-a109-0a11e15a3ad3"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:47:29 crc kubenswrapper[5052]: I1203 11:47:29.840706 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 11:47:29 crc kubenswrapper[5052]: I1203 11:47:29.852320 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d8de865a-0f6c-413d-8a37-3ec4d496c4bb-config-data-custom" (OuterVolumeSpecName: "config-data-custom") pod "d8de865a-0f6c-413d-8a37-3ec4d496c4bb" (UID: "d8de865a-0f6c-413d-8a37-3ec4d496c4bb"). InnerVolumeSpecName "config-data-custom". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:47:29 crc kubenswrapper[5052]: I1203 11:47:29.859416 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aa9a649b-9229-4cdd-8e27-8aedd48f4bbd-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "aa9a649b-9229-4cdd-8e27-8aedd48f4bbd" (UID: "aa9a649b-9229-4cdd-8e27-8aedd48f4bbd"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:47:29 crc kubenswrapper[5052]: I1203 11:47:29.865100 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d8de865a-0f6c-413d-8a37-3ec4d496c4bb-kube-api-access-bjktc" (OuterVolumeSpecName: "kube-api-access-bjktc") pod "d8de865a-0f6c-413d-8a37-3ec4d496c4bb" (UID: "d8de865a-0f6c-413d-8a37-3ec4d496c4bb"). InnerVolumeSpecName "kube-api-access-bjktc". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:47:29 crc kubenswrapper[5052]: I1203 11:47:29.865195 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:47:29 crc kubenswrapper[5052]: I1203 11:47:29.869945 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/39b9198e-e3d7-4f85-b6a2-55960bdda82a-kube-api-access-8pptj" (OuterVolumeSpecName: "kube-api-access-8pptj") pod "39b9198e-e3d7-4f85-b6a2-55960bdda82a" (UID: "39b9198e-e3d7-4f85-b6a2-55960bdda82a"). InnerVolumeSpecName "kube-api-access-8pptj". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:47:29 crc kubenswrapper[5052]: I1203 11:47:29.871945 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0b94b517-5f9b-485d-a109-0a11e15a3ad3-kube-api-access-brvb8" (OuterVolumeSpecName: "kube-api-access-brvb8") pod "0b94b517-5f9b-485d-a109-0a11e15a3ad3" (UID: "0b94b517-5f9b-485d-a109-0a11e15a3ad3"). InnerVolumeSpecName "kube-api-access-brvb8". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:47:29 crc kubenswrapper[5052]: I1203 11:47:29.872575 5052 scope.go:117] "RemoveContainer" containerID="32cafa0afafbb84110820599defab0c52864ba6ace469cdde0c2e43031966639" Dec 03 11:47:29 crc kubenswrapper[5052]: I1203 11:47:29.873023 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aa9a649b-9229-4cdd-8e27-8aedd48f4bbd-kube-api-access-l9m8g" (OuterVolumeSpecName: "kube-api-access-l9m8g") pod "aa9a649b-9229-4cdd-8e27-8aedd48f4bbd" (UID: "aa9a649b-9229-4cdd-8e27-8aedd48f4bbd"). InnerVolumeSpecName "kube-api-access-l9m8g". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:47:29 crc kubenswrapper[5052]: I1203 11:47:29.877213 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:47:29 crc kubenswrapper[5052]: I1203 11:47:29.889546 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d8de865a-0f6c-413d-8a37-3ec4d496c4bb-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "d8de865a-0f6c-413d-8a37-3ec4d496c4bb" (UID: "d8de865a-0f6c-413d-8a37-3ec4d496c4bb"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:47:29 crc kubenswrapper[5052]: I1203 11:47:29.897393 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 11:47:29 crc kubenswrapper[5052]: I1203 11:47:29.914109 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 11:47:29 crc kubenswrapper[5052]: I1203 11:47:29.916402 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d8de865a-0f6c-413d-8a37-3ec4d496c4bb-config-data" (OuterVolumeSpecName: "config-data") pod "d8de865a-0f6c-413d-8a37-3ec4d496c4bb" (UID: "d8de865a-0f6c-413d-8a37-3ec4d496c4bb"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:47:29 crc kubenswrapper[5052]: I1203 11:47:29.916402 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/39b9198e-e3d7-4f85-b6a2-55960bdda82a-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "39b9198e-e3d7-4f85-b6a2-55960bdda82a" (UID: "39b9198e-e3d7-4f85-b6a2-55960bdda82a"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:47:29 crc kubenswrapper[5052]: I1203 11:47:29.926939 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 11:47:29 crc kubenswrapper[5052]: I1203 11:47:29.933572 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 11:47:29 crc kubenswrapper[5052]: I1203 11:47:29.940592 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c9h9d\" (UniqueName: \"kubernetes.io/projected/e5339997-33d5-4794-8fa6-042b63e9391e-kube-api-access-c9h9d\") pod \"e5339997-33d5-4794-8fa6-042b63e9391e\" (UID: \"e5339997-33d5-4794-8fa6-042b63e9391e\") " Dec 03 11:47:29 crc kubenswrapper[5052]: I1203 11:47:29.940678 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z8p4p\" (UniqueName: \"kubernetes.io/projected/2d3a3054-6c8e-4808-bd5d-01413970167c-kube-api-access-z8p4p\") pod \"2d3a3054-6c8e-4808-bd5d-01413970167c\" (UID: \"2d3a3054-6c8e-4808-bd5d-01413970167c\") " Dec 03 11:47:29 crc kubenswrapper[5052]: I1203 11:47:29.940723 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/2d3a3054-6c8e-4808-bd5d-01413970167c-operator-scripts\") pod \"2d3a3054-6c8e-4808-bd5d-01413970167c\" (UID: \"2d3a3054-6c8e-4808-bd5d-01413970167c\") " Dec 03 11:47:29 crc kubenswrapper[5052]: I1203 11:47:29.940795 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/e5339997-33d5-4794-8fa6-042b63e9391e-operator-scripts\") pod \"e5339997-33d5-4794-8fa6-042b63e9391e\" (UID: \"e5339997-33d5-4794-8fa6-042b63e9391e\") " Dec 03 11:47:29 crc kubenswrapper[5052]: I1203 11:47:29.941242 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/39b9198e-e3d7-4f85-b6a2-55960bdda82a-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:29 crc kubenswrapper[5052]: I1203 11:47:29.941258 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-brvb8\" (UniqueName: \"kubernetes.io/projected/0b94b517-5f9b-485d-a109-0a11e15a3ad3-kube-api-access-brvb8\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:29 crc kubenswrapper[5052]: I1203 11:47:29.941271 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/d8de865a-0f6c-413d-8a37-3ec4d496c4bb-logs\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:29 crc kubenswrapper[5052]: I1203 11:47:29.941283 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/0b94b517-5f9b-485d-a109-0a11e15a3ad3-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:29 crc kubenswrapper[5052]: I1203 11:47:29.941296 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-8pptj\" (UniqueName: \"kubernetes.io/projected/39b9198e-e3d7-4f85-b6a2-55960bdda82a-kube-api-access-8pptj\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:29 crc kubenswrapper[5052]: I1203 11:47:29.941308 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-bjktc\" (UniqueName: \"kubernetes.io/projected/d8de865a-0f6c-413d-8a37-3ec4d496c4bb-kube-api-access-bjktc\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:29 crc kubenswrapper[5052]: I1203 11:47:29.941319 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-l9m8g\" (UniqueName: \"kubernetes.io/projected/aa9a649b-9229-4cdd-8e27-8aedd48f4bbd-kube-api-access-l9m8g\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:29 crc kubenswrapper[5052]: I1203 11:47:29.941332 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/d8de865a-0f6c-413d-8a37-3ec4d496c4bb-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:29 crc kubenswrapper[5052]: I1203 11:47:29.941344 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d8de865a-0f6c-413d-8a37-3ec4d496c4bb-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:29 crc kubenswrapper[5052]: I1203 11:47:29.941355 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/d8de865a-0f6c-413d-8a37-3ec4d496c4bb-config-data-custom\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:29 crc kubenswrapper[5052]: I1203 11:47:29.941366 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/aa9a649b-9229-4cdd-8e27-8aedd48f4bbd-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:29 crc kubenswrapper[5052]: I1203 11:47:29.941813 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e5339997-33d5-4794-8fa6-042b63e9391e-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "e5339997-33d5-4794-8fa6-042b63e9391e" (UID: "e5339997-33d5-4794-8fa6-042b63e9391e"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:47:29 crc kubenswrapper[5052]: I1203 11:47:29.941904 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2d3a3054-6c8e-4808-bd5d-01413970167c-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "2d3a3054-6c8e-4808-bd5d-01413970167c" (UID: "2d3a3054-6c8e-4808-bd5d-01413970167c"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:47:29 crc kubenswrapper[5052]: I1203 11:47:29.945972 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/placement-689c6465d6-nsjrx"] Dec 03 11:47:29 crc kubenswrapper[5052]: I1203 11:47:29.983610 5052 scope.go:117] "RemoveContainer" containerID="a47db30ff0a5b853393f81265de91d0780b0ae16051c26ce22422a53ba83dc46" Dec 03 11:47:29 crc kubenswrapper[5052]: I1203 11:47:29.984253 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2d3a3054-6c8e-4808-bd5d-01413970167c-kube-api-access-z8p4p" (OuterVolumeSpecName: "kube-api-access-z8p4p") pod "2d3a3054-6c8e-4808-bd5d-01413970167c" (UID: "2d3a3054-6c8e-4808-bd5d-01413970167c"). InnerVolumeSpecName "kube-api-access-z8p4p". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:47:29 crc kubenswrapper[5052]: I1203 11:47:29.986933 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/39b9198e-e3d7-4f85-b6a2-55960bdda82a-config-data" (OuterVolumeSpecName: "config-data") pod "39b9198e-e3d7-4f85-b6a2-55960bdda82a" (UID: "39b9198e-e3d7-4f85-b6a2-55960bdda82a"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:47:29 crc kubenswrapper[5052]: I1203 11:47:29.995050 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e5339997-33d5-4794-8fa6-042b63e9391e-kube-api-access-c9h9d" (OuterVolumeSpecName: "kube-api-access-c9h9d") pod "e5339997-33d5-4794-8fa6-042b63e9391e" (UID: "e5339997-33d5-4794-8fa6-042b63e9391e"). InnerVolumeSpecName "kube-api-access-c9h9d". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:47:30 crc kubenswrapper[5052]: I1203 11:47:30.003809 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/placement-689c6465d6-nsjrx"] Dec 03 11:47:30 crc kubenswrapper[5052]: I1203 11:47:30.009795 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 11:47:30 crc kubenswrapper[5052]: I1203 11:47:30.014135 5052 scope.go:117] "RemoveContainer" containerID="8ac2dc5e5a71365dcb309d32fb3d721ec328645681e822f6169600deaef3d18d" Dec 03 11:47:30 crc kubenswrapper[5052]: I1203 11:47:30.016871 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 11:47:30 crc kubenswrapper[5052]: I1203 11:47:30.024040 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-api-7cfb847cb8-jn85t"] Dec 03 11:47:30 crc kubenswrapper[5052]: I1203 11:47:30.030829 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/barbican-api-7cfb847cb8-jn85t"] Dec 03 11:47:30 crc kubenswrapper[5052]: I1203 11:47:30.031666 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_ovn-northd-0_83a10afe-30d8-443d-a4f7-28784a2ea068/ovn-northd/0.log" Dec 03 11:47:30 crc kubenswrapper[5052]: I1203 11:47:30.031703 5052 generic.go:334] "Generic (PLEG): container finished" podID="83a10afe-30d8-443d-a4f7-28784a2ea068" containerID="3b43b29211f1c83741ff8d9a266e5e11a2e9560cc3168a7ef1db16f7622c61ba" exitCode=139 Dec 03 11:47:30 crc kubenswrapper[5052]: I1203 11:47:30.031754 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovn-northd-0" event={"ID":"83a10afe-30d8-443d-a4f7-28784a2ea068","Type":"ContainerDied","Data":"3b43b29211f1c83741ff8d9a266e5e11a2e9560cc3168a7ef1db16f7622c61ba"} Dec 03 11:47:30 crc kubenswrapper[5052]: I1203 11:47:30.033813 5052 generic.go:334] "Generic (PLEG): container finished" podID="e5339997-33d5-4794-8fa6-042b63e9391e" containerID="3943c83b52b8dca3321af840235a5573bed2a0ef796bf876d3af6fcba17e64ff" exitCode=1 Dec 03 11:47:30 crc kubenswrapper[5052]: I1203 11:47:30.033891 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/novacell0f13b-account-delete-g7m79" event={"ID":"e5339997-33d5-4794-8fa6-042b63e9391e","Type":"ContainerDied","Data":"3943c83b52b8dca3321af840235a5573bed2a0ef796bf876d3af6fcba17e64ff"} Dec 03 11:47:30 crc kubenswrapper[5052]: I1203 11:47:30.033926 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/novacell0f13b-account-delete-g7m79" event={"ID":"e5339997-33d5-4794-8fa6-042b63e9391e","Type":"ContainerDied","Data":"6d74fedfd0678d095a656c2ce967b2160548b1dbc9117678c8bb6f3fb9a65c37"} Dec 03 11:47:30 crc kubenswrapper[5052]: I1203 11:47:30.034018 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/novacell0f13b-account-delete-g7m79" Dec 03 11:47:30 crc kubenswrapper[5052]: I1203 11:47:30.042868 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/39b9198e-e3d7-4f85-b6a2-55960bdda82a-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:30 crc kubenswrapper[5052]: I1203 11:47:30.042905 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-c9h9d\" (UniqueName: \"kubernetes.io/projected/e5339997-33d5-4794-8fa6-042b63e9391e-kube-api-access-c9h9d\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:30 crc kubenswrapper[5052]: I1203 11:47:30.042919 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-z8p4p\" (UniqueName: \"kubernetes.io/projected/2d3a3054-6c8e-4808-bd5d-01413970167c-kube-api-access-z8p4p\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:30 crc kubenswrapper[5052]: I1203 11:47:30.042931 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/2d3a3054-6c8e-4808-bd5d-01413970167c-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:30 crc kubenswrapper[5052]: I1203 11:47:30.042942 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/e5339997-33d5-4794-8fa6-042b63e9391e-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:30 crc kubenswrapper[5052]: I1203 11:47:30.044030 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"39b9198e-e3d7-4f85-b6a2-55960bdda82a","Type":"ContainerDied","Data":"2de361109ed724167cdae870a30228df924104c25e4439c710241e2b780c2e64"} Dec 03 11:47:30 crc kubenswrapper[5052]: I1203 11:47:30.044050 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:47:30 crc kubenswrapper[5052]: I1203 11:47:30.048939 5052 scope.go:117] "RemoveContainer" containerID="10f606fe8fa7b691b8917f929fb59e6261d389781afc79a34d36abb996d45333" Dec 03 11:47:30 crc kubenswrapper[5052]: I1203 11:47:30.052161 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-keystone-listener-6794d844d6-f4tr7" Dec 03 11:47:30 crc kubenswrapper[5052]: I1203 11:47:30.052872 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-keystone-listener-6794d844d6-f4tr7" event={"ID":"d8de865a-0f6c-413d-8a37-3ec4d496c4bb","Type":"ContainerDied","Data":"3b79aa1079659ba91b2bfc402ce89d6f8dcb2f07c0b860dd97bccda698d668a8"} Dec 03 11:47:30 crc kubenswrapper[5052]: I1203 11:47:30.058372 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/novaapi5919-account-delete-qkvzd" Dec 03 11:47:30 crc kubenswrapper[5052]: I1203 11:47:30.058182 5052 generic.go:334] "Generic (PLEG): container finished" podID="2d3a3054-6c8e-4808-bd5d-01413970167c" containerID="ae6315a718283526ad43a5d86df7f92e3baca5b6bd2a9ae8afdd33e574384a43" exitCode=1 Dec 03 11:47:30 crc kubenswrapper[5052]: I1203 11:47:30.058466 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement9a4f-account-delete-f6sjd" Dec 03 11:47:30 crc kubenswrapper[5052]: I1203 11:47:30.058470 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/novaapi5919-account-delete-qkvzd" event={"ID":"2d3a3054-6c8e-4808-bd5d-01413970167c","Type":"ContainerDied","Data":"ae6315a718283526ad43a5d86df7f92e3baca5b6bd2a9ae8afdd33e574384a43"} Dec 03 11:47:30 crc kubenswrapper[5052]: I1203 11:47:30.058554 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/novaapi5919-account-delete-qkvzd" event={"ID":"2d3a3054-6c8e-4808-bd5d-01413970167c","Type":"ContainerDied","Data":"a32e8eecdf37a0533359f0cca96b134dd958c5ee69c6850d73b655fa60a18f8d"} Dec 03 11:47:30 crc kubenswrapper[5052]: I1203 11:47:30.058657 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance3554-account-delete-8ghdx" Dec 03 11:47:30 crc kubenswrapper[5052]: I1203 11:47:30.058786 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron18a7-account-delete-x9lqd" Dec 03 11:47:30 crc kubenswrapper[5052]: I1203 11:47:30.059416 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder1bf0-account-delete-hs8j6" Dec 03 11:47:30 crc kubenswrapper[5052]: I1203 11:47:30.059416 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican5976-account-delete-q6j5s" Dec 03 11:47:30 crc kubenswrapper[5052]: I1203 11:47:30.067358 5052 scope.go:117] "RemoveContainer" containerID="95b8a0e92f2ae27cf80dcce2b2af098739e919963ebae7c22edd8d9c46c89ddc" Dec 03 11:47:30 crc kubenswrapper[5052]: I1203 11:47:30.087023 5052 scope.go:117] "RemoveContainer" containerID="08aececa6f47b8deb531a78d1231ed436b94566bc55dd432fa69a3f7a05d3ebf" Dec 03 11:47:30 crc kubenswrapper[5052]: I1203 11:47:30.112699 5052 scope.go:117] "RemoveContainer" containerID="5592513a2c756f332087769178ec0527d69dee3863e8ebc7da5e5d5ac9fefa4f" Dec 03 11:47:30 crc kubenswrapper[5052]: I1203 11:47:30.144819 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/novacell0f13b-account-delete-g7m79"] Dec 03 11:47:30 crc kubenswrapper[5052]: I1203 11:47:30.147785 5052 scope.go:117] "RemoveContainer" containerID="1714a77b6dd6ea9a3fe18b86a4ab32f16a2525fd31478012ffe19746b1260454" Dec 03 11:47:30 crc kubenswrapper[5052]: I1203 11:47:30.157068 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/novacell0f13b-account-delete-g7m79"] Dec 03 11:47:30 crc kubenswrapper[5052]: I1203 11:47:30.163172 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican5976-account-delete-q6j5s"] Dec 03 11:47:30 crc kubenswrapper[5052]: I1203 11:47:30.171252 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/barbican5976-account-delete-q6j5s"] Dec 03 11:47:30 crc kubenswrapper[5052]: I1203 11:47:30.171559 5052 scope.go:117] "RemoveContainer" containerID="2be919af42c62148fa0417519c4eea762d9886b79020bd92b5f10692d1f30f31" Dec 03 11:47:30 crc kubenswrapper[5052]: I1203 11:47:30.182890 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder1bf0-account-delete-hs8j6"] Dec 03 11:47:30 crc kubenswrapper[5052]: I1203 11:47:30.191126 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/cinder1bf0-account-delete-hs8j6"] Dec 03 11:47:30 crc kubenswrapper[5052]: I1203 11:47:30.197023 5052 scope.go:117] "RemoveContainer" containerID="1e380bc4c681225a08515748fc7e47b1e5ab3aca3455957a89e3055115aa44a9" Dec 03 11:47:30 crc kubenswrapper[5052]: I1203 11:47:30.199703 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 11:47:30 crc kubenswrapper[5052]: I1203 11:47:30.207954 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 11:47:30 crc kubenswrapper[5052]: I1203 11:47:30.215677 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/neutron18a7-account-delete-x9lqd"] Dec 03 11:47:30 crc kubenswrapper[5052]: I1203 11:47:30.219202 5052 scope.go:117] "RemoveContainer" containerID="25b77fa7d98d8777337c1b92110af89807efa838eb759959d62314939c420ccf" Dec 03 11:47:30 crc kubenswrapper[5052]: I1203 11:47:30.223429 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/neutron18a7-account-delete-x9lqd"] Dec 03 11:47:30 crc kubenswrapper[5052]: I1203 11:47:30.230723 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance3554-account-delete-8ghdx"] Dec 03 11:47:30 crc kubenswrapper[5052]: I1203 11:47:30.237520 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/glance3554-account-delete-8ghdx"] Dec 03 11:47:30 crc kubenswrapper[5052]: I1203 11:47:30.244360 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/placement9a4f-account-delete-f6sjd"] Dec 03 11:47:30 crc kubenswrapper[5052]: I1203 11:47:30.250794 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/placement9a4f-account-delete-f6sjd"] Dec 03 11:47:30 crc kubenswrapper[5052]: I1203 11:47:30.259294 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/novaapi5919-account-delete-qkvzd"] Dec 03 11:47:30 crc kubenswrapper[5052]: I1203 11:47:30.263079 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/novaapi5919-account-delete-qkvzd"] Dec 03 11:47:30 crc kubenswrapper[5052]: I1203 11:47:30.268896 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-keystone-listener-6794d844d6-f4tr7"] Dec 03 11:47:30 crc kubenswrapper[5052]: I1203 11:47:30.275385 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/barbican-keystone-listener-6794d844d6-f4tr7"] Dec 03 11:47:30 crc kubenswrapper[5052]: I1203 11:47:30.284313 5052 scope.go:117] "RemoveContainer" containerID="fa0c2ba3f86bf1385c8a82da17792deaa7cbb1c9c5250a02c070da47b2664562" Dec 03 11:47:30 crc kubenswrapper[5052]: I1203 11:47:30.311181 5052 scope.go:117] "RemoveContainer" containerID="b39f7b31c335958d6613651cea9f3ccbfee7c25b7c7641048321f3e460ee0193" Dec 03 11:47:30 crc kubenswrapper[5052]: I1203 11:47:30.340418 5052 scope.go:117] "RemoveContainer" containerID="4568b2e2e54107935eb412f6e606a9e5b4fd360b7f1d0fae94b2b383fd9205d0" Dec 03 11:47:30 crc kubenswrapper[5052]: E1203 11:47:30.349546 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/combined-ca-bundle: secret "combined-ca-bundle" not found Dec 03 11:47:30 crc kubenswrapper[5052]: E1203 11:47:30.349653 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f9b208b-086f-4671-8c3f-b3a275740ad7-combined-ca-bundle podName:6f9b208b-086f-4671-8c3f-b3a275740ad7 nodeName:}" failed. No retries permitted until 2025-12-03 11:47:34.34963353 +0000 UTC m=+1644.937875294 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "combined-ca-bundle" (UniqueName: "kubernetes.io/secret/6f9b208b-086f-4671-8c3f-b3a275740ad7-combined-ca-bundle") pod "memcached-0" (UID: "6f9b208b-086f-4671-8c3f-b3a275740ad7") : secret "combined-ca-bundle" not found Dec 03 11:47:30 crc kubenswrapper[5052]: E1203 11:47:30.350179 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-memcached-svc: secret "cert-memcached-svc" not found Dec 03 11:47:30 crc kubenswrapper[5052]: E1203 11:47:30.350216 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f9b208b-086f-4671-8c3f-b3a275740ad7-memcached-tls-certs podName:6f9b208b-086f-4671-8c3f-b3a275740ad7 nodeName:}" failed. No retries permitted until 2025-12-03 11:47:34.350205187 +0000 UTC m=+1644.938446951 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "memcached-tls-certs" (UniqueName: "kubernetes.io/secret/6f9b208b-086f-4671-8c3f-b3a275740ad7-memcached-tls-certs") pod "memcached-0" (UID: "6f9b208b-086f-4671-8c3f-b3a275740ad7") : secret "cert-memcached-svc" not found Dec 03 11:47:30 crc kubenswrapper[5052]: I1203 11:47:30.392998 5052 scope.go:117] "RemoveContainer" containerID="3943c83b52b8dca3321af840235a5573bed2a0ef796bf876d3af6fcba17e64ff" Dec 03 11:47:30 crc kubenswrapper[5052]: I1203 11:47:30.426114 5052 scope.go:117] "RemoveContainer" containerID="e8bef7322f8817aeda29d79f6179dd161790705405191177d57b564a93045852" Dec 03 11:47:30 crc kubenswrapper[5052]: I1203 11:47:30.488747 5052 scope.go:117] "RemoveContainer" containerID="3943c83b52b8dca3321af840235a5573bed2a0ef796bf876d3af6fcba17e64ff" Dec 03 11:47:30 crc kubenswrapper[5052]: E1203 11:47:30.489125 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3943c83b52b8dca3321af840235a5573bed2a0ef796bf876d3af6fcba17e64ff\": container with ID starting with 3943c83b52b8dca3321af840235a5573bed2a0ef796bf876d3af6fcba17e64ff not found: ID does not exist" containerID="3943c83b52b8dca3321af840235a5573bed2a0ef796bf876d3af6fcba17e64ff" Dec 03 11:47:30 crc kubenswrapper[5052]: I1203 11:47:30.489168 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3943c83b52b8dca3321af840235a5573bed2a0ef796bf876d3af6fcba17e64ff"} err="failed to get container status \"3943c83b52b8dca3321af840235a5573bed2a0ef796bf876d3af6fcba17e64ff\": rpc error: code = NotFound desc = could not find container \"3943c83b52b8dca3321af840235a5573bed2a0ef796bf876d3af6fcba17e64ff\": container with ID starting with 3943c83b52b8dca3321af840235a5573bed2a0ef796bf876d3af6fcba17e64ff not found: ID does not exist" Dec 03 11:47:30 crc kubenswrapper[5052]: I1203 11:47:30.489202 5052 scope.go:117] "RemoveContainer" containerID="e8bef7322f8817aeda29d79f6179dd161790705405191177d57b564a93045852" Dec 03 11:47:30 crc kubenswrapper[5052]: E1203 11:47:30.489479 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e8bef7322f8817aeda29d79f6179dd161790705405191177d57b564a93045852\": container with ID starting with e8bef7322f8817aeda29d79f6179dd161790705405191177d57b564a93045852 not found: ID does not exist" containerID="e8bef7322f8817aeda29d79f6179dd161790705405191177d57b564a93045852" Dec 03 11:47:30 crc kubenswrapper[5052]: I1203 11:47:30.489509 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e8bef7322f8817aeda29d79f6179dd161790705405191177d57b564a93045852"} err="failed to get container status \"e8bef7322f8817aeda29d79f6179dd161790705405191177d57b564a93045852\": rpc error: code = NotFound desc = could not find container \"e8bef7322f8817aeda29d79f6179dd161790705405191177d57b564a93045852\": container with ID starting with e8bef7322f8817aeda29d79f6179dd161790705405191177d57b564a93045852 not found: ID does not exist" Dec 03 11:47:30 crc kubenswrapper[5052]: I1203 11:47:30.489643 5052 scope.go:117] "RemoveContainer" containerID="1dbdda8905493662bc20bc18fcae5f54dd4929f65dec1b16df59f03dbfd76d8a" Dec 03 11:47:30 crc kubenswrapper[5052]: I1203 11:47:30.558500 5052 scope.go:117] "RemoveContainer" containerID="def922586d645f9987c75d4862d880ed855e05523f69e809e4d0637bfe42a9d8" Dec 03 11:47:30 crc kubenswrapper[5052]: I1203 11:47:30.585984 5052 scope.go:117] "RemoveContainer" containerID="775a6cb46d400df5368627d0a86f59d32f62f17aac77b3a75fe7d8c2928b8362" Dec 03 11:47:30 crc kubenswrapper[5052]: I1203 11:47:30.615307 5052 scope.go:117] "RemoveContainer" containerID="ae6315a718283526ad43a5d86df7f92e3baca5b6bd2a9ae8afdd33e574384a43" Dec 03 11:47:30 crc kubenswrapper[5052]: I1203 11:47:30.643987 5052 scope.go:117] "RemoveContainer" containerID="c9d59899fe21862fbe05cb4f9b40af488d9a06ef3ce0bca59d2b5c6edc68c2cd" Dec 03 11:47:30 crc kubenswrapper[5052]: I1203 11:47:30.720525 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_ovn-northd-0_83a10afe-30d8-443d-a4f7-28784a2ea068/ovn-northd/0.log" Dec 03 11:47:30 crc kubenswrapper[5052]: I1203 11:47:30.720606 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 11:47:30 crc kubenswrapper[5052]: I1203 11:47:30.738119 5052 scope.go:117] "RemoveContainer" containerID="ae6315a718283526ad43a5d86df7f92e3baca5b6bd2a9ae8afdd33e574384a43" Dec 03 11:47:30 crc kubenswrapper[5052]: E1203 11:47:30.738641 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ae6315a718283526ad43a5d86df7f92e3baca5b6bd2a9ae8afdd33e574384a43\": container with ID starting with ae6315a718283526ad43a5d86df7f92e3baca5b6bd2a9ae8afdd33e574384a43 not found: ID does not exist" containerID="ae6315a718283526ad43a5d86df7f92e3baca5b6bd2a9ae8afdd33e574384a43" Dec 03 11:47:30 crc kubenswrapper[5052]: I1203 11:47:30.738687 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ae6315a718283526ad43a5d86df7f92e3baca5b6bd2a9ae8afdd33e574384a43"} err="failed to get container status \"ae6315a718283526ad43a5d86df7f92e3baca5b6bd2a9ae8afdd33e574384a43\": rpc error: code = NotFound desc = could not find container \"ae6315a718283526ad43a5d86df7f92e3baca5b6bd2a9ae8afdd33e574384a43\": container with ID starting with ae6315a718283526ad43a5d86df7f92e3baca5b6bd2a9ae8afdd33e574384a43 not found: ID does not exist" Dec 03 11:47:30 crc kubenswrapper[5052]: I1203 11:47:30.738714 5052 scope.go:117] "RemoveContainer" containerID="c9d59899fe21862fbe05cb4f9b40af488d9a06ef3ce0bca59d2b5c6edc68c2cd" Dec 03 11:47:30 crc kubenswrapper[5052]: E1203 11:47:30.739066 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c9d59899fe21862fbe05cb4f9b40af488d9a06ef3ce0bca59d2b5c6edc68c2cd\": container with ID starting with c9d59899fe21862fbe05cb4f9b40af488d9a06ef3ce0bca59d2b5c6edc68c2cd not found: ID does not exist" containerID="c9d59899fe21862fbe05cb4f9b40af488d9a06ef3ce0bca59d2b5c6edc68c2cd" Dec 03 11:47:30 crc kubenswrapper[5052]: I1203 11:47:30.739209 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c9d59899fe21862fbe05cb4f9b40af488d9a06ef3ce0bca59d2b5c6edc68c2cd"} err="failed to get container status \"c9d59899fe21862fbe05cb4f9b40af488d9a06ef3ce0bca59d2b5c6edc68c2cd\": rpc error: code = NotFound desc = could not find container \"c9d59899fe21862fbe05cb4f9b40af488d9a06ef3ce0bca59d2b5c6edc68c2cd\": container with ID starting with c9d59899fe21862fbe05cb4f9b40af488d9a06ef3ce0bca59d2b5c6edc68c2cd not found: ID does not exist" Dec 03 11:47:30 crc kubenswrapper[5052]: I1203 11:47:30.874726 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xh5s8\" (UniqueName: \"kubernetes.io/projected/83a10afe-30d8-443d-a4f7-28784a2ea068-kube-api-access-xh5s8\") pod \"83a10afe-30d8-443d-a4f7-28784a2ea068\" (UID: \"83a10afe-30d8-443d-a4f7-28784a2ea068\") " Dec 03 11:47:30 crc kubenswrapper[5052]: I1203 11:47:30.874798 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovn-northd-tls-certs\" (UniqueName: \"kubernetes.io/secret/83a10afe-30d8-443d-a4f7-28784a2ea068-ovn-northd-tls-certs\") pod \"83a10afe-30d8-443d-a4f7-28784a2ea068\" (UID: \"83a10afe-30d8-443d-a4f7-28784a2ea068\") " Dec 03 11:47:30 crc kubenswrapper[5052]: I1203 11:47:30.874839 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/83a10afe-30d8-443d-a4f7-28784a2ea068-config\") pod \"83a10afe-30d8-443d-a4f7-28784a2ea068\" (UID: \"83a10afe-30d8-443d-a4f7-28784a2ea068\") " Dec 03 11:47:30 crc kubenswrapper[5052]: I1203 11:47:30.874880 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovn-rundir\" (UniqueName: \"kubernetes.io/empty-dir/83a10afe-30d8-443d-a4f7-28784a2ea068-ovn-rundir\") pod \"83a10afe-30d8-443d-a4f7-28784a2ea068\" (UID: \"83a10afe-30d8-443d-a4f7-28784a2ea068\") " Dec 03 11:47:30 crc kubenswrapper[5052]: I1203 11:47:30.874933 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/83a10afe-30d8-443d-a4f7-28784a2ea068-combined-ca-bundle\") pod \"83a10afe-30d8-443d-a4f7-28784a2ea068\" (UID: \"83a10afe-30d8-443d-a4f7-28784a2ea068\") " Dec 03 11:47:30 crc kubenswrapper[5052]: I1203 11:47:30.874989 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/83a10afe-30d8-443d-a4f7-28784a2ea068-metrics-certs-tls-certs\") pod \"83a10afe-30d8-443d-a4f7-28784a2ea068\" (UID: \"83a10afe-30d8-443d-a4f7-28784a2ea068\") " Dec 03 11:47:30 crc kubenswrapper[5052]: I1203 11:47:30.875023 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/83a10afe-30d8-443d-a4f7-28784a2ea068-scripts\") pod \"83a10afe-30d8-443d-a4f7-28784a2ea068\" (UID: \"83a10afe-30d8-443d-a4f7-28784a2ea068\") " Dec 03 11:47:30 crc kubenswrapper[5052]: I1203 11:47:30.875703 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/83a10afe-30d8-443d-a4f7-28784a2ea068-config" (OuterVolumeSpecName: "config") pod "83a10afe-30d8-443d-a4f7-28784a2ea068" (UID: "83a10afe-30d8-443d-a4f7-28784a2ea068"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:47:30 crc kubenswrapper[5052]: I1203 11:47:30.875980 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83a10afe-30d8-443d-a4f7-28784a2ea068-ovn-rundir" (OuterVolumeSpecName: "ovn-rundir") pod "83a10afe-30d8-443d-a4f7-28784a2ea068" (UID: "83a10afe-30d8-443d-a4f7-28784a2ea068"). InnerVolumeSpecName "ovn-rundir". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:47:30 crc kubenswrapper[5052]: I1203 11:47:30.876296 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/83a10afe-30d8-443d-a4f7-28784a2ea068-scripts" (OuterVolumeSpecName: "scripts") pod "83a10afe-30d8-443d-a4f7-28784a2ea068" (UID: "83a10afe-30d8-443d-a4f7-28784a2ea068"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:47:30 crc kubenswrapper[5052]: I1203 11:47:30.880946 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/83a10afe-30d8-443d-a4f7-28784a2ea068-kube-api-access-xh5s8" (OuterVolumeSpecName: "kube-api-access-xh5s8") pod "83a10afe-30d8-443d-a4f7-28784a2ea068" (UID: "83a10afe-30d8-443d-a4f7-28784a2ea068"). InnerVolumeSpecName "kube-api-access-xh5s8". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:47:30 crc kubenswrapper[5052]: I1203 11:47:30.897953 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/83a10afe-30d8-443d-a4f7-28784a2ea068-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "83a10afe-30d8-443d-a4f7-28784a2ea068" (UID: "83a10afe-30d8-443d-a4f7-28784a2ea068"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:47:30 crc kubenswrapper[5052]: I1203 11:47:30.937634 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/83a10afe-30d8-443d-a4f7-28784a2ea068-ovn-northd-tls-certs" (OuterVolumeSpecName: "ovn-northd-tls-certs") pod "83a10afe-30d8-443d-a4f7-28784a2ea068" (UID: "83a10afe-30d8-443d-a4f7-28784a2ea068"). InnerVolumeSpecName "ovn-northd-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:47:30 crc kubenswrapper[5052]: I1203 11:47:30.942071 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/83a10afe-30d8-443d-a4f7-28784a2ea068-metrics-certs-tls-certs" (OuterVolumeSpecName: "metrics-certs-tls-certs") pod "83a10afe-30d8-443d-a4f7-28784a2ea068" (UID: "83a10afe-30d8-443d-a4f7-28784a2ea068"). InnerVolumeSpecName "metrics-certs-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:47:30 crc kubenswrapper[5052]: I1203 11:47:30.977389 5052 reconciler_common.go:293] "Volume detached for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/83a10afe-30d8-443d-a4f7-28784a2ea068-metrics-certs-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:30 crc kubenswrapper[5052]: I1203 11:47:30.977429 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/83a10afe-30d8-443d-a4f7-28784a2ea068-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:30 crc kubenswrapper[5052]: I1203 11:47:30.977442 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-xh5s8\" (UniqueName: \"kubernetes.io/projected/83a10afe-30d8-443d-a4f7-28784a2ea068-kube-api-access-xh5s8\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:30 crc kubenswrapper[5052]: I1203 11:47:30.977455 5052 reconciler_common.go:293] "Volume detached for volume \"ovn-northd-tls-certs\" (UniqueName: \"kubernetes.io/secret/83a10afe-30d8-443d-a4f7-28784a2ea068-ovn-northd-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:30 crc kubenswrapper[5052]: I1203 11:47:30.977467 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/83a10afe-30d8-443d-a4f7-28784a2ea068-config\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:30 crc kubenswrapper[5052]: I1203 11:47:30.977479 5052 reconciler_common.go:293] "Volume detached for volume \"ovn-rundir\" (UniqueName: \"kubernetes.io/empty-dir/83a10afe-30d8-443d-a4f7-28784a2ea068-ovn-rundir\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:30 crc kubenswrapper[5052]: I1203 11:47:30.977491 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/83a10afe-30d8-443d-a4f7-28784a2ea068-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:31 crc kubenswrapper[5052]: I1203 11:47:31.073098 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovn-northd-0" event={"ID":"83a10afe-30d8-443d-a4f7-28784a2ea068","Type":"ContainerDied","Data":"4eb65027991ca77618d30f61f575326274e9222f67ae812b47b2bc67b410437c"} Dec 03 11:47:31 crc kubenswrapper[5052]: I1203 11:47:31.073151 5052 scope.go:117] "RemoveContainer" containerID="af748c115d51d513b98829b5044d30038362360180988763c14bac6e6886f163" Dec 03 11:47:31 crc kubenswrapper[5052]: I1203 11:47:31.073393 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 11:47:31 crc kubenswrapper[5052]: I1203 11:47:31.096973 5052 generic.go:334] "Generic (PLEG): container finished" podID="d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0" containerID="f1ccb0c2e8810415edac670e94d35e316eb9d118a2ee2cb2d84f782419550625" exitCode=0 Dec 03 11:47:31 crc kubenswrapper[5052]: I1203 11:47:31.097033 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-server-0" event={"ID":"d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0","Type":"ContainerDied","Data":"f1ccb0c2e8810415edac670e94d35e316eb9d118a2ee2cb2d84f782419550625"} Dec 03 11:47:31 crc kubenswrapper[5052]: I1203 11:47:31.137050 5052 scope.go:117] "RemoveContainer" containerID="3b43b29211f1c83741ff8d9a266e5e11a2e9560cc3168a7ef1db16f7622c61ba" Dec 03 11:47:31 crc kubenswrapper[5052]: I1203 11:47:31.144151 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ovn-northd-0"] Dec 03 11:47:31 crc kubenswrapper[5052]: I1203 11:47:31.147844 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ovn-northd-0"] Dec 03 11:47:31 crc kubenswrapper[5052]: I1203 11:47:31.361984 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 11:47:31 crc kubenswrapper[5052]: I1203 11:47:31.493830 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"plugins-conf\" (UniqueName: \"kubernetes.io/configmap/d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0-plugins-conf\") pod \"d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0\" (UID: \"d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0\") " Dec 03 11:47:31 crc kubenswrapper[5052]: I1203 11:47:31.493898 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"erlang-cookie-secret\" (UniqueName: \"kubernetes.io/secret/d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0-erlang-cookie-secret\") pod \"d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0\" (UID: \"d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0\") " Dec 03 11:47:31 crc kubenswrapper[5052]: I1203 11:47:31.493942 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"rabbitmq-erlang-cookie\" (UniqueName: \"kubernetes.io/empty-dir/d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0-rabbitmq-erlang-cookie\") pod \"d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0\" (UID: \"d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0\") " Dec 03 11:47:31 crc kubenswrapper[5052]: I1203 11:47:31.493983 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"server-conf\" (UniqueName: \"kubernetes.io/configmap/d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0-server-conf\") pod \"d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0\" (UID: \"d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0\") " Dec 03 11:47:31 crc kubenswrapper[5052]: I1203 11:47:31.494021 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"pod-info\" (UniqueName: \"kubernetes.io/downward-api/d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0-pod-info\") pod \"d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0\" (UID: \"d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0\") " Dec 03 11:47:31 crc kubenswrapper[5052]: I1203 11:47:31.494041 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"rabbitmq-confd\" (UniqueName: \"kubernetes.io/projected/d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0-rabbitmq-confd\") pod \"d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0\" (UID: \"d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0\") " Dec 03 11:47:31 crc kubenswrapper[5052]: I1203 11:47:31.494069 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"rabbitmq-tls\" (UniqueName: \"kubernetes.io/projected/d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0-rabbitmq-tls\") pod \"d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0\" (UID: \"d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0\") " Dec 03 11:47:31 crc kubenswrapper[5052]: I1203 11:47:31.494087 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"rabbitmq-plugins\" (UniqueName: \"kubernetes.io/empty-dir/d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0-rabbitmq-plugins\") pod \"d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0\" (UID: \"d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0\") " Dec 03 11:47:31 crc kubenswrapper[5052]: I1203 11:47:31.494106 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"persistence\" (UniqueName: \"kubernetes.io/local-volume/local-storage07-crc\") pod \"d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0\" (UID: \"d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0\") " Dec 03 11:47:31 crc kubenswrapper[5052]: I1203 11:47:31.494130 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pz65r\" (UniqueName: \"kubernetes.io/projected/d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0-kube-api-access-pz65r\") pod \"d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0\" (UID: \"d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0\") " Dec 03 11:47:31 crc kubenswrapper[5052]: I1203 11:47:31.494160 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0-config-data\") pod \"d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0\" (UID: \"d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0\") " Dec 03 11:47:31 crc kubenswrapper[5052]: I1203 11:47:31.496875 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0-rabbitmq-plugins" (OuterVolumeSpecName: "rabbitmq-plugins") pod "d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0" (UID: "d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0"). InnerVolumeSpecName "rabbitmq-plugins". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:47:31 crc kubenswrapper[5052]: I1203 11:47:31.497811 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0-plugins-conf" (OuterVolumeSpecName: "plugins-conf") pod "d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0" (UID: "d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0"). InnerVolumeSpecName "plugins-conf". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:47:31 crc kubenswrapper[5052]: I1203 11:47:31.498464 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage07-crc" (OuterVolumeSpecName: "persistence") pod "d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0" (UID: "d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0"). InnerVolumeSpecName "local-storage07-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 11:47:31 crc kubenswrapper[5052]: I1203 11:47:31.499243 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0-rabbitmq-erlang-cookie" (OuterVolumeSpecName: "rabbitmq-erlang-cookie") pod "d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0" (UID: "d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0"). InnerVolumeSpecName "rabbitmq-erlang-cookie". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:47:31 crc kubenswrapper[5052]: I1203 11:47:31.500707 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0-kube-api-access-pz65r" (OuterVolumeSpecName: "kube-api-access-pz65r") pod "d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0" (UID: "d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0"). InnerVolumeSpecName "kube-api-access-pz65r". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:47:31 crc kubenswrapper[5052]: I1203 11:47:31.501044 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0-erlang-cookie-secret" (OuterVolumeSpecName: "erlang-cookie-secret") pod "d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0" (UID: "d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0"). InnerVolumeSpecName "erlang-cookie-secret". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:47:31 crc kubenswrapper[5052]: I1203 11:47:31.501097 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0-rabbitmq-tls" (OuterVolumeSpecName: "rabbitmq-tls") pod "d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0" (UID: "d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0"). InnerVolumeSpecName "rabbitmq-tls". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:47:31 crc kubenswrapper[5052]: I1203 11:47:31.502091 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0-pod-info" (OuterVolumeSpecName: "pod-info") pod "d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0" (UID: "d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0"). InnerVolumeSpecName "pod-info". PluginName "kubernetes.io/downward-api", VolumeGidValue "" Dec 03 11:47:31 crc kubenswrapper[5052]: I1203 11:47:31.517295 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0-config-data" (OuterVolumeSpecName: "config-data") pod "d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0" (UID: "d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:47:31 crc kubenswrapper[5052]: I1203 11:47:31.562544 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0-server-conf" (OuterVolumeSpecName: "server-conf") pod "d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0" (UID: "d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0"). InnerVolumeSpecName "server-conf". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:47:31 crc kubenswrapper[5052]: I1203 11:47:31.582438 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0-rabbitmq-confd" (OuterVolumeSpecName: "rabbitmq-confd") pod "d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0" (UID: "d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0"). InnerVolumeSpecName "rabbitmq-confd". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:47:31 crc kubenswrapper[5052]: I1203 11:47:31.595575 5052 reconciler_common.go:293] "Volume detached for volume \"plugins-conf\" (UniqueName: \"kubernetes.io/configmap/d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0-plugins-conf\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:31 crc kubenswrapper[5052]: I1203 11:47:31.595610 5052 reconciler_common.go:293] "Volume detached for volume \"erlang-cookie-secret\" (UniqueName: \"kubernetes.io/secret/d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0-erlang-cookie-secret\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:31 crc kubenswrapper[5052]: I1203 11:47:31.595623 5052 reconciler_common.go:293] "Volume detached for volume \"rabbitmq-erlang-cookie\" (UniqueName: \"kubernetes.io/empty-dir/d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0-rabbitmq-erlang-cookie\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:31 crc kubenswrapper[5052]: I1203 11:47:31.595636 5052 reconciler_common.go:293] "Volume detached for volume \"server-conf\" (UniqueName: \"kubernetes.io/configmap/d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0-server-conf\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:31 crc kubenswrapper[5052]: I1203 11:47:31.595725 5052 reconciler_common.go:293] "Volume detached for volume \"pod-info\" (UniqueName: \"kubernetes.io/downward-api/d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0-pod-info\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:31 crc kubenswrapper[5052]: I1203 11:47:31.595739 5052 reconciler_common.go:293] "Volume detached for volume \"rabbitmq-confd\" (UniqueName: \"kubernetes.io/projected/d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0-rabbitmq-confd\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:31 crc kubenswrapper[5052]: I1203 11:47:31.595750 5052 reconciler_common.go:293] "Volume detached for volume \"rabbitmq-tls\" (UniqueName: \"kubernetes.io/projected/d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0-rabbitmq-tls\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:31 crc kubenswrapper[5052]: I1203 11:47:31.595773 5052 reconciler_common.go:293] "Volume detached for volume \"rabbitmq-plugins\" (UniqueName: \"kubernetes.io/empty-dir/d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0-rabbitmq-plugins\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:31 crc kubenswrapper[5052]: I1203 11:47:31.595800 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage07-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage07-crc\") on node \"crc\" " Dec 03 11:47:31 crc kubenswrapper[5052]: I1203 11:47:31.595814 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-pz65r\" (UniqueName: \"kubernetes.io/projected/d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0-kube-api-access-pz65r\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:31 crc kubenswrapper[5052]: I1203 11:47:31.595826 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:31 crc kubenswrapper[5052]: I1203 11:47:31.615582 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage07-crc" (UniqueName: "kubernetes.io/local-volume/local-storage07-crc") on node "crc" Dec 03 11:47:31 crc kubenswrapper[5052]: I1203 11:47:31.682181 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="00b12739-1437-4af3-ab6d-ad830b67ad2e" path="/var/lib/kubelet/pods/00b12739-1437-4af3-ab6d-ad830b67ad2e/volumes" Dec 03 11:47:31 crc kubenswrapper[5052]: I1203 11:47:31.682932 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0183ebb5-b91b-443a-9535-2008c9207ab1" path="/var/lib/kubelet/pods/0183ebb5-b91b-443a-9535-2008c9207ab1/volumes" Dec 03 11:47:31 crc kubenswrapper[5052]: I1203 11:47:31.683567 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0914ebfe-13c6-4309-bd10-a67850124e3f" path="/var/lib/kubelet/pods/0914ebfe-13c6-4309-bd10-a67850124e3f/volumes" Dec 03 11:47:31 crc kubenswrapper[5052]: I1203 11:47:31.684582 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0aefd6f6-cbb3-4c45-9f69-7124e43255e3" path="/var/lib/kubelet/pods/0aefd6f6-cbb3-4c45-9f69-7124e43255e3/volumes" Dec 03 11:47:31 crc kubenswrapper[5052]: I1203 11:47:31.685019 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0b94b517-5f9b-485d-a109-0a11e15a3ad3" path="/var/lib/kubelet/pods/0b94b517-5f9b-485d-a109-0a11e15a3ad3/volumes" Dec 03 11:47:31 crc kubenswrapper[5052]: I1203 11:47:31.685482 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1b45577a-92b7-4c67-941b-cfc2fbfa3eb2" path="/var/lib/kubelet/pods/1b45577a-92b7-4c67-941b-cfc2fbfa3eb2/volumes" Dec 03 11:47:31 crc kubenswrapper[5052]: I1203 11:47:31.686556 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2d3a3054-6c8e-4808-bd5d-01413970167c" path="/var/lib/kubelet/pods/2d3a3054-6c8e-4808-bd5d-01413970167c/volumes" Dec 03 11:47:31 crc kubenswrapper[5052]: I1203 11:47:31.686989 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="39b9198e-e3d7-4f85-b6a2-55960bdda82a" path="/var/lib/kubelet/pods/39b9198e-e3d7-4f85-b6a2-55960bdda82a/volumes" Dec 03 11:47:31 crc kubenswrapper[5052]: I1203 11:47:31.687505 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="65be66f1-c32d-40b5-81a4-a63258364d9d" path="/var/lib/kubelet/pods/65be66f1-c32d-40b5-81a4-a63258364d9d/volumes" Dec 03 11:47:31 crc kubenswrapper[5052]: I1203 11:47:31.688743 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="83a10afe-30d8-443d-a4f7-28784a2ea068" path="/var/lib/kubelet/pods/83a10afe-30d8-443d-a4f7-28784a2ea068/volumes" Dec 03 11:47:31 crc kubenswrapper[5052]: I1203 11:47:31.689275 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8c2252c0-5e30-4e38-b1a1-af0242df6ac0" path="/var/lib/kubelet/pods/8c2252c0-5e30-4e38-b1a1-af0242df6ac0/volumes" Dec 03 11:47:31 crc kubenswrapper[5052]: I1203 11:47:31.690321 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="950f0626-40d2-41c9-9701-cba50843c43e" path="/var/lib/kubelet/pods/950f0626-40d2-41c9-9701-cba50843c43e/volumes" Dec 03 11:47:31 crc kubenswrapper[5052]: I1203 11:47:31.690951 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="99f96963-d835-4445-84af-0976f1ba0701" path="/var/lib/kubelet/pods/99f96963-d835-4445-84af-0976f1ba0701/volumes" Dec 03 11:47:31 crc kubenswrapper[5052]: I1203 11:47:31.691601 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aa9a649b-9229-4cdd-8e27-8aedd48f4bbd" path="/var/lib/kubelet/pods/aa9a649b-9229-4cdd-8e27-8aedd48f4bbd/volumes" Dec 03 11:47:31 crc kubenswrapper[5052]: I1203 11:47:31.692623 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="acea154c-f0bb-4066-bd4d-2dda815f301c" path="/var/lib/kubelet/pods/acea154c-f0bb-4066-bd4d-2dda815f301c/volumes" Dec 03 11:47:31 crc kubenswrapper[5052]: I1203 11:47:31.693312 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bb51da3b-72cc-4008-aac7-737f6a3031cb" path="/var/lib/kubelet/pods/bb51da3b-72cc-4008-aac7-737f6a3031cb/volumes" Dec 03 11:47:31 crc kubenswrapper[5052]: I1203 11:47:31.693928 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cadac9a6-1d82-403a-b7a0-692a424dd0e2" path="/var/lib/kubelet/pods/cadac9a6-1d82-403a-b7a0-692a424dd0e2/volumes" Dec 03 11:47:31 crc kubenswrapper[5052]: I1203 11:47:31.695044 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d8de865a-0f6c-413d-8a37-3ec4d496c4bb" path="/var/lib/kubelet/pods/d8de865a-0f6c-413d-8a37-3ec4d496c4bb/volumes" Dec 03 11:47:31 crc kubenswrapper[5052]: I1203 11:47:31.695739 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e5339997-33d5-4794-8fa6-042b63e9391e" path="/var/lib/kubelet/pods/e5339997-33d5-4794-8fa6-042b63e9391e/volumes" Dec 03 11:47:31 crc kubenswrapper[5052]: I1203 11:47:31.696289 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f7123213-06e1-4b26-9892-c16c3d49c0ab" path="/var/lib/kubelet/pods/f7123213-06e1-4b26-9892-c16c3d49c0ab/volumes" Dec 03 11:47:31 crc kubenswrapper[5052]: I1203 11:47:31.697714 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage07-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage07-crc\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:32 crc kubenswrapper[5052]: I1203 11:47:32.141388 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-server-0" event={"ID":"d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0","Type":"ContainerDied","Data":"7711b51a7ed58585774b502006aeff8840e74cc12b62130717e231f47f598cac"} Dec 03 11:47:32 crc kubenswrapper[5052]: I1203 11:47:32.141444 5052 scope.go:117] "RemoveContainer" containerID="f1ccb0c2e8810415edac670e94d35e316eb9d118a2ee2cb2d84f782419550625" Dec 03 11:47:32 crc kubenswrapper[5052]: I1203 11:47:32.141598 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 11:47:32 crc kubenswrapper[5052]: I1203 11:47:32.160178 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/memcached-0"] Dec 03 11:47:32 crc kubenswrapper[5052]: I1203 11:47:32.160752 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/memcached-0" podUID="6f9b208b-086f-4671-8c3f-b3a275740ad7" containerName="memcached" containerID="cri-o://af1f689f6ec28c7e342622784d72e045524b9ad80e36d71f89fc38ea43408004" gracePeriod=30 Dec 03 11:47:32 crc kubenswrapper[5052]: I1203 11:47:32.192630 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/rabbitmq-server-0"] Dec 03 11:47:32 crc kubenswrapper[5052]: I1203 11:47:32.199585 5052 scope.go:117] "RemoveContainer" containerID="843eb248ababc1108cc5ab5df2a9259468619c4e57c04fb2e96d893def21be0a" Dec 03 11:47:32 crc kubenswrapper[5052]: I1203 11:47:32.200658 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/rabbitmq-server-0"] Dec 03 11:47:32 crc kubenswrapper[5052]: I1203 11:47:32.206811 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-db-sync-mbzlf"] Dec 03 11:47:32 crc kubenswrapper[5052]: I1203 11:47:32.212913 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-db-sync-mbzlf"] Dec 03 11:47:32 crc kubenswrapper[5052]: I1203 11:47:32.220268 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-0"] Dec 03 11:47:32 crc kubenswrapper[5052]: I1203 11:47:32.220624 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-cell1-conductor-0" podUID="507c60e5-056c-4661-a7a7-75c4d9bce7de" containerName="nova-cell1-conductor-conductor" containerID="cri-o://ad64b4dca94e6f61fc353d5aa054f7de7c10558b8c9671ffe67a4be04544faad" gracePeriod=30 Dec 03 11:47:32 crc kubenswrapper[5052]: I1203 11:47:32.224491 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-db-sync-42jqd"] Dec 03 11:47:32 crc kubenswrapper[5052]: I1203 11:47:32.228653 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-0"] Dec 03 11:47:32 crc kubenswrapper[5052]: I1203 11:47:32.228948 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-cell0-conductor-0" podUID="d6d9150c-e5b6-407a-b20a-4114a60312f8" containerName="nova-cell0-conductor-conductor" containerID="cri-o://bd5639bf60648b831ed05e44afedef6c3f8829e60604e5ebb80ecdd5b8960855" gracePeriod=30 Dec 03 11:47:32 crc kubenswrapper[5052]: I1203 11:47:32.233014 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-db-sync-42jqd"] Dec 03 11:47:32 crc kubenswrapper[5052]: I1203 11:47:32.673078 5052 scope.go:117] "RemoveContainer" containerID="25fe88f8d102f09983f8884e65f0c6c1575ade67921f10a630e6dcc3ea8b506e" Dec 03 11:47:32 crc kubenswrapper[5052]: E1203 11:47:32.673308 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 11:47:33 crc kubenswrapper[5052]: I1203 11:47:33.510758 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/memcached-0" Dec 03 11:47:33 crc kubenswrapper[5052]: I1203 11:47:33.624931 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/6f9b208b-086f-4671-8c3f-b3a275740ad7-config-data\") pod \"6f9b208b-086f-4671-8c3f-b3a275740ad7\" (UID: \"6f9b208b-086f-4671-8c3f-b3a275740ad7\") " Dec 03 11:47:33 crc kubenswrapper[5052]: I1203 11:47:33.625318 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6f9b208b-086f-4671-8c3f-b3a275740ad7-combined-ca-bundle\") pod \"6f9b208b-086f-4671-8c3f-b3a275740ad7\" (UID: \"6f9b208b-086f-4671-8c3f-b3a275740ad7\") " Dec 03 11:47:33 crc kubenswrapper[5052]: I1203 11:47:33.625436 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/6f9b208b-086f-4671-8c3f-b3a275740ad7-kolla-config\") pod \"6f9b208b-086f-4671-8c3f-b3a275740ad7\" (UID: \"6f9b208b-086f-4671-8c3f-b3a275740ad7\") " Dec 03 11:47:33 crc kubenswrapper[5052]: I1203 11:47:33.625465 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nn9xv\" (UniqueName: \"kubernetes.io/projected/6f9b208b-086f-4671-8c3f-b3a275740ad7-kube-api-access-nn9xv\") pod \"6f9b208b-086f-4671-8c3f-b3a275740ad7\" (UID: \"6f9b208b-086f-4671-8c3f-b3a275740ad7\") " Dec 03 11:47:33 crc kubenswrapper[5052]: I1203 11:47:33.625497 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"memcached-tls-certs\" (UniqueName: \"kubernetes.io/secret/6f9b208b-086f-4671-8c3f-b3a275740ad7-memcached-tls-certs\") pod \"6f9b208b-086f-4671-8c3f-b3a275740ad7\" (UID: \"6f9b208b-086f-4671-8c3f-b3a275740ad7\") " Dec 03 11:47:33 crc kubenswrapper[5052]: I1203 11:47:33.626470 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6f9b208b-086f-4671-8c3f-b3a275740ad7-config-data" (OuterVolumeSpecName: "config-data") pod "6f9b208b-086f-4671-8c3f-b3a275740ad7" (UID: "6f9b208b-086f-4671-8c3f-b3a275740ad7"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:47:33 crc kubenswrapper[5052]: I1203 11:47:33.627138 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6f9b208b-086f-4671-8c3f-b3a275740ad7-kolla-config" (OuterVolumeSpecName: "kolla-config") pod "6f9b208b-086f-4671-8c3f-b3a275740ad7" (UID: "6f9b208b-086f-4671-8c3f-b3a275740ad7"). InnerVolumeSpecName "kolla-config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:47:33 crc kubenswrapper[5052]: I1203 11:47:33.642407 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6f9b208b-086f-4671-8c3f-b3a275740ad7-kube-api-access-nn9xv" (OuterVolumeSpecName: "kube-api-access-nn9xv") pod "6f9b208b-086f-4671-8c3f-b3a275740ad7" (UID: "6f9b208b-086f-4671-8c3f-b3a275740ad7"). InnerVolumeSpecName "kube-api-access-nn9xv". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:47:33 crc kubenswrapper[5052]: I1203 11:47:33.659151 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6f9b208b-086f-4671-8c3f-b3a275740ad7-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "6f9b208b-086f-4671-8c3f-b3a275740ad7" (UID: "6f9b208b-086f-4671-8c3f-b3a275740ad7"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:47:33 crc kubenswrapper[5052]: I1203 11:47:33.679730 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6f9b208b-086f-4671-8c3f-b3a275740ad7-memcached-tls-certs" (OuterVolumeSpecName: "memcached-tls-certs") pod "6f9b208b-086f-4671-8c3f-b3a275740ad7" (UID: "6f9b208b-086f-4671-8c3f-b3a275740ad7"). InnerVolumeSpecName "memcached-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:47:33 crc kubenswrapper[5052]: I1203 11:47:33.690806 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8dcfd9c8-2ef8-44a2-ad54-5382d56c630d" path="/var/lib/kubelet/pods/8dcfd9c8-2ef8-44a2-ad54-5382d56c630d/volumes" Dec 03 11:47:33 crc kubenswrapper[5052]: I1203 11:47:33.691539 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b3524e58-0557-4bdd-9b3e-4584c5595b18" path="/var/lib/kubelet/pods/b3524e58-0557-4bdd-9b3e-4584c5595b18/volumes" Dec 03 11:47:33 crc kubenswrapper[5052]: I1203 11:47:33.692898 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0" path="/var/lib/kubelet/pods/d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0/volumes" Dec 03 11:47:33 crc kubenswrapper[5052]: I1203 11:47:33.711690 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/neutron-549dfc49dd-qtk5n" podUID="a4fa0509-8f85-462f-b7c3-4384346252bc" containerName="neutron-httpd" probeResult="failure" output="Get \"https://10.217.0.251:9696/\": dial tcp 10.217.0.251:9696: connect: connection refused" Dec 03 11:47:33 crc kubenswrapper[5052]: I1203 11:47:33.727638 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-nn9xv\" (UniqueName: \"kubernetes.io/projected/6f9b208b-086f-4671-8c3f-b3a275740ad7-kube-api-access-nn9xv\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:33 crc kubenswrapper[5052]: I1203 11:47:33.727672 5052 reconciler_common.go:293] "Volume detached for volume \"memcached-tls-certs\" (UniqueName: \"kubernetes.io/secret/6f9b208b-086f-4671-8c3f-b3a275740ad7-memcached-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:33 crc kubenswrapper[5052]: I1203 11:47:33.727682 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/6f9b208b-086f-4671-8c3f-b3a275740ad7-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:33 crc kubenswrapper[5052]: I1203 11:47:33.727691 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6f9b208b-086f-4671-8c3f-b3a275740ad7-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:33 crc kubenswrapper[5052]: I1203 11:47:33.727700 5052 reconciler_common.go:293] "Volume detached for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/6f9b208b-086f-4671-8c3f-b3a275740ad7-kolla-config\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:33 crc kubenswrapper[5052]: I1203 11:47:33.802627 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 11:47:33 crc kubenswrapper[5052]: I1203 11:47:33.929986 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d6d9150c-e5b6-407a-b20a-4114a60312f8-combined-ca-bundle\") pod \"d6d9150c-e5b6-407a-b20a-4114a60312f8\" (UID: \"d6d9150c-e5b6-407a-b20a-4114a60312f8\") " Dec 03 11:47:33 crc kubenswrapper[5052]: I1203 11:47:33.930061 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qs6nr\" (UniqueName: \"kubernetes.io/projected/d6d9150c-e5b6-407a-b20a-4114a60312f8-kube-api-access-qs6nr\") pod \"d6d9150c-e5b6-407a-b20a-4114a60312f8\" (UID: \"d6d9150c-e5b6-407a-b20a-4114a60312f8\") " Dec 03 11:47:33 crc kubenswrapper[5052]: I1203 11:47:33.930110 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/d6d9150c-e5b6-407a-b20a-4114a60312f8-config-data\") pod \"d6d9150c-e5b6-407a-b20a-4114a60312f8\" (UID: \"d6d9150c-e5b6-407a-b20a-4114a60312f8\") " Dec 03 11:47:33 crc kubenswrapper[5052]: I1203 11:47:33.936441 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d6d9150c-e5b6-407a-b20a-4114a60312f8-kube-api-access-qs6nr" (OuterVolumeSpecName: "kube-api-access-qs6nr") pod "d6d9150c-e5b6-407a-b20a-4114a60312f8" (UID: "d6d9150c-e5b6-407a-b20a-4114a60312f8"). InnerVolumeSpecName "kube-api-access-qs6nr". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:47:33 crc kubenswrapper[5052]: I1203 11:47:33.952790 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d6d9150c-e5b6-407a-b20a-4114a60312f8-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "d6d9150c-e5b6-407a-b20a-4114a60312f8" (UID: "d6d9150c-e5b6-407a-b20a-4114a60312f8"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:47:33 crc kubenswrapper[5052]: I1203 11:47:33.953427 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d6d9150c-e5b6-407a-b20a-4114a60312f8-config-data" (OuterVolumeSpecName: "config-data") pod "d6d9150c-e5b6-407a-b20a-4114a60312f8" (UID: "d6d9150c-e5b6-407a-b20a-4114a60312f8"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:47:34 crc kubenswrapper[5052]: I1203 11:47:34.032261 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d6d9150c-e5b6-407a-b20a-4114a60312f8-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:34 crc kubenswrapper[5052]: I1203 11:47:34.032292 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-qs6nr\" (UniqueName: \"kubernetes.io/projected/d6d9150c-e5b6-407a-b20a-4114a60312f8-kube-api-access-qs6nr\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:34 crc kubenswrapper[5052]: I1203 11:47:34.032302 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/d6d9150c-e5b6-407a-b20a-4114a60312f8-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:34 crc kubenswrapper[5052]: I1203 11:47:34.164538 5052 generic.go:334] "Generic (PLEG): container finished" podID="6f9b208b-086f-4671-8c3f-b3a275740ad7" containerID="af1f689f6ec28c7e342622784d72e045524b9ad80e36d71f89fc38ea43408004" exitCode=0 Dec 03 11:47:34 crc kubenswrapper[5052]: I1203 11:47:34.164591 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/memcached-0" Dec 03 11:47:34 crc kubenswrapper[5052]: I1203 11:47:34.164624 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/memcached-0" event={"ID":"6f9b208b-086f-4671-8c3f-b3a275740ad7","Type":"ContainerDied","Data":"af1f689f6ec28c7e342622784d72e045524b9ad80e36d71f89fc38ea43408004"} Dec 03 11:47:34 crc kubenswrapper[5052]: I1203 11:47:34.164815 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/memcached-0" event={"ID":"6f9b208b-086f-4671-8c3f-b3a275740ad7","Type":"ContainerDied","Data":"f836832155313a0381cab85f527e844cd45abd16cd9cc0a0b17fe34f70b740cf"} Dec 03 11:47:34 crc kubenswrapper[5052]: I1203 11:47:34.164848 5052 scope.go:117] "RemoveContainer" containerID="af1f689f6ec28c7e342622784d72e045524b9ad80e36d71f89fc38ea43408004" Dec 03 11:47:34 crc kubenswrapper[5052]: I1203 11:47:34.166527 5052 generic.go:334] "Generic (PLEG): container finished" podID="d6d9150c-e5b6-407a-b20a-4114a60312f8" containerID="bd5639bf60648b831ed05e44afedef6c3f8829e60604e5ebb80ecdd5b8960855" exitCode=0 Dec 03 11:47:34 crc kubenswrapper[5052]: I1203 11:47:34.166561 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-conductor-0" event={"ID":"d6d9150c-e5b6-407a-b20a-4114a60312f8","Type":"ContainerDied","Data":"bd5639bf60648b831ed05e44afedef6c3f8829e60604e5ebb80ecdd5b8960855"} Dec 03 11:47:34 crc kubenswrapper[5052]: I1203 11:47:34.166590 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-conductor-0" event={"ID":"d6d9150c-e5b6-407a-b20a-4114a60312f8","Type":"ContainerDied","Data":"c8b0bf7df94802874710eff6ff62a8275187b610c63db71e41dd8ed590a58aab"} Dec 03 11:47:34 crc kubenswrapper[5052]: I1203 11:47:34.166642 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 11:47:34 crc kubenswrapper[5052]: I1203 11:47:34.231112 5052 scope.go:117] "RemoveContainer" containerID="af1f689f6ec28c7e342622784d72e045524b9ad80e36d71f89fc38ea43408004" Dec 03 11:47:34 crc kubenswrapper[5052]: E1203 11:47:34.232306 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"af1f689f6ec28c7e342622784d72e045524b9ad80e36d71f89fc38ea43408004\": container with ID starting with af1f689f6ec28c7e342622784d72e045524b9ad80e36d71f89fc38ea43408004 not found: ID does not exist" containerID="af1f689f6ec28c7e342622784d72e045524b9ad80e36d71f89fc38ea43408004" Dec 03 11:47:34 crc kubenswrapper[5052]: I1203 11:47:34.232334 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"af1f689f6ec28c7e342622784d72e045524b9ad80e36d71f89fc38ea43408004"} err="failed to get container status \"af1f689f6ec28c7e342622784d72e045524b9ad80e36d71f89fc38ea43408004\": rpc error: code = NotFound desc = could not find container \"af1f689f6ec28c7e342622784d72e045524b9ad80e36d71f89fc38ea43408004\": container with ID starting with af1f689f6ec28c7e342622784d72e045524b9ad80e36d71f89fc38ea43408004 not found: ID does not exist" Dec 03 11:47:34 crc kubenswrapper[5052]: I1203 11:47:34.232361 5052 scope.go:117] "RemoveContainer" containerID="bd5639bf60648b831ed05e44afedef6c3f8829e60604e5ebb80ecdd5b8960855" Dec 03 11:47:34 crc kubenswrapper[5052]: I1203 11:47:34.233502 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/memcached-0"] Dec 03 11:47:34 crc kubenswrapper[5052]: I1203 11:47:34.243674 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/memcached-0"] Dec 03 11:47:34 crc kubenswrapper[5052]: I1203 11:47:34.250630 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-0"] Dec 03 11:47:34 crc kubenswrapper[5052]: I1203 11:47:34.254567 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-0"] Dec 03 11:47:34 crc kubenswrapper[5052]: I1203 11:47:34.256181 5052 scope.go:117] "RemoveContainer" containerID="bd5639bf60648b831ed05e44afedef6c3f8829e60604e5ebb80ecdd5b8960855" Dec 03 11:47:34 crc kubenswrapper[5052]: E1203 11:47:34.256626 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bd5639bf60648b831ed05e44afedef6c3f8829e60604e5ebb80ecdd5b8960855\": container with ID starting with bd5639bf60648b831ed05e44afedef6c3f8829e60604e5ebb80ecdd5b8960855 not found: ID does not exist" containerID="bd5639bf60648b831ed05e44afedef6c3f8829e60604e5ebb80ecdd5b8960855" Dec 03 11:47:34 crc kubenswrapper[5052]: I1203 11:47:34.256664 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bd5639bf60648b831ed05e44afedef6c3f8829e60604e5ebb80ecdd5b8960855"} err="failed to get container status \"bd5639bf60648b831ed05e44afedef6c3f8829e60604e5ebb80ecdd5b8960855\": rpc error: code = NotFound desc = could not find container \"bd5639bf60648b831ed05e44afedef6c3f8829e60604e5ebb80ecdd5b8960855\": container with ID starting with bd5639bf60648b831ed05e44afedef6c3f8829e60604e5ebb80ecdd5b8960855 not found: ID does not exist" Dec 03 11:47:35 crc kubenswrapper[5052]: I1203 11:47:35.681958 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6f9b208b-086f-4671-8c3f-b3a275740ad7" path="/var/lib/kubelet/pods/6f9b208b-086f-4671-8c3f-b3a275740ad7/volumes" Dec 03 11:47:35 crc kubenswrapper[5052]: I1203 11:47:35.682841 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d6d9150c-e5b6-407a-b20a-4114a60312f8" path="/var/lib/kubelet/pods/d6d9150c-e5b6-407a-b20a-4114a60312f8/volumes" Dec 03 11:47:36 crc kubenswrapper[5052]: E1203 11:47:36.513549 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="ad64b4dca94e6f61fc353d5aa054f7de7c10558b8c9671ffe67a4be04544faad" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 11:47:36 crc kubenswrapper[5052]: E1203 11:47:36.515959 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="ad64b4dca94e6f61fc353d5aa054f7de7c10558b8c9671ffe67a4be04544faad" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 11:47:36 crc kubenswrapper[5052]: E1203 11:47:36.517178 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="ad64b4dca94e6f61fc353d5aa054f7de7c10558b8c9671ffe67a4be04544faad" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 11:47:36 crc kubenswrapper[5052]: E1203 11:47:36.517215 5052 prober.go:104] "Probe errored" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" probeType="Readiness" pod="openstack-kuttl-tests/nova-cell1-conductor-0" podUID="507c60e5-056c-4661-a7a7-75c4d9bce7de" containerName="nova-cell1-conductor-conductor" Dec 03 11:47:36 crc kubenswrapper[5052]: E1203 11:47:36.578784 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/rabbitmq-cell1-config-data: configmap "rabbitmq-cell1-config-data" not found Dec 03 11:47:36 crc kubenswrapper[5052]: E1203 11:47:36.578841 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/combined-ca-bundle: secret "combined-ca-bundle" not found Dec 03 11:47:36 crc kubenswrapper[5052]: E1203 11:47:36.578882 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2766f489-9217-47ae-a595-869a26389587-config-data podName:2766f489-9217-47ae-a595-869a26389587 nodeName:}" failed. No retries permitted until 2025-12-03 11:47:52.578856553 +0000 UTC m=+1663.167098327 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "config-data" (UniqueName: "kubernetes.io/configmap/2766f489-9217-47ae-a595-869a26389587-config-data") pod "rabbitmq-cell1-server-0" (UID: "2766f489-9217-47ae-a595-869a26389587") : configmap "rabbitmq-cell1-config-data" not found Dec 03 11:47:36 crc kubenswrapper[5052]: E1203 11:47:36.578929 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/507c60e5-056c-4661-a7a7-75c4d9bce7de-combined-ca-bundle podName:507c60e5-056c-4661-a7a7-75c4d9bce7de nodeName:}" failed. No retries permitted until 2025-12-03 11:47:52.578906845 +0000 UTC m=+1663.167148609 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "combined-ca-bundle" (UniqueName: "kubernetes.io/secret/507c60e5-056c-4661-a7a7-75c4d9bce7de-combined-ca-bundle") pod "nova-cell1-conductor-0" (UID: "507c60e5-056c-4661-a7a7-75c4d9bce7de") : secret "combined-ca-bundle" not found Dec 03 11:47:37 crc kubenswrapper[5052]: E1203 11:47:37.696860 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cinder-scheduler-config-data: secret "cinder-scheduler-config-data" not found Dec 03 11:47:37 crc kubenswrapper[5052]: E1203 11:47:37.697173 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e947107f-104b-4699-a994-bc77285f0cf5-config-data-custom podName:e947107f-104b-4699-a994-bc77285f0cf5 nodeName:}" failed. No retries permitted until 2025-12-03 11:47:53.697154916 +0000 UTC m=+1664.285396690 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "config-data-custom" (UniqueName: "kubernetes.io/secret/e947107f-104b-4699-a994-bc77285f0cf5-config-data-custom") pod "cinder-scheduler-0" (UID: "e947107f-104b-4699-a994-bc77285f0cf5") : secret "cinder-scheduler-config-data" not found Dec 03 11:47:37 crc kubenswrapper[5052]: E1203 11:47:37.697230 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cinder-config-data: secret "cinder-config-data" not found Dec 03 11:47:37 crc kubenswrapper[5052]: E1203 11:47:37.697257 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e947107f-104b-4699-a994-bc77285f0cf5-config-data podName:e947107f-104b-4699-a994-bc77285f0cf5 nodeName:}" failed. No retries permitted until 2025-12-03 11:47:53.697247969 +0000 UTC m=+1664.285489743 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "config-data" (UniqueName: "kubernetes.io/secret/e947107f-104b-4699-a994-bc77285f0cf5-config-data") pod "cinder-scheduler-0" (UID: "e947107f-104b-4699-a994-bc77285f0cf5") : secret "cinder-config-data" not found Dec 03 11:47:37 crc kubenswrapper[5052]: E1203 11:47:37.697300 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/combined-ca-bundle: secret "combined-ca-bundle" not found Dec 03 11:47:37 crc kubenswrapper[5052]: E1203 11:47:37.697323 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e947107f-104b-4699-a994-bc77285f0cf5-combined-ca-bundle podName:e947107f-104b-4699-a994-bc77285f0cf5 nodeName:}" failed. No retries permitted until 2025-12-03 11:47:53.697315441 +0000 UTC m=+1664.285557215 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "combined-ca-bundle" (UniqueName: "kubernetes.io/secret/e947107f-104b-4699-a994-bc77285f0cf5-combined-ca-bundle") pod "cinder-scheduler-0" (UID: "e947107f-104b-4699-a994-bc77285f0cf5") : secret "combined-ca-bundle" not found Dec 03 11:47:37 crc kubenswrapper[5052]: E1203 11:47:37.697362 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cinder-scripts: secret "cinder-scripts" not found Dec 03 11:47:37 crc kubenswrapper[5052]: E1203 11:47:37.697384 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e947107f-104b-4699-a994-bc77285f0cf5-scripts podName:e947107f-104b-4699-a994-bc77285f0cf5 nodeName:}" failed. No retries permitted until 2025-12-03 11:47:53.697376543 +0000 UTC m=+1664.285618317 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "scripts" (UniqueName: "kubernetes.io/secret/e947107f-104b-4699-a994-bc77285f0cf5-scripts") pod "cinder-scheduler-0" (UID: "e947107f-104b-4699-a994-bc77285f0cf5") : secret "cinder-scripts" not found Dec 03 11:47:37 crc kubenswrapper[5052]: I1203 11:47:37.728897 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-lfl6r" Dec 03 11:47:37 crc kubenswrapper[5052]: I1203 11:47:37.801051 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-7d7b5bb58f-sgdms"] Dec 03 11:47:37 crc kubenswrapper[5052]: I1203 11:47:37.801373 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-7d7b5bb58f-sgdms" podUID="bbbc503e-d33d-42a9-856a-9ecc1d059f6b" containerName="dnsmasq-dns" containerID="cri-o://d0ee0cef3f1acf1c1868c3a8e765eb1d1fe5a4af1fd316d302221be7039841f9" gracePeriod=10 Dec 03 11:47:38 crc kubenswrapper[5052]: I1203 11:47:38.225669 5052 generic.go:334] "Generic (PLEG): container finished" podID="bbbc503e-d33d-42a9-856a-9ecc1d059f6b" containerID="d0ee0cef3f1acf1c1868c3a8e765eb1d1fe5a4af1fd316d302221be7039841f9" exitCode=0 Dec 03 11:47:38 crc kubenswrapper[5052]: I1203 11:47:38.226163 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-7d7b5bb58f-sgdms" event={"ID":"bbbc503e-d33d-42a9-856a-9ecc1d059f6b","Type":"ContainerDied","Data":"d0ee0cef3f1acf1c1868c3a8e765eb1d1fe5a4af1fd316d302221be7039841f9"} Dec 03 11:47:38 crc kubenswrapper[5052]: I1203 11:47:38.304977 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-7d7b5bb58f-sgdms" Dec 03 11:47:38 crc kubenswrapper[5052]: I1203 11:47:38.409896 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nctjk\" (UniqueName: \"kubernetes.io/projected/bbbc503e-d33d-42a9-856a-9ecc1d059f6b-kube-api-access-nctjk\") pod \"bbbc503e-d33d-42a9-856a-9ecc1d059f6b\" (UID: \"bbbc503e-d33d-42a9-856a-9ecc1d059f6b\") " Dec 03 11:47:38 crc kubenswrapper[5052]: I1203 11:47:38.409951 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"dns-swift-storage-0\" (UniqueName: \"kubernetes.io/configmap/bbbc503e-d33d-42a9-856a-9ecc1d059f6b-dns-swift-storage-0\") pod \"bbbc503e-d33d-42a9-856a-9ecc1d059f6b\" (UID: \"bbbc503e-d33d-42a9-856a-9ecc1d059f6b\") " Dec 03 11:47:38 crc kubenswrapper[5052]: I1203 11:47:38.410015 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/bbbc503e-d33d-42a9-856a-9ecc1d059f6b-dnsmasq-svc\") pod \"bbbc503e-d33d-42a9-856a-9ecc1d059f6b\" (UID: \"bbbc503e-d33d-42a9-856a-9ecc1d059f6b\") " Dec 03 11:47:38 crc kubenswrapper[5052]: I1203 11:47:38.410122 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/bbbc503e-d33d-42a9-856a-9ecc1d059f6b-config\") pod \"bbbc503e-d33d-42a9-856a-9ecc1d059f6b\" (UID: \"bbbc503e-d33d-42a9-856a-9ecc1d059f6b\") " Dec 03 11:47:38 crc kubenswrapper[5052]: I1203 11:47:38.415606 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bbbc503e-d33d-42a9-856a-9ecc1d059f6b-kube-api-access-nctjk" (OuterVolumeSpecName: "kube-api-access-nctjk") pod "bbbc503e-d33d-42a9-856a-9ecc1d059f6b" (UID: "bbbc503e-d33d-42a9-856a-9ecc1d059f6b"). InnerVolumeSpecName "kube-api-access-nctjk". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:47:38 crc kubenswrapper[5052]: I1203 11:47:38.450102 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bbbc503e-d33d-42a9-856a-9ecc1d059f6b-dns-swift-storage-0" (OuterVolumeSpecName: "dns-swift-storage-0") pod "bbbc503e-d33d-42a9-856a-9ecc1d059f6b" (UID: "bbbc503e-d33d-42a9-856a-9ecc1d059f6b"). InnerVolumeSpecName "dns-swift-storage-0". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:47:38 crc kubenswrapper[5052]: I1203 11:47:38.452284 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bbbc503e-d33d-42a9-856a-9ecc1d059f6b-dnsmasq-svc" (OuterVolumeSpecName: "dnsmasq-svc") pod "bbbc503e-d33d-42a9-856a-9ecc1d059f6b" (UID: "bbbc503e-d33d-42a9-856a-9ecc1d059f6b"). InnerVolumeSpecName "dnsmasq-svc". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:47:38 crc kubenswrapper[5052]: I1203 11:47:38.463181 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bbbc503e-d33d-42a9-856a-9ecc1d059f6b-config" (OuterVolumeSpecName: "config") pod "bbbc503e-d33d-42a9-856a-9ecc1d059f6b" (UID: "bbbc503e-d33d-42a9-856a-9ecc1d059f6b"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:47:38 crc kubenswrapper[5052]: I1203 11:47:38.511715 5052 reconciler_common.go:293] "Volume detached for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/bbbc503e-d33d-42a9-856a-9ecc1d059f6b-dnsmasq-svc\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:38 crc kubenswrapper[5052]: I1203 11:47:38.511755 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/bbbc503e-d33d-42a9-856a-9ecc1d059f6b-config\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:38 crc kubenswrapper[5052]: I1203 11:47:38.511769 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-nctjk\" (UniqueName: \"kubernetes.io/projected/bbbc503e-d33d-42a9-856a-9ecc1d059f6b-kube-api-access-nctjk\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:38 crc kubenswrapper[5052]: I1203 11:47:38.511782 5052 reconciler_common.go:293] "Volume detached for volume \"dns-swift-storage-0\" (UniqueName: \"kubernetes.io/configmap/bbbc503e-d33d-42a9-856a-9ecc1d059f6b-dns-swift-storage-0\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:39 crc kubenswrapper[5052]: I1203 11:47:39.243503 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-7d7b5bb58f-sgdms" event={"ID":"bbbc503e-d33d-42a9-856a-9ecc1d059f6b","Type":"ContainerDied","Data":"1397a5677def85e35981116a319fe663882469aeb115727b8a3e6bc09c12342e"} Dec 03 11:47:39 crc kubenswrapper[5052]: I1203 11:47:39.243593 5052 scope.go:117] "RemoveContainer" containerID="d0ee0cef3f1acf1c1868c3a8e765eb1d1fe5a4af1fd316d302221be7039841f9" Dec 03 11:47:39 crc kubenswrapper[5052]: I1203 11:47:39.245087 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-7d7b5bb58f-sgdms" Dec 03 11:47:39 crc kubenswrapper[5052]: I1203 11:47:39.289891 5052 scope.go:117] "RemoveContainer" containerID="a76bef2305c43c6073eaeef6861eaec8928c009de434c14bc97bc064ff85d56e" Dec 03 11:47:39 crc kubenswrapper[5052]: I1203 11:47:39.305962 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-7d7b5bb58f-sgdms"] Dec 03 11:47:39 crc kubenswrapper[5052]: I1203 11:47:39.311126 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-7d7b5bb58f-sgdms"] Dec 03 11:47:39 crc kubenswrapper[5052]: I1203 11:47:39.683438 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bbbc503e-d33d-42a9-856a-9ecc1d059f6b" path="/var/lib/kubelet/pods/bbbc503e-d33d-42a9-856a-9ecc1d059f6b/volumes" Dec 03 11:47:41 crc kubenswrapper[5052]: E1203 11:47:41.512301 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="ad64b4dca94e6f61fc353d5aa054f7de7c10558b8c9671ffe67a4be04544faad" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 11:47:41 crc kubenswrapper[5052]: E1203 11:47:41.513903 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="ad64b4dca94e6f61fc353d5aa054f7de7c10558b8c9671ffe67a4be04544faad" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 11:47:41 crc kubenswrapper[5052]: E1203 11:47:41.516298 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="ad64b4dca94e6f61fc353d5aa054f7de7c10558b8c9671ffe67a4be04544faad" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 11:47:41 crc kubenswrapper[5052]: E1203 11:47:41.516338 5052 prober.go:104] "Probe errored" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" probeType="Readiness" pod="openstack-kuttl-tests/nova-cell1-conductor-0" podUID="507c60e5-056c-4661-a7a7-75c4d9bce7de" containerName="nova-cell1-conductor-conductor" Dec 03 11:47:42 crc kubenswrapper[5052]: I1203 11:47:42.275911 5052 generic.go:334] "Generic (PLEG): container finished" podID="a4fa0509-8f85-462f-b7c3-4384346252bc" containerID="a10887f92ee55d6e2db028846bedc75386b0fc5b0814cc18643f424e258d8b3c" exitCode=0 Dec 03 11:47:42 crc kubenswrapper[5052]: I1203 11:47:42.275956 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-549dfc49dd-qtk5n" event={"ID":"a4fa0509-8f85-462f-b7c3-4384346252bc","Type":"ContainerDied","Data":"a10887f92ee55d6e2db028846bedc75386b0fc5b0814cc18643f424e258d8b3c"} Dec 03 11:47:42 crc kubenswrapper[5052]: I1203 11:47:42.455437 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-549dfc49dd-qtk5n" Dec 03 11:47:42 crc kubenswrapper[5052]: I1203 11:47:42.584433 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4zslh\" (UniqueName: \"kubernetes.io/projected/a4fa0509-8f85-462f-b7c3-4384346252bc-kube-api-access-4zslh\") pod \"a4fa0509-8f85-462f-b7c3-4384346252bc\" (UID: \"a4fa0509-8f85-462f-b7c3-4384346252bc\") " Dec 03 11:47:42 crc kubenswrapper[5052]: I1203 11:47:42.584509 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/a4fa0509-8f85-462f-b7c3-4384346252bc-httpd-config\") pod \"a4fa0509-8f85-462f-b7c3-4384346252bc\" (UID: \"a4fa0509-8f85-462f-b7c3-4384346252bc\") " Dec 03 11:47:42 crc kubenswrapper[5052]: I1203 11:47:42.584563 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/a4fa0509-8f85-462f-b7c3-4384346252bc-ovndb-tls-certs\") pod \"a4fa0509-8f85-462f-b7c3-4384346252bc\" (UID: \"a4fa0509-8f85-462f-b7c3-4384346252bc\") " Dec 03 11:47:42 crc kubenswrapper[5052]: I1203 11:47:42.584599 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a4fa0509-8f85-462f-b7c3-4384346252bc-combined-ca-bundle\") pod \"a4fa0509-8f85-462f-b7c3-4384346252bc\" (UID: \"a4fa0509-8f85-462f-b7c3-4384346252bc\") " Dec 03 11:47:42 crc kubenswrapper[5052]: I1203 11:47:42.584742 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/a4fa0509-8f85-462f-b7c3-4384346252bc-public-tls-certs\") pod \"a4fa0509-8f85-462f-b7c3-4384346252bc\" (UID: \"a4fa0509-8f85-462f-b7c3-4384346252bc\") " Dec 03 11:47:42 crc kubenswrapper[5052]: I1203 11:47:42.584852 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/a4fa0509-8f85-462f-b7c3-4384346252bc-config\") pod \"a4fa0509-8f85-462f-b7c3-4384346252bc\" (UID: \"a4fa0509-8f85-462f-b7c3-4384346252bc\") " Dec 03 11:47:42 crc kubenswrapper[5052]: I1203 11:47:42.584916 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/a4fa0509-8f85-462f-b7c3-4384346252bc-internal-tls-certs\") pod \"a4fa0509-8f85-462f-b7c3-4384346252bc\" (UID: \"a4fa0509-8f85-462f-b7c3-4384346252bc\") " Dec 03 11:47:42 crc kubenswrapper[5052]: I1203 11:47:42.591839 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a4fa0509-8f85-462f-b7c3-4384346252bc-httpd-config" (OuterVolumeSpecName: "httpd-config") pod "a4fa0509-8f85-462f-b7c3-4384346252bc" (UID: "a4fa0509-8f85-462f-b7c3-4384346252bc"). InnerVolumeSpecName "httpd-config". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:47:42 crc kubenswrapper[5052]: I1203 11:47:42.594338 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a4fa0509-8f85-462f-b7c3-4384346252bc-kube-api-access-4zslh" (OuterVolumeSpecName: "kube-api-access-4zslh") pod "a4fa0509-8f85-462f-b7c3-4384346252bc" (UID: "a4fa0509-8f85-462f-b7c3-4384346252bc"). InnerVolumeSpecName "kube-api-access-4zslh". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:47:42 crc kubenswrapper[5052]: I1203 11:47:42.632463 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a4fa0509-8f85-462f-b7c3-4384346252bc-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "a4fa0509-8f85-462f-b7c3-4384346252bc" (UID: "a4fa0509-8f85-462f-b7c3-4384346252bc"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:47:42 crc kubenswrapper[5052]: I1203 11:47:42.644016 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a4fa0509-8f85-462f-b7c3-4384346252bc-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "a4fa0509-8f85-462f-b7c3-4384346252bc" (UID: "a4fa0509-8f85-462f-b7c3-4384346252bc"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:47:42 crc kubenswrapper[5052]: I1203 11:47:42.644976 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a4fa0509-8f85-462f-b7c3-4384346252bc-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "a4fa0509-8f85-462f-b7c3-4384346252bc" (UID: "a4fa0509-8f85-462f-b7c3-4384346252bc"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:47:42 crc kubenswrapper[5052]: I1203 11:47:42.647985 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a4fa0509-8f85-462f-b7c3-4384346252bc-ovndb-tls-certs" (OuterVolumeSpecName: "ovndb-tls-certs") pod "a4fa0509-8f85-462f-b7c3-4384346252bc" (UID: "a4fa0509-8f85-462f-b7c3-4384346252bc"). InnerVolumeSpecName "ovndb-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:47:42 crc kubenswrapper[5052]: I1203 11:47:42.657064 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a4fa0509-8f85-462f-b7c3-4384346252bc-config" (OuterVolumeSpecName: "config") pod "a4fa0509-8f85-462f-b7c3-4384346252bc" (UID: "a4fa0509-8f85-462f-b7c3-4384346252bc"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:47:42 crc kubenswrapper[5052]: I1203 11:47:42.686842 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-4zslh\" (UniqueName: \"kubernetes.io/projected/a4fa0509-8f85-462f-b7c3-4384346252bc-kube-api-access-4zslh\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:42 crc kubenswrapper[5052]: I1203 11:47:42.687094 5052 reconciler_common.go:293] "Volume detached for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/a4fa0509-8f85-462f-b7c3-4384346252bc-httpd-config\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:42 crc kubenswrapper[5052]: I1203 11:47:42.687208 5052 reconciler_common.go:293] "Volume detached for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/a4fa0509-8f85-462f-b7c3-4384346252bc-ovndb-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:42 crc kubenswrapper[5052]: I1203 11:47:42.687322 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a4fa0509-8f85-462f-b7c3-4384346252bc-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:42 crc kubenswrapper[5052]: I1203 11:47:42.687399 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/a4fa0509-8f85-462f-b7c3-4384346252bc-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:42 crc kubenswrapper[5052]: I1203 11:47:42.687468 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/a4fa0509-8f85-462f-b7c3-4384346252bc-config\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:42 crc kubenswrapper[5052]: I1203 11:47:42.687564 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/a4fa0509-8f85-462f-b7c3-4384346252bc-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:43 crc kubenswrapper[5052]: I1203 11:47:43.286934 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-549dfc49dd-qtk5n" event={"ID":"a4fa0509-8f85-462f-b7c3-4384346252bc","Type":"ContainerDied","Data":"d721295f825c25b28d3ee5f53df5d8daf720db66d5c9eb7c09a79bb694afc720"} Dec 03 11:47:43 crc kubenswrapper[5052]: I1203 11:47:43.286992 5052 scope.go:117] "RemoveContainer" containerID="9ca8e5c7ceff313c395a8a6c7655cdc02bfef848fc00d1d1281da272c3a6f083" Dec 03 11:47:43 crc kubenswrapper[5052]: I1203 11:47:43.287006 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-549dfc49dd-qtk5n" Dec 03 11:47:43 crc kubenswrapper[5052]: I1203 11:47:43.317504 5052 scope.go:117] "RemoveContainer" containerID="a10887f92ee55d6e2db028846bedc75386b0fc5b0814cc18643f424e258d8b3c" Dec 03 11:47:43 crc kubenswrapper[5052]: I1203 11:47:43.321311 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/neutron-549dfc49dd-qtk5n"] Dec 03 11:47:43 crc kubenswrapper[5052]: I1203 11:47:43.330677 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/neutron-549dfc49dd-qtk5n"] Dec 03 11:47:43 crc kubenswrapper[5052]: I1203 11:47:43.685349 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a4fa0509-8f85-462f-b7c3-4384346252bc" path="/var/lib/kubelet/pods/a4fa0509-8f85-462f-b7c3-4384346252bc/volumes" Dec 03 11:47:45 crc kubenswrapper[5052]: I1203 11:47:45.674012 5052 scope.go:117] "RemoveContainer" containerID="25fe88f8d102f09983f8884e65f0c6c1575ade67921f10a630e6dcc3ea8b506e" Dec 03 11:47:45 crc kubenswrapper[5052]: E1203 11:47:45.675282 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 11:47:46 crc kubenswrapper[5052]: E1203 11:47:46.513488 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="ad64b4dca94e6f61fc353d5aa054f7de7c10558b8c9671ffe67a4be04544faad" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 11:47:46 crc kubenswrapper[5052]: E1203 11:47:46.515668 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="ad64b4dca94e6f61fc353d5aa054f7de7c10558b8c9671ffe67a4be04544faad" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 11:47:46 crc kubenswrapper[5052]: E1203 11:47:46.517625 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="ad64b4dca94e6f61fc353d5aa054f7de7c10558b8c9671ffe67a4be04544faad" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 11:47:46 crc kubenswrapper[5052]: E1203 11:47:46.517719 5052 prober.go:104] "Probe errored" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" probeType="Readiness" pod="openstack-kuttl-tests/nova-cell1-conductor-0" podUID="507c60e5-056c-4661-a7a7-75c4d9bce7de" containerName="nova-cell1-conductor-conductor" Dec 03 11:47:51 crc kubenswrapper[5052]: E1203 11:47:51.513909 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="ad64b4dca94e6f61fc353d5aa054f7de7c10558b8c9671ffe67a4be04544faad" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 11:47:51 crc kubenswrapper[5052]: E1203 11:47:51.515925 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="ad64b4dca94e6f61fc353d5aa054f7de7c10558b8c9671ffe67a4be04544faad" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 11:47:51 crc kubenswrapper[5052]: E1203 11:47:51.517350 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="ad64b4dca94e6f61fc353d5aa054f7de7c10558b8c9671ffe67a4be04544faad" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 11:47:51 crc kubenswrapper[5052]: E1203 11:47:51.517399 5052 prober.go:104] "Probe errored" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" probeType="Readiness" pod="openstack-kuttl-tests/nova-cell1-conductor-0" podUID="507c60e5-056c-4661-a7a7-75c4d9bce7de" containerName="nova-cell1-conductor-conductor" Dec 03 11:47:52 crc kubenswrapper[5052]: I1203 11:47:52.380360 5052 generic.go:334] "Generic (PLEG): container finished" podID="93ca3376-6916-4fc9-8b76-7cd9a930865e" containerID="57a92592bc143e7e16a01ba9cb34e8bc93f5c7a21ccf4e346f8763a448a75b42" exitCode=137 Dec 03 11:47:52 crc kubenswrapper[5052]: I1203 11:47:52.380747 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"93ca3376-6916-4fc9-8b76-7cd9a930865e","Type":"ContainerDied","Data":"57a92592bc143e7e16a01ba9cb34e8bc93f5c7a21ccf4e346f8763a448a75b42"} Dec 03 11:47:52 crc kubenswrapper[5052]: I1203 11:47:52.613378 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-storage-0" Dec 03 11:47:52 crc kubenswrapper[5052]: E1203 11:47:52.672335 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/rabbitmq-cell1-config-data: configmap "rabbitmq-cell1-config-data" not found Dec 03 11:47:52 crc kubenswrapper[5052]: E1203 11:47:52.672469 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2766f489-9217-47ae-a595-869a26389587-config-data podName:2766f489-9217-47ae-a595-869a26389587 nodeName:}" failed. No retries permitted until 2025-12-03 11:48:24.672444423 +0000 UTC m=+1695.260686187 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "config-data" (UniqueName: "kubernetes.io/configmap/2766f489-9217-47ae-a595-869a26389587-config-data") pod "rabbitmq-cell1-server-0" (UID: "2766f489-9217-47ae-a595-869a26389587") : configmap "rabbitmq-cell1-config-data" not found Dec 03 11:47:52 crc kubenswrapper[5052]: E1203 11:47:52.673135 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/combined-ca-bundle: secret "combined-ca-bundle" not found Dec 03 11:47:52 crc kubenswrapper[5052]: E1203 11:47:52.673200 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/507c60e5-056c-4661-a7a7-75c4d9bce7de-combined-ca-bundle podName:507c60e5-056c-4661-a7a7-75c4d9bce7de nodeName:}" failed. No retries permitted until 2025-12-03 11:48:24.673185274 +0000 UTC m=+1695.261427118 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "combined-ca-bundle" (UniqueName: "kubernetes.io/secret/507c60e5-056c-4661-a7a7-75c4d9bce7de-combined-ca-bundle") pod "nova-cell1-conductor-0" (UID: "507c60e5-056c-4661-a7a7-75c4d9bce7de") : secret "combined-ca-bundle" not found Dec 03 11:47:52 crc kubenswrapper[5052]: I1203 11:47:52.773429 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6qxz4\" (UniqueName: \"kubernetes.io/projected/93ca3376-6916-4fc9-8b76-7cd9a930865e-kube-api-access-6qxz4\") pod \"93ca3376-6916-4fc9-8b76-7cd9a930865e\" (UID: \"93ca3376-6916-4fc9-8b76-7cd9a930865e\") " Dec 03 11:47:52 crc kubenswrapper[5052]: I1203 11:47:52.773598 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"swift\" (UniqueName: \"kubernetes.io/local-volume/local-storage02-crc\") pod \"93ca3376-6916-4fc9-8b76-7cd9a930865e\" (UID: \"93ca3376-6916-4fc9-8b76-7cd9a930865e\") " Dec 03 11:47:52 crc kubenswrapper[5052]: I1203 11:47:52.773658 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/93ca3376-6916-4fc9-8b76-7cd9a930865e-etc-swift\") pod \"93ca3376-6916-4fc9-8b76-7cd9a930865e\" (UID: \"93ca3376-6916-4fc9-8b76-7cd9a930865e\") " Dec 03 11:47:52 crc kubenswrapper[5052]: I1203 11:47:52.773737 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"cache\" (UniqueName: \"kubernetes.io/empty-dir/93ca3376-6916-4fc9-8b76-7cd9a930865e-cache\") pod \"93ca3376-6916-4fc9-8b76-7cd9a930865e\" (UID: \"93ca3376-6916-4fc9-8b76-7cd9a930865e\") " Dec 03 11:47:52 crc kubenswrapper[5052]: I1203 11:47:52.773802 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"lock\" (UniqueName: \"kubernetes.io/empty-dir/93ca3376-6916-4fc9-8b76-7cd9a930865e-lock\") pod \"93ca3376-6916-4fc9-8b76-7cd9a930865e\" (UID: \"93ca3376-6916-4fc9-8b76-7cd9a930865e\") " Dec 03 11:47:52 crc kubenswrapper[5052]: I1203 11:47:52.774529 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93ca3376-6916-4fc9-8b76-7cd9a930865e-lock" (OuterVolumeSpecName: "lock") pod "93ca3376-6916-4fc9-8b76-7cd9a930865e" (UID: "93ca3376-6916-4fc9-8b76-7cd9a930865e"). InnerVolumeSpecName "lock". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:47:52 crc kubenswrapper[5052]: I1203 11:47:52.774903 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93ca3376-6916-4fc9-8b76-7cd9a930865e-cache" (OuterVolumeSpecName: "cache") pod "93ca3376-6916-4fc9-8b76-7cd9a930865e" (UID: "93ca3376-6916-4fc9-8b76-7cd9a930865e"). InnerVolumeSpecName "cache". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:47:52 crc kubenswrapper[5052]: I1203 11:47:52.779993 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage02-crc" (OuterVolumeSpecName: "swift") pod "93ca3376-6916-4fc9-8b76-7cd9a930865e" (UID: "93ca3376-6916-4fc9-8b76-7cd9a930865e"). InnerVolumeSpecName "local-storage02-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 11:47:52 crc kubenswrapper[5052]: I1203 11:47:52.780471 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/93ca3376-6916-4fc9-8b76-7cd9a930865e-kube-api-access-6qxz4" (OuterVolumeSpecName: "kube-api-access-6qxz4") pod "93ca3376-6916-4fc9-8b76-7cd9a930865e" (UID: "93ca3376-6916-4fc9-8b76-7cd9a930865e"). InnerVolumeSpecName "kube-api-access-6qxz4". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:47:52 crc kubenswrapper[5052]: I1203 11:47:52.780902 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/93ca3376-6916-4fc9-8b76-7cd9a930865e-etc-swift" (OuterVolumeSpecName: "etc-swift") pod "93ca3376-6916-4fc9-8b76-7cd9a930865e" (UID: "93ca3376-6916-4fc9-8b76-7cd9a930865e"). InnerVolumeSpecName "etc-swift". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:47:52 crc kubenswrapper[5052]: I1203 11:47:52.875379 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage02-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage02-crc\") on node \"crc\" " Dec 03 11:47:52 crc kubenswrapper[5052]: I1203 11:47:52.875573 5052 reconciler_common.go:293] "Volume detached for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/93ca3376-6916-4fc9-8b76-7cd9a930865e-etc-swift\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:52 crc kubenswrapper[5052]: I1203 11:47:52.875766 5052 reconciler_common.go:293] "Volume detached for volume \"cache\" (UniqueName: \"kubernetes.io/empty-dir/93ca3376-6916-4fc9-8b76-7cd9a930865e-cache\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:52 crc kubenswrapper[5052]: I1203 11:47:52.875799 5052 reconciler_common.go:293] "Volume detached for volume \"lock\" (UniqueName: \"kubernetes.io/empty-dir/93ca3376-6916-4fc9-8b76-7cd9a930865e-lock\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:52 crc kubenswrapper[5052]: I1203 11:47:52.875811 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-6qxz4\" (UniqueName: \"kubernetes.io/projected/93ca3376-6916-4fc9-8b76-7cd9a930865e-kube-api-access-6qxz4\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:52 crc kubenswrapper[5052]: I1203 11:47:52.889700 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage02-crc" (UniqueName: "kubernetes.io/local-volume/local-storage02-crc") on node "crc" Dec 03 11:47:52 crc kubenswrapper[5052]: I1203 11:47:52.978225 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage02-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage02-crc\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:53 crc kubenswrapper[5052]: I1203 11:47:53.398308 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"93ca3376-6916-4fc9-8b76-7cd9a930865e","Type":"ContainerDied","Data":"7ec32bb8331974e634a7411cf1d4536553c9750926536e8c626baddd97b10c7a"} Dec 03 11:47:53 crc kubenswrapper[5052]: I1203 11:47:53.398609 5052 scope.go:117] "RemoveContainer" containerID="57a92592bc143e7e16a01ba9cb34e8bc93f5c7a21ccf4e346f8763a448a75b42" Dec 03 11:47:53 crc kubenswrapper[5052]: I1203 11:47:53.398620 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-storage-0" Dec 03 11:47:53 crc kubenswrapper[5052]: I1203 11:47:53.433360 5052 scope.go:117] "RemoveContainer" containerID="aca4ad76bb32ce93c82a24cd5e11df447906446998393a1992802096707a00b7" Dec 03 11:47:53 crc kubenswrapper[5052]: I1203 11:47:53.445472 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/swift-storage-0"] Dec 03 11:47:53 crc kubenswrapper[5052]: I1203 11:47:53.452920 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/swift-storage-0"] Dec 03 11:47:53 crc kubenswrapper[5052]: I1203 11:47:53.464935 5052 scope.go:117] "RemoveContainer" containerID="dcf4a9a4ecfc80507c16a16774b51db2868255753c35cb58f980eb45993a0c29" Dec 03 11:47:53 crc kubenswrapper[5052]: I1203 11:47:53.519133 5052 scope.go:117] "RemoveContainer" containerID="11b133a6cbb654d7396b3e561b4ee9d5897ec4b55a4240fcf44eff0dc1c666b1" Dec 03 11:47:53 crc kubenswrapper[5052]: I1203 11:47:53.537343 5052 scope.go:117] "RemoveContainer" containerID="7c70d2e1650a64f27e594fadf25a119e4c9f5f13e151324319b5d9429e343564" Dec 03 11:47:53 crc kubenswrapper[5052]: I1203 11:47:53.554829 5052 scope.go:117] "RemoveContainer" containerID="fdfc3ff42d3ff2045f403d93c6d32c7111bdf6fba8059c6ba854561088de3a81" Dec 03 11:47:53 crc kubenswrapper[5052]: I1203 11:47:53.573173 5052 scope.go:117] "RemoveContainer" containerID="f561321b0fdca1154a6977a337503c90f840416e5bc0fa39babc9ea06bd5b9bf" Dec 03 11:47:53 crc kubenswrapper[5052]: I1203 11:47:53.594800 5052 scope.go:117] "RemoveContainer" containerID="0049fb7f1583b92a6e9922087915b30b055b3e38241dbdef833f7e931c597b18" Dec 03 11:47:53 crc kubenswrapper[5052]: I1203 11:47:53.613870 5052 scope.go:117] "RemoveContainer" containerID="767653c60f88687aa9e17c73cfb59cad0f979a47f7e0eb1bbebd1374e7223b77" Dec 03 11:47:53 crc kubenswrapper[5052]: I1203 11:47:53.640700 5052 scope.go:117] "RemoveContainer" containerID="170a73bd9b5a2ea0ebc55d043e19812afef343218daa53630096c9db6698e952" Dec 03 11:47:53 crc kubenswrapper[5052]: I1203 11:47:53.660100 5052 scope.go:117] "RemoveContainer" containerID="05a11312e8ec969e44a91b4857ff72727254fdf705f9356f00323d2cea9feb2f" Dec 03 11:47:53 crc kubenswrapper[5052]: I1203 11:47:53.683955 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="93ca3376-6916-4fc9-8b76-7cd9a930865e" path="/var/lib/kubelet/pods/93ca3376-6916-4fc9-8b76-7cd9a930865e/volumes" Dec 03 11:47:53 crc kubenswrapper[5052]: I1203 11:47:53.684654 5052 scope.go:117] "RemoveContainer" containerID="08d93d7b7a336f74cf614cfa15c40191bdfb66f13fc15d41dcf943b57d2e92db" Dec 03 11:47:53 crc kubenswrapper[5052]: I1203 11:47:53.711097 5052 scope.go:117] "RemoveContainer" containerID="e94a57b11b76a48c619f83900e8a10f2bdcc2a6fc844e2f0bb06dd5042373965" Dec 03 11:47:53 crc kubenswrapper[5052]: I1203 11:47:53.735099 5052 scope.go:117] "RemoveContainer" containerID="f13880839e746af36615e3cd0a68485f5974bfc02f15aacef8d8c58a4cd8e970" Dec 03 11:47:53 crc kubenswrapper[5052]: I1203 11:47:53.752187 5052 scope.go:117] "RemoveContainer" containerID="15eb357802bb079afbd7911a3c2b816f45034279f8a2a7e70d89277c5699b052" Dec 03 11:47:53 crc kubenswrapper[5052]: E1203 11:47:53.793129 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cinder-scripts: secret "cinder-scripts" not found Dec 03 11:47:53 crc kubenswrapper[5052]: E1203 11:47:53.793155 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cinder-scheduler-config-data: secret "cinder-scheduler-config-data" not found Dec 03 11:47:53 crc kubenswrapper[5052]: E1203 11:47:53.793213 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e947107f-104b-4699-a994-bc77285f0cf5-scripts podName:e947107f-104b-4699-a994-bc77285f0cf5 nodeName:}" failed. No retries permitted until 2025-12-03 11:48:25.793197566 +0000 UTC m=+1696.381439330 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "scripts" (UniqueName: "kubernetes.io/secret/e947107f-104b-4699-a994-bc77285f0cf5-scripts") pod "cinder-scheduler-0" (UID: "e947107f-104b-4699-a994-bc77285f0cf5") : secret "cinder-scripts" not found Dec 03 11:47:53 crc kubenswrapper[5052]: E1203 11:47:53.793143 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cinder-config-data: secret "cinder-config-data" not found Dec 03 11:47:53 crc kubenswrapper[5052]: E1203 11:47:53.793231 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e947107f-104b-4699-a994-bc77285f0cf5-config-data-custom podName:e947107f-104b-4699-a994-bc77285f0cf5 nodeName:}" failed. No retries permitted until 2025-12-03 11:48:25.793224606 +0000 UTC m=+1696.381466370 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "config-data-custom" (UniqueName: "kubernetes.io/secret/e947107f-104b-4699-a994-bc77285f0cf5-config-data-custom") pod "cinder-scheduler-0" (UID: "e947107f-104b-4699-a994-bc77285f0cf5") : secret "cinder-scheduler-config-data" not found Dec 03 11:47:53 crc kubenswrapper[5052]: E1203 11:47:53.793266 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e947107f-104b-4699-a994-bc77285f0cf5-config-data podName:e947107f-104b-4699-a994-bc77285f0cf5 nodeName:}" failed. No retries permitted until 2025-12-03 11:48:25.793244537 +0000 UTC m=+1696.381486311 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "config-data" (UniqueName: "kubernetes.io/secret/e947107f-104b-4699-a994-bc77285f0cf5-config-data") pod "cinder-scheduler-0" (UID: "e947107f-104b-4699-a994-bc77285f0cf5") : secret "cinder-config-data" not found Dec 03 11:47:53 crc kubenswrapper[5052]: E1203 11:47:53.793165 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/combined-ca-bundle: secret "combined-ca-bundle" not found Dec 03 11:47:53 crc kubenswrapper[5052]: E1203 11:47:53.793298 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e947107f-104b-4699-a994-bc77285f0cf5-combined-ca-bundle podName:e947107f-104b-4699-a994-bc77285f0cf5 nodeName:}" failed. No retries permitted until 2025-12-03 11:48:25.793290558 +0000 UTC m=+1696.381532342 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "combined-ca-bundle" (UniqueName: "kubernetes.io/secret/e947107f-104b-4699-a994-bc77285f0cf5-combined-ca-bundle") pod "cinder-scheduler-0" (UID: "e947107f-104b-4699-a994-bc77285f0cf5") : secret "combined-ca-bundle" not found Dec 03 11:47:54 crc kubenswrapper[5052]: I1203 11:47:54.410848 5052 generic.go:334] "Generic (PLEG): container finished" podID="e947107f-104b-4699-a994-bc77285f0cf5" containerID="30b3c8981cdec3a0ba91236f985bc49025e43effcb7cbf0b9e7cddcc3ba0a0d9" exitCode=137 Dec 03 11:47:54 crc kubenswrapper[5052]: I1203 11:47:54.410919 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"e947107f-104b-4699-a994-bc77285f0cf5","Type":"ContainerDied","Data":"30b3c8981cdec3a0ba91236f985bc49025e43effcb7cbf0b9e7cddcc3ba0a0d9"} Dec 03 11:47:54 crc kubenswrapper[5052]: I1203 11:47:54.411393 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"e947107f-104b-4699-a994-bc77285f0cf5","Type":"ContainerDied","Data":"4603e5e66c2357efa797fc51b20cdd6bd29abce2aa0235abf027cb539fb8a8a3"} Dec 03 11:47:54 crc kubenswrapper[5052]: I1203 11:47:54.411428 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4603e5e66c2357efa797fc51b20cdd6bd29abce2aa0235abf027cb539fb8a8a3" Dec 03 11:47:54 crc kubenswrapper[5052]: I1203 11:47:54.430829 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:47:54 crc kubenswrapper[5052]: I1203 11:47:54.502394 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/e947107f-104b-4699-a994-bc77285f0cf5-config-data-custom\") pod \"e947107f-104b-4699-a994-bc77285f0cf5\" (UID: \"e947107f-104b-4699-a994-bc77285f0cf5\") " Dec 03 11:47:54 crc kubenswrapper[5052]: I1203 11:47:54.502445 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e947107f-104b-4699-a994-bc77285f0cf5-combined-ca-bundle\") pod \"e947107f-104b-4699-a994-bc77285f0cf5\" (UID: \"e947107f-104b-4699-a994-bc77285f0cf5\") " Dec 03 11:47:54 crc kubenswrapper[5052]: I1203 11:47:54.502489 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/e947107f-104b-4699-a994-bc77285f0cf5-etc-machine-id\") pod \"e947107f-104b-4699-a994-bc77285f0cf5\" (UID: \"e947107f-104b-4699-a994-bc77285f0cf5\") " Dec 03 11:47:54 crc kubenswrapper[5052]: I1203 11:47:54.502590 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wnktp\" (UniqueName: \"kubernetes.io/projected/e947107f-104b-4699-a994-bc77285f0cf5-kube-api-access-wnktp\") pod \"e947107f-104b-4699-a994-bc77285f0cf5\" (UID: \"e947107f-104b-4699-a994-bc77285f0cf5\") " Dec 03 11:47:54 crc kubenswrapper[5052]: I1203 11:47:54.503190 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/e947107f-104b-4699-a994-bc77285f0cf5-scripts\") pod \"e947107f-104b-4699-a994-bc77285f0cf5\" (UID: \"e947107f-104b-4699-a994-bc77285f0cf5\") " Dec 03 11:47:54 crc kubenswrapper[5052]: I1203 11:47:54.502656 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/e947107f-104b-4699-a994-bc77285f0cf5-etc-machine-id" (OuterVolumeSpecName: "etc-machine-id") pod "e947107f-104b-4699-a994-bc77285f0cf5" (UID: "e947107f-104b-4699-a994-bc77285f0cf5"). InnerVolumeSpecName "etc-machine-id". PluginName "kubernetes.io/host-path", VolumeGidValue "" Dec 03 11:47:54 crc kubenswrapper[5052]: I1203 11:47:54.503247 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/e947107f-104b-4699-a994-bc77285f0cf5-config-data\") pod \"e947107f-104b-4699-a994-bc77285f0cf5\" (UID: \"e947107f-104b-4699-a994-bc77285f0cf5\") " Dec 03 11:47:54 crc kubenswrapper[5052]: I1203 11:47:54.503755 5052 reconciler_common.go:293] "Volume detached for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/e947107f-104b-4699-a994-bc77285f0cf5-etc-machine-id\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:54 crc kubenswrapper[5052]: I1203 11:47:54.509367 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e947107f-104b-4699-a994-bc77285f0cf5-scripts" (OuterVolumeSpecName: "scripts") pod "e947107f-104b-4699-a994-bc77285f0cf5" (UID: "e947107f-104b-4699-a994-bc77285f0cf5"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:47:54 crc kubenswrapper[5052]: I1203 11:47:54.509390 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e947107f-104b-4699-a994-bc77285f0cf5-kube-api-access-wnktp" (OuterVolumeSpecName: "kube-api-access-wnktp") pod "e947107f-104b-4699-a994-bc77285f0cf5" (UID: "e947107f-104b-4699-a994-bc77285f0cf5"). InnerVolumeSpecName "kube-api-access-wnktp". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:47:54 crc kubenswrapper[5052]: I1203 11:47:54.509772 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e947107f-104b-4699-a994-bc77285f0cf5-config-data-custom" (OuterVolumeSpecName: "config-data-custom") pod "e947107f-104b-4699-a994-bc77285f0cf5" (UID: "e947107f-104b-4699-a994-bc77285f0cf5"). InnerVolumeSpecName "config-data-custom". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:47:54 crc kubenswrapper[5052]: I1203 11:47:54.552583 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e947107f-104b-4699-a994-bc77285f0cf5-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "e947107f-104b-4699-a994-bc77285f0cf5" (UID: "e947107f-104b-4699-a994-bc77285f0cf5"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:47:54 crc kubenswrapper[5052]: I1203 11:47:54.575726 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e947107f-104b-4699-a994-bc77285f0cf5-config-data" (OuterVolumeSpecName: "config-data") pod "e947107f-104b-4699-a994-bc77285f0cf5" (UID: "e947107f-104b-4699-a994-bc77285f0cf5"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:47:54 crc kubenswrapper[5052]: I1203 11:47:54.605870 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-wnktp\" (UniqueName: \"kubernetes.io/projected/e947107f-104b-4699-a994-bc77285f0cf5-kube-api-access-wnktp\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:54 crc kubenswrapper[5052]: I1203 11:47:54.605907 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/e947107f-104b-4699-a994-bc77285f0cf5-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:54 crc kubenswrapper[5052]: I1203 11:47:54.605921 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/e947107f-104b-4699-a994-bc77285f0cf5-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:54 crc kubenswrapper[5052]: I1203 11:47:54.606088 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/e947107f-104b-4699-a994-bc77285f0cf5-config-data-custom\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:54 crc kubenswrapper[5052]: I1203 11:47:54.606107 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e947107f-104b-4699-a994-bc77285f0cf5-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:47:55 crc kubenswrapper[5052]: I1203 11:47:55.425134 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:47:55 crc kubenswrapper[5052]: I1203 11:47:55.462141 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 11:47:55 crc kubenswrapper[5052]: I1203 11:47:55.472113 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 11:47:55 crc kubenswrapper[5052]: I1203 11:47:55.685445 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e947107f-104b-4699-a994-bc77285f0cf5" path="/var/lib/kubelet/pods/e947107f-104b-4699-a994-bc77285f0cf5/volumes" Dec 03 11:47:56 crc kubenswrapper[5052]: E1203 11:47:56.516650 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="ad64b4dca94e6f61fc353d5aa054f7de7c10558b8c9671ffe67a4be04544faad" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 11:47:56 crc kubenswrapper[5052]: E1203 11:47:56.518759 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="ad64b4dca94e6f61fc353d5aa054f7de7c10558b8c9671ffe67a4be04544faad" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 11:47:56 crc kubenswrapper[5052]: E1203 11:47:56.520525 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="ad64b4dca94e6f61fc353d5aa054f7de7c10558b8c9671ffe67a4be04544faad" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 11:47:56 crc kubenswrapper[5052]: E1203 11:47:56.520669 5052 prober.go:104] "Probe errored" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" probeType="Readiness" pod="openstack-kuttl-tests/nova-cell1-conductor-0" podUID="507c60e5-056c-4661-a7a7-75c4d9bce7de" containerName="nova-cell1-conductor-conductor" Dec 03 11:47:59 crc kubenswrapper[5052]: I1203 11:47:59.678847 5052 scope.go:117] "RemoveContainer" containerID="25fe88f8d102f09983f8884e65f0c6c1575ade67921f10a630e6dcc3ea8b506e" Dec 03 11:47:59 crc kubenswrapper[5052]: E1203 11:47:59.679324 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 11:48:01 crc kubenswrapper[5052]: E1203 11:48:01.512886 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="ad64b4dca94e6f61fc353d5aa054f7de7c10558b8c9671ffe67a4be04544faad" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 11:48:01 crc kubenswrapper[5052]: E1203 11:48:01.515539 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="ad64b4dca94e6f61fc353d5aa054f7de7c10558b8c9671ffe67a4be04544faad" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 11:48:01 crc kubenswrapper[5052]: E1203 11:48:01.516687 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="ad64b4dca94e6f61fc353d5aa054f7de7c10558b8c9671ffe67a4be04544faad" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 11:48:01 crc kubenswrapper[5052]: E1203 11:48:01.516727 5052 prober.go:104] "Probe errored" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" probeType="Readiness" pod="openstack-kuttl-tests/nova-cell1-conductor-0" podUID="507c60e5-056c-4661-a7a7-75c4d9bce7de" containerName="nova-cell1-conductor-conductor" Dec 03 11:48:03 crc kubenswrapper[5052]: I1203 11:48:03.498200 5052 generic.go:334] "Generic (PLEG): container finished" podID="507c60e5-056c-4661-a7a7-75c4d9bce7de" containerID="ad64b4dca94e6f61fc353d5aa054f7de7c10558b8c9671ffe67a4be04544faad" exitCode=137 Dec 03 11:48:03 crc kubenswrapper[5052]: I1203 11:48:03.498326 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-conductor-0" event={"ID":"507c60e5-056c-4661-a7a7-75c4d9bce7de","Type":"ContainerDied","Data":"ad64b4dca94e6f61fc353d5aa054f7de7c10558b8c9671ffe67a4be04544faad"} Dec 03 11:48:03 crc kubenswrapper[5052]: I1203 11:48:03.996884 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 11:48:04 crc kubenswrapper[5052]: I1203 11:48:04.074070 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/507c60e5-056c-4661-a7a7-75c4d9bce7de-config-data\") pod \"507c60e5-056c-4661-a7a7-75c4d9bce7de\" (UID: \"507c60e5-056c-4661-a7a7-75c4d9bce7de\") " Dec 03 11:48:04 crc kubenswrapper[5052]: I1203 11:48:04.074874 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9cbzw\" (UniqueName: \"kubernetes.io/projected/507c60e5-056c-4661-a7a7-75c4d9bce7de-kube-api-access-9cbzw\") pod \"507c60e5-056c-4661-a7a7-75c4d9bce7de\" (UID: \"507c60e5-056c-4661-a7a7-75c4d9bce7de\") " Dec 03 11:48:04 crc kubenswrapper[5052]: I1203 11:48:04.075166 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/507c60e5-056c-4661-a7a7-75c4d9bce7de-combined-ca-bundle\") pod \"507c60e5-056c-4661-a7a7-75c4d9bce7de\" (UID: \"507c60e5-056c-4661-a7a7-75c4d9bce7de\") " Dec 03 11:48:04 crc kubenswrapper[5052]: I1203 11:48:04.080061 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/507c60e5-056c-4661-a7a7-75c4d9bce7de-kube-api-access-9cbzw" (OuterVolumeSpecName: "kube-api-access-9cbzw") pod "507c60e5-056c-4661-a7a7-75c4d9bce7de" (UID: "507c60e5-056c-4661-a7a7-75c4d9bce7de"). InnerVolumeSpecName "kube-api-access-9cbzw". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:48:04 crc kubenswrapper[5052]: I1203 11:48:04.096551 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/507c60e5-056c-4661-a7a7-75c4d9bce7de-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "507c60e5-056c-4661-a7a7-75c4d9bce7de" (UID: "507c60e5-056c-4661-a7a7-75c4d9bce7de"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:48:04 crc kubenswrapper[5052]: I1203 11:48:04.103367 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/507c60e5-056c-4661-a7a7-75c4d9bce7de-config-data" (OuterVolumeSpecName: "config-data") pod "507c60e5-056c-4661-a7a7-75c4d9bce7de" (UID: "507c60e5-056c-4661-a7a7-75c4d9bce7de"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:48:04 crc kubenswrapper[5052]: I1203 11:48:04.178214 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/507c60e5-056c-4661-a7a7-75c4d9bce7de-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:48:04 crc kubenswrapper[5052]: I1203 11:48:04.178262 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/507c60e5-056c-4661-a7a7-75c4d9bce7de-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:48:04 crc kubenswrapper[5052]: I1203 11:48:04.178393 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-9cbzw\" (UniqueName: \"kubernetes.io/projected/507c60e5-056c-4661-a7a7-75c4d9bce7de-kube-api-access-9cbzw\") on node \"crc\" DevicePath \"\"" Dec 03 11:48:04 crc kubenswrapper[5052]: I1203 11:48:04.514648 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-conductor-0" event={"ID":"507c60e5-056c-4661-a7a7-75c4d9bce7de","Type":"ContainerDied","Data":"231dd4c4a99d89479c2a56cf5b64bd4cfe99fd1b95806cca5cada9901e47dcc1"} Dec 03 11:48:04 crc kubenswrapper[5052]: I1203 11:48:04.514716 5052 scope.go:117] "RemoveContainer" containerID="ad64b4dca94e6f61fc353d5aa054f7de7c10558b8c9671ffe67a4be04544faad" Dec 03 11:48:04 crc kubenswrapper[5052]: I1203 11:48:04.514759 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 11:48:04 crc kubenswrapper[5052]: I1203 11:48:04.554599 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-0"] Dec 03 11:48:04 crc kubenswrapper[5052]: I1203 11:48:04.567079 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-0"] Dec 03 11:48:05 crc kubenswrapper[5052]: I1203 11:48:05.685841 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="507c60e5-056c-4661-a7a7-75c4d9bce7de" path="/var/lib/kubelet/pods/507c60e5-056c-4661-a7a7-75c4d9bce7de/volumes" Dec 03 11:48:12 crc kubenswrapper[5052]: I1203 11:48:12.673609 5052 scope.go:117] "RemoveContainer" containerID="25fe88f8d102f09983f8884e65f0c6c1575ade67921f10a630e6dcc3ea8b506e" Dec 03 11:48:12 crc kubenswrapper[5052]: E1203 11:48:12.674674 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 11:48:23 crc kubenswrapper[5052]: I1203 11:48:23.673787 5052 scope.go:117] "RemoveContainer" containerID="25fe88f8d102f09983f8884e65f0c6c1575ade67921f10a630e6dcc3ea8b506e" Dec 03 11:48:23 crc kubenswrapper[5052]: E1203 11:48:23.674989 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 11:48:24 crc kubenswrapper[5052]: E1203 11:48:24.744975 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/rabbitmq-cell1-config-data: configmap "rabbitmq-cell1-config-data" not found Dec 03 11:48:24 crc kubenswrapper[5052]: E1203 11:48:24.745441 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2766f489-9217-47ae-a595-869a26389587-config-data podName:2766f489-9217-47ae-a595-869a26389587 nodeName:}" failed. No retries permitted until 2025-12-03 11:49:28.745402608 +0000 UTC m=+1759.333644372 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "config-data" (UniqueName: "kubernetes.io/configmap/2766f489-9217-47ae-a595-869a26389587-config-data") pod "rabbitmq-cell1-server-0" (UID: "2766f489-9217-47ae-a595-869a26389587") : configmap "rabbitmq-cell1-config-data" not found Dec 03 11:48:25 crc kubenswrapper[5052]: E1203 11:48:25.752738 5052 handlers.go:78] "Exec lifecycle hook for Container in Pod failed" err=< Dec 03 11:48:25 crc kubenswrapper[5052]: command '/bin/bash -c if [ ! -z "$(cat /etc/pod-info/skipPreStopChecks)" ]; then exit 0; fi; rabbitmq-upgrade await_online_quorum_plus_one -t 604800 && rabbitmq-upgrade await_online_synchronized_mirror -t 604800 || true && rabbitmq-upgrade drain -t 604800' exited with 69: Error: unable to perform an operation on node 'rabbit@rabbitmq-cell1-server-0.rabbitmq-cell1-nodes.openstack-kuttl-tests'. Please see diagnostics information and suggestions below. Dec 03 11:48:25 crc kubenswrapper[5052]: Dec 03 11:48:25 crc kubenswrapper[5052]: Most common reasons for this are: Dec 03 11:48:25 crc kubenswrapper[5052]: Dec 03 11:48:25 crc kubenswrapper[5052]: * Target node is unreachable (e.g. due to hostname resolution, TCP connection or firewall issues) Dec 03 11:48:25 crc kubenswrapper[5052]: * CLI tool fails to authenticate with the server (e.g. due to CLI tool's Erlang cookie not matching that of the server) Dec 03 11:48:25 crc kubenswrapper[5052]: * Target node is not running Dec 03 11:48:25 crc kubenswrapper[5052]: Dec 03 11:48:25 crc kubenswrapper[5052]: In addition to the diagnostics info below: Dec 03 11:48:25 crc kubenswrapper[5052]: Dec 03 11:48:25 crc kubenswrapper[5052]: * See the CLI, clustering and networking guides on https://rabbitmq.com/documentation.html to learn more Dec 03 11:48:25 crc kubenswrapper[5052]: * Consult server logs on node rabbit@rabbitmq-cell1-server-0.rabbitmq-cell1-nodes.openstack-kuttl-tests Dec 03 11:48:25 crc kubenswrapper[5052]: * If target node is configured to use long node names, don't forget to use --longnames with CLI tools Dec 03 11:48:25 crc kubenswrapper[5052]: Dec 03 11:48:25 crc kubenswrapper[5052]: DIAGNOSTICS Dec 03 11:48:25 crc kubenswrapper[5052]: =========== Dec 03 11:48:25 crc kubenswrapper[5052]: Dec 03 11:48:25 crc kubenswrapper[5052]: attempted to contact: ['rabbit@rabbitmq-cell1-server-0.rabbitmq-cell1-nodes.openstack-kuttl-tests'] Dec 03 11:48:25 crc kubenswrapper[5052]: Dec 03 11:48:25 crc kubenswrapper[5052]: rabbit@rabbitmq-cell1-server-0.rabbitmq-cell1-nodes.openstack-kuttl-tests: Dec 03 11:48:25 crc kubenswrapper[5052]: * unable to connect to epmd (port 4369) on rabbitmq-cell1-server-0.rabbitmq-cell1-nodes.openstack-kuttl-tests: nxdomain (non-existing domain) Dec 03 11:48:25 crc kubenswrapper[5052]: Dec 03 11:48:25 crc kubenswrapper[5052]: Dec 03 11:48:25 crc kubenswrapper[5052]: Current node details: Dec 03 11:48:25 crc kubenswrapper[5052]: * node name: 'rabbitmqcli-692-rabbit@rabbitmq-cell1-server-0.rabbitmq-cell1-nodes.openstack-kuttl-tests' Dec 03 11:48:25 crc kubenswrapper[5052]: * effective user's home directory: /var/lib/rabbitmq Dec 03 11:48:25 crc kubenswrapper[5052]: * Erlang cookie hash: ke/4/tH1IG3RdXe51PKnxQ== Dec 03 11:48:25 crc kubenswrapper[5052]: Dec 03 11:48:25 crc kubenswrapper[5052]: Error: unable to perform an operation on node 'rabbit@rabbitmq-cell1-server-0.rabbitmq-cell1-nodes.openstack-kuttl-tests'. Please see diagnostics information and suggestions below. Dec 03 11:48:25 crc kubenswrapper[5052]: Dec 03 11:48:25 crc kubenswrapper[5052]: Most common reasons for this are: Dec 03 11:48:25 crc kubenswrapper[5052]: Dec 03 11:48:25 crc kubenswrapper[5052]: * Target node is unreachable (e.g. due to hostname resolution, TCP connection or firewall issues) Dec 03 11:48:25 crc kubenswrapper[5052]: * CLI tool fails to authenticate with the server (e.g. due to CLI tool's Erlang cookie not matching that of the server) Dec 03 11:48:25 crc kubenswrapper[5052]: * Target node is not running Dec 03 11:48:25 crc kubenswrapper[5052]: Dec 03 11:48:25 crc kubenswrapper[5052]: In addition to the diagnostics info below: Dec 03 11:48:25 crc kubenswrapper[5052]: Dec 03 11:48:25 crc kubenswrapper[5052]: * See the CLI, clustering and networking guides on https://rabbitmq.com/documentation.html to learn more Dec 03 11:48:25 crc kubenswrapper[5052]: * Consult server logs on node rabbit@rabbitmq-cell1-server-0.rabbitmq-cell1-nodes.openstack-kuttl-tests Dec 03 11:48:25 crc kubenswrapper[5052]: * If target node is configured to use long node names, don't forget to use --longnames with CLI tools Dec 03 11:48:25 crc kubenswrapper[5052]: Dec 03 11:48:25 crc kubenswrapper[5052]: DIAGNOSTICS Dec 03 11:48:25 crc kubenswrapper[5052]: =========== Dec 03 11:48:25 crc kubenswrapper[5052]: Dec 03 11:48:25 crc kubenswrapper[5052]: attempted to contact: ['rabbit@rabbitmq-cell1-server-0.rabbitmq-cell1-nodes.openstack-kuttl-tests'] Dec 03 11:48:25 crc kubenswrapper[5052]: Dec 03 11:48:25 crc kubenswrapper[5052]: rabbit@rabbitmq-cell1-server-0.rabbitmq-cell1-nodes.openstack-kuttl-tests: Dec 03 11:48:25 crc kubenswrapper[5052]: * unable to connect to epmd (port 4369) on rabbitmq-cell1-server-0.rabbitmq-cell1-nodes.openstack-kuttl-tests: nxdomain (non-existing domain) Dec 03 11:48:25 crc kubenswrapper[5052]: Dec 03 11:48:25 crc kubenswrapper[5052]: Dec 03 11:48:25 crc kubenswrapper[5052]: Current node details: Dec 03 11:48:25 crc kubenswrapper[5052]: * node name: 'rabbitmqcli-935-rabbit@rabbitmq-cell1-server-0.rabbitmq-cell1-nodes.openstack-kuttl-tests' Dec 03 11:48:25 crc kubenswrapper[5052]: * effective user's home directory: /var/lib/rabbitmq Dec 03 11:48:25 crc kubenswrapper[5052]: * Erlang cookie hash: ke/4/tH1IG3RdXe51PKnxQ== Dec 03 11:48:25 crc kubenswrapper[5052]: Dec 03 11:48:25 crc kubenswrapper[5052]: > execCommand=["/bin/bash","-c","if [ ! -z \"$(cat /etc/pod-info/skipPreStopChecks)\" ]; then exit 0; fi; rabbitmq-upgrade await_online_quorum_plus_one -t 604800 \u0026\u0026 rabbitmq-upgrade await_online_synchronized_mirror -t 604800 || true \u0026\u0026 rabbitmq-upgrade drain -t 604800"] containerName="rabbitmq" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" message=< Dec 03 11:48:25 crc kubenswrapper[5052]: Will put node rabbit@rabbitmq-cell1-server-0.rabbitmq-cell1-nodes.openstack-kuttl-tests into maintenance mode. The node will no longer serve any client traffic! Dec 03 11:48:25 crc kubenswrapper[5052]: Error: unable to perform an operation on node 'rabbit@rabbitmq-cell1-server-0.rabbitmq-cell1-nodes.openstack-kuttl-tests'. Please see diagnostics information and suggestions below. Dec 03 11:48:25 crc kubenswrapper[5052]: Dec 03 11:48:25 crc kubenswrapper[5052]: Most common reasons for this are: Dec 03 11:48:25 crc kubenswrapper[5052]: Dec 03 11:48:25 crc kubenswrapper[5052]: * Target node is unreachable (e.g. due to hostname resolution, TCP connection or firewall issues) Dec 03 11:48:25 crc kubenswrapper[5052]: * CLI tool fails to authenticate with the server (e.g. due to CLI tool's Erlang cookie not matching that of the server) Dec 03 11:48:25 crc kubenswrapper[5052]: * Target node is not running Dec 03 11:48:25 crc kubenswrapper[5052]: Dec 03 11:48:25 crc kubenswrapper[5052]: In addition to the diagnostics info below: Dec 03 11:48:25 crc kubenswrapper[5052]: Dec 03 11:48:25 crc kubenswrapper[5052]: * See the CLI, clustering and networking guides on https://rabbitmq.com/documentation.html to learn more Dec 03 11:48:25 crc kubenswrapper[5052]: * Consult server logs on node rabbit@rabbitmq-cell1-server-0.rabbitmq-cell1-nodes.openstack-kuttl-tests Dec 03 11:48:25 crc kubenswrapper[5052]: * If target node is configured to use long node names, don't forget to use --longnames with CLI tools Dec 03 11:48:25 crc kubenswrapper[5052]: Dec 03 11:48:25 crc kubenswrapper[5052]: DIAGNOSTICS Dec 03 11:48:25 crc kubenswrapper[5052]: =========== Dec 03 11:48:25 crc kubenswrapper[5052]: Dec 03 11:48:25 crc kubenswrapper[5052]: attempted to contact: ['rabbit@rabbitmq-cell1-server-0.rabbitmq-cell1-nodes.openstack-kuttl-tests'] Dec 03 11:48:25 crc kubenswrapper[5052]: Dec 03 11:48:25 crc kubenswrapper[5052]: rabbit@rabbitmq-cell1-server-0.rabbitmq-cell1-nodes.openstack-kuttl-tests: Dec 03 11:48:25 crc kubenswrapper[5052]: * unable to connect to epmd (port 4369) on rabbitmq-cell1-server-0.rabbitmq-cell1-nodes.openstack-kuttl-tests: nxdomain (non-existing domain) Dec 03 11:48:25 crc kubenswrapper[5052]: Dec 03 11:48:25 crc kubenswrapper[5052]: Dec 03 11:48:25 crc kubenswrapper[5052]: Current node details: Dec 03 11:48:25 crc kubenswrapper[5052]: * node name: 'rabbitmqcli-692-rabbit@rabbitmq-cell1-server-0.rabbitmq-cell1-nodes.openstack-kuttl-tests' Dec 03 11:48:25 crc kubenswrapper[5052]: * effective user's home directory: /var/lib/rabbitmq Dec 03 11:48:25 crc kubenswrapper[5052]: * Erlang cookie hash: ke/4/tH1IG3RdXe51PKnxQ== Dec 03 11:48:25 crc kubenswrapper[5052]: Dec 03 11:48:25 crc kubenswrapper[5052]: Error: unable to perform an operation on node 'rabbit@rabbitmq-cell1-server-0.rabbitmq-cell1-nodes.openstack-kuttl-tests'. Please see diagnostics information and suggestions below. Dec 03 11:48:25 crc kubenswrapper[5052]: Dec 03 11:48:25 crc kubenswrapper[5052]: Most common reasons for this are: Dec 03 11:48:25 crc kubenswrapper[5052]: Dec 03 11:48:25 crc kubenswrapper[5052]: * Target node is unreachable (e.g. due to hostname resolution, TCP connection or firewall issues) Dec 03 11:48:25 crc kubenswrapper[5052]: * CLI tool fails to authenticate with the server (e.g. due to CLI tool's Erlang cookie not matching that of the server) Dec 03 11:48:25 crc kubenswrapper[5052]: * Target node is not running Dec 03 11:48:25 crc kubenswrapper[5052]: Dec 03 11:48:25 crc kubenswrapper[5052]: In addition to the diagnostics info below: Dec 03 11:48:25 crc kubenswrapper[5052]: Dec 03 11:48:25 crc kubenswrapper[5052]: * See the CLI, clustering and networking guides on https://rabbitmq.com/documentation.html to learn more Dec 03 11:48:25 crc kubenswrapper[5052]: * Consult server logs on node rabbit@rabbitmq-cell1-server-0.rabbitmq-cell1-nodes.openstack-kuttl-tests Dec 03 11:48:25 crc kubenswrapper[5052]: * If target node is configured to use long node names, don't forget to use --longnames with CLI tools Dec 03 11:48:25 crc kubenswrapper[5052]: Dec 03 11:48:25 crc kubenswrapper[5052]: DIAGNOSTICS Dec 03 11:48:25 crc kubenswrapper[5052]: =========== Dec 03 11:48:25 crc kubenswrapper[5052]: Dec 03 11:48:25 crc kubenswrapper[5052]: attempted to contact: ['rabbit@rabbitmq-cell1-server-0.rabbitmq-cell1-nodes.openstack-kuttl-tests'] Dec 03 11:48:25 crc kubenswrapper[5052]: Dec 03 11:48:25 crc kubenswrapper[5052]: rabbit@rabbitmq-cell1-server-0.rabbitmq-cell1-nodes.openstack-kuttl-tests: Dec 03 11:48:25 crc kubenswrapper[5052]: * unable to connect to epmd (port 4369) on rabbitmq-cell1-server-0.rabbitmq-cell1-nodes.openstack-kuttl-tests: nxdomain (non-existing domain) Dec 03 11:48:25 crc kubenswrapper[5052]: Dec 03 11:48:25 crc kubenswrapper[5052]: Dec 03 11:48:25 crc kubenswrapper[5052]: Current node details: Dec 03 11:48:25 crc kubenswrapper[5052]: * node name: 'rabbitmqcli-935-rabbit@rabbitmq-cell1-server-0.rabbitmq-cell1-nodes.openstack-kuttl-tests' Dec 03 11:48:25 crc kubenswrapper[5052]: * effective user's home directory: /var/lib/rabbitmq Dec 03 11:48:25 crc kubenswrapper[5052]: * Erlang cookie hash: ke/4/tH1IG3RdXe51PKnxQ== Dec 03 11:48:25 crc kubenswrapper[5052]: Dec 03 11:48:25 crc kubenswrapper[5052]: > Dec 03 11:48:25 crc kubenswrapper[5052]: E1203 11:48:25.752814 5052 kuberuntime_container.go:691] "PreStop hook failed" err=< Dec 03 11:48:25 crc kubenswrapper[5052]: command '/bin/bash -c if [ ! -z "$(cat /etc/pod-info/skipPreStopChecks)" ]; then exit 0; fi; rabbitmq-upgrade await_online_quorum_plus_one -t 604800 && rabbitmq-upgrade await_online_synchronized_mirror -t 604800 || true && rabbitmq-upgrade drain -t 604800' exited with 69: Error: unable to perform an operation on node 'rabbit@rabbitmq-cell1-server-0.rabbitmq-cell1-nodes.openstack-kuttl-tests'. Please see diagnostics information and suggestions below. Dec 03 11:48:25 crc kubenswrapper[5052]: Dec 03 11:48:25 crc kubenswrapper[5052]: Most common reasons for this are: Dec 03 11:48:25 crc kubenswrapper[5052]: Dec 03 11:48:25 crc kubenswrapper[5052]: * Target node is unreachable (e.g. due to hostname resolution, TCP connection or firewall issues) Dec 03 11:48:25 crc kubenswrapper[5052]: * CLI tool fails to authenticate with the server (e.g. due to CLI tool's Erlang cookie not matching that of the server) Dec 03 11:48:25 crc kubenswrapper[5052]: * Target node is not running Dec 03 11:48:25 crc kubenswrapper[5052]: Dec 03 11:48:25 crc kubenswrapper[5052]: In addition to the diagnostics info below: Dec 03 11:48:25 crc kubenswrapper[5052]: Dec 03 11:48:25 crc kubenswrapper[5052]: * See the CLI, clustering and networking guides on https://rabbitmq.com/documentation.html to learn more Dec 03 11:48:25 crc kubenswrapper[5052]: * Consult server logs on node rabbit@rabbitmq-cell1-server-0.rabbitmq-cell1-nodes.openstack-kuttl-tests Dec 03 11:48:25 crc kubenswrapper[5052]: * If target node is configured to use long node names, don't forget to use --longnames with CLI tools Dec 03 11:48:25 crc kubenswrapper[5052]: Dec 03 11:48:25 crc kubenswrapper[5052]: DIAGNOSTICS Dec 03 11:48:25 crc kubenswrapper[5052]: =========== Dec 03 11:48:25 crc kubenswrapper[5052]: Dec 03 11:48:25 crc kubenswrapper[5052]: attempted to contact: ['rabbit@rabbitmq-cell1-server-0.rabbitmq-cell1-nodes.openstack-kuttl-tests'] Dec 03 11:48:25 crc kubenswrapper[5052]: Dec 03 11:48:25 crc kubenswrapper[5052]: rabbit@rabbitmq-cell1-server-0.rabbitmq-cell1-nodes.openstack-kuttl-tests: Dec 03 11:48:25 crc kubenswrapper[5052]: * unable to connect to epmd (port 4369) on rabbitmq-cell1-server-0.rabbitmq-cell1-nodes.openstack-kuttl-tests: nxdomain (non-existing domain) Dec 03 11:48:25 crc kubenswrapper[5052]: Dec 03 11:48:25 crc kubenswrapper[5052]: Dec 03 11:48:25 crc kubenswrapper[5052]: Current node details: Dec 03 11:48:25 crc kubenswrapper[5052]: * node name: 'rabbitmqcli-692-rabbit@rabbitmq-cell1-server-0.rabbitmq-cell1-nodes.openstack-kuttl-tests' Dec 03 11:48:25 crc kubenswrapper[5052]: * effective user's home directory: /var/lib/rabbitmq Dec 03 11:48:25 crc kubenswrapper[5052]: * Erlang cookie hash: ke/4/tH1IG3RdXe51PKnxQ== Dec 03 11:48:25 crc kubenswrapper[5052]: Dec 03 11:48:25 crc kubenswrapper[5052]: Error: unable to perform an operation on node 'rabbit@rabbitmq-cell1-server-0.rabbitmq-cell1-nodes.openstack-kuttl-tests'. Please see diagnostics information and suggestions below. Dec 03 11:48:25 crc kubenswrapper[5052]: Dec 03 11:48:25 crc kubenswrapper[5052]: Most common reasons for this are: Dec 03 11:48:25 crc kubenswrapper[5052]: Dec 03 11:48:25 crc kubenswrapper[5052]: * Target node is unreachable (e.g. due to hostname resolution, TCP connection or firewall issues) Dec 03 11:48:25 crc kubenswrapper[5052]: * CLI tool fails to authenticate with the server (e.g. due to CLI tool's Erlang cookie not matching that of the server) Dec 03 11:48:25 crc kubenswrapper[5052]: * Target node is not running Dec 03 11:48:25 crc kubenswrapper[5052]: Dec 03 11:48:25 crc kubenswrapper[5052]: In addition to the diagnostics info below: Dec 03 11:48:25 crc kubenswrapper[5052]: Dec 03 11:48:25 crc kubenswrapper[5052]: * See the CLI, clustering and networking guides on https://rabbitmq.com/documentation.html to learn more Dec 03 11:48:25 crc kubenswrapper[5052]: * Consult server logs on node rabbit@rabbitmq-cell1-server-0.rabbitmq-cell1-nodes.openstack-kuttl-tests Dec 03 11:48:25 crc kubenswrapper[5052]: * If target node is configured to use long node names, don't forget to use --longnames with CLI tools Dec 03 11:48:25 crc kubenswrapper[5052]: Dec 03 11:48:25 crc kubenswrapper[5052]: DIAGNOSTICS Dec 03 11:48:25 crc kubenswrapper[5052]: =========== Dec 03 11:48:25 crc kubenswrapper[5052]: Dec 03 11:48:25 crc kubenswrapper[5052]: attempted to contact: ['rabbit@rabbitmq-cell1-server-0.rabbitmq-cell1-nodes.openstack-kuttl-tests'] Dec 03 11:48:25 crc kubenswrapper[5052]: Dec 03 11:48:25 crc kubenswrapper[5052]: rabbit@rabbitmq-cell1-server-0.rabbitmq-cell1-nodes.openstack-kuttl-tests: Dec 03 11:48:25 crc kubenswrapper[5052]: * unable to connect to epmd (port 4369) on rabbitmq-cell1-server-0.rabbitmq-cell1-nodes.openstack-kuttl-tests: nxdomain (non-existing domain) Dec 03 11:48:25 crc kubenswrapper[5052]: Dec 03 11:48:25 crc kubenswrapper[5052]: Dec 03 11:48:25 crc kubenswrapper[5052]: Current node details: Dec 03 11:48:25 crc kubenswrapper[5052]: * node name: 'rabbitmqcli-935-rabbit@rabbitmq-cell1-server-0.rabbitmq-cell1-nodes.openstack-kuttl-tests' Dec 03 11:48:25 crc kubenswrapper[5052]: * effective user's home directory: /var/lib/rabbitmq Dec 03 11:48:25 crc kubenswrapper[5052]: * Erlang cookie hash: ke/4/tH1IG3RdXe51PKnxQ== Dec 03 11:48:25 crc kubenswrapper[5052]: Dec 03 11:48:25 crc kubenswrapper[5052]: > pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" podUID="2766f489-9217-47ae-a595-869a26389587" containerName="rabbitmq" containerID="cri-o://9725070208b6721d2746b3710550f0679a73b1a1ff7fc62cba3c5b2a68f65659" Dec 03 11:48:25 crc kubenswrapper[5052]: I1203 11:48:25.752872 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" podUID="2766f489-9217-47ae-a595-869a26389587" containerName="rabbitmq" containerID="cri-o://9725070208b6721d2746b3710550f0679a73b1a1ff7fc62cba3c5b2a68f65659" gracePeriod=604739 Dec 03 11:48:30 crc kubenswrapper[5052]: I1203 11:48:30.826133 5052 scope.go:117] "RemoveContainer" containerID="9a73f69a43329d67c0d372934e250290b21f5530d0b0079f8192b4709695cb6a" Dec 03 11:48:30 crc kubenswrapper[5052]: I1203 11:48:30.858669 5052 scope.go:117] "RemoveContainer" containerID="98a5d3b986b2516d0ac8f36fbeebe27e3364347af72f582505d55d4a22ea7445" Dec 03 11:48:30 crc kubenswrapper[5052]: I1203 11:48:30.923342 5052 scope.go:117] "RemoveContainer" containerID="ed4175dc9ade01ddd2b17bd932170a14f62de0010dc8527f7fa2c37dde74bfe6" Dec 03 11:48:30 crc kubenswrapper[5052]: I1203 11:48:30.951068 5052 scope.go:117] "RemoveContainer" containerID="7474492a6f12a4bba8719b2ffab92d689c3e76fc1c263f7a31eea5c422579780" Dec 03 11:48:30 crc kubenswrapper[5052]: I1203 11:48:30.976505 5052 scope.go:117] "RemoveContainer" containerID="9de6d546e32a6ebadd1f211b6d25ef1622cc49f940869c042b2e0e10bb7d3cb9" Dec 03 11:48:31 crc kubenswrapper[5052]: I1203 11:48:31.017105 5052 scope.go:117] "RemoveContainer" containerID="1fddb059a8118b5a724a5d5a74bfd9dcb6fdac1b88a0a1f2bd904834de863a32" Dec 03 11:48:31 crc kubenswrapper[5052]: I1203 11:48:31.040053 5052 scope.go:117] "RemoveContainer" containerID="2be85ceb7d3cd57a9befa38f17da587b414e83a180cebc28727fc4a5996939d1" Dec 03 11:48:31 crc kubenswrapper[5052]: I1203 11:48:31.061327 5052 scope.go:117] "RemoveContainer" containerID="910971ac3056ce793483491994f03b4796ccbd03d513c9fc09a55fb5a371b881" Dec 03 11:48:31 crc kubenswrapper[5052]: I1203 11:48:31.083105 5052 scope.go:117] "RemoveContainer" containerID="0bcbc83b612aeb3a9d9d8d6b5d25c20cfdd3b0b8c6002297d10049035fb2b217" Dec 03 11:48:31 crc kubenswrapper[5052]: I1203 11:48:31.105874 5052 scope.go:117] "RemoveContainer" containerID="b2697ef9ab442e1740e9992a4ae8a2ad183a400760ac5eea2d853dc944ed7fa8" Dec 03 11:48:31 crc kubenswrapper[5052]: I1203 11:48:31.121564 5052 scope.go:117] "RemoveContainer" containerID="aa0bc44913bc443884760e2a18faf9da4a9e4a859e8c121cd05e1cd62282505e" Dec 03 11:48:31 crc kubenswrapper[5052]: I1203 11:48:31.136146 5052 scope.go:117] "RemoveContainer" containerID="7d22ae141a66b8ccb952b7ed5aa56004799cd8db057a131cc24fa1331ea7e389" Dec 03 11:48:31 crc kubenswrapper[5052]: I1203 11:48:31.157647 5052 scope.go:117] "RemoveContainer" containerID="018800c14e3ce4d36361a2c7660a63547b7a9c5239ad164f8b420f98bb52cd55" Dec 03 11:48:31 crc kubenswrapper[5052]: I1203 11:48:31.176205 5052 scope.go:117] "RemoveContainer" containerID="1b0b626fe0bfe23d2e33c4a429245669c17a1a07a4936903e8ddb5ec47f1b434" Dec 03 11:48:32 crc kubenswrapper[5052]: I1203 11:48:32.820470 5052 generic.go:334] "Generic (PLEG): container finished" podID="2766f489-9217-47ae-a595-869a26389587" containerID="9725070208b6721d2746b3710550f0679a73b1a1ff7fc62cba3c5b2a68f65659" exitCode=0 Dec 03 11:48:32 crc kubenswrapper[5052]: I1203 11:48:32.820566 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" event={"ID":"2766f489-9217-47ae-a595-869a26389587","Type":"ContainerDied","Data":"9725070208b6721d2746b3710550f0679a73b1a1ff7fc62cba3c5b2a68f65659"} Dec 03 11:48:33 crc kubenswrapper[5052]: I1203 11:48:33.185486 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 11:48:33 crc kubenswrapper[5052]: I1203 11:48:33.292463 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"rabbitmq-plugins\" (UniqueName: \"kubernetes.io/empty-dir/2766f489-9217-47ae-a595-869a26389587-rabbitmq-plugins\") pod \"2766f489-9217-47ae-a595-869a26389587\" (UID: \"2766f489-9217-47ae-a595-869a26389587\") " Dec 03 11:48:33 crc kubenswrapper[5052]: I1203 11:48:33.292520 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"rabbitmq-erlang-cookie\" (UniqueName: \"kubernetes.io/empty-dir/2766f489-9217-47ae-a595-869a26389587-rabbitmq-erlang-cookie\") pod \"2766f489-9217-47ae-a595-869a26389587\" (UID: \"2766f489-9217-47ae-a595-869a26389587\") " Dec 03 11:48:33 crc kubenswrapper[5052]: I1203 11:48:33.292552 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"pod-info\" (UniqueName: \"kubernetes.io/downward-api/2766f489-9217-47ae-a595-869a26389587-pod-info\") pod \"2766f489-9217-47ae-a595-869a26389587\" (UID: \"2766f489-9217-47ae-a595-869a26389587\") " Dec 03 11:48:33 crc kubenswrapper[5052]: I1203 11:48:33.292608 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"rabbitmq-tls\" (UniqueName: \"kubernetes.io/projected/2766f489-9217-47ae-a595-869a26389587-rabbitmq-tls\") pod \"2766f489-9217-47ae-a595-869a26389587\" (UID: \"2766f489-9217-47ae-a595-869a26389587\") " Dec 03 11:48:33 crc kubenswrapper[5052]: I1203 11:48:33.292639 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"server-conf\" (UniqueName: \"kubernetes.io/configmap/2766f489-9217-47ae-a595-869a26389587-server-conf\") pod \"2766f489-9217-47ae-a595-869a26389587\" (UID: \"2766f489-9217-47ae-a595-869a26389587\") " Dec 03 11:48:33 crc kubenswrapper[5052]: I1203 11:48:33.292671 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"persistence\" (UniqueName: \"kubernetes.io/local-volume/local-storage18-crc\") pod \"2766f489-9217-47ae-a595-869a26389587\" (UID: \"2766f489-9217-47ae-a595-869a26389587\") " Dec 03 11:48:33 crc kubenswrapper[5052]: I1203 11:48:33.292728 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/2766f489-9217-47ae-a595-869a26389587-config-data\") pod \"2766f489-9217-47ae-a595-869a26389587\" (UID: \"2766f489-9217-47ae-a595-869a26389587\") " Dec 03 11:48:33 crc kubenswrapper[5052]: I1203 11:48:33.292780 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"erlang-cookie-secret\" (UniqueName: \"kubernetes.io/secret/2766f489-9217-47ae-a595-869a26389587-erlang-cookie-secret\") pod \"2766f489-9217-47ae-a595-869a26389587\" (UID: \"2766f489-9217-47ae-a595-869a26389587\") " Dec 03 11:48:33 crc kubenswrapper[5052]: I1203 11:48:33.292798 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"plugins-conf\" (UniqueName: \"kubernetes.io/configmap/2766f489-9217-47ae-a595-869a26389587-plugins-conf\") pod \"2766f489-9217-47ae-a595-869a26389587\" (UID: \"2766f489-9217-47ae-a595-869a26389587\") " Dec 03 11:48:33 crc kubenswrapper[5052]: I1203 11:48:33.292849 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nkgvw\" (UniqueName: \"kubernetes.io/projected/2766f489-9217-47ae-a595-869a26389587-kube-api-access-nkgvw\") pod \"2766f489-9217-47ae-a595-869a26389587\" (UID: \"2766f489-9217-47ae-a595-869a26389587\") " Dec 03 11:48:33 crc kubenswrapper[5052]: I1203 11:48:33.292878 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"rabbitmq-confd\" (UniqueName: \"kubernetes.io/projected/2766f489-9217-47ae-a595-869a26389587-rabbitmq-confd\") pod \"2766f489-9217-47ae-a595-869a26389587\" (UID: \"2766f489-9217-47ae-a595-869a26389587\") " Dec 03 11:48:33 crc kubenswrapper[5052]: I1203 11:48:33.294066 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2766f489-9217-47ae-a595-869a26389587-plugins-conf" (OuterVolumeSpecName: "plugins-conf") pod "2766f489-9217-47ae-a595-869a26389587" (UID: "2766f489-9217-47ae-a595-869a26389587"). InnerVolumeSpecName "plugins-conf". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:48:33 crc kubenswrapper[5052]: I1203 11:48:33.294489 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2766f489-9217-47ae-a595-869a26389587-rabbitmq-erlang-cookie" (OuterVolumeSpecName: "rabbitmq-erlang-cookie") pod "2766f489-9217-47ae-a595-869a26389587" (UID: "2766f489-9217-47ae-a595-869a26389587"). InnerVolumeSpecName "rabbitmq-erlang-cookie". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:48:33 crc kubenswrapper[5052]: I1203 11:48:33.294937 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2766f489-9217-47ae-a595-869a26389587-rabbitmq-plugins" (OuterVolumeSpecName: "rabbitmq-plugins") pod "2766f489-9217-47ae-a595-869a26389587" (UID: "2766f489-9217-47ae-a595-869a26389587"). InnerVolumeSpecName "rabbitmq-plugins". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:48:33 crc kubenswrapper[5052]: I1203 11:48:33.298789 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2766f489-9217-47ae-a595-869a26389587-erlang-cookie-secret" (OuterVolumeSpecName: "erlang-cookie-secret") pod "2766f489-9217-47ae-a595-869a26389587" (UID: "2766f489-9217-47ae-a595-869a26389587"). InnerVolumeSpecName "erlang-cookie-secret". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:48:33 crc kubenswrapper[5052]: I1203 11:48:33.298999 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2766f489-9217-47ae-a595-869a26389587-pod-info" (OuterVolumeSpecName: "pod-info") pod "2766f489-9217-47ae-a595-869a26389587" (UID: "2766f489-9217-47ae-a595-869a26389587"). InnerVolumeSpecName "pod-info". PluginName "kubernetes.io/downward-api", VolumeGidValue "" Dec 03 11:48:33 crc kubenswrapper[5052]: I1203 11:48:33.299974 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2766f489-9217-47ae-a595-869a26389587-kube-api-access-nkgvw" (OuterVolumeSpecName: "kube-api-access-nkgvw") pod "2766f489-9217-47ae-a595-869a26389587" (UID: "2766f489-9217-47ae-a595-869a26389587"). InnerVolumeSpecName "kube-api-access-nkgvw". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:48:33 crc kubenswrapper[5052]: I1203 11:48:33.301374 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage18-crc" (OuterVolumeSpecName: "persistence") pod "2766f489-9217-47ae-a595-869a26389587" (UID: "2766f489-9217-47ae-a595-869a26389587"). InnerVolumeSpecName "local-storage18-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 11:48:33 crc kubenswrapper[5052]: I1203 11:48:33.301574 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2766f489-9217-47ae-a595-869a26389587-rabbitmq-tls" (OuterVolumeSpecName: "rabbitmq-tls") pod "2766f489-9217-47ae-a595-869a26389587" (UID: "2766f489-9217-47ae-a595-869a26389587"). InnerVolumeSpecName "rabbitmq-tls". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:48:33 crc kubenswrapper[5052]: I1203 11:48:33.316351 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2766f489-9217-47ae-a595-869a26389587-config-data" (OuterVolumeSpecName: "config-data") pod "2766f489-9217-47ae-a595-869a26389587" (UID: "2766f489-9217-47ae-a595-869a26389587"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:48:33 crc kubenswrapper[5052]: I1203 11:48:33.342813 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2766f489-9217-47ae-a595-869a26389587-server-conf" (OuterVolumeSpecName: "server-conf") pod "2766f489-9217-47ae-a595-869a26389587" (UID: "2766f489-9217-47ae-a595-869a26389587"). InnerVolumeSpecName "server-conf". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:48:33 crc kubenswrapper[5052]: I1203 11:48:33.394268 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/2766f489-9217-47ae-a595-869a26389587-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:48:33 crc kubenswrapper[5052]: I1203 11:48:33.394538 5052 reconciler_common.go:293] "Volume detached for volume \"erlang-cookie-secret\" (UniqueName: \"kubernetes.io/secret/2766f489-9217-47ae-a595-869a26389587-erlang-cookie-secret\") on node \"crc\" DevicePath \"\"" Dec 03 11:48:33 crc kubenswrapper[5052]: I1203 11:48:33.394609 5052 reconciler_common.go:293] "Volume detached for volume \"plugins-conf\" (UniqueName: \"kubernetes.io/configmap/2766f489-9217-47ae-a595-869a26389587-plugins-conf\") on node \"crc\" DevicePath \"\"" Dec 03 11:48:33 crc kubenswrapper[5052]: I1203 11:48:33.394675 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-nkgvw\" (UniqueName: \"kubernetes.io/projected/2766f489-9217-47ae-a595-869a26389587-kube-api-access-nkgvw\") on node \"crc\" DevicePath \"\"" Dec 03 11:48:33 crc kubenswrapper[5052]: I1203 11:48:33.394739 5052 reconciler_common.go:293] "Volume detached for volume \"rabbitmq-plugins\" (UniqueName: \"kubernetes.io/empty-dir/2766f489-9217-47ae-a595-869a26389587-rabbitmq-plugins\") on node \"crc\" DevicePath \"\"" Dec 03 11:48:33 crc kubenswrapper[5052]: I1203 11:48:33.394796 5052 reconciler_common.go:293] "Volume detached for volume \"rabbitmq-erlang-cookie\" (UniqueName: \"kubernetes.io/empty-dir/2766f489-9217-47ae-a595-869a26389587-rabbitmq-erlang-cookie\") on node \"crc\" DevicePath \"\"" Dec 03 11:48:33 crc kubenswrapper[5052]: I1203 11:48:33.394872 5052 reconciler_common.go:293] "Volume detached for volume \"pod-info\" (UniqueName: \"kubernetes.io/downward-api/2766f489-9217-47ae-a595-869a26389587-pod-info\") on node \"crc\" DevicePath \"\"" Dec 03 11:48:33 crc kubenswrapper[5052]: I1203 11:48:33.394930 5052 reconciler_common.go:293] "Volume detached for volume \"rabbitmq-tls\" (UniqueName: \"kubernetes.io/projected/2766f489-9217-47ae-a595-869a26389587-rabbitmq-tls\") on node \"crc\" DevicePath \"\"" Dec 03 11:48:33 crc kubenswrapper[5052]: I1203 11:48:33.394988 5052 reconciler_common.go:293] "Volume detached for volume \"server-conf\" (UniqueName: \"kubernetes.io/configmap/2766f489-9217-47ae-a595-869a26389587-server-conf\") on node \"crc\" DevicePath \"\"" Dec 03 11:48:33 crc kubenswrapper[5052]: I1203 11:48:33.395134 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage18-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage18-crc\") on node \"crc\" " Dec 03 11:48:33 crc kubenswrapper[5052]: I1203 11:48:33.396370 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2766f489-9217-47ae-a595-869a26389587-rabbitmq-confd" (OuterVolumeSpecName: "rabbitmq-confd") pod "2766f489-9217-47ae-a595-869a26389587" (UID: "2766f489-9217-47ae-a595-869a26389587"). InnerVolumeSpecName "rabbitmq-confd". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:48:33 crc kubenswrapper[5052]: I1203 11:48:33.415161 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage18-crc" (UniqueName: "kubernetes.io/local-volume/local-storage18-crc") on node "crc" Dec 03 11:48:33 crc kubenswrapper[5052]: I1203 11:48:33.497032 5052 reconciler_common.go:293] "Volume detached for volume \"rabbitmq-confd\" (UniqueName: \"kubernetes.io/projected/2766f489-9217-47ae-a595-869a26389587-rabbitmq-confd\") on node \"crc\" DevicePath \"\"" Dec 03 11:48:33 crc kubenswrapper[5052]: I1203 11:48:33.497477 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage18-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage18-crc\") on node \"crc\" DevicePath \"\"" Dec 03 11:48:33 crc kubenswrapper[5052]: I1203 11:48:33.836859 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" event={"ID":"2766f489-9217-47ae-a595-869a26389587","Type":"ContainerDied","Data":"01a30dbd657eec98b6030db3277bea3757b501e6d8f09d2fab2597c1ed8d6dfb"} Dec 03 11:48:33 crc kubenswrapper[5052]: I1203 11:48:33.836928 5052 scope.go:117] "RemoveContainer" containerID="9725070208b6721d2746b3710550f0679a73b1a1ff7fc62cba3c5b2a68f65659" Dec 03 11:48:33 crc kubenswrapper[5052]: I1203 11:48:33.836952 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 11:48:33 crc kubenswrapper[5052]: I1203 11:48:33.861026 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/rabbitmq-cell1-server-0"] Dec 03 11:48:33 crc kubenswrapper[5052]: I1203 11:48:33.864014 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/rabbitmq-cell1-server-0"] Dec 03 11:48:33 crc kubenswrapper[5052]: I1203 11:48:33.879227 5052 scope.go:117] "RemoveContainer" containerID="3b75a48ddef04dba6740dfd97e3821f7c14d8f167622e54e606f6aae843251a9" Dec 03 11:48:35 crc kubenswrapper[5052]: I1203 11:48:35.696180 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2766f489-9217-47ae-a595-869a26389587" path="/var/lib/kubelet/pods/2766f489-9217-47ae-a595-869a26389587/volumes" Dec 03 11:48:36 crc kubenswrapper[5052]: I1203 11:48:36.674226 5052 scope.go:117] "RemoveContainer" containerID="25fe88f8d102f09983f8884e65f0c6c1575ade67921f10a630e6dcc3ea8b506e" Dec 03 11:48:36 crc kubenswrapper[5052]: E1203 11:48:36.674957 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 11:48:40 crc kubenswrapper[5052]: I1203 11:48:40.874812 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["crc-storage/crc-storage-crc-g5t8m"] Dec 03 11:48:40 crc kubenswrapper[5052]: I1203 11:48:40.879798 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["crc-storage/crc-storage-crc-g5t8m"] Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.009521 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["crc-storage/crc-storage-crc-8dcm2"] Dec 03 11:48:41 crc kubenswrapper[5052]: E1203 11:48:41.009791 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="1b45577a-92b7-4c67-941b-cfc2fbfa3eb2" containerName="sg-core" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.009803 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="1b45577a-92b7-4c67-941b-cfc2fbfa3eb2" containerName="sg-core" Dec 03 11:48:41 crc kubenswrapper[5052]: E1203 11:48:41.009819 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="950f0626-40d2-41c9-9701-cba50843c43e" containerName="mariadb-account-delete" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.009825 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="950f0626-40d2-41c9-9701-cba50843c43e" containerName="mariadb-account-delete" Dec 03 11:48:41 crc kubenswrapper[5052]: E1203 11:48:41.009831 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="e947107f-104b-4699-a994-bc77285f0cf5" containerName="probe" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.009836 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="e947107f-104b-4699-a994-bc77285f0cf5" containerName="probe" Dec 03 11:48:41 crc kubenswrapper[5052]: E1203 11:48:41.009847 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="0b94b517-5f9b-485d-a109-0a11e15a3ad3" containerName="mariadb-account-delete" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.009854 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="0b94b517-5f9b-485d-a109-0a11e15a3ad3" containerName="mariadb-account-delete" Dec 03 11:48:41 crc kubenswrapper[5052]: E1203 11:48:41.009860 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="9bdd2626-de50-4513-8313-3b25347ff641" containerName="ovsdbserver-sb" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.009866 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="9bdd2626-de50-4513-8313-3b25347ff641" containerName="ovsdbserver-sb" Dec 03 11:48:41 crc kubenswrapper[5052]: E1203 11:48:41.009876 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="a4fa0509-8f85-462f-b7c3-4384346252bc" containerName="neutron-api" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.009882 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="a4fa0509-8f85-462f-b7c3-4384346252bc" containerName="neutron-api" Dec 03 11:48:41 crc kubenswrapper[5052]: E1203 11:48:41.009890 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="84b8e975-30ee-42c6-8952-25aebf2c378b" containerName="nova-cell1-novncproxy-novncproxy" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.009896 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="84b8e975-30ee-42c6-8952-25aebf2c378b" containerName="nova-cell1-novncproxy-novncproxy" Dec 03 11:48:41 crc kubenswrapper[5052]: E1203 11:48:41.009907 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="d6d9150c-e5b6-407a-b20a-4114a60312f8" containerName="nova-cell0-conductor-conductor" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.009913 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="d6d9150c-e5b6-407a-b20a-4114a60312f8" containerName="nova-cell0-conductor-conductor" Dec 03 11:48:41 crc kubenswrapper[5052]: E1203 11:48:41.009922 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="93ca3376-6916-4fc9-8b76-7cd9a930865e" containerName="account-auditor" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.009927 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="93ca3376-6916-4fc9-8b76-7cd9a930865e" containerName="account-auditor" Dec 03 11:48:41 crc kubenswrapper[5052]: E1203 11:48:41.009938 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="93ca3376-6916-4fc9-8b76-7cd9a930865e" containerName="object-server" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.009943 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="93ca3376-6916-4fc9-8b76-7cd9a930865e" containerName="object-server" Dec 03 11:48:41 crc kubenswrapper[5052]: E1203 11:48:41.009950 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="00b12739-1437-4af3-ab6d-ad830b67ad2e" containerName="cinder-api-log" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.009955 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="00b12739-1437-4af3-ab6d-ad830b67ad2e" containerName="cinder-api-log" Dec 03 11:48:41 crc kubenswrapper[5052]: E1203 11:48:41.009962 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="e5339997-33d5-4794-8fa6-042b63e9391e" containerName="mariadb-account-delete" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.009968 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="e5339997-33d5-4794-8fa6-042b63e9391e" containerName="mariadb-account-delete" Dec 03 11:48:41 crc kubenswrapper[5052]: E1203 11:48:41.009975 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="e741a60a-1a4f-45a1-b315-064fb7458e8f" containerName="mysql-bootstrap" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.009982 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="e741a60a-1a4f-45a1-b315-064fb7458e8f" containerName="mysql-bootstrap" Dec 03 11:48:41 crc kubenswrapper[5052]: E1203 11:48:41.009991 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="2766f489-9217-47ae-a595-869a26389587" containerName="setup-container" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.009997 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="2766f489-9217-47ae-a595-869a26389587" containerName="setup-container" Dec 03 11:48:41 crc kubenswrapper[5052]: E1203 11:48:41.010008 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="8c2252c0-5e30-4e38-b1a1-af0242df6ac0" containerName="mariadb-account-delete" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.010014 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="8c2252c0-5e30-4e38-b1a1-af0242df6ac0" containerName="mariadb-account-delete" Dec 03 11:48:41 crc kubenswrapper[5052]: E1203 11:48:41.010023 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="00b12739-1437-4af3-ab6d-ad830b67ad2e" containerName="cinder-api" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.010030 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="00b12739-1437-4af3-ab6d-ad830b67ad2e" containerName="cinder-api" Dec 03 11:48:41 crc kubenswrapper[5052]: E1203 11:48:41.010036 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="1b45577a-92b7-4c67-941b-cfc2fbfa3eb2" containerName="ceilometer-central-agent" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.010042 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="1b45577a-92b7-4c67-941b-cfc2fbfa3eb2" containerName="ceilometer-central-agent" Dec 03 11:48:41 crc kubenswrapper[5052]: E1203 11:48:41.010048 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="0914ebfe-13c6-4309-bd10-a67850124e3f" containerName="glance-httpd" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.010054 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="0914ebfe-13c6-4309-bd10-a67850124e3f" containerName="glance-httpd" Dec 03 11:48:41 crc kubenswrapper[5052]: E1203 11:48:41.010061 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="2766f489-9217-47ae-a595-869a26389587" containerName="rabbitmq" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.010066 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="2766f489-9217-47ae-a595-869a26389587" containerName="rabbitmq" Dec 03 11:48:41 crc kubenswrapper[5052]: E1203 11:48:41.010078 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="9bdd2626-de50-4513-8313-3b25347ff641" containerName="openstack-network-exporter" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.010084 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="9bdd2626-de50-4513-8313-3b25347ff641" containerName="openstack-network-exporter" Dec 03 11:48:41 crc kubenswrapper[5052]: E1203 11:48:41.010093 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="496a1006-5380-40d1-998f-129a87233300" containerName="kube-state-metrics" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.010099 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="496a1006-5380-40d1-998f-129a87233300" containerName="kube-state-metrics" Dec 03 11:48:41 crc kubenswrapper[5052]: E1203 11:48:41.010111 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="93ca3376-6916-4fc9-8b76-7cd9a930865e" containerName="account-reaper" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.010116 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="93ca3376-6916-4fc9-8b76-7cd9a930865e" containerName="account-reaper" Dec 03 11:48:41 crc kubenswrapper[5052]: E1203 11:48:41.010123 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="950f0626-40d2-41c9-9701-cba50843c43e" containerName="mariadb-account-delete" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.010128 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="950f0626-40d2-41c9-9701-cba50843c43e" containerName="mariadb-account-delete" Dec 03 11:48:41 crc kubenswrapper[5052]: E1203 11:48:41.010135 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="a4fa0509-8f85-462f-b7c3-4384346252bc" containerName="neutron-httpd" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.010141 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="a4fa0509-8f85-462f-b7c3-4384346252bc" containerName="neutron-httpd" Dec 03 11:48:41 crc kubenswrapper[5052]: E1203 11:48:41.010150 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="bbbc503e-d33d-42a9-856a-9ecc1d059f6b" containerName="dnsmasq-dns" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.010156 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="bbbc503e-d33d-42a9-856a-9ecc1d059f6b" containerName="dnsmasq-dns" Dec 03 11:48:41 crc kubenswrapper[5052]: E1203 11:48:41.010165 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="83be8c2e-57f8-4b45-b3ca-096f72a1defd" containerName="mysql-bootstrap" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.010171 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="83be8c2e-57f8-4b45-b3ca-096f72a1defd" containerName="mysql-bootstrap" Dec 03 11:48:41 crc kubenswrapper[5052]: E1203 11:48:41.010177 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="93ca3376-6916-4fc9-8b76-7cd9a930865e" containerName="rsync" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.010182 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="93ca3376-6916-4fc9-8b76-7cd9a930865e" containerName="rsync" Dec 03 11:48:41 crc kubenswrapper[5052]: E1203 11:48:41.010190 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="0183ebb5-b91b-443a-9535-2008c9207ab1" containerName="placement-log" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.010196 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="0183ebb5-b91b-443a-9535-2008c9207ab1" containerName="placement-log" Dec 03 11:48:41 crc kubenswrapper[5052]: E1203 11:48:41.010207 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="0914ebfe-13c6-4309-bd10-a67850124e3f" containerName="glance-log" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.010213 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="0914ebfe-13c6-4309-bd10-a67850124e3f" containerName="glance-log" Dec 03 11:48:41 crc kubenswrapper[5052]: E1203 11:48:41.010221 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="2d3a3054-6c8e-4808-bd5d-01413970167c" containerName="mariadb-account-delete" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.010227 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d3a3054-6c8e-4808-bd5d-01413970167c" containerName="mariadb-account-delete" Dec 03 11:48:41 crc kubenswrapper[5052]: E1203 11:48:41.010234 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="84957e18-5747-4786-9c4f-6af023122726" containerName="proxy-server" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.010240 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="84957e18-5747-4786-9c4f-6af023122726" containerName="proxy-server" Dec 03 11:48:41 crc kubenswrapper[5052]: E1203 11:48:41.010248 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="acea154c-f0bb-4066-bd4d-2dda815f301c" containerName="glance-httpd" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.010253 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="acea154c-f0bb-4066-bd4d-2dda815f301c" containerName="glance-httpd" Dec 03 11:48:41 crc kubenswrapper[5052]: E1203 11:48:41.010261 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="93ca3376-6916-4fc9-8b76-7cd9a930865e" containerName="container-server" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.010266 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="93ca3376-6916-4fc9-8b76-7cd9a930865e" containerName="container-server" Dec 03 11:48:41 crc kubenswrapper[5052]: E1203 11:48:41.010274 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="93ca3376-6916-4fc9-8b76-7cd9a930865e" containerName="object-updater" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.010280 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="93ca3376-6916-4fc9-8b76-7cd9a930865e" containerName="object-updater" Dec 03 11:48:41 crc kubenswrapper[5052]: E1203 11:48:41.010287 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="0183ebb5-b91b-443a-9535-2008c9207ab1" containerName="placement-api" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.010293 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="0183ebb5-b91b-443a-9535-2008c9207ab1" containerName="placement-api" Dec 03 11:48:41 crc kubenswrapper[5052]: E1203 11:48:41.010299 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="93ca3376-6916-4fc9-8b76-7cd9a930865e" containerName="container-updater" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.010304 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="93ca3376-6916-4fc9-8b76-7cd9a930865e" containerName="container-updater" Dec 03 11:48:41 crc kubenswrapper[5052]: E1203 11:48:41.010311 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="e947107f-104b-4699-a994-bc77285f0cf5" containerName="cinder-scheduler" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.010316 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="e947107f-104b-4699-a994-bc77285f0cf5" containerName="cinder-scheduler" Dec 03 11:48:41 crc kubenswrapper[5052]: E1203 11:48:41.010322 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="93ca3376-6916-4fc9-8b76-7cd9a930865e" containerName="account-replicator" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.010328 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="93ca3376-6916-4fc9-8b76-7cd9a930865e" containerName="account-replicator" Dec 03 11:48:41 crc kubenswrapper[5052]: E1203 11:48:41.010334 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0" containerName="setup-container" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.010340 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0" containerName="setup-container" Dec 03 11:48:41 crc kubenswrapper[5052]: E1203 11:48:41.010348 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="65be66f1-c32d-40b5-81a4-a63258364d9d" containerName="keystone-api" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.010354 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="65be66f1-c32d-40b5-81a4-a63258364d9d" containerName="keystone-api" Dec 03 11:48:41 crc kubenswrapper[5052]: E1203 11:48:41.010362 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="68f829f3-e972-4168-8b75-27c086a91142" containerName="ovsdbserver-nb" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.010369 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="68f829f3-e972-4168-8b75-27c086a91142" containerName="ovsdbserver-nb" Dec 03 11:48:41 crc kubenswrapper[5052]: E1203 11:48:41.010377 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="cadac9a6-1d82-403a-b7a0-692a424dd0e2" containerName="barbican-worker-log" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.010385 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="cadac9a6-1d82-403a-b7a0-692a424dd0e2" containerName="barbican-worker-log" Dec 03 11:48:41 crc kubenswrapper[5052]: E1203 11:48:41.010401 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="2d3a3054-6c8e-4808-bd5d-01413970167c" containerName="mariadb-account-delete" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.010409 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d3a3054-6c8e-4808-bd5d-01413970167c" containerName="mariadb-account-delete" Dec 03 11:48:41 crc kubenswrapper[5052]: E1203 11:48:41.010421 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0" containerName="rabbitmq" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.010427 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0" containerName="rabbitmq" Dec 03 11:48:41 crc kubenswrapper[5052]: E1203 11:48:41.010434 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="bb51da3b-72cc-4008-aac7-737f6a3031cb" containerName="nova-metadata-log" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.010441 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb51da3b-72cc-4008-aac7-737f6a3031cb" containerName="nova-metadata-log" Dec 03 11:48:41 crc kubenswrapper[5052]: E1203 11:48:41.010448 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="83be8c2e-57f8-4b45-b3ca-096f72a1defd" containerName="galera" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.010455 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="83be8c2e-57f8-4b45-b3ca-096f72a1defd" containerName="galera" Dec 03 11:48:41 crc kubenswrapper[5052]: E1203 11:48:41.010468 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="93ca3376-6916-4fc9-8b76-7cd9a930865e" containerName="container-replicator" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.010474 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="93ca3376-6916-4fc9-8b76-7cd9a930865e" containerName="container-replicator" Dec 03 11:48:41 crc kubenswrapper[5052]: E1203 11:48:41.010482 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="6f9b208b-086f-4671-8c3f-b3a275740ad7" containerName="memcached" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.010489 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="6f9b208b-086f-4671-8c3f-b3a275740ad7" containerName="memcached" Dec 03 11:48:41 crc kubenswrapper[5052]: E1203 11:48:41.010497 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="83a10afe-30d8-443d-a4f7-28784a2ea068" containerName="ovn-northd" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.010503 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="83a10afe-30d8-443d-a4f7-28784a2ea068" containerName="ovn-northd" Dec 03 11:48:41 crc kubenswrapper[5052]: E1203 11:48:41.010533 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="93ca3376-6916-4fc9-8b76-7cd9a930865e" containerName="object-expirer" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.010541 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="93ca3376-6916-4fc9-8b76-7cd9a930865e" containerName="object-expirer" Dec 03 11:48:41 crc kubenswrapper[5052]: E1203 11:48:41.010552 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="93ca3376-6916-4fc9-8b76-7cd9a930865e" containerName="container-auditor" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.010558 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="93ca3376-6916-4fc9-8b76-7cd9a930865e" containerName="container-auditor" Dec 03 11:48:41 crc kubenswrapper[5052]: E1203 11:48:41.010564 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="93ca3376-6916-4fc9-8b76-7cd9a930865e" containerName="object-auditor" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.010570 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="93ca3376-6916-4fc9-8b76-7cd9a930865e" containerName="object-auditor" Dec 03 11:48:41 crc kubenswrapper[5052]: E1203 11:48:41.010578 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="d8de865a-0f6c-413d-8a37-3ec4d496c4bb" containerName="barbican-keystone-listener-log" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.010585 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="d8de865a-0f6c-413d-8a37-3ec4d496c4bb" containerName="barbican-keystone-listener-log" Dec 03 11:48:41 crc kubenswrapper[5052]: E1203 11:48:41.010599 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="0aefd6f6-cbb3-4c45-9f69-7124e43255e3" containerName="mariadb-account-delete" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.010607 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="0aefd6f6-cbb3-4c45-9f69-7124e43255e3" containerName="mariadb-account-delete" Dec 03 11:48:41 crc kubenswrapper[5052]: E1203 11:48:41.010620 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="68f829f3-e972-4168-8b75-27c086a91142" containerName="openstack-network-exporter" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.010627 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="68f829f3-e972-4168-8b75-27c086a91142" containerName="openstack-network-exporter" Dec 03 11:48:41 crc kubenswrapper[5052]: E1203 11:48:41.010635 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="aa9a649b-9229-4cdd-8e27-8aedd48f4bbd" containerName="mariadb-account-delete" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.010641 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa9a649b-9229-4cdd-8e27-8aedd48f4bbd" containerName="mariadb-account-delete" Dec 03 11:48:41 crc kubenswrapper[5052]: E1203 11:48:41.010648 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="93ca3376-6916-4fc9-8b76-7cd9a930865e" containerName="swift-recon-cron" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.010653 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="93ca3376-6916-4fc9-8b76-7cd9a930865e" containerName="swift-recon-cron" Dec 03 11:48:41 crc kubenswrapper[5052]: E1203 11:48:41.010664 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="83a10afe-30d8-443d-a4f7-28784a2ea068" containerName="openstack-network-exporter" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.010670 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="83a10afe-30d8-443d-a4f7-28784a2ea068" containerName="openstack-network-exporter" Dec 03 11:48:41 crc kubenswrapper[5052]: E1203 11:48:41.010680 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="93ca3376-6916-4fc9-8b76-7cd9a930865e" containerName="account-server" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.010686 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="93ca3376-6916-4fc9-8b76-7cd9a930865e" containerName="account-server" Dec 03 11:48:41 crc kubenswrapper[5052]: E1203 11:48:41.010693 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="93ca3376-6916-4fc9-8b76-7cd9a930865e" containerName="object-replicator" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.010701 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="93ca3376-6916-4fc9-8b76-7cd9a930865e" containerName="object-replicator" Dec 03 11:48:41 crc kubenswrapper[5052]: E1203 11:48:41.010711 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="cadac9a6-1d82-403a-b7a0-692a424dd0e2" containerName="barbican-worker" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.010719 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="cadac9a6-1d82-403a-b7a0-692a424dd0e2" containerName="barbican-worker" Dec 03 11:48:41 crc kubenswrapper[5052]: E1203 11:48:41.010731 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="bb51da3b-72cc-4008-aac7-737f6a3031cb" containerName="nova-metadata-metadata" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.010738 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb51da3b-72cc-4008-aac7-737f6a3031cb" containerName="nova-metadata-metadata" Dec 03 11:48:41 crc kubenswrapper[5052]: E1203 11:48:41.010745 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="99f96963-d835-4445-84af-0976f1ba0701" containerName="barbican-api-log" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.010752 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="99f96963-d835-4445-84af-0976f1ba0701" containerName="barbican-api-log" Dec 03 11:48:41 crc kubenswrapper[5052]: E1203 11:48:41.010778 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="f7123213-06e1-4b26-9892-c16c3d49c0ab" containerName="nova-api-api" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.010786 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7123213-06e1-4b26-9892-c16c3d49c0ab" containerName="nova-api-api" Dec 03 11:48:41 crc kubenswrapper[5052]: E1203 11:48:41.010793 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="e5339997-33d5-4794-8fa6-042b63e9391e" containerName="mariadb-account-delete" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.010801 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="e5339997-33d5-4794-8fa6-042b63e9391e" containerName="mariadb-account-delete" Dec 03 11:48:41 crc kubenswrapper[5052]: E1203 11:48:41.010811 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="99f96963-d835-4445-84af-0976f1ba0701" containerName="barbican-api" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.010819 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="99f96963-d835-4445-84af-0976f1ba0701" containerName="barbican-api" Dec 03 11:48:41 crc kubenswrapper[5052]: E1203 11:48:41.010832 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="bbbc503e-d33d-42a9-856a-9ecc1d059f6b" containerName="init" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.010838 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="bbbc503e-d33d-42a9-856a-9ecc1d059f6b" containerName="init" Dec 03 11:48:41 crc kubenswrapper[5052]: E1203 11:48:41.010850 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="acea154c-f0bb-4066-bd4d-2dda815f301c" containerName="glance-log" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.010857 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="acea154c-f0bb-4066-bd4d-2dda815f301c" containerName="glance-log" Dec 03 11:48:41 crc kubenswrapper[5052]: E1203 11:48:41.010867 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="1b45577a-92b7-4c67-941b-cfc2fbfa3eb2" containerName="ceilometer-notification-agent" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.010875 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="1b45577a-92b7-4c67-941b-cfc2fbfa3eb2" containerName="ceilometer-notification-agent" Dec 03 11:48:41 crc kubenswrapper[5052]: E1203 11:48:41.010885 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="1b45577a-92b7-4c67-941b-cfc2fbfa3eb2" containerName="proxy-httpd" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.010892 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="1b45577a-92b7-4c67-941b-cfc2fbfa3eb2" containerName="proxy-httpd" Dec 03 11:48:41 crc kubenswrapper[5052]: E1203 11:48:41.010902 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="f7123213-06e1-4b26-9892-c16c3d49c0ab" containerName="nova-api-log" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.010910 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7123213-06e1-4b26-9892-c16c3d49c0ab" containerName="nova-api-log" Dec 03 11:48:41 crc kubenswrapper[5052]: E1203 11:48:41.010919 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="507c60e5-056c-4661-a7a7-75c4d9bce7de" containerName="nova-cell1-conductor-conductor" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.010927 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="507c60e5-056c-4661-a7a7-75c4d9bce7de" containerName="nova-cell1-conductor-conductor" Dec 03 11:48:41 crc kubenswrapper[5052]: E1203 11:48:41.010939 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="e741a60a-1a4f-45a1-b315-064fb7458e8f" containerName="galera" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.010946 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="e741a60a-1a4f-45a1-b315-064fb7458e8f" containerName="galera" Dec 03 11:48:41 crc kubenswrapper[5052]: E1203 11:48:41.010955 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="d8de865a-0f6c-413d-8a37-3ec4d496c4bb" containerName="barbican-keystone-listener" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.010961 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="d8de865a-0f6c-413d-8a37-3ec4d496c4bb" containerName="barbican-keystone-listener" Dec 03 11:48:41 crc kubenswrapper[5052]: E1203 11:48:41.010973 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="84957e18-5747-4786-9c4f-6af023122726" containerName="proxy-httpd" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.010978 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="84957e18-5747-4786-9c4f-6af023122726" containerName="proxy-httpd" Dec 03 11:48:41 crc kubenswrapper[5052]: E1203 11:48:41.010986 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="39b9198e-e3d7-4f85-b6a2-55960bdda82a" containerName="nova-scheduler-scheduler" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.010992 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="39b9198e-e3d7-4f85-b6a2-55960bdda82a" containerName="nova-scheduler-scheduler" Dec 03 11:48:41 crc kubenswrapper[5052]: E1203 11:48:41.010998 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="aa9a649b-9229-4cdd-8e27-8aedd48f4bbd" containerName="mariadb-account-delete" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.011004 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa9a649b-9229-4cdd-8e27-8aedd48f4bbd" containerName="mariadb-account-delete" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.011125 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="39b9198e-e3d7-4f85-b6a2-55960bdda82a" containerName="nova-scheduler-scheduler" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.011133 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="aa9a649b-9229-4cdd-8e27-8aedd48f4bbd" containerName="mariadb-account-delete" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.011140 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="84b8e975-30ee-42c6-8952-25aebf2c378b" containerName="nova-cell1-novncproxy-novncproxy" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.011146 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="0aefd6f6-cbb3-4c45-9f69-7124e43255e3" containerName="mariadb-account-delete" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.011158 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="d8de865a-0f6c-413d-8a37-3ec4d496c4bb" containerName="barbican-keystone-listener" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.011166 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="e5339997-33d5-4794-8fa6-042b63e9391e" containerName="mariadb-account-delete" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.011174 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="bbbc503e-d33d-42a9-856a-9ecc1d059f6b" containerName="dnsmasq-dns" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.011182 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="00b12739-1437-4af3-ab6d-ad830b67ad2e" containerName="cinder-api" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.011192 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="1b45577a-92b7-4c67-941b-cfc2fbfa3eb2" containerName="proxy-httpd" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.011200 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="93ca3376-6916-4fc9-8b76-7cd9a930865e" containerName="object-server" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.011207 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="d8de865a-0f6c-413d-8a37-3ec4d496c4bb" containerName="barbican-keystone-listener-log" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.011213 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="e947107f-104b-4699-a994-bc77285f0cf5" containerName="cinder-scheduler" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.011220 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="bb51da3b-72cc-4008-aac7-737f6a3031cb" containerName="nova-metadata-log" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.011227 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="0914ebfe-13c6-4309-bd10-a67850124e3f" containerName="glance-log" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.011234 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="cadac9a6-1d82-403a-b7a0-692a424dd0e2" containerName="barbican-worker" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.011241 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="2d3a3054-6c8e-4808-bd5d-01413970167c" containerName="mariadb-account-delete" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.011249 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="93ca3376-6916-4fc9-8b76-7cd9a930865e" containerName="object-auditor" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.011257 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="e947107f-104b-4699-a994-bc77285f0cf5" containerName="probe" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.011266 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="d6d9150c-e5b6-407a-b20a-4114a60312f8" containerName="nova-cell0-conductor-conductor" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.011274 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="1b45577a-92b7-4c67-941b-cfc2fbfa3eb2" containerName="sg-core" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.011284 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="e741a60a-1a4f-45a1-b315-064fb7458e8f" containerName="galera" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.011289 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="1b45577a-92b7-4c67-941b-cfc2fbfa3eb2" containerName="ceilometer-central-agent" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.011297 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="acea154c-f0bb-4066-bd4d-2dda815f301c" containerName="glance-log" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.011307 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="1b45577a-92b7-4c67-941b-cfc2fbfa3eb2" containerName="ceilometer-notification-agent" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.011315 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="e5339997-33d5-4794-8fa6-042b63e9391e" containerName="mariadb-account-delete" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.011321 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="93ca3376-6916-4fc9-8b76-7cd9a930865e" containerName="rsync" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.011328 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="d40fbd0a-3169-4fd1-bb23-5c5344d9a0f0" containerName="rabbitmq" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.011335 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="0183ebb5-b91b-443a-9535-2008c9207ab1" containerName="placement-api" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.011342 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="93ca3376-6916-4fc9-8b76-7cd9a930865e" containerName="account-server" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.011348 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="93ca3376-6916-4fc9-8b76-7cd9a930865e" containerName="object-replicator" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.011355 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="f7123213-06e1-4b26-9892-c16c3d49c0ab" containerName="nova-api-log" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.011360 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="0183ebb5-b91b-443a-9535-2008c9207ab1" containerName="placement-log" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.011369 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="aa9a649b-9229-4cdd-8e27-8aedd48f4bbd" containerName="mariadb-account-delete" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.011378 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="950f0626-40d2-41c9-9701-cba50843c43e" containerName="mariadb-account-delete" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.011384 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="8c2252c0-5e30-4e38-b1a1-af0242df6ac0" containerName="mariadb-account-delete" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.011392 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="93ca3376-6916-4fc9-8b76-7cd9a930865e" containerName="container-server" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.011398 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="93ca3376-6916-4fc9-8b76-7cd9a930865e" containerName="object-updater" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.011405 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="93ca3376-6916-4fc9-8b76-7cd9a930865e" containerName="object-expirer" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.011412 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="a4fa0509-8f85-462f-b7c3-4384346252bc" containerName="neutron-httpd" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.011421 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="6f9b208b-086f-4671-8c3f-b3a275740ad7" containerName="memcached" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.011427 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="84957e18-5747-4786-9c4f-6af023122726" containerName="proxy-server" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.011437 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="2d3a3054-6c8e-4808-bd5d-01413970167c" containerName="mariadb-account-delete" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.011448 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="496a1006-5380-40d1-998f-129a87233300" containerName="kube-state-metrics" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.011456 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="93ca3376-6916-4fc9-8b76-7cd9a930865e" containerName="container-replicator" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.011465 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="a4fa0509-8f85-462f-b7c3-4384346252bc" containerName="neutron-api" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.011474 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="9bdd2626-de50-4513-8313-3b25347ff641" containerName="ovsdbserver-sb" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.011483 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="83be8c2e-57f8-4b45-b3ca-096f72a1defd" containerName="galera" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.011490 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="93ca3376-6916-4fc9-8b76-7cd9a930865e" containerName="account-reaper" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.011497 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="2766f489-9217-47ae-a595-869a26389587" containerName="rabbitmq" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.011528 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="93ca3376-6916-4fc9-8b76-7cd9a930865e" containerName="swift-recon-cron" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.011539 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="65be66f1-c32d-40b5-81a4-a63258364d9d" containerName="keystone-api" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.011545 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="00b12739-1437-4af3-ab6d-ad830b67ad2e" containerName="cinder-api-log" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.011556 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="950f0626-40d2-41c9-9701-cba50843c43e" containerName="mariadb-account-delete" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.011565 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="93ca3376-6916-4fc9-8b76-7cd9a930865e" containerName="account-auditor" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.011574 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="68f829f3-e972-4168-8b75-27c086a91142" containerName="openstack-network-exporter" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.011583 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="84957e18-5747-4786-9c4f-6af023122726" containerName="proxy-httpd" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.011589 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="93ca3376-6916-4fc9-8b76-7cd9a930865e" containerName="account-replicator" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.011598 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="bb51da3b-72cc-4008-aac7-737f6a3031cb" containerName="nova-metadata-metadata" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.011604 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="83a10afe-30d8-443d-a4f7-28784a2ea068" containerName="ovn-northd" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.011612 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="93ca3376-6916-4fc9-8b76-7cd9a930865e" containerName="container-auditor" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.011619 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="acea154c-f0bb-4066-bd4d-2dda815f301c" containerName="glance-httpd" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.011624 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="8c2252c0-5e30-4e38-b1a1-af0242df6ac0" containerName="mariadb-account-delete" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.011631 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="0b94b517-5f9b-485d-a109-0a11e15a3ad3" containerName="mariadb-account-delete" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.011640 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="507c60e5-056c-4661-a7a7-75c4d9bce7de" containerName="nova-cell1-conductor-conductor" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.011648 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="68f829f3-e972-4168-8b75-27c086a91142" containerName="ovsdbserver-nb" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.011653 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="93ca3376-6916-4fc9-8b76-7cd9a930865e" containerName="container-updater" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.011659 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="0914ebfe-13c6-4309-bd10-a67850124e3f" containerName="glance-httpd" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.011665 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="9bdd2626-de50-4513-8313-3b25347ff641" containerName="openstack-network-exporter" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.011672 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="cadac9a6-1d82-403a-b7a0-692a424dd0e2" containerName="barbican-worker-log" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.011678 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="99f96963-d835-4445-84af-0976f1ba0701" containerName="barbican-api" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.011684 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="f7123213-06e1-4b26-9892-c16c3d49c0ab" containerName="nova-api-api" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.011691 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="99f96963-d835-4445-84af-0976f1ba0701" containerName="barbican-api-log" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.011701 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="83a10afe-30d8-443d-a4f7-28784a2ea068" containerName="openstack-network-exporter" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.012970 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="crc-storage/crc-storage-crc-8dcm2" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.023935 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["crc-storage/crc-storage-crc-8dcm2"] Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.076156 5052 reflector.go:368] Caches populated for *v1.Secret from object-"crc-storage"/"crc-storage-dockercfg-mf76g" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.076343 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"crc-storage"/"openshift-service-ca.crt" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.077272 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"crc-storage"/"kube-root-ca.crt" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.081566 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"crc-storage"/"crc-storage" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.127199 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/0d3e5ff6-f008-44a0-a924-8cd86c58e200-node-mnt\") pod \"crc-storage-crc-8dcm2\" (UID: \"0d3e5ff6-f008-44a0-a924-8cd86c58e200\") " pod="crc-storage/crc-storage-crc-8dcm2" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.127336 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/0d3e5ff6-f008-44a0-a924-8cd86c58e200-crc-storage\") pod \"crc-storage-crc-8dcm2\" (UID: \"0d3e5ff6-f008-44a0-a924-8cd86c58e200\") " pod="crc-storage/crc-storage-crc-8dcm2" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.127370 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ttzts\" (UniqueName: \"kubernetes.io/projected/0d3e5ff6-f008-44a0-a924-8cd86c58e200-kube-api-access-ttzts\") pod \"crc-storage-crc-8dcm2\" (UID: \"0d3e5ff6-f008-44a0-a924-8cd86c58e200\") " pod="crc-storage/crc-storage-crc-8dcm2" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.229177 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/0d3e5ff6-f008-44a0-a924-8cd86c58e200-crc-storage\") pod \"crc-storage-crc-8dcm2\" (UID: \"0d3e5ff6-f008-44a0-a924-8cd86c58e200\") " pod="crc-storage/crc-storage-crc-8dcm2" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.229259 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-ttzts\" (UniqueName: \"kubernetes.io/projected/0d3e5ff6-f008-44a0-a924-8cd86c58e200-kube-api-access-ttzts\") pod \"crc-storage-crc-8dcm2\" (UID: \"0d3e5ff6-f008-44a0-a924-8cd86c58e200\") " pod="crc-storage/crc-storage-crc-8dcm2" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.229329 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/0d3e5ff6-f008-44a0-a924-8cd86c58e200-node-mnt\") pod \"crc-storage-crc-8dcm2\" (UID: \"0d3e5ff6-f008-44a0-a924-8cd86c58e200\") " pod="crc-storage/crc-storage-crc-8dcm2" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.229829 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/0d3e5ff6-f008-44a0-a924-8cd86c58e200-node-mnt\") pod \"crc-storage-crc-8dcm2\" (UID: \"0d3e5ff6-f008-44a0-a924-8cd86c58e200\") " pod="crc-storage/crc-storage-crc-8dcm2" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.230150 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/0d3e5ff6-f008-44a0-a924-8cd86c58e200-crc-storage\") pod \"crc-storage-crc-8dcm2\" (UID: \"0d3e5ff6-f008-44a0-a924-8cd86c58e200\") " pod="crc-storage/crc-storage-crc-8dcm2" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.252082 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-ttzts\" (UniqueName: \"kubernetes.io/projected/0d3e5ff6-f008-44a0-a924-8cd86c58e200-kube-api-access-ttzts\") pod \"crc-storage-crc-8dcm2\" (UID: \"0d3e5ff6-f008-44a0-a924-8cd86c58e200\") " pod="crc-storage/crc-storage-crc-8dcm2" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.403107 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="crc-storage/crc-storage-crc-8dcm2" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.627785 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["crc-storage/crc-storage-crc-8dcm2"] Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.647404 5052 provider.go:102] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.682749 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f9310868-40a7-4ede-bfb4-4f97a35345d0" path="/var/lib/kubelet/pods/f9310868-40a7-4ede-bfb4-4f97a35345d0/volumes" Dec 03 11:48:41 crc kubenswrapper[5052]: I1203 11:48:41.928230 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="crc-storage/crc-storage-crc-8dcm2" event={"ID":"0d3e5ff6-f008-44a0-a924-8cd86c58e200","Type":"ContainerStarted","Data":"67095dc52f0a9d4d066952a3030a01d525d40242bbe536757c6cecafb18daca2"} Dec 03 11:48:44 crc kubenswrapper[5052]: I1203 11:48:44.600709 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openshift-marketplace/redhat-marketplace-sbfvb"] Dec 03 11:48:44 crc kubenswrapper[5052]: E1203 11:48:44.604004 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="8c2252c0-5e30-4e38-b1a1-af0242df6ac0" containerName="mariadb-account-delete" Dec 03 11:48:44 crc kubenswrapper[5052]: I1203 11:48:44.604027 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="8c2252c0-5e30-4e38-b1a1-af0242df6ac0" containerName="mariadb-account-delete" Dec 03 11:48:44 crc kubenswrapper[5052]: I1203 11:48:44.605554 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/redhat-marketplace-sbfvb" Dec 03 11:48:44 crc kubenswrapper[5052]: I1203 11:48:44.610590 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/redhat-marketplace-sbfvb"] Dec 03 11:48:44 crc kubenswrapper[5052]: I1203 11:48:44.785243 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hqpht\" (UniqueName: \"kubernetes.io/projected/dd4c2563-ffde-4811-8705-79528793d93c-kube-api-access-hqpht\") pod \"redhat-marketplace-sbfvb\" (UID: \"dd4c2563-ffde-4811-8705-79528793d93c\") " pod="openshift-marketplace/redhat-marketplace-sbfvb" Dec 03 11:48:44 crc kubenswrapper[5052]: I1203 11:48:44.785308 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/dd4c2563-ffde-4811-8705-79528793d93c-catalog-content\") pod \"redhat-marketplace-sbfvb\" (UID: \"dd4c2563-ffde-4811-8705-79528793d93c\") " pod="openshift-marketplace/redhat-marketplace-sbfvb" Dec 03 11:48:44 crc kubenswrapper[5052]: I1203 11:48:44.785361 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/dd4c2563-ffde-4811-8705-79528793d93c-utilities\") pod \"redhat-marketplace-sbfvb\" (UID: \"dd4c2563-ffde-4811-8705-79528793d93c\") " pod="openshift-marketplace/redhat-marketplace-sbfvb" Dec 03 11:48:44 crc kubenswrapper[5052]: I1203 11:48:44.886945 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-hqpht\" (UniqueName: \"kubernetes.io/projected/dd4c2563-ffde-4811-8705-79528793d93c-kube-api-access-hqpht\") pod \"redhat-marketplace-sbfvb\" (UID: \"dd4c2563-ffde-4811-8705-79528793d93c\") " pod="openshift-marketplace/redhat-marketplace-sbfvb" Dec 03 11:48:44 crc kubenswrapper[5052]: I1203 11:48:44.887002 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/dd4c2563-ffde-4811-8705-79528793d93c-catalog-content\") pod \"redhat-marketplace-sbfvb\" (UID: \"dd4c2563-ffde-4811-8705-79528793d93c\") " pod="openshift-marketplace/redhat-marketplace-sbfvb" Dec 03 11:48:44 crc kubenswrapper[5052]: I1203 11:48:44.887068 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/dd4c2563-ffde-4811-8705-79528793d93c-utilities\") pod \"redhat-marketplace-sbfvb\" (UID: \"dd4c2563-ffde-4811-8705-79528793d93c\") " pod="openshift-marketplace/redhat-marketplace-sbfvb" Dec 03 11:48:44 crc kubenswrapper[5052]: I1203 11:48:44.887640 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/dd4c2563-ffde-4811-8705-79528793d93c-catalog-content\") pod \"redhat-marketplace-sbfvb\" (UID: \"dd4c2563-ffde-4811-8705-79528793d93c\") " pod="openshift-marketplace/redhat-marketplace-sbfvb" Dec 03 11:48:44 crc kubenswrapper[5052]: I1203 11:48:44.887706 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/dd4c2563-ffde-4811-8705-79528793d93c-utilities\") pod \"redhat-marketplace-sbfvb\" (UID: \"dd4c2563-ffde-4811-8705-79528793d93c\") " pod="openshift-marketplace/redhat-marketplace-sbfvb" Dec 03 11:48:44 crc kubenswrapper[5052]: I1203 11:48:44.909528 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-hqpht\" (UniqueName: \"kubernetes.io/projected/dd4c2563-ffde-4811-8705-79528793d93c-kube-api-access-hqpht\") pod \"redhat-marketplace-sbfvb\" (UID: \"dd4c2563-ffde-4811-8705-79528793d93c\") " pod="openshift-marketplace/redhat-marketplace-sbfvb" Dec 03 11:48:44 crc kubenswrapper[5052]: I1203 11:48:44.965594 5052 generic.go:334] "Generic (PLEG): container finished" podID="0d3e5ff6-f008-44a0-a924-8cd86c58e200" containerID="166938713bd25e93070764dc802907a92def42903c27f9f3ee3ea93bf02238e5" exitCode=0 Dec 03 11:48:44 crc kubenswrapper[5052]: I1203 11:48:44.965636 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="crc-storage/crc-storage-crc-8dcm2" event={"ID":"0d3e5ff6-f008-44a0-a924-8cd86c58e200","Type":"ContainerDied","Data":"166938713bd25e93070764dc802907a92def42903c27f9f3ee3ea93bf02238e5"} Dec 03 11:48:44 crc kubenswrapper[5052]: I1203 11:48:44.972929 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/redhat-marketplace-sbfvb" Dec 03 11:48:45 crc kubenswrapper[5052]: I1203 11:48:45.456181 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/redhat-marketplace-sbfvb"] Dec 03 11:48:45 crc kubenswrapper[5052]: W1203 11:48:45.464799 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd4c2563_ffde_4811_8705_79528793d93c.slice/crio-ae8801d41f19f4897f1084ba4ea5743064dac2ee9f0f66320f36c462b67a24f9 WatchSource:0}: Error finding container ae8801d41f19f4897f1084ba4ea5743064dac2ee9f0f66320f36c462b67a24f9: Status 404 returned error can't find the container with id ae8801d41f19f4897f1084ba4ea5743064dac2ee9f0f66320f36c462b67a24f9 Dec 03 11:48:45 crc kubenswrapper[5052]: I1203 11:48:45.974305 5052 generic.go:334] "Generic (PLEG): container finished" podID="dd4c2563-ffde-4811-8705-79528793d93c" containerID="fbb0c95d9cb1c2489a130115cfafc7255ea4fb866b07fe9bc58477851da897c9" exitCode=0 Dec 03 11:48:45 crc kubenswrapper[5052]: I1203 11:48:45.974450 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-marketplace-sbfvb" event={"ID":"dd4c2563-ffde-4811-8705-79528793d93c","Type":"ContainerDied","Data":"fbb0c95d9cb1c2489a130115cfafc7255ea4fb866b07fe9bc58477851da897c9"} Dec 03 11:48:45 crc kubenswrapper[5052]: I1203 11:48:45.974667 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-marketplace-sbfvb" event={"ID":"dd4c2563-ffde-4811-8705-79528793d93c","Type":"ContainerStarted","Data":"ae8801d41f19f4897f1084ba4ea5743064dac2ee9f0f66320f36c462b67a24f9"} Dec 03 11:48:46 crc kubenswrapper[5052]: I1203 11:48:46.265402 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="crc-storage/crc-storage-crc-8dcm2" Dec 03 11:48:46 crc kubenswrapper[5052]: I1203 11:48:46.407463 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ttzts\" (UniqueName: \"kubernetes.io/projected/0d3e5ff6-f008-44a0-a924-8cd86c58e200-kube-api-access-ttzts\") pod \"0d3e5ff6-f008-44a0-a924-8cd86c58e200\" (UID: \"0d3e5ff6-f008-44a0-a924-8cd86c58e200\") " Dec 03 11:48:46 crc kubenswrapper[5052]: I1203 11:48:46.407529 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/0d3e5ff6-f008-44a0-a924-8cd86c58e200-crc-storage\") pod \"0d3e5ff6-f008-44a0-a924-8cd86c58e200\" (UID: \"0d3e5ff6-f008-44a0-a924-8cd86c58e200\") " Dec 03 11:48:46 crc kubenswrapper[5052]: I1203 11:48:46.407613 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/0d3e5ff6-f008-44a0-a924-8cd86c58e200-node-mnt\") pod \"0d3e5ff6-f008-44a0-a924-8cd86c58e200\" (UID: \"0d3e5ff6-f008-44a0-a924-8cd86c58e200\") " Dec 03 11:48:46 crc kubenswrapper[5052]: I1203 11:48:46.407824 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/0d3e5ff6-f008-44a0-a924-8cd86c58e200-node-mnt" (OuterVolumeSpecName: "node-mnt") pod "0d3e5ff6-f008-44a0-a924-8cd86c58e200" (UID: "0d3e5ff6-f008-44a0-a924-8cd86c58e200"). InnerVolumeSpecName "node-mnt". PluginName "kubernetes.io/host-path", VolumeGidValue "" Dec 03 11:48:46 crc kubenswrapper[5052]: I1203 11:48:46.408013 5052 reconciler_common.go:293] "Volume detached for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/0d3e5ff6-f008-44a0-a924-8cd86c58e200-node-mnt\") on node \"crc\" DevicePath \"\"" Dec 03 11:48:46 crc kubenswrapper[5052]: I1203 11:48:46.415755 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0d3e5ff6-f008-44a0-a924-8cd86c58e200-kube-api-access-ttzts" (OuterVolumeSpecName: "kube-api-access-ttzts") pod "0d3e5ff6-f008-44a0-a924-8cd86c58e200" (UID: "0d3e5ff6-f008-44a0-a924-8cd86c58e200"). InnerVolumeSpecName "kube-api-access-ttzts". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:48:46 crc kubenswrapper[5052]: I1203 11:48:46.436389 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0d3e5ff6-f008-44a0-a924-8cd86c58e200-crc-storage" (OuterVolumeSpecName: "crc-storage") pod "0d3e5ff6-f008-44a0-a924-8cd86c58e200" (UID: "0d3e5ff6-f008-44a0-a924-8cd86c58e200"). InnerVolumeSpecName "crc-storage". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:48:46 crc kubenswrapper[5052]: I1203 11:48:46.510105 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-ttzts\" (UniqueName: \"kubernetes.io/projected/0d3e5ff6-f008-44a0-a924-8cd86c58e200-kube-api-access-ttzts\") on node \"crc\" DevicePath \"\"" Dec 03 11:48:46 crc kubenswrapper[5052]: I1203 11:48:46.510149 5052 reconciler_common.go:293] "Volume detached for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/0d3e5ff6-f008-44a0-a924-8cd86c58e200-crc-storage\") on node \"crc\" DevicePath \"\"" Dec 03 11:48:46 crc kubenswrapper[5052]: I1203 11:48:46.986448 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="crc-storage/crc-storage-crc-8dcm2" event={"ID":"0d3e5ff6-f008-44a0-a924-8cd86c58e200","Type":"ContainerDied","Data":"67095dc52f0a9d4d066952a3030a01d525d40242bbe536757c6cecafb18daca2"} Dec 03 11:48:46 crc kubenswrapper[5052]: I1203 11:48:46.986778 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="67095dc52f0a9d4d066952a3030a01d525d40242bbe536757c6cecafb18daca2" Dec 03 11:48:46 crc kubenswrapper[5052]: I1203 11:48:46.986489 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="crc-storage/crc-storage-crc-8dcm2" Dec 03 11:48:47 crc kubenswrapper[5052]: I1203 11:48:47.998186 5052 generic.go:334] "Generic (PLEG): container finished" podID="dd4c2563-ffde-4811-8705-79528793d93c" containerID="c4c24c798e9dc184297d47c688e65b5d9fd082fe66a0dd87714153cd96f9ebe2" exitCode=0 Dec 03 11:48:47 crc kubenswrapper[5052]: I1203 11:48:47.998231 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-marketplace-sbfvb" event={"ID":"dd4c2563-ffde-4811-8705-79528793d93c","Type":"ContainerDied","Data":"c4c24c798e9dc184297d47c688e65b5d9fd082fe66a0dd87714153cd96f9ebe2"} Dec 03 11:48:48 crc kubenswrapper[5052]: I1203 11:48:48.673385 5052 scope.go:117] "RemoveContainer" containerID="25fe88f8d102f09983f8884e65f0c6c1575ade67921f10a630e6dcc3ea8b506e" Dec 03 11:48:48 crc kubenswrapper[5052]: E1203 11:48:48.673883 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 11:48:49 crc kubenswrapper[5052]: I1203 11:48:49.013724 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-marketplace-sbfvb" event={"ID":"dd4c2563-ffde-4811-8705-79528793d93c","Type":"ContainerStarted","Data":"676f222e2427fdf810093272a42055ae13d78d9dbaf7d569b1f45fcdd24e2d6e"} Dec 03 11:48:49 crc kubenswrapper[5052]: I1203 11:48:49.046384 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-marketplace/redhat-marketplace-sbfvb" podStartSLOduration=2.610472947 podStartE2EDuration="5.046364927s" podCreationTimestamp="2025-12-03 11:48:44 +0000 UTC" firstStartedPulling="2025-12-03 11:48:45.975932327 +0000 UTC m=+1716.564174091" lastFinishedPulling="2025-12-03 11:48:48.411824267 +0000 UTC m=+1719.000066071" observedRunningTime="2025-12-03 11:48:49.037371209 +0000 UTC m=+1719.625612993" watchObservedRunningTime="2025-12-03 11:48:49.046364927 +0000 UTC m=+1719.634606701" Dec 03 11:48:49 crc kubenswrapper[5052]: I1203 11:48:49.685559 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["crc-storage/crc-storage-crc-8dcm2"] Dec 03 11:48:49 crc kubenswrapper[5052]: I1203 11:48:49.692350 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["crc-storage/crc-storage-crc-8dcm2"] Dec 03 11:48:49 crc kubenswrapper[5052]: I1203 11:48:49.826602 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["crc-storage/crc-storage-crc-6vsfp"] Dec 03 11:48:49 crc kubenswrapper[5052]: E1203 11:48:49.827120 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="0d3e5ff6-f008-44a0-a924-8cd86c58e200" containerName="storage" Dec 03 11:48:49 crc kubenswrapper[5052]: I1203 11:48:49.827189 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="0d3e5ff6-f008-44a0-a924-8cd86c58e200" containerName="storage" Dec 03 11:48:49 crc kubenswrapper[5052]: I1203 11:48:49.827415 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="0d3e5ff6-f008-44a0-a924-8cd86c58e200" containerName="storage" Dec 03 11:48:49 crc kubenswrapper[5052]: I1203 11:48:49.827966 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="crc-storage/crc-storage-crc-6vsfp" Dec 03 11:48:49 crc kubenswrapper[5052]: I1203 11:48:49.833310 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["crc-storage/crc-storage-crc-6vsfp"] Dec 03 11:48:49 crc kubenswrapper[5052]: I1203 11:48:49.834833 5052 reflector.go:368] Caches populated for *v1.Secret from object-"crc-storage"/"crc-storage-dockercfg-mf76g" Dec 03 11:48:49 crc kubenswrapper[5052]: I1203 11:48:49.834861 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"crc-storage"/"kube-root-ca.crt" Dec 03 11:48:49 crc kubenswrapper[5052]: I1203 11:48:49.836028 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"crc-storage"/"crc-storage" Dec 03 11:48:49 crc kubenswrapper[5052]: I1203 11:48:49.841012 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"crc-storage"/"openshift-service-ca.crt" Dec 03 11:48:49 crc kubenswrapper[5052]: I1203 11:48:49.958961 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bcnq9\" (UniqueName: \"kubernetes.io/projected/f1ba93eb-7c01-4add-b32d-8523c5e7808e-kube-api-access-bcnq9\") pod \"crc-storage-crc-6vsfp\" (UID: \"f1ba93eb-7c01-4add-b32d-8523c5e7808e\") " pod="crc-storage/crc-storage-crc-6vsfp" Dec 03 11:48:49 crc kubenswrapper[5052]: I1203 11:48:49.959019 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/f1ba93eb-7c01-4add-b32d-8523c5e7808e-crc-storage\") pod \"crc-storage-crc-6vsfp\" (UID: \"f1ba93eb-7c01-4add-b32d-8523c5e7808e\") " pod="crc-storage/crc-storage-crc-6vsfp" Dec 03 11:48:49 crc kubenswrapper[5052]: I1203 11:48:49.959647 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/f1ba93eb-7c01-4add-b32d-8523c5e7808e-node-mnt\") pod \"crc-storage-crc-6vsfp\" (UID: \"f1ba93eb-7c01-4add-b32d-8523c5e7808e\") " pod="crc-storage/crc-storage-crc-6vsfp" Dec 03 11:48:50 crc kubenswrapper[5052]: I1203 11:48:50.061735 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/f1ba93eb-7c01-4add-b32d-8523c5e7808e-node-mnt\") pod \"crc-storage-crc-6vsfp\" (UID: \"f1ba93eb-7c01-4add-b32d-8523c5e7808e\") " pod="crc-storage/crc-storage-crc-6vsfp" Dec 03 11:48:50 crc kubenswrapper[5052]: I1203 11:48:50.061824 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-bcnq9\" (UniqueName: \"kubernetes.io/projected/f1ba93eb-7c01-4add-b32d-8523c5e7808e-kube-api-access-bcnq9\") pod \"crc-storage-crc-6vsfp\" (UID: \"f1ba93eb-7c01-4add-b32d-8523c5e7808e\") " pod="crc-storage/crc-storage-crc-6vsfp" Dec 03 11:48:50 crc kubenswrapper[5052]: I1203 11:48:50.061862 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/f1ba93eb-7c01-4add-b32d-8523c5e7808e-crc-storage\") pod \"crc-storage-crc-6vsfp\" (UID: \"f1ba93eb-7c01-4add-b32d-8523c5e7808e\") " pod="crc-storage/crc-storage-crc-6vsfp" Dec 03 11:48:50 crc kubenswrapper[5052]: I1203 11:48:50.062110 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/f1ba93eb-7c01-4add-b32d-8523c5e7808e-node-mnt\") pod \"crc-storage-crc-6vsfp\" (UID: \"f1ba93eb-7c01-4add-b32d-8523c5e7808e\") " pod="crc-storage/crc-storage-crc-6vsfp" Dec 03 11:48:50 crc kubenswrapper[5052]: I1203 11:48:50.062809 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/f1ba93eb-7c01-4add-b32d-8523c5e7808e-crc-storage\") pod \"crc-storage-crc-6vsfp\" (UID: \"f1ba93eb-7c01-4add-b32d-8523c5e7808e\") " pod="crc-storage/crc-storage-crc-6vsfp" Dec 03 11:48:50 crc kubenswrapper[5052]: I1203 11:48:50.082940 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-bcnq9\" (UniqueName: \"kubernetes.io/projected/f1ba93eb-7c01-4add-b32d-8523c5e7808e-kube-api-access-bcnq9\") pod \"crc-storage-crc-6vsfp\" (UID: \"f1ba93eb-7c01-4add-b32d-8523c5e7808e\") " pod="crc-storage/crc-storage-crc-6vsfp" Dec 03 11:48:50 crc kubenswrapper[5052]: I1203 11:48:50.154386 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="crc-storage/crc-storage-crc-6vsfp" Dec 03 11:48:50 crc kubenswrapper[5052]: I1203 11:48:50.645128 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["crc-storage/crc-storage-crc-6vsfp"] Dec 03 11:48:50 crc kubenswrapper[5052]: W1203 11:48:50.647128 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf1ba93eb_7c01_4add_b32d_8523c5e7808e.slice/crio-e6e05179c1e647987fd0e6eb2f5418403a3735705539300a7090205961376fa5 WatchSource:0}: Error finding container e6e05179c1e647987fd0e6eb2f5418403a3735705539300a7090205961376fa5: Status 404 returned error can't find the container with id e6e05179c1e647987fd0e6eb2f5418403a3735705539300a7090205961376fa5 Dec 03 11:48:51 crc kubenswrapper[5052]: I1203 11:48:51.044994 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="crc-storage/crc-storage-crc-6vsfp" event={"ID":"f1ba93eb-7c01-4add-b32d-8523c5e7808e","Type":"ContainerStarted","Data":"e6e05179c1e647987fd0e6eb2f5418403a3735705539300a7090205961376fa5"} Dec 03 11:48:51 crc kubenswrapper[5052]: I1203 11:48:51.684538 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0d3e5ff6-f008-44a0-a924-8cd86c58e200" path="/var/lib/kubelet/pods/0d3e5ff6-f008-44a0-a924-8cd86c58e200/volumes" Dec 03 11:48:52 crc kubenswrapper[5052]: I1203 11:48:52.054939 5052 generic.go:334] "Generic (PLEG): container finished" podID="f1ba93eb-7c01-4add-b32d-8523c5e7808e" containerID="04d3797bc8a2e14f5d24f9cb2c6ee2c3f3c3bca4f308b9679823623d214051bf" exitCode=0 Dec 03 11:48:52 crc kubenswrapper[5052]: I1203 11:48:52.054994 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="crc-storage/crc-storage-crc-6vsfp" event={"ID":"f1ba93eb-7c01-4add-b32d-8523c5e7808e","Type":"ContainerDied","Data":"04d3797bc8a2e14f5d24f9cb2c6ee2c3f3c3bca4f308b9679823623d214051bf"} Dec 03 11:48:53 crc kubenswrapper[5052]: I1203 11:48:53.346468 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="crc-storage/crc-storage-crc-6vsfp" Dec 03 11:48:53 crc kubenswrapper[5052]: I1203 11:48:53.531327 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/f1ba93eb-7c01-4add-b32d-8523c5e7808e-node-mnt\") pod \"f1ba93eb-7c01-4add-b32d-8523c5e7808e\" (UID: \"f1ba93eb-7c01-4add-b32d-8523c5e7808e\") " Dec 03 11:48:53 crc kubenswrapper[5052]: I1203 11:48:53.531366 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/f1ba93eb-7c01-4add-b32d-8523c5e7808e-crc-storage\") pod \"f1ba93eb-7c01-4add-b32d-8523c5e7808e\" (UID: \"f1ba93eb-7c01-4add-b32d-8523c5e7808e\") " Dec 03 11:48:53 crc kubenswrapper[5052]: I1203 11:48:53.531455 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bcnq9\" (UniqueName: \"kubernetes.io/projected/f1ba93eb-7c01-4add-b32d-8523c5e7808e-kube-api-access-bcnq9\") pod \"f1ba93eb-7c01-4add-b32d-8523c5e7808e\" (UID: \"f1ba93eb-7c01-4add-b32d-8523c5e7808e\") " Dec 03 11:48:53 crc kubenswrapper[5052]: I1203 11:48:53.531501 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/f1ba93eb-7c01-4add-b32d-8523c5e7808e-node-mnt" (OuterVolumeSpecName: "node-mnt") pod "f1ba93eb-7c01-4add-b32d-8523c5e7808e" (UID: "f1ba93eb-7c01-4add-b32d-8523c5e7808e"). InnerVolumeSpecName "node-mnt". PluginName "kubernetes.io/host-path", VolumeGidValue "" Dec 03 11:48:53 crc kubenswrapper[5052]: I1203 11:48:53.531993 5052 reconciler_common.go:293] "Volume detached for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/f1ba93eb-7c01-4add-b32d-8523c5e7808e-node-mnt\") on node \"crc\" DevicePath \"\"" Dec 03 11:48:53 crc kubenswrapper[5052]: I1203 11:48:53.536816 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f1ba93eb-7c01-4add-b32d-8523c5e7808e-kube-api-access-bcnq9" (OuterVolumeSpecName: "kube-api-access-bcnq9") pod "f1ba93eb-7c01-4add-b32d-8523c5e7808e" (UID: "f1ba93eb-7c01-4add-b32d-8523c5e7808e"). InnerVolumeSpecName "kube-api-access-bcnq9". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:48:53 crc kubenswrapper[5052]: I1203 11:48:53.570669 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f1ba93eb-7c01-4add-b32d-8523c5e7808e-crc-storage" (OuterVolumeSpecName: "crc-storage") pod "f1ba93eb-7c01-4add-b32d-8523c5e7808e" (UID: "f1ba93eb-7c01-4add-b32d-8523c5e7808e"). InnerVolumeSpecName "crc-storage". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:48:53 crc kubenswrapper[5052]: I1203 11:48:53.633356 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-bcnq9\" (UniqueName: \"kubernetes.io/projected/f1ba93eb-7c01-4add-b32d-8523c5e7808e-kube-api-access-bcnq9\") on node \"crc\" DevicePath \"\"" Dec 03 11:48:53 crc kubenswrapper[5052]: I1203 11:48:53.633393 5052 reconciler_common.go:293] "Volume detached for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/f1ba93eb-7c01-4add-b32d-8523c5e7808e-crc-storage\") on node \"crc\" DevicePath \"\"" Dec 03 11:48:54 crc kubenswrapper[5052]: I1203 11:48:54.081291 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="crc-storage/crc-storage-crc-6vsfp" event={"ID":"f1ba93eb-7c01-4add-b32d-8523c5e7808e","Type":"ContainerDied","Data":"e6e05179c1e647987fd0e6eb2f5418403a3735705539300a7090205961376fa5"} Dec 03 11:48:54 crc kubenswrapper[5052]: I1203 11:48:54.081334 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="crc-storage/crc-storage-crc-6vsfp" Dec 03 11:48:54 crc kubenswrapper[5052]: I1203 11:48:54.081337 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e6e05179c1e647987fd0e6eb2f5418403a3735705539300a7090205961376fa5" Dec 03 11:48:54 crc kubenswrapper[5052]: I1203 11:48:54.973740 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openshift-marketplace/redhat-marketplace-sbfvb" Dec 03 11:48:54 crc kubenswrapper[5052]: I1203 11:48:54.975109 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-marketplace/redhat-marketplace-sbfvb" Dec 03 11:48:55 crc kubenswrapper[5052]: I1203 11:48:55.023804 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-marketplace/redhat-marketplace-sbfvb" Dec 03 11:48:55 crc kubenswrapper[5052]: I1203 11:48:55.141795 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-marketplace/redhat-marketplace-sbfvb" Dec 03 11:48:55 crc kubenswrapper[5052]: I1203 11:48:55.259265 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openshift-marketplace/redhat-marketplace-sbfvb"] Dec 03 11:48:57 crc kubenswrapper[5052]: I1203 11:48:57.105359 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openshift-marketplace/redhat-marketplace-sbfvb" podUID="dd4c2563-ffde-4811-8705-79528793d93c" containerName="registry-server" containerID="cri-o://676f222e2427fdf810093272a42055ae13d78d9dbaf7d569b1f45fcdd24e2d6e" gracePeriod=2 Dec 03 11:48:58 crc kubenswrapper[5052]: I1203 11:48:58.093292 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/redhat-marketplace-sbfvb" Dec 03 11:48:58 crc kubenswrapper[5052]: I1203 11:48:58.098386 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/dd4c2563-ffde-4811-8705-79528793d93c-utilities\") pod \"dd4c2563-ffde-4811-8705-79528793d93c\" (UID: \"dd4c2563-ffde-4811-8705-79528793d93c\") " Dec 03 11:48:58 crc kubenswrapper[5052]: I1203 11:48:58.098465 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/dd4c2563-ffde-4811-8705-79528793d93c-catalog-content\") pod \"dd4c2563-ffde-4811-8705-79528793d93c\" (UID: \"dd4c2563-ffde-4811-8705-79528793d93c\") " Dec 03 11:48:58 crc kubenswrapper[5052]: I1203 11:48:58.098545 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hqpht\" (UniqueName: \"kubernetes.io/projected/dd4c2563-ffde-4811-8705-79528793d93c-kube-api-access-hqpht\") pod \"dd4c2563-ffde-4811-8705-79528793d93c\" (UID: \"dd4c2563-ffde-4811-8705-79528793d93c\") " Dec 03 11:48:58 crc kubenswrapper[5052]: I1203 11:48:58.099701 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd4c2563-ffde-4811-8705-79528793d93c-utilities" (OuterVolumeSpecName: "utilities") pod "dd4c2563-ffde-4811-8705-79528793d93c" (UID: "dd4c2563-ffde-4811-8705-79528793d93c"). InnerVolumeSpecName "utilities". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:48:58 crc kubenswrapper[5052]: I1203 11:48:58.105275 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dd4c2563-ffde-4811-8705-79528793d93c-kube-api-access-hqpht" (OuterVolumeSpecName: "kube-api-access-hqpht") pod "dd4c2563-ffde-4811-8705-79528793d93c" (UID: "dd4c2563-ffde-4811-8705-79528793d93c"). InnerVolumeSpecName "kube-api-access-hqpht". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:48:58 crc kubenswrapper[5052]: I1203 11:48:58.125571 5052 generic.go:334] "Generic (PLEG): container finished" podID="dd4c2563-ffde-4811-8705-79528793d93c" containerID="676f222e2427fdf810093272a42055ae13d78d9dbaf7d569b1f45fcdd24e2d6e" exitCode=0 Dec 03 11:48:58 crc kubenswrapper[5052]: I1203 11:48:58.125629 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-marketplace-sbfvb" event={"ID":"dd4c2563-ffde-4811-8705-79528793d93c","Type":"ContainerDied","Data":"676f222e2427fdf810093272a42055ae13d78d9dbaf7d569b1f45fcdd24e2d6e"} Dec 03 11:48:58 crc kubenswrapper[5052]: I1203 11:48:58.125633 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/redhat-marketplace-sbfvb" Dec 03 11:48:58 crc kubenswrapper[5052]: I1203 11:48:58.125701 5052 scope.go:117] "RemoveContainer" containerID="676f222e2427fdf810093272a42055ae13d78d9dbaf7d569b1f45fcdd24e2d6e" Dec 03 11:48:58 crc kubenswrapper[5052]: I1203 11:48:58.125681 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-marketplace-sbfvb" event={"ID":"dd4c2563-ffde-4811-8705-79528793d93c","Type":"ContainerDied","Data":"ae8801d41f19f4897f1084ba4ea5743064dac2ee9f0f66320f36c462b67a24f9"} Dec 03 11:48:58 crc kubenswrapper[5052]: I1203 11:48:58.133368 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd4c2563-ffde-4811-8705-79528793d93c-catalog-content" (OuterVolumeSpecName: "catalog-content") pod "dd4c2563-ffde-4811-8705-79528793d93c" (UID: "dd4c2563-ffde-4811-8705-79528793d93c"). InnerVolumeSpecName "catalog-content". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:48:58 crc kubenswrapper[5052]: I1203 11:48:58.153778 5052 scope.go:117] "RemoveContainer" containerID="c4c24c798e9dc184297d47c688e65b5d9fd082fe66a0dd87714153cd96f9ebe2" Dec 03 11:48:58 crc kubenswrapper[5052]: I1203 11:48:58.175668 5052 scope.go:117] "RemoveContainer" containerID="fbb0c95d9cb1c2489a130115cfafc7255ea4fb866b07fe9bc58477851da897c9" Dec 03 11:48:58 crc kubenswrapper[5052]: I1203 11:48:58.201414 5052 reconciler_common.go:293] "Volume detached for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/dd4c2563-ffde-4811-8705-79528793d93c-catalog-content\") on node \"crc\" DevicePath \"\"" Dec 03 11:48:58 crc kubenswrapper[5052]: I1203 11:48:58.201449 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-hqpht\" (UniqueName: \"kubernetes.io/projected/dd4c2563-ffde-4811-8705-79528793d93c-kube-api-access-hqpht\") on node \"crc\" DevicePath \"\"" Dec 03 11:48:58 crc kubenswrapper[5052]: I1203 11:48:58.201467 5052 reconciler_common.go:293] "Volume detached for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/dd4c2563-ffde-4811-8705-79528793d93c-utilities\") on node \"crc\" DevicePath \"\"" Dec 03 11:48:58 crc kubenswrapper[5052]: I1203 11:48:58.216199 5052 scope.go:117] "RemoveContainer" containerID="676f222e2427fdf810093272a42055ae13d78d9dbaf7d569b1f45fcdd24e2d6e" Dec 03 11:48:58 crc kubenswrapper[5052]: E1203 11:48:58.216902 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"676f222e2427fdf810093272a42055ae13d78d9dbaf7d569b1f45fcdd24e2d6e\": container with ID starting with 676f222e2427fdf810093272a42055ae13d78d9dbaf7d569b1f45fcdd24e2d6e not found: ID does not exist" containerID="676f222e2427fdf810093272a42055ae13d78d9dbaf7d569b1f45fcdd24e2d6e" Dec 03 11:48:58 crc kubenswrapper[5052]: I1203 11:48:58.216937 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"676f222e2427fdf810093272a42055ae13d78d9dbaf7d569b1f45fcdd24e2d6e"} err="failed to get container status \"676f222e2427fdf810093272a42055ae13d78d9dbaf7d569b1f45fcdd24e2d6e\": rpc error: code = NotFound desc = could not find container \"676f222e2427fdf810093272a42055ae13d78d9dbaf7d569b1f45fcdd24e2d6e\": container with ID starting with 676f222e2427fdf810093272a42055ae13d78d9dbaf7d569b1f45fcdd24e2d6e not found: ID does not exist" Dec 03 11:48:58 crc kubenswrapper[5052]: I1203 11:48:58.216963 5052 scope.go:117] "RemoveContainer" containerID="c4c24c798e9dc184297d47c688e65b5d9fd082fe66a0dd87714153cd96f9ebe2" Dec 03 11:48:58 crc kubenswrapper[5052]: E1203 11:48:58.217908 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c4c24c798e9dc184297d47c688e65b5d9fd082fe66a0dd87714153cd96f9ebe2\": container with ID starting with c4c24c798e9dc184297d47c688e65b5d9fd082fe66a0dd87714153cd96f9ebe2 not found: ID does not exist" containerID="c4c24c798e9dc184297d47c688e65b5d9fd082fe66a0dd87714153cd96f9ebe2" Dec 03 11:48:58 crc kubenswrapper[5052]: I1203 11:48:58.218005 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c4c24c798e9dc184297d47c688e65b5d9fd082fe66a0dd87714153cd96f9ebe2"} err="failed to get container status \"c4c24c798e9dc184297d47c688e65b5d9fd082fe66a0dd87714153cd96f9ebe2\": rpc error: code = NotFound desc = could not find container \"c4c24c798e9dc184297d47c688e65b5d9fd082fe66a0dd87714153cd96f9ebe2\": container with ID starting with c4c24c798e9dc184297d47c688e65b5d9fd082fe66a0dd87714153cd96f9ebe2 not found: ID does not exist" Dec 03 11:48:58 crc kubenswrapper[5052]: I1203 11:48:58.218061 5052 scope.go:117] "RemoveContainer" containerID="fbb0c95d9cb1c2489a130115cfafc7255ea4fb866b07fe9bc58477851da897c9" Dec 03 11:48:58 crc kubenswrapper[5052]: E1203 11:48:58.218677 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fbb0c95d9cb1c2489a130115cfafc7255ea4fb866b07fe9bc58477851da897c9\": container with ID starting with fbb0c95d9cb1c2489a130115cfafc7255ea4fb866b07fe9bc58477851da897c9 not found: ID does not exist" containerID="fbb0c95d9cb1c2489a130115cfafc7255ea4fb866b07fe9bc58477851da897c9" Dec 03 11:48:58 crc kubenswrapper[5052]: I1203 11:48:58.218712 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fbb0c95d9cb1c2489a130115cfafc7255ea4fb866b07fe9bc58477851da897c9"} err="failed to get container status \"fbb0c95d9cb1c2489a130115cfafc7255ea4fb866b07fe9bc58477851da897c9\": rpc error: code = NotFound desc = could not find container \"fbb0c95d9cb1c2489a130115cfafc7255ea4fb866b07fe9bc58477851da897c9\": container with ID starting with fbb0c95d9cb1c2489a130115cfafc7255ea4fb866b07fe9bc58477851da897c9 not found: ID does not exist" Dec 03 11:48:58 crc kubenswrapper[5052]: I1203 11:48:58.465960 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openshift-marketplace/redhat-marketplace-sbfvb"] Dec 03 11:48:58 crc kubenswrapper[5052]: I1203 11:48:58.473890 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openshift-marketplace/redhat-marketplace-sbfvb"] Dec 03 11:48:59 crc kubenswrapper[5052]: I1203 11:48:59.685979 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dd4c2563-ffde-4811-8705-79528793d93c" path="/var/lib/kubelet/pods/dd4c2563-ffde-4811-8705-79528793d93c/volumes" Dec 03 11:49:01 crc kubenswrapper[5052]: I1203 11:49:01.673088 5052 scope.go:117] "RemoveContainer" containerID="25fe88f8d102f09983f8884e65f0c6c1575ade67921f10a630e6dcc3ea8b506e" Dec 03 11:49:01 crc kubenswrapper[5052]: E1203 11:49:01.673731 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 11:49:06 crc kubenswrapper[5052]: I1203 11:49:06.668498 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/rabbitmq-server-0"] Dec 03 11:49:06 crc kubenswrapper[5052]: E1203 11:49:06.669123 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="f1ba93eb-7c01-4add-b32d-8523c5e7808e" containerName="storage" Dec 03 11:49:06 crc kubenswrapper[5052]: I1203 11:49:06.669148 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="f1ba93eb-7c01-4add-b32d-8523c5e7808e" containerName="storage" Dec 03 11:49:06 crc kubenswrapper[5052]: E1203 11:49:06.669192 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="dd4c2563-ffde-4811-8705-79528793d93c" containerName="extract-utilities" Dec 03 11:49:06 crc kubenswrapper[5052]: I1203 11:49:06.669211 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="dd4c2563-ffde-4811-8705-79528793d93c" containerName="extract-utilities" Dec 03 11:49:06 crc kubenswrapper[5052]: E1203 11:49:06.669238 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="dd4c2563-ffde-4811-8705-79528793d93c" containerName="extract-content" Dec 03 11:49:06 crc kubenswrapper[5052]: I1203 11:49:06.669256 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="dd4c2563-ffde-4811-8705-79528793d93c" containerName="extract-content" Dec 03 11:49:06 crc kubenswrapper[5052]: E1203 11:49:06.669284 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="dd4c2563-ffde-4811-8705-79528793d93c" containerName="registry-server" Dec 03 11:49:06 crc kubenswrapper[5052]: I1203 11:49:06.669300 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="dd4c2563-ffde-4811-8705-79528793d93c" containerName="registry-server" Dec 03 11:49:06 crc kubenswrapper[5052]: I1203 11:49:06.670675 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="dd4c2563-ffde-4811-8705-79528793d93c" containerName="registry-server" Dec 03 11:49:06 crc kubenswrapper[5052]: I1203 11:49:06.670855 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="f1ba93eb-7c01-4add-b32d-8523c5e7808e" containerName="storage" Dec 03 11:49:06 crc kubenswrapper[5052]: I1203 11:49:06.672574 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 11:49:06 crc kubenswrapper[5052]: I1203 11:49:06.677775 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"rabbitmq-default-user" Dec 03 11:49:06 crc kubenswrapper[5052]: I1203 11:49:06.677959 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"rabbitmq-server-conf" Dec 03 11:49:06 crc kubenswrapper[5052]: I1203 11:49:06.678144 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"rabbitmq-erlang-cookie" Dec 03 11:49:06 crc kubenswrapper[5052]: I1203 11:49:06.678418 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-rabbitmq-svc" Dec 03 11:49:06 crc kubenswrapper[5052]: I1203 11:49:06.678980 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"rabbitmq-plugins-conf" Dec 03 11:49:06 crc kubenswrapper[5052]: I1203 11:49:06.679145 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"rabbitmq-config-data" Dec 03 11:49:06 crc kubenswrapper[5052]: I1203 11:49:06.679391 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"rabbitmq-server-dockercfg-mrg8q" Dec 03 11:49:06 crc kubenswrapper[5052]: I1203 11:49:06.694772 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/rabbitmq-server-0"] Dec 03 11:49:06 crc kubenswrapper[5052]: I1203 11:49:06.827471 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"rabbitmq-tls\" (UniqueName: \"kubernetes.io/projected/10e3094e-f625-4d69-b0e9-358220c0d195-rabbitmq-tls\") pod \"rabbitmq-server-0\" (UID: \"10e3094e-f625-4d69-b0e9-358220c0d195\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 11:49:06 crc kubenswrapper[5052]: I1203 11:49:06.827634 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"server-conf\" (UniqueName: \"kubernetes.io/configmap/10e3094e-f625-4d69-b0e9-358220c0d195-server-conf\") pod \"rabbitmq-server-0\" (UID: \"10e3094e-f625-4d69-b0e9-358220c0d195\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 11:49:06 crc kubenswrapper[5052]: I1203 11:49:06.827664 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"rabbitmq-erlang-cookie\" (UniqueName: \"kubernetes.io/empty-dir/10e3094e-f625-4d69-b0e9-358220c0d195-rabbitmq-erlang-cookie\") pod \"rabbitmq-server-0\" (UID: \"10e3094e-f625-4d69-b0e9-358220c0d195\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 11:49:06 crc kubenswrapper[5052]: I1203 11:49:06.827685 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/10e3094e-f625-4d69-b0e9-358220c0d195-config-data\") pod \"rabbitmq-server-0\" (UID: \"10e3094e-f625-4d69-b0e9-358220c0d195\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 11:49:06 crc kubenswrapper[5052]: I1203 11:49:06.827771 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"erlang-cookie-secret\" (UniqueName: \"kubernetes.io/secret/10e3094e-f625-4d69-b0e9-358220c0d195-erlang-cookie-secret\") pod \"rabbitmq-server-0\" (UID: \"10e3094e-f625-4d69-b0e9-358220c0d195\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 11:49:06 crc kubenswrapper[5052]: I1203 11:49:06.827833 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"pod-info\" (UniqueName: \"kubernetes.io/downward-api/10e3094e-f625-4d69-b0e9-358220c0d195-pod-info\") pod \"rabbitmq-server-0\" (UID: \"10e3094e-f625-4d69-b0e9-358220c0d195\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 11:49:06 crc kubenswrapper[5052]: I1203 11:49:06.827869 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"rabbitmq-confd\" (UniqueName: \"kubernetes.io/projected/10e3094e-f625-4d69-b0e9-358220c0d195-rabbitmq-confd\") pod \"rabbitmq-server-0\" (UID: \"10e3094e-f625-4d69-b0e9-358220c0d195\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 11:49:06 crc kubenswrapper[5052]: I1203 11:49:06.827892 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"rabbitmq-plugins\" (UniqueName: \"kubernetes.io/empty-dir/10e3094e-f625-4d69-b0e9-358220c0d195-rabbitmq-plugins\") pod \"rabbitmq-server-0\" (UID: \"10e3094e-f625-4d69-b0e9-358220c0d195\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 11:49:06 crc kubenswrapper[5052]: I1203 11:49:06.827926 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xsgxx\" (UniqueName: \"kubernetes.io/projected/10e3094e-f625-4d69-b0e9-358220c0d195-kube-api-access-xsgxx\") pod \"rabbitmq-server-0\" (UID: \"10e3094e-f625-4d69-b0e9-358220c0d195\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 11:49:06 crc kubenswrapper[5052]: I1203 11:49:06.827959 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-conf\" (UniqueName: \"kubernetes.io/configmap/10e3094e-f625-4d69-b0e9-358220c0d195-plugins-conf\") pod \"rabbitmq-server-0\" (UID: \"10e3094e-f625-4d69-b0e9-358220c0d195\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 11:49:06 crc kubenswrapper[5052]: I1203 11:49:06.827980 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage08-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage08-crc\") pod \"rabbitmq-server-0\" (UID: \"10e3094e-f625-4d69-b0e9-358220c0d195\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 11:49:06 crc kubenswrapper[5052]: I1203 11:49:06.928812 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"server-conf\" (UniqueName: \"kubernetes.io/configmap/10e3094e-f625-4d69-b0e9-358220c0d195-server-conf\") pod \"rabbitmq-server-0\" (UID: \"10e3094e-f625-4d69-b0e9-358220c0d195\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 11:49:06 crc kubenswrapper[5052]: I1203 11:49:06.928867 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"rabbitmq-erlang-cookie\" (UniqueName: \"kubernetes.io/empty-dir/10e3094e-f625-4d69-b0e9-358220c0d195-rabbitmq-erlang-cookie\") pod \"rabbitmq-server-0\" (UID: \"10e3094e-f625-4d69-b0e9-358220c0d195\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 11:49:06 crc kubenswrapper[5052]: I1203 11:49:06.928894 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/10e3094e-f625-4d69-b0e9-358220c0d195-config-data\") pod \"rabbitmq-server-0\" (UID: \"10e3094e-f625-4d69-b0e9-358220c0d195\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 11:49:06 crc kubenswrapper[5052]: I1203 11:49:06.928929 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"erlang-cookie-secret\" (UniqueName: \"kubernetes.io/secret/10e3094e-f625-4d69-b0e9-358220c0d195-erlang-cookie-secret\") pod \"rabbitmq-server-0\" (UID: \"10e3094e-f625-4d69-b0e9-358220c0d195\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 11:49:06 crc kubenswrapper[5052]: I1203 11:49:06.928959 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"pod-info\" (UniqueName: \"kubernetes.io/downward-api/10e3094e-f625-4d69-b0e9-358220c0d195-pod-info\") pod \"rabbitmq-server-0\" (UID: \"10e3094e-f625-4d69-b0e9-358220c0d195\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 11:49:06 crc kubenswrapper[5052]: I1203 11:49:06.928984 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"rabbitmq-confd\" (UniqueName: \"kubernetes.io/projected/10e3094e-f625-4d69-b0e9-358220c0d195-rabbitmq-confd\") pod \"rabbitmq-server-0\" (UID: \"10e3094e-f625-4d69-b0e9-358220c0d195\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 11:49:06 crc kubenswrapper[5052]: I1203 11:49:06.929011 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"rabbitmq-plugins\" (UniqueName: \"kubernetes.io/empty-dir/10e3094e-f625-4d69-b0e9-358220c0d195-rabbitmq-plugins\") pod \"rabbitmq-server-0\" (UID: \"10e3094e-f625-4d69-b0e9-358220c0d195\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 11:49:06 crc kubenswrapper[5052]: I1203 11:49:06.929044 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-xsgxx\" (UniqueName: \"kubernetes.io/projected/10e3094e-f625-4d69-b0e9-358220c0d195-kube-api-access-xsgxx\") pod \"rabbitmq-server-0\" (UID: \"10e3094e-f625-4d69-b0e9-358220c0d195\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 11:49:06 crc kubenswrapper[5052]: I1203 11:49:06.929075 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"plugins-conf\" (UniqueName: \"kubernetes.io/configmap/10e3094e-f625-4d69-b0e9-358220c0d195-plugins-conf\") pod \"rabbitmq-server-0\" (UID: \"10e3094e-f625-4d69-b0e9-358220c0d195\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 11:49:06 crc kubenswrapper[5052]: I1203 11:49:06.929096 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage08-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage08-crc\") pod \"rabbitmq-server-0\" (UID: \"10e3094e-f625-4d69-b0e9-358220c0d195\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 11:49:06 crc kubenswrapper[5052]: I1203 11:49:06.929127 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"rabbitmq-tls\" (UniqueName: \"kubernetes.io/projected/10e3094e-f625-4d69-b0e9-358220c0d195-rabbitmq-tls\") pod \"rabbitmq-server-0\" (UID: \"10e3094e-f625-4d69-b0e9-358220c0d195\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 11:49:06 crc kubenswrapper[5052]: I1203 11:49:06.930536 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"rabbitmq-erlang-cookie\" (UniqueName: \"kubernetes.io/empty-dir/10e3094e-f625-4d69-b0e9-358220c0d195-rabbitmq-erlang-cookie\") pod \"rabbitmq-server-0\" (UID: \"10e3094e-f625-4d69-b0e9-358220c0d195\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 11:49:06 crc kubenswrapper[5052]: I1203 11:49:06.931244 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"plugins-conf\" (UniqueName: \"kubernetes.io/configmap/10e3094e-f625-4d69-b0e9-358220c0d195-plugins-conf\") pod \"rabbitmq-server-0\" (UID: \"10e3094e-f625-4d69-b0e9-358220c0d195\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 11:49:06 crc kubenswrapper[5052]: I1203 11:49:06.931487 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/10e3094e-f625-4d69-b0e9-358220c0d195-config-data\") pod \"rabbitmq-server-0\" (UID: \"10e3094e-f625-4d69-b0e9-358220c0d195\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 11:49:06 crc kubenswrapper[5052]: I1203 11:49:06.931663 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"server-conf\" (UniqueName: \"kubernetes.io/configmap/10e3094e-f625-4d69-b0e9-358220c0d195-server-conf\") pod \"rabbitmq-server-0\" (UID: \"10e3094e-f625-4d69-b0e9-358220c0d195\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 11:49:06 crc kubenswrapper[5052]: I1203 11:49:06.931938 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage08-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage08-crc\") pod \"rabbitmq-server-0\" (UID: \"10e3094e-f625-4d69-b0e9-358220c0d195\") device mount path \"/mnt/openstack/pv08\"" pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 11:49:06 crc kubenswrapper[5052]: I1203 11:49:06.932014 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"rabbitmq-plugins\" (UniqueName: \"kubernetes.io/empty-dir/10e3094e-f625-4d69-b0e9-358220c0d195-rabbitmq-plugins\") pod \"rabbitmq-server-0\" (UID: \"10e3094e-f625-4d69-b0e9-358220c0d195\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 11:49:06 crc kubenswrapper[5052]: I1203 11:49:06.935804 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/rabbitmq-cell1-server-0"] Dec 03 11:49:06 crc kubenswrapper[5052]: I1203 11:49:06.936625 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"pod-info\" (UniqueName: \"kubernetes.io/downward-api/10e3094e-f625-4d69-b0e9-358220c0d195-pod-info\") pod \"rabbitmq-server-0\" (UID: \"10e3094e-f625-4d69-b0e9-358220c0d195\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 11:49:06 crc kubenswrapper[5052]: I1203 11:49:06.937175 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 11:49:06 crc kubenswrapper[5052]: I1203 11:49:06.938913 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"rabbitmq-cell1-erlang-cookie" Dec 03 11:49:06 crc kubenswrapper[5052]: I1203 11:49:06.939188 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-rabbitmq-cell1-svc" Dec 03 11:49:06 crc kubenswrapper[5052]: I1203 11:49:06.940122 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"rabbitmq-cell1-plugins-conf" Dec 03 11:49:06 crc kubenswrapper[5052]: I1203 11:49:06.940188 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"rabbitmq-confd\" (UniqueName: \"kubernetes.io/projected/10e3094e-f625-4d69-b0e9-358220c0d195-rabbitmq-confd\") pod \"rabbitmq-server-0\" (UID: \"10e3094e-f625-4d69-b0e9-358220c0d195\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 11:49:06 crc kubenswrapper[5052]: I1203 11:49:06.943917 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"rabbitmq-cell1-server-conf" Dec 03 11:49:06 crc kubenswrapper[5052]: I1203 11:49:06.944270 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"rabbitmq-cell1-default-user" Dec 03 11:49:06 crc kubenswrapper[5052]: I1203 11:49:06.944491 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"rabbitmq-cell1-server-dockercfg-spw52" Dec 03 11:49:06 crc kubenswrapper[5052]: I1203 11:49:06.944834 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"rabbitmq-cell1-config-data" Dec 03 11:49:06 crc kubenswrapper[5052]: I1203 11:49:06.948189 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"rabbitmq-tls\" (UniqueName: \"kubernetes.io/projected/10e3094e-f625-4d69-b0e9-358220c0d195-rabbitmq-tls\") pod \"rabbitmq-server-0\" (UID: \"10e3094e-f625-4d69-b0e9-358220c0d195\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 11:49:06 crc kubenswrapper[5052]: I1203 11:49:06.949143 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"erlang-cookie-secret\" (UniqueName: \"kubernetes.io/secret/10e3094e-f625-4d69-b0e9-358220c0d195-erlang-cookie-secret\") pod \"rabbitmq-server-0\" (UID: \"10e3094e-f625-4d69-b0e9-358220c0d195\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 11:49:06 crc kubenswrapper[5052]: I1203 11:49:06.955836 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-xsgxx\" (UniqueName: \"kubernetes.io/projected/10e3094e-f625-4d69-b0e9-358220c0d195-kube-api-access-xsgxx\") pod \"rabbitmq-server-0\" (UID: \"10e3094e-f625-4d69-b0e9-358220c0d195\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 11:49:06 crc kubenswrapper[5052]: I1203 11:49:06.968041 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/rabbitmq-cell1-server-0"] Dec 03 11:49:06 crc kubenswrapper[5052]: I1203 11:49:06.992855 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage08-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage08-crc\") pod \"rabbitmq-server-0\" (UID: \"10e3094e-f625-4d69-b0e9-358220c0d195\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 11:49:06 crc kubenswrapper[5052]: I1203 11:49:06.998893 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 11:49:07 crc kubenswrapper[5052]: I1203 11:49:07.035672 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"rabbitmq-plugins\" (UniqueName: \"kubernetes.io/empty-dir/a5ba4775-815a-46ef-bd2d-05dfa7513288-rabbitmq-plugins\") pod \"rabbitmq-cell1-server-0\" (UID: \"a5ba4775-815a-46ef-bd2d-05dfa7513288\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 11:49:07 crc kubenswrapper[5052]: I1203 11:49:07.035767 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xbr2g\" (UniqueName: \"kubernetes.io/projected/a5ba4775-815a-46ef-bd2d-05dfa7513288-kube-api-access-xbr2g\") pod \"rabbitmq-cell1-server-0\" (UID: \"a5ba4775-815a-46ef-bd2d-05dfa7513288\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 11:49:07 crc kubenswrapper[5052]: I1203 11:49:07.035809 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"rabbitmq-tls\" (UniqueName: \"kubernetes.io/projected/a5ba4775-815a-46ef-bd2d-05dfa7513288-rabbitmq-tls\") pod \"rabbitmq-cell1-server-0\" (UID: \"a5ba4775-815a-46ef-bd2d-05dfa7513288\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 11:49:07 crc kubenswrapper[5052]: I1203 11:49:07.035841 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/a5ba4775-815a-46ef-bd2d-05dfa7513288-config-data\") pod \"rabbitmq-cell1-server-0\" (UID: \"a5ba4775-815a-46ef-bd2d-05dfa7513288\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 11:49:07 crc kubenswrapper[5052]: I1203 11:49:07.035892 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"server-conf\" (UniqueName: \"kubernetes.io/configmap/a5ba4775-815a-46ef-bd2d-05dfa7513288-server-conf\") pod \"rabbitmq-cell1-server-0\" (UID: \"a5ba4775-815a-46ef-bd2d-05dfa7513288\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 11:49:07 crc kubenswrapper[5052]: I1203 11:49:07.035922 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"erlang-cookie-secret\" (UniqueName: \"kubernetes.io/secret/a5ba4775-815a-46ef-bd2d-05dfa7513288-erlang-cookie-secret\") pod \"rabbitmq-cell1-server-0\" (UID: \"a5ba4775-815a-46ef-bd2d-05dfa7513288\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 11:49:07 crc kubenswrapper[5052]: I1203 11:49:07.035942 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"rabbitmq-erlang-cookie\" (UniqueName: \"kubernetes.io/empty-dir/a5ba4775-815a-46ef-bd2d-05dfa7513288-rabbitmq-erlang-cookie\") pod \"rabbitmq-cell1-server-0\" (UID: \"a5ba4775-815a-46ef-bd2d-05dfa7513288\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 11:49:07 crc kubenswrapper[5052]: I1203 11:49:07.035966 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"pod-info\" (UniqueName: \"kubernetes.io/downward-api/a5ba4775-815a-46ef-bd2d-05dfa7513288-pod-info\") pod \"rabbitmq-cell1-server-0\" (UID: \"a5ba4775-815a-46ef-bd2d-05dfa7513288\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 11:49:07 crc kubenswrapper[5052]: I1203 11:49:07.035987 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"rabbitmq-confd\" (UniqueName: \"kubernetes.io/projected/a5ba4775-815a-46ef-bd2d-05dfa7513288-rabbitmq-confd\") pod \"rabbitmq-cell1-server-0\" (UID: \"a5ba4775-815a-46ef-bd2d-05dfa7513288\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 11:49:07 crc kubenswrapper[5052]: I1203 11:49:07.036061 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-conf\" (UniqueName: \"kubernetes.io/configmap/a5ba4775-815a-46ef-bd2d-05dfa7513288-plugins-conf\") pod \"rabbitmq-cell1-server-0\" (UID: \"a5ba4775-815a-46ef-bd2d-05dfa7513288\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 11:49:07 crc kubenswrapper[5052]: I1203 11:49:07.036100 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage11-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage11-crc\") pod \"rabbitmq-cell1-server-0\" (UID: \"a5ba4775-815a-46ef-bd2d-05dfa7513288\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 11:49:07 crc kubenswrapper[5052]: I1203 11:49:07.138075 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"server-conf\" (UniqueName: \"kubernetes.io/configmap/a5ba4775-815a-46ef-bd2d-05dfa7513288-server-conf\") pod \"rabbitmq-cell1-server-0\" (UID: \"a5ba4775-815a-46ef-bd2d-05dfa7513288\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 11:49:07 crc kubenswrapper[5052]: I1203 11:49:07.138218 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"erlang-cookie-secret\" (UniqueName: \"kubernetes.io/secret/a5ba4775-815a-46ef-bd2d-05dfa7513288-erlang-cookie-secret\") pod \"rabbitmq-cell1-server-0\" (UID: \"a5ba4775-815a-46ef-bd2d-05dfa7513288\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 11:49:07 crc kubenswrapper[5052]: I1203 11:49:07.138250 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"rabbitmq-erlang-cookie\" (UniqueName: \"kubernetes.io/empty-dir/a5ba4775-815a-46ef-bd2d-05dfa7513288-rabbitmq-erlang-cookie\") pod \"rabbitmq-cell1-server-0\" (UID: \"a5ba4775-815a-46ef-bd2d-05dfa7513288\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 11:49:07 crc kubenswrapper[5052]: I1203 11:49:07.138276 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"pod-info\" (UniqueName: \"kubernetes.io/downward-api/a5ba4775-815a-46ef-bd2d-05dfa7513288-pod-info\") pod \"rabbitmq-cell1-server-0\" (UID: \"a5ba4775-815a-46ef-bd2d-05dfa7513288\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 11:49:07 crc kubenswrapper[5052]: I1203 11:49:07.138305 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"rabbitmq-confd\" (UniqueName: \"kubernetes.io/projected/a5ba4775-815a-46ef-bd2d-05dfa7513288-rabbitmq-confd\") pod \"rabbitmq-cell1-server-0\" (UID: \"a5ba4775-815a-46ef-bd2d-05dfa7513288\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 11:49:07 crc kubenswrapper[5052]: I1203 11:49:07.138382 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"plugins-conf\" (UniqueName: \"kubernetes.io/configmap/a5ba4775-815a-46ef-bd2d-05dfa7513288-plugins-conf\") pod \"rabbitmq-cell1-server-0\" (UID: \"a5ba4775-815a-46ef-bd2d-05dfa7513288\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 11:49:07 crc kubenswrapper[5052]: I1203 11:49:07.138439 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage11-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage11-crc\") pod \"rabbitmq-cell1-server-0\" (UID: \"a5ba4775-815a-46ef-bd2d-05dfa7513288\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 11:49:07 crc kubenswrapper[5052]: I1203 11:49:07.138484 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"rabbitmq-plugins\" (UniqueName: \"kubernetes.io/empty-dir/a5ba4775-815a-46ef-bd2d-05dfa7513288-rabbitmq-plugins\") pod \"rabbitmq-cell1-server-0\" (UID: \"a5ba4775-815a-46ef-bd2d-05dfa7513288\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 11:49:07 crc kubenswrapper[5052]: I1203 11:49:07.138624 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-xbr2g\" (UniqueName: \"kubernetes.io/projected/a5ba4775-815a-46ef-bd2d-05dfa7513288-kube-api-access-xbr2g\") pod \"rabbitmq-cell1-server-0\" (UID: \"a5ba4775-815a-46ef-bd2d-05dfa7513288\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 11:49:07 crc kubenswrapper[5052]: I1203 11:49:07.138687 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"rabbitmq-tls\" (UniqueName: \"kubernetes.io/projected/a5ba4775-815a-46ef-bd2d-05dfa7513288-rabbitmq-tls\") pod \"rabbitmq-cell1-server-0\" (UID: \"a5ba4775-815a-46ef-bd2d-05dfa7513288\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 11:49:07 crc kubenswrapper[5052]: I1203 11:49:07.138727 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/a5ba4775-815a-46ef-bd2d-05dfa7513288-config-data\") pod \"rabbitmq-cell1-server-0\" (UID: \"a5ba4775-815a-46ef-bd2d-05dfa7513288\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 11:49:07 crc kubenswrapper[5052]: I1203 11:49:07.139179 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage11-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage11-crc\") pod \"rabbitmq-cell1-server-0\" (UID: \"a5ba4775-815a-46ef-bd2d-05dfa7513288\") device mount path \"/mnt/openstack/pv11\"" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 11:49:07 crc kubenswrapper[5052]: I1203 11:49:07.140315 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"server-conf\" (UniqueName: \"kubernetes.io/configmap/a5ba4775-815a-46ef-bd2d-05dfa7513288-server-conf\") pod \"rabbitmq-cell1-server-0\" (UID: \"a5ba4775-815a-46ef-bd2d-05dfa7513288\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 11:49:07 crc kubenswrapper[5052]: I1203 11:49:07.140594 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/a5ba4775-815a-46ef-bd2d-05dfa7513288-config-data\") pod \"rabbitmq-cell1-server-0\" (UID: \"a5ba4775-815a-46ef-bd2d-05dfa7513288\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 11:49:07 crc kubenswrapper[5052]: I1203 11:49:07.145232 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"plugins-conf\" (UniqueName: \"kubernetes.io/configmap/a5ba4775-815a-46ef-bd2d-05dfa7513288-plugins-conf\") pod \"rabbitmq-cell1-server-0\" (UID: \"a5ba4775-815a-46ef-bd2d-05dfa7513288\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 11:49:07 crc kubenswrapper[5052]: I1203 11:49:07.145288 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"erlang-cookie-secret\" (UniqueName: \"kubernetes.io/secret/a5ba4775-815a-46ef-bd2d-05dfa7513288-erlang-cookie-secret\") pod \"rabbitmq-cell1-server-0\" (UID: \"a5ba4775-815a-46ef-bd2d-05dfa7513288\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 11:49:07 crc kubenswrapper[5052]: I1203 11:49:07.158864 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"rabbitmq-plugins\" (UniqueName: \"kubernetes.io/empty-dir/a5ba4775-815a-46ef-bd2d-05dfa7513288-rabbitmq-plugins\") pod \"rabbitmq-cell1-server-0\" (UID: \"a5ba4775-815a-46ef-bd2d-05dfa7513288\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 11:49:07 crc kubenswrapper[5052]: I1203 11:49:07.159135 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"rabbitmq-erlang-cookie\" (UniqueName: \"kubernetes.io/empty-dir/a5ba4775-815a-46ef-bd2d-05dfa7513288-rabbitmq-erlang-cookie\") pod \"rabbitmq-cell1-server-0\" (UID: \"a5ba4775-815a-46ef-bd2d-05dfa7513288\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 11:49:07 crc kubenswrapper[5052]: I1203 11:49:07.161234 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"pod-info\" (UniqueName: \"kubernetes.io/downward-api/a5ba4775-815a-46ef-bd2d-05dfa7513288-pod-info\") pod \"rabbitmq-cell1-server-0\" (UID: \"a5ba4775-815a-46ef-bd2d-05dfa7513288\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 11:49:07 crc kubenswrapper[5052]: I1203 11:49:07.163269 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"rabbitmq-confd\" (UniqueName: \"kubernetes.io/projected/a5ba4775-815a-46ef-bd2d-05dfa7513288-rabbitmq-confd\") pod \"rabbitmq-cell1-server-0\" (UID: \"a5ba4775-815a-46ef-bd2d-05dfa7513288\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 11:49:07 crc kubenswrapper[5052]: I1203 11:49:07.165535 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"rabbitmq-tls\" (UniqueName: \"kubernetes.io/projected/a5ba4775-815a-46ef-bd2d-05dfa7513288-rabbitmq-tls\") pod \"rabbitmq-cell1-server-0\" (UID: \"a5ba4775-815a-46ef-bd2d-05dfa7513288\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 11:49:07 crc kubenswrapper[5052]: I1203 11:49:07.169270 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-xbr2g\" (UniqueName: \"kubernetes.io/projected/a5ba4775-815a-46ef-bd2d-05dfa7513288-kube-api-access-xbr2g\") pod \"rabbitmq-cell1-server-0\" (UID: \"a5ba4775-815a-46ef-bd2d-05dfa7513288\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 11:49:07 crc kubenswrapper[5052]: I1203 11:49:07.191707 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage11-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage11-crc\") pod \"rabbitmq-cell1-server-0\" (UID: \"a5ba4775-815a-46ef-bd2d-05dfa7513288\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 11:49:07 crc kubenswrapper[5052]: I1203 11:49:07.370338 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 11:49:07 crc kubenswrapper[5052]: I1203 11:49:07.501221 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/rabbitmq-server-0"] Dec 03 11:49:07 crc kubenswrapper[5052]: I1203 11:49:07.868610 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/rabbitmq-cell1-server-0"] Dec 03 11:49:07 crc kubenswrapper[5052]: W1203 11:49:07.873794 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5ba4775_815a_46ef_bd2d_05dfa7513288.slice/crio-9cddb4e8af380ee07414ad1c06f823e0dea2c6ce693e4ed3a60ef4221d3e87e4 WatchSource:0}: Error finding container 9cddb4e8af380ee07414ad1c06f823e0dea2c6ce693e4ed3a60ef4221d3e87e4: Status 404 returned error can't find the container with id 9cddb4e8af380ee07414ad1c06f823e0dea2c6ce693e4ed3a60ef4221d3e87e4 Dec 03 11:49:08 crc kubenswrapper[5052]: I1203 11:49:08.219316 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" event={"ID":"a5ba4775-815a-46ef-bd2d-05dfa7513288","Type":"ContainerStarted","Data":"9cddb4e8af380ee07414ad1c06f823e0dea2c6ce693e4ed3a60ef4221d3e87e4"} Dec 03 11:49:08 crc kubenswrapper[5052]: I1203 11:49:08.220591 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-server-0" event={"ID":"10e3094e-f625-4d69-b0e9-358220c0d195","Type":"ContainerStarted","Data":"551a3ead523383a5c92a144136bec5a30d11a6644fcc4ed6012c53179ab67346"} Dec 03 11:49:08 crc kubenswrapper[5052]: I1203 11:49:08.480213 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/openstack-galera-0"] Dec 03 11:49:08 crc kubenswrapper[5052]: I1203 11:49:08.482277 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 11:49:08 crc kubenswrapper[5052]: I1203 11:49:08.486160 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"openstack-scripts" Dec 03 11:49:08 crc kubenswrapper[5052]: I1203 11:49:08.486160 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"galera-openstack-dockercfg-jfp47" Dec 03 11:49:08 crc kubenswrapper[5052]: I1203 11:49:08.486171 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"openstack-config-data" Dec 03 11:49:08 crc kubenswrapper[5052]: I1203 11:49:08.486171 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-galera-openstack-svc" Dec 03 11:49:08 crc kubenswrapper[5052]: I1203 11:49:08.506985 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"combined-ca-bundle" Dec 03 11:49:08 crc kubenswrapper[5052]: I1203 11:49:08.509924 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/openstack-galera-0"] Dec 03 11:49:08 crc kubenswrapper[5052]: I1203 11:49:08.662295 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f856a02f-3f21-4fd4-8158-50a232de756e-combined-ca-bundle\") pod \"openstack-galera-0\" (UID: \"f856a02f-3f21-4fd4-8158-50a232de756e\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 11:49:08 crc kubenswrapper[5052]: I1203 11:49:08.662623 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/f856a02f-3f21-4fd4-8158-50a232de756e-config-data-default\") pod \"openstack-galera-0\" (UID: \"f856a02f-3f21-4fd4-8158-50a232de756e\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 11:49:08 crc kubenswrapper[5052]: I1203 11:49:08.662741 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage03-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage03-crc\") pod \"openstack-galera-0\" (UID: \"f856a02f-3f21-4fd4-8158-50a232de756e\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 11:49:08 crc kubenswrapper[5052]: I1203 11:49:08.662877 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/f856a02f-3f21-4fd4-8158-50a232de756e-operator-scripts\") pod \"openstack-galera-0\" (UID: \"f856a02f-3f21-4fd4-8158-50a232de756e\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 11:49:08 crc kubenswrapper[5052]: I1203 11:49:08.663000 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/f856a02f-3f21-4fd4-8158-50a232de756e-config-data-generated\") pod \"openstack-galera-0\" (UID: \"f856a02f-3f21-4fd4-8158-50a232de756e\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 11:49:08 crc kubenswrapper[5052]: I1203 11:49:08.663109 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9bf6j\" (UniqueName: \"kubernetes.io/projected/f856a02f-3f21-4fd4-8158-50a232de756e-kube-api-access-9bf6j\") pod \"openstack-galera-0\" (UID: \"f856a02f-3f21-4fd4-8158-50a232de756e\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 11:49:08 crc kubenswrapper[5052]: I1203 11:49:08.663216 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/f856a02f-3f21-4fd4-8158-50a232de756e-galera-tls-certs\") pod \"openstack-galera-0\" (UID: \"f856a02f-3f21-4fd4-8158-50a232de756e\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 11:49:08 crc kubenswrapper[5052]: I1203 11:49:08.663314 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/f856a02f-3f21-4fd4-8158-50a232de756e-kolla-config\") pod \"openstack-galera-0\" (UID: \"f856a02f-3f21-4fd4-8158-50a232de756e\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 11:49:08 crc kubenswrapper[5052]: I1203 11:49:08.764261 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage03-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage03-crc\") pod \"openstack-galera-0\" (UID: \"f856a02f-3f21-4fd4-8158-50a232de756e\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 11:49:08 crc kubenswrapper[5052]: I1203 11:49:08.764316 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/f856a02f-3f21-4fd4-8158-50a232de756e-operator-scripts\") pod \"openstack-galera-0\" (UID: \"f856a02f-3f21-4fd4-8158-50a232de756e\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 11:49:08 crc kubenswrapper[5052]: I1203 11:49:08.764381 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/f856a02f-3f21-4fd4-8158-50a232de756e-config-data-generated\") pod \"openstack-galera-0\" (UID: \"f856a02f-3f21-4fd4-8158-50a232de756e\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 11:49:08 crc kubenswrapper[5052]: I1203 11:49:08.764410 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-9bf6j\" (UniqueName: \"kubernetes.io/projected/f856a02f-3f21-4fd4-8158-50a232de756e-kube-api-access-9bf6j\") pod \"openstack-galera-0\" (UID: \"f856a02f-3f21-4fd4-8158-50a232de756e\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 11:49:08 crc kubenswrapper[5052]: I1203 11:49:08.764441 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/f856a02f-3f21-4fd4-8158-50a232de756e-galera-tls-certs\") pod \"openstack-galera-0\" (UID: \"f856a02f-3f21-4fd4-8158-50a232de756e\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 11:49:08 crc kubenswrapper[5052]: I1203 11:49:08.764467 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/f856a02f-3f21-4fd4-8158-50a232de756e-kolla-config\") pod \"openstack-galera-0\" (UID: \"f856a02f-3f21-4fd4-8158-50a232de756e\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 11:49:08 crc kubenswrapper[5052]: I1203 11:49:08.764557 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f856a02f-3f21-4fd4-8158-50a232de756e-combined-ca-bundle\") pod \"openstack-galera-0\" (UID: \"f856a02f-3f21-4fd4-8158-50a232de756e\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 11:49:08 crc kubenswrapper[5052]: I1203 11:49:08.764608 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/f856a02f-3f21-4fd4-8158-50a232de756e-config-data-default\") pod \"openstack-galera-0\" (UID: \"f856a02f-3f21-4fd4-8158-50a232de756e\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 11:49:08 crc kubenswrapper[5052]: I1203 11:49:08.764700 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage03-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage03-crc\") pod \"openstack-galera-0\" (UID: \"f856a02f-3f21-4fd4-8158-50a232de756e\") device mount path \"/mnt/openstack/pv03\"" pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 11:49:08 crc kubenswrapper[5052]: I1203 11:49:08.764848 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/f856a02f-3f21-4fd4-8158-50a232de756e-config-data-generated\") pod \"openstack-galera-0\" (UID: \"f856a02f-3f21-4fd4-8158-50a232de756e\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 11:49:08 crc kubenswrapper[5052]: I1203 11:49:08.765360 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/f856a02f-3f21-4fd4-8158-50a232de756e-kolla-config\") pod \"openstack-galera-0\" (UID: \"f856a02f-3f21-4fd4-8158-50a232de756e\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 11:49:08 crc kubenswrapper[5052]: I1203 11:49:08.765748 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/f856a02f-3f21-4fd4-8158-50a232de756e-operator-scripts\") pod \"openstack-galera-0\" (UID: \"f856a02f-3f21-4fd4-8158-50a232de756e\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 11:49:08 crc kubenswrapper[5052]: I1203 11:49:08.766168 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/f856a02f-3f21-4fd4-8158-50a232de756e-config-data-default\") pod \"openstack-galera-0\" (UID: \"f856a02f-3f21-4fd4-8158-50a232de756e\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 11:49:08 crc kubenswrapper[5052]: I1203 11:49:08.783503 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f856a02f-3f21-4fd4-8158-50a232de756e-combined-ca-bundle\") pod \"openstack-galera-0\" (UID: \"f856a02f-3f21-4fd4-8158-50a232de756e\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 11:49:08 crc kubenswrapper[5052]: I1203 11:49:08.786266 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/f856a02f-3f21-4fd4-8158-50a232de756e-galera-tls-certs\") pod \"openstack-galera-0\" (UID: \"f856a02f-3f21-4fd4-8158-50a232de756e\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 11:49:08 crc kubenswrapper[5052]: I1203 11:49:08.787464 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-9bf6j\" (UniqueName: \"kubernetes.io/projected/f856a02f-3f21-4fd4-8158-50a232de756e-kube-api-access-9bf6j\") pod \"openstack-galera-0\" (UID: \"f856a02f-3f21-4fd4-8158-50a232de756e\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 11:49:08 crc kubenswrapper[5052]: I1203 11:49:08.802284 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage03-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage03-crc\") pod \"openstack-galera-0\" (UID: \"f856a02f-3f21-4fd4-8158-50a232de756e\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 11:49:09 crc kubenswrapper[5052]: I1203 11:49:09.102722 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 11:49:09 crc kubenswrapper[5052]: I1203 11:49:09.230689 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" event={"ID":"a5ba4775-815a-46ef-bd2d-05dfa7513288","Type":"ContainerStarted","Data":"f2cfc34275d0c16e884df462f785981d52cd8107d11e34b450fe684cd48e4f3b"} Dec 03 11:49:09 crc kubenswrapper[5052]: I1203 11:49:09.233122 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-server-0" event={"ID":"10e3094e-f625-4d69-b0e9-358220c0d195","Type":"ContainerStarted","Data":"c42837bc18a40995ef2cc940533425d21d777f73ad9d7999540fb90eedd02248"} Dec 03 11:49:09 crc kubenswrapper[5052]: I1203 11:49:09.445845 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/openstack-galera-0"] Dec 03 11:49:09 crc kubenswrapper[5052]: I1203 11:49:09.842379 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/openstack-cell1-galera-0"] Dec 03 11:49:09 crc kubenswrapper[5052]: I1203 11:49:09.844196 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 11:49:09 crc kubenswrapper[5052]: I1203 11:49:09.846118 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-galera-openstack-cell1-svc" Dec 03 11:49:09 crc kubenswrapper[5052]: I1203 11:49:09.846470 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"galera-openstack-cell1-dockercfg-cv9ln" Dec 03 11:49:09 crc kubenswrapper[5052]: I1203 11:49:09.846578 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"openstack-cell1-config-data" Dec 03 11:49:09 crc kubenswrapper[5052]: I1203 11:49:09.850276 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"openstack-cell1-scripts" Dec 03 11:49:09 crc kubenswrapper[5052]: I1203 11:49:09.863731 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/openstack-cell1-galera-0"] Dec 03 11:49:09 crc kubenswrapper[5052]: I1203 11:49:09.984912 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage17-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage17-crc\") pod \"openstack-cell1-galera-0\" (UID: \"8dd63158-2c75-455f-8f58-57a55c7e0f22\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 11:49:09 crc kubenswrapper[5052]: I1203 11:49:09.984969 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8dd63158-2c75-455f-8f58-57a55c7e0f22-combined-ca-bundle\") pod \"openstack-cell1-galera-0\" (UID: \"8dd63158-2c75-455f-8f58-57a55c7e0f22\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 11:49:09 crc kubenswrapper[5052]: I1203 11:49:09.985004 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/8dd63158-2c75-455f-8f58-57a55c7e0f22-kolla-config\") pod \"openstack-cell1-galera-0\" (UID: \"8dd63158-2c75-455f-8f58-57a55c7e0f22\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 11:49:09 crc kubenswrapper[5052]: I1203 11:49:09.985034 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/8dd63158-2c75-455f-8f58-57a55c7e0f22-config-data-default\") pod \"openstack-cell1-galera-0\" (UID: \"8dd63158-2c75-455f-8f58-57a55c7e0f22\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 11:49:09 crc kubenswrapper[5052]: I1203 11:49:09.985066 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/8dd63158-2c75-455f-8f58-57a55c7e0f22-config-data-generated\") pod \"openstack-cell1-galera-0\" (UID: \"8dd63158-2c75-455f-8f58-57a55c7e0f22\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 11:49:09 crc kubenswrapper[5052]: I1203 11:49:09.985120 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/8dd63158-2c75-455f-8f58-57a55c7e0f22-galera-tls-certs\") pod \"openstack-cell1-galera-0\" (UID: \"8dd63158-2c75-455f-8f58-57a55c7e0f22\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 11:49:09 crc kubenswrapper[5052]: I1203 11:49:09.985145 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-swksq\" (UniqueName: \"kubernetes.io/projected/8dd63158-2c75-455f-8f58-57a55c7e0f22-kube-api-access-swksq\") pod \"openstack-cell1-galera-0\" (UID: \"8dd63158-2c75-455f-8f58-57a55c7e0f22\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 11:49:09 crc kubenswrapper[5052]: I1203 11:49:09.985237 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/8dd63158-2c75-455f-8f58-57a55c7e0f22-operator-scripts\") pod \"openstack-cell1-galera-0\" (UID: \"8dd63158-2c75-455f-8f58-57a55c7e0f22\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 11:49:10 crc kubenswrapper[5052]: I1203 11:49:10.086345 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/8dd63158-2c75-455f-8f58-57a55c7e0f22-operator-scripts\") pod \"openstack-cell1-galera-0\" (UID: \"8dd63158-2c75-455f-8f58-57a55c7e0f22\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 11:49:10 crc kubenswrapper[5052]: I1203 11:49:10.086398 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage17-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage17-crc\") pod \"openstack-cell1-galera-0\" (UID: \"8dd63158-2c75-455f-8f58-57a55c7e0f22\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 11:49:10 crc kubenswrapper[5052]: I1203 11:49:10.086416 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8dd63158-2c75-455f-8f58-57a55c7e0f22-combined-ca-bundle\") pod \"openstack-cell1-galera-0\" (UID: \"8dd63158-2c75-455f-8f58-57a55c7e0f22\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 11:49:10 crc kubenswrapper[5052]: I1203 11:49:10.086440 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/8dd63158-2c75-455f-8f58-57a55c7e0f22-kolla-config\") pod \"openstack-cell1-galera-0\" (UID: \"8dd63158-2c75-455f-8f58-57a55c7e0f22\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 11:49:10 crc kubenswrapper[5052]: I1203 11:49:10.086773 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage17-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage17-crc\") pod \"openstack-cell1-galera-0\" (UID: \"8dd63158-2c75-455f-8f58-57a55c7e0f22\") device mount path \"/mnt/openstack/pv17\"" pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 11:49:10 crc kubenswrapper[5052]: I1203 11:49:10.087355 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/8dd63158-2c75-455f-8f58-57a55c7e0f22-kolla-config\") pod \"openstack-cell1-galera-0\" (UID: \"8dd63158-2c75-455f-8f58-57a55c7e0f22\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 11:49:10 crc kubenswrapper[5052]: I1203 11:49:10.087404 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/8dd63158-2c75-455f-8f58-57a55c7e0f22-config-data-default\") pod \"openstack-cell1-galera-0\" (UID: \"8dd63158-2c75-455f-8f58-57a55c7e0f22\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 11:49:10 crc kubenswrapper[5052]: I1203 11:49:10.087450 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/8dd63158-2c75-455f-8f58-57a55c7e0f22-config-data-generated\") pod \"openstack-cell1-galera-0\" (UID: \"8dd63158-2c75-455f-8f58-57a55c7e0f22\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 11:49:10 crc kubenswrapper[5052]: I1203 11:49:10.087493 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/8dd63158-2c75-455f-8f58-57a55c7e0f22-config-data-default\") pod \"openstack-cell1-galera-0\" (UID: \"8dd63158-2c75-455f-8f58-57a55c7e0f22\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 11:49:10 crc kubenswrapper[5052]: I1203 11:49:10.087765 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/8dd63158-2c75-455f-8f58-57a55c7e0f22-config-data-generated\") pod \"openstack-cell1-galera-0\" (UID: \"8dd63158-2c75-455f-8f58-57a55c7e0f22\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 11:49:10 crc kubenswrapper[5052]: I1203 11:49:10.087492 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/8dd63158-2c75-455f-8f58-57a55c7e0f22-galera-tls-certs\") pod \"openstack-cell1-galera-0\" (UID: \"8dd63158-2c75-455f-8f58-57a55c7e0f22\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 11:49:10 crc kubenswrapper[5052]: I1203 11:49:10.087839 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-swksq\" (UniqueName: \"kubernetes.io/projected/8dd63158-2c75-455f-8f58-57a55c7e0f22-kube-api-access-swksq\") pod \"openstack-cell1-galera-0\" (UID: \"8dd63158-2c75-455f-8f58-57a55c7e0f22\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 11:49:10 crc kubenswrapper[5052]: I1203 11:49:10.087911 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/8dd63158-2c75-455f-8f58-57a55c7e0f22-operator-scripts\") pod \"openstack-cell1-galera-0\" (UID: \"8dd63158-2c75-455f-8f58-57a55c7e0f22\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 11:49:10 crc kubenswrapper[5052]: I1203 11:49:10.092829 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8dd63158-2c75-455f-8f58-57a55c7e0f22-combined-ca-bundle\") pod \"openstack-cell1-galera-0\" (UID: \"8dd63158-2c75-455f-8f58-57a55c7e0f22\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 11:49:10 crc kubenswrapper[5052]: I1203 11:49:10.094299 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/8dd63158-2c75-455f-8f58-57a55c7e0f22-galera-tls-certs\") pod \"openstack-cell1-galera-0\" (UID: \"8dd63158-2c75-455f-8f58-57a55c7e0f22\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 11:49:10 crc kubenswrapper[5052]: I1203 11:49:10.108239 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-swksq\" (UniqueName: \"kubernetes.io/projected/8dd63158-2c75-455f-8f58-57a55c7e0f22-kube-api-access-swksq\") pod \"openstack-cell1-galera-0\" (UID: \"8dd63158-2c75-455f-8f58-57a55c7e0f22\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 11:49:10 crc kubenswrapper[5052]: I1203 11:49:10.122930 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage17-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage17-crc\") pod \"openstack-cell1-galera-0\" (UID: \"8dd63158-2c75-455f-8f58-57a55c7e0f22\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 11:49:10 crc kubenswrapper[5052]: I1203 11:49:10.158638 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 11:49:10 crc kubenswrapper[5052]: I1203 11:49:10.202886 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/memcached-0"] Dec 03 11:49:10 crc kubenswrapper[5052]: I1203 11:49:10.213541 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/memcached-0" Dec 03 11:49:10 crc kubenswrapper[5052]: I1203 11:49:10.217014 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-memcached-svc" Dec 03 11:49:10 crc kubenswrapper[5052]: I1203 11:49:10.217275 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"memcached-config-data" Dec 03 11:49:10 crc kubenswrapper[5052]: I1203 11:49:10.217472 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"memcached-memcached-dockercfg-qj6hc" Dec 03 11:49:10 crc kubenswrapper[5052]: I1203 11:49:10.225314 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/memcached-0"] Dec 03 11:49:10 crc kubenswrapper[5052]: I1203 11:49:10.252583 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-galera-0" event={"ID":"f856a02f-3f21-4fd4-8158-50a232de756e","Type":"ContainerStarted","Data":"14eb89a7a38e6b85144c32603126a266bb620c9850f97165f7900e4b5c32df21"} Dec 03 11:49:10 crc kubenswrapper[5052]: I1203 11:49:10.252665 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-galera-0" event={"ID":"f856a02f-3f21-4fd4-8158-50a232de756e","Type":"ContainerStarted","Data":"e9a78ff070e5ba47efabe3e32a44713239d9f7733bea04dd31fc687e43dc182e"} Dec 03 11:49:10 crc kubenswrapper[5052]: I1203 11:49:10.292001 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"memcached-tls-certs\" (UniqueName: \"kubernetes.io/secret/53a152d4-c58c-40a7-b4ce-b9a9dc42337d-memcached-tls-certs\") pod \"memcached-0\" (UID: \"53a152d4-c58c-40a7-b4ce-b9a9dc42337d\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 11:49:10 crc kubenswrapper[5052]: I1203 11:49:10.292048 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/53a152d4-c58c-40a7-b4ce-b9a9dc42337d-kolla-config\") pod \"memcached-0\" (UID: \"53a152d4-c58c-40a7-b4ce-b9a9dc42337d\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 11:49:10 crc kubenswrapper[5052]: I1203 11:49:10.292079 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/53a152d4-c58c-40a7-b4ce-b9a9dc42337d-config-data\") pod \"memcached-0\" (UID: \"53a152d4-c58c-40a7-b4ce-b9a9dc42337d\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 11:49:10 crc kubenswrapper[5052]: I1203 11:49:10.292163 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/53a152d4-c58c-40a7-b4ce-b9a9dc42337d-combined-ca-bundle\") pod \"memcached-0\" (UID: \"53a152d4-c58c-40a7-b4ce-b9a9dc42337d\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 11:49:10 crc kubenswrapper[5052]: I1203 11:49:10.292211 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j5gfw\" (UniqueName: \"kubernetes.io/projected/53a152d4-c58c-40a7-b4ce-b9a9dc42337d-kube-api-access-j5gfw\") pod \"memcached-0\" (UID: \"53a152d4-c58c-40a7-b4ce-b9a9dc42337d\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 11:49:10 crc kubenswrapper[5052]: I1203 11:49:10.401023 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/53a152d4-c58c-40a7-b4ce-b9a9dc42337d-combined-ca-bundle\") pod \"memcached-0\" (UID: \"53a152d4-c58c-40a7-b4ce-b9a9dc42337d\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 11:49:10 crc kubenswrapper[5052]: I1203 11:49:10.401068 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-j5gfw\" (UniqueName: \"kubernetes.io/projected/53a152d4-c58c-40a7-b4ce-b9a9dc42337d-kube-api-access-j5gfw\") pod \"memcached-0\" (UID: \"53a152d4-c58c-40a7-b4ce-b9a9dc42337d\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 11:49:10 crc kubenswrapper[5052]: I1203 11:49:10.401168 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"memcached-tls-certs\" (UniqueName: \"kubernetes.io/secret/53a152d4-c58c-40a7-b4ce-b9a9dc42337d-memcached-tls-certs\") pod \"memcached-0\" (UID: \"53a152d4-c58c-40a7-b4ce-b9a9dc42337d\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 11:49:10 crc kubenswrapper[5052]: I1203 11:49:10.401192 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/53a152d4-c58c-40a7-b4ce-b9a9dc42337d-kolla-config\") pod \"memcached-0\" (UID: \"53a152d4-c58c-40a7-b4ce-b9a9dc42337d\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 11:49:10 crc kubenswrapper[5052]: I1203 11:49:10.401214 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/53a152d4-c58c-40a7-b4ce-b9a9dc42337d-config-data\") pod \"memcached-0\" (UID: \"53a152d4-c58c-40a7-b4ce-b9a9dc42337d\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 11:49:10 crc kubenswrapper[5052]: I1203 11:49:10.402033 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/53a152d4-c58c-40a7-b4ce-b9a9dc42337d-config-data\") pod \"memcached-0\" (UID: \"53a152d4-c58c-40a7-b4ce-b9a9dc42337d\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 11:49:10 crc kubenswrapper[5052]: I1203 11:49:10.408678 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/53a152d4-c58c-40a7-b4ce-b9a9dc42337d-kolla-config\") pod \"memcached-0\" (UID: \"53a152d4-c58c-40a7-b4ce-b9a9dc42337d\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 11:49:10 crc kubenswrapper[5052]: I1203 11:49:10.410921 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"memcached-tls-certs\" (UniqueName: \"kubernetes.io/secret/53a152d4-c58c-40a7-b4ce-b9a9dc42337d-memcached-tls-certs\") pod \"memcached-0\" (UID: \"53a152d4-c58c-40a7-b4ce-b9a9dc42337d\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 11:49:10 crc kubenswrapper[5052]: I1203 11:49:10.426576 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-j5gfw\" (UniqueName: \"kubernetes.io/projected/53a152d4-c58c-40a7-b4ce-b9a9dc42337d-kube-api-access-j5gfw\") pod \"memcached-0\" (UID: \"53a152d4-c58c-40a7-b4ce-b9a9dc42337d\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 11:49:10 crc kubenswrapper[5052]: I1203 11:49:10.427237 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/53a152d4-c58c-40a7-b4ce-b9a9dc42337d-combined-ca-bundle\") pod \"memcached-0\" (UID: \"53a152d4-c58c-40a7-b4ce-b9a9dc42337d\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 11:49:10 crc kubenswrapper[5052]: I1203 11:49:10.561766 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/memcached-0" Dec 03 11:49:10 crc kubenswrapper[5052]: I1203 11:49:10.776739 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/openstack-cell1-galera-0"] Dec 03 11:49:10 crc kubenswrapper[5052]: W1203 11:49:10.781944 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8dd63158_2c75_455f_8f58_57a55c7e0f22.slice/crio-c0b9acd24dac2bb1464d4eb45c3f207e86ea3ec833bc0cc1d710c34de7af5f71 WatchSource:0}: Error finding container c0b9acd24dac2bb1464d4eb45c3f207e86ea3ec833bc0cc1d710c34de7af5f71: Status 404 returned error can't find the container with id c0b9acd24dac2bb1464d4eb45c3f207e86ea3ec833bc0cc1d710c34de7af5f71 Dec 03 11:49:10 crc kubenswrapper[5052]: I1203 11:49:10.981298 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/memcached-0"] Dec 03 11:49:10 crc kubenswrapper[5052]: W1203 11:49:10.989865 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod53a152d4_c58c_40a7_b4ce_b9a9dc42337d.slice/crio-7699d5b4862dda1b26c99cc5f67d2c38366ff6be793417801fb8d505bf49ce00 WatchSource:0}: Error finding container 7699d5b4862dda1b26c99cc5f67d2c38366ff6be793417801fb8d505bf49ce00: Status 404 returned error can't find the container with id 7699d5b4862dda1b26c99cc5f67d2c38366ff6be793417801fb8d505bf49ce00 Dec 03 11:49:11 crc kubenswrapper[5052]: I1203 11:49:11.261493 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/memcached-0" event={"ID":"53a152d4-c58c-40a7-b4ce-b9a9dc42337d","Type":"ContainerStarted","Data":"c2bfc844a9ad9b1a7f5228851e78306069141a35c777c8e2d5f4055edf6d1d62"} Dec 03 11:49:11 crc kubenswrapper[5052]: I1203 11:49:11.261556 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/memcached-0" event={"ID":"53a152d4-c58c-40a7-b4ce-b9a9dc42337d","Type":"ContainerStarted","Data":"7699d5b4862dda1b26c99cc5f67d2c38366ff6be793417801fb8d505bf49ce00"} Dec 03 11:49:11 crc kubenswrapper[5052]: I1203 11:49:11.262381 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/memcached-0" Dec 03 11:49:11 crc kubenswrapper[5052]: I1203 11:49:11.262990 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-cell1-galera-0" event={"ID":"8dd63158-2c75-455f-8f58-57a55c7e0f22","Type":"ContainerStarted","Data":"129c9172d0e6d0d175261d5f3c8438735b0e303535baa4945b26905504879c36"} Dec 03 11:49:11 crc kubenswrapper[5052]: I1203 11:49:11.263029 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-cell1-galera-0" event={"ID":"8dd63158-2c75-455f-8f58-57a55c7e0f22","Type":"ContainerStarted","Data":"c0b9acd24dac2bb1464d4eb45c3f207e86ea3ec833bc0cc1d710c34de7af5f71"} Dec 03 11:49:11 crc kubenswrapper[5052]: I1203 11:49:11.280812 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/memcached-0" podStartSLOduration=1.280794277 podStartE2EDuration="1.280794277s" podCreationTimestamp="2025-12-03 11:49:10 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:49:11.276390861 +0000 UTC m=+1741.864632625" watchObservedRunningTime="2025-12-03 11:49:11.280794277 +0000 UTC m=+1741.869036041" Dec 03 11:49:11 crc kubenswrapper[5052]: I1203 11:49:11.942778 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/kube-state-metrics-0"] Dec 03 11:49:11 crc kubenswrapper[5052]: I1203 11:49:11.943900 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 11:49:11 crc kubenswrapper[5052]: I1203 11:49:11.953110 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"telemetry-ceilometer-dockercfg-97ztv" Dec 03 11:49:11 crc kubenswrapper[5052]: I1203 11:49:11.969319 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/kube-state-metrics-0"] Dec 03 11:49:12 crc kubenswrapper[5052]: I1203 11:49:12.125968 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-84ptz\" (UniqueName: \"kubernetes.io/projected/d21ea848-4642-4f06-b777-ae420fd6330e-kube-api-access-84ptz\") pod \"kube-state-metrics-0\" (UID: \"d21ea848-4642-4f06-b777-ae420fd6330e\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 11:49:12 crc kubenswrapper[5052]: I1203 11:49:12.227914 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-84ptz\" (UniqueName: \"kubernetes.io/projected/d21ea848-4642-4f06-b777-ae420fd6330e-kube-api-access-84ptz\") pod \"kube-state-metrics-0\" (UID: \"d21ea848-4642-4f06-b777-ae420fd6330e\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 11:49:12 crc kubenswrapper[5052]: I1203 11:49:12.251917 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-84ptz\" (UniqueName: \"kubernetes.io/projected/d21ea848-4642-4f06-b777-ae420fd6330e-kube-api-access-84ptz\") pod \"kube-state-metrics-0\" (UID: \"d21ea848-4642-4f06-b777-ae420fd6330e\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 11:49:12 crc kubenswrapper[5052]: I1203 11:49:12.261778 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 11:49:12 crc kubenswrapper[5052]: I1203 11:49:12.739132 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/kube-state-metrics-0"] Dec 03 11:49:13 crc kubenswrapper[5052]: I1203 11:49:13.283751 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/kube-state-metrics-0" event={"ID":"d21ea848-4642-4f06-b777-ae420fd6330e","Type":"ContainerStarted","Data":"643de97d8ca468a005dd92098fc487f71f02b13e452f12905f1e816ee46ad6d0"} Dec 03 11:49:13 crc kubenswrapper[5052]: I1203 11:49:13.672490 5052 scope.go:117] "RemoveContainer" containerID="25fe88f8d102f09983f8884e65f0c6c1575ade67921f10a630e6dcc3ea8b506e" Dec 03 11:49:13 crc kubenswrapper[5052]: E1203 11:49:13.672814 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 11:49:14 crc kubenswrapper[5052]: I1203 11:49:14.295315 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/kube-state-metrics-0" event={"ID":"d21ea848-4642-4f06-b777-ae420fd6330e","Type":"ContainerStarted","Data":"3a04155db4f0c7a1d45628519f9e6444be8550227cc239c1ea6e91f467326d18"} Dec 03 11:49:14 crc kubenswrapper[5052]: I1203 11:49:14.295649 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 11:49:14 crc kubenswrapper[5052]: I1203 11:49:14.297774 5052 generic.go:334] "Generic (PLEG): container finished" podID="f856a02f-3f21-4fd4-8158-50a232de756e" containerID="14eb89a7a38e6b85144c32603126a266bb620c9850f97165f7900e4b5c32df21" exitCode=0 Dec 03 11:49:14 crc kubenswrapper[5052]: I1203 11:49:14.297835 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-galera-0" event={"ID":"f856a02f-3f21-4fd4-8158-50a232de756e","Type":"ContainerDied","Data":"14eb89a7a38e6b85144c32603126a266bb620c9850f97165f7900e4b5c32df21"} Dec 03 11:49:14 crc kubenswrapper[5052]: I1203 11:49:14.320759 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/kube-state-metrics-0" podStartSLOduration=2.9532718940000002 podStartE2EDuration="3.320736612s" podCreationTimestamp="2025-12-03 11:49:11 +0000 UTC" firstStartedPulling="2025-12-03 11:49:12.74428494 +0000 UTC m=+1743.332526704" lastFinishedPulling="2025-12-03 11:49:13.111749658 +0000 UTC m=+1743.699991422" observedRunningTime="2025-12-03 11:49:14.311081675 +0000 UTC m=+1744.899323439" watchObservedRunningTime="2025-12-03 11:49:14.320736612 +0000 UTC m=+1744.908978386" Dec 03 11:49:15 crc kubenswrapper[5052]: I1203 11:49:15.307440 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-galera-0" event={"ID":"f856a02f-3f21-4fd4-8158-50a232de756e","Type":"ContainerStarted","Data":"8d6e9aac3a6cab1fd7cc9acbcd622c819b6570c8ebf974e1634165fe0334ddb0"} Dec 03 11:49:15 crc kubenswrapper[5052]: I1203 11:49:15.309764 5052 generic.go:334] "Generic (PLEG): container finished" podID="8dd63158-2c75-455f-8f58-57a55c7e0f22" containerID="129c9172d0e6d0d175261d5f3c8438735b0e303535baa4945b26905504879c36" exitCode=0 Dec 03 11:49:15 crc kubenswrapper[5052]: I1203 11:49:15.309838 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-cell1-galera-0" event={"ID":"8dd63158-2c75-455f-8f58-57a55c7e0f22","Type":"ContainerDied","Data":"129c9172d0e6d0d175261d5f3c8438735b0e303535baa4945b26905504879c36"} Dec 03 11:49:15 crc kubenswrapper[5052]: I1203 11:49:15.335887 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/openstack-galera-0" podStartSLOduration=8.335859076 podStartE2EDuration="8.335859076s" podCreationTimestamp="2025-12-03 11:49:07 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:49:15.328591128 +0000 UTC m=+1745.916832912" watchObservedRunningTime="2025-12-03 11:49:15.335859076 +0000 UTC m=+1745.924100860" Dec 03 11:49:15 crc kubenswrapper[5052]: I1203 11:49:15.969021 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ovsdbserver-nb-0"] Dec 03 11:49:15 crc kubenswrapper[5052]: I1203 11:49:15.970364 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 11:49:15 crc kubenswrapper[5052]: I1203 11:49:15.975915 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"ovndbcluster-nb-scripts" Dec 03 11:49:15 crc kubenswrapper[5052]: I1203 11:49:15.976039 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"ovndbcluster-nb-config" Dec 03 11:49:15 crc kubenswrapper[5052]: I1203 11:49:15.976092 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-ovndbcluster-nb-ovndbs" Dec 03 11:49:15 crc kubenswrapper[5052]: I1203 11:49:15.976040 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-ovn-metrics" Dec 03 11:49:15 crc kubenswrapper[5052]: I1203 11:49:15.982964 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-nb-0"] Dec 03 11:49:15 crc kubenswrapper[5052]: I1203 11:49:15.985601 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ovncluster-ovndbcluster-nb-dockercfg-pdh99" Dec 03 11:49:16 crc kubenswrapper[5052]: I1203 11:49:16.084546 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/b4961b81-97b8-4174-82bf-2d1d02cf8335-ovsdb-rundir\") pod \"ovsdbserver-nb-0\" (UID: \"b4961b81-97b8-4174-82bf-2d1d02cf8335\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 11:49:16 crc kubenswrapper[5052]: I1203 11:49:16.084594 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/b4961b81-97b8-4174-82bf-2d1d02cf8335-metrics-certs-tls-certs\") pod \"ovsdbserver-nb-0\" (UID: \"b4961b81-97b8-4174-82bf-2d1d02cf8335\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 11:49:16 crc kubenswrapper[5052]: I1203 11:49:16.084617 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/b4961b81-97b8-4174-82bf-2d1d02cf8335-scripts\") pod \"ovsdbserver-nb-0\" (UID: \"b4961b81-97b8-4174-82bf-2d1d02cf8335\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 11:49:16 crc kubenswrapper[5052]: I1203 11:49:16.084637 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2drl2\" (UniqueName: \"kubernetes.io/projected/b4961b81-97b8-4174-82bf-2d1d02cf8335-kube-api-access-2drl2\") pod \"ovsdbserver-nb-0\" (UID: \"b4961b81-97b8-4174-82bf-2d1d02cf8335\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 11:49:16 crc kubenswrapper[5052]: I1203 11:49:16.084684 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage19-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage19-crc\") pod \"ovsdbserver-nb-0\" (UID: \"b4961b81-97b8-4174-82bf-2d1d02cf8335\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 11:49:16 crc kubenswrapper[5052]: I1203 11:49:16.084701 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b4961b81-97b8-4174-82bf-2d1d02cf8335-combined-ca-bundle\") pod \"ovsdbserver-nb-0\" (UID: \"b4961b81-97b8-4174-82bf-2d1d02cf8335\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 11:49:16 crc kubenswrapper[5052]: I1203 11:49:16.084727 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovsdbserver-nb-tls-certs\" (UniqueName: \"kubernetes.io/secret/b4961b81-97b8-4174-82bf-2d1d02cf8335-ovsdbserver-nb-tls-certs\") pod \"ovsdbserver-nb-0\" (UID: \"b4961b81-97b8-4174-82bf-2d1d02cf8335\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 11:49:16 crc kubenswrapper[5052]: I1203 11:49:16.084752 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/b4961b81-97b8-4174-82bf-2d1d02cf8335-config\") pod \"ovsdbserver-nb-0\" (UID: \"b4961b81-97b8-4174-82bf-2d1d02cf8335\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 11:49:16 crc kubenswrapper[5052]: I1203 11:49:16.186198 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/b4961b81-97b8-4174-82bf-2d1d02cf8335-ovsdb-rundir\") pod \"ovsdbserver-nb-0\" (UID: \"b4961b81-97b8-4174-82bf-2d1d02cf8335\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 11:49:16 crc kubenswrapper[5052]: I1203 11:49:16.186238 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/b4961b81-97b8-4174-82bf-2d1d02cf8335-metrics-certs-tls-certs\") pod \"ovsdbserver-nb-0\" (UID: \"b4961b81-97b8-4174-82bf-2d1d02cf8335\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 11:49:16 crc kubenswrapper[5052]: I1203 11:49:16.186264 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/b4961b81-97b8-4174-82bf-2d1d02cf8335-scripts\") pod \"ovsdbserver-nb-0\" (UID: \"b4961b81-97b8-4174-82bf-2d1d02cf8335\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 11:49:16 crc kubenswrapper[5052]: I1203 11:49:16.186290 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-2drl2\" (UniqueName: \"kubernetes.io/projected/b4961b81-97b8-4174-82bf-2d1d02cf8335-kube-api-access-2drl2\") pod \"ovsdbserver-nb-0\" (UID: \"b4961b81-97b8-4174-82bf-2d1d02cf8335\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 11:49:16 crc kubenswrapper[5052]: I1203 11:49:16.186339 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage19-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage19-crc\") pod \"ovsdbserver-nb-0\" (UID: \"b4961b81-97b8-4174-82bf-2d1d02cf8335\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 11:49:16 crc kubenswrapper[5052]: I1203 11:49:16.186356 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b4961b81-97b8-4174-82bf-2d1d02cf8335-combined-ca-bundle\") pod \"ovsdbserver-nb-0\" (UID: \"b4961b81-97b8-4174-82bf-2d1d02cf8335\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 11:49:16 crc kubenswrapper[5052]: I1203 11:49:16.186381 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovsdbserver-nb-tls-certs\" (UniqueName: \"kubernetes.io/secret/b4961b81-97b8-4174-82bf-2d1d02cf8335-ovsdbserver-nb-tls-certs\") pod \"ovsdbserver-nb-0\" (UID: \"b4961b81-97b8-4174-82bf-2d1d02cf8335\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 11:49:16 crc kubenswrapper[5052]: I1203 11:49:16.186403 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/b4961b81-97b8-4174-82bf-2d1d02cf8335-config\") pod \"ovsdbserver-nb-0\" (UID: \"b4961b81-97b8-4174-82bf-2d1d02cf8335\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 11:49:16 crc kubenswrapper[5052]: I1203 11:49:16.186956 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage19-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage19-crc\") pod \"ovsdbserver-nb-0\" (UID: \"b4961b81-97b8-4174-82bf-2d1d02cf8335\") device mount path \"/mnt/openstack/pv19\"" pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 11:49:16 crc kubenswrapper[5052]: I1203 11:49:16.187480 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/b4961b81-97b8-4174-82bf-2d1d02cf8335-config\") pod \"ovsdbserver-nb-0\" (UID: \"b4961b81-97b8-4174-82bf-2d1d02cf8335\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 11:49:16 crc kubenswrapper[5052]: I1203 11:49:16.187768 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/b4961b81-97b8-4174-82bf-2d1d02cf8335-scripts\") pod \"ovsdbserver-nb-0\" (UID: \"b4961b81-97b8-4174-82bf-2d1d02cf8335\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 11:49:16 crc kubenswrapper[5052]: I1203 11:49:16.187865 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/b4961b81-97b8-4174-82bf-2d1d02cf8335-ovsdb-rundir\") pod \"ovsdbserver-nb-0\" (UID: \"b4961b81-97b8-4174-82bf-2d1d02cf8335\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 11:49:16 crc kubenswrapper[5052]: I1203 11:49:16.191925 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b4961b81-97b8-4174-82bf-2d1d02cf8335-combined-ca-bundle\") pod \"ovsdbserver-nb-0\" (UID: \"b4961b81-97b8-4174-82bf-2d1d02cf8335\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 11:49:16 crc kubenswrapper[5052]: I1203 11:49:16.193065 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/b4961b81-97b8-4174-82bf-2d1d02cf8335-metrics-certs-tls-certs\") pod \"ovsdbserver-nb-0\" (UID: \"b4961b81-97b8-4174-82bf-2d1d02cf8335\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 11:49:16 crc kubenswrapper[5052]: I1203 11:49:16.193161 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovsdbserver-nb-tls-certs\" (UniqueName: \"kubernetes.io/secret/b4961b81-97b8-4174-82bf-2d1d02cf8335-ovsdbserver-nb-tls-certs\") pod \"ovsdbserver-nb-0\" (UID: \"b4961b81-97b8-4174-82bf-2d1d02cf8335\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 11:49:16 crc kubenswrapper[5052]: I1203 11:49:16.207688 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-2drl2\" (UniqueName: \"kubernetes.io/projected/b4961b81-97b8-4174-82bf-2d1d02cf8335-kube-api-access-2drl2\") pod \"ovsdbserver-nb-0\" (UID: \"b4961b81-97b8-4174-82bf-2d1d02cf8335\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 11:49:16 crc kubenswrapper[5052]: I1203 11:49:16.212149 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage19-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage19-crc\") pod \"ovsdbserver-nb-0\" (UID: \"b4961b81-97b8-4174-82bf-2d1d02cf8335\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 11:49:16 crc kubenswrapper[5052]: I1203 11:49:16.286602 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 11:49:16 crc kubenswrapper[5052]: I1203 11:49:16.319148 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-cell1-galera-0" event={"ID":"8dd63158-2c75-455f-8f58-57a55c7e0f22","Type":"ContainerStarted","Data":"77ae7474c6477ac30175222ea524c1ac6cf15f586c4f63872104bcf7083b3836"} Dec 03 11:49:16 crc kubenswrapper[5052]: I1203 11:49:16.348088 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/openstack-cell1-galera-0" podStartSLOduration=8.348069646 podStartE2EDuration="8.348069646s" podCreationTimestamp="2025-12-03 11:49:08 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:49:16.344322958 +0000 UTC m=+1746.932564742" watchObservedRunningTime="2025-12-03 11:49:16.348069646 +0000 UTC m=+1746.936311400" Dec 03 11:49:16 crc kubenswrapper[5052]: I1203 11:49:16.704410 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-nb-0"] Dec 03 11:49:17 crc kubenswrapper[5052]: I1203 11:49:17.331396 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-nb-0" event={"ID":"b4961b81-97b8-4174-82bf-2d1d02cf8335","Type":"ContainerStarted","Data":"e54d2995d89269947a87a50c5c5e626a778c66e141e2d728f3fafc46c76e5a9b"} Dec 03 11:49:17 crc kubenswrapper[5052]: I1203 11:49:17.331726 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-nb-0" event={"ID":"b4961b81-97b8-4174-82bf-2d1d02cf8335","Type":"ContainerStarted","Data":"434f07c085aa632ec76c3e82580e18718305175f93ff194a656a5b66cfbf2180"} Dec 03 11:49:17 crc kubenswrapper[5052]: I1203 11:49:17.331740 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-nb-0" event={"ID":"b4961b81-97b8-4174-82bf-2d1d02cf8335","Type":"ContainerStarted","Data":"e519c2d639554564bd415c09d122b9a2625467e70003e8cbac73c52a20dbb731"} Dec 03 11:49:18 crc kubenswrapper[5052]: I1203 11:49:18.364450 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/ovsdbserver-nb-0" podStartSLOduration=4.364433805 podStartE2EDuration="4.364433805s" podCreationTimestamp="2025-12-03 11:49:14 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:49:17.364642721 +0000 UTC m=+1747.952884485" watchObservedRunningTime="2025-12-03 11:49:18.364433805 +0000 UTC m=+1748.952675569" Dec 03 11:49:18 crc kubenswrapper[5052]: I1203 11:49:18.370554 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ovsdbserver-sb-0"] Dec 03 11:49:18 crc kubenswrapper[5052]: I1203 11:49:18.371722 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 11:49:18 crc kubenswrapper[5052]: I1203 11:49:18.374375 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"ovndbcluster-sb-scripts" Dec 03 11:49:18 crc kubenswrapper[5052]: I1203 11:49:18.374431 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-ovndbcluster-sb-ovndbs" Dec 03 11:49:18 crc kubenswrapper[5052]: I1203 11:49:18.374618 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"ovndbcluster-sb-config" Dec 03 11:49:18 crc kubenswrapper[5052]: I1203 11:49:18.375626 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ovncluster-ovndbcluster-sb-dockercfg-9rtc8" Dec 03 11:49:18 crc kubenswrapper[5052]: I1203 11:49:18.382611 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-sb-0"] Dec 03 11:49:18 crc kubenswrapper[5052]: I1203 11:49:18.522095 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/75dd245b-b54f-4a9a-9627-cbf338a69566-config\") pod \"ovsdbserver-sb-0\" (UID: \"75dd245b-b54f-4a9a-9627-cbf338a69566\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 11:49:18 crc kubenswrapper[5052]: I1203 11:49:18.522143 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/75dd245b-b54f-4a9a-9627-cbf338a69566-ovsdb-rundir\") pod \"ovsdbserver-sb-0\" (UID: \"75dd245b-b54f-4a9a-9627-cbf338a69566\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 11:49:18 crc kubenswrapper[5052]: I1203 11:49:18.522244 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage01-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage01-crc\") pod \"ovsdbserver-sb-0\" (UID: \"75dd245b-b54f-4a9a-9627-cbf338a69566\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 11:49:18 crc kubenswrapper[5052]: I1203 11:49:18.522304 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/75dd245b-b54f-4a9a-9627-cbf338a69566-combined-ca-bundle\") pod \"ovsdbserver-sb-0\" (UID: \"75dd245b-b54f-4a9a-9627-cbf338a69566\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 11:49:18 crc kubenswrapper[5052]: I1203 11:49:18.522333 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/75dd245b-b54f-4a9a-9627-cbf338a69566-scripts\") pod \"ovsdbserver-sb-0\" (UID: \"75dd245b-b54f-4a9a-9627-cbf338a69566\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 11:49:18 crc kubenswrapper[5052]: I1203 11:49:18.522473 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovsdbserver-sb-tls-certs\" (UniqueName: \"kubernetes.io/secret/75dd245b-b54f-4a9a-9627-cbf338a69566-ovsdbserver-sb-tls-certs\") pod \"ovsdbserver-sb-0\" (UID: \"75dd245b-b54f-4a9a-9627-cbf338a69566\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 11:49:18 crc kubenswrapper[5052]: I1203 11:49:18.522686 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/75dd245b-b54f-4a9a-9627-cbf338a69566-metrics-certs-tls-certs\") pod \"ovsdbserver-sb-0\" (UID: \"75dd245b-b54f-4a9a-9627-cbf338a69566\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 11:49:18 crc kubenswrapper[5052]: I1203 11:49:18.522730 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cqprg\" (UniqueName: \"kubernetes.io/projected/75dd245b-b54f-4a9a-9627-cbf338a69566-kube-api-access-cqprg\") pod \"ovsdbserver-sb-0\" (UID: \"75dd245b-b54f-4a9a-9627-cbf338a69566\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 11:49:18 crc kubenswrapper[5052]: I1203 11:49:18.624530 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/75dd245b-b54f-4a9a-9627-cbf338a69566-combined-ca-bundle\") pod \"ovsdbserver-sb-0\" (UID: \"75dd245b-b54f-4a9a-9627-cbf338a69566\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 11:49:18 crc kubenswrapper[5052]: I1203 11:49:18.624581 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/75dd245b-b54f-4a9a-9627-cbf338a69566-scripts\") pod \"ovsdbserver-sb-0\" (UID: \"75dd245b-b54f-4a9a-9627-cbf338a69566\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 11:49:18 crc kubenswrapper[5052]: I1203 11:49:18.624625 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovsdbserver-sb-tls-certs\" (UniqueName: \"kubernetes.io/secret/75dd245b-b54f-4a9a-9627-cbf338a69566-ovsdbserver-sb-tls-certs\") pod \"ovsdbserver-sb-0\" (UID: \"75dd245b-b54f-4a9a-9627-cbf338a69566\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 11:49:18 crc kubenswrapper[5052]: I1203 11:49:18.624689 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/75dd245b-b54f-4a9a-9627-cbf338a69566-metrics-certs-tls-certs\") pod \"ovsdbserver-sb-0\" (UID: \"75dd245b-b54f-4a9a-9627-cbf338a69566\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 11:49:18 crc kubenswrapper[5052]: I1203 11:49:18.624717 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-cqprg\" (UniqueName: \"kubernetes.io/projected/75dd245b-b54f-4a9a-9627-cbf338a69566-kube-api-access-cqprg\") pod \"ovsdbserver-sb-0\" (UID: \"75dd245b-b54f-4a9a-9627-cbf338a69566\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 11:49:18 crc kubenswrapper[5052]: I1203 11:49:18.624744 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/75dd245b-b54f-4a9a-9627-cbf338a69566-config\") pod \"ovsdbserver-sb-0\" (UID: \"75dd245b-b54f-4a9a-9627-cbf338a69566\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 11:49:18 crc kubenswrapper[5052]: I1203 11:49:18.624772 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/75dd245b-b54f-4a9a-9627-cbf338a69566-ovsdb-rundir\") pod \"ovsdbserver-sb-0\" (UID: \"75dd245b-b54f-4a9a-9627-cbf338a69566\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 11:49:18 crc kubenswrapper[5052]: I1203 11:49:18.624827 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage01-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage01-crc\") pod \"ovsdbserver-sb-0\" (UID: \"75dd245b-b54f-4a9a-9627-cbf338a69566\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 11:49:18 crc kubenswrapper[5052]: I1203 11:49:18.625188 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage01-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage01-crc\") pod \"ovsdbserver-sb-0\" (UID: \"75dd245b-b54f-4a9a-9627-cbf338a69566\") device mount path \"/mnt/openstack/pv01\"" pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 11:49:18 crc kubenswrapper[5052]: I1203 11:49:18.625644 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/75dd245b-b54f-4a9a-9627-cbf338a69566-ovsdb-rundir\") pod \"ovsdbserver-sb-0\" (UID: \"75dd245b-b54f-4a9a-9627-cbf338a69566\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 11:49:18 crc kubenswrapper[5052]: I1203 11:49:18.625677 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/75dd245b-b54f-4a9a-9627-cbf338a69566-scripts\") pod \"ovsdbserver-sb-0\" (UID: \"75dd245b-b54f-4a9a-9627-cbf338a69566\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 11:49:18 crc kubenswrapper[5052]: I1203 11:49:18.626417 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/75dd245b-b54f-4a9a-9627-cbf338a69566-config\") pod \"ovsdbserver-sb-0\" (UID: \"75dd245b-b54f-4a9a-9627-cbf338a69566\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 11:49:18 crc kubenswrapper[5052]: I1203 11:49:18.635212 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/75dd245b-b54f-4a9a-9627-cbf338a69566-metrics-certs-tls-certs\") pod \"ovsdbserver-sb-0\" (UID: \"75dd245b-b54f-4a9a-9627-cbf338a69566\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 11:49:18 crc kubenswrapper[5052]: I1203 11:49:18.638275 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovsdbserver-sb-tls-certs\" (UniqueName: \"kubernetes.io/secret/75dd245b-b54f-4a9a-9627-cbf338a69566-ovsdbserver-sb-tls-certs\") pod \"ovsdbserver-sb-0\" (UID: \"75dd245b-b54f-4a9a-9627-cbf338a69566\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 11:49:18 crc kubenswrapper[5052]: I1203 11:49:18.642195 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/75dd245b-b54f-4a9a-9627-cbf338a69566-combined-ca-bundle\") pod \"ovsdbserver-sb-0\" (UID: \"75dd245b-b54f-4a9a-9627-cbf338a69566\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 11:49:18 crc kubenswrapper[5052]: I1203 11:49:18.646321 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-cqprg\" (UniqueName: \"kubernetes.io/projected/75dd245b-b54f-4a9a-9627-cbf338a69566-kube-api-access-cqprg\") pod \"ovsdbserver-sb-0\" (UID: \"75dd245b-b54f-4a9a-9627-cbf338a69566\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 11:49:18 crc kubenswrapper[5052]: I1203 11:49:18.648377 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage01-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage01-crc\") pod \"ovsdbserver-sb-0\" (UID: \"75dd245b-b54f-4a9a-9627-cbf338a69566\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 11:49:18 crc kubenswrapper[5052]: I1203 11:49:18.690920 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 11:49:19 crc kubenswrapper[5052]: I1203 11:49:19.103462 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 11:49:19 crc kubenswrapper[5052]: I1203 11:49:19.103611 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 11:49:19 crc kubenswrapper[5052]: I1203 11:49:19.132877 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-sb-0"] Dec 03 11:49:19 crc kubenswrapper[5052]: W1203 11:49:19.137831 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod75dd245b_b54f_4a9a_9627_cbf338a69566.slice/crio-3ca68891bf73f4d039a81c4089ffa5c05d2005f23b02f62e0858bc1c390192eb WatchSource:0}: Error finding container 3ca68891bf73f4d039a81c4089ffa5c05d2005f23b02f62e0858bc1c390192eb: Status 404 returned error can't find the container with id 3ca68891bf73f4d039a81c4089ffa5c05d2005f23b02f62e0858bc1c390192eb Dec 03 11:49:19 crc kubenswrapper[5052]: I1203 11:49:19.223279 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 11:49:19 crc kubenswrapper[5052]: I1203 11:49:19.287056 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 11:49:19 crc kubenswrapper[5052]: I1203 11:49:19.333244 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 11:49:19 crc kubenswrapper[5052]: I1203 11:49:19.348619 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-sb-0" event={"ID":"75dd245b-b54f-4a9a-9627-cbf338a69566","Type":"ContainerStarted","Data":"a3ef9454b0aa12b5e49b5ada9b84ef790f5b7a8483610db37b8081939ff19a0a"} Dec 03 11:49:19 crc kubenswrapper[5052]: I1203 11:49:19.348923 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-sb-0" event={"ID":"75dd245b-b54f-4a9a-9627-cbf338a69566","Type":"ContainerStarted","Data":"3ca68891bf73f4d039a81c4089ffa5c05d2005f23b02f62e0858bc1c390192eb"} Dec 03 11:49:19 crc kubenswrapper[5052]: I1203 11:49:19.349038 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 11:49:19 crc kubenswrapper[5052]: I1203 11:49:19.429863 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 11:49:20 crc kubenswrapper[5052]: I1203 11:49:20.107595 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/keystone-db-create-fj555"] Dec 03 11:49:20 crc kubenswrapper[5052]: I1203 11:49:20.108722 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-db-create-fj555" Dec 03 11:49:20 crc kubenswrapper[5052]: I1203 11:49:20.122679 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-db-create-fj555"] Dec 03 11:49:20 crc kubenswrapper[5052]: I1203 11:49:20.154843 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/f009f642-7d9a-4739-9e45-bf734d0a0eea-operator-scripts\") pod \"keystone-db-create-fj555\" (UID: \"f009f642-7d9a-4739-9e45-bf734d0a0eea\") " pod="openstack-kuttl-tests/keystone-db-create-fj555" Dec 03 11:49:20 crc kubenswrapper[5052]: I1203 11:49:20.154961 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p7pzx\" (UniqueName: \"kubernetes.io/projected/f009f642-7d9a-4739-9e45-bf734d0a0eea-kube-api-access-p7pzx\") pod \"keystone-db-create-fj555\" (UID: \"f009f642-7d9a-4739-9e45-bf734d0a0eea\") " pod="openstack-kuttl-tests/keystone-db-create-fj555" Dec 03 11:49:20 crc kubenswrapper[5052]: I1203 11:49:20.159516 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 11:49:20 crc kubenswrapper[5052]: I1203 11:49:20.159746 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 11:49:20 crc kubenswrapper[5052]: I1203 11:49:20.214290 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/keystone-42e8-account-create-update-7bd5x"] Dec 03 11:49:20 crc kubenswrapper[5052]: I1203 11:49:20.215242 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-42e8-account-create-update-7bd5x" Dec 03 11:49:20 crc kubenswrapper[5052]: I1203 11:49:20.218475 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone-db-secret" Dec 03 11:49:20 crc kubenswrapper[5052]: I1203 11:49:20.231602 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-42e8-account-create-update-7bd5x"] Dec 03 11:49:20 crc kubenswrapper[5052]: I1203 11:49:20.256943 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/f009f642-7d9a-4739-9e45-bf734d0a0eea-operator-scripts\") pod \"keystone-db-create-fj555\" (UID: \"f009f642-7d9a-4739-9e45-bf734d0a0eea\") " pod="openstack-kuttl-tests/keystone-db-create-fj555" Dec 03 11:49:20 crc kubenswrapper[5052]: I1203 11:49:20.257015 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/f5994dc8-c6d2-43c7-8035-7e0bfdc09445-operator-scripts\") pod \"keystone-42e8-account-create-update-7bd5x\" (UID: \"f5994dc8-c6d2-43c7-8035-7e0bfdc09445\") " pod="openstack-kuttl-tests/keystone-42e8-account-create-update-7bd5x" Dec 03 11:49:20 crc kubenswrapper[5052]: I1203 11:49:20.257070 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wj55b\" (UniqueName: \"kubernetes.io/projected/f5994dc8-c6d2-43c7-8035-7e0bfdc09445-kube-api-access-wj55b\") pod \"keystone-42e8-account-create-update-7bd5x\" (UID: \"f5994dc8-c6d2-43c7-8035-7e0bfdc09445\") " pod="openstack-kuttl-tests/keystone-42e8-account-create-update-7bd5x" Dec 03 11:49:20 crc kubenswrapper[5052]: I1203 11:49:20.257120 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-p7pzx\" (UniqueName: \"kubernetes.io/projected/f009f642-7d9a-4739-9e45-bf734d0a0eea-kube-api-access-p7pzx\") pod \"keystone-db-create-fj555\" (UID: \"f009f642-7d9a-4739-9e45-bf734d0a0eea\") " pod="openstack-kuttl-tests/keystone-db-create-fj555" Dec 03 11:49:20 crc kubenswrapper[5052]: I1203 11:49:20.258436 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/f009f642-7d9a-4739-9e45-bf734d0a0eea-operator-scripts\") pod \"keystone-db-create-fj555\" (UID: \"f009f642-7d9a-4739-9e45-bf734d0a0eea\") " pod="openstack-kuttl-tests/keystone-db-create-fj555" Dec 03 11:49:20 crc kubenswrapper[5052]: I1203 11:49:20.277064 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-p7pzx\" (UniqueName: \"kubernetes.io/projected/f009f642-7d9a-4739-9e45-bf734d0a0eea-kube-api-access-p7pzx\") pod \"keystone-db-create-fj555\" (UID: \"f009f642-7d9a-4739-9e45-bf734d0a0eea\") " pod="openstack-kuttl-tests/keystone-db-create-fj555" Dec 03 11:49:20 crc kubenswrapper[5052]: I1203 11:49:20.357616 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-sb-0" event={"ID":"75dd245b-b54f-4a9a-9627-cbf338a69566","Type":"ContainerStarted","Data":"ed36cfdff4484c95d4ba9ac84fbe8a541c4883ce7afbed30c9761965aace756c"} Dec 03 11:49:20 crc kubenswrapper[5052]: I1203 11:49:20.358976 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/f5994dc8-c6d2-43c7-8035-7e0bfdc09445-operator-scripts\") pod \"keystone-42e8-account-create-update-7bd5x\" (UID: \"f5994dc8-c6d2-43c7-8035-7e0bfdc09445\") " pod="openstack-kuttl-tests/keystone-42e8-account-create-update-7bd5x" Dec 03 11:49:20 crc kubenswrapper[5052]: I1203 11:49:20.359156 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-wj55b\" (UniqueName: \"kubernetes.io/projected/f5994dc8-c6d2-43c7-8035-7e0bfdc09445-kube-api-access-wj55b\") pod \"keystone-42e8-account-create-update-7bd5x\" (UID: \"f5994dc8-c6d2-43c7-8035-7e0bfdc09445\") " pod="openstack-kuttl-tests/keystone-42e8-account-create-update-7bd5x" Dec 03 11:49:20 crc kubenswrapper[5052]: I1203 11:49:20.360009 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/f5994dc8-c6d2-43c7-8035-7e0bfdc09445-operator-scripts\") pod \"keystone-42e8-account-create-update-7bd5x\" (UID: \"f5994dc8-c6d2-43c7-8035-7e0bfdc09445\") " pod="openstack-kuttl-tests/keystone-42e8-account-create-update-7bd5x" Dec 03 11:49:20 crc kubenswrapper[5052]: I1203 11:49:20.379988 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-wj55b\" (UniqueName: \"kubernetes.io/projected/f5994dc8-c6d2-43c7-8035-7e0bfdc09445-kube-api-access-wj55b\") pod \"keystone-42e8-account-create-update-7bd5x\" (UID: \"f5994dc8-c6d2-43c7-8035-7e0bfdc09445\") " pod="openstack-kuttl-tests/keystone-42e8-account-create-update-7bd5x" Dec 03 11:49:20 crc kubenswrapper[5052]: I1203 11:49:20.385420 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/ovsdbserver-sb-0" podStartSLOduration=3.385401376 podStartE2EDuration="3.385401376s" podCreationTimestamp="2025-12-03 11:49:17 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:49:20.374119722 +0000 UTC m=+1750.962361476" watchObservedRunningTime="2025-12-03 11:49:20.385401376 +0000 UTC m=+1750.973643140" Dec 03 11:49:20 crc kubenswrapper[5052]: I1203 11:49:20.424966 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 11:49:20 crc kubenswrapper[5052]: I1203 11:49:20.430358 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-db-create-fj555" Dec 03 11:49:20 crc kubenswrapper[5052]: I1203 11:49:20.441954 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/placement-db-create-8bls7"] Dec 03 11:49:20 crc kubenswrapper[5052]: I1203 11:49:20.443087 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-db-create-8bls7" Dec 03 11:49:20 crc kubenswrapper[5052]: I1203 11:49:20.459108 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/placement-db-create-8bls7"] Dec 03 11:49:20 crc kubenswrapper[5052]: I1203 11:49:20.527044 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/placement-2868-account-create-update-nbw4x"] Dec 03 11:49:20 crc kubenswrapper[5052]: I1203 11:49:20.528577 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-2868-account-create-update-nbw4x" Dec 03 11:49:20 crc kubenswrapper[5052]: I1203 11:49:20.530799 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"placement-db-secret" Dec 03 11:49:20 crc kubenswrapper[5052]: I1203 11:49:20.532958 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-42e8-account-create-update-7bd5x" Dec 03 11:49:20 crc kubenswrapper[5052]: I1203 11:49:20.536403 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/placement-2868-account-create-update-nbw4x"] Dec 03 11:49:20 crc kubenswrapper[5052]: I1203 11:49:20.579670 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/memcached-0" Dec 03 11:49:20 crc kubenswrapper[5052]: I1203 11:49:20.676530 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/79fd3287-4c54-497c-9081-16fbf13bbb48-operator-scripts\") pod \"placement-2868-account-create-update-nbw4x\" (UID: \"79fd3287-4c54-497c-9081-16fbf13bbb48\") " pod="openstack-kuttl-tests/placement-2868-account-create-update-nbw4x" Dec 03 11:49:20 crc kubenswrapper[5052]: I1203 11:49:20.676857 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/3df6012c-618d-434a-bfdb-56f8f602b05d-operator-scripts\") pod \"placement-db-create-8bls7\" (UID: \"3df6012c-618d-434a-bfdb-56f8f602b05d\") " pod="openstack-kuttl-tests/placement-db-create-8bls7" Dec 03 11:49:20 crc kubenswrapper[5052]: I1203 11:49:20.676892 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sz9gk\" (UniqueName: \"kubernetes.io/projected/79fd3287-4c54-497c-9081-16fbf13bbb48-kube-api-access-sz9gk\") pod \"placement-2868-account-create-update-nbw4x\" (UID: \"79fd3287-4c54-497c-9081-16fbf13bbb48\") " pod="openstack-kuttl-tests/placement-2868-account-create-update-nbw4x" Dec 03 11:49:20 crc kubenswrapper[5052]: I1203 11:49:20.676933 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rm6jg\" (UniqueName: \"kubernetes.io/projected/3df6012c-618d-434a-bfdb-56f8f602b05d-kube-api-access-rm6jg\") pod \"placement-db-create-8bls7\" (UID: \"3df6012c-618d-434a-bfdb-56f8f602b05d\") " pod="openstack-kuttl-tests/placement-db-create-8bls7" Dec 03 11:49:20 crc kubenswrapper[5052]: I1203 11:49:20.725833 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/glance-db-create-m6rsr"] Dec 03 11:49:20 crc kubenswrapper[5052]: I1203 11:49:20.726801 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-db-create-m6rsr" Dec 03 11:49:20 crc kubenswrapper[5052]: I1203 11:49:20.757636 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-db-create-m6rsr"] Dec 03 11:49:20 crc kubenswrapper[5052]: I1203 11:49:20.793440 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/79fd3287-4c54-497c-9081-16fbf13bbb48-operator-scripts\") pod \"placement-2868-account-create-update-nbw4x\" (UID: \"79fd3287-4c54-497c-9081-16fbf13bbb48\") " pod="openstack-kuttl-tests/placement-2868-account-create-update-nbw4x" Dec 03 11:49:20 crc kubenswrapper[5052]: I1203 11:49:20.793664 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/3df6012c-618d-434a-bfdb-56f8f602b05d-operator-scripts\") pod \"placement-db-create-8bls7\" (UID: \"3df6012c-618d-434a-bfdb-56f8f602b05d\") " pod="openstack-kuttl-tests/placement-db-create-8bls7" Dec 03 11:49:20 crc kubenswrapper[5052]: I1203 11:49:20.793797 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-sz9gk\" (UniqueName: \"kubernetes.io/projected/79fd3287-4c54-497c-9081-16fbf13bbb48-kube-api-access-sz9gk\") pod \"placement-2868-account-create-update-nbw4x\" (UID: \"79fd3287-4c54-497c-9081-16fbf13bbb48\") " pod="openstack-kuttl-tests/placement-2868-account-create-update-nbw4x" Dec 03 11:49:20 crc kubenswrapper[5052]: I1203 11:49:20.793913 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-rm6jg\" (UniqueName: \"kubernetes.io/projected/3df6012c-618d-434a-bfdb-56f8f602b05d-kube-api-access-rm6jg\") pod \"placement-db-create-8bls7\" (UID: \"3df6012c-618d-434a-bfdb-56f8f602b05d\") " pod="openstack-kuttl-tests/placement-db-create-8bls7" Dec 03 11:49:20 crc kubenswrapper[5052]: I1203 11:49:20.795368 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/79fd3287-4c54-497c-9081-16fbf13bbb48-operator-scripts\") pod \"placement-2868-account-create-update-nbw4x\" (UID: \"79fd3287-4c54-497c-9081-16fbf13bbb48\") " pod="openstack-kuttl-tests/placement-2868-account-create-update-nbw4x" Dec 03 11:49:20 crc kubenswrapper[5052]: I1203 11:49:20.796386 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/3df6012c-618d-434a-bfdb-56f8f602b05d-operator-scripts\") pod \"placement-db-create-8bls7\" (UID: \"3df6012c-618d-434a-bfdb-56f8f602b05d\") " pod="openstack-kuttl-tests/placement-db-create-8bls7" Dec 03 11:49:20 crc kubenswrapper[5052]: I1203 11:49:20.834889 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-rm6jg\" (UniqueName: \"kubernetes.io/projected/3df6012c-618d-434a-bfdb-56f8f602b05d-kube-api-access-rm6jg\") pod \"placement-db-create-8bls7\" (UID: \"3df6012c-618d-434a-bfdb-56f8f602b05d\") " pod="openstack-kuttl-tests/placement-db-create-8bls7" Dec 03 11:49:20 crc kubenswrapper[5052]: I1203 11:49:20.838028 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-sz9gk\" (UniqueName: \"kubernetes.io/projected/79fd3287-4c54-497c-9081-16fbf13bbb48-kube-api-access-sz9gk\") pod \"placement-2868-account-create-update-nbw4x\" (UID: \"79fd3287-4c54-497c-9081-16fbf13bbb48\") " pod="openstack-kuttl-tests/placement-2868-account-create-update-nbw4x" Dec 03 11:49:20 crc kubenswrapper[5052]: I1203 11:49:20.839251 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-db-create-8bls7" Dec 03 11:49:20 crc kubenswrapper[5052]: I1203 11:49:20.843718 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/glance-634c-account-create-update-mmdsg"] Dec 03 11:49:20 crc kubenswrapper[5052]: I1203 11:49:20.845447 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-634c-account-create-update-mmdsg" Dec 03 11:49:20 crc kubenswrapper[5052]: I1203 11:49:20.848227 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-db-secret" Dec 03 11:49:20 crc kubenswrapper[5052]: I1203 11:49:20.853371 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-634c-account-create-update-mmdsg"] Dec 03 11:49:20 crc kubenswrapper[5052]: I1203 11:49:20.897278 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zpwp8\" (UniqueName: \"kubernetes.io/projected/0071cca3-cb80-44f5-86f6-8abc8399d732-kube-api-access-zpwp8\") pod \"glance-634c-account-create-update-mmdsg\" (UID: \"0071cca3-cb80-44f5-86f6-8abc8399d732\") " pod="openstack-kuttl-tests/glance-634c-account-create-update-mmdsg" Dec 03 11:49:20 crc kubenswrapper[5052]: I1203 11:49:20.897404 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/00b2551c-6bbe-444d-abb0-2267676dfb09-operator-scripts\") pod \"glance-db-create-m6rsr\" (UID: \"00b2551c-6bbe-444d-abb0-2267676dfb09\") " pod="openstack-kuttl-tests/glance-db-create-m6rsr" Dec 03 11:49:20 crc kubenswrapper[5052]: I1203 11:49:20.897460 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nshcs\" (UniqueName: \"kubernetes.io/projected/00b2551c-6bbe-444d-abb0-2267676dfb09-kube-api-access-nshcs\") pod \"glance-db-create-m6rsr\" (UID: \"00b2551c-6bbe-444d-abb0-2267676dfb09\") " pod="openstack-kuttl-tests/glance-db-create-m6rsr" Dec 03 11:49:20 crc kubenswrapper[5052]: I1203 11:49:20.897493 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/0071cca3-cb80-44f5-86f6-8abc8399d732-operator-scripts\") pod \"glance-634c-account-create-update-mmdsg\" (UID: \"0071cca3-cb80-44f5-86f6-8abc8399d732\") " pod="openstack-kuttl-tests/glance-634c-account-create-update-mmdsg" Dec 03 11:49:20 crc kubenswrapper[5052]: I1203 11:49:20.916729 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-2868-account-create-update-nbw4x" Dec 03 11:49:20 crc kubenswrapper[5052]: I1203 11:49:20.996299 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-db-create-fj555"] Dec 03 11:49:20 crc kubenswrapper[5052]: I1203 11:49:20.999348 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-nshcs\" (UniqueName: \"kubernetes.io/projected/00b2551c-6bbe-444d-abb0-2267676dfb09-kube-api-access-nshcs\") pod \"glance-db-create-m6rsr\" (UID: \"00b2551c-6bbe-444d-abb0-2267676dfb09\") " pod="openstack-kuttl-tests/glance-db-create-m6rsr" Dec 03 11:49:20 crc kubenswrapper[5052]: I1203 11:49:20.999399 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/0071cca3-cb80-44f5-86f6-8abc8399d732-operator-scripts\") pod \"glance-634c-account-create-update-mmdsg\" (UID: \"0071cca3-cb80-44f5-86f6-8abc8399d732\") " pod="openstack-kuttl-tests/glance-634c-account-create-update-mmdsg" Dec 03 11:49:20 crc kubenswrapper[5052]: I1203 11:49:20.999445 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-zpwp8\" (UniqueName: \"kubernetes.io/projected/0071cca3-cb80-44f5-86f6-8abc8399d732-kube-api-access-zpwp8\") pod \"glance-634c-account-create-update-mmdsg\" (UID: \"0071cca3-cb80-44f5-86f6-8abc8399d732\") " pod="openstack-kuttl-tests/glance-634c-account-create-update-mmdsg" Dec 03 11:49:20 crc kubenswrapper[5052]: I1203 11:49:20.999663 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/00b2551c-6bbe-444d-abb0-2267676dfb09-operator-scripts\") pod \"glance-db-create-m6rsr\" (UID: \"00b2551c-6bbe-444d-abb0-2267676dfb09\") " pod="openstack-kuttl-tests/glance-db-create-m6rsr" Dec 03 11:49:21 crc kubenswrapper[5052]: I1203 11:49:21.002374 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/0071cca3-cb80-44f5-86f6-8abc8399d732-operator-scripts\") pod \"glance-634c-account-create-update-mmdsg\" (UID: \"0071cca3-cb80-44f5-86f6-8abc8399d732\") " pod="openstack-kuttl-tests/glance-634c-account-create-update-mmdsg" Dec 03 11:49:21 crc kubenswrapper[5052]: I1203 11:49:21.002398 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/00b2551c-6bbe-444d-abb0-2267676dfb09-operator-scripts\") pod \"glance-db-create-m6rsr\" (UID: \"00b2551c-6bbe-444d-abb0-2267676dfb09\") " pod="openstack-kuttl-tests/glance-db-create-m6rsr" Dec 03 11:49:21 crc kubenswrapper[5052]: I1203 11:49:21.019087 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-nshcs\" (UniqueName: \"kubernetes.io/projected/00b2551c-6bbe-444d-abb0-2267676dfb09-kube-api-access-nshcs\") pod \"glance-db-create-m6rsr\" (UID: \"00b2551c-6bbe-444d-abb0-2267676dfb09\") " pod="openstack-kuttl-tests/glance-db-create-m6rsr" Dec 03 11:49:21 crc kubenswrapper[5052]: I1203 11:49:21.019685 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-zpwp8\" (UniqueName: \"kubernetes.io/projected/0071cca3-cb80-44f5-86f6-8abc8399d732-kube-api-access-zpwp8\") pod \"glance-634c-account-create-update-mmdsg\" (UID: \"0071cca3-cb80-44f5-86f6-8abc8399d732\") " pod="openstack-kuttl-tests/glance-634c-account-create-update-mmdsg" Dec 03 11:49:21 crc kubenswrapper[5052]: I1203 11:49:21.051087 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-db-create-m6rsr" Dec 03 11:49:21 crc kubenswrapper[5052]: I1203 11:49:21.117846 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-42e8-account-create-update-7bd5x"] Dec 03 11:49:21 crc kubenswrapper[5052]: W1203 11:49:21.122504 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf5994dc8_c6d2_43c7_8035_7e0bfdc09445.slice/crio-8f8ce9b973913e1ea00735b528305afb926208f1c72085166040a0b0b523d193 WatchSource:0}: Error finding container 8f8ce9b973913e1ea00735b528305afb926208f1c72085166040a0b0b523d193: Status 404 returned error can't find the container with id 8f8ce9b973913e1ea00735b528305afb926208f1c72085166040a0b0b523d193 Dec 03 11:49:21 crc kubenswrapper[5052]: I1203 11:49:21.260232 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-634c-account-create-update-mmdsg" Dec 03 11:49:21 crc kubenswrapper[5052]: I1203 11:49:21.313344 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/placement-db-create-8bls7"] Dec 03 11:49:21 crc kubenswrapper[5052]: W1203 11:49:21.327647 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3df6012c_618d_434a_bfdb_56f8f602b05d.slice/crio-46dac6b242f18b7c22beb5db8d354060a3785b07c0db6e6d6a6c4f2c7188a47a WatchSource:0}: Error finding container 46dac6b242f18b7c22beb5db8d354060a3785b07c0db6e6d6a6c4f2c7188a47a: Status 404 returned error can't find the container with id 46dac6b242f18b7c22beb5db8d354060a3785b07c0db6e6d6a6c4f2c7188a47a Dec 03 11:49:21 crc kubenswrapper[5052]: I1203 11:49:21.331936 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 11:49:21 crc kubenswrapper[5052]: I1203 11:49:21.383172 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-db-create-fj555" event={"ID":"f009f642-7d9a-4739-9e45-bf734d0a0eea","Type":"ContainerStarted","Data":"1f0ebc3a3c2e59c7963f57f2c53a6a37ed280f6b4a8ba80dc35ffc947af1383a"} Dec 03 11:49:21 crc kubenswrapper[5052]: I1203 11:49:21.383232 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-db-create-fj555" event={"ID":"f009f642-7d9a-4739-9e45-bf734d0a0eea","Type":"ContainerStarted","Data":"548e9ae9274e5ec1585470e5c08c1b87850aacb4d6805cade35bf74f160c8ece"} Dec 03 11:49:21 crc kubenswrapper[5052]: I1203 11:49:21.389097 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-42e8-account-create-update-7bd5x" event={"ID":"f5994dc8-c6d2-43c7-8035-7e0bfdc09445","Type":"ContainerStarted","Data":"a23cab28141cdcf3f5726567816a96041bb63b99717c6ec1de9a11677f249a09"} Dec 03 11:49:21 crc kubenswrapper[5052]: I1203 11:49:21.389158 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-42e8-account-create-update-7bd5x" event={"ID":"f5994dc8-c6d2-43c7-8035-7e0bfdc09445","Type":"ContainerStarted","Data":"8f8ce9b973913e1ea00735b528305afb926208f1c72085166040a0b0b523d193"} Dec 03 11:49:21 crc kubenswrapper[5052]: I1203 11:49:21.397765 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-db-create-8bls7" event={"ID":"3df6012c-618d-434a-bfdb-56f8f602b05d","Type":"ContainerStarted","Data":"46dac6b242f18b7c22beb5db8d354060a3785b07c0db6e6d6a6c4f2c7188a47a"} Dec 03 11:49:21 crc kubenswrapper[5052]: I1203 11:49:21.434821 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/placement-2868-account-create-update-nbw4x"] Dec 03 11:49:21 crc kubenswrapper[5052]: I1203 11:49:21.442814 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/keystone-42e8-account-create-update-7bd5x" podStartSLOduration=1.442791742 podStartE2EDuration="1.442791742s" podCreationTimestamp="2025-12-03 11:49:20 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:49:21.425586889 +0000 UTC m=+1752.013828653" watchObservedRunningTime="2025-12-03 11:49:21.442791742 +0000 UTC m=+1752.031033506" Dec 03 11:49:21 crc kubenswrapper[5052]: W1203 11:49:21.444539 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod79fd3287_4c54_497c_9081_16fbf13bbb48.slice/crio-aec0da91c4a4f259e563e53d4092e488eb63daa8e0dc97f3fae036c1cd62efc5 WatchSource:0}: Error finding container aec0da91c4a4f259e563e53d4092e488eb63daa8e0dc97f3fae036c1cd62efc5: Status 404 returned error can't find the container with id aec0da91c4a4f259e563e53d4092e488eb63daa8e0dc97f3fae036c1cd62efc5 Dec 03 11:49:21 crc kubenswrapper[5052]: I1203 11:49:21.484291 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 11:49:21 crc kubenswrapper[5052]: I1203 11:49:21.532265 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-db-create-m6rsr"] Dec 03 11:49:21 crc kubenswrapper[5052]: I1203 11:49:21.693388 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 11:49:21 crc kubenswrapper[5052]: I1203 11:49:21.787273 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-634c-account-create-update-mmdsg"] Dec 03 11:49:21 crc kubenswrapper[5052]: W1203 11:49:21.794755 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0071cca3_cb80_44f5_86f6_8abc8399d732.slice/crio-564f149100fa628b63d915a8d30de3e83dc3472a21412ca8dc946a55c4c593c5 WatchSource:0}: Error finding container 564f149100fa628b63d915a8d30de3e83dc3472a21412ca8dc946a55c4c593c5: Status 404 returned error can't find the container with id 564f149100fa628b63d915a8d30de3e83dc3472a21412ca8dc946a55c4c593c5 Dec 03 11:49:21 crc kubenswrapper[5052]: E1203 11:49:21.889344 5052 cadvisor_stats_provider.go:516] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3df6012c_618d_434a_bfdb_56f8f602b05d.slice/crio-conmon-a796623231501a155ae6a6f3667bcb4114e657680ac474de044e4ecdac197168.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3df6012c_618d_434a_bfdb_56f8f602b05d.slice/crio-a796623231501a155ae6a6f3667bcb4114e657680ac474de044e4ecdac197168.scope\": RecentStats: unable to find data in memory cache]" Dec 03 11:49:22 crc kubenswrapper[5052]: I1203 11:49:22.275018 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 11:49:22 crc kubenswrapper[5052]: I1203 11:49:22.413756 5052 generic.go:334] "Generic (PLEG): container finished" podID="f009f642-7d9a-4739-9e45-bf734d0a0eea" containerID="1f0ebc3a3c2e59c7963f57f2c53a6a37ed280f6b4a8ba80dc35ffc947af1383a" exitCode=0 Dec 03 11:49:22 crc kubenswrapper[5052]: I1203 11:49:22.413821 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-db-create-fj555" event={"ID":"f009f642-7d9a-4739-9e45-bf734d0a0eea","Type":"ContainerDied","Data":"1f0ebc3a3c2e59c7963f57f2c53a6a37ed280f6b4a8ba80dc35ffc947af1383a"} Dec 03 11:49:22 crc kubenswrapper[5052]: I1203 11:49:22.415800 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-634c-account-create-update-mmdsg" event={"ID":"0071cca3-cb80-44f5-86f6-8abc8399d732","Type":"ContainerStarted","Data":"783ae349abd4666b2f1a54103fcd03ff8d6cf865a685a1b5c3faed8dae76e348"} Dec 03 11:49:22 crc kubenswrapper[5052]: I1203 11:49:22.415825 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-634c-account-create-update-mmdsg" event={"ID":"0071cca3-cb80-44f5-86f6-8abc8399d732","Type":"ContainerStarted","Data":"564f149100fa628b63d915a8d30de3e83dc3472a21412ca8dc946a55c4c593c5"} Dec 03 11:49:22 crc kubenswrapper[5052]: I1203 11:49:22.417896 5052 generic.go:334] "Generic (PLEG): container finished" podID="f5994dc8-c6d2-43c7-8035-7e0bfdc09445" containerID="a23cab28141cdcf3f5726567816a96041bb63b99717c6ec1de9a11677f249a09" exitCode=0 Dec 03 11:49:22 crc kubenswrapper[5052]: I1203 11:49:22.417939 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-42e8-account-create-update-7bd5x" event={"ID":"f5994dc8-c6d2-43c7-8035-7e0bfdc09445","Type":"ContainerDied","Data":"a23cab28141cdcf3f5726567816a96041bb63b99717c6ec1de9a11677f249a09"} Dec 03 11:49:22 crc kubenswrapper[5052]: I1203 11:49:22.418891 5052 generic.go:334] "Generic (PLEG): container finished" podID="79fd3287-4c54-497c-9081-16fbf13bbb48" containerID="586f73aa377f7546d2b8ba5198112bca3f096ed69864b933f12805e0f366fdea" exitCode=0 Dec 03 11:49:22 crc kubenswrapper[5052]: I1203 11:49:22.418941 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-2868-account-create-update-nbw4x" event={"ID":"79fd3287-4c54-497c-9081-16fbf13bbb48","Type":"ContainerDied","Data":"586f73aa377f7546d2b8ba5198112bca3f096ed69864b933f12805e0f366fdea"} Dec 03 11:49:22 crc kubenswrapper[5052]: I1203 11:49:22.418957 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-2868-account-create-update-nbw4x" event={"ID":"79fd3287-4c54-497c-9081-16fbf13bbb48","Type":"ContainerStarted","Data":"aec0da91c4a4f259e563e53d4092e488eb63daa8e0dc97f3fae036c1cd62efc5"} Dec 03 11:49:22 crc kubenswrapper[5052]: I1203 11:49:22.419967 5052 generic.go:334] "Generic (PLEG): container finished" podID="00b2551c-6bbe-444d-abb0-2267676dfb09" containerID="3075a9e061bf97d3b130a49220cf7a765251499b5d0b7a0bd53a3647974aaaee" exitCode=0 Dec 03 11:49:22 crc kubenswrapper[5052]: I1203 11:49:22.420006 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-db-create-m6rsr" event={"ID":"00b2551c-6bbe-444d-abb0-2267676dfb09","Type":"ContainerDied","Data":"3075a9e061bf97d3b130a49220cf7a765251499b5d0b7a0bd53a3647974aaaee"} Dec 03 11:49:22 crc kubenswrapper[5052]: I1203 11:49:22.420020 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-db-create-m6rsr" event={"ID":"00b2551c-6bbe-444d-abb0-2267676dfb09","Type":"ContainerStarted","Data":"4ffedbb9a4d4d69a937f1e4074b0b5d9ce5041b5b96898a66a4a0bed088b8d9f"} Dec 03 11:49:22 crc kubenswrapper[5052]: I1203 11:49:22.422033 5052 generic.go:334] "Generic (PLEG): container finished" podID="3df6012c-618d-434a-bfdb-56f8f602b05d" containerID="a796623231501a155ae6a6f3667bcb4114e657680ac474de044e4ecdac197168" exitCode=0 Dec 03 11:49:22 crc kubenswrapper[5052]: I1203 11:49:22.422943 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-db-create-8bls7" event={"ID":"3df6012c-618d-434a-bfdb-56f8f602b05d","Type":"ContainerDied","Data":"a796623231501a155ae6a6f3667bcb4114e657680ac474de044e4ecdac197168"} Dec 03 11:49:22 crc kubenswrapper[5052]: I1203 11:49:22.438660 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/glance-634c-account-create-update-mmdsg" podStartSLOduration=2.438625463 podStartE2EDuration="2.438625463s" podCreationTimestamp="2025-12-03 11:49:20 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:49:22.438020765 +0000 UTC m=+1753.026262529" watchObservedRunningTime="2025-12-03 11:49:22.438625463 +0000 UTC m=+1753.026867227" Dec 03 11:49:22 crc kubenswrapper[5052]: I1203 11:49:22.783208 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-db-create-fj555" Dec 03 11:49:22 crc kubenswrapper[5052]: I1203 11:49:22.951246 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p7pzx\" (UniqueName: \"kubernetes.io/projected/f009f642-7d9a-4739-9e45-bf734d0a0eea-kube-api-access-p7pzx\") pod \"f009f642-7d9a-4739-9e45-bf734d0a0eea\" (UID: \"f009f642-7d9a-4739-9e45-bf734d0a0eea\") " Dec 03 11:49:22 crc kubenswrapper[5052]: I1203 11:49:22.951339 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/f009f642-7d9a-4739-9e45-bf734d0a0eea-operator-scripts\") pod \"f009f642-7d9a-4739-9e45-bf734d0a0eea\" (UID: \"f009f642-7d9a-4739-9e45-bf734d0a0eea\") " Dec 03 11:49:22 crc kubenswrapper[5052]: I1203 11:49:22.951949 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f009f642-7d9a-4739-9e45-bf734d0a0eea-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "f009f642-7d9a-4739-9e45-bf734d0a0eea" (UID: "f009f642-7d9a-4739-9e45-bf734d0a0eea"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:49:22 crc kubenswrapper[5052]: I1203 11:49:22.957850 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f009f642-7d9a-4739-9e45-bf734d0a0eea-kube-api-access-p7pzx" (OuterVolumeSpecName: "kube-api-access-p7pzx") pod "f009f642-7d9a-4739-9e45-bf734d0a0eea" (UID: "f009f642-7d9a-4739-9e45-bf734d0a0eea"). InnerVolumeSpecName "kube-api-access-p7pzx". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:49:23 crc kubenswrapper[5052]: I1203 11:49:23.053127 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-p7pzx\" (UniqueName: \"kubernetes.io/projected/f009f642-7d9a-4739-9e45-bf734d0a0eea-kube-api-access-p7pzx\") on node \"crc\" DevicePath \"\"" Dec 03 11:49:23 crc kubenswrapper[5052]: I1203 11:49:23.053165 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/f009f642-7d9a-4739-9e45-bf734d0a0eea-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:49:23 crc kubenswrapper[5052]: I1203 11:49:23.280248 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/swift-storage-0"] Dec 03 11:49:23 crc kubenswrapper[5052]: E1203 11:49:23.280688 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="f009f642-7d9a-4739-9e45-bf734d0a0eea" containerName="mariadb-database-create" Dec 03 11:49:23 crc kubenswrapper[5052]: I1203 11:49:23.280711 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="f009f642-7d9a-4739-9e45-bf734d0a0eea" containerName="mariadb-database-create" Dec 03 11:49:23 crc kubenswrapper[5052]: I1203 11:49:23.280898 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="f009f642-7d9a-4739-9e45-bf734d0a0eea" containerName="mariadb-database-create" Dec 03 11:49:23 crc kubenswrapper[5052]: I1203 11:49:23.286209 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-storage-0" Dec 03 11:49:23 crc kubenswrapper[5052]: I1203 11:49:23.292014 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"swift-conf" Dec 03 11:49:23 crc kubenswrapper[5052]: I1203 11:49:23.292201 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"swift-storage-config-data" Dec 03 11:49:23 crc kubenswrapper[5052]: I1203 11:49:23.292276 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"swift-swift-dockercfg-6cgv9" Dec 03 11:49:23 crc kubenswrapper[5052]: I1203 11:49:23.292488 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"swift-ring-files" Dec 03 11:49:23 crc kubenswrapper[5052]: I1203 11:49:23.304707 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/swift-storage-0"] Dec 03 11:49:23 crc kubenswrapper[5052]: I1203 11:49:23.433395 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-db-create-fj555" event={"ID":"f009f642-7d9a-4739-9e45-bf734d0a0eea","Type":"ContainerDied","Data":"548e9ae9274e5ec1585470e5c08c1b87850aacb4d6805cade35bf74f160c8ece"} Dec 03 11:49:23 crc kubenswrapper[5052]: I1203 11:49:23.433458 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="548e9ae9274e5ec1585470e5c08c1b87850aacb4d6805cade35bf74f160c8ece" Dec 03 11:49:23 crc kubenswrapper[5052]: I1203 11:49:23.433484 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-db-create-fj555" Dec 03 11:49:23 crc kubenswrapper[5052]: I1203 11:49:23.435372 5052 generic.go:334] "Generic (PLEG): container finished" podID="0071cca3-cb80-44f5-86f6-8abc8399d732" containerID="783ae349abd4666b2f1a54103fcd03ff8d6cf865a685a1b5c3faed8dae76e348" exitCode=0 Dec 03 11:49:23 crc kubenswrapper[5052]: I1203 11:49:23.435451 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-634c-account-create-update-mmdsg" event={"ID":"0071cca3-cb80-44f5-86f6-8abc8399d732","Type":"ContainerDied","Data":"783ae349abd4666b2f1a54103fcd03ff8d6cf865a685a1b5c3faed8dae76e348"} Dec 03 11:49:23 crc kubenswrapper[5052]: I1203 11:49:23.459625 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cache\" (UniqueName: \"kubernetes.io/empty-dir/337bf4ce-ed6a-47bb-820d-6d64cd15e16e-cache\") pod \"swift-storage-0\" (UID: \"337bf4ce-ed6a-47bb-820d-6d64cd15e16e\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 11:49:23 crc kubenswrapper[5052]: I1203 11:49:23.459708 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage06-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage06-crc\") pod \"swift-storage-0\" (UID: \"337bf4ce-ed6a-47bb-820d-6d64cd15e16e\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 11:49:23 crc kubenswrapper[5052]: I1203 11:49:23.459732 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/337bf4ce-ed6a-47bb-820d-6d64cd15e16e-etc-swift\") pod \"swift-storage-0\" (UID: \"337bf4ce-ed6a-47bb-820d-6d64cd15e16e\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 11:49:23 crc kubenswrapper[5052]: I1203 11:49:23.459773 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gwmsq\" (UniqueName: \"kubernetes.io/projected/337bf4ce-ed6a-47bb-820d-6d64cd15e16e-kube-api-access-gwmsq\") pod \"swift-storage-0\" (UID: \"337bf4ce-ed6a-47bb-820d-6d64cd15e16e\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 11:49:23 crc kubenswrapper[5052]: I1203 11:49:23.459849 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lock\" (UniqueName: \"kubernetes.io/empty-dir/337bf4ce-ed6a-47bb-820d-6d64cd15e16e-lock\") pod \"swift-storage-0\" (UID: \"337bf4ce-ed6a-47bb-820d-6d64cd15e16e\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 11:49:23 crc kubenswrapper[5052]: I1203 11:49:23.561007 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage06-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage06-crc\") pod \"swift-storage-0\" (UID: \"337bf4ce-ed6a-47bb-820d-6d64cd15e16e\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 11:49:23 crc kubenswrapper[5052]: I1203 11:49:23.561048 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/337bf4ce-ed6a-47bb-820d-6d64cd15e16e-etc-swift\") pod \"swift-storage-0\" (UID: \"337bf4ce-ed6a-47bb-820d-6d64cd15e16e\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 11:49:23 crc kubenswrapper[5052]: I1203 11:49:23.561099 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-gwmsq\" (UniqueName: \"kubernetes.io/projected/337bf4ce-ed6a-47bb-820d-6d64cd15e16e-kube-api-access-gwmsq\") pod \"swift-storage-0\" (UID: \"337bf4ce-ed6a-47bb-820d-6d64cd15e16e\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 11:49:23 crc kubenswrapper[5052]: I1203 11:49:23.561154 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"lock\" (UniqueName: \"kubernetes.io/empty-dir/337bf4ce-ed6a-47bb-820d-6d64cd15e16e-lock\") pod \"swift-storage-0\" (UID: \"337bf4ce-ed6a-47bb-820d-6d64cd15e16e\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 11:49:23 crc kubenswrapper[5052]: I1203 11:49:23.561199 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"cache\" (UniqueName: \"kubernetes.io/empty-dir/337bf4ce-ed6a-47bb-820d-6d64cd15e16e-cache\") pod \"swift-storage-0\" (UID: \"337bf4ce-ed6a-47bb-820d-6d64cd15e16e\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 11:49:23 crc kubenswrapper[5052]: I1203 11:49:23.561668 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"cache\" (UniqueName: \"kubernetes.io/empty-dir/337bf4ce-ed6a-47bb-820d-6d64cd15e16e-cache\") pod \"swift-storage-0\" (UID: \"337bf4ce-ed6a-47bb-820d-6d64cd15e16e\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 11:49:23 crc kubenswrapper[5052]: E1203 11:49:23.561821 5052 projected.go:288] Couldn't get configMap openstack-kuttl-tests/swift-ring-files: configmap "swift-ring-files" not found Dec 03 11:49:23 crc kubenswrapper[5052]: E1203 11:49:23.561864 5052 projected.go:194] Error preparing data for projected volume etc-swift for pod openstack-kuttl-tests/swift-storage-0: configmap "swift-ring-files" not found Dec 03 11:49:23 crc kubenswrapper[5052]: I1203 11:49:23.561886 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"lock\" (UniqueName: \"kubernetes.io/empty-dir/337bf4ce-ed6a-47bb-820d-6d64cd15e16e-lock\") pod \"swift-storage-0\" (UID: \"337bf4ce-ed6a-47bb-820d-6d64cd15e16e\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 11:49:23 crc kubenswrapper[5052]: E1203 11:49:23.561921 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/337bf4ce-ed6a-47bb-820d-6d64cd15e16e-etc-swift podName:337bf4ce-ed6a-47bb-820d-6d64cd15e16e nodeName:}" failed. No retries permitted until 2025-12-03 11:49:24.061904798 +0000 UTC m=+1754.650146562 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "etc-swift" (UniqueName: "kubernetes.io/projected/337bf4ce-ed6a-47bb-820d-6d64cd15e16e-etc-swift") pod "swift-storage-0" (UID: "337bf4ce-ed6a-47bb-820d-6d64cd15e16e") : configmap "swift-ring-files" not found Dec 03 11:49:23 crc kubenswrapper[5052]: I1203 11:49:23.562118 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage06-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage06-crc\") pod \"swift-storage-0\" (UID: \"337bf4ce-ed6a-47bb-820d-6d64cd15e16e\") device mount path \"/mnt/openstack/pv06\"" pod="openstack-kuttl-tests/swift-storage-0" Dec 03 11:49:23 crc kubenswrapper[5052]: I1203 11:49:23.579769 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-gwmsq\" (UniqueName: \"kubernetes.io/projected/337bf4ce-ed6a-47bb-820d-6d64cd15e16e-kube-api-access-gwmsq\") pod \"swift-storage-0\" (UID: \"337bf4ce-ed6a-47bb-820d-6d64cd15e16e\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 11:49:23 crc kubenswrapper[5052]: I1203 11:49:23.581148 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage06-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage06-crc\") pod \"swift-storage-0\" (UID: \"337bf4ce-ed6a-47bb-820d-6d64cd15e16e\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 11:49:23 crc kubenswrapper[5052]: I1203 11:49:23.692099 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 11:49:23 crc kubenswrapper[5052]: I1203 11:49:23.868544 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-42e8-account-create-update-7bd5x" Dec 03 11:49:23 crc kubenswrapper[5052]: I1203 11:49:23.968245 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wj55b\" (UniqueName: \"kubernetes.io/projected/f5994dc8-c6d2-43c7-8035-7e0bfdc09445-kube-api-access-wj55b\") pod \"f5994dc8-c6d2-43c7-8035-7e0bfdc09445\" (UID: \"f5994dc8-c6d2-43c7-8035-7e0bfdc09445\") " Dec 03 11:49:23 crc kubenswrapper[5052]: I1203 11:49:23.968588 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/f5994dc8-c6d2-43c7-8035-7e0bfdc09445-operator-scripts\") pod \"f5994dc8-c6d2-43c7-8035-7e0bfdc09445\" (UID: \"f5994dc8-c6d2-43c7-8035-7e0bfdc09445\") " Dec 03 11:49:23 crc kubenswrapper[5052]: I1203 11:49:23.969356 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f5994dc8-c6d2-43c7-8035-7e0bfdc09445-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "f5994dc8-c6d2-43c7-8035-7e0bfdc09445" (UID: "f5994dc8-c6d2-43c7-8035-7e0bfdc09445"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:49:23 crc kubenswrapper[5052]: I1203 11:49:23.971576 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/f5994dc8-c6d2-43c7-8035-7e0bfdc09445-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:49:23 crc kubenswrapper[5052]: I1203 11:49:23.972693 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f5994dc8-c6d2-43c7-8035-7e0bfdc09445-kube-api-access-wj55b" (OuterVolumeSpecName: "kube-api-access-wj55b") pod "f5994dc8-c6d2-43c7-8035-7e0bfdc09445" (UID: "f5994dc8-c6d2-43c7-8035-7e0bfdc09445"). InnerVolumeSpecName "kube-api-access-wj55b". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:49:24 crc kubenswrapper[5052]: I1203 11:49:24.025376 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-db-create-8bls7" Dec 03 11:49:24 crc kubenswrapper[5052]: I1203 11:49:24.043957 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-db-create-m6rsr" Dec 03 11:49:24 crc kubenswrapper[5052]: I1203 11:49:24.076588 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/337bf4ce-ed6a-47bb-820d-6d64cd15e16e-etc-swift\") pod \"swift-storage-0\" (UID: \"337bf4ce-ed6a-47bb-820d-6d64cd15e16e\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 11:49:24 crc kubenswrapper[5052]: I1203 11:49:24.076807 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-wj55b\" (UniqueName: \"kubernetes.io/projected/f5994dc8-c6d2-43c7-8035-7e0bfdc09445-kube-api-access-wj55b\") on node \"crc\" DevicePath \"\"" Dec 03 11:49:24 crc kubenswrapper[5052]: E1203 11:49:24.076947 5052 projected.go:288] Couldn't get configMap openstack-kuttl-tests/swift-ring-files: configmap "swift-ring-files" not found Dec 03 11:49:24 crc kubenswrapper[5052]: E1203 11:49:24.076964 5052 projected.go:194] Error preparing data for projected volume etc-swift for pod openstack-kuttl-tests/swift-storage-0: configmap "swift-ring-files" not found Dec 03 11:49:24 crc kubenswrapper[5052]: E1203 11:49:24.077017 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/337bf4ce-ed6a-47bb-820d-6d64cd15e16e-etc-swift podName:337bf4ce-ed6a-47bb-820d-6d64cd15e16e nodeName:}" failed. No retries permitted until 2025-12-03 11:49:25.076999441 +0000 UTC m=+1755.665241205 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "etc-swift" (UniqueName: "kubernetes.io/projected/337bf4ce-ed6a-47bb-820d-6d64cd15e16e-etc-swift") pod "swift-storage-0" (UID: "337bf4ce-ed6a-47bb-820d-6d64cd15e16e") : configmap "swift-ring-files" not found Dec 03 11:49:24 crc kubenswrapper[5052]: I1203 11:49:24.085262 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-2868-account-create-update-nbw4x" Dec 03 11:49:24 crc kubenswrapper[5052]: I1203 11:49:24.178005 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/3df6012c-618d-434a-bfdb-56f8f602b05d-operator-scripts\") pod \"3df6012c-618d-434a-bfdb-56f8f602b05d\" (UID: \"3df6012c-618d-434a-bfdb-56f8f602b05d\") " Dec 03 11:49:24 crc kubenswrapper[5052]: I1203 11:49:24.178378 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/00b2551c-6bbe-444d-abb0-2267676dfb09-operator-scripts\") pod \"00b2551c-6bbe-444d-abb0-2267676dfb09\" (UID: \"00b2551c-6bbe-444d-abb0-2267676dfb09\") " Dec 03 11:49:24 crc kubenswrapper[5052]: I1203 11:49:24.178443 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/79fd3287-4c54-497c-9081-16fbf13bbb48-operator-scripts\") pod \"79fd3287-4c54-497c-9081-16fbf13bbb48\" (UID: \"79fd3287-4c54-497c-9081-16fbf13bbb48\") " Dec 03 11:49:24 crc kubenswrapper[5052]: I1203 11:49:24.178473 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rm6jg\" (UniqueName: \"kubernetes.io/projected/3df6012c-618d-434a-bfdb-56f8f602b05d-kube-api-access-rm6jg\") pod \"3df6012c-618d-434a-bfdb-56f8f602b05d\" (UID: \"3df6012c-618d-434a-bfdb-56f8f602b05d\") " Dec 03 11:49:24 crc kubenswrapper[5052]: I1203 11:49:24.178534 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sz9gk\" (UniqueName: \"kubernetes.io/projected/79fd3287-4c54-497c-9081-16fbf13bbb48-kube-api-access-sz9gk\") pod \"79fd3287-4c54-497c-9081-16fbf13bbb48\" (UID: \"79fd3287-4c54-497c-9081-16fbf13bbb48\") " Dec 03 11:49:24 crc kubenswrapper[5052]: I1203 11:49:24.178575 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nshcs\" (UniqueName: \"kubernetes.io/projected/00b2551c-6bbe-444d-abb0-2267676dfb09-kube-api-access-nshcs\") pod \"00b2551c-6bbe-444d-abb0-2267676dfb09\" (UID: \"00b2551c-6bbe-444d-abb0-2267676dfb09\") " Dec 03 11:49:24 crc kubenswrapper[5052]: I1203 11:49:24.178918 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3df6012c-618d-434a-bfdb-56f8f602b05d-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "3df6012c-618d-434a-bfdb-56f8f602b05d" (UID: "3df6012c-618d-434a-bfdb-56f8f602b05d"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:49:24 crc kubenswrapper[5052]: I1203 11:49:24.179118 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/00b2551c-6bbe-444d-abb0-2267676dfb09-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "00b2551c-6bbe-444d-abb0-2267676dfb09" (UID: "00b2551c-6bbe-444d-abb0-2267676dfb09"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:49:24 crc kubenswrapper[5052]: I1203 11:49:24.179363 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/79fd3287-4c54-497c-9081-16fbf13bbb48-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "79fd3287-4c54-497c-9081-16fbf13bbb48" (UID: "79fd3287-4c54-497c-9081-16fbf13bbb48"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:49:24 crc kubenswrapper[5052]: I1203 11:49:24.181854 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/00b2551c-6bbe-444d-abb0-2267676dfb09-kube-api-access-nshcs" (OuterVolumeSpecName: "kube-api-access-nshcs") pod "00b2551c-6bbe-444d-abb0-2267676dfb09" (UID: "00b2551c-6bbe-444d-abb0-2267676dfb09"). InnerVolumeSpecName "kube-api-access-nshcs". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:49:24 crc kubenswrapper[5052]: I1203 11:49:24.182812 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3df6012c-618d-434a-bfdb-56f8f602b05d-kube-api-access-rm6jg" (OuterVolumeSpecName: "kube-api-access-rm6jg") pod "3df6012c-618d-434a-bfdb-56f8f602b05d" (UID: "3df6012c-618d-434a-bfdb-56f8f602b05d"). InnerVolumeSpecName "kube-api-access-rm6jg". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:49:24 crc kubenswrapper[5052]: I1203 11:49:24.183332 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/79fd3287-4c54-497c-9081-16fbf13bbb48-kube-api-access-sz9gk" (OuterVolumeSpecName: "kube-api-access-sz9gk") pod "79fd3287-4c54-497c-9081-16fbf13bbb48" (UID: "79fd3287-4c54-497c-9081-16fbf13bbb48"). InnerVolumeSpecName "kube-api-access-sz9gk". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:49:24 crc kubenswrapper[5052]: I1203 11:49:24.280675 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/00b2551c-6bbe-444d-abb0-2267676dfb09-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:49:24 crc kubenswrapper[5052]: I1203 11:49:24.280712 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/79fd3287-4c54-497c-9081-16fbf13bbb48-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:49:24 crc kubenswrapper[5052]: I1203 11:49:24.280727 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-rm6jg\" (UniqueName: \"kubernetes.io/projected/3df6012c-618d-434a-bfdb-56f8f602b05d-kube-api-access-rm6jg\") on node \"crc\" DevicePath \"\"" Dec 03 11:49:24 crc kubenswrapper[5052]: I1203 11:49:24.280742 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-sz9gk\" (UniqueName: \"kubernetes.io/projected/79fd3287-4c54-497c-9081-16fbf13bbb48-kube-api-access-sz9gk\") on node \"crc\" DevicePath \"\"" Dec 03 11:49:24 crc kubenswrapper[5052]: I1203 11:49:24.280755 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-nshcs\" (UniqueName: \"kubernetes.io/projected/00b2551c-6bbe-444d-abb0-2267676dfb09-kube-api-access-nshcs\") on node \"crc\" DevicePath \"\"" Dec 03 11:49:24 crc kubenswrapper[5052]: I1203 11:49:24.280767 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/3df6012c-618d-434a-bfdb-56f8f602b05d-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:49:24 crc kubenswrapper[5052]: I1203 11:49:24.447557 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-42e8-account-create-update-7bd5x" Dec 03 11:49:24 crc kubenswrapper[5052]: I1203 11:49:24.448825 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-42e8-account-create-update-7bd5x" event={"ID":"f5994dc8-c6d2-43c7-8035-7e0bfdc09445","Type":"ContainerDied","Data":"8f8ce9b973913e1ea00735b528305afb926208f1c72085166040a0b0b523d193"} Dec 03 11:49:24 crc kubenswrapper[5052]: I1203 11:49:24.448881 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8f8ce9b973913e1ea00735b528305afb926208f1c72085166040a0b0b523d193" Dec 03 11:49:24 crc kubenswrapper[5052]: I1203 11:49:24.454807 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-2868-account-create-update-nbw4x" event={"ID":"79fd3287-4c54-497c-9081-16fbf13bbb48","Type":"ContainerDied","Data":"aec0da91c4a4f259e563e53d4092e488eb63daa8e0dc97f3fae036c1cd62efc5"} Dec 03 11:49:24 crc kubenswrapper[5052]: I1203 11:49:24.454861 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="aec0da91c4a4f259e563e53d4092e488eb63daa8e0dc97f3fae036c1cd62efc5" Dec 03 11:49:24 crc kubenswrapper[5052]: I1203 11:49:24.454864 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-2868-account-create-update-nbw4x" Dec 03 11:49:24 crc kubenswrapper[5052]: I1203 11:49:24.456814 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-db-create-m6rsr" event={"ID":"00b2551c-6bbe-444d-abb0-2267676dfb09","Type":"ContainerDied","Data":"4ffedbb9a4d4d69a937f1e4074b0b5d9ce5041b5b96898a66a4a0bed088b8d9f"} Dec 03 11:49:24 crc kubenswrapper[5052]: I1203 11:49:24.456849 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4ffedbb9a4d4d69a937f1e4074b0b5d9ce5041b5b96898a66a4a0bed088b8d9f" Dec 03 11:49:24 crc kubenswrapper[5052]: I1203 11:49:24.456826 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-db-create-m6rsr" Dec 03 11:49:24 crc kubenswrapper[5052]: I1203 11:49:24.458366 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-db-create-8bls7" Dec 03 11:49:24 crc kubenswrapper[5052]: I1203 11:49:24.458375 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-db-create-8bls7" event={"ID":"3df6012c-618d-434a-bfdb-56f8f602b05d","Type":"ContainerDied","Data":"46dac6b242f18b7c22beb5db8d354060a3785b07c0db6e6d6a6c4f2c7188a47a"} Dec 03 11:49:24 crc kubenswrapper[5052]: I1203 11:49:24.458497 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="46dac6b242f18b7c22beb5db8d354060a3785b07c0db6e6d6a6c4f2c7188a47a" Dec 03 11:49:24 crc kubenswrapper[5052]: I1203 11:49:24.673101 5052 scope.go:117] "RemoveContainer" containerID="25fe88f8d102f09983f8884e65f0c6c1575ade67921f10a630e6dcc3ea8b506e" Dec 03 11:49:24 crc kubenswrapper[5052]: E1203 11:49:24.673446 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 11:49:24 crc kubenswrapper[5052]: I1203 11:49:24.741313 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 11:49:24 crc kubenswrapper[5052]: I1203 11:49:24.789234 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 11:49:24 crc kubenswrapper[5052]: I1203 11:49:24.859297 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-634c-account-create-update-mmdsg" Dec 03 11:49:24 crc kubenswrapper[5052]: I1203 11:49:24.958478 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ovn-northd-0"] Dec 03 11:49:24 crc kubenswrapper[5052]: E1203 11:49:24.961881 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="f5994dc8-c6d2-43c7-8035-7e0bfdc09445" containerName="mariadb-account-create-update" Dec 03 11:49:24 crc kubenswrapper[5052]: I1203 11:49:24.961904 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="f5994dc8-c6d2-43c7-8035-7e0bfdc09445" containerName="mariadb-account-create-update" Dec 03 11:49:24 crc kubenswrapper[5052]: E1203 11:49:24.961923 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="0071cca3-cb80-44f5-86f6-8abc8399d732" containerName="mariadb-account-create-update" Dec 03 11:49:24 crc kubenswrapper[5052]: I1203 11:49:24.961929 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="0071cca3-cb80-44f5-86f6-8abc8399d732" containerName="mariadb-account-create-update" Dec 03 11:49:24 crc kubenswrapper[5052]: E1203 11:49:24.961939 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="00b2551c-6bbe-444d-abb0-2267676dfb09" containerName="mariadb-database-create" Dec 03 11:49:24 crc kubenswrapper[5052]: I1203 11:49:24.961946 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="00b2551c-6bbe-444d-abb0-2267676dfb09" containerName="mariadb-database-create" Dec 03 11:49:24 crc kubenswrapper[5052]: E1203 11:49:24.961957 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="79fd3287-4c54-497c-9081-16fbf13bbb48" containerName="mariadb-account-create-update" Dec 03 11:49:24 crc kubenswrapper[5052]: I1203 11:49:24.961964 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="79fd3287-4c54-497c-9081-16fbf13bbb48" containerName="mariadb-account-create-update" Dec 03 11:49:24 crc kubenswrapper[5052]: E1203 11:49:24.961982 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="3df6012c-618d-434a-bfdb-56f8f602b05d" containerName="mariadb-database-create" Dec 03 11:49:24 crc kubenswrapper[5052]: I1203 11:49:24.961988 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="3df6012c-618d-434a-bfdb-56f8f602b05d" containerName="mariadb-database-create" Dec 03 11:49:24 crc kubenswrapper[5052]: I1203 11:49:24.962158 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="79fd3287-4c54-497c-9081-16fbf13bbb48" containerName="mariadb-account-create-update" Dec 03 11:49:24 crc kubenswrapper[5052]: I1203 11:49:24.962179 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="0071cca3-cb80-44f5-86f6-8abc8399d732" containerName="mariadb-account-create-update" Dec 03 11:49:24 crc kubenswrapper[5052]: I1203 11:49:24.962189 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="3df6012c-618d-434a-bfdb-56f8f602b05d" containerName="mariadb-database-create" Dec 03 11:49:24 crc kubenswrapper[5052]: I1203 11:49:24.962203 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="00b2551c-6bbe-444d-abb0-2267676dfb09" containerName="mariadb-database-create" Dec 03 11:49:24 crc kubenswrapper[5052]: I1203 11:49:24.962216 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="f5994dc8-c6d2-43c7-8035-7e0bfdc09445" containerName="mariadb-account-create-update" Dec 03 11:49:24 crc kubenswrapper[5052]: I1203 11:49:24.963188 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 11:49:24 crc kubenswrapper[5052]: I1203 11:49:24.974494 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"ovnnorthd-scripts" Dec 03 11:49:24 crc kubenswrapper[5052]: I1203 11:49:24.974719 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-ovnnorthd-ovndbs" Dec 03 11:49:24 crc kubenswrapper[5052]: I1203 11:49:24.974870 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ovnnorthd-ovnnorthd-dockercfg-v8chk" Dec 03 11:49:24 crc kubenswrapper[5052]: I1203 11:49:24.975026 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"ovnnorthd-config" Dec 03 11:49:24 crc kubenswrapper[5052]: I1203 11:49:24.979937 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ovn-northd-0"] Dec 03 11:49:24 crc kubenswrapper[5052]: I1203 11:49:24.992048 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zpwp8\" (UniqueName: \"kubernetes.io/projected/0071cca3-cb80-44f5-86f6-8abc8399d732-kube-api-access-zpwp8\") pod \"0071cca3-cb80-44f5-86f6-8abc8399d732\" (UID: \"0071cca3-cb80-44f5-86f6-8abc8399d732\") " Dec 03 11:49:24 crc kubenswrapper[5052]: I1203 11:49:24.992251 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/0071cca3-cb80-44f5-86f6-8abc8399d732-operator-scripts\") pod \"0071cca3-cb80-44f5-86f6-8abc8399d732\" (UID: \"0071cca3-cb80-44f5-86f6-8abc8399d732\") " Dec 03 11:49:24 crc kubenswrapper[5052]: I1203 11:49:24.993260 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0071cca3-cb80-44f5-86f6-8abc8399d732-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "0071cca3-cb80-44f5-86f6-8abc8399d732" (UID: "0071cca3-cb80-44f5-86f6-8abc8399d732"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:49:25 crc kubenswrapper[5052]: I1203 11:49:25.010883 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0071cca3-cb80-44f5-86f6-8abc8399d732-kube-api-access-zpwp8" (OuterVolumeSpecName: "kube-api-access-zpwp8") pod "0071cca3-cb80-44f5-86f6-8abc8399d732" (UID: "0071cca3-cb80-44f5-86f6-8abc8399d732"). InnerVolumeSpecName "kube-api-access-zpwp8". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:49:25 crc kubenswrapper[5052]: I1203 11:49:25.094649 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/9d8cd78f-2670-4933-81dd-01cdc0044454-config\") pod \"ovn-northd-0\" (UID: \"9d8cd78f-2670-4933-81dd-01cdc0044454\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 11:49:25 crc kubenswrapper[5052]: I1203 11:49:25.094717 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-rundir\" (UniqueName: \"kubernetes.io/empty-dir/9d8cd78f-2670-4933-81dd-01cdc0044454-ovn-rundir\") pod \"ovn-northd-0\" (UID: \"9d8cd78f-2670-4933-81dd-01cdc0044454\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 11:49:25 crc kubenswrapper[5052]: I1203 11:49:25.094762 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/337bf4ce-ed6a-47bb-820d-6d64cd15e16e-etc-swift\") pod \"swift-storage-0\" (UID: \"337bf4ce-ed6a-47bb-820d-6d64cd15e16e\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 11:49:25 crc kubenswrapper[5052]: I1203 11:49:25.094880 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-northd-tls-certs\" (UniqueName: \"kubernetes.io/secret/9d8cd78f-2670-4933-81dd-01cdc0044454-ovn-northd-tls-certs\") pod \"ovn-northd-0\" (UID: \"9d8cd78f-2670-4933-81dd-01cdc0044454\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 11:49:25 crc kubenswrapper[5052]: E1203 11:49:25.094908 5052 projected.go:288] Couldn't get configMap openstack-kuttl-tests/swift-ring-files: configmap "swift-ring-files" not found Dec 03 11:49:25 crc kubenswrapper[5052]: E1203 11:49:25.094928 5052 projected.go:194] Error preparing data for projected volume etc-swift for pod openstack-kuttl-tests/swift-storage-0: configmap "swift-ring-files" not found Dec 03 11:49:25 crc kubenswrapper[5052]: I1203 11:49:25.094944 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/9d8cd78f-2670-4933-81dd-01cdc0044454-metrics-certs-tls-certs\") pod \"ovn-northd-0\" (UID: \"9d8cd78f-2670-4933-81dd-01cdc0044454\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 11:49:25 crc kubenswrapper[5052]: E1203 11:49:25.094985 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/337bf4ce-ed6a-47bb-820d-6d64cd15e16e-etc-swift podName:337bf4ce-ed6a-47bb-820d-6d64cd15e16e nodeName:}" failed. No retries permitted until 2025-12-03 11:49:27.094966836 +0000 UTC m=+1757.683208760 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "etc-swift" (UniqueName: "kubernetes.io/projected/337bf4ce-ed6a-47bb-820d-6d64cd15e16e-etc-swift") pod "swift-storage-0" (UID: "337bf4ce-ed6a-47bb-820d-6d64cd15e16e") : configmap "swift-ring-files" not found Dec 03 11:49:25 crc kubenswrapper[5052]: I1203 11:49:25.095001 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/9d8cd78f-2670-4933-81dd-01cdc0044454-scripts\") pod \"ovn-northd-0\" (UID: \"9d8cd78f-2670-4933-81dd-01cdc0044454\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 11:49:25 crc kubenswrapper[5052]: I1203 11:49:25.095024 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sw2xt\" (UniqueName: \"kubernetes.io/projected/9d8cd78f-2670-4933-81dd-01cdc0044454-kube-api-access-sw2xt\") pod \"ovn-northd-0\" (UID: \"9d8cd78f-2670-4933-81dd-01cdc0044454\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 11:49:25 crc kubenswrapper[5052]: I1203 11:49:25.095059 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9d8cd78f-2670-4933-81dd-01cdc0044454-combined-ca-bundle\") pod \"ovn-northd-0\" (UID: \"9d8cd78f-2670-4933-81dd-01cdc0044454\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 11:49:25 crc kubenswrapper[5052]: I1203 11:49:25.095186 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/0071cca3-cb80-44f5-86f6-8abc8399d732-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:49:25 crc kubenswrapper[5052]: I1203 11:49:25.095202 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-zpwp8\" (UniqueName: \"kubernetes.io/projected/0071cca3-cb80-44f5-86f6-8abc8399d732-kube-api-access-zpwp8\") on node \"crc\" DevicePath \"\"" Dec 03 11:49:25 crc kubenswrapper[5052]: I1203 11:49:25.196931 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovn-northd-tls-certs\" (UniqueName: \"kubernetes.io/secret/9d8cd78f-2670-4933-81dd-01cdc0044454-ovn-northd-tls-certs\") pod \"ovn-northd-0\" (UID: \"9d8cd78f-2670-4933-81dd-01cdc0044454\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 11:49:25 crc kubenswrapper[5052]: I1203 11:49:25.196987 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/9d8cd78f-2670-4933-81dd-01cdc0044454-metrics-certs-tls-certs\") pod \"ovn-northd-0\" (UID: \"9d8cd78f-2670-4933-81dd-01cdc0044454\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 11:49:25 crc kubenswrapper[5052]: I1203 11:49:25.197021 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/9d8cd78f-2670-4933-81dd-01cdc0044454-scripts\") pod \"ovn-northd-0\" (UID: \"9d8cd78f-2670-4933-81dd-01cdc0044454\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 11:49:25 crc kubenswrapper[5052]: I1203 11:49:25.197044 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-sw2xt\" (UniqueName: \"kubernetes.io/projected/9d8cd78f-2670-4933-81dd-01cdc0044454-kube-api-access-sw2xt\") pod \"ovn-northd-0\" (UID: \"9d8cd78f-2670-4933-81dd-01cdc0044454\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 11:49:25 crc kubenswrapper[5052]: I1203 11:49:25.197072 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9d8cd78f-2670-4933-81dd-01cdc0044454-combined-ca-bundle\") pod \"ovn-northd-0\" (UID: \"9d8cd78f-2670-4933-81dd-01cdc0044454\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 11:49:25 crc kubenswrapper[5052]: I1203 11:49:25.197170 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/9d8cd78f-2670-4933-81dd-01cdc0044454-config\") pod \"ovn-northd-0\" (UID: \"9d8cd78f-2670-4933-81dd-01cdc0044454\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 11:49:25 crc kubenswrapper[5052]: I1203 11:49:25.197214 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovn-rundir\" (UniqueName: \"kubernetes.io/empty-dir/9d8cd78f-2670-4933-81dd-01cdc0044454-ovn-rundir\") pod \"ovn-northd-0\" (UID: \"9d8cd78f-2670-4933-81dd-01cdc0044454\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 11:49:25 crc kubenswrapper[5052]: I1203 11:49:25.197786 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovn-rundir\" (UniqueName: \"kubernetes.io/empty-dir/9d8cd78f-2670-4933-81dd-01cdc0044454-ovn-rundir\") pod \"ovn-northd-0\" (UID: \"9d8cd78f-2670-4933-81dd-01cdc0044454\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 11:49:25 crc kubenswrapper[5052]: I1203 11:49:25.198769 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/9d8cd78f-2670-4933-81dd-01cdc0044454-scripts\") pod \"ovn-northd-0\" (UID: \"9d8cd78f-2670-4933-81dd-01cdc0044454\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 11:49:25 crc kubenswrapper[5052]: I1203 11:49:25.199397 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/9d8cd78f-2670-4933-81dd-01cdc0044454-config\") pod \"ovn-northd-0\" (UID: \"9d8cd78f-2670-4933-81dd-01cdc0044454\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 11:49:25 crc kubenswrapper[5052]: I1203 11:49:25.201436 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/9d8cd78f-2670-4933-81dd-01cdc0044454-metrics-certs-tls-certs\") pod \"ovn-northd-0\" (UID: \"9d8cd78f-2670-4933-81dd-01cdc0044454\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 11:49:25 crc kubenswrapper[5052]: I1203 11:49:25.201850 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovn-northd-tls-certs\" (UniqueName: \"kubernetes.io/secret/9d8cd78f-2670-4933-81dd-01cdc0044454-ovn-northd-tls-certs\") pod \"ovn-northd-0\" (UID: \"9d8cd78f-2670-4933-81dd-01cdc0044454\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 11:49:25 crc kubenswrapper[5052]: I1203 11:49:25.202183 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9d8cd78f-2670-4933-81dd-01cdc0044454-combined-ca-bundle\") pod \"ovn-northd-0\" (UID: \"9d8cd78f-2670-4933-81dd-01cdc0044454\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 11:49:25 crc kubenswrapper[5052]: I1203 11:49:25.235686 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-sw2xt\" (UniqueName: \"kubernetes.io/projected/9d8cd78f-2670-4933-81dd-01cdc0044454-kube-api-access-sw2xt\") pod \"ovn-northd-0\" (UID: \"9d8cd78f-2670-4933-81dd-01cdc0044454\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 11:49:25 crc kubenswrapper[5052]: I1203 11:49:25.365690 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 11:49:25 crc kubenswrapper[5052]: I1203 11:49:25.471664 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-634c-account-create-update-mmdsg" Dec 03 11:49:25 crc kubenswrapper[5052]: I1203 11:49:25.472301 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-634c-account-create-update-mmdsg" event={"ID":"0071cca3-cb80-44f5-86f6-8abc8399d732","Type":"ContainerDied","Data":"564f149100fa628b63d915a8d30de3e83dc3472a21412ca8dc946a55c4c593c5"} Dec 03 11:49:25 crc kubenswrapper[5052]: I1203 11:49:25.472342 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="564f149100fa628b63d915a8d30de3e83dc3472a21412ca8dc946a55c4c593c5" Dec 03 11:49:25 crc kubenswrapper[5052]: I1203 11:49:25.826588 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ovn-northd-0"] Dec 03 11:49:25 crc kubenswrapper[5052]: W1203 11:49:25.833070 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9d8cd78f_2670_4933_81dd_01cdc0044454.slice/crio-fe5a7b41847839b2299acdb962194c6f5f4679c210cf40fde158b31f5dd06532 WatchSource:0}: Error finding container fe5a7b41847839b2299acdb962194c6f5f4679c210cf40fde158b31f5dd06532: Status 404 returned error can't find the container with id fe5a7b41847839b2299acdb962194c6f5f4679c210cf40fde158b31f5dd06532 Dec 03 11:49:26 crc kubenswrapper[5052]: I1203 11:49:26.046794 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/glance-db-sync-zzt28"] Dec 03 11:49:26 crc kubenswrapper[5052]: I1203 11:49:26.048688 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-db-sync-zzt28" Dec 03 11:49:26 crc kubenswrapper[5052]: I1203 11:49:26.081245 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-config-data" Dec 03 11:49:26 crc kubenswrapper[5052]: I1203 11:49:26.081487 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-glance-dockercfg-zntkb" Dec 03 11:49:26 crc kubenswrapper[5052]: I1203 11:49:26.089589 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-db-sync-zzt28"] Dec 03 11:49:26 crc kubenswrapper[5052]: I1203 11:49:26.111074 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-49dgg\" (UniqueName: \"kubernetes.io/projected/a7ea647c-c7da-4001-befa-c8412db3570b-kube-api-access-49dgg\") pod \"glance-db-sync-zzt28\" (UID: \"a7ea647c-c7da-4001-befa-c8412db3570b\") " pod="openstack-kuttl-tests/glance-db-sync-zzt28" Dec 03 11:49:26 crc kubenswrapper[5052]: I1203 11:49:26.111221 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a7ea647c-c7da-4001-befa-c8412db3570b-config-data\") pod \"glance-db-sync-zzt28\" (UID: \"a7ea647c-c7da-4001-befa-c8412db3570b\") " pod="openstack-kuttl-tests/glance-db-sync-zzt28" Dec 03 11:49:26 crc kubenswrapper[5052]: I1203 11:49:26.111260 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/a7ea647c-c7da-4001-befa-c8412db3570b-db-sync-config-data\") pod \"glance-db-sync-zzt28\" (UID: \"a7ea647c-c7da-4001-befa-c8412db3570b\") " pod="openstack-kuttl-tests/glance-db-sync-zzt28" Dec 03 11:49:26 crc kubenswrapper[5052]: I1203 11:49:26.111359 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a7ea647c-c7da-4001-befa-c8412db3570b-combined-ca-bundle\") pod \"glance-db-sync-zzt28\" (UID: \"a7ea647c-c7da-4001-befa-c8412db3570b\") " pod="openstack-kuttl-tests/glance-db-sync-zzt28" Dec 03 11:49:26 crc kubenswrapper[5052]: I1203 11:49:26.213733 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a7ea647c-c7da-4001-befa-c8412db3570b-combined-ca-bundle\") pod \"glance-db-sync-zzt28\" (UID: \"a7ea647c-c7da-4001-befa-c8412db3570b\") " pod="openstack-kuttl-tests/glance-db-sync-zzt28" Dec 03 11:49:26 crc kubenswrapper[5052]: I1203 11:49:26.213789 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-49dgg\" (UniqueName: \"kubernetes.io/projected/a7ea647c-c7da-4001-befa-c8412db3570b-kube-api-access-49dgg\") pod \"glance-db-sync-zzt28\" (UID: \"a7ea647c-c7da-4001-befa-c8412db3570b\") " pod="openstack-kuttl-tests/glance-db-sync-zzt28" Dec 03 11:49:26 crc kubenswrapper[5052]: I1203 11:49:26.213855 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a7ea647c-c7da-4001-befa-c8412db3570b-config-data\") pod \"glance-db-sync-zzt28\" (UID: \"a7ea647c-c7da-4001-befa-c8412db3570b\") " pod="openstack-kuttl-tests/glance-db-sync-zzt28" Dec 03 11:49:26 crc kubenswrapper[5052]: I1203 11:49:26.213876 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/a7ea647c-c7da-4001-befa-c8412db3570b-db-sync-config-data\") pod \"glance-db-sync-zzt28\" (UID: \"a7ea647c-c7da-4001-befa-c8412db3570b\") " pod="openstack-kuttl-tests/glance-db-sync-zzt28" Dec 03 11:49:26 crc kubenswrapper[5052]: I1203 11:49:26.216731 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/a7ea647c-c7da-4001-befa-c8412db3570b-db-sync-config-data\") pod \"glance-db-sync-zzt28\" (UID: \"a7ea647c-c7da-4001-befa-c8412db3570b\") " pod="openstack-kuttl-tests/glance-db-sync-zzt28" Dec 03 11:49:26 crc kubenswrapper[5052]: I1203 11:49:26.217170 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a7ea647c-c7da-4001-befa-c8412db3570b-combined-ca-bundle\") pod \"glance-db-sync-zzt28\" (UID: \"a7ea647c-c7da-4001-befa-c8412db3570b\") " pod="openstack-kuttl-tests/glance-db-sync-zzt28" Dec 03 11:49:26 crc kubenswrapper[5052]: I1203 11:49:26.217980 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a7ea647c-c7da-4001-befa-c8412db3570b-config-data\") pod \"glance-db-sync-zzt28\" (UID: \"a7ea647c-c7da-4001-befa-c8412db3570b\") " pod="openstack-kuttl-tests/glance-db-sync-zzt28" Dec 03 11:49:26 crc kubenswrapper[5052]: I1203 11:49:26.229483 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-49dgg\" (UniqueName: \"kubernetes.io/projected/a7ea647c-c7da-4001-befa-c8412db3570b-kube-api-access-49dgg\") pod \"glance-db-sync-zzt28\" (UID: \"a7ea647c-c7da-4001-befa-c8412db3570b\") " pod="openstack-kuttl-tests/glance-db-sync-zzt28" Dec 03 11:49:26 crc kubenswrapper[5052]: I1203 11:49:26.408134 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-db-sync-zzt28" Dec 03 11:49:26 crc kubenswrapper[5052]: I1203 11:49:26.483030 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovn-northd-0" event={"ID":"9d8cd78f-2670-4933-81dd-01cdc0044454","Type":"ContainerStarted","Data":"a00e41ad3b048582d984194b4e78df8f197fd9e5a5af35f2e726a5b8f13fb9fb"} Dec 03 11:49:26 crc kubenswrapper[5052]: I1203 11:49:26.483076 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovn-northd-0" event={"ID":"9d8cd78f-2670-4933-81dd-01cdc0044454","Type":"ContainerStarted","Data":"2cf5c7f704cdea9711753e85146be0ca1fd13eb56b9658c079f3f91d897ce7a3"} Dec 03 11:49:26 crc kubenswrapper[5052]: I1203 11:49:26.483086 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovn-northd-0" event={"ID":"9d8cd78f-2670-4933-81dd-01cdc0044454","Type":"ContainerStarted","Data":"fe5a7b41847839b2299acdb962194c6f5f4679c210cf40fde158b31f5dd06532"} Dec 03 11:49:26 crc kubenswrapper[5052]: I1203 11:49:26.483208 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 11:49:26 crc kubenswrapper[5052]: I1203 11:49:26.514687 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/ovn-northd-0" podStartSLOduration=2.514665663 podStartE2EDuration="2.514665663s" podCreationTimestamp="2025-12-03 11:49:24 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:49:26.505805999 +0000 UTC m=+1757.094047763" watchObservedRunningTime="2025-12-03 11:49:26.514665663 +0000 UTC m=+1757.102907427" Dec 03 11:49:26 crc kubenswrapper[5052]: I1203 11:49:26.843774 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-db-sync-zzt28"] Dec 03 11:49:26 crc kubenswrapper[5052]: W1203 11:49:26.847623 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda7ea647c_c7da_4001_befa_c8412db3570b.slice/crio-9ea6a1ae1be6707cf7c0d5a03f30229b23b3a842302c4cdc10869f2767e0c9e2 WatchSource:0}: Error finding container 9ea6a1ae1be6707cf7c0d5a03f30229b23b3a842302c4cdc10869f2767e0c9e2: Status 404 returned error can't find the container with id 9ea6a1ae1be6707cf7c0d5a03f30229b23b3a842302c4cdc10869f2767e0c9e2 Dec 03 11:49:27 crc kubenswrapper[5052]: I1203 11:49:27.131389 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/337bf4ce-ed6a-47bb-820d-6d64cd15e16e-etc-swift\") pod \"swift-storage-0\" (UID: \"337bf4ce-ed6a-47bb-820d-6d64cd15e16e\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 11:49:27 crc kubenswrapper[5052]: E1203 11:49:27.131595 5052 projected.go:288] Couldn't get configMap openstack-kuttl-tests/swift-ring-files: configmap "swift-ring-files" not found Dec 03 11:49:27 crc kubenswrapper[5052]: E1203 11:49:27.131631 5052 projected.go:194] Error preparing data for projected volume etc-swift for pod openstack-kuttl-tests/swift-storage-0: configmap "swift-ring-files" not found Dec 03 11:49:27 crc kubenswrapper[5052]: E1203 11:49:27.131702 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/337bf4ce-ed6a-47bb-820d-6d64cd15e16e-etc-swift podName:337bf4ce-ed6a-47bb-820d-6d64cd15e16e nodeName:}" failed. No retries permitted until 2025-12-03 11:49:31.131679758 +0000 UTC m=+1761.719921582 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "etc-swift" (UniqueName: "kubernetes.io/projected/337bf4ce-ed6a-47bb-820d-6d64cd15e16e-etc-swift") pod "swift-storage-0" (UID: "337bf4ce-ed6a-47bb-820d-6d64cd15e16e") : configmap "swift-ring-files" not found Dec 03 11:49:27 crc kubenswrapper[5052]: I1203 11:49:27.170239 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/swift-ring-rebalance-g5xx8"] Dec 03 11:49:27 crc kubenswrapper[5052]: I1203 11:49:27.171325 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-ring-rebalance-g5xx8" Dec 03 11:49:27 crc kubenswrapper[5052]: I1203 11:49:27.172880 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"swift-proxy-config-data" Dec 03 11:49:27 crc kubenswrapper[5052]: I1203 11:49:27.172963 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"swift-ring-scripts" Dec 03 11:49:27 crc kubenswrapper[5052]: I1203 11:49:27.173175 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"swift-ring-config-data" Dec 03 11:49:27 crc kubenswrapper[5052]: I1203 11:49:27.190524 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/swift-ring-rebalance-g5xx8"] Dec 03 11:49:27 crc kubenswrapper[5052]: I1203 11:49:27.233626 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3024dd24-2f4a-454d-b7a6-d7ba158c6f33-combined-ca-bundle\") pod \"swift-ring-rebalance-g5xx8\" (UID: \"3024dd24-2f4a-454d-b7a6-d7ba158c6f33\") " pod="openstack-kuttl-tests/swift-ring-rebalance-g5xx8" Dec 03 11:49:27 crc kubenswrapper[5052]: I1203 11:49:27.233670 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dispersionconf\" (UniqueName: \"kubernetes.io/secret/3024dd24-2f4a-454d-b7a6-d7ba158c6f33-dispersionconf\") pod \"swift-ring-rebalance-g5xx8\" (UID: \"3024dd24-2f4a-454d-b7a6-d7ba158c6f33\") " pod="openstack-kuttl-tests/swift-ring-rebalance-g5xx8" Dec 03 11:49:27 crc kubenswrapper[5052]: I1203 11:49:27.233686 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"swiftconf\" (UniqueName: \"kubernetes.io/secret/3024dd24-2f4a-454d-b7a6-d7ba158c6f33-swiftconf\") pod \"swift-ring-rebalance-g5xx8\" (UID: \"3024dd24-2f4a-454d-b7a6-d7ba158c6f33\") " pod="openstack-kuttl-tests/swift-ring-rebalance-g5xx8" Dec 03 11:49:27 crc kubenswrapper[5052]: I1203 11:49:27.233721 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ring-data-devices\" (UniqueName: \"kubernetes.io/configmap/3024dd24-2f4a-454d-b7a6-d7ba158c6f33-ring-data-devices\") pod \"swift-ring-rebalance-g5xx8\" (UID: \"3024dd24-2f4a-454d-b7a6-d7ba158c6f33\") " pod="openstack-kuttl-tests/swift-ring-rebalance-g5xx8" Dec 03 11:49:27 crc kubenswrapper[5052]: I1203 11:49:27.233767 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/3024dd24-2f4a-454d-b7a6-d7ba158c6f33-scripts\") pod \"swift-ring-rebalance-g5xx8\" (UID: \"3024dd24-2f4a-454d-b7a6-d7ba158c6f33\") " pod="openstack-kuttl-tests/swift-ring-rebalance-g5xx8" Dec 03 11:49:27 crc kubenswrapper[5052]: I1203 11:49:27.233800 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/empty-dir/3024dd24-2f4a-454d-b7a6-d7ba158c6f33-etc-swift\") pod \"swift-ring-rebalance-g5xx8\" (UID: \"3024dd24-2f4a-454d-b7a6-d7ba158c6f33\") " pod="openstack-kuttl-tests/swift-ring-rebalance-g5xx8" Dec 03 11:49:27 crc kubenswrapper[5052]: I1203 11:49:27.233824 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l6p8s\" (UniqueName: \"kubernetes.io/projected/3024dd24-2f4a-454d-b7a6-d7ba158c6f33-kube-api-access-l6p8s\") pod \"swift-ring-rebalance-g5xx8\" (UID: \"3024dd24-2f4a-454d-b7a6-d7ba158c6f33\") " pod="openstack-kuttl-tests/swift-ring-rebalance-g5xx8" Dec 03 11:49:27 crc kubenswrapper[5052]: I1203 11:49:27.335598 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/empty-dir/3024dd24-2f4a-454d-b7a6-d7ba158c6f33-etc-swift\") pod \"swift-ring-rebalance-g5xx8\" (UID: \"3024dd24-2f4a-454d-b7a6-d7ba158c6f33\") " pod="openstack-kuttl-tests/swift-ring-rebalance-g5xx8" Dec 03 11:49:27 crc kubenswrapper[5052]: I1203 11:49:27.335660 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-l6p8s\" (UniqueName: \"kubernetes.io/projected/3024dd24-2f4a-454d-b7a6-d7ba158c6f33-kube-api-access-l6p8s\") pod \"swift-ring-rebalance-g5xx8\" (UID: \"3024dd24-2f4a-454d-b7a6-d7ba158c6f33\") " pod="openstack-kuttl-tests/swift-ring-rebalance-g5xx8" Dec 03 11:49:27 crc kubenswrapper[5052]: I1203 11:49:27.335767 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3024dd24-2f4a-454d-b7a6-d7ba158c6f33-combined-ca-bundle\") pod \"swift-ring-rebalance-g5xx8\" (UID: \"3024dd24-2f4a-454d-b7a6-d7ba158c6f33\") " pod="openstack-kuttl-tests/swift-ring-rebalance-g5xx8" Dec 03 11:49:27 crc kubenswrapper[5052]: I1203 11:49:27.335794 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"dispersionconf\" (UniqueName: \"kubernetes.io/secret/3024dd24-2f4a-454d-b7a6-d7ba158c6f33-dispersionconf\") pod \"swift-ring-rebalance-g5xx8\" (UID: \"3024dd24-2f4a-454d-b7a6-d7ba158c6f33\") " pod="openstack-kuttl-tests/swift-ring-rebalance-g5xx8" Dec 03 11:49:27 crc kubenswrapper[5052]: I1203 11:49:27.335816 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"swiftconf\" (UniqueName: \"kubernetes.io/secret/3024dd24-2f4a-454d-b7a6-d7ba158c6f33-swiftconf\") pod \"swift-ring-rebalance-g5xx8\" (UID: \"3024dd24-2f4a-454d-b7a6-d7ba158c6f33\") " pod="openstack-kuttl-tests/swift-ring-rebalance-g5xx8" Dec 03 11:49:27 crc kubenswrapper[5052]: I1203 11:49:27.335861 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ring-data-devices\" (UniqueName: \"kubernetes.io/configmap/3024dd24-2f4a-454d-b7a6-d7ba158c6f33-ring-data-devices\") pod \"swift-ring-rebalance-g5xx8\" (UID: \"3024dd24-2f4a-454d-b7a6-d7ba158c6f33\") " pod="openstack-kuttl-tests/swift-ring-rebalance-g5xx8" Dec 03 11:49:27 crc kubenswrapper[5052]: I1203 11:49:27.335912 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/3024dd24-2f4a-454d-b7a6-d7ba158c6f33-scripts\") pod \"swift-ring-rebalance-g5xx8\" (UID: \"3024dd24-2f4a-454d-b7a6-d7ba158c6f33\") " pod="openstack-kuttl-tests/swift-ring-rebalance-g5xx8" Dec 03 11:49:27 crc kubenswrapper[5052]: I1203 11:49:27.336024 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"etc-swift\" (UniqueName: \"kubernetes.io/empty-dir/3024dd24-2f4a-454d-b7a6-d7ba158c6f33-etc-swift\") pod \"swift-ring-rebalance-g5xx8\" (UID: \"3024dd24-2f4a-454d-b7a6-d7ba158c6f33\") " pod="openstack-kuttl-tests/swift-ring-rebalance-g5xx8" Dec 03 11:49:27 crc kubenswrapper[5052]: I1203 11:49:27.336674 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/3024dd24-2f4a-454d-b7a6-d7ba158c6f33-scripts\") pod \"swift-ring-rebalance-g5xx8\" (UID: \"3024dd24-2f4a-454d-b7a6-d7ba158c6f33\") " pod="openstack-kuttl-tests/swift-ring-rebalance-g5xx8" Dec 03 11:49:27 crc kubenswrapper[5052]: I1203 11:49:27.337673 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ring-data-devices\" (UniqueName: \"kubernetes.io/configmap/3024dd24-2f4a-454d-b7a6-d7ba158c6f33-ring-data-devices\") pod \"swift-ring-rebalance-g5xx8\" (UID: \"3024dd24-2f4a-454d-b7a6-d7ba158c6f33\") " pod="openstack-kuttl-tests/swift-ring-rebalance-g5xx8" Dec 03 11:49:27 crc kubenswrapper[5052]: I1203 11:49:27.348013 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"swiftconf\" (UniqueName: \"kubernetes.io/secret/3024dd24-2f4a-454d-b7a6-d7ba158c6f33-swiftconf\") pod \"swift-ring-rebalance-g5xx8\" (UID: \"3024dd24-2f4a-454d-b7a6-d7ba158c6f33\") " pod="openstack-kuttl-tests/swift-ring-rebalance-g5xx8" Dec 03 11:49:27 crc kubenswrapper[5052]: I1203 11:49:27.349274 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3024dd24-2f4a-454d-b7a6-d7ba158c6f33-combined-ca-bundle\") pod \"swift-ring-rebalance-g5xx8\" (UID: \"3024dd24-2f4a-454d-b7a6-d7ba158c6f33\") " pod="openstack-kuttl-tests/swift-ring-rebalance-g5xx8" Dec 03 11:49:27 crc kubenswrapper[5052]: I1203 11:49:27.352986 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-l6p8s\" (UniqueName: \"kubernetes.io/projected/3024dd24-2f4a-454d-b7a6-d7ba158c6f33-kube-api-access-l6p8s\") pod \"swift-ring-rebalance-g5xx8\" (UID: \"3024dd24-2f4a-454d-b7a6-d7ba158c6f33\") " pod="openstack-kuttl-tests/swift-ring-rebalance-g5xx8" Dec 03 11:49:27 crc kubenswrapper[5052]: I1203 11:49:27.360035 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"dispersionconf\" (UniqueName: \"kubernetes.io/secret/3024dd24-2f4a-454d-b7a6-d7ba158c6f33-dispersionconf\") pod \"swift-ring-rebalance-g5xx8\" (UID: \"3024dd24-2f4a-454d-b7a6-d7ba158c6f33\") " pod="openstack-kuttl-tests/swift-ring-rebalance-g5xx8" Dec 03 11:49:27 crc kubenswrapper[5052]: I1203 11:49:27.494705 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-db-sync-zzt28" event={"ID":"a7ea647c-c7da-4001-befa-c8412db3570b","Type":"ContainerStarted","Data":"8dd2dc759600bdcc2a723346b6089dd30b5efa1d3c638543a2d02e0bfda6a7f1"} Dec 03 11:49:27 crc kubenswrapper[5052]: I1203 11:49:27.495746 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-db-sync-zzt28" event={"ID":"a7ea647c-c7da-4001-befa-c8412db3570b","Type":"ContainerStarted","Data":"9ea6a1ae1be6707cf7c0d5a03f30229b23b3a842302c4cdc10869f2767e0c9e2"} Dec 03 11:49:27 crc kubenswrapper[5052]: I1203 11:49:27.509194 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-ring-rebalance-g5xx8" Dec 03 11:49:27 crc kubenswrapper[5052]: I1203 11:49:27.529288 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/glance-db-sync-zzt28" podStartSLOduration=1.5292535699999998 podStartE2EDuration="1.52925357s" podCreationTimestamp="2025-12-03 11:49:26 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:49:27.515178206 +0000 UTC m=+1758.103419970" watchObservedRunningTime="2025-12-03 11:49:27.52925357 +0000 UTC m=+1758.117495334" Dec 03 11:49:27 crc kubenswrapper[5052]: I1203 11:49:27.947844 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/swift-ring-rebalance-g5xx8"] Dec 03 11:49:27 crc kubenswrapper[5052]: W1203 11:49:27.954501 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3024dd24_2f4a_454d_b7a6_d7ba158c6f33.slice/crio-dd8788a6f530f455ada1b5638944166a0711e5f22b28c50794c49d6ef9441e9d WatchSource:0}: Error finding container dd8788a6f530f455ada1b5638944166a0711e5f22b28c50794c49d6ef9441e9d: Status 404 returned error can't find the container with id dd8788a6f530f455ada1b5638944166a0711e5f22b28c50794c49d6ef9441e9d Dec 03 11:49:28 crc kubenswrapper[5052]: I1203 11:49:28.505627 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-ring-rebalance-g5xx8" event={"ID":"3024dd24-2f4a-454d-b7a6-d7ba158c6f33","Type":"ContainerStarted","Data":"92f0af5182d1bd32fb01791a918d23abf78685fd72687f4722dc812f9d7e2f5e"} Dec 03 11:49:28 crc kubenswrapper[5052]: I1203 11:49:28.505983 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-ring-rebalance-g5xx8" event={"ID":"3024dd24-2f4a-454d-b7a6-d7ba158c6f33","Type":"ContainerStarted","Data":"dd8788a6f530f455ada1b5638944166a0711e5f22b28c50794c49d6ef9441e9d"} Dec 03 11:49:31 crc kubenswrapper[5052]: I1203 11:49:31.214679 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/337bf4ce-ed6a-47bb-820d-6d64cd15e16e-etc-swift\") pod \"swift-storage-0\" (UID: \"337bf4ce-ed6a-47bb-820d-6d64cd15e16e\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 11:49:31 crc kubenswrapper[5052]: E1203 11:49:31.214891 5052 projected.go:288] Couldn't get configMap openstack-kuttl-tests/swift-ring-files: configmap "swift-ring-files" not found Dec 03 11:49:31 crc kubenswrapper[5052]: E1203 11:49:31.215162 5052 projected.go:194] Error preparing data for projected volume etc-swift for pod openstack-kuttl-tests/swift-storage-0: configmap "swift-ring-files" not found Dec 03 11:49:31 crc kubenswrapper[5052]: E1203 11:49:31.215241 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/337bf4ce-ed6a-47bb-820d-6d64cd15e16e-etc-swift podName:337bf4ce-ed6a-47bb-820d-6d64cd15e16e nodeName:}" failed. No retries permitted until 2025-12-03 11:49:39.215218943 +0000 UTC m=+1769.803460727 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "etc-swift" (UniqueName: "kubernetes.io/projected/337bf4ce-ed6a-47bb-820d-6d64cd15e16e-etc-swift") pod "swift-storage-0" (UID: "337bf4ce-ed6a-47bb-820d-6d64cd15e16e") : configmap "swift-ring-files" not found Dec 03 11:49:31 crc kubenswrapper[5052]: I1203 11:49:31.493986 5052 scope.go:117] "RemoveContainer" containerID="553635edc0dee6e3978dee3481740c8629f4746318a9834f801f334db1a16c3d" Dec 03 11:49:31 crc kubenswrapper[5052]: I1203 11:49:31.514298 5052 scope.go:117] "RemoveContainer" containerID="e091f633e39fe79c62c14d716851b357c6ec86a4712a030e1839dbe830fb60a3" Dec 03 11:49:31 crc kubenswrapper[5052]: I1203 11:49:31.551063 5052 scope.go:117] "RemoveContainer" containerID="649f1e16c0c9e3ca2dc574767b85143b4bf02e3118f334d98a6e745e58991caa" Dec 03 11:49:31 crc kubenswrapper[5052]: I1203 11:49:31.570323 5052 scope.go:117] "RemoveContainer" containerID="4d446fd67e4ff7b7abd8b762a4e09b8cacfc730c8985d62af75216baa471e69c" Dec 03 11:49:31 crc kubenswrapper[5052]: I1203 11:49:31.626470 5052 scope.go:117] "RemoveContainer" containerID="b4718facd796f54b8c673f27e81c4cebe41d2467bb0f69d919998113712b9cce" Dec 03 11:49:31 crc kubenswrapper[5052]: I1203 11:49:31.646447 5052 scope.go:117] "RemoveContainer" containerID="5bb72fbb2016062939d5ea85849b5d6c4e299c96778da77c8f957ba54d086e4c" Dec 03 11:49:31 crc kubenswrapper[5052]: I1203 11:49:31.679134 5052 scope.go:117] "RemoveContainer" containerID="b61c583a8237d6942c8885a37c89d5361107f0bd48e80748feca1601cb46e570" Dec 03 11:49:31 crc kubenswrapper[5052]: I1203 11:49:31.705183 5052 scope.go:117] "RemoveContainer" containerID="4a26b9bf2f464394dfe8329500db73ffc7c0cc73ad437ed732610bb4cd958539" Dec 03 11:49:31 crc kubenswrapper[5052]: I1203 11:49:31.722524 5052 scope.go:117] "RemoveContainer" containerID="1b505b87190dc754cde56718805d0189a89e23308cc1e51b861d4ddf4b6bfc89" Dec 03 11:49:31 crc kubenswrapper[5052]: I1203 11:49:31.764241 5052 scope.go:117] "RemoveContainer" containerID="a7e16a1558edb19db29f41cde5903f2a8a903af9d50cea81f332f651715f1903" Dec 03 11:49:31 crc kubenswrapper[5052]: I1203 11:49:31.786589 5052 scope.go:117] "RemoveContainer" containerID="0b842a895f06933ddfc5ac479287d4141d4c61e4c5d4906486c088a12e96d8e7" Dec 03 11:49:33 crc kubenswrapper[5052]: I1203 11:49:33.556826 5052 generic.go:334] "Generic (PLEG): container finished" podID="a7ea647c-c7da-4001-befa-c8412db3570b" containerID="8dd2dc759600bdcc2a723346b6089dd30b5efa1d3c638543a2d02e0bfda6a7f1" exitCode=0 Dec 03 11:49:33 crc kubenswrapper[5052]: I1203 11:49:33.556952 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-db-sync-zzt28" event={"ID":"a7ea647c-c7da-4001-befa-c8412db3570b","Type":"ContainerDied","Data":"8dd2dc759600bdcc2a723346b6089dd30b5efa1d3c638543a2d02e0bfda6a7f1"} Dec 03 11:49:33 crc kubenswrapper[5052]: I1203 11:49:33.575331 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/swift-ring-rebalance-g5xx8" podStartSLOduration=6.57530517 podStartE2EDuration="6.57530517s" podCreationTimestamp="2025-12-03 11:49:27 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:49:28.533140852 +0000 UTC m=+1759.121382616" watchObservedRunningTime="2025-12-03 11:49:33.57530517 +0000 UTC m=+1764.163546944" Dec 03 11:49:34 crc kubenswrapper[5052]: I1203 11:49:34.983444 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-db-sync-zzt28" Dec 03 11:49:35 crc kubenswrapper[5052]: I1203 11:49:35.087443 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a7ea647c-c7da-4001-befa-c8412db3570b-combined-ca-bundle\") pod \"a7ea647c-c7da-4001-befa-c8412db3570b\" (UID: \"a7ea647c-c7da-4001-befa-c8412db3570b\") " Dec 03 11:49:35 crc kubenswrapper[5052]: I1203 11:49:35.087570 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/a7ea647c-c7da-4001-befa-c8412db3570b-db-sync-config-data\") pod \"a7ea647c-c7da-4001-befa-c8412db3570b\" (UID: \"a7ea647c-c7da-4001-befa-c8412db3570b\") " Dec 03 11:49:35 crc kubenswrapper[5052]: I1203 11:49:35.087668 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a7ea647c-c7da-4001-befa-c8412db3570b-config-data\") pod \"a7ea647c-c7da-4001-befa-c8412db3570b\" (UID: \"a7ea647c-c7da-4001-befa-c8412db3570b\") " Dec 03 11:49:35 crc kubenswrapper[5052]: I1203 11:49:35.087695 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-49dgg\" (UniqueName: \"kubernetes.io/projected/a7ea647c-c7da-4001-befa-c8412db3570b-kube-api-access-49dgg\") pod \"a7ea647c-c7da-4001-befa-c8412db3570b\" (UID: \"a7ea647c-c7da-4001-befa-c8412db3570b\") " Dec 03 11:49:35 crc kubenswrapper[5052]: I1203 11:49:35.092943 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a7ea647c-c7da-4001-befa-c8412db3570b-kube-api-access-49dgg" (OuterVolumeSpecName: "kube-api-access-49dgg") pod "a7ea647c-c7da-4001-befa-c8412db3570b" (UID: "a7ea647c-c7da-4001-befa-c8412db3570b"). InnerVolumeSpecName "kube-api-access-49dgg". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:49:35 crc kubenswrapper[5052]: I1203 11:49:35.092959 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a7ea647c-c7da-4001-befa-c8412db3570b-db-sync-config-data" (OuterVolumeSpecName: "db-sync-config-data") pod "a7ea647c-c7da-4001-befa-c8412db3570b" (UID: "a7ea647c-c7da-4001-befa-c8412db3570b"). InnerVolumeSpecName "db-sync-config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:49:35 crc kubenswrapper[5052]: I1203 11:49:35.112196 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a7ea647c-c7da-4001-befa-c8412db3570b-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "a7ea647c-c7da-4001-befa-c8412db3570b" (UID: "a7ea647c-c7da-4001-befa-c8412db3570b"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:49:35 crc kubenswrapper[5052]: I1203 11:49:35.151645 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a7ea647c-c7da-4001-befa-c8412db3570b-config-data" (OuterVolumeSpecName: "config-data") pod "a7ea647c-c7da-4001-befa-c8412db3570b" (UID: "a7ea647c-c7da-4001-befa-c8412db3570b"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:49:35 crc kubenswrapper[5052]: I1203 11:49:35.189893 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a7ea647c-c7da-4001-befa-c8412db3570b-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:49:35 crc kubenswrapper[5052]: I1203 11:49:35.190057 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-49dgg\" (UniqueName: \"kubernetes.io/projected/a7ea647c-c7da-4001-befa-c8412db3570b-kube-api-access-49dgg\") on node \"crc\" DevicePath \"\"" Dec 03 11:49:35 crc kubenswrapper[5052]: I1203 11:49:35.190132 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a7ea647c-c7da-4001-befa-c8412db3570b-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:49:35 crc kubenswrapper[5052]: I1203 11:49:35.190224 5052 reconciler_common.go:293] "Volume detached for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/a7ea647c-c7da-4001-befa-c8412db3570b-db-sync-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:49:35 crc kubenswrapper[5052]: I1203 11:49:35.581802 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-db-sync-zzt28" event={"ID":"a7ea647c-c7da-4001-befa-c8412db3570b","Type":"ContainerDied","Data":"9ea6a1ae1be6707cf7c0d5a03f30229b23b3a842302c4cdc10869f2767e0c9e2"} Dec 03 11:49:35 crc kubenswrapper[5052]: I1203 11:49:35.582088 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9ea6a1ae1be6707cf7c0d5a03f30229b23b3a842302c4cdc10869f2767e0c9e2" Dec 03 11:49:35 crc kubenswrapper[5052]: I1203 11:49:35.581893 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-db-sync-zzt28" Dec 03 11:49:35 crc kubenswrapper[5052]: I1203 11:49:35.589119 5052 generic.go:334] "Generic (PLEG): container finished" podID="3024dd24-2f4a-454d-b7a6-d7ba158c6f33" containerID="92f0af5182d1bd32fb01791a918d23abf78685fd72687f4722dc812f9d7e2f5e" exitCode=0 Dec 03 11:49:35 crc kubenswrapper[5052]: I1203 11:49:35.589166 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-ring-rebalance-g5xx8" event={"ID":"3024dd24-2f4a-454d-b7a6-d7ba158c6f33","Type":"ContainerDied","Data":"92f0af5182d1bd32fb01791a918d23abf78685fd72687f4722dc812f9d7e2f5e"} Dec 03 11:49:35 crc kubenswrapper[5052]: I1203 11:49:35.672999 5052 scope.go:117] "RemoveContainer" containerID="25fe88f8d102f09983f8884e65f0c6c1575ade67921f10a630e6dcc3ea8b506e" Dec 03 11:49:35 crc kubenswrapper[5052]: E1203 11:49:35.673296 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 11:49:37 crc kubenswrapper[5052]: I1203 11:49:37.001983 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-ring-rebalance-g5xx8" Dec 03 11:49:37 crc kubenswrapper[5052]: I1203 11:49:37.121032 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/empty-dir/3024dd24-2f4a-454d-b7a6-d7ba158c6f33-etc-swift\") pod \"3024dd24-2f4a-454d-b7a6-d7ba158c6f33\" (UID: \"3024dd24-2f4a-454d-b7a6-d7ba158c6f33\") " Dec 03 11:49:37 crc kubenswrapper[5052]: I1203 11:49:37.121126 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l6p8s\" (UniqueName: \"kubernetes.io/projected/3024dd24-2f4a-454d-b7a6-d7ba158c6f33-kube-api-access-l6p8s\") pod \"3024dd24-2f4a-454d-b7a6-d7ba158c6f33\" (UID: \"3024dd24-2f4a-454d-b7a6-d7ba158c6f33\") " Dec 03 11:49:37 crc kubenswrapper[5052]: I1203 11:49:37.121193 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"dispersionconf\" (UniqueName: \"kubernetes.io/secret/3024dd24-2f4a-454d-b7a6-d7ba158c6f33-dispersionconf\") pod \"3024dd24-2f4a-454d-b7a6-d7ba158c6f33\" (UID: \"3024dd24-2f4a-454d-b7a6-d7ba158c6f33\") " Dec 03 11:49:37 crc kubenswrapper[5052]: I1203 11:49:37.121250 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/3024dd24-2f4a-454d-b7a6-d7ba158c6f33-scripts\") pod \"3024dd24-2f4a-454d-b7a6-d7ba158c6f33\" (UID: \"3024dd24-2f4a-454d-b7a6-d7ba158c6f33\") " Dec 03 11:49:37 crc kubenswrapper[5052]: I1203 11:49:37.121280 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ring-data-devices\" (UniqueName: \"kubernetes.io/configmap/3024dd24-2f4a-454d-b7a6-d7ba158c6f33-ring-data-devices\") pod \"3024dd24-2f4a-454d-b7a6-d7ba158c6f33\" (UID: \"3024dd24-2f4a-454d-b7a6-d7ba158c6f33\") " Dec 03 11:49:37 crc kubenswrapper[5052]: I1203 11:49:37.121315 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3024dd24-2f4a-454d-b7a6-d7ba158c6f33-combined-ca-bundle\") pod \"3024dd24-2f4a-454d-b7a6-d7ba158c6f33\" (UID: \"3024dd24-2f4a-454d-b7a6-d7ba158c6f33\") " Dec 03 11:49:37 crc kubenswrapper[5052]: I1203 11:49:37.121346 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"swiftconf\" (UniqueName: \"kubernetes.io/secret/3024dd24-2f4a-454d-b7a6-d7ba158c6f33-swiftconf\") pod \"3024dd24-2f4a-454d-b7a6-d7ba158c6f33\" (UID: \"3024dd24-2f4a-454d-b7a6-d7ba158c6f33\") " Dec 03 11:49:37 crc kubenswrapper[5052]: I1203 11:49:37.122106 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3024dd24-2f4a-454d-b7a6-d7ba158c6f33-etc-swift" (OuterVolumeSpecName: "etc-swift") pod "3024dd24-2f4a-454d-b7a6-d7ba158c6f33" (UID: "3024dd24-2f4a-454d-b7a6-d7ba158c6f33"). InnerVolumeSpecName "etc-swift". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:49:37 crc kubenswrapper[5052]: I1203 11:49:37.122235 5052 reconciler_common.go:293] "Volume detached for volume \"etc-swift\" (UniqueName: \"kubernetes.io/empty-dir/3024dd24-2f4a-454d-b7a6-d7ba158c6f33-etc-swift\") on node \"crc\" DevicePath \"\"" Dec 03 11:49:37 crc kubenswrapper[5052]: I1203 11:49:37.122344 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3024dd24-2f4a-454d-b7a6-d7ba158c6f33-ring-data-devices" (OuterVolumeSpecName: "ring-data-devices") pod "3024dd24-2f4a-454d-b7a6-d7ba158c6f33" (UID: "3024dd24-2f4a-454d-b7a6-d7ba158c6f33"). InnerVolumeSpecName "ring-data-devices". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:49:37 crc kubenswrapper[5052]: I1203 11:49:37.127808 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3024dd24-2f4a-454d-b7a6-d7ba158c6f33-kube-api-access-l6p8s" (OuterVolumeSpecName: "kube-api-access-l6p8s") pod "3024dd24-2f4a-454d-b7a6-d7ba158c6f33" (UID: "3024dd24-2f4a-454d-b7a6-d7ba158c6f33"). InnerVolumeSpecName "kube-api-access-l6p8s". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:49:37 crc kubenswrapper[5052]: I1203 11:49:37.130241 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3024dd24-2f4a-454d-b7a6-d7ba158c6f33-dispersionconf" (OuterVolumeSpecName: "dispersionconf") pod "3024dd24-2f4a-454d-b7a6-d7ba158c6f33" (UID: "3024dd24-2f4a-454d-b7a6-d7ba158c6f33"). InnerVolumeSpecName "dispersionconf". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:49:37 crc kubenswrapper[5052]: I1203 11:49:37.150818 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3024dd24-2f4a-454d-b7a6-d7ba158c6f33-scripts" (OuterVolumeSpecName: "scripts") pod "3024dd24-2f4a-454d-b7a6-d7ba158c6f33" (UID: "3024dd24-2f4a-454d-b7a6-d7ba158c6f33"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:49:37 crc kubenswrapper[5052]: I1203 11:49:37.152980 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3024dd24-2f4a-454d-b7a6-d7ba158c6f33-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "3024dd24-2f4a-454d-b7a6-d7ba158c6f33" (UID: "3024dd24-2f4a-454d-b7a6-d7ba158c6f33"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:49:37 crc kubenswrapper[5052]: I1203 11:49:37.156937 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3024dd24-2f4a-454d-b7a6-d7ba158c6f33-swiftconf" (OuterVolumeSpecName: "swiftconf") pod "3024dd24-2f4a-454d-b7a6-d7ba158c6f33" (UID: "3024dd24-2f4a-454d-b7a6-d7ba158c6f33"). InnerVolumeSpecName "swiftconf". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:49:37 crc kubenswrapper[5052]: I1203 11:49:37.224573 5052 reconciler_common.go:293] "Volume detached for volume \"ring-data-devices\" (UniqueName: \"kubernetes.io/configmap/3024dd24-2f4a-454d-b7a6-d7ba158c6f33-ring-data-devices\") on node \"crc\" DevicePath \"\"" Dec 03 11:49:37 crc kubenswrapper[5052]: I1203 11:49:37.224636 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3024dd24-2f4a-454d-b7a6-d7ba158c6f33-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:49:37 crc kubenswrapper[5052]: I1203 11:49:37.224655 5052 reconciler_common.go:293] "Volume detached for volume \"swiftconf\" (UniqueName: \"kubernetes.io/secret/3024dd24-2f4a-454d-b7a6-d7ba158c6f33-swiftconf\") on node \"crc\" DevicePath \"\"" Dec 03 11:49:37 crc kubenswrapper[5052]: I1203 11:49:37.224679 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-l6p8s\" (UniqueName: \"kubernetes.io/projected/3024dd24-2f4a-454d-b7a6-d7ba158c6f33-kube-api-access-l6p8s\") on node \"crc\" DevicePath \"\"" Dec 03 11:49:37 crc kubenswrapper[5052]: I1203 11:49:37.224700 5052 reconciler_common.go:293] "Volume detached for volume \"dispersionconf\" (UniqueName: \"kubernetes.io/secret/3024dd24-2f4a-454d-b7a6-d7ba158c6f33-dispersionconf\") on node \"crc\" DevicePath \"\"" Dec 03 11:49:37 crc kubenswrapper[5052]: I1203 11:49:37.224719 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/3024dd24-2f4a-454d-b7a6-d7ba158c6f33-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:49:37 crc kubenswrapper[5052]: I1203 11:49:37.606671 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-ring-rebalance-g5xx8" event={"ID":"3024dd24-2f4a-454d-b7a6-d7ba158c6f33","Type":"ContainerDied","Data":"dd8788a6f530f455ada1b5638944166a0711e5f22b28c50794c49d6ef9441e9d"} Dec 03 11:49:37 crc kubenswrapper[5052]: I1203 11:49:37.606726 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="dd8788a6f530f455ada1b5638944166a0711e5f22b28c50794c49d6ef9441e9d" Dec 03 11:49:37 crc kubenswrapper[5052]: I1203 11:49:37.606750 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-ring-rebalance-g5xx8" Dec 03 11:49:39 crc kubenswrapper[5052]: I1203 11:49:39.257491 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/337bf4ce-ed6a-47bb-820d-6d64cd15e16e-etc-swift\") pod \"swift-storage-0\" (UID: \"337bf4ce-ed6a-47bb-820d-6d64cd15e16e\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 11:49:39 crc kubenswrapper[5052]: I1203 11:49:39.266943 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/337bf4ce-ed6a-47bb-820d-6d64cd15e16e-etc-swift\") pod \"swift-storage-0\" (UID: \"337bf4ce-ed6a-47bb-820d-6d64cd15e16e\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 11:49:39 crc kubenswrapper[5052]: I1203 11:49:39.503295 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-storage-0" Dec 03 11:49:39 crc kubenswrapper[5052]: I1203 11:49:39.970162 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/swift-storage-0"] Dec 03 11:49:40 crc kubenswrapper[5052]: I1203 11:49:40.444116 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 11:49:40 crc kubenswrapper[5052]: I1203 11:49:40.633603 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"337bf4ce-ed6a-47bb-820d-6d64cd15e16e","Type":"ContainerStarted","Data":"da24bea2b91d001778224eaa9d20a940c1cfa8aec9ca78a78f4063dc10e39b5e"} Dec 03 11:49:40 crc kubenswrapper[5052]: I1203 11:49:40.633648 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"337bf4ce-ed6a-47bb-820d-6d64cd15e16e","Type":"ContainerStarted","Data":"fc539dbf46bdf8b30ab4b7079777cd46ae1b263b2a2b5b2e7afc75cabc6b5735"} Dec 03 11:49:41 crc kubenswrapper[5052]: I1203 11:49:41.641679 5052 generic.go:334] "Generic (PLEG): container finished" podID="10e3094e-f625-4d69-b0e9-358220c0d195" containerID="c42837bc18a40995ef2cc940533425d21d777f73ad9d7999540fb90eedd02248" exitCode=0 Dec 03 11:49:41 crc kubenswrapper[5052]: I1203 11:49:41.641768 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-server-0" event={"ID":"10e3094e-f625-4d69-b0e9-358220c0d195","Type":"ContainerDied","Data":"c42837bc18a40995ef2cc940533425d21d777f73ad9d7999540fb90eedd02248"} Dec 03 11:49:41 crc kubenswrapper[5052]: I1203 11:49:41.660310 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"337bf4ce-ed6a-47bb-820d-6d64cd15e16e","Type":"ContainerStarted","Data":"cac91cb8a133117b6ac9ef92818b0c43f08504bb99f13b28c87c24ca86efd6f6"} Dec 03 11:49:41 crc kubenswrapper[5052]: I1203 11:49:41.660607 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"337bf4ce-ed6a-47bb-820d-6d64cd15e16e","Type":"ContainerStarted","Data":"187fe18924b493edafcac4ae2021f9d0b6680c618116eb5301da25207a0518fb"} Dec 03 11:49:41 crc kubenswrapper[5052]: I1203 11:49:41.660622 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"337bf4ce-ed6a-47bb-820d-6d64cd15e16e","Type":"ContainerStarted","Data":"36f10fc8c3ae11fad49f741b63364021479706c1dcab5d1f51a1da9b35e558c2"} Dec 03 11:49:41 crc kubenswrapper[5052]: I1203 11:49:41.660633 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"337bf4ce-ed6a-47bb-820d-6d64cd15e16e","Type":"ContainerStarted","Data":"1572718ba303c5fe3c2aff8c645fa6d74e146847e7945778af6c12cfd1f31c90"} Dec 03 11:49:41 crc kubenswrapper[5052]: I1203 11:49:41.663450 5052 generic.go:334] "Generic (PLEG): container finished" podID="a5ba4775-815a-46ef-bd2d-05dfa7513288" containerID="f2cfc34275d0c16e884df462f785981d52cd8107d11e34b450fe684cd48e4f3b" exitCode=0 Dec 03 11:49:41 crc kubenswrapper[5052]: I1203 11:49:41.663485 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" event={"ID":"a5ba4775-815a-46ef-bd2d-05dfa7513288","Type":"ContainerDied","Data":"f2cfc34275d0c16e884df462f785981d52cd8107d11e34b450fe684cd48e4f3b"} Dec 03 11:49:42 crc kubenswrapper[5052]: I1203 11:49:42.692103 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" event={"ID":"a5ba4775-815a-46ef-bd2d-05dfa7513288","Type":"ContainerStarted","Data":"c25fbe13b5143561b88ab53fac63216ac379a26107913a3a532fbd7033bbeba6"} Dec 03 11:49:42 crc kubenswrapper[5052]: I1203 11:49:42.694240 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 11:49:42 crc kubenswrapper[5052]: I1203 11:49:42.703292 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-server-0" event={"ID":"10e3094e-f625-4d69-b0e9-358220c0d195","Type":"ContainerStarted","Data":"db5a765ca282bd8e9880ef9379b02a9202ec6dc773b029eb3abdd3d2f5581f14"} Dec 03 11:49:42 crc kubenswrapper[5052]: I1203 11:49:42.719133 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"337bf4ce-ed6a-47bb-820d-6d64cd15e16e","Type":"ContainerStarted","Data":"0bdc5789ab44d02d9702c5cf3e475db9c7a7ce6bf079ab5b06570c12673c510c"} Dec 03 11:49:42 crc kubenswrapper[5052]: I1203 11:49:42.719377 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"337bf4ce-ed6a-47bb-820d-6d64cd15e16e","Type":"ContainerStarted","Data":"ef2775155138fd30b620bc0fbab9074d6f7cd4364b1cbc82c977fd6b48054772"} Dec 03 11:49:42 crc kubenswrapper[5052]: I1203 11:49:42.719397 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"337bf4ce-ed6a-47bb-820d-6d64cd15e16e","Type":"ContainerStarted","Data":"bf4d57ce3d071492f7492f8995fdfdb4a50c770c3ee835e5864c7889d9aca849"} Dec 03 11:49:42 crc kubenswrapper[5052]: I1203 11:49:42.719448 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"337bf4ce-ed6a-47bb-820d-6d64cd15e16e","Type":"ContainerStarted","Data":"6b9be92cb0d9072365edb2b4115b091dd2b7a90d8f70424fe8b36078d19bcab6"} Dec 03 11:49:42 crc kubenswrapper[5052]: I1203 11:49:42.719459 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"337bf4ce-ed6a-47bb-820d-6d64cd15e16e","Type":"ContainerStarted","Data":"aeee80ed3e9e151ab7e9e179efcf5113a7eda44c709ef92d44f8e5e51a28368d"} Dec 03 11:49:42 crc kubenswrapper[5052]: I1203 11:49:42.719470 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"337bf4ce-ed6a-47bb-820d-6d64cd15e16e","Type":"ContainerStarted","Data":"6b146d4fbd455766a79f81d255c8552e4579557dae733283b1a6647e1871ce4a"} Dec 03 11:49:42 crc kubenswrapper[5052]: I1203 11:49:42.719481 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"337bf4ce-ed6a-47bb-820d-6d64cd15e16e","Type":"ContainerStarted","Data":"bf27f4652a0cd9c65c3371e4a8f8b663bd4dc1a05420dbf79f77e2f00185a31d"} Dec 03 11:49:42 crc kubenswrapper[5052]: I1203 11:49:42.744712 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" podStartSLOduration=37.744683006 podStartE2EDuration="37.744683006s" podCreationTimestamp="2025-12-03 11:49:05 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:49:42.714618714 +0000 UTC m=+1773.302860478" watchObservedRunningTime="2025-12-03 11:49:42.744683006 +0000 UTC m=+1773.332924780" Dec 03 11:49:42 crc kubenswrapper[5052]: I1203 11:49:42.769460 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/rabbitmq-server-0" podStartSLOduration=37.769438576 podStartE2EDuration="37.769438576s" podCreationTimestamp="2025-12-03 11:49:05 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:49:42.748391363 +0000 UTC m=+1773.336633127" watchObservedRunningTime="2025-12-03 11:49:42.769438576 +0000 UTC m=+1773.357680340" Dec 03 11:49:43 crc kubenswrapper[5052]: I1203 11:49:43.735541 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"337bf4ce-ed6a-47bb-820d-6d64cd15e16e","Type":"ContainerStarted","Data":"adcb7e1e3f0cfa3f2f565b2ee561330619830866aab79c1bb82dc8a5cacf862d"} Dec 03 11:49:43 crc kubenswrapper[5052]: I1203 11:49:43.736169 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"337bf4ce-ed6a-47bb-820d-6d64cd15e16e","Type":"ContainerStarted","Data":"54a021ff30a08f0f3bd288edbc672e78a69a0e8398c93034e0304c71564937f1"} Dec 03 11:49:43 crc kubenswrapper[5052]: I1203 11:49:43.736253 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"337bf4ce-ed6a-47bb-820d-6d64cd15e16e","Type":"ContainerStarted","Data":"308f757a4c5baeeb088bbd69071d883ee0fd5f6717ce1fb801584cc7308cf01e"} Dec 03 11:49:43 crc kubenswrapper[5052]: I1203 11:49:43.776260 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/swift-storage-0" podStartSLOduration=21.776237812 podStartE2EDuration="21.776237812s" podCreationTimestamp="2025-12-03 11:49:22 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:49:43.767451229 +0000 UTC m=+1774.355692993" watchObservedRunningTime="2025-12-03 11:49:43.776237812 +0000 UTC m=+1774.364479576" Dec 03 11:49:43 crc kubenswrapper[5052]: I1203 11:49:43.904495 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-88667bdf5-ndf84"] Dec 03 11:49:43 crc kubenswrapper[5052]: E1203 11:49:43.904910 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="a7ea647c-c7da-4001-befa-c8412db3570b" containerName="glance-db-sync" Dec 03 11:49:43 crc kubenswrapper[5052]: I1203 11:49:43.904942 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="a7ea647c-c7da-4001-befa-c8412db3570b" containerName="glance-db-sync" Dec 03 11:49:43 crc kubenswrapper[5052]: E1203 11:49:43.904972 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="3024dd24-2f4a-454d-b7a6-d7ba158c6f33" containerName="swift-ring-rebalance" Dec 03 11:49:43 crc kubenswrapper[5052]: I1203 11:49:43.904979 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="3024dd24-2f4a-454d-b7a6-d7ba158c6f33" containerName="swift-ring-rebalance" Dec 03 11:49:43 crc kubenswrapper[5052]: I1203 11:49:43.905141 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="a7ea647c-c7da-4001-befa-c8412db3570b" containerName="glance-db-sync" Dec 03 11:49:43 crc kubenswrapper[5052]: I1203 11:49:43.905160 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="3024dd24-2f4a-454d-b7a6-d7ba158c6f33" containerName="swift-ring-rebalance" Dec 03 11:49:43 crc kubenswrapper[5052]: I1203 11:49:43.906165 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-88667bdf5-ndf84" Dec 03 11:49:43 crc kubenswrapper[5052]: I1203 11:49:43.909820 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"dns-swift-storage-0" Dec 03 11:49:43 crc kubenswrapper[5052]: I1203 11:49:43.914021 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-88667bdf5-ndf84"] Dec 03 11:49:44 crc kubenswrapper[5052]: I1203 11:49:44.030102 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lqx4d\" (UniqueName: \"kubernetes.io/projected/5d0169e1-4f74-4f6e-89cc-70d2f79c34e2-kube-api-access-lqx4d\") pod \"dnsmasq-dnsmasq-88667bdf5-ndf84\" (UID: \"5d0169e1-4f74-4f6e-89cc-70d2f79c34e2\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-88667bdf5-ndf84" Dec 03 11:49:44 crc kubenswrapper[5052]: I1203 11:49:44.030590 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dns-swift-storage-0\" (UniqueName: \"kubernetes.io/configmap/5d0169e1-4f74-4f6e-89cc-70d2f79c34e2-dns-swift-storage-0\") pod \"dnsmasq-dnsmasq-88667bdf5-ndf84\" (UID: \"5d0169e1-4f74-4f6e-89cc-70d2f79c34e2\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-88667bdf5-ndf84" Dec 03 11:49:44 crc kubenswrapper[5052]: I1203 11:49:44.030678 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5d0169e1-4f74-4f6e-89cc-70d2f79c34e2-config\") pod \"dnsmasq-dnsmasq-88667bdf5-ndf84\" (UID: \"5d0169e1-4f74-4f6e-89cc-70d2f79c34e2\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-88667bdf5-ndf84" Dec 03 11:49:44 crc kubenswrapper[5052]: I1203 11:49:44.030764 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/5d0169e1-4f74-4f6e-89cc-70d2f79c34e2-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-88667bdf5-ndf84\" (UID: \"5d0169e1-4f74-4f6e-89cc-70d2f79c34e2\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-88667bdf5-ndf84" Dec 03 11:49:44 crc kubenswrapper[5052]: I1203 11:49:44.132605 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"dns-swift-storage-0\" (UniqueName: \"kubernetes.io/configmap/5d0169e1-4f74-4f6e-89cc-70d2f79c34e2-dns-swift-storage-0\") pod \"dnsmasq-dnsmasq-88667bdf5-ndf84\" (UID: \"5d0169e1-4f74-4f6e-89cc-70d2f79c34e2\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-88667bdf5-ndf84" Dec 03 11:49:44 crc kubenswrapper[5052]: I1203 11:49:44.132876 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5d0169e1-4f74-4f6e-89cc-70d2f79c34e2-config\") pod \"dnsmasq-dnsmasq-88667bdf5-ndf84\" (UID: \"5d0169e1-4f74-4f6e-89cc-70d2f79c34e2\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-88667bdf5-ndf84" Dec 03 11:49:44 crc kubenswrapper[5052]: I1203 11:49:44.132969 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/5d0169e1-4f74-4f6e-89cc-70d2f79c34e2-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-88667bdf5-ndf84\" (UID: \"5d0169e1-4f74-4f6e-89cc-70d2f79c34e2\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-88667bdf5-ndf84" Dec 03 11:49:44 crc kubenswrapper[5052]: I1203 11:49:44.133136 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-lqx4d\" (UniqueName: \"kubernetes.io/projected/5d0169e1-4f74-4f6e-89cc-70d2f79c34e2-kube-api-access-lqx4d\") pod \"dnsmasq-dnsmasq-88667bdf5-ndf84\" (UID: \"5d0169e1-4f74-4f6e-89cc-70d2f79c34e2\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-88667bdf5-ndf84" Dec 03 11:49:44 crc kubenswrapper[5052]: I1203 11:49:44.133427 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"dns-swift-storage-0\" (UniqueName: \"kubernetes.io/configmap/5d0169e1-4f74-4f6e-89cc-70d2f79c34e2-dns-swift-storage-0\") pod \"dnsmasq-dnsmasq-88667bdf5-ndf84\" (UID: \"5d0169e1-4f74-4f6e-89cc-70d2f79c34e2\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-88667bdf5-ndf84" Dec 03 11:49:44 crc kubenswrapper[5052]: I1203 11:49:44.133696 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5d0169e1-4f74-4f6e-89cc-70d2f79c34e2-config\") pod \"dnsmasq-dnsmasq-88667bdf5-ndf84\" (UID: \"5d0169e1-4f74-4f6e-89cc-70d2f79c34e2\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-88667bdf5-ndf84" Dec 03 11:49:44 crc kubenswrapper[5052]: I1203 11:49:44.133891 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/5d0169e1-4f74-4f6e-89cc-70d2f79c34e2-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-88667bdf5-ndf84\" (UID: \"5d0169e1-4f74-4f6e-89cc-70d2f79c34e2\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-88667bdf5-ndf84" Dec 03 11:49:44 crc kubenswrapper[5052]: I1203 11:49:44.150325 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-lqx4d\" (UniqueName: \"kubernetes.io/projected/5d0169e1-4f74-4f6e-89cc-70d2f79c34e2-kube-api-access-lqx4d\") pod \"dnsmasq-dnsmasq-88667bdf5-ndf84\" (UID: \"5d0169e1-4f74-4f6e-89cc-70d2f79c34e2\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-88667bdf5-ndf84" Dec 03 11:49:44 crc kubenswrapper[5052]: I1203 11:49:44.220651 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-88667bdf5-ndf84" Dec 03 11:49:44 crc kubenswrapper[5052]: I1203 11:49:44.677617 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-88667bdf5-ndf84"] Dec 03 11:49:44 crc kubenswrapper[5052]: W1203 11:49:44.680965 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5d0169e1_4f74_4f6e_89cc_70d2f79c34e2.slice/crio-369282bf4385da505931e707f888035ed5b46cf28271b9962d446947f428c38a WatchSource:0}: Error finding container 369282bf4385da505931e707f888035ed5b46cf28271b9962d446947f428c38a: Status 404 returned error can't find the container with id 369282bf4385da505931e707f888035ed5b46cf28271b9962d446947f428c38a Dec 03 11:49:44 crc kubenswrapper[5052]: I1203 11:49:44.751199 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-88667bdf5-ndf84" event={"ID":"5d0169e1-4f74-4f6e-89cc-70d2f79c34e2","Type":"ContainerStarted","Data":"369282bf4385da505931e707f888035ed5b46cf28271b9962d446947f428c38a"} Dec 03 11:49:45 crc kubenswrapper[5052]: I1203 11:49:45.764546 5052 generic.go:334] "Generic (PLEG): container finished" podID="5d0169e1-4f74-4f6e-89cc-70d2f79c34e2" containerID="ec1fb887baa43947f24b5957fd0b17eec560f3a84fa381dab76beddac96dcb20" exitCode=0 Dec 03 11:49:45 crc kubenswrapper[5052]: I1203 11:49:45.764868 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-88667bdf5-ndf84" event={"ID":"5d0169e1-4f74-4f6e-89cc-70d2f79c34e2","Type":"ContainerDied","Data":"ec1fb887baa43947f24b5957fd0b17eec560f3a84fa381dab76beddac96dcb20"} Dec 03 11:49:46 crc kubenswrapper[5052]: I1203 11:49:46.777189 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-88667bdf5-ndf84" event={"ID":"5d0169e1-4f74-4f6e-89cc-70d2f79c34e2","Type":"ContainerStarted","Data":"5941b7d31e38a458b2b2f04a347d19ebe8ee3aa64cc1993b16459efd04d966e0"} Dec 03 11:49:46 crc kubenswrapper[5052]: I1203 11:49:46.777793 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-88667bdf5-ndf84" Dec 03 11:49:46 crc kubenswrapper[5052]: I1203 11:49:46.820899 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-88667bdf5-ndf84" podStartSLOduration=3.820871441 podStartE2EDuration="3.820871441s" podCreationTimestamp="2025-12-03 11:49:43 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:49:46.806619272 +0000 UTC m=+1777.394861036" watchObservedRunningTime="2025-12-03 11:49:46.820871441 +0000 UTC m=+1777.409113215" Dec 03 11:49:47 crc kubenswrapper[5052]: I1203 11:49:47.000051 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 11:49:49 crc kubenswrapper[5052]: I1203 11:49:49.680847 5052 scope.go:117] "RemoveContainer" containerID="25fe88f8d102f09983f8884e65f0c6c1575ade67921f10a630e6dcc3ea8b506e" Dec 03 11:49:49 crc kubenswrapper[5052]: E1203 11:49:49.681847 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 11:49:54 crc kubenswrapper[5052]: I1203 11:49:54.222698 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-88667bdf5-ndf84" Dec 03 11:49:54 crc kubenswrapper[5052]: I1203 11:49:54.274657 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-lfl6r"] Dec 03 11:49:54 crc kubenswrapper[5052]: I1203 11:49:54.274925 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-lfl6r" podUID="93fb4fd0-aa18-4a2d-bfb4-b65bbc378ae5" containerName="dnsmasq-dns" containerID="cri-o://ea6a0525e4279380cf0ca84429fe6bc0012e5082feedb56ba078f355d1fafc6d" gracePeriod=10 Dec 03 11:49:54 crc kubenswrapper[5052]: I1203 11:49:54.687587 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-lfl6r" Dec 03 11:49:54 crc kubenswrapper[5052]: I1203 11:49:54.714048 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/93fb4fd0-aa18-4a2d-bfb4-b65bbc378ae5-dnsmasq-svc\") pod \"93fb4fd0-aa18-4a2d-bfb4-b65bbc378ae5\" (UID: \"93fb4fd0-aa18-4a2d-bfb4-b65bbc378ae5\") " Dec 03 11:49:54 crc kubenswrapper[5052]: I1203 11:49:54.714092 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hht6w\" (UniqueName: \"kubernetes.io/projected/93fb4fd0-aa18-4a2d-bfb4-b65bbc378ae5-kube-api-access-hht6w\") pod \"93fb4fd0-aa18-4a2d-bfb4-b65bbc378ae5\" (UID: \"93fb4fd0-aa18-4a2d-bfb4-b65bbc378ae5\") " Dec 03 11:49:54 crc kubenswrapper[5052]: I1203 11:49:54.714117 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/93fb4fd0-aa18-4a2d-bfb4-b65bbc378ae5-config\") pod \"93fb4fd0-aa18-4a2d-bfb4-b65bbc378ae5\" (UID: \"93fb4fd0-aa18-4a2d-bfb4-b65bbc378ae5\") " Dec 03 11:49:54 crc kubenswrapper[5052]: I1203 11:49:54.719396 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/93fb4fd0-aa18-4a2d-bfb4-b65bbc378ae5-kube-api-access-hht6w" (OuterVolumeSpecName: "kube-api-access-hht6w") pod "93fb4fd0-aa18-4a2d-bfb4-b65bbc378ae5" (UID: "93fb4fd0-aa18-4a2d-bfb4-b65bbc378ae5"). InnerVolumeSpecName "kube-api-access-hht6w". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:49:54 crc kubenswrapper[5052]: I1203 11:49:54.758914 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/93fb4fd0-aa18-4a2d-bfb4-b65bbc378ae5-config" (OuterVolumeSpecName: "config") pod "93fb4fd0-aa18-4a2d-bfb4-b65bbc378ae5" (UID: "93fb4fd0-aa18-4a2d-bfb4-b65bbc378ae5"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:49:54 crc kubenswrapper[5052]: I1203 11:49:54.775648 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/93fb4fd0-aa18-4a2d-bfb4-b65bbc378ae5-dnsmasq-svc" (OuterVolumeSpecName: "dnsmasq-svc") pod "93fb4fd0-aa18-4a2d-bfb4-b65bbc378ae5" (UID: "93fb4fd0-aa18-4a2d-bfb4-b65bbc378ae5"). InnerVolumeSpecName "dnsmasq-svc". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:49:54 crc kubenswrapper[5052]: I1203 11:49:54.815570 5052 reconciler_common.go:293] "Volume detached for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/93fb4fd0-aa18-4a2d-bfb4-b65bbc378ae5-dnsmasq-svc\") on node \"crc\" DevicePath \"\"" Dec 03 11:49:54 crc kubenswrapper[5052]: I1203 11:49:54.815597 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-hht6w\" (UniqueName: \"kubernetes.io/projected/93fb4fd0-aa18-4a2d-bfb4-b65bbc378ae5-kube-api-access-hht6w\") on node \"crc\" DevicePath \"\"" Dec 03 11:49:54 crc kubenswrapper[5052]: I1203 11:49:54.815608 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/93fb4fd0-aa18-4a2d-bfb4-b65bbc378ae5-config\") on node \"crc\" DevicePath \"\"" Dec 03 11:49:54 crc kubenswrapper[5052]: I1203 11:49:54.859486 5052 generic.go:334] "Generic (PLEG): container finished" podID="93fb4fd0-aa18-4a2d-bfb4-b65bbc378ae5" containerID="ea6a0525e4279380cf0ca84429fe6bc0012e5082feedb56ba078f355d1fafc6d" exitCode=0 Dec 03 11:49:54 crc kubenswrapper[5052]: I1203 11:49:54.859575 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-lfl6r" event={"ID":"93fb4fd0-aa18-4a2d-bfb4-b65bbc378ae5","Type":"ContainerDied","Data":"ea6a0525e4279380cf0ca84429fe6bc0012e5082feedb56ba078f355d1fafc6d"} Dec 03 11:49:54 crc kubenswrapper[5052]: I1203 11:49:54.859613 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-lfl6r" event={"ID":"93fb4fd0-aa18-4a2d-bfb4-b65bbc378ae5","Type":"ContainerDied","Data":"395380eced40d52322ffade7efa7a58446308db3cecd621bc7773f272b9c84e0"} Dec 03 11:49:54 crc kubenswrapper[5052]: I1203 11:49:54.859642 5052 scope.go:117] "RemoveContainer" containerID="ea6a0525e4279380cf0ca84429fe6bc0012e5082feedb56ba078f355d1fafc6d" Dec 03 11:49:54 crc kubenswrapper[5052]: I1203 11:49:54.859845 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-lfl6r" Dec 03 11:49:54 crc kubenswrapper[5052]: I1203 11:49:54.893493 5052 scope.go:117] "RemoveContainer" containerID="a278a573e396cd799f85833fa26f6bb09544244a8956130e15e5c8b299a9a039" Dec 03 11:49:54 crc kubenswrapper[5052]: I1203 11:49:54.913140 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-lfl6r"] Dec 03 11:49:54 crc kubenswrapper[5052]: I1203 11:49:54.919397 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-lfl6r"] Dec 03 11:49:54 crc kubenswrapper[5052]: I1203 11:49:54.944108 5052 scope.go:117] "RemoveContainer" containerID="ea6a0525e4279380cf0ca84429fe6bc0012e5082feedb56ba078f355d1fafc6d" Dec 03 11:49:54 crc kubenswrapper[5052]: E1203 11:49:54.944591 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ea6a0525e4279380cf0ca84429fe6bc0012e5082feedb56ba078f355d1fafc6d\": container with ID starting with ea6a0525e4279380cf0ca84429fe6bc0012e5082feedb56ba078f355d1fafc6d not found: ID does not exist" containerID="ea6a0525e4279380cf0ca84429fe6bc0012e5082feedb56ba078f355d1fafc6d" Dec 03 11:49:54 crc kubenswrapper[5052]: I1203 11:49:54.944631 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ea6a0525e4279380cf0ca84429fe6bc0012e5082feedb56ba078f355d1fafc6d"} err="failed to get container status \"ea6a0525e4279380cf0ca84429fe6bc0012e5082feedb56ba078f355d1fafc6d\": rpc error: code = NotFound desc = could not find container \"ea6a0525e4279380cf0ca84429fe6bc0012e5082feedb56ba078f355d1fafc6d\": container with ID starting with ea6a0525e4279380cf0ca84429fe6bc0012e5082feedb56ba078f355d1fafc6d not found: ID does not exist" Dec 03 11:49:54 crc kubenswrapper[5052]: I1203 11:49:54.944655 5052 scope.go:117] "RemoveContainer" containerID="a278a573e396cd799f85833fa26f6bb09544244a8956130e15e5c8b299a9a039" Dec 03 11:49:54 crc kubenswrapper[5052]: E1203 11:49:54.945251 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a278a573e396cd799f85833fa26f6bb09544244a8956130e15e5c8b299a9a039\": container with ID starting with a278a573e396cd799f85833fa26f6bb09544244a8956130e15e5c8b299a9a039 not found: ID does not exist" containerID="a278a573e396cd799f85833fa26f6bb09544244a8956130e15e5c8b299a9a039" Dec 03 11:49:54 crc kubenswrapper[5052]: I1203 11:49:54.945273 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a278a573e396cd799f85833fa26f6bb09544244a8956130e15e5c8b299a9a039"} err="failed to get container status \"a278a573e396cd799f85833fa26f6bb09544244a8956130e15e5c8b299a9a039\": rpc error: code = NotFound desc = could not find container \"a278a573e396cd799f85833fa26f6bb09544244a8956130e15e5c8b299a9a039\": container with ID starting with a278a573e396cd799f85833fa26f6bb09544244a8956130e15e5c8b299a9a039 not found: ID does not exist" Dec 03 11:49:55 crc kubenswrapper[5052]: I1203 11:49:55.681930 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="93fb4fd0-aa18-4a2d-bfb4-b65bbc378ae5" path="/var/lib/kubelet/pods/93fb4fd0-aa18-4a2d-bfb4-b65bbc378ae5/volumes" Dec 03 11:49:57 crc kubenswrapper[5052]: I1203 11:49:57.002720 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 11:49:57 crc kubenswrapper[5052]: I1203 11:49:57.283534 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/cinder-db-create-kddz8"] Dec 03 11:49:57 crc kubenswrapper[5052]: E1203 11:49:57.284194 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="93fb4fd0-aa18-4a2d-bfb4-b65bbc378ae5" containerName="dnsmasq-dns" Dec 03 11:49:57 crc kubenswrapper[5052]: I1203 11:49:57.284209 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="93fb4fd0-aa18-4a2d-bfb4-b65bbc378ae5" containerName="dnsmasq-dns" Dec 03 11:49:57 crc kubenswrapper[5052]: E1203 11:49:57.284227 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="93fb4fd0-aa18-4a2d-bfb4-b65bbc378ae5" containerName="init" Dec 03 11:49:57 crc kubenswrapper[5052]: I1203 11:49:57.284234 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="93fb4fd0-aa18-4a2d-bfb4-b65bbc378ae5" containerName="init" Dec 03 11:49:57 crc kubenswrapper[5052]: I1203 11:49:57.284394 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="93fb4fd0-aa18-4a2d-bfb4-b65bbc378ae5" containerName="dnsmasq-dns" Dec 03 11:49:57 crc kubenswrapper[5052]: I1203 11:49:57.284952 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-db-create-kddz8" Dec 03 11:49:57 crc kubenswrapper[5052]: I1203 11:49:57.308105 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-db-create-kddz8"] Dec 03 11:49:57 crc kubenswrapper[5052]: I1203 11:49:57.355421 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/3f0bc56f-8f7f-4ebf-94f2-e02dc2edd717-operator-scripts\") pod \"cinder-db-create-kddz8\" (UID: \"3f0bc56f-8f7f-4ebf-94f2-e02dc2edd717\") " pod="openstack-kuttl-tests/cinder-db-create-kddz8" Dec 03 11:49:57 crc kubenswrapper[5052]: I1203 11:49:57.355577 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nb47c\" (UniqueName: \"kubernetes.io/projected/3f0bc56f-8f7f-4ebf-94f2-e02dc2edd717-kube-api-access-nb47c\") pod \"cinder-db-create-kddz8\" (UID: \"3f0bc56f-8f7f-4ebf-94f2-e02dc2edd717\") " pod="openstack-kuttl-tests/cinder-db-create-kddz8" Dec 03 11:49:57 crc kubenswrapper[5052]: I1203 11:49:57.373737 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 11:49:57 crc kubenswrapper[5052]: I1203 11:49:57.388812 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/barbican-db-create-rzt98"] Dec 03 11:49:57 crc kubenswrapper[5052]: I1203 11:49:57.390092 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-db-create-rzt98" Dec 03 11:49:57 crc kubenswrapper[5052]: I1203 11:49:57.398079 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/barbican-4385-account-create-update-52gf9"] Dec 03 11:49:57 crc kubenswrapper[5052]: I1203 11:49:57.399229 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-4385-account-create-update-52gf9" Dec 03 11:49:57 crc kubenswrapper[5052]: I1203 11:49:57.402663 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"barbican-db-secret" Dec 03 11:49:57 crc kubenswrapper[5052]: I1203 11:49:57.415318 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-db-create-rzt98"] Dec 03 11:49:57 crc kubenswrapper[5052]: I1203 11:49:57.433934 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-4385-account-create-update-52gf9"] Dec 03 11:49:57 crc kubenswrapper[5052]: I1203 11:49:57.457023 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-79tlr\" (UniqueName: \"kubernetes.io/projected/84a261ec-41e5-487b-b5a1-3743d98c7a76-kube-api-access-79tlr\") pod \"barbican-db-create-rzt98\" (UID: \"84a261ec-41e5-487b-b5a1-3743d98c7a76\") " pod="openstack-kuttl-tests/barbican-db-create-rzt98" Dec 03 11:49:57 crc kubenswrapper[5052]: I1203 11:49:57.457076 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/3f0bc56f-8f7f-4ebf-94f2-e02dc2edd717-operator-scripts\") pod \"cinder-db-create-kddz8\" (UID: \"3f0bc56f-8f7f-4ebf-94f2-e02dc2edd717\") " pod="openstack-kuttl-tests/cinder-db-create-kddz8" Dec 03 11:49:57 crc kubenswrapper[5052]: I1203 11:49:57.457136 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/896b9690-42d0-4528-886d-e2e4984f1aee-operator-scripts\") pod \"barbican-4385-account-create-update-52gf9\" (UID: \"896b9690-42d0-4528-886d-e2e4984f1aee\") " pod="openstack-kuttl-tests/barbican-4385-account-create-update-52gf9" Dec 03 11:49:57 crc kubenswrapper[5052]: I1203 11:49:57.457219 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zsrtf\" (UniqueName: \"kubernetes.io/projected/896b9690-42d0-4528-886d-e2e4984f1aee-kube-api-access-zsrtf\") pod \"barbican-4385-account-create-update-52gf9\" (UID: \"896b9690-42d0-4528-886d-e2e4984f1aee\") " pod="openstack-kuttl-tests/barbican-4385-account-create-update-52gf9" Dec 03 11:49:57 crc kubenswrapper[5052]: I1203 11:49:57.457271 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-nb47c\" (UniqueName: \"kubernetes.io/projected/3f0bc56f-8f7f-4ebf-94f2-e02dc2edd717-kube-api-access-nb47c\") pod \"cinder-db-create-kddz8\" (UID: \"3f0bc56f-8f7f-4ebf-94f2-e02dc2edd717\") " pod="openstack-kuttl-tests/cinder-db-create-kddz8" Dec 03 11:49:57 crc kubenswrapper[5052]: I1203 11:49:57.457294 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/84a261ec-41e5-487b-b5a1-3743d98c7a76-operator-scripts\") pod \"barbican-db-create-rzt98\" (UID: \"84a261ec-41e5-487b-b5a1-3743d98c7a76\") " pod="openstack-kuttl-tests/barbican-db-create-rzt98" Dec 03 11:49:57 crc kubenswrapper[5052]: I1203 11:49:57.458703 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/3f0bc56f-8f7f-4ebf-94f2-e02dc2edd717-operator-scripts\") pod \"cinder-db-create-kddz8\" (UID: \"3f0bc56f-8f7f-4ebf-94f2-e02dc2edd717\") " pod="openstack-kuttl-tests/cinder-db-create-kddz8" Dec 03 11:49:57 crc kubenswrapper[5052]: I1203 11:49:57.485533 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-nb47c\" (UniqueName: \"kubernetes.io/projected/3f0bc56f-8f7f-4ebf-94f2-e02dc2edd717-kube-api-access-nb47c\") pod \"cinder-db-create-kddz8\" (UID: \"3f0bc56f-8f7f-4ebf-94f2-e02dc2edd717\") " pod="openstack-kuttl-tests/cinder-db-create-kddz8" Dec 03 11:49:57 crc kubenswrapper[5052]: I1203 11:49:57.512459 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/cinder-6fd6-account-create-update-sz9zj"] Dec 03 11:49:57 crc kubenswrapper[5052]: I1203 11:49:57.517404 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-6fd6-account-create-update-sz9zj" Dec 03 11:49:57 crc kubenswrapper[5052]: I1203 11:49:57.520040 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cinder-db-secret" Dec 03 11:49:57 crc kubenswrapper[5052]: I1203 11:49:57.547346 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-6fd6-account-create-update-sz9zj"] Dec 03 11:49:57 crc kubenswrapper[5052]: I1203 11:49:57.559429 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/896b9690-42d0-4528-886d-e2e4984f1aee-operator-scripts\") pod \"barbican-4385-account-create-update-52gf9\" (UID: \"896b9690-42d0-4528-886d-e2e4984f1aee\") " pod="openstack-kuttl-tests/barbican-4385-account-create-update-52gf9" Dec 03 11:49:57 crc kubenswrapper[5052]: I1203 11:49:57.559485 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/d815bd05-9b18-4ef8-ab32-60bee21bf881-operator-scripts\") pod \"cinder-6fd6-account-create-update-sz9zj\" (UID: \"d815bd05-9b18-4ef8-ab32-60bee21bf881\") " pod="openstack-kuttl-tests/cinder-6fd6-account-create-update-sz9zj" Dec 03 11:49:57 crc kubenswrapper[5052]: I1203 11:49:57.559555 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-zsrtf\" (UniqueName: \"kubernetes.io/projected/896b9690-42d0-4528-886d-e2e4984f1aee-kube-api-access-zsrtf\") pod \"barbican-4385-account-create-update-52gf9\" (UID: \"896b9690-42d0-4528-886d-e2e4984f1aee\") " pod="openstack-kuttl-tests/barbican-4385-account-create-update-52gf9" Dec 03 11:49:57 crc kubenswrapper[5052]: I1203 11:49:57.559596 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6nl86\" (UniqueName: \"kubernetes.io/projected/d815bd05-9b18-4ef8-ab32-60bee21bf881-kube-api-access-6nl86\") pod \"cinder-6fd6-account-create-update-sz9zj\" (UID: \"d815bd05-9b18-4ef8-ab32-60bee21bf881\") " pod="openstack-kuttl-tests/cinder-6fd6-account-create-update-sz9zj" Dec 03 11:49:57 crc kubenswrapper[5052]: I1203 11:49:57.559629 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/84a261ec-41e5-487b-b5a1-3743d98c7a76-operator-scripts\") pod \"barbican-db-create-rzt98\" (UID: \"84a261ec-41e5-487b-b5a1-3743d98c7a76\") " pod="openstack-kuttl-tests/barbican-db-create-rzt98" Dec 03 11:49:57 crc kubenswrapper[5052]: I1203 11:49:57.559738 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-79tlr\" (UniqueName: \"kubernetes.io/projected/84a261ec-41e5-487b-b5a1-3743d98c7a76-kube-api-access-79tlr\") pod \"barbican-db-create-rzt98\" (UID: \"84a261ec-41e5-487b-b5a1-3743d98c7a76\") " pod="openstack-kuttl-tests/barbican-db-create-rzt98" Dec 03 11:49:57 crc kubenswrapper[5052]: I1203 11:49:57.560906 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/896b9690-42d0-4528-886d-e2e4984f1aee-operator-scripts\") pod \"barbican-4385-account-create-update-52gf9\" (UID: \"896b9690-42d0-4528-886d-e2e4984f1aee\") " pod="openstack-kuttl-tests/barbican-4385-account-create-update-52gf9" Dec 03 11:49:57 crc kubenswrapper[5052]: I1203 11:49:57.561472 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/84a261ec-41e5-487b-b5a1-3743d98c7a76-operator-scripts\") pod \"barbican-db-create-rzt98\" (UID: \"84a261ec-41e5-487b-b5a1-3743d98c7a76\") " pod="openstack-kuttl-tests/barbican-db-create-rzt98" Dec 03 11:49:57 crc kubenswrapper[5052]: I1203 11:49:57.586624 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-79tlr\" (UniqueName: \"kubernetes.io/projected/84a261ec-41e5-487b-b5a1-3743d98c7a76-kube-api-access-79tlr\") pod \"barbican-db-create-rzt98\" (UID: \"84a261ec-41e5-487b-b5a1-3743d98c7a76\") " pod="openstack-kuttl-tests/barbican-db-create-rzt98" Dec 03 11:49:57 crc kubenswrapper[5052]: I1203 11:49:57.587760 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-zsrtf\" (UniqueName: \"kubernetes.io/projected/896b9690-42d0-4528-886d-e2e4984f1aee-kube-api-access-zsrtf\") pod \"barbican-4385-account-create-update-52gf9\" (UID: \"896b9690-42d0-4528-886d-e2e4984f1aee\") " pod="openstack-kuttl-tests/barbican-4385-account-create-update-52gf9" Dec 03 11:49:57 crc kubenswrapper[5052]: I1203 11:49:57.601023 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-db-create-kddz8" Dec 03 11:49:57 crc kubenswrapper[5052]: I1203 11:49:57.602106 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/neutron-db-create-lfrbv"] Dec 03 11:49:57 crc kubenswrapper[5052]: I1203 11:49:57.603685 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-db-create-lfrbv" Dec 03 11:49:57 crc kubenswrapper[5052]: I1203 11:49:57.630682 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-db-create-lfrbv"] Dec 03 11:49:57 crc kubenswrapper[5052]: I1203 11:49:57.663454 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/760bae42-69a5-4a98-8a7d-5f1fd7143591-operator-scripts\") pod \"neutron-db-create-lfrbv\" (UID: \"760bae42-69a5-4a98-8a7d-5f1fd7143591\") " pod="openstack-kuttl-tests/neutron-db-create-lfrbv" Dec 03 11:49:57 crc kubenswrapper[5052]: I1203 11:49:57.663538 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-6nl86\" (UniqueName: \"kubernetes.io/projected/d815bd05-9b18-4ef8-ab32-60bee21bf881-kube-api-access-6nl86\") pod \"cinder-6fd6-account-create-update-sz9zj\" (UID: \"d815bd05-9b18-4ef8-ab32-60bee21bf881\") " pod="openstack-kuttl-tests/cinder-6fd6-account-create-update-sz9zj" Dec 03 11:49:57 crc kubenswrapper[5052]: I1203 11:49:57.663617 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dbshc\" (UniqueName: \"kubernetes.io/projected/760bae42-69a5-4a98-8a7d-5f1fd7143591-kube-api-access-dbshc\") pod \"neutron-db-create-lfrbv\" (UID: \"760bae42-69a5-4a98-8a7d-5f1fd7143591\") " pod="openstack-kuttl-tests/neutron-db-create-lfrbv" Dec 03 11:49:57 crc kubenswrapper[5052]: I1203 11:49:57.663707 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/d815bd05-9b18-4ef8-ab32-60bee21bf881-operator-scripts\") pod \"cinder-6fd6-account-create-update-sz9zj\" (UID: \"d815bd05-9b18-4ef8-ab32-60bee21bf881\") " pod="openstack-kuttl-tests/cinder-6fd6-account-create-update-sz9zj" Dec 03 11:49:57 crc kubenswrapper[5052]: I1203 11:49:57.668124 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/d815bd05-9b18-4ef8-ab32-60bee21bf881-operator-scripts\") pod \"cinder-6fd6-account-create-update-sz9zj\" (UID: \"d815bd05-9b18-4ef8-ab32-60bee21bf881\") " pod="openstack-kuttl-tests/cinder-6fd6-account-create-update-sz9zj" Dec 03 11:49:57 crc kubenswrapper[5052]: I1203 11:49:57.690164 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-6nl86\" (UniqueName: \"kubernetes.io/projected/d815bd05-9b18-4ef8-ab32-60bee21bf881-kube-api-access-6nl86\") pod \"cinder-6fd6-account-create-update-sz9zj\" (UID: \"d815bd05-9b18-4ef8-ab32-60bee21bf881\") " pod="openstack-kuttl-tests/cinder-6fd6-account-create-update-sz9zj" Dec 03 11:49:57 crc kubenswrapper[5052]: I1203 11:49:57.702356 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/neutron-3cf1-account-create-update-96n2n"] Dec 03 11:49:57 crc kubenswrapper[5052]: I1203 11:49:57.703640 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-3cf1-account-create-update-96n2n" Dec 03 11:49:57 crc kubenswrapper[5052]: I1203 11:49:57.706697 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"neutron-db-secret" Dec 03 11:49:57 crc kubenswrapper[5052]: I1203 11:49:57.706959 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-db-create-rzt98" Dec 03 11:49:57 crc kubenswrapper[5052]: I1203 11:49:57.713613 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-3cf1-account-create-update-96n2n"] Dec 03 11:49:57 crc kubenswrapper[5052]: I1203 11:49:57.720489 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-4385-account-create-update-52gf9" Dec 03 11:49:57 crc kubenswrapper[5052]: I1203 11:49:57.789850 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-dbshc\" (UniqueName: \"kubernetes.io/projected/760bae42-69a5-4a98-8a7d-5f1fd7143591-kube-api-access-dbshc\") pod \"neutron-db-create-lfrbv\" (UID: \"760bae42-69a5-4a98-8a7d-5f1fd7143591\") " pod="openstack-kuttl-tests/neutron-db-create-lfrbv" Dec 03 11:49:57 crc kubenswrapper[5052]: I1203 11:49:57.789929 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/e7c839dc-afd1-4cd2-a6a7-a24a5f716f13-operator-scripts\") pod \"neutron-3cf1-account-create-update-96n2n\" (UID: \"e7c839dc-afd1-4cd2-a6a7-a24a5f716f13\") " pod="openstack-kuttl-tests/neutron-3cf1-account-create-update-96n2n" Dec 03 11:49:57 crc kubenswrapper[5052]: I1203 11:49:57.790081 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-284d8\" (UniqueName: \"kubernetes.io/projected/e7c839dc-afd1-4cd2-a6a7-a24a5f716f13-kube-api-access-284d8\") pod \"neutron-3cf1-account-create-update-96n2n\" (UID: \"e7c839dc-afd1-4cd2-a6a7-a24a5f716f13\") " pod="openstack-kuttl-tests/neutron-3cf1-account-create-update-96n2n" Dec 03 11:49:57 crc kubenswrapper[5052]: I1203 11:49:57.790324 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/760bae42-69a5-4a98-8a7d-5f1fd7143591-operator-scripts\") pod \"neutron-db-create-lfrbv\" (UID: \"760bae42-69a5-4a98-8a7d-5f1fd7143591\") " pod="openstack-kuttl-tests/neutron-db-create-lfrbv" Dec 03 11:49:57 crc kubenswrapper[5052]: I1203 11:49:57.791553 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/760bae42-69a5-4a98-8a7d-5f1fd7143591-operator-scripts\") pod \"neutron-db-create-lfrbv\" (UID: \"760bae42-69a5-4a98-8a7d-5f1fd7143591\") " pod="openstack-kuttl-tests/neutron-db-create-lfrbv" Dec 03 11:49:57 crc kubenswrapper[5052]: I1203 11:49:57.795341 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/keystone-db-sync-l74fc"] Dec 03 11:49:57 crc kubenswrapper[5052]: I1203 11:49:57.801971 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-db-sync-l74fc" Dec 03 11:49:57 crc kubenswrapper[5052]: I1203 11:49:57.808198 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone-scripts" Dec 03 11:49:57 crc kubenswrapper[5052]: I1203 11:49:57.808833 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone-keystone-dockercfg-xtl6f" Dec 03 11:49:57 crc kubenswrapper[5052]: I1203 11:49:57.809875 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-dbshc\" (UniqueName: \"kubernetes.io/projected/760bae42-69a5-4a98-8a7d-5f1fd7143591-kube-api-access-dbshc\") pod \"neutron-db-create-lfrbv\" (UID: \"760bae42-69a5-4a98-8a7d-5f1fd7143591\") " pod="openstack-kuttl-tests/neutron-db-create-lfrbv" Dec 03 11:49:57 crc kubenswrapper[5052]: I1203 11:49:57.811264 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone" Dec 03 11:49:57 crc kubenswrapper[5052]: I1203 11:49:57.827274 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone-config-data" Dec 03 11:49:57 crc kubenswrapper[5052]: I1203 11:49:57.832250 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-db-sync-l74fc"] Dec 03 11:49:57 crc kubenswrapper[5052]: I1203 11:49:57.848471 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-6fd6-account-create-update-sz9zj" Dec 03 11:49:57 crc kubenswrapper[5052]: I1203 11:49:57.891587 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/e7c839dc-afd1-4cd2-a6a7-a24a5f716f13-operator-scripts\") pod \"neutron-3cf1-account-create-update-96n2n\" (UID: \"e7c839dc-afd1-4cd2-a6a7-a24a5f716f13\") " pod="openstack-kuttl-tests/neutron-3cf1-account-create-update-96n2n" Dec 03 11:49:57 crc kubenswrapper[5052]: I1203 11:49:57.891677 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-284d8\" (UniqueName: \"kubernetes.io/projected/e7c839dc-afd1-4cd2-a6a7-a24a5f716f13-kube-api-access-284d8\") pod \"neutron-3cf1-account-create-update-96n2n\" (UID: \"e7c839dc-afd1-4cd2-a6a7-a24a5f716f13\") " pod="openstack-kuttl-tests/neutron-3cf1-account-create-update-96n2n" Dec 03 11:49:57 crc kubenswrapper[5052]: I1203 11:49:57.892930 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/e7c839dc-afd1-4cd2-a6a7-a24a5f716f13-operator-scripts\") pod \"neutron-3cf1-account-create-update-96n2n\" (UID: \"e7c839dc-afd1-4cd2-a6a7-a24a5f716f13\") " pod="openstack-kuttl-tests/neutron-3cf1-account-create-update-96n2n" Dec 03 11:49:57 crc kubenswrapper[5052]: I1203 11:49:57.910144 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-284d8\" (UniqueName: \"kubernetes.io/projected/e7c839dc-afd1-4cd2-a6a7-a24a5f716f13-kube-api-access-284d8\") pod \"neutron-3cf1-account-create-update-96n2n\" (UID: \"e7c839dc-afd1-4cd2-a6a7-a24a5f716f13\") " pod="openstack-kuttl-tests/neutron-3cf1-account-create-update-96n2n" Dec 03 11:49:57 crc kubenswrapper[5052]: I1203 11:49:57.991307 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-db-create-kddz8"] Dec 03 11:49:57 crc kubenswrapper[5052]: I1203 11:49:57.993188 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/01498d4a-9b4a-4991-9bb0-a5c77f885672-combined-ca-bundle\") pod \"keystone-db-sync-l74fc\" (UID: \"01498d4a-9b4a-4991-9bb0-a5c77f885672\") " pod="openstack-kuttl-tests/keystone-db-sync-l74fc" Dec 03 11:49:57 crc kubenswrapper[5052]: I1203 11:49:57.993413 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dpstd\" (UniqueName: \"kubernetes.io/projected/01498d4a-9b4a-4991-9bb0-a5c77f885672-kube-api-access-dpstd\") pod \"keystone-db-sync-l74fc\" (UID: \"01498d4a-9b4a-4991-9bb0-a5c77f885672\") " pod="openstack-kuttl-tests/keystone-db-sync-l74fc" Dec 03 11:49:57 crc kubenswrapper[5052]: I1203 11:49:57.993598 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/01498d4a-9b4a-4991-9bb0-a5c77f885672-config-data\") pod \"keystone-db-sync-l74fc\" (UID: \"01498d4a-9b4a-4991-9bb0-a5c77f885672\") " pod="openstack-kuttl-tests/keystone-db-sync-l74fc" Dec 03 11:49:58 crc kubenswrapper[5052]: W1203 11:49:58.004104 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3f0bc56f_8f7f_4ebf_94f2_e02dc2edd717.slice/crio-244cc0709af85ef689661f7b34631a574dffe324eadb5d89b1a5288c16441e56 WatchSource:0}: Error finding container 244cc0709af85ef689661f7b34631a574dffe324eadb5d89b1a5288c16441e56: Status 404 returned error can't find the container with id 244cc0709af85ef689661f7b34631a574dffe324eadb5d89b1a5288c16441e56 Dec 03 11:49:58 crc kubenswrapper[5052]: I1203 11:49:58.057592 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-db-create-lfrbv" Dec 03 11:49:58 crc kubenswrapper[5052]: I1203 11:49:58.083863 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-3cf1-account-create-update-96n2n" Dec 03 11:49:58 crc kubenswrapper[5052]: I1203 11:49:58.096089 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-dpstd\" (UniqueName: \"kubernetes.io/projected/01498d4a-9b4a-4991-9bb0-a5c77f885672-kube-api-access-dpstd\") pod \"keystone-db-sync-l74fc\" (UID: \"01498d4a-9b4a-4991-9bb0-a5c77f885672\") " pod="openstack-kuttl-tests/keystone-db-sync-l74fc" Dec 03 11:49:58 crc kubenswrapper[5052]: I1203 11:49:58.096195 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/01498d4a-9b4a-4991-9bb0-a5c77f885672-config-data\") pod \"keystone-db-sync-l74fc\" (UID: \"01498d4a-9b4a-4991-9bb0-a5c77f885672\") " pod="openstack-kuttl-tests/keystone-db-sync-l74fc" Dec 03 11:49:58 crc kubenswrapper[5052]: I1203 11:49:58.096252 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/01498d4a-9b4a-4991-9bb0-a5c77f885672-combined-ca-bundle\") pod \"keystone-db-sync-l74fc\" (UID: \"01498d4a-9b4a-4991-9bb0-a5c77f885672\") " pod="openstack-kuttl-tests/keystone-db-sync-l74fc" Dec 03 11:49:58 crc kubenswrapper[5052]: I1203 11:49:58.105324 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/01498d4a-9b4a-4991-9bb0-a5c77f885672-config-data\") pod \"keystone-db-sync-l74fc\" (UID: \"01498d4a-9b4a-4991-9bb0-a5c77f885672\") " pod="openstack-kuttl-tests/keystone-db-sync-l74fc" Dec 03 11:49:58 crc kubenswrapper[5052]: I1203 11:49:58.105990 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/01498d4a-9b4a-4991-9bb0-a5c77f885672-combined-ca-bundle\") pod \"keystone-db-sync-l74fc\" (UID: \"01498d4a-9b4a-4991-9bb0-a5c77f885672\") " pod="openstack-kuttl-tests/keystone-db-sync-l74fc" Dec 03 11:49:58 crc kubenswrapper[5052]: I1203 11:49:58.117916 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-dpstd\" (UniqueName: \"kubernetes.io/projected/01498d4a-9b4a-4991-9bb0-a5c77f885672-kube-api-access-dpstd\") pod \"keystone-db-sync-l74fc\" (UID: \"01498d4a-9b4a-4991-9bb0-a5c77f885672\") " pod="openstack-kuttl-tests/keystone-db-sync-l74fc" Dec 03 11:49:58 crc kubenswrapper[5052]: I1203 11:49:58.138270 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-db-sync-l74fc" Dec 03 11:49:58 crc kubenswrapper[5052]: I1203 11:49:58.295904 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-db-create-rzt98"] Dec 03 11:49:58 crc kubenswrapper[5052]: W1203 11:49:58.306984 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod84a261ec_41e5_487b_b5a1_3743d98c7a76.slice/crio-9e144e85b5ab9abef569f4916096d7ab3da69a6b2c435c0ee7ad483f1bce5ec0 WatchSource:0}: Error finding container 9e144e85b5ab9abef569f4916096d7ab3da69a6b2c435c0ee7ad483f1bce5ec0: Status 404 returned error can't find the container with id 9e144e85b5ab9abef569f4916096d7ab3da69a6b2c435c0ee7ad483f1bce5ec0 Dec 03 11:49:58 crc kubenswrapper[5052]: I1203 11:49:58.379688 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-6fd6-account-create-update-sz9zj"] Dec 03 11:49:58 crc kubenswrapper[5052]: W1203 11:49:58.391981 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd815bd05_9b18_4ef8_ab32_60bee21bf881.slice/crio-6d178a3cf80582a2af7814a279023d610d4d7be0881be7a998f0e4f0b4520081 WatchSource:0}: Error finding container 6d178a3cf80582a2af7814a279023d610d4d7be0881be7a998f0e4f0b4520081: Status 404 returned error can't find the container with id 6d178a3cf80582a2af7814a279023d610d4d7be0881be7a998f0e4f0b4520081 Dec 03 11:49:58 crc kubenswrapper[5052]: I1203 11:49:58.394881 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-4385-account-create-update-52gf9"] Dec 03 11:49:58 crc kubenswrapper[5052]: W1203 11:49:58.396837 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod896b9690_42d0_4528_886d_e2e4984f1aee.slice/crio-c8c3ee26357f77ad06592bd8444aee33c0edba92dd078ce46c9a9bcfce96c823 WatchSource:0}: Error finding container c8c3ee26357f77ad06592bd8444aee33c0edba92dd078ce46c9a9bcfce96c823: Status 404 returned error can't find the container with id c8c3ee26357f77ad06592bd8444aee33c0edba92dd078ce46c9a9bcfce96c823 Dec 03 11:49:58 crc kubenswrapper[5052]: I1203 11:49:58.576829 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-db-create-lfrbv"] Dec 03 11:49:58 crc kubenswrapper[5052]: W1203 11:49:58.578585 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod760bae42_69a5_4a98_8a7d_5f1fd7143591.slice/crio-3a319f699dbb7668ddf14dc09b6cbe341445028ac405dc7ac49f03735a21b15a WatchSource:0}: Error finding container 3a319f699dbb7668ddf14dc09b6cbe341445028ac405dc7ac49f03735a21b15a: Status 404 returned error can't find the container with id 3a319f699dbb7668ddf14dc09b6cbe341445028ac405dc7ac49f03735a21b15a Dec 03 11:49:58 crc kubenswrapper[5052]: I1203 11:49:58.664798 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-3cf1-account-create-update-96n2n"] Dec 03 11:49:58 crc kubenswrapper[5052]: W1203 11:49:58.673372 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode7c839dc_afd1_4cd2_a6a7_a24a5f716f13.slice/crio-cda1fc2ef75d4519ea906425fdb30e169fb9053da846251d85abf3eabac2dcc1 WatchSource:0}: Error finding container cda1fc2ef75d4519ea906425fdb30e169fb9053da846251d85abf3eabac2dcc1: Status 404 returned error can't find the container with id cda1fc2ef75d4519ea906425fdb30e169fb9053da846251d85abf3eabac2dcc1 Dec 03 11:49:58 crc kubenswrapper[5052]: W1203 11:49:58.684879 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod01498d4a_9b4a_4991_9bb0_a5c77f885672.slice/crio-13fde770f4f95f27ef99617c31d93e7471715ef6f4f8058ac1acee39ddeeb328 WatchSource:0}: Error finding container 13fde770f4f95f27ef99617c31d93e7471715ef6f4f8058ac1acee39ddeeb328: Status 404 returned error can't find the container with id 13fde770f4f95f27ef99617c31d93e7471715ef6f4f8058ac1acee39ddeeb328 Dec 03 11:49:58 crc kubenswrapper[5052]: I1203 11:49:58.686068 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-db-sync-l74fc"] Dec 03 11:49:58 crc kubenswrapper[5052]: I1203 11:49:58.915882 5052 generic.go:334] "Generic (PLEG): container finished" podID="3f0bc56f-8f7f-4ebf-94f2-e02dc2edd717" containerID="f66d133226f04488b65271e680367fcbcbdfdc8cbbafb3a1d5494736b7c5cfb3" exitCode=0 Dec 03 11:49:58 crc kubenswrapper[5052]: I1203 11:49:58.915976 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-db-create-kddz8" event={"ID":"3f0bc56f-8f7f-4ebf-94f2-e02dc2edd717","Type":"ContainerDied","Data":"f66d133226f04488b65271e680367fcbcbdfdc8cbbafb3a1d5494736b7c5cfb3"} Dec 03 11:49:58 crc kubenswrapper[5052]: I1203 11:49:58.916199 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-db-create-kddz8" event={"ID":"3f0bc56f-8f7f-4ebf-94f2-e02dc2edd717","Type":"ContainerStarted","Data":"244cc0709af85ef689661f7b34631a574dffe324eadb5d89b1a5288c16441e56"} Dec 03 11:49:58 crc kubenswrapper[5052]: I1203 11:49:58.917682 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-3cf1-account-create-update-96n2n" event={"ID":"e7c839dc-afd1-4cd2-a6a7-a24a5f716f13","Type":"ContainerStarted","Data":"cda1fc2ef75d4519ea906425fdb30e169fb9053da846251d85abf3eabac2dcc1"} Dec 03 11:49:58 crc kubenswrapper[5052]: I1203 11:49:58.919086 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-4385-account-create-update-52gf9" event={"ID":"896b9690-42d0-4528-886d-e2e4984f1aee","Type":"ContainerStarted","Data":"8393f7f6989898dc3fee2162e87f42b6979e4c1290368f41d1aa48d371d31314"} Dec 03 11:49:58 crc kubenswrapper[5052]: I1203 11:49:58.919116 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-4385-account-create-update-52gf9" event={"ID":"896b9690-42d0-4528-886d-e2e4984f1aee","Type":"ContainerStarted","Data":"c8c3ee26357f77ad06592bd8444aee33c0edba92dd078ce46c9a9bcfce96c823"} Dec 03 11:49:58 crc kubenswrapper[5052]: I1203 11:49:58.920142 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-db-sync-l74fc" event={"ID":"01498d4a-9b4a-4991-9bb0-a5c77f885672","Type":"ContainerStarted","Data":"13fde770f4f95f27ef99617c31d93e7471715ef6f4f8058ac1acee39ddeeb328"} Dec 03 11:49:58 crc kubenswrapper[5052]: I1203 11:49:58.921516 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-6fd6-account-create-update-sz9zj" event={"ID":"d815bd05-9b18-4ef8-ab32-60bee21bf881","Type":"ContainerStarted","Data":"aaa2d28c5e3338722d4f5eaedeeb8a30c10f86ffe40a62d737b912ad57d3b9b1"} Dec 03 11:49:58 crc kubenswrapper[5052]: I1203 11:49:58.921543 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-6fd6-account-create-update-sz9zj" event={"ID":"d815bd05-9b18-4ef8-ab32-60bee21bf881","Type":"ContainerStarted","Data":"6d178a3cf80582a2af7814a279023d610d4d7be0881be7a998f0e4f0b4520081"} Dec 03 11:49:58 crc kubenswrapper[5052]: I1203 11:49:58.922694 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-db-create-lfrbv" event={"ID":"760bae42-69a5-4a98-8a7d-5f1fd7143591","Type":"ContainerStarted","Data":"3a319f699dbb7668ddf14dc09b6cbe341445028ac405dc7ac49f03735a21b15a"} Dec 03 11:49:58 crc kubenswrapper[5052]: I1203 11:49:58.924074 5052 generic.go:334] "Generic (PLEG): container finished" podID="84a261ec-41e5-487b-b5a1-3743d98c7a76" containerID="87e11068f85c53dfc10a1bce0f37811f9e183ed6ffd052218a764a795d3ec700" exitCode=0 Dec 03 11:49:58 crc kubenswrapper[5052]: I1203 11:49:58.924103 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-db-create-rzt98" event={"ID":"84a261ec-41e5-487b-b5a1-3743d98c7a76","Type":"ContainerDied","Data":"87e11068f85c53dfc10a1bce0f37811f9e183ed6ffd052218a764a795d3ec700"} Dec 03 11:49:58 crc kubenswrapper[5052]: I1203 11:49:58.924120 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-db-create-rzt98" event={"ID":"84a261ec-41e5-487b-b5a1-3743d98c7a76","Type":"ContainerStarted","Data":"9e144e85b5ab9abef569f4916096d7ab3da69a6b2c435c0ee7ad483f1bce5ec0"} Dec 03 11:49:58 crc kubenswrapper[5052]: I1203 11:49:58.964239 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/barbican-4385-account-create-update-52gf9" podStartSLOduration=1.964222739 podStartE2EDuration="1.964222739s" podCreationTimestamp="2025-12-03 11:49:57 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:49:58.958722131 +0000 UTC m=+1789.546963895" watchObservedRunningTime="2025-12-03 11:49:58.964222739 +0000 UTC m=+1789.552464503" Dec 03 11:49:59 crc kubenswrapper[5052]: I1203 11:49:59.024009 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/cinder-6fd6-account-create-update-sz9zj" podStartSLOduration=2.023992713 podStartE2EDuration="2.023992713s" podCreationTimestamp="2025-12-03 11:49:57 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:49:59.022534671 +0000 UTC m=+1789.610776435" watchObservedRunningTime="2025-12-03 11:49:59.023992713 +0000 UTC m=+1789.612234477" Dec 03 11:49:59 crc kubenswrapper[5052]: I1203 11:49:59.936750 5052 generic.go:334] "Generic (PLEG): container finished" podID="e7c839dc-afd1-4cd2-a6a7-a24a5f716f13" containerID="2d4901c4da8c5451102183e62f966e62a2dd0b19c0f466d04d0154b54dcb5f2d" exitCode=0 Dec 03 11:49:59 crc kubenswrapper[5052]: I1203 11:49:59.936801 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-3cf1-account-create-update-96n2n" event={"ID":"e7c839dc-afd1-4cd2-a6a7-a24a5f716f13","Type":"ContainerDied","Data":"2d4901c4da8c5451102183e62f966e62a2dd0b19c0f466d04d0154b54dcb5f2d"} Dec 03 11:49:59 crc kubenswrapper[5052]: I1203 11:49:59.938978 5052 generic.go:334] "Generic (PLEG): container finished" podID="896b9690-42d0-4528-886d-e2e4984f1aee" containerID="8393f7f6989898dc3fee2162e87f42b6979e4c1290368f41d1aa48d371d31314" exitCode=0 Dec 03 11:49:59 crc kubenswrapper[5052]: I1203 11:49:59.939124 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-4385-account-create-update-52gf9" event={"ID":"896b9690-42d0-4528-886d-e2e4984f1aee","Type":"ContainerDied","Data":"8393f7f6989898dc3fee2162e87f42b6979e4c1290368f41d1aa48d371d31314"} Dec 03 11:49:59 crc kubenswrapper[5052]: I1203 11:49:59.944541 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-db-sync-l74fc" event={"ID":"01498d4a-9b4a-4991-9bb0-a5c77f885672","Type":"ContainerStarted","Data":"845fffc100a142a0e49faa1925c6d2a46f3a8982adbaa217c0cfa62f30ea7ce7"} Dec 03 11:49:59 crc kubenswrapper[5052]: I1203 11:49:59.948399 5052 generic.go:334] "Generic (PLEG): container finished" podID="d815bd05-9b18-4ef8-ab32-60bee21bf881" containerID="aaa2d28c5e3338722d4f5eaedeeb8a30c10f86ffe40a62d737b912ad57d3b9b1" exitCode=0 Dec 03 11:49:59 crc kubenswrapper[5052]: I1203 11:49:59.948456 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-6fd6-account-create-update-sz9zj" event={"ID":"d815bd05-9b18-4ef8-ab32-60bee21bf881","Type":"ContainerDied","Data":"aaa2d28c5e3338722d4f5eaedeeb8a30c10f86ffe40a62d737b912ad57d3b9b1"} Dec 03 11:49:59 crc kubenswrapper[5052]: I1203 11:49:59.951446 5052 generic.go:334] "Generic (PLEG): container finished" podID="760bae42-69a5-4a98-8a7d-5f1fd7143591" containerID="8664cd2b37abb007f9897daeb6f6c5de1533943a513fca1505b8bf8ee642c203" exitCode=0 Dec 03 11:49:59 crc kubenswrapper[5052]: I1203 11:49:59.951543 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-db-create-lfrbv" event={"ID":"760bae42-69a5-4a98-8a7d-5f1fd7143591","Type":"ContainerDied","Data":"8664cd2b37abb007f9897daeb6f6c5de1533943a513fca1505b8bf8ee642c203"} Dec 03 11:50:00 crc kubenswrapper[5052]: I1203 11:50:00.023160 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/keystone-db-sync-l74fc" podStartSLOduration=3.023142389 podStartE2EDuration="3.023142389s" podCreationTimestamp="2025-12-03 11:49:57 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:50:00.018920738 +0000 UTC m=+1790.607162502" watchObservedRunningTime="2025-12-03 11:50:00.023142389 +0000 UTC m=+1790.611384153" Dec 03 11:50:00 crc kubenswrapper[5052]: I1203 11:50:00.465810 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-db-create-kddz8" Dec 03 11:50:00 crc kubenswrapper[5052]: I1203 11:50:00.472357 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-db-create-rzt98" Dec 03 11:50:00 crc kubenswrapper[5052]: I1203 11:50:00.555579 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nb47c\" (UniqueName: \"kubernetes.io/projected/3f0bc56f-8f7f-4ebf-94f2-e02dc2edd717-kube-api-access-nb47c\") pod \"3f0bc56f-8f7f-4ebf-94f2-e02dc2edd717\" (UID: \"3f0bc56f-8f7f-4ebf-94f2-e02dc2edd717\") " Dec 03 11:50:00 crc kubenswrapper[5052]: I1203 11:50:00.555686 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-79tlr\" (UniqueName: \"kubernetes.io/projected/84a261ec-41e5-487b-b5a1-3743d98c7a76-kube-api-access-79tlr\") pod \"84a261ec-41e5-487b-b5a1-3743d98c7a76\" (UID: \"84a261ec-41e5-487b-b5a1-3743d98c7a76\") " Dec 03 11:50:00 crc kubenswrapper[5052]: I1203 11:50:00.555736 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/84a261ec-41e5-487b-b5a1-3743d98c7a76-operator-scripts\") pod \"84a261ec-41e5-487b-b5a1-3743d98c7a76\" (UID: \"84a261ec-41e5-487b-b5a1-3743d98c7a76\") " Dec 03 11:50:00 crc kubenswrapper[5052]: I1203 11:50:00.555811 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/3f0bc56f-8f7f-4ebf-94f2-e02dc2edd717-operator-scripts\") pod \"3f0bc56f-8f7f-4ebf-94f2-e02dc2edd717\" (UID: \"3f0bc56f-8f7f-4ebf-94f2-e02dc2edd717\") " Dec 03 11:50:00 crc kubenswrapper[5052]: I1203 11:50:00.556225 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/84a261ec-41e5-487b-b5a1-3743d98c7a76-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "84a261ec-41e5-487b-b5a1-3743d98c7a76" (UID: "84a261ec-41e5-487b-b5a1-3743d98c7a76"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:50:00 crc kubenswrapper[5052]: I1203 11:50:00.556642 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3f0bc56f-8f7f-4ebf-94f2-e02dc2edd717-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "3f0bc56f-8f7f-4ebf-94f2-e02dc2edd717" (UID: "3f0bc56f-8f7f-4ebf-94f2-e02dc2edd717"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:50:00 crc kubenswrapper[5052]: I1203 11:50:00.586646 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/84a261ec-41e5-487b-b5a1-3743d98c7a76-kube-api-access-79tlr" (OuterVolumeSpecName: "kube-api-access-79tlr") pod "84a261ec-41e5-487b-b5a1-3743d98c7a76" (UID: "84a261ec-41e5-487b-b5a1-3743d98c7a76"). InnerVolumeSpecName "kube-api-access-79tlr". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:50:00 crc kubenswrapper[5052]: I1203 11:50:00.587744 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3f0bc56f-8f7f-4ebf-94f2-e02dc2edd717-kube-api-access-nb47c" (OuterVolumeSpecName: "kube-api-access-nb47c") pod "3f0bc56f-8f7f-4ebf-94f2-e02dc2edd717" (UID: "3f0bc56f-8f7f-4ebf-94f2-e02dc2edd717"). InnerVolumeSpecName "kube-api-access-nb47c". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:50:00 crc kubenswrapper[5052]: I1203 11:50:00.656809 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-nb47c\" (UniqueName: \"kubernetes.io/projected/3f0bc56f-8f7f-4ebf-94f2-e02dc2edd717-kube-api-access-nb47c\") on node \"crc\" DevicePath \"\"" Dec 03 11:50:00 crc kubenswrapper[5052]: I1203 11:50:00.656835 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-79tlr\" (UniqueName: \"kubernetes.io/projected/84a261ec-41e5-487b-b5a1-3743d98c7a76-kube-api-access-79tlr\") on node \"crc\" DevicePath \"\"" Dec 03 11:50:00 crc kubenswrapper[5052]: I1203 11:50:00.656857 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/84a261ec-41e5-487b-b5a1-3743d98c7a76-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:50:00 crc kubenswrapper[5052]: I1203 11:50:00.656866 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/3f0bc56f-8f7f-4ebf-94f2-e02dc2edd717-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:50:00 crc kubenswrapper[5052]: I1203 11:50:00.962933 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-db-create-kddz8" event={"ID":"3f0bc56f-8f7f-4ebf-94f2-e02dc2edd717","Type":"ContainerDied","Data":"244cc0709af85ef689661f7b34631a574dffe324eadb5d89b1a5288c16441e56"} Dec 03 11:50:00 crc kubenswrapper[5052]: I1203 11:50:00.962977 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="244cc0709af85ef689661f7b34631a574dffe324eadb5d89b1a5288c16441e56" Dec 03 11:50:00 crc kubenswrapper[5052]: I1203 11:50:00.962979 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-db-create-kddz8" Dec 03 11:50:00 crc kubenswrapper[5052]: I1203 11:50:00.964188 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-db-create-rzt98" event={"ID":"84a261ec-41e5-487b-b5a1-3743d98c7a76","Type":"ContainerDied","Data":"9e144e85b5ab9abef569f4916096d7ab3da69a6b2c435c0ee7ad483f1bce5ec0"} Dec 03 11:50:00 crc kubenswrapper[5052]: I1203 11:50:00.964280 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9e144e85b5ab9abef569f4916096d7ab3da69a6b2c435c0ee7ad483f1bce5ec0" Dec 03 11:50:00 crc kubenswrapper[5052]: I1203 11:50:00.964336 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-db-create-rzt98" Dec 03 11:50:01 crc kubenswrapper[5052]: I1203 11:50:01.323017 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-3cf1-account-create-update-96n2n" Dec 03 11:50:01 crc kubenswrapper[5052]: I1203 11:50:01.369165 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/e7c839dc-afd1-4cd2-a6a7-a24a5f716f13-operator-scripts\") pod \"e7c839dc-afd1-4cd2-a6a7-a24a5f716f13\" (UID: \"e7c839dc-afd1-4cd2-a6a7-a24a5f716f13\") " Dec 03 11:50:01 crc kubenswrapper[5052]: I1203 11:50:01.369221 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-284d8\" (UniqueName: \"kubernetes.io/projected/e7c839dc-afd1-4cd2-a6a7-a24a5f716f13-kube-api-access-284d8\") pod \"e7c839dc-afd1-4cd2-a6a7-a24a5f716f13\" (UID: \"e7c839dc-afd1-4cd2-a6a7-a24a5f716f13\") " Dec 03 11:50:01 crc kubenswrapper[5052]: I1203 11:50:01.369809 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e7c839dc-afd1-4cd2-a6a7-a24a5f716f13-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "e7c839dc-afd1-4cd2-a6a7-a24a5f716f13" (UID: "e7c839dc-afd1-4cd2-a6a7-a24a5f716f13"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:50:01 crc kubenswrapper[5052]: I1203 11:50:01.370038 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/e7c839dc-afd1-4cd2-a6a7-a24a5f716f13-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:50:01 crc kubenswrapper[5052]: I1203 11:50:01.374616 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e7c839dc-afd1-4cd2-a6a7-a24a5f716f13-kube-api-access-284d8" (OuterVolumeSpecName: "kube-api-access-284d8") pod "e7c839dc-afd1-4cd2-a6a7-a24a5f716f13" (UID: "e7c839dc-afd1-4cd2-a6a7-a24a5f716f13"). InnerVolumeSpecName "kube-api-access-284d8". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:50:01 crc kubenswrapper[5052]: I1203 11:50:01.449851 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-db-create-lfrbv" Dec 03 11:50:01 crc kubenswrapper[5052]: I1203 11:50:01.467234 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-6fd6-account-create-update-sz9zj" Dec 03 11:50:01 crc kubenswrapper[5052]: I1203 11:50:01.470578 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/760bae42-69a5-4a98-8a7d-5f1fd7143591-operator-scripts\") pod \"760bae42-69a5-4a98-8a7d-5f1fd7143591\" (UID: \"760bae42-69a5-4a98-8a7d-5f1fd7143591\") " Dec 03 11:50:01 crc kubenswrapper[5052]: I1203 11:50:01.470792 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dbshc\" (UniqueName: \"kubernetes.io/projected/760bae42-69a5-4a98-8a7d-5f1fd7143591-kube-api-access-dbshc\") pod \"760bae42-69a5-4a98-8a7d-5f1fd7143591\" (UID: \"760bae42-69a5-4a98-8a7d-5f1fd7143591\") " Dec 03 11:50:01 crc kubenswrapper[5052]: I1203 11:50:01.471082 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/760bae42-69a5-4a98-8a7d-5f1fd7143591-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "760bae42-69a5-4a98-8a7d-5f1fd7143591" (UID: "760bae42-69a5-4a98-8a7d-5f1fd7143591"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:50:01 crc kubenswrapper[5052]: I1203 11:50:01.471100 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-284d8\" (UniqueName: \"kubernetes.io/projected/e7c839dc-afd1-4cd2-a6a7-a24a5f716f13-kube-api-access-284d8\") on node \"crc\" DevicePath \"\"" Dec 03 11:50:01 crc kubenswrapper[5052]: I1203 11:50:01.474312 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/760bae42-69a5-4a98-8a7d-5f1fd7143591-kube-api-access-dbshc" (OuterVolumeSpecName: "kube-api-access-dbshc") pod "760bae42-69a5-4a98-8a7d-5f1fd7143591" (UID: "760bae42-69a5-4a98-8a7d-5f1fd7143591"). InnerVolumeSpecName "kube-api-access-dbshc". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:50:01 crc kubenswrapper[5052]: I1203 11:50:01.474485 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-4385-account-create-update-52gf9" Dec 03 11:50:01 crc kubenswrapper[5052]: I1203 11:50:01.571755 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/d815bd05-9b18-4ef8-ab32-60bee21bf881-operator-scripts\") pod \"d815bd05-9b18-4ef8-ab32-60bee21bf881\" (UID: \"d815bd05-9b18-4ef8-ab32-60bee21bf881\") " Dec 03 11:50:01 crc kubenswrapper[5052]: I1203 11:50:01.571907 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6nl86\" (UniqueName: \"kubernetes.io/projected/d815bd05-9b18-4ef8-ab32-60bee21bf881-kube-api-access-6nl86\") pod \"d815bd05-9b18-4ef8-ab32-60bee21bf881\" (UID: \"d815bd05-9b18-4ef8-ab32-60bee21bf881\") " Dec 03 11:50:01 crc kubenswrapper[5052]: I1203 11:50:01.571996 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/896b9690-42d0-4528-886d-e2e4984f1aee-operator-scripts\") pod \"896b9690-42d0-4528-886d-e2e4984f1aee\" (UID: \"896b9690-42d0-4528-886d-e2e4984f1aee\") " Dec 03 11:50:01 crc kubenswrapper[5052]: I1203 11:50:01.572093 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zsrtf\" (UniqueName: \"kubernetes.io/projected/896b9690-42d0-4528-886d-e2e4984f1aee-kube-api-access-zsrtf\") pod \"896b9690-42d0-4528-886d-e2e4984f1aee\" (UID: \"896b9690-42d0-4528-886d-e2e4984f1aee\") " Dec 03 11:50:01 crc kubenswrapper[5052]: I1203 11:50:01.572728 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/760bae42-69a5-4a98-8a7d-5f1fd7143591-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:50:01 crc kubenswrapper[5052]: I1203 11:50:01.572759 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-dbshc\" (UniqueName: \"kubernetes.io/projected/760bae42-69a5-4a98-8a7d-5f1fd7143591-kube-api-access-dbshc\") on node \"crc\" DevicePath \"\"" Dec 03 11:50:01 crc kubenswrapper[5052]: I1203 11:50:01.573882 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d815bd05-9b18-4ef8-ab32-60bee21bf881-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "d815bd05-9b18-4ef8-ab32-60bee21bf881" (UID: "d815bd05-9b18-4ef8-ab32-60bee21bf881"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:50:01 crc kubenswrapper[5052]: I1203 11:50:01.573903 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/896b9690-42d0-4528-886d-e2e4984f1aee-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "896b9690-42d0-4528-886d-e2e4984f1aee" (UID: "896b9690-42d0-4528-886d-e2e4984f1aee"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:50:01 crc kubenswrapper[5052]: I1203 11:50:01.577400 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d815bd05-9b18-4ef8-ab32-60bee21bf881-kube-api-access-6nl86" (OuterVolumeSpecName: "kube-api-access-6nl86") pod "d815bd05-9b18-4ef8-ab32-60bee21bf881" (UID: "d815bd05-9b18-4ef8-ab32-60bee21bf881"). InnerVolumeSpecName "kube-api-access-6nl86". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:50:01 crc kubenswrapper[5052]: I1203 11:50:01.577592 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/896b9690-42d0-4528-886d-e2e4984f1aee-kube-api-access-zsrtf" (OuterVolumeSpecName: "kube-api-access-zsrtf") pod "896b9690-42d0-4528-886d-e2e4984f1aee" (UID: "896b9690-42d0-4528-886d-e2e4984f1aee"). InnerVolumeSpecName "kube-api-access-zsrtf". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:50:01 crc kubenswrapper[5052]: I1203 11:50:01.674268 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/d815bd05-9b18-4ef8-ab32-60bee21bf881-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:50:01 crc kubenswrapper[5052]: I1203 11:50:01.674307 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-6nl86\" (UniqueName: \"kubernetes.io/projected/d815bd05-9b18-4ef8-ab32-60bee21bf881-kube-api-access-6nl86\") on node \"crc\" DevicePath \"\"" Dec 03 11:50:01 crc kubenswrapper[5052]: I1203 11:50:01.674319 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/896b9690-42d0-4528-886d-e2e4984f1aee-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:50:01 crc kubenswrapper[5052]: I1203 11:50:01.674328 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-zsrtf\" (UniqueName: \"kubernetes.io/projected/896b9690-42d0-4528-886d-e2e4984f1aee-kube-api-access-zsrtf\") on node \"crc\" DevicePath \"\"" Dec 03 11:50:01 crc kubenswrapper[5052]: I1203 11:50:01.976862 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-6fd6-account-create-update-sz9zj" event={"ID":"d815bd05-9b18-4ef8-ab32-60bee21bf881","Type":"ContainerDied","Data":"6d178a3cf80582a2af7814a279023d610d4d7be0881be7a998f0e4f0b4520081"} Dec 03 11:50:01 crc kubenswrapper[5052]: I1203 11:50:01.976881 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-6fd6-account-create-update-sz9zj" Dec 03 11:50:01 crc kubenswrapper[5052]: I1203 11:50:01.977016 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6d178a3cf80582a2af7814a279023d610d4d7be0881be7a998f0e4f0b4520081" Dec 03 11:50:01 crc kubenswrapper[5052]: I1203 11:50:01.979047 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-db-create-lfrbv" event={"ID":"760bae42-69a5-4a98-8a7d-5f1fd7143591","Type":"ContainerDied","Data":"3a319f699dbb7668ddf14dc09b6cbe341445028ac405dc7ac49f03735a21b15a"} Dec 03 11:50:01 crc kubenswrapper[5052]: I1203 11:50:01.979096 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3a319f699dbb7668ddf14dc09b6cbe341445028ac405dc7ac49f03735a21b15a" Dec 03 11:50:01 crc kubenswrapper[5052]: I1203 11:50:01.979117 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-db-create-lfrbv" Dec 03 11:50:01 crc kubenswrapper[5052]: I1203 11:50:01.982426 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-3cf1-account-create-update-96n2n" event={"ID":"e7c839dc-afd1-4cd2-a6a7-a24a5f716f13","Type":"ContainerDied","Data":"cda1fc2ef75d4519ea906425fdb30e169fb9053da846251d85abf3eabac2dcc1"} Dec 03 11:50:01 crc kubenswrapper[5052]: I1203 11:50:01.982457 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cda1fc2ef75d4519ea906425fdb30e169fb9053da846251d85abf3eabac2dcc1" Dec 03 11:50:01 crc kubenswrapper[5052]: I1203 11:50:01.982502 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-3cf1-account-create-update-96n2n" Dec 03 11:50:01 crc kubenswrapper[5052]: I1203 11:50:01.987702 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-4385-account-create-update-52gf9" event={"ID":"896b9690-42d0-4528-886d-e2e4984f1aee","Type":"ContainerDied","Data":"c8c3ee26357f77ad06592bd8444aee33c0edba92dd078ce46c9a9bcfce96c823"} Dec 03 11:50:01 crc kubenswrapper[5052]: I1203 11:50:01.987796 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-4385-account-create-update-52gf9" Dec 03 11:50:01 crc kubenswrapper[5052]: I1203 11:50:01.987792 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c8c3ee26357f77ad06592bd8444aee33c0edba92dd078ce46c9a9bcfce96c823" Dec 03 11:50:03 crc kubenswrapper[5052]: I1203 11:50:03.673828 5052 scope.go:117] "RemoveContainer" containerID="25fe88f8d102f09983f8884e65f0c6c1575ade67921f10a630e6dcc3ea8b506e" Dec 03 11:50:03 crc kubenswrapper[5052]: E1203 11:50:03.674337 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 11:50:10 crc kubenswrapper[5052]: I1203 11:50:10.059932 5052 generic.go:334] "Generic (PLEG): container finished" podID="01498d4a-9b4a-4991-9bb0-a5c77f885672" containerID="845fffc100a142a0e49faa1925c6d2a46f3a8982adbaa217c0cfa62f30ea7ce7" exitCode=0 Dec 03 11:50:10 crc kubenswrapper[5052]: I1203 11:50:10.060014 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-db-sync-l74fc" event={"ID":"01498d4a-9b4a-4991-9bb0-a5c77f885672","Type":"ContainerDied","Data":"845fffc100a142a0e49faa1925c6d2a46f3a8982adbaa217c0cfa62f30ea7ce7"} Dec 03 11:50:11 crc kubenswrapper[5052]: I1203 11:50:11.402881 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-db-sync-l74fc" Dec 03 11:50:11 crc kubenswrapper[5052]: I1203 11:50:11.433112 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/01498d4a-9b4a-4991-9bb0-a5c77f885672-config-data\") pod \"01498d4a-9b4a-4991-9bb0-a5c77f885672\" (UID: \"01498d4a-9b4a-4991-9bb0-a5c77f885672\") " Dec 03 11:50:11 crc kubenswrapper[5052]: I1203 11:50:11.434632 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/01498d4a-9b4a-4991-9bb0-a5c77f885672-combined-ca-bundle\") pod \"01498d4a-9b4a-4991-9bb0-a5c77f885672\" (UID: \"01498d4a-9b4a-4991-9bb0-a5c77f885672\") " Dec 03 11:50:11 crc kubenswrapper[5052]: I1203 11:50:11.434749 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dpstd\" (UniqueName: \"kubernetes.io/projected/01498d4a-9b4a-4991-9bb0-a5c77f885672-kube-api-access-dpstd\") pod \"01498d4a-9b4a-4991-9bb0-a5c77f885672\" (UID: \"01498d4a-9b4a-4991-9bb0-a5c77f885672\") " Dec 03 11:50:11 crc kubenswrapper[5052]: I1203 11:50:11.440257 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/01498d4a-9b4a-4991-9bb0-a5c77f885672-kube-api-access-dpstd" (OuterVolumeSpecName: "kube-api-access-dpstd") pod "01498d4a-9b4a-4991-9bb0-a5c77f885672" (UID: "01498d4a-9b4a-4991-9bb0-a5c77f885672"). InnerVolumeSpecName "kube-api-access-dpstd". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:50:11 crc kubenswrapper[5052]: I1203 11:50:11.473187 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/01498d4a-9b4a-4991-9bb0-a5c77f885672-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "01498d4a-9b4a-4991-9bb0-a5c77f885672" (UID: "01498d4a-9b4a-4991-9bb0-a5c77f885672"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:50:11 crc kubenswrapper[5052]: I1203 11:50:11.488477 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/01498d4a-9b4a-4991-9bb0-a5c77f885672-config-data" (OuterVolumeSpecName: "config-data") pod "01498d4a-9b4a-4991-9bb0-a5c77f885672" (UID: "01498d4a-9b4a-4991-9bb0-a5c77f885672"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:50:11 crc kubenswrapper[5052]: I1203 11:50:11.536542 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/01498d4a-9b4a-4991-9bb0-a5c77f885672-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:50:11 crc kubenswrapper[5052]: I1203 11:50:11.536594 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/01498d4a-9b4a-4991-9bb0-a5c77f885672-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:50:11 crc kubenswrapper[5052]: I1203 11:50:11.536613 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-dpstd\" (UniqueName: \"kubernetes.io/projected/01498d4a-9b4a-4991-9bb0-a5c77f885672-kube-api-access-dpstd\") on node \"crc\" DevicePath \"\"" Dec 03 11:50:12 crc kubenswrapper[5052]: I1203 11:50:12.085118 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-db-sync-l74fc" event={"ID":"01498d4a-9b4a-4991-9bb0-a5c77f885672","Type":"ContainerDied","Data":"13fde770f4f95f27ef99617c31d93e7471715ef6f4f8058ac1acee39ddeeb328"} Dec 03 11:50:12 crc kubenswrapper[5052]: I1203 11:50:12.085191 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-db-sync-l74fc" Dec 03 11:50:12 crc kubenswrapper[5052]: I1203 11:50:12.085204 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="13fde770f4f95f27ef99617c31d93e7471715ef6f4f8058ac1acee39ddeeb328" Dec 03 11:50:12 crc kubenswrapper[5052]: I1203 11:50:12.268444 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/keystone-bootstrap-vbjb4"] Dec 03 11:50:12 crc kubenswrapper[5052]: E1203 11:50:12.269031 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="760bae42-69a5-4a98-8a7d-5f1fd7143591" containerName="mariadb-database-create" Dec 03 11:50:12 crc kubenswrapper[5052]: I1203 11:50:12.269049 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="760bae42-69a5-4a98-8a7d-5f1fd7143591" containerName="mariadb-database-create" Dec 03 11:50:12 crc kubenswrapper[5052]: E1203 11:50:12.269060 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="3f0bc56f-8f7f-4ebf-94f2-e02dc2edd717" containerName="mariadb-database-create" Dec 03 11:50:12 crc kubenswrapper[5052]: I1203 11:50:12.269067 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="3f0bc56f-8f7f-4ebf-94f2-e02dc2edd717" containerName="mariadb-database-create" Dec 03 11:50:12 crc kubenswrapper[5052]: E1203 11:50:12.269077 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="84a261ec-41e5-487b-b5a1-3743d98c7a76" containerName="mariadb-database-create" Dec 03 11:50:12 crc kubenswrapper[5052]: I1203 11:50:12.269083 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="84a261ec-41e5-487b-b5a1-3743d98c7a76" containerName="mariadb-database-create" Dec 03 11:50:12 crc kubenswrapper[5052]: E1203 11:50:12.269090 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="01498d4a-9b4a-4991-9bb0-a5c77f885672" containerName="keystone-db-sync" Dec 03 11:50:12 crc kubenswrapper[5052]: I1203 11:50:12.269096 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="01498d4a-9b4a-4991-9bb0-a5c77f885672" containerName="keystone-db-sync" Dec 03 11:50:12 crc kubenswrapper[5052]: E1203 11:50:12.269108 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="896b9690-42d0-4528-886d-e2e4984f1aee" containerName="mariadb-account-create-update" Dec 03 11:50:12 crc kubenswrapper[5052]: I1203 11:50:12.269114 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="896b9690-42d0-4528-886d-e2e4984f1aee" containerName="mariadb-account-create-update" Dec 03 11:50:12 crc kubenswrapper[5052]: E1203 11:50:12.269122 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="d815bd05-9b18-4ef8-ab32-60bee21bf881" containerName="mariadb-account-create-update" Dec 03 11:50:12 crc kubenswrapper[5052]: I1203 11:50:12.269128 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="d815bd05-9b18-4ef8-ab32-60bee21bf881" containerName="mariadb-account-create-update" Dec 03 11:50:12 crc kubenswrapper[5052]: E1203 11:50:12.269143 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="e7c839dc-afd1-4cd2-a6a7-a24a5f716f13" containerName="mariadb-account-create-update" Dec 03 11:50:12 crc kubenswrapper[5052]: I1203 11:50:12.269149 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="e7c839dc-afd1-4cd2-a6a7-a24a5f716f13" containerName="mariadb-account-create-update" Dec 03 11:50:12 crc kubenswrapper[5052]: I1203 11:50:12.269285 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="3f0bc56f-8f7f-4ebf-94f2-e02dc2edd717" containerName="mariadb-database-create" Dec 03 11:50:12 crc kubenswrapper[5052]: I1203 11:50:12.269301 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="01498d4a-9b4a-4991-9bb0-a5c77f885672" containerName="keystone-db-sync" Dec 03 11:50:12 crc kubenswrapper[5052]: I1203 11:50:12.269309 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="d815bd05-9b18-4ef8-ab32-60bee21bf881" containerName="mariadb-account-create-update" Dec 03 11:50:12 crc kubenswrapper[5052]: I1203 11:50:12.269316 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="84a261ec-41e5-487b-b5a1-3743d98c7a76" containerName="mariadb-database-create" Dec 03 11:50:12 crc kubenswrapper[5052]: I1203 11:50:12.269324 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="e7c839dc-afd1-4cd2-a6a7-a24a5f716f13" containerName="mariadb-account-create-update" Dec 03 11:50:12 crc kubenswrapper[5052]: I1203 11:50:12.269331 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="896b9690-42d0-4528-886d-e2e4984f1aee" containerName="mariadb-account-create-update" Dec 03 11:50:12 crc kubenswrapper[5052]: I1203 11:50:12.269348 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="760bae42-69a5-4a98-8a7d-5f1fd7143591" containerName="mariadb-database-create" Dec 03 11:50:12 crc kubenswrapper[5052]: I1203 11:50:12.269887 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-bootstrap-vbjb4" Dec 03 11:50:12 crc kubenswrapper[5052]: I1203 11:50:12.272280 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone-scripts" Dec 03 11:50:12 crc kubenswrapper[5052]: I1203 11:50:12.272428 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone-config-data" Dec 03 11:50:12 crc kubenswrapper[5052]: I1203 11:50:12.272976 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"osp-secret" Dec 03 11:50:12 crc kubenswrapper[5052]: I1203 11:50:12.274935 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone" Dec 03 11:50:12 crc kubenswrapper[5052]: I1203 11:50:12.275524 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone-keystone-dockercfg-xtl6f" Dec 03 11:50:12 crc kubenswrapper[5052]: I1203 11:50:12.291847 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-bootstrap-vbjb4"] Dec 03 11:50:12 crc kubenswrapper[5052]: I1203 11:50:12.348977 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-grwkk\" (UniqueName: \"kubernetes.io/projected/20d3b16f-b0ce-4c24-b665-3be3a2da781a-kube-api-access-grwkk\") pod \"keystone-bootstrap-vbjb4\" (UID: \"20d3b16f-b0ce-4c24-b665-3be3a2da781a\") " pod="openstack-kuttl-tests/keystone-bootstrap-vbjb4" Dec 03 11:50:12 crc kubenswrapper[5052]: I1203 11:50:12.349059 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/20d3b16f-b0ce-4c24-b665-3be3a2da781a-fernet-keys\") pod \"keystone-bootstrap-vbjb4\" (UID: \"20d3b16f-b0ce-4c24-b665-3be3a2da781a\") " pod="openstack-kuttl-tests/keystone-bootstrap-vbjb4" Dec 03 11:50:12 crc kubenswrapper[5052]: I1203 11:50:12.349095 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/20d3b16f-b0ce-4c24-b665-3be3a2da781a-config-data\") pod \"keystone-bootstrap-vbjb4\" (UID: \"20d3b16f-b0ce-4c24-b665-3be3a2da781a\") " pod="openstack-kuttl-tests/keystone-bootstrap-vbjb4" Dec 03 11:50:12 crc kubenswrapper[5052]: I1203 11:50:12.349137 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/20d3b16f-b0ce-4c24-b665-3be3a2da781a-scripts\") pod \"keystone-bootstrap-vbjb4\" (UID: \"20d3b16f-b0ce-4c24-b665-3be3a2da781a\") " pod="openstack-kuttl-tests/keystone-bootstrap-vbjb4" Dec 03 11:50:12 crc kubenswrapper[5052]: I1203 11:50:12.349249 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/20d3b16f-b0ce-4c24-b665-3be3a2da781a-credential-keys\") pod \"keystone-bootstrap-vbjb4\" (UID: \"20d3b16f-b0ce-4c24-b665-3be3a2da781a\") " pod="openstack-kuttl-tests/keystone-bootstrap-vbjb4" Dec 03 11:50:12 crc kubenswrapper[5052]: I1203 11:50:12.349326 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/20d3b16f-b0ce-4c24-b665-3be3a2da781a-combined-ca-bundle\") pod \"keystone-bootstrap-vbjb4\" (UID: \"20d3b16f-b0ce-4c24-b665-3be3a2da781a\") " pod="openstack-kuttl-tests/keystone-bootstrap-vbjb4" Dec 03 11:50:12 crc kubenswrapper[5052]: I1203 11:50:12.452848 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/20d3b16f-b0ce-4c24-b665-3be3a2da781a-fernet-keys\") pod \"keystone-bootstrap-vbjb4\" (UID: \"20d3b16f-b0ce-4c24-b665-3be3a2da781a\") " pod="openstack-kuttl-tests/keystone-bootstrap-vbjb4" Dec 03 11:50:12 crc kubenswrapper[5052]: I1203 11:50:12.452907 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/20d3b16f-b0ce-4c24-b665-3be3a2da781a-config-data\") pod \"keystone-bootstrap-vbjb4\" (UID: \"20d3b16f-b0ce-4c24-b665-3be3a2da781a\") " pod="openstack-kuttl-tests/keystone-bootstrap-vbjb4" Dec 03 11:50:12 crc kubenswrapper[5052]: I1203 11:50:12.452942 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/20d3b16f-b0ce-4c24-b665-3be3a2da781a-scripts\") pod \"keystone-bootstrap-vbjb4\" (UID: \"20d3b16f-b0ce-4c24-b665-3be3a2da781a\") " pod="openstack-kuttl-tests/keystone-bootstrap-vbjb4" Dec 03 11:50:12 crc kubenswrapper[5052]: I1203 11:50:12.452984 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/20d3b16f-b0ce-4c24-b665-3be3a2da781a-credential-keys\") pod \"keystone-bootstrap-vbjb4\" (UID: \"20d3b16f-b0ce-4c24-b665-3be3a2da781a\") " pod="openstack-kuttl-tests/keystone-bootstrap-vbjb4" Dec 03 11:50:12 crc kubenswrapper[5052]: I1203 11:50:12.453033 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/20d3b16f-b0ce-4c24-b665-3be3a2da781a-combined-ca-bundle\") pod \"keystone-bootstrap-vbjb4\" (UID: \"20d3b16f-b0ce-4c24-b665-3be3a2da781a\") " pod="openstack-kuttl-tests/keystone-bootstrap-vbjb4" Dec 03 11:50:12 crc kubenswrapper[5052]: I1203 11:50:12.453088 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-grwkk\" (UniqueName: \"kubernetes.io/projected/20d3b16f-b0ce-4c24-b665-3be3a2da781a-kube-api-access-grwkk\") pod \"keystone-bootstrap-vbjb4\" (UID: \"20d3b16f-b0ce-4c24-b665-3be3a2da781a\") " pod="openstack-kuttl-tests/keystone-bootstrap-vbjb4" Dec 03 11:50:12 crc kubenswrapper[5052]: I1203 11:50:12.459798 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/20d3b16f-b0ce-4c24-b665-3be3a2da781a-fernet-keys\") pod \"keystone-bootstrap-vbjb4\" (UID: \"20d3b16f-b0ce-4c24-b665-3be3a2da781a\") " pod="openstack-kuttl-tests/keystone-bootstrap-vbjb4" Dec 03 11:50:12 crc kubenswrapper[5052]: I1203 11:50:12.477557 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/cinder-db-sync-lgtfz"] Dec 03 11:50:12 crc kubenswrapper[5052]: I1203 11:50:12.478836 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-db-sync-lgtfz" Dec 03 11:50:12 crc kubenswrapper[5052]: I1203 11:50:12.482325 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/20d3b16f-b0ce-4c24-b665-3be3a2da781a-scripts\") pod \"keystone-bootstrap-vbjb4\" (UID: \"20d3b16f-b0ce-4c24-b665-3be3a2da781a\") " pod="openstack-kuttl-tests/keystone-bootstrap-vbjb4" Dec 03 11:50:12 crc kubenswrapper[5052]: I1203 11:50:12.483329 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/20d3b16f-b0ce-4c24-b665-3be3a2da781a-credential-keys\") pod \"keystone-bootstrap-vbjb4\" (UID: \"20d3b16f-b0ce-4c24-b665-3be3a2da781a\") " pod="openstack-kuttl-tests/keystone-bootstrap-vbjb4" Dec 03 11:50:12 crc kubenswrapper[5052]: I1203 11:50:12.483343 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-grwkk\" (UniqueName: \"kubernetes.io/projected/20d3b16f-b0ce-4c24-b665-3be3a2da781a-kube-api-access-grwkk\") pod \"keystone-bootstrap-vbjb4\" (UID: \"20d3b16f-b0ce-4c24-b665-3be3a2da781a\") " pod="openstack-kuttl-tests/keystone-bootstrap-vbjb4" Dec 03 11:50:12 crc kubenswrapper[5052]: I1203 11:50:12.483904 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/20d3b16f-b0ce-4c24-b665-3be3a2da781a-combined-ca-bundle\") pod \"keystone-bootstrap-vbjb4\" (UID: \"20d3b16f-b0ce-4c24-b665-3be3a2da781a\") " pod="openstack-kuttl-tests/keystone-bootstrap-vbjb4" Dec 03 11:50:12 crc kubenswrapper[5052]: I1203 11:50:12.486503 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/20d3b16f-b0ce-4c24-b665-3be3a2da781a-config-data\") pod \"keystone-bootstrap-vbjb4\" (UID: \"20d3b16f-b0ce-4c24-b665-3be3a2da781a\") " pod="openstack-kuttl-tests/keystone-bootstrap-vbjb4" Dec 03 11:50:12 crc kubenswrapper[5052]: I1203 11:50:12.489095 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cinder-scripts" Dec 03 11:50:12 crc kubenswrapper[5052]: I1203 11:50:12.489189 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cinder-cinder-dockercfg-vkzbs" Dec 03 11:50:12 crc kubenswrapper[5052]: I1203 11:50:12.489452 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cinder-config-data" Dec 03 11:50:12 crc kubenswrapper[5052]: I1203 11:50:12.490560 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/neutron-db-sync-ghfwv"] Dec 03 11:50:12 crc kubenswrapper[5052]: I1203 11:50:12.491944 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-db-sync-ghfwv" Dec 03 11:50:12 crc kubenswrapper[5052]: I1203 11:50:12.496649 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"neutron-neutron-dockercfg-nslhs" Dec 03 11:50:12 crc kubenswrapper[5052]: I1203 11:50:12.496710 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"neutron-httpd-config" Dec 03 11:50:12 crc kubenswrapper[5052]: I1203 11:50:12.498527 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"neutron-config" Dec 03 11:50:12 crc kubenswrapper[5052]: I1203 11:50:12.499691 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-db-sync-lgtfz"] Dec 03 11:50:12 crc kubenswrapper[5052]: I1203 11:50:12.514885 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-db-sync-ghfwv"] Dec 03 11:50:12 crc kubenswrapper[5052]: I1203 11:50:12.554008 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/5ddb757c-c235-4a2f-b6db-0d120215edcf-db-sync-config-data\") pod \"cinder-db-sync-lgtfz\" (UID: \"5ddb757c-c235-4a2f-b6db-0d120215edcf\") " pod="openstack-kuttl-tests/cinder-db-sync-lgtfz" Dec 03 11:50:12 crc kubenswrapper[5052]: I1203 11:50:12.554047 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/d6eac989-1eec-4b3a-b9d7-d5a8da2c1a27-config\") pod \"neutron-db-sync-ghfwv\" (UID: \"d6eac989-1eec-4b3a-b9d7-d5a8da2c1a27\") " pod="openstack-kuttl-tests/neutron-db-sync-ghfwv" Dec 03 11:50:12 crc kubenswrapper[5052]: I1203 11:50:12.554092 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/5ddb757c-c235-4a2f-b6db-0d120215edcf-scripts\") pod \"cinder-db-sync-lgtfz\" (UID: \"5ddb757c-c235-4a2f-b6db-0d120215edcf\") " pod="openstack-kuttl-tests/cinder-db-sync-lgtfz" Dec 03 11:50:12 crc kubenswrapper[5052]: I1203 11:50:12.554133 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/5ddb757c-c235-4a2f-b6db-0d120215edcf-etc-machine-id\") pod \"cinder-db-sync-lgtfz\" (UID: \"5ddb757c-c235-4a2f-b6db-0d120215edcf\") " pod="openstack-kuttl-tests/cinder-db-sync-lgtfz" Dec 03 11:50:12 crc kubenswrapper[5052]: I1203 11:50:12.554156 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/5ddb757c-c235-4a2f-b6db-0d120215edcf-config-data\") pod \"cinder-db-sync-lgtfz\" (UID: \"5ddb757c-c235-4a2f-b6db-0d120215edcf\") " pod="openstack-kuttl-tests/cinder-db-sync-lgtfz" Dec 03 11:50:12 crc kubenswrapper[5052]: I1203 11:50:12.554337 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d6eac989-1eec-4b3a-b9d7-d5a8da2c1a27-combined-ca-bundle\") pod \"neutron-db-sync-ghfwv\" (UID: \"d6eac989-1eec-4b3a-b9d7-d5a8da2c1a27\") " pod="openstack-kuttl-tests/neutron-db-sync-ghfwv" Dec 03 11:50:12 crc kubenswrapper[5052]: I1203 11:50:12.554419 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sjbmp\" (UniqueName: \"kubernetes.io/projected/5ddb757c-c235-4a2f-b6db-0d120215edcf-kube-api-access-sjbmp\") pod \"cinder-db-sync-lgtfz\" (UID: \"5ddb757c-c235-4a2f-b6db-0d120215edcf\") " pod="openstack-kuttl-tests/cinder-db-sync-lgtfz" Dec 03 11:50:12 crc kubenswrapper[5052]: I1203 11:50:12.554446 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9xwxv\" (UniqueName: \"kubernetes.io/projected/d6eac989-1eec-4b3a-b9d7-d5a8da2c1a27-kube-api-access-9xwxv\") pod \"neutron-db-sync-ghfwv\" (UID: \"d6eac989-1eec-4b3a-b9d7-d5a8da2c1a27\") " pod="openstack-kuttl-tests/neutron-db-sync-ghfwv" Dec 03 11:50:12 crc kubenswrapper[5052]: I1203 11:50:12.554475 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5ddb757c-c235-4a2f-b6db-0d120215edcf-combined-ca-bundle\") pod \"cinder-db-sync-lgtfz\" (UID: \"5ddb757c-c235-4a2f-b6db-0d120215edcf\") " pod="openstack-kuttl-tests/cinder-db-sync-lgtfz" Dec 03 11:50:12 crc kubenswrapper[5052]: I1203 11:50:12.579941 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/placement-db-sync-nqvdz"] Dec 03 11:50:12 crc kubenswrapper[5052]: I1203 11:50:12.581373 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-db-sync-nqvdz" Dec 03 11:50:12 crc kubenswrapper[5052]: I1203 11:50:12.589671 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-bootstrap-vbjb4" Dec 03 11:50:12 crc kubenswrapper[5052]: I1203 11:50:12.590068 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"placement-scripts" Dec 03 11:50:12 crc kubenswrapper[5052]: I1203 11:50:12.590161 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"placement-placement-dockercfg-kv7xq" Dec 03 11:50:12 crc kubenswrapper[5052]: I1203 11:50:12.590295 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"placement-config-data" Dec 03 11:50:12 crc kubenswrapper[5052]: I1203 11:50:12.598890 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/placement-db-sync-nqvdz"] Dec 03 11:50:12 crc kubenswrapper[5052]: I1203 11:50:12.619606 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/barbican-db-sync-847xp"] Dec 03 11:50:12 crc kubenswrapper[5052]: I1203 11:50:12.621078 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-db-sync-847xp" Dec 03 11:50:12 crc kubenswrapper[5052]: I1203 11:50:12.624762 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"barbican-config-data" Dec 03 11:50:12 crc kubenswrapper[5052]: I1203 11:50:12.624804 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"barbican-barbican-dockercfg-wsvrm" Dec 03 11:50:12 crc kubenswrapper[5052]: I1203 11:50:12.632679 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-db-sync-847xp"] Dec 03 11:50:12 crc kubenswrapper[5052]: I1203 11:50:12.657019 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/3a49f2d7-59af-4bc8-8d05-641b33c7a24c-scripts\") pod \"placement-db-sync-nqvdz\" (UID: \"3a49f2d7-59af-4bc8-8d05-641b33c7a24c\") " pod="openstack-kuttl-tests/placement-db-sync-nqvdz" Dec 03 11:50:12 crc kubenswrapper[5052]: I1203 11:50:12.657080 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/3a49f2d7-59af-4bc8-8d05-641b33c7a24c-config-data\") pod \"placement-db-sync-nqvdz\" (UID: \"3a49f2d7-59af-4bc8-8d05-641b33c7a24c\") " pod="openstack-kuttl-tests/placement-db-sync-nqvdz" Dec 03 11:50:12 crc kubenswrapper[5052]: I1203 11:50:12.657129 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d6eac989-1eec-4b3a-b9d7-d5a8da2c1a27-combined-ca-bundle\") pod \"neutron-db-sync-ghfwv\" (UID: \"d6eac989-1eec-4b3a-b9d7-d5a8da2c1a27\") " pod="openstack-kuttl-tests/neutron-db-sync-ghfwv" Dec 03 11:50:12 crc kubenswrapper[5052]: I1203 11:50:12.657162 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-sjbmp\" (UniqueName: \"kubernetes.io/projected/5ddb757c-c235-4a2f-b6db-0d120215edcf-kube-api-access-sjbmp\") pod \"cinder-db-sync-lgtfz\" (UID: \"5ddb757c-c235-4a2f-b6db-0d120215edcf\") " pod="openstack-kuttl-tests/cinder-db-sync-lgtfz" Dec 03 11:50:12 crc kubenswrapper[5052]: I1203 11:50:12.657189 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-9xwxv\" (UniqueName: \"kubernetes.io/projected/d6eac989-1eec-4b3a-b9d7-d5a8da2c1a27-kube-api-access-9xwxv\") pod \"neutron-db-sync-ghfwv\" (UID: \"d6eac989-1eec-4b3a-b9d7-d5a8da2c1a27\") " pod="openstack-kuttl-tests/neutron-db-sync-ghfwv" Dec 03 11:50:12 crc kubenswrapper[5052]: I1203 11:50:12.657213 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5ddb757c-c235-4a2f-b6db-0d120215edcf-combined-ca-bundle\") pod \"cinder-db-sync-lgtfz\" (UID: \"5ddb757c-c235-4a2f-b6db-0d120215edcf\") " pod="openstack-kuttl-tests/cinder-db-sync-lgtfz" Dec 03 11:50:12 crc kubenswrapper[5052]: I1203 11:50:12.657243 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/3a49f2d7-59af-4bc8-8d05-641b33c7a24c-logs\") pod \"placement-db-sync-nqvdz\" (UID: \"3a49f2d7-59af-4bc8-8d05-641b33c7a24c\") " pod="openstack-kuttl-tests/placement-db-sync-nqvdz" Dec 03 11:50:12 crc kubenswrapper[5052]: I1203 11:50:12.657282 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/97107a61-a19d-4494-bcdb-a8d8f6223cb5-combined-ca-bundle\") pod \"barbican-db-sync-847xp\" (UID: \"97107a61-a19d-4494-bcdb-a8d8f6223cb5\") " pod="openstack-kuttl-tests/barbican-db-sync-847xp" Dec 03 11:50:12 crc kubenswrapper[5052]: I1203 11:50:12.657310 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/5ddb757c-c235-4a2f-b6db-0d120215edcf-db-sync-config-data\") pod \"cinder-db-sync-lgtfz\" (UID: \"5ddb757c-c235-4a2f-b6db-0d120215edcf\") " pod="openstack-kuttl-tests/cinder-db-sync-lgtfz" Dec 03 11:50:12 crc kubenswrapper[5052]: I1203 11:50:12.657337 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/d6eac989-1eec-4b3a-b9d7-d5a8da2c1a27-config\") pod \"neutron-db-sync-ghfwv\" (UID: \"d6eac989-1eec-4b3a-b9d7-d5a8da2c1a27\") " pod="openstack-kuttl-tests/neutron-db-sync-ghfwv" Dec 03 11:50:12 crc kubenswrapper[5052]: I1203 11:50:12.657383 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/5ddb757c-c235-4a2f-b6db-0d120215edcf-scripts\") pod \"cinder-db-sync-lgtfz\" (UID: \"5ddb757c-c235-4a2f-b6db-0d120215edcf\") " pod="openstack-kuttl-tests/cinder-db-sync-lgtfz" Dec 03 11:50:12 crc kubenswrapper[5052]: I1203 11:50:12.657416 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/97107a61-a19d-4494-bcdb-a8d8f6223cb5-db-sync-config-data\") pod \"barbican-db-sync-847xp\" (UID: \"97107a61-a19d-4494-bcdb-a8d8f6223cb5\") " pod="openstack-kuttl-tests/barbican-db-sync-847xp" Dec 03 11:50:12 crc kubenswrapper[5052]: I1203 11:50:12.657441 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3a49f2d7-59af-4bc8-8d05-641b33c7a24c-combined-ca-bundle\") pod \"placement-db-sync-nqvdz\" (UID: \"3a49f2d7-59af-4bc8-8d05-641b33c7a24c\") " pod="openstack-kuttl-tests/placement-db-sync-nqvdz" Dec 03 11:50:12 crc kubenswrapper[5052]: I1203 11:50:12.657473 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dg59k\" (UniqueName: \"kubernetes.io/projected/3a49f2d7-59af-4bc8-8d05-641b33c7a24c-kube-api-access-dg59k\") pod \"placement-db-sync-nqvdz\" (UID: \"3a49f2d7-59af-4bc8-8d05-641b33c7a24c\") " pod="openstack-kuttl-tests/placement-db-sync-nqvdz" Dec 03 11:50:12 crc kubenswrapper[5052]: I1203 11:50:12.657522 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/5ddb757c-c235-4a2f-b6db-0d120215edcf-config-data\") pod \"cinder-db-sync-lgtfz\" (UID: \"5ddb757c-c235-4a2f-b6db-0d120215edcf\") " pod="openstack-kuttl-tests/cinder-db-sync-lgtfz" Dec 03 11:50:12 crc kubenswrapper[5052]: I1203 11:50:12.657551 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7jnbj\" (UniqueName: \"kubernetes.io/projected/97107a61-a19d-4494-bcdb-a8d8f6223cb5-kube-api-access-7jnbj\") pod \"barbican-db-sync-847xp\" (UID: \"97107a61-a19d-4494-bcdb-a8d8f6223cb5\") " pod="openstack-kuttl-tests/barbican-db-sync-847xp" Dec 03 11:50:12 crc kubenswrapper[5052]: I1203 11:50:12.657577 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/5ddb757c-c235-4a2f-b6db-0d120215edcf-etc-machine-id\") pod \"cinder-db-sync-lgtfz\" (UID: \"5ddb757c-c235-4a2f-b6db-0d120215edcf\") " pod="openstack-kuttl-tests/cinder-db-sync-lgtfz" Dec 03 11:50:12 crc kubenswrapper[5052]: I1203 11:50:12.657703 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/5ddb757c-c235-4a2f-b6db-0d120215edcf-etc-machine-id\") pod \"cinder-db-sync-lgtfz\" (UID: \"5ddb757c-c235-4a2f-b6db-0d120215edcf\") " pod="openstack-kuttl-tests/cinder-db-sync-lgtfz" Dec 03 11:50:12 crc kubenswrapper[5052]: I1203 11:50:12.664333 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/d6eac989-1eec-4b3a-b9d7-d5a8da2c1a27-config\") pod \"neutron-db-sync-ghfwv\" (UID: \"d6eac989-1eec-4b3a-b9d7-d5a8da2c1a27\") " pod="openstack-kuttl-tests/neutron-db-sync-ghfwv" Dec 03 11:50:12 crc kubenswrapper[5052]: I1203 11:50:12.676182 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/5ddb757c-c235-4a2f-b6db-0d120215edcf-scripts\") pod \"cinder-db-sync-lgtfz\" (UID: \"5ddb757c-c235-4a2f-b6db-0d120215edcf\") " pod="openstack-kuttl-tests/cinder-db-sync-lgtfz" Dec 03 11:50:12 crc kubenswrapper[5052]: I1203 11:50:12.676954 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5ddb757c-c235-4a2f-b6db-0d120215edcf-combined-ca-bundle\") pod \"cinder-db-sync-lgtfz\" (UID: \"5ddb757c-c235-4a2f-b6db-0d120215edcf\") " pod="openstack-kuttl-tests/cinder-db-sync-lgtfz" Dec 03 11:50:12 crc kubenswrapper[5052]: I1203 11:50:12.681089 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/5ddb757c-c235-4a2f-b6db-0d120215edcf-config-data\") pod \"cinder-db-sync-lgtfz\" (UID: \"5ddb757c-c235-4a2f-b6db-0d120215edcf\") " pod="openstack-kuttl-tests/cinder-db-sync-lgtfz" Dec 03 11:50:12 crc kubenswrapper[5052]: I1203 11:50:12.682819 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d6eac989-1eec-4b3a-b9d7-d5a8da2c1a27-combined-ca-bundle\") pod \"neutron-db-sync-ghfwv\" (UID: \"d6eac989-1eec-4b3a-b9d7-d5a8da2c1a27\") " pod="openstack-kuttl-tests/neutron-db-sync-ghfwv" Dec 03 11:50:12 crc kubenswrapper[5052]: I1203 11:50:12.684813 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-sjbmp\" (UniqueName: \"kubernetes.io/projected/5ddb757c-c235-4a2f-b6db-0d120215edcf-kube-api-access-sjbmp\") pod \"cinder-db-sync-lgtfz\" (UID: \"5ddb757c-c235-4a2f-b6db-0d120215edcf\") " pod="openstack-kuttl-tests/cinder-db-sync-lgtfz" Dec 03 11:50:12 crc kubenswrapper[5052]: I1203 11:50:12.684895 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/5ddb757c-c235-4a2f-b6db-0d120215edcf-db-sync-config-data\") pod \"cinder-db-sync-lgtfz\" (UID: \"5ddb757c-c235-4a2f-b6db-0d120215edcf\") " pod="openstack-kuttl-tests/cinder-db-sync-lgtfz" Dec 03 11:50:12 crc kubenswrapper[5052]: I1203 11:50:12.696878 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-9xwxv\" (UniqueName: \"kubernetes.io/projected/d6eac989-1eec-4b3a-b9d7-d5a8da2c1a27-kube-api-access-9xwxv\") pod \"neutron-db-sync-ghfwv\" (UID: \"d6eac989-1eec-4b3a-b9d7-d5a8da2c1a27\") " pod="openstack-kuttl-tests/neutron-db-sync-ghfwv" Dec 03 11:50:12 crc kubenswrapper[5052]: I1203 11:50:12.721259 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:50:12 crc kubenswrapper[5052]: I1203 11:50:12.728744 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:50:12 crc kubenswrapper[5052]: I1203 11:50:12.735611 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-config-data" Dec 03 11:50:12 crc kubenswrapper[5052]: I1203 11:50:12.744316 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-scripts" Dec 03 11:50:12 crc kubenswrapper[5052]: I1203 11:50:12.759480 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/666333b4-b787-4efa-9681-e6dc65f42542-config-data\") pod \"ceilometer-0\" (UID: \"666333b4-b787-4efa-9681-e6dc65f42542\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:50:12 crc kubenswrapper[5052]: I1203 11:50:12.759535 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/666333b4-b787-4efa-9681-e6dc65f42542-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"666333b4-b787-4efa-9681-e6dc65f42542\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:50:12 crc kubenswrapper[5052]: I1203 11:50:12.759620 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/666333b4-b787-4efa-9681-e6dc65f42542-run-httpd\") pod \"ceilometer-0\" (UID: \"666333b4-b787-4efa-9681-e6dc65f42542\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:50:12 crc kubenswrapper[5052]: I1203 11:50:12.759748 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/97107a61-a19d-4494-bcdb-a8d8f6223cb5-db-sync-config-data\") pod \"barbican-db-sync-847xp\" (UID: \"97107a61-a19d-4494-bcdb-a8d8f6223cb5\") " pod="openstack-kuttl-tests/barbican-db-sync-847xp" Dec 03 11:50:12 crc kubenswrapper[5052]: I1203 11:50:12.759772 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3a49f2d7-59af-4bc8-8d05-641b33c7a24c-combined-ca-bundle\") pod \"placement-db-sync-nqvdz\" (UID: \"3a49f2d7-59af-4bc8-8d05-641b33c7a24c\") " pod="openstack-kuttl-tests/placement-db-sync-nqvdz" Dec 03 11:50:12 crc kubenswrapper[5052]: I1203 11:50:12.759813 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-dg59k\" (UniqueName: \"kubernetes.io/projected/3a49f2d7-59af-4bc8-8d05-641b33c7a24c-kube-api-access-dg59k\") pod \"placement-db-sync-nqvdz\" (UID: \"3a49f2d7-59af-4bc8-8d05-641b33c7a24c\") " pod="openstack-kuttl-tests/placement-db-sync-nqvdz" Dec 03 11:50:12 crc kubenswrapper[5052]: I1203 11:50:12.759846 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-7jnbj\" (UniqueName: \"kubernetes.io/projected/97107a61-a19d-4494-bcdb-a8d8f6223cb5-kube-api-access-7jnbj\") pod \"barbican-db-sync-847xp\" (UID: \"97107a61-a19d-4494-bcdb-a8d8f6223cb5\") " pod="openstack-kuttl-tests/barbican-db-sync-847xp" Dec 03 11:50:12 crc kubenswrapper[5052]: I1203 11:50:12.759902 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/666333b4-b787-4efa-9681-e6dc65f42542-log-httpd\") pod \"ceilometer-0\" (UID: \"666333b4-b787-4efa-9681-e6dc65f42542\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:50:12 crc kubenswrapper[5052]: I1203 11:50:12.759994 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/3a49f2d7-59af-4bc8-8d05-641b33c7a24c-scripts\") pod \"placement-db-sync-nqvdz\" (UID: \"3a49f2d7-59af-4bc8-8d05-641b33c7a24c\") " pod="openstack-kuttl-tests/placement-db-sync-nqvdz" Dec 03 11:50:12 crc kubenswrapper[5052]: I1203 11:50:12.760016 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7zxf6\" (UniqueName: \"kubernetes.io/projected/666333b4-b787-4efa-9681-e6dc65f42542-kube-api-access-7zxf6\") pod \"ceilometer-0\" (UID: \"666333b4-b787-4efa-9681-e6dc65f42542\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:50:12 crc kubenswrapper[5052]: I1203 11:50:12.760045 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/666333b4-b787-4efa-9681-e6dc65f42542-scripts\") pod \"ceilometer-0\" (UID: \"666333b4-b787-4efa-9681-e6dc65f42542\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:50:12 crc kubenswrapper[5052]: I1203 11:50:12.760071 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/3a49f2d7-59af-4bc8-8d05-641b33c7a24c-config-data\") pod \"placement-db-sync-nqvdz\" (UID: \"3a49f2d7-59af-4bc8-8d05-641b33c7a24c\") " pod="openstack-kuttl-tests/placement-db-sync-nqvdz" Dec 03 11:50:12 crc kubenswrapper[5052]: I1203 11:50:12.760146 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/666333b4-b787-4efa-9681-e6dc65f42542-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"666333b4-b787-4efa-9681-e6dc65f42542\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:50:12 crc kubenswrapper[5052]: I1203 11:50:12.760213 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/3a49f2d7-59af-4bc8-8d05-641b33c7a24c-logs\") pod \"placement-db-sync-nqvdz\" (UID: \"3a49f2d7-59af-4bc8-8d05-641b33c7a24c\") " pod="openstack-kuttl-tests/placement-db-sync-nqvdz" Dec 03 11:50:12 crc kubenswrapper[5052]: I1203 11:50:12.760259 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/97107a61-a19d-4494-bcdb-a8d8f6223cb5-combined-ca-bundle\") pod \"barbican-db-sync-847xp\" (UID: \"97107a61-a19d-4494-bcdb-a8d8f6223cb5\") " pod="openstack-kuttl-tests/barbican-db-sync-847xp" Dec 03 11:50:12 crc kubenswrapper[5052]: I1203 11:50:12.763045 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/3a49f2d7-59af-4bc8-8d05-641b33c7a24c-logs\") pod \"placement-db-sync-nqvdz\" (UID: \"3a49f2d7-59af-4bc8-8d05-641b33c7a24c\") " pod="openstack-kuttl-tests/placement-db-sync-nqvdz" Dec 03 11:50:12 crc kubenswrapper[5052]: I1203 11:50:12.771579 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/97107a61-a19d-4494-bcdb-a8d8f6223cb5-db-sync-config-data\") pod \"barbican-db-sync-847xp\" (UID: \"97107a61-a19d-4494-bcdb-a8d8f6223cb5\") " pod="openstack-kuttl-tests/barbican-db-sync-847xp" Dec 03 11:50:12 crc kubenswrapper[5052]: I1203 11:50:12.772534 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3a49f2d7-59af-4bc8-8d05-641b33c7a24c-combined-ca-bundle\") pod \"placement-db-sync-nqvdz\" (UID: \"3a49f2d7-59af-4bc8-8d05-641b33c7a24c\") " pod="openstack-kuttl-tests/placement-db-sync-nqvdz" Dec 03 11:50:12 crc kubenswrapper[5052]: I1203 11:50:12.785284 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/3a49f2d7-59af-4bc8-8d05-641b33c7a24c-config-data\") pod \"placement-db-sync-nqvdz\" (UID: \"3a49f2d7-59af-4bc8-8d05-641b33c7a24c\") " pod="openstack-kuttl-tests/placement-db-sync-nqvdz" Dec 03 11:50:12 crc kubenswrapper[5052]: I1203 11:50:12.786372 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/3a49f2d7-59af-4bc8-8d05-641b33c7a24c-scripts\") pod \"placement-db-sync-nqvdz\" (UID: \"3a49f2d7-59af-4bc8-8d05-641b33c7a24c\") " pod="openstack-kuttl-tests/placement-db-sync-nqvdz" Dec 03 11:50:12 crc kubenswrapper[5052]: I1203 11:50:12.788377 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/97107a61-a19d-4494-bcdb-a8d8f6223cb5-combined-ca-bundle\") pod \"barbican-db-sync-847xp\" (UID: \"97107a61-a19d-4494-bcdb-a8d8f6223cb5\") " pod="openstack-kuttl-tests/barbican-db-sync-847xp" Dec 03 11:50:12 crc kubenswrapper[5052]: I1203 11:50:12.789140 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:50:12 crc kubenswrapper[5052]: I1203 11:50:12.794126 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-7jnbj\" (UniqueName: \"kubernetes.io/projected/97107a61-a19d-4494-bcdb-a8d8f6223cb5-kube-api-access-7jnbj\") pod \"barbican-db-sync-847xp\" (UID: \"97107a61-a19d-4494-bcdb-a8d8f6223cb5\") " pod="openstack-kuttl-tests/barbican-db-sync-847xp" Dec 03 11:50:12 crc kubenswrapper[5052]: I1203 11:50:12.801126 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-dg59k\" (UniqueName: \"kubernetes.io/projected/3a49f2d7-59af-4bc8-8d05-641b33c7a24c-kube-api-access-dg59k\") pod \"placement-db-sync-nqvdz\" (UID: \"3a49f2d7-59af-4bc8-8d05-641b33c7a24c\") " pod="openstack-kuttl-tests/placement-db-sync-nqvdz" Dec 03 11:50:12 crc kubenswrapper[5052]: I1203 11:50:12.862807 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/666333b4-b787-4efa-9681-e6dc65f42542-log-httpd\") pod \"ceilometer-0\" (UID: \"666333b4-b787-4efa-9681-e6dc65f42542\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:50:12 crc kubenswrapper[5052]: I1203 11:50:12.863310 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-7zxf6\" (UniqueName: \"kubernetes.io/projected/666333b4-b787-4efa-9681-e6dc65f42542-kube-api-access-7zxf6\") pod \"ceilometer-0\" (UID: \"666333b4-b787-4efa-9681-e6dc65f42542\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:50:12 crc kubenswrapper[5052]: I1203 11:50:12.863361 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/666333b4-b787-4efa-9681-e6dc65f42542-scripts\") pod \"ceilometer-0\" (UID: \"666333b4-b787-4efa-9681-e6dc65f42542\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:50:12 crc kubenswrapper[5052]: I1203 11:50:12.863428 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/666333b4-b787-4efa-9681-e6dc65f42542-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"666333b4-b787-4efa-9681-e6dc65f42542\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:50:12 crc kubenswrapper[5052]: I1203 11:50:12.863911 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/666333b4-b787-4efa-9681-e6dc65f42542-config-data\") pod \"ceilometer-0\" (UID: \"666333b4-b787-4efa-9681-e6dc65f42542\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:50:12 crc kubenswrapper[5052]: I1203 11:50:12.863957 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/666333b4-b787-4efa-9681-e6dc65f42542-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"666333b4-b787-4efa-9681-e6dc65f42542\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:50:12 crc kubenswrapper[5052]: I1203 11:50:12.863983 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/666333b4-b787-4efa-9681-e6dc65f42542-run-httpd\") pod \"ceilometer-0\" (UID: \"666333b4-b787-4efa-9681-e6dc65f42542\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:50:12 crc kubenswrapper[5052]: I1203 11:50:12.865201 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/666333b4-b787-4efa-9681-e6dc65f42542-log-httpd\") pod \"ceilometer-0\" (UID: \"666333b4-b787-4efa-9681-e6dc65f42542\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:50:12 crc kubenswrapper[5052]: I1203 11:50:12.867188 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/666333b4-b787-4efa-9681-e6dc65f42542-run-httpd\") pod \"ceilometer-0\" (UID: \"666333b4-b787-4efa-9681-e6dc65f42542\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:50:12 crc kubenswrapper[5052]: I1203 11:50:12.870390 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/666333b4-b787-4efa-9681-e6dc65f42542-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"666333b4-b787-4efa-9681-e6dc65f42542\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:50:12 crc kubenswrapper[5052]: I1203 11:50:12.870562 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/666333b4-b787-4efa-9681-e6dc65f42542-scripts\") pod \"ceilometer-0\" (UID: \"666333b4-b787-4efa-9681-e6dc65f42542\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:50:12 crc kubenswrapper[5052]: I1203 11:50:12.870735 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/666333b4-b787-4efa-9681-e6dc65f42542-config-data\") pod \"ceilometer-0\" (UID: \"666333b4-b787-4efa-9681-e6dc65f42542\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:50:12 crc kubenswrapper[5052]: I1203 11:50:12.870760 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-db-sync-lgtfz" Dec 03 11:50:12 crc kubenswrapper[5052]: I1203 11:50:12.878706 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/666333b4-b787-4efa-9681-e6dc65f42542-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"666333b4-b787-4efa-9681-e6dc65f42542\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:50:12 crc kubenswrapper[5052]: I1203 11:50:12.883665 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-7zxf6\" (UniqueName: \"kubernetes.io/projected/666333b4-b787-4efa-9681-e6dc65f42542-kube-api-access-7zxf6\") pod \"ceilometer-0\" (UID: \"666333b4-b787-4efa-9681-e6dc65f42542\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:50:12 crc kubenswrapper[5052]: I1203 11:50:12.889989 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-db-sync-ghfwv" Dec 03 11:50:12 crc kubenswrapper[5052]: I1203 11:50:12.904079 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-db-sync-nqvdz" Dec 03 11:50:13 crc kubenswrapper[5052]: I1203 11:50:13.059126 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-db-sync-847xp" Dec 03 11:50:13 crc kubenswrapper[5052]: I1203 11:50:13.089415 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:50:13 crc kubenswrapper[5052]: I1203 11:50:13.140934 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-bootstrap-vbjb4"] Dec 03 11:50:13 crc kubenswrapper[5052]: I1203 11:50:13.253397 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/placement-db-sync-nqvdz"] Dec 03 11:50:13 crc kubenswrapper[5052]: I1203 11:50:13.346130 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-db-sync-lgtfz"] Dec 03 11:50:13 crc kubenswrapper[5052]: I1203 11:50:13.415942 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 11:50:13 crc kubenswrapper[5052]: I1203 11:50:13.417530 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:50:13 crc kubenswrapper[5052]: I1203 11:50:13.419671 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-default-external-config-data" Dec 03 11:50:13 crc kubenswrapper[5052]: I1203 11:50:13.419996 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-glance-dockercfg-zntkb" Dec 03 11:50:13 crc kubenswrapper[5052]: I1203 11:50:13.420123 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-glance-default-public-svc" Dec 03 11:50:13 crc kubenswrapper[5052]: I1203 11:50:13.420280 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-scripts" Dec 03 11:50:13 crc kubenswrapper[5052]: I1203 11:50:13.424488 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 11:50:13 crc kubenswrapper[5052]: I1203 11:50:13.514296 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 11:50:13 crc kubenswrapper[5052]: I1203 11:50:13.515608 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:50:13 crc kubenswrapper[5052]: I1203 11:50:13.525432 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-glance-default-internal-svc" Dec 03 11:50:13 crc kubenswrapper[5052]: I1203 11:50:13.525881 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-default-internal-config-data" Dec 03 11:50:13 crc kubenswrapper[5052]: I1203 11:50:13.550608 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 11:50:13 crc kubenswrapper[5052]: I1203 11:50:13.596971 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/7795810f-2574-4fd4-8761-a03317434f92-config-data\") pod \"glance-default-external-api-0\" (UID: \"7795810f-2574-4fd4-8761-a03317434f92\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:50:13 crc kubenswrapper[5052]: I1203 11:50:13.597042 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/7795810f-2574-4fd4-8761-a03317434f92-public-tls-certs\") pod \"glance-default-external-api-0\" (UID: \"7795810f-2574-4fd4-8761-a03317434f92\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:50:13 crc kubenswrapper[5052]: I1203 11:50:13.597061 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/7795810f-2574-4fd4-8761-a03317434f92-httpd-run\") pod \"glance-default-external-api-0\" (UID: \"7795810f-2574-4fd4-8761-a03317434f92\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:50:13 crc kubenswrapper[5052]: I1203 11:50:13.597104 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/7795810f-2574-4fd4-8761-a03317434f92-logs\") pod \"glance-default-external-api-0\" (UID: \"7795810f-2574-4fd4-8761-a03317434f92\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:50:13 crc kubenswrapper[5052]: I1203 11:50:13.597139 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/7795810f-2574-4fd4-8761-a03317434f92-scripts\") pod \"glance-default-external-api-0\" (UID: \"7795810f-2574-4fd4-8761-a03317434f92\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:50:13 crc kubenswrapper[5052]: I1203 11:50:13.597183 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lrbjj\" (UniqueName: \"kubernetes.io/projected/7795810f-2574-4fd4-8761-a03317434f92-kube-api-access-lrbjj\") pod \"glance-default-external-api-0\" (UID: \"7795810f-2574-4fd4-8761-a03317434f92\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:50:13 crc kubenswrapper[5052]: I1203 11:50:13.597204 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage09-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage09-crc\") pod \"glance-default-external-api-0\" (UID: \"7795810f-2574-4fd4-8761-a03317434f92\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:50:13 crc kubenswrapper[5052]: I1203 11:50:13.597218 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/7795810f-2574-4fd4-8761-a03317434f92-combined-ca-bundle\") pod \"glance-default-external-api-0\" (UID: \"7795810f-2574-4fd4-8761-a03317434f92\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:50:13 crc kubenswrapper[5052]: W1203 11:50:13.613857 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod97107a61_a19d_4494_bcdb_a8d8f6223cb5.slice/crio-5a2f374beafdeba46ec6fed9c5f9fcd800d1968d84009fd8fa6a84b47f6f9e7d WatchSource:0}: Error finding container 5a2f374beafdeba46ec6fed9c5f9fcd800d1968d84009fd8fa6a84b47f6f9e7d: Status 404 returned error can't find the container with id 5a2f374beafdeba46ec6fed9c5f9fcd800d1968d84009fd8fa6a84b47f6f9e7d Dec 03 11:50:13 crc kubenswrapper[5052]: I1203 11:50:13.619082 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-db-sync-847xp"] Dec 03 11:50:13 crc kubenswrapper[5052]: W1203 11:50:13.620145 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd6eac989_1eec_4b3a_b9d7_d5a8da2c1a27.slice/crio-f798e9ff4b47add1643ceb7425c2e5044ecab55dae14267b62eac3c86bf67a70 WatchSource:0}: Error finding container f798e9ff4b47add1643ceb7425c2e5044ecab55dae14267b62eac3c86bf67a70: Status 404 returned error can't find the container with id f798e9ff4b47add1643ceb7425c2e5044ecab55dae14267b62eac3c86bf67a70 Dec 03 11:50:13 crc kubenswrapper[5052]: I1203 11:50:13.642259 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-db-sync-ghfwv"] Dec 03 11:50:13 crc kubenswrapper[5052]: I1203 11:50:13.713022 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/30e3cc7a-6b64-4aa8-924e-6e195451067b-config-data\") pod \"glance-default-internal-api-0\" (UID: \"30e3cc7a-6b64-4aa8-924e-6e195451067b\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:50:13 crc kubenswrapper[5052]: I1203 11:50:13.713116 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/30e3cc7a-6b64-4aa8-924e-6e195451067b-scripts\") pod \"glance-default-internal-api-0\" (UID: \"30e3cc7a-6b64-4aa8-924e-6e195451067b\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:50:13 crc kubenswrapper[5052]: I1203 11:50:13.713169 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-lrbjj\" (UniqueName: \"kubernetes.io/projected/7795810f-2574-4fd4-8761-a03317434f92-kube-api-access-lrbjj\") pod \"glance-default-external-api-0\" (UID: \"7795810f-2574-4fd4-8761-a03317434f92\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:50:13 crc kubenswrapper[5052]: I1203 11:50:13.713231 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage09-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage09-crc\") pod \"glance-default-external-api-0\" (UID: \"7795810f-2574-4fd4-8761-a03317434f92\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:50:13 crc kubenswrapper[5052]: I1203 11:50:13.713267 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/7795810f-2574-4fd4-8761-a03317434f92-combined-ca-bundle\") pod \"glance-default-external-api-0\" (UID: \"7795810f-2574-4fd4-8761-a03317434f92\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:50:13 crc kubenswrapper[5052]: I1203 11:50:13.713327 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/7795810f-2574-4fd4-8761-a03317434f92-config-data\") pod \"glance-default-external-api-0\" (UID: \"7795810f-2574-4fd4-8761-a03317434f92\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:50:13 crc kubenswrapper[5052]: I1203 11:50:13.713388 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/7795810f-2574-4fd4-8761-a03317434f92-public-tls-certs\") pod \"glance-default-external-api-0\" (UID: \"7795810f-2574-4fd4-8761-a03317434f92\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:50:13 crc kubenswrapper[5052]: I1203 11:50:13.713428 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/7795810f-2574-4fd4-8761-a03317434f92-httpd-run\") pod \"glance-default-external-api-0\" (UID: \"7795810f-2574-4fd4-8761-a03317434f92\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:50:13 crc kubenswrapper[5052]: I1203 11:50:13.713470 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/30e3cc7a-6b64-4aa8-924e-6e195451067b-logs\") pod \"glance-default-internal-api-0\" (UID: \"30e3cc7a-6b64-4aa8-924e-6e195451067b\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:50:13 crc kubenswrapper[5052]: I1203 11:50:13.713584 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/7795810f-2574-4fd4-8761-a03317434f92-logs\") pod \"glance-default-external-api-0\" (UID: \"7795810f-2574-4fd4-8761-a03317434f92\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:50:13 crc kubenswrapper[5052]: I1203 11:50:13.713625 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/30e3cc7a-6b64-4aa8-924e-6e195451067b-internal-tls-certs\") pod \"glance-default-internal-api-0\" (UID: \"30e3cc7a-6b64-4aa8-924e-6e195451067b\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:50:13 crc kubenswrapper[5052]: I1203 11:50:13.713673 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/30e3cc7a-6b64-4aa8-924e-6e195451067b-combined-ca-bundle\") pod \"glance-default-internal-api-0\" (UID: \"30e3cc7a-6b64-4aa8-924e-6e195451067b\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:50:13 crc kubenswrapper[5052]: I1203 11:50:13.713704 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/30e3cc7a-6b64-4aa8-924e-6e195451067b-httpd-run\") pod \"glance-default-internal-api-0\" (UID: \"30e3cc7a-6b64-4aa8-924e-6e195451067b\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:50:13 crc kubenswrapper[5052]: I1203 11:50:13.713747 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/7795810f-2574-4fd4-8761-a03317434f92-scripts\") pod \"glance-default-external-api-0\" (UID: \"7795810f-2574-4fd4-8761-a03317434f92\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:50:13 crc kubenswrapper[5052]: I1203 11:50:13.713785 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rklwd\" (UniqueName: \"kubernetes.io/projected/30e3cc7a-6b64-4aa8-924e-6e195451067b-kube-api-access-rklwd\") pod \"glance-default-internal-api-0\" (UID: \"30e3cc7a-6b64-4aa8-924e-6e195451067b\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:50:13 crc kubenswrapper[5052]: I1203 11:50:13.713819 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage02-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage02-crc\") pod \"glance-default-internal-api-0\" (UID: \"30e3cc7a-6b64-4aa8-924e-6e195451067b\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:50:13 crc kubenswrapper[5052]: I1203 11:50:13.714108 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage09-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage09-crc\") pod \"glance-default-external-api-0\" (UID: \"7795810f-2574-4fd4-8761-a03317434f92\") device mount path \"/mnt/openstack/pv09\"" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:50:13 crc kubenswrapper[5052]: I1203 11:50:13.717352 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/7795810f-2574-4fd4-8761-a03317434f92-httpd-run\") pod \"glance-default-external-api-0\" (UID: \"7795810f-2574-4fd4-8761-a03317434f92\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:50:13 crc kubenswrapper[5052]: I1203 11:50:13.717608 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/7795810f-2574-4fd4-8761-a03317434f92-logs\") pod \"glance-default-external-api-0\" (UID: \"7795810f-2574-4fd4-8761-a03317434f92\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:50:13 crc kubenswrapper[5052]: I1203 11:50:13.720297 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/7795810f-2574-4fd4-8761-a03317434f92-config-data\") pod \"glance-default-external-api-0\" (UID: \"7795810f-2574-4fd4-8761-a03317434f92\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:50:13 crc kubenswrapper[5052]: I1203 11:50:13.724180 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/7795810f-2574-4fd4-8761-a03317434f92-scripts\") pod \"glance-default-external-api-0\" (UID: \"7795810f-2574-4fd4-8761-a03317434f92\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:50:13 crc kubenswrapper[5052]: I1203 11:50:13.725128 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/7795810f-2574-4fd4-8761-a03317434f92-combined-ca-bundle\") pod \"glance-default-external-api-0\" (UID: \"7795810f-2574-4fd4-8761-a03317434f92\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:50:13 crc kubenswrapper[5052]: I1203 11:50:13.725741 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/7795810f-2574-4fd4-8761-a03317434f92-public-tls-certs\") pod \"glance-default-external-api-0\" (UID: \"7795810f-2574-4fd4-8761-a03317434f92\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:50:13 crc kubenswrapper[5052]: I1203 11:50:13.732274 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-lrbjj\" (UniqueName: \"kubernetes.io/projected/7795810f-2574-4fd4-8761-a03317434f92-kube-api-access-lrbjj\") pod \"glance-default-external-api-0\" (UID: \"7795810f-2574-4fd4-8761-a03317434f92\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:50:13 crc kubenswrapper[5052]: I1203 11:50:13.756891 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage09-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage09-crc\") pod \"glance-default-external-api-0\" (UID: \"7795810f-2574-4fd4-8761-a03317434f92\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:50:13 crc kubenswrapper[5052]: I1203 11:50:13.815025 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/30e3cc7a-6b64-4aa8-924e-6e195451067b-combined-ca-bundle\") pod \"glance-default-internal-api-0\" (UID: \"30e3cc7a-6b64-4aa8-924e-6e195451067b\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:50:13 crc kubenswrapper[5052]: I1203 11:50:13.815347 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/30e3cc7a-6b64-4aa8-924e-6e195451067b-httpd-run\") pod \"glance-default-internal-api-0\" (UID: \"30e3cc7a-6b64-4aa8-924e-6e195451067b\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:50:13 crc kubenswrapper[5052]: I1203 11:50:13.815418 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-rklwd\" (UniqueName: \"kubernetes.io/projected/30e3cc7a-6b64-4aa8-924e-6e195451067b-kube-api-access-rklwd\") pod \"glance-default-internal-api-0\" (UID: \"30e3cc7a-6b64-4aa8-924e-6e195451067b\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:50:13 crc kubenswrapper[5052]: I1203 11:50:13.815449 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage02-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage02-crc\") pod \"glance-default-internal-api-0\" (UID: \"30e3cc7a-6b64-4aa8-924e-6e195451067b\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:50:13 crc kubenswrapper[5052]: I1203 11:50:13.815504 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/30e3cc7a-6b64-4aa8-924e-6e195451067b-config-data\") pod \"glance-default-internal-api-0\" (UID: \"30e3cc7a-6b64-4aa8-924e-6e195451067b\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:50:13 crc kubenswrapper[5052]: I1203 11:50:13.815555 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/30e3cc7a-6b64-4aa8-924e-6e195451067b-scripts\") pod \"glance-default-internal-api-0\" (UID: \"30e3cc7a-6b64-4aa8-924e-6e195451067b\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:50:13 crc kubenswrapper[5052]: I1203 11:50:13.815676 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/30e3cc7a-6b64-4aa8-924e-6e195451067b-logs\") pod \"glance-default-internal-api-0\" (UID: \"30e3cc7a-6b64-4aa8-924e-6e195451067b\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:50:13 crc kubenswrapper[5052]: I1203 11:50:13.815741 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/30e3cc7a-6b64-4aa8-924e-6e195451067b-internal-tls-certs\") pod \"glance-default-internal-api-0\" (UID: \"30e3cc7a-6b64-4aa8-924e-6e195451067b\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:50:13 crc kubenswrapper[5052]: I1203 11:50:13.818759 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage02-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage02-crc\") pod \"glance-default-internal-api-0\" (UID: \"30e3cc7a-6b64-4aa8-924e-6e195451067b\") device mount path \"/mnt/openstack/pv02\"" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:50:13 crc kubenswrapper[5052]: I1203 11:50:13.821665 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/30e3cc7a-6b64-4aa8-924e-6e195451067b-logs\") pod \"glance-default-internal-api-0\" (UID: \"30e3cc7a-6b64-4aa8-924e-6e195451067b\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:50:13 crc kubenswrapper[5052]: I1203 11:50:13.822077 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/30e3cc7a-6b64-4aa8-924e-6e195451067b-httpd-run\") pod \"glance-default-internal-api-0\" (UID: \"30e3cc7a-6b64-4aa8-924e-6e195451067b\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:50:13 crc kubenswrapper[5052]: I1203 11:50:13.822926 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/30e3cc7a-6b64-4aa8-924e-6e195451067b-config-data\") pod \"glance-default-internal-api-0\" (UID: \"30e3cc7a-6b64-4aa8-924e-6e195451067b\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:50:13 crc kubenswrapper[5052]: I1203 11:50:13.823024 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/30e3cc7a-6b64-4aa8-924e-6e195451067b-internal-tls-certs\") pod \"glance-default-internal-api-0\" (UID: \"30e3cc7a-6b64-4aa8-924e-6e195451067b\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:50:13 crc kubenswrapper[5052]: I1203 11:50:13.824329 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/30e3cc7a-6b64-4aa8-924e-6e195451067b-scripts\") pod \"glance-default-internal-api-0\" (UID: \"30e3cc7a-6b64-4aa8-924e-6e195451067b\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:50:13 crc kubenswrapper[5052]: I1203 11:50:13.826841 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/30e3cc7a-6b64-4aa8-924e-6e195451067b-combined-ca-bundle\") pod \"glance-default-internal-api-0\" (UID: \"30e3cc7a-6b64-4aa8-924e-6e195451067b\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:50:13 crc kubenswrapper[5052]: I1203 11:50:13.841542 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-rklwd\" (UniqueName: \"kubernetes.io/projected/30e3cc7a-6b64-4aa8-924e-6e195451067b-kube-api-access-rklwd\") pod \"glance-default-internal-api-0\" (UID: \"30e3cc7a-6b64-4aa8-924e-6e195451067b\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:50:13 crc kubenswrapper[5052]: I1203 11:50:13.856078 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:50:13 crc kubenswrapper[5052]: I1203 11:50:13.860866 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage02-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage02-crc\") pod \"glance-default-internal-api-0\" (UID: \"30e3cc7a-6b64-4aa8-924e-6e195451067b\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:50:13 crc kubenswrapper[5052]: W1203 11:50:13.941815 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod666333b4_b787_4efa_9681_e6dc65f42542.slice/crio-b3f23b53e27db5f3c7c6c40fb8600aba44f52b28e0eb44e5247fee3b74a063f8 WatchSource:0}: Error finding container b3f23b53e27db5f3c7c6c40fb8600aba44f52b28e0eb44e5247fee3b74a063f8: Status 404 returned error can't find the container with id b3f23b53e27db5f3c7c6c40fb8600aba44f52b28e0eb44e5247fee3b74a063f8 Dec 03 11:50:13 crc kubenswrapper[5052]: I1203 11:50:13.964040 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:50:13 crc kubenswrapper[5052]: I1203 11:50:13.968484 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:50:14 crc kubenswrapper[5052]: I1203 11:50:14.196619 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-db-sync-lgtfz" event={"ID":"5ddb757c-c235-4a2f-b6db-0d120215edcf","Type":"ContainerStarted","Data":"54c691cdb66a1ef8d58fcbeabe48d93756fdbed671d58afd1a0870677ec28549"} Dec 03 11:50:14 crc kubenswrapper[5052]: I1203 11:50:14.206754 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"666333b4-b787-4efa-9681-e6dc65f42542","Type":"ContainerStarted","Data":"b3f23b53e27db5f3c7c6c40fb8600aba44f52b28e0eb44e5247fee3b74a063f8"} Dec 03 11:50:14 crc kubenswrapper[5052]: I1203 11:50:14.213526 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-db-sync-nqvdz" event={"ID":"3a49f2d7-59af-4bc8-8d05-641b33c7a24c","Type":"ContainerStarted","Data":"d4027f144e15ab4f41b2a5336a39af9c79ec80a702f05271b93953e9587b9570"} Dec 03 11:50:14 crc kubenswrapper[5052]: I1203 11:50:14.215069 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-bootstrap-vbjb4" event={"ID":"20d3b16f-b0ce-4c24-b665-3be3a2da781a","Type":"ContainerStarted","Data":"ef0059d3a7b4a9c6fc5cb0bb9f3ef07ac616881c592a0fb5a6259ef33f4a5590"} Dec 03 11:50:14 crc kubenswrapper[5052]: I1203 11:50:14.218754 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-db-sync-847xp" event={"ID":"97107a61-a19d-4494-bcdb-a8d8f6223cb5","Type":"ContainerStarted","Data":"5a2f374beafdeba46ec6fed9c5f9fcd800d1968d84009fd8fa6a84b47f6f9e7d"} Dec 03 11:50:14 crc kubenswrapper[5052]: I1203 11:50:14.220649 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-db-sync-ghfwv" event={"ID":"d6eac989-1eec-4b3a-b9d7-d5a8da2c1a27","Type":"ContainerStarted","Data":"f798e9ff4b47add1643ceb7425c2e5044ecab55dae14267b62eac3c86bf67a70"} Dec 03 11:50:14 crc kubenswrapper[5052]: I1203 11:50:14.584648 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 11:50:14 crc kubenswrapper[5052]: I1203 11:50:14.698549 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 11:50:14 crc kubenswrapper[5052]: W1203 11:50:14.700373 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7795810f_2574_4fd4_8761_a03317434f92.slice/crio-48fbed42f91637bbfee35a6f8135d2292913ba44c810975fb5d89400c9f408da WatchSource:0}: Error finding container 48fbed42f91637bbfee35a6f8135d2292913ba44c810975fb5d89400c9f408da: Status 404 returned error can't find the container with id 48fbed42f91637bbfee35a6f8135d2292913ba44c810975fb5d89400c9f408da Dec 03 11:50:14 crc kubenswrapper[5052]: I1203 11:50:14.820058 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 11:50:14 crc kubenswrapper[5052]: I1203 11:50:14.918432 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:50:14 crc kubenswrapper[5052]: I1203 11:50:14.994872 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 11:50:15 crc kubenswrapper[5052]: I1203 11:50:15.245788 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-db-sync-nqvdz" event={"ID":"3a49f2d7-59af-4bc8-8d05-641b33c7a24c","Type":"ContainerStarted","Data":"4ca16aced64bf17827b7e9533988c8f6aa76a1efc00408bc2b421323bbd6026b"} Dec 03 11:50:15 crc kubenswrapper[5052]: I1203 11:50:15.248877 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-bootstrap-vbjb4" event={"ID":"20d3b16f-b0ce-4c24-b665-3be3a2da781a","Type":"ContainerStarted","Data":"97f15cadb1a96ebd3eddcbe674bed92f481485532cc8ed2bb6b1191062dafefa"} Dec 03 11:50:15 crc kubenswrapper[5052]: I1203 11:50:15.250910 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"7795810f-2574-4fd4-8761-a03317434f92","Type":"ContainerStarted","Data":"48fbed42f91637bbfee35a6f8135d2292913ba44c810975fb5d89400c9f408da"} Dec 03 11:50:15 crc kubenswrapper[5052]: I1203 11:50:15.254869 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-db-sync-847xp" event={"ID":"97107a61-a19d-4494-bcdb-a8d8f6223cb5","Type":"ContainerStarted","Data":"57ea218fb8f0a0038fe40dcb1ad5997273497a2c37f8b30405d5fc310990e84b"} Dec 03 11:50:15 crc kubenswrapper[5052]: I1203 11:50:15.264673 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-db-sync-ghfwv" event={"ID":"d6eac989-1eec-4b3a-b9d7-d5a8da2c1a27","Type":"ContainerStarted","Data":"ad2e20f86ff81ee90b61200f14867c2ba74e9e0bbf7e5ed7916c06cfc4183fad"} Dec 03 11:50:15 crc kubenswrapper[5052]: I1203 11:50:15.267405 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"30e3cc7a-6b64-4aa8-924e-6e195451067b","Type":"ContainerStarted","Data":"1511871fccd2bcf35d2e1c1f4ed48f90dadeeb04aa5a73da3ff77560df192897"} Dec 03 11:50:16 crc kubenswrapper[5052]: I1203 11:50:16.279361 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"30e3cc7a-6b64-4aa8-924e-6e195451067b","Type":"ContainerStarted","Data":"5cf799e5678d6cf72e32aadb81784fc1701308302bc719f80ca2ee2be0f8df50"} Dec 03 11:50:16 crc kubenswrapper[5052]: I1203 11:50:16.281331 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"7795810f-2574-4fd4-8761-a03317434f92","Type":"ContainerStarted","Data":"bcf0f42cb30509317e257b170f7b0606aaf126c6e75998ff5310aea5330477a6"} Dec 03 11:50:16 crc kubenswrapper[5052]: I1203 11:50:16.285485 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-db-sync-lgtfz" event={"ID":"5ddb757c-c235-4a2f-b6db-0d120215edcf","Type":"ContainerStarted","Data":"24721104cb9088d6a7d4f0ae127c54a770da6fbf71ba161bc9d6020ff94a6ce7"} Dec 03 11:50:16 crc kubenswrapper[5052]: I1203 11:50:16.307782 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/keystone-bootstrap-vbjb4" podStartSLOduration=4.307762439 podStartE2EDuration="4.307762439s" podCreationTimestamp="2025-12-03 11:50:12 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:50:16.307668096 +0000 UTC m=+1806.895909860" watchObservedRunningTime="2025-12-03 11:50:16.307762439 +0000 UTC m=+1806.896004223" Dec 03 11:50:16 crc kubenswrapper[5052]: I1203 11:50:16.341626 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/cinder-db-sync-lgtfz" podStartSLOduration=4.341585509 podStartE2EDuration="4.341585509s" podCreationTimestamp="2025-12-03 11:50:12 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:50:16.338790029 +0000 UTC m=+1806.927031793" watchObservedRunningTime="2025-12-03 11:50:16.341585509 +0000 UTC m=+1806.929827273" Dec 03 11:50:16 crc kubenswrapper[5052]: I1203 11:50:16.353991 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/placement-db-sync-nqvdz" podStartSLOduration=4.353974494 podStartE2EDuration="4.353974494s" podCreationTimestamp="2025-12-03 11:50:12 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:50:16.352708708 +0000 UTC m=+1806.940950472" watchObservedRunningTime="2025-12-03 11:50:16.353974494 +0000 UTC m=+1806.942216258" Dec 03 11:50:16 crc kubenswrapper[5052]: I1203 11:50:16.379791 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/neutron-db-sync-ghfwv" podStartSLOduration=4.379766164 podStartE2EDuration="4.379766164s" podCreationTimestamp="2025-12-03 11:50:12 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:50:16.371829076 +0000 UTC m=+1806.960070830" watchObservedRunningTime="2025-12-03 11:50:16.379766164 +0000 UTC m=+1806.968007928" Dec 03 11:50:16 crc kubenswrapper[5052]: I1203 11:50:16.405151 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/barbican-db-sync-847xp" podStartSLOduration=4.405119991 podStartE2EDuration="4.405119991s" podCreationTimestamp="2025-12-03 11:50:12 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:50:16.397661527 +0000 UTC m=+1806.985903301" watchObservedRunningTime="2025-12-03 11:50:16.405119991 +0000 UTC m=+1806.993361755" Dec 03 11:50:17 crc kubenswrapper[5052]: I1203 11:50:17.293567 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"7795810f-2574-4fd4-8761-a03317434f92","Type":"ContainerStarted","Data":"e2dcbec6b2e8c891b8578e50f9bbea6dfc08180b33648c434d680086a3f3176a"} Dec 03 11:50:17 crc kubenswrapper[5052]: I1203 11:50:17.295805 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"666333b4-b787-4efa-9681-e6dc65f42542","Type":"ContainerStarted","Data":"e81be556dd24de0c9fb321aafa9d8aaf412a0390f6ebf245b845b08b17a4b778"} Dec 03 11:50:17 crc kubenswrapper[5052]: I1203 11:50:17.673193 5052 scope.go:117] "RemoveContainer" containerID="25fe88f8d102f09983f8884e65f0c6c1575ade67921f10a630e6dcc3ea8b506e" Dec 03 11:50:17 crc kubenswrapper[5052]: E1203 11:50:17.673832 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 11:50:18 crc kubenswrapper[5052]: I1203 11:50:18.305269 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"30e3cc7a-6b64-4aa8-924e-6e195451067b","Type":"ContainerStarted","Data":"b69f65bd466afe1ddecaf6e8c8e421c99ec09f442299057fb0ff45ce1848f532"} Dec 03 11:50:18 crc kubenswrapper[5052]: I1203 11:50:18.305374 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-internal-api-0" podUID="30e3cc7a-6b64-4aa8-924e-6e195451067b" containerName="glance-log" containerID="cri-o://5cf799e5678d6cf72e32aadb81784fc1701308302bc719f80ca2ee2be0f8df50" gracePeriod=30 Dec 03 11:50:18 crc kubenswrapper[5052]: I1203 11:50:18.305441 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-external-api-0" podUID="7795810f-2574-4fd4-8761-a03317434f92" containerName="glance-log" containerID="cri-o://bcf0f42cb30509317e257b170f7b0606aaf126c6e75998ff5310aea5330477a6" gracePeriod=30 Dec 03 11:50:18 crc kubenswrapper[5052]: I1203 11:50:18.305528 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-external-api-0" podUID="7795810f-2574-4fd4-8761-a03317434f92" containerName="glance-httpd" containerID="cri-o://e2dcbec6b2e8c891b8578e50f9bbea6dfc08180b33648c434d680086a3f3176a" gracePeriod=30 Dec 03 11:50:18 crc kubenswrapper[5052]: I1203 11:50:18.305467 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-internal-api-0" podUID="30e3cc7a-6b64-4aa8-924e-6e195451067b" containerName="glance-httpd" containerID="cri-o://b69f65bd466afe1ddecaf6e8c8e421c99ec09f442299057fb0ff45ce1848f532" gracePeriod=30 Dec 03 11:50:18 crc kubenswrapper[5052]: I1203 11:50:18.348237 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/glance-default-external-api-0" podStartSLOduration=6.348214828 podStartE2EDuration="6.348214828s" podCreationTimestamp="2025-12-03 11:50:12 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:50:18.343557824 +0000 UTC m=+1808.931799628" watchObservedRunningTime="2025-12-03 11:50:18.348214828 +0000 UTC m=+1808.936456592" Dec 03 11:50:18 crc kubenswrapper[5052]: I1203 11:50:18.379597 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/glance-default-internal-api-0" podStartSLOduration=6.379576157 podStartE2EDuration="6.379576157s" podCreationTimestamp="2025-12-03 11:50:12 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:50:18.377945891 +0000 UTC m=+1808.966187665" watchObservedRunningTime="2025-12-03 11:50:18.379576157 +0000 UTC m=+1808.967817921" Dec 03 11:50:19 crc kubenswrapper[5052]: I1203 11:50:19.319204 5052 generic.go:334] "Generic (PLEG): container finished" podID="30e3cc7a-6b64-4aa8-924e-6e195451067b" containerID="5cf799e5678d6cf72e32aadb81784fc1701308302bc719f80ca2ee2be0f8df50" exitCode=143 Dec 03 11:50:19 crc kubenswrapper[5052]: I1203 11:50:19.319278 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"30e3cc7a-6b64-4aa8-924e-6e195451067b","Type":"ContainerDied","Data":"5cf799e5678d6cf72e32aadb81784fc1701308302bc719f80ca2ee2be0f8df50"} Dec 03 11:50:19 crc kubenswrapper[5052]: I1203 11:50:19.324845 5052 generic.go:334] "Generic (PLEG): container finished" podID="7795810f-2574-4fd4-8761-a03317434f92" containerID="bcf0f42cb30509317e257b170f7b0606aaf126c6e75998ff5310aea5330477a6" exitCode=143 Dec 03 11:50:19 crc kubenswrapper[5052]: I1203 11:50:19.324866 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"7795810f-2574-4fd4-8761-a03317434f92","Type":"ContainerDied","Data":"bcf0f42cb30509317e257b170f7b0606aaf126c6e75998ff5310aea5330477a6"} Dec 03 11:50:20 crc kubenswrapper[5052]: I1203 11:50:20.236899 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:50:20 crc kubenswrapper[5052]: I1203 11:50:20.244232 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:50:20 crc kubenswrapper[5052]: I1203 11:50:20.337998 5052 generic.go:334] "Generic (PLEG): container finished" podID="7795810f-2574-4fd4-8761-a03317434f92" containerID="e2dcbec6b2e8c891b8578e50f9bbea6dfc08180b33648c434d680086a3f3176a" exitCode=0 Dec 03 11:50:20 crc kubenswrapper[5052]: I1203 11:50:20.338077 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"7795810f-2574-4fd4-8761-a03317434f92","Type":"ContainerDied","Data":"e2dcbec6b2e8c891b8578e50f9bbea6dfc08180b33648c434d680086a3f3176a"} Dec 03 11:50:20 crc kubenswrapper[5052]: I1203 11:50:20.338114 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"7795810f-2574-4fd4-8761-a03317434f92","Type":"ContainerDied","Data":"48fbed42f91637bbfee35a6f8135d2292913ba44c810975fb5d89400c9f408da"} Dec 03 11:50:20 crc kubenswrapper[5052]: I1203 11:50:20.338135 5052 scope.go:117] "RemoveContainer" containerID="e2dcbec6b2e8c891b8578e50f9bbea6dfc08180b33648c434d680086a3f3176a" Dec 03 11:50:20 crc kubenswrapper[5052]: I1203 11:50:20.338256 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:50:20 crc kubenswrapper[5052]: I1203 11:50:20.341304 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/30e3cc7a-6b64-4aa8-924e-6e195451067b-scripts\") pod \"30e3cc7a-6b64-4aa8-924e-6e195451067b\" (UID: \"30e3cc7a-6b64-4aa8-924e-6e195451067b\") " Dec 03 11:50:20 crc kubenswrapper[5052]: I1203 11:50:20.341380 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/7795810f-2574-4fd4-8761-a03317434f92-scripts\") pod \"7795810f-2574-4fd4-8761-a03317434f92\" (UID: \"7795810f-2574-4fd4-8761-a03317434f92\") " Dec 03 11:50:20 crc kubenswrapper[5052]: I1203 11:50:20.341410 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lrbjj\" (UniqueName: \"kubernetes.io/projected/7795810f-2574-4fd4-8761-a03317434f92-kube-api-access-lrbjj\") pod \"7795810f-2574-4fd4-8761-a03317434f92\" (UID: \"7795810f-2574-4fd4-8761-a03317434f92\") " Dec 03 11:50:20 crc kubenswrapper[5052]: I1203 11:50:20.341451 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"glance\" (UniqueName: \"kubernetes.io/local-volume/local-storage02-crc\") pod \"30e3cc7a-6b64-4aa8-924e-6e195451067b\" (UID: \"30e3cc7a-6b64-4aa8-924e-6e195451067b\") " Dec 03 11:50:20 crc kubenswrapper[5052]: I1203 11:50:20.341485 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/30e3cc7a-6b64-4aa8-924e-6e195451067b-httpd-run\") pod \"30e3cc7a-6b64-4aa8-924e-6e195451067b\" (UID: \"30e3cc7a-6b64-4aa8-924e-6e195451067b\") " Dec 03 11:50:20 crc kubenswrapper[5052]: I1203 11:50:20.341536 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"glance\" (UniqueName: \"kubernetes.io/local-volume/local-storage09-crc\") pod \"7795810f-2574-4fd4-8761-a03317434f92\" (UID: \"7795810f-2574-4fd4-8761-a03317434f92\") " Dec 03 11:50:20 crc kubenswrapper[5052]: I1203 11:50:20.341578 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/30e3cc7a-6b64-4aa8-924e-6e195451067b-combined-ca-bundle\") pod \"30e3cc7a-6b64-4aa8-924e-6e195451067b\" (UID: \"30e3cc7a-6b64-4aa8-924e-6e195451067b\") " Dec 03 11:50:20 crc kubenswrapper[5052]: I1203 11:50:20.341617 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/7795810f-2574-4fd4-8761-a03317434f92-httpd-run\") pod \"7795810f-2574-4fd4-8761-a03317434f92\" (UID: \"7795810f-2574-4fd4-8761-a03317434f92\") " Dec 03 11:50:20 crc kubenswrapper[5052]: I1203 11:50:20.341678 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/30e3cc7a-6b64-4aa8-924e-6e195451067b-config-data\") pod \"30e3cc7a-6b64-4aa8-924e-6e195451067b\" (UID: \"30e3cc7a-6b64-4aa8-924e-6e195451067b\") " Dec 03 11:50:20 crc kubenswrapper[5052]: I1203 11:50:20.341721 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/7795810f-2574-4fd4-8761-a03317434f92-combined-ca-bundle\") pod \"7795810f-2574-4fd4-8761-a03317434f92\" (UID: \"7795810f-2574-4fd4-8761-a03317434f92\") " Dec 03 11:50:20 crc kubenswrapper[5052]: I1203 11:50:20.341745 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rklwd\" (UniqueName: \"kubernetes.io/projected/30e3cc7a-6b64-4aa8-924e-6e195451067b-kube-api-access-rklwd\") pod \"30e3cc7a-6b64-4aa8-924e-6e195451067b\" (UID: \"30e3cc7a-6b64-4aa8-924e-6e195451067b\") " Dec 03 11:50:20 crc kubenswrapper[5052]: I1203 11:50:20.341807 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/30e3cc7a-6b64-4aa8-924e-6e195451067b-internal-tls-certs\") pod \"30e3cc7a-6b64-4aa8-924e-6e195451067b\" (UID: \"30e3cc7a-6b64-4aa8-924e-6e195451067b\") " Dec 03 11:50:20 crc kubenswrapper[5052]: I1203 11:50:20.341841 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/30e3cc7a-6b64-4aa8-924e-6e195451067b-logs\") pod \"30e3cc7a-6b64-4aa8-924e-6e195451067b\" (UID: \"30e3cc7a-6b64-4aa8-924e-6e195451067b\") " Dec 03 11:50:20 crc kubenswrapper[5052]: I1203 11:50:20.341888 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/7795810f-2574-4fd4-8761-a03317434f92-public-tls-certs\") pod \"7795810f-2574-4fd4-8761-a03317434f92\" (UID: \"7795810f-2574-4fd4-8761-a03317434f92\") " Dec 03 11:50:20 crc kubenswrapper[5052]: I1203 11:50:20.341920 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/7795810f-2574-4fd4-8761-a03317434f92-logs\") pod \"7795810f-2574-4fd4-8761-a03317434f92\" (UID: \"7795810f-2574-4fd4-8761-a03317434f92\") " Dec 03 11:50:20 crc kubenswrapper[5052]: I1203 11:50:20.341944 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/7795810f-2574-4fd4-8761-a03317434f92-config-data\") pod \"7795810f-2574-4fd4-8761-a03317434f92\" (UID: \"7795810f-2574-4fd4-8761-a03317434f92\") " Dec 03 11:50:20 crc kubenswrapper[5052]: I1203 11:50:20.342813 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7795810f-2574-4fd4-8761-a03317434f92-httpd-run" (OuterVolumeSpecName: "httpd-run") pod "7795810f-2574-4fd4-8761-a03317434f92" (UID: "7795810f-2574-4fd4-8761-a03317434f92"). InnerVolumeSpecName "httpd-run". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:50:20 crc kubenswrapper[5052]: I1203 11:50:20.342989 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"666333b4-b787-4efa-9681-e6dc65f42542","Type":"ContainerStarted","Data":"6e0bf73696d9f9024b441df39c60ce218057276df5cc86577e519bd2072c4a62"} Dec 03 11:50:20 crc kubenswrapper[5052]: I1203 11:50:20.343315 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/30e3cc7a-6b64-4aa8-924e-6e195451067b-logs" (OuterVolumeSpecName: "logs") pod "30e3cc7a-6b64-4aa8-924e-6e195451067b" (UID: "30e3cc7a-6b64-4aa8-924e-6e195451067b"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:50:20 crc kubenswrapper[5052]: I1203 11:50:20.343727 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/30e3cc7a-6b64-4aa8-924e-6e195451067b-httpd-run" (OuterVolumeSpecName: "httpd-run") pod "30e3cc7a-6b64-4aa8-924e-6e195451067b" (UID: "30e3cc7a-6b64-4aa8-924e-6e195451067b"). InnerVolumeSpecName "httpd-run". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:50:20 crc kubenswrapper[5052]: I1203 11:50:20.343771 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7795810f-2574-4fd4-8761-a03317434f92-logs" (OuterVolumeSpecName: "logs") pod "7795810f-2574-4fd4-8761-a03317434f92" (UID: "7795810f-2574-4fd4-8761-a03317434f92"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:50:20 crc kubenswrapper[5052]: I1203 11:50:20.347162 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7795810f-2574-4fd4-8761-a03317434f92-kube-api-access-lrbjj" (OuterVolumeSpecName: "kube-api-access-lrbjj") pod "7795810f-2574-4fd4-8761-a03317434f92" (UID: "7795810f-2574-4fd4-8761-a03317434f92"). InnerVolumeSpecName "kube-api-access-lrbjj". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:50:20 crc kubenswrapper[5052]: I1203 11:50:20.347349 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage09-crc" (OuterVolumeSpecName: "glance") pod "7795810f-2574-4fd4-8761-a03317434f92" (UID: "7795810f-2574-4fd4-8761-a03317434f92"). InnerVolumeSpecName "local-storage09-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 11:50:20 crc kubenswrapper[5052]: I1203 11:50:20.348647 5052 generic.go:334] "Generic (PLEG): container finished" podID="30e3cc7a-6b64-4aa8-924e-6e195451067b" containerID="b69f65bd466afe1ddecaf6e8c8e421c99ec09f442299057fb0ff45ce1848f532" exitCode=0 Dec 03 11:50:20 crc kubenswrapper[5052]: I1203 11:50:20.348684 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"30e3cc7a-6b64-4aa8-924e-6e195451067b","Type":"ContainerDied","Data":"b69f65bd466afe1ddecaf6e8c8e421c99ec09f442299057fb0ff45ce1848f532"} Dec 03 11:50:20 crc kubenswrapper[5052]: I1203 11:50:20.348710 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"30e3cc7a-6b64-4aa8-924e-6e195451067b","Type":"ContainerDied","Data":"1511871fccd2bcf35d2e1c1f4ed48f90dadeeb04aa5a73da3ff77560df192897"} Dec 03 11:50:20 crc kubenswrapper[5052]: I1203 11:50:20.348713 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:50:20 crc kubenswrapper[5052]: I1203 11:50:20.350382 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage02-crc" (OuterVolumeSpecName: "glance") pod "30e3cc7a-6b64-4aa8-924e-6e195451067b" (UID: "30e3cc7a-6b64-4aa8-924e-6e195451067b"). InnerVolumeSpecName "local-storage02-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 11:50:20 crc kubenswrapper[5052]: I1203 11:50:20.350408 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/30e3cc7a-6b64-4aa8-924e-6e195451067b-scripts" (OuterVolumeSpecName: "scripts") pod "30e3cc7a-6b64-4aa8-924e-6e195451067b" (UID: "30e3cc7a-6b64-4aa8-924e-6e195451067b"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:50:20 crc kubenswrapper[5052]: I1203 11:50:20.350452 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7795810f-2574-4fd4-8761-a03317434f92-scripts" (OuterVolumeSpecName: "scripts") pod "7795810f-2574-4fd4-8761-a03317434f92" (UID: "7795810f-2574-4fd4-8761-a03317434f92"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:50:20 crc kubenswrapper[5052]: I1203 11:50:20.350757 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/30e3cc7a-6b64-4aa8-924e-6e195451067b-kube-api-access-rklwd" (OuterVolumeSpecName: "kube-api-access-rklwd") pod "30e3cc7a-6b64-4aa8-924e-6e195451067b" (UID: "30e3cc7a-6b64-4aa8-924e-6e195451067b"). InnerVolumeSpecName "kube-api-access-rklwd". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:50:20 crc kubenswrapper[5052]: I1203 11:50:20.384297 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7795810f-2574-4fd4-8761-a03317434f92-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "7795810f-2574-4fd4-8761-a03317434f92" (UID: "7795810f-2574-4fd4-8761-a03317434f92"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:50:20 crc kubenswrapper[5052]: I1203 11:50:20.384956 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/30e3cc7a-6b64-4aa8-924e-6e195451067b-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "30e3cc7a-6b64-4aa8-924e-6e195451067b" (UID: "30e3cc7a-6b64-4aa8-924e-6e195451067b"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:50:20 crc kubenswrapper[5052]: I1203 11:50:20.407256 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/30e3cc7a-6b64-4aa8-924e-6e195451067b-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "30e3cc7a-6b64-4aa8-924e-6e195451067b" (UID: "30e3cc7a-6b64-4aa8-924e-6e195451067b"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:50:20 crc kubenswrapper[5052]: I1203 11:50:20.410783 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7795810f-2574-4fd4-8761-a03317434f92-config-data" (OuterVolumeSpecName: "config-data") pod "7795810f-2574-4fd4-8761-a03317434f92" (UID: "7795810f-2574-4fd4-8761-a03317434f92"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:50:20 crc kubenswrapper[5052]: I1203 11:50:20.420706 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7795810f-2574-4fd4-8761-a03317434f92-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "7795810f-2574-4fd4-8761-a03317434f92" (UID: "7795810f-2574-4fd4-8761-a03317434f92"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:50:20 crc kubenswrapper[5052]: I1203 11:50:20.421017 5052 scope.go:117] "RemoveContainer" containerID="bcf0f42cb30509317e257b170f7b0606aaf126c6e75998ff5310aea5330477a6" Dec 03 11:50:20 crc kubenswrapper[5052]: I1203 11:50:20.438952 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/30e3cc7a-6b64-4aa8-924e-6e195451067b-config-data" (OuterVolumeSpecName: "config-data") pod "30e3cc7a-6b64-4aa8-924e-6e195451067b" (UID: "30e3cc7a-6b64-4aa8-924e-6e195451067b"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:50:20 crc kubenswrapper[5052]: I1203 11:50:20.443434 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/7795810f-2574-4fd4-8761-a03317434f92-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:50:20 crc kubenswrapper[5052]: I1203 11:50:20.443471 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-rklwd\" (UniqueName: \"kubernetes.io/projected/30e3cc7a-6b64-4aa8-924e-6e195451067b-kube-api-access-rklwd\") on node \"crc\" DevicePath \"\"" Dec 03 11:50:20 crc kubenswrapper[5052]: I1203 11:50:20.443484 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/30e3cc7a-6b64-4aa8-924e-6e195451067b-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 11:50:20 crc kubenswrapper[5052]: I1203 11:50:20.443492 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/30e3cc7a-6b64-4aa8-924e-6e195451067b-logs\") on node \"crc\" DevicePath \"\"" Dec 03 11:50:20 crc kubenswrapper[5052]: I1203 11:50:20.443501 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/7795810f-2574-4fd4-8761-a03317434f92-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 11:50:20 crc kubenswrapper[5052]: I1203 11:50:20.443530 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/7795810f-2574-4fd4-8761-a03317434f92-logs\") on node \"crc\" DevicePath \"\"" Dec 03 11:50:20 crc kubenswrapper[5052]: I1203 11:50:20.443561 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/7795810f-2574-4fd4-8761-a03317434f92-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:50:20 crc kubenswrapper[5052]: I1203 11:50:20.443569 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/30e3cc7a-6b64-4aa8-924e-6e195451067b-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:50:20 crc kubenswrapper[5052]: I1203 11:50:20.443578 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/7795810f-2574-4fd4-8761-a03317434f92-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:50:20 crc kubenswrapper[5052]: I1203 11:50:20.443586 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-lrbjj\" (UniqueName: \"kubernetes.io/projected/7795810f-2574-4fd4-8761-a03317434f92-kube-api-access-lrbjj\") on node \"crc\" DevicePath \"\"" Dec 03 11:50:20 crc kubenswrapper[5052]: I1203 11:50:20.443620 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage02-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage02-crc\") on node \"crc\" " Dec 03 11:50:20 crc kubenswrapper[5052]: I1203 11:50:20.443628 5052 reconciler_common.go:293] "Volume detached for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/30e3cc7a-6b64-4aa8-924e-6e195451067b-httpd-run\") on node \"crc\" DevicePath \"\"" Dec 03 11:50:20 crc kubenswrapper[5052]: I1203 11:50:20.443642 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage09-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage09-crc\") on node \"crc\" " Dec 03 11:50:20 crc kubenswrapper[5052]: I1203 11:50:20.443652 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/30e3cc7a-6b64-4aa8-924e-6e195451067b-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:50:20 crc kubenswrapper[5052]: I1203 11:50:20.443660 5052 reconciler_common.go:293] "Volume detached for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/7795810f-2574-4fd4-8761-a03317434f92-httpd-run\") on node \"crc\" DevicePath \"\"" Dec 03 11:50:20 crc kubenswrapper[5052]: I1203 11:50:20.443669 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/30e3cc7a-6b64-4aa8-924e-6e195451067b-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:50:20 crc kubenswrapper[5052]: I1203 11:50:20.462804 5052 scope.go:117] "RemoveContainer" containerID="e2dcbec6b2e8c891b8578e50f9bbea6dfc08180b33648c434d680086a3f3176a" Dec 03 11:50:20 crc kubenswrapper[5052]: E1203 11:50:20.463824 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e2dcbec6b2e8c891b8578e50f9bbea6dfc08180b33648c434d680086a3f3176a\": container with ID starting with e2dcbec6b2e8c891b8578e50f9bbea6dfc08180b33648c434d680086a3f3176a not found: ID does not exist" containerID="e2dcbec6b2e8c891b8578e50f9bbea6dfc08180b33648c434d680086a3f3176a" Dec 03 11:50:20 crc kubenswrapper[5052]: I1203 11:50:20.463869 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e2dcbec6b2e8c891b8578e50f9bbea6dfc08180b33648c434d680086a3f3176a"} err="failed to get container status \"e2dcbec6b2e8c891b8578e50f9bbea6dfc08180b33648c434d680086a3f3176a\": rpc error: code = NotFound desc = could not find container \"e2dcbec6b2e8c891b8578e50f9bbea6dfc08180b33648c434d680086a3f3176a\": container with ID starting with e2dcbec6b2e8c891b8578e50f9bbea6dfc08180b33648c434d680086a3f3176a not found: ID does not exist" Dec 03 11:50:20 crc kubenswrapper[5052]: I1203 11:50:20.463901 5052 scope.go:117] "RemoveContainer" containerID="bcf0f42cb30509317e257b170f7b0606aaf126c6e75998ff5310aea5330477a6" Dec 03 11:50:20 crc kubenswrapper[5052]: E1203 11:50:20.464245 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bcf0f42cb30509317e257b170f7b0606aaf126c6e75998ff5310aea5330477a6\": container with ID starting with bcf0f42cb30509317e257b170f7b0606aaf126c6e75998ff5310aea5330477a6 not found: ID does not exist" containerID="bcf0f42cb30509317e257b170f7b0606aaf126c6e75998ff5310aea5330477a6" Dec 03 11:50:20 crc kubenswrapper[5052]: I1203 11:50:20.464279 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bcf0f42cb30509317e257b170f7b0606aaf126c6e75998ff5310aea5330477a6"} err="failed to get container status \"bcf0f42cb30509317e257b170f7b0606aaf126c6e75998ff5310aea5330477a6\": rpc error: code = NotFound desc = could not find container \"bcf0f42cb30509317e257b170f7b0606aaf126c6e75998ff5310aea5330477a6\": container with ID starting with bcf0f42cb30509317e257b170f7b0606aaf126c6e75998ff5310aea5330477a6 not found: ID does not exist" Dec 03 11:50:20 crc kubenswrapper[5052]: I1203 11:50:20.464301 5052 scope.go:117] "RemoveContainer" containerID="b69f65bd466afe1ddecaf6e8c8e421c99ec09f442299057fb0ff45ce1848f532" Dec 03 11:50:20 crc kubenswrapper[5052]: I1203 11:50:20.468756 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage02-crc" (UniqueName: "kubernetes.io/local-volume/local-storage02-crc") on node "crc" Dec 03 11:50:20 crc kubenswrapper[5052]: I1203 11:50:20.478545 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage09-crc" (UniqueName: "kubernetes.io/local-volume/local-storage09-crc") on node "crc" Dec 03 11:50:20 crc kubenswrapper[5052]: I1203 11:50:20.523235 5052 scope.go:117] "RemoveContainer" containerID="5cf799e5678d6cf72e32aadb81784fc1701308302bc719f80ca2ee2be0f8df50" Dec 03 11:50:20 crc kubenswrapper[5052]: I1203 11:50:20.541206 5052 scope.go:117] "RemoveContainer" containerID="b69f65bd466afe1ddecaf6e8c8e421c99ec09f442299057fb0ff45ce1848f532" Dec 03 11:50:20 crc kubenswrapper[5052]: E1203 11:50:20.541876 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b69f65bd466afe1ddecaf6e8c8e421c99ec09f442299057fb0ff45ce1848f532\": container with ID starting with b69f65bd466afe1ddecaf6e8c8e421c99ec09f442299057fb0ff45ce1848f532 not found: ID does not exist" containerID="b69f65bd466afe1ddecaf6e8c8e421c99ec09f442299057fb0ff45ce1848f532" Dec 03 11:50:20 crc kubenswrapper[5052]: I1203 11:50:20.541909 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b69f65bd466afe1ddecaf6e8c8e421c99ec09f442299057fb0ff45ce1848f532"} err="failed to get container status \"b69f65bd466afe1ddecaf6e8c8e421c99ec09f442299057fb0ff45ce1848f532\": rpc error: code = NotFound desc = could not find container \"b69f65bd466afe1ddecaf6e8c8e421c99ec09f442299057fb0ff45ce1848f532\": container with ID starting with b69f65bd466afe1ddecaf6e8c8e421c99ec09f442299057fb0ff45ce1848f532 not found: ID does not exist" Dec 03 11:50:20 crc kubenswrapper[5052]: I1203 11:50:20.541950 5052 scope.go:117] "RemoveContainer" containerID="5cf799e5678d6cf72e32aadb81784fc1701308302bc719f80ca2ee2be0f8df50" Dec 03 11:50:20 crc kubenswrapper[5052]: E1203 11:50:20.542195 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5cf799e5678d6cf72e32aadb81784fc1701308302bc719f80ca2ee2be0f8df50\": container with ID starting with 5cf799e5678d6cf72e32aadb81784fc1701308302bc719f80ca2ee2be0f8df50 not found: ID does not exist" containerID="5cf799e5678d6cf72e32aadb81784fc1701308302bc719f80ca2ee2be0f8df50" Dec 03 11:50:20 crc kubenswrapper[5052]: I1203 11:50:20.542221 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5cf799e5678d6cf72e32aadb81784fc1701308302bc719f80ca2ee2be0f8df50"} err="failed to get container status \"5cf799e5678d6cf72e32aadb81784fc1701308302bc719f80ca2ee2be0f8df50\": rpc error: code = NotFound desc = could not find container \"5cf799e5678d6cf72e32aadb81784fc1701308302bc719f80ca2ee2be0f8df50\": container with ID starting with 5cf799e5678d6cf72e32aadb81784fc1701308302bc719f80ca2ee2be0f8df50 not found: ID does not exist" Dec 03 11:50:20 crc kubenswrapper[5052]: I1203 11:50:20.545275 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage02-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage02-crc\") on node \"crc\" DevicePath \"\"" Dec 03 11:50:20 crc kubenswrapper[5052]: I1203 11:50:20.545293 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage09-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage09-crc\") on node \"crc\" DevicePath \"\"" Dec 03 11:50:20 crc kubenswrapper[5052]: I1203 11:50:20.674540 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 11:50:20 crc kubenswrapper[5052]: I1203 11:50:20.691217 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 11:50:20 crc kubenswrapper[5052]: I1203 11:50:20.730910 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 11:50:20 crc kubenswrapper[5052]: E1203 11:50:20.731776 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="7795810f-2574-4fd4-8761-a03317434f92" containerName="glance-log" Dec 03 11:50:20 crc kubenswrapper[5052]: I1203 11:50:20.731798 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="7795810f-2574-4fd4-8761-a03317434f92" containerName="glance-log" Dec 03 11:50:20 crc kubenswrapper[5052]: E1203 11:50:20.731835 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="7795810f-2574-4fd4-8761-a03317434f92" containerName="glance-httpd" Dec 03 11:50:20 crc kubenswrapper[5052]: I1203 11:50:20.731841 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="7795810f-2574-4fd4-8761-a03317434f92" containerName="glance-httpd" Dec 03 11:50:20 crc kubenswrapper[5052]: E1203 11:50:20.731861 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="30e3cc7a-6b64-4aa8-924e-6e195451067b" containerName="glance-log" Dec 03 11:50:20 crc kubenswrapper[5052]: I1203 11:50:20.731869 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="30e3cc7a-6b64-4aa8-924e-6e195451067b" containerName="glance-log" Dec 03 11:50:20 crc kubenswrapper[5052]: E1203 11:50:20.731891 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="30e3cc7a-6b64-4aa8-924e-6e195451067b" containerName="glance-httpd" Dec 03 11:50:20 crc kubenswrapper[5052]: I1203 11:50:20.731897 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="30e3cc7a-6b64-4aa8-924e-6e195451067b" containerName="glance-httpd" Dec 03 11:50:20 crc kubenswrapper[5052]: I1203 11:50:20.732168 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="30e3cc7a-6b64-4aa8-924e-6e195451067b" containerName="glance-httpd" Dec 03 11:50:20 crc kubenswrapper[5052]: I1203 11:50:20.732198 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="7795810f-2574-4fd4-8761-a03317434f92" containerName="glance-httpd" Dec 03 11:50:20 crc kubenswrapper[5052]: I1203 11:50:20.732219 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="7795810f-2574-4fd4-8761-a03317434f92" containerName="glance-log" Dec 03 11:50:20 crc kubenswrapper[5052]: I1203 11:50:20.732231 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="30e3cc7a-6b64-4aa8-924e-6e195451067b" containerName="glance-log" Dec 03 11:50:20 crc kubenswrapper[5052]: I1203 11:50:20.733816 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:50:20 crc kubenswrapper[5052]: I1203 11:50:20.747678 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 11:50:20 crc kubenswrapper[5052]: I1203 11:50:20.766412 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-glance-default-public-svc" Dec 03 11:50:20 crc kubenswrapper[5052]: I1203 11:50:20.769938 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-scripts" Dec 03 11:50:20 crc kubenswrapper[5052]: I1203 11:50:20.770214 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-default-external-config-data" Dec 03 11:50:20 crc kubenswrapper[5052]: I1203 11:50:20.772658 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 11:50:20 crc kubenswrapper[5052]: I1203 11:50:20.773023 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-glance-dockercfg-zntkb" Dec 03 11:50:20 crc kubenswrapper[5052]: I1203 11:50:20.785551 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 11:50:20 crc kubenswrapper[5052]: I1203 11:50:20.801125 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 11:50:20 crc kubenswrapper[5052]: I1203 11:50:20.802681 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:50:20 crc kubenswrapper[5052]: I1203 11:50:20.806174 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-default-internal-config-data" Dec 03 11:50:20 crc kubenswrapper[5052]: I1203 11:50:20.806501 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-glance-default-internal-svc" Dec 03 11:50:20 crc kubenswrapper[5052]: I1203 11:50:20.811478 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 11:50:20 crc kubenswrapper[5052]: I1203 11:50:20.851382 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/5983840b-20da-49d2-aaa4-7be0eb36b0c4-scripts\") pod \"glance-default-external-api-0\" (UID: \"5983840b-20da-49d2-aaa4-7be0eb36b0c4\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:50:20 crc kubenswrapper[5052]: I1203 11:50:20.851582 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/5983840b-20da-49d2-aaa4-7be0eb36b0c4-httpd-run\") pod \"glance-default-external-api-0\" (UID: \"5983840b-20da-49d2-aaa4-7be0eb36b0c4\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:50:20 crc kubenswrapper[5052]: I1203 11:50:20.851612 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/760038e5-c49a-4c9e-b236-a4b98d58720e-internal-tls-certs\") pod \"glance-default-internal-api-0\" (UID: \"760038e5-c49a-4c9e-b236-a4b98d58720e\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:50:20 crc kubenswrapper[5052]: I1203 11:50:20.851636 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/5983840b-20da-49d2-aaa4-7be0eb36b0c4-logs\") pod \"glance-default-external-api-0\" (UID: \"5983840b-20da-49d2-aaa4-7be0eb36b0c4\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:50:20 crc kubenswrapper[5052]: I1203 11:50:20.851662 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/5983840b-20da-49d2-aaa4-7be0eb36b0c4-config-data\") pod \"glance-default-external-api-0\" (UID: \"5983840b-20da-49d2-aaa4-7be0eb36b0c4\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:50:20 crc kubenswrapper[5052]: I1203 11:50:20.851693 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/760038e5-c49a-4c9e-b236-a4b98d58720e-config-data\") pod \"glance-default-internal-api-0\" (UID: \"760038e5-c49a-4c9e-b236-a4b98d58720e\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:50:20 crc kubenswrapper[5052]: I1203 11:50:20.851714 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/760038e5-c49a-4c9e-b236-a4b98d58720e-scripts\") pod \"glance-default-internal-api-0\" (UID: \"760038e5-c49a-4c9e-b236-a4b98d58720e\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:50:20 crc kubenswrapper[5052]: I1203 11:50:20.851740 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage09-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage09-crc\") pod \"glance-default-external-api-0\" (UID: \"5983840b-20da-49d2-aaa4-7be0eb36b0c4\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:50:20 crc kubenswrapper[5052]: I1203 11:50:20.851788 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/760038e5-c49a-4c9e-b236-a4b98d58720e-httpd-run\") pod \"glance-default-internal-api-0\" (UID: \"760038e5-c49a-4c9e-b236-a4b98d58720e\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:50:20 crc kubenswrapper[5052]: I1203 11:50:20.851824 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/760038e5-c49a-4c9e-b236-a4b98d58720e-combined-ca-bundle\") pod \"glance-default-internal-api-0\" (UID: \"760038e5-c49a-4c9e-b236-a4b98d58720e\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:50:20 crc kubenswrapper[5052]: I1203 11:50:20.851875 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/5983840b-20da-49d2-aaa4-7be0eb36b0c4-public-tls-certs\") pod \"glance-default-external-api-0\" (UID: \"5983840b-20da-49d2-aaa4-7be0eb36b0c4\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:50:20 crc kubenswrapper[5052]: I1203 11:50:20.851908 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/760038e5-c49a-4c9e-b236-a4b98d58720e-logs\") pod \"glance-default-internal-api-0\" (UID: \"760038e5-c49a-4c9e-b236-a4b98d58720e\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:50:20 crc kubenswrapper[5052]: I1203 11:50:20.851948 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d6s7m\" (UniqueName: \"kubernetes.io/projected/760038e5-c49a-4c9e-b236-a4b98d58720e-kube-api-access-d6s7m\") pod \"glance-default-internal-api-0\" (UID: \"760038e5-c49a-4c9e-b236-a4b98d58720e\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:50:20 crc kubenswrapper[5052]: I1203 11:50:20.851985 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dtdbk\" (UniqueName: \"kubernetes.io/projected/5983840b-20da-49d2-aaa4-7be0eb36b0c4-kube-api-access-dtdbk\") pod \"glance-default-external-api-0\" (UID: \"5983840b-20da-49d2-aaa4-7be0eb36b0c4\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:50:20 crc kubenswrapper[5052]: I1203 11:50:20.852006 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5983840b-20da-49d2-aaa4-7be0eb36b0c4-combined-ca-bundle\") pod \"glance-default-external-api-0\" (UID: \"5983840b-20da-49d2-aaa4-7be0eb36b0c4\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:50:20 crc kubenswrapper[5052]: I1203 11:50:20.852034 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage02-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage02-crc\") pod \"glance-default-internal-api-0\" (UID: \"760038e5-c49a-4c9e-b236-a4b98d58720e\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:50:20 crc kubenswrapper[5052]: I1203 11:50:20.953342 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-dtdbk\" (UniqueName: \"kubernetes.io/projected/5983840b-20da-49d2-aaa4-7be0eb36b0c4-kube-api-access-dtdbk\") pod \"glance-default-external-api-0\" (UID: \"5983840b-20da-49d2-aaa4-7be0eb36b0c4\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:50:20 crc kubenswrapper[5052]: I1203 11:50:20.953395 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5983840b-20da-49d2-aaa4-7be0eb36b0c4-combined-ca-bundle\") pod \"glance-default-external-api-0\" (UID: \"5983840b-20da-49d2-aaa4-7be0eb36b0c4\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:50:20 crc kubenswrapper[5052]: I1203 11:50:20.953423 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage02-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage02-crc\") pod \"glance-default-internal-api-0\" (UID: \"760038e5-c49a-4c9e-b236-a4b98d58720e\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:50:20 crc kubenswrapper[5052]: I1203 11:50:20.953457 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/5983840b-20da-49d2-aaa4-7be0eb36b0c4-scripts\") pod \"glance-default-external-api-0\" (UID: \"5983840b-20da-49d2-aaa4-7be0eb36b0c4\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:50:20 crc kubenswrapper[5052]: I1203 11:50:20.953473 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/5983840b-20da-49d2-aaa4-7be0eb36b0c4-httpd-run\") pod \"glance-default-external-api-0\" (UID: \"5983840b-20da-49d2-aaa4-7be0eb36b0c4\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:50:20 crc kubenswrapper[5052]: I1203 11:50:20.953490 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/760038e5-c49a-4c9e-b236-a4b98d58720e-internal-tls-certs\") pod \"glance-default-internal-api-0\" (UID: \"760038e5-c49a-4c9e-b236-a4b98d58720e\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:50:20 crc kubenswrapper[5052]: I1203 11:50:20.953528 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/5983840b-20da-49d2-aaa4-7be0eb36b0c4-logs\") pod \"glance-default-external-api-0\" (UID: \"5983840b-20da-49d2-aaa4-7be0eb36b0c4\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:50:20 crc kubenswrapper[5052]: I1203 11:50:20.953555 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/5983840b-20da-49d2-aaa4-7be0eb36b0c4-config-data\") pod \"glance-default-external-api-0\" (UID: \"5983840b-20da-49d2-aaa4-7be0eb36b0c4\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:50:20 crc kubenswrapper[5052]: I1203 11:50:20.953584 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/760038e5-c49a-4c9e-b236-a4b98d58720e-config-data\") pod \"glance-default-internal-api-0\" (UID: \"760038e5-c49a-4c9e-b236-a4b98d58720e\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:50:20 crc kubenswrapper[5052]: I1203 11:50:20.953600 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/760038e5-c49a-4c9e-b236-a4b98d58720e-scripts\") pod \"glance-default-internal-api-0\" (UID: \"760038e5-c49a-4c9e-b236-a4b98d58720e\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:50:20 crc kubenswrapper[5052]: I1203 11:50:20.953619 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage09-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage09-crc\") pod \"glance-default-external-api-0\" (UID: \"5983840b-20da-49d2-aaa4-7be0eb36b0c4\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:50:20 crc kubenswrapper[5052]: I1203 11:50:20.953652 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/760038e5-c49a-4c9e-b236-a4b98d58720e-httpd-run\") pod \"glance-default-internal-api-0\" (UID: \"760038e5-c49a-4c9e-b236-a4b98d58720e\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:50:20 crc kubenswrapper[5052]: I1203 11:50:20.953677 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/760038e5-c49a-4c9e-b236-a4b98d58720e-combined-ca-bundle\") pod \"glance-default-internal-api-0\" (UID: \"760038e5-c49a-4c9e-b236-a4b98d58720e\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:50:20 crc kubenswrapper[5052]: I1203 11:50:20.953722 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/5983840b-20da-49d2-aaa4-7be0eb36b0c4-public-tls-certs\") pod \"glance-default-external-api-0\" (UID: \"5983840b-20da-49d2-aaa4-7be0eb36b0c4\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:50:20 crc kubenswrapper[5052]: I1203 11:50:20.953743 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/760038e5-c49a-4c9e-b236-a4b98d58720e-logs\") pod \"glance-default-internal-api-0\" (UID: \"760038e5-c49a-4c9e-b236-a4b98d58720e\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:50:20 crc kubenswrapper[5052]: I1203 11:50:20.953772 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-d6s7m\" (UniqueName: \"kubernetes.io/projected/760038e5-c49a-4c9e-b236-a4b98d58720e-kube-api-access-d6s7m\") pod \"glance-default-internal-api-0\" (UID: \"760038e5-c49a-4c9e-b236-a4b98d58720e\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:50:20 crc kubenswrapper[5052]: I1203 11:50:20.955359 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/760038e5-c49a-4c9e-b236-a4b98d58720e-httpd-run\") pod \"glance-default-internal-api-0\" (UID: \"760038e5-c49a-4c9e-b236-a4b98d58720e\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:50:20 crc kubenswrapper[5052]: I1203 11:50:20.955665 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage02-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage02-crc\") pod \"glance-default-internal-api-0\" (UID: \"760038e5-c49a-4c9e-b236-a4b98d58720e\") device mount path \"/mnt/openstack/pv02\"" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:50:20 crc kubenswrapper[5052]: I1203 11:50:20.955699 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage09-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage09-crc\") pod \"glance-default-external-api-0\" (UID: \"5983840b-20da-49d2-aaa4-7be0eb36b0c4\") device mount path \"/mnt/openstack/pv09\"" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:50:20 crc kubenswrapper[5052]: I1203 11:50:20.955746 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/760038e5-c49a-4c9e-b236-a4b98d58720e-logs\") pod \"glance-default-internal-api-0\" (UID: \"760038e5-c49a-4c9e-b236-a4b98d58720e\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:50:20 crc kubenswrapper[5052]: I1203 11:50:20.957042 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/5983840b-20da-49d2-aaa4-7be0eb36b0c4-httpd-run\") pod \"glance-default-external-api-0\" (UID: \"5983840b-20da-49d2-aaa4-7be0eb36b0c4\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:50:20 crc kubenswrapper[5052]: I1203 11:50:20.957116 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/5983840b-20da-49d2-aaa4-7be0eb36b0c4-logs\") pod \"glance-default-external-api-0\" (UID: \"5983840b-20da-49d2-aaa4-7be0eb36b0c4\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:50:20 crc kubenswrapper[5052]: I1203 11:50:20.959568 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5983840b-20da-49d2-aaa4-7be0eb36b0c4-combined-ca-bundle\") pod \"glance-default-external-api-0\" (UID: \"5983840b-20da-49d2-aaa4-7be0eb36b0c4\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:50:20 crc kubenswrapper[5052]: I1203 11:50:20.959631 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/5983840b-20da-49d2-aaa4-7be0eb36b0c4-public-tls-certs\") pod \"glance-default-external-api-0\" (UID: \"5983840b-20da-49d2-aaa4-7be0eb36b0c4\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:50:20 crc kubenswrapper[5052]: I1203 11:50:20.960586 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/760038e5-c49a-4c9e-b236-a4b98d58720e-internal-tls-certs\") pod \"glance-default-internal-api-0\" (UID: \"760038e5-c49a-4c9e-b236-a4b98d58720e\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:50:20 crc kubenswrapper[5052]: I1203 11:50:20.968945 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/5983840b-20da-49d2-aaa4-7be0eb36b0c4-scripts\") pod \"glance-default-external-api-0\" (UID: \"5983840b-20da-49d2-aaa4-7be0eb36b0c4\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:50:20 crc kubenswrapper[5052]: I1203 11:50:20.969498 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/760038e5-c49a-4c9e-b236-a4b98d58720e-combined-ca-bundle\") pod \"glance-default-internal-api-0\" (UID: \"760038e5-c49a-4c9e-b236-a4b98d58720e\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:50:20 crc kubenswrapper[5052]: I1203 11:50:20.969870 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/760038e5-c49a-4c9e-b236-a4b98d58720e-config-data\") pod \"glance-default-internal-api-0\" (UID: \"760038e5-c49a-4c9e-b236-a4b98d58720e\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:50:20 crc kubenswrapper[5052]: I1203 11:50:20.969994 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/760038e5-c49a-4c9e-b236-a4b98d58720e-scripts\") pod \"glance-default-internal-api-0\" (UID: \"760038e5-c49a-4c9e-b236-a4b98d58720e\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:50:20 crc kubenswrapper[5052]: I1203 11:50:20.972458 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-d6s7m\" (UniqueName: \"kubernetes.io/projected/760038e5-c49a-4c9e-b236-a4b98d58720e-kube-api-access-d6s7m\") pod \"glance-default-internal-api-0\" (UID: \"760038e5-c49a-4c9e-b236-a4b98d58720e\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:50:20 crc kubenswrapper[5052]: I1203 11:50:20.975903 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/5983840b-20da-49d2-aaa4-7be0eb36b0c4-config-data\") pod \"glance-default-external-api-0\" (UID: \"5983840b-20da-49d2-aaa4-7be0eb36b0c4\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:50:20 crc kubenswrapper[5052]: I1203 11:50:20.985582 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-dtdbk\" (UniqueName: \"kubernetes.io/projected/5983840b-20da-49d2-aaa4-7be0eb36b0c4-kube-api-access-dtdbk\") pod \"glance-default-external-api-0\" (UID: \"5983840b-20da-49d2-aaa4-7be0eb36b0c4\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:50:20 crc kubenswrapper[5052]: I1203 11:50:20.990814 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage09-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage09-crc\") pod \"glance-default-external-api-0\" (UID: \"5983840b-20da-49d2-aaa4-7be0eb36b0c4\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:50:20 crc kubenswrapper[5052]: I1203 11:50:20.992709 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage02-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage02-crc\") pod \"glance-default-internal-api-0\" (UID: \"760038e5-c49a-4c9e-b236-a4b98d58720e\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:50:21 crc kubenswrapper[5052]: I1203 11:50:21.068232 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:50:21 crc kubenswrapper[5052]: I1203 11:50:21.130116 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:50:21 crc kubenswrapper[5052]: I1203 11:50:21.605689 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 11:50:21 crc kubenswrapper[5052]: I1203 11:50:21.684809 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="30e3cc7a-6b64-4aa8-924e-6e195451067b" path="/var/lib/kubelet/pods/30e3cc7a-6b64-4aa8-924e-6e195451067b/volumes" Dec 03 11:50:21 crc kubenswrapper[5052]: I1203 11:50:21.686127 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7795810f-2574-4fd4-8761-a03317434f92" path="/var/lib/kubelet/pods/7795810f-2574-4fd4-8761-a03317434f92/volumes" Dec 03 11:50:21 crc kubenswrapper[5052]: W1203 11:50:21.721792 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5983840b_20da_49d2_aaa4_7be0eb36b0c4.slice/crio-e6c2bd43cdad3d2765bde41c0efd46bd1d126d31699d04edbb3daf8f80d5783f WatchSource:0}: Error finding container e6c2bd43cdad3d2765bde41c0efd46bd1d126d31699d04edbb3daf8f80d5783f: Status 404 returned error can't find the container with id e6c2bd43cdad3d2765bde41c0efd46bd1d126d31699d04edbb3daf8f80d5783f Dec 03 11:50:21 crc kubenswrapper[5052]: I1203 11:50:21.732025 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 11:50:22 crc kubenswrapper[5052]: I1203 11:50:22.377778 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"760038e5-c49a-4c9e-b236-a4b98d58720e","Type":"ContainerStarted","Data":"d0fbfa82077cfcd4d66763e4991cff3fbd800f1420c599ba5d5c5c25bf98d078"} Dec 03 11:50:22 crc kubenswrapper[5052]: I1203 11:50:22.379612 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"5983840b-20da-49d2-aaa4-7be0eb36b0c4","Type":"ContainerStarted","Data":"e6c2bd43cdad3d2765bde41c0efd46bd1d126d31699d04edbb3daf8f80d5783f"} Dec 03 11:50:23 crc kubenswrapper[5052]: I1203 11:50:23.389063 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"666333b4-b787-4efa-9681-e6dc65f42542","Type":"ContainerStarted","Data":"36d042927e39fe3c63e7087185065a06016fb4bef2a36bbd768bcb479ab5a83d"} Dec 03 11:50:24 crc kubenswrapper[5052]: I1203 11:50:24.401817 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"760038e5-c49a-4c9e-b236-a4b98d58720e","Type":"ContainerStarted","Data":"3087941b5ebefca5a16d78bd326e84502e4ef383a38ad5c7e648b61e640cc09f"} Dec 03 11:50:24 crc kubenswrapper[5052]: I1203 11:50:24.403819 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"5983840b-20da-49d2-aaa4-7be0eb36b0c4","Type":"ContainerStarted","Data":"6ed3a69c28ece77aa5d36034ff2cca030b4a7f6251d240bbd51afcf3f4a9773d"} Dec 03 11:50:27 crc kubenswrapper[5052]: I1203 11:50:27.435746 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"760038e5-c49a-4c9e-b236-a4b98d58720e","Type":"ContainerStarted","Data":"3d55abecfc0f796ab8e6c6031b93876ae418a7296b5273ed839b6fe1ac3cc9eb"} Dec 03 11:50:27 crc kubenswrapper[5052]: I1203 11:50:27.438066 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"5983840b-20da-49d2-aaa4-7be0eb36b0c4","Type":"ContainerStarted","Data":"6d7e2f9948e6e8fb4bfee608b3eeaee15339ef55c9a1c50f68aa577ee7ae17b4"} Dec 03 11:50:27 crc kubenswrapper[5052]: I1203 11:50:27.456397 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/glance-default-internal-api-0" podStartSLOduration=7.4563786499999996 podStartE2EDuration="7.45637865s" podCreationTimestamp="2025-12-03 11:50:20 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:50:27.453466316 +0000 UTC m=+1818.041708080" watchObservedRunningTime="2025-12-03 11:50:27.45637865 +0000 UTC m=+1818.044620414" Dec 03 11:50:27 crc kubenswrapper[5052]: I1203 11:50:27.511641 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/glance-default-external-api-0" podStartSLOduration=7.511623404 podStartE2EDuration="7.511623404s" podCreationTimestamp="2025-12-03 11:50:20 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:50:27.499091925 +0000 UTC m=+1818.087333739" watchObservedRunningTime="2025-12-03 11:50:27.511623404 +0000 UTC m=+1818.099865158" Dec 03 11:50:28 crc kubenswrapper[5052]: I1203 11:50:28.448679 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"666333b4-b787-4efa-9681-e6dc65f42542","Type":"ContainerStarted","Data":"b9e1a4f16e0e745d7e4576d1aa52f8eb1f3ed932ba9faf9370693ff78039f909"} Dec 03 11:50:28 crc kubenswrapper[5052]: I1203 11:50:28.449070 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="666333b4-b787-4efa-9681-e6dc65f42542" containerName="ceilometer-central-agent" containerID="cri-o://e81be556dd24de0c9fb321aafa9d8aaf412a0390f6ebf245b845b08b17a4b778" gracePeriod=30 Dec 03 11:50:28 crc kubenswrapper[5052]: I1203 11:50:28.449171 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="666333b4-b787-4efa-9681-e6dc65f42542" containerName="proxy-httpd" containerID="cri-o://b9e1a4f16e0e745d7e4576d1aa52f8eb1f3ed932ba9faf9370693ff78039f909" gracePeriod=30 Dec 03 11:50:28 crc kubenswrapper[5052]: I1203 11:50:28.449210 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="666333b4-b787-4efa-9681-e6dc65f42542" containerName="sg-core" containerID="cri-o://36d042927e39fe3c63e7087185065a06016fb4bef2a36bbd768bcb479ab5a83d" gracePeriod=30 Dec 03 11:50:28 crc kubenswrapper[5052]: I1203 11:50:28.449240 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="666333b4-b787-4efa-9681-e6dc65f42542" containerName="ceilometer-notification-agent" containerID="cri-o://6e0bf73696d9f9024b441df39c60ce218057276df5cc86577e519bd2072c4a62" gracePeriod=30 Dec 03 11:50:28 crc kubenswrapper[5052]: I1203 11:50:28.474476 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/ceilometer-0" podStartSLOduration=2.965317146 podStartE2EDuration="16.474456707s" podCreationTimestamp="2025-12-03 11:50:12 +0000 UTC" firstStartedPulling="2025-12-03 11:50:13.957715009 +0000 UTC m=+1804.545956773" lastFinishedPulling="2025-12-03 11:50:27.46685457 +0000 UTC m=+1818.055096334" observedRunningTime="2025-12-03 11:50:28.469246488 +0000 UTC m=+1819.057488262" watchObservedRunningTime="2025-12-03 11:50:28.474456707 +0000 UTC m=+1819.062698491" Dec 03 11:50:29 crc kubenswrapper[5052]: I1203 11:50:29.465602 5052 generic.go:334] "Generic (PLEG): container finished" podID="666333b4-b787-4efa-9681-e6dc65f42542" containerID="b9e1a4f16e0e745d7e4576d1aa52f8eb1f3ed932ba9faf9370693ff78039f909" exitCode=0 Dec 03 11:50:29 crc kubenswrapper[5052]: I1203 11:50:29.465660 5052 generic.go:334] "Generic (PLEG): container finished" podID="666333b4-b787-4efa-9681-e6dc65f42542" containerID="36d042927e39fe3c63e7087185065a06016fb4bef2a36bbd768bcb479ab5a83d" exitCode=2 Dec 03 11:50:29 crc kubenswrapper[5052]: I1203 11:50:29.465672 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"666333b4-b787-4efa-9681-e6dc65f42542","Type":"ContainerDied","Data":"b9e1a4f16e0e745d7e4576d1aa52f8eb1f3ed932ba9faf9370693ff78039f909"} Dec 03 11:50:29 crc kubenswrapper[5052]: I1203 11:50:29.465743 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"666333b4-b787-4efa-9681-e6dc65f42542","Type":"ContainerDied","Data":"36d042927e39fe3c63e7087185065a06016fb4bef2a36bbd768bcb479ab5a83d"} Dec 03 11:50:30 crc kubenswrapper[5052]: I1203 11:50:30.429246 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:50:30 crc kubenswrapper[5052]: I1203 11:50:30.456606 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/666333b4-b787-4efa-9681-e6dc65f42542-scripts\") pod \"666333b4-b787-4efa-9681-e6dc65f42542\" (UID: \"666333b4-b787-4efa-9681-e6dc65f42542\") " Dec 03 11:50:30 crc kubenswrapper[5052]: I1203 11:50:30.456671 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/666333b4-b787-4efa-9681-e6dc65f42542-combined-ca-bundle\") pod \"666333b4-b787-4efa-9681-e6dc65f42542\" (UID: \"666333b4-b787-4efa-9681-e6dc65f42542\") " Dec 03 11:50:30 crc kubenswrapper[5052]: I1203 11:50:30.456690 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/666333b4-b787-4efa-9681-e6dc65f42542-config-data\") pod \"666333b4-b787-4efa-9681-e6dc65f42542\" (UID: \"666333b4-b787-4efa-9681-e6dc65f42542\") " Dec 03 11:50:30 crc kubenswrapper[5052]: I1203 11:50:30.456739 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7zxf6\" (UniqueName: \"kubernetes.io/projected/666333b4-b787-4efa-9681-e6dc65f42542-kube-api-access-7zxf6\") pod \"666333b4-b787-4efa-9681-e6dc65f42542\" (UID: \"666333b4-b787-4efa-9681-e6dc65f42542\") " Dec 03 11:50:30 crc kubenswrapper[5052]: I1203 11:50:30.456783 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/666333b4-b787-4efa-9681-e6dc65f42542-run-httpd\") pod \"666333b4-b787-4efa-9681-e6dc65f42542\" (UID: \"666333b4-b787-4efa-9681-e6dc65f42542\") " Dec 03 11:50:30 crc kubenswrapper[5052]: I1203 11:50:30.456800 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/666333b4-b787-4efa-9681-e6dc65f42542-log-httpd\") pod \"666333b4-b787-4efa-9681-e6dc65f42542\" (UID: \"666333b4-b787-4efa-9681-e6dc65f42542\") " Dec 03 11:50:30 crc kubenswrapper[5052]: I1203 11:50:30.456831 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/666333b4-b787-4efa-9681-e6dc65f42542-sg-core-conf-yaml\") pod \"666333b4-b787-4efa-9681-e6dc65f42542\" (UID: \"666333b4-b787-4efa-9681-e6dc65f42542\") " Dec 03 11:50:30 crc kubenswrapper[5052]: I1203 11:50:30.457193 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/666333b4-b787-4efa-9681-e6dc65f42542-run-httpd" (OuterVolumeSpecName: "run-httpd") pod "666333b4-b787-4efa-9681-e6dc65f42542" (UID: "666333b4-b787-4efa-9681-e6dc65f42542"). InnerVolumeSpecName "run-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:50:30 crc kubenswrapper[5052]: I1203 11:50:30.457480 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/666333b4-b787-4efa-9681-e6dc65f42542-log-httpd" (OuterVolumeSpecName: "log-httpd") pod "666333b4-b787-4efa-9681-e6dc65f42542" (UID: "666333b4-b787-4efa-9681-e6dc65f42542"). InnerVolumeSpecName "log-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:50:30 crc kubenswrapper[5052]: I1203 11:50:30.472681 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/666333b4-b787-4efa-9681-e6dc65f42542-scripts" (OuterVolumeSpecName: "scripts") pod "666333b4-b787-4efa-9681-e6dc65f42542" (UID: "666333b4-b787-4efa-9681-e6dc65f42542"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:50:30 crc kubenswrapper[5052]: I1203 11:50:30.480659 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/666333b4-b787-4efa-9681-e6dc65f42542-kube-api-access-7zxf6" (OuterVolumeSpecName: "kube-api-access-7zxf6") pod "666333b4-b787-4efa-9681-e6dc65f42542" (UID: "666333b4-b787-4efa-9681-e6dc65f42542"). InnerVolumeSpecName "kube-api-access-7zxf6". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:50:30 crc kubenswrapper[5052]: I1203 11:50:30.498079 5052 generic.go:334] "Generic (PLEG): container finished" podID="666333b4-b787-4efa-9681-e6dc65f42542" containerID="6e0bf73696d9f9024b441df39c60ce218057276df5cc86577e519bd2072c4a62" exitCode=0 Dec 03 11:50:30 crc kubenswrapper[5052]: I1203 11:50:30.498108 5052 generic.go:334] "Generic (PLEG): container finished" podID="666333b4-b787-4efa-9681-e6dc65f42542" containerID="e81be556dd24de0c9fb321aafa9d8aaf412a0390f6ebf245b845b08b17a4b778" exitCode=0 Dec 03 11:50:30 crc kubenswrapper[5052]: I1203 11:50:30.498128 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"666333b4-b787-4efa-9681-e6dc65f42542","Type":"ContainerDied","Data":"6e0bf73696d9f9024b441df39c60ce218057276df5cc86577e519bd2072c4a62"} Dec 03 11:50:30 crc kubenswrapper[5052]: I1203 11:50:30.498157 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"666333b4-b787-4efa-9681-e6dc65f42542","Type":"ContainerDied","Data":"e81be556dd24de0c9fb321aafa9d8aaf412a0390f6ebf245b845b08b17a4b778"} Dec 03 11:50:30 crc kubenswrapper[5052]: I1203 11:50:30.498185 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"666333b4-b787-4efa-9681-e6dc65f42542","Type":"ContainerDied","Data":"b3f23b53e27db5f3c7c6c40fb8600aba44f52b28e0eb44e5247fee3b74a063f8"} Dec 03 11:50:30 crc kubenswrapper[5052]: I1203 11:50:30.498204 5052 scope.go:117] "RemoveContainer" containerID="b9e1a4f16e0e745d7e4576d1aa52f8eb1f3ed932ba9faf9370693ff78039f909" Dec 03 11:50:30 crc kubenswrapper[5052]: I1203 11:50:30.498395 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:50:30 crc kubenswrapper[5052]: I1203 11:50:30.502637 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/666333b4-b787-4efa-9681-e6dc65f42542-sg-core-conf-yaml" (OuterVolumeSpecName: "sg-core-conf-yaml") pod "666333b4-b787-4efa-9681-e6dc65f42542" (UID: "666333b4-b787-4efa-9681-e6dc65f42542"). InnerVolumeSpecName "sg-core-conf-yaml". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:50:30 crc kubenswrapper[5052]: I1203 11:50:30.550403 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/666333b4-b787-4efa-9681-e6dc65f42542-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "666333b4-b787-4efa-9681-e6dc65f42542" (UID: "666333b4-b787-4efa-9681-e6dc65f42542"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:50:30 crc kubenswrapper[5052]: I1203 11:50:30.559662 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-7zxf6\" (UniqueName: \"kubernetes.io/projected/666333b4-b787-4efa-9681-e6dc65f42542-kube-api-access-7zxf6\") on node \"crc\" DevicePath \"\"" Dec 03 11:50:30 crc kubenswrapper[5052]: I1203 11:50:30.559694 5052 reconciler_common.go:293] "Volume detached for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/666333b4-b787-4efa-9681-e6dc65f42542-run-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 11:50:30 crc kubenswrapper[5052]: I1203 11:50:30.559703 5052 reconciler_common.go:293] "Volume detached for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/666333b4-b787-4efa-9681-e6dc65f42542-log-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 11:50:30 crc kubenswrapper[5052]: I1203 11:50:30.559712 5052 reconciler_common.go:293] "Volume detached for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/666333b4-b787-4efa-9681-e6dc65f42542-sg-core-conf-yaml\") on node \"crc\" DevicePath \"\"" Dec 03 11:50:30 crc kubenswrapper[5052]: I1203 11:50:30.559721 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/666333b4-b787-4efa-9681-e6dc65f42542-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:50:30 crc kubenswrapper[5052]: I1203 11:50:30.559729 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/666333b4-b787-4efa-9681-e6dc65f42542-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:50:30 crc kubenswrapper[5052]: I1203 11:50:30.593959 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/666333b4-b787-4efa-9681-e6dc65f42542-config-data" (OuterVolumeSpecName: "config-data") pod "666333b4-b787-4efa-9681-e6dc65f42542" (UID: "666333b4-b787-4efa-9681-e6dc65f42542"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:50:30 crc kubenswrapper[5052]: I1203 11:50:30.603082 5052 scope.go:117] "RemoveContainer" containerID="36d042927e39fe3c63e7087185065a06016fb4bef2a36bbd768bcb479ab5a83d" Dec 03 11:50:30 crc kubenswrapper[5052]: I1203 11:50:30.626815 5052 scope.go:117] "RemoveContainer" containerID="6e0bf73696d9f9024b441df39c60ce218057276df5cc86577e519bd2072c4a62" Dec 03 11:50:30 crc kubenswrapper[5052]: I1203 11:50:30.647085 5052 scope.go:117] "RemoveContainer" containerID="e81be556dd24de0c9fb321aafa9d8aaf412a0390f6ebf245b845b08b17a4b778" Dec 03 11:50:30 crc kubenswrapper[5052]: I1203 11:50:30.661168 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/666333b4-b787-4efa-9681-e6dc65f42542-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:50:30 crc kubenswrapper[5052]: I1203 11:50:30.677612 5052 scope.go:117] "RemoveContainer" containerID="b9e1a4f16e0e745d7e4576d1aa52f8eb1f3ed932ba9faf9370693ff78039f909" Dec 03 11:50:30 crc kubenswrapper[5052]: E1203 11:50:30.678163 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b9e1a4f16e0e745d7e4576d1aa52f8eb1f3ed932ba9faf9370693ff78039f909\": container with ID starting with b9e1a4f16e0e745d7e4576d1aa52f8eb1f3ed932ba9faf9370693ff78039f909 not found: ID does not exist" containerID="b9e1a4f16e0e745d7e4576d1aa52f8eb1f3ed932ba9faf9370693ff78039f909" Dec 03 11:50:30 crc kubenswrapper[5052]: I1203 11:50:30.678215 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b9e1a4f16e0e745d7e4576d1aa52f8eb1f3ed932ba9faf9370693ff78039f909"} err="failed to get container status \"b9e1a4f16e0e745d7e4576d1aa52f8eb1f3ed932ba9faf9370693ff78039f909\": rpc error: code = NotFound desc = could not find container \"b9e1a4f16e0e745d7e4576d1aa52f8eb1f3ed932ba9faf9370693ff78039f909\": container with ID starting with b9e1a4f16e0e745d7e4576d1aa52f8eb1f3ed932ba9faf9370693ff78039f909 not found: ID does not exist" Dec 03 11:50:30 crc kubenswrapper[5052]: I1203 11:50:30.678249 5052 scope.go:117] "RemoveContainer" containerID="36d042927e39fe3c63e7087185065a06016fb4bef2a36bbd768bcb479ab5a83d" Dec 03 11:50:30 crc kubenswrapper[5052]: E1203 11:50:30.678759 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"36d042927e39fe3c63e7087185065a06016fb4bef2a36bbd768bcb479ab5a83d\": container with ID starting with 36d042927e39fe3c63e7087185065a06016fb4bef2a36bbd768bcb479ab5a83d not found: ID does not exist" containerID="36d042927e39fe3c63e7087185065a06016fb4bef2a36bbd768bcb479ab5a83d" Dec 03 11:50:30 crc kubenswrapper[5052]: I1203 11:50:30.678803 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"36d042927e39fe3c63e7087185065a06016fb4bef2a36bbd768bcb479ab5a83d"} err="failed to get container status \"36d042927e39fe3c63e7087185065a06016fb4bef2a36bbd768bcb479ab5a83d\": rpc error: code = NotFound desc = could not find container \"36d042927e39fe3c63e7087185065a06016fb4bef2a36bbd768bcb479ab5a83d\": container with ID starting with 36d042927e39fe3c63e7087185065a06016fb4bef2a36bbd768bcb479ab5a83d not found: ID does not exist" Dec 03 11:50:30 crc kubenswrapper[5052]: I1203 11:50:30.678828 5052 scope.go:117] "RemoveContainer" containerID="6e0bf73696d9f9024b441df39c60ce218057276df5cc86577e519bd2072c4a62" Dec 03 11:50:30 crc kubenswrapper[5052]: E1203 11:50:30.679131 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6e0bf73696d9f9024b441df39c60ce218057276df5cc86577e519bd2072c4a62\": container with ID starting with 6e0bf73696d9f9024b441df39c60ce218057276df5cc86577e519bd2072c4a62 not found: ID does not exist" containerID="6e0bf73696d9f9024b441df39c60ce218057276df5cc86577e519bd2072c4a62" Dec 03 11:50:30 crc kubenswrapper[5052]: I1203 11:50:30.679151 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6e0bf73696d9f9024b441df39c60ce218057276df5cc86577e519bd2072c4a62"} err="failed to get container status \"6e0bf73696d9f9024b441df39c60ce218057276df5cc86577e519bd2072c4a62\": rpc error: code = NotFound desc = could not find container \"6e0bf73696d9f9024b441df39c60ce218057276df5cc86577e519bd2072c4a62\": container with ID starting with 6e0bf73696d9f9024b441df39c60ce218057276df5cc86577e519bd2072c4a62 not found: ID does not exist" Dec 03 11:50:30 crc kubenswrapper[5052]: I1203 11:50:30.679167 5052 scope.go:117] "RemoveContainer" containerID="e81be556dd24de0c9fb321aafa9d8aaf412a0390f6ebf245b845b08b17a4b778" Dec 03 11:50:30 crc kubenswrapper[5052]: E1203 11:50:30.679387 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e81be556dd24de0c9fb321aafa9d8aaf412a0390f6ebf245b845b08b17a4b778\": container with ID starting with e81be556dd24de0c9fb321aafa9d8aaf412a0390f6ebf245b845b08b17a4b778 not found: ID does not exist" containerID="e81be556dd24de0c9fb321aafa9d8aaf412a0390f6ebf245b845b08b17a4b778" Dec 03 11:50:30 crc kubenswrapper[5052]: I1203 11:50:30.679417 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e81be556dd24de0c9fb321aafa9d8aaf412a0390f6ebf245b845b08b17a4b778"} err="failed to get container status \"e81be556dd24de0c9fb321aafa9d8aaf412a0390f6ebf245b845b08b17a4b778\": rpc error: code = NotFound desc = could not find container \"e81be556dd24de0c9fb321aafa9d8aaf412a0390f6ebf245b845b08b17a4b778\": container with ID starting with e81be556dd24de0c9fb321aafa9d8aaf412a0390f6ebf245b845b08b17a4b778 not found: ID does not exist" Dec 03 11:50:30 crc kubenswrapper[5052]: I1203 11:50:30.679437 5052 scope.go:117] "RemoveContainer" containerID="b9e1a4f16e0e745d7e4576d1aa52f8eb1f3ed932ba9faf9370693ff78039f909" Dec 03 11:50:30 crc kubenswrapper[5052]: I1203 11:50:30.679913 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b9e1a4f16e0e745d7e4576d1aa52f8eb1f3ed932ba9faf9370693ff78039f909"} err="failed to get container status \"b9e1a4f16e0e745d7e4576d1aa52f8eb1f3ed932ba9faf9370693ff78039f909\": rpc error: code = NotFound desc = could not find container \"b9e1a4f16e0e745d7e4576d1aa52f8eb1f3ed932ba9faf9370693ff78039f909\": container with ID starting with b9e1a4f16e0e745d7e4576d1aa52f8eb1f3ed932ba9faf9370693ff78039f909 not found: ID does not exist" Dec 03 11:50:30 crc kubenswrapper[5052]: I1203 11:50:30.679938 5052 scope.go:117] "RemoveContainer" containerID="36d042927e39fe3c63e7087185065a06016fb4bef2a36bbd768bcb479ab5a83d" Dec 03 11:50:30 crc kubenswrapper[5052]: I1203 11:50:30.680305 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"36d042927e39fe3c63e7087185065a06016fb4bef2a36bbd768bcb479ab5a83d"} err="failed to get container status \"36d042927e39fe3c63e7087185065a06016fb4bef2a36bbd768bcb479ab5a83d\": rpc error: code = NotFound desc = could not find container \"36d042927e39fe3c63e7087185065a06016fb4bef2a36bbd768bcb479ab5a83d\": container with ID starting with 36d042927e39fe3c63e7087185065a06016fb4bef2a36bbd768bcb479ab5a83d not found: ID does not exist" Dec 03 11:50:30 crc kubenswrapper[5052]: I1203 11:50:30.680328 5052 scope.go:117] "RemoveContainer" containerID="6e0bf73696d9f9024b441df39c60ce218057276df5cc86577e519bd2072c4a62" Dec 03 11:50:30 crc kubenswrapper[5052]: I1203 11:50:30.680601 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6e0bf73696d9f9024b441df39c60ce218057276df5cc86577e519bd2072c4a62"} err="failed to get container status \"6e0bf73696d9f9024b441df39c60ce218057276df5cc86577e519bd2072c4a62\": rpc error: code = NotFound desc = could not find container \"6e0bf73696d9f9024b441df39c60ce218057276df5cc86577e519bd2072c4a62\": container with ID starting with 6e0bf73696d9f9024b441df39c60ce218057276df5cc86577e519bd2072c4a62 not found: ID does not exist" Dec 03 11:50:30 crc kubenswrapper[5052]: I1203 11:50:30.680618 5052 scope.go:117] "RemoveContainer" containerID="e81be556dd24de0c9fb321aafa9d8aaf412a0390f6ebf245b845b08b17a4b778" Dec 03 11:50:30 crc kubenswrapper[5052]: I1203 11:50:30.680886 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e81be556dd24de0c9fb321aafa9d8aaf412a0390f6ebf245b845b08b17a4b778"} err="failed to get container status \"e81be556dd24de0c9fb321aafa9d8aaf412a0390f6ebf245b845b08b17a4b778\": rpc error: code = NotFound desc = could not find container \"e81be556dd24de0c9fb321aafa9d8aaf412a0390f6ebf245b845b08b17a4b778\": container with ID starting with e81be556dd24de0c9fb321aafa9d8aaf412a0390f6ebf245b845b08b17a4b778 not found: ID does not exist" Dec 03 11:50:30 crc kubenswrapper[5052]: I1203 11:50:30.851756 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:50:30 crc kubenswrapper[5052]: I1203 11:50:30.859606 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:50:30 crc kubenswrapper[5052]: I1203 11:50:30.864934 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:50:30 crc kubenswrapper[5052]: E1203 11:50:30.865282 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="666333b4-b787-4efa-9681-e6dc65f42542" containerName="proxy-httpd" Dec 03 11:50:30 crc kubenswrapper[5052]: I1203 11:50:30.865298 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="666333b4-b787-4efa-9681-e6dc65f42542" containerName="proxy-httpd" Dec 03 11:50:30 crc kubenswrapper[5052]: E1203 11:50:30.865307 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="666333b4-b787-4efa-9681-e6dc65f42542" containerName="ceilometer-central-agent" Dec 03 11:50:30 crc kubenswrapper[5052]: I1203 11:50:30.865313 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="666333b4-b787-4efa-9681-e6dc65f42542" containerName="ceilometer-central-agent" Dec 03 11:50:30 crc kubenswrapper[5052]: E1203 11:50:30.865336 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="666333b4-b787-4efa-9681-e6dc65f42542" containerName="ceilometer-notification-agent" Dec 03 11:50:30 crc kubenswrapper[5052]: I1203 11:50:30.865342 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="666333b4-b787-4efa-9681-e6dc65f42542" containerName="ceilometer-notification-agent" Dec 03 11:50:30 crc kubenswrapper[5052]: E1203 11:50:30.865352 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="666333b4-b787-4efa-9681-e6dc65f42542" containerName="sg-core" Dec 03 11:50:30 crc kubenswrapper[5052]: I1203 11:50:30.865357 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="666333b4-b787-4efa-9681-e6dc65f42542" containerName="sg-core" Dec 03 11:50:30 crc kubenswrapper[5052]: I1203 11:50:30.865518 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="666333b4-b787-4efa-9681-e6dc65f42542" containerName="ceilometer-central-agent" Dec 03 11:50:30 crc kubenswrapper[5052]: I1203 11:50:30.865528 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="666333b4-b787-4efa-9681-e6dc65f42542" containerName="ceilometer-notification-agent" Dec 03 11:50:30 crc kubenswrapper[5052]: I1203 11:50:30.865539 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="666333b4-b787-4efa-9681-e6dc65f42542" containerName="sg-core" Dec 03 11:50:30 crc kubenswrapper[5052]: I1203 11:50:30.865555 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="666333b4-b787-4efa-9681-e6dc65f42542" containerName="proxy-httpd" Dec 03 11:50:30 crc kubenswrapper[5052]: I1203 11:50:30.866980 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:50:30 crc kubenswrapper[5052]: I1203 11:50:30.870308 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-scripts" Dec 03 11:50:30 crc kubenswrapper[5052]: I1203 11:50:30.870650 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-config-data" Dec 03 11:50:30 crc kubenswrapper[5052]: I1203 11:50:30.899082 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:50:30 crc kubenswrapper[5052]: I1203 11:50:30.965947 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ksbm4\" (UniqueName: \"kubernetes.io/projected/f8c1750d-cf4a-49cc-a5ba-d40f19a6aad8-kube-api-access-ksbm4\") pod \"ceilometer-0\" (UID: \"f8c1750d-cf4a-49cc-a5ba-d40f19a6aad8\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:50:30 crc kubenswrapper[5052]: I1203 11:50:30.966043 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/f8c1750d-cf4a-49cc-a5ba-d40f19a6aad8-log-httpd\") pod \"ceilometer-0\" (UID: \"f8c1750d-cf4a-49cc-a5ba-d40f19a6aad8\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:50:30 crc kubenswrapper[5052]: I1203 11:50:30.966083 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f8c1750d-cf4a-49cc-a5ba-d40f19a6aad8-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"f8c1750d-cf4a-49cc-a5ba-d40f19a6aad8\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:50:30 crc kubenswrapper[5052]: I1203 11:50:30.966134 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/f8c1750d-cf4a-49cc-a5ba-d40f19a6aad8-scripts\") pod \"ceilometer-0\" (UID: \"f8c1750d-cf4a-49cc-a5ba-d40f19a6aad8\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:50:30 crc kubenswrapper[5052]: I1203 11:50:30.966186 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/f8c1750d-cf4a-49cc-a5ba-d40f19a6aad8-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"f8c1750d-cf4a-49cc-a5ba-d40f19a6aad8\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:50:30 crc kubenswrapper[5052]: I1203 11:50:30.966223 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/f8c1750d-cf4a-49cc-a5ba-d40f19a6aad8-run-httpd\") pod \"ceilometer-0\" (UID: \"f8c1750d-cf4a-49cc-a5ba-d40f19a6aad8\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:50:30 crc kubenswrapper[5052]: I1203 11:50:30.966245 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/f8c1750d-cf4a-49cc-a5ba-d40f19a6aad8-config-data\") pod \"ceilometer-0\" (UID: \"f8c1750d-cf4a-49cc-a5ba-d40f19a6aad8\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:50:31 crc kubenswrapper[5052]: I1203 11:50:31.068446 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:50:31 crc kubenswrapper[5052]: I1203 11:50:31.068557 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f8c1750d-cf4a-49cc-a5ba-d40f19a6aad8-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"f8c1750d-cf4a-49cc-a5ba-d40f19a6aad8\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:50:31 crc kubenswrapper[5052]: I1203 11:50:31.068705 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/f8c1750d-cf4a-49cc-a5ba-d40f19a6aad8-scripts\") pod \"ceilometer-0\" (UID: \"f8c1750d-cf4a-49cc-a5ba-d40f19a6aad8\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:50:31 crc kubenswrapper[5052]: I1203 11:50:31.068861 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/f8c1750d-cf4a-49cc-a5ba-d40f19a6aad8-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"f8c1750d-cf4a-49cc-a5ba-d40f19a6aad8\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:50:31 crc kubenswrapper[5052]: I1203 11:50:31.068953 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/f8c1750d-cf4a-49cc-a5ba-d40f19a6aad8-run-httpd\") pod \"ceilometer-0\" (UID: \"f8c1750d-cf4a-49cc-a5ba-d40f19a6aad8\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:50:31 crc kubenswrapper[5052]: I1203 11:50:31.069000 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:50:31 crc kubenswrapper[5052]: I1203 11:50:31.069016 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/f8c1750d-cf4a-49cc-a5ba-d40f19a6aad8-config-data\") pod \"ceilometer-0\" (UID: \"f8c1750d-cf4a-49cc-a5ba-d40f19a6aad8\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:50:31 crc kubenswrapper[5052]: I1203 11:50:31.069105 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-ksbm4\" (UniqueName: \"kubernetes.io/projected/f8c1750d-cf4a-49cc-a5ba-d40f19a6aad8-kube-api-access-ksbm4\") pod \"ceilometer-0\" (UID: \"f8c1750d-cf4a-49cc-a5ba-d40f19a6aad8\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:50:31 crc kubenswrapper[5052]: I1203 11:50:31.069214 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/f8c1750d-cf4a-49cc-a5ba-d40f19a6aad8-log-httpd\") pod \"ceilometer-0\" (UID: \"f8c1750d-cf4a-49cc-a5ba-d40f19a6aad8\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:50:31 crc kubenswrapper[5052]: I1203 11:50:31.069502 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/f8c1750d-cf4a-49cc-a5ba-d40f19a6aad8-run-httpd\") pod \"ceilometer-0\" (UID: \"f8c1750d-cf4a-49cc-a5ba-d40f19a6aad8\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:50:31 crc kubenswrapper[5052]: I1203 11:50:31.070016 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/f8c1750d-cf4a-49cc-a5ba-d40f19a6aad8-log-httpd\") pod \"ceilometer-0\" (UID: \"f8c1750d-cf4a-49cc-a5ba-d40f19a6aad8\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:50:31 crc kubenswrapper[5052]: I1203 11:50:31.073055 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/f8c1750d-cf4a-49cc-a5ba-d40f19a6aad8-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"f8c1750d-cf4a-49cc-a5ba-d40f19a6aad8\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:50:31 crc kubenswrapper[5052]: I1203 11:50:31.073376 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f8c1750d-cf4a-49cc-a5ba-d40f19a6aad8-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"f8c1750d-cf4a-49cc-a5ba-d40f19a6aad8\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:50:31 crc kubenswrapper[5052]: I1203 11:50:31.073763 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/f8c1750d-cf4a-49cc-a5ba-d40f19a6aad8-config-data\") pod \"ceilometer-0\" (UID: \"f8c1750d-cf4a-49cc-a5ba-d40f19a6aad8\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:50:31 crc kubenswrapper[5052]: I1203 11:50:31.075288 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/f8c1750d-cf4a-49cc-a5ba-d40f19a6aad8-scripts\") pod \"ceilometer-0\" (UID: \"f8c1750d-cf4a-49cc-a5ba-d40f19a6aad8\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:50:31 crc kubenswrapper[5052]: I1203 11:50:31.094374 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-ksbm4\" (UniqueName: \"kubernetes.io/projected/f8c1750d-cf4a-49cc-a5ba-d40f19a6aad8-kube-api-access-ksbm4\") pod \"ceilometer-0\" (UID: \"f8c1750d-cf4a-49cc-a5ba-d40f19a6aad8\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:50:31 crc kubenswrapper[5052]: I1203 11:50:31.099219 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:50:31 crc kubenswrapper[5052]: I1203 11:50:31.107973 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:50:31 crc kubenswrapper[5052]: I1203 11:50:31.130879 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:50:31 crc kubenswrapper[5052]: I1203 11:50:31.130946 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:50:31 crc kubenswrapper[5052]: I1203 11:50:31.166987 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:50:31 crc kubenswrapper[5052]: I1203 11:50:31.182055 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:50:31 crc kubenswrapper[5052]: I1203 11:50:31.199340 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:50:31 crc kubenswrapper[5052]: I1203 11:50:31.509656 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:50:31 crc kubenswrapper[5052]: I1203 11:50:31.509904 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:50:31 crc kubenswrapper[5052]: I1203 11:50:31.509973 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:50:31 crc kubenswrapper[5052]: I1203 11:50:31.509989 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:50:31 crc kubenswrapper[5052]: I1203 11:50:31.683642 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="666333b4-b787-4efa-9681-e6dc65f42542" path="/var/lib/kubelet/pods/666333b4-b787-4efa-9681-e6dc65f42542/volumes" Dec 03 11:50:31 crc kubenswrapper[5052]: I1203 11:50:31.810609 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:50:32 crc kubenswrapper[5052]: I1203 11:50:32.055256 5052 scope.go:117] "RemoveContainer" containerID="5e4ba37f3dc718fe1333c37c5dbab54711be06535a90da3c06591ab9c24f200c" Dec 03 11:50:32 crc kubenswrapper[5052]: I1203 11:50:32.075297 5052 scope.go:117] "RemoveContainer" containerID="52f339b3d910dbf9c4fb68846eb91d563721d72aa18e7b140f1a773dba41565c" Dec 03 11:50:32 crc kubenswrapper[5052]: I1203 11:50:32.142339 5052 scope.go:117] "RemoveContainer" containerID="38aac1236f884983fd56130cde1141029030d5995e585474bc43e38342fcd566" Dec 03 11:50:32 crc kubenswrapper[5052]: I1203 11:50:32.169637 5052 scope.go:117] "RemoveContainer" containerID="562c233abb263737b9f5f1e4e7573903554a8770ec0a228f0ae6b1eaeed438cd" Dec 03 11:50:32 crc kubenswrapper[5052]: I1203 11:50:32.191355 5052 scope.go:117] "RemoveContainer" containerID="4d318f4a7ce116ebcf5a2ea62f382127bf1d126d796753d913aaafd4044c08be" Dec 03 11:50:32 crc kubenswrapper[5052]: I1203 11:50:32.209658 5052 scope.go:117] "RemoveContainer" containerID="b6e79fea7226f005ad352e5ed4669dd04e2f833341d603135672d32a6a08851b" Dec 03 11:50:32 crc kubenswrapper[5052]: I1203 11:50:32.230519 5052 scope.go:117] "RemoveContainer" containerID="57ac6bf32578c2c473e847077b4327901f57601c65faea43ff20034734be5d19" Dec 03 11:50:32 crc kubenswrapper[5052]: I1203 11:50:32.290278 5052 scope.go:117] "RemoveContainer" containerID="49cca97ca89a76c55abbbc2cf6873647427b19a37272e11075513a988efda003" Dec 03 11:50:32 crc kubenswrapper[5052]: I1203 11:50:32.333843 5052 scope.go:117] "RemoveContainer" containerID="484767d2c94ae919e47f683d5bfab4a9ab2ee0aca466f6bef5f32634711c335c" Dec 03 11:50:32 crc kubenswrapper[5052]: I1203 11:50:32.384154 5052 scope.go:117] "RemoveContainer" containerID="b0260015d44c0e665a4e129dce5d0997d993c150242031889ee970b1dbfc89df" Dec 03 11:50:32 crc kubenswrapper[5052]: I1203 11:50:32.417580 5052 scope.go:117] "RemoveContainer" containerID="f64575a6937b334384672b5b5ceb671a3f4b664259bb329427fc96882137e948" Dec 03 11:50:32 crc kubenswrapper[5052]: I1203 11:50:32.440164 5052 scope.go:117] "RemoveContainer" containerID="c9c41a712ce14f494b913d7d4165d8d399a912c664179fc011621e5b9c476b1c" Dec 03 11:50:32 crc kubenswrapper[5052]: I1203 11:50:32.501678 5052 scope.go:117] "RemoveContainer" containerID="3164682c6c7ed301d0d1fe1c0872340c3ccf49ae8dd5ae7e5c42e69a4c52010a" Dec 03 11:50:32 crc kubenswrapper[5052]: I1203 11:50:32.531578 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"f8c1750d-cf4a-49cc-a5ba-d40f19a6aad8","Type":"ContainerStarted","Data":"95da1c09d4a814655d8ff814d18772082ba427c8e948f5716fc3c2800d6ee734"} Dec 03 11:50:32 crc kubenswrapper[5052]: I1203 11:50:32.543878 5052 scope.go:117] "RemoveContainer" containerID="decc6b94b55978b5d80c7e0c6d2f7fe37d1a02b9b0af94ffa4573365f43e6d12" Dec 03 11:50:32 crc kubenswrapper[5052]: I1203 11:50:32.563379 5052 scope.go:117] "RemoveContainer" containerID="d611ec42cb4376e27185390a976f403ae2efb08d4970d4d342ad87bfe46b3941" Dec 03 11:50:32 crc kubenswrapper[5052]: I1203 11:50:32.605810 5052 scope.go:117] "RemoveContainer" containerID="2beb36e19d46bc6b746913b09b005f085e7e4ab7f58b32b97740b3059d80cf36" Dec 03 11:50:32 crc kubenswrapper[5052]: I1203 11:50:32.640315 5052 scope.go:117] "RemoveContainer" containerID="e8f9201ec877119fe422d30c220b1d235ac9fd50b57cec45153ccb587583396d" Dec 03 11:50:32 crc kubenswrapper[5052]: I1203 11:50:32.672072 5052 scope.go:117] "RemoveContainer" containerID="0d7ccf70d7dc58060ea283ad2f6c964fb25e52537fe4fef671cdf5a0610a07dc" Dec 03 11:50:32 crc kubenswrapper[5052]: I1203 11:50:32.672938 5052 scope.go:117] "RemoveContainer" containerID="25fe88f8d102f09983f8884e65f0c6c1575ade67921f10a630e6dcc3ea8b506e" Dec 03 11:50:32 crc kubenswrapper[5052]: E1203 11:50:32.673165 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 11:50:32 crc kubenswrapper[5052]: I1203 11:50:32.724602 5052 scope.go:117] "RemoveContainer" containerID="58159cdbdb384b445bbfc44b06917784c26a4f04d2b5e2146d782963eab0eab7" Dec 03 11:50:33 crc kubenswrapper[5052]: I1203 11:50:33.572132 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:50:33 crc kubenswrapper[5052]: I1203 11:50:33.577917 5052 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Dec 03 11:50:33 crc kubenswrapper[5052]: I1203 11:50:33.652879 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:50:33 crc kubenswrapper[5052]: I1203 11:50:33.703094 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:50:34 crc kubenswrapper[5052]: I1203 11:50:34.482675 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:50:35 crc kubenswrapper[5052]: I1203 11:50:35.568889 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"f8c1750d-cf4a-49cc-a5ba-d40f19a6aad8","Type":"ContainerStarted","Data":"eb33c766c34b7def659e87f673ae82230a201d258f7a88dee07749f3cbd8d1e0"} Dec 03 11:50:38 crc kubenswrapper[5052]: I1203 11:50:38.603225 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"f8c1750d-cf4a-49cc-a5ba-d40f19a6aad8","Type":"ContainerStarted","Data":"8f596e1b0a79e1f1f823327015afe83ef471316845e5e0fd8516f51ce4c105c2"} Dec 03 11:50:43 crc kubenswrapper[5052]: I1203 11:50:43.662141 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"f8c1750d-cf4a-49cc-a5ba-d40f19a6aad8","Type":"ContainerStarted","Data":"152e33660b972abe070d51c7baf270cd5dab59b3cbe27ab26e323e676c508393"} Dec 03 11:50:46 crc kubenswrapper[5052]: I1203 11:50:46.672719 5052 scope.go:117] "RemoveContainer" containerID="25fe88f8d102f09983f8884e65f0c6c1575ade67921f10a630e6dcc3ea8b506e" Dec 03 11:50:46 crc kubenswrapper[5052]: E1203 11:50:46.673409 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 11:50:49 crc kubenswrapper[5052]: I1203 11:50:49.723541 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"f8c1750d-cf4a-49cc-a5ba-d40f19a6aad8","Type":"ContainerStarted","Data":"2807aed7a69b1fdeca76bc8aeda707467e8f044feefef396c4c3ef5f25917a3b"} Dec 03 11:50:49 crc kubenswrapper[5052]: I1203 11:50:49.724058 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:50:49 crc kubenswrapper[5052]: I1203 11:50:49.753914 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/ceilometer-0" podStartSLOduration=2.753964373 podStartE2EDuration="19.753895649s" podCreationTimestamp="2025-12-03 11:50:30 +0000 UTC" firstStartedPulling="2025-12-03 11:50:31.783662935 +0000 UTC m=+1822.371904699" lastFinishedPulling="2025-12-03 11:50:48.783594181 +0000 UTC m=+1839.371835975" observedRunningTime="2025-12-03 11:50:49.753640081 +0000 UTC m=+1840.341881845" watchObservedRunningTime="2025-12-03 11:50:49.753895649 +0000 UTC m=+1840.342137423" Dec 03 11:50:58 crc kubenswrapper[5052]: I1203 11:50:58.673939 5052 scope.go:117] "RemoveContainer" containerID="25fe88f8d102f09983f8884e65f0c6c1575ade67921f10a630e6dcc3ea8b506e" Dec 03 11:50:58 crc kubenswrapper[5052]: E1203 11:50:58.675104 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 11:50:59 crc kubenswrapper[5052]: I1203 11:50:59.821674 5052 generic.go:334] "Generic (PLEG): container finished" podID="20d3b16f-b0ce-4c24-b665-3be3a2da781a" containerID="97f15cadb1a96ebd3eddcbe674bed92f481485532cc8ed2bb6b1191062dafefa" exitCode=0 Dec 03 11:50:59 crc kubenswrapper[5052]: I1203 11:50:59.821818 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-bootstrap-vbjb4" event={"ID":"20d3b16f-b0ce-4c24-b665-3be3a2da781a","Type":"ContainerDied","Data":"97f15cadb1a96ebd3eddcbe674bed92f481485532cc8ed2bb6b1191062dafefa"} Dec 03 11:51:00 crc kubenswrapper[5052]: I1203 11:51:00.834812 5052 generic.go:334] "Generic (PLEG): container finished" podID="3a49f2d7-59af-4bc8-8d05-641b33c7a24c" containerID="4ca16aced64bf17827b7e9533988c8f6aa76a1efc00408bc2b421323bbd6026b" exitCode=0 Dec 03 11:51:00 crc kubenswrapper[5052]: I1203 11:51:00.834913 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-db-sync-nqvdz" event={"ID":"3a49f2d7-59af-4bc8-8d05-641b33c7a24c","Type":"ContainerDied","Data":"4ca16aced64bf17827b7e9533988c8f6aa76a1efc00408bc2b421323bbd6026b"} Dec 03 11:51:01 crc kubenswrapper[5052]: I1203 11:51:01.141058 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-bootstrap-vbjb4" Dec 03 11:51:01 crc kubenswrapper[5052]: I1203 11:51:01.203531 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:51:01 crc kubenswrapper[5052]: I1203 11:51:01.203888 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/20d3b16f-b0ce-4c24-b665-3be3a2da781a-config-data\") pod \"20d3b16f-b0ce-4c24-b665-3be3a2da781a\" (UID: \"20d3b16f-b0ce-4c24-b665-3be3a2da781a\") " Dec 03 11:51:01 crc kubenswrapper[5052]: I1203 11:51:01.203943 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/20d3b16f-b0ce-4c24-b665-3be3a2da781a-combined-ca-bundle\") pod \"20d3b16f-b0ce-4c24-b665-3be3a2da781a\" (UID: \"20d3b16f-b0ce-4c24-b665-3be3a2da781a\") " Dec 03 11:51:01 crc kubenswrapper[5052]: I1203 11:51:01.204025 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-grwkk\" (UniqueName: \"kubernetes.io/projected/20d3b16f-b0ce-4c24-b665-3be3a2da781a-kube-api-access-grwkk\") pod \"20d3b16f-b0ce-4c24-b665-3be3a2da781a\" (UID: \"20d3b16f-b0ce-4c24-b665-3be3a2da781a\") " Dec 03 11:51:01 crc kubenswrapper[5052]: I1203 11:51:01.205071 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/20d3b16f-b0ce-4c24-b665-3be3a2da781a-fernet-keys\") pod \"20d3b16f-b0ce-4c24-b665-3be3a2da781a\" (UID: \"20d3b16f-b0ce-4c24-b665-3be3a2da781a\") " Dec 03 11:51:01 crc kubenswrapper[5052]: I1203 11:51:01.205153 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/20d3b16f-b0ce-4c24-b665-3be3a2da781a-scripts\") pod \"20d3b16f-b0ce-4c24-b665-3be3a2da781a\" (UID: \"20d3b16f-b0ce-4c24-b665-3be3a2da781a\") " Dec 03 11:51:01 crc kubenswrapper[5052]: I1203 11:51:01.205270 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/20d3b16f-b0ce-4c24-b665-3be3a2da781a-credential-keys\") pod \"20d3b16f-b0ce-4c24-b665-3be3a2da781a\" (UID: \"20d3b16f-b0ce-4c24-b665-3be3a2da781a\") " Dec 03 11:51:01 crc kubenswrapper[5052]: I1203 11:51:01.209809 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/20d3b16f-b0ce-4c24-b665-3be3a2da781a-fernet-keys" (OuterVolumeSpecName: "fernet-keys") pod "20d3b16f-b0ce-4c24-b665-3be3a2da781a" (UID: "20d3b16f-b0ce-4c24-b665-3be3a2da781a"). InnerVolumeSpecName "fernet-keys". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:51:01 crc kubenswrapper[5052]: I1203 11:51:01.216214 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/20d3b16f-b0ce-4c24-b665-3be3a2da781a-scripts" (OuterVolumeSpecName: "scripts") pod "20d3b16f-b0ce-4c24-b665-3be3a2da781a" (UID: "20d3b16f-b0ce-4c24-b665-3be3a2da781a"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:51:01 crc kubenswrapper[5052]: I1203 11:51:01.216434 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/20d3b16f-b0ce-4c24-b665-3be3a2da781a-kube-api-access-grwkk" (OuterVolumeSpecName: "kube-api-access-grwkk") pod "20d3b16f-b0ce-4c24-b665-3be3a2da781a" (UID: "20d3b16f-b0ce-4c24-b665-3be3a2da781a"). InnerVolumeSpecName "kube-api-access-grwkk". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:51:01 crc kubenswrapper[5052]: I1203 11:51:01.216545 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/20d3b16f-b0ce-4c24-b665-3be3a2da781a-credential-keys" (OuterVolumeSpecName: "credential-keys") pod "20d3b16f-b0ce-4c24-b665-3be3a2da781a" (UID: "20d3b16f-b0ce-4c24-b665-3be3a2da781a"). InnerVolumeSpecName "credential-keys". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:51:01 crc kubenswrapper[5052]: I1203 11:51:01.229718 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/20d3b16f-b0ce-4c24-b665-3be3a2da781a-config-data" (OuterVolumeSpecName: "config-data") pod "20d3b16f-b0ce-4c24-b665-3be3a2da781a" (UID: "20d3b16f-b0ce-4c24-b665-3be3a2da781a"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:51:01 crc kubenswrapper[5052]: I1203 11:51:01.261554 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/20d3b16f-b0ce-4c24-b665-3be3a2da781a-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "20d3b16f-b0ce-4c24-b665-3be3a2da781a" (UID: "20d3b16f-b0ce-4c24-b665-3be3a2da781a"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:51:01 crc kubenswrapper[5052]: I1203 11:51:01.308259 5052 reconciler_common.go:293] "Volume detached for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/20d3b16f-b0ce-4c24-b665-3be3a2da781a-fernet-keys\") on node \"crc\" DevicePath \"\"" Dec 03 11:51:01 crc kubenswrapper[5052]: I1203 11:51:01.308310 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/20d3b16f-b0ce-4c24-b665-3be3a2da781a-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:51:01 crc kubenswrapper[5052]: I1203 11:51:01.308320 5052 reconciler_common.go:293] "Volume detached for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/20d3b16f-b0ce-4c24-b665-3be3a2da781a-credential-keys\") on node \"crc\" DevicePath \"\"" Dec 03 11:51:01 crc kubenswrapper[5052]: I1203 11:51:01.308332 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/20d3b16f-b0ce-4c24-b665-3be3a2da781a-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:51:01 crc kubenswrapper[5052]: I1203 11:51:01.308342 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/20d3b16f-b0ce-4c24-b665-3be3a2da781a-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:51:01 crc kubenswrapper[5052]: I1203 11:51:01.308351 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-grwkk\" (UniqueName: \"kubernetes.io/projected/20d3b16f-b0ce-4c24-b665-3be3a2da781a-kube-api-access-grwkk\") on node \"crc\" DevicePath \"\"" Dec 03 11:51:01 crc kubenswrapper[5052]: I1203 11:51:01.850153 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-bootstrap-vbjb4" Dec 03 11:51:01 crc kubenswrapper[5052]: I1203 11:51:01.850146 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-bootstrap-vbjb4" event={"ID":"20d3b16f-b0ce-4c24-b665-3be3a2da781a","Type":"ContainerDied","Data":"ef0059d3a7b4a9c6fc5cb0bb9f3ef07ac616881c592a0fb5a6259ef33f4a5590"} Dec 03 11:51:01 crc kubenswrapper[5052]: I1203 11:51:01.850713 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ef0059d3a7b4a9c6fc5cb0bb9f3ef07ac616881c592a0fb5a6259ef33f4a5590" Dec 03 11:51:02 crc kubenswrapper[5052]: I1203 11:51:02.028418 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/keystone-bootstrap-vbjb4"] Dec 03 11:51:02 crc kubenswrapper[5052]: I1203 11:51:02.036878 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/keystone-bootstrap-vbjb4"] Dec 03 11:51:02 crc kubenswrapper[5052]: I1203 11:51:02.130162 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/keystone-bootstrap-hv96x"] Dec 03 11:51:02 crc kubenswrapper[5052]: E1203 11:51:02.130560 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="20d3b16f-b0ce-4c24-b665-3be3a2da781a" containerName="keystone-bootstrap" Dec 03 11:51:02 crc kubenswrapper[5052]: I1203 11:51:02.130579 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="20d3b16f-b0ce-4c24-b665-3be3a2da781a" containerName="keystone-bootstrap" Dec 03 11:51:02 crc kubenswrapper[5052]: I1203 11:51:02.130796 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="20d3b16f-b0ce-4c24-b665-3be3a2da781a" containerName="keystone-bootstrap" Dec 03 11:51:02 crc kubenswrapper[5052]: I1203 11:51:02.131422 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-bootstrap-hv96x" Dec 03 11:51:02 crc kubenswrapper[5052]: I1203 11:51:02.133794 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"osp-secret" Dec 03 11:51:02 crc kubenswrapper[5052]: I1203 11:51:02.134125 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone-keystone-dockercfg-xtl6f" Dec 03 11:51:02 crc kubenswrapper[5052]: I1203 11:51:02.134366 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone-scripts" Dec 03 11:51:02 crc kubenswrapper[5052]: I1203 11:51:02.134548 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone" Dec 03 11:51:02 crc kubenswrapper[5052]: I1203 11:51:02.137187 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone-config-data" Dec 03 11:51:02 crc kubenswrapper[5052]: I1203 11:51:02.141828 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-bootstrap-hv96x"] Dec 03 11:51:02 crc kubenswrapper[5052]: I1203 11:51:02.163956 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-db-sync-nqvdz" Dec 03 11:51:02 crc kubenswrapper[5052]: I1203 11:51:02.224333 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/3a49f2d7-59af-4bc8-8d05-641b33c7a24c-config-data\") pod \"3a49f2d7-59af-4bc8-8d05-641b33c7a24c\" (UID: \"3a49f2d7-59af-4bc8-8d05-641b33c7a24c\") " Dec 03 11:51:02 crc kubenswrapper[5052]: I1203 11:51:02.224440 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dg59k\" (UniqueName: \"kubernetes.io/projected/3a49f2d7-59af-4bc8-8d05-641b33c7a24c-kube-api-access-dg59k\") pod \"3a49f2d7-59af-4bc8-8d05-641b33c7a24c\" (UID: \"3a49f2d7-59af-4bc8-8d05-641b33c7a24c\") " Dec 03 11:51:02 crc kubenswrapper[5052]: I1203 11:51:02.224517 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3a49f2d7-59af-4bc8-8d05-641b33c7a24c-combined-ca-bundle\") pod \"3a49f2d7-59af-4bc8-8d05-641b33c7a24c\" (UID: \"3a49f2d7-59af-4bc8-8d05-641b33c7a24c\") " Dec 03 11:51:02 crc kubenswrapper[5052]: I1203 11:51:02.224589 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/3a49f2d7-59af-4bc8-8d05-641b33c7a24c-logs\") pod \"3a49f2d7-59af-4bc8-8d05-641b33c7a24c\" (UID: \"3a49f2d7-59af-4bc8-8d05-641b33c7a24c\") " Dec 03 11:51:02 crc kubenswrapper[5052]: I1203 11:51:02.224679 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/3a49f2d7-59af-4bc8-8d05-641b33c7a24c-scripts\") pod \"3a49f2d7-59af-4bc8-8d05-641b33c7a24c\" (UID: \"3a49f2d7-59af-4bc8-8d05-641b33c7a24c\") " Dec 03 11:51:02 crc kubenswrapper[5052]: I1203 11:51:02.225273 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3a49f2d7-59af-4bc8-8d05-641b33c7a24c-logs" (OuterVolumeSpecName: "logs") pod "3a49f2d7-59af-4bc8-8d05-641b33c7a24c" (UID: "3a49f2d7-59af-4bc8-8d05-641b33c7a24c"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:51:02 crc kubenswrapper[5052]: I1203 11:51:02.225426 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c7flf\" (UniqueName: \"kubernetes.io/projected/c86d8ada-bec6-4d35-978b-23e5cbac83ff-kube-api-access-c7flf\") pod \"keystone-bootstrap-hv96x\" (UID: \"c86d8ada-bec6-4d35-978b-23e5cbac83ff\") " pod="openstack-kuttl-tests/keystone-bootstrap-hv96x" Dec 03 11:51:02 crc kubenswrapper[5052]: I1203 11:51:02.225458 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/c86d8ada-bec6-4d35-978b-23e5cbac83ff-config-data\") pod \"keystone-bootstrap-hv96x\" (UID: \"c86d8ada-bec6-4d35-978b-23e5cbac83ff\") " pod="openstack-kuttl-tests/keystone-bootstrap-hv96x" Dec 03 11:51:02 crc kubenswrapper[5052]: I1203 11:51:02.225829 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/c86d8ada-bec6-4d35-978b-23e5cbac83ff-fernet-keys\") pod \"keystone-bootstrap-hv96x\" (UID: \"c86d8ada-bec6-4d35-978b-23e5cbac83ff\") " pod="openstack-kuttl-tests/keystone-bootstrap-hv96x" Dec 03 11:51:02 crc kubenswrapper[5052]: I1203 11:51:02.225973 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c86d8ada-bec6-4d35-978b-23e5cbac83ff-combined-ca-bundle\") pod \"keystone-bootstrap-hv96x\" (UID: \"c86d8ada-bec6-4d35-978b-23e5cbac83ff\") " pod="openstack-kuttl-tests/keystone-bootstrap-hv96x" Dec 03 11:51:02 crc kubenswrapper[5052]: I1203 11:51:02.226084 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/c86d8ada-bec6-4d35-978b-23e5cbac83ff-scripts\") pod \"keystone-bootstrap-hv96x\" (UID: \"c86d8ada-bec6-4d35-978b-23e5cbac83ff\") " pod="openstack-kuttl-tests/keystone-bootstrap-hv96x" Dec 03 11:51:02 crc kubenswrapper[5052]: I1203 11:51:02.226207 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/c86d8ada-bec6-4d35-978b-23e5cbac83ff-credential-keys\") pod \"keystone-bootstrap-hv96x\" (UID: \"c86d8ada-bec6-4d35-978b-23e5cbac83ff\") " pod="openstack-kuttl-tests/keystone-bootstrap-hv96x" Dec 03 11:51:02 crc kubenswrapper[5052]: I1203 11:51:02.226559 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/3a49f2d7-59af-4bc8-8d05-641b33c7a24c-logs\") on node \"crc\" DevicePath \"\"" Dec 03 11:51:02 crc kubenswrapper[5052]: I1203 11:51:02.239036 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3a49f2d7-59af-4bc8-8d05-641b33c7a24c-scripts" (OuterVolumeSpecName: "scripts") pod "3a49f2d7-59af-4bc8-8d05-641b33c7a24c" (UID: "3a49f2d7-59af-4bc8-8d05-641b33c7a24c"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:51:02 crc kubenswrapper[5052]: I1203 11:51:02.239465 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3a49f2d7-59af-4bc8-8d05-641b33c7a24c-kube-api-access-dg59k" (OuterVolumeSpecName: "kube-api-access-dg59k") pod "3a49f2d7-59af-4bc8-8d05-641b33c7a24c" (UID: "3a49f2d7-59af-4bc8-8d05-641b33c7a24c"). InnerVolumeSpecName "kube-api-access-dg59k". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:51:02 crc kubenswrapper[5052]: I1203 11:51:02.249124 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3a49f2d7-59af-4bc8-8d05-641b33c7a24c-config-data" (OuterVolumeSpecName: "config-data") pod "3a49f2d7-59af-4bc8-8d05-641b33c7a24c" (UID: "3a49f2d7-59af-4bc8-8d05-641b33c7a24c"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:51:02 crc kubenswrapper[5052]: I1203 11:51:02.250518 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3a49f2d7-59af-4bc8-8d05-641b33c7a24c-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "3a49f2d7-59af-4bc8-8d05-641b33c7a24c" (UID: "3a49f2d7-59af-4bc8-8d05-641b33c7a24c"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:51:02 crc kubenswrapper[5052]: I1203 11:51:02.328105 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/c86d8ada-bec6-4d35-978b-23e5cbac83ff-config-data\") pod \"keystone-bootstrap-hv96x\" (UID: \"c86d8ada-bec6-4d35-978b-23e5cbac83ff\") " pod="openstack-kuttl-tests/keystone-bootstrap-hv96x" Dec 03 11:51:02 crc kubenswrapper[5052]: I1203 11:51:02.328197 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/c86d8ada-bec6-4d35-978b-23e5cbac83ff-fernet-keys\") pod \"keystone-bootstrap-hv96x\" (UID: \"c86d8ada-bec6-4d35-978b-23e5cbac83ff\") " pod="openstack-kuttl-tests/keystone-bootstrap-hv96x" Dec 03 11:51:02 crc kubenswrapper[5052]: I1203 11:51:02.328224 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c86d8ada-bec6-4d35-978b-23e5cbac83ff-combined-ca-bundle\") pod \"keystone-bootstrap-hv96x\" (UID: \"c86d8ada-bec6-4d35-978b-23e5cbac83ff\") " pod="openstack-kuttl-tests/keystone-bootstrap-hv96x" Dec 03 11:51:02 crc kubenswrapper[5052]: I1203 11:51:02.328250 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/c86d8ada-bec6-4d35-978b-23e5cbac83ff-scripts\") pod \"keystone-bootstrap-hv96x\" (UID: \"c86d8ada-bec6-4d35-978b-23e5cbac83ff\") " pod="openstack-kuttl-tests/keystone-bootstrap-hv96x" Dec 03 11:51:02 crc kubenswrapper[5052]: I1203 11:51:02.328281 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/c86d8ada-bec6-4d35-978b-23e5cbac83ff-credential-keys\") pod \"keystone-bootstrap-hv96x\" (UID: \"c86d8ada-bec6-4d35-978b-23e5cbac83ff\") " pod="openstack-kuttl-tests/keystone-bootstrap-hv96x" Dec 03 11:51:02 crc kubenswrapper[5052]: I1203 11:51:02.328338 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-c7flf\" (UniqueName: \"kubernetes.io/projected/c86d8ada-bec6-4d35-978b-23e5cbac83ff-kube-api-access-c7flf\") pod \"keystone-bootstrap-hv96x\" (UID: \"c86d8ada-bec6-4d35-978b-23e5cbac83ff\") " pod="openstack-kuttl-tests/keystone-bootstrap-hv96x" Dec 03 11:51:02 crc kubenswrapper[5052]: I1203 11:51:02.328938 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-dg59k\" (UniqueName: \"kubernetes.io/projected/3a49f2d7-59af-4bc8-8d05-641b33c7a24c-kube-api-access-dg59k\") on node \"crc\" DevicePath \"\"" Dec 03 11:51:02 crc kubenswrapper[5052]: I1203 11:51:02.328965 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3a49f2d7-59af-4bc8-8d05-641b33c7a24c-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:51:02 crc kubenswrapper[5052]: I1203 11:51:02.328977 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/3a49f2d7-59af-4bc8-8d05-641b33c7a24c-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:51:02 crc kubenswrapper[5052]: I1203 11:51:02.328989 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/3a49f2d7-59af-4bc8-8d05-641b33c7a24c-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:51:02 crc kubenswrapper[5052]: I1203 11:51:02.332179 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/c86d8ada-bec6-4d35-978b-23e5cbac83ff-config-data\") pod \"keystone-bootstrap-hv96x\" (UID: \"c86d8ada-bec6-4d35-978b-23e5cbac83ff\") " pod="openstack-kuttl-tests/keystone-bootstrap-hv96x" Dec 03 11:51:02 crc kubenswrapper[5052]: I1203 11:51:02.332187 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c86d8ada-bec6-4d35-978b-23e5cbac83ff-combined-ca-bundle\") pod \"keystone-bootstrap-hv96x\" (UID: \"c86d8ada-bec6-4d35-978b-23e5cbac83ff\") " pod="openstack-kuttl-tests/keystone-bootstrap-hv96x" Dec 03 11:51:02 crc kubenswrapper[5052]: I1203 11:51:02.332740 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/c86d8ada-bec6-4d35-978b-23e5cbac83ff-scripts\") pod \"keystone-bootstrap-hv96x\" (UID: \"c86d8ada-bec6-4d35-978b-23e5cbac83ff\") " pod="openstack-kuttl-tests/keystone-bootstrap-hv96x" Dec 03 11:51:02 crc kubenswrapper[5052]: I1203 11:51:02.333277 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/c86d8ada-bec6-4d35-978b-23e5cbac83ff-credential-keys\") pod \"keystone-bootstrap-hv96x\" (UID: \"c86d8ada-bec6-4d35-978b-23e5cbac83ff\") " pod="openstack-kuttl-tests/keystone-bootstrap-hv96x" Dec 03 11:51:02 crc kubenswrapper[5052]: I1203 11:51:02.333450 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/c86d8ada-bec6-4d35-978b-23e5cbac83ff-fernet-keys\") pod \"keystone-bootstrap-hv96x\" (UID: \"c86d8ada-bec6-4d35-978b-23e5cbac83ff\") " pod="openstack-kuttl-tests/keystone-bootstrap-hv96x" Dec 03 11:51:02 crc kubenswrapper[5052]: I1203 11:51:02.346651 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-c7flf\" (UniqueName: \"kubernetes.io/projected/c86d8ada-bec6-4d35-978b-23e5cbac83ff-kube-api-access-c7flf\") pod \"keystone-bootstrap-hv96x\" (UID: \"c86d8ada-bec6-4d35-978b-23e5cbac83ff\") " pod="openstack-kuttl-tests/keystone-bootstrap-hv96x" Dec 03 11:51:02 crc kubenswrapper[5052]: I1203 11:51:02.484660 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-bootstrap-hv96x" Dec 03 11:51:02 crc kubenswrapper[5052]: I1203 11:51:02.867635 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-db-sync-nqvdz" Dec 03 11:51:02 crc kubenswrapper[5052]: I1203 11:51:02.867624 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-db-sync-nqvdz" event={"ID":"3a49f2d7-59af-4bc8-8d05-641b33c7a24c","Type":"ContainerDied","Data":"d4027f144e15ab4f41b2a5336a39af9c79ec80a702f05271b93953e9587b9570"} Dec 03 11:51:02 crc kubenswrapper[5052]: I1203 11:51:02.867772 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d4027f144e15ab4f41b2a5336a39af9c79ec80a702f05271b93953e9587b9570" Dec 03 11:51:02 crc kubenswrapper[5052]: I1203 11:51:02.871980 5052 generic.go:334] "Generic (PLEG): container finished" podID="97107a61-a19d-4494-bcdb-a8d8f6223cb5" containerID="57ea218fb8f0a0038fe40dcb1ad5997273497a2c37f8b30405d5fc310990e84b" exitCode=0 Dec 03 11:51:02 crc kubenswrapper[5052]: I1203 11:51:02.872029 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-db-sync-847xp" event={"ID":"97107a61-a19d-4494-bcdb-a8d8f6223cb5","Type":"ContainerDied","Data":"57ea218fb8f0a0038fe40dcb1ad5997273497a2c37f8b30405d5fc310990e84b"} Dec 03 11:51:02 crc kubenswrapper[5052]: I1203 11:51:02.954825 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/placement-5fc4755558-qz76x"] Dec 03 11:51:02 crc kubenswrapper[5052]: E1203 11:51:02.955375 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="3a49f2d7-59af-4bc8-8d05-641b33c7a24c" containerName="placement-db-sync" Dec 03 11:51:02 crc kubenswrapper[5052]: I1203 11:51:02.955581 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="3a49f2d7-59af-4bc8-8d05-641b33c7a24c" containerName="placement-db-sync" Dec 03 11:51:02 crc kubenswrapper[5052]: I1203 11:51:02.955816 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="3a49f2d7-59af-4bc8-8d05-641b33c7a24c" containerName="placement-db-sync" Dec 03 11:51:02 crc kubenswrapper[5052]: I1203 11:51:02.956755 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-5fc4755558-qz76x" Dec 03 11:51:02 crc kubenswrapper[5052]: I1203 11:51:02.962402 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"placement-scripts" Dec 03 11:51:02 crc kubenswrapper[5052]: I1203 11:51:02.962681 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"placement-placement-dockercfg-kv7xq" Dec 03 11:51:02 crc kubenswrapper[5052]: I1203 11:51:02.965057 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-placement-public-svc" Dec 03 11:51:02 crc kubenswrapper[5052]: I1203 11:51:02.965665 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"placement-config-data" Dec 03 11:51:02 crc kubenswrapper[5052]: I1203 11:51:02.966017 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-placement-internal-svc" Dec 03 11:51:02 crc kubenswrapper[5052]: I1203 11:51:02.973068 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/placement-5fc4755558-qz76x"] Dec 03 11:51:03 crc kubenswrapper[5052]: I1203 11:51:03.014613 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-bootstrap-hv96x"] Dec 03 11:51:03 crc kubenswrapper[5052]: I1203 11:51:03.043162 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/428e9203-aa9c-48d3-84c9-4dfaa806140d-public-tls-certs\") pod \"placement-5fc4755558-qz76x\" (UID: \"428e9203-aa9c-48d3-84c9-4dfaa806140d\") " pod="openstack-kuttl-tests/placement-5fc4755558-qz76x" Dec 03 11:51:03 crc kubenswrapper[5052]: I1203 11:51:03.043205 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/428e9203-aa9c-48d3-84c9-4dfaa806140d-config-data\") pod \"placement-5fc4755558-qz76x\" (UID: \"428e9203-aa9c-48d3-84c9-4dfaa806140d\") " pod="openstack-kuttl-tests/placement-5fc4755558-qz76x" Dec 03 11:51:03 crc kubenswrapper[5052]: I1203 11:51:03.043300 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sfv2r\" (UniqueName: \"kubernetes.io/projected/428e9203-aa9c-48d3-84c9-4dfaa806140d-kube-api-access-sfv2r\") pod \"placement-5fc4755558-qz76x\" (UID: \"428e9203-aa9c-48d3-84c9-4dfaa806140d\") " pod="openstack-kuttl-tests/placement-5fc4755558-qz76x" Dec 03 11:51:03 crc kubenswrapper[5052]: I1203 11:51:03.043334 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/428e9203-aa9c-48d3-84c9-4dfaa806140d-internal-tls-certs\") pod \"placement-5fc4755558-qz76x\" (UID: \"428e9203-aa9c-48d3-84c9-4dfaa806140d\") " pod="openstack-kuttl-tests/placement-5fc4755558-qz76x" Dec 03 11:51:03 crc kubenswrapper[5052]: I1203 11:51:03.043376 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/428e9203-aa9c-48d3-84c9-4dfaa806140d-logs\") pod \"placement-5fc4755558-qz76x\" (UID: \"428e9203-aa9c-48d3-84c9-4dfaa806140d\") " pod="openstack-kuttl-tests/placement-5fc4755558-qz76x" Dec 03 11:51:03 crc kubenswrapper[5052]: I1203 11:51:03.043403 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/428e9203-aa9c-48d3-84c9-4dfaa806140d-scripts\") pod \"placement-5fc4755558-qz76x\" (UID: \"428e9203-aa9c-48d3-84c9-4dfaa806140d\") " pod="openstack-kuttl-tests/placement-5fc4755558-qz76x" Dec 03 11:51:03 crc kubenswrapper[5052]: I1203 11:51:03.043451 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/428e9203-aa9c-48d3-84c9-4dfaa806140d-combined-ca-bundle\") pod \"placement-5fc4755558-qz76x\" (UID: \"428e9203-aa9c-48d3-84c9-4dfaa806140d\") " pod="openstack-kuttl-tests/placement-5fc4755558-qz76x" Dec 03 11:51:03 crc kubenswrapper[5052]: I1203 11:51:03.145033 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/428e9203-aa9c-48d3-84c9-4dfaa806140d-logs\") pod \"placement-5fc4755558-qz76x\" (UID: \"428e9203-aa9c-48d3-84c9-4dfaa806140d\") " pod="openstack-kuttl-tests/placement-5fc4755558-qz76x" Dec 03 11:51:03 crc kubenswrapper[5052]: I1203 11:51:03.145436 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/428e9203-aa9c-48d3-84c9-4dfaa806140d-scripts\") pod \"placement-5fc4755558-qz76x\" (UID: \"428e9203-aa9c-48d3-84c9-4dfaa806140d\") " pod="openstack-kuttl-tests/placement-5fc4755558-qz76x" Dec 03 11:51:03 crc kubenswrapper[5052]: I1203 11:51:03.145489 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/428e9203-aa9c-48d3-84c9-4dfaa806140d-combined-ca-bundle\") pod \"placement-5fc4755558-qz76x\" (UID: \"428e9203-aa9c-48d3-84c9-4dfaa806140d\") " pod="openstack-kuttl-tests/placement-5fc4755558-qz76x" Dec 03 11:51:03 crc kubenswrapper[5052]: I1203 11:51:03.145534 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/428e9203-aa9c-48d3-84c9-4dfaa806140d-config-data\") pod \"placement-5fc4755558-qz76x\" (UID: \"428e9203-aa9c-48d3-84c9-4dfaa806140d\") " pod="openstack-kuttl-tests/placement-5fc4755558-qz76x" Dec 03 11:51:03 crc kubenswrapper[5052]: I1203 11:51:03.145555 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/428e9203-aa9c-48d3-84c9-4dfaa806140d-public-tls-certs\") pod \"placement-5fc4755558-qz76x\" (UID: \"428e9203-aa9c-48d3-84c9-4dfaa806140d\") " pod="openstack-kuttl-tests/placement-5fc4755558-qz76x" Dec 03 11:51:03 crc kubenswrapper[5052]: I1203 11:51:03.145647 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-sfv2r\" (UniqueName: \"kubernetes.io/projected/428e9203-aa9c-48d3-84c9-4dfaa806140d-kube-api-access-sfv2r\") pod \"placement-5fc4755558-qz76x\" (UID: \"428e9203-aa9c-48d3-84c9-4dfaa806140d\") " pod="openstack-kuttl-tests/placement-5fc4755558-qz76x" Dec 03 11:51:03 crc kubenswrapper[5052]: I1203 11:51:03.145684 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/428e9203-aa9c-48d3-84c9-4dfaa806140d-internal-tls-certs\") pod \"placement-5fc4755558-qz76x\" (UID: \"428e9203-aa9c-48d3-84c9-4dfaa806140d\") " pod="openstack-kuttl-tests/placement-5fc4755558-qz76x" Dec 03 11:51:03 crc kubenswrapper[5052]: I1203 11:51:03.145793 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/428e9203-aa9c-48d3-84c9-4dfaa806140d-logs\") pod \"placement-5fc4755558-qz76x\" (UID: \"428e9203-aa9c-48d3-84c9-4dfaa806140d\") " pod="openstack-kuttl-tests/placement-5fc4755558-qz76x" Dec 03 11:51:03 crc kubenswrapper[5052]: I1203 11:51:03.148842 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/428e9203-aa9c-48d3-84c9-4dfaa806140d-public-tls-certs\") pod \"placement-5fc4755558-qz76x\" (UID: \"428e9203-aa9c-48d3-84c9-4dfaa806140d\") " pod="openstack-kuttl-tests/placement-5fc4755558-qz76x" Dec 03 11:51:03 crc kubenswrapper[5052]: I1203 11:51:03.149222 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/428e9203-aa9c-48d3-84c9-4dfaa806140d-config-data\") pod \"placement-5fc4755558-qz76x\" (UID: \"428e9203-aa9c-48d3-84c9-4dfaa806140d\") " pod="openstack-kuttl-tests/placement-5fc4755558-qz76x" Dec 03 11:51:03 crc kubenswrapper[5052]: I1203 11:51:03.149527 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/428e9203-aa9c-48d3-84c9-4dfaa806140d-scripts\") pod \"placement-5fc4755558-qz76x\" (UID: \"428e9203-aa9c-48d3-84c9-4dfaa806140d\") " pod="openstack-kuttl-tests/placement-5fc4755558-qz76x" Dec 03 11:51:03 crc kubenswrapper[5052]: I1203 11:51:03.151880 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/428e9203-aa9c-48d3-84c9-4dfaa806140d-combined-ca-bundle\") pod \"placement-5fc4755558-qz76x\" (UID: \"428e9203-aa9c-48d3-84c9-4dfaa806140d\") " pod="openstack-kuttl-tests/placement-5fc4755558-qz76x" Dec 03 11:51:03 crc kubenswrapper[5052]: I1203 11:51:03.151933 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/428e9203-aa9c-48d3-84c9-4dfaa806140d-internal-tls-certs\") pod \"placement-5fc4755558-qz76x\" (UID: \"428e9203-aa9c-48d3-84c9-4dfaa806140d\") " pod="openstack-kuttl-tests/placement-5fc4755558-qz76x" Dec 03 11:51:03 crc kubenswrapper[5052]: I1203 11:51:03.162908 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-sfv2r\" (UniqueName: \"kubernetes.io/projected/428e9203-aa9c-48d3-84c9-4dfaa806140d-kube-api-access-sfv2r\") pod \"placement-5fc4755558-qz76x\" (UID: \"428e9203-aa9c-48d3-84c9-4dfaa806140d\") " pod="openstack-kuttl-tests/placement-5fc4755558-qz76x" Dec 03 11:51:03 crc kubenswrapper[5052]: I1203 11:51:03.278595 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-5fc4755558-qz76x" Dec 03 11:51:03 crc kubenswrapper[5052]: I1203 11:51:03.524419 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/placement-5fc4755558-qz76x"] Dec 03 11:51:03 crc kubenswrapper[5052]: I1203 11:51:03.683383 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="20d3b16f-b0ce-4c24-b665-3be3a2da781a" path="/var/lib/kubelet/pods/20d3b16f-b0ce-4c24-b665-3be3a2da781a/volumes" Dec 03 11:51:03 crc kubenswrapper[5052]: I1203 11:51:03.887159 5052 generic.go:334] "Generic (PLEG): container finished" podID="5ddb757c-c235-4a2f-b6db-0d120215edcf" containerID="24721104cb9088d6a7d4f0ae127c54a770da6fbf71ba161bc9d6020ff94a6ce7" exitCode=0 Dec 03 11:51:03 crc kubenswrapper[5052]: I1203 11:51:03.887265 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-db-sync-lgtfz" event={"ID":"5ddb757c-c235-4a2f-b6db-0d120215edcf","Type":"ContainerDied","Data":"24721104cb9088d6a7d4f0ae127c54a770da6fbf71ba161bc9d6020ff94a6ce7"} Dec 03 11:51:03 crc kubenswrapper[5052]: I1203 11:51:03.891744 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-5fc4755558-qz76x" event={"ID":"428e9203-aa9c-48d3-84c9-4dfaa806140d","Type":"ContainerStarted","Data":"932117c4bd5dd37889c00f1528583dfd60eee1eb15198c88c121a5439e07b994"} Dec 03 11:51:03 crc kubenswrapper[5052]: I1203 11:51:03.892073 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-5fc4755558-qz76x" event={"ID":"428e9203-aa9c-48d3-84c9-4dfaa806140d","Type":"ContainerStarted","Data":"08e19dd9ccd6aec98e6f2274df9787afa9e6ed5538a316cc3e37b8118cbcb222"} Dec 03 11:51:03 crc kubenswrapper[5052]: I1203 11:51:03.893478 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-bootstrap-hv96x" event={"ID":"c86d8ada-bec6-4d35-978b-23e5cbac83ff","Type":"ContainerStarted","Data":"9393f4a1875bd93dade4f4a250ae2161d3bae5bab0dfeec9d201f006f152194d"} Dec 03 11:51:03 crc kubenswrapper[5052]: I1203 11:51:03.893549 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-bootstrap-hv96x" event={"ID":"c86d8ada-bec6-4d35-978b-23e5cbac83ff","Type":"ContainerStarted","Data":"ff809a113fa8be6a9a42890ed1a8dd80a83774c6f34292b0d359f00c261c5a40"} Dec 03 11:51:03 crc kubenswrapper[5052]: I1203 11:51:03.932899 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/keystone-bootstrap-hv96x" podStartSLOduration=1.932882529 podStartE2EDuration="1.932882529s" podCreationTimestamp="2025-12-03 11:51:02 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:51:03.927970568 +0000 UTC m=+1854.516212352" watchObservedRunningTime="2025-12-03 11:51:03.932882529 +0000 UTC m=+1854.521124293" Dec 03 11:51:04 crc kubenswrapper[5052]: I1203 11:51:04.211293 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-db-sync-847xp" Dec 03 11:51:04 crc kubenswrapper[5052]: I1203 11:51:04.265604 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/97107a61-a19d-4494-bcdb-a8d8f6223cb5-combined-ca-bundle\") pod \"97107a61-a19d-4494-bcdb-a8d8f6223cb5\" (UID: \"97107a61-a19d-4494-bcdb-a8d8f6223cb5\") " Dec 03 11:51:04 crc kubenswrapper[5052]: I1203 11:51:04.265818 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7jnbj\" (UniqueName: \"kubernetes.io/projected/97107a61-a19d-4494-bcdb-a8d8f6223cb5-kube-api-access-7jnbj\") pod \"97107a61-a19d-4494-bcdb-a8d8f6223cb5\" (UID: \"97107a61-a19d-4494-bcdb-a8d8f6223cb5\") " Dec 03 11:51:04 crc kubenswrapper[5052]: I1203 11:51:04.265868 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/97107a61-a19d-4494-bcdb-a8d8f6223cb5-db-sync-config-data\") pod \"97107a61-a19d-4494-bcdb-a8d8f6223cb5\" (UID: \"97107a61-a19d-4494-bcdb-a8d8f6223cb5\") " Dec 03 11:51:04 crc kubenswrapper[5052]: I1203 11:51:04.270456 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/97107a61-a19d-4494-bcdb-a8d8f6223cb5-db-sync-config-data" (OuterVolumeSpecName: "db-sync-config-data") pod "97107a61-a19d-4494-bcdb-a8d8f6223cb5" (UID: "97107a61-a19d-4494-bcdb-a8d8f6223cb5"). InnerVolumeSpecName "db-sync-config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:51:04 crc kubenswrapper[5052]: I1203 11:51:04.270630 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/97107a61-a19d-4494-bcdb-a8d8f6223cb5-kube-api-access-7jnbj" (OuterVolumeSpecName: "kube-api-access-7jnbj") pod "97107a61-a19d-4494-bcdb-a8d8f6223cb5" (UID: "97107a61-a19d-4494-bcdb-a8d8f6223cb5"). InnerVolumeSpecName "kube-api-access-7jnbj". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:51:04 crc kubenswrapper[5052]: I1203 11:51:04.303488 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/97107a61-a19d-4494-bcdb-a8d8f6223cb5-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "97107a61-a19d-4494-bcdb-a8d8f6223cb5" (UID: "97107a61-a19d-4494-bcdb-a8d8f6223cb5"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:51:04 crc kubenswrapper[5052]: I1203 11:51:04.370532 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-7jnbj\" (UniqueName: \"kubernetes.io/projected/97107a61-a19d-4494-bcdb-a8d8f6223cb5-kube-api-access-7jnbj\") on node \"crc\" DevicePath \"\"" Dec 03 11:51:04 crc kubenswrapper[5052]: I1203 11:51:04.370982 5052 reconciler_common.go:293] "Volume detached for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/97107a61-a19d-4494-bcdb-a8d8f6223cb5-db-sync-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:51:04 crc kubenswrapper[5052]: I1203 11:51:04.370999 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/97107a61-a19d-4494-bcdb-a8d8f6223cb5-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:51:04 crc kubenswrapper[5052]: I1203 11:51:04.903276 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-5fc4755558-qz76x" event={"ID":"428e9203-aa9c-48d3-84c9-4dfaa806140d","Type":"ContainerStarted","Data":"f7f46b76f15d594e9e319920752ee78fe4901c945bcac1aa5273dd6bd20b6164"} Dec 03 11:51:04 crc kubenswrapper[5052]: I1203 11:51:04.904854 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/placement-5fc4755558-qz76x" Dec 03 11:51:04 crc kubenswrapper[5052]: I1203 11:51:04.905249 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-db-sync-847xp" Dec 03 11:51:04 crc kubenswrapper[5052]: I1203 11:51:04.905461 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-db-sync-847xp" event={"ID":"97107a61-a19d-4494-bcdb-a8d8f6223cb5","Type":"ContainerDied","Data":"5a2f374beafdeba46ec6fed9c5f9fcd800d1968d84009fd8fa6a84b47f6f9e7d"} Dec 03 11:51:04 crc kubenswrapper[5052]: I1203 11:51:04.905619 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5a2f374beafdeba46ec6fed9c5f9fcd800d1968d84009fd8fa6a84b47f6f9e7d" Dec 03 11:51:04 crc kubenswrapper[5052]: I1203 11:51:04.942079 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/placement-5fc4755558-qz76x" podStartSLOduration=2.942058452 podStartE2EDuration="2.942058452s" podCreationTimestamp="2025-12-03 11:51:02 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:51:04.928653228 +0000 UTC m=+1855.516895052" watchObservedRunningTime="2025-12-03 11:51:04.942058452 +0000 UTC m=+1855.530300216" Dec 03 11:51:05 crc kubenswrapper[5052]: I1203 11:51:05.132895 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/barbican-keystone-listener-7bf89d7c74-c4srn"] Dec 03 11:51:05 crc kubenswrapper[5052]: E1203 11:51:05.133682 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="97107a61-a19d-4494-bcdb-a8d8f6223cb5" containerName="barbican-db-sync" Dec 03 11:51:05 crc kubenswrapper[5052]: I1203 11:51:05.133704 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="97107a61-a19d-4494-bcdb-a8d8f6223cb5" containerName="barbican-db-sync" Dec 03 11:51:05 crc kubenswrapper[5052]: I1203 11:51:05.133866 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="97107a61-a19d-4494-bcdb-a8d8f6223cb5" containerName="barbican-db-sync" Dec 03 11:51:05 crc kubenswrapper[5052]: I1203 11:51:05.134749 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-keystone-listener-7bf89d7c74-c4srn" Dec 03 11:51:05 crc kubenswrapper[5052]: I1203 11:51:05.140150 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"barbican-barbican-dockercfg-wsvrm" Dec 03 11:51:05 crc kubenswrapper[5052]: I1203 11:51:05.140307 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"barbican-config-data" Dec 03 11:51:05 crc kubenswrapper[5052]: I1203 11:51:05.140412 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"barbican-keystone-listener-config-data" Dec 03 11:51:05 crc kubenswrapper[5052]: I1203 11:51:05.146136 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/barbican-worker-6d7f657c65-cqxbg"] Dec 03 11:51:05 crc kubenswrapper[5052]: I1203 11:51:05.147993 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-worker-6d7f657c65-cqxbg" Dec 03 11:51:05 crc kubenswrapper[5052]: I1203 11:51:05.153191 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"barbican-worker-config-data" Dec 03 11:51:05 crc kubenswrapper[5052]: I1203 11:51:05.156732 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-keystone-listener-7bf89d7c74-c4srn"] Dec 03 11:51:05 crc kubenswrapper[5052]: I1203 11:51:05.164753 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-worker-6d7f657c65-cqxbg"] Dec 03 11:51:05 crc kubenswrapper[5052]: I1203 11:51:05.186216 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7csfc\" (UniqueName: \"kubernetes.io/projected/5679a95d-a498-448e-97ed-497496520506-kube-api-access-7csfc\") pod \"barbican-keystone-listener-7bf89d7c74-c4srn\" (UID: \"5679a95d-a498-448e-97ed-497496520506\") " pod="openstack-kuttl-tests/barbican-keystone-listener-7bf89d7c74-c4srn" Dec 03 11:51:05 crc kubenswrapper[5052]: I1203 11:51:05.186332 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/584deaba-f945-42a2-8c0d-e531d6f48fd8-config-data-custom\") pod \"barbican-worker-6d7f657c65-cqxbg\" (UID: \"584deaba-f945-42a2-8c0d-e531d6f48fd8\") " pod="openstack-kuttl-tests/barbican-worker-6d7f657c65-cqxbg" Dec 03 11:51:05 crc kubenswrapper[5052]: I1203 11:51:05.186364 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/5679a95d-a498-448e-97ed-497496520506-config-data-custom\") pod \"barbican-keystone-listener-7bf89d7c74-c4srn\" (UID: \"5679a95d-a498-448e-97ed-497496520506\") " pod="openstack-kuttl-tests/barbican-keystone-listener-7bf89d7c74-c4srn" Dec 03 11:51:05 crc kubenswrapper[5052]: I1203 11:51:05.186383 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/5679a95d-a498-448e-97ed-497496520506-config-data\") pod \"barbican-keystone-listener-7bf89d7c74-c4srn\" (UID: \"5679a95d-a498-448e-97ed-497496520506\") " pod="openstack-kuttl-tests/barbican-keystone-listener-7bf89d7c74-c4srn" Dec 03 11:51:05 crc kubenswrapper[5052]: I1203 11:51:05.186429 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/584deaba-f945-42a2-8c0d-e531d6f48fd8-logs\") pod \"barbican-worker-6d7f657c65-cqxbg\" (UID: \"584deaba-f945-42a2-8c0d-e531d6f48fd8\") " pod="openstack-kuttl-tests/barbican-worker-6d7f657c65-cqxbg" Dec 03 11:51:05 crc kubenswrapper[5052]: I1203 11:51:05.186451 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/584deaba-f945-42a2-8c0d-e531d6f48fd8-combined-ca-bundle\") pod \"barbican-worker-6d7f657c65-cqxbg\" (UID: \"584deaba-f945-42a2-8c0d-e531d6f48fd8\") " pod="openstack-kuttl-tests/barbican-worker-6d7f657c65-cqxbg" Dec 03 11:51:05 crc kubenswrapper[5052]: I1203 11:51:05.186575 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/584deaba-f945-42a2-8c0d-e531d6f48fd8-config-data\") pod \"barbican-worker-6d7f657c65-cqxbg\" (UID: \"584deaba-f945-42a2-8c0d-e531d6f48fd8\") " pod="openstack-kuttl-tests/barbican-worker-6d7f657c65-cqxbg" Dec 03 11:51:05 crc kubenswrapper[5052]: I1203 11:51:05.186611 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/5679a95d-a498-448e-97ed-497496520506-logs\") pod \"barbican-keystone-listener-7bf89d7c74-c4srn\" (UID: \"5679a95d-a498-448e-97ed-497496520506\") " pod="openstack-kuttl-tests/barbican-keystone-listener-7bf89d7c74-c4srn" Dec 03 11:51:05 crc kubenswrapper[5052]: I1203 11:51:05.186634 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cjpxg\" (UniqueName: \"kubernetes.io/projected/584deaba-f945-42a2-8c0d-e531d6f48fd8-kube-api-access-cjpxg\") pod \"barbican-worker-6d7f657c65-cqxbg\" (UID: \"584deaba-f945-42a2-8c0d-e531d6f48fd8\") " pod="openstack-kuttl-tests/barbican-worker-6d7f657c65-cqxbg" Dec 03 11:51:05 crc kubenswrapper[5052]: I1203 11:51:05.186653 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5679a95d-a498-448e-97ed-497496520506-combined-ca-bundle\") pod \"barbican-keystone-listener-7bf89d7c74-c4srn\" (UID: \"5679a95d-a498-448e-97ed-497496520506\") " pod="openstack-kuttl-tests/barbican-keystone-listener-7bf89d7c74-c4srn" Dec 03 11:51:05 crc kubenswrapper[5052]: I1203 11:51:05.292364 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/5679a95d-a498-448e-97ed-497496520506-config-data\") pod \"barbican-keystone-listener-7bf89d7c74-c4srn\" (UID: \"5679a95d-a498-448e-97ed-497496520506\") " pod="openstack-kuttl-tests/barbican-keystone-listener-7bf89d7c74-c4srn" Dec 03 11:51:05 crc kubenswrapper[5052]: I1203 11:51:05.292430 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/584deaba-f945-42a2-8c0d-e531d6f48fd8-logs\") pod \"barbican-worker-6d7f657c65-cqxbg\" (UID: \"584deaba-f945-42a2-8c0d-e531d6f48fd8\") " pod="openstack-kuttl-tests/barbican-worker-6d7f657c65-cqxbg" Dec 03 11:51:05 crc kubenswrapper[5052]: I1203 11:51:05.292454 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/584deaba-f945-42a2-8c0d-e531d6f48fd8-combined-ca-bundle\") pod \"barbican-worker-6d7f657c65-cqxbg\" (UID: \"584deaba-f945-42a2-8c0d-e531d6f48fd8\") " pod="openstack-kuttl-tests/barbican-worker-6d7f657c65-cqxbg" Dec 03 11:51:05 crc kubenswrapper[5052]: I1203 11:51:05.292566 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/584deaba-f945-42a2-8c0d-e531d6f48fd8-config-data\") pod \"barbican-worker-6d7f657c65-cqxbg\" (UID: \"584deaba-f945-42a2-8c0d-e531d6f48fd8\") " pod="openstack-kuttl-tests/barbican-worker-6d7f657c65-cqxbg" Dec 03 11:51:05 crc kubenswrapper[5052]: I1203 11:51:05.292587 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/5679a95d-a498-448e-97ed-497496520506-logs\") pod \"barbican-keystone-listener-7bf89d7c74-c4srn\" (UID: \"5679a95d-a498-448e-97ed-497496520506\") " pod="openstack-kuttl-tests/barbican-keystone-listener-7bf89d7c74-c4srn" Dec 03 11:51:05 crc kubenswrapper[5052]: I1203 11:51:05.292604 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-cjpxg\" (UniqueName: \"kubernetes.io/projected/584deaba-f945-42a2-8c0d-e531d6f48fd8-kube-api-access-cjpxg\") pod \"barbican-worker-6d7f657c65-cqxbg\" (UID: \"584deaba-f945-42a2-8c0d-e531d6f48fd8\") " pod="openstack-kuttl-tests/barbican-worker-6d7f657c65-cqxbg" Dec 03 11:51:05 crc kubenswrapper[5052]: I1203 11:51:05.292622 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5679a95d-a498-448e-97ed-497496520506-combined-ca-bundle\") pod \"barbican-keystone-listener-7bf89d7c74-c4srn\" (UID: \"5679a95d-a498-448e-97ed-497496520506\") " pod="openstack-kuttl-tests/barbican-keystone-listener-7bf89d7c74-c4srn" Dec 03 11:51:05 crc kubenswrapper[5052]: I1203 11:51:05.292685 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-7csfc\" (UniqueName: \"kubernetes.io/projected/5679a95d-a498-448e-97ed-497496520506-kube-api-access-7csfc\") pod \"barbican-keystone-listener-7bf89d7c74-c4srn\" (UID: \"5679a95d-a498-448e-97ed-497496520506\") " pod="openstack-kuttl-tests/barbican-keystone-listener-7bf89d7c74-c4srn" Dec 03 11:51:05 crc kubenswrapper[5052]: I1203 11:51:05.292725 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/584deaba-f945-42a2-8c0d-e531d6f48fd8-config-data-custom\") pod \"barbican-worker-6d7f657c65-cqxbg\" (UID: \"584deaba-f945-42a2-8c0d-e531d6f48fd8\") " pod="openstack-kuttl-tests/barbican-worker-6d7f657c65-cqxbg" Dec 03 11:51:05 crc kubenswrapper[5052]: I1203 11:51:05.292746 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/5679a95d-a498-448e-97ed-497496520506-config-data-custom\") pod \"barbican-keystone-listener-7bf89d7c74-c4srn\" (UID: \"5679a95d-a498-448e-97ed-497496520506\") " pod="openstack-kuttl-tests/barbican-keystone-listener-7bf89d7c74-c4srn" Dec 03 11:51:05 crc kubenswrapper[5052]: I1203 11:51:05.294303 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/5679a95d-a498-448e-97ed-497496520506-logs\") pod \"barbican-keystone-listener-7bf89d7c74-c4srn\" (UID: \"5679a95d-a498-448e-97ed-497496520506\") " pod="openstack-kuttl-tests/barbican-keystone-listener-7bf89d7c74-c4srn" Dec 03 11:51:05 crc kubenswrapper[5052]: I1203 11:51:05.295838 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/584deaba-f945-42a2-8c0d-e531d6f48fd8-logs\") pod \"barbican-worker-6d7f657c65-cqxbg\" (UID: \"584deaba-f945-42a2-8c0d-e531d6f48fd8\") " pod="openstack-kuttl-tests/barbican-worker-6d7f657c65-cqxbg" Dec 03 11:51:05 crc kubenswrapper[5052]: I1203 11:51:05.300321 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/584deaba-f945-42a2-8c0d-e531d6f48fd8-config-data-custom\") pod \"barbican-worker-6d7f657c65-cqxbg\" (UID: \"584deaba-f945-42a2-8c0d-e531d6f48fd8\") " pod="openstack-kuttl-tests/barbican-worker-6d7f657c65-cqxbg" Dec 03 11:51:05 crc kubenswrapper[5052]: I1203 11:51:05.300706 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/584deaba-f945-42a2-8c0d-e531d6f48fd8-combined-ca-bundle\") pod \"barbican-worker-6d7f657c65-cqxbg\" (UID: \"584deaba-f945-42a2-8c0d-e531d6f48fd8\") " pod="openstack-kuttl-tests/barbican-worker-6d7f657c65-cqxbg" Dec 03 11:51:05 crc kubenswrapper[5052]: I1203 11:51:05.305428 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/5679a95d-a498-448e-97ed-497496520506-config-data\") pod \"barbican-keystone-listener-7bf89d7c74-c4srn\" (UID: \"5679a95d-a498-448e-97ed-497496520506\") " pod="openstack-kuttl-tests/barbican-keystone-listener-7bf89d7c74-c4srn" Dec 03 11:51:05 crc kubenswrapper[5052]: I1203 11:51:05.307547 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/584deaba-f945-42a2-8c0d-e531d6f48fd8-config-data\") pod \"barbican-worker-6d7f657c65-cqxbg\" (UID: \"584deaba-f945-42a2-8c0d-e531d6f48fd8\") " pod="openstack-kuttl-tests/barbican-worker-6d7f657c65-cqxbg" Dec 03 11:51:05 crc kubenswrapper[5052]: I1203 11:51:05.324055 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/5679a95d-a498-448e-97ed-497496520506-config-data-custom\") pod \"barbican-keystone-listener-7bf89d7c74-c4srn\" (UID: \"5679a95d-a498-448e-97ed-497496520506\") " pod="openstack-kuttl-tests/barbican-keystone-listener-7bf89d7c74-c4srn" Dec 03 11:51:05 crc kubenswrapper[5052]: I1203 11:51:05.325159 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5679a95d-a498-448e-97ed-497496520506-combined-ca-bundle\") pod \"barbican-keystone-listener-7bf89d7c74-c4srn\" (UID: \"5679a95d-a498-448e-97ed-497496520506\") " pod="openstack-kuttl-tests/barbican-keystone-listener-7bf89d7c74-c4srn" Dec 03 11:51:05 crc kubenswrapper[5052]: I1203 11:51:05.326619 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/barbican-api-67d5b9d8c8-4gw9z"] Dec 03 11:51:05 crc kubenswrapper[5052]: I1203 11:51:05.333148 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-api-67d5b9d8c8-4gw9z" Dec 03 11:51:05 crc kubenswrapper[5052]: I1203 11:51:05.341395 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"barbican-api-config-data" Dec 03 11:51:05 crc kubenswrapper[5052]: I1203 11:51:05.349103 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-7csfc\" (UniqueName: \"kubernetes.io/projected/5679a95d-a498-448e-97ed-497496520506-kube-api-access-7csfc\") pod \"barbican-keystone-listener-7bf89d7c74-c4srn\" (UID: \"5679a95d-a498-448e-97ed-497496520506\") " pod="openstack-kuttl-tests/barbican-keystone-listener-7bf89d7c74-c4srn" Dec 03 11:51:05 crc kubenswrapper[5052]: I1203 11:51:05.370642 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-cjpxg\" (UniqueName: \"kubernetes.io/projected/584deaba-f945-42a2-8c0d-e531d6f48fd8-kube-api-access-cjpxg\") pod \"barbican-worker-6d7f657c65-cqxbg\" (UID: \"584deaba-f945-42a2-8c0d-e531d6f48fd8\") " pod="openstack-kuttl-tests/barbican-worker-6d7f657c65-cqxbg" Dec 03 11:51:05 crc kubenswrapper[5052]: I1203 11:51:05.385833 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-api-67d5b9d8c8-4gw9z"] Dec 03 11:51:05 crc kubenswrapper[5052]: I1203 11:51:05.404984 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2b36c173-6bfb-4fcc-858f-19e0ababbf95-combined-ca-bundle\") pod \"barbican-api-67d5b9d8c8-4gw9z\" (UID: \"2b36c173-6bfb-4fcc-858f-19e0ababbf95\") " pod="openstack-kuttl-tests/barbican-api-67d5b9d8c8-4gw9z" Dec 03 11:51:05 crc kubenswrapper[5052]: I1203 11:51:05.405031 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/2b36c173-6bfb-4fcc-858f-19e0ababbf95-config-data\") pod \"barbican-api-67d5b9d8c8-4gw9z\" (UID: \"2b36c173-6bfb-4fcc-858f-19e0ababbf95\") " pod="openstack-kuttl-tests/barbican-api-67d5b9d8c8-4gw9z" Dec 03 11:51:05 crc kubenswrapper[5052]: I1203 11:51:05.405135 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rfgp2\" (UniqueName: \"kubernetes.io/projected/2b36c173-6bfb-4fcc-858f-19e0ababbf95-kube-api-access-rfgp2\") pod \"barbican-api-67d5b9d8c8-4gw9z\" (UID: \"2b36c173-6bfb-4fcc-858f-19e0ababbf95\") " pod="openstack-kuttl-tests/barbican-api-67d5b9d8c8-4gw9z" Dec 03 11:51:05 crc kubenswrapper[5052]: I1203 11:51:05.405274 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/2b36c173-6bfb-4fcc-858f-19e0ababbf95-config-data-custom\") pod \"barbican-api-67d5b9d8c8-4gw9z\" (UID: \"2b36c173-6bfb-4fcc-858f-19e0ababbf95\") " pod="openstack-kuttl-tests/barbican-api-67d5b9d8c8-4gw9z" Dec 03 11:51:05 crc kubenswrapper[5052]: I1203 11:51:05.405317 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/2b36c173-6bfb-4fcc-858f-19e0ababbf95-logs\") pod \"barbican-api-67d5b9d8c8-4gw9z\" (UID: \"2b36c173-6bfb-4fcc-858f-19e0ababbf95\") " pod="openstack-kuttl-tests/barbican-api-67d5b9d8c8-4gw9z" Dec 03 11:51:05 crc kubenswrapper[5052]: I1203 11:51:05.449189 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-db-sync-lgtfz" Dec 03 11:51:05 crc kubenswrapper[5052]: I1203 11:51:05.479960 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-keystone-listener-7bf89d7c74-c4srn" Dec 03 11:51:05 crc kubenswrapper[5052]: I1203 11:51:05.495807 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-worker-6d7f657c65-cqxbg" Dec 03 11:51:05 crc kubenswrapper[5052]: I1203 11:51:05.508522 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5ddb757c-c235-4a2f-b6db-0d120215edcf-combined-ca-bundle\") pod \"5ddb757c-c235-4a2f-b6db-0d120215edcf\" (UID: \"5ddb757c-c235-4a2f-b6db-0d120215edcf\") " Dec 03 11:51:05 crc kubenswrapper[5052]: I1203 11:51:05.508781 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sjbmp\" (UniqueName: \"kubernetes.io/projected/5ddb757c-c235-4a2f-b6db-0d120215edcf-kube-api-access-sjbmp\") pod \"5ddb757c-c235-4a2f-b6db-0d120215edcf\" (UID: \"5ddb757c-c235-4a2f-b6db-0d120215edcf\") " Dec 03 11:51:05 crc kubenswrapper[5052]: I1203 11:51:05.508900 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/5ddb757c-c235-4a2f-b6db-0d120215edcf-etc-machine-id\") pod \"5ddb757c-c235-4a2f-b6db-0d120215edcf\" (UID: \"5ddb757c-c235-4a2f-b6db-0d120215edcf\") " Dec 03 11:51:05 crc kubenswrapper[5052]: I1203 11:51:05.509913 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/5ddb757c-c235-4a2f-b6db-0d120215edcf-db-sync-config-data\") pod \"5ddb757c-c235-4a2f-b6db-0d120215edcf\" (UID: \"5ddb757c-c235-4a2f-b6db-0d120215edcf\") " Dec 03 11:51:05 crc kubenswrapper[5052]: I1203 11:51:05.510140 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/5ddb757c-c235-4a2f-b6db-0d120215edcf-scripts\") pod \"5ddb757c-c235-4a2f-b6db-0d120215edcf\" (UID: \"5ddb757c-c235-4a2f-b6db-0d120215edcf\") " Dec 03 11:51:05 crc kubenswrapper[5052]: I1203 11:51:05.510637 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/5ddb757c-c235-4a2f-b6db-0d120215edcf-config-data\") pod \"5ddb757c-c235-4a2f-b6db-0d120215edcf\" (UID: \"5ddb757c-c235-4a2f-b6db-0d120215edcf\") " Dec 03 11:51:05 crc kubenswrapper[5052]: I1203 11:51:05.511118 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/2b36c173-6bfb-4fcc-858f-19e0ababbf95-config-data-custom\") pod \"barbican-api-67d5b9d8c8-4gw9z\" (UID: \"2b36c173-6bfb-4fcc-858f-19e0ababbf95\") " pod="openstack-kuttl-tests/barbican-api-67d5b9d8c8-4gw9z" Dec 03 11:51:05 crc kubenswrapper[5052]: I1203 11:51:05.511376 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/2b36c173-6bfb-4fcc-858f-19e0ababbf95-logs\") pod \"barbican-api-67d5b9d8c8-4gw9z\" (UID: \"2b36c173-6bfb-4fcc-858f-19e0ababbf95\") " pod="openstack-kuttl-tests/barbican-api-67d5b9d8c8-4gw9z" Dec 03 11:51:05 crc kubenswrapper[5052]: I1203 11:51:05.511605 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2b36c173-6bfb-4fcc-858f-19e0ababbf95-combined-ca-bundle\") pod \"barbican-api-67d5b9d8c8-4gw9z\" (UID: \"2b36c173-6bfb-4fcc-858f-19e0ababbf95\") " pod="openstack-kuttl-tests/barbican-api-67d5b9d8c8-4gw9z" Dec 03 11:51:05 crc kubenswrapper[5052]: I1203 11:51:05.511668 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/5ddb757c-c235-4a2f-b6db-0d120215edcf-etc-machine-id" (OuterVolumeSpecName: "etc-machine-id") pod "5ddb757c-c235-4a2f-b6db-0d120215edcf" (UID: "5ddb757c-c235-4a2f-b6db-0d120215edcf"). InnerVolumeSpecName "etc-machine-id". PluginName "kubernetes.io/host-path", VolumeGidValue "" Dec 03 11:51:05 crc kubenswrapper[5052]: I1203 11:51:05.511786 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/2b36c173-6bfb-4fcc-858f-19e0ababbf95-config-data\") pod \"barbican-api-67d5b9d8c8-4gw9z\" (UID: \"2b36c173-6bfb-4fcc-858f-19e0ababbf95\") " pod="openstack-kuttl-tests/barbican-api-67d5b9d8c8-4gw9z" Dec 03 11:51:05 crc kubenswrapper[5052]: I1203 11:51:05.512232 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-rfgp2\" (UniqueName: \"kubernetes.io/projected/2b36c173-6bfb-4fcc-858f-19e0ababbf95-kube-api-access-rfgp2\") pod \"barbican-api-67d5b9d8c8-4gw9z\" (UID: \"2b36c173-6bfb-4fcc-858f-19e0ababbf95\") " pod="openstack-kuttl-tests/barbican-api-67d5b9d8c8-4gw9z" Dec 03 11:51:05 crc kubenswrapper[5052]: I1203 11:51:05.512406 5052 reconciler_common.go:293] "Volume detached for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/5ddb757c-c235-4a2f-b6db-0d120215edcf-etc-machine-id\") on node \"crc\" DevicePath \"\"" Dec 03 11:51:05 crc kubenswrapper[5052]: I1203 11:51:05.512649 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5ddb757c-c235-4a2f-b6db-0d120215edcf-kube-api-access-sjbmp" (OuterVolumeSpecName: "kube-api-access-sjbmp") pod "5ddb757c-c235-4a2f-b6db-0d120215edcf" (UID: "5ddb757c-c235-4a2f-b6db-0d120215edcf"). InnerVolumeSpecName "kube-api-access-sjbmp". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:51:05 crc kubenswrapper[5052]: I1203 11:51:05.513191 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/2b36c173-6bfb-4fcc-858f-19e0ababbf95-logs\") pod \"barbican-api-67d5b9d8c8-4gw9z\" (UID: \"2b36c173-6bfb-4fcc-858f-19e0ababbf95\") " pod="openstack-kuttl-tests/barbican-api-67d5b9d8c8-4gw9z" Dec 03 11:51:05 crc kubenswrapper[5052]: I1203 11:51:05.518595 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/2b36c173-6bfb-4fcc-858f-19e0ababbf95-config-data-custom\") pod \"barbican-api-67d5b9d8c8-4gw9z\" (UID: \"2b36c173-6bfb-4fcc-858f-19e0ababbf95\") " pod="openstack-kuttl-tests/barbican-api-67d5b9d8c8-4gw9z" Dec 03 11:51:05 crc kubenswrapper[5052]: I1203 11:51:05.518721 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5ddb757c-c235-4a2f-b6db-0d120215edcf-scripts" (OuterVolumeSpecName: "scripts") pod "5ddb757c-c235-4a2f-b6db-0d120215edcf" (UID: "5ddb757c-c235-4a2f-b6db-0d120215edcf"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:51:05 crc kubenswrapper[5052]: I1203 11:51:05.519226 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2b36c173-6bfb-4fcc-858f-19e0ababbf95-combined-ca-bundle\") pod \"barbican-api-67d5b9d8c8-4gw9z\" (UID: \"2b36c173-6bfb-4fcc-858f-19e0ababbf95\") " pod="openstack-kuttl-tests/barbican-api-67d5b9d8c8-4gw9z" Dec 03 11:51:05 crc kubenswrapper[5052]: I1203 11:51:05.520696 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/2b36c173-6bfb-4fcc-858f-19e0ababbf95-config-data\") pod \"barbican-api-67d5b9d8c8-4gw9z\" (UID: \"2b36c173-6bfb-4fcc-858f-19e0ababbf95\") " pod="openstack-kuttl-tests/barbican-api-67d5b9d8c8-4gw9z" Dec 03 11:51:05 crc kubenswrapper[5052]: I1203 11:51:05.528350 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5ddb757c-c235-4a2f-b6db-0d120215edcf-db-sync-config-data" (OuterVolumeSpecName: "db-sync-config-data") pod "5ddb757c-c235-4a2f-b6db-0d120215edcf" (UID: "5ddb757c-c235-4a2f-b6db-0d120215edcf"). InnerVolumeSpecName "db-sync-config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:51:05 crc kubenswrapper[5052]: I1203 11:51:05.529590 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-rfgp2\" (UniqueName: \"kubernetes.io/projected/2b36c173-6bfb-4fcc-858f-19e0ababbf95-kube-api-access-rfgp2\") pod \"barbican-api-67d5b9d8c8-4gw9z\" (UID: \"2b36c173-6bfb-4fcc-858f-19e0ababbf95\") " pod="openstack-kuttl-tests/barbican-api-67d5b9d8c8-4gw9z" Dec 03 11:51:05 crc kubenswrapper[5052]: I1203 11:51:05.557396 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5ddb757c-c235-4a2f-b6db-0d120215edcf-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "5ddb757c-c235-4a2f-b6db-0d120215edcf" (UID: "5ddb757c-c235-4a2f-b6db-0d120215edcf"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:51:05 crc kubenswrapper[5052]: I1203 11:51:05.581306 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5ddb757c-c235-4a2f-b6db-0d120215edcf-config-data" (OuterVolumeSpecName: "config-data") pod "5ddb757c-c235-4a2f-b6db-0d120215edcf" (UID: "5ddb757c-c235-4a2f-b6db-0d120215edcf"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:51:05 crc kubenswrapper[5052]: I1203 11:51:05.618440 5052 reconciler_common.go:293] "Volume detached for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/5ddb757c-c235-4a2f-b6db-0d120215edcf-db-sync-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:51:05 crc kubenswrapper[5052]: I1203 11:51:05.618764 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/5ddb757c-c235-4a2f-b6db-0d120215edcf-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:51:05 crc kubenswrapper[5052]: I1203 11:51:05.618775 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/5ddb757c-c235-4a2f-b6db-0d120215edcf-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:51:05 crc kubenswrapper[5052]: I1203 11:51:05.618784 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5ddb757c-c235-4a2f-b6db-0d120215edcf-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:51:05 crc kubenswrapper[5052]: I1203 11:51:05.618793 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-sjbmp\" (UniqueName: \"kubernetes.io/projected/5ddb757c-c235-4a2f-b6db-0d120215edcf-kube-api-access-sjbmp\") on node \"crc\" DevicePath \"\"" Dec 03 11:51:05 crc kubenswrapper[5052]: I1203 11:51:05.761536 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-api-67d5b9d8c8-4gw9z" Dec 03 11:51:05 crc kubenswrapper[5052]: I1203 11:51:05.768218 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-worker-6d7f657c65-cqxbg"] Dec 03 11:51:05 crc kubenswrapper[5052]: I1203 11:51:05.929674 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-db-sync-lgtfz" event={"ID":"5ddb757c-c235-4a2f-b6db-0d120215edcf","Type":"ContainerDied","Data":"54c691cdb66a1ef8d58fcbeabe48d93756fdbed671d58afd1a0870677ec28549"} Dec 03 11:51:05 crc kubenswrapper[5052]: I1203 11:51:05.929984 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="54c691cdb66a1ef8d58fcbeabe48d93756fdbed671d58afd1a0870677ec28549" Dec 03 11:51:05 crc kubenswrapper[5052]: I1203 11:51:05.930067 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-db-sync-lgtfz" Dec 03 11:51:05 crc kubenswrapper[5052]: I1203 11:51:05.942369 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-worker-6d7f657c65-cqxbg" event={"ID":"584deaba-f945-42a2-8c0d-e531d6f48fd8","Type":"ContainerStarted","Data":"945068210f2d3452572796b34ccff45ea13b851c61feca7b498419c59ada21cc"} Dec 03 11:51:05 crc kubenswrapper[5052]: I1203 11:51:05.942415 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/placement-5fc4755558-qz76x" Dec 03 11:51:06 crc kubenswrapper[5052]: I1203 11:51:06.041271 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-keystone-listener-7bf89d7c74-c4srn"] Dec 03 11:51:06 crc kubenswrapper[5052]: W1203 11:51:06.055619 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5679a95d_a498_448e_97ed_497496520506.slice/crio-53669a1fe2b33a6066722e089f916ce83215ef76bae4fcb17bcd2d37ebe966bb WatchSource:0}: Error finding container 53669a1fe2b33a6066722e089f916ce83215ef76bae4fcb17bcd2d37ebe966bb: Status 404 returned error can't find the container with id 53669a1fe2b33a6066722e089f916ce83215ef76bae4fcb17bcd2d37ebe966bb Dec 03 11:51:06 crc kubenswrapper[5052]: I1203 11:51:06.250012 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 11:51:06 crc kubenswrapper[5052]: E1203 11:51:06.250563 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="5ddb757c-c235-4a2f-b6db-0d120215edcf" containerName="cinder-db-sync" Dec 03 11:51:06 crc kubenswrapper[5052]: I1203 11:51:06.250592 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ddb757c-c235-4a2f-b6db-0d120215edcf" containerName="cinder-db-sync" Dec 03 11:51:06 crc kubenswrapper[5052]: I1203 11:51:06.250856 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="5ddb757c-c235-4a2f-b6db-0d120215edcf" containerName="cinder-db-sync" Dec 03 11:51:06 crc kubenswrapper[5052]: I1203 11:51:06.252151 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:51:06 crc kubenswrapper[5052]: I1203 11:51:06.256304 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cinder-cinder-dockercfg-vkzbs" Dec 03 11:51:06 crc kubenswrapper[5052]: I1203 11:51:06.256592 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cinder-scripts" Dec 03 11:51:06 crc kubenswrapper[5052]: I1203 11:51:06.256613 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cinder-scheduler-config-data" Dec 03 11:51:06 crc kubenswrapper[5052]: I1203 11:51:06.256808 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cinder-config-data" Dec 03 11:51:06 crc kubenswrapper[5052]: I1203 11:51:06.260577 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 11:51:06 crc kubenswrapper[5052]: I1203 11:51:06.272270 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-api-67d5b9d8c8-4gw9z"] Dec 03 11:51:06 crc kubenswrapper[5052]: I1203 11:51:06.331540 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/61d6dd4f-56ef-4f1d-8536-ea66f56188ce-combined-ca-bundle\") pod \"cinder-scheduler-0\" (UID: \"61d6dd4f-56ef-4f1d-8536-ea66f56188ce\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:51:06 crc kubenswrapper[5052]: I1203 11:51:06.331932 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/61d6dd4f-56ef-4f1d-8536-ea66f56188ce-etc-machine-id\") pod \"cinder-scheduler-0\" (UID: \"61d6dd4f-56ef-4f1d-8536-ea66f56188ce\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:51:06 crc kubenswrapper[5052]: I1203 11:51:06.332021 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/61d6dd4f-56ef-4f1d-8536-ea66f56188ce-config-data\") pod \"cinder-scheduler-0\" (UID: \"61d6dd4f-56ef-4f1d-8536-ea66f56188ce\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:51:06 crc kubenswrapper[5052]: I1203 11:51:06.332072 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/61d6dd4f-56ef-4f1d-8536-ea66f56188ce-scripts\") pod \"cinder-scheduler-0\" (UID: \"61d6dd4f-56ef-4f1d-8536-ea66f56188ce\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:51:06 crc kubenswrapper[5052]: I1203 11:51:06.332105 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-79djt\" (UniqueName: \"kubernetes.io/projected/61d6dd4f-56ef-4f1d-8536-ea66f56188ce-kube-api-access-79djt\") pod \"cinder-scheduler-0\" (UID: \"61d6dd4f-56ef-4f1d-8536-ea66f56188ce\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:51:06 crc kubenswrapper[5052]: I1203 11:51:06.332173 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/61d6dd4f-56ef-4f1d-8536-ea66f56188ce-config-data-custom\") pod \"cinder-scheduler-0\" (UID: \"61d6dd4f-56ef-4f1d-8536-ea66f56188ce\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:51:06 crc kubenswrapper[5052]: I1203 11:51:06.409960 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 11:51:06 crc kubenswrapper[5052]: I1203 11:51:06.414036 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:51:06 crc kubenswrapper[5052]: I1203 11:51:06.420386 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 11:51:06 crc kubenswrapper[5052]: I1203 11:51:06.422951 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cinder-api-config-data" Dec 03 11:51:06 crc kubenswrapper[5052]: I1203 11:51:06.436599 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/61d6dd4f-56ef-4f1d-8536-ea66f56188ce-config-data\") pod \"cinder-scheduler-0\" (UID: \"61d6dd4f-56ef-4f1d-8536-ea66f56188ce\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:51:06 crc kubenswrapper[5052]: I1203 11:51:06.436667 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/61d6dd4f-56ef-4f1d-8536-ea66f56188ce-scripts\") pod \"cinder-scheduler-0\" (UID: \"61d6dd4f-56ef-4f1d-8536-ea66f56188ce\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:51:06 crc kubenswrapper[5052]: I1203 11:51:06.436696 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-79djt\" (UniqueName: \"kubernetes.io/projected/61d6dd4f-56ef-4f1d-8536-ea66f56188ce-kube-api-access-79djt\") pod \"cinder-scheduler-0\" (UID: \"61d6dd4f-56ef-4f1d-8536-ea66f56188ce\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:51:06 crc kubenswrapper[5052]: I1203 11:51:06.436740 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/61d6dd4f-56ef-4f1d-8536-ea66f56188ce-config-data-custom\") pod \"cinder-scheduler-0\" (UID: \"61d6dd4f-56ef-4f1d-8536-ea66f56188ce\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:51:06 crc kubenswrapper[5052]: I1203 11:51:06.436782 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/61d6dd4f-56ef-4f1d-8536-ea66f56188ce-combined-ca-bundle\") pod \"cinder-scheduler-0\" (UID: \"61d6dd4f-56ef-4f1d-8536-ea66f56188ce\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:51:06 crc kubenswrapper[5052]: I1203 11:51:06.436822 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/61d6dd4f-56ef-4f1d-8536-ea66f56188ce-etc-machine-id\") pod \"cinder-scheduler-0\" (UID: \"61d6dd4f-56ef-4f1d-8536-ea66f56188ce\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:51:06 crc kubenswrapper[5052]: I1203 11:51:06.436900 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/61d6dd4f-56ef-4f1d-8536-ea66f56188ce-etc-machine-id\") pod \"cinder-scheduler-0\" (UID: \"61d6dd4f-56ef-4f1d-8536-ea66f56188ce\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:51:06 crc kubenswrapper[5052]: I1203 11:51:06.448697 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/61d6dd4f-56ef-4f1d-8536-ea66f56188ce-config-data\") pod \"cinder-scheduler-0\" (UID: \"61d6dd4f-56ef-4f1d-8536-ea66f56188ce\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:51:06 crc kubenswrapper[5052]: I1203 11:51:06.449033 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/61d6dd4f-56ef-4f1d-8536-ea66f56188ce-scripts\") pod \"cinder-scheduler-0\" (UID: \"61d6dd4f-56ef-4f1d-8536-ea66f56188ce\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:51:06 crc kubenswrapper[5052]: I1203 11:51:06.451310 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/61d6dd4f-56ef-4f1d-8536-ea66f56188ce-config-data-custom\") pod \"cinder-scheduler-0\" (UID: \"61d6dd4f-56ef-4f1d-8536-ea66f56188ce\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:51:06 crc kubenswrapper[5052]: I1203 11:51:06.457672 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/61d6dd4f-56ef-4f1d-8536-ea66f56188ce-combined-ca-bundle\") pod \"cinder-scheduler-0\" (UID: \"61d6dd4f-56ef-4f1d-8536-ea66f56188ce\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:51:06 crc kubenswrapper[5052]: I1203 11:51:06.477047 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-79djt\" (UniqueName: \"kubernetes.io/projected/61d6dd4f-56ef-4f1d-8536-ea66f56188ce-kube-api-access-79djt\") pod \"cinder-scheduler-0\" (UID: \"61d6dd4f-56ef-4f1d-8536-ea66f56188ce\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:51:06 crc kubenswrapper[5052]: I1203 11:51:06.538709 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a2fba22f-a015-40f4-8a93-e1011fe64e09-combined-ca-bundle\") pod \"cinder-api-0\" (UID: \"a2fba22f-a015-40f4-8a93-e1011fe64e09\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:51:06 crc kubenswrapper[5052]: I1203 11:51:06.538777 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/a2fba22f-a015-40f4-8a93-e1011fe64e09-etc-machine-id\") pod \"cinder-api-0\" (UID: \"a2fba22f-a015-40f4-8a93-e1011fe64e09\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:51:06 crc kubenswrapper[5052]: I1203 11:51:06.538818 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/a2fba22f-a015-40f4-8a93-e1011fe64e09-config-data-custom\") pod \"cinder-api-0\" (UID: \"a2fba22f-a015-40f4-8a93-e1011fe64e09\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:51:06 crc kubenswrapper[5052]: I1203 11:51:06.538894 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/a2fba22f-a015-40f4-8a93-e1011fe64e09-scripts\") pod \"cinder-api-0\" (UID: \"a2fba22f-a015-40f4-8a93-e1011fe64e09\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:51:06 crc kubenswrapper[5052]: I1203 11:51:06.538918 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zm95z\" (UniqueName: \"kubernetes.io/projected/a2fba22f-a015-40f4-8a93-e1011fe64e09-kube-api-access-zm95z\") pod \"cinder-api-0\" (UID: \"a2fba22f-a015-40f4-8a93-e1011fe64e09\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:51:06 crc kubenswrapper[5052]: I1203 11:51:06.538938 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a2fba22f-a015-40f4-8a93-e1011fe64e09-config-data\") pod \"cinder-api-0\" (UID: \"a2fba22f-a015-40f4-8a93-e1011fe64e09\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:51:06 crc kubenswrapper[5052]: I1203 11:51:06.538964 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/a2fba22f-a015-40f4-8a93-e1011fe64e09-logs\") pod \"cinder-api-0\" (UID: \"a2fba22f-a015-40f4-8a93-e1011fe64e09\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:51:06 crc kubenswrapper[5052]: I1203 11:51:06.589097 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:51:06 crc kubenswrapper[5052]: I1203 11:51:06.640355 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/a2fba22f-a015-40f4-8a93-e1011fe64e09-scripts\") pod \"cinder-api-0\" (UID: \"a2fba22f-a015-40f4-8a93-e1011fe64e09\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:51:06 crc kubenswrapper[5052]: I1203 11:51:06.640391 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-zm95z\" (UniqueName: \"kubernetes.io/projected/a2fba22f-a015-40f4-8a93-e1011fe64e09-kube-api-access-zm95z\") pod \"cinder-api-0\" (UID: \"a2fba22f-a015-40f4-8a93-e1011fe64e09\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:51:06 crc kubenswrapper[5052]: I1203 11:51:06.640410 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a2fba22f-a015-40f4-8a93-e1011fe64e09-config-data\") pod \"cinder-api-0\" (UID: \"a2fba22f-a015-40f4-8a93-e1011fe64e09\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:51:06 crc kubenswrapper[5052]: I1203 11:51:06.640436 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/a2fba22f-a015-40f4-8a93-e1011fe64e09-logs\") pod \"cinder-api-0\" (UID: \"a2fba22f-a015-40f4-8a93-e1011fe64e09\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:51:06 crc kubenswrapper[5052]: I1203 11:51:06.640501 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a2fba22f-a015-40f4-8a93-e1011fe64e09-combined-ca-bundle\") pod \"cinder-api-0\" (UID: \"a2fba22f-a015-40f4-8a93-e1011fe64e09\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:51:06 crc kubenswrapper[5052]: I1203 11:51:06.640546 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/a2fba22f-a015-40f4-8a93-e1011fe64e09-etc-machine-id\") pod \"cinder-api-0\" (UID: \"a2fba22f-a015-40f4-8a93-e1011fe64e09\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:51:06 crc kubenswrapper[5052]: I1203 11:51:06.640573 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/a2fba22f-a015-40f4-8a93-e1011fe64e09-config-data-custom\") pod \"cinder-api-0\" (UID: \"a2fba22f-a015-40f4-8a93-e1011fe64e09\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:51:06 crc kubenswrapper[5052]: I1203 11:51:06.640935 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/a2fba22f-a015-40f4-8a93-e1011fe64e09-etc-machine-id\") pod \"cinder-api-0\" (UID: \"a2fba22f-a015-40f4-8a93-e1011fe64e09\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:51:06 crc kubenswrapper[5052]: I1203 11:51:06.641670 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/a2fba22f-a015-40f4-8a93-e1011fe64e09-logs\") pod \"cinder-api-0\" (UID: \"a2fba22f-a015-40f4-8a93-e1011fe64e09\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:51:06 crc kubenswrapper[5052]: I1203 11:51:06.644163 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/a2fba22f-a015-40f4-8a93-e1011fe64e09-config-data-custom\") pod \"cinder-api-0\" (UID: \"a2fba22f-a015-40f4-8a93-e1011fe64e09\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:51:06 crc kubenswrapper[5052]: I1203 11:51:06.645013 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a2fba22f-a015-40f4-8a93-e1011fe64e09-combined-ca-bundle\") pod \"cinder-api-0\" (UID: \"a2fba22f-a015-40f4-8a93-e1011fe64e09\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:51:06 crc kubenswrapper[5052]: I1203 11:51:06.645287 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a2fba22f-a015-40f4-8a93-e1011fe64e09-config-data\") pod \"cinder-api-0\" (UID: \"a2fba22f-a015-40f4-8a93-e1011fe64e09\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:51:06 crc kubenswrapper[5052]: I1203 11:51:06.651075 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/a2fba22f-a015-40f4-8a93-e1011fe64e09-scripts\") pod \"cinder-api-0\" (UID: \"a2fba22f-a015-40f4-8a93-e1011fe64e09\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:51:06 crc kubenswrapper[5052]: I1203 11:51:06.662886 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-zm95z\" (UniqueName: \"kubernetes.io/projected/a2fba22f-a015-40f4-8a93-e1011fe64e09-kube-api-access-zm95z\") pod \"cinder-api-0\" (UID: \"a2fba22f-a015-40f4-8a93-e1011fe64e09\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:51:06 crc kubenswrapper[5052]: I1203 11:51:06.834082 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:51:07 crc kubenswrapper[5052]: I1203 11:51:06.970834 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-keystone-listener-7bf89d7c74-c4srn" event={"ID":"5679a95d-a498-448e-97ed-497496520506","Type":"ContainerStarted","Data":"3125ced637b6686a088afdde74ed21eda056777bed8dca841a4e6549c3cf70c8"} Dec 03 11:51:07 crc kubenswrapper[5052]: I1203 11:51:06.970876 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-keystone-listener-7bf89d7c74-c4srn" event={"ID":"5679a95d-a498-448e-97ed-497496520506","Type":"ContainerStarted","Data":"b657a1c95ebe202d32286143c3e1cb3b321d2c67300afcf9cb53509b8a6aee42"} Dec 03 11:51:07 crc kubenswrapper[5052]: I1203 11:51:06.970886 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-keystone-listener-7bf89d7c74-c4srn" event={"ID":"5679a95d-a498-448e-97ed-497496520506","Type":"ContainerStarted","Data":"53669a1fe2b33a6066722e089f916ce83215ef76bae4fcb17bcd2d37ebe966bb"} Dec 03 11:51:07 crc kubenswrapper[5052]: I1203 11:51:06.991424 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/barbican-keystone-listener-7bf89d7c74-c4srn" podStartSLOduration=1.9914099269999999 podStartE2EDuration="1.991409927s" podCreationTimestamp="2025-12-03 11:51:05 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:51:06.988833993 +0000 UTC m=+1857.577075757" watchObservedRunningTime="2025-12-03 11:51:06.991409927 +0000 UTC m=+1857.579651681" Dec 03 11:51:07 crc kubenswrapper[5052]: I1203 11:51:06.993766 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-67d5b9d8c8-4gw9z" event={"ID":"2b36c173-6bfb-4fcc-858f-19e0ababbf95","Type":"ContainerStarted","Data":"d6f5255145150903893d430668b91fdf726ecc26c094e89b13fdc75a3e249f5d"} Dec 03 11:51:07 crc kubenswrapper[5052]: I1203 11:51:06.993804 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-67d5b9d8c8-4gw9z" event={"ID":"2b36c173-6bfb-4fcc-858f-19e0ababbf95","Type":"ContainerStarted","Data":"61036e95ab93308bc10e8d596f78981c7d8a8b273993f37dcf6c5a34caac782a"} Dec 03 11:51:07 crc kubenswrapper[5052]: I1203 11:51:06.993815 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-67d5b9d8c8-4gw9z" event={"ID":"2b36c173-6bfb-4fcc-858f-19e0ababbf95","Type":"ContainerStarted","Data":"9f9555b543dd7d0083b56266b19403629f3d7459858e6f043e0f32e7c4f7e521"} Dec 03 11:51:07 crc kubenswrapper[5052]: I1203 11:51:06.993840 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/barbican-api-67d5b9d8c8-4gw9z" Dec 03 11:51:07 crc kubenswrapper[5052]: I1203 11:51:06.993854 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/barbican-api-67d5b9d8c8-4gw9z" Dec 03 11:51:07 crc kubenswrapper[5052]: I1203 11:51:07.016710 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/barbican-api-67d5b9d8c8-4gw9z" podStartSLOduration=2.016691442 podStartE2EDuration="2.016691442s" podCreationTimestamp="2025-12-03 11:51:05 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:51:07.014725085 +0000 UTC m=+1857.602966859" watchObservedRunningTime="2025-12-03 11:51:07.016691442 +0000 UTC m=+1857.604933206" Dec 03 11:51:07 crc kubenswrapper[5052]: I1203 11:51:07.024615 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-worker-6d7f657c65-cqxbg" event={"ID":"584deaba-f945-42a2-8c0d-e531d6f48fd8","Type":"ContainerStarted","Data":"3e624a11d1616304ada395cbc34d76126b3b0f68d661f5c753eefaa96b499a5c"} Dec 03 11:51:07 crc kubenswrapper[5052]: I1203 11:51:07.024688 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-worker-6d7f657c65-cqxbg" event={"ID":"584deaba-f945-42a2-8c0d-e531d6f48fd8","Type":"ContainerStarted","Data":"2ee3c0361bbf90cf40dcd13279e44150c5f8ed97f268cf91a9701bccff60b20b"} Dec 03 11:51:07 crc kubenswrapper[5052]: I1203 11:51:07.038075 5052 generic.go:334] "Generic (PLEG): container finished" podID="c86d8ada-bec6-4d35-978b-23e5cbac83ff" containerID="9393f4a1875bd93dade4f4a250ae2161d3bae5bab0dfeec9d201f006f152194d" exitCode=0 Dec 03 11:51:07 crc kubenswrapper[5052]: I1203 11:51:07.038890 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-bootstrap-hv96x" event={"ID":"c86d8ada-bec6-4d35-978b-23e5cbac83ff","Type":"ContainerDied","Data":"9393f4a1875bd93dade4f4a250ae2161d3bae5bab0dfeec9d201f006f152194d"} Dec 03 11:51:07 crc kubenswrapper[5052]: I1203 11:51:07.069413 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/barbican-worker-6d7f657c65-cqxbg" podStartSLOduration=2.069391763 podStartE2EDuration="2.069391763s" podCreationTimestamp="2025-12-03 11:51:05 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:51:07.061848437 +0000 UTC m=+1857.650090201" watchObservedRunningTime="2025-12-03 11:51:07.069391763 +0000 UTC m=+1857.657633527" Dec 03 11:51:07 crc kubenswrapper[5052]: I1203 11:51:07.148198 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 11:51:07 crc kubenswrapper[5052]: I1203 11:51:07.923636 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 11:51:08 crc kubenswrapper[5052]: I1203 11:51:08.051136 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"a2fba22f-a015-40f4-8a93-e1011fe64e09","Type":"ContainerStarted","Data":"748b422e8a05a370a0584564fb275ccb549575f174c156d7ba55a98bf54cfb5c"} Dec 03 11:51:08 crc kubenswrapper[5052]: I1203 11:51:08.054456 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"61d6dd4f-56ef-4f1d-8536-ea66f56188ce","Type":"ContainerStarted","Data":"88bb12213446d0cc45586efb2f6ce025322fc258ef04095763e5dd0212b596c5"} Dec 03 11:51:08 crc kubenswrapper[5052]: I1203 11:51:08.054528 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"61d6dd4f-56ef-4f1d-8536-ea66f56188ce","Type":"ContainerStarted","Data":"c853d087f6b4b77264beb515e122dbd4f993224b36caaa255ab796b5224a6ecb"} Dec 03 11:51:08 crc kubenswrapper[5052]: I1203 11:51:08.287887 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-bootstrap-hv96x" Dec 03 11:51:08 crc kubenswrapper[5052]: I1203 11:51:08.375825 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/c86d8ada-bec6-4d35-978b-23e5cbac83ff-config-data\") pod \"c86d8ada-bec6-4d35-978b-23e5cbac83ff\" (UID: \"c86d8ada-bec6-4d35-978b-23e5cbac83ff\") " Dec 03 11:51:08 crc kubenswrapper[5052]: I1203 11:51:08.375993 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/c86d8ada-bec6-4d35-978b-23e5cbac83ff-fernet-keys\") pod \"c86d8ada-bec6-4d35-978b-23e5cbac83ff\" (UID: \"c86d8ada-bec6-4d35-978b-23e5cbac83ff\") " Dec 03 11:51:08 crc kubenswrapper[5052]: I1203 11:51:08.376216 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c7flf\" (UniqueName: \"kubernetes.io/projected/c86d8ada-bec6-4d35-978b-23e5cbac83ff-kube-api-access-c7flf\") pod \"c86d8ada-bec6-4d35-978b-23e5cbac83ff\" (UID: \"c86d8ada-bec6-4d35-978b-23e5cbac83ff\") " Dec 03 11:51:08 crc kubenswrapper[5052]: I1203 11:51:08.376310 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c86d8ada-bec6-4d35-978b-23e5cbac83ff-combined-ca-bundle\") pod \"c86d8ada-bec6-4d35-978b-23e5cbac83ff\" (UID: \"c86d8ada-bec6-4d35-978b-23e5cbac83ff\") " Dec 03 11:51:08 crc kubenswrapper[5052]: I1203 11:51:08.376377 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/c86d8ada-bec6-4d35-978b-23e5cbac83ff-scripts\") pod \"c86d8ada-bec6-4d35-978b-23e5cbac83ff\" (UID: \"c86d8ada-bec6-4d35-978b-23e5cbac83ff\") " Dec 03 11:51:08 crc kubenswrapper[5052]: I1203 11:51:08.376457 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/c86d8ada-bec6-4d35-978b-23e5cbac83ff-credential-keys\") pod \"c86d8ada-bec6-4d35-978b-23e5cbac83ff\" (UID: \"c86d8ada-bec6-4d35-978b-23e5cbac83ff\") " Dec 03 11:51:08 crc kubenswrapper[5052]: I1203 11:51:08.385745 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c86d8ada-bec6-4d35-978b-23e5cbac83ff-kube-api-access-c7flf" (OuterVolumeSpecName: "kube-api-access-c7flf") pod "c86d8ada-bec6-4d35-978b-23e5cbac83ff" (UID: "c86d8ada-bec6-4d35-978b-23e5cbac83ff"). InnerVolumeSpecName "kube-api-access-c7flf". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:51:08 crc kubenswrapper[5052]: I1203 11:51:08.391850 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c86d8ada-bec6-4d35-978b-23e5cbac83ff-credential-keys" (OuterVolumeSpecName: "credential-keys") pod "c86d8ada-bec6-4d35-978b-23e5cbac83ff" (UID: "c86d8ada-bec6-4d35-978b-23e5cbac83ff"). InnerVolumeSpecName "credential-keys". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:51:08 crc kubenswrapper[5052]: I1203 11:51:08.393198 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c86d8ada-bec6-4d35-978b-23e5cbac83ff-fernet-keys" (OuterVolumeSpecName: "fernet-keys") pod "c86d8ada-bec6-4d35-978b-23e5cbac83ff" (UID: "c86d8ada-bec6-4d35-978b-23e5cbac83ff"). InnerVolumeSpecName "fernet-keys". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:51:08 crc kubenswrapper[5052]: I1203 11:51:08.394238 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c86d8ada-bec6-4d35-978b-23e5cbac83ff-scripts" (OuterVolumeSpecName: "scripts") pod "c86d8ada-bec6-4d35-978b-23e5cbac83ff" (UID: "c86d8ada-bec6-4d35-978b-23e5cbac83ff"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:51:08 crc kubenswrapper[5052]: I1203 11:51:08.432060 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c86d8ada-bec6-4d35-978b-23e5cbac83ff-config-data" (OuterVolumeSpecName: "config-data") pod "c86d8ada-bec6-4d35-978b-23e5cbac83ff" (UID: "c86d8ada-bec6-4d35-978b-23e5cbac83ff"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:51:08 crc kubenswrapper[5052]: I1203 11:51:08.433116 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c86d8ada-bec6-4d35-978b-23e5cbac83ff-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "c86d8ada-bec6-4d35-978b-23e5cbac83ff" (UID: "c86d8ada-bec6-4d35-978b-23e5cbac83ff"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:51:08 crc kubenswrapper[5052]: I1203 11:51:08.479046 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-c7flf\" (UniqueName: \"kubernetes.io/projected/c86d8ada-bec6-4d35-978b-23e5cbac83ff-kube-api-access-c7flf\") on node \"crc\" DevicePath \"\"" Dec 03 11:51:08 crc kubenswrapper[5052]: I1203 11:51:08.479082 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c86d8ada-bec6-4d35-978b-23e5cbac83ff-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:51:08 crc kubenswrapper[5052]: I1203 11:51:08.479092 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/c86d8ada-bec6-4d35-978b-23e5cbac83ff-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:51:08 crc kubenswrapper[5052]: I1203 11:51:08.479101 5052 reconciler_common.go:293] "Volume detached for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/c86d8ada-bec6-4d35-978b-23e5cbac83ff-credential-keys\") on node \"crc\" DevicePath \"\"" Dec 03 11:51:08 crc kubenswrapper[5052]: I1203 11:51:08.479110 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/c86d8ada-bec6-4d35-978b-23e5cbac83ff-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:51:08 crc kubenswrapper[5052]: I1203 11:51:08.479119 5052 reconciler_common.go:293] "Volume detached for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/c86d8ada-bec6-4d35-978b-23e5cbac83ff-fernet-keys\") on node \"crc\" DevicePath \"\"" Dec 03 11:51:09 crc kubenswrapper[5052]: I1203 11:51:09.067475 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-bootstrap-hv96x" event={"ID":"c86d8ada-bec6-4d35-978b-23e5cbac83ff","Type":"ContainerDied","Data":"ff809a113fa8be6a9a42890ed1a8dd80a83774c6f34292b0d359f00c261c5a40"} Dec 03 11:51:09 crc kubenswrapper[5052]: I1203 11:51:09.067843 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ff809a113fa8be6a9a42890ed1a8dd80a83774c6f34292b0d359f00c261c5a40" Dec 03 11:51:09 crc kubenswrapper[5052]: I1203 11:51:09.067911 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-bootstrap-hv96x" Dec 03 11:51:09 crc kubenswrapper[5052]: I1203 11:51:09.073384 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"61d6dd4f-56ef-4f1d-8536-ea66f56188ce","Type":"ContainerStarted","Data":"ff503e474d5755878e81b768ad453fa52004dce2cb8e5b92698649c6b7907845"} Dec 03 11:51:09 crc kubenswrapper[5052]: I1203 11:51:09.075312 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"a2fba22f-a015-40f4-8a93-e1011fe64e09","Type":"ContainerStarted","Data":"619259e4397089b6aa65dec5b558419311effe8a5ca7087e33cd64a4922abb5f"} Dec 03 11:51:09 crc kubenswrapper[5052]: I1203 11:51:09.096677 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/cinder-scheduler-0" podStartSLOduration=3.096658895 podStartE2EDuration="3.096658895s" podCreationTimestamp="2025-12-03 11:51:06 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:51:09.093610527 +0000 UTC m=+1859.681852301" watchObservedRunningTime="2025-12-03 11:51:09.096658895 +0000 UTC m=+1859.684900659" Dec 03 11:51:09 crc kubenswrapper[5052]: I1203 11:51:09.196606 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/keystone-94f46848f-wgzvx"] Dec 03 11:51:09 crc kubenswrapper[5052]: E1203 11:51:09.197069 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="c86d8ada-bec6-4d35-978b-23e5cbac83ff" containerName="keystone-bootstrap" Dec 03 11:51:09 crc kubenswrapper[5052]: I1203 11:51:09.197090 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="c86d8ada-bec6-4d35-978b-23e5cbac83ff" containerName="keystone-bootstrap" Dec 03 11:51:09 crc kubenswrapper[5052]: I1203 11:51:09.197318 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="c86d8ada-bec6-4d35-978b-23e5cbac83ff" containerName="keystone-bootstrap" Dec 03 11:51:09 crc kubenswrapper[5052]: I1203 11:51:09.199502 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-94f46848f-wgzvx" Dec 03 11:51:09 crc kubenswrapper[5052]: I1203 11:51:09.208364 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-keystone-internal-svc" Dec 03 11:51:09 crc kubenswrapper[5052]: I1203 11:51:09.209039 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone-config-data" Dec 03 11:51:09 crc kubenswrapper[5052]: I1203 11:51:09.209159 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone" Dec 03 11:51:09 crc kubenswrapper[5052]: I1203 11:51:09.209276 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-keystone-public-svc" Dec 03 11:51:09 crc kubenswrapper[5052]: I1203 11:51:09.209372 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone-scripts" Dec 03 11:51:09 crc kubenswrapper[5052]: I1203 11:51:09.209465 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone-keystone-dockercfg-xtl6f" Dec 03 11:51:09 crc kubenswrapper[5052]: I1203 11:51:09.226842 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-94f46848f-wgzvx"] Dec 03 11:51:09 crc kubenswrapper[5052]: I1203 11:51:09.297038 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/775d2ccf-9b52-4274-99ef-2c33b18beaea-scripts\") pod \"keystone-94f46848f-wgzvx\" (UID: \"775d2ccf-9b52-4274-99ef-2c33b18beaea\") " pod="openstack-kuttl-tests/keystone-94f46848f-wgzvx" Dec 03 11:51:09 crc kubenswrapper[5052]: I1203 11:51:09.297102 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/775d2ccf-9b52-4274-99ef-2c33b18beaea-internal-tls-certs\") pod \"keystone-94f46848f-wgzvx\" (UID: \"775d2ccf-9b52-4274-99ef-2c33b18beaea\") " pod="openstack-kuttl-tests/keystone-94f46848f-wgzvx" Dec 03 11:51:09 crc kubenswrapper[5052]: I1203 11:51:09.297203 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c69gm\" (UniqueName: \"kubernetes.io/projected/775d2ccf-9b52-4274-99ef-2c33b18beaea-kube-api-access-c69gm\") pod \"keystone-94f46848f-wgzvx\" (UID: \"775d2ccf-9b52-4274-99ef-2c33b18beaea\") " pod="openstack-kuttl-tests/keystone-94f46848f-wgzvx" Dec 03 11:51:09 crc kubenswrapper[5052]: I1203 11:51:09.297295 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/775d2ccf-9b52-4274-99ef-2c33b18beaea-config-data\") pod \"keystone-94f46848f-wgzvx\" (UID: \"775d2ccf-9b52-4274-99ef-2c33b18beaea\") " pod="openstack-kuttl-tests/keystone-94f46848f-wgzvx" Dec 03 11:51:09 crc kubenswrapper[5052]: I1203 11:51:09.297424 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/775d2ccf-9b52-4274-99ef-2c33b18beaea-credential-keys\") pod \"keystone-94f46848f-wgzvx\" (UID: \"775d2ccf-9b52-4274-99ef-2c33b18beaea\") " pod="openstack-kuttl-tests/keystone-94f46848f-wgzvx" Dec 03 11:51:09 crc kubenswrapper[5052]: I1203 11:51:09.297495 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/775d2ccf-9b52-4274-99ef-2c33b18beaea-fernet-keys\") pod \"keystone-94f46848f-wgzvx\" (UID: \"775d2ccf-9b52-4274-99ef-2c33b18beaea\") " pod="openstack-kuttl-tests/keystone-94f46848f-wgzvx" Dec 03 11:51:09 crc kubenswrapper[5052]: I1203 11:51:09.297632 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/775d2ccf-9b52-4274-99ef-2c33b18beaea-combined-ca-bundle\") pod \"keystone-94f46848f-wgzvx\" (UID: \"775d2ccf-9b52-4274-99ef-2c33b18beaea\") " pod="openstack-kuttl-tests/keystone-94f46848f-wgzvx" Dec 03 11:51:09 crc kubenswrapper[5052]: I1203 11:51:09.297846 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/775d2ccf-9b52-4274-99ef-2c33b18beaea-public-tls-certs\") pod \"keystone-94f46848f-wgzvx\" (UID: \"775d2ccf-9b52-4274-99ef-2c33b18beaea\") " pod="openstack-kuttl-tests/keystone-94f46848f-wgzvx" Dec 03 11:51:09 crc kubenswrapper[5052]: I1203 11:51:09.399723 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/775d2ccf-9b52-4274-99ef-2c33b18beaea-config-data\") pod \"keystone-94f46848f-wgzvx\" (UID: \"775d2ccf-9b52-4274-99ef-2c33b18beaea\") " pod="openstack-kuttl-tests/keystone-94f46848f-wgzvx" Dec 03 11:51:09 crc kubenswrapper[5052]: I1203 11:51:09.399793 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/775d2ccf-9b52-4274-99ef-2c33b18beaea-credential-keys\") pod \"keystone-94f46848f-wgzvx\" (UID: \"775d2ccf-9b52-4274-99ef-2c33b18beaea\") " pod="openstack-kuttl-tests/keystone-94f46848f-wgzvx" Dec 03 11:51:09 crc kubenswrapper[5052]: I1203 11:51:09.399827 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/775d2ccf-9b52-4274-99ef-2c33b18beaea-fernet-keys\") pod \"keystone-94f46848f-wgzvx\" (UID: \"775d2ccf-9b52-4274-99ef-2c33b18beaea\") " pod="openstack-kuttl-tests/keystone-94f46848f-wgzvx" Dec 03 11:51:09 crc kubenswrapper[5052]: I1203 11:51:09.399863 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/775d2ccf-9b52-4274-99ef-2c33b18beaea-combined-ca-bundle\") pod \"keystone-94f46848f-wgzvx\" (UID: \"775d2ccf-9b52-4274-99ef-2c33b18beaea\") " pod="openstack-kuttl-tests/keystone-94f46848f-wgzvx" Dec 03 11:51:09 crc kubenswrapper[5052]: I1203 11:51:09.399903 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/775d2ccf-9b52-4274-99ef-2c33b18beaea-public-tls-certs\") pod \"keystone-94f46848f-wgzvx\" (UID: \"775d2ccf-9b52-4274-99ef-2c33b18beaea\") " pod="openstack-kuttl-tests/keystone-94f46848f-wgzvx" Dec 03 11:51:09 crc kubenswrapper[5052]: I1203 11:51:09.399936 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/775d2ccf-9b52-4274-99ef-2c33b18beaea-scripts\") pod \"keystone-94f46848f-wgzvx\" (UID: \"775d2ccf-9b52-4274-99ef-2c33b18beaea\") " pod="openstack-kuttl-tests/keystone-94f46848f-wgzvx" Dec 03 11:51:09 crc kubenswrapper[5052]: I1203 11:51:09.399961 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/775d2ccf-9b52-4274-99ef-2c33b18beaea-internal-tls-certs\") pod \"keystone-94f46848f-wgzvx\" (UID: \"775d2ccf-9b52-4274-99ef-2c33b18beaea\") " pod="openstack-kuttl-tests/keystone-94f46848f-wgzvx" Dec 03 11:51:09 crc kubenswrapper[5052]: I1203 11:51:09.399980 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-c69gm\" (UniqueName: \"kubernetes.io/projected/775d2ccf-9b52-4274-99ef-2c33b18beaea-kube-api-access-c69gm\") pod \"keystone-94f46848f-wgzvx\" (UID: \"775d2ccf-9b52-4274-99ef-2c33b18beaea\") " pod="openstack-kuttl-tests/keystone-94f46848f-wgzvx" Dec 03 11:51:09 crc kubenswrapper[5052]: I1203 11:51:09.405946 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/775d2ccf-9b52-4274-99ef-2c33b18beaea-fernet-keys\") pod \"keystone-94f46848f-wgzvx\" (UID: \"775d2ccf-9b52-4274-99ef-2c33b18beaea\") " pod="openstack-kuttl-tests/keystone-94f46848f-wgzvx" Dec 03 11:51:09 crc kubenswrapper[5052]: I1203 11:51:09.406051 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/775d2ccf-9b52-4274-99ef-2c33b18beaea-public-tls-certs\") pod \"keystone-94f46848f-wgzvx\" (UID: \"775d2ccf-9b52-4274-99ef-2c33b18beaea\") " pod="openstack-kuttl-tests/keystone-94f46848f-wgzvx" Dec 03 11:51:09 crc kubenswrapper[5052]: I1203 11:51:09.406463 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/775d2ccf-9b52-4274-99ef-2c33b18beaea-scripts\") pod \"keystone-94f46848f-wgzvx\" (UID: \"775d2ccf-9b52-4274-99ef-2c33b18beaea\") " pod="openstack-kuttl-tests/keystone-94f46848f-wgzvx" Dec 03 11:51:09 crc kubenswrapper[5052]: I1203 11:51:09.406752 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/775d2ccf-9b52-4274-99ef-2c33b18beaea-combined-ca-bundle\") pod \"keystone-94f46848f-wgzvx\" (UID: \"775d2ccf-9b52-4274-99ef-2c33b18beaea\") " pod="openstack-kuttl-tests/keystone-94f46848f-wgzvx" Dec 03 11:51:09 crc kubenswrapper[5052]: I1203 11:51:09.406786 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/775d2ccf-9b52-4274-99ef-2c33b18beaea-config-data\") pod \"keystone-94f46848f-wgzvx\" (UID: \"775d2ccf-9b52-4274-99ef-2c33b18beaea\") " pod="openstack-kuttl-tests/keystone-94f46848f-wgzvx" Dec 03 11:51:09 crc kubenswrapper[5052]: I1203 11:51:09.407236 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/775d2ccf-9b52-4274-99ef-2c33b18beaea-credential-keys\") pod \"keystone-94f46848f-wgzvx\" (UID: \"775d2ccf-9b52-4274-99ef-2c33b18beaea\") " pod="openstack-kuttl-tests/keystone-94f46848f-wgzvx" Dec 03 11:51:09 crc kubenswrapper[5052]: I1203 11:51:09.415978 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/775d2ccf-9b52-4274-99ef-2c33b18beaea-internal-tls-certs\") pod \"keystone-94f46848f-wgzvx\" (UID: \"775d2ccf-9b52-4274-99ef-2c33b18beaea\") " pod="openstack-kuttl-tests/keystone-94f46848f-wgzvx" Dec 03 11:51:09 crc kubenswrapper[5052]: I1203 11:51:09.424242 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-c69gm\" (UniqueName: \"kubernetes.io/projected/775d2ccf-9b52-4274-99ef-2c33b18beaea-kube-api-access-c69gm\") pod \"keystone-94f46848f-wgzvx\" (UID: \"775d2ccf-9b52-4274-99ef-2c33b18beaea\") " pod="openstack-kuttl-tests/keystone-94f46848f-wgzvx" Dec 03 11:51:09 crc kubenswrapper[5052]: I1203 11:51:09.525401 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-94f46848f-wgzvx" Dec 03 11:51:10 crc kubenswrapper[5052]: I1203 11:51:09.999990 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-94f46848f-wgzvx"] Dec 03 11:51:10 crc kubenswrapper[5052]: W1203 11:51:10.003335 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod775d2ccf_9b52_4274_99ef_2c33b18beaea.slice/crio-baba821dc78b24dbf946de1b85b6619340cdd58507e31c86dc21895cd0437171 WatchSource:0}: Error finding container baba821dc78b24dbf946de1b85b6619340cdd58507e31c86dc21895cd0437171: Status 404 returned error can't find the container with id baba821dc78b24dbf946de1b85b6619340cdd58507e31c86dc21895cd0437171 Dec 03 11:51:10 crc kubenswrapper[5052]: I1203 11:51:10.065663 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 11:51:10 crc kubenswrapper[5052]: I1203 11:51:10.112328 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-94f46848f-wgzvx" event={"ID":"775d2ccf-9b52-4274-99ef-2c33b18beaea","Type":"ContainerStarted","Data":"baba821dc78b24dbf946de1b85b6619340cdd58507e31c86dc21895cd0437171"} Dec 03 11:51:10 crc kubenswrapper[5052]: I1203 11:51:10.115920 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"a2fba22f-a015-40f4-8a93-e1011fe64e09","Type":"ContainerStarted","Data":"839a0c44b0dd45ea4509839490bafa1853a869568fc2d60a35de8f8da6c90fa4"} Dec 03 11:51:10 crc kubenswrapper[5052]: I1203 11:51:10.136930 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/cinder-api-0" podStartSLOduration=4.136909179 podStartE2EDuration="4.136909179s" podCreationTimestamp="2025-12-03 11:51:06 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:51:10.131273038 +0000 UTC m=+1860.719514812" watchObservedRunningTime="2025-12-03 11:51:10.136909179 +0000 UTC m=+1860.725150943" Dec 03 11:51:11 crc kubenswrapper[5052]: I1203 11:51:11.127757 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-94f46848f-wgzvx" event={"ID":"775d2ccf-9b52-4274-99ef-2c33b18beaea","Type":"ContainerStarted","Data":"92f19f59b5c7604cf76509c11bcfe787a7d812a120d0641a971265397e22c941"} Dec 03 11:51:11 crc kubenswrapper[5052]: I1203 11:51:11.128004 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/cinder-api-0" podUID="a2fba22f-a015-40f4-8a93-e1011fe64e09" containerName="cinder-api" containerID="cri-o://839a0c44b0dd45ea4509839490bafa1853a869568fc2d60a35de8f8da6c90fa4" gracePeriod=30 Dec 03 11:51:11 crc kubenswrapper[5052]: I1203 11:51:11.127891 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/cinder-api-0" podUID="a2fba22f-a015-40f4-8a93-e1011fe64e09" containerName="cinder-api-log" containerID="cri-o://619259e4397089b6aa65dec5b558419311effe8a5ca7087e33cd64a4922abb5f" gracePeriod=30 Dec 03 11:51:11 crc kubenswrapper[5052]: I1203 11:51:11.129308 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:51:11 crc kubenswrapper[5052]: I1203 11:51:11.160143 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/keystone-94f46848f-wgzvx" podStartSLOduration=2.160125355 podStartE2EDuration="2.160125355s" podCreationTimestamp="2025-12-03 11:51:09 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:51:11.155971646 +0000 UTC m=+1861.744213410" watchObservedRunningTime="2025-12-03 11:51:11.160125355 +0000 UTC m=+1861.748367119" Dec 03 11:51:11 crc kubenswrapper[5052]: I1203 11:51:11.589589 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:51:11 crc kubenswrapper[5052]: I1203 11:51:11.607974 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/barbican-api-5758478fc6-5g46v"] Dec 03 11:51:11 crc kubenswrapper[5052]: I1203 11:51:11.609940 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-api-5758478fc6-5g46v" Dec 03 11:51:11 crc kubenswrapper[5052]: I1203 11:51:11.614734 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-barbican-internal-svc" Dec 03 11:51:11 crc kubenswrapper[5052]: I1203 11:51:11.615070 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-barbican-public-svc" Dec 03 11:51:11 crc kubenswrapper[5052]: I1203 11:51:11.629885 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-api-5758478fc6-5g46v"] Dec 03 11:51:11 crc kubenswrapper[5052]: I1203 11:51:11.642026 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a18602ce-7206-4edc-a8d3-5d0754c3da02-combined-ca-bundle\") pod \"barbican-api-5758478fc6-5g46v\" (UID: \"a18602ce-7206-4edc-a8d3-5d0754c3da02\") " pod="openstack-kuttl-tests/barbican-api-5758478fc6-5g46v" Dec 03 11:51:11 crc kubenswrapper[5052]: I1203 11:51:11.642092 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a18602ce-7206-4edc-a8d3-5d0754c3da02-config-data\") pod \"barbican-api-5758478fc6-5g46v\" (UID: \"a18602ce-7206-4edc-a8d3-5d0754c3da02\") " pod="openstack-kuttl-tests/barbican-api-5758478fc6-5g46v" Dec 03 11:51:11 crc kubenswrapper[5052]: I1203 11:51:11.642185 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fkmkd\" (UniqueName: \"kubernetes.io/projected/a18602ce-7206-4edc-a8d3-5d0754c3da02-kube-api-access-fkmkd\") pod \"barbican-api-5758478fc6-5g46v\" (UID: \"a18602ce-7206-4edc-a8d3-5d0754c3da02\") " pod="openstack-kuttl-tests/barbican-api-5758478fc6-5g46v" Dec 03 11:51:11 crc kubenswrapper[5052]: I1203 11:51:11.642371 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/a18602ce-7206-4edc-a8d3-5d0754c3da02-public-tls-certs\") pod \"barbican-api-5758478fc6-5g46v\" (UID: \"a18602ce-7206-4edc-a8d3-5d0754c3da02\") " pod="openstack-kuttl-tests/barbican-api-5758478fc6-5g46v" Dec 03 11:51:11 crc kubenswrapper[5052]: I1203 11:51:11.642424 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/a18602ce-7206-4edc-a8d3-5d0754c3da02-internal-tls-certs\") pod \"barbican-api-5758478fc6-5g46v\" (UID: \"a18602ce-7206-4edc-a8d3-5d0754c3da02\") " pod="openstack-kuttl-tests/barbican-api-5758478fc6-5g46v" Dec 03 11:51:11 crc kubenswrapper[5052]: I1203 11:51:11.642453 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/a18602ce-7206-4edc-a8d3-5d0754c3da02-logs\") pod \"barbican-api-5758478fc6-5g46v\" (UID: \"a18602ce-7206-4edc-a8d3-5d0754c3da02\") " pod="openstack-kuttl-tests/barbican-api-5758478fc6-5g46v" Dec 03 11:51:11 crc kubenswrapper[5052]: I1203 11:51:11.642586 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/a18602ce-7206-4edc-a8d3-5d0754c3da02-config-data-custom\") pod \"barbican-api-5758478fc6-5g46v\" (UID: \"a18602ce-7206-4edc-a8d3-5d0754c3da02\") " pod="openstack-kuttl-tests/barbican-api-5758478fc6-5g46v" Dec 03 11:51:11 crc kubenswrapper[5052]: I1203 11:51:11.744470 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/a18602ce-7206-4edc-a8d3-5d0754c3da02-config-data-custom\") pod \"barbican-api-5758478fc6-5g46v\" (UID: \"a18602ce-7206-4edc-a8d3-5d0754c3da02\") " pod="openstack-kuttl-tests/barbican-api-5758478fc6-5g46v" Dec 03 11:51:11 crc kubenswrapper[5052]: I1203 11:51:11.744597 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a18602ce-7206-4edc-a8d3-5d0754c3da02-combined-ca-bundle\") pod \"barbican-api-5758478fc6-5g46v\" (UID: \"a18602ce-7206-4edc-a8d3-5d0754c3da02\") " pod="openstack-kuttl-tests/barbican-api-5758478fc6-5g46v" Dec 03 11:51:11 crc kubenswrapper[5052]: I1203 11:51:11.744618 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a18602ce-7206-4edc-a8d3-5d0754c3da02-config-data\") pod \"barbican-api-5758478fc6-5g46v\" (UID: \"a18602ce-7206-4edc-a8d3-5d0754c3da02\") " pod="openstack-kuttl-tests/barbican-api-5758478fc6-5g46v" Dec 03 11:51:11 crc kubenswrapper[5052]: I1203 11:51:11.744651 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-fkmkd\" (UniqueName: \"kubernetes.io/projected/a18602ce-7206-4edc-a8d3-5d0754c3da02-kube-api-access-fkmkd\") pod \"barbican-api-5758478fc6-5g46v\" (UID: \"a18602ce-7206-4edc-a8d3-5d0754c3da02\") " pod="openstack-kuttl-tests/barbican-api-5758478fc6-5g46v" Dec 03 11:51:11 crc kubenswrapper[5052]: I1203 11:51:11.744735 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/a18602ce-7206-4edc-a8d3-5d0754c3da02-public-tls-certs\") pod \"barbican-api-5758478fc6-5g46v\" (UID: \"a18602ce-7206-4edc-a8d3-5d0754c3da02\") " pod="openstack-kuttl-tests/barbican-api-5758478fc6-5g46v" Dec 03 11:51:11 crc kubenswrapper[5052]: I1203 11:51:11.744755 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/a18602ce-7206-4edc-a8d3-5d0754c3da02-internal-tls-certs\") pod \"barbican-api-5758478fc6-5g46v\" (UID: \"a18602ce-7206-4edc-a8d3-5d0754c3da02\") " pod="openstack-kuttl-tests/barbican-api-5758478fc6-5g46v" Dec 03 11:51:11 crc kubenswrapper[5052]: I1203 11:51:11.744775 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/a18602ce-7206-4edc-a8d3-5d0754c3da02-logs\") pod \"barbican-api-5758478fc6-5g46v\" (UID: \"a18602ce-7206-4edc-a8d3-5d0754c3da02\") " pod="openstack-kuttl-tests/barbican-api-5758478fc6-5g46v" Dec 03 11:51:11 crc kubenswrapper[5052]: I1203 11:51:11.745700 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/a18602ce-7206-4edc-a8d3-5d0754c3da02-logs\") pod \"barbican-api-5758478fc6-5g46v\" (UID: \"a18602ce-7206-4edc-a8d3-5d0754c3da02\") " pod="openstack-kuttl-tests/barbican-api-5758478fc6-5g46v" Dec 03 11:51:11 crc kubenswrapper[5052]: I1203 11:51:11.750502 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/a18602ce-7206-4edc-a8d3-5d0754c3da02-public-tls-certs\") pod \"barbican-api-5758478fc6-5g46v\" (UID: \"a18602ce-7206-4edc-a8d3-5d0754c3da02\") " pod="openstack-kuttl-tests/barbican-api-5758478fc6-5g46v" Dec 03 11:51:11 crc kubenswrapper[5052]: I1203 11:51:11.751065 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/a18602ce-7206-4edc-a8d3-5d0754c3da02-config-data-custom\") pod \"barbican-api-5758478fc6-5g46v\" (UID: \"a18602ce-7206-4edc-a8d3-5d0754c3da02\") " pod="openstack-kuttl-tests/barbican-api-5758478fc6-5g46v" Dec 03 11:51:11 crc kubenswrapper[5052]: I1203 11:51:11.751429 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a18602ce-7206-4edc-a8d3-5d0754c3da02-config-data\") pod \"barbican-api-5758478fc6-5g46v\" (UID: \"a18602ce-7206-4edc-a8d3-5d0754c3da02\") " pod="openstack-kuttl-tests/barbican-api-5758478fc6-5g46v" Dec 03 11:51:11 crc kubenswrapper[5052]: I1203 11:51:11.753145 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/a18602ce-7206-4edc-a8d3-5d0754c3da02-internal-tls-certs\") pod \"barbican-api-5758478fc6-5g46v\" (UID: \"a18602ce-7206-4edc-a8d3-5d0754c3da02\") " pod="openstack-kuttl-tests/barbican-api-5758478fc6-5g46v" Dec 03 11:51:11 crc kubenswrapper[5052]: I1203 11:51:11.753992 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a18602ce-7206-4edc-a8d3-5d0754c3da02-combined-ca-bundle\") pod \"barbican-api-5758478fc6-5g46v\" (UID: \"a18602ce-7206-4edc-a8d3-5d0754c3da02\") " pod="openstack-kuttl-tests/barbican-api-5758478fc6-5g46v" Dec 03 11:51:11 crc kubenswrapper[5052]: I1203 11:51:11.767309 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-fkmkd\" (UniqueName: \"kubernetes.io/projected/a18602ce-7206-4edc-a8d3-5d0754c3da02-kube-api-access-fkmkd\") pod \"barbican-api-5758478fc6-5g46v\" (UID: \"a18602ce-7206-4edc-a8d3-5d0754c3da02\") " pod="openstack-kuttl-tests/barbican-api-5758478fc6-5g46v" Dec 03 11:51:11 crc kubenswrapper[5052]: I1203 11:51:11.939105 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-api-5758478fc6-5g46v" Dec 03 11:51:12 crc kubenswrapper[5052]: I1203 11:51:12.148074 5052 generic.go:334] "Generic (PLEG): container finished" podID="a2fba22f-a015-40f4-8a93-e1011fe64e09" containerID="839a0c44b0dd45ea4509839490bafa1853a869568fc2d60a35de8f8da6c90fa4" exitCode=0 Dec 03 11:51:12 crc kubenswrapper[5052]: I1203 11:51:12.148444 5052 generic.go:334] "Generic (PLEG): container finished" podID="a2fba22f-a015-40f4-8a93-e1011fe64e09" containerID="619259e4397089b6aa65dec5b558419311effe8a5ca7087e33cd64a4922abb5f" exitCode=143 Dec 03 11:51:12 crc kubenswrapper[5052]: I1203 11:51:12.148250 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"a2fba22f-a015-40f4-8a93-e1011fe64e09","Type":"ContainerDied","Data":"839a0c44b0dd45ea4509839490bafa1853a869568fc2d60a35de8f8da6c90fa4"} Dec 03 11:51:12 crc kubenswrapper[5052]: I1203 11:51:12.148493 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"a2fba22f-a015-40f4-8a93-e1011fe64e09","Type":"ContainerDied","Data":"619259e4397089b6aa65dec5b558419311effe8a5ca7087e33cd64a4922abb5f"} Dec 03 11:51:12 crc kubenswrapper[5052]: I1203 11:51:12.149647 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/keystone-94f46848f-wgzvx" Dec 03 11:51:12 crc kubenswrapper[5052]: I1203 11:51:12.272962 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:51:12 crc kubenswrapper[5052]: I1203 11:51:12.361774 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zm95z\" (UniqueName: \"kubernetes.io/projected/a2fba22f-a015-40f4-8a93-e1011fe64e09-kube-api-access-zm95z\") pod \"a2fba22f-a015-40f4-8a93-e1011fe64e09\" (UID: \"a2fba22f-a015-40f4-8a93-e1011fe64e09\") " Dec 03 11:51:12 crc kubenswrapper[5052]: I1203 11:51:12.361904 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a2fba22f-a015-40f4-8a93-e1011fe64e09-config-data\") pod \"a2fba22f-a015-40f4-8a93-e1011fe64e09\" (UID: \"a2fba22f-a015-40f4-8a93-e1011fe64e09\") " Dec 03 11:51:12 crc kubenswrapper[5052]: I1203 11:51:12.361927 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/a2fba22f-a015-40f4-8a93-e1011fe64e09-config-data-custom\") pod \"a2fba22f-a015-40f4-8a93-e1011fe64e09\" (UID: \"a2fba22f-a015-40f4-8a93-e1011fe64e09\") " Dec 03 11:51:12 crc kubenswrapper[5052]: I1203 11:51:12.361947 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a2fba22f-a015-40f4-8a93-e1011fe64e09-combined-ca-bundle\") pod \"a2fba22f-a015-40f4-8a93-e1011fe64e09\" (UID: \"a2fba22f-a015-40f4-8a93-e1011fe64e09\") " Dec 03 11:51:12 crc kubenswrapper[5052]: I1203 11:51:12.362007 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/a2fba22f-a015-40f4-8a93-e1011fe64e09-logs\") pod \"a2fba22f-a015-40f4-8a93-e1011fe64e09\" (UID: \"a2fba22f-a015-40f4-8a93-e1011fe64e09\") " Dec 03 11:51:12 crc kubenswrapper[5052]: I1203 11:51:12.362047 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/a2fba22f-a015-40f4-8a93-e1011fe64e09-scripts\") pod \"a2fba22f-a015-40f4-8a93-e1011fe64e09\" (UID: \"a2fba22f-a015-40f4-8a93-e1011fe64e09\") " Dec 03 11:51:12 crc kubenswrapper[5052]: I1203 11:51:12.362095 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/a2fba22f-a015-40f4-8a93-e1011fe64e09-etc-machine-id\") pod \"a2fba22f-a015-40f4-8a93-e1011fe64e09\" (UID: \"a2fba22f-a015-40f4-8a93-e1011fe64e09\") " Dec 03 11:51:12 crc kubenswrapper[5052]: I1203 11:51:12.362599 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/a2fba22f-a015-40f4-8a93-e1011fe64e09-etc-machine-id" (OuterVolumeSpecName: "etc-machine-id") pod "a2fba22f-a015-40f4-8a93-e1011fe64e09" (UID: "a2fba22f-a015-40f4-8a93-e1011fe64e09"). InnerVolumeSpecName "etc-machine-id". PluginName "kubernetes.io/host-path", VolumeGidValue "" Dec 03 11:51:12 crc kubenswrapper[5052]: I1203 11:51:12.364164 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2fba22f-a015-40f4-8a93-e1011fe64e09-logs" (OuterVolumeSpecName: "logs") pod "a2fba22f-a015-40f4-8a93-e1011fe64e09" (UID: "a2fba22f-a015-40f4-8a93-e1011fe64e09"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:51:12 crc kubenswrapper[5052]: I1203 11:51:12.367678 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a2fba22f-a015-40f4-8a93-e1011fe64e09-kube-api-access-zm95z" (OuterVolumeSpecName: "kube-api-access-zm95z") pod "a2fba22f-a015-40f4-8a93-e1011fe64e09" (UID: "a2fba22f-a015-40f4-8a93-e1011fe64e09"). InnerVolumeSpecName "kube-api-access-zm95z". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:51:12 crc kubenswrapper[5052]: I1203 11:51:12.367852 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a2fba22f-a015-40f4-8a93-e1011fe64e09-scripts" (OuterVolumeSpecName: "scripts") pod "a2fba22f-a015-40f4-8a93-e1011fe64e09" (UID: "a2fba22f-a015-40f4-8a93-e1011fe64e09"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:51:12 crc kubenswrapper[5052]: I1203 11:51:12.369568 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a2fba22f-a015-40f4-8a93-e1011fe64e09-config-data-custom" (OuterVolumeSpecName: "config-data-custom") pod "a2fba22f-a015-40f4-8a93-e1011fe64e09" (UID: "a2fba22f-a015-40f4-8a93-e1011fe64e09"). InnerVolumeSpecName "config-data-custom". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:51:12 crc kubenswrapper[5052]: I1203 11:51:12.408048 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a2fba22f-a015-40f4-8a93-e1011fe64e09-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "a2fba22f-a015-40f4-8a93-e1011fe64e09" (UID: "a2fba22f-a015-40f4-8a93-e1011fe64e09"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:51:12 crc kubenswrapper[5052]: I1203 11:51:12.416254 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a2fba22f-a015-40f4-8a93-e1011fe64e09-config-data" (OuterVolumeSpecName: "config-data") pod "a2fba22f-a015-40f4-8a93-e1011fe64e09" (UID: "a2fba22f-a015-40f4-8a93-e1011fe64e09"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:51:12 crc kubenswrapper[5052]: I1203 11:51:12.468773 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a2fba22f-a015-40f4-8a93-e1011fe64e09-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:51:12 crc kubenswrapper[5052]: I1203 11:51:12.468814 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/a2fba22f-a015-40f4-8a93-e1011fe64e09-config-data-custom\") on node \"crc\" DevicePath \"\"" Dec 03 11:51:12 crc kubenswrapper[5052]: I1203 11:51:12.468829 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a2fba22f-a015-40f4-8a93-e1011fe64e09-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:51:12 crc kubenswrapper[5052]: I1203 11:51:12.468841 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/a2fba22f-a015-40f4-8a93-e1011fe64e09-logs\") on node \"crc\" DevicePath \"\"" Dec 03 11:51:12 crc kubenswrapper[5052]: I1203 11:51:12.468852 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/a2fba22f-a015-40f4-8a93-e1011fe64e09-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:51:12 crc kubenswrapper[5052]: I1203 11:51:12.468864 5052 reconciler_common.go:293] "Volume detached for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/a2fba22f-a015-40f4-8a93-e1011fe64e09-etc-machine-id\") on node \"crc\" DevicePath \"\"" Dec 03 11:51:12 crc kubenswrapper[5052]: I1203 11:51:12.468875 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-zm95z\" (UniqueName: \"kubernetes.io/projected/a2fba22f-a015-40f4-8a93-e1011fe64e09-kube-api-access-zm95z\") on node \"crc\" DevicePath \"\"" Dec 03 11:51:12 crc kubenswrapper[5052]: I1203 11:51:12.471842 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-api-5758478fc6-5g46v"] Dec 03 11:51:12 crc kubenswrapper[5052]: W1203 11:51:12.473313 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda18602ce_7206_4edc_a8d3_5d0754c3da02.slice/crio-cc433ae07b2f7176b8b5d5186d1280a020187109a3af8495da69733e3bc0551d WatchSource:0}: Error finding container cc433ae07b2f7176b8b5d5186d1280a020187109a3af8495da69733e3bc0551d: Status 404 returned error can't find the container with id cc433ae07b2f7176b8b5d5186d1280a020187109a3af8495da69733e3bc0551d Dec 03 11:51:12 crc kubenswrapper[5052]: I1203 11:51:12.672867 5052 scope.go:117] "RemoveContainer" containerID="25fe88f8d102f09983f8884e65f0c6c1575ade67921f10a630e6dcc3ea8b506e" Dec 03 11:51:13 crc kubenswrapper[5052]: I1203 11:51:13.161309 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" event={"ID":"10c237bc-72e7-4da3-9f2c-b1688395bf6e","Type":"ContainerStarted","Data":"cc4f296896bbf8aa4d2c78b7e4045d3222cac9b2a6ed232c44ec9d7458bb3338"} Dec 03 11:51:13 crc kubenswrapper[5052]: I1203 11:51:13.163395 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-5758478fc6-5g46v" event={"ID":"a18602ce-7206-4edc-a8d3-5d0754c3da02","Type":"ContainerStarted","Data":"9684094fee34437582198b41d3a72ef6d576587c1466aadd379f8be1cedcf6af"} Dec 03 11:51:13 crc kubenswrapper[5052]: I1203 11:51:13.163434 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-5758478fc6-5g46v" event={"ID":"a18602ce-7206-4edc-a8d3-5d0754c3da02","Type":"ContainerStarted","Data":"b8bffbbe67b941eebfcc7d809311107fe0d16c3fd77f96571d99325342cabd72"} Dec 03 11:51:13 crc kubenswrapper[5052]: I1203 11:51:13.163448 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-5758478fc6-5g46v" event={"ID":"a18602ce-7206-4edc-a8d3-5d0754c3da02","Type":"ContainerStarted","Data":"cc433ae07b2f7176b8b5d5186d1280a020187109a3af8495da69733e3bc0551d"} Dec 03 11:51:13 crc kubenswrapper[5052]: I1203 11:51:13.163540 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/barbican-api-5758478fc6-5g46v" Dec 03 11:51:13 crc kubenswrapper[5052]: I1203 11:51:13.163557 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/barbican-api-5758478fc6-5g46v" Dec 03 11:51:13 crc kubenswrapper[5052]: I1203 11:51:13.166521 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:51:13 crc kubenswrapper[5052]: I1203 11:51:13.166835 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"a2fba22f-a015-40f4-8a93-e1011fe64e09","Type":"ContainerDied","Data":"748b422e8a05a370a0584564fb275ccb549575f174c156d7ba55a98bf54cfb5c"} Dec 03 11:51:13 crc kubenswrapper[5052]: I1203 11:51:13.166880 5052 scope.go:117] "RemoveContainer" containerID="839a0c44b0dd45ea4509839490bafa1853a869568fc2d60a35de8f8da6c90fa4" Dec 03 11:51:13 crc kubenswrapper[5052]: I1203 11:51:13.231652 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/barbican-api-5758478fc6-5g46v" podStartSLOduration=2.231627505 podStartE2EDuration="2.231627505s" podCreationTimestamp="2025-12-03 11:51:11 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:51:13.221388322 +0000 UTC m=+1863.809630096" watchObservedRunningTime="2025-12-03 11:51:13.231627505 +0000 UTC m=+1863.819869269" Dec 03 11:51:13 crc kubenswrapper[5052]: I1203 11:51:13.233841 5052 scope.go:117] "RemoveContainer" containerID="619259e4397089b6aa65dec5b558419311effe8a5ca7087e33cd64a4922abb5f" Dec 03 11:51:13 crc kubenswrapper[5052]: I1203 11:51:13.260905 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 11:51:13 crc kubenswrapper[5052]: I1203 11:51:13.303248 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 11:51:13 crc kubenswrapper[5052]: I1203 11:51:13.325683 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 11:51:13 crc kubenswrapper[5052]: E1203 11:51:13.326450 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="a2fba22f-a015-40f4-8a93-e1011fe64e09" containerName="cinder-api" Dec 03 11:51:13 crc kubenswrapper[5052]: I1203 11:51:13.326560 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="a2fba22f-a015-40f4-8a93-e1011fe64e09" containerName="cinder-api" Dec 03 11:51:13 crc kubenswrapper[5052]: E1203 11:51:13.326628 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="a2fba22f-a015-40f4-8a93-e1011fe64e09" containerName="cinder-api-log" Dec 03 11:51:13 crc kubenswrapper[5052]: I1203 11:51:13.326672 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="a2fba22f-a015-40f4-8a93-e1011fe64e09" containerName="cinder-api-log" Dec 03 11:51:13 crc kubenswrapper[5052]: I1203 11:51:13.326898 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="a2fba22f-a015-40f4-8a93-e1011fe64e09" containerName="cinder-api-log" Dec 03 11:51:13 crc kubenswrapper[5052]: I1203 11:51:13.326965 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="a2fba22f-a015-40f4-8a93-e1011fe64e09" containerName="cinder-api" Dec 03 11:51:13 crc kubenswrapper[5052]: I1203 11:51:13.327957 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:51:13 crc kubenswrapper[5052]: I1203 11:51:13.330833 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-cinder-internal-svc" Dec 03 11:51:13 crc kubenswrapper[5052]: I1203 11:51:13.330853 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-cinder-public-svc" Dec 03 11:51:13 crc kubenswrapper[5052]: I1203 11:51:13.332382 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cinder-api-config-data" Dec 03 11:51:13 crc kubenswrapper[5052]: I1203 11:51:13.335667 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 11:51:13 crc kubenswrapper[5052]: I1203 11:51:13.415985 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/9a4b1245-d95c-4eb6-8d2b-fdd93d98bb1e-scripts\") pod \"cinder-api-0\" (UID: \"9a4b1245-d95c-4eb6-8d2b-fdd93d98bb1e\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:51:13 crc kubenswrapper[5052]: I1203 11:51:13.416316 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/9a4b1245-d95c-4eb6-8d2b-fdd93d98bb1e-public-tls-certs\") pod \"cinder-api-0\" (UID: \"9a4b1245-d95c-4eb6-8d2b-fdd93d98bb1e\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:51:13 crc kubenswrapper[5052]: I1203 11:51:13.416410 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/9a4b1245-d95c-4eb6-8d2b-fdd93d98bb1e-logs\") pod \"cinder-api-0\" (UID: \"9a4b1245-d95c-4eb6-8d2b-fdd93d98bb1e\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:51:13 crc kubenswrapper[5052]: I1203 11:51:13.416618 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/9a4b1245-d95c-4eb6-8d2b-fdd93d98bb1e-etc-machine-id\") pod \"cinder-api-0\" (UID: \"9a4b1245-d95c-4eb6-8d2b-fdd93d98bb1e\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:51:13 crc kubenswrapper[5052]: I1203 11:51:13.416719 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/9a4b1245-d95c-4eb6-8d2b-fdd93d98bb1e-config-data-custom\") pod \"cinder-api-0\" (UID: \"9a4b1245-d95c-4eb6-8d2b-fdd93d98bb1e\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:51:13 crc kubenswrapper[5052]: I1203 11:51:13.416871 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9a4b1245-d95c-4eb6-8d2b-fdd93d98bb1e-combined-ca-bundle\") pod \"cinder-api-0\" (UID: \"9a4b1245-d95c-4eb6-8d2b-fdd93d98bb1e\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:51:13 crc kubenswrapper[5052]: I1203 11:51:13.417013 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/9a4b1245-d95c-4eb6-8d2b-fdd93d98bb1e-config-data\") pod \"cinder-api-0\" (UID: \"9a4b1245-d95c-4eb6-8d2b-fdd93d98bb1e\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:51:13 crc kubenswrapper[5052]: I1203 11:51:13.417116 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8k5pl\" (UniqueName: \"kubernetes.io/projected/9a4b1245-d95c-4eb6-8d2b-fdd93d98bb1e-kube-api-access-8k5pl\") pod \"cinder-api-0\" (UID: \"9a4b1245-d95c-4eb6-8d2b-fdd93d98bb1e\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:51:13 crc kubenswrapper[5052]: I1203 11:51:13.417214 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/9a4b1245-d95c-4eb6-8d2b-fdd93d98bb1e-internal-tls-certs\") pod \"cinder-api-0\" (UID: \"9a4b1245-d95c-4eb6-8d2b-fdd93d98bb1e\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:51:13 crc kubenswrapper[5052]: I1203 11:51:13.518650 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/9a4b1245-d95c-4eb6-8d2b-fdd93d98bb1e-config-data\") pod \"cinder-api-0\" (UID: \"9a4b1245-d95c-4eb6-8d2b-fdd93d98bb1e\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:51:13 crc kubenswrapper[5052]: I1203 11:51:13.518696 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-8k5pl\" (UniqueName: \"kubernetes.io/projected/9a4b1245-d95c-4eb6-8d2b-fdd93d98bb1e-kube-api-access-8k5pl\") pod \"cinder-api-0\" (UID: \"9a4b1245-d95c-4eb6-8d2b-fdd93d98bb1e\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:51:13 crc kubenswrapper[5052]: I1203 11:51:13.518733 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/9a4b1245-d95c-4eb6-8d2b-fdd93d98bb1e-internal-tls-certs\") pod \"cinder-api-0\" (UID: \"9a4b1245-d95c-4eb6-8d2b-fdd93d98bb1e\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:51:13 crc kubenswrapper[5052]: I1203 11:51:13.518795 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/9a4b1245-d95c-4eb6-8d2b-fdd93d98bb1e-scripts\") pod \"cinder-api-0\" (UID: \"9a4b1245-d95c-4eb6-8d2b-fdd93d98bb1e\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:51:13 crc kubenswrapper[5052]: I1203 11:51:13.518831 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/9a4b1245-d95c-4eb6-8d2b-fdd93d98bb1e-public-tls-certs\") pod \"cinder-api-0\" (UID: \"9a4b1245-d95c-4eb6-8d2b-fdd93d98bb1e\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:51:13 crc kubenswrapper[5052]: I1203 11:51:13.518852 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/9a4b1245-d95c-4eb6-8d2b-fdd93d98bb1e-logs\") pod \"cinder-api-0\" (UID: \"9a4b1245-d95c-4eb6-8d2b-fdd93d98bb1e\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:51:13 crc kubenswrapper[5052]: I1203 11:51:13.518871 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/9a4b1245-d95c-4eb6-8d2b-fdd93d98bb1e-etc-machine-id\") pod \"cinder-api-0\" (UID: \"9a4b1245-d95c-4eb6-8d2b-fdd93d98bb1e\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:51:13 crc kubenswrapper[5052]: I1203 11:51:13.518897 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/9a4b1245-d95c-4eb6-8d2b-fdd93d98bb1e-config-data-custom\") pod \"cinder-api-0\" (UID: \"9a4b1245-d95c-4eb6-8d2b-fdd93d98bb1e\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:51:13 crc kubenswrapper[5052]: I1203 11:51:13.518966 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9a4b1245-d95c-4eb6-8d2b-fdd93d98bb1e-combined-ca-bundle\") pod \"cinder-api-0\" (UID: \"9a4b1245-d95c-4eb6-8d2b-fdd93d98bb1e\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:51:13 crc kubenswrapper[5052]: I1203 11:51:13.520164 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/9a4b1245-d95c-4eb6-8d2b-fdd93d98bb1e-etc-machine-id\") pod \"cinder-api-0\" (UID: \"9a4b1245-d95c-4eb6-8d2b-fdd93d98bb1e\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:51:13 crc kubenswrapper[5052]: I1203 11:51:13.520900 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/9a4b1245-d95c-4eb6-8d2b-fdd93d98bb1e-logs\") pod \"cinder-api-0\" (UID: \"9a4b1245-d95c-4eb6-8d2b-fdd93d98bb1e\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:51:13 crc kubenswrapper[5052]: I1203 11:51:13.525060 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/9a4b1245-d95c-4eb6-8d2b-fdd93d98bb1e-config-data-custom\") pod \"cinder-api-0\" (UID: \"9a4b1245-d95c-4eb6-8d2b-fdd93d98bb1e\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:51:13 crc kubenswrapper[5052]: I1203 11:51:13.526106 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/9a4b1245-d95c-4eb6-8d2b-fdd93d98bb1e-scripts\") pod \"cinder-api-0\" (UID: \"9a4b1245-d95c-4eb6-8d2b-fdd93d98bb1e\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:51:13 crc kubenswrapper[5052]: I1203 11:51:13.526589 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9a4b1245-d95c-4eb6-8d2b-fdd93d98bb1e-combined-ca-bundle\") pod \"cinder-api-0\" (UID: \"9a4b1245-d95c-4eb6-8d2b-fdd93d98bb1e\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:51:13 crc kubenswrapper[5052]: I1203 11:51:13.528141 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/9a4b1245-d95c-4eb6-8d2b-fdd93d98bb1e-public-tls-certs\") pod \"cinder-api-0\" (UID: \"9a4b1245-d95c-4eb6-8d2b-fdd93d98bb1e\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:51:13 crc kubenswrapper[5052]: I1203 11:51:13.534363 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/9a4b1245-d95c-4eb6-8d2b-fdd93d98bb1e-config-data\") pod \"cinder-api-0\" (UID: \"9a4b1245-d95c-4eb6-8d2b-fdd93d98bb1e\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:51:13 crc kubenswrapper[5052]: I1203 11:51:13.540660 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-8k5pl\" (UniqueName: \"kubernetes.io/projected/9a4b1245-d95c-4eb6-8d2b-fdd93d98bb1e-kube-api-access-8k5pl\") pod \"cinder-api-0\" (UID: \"9a4b1245-d95c-4eb6-8d2b-fdd93d98bb1e\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:51:13 crc kubenswrapper[5052]: I1203 11:51:13.545266 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/9a4b1245-d95c-4eb6-8d2b-fdd93d98bb1e-internal-tls-certs\") pod \"cinder-api-0\" (UID: \"9a4b1245-d95c-4eb6-8d2b-fdd93d98bb1e\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:51:13 crc kubenswrapper[5052]: I1203 11:51:13.684267 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a2fba22f-a015-40f4-8a93-e1011fe64e09" path="/var/lib/kubelet/pods/a2fba22f-a015-40f4-8a93-e1011fe64e09/volumes" Dec 03 11:51:13 crc kubenswrapper[5052]: I1203 11:51:13.693297 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:51:14 crc kubenswrapper[5052]: W1203 11:51:14.104921 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9a4b1245_d95c_4eb6_8d2b_fdd93d98bb1e.slice/crio-6164f84f9476faf7069883e2525db6c95bc01479dc7ece67cdb31494c9c1c048 WatchSource:0}: Error finding container 6164f84f9476faf7069883e2525db6c95bc01479dc7ece67cdb31494c9c1c048: Status 404 returned error can't find the container with id 6164f84f9476faf7069883e2525db6c95bc01479dc7ece67cdb31494c9c1c048 Dec 03 11:51:14 crc kubenswrapper[5052]: I1203 11:51:14.106712 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 11:51:14 crc kubenswrapper[5052]: I1203 11:51:14.182480 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"9a4b1245-d95c-4eb6-8d2b-fdd93d98bb1e","Type":"ContainerStarted","Data":"6164f84f9476faf7069883e2525db6c95bc01479dc7ece67cdb31494c9c1c048"} Dec 03 11:51:15 crc kubenswrapper[5052]: I1203 11:51:15.204553 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"9a4b1245-d95c-4eb6-8d2b-fdd93d98bb1e","Type":"ContainerStarted","Data":"8c286da6c5ac86a7bafdea57cd407cfa1b12fb35b2f4dd6c013af4a83aa8fbb3"} Dec 03 11:51:16 crc kubenswrapper[5052]: I1203 11:51:16.216500 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"9a4b1245-d95c-4eb6-8d2b-fdd93d98bb1e","Type":"ContainerStarted","Data":"b4835f9bc39d3950a321af04adbfa74294d65285b1d19075ec93d354b42417ef"} Dec 03 11:51:16 crc kubenswrapper[5052]: I1203 11:51:16.217200 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:51:16 crc kubenswrapper[5052]: I1203 11:51:16.840252 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:51:16 crc kubenswrapper[5052]: I1203 11:51:16.866302 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/cinder-api-0" podStartSLOduration=3.866279686 podStartE2EDuration="3.866279686s" podCreationTimestamp="2025-12-03 11:51:13 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:51:16.264338963 +0000 UTC m=+1866.852580727" watchObservedRunningTime="2025-12-03 11:51:16.866279686 +0000 UTC m=+1867.454521450" Dec 03 11:51:16 crc kubenswrapper[5052]: I1203 11:51:16.881964 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 11:51:17 crc kubenswrapper[5052]: I1203 11:51:17.228745 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/cinder-scheduler-0" podUID="61d6dd4f-56ef-4f1d-8536-ea66f56188ce" containerName="cinder-scheduler" containerID="cri-o://88bb12213446d0cc45586efb2f6ce025322fc258ef04095763e5dd0212b596c5" gracePeriod=30 Dec 03 11:51:17 crc kubenswrapper[5052]: I1203 11:51:17.228929 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/cinder-scheduler-0" podUID="61d6dd4f-56ef-4f1d-8536-ea66f56188ce" containerName="probe" containerID="cri-o://ff503e474d5755878e81b768ad453fa52004dce2cb8e5b92698649c6b7907845" gracePeriod=30 Dec 03 11:51:17 crc kubenswrapper[5052]: I1203 11:51:17.308205 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/barbican-api-67d5b9d8c8-4gw9z" Dec 03 11:51:17 crc kubenswrapper[5052]: I1203 11:51:17.328584 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/barbican-api-67d5b9d8c8-4gw9z" Dec 03 11:51:18 crc kubenswrapper[5052]: I1203 11:51:18.239462 5052 generic.go:334] "Generic (PLEG): container finished" podID="61d6dd4f-56ef-4f1d-8536-ea66f56188ce" containerID="ff503e474d5755878e81b768ad453fa52004dce2cb8e5b92698649c6b7907845" exitCode=0 Dec 03 11:51:18 crc kubenswrapper[5052]: I1203 11:51:18.239717 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"61d6dd4f-56ef-4f1d-8536-ea66f56188ce","Type":"ContainerDied","Data":"ff503e474d5755878e81b768ad453fa52004dce2cb8e5b92698649c6b7907845"} Dec 03 11:51:19 crc kubenswrapper[5052]: I1203 11:51:19.253914 5052 generic.go:334] "Generic (PLEG): container finished" podID="61d6dd4f-56ef-4f1d-8536-ea66f56188ce" containerID="88bb12213446d0cc45586efb2f6ce025322fc258ef04095763e5dd0212b596c5" exitCode=0 Dec 03 11:51:19 crc kubenswrapper[5052]: I1203 11:51:19.254105 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"61d6dd4f-56ef-4f1d-8536-ea66f56188ce","Type":"ContainerDied","Data":"88bb12213446d0cc45586efb2f6ce025322fc258ef04095763e5dd0212b596c5"} Dec 03 11:51:19 crc kubenswrapper[5052]: I1203 11:51:19.258262 5052 generic.go:334] "Generic (PLEG): container finished" podID="d6eac989-1eec-4b3a-b9d7-d5a8da2c1a27" containerID="ad2e20f86ff81ee90b61200f14867c2ba74e9e0bbf7e5ed7916c06cfc4183fad" exitCode=0 Dec 03 11:51:19 crc kubenswrapper[5052]: I1203 11:51:19.258310 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-db-sync-ghfwv" event={"ID":"d6eac989-1eec-4b3a-b9d7-d5a8da2c1a27","Type":"ContainerDied","Data":"ad2e20f86ff81ee90b61200f14867c2ba74e9e0bbf7e5ed7916c06cfc4183fad"} Dec 03 11:51:19 crc kubenswrapper[5052]: I1203 11:51:19.569223 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:51:19 crc kubenswrapper[5052]: I1203 11:51:19.729704 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/61d6dd4f-56ef-4f1d-8536-ea66f56188ce-scripts\") pod \"61d6dd4f-56ef-4f1d-8536-ea66f56188ce\" (UID: \"61d6dd4f-56ef-4f1d-8536-ea66f56188ce\") " Dec 03 11:51:19 crc kubenswrapper[5052]: I1203 11:51:19.730106 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/61d6dd4f-56ef-4f1d-8536-ea66f56188ce-etc-machine-id\") pod \"61d6dd4f-56ef-4f1d-8536-ea66f56188ce\" (UID: \"61d6dd4f-56ef-4f1d-8536-ea66f56188ce\") " Dec 03 11:51:19 crc kubenswrapper[5052]: I1203 11:51:19.730237 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/61d6dd4f-56ef-4f1d-8536-ea66f56188ce-config-data\") pod \"61d6dd4f-56ef-4f1d-8536-ea66f56188ce\" (UID: \"61d6dd4f-56ef-4f1d-8536-ea66f56188ce\") " Dec 03 11:51:19 crc kubenswrapper[5052]: I1203 11:51:19.730291 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-79djt\" (UniqueName: \"kubernetes.io/projected/61d6dd4f-56ef-4f1d-8536-ea66f56188ce-kube-api-access-79djt\") pod \"61d6dd4f-56ef-4f1d-8536-ea66f56188ce\" (UID: \"61d6dd4f-56ef-4f1d-8536-ea66f56188ce\") " Dec 03 11:51:19 crc kubenswrapper[5052]: I1203 11:51:19.730314 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/61d6dd4f-56ef-4f1d-8536-ea66f56188ce-combined-ca-bundle\") pod \"61d6dd4f-56ef-4f1d-8536-ea66f56188ce\" (UID: \"61d6dd4f-56ef-4f1d-8536-ea66f56188ce\") " Dec 03 11:51:19 crc kubenswrapper[5052]: I1203 11:51:19.730368 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/61d6dd4f-56ef-4f1d-8536-ea66f56188ce-config-data-custom\") pod \"61d6dd4f-56ef-4f1d-8536-ea66f56188ce\" (UID: \"61d6dd4f-56ef-4f1d-8536-ea66f56188ce\") " Dec 03 11:51:19 crc kubenswrapper[5052]: I1203 11:51:19.730364 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/61d6dd4f-56ef-4f1d-8536-ea66f56188ce-etc-machine-id" (OuterVolumeSpecName: "etc-machine-id") pod "61d6dd4f-56ef-4f1d-8536-ea66f56188ce" (UID: "61d6dd4f-56ef-4f1d-8536-ea66f56188ce"). InnerVolumeSpecName "etc-machine-id". PluginName "kubernetes.io/host-path", VolumeGidValue "" Dec 03 11:51:19 crc kubenswrapper[5052]: I1203 11:51:19.730843 5052 reconciler_common.go:293] "Volume detached for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/61d6dd4f-56ef-4f1d-8536-ea66f56188ce-etc-machine-id\") on node \"crc\" DevicePath \"\"" Dec 03 11:51:19 crc kubenswrapper[5052]: I1203 11:51:19.736330 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/61d6dd4f-56ef-4f1d-8536-ea66f56188ce-kube-api-access-79djt" (OuterVolumeSpecName: "kube-api-access-79djt") pod "61d6dd4f-56ef-4f1d-8536-ea66f56188ce" (UID: "61d6dd4f-56ef-4f1d-8536-ea66f56188ce"). InnerVolumeSpecName "kube-api-access-79djt". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:51:19 crc kubenswrapper[5052]: I1203 11:51:19.736443 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/61d6dd4f-56ef-4f1d-8536-ea66f56188ce-config-data-custom" (OuterVolumeSpecName: "config-data-custom") pod "61d6dd4f-56ef-4f1d-8536-ea66f56188ce" (UID: "61d6dd4f-56ef-4f1d-8536-ea66f56188ce"). InnerVolumeSpecName "config-data-custom". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:51:19 crc kubenswrapper[5052]: I1203 11:51:19.737440 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/61d6dd4f-56ef-4f1d-8536-ea66f56188ce-scripts" (OuterVolumeSpecName: "scripts") pod "61d6dd4f-56ef-4f1d-8536-ea66f56188ce" (UID: "61d6dd4f-56ef-4f1d-8536-ea66f56188ce"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:51:19 crc kubenswrapper[5052]: I1203 11:51:19.781146 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/61d6dd4f-56ef-4f1d-8536-ea66f56188ce-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "61d6dd4f-56ef-4f1d-8536-ea66f56188ce" (UID: "61d6dd4f-56ef-4f1d-8536-ea66f56188ce"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:51:19 crc kubenswrapper[5052]: I1203 11:51:19.821695 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/61d6dd4f-56ef-4f1d-8536-ea66f56188ce-config-data" (OuterVolumeSpecName: "config-data") pod "61d6dd4f-56ef-4f1d-8536-ea66f56188ce" (UID: "61d6dd4f-56ef-4f1d-8536-ea66f56188ce"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:51:19 crc kubenswrapper[5052]: I1203 11:51:19.834677 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/61d6dd4f-56ef-4f1d-8536-ea66f56188ce-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:51:19 crc kubenswrapper[5052]: I1203 11:51:19.834718 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-79djt\" (UniqueName: \"kubernetes.io/projected/61d6dd4f-56ef-4f1d-8536-ea66f56188ce-kube-api-access-79djt\") on node \"crc\" DevicePath \"\"" Dec 03 11:51:19 crc kubenswrapper[5052]: I1203 11:51:19.834731 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/61d6dd4f-56ef-4f1d-8536-ea66f56188ce-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:51:19 crc kubenswrapper[5052]: I1203 11:51:19.834741 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/61d6dd4f-56ef-4f1d-8536-ea66f56188ce-config-data-custom\") on node \"crc\" DevicePath \"\"" Dec 03 11:51:19 crc kubenswrapper[5052]: I1203 11:51:19.834751 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/61d6dd4f-56ef-4f1d-8536-ea66f56188ce-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:51:20 crc kubenswrapper[5052]: I1203 11:51:20.270067 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"61d6dd4f-56ef-4f1d-8536-ea66f56188ce","Type":"ContainerDied","Data":"c853d087f6b4b77264beb515e122dbd4f993224b36caaa255ab796b5224a6ecb"} Dec 03 11:51:20 crc kubenswrapper[5052]: I1203 11:51:20.270104 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:51:20 crc kubenswrapper[5052]: I1203 11:51:20.270127 5052 scope.go:117] "RemoveContainer" containerID="ff503e474d5755878e81b768ad453fa52004dce2cb8e5b92698649c6b7907845" Dec 03 11:51:20 crc kubenswrapper[5052]: I1203 11:51:20.315278 5052 scope.go:117] "RemoveContainer" containerID="88bb12213446d0cc45586efb2f6ce025322fc258ef04095763e5dd0212b596c5" Dec 03 11:51:20 crc kubenswrapper[5052]: I1203 11:51:20.315296 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 11:51:20 crc kubenswrapper[5052]: I1203 11:51:20.326926 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 11:51:20 crc kubenswrapper[5052]: I1203 11:51:20.356544 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 11:51:20 crc kubenswrapper[5052]: E1203 11:51:20.356927 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="61d6dd4f-56ef-4f1d-8536-ea66f56188ce" containerName="cinder-scheduler" Dec 03 11:51:20 crc kubenswrapper[5052]: I1203 11:51:20.356944 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="61d6dd4f-56ef-4f1d-8536-ea66f56188ce" containerName="cinder-scheduler" Dec 03 11:51:20 crc kubenswrapper[5052]: E1203 11:51:20.356965 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="61d6dd4f-56ef-4f1d-8536-ea66f56188ce" containerName="probe" Dec 03 11:51:20 crc kubenswrapper[5052]: I1203 11:51:20.356971 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="61d6dd4f-56ef-4f1d-8536-ea66f56188ce" containerName="probe" Dec 03 11:51:20 crc kubenswrapper[5052]: I1203 11:51:20.357189 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="61d6dd4f-56ef-4f1d-8536-ea66f56188ce" containerName="probe" Dec 03 11:51:20 crc kubenswrapper[5052]: I1203 11:51:20.357223 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="61d6dd4f-56ef-4f1d-8536-ea66f56188ce" containerName="cinder-scheduler" Dec 03 11:51:20 crc kubenswrapper[5052]: I1203 11:51:20.359197 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:51:20 crc kubenswrapper[5052]: I1203 11:51:20.360935 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cinder-scheduler-config-data" Dec 03 11:51:20 crc kubenswrapper[5052]: I1203 11:51:20.373759 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 11:51:20 crc kubenswrapper[5052]: I1203 11:51:20.448792 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/0201d941-cd05-4d1c-9af6-f3a82a375a0f-config-data-custom\") pod \"cinder-scheduler-0\" (UID: \"0201d941-cd05-4d1c-9af6-f3a82a375a0f\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:51:20 crc kubenswrapper[5052]: I1203 11:51:20.449097 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/0201d941-cd05-4d1c-9af6-f3a82a375a0f-etc-machine-id\") pod \"cinder-scheduler-0\" (UID: \"0201d941-cd05-4d1c-9af6-f3a82a375a0f\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:51:20 crc kubenswrapper[5052]: I1203 11:51:20.449141 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zczqd\" (UniqueName: \"kubernetes.io/projected/0201d941-cd05-4d1c-9af6-f3a82a375a0f-kube-api-access-zczqd\") pod \"cinder-scheduler-0\" (UID: \"0201d941-cd05-4d1c-9af6-f3a82a375a0f\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:51:20 crc kubenswrapper[5052]: I1203 11:51:20.449221 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/0201d941-cd05-4d1c-9af6-f3a82a375a0f-config-data\") pod \"cinder-scheduler-0\" (UID: \"0201d941-cd05-4d1c-9af6-f3a82a375a0f\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:51:20 crc kubenswrapper[5052]: I1203 11:51:20.449236 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/0201d941-cd05-4d1c-9af6-f3a82a375a0f-scripts\") pod \"cinder-scheduler-0\" (UID: \"0201d941-cd05-4d1c-9af6-f3a82a375a0f\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:51:20 crc kubenswrapper[5052]: I1203 11:51:20.449341 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0201d941-cd05-4d1c-9af6-f3a82a375a0f-combined-ca-bundle\") pod \"cinder-scheduler-0\" (UID: \"0201d941-cd05-4d1c-9af6-f3a82a375a0f\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:51:20 crc kubenswrapper[5052]: I1203 11:51:20.551018 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0201d941-cd05-4d1c-9af6-f3a82a375a0f-combined-ca-bundle\") pod \"cinder-scheduler-0\" (UID: \"0201d941-cd05-4d1c-9af6-f3a82a375a0f\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:51:20 crc kubenswrapper[5052]: I1203 11:51:20.551101 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/0201d941-cd05-4d1c-9af6-f3a82a375a0f-config-data-custom\") pod \"cinder-scheduler-0\" (UID: \"0201d941-cd05-4d1c-9af6-f3a82a375a0f\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:51:20 crc kubenswrapper[5052]: I1203 11:51:20.551128 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/0201d941-cd05-4d1c-9af6-f3a82a375a0f-etc-machine-id\") pod \"cinder-scheduler-0\" (UID: \"0201d941-cd05-4d1c-9af6-f3a82a375a0f\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:51:20 crc kubenswrapper[5052]: I1203 11:51:20.551155 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-zczqd\" (UniqueName: \"kubernetes.io/projected/0201d941-cd05-4d1c-9af6-f3a82a375a0f-kube-api-access-zczqd\") pod \"cinder-scheduler-0\" (UID: \"0201d941-cd05-4d1c-9af6-f3a82a375a0f\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:51:20 crc kubenswrapper[5052]: I1203 11:51:20.551204 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/0201d941-cd05-4d1c-9af6-f3a82a375a0f-etc-machine-id\") pod \"cinder-scheduler-0\" (UID: \"0201d941-cd05-4d1c-9af6-f3a82a375a0f\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:51:20 crc kubenswrapper[5052]: I1203 11:51:20.551208 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/0201d941-cd05-4d1c-9af6-f3a82a375a0f-config-data\") pod \"cinder-scheduler-0\" (UID: \"0201d941-cd05-4d1c-9af6-f3a82a375a0f\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:51:20 crc kubenswrapper[5052]: I1203 11:51:20.551560 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/0201d941-cd05-4d1c-9af6-f3a82a375a0f-scripts\") pod \"cinder-scheduler-0\" (UID: \"0201d941-cd05-4d1c-9af6-f3a82a375a0f\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:51:20 crc kubenswrapper[5052]: I1203 11:51:20.556848 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/0201d941-cd05-4d1c-9af6-f3a82a375a0f-scripts\") pod \"cinder-scheduler-0\" (UID: \"0201d941-cd05-4d1c-9af6-f3a82a375a0f\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:51:20 crc kubenswrapper[5052]: I1203 11:51:20.556879 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/0201d941-cd05-4d1c-9af6-f3a82a375a0f-config-data-custom\") pod \"cinder-scheduler-0\" (UID: \"0201d941-cd05-4d1c-9af6-f3a82a375a0f\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:51:20 crc kubenswrapper[5052]: I1203 11:51:20.569702 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0201d941-cd05-4d1c-9af6-f3a82a375a0f-combined-ca-bundle\") pod \"cinder-scheduler-0\" (UID: \"0201d941-cd05-4d1c-9af6-f3a82a375a0f\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:51:20 crc kubenswrapper[5052]: I1203 11:51:20.570266 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/0201d941-cd05-4d1c-9af6-f3a82a375a0f-config-data\") pod \"cinder-scheduler-0\" (UID: \"0201d941-cd05-4d1c-9af6-f3a82a375a0f\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:51:20 crc kubenswrapper[5052]: I1203 11:51:20.572070 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-zczqd\" (UniqueName: \"kubernetes.io/projected/0201d941-cd05-4d1c-9af6-f3a82a375a0f-kube-api-access-zczqd\") pod \"cinder-scheduler-0\" (UID: \"0201d941-cd05-4d1c-9af6-f3a82a375a0f\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:51:20 crc kubenswrapper[5052]: I1203 11:51:20.651444 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-db-sync-ghfwv" Dec 03 11:51:20 crc kubenswrapper[5052]: I1203 11:51:20.681171 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:51:20 crc kubenswrapper[5052]: I1203 11:51:20.759287 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d6eac989-1eec-4b3a-b9d7-d5a8da2c1a27-combined-ca-bundle\") pod \"d6eac989-1eec-4b3a-b9d7-d5a8da2c1a27\" (UID: \"d6eac989-1eec-4b3a-b9d7-d5a8da2c1a27\") " Dec 03 11:51:20 crc kubenswrapper[5052]: I1203 11:51:20.759349 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9xwxv\" (UniqueName: \"kubernetes.io/projected/d6eac989-1eec-4b3a-b9d7-d5a8da2c1a27-kube-api-access-9xwxv\") pod \"d6eac989-1eec-4b3a-b9d7-d5a8da2c1a27\" (UID: \"d6eac989-1eec-4b3a-b9d7-d5a8da2c1a27\") " Dec 03 11:51:20 crc kubenswrapper[5052]: I1203 11:51:20.759445 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/d6eac989-1eec-4b3a-b9d7-d5a8da2c1a27-config\") pod \"d6eac989-1eec-4b3a-b9d7-d5a8da2c1a27\" (UID: \"d6eac989-1eec-4b3a-b9d7-d5a8da2c1a27\") " Dec 03 11:51:20 crc kubenswrapper[5052]: I1203 11:51:20.765592 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d6eac989-1eec-4b3a-b9d7-d5a8da2c1a27-kube-api-access-9xwxv" (OuterVolumeSpecName: "kube-api-access-9xwxv") pod "d6eac989-1eec-4b3a-b9d7-d5a8da2c1a27" (UID: "d6eac989-1eec-4b3a-b9d7-d5a8da2c1a27"). InnerVolumeSpecName "kube-api-access-9xwxv". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:51:20 crc kubenswrapper[5052]: I1203 11:51:20.782000 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d6eac989-1eec-4b3a-b9d7-d5a8da2c1a27-config" (OuterVolumeSpecName: "config") pod "d6eac989-1eec-4b3a-b9d7-d5a8da2c1a27" (UID: "d6eac989-1eec-4b3a-b9d7-d5a8da2c1a27"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:51:20 crc kubenswrapper[5052]: I1203 11:51:20.792685 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d6eac989-1eec-4b3a-b9d7-d5a8da2c1a27-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "d6eac989-1eec-4b3a-b9d7-d5a8da2c1a27" (UID: "d6eac989-1eec-4b3a-b9d7-d5a8da2c1a27"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:51:20 crc kubenswrapper[5052]: I1203 11:51:20.861443 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d6eac989-1eec-4b3a-b9d7-d5a8da2c1a27-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:51:20 crc kubenswrapper[5052]: I1203 11:51:20.861489 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-9xwxv\" (UniqueName: \"kubernetes.io/projected/d6eac989-1eec-4b3a-b9d7-d5a8da2c1a27-kube-api-access-9xwxv\") on node \"crc\" DevicePath \"\"" Dec 03 11:51:20 crc kubenswrapper[5052]: I1203 11:51:20.861532 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/d6eac989-1eec-4b3a-b9d7-d5a8da2c1a27-config\") on node \"crc\" DevicePath \"\"" Dec 03 11:51:21 crc kubenswrapper[5052]: I1203 11:51:21.149035 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 11:51:21 crc kubenswrapper[5052]: I1203 11:51:21.286461 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"0201d941-cd05-4d1c-9af6-f3a82a375a0f","Type":"ContainerStarted","Data":"a172d8e41b4536cc8c4a79909bc82ba0ecbe2bd22bde8680827a3a0781216413"} Dec 03 11:51:21 crc kubenswrapper[5052]: I1203 11:51:21.310681 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-db-sync-ghfwv" event={"ID":"d6eac989-1eec-4b3a-b9d7-d5a8da2c1a27","Type":"ContainerDied","Data":"f798e9ff4b47add1643ceb7425c2e5044ecab55dae14267b62eac3c86bf67a70"} Dec 03 11:51:21 crc kubenswrapper[5052]: I1203 11:51:21.310728 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f798e9ff4b47add1643ceb7425c2e5044ecab55dae14267b62eac3c86bf67a70" Dec 03 11:51:21 crc kubenswrapper[5052]: I1203 11:51:21.311611 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-db-sync-ghfwv" Dec 03 11:51:21 crc kubenswrapper[5052]: I1203 11:51:21.545663 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/neutron-68d4b65fff-k4c5r"] Dec 03 11:51:21 crc kubenswrapper[5052]: E1203 11:51:21.547277 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="d6eac989-1eec-4b3a-b9d7-d5a8da2c1a27" containerName="neutron-db-sync" Dec 03 11:51:21 crc kubenswrapper[5052]: I1203 11:51:21.547290 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="d6eac989-1eec-4b3a-b9d7-d5a8da2c1a27" containerName="neutron-db-sync" Dec 03 11:51:21 crc kubenswrapper[5052]: I1203 11:51:21.547490 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="d6eac989-1eec-4b3a-b9d7-d5a8da2c1a27" containerName="neutron-db-sync" Dec 03 11:51:21 crc kubenswrapper[5052]: I1203 11:51:21.548365 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-68d4b65fff-k4c5r" Dec 03 11:51:21 crc kubenswrapper[5052]: I1203 11:51:21.551134 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-neutron-ovndbs" Dec 03 11:51:21 crc kubenswrapper[5052]: I1203 11:51:21.551269 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"neutron-httpd-config" Dec 03 11:51:21 crc kubenswrapper[5052]: I1203 11:51:21.551372 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"neutron-neutron-dockercfg-nslhs" Dec 03 11:51:21 crc kubenswrapper[5052]: I1203 11:51:21.557236 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"neutron-config" Dec 03 11:51:21 crc kubenswrapper[5052]: I1203 11:51:21.574464 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-68d4b65fff-k4c5r"] Dec 03 11:51:21 crc kubenswrapper[5052]: I1203 11:51:21.583808 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/d611c43f-d958-4cef-86d5-32298150487c-ovndb-tls-certs\") pod \"neutron-68d4b65fff-k4c5r\" (UID: \"d611c43f-d958-4cef-86d5-32298150487c\") " pod="openstack-kuttl-tests/neutron-68d4b65fff-k4c5r" Dec 03 11:51:21 crc kubenswrapper[5052]: I1203 11:51:21.583876 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/d611c43f-d958-4cef-86d5-32298150487c-config\") pod \"neutron-68d4b65fff-k4c5r\" (UID: \"d611c43f-d958-4cef-86d5-32298150487c\") " pod="openstack-kuttl-tests/neutron-68d4b65fff-k4c5r" Dec 03 11:51:21 crc kubenswrapper[5052]: I1203 11:51:21.583909 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n55xl\" (UniqueName: \"kubernetes.io/projected/d611c43f-d958-4cef-86d5-32298150487c-kube-api-access-n55xl\") pod \"neutron-68d4b65fff-k4c5r\" (UID: \"d611c43f-d958-4cef-86d5-32298150487c\") " pod="openstack-kuttl-tests/neutron-68d4b65fff-k4c5r" Dec 03 11:51:21 crc kubenswrapper[5052]: I1203 11:51:21.583945 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d611c43f-d958-4cef-86d5-32298150487c-combined-ca-bundle\") pod \"neutron-68d4b65fff-k4c5r\" (UID: \"d611c43f-d958-4cef-86d5-32298150487c\") " pod="openstack-kuttl-tests/neutron-68d4b65fff-k4c5r" Dec 03 11:51:21 crc kubenswrapper[5052]: I1203 11:51:21.584078 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/d611c43f-d958-4cef-86d5-32298150487c-httpd-config\") pod \"neutron-68d4b65fff-k4c5r\" (UID: \"d611c43f-d958-4cef-86d5-32298150487c\") " pod="openstack-kuttl-tests/neutron-68d4b65fff-k4c5r" Dec 03 11:51:21 crc kubenswrapper[5052]: I1203 11:51:21.688536 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="61d6dd4f-56ef-4f1d-8536-ea66f56188ce" path="/var/lib/kubelet/pods/61d6dd4f-56ef-4f1d-8536-ea66f56188ce/volumes" Dec 03 11:51:21 crc kubenswrapper[5052]: I1203 11:51:21.725679 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/d611c43f-d958-4cef-86d5-32298150487c-ovndb-tls-certs\") pod \"neutron-68d4b65fff-k4c5r\" (UID: \"d611c43f-d958-4cef-86d5-32298150487c\") " pod="openstack-kuttl-tests/neutron-68d4b65fff-k4c5r" Dec 03 11:51:21 crc kubenswrapper[5052]: I1203 11:51:21.725756 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/d611c43f-d958-4cef-86d5-32298150487c-config\") pod \"neutron-68d4b65fff-k4c5r\" (UID: \"d611c43f-d958-4cef-86d5-32298150487c\") " pod="openstack-kuttl-tests/neutron-68d4b65fff-k4c5r" Dec 03 11:51:21 crc kubenswrapper[5052]: I1203 11:51:21.725787 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-n55xl\" (UniqueName: \"kubernetes.io/projected/d611c43f-d958-4cef-86d5-32298150487c-kube-api-access-n55xl\") pod \"neutron-68d4b65fff-k4c5r\" (UID: \"d611c43f-d958-4cef-86d5-32298150487c\") " pod="openstack-kuttl-tests/neutron-68d4b65fff-k4c5r" Dec 03 11:51:21 crc kubenswrapper[5052]: I1203 11:51:21.725822 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d611c43f-d958-4cef-86d5-32298150487c-combined-ca-bundle\") pod \"neutron-68d4b65fff-k4c5r\" (UID: \"d611c43f-d958-4cef-86d5-32298150487c\") " pod="openstack-kuttl-tests/neutron-68d4b65fff-k4c5r" Dec 03 11:51:21 crc kubenswrapper[5052]: I1203 11:51:21.725968 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/d611c43f-d958-4cef-86d5-32298150487c-httpd-config\") pod \"neutron-68d4b65fff-k4c5r\" (UID: \"d611c43f-d958-4cef-86d5-32298150487c\") " pod="openstack-kuttl-tests/neutron-68d4b65fff-k4c5r" Dec 03 11:51:21 crc kubenswrapper[5052]: I1203 11:51:21.730923 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d611c43f-d958-4cef-86d5-32298150487c-combined-ca-bundle\") pod \"neutron-68d4b65fff-k4c5r\" (UID: \"d611c43f-d958-4cef-86d5-32298150487c\") " pod="openstack-kuttl-tests/neutron-68d4b65fff-k4c5r" Dec 03 11:51:21 crc kubenswrapper[5052]: I1203 11:51:21.731859 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/d611c43f-d958-4cef-86d5-32298150487c-ovndb-tls-certs\") pod \"neutron-68d4b65fff-k4c5r\" (UID: \"d611c43f-d958-4cef-86d5-32298150487c\") " pod="openstack-kuttl-tests/neutron-68d4b65fff-k4c5r" Dec 03 11:51:21 crc kubenswrapper[5052]: I1203 11:51:21.737492 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/d611c43f-d958-4cef-86d5-32298150487c-config\") pod \"neutron-68d4b65fff-k4c5r\" (UID: \"d611c43f-d958-4cef-86d5-32298150487c\") " pod="openstack-kuttl-tests/neutron-68d4b65fff-k4c5r" Dec 03 11:51:21 crc kubenswrapper[5052]: I1203 11:51:21.747708 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/d611c43f-d958-4cef-86d5-32298150487c-httpd-config\") pod \"neutron-68d4b65fff-k4c5r\" (UID: \"d611c43f-d958-4cef-86d5-32298150487c\") " pod="openstack-kuttl-tests/neutron-68d4b65fff-k4c5r" Dec 03 11:51:21 crc kubenswrapper[5052]: I1203 11:51:21.754381 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-n55xl\" (UniqueName: \"kubernetes.io/projected/d611c43f-d958-4cef-86d5-32298150487c-kube-api-access-n55xl\") pod \"neutron-68d4b65fff-k4c5r\" (UID: \"d611c43f-d958-4cef-86d5-32298150487c\") " pod="openstack-kuttl-tests/neutron-68d4b65fff-k4c5r" Dec 03 11:51:21 crc kubenswrapper[5052]: I1203 11:51:21.942827 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-68d4b65fff-k4c5r" Dec 03 11:51:22 crc kubenswrapper[5052]: I1203 11:51:22.345597 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-68d4b65fff-k4c5r"] Dec 03 11:51:22 crc kubenswrapper[5052]: I1203 11:51:22.376204 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"0201d941-cd05-4d1c-9af6-f3a82a375a0f","Type":"ContainerStarted","Data":"24b9c22c01a17699dc23692d82c8a6d2aba256d2fe1411900dc2de65cbd33d93"} Dec 03 11:51:23 crc kubenswrapper[5052]: I1203 11:51:23.386538 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-68d4b65fff-k4c5r" event={"ID":"d611c43f-d958-4cef-86d5-32298150487c","Type":"ContainerStarted","Data":"281c11f92f4f6b8ca18c5f7fb7584ce5ead1606dbee8e1ff85ce5bd12ce1c76b"} Dec 03 11:51:23 crc kubenswrapper[5052]: I1203 11:51:23.387117 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-68d4b65fff-k4c5r" event={"ID":"d611c43f-d958-4cef-86d5-32298150487c","Type":"ContainerStarted","Data":"654e271093b52e242fc7152c45ab8928f835c7293e10fd27319b88f10d29b882"} Dec 03 11:51:23 crc kubenswrapper[5052]: I1203 11:51:23.387129 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-68d4b65fff-k4c5r" event={"ID":"d611c43f-d958-4cef-86d5-32298150487c","Type":"ContainerStarted","Data":"08cf4bb00114f457940aa1dbde1e6d7d4d4450b512b0bd8ca885622ef40792b5"} Dec 03 11:51:23 crc kubenswrapper[5052]: I1203 11:51:23.389403 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"0201d941-cd05-4d1c-9af6-f3a82a375a0f","Type":"ContainerStarted","Data":"f0df1e8717d84a41bd64b8ee9a76553b85d16b020416be110d56d085ed4f7b45"} Dec 03 11:51:23 crc kubenswrapper[5052]: I1203 11:51:23.857990 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/barbican-api-5758478fc6-5g46v" Dec 03 11:51:23 crc kubenswrapper[5052]: I1203 11:51:23.864804 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/barbican-api-5758478fc6-5g46v" Dec 03 11:51:23 crc kubenswrapper[5052]: I1203 11:51:23.998093 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-api-67d5b9d8c8-4gw9z"] Dec 03 11:51:23 crc kubenswrapper[5052]: I1203 11:51:23.998391 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/barbican-api-67d5b9d8c8-4gw9z" podUID="2b36c173-6bfb-4fcc-858f-19e0ababbf95" containerName="barbican-api-log" containerID="cri-o://61036e95ab93308bc10e8d596f78981c7d8a8b273993f37dcf6c5a34caac782a" gracePeriod=30 Dec 03 11:51:23 crc kubenswrapper[5052]: I1203 11:51:23.998580 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/barbican-api-67d5b9d8c8-4gw9z" podUID="2b36c173-6bfb-4fcc-858f-19e0ababbf95" containerName="barbican-api" containerID="cri-o://d6f5255145150903893d430668b91fdf726ecc26c094e89b13fdc75a3e249f5d" gracePeriod=30 Dec 03 11:51:24 crc kubenswrapper[5052]: I1203 11:51:24.402845 5052 generic.go:334] "Generic (PLEG): container finished" podID="2b36c173-6bfb-4fcc-858f-19e0ababbf95" containerID="61036e95ab93308bc10e8d596f78981c7d8a8b273993f37dcf6c5a34caac782a" exitCode=143 Dec 03 11:51:24 crc kubenswrapper[5052]: I1203 11:51:24.404048 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-67d5b9d8c8-4gw9z" event={"ID":"2b36c173-6bfb-4fcc-858f-19e0ababbf95","Type":"ContainerDied","Data":"61036e95ab93308bc10e8d596f78981c7d8a8b273993f37dcf6c5a34caac782a"} Dec 03 11:51:24 crc kubenswrapper[5052]: I1203 11:51:24.404224 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/neutron-68d4b65fff-k4c5r" Dec 03 11:51:24 crc kubenswrapper[5052]: I1203 11:51:24.428766 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/cinder-scheduler-0" podStartSLOduration=4.428747967 podStartE2EDuration="4.428747967s" podCreationTimestamp="2025-12-03 11:51:20 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:51:24.423802915 +0000 UTC m=+1875.012044679" watchObservedRunningTime="2025-12-03 11:51:24.428747967 +0000 UTC m=+1875.016989731" Dec 03 11:51:24 crc kubenswrapper[5052]: I1203 11:51:24.461040 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/neutron-68d4b65fff-k4c5r" podStartSLOduration=3.461015883 podStartE2EDuration="3.461015883s" podCreationTimestamp="2025-12-03 11:51:21 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:51:24.460811147 +0000 UTC m=+1875.049052921" watchObservedRunningTime="2025-12-03 11:51:24.461015883 +0000 UTC m=+1875.049257647" Dec 03 11:51:24 crc kubenswrapper[5052]: I1203 11:51:24.858905 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/neutron-5ff446f48b-7hc25"] Dec 03 11:51:24 crc kubenswrapper[5052]: I1203 11:51:24.860387 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-5ff446f48b-7hc25" Dec 03 11:51:24 crc kubenswrapper[5052]: I1203 11:51:24.869207 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-neutron-internal-svc" Dec 03 11:51:24 crc kubenswrapper[5052]: I1203 11:51:24.869458 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-neutron-public-svc" Dec 03 11:51:24 crc kubenswrapper[5052]: I1203 11:51:24.875463 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-5ff446f48b-7hc25"] Dec 03 11:51:25 crc kubenswrapper[5052]: I1203 11:51:25.006303 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/83b9fcfd-8007-4d6b-9090-d2e536cb1bbc-internal-tls-certs\") pod \"neutron-5ff446f48b-7hc25\" (UID: \"83b9fcfd-8007-4d6b-9090-d2e536cb1bbc\") " pod="openstack-kuttl-tests/neutron-5ff446f48b-7hc25" Dec 03 11:51:25 crc kubenswrapper[5052]: I1203 11:51:25.006388 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/83b9fcfd-8007-4d6b-9090-d2e536cb1bbc-config\") pod \"neutron-5ff446f48b-7hc25\" (UID: \"83b9fcfd-8007-4d6b-9090-d2e536cb1bbc\") " pod="openstack-kuttl-tests/neutron-5ff446f48b-7hc25" Dec 03 11:51:25 crc kubenswrapper[5052]: I1203 11:51:25.006412 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/83b9fcfd-8007-4d6b-9090-d2e536cb1bbc-combined-ca-bundle\") pod \"neutron-5ff446f48b-7hc25\" (UID: \"83b9fcfd-8007-4d6b-9090-d2e536cb1bbc\") " pod="openstack-kuttl-tests/neutron-5ff446f48b-7hc25" Dec 03 11:51:25 crc kubenswrapper[5052]: I1203 11:51:25.006429 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/83b9fcfd-8007-4d6b-9090-d2e536cb1bbc-httpd-config\") pod \"neutron-5ff446f48b-7hc25\" (UID: \"83b9fcfd-8007-4d6b-9090-d2e536cb1bbc\") " pod="openstack-kuttl-tests/neutron-5ff446f48b-7hc25" Dec 03 11:51:25 crc kubenswrapper[5052]: I1203 11:51:25.006474 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6lpvb\" (UniqueName: \"kubernetes.io/projected/83b9fcfd-8007-4d6b-9090-d2e536cb1bbc-kube-api-access-6lpvb\") pod \"neutron-5ff446f48b-7hc25\" (UID: \"83b9fcfd-8007-4d6b-9090-d2e536cb1bbc\") " pod="openstack-kuttl-tests/neutron-5ff446f48b-7hc25" Dec 03 11:51:25 crc kubenswrapper[5052]: I1203 11:51:25.006732 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/83b9fcfd-8007-4d6b-9090-d2e536cb1bbc-public-tls-certs\") pod \"neutron-5ff446f48b-7hc25\" (UID: \"83b9fcfd-8007-4d6b-9090-d2e536cb1bbc\") " pod="openstack-kuttl-tests/neutron-5ff446f48b-7hc25" Dec 03 11:51:25 crc kubenswrapper[5052]: I1203 11:51:25.006860 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/83b9fcfd-8007-4d6b-9090-d2e536cb1bbc-ovndb-tls-certs\") pod \"neutron-5ff446f48b-7hc25\" (UID: \"83b9fcfd-8007-4d6b-9090-d2e536cb1bbc\") " pod="openstack-kuttl-tests/neutron-5ff446f48b-7hc25" Dec 03 11:51:25 crc kubenswrapper[5052]: I1203 11:51:25.107934 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/83b9fcfd-8007-4d6b-9090-d2e536cb1bbc-public-tls-certs\") pod \"neutron-5ff446f48b-7hc25\" (UID: \"83b9fcfd-8007-4d6b-9090-d2e536cb1bbc\") " pod="openstack-kuttl-tests/neutron-5ff446f48b-7hc25" Dec 03 11:51:25 crc kubenswrapper[5052]: I1203 11:51:25.107994 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/83b9fcfd-8007-4d6b-9090-d2e536cb1bbc-ovndb-tls-certs\") pod \"neutron-5ff446f48b-7hc25\" (UID: \"83b9fcfd-8007-4d6b-9090-d2e536cb1bbc\") " pod="openstack-kuttl-tests/neutron-5ff446f48b-7hc25" Dec 03 11:51:25 crc kubenswrapper[5052]: I1203 11:51:25.108029 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/83b9fcfd-8007-4d6b-9090-d2e536cb1bbc-internal-tls-certs\") pod \"neutron-5ff446f48b-7hc25\" (UID: \"83b9fcfd-8007-4d6b-9090-d2e536cb1bbc\") " pod="openstack-kuttl-tests/neutron-5ff446f48b-7hc25" Dec 03 11:51:25 crc kubenswrapper[5052]: I1203 11:51:25.108096 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/83b9fcfd-8007-4d6b-9090-d2e536cb1bbc-config\") pod \"neutron-5ff446f48b-7hc25\" (UID: \"83b9fcfd-8007-4d6b-9090-d2e536cb1bbc\") " pod="openstack-kuttl-tests/neutron-5ff446f48b-7hc25" Dec 03 11:51:25 crc kubenswrapper[5052]: I1203 11:51:25.108127 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/83b9fcfd-8007-4d6b-9090-d2e536cb1bbc-combined-ca-bundle\") pod \"neutron-5ff446f48b-7hc25\" (UID: \"83b9fcfd-8007-4d6b-9090-d2e536cb1bbc\") " pod="openstack-kuttl-tests/neutron-5ff446f48b-7hc25" Dec 03 11:51:25 crc kubenswrapper[5052]: I1203 11:51:25.108149 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/83b9fcfd-8007-4d6b-9090-d2e536cb1bbc-httpd-config\") pod \"neutron-5ff446f48b-7hc25\" (UID: \"83b9fcfd-8007-4d6b-9090-d2e536cb1bbc\") " pod="openstack-kuttl-tests/neutron-5ff446f48b-7hc25" Dec 03 11:51:25 crc kubenswrapper[5052]: I1203 11:51:25.108207 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-6lpvb\" (UniqueName: \"kubernetes.io/projected/83b9fcfd-8007-4d6b-9090-d2e536cb1bbc-kube-api-access-6lpvb\") pod \"neutron-5ff446f48b-7hc25\" (UID: \"83b9fcfd-8007-4d6b-9090-d2e536cb1bbc\") " pod="openstack-kuttl-tests/neutron-5ff446f48b-7hc25" Dec 03 11:51:25 crc kubenswrapper[5052]: I1203 11:51:25.114210 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/83b9fcfd-8007-4d6b-9090-d2e536cb1bbc-httpd-config\") pod \"neutron-5ff446f48b-7hc25\" (UID: \"83b9fcfd-8007-4d6b-9090-d2e536cb1bbc\") " pod="openstack-kuttl-tests/neutron-5ff446f48b-7hc25" Dec 03 11:51:25 crc kubenswrapper[5052]: I1203 11:51:25.114447 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/83b9fcfd-8007-4d6b-9090-d2e536cb1bbc-internal-tls-certs\") pod \"neutron-5ff446f48b-7hc25\" (UID: \"83b9fcfd-8007-4d6b-9090-d2e536cb1bbc\") " pod="openstack-kuttl-tests/neutron-5ff446f48b-7hc25" Dec 03 11:51:25 crc kubenswrapper[5052]: I1203 11:51:25.117063 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/83b9fcfd-8007-4d6b-9090-d2e536cb1bbc-combined-ca-bundle\") pod \"neutron-5ff446f48b-7hc25\" (UID: \"83b9fcfd-8007-4d6b-9090-d2e536cb1bbc\") " pod="openstack-kuttl-tests/neutron-5ff446f48b-7hc25" Dec 03 11:51:25 crc kubenswrapper[5052]: I1203 11:51:25.117231 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/83b9fcfd-8007-4d6b-9090-d2e536cb1bbc-ovndb-tls-certs\") pod \"neutron-5ff446f48b-7hc25\" (UID: \"83b9fcfd-8007-4d6b-9090-d2e536cb1bbc\") " pod="openstack-kuttl-tests/neutron-5ff446f48b-7hc25" Dec 03 11:51:25 crc kubenswrapper[5052]: I1203 11:51:25.117939 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/83b9fcfd-8007-4d6b-9090-d2e536cb1bbc-config\") pod \"neutron-5ff446f48b-7hc25\" (UID: \"83b9fcfd-8007-4d6b-9090-d2e536cb1bbc\") " pod="openstack-kuttl-tests/neutron-5ff446f48b-7hc25" Dec 03 11:51:25 crc kubenswrapper[5052]: I1203 11:51:25.131900 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-6lpvb\" (UniqueName: \"kubernetes.io/projected/83b9fcfd-8007-4d6b-9090-d2e536cb1bbc-kube-api-access-6lpvb\") pod \"neutron-5ff446f48b-7hc25\" (UID: \"83b9fcfd-8007-4d6b-9090-d2e536cb1bbc\") " pod="openstack-kuttl-tests/neutron-5ff446f48b-7hc25" Dec 03 11:51:25 crc kubenswrapper[5052]: I1203 11:51:25.133406 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/83b9fcfd-8007-4d6b-9090-d2e536cb1bbc-public-tls-certs\") pod \"neutron-5ff446f48b-7hc25\" (UID: \"83b9fcfd-8007-4d6b-9090-d2e536cb1bbc\") " pod="openstack-kuttl-tests/neutron-5ff446f48b-7hc25" Dec 03 11:51:25 crc kubenswrapper[5052]: I1203 11:51:25.184618 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-5ff446f48b-7hc25" Dec 03 11:51:25 crc kubenswrapper[5052]: I1203 11:51:25.689442 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-5ff446f48b-7hc25"] Dec 03 11:51:25 crc kubenswrapper[5052]: I1203 11:51:25.689709 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:51:26 crc kubenswrapper[5052]: I1203 11:51:26.016541 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:51:26 crc kubenswrapper[5052]: I1203 11:51:26.434274 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-5ff446f48b-7hc25" event={"ID":"83b9fcfd-8007-4d6b-9090-d2e536cb1bbc","Type":"ContainerStarted","Data":"d4b83b49b57a27fa3f6b26ff04edd361dfcc21c0d52243555076b619c81a9ec9"} Dec 03 11:51:26 crc kubenswrapper[5052]: I1203 11:51:26.434324 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-5ff446f48b-7hc25" event={"ID":"83b9fcfd-8007-4d6b-9090-d2e536cb1bbc","Type":"ContainerStarted","Data":"cd35d473482d3a4b6bc6f957931d66333a4d95cf45ac228f925b1160c222f1f6"} Dec 03 11:51:26 crc kubenswrapper[5052]: I1203 11:51:26.434336 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-5ff446f48b-7hc25" event={"ID":"83b9fcfd-8007-4d6b-9090-d2e536cb1bbc","Type":"ContainerStarted","Data":"d9a6cf6c4902e067e4f6982221164cc67224cb6bdd2f6ecce5a98dcf499fa966"} Dec 03 11:51:26 crc kubenswrapper[5052]: I1203 11:51:26.434484 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/neutron-5ff446f48b-7hc25" Dec 03 11:51:26 crc kubenswrapper[5052]: I1203 11:51:26.453613 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/neutron-5ff446f48b-7hc25" podStartSLOduration=2.453593099 podStartE2EDuration="2.453593099s" podCreationTimestamp="2025-12-03 11:51:24 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:51:26.452498618 +0000 UTC m=+1877.040740392" watchObservedRunningTime="2025-12-03 11:51:26.453593099 +0000 UTC m=+1877.041834873" Dec 03 11:51:27 crc kubenswrapper[5052]: I1203 11:51:27.202485 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/barbican-api-67d5b9d8c8-4gw9z" podUID="2b36c173-6bfb-4fcc-858f-19e0ababbf95" containerName="barbican-api-log" probeResult="failure" output="Get \"http://10.217.1.91:9311/healthcheck\": read tcp 10.217.0.2:55940->10.217.1.91:9311: read: connection reset by peer" Dec 03 11:51:27 crc kubenswrapper[5052]: I1203 11:51:27.202572 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/barbican-api-67d5b9d8c8-4gw9z" podUID="2b36c173-6bfb-4fcc-858f-19e0ababbf95" containerName="barbican-api" probeResult="failure" output="Get \"http://10.217.1.91:9311/healthcheck\": read tcp 10.217.0.2:55942->10.217.1.91:9311: read: connection reset by peer" Dec 03 11:51:27 crc kubenswrapper[5052]: I1203 11:51:27.455374 5052 generic.go:334] "Generic (PLEG): container finished" podID="2b36c173-6bfb-4fcc-858f-19e0ababbf95" containerID="d6f5255145150903893d430668b91fdf726ecc26c094e89b13fdc75a3e249f5d" exitCode=0 Dec 03 11:51:27 crc kubenswrapper[5052]: I1203 11:51:27.455915 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-67d5b9d8c8-4gw9z" event={"ID":"2b36c173-6bfb-4fcc-858f-19e0ababbf95","Type":"ContainerDied","Data":"d6f5255145150903893d430668b91fdf726ecc26c094e89b13fdc75a3e249f5d"} Dec 03 11:51:27 crc kubenswrapper[5052]: I1203 11:51:27.674807 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-api-67d5b9d8c8-4gw9z" Dec 03 11:51:27 crc kubenswrapper[5052]: I1203 11:51:27.762131 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/2b36c173-6bfb-4fcc-858f-19e0ababbf95-config-data-custom\") pod \"2b36c173-6bfb-4fcc-858f-19e0ababbf95\" (UID: \"2b36c173-6bfb-4fcc-858f-19e0ababbf95\") " Dec 03 11:51:27 crc kubenswrapper[5052]: I1203 11:51:27.762294 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/2b36c173-6bfb-4fcc-858f-19e0ababbf95-config-data\") pod \"2b36c173-6bfb-4fcc-858f-19e0ababbf95\" (UID: \"2b36c173-6bfb-4fcc-858f-19e0ababbf95\") " Dec 03 11:51:27 crc kubenswrapper[5052]: I1203 11:51:27.762372 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/2b36c173-6bfb-4fcc-858f-19e0ababbf95-logs\") pod \"2b36c173-6bfb-4fcc-858f-19e0ababbf95\" (UID: \"2b36c173-6bfb-4fcc-858f-19e0ababbf95\") " Dec 03 11:51:27 crc kubenswrapper[5052]: I1203 11:51:27.762396 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2b36c173-6bfb-4fcc-858f-19e0ababbf95-combined-ca-bundle\") pod \"2b36c173-6bfb-4fcc-858f-19e0ababbf95\" (UID: \"2b36c173-6bfb-4fcc-858f-19e0ababbf95\") " Dec 03 11:51:27 crc kubenswrapper[5052]: I1203 11:51:27.762439 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rfgp2\" (UniqueName: \"kubernetes.io/projected/2b36c173-6bfb-4fcc-858f-19e0ababbf95-kube-api-access-rfgp2\") pod \"2b36c173-6bfb-4fcc-858f-19e0ababbf95\" (UID: \"2b36c173-6bfb-4fcc-858f-19e0ababbf95\") " Dec 03 11:51:27 crc kubenswrapper[5052]: I1203 11:51:27.765520 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2b36c173-6bfb-4fcc-858f-19e0ababbf95-logs" (OuterVolumeSpecName: "logs") pod "2b36c173-6bfb-4fcc-858f-19e0ababbf95" (UID: "2b36c173-6bfb-4fcc-858f-19e0ababbf95"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:51:27 crc kubenswrapper[5052]: I1203 11:51:27.770238 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2b36c173-6bfb-4fcc-858f-19e0ababbf95-kube-api-access-rfgp2" (OuterVolumeSpecName: "kube-api-access-rfgp2") pod "2b36c173-6bfb-4fcc-858f-19e0ababbf95" (UID: "2b36c173-6bfb-4fcc-858f-19e0ababbf95"). InnerVolumeSpecName "kube-api-access-rfgp2". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:51:27 crc kubenswrapper[5052]: I1203 11:51:27.776295 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2b36c173-6bfb-4fcc-858f-19e0ababbf95-config-data-custom" (OuterVolumeSpecName: "config-data-custom") pod "2b36c173-6bfb-4fcc-858f-19e0ababbf95" (UID: "2b36c173-6bfb-4fcc-858f-19e0ababbf95"). InnerVolumeSpecName "config-data-custom". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:51:27 crc kubenswrapper[5052]: I1203 11:51:27.797155 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2b36c173-6bfb-4fcc-858f-19e0ababbf95-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "2b36c173-6bfb-4fcc-858f-19e0ababbf95" (UID: "2b36c173-6bfb-4fcc-858f-19e0ababbf95"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:51:27 crc kubenswrapper[5052]: I1203 11:51:27.815699 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2b36c173-6bfb-4fcc-858f-19e0ababbf95-config-data" (OuterVolumeSpecName: "config-data") pod "2b36c173-6bfb-4fcc-858f-19e0ababbf95" (UID: "2b36c173-6bfb-4fcc-858f-19e0ababbf95"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:51:27 crc kubenswrapper[5052]: I1203 11:51:27.865017 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/2b36c173-6bfb-4fcc-858f-19e0ababbf95-config-data-custom\") on node \"crc\" DevicePath \"\"" Dec 03 11:51:27 crc kubenswrapper[5052]: I1203 11:51:27.865334 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/2b36c173-6bfb-4fcc-858f-19e0ababbf95-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:51:27 crc kubenswrapper[5052]: I1203 11:51:27.865343 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/2b36c173-6bfb-4fcc-858f-19e0ababbf95-logs\") on node \"crc\" DevicePath \"\"" Dec 03 11:51:27 crc kubenswrapper[5052]: I1203 11:51:27.865353 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2b36c173-6bfb-4fcc-858f-19e0ababbf95-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:51:27 crc kubenswrapper[5052]: I1203 11:51:27.865363 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-rfgp2\" (UniqueName: \"kubernetes.io/projected/2b36c173-6bfb-4fcc-858f-19e0ababbf95-kube-api-access-rfgp2\") on node \"crc\" DevicePath \"\"" Dec 03 11:51:28 crc kubenswrapper[5052]: I1203 11:51:28.471041 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-67d5b9d8c8-4gw9z" event={"ID":"2b36c173-6bfb-4fcc-858f-19e0ababbf95","Type":"ContainerDied","Data":"9f9555b543dd7d0083b56266b19403629f3d7459858e6f043e0f32e7c4f7e521"} Dec 03 11:51:28 crc kubenswrapper[5052]: I1203 11:51:28.471098 5052 scope.go:117] "RemoveContainer" containerID="d6f5255145150903893d430668b91fdf726ecc26c094e89b13fdc75a3e249f5d" Dec 03 11:51:28 crc kubenswrapper[5052]: I1203 11:51:28.471271 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-api-67d5b9d8c8-4gw9z" Dec 03 11:51:28 crc kubenswrapper[5052]: I1203 11:51:28.497791 5052 scope.go:117] "RemoveContainer" containerID="61036e95ab93308bc10e8d596f78981c7d8a8b273993f37dcf6c5a34caac782a" Dec 03 11:51:28 crc kubenswrapper[5052]: I1203 11:51:28.518700 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-api-67d5b9d8c8-4gw9z"] Dec 03 11:51:28 crc kubenswrapper[5052]: I1203 11:51:28.526147 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/barbican-api-67d5b9d8c8-4gw9z"] Dec 03 11:51:29 crc kubenswrapper[5052]: I1203 11:51:29.683998 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2b36c173-6bfb-4fcc-858f-19e0ababbf95" path="/var/lib/kubelet/pods/2b36c173-6bfb-4fcc-858f-19e0ababbf95/volumes" Dec 03 11:51:30 crc kubenswrapper[5052]: I1203 11:51:30.874763 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:51:33 crc kubenswrapper[5052]: I1203 11:51:33.072453 5052 scope.go:117] "RemoveContainer" containerID="d26bbcf3d9f20787b3d596993e0924eb679af95f833778f1f8ad33c49224f4d8" Dec 03 11:51:33 crc kubenswrapper[5052]: I1203 11:51:33.094879 5052 scope.go:117] "RemoveContainer" containerID="08b743014f4f3e11912f7da330bbe8c095df94c7cd01737b812783a0bd8af17b" Dec 03 11:51:33 crc kubenswrapper[5052]: I1203 11:51:33.135263 5052 scope.go:117] "RemoveContainer" containerID="aaed7fd471350da406347ddf3e6da7df5351689cca457e45e0443f8f32533c5e" Dec 03 11:51:33 crc kubenswrapper[5052]: I1203 11:51:33.174736 5052 scope.go:117] "RemoveContainer" containerID="18fb8243e49f7c54143f6d0ab14bae1d97d986b3e3dbf996c8186febab430a50" Dec 03 11:51:33 crc kubenswrapper[5052]: I1203 11:51:33.197445 5052 scope.go:117] "RemoveContainer" containerID="30b3c8981cdec3a0ba91236f985bc49025e43effcb7cbf0b9e7cddcc3ba0a0d9" Dec 03 11:51:33 crc kubenswrapper[5052]: I1203 11:51:33.236273 5052 scope.go:117] "RemoveContainer" containerID="1b7ece73f44642bcedc96bdc0480c0038d10712c4f5cc1cf1e1c3d04d924e6f9" Dec 03 11:51:33 crc kubenswrapper[5052]: I1203 11:51:33.252271 5052 scope.go:117] "RemoveContainer" containerID="de182e92eb499a14ca6d41657fcf50d4c07d51d2e7062a139fcdbe3c0e220744" Dec 03 11:51:33 crc kubenswrapper[5052]: I1203 11:51:33.292488 5052 scope.go:117] "RemoveContainer" containerID="5bd53afa88afc24e4d88a692fa06eded59d58f9f609240bdbab89ff86313aa6d" Dec 03 11:51:33 crc kubenswrapper[5052]: I1203 11:51:33.337299 5052 scope.go:117] "RemoveContainer" containerID="4add9b48367f94bee74008ec0e801ddbd291eb262a94b430d5c2adfb689ac2b9" Dec 03 11:51:33 crc kubenswrapper[5052]: I1203 11:51:33.358947 5052 scope.go:117] "RemoveContainer" containerID="988a528826a2862aed42fdaece9c254222b91bc11952d4cd54b0b18c778396b2" Dec 03 11:51:33 crc kubenswrapper[5052]: I1203 11:51:33.380081 5052 scope.go:117] "RemoveContainer" containerID="ef839a7336e307d7216464809bb35eccbc476e002b4d08e14e48e0259172f844" Dec 03 11:51:34 crc kubenswrapper[5052]: I1203 11:51:34.378109 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/placement-5fc4755558-qz76x" Dec 03 11:51:34 crc kubenswrapper[5052]: I1203 11:51:34.379220 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/placement-5fc4755558-qz76x" Dec 03 11:51:41 crc kubenswrapper[5052]: I1203 11:51:41.316130 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/keystone-94f46848f-wgzvx" Dec 03 11:51:41 crc kubenswrapper[5052]: I1203 11:51:41.728484 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/openstackclient"] Dec 03 11:51:41 crc kubenswrapper[5052]: E1203 11:51:41.730553 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="2b36c173-6bfb-4fcc-858f-19e0ababbf95" containerName="barbican-api-log" Dec 03 11:51:41 crc kubenswrapper[5052]: I1203 11:51:41.730573 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="2b36c173-6bfb-4fcc-858f-19e0ababbf95" containerName="barbican-api-log" Dec 03 11:51:41 crc kubenswrapper[5052]: E1203 11:51:41.730600 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="2b36c173-6bfb-4fcc-858f-19e0ababbf95" containerName="barbican-api" Dec 03 11:51:41 crc kubenswrapper[5052]: I1203 11:51:41.730610 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="2b36c173-6bfb-4fcc-858f-19e0ababbf95" containerName="barbican-api" Dec 03 11:51:41 crc kubenswrapper[5052]: I1203 11:51:41.730791 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="2b36c173-6bfb-4fcc-858f-19e0ababbf95" containerName="barbican-api-log" Dec 03 11:51:41 crc kubenswrapper[5052]: I1203 11:51:41.730810 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="2b36c173-6bfb-4fcc-858f-19e0ababbf95" containerName="barbican-api" Dec 03 11:51:41 crc kubenswrapper[5052]: I1203 11:51:41.731528 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstackclient" Dec 03 11:51:41 crc kubenswrapper[5052]: I1203 11:51:41.735407 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"openstack-config-secret" Dec 03 11:51:41 crc kubenswrapper[5052]: I1203 11:51:41.735408 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"openstackclient-openstackclient-dockercfg-hk87k" Dec 03 11:51:41 crc kubenswrapper[5052]: I1203 11:51:41.735764 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"openstack-config" Dec 03 11:51:41 crc kubenswrapper[5052]: I1203 11:51:41.741304 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/openstackclient"] Dec 03 11:51:41 crc kubenswrapper[5052]: I1203 11:51:41.750439 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openstack-config\" (UniqueName: \"kubernetes.io/configmap/4c81b4dc-4f69-4447-8e5e-043aa0c7dd59-openstack-config\") pod \"openstackclient\" (UID: \"4c81b4dc-4f69-4447-8e5e-043aa0c7dd59\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 11:51:41 crc kubenswrapper[5052]: I1203 11:51:41.750758 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4c81b4dc-4f69-4447-8e5e-043aa0c7dd59-combined-ca-bundle\") pod \"openstackclient\" (UID: \"4c81b4dc-4f69-4447-8e5e-043aa0c7dd59\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 11:51:41 crc kubenswrapper[5052]: I1203 11:51:41.750935 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openstack-config-secret\" (UniqueName: \"kubernetes.io/secret/4c81b4dc-4f69-4447-8e5e-043aa0c7dd59-openstack-config-secret\") pod \"openstackclient\" (UID: \"4c81b4dc-4f69-4447-8e5e-043aa0c7dd59\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 11:51:41 crc kubenswrapper[5052]: I1203 11:51:41.751058 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-68s8l\" (UniqueName: \"kubernetes.io/projected/4c81b4dc-4f69-4447-8e5e-043aa0c7dd59-kube-api-access-68s8l\") pod \"openstackclient\" (UID: \"4c81b4dc-4f69-4447-8e5e-043aa0c7dd59\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 11:51:41 crc kubenswrapper[5052]: I1203 11:51:41.853171 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4c81b4dc-4f69-4447-8e5e-043aa0c7dd59-combined-ca-bundle\") pod \"openstackclient\" (UID: \"4c81b4dc-4f69-4447-8e5e-043aa0c7dd59\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 11:51:41 crc kubenswrapper[5052]: I1203 11:51:41.853275 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"openstack-config-secret\" (UniqueName: \"kubernetes.io/secret/4c81b4dc-4f69-4447-8e5e-043aa0c7dd59-openstack-config-secret\") pod \"openstackclient\" (UID: \"4c81b4dc-4f69-4447-8e5e-043aa0c7dd59\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 11:51:41 crc kubenswrapper[5052]: I1203 11:51:41.853329 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-68s8l\" (UniqueName: \"kubernetes.io/projected/4c81b4dc-4f69-4447-8e5e-043aa0c7dd59-kube-api-access-68s8l\") pod \"openstackclient\" (UID: \"4c81b4dc-4f69-4447-8e5e-043aa0c7dd59\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 11:51:41 crc kubenswrapper[5052]: I1203 11:51:41.853363 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"openstack-config\" (UniqueName: \"kubernetes.io/configmap/4c81b4dc-4f69-4447-8e5e-043aa0c7dd59-openstack-config\") pod \"openstackclient\" (UID: \"4c81b4dc-4f69-4447-8e5e-043aa0c7dd59\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 11:51:41 crc kubenswrapper[5052]: I1203 11:51:41.854656 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"openstack-config\" (UniqueName: \"kubernetes.io/configmap/4c81b4dc-4f69-4447-8e5e-043aa0c7dd59-openstack-config\") pod \"openstackclient\" (UID: \"4c81b4dc-4f69-4447-8e5e-043aa0c7dd59\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 11:51:41 crc kubenswrapper[5052]: I1203 11:51:41.861224 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"openstack-config-secret\" (UniqueName: \"kubernetes.io/secret/4c81b4dc-4f69-4447-8e5e-043aa0c7dd59-openstack-config-secret\") pod \"openstackclient\" (UID: \"4c81b4dc-4f69-4447-8e5e-043aa0c7dd59\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 11:51:41 crc kubenswrapper[5052]: I1203 11:51:41.871861 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4c81b4dc-4f69-4447-8e5e-043aa0c7dd59-combined-ca-bundle\") pod \"openstackclient\" (UID: \"4c81b4dc-4f69-4447-8e5e-043aa0c7dd59\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 11:51:41 crc kubenswrapper[5052]: I1203 11:51:41.874460 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-68s8l\" (UniqueName: \"kubernetes.io/projected/4c81b4dc-4f69-4447-8e5e-043aa0c7dd59-kube-api-access-68s8l\") pod \"openstackclient\" (UID: \"4c81b4dc-4f69-4447-8e5e-043aa0c7dd59\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 11:51:42 crc kubenswrapper[5052]: I1203 11:51:42.054032 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstackclient" Dec 03 11:51:42 crc kubenswrapper[5052]: I1203 11:51:42.517686 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/openstackclient"] Dec 03 11:51:42 crc kubenswrapper[5052]: W1203 11:51:42.517755 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4c81b4dc_4f69_4447_8e5e_043aa0c7dd59.slice/crio-6131cd0a2609f5f05b97af6f0d0cdfbfc3375ed10650db4475efc03c4c130825 WatchSource:0}: Error finding container 6131cd0a2609f5f05b97af6f0d0cdfbfc3375ed10650db4475efc03c4c130825: Status 404 returned error can't find the container with id 6131cd0a2609f5f05b97af6f0d0cdfbfc3375ed10650db4475efc03c4c130825 Dec 03 11:51:42 crc kubenswrapper[5052]: I1203 11:51:42.605733 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstackclient" event={"ID":"4c81b4dc-4f69-4447-8e5e-043aa0c7dd59","Type":"ContainerStarted","Data":"6131cd0a2609f5f05b97af6f0d0cdfbfc3375ed10650db4475efc03c4c130825"} Dec 03 11:51:43 crc kubenswrapper[5052]: I1203 11:51:43.615444 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstackclient" event={"ID":"4c81b4dc-4f69-4447-8e5e-043aa0c7dd59","Type":"ContainerStarted","Data":"d40e529ae6719c4ad0db1e286623449a547ab4530cef62b4eede6892bcc97992"} Dec 03 11:51:43 crc kubenswrapper[5052]: I1203 11:51:43.634184 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/openstackclient" podStartSLOduration=2.634168143 podStartE2EDuration="2.634168143s" podCreationTimestamp="2025-12-03 11:51:41 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:51:43.631541148 +0000 UTC m=+1894.219782912" watchObservedRunningTime="2025-12-03 11:51:43.634168143 +0000 UTC m=+1894.222409907" Dec 03 11:51:46 crc kubenswrapper[5052]: I1203 11:51:46.089786 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:51:46 crc kubenswrapper[5052]: I1203 11:51:46.090629 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="f8c1750d-cf4a-49cc-a5ba-d40f19a6aad8" containerName="ceilometer-central-agent" containerID="cri-o://eb33c766c34b7def659e87f673ae82230a201d258f7a88dee07749f3cbd8d1e0" gracePeriod=30 Dec 03 11:51:46 crc kubenswrapper[5052]: I1203 11:51:46.090686 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="f8c1750d-cf4a-49cc-a5ba-d40f19a6aad8" containerName="proxy-httpd" containerID="cri-o://2807aed7a69b1fdeca76bc8aeda707467e8f044feefef396c4c3ef5f25917a3b" gracePeriod=30 Dec 03 11:51:46 crc kubenswrapper[5052]: I1203 11:51:46.090750 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="f8c1750d-cf4a-49cc-a5ba-d40f19a6aad8" containerName="ceilometer-notification-agent" containerID="cri-o://8f596e1b0a79e1f1f823327015afe83ef471316845e5e0fd8516f51ce4c105c2" gracePeriod=30 Dec 03 11:51:46 crc kubenswrapper[5052]: I1203 11:51:46.090754 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="f8c1750d-cf4a-49cc-a5ba-d40f19a6aad8" containerName="sg-core" containerID="cri-o://152e33660b972abe070d51c7baf270cd5dab59b3cbe27ab26e323e676c508393" gracePeriod=30 Dec 03 11:51:46 crc kubenswrapper[5052]: I1203 11:51:46.113617 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/swift-proxy-b9d8f996b-brs7h"] Dec 03 11:51:46 crc kubenswrapper[5052]: I1203 11:51:46.115403 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-proxy-b9d8f996b-brs7h" Dec 03 11:51:46 crc kubenswrapper[5052]: I1203 11:51:46.117426 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-swift-internal-svc" Dec 03 11:51:46 crc kubenswrapper[5052]: I1203 11:51:46.117967 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"swift-proxy-config-data" Dec 03 11:51:46 crc kubenswrapper[5052]: I1203 11:51:46.119074 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-swift-public-svc" Dec 03 11:51:46 crc kubenswrapper[5052]: I1203 11:51:46.128489 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/swift-proxy-b9d8f996b-brs7h"] Dec 03 11:51:46 crc kubenswrapper[5052]: I1203 11:51:46.233768 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/036bc2b0-bc50-4c91-bea5-882b9e592e4f-etc-swift\") pod \"swift-proxy-b9d8f996b-brs7h\" (UID: \"036bc2b0-bc50-4c91-bea5-882b9e592e4f\") " pod="openstack-kuttl-tests/swift-proxy-b9d8f996b-brs7h" Dec 03 11:51:46 crc kubenswrapper[5052]: I1203 11:51:46.233823 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/036bc2b0-bc50-4c91-bea5-882b9e592e4f-combined-ca-bundle\") pod \"swift-proxy-b9d8f996b-brs7h\" (UID: \"036bc2b0-bc50-4c91-bea5-882b9e592e4f\") " pod="openstack-kuttl-tests/swift-proxy-b9d8f996b-brs7h" Dec 03 11:51:46 crc kubenswrapper[5052]: I1203 11:51:46.233884 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f8hmk\" (UniqueName: \"kubernetes.io/projected/036bc2b0-bc50-4c91-bea5-882b9e592e4f-kube-api-access-f8hmk\") pod \"swift-proxy-b9d8f996b-brs7h\" (UID: \"036bc2b0-bc50-4c91-bea5-882b9e592e4f\") " pod="openstack-kuttl-tests/swift-proxy-b9d8f996b-brs7h" Dec 03 11:51:46 crc kubenswrapper[5052]: I1203 11:51:46.234407 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/036bc2b0-bc50-4c91-bea5-882b9e592e4f-public-tls-certs\") pod \"swift-proxy-b9d8f996b-brs7h\" (UID: \"036bc2b0-bc50-4c91-bea5-882b9e592e4f\") " pod="openstack-kuttl-tests/swift-proxy-b9d8f996b-brs7h" Dec 03 11:51:46 crc kubenswrapper[5052]: I1203 11:51:46.234764 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/036bc2b0-bc50-4c91-bea5-882b9e592e4f-internal-tls-certs\") pod \"swift-proxy-b9d8f996b-brs7h\" (UID: \"036bc2b0-bc50-4c91-bea5-882b9e592e4f\") " pod="openstack-kuttl-tests/swift-proxy-b9d8f996b-brs7h" Dec 03 11:51:46 crc kubenswrapper[5052]: I1203 11:51:46.234874 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/036bc2b0-bc50-4c91-bea5-882b9e592e4f-config-data\") pod \"swift-proxy-b9d8f996b-brs7h\" (UID: \"036bc2b0-bc50-4c91-bea5-882b9e592e4f\") " pod="openstack-kuttl-tests/swift-proxy-b9d8f996b-brs7h" Dec 03 11:51:46 crc kubenswrapper[5052]: I1203 11:51:46.234914 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/036bc2b0-bc50-4c91-bea5-882b9e592e4f-log-httpd\") pod \"swift-proxy-b9d8f996b-brs7h\" (UID: \"036bc2b0-bc50-4c91-bea5-882b9e592e4f\") " pod="openstack-kuttl-tests/swift-proxy-b9d8f996b-brs7h" Dec 03 11:51:46 crc kubenswrapper[5052]: I1203 11:51:46.234972 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/036bc2b0-bc50-4c91-bea5-882b9e592e4f-run-httpd\") pod \"swift-proxy-b9d8f996b-brs7h\" (UID: \"036bc2b0-bc50-4c91-bea5-882b9e592e4f\") " pod="openstack-kuttl-tests/swift-proxy-b9d8f996b-brs7h" Dec 03 11:51:46 crc kubenswrapper[5052]: I1203 11:51:46.336378 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/036bc2b0-bc50-4c91-bea5-882b9e592e4f-etc-swift\") pod \"swift-proxy-b9d8f996b-brs7h\" (UID: \"036bc2b0-bc50-4c91-bea5-882b9e592e4f\") " pod="openstack-kuttl-tests/swift-proxy-b9d8f996b-brs7h" Dec 03 11:51:46 crc kubenswrapper[5052]: I1203 11:51:46.336431 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/036bc2b0-bc50-4c91-bea5-882b9e592e4f-combined-ca-bundle\") pod \"swift-proxy-b9d8f996b-brs7h\" (UID: \"036bc2b0-bc50-4c91-bea5-882b9e592e4f\") " pod="openstack-kuttl-tests/swift-proxy-b9d8f996b-brs7h" Dec 03 11:51:46 crc kubenswrapper[5052]: I1203 11:51:46.336455 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-f8hmk\" (UniqueName: \"kubernetes.io/projected/036bc2b0-bc50-4c91-bea5-882b9e592e4f-kube-api-access-f8hmk\") pod \"swift-proxy-b9d8f996b-brs7h\" (UID: \"036bc2b0-bc50-4c91-bea5-882b9e592e4f\") " pod="openstack-kuttl-tests/swift-proxy-b9d8f996b-brs7h" Dec 03 11:51:46 crc kubenswrapper[5052]: I1203 11:51:46.336548 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/036bc2b0-bc50-4c91-bea5-882b9e592e4f-public-tls-certs\") pod \"swift-proxy-b9d8f996b-brs7h\" (UID: \"036bc2b0-bc50-4c91-bea5-882b9e592e4f\") " pod="openstack-kuttl-tests/swift-proxy-b9d8f996b-brs7h" Dec 03 11:51:46 crc kubenswrapper[5052]: I1203 11:51:46.336614 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/036bc2b0-bc50-4c91-bea5-882b9e592e4f-internal-tls-certs\") pod \"swift-proxy-b9d8f996b-brs7h\" (UID: \"036bc2b0-bc50-4c91-bea5-882b9e592e4f\") " pod="openstack-kuttl-tests/swift-proxy-b9d8f996b-brs7h" Dec 03 11:51:46 crc kubenswrapper[5052]: I1203 11:51:46.336641 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/036bc2b0-bc50-4c91-bea5-882b9e592e4f-config-data\") pod \"swift-proxy-b9d8f996b-brs7h\" (UID: \"036bc2b0-bc50-4c91-bea5-882b9e592e4f\") " pod="openstack-kuttl-tests/swift-proxy-b9d8f996b-brs7h" Dec 03 11:51:46 crc kubenswrapper[5052]: I1203 11:51:46.336660 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/036bc2b0-bc50-4c91-bea5-882b9e592e4f-log-httpd\") pod \"swift-proxy-b9d8f996b-brs7h\" (UID: \"036bc2b0-bc50-4c91-bea5-882b9e592e4f\") " pod="openstack-kuttl-tests/swift-proxy-b9d8f996b-brs7h" Dec 03 11:51:46 crc kubenswrapper[5052]: I1203 11:51:46.336703 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/036bc2b0-bc50-4c91-bea5-882b9e592e4f-run-httpd\") pod \"swift-proxy-b9d8f996b-brs7h\" (UID: \"036bc2b0-bc50-4c91-bea5-882b9e592e4f\") " pod="openstack-kuttl-tests/swift-proxy-b9d8f996b-brs7h" Dec 03 11:51:46 crc kubenswrapper[5052]: I1203 11:51:46.337399 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/036bc2b0-bc50-4c91-bea5-882b9e592e4f-run-httpd\") pod \"swift-proxy-b9d8f996b-brs7h\" (UID: \"036bc2b0-bc50-4c91-bea5-882b9e592e4f\") " pod="openstack-kuttl-tests/swift-proxy-b9d8f996b-brs7h" Dec 03 11:51:46 crc kubenswrapper[5052]: I1203 11:51:46.337842 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/036bc2b0-bc50-4c91-bea5-882b9e592e4f-log-httpd\") pod \"swift-proxy-b9d8f996b-brs7h\" (UID: \"036bc2b0-bc50-4c91-bea5-882b9e592e4f\") " pod="openstack-kuttl-tests/swift-proxy-b9d8f996b-brs7h" Dec 03 11:51:46 crc kubenswrapper[5052]: I1203 11:51:46.343474 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/036bc2b0-bc50-4c91-bea5-882b9e592e4f-public-tls-certs\") pod \"swift-proxy-b9d8f996b-brs7h\" (UID: \"036bc2b0-bc50-4c91-bea5-882b9e592e4f\") " pod="openstack-kuttl-tests/swift-proxy-b9d8f996b-brs7h" Dec 03 11:51:46 crc kubenswrapper[5052]: I1203 11:51:46.343899 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/036bc2b0-bc50-4c91-bea5-882b9e592e4f-config-data\") pod \"swift-proxy-b9d8f996b-brs7h\" (UID: \"036bc2b0-bc50-4c91-bea5-882b9e592e4f\") " pod="openstack-kuttl-tests/swift-proxy-b9d8f996b-brs7h" Dec 03 11:51:46 crc kubenswrapper[5052]: I1203 11:51:46.344062 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/036bc2b0-bc50-4c91-bea5-882b9e592e4f-internal-tls-certs\") pod \"swift-proxy-b9d8f996b-brs7h\" (UID: \"036bc2b0-bc50-4c91-bea5-882b9e592e4f\") " pod="openstack-kuttl-tests/swift-proxy-b9d8f996b-brs7h" Dec 03 11:51:46 crc kubenswrapper[5052]: I1203 11:51:46.345706 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/036bc2b0-bc50-4c91-bea5-882b9e592e4f-etc-swift\") pod \"swift-proxy-b9d8f996b-brs7h\" (UID: \"036bc2b0-bc50-4c91-bea5-882b9e592e4f\") " pod="openstack-kuttl-tests/swift-proxy-b9d8f996b-brs7h" Dec 03 11:51:46 crc kubenswrapper[5052]: I1203 11:51:46.347328 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/036bc2b0-bc50-4c91-bea5-882b9e592e4f-combined-ca-bundle\") pod \"swift-proxy-b9d8f996b-brs7h\" (UID: \"036bc2b0-bc50-4c91-bea5-882b9e592e4f\") " pod="openstack-kuttl-tests/swift-proxy-b9d8f996b-brs7h" Dec 03 11:51:46 crc kubenswrapper[5052]: I1203 11:51:46.357840 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-f8hmk\" (UniqueName: \"kubernetes.io/projected/036bc2b0-bc50-4c91-bea5-882b9e592e4f-kube-api-access-f8hmk\") pod \"swift-proxy-b9d8f996b-brs7h\" (UID: \"036bc2b0-bc50-4c91-bea5-882b9e592e4f\") " pod="openstack-kuttl-tests/swift-proxy-b9d8f996b-brs7h" Dec 03 11:51:46 crc kubenswrapper[5052]: I1203 11:51:46.450347 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-proxy-b9d8f996b-brs7h" Dec 03 11:51:46 crc kubenswrapper[5052]: I1203 11:51:46.648375 5052 generic.go:334] "Generic (PLEG): container finished" podID="f8c1750d-cf4a-49cc-a5ba-d40f19a6aad8" containerID="2807aed7a69b1fdeca76bc8aeda707467e8f044feefef396c4c3ef5f25917a3b" exitCode=0 Dec 03 11:51:46 crc kubenswrapper[5052]: I1203 11:51:46.648402 5052 generic.go:334] "Generic (PLEG): container finished" podID="f8c1750d-cf4a-49cc-a5ba-d40f19a6aad8" containerID="152e33660b972abe070d51c7baf270cd5dab59b3cbe27ab26e323e676c508393" exitCode=2 Dec 03 11:51:46 crc kubenswrapper[5052]: I1203 11:51:46.648423 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"f8c1750d-cf4a-49cc-a5ba-d40f19a6aad8","Type":"ContainerDied","Data":"2807aed7a69b1fdeca76bc8aeda707467e8f044feefef396c4c3ef5f25917a3b"} Dec 03 11:51:46 crc kubenswrapper[5052]: I1203 11:51:46.648449 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"f8c1750d-cf4a-49cc-a5ba-d40f19a6aad8","Type":"ContainerDied","Data":"152e33660b972abe070d51c7baf270cd5dab59b3cbe27ab26e323e676c508393"} Dec 03 11:51:46 crc kubenswrapper[5052]: I1203 11:51:46.874554 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/swift-proxy-b9d8f996b-brs7h"] Dec 03 11:51:47 crc kubenswrapper[5052]: I1203 11:51:47.592501 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-api-db-create-4fs9m"] Dec 03 11:51:47 crc kubenswrapper[5052]: I1203 11:51:47.594381 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-db-create-4fs9m"] Dec 03 11:51:47 crc kubenswrapper[5052]: I1203 11:51:47.594465 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-db-create-4fs9m" Dec 03 11:51:47 crc kubenswrapper[5052]: I1203 11:51:47.659708 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-proxy-b9d8f996b-brs7h" event={"ID":"036bc2b0-bc50-4c91-bea5-882b9e592e4f","Type":"ContainerStarted","Data":"b83ee0552872f35b33c41234c7f87cdaa58debb1beeb08c6e6d9b6f3a591cb00"} Dec 03 11:51:47 crc kubenswrapper[5052]: I1203 11:51:47.659959 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-proxy-b9d8f996b-brs7h" event={"ID":"036bc2b0-bc50-4c91-bea5-882b9e592e4f","Type":"ContainerStarted","Data":"f0b09f86c71e6c4c0eeb418caafc02a9fd15b0cade1aeff34665731a0a629c87"} Dec 03 11:51:47 crc kubenswrapper[5052]: I1203 11:51:47.662160 5052 generic.go:334] "Generic (PLEG): container finished" podID="f8c1750d-cf4a-49cc-a5ba-d40f19a6aad8" containerID="eb33c766c34b7def659e87f673ae82230a201d258f7a88dee07749f3cbd8d1e0" exitCode=0 Dec 03 11:51:47 crc kubenswrapper[5052]: I1203 11:51:47.662201 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"f8c1750d-cf4a-49cc-a5ba-d40f19a6aad8","Type":"ContainerDied","Data":"eb33c766c34b7def659e87f673ae82230a201d258f7a88dee07749f3cbd8d1e0"} Dec 03 11:51:47 crc kubenswrapper[5052]: I1203 11:51:47.670675 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/c1c20a0e-ee8a-416e-b739-05f16a536cfa-operator-scripts\") pod \"nova-api-db-create-4fs9m\" (UID: \"c1c20a0e-ee8a-416e-b739-05f16a536cfa\") " pod="openstack-kuttl-tests/nova-api-db-create-4fs9m" Dec 03 11:51:47 crc kubenswrapper[5052]: I1203 11:51:47.670870 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5v2k2\" (UniqueName: \"kubernetes.io/projected/c1c20a0e-ee8a-416e-b739-05f16a536cfa-kube-api-access-5v2k2\") pod \"nova-api-db-create-4fs9m\" (UID: \"c1c20a0e-ee8a-416e-b739-05f16a536cfa\") " pod="openstack-kuttl-tests/nova-api-db-create-4fs9m" Dec 03 11:51:47 crc kubenswrapper[5052]: I1203 11:51:47.749797 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell0-db-create-42nm2"] Dec 03 11:51:47 crc kubenswrapper[5052]: I1203 11:51:47.751768 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-db-create-42nm2" Dec 03 11:51:47 crc kubenswrapper[5052]: I1203 11:51:47.768581 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell0-db-create-42nm2"] Dec 03 11:51:47 crc kubenswrapper[5052]: I1203 11:51:47.772491 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/c1c20a0e-ee8a-416e-b739-05f16a536cfa-operator-scripts\") pod \"nova-api-db-create-4fs9m\" (UID: \"c1c20a0e-ee8a-416e-b739-05f16a536cfa\") " pod="openstack-kuttl-tests/nova-api-db-create-4fs9m" Dec 03 11:51:47 crc kubenswrapper[5052]: I1203 11:51:47.772672 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-5v2k2\" (UniqueName: \"kubernetes.io/projected/c1c20a0e-ee8a-416e-b739-05f16a536cfa-kube-api-access-5v2k2\") pod \"nova-api-db-create-4fs9m\" (UID: \"c1c20a0e-ee8a-416e-b739-05f16a536cfa\") " pod="openstack-kuttl-tests/nova-api-db-create-4fs9m" Dec 03 11:51:47 crc kubenswrapper[5052]: I1203 11:51:47.773945 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/c1c20a0e-ee8a-416e-b739-05f16a536cfa-operator-scripts\") pod \"nova-api-db-create-4fs9m\" (UID: \"c1c20a0e-ee8a-416e-b739-05f16a536cfa\") " pod="openstack-kuttl-tests/nova-api-db-create-4fs9m" Dec 03 11:51:47 crc kubenswrapper[5052]: I1203 11:51:47.781588 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-api-e4da-account-create-update-rqlfw"] Dec 03 11:51:47 crc kubenswrapper[5052]: I1203 11:51:47.783069 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-e4da-account-create-update-rqlfw" Dec 03 11:51:47 crc kubenswrapper[5052]: I1203 11:51:47.786078 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-api-db-secret" Dec 03 11:51:47 crc kubenswrapper[5052]: I1203 11:51:47.810160 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-5v2k2\" (UniqueName: \"kubernetes.io/projected/c1c20a0e-ee8a-416e-b739-05f16a536cfa-kube-api-access-5v2k2\") pod \"nova-api-db-create-4fs9m\" (UID: \"c1c20a0e-ee8a-416e-b739-05f16a536cfa\") " pod="openstack-kuttl-tests/nova-api-db-create-4fs9m" Dec 03 11:51:47 crc kubenswrapper[5052]: I1203 11:51:47.810228 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-e4da-account-create-update-rqlfw"] Dec 03 11:51:47 crc kubenswrapper[5052]: I1203 11:51:47.858221 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell1-db-create-8vmt9"] Dec 03 11:51:47 crc kubenswrapper[5052]: I1203 11:51:47.859577 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-db-create-8vmt9" Dec 03 11:51:47 crc kubenswrapper[5052]: I1203 11:51:47.869648 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-db-create-8vmt9"] Dec 03 11:51:47 crc kubenswrapper[5052]: I1203 11:51:47.874458 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-62tjt\" (UniqueName: \"kubernetes.io/projected/a4aab9e3-5790-4143-948b-e24a52e552a5-kube-api-access-62tjt\") pod \"nova-cell0-db-create-42nm2\" (UID: \"a4aab9e3-5790-4143-948b-e24a52e552a5\") " pod="openstack-kuttl-tests/nova-cell0-db-create-42nm2" Dec 03 11:51:47 crc kubenswrapper[5052]: I1203 11:51:47.874790 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/a4aab9e3-5790-4143-948b-e24a52e552a5-operator-scripts\") pod \"nova-cell0-db-create-42nm2\" (UID: \"a4aab9e3-5790-4143-948b-e24a52e552a5\") " pod="openstack-kuttl-tests/nova-cell0-db-create-42nm2" Dec 03 11:51:47 crc kubenswrapper[5052]: I1203 11:51:47.932241 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-db-create-4fs9m" Dec 03 11:51:47 crc kubenswrapper[5052]: I1203 11:51:47.968741 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell0-745f-account-create-update-5v2dr"] Dec 03 11:51:47 crc kubenswrapper[5052]: I1203 11:51:47.972624 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-745f-account-create-update-5v2dr" Dec 03 11:51:47 crc kubenswrapper[5052]: I1203 11:51:47.982775 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6fjp6\" (UniqueName: \"kubernetes.io/projected/80d855f5-afda-46c0-9d72-01dee8dd93fb-kube-api-access-6fjp6\") pod \"nova-api-e4da-account-create-update-rqlfw\" (UID: \"80d855f5-afda-46c0-9d72-01dee8dd93fb\") " pod="openstack-kuttl-tests/nova-api-e4da-account-create-update-rqlfw" Dec 03 11:51:47 crc kubenswrapper[5052]: I1203 11:51:47.983011 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p5p88\" (UniqueName: \"kubernetes.io/projected/18c1240f-c951-40de-a0da-3b50f11f4cf1-kube-api-access-p5p88\") pod \"nova-cell1-db-create-8vmt9\" (UID: \"18c1240f-c951-40de-a0da-3b50f11f4cf1\") " pod="openstack-kuttl-tests/nova-cell1-db-create-8vmt9" Dec 03 11:51:47 crc kubenswrapper[5052]: I1203 11:51:47.983104 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-62tjt\" (UniqueName: \"kubernetes.io/projected/a4aab9e3-5790-4143-948b-e24a52e552a5-kube-api-access-62tjt\") pod \"nova-cell0-db-create-42nm2\" (UID: \"a4aab9e3-5790-4143-948b-e24a52e552a5\") " pod="openstack-kuttl-tests/nova-cell0-db-create-42nm2" Dec 03 11:51:47 crc kubenswrapper[5052]: I1203 11:51:47.983222 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/18c1240f-c951-40de-a0da-3b50f11f4cf1-operator-scripts\") pod \"nova-cell1-db-create-8vmt9\" (UID: \"18c1240f-c951-40de-a0da-3b50f11f4cf1\") " pod="openstack-kuttl-tests/nova-cell1-db-create-8vmt9" Dec 03 11:51:47 crc kubenswrapper[5052]: I1203 11:51:47.983317 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/a4aab9e3-5790-4143-948b-e24a52e552a5-operator-scripts\") pod \"nova-cell0-db-create-42nm2\" (UID: \"a4aab9e3-5790-4143-948b-e24a52e552a5\") " pod="openstack-kuttl-tests/nova-cell0-db-create-42nm2" Dec 03 11:51:47 crc kubenswrapper[5052]: I1203 11:51:47.983395 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/80d855f5-afda-46c0-9d72-01dee8dd93fb-operator-scripts\") pod \"nova-api-e4da-account-create-update-rqlfw\" (UID: \"80d855f5-afda-46c0-9d72-01dee8dd93fb\") " pod="openstack-kuttl-tests/nova-api-e4da-account-create-update-rqlfw" Dec 03 11:51:47 crc kubenswrapper[5052]: I1203 11:51:47.984420 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/a4aab9e3-5790-4143-948b-e24a52e552a5-operator-scripts\") pod \"nova-cell0-db-create-42nm2\" (UID: \"a4aab9e3-5790-4143-948b-e24a52e552a5\") " pod="openstack-kuttl-tests/nova-cell0-db-create-42nm2" Dec 03 11:51:47 crc kubenswrapper[5052]: I1203 11:51:47.985198 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell0-db-secret" Dec 03 11:51:48 crc kubenswrapper[5052]: I1203 11:51:48.001927 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell0-745f-account-create-update-5v2dr"] Dec 03 11:51:48 crc kubenswrapper[5052]: I1203 11:51:48.005908 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-62tjt\" (UniqueName: \"kubernetes.io/projected/a4aab9e3-5790-4143-948b-e24a52e552a5-kube-api-access-62tjt\") pod \"nova-cell0-db-create-42nm2\" (UID: \"a4aab9e3-5790-4143-948b-e24a52e552a5\") " pod="openstack-kuttl-tests/nova-cell0-db-create-42nm2" Dec 03 11:51:48 crc kubenswrapper[5052]: I1203 11:51:48.075590 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell1-4fec-account-create-update-x8rjw"] Dec 03 11:51:48 crc kubenswrapper[5052]: I1203 11:51:48.077083 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-4fec-account-create-update-x8rjw" Dec 03 11:51:48 crc kubenswrapper[5052]: I1203 11:51:48.078830 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-db-create-42nm2" Dec 03 11:51:48 crc kubenswrapper[5052]: I1203 11:51:48.080282 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell1-db-secret" Dec 03 11:51:48 crc kubenswrapper[5052]: I1203 11:51:48.084464 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/fa84c362-f076-4b47-8d2b-9eadd3f2d10c-operator-scripts\") pod \"nova-cell0-745f-account-create-update-5v2dr\" (UID: \"fa84c362-f076-4b47-8d2b-9eadd3f2d10c\") " pod="openstack-kuttl-tests/nova-cell0-745f-account-create-update-5v2dr" Dec 03 11:51:48 crc kubenswrapper[5052]: I1203 11:51:48.084538 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/80d855f5-afda-46c0-9d72-01dee8dd93fb-operator-scripts\") pod \"nova-api-e4da-account-create-update-rqlfw\" (UID: \"80d855f5-afda-46c0-9d72-01dee8dd93fb\") " pod="openstack-kuttl-tests/nova-api-e4da-account-create-update-rqlfw" Dec 03 11:51:48 crc kubenswrapper[5052]: I1203 11:51:48.084570 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-6fjp6\" (UniqueName: \"kubernetes.io/projected/80d855f5-afda-46c0-9d72-01dee8dd93fb-kube-api-access-6fjp6\") pod \"nova-api-e4da-account-create-update-rqlfw\" (UID: \"80d855f5-afda-46c0-9d72-01dee8dd93fb\") " pod="openstack-kuttl-tests/nova-api-e4da-account-create-update-rqlfw" Dec 03 11:51:48 crc kubenswrapper[5052]: I1203 11:51:48.084600 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jw7pd\" (UniqueName: \"kubernetes.io/projected/fa84c362-f076-4b47-8d2b-9eadd3f2d10c-kube-api-access-jw7pd\") pod \"nova-cell0-745f-account-create-update-5v2dr\" (UID: \"fa84c362-f076-4b47-8d2b-9eadd3f2d10c\") " pod="openstack-kuttl-tests/nova-cell0-745f-account-create-update-5v2dr" Dec 03 11:51:48 crc kubenswrapper[5052]: I1203 11:51:48.084650 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-p5p88\" (UniqueName: \"kubernetes.io/projected/18c1240f-c951-40de-a0da-3b50f11f4cf1-kube-api-access-p5p88\") pod \"nova-cell1-db-create-8vmt9\" (UID: \"18c1240f-c951-40de-a0da-3b50f11f4cf1\") " pod="openstack-kuttl-tests/nova-cell1-db-create-8vmt9" Dec 03 11:51:48 crc kubenswrapper[5052]: I1203 11:51:48.084748 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/18c1240f-c951-40de-a0da-3b50f11f4cf1-operator-scripts\") pod \"nova-cell1-db-create-8vmt9\" (UID: \"18c1240f-c951-40de-a0da-3b50f11f4cf1\") " pod="openstack-kuttl-tests/nova-cell1-db-create-8vmt9" Dec 03 11:51:48 crc kubenswrapper[5052]: I1203 11:51:48.085599 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/18c1240f-c951-40de-a0da-3b50f11f4cf1-operator-scripts\") pod \"nova-cell1-db-create-8vmt9\" (UID: \"18c1240f-c951-40de-a0da-3b50f11f4cf1\") " pod="openstack-kuttl-tests/nova-cell1-db-create-8vmt9" Dec 03 11:51:48 crc kubenswrapper[5052]: I1203 11:51:48.086265 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/80d855f5-afda-46c0-9d72-01dee8dd93fb-operator-scripts\") pod \"nova-api-e4da-account-create-update-rqlfw\" (UID: \"80d855f5-afda-46c0-9d72-01dee8dd93fb\") " pod="openstack-kuttl-tests/nova-api-e4da-account-create-update-rqlfw" Dec 03 11:51:48 crc kubenswrapper[5052]: I1203 11:51:48.092307 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-4fec-account-create-update-x8rjw"] Dec 03 11:51:48 crc kubenswrapper[5052]: I1203 11:51:48.109408 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-p5p88\" (UniqueName: \"kubernetes.io/projected/18c1240f-c951-40de-a0da-3b50f11f4cf1-kube-api-access-p5p88\") pod \"nova-cell1-db-create-8vmt9\" (UID: \"18c1240f-c951-40de-a0da-3b50f11f4cf1\") " pod="openstack-kuttl-tests/nova-cell1-db-create-8vmt9" Dec 03 11:51:48 crc kubenswrapper[5052]: I1203 11:51:48.116018 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-6fjp6\" (UniqueName: \"kubernetes.io/projected/80d855f5-afda-46c0-9d72-01dee8dd93fb-kube-api-access-6fjp6\") pod \"nova-api-e4da-account-create-update-rqlfw\" (UID: \"80d855f5-afda-46c0-9d72-01dee8dd93fb\") " pod="openstack-kuttl-tests/nova-api-e4da-account-create-update-rqlfw" Dec 03 11:51:48 crc kubenswrapper[5052]: I1203 11:51:48.181255 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-e4da-account-create-update-rqlfw" Dec 03 11:51:48 crc kubenswrapper[5052]: I1203 11:51:48.186315 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/cca0de9c-68fc-44d4-b6b2-2be05e5d9995-operator-scripts\") pod \"nova-cell1-4fec-account-create-update-x8rjw\" (UID: \"cca0de9c-68fc-44d4-b6b2-2be05e5d9995\") " pod="openstack-kuttl-tests/nova-cell1-4fec-account-create-update-x8rjw" Dec 03 11:51:48 crc kubenswrapper[5052]: I1203 11:51:48.186399 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-jw7pd\" (UniqueName: \"kubernetes.io/projected/fa84c362-f076-4b47-8d2b-9eadd3f2d10c-kube-api-access-jw7pd\") pod \"nova-cell0-745f-account-create-update-5v2dr\" (UID: \"fa84c362-f076-4b47-8d2b-9eadd3f2d10c\") " pod="openstack-kuttl-tests/nova-cell0-745f-account-create-update-5v2dr" Dec 03 11:51:48 crc kubenswrapper[5052]: I1203 11:51:48.186564 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vmz62\" (UniqueName: \"kubernetes.io/projected/cca0de9c-68fc-44d4-b6b2-2be05e5d9995-kube-api-access-vmz62\") pod \"nova-cell1-4fec-account-create-update-x8rjw\" (UID: \"cca0de9c-68fc-44d4-b6b2-2be05e5d9995\") " pod="openstack-kuttl-tests/nova-cell1-4fec-account-create-update-x8rjw" Dec 03 11:51:48 crc kubenswrapper[5052]: I1203 11:51:48.186605 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/fa84c362-f076-4b47-8d2b-9eadd3f2d10c-operator-scripts\") pod \"nova-cell0-745f-account-create-update-5v2dr\" (UID: \"fa84c362-f076-4b47-8d2b-9eadd3f2d10c\") " pod="openstack-kuttl-tests/nova-cell0-745f-account-create-update-5v2dr" Dec 03 11:51:48 crc kubenswrapper[5052]: I1203 11:51:48.187210 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/fa84c362-f076-4b47-8d2b-9eadd3f2d10c-operator-scripts\") pod \"nova-cell0-745f-account-create-update-5v2dr\" (UID: \"fa84c362-f076-4b47-8d2b-9eadd3f2d10c\") " pod="openstack-kuttl-tests/nova-cell0-745f-account-create-update-5v2dr" Dec 03 11:51:48 crc kubenswrapper[5052]: I1203 11:51:48.202942 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-db-create-8vmt9" Dec 03 11:51:48 crc kubenswrapper[5052]: I1203 11:51:48.203059 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-jw7pd\" (UniqueName: \"kubernetes.io/projected/fa84c362-f076-4b47-8d2b-9eadd3f2d10c-kube-api-access-jw7pd\") pod \"nova-cell0-745f-account-create-update-5v2dr\" (UID: \"fa84c362-f076-4b47-8d2b-9eadd3f2d10c\") " pod="openstack-kuttl-tests/nova-cell0-745f-account-create-update-5v2dr" Dec 03 11:51:48 crc kubenswrapper[5052]: I1203 11:51:48.289886 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-vmz62\" (UniqueName: \"kubernetes.io/projected/cca0de9c-68fc-44d4-b6b2-2be05e5d9995-kube-api-access-vmz62\") pod \"nova-cell1-4fec-account-create-update-x8rjw\" (UID: \"cca0de9c-68fc-44d4-b6b2-2be05e5d9995\") " pod="openstack-kuttl-tests/nova-cell1-4fec-account-create-update-x8rjw" Dec 03 11:51:48 crc kubenswrapper[5052]: I1203 11:51:48.289964 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/cca0de9c-68fc-44d4-b6b2-2be05e5d9995-operator-scripts\") pod \"nova-cell1-4fec-account-create-update-x8rjw\" (UID: \"cca0de9c-68fc-44d4-b6b2-2be05e5d9995\") " pod="openstack-kuttl-tests/nova-cell1-4fec-account-create-update-x8rjw" Dec 03 11:51:48 crc kubenswrapper[5052]: I1203 11:51:48.290667 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/cca0de9c-68fc-44d4-b6b2-2be05e5d9995-operator-scripts\") pod \"nova-cell1-4fec-account-create-update-x8rjw\" (UID: \"cca0de9c-68fc-44d4-b6b2-2be05e5d9995\") " pod="openstack-kuttl-tests/nova-cell1-4fec-account-create-update-x8rjw" Dec 03 11:51:48 crc kubenswrapper[5052]: I1203 11:51:48.306271 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-745f-account-create-update-5v2dr" Dec 03 11:51:48 crc kubenswrapper[5052]: I1203 11:51:48.325960 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-vmz62\" (UniqueName: \"kubernetes.io/projected/cca0de9c-68fc-44d4-b6b2-2be05e5d9995-kube-api-access-vmz62\") pod \"nova-cell1-4fec-account-create-update-x8rjw\" (UID: \"cca0de9c-68fc-44d4-b6b2-2be05e5d9995\") " pod="openstack-kuttl-tests/nova-cell1-4fec-account-create-update-x8rjw" Dec 03 11:51:48 crc kubenswrapper[5052]: I1203 11:51:48.423620 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-4fec-account-create-update-x8rjw" Dec 03 11:51:48 crc kubenswrapper[5052]: I1203 11:51:48.461635 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-db-create-4fs9m"] Dec 03 11:51:48 crc kubenswrapper[5052]: I1203 11:51:48.697864 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell0-db-create-42nm2"] Dec 03 11:51:48 crc kubenswrapper[5052]: I1203 11:51:48.699177 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-db-create-4fs9m" event={"ID":"c1c20a0e-ee8a-416e-b739-05f16a536cfa","Type":"ContainerStarted","Data":"9f4811ea7b06ca5b60ffbf42efb89aa417c5b3e4d95c40d1509dff1bb17016c8"} Dec 03 11:51:48 crc kubenswrapper[5052]: I1203 11:51:48.710301 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-proxy-b9d8f996b-brs7h" event={"ID":"036bc2b0-bc50-4c91-bea5-882b9e592e4f","Type":"ContainerStarted","Data":"47975dc763fa99fd8e031feff26f6ec865ddd0443ad75de0e4f2b3626568c51b"} Dec 03 11:51:48 crc kubenswrapper[5052]: I1203 11:51:48.710615 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/swift-proxy-b9d8f996b-brs7h" Dec 03 11:51:48 crc kubenswrapper[5052]: I1203 11:51:48.710704 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/swift-proxy-b9d8f996b-brs7h" Dec 03 11:51:48 crc kubenswrapper[5052]: I1203 11:51:48.781543 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/swift-proxy-b9d8f996b-brs7h" podStartSLOduration=2.781505712 podStartE2EDuration="2.781505712s" podCreationTimestamp="2025-12-03 11:51:46 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:51:48.744915389 +0000 UTC m=+1899.333157153" watchObservedRunningTime="2025-12-03 11:51:48.781505712 +0000 UTC m=+1899.369747476" Dec 03 11:51:48 crc kubenswrapper[5052]: I1203 11:51:48.785090 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-db-create-8vmt9"] Dec 03 11:51:48 crc kubenswrapper[5052]: I1203 11:51:48.795403 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-e4da-account-create-update-rqlfw"] Dec 03 11:51:48 crc kubenswrapper[5052]: W1203 11:51:48.803443 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod80d855f5_afda_46c0_9d72_01dee8dd93fb.slice/crio-f42f75d99bdec801338c55be6287930336d0363dc46f963fd24aa54bf5edd588 WatchSource:0}: Error finding container f42f75d99bdec801338c55be6287930336d0363dc46f963fd24aa54bf5edd588: Status 404 returned error can't find the container with id f42f75d99bdec801338c55be6287930336d0363dc46f963fd24aa54bf5edd588 Dec 03 11:51:48 crc kubenswrapper[5052]: W1203 11:51:48.943690 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfa84c362_f076_4b47_8d2b_9eadd3f2d10c.slice/crio-788b6bce85e49bfb7ee1f44e73114ed44d95d7bfe62ad21e87de54624ba2f02e WatchSource:0}: Error finding container 788b6bce85e49bfb7ee1f44e73114ed44d95d7bfe62ad21e87de54624ba2f02e: Status 404 returned error can't find the container with id 788b6bce85e49bfb7ee1f44e73114ed44d95d7bfe62ad21e87de54624ba2f02e Dec 03 11:51:48 crc kubenswrapper[5052]: I1203 11:51:48.945942 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell0-745f-account-create-update-5v2dr"] Dec 03 11:51:49 crc kubenswrapper[5052]: W1203 11:51:49.050905 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcca0de9c_68fc_44d4_b6b2_2be05e5d9995.slice/crio-230228de8894457a35c0ce7b71ba8cc76e8b8429308ea9fb308d6dcf9dce4ab6 WatchSource:0}: Error finding container 230228de8894457a35c0ce7b71ba8cc76e8b8429308ea9fb308d6dcf9dce4ab6: Status 404 returned error can't find the container with id 230228de8894457a35c0ce7b71ba8cc76e8b8429308ea9fb308d6dcf9dce4ab6 Dec 03 11:51:49 crc kubenswrapper[5052]: I1203 11:51:49.052111 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-4fec-account-create-update-x8rjw"] Dec 03 11:51:49 crc kubenswrapper[5052]: I1203 11:51:49.720777 5052 generic.go:334] "Generic (PLEG): container finished" podID="a4aab9e3-5790-4143-948b-e24a52e552a5" containerID="7458d0a52664f876e9b1ff1ffa9b479a1d19950d3701f07e57764be8b8c11e99" exitCode=0 Dec 03 11:51:49 crc kubenswrapper[5052]: I1203 11:51:49.720869 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-db-create-42nm2" event={"ID":"a4aab9e3-5790-4143-948b-e24a52e552a5","Type":"ContainerDied","Data":"7458d0a52664f876e9b1ff1ffa9b479a1d19950d3701f07e57764be8b8c11e99"} Dec 03 11:51:49 crc kubenswrapper[5052]: I1203 11:51:49.721140 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-db-create-42nm2" event={"ID":"a4aab9e3-5790-4143-948b-e24a52e552a5","Type":"ContainerStarted","Data":"70201196781bc805f13b6ec2ad822a7fb5f663322ec17cbe821cfa4c2124ebf4"} Dec 03 11:51:49 crc kubenswrapper[5052]: I1203 11:51:49.723141 5052 generic.go:334] "Generic (PLEG): container finished" podID="18c1240f-c951-40de-a0da-3b50f11f4cf1" containerID="39d9f6dd82140e8dbcb6a630404644ad9185d3873550446e8d89fa09e6c54eb8" exitCode=0 Dec 03 11:51:49 crc kubenswrapper[5052]: I1203 11:51:49.723187 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-db-create-8vmt9" event={"ID":"18c1240f-c951-40de-a0da-3b50f11f4cf1","Type":"ContainerDied","Data":"39d9f6dd82140e8dbcb6a630404644ad9185d3873550446e8d89fa09e6c54eb8"} Dec 03 11:51:49 crc kubenswrapper[5052]: I1203 11:51:49.723204 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-db-create-8vmt9" event={"ID":"18c1240f-c951-40de-a0da-3b50f11f4cf1","Type":"ContainerStarted","Data":"9b3eecd7df306850acf33f9054575c0a709af681724dce56aba72095a47d69ad"} Dec 03 11:51:49 crc kubenswrapper[5052]: I1203 11:51:49.724902 5052 generic.go:334] "Generic (PLEG): container finished" podID="80d855f5-afda-46c0-9d72-01dee8dd93fb" containerID="89d88ba4bc869ab32cad85adfbac99355488669b3081a20686a3e517b6deb8f8" exitCode=0 Dec 03 11:51:49 crc kubenswrapper[5052]: I1203 11:51:49.724959 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-e4da-account-create-update-rqlfw" event={"ID":"80d855f5-afda-46c0-9d72-01dee8dd93fb","Type":"ContainerDied","Data":"89d88ba4bc869ab32cad85adfbac99355488669b3081a20686a3e517b6deb8f8"} Dec 03 11:51:49 crc kubenswrapper[5052]: I1203 11:51:49.724982 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-e4da-account-create-update-rqlfw" event={"ID":"80d855f5-afda-46c0-9d72-01dee8dd93fb","Type":"ContainerStarted","Data":"f42f75d99bdec801338c55be6287930336d0363dc46f963fd24aa54bf5edd588"} Dec 03 11:51:49 crc kubenswrapper[5052]: I1203 11:51:49.728281 5052 generic.go:334] "Generic (PLEG): container finished" podID="fa84c362-f076-4b47-8d2b-9eadd3f2d10c" containerID="160567c413c48ad58e663b87e889640ffaea914541b8bd2d0d0fe0e8fa1833b4" exitCode=0 Dec 03 11:51:49 crc kubenswrapper[5052]: I1203 11:51:49.728351 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-745f-account-create-update-5v2dr" event={"ID":"fa84c362-f076-4b47-8d2b-9eadd3f2d10c","Type":"ContainerDied","Data":"160567c413c48ad58e663b87e889640ffaea914541b8bd2d0d0fe0e8fa1833b4"} Dec 03 11:51:49 crc kubenswrapper[5052]: I1203 11:51:49.728376 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-745f-account-create-update-5v2dr" event={"ID":"fa84c362-f076-4b47-8d2b-9eadd3f2d10c","Type":"ContainerStarted","Data":"788b6bce85e49bfb7ee1f44e73114ed44d95d7bfe62ad21e87de54624ba2f02e"} Dec 03 11:51:49 crc kubenswrapper[5052]: I1203 11:51:49.730153 5052 generic.go:334] "Generic (PLEG): container finished" podID="c1c20a0e-ee8a-416e-b739-05f16a536cfa" containerID="12163794070d86f13c99de7a2ca7f5dc22e8548c4c16239dc6f3c9916233ac1d" exitCode=0 Dec 03 11:51:49 crc kubenswrapper[5052]: I1203 11:51:49.730215 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-db-create-4fs9m" event={"ID":"c1c20a0e-ee8a-416e-b739-05f16a536cfa","Type":"ContainerDied","Data":"12163794070d86f13c99de7a2ca7f5dc22e8548c4c16239dc6f3c9916233ac1d"} Dec 03 11:51:49 crc kubenswrapper[5052]: I1203 11:51:49.732286 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-4fec-account-create-update-x8rjw" event={"ID":"cca0de9c-68fc-44d4-b6b2-2be05e5d9995","Type":"ContainerStarted","Data":"ca10bcd783453e6f7e39b4abea8415b2fde88b6f5ba39461841a0649464dfe87"} Dec 03 11:51:49 crc kubenswrapper[5052]: I1203 11:51:49.732412 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-4fec-account-create-update-x8rjw" event={"ID":"cca0de9c-68fc-44d4-b6b2-2be05e5d9995","Type":"ContainerStarted","Data":"230228de8894457a35c0ce7b71ba8cc76e8b8429308ea9fb308d6dcf9dce4ab6"} Dec 03 11:51:49 crc kubenswrapper[5052]: I1203 11:51:49.820635 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-cell1-4fec-account-create-update-x8rjw" podStartSLOduration=1.820617921 podStartE2EDuration="1.820617921s" podCreationTimestamp="2025-12-03 11:51:48 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:51:49.813176462 +0000 UTC m=+1900.401418236" watchObservedRunningTime="2025-12-03 11:51:49.820617921 +0000 UTC m=+1900.408859685" Dec 03 11:51:50 crc kubenswrapper[5052]: I1203 11:51:50.755074 5052 generic.go:334] "Generic (PLEG): container finished" podID="cca0de9c-68fc-44d4-b6b2-2be05e5d9995" containerID="ca10bcd783453e6f7e39b4abea8415b2fde88b6f5ba39461841a0649464dfe87" exitCode=0 Dec 03 11:51:50 crc kubenswrapper[5052]: I1203 11:51:50.755269 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-4fec-account-create-update-x8rjw" event={"ID":"cca0de9c-68fc-44d4-b6b2-2be05e5d9995","Type":"ContainerDied","Data":"ca10bcd783453e6f7e39b4abea8415b2fde88b6f5ba39461841a0649464dfe87"} Dec 03 11:51:50 crc kubenswrapper[5052]: I1203 11:51:50.761559 5052 generic.go:334] "Generic (PLEG): container finished" podID="f8c1750d-cf4a-49cc-a5ba-d40f19a6aad8" containerID="8f596e1b0a79e1f1f823327015afe83ef471316845e5e0fd8516f51ce4c105c2" exitCode=0 Dec 03 11:51:50 crc kubenswrapper[5052]: I1203 11:51:50.761654 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"f8c1750d-cf4a-49cc-a5ba-d40f19a6aad8","Type":"ContainerDied","Data":"8f596e1b0a79e1f1f823327015afe83ef471316845e5e0fd8516f51ce4c105c2"} Dec 03 11:51:51 crc kubenswrapper[5052]: I1203 11:51:51.014054 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:51:51 crc kubenswrapper[5052]: I1203 11:51:51.144865 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/f8c1750d-cf4a-49cc-a5ba-d40f19a6aad8-config-data\") pod \"f8c1750d-cf4a-49cc-a5ba-d40f19a6aad8\" (UID: \"f8c1750d-cf4a-49cc-a5ba-d40f19a6aad8\") " Dec 03 11:51:51 crc kubenswrapper[5052]: I1203 11:51:51.144975 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/f8c1750d-cf4a-49cc-a5ba-d40f19a6aad8-log-httpd\") pod \"f8c1750d-cf4a-49cc-a5ba-d40f19a6aad8\" (UID: \"f8c1750d-cf4a-49cc-a5ba-d40f19a6aad8\") " Dec 03 11:51:51 crc kubenswrapper[5052]: I1203 11:51:51.145027 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/f8c1750d-cf4a-49cc-a5ba-d40f19a6aad8-run-httpd\") pod \"f8c1750d-cf4a-49cc-a5ba-d40f19a6aad8\" (UID: \"f8c1750d-cf4a-49cc-a5ba-d40f19a6aad8\") " Dec 03 11:51:51 crc kubenswrapper[5052]: I1203 11:51:51.145062 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f8c1750d-cf4a-49cc-a5ba-d40f19a6aad8-combined-ca-bundle\") pod \"f8c1750d-cf4a-49cc-a5ba-d40f19a6aad8\" (UID: \"f8c1750d-cf4a-49cc-a5ba-d40f19a6aad8\") " Dec 03 11:51:51 crc kubenswrapper[5052]: I1203 11:51:51.145106 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/f8c1750d-cf4a-49cc-a5ba-d40f19a6aad8-scripts\") pod \"f8c1750d-cf4a-49cc-a5ba-d40f19a6aad8\" (UID: \"f8c1750d-cf4a-49cc-a5ba-d40f19a6aad8\") " Dec 03 11:51:51 crc kubenswrapper[5052]: I1203 11:51:51.145157 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ksbm4\" (UniqueName: \"kubernetes.io/projected/f8c1750d-cf4a-49cc-a5ba-d40f19a6aad8-kube-api-access-ksbm4\") pod \"f8c1750d-cf4a-49cc-a5ba-d40f19a6aad8\" (UID: \"f8c1750d-cf4a-49cc-a5ba-d40f19a6aad8\") " Dec 03 11:51:51 crc kubenswrapper[5052]: I1203 11:51:51.145186 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/f8c1750d-cf4a-49cc-a5ba-d40f19a6aad8-sg-core-conf-yaml\") pod \"f8c1750d-cf4a-49cc-a5ba-d40f19a6aad8\" (UID: \"f8c1750d-cf4a-49cc-a5ba-d40f19a6aad8\") " Dec 03 11:51:51 crc kubenswrapper[5052]: I1203 11:51:51.145583 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f8c1750d-cf4a-49cc-a5ba-d40f19a6aad8-log-httpd" (OuterVolumeSpecName: "log-httpd") pod "f8c1750d-cf4a-49cc-a5ba-d40f19a6aad8" (UID: "f8c1750d-cf4a-49cc-a5ba-d40f19a6aad8"). InnerVolumeSpecName "log-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:51:51 crc kubenswrapper[5052]: I1203 11:51:51.146266 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f8c1750d-cf4a-49cc-a5ba-d40f19a6aad8-run-httpd" (OuterVolumeSpecName: "run-httpd") pod "f8c1750d-cf4a-49cc-a5ba-d40f19a6aad8" (UID: "f8c1750d-cf4a-49cc-a5ba-d40f19a6aad8"). InnerVolumeSpecName "run-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:51:51 crc kubenswrapper[5052]: I1203 11:51:51.179720 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f8c1750d-cf4a-49cc-a5ba-d40f19a6aad8-scripts" (OuterVolumeSpecName: "scripts") pod "f8c1750d-cf4a-49cc-a5ba-d40f19a6aad8" (UID: "f8c1750d-cf4a-49cc-a5ba-d40f19a6aad8"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:51:51 crc kubenswrapper[5052]: I1203 11:51:51.192794 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f8c1750d-cf4a-49cc-a5ba-d40f19a6aad8-kube-api-access-ksbm4" (OuterVolumeSpecName: "kube-api-access-ksbm4") pod "f8c1750d-cf4a-49cc-a5ba-d40f19a6aad8" (UID: "f8c1750d-cf4a-49cc-a5ba-d40f19a6aad8"). InnerVolumeSpecName "kube-api-access-ksbm4". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:51:51 crc kubenswrapper[5052]: I1203 11:51:51.226278 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f8c1750d-cf4a-49cc-a5ba-d40f19a6aad8-sg-core-conf-yaml" (OuterVolumeSpecName: "sg-core-conf-yaml") pod "f8c1750d-cf4a-49cc-a5ba-d40f19a6aad8" (UID: "f8c1750d-cf4a-49cc-a5ba-d40f19a6aad8"). InnerVolumeSpecName "sg-core-conf-yaml". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:51:51 crc kubenswrapper[5052]: I1203 11:51:51.247940 5052 reconciler_common.go:293] "Volume detached for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/f8c1750d-cf4a-49cc-a5ba-d40f19a6aad8-log-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 11:51:51 crc kubenswrapper[5052]: I1203 11:51:51.247974 5052 reconciler_common.go:293] "Volume detached for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/f8c1750d-cf4a-49cc-a5ba-d40f19a6aad8-run-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 11:51:51 crc kubenswrapper[5052]: I1203 11:51:51.247985 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/f8c1750d-cf4a-49cc-a5ba-d40f19a6aad8-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:51:51 crc kubenswrapper[5052]: I1203 11:51:51.248694 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-ksbm4\" (UniqueName: \"kubernetes.io/projected/f8c1750d-cf4a-49cc-a5ba-d40f19a6aad8-kube-api-access-ksbm4\") on node \"crc\" DevicePath \"\"" Dec 03 11:51:51 crc kubenswrapper[5052]: I1203 11:51:51.248711 5052 reconciler_common.go:293] "Volume detached for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/f8c1750d-cf4a-49cc-a5ba-d40f19a6aad8-sg-core-conf-yaml\") on node \"crc\" DevicePath \"\"" Dec 03 11:51:51 crc kubenswrapper[5052]: I1203 11:51:51.280009 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-db-create-4fs9m" Dec 03 11:51:51 crc kubenswrapper[5052]: I1203 11:51:51.290402 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f8c1750d-cf4a-49cc-a5ba-d40f19a6aad8-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "f8c1750d-cf4a-49cc-a5ba-d40f19a6aad8" (UID: "f8c1750d-cf4a-49cc-a5ba-d40f19a6aad8"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:51:51 crc kubenswrapper[5052]: I1203 11:51:51.298778 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-db-create-42nm2" Dec 03 11:51:51 crc kubenswrapper[5052]: I1203 11:51:51.350339 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5v2k2\" (UniqueName: \"kubernetes.io/projected/c1c20a0e-ee8a-416e-b739-05f16a536cfa-kube-api-access-5v2k2\") pod \"c1c20a0e-ee8a-416e-b739-05f16a536cfa\" (UID: \"c1c20a0e-ee8a-416e-b739-05f16a536cfa\") " Dec 03 11:51:51 crc kubenswrapper[5052]: I1203 11:51:51.350396 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/c1c20a0e-ee8a-416e-b739-05f16a536cfa-operator-scripts\") pod \"c1c20a0e-ee8a-416e-b739-05f16a536cfa\" (UID: \"c1c20a0e-ee8a-416e-b739-05f16a536cfa\") " Dec 03 11:51:51 crc kubenswrapper[5052]: I1203 11:51:51.350913 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f8c1750d-cf4a-49cc-a5ba-d40f19a6aad8-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:51:51 crc kubenswrapper[5052]: I1203 11:51:51.351423 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c1c20a0e-ee8a-416e-b739-05f16a536cfa-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "c1c20a0e-ee8a-416e-b739-05f16a536cfa" (UID: "c1c20a0e-ee8a-416e-b739-05f16a536cfa"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:51:51 crc kubenswrapper[5052]: I1203 11:51:51.356467 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c1c20a0e-ee8a-416e-b739-05f16a536cfa-kube-api-access-5v2k2" (OuterVolumeSpecName: "kube-api-access-5v2k2") pod "c1c20a0e-ee8a-416e-b739-05f16a536cfa" (UID: "c1c20a0e-ee8a-416e-b739-05f16a536cfa"). InnerVolumeSpecName "kube-api-access-5v2k2". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:51:51 crc kubenswrapper[5052]: I1203 11:51:51.363438 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f8c1750d-cf4a-49cc-a5ba-d40f19a6aad8-config-data" (OuterVolumeSpecName: "config-data") pod "f8c1750d-cf4a-49cc-a5ba-d40f19a6aad8" (UID: "f8c1750d-cf4a-49cc-a5ba-d40f19a6aad8"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:51:51 crc kubenswrapper[5052]: I1203 11:51:51.423161 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-745f-account-create-update-5v2dr" Dec 03 11:51:51 crc kubenswrapper[5052]: I1203 11:51:51.446646 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-e4da-account-create-update-rqlfw" Dec 03 11:51:51 crc kubenswrapper[5052]: I1203 11:51:51.453209 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/a4aab9e3-5790-4143-948b-e24a52e552a5-operator-scripts\") pod \"a4aab9e3-5790-4143-948b-e24a52e552a5\" (UID: \"a4aab9e3-5790-4143-948b-e24a52e552a5\") " Dec 03 11:51:51 crc kubenswrapper[5052]: I1203 11:51:51.453346 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-62tjt\" (UniqueName: \"kubernetes.io/projected/a4aab9e3-5790-4143-948b-e24a52e552a5-kube-api-access-62tjt\") pod \"a4aab9e3-5790-4143-948b-e24a52e552a5\" (UID: \"a4aab9e3-5790-4143-948b-e24a52e552a5\") " Dec 03 11:51:51 crc kubenswrapper[5052]: I1203 11:51:51.453820 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/f8c1750d-cf4a-49cc-a5ba-d40f19a6aad8-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:51:51 crc kubenswrapper[5052]: I1203 11:51:51.453838 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-5v2k2\" (UniqueName: \"kubernetes.io/projected/c1c20a0e-ee8a-416e-b739-05f16a536cfa-kube-api-access-5v2k2\") on node \"crc\" DevicePath \"\"" Dec 03 11:51:51 crc kubenswrapper[5052]: I1203 11:51:51.453849 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/c1c20a0e-ee8a-416e-b739-05f16a536cfa-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:51:51 crc kubenswrapper[5052]: I1203 11:51:51.454418 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a4aab9e3-5790-4143-948b-e24a52e552a5-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "a4aab9e3-5790-4143-948b-e24a52e552a5" (UID: "a4aab9e3-5790-4143-948b-e24a52e552a5"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:51:51 crc kubenswrapper[5052]: I1203 11:51:51.457612 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a4aab9e3-5790-4143-948b-e24a52e552a5-kube-api-access-62tjt" (OuterVolumeSpecName: "kube-api-access-62tjt") pod "a4aab9e3-5790-4143-948b-e24a52e552a5" (UID: "a4aab9e3-5790-4143-948b-e24a52e552a5"). InnerVolumeSpecName "kube-api-access-62tjt". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:51:51 crc kubenswrapper[5052]: I1203 11:51:51.490010 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-db-create-8vmt9" Dec 03 11:51:51 crc kubenswrapper[5052]: I1203 11:51:51.555305 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/fa84c362-f076-4b47-8d2b-9eadd3f2d10c-operator-scripts\") pod \"fa84c362-f076-4b47-8d2b-9eadd3f2d10c\" (UID: \"fa84c362-f076-4b47-8d2b-9eadd3f2d10c\") " Dec 03 11:51:51 crc kubenswrapper[5052]: I1203 11:51:51.555371 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6fjp6\" (UniqueName: \"kubernetes.io/projected/80d855f5-afda-46c0-9d72-01dee8dd93fb-kube-api-access-6fjp6\") pod \"80d855f5-afda-46c0-9d72-01dee8dd93fb\" (UID: \"80d855f5-afda-46c0-9d72-01dee8dd93fb\") " Dec 03 11:51:51 crc kubenswrapper[5052]: I1203 11:51:51.555443 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jw7pd\" (UniqueName: \"kubernetes.io/projected/fa84c362-f076-4b47-8d2b-9eadd3f2d10c-kube-api-access-jw7pd\") pod \"fa84c362-f076-4b47-8d2b-9eadd3f2d10c\" (UID: \"fa84c362-f076-4b47-8d2b-9eadd3f2d10c\") " Dec 03 11:51:51 crc kubenswrapper[5052]: I1203 11:51:51.555564 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/80d855f5-afda-46c0-9d72-01dee8dd93fb-operator-scripts\") pod \"80d855f5-afda-46c0-9d72-01dee8dd93fb\" (UID: \"80d855f5-afda-46c0-9d72-01dee8dd93fb\") " Dec 03 11:51:51 crc kubenswrapper[5052]: I1203 11:51:51.555845 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fa84c362-f076-4b47-8d2b-9eadd3f2d10c-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "fa84c362-f076-4b47-8d2b-9eadd3f2d10c" (UID: "fa84c362-f076-4b47-8d2b-9eadd3f2d10c"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:51:51 crc kubenswrapper[5052]: I1203 11:51:51.556324 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/80d855f5-afda-46c0-9d72-01dee8dd93fb-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "80d855f5-afda-46c0-9d72-01dee8dd93fb" (UID: "80d855f5-afda-46c0-9d72-01dee8dd93fb"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:51:51 crc kubenswrapper[5052]: I1203 11:51:51.556405 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/fa84c362-f076-4b47-8d2b-9eadd3f2d10c-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:51:51 crc kubenswrapper[5052]: I1203 11:51:51.556423 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/a4aab9e3-5790-4143-948b-e24a52e552a5-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:51:51 crc kubenswrapper[5052]: I1203 11:51:51.556440 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-62tjt\" (UniqueName: \"kubernetes.io/projected/a4aab9e3-5790-4143-948b-e24a52e552a5-kube-api-access-62tjt\") on node \"crc\" DevicePath \"\"" Dec 03 11:51:51 crc kubenswrapper[5052]: I1203 11:51:51.559846 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/80d855f5-afda-46c0-9d72-01dee8dd93fb-kube-api-access-6fjp6" (OuterVolumeSpecName: "kube-api-access-6fjp6") pod "80d855f5-afda-46c0-9d72-01dee8dd93fb" (UID: "80d855f5-afda-46c0-9d72-01dee8dd93fb"). InnerVolumeSpecName "kube-api-access-6fjp6". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:51:51 crc kubenswrapper[5052]: I1203 11:51:51.563077 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fa84c362-f076-4b47-8d2b-9eadd3f2d10c-kube-api-access-jw7pd" (OuterVolumeSpecName: "kube-api-access-jw7pd") pod "fa84c362-f076-4b47-8d2b-9eadd3f2d10c" (UID: "fa84c362-f076-4b47-8d2b-9eadd3f2d10c"). InnerVolumeSpecName "kube-api-access-jw7pd". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:51:51 crc kubenswrapper[5052]: I1203 11:51:51.657268 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/18c1240f-c951-40de-a0da-3b50f11f4cf1-operator-scripts\") pod \"18c1240f-c951-40de-a0da-3b50f11f4cf1\" (UID: \"18c1240f-c951-40de-a0da-3b50f11f4cf1\") " Dec 03 11:51:51 crc kubenswrapper[5052]: I1203 11:51:51.657490 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p5p88\" (UniqueName: \"kubernetes.io/projected/18c1240f-c951-40de-a0da-3b50f11f4cf1-kube-api-access-p5p88\") pod \"18c1240f-c951-40de-a0da-3b50f11f4cf1\" (UID: \"18c1240f-c951-40de-a0da-3b50f11f4cf1\") " Dec 03 11:51:51 crc kubenswrapper[5052]: I1203 11:51:51.657858 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/80d855f5-afda-46c0-9d72-01dee8dd93fb-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:51:51 crc kubenswrapper[5052]: I1203 11:51:51.657891 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-6fjp6\" (UniqueName: \"kubernetes.io/projected/80d855f5-afda-46c0-9d72-01dee8dd93fb-kube-api-access-6fjp6\") on node \"crc\" DevicePath \"\"" Dec 03 11:51:51 crc kubenswrapper[5052]: I1203 11:51:51.657906 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-jw7pd\" (UniqueName: \"kubernetes.io/projected/fa84c362-f076-4b47-8d2b-9eadd3f2d10c-kube-api-access-jw7pd\") on node \"crc\" DevicePath \"\"" Dec 03 11:51:51 crc kubenswrapper[5052]: I1203 11:51:51.657850 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/18c1240f-c951-40de-a0da-3b50f11f4cf1-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "18c1240f-c951-40de-a0da-3b50f11f4cf1" (UID: "18c1240f-c951-40de-a0da-3b50f11f4cf1"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:51:51 crc kubenswrapper[5052]: I1203 11:51:51.661775 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/18c1240f-c951-40de-a0da-3b50f11f4cf1-kube-api-access-p5p88" (OuterVolumeSpecName: "kube-api-access-p5p88") pod "18c1240f-c951-40de-a0da-3b50f11f4cf1" (UID: "18c1240f-c951-40de-a0da-3b50f11f4cf1"). InnerVolumeSpecName "kube-api-access-p5p88". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:51:51 crc kubenswrapper[5052]: I1203 11:51:51.760020 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-p5p88\" (UniqueName: \"kubernetes.io/projected/18c1240f-c951-40de-a0da-3b50f11f4cf1-kube-api-access-p5p88\") on node \"crc\" DevicePath \"\"" Dec 03 11:51:51 crc kubenswrapper[5052]: I1203 11:51:51.760052 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/18c1240f-c951-40de-a0da-3b50f11f4cf1-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:51:51 crc kubenswrapper[5052]: I1203 11:51:51.774801 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"f8c1750d-cf4a-49cc-a5ba-d40f19a6aad8","Type":"ContainerDied","Data":"95da1c09d4a814655d8ff814d18772082ba427c8e948f5716fc3c2800d6ee734"} Dec 03 11:51:51 crc kubenswrapper[5052]: I1203 11:51:51.774856 5052 scope.go:117] "RemoveContainer" containerID="2807aed7a69b1fdeca76bc8aeda707467e8f044feefef396c4c3ef5f25917a3b" Dec 03 11:51:51 crc kubenswrapper[5052]: I1203 11:51:51.775010 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:51:51 crc kubenswrapper[5052]: I1203 11:51:51.779503 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-db-create-42nm2" event={"ID":"a4aab9e3-5790-4143-948b-e24a52e552a5","Type":"ContainerDied","Data":"70201196781bc805f13b6ec2ad822a7fb5f663322ec17cbe821cfa4c2124ebf4"} Dec 03 11:51:51 crc kubenswrapper[5052]: I1203 11:51:51.780849 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="70201196781bc805f13b6ec2ad822a7fb5f663322ec17cbe821cfa4c2124ebf4" Dec 03 11:51:51 crc kubenswrapper[5052]: I1203 11:51:51.781714 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-db-create-42nm2" Dec 03 11:51:51 crc kubenswrapper[5052]: I1203 11:51:51.783632 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-db-create-8vmt9" event={"ID":"18c1240f-c951-40de-a0da-3b50f11f4cf1","Type":"ContainerDied","Data":"9b3eecd7df306850acf33f9054575c0a709af681724dce56aba72095a47d69ad"} Dec 03 11:51:51 crc kubenswrapper[5052]: I1203 11:51:51.783794 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9b3eecd7df306850acf33f9054575c0a709af681724dce56aba72095a47d69ad" Dec 03 11:51:51 crc kubenswrapper[5052]: I1203 11:51:51.783648 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-db-create-8vmt9" Dec 03 11:51:51 crc kubenswrapper[5052]: I1203 11:51:51.786150 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-e4da-account-create-update-rqlfw" Dec 03 11:51:51 crc kubenswrapper[5052]: I1203 11:51:51.786196 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-e4da-account-create-update-rqlfw" event={"ID":"80d855f5-afda-46c0-9d72-01dee8dd93fb","Type":"ContainerDied","Data":"f42f75d99bdec801338c55be6287930336d0363dc46f963fd24aa54bf5edd588"} Dec 03 11:51:51 crc kubenswrapper[5052]: I1203 11:51:51.786449 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f42f75d99bdec801338c55be6287930336d0363dc46f963fd24aa54bf5edd588" Dec 03 11:51:51 crc kubenswrapper[5052]: I1203 11:51:51.789354 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-745f-account-create-update-5v2dr" event={"ID":"fa84c362-f076-4b47-8d2b-9eadd3f2d10c","Type":"ContainerDied","Data":"788b6bce85e49bfb7ee1f44e73114ed44d95d7bfe62ad21e87de54624ba2f02e"} Dec 03 11:51:51 crc kubenswrapper[5052]: I1203 11:51:51.789386 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="788b6bce85e49bfb7ee1f44e73114ed44d95d7bfe62ad21e87de54624ba2f02e" Dec 03 11:51:51 crc kubenswrapper[5052]: I1203 11:51:51.789477 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-745f-account-create-update-5v2dr" Dec 03 11:51:51 crc kubenswrapper[5052]: I1203 11:51:51.791643 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-db-create-4fs9m" Dec 03 11:51:51 crc kubenswrapper[5052]: I1203 11:51:51.791645 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-db-create-4fs9m" event={"ID":"c1c20a0e-ee8a-416e-b739-05f16a536cfa","Type":"ContainerDied","Data":"9f4811ea7b06ca5b60ffbf42efb89aa417c5b3e4d95c40d1509dff1bb17016c8"} Dec 03 11:51:51 crc kubenswrapper[5052]: I1203 11:51:51.791763 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9f4811ea7b06ca5b60ffbf42efb89aa417c5b3e4d95c40d1509dff1bb17016c8" Dec 03 11:51:51 crc kubenswrapper[5052]: I1203 11:51:51.808283 5052 scope.go:117] "RemoveContainer" containerID="152e33660b972abe070d51c7baf270cd5dab59b3cbe27ab26e323e676c508393" Dec 03 11:51:51 crc kubenswrapper[5052]: I1203 11:51:51.833443 5052 scope.go:117] "RemoveContainer" containerID="8f596e1b0a79e1f1f823327015afe83ef471316845e5e0fd8516f51ce4c105c2" Dec 03 11:51:51 crc kubenswrapper[5052]: I1203 11:51:51.847492 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:51:51 crc kubenswrapper[5052]: I1203 11:51:51.871320 5052 scope.go:117] "RemoveContainer" containerID="eb33c766c34b7def659e87f673ae82230a201d258f7a88dee07749f3cbd8d1e0" Dec 03 11:51:51 crc kubenswrapper[5052]: I1203 11:51:51.872682 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:51:51 crc kubenswrapper[5052]: I1203 11:51:51.896129 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:51:51 crc kubenswrapper[5052]: E1203 11:51:51.897727 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="a4aab9e3-5790-4143-948b-e24a52e552a5" containerName="mariadb-database-create" Dec 03 11:51:51 crc kubenswrapper[5052]: I1203 11:51:51.897746 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="a4aab9e3-5790-4143-948b-e24a52e552a5" containerName="mariadb-database-create" Dec 03 11:51:51 crc kubenswrapper[5052]: E1203 11:51:51.897761 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="fa84c362-f076-4b47-8d2b-9eadd3f2d10c" containerName="mariadb-account-create-update" Dec 03 11:51:51 crc kubenswrapper[5052]: I1203 11:51:51.897767 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="fa84c362-f076-4b47-8d2b-9eadd3f2d10c" containerName="mariadb-account-create-update" Dec 03 11:51:51 crc kubenswrapper[5052]: E1203 11:51:51.897783 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="f8c1750d-cf4a-49cc-a5ba-d40f19a6aad8" containerName="ceilometer-notification-agent" Dec 03 11:51:51 crc kubenswrapper[5052]: I1203 11:51:51.897789 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="f8c1750d-cf4a-49cc-a5ba-d40f19a6aad8" containerName="ceilometer-notification-agent" Dec 03 11:51:51 crc kubenswrapper[5052]: E1203 11:51:51.897800 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="80d855f5-afda-46c0-9d72-01dee8dd93fb" containerName="mariadb-account-create-update" Dec 03 11:51:51 crc kubenswrapper[5052]: I1203 11:51:51.897805 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="80d855f5-afda-46c0-9d72-01dee8dd93fb" containerName="mariadb-account-create-update" Dec 03 11:51:51 crc kubenswrapper[5052]: E1203 11:51:51.897823 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="18c1240f-c951-40de-a0da-3b50f11f4cf1" containerName="mariadb-database-create" Dec 03 11:51:51 crc kubenswrapper[5052]: I1203 11:51:51.897828 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="18c1240f-c951-40de-a0da-3b50f11f4cf1" containerName="mariadb-database-create" Dec 03 11:51:51 crc kubenswrapper[5052]: E1203 11:51:51.897840 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="f8c1750d-cf4a-49cc-a5ba-d40f19a6aad8" containerName="sg-core" Dec 03 11:51:51 crc kubenswrapper[5052]: I1203 11:51:51.897845 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="f8c1750d-cf4a-49cc-a5ba-d40f19a6aad8" containerName="sg-core" Dec 03 11:51:51 crc kubenswrapper[5052]: E1203 11:51:51.897854 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="f8c1750d-cf4a-49cc-a5ba-d40f19a6aad8" containerName="proxy-httpd" Dec 03 11:51:51 crc kubenswrapper[5052]: I1203 11:51:51.897859 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="f8c1750d-cf4a-49cc-a5ba-d40f19a6aad8" containerName="proxy-httpd" Dec 03 11:51:51 crc kubenswrapper[5052]: E1203 11:51:51.897868 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="c1c20a0e-ee8a-416e-b739-05f16a536cfa" containerName="mariadb-database-create" Dec 03 11:51:51 crc kubenswrapper[5052]: I1203 11:51:51.897874 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="c1c20a0e-ee8a-416e-b739-05f16a536cfa" containerName="mariadb-database-create" Dec 03 11:51:51 crc kubenswrapper[5052]: E1203 11:51:51.897891 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="f8c1750d-cf4a-49cc-a5ba-d40f19a6aad8" containerName="ceilometer-central-agent" Dec 03 11:51:51 crc kubenswrapper[5052]: I1203 11:51:51.897897 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="f8c1750d-cf4a-49cc-a5ba-d40f19a6aad8" containerName="ceilometer-central-agent" Dec 03 11:51:51 crc kubenswrapper[5052]: I1203 11:51:51.898043 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="f8c1750d-cf4a-49cc-a5ba-d40f19a6aad8" containerName="ceilometer-central-agent" Dec 03 11:51:51 crc kubenswrapper[5052]: I1203 11:51:51.898056 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="c1c20a0e-ee8a-416e-b739-05f16a536cfa" containerName="mariadb-database-create" Dec 03 11:51:51 crc kubenswrapper[5052]: I1203 11:51:51.898068 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="f8c1750d-cf4a-49cc-a5ba-d40f19a6aad8" containerName="proxy-httpd" Dec 03 11:51:51 crc kubenswrapper[5052]: I1203 11:51:51.898081 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="fa84c362-f076-4b47-8d2b-9eadd3f2d10c" containerName="mariadb-account-create-update" Dec 03 11:51:51 crc kubenswrapper[5052]: I1203 11:51:51.898090 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="f8c1750d-cf4a-49cc-a5ba-d40f19a6aad8" containerName="ceilometer-notification-agent" Dec 03 11:51:51 crc kubenswrapper[5052]: I1203 11:51:51.898097 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="a4aab9e3-5790-4143-948b-e24a52e552a5" containerName="mariadb-database-create" Dec 03 11:51:51 crc kubenswrapper[5052]: I1203 11:51:51.898107 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="f8c1750d-cf4a-49cc-a5ba-d40f19a6aad8" containerName="sg-core" Dec 03 11:51:51 crc kubenswrapper[5052]: I1203 11:51:51.898118 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="80d855f5-afda-46c0-9d72-01dee8dd93fb" containerName="mariadb-account-create-update" Dec 03 11:51:51 crc kubenswrapper[5052]: I1203 11:51:51.898128 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="18c1240f-c951-40de-a0da-3b50f11f4cf1" containerName="mariadb-database-create" Dec 03 11:51:51 crc kubenswrapper[5052]: I1203 11:51:51.903081 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:51:51 crc kubenswrapper[5052]: I1203 11:51:51.906024 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-scripts" Dec 03 11:51:51 crc kubenswrapper[5052]: I1203 11:51:51.906242 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-config-data" Dec 03 11:51:51 crc kubenswrapper[5052]: I1203 11:51:51.911779 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:51:51 crc kubenswrapper[5052]: I1203 11:51:51.963437 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2b63c55a-e29f-45a8-8f5d-4166c162c3cb-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"2b63c55a-e29f-45a8-8f5d-4166c162c3cb\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:51:51 crc kubenswrapper[5052]: I1203 11:51:51.963820 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/2b63c55a-e29f-45a8-8f5d-4166c162c3cb-log-httpd\") pod \"ceilometer-0\" (UID: \"2b63c55a-e29f-45a8-8f5d-4166c162c3cb\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:51:51 crc kubenswrapper[5052]: I1203 11:51:51.963865 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4xxkf\" (UniqueName: \"kubernetes.io/projected/2b63c55a-e29f-45a8-8f5d-4166c162c3cb-kube-api-access-4xxkf\") pod \"ceilometer-0\" (UID: \"2b63c55a-e29f-45a8-8f5d-4166c162c3cb\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:51:51 crc kubenswrapper[5052]: I1203 11:51:51.963884 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/2b63c55a-e29f-45a8-8f5d-4166c162c3cb-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"2b63c55a-e29f-45a8-8f5d-4166c162c3cb\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:51:51 crc kubenswrapper[5052]: I1203 11:51:51.963914 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/2b63c55a-e29f-45a8-8f5d-4166c162c3cb-run-httpd\") pod \"ceilometer-0\" (UID: \"2b63c55a-e29f-45a8-8f5d-4166c162c3cb\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:51:51 crc kubenswrapper[5052]: I1203 11:51:51.963990 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/2b63c55a-e29f-45a8-8f5d-4166c162c3cb-scripts\") pod \"ceilometer-0\" (UID: \"2b63c55a-e29f-45a8-8f5d-4166c162c3cb\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:51:51 crc kubenswrapper[5052]: I1203 11:51:51.964016 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/2b63c55a-e29f-45a8-8f5d-4166c162c3cb-config-data\") pod \"ceilometer-0\" (UID: \"2b63c55a-e29f-45a8-8f5d-4166c162c3cb\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:51:51 crc kubenswrapper[5052]: I1203 11:51:51.971933 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/neutron-68d4b65fff-k4c5r" Dec 03 11:51:52 crc kubenswrapper[5052]: I1203 11:51:52.065293 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/2b63c55a-e29f-45a8-8f5d-4166c162c3cb-scripts\") pod \"ceilometer-0\" (UID: \"2b63c55a-e29f-45a8-8f5d-4166c162c3cb\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:51:52 crc kubenswrapper[5052]: I1203 11:51:52.065340 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/2b63c55a-e29f-45a8-8f5d-4166c162c3cb-config-data\") pod \"ceilometer-0\" (UID: \"2b63c55a-e29f-45a8-8f5d-4166c162c3cb\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:51:52 crc kubenswrapper[5052]: I1203 11:51:52.065420 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2b63c55a-e29f-45a8-8f5d-4166c162c3cb-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"2b63c55a-e29f-45a8-8f5d-4166c162c3cb\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:51:52 crc kubenswrapper[5052]: I1203 11:51:52.065477 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/2b63c55a-e29f-45a8-8f5d-4166c162c3cb-log-httpd\") pod \"ceilometer-0\" (UID: \"2b63c55a-e29f-45a8-8f5d-4166c162c3cb\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:51:52 crc kubenswrapper[5052]: I1203 11:51:52.065580 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-4xxkf\" (UniqueName: \"kubernetes.io/projected/2b63c55a-e29f-45a8-8f5d-4166c162c3cb-kube-api-access-4xxkf\") pod \"ceilometer-0\" (UID: \"2b63c55a-e29f-45a8-8f5d-4166c162c3cb\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:51:52 crc kubenswrapper[5052]: I1203 11:51:52.065610 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/2b63c55a-e29f-45a8-8f5d-4166c162c3cb-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"2b63c55a-e29f-45a8-8f5d-4166c162c3cb\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:51:52 crc kubenswrapper[5052]: I1203 11:51:52.065645 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/2b63c55a-e29f-45a8-8f5d-4166c162c3cb-run-httpd\") pod \"ceilometer-0\" (UID: \"2b63c55a-e29f-45a8-8f5d-4166c162c3cb\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:51:52 crc kubenswrapper[5052]: I1203 11:51:52.068917 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/2b63c55a-e29f-45a8-8f5d-4166c162c3cb-run-httpd\") pod \"ceilometer-0\" (UID: \"2b63c55a-e29f-45a8-8f5d-4166c162c3cb\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:51:52 crc kubenswrapper[5052]: I1203 11:51:52.069689 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/2b63c55a-e29f-45a8-8f5d-4166c162c3cb-log-httpd\") pod \"ceilometer-0\" (UID: \"2b63c55a-e29f-45a8-8f5d-4166c162c3cb\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:51:52 crc kubenswrapper[5052]: I1203 11:51:52.083039 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/2b63c55a-e29f-45a8-8f5d-4166c162c3cb-scripts\") pod \"ceilometer-0\" (UID: \"2b63c55a-e29f-45a8-8f5d-4166c162c3cb\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:51:52 crc kubenswrapper[5052]: I1203 11:51:52.088230 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2b63c55a-e29f-45a8-8f5d-4166c162c3cb-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"2b63c55a-e29f-45a8-8f5d-4166c162c3cb\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:51:52 crc kubenswrapper[5052]: I1203 11:51:52.096816 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/2b63c55a-e29f-45a8-8f5d-4166c162c3cb-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"2b63c55a-e29f-45a8-8f5d-4166c162c3cb\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:51:52 crc kubenswrapper[5052]: I1203 11:51:52.099975 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-4xxkf\" (UniqueName: \"kubernetes.io/projected/2b63c55a-e29f-45a8-8f5d-4166c162c3cb-kube-api-access-4xxkf\") pod \"ceilometer-0\" (UID: \"2b63c55a-e29f-45a8-8f5d-4166c162c3cb\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:51:52 crc kubenswrapper[5052]: I1203 11:51:52.105861 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/2b63c55a-e29f-45a8-8f5d-4166c162c3cb-config-data\") pod \"ceilometer-0\" (UID: \"2b63c55a-e29f-45a8-8f5d-4166c162c3cb\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:51:52 crc kubenswrapper[5052]: I1203 11:51:52.195248 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-4fec-account-create-update-x8rjw" Dec 03 11:51:52 crc kubenswrapper[5052]: I1203 11:51:52.234967 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:51:52 crc kubenswrapper[5052]: I1203 11:51:52.272523 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/cca0de9c-68fc-44d4-b6b2-2be05e5d9995-operator-scripts\") pod \"cca0de9c-68fc-44d4-b6b2-2be05e5d9995\" (UID: \"cca0de9c-68fc-44d4-b6b2-2be05e5d9995\") " Dec 03 11:51:52 crc kubenswrapper[5052]: I1203 11:51:52.272725 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vmz62\" (UniqueName: \"kubernetes.io/projected/cca0de9c-68fc-44d4-b6b2-2be05e5d9995-kube-api-access-vmz62\") pod \"cca0de9c-68fc-44d4-b6b2-2be05e5d9995\" (UID: \"cca0de9c-68fc-44d4-b6b2-2be05e5d9995\") " Dec 03 11:51:52 crc kubenswrapper[5052]: I1203 11:51:52.273042 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cca0de9c-68fc-44d4-b6b2-2be05e5d9995-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "cca0de9c-68fc-44d4-b6b2-2be05e5d9995" (UID: "cca0de9c-68fc-44d4-b6b2-2be05e5d9995"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:51:52 crc kubenswrapper[5052]: I1203 11:51:52.273315 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/cca0de9c-68fc-44d4-b6b2-2be05e5d9995-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:51:52 crc kubenswrapper[5052]: I1203 11:51:52.275761 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cca0de9c-68fc-44d4-b6b2-2be05e5d9995-kube-api-access-vmz62" (OuterVolumeSpecName: "kube-api-access-vmz62") pod "cca0de9c-68fc-44d4-b6b2-2be05e5d9995" (UID: "cca0de9c-68fc-44d4-b6b2-2be05e5d9995"). InnerVolumeSpecName "kube-api-access-vmz62". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:51:52 crc kubenswrapper[5052]: I1203 11:51:52.377064 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-vmz62\" (UniqueName: \"kubernetes.io/projected/cca0de9c-68fc-44d4-b6b2-2be05e5d9995-kube-api-access-vmz62\") on node \"crc\" DevicePath \"\"" Dec 03 11:51:52 crc kubenswrapper[5052]: I1203 11:51:52.702054 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:51:52 crc kubenswrapper[5052]: I1203 11:51:52.801528 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-4fec-account-create-update-x8rjw" Dec 03 11:51:52 crc kubenswrapper[5052]: I1203 11:51:52.801528 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-4fec-account-create-update-x8rjw" event={"ID":"cca0de9c-68fc-44d4-b6b2-2be05e5d9995","Type":"ContainerDied","Data":"230228de8894457a35c0ce7b71ba8cc76e8b8429308ea9fb308d6dcf9dce4ab6"} Dec 03 11:51:52 crc kubenswrapper[5052]: I1203 11:51:52.801574 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="230228de8894457a35c0ce7b71ba8cc76e8b8429308ea9fb308d6dcf9dce4ab6" Dec 03 11:51:52 crc kubenswrapper[5052]: I1203 11:51:52.804218 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"2b63c55a-e29f-45a8-8f5d-4166c162c3cb","Type":"ContainerStarted","Data":"8371c14a38dfdfc474cdbe89b3b61e22c5f74d517c4df4efd0c4444be88e5a4d"} Dec 03 11:51:53 crc kubenswrapper[5052]: I1203 11:51:53.153974 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-db-sync-tttqk"] Dec 03 11:51:53 crc kubenswrapper[5052]: E1203 11:51:53.154775 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="cca0de9c-68fc-44d4-b6b2-2be05e5d9995" containerName="mariadb-account-create-update" Dec 03 11:51:53 crc kubenswrapper[5052]: I1203 11:51:53.154793 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="cca0de9c-68fc-44d4-b6b2-2be05e5d9995" containerName="mariadb-account-create-update" Dec 03 11:51:53 crc kubenswrapper[5052]: I1203 11:51:53.154996 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="cca0de9c-68fc-44d4-b6b2-2be05e5d9995" containerName="mariadb-account-create-update" Dec 03 11:51:53 crc kubenswrapper[5052]: I1203 11:51:53.155913 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-tttqk" Dec 03 11:51:53 crc kubenswrapper[5052]: I1203 11:51:53.158636 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-nova-dockercfg-8c458" Dec 03 11:51:53 crc kubenswrapper[5052]: I1203 11:51:53.158791 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell0-conductor-scripts" Dec 03 11:51:53 crc kubenswrapper[5052]: I1203 11:51:53.158790 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell0-conductor-config-data" Dec 03 11:51:53 crc kubenswrapper[5052]: I1203 11:51:53.174989 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-db-sync-tttqk"] Dec 03 11:51:53 crc kubenswrapper[5052]: I1203 11:51:53.293988 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/c84c82ae-32a2-4965-9ad3-7eb90e03657c-scripts\") pod \"nova-cell0-conductor-db-sync-tttqk\" (UID: \"c84c82ae-32a2-4965-9ad3-7eb90e03657c\") " pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-tttqk" Dec 03 11:51:53 crc kubenswrapper[5052]: I1203 11:51:53.294068 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q7gkw\" (UniqueName: \"kubernetes.io/projected/c84c82ae-32a2-4965-9ad3-7eb90e03657c-kube-api-access-q7gkw\") pod \"nova-cell0-conductor-db-sync-tttqk\" (UID: \"c84c82ae-32a2-4965-9ad3-7eb90e03657c\") " pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-tttqk" Dec 03 11:51:53 crc kubenswrapper[5052]: I1203 11:51:53.294156 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c84c82ae-32a2-4965-9ad3-7eb90e03657c-combined-ca-bundle\") pod \"nova-cell0-conductor-db-sync-tttqk\" (UID: \"c84c82ae-32a2-4965-9ad3-7eb90e03657c\") " pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-tttqk" Dec 03 11:51:53 crc kubenswrapper[5052]: I1203 11:51:53.294221 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/c84c82ae-32a2-4965-9ad3-7eb90e03657c-config-data\") pod \"nova-cell0-conductor-db-sync-tttqk\" (UID: \"c84c82ae-32a2-4965-9ad3-7eb90e03657c\") " pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-tttqk" Dec 03 11:51:53 crc kubenswrapper[5052]: I1203 11:51:53.395708 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c84c82ae-32a2-4965-9ad3-7eb90e03657c-combined-ca-bundle\") pod \"nova-cell0-conductor-db-sync-tttqk\" (UID: \"c84c82ae-32a2-4965-9ad3-7eb90e03657c\") " pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-tttqk" Dec 03 11:51:53 crc kubenswrapper[5052]: I1203 11:51:53.395790 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/c84c82ae-32a2-4965-9ad3-7eb90e03657c-config-data\") pod \"nova-cell0-conductor-db-sync-tttqk\" (UID: \"c84c82ae-32a2-4965-9ad3-7eb90e03657c\") " pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-tttqk" Dec 03 11:51:53 crc kubenswrapper[5052]: I1203 11:51:53.395837 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/c84c82ae-32a2-4965-9ad3-7eb90e03657c-scripts\") pod \"nova-cell0-conductor-db-sync-tttqk\" (UID: \"c84c82ae-32a2-4965-9ad3-7eb90e03657c\") " pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-tttqk" Dec 03 11:51:53 crc kubenswrapper[5052]: I1203 11:51:53.395880 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-q7gkw\" (UniqueName: \"kubernetes.io/projected/c84c82ae-32a2-4965-9ad3-7eb90e03657c-kube-api-access-q7gkw\") pod \"nova-cell0-conductor-db-sync-tttqk\" (UID: \"c84c82ae-32a2-4965-9ad3-7eb90e03657c\") " pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-tttqk" Dec 03 11:51:53 crc kubenswrapper[5052]: I1203 11:51:53.401339 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/c84c82ae-32a2-4965-9ad3-7eb90e03657c-scripts\") pod \"nova-cell0-conductor-db-sync-tttqk\" (UID: \"c84c82ae-32a2-4965-9ad3-7eb90e03657c\") " pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-tttqk" Dec 03 11:51:53 crc kubenswrapper[5052]: I1203 11:51:53.401399 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c84c82ae-32a2-4965-9ad3-7eb90e03657c-combined-ca-bundle\") pod \"nova-cell0-conductor-db-sync-tttqk\" (UID: \"c84c82ae-32a2-4965-9ad3-7eb90e03657c\") " pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-tttqk" Dec 03 11:51:53 crc kubenswrapper[5052]: I1203 11:51:53.401466 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/c84c82ae-32a2-4965-9ad3-7eb90e03657c-config-data\") pod \"nova-cell0-conductor-db-sync-tttqk\" (UID: \"c84c82ae-32a2-4965-9ad3-7eb90e03657c\") " pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-tttqk" Dec 03 11:51:53 crc kubenswrapper[5052]: I1203 11:51:53.416068 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-q7gkw\" (UniqueName: \"kubernetes.io/projected/c84c82ae-32a2-4965-9ad3-7eb90e03657c-kube-api-access-q7gkw\") pod \"nova-cell0-conductor-db-sync-tttqk\" (UID: \"c84c82ae-32a2-4965-9ad3-7eb90e03657c\") " pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-tttqk" Dec 03 11:51:53 crc kubenswrapper[5052]: I1203 11:51:53.475222 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-tttqk" Dec 03 11:51:53 crc kubenswrapper[5052]: I1203 11:51:53.686455 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f8c1750d-cf4a-49cc-a5ba-d40f19a6aad8" path="/var/lib/kubelet/pods/f8c1750d-cf4a-49cc-a5ba-d40f19a6aad8/volumes" Dec 03 11:51:53 crc kubenswrapper[5052]: I1203 11:51:53.814304 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"2b63c55a-e29f-45a8-8f5d-4166c162c3cb","Type":"ContainerStarted","Data":"2693072ee37641b8c8369038546f2b36325a5526bdc8ec1217de284d5fa41724"} Dec 03 11:51:53 crc kubenswrapper[5052]: I1203 11:51:53.975410 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-db-sync-tttqk"] Dec 03 11:51:53 crc kubenswrapper[5052]: W1203 11:51:53.985559 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc84c82ae_32a2_4965_9ad3_7eb90e03657c.slice/crio-a95c6fd939b49c661e3261a2a1d48d0e84df2a93d0a77a9c83f3f241440061ce WatchSource:0}: Error finding container a95c6fd939b49c661e3261a2a1d48d0e84df2a93d0a77a9c83f3f241440061ce: Status 404 returned error can't find the container with id a95c6fd939b49c661e3261a2a1d48d0e84df2a93d0a77a9c83f3f241440061ce Dec 03 11:51:54 crc kubenswrapper[5052]: I1203 11:51:54.828253 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-tttqk" event={"ID":"c84c82ae-32a2-4965-9ad3-7eb90e03657c","Type":"ContainerStarted","Data":"6abb759f1844b571cc5be4dcbd64ddf7b2892093b015739e89b0a35c6e169927"} Dec 03 11:51:54 crc kubenswrapper[5052]: I1203 11:51:54.828636 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-tttqk" event={"ID":"c84c82ae-32a2-4965-9ad3-7eb90e03657c","Type":"ContainerStarted","Data":"a95c6fd939b49c661e3261a2a1d48d0e84df2a93d0a77a9c83f3f241440061ce"} Dec 03 11:51:54 crc kubenswrapper[5052]: I1203 11:51:54.833658 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"2b63c55a-e29f-45a8-8f5d-4166c162c3cb","Type":"ContainerStarted","Data":"9cfba92208de175be52f5d19cc51bdcc6e1f1b25e40e4c3ce4647374108ea79a"} Dec 03 11:51:54 crc kubenswrapper[5052]: I1203 11:51:54.843705 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-tttqk" podStartSLOduration=1.8436808139999998 podStartE2EDuration="1.843680814s" podCreationTimestamp="2025-12-03 11:51:53 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:51:54.841424042 +0000 UTC m=+1905.429665806" watchObservedRunningTime="2025-12-03 11:51:54.843680814 +0000 UTC m=+1905.431922578" Dec 03 11:51:55 crc kubenswrapper[5052]: I1203 11:51:55.196377 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/neutron-5ff446f48b-7hc25" Dec 03 11:51:55 crc kubenswrapper[5052]: I1203 11:51:55.285170 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/neutron-68d4b65fff-k4c5r"] Dec 03 11:51:55 crc kubenswrapper[5052]: I1203 11:51:55.285441 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/neutron-68d4b65fff-k4c5r" podUID="d611c43f-d958-4cef-86d5-32298150487c" containerName="neutron-api" containerID="cri-o://654e271093b52e242fc7152c45ab8928f835c7293e10fd27319b88f10d29b882" gracePeriod=30 Dec 03 11:51:55 crc kubenswrapper[5052]: I1203 11:51:55.285897 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/neutron-68d4b65fff-k4c5r" podUID="d611c43f-d958-4cef-86d5-32298150487c" containerName="neutron-httpd" containerID="cri-o://281c11f92f4f6b8ca18c5f7fb7584ce5ead1606dbee8e1ff85ce5bd12ce1c76b" gracePeriod=30 Dec 03 11:51:55 crc kubenswrapper[5052]: I1203 11:51:55.844113 5052 generic.go:334] "Generic (PLEG): container finished" podID="d611c43f-d958-4cef-86d5-32298150487c" containerID="281c11f92f4f6b8ca18c5f7fb7584ce5ead1606dbee8e1ff85ce5bd12ce1c76b" exitCode=0 Dec 03 11:51:55 crc kubenswrapper[5052]: I1203 11:51:55.845363 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-68d4b65fff-k4c5r" event={"ID":"d611c43f-d958-4cef-86d5-32298150487c","Type":"ContainerDied","Data":"281c11f92f4f6b8ca18c5f7fb7584ce5ead1606dbee8e1ff85ce5bd12ce1c76b"} Dec 03 11:51:56 crc kubenswrapper[5052]: I1203 11:51:56.456954 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/swift-proxy-b9d8f996b-brs7h" Dec 03 11:51:56 crc kubenswrapper[5052]: I1203 11:51:56.457425 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/swift-proxy-b9d8f996b-brs7h" Dec 03 11:51:56 crc kubenswrapper[5052]: I1203 11:51:56.856379 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"2b63c55a-e29f-45a8-8f5d-4166c162c3cb","Type":"ContainerStarted","Data":"cfd665967d473e48dc565759604a79b625f48c47129f3c1507c5285ebf4de4e1"} Dec 03 11:51:59 crc kubenswrapper[5052]: I1203 11:51:59.907783 5052 generic.go:334] "Generic (PLEG): container finished" podID="d611c43f-d958-4cef-86d5-32298150487c" containerID="654e271093b52e242fc7152c45ab8928f835c7293e10fd27319b88f10d29b882" exitCode=0 Dec 03 11:51:59 crc kubenswrapper[5052]: I1203 11:51:59.907889 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-68d4b65fff-k4c5r" event={"ID":"d611c43f-d958-4cef-86d5-32298150487c","Type":"ContainerDied","Data":"654e271093b52e242fc7152c45ab8928f835c7293e10fd27319b88f10d29b882"} Dec 03 11:52:02 crc kubenswrapper[5052]: I1203 11:52:02.193637 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-68d4b65fff-k4c5r" Dec 03 11:52:02 crc kubenswrapper[5052]: I1203 11:52:02.304947 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/d611c43f-d958-4cef-86d5-32298150487c-httpd-config\") pod \"d611c43f-d958-4cef-86d5-32298150487c\" (UID: \"d611c43f-d958-4cef-86d5-32298150487c\") " Dec 03 11:52:02 crc kubenswrapper[5052]: I1203 11:52:02.305026 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d611c43f-d958-4cef-86d5-32298150487c-combined-ca-bundle\") pod \"d611c43f-d958-4cef-86d5-32298150487c\" (UID: \"d611c43f-d958-4cef-86d5-32298150487c\") " Dec 03 11:52:02 crc kubenswrapper[5052]: I1203 11:52:02.305099 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/d611c43f-d958-4cef-86d5-32298150487c-config\") pod \"d611c43f-d958-4cef-86d5-32298150487c\" (UID: \"d611c43f-d958-4cef-86d5-32298150487c\") " Dec 03 11:52:02 crc kubenswrapper[5052]: I1203 11:52:02.305273 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/d611c43f-d958-4cef-86d5-32298150487c-ovndb-tls-certs\") pod \"d611c43f-d958-4cef-86d5-32298150487c\" (UID: \"d611c43f-d958-4cef-86d5-32298150487c\") " Dec 03 11:52:02 crc kubenswrapper[5052]: I1203 11:52:02.305317 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n55xl\" (UniqueName: \"kubernetes.io/projected/d611c43f-d958-4cef-86d5-32298150487c-kube-api-access-n55xl\") pod \"d611c43f-d958-4cef-86d5-32298150487c\" (UID: \"d611c43f-d958-4cef-86d5-32298150487c\") " Dec 03 11:52:02 crc kubenswrapper[5052]: I1203 11:52:02.309495 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d611c43f-d958-4cef-86d5-32298150487c-httpd-config" (OuterVolumeSpecName: "httpd-config") pod "d611c43f-d958-4cef-86d5-32298150487c" (UID: "d611c43f-d958-4cef-86d5-32298150487c"). InnerVolumeSpecName "httpd-config". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:52:02 crc kubenswrapper[5052]: I1203 11:52:02.310267 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d611c43f-d958-4cef-86d5-32298150487c-kube-api-access-n55xl" (OuterVolumeSpecName: "kube-api-access-n55xl") pod "d611c43f-d958-4cef-86d5-32298150487c" (UID: "d611c43f-d958-4cef-86d5-32298150487c"). InnerVolumeSpecName "kube-api-access-n55xl". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:52:02 crc kubenswrapper[5052]: I1203 11:52:02.371386 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d611c43f-d958-4cef-86d5-32298150487c-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "d611c43f-d958-4cef-86d5-32298150487c" (UID: "d611c43f-d958-4cef-86d5-32298150487c"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:52:02 crc kubenswrapper[5052]: I1203 11:52:02.382744 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d611c43f-d958-4cef-86d5-32298150487c-config" (OuterVolumeSpecName: "config") pod "d611c43f-d958-4cef-86d5-32298150487c" (UID: "d611c43f-d958-4cef-86d5-32298150487c"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:52:02 crc kubenswrapper[5052]: I1203 11:52:02.397218 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d611c43f-d958-4cef-86d5-32298150487c-ovndb-tls-certs" (OuterVolumeSpecName: "ovndb-tls-certs") pod "d611c43f-d958-4cef-86d5-32298150487c" (UID: "d611c43f-d958-4cef-86d5-32298150487c"). InnerVolumeSpecName "ovndb-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:52:02 crc kubenswrapper[5052]: I1203 11:52:02.407473 5052 reconciler_common.go:293] "Volume detached for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/d611c43f-d958-4cef-86d5-32298150487c-ovndb-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 11:52:02 crc kubenswrapper[5052]: I1203 11:52:02.407513 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-n55xl\" (UniqueName: \"kubernetes.io/projected/d611c43f-d958-4cef-86d5-32298150487c-kube-api-access-n55xl\") on node \"crc\" DevicePath \"\"" Dec 03 11:52:02 crc kubenswrapper[5052]: I1203 11:52:02.407527 5052 reconciler_common.go:293] "Volume detached for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/d611c43f-d958-4cef-86d5-32298150487c-httpd-config\") on node \"crc\" DevicePath \"\"" Dec 03 11:52:02 crc kubenswrapper[5052]: I1203 11:52:02.407538 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d611c43f-d958-4cef-86d5-32298150487c-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:52:02 crc kubenswrapper[5052]: I1203 11:52:02.407568 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/d611c43f-d958-4cef-86d5-32298150487c-config\") on node \"crc\" DevicePath \"\"" Dec 03 11:52:02 crc kubenswrapper[5052]: I1203 11:52:02.944160 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"2b63c55a-e29f-45a8-8f5d-4166c162c3cb","Type":"ContainerStarted","Data":"607973c8134fdca1237afb09655b62bca7d33c020ad583289303870cfce7e9e2"} Dec 03 11:52:02 crc kubenswrapper[5052]: I1203 11:52:02.944394 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:52:02 crc kubenswrapper[5052]: I1203 11:52:02.947158 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-68d4b65fff-k4c5r" event={"ID":"d611c43f-d958-4cef-86d5-32298150487c","Type":"ContainerDied","Data":"08cf4bb00114f457940aa1dbde1e6d7d4d4450b512b0bd8ca885622ef40792b5"} Dec 03 11:52:02 crc kubenswrapper[5052]: I1203 11:52:02.947228 5052 scope.go:117] "RemoveContainer" containerID="281c11f92f4f6b8ca18c5f7fb7584ce5ead1606dbee8e1ff85ce5bd12ce1c76b" Dec 03 11:52:02 crc kubenswrapper[5052]: I1203 11:52:02.947411 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-68d4b65fff-k4c5r" Dec 03 11:52:02 crc kubenswrapper[5052]: I1203 11:52:02.973198 5052 scope.go:117] "RemoveContainer" containerID="654e271093b52e242fc7152c45ab8928f835c7293e10fd27319b88f10d29b882" Dec 03 11:52:02 crc kubenswrapper[5052]: I1203 11:52:02.994797 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/ceilometer-0" podStartSLOduration=2.40604748 podStartE2EDuration="11.99477472s" podCreationTimestamp="2025-12-03 11:51:51 +0000 UTC" firstStartedPulling="2025-12-03 11:51:52.708839079 +0000 UTC m=+1903.297080843" lastFinishedPulling="2025-12-03 11:52:02.297566319 +0000 UTC m=+1912.885808083" observedRunningTime="2025-12-03 11:52:02.982549938 +0000 UTC m=+1913.570791702" watchObservedRunningTime="2025-12-03 11:52:02.99477472 +0000 UTC m=+1913.583016494" Dec 03 11:52:03 crc kubenswrapper[5052]: I1203 11:52:03.016582 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/neutron-68d4b65fff-k4c5r"] Dec 03 11:52:03 crc kubenswrapper[5052]: I1203 11:52:03.034419 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/neutron-68d4b65fff-k4c5r"] Dec 03 11:52:03 crc kubenswrapper[5052]: I1203 11:52:03.514839 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:52:03 crc kubenswrapper[5052]: I1203 11:52:03.685070 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d611c43f-d958-4cef-86d5-32298150487c" path="/var/lib/kubelet/pods/d611c43f-d958-4cef-86d5-32298150487c/volumes" Dec 03 11:52:04 crc kubenswrapper[5052]: I1203 11:52:04.966164 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="2b63c55a-e29f-45a8-8f5d-4166c162c3cb" containerName="ceilometer-central-agent" containerID="cri-o://2693072ee37641b8c8369038546f2b36325a5526bdc8ec1217de284d5fa41724" gracePeriod=30 Dec 03 11:52:04 crc kubenswrapper[5052]: I1203 11:52:04.966216 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="2b63c55a-e29f-45a8-8f5d-4166c162c3cb" containerName="proxy-httpd" containerID="cri-o://607973c8134fdca1237afb09655b62bca7d33c020ad583289303870cfce7e9e2" gracePeriod=30 Dec 03 11:52:04 crc kubenswrapper[5052]: I1203 11:52:04.966266 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="2b63c55a-e29f-45a8-8f5d-4166c162c3cb" containerName="ceilometer-notification-agent" containerID="cri-o://9cfba92208de175be52f5d19cc51bdcc6e1f1b25e40e4c3ce4647374108ea79a" gracePeriod=30 Dec 03 11:52:04 crc kubenswrapper[5052]: I1203 11:52:04.966241 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="2b63c55a-e29f-45a8-8f5d-4166c162c3cb" containerName="sg-core" containerID="cri-o://cfd665967d473e48dc565759604a79b625f48c47129f3c1507c5285ebf4de4e1" gracePeriod=30 Dec 03 11:52:05 crc kubenswrapper[5052]: I1203 11:52:05.980823 5052 generic.go:334] "Generic (PLEG): container finished" podID="2b63c55a-e29f-45a8-8f5d-4166c162c3cb" containerID="607973c8134fdca1237afb09655b62bca7d33c020ad583289303870cfce7e9e2" exitCode=0 Dec 03 11:52:05 crc kubenswrapper[5052]: I1203 11:52:05.981152 5052 generic.go:334] "Generic (PLEG): container finished" podID="2b63c55a-e29f-45a8-8f5d-4166c162c3cb" containerID="cfd665967d473e48dc565759604a79b625f48c47129f3c1507c5285ebf4de4e1" exitCode=2 Dec 03 11:52:05 crc kubenswrapper[5052]: I1203 11:52:05.980897 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"2b63c55a-e29f-45a8-8f5d-4166c162c3cb","Type":"ContainerDied","Data":"607973c8134fdca1237afb09655b62bca7d33c020ad583289303870cfce7e9e2"} Dec 03 11:52:05 crc kubenswrapper[5052]: I1203 11:52:05.981207 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"2b63c55a-e29f-45a8-8f5d-4166c162c3cb","Type":"ContainerDied","Data":"cfd665967d473e48dc565759604a79b625f48c47129f3c1507c5285ebf4de4e1"} Dec 03 11:52:06 crc kubenswrapper[5052]: I1203 11:52:06.991901 5052 generic.go:334] "Generic (PLEG): container finished" podID="2b63c55a-e29f-45a8-8f5d-4166c162c3cb" containerID="9cfba92208de175be52f5d19cc51bdcc6e1f1b25e40e4c3ce4647374108ea79a" exitCode=0 Dec 03 11:52:06 crc kubenswrapper[5052]: I1203 11:52:06.991943 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"2b63c55a-e29f-45a8-8f5d-4166c162c3cb","Type":"ContainerDied","Data":"9cfba92208de175be52f5d19cc51bdcc6e1f1b25e40e4c3ce4647374108ea79a"} Dec 03 11:52:10 crc kubenswrapper[5052]: I1203 11:52:10.029848 5052 generic.go:334] "Generic (PLEG): container finished" podID="2b63c55a-e29f-45a8-8f5d-4166c162c3cb" containerID="2693072ee37641b8c8369038546f2b36325a5526bdc8ec1217de284d5fa41724" exitCode=0 Dec 03 11:52:10 crc kubenswrapper[5052]: I1203 11:52:10.029945 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"2b63c55a-e29f-45a8-8f5d-4166c162c3cb","Type":"ContainerDied","Data":"2693072ee37641b8c8369038546f2b36325a5526bdc8ec1217de284d5fa41724"} Dec 03 11:52:10 crc kubenswrapper[5052]: I1203 11:52:10.123111 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:52:10 crc kubenswrapper[5052]: I1203 11:52:10.263542 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/2b63c55a-e29f-45a8-8f5d-4166c162c3cb-scripts\") pod \"2b63c55a-e29f-45a8-8f5d-4166c162c3cb\" (UID: \"2b63c55a-e29f-45a8-8f5d-4166c162c3cb\") " Dec 03 11:52:10 crc kubenswrapper[5052]: I1203 11:52:10.263581 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4xxkf\" (UniqueName: \"kubernetes.io/projected/2b63c55a-e29f-45a8-8f5d-4166c162c3cb-kube-api-access-4xxkf\") pod \"2b63c55a-e29f-45a8-8f5d-4166c162c3cb\" (UID: \"2b63c55a-e29f-45a8-8f5d-4166c162c3cb\") " Dec 03 11:52:10 crc kubenswrapper[5052]: I1203 11:52:10.263698 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/2b63c55a-e29f-45a8-8f5d-4166c162c3cb-log-httpd\") pod \"2b63c55a-e29f-45a8-8f5d-4166c162c3cb\" (UID: \"2b63c55a-e29f-45a8-8f5d-4166c162c3cb\") " Dec 03 11:52:10 crc kubenswrapper[5052]: I1203 11:52:10.263776 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/2b63c55a-e29f-45a8-8f5d-4166c162c3cb-sg-core-conf-yaml\") pod \"2b63c55a-e29f-45a8-8f5d-4166c162c3cb\" (UID: \"2b63c55a-e29f-45a8-8f5d-4166c162c3cb\") " Dec 03 11:52:10 crc kubenswrapper[5052]: I1203 11:52:10.263810 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/2b63c55a-e29f-45a8-8f5d-4166c162c3cb-config-data\") pod \"2b63c55a-e29f-45a8-8f5d-4166c162c3cb\" (UID: \"2b63c55a-e29f-45a8-8f5d-4166c162c3cb\") " Dec 03 11:52:10 crc kubenswrapper[5052]: I1203 11:52:10.263903 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/2b63c55a-e29f-45a8-8f5d-4166c162c3cb-run-httpd\") pod \"2b63c55a-e29f-45a8-8f5d-4166c162c3cb\" (UID: \"2b63c55a-e29f-45a8-8f5d-4166c162c3cb\") " Dec 03 11:52:10 crc kubenswrapper[5052]: I1203 11:52:10.263952 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2b63c55a-e29f-45a8-8f5d-4166c162c3cb-combined-ca-bundle\") pod \"2b63c55a-e29f-45a8-8f5d-4166c162c3cb\" (UID: \"2b63c55a-e29f-45a8-8f5d-4166c162c3cb\") " Dec 03 11:52:10 crc kubenswrapper[5052]: I1203 11:52:10.264175 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2b63c55a-e29f-45a8-8f5d-4166c162c3cb-run-httpd" (OuterVolumeSpecName: "run-httpd") pod "2b63c55a-e29f-45a8-8f5d-4166c162c3cb" (UID: "2b63c55a-e29f-45a8-8f5d-4166c162c3cb"). InnerVolumeSpecName "run-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:52:10 crc kubenswrapper[5052]: I1203 11:52:10.264250 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2b63c55a-e29f-45a8-8f5d-4166c162c3cb-log-httpd" (OuterVolumeSpecName: "log-httpd") pod "2b63c55a-e29f-45a8-8f5d-4166c162c3cb" (UID: "2b63c55a-e29f-45a8-8f5d-4166c162c3cb"). InnerVolumeSpecName "log-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:52:10 crc kubenswrapper[5052]: I1203 11:52:10.264629 5052 reconciler_common.go:293] "Volume detached for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/2b63c55a-e29f-45a8-8f5d-4166c162c3cb-log-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 11:52:10 crc kubenswrapper[5052]: I1203 11:52:10.264649 5052 reconciler_common.go:293] "Volume detached for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/2b63c55a-e29f-45a8-8f5d-4166c162c3cb-run-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 11:52:10 crc kubenswrapper[5052]: I1203 11:52:10.268977 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2b63c55a-e29f-45a8-8f5d-4166c162c3cb-kube-api-access-4xxkf" (OuterVolumeSpecName: "kube-api-access-4xxkf") pod "2b63c55a-e29f-45a8-8f5d-4166c162c3cb" (UID: "2b63c55a-e29f-45a8-8f5d-4166c162c3cb"). InnerVolumeSpecName "kube-api-access-4xxkf". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:52:10 crc kubenswrapper[5052]: I1203 11:52:10.270022 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2b63c55a-e29f-45a8-8f5d-4166c162c3cb-scripts" (OuterVolumeSpecName: "scripts") pod "2b63c55a-e29f-45a8-8f5d-4166c162c3cb" (UID: "2b63c55a-e29f-45a8-8f5d-4166c162c3cb"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:52:10 crc kubenswrapper[5052]: I1203 11:52:10.293062 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2b63c55a-e29f-45a8-8f5d-4166c162c3cb-sg-core-conf-yaml" (OuterVolumeSpecName: "sg-core-conf-yaml") pod "2b63c55a-e29f-45a8-8f5d-4166c162c3cb" (UID: "2b63c55a-e29f-45a8-8f5d-4166c162c3cb"). InnerVolumeSpecName "sg-core-conf-yaml". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:52:10 crc kubenswrapper[5052]: I1203 11:52:10.346160 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2b63c55a-e29f-45a8-8f5d-4166c162c3cb-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "2b63c55a-e29f-45a8-8f5d-4166c162c3cb" (UID: "2b63c55a-e29f-45a8-8f5d-4166c162c3cb"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:52:10 crc kubenswrapper[5052]: I1203 11:52:10.366501 5052 reconciler_common.go:293] "Volume detached for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/2b63c55a-e29f-45a8-8f5d-4166c162c3cb-sg-core-conf-yaml\") on node \"crc\" DevicePath \"\"" Dec 03 11:52:10 crc kubenswrapper[5052]: I1203 11:52:10.366553 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2b63c55a-e29f-45a8-8f5d-4166c162c3cb-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:52:10 crc kubenswrapper[5052]: I1203 11:52:10.366567 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/2b63c55a-e29f-45a8-8f5d-4166c162c3cb-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:52:10 crc kubenswrapper[5052]: I1203 11:52:10.366580 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-4xxkf\" (UniqueName: \"kubernetes.io/projected/2b63c55a-e29f-45a8-8f5d-4166c162c3cb-kube-api-access-4xxkf\") on node \"crc\" DevicePath \"\"" Dec 03 11:52:10 crc kubenswrapper[5052]: I1203 11:52:10.373835 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2b63c55a-e29f-45a8-8f5d-4166c162c3cb-config-data" (OuterVolumeSpecName: "config-data") pod "2b63c55a-e29f-45a8-8f5d-4166c162c3cb" (UID: "2b63c55a-e29f-45a8-8f5d-4166c162c3cb"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:52:10 crc kubenswrapper[5052]: I1203 11:52:10.468042 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/2b63c55a-e29f-45a8-8f5d-4166c162c3cb-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:52:11 crc kubenswrapper[5052]: I1203 11:52:11.042778 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"2b63c55a-e29f-45a8-8f5d-4166c162c3cb","Type":"ContainerDied","Data":"8371c14a38dfdfc474cdbe89b3b61e22c5f74d517c4df4efd0c4444be88e5a4d"} Dec 03 11:52:11 crc kubenswrapper[5052]: I1203 11:52:11.042844 5052 scope.go:117] "RemoveContainer" containerID="607973c8134fdca1237afb09655b62bca7d33c020ad583289303870cfce7e9e2" Dec 03 11:52:11 crc kubenswrapper[5052]: I1203 11:52:11.042885 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:52:11 crc kubenswrapper[5052]: I1203 11:52:11.098587 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:52:11 crc kubenswrapper[5052]: I1203 11:52:11.099664 5052 scope.go:117] "RemoveContainer" containerID="cfd665967d473e48dc565759604a79b625f48c47129f3c1507c5285ebf4de4e1" Dec 03 11:52:11 crc kubenswrapper[5052]: I1203 11:52:11.108293 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:52:11 crc kubenswrapper[5052]: I1203 11:52:11.124973 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:52:11 crc kubenswrapper[5052]: E1203 11:52:11.125383 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="2b63c55a-e29f-45a8-8f5d-4166c162c3cb" containerName="sg-core" Dec 03 11:52:11 crc kubenswrapper[5052]: I1203 11:52:11.125402 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="2b63c55a-e29f-45a8-8f5d-4166c162c3cb" containerName="sg-core" Dec 03 11:52:11 crc kubenswrapper[5052]: E1203 11:52:11.125421 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="d611c43f-d958-4cef-86d5-32298150487c" containerName="neutron-api" Dec 03 11:52:11 crc kubenswrapper[5052]: I1203 11:52:11.125428 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="d611c43f-d958-4cef-86d5-32298150487c" containerName="neutron-api" Dec 03 11:52:11 crc kubenswrapper[5052]: E1203 11:52:11.125440 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="2b63c55a-e29f-45a8-8f5d-4166c162c3cb" containerName="ceilometer-central-agent" Dec 03 11:52:11 crc kubenswrapper[5052]: I1203 11:52:11.125446 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="2b63c55a-e29f-45a8-8f5d-4166c162c3cb" containerName="ceilometer-central-agent" Dec 03 11:52:11 crc kubenswrapper[5052]: E1203 11:52:11.125456 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="d611c43f-d958-4cef-86d5-32298150487c" containerName="neutron-httpd" Dec 03 11:52:11 crc kubenswrapper[5052]: I1203 11:52:11.125461 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="d611c43f-d958-4cef-86d5-32298150487c" containerName="neutron-httpd" Dec 03 11:52:11 crc kubenswrapper[5052]: E1203 11:52:11.125491 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="2b63c55a-e29f-45a8-8f5d-4166c162c3cb" containerName="ceilometer-notification-agent" Dec 03 11:52:11 crc kubenswrapper[5052]: I1203 11:52:11.134593 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="2b63c55a-e29f-45a8-8f5d-4166c162c3cb" containerName="ceilometer-notification-agent" Dec 03 11:52:11 crc kubenswrapper[5052]: E1203 11:52:11.134637 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="2b63c55a-e29f-45a8-8f5d-4166c162c3cb" containerName="proxy-httpd" Dec 03 11:52:11 crc kubenswrapper[5052]: I1203 11:52:11.134643 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="2b63c55a-e29f-45a8-8f5d-4166c162c3cb" containerName="proxy-httpd" Dec 03 11:52:11 crc kubenswrapper[5052]: I1203 11:52:11.133714 5052 scope.go:117] "RemoveContainer" containerID="9cfba92208de175be52f5d19cc51bdcc6e1f1b25e40e4c3ce4647374108ea79a" Dec 03 11:52:11 crc kubenswrapper[5052]: I1203 11:52:11.134966 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="2b63c55a-e29f-45a8-8f5d-4166c162c3cb" containerName="ceilometer-notification-agent" Dec 03 11:52:11 crc kubenswrapper[5052]: I1203 11:52:11.134982 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="2b63c55a-e29f-45a8-8f5d-4166c162c3cb" containerName="sg-core" Dec 03 11:52:11 crc kubenswrapper[5052]: I1203 11:52:11.135000 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="2b63c55a-e29f-45a8-8f5d-4166c162c3cb" containerName="ceilometer-central-agent" Dec 03 11:52:11 crc kubenswrapper[5052]: I1203 11:52:11.135016 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="d611c43f-d958-4cef-86d5-32298150487c" containerName="neutron-httpd" Dec 03 11:52:11 crc kubenswrapper[5052]: I1203 11:52:11.135036 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="2b63c55a-e29f-45a8-8f5d-4166c162c3cb" containerName="proxy-httpd" Dec 03 11:52:11 crc kubenswrapper[5052]: I1203 11:52:11.135045 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="d611c43f-d958-4cef-86d5-32298150487c" containerName="neutron-api" Dec 03 11:52:11 crc kubenswrapper[5052]: I1203 11:52:11.177856 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:52:11 crc kubenswrapper[5052]: I1203 11:52:11.182837 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-config-data" Dec 03 11:52:11 crc kubenswrapper[5052]: I1203 11:52:11.183155 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-scripts" Dec 03 11:52:11 crc kubenswrapper[5052]: I1203 11:52:11.206842 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:52:11 crc kubenswrapper[5052]: I1203 11:52:11.253745 5052 scope.go:117] "RemoveContainer" containerID="2693072ee37641b8c8369038546f2b36325a5526bdc8ec1217de284d5fa41724" Dec 03 11:52:11 crc kubenswrapper[5052]: I1203 11:52:11.283315 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/6409124d-6776-45c8-9a15-a96771e5fdc0-log-httpd\") pod \"ceilometer-0\" (UID: \"6409124d-6776-45c8-9a15-a96771e5fdc0\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:52:11 crc kubenswrapper[5052]: I1203 11:52:11.283381 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/6409124d-6776-45c8-9a15-a96771e5fdc0-run-httpd\") pod \"ceilometer-0\" (UID: \"6409124d-6776-45c8-9a15-a96771e5fdc0\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:52:11 crc kubenswrapper[5052]: I1203 11:52:11.283402 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/6409124d-6776-45c8-9a15-a96771e5fdc0-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"6409124d-6776-45c8-9a15-a96771e5fdc0\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:52:11 crc kubenswrapper[5052]: I1203 11:52:11.283440 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4lc8j\" (UniqueName: \"kubernetes.io/projected/6409124d-6776-45c8-9a15-a96771e5fdc0-kube-api-access-4lc8j\") pod \"ceilometer-0\" (UID: \"6409124d-6776-45c8-9a15-a96771e5fdc0\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:52:11 crc kubenswrapper[5052]: I1203 11:52:11.283488 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/6409124d-6776-45c8-9a15-a96771e5fdc0-scripts\") pod \"ceilometer-0\" (UID: \"6409124d-6776-45c8-9a15-a96771e5fdc0\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:52:11 crc kubenswrapper[5052]: I1203 11:52:11.283529 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6409124d-6776-45c8-9a15-a96771e5fdc0-config-data\") pod \"ceilometer-0\" (UID: \"6409124d-6776-45c8-9a15-a96771e5fdc0\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:52:11 crc kubenswrapper[5052]: I1203 11:52:11.283551 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6409124d-6776-45c8-9a15-a96771e5fdc0-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"6409124d-6776-45c8-9a15-a96771e5fdc0\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:52:11 crc kubenswrapper[5052]: I1203 11:52:11.385250 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-4lc8j\" (UniqueName: \"kubernetes.io/projected/6409124d-6776-45c8-9a15-a96771e5fdc0-kube-api-access-4lc8j\") pod \"ceilometer-0\" (UID: \"6409124d-6776-45c8-9a15-a96771e5fdc0\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:52:11 crc kubenswrapper[5052]: I1203 11:52:11.385333 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/6409124d-6776-45c8-9a15-a96771e5fdc0-scripts\") pod \"ceilometer-0\" (UID: \"6409124d-6776-45c8-9a15-a96771e5fdc0\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:52:11 crc kubenswrapper[5052]: I1203 11:52:11.385364 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6409124d-6776-45c8-9a15-a96771e5fdc0-config-data\") pod \"ceilometer-0\" (UID: \"6409124d-6776-45c8-9a15-a96771e5fdc0\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:52:11 crc kubenswrapper[5052]: I1203 11:52:11.385386 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6409124d-6776-45c8-9a15-a96771e5fdc0-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"6409124d-6776-45c8-9a15-a96771e5fdc0\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:52:11 crc kubenswrapper[5052]: I1203 11:52:11.385454 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/6409124d-6776-45c8-9a15-a96771e5fdc0-log-httpd\") pod \"ceilometer-0\" (UID: \"6409124d-6776-45c8-9a15-a96771e5fdc0\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:52:11 crc kubenswrapper[5052]: I1203 11:52:11.385484 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/6409124d-6776-45c8-9a15-a96771e5fdc0-run-httpd\") pod \"ceilometer-0\" (UID: \"6409124d-6776-45c8-9a15-a96771e5fdc0\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:52:11 crc kubenswrapper[5052]: I1203 11:52:11.385500 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/6409124d-6776-45c8-9a15-a96771e5fdc0-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"6409124d-6776-45c8-9a15-a96771e5fdc0\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:52:11 crc kubenswrapper[5052]: I1203 11:52:11.386160 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/6409124d-6776-45c8-9a15-a96771e5fdc0-log-httpd\") pod \"ceilometer-0\" (UID: \"6409124d-6776-45c8-9a15-a96771e5fdc0\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:52:11 crc kubenswrapper[5052]: I1203 11:52:11.386165 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/6409124d-6776-45c8-9a15-a96771e5fdc0-run-httpd\") pod \"ceilometer-0\" (UID: \"6409124d-6776-45c8-9a15-a96771e5fdc0\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:52:11 crc kubenswrapper[5052]: I1203 11:52:11.392320 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6409124d-6776-45c8-9a15-a96771e5fdc0-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"6409124d-6776-45c8-9a15-a96771e5fdc0\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:52:11 crc kubenswrapper[5052]: I1203 11:52:11.392330 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/6409124d-6776-45c8-9a15-a96771e5fdc0-scripts\") pod \"ceilometer-0\" (UID: \"6409124d-6776-45c8-9a15-a96771e5fdc0\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:52:11 crc kubenswrapper[5052]: I1203 11:52:11.393083 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6409124d-6776-45c8-9a15-a96771e5fdc0-config-data\") pod \"ceilometer-0\" (UID: \"6409124d-6776-45c8-9a15-a96771e5fdc0\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:52:11 crc kubenswrapper[5052]: I1203 11:52:11.403027 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/6409124d-6776-45c8-9a15-a96771e5fdc0-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"6409124d-6776-45c8-9a15-a96771e5fdc0\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:52:11 crc kubenswrapper[5052]: I1203 11:52:11.409244 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-4lc8j\" (UniqueName: \"kubernetes.io/projected/6409124d-6776-45c8-9a15-a96771e5fdc0-kube-api-access-4lc8j\") pod \"ceilometer-0\" (UID: \"6409124d-6776-45c8-9a15-a96771e5fdc0\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:52:11 crc kubenswrapper[5052]: I1203 11:52:11.553868 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:52:11 crc kubenswrapper[5052]: I1203 11:52:11.683602 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2b63c55a-e29f-45a8-8f5d-4166c162c3cb" path="/var/lib/kubelet/pods/2b63c55a-e29f-45a8-8f5d-4166c162c3cb/volumes" Dec 03 11:52:12 crc kubenswrapper[5052]: I1203 11:52:12.057286 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:52:13 crc kubenswrapper[5052]: I1203 11:52:13.071399 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"6409124d-6776-45c8-9a15-a96771e5fdc0","Type":"ContainerStarted","Data":"d70b151307207f9c1fc099e2a6ed8895a49b5da5d712333d4b90d7a7d4de82dd"} Dec 03 11:52:14 crc kubenswrapper[5052]: I1203 11:52:14.091127 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"6409124d-6776-45c8-9a15-a96771e5fdc0","Type":"ContainerStarted","Data":"4ed7a40265188a06aceab89b1d6ec5a6368e1490d257b4dcc9660cb8319e4d81"} Dec 03 11:52:15 crc kubenswrapper[5052]: I1203 11:52:15.101057 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"6409124d-6776-45c8-9a15-a96771e5fdc0","Type":"ContainerStarted","Data":"7f99c2eea2ed8331b29ec6d6324f201e6ebb4ae6f71c9cf21e24b8b48961ed39"} Dec 03 11:52:16 crc kubenswrapper[5052]: I1203 11:52:16.110644 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"6409124d-6776-45c8-9a15-a96771e5fdc0","Type":"ContainerStarted","Data":"06ed60c6e80c4e7a7c82dba95212d5392a25092552f92b492b2ae1d402182b62"} Dec 03 11:52:16 crc kubenswrapper[5052]: E1203 11:52:16.170975 5052 cadvisor_stats_provider.go:516] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2b63c55a_e29f_45a8_8f5d_4166c162c3cb.slice/crio-conmon-2693072ee37641b8c8369038546f2b36325a5526bdc8ec1217de284d5fa41724.scope\": RecentStats: unable to find data in memory cache]" Dec 03 11:52:17 crc kubenswrapper[5052]: I1203 11:52:17.121077 5052 generic.go:334] "Generic (PLEG): container finished" podID="c84c82ae-32a2-4965-9ad3-7eb90e03657c" containerID="6abb759f1844b571cc5be4dcbd64ddf7b2892093b015739e89b0a35c6e169927" exitCode=0 Dec 03 11:52:17 crc kubenswrapper[5052]: I1203 11:52:17.121120 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-tttqk" event={"ID":"c84c82ae-32a2-4965-9ad3-7eb90e03657c","Type":"ContainerDied","Data":"6abb759f1844b571cc5be4dcbd64ddf7b2892093b015739e89b0a35c6e169927"} Dec 03 11:52:18 crc kubenswrapper[5052]: I1203 11:52:18.058045 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:52:18 crc kubenswrapper[5052]: I1203 11:52:18.131648 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"6409124d-6776-45c8-9a15-a96771e5fdc0","Type":"ContainerStarted","Data":"7cd2751dcd1fdf109337202bccddbfe91e1a63d9b3956770539ce38cc518a500"} Dec 03 11:52:18 crc kubenswrapper[5052]: I1203 11:52:18.160957 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/ceilometer-0" podStartSLOduration=2.290147072 podStartE2EDuration="7.160935557s" podCreationTimestamp="2025-12-03 11:52:11 +0000 UTC" firstStartedPulling="2025-12-03 11:52:12.055242551 +0000 UTC m=+1922.643484315" lastFinishedPulling="2025-12-03 11:52:16.926031036 +0000 UTC m=+1927.514272800" observedRunningTime="2025-12-03 11:52:18.148761817 +0000 UTC m=+1928.737003591" watchObservedRunningTime="2025-12-03 11:52:18.160935557 +0000 UTC m=+1928.749177331" Dec 03 11:52:18 crc kubenswrapper[5052]: I1203 11:52:18.507299 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-tttqk" Dec 03 11:52:18 crc kubenswrapper[5052]: I1203 11:52:18.618413 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q7gkw\" (UniqueName: \"kubernetes.io/projected/c84c82ae-32a2-4965-9ad3-7eb90e03657c-kube-api-access-q7gkw\") pod \"c84c82ae-32a2-4965-9ad3-7eb90e03657c\" (UID: \"c84c82ae-32a2-4965-9ad3-7eb90e03657c\") " Dec 03 11:52:18 crc kubenswrapper[5052]: I1203 11:52:18.618617 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/c84c82ae-32a2-4965-9ad3-7eb90e03657c-scripts\") pod \"c84c82ae-32a2-4965-9ad3-7eb90e03657c\" (UID: \"c84c82ae-32a2-4965-9ad3-7eb90e03657c\") " Dec 03 11:52:18 crc kubenswrapper[5052]: I1203 11:52:18.618666 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/c84c82ae-32a2-4965-9ad3-7eb90e03657c-config-data\") pod \"c84c82ae-32a2-4965-9ad3-7eb90e03657c\" (UID: \"c84c82ae-32a2-4965-9ad3-7eb90e03657c\") " Dec 03 11:52:18 crc kubenswrapper[5052]: I1203 11:52:18.618761 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c84c82ae-32a2-4965-9ad3-7eb90e03657c-combined-ca-bundle\") pod \"c84c82ae-32a2-4965-9ad3-7eb90e03657c\" (UID: \"c84c82ae-32a2-4965-9ad3-7eb90e03657c\") " Dec 03 11:52:18 crc kubenswrapper[5052]: I1203 11:52:18.624135 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c84c82ae-32a2-4965-9ad3-7eb90e03657c-scripts" (OuterVolumeSpecName: "scripts") pod "c84c82ae-32a2-4965-9ad3-7eb90e03657c" (UID: "c84c82ae-32a2-4965-9ad3-7eb90e03657c"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:52:18 crc kubenswrapper[5052]: I1203 11:52:18.636112 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c84c82ae-32a2-4965-9ad3-7eb90e03657c-kube-api-access-q7gkw" (OuterVolumeSpecName: "kube-api-access-q7gkw") pod "c84c82ae-32a2-4965-9ad3-7eb90e03657c" (UID: "c84c82ae-32a2-4965-9ad3-7eb90e03657c"). InnerVolumeSpecName "kube-api-access-q7gkw". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:52:18 crc kubenswrapper[5052]: I1203 11:52:18.653601 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c84c82ae-32a2-4965-9ad3-7eb90e03657c-config-data" (OuterVolumeSpecName: "config-data") pod "c84c82ae-32a2-4965-9ad3-7eb90e03657c" (UID: "c84c82ae-32a2-4965-9ad3-7eb90e03657c"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:52:18 crc kubenswrapper[5052]: I1203 11:52:18.654923 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c84c82ae-32a2-4965-9ad3-7eb90e03657c-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "c84c82ae-32a2-4965-9ad3-7eb90e03657c" (UID: "c84c82ae-32a2-4965-9ad3-7eb90e03657c"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:52:18 crc kubenswrapper[5052]: I1203 11:52:18.721489 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c84c82ae-32a2-4965-9ad3-7eb90e03657c-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:52:18 crc kubenswrapper[5052]: I1203 11:52:18.721535 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-q7gkw\" (UniqueName: \"kubernetes.io/projected/c84c82ae-32a2-4965-9ad3-7eb90e03657c-kube-api-access-q7gkw\") on node \"crc\" DevicePath \"\"" Dec 03 11:52:18 crc kubenswrapper[5052]: I1203 11:52:18.721549 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/c84c82ae-32a2-4965-9ad3-7eb90e03657c-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:52:18 crc kubenswrapper[5052]: I1203 11:52:18.721559 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/c84c82ae-32a2-4965-9ad3-7eb90e03657c-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:52:18 crc kubenswrapper[5052]: I1203 11:52:18.819109 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 11:52:18 crc kubenswrapper[5052]: I1203 11:52:18.819350 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-external-api-0" podUID="5983840b-20da-49d2-aaa4-7be0eb36b0c4" containerName="glance-log" containerID="cri-o://6ed3a69c28ece77aa5d36034ff2cca030b4a7f6251d240bbd51afcf3f4a9773d" gracePeriod=30 Dec 03 11:52:18 crc kubenswrapper[5052]: I1203 11:52:18.819442 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-external-api-0" podUID="5983840b-20da-49d2-aaa4-7be0eb36b0c4" containerName="glance-httpd" containerID="cri-o://6d7e2f9948e6e8fb4bfee608b3eeaee15339ef55c9a1c50f68aa577ee7ae17b4" gracePeriod=30 Dec 03 11:52:19 crc kubenswrapper[5052]: I1203 11:52:19.141326 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-tttqk" event={"ID":"c84c82ae-32a2-4965-9ad3-7eb90e03657c","Type":"ContainerDied","Data":"a95c6fd939b49c661e3261a2a1d48d0e84df2a93d0a77a9c83f3f241440061ce"} Dec 03 11:52:19 crc kubenswrapper[5052]: I1203 11:52:19.141371 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-tttqk" Dec 03 11:52:19 crc kubenswrapper[5052]: I1203 11:52:19.141382 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a95c6fd939b49c661e3261a2a1d48d0e84df2a93d0a77a9c83f3f241440061ce" Dec 03 11:52:19 crc kubenswrapper[5052]: I1203 11:52:19.144093 5052 generic.go:334] "Generic (PLEG): container finished" podID="5983840b-20da-49d2-aaa4-7be0eb36b0c4" containerID="6ed3a69c28ece77aa5d36034ff2cca030b4a7f6251d240bbd51afcf3f4a9773d" exitCode=143 Dec 03 11:52:19 crc kubenswrapper[5052]: I1203 11:52:19.144332 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="6409124d-6776-45c8-9a15-a96771e5fdc0" containerName="ceilometer-central-agent" containerID="cri-o://4ed7a40265188a06aceab89b1d6ec5a6368e1490d257b4dcc9660cb8319e4d81" gracePeriod=30 Dec 03 11:52:19 crc kubenswrapper[5052]: I1203 11:52:19.144627 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"5983840b-20da-49d2-aaa4-7be0eb36b0c4","Type":"ContainerDied","Data":"6ed3a69c28ece77aa5d36034ff2cca030b4a7f6251d240bbd51afcf3f4a9773d"} Dec 03 11:52:19 crc kubenswrapper[5052]: I1203 11:52:19.144687 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:52:19 crc kubenswrapper[5052]: I1203 11:52:19.144951 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="6409124d-6776-45c8-9a15-a96771e5fdc0" containerName="proxy-httpd" containerID="cri-o://7cd2751dcd1fdf109337202bccddbfe91e1a63d9b3956770539ce38cc518a500" gracePeriod=30 Dec 03 11:52:19 crc kubenswrapper[5052]: I1203 11:52:19.145002 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="6409124d-6776-45c8-9a15-a96771e5fdc0" containerName="sg-core" containerID="cri-o://06ed60c6e80c4e7a7c82dba95212d5392a25092552f92b492b2ae1d402182b62" gracePeriod=30 Dec 03 11:52:19 crc kubenswrapper[5052]: I1203 11:52:19.145038 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="6409124d-6776-45c8-9a15-a96771e5fdc0" containerName="ceilometer-notification-agent" containerID="cri-o://7f99c2eea2ed8331b29ec6d6324f201e6ebb4ae6f71c9cf21e24b8b48961ed39" gracePeriod=30 Dec 03 11:52:19 crc kubenswrapper[5052]: I1203 11:52:19.241584 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-0"] Dec 03 11:52:19 crc kubenswrapper[5052]: E1203 11:52:19.243151 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="c84c82ae-32a2-4965-9ad3-7eb90e03657c" containerName="nova-cell0-conductor-db-sync" Dec 03 11:52:19 crc kubenswrapper[5052]: I1203 11:52:19.243235 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="c84c82ae-32a2-4965-9ad3-7eb90e03657c" containerName="nova-cell0-conductor-db-sync" Dec 03 11:52:19 crc kubenswrapper[5052]: I1203 11:52:19.243577 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="c84c82ae-32a2-4965-9ad3-7eb90e03657c" containerName="nova-cell0-conductor-db-sync" Dec 03 11:52:19 crc kubenswrapper[5052]: I1203 11:52:19.244428 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 11:52:19 crc kubenswrapper[5052]: I1203 11:52:19.246806 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell0-conductor-config-data" Dec 03 11:52:19 crc kubenswrapper[5052]: I1203 11:52:19.249406 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-nova-dockercfg-8c458" Dec 03 11:52:19 crc kubenswrapper[5052]: I1203 11:52:19.272786 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-0"] Dec 03 11:52:19 crc kubenswrapper[5052]: I1203 11:52:19.332004 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/9f1ae28a-0533-4433-9897-741a5d998529-config-data\") pod \"nova-cell0-conductor-0\" (UID: \"9f1ae28a-0533-4433-9897-741a5d998529\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 11:52:19 crc kubenswrapper[5052]: I1203 11:52:19.332360 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9f1ae28a-0533-4433-9897-741a5d998529-combined-ca-bundle\") pod \"nova-cell0-conductor-0\" (UID: \"9f1ae28a-0533-4433-9897-741a5d998529\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 11:52:19 crc kubenswrapper[5052]: I1203 11:52:19.332788 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n9z96\" (UniqueName: \"kubernetes.io/projected/9f1ae28a-0533-4433-9897-741a5d998529-kube-api-access-n9z96\") pod \"nova-cell0-conductor-0\" (UID: \"9f1ae28a-0533-4433-9897-741a5d998529\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 11:52:19 crc kubenswrapper[5052]: I1203 11:52:19.436875 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/9f1ae28a-0533-4433-9897-741a5d998529-config-data\") pod \"nova-cell0-conductor-0\" (UID: \"9f1ae28a-0533-4433-9897-741a5d998529\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 11:52:19 crc kubenswrapper[5052]: I1203 11:52:19.436923 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9f1ae28a-0533-4433-9897-741a5d998529-combined-ca-bundle\") pod \"nova-cell0-conductor-0\" (UID: \"9f1ae28a-0533-4433-9897-741a5d998529\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 11:52:19 crc kubenswrapper[5052]: I1203 11:52:19.437046 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-n9z96\" (UniqueName: \"kubernetes.io/projected/9f1ae28a-0533-4433-9897-741a5d998529-kube-api-access-n9z96\") pod \"nova-cell0-conductor-0\" (UID: \"9f1ae28a-0533-4433-9897-741a5d998529\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 11:52:19 crc kubenswrapper[5052]: I1203 11:52:19.451541 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9f1ae28a-0533-4433-9897-741a5d998529-combined-ca-bundle\") pod \"nova-cell0-conductor-0\" (UID: \"9f1ae28a-0533-4433-9897-741a5d998529\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 11:52:19 crc kubenswrapper[5052]: I1203 11:52:19.452700 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/9f1ae28a-0533-4433-9897-741a5d998529-config-data\") pod \"nova-cell0-conductor-0\" (UID: \"9f1ae28a-0533-4433-9897-741a5d998529\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 11:52:19 crc kubenswrapper[5052]: I1203 11:52:19.486075 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-n9z96\" (UniqueName: \"kubernetes.io/projected/9f1ae28a-0533-4433-9897-741a5d998529-kube-api-access-n9z96\") pod \"nova-cell0-conductor-0\" (UID: \"9f1ae28a-0533-4433-9897-741a5d998529\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 11:52:19 crc kubenswrapper[5052]: I1203 11:52:19.604442 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 11:52:20 crc kubenswrapper[5052]: I1203 11:52:20.016378 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-0"] Dec 03 11:52:20 crc kubenswrapper[5052]: I1203 11:52:20.058339 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 11:52:20 crc kubenswrapper[5052]: I1203 11:52:20.058667 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-internal-api-0" podUID="760038e5-c49a-4c9e-b236-a4b98d58720e" containerName="glance-log" containerID="cri-o://3087941b5ebefca5a16d78bd326e84502e4ef383a38ad5c7e648b61e640cc09f" gracePeriod=30 Dec 03 11:52:20 crc kubenswrapper[5052]: I1203 11:52:20.058798 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-internal-api-0" podUID="760038e5-c49a-4c9e-b236-a4b98d58720e" containerName="glance-httpd" containerID="cri-o://3d55abecfc0f796ab8e6c6031b93876ae418a7296b5273ed839b6fe1ac3cc9eb" gracePeriod=30 Dec 03 11:52:20 crc kubenswrapper[5052]: I1203 11:52:20.081338 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-0"] Dec 03 11:52:20 crc kubenswrapper[5052]: I1203 11:52:20.162709 5052 generic.go:334] "Generic (PLEG): container finished" podID="6409124d-6776-45c8-9a15-a96771e5fdc0" containerID="7cd2751dcd1fdf109337202bccddbfe91e1a63d9b3956770539ce38cc518a500" exitCode=0 Dec 03 11:52:20 crc kubenswrapper[5052]: I1203 11:52:20.163017 5052 generic.go:334] "Generic (PLEG): container finished" podID="6409124d-6776-45c8-9a15-a96771e5fdc0" containerID="06ed60c6e80c4e7a7c82dba95212d5392a25092552f92b492b2ae1d402182b62" exitCode=2 Dec 03 11:52:20 crc kubenswrapper[5052]: I1203 11:52:20.163031 5052 generic.go:334] "Generic (PLEG): container finished" podID="6409124d-6776-45c8-9a15-a96771e5fdc0" containerID="7f99c2eea2ed8331b29ec6d6324f201e6ebb4ae6f71c9cf21e24b8b48961ed39" exitCode=0 Dec 03 11:52:20 crc kubenswrapper[5052]: I1203 11:52:20.163040 5052 generic.go:334] "Generic (PLEG): container finished" podID="6409124d-6776-45c8-9a15-a96771e5fdc0" containerID="4ed7a40265188a06aceab89b1d6ec5a6368e1490d257b4dcc9660cb8319e4d81" exitCode=0 Dec 03 11:52:20 crc kubenswrapper[5052]: I1203 11:52:20.163120 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"6409124d-6776-45c8-9a15-a96771e5fdc0","Type":"ContainerDied","Data":"7cd2751dcd1fdf109337202bccddbfe91e1a63d9b3956770539ce38cc518a500"} Dec 03 11:52:20 crc kubenswrapper[5052]: I1203 11:52:20.163180 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"6409124d-6776-45c8-9a15-a96771e5fdc0","Type":"ContainerDied","Data":"06ed60c6e80c4e7a7c82dba95212d5392a25092552f92b492b2ae1d402182b62"} Dec 03 11:52:20 crc kubenswrapper[5052]: I1203 11:52:20.163199 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"6409124d-6776-45c8-9a15-a96771e5fdc0","Type":"ContainerDied","Data":"7f99c2eea2ed8331b29ec6d6324f201e6ebb4ae6f71c9cf21e24b8b48961ed39"} Dec 03 11:52:20 crc kubenswrapper[5052]: I1203 11:52:20.163212 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"6409124d-6776-45c8-9a15-a96771e5fdc0","Type":"ContainerDied","Data":"4ed7a40265188a06aceab89b1d6ec5a6368e1490d257b4dcc9660cb8319e4d81"} Dec 03 11:52:20 crc kubenswrapper[5052]: I1203 11:52:20.165338 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-conductor-0" event={"ID":"9f1ae28a-0533-4433-9897-741a5d998529","Type":"ContainerStarted","Data":"8d528a4478d2c3957b506756b35b44300d544505e56daccc465085f986f858de"} Dec 03 11:52:20 crc kubenswrapper[5052]: I1203 11:52:20.313438 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:52:20 crc kubenswrapper[5052]: I1203 11:52:20.353285 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/6409124d-6776-45c8-9a15-a96771e5fdc0-scripts\") pod \"6409124d-6776-45c8-9a15-a96771e5fdc0\" (UID: \"6409124d-6776-45c8-9a15-a96771e5fdc0\") " Dec 03 11:52:20 crc kubenswrapper[5052]: I1203 11:52:20.353338 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4lc8j\" (UniqueName: \"kubernetes.io/projected/6409124d-6776-45c8-9a15-a96771e5fdc0-kube-api-access-4lc8j\") pod \"6409124d-6776-45c8-9a15-a96771e5fdc0\" (UID: \"6409124d-6776-45c8-9a15-a96771e5fdc0\") " Dec 03 11:52:20 crc kubenswrapper[5052]: I1203 11:52:20.353401 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/6409124d-6776-45c8-9a15-a96771e5fdc0-run-httpd\") pod \"6409124d-6776-45c8-9a15-a96771e5fdc0\" (UID: \"6409124d-6776-45c8-9a15-a96771e5fdc0\") " Dec 03 11:52:20 crc kubenswrapper[5052]: I1203 11:52:20.353501 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6409124d-6776-45c8-9a15-a96771e5fdc0-combined-ca-bundle\") pod \"6409124d-6776-45c8-9a15-a96771e5fdc0\" (UID: \"6409124d-6776-45c8-9a15-a96771e5fdc0\") " Dec 03 11:52:20 crc kubenswrapper[5052]: I1203 11:52:20.353583 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/6409124d-6776-45c8-9a15-a96771e5fdc0-log-httpd\") pod \"6409124d-6776-45c8-9a15-a96771e5fdc0\" (UID: \"6409124d-6776-45c8-9a15-a96771e5fdc0\") " Dec 03 11:52:20 crc kubenswrapper[5052]: I1203 11:52:20.353599 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6409124d-6776-45c8-9a15-a96771e5fdc0-config-data\") pod \"6409124d-6776-45c8-9a15-a96771e5fdc0\" (UID: \"6409124d-6776-45c8-9a15-a96771e5fdc0\") " Dec 03 11:52:20 crc kubenswrapper[5052]: I1203 11:52:20.353667 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/6409124d-6776-45c8-9a15-a96771e5fdc0-sg-core-conf-yaml\") pod \"6409124d-6776-45c8-9a15-a96771e5fdc0\" (UID: \"6409124d-6776-45c8-9a15-a96771e5fdc0\") " Dec 03 11:52:20 crc kubenswrapper[5052]: I1203 11:52:20.354404 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6409124d-6776-45c8-9a15-a96771e5fdc0-run-httpd" (OuterVolumeSpecName: "run-httpd") pod "6409124d-6776-45c8-9a15-a96771e5fdc0" (UID: "6409124d-6776-45c8-9a15-a96771e5fdc0"). InnerVolumeSpecName "run-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:52:20 crc kubenswrapper[5052]: I1203 11:52:20.357011 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6409124d-6776-45c8-9a15-a96771e5fdc0-log-httpd" (OuterVolumeSpecName: "log-httpd") pod "6409124d-6776-45c8-9a15-a96771e5fdc0" (UID: "6409124d-6776-45c8-9a15-a96771e5fdc0"). InnerVolumeSpecName "log-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:52:20 crc kubenswrapper[5052]: I1203 11:52:20.359421 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6409124d-6776-45c8-9a15-a96771e5fdc0-scripts" (OuterVolumeSpecName: "scripts") pod "6409124d-6776-45c8-9a15-a96771e5fdc0" (UID: "6409124d-6776-45c8-9a15-a96771e5fdc0"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:52:20 crc kubenswrapper[5052]: I1203 11:52:20.364897 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6409124d-6776-45c8-9a15-a96771e5fdc0-kube-api-access-4lc8j" (OuterVolumeSpecName: "kube-api-access-4lc8j") pod "6409124d-6776-45c8-9a15-a96771e5fdc0" (UID: "6409124d-6776-45c8-9a15-a96771e5fdc0"). InnerVolumeSpecName "kube-api-access-4lc8j". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:52:20 crc kubenswrapper[5052]: I1203 11:52:20.390899 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6409124d-6776-45c8-9a15-a96771e5fdc0-sg-core-conf-yaml" (OuterVolumeSpecName: "sg-core-conf-yaml") pod "6409124d-6776-45c8-9a15-a96771e5fdc0" (UID: "6409124d-6776-45c8-9a15-a96771e5fdc0"). InnerVolumeSpecName "sg-core-conf-yaml". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:52:20 crc kubenswrapper[5052]: I1203 11:52:20.448906 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6409124d-6776-45c8-9a15-a96771e5fdc0-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "6409124d-6776-45c8-9a15-a96771e5fdc0" (UID: "6409124d-6776-45c8-9a15-a96771e5fdc0"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:52:20 crc kubenswrapper[5052]: I1203 11:52:20.456303 5052 reconciler_common.go:293] "Volume detached for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/6409124d-6776-45c8-9a15-a96771e5fdc0-run-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 11:52:20 crc kubenswrapper[5052]: I1203 11:52:20.456344 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6409124d-6776-45c8-9a15-a96771e5fdc0-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:52:20 crc kubenswrapper[5052]: I1203 11:52:20.456360 5052 reconciler_common.go:293] "Volume detached for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/6409124d-6776-45c8-9a15-a96771e5fdc0-log-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 11:52:20 crc kubenswrapper[5052]: I1203 11:52:20.456373 5052 reconciler_common.go:293] "Volume detached for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/6409124d-6776-45c8-9a15-a96771e5fdc0-sg-core-conf-yaml\") on node \"crc\" DevicePath \"\"" Dec 03 11:52:20 crc kubenswrapper[5052]: I1203 11:52:20.456384 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/6409124d-6776-45c8-9a15-a96771e5fdc0-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:52:20 crc kubenswrapper[5052]: I1203 11:52:20.456397 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-4lc8j\" (UniqueName: \"kubernetes.io/projected/6409124d-6776-45c8-9a15-a96771e5fdc0-kube-api-access-4lc8j\") on node \"crc\" DevicePath \"\"" Dec 03 11:52:20 crc kubenswrapper[5052]: I1203 11:52:20.483792 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6409124d-6776-45c8-9a15-a96771e5fdc0-config-data" (OuterVolumeSpecName: "config-data") pod "6409124d-6776-45c8-9a15-a96771e5fdc0" (UID: "6409124d-6776-45c8-9a15-a96771e5fdc0"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:52:20 crc kubenswrapper[5052]: I1203 11:52:20.558412 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6409124d-6776-45c8-9a15-a96771e5fdc0-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:52:21 crc kubenswrapper[5052]: I1203 11:52:21.175241 5052 generic.go:334] "Generic (PLEG): container finished" podID="760038e5-c49a-4c9e-b236-a4b98d58720e" containerID="3087941b5ebefca5a16d78bd326e84502e4ef383a38ad5c7e648b61e640cc09f" exitCode=143 Dec 03 11:52:21 crc kubenswrapper[5052]: I1203 11:52:21.175336 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"760038e5-c49a-4c9e-b236-a4b98d58720e","Type":"ContainerDied","Data":"3087941b5ebefca5a16d78bd326e84502e4ef383a38ad5c7e648b61e640cc09f"} Dec 03 11:52:21 crc kubenswrapper[5052]: I1203 11:52:21.179778 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"6409124d-6776-45c8-9a15-a96771e5fdc0","Type":"ContainerDied","Data":"d70b151307207f9c1fc099e2a6ed8895a49b5da5d712333d4b90d7a7d4de82dd"} Dec 03 11:52:21 crc kubenswrapper[5052]: I1203 11:52:21.179816 5052 scope.go:117] "RemoveContainer" containerID="7cd2751dcd1fdf109337202bccddbfe91e1a63d9b3956770539ce38cc518a500" Dec 03 11:52:21 crc kubenswrapper[5052]: I1203 11:52:21.179943 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:52:21 crc kubenswrapper[5052]: I1203 11:52:21.185912 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-conductor-0" event={"ID":"9f1ae28a-0533-4433-9897-741a5d998529","Type":"ContainerStarted","Data":"b40911b86cf89ca99d8cdb40b8ce8c3d300e7a5c1b0bccb9e400e8d7348fbc2f"} Dec 03 11:52:21 crc kubenswrapper[5052]: I1203 11:52:21.186044 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-cell0-conductor-0" podUID="9f1ae28a-0533-4433-9897-741a5d998529" containerName="nova-cell0-conductor-conductor" containerID="cri-o://b40911b86cf89ca99d8cdb40b8ce8c3d300e7a5c1b0bccb9e400e8d7348fbc2f" gracePeriod=30 Dec 03 11:52:21 crc kubenswrapper[5052]: I1203 11:52:21.186071 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 11:52:21 crc kubenswrapper[5052]: I1203 11:52:21.212239 5052 scope.go:117] "RemoveContainer" containerID="06ed60c6e80c4e7a7c82dba95212d5392a25092552f92b492b2ae1d402182b62" Dec 03 11:52:21 crc kubenswrapper[5052]: I1203 11:52:21.212396 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-cell0-conductor-0" podStartSLOduration=2.21238531 podStartE2EDuration="2.21238531s" podCreationTimestamp="2025-12-03 11:52:19 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:52:21.205115907 +0000 UTC m=+1931.793357691" watchObservedRunningTime="2025-12-03 11:52:21.21238531 +0000 UTC m=+1931.800627074" Dec 03 11:52:21 crc kubenswrapper[5052]: I1203 11:52:21.230289 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:52:21 crc kubenswrapper[5052]: I1203 11:52:21.236343 5052 scope.go:117] "RemoveContainer" containerID="7f99c2eea2ed8331b29ec6d6324f201e6ebb4ae6f71c9cf21e24b8b48961ed39" Dec 03 11:52:21 crc kubenswrapper[5052]: I1203 11:52:21.238888 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:52:21 crc kubenswrapper[5052]: I1203 11:52:21.255590 5052 scope.go:117] "RemoveContainer" containerID="4ed7a40265188a06aceab89b1d6ec5a6368e1490d257b4dcc9660cb8319e4d81" Dec 03 11:52:21 crc kubenswrapper[5052]: I1203 11:52:21.256931 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:52:21 crc kubenswrapper[5052]: E1203 11:52:21.260881 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="6409124d-6776-45c8-9a15-a96771e5fdc0" containerName="sg-core" Dec 03 11:52:21 crc kubenswrapper[5052]: I1203 11:52:21.260953 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="6409124d-6776-45c8-9a15-a96771e5fdc0" containerName="sg-core" Dec 03 11:52:21 crc kubenswrapper[5052]: E1203 11:52:21.261037 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="6409124d-6776-45c8-9a15-a96771e5fdc0" containerName="ceilometer-central-agent" Dec 03 11:52:21 crc kubenswrapper[5052]: I1203 11:52:21.261047 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="6409124d-6776-45c8-9a15-a96771e5fdc0" containerName="ceilometer-central-agent" Dec 03 11:52:21 crc kubenswrapper[5052]: E1203 11:52:21.261075 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="6409124d-6776-45c8-9a15-a96771e5fdc0" containerName="ceilometer-notification-agent" Dec 03 11:52:21 crc kubenswrapper[5052]: I1203 11:52:21.261083 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="6409124d-6776-45c8-9a15-a96771e5fdc0" containerName="ceilometer-notification-agent" Dec 03 11:52:21 crc kubenswrapper[5052]: E1203 11:52:21.261096 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="6409124d-6776-45c8-9a15-a96771e5fdc0" containerName="proxy-httpd" Dec 03 11:52:21 crc kubenswrapper[5052]: I1203 11:52:21.261103 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="6409124d-6776-45c8-9a15-a96771e5fdc0" containerName="proxy-httpd" Dec 03 11:52:21 crc kubenswrapper[5052]: I1203 11:52:21.261424 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="6409124d-6776-45c8-9a15-a96771e5fdc0" containerName="ceilometer-central-agent" Dec 03 11:52:21 crc kubenswrapper[5052]: I1203 11:52:21.261449 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="6409124d-6776-45c8-9a15-a96771e5fdc0" containerName="ceilometer-notification-agent" Dec 03 11:52:21 crc kubenswrapper[5052]: I1203 11:52:21.261468 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="6409124d-6776-45c8-9a15-a96771e5fdc0" containerName="proxy-httpd" Dec 03 11:52:21 crc kubenswrapper[5052]: I1203 11:52:21.261485 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="6409124d-6776-45c8-9a15-a96771e5fdc0" containerName="sg-core" Dec 03 11:52:21 crc kubenswrapper[5052]: I1203 11:52:21.263266 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:52:21 crc kubenswrapper[5052]: I1203 11:52:21.266704 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-scripts" Dec 03 11:52:21 crc kubenswrapper[5052]: I1203 11:52:21.266714 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-config-data" Dec 03 11:52:21 crc kubenswrapper[5052]: I1203 11:52:21.284939 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:52:21 crc kubenswrapper[5052]: I1203 11:52:21.373452 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/0b1e715b-a818-4f92-b56c-3bc5f53dd41b-config-data\") pod \"ceilometer-0\" (UID: \"0b1e715b-a818-4f92-b56c-3bc5f53dd41b\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:52:21 crc kubenswrapper[5052]: I1203 11:52:21.373494 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/0b1e715b-a818-4f92-b56c-3bc5f53dd41b-log-httpd\") pod \"ceilometer-0\" (UID: \"0b1e715b-a818-4f92-b56c-3bc5f53dd41b\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:52:21 crc kubenswrapper[5052]: I1203 11:52:21.373558 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/0b1e715b-a818-4f92-b56c-3bc5f53dd41b-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"0b1e715b-a818-4f92-b56c-3bc5f53dd41b\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:52:21 crc kubenswrapper[5052]: I1203 11:52:21.373589 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xxrsz\" (UniqueName: \"kubernetes.io/projected/0b1e715b-a818-4f92-b56c-3bc5f53dd41b-kube-api-access-xxrsz\") pod \"ceilometer-0\" (UID: \"0b1e715b-a818-4f92-b56c-3bc5f53dd41b\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:52:21 crc kubenswrapper[5052]: I1203 11:52:21.373610 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/0b1e715b-a818-4f92-b56c-3bc5f53dd41b-run-httpd\") pod \"ceilometer-0\" (UID: \"0b1e715b-a818-4f92-b56c-3bc5f53dd41b\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:52:21 crc kubenswrapper[5052]: I1203 11:52:21.373640 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0b1e715b-a818-4f92-b56c-3bc5f53dd41b-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"0b1e715b-a818-4f92-b56c-3bc5f53dd41b\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:52:21 crc kubenswrapper[5052]: I1203 11:52:21.373749 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/0b1e715b-a818-4f92-b56c-3bc5f53dd41b-scripts\") pod \"ceilometer-0\" (UID: \"0b1e715b-a818-4f92-b56c-3bc5f53dd41b\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:52:21 crc kubenswrapper[5052]: I1203 11:52:21.474956 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/0b1e715b-a818-4f92-b56c-3bc5f53dd41b-scripts\") pod \"ceilometer-0\" (UID: \"0b1e715b-a818-4f92-b56c-3bc5f53dd41b\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:52:21 crc kubenswrapper[5052]: I1203 11:52:21.475058 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/0b1e715b-a818-4f92-b56c-3bc5f53dd41b-config-data\") pod \"ceilometer-0\" (UID: \"0b1e715b-a818-4f92-b56c-3bc5f53dd41b\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:52:21 crc kubenswrapper[5052]: I1203 11:52:21.475085 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/0b1e715b-a818-4f92-b56c-3bc5f53dd41b-log-httpd\") pod \"ceilometer-0\" (UID: \"0b1e715b-a818-4f92-b56c-3bc5f53dd41b\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:52:21 crc kubenswrapper[5052]: I1203 11:52:21.475139 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/0b1e715b-a818-4f92-b56c-3bc5f53dd41b-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"0b1e715b-a818-4f92-b56c-3bc5f53dd41b\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:52:21 crc kubenswrapper[5052]: I1203 11:52:21.475174 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-xxrsz\" (UniqueName: \"kubernetes.io/projected/0b1e715b-a818-4f92-b56c-3bc5f53dd41b-kube-api-access-xxrsz\") pod \"ceilometer-0\" (UID: \"0b1e715b-a818-4f92-b56c-3bc5f53dd41b\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:52:21 crc kubenswrapper[5052]: I1203 11:52:21.475197 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/0b1e715b-a818-4f92-b56c-3bc5f53dd41b-run-httpd\") pod \"ceilometer-0\" (UID: \"0b1e715b-a818-4f92-b56c-3bc5f53dd41b\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:52:21 crc kubenswrapper[5052]: I1203 11:52:21.475227 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0b1e715b-a818-4f92-b56c-3bc5f53dd41b-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"0b1e715b-a818-4f92-b56c-3bc5f53dd41b\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:52:21 crc kubenswrapper[5052]: I1203 11:52:21.475635 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/0b1e715b-a818-4f92-b56c-3bc5f53dd41b-log-httpd\") pod \"ceilometer-0\" (UID: \"0b1e715b-a818-4f92-b56c-3bc5f53dd41b\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:52:21 crc kubenswrapper[5052]: I1203 11:52:21.475823 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/0b1e715b-a818-4f92-b56c-3bc5f53dd41b-run-httpd\") pod \"ceilometer-0\" (UID: \"0b1e715b-a818-4f92-b56c-3bc5f53dd41b\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:52:21 crc kubenswrapper[5052]: I1203 11:52:21.480107 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/0b1e715b-a818-4f92-b56c-3bc5f53dd41b-scripts\") pod \"ceilometer-0\" (UID: \"0b1e715b-a818-4f92-b56c-3bc5f53dd41b\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:52:21 crc kubenswrapper[5052]: I1203 11:52:21.480113 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/0b1e715b-a818-4f92-b56c-3bc5f53dd41b-config-data\") pod \"ceilometer-0\" (UID: \"0b1e715b-a818-4f92-b56c-3bc5f53dd41b\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:52:21 crc kubenswrapper[5052]: I1203 11:52:21.480178 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0b1e715b-a818-4f92-b56c-3bc5f53dd41b-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"0b1e715b-a818-4f92-b56c-3bc5f53dd41b\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:52:21 crc kubenswrapper[5052]: I1203 11:52:21.480334 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/0b1e715b-a818-4f92-b56c-3bc5f53dd41b-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"0b1e715b-a818-4f92-b56c-3bc5f53dd41b\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:52:21 crc kubenswrapper[5052]: I1203 11:52:21.499775 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-xxrsz\" (UniqueName: \"kubernetes.io/projected/0b1e715b-a818-4f92-b56c-3bc5f53dd41b-kube-api-access-xxrsz\") pod \"ceilometer-0\" (UID: \"0b1e715b-a818-4f92-b56c-3bc5f53dd41b\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:52:21 crc kubenswrapper[5052]: I1203 11:52:21.583177 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:52:21 crc kubenswrapper[5052]: I1203 11:52:21.695815 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6409124d-6776-45c8-9a15-a96771e5fdc0" path="/var/lib/kubelet/pods/6409124d-6776-45c8-9a15-a96771e5fdc0/volumes" Dec 03 11:52:22 crc kubenswrapper[5052]: I1203 11:52:22.001237 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/glance-default-external-api-0" podUID="5983840b-20da-49d2-aaa4-7be0eb36b0c4" containerName="glance-httpd" probeResult="failure" output="Get \"https://10.217.1.84:9292/healthcheck\": read tcp 10.217.0.2:37224->10.217.1.84:9292: read: connection reset by peer" Dec 03 11:52:22 crc kubenswrapper[5052]: I1203 11:52:22.001306 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/glance-default-external-api-0" podUID="5983840b-20da-49d2-aaa4-7be0eb36b0c4" containerName="glance-log" probeResult="failure" output="Get \"https://10.217.1.84:9292/healthcheck\": read tcp 10.217.0.2:37226->10.217.1.84:9292: read: connection reset by peer" Dec 03 11:52:22 crc kubenswrapper[5052]: I1203 11:52:22.140488 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:52:22 crc kubenswrapper[5052]: W1203 11:52:22.144564 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0b1e715b_a818_4f92_b56c_3bc5f53dd41b.slice/crio-ebef7dbe1c54b8bb7ba646cac358f25d1aeecb92a562a6dbd445013c5fab5b8f WatchSource:0}: Error finding container ebef7dbe1c54b8bb7ba646cac358f25d1aeecb92a562a6dbd445013c5fab5b8f: Status 404 returned error can't find the container with id ebef7dbe1c54b8bb7ba646cac358f25d1aeecb92a562a6dbd445013c5fab5b8f Dec 03 11:52:22 crc kubenswrapper[5052]: I1203 11:52:22.214338 5052 generic.go:334] "Generic (PLEG): container finished" podID="5983840b-20da-49d2-aaa4-7be0eb36b0c4" containerID="6d7e2f9948e6e8fb4bfee608b3eeaee15339ef55c9a1c50f68aa577ee7ae17b4" exitCode=0 Dec 03 11:52:22 crc kubenswrapper[5052]: I1203 11:52:22.214434 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"5983840b-20da-49d2-aaa4-7be0eb36b0c4","Type":"ContainerDied","Data":"6d7e2f9948e6e8fb4bfee608b3eeaee15339ef55c9a1c50f68aa577ee7ae17b4"} Dec 03 11:52:22 crc kubenswrapper[5052]: I1203 11:52:22.216620 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"0b1e715b-a818-4f92-b56c-3bc5f53dd41b","Type":"ContainerStarted","Data":"ebef7dbe1c54b8bb7ba646cac358f25d1aeecb92a562a6dbd445013c5fab5b8f"} Dec 03 11:52:22 crc kubenswrapper[5052]: I1203 11:52:22.360463 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:52:22 crc kubenswrapper[5052]: I1203 11:52:22.492919 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/5983840b-20da-49d2-aaa4-7be0eb36b0c4-config-data\") pod \"5983840b-20da-49d2-aaa4-7be0eb36b0c4\" (UID: \"5983840b-20da-49d2-aaa4-7be0eb36b0c4\") " Dec 03 11:52:22 crc kubenswrapper[5052]: I1203 11:52:22.492981 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/5983840b-20da-49d2-aaa4-7be0eb36b0c4-public-tls-certs\") pod \"5983840b-20da-49d2-aaa4-7be0eb36b0c4\" (UID: \"5983840b-20da-49d2-aaa4-7be0eb36b0c4\") " Dec 03 11:52:22 crc kubenswrapper[5052]: I1203 11:52:22.493047 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/5983840b-20da-49d2-aaa4-7be0eb36b0c4-logs\") pod \"5983840b-20da-49d2-aaa4-7be0eb36b0c4\" (UID: \"5983840b-20da-49d2-aaa4-7be0eb36b0c4\") " Dec 03 11:52:22 crc kubenswrapper[5052]: I1203 11:52:22.493080 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"glance\" (UniqueName: \"kubernetes.io/local-volume/local-storage09-crc\") pod \"5983840b-20da-49d2-aaa4-7be0eb36b0c4\" (UID: \"5983840b-20da-49d2-aaa4-7be0eb36b0c4\") " Dec 03 11:52:22 crc kubenswrapper[5052]: I1203 11:52:22.493145 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/5983840b-20da-49d2-aaa4-7be0eb36b0c4-httpd-run\") pod \"5983840b-20da-49d2-aaa4-7be0eb36b0c4\" (UID: \"5983840b-20da-49d2-aaa4-7be0eb36b0c4\") " Dec 03 11:52:22 crc kubenswrapper[5052]: I1203 11:52:22.493262 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/5983840b-20da-49d2-aaa4-7be0eb36b0c4-scripts\") pod \"5983840b-20da-49d2-aaa4-7be0eb36b0c4\" (UID: \"5983840b-20da-49d2-aaa4-7be0eb36b0c4\") " Dec 03 11:52:22 crc kubenswrapper[5052]: I1203 11:52:22.493322 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dtdbk\" (UniqueName: \"kubernetes.io/projected/5983840b-20da-49d2-aaa4-7be0eb36b0c4-kube-api-access-dtdbk\") pod \"5983840b-20da-49d2-aaa4-7be0eb36b0c4\" (UID: \"5983840b-20da-49d2-aaa4-7be0eb36b0c4\") " Dec 03 11:52:22 crc kubenswrapper[5052]: I1203 11:52:22.493352 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5983840b-20da-49d2-aaa4-7be0eb36b0c4-combined-ca-bundle\") pod \"5983840b-20da-49d2-aaa4-7be0eb36b0c4\" (UID: \"5983840b-20da-49d2-aaa4-7be0eb36b0c4\") " Dec 03 11:52:22 crc kubenswrapper[5052]: I1203 11:52:22.494551 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5983840b-20da-49d2-aaa4-7be0eb36b0c4-logs" (OuterVolumeSpecName: "logs") pod "5983840b-20da-49d2-aaa4-7be0eb36b0c4" (UID: "5983840b-20da-49d2-aaa4-7be0eb36b0c4"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:52:22 crc kubenswrapper[5052]: I1203 11:52:22.494701 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5983840b-20da-49d2-aaa4-7be0eb36b0c4-httpd-run" (OuterVolumeSpecName: "httpd-run") pod "5983840b-20da-49d2-aaa4-7be0eb36b0c4" (UID: "5983840b-20da-49d2-aaa4-7be0eb36b0c4"). InnerVolumeSpecName "httpd-run". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:52:22 crc kubenswrapper[5052]: I1203 11:52:22.499236 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage09-crc" (OuterVolumeSpecName: "glance") pod "5983840b-20da-49d2-aaa4-7be0eb36b0c4" (UID: "5983840b-20da-49d2-aaa4-7be0eb36b0c4"). InnerVolumeSpecName "local-storage09-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 11:52:22 crc kubenswrapper[5052]: I1203 11:52:22.499334 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5983840b-20da-49d2-aaa4-7be0eb36b0c4-scripts" (OuterVolumeSpecName: "scripts") pod "5983840b-20da-49d2-aaa4-7be0eb36b0c4" (UID: "5983840b-20da-49d2-aaa4-7be0eb36b0c4"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:52:22 crc kubenswrapper[5052]: I1203 11:52:22.499664 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5983840b-20da-49d2-aaa4-7be0eb36b0c4-kube-api-access-dtdbk" (OuterVolumeSpecName: "kube-api-access-dtdbk") pod "5983840b-20da-49d2-aaa4-7be0eb36b0c4" (UID: "5983840b-20da-49d2-aaa4-7be0eb36b0c4"). InnerVolumeSpecName "kube-api-access-dtdbk". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:52:22 crc kubenswrapper[5052]: I1203 11:52:22.533917 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5983840b-20da-49d2-aaa4-7be0eb36b0c4-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "5983840b-20da-49d2-aaa4-7be0eb36b0c4" (UID: "5983840b-20da-49d2-aaa4-7be0eb36b0c4"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:52:22 crc kubenswrapper[5052]: I1203 11:52:22.559112 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5983840b-20da-49d2-aaa4-7be0eb36b0c4-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "5983840b-20da-49d2-aaa4-7be0eb36b0c4" (UID: "5983840b-20da-49d2-aaa4-7be0eb36b0c4"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:52:22 crc kubenswrapper[5052]: I1203 11:52:22.561964 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5983840b-20da-49d2-aaa4-7be0eb36b0c4-config-data" (OuterVolumeSpecName: "config-data") pod "5983840b-20da-49d2-aaa4-7be0eb36b0c4" (UID: "5983840b-20da-49d2-aaa4-7be0eb36b0c4"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:52:22 crc kubenswrapper[5052]: I1203 11:52:22.595343 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/5983840b-20da-49d2-aaa4-7be0eb36b0c4-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:52:22 crc kubenswrapper[5052]: I1203 11:52:22.595673 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/5983840b-20da-49d2-aaa4-7be0eb36b0c4-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 11:52:22 crc kubenswrapper[5052]: I1203 11:52:22.595689 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/5983840b-20da-49d2-aaa4-7be0eb36b0c4-logs\") on node \"crc\" DevicePath \"\"" Dec 03 11:52:22 crc kubenswrapper[5052]: I1203 11:52:22.595720 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage09-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage09-crc\") on node \"crc\" " Dec 03 11:52:22 crc kubenswrapper[5052]: I1203 11:52:22.595733 5052 reconciler_common.go:293] "Volume detached for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/5983840b-20da-49d2-aaa4-7be0eb36b0c4-httpd-run\") on node \"crc\" DevicePath \"\"" Dec 03 11:52:22 crc kubenswrapper[5052]: I1203 11:52:22.595744 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/5983840b-20da-49d2-aaa4-7be0eb36b0c4-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:52:22 crc kubenswrapper[5052]: I1203 11:52:22.595757 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-dtdbk\" (UniqueName: \"kubernetes.io/projected/5983840b-20da-49d2-aaa4-7be0eb36b0c4-kube-api-access-dtdbk\") on node \"crc\" DevicePath \"\"" Dec 03 11:52:22 crc kubenswrapper[5052]: I1203 11:52:22.595769 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5983840b-20da-49d2-aaa4-7be0eb36b0c4-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:52:22 crc kubenswrapper[5052]: I1203 11:52:22.620893 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage09-crc" (UniqueName: "kubernetes.io/local-volume/local-storage09-crc") on node "crc" Dec 03 11:52:22 crc kubenswrapper[5052]: I1203 11:52:22.697280 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage09-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage09-crc\") on node \"crc\" DevicePath \"\"" Dec 03 11:52:22 crc kubenswrapper[5052]: I1203 11:52:22.944981 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:52:23 crc kubenswrapper[5052]: I1203 11:52:23.246501 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"5983840b-20da-49d2-aaa4-7be0eb36b0c4","Type":"ContainerDied","Data":"e6c2bd43cdad3d2765bde41c0efd46bd1d126d31699d04edbb3daf8f80d5783f"} Dec 03 11:52:23 crc kubenswrapper[5052]: I1203 11:52:23.246579 5052 scope.go:117] "RemoveContainer" containerID="6d7e2f9948e6e8fb4bfee608b3eeaee15339ef55c9a1c50f68aa577ee7ae17b4" Dec 03 11:52:23 crc kubenswrapper[5052]: I1203 11:52:23.246588 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:52:23 crc kubenswrapper[5052]: I1203 11:52:23.260286 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"0b1e715b-a818-4f92-b56c-3bc5f53dd41b","Type":"ContainerStarted","Data":"5b169bb512dbd1668acdc5b85816f5b2f20666e2a7728f2199a7f78b7ccc7e33"} Dec 03 11:52:23 crc kubenswrapper[5052]: I1203 11:52:23.293680 5052 scope.go:117] "RemoveContainer" containerID="6ed3a69c28ece77aa5d36034ff2cca030b4a7f6251d240bbd51afcf3f4a9773d" Dec 03 11:52:23 crc kubenswrapper[5052]: I1203 11:52:23.301576 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 11:52:23 crc kubenswrapper[5052]: I1203 11:52:23.312834 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 11:52:23 crc kubenswrapper[5052]: I1203 11:52:23.335659 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 11:52:23 crc kubenswrapper[5052]: E1203 11:52:23.336203 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="5983840b-20da-49d2-aaa4-7be0eb36b0c4" containerName="glance-httpd" Dec 03 11:52:23 crc kubenswrapper[5052]: I1203 11:52:23.336228 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="5983840b-20da-49d2-aaa4-7be0eb36b0c4" containerName="glance-httpd" Dec 03 11:52:23 crc kubenswrapper[5052]: E1203 11:52:23.336255 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="5983840b-20da-49d2-aaa4-7be0eb36b0c4" containerName="glance-log" Dec 03 11:52:23 crc kubenswrapper[5052]: I1203 11:52:23.336280 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="5983840b-20da-49d2-aaa4-7be0eb36b0c4" containerName="glance-log" Dec 03 11:52:23 crc kubenswrapper[5052]: I1203 11:52:23.336543 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="5983840b-20da-49d2-aaa4-7be0eb36b0c4" containerName="glance-httpd" Dec 03 11:52:23 crc kubenswrapper[5052]: I1203 11:52:23.336595 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="5983840b-20da-49d2-aaa4-7be0eb36b0c4" containerName="glance-log" Dec 03 11:52:23 crc kubenswrapper[5052]: I1203 11:52:23.337830 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:52:23 crc kubenswrapper[5052]: I1203 11:52:23.345456 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-glance-default-public-svc" Dec 03 11:52:23 crc kubenswrapper[5052]: I1203 11:52:23.345830 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-default-external-config-data" Dec 03 11:52:23 crc kubenswrapper[5052]: I1203 11:52:23.371094 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 11:52:23 crc kubenswrapper[5052]: I1203 11:52:23.427324 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/f32095c7-4e6f-43c7-a16f-513e89f7c28e-httpd-run\") pod \"glance-default-external-api-0\" (UID: \"f32095c7-4e6f-43c7-a16f-513e89f7c28e\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:52:23 crc kubenswrapper[5052]: I1203 11:52:23.427794 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/f32095c7-4e6f-43c7-a16f-513e89f7c28e-config-data\") pod \"glance-default-external-api-0\" (UID: \"f32095c7-4e6f-43c7-a16f-513e89f7c28e\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:52:23 crc kubenswrapper[5052]: I1203 11:52:23.427876 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f32095c7-4e6f-43c7-a16f-513e89f7c28e-combined-ca-bundle\") pod \"glance-default-external-api-0\" (UID: \"f32095c7-4e6f-43c7-a16f-513e89f7c28e\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:52:23 crc kubenswrapper[5052]: I1203 11:52:23.427910 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage09-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage09-crc\") pod \"glance-default-external-api-0\" (UID: \"f32095c7-4e6f-43c7-a16f-513e89f7c28e\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:52:23 crc kubenswrapper[5052]: I1203 11:52:23.427977 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7l4t2\" (UniqueName: \"kubernetes.io/projected/f32095c7-4e6f-43c7-a16f-513e89f7c28e-kube-api-access-7l4t2\") pod \"glance-default-external-api-0\" (UID: \"f32095c7-4e6f-43c7-a16f-513e89f7c28e\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:52:23 crc kubenswrapper[5052]: I1203 11:52:23.428035 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/f32095c7-4e6f-43c7-a16f-513e89f7c28e-scripts\") pod \"glance-default-external-api-0\" (UID: \"f32095c7-4e6f-43c7-a16f-513e89f7c28e\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:52:23 crc kubenswrapper[5052]: I1203 11:52:23.428082 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/f32095c7-4e6f-43c7-a16f-513e89f7c28e-logs\") pod \"glance-default-external-api-0\" (UID: \"f32095c7-4e6f-43c7-a16f-513e89f7c28e\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:52:23 crc kubenswrapper[5052]: I1203 11:52:23.428131 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/f32095c7-4e6f-43c7-a16f-513e89f7c28e-public-tls-certs\") pod \"glance-default-external-api-0\" (UID: \"f32095c7-4e6f-43c7-a16f-513e89f7c28e\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:52:23 crc kubenswrapper[5052]: I1203 11:52:23.475199 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/glance-default-internal-api-0" podUID="760038e5-c49a-4c9e-b236-a4b98d58720e" containerName="glance-log" probeResult="failure" output="Get \"https://10.217.1.85:9292/healthcheck\": read tcp 10.217.0.2:48652->10.217.1.85:9292: read: connection reset by peer" Dec 03 11:52:23 crc kubenswrapper[5052]: I1203 11:52:23.475713 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/glance-default-internal-api-0" podUID="760038e5-c49a-4c9e-b236-a4b98d58720e" containerName="glance-httpd" probeResult="failure" output="Get \"https://10.217.1.85:9292/healthcheck\": read tcp 10.217.0.2:48650->10.217.1.85:9292: read: connection reset by peer" Dec 03 11:52:23 crc kubenswrapper[5052]: I1203 11:52:23.532981 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/f32095c7-4e6f-43c7-a16f-513e89f7c28e-logs\") pod \"glance-default-external-api-0\" (UID: \"f32095c7-4e6f-43c7-a16f-513e89f7c28e\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:52:23 crc kubenswrapper[5052]: I1203 11:52:23.533063 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/f32095c7-4e6f-43c7-a16f-513e89f7c28e-public-tls-certs\") pod \"glance-default-external-api-0\" (UID: \"f32095c7-4e6f-43c7-a16f-513e89f7c28e\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:52:23 crc kubenswrapper[5052]: I1203 11:52:23.533121 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/f32095c7-4e6f-43c7-a16f-513e89f7c28e-httpd-run\") pod \"glance-default-external-api-0\" (UID: \"f32095c7-4e6f-43c7-a16f-513e89f7c28e\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:52:23 crc kubenswrapper[5052]: I1203 11:52:23.533178 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/f32095c7-4e6f-43c7-a16f-513e89f7c28e-config-data\") pod \"glance-default-external-api-0\" (UID: \"f32095c7-4e6f-43c7-a16f-513e89f7c28e\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:52:23 crc kubenswrapper[5052]: I1203 11:52:23.533233 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f32095c7-4e6f-43c7-a16f-513e89f7c28e-combined-ca-bundle\") pod \"glance-default-external-api-0\" (UID: \"f32095c7-4e6f-43c7-a16f-513e89f7c28e\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:52:23 crc kubenswrapper[5052]: I1203 11:52:23.533267 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage09-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage09-crc\") pod \"glance-default-external-api-0\" (UID: \"f32095c7-4e6f-43c7-a16f-513e89f7c28e\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:52:23 crc kubenswrapper[5052]: I1203 11:52:23.533319 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-7l4t2\" (UniqueName: \"kubernetes.io/projected/f32095c7-4e6f-43c7-a16f-513e89f7c28e-kube-api-access-7l4t2\") pod \"glance-default-external-api-0\" (UID: \"f32095c7-4e6f-43c7-a16f-513e89f7c28e\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:52:23 crc kubenswrapper[5052]: I1203 11:52:23.533364 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/f32095c7-4e6f-43c7-a16f-513e89f7c28e-scripts\") pod \"glance-default-external-api-0\" (UID: \"f32095c7-4e6f-43c7-a16f-513e89f7c28e\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:52:23 crc kubenswrapper[5052]: I1203 11:52:23.533898 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage09-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage09-crc\") pod \"glance-default-external-api-0\" (UID: \"f32095c7-4e6f-43c7-a16f-513e89f7c28e\") device mount path \"/mnt/openstack/pv09\"" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:52:23 crc kubenswrapper[5052]: I1203 11:52:23.535616 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/f32095c7-4e6f-43c7-a16f-513e89f7c28e-logs\") pod \"glance-default-external-api-0\" (UID: \"f32095c7-4e6f-43c7-a16f-513e89f7c28e\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:52:23 crc kubenswrapper[5052]: I1203 11:52:23.536050 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/f32095c7-4e6f-43c7-a16f-513e89f7c28e-httpd-run\") pod \"glance-default-external-api-0\" (UID: \"f32095c7-4e6f-43c7-a16f-513e89f7c28e\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:52:23 crc kubenswrapper[5052]: I1203 11:52:23.537679 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/f32095c7-4e6f-43c7-a16f-513e89f7c28e-config-data\") pod \"glance-default-external-api-0\" (UID: \"f32095c7-4e6f-43c7-a16f-513e89f7c28e\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:52:23 crc kubenswrapper[5052]: I1203 11:52:23.539194 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/f32095c7-4e6f-43c7-a16f-513e89f7c28e-scripts\") pod \"glance-default-external-api-0\" (UID: \"f32095c7-4e6f-43c7-a16f-513e89f7c28e\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:52:23 crc kubenswrapper[5052]: I1203 11:52:23.542015 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/f32095c7-4e6f-43c7-a16f-513e89f7c28e-public-tls-certs\") pod \"glance-default-external-api-0\" (UID: \"f32095c7-4e6f-43c7-a16f-513e89f7c28e\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:52:23 crc kubenswrapper[5052]: I1203 11:52:23.554574 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f32095c7-4e6f-43c7-a16f-513e89f7c28e-combined-ca-bundle\") pod \"glance-default-external-api-0\" (UID: \"f32095c7-4e6f-43c7-a16f-513e89f7c28e\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:52:23 crc kubenswrapper[5052]: I1203 11:52:23.558889 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-7l4t2\" (UniqueName: \"kubernetes.io/projected/f32095c7-4e6f-43c7-a16f-513e89f7c28e-kube-api-access-7l4t2\") pod \"glance-default-external-api-0\" (UID: \"f32095c7-4e6f-43c7-a16f-513e89f7c28e\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:52:23 crc kubenswrapper[5052]: I1203 11:52:23.581448 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage09-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage09-crc\") pod \"glance-default-external-api-0\" (UID: \"f32095c7-4e6f-43c7-a16f-513e89f7c28e\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:52:23 crc kubenswrapper[5052]: I1203 11:52:23.689177 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5983840b-20da-49d2-aaa4-7be0eb36b0c4" path="/var/lib/kubelet/pods/5983840b-20da-49d2-aaa4-7be0eb36b0c4/volumes" Dec 03 11:52:23 crc kubenswrapper[5052]: I1203 11:52:23.692188 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:52:24 crc kubenswrapper[5052]: I1203 11:52:24.204938 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 11:52:24 crc kubenswrapper[5052]: W1203 11:52:24.217345 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf32095c7_4e6f_43c7_a16f_513e89f7c28e.slice/crio-c51dcbab996d3c90af5356485f15758b6ccebf38d9965e40f0c878c24fb68968 WatchSource:0}: Error finding container c51dcbab996d3c90af5356485f15758b6ccebf38d9965e40f0c878c24fb68968: Status 404 returned error can't find the container with id c51dcbab996d3c90af5356485f15758b6ccebf38d9965e40f0c878c24fb68968 Dec 03 11:52:24 crc kubenswrapper[5052]: I1203 11:52:24.270413 5052 generic.go:334] "Generic (PLEG): container finished" podID="760038e5-c49a-4c9e-b236-a4b98d58720e" containerID="3d55abecfc0f796ab8e6c6031b93876ae418a7296b5273ed839b6fe1ac3cc9eb" exitCode=0 Dec 03 11:52:24 crc kubenswrapper[5052]: I1203 11:52:24.270533 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"760038e5-c49a-4c9e-b236-a4b98d58720e","Type":"ContainerDied","Data":"3d55abecfc0f796ab8e6c6031b93876ae418a7296b5273ed839b6fe1ac3cc9eb"} Dec 03 11:52:24 crc kubenswrapper[5052]: I1203 11:52:24.273476 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"f32095c7-4e6f-43c7-a16f-513e89f7c28e","Type":"ContainerStarted","Data":"c51dcbab996d3c90af5356485f15758b6ccebf38d9965e40f0c878c24fb68968"} Dec 03 11:52:24 crc kubenswrapper[5052]: I1203 11:52:24.282309 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"0b1e715b-a818-4f92-b56c-3bc5f53dd41b","Type":"ContainerStarted","Data":"665790873f6c55ad2c188d468337affcedf0bb7b24e2eac8edb9cafae8234e8f"} Dec 03 11:52:24 crc kubenswrapper[5052]: I1203 11:52:24.754428 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:52:24 crc kubenswrapper[5052]: I1203 11:52:24.853641 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/760038e5-c49a-4c9e-b236-a4b98d58720e-config-data\") pod \"760038e5-c49a-4c9e-b236-a4b98d58720e\" (UID: \"760038e5-c49a-4c9e-b236-a4b98d58720e\") " Dec 03 11:52:24 crc kubenswrapper[5052]: I1203 11:52:24.854027 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/760038e5-c49a-4c9e-b236-a4b98d58720e-logs\") pod \"760038e5-c49a-4c9e-b236-a4b98d58720e\" (UID: \"760038e5-c49a-4c9e-b236-a4b98d58720e\") " Dec 03 11:52:24 crc kubenswrapper[5052]: I1203 11:52:24.854066 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/760038e5-c49a-4c9e-b236-a4b98d58720e-combined-ca-bundle\") pod \"760038e5-c49a-4c9e-b236-a4b98d58720e\" (UID: \"760038e5-c49a-4c9e-b236-a4b98d58720e\") " Dec 03 11:52:24 crc kubenswrapper[5052]: I1203 11:52:24.854108 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/760038e5-c49a-4c9e-b236-a4b98d58720e-internal-tls-certs\") pod \"760038e5-c49a-4c9e-b236-a4b98d58720e\" (UID: \"760038e5-c49a-4c9e-b236-a4b98d58720e\") " Dec 03 11:52:24 crc kubenswrapper[5052]: I1203 11:52:24.854139 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d6s7m\" (UniqueName: \"kubernetes.io/projected/760038e5-c49a-4c9e-b236-a4b98d58720e-kube-api-access-d6s7m\") pod \"760038e5-c49a-4c9e-b236-a4b98d58720e\" (UID: \"760038e5-c49a-4c9e-b236-a4b98d58720e\") " Dec 03 11:52:24 crc kubenswrapper[5052]: I1203 11:52:24.854178 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/760038e5-c49a-4c9e-b236-a4b98d58720e-httpd-run\") pod \"760038e5-c49a-4c9e-b236-a4b98d58720e\" (UID: \"760038e5-c49a-4c9e-b236-a4b98d58720e\") " Dec 03 11:52:24 crc kubenswrapper[5052]: I1203 11:52:24.854276 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/760038e5-c49a-4c9e-b236-a4b98d58720e-scripts\") pod \"760038e5-c49a-4c9e-b236-a4b98d58720e\" (UID: \"760038e5-c49a-4c9e-b236-a4b98d58720e\") " Dec 03 11:52:24 crc kubenswrapper[5052]: I1203 11:52:24.854316 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"glance\" (UniqueName: \"kubernetes.io/local-volume/local-storage02-crc\") pod \"760038e5-c49a-4c9e-b236-a4b98d58720e\" (UID: \"760038e5-c49a-4c9e-b236-a4b98d58720e\") " Dec 03 11:52:24 crc kubenswrapper[5052]: I1203 11:52:24.855284 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/760038e5-c49a-4c9e-b236-a4b98d58720e-httpd-run" (OuterVolumeSpecName: "httpd-run") pod "760038e5-c49a-4c9e-b236-a4b98d58720e" (UID: "760038e5-c49a-4c9e-b236-a4b98d58720e"). InnerVolumeSpecName "httpd-run". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:52:24 crc kubenswrapper[5052]: I1203 11:52:24.855298 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/760038e5-c49a-4c9e-b236-a4b98d58720e-logs" (OuterVolumeSpecName: "logs") pod "760038e5-c49a-4c9e-b236-a4b98d58720e" (UID: "760038e5-c49a-4c9e-b236-a4b98d58720e"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:52:24 crc kubenswrapper[5052]: I1203 11:52:24.866627 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage02-crc" (OuterVolumeSpecName: "glance") pod "760038e5-c49a-4c9e-b236-a4b98d58720e" (UID: "760038e5-c49a-4c9e-b236-a4b98d58720e"). InnerVolumeSpecName "local-storage02-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 11:52:24 crc kubenswrapper[5052]: I1203 11:52:24.866747 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/760038e5-c49a-4c9e-b236-a4b98d58720e-kube-api-access-d6s7m" (OuterVolumeSpecName: "kube-api-access-d6s7m") pod "760038e5-c49a-4c9e-b236-a4b98d58720e" (UID: "760038e5-c49a-4c9e-b236-a4b98d58720e"). InnerVolumeSpecName "kube-api-access-d6s7m". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:52:24 crc kubenswrapper[5052]: I1203 11:52:24.869770 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/760038e5-c49a-4c9e-b236-a4b98d58720e-scripts" (OuterVolumeSpecName: "scripts") pod "760038e5-c49a-4c9e-b236-a4b98d58720e" (UID: "760038e5-c49a-4c9e-b236-a4b98d58720e"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:52:24 crc kubenswrapper[5052]: I1203 11:52:24.898921 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/760038e5-c49a-4c9e-b236-a4b98d58720e-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "760038e5-c49a-4c9e-b236-a4b98d58720e" (UID: "760038e5-c49a-4c9e-b236-a4b98d58720e"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:52:24 crc kubenswrapper[5052]: I1203 11:52:24.915732 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/760038e5-c49a-4c9e-b236-a4b98d58720e-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "760038e5-c49a-4c9e-b236-a4b98d58720e" (UID: "760038e5-c49a-4c9e-b236-a4b98d58720e"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:52:24 crc kubenswrapper[5052]: I1203 11:52:24.918709 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/760038e5-c49a-4c9e-b236-a4b98d58720e-config-data" (OuterVolumeSpecName: "config-data") pod "760038e5-c49a-4c9e-b236-a4b98d58720e" (UID: "760038e5-c49a-4c9e-b236-a4b98d58720e"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:52:24 crc kubenswrapper[5052]: I1203 11:52:24.956784 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/760038e5-c49a-4c9e-b236-a4b98d58720e-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:52:24 crc kubenswrapper[5052]: I1203 11:52:24.956825 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/760038e5-c49a-4c9e-b236-a4b98d58720e-logs\") on node \"crc\" DevicePath \"\"" Dec 03 11:52:24 crc kubenswrapper[5052]: I1203 11:52:24.956834 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/760038e5-c49a-4c9e-b236-a4b98d58720e-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:52:24 crc kubenswrapper[5052]: I1203 11:52:24.956846 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/760038e5-c49a-4c9e-b236-a4b98d58720e-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 11:52:24 crc kubenswrapper[5052]: I1203 11:52:24.956854 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-d6s7m\" (UniqueName: \"kubernetes.io/projected/760038e5-c49a-4c9e-b236-a4b98d58720e-kube-api-access-d6s7m\") on node \"crc\" DevicePath \"\"" Dec 03 11:52:24 crc kubenswrapper[5052]: I1203 11:52:24.956863 5052 reconciler_common.go:293] "Volume detached for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/760038e5-c49a-4c9e-b236-a4b98d58720e-httpd-run\") on node \"crc\" DevicePath \"\"" Dec 03 11:52:24 crc kubenswrapper[5052]: I1203 11:52:24.956872 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/760038e5-c49a-4c9e-b236-a4b98d58720e-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:52:24 crc kubenswrapper[5052]: I1203 11:52:24.956908 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage02-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage02-crc\") on node \"crc\" " Dec 03 11:52:24 crc kubenswrapper[5052]: I1203 11:52:24.977550 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage02-crc" (UniqueName: "kubernetes.io/local-volume/local-storage02-crc") on node "crc" Dec 03 11:52:25 crc kubenswrapper[5052]: I1203 11:52:25.058222 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage02-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage02-crc\") on node \"crc\" DevicePath \"\"" Dec 03 11:52:25 crc kubenswrapper[5052]: I1203 11:52:25.314895 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"760038e5-c49a-4c9e-b236-a4b98d58720e","Type":"ContainerDied","Data":"d0fbfa82077cfcd4d66763e4991cff3fbd800f1420c599ba5d5c5c25bf98d078"} Dec 03 11:52:25 crc kubenswrapper[5052]: I1203 11:52:25.314977 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:52:25 crc kubenswrapper[5052]: I1203 11:52:25.314988 5052 scope.go:117] "RemoveContainer" containerID="3d55abecfc0f796ab8e6c6031b93876ae418a7296b5273ed839b6fe1ac3cc9eb" Dec 03 11:52:25 crc kubenswrapper[5052]: I1203 11:52:25.371248 5052 scope.go:117] "RemoveContainer" containerID="3087941b5ebefca5a16d78bd326e84502e4ef383a38ad5c7e648b61e640cc09f" Dec 03 11:52:25 crc kubenswrapper[5052]: I1203 11:52:25.382629 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 11:52:25 crc kubenswrapper[5052]: I1203 11:52:25.393943 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 11:52:25 crc kubenswrapper[5052]: I1203 11:52:25.406720 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 11:52:25 crc kubenswrapper[5052]: E1203 11:52:25.407208 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="760038e5-c49a-4c9e-b236-a4b98d58720e" containerName="glance-httpd" Dec 03 11:52:25 crc kubenswrapper[5052]: I1203 11:52:25.407233 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="760038e5-c49a-4c9e-b236-a4b98d58720e" containerName="glance-httpd" Dec 03 11:52:25 crc kubenswrapper[5052]: E1203 11:52:25.407247 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="760038e5-c49a-4c9e-b236-a4b98d58720e" containerName="glance-log" Dec 03 11:52:25 crc kubenswrapper[5052]: I1203 11:52:25.407257 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="760038e5-c49a-4c9e-b236-a4b98d58720e" containerName="glance-log" Dec 03 11:52:25 crc kubenswrapper[5052]: I1203 11:52:25.407531 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="760038e5-c49a-4c9e-b236-a4b98d58720e" containerName="glance-log" Dec 03 11:52:25 crc kubenswrapper[5052]: I1203 11:52:25.407562 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="760038e5-c49a-4c9e-b236-a4b98d58720e" containerName="glance-httpd" Dec 03 11:52:25 crc kubenswrapper[5052]: I1203 11:52:25.409186 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:52:25 crc kubenswrapper[5052]: I1203 11:52:25.416014 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-default-internal-config-data" Dec 03 11:52:25 crc kubenswrapper[5052]: I1203 11:52:25.431602 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 11:52:25 crc kubenswrapper[5052]: I1203 11:52:25.435029 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-glance-default-internal-svc" Dec 03 11:52:25 crc kubenswrapper[5052]: I1203 11:52:25.464993 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/d541d73d-2f95-43a8-8409-40f5f2ff54c4-internal-tls-certs\") pod \"glance-default-internal-api-0\" (UID: \"d541d73d-2f95-43a8-8409-40f5f2ff54c4\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:52:25 crc kubenswrapper[5052]: I1203 11:52:25.465041 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage02-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage02-crc\") pod \"glance-default-internal-api-0\" (UID: \"d541d73d-2f95-43a8-8409-40f5f2ff54c4\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:52:25 crc kubenswrapper[5052]: I1203 11:52:25.465068 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/d541d73d-2f95-43a8-8409-40f5f2ff54c4-httpd-run\") pod \"glance-default-internal-api-0\" (UID: \"d541d73d-2f95-43a8-8409-40f5f2ff54c4\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:52:25 crc kubenswrapper[5052]: I1203 11:52:25.465101 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/d541d73d-2f95-43a8-8409-40f5f2ff54c4-logs\") pod \"glance-default-internal-api-0\" (UID: \"d541d73d-2f95-43a8-8409-40f5f2ff54c4\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:52:25 crc kubenswrapper[5052]: I1203 11:52:25.465117 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/d541d73d-2f95-43a8-8409-40f5f2ff54c4-scripts\") pod \"glance-default-internal-api-0\" (UID: \"d541d73d-2f95-43a8-8409-40f5f2ff54c4\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:52:25 crc kubenswrapper[5052]: I1203 11:52:25.465134 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d541d73d-2f95-43a8-8409-40f5f2ff54c4-combined-ca-bundle\") pod \"glance-default-internal-api-0\" (UID: \"d541d73d-2f95-43a8-8409-40f5f2ff54c4\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:52:25 crc kubenswrapper[5052]: I1203 11:52:25.465168 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hvhb2\" (UniqueName: \"kubernetes.io/projected/d541d73d-2f95-43a8-8409-40f5f2ff54c4-kube-api-access-hvhb2\") pod \"glance-default-internal-api-0\" (UID: \"d541d73d-2f95-43a8-8409-40f5f2ff54c4\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:52:25 crc kubenswrapper[5052]: I1203 11:52:25.465226 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/d541d73d-2f95-43a8-8409-40f5f2ff54c4-config-data\") pod \"glance-default-internal-api-0\" (UID: \"d541d73d-2f95-43a8-8409-40f5f2ff54c4\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:52:25 crc kubenswrapper[5052]: I1203 11:52:25.566351 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage02-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage02-crc\") pod \"glance-default-internal-api-0\" (UID: \"d541d73d-2f95-43a8-8409-40f5f2ff54c4\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:52:25 crc kubenswrapper[5052]: I1203 11:52:25.566719 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/d541d73d-2f95-43a8-8409-40f5f2ff54c4-httpd-run\") pod \"glance-default-internal-api-0\" (UID: \"d541d73d-2f95-43a8-8409-40f5f2ff54c4\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:52:25 crc kubenswrapper[5052]: I1203 11:52:25.566760 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/d541d73d-2f95-43a8-8409-40f5f2ff54c4-logs\") pod \"glance-default-internal-api-0\" (UID: \"d541d73d-2f95-43a8-8409-40f5f2ff54c4\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:52:25 crc kubenswrapper[5052]: I1203 11:52:25.566776 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/d541d73d-2f95-43a8-8409-40f5f2ff54c4-scripts\") pod \"glance-default-internal-api-0\" (UID: \"d541d73d-2f95-43a8-8409-40f5f2ff54c4\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:52:25 crc kubenswrapper[5052]: I1203 11:52:25.566795 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d541d73d-2f95-43a8-8409-40f5f2ff54c4-combined-ca-bundle\") pod \"glance-default-internal-api-0\" (UID: \"d541d73d-2f95-43a8-8409-40f5f2ff54c4\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:52:25 crc kubenswrapper[5052]: I1203 11:52:25.566851 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-hvhb2\" (UniqueName: \"kubernetes.io/projected/d541d73d-2f95-43a8-8409-40f5f2ff54c4-kube-api-access-hvhb2\") pod \"glance-default-internal-api-0\" (UID: \"d541d73d-2f95-43a8-8409-40f5f2ff54c4\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:52:25 crc kubenswrapper[5052]: I1203 11:52:25.566925 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/d541d73d-2f95-43a8-8409-40f5f2ff54c4-config-data\") pod \"glance-default-internal-api-0\" (UID: \"d541d73d-2f95-43a8-8409-40f5f2ff54c4\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:52:25 crc kubenswrapper[5052]: I1203 11:52:25.566938 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage02-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage02-crc\") pod \"glance-default-internal-api-0\" (UID: \"d541d73d-2f95-43a8-8409-40f5f2ff54c4\") device mount path \"/mnt/openstack/pv02\"" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:52:25 crc kubenswrapper[5052]: I1203 11:52:25.567444 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/d541d73d-2f95-43a8-8409-40f5f2ff54c4-httpd-run\") pod \"glance-default-internal-api-0\" (UID: \"d541d73d-2f95-43a8-8409-40f5f2ff54c4\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:52:25 crc kubenswrapper[5052]: I1203 11:52:25.567782 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/d541d73d-2f95-43a8-8409-40f5f2ff54c4-logs\") pod \"glance-default-internal-api-0\" (UID: \"d541d73d-2f95-43a8-8409-40f5f2ff54c4\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:52:25 crc kubenswrapper[5052]: I1203 11:52:25.567822 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/d541d73d-2f95-43a8-8409-40f5f2ff54c4-internal-tls-certs\") pod \"glance-default-internal-api-0\" (UID: \"d541d73d-2f95-43a8-8409-40f5f2ff54c4\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:52:25 crc kubenswrapper[5052]: I1203 11:52:25.571177 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d541d73d-2f95-43a8-8409-40f5f2ff54c4-combined-ca-bundle\") pod \"glance-default-internal-api-0\" (UID: \"d541d73d-2f95-43a8-8409-40f5f2ff54c4\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:52:25 crc kubenswrapper[5052]: I1203 11:52:25.572729 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/d541d73d-2f95-43a8-8409-40f5f2ff54c4-scripts\") pod \"glance-default-internal-api-0\" (UID: \"d541d73d-2f95-43a8-8409-40f5f2ff54c4\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:52:25 crc kubenswrapper[5052]: I1203 11:52:25.572918 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/d541d73d-2f95-43a8-8409-40f5f2ff54c4-config-data\") pod \"glance-default-internal-api-0\" (UID: \"d541d73d-2f95-43a8-8409-40f5f2ff54c4\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:52:25 crc kubenswrapper[5052]: I1203 11:52:25.586455 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/d541d73d-2f95-43a8-8409-40f5f2ff54c4-internal-tls-certs\") pod \"glance-default-internal-api-0\" (UID: \"d541d73d-2f95-43a8-8409-40f5f2ff54c4\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:52:25 crc kubenswrapper[5052]: I1203 11:52:25.591296 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-hvhb2\" (UniqueName: \"kubernetes.io/projected/d541d73d-2f95-43a8-8409-40f5f2ff54c4-kube-api-access-hvhb2\") pod \"glance-default-internal-api-0\" (UID: \"d541d73d-2f95-43a8-8409-40f5f2ff54c4\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:52:25 crc kubenswrapper[5052]: I1203 11:52:25.601170 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage02-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage02-crc\") pod \"glance-default-internal-api-0\" (UID: \"d541d73d-2f95-43a8-8409-40f5f2ff54c4\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:52:25 crc kubenswrapper[5052]: I1203 11:52:25.686626 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="760038e5-c49a-4c9e-b236-a4b98d58720e" path="/var/lib/kubelet/pods/760038e5-c49a-4c9e-b236-a4b98d58720e/volumes" Dec 03 11:52:25 crc kubenswrapper[5052]: I1203 11:52:25.749805 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:52:26 crc kubenswrapper[5052]: I1203 11:52:26.234586 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 11:52:26 crc kubenswrapper[5052]: I1203 11:52:26.337188 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"f32095c7-4e6f-43c7-a16f-513e89f7c28e","Type":"ContainerStarted","Data":"a9bd9892b49a6bfba8e02d24efa8266231be0e4d99061688b1aeffccdabed81f"} Dec 03 11:52:26 crc kubenswrapper[5052]: I1203 11:52:26.339154 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"d541d73d-2f95-43a8-8409-40f5f2ff54c4","Type":"ContainerStarted","Data":"c8dfb8aa05e13b277498c99703ea5496b118e00d18f48f2fc71e83350fd07c6e"} Dec 03 11:52:26 crc kubenswrapper[5052]: E1203 11:52:26.433440 5052 cadvisor_stats_provider.go:516] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2b63c55a_e29f_45a8_8f5d_4166c162c3cb.slice/crio-conmon-2693072ee37641b8c8369038546f2b36325a5526bdc8ec1217de284d5fa41724.scope\": RecentStats: unable to find data in memory cache]" Dec 03 11:52:27 crc kubenswrapper[5052]: I1203 11:52:27.355002 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"d541d73d-2f95-43a8-8409-40f5f2ff54c4","Type":"ContainerStarted","Data":"3b60c38b2468b7d978dcdc73d52fcc2371995bf05e7490d1357efab444e714d0"} Dec 03 11:52:27 crc kubenswrapper[5052]: I1203 11:52:27.358465 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"f32095c7-4e6f-43c7-a16f-513e89f7c28e","Type":"ContainerStarted","Data":"9b77bce42854e145583416efda6ad1a83fa0803e3bbc085d98e3a15ec1a50003"} Dec 03 11:52:27 crc kubenswrapper[5052]: I1203 11:52:27.385027 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/glance-default-external-api-0" podStartSLOduration=4.385006923 podStartE2EDuration="4.385006923s" podCreationTimestamp="2025-12-03 11:52:23 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:52:27.378819006 +0000 UTC m=+1937.967060770" watchObservedRunningTime="2025-12-03 11:52:27.385006923 +0000 UTC m=+1937.973248687" Dec 03 11:52:28 crc kubenswrapper[5052]: I1203 11:52:28.372546 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"d541d73d-2f95-43a8-8409-40f5f2ff54c4","Type":"ContainerStarted","Data":"756be0d767606b67fd07bb5cf4a07ee4a5ec3d33b08c84185318683745d97e1b"} Dec 03 11:52:28 crc kubenswrapper[5052]: I1203 11:52:28.374852 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"0b1e715b-a818-4f92-b56c-3bc5f53dd41b","Type":"ContainerStarted","Data":"caa4105376a4fd91a83a73b83c341fe59f8dc60a5bc1121168be275047db84c0"} Dec 03 11:52:29 crc kubenswrapper[5052]: I1203 11:52:29.410545 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/glance-default-internal-api-0" podStartSLOduration=4.410528186 podStartE2EDuration="4.410528186s" podCreationTimestamp="2025-12-03 11:52:25 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:52:29.407065667 +0000 UTC m=+1939.995307431" watchObservedRunningTime="2025-12-03 11:52:29.410528186 +0000 UTC m=+1939.998769950" Dec 03 11:52:29 crc kubenswrapper[5052]: E1203 11:52:29.606889 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="b40911b86cf89ca99d8cdb40b8ce8c3d300e7a5c1b0bccb9e400e8d7348fbc2f" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 11:52:29 crc kubenswrapper[5052]: E1203 11:52:29.608660 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="b40911b86cf89ca99d8cdb40b8ce8c3d300e7a5c1b0bccb9e400e8d7348fbc2f" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 11:52:29 crc kubenswrapper[5052]: E1203 11:52:29.613970 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="b40911b86cf89ca99d8cdb40b8ce8c3d300e7a5c1b0bccb9e400e8d7348fbc2f" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 11:52:29 crc kubenswrapper[5052]: E1203 11:52:29.614050 5052 prober.go:104] "Probe errored" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" probeType="Readiness" pod="openstack-kuttl-tests/nova-cell0-conductor-0" podUID="9f1ae28a-0533-4433-9897-741a5d998529" containerName="nova-cell0-conductor-conductor" Dec 03 11:52:30 crc kubenswrapper[5052]: I1203 11:52:30.398861 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"0b1e715b-a818-4f92-b56c-3bc5f53dd41b","Type":"ContainerStarted","Data":"b297067c85cdfb444ef47e977436dd7c0943a8de9f725b758f5f37ef9fb27e07"} Dec 03 11:52:30 crc kubenswrapper[5052]: I1203 11:52:30.399147 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="0b1e715b-a818-4f92-b56c-3bc5f53dd41b" containerName="ceilometer-central-agent" containerID="cri-o://5b169bb512dbd1668acdc5b85816f5b2f20666e2a7728f2199a7f78b7ccc7e33" gracePeriod=30 Dec 03 11:52:30 crc kubenswrapper[5052]: I1203 11:52:30.399225 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:52:30 crc kubenswrapper[5052]: I1203 11:52:30.399276 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="0b1e715b-a818-4f92-b56c-3bc5f53dd41b" containerName="sg-core" containerID="cri-o://caa4105376a4fd91a83a73b83c341fe59f8dc60a5bc1121168be275047db84c0" gracePeriod=30 Dec 03 11:52:30 crc kubenswrapper[5052]: I1203 11:52:30.399279 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="0b1e715b-a818-4f92-b56c-3bc5f53dd41b" containerName="ceilometer-notification-agent" containerID="cri-o://665790873f6c55ad2c188d468337affcedf0bb7b24e2eac8edb9cafae8234e8f" gracePeriod=30 Dec 03 11:52:30 crc kubenswrapper[5052]: I1203 11:52:30.399391 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="0b1e715b-a818-4f92-b56c-3bc5f53dd41b" containerName="proxy-httpd" containerID="cri-o://b297067c85cdfb444ef47e977436dd7c0943a8de9f725b758f5f37ef9fb27e07" gracePeriod=30 Dec 03 11:52:30 crc kubenswrapper[5052]: I1203 11:52:30.432790 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/ceilometer-0" podStartSLOduration=2.27442876 podStartE2EDuration="9.432767045s" podCreationTimestamp="2025-12-03 11:52:21 +0000 UTC" firstStartedPulling="2025-12-03 11:52:22.147261507 +0000 UTC m=+1932.735503271" lastFinishedPulling="2025-12-03 11:52:29.305599792 +0000 UTC m=+1939.893841556" observedRunningTime="2025-12-03 11:52:30.420332279 +0000 UTC m=+1941.008574053" watchObservedRunningTime="2025-12-03 11:52:30.432767045 +0000 UTC m=+1941.021008819" Dec 03 11:52:31 crc kubenswrapper[5052]: I1203 11:52:31.420499 5052 generic.go:334] "Generic (PLEG): container finished" podID="0b1e715b-a818-4f92-b56c-3bc5f53dd41b" containerID="b297067c85cdfb444ef47e977436dd7c0943a8de9f725b758f5f37ef9fb27e07" exitCode=0 Dec 03 11:52:31 crc kubenswrapper[5052]: I1203 11:52:31.420838 5052 generic.go:334] "Generic (PLEG): container finished" podID="0b1e715b-a818-4f92-b56c-3bc5f53dd41b" containerID="caa4105376a4fd91a83a73b83c341fe59f8dc60a5bc1121168be275047db84c0" exitCode=2 Dec 03 11:52:31 crc kubenswrapper[5052]: I1203 11:52:31.420848 5052 generic.go:334] "Generic (PLEG): container finished" podID="0b1e715b-a818-4f92-b56c-3bc5f53dd41b" containerID="665790873f6c55ad2c188d468337affcedf0bb7b24e2eac8edb9cafae8234e8f" exitCode=0 Dec 03 11:52:31 crc kubenswrapper[5052]: I1203 11:52:31.420544 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"0b1e715b-a818-4f92-b56c-3bc5f53dd41b","Type":"ContainerDied","Data":"b297067c85cdfb444ef47e977436dd7c0943a8de9f725b758f5f37ef9fb27e07"} Dec 03 11:52:31 crc kubenswrapper[5052]: I1203 11:52:31.420909 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"0b1e715b-a818-4f92-b56c-3bc5f53dd41b","Type":"ContainerDied","Data":"caa4105376a4fd91a83a73b83c341fe59f8dc60a5bc1121168be275047db84c0"} Dec 03 11:52:31 crc kubenswrapper[5052]: I1203 11:52:31.420941 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"0b1e715b-a818-4f92-b56c-3bc5f53dd41b","Type":"ContainerDied","Data":"665790873f6c55ad2c188d468337affcedf0bb7b24e2eac8edb9cafae8234e8f"} Dec 03 11:52:32 crc kubenswrapper[5052]: I1203 11:52:32.444993 5052 generic.go:334] "Generic (PLEG): container finished" podID="0b1e715b-a818-4f92-b56c-3bc5f53dd41b" containerID="5b169bb512dbd1668acdc5b85816f5b2f20666e2a7728f2199a7f78b7ccc7e33" exitCode=0 Dec 03 11:52:32 crc kubenswrapper[5052]: I1203 11:52:32.445056 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"0b1e715b-a818-4f92-b56c-3bc5f53dd41b","Type":"ContainerDied","Data":"5b169bb512dbd1668acdc5b85816f5b2f20666e2a7728f2199a7f78b7ccc7e33"} Dec 03 11:52:32 crc kubenswrapper[5052]: I1203 11:52:32.445244 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"0b1e715b-a818-4f92-b56c-3bc5f53dd41b","Type":"ContainerDied","Data":"ebef7dbe1c54b8bb7ba646cac358f25d1aeecb92a562a6dbd445013c5fab5b8f"} Dec 03 11:52:32 crc kubenswrapper[5052]: I1203 11:52:32.445261 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ebef7dbe1c54b8bb7ba646cac358f25d1aeecb92a562a6dbd445013c5fab5b8f" Dec 03 11:52:32 crc kubenswrapper[5052]: I1203 11:52:32.452435 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:52:32 crc kubenswrapper[5052]: I1203 11:52:32.610755 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/0b1e715b-a818-4f92-b56c-3bc5f53dd41b-log-httpd\") pod \"0b1e715b-a818-4f92-b56c-3bc5f53dd41b\" (UID: \"0b1e715b-a818-4f92-b56c-3bc5f53dd41b\") " Dec 03 11:52:32 crc kubenswrapper[5052]: I1203 11:52:32.610845 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/0b1e715b-a818-4f92-b56c-3bc5f53dd41b-config-data\") pod \"0b1e715b-a818-4f92-b56c-3bc5f53dd41b\" (UID: \"0b1e715b-a818-4f92-b56c-3bc5f53dd41b\") " Dec 03 11:52:32 crc kubenswrapper[5052]: I1203 11:52:32.610925 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/0b1e715b-a818-4f92-b56c-3bc5f53dd41b-sg-core-conf-yaml\") pod \"0b1e715b-a818-4f92-b56c-3bc5f53dd41b\" (UID: \"0b1e715b-a818-4f92-b56c-3bc5f53dd41b\") " Dec 03 11:52:32 crc kubenswrapper[5052]: I1203 11:52:32.610982 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0b1e715b-a818-4f92-b56c-3bc5f53dd41b-combined-ca-bundle\") pod \"0b1e715b-a818-4f92-b56c-3bc5f53dd41b\" (UID: \"0b1e715b-a818-4f92-b56c-3bc5f53dd41b\") " Dec 03 11:52:32 crc kubenswrapper[5052]: I1203 11:52:32.611096 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/0b1e715b-a818-4f92-b56c-3bc5f53dd41b-run-httpd\") pod \"0b1e715b-a818-4f92-b56c-3bc5f53dd41b\" (UID: \"0b1e715b-a818-4f92-b56c-3bc5f53dd41b\") " Dec 03 11:52:32 crc kubenswrapper[5052]: I1203 11:52:32.611163 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xxrsz\" (UniqueName: \"kubernetes.io/projected/0b1e715b-a818-4f92-b56c-3bc5f53dd41b-kube-api-access-xxrsz\") pod \"0b1e715b-a818-4f92-b56c-3bc5f53dd41b\" (UID: \"0b1e715b-a818-4f92-b56c-3bc5f53dd41b\") " Dec 03 11:52:32 crc kubenswrapper[5052]: I1203 11:52:32.611229 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/0b1e715b-a818-4f92-b56c-3bc5f53dd41b-scripts\") pod \"0b1e715b-a818-4f92-b56c-3bc5f53dd41b\" (UID: \"0b1e715b-a818-4f92-b56c-3bc5f53dd41b\") " Dec 03 11:52:32 crc kubenswrapper[5052]: I1203 11:52:32.611374 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0b1e715b-a818-4f92-b56c-3bc5f53dd41b-log-httpd" (OuterVolumeSpecName: "log-httpd") pod "0b1e715b-a818-4f92-b56c-3bc5f53dd41b" (UID: "0b1e715b-a818-4f92-b56c-3bc5f53dd41b"). InnerVolumeSpecName "log-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:52:32 crc kubenswrapper[5052]: I1203 11:52:32.611419 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0b1e715b-a818-4f92-b56c-3bc5f53dd41b-run-httpd" (OuterVolumeSpecName: "run-httpd") pod "0b1e715b-a818-4f92-b56c-3bc5f53dd41b" (UID: "0b1e715b-a818-4f92-b56c-3bc5f53dd41b"). InnerVolumeSpecName "run-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:52:32 crc kubenswrapper[5052]: I1203 11:52:32.612046 5052 reconciler_common.go:293] "Volume detached for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/0b1e715b-a818-4f92-b56c-3bc5f53dd41b-run-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 11:52:32 crc kubenswrapper[5052]: I1203 11:52:32.612073 5052 reconciler_common.go:293] "Volume detached for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/0b1e715b-a818-4f92-b56c-3bc5f53dd41b-log-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 11:52:32 crc kubenswrapper[5052]: I1203 11:52:32.616047 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0b1e715b-a818-4f92-b56c-3bc5f53dd41b-scripts" (OuterVolumeSpecName: "scripts") pod "0b1e715b-a818-4f92-b56c-3bc5f53dd41b" (UID: "0b1e715b-a818-4f92-b56c-3bc5f53dd41b"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:52:32 crc kubenswrapper[5052]: I1203 11:52:32.617435 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0b1e715b-a818-4f92-b56c-3bc5f53dd41b-kube-api-access-xxrsz" (OuterVolumeSpecName: "kube-api-access-xxrsz") pod "0b1e715b-a818-4f92-b56c-3bc5f53dd41b" (UID: "0b1e715b-a818-4f92-b56c-3bc5f53dd41b"). InnerVolumeSpecName "kube-api-access-xxrsz". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:52:32 crc kubenswrapper[5052]: I1203 11:52:32.637783 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0b1e715b-a818-4f92-b56c-3bc5f53dd41b-sg-core-conf-yaml" (OuterVolumeSpecName: "sg-core-conf-yaml") pod "0b1e715b-a818-4f92-b56c-3bc5f53dd41b" (UID: "0b1e715b-a818-4f92-b56c-3bc5f53dd41b"). InnerVolumeSpecName "sg-core-conf-yaml". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:52:32 crc kubenswrapper[5052]: I1203 11:52:32.686796 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0b1e715b-a818-4f92-b56c-3bc5f53dd41b-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "0b1e715b-a818-4f92-b56c-3bc5f53dd41b" (UID: "0b1e715b-a818-4f92-b56c-3bc5f53dd41b"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:52:32 crc kubenswrapper[5052]: I1203 11:52:32.710907 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0b1e715b-a818-4f92-b56c-3bc5f53dd41b-config-data" (OuterVolumeSpecName: "config-data") pod "0b1e715b-a818-4f92-b56c-3bc5f53dd41b" (UID: "0b1e715b-a818-4f92-b56c-3bc5f53dd41b"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:52:32 crc kubenswrapper[5052]: I1203 11:52:32.714114 5052 reconciler_common.go:293] "Volume detached for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/0b1e715b-a818-4f92-b56c-3bc5f53dd41b-sg-core-conf-yaml\") on node \"crc\" DevicePath \"\"" Dec 03 11:52:32 crc kubenswrapper[5052]: I1203 11:52:32.714145 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0b1e715b-a818-4f92-b56c-3bc5f53dd41b-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:52:32 crc kubenswrapper[5052]: I1203 11:52:32.714155 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-xxrsz\" (UniqueName: \"kubernetes.io/projected/0b1e715b-a818-4f92-b56c-3bc5f53dd41b-kube-api-access-xxrsz\") on node \"crc\" DevicePath \"\"" Dec 03 11:52:32 crc kubenswrapper[5052]: I1203 11:52:32.714166 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/0b1e715b-a818-4f92-b56c-3bc5f53dd41b-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:52:32 crc kubenswrapper[5052]: I1203 11:52:32.714174 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/0b1e715b-a818-4f92-b56c-3bc5f53dd41b-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:52:33 crc kubenswrapper[5052]: I1203 11:52:33.452852 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:52:33 crc kubenswrapper[5052]: I1203 11:52:33.492171 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:52:33 crc kubenswrapper[5052]: I1203 11:52:33.503710 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:52:33 crc kubenswrapper[5052]: I1203 11:52:33.537950 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:52:33 crc kubenswrapper[5052]: E1203 11:52:33.538464 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="0b1e715b-a818-4f92-b56c-3bc5f53dd41b" containerName="sg-core" Dec 03 11:52:33 crc kubenswrapper[5052]: I1203 11:52:33.538485 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="0b1e715b-a818-4f92-b56c-3bc5f53dd41b" containerName="sg-core" Dec 03 11:52:33 crc kubenswrapper[5052]: E1203 11:52:33.538503 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="0b1e715b-a818-4f92-b56c-3bc5f53dd41b" containerName="ceilometer-central-agent" Dec 03 11:52:33 crc kubenswrapper[5052]: I1203 11:52:33.538531 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="0b1e715b-a818-4f92-b56c-3bc5f53dd41b" containerName="ceilometer-central-agent" Dec 03 11:52:33 crc kubenswrapper[5052]: E1203 11:52:33.538549 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="0b1e715b-a818-4f92-b56c-3bc5f53dd41b" containerName="proxy-httpd" Dec 03 11:52:33 crc kubenswrapper[5052]: I1203 11:52:33.538559 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="0b1e715b-a818-4f92-b56c-3bc5f53dd41b" containerName="proxy-httpd" Dec 03 11:52:33 crc kubenswrapper[5052]: E1203 11:52:33.538582 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="0b1e715b-a818-4f92-b56c-3bc5f53dd41b" containerName="ceilometer-notification-agent" Dec 03 11:52:33 crc kubenswrapper[5052]: I1203 11:52:33.538589 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="0b1e715b-a818-4f92-b56c-3bc5f53dd41b" containerName="ceilometer-notification-agent" Dec 03 11:52:33 crc kubenswrapper[5052]: I1203 11:52:33.538815 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="0b1e715b-a818-4f92-b56c-3bc5f53dd41b" containerName="proxy-httpd" Dec 03 11:52:33 crc kubenswrapper[5052]: I1203 11:52:33.538843 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="0b1e715b-a818-4f92-b56c-3bc5f53dd41b" containerName="sg-core" Dec 03 11:52:33 crc kubenswrapper[5052]: I1203 11:52:33.538858 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="0b1e715b-a818-4f92-b56c-3bc5f53dd41b" containerName="ceilometer-notification-agent" Dec 03 11:52:33 crc kubenswrapper[5052]: I1203 11:52:33.538871 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="0b1e715b-a818-4f92-b56c-3bc5f53dd41b" containerName="ceilometer-central-agent" Dec 03 11:52:33 crc kubenswrapper[5052]: I1203 11:52:33.540840 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:52:33 crc kubenswrapper[5052]: I1203 11:52:33.544466 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-config-data" Dec 03 11:52:33 crc kubenswrapper[5052]: I1203 11:52:33.549720 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-scripts" Dec 03 11:52:33 crc kubenswrapper[5052]: I1203 11:52:33.553228 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:52:33 crc kubenswrapper[5052]: I1203 11:52:33.615947 5052 scope.go:117] "RemoveContainer" containerID="f671b26620e48a89be3dad617c0f1f2474db5f99b8b9f90227b82a243b116141" Dec 03 11:52:33 crc kubenswrapper[5052]: I1203 11:52:33.630223 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sjtbn\" (UniqueName: \"kubernetes.io/projected/3bae2410-56dc-42bc-8974-4da004ac0b28-kube-api-access-sjtbn\") pod \"ceilometer-0\" (UID: \"3bae2410-56dc-42bc-8974-4da004ac0b28\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:52:33 crc kubenswrapper[5052]: I1203 11:52:33.630678 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/3bae2410-56dc-42bc-8974-4da004ac0b28-scripts\") pod \"ceilometer-0\" (UID: \"3bae2410-56dc-42bc-8974-4da004ac0b28\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:52:33 crc kubenswrapper[5052]: I1203 11:52:33.630707 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/3bae2410-56dc-42bc-8974-4da004ac0b28-run-httpd\") pod \"ceilometer-0\" (UID: \"3bae2410-56dc-42bc-8974-4da004ac0b28\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:52:33 crc kubenswrapper[5052]: I1203 11:52:33.630751 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/3bae2410-56dc-42bc-8974-4da004ac0b28-config-data\") pod \"ceilometer-0\" (UID: \"3bae2410-56dc-42bc-8974-4da004ac0b28\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:52:33 crc kubenswrapper[5052]: I1203 11:52:33.630789 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3bae2410-56dc-42bc-8974-4da004ac0b28-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"3bae2410-56dc-42bc-8974-4da004ac0b28\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:52:33 crc kubenswrapper[5052]: I1203 11:52:33.630840 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/3bae2410-56dc-42bc-8974-4da004ac0b28-log-httpd\") pod \"ceilometer-0\" (UID: \"3bae2410-56dc-42bc-8974-4da004ac0b28\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:52:33 crc kubenswrapper[5052]: I1203 11:52:33.630868 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/3bae2410-56dc-42bc-8974-4da004ac0b28-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"3bae2410-56dc-42bc-8974-4da004ac0b28\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:52:33 crc kubenswrapper[5052]: I1203 11:52:33.685886 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0b1e715b-a818-4f92-b56c-3bc5f53dd41b" path="/var/lib/kubelet/pods/0b1e715b-a818-4f92-b56c-3bc5f53dd41b/volumes" Dec 03 11:52:33 crc kubenswrapper[5052]: I1203 11:52:33.687198 5052 scope.go:117] "RemoveContainer" containerID="90f0d1ff5147afe42adcfb385bda757769b246377b2cbf0ce7d7370e10db5e32" Dec 03 11:52:33 crc kubenswrapper[5052]: I1203 11:52:33.692537 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:52:33 crc kubenswrapper[5052]: I1203 11:52:33.694793 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:52:33 crc kubenswrapper[5052]: I1203 11:52:33.732562 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/3bae2410-56dc-42bc-8974-4da004ac0b28-scripts\") pod \"ceilometer-0\" (UID: \"3bae2410-56dc-42bc-8974-4da004ac0b28\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:52:33 crc kubenswrapper[5052]: I1203 11:52:33.732863 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/3bae2410-56dc-42bc-8974-4da004ac0b28-run-httpd\") pod \"ceilometer-0\" (UID: \"3bae2410-56dc-42bc-8974-4da004ac0b28\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:52:33 crc kubenswrapper[5052]: I1203 11:52:33.732981 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/3bae2410-56dc-42bc-8974-4da004ac0b28-config-data\") pod \"ceilometer-0\" (UID: \"3bae2410-56dc-42bc-8974-4da004ac0b28\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:52:33 crc kubenswrapper[5052]: I1203 11:52:33.733106 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3bae2410-56dc-42bc-8974-4da004ac0b28-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"3bae2410-56dc-42bc-8974-4da004ac0b28\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:52:33 crc kubenswrapper[5052]: I1203 11:52:33.733228 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/3bae2410-56dc-42bc-8974-4da004ac0b28-log-httpd\") pod \"ceilometer-0\" (UID: \"3bae2410-56dc-42bc-8974-4da004ac0b28\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:52:33 crc kubenswrapper[5052]: I1203 11:52:33.733318 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/3bae2410-56dc-42bc-8974-4da004ac0b28-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"3bae2410-56dc-42bc-8974-4da004ac0b28\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:52:33 crc kubenswrapper[5052]: I1203 11:52:33.733529 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-sjtbn\" (UniqueName: \"kubernetes.io/projected/3bae2410-56dc-42bc-8974-4da004ac0b28-kube-api-access-sjtbn\") pod \"ceilometer-0\" (UID: \"3bae2410-56dc-42bc-8974-4da004ac0b28\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:52:33 crc kubenswrapper[5052]: I1203 11:52:33.734359 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/3bae2410-56dc-42bc-8974-4da004ac0b28-run-httpd\") pod \"ceilometer-0\" (UID: \"3bae2410-56dc-42bc-8974-4da004ac0b28\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:52:33 crc kubenswrapper[5052]: I1203 11:52:33.734502 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/3bae2410-56dc-42bc-8974-4da004ac0b28-log-httpd\") pod \"ceilometer-0\" (UID: \"3bae2410-56dc-42bc-8974-4da004ac0b28\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:52:33 crc kubenswrapper[5052]: I1203 11:52:33.742431 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3bae2410-56dc-42bc-8974-4da004ac0b28-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"3bae2410-56dc-42bc-8974-4da004ac0b28\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:52:33 crc kubenswrapper[5052]: I1203 11:52:33.743032 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/3bae2410-56dc-42bc-8974-4da004ac0b28-scripts\") pod \"ceilometer-0\" (UID: \"3bae2410-56dc-42bc-8974-4da004ac0b28\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:52:33 crc kubenswrapper[5052]: I1203 11:52:33.743476 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/3bae2410-56dc-42bc-8974-4da004ac0b28-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"3bae2410-56dc-42bc-8974-4da004ac0b28\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:52:33 crc kubenswrapper[5052]: I1203 11:52:33.754039 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/3bae2410-56dc-42bc-8974-4da004ac0b28-config-data\") pod \"ceilometer-0\" (UID: \"3bae2410-56dc-42bc-8974-4da004ac0b28\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:52:33 crc kubenswrapper[5052]: I1203 11:52:33.772724 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-sjtbn\" (UniqueName: \"kubernetes.io/projected/3bae2410-56dc-42bc-8974-4da004ac0b28-kube-api-access-sjtbn\") pod \"ceilometer-0\" (UID: \"3bae2410-56dc-42bc-8974-4da004ac0b28\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:52:33 crc kubenswrapper[5052]: I1203 11:52:33.782835 5052 scope.go:117] "RemoveContainer" containerID="57aafbdbe2a86778650bc79b3ec5dc6502116268591f2f9f5ff1056d241c1f66" Dec 03 11:52:33 crc kubenswrapper[5052]: I1203 11:52:33.798838 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:52:33 crc kubenswrapper[5052]: I1203 11:52:33.807190 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:52:33 crc kubenswrapper[5052]: I1203 11:52:33.859067 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:52:33 crc kubenswrapper[5052]: I1203 11:52:33.870933 5052 scope.go:117] "RemoveContainer" containerID="f94cadcc0ee480f6891ba5c2d39b97896b89ebe3f6eee06d110b630861ea1a90" Dec 03 11:52:34 crc kubenswrapper[5052]: I1203 11:52:34.394029 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:52:34 crc kubenswrapper[5052]: I1203 11:52:34.461634 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"3bae2410-56dc-42bc-8974-4da004ac0b28","Type":"ContainerStarted","Data":"928198eda633511f4c6b051750375a81be21cad5956be1efa9e984cf4c061d28"} Dec 03 11:52:34 crc kubenswrapper[5052]: I1203 11:52:34.461931 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:52:34 crc kubenswrapper[5052]: I1203 11:52:34.461950 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:52:34 crc kubenswrapper[5052]: E1203 11:52:34.608168 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="b40911b86cf89ca99d8cdb40b8ce8c3d300e7a5c1b0bccb9e400e8d7348fbc2f" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 11:52:34 crc kubenswrapper[5052]: E1203 11:52:34.609631 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="b40911b86cf89ca99d8cdb40b8ce8c3d300e7a5c1b0bccb9e400e8d7348fbc2f" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 11:52:34 crc kubenswrapper[5052]: E1203 11:52:34.610873 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="b40911b86cf89ca99d8cdb40b8ce8c3d300e7a5c1b0bccb9e400e8d7348fbc2f" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 11:52:34 crc kubenswrapper[5052]: E1203 11:52:34.610922 5052 prober.go:104] "Probe errored" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" probeType="Readiness" pod="openstack-kuttl-tests/nova-cell0-conductor-0" podUID="9f1ae28a-0533-4433-9897-741a5d998529" containerName="nova-cell0-conductor-conductor" Dec 03 11:52:35 crc kubenswrapper[5052]: I1203 11:52:35.750185 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:52:35 crc kubenswrapper[5052]: I1203 11:52:35.750864 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:52:35 crc kubenswrapper[5052]: I1203 11:52:35.787114 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:52:35 crc kubenswrapper[5052]: I1203 11:52:35.800943 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:52:36 crc kubenswrapper[5052]: I1203 11:52:36.352141 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:52:36 crc kubenswrapper[5052]: I1203 11:52:36.499983 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"3bae2410-56dc-42bc-8974-4da004ac0b28","Type":"ContainerStarted","Data":"3d4e37cd5ebdf25795857b70ca2eda328dcf4ce9f44215affa2902c0ee5c7ec8"} Dec 03 11:52:36 crc kubenswrapper[5052]: I1203 11:52:36.500056 5052 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Dec 03 11:52:36 crc kubenswrapper[5052]: I1203 11:52:36.500381 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:52:36 crc kubenswrapper[5052]: I1203 11:52:36.500407 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:52:36 crc kubenswrapper[5052]: I1203 11:52:36.505773 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:52:36 crc kubenswrapper[5052]: E1203 11:52:36.672402 5052 cadvisor_stats_provider.go:516] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2b63c55a_e29f_45a8_8f5d_4166c162c3cb.slice/crio-conmon-2693072ee37641b8c8369038546f2b36325a5526bdc8ec1217de284d5fa41724.scope\": RecentStats: unable to find data in memory cache]" Dec 03 11:52:38 crc kubenswrapper[5052]: I1203 11:52:38.439574 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:52:38 crc kubenswrapper[5052]: I1203 11:52:38.479498 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:52:38 crc kubenswrapper[5052]: I1203 11:52:38.531299 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"3bae2410-56dc-42bc-8974-4da004ac0b28","Type":"ContainerStarted","Data":"8edea1b7e7603813f9cee40e80060c8fe7cdde3a3fe81da594f714c8848d10a6"} Dec 03 11:52:39 crc kubenswrapper[5052]: E1203 11:52:39.606088 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="b40911b86cf89ca99d8cdb40b8ce8c3d300e7a5c1b0bccb9e400e8d7348fbc2f" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 11:52:39 crc kubenswrapper[5052]: E1203 11:52:39.607821 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="b40911b86cf89ca99d8cdb40b8ce8c3d300e7a5c1b0bccb9e400e8d7348fbc2f" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 11:52:39 crc kubenswrapper[5052]: E1203 11:52:39.608753 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="b40911b86cf89ca99d8cdb40b8ce8c3d300e7a5c1b0bccb9e400e8d7348fbc2f" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 11:52:39 crc kubenswrapper[5052]: E1203 11:52:39.608787 5052 prober.go:104] "Probe errored" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" probeType="Readiness" pod="openstack-kuttl-tests/nova-cell0-conductor-0" podUID="9f1ae28a-0533-4433-9897-741a5d998529" containerName="nova-cell0-conductor-conductor" Dec 03 11:52:40 crc kubenswrapper[5052]: I1203 11:52:40.555609 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"3bae2410-56dc-42bc-8974-4da004ac0b28","Type":"ContainerStarted","Data":"dd21b0bd313fbea7fd9fbbc6df20b5977fa0f518bc12404bb5083a4c2ee3b4ce"} Dec 03 11:52:41 crc kubenswrapper[5052]: I1203 11:52:41.566751 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"3bae2410-56dc-42bc-8974-4da004ac0b28","Type":"ContainerStarted","Data":"f9fcf25c0c0d7f76435313ac33854df45d1892029892dae5e2816386e51e35d6"} Dec 03 11:52:41 crc kubenswrapper[5052]: I1203 11:52:41.567284 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:52:41 crc kubenswrapper[5052]: I1203 11:52:41.598627 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/ceilometer-0" podStartSLOduration=2.039130093 podStartE2EDuration="8.598601232s" podCreationTimestamp="2025-12-03 11:52:33 +0000 UTC" firstStartedPulling="2025-12-03 11:52:34.397902454 +0000 UTC m=+1944.986144208" lastFinishedPulling="2025-12-03 11:52:40.957373583 +0000 UTC m=+1951.545615347" observedRunningTime="2025-12-03 11:52:41.590650014 +0000 UTC m=+1952.178891788" watchObservedRunningTime="2025-12-03 11:52:41.598601232 +0000 UTC m=+1952.186842996" Dec 03 11:52:44 crc kubenswrapper[5052]: E1203 11:52:44.607359 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="b40911b86cf89ca99d8cdb40b8ce8c3d300e7a5c1b0bccb9e400e8d7348fbc2f" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 11:52:44 crc kubenswrapper[5052]: E1203 11:52:44.609379 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="b40911b86cf89ca99d8cdb40b8ce8c3d300e7a5c1b0bccb9e400e8d7348fbc2f" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 11:52:44 crc kubenswrapper[5052]: E1203 11:52:44.611344 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="b40911b86cf89ca99d8cdb40b8ce8c3d300e7a5c1b0bccb9e400e8d7348fbc2f" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 11:52:44 crc kubenswrapper[5052]: E1203 11:52:44.611430 5052 prober.go:104] "Probe errored" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" probeType="Readiness" pod="openstack-kuttl-tests/nova-cell0-conductor-0" podUID="9f1ae28a-0533-4433-9897-741a5d998529" containerName="nova-cell0-conductor-conductor" Dec 03 11:52:46 crc kubenswrapper[5052]: E1203 11:52:46.898758 5052 cadvisor_stats_provider.go:516] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2b63c55a_e29f_45a8_8f5d_4166c162c3cb.slice/crio-conmon-2693072ee37641b8c8369038546f2b36325a5526bdc8ec1217de284d5fa41724.scope\": RecentStats: unable to find data in memory cache]" Dec 03 11:52:49 crc kubenswrapper[5052]: E1203 11:52:49.606739 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="b40911b86cf89ca99d8cdb40b8ce8c3d300e7a5c1b0bccb9e400e8d7348fbc2f" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 11:52:49 crc kubenswrapper[5052]: E1203 11:52:49.608522 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="b40911b86cf89ca99d8cdb40b8ce8c3d300e7a5c1b0bccb9e400e8d7348fbc2f" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 11:52:49 crc kubenswrapper[5052]: E1203 11:52:49.609992 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="b40911b86cf89ca99d8cdb40b8ce8c3d300e7a5c1b0bccb9e400e8d7348fbc2f" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 11:52:49 crc kubenswrapper[5052]: E1203 11:52:49.610022 5052 prober.go:104] "Probe errored" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" probeType="Readiness" pod="openstack-kuttl-tests/nova-cell0-conductor-0" podUID="9f1ae28a-0533-4433-9897-741a5d998529" containerName="nova-cell0-conductor-conductor" Dec 03 11:52:51 crc kubenswrapper[5052]: I1203 11:52:51.588636 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 11:52:51 crc kubenswrapper[5052]: I1203 11:52:51.656748 5052 generic.go:334] "Generic (PLEG): container finished" podID="9f1ae28a-0533-4433-9897-741a5d998529" containerID="b40911b86cf89ca99d8cdb40b8ce8c3d300e7a5c1b0bccb9e400e8d7348fbc2f" exitCode=137 Dec 03 11:52:51 crc kubenswrapper[5052]: I1203 11:52:51.656805 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-conductor-0" event={"ID":"9f1ae28a-0533-4433-9897-741a5d998529","Type":"ContainerDied","Data":"b40911b86cf89ca99d8cdb40b8ce8c3d300e7a5c1b0bccb9e400e8d7348fbc2f"} Dec 03 11:52:51 crc kubenswrapper[5052]: I1203 11:52:51.656836 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-conductor-0" event={"ID":"9f1ae28a-0533-4433-9897-741a5d998529","Type":"ContainerDied","Data":"8d528a4478d2c3957b506756b35b44300d544505e56daccc465085f986f858de"} Dec 03 11:52:51 crc kubenswrapper[5052]: I1203 11:52:51.656853 5052 scope.go:117] "RemoveContainer" containerID="b40911b86cf89ca99d8cdb40b8ce8c3d300e7a5c1b0bccb9e400e8d7348fbc2f" Dec 03 11:52:51 crc kubenswrapper[5052]: I1203 11:52:51.656973 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 11:52:51 crc kubenswrapper[5052]: I1203 11:52:51.678016 5052 scope.go:117] "RemoveContainer" containerID="b40911b86cf89ca99d8cdb40b8ce8c3d300e7a5c1b0bccb9e400e8d7348fbc2f" Dec 03 11:52:51 crc kubenswrapper[5052]: E1203 11:52:51.680958 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b40911b86cf89ca99d8cdb40b8ce8c3d300e7a5c1b0bccb9e400e8d7348fbc2f\": container with ID starting with b40911b86cf89ca99d8cdb40b8ce8c3d300e7a5c1b0bccb9e400e8d7348fbc2f not found: ID does not exist" containerID="b40911b86cf89ca99d8cdb40b8ce8c3d300e7a5c1b0bccb9e400e8d7348fbc2f" Dec 03 11:52:51 crc kubenswrapper[5052]: I1203 11:52:51.680995 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b40911b86cf89ca99d8cdb40b8ce8c3d300e7a5c1b0bccb9e400e8d7348fbc2f"} err="failed to get container status \"b40911b86cf89ca99d8cdb40b8ce8c3d300e7a5c1b0bccb9e400e8d7348fbc2f\": rpc error: code = NotFound desc = could not find container \"b40911b86cf89ca99d8cdb40b8ce8c3d300e7a5c1b0bccb9e400e8d7348fbc2f\": container with ID starting with b40911b86cf89ca99d8cdb40b8ce8c3d300e7a5c1b0bccb9e400e8d7348fbc2f not found: ID does not exist" Dec 03 11:52:51 crc kubenswrapper[5052]: I1203 11:52:51.719342 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9f1ae28a-0533-4433-9897-741a5d998529-combined-ca-bundle\") pod \"9f1ae28a-0533-4433-9897-741a5d998529\" (UID: \"9f1ae28a-0533-4433-9897-741a5d998529\") " Dec 03 11:52:51 crc kubenswrapper[5052]: I1203 11:52:51.719674 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n9z96\" (UniqueName: \"kubernetes.io/projected/9f1ae28a-0533-4433-9897-741a5d998529-kube-api-access-n9z96\") pod \"9f1ae28a-0533-4433-9897-741a5d998529\" (UID: \"9f1ae28a-0533-4433-9897-741a5d998529\") " Dec 03 11:52:51 crc kubenswrapper[5052]: I1203 11:52:51.719698 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/9f1ae28a-0533-4433-9897-741a5d998529-config-data\") pod \"9f1ae28a-0533-4433-9897-741a5d998529\" (UID: \"9f1ae28a-0533-4433-9897-741a5d998529\") " Dec 03 11:52:51 crc kubenswrapper[5052]: I1203 11:52:51.724719 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9f1ae28a-0533-4433-9897-741a5d998529-kube-api-access-n9z96" (OuterVolumeSpecName: "kube-api-access-n9z96") pod "9f1ae28a-0533-4433-9897-741a5d998529" (UID: "9f1ae28a-0533-4433-9897-741a5d998529"). InnerVolumeSpecName "kube-api-access-n9z96". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:52:51 crc kubenswrapper[5052]: I1203 11:52:51.745096 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9f1ae28a-0533-4433-9897-741a5d998529-config-data" (OuterVolumeSpecName: "config-data") pod "9f1ae28a-0533-4433-9897-741a5d998529" (UID: "9f1ae28a-0533-4433-9897-741a5d998529"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:52:51 crc kubenswrapper[5052]: I1203 11:52:51.746326 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9f1ae28a-0533-4433-9897-741a5d998529-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "9f1ae28a-0533-4433-9897-741a5d998529" (UID: "9f1ae28a-0533-4433-9897-741a5d998529"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:52:51 crc kubenswrapper[5052]: I1203 11:52:51.822740 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-n9z96\" (UniqueName: \"kubernetes.io/projected/9f1ae28a-0533-4433-9897-741a5d998529-kube-api-access-n9z96\") on node \"crc\" DevicePath \"\"" Dec 03 11:52:51 crc kubenswrapper[5052]: I1203 11:52:51.822801 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/9f1ae28a-0533-4433-9897-741a5d998529-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:52:51 crc kubenswrapper[5052]: I1203 11:52:51.822814 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9f1ae28a-0533-4433-9897-741a5d998529-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:52:51 crc kubenswrapper[5052]: I1203 11:52:51.989015 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-0"] Dec 03 11:52:51 crc kubenswrapper[5052]: I1203 11:52:51.999360 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-0"] Dec 03 11:52:52 crc kubenswrapper[5052]: I1203 11:52:52.013422 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-0"] Dec 03 11:52:52 crc kubenswrapper[5052]: E1203 11:52:52.013817 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="9f1ae28a-0533-4433-9897-741a5d998529" containerName="nova-cell0-conductor-conductor" Dec 03 11:52:52 crc kubenswrapper[5052]: I1203 11:52:52.013835 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="9f1ae28a-0533-4433-9897-741a5d998529" containerName="nova-cell0-conductor-conductor" Dec 03 11:52:52 crc kubenswrapper[5052]: I1203 11:52:52.014018 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="9f1ae28a-0533-4433-9897-741a5d998529" containerName="nova-cell0-conductor-conductor" Dec 03 11:52:52 crc kubenswrapper[5052]: I1203 11:52:52.014739 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 11:52:52 crc kubenswrapper[5052]: I1203 11:52:52.017571 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell0-conductor-config-data" Dec 03 11:52:52 crc kubenswrapper[5052]: I1203 11:52:52.018258 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-nova-dockercfg-8c458" Dec 03 11:52:52 crc kubenswrapper[5052]: I1203 11:52:52.044637 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-0"] Dec 03 11:52:52 crc kubenswrapper[5052]: I1203 11:52:52.133874 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nnk5d\" (UniqueName: \"kubernetes.io/projected/4ec0d450-cd52-432e-8bb2-1ef9698821c1-kube-api-access-nnk5d\") pod \"nova-cell0-conductor-0\" (UID: \"4ec0d450-cd52-432e-8bb2-1ef9698821c1\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 11:52:52 crc kubenswrapper[5052]: I1203 11:52:52.134078 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/4ec0d450-cd52-432e-8bb2-1ef9698821c1-config-data\") pod \"nova-cell0-conductor-0\" (UID: \"4ec0d450-cd52-432e-8bb2-1ef9698821c1\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 11:52:52 crc kubenswrapper[5052]: I1203 11:52:52.134270 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4ec0d450-cd52-432e-8bb2-1ef9698821c1-combined-ca-bundle\") pod \"nova-cell0-conductor-0\" (UID: \"4ec0d450-cd52-432e-8bb2-1ef9698821c1\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 11:52:52 crc kubenswrapper[5052]: I1203 11:52:52.236351 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/4ec0d450-cd52-432e-8bb2-1ef9698821c1-config-data\") pod \"nova-cell0-conductor-0\" (UID: \"4ec0d450-cd52-432e-8bb2-1ef9698821c1\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 11:52:52 crc kubenswrapper[5052]: I1203 11:52:52.236805 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4ec0d450-cd52-432e-8bb2-1ef9698821c1-combined-ca-bundle\") pod \"nova-cell0-conductor-0\" (UID: \"4ec0d450-cd52-432e-8bb2-1ef9698821c1\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 11:52:52 crc kubenswrapper[5052]: I1203 11:52:52.237081 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-nnk5d\" (UniqueName: \"kubernetes.io/projected/4ec0d450-cd52-432e-8bb2-1ef9698821c1-kube-api-access-nnk5d\") pod \"nova-cell0-conductor-0\" (UID: \"4ec0d450-cd52-432e-8bb2-1ef9698821c1\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 11:52:52 crc kubenswrapper[5052]: I1203 11:52:52.240386 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4ec0d450-cd52-432e-8bb2-1ef9698821c1-combined-ca-bundle\") pod \"nova-cell0-conductor-0\" (UID: \"4ec0d450-cd52-432e-8bb2-1ef9698821c1\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 11:52:52 crc kubenswrapper[5052]: I1203 11:52:52.240909 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/4ec0d450-cd52-432e-8bb2-1ef9698821c1-config-data\") pod \"nova-cell0-conductor-0\" (UID: \"4ec0d450-cd52-432e-8bb2-1ef9698821c1\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 11:52:52 crc kubenswrapper[5052]: I1203 11:52:52.254358 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-nnk5d\" (UniqueName: \"kubernetes.io/projected/4ec0d450-cd52-432e-8bb2-1ef9698821c1-kube-api-access-nnk5d\") pod \"nova-cell0-conductor-0\" (UID: \"4ec0d450-cd52-432e-8bb2-1ef9698821c1\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 11:52:52 crc kubenswrapper[5052]: I1203 11:52:52.345138 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 11:52:52 crc kubenswrapper[5052]: I1203 11:52:52.763477 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-0"] Dec 03 11:52:53 crc kubenswrapper[5052]: I1203 11:52:53.683691 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9f1ae28a-0533-4433-9897-741a5d998529" path="/var/lib/kubelet/pods/9f1ae28a-0533-4433-9897-741a5d998529/volumes" Dec 03 11:52:53 crc kubenswrapper[5052]: I1203 11:52:53.684661 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 11:52:53 crc kubenswrapper[5052]: I1203 11:52:53.684696 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-conductor-0" event={"ID":"4ec0d450-cd52-432e-8bb2-1ef9698821c1","Type":"ContainerStarted","Data":"c75d78e4eb183b74af6b09e658fe926d3135454064d32c91f94389213e67abcc"} Dec 03 11:52:53 crc kubenswrapper[5052]: I1203 11:52:53.684712 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-conductor-0" event={"ID":"4ec0d450-cd52-432e-8bb2-1ef9698821c1","Type":"ContainerStarted","Data":"7cfae3384dfc64e883acd3f6252258d3a8c0820ae83f7b47b7037b2129664b37"} Dec 03 11:52:53 crc kubenswrapper[5052]: I1203 11:52:53.701983 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-cell0-conductor-0" podStartSLOduration=2.701967074 podStartE2EDuration="2.701967074s" podCreationTimestamp="2025-12-03 11:52:51 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:52:53.698775653 +0000 UTC m=+1964.287017417" watchObservedRunningTime="2025-12-03 11:52:53.701967074 +0000 UTC m=+1964.290208838" Dec 03 11:52:57 crc kubenswrapper[5052]: E1203 11:52:57.193136 5052 cadvisor_stats_provider.go:516] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2b63c55a_e29f_45a8_8f5d_4166c162c3cb.slice/crio-conmon-2693072ee37641b8c8369038546f2b36325a5526bdc8ec1217de284d5fa41724.scope\": RecentStats: unable to find data in memory cache]" Dec 03 11:52:57 crc kubenswrapper[5052]: I1203 11:52:57.375715 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 11:52:57 crc kubenswrapper[5052]: I1203 11:52:57.856327 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell0-cell-mapping-7v46q"] Dec 03 11:52:57 crc kubenswrapper[5052]: I1203 11:52:57.858041 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-cell-mapping-7v46q" Dec 03 11:52:57 crc kubenswrapper[5052]: I1203 11:52:57.860063 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell0-manage-config-data" Dec 03 11:52:57 crc kubenswrapper[5052]: I1203 11:52:57.860165 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell0-manage-scripts" Dec 03 11:52:57 crc kubenswrapper[5052]: I1203 11:52:57.875098 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell0-cell-mapping-7v46q"] Dec 03 11:52:57 crc kubenswrapper[5052]: I1203 11:52:57.950965 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rx6bk\" (UniqueName: \"kubernetes.io/projected/6500a947-382b-4269-b31b-5b518c858a6f-kube-api-access-rx6bk\") pod \"nova-cell0-cell-mapping-7v46q\" (UID: \"6500a947-382b-4269-b31b-5b518c858a6f\") " pod="openstack-kuttl-tests/nova-cell0-cell-mapping-7v46q" Dec 03 11:52:57 crc kubenswrapper[5052]: I1203 11:52:57.951083 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/6500a947-382b-4269-b31b-5b518c858a6f-scripts\") pod \"nova-cell0-cell-mapping-7v46q\" (UID: \"6500a947-382b-4269-b31b-5b518c858a6f\") " pod="openstack-kuttl-tests/nova-cell0-cell-mapping-7v46q" Dec 03 11:52:57 crc kubenswrapper[5052]: I1203 11:52:57.951113 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6500a947-382b-4269-b31b-5b518c858a6f-config-data\") pod \"nova-cell0-cell-mapping-7v46q\" (UID: \"6500a947-382b-4269-b31b-5b518c858a6f\") " pod="openstack-kuttl-tests/nova-cell0-cell-mapping-7v46q" Dec 03 11:52:57 crc kubenswrapper[5052]: I1203 11:52:57.951289 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6500a947-382b-4269-b31b-5b518c858a6f-combined-ca-bundle\") pod \"nova-cell0-cell-mapping-7v46q\" (UID: \"6500a947-382b-4269-b31b-5b518c858a6f\") " pod="openstack-kuttl-tests/nova-cell0-cell-mapping-7v46q" Dec 03 11:52:57 crc kubenswrapper[5052]: I1203 11:52:57.988736 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 11:52:57 crc kubenswrapper[5052]: I1203 11:52:57.997900 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 11:52:58 crc kubenswrapper[5052]: I1203 11:52:58.001122 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell1-novncproxy-config-data" Dec 03 11:52:58 crc kubenswrapper[5052]: I1203 11:52:58.008457 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 11:52:58 crc kubenswrapper[5052]: I1203 11:52:58.039432 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 11:52:58 crc kubenswrapper[5052]: I1203 11:52:58.040528 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:52:58 crc kubenswrapper[5052]: I1203 11:52:58.044874 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-scheduler-config-data" Dec 03 11:52:58 crc kubenswrapper[5052]: I1203 11:52:58.054719 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-rx6bk\" (UniqueName: \"kubernetes.io/projected/6500a947-382b-4269-b31b-5b518c858a6f-kube-api-access-rx6bk\") pod \"nova-cell0-cell-mapping-7v46q\" (UID: \"6500a947-382b-4269-b31b-5b518c858a6f\") " pod="openstack-kuttl-tests/nova-cell0-cell-mapping-7v46q" Dec 03 11:52:58 crc kubenswrapper[5052]: I1203 11:52:58.054815 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/6500a947-382b-4269-b31b-5b518c858a6f-scripts\") pod \"nova-cell0-cell-mapping-7v46q\" (UID: \"6500a947-382b-4269-b31b-5b518c858a6f\") " pod="openstack-kuttl-tests/nova-cell0-cell-mapping-7v46q" Dec 03 11:52:58 crc kubenswrapper[5052]: I1203 11:52:58.054847 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6500a947-382b-4269-b31b-5b518c858a6f-config-data\") pod \"nova-cell0-cell-mapping-7v46q\" (UID: \"6500a947-382b-4269-b31b-5b518c858a6f\") " pod="openstack-kuttl-tests/nova-cell0-cell-mapping-7v46q" Dec 03 11:52:58 crc kubenswrapper[5052]: I1203 11:52:58.054899 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6500a947-382b-4269-b31b-5b518c858a6f-combined-ca-bundle\") pod \"nova-cell0-cell-mapping-7v46q\" (UID: \"6500a947-382b-4269-b31b-5b518c858a6f\") " pod="openstack-kuttl-tests/nova-cell0-cell-mapping-7v46q" Dec 03 11:52:58 crc kubenswrapper[5052]: I1203 11:52:58.054950 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5717ffbe-ecef-4f7a-aa28-0128666af1f2-combined-ca-bundle\") pod \"nova-cell1-novncproxy-0\" (UID: \"5717ffbe-ecef-4f7a-aa28-0128666af1f2\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 11:52:58 crc kubenswrapper[5052]: I1203 11:52:58.055015 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cvc7q\" (UniqueName: \"kubernetes.io/projected/5717ffbe-ecef-4f7a-aa28-0128666af1f2-kube-api-access-cvc7q\") pod \"nova-cell1-novncproxy-0\" (UID: \"5717ffbe-ecef-4f7a-aa28-0128666af1f2\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 11:52:58 crc kubenswrapper[5052]: I1203 11:52:58.055067 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/5717ffbe-ecef-4f7a-aa28-0128666af1f2-config-data\") pod \"nova-cell1-novncproxy-0\" (UID: \"5717ffbe-ecef-4f7a-aa28-0128666af1f2\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 11:52:58 crc kubenswrapper[5052]: I1203 11:52:58.063213 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6500a947-382b-4269-b31b-5b518c858a6f-config-data\") pod \"nova-cell0-cell-mapping-7v46q\" (UID: \"6500a947-382b-4269-b31b-5b518c858a6f\") " pod="openstack-kuttl-tests/nova-cell0-cell-mapping-7v46q" Dec 03 11:52:58 crc kubenswrapper[5052]: I1203 11:52:58.065457 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6500a947-382b-4269-b31b-5b518c858a6f-combined-ca-bundle\") pod \"nova-cell0-cell-mapping-7v46q\" (UID: \"6500a947-382b-4269-b31b-5b518c858a6f\") " pod="openstack-kuttl-tests/nova-cell0-cell-mapping-7v46q" Dec 03 11:52:58 crc kubenswrapper[5052]: I1203 11:52:58.070016 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/6500a947-382b-4269-b31b-5b518c858a6f-scripts\") pod \"nova-cell0-cell-mapping-7v46q\" (UID: \"6500a947-382b-4269-b31b-5b518c858a6f\") " pod="openstack-kuttl-tests/nova-cell0-cell-mapping-7v46q" Dec 03 11:52:58 crc kubenswrapper[5052]: I1203 11:52:58.072294 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 11:52:58 crc kubenswrapper[5052]: I1203 11:52:58.106061 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-rx6bk\" (UniqueName: \"kubernetes.io/projected/6500a947-382b-4269-b31b-5b518c858a6f-kube-api-access-rx6bk\") pod \"nova-cell0-cell-mapping-7v46q\" (UID: \"6500a947-382b-4269-b31b-5b518c858a6f\") " pod="openstack-kuttl-tests/nova-cell0-cell-mapping-7v46q" Dec 03 11:52:58 crc kubenswrapper[5052]: I1203 11:52:58.126168 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 11:52:58 crc kubenswrapper[5052]: I1203 11:52:58.135089 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:52:58 crc kubenswrapper[5052]: I1203 11:52:58.140416 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-metadata-config-data" Dec 03 11:52:58 crc kubenswrapper[5052]: I1203 11:52:58.157129 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/c32dead2-3fd2-497d-b4a1-56c24d3cd8dd-config-data\") pod \"nova-scheduler-0\" (UID: \"c32dead2-3fd2-497d-b4a1-56c24d3cd8dd\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:52:58 crc kubenswrapper[5052]: I1203 11:52:58.157190 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c32dead2-3fd2-497d-b4a1-56c24d3cd8dd-combined-ca-bundle\") pod \"nova-scheduler-0\" (UID: \"c32dead2-3fd2-497d-b4a1-56c24d3cd8dd\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:52:58 crc kubenswrapper[5052]: I1203 11:52:58.157224 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5717ffbe-ecef-4f7a-aa28-0128666af1f2-combined-ca-bundle\") pod \"nova-cell1-novncproxy-0\" (UID: \"5717ffbe-ecef-4f7a-aa28-0128666af1f2\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 11:52:58 crc kubenswrapper[5052]: I1203 11:52:58.157267 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-cvc7q\" (UniqueName: \"kubernetes.io/projected/5717ffbe-ecef-4f7a-aa28-0128666af1f2-kube-api-access-cvc7q\") pod \"nova-cell1-novncproxy-0\" (UID: \"5717ffbe-ecef-4f7a-aa28-0128666af1f2\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 11:52:58 crc kubenswrapper[5052]: I1203 11:52:58.157302 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/5717ffbe-ecef-4f7a-aa28-0128666af1f2-config-data\") pod \"nova-cell1-novncproxy-0\" (UID: \"5717ffbe-ecef-4f7a-aa28-0128666af1f2\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 11:52:58 crc kubenswrapper[5052]: I1203 11:52:58.157345 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mtdwx\" (UniqueName: \"kubernetes.io/projected/c32dead2-3fd2-497d-b4a1-56c24d3cd8dd-kube-api-access-mtdwx\") pod \"nova-scheduler-0\" (UID: \"c32dead2-3fd2-497d-b4a1-56c24d3cd8dd\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:52:58 crc kubenswrapper[5052]: I1203 11:52:58.164614 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5717ffbe-ecef-4f7a-aa28-0128666af1f2-combined-ca-bundle\") pod \"nova-cell1-novncproxy-0\" (UID: \"5717ffbe-ecef-4f7a-aa28-0128666af1f2\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 11:52:58 crc kubenswrapper[5052]: I1203 11:52:58.169057 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/5717ffbe-ecef-4f7a-aa28-0128666af1f2-config-data\") pod \"nova-cell1-novncproxy-0\" (UID: \"5717ffbe-ecef-4f7a-aa28-0128666af1f2\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 11:52:58 crc kubenswrapper[5052]: I1203 11:52:58.178982 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 11:52:58 crc kubenswrapper[5052]: I1203 11:52:58.184889 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-cell-mapping-7v46q" Dec 03 11:52:58 crc kubenswrapper[5052]: I1203 11:52:58.193246 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 11:52:58 crc kubenswrapper[5052]: I1203 11:52:58.194790 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:52:58 crc kubenswrapper[5052]: I1203 11:52:58.203369 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-api-config-data" Dec 03 11:52:58 crc kubenswrapper[5052]: I1203 11:52:58.205145 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-cvc7q\" (UniqueName: \"kubernetes.io/projected/5717ffbe-ecef-4f7a-aa28-0128666af1f2-kube-api-access-cvc7q\") pod \"nova-cell1-novncproxy-0\" (UID: \"5717ffbe-ecef-4f7a-aa28-0128666af1f2\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 11:52:58 crc kubenswrapper[5052]: I1203 11:52:58.221655 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 11:52:58 crc kubenswrapper[5052]: I1203 11:52:58.259602 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8d641b96-28f4-4adc-9436-3c55585efaaa-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"8d641b96-28f4-4adc-9436-3c55585efaaa\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:52:58 crc kubenswrapper[5052]: I1203 11:52:58.260229 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/c32dead2-3fd2-497d-b4a1-56c24d3cd8dd-config-data\") pod \"nova-scheduler-0\" (UID: \"c32dead2-3fd2-497d-b4a1-56c24d3cd8dd\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:52:58 crc kubenswrapper[5052]: I1203 11:52:58.261433 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c32dead2-3fd2-497d-b4a1-56c24d3cd8dd-combined-ca-bundle\") pod \"nova-scheduler-0\" (UID: \"c32dead2-3fd2-497d-b4a1-56c24d3cd8dd\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:52:58 crc kubenswrapper[5052]: I1203 11:52:58.261776 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/8d641b96-28f4-4adc-9436-3c55585efaaa-logs\") pod \"nova-metadata-0\" (UID: \"8d641b96-28f4-4adc-9436-3c55585efaaa\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:52:58 crc kubenswrapper[5052]: I1203 11:52:58.261863 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a4ee181e-0136-4d6c-aad6-e0ffcc2cb9db-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"a4ee181e-0136-4d6c-aad6-e0ffcc2cb9db\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:52:58 crc kubenswrapper[5052]: I1203 11:52:58.261889 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x9zjm\" (UniqueName: \"kubernetes.io/projected/8d641b96-28f4-4adc-9436-3c55585efaaa-kube-api-access-x9zjm\") pod \"nova-metadata-0\" (UID: \"8d641b96-28f4-4adc-9436-3c55585efaaa\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:52:58 crc kubenswrapper[5052]: I1203 11:52:58.261921 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a4ee181e-0136-4d6c-aad6-e0ffcc2cb9db-config-data\") pod \"nova-api-0\" (UID: \"a4ee181e-0136-4d6c-aad6-e0ffcc2cb9db\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:52:58 crc kubenswrapper[5052]: I1203 11:52:58.262032 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/a4ee181e-0136-4d6c-aad6-e0ffcc2cb9db-logs\") pod \"nova-api-0\" (UID: \"a4ee181e-0136-4d6c-aad6-e0ffcc2cb9db\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:52:58 crc kubenswrapper[5052]: I1203 11:52:58.262476 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-mtdwx\" (UniqueName: \"kubernetes.io/projected/c32dead2-3fd2-497d-b4a1-56c24d3cd8dd-kube-api-access-mtdwx\") pod \"nova-scheduler-0\" (UID: \"c32dead2-3fd2-497d-b4a1-56c24d3cd8dd\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:52:58 crc kubenswrapper[5052]: I1203 11:52:58.262688 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rgn28\" (UniqueName: \"kubernetes.io/projected/a4ee181e-0136-4d6c-aad6-e0ffcc2cb9db-kube-api-access-rgn28\") pod \"nova-api-0\" (UID: \"a4ee181e-0136-4d6c-aad6-e0ffcc2cb9db\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:52:58 crc kubenswrapper[5052]: I1203 11:52:58.262876 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/8d641b96-28f4-4adc-9436-3c55585efaaa-config-data\") pod \"nova-metadata-0\" (UID: \"8d641b96-28f4-4adc-9436-3c55585efaaa\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:52:58 crc kubenswrapper[5052]: I1203 11:52:58.268072 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/c32dead2-3fd2-497d-b4a1-56c24d3cd8dd-config-data\") pod \"nova-scheduler-0\" (UID: \"c32dead2-3fd2-497d-b4a1-56c24d3cd8dd\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:52:58 crc kubenswrapper[5052]: I1203 11:52:58.277872 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c32dead2-3fd2-497d-b4a1-56c24d3cd8dd-combined-ca-bundle\") pod \"nova-scheduler-0\" (UID: \"c32dead2-3fd2-497d-b4a1-56c24d3cd8dd\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:52:58 crc kubenswrapper[5052]: I1203 11:52:58.281309 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-mtdwx\" (UniqueName: \"kubernetes.io/projected/c32dead2-3fd2-497d-b4a1-56c24d3cd8dd-kube-api-access-mtdwx\") pod \"nova-scheduler-0\" (UID: \"c32dead2-3fd2-497d-b4a1-56c24d3cd8dd\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:52:58 crc kubenswrapper[5052]: I1203 11:52:58.317028 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 11:52:58 crc kubenswrapper[5052]: I1203 11:52:58.368031 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-rgn28\" (UniqueName: \"kubernetes.io/projected/a4ee181e-0136-4d6c-aad6-e0ffcc2cb9db-kube-api-access-rgn28\") pod \"nova-api-0\" (UID: \"a4ee181e-0136-4d6c-aad6-e0ffcc2cb9db\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:52:58 crc kubenswrapper[5052]: I1203 11:52:58.368108 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/8d641b96-28f4-4adc-9436-3c55585efaaa-config-data\") pod \"nova-metadata-0\" (UID: \"8d641b96-28f4-4adc-9436-3c55585efaaa\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:52:58 crc kubenswrapper[5052]: I1203 11:52:58.368145 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8d641b96-28f4-4adc-9436-3c55585efaaa-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"8d641b96-28f4-4adc-9436-3c55585efaaa\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:52:58 crc kubenswrapper[5052]: I1203 11:52:58.368232 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/8d641b96-28f4-4adc-9436-3c55585efaaa-logs\") pod \"nova-metadata-0\" (UID: \"8d641b96-28f4-4adc-9436-3c55585efaaa\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:52:58 crc kubenswrapper[5052]: I1203 11:52:58.368275 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a4ee181e-0136-4d6c-aad6-e0ffcc2cb9db-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"a4ee181e-0136-4d6c-aad6-e0ffcc2cb9db\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:52:58 crc kubenswrapper[5052]: I1203 11:52:58.368298 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-x9zjm\" (UniqueName: \"kubernetes.io/projected/8d641b96-28f4-4adc-9436-3c55585efaaa-kube-api-access-x9zjm\") pod \"nova-metadata-0\" (UID: \"8d641b96-28f4-4adc-9436-3c55585efaaa\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:52:58 crc kubenswrapper[5052]: I1203 11:52:58.368320 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a4ee181e-0136-4d6c-aad6-e0ffcc2cb9db-config-data\") pod \"nova-api-0\" (UID: \"a4ee181e-0136-4d6c-aad6-e0ffcc2cb9db\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:52:58 crc kubenswrapper[5052]: I1203 11:52:58.368355 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/a4ee181e-0136-4d6c-aad6-e0ffcc2cb9db-logs\") pod \"nova-api-0\" (UID: \"a4ee181e-0136-4d6c-aad6-e0ffcc2cb9db\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:52:58 crc kubenswrapper[5052]: I1203 11:52:58.371116 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/8d641b96-28f4-4adc-9436-3c55585efaaa-logs\") pod \"nova-metadata-0\" (UID: \"8d641b96-28f4-4adc-9436-3c55585efaaa\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:52:58 crc kubenswrapper[5052]: I1203 11:52:58.372496 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8d641b96-28f4-4adc-9436-3c55585efaaa-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"8d641b96-28f4-4adc-9436-3c55585efaaa\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:52:58 crc kubenswrapper[5052]: I1203 11:52:58.374834 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/8d641b96-28f4-4adc-9436-3c55585efaaa-config-data\") pod \"nova-metadata-0\" (UID: \"8d641b96-28f4-4adc-9436-3c55585efaaa\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:52:58 crc kubenswrapper[5052]: I1203 11:52:58.375229 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/a4ee181e-0136-4d6c-aad6-e0ffcc2cb9db-logs\") pod \"nova-api-0\" (UID: \"a4ee181e-0136-4d6c-aad6-e0ffcc2cb9db\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:52:58 crc kubenswrapper[5052]: I1203 11:52:58.378935 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a4ee181e-0136-4d6c-aad6-e0ffcc2cb9db-config-data\") pod \"nova-api-0\" (UID: \"a4ee181e-0136-4d6c-aad6-e0ffcc2cb9db\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:52:58 crc kubenswrapper[5052]: I1203 11:52:58.401482 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a4ee181e-0136-4d6c-aad6-e0ffcc2cb9db-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"a4ee181e-0136-4d6c-aad6-e0ffcc2cb9db\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:52:58 crc kubenswrapper[5052]: I1203 11:52:58.404031 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-x9zjm\" (UniqueName: \"kubernetes.io/projected/8d641b96-28f4-4adc-9436-3c55585efaaa-kube-api-access-x9zjm\") pod \"nova-metadata-0\" (UID: \"8d641b96-28f4-4adc-9436-3c55585efaaa\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:52:58 crc kubenswrapper[5052]: I1203 11:52:58.411012 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-rgn28\" (UniqueName: \"kubernetes.io/projected/a4ee181e-0136-4d6c-aad6-e0ffcc2cb9db-kube-api-access-rgn28\") pod \"nova-api-0\" (UID: \"a4ee181e-0136-4d6c-aad6-e0ffcc2cb9db\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:52:58 crc kubenswrapper[5052]: I1203 11:52:58.559743 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:52:58 crc kubenswrapper[5052]: I1203 11:52:58.572799 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:52:58 crc kubenswrapper[5052]: I1203 11:52:58.591834 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:52:58 crc kubenswrapper[5052]: I1203 11:52:58.737073 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell0-cell-mapping-7v46q"] Dec 03 11:52:58 crc kubenswrapper[5052]: I1203 11:52:58.823265 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 11:52:58 crc kubenswrapper[5052]: I1203 11:52:58.954334 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-db-sync-xr6nj"] Dec 03 11:52:58 crc kubenswrapper[5052]: I1203 11:52:58.955871 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-xr6nj" Dec 03 11:52:58 crc kubenswrapper[5052]: I1203 11:52:58.962176 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell1-conductor-config-data" Dec 03 11:52:58 crc kubenswrapper[5052]: I1203 11:52:58.962464 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell1-conductor-scripts" Dec 03 11:52:58 crc kubenswrapper[5052]: I1203 11:52:58.988250 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-db-sync-xr6nj"] Dec 03 11:52:59 crc kubenswrapper[5052]: I1203 11:52:59.082173 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/511d01be-4413-4580-b284-d49c914d0b8a-combined-ca-bundle\") pod \"nova-cell1-conductor-db-sync-xr6nj\" (UID: \"511d01be-4413-4580-b284-d49c914d0b8a\") " pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-xr6nj" Dec 03 11:52:59 crc kubenswrapper[5052]: I1203 11:52:59.082223 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rk5gf\" (UniqueName: \"kubernetes.io/projected/511d01be-4413-4580-b284-d49c914d0b8a-kube-api-access-rk5gf\") pod \"nova-cell1-conductor-db-sync-xr6nj\" (UID: \"511d01be-4413-4580-b284-d49c914d0b8a\") " pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-xr6nj" Dec 03 11:52:59 crc kubenswrapper[5052]: I1203 11:52:59.082360 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/511d01be-4413-4580-b284-d49c914d0b8a-scripts\") pod \"nova-cell1-conductor-db-sync-xr6nj\" (UID: \"511d01be-4413-4580-b284-d49c914d0b8a\") " pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-xr6nj" Dec 03 11:52:59 crc kubenswrapper[5052]: I1203 11:52:59.082398 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/511d01be-4413-4580-b284-d49c914d0b8a-config-data\") pod \"nova-cell1-conductor-db-sync-xr6nj\" (UID: \"511d01be-4413-4580-b284-d49c914d0b8a\") " pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-xr6nj" Dec 03 11:52:59 crc kubenswrapper[5052]: I1203 11:52:59.110861 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 11:52:59 crc kubenswrapper[5052]: W1203 11:52:59.131649 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8d641b96_28f4_4adc_9436_3c55585efaaa.slice/crio-433c28123208e2ac7640c3163ae547684125a775aa16b8437c292c8b1d656025 WatchSource:0}: Error finding container 433c28123208e2ac7640c3163ae547684125a775aa16b8437c292c8b1d656025: Status 404 returned error can't find the container with id 433c28123208e2ac7640c3163ae547684125a775aa16b8437c292c8b1d656025 Dec 03 11:52:59 crc kubenswrapper[5052]: I1203 11:52:59.132020 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 11:52:59 crc kubenswrapper[5052]: W1203 11:52:59.140180 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc32dead2_3fd2_497d_b4a1_56c24d3cd8dd.slice/crio-a44d845b110bda884e47e5e504790e881029f210274249b4cfe3ac417bf1ed1a WatchSource:0}: Error finding container a44d845b110bda884e47e5e504790e881029f210274249b4cfe3ac417bf1ed1a: Status 404 returned error can't find the container with id a44d845b110bda884e47e5e504790e881029f210274249b4cfe3ac417bf1ed1a Dec 03 11:52:59 crc kubenswrapper[5052]: I1203 11:52:59.183830 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/511d01be-4413-4580-b284-d49c914d0b8a-config-data\") pod \"nova-cell1-conductor-db-sync-xr6nj\" (UID: \"511d01be-4413-4580-b284-d49c914d0b8a\") " pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-xr6nj" Dec 03 11:52:59 crc kubenswrapper[5052]: I1203 11:52:59.184432 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/511d01be-4413-4580-b284-d49c914d0b8a-combined-ca-bundle\") pod \"nova-cell1-conductor-db-sync-xr6nj\" (UID: \"511d01be-4413-4580-b284-d49c914d0b8a\") " pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-xr6nj" Dec 03 11:52:59 crc kubenswrapper[5052]: I1203 11:52:59.184564 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-rk5gf\" (UniqueName: \"kubernetes.io/projected/511d01be-4413-4580-b284-d49c914d0b8a-kube-api-access-rk5gf\") pod \"nova-cell1-conductor-db-sync-xr6nj\" (UID: \"511d01be-4413-4580-b284-d49c914d0b8a\") " pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-xr6nj" Dec 03 11:52:59 crc kubenswrapper[5052]: I1203 11:52:59.184825 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/511d01be-4413-4580-b284-d49c914d0b8a-scripts\") pod \"nova-cell1-conductor-db-sync-xr6nj\" (UID: \"511d01be-4413-4580-b284-d49c914d0b8a\") " pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-xr6nj" Dec 03 11:52:59 crc kubenswrapper[5052]: I1203 11:52:59.190046 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/511d01be-4413-4580-b284-d49c914d0b8a-combined-ca-bundle\") pod \"nova-cell1-conductor-db-sync-xr6nj\" (UID: \"511d01be-4413-4580-b284-d49c914d0b8a\") " pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-xr6nj" Dec 03 11:52:59 crc kubenswrapper[5052]: I1203 11:52:59.195916 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/511d01be-4413-4580-b284-d49c914d0b8a-scripts\") pod \"nova-cell1-conductor-db-sync-xr6nj\" (UID: \"511d01be-4413-4580-b284-d49c914d0b8a\") " pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-xr6nj" Dec 03 11:52:59 crc kubenswrapper[5052]: I1203 11:52:59.196171 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/511d01be-4413-4580-b284-d49c914d0b8a-config-data\") pod \"nova-cell1-conductor-db-sync-xr6nj\" (UID: \"511d01be-4413-4580-b284-d49c914d0b8a\") " pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-xr6nj" Dec 03 11:52:59 crc kubenswrapper[5052]: I1203 11:52:59.200407 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-rk5gf\" (UniqueName: \"kubernetes.io/projected/511d01be-4413-4580-b284-d49c914d0b8a-kube-api-access-rk5gf\") pod \"nova-cell1-conductor-db-sync-xr6nj\" (UID: \"511d01be-4413-4580-b284-d49c914d0b8a\") " pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-xr6nj" Dec 03 11:52:59 crc kubenswrapper[5052]: I1203 11:52:59.268898 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 11:52:59 crc kubenswrapper[5052]: W1203 11:52:59.269345 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda4ee181e_0136_4d6c_aad6_e0ffcc2cb9db.slice/crio-b4b1f46f1d7bf1e1ee51bcd6ab184eaf68fe533d92da86903f24b02ad60a0af3 WatchSource:0}: Error finding container b4b1f46f1d7bf1e1ee51bcd6ab184eaf68fe533d92da86903f24b02ad60a0af3: Status 404 returned error can't find the container with id b4b1f46f1d7bf1e1ee51bcd6ab184eaf68fe533d92da86903f24b02ad60a0af3 Dec 03 11:52:59 crc kubenswrapper[5052]: I1203 11:52:59.288540 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-xr6nj" Dec 03 11:52:59 crc kubenswrapper[5052]: I1203 11:52:59.806733 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"a4ee181e-0136-4d6c-aad6-e0ffcc2cb9db","Type":"ContainerStarted","Data":"70062d4b63028d8ee6f2b890b4bb61d128ba1af7288be9bc6c1fe530ecb428dc"} Dec 03 11:52:59 crc kubenswrapper[5052]: I1203 11:52:59.807093 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"a4ee181e-0136-4d6c-aad6-e0ffcc2cb9db","Type":"ContainerStarted","Data":"b4b1f46f1d7bf1e1ee51bcd6ab184eaf68fe533d92da86903f24b02ad60a0af3"} Dec 03 11:52:59 crc kubenswrapper[5052]: I1203 11:52:59.818031 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-db-sync-xr6nj"] Dec 03 11:52:59 crc kubenswrapper[5052]: W1203 11:52:59.826663 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod511d01be_4413_4580_b284_d49c914d0b8a.slice/crio-9471a7fcd0315599cb5117399f326d8120bfccd525744904148a255798d95beb WatchSource:0}: Error finding container 9471a7fcd0315599cb5117399f326d8120bfccd525744904148a255798d95beb: Status 404 returned error can't find the container with id 9471a7fcd0315599cb5117399f326d8120bfccd525744904148a255798d95beb Dec 03 11:52:59 crc kubenswrapper[5052]: I1203 11:52:59.827376 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"c32dead2-3fd2-497d-b4a1-56c24d3cd8dd","Type":"ContainerStarted","Data":"8bd79e5e170a8c0b2749fe7d7e6882a7f14fa8575090df791ae6d359ca030e14"} Dec 03 11:52:59 crc kubenswrapper[5052]: I1203 11:52:59.827411 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"c32dead2-3fd2-497d-b4a1-56c24d3cd8dd","Type":"ContainerStarted","Data":"a44d845b110bda884e47e5e504790e881029f210274249b4cfe3ac417bf1ed1a"} Dec 03 11:52:59 crc kubenswrapper[5052]: I1203 11:52:59.845860 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" event={"ID":"5717ffbe-ecef-4f7a-aa28-0128666af1f2","Type":"ContainerStarted","Data":"16ff4417366a31eb4b5554be80d8d29d8be9a3f9f1eab8a21506a01190850710"} Dec 03 11:52:59 crc kubenswrapper[5052]: I1203 11:52:59.845925 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" event={"ID":"5717ffbe-ecef-4f7a-aa28-0128666af1f2","Type":"ContainerStarted","Data":"47a22a99234d9d44f994a30cd7600b8d604bf38d8f6d03419963b86c52db8f18"} Dec 03 11:52:59 crc kubenswrapper[5052]: I1203 11:52:59.849899 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-cell-mapping-7v46q" event={"ID":"6500a947-382b-4269-b31b-5b518c858a6f","Type":"ContainerStarted","Data":"47ff9ffb2ee04fc839b0023d8e10c46ffdf46d84f9e4a70a36307e82f3f234da"} Dec 03 11:52:59 crc kubenswrapper[5052]: I1203 11:52:59.849964 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-cell-mapping-7v46q" event={"ID":"6500a947-382b-4269-b31b-5b518c858a6f","Type":"ContainerStarted","Data":"2a84acd78c7cc5ac9aee56a708c77de8df6fa014f845f01788f398d582b28969"} Dec 03 11:52:59 crc kubenswrapper[5052]: I1203 11:52:59.866850 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"8d641b96-28f4-4adc-9436-3c55585efaaa","Type":"ContainerStarted","Data":"c281e2aaabcb5590ee74e3835d9f310c5a45ca47f571d3e3f6c6ed7528906a1e"} Dec 03 11:52:59 crc kubenswrapper[5052]: I1203 11:52:59.866935 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"8d641b96-28f4-4adc-9436-3c55585efaaa","Type":"ContainerStarted","Data":"433c28123208e2ac7640c3163ae547684125a775aa16b8437c292c8b1d656025"} Dec 03 11:52:59 crc kubenswrapper[5052]: I1203 11:52:59.948311 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-cell0-cell-mapping-7v46q" podStartSLOduration=2.948288187 podStartE2EDuration="2.948288187s" podCreationTimestamp="2025-12-03 11:52:57 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:52:59.921914842 +0000 UTC m=+1970.510156596" watchObservedRunningTime="2025-12-03 11:52:59.948288187 +0000 UTC m=+1970.536529961" Dec 03 11:52:59 crc kubenswrapper[5052]: I1203 11:52:59.960212 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" podStartSLOduration=2.958583462 podStartE2EDuration="2.958583462s" podCreationTimestamp="2025-12-03 11:52:57 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:52:59.949180203 +0000 UTC m=+1970.537421967" watchObservedRunningTime="2025-12-03 11:52:59.958583462 +0000 UTC m=+1970.546825236" Dec 03 11:52:59 crc kubenswrapper[5052]: I1203 11:52:59.982966 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-scheduler-0" podStartSLOduration=1.98295025 podStartE2EDuration="1.98295025s" podCreationTimestamp="2025-12-03 11:52:58 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:52:59.973057777 +0000 UTC m=+1970.561299551" watchObservedRunningTime="2025-12-03 11:52:59.98295025 +0000 UTC m=+1970.571192014" Dec 03 11:53:00 crc kubenswrapper[5052]: I1203 11:53:00.932798 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-xr6nj" event={"ID":"511d01be-4413-4580-b284-d49c914d0b8a","Type":"ContainerStarted","Data":"974df8d52afeb94ca7f6376aa32c93e7908382a0a0e609c4d64eb521bfbc0a41"} Dec 03 11:53:00 crc kubenswrapper[5052]: I1203 11:53:00.933141 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-xr6nj" event={"ID":"511d01be-4413-4580-b284-d49c914d0b8a","Type":"ContainerStarted","Data":"9471a7fcd0315599cb5117399f326d8120bfccd525744904148a255798d95beb"} Dec 03 11:53:00 crc kubenswrapper[5052]: I1203 11:53:00.950206 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-xr6nj" podStartSLOduration=2.950187714 podStartE2EDuration="2.950187714s" podCreationTimestamp="2025-12-03 11:52:58 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:53:00.948375822 +0000 UTC m=+1971.536617596" watchObservedRunningTime="2025-12-03 11:53:00.950187714 +0000 UTC m=+1971.538429478" Dec 03 11:53:00 crc kubenswrapper[5052]: I1203 11:53:00.954781 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"8d641b96-28f4-4adc-9436-3c55585efaaa","Type":"ContainerStarted","Data":"776190df2766601f4a5f2a415e444173e4d84a3653efda9cbec92bd5b8ac9b23"} Dec 03 11:53:00 crc kubenswrapper[5052]: I1203 11:53:00.964576 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"a4ee181e-0136-4d6c-aad6-e0ffcc2cb9db","Type":"ContainerStarted","Data":"f681937d4e3600c07f0e7a5f6674a949192d35f7d97ddf1a5a4f6b78e7c5ca57"} Dec 03 11:53:00 crc kubenswrapper[5052]: I1203 11:53:00.983404 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-metadata-0" podStartSLOduration=2.983387644 podStartE2EDuration="2.983387644s" podCreationTimestamp="2025-12-03 11:52:58 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:53:00.97172284 +0000 UTC m=+1971.559964624" watchObservedRunningTime="2025-12-03 11:53:00.983387644 +0000 UTC m=+1971.571629408" Dec 03 11:53:00 crc kubenswrapper[5052]: I1203 11:53:00.994063 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-api-0" podStartSLOduration=2.99404595 podStartE2EDuration="2.99404595s" podCreationTimestamp="2025-12-03 11:52:58 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:53:00.991862167 +0000 UTC m=+1971.580103931" watchObservedRunningTime="2025-12-03 11:53:00.99404595 +0000 UTC m=+1971.582287714" Dec 03 11:53:01 crc kubenswrapper[5052]: I1203 11:53:01.565653 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 11:53:01 crc kubenswrapper[5052]: I1203 11:53:01.583263 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 11:53:01 crc kubenswrapper[5052]: I1203 11:53:01.971968 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" podUID="5717ffbe-ecef-4f7a-aa28-0128666af1f2" containerName="nova-cell1-novncproxy-novncproxy" containerID="cri-o://16ff4417366a31eb4b5554be80d8d29d8be9a3f9f1eab8a21506a01190850710" gracePeriod=30 Dec 03 11:53:02 crc kubenswrapper[5052]: I1203 11:53:02.750686 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 11:53:02 crc kubenswrapper[5052]: I1203 11:53:02.884471 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5717ffbe-ecef-4f7a-aa28-0128666af1f2-combined-ca-bundle\") pod \"5717ffbe-ecef-4f7a-aa28-0128666af1f2\" (UID: \"5717ffbe-ecef-4f7a-aa28-0128666af1f2\") " Dec 03 11:53:02 crc kubenswrapper[5052]: I1203 11:53:02.884595 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/5717ffbe-ecef-4f7a-aa28-0128666af1f2-config-data\") pod \"5717ffbe-ecef-4f7a-aa28-0128666af1f2\" (UID: \"5717ffbe-ecef-4f7a-aa28-0128666af1f2\") " Dec 03 11:53:02 crc kubenswrapper[5052]: I1203 11:53:02.884679 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cvc7q\" (UniqueName: \"kubernetes.io/projected/5717ffbe-ecef-4f7a-aa28-0128666af1f2-kube-api-access-cvc7q\") pod \"5717ffbe-ecef-4f7a-aa28-0128666af1f2\" (UID: \"5717ffbe-ecef-4f7a-aa28-0128666af1f2\") " Dec 03 11:53:02 crc kubenswrapper[5052]: I1203 11:53:02.889538 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5717ffbe-ecef-4f7a-aa28-0128666af1f2-kube-api-access-cvc7q" (OuterVolumeSpecName: "kube-api-access-cvc7q") pod "5717ffbe-ecef-4f7a-aa28-0128666af1f2" (UID: "5717ffbe-ecef-4f7a-aa28-0128666af1f2"). InnerVolumeSpecName "kube-api-access-cvc7q". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:53:02 crc kubenswrapper[5052]: I1203 11:53:02.912608 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5717ffbe-ecef-4f7a-aa28-0128666af1f2-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "5717ffbe-ecef-4f7a-aa28-0128666af1f2" (UID: "5717ffbe-ecef-4f7a-aa28-0128666af1f2"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:53:02 crc kubenswrapper[5052]: I1203 11:53:02.920963 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5717ffbe-ecef-4f7a-aa28-0128666af1f2-config-data" (OuterVolumeSpecName: "config-data") pod "5717ffbe-ecef-4f7a-aa28-0128666af1f2" (UID: "5717ffbe-ecef-4f7a-aa28-0128666af1f2"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:53:02 crc kubenswrapper[5052]: I1203 11:53:02.979940 5052 generic.go:334] "Generic (PLEG): container finished" podID="5717ffbe-ecef-4f7a-aa28-0128666af1f2" containerID="16ff4417366a31eb4b5554be80d8d29d8be9a3f9f1eab8a21506a01190850710" exitCode=0 Dec 03 11:53:02 crc kubenswrapper[5052]: I1203 11:53:02.980020 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 11:53:02 crc kubenswrapper[5052]: I1203 11:53:02.980040 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" event={"ID":"5717ffbe-ecef-4f7a-aa28-0128666af1f2","Type":"ContainerDied","Data":"16ff4417366a31eb4b5554be80d8d29d8be9a3f9f1eab8a21506a01190850710"} Dec 03 11:53:02 crc kubenswrapper[5052]: I1203 11:53:02.980077 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" event={"ID":"5717ffbe-ecef-4f7a-aa28-0128666af1f2","Type":"ContainerDied","Data":"47a22a99234d9d44f994a30cd7600b8d604bf38d8f6d03419963b86c52db8f18"} Dec 03 11:53:02 crc kubenswrapper[5052]: I1203 11:53:02.980106 5052 scope.go:117] "RemoveContainer" containerID="16ff4417366a31eb4b5554be80d8d29d8be9a3f9f1eab8a21506a01190850710" Dec 03 11:53:02 crc kubenswrapper[5052]: I1203 11:53:02.980612 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-metadata-0" podUID="8d641b96-28f4-4adc-9436-3c55585efaaa" containerName="nova-metadata-log" containerID="cri-o://c281e2aaabcb5590ee74e3835d9f310c5a45ca47f571d3e3f6c6ed7528906a1e" gracePeriod=30 Dec 03 11:53:02 crc kubenswrapper[5052]: I1203 11:53:02.980668 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-metadata-0" podUID="8d641b96-28f4-4adc-9436-3c55585efaaa" containerName="nova-metadata-metadata" containerID="cri-o://776190df2766601f4a5f2a415e444173e4d84a3653efda9cbec92bd5b8ac9b23" gracePeriod=30 Dec 03 11:53:02 crc kubenswrapper[5052]: I1203 11:53:02.986541 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-cvc7q\" (UniqueName: \"kubernetes.io/projected/5717ffbe-ecef-4f7a-aa28-0128666af1f2-kube-api-access-cvc7q\") on node \"crc\" DevicePath \"\"" Dec 03 11:53:02 crc kubenswrapper[5052]: I1203 11:53:02.986585 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5717ffbe-ecef-4f7a-aa28-0128666af1f2-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:53:02 crc kubenswrapper[5052]: I1203 11:53:02.986596 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/5717ffbe-ecef-4f7a-aa28-0128666af1f2-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:53:03 crc kubenswrapper[5052]: I1203 11:53:03.020164 5052 scope.go:117] "RemoveContainer" containerID="16ff4417366a31eb4b5554be80d8d29d8be9a3f9f1eab8a21506a01190850710" Dec 03 11:53:03 crc kubenswrapper[5052]: E1203 11:53:03.020487 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"16ff4417366a31eb4b5554be80d8d29d8be9a3f9f1eab8a21506a01190850710\": container with ID starting with 16ff4417366a31eb4b5554be80d8d29d8be9a3f9f1eab8a21506a01190850710 not found: ID does not exist" containerID="16ff4417366a31eb4b5554be80d8d29d8be9a3f9f1eab8a21506a01190850710" Dec 03 11:53:03 crc kubenswrapper[5052]: I1203 11:53:03.020546 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"16ff4417366a31eb4b5554be80d8d29d8be9a3f9f1eab8a21506a01190850710"} err="failed to get container status \"16ff4417366a31eb4b5554be80d8d29d8be9a3f9f1eab8a21506a01190850710\": rpc error: code = NotFound desc = could not find container \"16ff4417366a31eb4b5554be80d8d29d8be9a3f9f1eab8a21506a01190850710\": container with ID starting with 16ff4417366a31eb4b5554be80d8d29d8be9a3f9f1eab8a21506a01190850710 not found: ID does not exist" Dec 03 11:53:03 crc kubenswrapper[5052]: I1203 11:53:03.027112 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 11:53:03 crc kubenswrapper[5052]: I1203 11:53:03.038957 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 11:53:03 crc kubenswrapper[5052]: I1203 11:53:03.052796 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 11:53:03 crc kubenswrapper[5052]: E1203 11:53:03.053308 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="5717ffbe-ecef-4f7a-aa28-0128666af1f2" containerName="nova-cell1-novncproxy-novncproxy" Dec 03 11:53:03 crc kubenswrapper[5052]: I1203 11:53:03.053335 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="5717ffbe-ecef-4f7a-aa28-0128666af1f2" containerName="nova-cell1-novncproxy-novncproxy" Dec 03 11:53:03 crc kubenswrapper[5052]: I1203 11:53:03.053646 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="5717ffbe-ecef-4f7a-aa28-0128666af1f2" containerName="nova-cell1-novncproxy-novncproxy" Dec 03 11:53:03 crc kubenswrapper[5052]: I1203 11:53:03.054450 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 11:53:03 crc kubenswrapper[5052]: I1203 11:53:03.057978 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-nova-novncproxy-cell1-public-svc" Dec 03 11:53:03 crc kubenswrapper[5052]: I1203 11:53:03.058155 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell1-novncproxy-config-data" Dec 03 11:53:03 crc kubenswrapper[5052]: I1203 11:53:03.058403 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-nova-novncproxy-cell1-vencrypt" Dec 03 11:53:03 crc kubenswrapper[5052]: I1203 11:53:03.068503 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 11:53:03 crc kubenswrapper[5052]: I1203 11:53:03.190485 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/da3a810d-9fd0-4f18-8f4a-e896b94b1431-combined-ca-bundle\") pod \"nova-cell1-novncproxy-0\" (UID: \"da3a810d-9fd0-4f18-8f4a-e896b94b1431\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 11:53:03 crc kubenswrapper[5052]: I1203 11:53:03.190556 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nova-novncproxy-tls-certs\" (UniqueName: \"kubernetes.io/secret/da3a810d-9fd0-4f18-8f4a-e896b94b1431-nova-novncproxy-tls-certs\") pod \"nova-cell1-novncproxy-0\" (UID: \"da3a810d-9fd0-4f18-8f4a-e896b94b1431\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 11:53:03 crc kubenswrapper[5052]: I1203 11:53:03.190699 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/da3a810d-9fd0-4f18-8f4a-e896b94b1431-config-data\") pod \"nova-cell1-novncproxy-0\" (UID: \"da3a810d-9fd0-4f18-8f4a-e896b94b1431\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 11:53:03 crc kubenswrapper[5052]: I1203 11:53:03.190778 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"vencrypt-tls-certs\" (UniqueName: \"kubernetes.io/secret/da3a810d-9fd0-4f18-8f4a-e896b94b1431-vencrypt-tls-certs\") pod \"nova-cell1-novncproxy-0\" (UID: \"da3a810d-9fd0-4f18-8f4a-e896b94b1431\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 11:53:03 crc kubenswrapper[5052]: I1203 11:53:03.190888 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nm6dm\" (UniqueName: \"kubernetes.io/projected/da3a810d-9fd0-4f18-8f4a-e896b94b1431-kube-api-access-nm6dm\") pod \"nova-cell1-novncproxy-0\" (UID: \"da3a810d-9fd0-4f18-8f4a-e896b94b1431\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 11:53:03 crc kubenswrapper[5052]: I1203 11:53:03.293178 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"vencrypt-tls-certs\" (UniqueName: \"kubernetes.io/secret/da3a810d-9fd0-4f18-8f4a-e896b94b1431-vencrypt-tls-certs\") pod \"nova-cell1-novncproxy-0\" (UID: \"da3a810d-9fd0-4f18-8f4a-e896b94b1431\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 11:53:03 crc kubenswrapper[5052]: I1203 11:53:03.293233 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-nm6dm\" (UniqueName: \"kubernetes.io/projected/da3a810d-9fd0-4f18-8f4a-e896b94b1431-kube-api-access-nm6dm\") pod \"nova-cell1-novncproxy-0\" (UID: \"da3a810d-9fd0-4f18-8f4a-e896b94b1431\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 11:53:03 crc kubenswrapper[5052]: I1203 11:53:03.293338 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/da3a810d-9fd0-4f18-8f4a-e896b94b1431-combined-ca-bundle\") pod \"nova-cell1-novncproxy-0\" (UID: \"da3a810d-9fd0-4f18-8f4a-e896b94b1431\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 11:53:03 crc kubenswrapper[5052]: I1203 11:53:03.293364 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"nova-novncproxy-tls-certs\" (UniqueName: \"kubernetes.io/secret/da3a810d-9fd0-4f18-8f4a-e896b94b1431-nova-novncproxy-tls-certs\") pod \"nova-cell1-novncproxy-0\" (UID: \"da3a810d-9fd0-4f18-8f4a-e896b94b1431\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 11:53:03 crc kubenswrapper[5052]: I1203 11:53:03.293463 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/da3a810d-9fd0-4f18-8f4a-e896b94b1431-config-data\") pod \"nova-cell1-novncproxy-0\" (UID: \"da3a810d-9fd0-4f18-8f4a-e896b94b1431\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 11:53:03 crc kubenswrapper[5052]: I1203 11:53:03.298426 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"vencrypt-tls-certs\" (UniqueName: \"kubernetes.io/secret/da3a810d-9fd0-4f18-8f4a-e896b94b1431-vencrypt-tls-certs\") pod \"nova-cell1-novncproxy-0\" (UID: \"da3a810d-9fd0-4f18-8f4a-e896b94b1431\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 11:53:03 crc kubenswrapper[5052]: I1203 11:53:03.299021 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/da3a810d-9fd0-4f18-8f4a-e896b94b1431-config-data\") pod \"nova-cell1-novncproxy-0\" (UID: \"da3a810d-9fd0-4f18-8f4a-e896b94b1431\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 11:53:03 crc kubenswrapper[5052]: I1203 11:53:03.299081 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/da3a810d-9fd0-4f18-8f4a-e896b94b1431-combined-ca-bundle\") pod \"nova-cell1-novncproxy-0\" (UID: \"da3a810d-9fd0-4f18-8f4a-e896b94b1431\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 11:53:03 crc kubenswrapper[5052]: I1203 11:53:03.299487 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"nova-novncproxy-tls-certs\" (UniqueName: \"kubernetes.io/secret/da3a810d-9fd0-4f18-8f4a-e896b94b1431-nova-novncproxy-tls-certs\") pod \"nova-cell1-novncproxy-0\" (UID: \"da3a810d-9fd0-4f18-8f4a-e896b94b1431\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 11:53:03 crc kubenswrapper[5052]: I1203 11:53:03.317704 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-nm6dm\" (UniqueName: \"kubernetes.io/projected/da3a810d-9fd0-4f18-8f4a-e896b94b1431-kube-api-access-nm6dm\") pod \"nova-cell1-novncproxy-0\" (UID: \"da3a810d-9fd0-4f18-8f4a-e896b94b1431\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 11:53:03 crc kubenswrapper[5052]: I1203 11:53:03.372385 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 11:53:03 crc kubenswrapper[5052]: I1203 11:53:03.559932 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:53:03 crc kubenswrapper[5052]: I1203 11:53:03.573062 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:53:03 crc kubenswrapper[5052]: I1203 11:53:03.573113 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:53:03 crc kubenswrapper[5052]: I1203 11:53:03.661626 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:53:03 crc kubenswrapper[5052]: I1203 11:53:03.690115 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5717ffbe-ecef-4f7a-aa28-0128666af1f2" path="/var/lib/kubelet/pods/5717ffbe-ecef-4f7a-aa28-0128666af1f2/volumes" Dec 03 11:53:03 crc kubenswrapper[5052]: I1203 11:53:03.821579 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/8d641b96-28f4-4adc-9436-3c55585efaaa-logs\") pod \"8d641b96-28f4-4adc-9436-3c55585efaaa\" (UID: \"8d641b96-28f4-4adc-9436-3c55585efaaa\") " Dec 03 11:53:03 crc kubenswrapper[5052]: I1203 11:53:03.821656 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/8d641b96-28f4-4adc-9436-3c55585efaaa-config-data\") pod \"8d641b96-28f4-4adc-9436-3c55585efaaa\" (UID: \"8d641b96-28f4-4adc-9436-3c55585efaaa\") " Dec 03 11:53:03 crc kubenswrapper[5052]: I1203 11:53:03.821710 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x9zjm\" (UniqueName: \"kubernetes.io/projected/8d641b96-28f4-4adc-9436-3c55585efaaa-kube-api-access-x9zjm\") pod \"8d641b96-28f4-4adc-9436-3c55585efaaa\" (UID: \"8d641b96-28f4-4adc-9436-3c55585efaaa\") " Dec 03 11:53:03 crc kubenswrapper[5052]: I1203 11:53:03.821829 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8d641b96-28f4-4adc-9436-3c55585efaaa-combined-ca-bundle\") pod \"8d641b96-28f4-4adc-9436-3c55585efaaa\" (UID: \"8d641b96-28f4-4adc-9436-3c55585efaaa\") " Dec 03 11:53:03 crc kubenswrapper[5052]: I1203 11:53:03.821960 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8d641b96-28f4-4adc-9436-3c55585efaaa-logs" (OuterVolumeSpecName: "logs") pod "8d641b96-28f4-4adc-9436-3c55585efaaa" (UID: "8d641b96-28f4-4adc-9436-3c55585efaaa"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:53:03 crc kubenswrapper[5052]: I1203 11:53:03.822413 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/8d641b96-28f4-4adc-9436-3c55585efaaa-logs\") on node \"crc\" DevicePath \"\"" Dec 03 11:53:03 crc kubenswrapper[5052]: I1203 11:53:03.827734 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8d641b96-28f4-4adc-9436-3c55585efaaa-kube-api-access-x9zjm" (OuterVolumeSpecName: "kube-api-access-x9zjm") pod "8d641b96-28f4-4adc-9436-3c55585efaaa" (UID: "8d641b96-28f4-4adc-9436-3c55585efaaa"). InnerVolumeSpecName "kube-api-access-x9zjm". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:53:03 crc kubenswrapper[5052]: W1203 11:53:03.835885 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podda3a810d_9fd0_4f18_8f4a_e896b94b1431.slice/crio-3a8314004123a6281ae05824b2b1fadbf5096f8a07470680389b4ffa3aff159c WatchSource:0}: Error finding container 3a8314004123a6281ae05824b2b1fadbf5096f8a07470680389b4ffa3aff159c: Status 404 returned error can't find the container with id 3a8314004123a6281ae05824b2b1fadbf5096f8a07470680389b4ffa3aff159c Dec 03 11:53:03 crc kubenswrapper[5052]: I1203 11:53:03.839945 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 11:53:03 crc kubenswrapper[5052]: I1203 11:53:03.853573 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8d641b96-28f4-4adc-9436-3c55585efaaa-config-data" (OuterVolumeSpecName: "config-data") pod "8d641b96-28f4-4adc-9436-3c55585efaaa" (UID: "8d641b96-28f4-4adc-9436-3c55585efaaa"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:53:03 crc kubenswrapper[5052]: I1203 11:53:03.860132 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8d641b96-28f4-4adc-9436-3c55585efaaa-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "8d641b96-28f4-4adc-9436-3c55585efaaa" (UID: "8d641b96-28f4-4adc-9436-3c55585efaaa"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:53:03 crc kubenswrapper[5052]: I1203 11:53:03.865696 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:53:03 crc kubenswrapper[5052]: I1203 11:53:03.929726 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8d641b96-28f4-4adc-9436-3c55585efaaa-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:53:03 crc kubenswrapper[5052]: I1203 11:53:03.930074 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/8d641b96-28f4-4adc-9436-3c55585efaaa-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:53:03 crc kubenswrapper[5052]: I1203 11:53:03.930086 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-x9zjm\" (UniqueName: \"kubernetes.io/projected/8d641b96-28f4-4adc-9436-3c55585efaaa-kube-api-access-x9zjm\") on node \"crc\" DevicePath \"\"" Dec 03 11:53:03 crc kubenswrapper[5052]: I1203 11:53:03.992608 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" event={"ID":"da3a810d-9fd0-4f18-8f4a-e896b94b1431","Type":"ContainerStarted","Data":"3a8314004123a6281ae05824b2b1fadbf5096f8a07470680389b4ffa3aff159c"} Dec 03 11:53:03 crc kubenswrapper[5052]: I1203 11:53:03.996063 5052 generic.go:334] "Generic (PLEG): container finished" podID="8d641b96-28f4-4adc-9436-3c55585efaaa" containerID="776190df2766601f4a5f2a415e444173e4d84a3653efda9cbec92bd5b8ac9b23" exitCode=0 Dec 03 11:53:03 crc kubenswrapper[5052]: I1203 11:53:03.996102 5052 generic.go:334] "Generic (PLEG): container finished" podID="8d641b96-28f4-4adc-9436-3c55585efaaa" containerID="c281e2aaabcb5590ee74e3835d9f310c5a45ca47f571d3e3f6c6ed7528906a1e" exitCode=143 Dec 03 11:53:03 crc kubenswrapper[5052]: I1203 11:53:03.996095 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"8d641b96-28f4-4adc-9436-3c55585efaaa","Type":"ContainerDied","Data":"776190df2766601f4a5f2a415e444173e4d84a3653efda9cbec92bd5b8ac9b23"} Dec 03 11:53:03 crc kubenswrapper[5052]: I1203 11:53:03.996172 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"8d641b96-28f4-4adc-9436-3c55585efaaa","Type":"ContainerDied","Data":"c281e2aaabcb5590ee74e3835d9f310c5a45ca47f571d3e3f6c6ed7528906a1e"} Dec 03 11:53:03 crc kubenswrapper[5052]: I1203 11:53:03.996186 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"8d641b96-28f4-4adc-9436-3c55585efaaa","Type":"ContainerDied","Data":"433c28123208e2ac7640c3163ae547684125a775aa16b8437c292c8b1d656025"} Dec 03 11:53:03 crc kubenswrapper[5052]: I1203 11:53:03.996207 5052 scope.go:117] "RemoveContainer" containerID="776190df2766601f4a5f2a415e444173e4d84a3653efda9cbec92bd5b8ac9b23" Dec 03 11:53:03 crc kubenswrapper[5052]: I1203 11:53:03.996140 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:53:04 crc kubenswrapper[5052]: I1203 11:53:04.024961 5052 scope.go:117] "RemoveContainer" containerID="c281e2aaabcb5590ee74e3835d9f310c5a45ca47f571d3e3f6c6ed7528906a1e" Dec 03 11:53:04 crc kubenswrapper[5052]: I1203 11:53:04.065435 5052 scope.go:117] "RemoveContainer" containerID="776190df2766601f4a5f2a415e444173e4d84a3653efda9cbec92bd5b8ac9b23" Dec 03 11:53:04 crc kubenswrapper[5052]: E1203 11:53:04.069110 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"776190df2766601f4a5f2a415e444173e4d84a3653efda9cbec92bd5b8ac9b23\": container with ID starting with 776190df2766601f4a5f2a415e444173e4d84a3653efda9cbec92bd5b8ac9b23 not found: ID does not exist" containerID="776190df2766601f4a5f2a415e444173e4d84a3653efda9cbec92bd5b8ac9b23" Dec 03 11:53:04 crc kubenswrapper[5052]: I1203 11:53:04.069153 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"776190df2766601f4a5f2a415e444173e4d84a3653efda9cbec92bd5b8ac9b23"} err="failed to get container status \"776190df2766601f4a5f2a415e444173e4d84a3653efda9cbec92bd5b8ac9b23\": rpc error: code = NotFound desc = could not find container \"776190df2766601f4a5f2a415e444173e4d84a3653efda9cbec92bd5b8ac9b23\": container with ID starting with 776190df2766601f4a5f2a415e444173e4d84a3653efda9cbec92bd5b8ac9b23 not found: ID does not exist" Dec 03 11:53:04 crc kubenswrapper[5052]: I1203 11:53:04.069177 5052 scope.go:117] "RemoveContainer" containerID="c281e2aaabcb5590ee74e3835d9f310c5a45ca47f571d3e3f6c6ed7528906a1e" Dec 03 11:53:04 crc kubenswrapper[5052]: E1203 11:53:04.072619 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c281e2aaabcb5590ee74e3835d9f310c5a45ca47f571d3e3f6c6ed7528906a1e\": container with ID starting with c281e2aaabcb5590ee74e3835d9f310c5a45ca47f571d3e3f6c6ed7528906a1e not found: ID does not exist" containerID="c281e2aaabcb5590ee74e3835d9f310c5a45ca47f571d3e3f6c6ed7528906a1e" Dec 03 11:53:04 crc kubenswrapper[5052]: I1203 11:53:04.072673 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c281e2aaabcb5590ee74e3835d9f310c5a45ca47f571d3e3f6c6ed7528906a1e"} err="failed to get container status \"c281e2aaabcb5590ee74e3835d9f310c5a45ca47f571d3e3f6c6ed7528906a1e\": rpc error: code = NotFound desc = could not find container \"c281e2aaabcb5590ee74e3835d9f310c5a45ca47f571d3e3f6c6ed7528906a1e\": container with ID starting with c281e2aaabcb5590ee74e3835d9f310c5a45ca47f571d3e3f6c6ed7528906a1e not found: ID does not exist" Dec 03 11:53:04 crc kubenswrapper[5052]: I1203 11:53:04.072706 5052 scope.go:117] "RemoveContainer" containerID="776190df2766601f4a5f2a415e444173e4d84a3653efda9cbec92bd5b8ac9b23" Dec 03 11:53:04 crc kubenswrapper[5052]: I1203 11:53:04.072859 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 11:53:04 crc kubenswrapper[5052]: I1203 11:53:04.074099 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"776190df2766601f4a5f2a415e444173e4d84a3653efda9cbec92bd5b8ac9b23"} err="failed to get container status \"776190df2766601f4a5f2a415e444173e4d84a3653efda9cbec92bd5b8ac9b23\": rpc error: code = NotFound desc = could not find container \"776190df2766601f4a5f2a415e444173e4d84a3653efda9cbec92bd5b8ac9b23\": container with ID starting with 776190df2766601f4a5f2a415e444173e4d84a3653efda9cbec92bd5b8ac9b23 not found: ID does not exist" Dec 03 11:53:04 crc kubenswrapper[5052]: I1203 11:53:04.074140 5052 scope.go:117] "RemoveContainer" containerID="c281e2aaabcb5590ee74e3835d9f310c5a45ca47f571d3e3f6c6ed7528906a1e" Dec 03 11:53:04 crc kubenswrapper[5052]: I1203 11:53:04.077134 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c281e2aaabcb5590ee74e3835d9f310c5a45ca47f571d3e3f6c6ed7528906a1e"} err="failed to get container status \"c281e2aaabcb5590ee74e3835d9f310c5a45ca47f571d3e3f6c6ed7528906a1e\": rpc error: code = NotFound desc = could not find container \"c281e2aaabcb5590ee74e3835d9f310c5a45ca47f571d3e3f6c6ed7528906a1e\": container with ID starting with c281e2aaabcb5590ee74e3835d9f310c5a45ca47f571d3e3f6c6ed7528906a1e not found: ID does not exist" Dec 03 11:53:04 crc kubenswrapper[5052]: I1203 11:53:04.096480 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 11:53:04 crc kubenswrapper[5052]: I1203 11:53:04.118427 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 11:53:04 crc kubenswrapper[5052]: E1203 11:53:04.119968 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="8d641b96-28f4-4adc-9436-3c55585efaaa" containerName="nova-metadata-metadata" Dec 03 11:53:04 crc kubenswrapper[5052]: I1203 11:53:04.119997 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="8d641b96-28f4-4adc-9436-3c55585efaaa" containerName="nova-metadata-metadata" Dec 03 11:53:04 crc kubenswrapper[5052]: E1203 11:53:04.120032 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="8d641b96-28f4-4adc-9436-3c55585efaaa" containerName="nova-metadata-log" Dec 03 11:53:04 crc kubenswrapper[5052]: I1203 11:53:04.120043 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="8d641b96-28f4-4adc-9436-3c55585efaaa" containerName="nova-metadata-log" Dec 03 11:53:04 crc kubenswrapper[5052]: I1203 11:53:04.120247 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="8d641b96-28f4-4adc-9436-3c55585efaaa" containerName="nova-metadata-log" Dec 03 11:53:04 crc kubenswrapper[5052]: I1203 11:53:04.120268 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="8d641b96-28f4-4adc-9436-3c55585efaaa" containerName="nova-metadata-metadata" Dec 03 11:53:04 crc kubenswrapper[5052]: I1203 11:53:04.122117 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:53:04 crc kubenswrapper[5052]: I1203 11:53:04.124311 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-nova-metadata-internal-svc" Dec 03 11:53:04 crc kubenswrapper[5052]: I1203 11:53:04.125158 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-metadata-config-data" Dec 03 11:53:04 crc kubenswrapper[5052]: I1203 11:53:04.131370 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 11:53:04 crc kubenswrapper[5052]: I1203 11:53:04.236771 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8699ba4f-1002-4ba2-9358-17a5600bcff8-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"8699ba4f-1002-4ba2-9358-17a5600bcff8\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:53:04 crc kubenswrapper[5052]: I1203 11:53:04.236837 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sgwwt\" (UniqueName: \"kubernetes.io/projected/8699ba4f-1002-4ba2-9358-17a5600bcff8-kube-api-access-sgwwt\") pod \"nova-metadata-0\" (UID: \"8699ba4f-1002-4ba2-9358-17a5600bcff8\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:53:04 crc kubenswrapper[5052]: I1203 11:53:04.236870 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/8699ba4f-1002-4ba2-9358-17a5600bcff8-config-data\") pod \"nova-metadata-0\" (UID: \"8699ba4f-1002-4ba2-9358-17a5600bcff8\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:53:04 crc kubenswrapper[5052]: I1203 11:53:04.236892 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/8699ba4f-1002-4ba2-9358-17a5600bcff8-logs\") pod \"nova-metadata-0\" (UID: \"8699ba4f-1002-4ba2-9358-17a5600bcff8\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:53:04 crc kubenswrapper[5052]: I1203 11:53:04.237041 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/8699ba4f-1002-4ba2-9358-17a5600bcff8-nova-metadata-tls-certs\") pod \"nova-metadata-0\" (UID: \"8699ba4f-1002-4ba2-9358-17a5600bcff8\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:53:04 crc kubenswrapper[5052]: I1203 11:53:04.338778 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/8699ba4f-1002-4ba2-9358-17a5600bcff8-nova-metadata-tls-certs\") pod \"nova-metadata-0\" (UID: \"8699ba4f-1002-4ba2-9358-17a5600bcff8\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:53:04 crc kubenswrapper[5052]: I1203 11:53:04.338906 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8699ba4f-1002-4ba2-9358-17a5600bcff8-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"8699ba4f-1002-4ba2-9358-17a5600bcff8\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:53:04 crc kubenswrapper[5052]: I1203 11:53:04.338941 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-sgwwt\" (UniqueName: \"kubernetes.io/projected/8699ba4f-1002-4ba2-9358-17a5600bcff8-kube-api-access-sgwwt\") pod \"nova-metadata-0\" (UID: \"8699ba4f-1002-4ba2-9358-17a5600bcff8\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:53:04 crc kubenswrapper[5052]: I1203 11:53:04.338965 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/8699ba4f-1002-4ba2-9358-17a5600bcff8-config-data\") pod \"nova-metadata-0\" (UID: \"8699ba4f-1002-4ba2-9358-17a5600bcff8\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:53:04 crc kubenswrapper[5052]: I1203 11:53:04.338993 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/8699ba4f-1002-4ba2-9358-17a5600bcff8-logs\") pod \"nova-metadata-0\" (UID: \"8699ba4f-1002-4ba2-9358-17a5600bcff8\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:53:04 crc kubenswrapper[5052]: I1203 11:53:04.339551 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/8699ba4f-1002-4ba2-9358-17a5600bcff8-logs\") pod \"nova-metadata-0\" (UID: \"8699ba4f-1002-4ba2-9358-17a5600bcff8\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:53:04 crc kubenswrapper[5052]: I1203 11:53:04.344600 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/8699ba4f-1002-4ba2-9358-17a5600bcff8-config-data\") pod \"nova-metadata-0\" (UID: \"8699ba4f-1002-4ba2-9358-17a5600bcff8\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:53:04 crc kubenswrapper[5052]: I1203 11:53:04.345734 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8699ba4f-1002-4ba2-9358-17a5600bcff8-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"8699ba4f-1002-4ba2-9358-17a5600bcff8\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:53:04 crc kubenswrapper[5052]: I1203 11:53:04.346382 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/8699ba4f-1002-4ba2-9358-17a5600bcff8-nova-metadata-tls-certs\") pod \"nova-metadata-0\" (UID: \"8699ba4f-1002-4ba2-9358-17a5600bcff8\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:53:04 crc kubenswrapper[5052]: I1203 11:53:04.361241 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-sgwwt\" (UniqueName: \"kubernetes.io/projected/8699ba4f-1002-4ba2-9358-17a5600bcff8-kube-api-access-sgwwt\") pod \"nova-metadata-0\" (UID: \"8699ba4f-1002-4ba2-9358-17a5600bcff8\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:53:04 crc kubenswrapper[5052]: I1203 11:53:04.445404 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:53:04 crc kubenswrapper[5052]: I1203 11:53:04.902323 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 11:53:05 crc kubenswrapper[5052]: I1203 11:53:05.009420 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"8699ba4f-1002-4ba2-9358-17a5600bcff8","Type":"ContainerStarted","Data":"cb4638009981ca86353947fed0da344a6eef3f474f703ad176eb004b88a85f26"} Dec 03 11:53:05 crc kubenswrapper[5052]: I1203 11:53:05.011443 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" event={"ID":"da3a810d-9fd0-4f18-8f4a-e896b94b1431","Type":"ContainerStarted","Data":"80ffd33af4023d6b6f0f24ec3bd666459e45b4216c33dab791e932793c7535bf"} Dec 03 11:53:05 crc kubenswrapper[5052]: I1203 11:53:05.042803 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" podStartSLOduration=2.042784631 podStartE2EDuration="2.042784631s" podCreationTimestamp="2025-12-03 11:53:03 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:53:05.028313788 +0000 UTC m=+1975.616555552" watchObservedRunningTime="2025-12-03 11:53:05.042784631 +0000 UTC m=+1975.631026395" Dec 03 11:53:05 crc kubenswrapper[5052]: I1203 11:53:05.686979 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8d641b96-28f4-4adc-9436-3c55585efaaa" path="/var/lib/kubelet/pods/8d641b96-28f4-4adc-9436-3c55585efaaa/volumes" Dec 03 11:53:06 crc kubenswrapper[5052]: I1203 11:53:06.023791 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"8699ba4f-1002-4ba2-9358-17a5600bcff8","Type":"ContainerStarted","Data":"03c746c1a2fa9c652460e2d9021afa7c6d8a54756fd60dd1a4125e6904207ea1"} Dec 03 11:53:06 crc kubenswrapper[5052]: I1203 11:53:06.023863 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"8699ba4f-1002-4ba2-9358-17a5600bcff8","Type":"ContainerStarted","Data":"82b44062050a9b637ebc60692bad3aafe0bcf829534275f1a2fec3faecd8b697"} Dec 03 11:53:06 crc kubenswrapper[5052]: I1203 11:53:06.045359 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-metadata-0" podStartSLOduration=2.045337566 podStartE2EDuration="2.045337566s" podCreationTimestamp="2025-12-03 11:53:04 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:53:06.039749246 +0000 UTC m=+1976.627991070" watchObservedRunningTime="2025-12-03 11:53:06.045337566 +0000 UTC m=+1976.633579330" Dec 03 11:53:07 crc kubenswrapper[5052]: I1203 11:53:07.411663 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/kube-state-metrics-0"] Dec 03 11:53:07 crc kubenswrapper[5052]: I1203 11:53:07.412777 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/kube-state-metrics-0" podUID="d21ea848-4642-4f06-b777-ae420fd6330e" containerName="kube-state-metrics" containerID="cri-o://3a04155db4f0c7a1d45628519f9e6444be8550227cc239c1ea6e91f467326d18" gracePeriod=30 Dec 03 11:53:07 crc kubenswrapper[5052]: E1203 11:53:07.437211 5052 cadvisor_stats_provider.go:516] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2b63c55a_e29f_45a8_8f5d_4166c162c3cb.slice/crio-conmon-2693072ee37641b8c8369038546f2b36325a5526bdc8ec1217de284d5fa41724.scope\": RecentStats: unable to find data in memory cache]" Dec 03 11:53:07 crc kubenswrapper[5052]: I1203 11:53:07.920224 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 11:53:08 crc kubenswrapper[5052]: I1203 11:53:08.015562 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-84ptz\" (UniqueName: \"kubernetes.io/projected/d21ea848-4642-4f06-b777-ae420fd6330e-kube-api-access-84ptz\") pod \"d21ea848-4642-4f06-b777-ae420fd6330e\" (UID: \"d21ea848-4642-4f06-b777-ae420fd6330e\") " Dec 03 11:53:08 crc kubenswrapper[5052]: I1203 11:53:08.026927 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d21ea848-4642-4f06-b777-ae420fd6330e-kube-api-access-84ptz" (OuterVolumeSpecName: "kube-api-access-84ptz") pod "d21ea848-4642-4f06-b777-ae420fd6330e" (UID: "d21ea848-4642-4f06-b777-ae420fd6330e"). InnerVolumeSpecName "kube-api-access-84ptz". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:53:08 crc kubenswrapper[5052]: I1203 11:53:08.044466 5052 generic.go:334] "Generic (PLEG): container finished" podID="6500a947-382b-4269-b31b-5b518c858a6f" containerID="47ff9ffb2ee04fc839b0023d8e10c46ffdf46d84f9e4a70a36307e82f3f234da" exitCode=0 Dec 03 11:53:08 crc kubenswrapper[5052]: I1203 11:53:08.044553 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-cell-mapping-7v46q" event={"ID":"6500a947-382b-4269-b31b-5b518c858a6f","Type":"ContainerDied","Data":"47ff9ffb2ee04fc839b0023d8e10c46ffdf46d84f9e4a70a36307e82f3f234da"} Dec 03 11:53:08 crc kubenswrapper[5052]: I1203 11:53:08.047616 5052 generic.go:334] "Generic (PLEG): container finished" podID="d21ea848-4642-4f06-b777-ae420fd6330e" containerID="3a04155db4f0c7a1d45628519f9e6444be8550227cc239c1ea6e91f467326d18" exitCode=2 Dec 03 11:53:08 crc kubenswrapper[5052]: I1203 11:53:08.047698 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 11:53:08 crc kubenswrapper[5052]: I1203 11:53:08.052183 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/kube-state-metrics-0" event={"ID":"d21ea848-4642-4f06-b777-ae420fd6330e","Type":"ContainerDied","Data":"3a04155db4f0c7a1d45628519f9e6444be8550227cc239c1ea6e91f467326d18"} Dec 03 11:53:08 crc kubenswrapper[5052]: I1203 11:53:08.052269 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/kube-state-metrics-0" event={"ID":"d21ea848-4642-4f06-b777-ae420fd6330e","Type":"ContainerDied","Data":"643de97d8ca468a005dd92098fc487f71f02b13e452f12905f1e816ee46ad6d0"} Dec 03 11:53:08 crc kubenswrapper[5052]: I1203 11:53:08.052297 5052 scope.go:117] "RemoveContainer" containerID="3a04155db4f0c7a1d45628519f9e6444be8550227cc239c1ea6e91f467326d18" Dec 03 11:53:08 crc kubenswrapper[5052]: I1203 11:53:08.103723 5052 scope.go:117] "RemoveContainer" containerID="3a04155db4f0c7a1d45628519f9e6444be8550227cc239c1ea6e91f467326d18" Dec 03 11:53:08 crc kubenswrapper[5052]: E1203 11:53:08.107835 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3a04155db4f0c7a1d45628519f9e6444be8550227cc239c1ea6e91f467326d18\": container with ID starting with 3a04155db4f0c7a1d45628519f9e6444be8550227cc239c1ea6e91f467326d18 not found: ID does not exist" containerID="3a04155db4f0c7a1d45628519f9e6444be8550227cc239c1ea6e91f467326d18" Dec 03 11:53:08 crc kubenswrapper[5052]: I1203 11:53:08.107910 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3a04155db4f0c7a1d45628519f9e6444be8550227cc239c1ea6e91f467326d18"} err="failed to get container status \"3a04155db4f0c7a1d45628519f9e6444be8550227cc239c1ea6e91f467326d18\": rpc error: code = NotFound desc = could not find container \"3a04155db4f0c7a1d45628519f9e6444be8550227cc239c1ea6e91f467326d18\": container with ID starting with 3a04155db4f0c7a1d45628519f9e6444be8550227cc239c1ea6e91f467326d18 not found: ID does not exist" Dec 03 11:53:08 crc kubenswrapper[5052]: I1203 11:53:08.109370 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/kube-state-metrics-0"] Dec 03 11:53:08 crc kubenswrapper[5052]: I1203 11:53:08.117420 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-84ptz\" (UniqueName: \"kubernetes.io/projected/d21ea848-4642-4f06-b777-ae420fd6330e-kube-api-access-84ptz\") on node \"crc\" DevicePath \"\"" Dec 03 11:53:08 crc kubenswrapper[5052]: I1203 11:53:08.120929 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/kube-state-metrics-0"] Dec 03 11:53:08 crc kubenswrapper[5052]: I1203 11:53:08.133181 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/kube-state-metrics-0"] Dec 03 11:53:08 crc kubenswrapper[5052]: E1203 11:53:08.133619 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="d21ea848-4642-4f06-b777-ae420fd6330e" containerName="kube-state-metrics" Dec 03 11:53:08 crc kubenswrapper[5052]: I1203 11:53:08.133638 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="d21ea848-4642-4f06-b777-ae420fd6330e" containerName="kube-state-metrics" Dec 03 11:53:08 crc kubenswrapper[5052]: I1203 11:53:08.133815 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="d21ea848-4642-4f06-b777-ae420fd6330e" containerName="kube-state-metrics" Dec 03 11:53:08 crc kubenswrapper[5052]: I1203 11:53:08.134502 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 11:53:08 crc kubenswrapper[5052]: I1203 11:53:08.138676 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-kube-state-metrics-svc" Dec 03 11:53:08 crc kubenswrapper[5052]: I1203 11:53:08.138926 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"kube-state-metrics-tls-config" Dec 03 11:53:08 crc kubenswrapper[5052]: I1203 11:53:08.149166 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/kube-state-metrics-0"] Dec 03 11:53:08 crc kubenswrapper[5052]: I1203 11:53:08.219805 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls-certs\" (UniqueName: \"kubernetes.io/secret/88e7db65-8253-474b-b115-cd3f6edaf85c-kube-state-metrics-tls-certs\") pod \"kube-state-metrics-0\" (UID: \"88e7db65-8253-474b-b115-cd3f6edaf85c\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 11:53:08 crc kubenswrapper[5052]: I1203 11:53:08.220161 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7jb5v\" (UniqueName: \"kubernetes.io/projected/88e7db65-8253-474b-b115-cd3f6edaf85c-kube-api-access-7jb5v\") pod \"kube-state-metrics-0\" (UID: \"88e7db65-8253-474b-b115-cd3f6edaf85c\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 11:53:08 crc kubenswrapper[5052]: I1203 11:53:08.220293 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/88e7db65-8253-474b-b115-cd3f6edaf85c-combined-ca-bundle\") pod \"kube-state-metrics-0\" (UID: \"88e7db65-8253-474b-b115-cd3f6edaf85c\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 11:53:08 crc kubenswrapper[5052]: I1203 11:53:08.220358 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls-config\" (UniqueName: \"kubernetes.io/secret/88e7db65-8253-474b-b115-cd3f6edaf85c-kube-state-metrics-tls-config\") pod \"kube-state-metrics-0\" (UID: \"88e7db65-8253-474b-b115-cd3f6edaf85c\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 11:53:08 crc kubenswrapper[5052]: I1203 11:53:08.322799 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-7jb5v\" (UniqueName: \"kubernetes.io/projected/88e7db65-8253-474b-b115-cd3f6edaf85c-kube-api-access-7jb5v\") pod \"kube-state-metrics-0\" (UID: \"88e7db65-8253-474b-b115-cd3f6edaf85c\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 11:53:08 crc kubenswrapper[5052]: I1203 11:53:08.322852 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/88e7db65-8253-474b-b115-cd3f6edaf85c-combined-ca-bundle\") pod \"kube-state-metrics-0\" (UID: \"88e7db65-8253-474b-b115-cd3f6edaf85c\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 11:53:08 crc kubenswrapper[5052]: I1203 11:53:08.322889 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls-config\" (UniqueName: \"kubernetes.io/secret/88e7db65-8253-474b-b115-cd3f6edaf85c-kube-state-metrics-tls-config\") pod \"kube-state-metrics-0\" (UID: \"88e7db65-8253-474b-b115-cd3f6edaf85c\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 11:53:08 crc kubenswrapper[5052]: I1203 11:53:08.322955 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls-certs\" (UniqueName: \"kubernetes.io/secret/88e7db65-8253-474b-b115-cd3f6edaf85c-kube-state-metrics-tls-certs\") pod \"kube-state-metrics-0\" (UID: \"88e7db65-8253-474b-b115-cd3f6edaf85c\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 11:53:08 crc kubenswrapper[5052]: I1203 11:53:08.327816 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls-config\" (UniqueName: \"kubernetes.io/secret/88e7db65-8253-474b-b115-cd3f6edaf85c-kube-state-metrics-tls-config\") pod \"kube-state-metrics-0\" (UID: \"88e7db65-8253-474b-b115-cd3f6edaf85c\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 11:53:08 crc kubenswrapper[5052]: I1203 11:53:08.327907 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/88e7db65-8253-474b-b115-cd3f6edaf85c-combined-ca-bundle\") pod \"kube-state-metrics-0\" (UID: \"88e7db65-8253-474b-b115-cd3f6edaf85c\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 11:53:08 crc kubenswrapper[5052]: I1203 11:53:08.328852 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls-certs\" (UniqueName: \"kubernetes.io/secret/88e7db65-8253-474b-b115-cd3f6edaf85c-kube-state-metrics-tls-certs\") pod \"kube-state-metrics-0\" (UID: \"88e7db65-8253-474b-b115-cd3f6edaf85c\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 11:53:08 crc kubenswrapper[5052]: I1203 11:53:08.344836 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-7jb5v\" (UniqueName: \"kubernetes.io/projected/88e7db65-8253-474b-b115-cd3f6edaf85c-kube-api-access-7jb5v\") pod \"kube-state-metrics-0\" (UID: \"88e7db65-8253-474b-b115-cd3f6edaf85c\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 11:53:08 crc kubenswrapper[5052]: I1203 11:53:08.375461 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 11:53:08 crc kubenswrapper[5052]: I1203 11:53:08.460184 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 11:53:08 crc kubenswrapper[5052]: I1203 11:53:08.560197 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:53:08 crc kubenswrapper[5052]: I1203 11:53:08.592568 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:53:08 crc kubenswrapper[5052]: I1203 11:53:08.592644 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:53:08 crc kubenswrapper[5052]: I1203 11:53:08.599124 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:53:08 crc kubenswrapper[5052]: I1203 11:53:08.919622 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/kube-state-metrics-0"] Dec 03 11:53:09 crc kubenswrapper[5052]: I1203 11:53:09.059940 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/kube-state-metrics-0" event={"ID":"88e7db65-8253-474b-b115-cd3f6edaf85c","Type":"ContainerStarted","Data":"7a7ebfc7c11351ab35dc3f7e2e09b70208c51b2ecfaedabd2d35aa5d1babb04f"} Dec 03 11:53:09 crc kubenswrapper[5052]: I1203 11:53:09.107264 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:53:09 crc kubenswrapper[5052]: I1203 11:53:09.264673 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:53:09 crc kubenswrapper[5052]: I1203 11:53:09.264916 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="3bae2410-56dc-42bc-8974-4da004ac0b28" containerName="ceilometer-central-agent" containerID="cri-o://3d4e37cd5ebdf25795857b70ca2eda328dcf4ce9f44215affa2902c0ee5c7ec8" gracePeriod=30 Dec 03 11:53:09 crc kubenswrapper[5052]: I1203 11:53:09.265329 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="3bae2410-56dc-42bc-8974-4da004ac0b28" containerName="proxy-httpd" containerID="cri-o://f9fcf25c0c0d7f76435313ac33854df45d1892029892dae5e2816386e51e35d6" gracePeriod=30 Dec 03 11:53:09 crc kubenswrapper[5052]: I1203 11:53:09.265349 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="3bae2410-56dc-42bc-8974-4da004ac0b28" containerName="ceilometer-notification-agent" containerID="cri-o://8edea1b7e7603813f9cee40e80060c8fe7cdde3a3fe81da594f714c8848d10a6" gracePeriod=30 Dec 03 11:53:09 crc kubenswrapper[5052]: I1203 11:53:09.265333 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="3bae2410-56dc-42bc-8974-4da004ac0b28" containerName="sg-core" containerID="cri-o://dd21b0bd313fbea7fd9fbbc6df20b5977fa0f518bc12404bb5083a4c2ee3b4ce" gracePeriod=30 Dec 03 11:53:09 crc kubenswrapper[5052]: I1203 11:53:09.445912 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:53:09 crc kubenswrapper[5052]: I1203 11:53:09.446171 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:53:09 crc kubenswrapper[5052]: I1203 11:53:09.682783 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-api-0" podUID="a4ee181e-0136-4d6c-aad6-e0ffcc2cb9db" containerName="nova-api-api" probeResult="failure" output="Get \"http://10.217.1.121:8774/\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Dec 03 11:53:09 crc kubenswrapper[5052]: I1203 11:53:09.682974 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-api-0" podUID="a4ee181e-0136-4d6c-aad6-e0ffcc2cb9db" containerName="nova-api-log" probeResult="failure" output="Get \"http://10.217.1.121:8774/\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Dec 03 11:53:09 crc kubenswrapper[5052]: I1203 11:53:09.722103 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d21ea848-4642-4f06-b777-ae420fd6330e" path="/var/lib/kubelet/pods/d21ea848-4642-4f06-b777-ae420fd6330e/volumes" Dec 03 11:53:09 crc kubenswrapper[5052]: I1203 11:53:09.752263 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-cell-mapping-7v46q" Dec 03 11:53:09 crc kubenswrapper[5052]: I1203 11:53:09.858222 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/6500a947-382b-4269-b31b-5b518c858a6f-scripts\") pod \"6500a947-382b-4269-b31b-5b518c858a6f\" (UID: \"6500a947-382b-4269-b31b-5b518c858a6f\") " Dec 03 11:53:09 crc kubenswrapper[5052]: I1203 11:53:09.858605 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6500a947-382b-4269-b31b-5b518c858a6f-config-data\") pod \"6500a947-382b-4269-b31b-5b518c858a6f\" (UID: \"6500a947-382b-4269-b31b-5b518c858a6f\") " Dec 03 11:53:09 crc kubenswrapper[5052]: I1203 11:53:09.858629 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rx6bk\" (UniqueName: \"kubernetes.io/projected/6500a947-382b-4269-b31b-5b518c858a6f-kube-api-access-rx6bk\") pod \"6500a947-382b-4269-b31b-5b518c858a6f\" (UID: \"6500a947-382b-4269-b31b-5b518c858a6f\") " Dec 03 11:53:09 crc kubenswrapper[5052]: I1203 11:53:09.858649 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6500a947-382b-4269-b31b-5b518c858a6f-combined-ca-bundle\") pod \"6500a947-382b-4269-b31b-5b518c858a6f\" (UID: \"6500a947-382b-4269-b31b-5b518c858a6f\") " Dec 03 11:53:09 crc kubenswrapper[5052]: I1203 11:53:09.863290 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6500a947-382b-4269-b31b-5b518c858a6f-scripts" (OuterVolumeSpecName: "scripts") pod "6500a947-382b-4269-b31b-5b518c858a6f" (UID: "6500a947-382b-4269-b31b-5b518c858a6f"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:53:09 crc kubenswrapper[5052]: I1203 11:53:09.866229 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6500a947-382b-4269-b31b-5b518c858a6f-kube-api-access-rx6bk" (OuterVolumeSpecName: "kube-api-access-rx6bk") pod "6500a947-382b-4269-b31b-5b518c858a6f" (UID: "6500a947-382b-4269-b31b-5b518c858a6f"). InnerVolumeSpecName "kube-api-access-rx6bk". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:53:09 crc kubenswrapper[5052]: I1203 11:53:09.893685 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6500a947-382b-4269-b31b-5b518c858a6f-config-data" (OuterVolumeSpecName: "config-data") pod "6500a947-382b-4269-b31b-5b518c858a6f" (UID: "6500a947-382b-4269-b31b-5b518c858a6f"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:53:09 crc kubenswrapper[5052]: I1203 11:53:09.916011 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6500a947-382b-4269-b31b-5b518c858a6f-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "6500a947-382b-4269-b31b-5b518c858a6f" (UID: "6500a947-382b-4269-b31b-5b518c858a6f"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:53:09 crc kubenswrapper[5052]: I1203 11:53:09.960631 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-rx6bk\" (UniqueName: \"kubernetes.io/projected/6500a947-382b-4269-b31b-5b518c858a6f-kube-api-access-rx6bk\") on node \"crc\" DevicePath \"\"" Dec 03 11:53:09 crc kubenswrapper[5052]: I1203 11:53:09.960674 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6500a947-382b-4269-b31b-5b518c858a6f-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:53:09 crc kubenswrapper[5052]: I1203 11:53:09.960688 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6500a947-382b-4269-b31b-5b518c858a6f-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:53:09 crc kubenswrapper[5052]: I1203 11:53:09.960699 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/6500a947-382b-4269-b31b-5b518c858a6f-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:53:10 crc kubenswrapper[5052]: I1203 11:53:10.072759 5052 generic.go:334] "Generic (PLEG): container finished" podID="511d01be-4413-4580-b284-d49c914d0b8a" containerID="974df8d52afeb94ca7f6376aa32c93e7908382a0a0e609c4d64eb521bfbc0a41" exitCode=0 Dec 03 11:53:10 crc kubenswrapper[5052]: I1203 11:53:10.072852 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-xr6nj" event={"ID":"511d01be-4413-4580-b284-d49c914d0b8a","Type":"ContainerDied","Data":"974df8d52afeb94ca7f6376aa32c93e7908382a0a0e609c4d64eb521bfbc0a41"} Dec 03 11:53:10 crc kubenswrapper[5052]: I1203 11:53:10.074684 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-cell-mapping-7v46q" event={"ID":"6500a947-382b-4269-b31b-5b518c858a6f","Type":"ContainerDied","Data":"2a84acd78c7cc5ac9aee56a708c77de8df6fa014f845f01788f398d582b28969"} Dec 03 11:53:10 crc kubenswrapper[5052]: I1203 11:53:10.074723 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2a84acd78c7cc5ac9aee56a708c77de8df6fa014f845f01788f398d582b28969" Dec 03 11:53:10 crc kubenswrapper[5052]: I1203 11:53:10.074724 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-cell-mapping-7v46q" Dec 03 11:53:10 crc kubenswrapper[5052]: I1203 11:53:10.077982 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/kube-state-metrics-0" event={"ID":"88e7db65-8253-474b-b115-cd3f6edaf85c","Type":"ContainerStarted","Data":"d035b8a698baa861b1fde789196a379e9a2b04d327860b01e3e264af295b680f"} Dec 03 11:53:10 crc kubenswrapper[5052]: I1203 11:53:10.078246 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 11:53:10 crc kubenswrapper[5052]: I1203 11:53:10.080731 5052 generic.go:334] "Generic (PLEG): container finished" podID="3bae2410-56dc-42bc-8974-4da004ac0b28" containerID="f9fcf25c0c0d7f76435313ac33854df45d1892029892dae5e2816386e51e35d6" exitCode=0 Dec 03 11:53:10 crc kubenswrapper[5052]: I1203 11:53:10.080825 5052 generic.go:334] "Generic (PLEG): container finished" podID="3bae2410-56dc-42bc-8974-4da004ac0b28" containerID="dd21b0bd313fbea7fd9fbbc6df20b5977fa0f518bc12404bb5083a4c2ee3b4ce" exitCode=2 Dec 03 11:53:10 crc kubenswrapper[5052]: I1203 11:53:10.080911 5052 generic.go:334] "Generic (PLEG): container finished" podID="3bae2410-56dc-42bc-8974-4da004ac0b28" containerID="3d4e37cd5ebdf25795857b70ca2eda328dcf4ce9f44215affa2902c0ee5c7ec8" exitCode=0 Dec 03 11:53:10 crc kubenswrapper[5052]: I1203 11:53:10.080949 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"3bae2410-56dc-42bc-8974-4da004ac0b28","Type":"ContainerDied","Data":"f9fcf25c0c0d7f76435313ac33854df45d1892029892dae5e2816386e51e35d6"} Dec 03 11:53:10 crc kubenswrapper[5052]: I1203 11:53:10.081237 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"3bae2410-56dc-42bc-8974-4da004ac0b28","Type":"ContainerDied","Data":"dd21b0bd313fbea7fd9fbbc6df20b5977fa0f518bc12404bb5083a4c2ee3b4ce"} Dec 03 11:53:10 crc kubenswrapper[5052]: I1203 11:53:10.081258 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"3bae2410-56dc-42bc-8974-4da004ac0b28","Type":"ContainerDied","Data":"3d4e37cd5ebdf25795857b70ca2eda328dcf4ce9f44215affa2902c0ee5c7ec8"} Dec 03 11:53:10 crc kubenswrapper[5052]: I1203 11:53:10.123442 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/kube-state-metrics-0" podStartSLOduration=1.629844207 podStartE2EDuration="2.123421119s" podCreationTimestamp="2025-12-03 11:53:08 +0000 UTC" firstStartedPulling="2025-12-03 11:53:08.926813808 +0000 UTC m=+1979.515055572" lastFinishedPulling="2025-12-03 11:53:09.42039072 +0000 UTC m=+1980.008632484" observedRunningTime="2025-12-03 11:53:10.104724264 +0000 UTC m=+1980.692966048" watchObservedRunningTime="2025-12-03 11:53:10.123421119 +0000 UTC m=+1980.711662903" Dec 03 11:53:10 crc kubenswrapper[5052]: I1203 11:53:10.245218 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 11:53:10 crc kubenswrapper[5052]: I1203 11:53:10.245512 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-api-0" podUID="a4ee181e-0136-4d6c-aad6-e0ffcc2cb9db" containerName="nova-api-log" containerID="cri-o://70062d4b63028d8ee6f2b890b4bb61d128ba1af7288be9bc6c1fe530ecb428dc" gracePeriod=30 Dec 03 11:53:10 crc kubenswrapper[5052]: I1203 11:53:10.245607 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-api-0" podUID="a4ee181e-0136-4d6c-aad6-e0ffcc2cb9db" containerName="nova-api-api" containerID="cri-o://f681937d4e3600c07f0e7a5f6674a949192d35f7d97ddf1a5a4f6b78e7c5ca57" gracePeriod=30 Dec 03 11:53:10 crc kubenswrapper[5052]: I1203 11:53:10.279469 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 11:53:10 crc kubenswrapper[5052]: I1203 11:53:10.288288 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 11:53:10 crc kubenswrapper[5052]: I1203 11:53:10.288504 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-metadata-0" podUID="8699ba4f-1002-4ba2-9358-17a5600bcff8" containerName="nova-metadata-log" containerID="cri-o://82b44062050a9b637ebc60692bad3aafe0bcf829534275f1a2fec3faecd8b697" gracePeriod=30 Dec 03 11:53:10 crc kubenswrapper[5052]: I1203 11:53:10.288580 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-metadata-0" podUID="8699ba4f-1002-4ba2-9358-17a5600bcff8" containerName="nova-metadata-metadata" containerID="cri-o://03c746c1a2fa9c652460e2d9021afa7c6d8a54756fd60dd1a4125e6904207ea1" gracePeriod=30 Dec 03 11:53:11 crc kubenswrapper[5052]: I1203 11:53:11.091247 5052 generic.go:334] "Generic (PLEG): container finished" podID="8699ba4f-1002-4ba2-9358-17a5600bcff8" containerID="03c746c1a2fa9c652460e2d9021afa7c6d8a54756fd60dd1a4125e6904207ea1" exitCode=0 Dec 03 11:53:11 crc kubenswrapper[5052]: I1203 11:53:11.091567 5052 generic.go:334] "Generic (PLEG): container finished" podID="8699ba4f-1002-4ba2-9358-17a5600bcff8" containerID="82b44062050a9b637ebc60692bad3aafe0bcf829534275f1a2fec3faecd8b697" exitCode=143 Dec 03 11:53:11 crc kubenswrapper[5052]: I1203 11:53:11.091341 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"8699ba4f-1002-4ba2-9358-17a5600bcff8","Type":"ContainerDied","Data":"03c746c1a2fa9c652460e2d9021afa7c6d8a54756fd60dd1a4125e6904207ea1"} Dec 03 11:53:11 crc kubenswrapper[5052]: I1203 11:53:11.091642 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"8699ba4f-1002-4ba2-9358-17a5600bcff8","Type":"ContainerDied","Data":"82b44062050a9b637ebc60692bad3aafe0bcf829534275f1a2fec3faecd8b697"} Dec 03 11:53:11 crc kubenswrapper[5052]: I1203 11:53:11.093337 5052 generic.go:334] "Generic (PLEG): container finished" podID="a4ee181e-0136-4d6c-aad6-e0ffcc2cb9db" containerID="70062d4b63028d8ee6f2b890b4bb61d128ba1af7288be9bc6c1fe530ecb428dc" exitCode=143 Dec 03 11:53:11 crc kubenswrapper[5052]: I1203 11:53:11.093447 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"a4ee181e-0136-4d6c-aad6-e0ffcc2cb9db","Type":"ContainerDied","Data":"70062d4b63028d8ee6f2b890b4bb61d128ba1af7288be9bc6c1fe530ecb428dc"} Dec 03 11:53:11 crc kubenswrapper[5052]: I1203 11:53:11.093868 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-scheduler-0" podUID="c32dead2-3fd2-497d-b4a1-56c24d3cd8dd" containerName="nova-scheduler-scheduler" containerID="cri-o://8bd79e5e170a8c0b2749fe7d7e6882a7f14fa8575090df791ae6d359ca030e14" gracePeriod=30 Dec 03 11:53:11 crc kubenswrapper[5052]: I1203 11:53:11.196872 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:53:11 crc kubenswrapper[5052]: I1203 11:53:11.295569 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/8699ba4f-1002-4ba2-9358-17a5600bcff8-config-data\") pod \"8699ba4f-1002-4ba2-9358-17a5600bcff8\" (UID: \"8699ba4f-1002-4ba2-9358-17a5600bcff8\") " Dec 03 11:53:11 crc kubenswrapper[5052]: I1203 11:53:11.295665 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8699ba4f-1002-4ba2-9358-17a5600bcff8-combined-ca-bundle\") pod \"8699ba4f-1002-4ba2-9358-17a5600bcff8\" (UID: \"8699ba4f-1002-4ba2-9358-17a5600bcff8\") " Dec 03 11:53:11 crc kubenswrapper[5052]: I1203 11:53:11.295761 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sgwwt\" (UniqueName: \"kubernetes.io/projected/8699ba4f-1002-4ba2-9358-17a5600bcff8-kube-api-access-sgwwt\") pod \"8699ba4f-1002-4ba2-9358-17a5600bcff8\" (UID: \"8699ba4f-1002-4ba2-9358-17a5600bcff8\") " Dec 03 11:53:11 crc kubenswrapper[5052]: I1203 11:53:11.295790 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/8699ba4f-1002-4ba2-9358-17a5600bcff8-logs\") pod \"8699ba4f-1002-4ba2-9358-17a5600bcff8\" (UID: \"8699ba4f-1002-4ba2-9358-17a5600bcff8\") " Dec 03 11:53:11 crc kubenswrapper[5052]: I1203 11:53:11.295831 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/8699ba4f-1002-4ba2-9358-17a5600bcff8-nova-metadata-tls-certs\") pod \"8699ba4f-1002-4ba2-9358-17a5600bcff8\" (UID: \"8699ba4f-1002-4ba2-9358-17a5600bcff8\") " Dec 03 11:53:11 crc kubenswrapper[5052]: I1203 11:53:11.296785 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8699ba4f-1002-4ba2-9358-17a5600bcff8-logs" (OuterVolumeSpecName: "logs") pod "8699ba4f-1002-4ba2-9358-17a5600bcff8" (UID: "8699ba4f-1002-4ba2-9358-17a5600bcff8"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:53:11 crc kubenswrapper[5052]: I1203 11:53:11.318339 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8699ba4f-1002-4ba2-9358-17a5600bcff8-kube-api-access-sgwwt" (OuterVolumeSpecName: "kube-api-access-sgwwt") pod "8699ba4f-1002-4ba2-9358-17a5600bcff8" (UID: "8699ba4f-1002-4ba2-9358-17a5600bcff8"). InnerVolumeSpecName "kube-api-access-sgwwt". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:53:11 crc kubenswrapper[5052]: I1203 11:53:11.341574 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8699ba4f-1002-4ba2-9358-17a5600bcff8-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "8699ba4f-1002-4ba2-9358-17a5600bcff8" (UID: "8699ba4f-1002-4ba2-9358-17a5600bcff8"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:53:11 crc kubenswrapper[5052]: I1203 11:53:11.355800 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8699ba4f-1002-4ba2-9358-17a5600bcff8-config-data" (OuterVolumeSpecName: "config-data") pod "8699ba4f-1002-4ba2-9358-17a5600bcff8" (UID: "8699ba4f-1002-4ba2-9358-17a5600bcff8"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:53:11 crc kubenswrapper[5052]: I1203 11:53:11.355907 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-xr6nj" Dec 03 11:53:11 crc kubenswrapper[5052]: I1203 11:53:11.361493 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8699ba4f-1002-4ba2-9358-17a5600bcff8-nova-metadata-tls-certs" (OuterVolumeSpecName: "nova-metadata-tls-certs") pod "8699ba4f-1002-4ba2-9358-17a5600bcff8" (UID: "8699ba4f-1002-4ba2-9358-17a5600bcff8"). InnerVolumeSpecName "nova-metadata-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:53:11 crc kubenswrapper[5052]: I1203 11:53:11.398330 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-sgwwt\" (UniqueName: \"kubernetes.io/projected/8699ba4f-1002-4ba2-9358-17a5600bcff8-kube-api-access-sgwwt\") on node \"crc\" DevicePath \"\"" Dec 03 11:53:11 crc kubenswrapper[5052]: I1203 11:53:11.398363 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/8699ba4f-1002-4ba2-9358-17a5600bcff8-logs\") on node \"crc\" DevicePath \"\"" Dec 03 11:53:11 crc kubenswrapper[5052]: I1203 11:53:11.398373 5052 reconciler_common.go:293] "Volume detached for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/8699ba4f-1002-4ba2-9358-17a5600bcff8-nova-metadata-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 11:53:11 crc kubenswrapper[5052]: I1203 11:53:11.398383 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/8699ba4f-1002-4ba2-9358-17a5600bcff8-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:53:11 crc kubenswrapper[5052]: I1203 11:53:11.398391 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8699ba4f-1002-4ba2-9358-17a5600bcff8-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:53:11 crc kubenswrapper[5052]: I1203 11:53:11.499484 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/511d01be-4413-4580-b284-d49c914d0b8a-scripts\") pod \"511d01be-4413-4580-b284-d49c914d0b8a\" (UID: \"511d01be-4413-4580-b284-d49c914d0b8a\") " Dec 03 11:53:11 crc kubenswrapper[5052]: I1203 11:53:11.499604 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rk5gf\" (UniqueName: \"kubernetes.io/projected/511d01be-4413-4580-b284-d49c914d0b8a-kube-api-access-rk5gf\") pod \"511d01be-4413-4580-b284-d49c914d0b8a\" (UID: \"511d01be-4413-4580-b284-d49c914d0b8a\") " Dec 03 11:53:11 crc kubenswrapper[5052]: I1203 11:53:11.499674 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/511d01be-4413-4580-b284-d49c914d0b8a-combined-ca-bundle\") pod \"511d01be-4413-4580-b284-d49c914d0b8a\" (UID: \"511d01be-4413-4580-b284-d49c914d0b8a\") " Dec 03 11:53:11 crc kubenswrapper[5052]: I1203 11:53:11.499725 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/511d01be-4413-4580-b284-d49c914d0b8a-config-data\") pod \"511d01be-4413-4580-b284-d49c914d0b8a\" (UID: \"511d01be-4413-4580-b284-d49c914d0b8a\") " Dec 03 11:53:11 crc kubenswrapper[5052]: I1203 11:53:11.507280 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/511d01be-4413-4580-b284-d49c914d0b8a-kube-api-access-rk5gf" (OuterVolumeSpecName: "kube-api-access-rk5gf") pod "511d01be-4413-4580-b284-d49c914d0b8a" (UID: "511d01be-4413-4580-b284-d49c914d0b8a"). InnerVolumeSpecName "kube-api-access-rk5gf". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:53:11 crc kubenswrapper[5052]: I1203 11:53:11.517320 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/511d01be-4413-4580-b284-d49c914d0b8a-scripts" (OuterVolumeSpecName: "scripts") pod "511d01be-4413-4580-b284-d49c914d0b8a" (UID: "511d01be-4413-4580-b284-d49c914d0b8a"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:53:11 crc kubenswrapper[5052]: I1203 11:53:11.526173 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/511d01be-4413-4580-b284-d49c914d0b8a-config-data" (OuterVolumeSpecName: "config-data") pod "511d01be-4413-4580-b284-d49c914d0b8a" (UID: "511d01be-4413-4580-b284-d49c914d0b8a"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:53:11 crc kubenswrapper[5052]: I1203 11:53:11.526817 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/511d01be-4413-4580-b284-d49c914d0b8a-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "511d01be-4413-4580-b284-d49c914d0b8a" (UID: "511d01be-4413-4580-b284-d49c914d0b8a"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:53:11 crc kubenswrapper[5052]: I1203 11:53:11.602108 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/511d01be-4413-4580-b284-d49c914d0b8a-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:53:11 crc kubenswrapper[5052]: I1203 11:53:11.602148 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-rk5gf\" (UniqueName: \"kubernetes.io/projected/511d01be-4413-4580-b284-d49c914d0b8a-kube-api-access-rk5gf\") on node \"crc\" DevicePath \"\"" Dec 03 11:53:11 crc kubenswrapper[5052]: I1203 11:53:11.602163 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/511d01be-4413-4580-b284-d49c914d0b8a-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:53:11 crc kubenswrapper[5052]: I1203 11:53:11.602175 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/511d01be-4413-4580-b284-d49c914d0b8a-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:53:12 crc kubenswrapper[5052]: I1203 11:53:12.120984 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-xr6nj" event={"ID":"511d01be-4413-4580-b284-d49c914d0b8a","Type":"ContainerDied","Data":"9471a7fcd0315599cb5117399f326d8120bfccd525744904148a255798d95beb"} Dec 03 11:53:12 crc kubenswrapper[5052]: I1203 11:53:12.121027 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9471a7fcd0315599cb5117399f326d8120bfccd525744904148a255798d95beb" Dec 03 11:53:12 crc kubenswrapper[5052]: I1203 11:53:12.121089 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-xr6nj" Dec 03 11:53:12 crc kubenswrapper[5052]: I1203 11:53:12.134000 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"8699ba4f-1002-4ba2-9358-17a5600bcff8","Type":"ContainerDied","Data":"cb4638009981ca86353947fed0da344a6eef3f474f703ad176eb004b88a85f26"} Dec 03 11:53:12 crc kubenswrapper[5052]: I1203 11:53:12.134061 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:53:12 crc kubenswrapper[5052]: I1203 11:53:12.134078 5052 scope.go:117] "RemoveContainer" containerID="03c746c1a2fa9c652460e2d9021afa7c6d8a54756fd60dd1a4125e6904207ea1" Dec 03 11:53:12 crc kubenswrapper[5052]: I1203 11:53:12.176680 5052 scope.go:117] "RemoveContainer" containerID="82b44062050a9b637ebc60692bad3aafe0bcf829534275f1a2fec3faecd8b697" Dec 03 11:53:12 crc kubenswrapper[5052]: I1203 11:53:12.181487 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 11:53:12 crc kubenswrapper[5052]: I1203 11:53:12.199579 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 11:53:12 crc kubenswrapper[5052]: I1203 11:53:12.222665 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-0"] Dec 03 11:53:12 crc kubenswrapper[5052]: E1203 11:53:12.223097 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="511d01be-4413-4580-b284-d49c914d0b8a" containerName="nova-cell1-conductor-db-sync" Dec 03 11:53:12 crc kubenswrapper[5052]: I1203 11:53:12.223115 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="511d01be-4413-4580-b284-d49c914d0b8a" containerName="nova-cell1-conductor-db-sync" Dec 03 11:53:12 crc kubenswrapper[5052]: E1203 11:53:12.223134 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="6500a947-382b-4269-b31b-5b518c858a6f" containerName="nova-manage" Dec 03 11:53:12 crc kubenswrapper[5052]: I1203 11:53:12.223141 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="6500a947-382b-4269-b31b-5b518c858a6f" containerName="nova-manage" Dec 03 11:53:12 crc kubenswrapper[5052]: E1203 11:53:12.223171 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="8699ba4f-1002-4ba2-9358-17a5600bcff8" containerName="nova-metadata-log" Dec 03 11:53:12 crc kubenswrapper[5052]: I1203 11:53:12.223177 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="8699ba4f-1002-4ba2-9358-17a5600bcff8" containerName="nova-metadata-log" Dec 03 11:53:12 crc kubenswrapper[5052]: E1203 11:53:12.223190 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="8699ba4f-1002-4ba2-9358-17a5600bcff8" containerName="nova-metadata-metadata" Dec 03 11:53:12 crc kubenswrapper[5052]: I1203 11:53:12.223196 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="8699ba4f-1002-4ba2-9358-17a5600bcff8" containerName="nova-metadata-metadata" Dec 03 11:53:12 crc kubenswrapper[5052]: I1203 11:53:12.223359 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="6500a947-382b-4269-b31b-5b518c858a6f" containerName="nova-manage" Dec 03 11:53:12 crc kubenswrapper[5052]: I1203 11:53:12.223377 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="8699ba4f-1002-4ba2-9358-17a5600bcff8" containerName="nova-metadata-metadata" Dec 03 11:53:12 crc kubenswrapper[5052]: I1203 11:53:12.223388 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="511d01be-4413-4580-b284-d49c914d0b8a" containerName="nova-cell1-conductor-db-sync" Dec 03 11:53:12 crc kubenswrapper[5052]: I1203 11:53:12.223399 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="8699ba4f-1002-4ba2-9358-17a5600bcff8" containerName="nova-metadata-log" Dec 03 11:53:12 crc kubenswrapper[5052]: I1203 11:53:12.224000 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 11:53:12 crc kubenswrapper[5052]: I1203 11:53:12.228146 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell1-conductor-config-data" Dec 03 11:53:12 crc kubenswrapper[5052]: I1203 11:53:12.233267 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 11:53:12 crc kubenswrapper[5052]: I1203 11:53:12.234833 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:53:12 crc kubenswrapper[5052]: I1203 11:53:12.237443 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-nova-metadata-internal-svc" Dec 03 11:53:12 crc kubenswrapper[5052]: I1203 11:53:12.237581 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-metadata-config-data" Dec 03 11:53:12 crc kubenswrapper[5052]: I1203 11:53:12.247129 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-0"] Dec 03 11:53:12 crc kubenswrapper[5052]: I1203 11:53:12.266924 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 11:53:12 crc kubenswrapper[5052]: I1203 11:53:12.315937 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5zm2j\" (UniqueName: \"kubernetes.io/projected/8a2597b5-e6ec-40e9-a992-ea2ee71ae759-kube-api-access-5zm2j\") pod \"nova-metadata-0\" (UID: \"8a2597b5-e6ec-40e9-a992-ea2ee71ae759\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:53:12 crc kubenswrapper[5052]: I1203 11:53:12.316053 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5mmcz\" (UniqueName: \"kubernetes.io/projected/4a181d2c-7595-4394-98de-957451309c98-kube-api-access-5mmcz\") pod \"nova-cell1-conductor-0\" (UID: \"4a181d2c-7595-4394-98de-957451309c98\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 11:53:12 crc kubenswrapper[5052]: I1203 11:53:12.316107 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/8a2597b5-e6ec-40e9-a992-ea2ee71ae759-nova-metadata-tls-certs\") pod \"nova-metadata-0\" (UID: \"8a2597b5-e6ec-40e9-a992-ea2ee71ae759\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:53:12 crc kubenswrapper[5052]: I1203 11:53:12.316206 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/8a2597b5-e6ec-40e9-a992-ea2ee71ae759-config-data\") pod \"nova-metadata-0\" (UID: \"8a2597b5-e6ec-40e9-a992-ea2ee71ae759\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:53:12 crc kubenswrapper[5052]: I1203 11:53:12.316277 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/8a2597b5-e6ec-40e9-a992-ea2ee71ae759-logs\") pod \"nova-metadata-0\" (UID: \"8a2597b5-e6ec-40e9-a992-ea2ee71ae759\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:53:12 crc kubenswrapper[5052]: I1203 11:53:12.316553 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/4a181d2c-7595-4394-98de-957451309c98-config-data\") pod \"nova-cell1-conductor-0\" (UID: \"4a181d2c-7595-4394-98de-957451309c98\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 11:53:12 crc kubenswrapper[5052]: I1203 11:53:12.316608 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4a181d2c-7595-4394-98de-957451309c98-combined-ca-bundle\") pod \"nova-cell1-conductor-0\" (UID: \"4a181d2c-7595-4394-98de-957451309c98\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 11:53:12 crc kubenswrapper[5052]: I1203 11:53:12.316685 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8a2597b5-e6ec-40e9-a992-ea2ee71ae759-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"8a2597b5-e6ec-40e9-a992-ea2ee71ae759\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:53:12 crc kubenswrapper[5052]: I1203 11:53:12.418385 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-5mmcz\" (UniqueName: \"kubernetes.io/projected/4a181d2c-7595-4394-98de-957451309c98-kube-api-access-5mmcz\") pod \"nova-cell1-conductor-0\" (UID: \"4a181d2c-7595-4394-98de-957451309c98\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 11:53:12 crc kubenswrapper[5052]: I1203 11:53:12.418763 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-5zm2j\" (UniqueName: \"kubernetes.io/projected/8a2597b5-e6ec-40e9-a992-ea2ee71ae759-kube-api-access-5zm2j\") pod \"nova-metadata-0\" (UID: \"8a2597b5-e6ec-40e9-a992-ea2ee71ae759\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:53:12 crc kubenswrapper[5052]: I1203 11:53:12.418808 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/8a2597b5-e6ec-40e9-a992-ea2ee71ae759-nova-metadata-tls-certs\") pod \"nova-metadata-0\" (UID: \"8a2597b5-e6ec-40e9-a992-ea2ee71ae759\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:53:12 crc kubenswrapper[5052]: I1203 11:53:12.418841 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/8a2597b5-e6ec-40e9-a992-ea2ee71ae759-config-data\") pod \"nova-metadata-0\" (UID: \"8a2597b5-e6ec-40e9-a992-ea2ee71ae759\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:53:12 crc kubenswrapper[5052]: I1203 11:53:12.418901 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/8a2597b5-e6ec-40e9-a992-ea2ee71ae759-logs\") pod \"nova-metadata-0\" (UID: \"8a2597b5-e6ec-40e9-a992-ea2ee71ae759\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:53:12 crc kubenswrapper[5052]: I1203 11:53:12.419402 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/8a2597b5-e6ec-40e9-a992-ea2ee71ae759-logs\") pod \"nova-metadata-0\" (UID: \"8a2597b5-e6ec-40e9-a992-ea2ee71ae759\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:53:12 crc kubenswrapper[5052]: I1203 11:53:12.419551 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/4a181d2c-7595-4394-98de-957451309c98-config-data\") pod \"nova-cell1-conductor-0\" (UID: \"4a181d2c-7595-4394-98de-957451309c98\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 11:53:12 crc kubenswrapper[5052]: I1203 11:53:12.421172 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4a181d2c-7595-4394-98de-957451309c98-combined-ca-bundle\") pod \"nova-cell1-conductor-0\" (UID: \"4a181d2c-7595-4394-98de-957451309c98\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 11:53:12 crc kubenswrapper[5052]: I1203 11:53:12.421433 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8a2597b5-e6ec-40e9-a992-ea2ee71ae759-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"8a2597b5-e6ec-40e9-a992-ea2ee71ae759\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:53:12 crc kubenswrapper[5052]: I1203 11:53:12.423999 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/8a2597b5-e6ec-40e9-a992-ea2ee71ae759-nova-metadata-tls-certs\") pod \"nova-metadata-0\" (UID: \"8a2597b5-e6ec-40e9-a992-ea2ee71ae759\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:53:12 crc kubenswrapper[5052]: I1203 11:53:12.424004 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4a181d2c-7595-4394-98de-957451309c98-combined-ca-bundle\") pod \"nova-cell1-conductor-0\" (UID: \"4a181d2c-7595-4394-98de-957451309c98\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 11:53:12 crc kubenswrapper[5052]: I1203 11:53:12.424177 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/8a2597b5-e6ec-40e9-a992-ea2ee71ae759-config-data\") pod \"nova-metadata-0\" (UID: \"8a2597b5-e6ec-40e9-a992-ea2ee71ae759\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:53:12 crc kubenswrapper[5052]: I1203 11:53:12.427604 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/4a181d2c-7595-4394-98de-957451309c98-config-data\") pod \"nova-cell1-conductor-0\" (UID: \"4a181d2c-7595-4394-98de-957451309c98\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 11:53:12 crc kubenswrapper[5052]: I1203 11:53:12.430416 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8a2597b5-e6ec-40e9-a992-ea2ee71ae759-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"8a2597b5-e6ec-40e9-a992-ea2ee71ae759\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:53:12 crc kubenswrapper[5052]: I1203 11:53:12.442946 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-5zm2j\" (UniqueName: \"kubernetes.io/projected/8a2597b5-e6ec-40e9-a992-ea2ee71ae759-kube-api-access-5zm2j\") pod \"nova-metadata-0\" (UID: \"8a2597b5-e6ec-40e9-a992-ea2ee71ae759\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:53:12 crc kubenswrapper[5052]: I1203 11:53:12.450877 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-5mmcz\" (UniqueName: \"kubernetes.io/projected/4a181d2c-7595-4394-98de-957451309c98-kube-api-access-5mmcz\") pod \"nova-cell1-conductor-0\" (UID: \"4a181d2c-7595-4394-98de-957451309c98\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 11:53:12 crc kubenswrapper[5052]: I1203 11:53:12.550754 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 11:53:12 crc kubenswrapper[5052]: I1203 11:53:12.563039 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:53:13 crc kubenswrapper[5052]: I1203 11:53:13.045348 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-0"] Dec 03 11:53:13 crc kubenswrapper[5052]: W1203 11:53:13.048774 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4a181d2c_7595_4394_98de_957451309c98.slice/crio-ee653fb33989f4aca07fcbe1bf8a3dbf2e60a62c48b8ca19f6784644b6e2efd2 WatchSource:0}: Error finding container ee653fb33989f4aca07fcbe1bf8a3dbf2e60a62c48b8ca19f6784644b6e2efd2: Status 404 returned error can't find the container with id ee653fb33989f4aca07fcbe1bf8a3dbf2e60a62c48b8ca19f6784644b6e2efd2 Dec 03 11:53:13 crc kubenswrapper[5052]: W1203 11:53:13.114877 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8a2597b5_e6ec_40e9_a992_ea2ee71ae759.slice/crio-0d9db8df6b44b1c24186afdd2064dc1ea8b0415d4cf50ef5fca6429bf616c7fb WatchSource:0}: Error finding container 0d9db8df6b44b1c24186afdd2064dc1ea8b0415d4cf50ef5fca6429bf616c7fb: Status 404 returned error can't find the container with id 0d9db8df6b44b1c24186afdd2064dc1ea8b0415d4cf50ef5fca6429bf616c7fb Dec 03 11:53:13 crc kubenswrapper[5052]: I1203 11:53:13.115841 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 11:53:13 crc kubenswrapper[5052]: I1203 11:53:13.150911 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"8a2597b5-e6ec-40e9-a992-ea2ee71ae759","Type":"ContainerStarted","Data":"0d9db8df6b44b1c24186afdd2064dc1ea8b0415d4cf50ef5fca6429bf616c7fb"} Dec 03 11:53:13 crc kubenswrapper[5052]: I1203 11:53:13.153478 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-conductor-0" event={"ID":"4a181d2c-7595-4394-98de-957451309c98","Type":"ContainerStarted","Data":"ee653fb33989f4aca07fcbe1bf8a3dbf2e60a62c48b8ca19f6784644b6e2efd2"} Dec 03 11:53:13 crc kubenswrapper[5052]: I1203 11:53:13.373002 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 11:53:13 crc kubenswrapper[5052]: I1203 11:53:13.397347 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 11:53:13 crc kubenswrapper[5052]: E1203 11:53:13.561909 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="8bd79e5e170a8c0b2749fe7d7e6882a7f14fa8575090df791ae6d359ca030e14" cmd=["/usr/bin/pgrep","-r","DRST","nova-scheduler"] Dec 03 11:53:13 crc kubenswrapper[5052]: E1203 11:53:13.563249 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="8bd79e5e170a8c0b2749fe7d7e6882a7f14fa8575090df791ae6d359ca030e14" cmd=["/usr/bin/pgrep","-r","DRST","nova-scheduler"] Dec 03 11:53:13 crc kubenswrapper[5052]: E1203 11:53:13.565758 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="8bd79e5e170a8c0b2749fe7d7e6882a7f14fa8575090df791ae6d359ca030e14" cmd=["/usr/bin/pgrep","-r","DRST","nova-scheduler"] Dec 03 11:53:13 crc kubenswrapper[5052]: E1203 11:53:13.565802 5052 prober.go:104] "Probe errored" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" probeType="Readiness" pod="openstack-kuttl-tests/nova-scheduler-0" podUID="c32dead2-3fd2-497d-b4a1-56c24d3cd8dd" containerName="nova-scheduler-scheduler" Dec 03 11:53:13 crc kubenswrapper[5052]: I1203 11:53:13.684292 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8699ba4f-1002-4ba2-9358-17a5600bcff8" path="/var/lib/kubelet/pods/8699ba4f-1002-4ba2-9358-17a5600bcff8/volumes" Dec 03 11:53:13 crc kubenswrapper[5052]: I1203 11:53:13.690748 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openshift-marketplace/redhat-operators-cjtc5"] Dec 03 11:53:13 crc kubenswrapper[5052]: I1203 11:53:13.692788 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/redhat-operators-cjtc5" Dec 03 11:53:13 crc kubenswrapper[5052]: I1203 11:53:13.709461 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/redhat-operators-cjtc5"] Dec 03 11:53:13 crc kubenswrapper[5052]: I1203 11:53:13.847665 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gntth\" (UniqueName: \"kubernetes.io/projected/5d4ac632-b94b-4371-a605-458e4cd92f23-kube-api-access-gntth\") pod \"redhat-operators-cjtc5\" (UID: \"5d4ac632-b94b-4371-a605-458e4cd92f23\") " pod="openshift-marketplace/redhat-operators-cjtc5" Dec 03 11:53:13 crc kubenswrapper[5052]: I1203 11:53:13.847888 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/5d4ac632-b94b-4371-a605-458e4cd92f23-utilities\") pod \"redhat-operators-cjtc5\" (UID: \"5d4ac632-b94b-4371-a605-458e4cd92f23\") " pod="openshift-marketplace/redhat-operators-cjtc5" Dec 03 11:53:13 crc kubenswrapper[5052]: I1203 11:53:13.847921 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/5d4ac632-b94b-4371-a605-458e4cd92f23-catalog-content\") pod \"redhat-operators-cjtc5\" (UID: \"5d4ac632-b94b-4371-a605-458e4cd92f23\") " pod="openshift-marketplace/redhat-operators-cjtc5" Dec 03 11:53:13 crc kubenswrapper[5052]: I1203 11:53:13.949136 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/5d4ac632-b94b-4371-a605-458e4cd92f23-utilities\") pod \"redhat-operators-cjtc5\" (UID: \"5d4ac632-b94b-4371-a605-458e4cd92f23\") " pod="openshift-marketplace/redhat-operators-cjtc5" Dec 03 11:53:13 crc kubenswrapper[5052]: I1203 11:53:13.949217 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/5d4ac632-b94b-4371-a605-458e4cd92f23-catalog-content\") pod \"redhat-operators-cjtc5\" (UID: \"5d4ac632-b94b-4371-a605-458e4cd92f23\") " pod="openshift-marketplace/redhat-operators-cjtc5" Dec 03 11:53:13 crc kubenswrapper[5052]: I1203 11:53:13.949308 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-gntth\" (UniqueName: \"kubernetes.io/projected/5d4ac632-b94b-4371-a605-458e4cd92f23-kube-api-access-gntth\") pod \"redhat-operators-cjtc5\" (UID: \"5d4ac632-b94b-4371-a605-458e4cd92f23\") " pod="openshift-marketplace/redhat-operators-cjtc5" Dec 03 11:53:13 crc kubenswrapper[5052]: I1203 11:53:13.949723 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/5d4ac632-b94b-4371-a605-458e4cd92f23-catalog-content\") pod \"redhat-operators-cjtc5\" (UID: \"5d4ac632-b94b-4371-a605-458e4cd92f23\") " pod="openshift-marketplace/redhat-operators-cjtc5" Dec 03 11:53:13 crc kubenswrapper[5052]: I1203 11:53:13.950015 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/5d4ac632-b94b-4371-a605-458e4cd92f23-utilities\") pod \"redhat-operators-cjtc5\" (UID: \"5d4ac632-b94b-4371-a605-458e4cd92f23\") " pod="openshift-marketplace/redhat-operators-cjtc5" Dec 03 11:53:13 crc kubenswrapper[5052]: I1203 11:53:13.978736 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-gntth\" (UniqueName: \"kubernetes.io/projected/5d4ac632-b94b-4371-a605-458e4cd92f23-kube-api-access-gntth\") pod \"redhat-operators-cjtc5\" (UID: \"5d4ac632-b94b-4371-a605-458e4cd92f23\") " pod="openshift-marketplace/redhat-operators-cjtc5" Dec 03 11:53:14 crc kubenswrapper[5052]: I1203 11:53:14.030134 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/redhat-operators-cjtc5" Dec 03 11:53:14 crc kubenswrapper[5052]: I1203 11:53:14.198099 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-conductor-0" event={"ID":"4a181d2c-7595-4394-98de-957451309c98","Type":"ContainerStarted","Data":"c3cb1469712553e8283f9907b8e296f175dfb01cae7b28331f721bbf4e80e204"} Dec 03 11:53:14 crc kubenswrapper[5052]: I1203 11:53:14.198367 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 11:53:14 crc kubenswrapper[5052]: I1203 11:53:14.206832 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"8a2597b5-e6ec-40e9-a992-ea2ee71ae759","Type":"ContainerStarted","Data":"8f647efbd4f4f64fedeccae7318c4ecd9cfbd4fce9fa538a4df1495f8242c11e"} Dec 03 11:53:14 crc kubenswrapper[5052]: I1203 11:53:14.206883 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"8a2597b5-e6ec-40e9-a992-ea2ee71ae759","Type":"ContainerStarted","Data":"e993f2c99fe8426fc231da418041c0bcb56e7d6a81c4f52c27e7ad9249e1bea6"} Dec 03 11:53:14 crc kubenswrapper[5052]: I1203 11:53:14.230201 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 11:53:14 crc kubenswrapper[5052]: I1203 11:53:14.232898 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-cell1-conductor-0" podStartSLOduration=2.23287646 podStartE2EDuration="2.23287646s" podCreationTimestamp="2025-12-03 11:53:12 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:53:14.21891514 +0000 UTC m=+1984.807156914" watchObservedRunningTime="2025-12-03 11:53:14.23287646 +0000 UTC m=+1984.821118254" Dec 03 11:53:14 crc kubenswrapper[5052]: I1203 11:53:14.253566 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-metadata-0" podStartSLOduration=2.253544442 podStartE2EDuration="2.253544442s" podCreationTimestamp="2025-12-03 11:53:12 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:53:14.240931851 +0000 UTC m=+1984.829173615" watchObservedRunningTime="2025-12-03 11:53:14.253544442 +0000 UTC m=+1984.841786206" Dec 03 11:53:14 crc kubenswrapper[5052]: I1203 11:53:14.512470 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/redhat-operators-cjtc5"] Dec 03 11:53:14 crc kubenswrapper[5052]: W1203 11:53:14.516944 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d4ac632_b94b_4371_a605_458e4cd92f23.slice/crio-16e72209bded8576b920281b541b9c2b1dbc91083750dce94a6f5b331d751672 WatchSource:0}: Error finding container 16e72209bded8576b920281b541b9c2b1dbc91083750dce94a6f5b331d751672: Status 404 returned error can't find the container with id 16e72209bded8576b920281b541b9c2b1dbc91083750dce94a6f5b331d751672 Dec 03 11:53:14 crc kubenswrapper[5052]: I1203 11:53:14.867328 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:53:14 crc kubenswrapper[5052]: I1203 11:53:14.970708 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3bae2410-56dc-42bc-8974-4da004ac0b28-combined-ca-bundle\") pod \"3bae2410-56dc-42bc-8974-4da004ac0b28\" (UID: \"3bae2410-56dc-42bc-8974-4da004ac0b28\") " Dec 03 11:53:14 crc kubenswrapper[5052]: I1203 11:53:14.970786 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/3bae2410-56dc-42bc-8974-4da004ac0b28-config-data\") pod \"3bae2410-56dc-42bc-8974-4da004ac0b28\" (UID: \"3bae2410-56dc-42bc-8974-4da004ac0b28\") " Dec 03 11:53:14 crc kubenswrapper[5052]: I1203 11:53:14.970943 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/3bae2410-56dc-42bc-8974-4da004ac0b28-sg-core-conf-yaml\") pod \"3bae2410-56dc-42bc-8974-4da004ac0b28\" (UID: \"3bae2410-56dc-42bc-8974-4da004ac0b28\") " Dec 03 11:53:14 crc kubenswrapper[5052]: I1203 11:53:14.970993 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/3bae2410-56dc-42bc-8974-4da004ac0b28-scripts\") pod \"3bae2410-56dc-42bc-8974-4da004ac0b28\" (UID: \"3bae2410-56dc-42bc-8974-4da004ac0b28\") " Dec 03 11:53:14 crc kubenswrapper[5052]: I1203 11:53:14.971042 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/3bae2410-56dc-42bc-8974-4da004ac0b28-log-httpd\") pod \"3bae2410-56dc-42bc-8974-4da004ac0b28\" (UID: \"3bae2410-56dc-42bc-8974-4da004ac0b28\") " Dec 03 11:53:14 crc kubenswrapper[5052]: I1203 11:53:14.971087 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/3bae2410-56dc-42bc-8974-4da004ac0b28-run-httpd\") pod \"3bae2410-56dc-42bc-8974-4da004ac0b28\" (UID: \"3bae2410-56dc-42bc-8974-4da004ac0b28\") " Dec 03 11:53:14 crc kubenswrapper[5052]: I1203 11:53:14.971120 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sjtbn\" (UniqueName: \"kubernetes.io/projected/3bae2410-56dc-42bc-8974-4da004ac0b28-kube-api-access-sjtbn\") pod \"3bae2410-56dc-42bc-8974-4da004ac0b28\" (UID: \"3bae2410-56dc-42bc-8974-4da004ac0b28\") " Dec 03 11:53:14 crc kubenswrapper[5052]: I1203 11:53:14.971540 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3bae2410-56dc-42bc-8974-4da004ac0b28-run-httpd" (OuterVolumeSpecName: "run-httpd") pod "3bae2410-56dc-42bc-8974-4da004ac0b28" (UID: "3bae2410-56dc-42bc-8974-4da004ac0b28"). InnerVolumeSpecName "run-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:53:14 crc kubenswrapper[5052]: I1203 11:53:14.971804 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3bae2410-56dc-42bc-8974-4da004ac0b28-log-httpd" (OuterVolumeSpecName: "log-httpd") pod "3bae2410-56dc-42bc-8974-4da004ac0b28" (UID: "3bae2410-56dc-42bc-8974-4da004ac0b28"). InnerVolumeSpecName "log-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:53:14 crc kubenswrapper[5052]: I1203 11:53:14.977948 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3bae2410-56dc-42bc-8974-4da004ac0b28-kube-api-access-sjtbn" (OuterVolumeSpecName: "kube-api-access-sjtbn") pod "3bae2410-56dc-42bc-8974-4da004ac0b28" (UID: "3bae2410-56dc-42bc-8974-4da004ac0b28"). InnerVolumeSpecName "kube-api-access-sjtbn". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:53:14 crc kubenswrapper[5052]: I1203 11:53:14.987550 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3bae2410-56dc-42bc-8974-4da004ac0b28-scripts" (OuterVolumeSpecName: "scripts") pod "3bae2410-56dc-42bc-8974-4da004ac0b28" (UID: "3bae2410-56dc-42bc-8974-4da004ac0b28"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:53:15 crc kubenswrapper[5052]: I1203 11:53:15.005935 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3bae2410-56dc-42bc-8974-4da004ac0b28-sg-core-conf-yaml" (OuterVolumeSpecName: "sg-core-conf-yaml") pod "3bae2410-56dc-42bc-8974-4da004ac0b28" (UID: "3bae2410-56dc-42bc-8974-4da004ac0b28"). InnerVolumeSpecName "sg-core-conf-yaml". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:53:15 crc kubenswrapper[5052]: I1203 11:53:15.069875 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3bae2410-56dc-42bc-8974-4da004ac0b28-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "3bae2410-56dc-42bc-8974-4da004ac0b28" (UID: "3bae2410-56dc-42bc-8974-4da004ac0b28"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:53:15 crc kubenswrapper[5052]: I1203 11:53:15.073422 5052 reconciler_common.go:293] "Volume detached for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/3bae2410-56dc-42bc-8974-4da004ac0b28-log-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 11:53:15 crc kubenswrapper[5052]: I1203 11:53:15.073456 5052 reconciler_common.go:293] "Volume detached for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/3bae2410-56dc-42bc-8974-4da004ac0b28-run-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 11:53:15 crc kubenswrapper[5052]: I1203 11:53:15.073467 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-sjtbn\" (UniqueName: \"kubernetes.io/projected/3bae2410-56dc-42bc-8974-4da004ac0b28-kube-api-access-sjtbn\") on node \"crc\" DevicePath \"\"" Dec 03 11:53:15 crc kubenswrapper[5052]: I1203 11:53:15.073491 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3bae2410-56dc-42bc-8974-4da004ac0b28-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:53:15 crc kubenswrapper[5052]: I1203 11:53:15.073500 5052 reconciler_common.go:293] "Volume detached for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/3bae2410-56dc-42bc-8974-4da004ac0b28-sg-core-conf-yaml\") on node \"crc\" DevicePath \"\"" Dec 03 11:53:15 crc kubenswrapper[5052]: I1203 11:53:15.073536 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/3bae2410-56dc-42bc-8974-4da004ac0b28-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:53:15 crc kubenswrapper[5052]: I1203 11:53:15.110171 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3bae2410-56dc-42bc-8974-4da004ac0b28-config-data" (OuterVolumeSpecName: "config-data") pod "3bae2410-56dc-42bc-8974-4da004ac0b28" (UID: "3bae2410-56dc-42bc-8974-4da004ac0b28"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:53:15 crc kubenswrapper[5052]: I1203 11:53:15.174969 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/3bae2410-56dc-42bc-8974-4da004ac0b28-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:53:15 crc kubenswrapper[5052]: I1203 11:53:15.219463 5052 generic.go:334] "Generic (PLEG): container finished" podID="3bae2410-56dc-42bc-8974-4da004ac0b28" containerID="8edea1b7e7603813f9cee40e80060c8fe7cdde3a3fe81da594f714c8848d10a6" exitCode=0 Dec 03 11:53:15 crc kubenswrapper[5052]: I1203 11:53:15.219540 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"3bae2410-56dc-42bc-8974-4da004ac0b28","Type":"ContainerDied","Data":"8edea1b7e7603813f9cee40e80060c8fe7cdde3a3fe81da594f714c8848d10a6"} Dec 03 11:53:15 crc kubenswrapper[5052]: I1203 11:53:15.219568 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"3bae2410-56dc-42bc-8974-4da004ac0b28","Type":"ContainerDied","Data":"928198eda633511f4c6b051750375a81be21cad5956be1efa9e984cf4c061d28"} Dec 03 11:53:15 crc kubenswrapper[5052]: I1203 11:53:15.219584 5052 scope.go:117] "RemoveContainer" containerID="f9fcf25c0c0d7f76435313ac33854df45d1892029892dae5e2816386e51e35d6" Dec 03 11:53:15 crc kubenswrapper[5052]: I1203 11:53:15.219739 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:53:15 crc kubenswrapper[5052]: I1203 11:53:15.229789 5052 generic.go:334] "Generic (PLEG): container finished" podID="5d4ac632-b94b-4371-a605-458e4cd92f23" containerID="9a0b11c711251ed32c98116acb7e26a2fe5a715ace90e8fc1c90d87db3a20482" exitCode=0 Dec 03 11:53:15 crc kubenswrapper[5052]: I1203 11:53:15.229892 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-operators-cjtc5" event={"ID":"5d4ac632-b94b-4371-a605-458e4cd92f23","Type":"ContainerDied","Data":"9a0b11c711251ed32c98116acb7e26a2fe5a715ace90e8fc1c90d87db3a20482"} Dec 03 11:53:15 crc kubenswrapper[5052]: I1203 11:53:15.229929 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-operators-cjtc5" event={"ID":"5d4ac632-b94b-4371-a605-458e4cd92f23","Type":"ContainerStarted","Data":"16e72209bded8576b920281b541b9c2b1dbc91083750dce94a6f5b331d751672"} Dec 03 11:53:15 crc kubenswrapper[5052]: I1203 11:53:15.250763 5052 scope.go:117] "RemoveContainer" containerID="dd21b0bd313fbea7fd9fbbc6df20b5977fa0f518bc12404bb5083a4c2ee3b4ce" Dec 03 11:53:15 crc kubenswrapper[5052]: I1203 11:53:15.297496 5052 scope.go:117] "RemoveContainer" containerID="8edea1b7e7603813f9cee40e80060c8fe7cdde3a3fe81da594f714c8848d10a6" Dec 03 11:53:15 crc kubenswrapper[5052]: I1203 11:53:15.308745 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:53:15 crc kubenswrapper[5052]: I1203 11:53:15.323687 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:53:15 crc kubenswrapper[5052]: I1203 11:53:15.332168 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:53:15 crc kubenswrapper[5052]: E1203 11:53:15.332675 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="3bae2410-56dc-42bc-8974-4da004ac0b28" containerName="ceilometer-central-agent" Dec 03 11:53:15 crc kubenswrapper[5052]: I1203 11:53:15.332735 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="3bae2410-56dc-42bc-8974-4da004ac0b28" containerName="ceilometer-central-agent" Dec 03 11:53:15 crc kubenswrapper[5052]: E1203 11:53:15.332758 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="3bae2410-56dc-42bc-8974-4da004ac0b28" containerName="sg-core" Dec 03 11:53:15 crc kubenswrapper[5052]: I1203 11:53:15.332767 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="3bae2410-56dc-42bc-8974-4da004ac0b28" containerName="sg-core" Dec 03 11:53:15 crc kubenswrapper[5052]: E1203 11:53:15.332781 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="3bae2410-56dc-42bc-8974-4da004ac0b28" containerName="ceilometer-notification-agent" Dec 03 11:53:15 crc kubenswrapper[5052]: I1203 11:53:15.332790 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="3bae2410-56dc-42bc-8974-4da004ac0b28" containerName="ceilometer-notification-agent" Dec 03 11:53:15 crc kubenswrapper[5052]: E1203 11:53:15.332853 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="3bae2410-56dc-42bc-8974-4da004ac0b28" containerName="proxy-httpd" Dec 03 11:53:15 crc kubenswrapper[5052]: I1203 11:53:15.332861 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="3bae2410-56dc-42bc-8974-4da004ac0b28" containerName="proxy-httpd" Dec 03 11:53:15 crc kubenswrapper[5052]: I1203 11:53:15.333189 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="3bae2410-56dc-42bc-8974-4da004ac0b28" containerName="sg-core" Dec 03 11:53:15 crc kubenswrapper[5052]: I1203 11:53:15.333209 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="3bae2410-56dc-42bc-8974-4da004ac0b28" containerName="proxy-httpd" Dec 03 11:53:15 crc kubenswrapper[5052]: I1203 11:53:15.333256 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="3bae2410-56dc-42bc-8974-4da004ac0b28" containerName="ceilometer-notification-agent" Dec 03 11:53:15 crc kubenswrapper[5052]: I1203 11:53:15.333269 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="3bae2410-56dc-42bc-8974-4da004ac0b28" containerName="ceilometer-central-agent" Dec 03 11:53:15 crc kubenswrapper[5052]: I1203 11:53:15.335290 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:53:15 crc kubenswrapper[5052]: I1203 11:53:15.337192 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-ceilometer-internal-svc" Dec 03 11:53:15 crc kubenswrapper[5052]: I1203 11:53:15.337400 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-config-data" Dec 03 11:53:15 crc kubenswrapper[5052]: I1203 11:53:15.337555 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-scripts" Dec 03 11:53:15 crc kubenswrapper[5052]: I1203 11:53:15.337920 5052 scope.go:117] "RemoveContainer" containerID="3d4e37cd5ebdf25795857b70ca2eda328dcf4ce9f44215affa2902c0ee5c7ec8" Dec 03 11:53:15 crc kubenswrapper[5052]: I1203 11:53:15.348711 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:53:15 crc kubenswrapper[5052]: I1203 11:53:15.364826 5052 scope.go:117] "RemoveContainer" containerID="f9fcf25c0c0d7f76435313ac33854df45d1892029892dae5e2816386e51e35d6" Dec 03 11:53:15 crc kubenswrapper[5052]: E1203 11:53:15.365384 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f9fcf25c0c0d7f76435313ac33854df45d1892029892dae5e2816386e51e35d6\": container with ID starting with f9fcf25c0c0d7f76435313ac33854df45d1892029892dae5e2816386e51e35d6 not found: ID does not exist" containerID="f9fcf25c0c0d7f76435313ac33854df45d1892029892dae5e2816386e51e35d6" Dec 03 11:53:15 crc kubenswrapper[5052]: I1203 11:53:15.365427 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f9fcf25c0c0d7f76435313ac33854df45d1892029892dae5e2816386e51e35d6"} err="failed to get container status \"f9fcf25c0c0d7f76435313ac33854df45d1892029892dae5e2816386e51e35d6\": rpc error: code = NotFound desc = could not find container \"f9fcf25c0c0d7f76435313ac33854df45d1892029892dae5e2816386e51e35d6\": container with ID starting with f9fcf25c0c0d7f76435313ac33854df45d1892029892dae5e2816386e51e35d6 not found: ID does not exist" Dec 03 11:53:15 crc kubenswrapper[5052]: I1203 11:53:15.365448 5052 scope.go:117] "RemoveContainer" containerID="dd21b0bd313fbea7fd9fbbc6df20b5977fa0f518bc12404bb5083a4c2ee3b4ce" Dec 03 11:53:15 crc kubenswrapper[5052]: E1203 11:53:15.365959 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dd21b0bd313fbea7fd9fbbc6df20b5977fa0f518bc12404bb5083a4c2ee3b4ce\": container with ID starting with dd21b0bd313fbea7fd9fbbc6df20b5977fa0f518bc12404bb5083a4c2ee3b4ce not found: ID does not exist" containerID="dd21b0bd313fbea7fd9fbbc6df20b5977fa0f518bc12404bb5083a4c2ee3b4ce" Dec 03 11:53:15 crc kubenswrapper[5052]: I1203 11:53:15.365979 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dd21b0bd313fbea7fd9fbbc6df20b5977fa0f518bc12404bb5083a4c2ee3b4ce"} err="failed to get container status \"dd21b0bd313fbea7fd9fbbc6df20b5977fa0f518bc12404bb5083a4c2ee3b4ce\": rpc error: code = NotFound desc = could not find container \"dd21b0bd313fbea7fd9fbbc6df20b5977fa0f518bc12404bb5083a4c2ee3b4ce\": container with ID starting with dd21b0bd313fbea7fd9fbbc6df20b5977fa0f518bc12404bb5083a4c2ee3b4ce not found: ID does not exist" Dec 03 11:53:15 crc kubenswrapper[5052]: I1203 11:53:15.365994 5052 scope.go:117] "RemoveContainer" containerID="8edea1b7e7603813f9cee40e80060c8fe7cdde3a3fe81da594f714c8848d10a6" Dec 03 11:53:15 crc kubenswrapper[5052]: E1203 11:53:15.368778 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8edea1b7e7603813f9cee40e80060c8fe7cdde3a3fe81da594f714c8848d10a6\": container with ID starting with 8edea1b7e7603813f9cee40e80060c8fe7cdde3a3fe81da594f714c8848d10a6 not found: ID does not exist" containerID="8edea1b7e7603813f9cee40e80060c8fe7cdde3a3fe81da594f714c8848d10a6" Dec 03 11:53:15 crc kubenswrapper[5052]: I1203 11:53:15.368845 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8edea1b7e7603813f9cee40e80060c8fe7cdde3a3fe81da594f714c8848d10a6"} err="failed to get container status \"8edea1b7e7603813f9cee40e80060c8fe7cdde3a3fe81da594f714c8848d10a6\": rpc error: code = NotFound desc = could not find container \"8edea1b7e7603813f9cee40e80060c8fe7cdde3a3fe81da594f714c8848d10a6\": container with ID starting with 8edea1b7e7603813f9cee40e80060c8fe7cdde3a3fe81da594f714c8848d10a6 not found: ID does not exist" Dec 03 11:53:15 crc kubenswrapper[5052]: I1203 11:53:15.368878 5052 scope.go:117] "RemoveContainer" containerID="3d4e37cd5ebdf25795857b70ca2eda328dcf4ce9f44215affa2902c0ee5c7ec8" Dec 03 11:53:15 crc kubenswrapper[5052]: E1203 11:53:15.373078 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3d4e37cd5ebdf25795857b70ca2eda328dcf4ce9f44215affa2902c0ee5c7ec8\": container with ID starting with 3d4e37cd5ebdf25795857b70ca2eda328dcf4ce9f44215affa2902c0ee5c7ec8 not found: ID does not exist" containerID="3d4e37cd5ebdf25795857b70ca2eda328dcf4ce9f44215affa2902c0ee5c7ec8" Dec 03 11:53:15 crc kubenswrapper[5052]: I1203 11:53:15.373117 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3d4e37cd5ebdf25795857b70ca2eda328dcf4ce9f44215affa2902c0ee5c7ec8"} err="failed to get container status \"3d4e37cd5ebdf25795857b70ca2eda328dcf4ce9f44215affa2902c0ee5c7ec8\": rpc error: code = NotFound desc = could not find container \"3d4e37cd5ebdf25795857b70ca2eda328dcf4ce9f44215affa2902c0ee5c7ec8\": container with ID starting with 3d4e37cd5ebdf25795857b70ca2eda328dcf4ce9f44215affa2902c0ee5c7ec8 not found: ID does not exist" Dec 03 11:53:15 crc kubenswrapper[5052]: I1203 11:53:15.486426 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/eec21126-4992-40a1-ad21-4a005a5f7b1c-config-data\") pod \"ceilometer-0\" (UID: \"eec21126-4992-40a1-ad21-4a005a5f7b1c\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:53:15 crc kubenswrapper[5052]: I1203 11:53:15.486470 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/eec21126-4992-40a1-ad21-4a005a5f7b1c-run-httpd\") pod \"ceilometer-0\" (UID: \"eec21126-4992-40a1-ad21-4a005a5f7b1c\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:53:15 crc kubenswrapper[5052]: I1203 11:53:15.486624 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/eec21126-4992-40a1-ad21-4a005a5f7b1c-ceilometer-tls-certs\") pod \"ceilometer-0\" (UID: \"eec21126-4992-40a1-ad21-4a005a5f7b1c\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:53:15 crc kubenswrapper[5052]: I1203 11:53:15.486930 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/eec21126-4992-40a1-ad21-4a005a5f7b1c-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"eec21126-4992-40a1-ad21-4a005a5f7b1c\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:53:15 crc kubenswrapper[5052]: I1203 11:53:15.487011 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/eec21126-4992-40a1-ad21-4a005a5f7b1c-log-httpd\") pod \"ceilometer-0\" (UID: \"eec21126-4992-40a1-ad21-4a005a5f7b1c\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:53:15 crc kubenswrapper[5052]: I1203 11:53:15.487090 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/eec21126-4992-40a1-ad21-4a005a5f7b1c-scripts\") pod \"ceilometer-0\" (UID: \"eec21126-4992-40a1-ad21-4a005a5f7b1c\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:53:15 crc kubenswrapper[5052]: I1203 11:53:15.487134 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lbjk8\" (UniqueName: \"kubernetes.io/projected/eec21126-4992-40a1-ad21-4a005a5f7b1c-kube-api-access-lbjk8\") pod \"ceilometer-0\" (UID: \"eec21126-4992-40a1-ad21-4a005a5f7b1c\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:53:15 crc kubenswrapper[5052]: I1203 11:53:15.487165 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/eec21126-4992-40a1-ad21-4a005a5f7b1c-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"eec21126-4992-40a1-ad21-4a005a5f7b1c\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:53:15 crc kubenswrapper[5052]: I1203 11:53:15.588909 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/eec21126-4992-40a1-ad21-4a005a5f7b1c-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"eec21126-4992-40a1-ad21-4a005a5f7b1c\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:53:15 crc kubenswrapper[5052]: I1203 11:53:15.588975 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/eec21126-4992-40a1-ad21-4a005a5f7b1c-log-httpd\") pod \"ceilometer-0\" (UID: \"eec21126-4992-40a1-ad21-4a005a5f7b1c\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:53:15 crc kubenswrapper[5052]: I1203 11:53:15.589018 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/eec21126-4992-40a1-ad21-4a005a5f7b1c-scripts\") pod \"ceilometer-0\" (UID: \"eec21126-4992-40a1-ad21-4a005a5f7b1c\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:53:15 crc kubenswrapper[5052]: I1203 11:53:15.589045 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-lbjk8\" (UniqueName: \"kubernetes.io/projected/eec21126-4992-40a1-ad21-4a005a5f7b1c-kube-api-access-lbjk8\") pod \"ceilometer-0\" (UID: \"eec21126-4992-40a1-ad21-4a005a5f7b1c\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:53:15 crc kubenswrapper[5052]: I1203 11:53:15.589077 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/eec21126-4992-40a1-ad21-4a005a5f7b1c-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"eec21126-4992-40a1-ad21-4a005a5f7b1c\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:53:15 crc kubenswrapper[5052]: I1203 11:53:15.589178 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/eec21126-4992-40a1-ad21-4a005a5f7b1c-config-data\") pod \"ceilometer-0\" (UID: \"eec21126-4992-40a1-ad21-4a005a5f7b1c\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:53:15 crc kubenswrapper[5052]: I1203 11:53:15.589199 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/eec21126-4992-40a1-ad21-4a005a5f7b1c-run-httpd\") pod \"ceilometer-0\" (UID: \"eec21126-4992-40a1-ad21-4a005a5f7b1c\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:53:15 crc kubenswrapper[5052]: I1203 11:53:15.589234 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/eec21126-4992-40a1-ad21-4a005a5f7b1c-ceilometer-tls-certs\") pod \"ceilometer-0\" (UID: \"eec21126-4992-40a1-ad21-4a005a5f7b1c\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:53:15 crc kubenswrapper[5052]: I1203 11:53:15.590130 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/eec21126-4992-40a1-ad21-4a005a5f7b1c-log-httpd\") pod \"ceilometer-0\" (UID: \"eec21126-4992-40a1-ad21-4a005a5f7b1c\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:53:15 crc kubenswrapper[5052]: I1203 11:53:15.590483 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/eec21126-4992-40a1-ad21-4a005a5f7b1c-run-httpd\") pod \"ceilometer-0\" (UID: \"eec21126-4992-40a1-ad21-4a005a5f7b1c\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:53:15 crc kubenswrapper[5052]: I1203 11:53:15.593785 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/eec21126-4992-40a1-ad21-4a005a5f7b1c-scripts\") pod \"ceilometer-0\" (UID: \"eec21126-4992-40a1-ad21-4a005a5f7b1c\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:53:15 crc kubenswrapper[5052]: I1203 11:53:15.595256 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/eec21126-4992-40a1-ad21-4a005a5f7b1c-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"eec21126-4992-40a1-ad21-4a005a5f7b1c\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:53:15 crc kubenswrapper[5052]: I1203 11:53:15.595257 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/eec21126-4992-40a1-ad21-4a005a5f7b1c-ceilometer-tls-certs\") pod \"ceilometer-0\" (UID: \"eec21126-4992-40a1-ad21-4a005a5f7b1c\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:53:15 crc kubenswrapper[5052]: I1203 11:53:15.596311 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/eec21126-4992-40a1-ad21-4a005a5f7b1c-config-data\") pod \"ceilometer-0\" (UID: \"eec21126-4992-40a1-ad21-4a005a5f7b1c\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:53:15 crc kubenswrapper[5052]: I1203 11:53:15.597138 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/eec21126-4992-40a1-ad21-4a005a5f7b1c-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"eec21126-4992-40a1-ad21-4a005a5f7b1c\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:53:15 crc kubenswrapper[5052]: I1203 11:53:15.610455 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-lbjk8\" (UniqueName: \"kubernetes.io/projected/eec21126-4992-40a1-ad21-4a005a5f7b1c-kube-api-access-lbjk8\") pod \"ceilometer-0\" (UID: \"eec21126-4992-40a1-ad21-4a005a5f7b1c\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:53:15 crc kubenswrapper[5052]: I1203 11:53:15.659327 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:53:15 crc kubenswrapper[5052]: I1203 11:53:15.749663 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3bae2410-56dc-42bc-8974-4da004ac0b28" path="/var/lib/kubelet/pods/3bae2410-56dc-42bc-8974-4da004ac0b28/volumes" Dec 03 11:53:16 crc kubenswrapper[5052]: I1203 11:53:16.142697 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:53:16 crc kubenswrapper[5052]: W1203 11:53:16.145838 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podeec21126_4992_40a1_ad21_4a005a5f7b1c.slice/crio-9f1bc73aa05c422d96db7f00c5b657494b348132ea05203d632c3ef351760039 WatchSource:0}: Error finding container 9f1bc73aa05c422d96db7f00c5b657494b348132ea05203d632c3ef351760039: Status 404 returned error can't find the container with id 9f1bc73aa05c422d96db7f00c5b657494b348132ea05203d632c3ef351760039 Dec 03 11:53:16 crc kubenswrapper[5052]: I1203 11:53:16.240476 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-operators-cjtc5" event={"ID":"5d4ac632-b94b-4371-a605-458e4cd92f23","Type":"ContainerStarted","Data":"415388246ab7884c9c03ffa53307c26fad0811817bdf7ab961534d37b83287cd"} Dec 03 11:53:16 crc kubenswrapper[5052]: I1203 11:53:16.243054 5052 generic.go:334] "Generic (PLEG): container finished" podID="a4ee181e-0136-4d6c-aad6-e0ffcc2cb9db" containerID="f681937d4e3600c07f0e7a5f6674a949192d35f7d97ddf1a5a4f6b78e7c5ca57" exitCode=0 Dec 03 11:53:16 crc kubenswrapper[5052]: I1203 11:53:16.243111 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"a4ee181e-0136-4d6c-aad6-e0ffcc2cb9db","Type":"ContainerDied","Data":"f681937d4e3600c07f0e7a5f6674a949192d35f7d97ddf1a5a4f6b78e7c5ca57"} Dec 03 11:53:16 crc kubenswrapper[5052]: I1203 11:53:16.243131 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"a4ee181e-0136-4d6c-aad6-e0ffcc2cb9db","Type":"ContainerDied","Data":"b4b1f46f1d7bf1e1ee51bcd6ab184eaf68fe533d92da86903f24b02ad60a0af3"} Dec 03 11:53:16 crc kubenswrapper[5052]: I1203 11:53:16.243142 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b4b1f46f1d7bf1e1ee51bcd6ab184eaf68fe533d92da86903f24b02ad60a0af3" Dec 03 11:53:16 crc kubenswrapper[5052]: I1203 11:53:16.243619 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:53:16 crc kubenswrapper[5052]: I1203 11:53:16.246055 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"eec21126-4992-40a1-ad21-4a005a5f7b1c","Type":"ContainerStarted","Data":"9f1bc73aa05c422d96db7f00c5b657494b348132ea05203d632c3ef351760039"} Dec 03 11:53:16 crc kubenswrapper[5052]: I1203 11:53:16.419887 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/a4ee181e-0136-4d6c-aad6-e0ffcc2cb9db-logs\") pod \"a4ee181e-0136-4d6c-aad6-e0ffcc2cb9db\" (UID: \"a4ee181e-0136-4d6c-aad6-e0ffcc2cb9db\") " Dec 03 11:53:16 crc kubenswrapper[5052]: I1203 11:53:16.420438 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a4ee181e-0136-4d6c-aad6-e0ffcc2cb9db-combined-ca-bundle\") pod \"a4ee181e-0136-4d6c-aad6-e0ffcc2cb9db\" (UID: \"a4ee181e-0136-4d6c-aad6-e0ffcc2cb9db\") " Dec 03 11:53:16 crc kubenswrapper[5052]: I1203 11:53:16.420467 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a4ee181e-0136-4d6c-aad6-e0ffcc2cb9db-config-data\") pod \"a4ee181e-0136-4d6c-aad6-e0ffcc2cb9db\" (UID: \"a4ee181e-0136-4d6c-aad6-e0ffcc2cb9db\") " Dec 03 11:53:16 crc kubenswrapper[5052]: I1203 11:53:16.420435 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4ee181e-0136-4d6c-aad6-e0ffcc2cb9db-logs" (OuterVolumeSpecName: "logs") pod "a4ee181e-0136-4d6c-aad6-e0ffcc2cb9db" (UID: "a4ee181e-0136-4d6c-aad6-e0ffcc2cb9db"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:53:16 crc kubenswrapper[5052]: I1203 11:53:16.420687 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rgn28\" (UniqueName: \"kubernetes.io/projected/a4ee181e-0136-4d6c-aad6-e0ffcc2cb9db-kube-api-access-rgn28\") pod \"a4ee181e-0136-4d6c-aad6-e0ffcc2cb9db\" (UID: \"a4ee181e-0136-4d6c-aad6-e0ffcc2cb9db\") " Dec 03 11:53:16 crc kubenswrapper[5052]: I1203 11:53:16.422063 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/a4ee181e-0136-4d6c-aad6-e0ffcc2cb9db-logs\") on node \"crc\" DevicePath \"\"" Dec 03 11:53:16 crc kubenswrapper[5052]: I1203 11:53:16.426157 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a4ee181e-0136-4d6c-aad6-e0ffcc2cb9db-kube-api-access-rgn28" (OuterVolumeSpecName: "kube-api-access-rgn28") pod "a4ee181e-0136-4d6c-aad6-e0ffcc2cb9db" (UID: "a4ee181e-0136-4d6c-aad6-e0ffcc2cb9db"). InnerVolumeSpecName "kube-api-access-rgn28". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:53:16 crc kubenswrapper[5052]: I1203 11:53:16.455711 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a4ee181e-0136-4d6c-aad6-e0ffcc2cb9db-config-data" (OuterVolumeSpecName: "config-data") pod "a4ee181e-0136-4d6c-aad6-e0ffcc2cb9db" (UID: "a4ee181e-0136-4d6c-aad6-e0ffcc2cb9db"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:53:16 crc kubenswrapper[5052]: I1203 11:53:16.461609 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a4ee181e-0136-4d6c-aad6-e0ffcc2cb9db-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "a4ee181e-0136-4d6c-aad6-e0ffcc2cb9db" (UID: "a4ee181e-0136-4d6c-aad6-e0ffcc2cb9db"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:53:16 crc kubenswrapper[5052]: I1203 11:53:16.522721 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-rgn28\" (UniqueName: \"kubernetes.io/projected/a4ee181e-0136-4d6c-aad6-e0ffcc2cb9db-kube-api-access-rgn28\") on node \"crc\" DevicePath \"\"" Dec 03 11:53:16 crc kubenswrapper[5052]: I1203 11:53:16.522759 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a4ee181e-0136-4d6c-aad6-e0ffcc2cb9db-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:53:16 crc kubenswrapper[5052]: I1203 11:53:16.522772 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a4ee181e-0136-4d6c-aad6-e0ffcc2cb9db-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:53:16 crc kubenswrapper[5052]: I1203 11:53:16.644838 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:53:16 crc kubenswrapper[5052]: I1203 11:53:16.727072 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/c32dead2-3fd2-497d-b4a1-56c24d3cd8dd-config-data\") pod \"c32dead2-3fd2-497d-b4a1-56c24d3cd8dd\" (UID: \"c32dead2-3fd2-497d-b4a1-56c24d3cd8dd\") " Dec 03 11:53:16 crc kubenswrapper[5052]: I1203 11:53:16.727564 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c32dead2-3fd2-497d-b4a1-56c24d3cd8dd-combined-ca-bundle\") pod \"c32dead2-3fd2-497d-b4a1-56c24d3cd8dd\" (UID: \"c32dead2-3fd2-497d-b4a1-56c24d3cd8dd\") " Dec 03 11:53:16 crc kubenswrapper[5052]: I1203 11:53:16.727744 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mtdwx\" (UniqueName: \"kubernetes.io/projected/c32dead2-3fd2-497d-b4a1-56c24d3cd8dd-kube-api-access-mtdwx\") pod \"c32dead2-3fd2-497d-b4a1-56c24d3cd8dd\" (UID: \"c32dead2-3fd2-497d-b4a1-56c24d3cd8dd\") " Dec 03 11:53:16 crc kubenswrapper[5052]: I1203 11:53:16.733604 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c32dead2-3fd2-497d-b4a1-56c24d3cd8dd-kube-api-access-mtdwx" (OuterVolumeSpecName: "kube-api-access-mtdwx") pod "c32dead2-3fd2-497d-b4a1-56c24d3cd8dd" (UID: "c32dead2-3fd2-497d-b4a1-56c24d3cd8dd"). InnerVolumeSpecName "kube-api-access-mtdwx". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:53:16 crc kubenswrapper[5052]: I1203 11:53:16.775066 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c32dead2-3fd2-497d-b4a1-56c24d3cd8dd-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "c32dead2-3fd2-497d-b4a1-56c24d3cd8dd" (UID: "c32dead2-3fd2-497d-b4a1-56c24d3cd8dd"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:53:16 crc kubenswrapper[5052]: I1203 11:53:16.776382 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c32dead2-3fd2-497d-b4a1-56c24d3cd8dd-config-data" (OuterVolumeSpecName: "config-data") pod "c32dead2-3fd2-497d-b4a1-56c24d3cd8dd" (UID: "c32dead2-3fd2-497d-b4a1-56c24d3cd8dd"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:53:16 crc kubenswrapper[5052]: I1203 11:53:16.829843 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c32dead2-3fd2-497d-b4a1-56c24d3cd8dd-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:53:16 crc kubenswrapper[5052]: I1203 11:53:16.829886 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-mtdwx\" (UniqueName: \"kubernetes.io/projected/c32dead2-3fd2-497d-b4a1-56c24d3cd8dd-kube-api-access-mtdwx\") on node \"crc\" DevicePath \"\"" Dec 03 11:53:16 crc kubenswrapper[5052]: I1203 11:53:16.829908 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/c32dead2-3fd2-497d-b4a1-56c24d3cd8dd-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:53:17 crc kubenswrapper[5052]: I1203 11:53:17.257773 5052 generic.go:334] "Generic (PLEG): container finished" podID="c32dead2-3fd2-497d-b4a1-56c24d3cd8dd" containerID="8bd79e5e170a8c0b2749fe7d7e6882a7f14fa8575090df791ae6d359ca030e14" exitCode=0 Dec 03 11:53:17 crc kubenswrapper[5052]: I1203 11:53:17.257822 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"c32dead2-3fd2-497d-b4a1-56c24d3cd8dd","Type":"ContainerDied","Data":"8bd79e5e170a8c0b2749fe7d7e6882a7f14fa8575090df791ae6d359ca030e14"} Dec 03 11:53:17 crc kubenswrapper[5052]: I1203 11:53:17.257864 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:53:17 crc kubenswrapper[5052]: I1203 11:53:17.257893 5052 scope.go:117] "RemoveContainer" containerID="8bd79e5e170a8c0b2749fe7d7e6882a7f14fa8575090df791ae6d359ca030e14" Dec 03 11:53:17 crc kubenswrapper[5052]: I1203 11:53:17.257869 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"c32dead2-3fd2-497d-b4a1-56c24d3cd8dd","Type":"ContainerDied","Data":"a44d845b110bda884e47e5e504790e881029f210274249b4cfe3ac417bf1ed1a"} Dec 03 11:53:17 crc kubenswrapper[5052]: I1203 11:53:17.260596 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"eec21126-4992-40a1-ad21-4a005a5f7b1c","Type":"ContainerStarted","Data":"bb276bcdc3ab1893514f1a6e8c1d6bce34dced631ba9630ce9d6aa621798b31b"} Dec 03 11:53:17 crc kubenswrapper[5052]: I1203 11:53:17.263469 5052 generic.go:334] "Generic (PLEG): container finished" podID="5d4ac632-b94b-4371-a605-458e4cd92f23" containerID="415388246ab7884c9c03ffa53307c26fad0811817bdf7ab961534d37b83287cd" exitCode=0 Dec 03 11:53:17 crc kubenswrapper[5052]: I1203 11:53:17.263597 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:53:17 crc kubenswrapper[5052]: I1203 11:53:17.263649 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-operators-cjtc5" event={"ID":"5d4ac632-b94b-4371-a605-458e4cd92f23","Type":"ContainerDied","Data":"415388246ab7884c9c03ffa53307c26fad0811817bdf7ab961534d37b83287cd"} Dec 03 11:53:17 crc kubenswrapper[5052]: I1203 11:53:17.291665 5052 scope.go:117] "RemoveContainer" containerID="8bd79e5e170a8c0b2749fe7d7e6882a7f14fa8575090df791ae6d359ca030e14" Dec 03 11:53:17 crc kubenswrapper[5052]: E1203 11:53:17.292168 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8bd79e5e170a8c0b2749fe7d7e6882a7f14fa8575090df791ae6d359ca030e14\": container with ID starting with 8bd79e5e170a8c0b2749fe7d7e6882a7f14fa8575090df791ae6d359ca030e14 not found: ID does not exist" containerID="8bd79e5e170a8c0b2749fe7d7e6882a7f14fa8575090df791ae6d359ca030e14" Dec 03 11:53:17 crc kubenswrapper[5052]: I1203 11:53:17.292216 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8bd79e5e170a8c0b2749fe7d7e6882a7f14fa8575090df791ae6d359ca030e14"} err="failed to get container status \"8bd79e5e170a8c0b2749fe7d7e6882a7f14fa8575090df791ae6d359ca030e14\": rpc error: code = NotFound desc = could not find container \"8bd79e5e170a8c0b2749fe7d7e6882a7f14fa8575090df791ae6d359ca030e14\": container with ID starting with 8bd79e5e170a8c0b2749fe7d7e6882a7f14fa8575090df791ae6d359ca030e14 not found: ID does not exist" Dec 03 11:53:17 crc kubenswrapper[5052]: I1203 11:53:17.321613 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 11:53:17 crc kubenswrapper[5052]: I1203 11:53:17.336778 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 11:53:17 crc kubenswrapper[5052]: I1203 11:53:17.344984 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 11:53:17 crc kubenswrapper[5052]: I1203 11:53:17.354252 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 11:53:17 crc kubenswrapper[5052]: I1203 11:53:17.362935 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 11:53:17 crc kubenswrapper[5052]: E1203 11:53:17.363331 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="a4ee181e-0136-4d6c-aad6-e0ffcc2cb9db" containerName="nova-api-api" Dec 03 11:53:17 crc kubenswrapper[5052]: I1203 11:53:17.363346 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="a4ee181e-0136-4d6c-aad6-e0ffcc2cb9db" containerName="nova-api-api" Dec 03 11:53:17 crc kubenswrapper[5052]: E1203 11:53:17.363383 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="a4ee181e-0136-4d6c-aad6-e0ffcc2cb9db" containerName="nova-api-log" Dec 03 11:53:17 crc kubenswrapper[5052]: I1203 11:53:17.363389 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="a4ee181e-0136-4d6c-aad6-e0ffcc2cb9db" containerName="nova-api-log" Dec 03 11:53:17 crc kubenswrapper[5052]: E1203 11:53:17.363406 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="c32dead2-3fd2-497d-b4a1-56c24d3cd8dd" containerName="nova-scheduler-scheduler" Dec 03 11:53:17 crc kubenswrapper[5052]: I1203 11:53:17.363414 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="c32dead2-3fd2-497d-b4a1-56c24d3cd8dd" containerName="nova-scheduler-scheduler" Dec 03 11:53:17 crc kubenswrapper[5052]: I1203 11:53:17.363625 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="a4ee181e-0136-4d6c-aad6-e0ffcc2cb9db" containerName="nova-api-log" Dec 03 11:53:17 crc kubenswrapper[5052]: I1203 11:53:17.363643 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="c32dead2-3fd2-497d-b4a1-56c24d3cd8dd" containerName="nova-scheduler-scheduler" Dec 03 11:53:17 crc kubenswrapper[5052]: I1203 11:53:17.363656 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="a4ee181e-0136-4d6c-aad6-e0ffcc2cb9db" containerName="nova-api-api" Dec 03 11:53:17 crc kubenswrapper[5052]: I1203 11:53:17.364375 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:53:17 crc kubenswrapper[5052]: I1203 11:53:17.372777 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-scheduler-config-data" Dec 03 11:53:17 crc kubenswrapper[5052]: I1203 11:53:17.377466 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 11:53:17 crc kubenswrapper[5052]: I1203 11:53:17.387432 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 11:53:17 crc kubenswrapper[5052]: I1203 11:53:17.389104 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:53:17 crc kubenswrapper[5052]: I1203 11:53:17.391855 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-api-config-data" Dec 03 11:53:17 crc kubenswrapper[5052]: I1203 11:53:17.408145 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 11:53:17 crc kubenswrapper[5052]: I1203 11:53:17.541966 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c8f85677-5515-4318-9c54-66505a4c416c-combined-ca-bundle\") pod \"nova-scheduler-0\" (UID: \"c8f85677-5515-4318-9c54-66505a4c416c\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:53:17 crc kubenswrapper[5052]: I1203 11:53:17.542372 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/9a82d4a4-d01b-475f-87ac-c7a3b8f353a6-config-data\") pod \"nova-api-0\" (UID: \"9a82d4a4-d01b-475f-87ac-c7a3b8f353a6\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:53:17 crc kubenswrapper[5052]: I1203 11:53:17.542413 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/c8f85677-5515-4318-9c54-66505a4c416c-config-data\") pod \"nova-scheduler-0\" (UID: \"c8f85677-5515-4318-9c54-66505a4c416c\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:53:17 crc kubenswrapper[5052]: I1203 11:53:17.542564 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/9a82d4a4-d01b-475f-87ac-c7a3b8f353a6-logs\") pod \"nova-api-0\" (UID: \"9a82d4a4-d01b-475f-87ac-c7a3b8f353a6\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:53:17 crc kubenswrapper[5052]: I1203 11:53:17.542663 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g6g9g\" (UniqueName: \"kubernetes.io/projected/c8f85677-5515-4318-9c54-66505a4c416c-kube-api-access-g6g9g\") pod \"nova-scheduler-0\" (UID: \"c8f85677-5515-4318-9c54-66505a4c416c\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:53:17 crc kubenswrapper[5052]: I1203 11:53:17.542722 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9a82d4a4-d01b-475f-87ac-c7a3b8f353a6-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"9a82d4a4-d01b-475f-87ac-c7a3b8f353a6\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:53:17 crc kubenswrapper[5052]: I1203 11:53:17.542775 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2x85h\" (UniqueName: \"kubernetes.io/projected/9a82d4a4-d01b-475f-87ac-c7a3b8f353a6-kube-api-access-2x85h\") pod \"nova-api-0\" (UID: \"9a82d4a4-d01b-475f-87ac-c7a3b8f353a6\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:53:17 crc kubenswrapper[5052]: I1203 11:53:17.568679 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:53:17 crc kubenswrapper[5052]: I1203 11:53:17.568814 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:53:17 crc kubenswrapper[5052]: I1203 11:53:17.644676 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/9a82d4a4-d01b-475f-87ac-c7a3b8f353a6-logs\") pod \"nova-api-0\" (UID: \"9a82d4a4-d01b-475f-87ac-c7a3b8f353a6\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:53:17 crc kubenswrapper[5052]: I1203 11:53:17.644739 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-g6g9g\" (UniqueName: \"kubernetes.io/projected/c8f85677-5515-4318-9c54-66505a4c416c-kube-api-access-g6g9g\") pod \"nova-scheduler-0\" (UID: \"c8f85677-5515-4318-9c54-66505a4c416c\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:53:17 crc kubenswrapper[5052]: I1203 11:53:17.644776 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9a82d4a4-d01b-475f-87ac-c7a3b8f353a6-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"9a82d4a4-d01b-475f-87ac-c7a3b8f353a6\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:53:17 crc kubenswrapper[5052]: I1203 11:53:17.644804 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-2x85h\" (UniqueName: \"kubernetes.io/projected/9a82d4a4-d01b-475f-87ac-c7a3b8f353a6-kube-api-access-2x85h\") pod \"nova-api-0\" (UID: \"9a82d4a4-d01b-475f-87ac-c7a3b8f353a6\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:53:17 crc kubenswrapper[5052]: I1203 11:53:17.644844 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c8f85677-5515-4318-9c54-66505a4c416c-combined-ca-bundle\") pod \"nova-scheduler-0\" (UID: \"c8f85677-5515-4318-9c54-66505a4c416c\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:53:17 crc kubenswrapper[5052]: I1203 11:53:17.644906 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/9a82d4a4-d01b-475f-87ac-c7a3b8f353a6-config-data\") pod \"nova-api-0\" (UID: \"9a82d4a4-d01b-475f-87ac-c7a3b8f353a6\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:53:17 crc kubenswrapper[5052]: I1203 11:53:17.644933 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/c8f85677-5515-4318-9c54-66505a4c416c-config-data\") pod \"nova-scheduler-0\" (UID: \"c8f85677-5515-4318-9c54-66505a4c416c\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:53:17 crc kubenswrapper[5052]: I1203 11:53:17.646257 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/9a82d4a4-d01b-475f-87ac-c7a3b8f353a6-logs\") pod \"nova-api-0\" (UID: \"9a82d4a4-d01b-475f-87ac-c7a3b8f353a6\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:53:17 crc kubenswrapper[5052]: I1203 11:53:17.648262 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c8f85677-5515-4318-9c54-66505a4c416c-combined-ca-bundle\") pod \"nova-scheduler-0\" (UID: \"c8f85677-5515-4318-9c54-66505a4c416c\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:53:17 crc kubenswrapper[5052]: I1203 11:53:17.649189 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9a82d4a4-d01b-475f-87ac-c7a3b8f353a6-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"9a82d4a4-d01b-475f-87ac-c7a3b8f353a6\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:53:17 crc kubenswrapper[5052]: I1203 11:53:17.651324 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/9a82d4a4-d01b-475f-87ac-c7a3b8f353a6-config-data\") pod \"nova-api-0\" (UID: \"9a82d4a4-d01b-475f-87ac-c7a3b8f353a6\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:53:17 crc kubenswrapper[5052]: I1203 11:53:17.653689 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/c8f85677-5515-4318-9c54-66505a4c416c-config-data\") pod \"nova-scheduler-0\" (UID: \"c8f85677-5515-4318-9c54-66505a4c416c\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:53:17 crc kubenswrapper[5052]: I1203 11:53:17.665156 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-2x85h\" (UniqueName: \"kubernetes.io/projected/9a82d4a4-d01b-475f-87ac-c7a3b8f353a6-kube-api-access-2x85h\") pod \"nova-api-0\" (UID: \"9a82d4a4-d01b-475f-87ac-c7a3b8f353a6\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:53:17 crc kubenswrapper[5052]: I1203 11:53:17.670031 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-g6g9g\" (UniqueName: \"kubernetes.io/projected/c8f85677-5515-4318-9c54-66505a4c416c-kube-api-access-g6g9g\") pod \"nova-scheduler-0\" (UID: \"c8f85677-5515-4318-9c54-66505a4c416c\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:53:17 crc kubenswrapper[5052]: I1203 11:53:17.686004 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a4ee181e-0136-4d6c-aad6-e0ffcc2cb9db" path="/var/lib/kubelet/pods/a4ee181e-0136-4d6c-aad6-e0ffcc2cb9db/volumes" Dec 03 11:53:17 crc kubenswrapper[5052]: I1203 11:53:17.687071 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c32dead2-3fd2-497d-b4a1-56c24d3cd8dd" path="/var/lib/kubelet/pods/c32dead2-3fd2-497d-b4a1-56c24d3cd8dd/volumes" Dec 03 11:53:17 crc kubenswrapper[5052]: I1203 11:53:17.687868 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:53:17 crc kubenswrapper[5052]: I1203 11:53:17.705057 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:53:18 crc kubenswrapper[5052]: I1203 11:53:18.155686 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 11:53:18 crc kubenswrapper[5052]: I1203 11:53:18.240449 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 11:53:18 crc kubenswrapper[5052]: I1203 11:53:18.275703 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"eec21126-4992-40a1-ad21-4a005a5f7b1c","Type":"ContainerStarted","Data":"f5421d640c3d65fd8d438e37a5d376270f59fc2eef924597b04b9ca53af1eed4"} Dec 03 11:53:18 crc kubenswrapper[5052]: I1203 11:53:18.276838 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"c8f85677-5515-4318-9c54-66505a4c416c","Type":"ContainerStarted","Data":"e4c0fd0cab706599a087e52a95b0ff2b391a487409fb90a655bce6a601ddd9c5"} Dec 03 11:53:18 crc kubenswrapper[5052]: I1203 11:53:18.278825 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"9a82d4a4-d01b-475f-87ac-c7a3b8f353a6","Type":"ContainerStarted","Data":"7881219e0f5dd3f01b64e49672bb3184d2cf074949e6d28ccbc8127263654aea"} Dec 03 11:53:18 crc kubenswrapper[5052]: I1203 11:53:18.472141 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 11:53:19 crc kubenswrapper[5052]: I1203 11:53:19.292486 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"eec21126-4992-40a1-ad21-4a005a5f7b1c","Type":"ContainerStarted","Data":"2d366eb586a77209ca68fbef78c7a3d89603fae204b9708a92563b46fcfbb032"} Dec 03 11:53:19 crc kubenswrapper[5052]: I1203 11:53:19.293998 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"c8f85677-5515-4318-9c54-66505a4c416c","Type":"ContainerStarted","Data":"6901b1ff77b02438397e55f7ee6fdb393d6f8e2b944de48831f9519f83e59546"} Dec 03 11:53:19 crc kubenswrapper[5052]: I1203 11:53:19.308162 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-operators-cjtc5" event={"ID":"5d4ac632-b94b-4371-a605-458e4cd92f23","Type":"ContainerStarted","Data":"b208896c5863e9021e60fd793f2b067644ac23e34dd7d6e5cf9c008f1ce97991"} Dec 03 11:53:19 crc kubenswrapper[5052]: I1203 11:53:19.314972 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"9a82d4a4-d01b-475f-87ac-c7a3b8f353a6","Type":"ContainerStarted","Data":"2a583d85b5f9e4a6e90e25d7a948ae635e6695ec5893b35762c3f5cb63b8041b"} Dec 03 11:53:19 crc kubenswrapper[5052]: I1203 11:53:19.315020 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"9a82d4a4-d01b-475f-87ac-c7a3b8f353a6","Type":"ContainerStarted","Data":"620cafd319adde74eb2596aae4bd6d78515283619ae6e97aa04cddd040616ed9"} Dec 03 11:53:19 crc kubenswrapper[5052]: I1203 11:53:19.318502 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-scheduler-0" podStartSLOduration=2.318487291 podStartE2EDuration="2.318487291s" podCreationTimestamp="2025-12-03 11:53:17 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:53:19.312471999 +0000 UTC m=+1989.900713773" watchObservedRunningTime="2025-12-03 11:53:19.318487291 +0000 UTC m=+1989.906729055" Dec 03 11:53:19 crc kubenswrapper[5052]: I1203 11:53:19.332872 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-marketplace/redhat-operators-cjtc5" podStartSLOduration=3.453041148 podStartE2EDuration="6.332850012s" podCreationTimestamp="2025-12-03 11:53:13 +0000 UTC" firstStartedPulling="2025-12-03 11:53:15.232238094 +0000 UTC m=+1985.820479858" lastFinishedPulling="2025-12-03 11:53:18.112046958 +0000 UTC m=+1988.700288722" observedRunningTime="2025-12-03 11:53:19.331675098 +0000 UTC m=+1989.919916862" watchObservedRunningTime="2025-12-03 11:53:19.332850012 +0000 UTC m=+1989.921091776" Dec 03 11:53:19 crc kubenswrapper[5052]: I1203 11:53:19.368774 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-api-0" podStartSLOduration=2.368743989 podStartE2EDuration="2.368743989s" podCreationTimestamp="2025-12-03 11:53:17 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:53:19.355246492 +0000 UTC m=+1989.943488256" watchObservedRunningTime="2025-12-03 11:53:19.368743989 +0000 UTC m=+1989.956985783" Dec 03 11:53:21 crc kubenswrapper[5052]: I1203 11:53:21.339834 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"eec21126-4992-40a1-ad21-4a005a5f7b1c","Type":"ContainerStarted","Data":"1f882f2f0bb94e8f107211926dcf76256c015c58e2c4caf41052300a24daec33"} Dec 03 11:53:21 crc kubenswrapper[5052]: I1203 11:53:21.340286 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:53:22 crc kubenswrapper[5052]: I1203 11:53:22.563881 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:53:22 crc kubenswrapper[5052]: I1203 11:53:22.564538 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:53:22 crc kubenswrapper[5052]: I1203 11:53:22.596363 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 11:53:22 crc kubenswrapper[5052]: I1203 11:53:22.623588 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/ceilometer-0" podStartSLOduration=3.143090447 podStartE2EDuration="7.623570861s" podCreationTimestamp="2025-12-03 11:53:15 +0000 UTC" firstStartedPulling="2025-12-03 11:53:16.149926649 +0000 UTC m=+1986.738168413" lastFinishedPulling="2025-12-03 11:53:20.630407073 +0000 UTC m=+1991.218648827" observedRunningTime="2025-12-03 11:53:21.37772247 +0000 UTC m=+1991.965964244" watchObservedRunningTime="2025-12-03 11:53:22.623570861 +0000 UTC m=+1993.211812625" Dec 03 11:53:22 crc kubenswrapper[5052]: I1203 11:53:22.688834 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:53:23 crc kubenswrapper[5052]: I1203 11:53:23.112303 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell1-cell-mapping-ptcx4"] Dec 03 11:53:23 crc kubenswrapper[5052]: I1203 11:53:23.113601 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-cell-mapping-ptcx4" Dec 03 11:53:23 crc kubenswrapper[5052]: I1203 11:53:23.119011 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell1-manage-scripts" Dec 03 11:53:23 crc kubenswrapper[5052]: I1203 11:53:23.119109 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell1-manage-config-data" Dec 03 11:53:23 crc kubenswrapper[5052]: I1203 11:53:23.150364 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-cell-mapping-ptcx4"] Dec 03 11:53:23 crc kubenswrapper[5052]: I1203 11:53:23.263309 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j2974\" (UniqueName: \"kubernetes.io/projected/b72434ac-6b58-49a8-aaa1-6d5aa7b920f2-kube-api-access-j2974\") pod \"nova-cell1-cell-mapping-ptcx4\" (UID: \"b72434ac-6b58-49a8-aaa1-6d5aa7b920f2\") " pod="openstack-kuttl-tests/nova-cell1-cell-mapping-ptcx4" Dec 03 11:53:23 crc kubenswrapper[5052]: I1203 11:53:23.263357 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/b72434ac-6b58-49a8-aaa1-6d5aa7b920f2-config-data\") pod \"nova-cell1-cell-mapping-ptcx4\" (UID: \"b72434ac-6b58-49a8-aaa1-6d5aa7b920f2\") " pod="openstack-kuttl-tests/nova-cell1-cell-mapping-ptcx4" Dec 03 11:53:23 crc kubenswrapper[5052]: I1203 11:53:23.263399 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/b72434ac-6b58-49a8-aaa1-6d5aa7b920f2-scripts\") pod \"nova-cell1-cell-mapping-ptcx4\" (UID: \"b72434ac-6b58-49a8-aaa1-6d5aa7b920f2\") " pod="openstack-kuttl-tests/nova-cell1-cell-mapping-ptcx4" Dec 03 11:53:23 crc kubenswrapper[5052]: I1203 11:53:23.263482 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b72434ac-6b58-49a8-aaa1-6d5aa7b920f2-combined-ca-bundle\") pod \"nova-cell1-cell-mapping-ptcx4\" (UID: \"b72434ac-6b58-49a8-aaa1-6d5aa7b920f2\") " pod="openstack-kuttl-tests/nova-cell1-cell-mapping-ptcx4" Dec 03 11:53:23 crc kubenswrapper[5052]: I1203 11:53:23.365403 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/b72434ac-6b58-49a8-aaa1-6d5aa7b920f2-scripts\") pod \"nova-cell1-cell-mapping-ptcx4\" (UID: \"b72434ac-6b58-49a8-aaa1-6d5aa7b920f2\") " pod="openstack-kuttl-tests/nova-cell1-cell-mapping-ptcx4" Dec 03 11:53:23 crc kubenswrapper[5052]: I1203 11:53:23.365615 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b72434ac-6b58-49a8-aaa1-6d5aa7b920f2-combined-ca-bundle\") pod \"nova-cell1-cell-mapping-ptcx4\" (UID: \"b72434ac-6b58-49a8-aaa1-6d5aa7b920f2\") " pod="openstack-kuttl-tests/nova-cell1-cell-mapping-ptcx4" Dec 03 11:53:23 crc kubenswrapper[5052]: I1203 11:53:23.365695 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-j2974\" (UniqueName: \"kubernetes.io/projected/b72434ac-6b58-49a8-aaa1-6d5aa7b920f2-kube-api-access-j2974\") pod \"nova-cell1-cell-mapping-ptcx4\" (UID: \"b72434ac-6b58-49a8-aaa1-6d5aa7b920f2\") " pod="openstack-kuttl-tests/nova-cell1-cell-mapping-ptcx4" Dec 03 11:53:23 crc kubenswrapper[5052]: I1203 11:53:23.365718 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/b72434ac-6b58-49a8-aaa1-6d5aa7b920f2-config-data\") pod \"nova-cell1-cell-mapping-ptcx4\" (UID: \"b72434ac-6b58-49a8-aaa1-6d5aa7b920f2\") " pod="openstack-kuttl-tests/nova-cell1-cell-mapping-ptcx4" Dec 03 11:53:23 crc kubenswrapper[5052]: I1203 11:53:23.372865 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/b72434ac-6b58-49a8-aaa1-6d5aa7b920f2-config-data\") pod \"nova-cell1-cell-mapping-ptcx4\" (UID: \"b72434ac-6b58-49a8-aaa1-6d5aa7b920f2\") " pod="openstack-kuttl-tests/nova-cell1-cell-mapping-ptcx4" Dec 03 11:53:23 crc kubenswrapper[5052]: I1203 11:53:23.374019 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/b72434ac-6b58-49a8-aaa1-6d5aa7b920f2-scripts\") pod \"nova-cell1-cell-mapping-ptcx4\" (UID: \"b72434ac-6b58-49a8-aaa1-6d5aa7b920f2\") " pod="openstack-kuttl-tests/nova-cell1-cell-mapping-ptcx4" Dec 03 11:53:23 crc kubenswrapper[5052]: I1203 11:53:23.375078 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b72434ac-6b58-49a8-aaa1-6d5aa7b920f2-combined-ca-bundle\") pod \"nova-cell1-cell-mapping-ptcx4\" (UID: \"b72434ac-6b58-49a8-aaa1-6d5aa7b920f2\") " pod="openstack-kuttl-tests/nova-cell1-cell-mapping-ptcx4" Dec 03 11:53:23 crc kubenswrapper[5052]: I1203 11:53:23.401039 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-j2974\" (UniqueName: \"kubernetes.io/projected/b72434ac-6b58-49a8-aaa1-6d5aa7b920f2-kube-api-access-j2974\") pod \"nova-cell1-cell-mapping-ptcx4\" (UID: \"b72434ac-6b58-49a8-aaa1-6d5aa7b920f2\") " pod="openstack-kuttl-tests/nova-cell1-cell-mapping-ptcx4" Dec 03 11:53:23 crc kubenswrapper[5052]: I1203 11:53:23.429296 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-cell-mapping-ptcx4" Dec 03 11:53:23 crc kubenswrapper[5052]: I1203 11:53:23.576781 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-metadata-0" podUID="8a2597b5-e6ec-40e9-a992-ea2ee71ae759" containerName="nova-metadata-log" probeResult="failure" output="Get \"https://10.217.1.127:8775/\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Dec 03 11:53:23 crc kubenswrapper[5052]: I1203 11:53:23.577100 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-metadata-0" podUID="8a2597b5-e6ec-40e9-a992-ea2ee71ae759" containerName="nova-metadata-metadata" probeResult="failure" output="Get \"https://10.217.1.127:8775/\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Dec 03 11:53:24 crc kubenswrapper[5052]: I1203 11:53:24.088533 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-marketplace/redhat-operators-cjtc5" Dec 03 11:53:24 crc kubenswrapper[5052]: I1203 11:53:24.088780 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openshift-marketplace/redhat-operators-cjtc5" Dec 03 11:53:24 crc kubenswrapper[5052]: I1203 11:53:24.140666 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-cell-mapping-ptcx4"] Dec 03 11:53:24 crc kubenswrapper[5052]: W1203 11:53:24.142920 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb72434ac_6b58_49a8_aaa1_6d5aa7b920f2.slice/crio-6c481159b15060ab40e2d7e7e46b293c89bcc668ea35f05d109418fbed75144f WatchSource:0}: Error finding container 6c481159b15060ab40e2d7e7e46b293c89bcc668ea35f05d109418fbed75144f: Status 404 returned error can't find the container with id 6c481159b15060ab40e2d7e7e46b293c89bcc668ea35f05d109418fbed75144f Dec 03 11:53:24 crc kubenswrapper[5052]: I1203 11:53:24.192648 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-marketplace/redhat-operators-cjtc5" Dec 03 11:53:24 crc kubenswrapper[5052]: I1203 11:53:24.373273 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-cell-mapping-ptcx4" event={"ID":"b72434ac-6b58-49a8-aaa1-6d5aa7b920f2","Type":"ContainerStarted","Data":"6c481159b15060ab40e2d7e7e46b293c89bcc668ea35f05d109418fbed75144f"} Dec 03 11:53:24 crc kubenswrapper[5052]: I1203 11:53:24.438302 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-marketplace/redhat-operators-cjtc5" Dec 03 11:53:24 crc kubenswrapper[5052]: I1203 11:53:24.497039 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openshift-marketplace/redhat-operators-cjtc5"] Dec 03 11:53:25 crc kubenswrapper[5052]: I1203 11:53:25.393892 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-cell-mapping-ptcx4" event={"ID":"b72434ac-6b58-49a8-aaa1-6d5aa7b920f2","Type":"ContainerStarted","Data":"45b01a8f4fda9b4f932ead3e127a5cc89388e307cc160b1e9e3f45111d4ef6b0"} Dec 03 11:53:25 crc kubenswrapper[5052]: I1203 11:53:25.414735 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-cell1-cell-mapping-ptcx4" podStartSLOduration=2.414716236 podStartE2EDuration="2.414716236s" podCreationTimestamp="2025-12-03 11:53:23 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:53:25.412570575 +0000 UTC m=+1996.000812339" watchObservedRunningTime="2025-12-03 11:53:25.414716236 +0000 UTC m=+1996.002958000" Dec 03 11:53:26 crc kubenswrapper[5052]: I1203 11:53:26.401704 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openshift-marketplace/redhat-operators-cjtc5" podUID="5d4ac632-b94b-4371-a605-458e4cd92f23" containerName="registry-server" containerID="cri-o://b208896c5863e9021e60fd793f2b067644ac23e34dd7d6e5cf9c008f1ce97991" gracePeriod=2 Dec 03 11:53:26 crc kubenswrapper[5052]: I1203 11:53:26.967418 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/redhat-operators-cjtc5" Dec 03 11:53:27 crc kubenswrapper[5052]: I1203 11:53:27.137263 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/5d4ac632-b94b-4371-a605-458e4cd92f23-utilities\") pod \"5d4ac632-b94b-4371-a605-458e4cd92f23\" (UID: \"5d4ac632-b94b-4371-a605-458e4cd92f23\") " Dec 03 11:53:27 crc kubenswrapper[5052]: I1203 11:53:27.138742 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/5d4ac632-b94b-4371-a605-458e4cd92f23-catalog-content\") pod \"5d4ac632-b94b-4371-a605-458e4cd92f23\" (UID: \"5d4ac632-b94b-4371-a605-458e4cd92f23\") " Dec 03 11:53:27 crc kubenswrapper[5052]: I1203 11:53:27.138872 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gntth\" (UniqueName: \"kubernetes.io/projected/5d4ac632-b94b-4371-a605-458e4cd92f23-kube-api-access-gntth\") pod \"5d4ac632-b94b-4371-a605-458e4cd92f23\" (UID: \"5d4ac632-b94b-4371-a605-458e4cd92f23\") " Dec 03 11:53:27 crc kubenswrapper[5052]: I1203 11:53:27.139255 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5d4ac632-b94b-4371-a605-458e4cd92f23-utilities" (OuterVolumeSpecName: "utilities") pod "5d4ac632-b94b-4371-a605-458e4cd92f23" (UID: "5d4ac632-b94b-4371-a605-458e4cd92f23"). InnerVolumeSpecName "utilities". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:53:27 crc kubenswrapper[5052]: I1203 11:53:27.144369 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5d4ac632-b94b-4371-a605-458e4cd92f23-kube-api-access-gntth" (OuterVolumeSpecName: "kube-api-access-gntth") pod "5d4ac632-b94b-4371-a605-458e4cd92f23" (UID: "5d4ac632-b94b-4371-a605-458e4cd92f23"). InnerVolumeSpecName "kube-api-access-gntth". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:53:27 crc kubenswrapper[5052]: I1203 11:53:27.227266 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5d4ac632-b94b-4371-a605-458e4cd92f23-catalog-content" (OuterVolumeSpecName: "catalog-content") pod "5d4ac632-b94b-4371-a605-458e4cd92f23" (UID: "5d4ac632-b94b-4371-a605-458e4cd92f23"). InnerVolumeSpecName "catalog-content". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:53:27 crc kubenswrapper[5052]: I1203 11:53:27.240037 5052 reconciler_common.go:293] "Volume detached for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/5d4ac632-b94b-4371-a605-458e4cd92f23-utilities\") on node \"crc\" DevicePath \"\"" Dec 03 11:53:27 crc kubenswrapper[5052]: I1203 11:53:27.240265 5052 reconciler_common.go:293] "Volume detached for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/5d4ac632-b94b-4371-a605-458e4cd92f23-catalog-content\") on node \"crc\" DevicePath \"\"" Dec 03 11:53:27 crc kubenswrapper[5052]: I1203 11:53:27.240343 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-gntth\" (UniqueName: \"kubernetes.io/projected/5d4ac632-b94b-4371-a605-458e4cd92f23-kube-api-access-gntth\") on node \"crc\" DevicePath \"\"" Dec 03 11:53:27 crc kubenswrapper[5052]: I1203 11:53:27.413592 5052 generic.go:334] "Generic (PLEG): container finished" podID="5d4ac632-b94b-4371-a605-458e4cd92f23" containerID="b208896c5863e9021e60fd793f2b067644ac23e34dd7d6e5cf9c008f1ce97991" exitCode=0 Dec 03 11:53:27 crc kubenswrapper[5052]: I1203 11:53:27.413648 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/redhat-operators-cjtc5" Dec 03 11:53:27 crc kubenswrapper[5052]: I1203 11:53:27.415017 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-operators-cjtc5" event={"ID":"5d4ac632-b94b-4371-a605-458e4cd92f23","Type":"ContainerDied","Data":"b208896c5863e9021e60fd793f2b067644ac23e34dd7d6e5cf9c008f1ce97991"} Dec 03 11:53:27 crc kubenswrapper[5052]: I1203 11:53:27.415203 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-operators-cjtc5" event={"ID":"5d4ac632-b94b-4371-a605-458e4cd92f23","Type":"ContainerDied","Data":"16e72209bded8576b920281b541b9c2b1dbc91083750dce94a6f5b331d751672"} Dec 03 11:53:27 crc kubenswrapper[5052]: I1203 11:53:27.415374 5052 scope.go:117] "RemoveContainer" containerID="b208896c5863e9021e60fd793f2b067644ac23e34dd7d6e5cf9c008f1ce97991" Dec 03 11:53:27 crc kubenswrapper[5052]: I1203 11:53:27.453023 5052 scope.go:117] "RemoveContainer" containerID="415388246ab7884c9c03ffa53307c26fad0811817bdf7ab961534d37b83287cd" Dec 03 11:53:27 crc kubenswrapper[5052]: I1203 11:53:27.456618 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openshift-marketplace/redhat-operators-cjtc5"] Dec 03 11:53:27 crc kubenswrapper[5052]: I1203 11:53:27.465019 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openshift-marketplace/redhat-operators-cjtc5"] Dec 03 11:53:27 crc kubenswrapper[5052]: I1203 11:53:27.479077 5052 scope.go:117] "RemoveContainer" containerID="9a0b11c711251ed32c98116acb7e26a2fe5a715ace90e8fc1c90d87db3a20482" Dec 03 11:53:27 crc kubenswrapper[5052]: I1203 11:53:27.519469 5052 scope.go:117] "RemoveContainer" containerID="b208896c5863e9021e60fd793f2b067644ac23e34dd7d6e5cf9c008f1ce97991" Dec 03 11:53:27 crc kubenswrapper[5052]: E1203 11:53:27.519893 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b208896c5863e9021e60fd793f2b067644ac23e34dd7d6e5cf9c008f1ce97991\": container with ID starting with b208896c5863e9021e60fd793f2b067644ac23e34dd7d6e5cf9c008f1ce97991 not found: ID does not exist" containerID="b208896c5863e9021e60fd793f2b067644ac23e34dd7d6e5cf9c008f1ce97991" Dec 03 11:53:27 crc kubenswrapper[5052]: I1203 11:53:27.519921 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b208896c5863e9021e60fd793f2b067644ac23e34dd7d6e5cf9c008f1ce97991"} err="failed to get container status \"b208896c5863e9021e60fd793f2b067644ac23e34dd7d6e5cf9c008f1ce97991\": rpc error: code = NotFound desc = could not find container \"b208896c5863e9021e60fd793f2b067644ac23e34dd7d6e5cf9c008f1ce97991\": container with ID starting with b208896c5863e9021e60fd793f2b067644ac23e34dd7d6e5cf9c008f1ce97991 not found: ID does not exist" Dec 03 11:53:27 crc kubenswrapper[5052]: I1203 11:53:27.519942 5052 scope.go:117] "RemoveContainer" containerID="415388246ab7884c9c03ffa53307c26fad0811817bdf7ab961534d37b83287cd" Dec 03 11:53:27 crc kubenswrapper[5052]: E1203 11:53:27.520247 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"415388246ab7884c9c03ffa53307c26fad0811817bdf7ab961534d37b83287cd\": container with ID starting with 415388246ab7884c9c03ffa53307c26fad0811817bdf7ab961534d37b83287cd not found: ID does not exist" containerID="415388246ab7884c9c03ffa53307c26fad0811817bdf7ab961534d37b83287cd" Dec 03 11:53:27 crc kubenswrapper[5052]: I1203 11:53:27.520291 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"415388246ab7884c9c03ffa53307c26fad0811817bdf7ab961534d37b83287cd"} err="failed to get container status \"415388246ab7884c9c03ffa53307c26fad0811817bdf7ab961534d37b83287cd\": rpc error: code = NotFound desc = could not find container \"415388246ab7884c9c03ffa53307c26fad0811817bdf7ab961534d37b83287cd\": container with ID starting with 415388246ab7884c9c03ffa53307c26fad0811817bdf7ab961534d37b83287cd not found: ID does not exist" Dec 03 11:53:27 crc kubenswrapper[5052]: I1203 11:53:27.520321 5052 scope.go:117] "RemoveContainer" containerID="9a0b11c711251ed32c98116acb7e26a2fe5a715ace90e8fc1c90d87db3a20482" Dec 03 11:53:27 crc kubenswrapper[5052]: E1203 11:53:27.520794 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9a0b11c711251ed32c98116acb7e26a2fe5a715ace90e8fc1c90d87db3a20482\": container with ID starting with 9a0b11c711251ed32c98116acb7e26a2fe5a715ace90e8fc1c90d87db3a20482 not found: ID does not exist" containerID="9a0b11c711251ed32c98116acb7e26a2fe5a715ace90e8fc1c90d87db3a20482" Dec 03 11:53:27 crc kubenswrapper[5052]: I1203 11:53:27.520815 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9a0b11c711251ed32c98116acb7e26a2fe5a715ace90e8fc1c90d87db3a20482"} err="failed to get container status \"9a0b11c711251ed32c98116acb7e26a2fe5a715ace90e8fc1c90d87db3a20482\": rpc error: code = NotFound desc = could not find container \"9a0b11c711251ed32c98116acb7e26a2fe5a715ace90e8fc1c90d87db3a20482\": container with ID starting with 9a0b11c711251ed32c98116acb7e26a2fe5a715ace90e8fc1c90d87db3a20482 not found: ID does not exist" Dec 03 11:53:27 crc kubenswrapper[5052]: I1203 11:53:27.685579 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5d4ac632-b94b-4371-a605-458e4cd92f23" path="/var/lib/kubelet/pods/5d4ac632-b94b-4371-a605-458e4cd92f23/volumes" Dec 03 11:53:27 crc kubenswrapper[5052]: I1203 11:53:27.688692 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:53:27 crc kubenswrapper[5052]: I1203 11:53:27.707158 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:53:27 crc kubenswrapper[5052]: I1203 11:53:27.707222 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:53:27 crc kubenswrapper[5052]: I1203 11:53:27.719985 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:53:28 crc kubenswrapper[5052]: I1203 11:53:28.460797 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:53:28 crc kubenswrapper[5052]: I1203 11:53:28.789859 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-api-0" podUID="9a82d4a4-d01b-475f-87ac-c7a3b8f353a6" containerName="nova-api-api" probeResult="failure" output="Get \"http://10.217.1.131:8774/\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Dec 03 11:53:28 crc kubenswrapper[5052]: I1203 11:53:28.789932 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-api-0" podUID="9a82d4a4-d01b-475f-87ac-c7a3b8f353a6" containerName="nova-api-log" probeResult="failure" output="Get \"http://10.217.1.131:8774/\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Dec 03 11:53:29 crc kubenswrapper[5052]: I1203 11:53:29.379266 5052 patch_prober.go:28] interesting pod/machine-config-daemon-74bj6 container/machine-config-daemon namespace/openshift-machine-config-operator: Liveness probe status=failure output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" start-of-body= Dec 03 11:53:29 crc kubenswrapper[5052]: I1203 11:53:29.379316 5052 prober.go:107] "Probe failed" probeType="Liveness" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" containerName="machine-config-daemon" probeResult="failure" output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" Dec 03 11:53:30 crc kubenswrapper[5052]: I1203 11:53:30.450803 5052 generic.go:334] "Generic (PLEG): container finished" podID="b72434ac-6b58-49a8-aaa1-6d5aa7b920f2" containerID="45b01a8f4fda9b4f932ead3e127a5cc89388e307cc160b1e9e3f45111d4ef6b0" exitCode=0 Dec 03 11:53:30 crc kubenswrapper[5052]: I1203 11:53:30.451116 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-cell-mapping-ptcx4" event={"ID":"b72434ac-6b58-49a8-aaa1-6d5aa7b920f2","Type":"ContainerDied","Data":"45b01a8f4fda9b4f932ead3e127a5cc89388e307cc160b1e9e3f45111d4ef6b0"} Dec 03 11:53:31 crc kubenswrapper[5052]: I1203 11:53:31.832923 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-cell-mapping-ptcx4" Dec 03 11:53:32 crc kubenswrapper[5052]: I1203 11:53:32.025735 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b72434ac-6b58-49a8-aaa1-6d5aa7b920f2-combined-ca-bundle\") pod \"b72434ac-6b58-49a8-aaa1-6d5aa7b920f2\" (UID: \"b72434ac-6b58-49a8-aaa1-6d5aa7b920f2\") " Dec 03 11:53:32 crc kubenswrapper[5052]: I1203 11:53:32.025875 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/b72434ac-6b58-49a8-aaa1-6d5aa7b920f2-config-data\") pod \"b72434ac-6b58-49a8-aaa1-6d5aa7b920f2\" (UID: \"b72434ac-6b58-49a8-aaa1-6d5aa7b920f2\") " Dec 03 11:53:32 crc kubenswrapper[5052]: I1203 11:53:32.025955 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/b72434ac-6b58-49a8-aaa1-6d5aa7b920f2-scripts\") pod \"b72434ac-6b58-49a8-aaa1-6d5aa7b920f2\" (UID: \"b72434ac-6b58-49a8-aaa1-6d5aa7b920f2\") " Dec 03 11:53:32 crc kubenswrapper[5052]: I1203 11:53:32.026059 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j2974\" (UniqueName: \"kubernetes.io/projected/b72434ac-6b58-49a8-aaa1-6d5aa7b920f2-kube-api-access-j2974\") pod \"b72434ac-6b58-49a8-aaa1-6d5aa7b920f2\" (UID: \"b72434ac-6b58-49a8-aaa1-6d5aa7b920f2\") " Dec 03 11:53:32 crc kubenswrapper[5052]: I1203 11:53:32.031749 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b72434ac-6b58-49a8-aaa1-6d5aa7b920f2-scripts" (OuterVolumeSpecName: "scripts") pod "b72434ac-6b58-49a8-aaa1-6d5aa7b920f2" (UID: "b72434ac-6b58-49a8-aaa1-6d5aa7b920f2"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:53:32 crc kubenswrapper[5052]: I1203 11:53:32.040908 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b72434ac-6b58-49a8-aaa1-6d5aa7b920f2-kube-api-access-j2974" (OuterVolumeSpecName: "kube-api-access-j2974") pod "b72434ac-6b58-49a8-aaa1-6d5aa7b920f2" (UID: "b72434ac-6b58-49a8-aaa1-6d5aa7b920f2"). InnerVolumeSpecName "kube-api-access-j2974". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:53:32 crc kubenswrapper[5052]: I1203 11:53:32.059137 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b72434ac-6b58-49a8-aaa1-6d5aa7b920f2-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "b72434ac-6b58-49a8-aaa1-6d5aa7b920f2" (UID: "b72434ac-6b58-49a8-aaa1-6d5aa7b920f2"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:53:32 crc kubenswrapper[5052]: I1203 11:53:32.060547 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b72434ac-6b58-49a8-aaa1-6d5aa7b920f2-config-data" (OuterVolumeSpecName: "config-data") pod "b72434ac-6b58-49a8-aaa1-6d5aa7b920f2" (UID: "b72434ac-6b58-49a8-aaa1-6d5aa7b920f2"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:53:32 crc kubenswrapper[5052]: I1203 11:53:32.129082 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b72434ac-6b58-49a8-aaa1-6d5aa7b920f2-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:53:32 crc kubenswrapper[5052]: I1203 11:53:32.129119 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/b72434ac-6b58-49a8-aaa1-6d5aa7b920f2-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:53:32 crc kubenswrapper[5052]: I1203 11:53:32.129129 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/b72434ac-6b58-49a8-aaa1-6d5aa7b920f2-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:53:32 crc kubenswrapper[5052]: I1203 11:53:32.129138 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-j2974\" (UniqueName: \"kubernetes.io/projected/b72434ac-6b58-49a8-aaa1-6d5aa7b920f2-kube-api-access-j2974\") on node \"crc\" DevicePath \"\"" Dec 03 11:53:32 crc kubenswrapper[5052]: I1203 11:53:32.471278 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-cell-mapping-ptcx4" event={"ID":"b72434ac-6b58-49a8-aaa1-6d5aa7b920f2","Type":"ContainerDied","Data":"6c481159b15060ab40e2d7e7e46b293c89bcc668ea35f05d109418fbed75144f"} Dec 03 11:53:32 crc kubenswrapper[5052]: I1203 11:53:32.471794 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6c481159b15060ab40e2d7e7e46b293c89bcc668ea35f05d109418fbed75144f" Dec 03 11:53:32 crc kubenswrapper[5052]: I1203 11:53:32.471364 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-cell-mapping-ptcx4" Dec 03 11:53:32 crc kubenswrapper[5052]: I1203 11:53:32.568080 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:53:32 crc kubenswrapper[5052]: I1203 11:53:32.568378 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:53:32 crc kubenswrapper[5052]: I1203 11:53:32.574080 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:53:32 crc kubenswrapper[5052]: I1203 11:53:32.574598 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:53:32 crc kubenswrapper[5052]: I1203 11:53:32.755104 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 11:53:32 crc kubenswrapper[5052]: I1203 11:53:32.755346 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-api-0" podUID="9a82d4a4-d01b-475f-87ac-c7a3b8f353a6" containerName="nova-api-log" containerID="cri-o://620cafd319adde74eb2596aae4bd6d78515283619ae6e97aa04cddd040616ed9" gracePeriod=30 Dec 03 11:53:32 crc kubenswrapper[5052]: I1203 11:53:32.755421 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-api-0" podUID="9a82d4a4-d01b-475f-87ac-c7a3b8f353a6" containerName="nova-api-api" containerID="cri-o://2a583d85b5f9e4a6e90e25d7a948ae635e6695ec5893b35762c3f5cb63b8041b" gracePeriod=30 Dec 03 11:53:32 crc kubenswrapper[5052]: I1203 11:53:32.771342 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 11:53:32 crc kubenswrapper[5052]: I1203 11:53:32.771563 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-scheduler-0" podUID="c8f85677-5515-4318-9c54-66505a4c416c" containerName="nova-scheduler-scheduler" containerID="cri-o://6901b1ff77b02438397e55f7ee6fdb393d6f8e2b944de48831f9519f83e59546" gracePeriod=30 Dec 03 11:53:32 crc kubenswrapper[5052]: I1203 11:53:32.795859 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 11:53:33 crc kubenswrapper[5052]: I1203 11:53:33.482113 5052 generic.go:334] "Generic (PLEG): container finished" podID="9a82d4a4-d01b-475f-87ac-c7a3b8f353a6" containerID="620cafd319adde74eb2596aae4bd6d78515283619ae6e97aa04cddd040616ed9" exitCode=143 Dec 03 11:53:33 crc kubenswrapper[5052]: I1203 11:53:33.482260 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"9a82d4a4-d01b-475f-87ac-c7a3b8f353a6","Type":"ContainerDied","Data":"620cafd319adde74eb2596aae4bd6d78515283619ae6e97aa04cddd040616ed9"} Dec 03 11:53:34 crc kubenswrapper[5052]: I1203 11:53:34.107160 5052 scope.go:117] "RemoveContainer" containerID="92c1a0e919d06f22f9f6723e6dddde204ff33caafc0c8594584022f38ab81e75" Dec 03 11:53:34 crc kubenswrapper[5052]: I1203 11:53:34.133184 5052 scope.go:117] "RemoveContainer" containerID="19fe01d36f124c6e5b005cff9acb3ba773fdc1a82d383392d6ca61a3276e293b" Dec 03 11:53:34 crc kubenswrapper[5052]: I1203 11:53:34.175858 5052 scope.go:117] "RemoveContainer" containerID="d29340e2b19190d615ecfc711e5c8befec183416c40af1d92bde9b4447a33954" Dec 03 11:53:34 crc kubenswrapper[5052]: I1203 11:53:34.225011 5052 scope.go:117] "RemoveContainer" containerID="92cc714d81001cf63bf9236d9304b62df8d8ba4cf9d19b8376165ae7a9509d8a" Dec 03 11:53:34 crc kubenswrapper[5052]: I1203 11:53:34.278940 5052 scope.go:117] "RemoveContainer" containerID="12cedf118f41103f94919b8993790529cc71a18a79a829fcc52ee74432f113ee" Dec 03 11:53:34 crc kubenswrapper[5052]: I1203 11:53:34.316757 5052 scope.go:117] "RemoveContainer" containerID="c1f0fb7eccffe8697552aa5a47d51d47b7e88244cbd383a6c79b3d22fc46417d" Dec 03 11:53:34 crc kubenswrapper[5052]: I1203 11:53:34.492833 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-metadata-0" podUID="8a2597b5-e6ec-40e9-a992-ea2ee71ae759" containerName="nova-metadata-log" containerID="cri-o://e993f2c99fe8426fc231da418041c0bcb56e7d6a81c4f52c27e7ad9249e1bea6" gracePeriod=30 Dec 03 11:53:34 crc kubenswrapper[5052]: I1203 11:53:34.492847 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-metadata-0" podUID="8a2597b5-e6ec-40e9-a992-ea2ee71ae759" containerName="nova-metadata-metadata" containerID="cri-o://8f647efbd4f4f64fedeccae7318c4ecd9cfbd4fce9fa538a4df1495f8242c11e" gracePeriod=30 Dec 03 11:53:35 crc kubenswrapper[5052]: I1203 11:53:35.506917 5052 generic.go:334] "Generic (PLEG): container finished" podID="8a2597b5-e6ec-40e9-a992-ea2ee71ae759" containerID="e993f2c99fe8426fc231da418041c0bcb56e7d6a81c4f52c27e7ad9249e1bea6" exitCode=143 Dec 03 11:53:35 crc kubenswrapper[5052]: I1203 11:53:35.507012 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"8a2597b5-e6ec-40e9-a992-ea2ee71ae759","Type":"ContainerDied","Data":"e993f2c99fe8426fc231da418041c0bcb56e7d6a81c4f52c27e7ad9249e1bea6"} Dec 03 11:53:35 crc kubenswrapper[5052]: I1203 11:53:35.509052 5052 generic.go:334] "Generic (PLEG): container finished" podID="c8f85677-5515-4318-9c54-66505a4c416c" containerID="6901b1ff77b02438397e55f7ee6fdb393d6f8e2b944de48831f9519f83e59546" exitCode=0 Dec 03 11:53:35 crc kubenswrapper[5052]: I1203 11:53:35.509116 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"c8f85677-5515-4318-9c54-66505a4c416c","Type":"ContainerDied","Data":"6901b1ff77b02438397e55f7ee6fdb393d6f8e2b944de48831f9519f83e59546"} Dec 03 11:53:35 crc kubenswrapper[5052]: I1203 11:53:35.843853 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:53:36 crc kubenswrapper[5052]: I1203 11:53:36.012026 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/c8f85677-5515-4318-9c54-66505a4c416c-config-data\") pod \"c8f85677-5515-4318-9c54-66505a4c416c\" (UID: \"c8f85677-5515-4318-9c54-66505a4c416c\") " Dec 03 11:53:36 crc kubenswrapper[5052]: I1203 11:53:36.012078 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c8f85677-5515-4318-9c54-66505a4c416c-combined-ca-bundle\") pod \"c8f85677-5515-4318-9c54-66505a4c416c\" (UID: \"c8f85677-5515-4318-9c54-66505a4c416c\") " Dec 03 11:53:36 crc kubenswrapper[5052]: I1203 11:53:36.012231 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g6g9g\" (UniqueName: \"kubernetes.io/projected/c8f85677-5515-4318-9c54-66505a4c416c-kube-api-access-g6g9g\") pod \"c8f85677-5515-4318-9c54-66505a4c416c\" (UID: \"c8f85677-5515-4318-9c54-66505a4c416c\") " Dec 03 11:53:36 crc kubenswrapper[5052]: I1203 11:53:36.019262 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c8f85677-5515-4318-9c54-66505a4c416c-kube-api-access-g6g9g" (OuterVolumeSpecName: "kube-api-access-g6g9g") pod "c8f85677-5515-4318-9c54-66505a4c416c" (UID: "c8f85677-5515-4318-9c54-66505a4c416c"). InnerVolumeSpecName "kube-api-access-g6g9g". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:53:36 crc kubenswrapper[5052]: I1203 11:53:36.039460 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c8f85677-5515-4318-9c54-66505a4c416c-config-data" (OuterVolumeSpecName: "config-data") pod "c8f85677-5515-4318-9c54-66505a4c416c" (UID: "c8f85677-5515-4318-9c54-66505a4c416c"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:53:36 crc kubenswrapper[5052]: I1203 11:53:36.041553 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c8f85677-5515-4318-9c54-66505a4c416c-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "c8f85677-5515-4318-9c54-66505a4c416c" (UID: "c8f85677-5515-4318-9c54-66505a4c416c"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:53:36 crc kubenswrapper[5052]: I1203 11:53:36.114690 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/c8f85677-5515-4318-9c54-66505a4c416c-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:53:36 crc kubenswrapper[5052]: I1203 11:53:36.114979 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c8f85677-5515-4318-9c54-66505a4c416c-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:53:36 crc kubenswrapper[5052]: I1203 11:53:36.114992 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-g6g9g\" (UniqueName: \"kubernetes.io/projected/c8f85677-5515-4318-9c54-66505a4c416c-kube-api-access-g6g9g\") on node \"crc\" DevicePath \"\"" Dec 03 11:53:36 crc kubenswrapper[5052]: I1203 11:53:36.398433 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:53:36 crc kubenswrapper[5052]: I1203 11:53:36.419846 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2x85h\" (UniqueName: \"kubernetes.io/projected/9a82d4a4-d01b-475f-87ac-c7a3b8f353a6-kube-api-access-2x85h\") pod \"9a82d4a4-d01b-475f-87ac-c7a3b8f353a6\" (UID: \"9a82d4a4-d01b-475f-87ac-c7a3b8f353a6\") " Dec 03 11:53:36 crc kubenswrapper[5052]: I1203 11:53:36.419979 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/9a82d4a4-d01b-475f-87ac-c7a3b8f353a6-logs\") pod \"9a82d4a4-d01b-475f-87ac-c7a3b8f353a6\" (UID: \"9a82d4a4-d01b-475f-87ac-c7a3b8f353a6\") " Dec 03 11:53:36 crc kubenswrapper[5052]: I1203 11:53:36.420028 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/9a82d4a4-d01b-475f-87ac-c7a3b8f353a6-config-data\") pod \"9a82d4a4-d01b-475f-87ac-c7a3b8f353a6\" (UID: \"9a82d4a4-d01b-475f-87ac-c7a3b8f353a6\") " Dec 03 11:53:36 crc kubenswrapper[5052]: I1203 11:53:36.420048 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9a82d4a4-d01b-475f-87ac-c7a3b8f353a6-combined-ca-bundle\") pod \"9a82d4a4-d01b-475f-87ac-c7a3b8f353a6\" (UID: \"9a82d4a4-d01b-475f-87ac-c7a3b8f353a6\") " Dec 03 11:53:36 crc kubenswrapper[5052]: I1203 11:53:36.420535 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9a82d4a4-d01b-475f-87ac-c7a3b8f353a6-logs" (OuterVolumeSpecName: "logs") pod "9a82d4a4-d01b-475f-87ac-c7a3b8f353a6" (UID: "9a82d4a4-d01b-475f-87ac-c7a3b8f353a6"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:53:36 crc kubenswrapper[5052]: I1203 11:53:36.424351 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9a82d4a4-d01b-475f-87ac-c7a3b8f353a6-kube-api-access-2x85h" (OuterVolumeSpecName: "kube-api-access-2x85h") pod "9a82d4a4-d01b-475f-87ac-c7a3b8f353a6" (UID: "9a82d4a4-d01b-475f-87ac-c7a3b8f353a6"). InnerVolumeSpecName "kube-api-access-2x85h". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:53:36 crc kubenswrapper[5052]: I1203 11:53:36.445667 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9a82d4a4-d01b-475f-87ac-c7a3b8f353a6-config-data" (OuterVolumeSpecName: "config-data") pod "9a82d4a4-d01b-475f-87ac-c7a3b8f353a6" (UID: "9a82d4a4-d01b-475f-87ac-c7a3b8f353a6"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:53:36 crc kubenswrapper[5052]: I1203 11:53:36.454178 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9a82d4a4-d01b-475f-87ac-c7a3b8f353a6-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "9a82d4a4-d01b-475f-87ac-c7a3b8f353a6" (UID: "9a82d4a4-d01b-475f-87ac-c7a3b8f353a6"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:53:36 crc kubenswrapper[5052]: I1203 11:53:36.517702 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"c8f85677-5515-4318-9c54-66505a4c416c","Type":"ContainerDied","Data":"e4c0fd0cab706599a087e52a95b0ff2b391a487409fb90a655bce6a601ddd9c5"} Dec 03 11:53:36 crc kubenswrapper[5052]: I1203 11:53:36.518764 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:53:36 crc kubenswrapper[5052]: I1203 11:53:36.519087 5052 scope.go:117] "RemoveContainer" containerID="6901b1ff77b02438397e55f7ee6fdb393d6f8e2b944de48831f9519f83e59546" Dec 03 11:53:36 crc kubenswrapper[5052]: I1203 11:53:36.521936 5052 generic.go:334] "Generic (PLEG): container finished" podID="9a82d4a4-d01b-475f-87ac-c7a3b8f353a6" containerID="2a583d85b5f9e4a6e90e25d7a948ae635e6695ec5893b35762c3f5cb63b8041b" exitCode=0 Dec 03 11:53:36 crc kubenswrapper[5052]: I1203 11:53:36.521971 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"9a82d4a4-d01b-475f-87ac-c7a3b8f353a6","Type":"ContainerDied","Data":"2a583d85b5f9e4a6e90e25d7a948ae635e6695ec5893b35762c3f5cb63b8041b"} Dec 03 11:53:36 crc kubenswrapper[5052]: I1203 11:53:36.521994 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"9a82d4a4-d01b-475f-87ac-c7a3b8f353a6","Type":"ContainerDied","Data":"7881219e0f5dd3f01b64e49672bb3184d2cf074949e6d28ccbc8127263654aea"} Dec 03 11:53:36 crc kubenswrapper[5052]: I1203 11:53:36.522059 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:53:36 crc kubenswrapper[5052]: I1203 11:53:36.526260 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/9a82d4a4-d01b-475f-87ac-c7a3b8f353a6-logs\") on node \"crc\" DevicePath \"\"" Dec 03 11:53:36 crc kubenswrapper[5052]: I1203 11:53:36.526284 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/9a82d4a4-d01b-475f-87ac-c7a3b8f353a6-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:53:36 crc kubenswrapper[5052]: I1203 11:53:36.526297 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9a82d4a4-d01b-475f-87ac-c7a3b8f353a6-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:53:36 crc kubenswrapper[5052]: I1203 11:53:36.526309 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-2x85h\" (UniqueName: \"kubernetes.io/projected/9a82d4a4-d01b-475f-87ac-c7a3b8f353a6-kube-api-access-2x85h\") on node \"crc\" DevicePath \"\"" Dec 03 11:53:36 crc kubenswrapper[5052]: I1203 11:53:36.549214 5052 scope.go:117] "RemoveContainer" containerID="2a583d85b5f9e4a6e90e25d7a948ae635e6695ec5893b35762c3f5cb63b8041b" Dec 03 11:53:36 crc kubenswrapper[5052]: I1203 11:53:36.585112 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 11:53:36 crc kubenswrapper[5052]: I1203 11:53:36.596628 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 11:53:36 crc kubenswrapper[5052]: I1203 11:53:36.606402 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 11:53:36 crc kubenswrapper[5052]: I1203 11:53:36.608976 5052 scope.go:117] "RemoveContainer" containerID="620cafd319adde74eb2596aae4bd6d78515283619ae6e97aa04cddd040616ed9" Dec 03 11:53:36 crc kubenswrapper[5052]: I1203 11:53:36.619934 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 11:53:36 crc kubenswrapper[5052]: E1203 11:53:36.620349 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="5d4ac632-b94b-4371-a605-458e4cd92f23" containerName="registry-server" Dec 03 11:53:36 crc kubenswrapper[5052]: I1203 11:53:36.620366 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="5d4ac632-b94b-4371-a605-458e4cd92f23" containerName="registry-server" Dec 03 11:53:36 crc kubenswrapper[5052]: E1203 11:53:36.620388 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="5d4ac632-b94b-4371-a605-458e4cd92f23" containerName="extract-content" Dec 03 11:53:36 crc kubenswrapper[5052]: I1203 11:53:36.620396 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="5d4ac632-b94b-4371-a605-458e4cd92f23" containerName="extract-content" Dec 03 11:53:36 crc kubenswrapper[5052]: E1203 11:53:36.620413 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="5d4ac632-b94b-4371-a605-458e4cd92f23" containerName="extract-utilities" Dec 03 11:53:36 crc kubenswrapper[5052]: I1203 11:53:36.620420 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="5d4ac632-b94b-4371-a605-458e4cd92f23" containerName="extract-utilities" Dec 03 11:53:36 crc kubenswrapper[5052]: E1203 11:53:36.620429 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="b72434ac-6b58-49a8-aaa1-6d5aa7b920f2" containerName="nova-manage" Dec 03 11:53:36 crc kubenswrapper[5052]: I1203 11:53:36.620435 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="b72434ac-6b58-49a8-aaa1-6d5aa7b920f2" containerName="nova-manage" Dec 03 11:53:36 crc kubenswrapper[5052]: E1203 11:53:36.620445 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="9a82d4a4-d01b-475f-87ac-c7a3b8f353a6" containerName="nova-api-log" Dec 03 11:53:36 crc kubenswrapper[5052]: I1203 11:53:36.620451 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="9a82d4a4-d01b-475f-87ac-c7a3b8f353a6" containerName="nova-api-log" Dec 03 11:53:36 crc kubenswrapper[5052]: E1203 11:53:36.620471 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="c8f85677-5515-4318-9c54-66505a4c416c" containerName="nova-scheduler-scheduler" Dec 03 11:53:36 crc kubenswrapper[5052]: I1203 11:53:36.620477 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="c8f85677-5515-4318-9c54-66505a4c416c" containerName="nova-scheduler-scheduler" Dec 03 11:53:36 crc kubenswrapper[5052]: E1203 11:53:36.620487 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="9a82d4a4-d01b-475f-87ac-c7a3b8f353a6" containerName="nova-api-api" Dec 03 11:53:36 crc kubenswrapper[5052]: I1203 11:53:36.620492 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="9a82d4a4-d01b-475f-87ac-c7a3b8f353a6" containerName="nova-api-api" Dec 03 11:53:36 crc kubenswrapper[5052]: I1203 11:53:36.620694 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="5d4ac632-b94b-4371-a605-458e4cd92f23" containerName="registry-server" Dec 03 11:53:36 crc kubenswrapper[5052]: I1203 11:53:36.620711 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="c8f85677-5515-4318-9c54-66505a4c416c" containerName="nova-scheduler-scheduler" Dec 03 11:53:36 crc kubenswrapper[5052]: I1203 11:53:36.620729 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="9a82d4a4-d01b-475f-87ac-c7a3b8f353a6" containerName="nova-api-log" Dec 03 11:53:36 crc kubenswrapper[5052]: I1203 11:53:36.620736 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="b72434ac-6b58-49a8-aaa1-6d5aa7b920f2" containerName="nova-manage" Dec 03 11:53:36 crc kubenswrapper[5052]: I1203 11:53:36.620745 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="9a82d4a4-d01b-475f-87ac-c7a3b8f353a6" containerName="nova-api-api" Dec 03 11:53:36 crc kubenswrapper[5052]: I1203 11:53:36.621341 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:53:36 crc kubenswrapper[5052]: I1203 11:53:36.624268 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-scheduler-config-data" Dec 03 11:53:36 crc kubenswrapper[5052]: I1203 11:53:36.627589 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/ec16e8c3-9750-4475-a671-17b0c16826c8-config-data\") pod \"nova-scheduler-0\" (UID: \"ec16e8c3-9750-4475-a671-17b0c16826c8\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:53:36 crc kubenswrapper[5052]: I1203 11:53:36.627665 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ec16e8c3-9750-4475-a671-17b0c16826c8-combined-ca-bundle\") pod \"nova-scheduler-0\" (UID: \"ec16e8c3-9750-4475-a671-17b0c16826c8\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:53:36 crc kubenswrapper[5052]: I1203 11:53:36.627824 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q8jkt\" (UniqueName: \"kubernetes.io/projected/ec16e8c3-9750-4475-a671-17b0c16826c8-kube-api-access-q8jkt\") pod \"nova-scheduler-0\" (UID: \"ec16e8c3-9750-4475-a671-17b0c16826c8\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:53:36 crc kubenswrapper[5052]: I1203 11:53:36.629543 5052 scope.go:117] "RemoveContainer" containerID="2a583d85b5f9e4a6e90e25d7a948ae635e6695ec5893b35762c3f5cb63b8041b" Dec 03 11:53:36 crc kubenswrapper[5052]: E1203 11:53:36.629953 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2a583d85b5f9e4a6e90e25d7a948ae635e6695ec5893b35762c3f5cb63b8041b\": container with ID starting with 2a583d85b5f9e4a6e90e25d7a948ae635e6695ec5893b35762c3f5cb63b8041b not found: ID does not exist" containerID="2a583d85b5f9e4a6e90e25d7a948ae635e6695ec5893b35762c3f5cb63b8041b" Dec 03 11:53:36 crc kubenswrapper[5052]: I1203 11:53:36.629982 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2a583d85b5f9e4a6e90e25d7a948ae635e6695ec5893b35762c3f5cb63b8041b"} err="failed to get container status \"2a583d85b5f9e4a6e90e25d7a948ae635e6695ec5893b35762c3f5cb63b8041b\": rpc error: code = NotFound desc = could not find container \"2a583d85b5f9e4a6e90e25d7a948ae635e6695ec5893b35762c3f5cb63b8041b\": container with ID starting with 2a583d85b5f9e4a6e90e25d7a948ae635e6695ec5893b35762c3f5cb63b8041b not found: ID does not exist" Dec 03 11:53:36 crc kubenswrapper[5052]: I1203 11:53:36.630002 5052 scope.go:117] "RemoveContainer" containerID="620cafd319adde74eb2596aae4bd6d78515283619ae6e97aa04cddd040616ed9" Dec 03 11:53:36 crc kubenswrapper[5052]: E1203 11:53:36.630236 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"620cafd319adde74eb2596aae4bd6d78515283619ae6e97aa04cddd040616ed9\": container with ID starting with 620cafd319adde74eb2596aae4bd6d78515283619ae6e97aa04cddd040616ed9 not found: ID does not exist" containerID="620cafd319adde74eb2596aae4bd6d78515283619ae6e97aa04cddd040616ed9" Dec 03 11:53:36 crc kubenswrapper[5052]: I1203 11:53:36.630271 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"620cafd319adde74eb2596aae4bd6d78515283619ae6e97aa04cddd040616ed9"} err="failed to get container status \"620cafd319adde74eb2596aae4bd6d78515283619ae6e97aa04cddd040616ed9\": rpc error: code = NotFound desc = could not find container \"620cafd319adde74eb2596aae4bd6d78515283619ae6e97aa04cddd040616ed9\": container with ID starting with 620cafd319adde74eb2596aae4bd6d78515283619ae6e97aa04cddd040616ed9 not found: ID does not exist" Dec 03 11:53:36 crc kubenswrapper[5052]: I1203 11:53:36.637946 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 11:53:36 crc kubenswrapper[5052]: I1203 11:53:36.646847 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 11:53:36 crc kubenswrapper[5052]: I1203 11:53:36.656138 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 11:53:36 crc kubenswrapper[5052]: I1203 11:53:36.657803 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:53:36 crc kubenswrapper[5052]: I1203 11:53:36.659861 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-api-config-data" Dec 03 11:53:36 crc kubenswrapper[5052]: I1203 11:53:36.675156 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 11:53:36 crc kubenswrapper[5052]: I1203 11:53:36.730310 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/ec16e8c3-9750-4475-a671-17b0c16826c8-config-data\") pod \"nova-scheduler-0\" (UID: \"ec16e8c3-9750-4475-a671-17b0c16826c8\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:53:36 crc kubenswrapper[5052]: I1203 11:53:36.730371 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a759a8ef-882c-4eb0-8336-9aa26c80b488-config-data\") pod \"nova-api-0\" (UID: \"a759a8ef-882c-4eb0-8336-9aa26c80b488\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:53:36 crc kubenswrapper[5052]: I1203 11:53:36.730420 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a759a8ef-882c-4eb0-8336-9aa26c80b488-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"a759a8ef-882c-4eb0-8336-9aa26c80b488\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:53:36 crc kubenswrapper[5052]: I1203 11:53:36.730450 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fbnqh\" (UniqueName: \"kubernetes.io/projected/a759a8ef-882c-4eb0-8336-9aa26c80b488-kube-api-access-fbnqh\") pod \"nova-api-0\" (UID: \"a759a8ef-882c-4eb0-8336-9aa26c80b488\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:53:36 crc kubenswrapper[5052]: I1203 11:53:36.730482 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ec16e8c3-9750-4475-a671-17b0c16826c8-combined-ca-bundle\") pod \"nova-scheduler-0\" (UID: \"ec16e8c3-9750-4475-a671-17b0c16826c8\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:53:36 crc kubenswrapper[5052]: I1203 11:53:36.730630 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-q8jkt\" (UniqueName: \"kubernetes.io/projected/ec16e8c3-9750-4475-a671-17b0c16826c8-kube-api-access-q8jkt\") pod \"nova-scheduler-0\" (UID: \"ec16e8c3-9750-4475-a671-17b0c16826c8\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:53:36 crc kubenswrapper[5052]: I1203 11:53:36.730684 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/a759a8ef-882c-4eb0-8336-9aa26c80b488-logs\") pod \"nova-api-0\" (UID: \"a759a8ef-882c-4eb0-8336-9aa26c80b488\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:53:36 crc kubenswrapper[5052]: I1203 11:53:36.735209 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ec16e8c3-9750-4475-a671-17b0c16826c8-combined-ca-bundle\") pod \"nova-scheduler-0\" (UID: \"ec16e8c3-9750-4475-a671-17b0c16826c8\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:53:36 crc kubenswrapper[5052]: I1203 11:53:36.735763 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/ec16e8c3-9750-4475-a671-17b0c16826c8-config-data\") pod \"nova-scheduler-0\" (UID: \"ec16e8c3-9750-4475-a671-17b0c16826c8\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:53:36 crc kubenswrapper[5052]: I1203 11:53:36.748932 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-q8jkt\" (UniqueName: \"kubernetes.io/projected/ec16e8c3-9750-4475-a671-17b0c16826c8-kube-api-access-q8jkt\") pod \"nova-scheduler-0\" (UID: \"ec16e8c3-9750-4475-a671-17b0c16826c8\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:53:36 crc kubenswrapper[5052]: I1203 11:53:36.832722 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/a759a8ef-882c-4eb0-8336-9aa26c80b488-logs\") pod \"nova-api-0\" (UID: \"a759a8ef-882c-4eb0-8336-9aa26c80b488\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:53:36 crc kubenswrapper[5052]: I1203 11:53:36.833235 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/a759a8ef-882c-4eb0-8336-9aa26c80b488-logs\") pod \"nova-api-0\" (UID: \"a759a8ef-882c-4eb0-8336-9aa26c80b488\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:53:36 crc kubenswrapper[5052]: I1203 11:53:36.833446 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a759a8ef-882c-4eb0-8336-9aa26c80b488-config-data\") pod \"nova-api-0\" (UID: \"a759a8ef-882c-4eb0-8336-9aa26c80b488\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:53:36 crc kubenswrapper[5052]: I1203 11:53:36.833647 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a759a8ef-882c-4eb0-8336-9aa26c80b488-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"a759a8ef-882c-4eb0-8336-9aa26c80b488\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:53:36 crc kubenswrapper[5052]: I1203 11:53:36.834246 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-fbnqh\" (UniqueName: \"kubernetes.io/projected/a759a8ef-882c-4eb0-8336-9aa26c80b488-kube-api-access-fbnqh\") pod \"nova-api-0\" (UID: \"a759a8ef-882c-4eb0-8336-9aa26c80b488\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:53:36 crc kubenswrapper[5052]: I1203 11:53:36.836770 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a759a8ef-882c-4eb0-8336-9aa26c80b488-config-data\") pod \"nova-api-0\" (UID: \"a759a8ef-882c-4eb0-8336-9aa26c80b488\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:53:36 crc kubenswrapper[5052]: I1203 11:53:36.837457 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a759a8ef-882c-4eb0-8336-9aa26c80b488-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"a759a8ef-882c-4eb0-8336-9aa26c80b488\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:53:36 crc kubenswrapper[5052]: I1203 11:53:36.849761 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-fbnqh\" (UniqueName: \"kubernetes.io/projected/a759a8ef-882c-4eb0-8336-9aa26c80b488-kube-api-access-fbnqh\") pod \"nova-api-0\" (UID: \"a759a8ef-882c-4eb0-8336-9aa26c80b488\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:53:36 crc kubenswrapper[5052]: I1203 11:53:36.937577 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:53:36 crc kubenswrapper[5052]: I1203 11:53:36.974730 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:53:37 crc kubenswrapper[5052]: W1203 11:53:37.352788 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podec16e8c3_9750_4475_a671_17b0c16826c8.slice/crio-61c38ece388590278bd72d985a321f75db09a50bb828355913a9b0bdfa05b5a8 WatchSource:0}: Error finding container 61c38ece388590278bd72d985a321f75db09a50bb828355913a9b0bdfa05b5a8: Status 404 returned error can't find the container with id 61c38ece388590278bd72d985a321f75db09a50bb828355913a9b0bdfa05b5a8 Dec 03 11:53:37 crc kubenswrapper[5052]: I1203 11:53:37.353265 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 11:53:37 crc kubenswrapper[5052]: W1203 11:53:37.450526 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda759a8ef_882c_4eb0_8336_9aa26c80b488.slice/crio-2f140cc1b5185e35ab25e66348f42d4605a62b533777b9385259815203c3ebaa WatchSource:0}: Error finding container 2f140cc1b5185e35ab25e66348f42d4605a62b533777b9385259815203c3ebaa: Status 404 returned error can't find the container with id 2f140cc1b5185e35ab25e66348f42d4605a62b533777b9385259815203c3ebaa Dec 03 11:53:37 crc kubenswrapper[5052]: I1203 11:53:37.452397 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 11:53:37 crc kubenswrapper[5052]: I1203 11:53:37.538100 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"ec16e8c3-9750-4475-a671-17b0c16826c8","Type":"ContainerStarted","Data":"61c38ece388590278bd72d985a321f75db09a50bb828355913a9b0bdfa05b5a8"} Dec 03 11:53:37 crc kubenswrapper[5052]: I1203 11:53:37.542697 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"a759a8ef-882c-4eb0-8336-9aa26c80b488","Type":"ContainerStarted","Data":"2f140cc1b5185e35ab25e66348f42d4605a62b533777b9385259815203c3ebaa"} Dec 03 11:53:37 crc kubenswrapper[5052]: I1203 11:53:37.616689 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/nova-metadata-0" podUID="8a2597b5-e6ec-40e9-a992-ea2ee71ae759" containerName="nova-metadata-metadata" probeResult="failure" output="Get \"https://10.217.1.127:8775/\": read tcp 10.217.0.2:57408->10.217.1.127:8775: read: connection reset by peer" Dec 03 11:53:37 crc kubenswrapper[5052]: I1203 11:53:37.616703 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/nova-metadata-0" podUID="8a2597b5-e6ec-40e9-a992-ea2ee71ae759" containerName="nova-metadata-log" probeResult="failure" output="Get \"https://10.217.1.127:8775/\": read tcp 10.217.0.2:57424->10.217.1.127:8775: read: connection reset by peer" Dec 03 11:53:37 crc kubenswrapper[5052]: I1203 11:53:37.683543 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9a82d4a4-d01b-475f-87ac-c7a3b8f353a6" path="/var/lib/kubelet/pods/9a82d4a4-d01b-475f-87ac-c7a3b8f353a6/volumes" Dec 03 11:53:37 crc kubenswrapper[5052]: I1203 11:53:37.684128 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c8f85677-5515-4318-9c54-66505a4c416c" path="/var/lib/kubelet/pods/c8f85677-5515-4318-9c54-66505a4c416c/volumes" Dec 03 11:53:37 crc kubenswrapper[5052]: I1203 11:53:37.954725 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:53:38 crc kubenswrapper[5052]: I1203 11:53:38.053795 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8a2597b5-e6ec-40e9-a992-ea2ee71ae759-combined-ca-bundle\") pod \"8a2597b5-e6ec-40e9-a992-ea2ee71ae759\" (UID: \"8a2597b5-e6ec-40e9-a992-ea2ee71ae759\") " Dec 03 11:53:38 crc kubenswrapper[5052]: I1203 11:53:38.055150 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5zm2j\" (UniqueName: \"kubernetes.io/projected/8a2597b5-e6ec-40e9-a992-ea2ee71ae759-kube-api-access-5zm2j\") pod \"8a2597b5-e6ec-40e9-a992-ea2ee71ae759\" (UID: \"8a2597b5-e6ec-40e9-a992-ea2ee71ae759\") " Dec 03 11:53:38 crc kubenswrapper[5052]: I1203 11:53:38.055245 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/8a2597b5-e6ec-40e9-a992-ea2ee71ae759-logs\") pod \"8a2597b5-e6ec-40e9-a992-ea2ee71ae759\" (UID: \"8a2597b5-e6ec-40e9-a992-ea2ee71ae759\") " Dec 03 11:53:38 crc kubenswrapper[5052]: I1203 11:53:38.055286 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/8a2597b5-e6ec-40e9-a992-ea2ee71ae759-config-data\") pod \"8a2597b5-e6ec-40e9-a992-ea2ee71ae759\" (UID: \"8a2597b5-e6ec-40e9-a992-ea2ee71ae759\") " Dec 03 11:53:38 crc kubenswrapper[5052]: I1203 11:53:38.055474 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/8a2597b5-e6ec-40e9-a992-ea2ee71ae759-nova-metadata-tls-certs\") pod \"8a2597b5-e6ec-40e9-a992-ea2ee71ae759\" (UID: \"8a2597b5-e6ec-40e9-a992-ea2ee71ae759\") " Dec 03 11:53:38 crc kubenswrapper[5052]: I1203 11:53:38.057108 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8a2597b5-e6ec-40e9-a992-ea2ee71ae759-logs" (OuterVolumeSpecName: "logs") pod "8a2597b5-e6ec-40e9-a992-ea2ee71ae759" (UID: "8a2597b5-e6ec-40e9-a992-ea2ee71ae759"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:53:38 crc kubenswrapper[5052]: I1203 11:53:38.059228 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8a2597b5-e6ec-40e9-a992-ea2ee71ae759-kube-api-access-5zm2j" (OuterVolumeSpecName: "kube-api-access-5zm2j") pod "8a2597b5-e6ec-40e9-a992-ea2ee71ae759" (UID: "8a2597b5-e6ec-40e9-a992-ea2ee71ae759"). InnerVolumeSpecName "kube-api-access-5zm2j". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:53:38 crc kubenswrapper[5052]: I1203 11:53:38.084209 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8a2597b5-e6ec-40e9-a992-ea2ee71ae759-config-data" (OuterVolumeSpecName: "config-data") pod "8a2597b5-e6ec-40e9-a992-ea2ee71ae759" (UID: "8a2597b5-e6ec-40e9-a992-ea2ee71ae759"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:53:38 crc kubenswrapper[5052]: I1203 11:53:38.087387 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8a2597b5-e6ec-40e9-a992-ea2ee71ae759-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "8a2597b5-e6ec-40e9-a992-ea2ee71ae759" (UID: "8a2597b5-e6ec-40e9-a992-ea2ee71ae759"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:53:38 crc kubenswrapper[5052]: I1203 11:53:38.126695 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8a2597b5-e6ec-40e9-a992-ea2ee71ae759-nova-metadata-tls-certs" (OuterVolumeSpecName: "nova-metadata-tls-certs") pod "8a2597b5-e6ec-40e9-a992-ea2ee71ae759" (UID: "8a2597b5-e6ec-40e9-a992-ea2ee71ae759"). InnerVolumeSpecName "nova-metadata-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:53:38 crc kubenswrapper[5052]: I1203 11:53:38.158080 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8a2597b5-e6ec-40e9-a992-ea2ee71ae759-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:53:38 crc kubenswrapper[5052]: I1203 11:53:38.158127 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-5zm2j\" (UniqueName: \"kubernetes.io/projected/8a2597b5-e6ec-40e9-a992-ea2ee71ae759-kube-api-access-5zm2j\") on node \"crc\" DevicePath \"\"" Dec 03 11:53:38 crc kubenswrapper[5052]: I1203 11:53:38.158144 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/8a2597b5-e6ec-40e9-a992-ea2ee71ae759-logs\") on node \"crc\" DevicePath \"\"" Dec 03 11:53:38 crc kubenswrapper[5052]: I1203 11:53:38.158159 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/8a2597b5-e6ec-40e9-a992-ea2ee71ae759-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:53:38 crc kubenswrapper[5052]: I1203 11:53:38.158172 5052 reconciler_common.go:293] "Volume detached for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/8a2597b5-e6ec-40e9-a992-ea2ee71ae759-nova-metadata-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 11:53:38 crc kubenswrapper[5052]: I1203 11:53:38.554117 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"ec16e8c3-9750-4475-a671-17b0c16826c8","Type":"ContainerStarted","Data":"21459936e0c2e5ac30de5ff0dac1297b633e3badacbcf9ad455e81ffb5671c96"} Dec 03 11:53:38 crc kubenswrapper[5052]: I1203 11:53:38.556245 5052 generic.go:334] "Generic (PLEG): container finished" podID="8a2597b5-e6ec-40e9-a992-ea2ee71ae759" containerID="8f647efbd4f4f64fedeccae7318c4ecd9cfbd4fce9fa538a4df1495f8242c11e" exitCode=0 Dec 03 11:53:38 crc kubenswrapper[5052]: I1203 11:53:38.556312 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:53:38 crc kubenswrapper[5052]: I1203 11:53:38.556355 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"8a2597b5-e6ec-40e9-a992-ea2ee71ae759","Type":"ContainerDied","Data":"8f647efbd4f4f64fedeccae7318c4ecd9cfbd4fce9fa538a4df1495f8242c11e"} Dec 03 11:53:38 crc kubenswrapper[5052]: I1203 11:53:38.556386 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"8a2597b5-e6ec-40e9-a992-ea2ee71ae759","Type":"ContainerDied","Data":"0d9db8df6b44b1c24186afdd2064dc1ea8b0415d4cf50ef5fca6429bf616c7fb"} Dec 03 11:53:38 crc kubenswrapper[5052]: I1203 11:53:38.556408 5052 scope.go:117] "RemoveContainer" containerID="8f647efbd4f4f64fedeccae7318c4ecd9cfbd4fce9fa538a4df1495f8242c11e" Dec 03 11:53:38 crc kubenswrapper[5052]: I1203 11:53:38.558161 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"a759a8ef-882c-4eb0-8336-9aa26c80b488","Type":"ContainerStarted","Data":"1b0b289d84e0446d67b376b1b82446636f101d092aa2c630b25ef98f06792d0e"} Dec 03 11:53:38 crc kubenswrapper[5052]: I1203 11:53:38.558187 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"a759a8ef-882c-4eb0-8336-9aa26c80b488","Type":"ContainerStarted","Data":"8aba5a231952e5e90e8bda795fd346964cfc9876df0c0e4eb4f010f93cd471fe"} Dec 03 11:53:38 crc kubenswrapper[5052]: I1203 11:53:38.577024 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-scheduler-0" podStartSLOduration=2.577002875 podStartE2EDuration="2.577002875s" podCreationTimestamp="2025-12-03 11:53:36 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:53:38.57263693 +0000 UTC m=+2009.160878684" watchObservedRunningTime="2025-12-03 11:53:38.577002875 +0000 UTC m=+2009.165244649" Dec 03 11:53:38 crc kubenswrapper[5052]: I1203 11:53:38.582886 5052 scope.go:117] "RemoveContainer" containerID="e993f2c99fe8426fc231da418041c0bcb56e7d6a81c4f52c27e7ad9249e1bea6" Dec 03 11:53:38 crc kubenswrapper[5052]: I1203 11:53:38.601417 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-api-0" podStartSLOduration=2.601393784 podStartE2EDuration="2.601393784s" podCreationTimestamp="2025-12-03 11:53:36 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:53:38.596612607 +0000 UTC m=+2009.184854381" watchObservedRunningTime="2025-12-03 11:53:38.601393784 +0000 UTC m=+2009.189635548" Dec 03 11:53:38 crc kubenswrapper[5052]: I1203 11:53:38.610495 5052 scope.go:117] "RemoveContainer" containerID="8f647efbd4f4f64fedeccae7318c4ecd9cfbd4fce9fa538a4df1495f8242c11e" Dec 03 11:53:38 crc kubenswrapper[5052]: E1203 11:53:38.612189 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8f647efbd4f4f64fedeccae7318c4ecd9cfbd4fce9fa538a4df1495f8242c11e\": container with ID starting with 8f647efbd4f4f64fedeccae7318c4ecd9cfbd4fce9fa538a4df1495f8242c11e not found: ID does not exist" containerID="8f647efbd4f4f64fedeccae7318c4ecd9cfbd4fce9fa538a4df1495f8242c11e" Dec 03 11:53:38 crc kubenswrapper[5052]: I1203 11:53:38.612313 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8f647efbd4f4f64fedeccae7318c4ecd9cfbd4fce9fa538a4df1495f8242c11e"} err="failed to get container status \"8f647efbd4f4f64fedeccae7318c4ecd9cfbd4fce9fa538a4df1495f8242c11e\": rpc error: code = NotFound desc = could not find container \"8f647efbd4f4f64fedeccae7318c4ecd9cfbd4fce9fa538a4df1495f8242c11e\": container with ID starting with 8f647efbd4f4f64fedeccae7318c4ecd9cfbd4fce9fa538a4df1495f8242c11e not found: ID does not exist" Dec 03 11:53:38 crc kubenswrapper[5052]: I1203 11:53:38.612397 5052 scope.go:117] "RemoveContainer" containerID="e993f2c99fe8426fc231da418041c0bcb56e7d6a81c4f52c27e7ad9249e1bea6" Dec 03 11:53:38 crc kubenswrapper[5052]: E1203 11:53:38.612947 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e993f2c99fe8426fc231da418041c0bcb56e7d6a81c4f52c27e7ad9249e1bea6\": container with ID starting with e993f2c99fe8426fc231da418041c0bcb56e7d6a81c4f52c27e7ad9249e1bea6 not found: ID does not exist" containerID="e993f2c99fe8426fc231da418041c0bcb56e7d6a81c4f52c27e7ad9249e1bea6" Dec 03 11:53:38 crc kubenswrapper[5052]: I1203 11:53:38.613068 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e993f2c99fe8426fc231da418041c0bcb56e7d6a81c4f52c27e7ad9249e1bea6"} err="failed to get container status \"e993f2c99fe8426fc231da418041c0bcb56e7d6a81c4f52c27e7ad9249e1bea6\": rpc error: code = NotFound desc = could not find container \"e993f2c99fe8426fc231da418041c0bcb56e7d6a81c4f52c27e7ad9249e1bea6\": container with ID starting with e993f2c99fe8426fc231da418041c0bcb56e7d6a81c4f52c27e7ad9249e1bea6 not found: ID does not exist" Dec 03 11:53:38 crc kubenswrapper[5052]: I1203 11:53:38.624771 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 11:53:38 crc kubenswrapper[5052]: I1203 11:53:38.642531 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 11:53:38 crc kubenswrapper[5052]: I1203 11:53:38.660255 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 11:53:38 crc kubenswrapper[5052]: E1203 11:53:38.660746 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="8a2597b5-e6ec-40e9-a992-ea2ee71ae759" containerName="nova-metadata-metadata" Dec 03 11:53:38 crc kubenswrapper[5052]: I1203 11:53:38.660773 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="8a2597b5-e6ec-40e9-a992-ea2ee71ae759" containerName="nova-metadata-metadata" Dec 03 11:53:38 crc kubenswrapper[5052]: E1203 11:53:38.660790 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="8a2597b5-e6ec-40e9-a992-ea2ee71ae759" containerName="nova-metadata-log" Dec 03 11:53:38 crc kubenswrapper[5052]: I1203 11:53:38.660797 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="8a2597b5-e6ec-40e9-a992-ea2ee71ae759" containerName="nova-metadata-log" Dec 03 11:53:38 crc kubenswrapper[5052]: I1203 11:53:38.661058 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="8a2597b5-e6ec-40e9-a992-ea2ee71ae759" containerName="nova-metadata-log" Dec 03 11:53:38 crc kubenswrapper[5052]: I1203 11:53:38.661089 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="8a2597b5-e6ec-40e9-a992-ea2ee71ae759" containerName="nova-metadata-metadata" Dec 03 11:53:38 crc kubenswrapper[5052]: I1203 11:53:38.662433 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:53:38 crc kubenswrapper[5052]: I1203 11:53:38.663821 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 11:53:38 crc kubenswrapper[5052]: I1203 11:53:38.668525 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-nova-metadata-internal-svc" Dec 03 11:53:38 crc kubenswrapper[5052]: I1203 11:53:38.668799 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-metadata-config-data" Dec 03 11:53:38 crc kubenswrapper[5052]: I1203 11:53:38.767999 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/653882ec-4522-4979-b1aa-7fad5023605c-config-data\") pod \"nova-metadata-0\" (UID: \"653882ec-4522-4979-b1aa-7fad5023605c\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:53:38 crc kubenswrapper[5052]: I1203 11:53:38.768107 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/653882ec-4522-4979-b1aa-7fad5023605c-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"653882ec-4522-4979-b1aa-7fad5023605c\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:53:38 crc kubenswrapper[5052]: I1203 11:53:38.768196 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/653882ec-4522-4979-b1aa-7fad5023605c-logs\") pod \"nova-metadata-0\" (UID: \"653882ec-4522-4979-b1aa-7fad5023605c\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:53:38 crc kubenswrapper[5052]: I1203 11:53:38.768296 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/653882ec-4522-4979-b1aa-7fad5023605c-nova-metadata-tls-certs\") pod \"nova-metadata-0\" (UID: \"653882ec-4522-4979-b1aa-7fad5023605c\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:53:38 crc kubenswrapper[5052]: I1203 11:53:38.768328 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-72gng\" (UniqueName: \"kubernetes.io/projected/653882ec-4522-4979-b1aa-7fad5023605c-kube-api-access-72gng\") pod \"nova-metadata-0\" (UID: \"653882ec-4522-4979-b1aa-7fad5023605c\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:53:38 crc kubenswrapper[5052]: I1203 11:53:38.870489 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/653882ec-4522-4979-b1aa-7fad5023605c-logs\") pod \"nova-metadata-0\" (UID: \"653882ec-4522-4979-b1aa-7fad5023605c\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:53:38 crc kubenswrapper[5052]: I1203 11:53:38.870622 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/653882ec-4522-4979-b1aa-7fad5023605c-nova-metadata-tls-certs\") pod \"nova-metadata-0\" (UID: \"653882ec-4522-4979-b1aa-7fad5023605c\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:53:38 crc kubenswrapper[5052]: I1203 11:53:38.870657 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-72gng\" (UniqueName: \"kubernetes.io/projected/653882ec-4522-4979-b1aa-7fad5023605c-kube-api-access-72gng\") pod \"nova-metadata-0\" (UID: \"653882ec-4522-4979-b1aa-7fad5023605c\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:53:38 crc kubenswrapper[5052]: I1203 11:53:38.870694 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/653882ec-4522-4979-b1aa-7fad5023605c-config-data\") pod \"nova-metadata-0\" (UID: \"653882ec-4522-4979-b1aa-7fad5023605c\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:53:38 crc kubenswrapper[5052]: I1203 11:53:38.870743 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/653882ec-4522-4979-b1aa-7fad5023605c-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"653882ec-4522-4979-b1aa-7fad5023605c\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:53:38 crc kubenswrapper[5052]: I1203 11:53:38.871876 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/653882ec-4522-4979-b1aa-7fad5023605c-logs\") pod \"nova-metadata-0\" (UID: \"653882ec-4522-4979-b1aa-7fad5023605c\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:53:38 crc kubenswrapper[5052]: I1203 11:53:38.880237 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/653882ec-4522-4979-b1aa-7fad5023605c-nova-metadata-tls-certs\") pod \"nova-metadata-0\" (UID: \"653882ec-4522-4979-b1aa-7fad5023605c\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:53:38 crc kubenswrapper[5052]: I1203 11:53:38.886228 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/653882ec-4522-4979-b1aa-7fad5023605c-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"653882ec-4522-4979-b1aa-7fad5023605c\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:53:38 crc kubenswrapper[5052]: I1203 11:53:38.889748 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/653882ec-4522-4979-b1aa-7fad5023605c-config-data\") pod \"nova-metadata-0\" (UID: \"653882ec-4522-4979-b1aa-7fad5023605c\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:53:38 crc kubenswrapper[5052]: I1203 11:53:38.889752 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-72gng\" (UniqueName: \"kubernetes.io/projected/653882ec-4522-4979-b1aa-7fad5023605c-kube-api-access-72gng\") pod \"nova-metadata-0\" (UID: \"653882ec-4522-4979-b1aa-7fad5023605c\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:53:38 crc kubenswrapper[5052]: I1203 11:53:38.991739 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:53:39 crc kubenswrapper[5052]: I1203 11:53:39.447657 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 11:53:39 crc kubenswrapper[5052]: I1203 11:53:39.575051 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"653882ec-4522-4979-b1aa-7fad5023605c","Type":"ContainerStarted","Data":"8cf209cc3fa5306baea03e779a78d1b18f9d6410f0d70d3533fdca6d5197279e"} Dec 03 11:53:39 crc kubenswrapper[5052]: I1203 11:53:39.686733 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8a2597b5-e6ec-40e9-a992-ea2ee71ae759" path="/var/lib/kubelet/pods/8a2597b5-e6ec-40e9-a992-ea2ee71ae759/volumes" Dec 03 11:53:40 crc kubenswrapper[5052]: I1203 11:53:40.583636 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"653882ec-4522-4979-b1aa-7fad5023605c","Type":"ContainerStarted","Data":"5c59824fa390c99841f43488a426aa7880afaa22f074091daf68f834930db61d"} Dec 03 11:53:40 crc kubenswrapper[5052]: I1203 11:53:40.583687 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"653882ec-4522-4979-b1aa-7fad5023605c","Type":"ContainerStarted","Data":"cbf6b026a9b62a7464f44bbde7effeb8a0aa796789269fbcd6b9b3589fc439c9"} Dec 03 11:53:40 crc kubenswrapper[5052]: I1203 11:53:40.607007 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-metadata-0" podStartSLOduration=2.606988898 podStartE2EDuration="2.606988898s" podCreationTimestamp="2025-12-03 11:53:38 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:53:40.602491769 +0000 UTC m=+2011.190733533" watchObservedRunningTime="2025-12-03 11:53:40.606988898 +0000 UTC m=+2011.195230662" Dec 03 11:53:41 crc kubenswrapper[5052]: I1203 11:53:41.938428 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:53:43 crc kubenswrapper[5052]: I1203 11:53:43.992639 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:53:43 crc kubenswrapper[5052]: I1203 11:53:43.992708 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:53:45 crc kubenswrapper[5052]: I1203 11:53:45.669132 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:53:46 crc kubenswrapper[5052]: I1203 11:53:46.938789 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:53:46 crc kubenswrapper[5052]: I1203 11:53:46.966589 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:53:46 crc kubenswrapper[5052]: I1203 11:53:46.975628 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:53:46 crc kubenswrapper[5052]: I1203 11:53:46.975678 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:53:47 crc kubenswrapper[5052]: I1203 11:53:47.684961 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:53:48 crc kubenswrapper[5052]: I1203 11:53:48.058726 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-api-0" podUID="a759a8ef-882c-4eb0-8336-9aa26c80b488" containerName="nova-api-log" probeResult="failure" output="Get \"http://10.217.1.134:8774/\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Dec 03 11:53:48 crc kubenswrapper[5052]: I1203 11:53:48.058766 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-api-0" podUID="a759a8ef-882c-4eb0-8336-9aa26c80b488" containerName="nova-api-api" probeResult="failure" output="Get \"http://10.217.1.134:8774/\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Dec 03 11:53:48 crc kubenswrapper[5052]: I1203 11:53:48.992009 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:53:48 crc kubenswrapper[5052]: I1203 11:53:48.992398 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:53:50 crc kubenswrapper[5052]: I1203 11:53:50.004677 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-metadata-0" podUID="653882ec-4522-4979-b1aa-7fad5023605c" containerName="nova-metadata-log" probeResult="failure" output="Get \"https://10.217.1.135:8775/\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Dec 03 11:53:50 crc kubenswrapper[5052]: I1203 11:53:50.004703 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-metadata-0" podUID="653882ec-4522-4979-b1aa-7fad5023605c" containerName="nova-metadata-metadata" probeResult="failure" output="Get \"https://10.217.1.135:8775/\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Dec 03 11:53:56 crc kubenswrapper[5052]: I1203 11:53:56.979331 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:53:56 crc kubenswrapper[5052]: I1203 11:53:56.980177 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:53:56 crc kubenswrapper[5052]: I1203 11:53:56.980647 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:53:56 crc kubenswrapper[5052]: I1203 11:53:56.980691 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:53:56 crc kubenswrapper[5052]: I1203 11:53:56.983621 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:53:56 crc kubenswrapper[5052]: I1203 11:53:56.988957 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:53:58 crc kubenswrapper[5052]: I1203 11:53:58.982064 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:53:58 crc kubenswrapper[5052]: I1203 11:53:58.983602 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="eec21126-4992-40a1-ad21-4a005a5f7b1c" containerName="ceilometer-central-agent" containerID="cri-o://bb276bcdc3ab1893514f1a6e8c1d6bce34dced631ba9630ce9d6aa621798b31b" gracePeriod=30 Dec 03 11:53:58 crc kubenswrapper[5052]: I1203 11:53:58.983680 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="eec21126-4992-40a1-ad21-4a005a5f7b1c" containerName="proxy-httpd" containerID="cri-o://1f882f2f0bb94e8f107211926dcf76256c015c58e2c4caf41052300a24daec33" gracePeriod=30 Dec 03 11:53:58 crc kubenswrapper[5052]: I1203 11:53:58.983733 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="eec21126-4992-40a1-ad21-4a005a5f7b1c" containerName="sg-core" containerID="cri-o://2d366eb586a77209ca68fbef78c7a3d89603fae204b9708a92563b46fcfbb032" gracePeriod=30 Dec 03 11:53:58 crc kubenswrapper[5052]: I1203 11:53:58.983864 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="eec21126-4992-40a1-ad21-4a005a5f7b1c" containerName="ceilometer-notification-agent" containerID="cri-o://f5421d640c3d65fd8d438e37a5d376270f59fc2eef924597b04b9ca53af1eed4" gracePeriod=30 Dec 03 11:53:59 crc kubenswrapper[5052]: I1203 11:53:59.003413 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:53:59 crc kubenswrapper[5052]: I1203 11:53:59.003488 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:53:59 crc kubenswrapper[5052]: I1203 11:53:59.011422 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:53:59 crc kubenswrapper[5052]: I1203 11:53:59.021099 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:53:59 crc kubenswrapper[5052]: I1203 11:53:59.379937 5052 patch_prober.go:28] interesting pod/machine-config-daemon-74bj6 container/machine-config-daemon namespace/openshift-machine-config-operator: Liveness probe status=failure output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" start-of-body= Dec 03 11:53:59 crc kubenswrapper[5052]: I1203 11:53:59.380322 5052 prober.go:107] "Probe failed" probeType="Liveness" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" containerName="machine-config-daemon" probeResult="failure" output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" Dec 03 11:53:59 crc kubenswrapper[5052]: I1203 11:53:59.784910 5052 generic.go:334] "Generic (PLEG): container finished" podID="eec21126-4992-40a1-ad21-4a005a5f7b1c" containerID="1f882f2f0bb94e8f107211926dcf76256c015c58e2c4caf41052300a24daec33" exitCode=0 Dec 03 11:53:59 crc kubenswrapper[5052]: I1203 11:53:59.784940 5052 generic.go:334] "Generic (PLEG): container finished" podID="eec21126-4992-40a1-ad21-4a005a5f7b1c" containerID="2d366eb586a77209ca68fbef78c7a3d89603fae204b9708a92563b46fcfbb032" exitCode=2 Dec 03 11:53:59 crc kubenswrapper[5052]: I1203 11:53:59.784949 5052 generic.go:334] "Generic (PLEG): container finished" podID="eec21126-4992-40a1-ad21-4a005a5f7b1c" containerID="bb276bcdc3ab1893514f1a6e8c1d6bce34dced631ba9630ce9d6aa621798b31b" exitCode=0 Dec 03 11:53:59 crc kubenswrapper[5052]: I1203 11:53:59.785721 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"eec21126-4992-40a1-ad21-4a005a5f7b1c","Type":"ContainerDied","Data":"1f882f2f0bb94e8f107211926dcf76256c015c58e2c4caf41052300a24daec33"} Dec 03 11:53:59 crc kubenswrapper[5052]: I1203 11:53:59.785774 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"eec21126-4992-40a1-ad21-4a005a5f7b1c","Type":"ContainerDied","Data":"2d366eb586a77209ca68fbef78c7a3d89603fae204b9708a92563b46fcfbb032"} Dec 03 11:53:59 crc kubenswrapper[5052]: I1203 11:53:59.785787 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"eec21126-4992-40a1-ad21-4a005a5f7b1c","Type":"ContainerDied","Data":"bb276bcdc3ab1893514f1a6e8c1d6bce34dced631ba9630ce9d6aa621798b31b"} Dec 03 11:53:59 crc kubenswrapper[5052]: I1203 11:53:59.914796 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 11:53:59 crc kubenswrapper[5052]: I1203 11:53:59.915209 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-api-0" podUID="a759a8ef-882c-4eb0-8336-9aa26c80b488" containerName="nova-api-api" containerID="cri-o://1b0b289d84e0446d67b376b1b82446636f101d092aa2c630b25ef98f06792d0e" gracePeriod=30 Dec 03 11:53:59 crc kubenswrapper[5052]: I1203 11:53:59.915067 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-api-0" podUID="a759a8ef-882c-4eb0-8336-9aa26c80b488" containerName="nova-api-log" containerID="cri-o://8aba5a231952e5e90e8bda795fd346964cfc9876df0c0e4eb4f010f93cd471fe" gracePeriod=30 Dec 03 11:54:00 crc kubenswrapper[5052]: I1203 11:54:00.796728 5052 generic.go:334] "Generic (PLEG): container finished" podID="a759a8ef-882c-4eb0-8336-9aa26c80b488" containerID="8aba5a231952e5e90e8bda795fd346964cfc9876df0c0e4eb4f010f93cd471fe" exitCode=143 Dec 03 11:54:00 crc kubenswrapper[5052]: I1203 11:54:00.796831 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"a759a8ef-882c-4eb0-8336-9aa26c80b488","Type":"ContainerDied","Data":"8aba5a231952e5e90e8bda795fd346964cfc9876df0c0e4eb4f010f93cd471fe"} Dec 03 11:54:01 crc kubenswrapper[5052]: I1203 11:54:01.423044 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:54:01 crc kubenswrapper[5052]: I1203 11:54:01.535747 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/eec21126-4992-40a1-ad21-4a005a5f7b1c-ceilometer-tls-certs\") pod \"eec21126-4992-40a1-ad21-4a005a5f7b1c\" (UID: \"eec21126-4992-40a1-ad21-4a005a5f7b1c\") " Dec 03 11:54:01 crc kubenswrapper[5052]: I1203 11:54:01.535790 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/eec21126-4992-40a1-ad21-4a005a5f7b1c-sg-core-conf-yaml\") pod \"eec21126-4992-40a1-ad21-4a005a5f7b1c\" (UID: \"eec21126-4992-40a1-ad21-4a005a5f7b1c\") " Dec 03 11:54:01 crc kubenswrapper[5052]: I1203 11:54:01.535901 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/eec21126-4992-40a1-ad21-4a005a5f7b1c-scripts\") pod \"eec21126-4992-40a1-ad21-4a005a5f7b1c\" (UID: \"eec21126-4992-40a1-ad21-4a005a5f7b1c\") " Dec 03 11:54:01 crc kubenswrapper[5052]: I1203 11:54:01.535958 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/eec21126-4992-40a1-ad21-4a005a5f7b1c-config-data\") pod \"eec21126-4992-40a1-ad21-4a005a5f7b1c\" (UID: \"eec21126-4992-40a1-ad21-4a005a5f7b1c\") " Dec 03 11:54:01 crc kubenswrapper[5052]: I1203 11:54:01.536006 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/eec21126-4992-40a1-ad21-4a005a5f7b1c-combined-ca-bundle\") pod \"eec21126-4992-40a1-ad21-4a005a5f7b1c\" (UID: \"eec21126-4992-40a1-ad21-4a005a5f7b1c\") " Dec 03 11:54:01 crc kubenswrapper[5052]: I1203 11:54:01.536058 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/eec21126-4992-40a1-ad21-4a005a5f7b1c-log-httpd\") pod \"eec21126-4992-40a1-ad21-4a005a5f7b1c\" (UID: \"eec21126-4992-40a1-ad21-4a005a5f7b1c\") " Dec 03 11:54:01 crc kubenswrapper[5052]: I1203 11:54:01.536095 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/eec21126-4992-40a1-ad21-4a005a5f7b1c-run-httpd\") pod \"eec21126-4992-40a1-ad21-4a005a5f7b1c\" (UID: \"eec21126-4992-40a1-ad21-4a005a5f7b1c\") " Dec 03 11:54:01 crc kubenswrapper[5052]: I1203 11:54:01.536142 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lbjk8\" (UniqueName: \"kubernetes.io/projected/eec21126-4992-40a1-ad21-4a005a5f7b1c-kube-api-access-lbjk8\") pod \"eec21126-4992-40a1-ad21-4a005a5f7b1c\" (UID: \"eec21126-4992-40a1-ad21-4a005a5f7b1c\") " Dec 03 11:54:01 crc kubenswrapper[5052]: I1203 11:54:01.536473 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eec21126-4992-40a1-ad21-4a005a5f7b1c-run-httpd" (OuterVolumeSpecName: "run-httpd") pod "eec21126-4992-40a1-ad21-4a005a5f7b1c" (UID: "eec21126-4992-40a1-ad21-4a005a5f7b1c"). InnerVolumeSpecName "run-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:54:01 crc kubenswrapper[5052]: I1203 11:54:01.536533 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eec21126-4992-40a1-ad21-4a005a5f7b1c-log-httpd" (OuterVolumeSpecName: "log-httpd") pod "eec21126-4992-40a1-ad21-4a005a5f7b1c" (UID: "eec21126-4992-40a1-ad21-4a005a5f7b1c"). InnerVolumeSpecName "log-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:54:01 crc kubenswrapper[5052]: I1203 11:54:01.542742 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eec21126-4992-40a1-ad21-4a005a5f7b1c-kube-api-access-lbjk8" (OuterVolumeSpecName: "kube-api-access-lbjk8") pod "eec21126-4992-40a1-ad21-4a005a5f7b1c" (UID: "eec21126-4992-40a1-ad21-4a005a5f7b1c"). InnerVolumeSpecName "kube-api-access-lbjk8". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:54:01 crc kubenswrapper[5052]: I1203 11:54:01.545394 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eec21126-4992-40a1-ad21-4a005a5f7b1c-scripts" (OuterVolumeSpecName: "scripts") pod "eec21126-4992-40a1-ad21-4a005a5f7b1c" (UID: "eec21126-4992-40a1-ad21-4a005a5f7b1c"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:54:01 crc kubenswrapper[5052]: I1203 11:54:01.569737 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eec21126-4992-40a1-ad21-4a005a5f7b1c-sg-core-conf-yaml" (OuterVolumeSpecName: "sg-core-conf-yaml") pod "eec21126-4992-40a1-ad21-4a005a5f7b1c" (UID: "eec21126-4992-40a1-ad21-4a005a5f7b1c"). InnerVolumeSpecName "sg-core-conf-yaml". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:54:01 crc kubenswrapper[5052]: I1203 11:54:01.627096 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eec21126-4992-40a1-ad21-4a005a5f7b1c-ceilometer-tls-certs" (OuterVolumeSpecName: "ceilometer-tls-certs") pod "eec21126-4992-40a1-ad21-4a005a5f7b1c" (UID: "eec21126-4992-40a1-ad21-4a005a5f7b1c"). InnerVolumeSpecName "ceilometer-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:54:01 crc kubenswrapper[5052]: I1203 11:54:01.637882 5052 reconciler_common.go:293] "Volume detached for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/eec21126-4992-40a1-ad21-4a005a5f7b1c-log-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:01 crc kubenswrapper[5052]: I1203 11:54:01.637908 5052 reconciler_common.go:293] "Volume detached for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/eec21126-4992-40a1-ad21-4a005a5f7b1c-run-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:01 crc kubenswrapper[5052]: I1203 11:54:01.637918 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-lbjk8\" (UniqueName: \"kubernetes.io/projected/eec21126-4992-40a1-ad21-4a005a5f7b1c-kube-api-access-lbjk8\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:01 crc kubenswrapper[5052]: I1203 11:54:01.637926 5052 reconciler_common.go:293] "Volume detached for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/eec21126-4992-40a1-ad21-4a005a5f7b1c-ceilometer-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:01 crc kubenswrapper[5052]: I1203 11:54:01.638495 5052 reconciler_common.go:293] "Volume detached for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/eec21126-4992-40a1-ad21-4a005a5f7b1c-sg-core-conf-yaml\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:01 crc kubenswrapper[5052]: I1203 11:54:01.638548 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/eec21126-4992-40a1-ad21-4a005a5f7b1c-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:01 crc kubenswrapper[5052]: I1203 11:54:01.640115 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eec21126-4992-40a1-ad21-4a005a5f7b1c-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "eec21126-4992-40a1-ad21-4a005a5f7b1c" (UID: "eec21126-4992-40a1-ad21-4a005a5f7b1c"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:54:01 crc kubenswrapper[5052]: I1203 11:54:01.657698 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eec21126-4992-40a1-ad21-4a005a5f7b1c-config-data" (OuterVolumeSpecName: "config-data") pod "eec21126-4992-40a1-ad21-4a005a5f7b1c" (UID: "eec21126-4992-40a1-ad21-4a005a5f7b1c"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:54:01 crc kubenswrapper[5052]: I1203 11:54:01.741856 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/eec21126-4992-40a1-ad21-4a005a5f7b1c-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:01 crc kubenswrapper[5052]: I1203 11:54:01.741889 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/eec21126-4992-40a1-ad21-4a005a5f7b1c-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:01 crc kubenswrapper[5052]: I1203 11:54:01.806496 5052 generic.go:334] "Generic (PLEG): container finished" podID="eec21126-4992-40a1-ad21-4a005a5f7b1c" containerID="f5421d640c3d65fd8d438e37a5d376270f59fc2eef924597b04b9ca53af1eed4" exitCode=0 Dec 03 11:54:01 crc kubenswrapper[5052]: I1203 11:54:01.806563 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"eec21126-4992-40a1-ad21-4a005a5f7b1c","Type":"ContainerDied","Data":"f5421d640c3d65fd8d438e37a5d376270f59fc2eef924597b04b9ca53af1eed4"} Dec 03 11:54:01 crc kubenswrapper[5052]: I1203 11:54:01.806596 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"eec21126-4992-40a1-ad21-4a005a5f7b1c","Type":"ContainerDied","Data":"9f1bc73aa05c422d96db7f00c5b657494b348132ea05203d632c3ef351760039"} Dec 03 11:54:01 crc kubenswrapper[5052]: I1203 11:54:01.806616 5052 scope.go:117] "RemoveContainer" containerID="1f882f2f0bb94e8f107211926dcf76256c015c58e2c4caf41052300a24daec33" Dec 03 11:54:01 crc kubenswrapper[5052]: I1203 11:54:01.806773 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:54:01 crc kubenswrapper[5052]: I1203 11:54:01.840870 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:54:01 crc kubenswrapper[5052]: I1203 11:54:01.848637 5052 scope.go:117] "RemoveContainer" containerID="2d366eb586a77209ca68fbef78c7a3d89603fae204b9708a92563b46fcfbb032" Dec 03 11:54:01 crc kubenswrapper[5052]: I1203 11:54:01.850422 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:54:01 crc kubenswrapper[5052]: I1203 11:54:01.873679 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:54:01 crc kubenswrapper[5052]: E1203 11:54:01.874177 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="eec21126-4992-40a1-ad21-4a005a5f7b1c" containerName="ceilometer-notification-agent" Dec 03 11:54:01 crc kubenswrapper[5052]: I1203 11:54:01.874197 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="eec21126-4992-40a1-ad21-4a005a5f7b1c" containerName="ceilometer-notification-agent" Dec 03 11:54:01 crc kubenswrapper[5052]: E1203 11:54:01.874213 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="eec21126-4992-40a1-ad21-4a005a5f7b1c" containerName="sg-core" Dec 03 11:54:01 crc kubenswrapper[5052]: I1203 11:54:01.874223 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="eec21126-4992-40a1-ad21-4a005a5f7b1c" containerName="sg-core" Dec 03 11:54:01 crc kubenswrapper[5052]: E1203 11:54:01.874245 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="eec21126-4992-40a1-ad21-4a005a5f7b1c" containerName="ceilometer-central-agent" Dec 03 11:54:01 crc kubenswrapper[5052]: I1203 11:54:01.874252 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="eec21126-4992-40a1-ad21-4a005a5f7b1c" containerName="ceilometer-central-agent" Dec 03 11:54:01 crc kubenswrapper[5052]: E1203 11:54:01.874279 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="eec21126-4992-40a1-ad21-4a005a5f7b1c" containerName="proxy-httpd" Dec 03 11:54:01 crc kubenswrapper[5052]: I1203 11:54:01.874286 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="eec21126-4992-40a1-ad21-4a005a5f7b1c" containerName="proxy-httpd" Dec 03 11:54:01 crc kubenswrapper[5052]: I1203 11:54:01.874537 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="eec21126-4992-40a1-ad21-4a005a5f7b1c" containerName="ceilometer-notification-agent" Dec 03 11:54:01 crc kubenswrapper[5052]: I1203 11:54:01.874562 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="eec21126-4992-40a1-ad21-4a005a5f7b1c" containerName="sg-core" Dec 03 11:54:01 crc kubenswrapper[5052]: I1203 11:54:01.874581 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="eec21126-4992-40a1-ad21-4a005a5f7b1c" containerName="proxy-httpd" Dec 03 11:54:01 crc kubenswrapper[5052]: I1203 11:54:01.874596 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="eec21126-4992-40a1-ad21-4a005a5f7b1c" containerName="ceilometer-central-agent" Dec 03 11:54:01 crc kubenswrapper[5052]: I1203 11:54:01.876859 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:54:01 crc kubenswrapper[5052]: I1203 11:54:01.877646 5052 scope.go:117] "RemoveContainer" containerID="f5421d640c3d65fd8d438e37a5d376270f59fc2eef924597b04b9ca53af1eed4" Dec 03 11:54:01 crc kubenswrapper[5052]: I1203 11:54:01.896641 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-scripts" Dec 03 11:54:01 crc kubenswrapper[5052]: I1203 11:54:01.896883 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-ceilometer-internal-svc" Dec 03 11:54:01 crc kubenswrapper[5052]: I1203 11:54:01.898081 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-config-data" Dec 03 11:54:01 crc kubenswrapper[5052]: I1203 11:54:01.906195 5052 scope.go:117] "RemoveContainer" containerID="bb276bcdc3ab1893514f1a6e8c1d6bce34dced631ba9630ce9d6aa621798b31b" Dec 03 11:54:01 crc kubenswrapper[5052]: I1203 11:54:01.925924 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:54:01 crc kubenswrapper[5052]: I1203 11:54:01.934757 5052 scope.go:117] "RemoveContainer" containerID="1f882f2f0bb94e8f107211926dcf76256c015c58e2c4caf41052300a24daec33" Dec 03 11:54:01 crc kubenswrapper[5052]: E1203 11:54:01.935332 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1f882f2f0bb94e8f107211926dcf76256c015c58e2c4caf41052300a24daec33\": container with ID starting with 1f882f2f0bb94e8f107211926dcf76256c015c58e2c4caf41052300a24daec33 not found: ID does not exist" containerID="1f882f2f0bb94e8f107211926dcf76256c015c58e2c4caf41052300a24daec33" Dec 03 11:54:01 crc kubenswrapper[5052]: I1203 11:54:01.935371 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1f882f2f0bb94e8f107211926dcf76256c015c58e2c4caf41052300a24daec33"} err="failed to get container status \"1f882f2f0bb94e8f107211926dcf76256c015c58e2c4caf41052300a24daec33\": rpc error: code = NotFound desc = could not find container \"1f882f2f0bb94e8f107211926dcf76256c015c58e2c4caf41052300a24daec33\": container with ID starting with 1f882f2f0bb94e8f107211926dcf76256c015c58e2c4caf41052300a24daec33 not found: ID does not exist" Dec 03 11:54:01 crc kubenswrapper[5052]: I1203 11:54:01.935399 5052 scope.go:117] "RemoveContainer" containerID="2d366eb586a77209ca68fbef78c7a3d89603fae204b9708a92563b46fcfbb032" Dec 03 11:54:01 crc kubenswrapper[5052]: E1203 11:54:01.935970 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2d366eb586a77209ca68fbef78c7a3d89603fae204b9708a92563b46fcfbb032\": container with ID starting with 2d366eb586a77209ca68fbef78c7a3d89603fae204b9708a92563b46fcfbb032 not found: ID does not exist" containerID="2d366eb586a77209ca68fbef78c7a3d89603fae204b9708a92563b46fcfbb032" Dec 03 11:54:01 crc kubenswrapper[5052]: I1203 11:54:01.936049 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d366eb586a77209ca68fbef78c7a3d89603fae204b9708a92563b46fcfbb032"} err="failed to get container status \"2d366eb586a77209ca68fbef78c7a3d89603fae204b9708a92563b46fcfbb032\": rpc error: code = NotFound desc = could not find container \"2d366eb586a77209ca68fbef78c7a3d89603fae204b9708a92563b46fcfbb032\": container with ID starting with 2d366eb586a77209ca68fbef78c7a3d89603fae204b9708a92563b46fcfbb032 not found: ID does not exist" Dec 03 11:54:01 crc kubenswrapper[5052]: I1203 11:54:01.936078 5052 scope.go:117] "RemoveContainer" containerID="f5421d640c3d65fd8d438e37a5d376270f59fc2eef924597b04b9ca53af1eed4" Dec 03 11:54:01 crc kubenswrapper[5052]: E1203 11:54:01.936397 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f5421d640c3d65fd8d438e37a5d376270f59fc2eef924597b04b9ca53af1eed4\": container with ID starting with f5421d640c3d65fd8d438e37a5d376270f59fc2eef924597b04b9ca53af1eed4 not found: ID does not exist" containerID="f5421d640c3d65fd8d438e37a5d376270f59fc2eef924597b04b9ca53af1eed4" Dec 03 11:54:01 crc kubenswrapper[5052]: I1203 11:54:01.936445 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f5421d640c3d65fd8d438e37a5d376270f59fc2eef924597b04b9ca53af1eed4"} err="failed to get container status \"f5421d640c3d65fd8d438e37a5d376270f59fc2eef924597b04b9ca53af1eed4\": rpc error: code = NotFound desc = could not find container \"f5421d640c3d65fd8d438e37a5d376270f59fc2eef924597b04b9ca53af1eed4\": container with ID starting with f5421d640c3d65fd8d438e37a5d376270f59fc2eef924597b04b9ca53af1eed4 not found: ID does not exist" Dec 03 11:54:01 crc kubenswrapper[5052]: I1203 11:54:01.936466 5052 scope.go:117] "RemoveContainer" containerID="bb276bcdc3ab1893514f1a6e8c1d6bce34dced631ba9630ce9d6aa621798b31b" Dec 03 11:54:01 crc kubenswrapper[5052]: E1203 11:54:01.937146 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bb276bcdc3ab1893514f1a6e8c1d6bce34dced631ba9630ce9d6aa621798b31b\": container with ID starting with bb276bcdc3ab1893514f1a6e8c1d6bce34dced631ba9630ce9d6aa621798b31b not found: ID does not exist" containerID="bb276bcdc3ab1893514f1a6e8c1d6bce34dced631ba9630ce9d6aa621798b31b" Dec 03 11:54:01 crc kubenswrapper[5052]: I1203 11:54:01.937171 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bb276bcdc3ab1893514f1a6e8c1d6bce34dced631ba9630ce9d6aa621798b31b"} err="failed to get container status \"bb276bcdc3ab1893514f1a6e8c1d6bce34dced631ba9630ce9d6aa621798b31b\": rpc error: code = NotFound desc = could not find container \"bb276bcdc3ab1893514f1a6e8c1d6bce34dced631ba9630ce9d6aa621798b31b\": container with ID starting with bb276bcdc3ab1893514f1a6e8c1d6bce34dced631ba9630ce9d6aa621798b31b not found: ID does not exist" Dec 03 11:54:02 crc kubenswrapper[5052]: I1203 11:54:02.047310 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/a5ff0249-173a-4793-8cbe-0aa749f34632-log-httpd\") pod \"ceilometer-0\" (UID: \"a5ff0249-173a-4793-8cbe-0aa749f34632\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:54:02 crc kubenswrapper[5052]: I1203 11:54:02.047403 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/a5ff0249-173a-4793-8cbe-0aa749f34632-ceilometer-tls-certs\") pod \"ceilometer-0\" (UID: \"a5ff0249-173a-4793-8cbe-0aa749f34632\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:54:02 crc kubenswrapper[5052]: I1203 11:54:02.047489 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a5ff0249-173a-4793-8cbe-0aa749f34632-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"a5ff0249-173a-4793-8cbe-0aa749f34632\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:54:02 crc kubenswrapper[5052]: I1203 11:54:02.047539 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/a5ff0249-173a-4793-8cbe-0aa749f34632-scripts\") pod \"ceilometer-0\" (UID: \"a5ff0249-173a-4793-8cbe-0aa749f34632\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:54:02 crc kubenswrapper[5052]: I1203 11:54:02.047616 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a5ff0249-173a-4793-8cbe-0aa749f34632-config-data\") pod \"ceilometer-0\" (UID: \"a5ff0249-173a-4793-8cbe-0aa749f34632\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:54:02 crc kubenswrapper[5052]: I1203 11:54:02.047655 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4v26g\" (UniqueName: \"kubernetes.io/projected/a5ff0249-173a-4793-8cbe-0aa749f34632-kube-api-access-4v26g\") pod \"ceilometer-0\" (UID: \"a5ff0249-173a-4793-8cbe-0aa749f34632\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:54:02 crc kubenswrapper[5052]: I1203 11:54:02.047689 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/a5ff0249-173a-4793-8cbe-0aa749f34632-run-httpd\") pod \"ceilometer-0\" (UID: \"a5ff0249-173a-4793-8cbe-0aa749f34632\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:54:02 crc kubenswrapper[5052]: I1203 11:54:02.047742 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/a5ff0249-173a-4793-8cbe-0aa749f34632-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"a5ff0249-173a-4793-8cbe-0aa749f34632\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:54:02 crc kubenswrapper[5052]: I1203 11:54:02.149124 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a5ff0249-173a-4793-8cbe-0aa749f34632-config-data\") pod \"ceilometer-0\" (UID: \"a5ff0249-173a-4793-8cbe-0aa749f34632\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:54:02 crc kubenswrapper[5052]: I1203 11:54:02.149187 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-4v26g\" (UniqueName: \"kubernetes.io/projected/a5ff0249-173a-4793-8cbe-0aa749f34632-kube-api-access-4v26g\") pod \"ceilometer-0\" (UID: \"a5ff0249-173a-4793-8cbe-0aa749f34632\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:54:02 crc kubenswrapper[5052]: I1203 11:54:02.149214 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/a5ff0249-173a-4793-8cbe-0aa749f34632-run-httpd\") pod \"ceilometer-0\" (UID: \"a5ff0249-173a-4793-8cbe-0aa749f34632\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:54:02 crc kubenswrapper[5052]: I1203 11:54:02.149246 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/a5ff0249-173a-4793-8cbe-0aa749f34632-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"a5ff0249-173a-4793-8cbe-0aa749f34632\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:54:02 crc kubenswrapper[5052]: I1203 11:54:02.149278 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/a5ff0249-173a-4793-8cbe-0aa749f34632-log-httpd\") pod \"ceilometer-0\" (UID: \"a5ff0249-173a-4793-8cbe-0aa749f34632\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:54:02 crc kubenswrapper[5052]: I1203 11:54:02.149318 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/a5ff0249-173a-4793-8cbe-0aa749f34632-ceilometer-tls-certs\") pod \"ceilometer-0\" (UID: \"a5ff0249-173a-4793-8cbe-0aa749f34632\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:54:02 crc kubenswrapper[5052]: I1203 11:54:02.149356 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a5ff0249-173a-4793-8cbe-0aa749f34632-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"a5ff0249-173a-4793-8cbe-0aa749f34632\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:54:02 crc kubenswrapper[5052]: I1203 11:54:02.149381 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/a5ff0249-173a-4793-8cbe-0aa749f34632-scripts\") pod \"ceilometer-0\" (UID: \"a5ff0249-173a-4793-8cbe-0aa749f34632\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:54:02 crc kubenswrapper[5052]: I1203 11:54:02.150205 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/a5ff0249-173a-4793-8cbe-0aa749f34632-log-httpd\") pod \"ceilometer-0\" (UID: \"a5ff0249-173a-4793-8cbe-0aa749f34632\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:54:02 crc kubenswrapper[5052]: I1203 11:54:02.150813 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/a5ff0249-173a-4793-8cbe-0aa749f34632-run-httpd\") pod \"ceilometer-0\" (UID: \"a5ff0249-173a-4793-8cbe-0aa749f34632\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:54:02 crc kubenswrapper[5052]: I1203 11:54:02.154652 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/a5ff0249-173a-4793-8cbe-0aa749f34632-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"a5ff0249-173a-4793-8cbe-0aa749f34632\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:54:02 crc kubenswrapper[5052]: I1203 11:54:02.156238 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a5ff0249-173a-4793-8cbe-0aa749f34632-config-data\") pod \"ceilometer-0\" (UID: \"a5ff0249-173a-4793-8cbe-0aa749f34632\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:54:02 crc kubenswrapper[5052]: I1203 11:54:02.156379 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/a5ff0249-173a-4793-8cbe-0aa749f34632-ceilometer-tls-certs\") pod \"ceilometer-0\" (UID: \"a5ff0249-173a-4793-8cbe-0aa749f34632\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:54:02 crc kubenswrapper[5052]: I1203 11:54:02.163945 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a5ff0249-173a-4793-8cbe-0aa749f34632-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"a5ff0249-173a-4793-8cbe-0aa749f34632\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:54:02 crc kubenswrapper[5052]: I1203 11:54:02.164298 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/a5ff0249-173a-4793-8cbe-0aa749f34632-scripts\") pod \"ceilometer-0\" (UID: \"a5ff0249-173a-4793-8cbe-0aa749f34632\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:54:02 crc kubenswrapper[5052]: I1203 11:54:02.179327 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-4v26g\" (UniqueName: \"kubernetes.io/projected/a5ff0249-173a-4793-8cbe-0aa749f34632-kube-api-access-4v26g\") pod \"ceilometer-0\" (UID: \"a5ff0249-173a-4793-8cbe-0aa749f34632\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:54:02 crc kubenswrapper[5052]: I1203 11:54:02.221788 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:54:02 crc kubenswrapper[5052]: I1203 11:54:02.693214 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:54:02 crc kubenswrapper[5052]: I1203 11:54:02.699719 5052 provider.go:102] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Dec 03 11:54:02 crc kubenswrapper[5052]: I1203 11:54:02.818283 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"a5ff0249-173a-4793-8cbe-0aa749f34632","Type":"ContainerStarted","Data":"42ce9b7cb4235ab65f20054ec3746eea9e53fe033f16c4c19190abce6617e7a5"} Dec 03 11:54:03 crc kubenswrapper[5052]: I1203 11:54:03.516120 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:54:03 crc kubenswrapper[5052]: I1203 11:54:03.678348 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/a759a8ef-882c-4eb0-8336-9aa26c80b488-logs\") pod \"a759a8ef-882c-4eb0-8336-9aa26c80b488\" (UID: \"a759a8ef-882c-4eb0-8336-9aa26c80b488\") " Dec 03 11:54:03 crc kubenswrapper[5052]: I1203 11:54:03.678784 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a759a8ef-882c-4eb0-8336-9aa26c80b488-config-data\") pod \"a759a8ef-882c-4eb0-8336-9aa26c80b488\" (UID: \"a759a8ef-882c-4eb0-8336-9aa26c80b488\") " Dec 03 11:54:03 crc kubenswrapper[5052]: I1203 11:54:03.678837 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a759a8ef-882c-4eb0-8336-9aa26c80b488-logs" (OuterVolumeSpecName: "logs") pod "a759a8ef-882c-4eb0-8336-9aa26c80b488" (UID: "a759a8ef-882c-4eb0-8336-9aa26c80b488"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:54:03 crc kubenswrapper[5052]: I1203 11:54:03.678891 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a759a8ef-882c-4eb0-8336-9aa26c80b488-combined-ca-bundle\") pod \"a759a8ef-882c-4eb0-8336-9aa26c80b488\" (UID: \"a759a8ef-882c-4eb0-8336-9aa26c80b488\") " Dec 03 11:54:03 crc kubenswrapper[5052]: I1203 11:54:03.678926 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fbnqh\" (UniqueName: \"kubernetes.io/projected/a759a8ef-882c-4eb0-8336-9aa26c80b488-kube-api-access-fbnqh\") pod \"a759a8ef-882c-4eb0-8336-9aa26c80b488\" (UID: \"a759a8ef-882c-4eb0-8336-9aa26c80b488\") " Dec 03 11:54:03 crc kubenswrapper[5052]: I1203 11:54:03.679733 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/a759a8ef-882c-4eb0-8336-9aa26c80b488-logs\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:03 crc kubenswrapper[5052]: I1203 11:54:03.685130 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eec21126-4992-40a1-ad21-4a005a5f7b1c" path="/var/lib/kubelet/pods/eec21126-4992-40a1-ad21-4a005a5f7b1c/volumes" Dec 03 11:54:03 crc kubenswrapper[5052]: I1203 11:54:03.686622 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a759a8ef-882c-4eb0-8336-9aa26c80b488-kube-api-access-fbnqh" (OuterVolumeSpecName: "kube-api-access-fbnqh") pod "a759a8ef-882c-4eb0-8336-9aa26c80b488" (UID: "a759a8ef-882c-4eb0-8336-9aa26c80b488"). InnerVolumeSpecName "kube-api-access-fbnqh". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:54:03 crc kubenswrapper[5052]: I1203 11:54:03.710265 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a759a8ef-882c-4eb0-8336-9aa26c80b488-config-data" (OuterVolumeSpecName: "config-data") pod "a759a8ef-882c-4eb0-8336-9aa26c80b488" (UID: "a759a8ef-882c-4eb0-8336-9aa26c80b488"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:54:03 crc kubenswrapper[5052]: I1203 11:54:03.715591 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a759a8ef-882c-4eb0-8336-9aa26c80b488-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "a759a8ef-882c-4eb0-8336-9aa26c80b488" (UID: "a759a8ef-882c-4eb0-8336-9aa26c80b488"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:54:03 crc kubenswrapper[5052]: I1203 11:54:03.781372 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a759a8ef-882c-4eb0-8336-9aa26c80b488-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:03 crc kubenswrapper[5052]: I1203 11:54:03.781489 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a759a8ef-882c-4eb0-8336-9aa26c80b488-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:03 crc kubenswrapper[5052]: I1203 11:54:03.781576 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-fbnqh\" (UniqueName: \"kubernetes.io/projected/a759a8ef-882c-4eb0-8336-9aa26c80b488-kube-api-access-fbnqh\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:03 crc kubenswrapper[5052]: I1203 11:54:03.830523 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"a5ff0249-173a-4793-8cbe-0aa749f34632","Type":"ContainerStarted","Data":"7bad91ec5c27b9fd97fb06baef39407cc01e86e5c940cc4ffd9ae98b217e02e2"} Dec 03 11:54:03 crc kubenswrapper[5052]: I1203 11:54:03.832698 5052 generic.go:334] "Generic (PLEG): container finished" podID="a759a8ef-882c-4eb0-8336-9aa26c80b488" containerID="1b0b289d84e0446d67b376b1b82446636f101d092aa2c630b25ef98f06792d0e" exitCode=0 Dec 03 11:54:03 crc kubenswrapper[5052]: I1203 11:54:03.832766 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:54:03 crc kubenswrapper[5052]: I1203 11:54:03.832760 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"a759a8ef-882c-4eb0-8336-9aa26c80b488","Type":"ContainerDied","Data":"1b0b289d84e0446d67b376b1b82446636f101d092aa2c630b25ef98f06792d0e"} Dec 03 11:54:03 crc kubenswrapper[5052]: I1203 11:54:03.832861 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"a759a8ef-882c-4eb0-8336-9aa26c80b488","Type":"ContainerDied","Data":"2f140cc1b5185e35ab25e66348f42d4605a62b533777b9385259815203c3ebaa"} Dec 03 11:54:03 crc kubenswrapper[5052]: I1203 11:54:03.832897 5052 scope.go:117] "RemoveContainer" containerID="1b0b289d84e0446d67b376b1b82446636f101d092aa2c630b25ef98f06792d0e" Dec 03 11:54:03 crc kubenswrapper[5052]: I1203 11:54:03.876467 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 11:54:03 crc kubenswrapper[5052]: I1203 11:54:03.898912 5052 scope.go:117] "RemoveContainer" containerID="8aba5a231952e5e90e8bda795fd346964cfc9876df0c0e4eb4f010f93cd471fe" Dec 03 11:54:03 crc kubenswrapper[5052]: I1203 11:54:03.900786 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 11:54:03 crc kubenswrapper[5052]: I1203 11:54:03.917791 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 11:54:03 crc kubenswrapper[5052]: E1203 11:54:03.918211 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="a759a8ef-882c-4eb0-8336-9aa26c80b488" containerName="nova-api-log" Dec 03 11:54:03 crc kubenswrapper[5052]: I1203 11:54:03.918227 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="a759a8ef-882c-4eb0-8336-9aa26c80b488" containerName="nova-api-log" Dec 03 11:54:03 crc kubenswrapper[5052]: E1203 11:54:03.918237 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="a759a8ef-882c-4eb0-8336-9aa26c80b488" containerName="nova-api-api" Dec 03 11:54:03 crc kubenswrapper[5052]: I1203 11:54:03.918243 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="a759a8ef-882c-4eb0-8336-9aa26c80b488" containerName="nova-api-api" Dec 03 11:54:03 crc kubenswrapper[5052]: I1203 11:54:03.918454 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="a759a8ef-882c-4eb0-8336-9aa26c80b488" containerName="nova-api-log" Dec 03 11:54:03 crc kubenswrapper[5052]: I1203 11:54:03.918473 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="a759a8ef-882c-4eb0-8336-9aa26c80b488" containerName="nova-api-api" Dec 03 11:54:03 crc kubenswrapper[5052]: I1203 11:54:03.920554 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:54:03 crc kubenswrapper[5052]: I1203 11:54:03.923966 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-api-config-data" Dec 03 11:54:03 crc kubenswrapper[5052]: I1203 11:54:03.924323 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-nova-internal-svc" Dec 03 11:54:03 crc kubenswrapper[5052]: I1203 11:54:03.924538 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-nova-public-svc" Dec 03 11:54:03 crc kubenswrapper[5052]: I1203 11:54:03.931243 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 11:54:03 crc kubenswrapper[5052]: I1203 11:54:03.938302 5052 scope.go:117] "RemoveContainer" containerID="1b0b289d84e0446d67b376b1b82446636f101d092aa2c630b25ef98f06792d0e" Dec 03 11:54:03 crc kubenswrapper[5052]: E1203 11:54:03.939159 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1b0b289d84e0446d67b376b1b82446636f101d092aa2c630b25ef98f06792d0e\": container with ID starting with 1b0b289d84e0446d67b376b1b82446636f101d092aa2c630b25ef98f06792d0e not found: ID does not exist" containerID="1b0b289d84e0446d67b376b1b82446636f101d092aa2c630b25ef98f06792d0e" Dec 03 11:54:03 crc kubenswrapper[5052]: I1203 11:54:03.939214 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1b0b289d84e0446d67b376b1b82446636f101d092aa2c630b25ef98f06792d0e"} err="failed to get container status \"1b0b289d84e0446d67b376b1b82446636f101d092aa2c630b25ef98f06792d0e\": rpc error: code = NotFound desc = could not find container \"1b0b289d84e0446d67b376b1b82446636f101d092aa2c630b25ef98f06792d0e\": container with ID starting with 1b0b289d84e0446d67b376b1b82446636f101d092aa2c630b25ef98f06792d0e not found: ID does not exist" Dec 03 11:54:03 crc kubenswrapper[5052]: I1203 11:54:03.939244 5052 scope.go:117] "RemoveContainer" containerID="8aba5a231952e5e90e8bda795fd346964cfc9876df0c0e4eb4f010f93cd471fe" Dec 03 11:54:03 crc kubenswrapper[5052]: E1203 11:54:03.939568 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8aba5a231952e5e90e8bda795fd346964cfc9876df0c0e4eb4f010f93cd471fe\": container with ID starting with 8aba5a231952e5e90e8bda795fd346964cfc9876df0c0e4eb4f010f93cd471fe not found: ID does not exist" containerID="8aba5a231952e5e90e8bda795fd346964cfc9876df0c0e4eb4f010f93cd471fe" Dec 03 11:54:03 crc kubenswrapper[5052]: I1203 11:54:03.939617 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8aba5a231952e5e90e8bda795fd346964cfc9876df0c0e4eb4f010f93cd471fe"} err="failed to get container status \"8aba5a231952e5e90e8bda795fd346964cfc9876df0c0e4eb4f010f93cd471fe\": rpc error: code = NotFound desc = could not find container \"8aba5a231952e5e90e8bda795fd346964cfc9876df0c0e4eb4f010f93cd471fe\": container with ID starting with 8aba5a231952e5e90e8bda795fd346964cfc9876df0c0e4eb4f010f93cd471fe not found: ID does not exist" Dec 03 11:54:04 crc kubenswrapper[5052]: I1203 11:54:04.087550 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/d3222c69-47bc-46d8-a025-cc457881cac9-config-data\") pod \"nova-api-0\" (UID: \"d3222c69-47bc-46d8-a025-cc457881cac9\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:54:04 crc kubenswrapper[5052]: I1203 11:54:04.087610 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/d3222c69-47bc-46d8-a025-cc457881cac9-public-tls-certs\") pod \"nova-api-0\" (UID: \"d3222c69-47bc-46d8-a025-cc457881cac9\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:54:04 crc kubenswrapper[5052]: I1203 11:54:04.087677 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d3222c69-47bc-46d8-a025-cc457881cac9-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"d3222c69-47bc-46d8-a025-cc457881cac9\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:54:04 crc kubenswrapper[5052]: I1203 11:54:04.087818 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/d3222c69-47bc-46d8-a025-cc457881cac9-logs\") pod \"nova-api-0\" (UID: \"d3222c69-47bc-46d8-a025-cc457881cac9\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:54:04 crc kubenswrapper[5052]: I1203 11:54:04.087848 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gfqkf\" (UniqueName: \"kubernetes.io/projected/d3222c69-47bc-46d8-a025-cc457881cac9-kube-api-access-gfqkf\") pod \"nova-api-0\" (UID: \"d3222c69-47bc-46d8-a025-cc457881cac9\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:54:04 crc kubenswrapper[5052]: I1203 11:54:04.087904 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/d3222c69-47bc-46d8-a025-cc457881cac9-internal-tls-certs\") pod \"nova-api-0\" (UID: \"d3222c69-47bc-46d8-a025-cc457881cac9\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:54:04 crc kubenswrapper[5052]: I1203 11:54:04.190491 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/d3222c69-47bc-46d8-a025-cc457881cac9-config-data\") pod \"nova-api-0\" (UID: \"d3222c69-47bc-46d8-a025-cc457881cac9\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:54:04 crc kubenswrapper[5052]: I1203 11:54:04.190573 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/d3222c69-47bc-46d8-a025-cc457881cac9-public-tls-certs\") pod \"nova-api-0\" (UID: \"d3222c69-47bc-46d8-a025-cc457881cac9\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:54:04 crc kubenswrapper[5052]: I1203 11:54:04.190650 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d3222c69-47bc-46d8-a025-cc457881cac9-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"d3222c69-47bc-46d8-a025-cc457881cac9\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:54:04 crc kubenswrapper[5052]: I1203 11:54:04.191258 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/d3222c69-47bc-46d8-a025-cc457881cac9-logs\") pod \"nova-api-0\" (UID: \"d3222c69-47bc-46d8-a025-cc457881cac9\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:54:04 crc kubenswrapper[5052]: I1203 11:54:04.191296 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-gfqkf\" (UniqueName: \"kubernetes.io/projected/d3222c69-47bc-46d8-a025-cc457881cac9-kube-api-access-gfqkf\") pod \"nova-api-0\" (UID: \"d3222c69-47bc-46d8-a025-cc457881cac9\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:54:04 crc kubenswrapper[5052]: I1203 11:54:04.191319 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/d3222c69-47bc-46d8-a025-cc457881cac9-internal-tls-certs\") pod \"nova-api-0\" (UID: \"d3222c69-47bc-46d8-a025-cc457881cac9\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:54:04 crc kubenswrapper[5052]: I1203 11:54:04.191806 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/d3222c69-47bc-46d8-a025-cc457881cac9-logs\") pod \"nova-api-0\" (UID: \"d3222c69-47bc-46d8-a025-cc457881cac9\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:54:04 crc kubenswrapper[5052]: I1203 11:54:04.195723 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/d3222c69-47bc-46d8-a025-cc457881cac9-public-tls-certs\") pod \"nova-api-0\" (UID: \"d3222c69-47bc-46d8-a025-cc457881cac9\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:54:04 crc kubenswrapper[5052]: I1203 11:54:04.195817 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d3222c69-47bc-46d8-a025-cc457881cac9-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"d3222c69-47bc-46d8-a025-cc457881cac9\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:54:04 crc kubenswrapper[5052]: I1203 11:54:04.196215 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/d3222c69-47bc-46d8-a025-cc457881cac9-internal-tls-certs\") pod \"nova-api-0\" (UID: \"d3222c69-47bc-46d8-a025-cc457881cac9\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:54:04 crc kubenswrapper[5052]: I1203 11:54:04.196830 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/d3222c69-47bc-46d8-a025-cc457881cac9-config-data\") pod \"nova-api-0\" (UID: \"d3222c69-47bc-46d8-a025-cc457881cac9\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:54:04 crc kubenswrapper[5052]: I1203 11:54:04.208787 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-gfqkf\" (UniqueName: \"kubernetes.io/projected/d3222c69-47bc-46d8-a025-cc457881cac9-kube-api-access-gfqkf\") pod \"nova-api-0\" (UID: \"d3222c69-47bc-46d8-a025-cc457881cac9\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:54:04 crc kubenswrapper[5052]: I1203 11:54:04.250186 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:54:04 crc kubenswrapper[5052]: I1203 11:54:04.732697 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 11:54:04 crc kubenswrapper[5052]: W1203 11:54:04.735392 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd3222c69_47bc_46d8_a025_cc457881cac9.slice/crio-468d819d57751760180e3421d5195178c6b74c7e66737c3521cc9374ec0895b0 WatchSource:0}: Error finding container 468d819d57751760180e3421d5195178c6b74c7e66737c3521cc9374ec0895b0: Status 404 returned error can't find the container with id 468d819d57751760180e3421d5195178c6b74c7e66737c3521cc9374ec0895b0 Dec 03 11:54:04 crc kubenswrapper[5052]: I1203 11:54:04.843772 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"d3222c69-47bc-46d8-a025-cc457881cac9","Type":"ContainerStarted","Data":"468d819d57751760180e3421d5195178c6b74c7e66737c3521cc9374ec0895b0"} Dec 03 11:54:04 crc kubenswrapper[5052]: I1203 11:54:04.846156 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"a5ff0249-173a-4793-8cbe-0aa749f34632","Type":"ContainerStarted","Data":"cf164870d3f804d61f70d5e02392ceff8cd44fdf830fb056d8e5f574d1ef05c1"} Dec 03 11:54:05 crc kubenswrapper[5052]: I1203 11:54:05.689679 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a759a8ef-882c-4eb0-8336-9aa26c80b488" path="/var/lib/kubelet/pods/a759a8ef-882c-4eb0-8336-9aa26c80b488/volumes" Dec 03 11:54:05 crc kubenswrapper[5052]: I1203 11:54:05.878117 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"d3222c69-47bc-46d8-a025-cc457881cac9","Type":"ContainerStarted","Data":"13ed5d128b8c9df19939f5f214d29b30d31dc1296f2560559e840f9d3ffbfd12"} Dec 03 11:54:05 crc kubenswrapper[5052]: I1203 11:54:05.878187 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"d3222c69-47bc-46d8-a025-cc457881cac9","Type":"ContainerStarted","Data":"d7f3ac6af5a2c0d33f118ffd9646c754d018fd85708d55c58a60a5cc00574cd1"} Dec 03 11:54:05 crc kubenswrapper[5052]: I1203 11:54:05.892385 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"a5ff0249-173a-4793-8cbe-0aa749f34632","Type":"ContainerStarted","Data":"14c4e0b558f59a7e38279d4f104332c292bd1c29af9d8ae1768b1a9ec55a8d31"} Dec 03 11:54:05 crc kubenswrapper[5052]: I1203 11:54:05.909321 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-api-0" podStartSLOduration=2.909295377 podStartE2EDuration="2.909295377s" podCreationTimestamp="2025-12-03 11:54:03 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:54:05.896721237 +0000 UTC m=+2036.484963031" watchObservedRunningTime="2025-12-03 11:54:05.909295377 +0000 UTC m=+2036.497537161" Dec 03 11:54:07 crc kubenswrapper[5052]: I1203 11:54:07.909383 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"a5ff0249-173a-4793-8cbe-0aa749f34632","Type":"ContainerStarted","Data":"615dcdb302a4fa77dc8006892c9f0c5509fe1f5ef890116dd6a627f53880b9b8"} Dec 03 11:54:07 crc kubenswrapper[5052]: I1203 11:54:07.910132 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:54:14 crc kubenswrapper[5052]: I1203 11:54:14.252004 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:54:14 crc kubenswrapper[5052]: I1203 11:54:14.252737 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:54:15 crc kubenswrapper[5052]: I1203 11:54:15.271723 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-api-0" podUID="d3222c69-47bc-46d8-a025-cc457881cac9" containerName="nova-api-log" probeResult="failure" output="Get \"https://10.217.1.137:8774/\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Dec 03 11:54:15 crc kubenswrapper[5052]: I1203 11:54:15.271751 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-api-0" podUID="d3222c69-47bc-46d8-a025-cc457881cac9" containerName="nova-api-api" probeResult="failure" output="Get \"https://10.217.1.137:8774/\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Dec 03 11:54:24 crc kubenswrapper[5052]: I1203 11:54:24.258255 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:54:24 crc kubenswrapper[5052]: I1203 11:54:24.260174 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:54:24 crc kubenswrapper[5052]: I1203 11:54:24.260409 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:54:24 crc kubenswrapper[5052]: I1203 11:54:24.267450 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:54:24 crc kubenswrapper[5052]: I1203 11:54:24.388101 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/ceilometer-0" podStartSLOduration=18.932842335 podStartE2EDuration="23.388077707s" podCreationTimestamp="2025-12-03 11:54:01 +0000 UTC" firstStartedPulling="2025-12-03 11:54:02.699488834 +0000 UTC m=+2033.287730598" lastFinishedPulling="2025-12-03 11:54:07.154724196 +0000 UTC m=+2037.742965970" observedRunningTime="2025-12-03 11:54:07.928360397 +0000 UTC m=+2038.516602171" watchObservedRunningTime="2025-12-03 11:54:24.388077707 +0000 UTC m=+2054.976319471" Dec 03 11:54:25 crc kubenswrapper[5052]: I1203 11:54:25.071280 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:54:25 crc kubenswrapper[5052]: I1203 11:54:25.078576 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:54:29 crc kubenswrapper[5052]: I1203 11:54:29.378874 5052 patch_prober.go:28] interesting pod/machine-config-daemon-74bj6 container/machine-config-daemon namespace/openshift-machine-config-operator: Liveness probe status=failure output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" start-of-body= Dec 03 11:54:29 crc kubenswrapper[5052]: I1203 11:54:29.379195 5052 prober.go:107] "Probe failed" probeType="Liveness" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" containerName="machine-config-daemon" probeResult="failure" output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" Dec 03 11:54:29 crc kubenswrapper[5052]: I1203 11:54:29.379238 5052 kubelet.go:2542] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" Dec 03 11:54:29 crc kubenswrapper[5052]: I1203 11:54:29.379945 5052 kuberuntime_manager.go:1027] "Message for Container of pod" containerName="machine-config-daemon" containerStatusID={"Type":"cri-o","ID":"cc4f296896bbf8aa4d2c78b7e4045d3222cac9b2a6ed232c44ec9d7458bb3338"} pod="openshift-machine-config-operator/machine-config-daemon-74bj6" containerMessage="Container machine-config-daemon failed liveness probe, will be restarted" Dec 03 11:54:29 crc kubenswrapper[5052]: I1203 11:54:29.380002 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" containerName="machine-config-daemon" containerID="cri-o://cc4f296896bbf8aa4d2c78b7e4045d3222cac9b2a6ed232c44ec9d7458bb3338" gracePeriod=600 Dec 03 11:54:29 crc kubenswrapper[5052]: E1203 11:54:29.508801 5052 cadvisor_stats_provider.go:516] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10c237bc_72e7_4da3_9f2c_b1688395bf6e.slice/crio-cc4f296896bbf8aa4d2c78b7e4045d3222cac9b2a6ed232c44ec9d7458bb3338.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10c237bc_72e7_4da3_9f2c_b1688395bf6e.slice/crio-conmon-cc4f296896bbf8aa4d2c78b7e4045d3222cac9b2a6ed232c44ec9d7458bb3338.scope\": RecentStats: unable to find data in memory cache]" Dec 03 11:54:30 crc kubenswrapper[5052]: I1203 11:54:30.114403 5052 generic.go:334] "Generic (PLEG): container finished" podID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" containerID="cc4f296896bbf8aa4d2c78b7e4045d3222cac9b2a6ed232c44ec9d7458bb3338" exitCode=0 Dec 03 11:54:30 crc kubenswrapper[5052]: I1203 11:54:30.114499 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" event={"ID":"10c237bc-72e7-4da3-9f2c-b1688395bf6e","Type":"ContainerDied","Data":"cc4f296896bbf8aa4d2c78b7e4045d3222cac9b2a6ed232c44ec9d7458bb3338"} Dec 03 11:54:30 crc kubenswrapper[5052]: I1203 11:54:30.115091 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" event={"ID":"10c237bc-72e7-4da3-9f2c-b1688395bf6e","Type":"ContainerStarted","Data":"d224717c6d9ca8135fd510e03a6c40c9f2b1a56dd8b4a3b685af3e2dfa855982"} Dec 03 11:54:30 crc kubenswrapper[5052]: I1203 11:54:30.115114 5052 scope.go:117] "RemoveContainer" containerID="25fe88f8d102f09983f8884e65f0c6c1575ade67921f10a630e6dcc3ea8b506e" Dec 03 11:54:32 crc kubenswrapper[5052]: I1203 11:54:32.230261 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:54:36 crc kubenswrapper[5052]: I1203 11:54:36.662335 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ovn-controller-9m9bm"] Dec 03 11:54:36 crc kubenswrapper[5052]: I1203 11:54:36.664317 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovn-controller-9m9bm" Dec 03 11:54:36 crc kubenswrapper[5052]: I1203 11:54:36.666676 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ovncontroller-ovncontroller-dockercfg-pmp29" Dec 03 11:54:36 crc kubenswrapper[5052]: I1203 11:54:36.670174 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-ovncontroller-ovndbs" Dec 03 11:54:36 crc kubenswrapper[5052]: I1203 11:54:36.670291 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"ovncontroller-scripts" Dec 03 11:54:36 crc kubenswrapper[5052]: I1203 11:54:36.672358 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ovn-controller-9m9bm"] Dec 03 11:54:36 crc kubenswrapper[5052]: I1203 11:54:36.713661 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ovn-controller-ovs-2bfws"] Dec 03 11:54:36 crc kubenswrapper[5052]: I1203 11:54:36.716106 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovn-controller-ovs-2bfws" Dec 03 11:54:36 crc kubenswrapper[5052]: I1203 11:54:36.727464 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ovn-controller-ovs-2bfws"] Dec 03 11:54:36 crc kubenswrapper[5052]: I1203 11:54:36.743107 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ovn-controller-metrics-mxqpj"] Dec 03 11:54:36 crc kubenswrapper[5052]: I1203 11:54:36.744734 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovn-controller-metrics-mxqpj" Dec 03 11:54:36 crc kubenswrapper[5052]: I1203 11:54:36.746866 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"ovncontroller-metrics-config" Dec 03 11:54:36 crc kubenswrapper[5052]: I1203 11:54:36.757710 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ovn-controller-metrics-mxqpj"] Dec 03 11:54:36 crc kubenswrapper[5052]: I1203 11:54:36.814931 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/477501bc-7bfe-4712-b61e-e566d7e48037-combined-ca-bundle\") pod \"ovn-controller-9m9bm\" (UID: \"477501bc-7bfe-4712-b61e-e566d7e48037\") " pod="openstack-kuttl-tests/ovn-controller-9m9bm" Dec 03 11:54:36 crc kubenswrapper[5052]: I1203 11:54:36.814995 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib\" (UniqueName: \"kubernetes.io/host-path/7f7e6c2a-8c03-435d-9b23-3ed37bb9811d-var-lib\") pod \"ovn-controller-ovs-2bfws\" (UID: \"7f7e6c2a-8c03-435d-9b23-3ed37bb9811d\") " pod="openstack-kuttl-tests/ovn-controller-ovs-2bfws" Dec 03 11:54:36 crc kubenswrapper[5052]: I1203 11:54:36.815021 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-68v7z\" (UniqueName: \"kubernetes.io/projected/7f7e6c2a-8c03-435d-9b23-3ed37bb9811d-kube-api-access-68v7z\") pod \"ovn-controller-ovs-2bfws\" (UID: \"7f7e6c2a-8c03-435d-9b23-3ed37bb9811d\") " pod="openstack-kuttl-tests/ovn-controller-ovs-2bfws" Dec 03 11:54:36 crc kubenswrapper[5052]: I1203 11:54:36.815052 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8cpnv\" (UniqueName: \"kubernetes.io/projected/477501bc-7bfe-4712-b61e-e566d7e48037-kube-api-access-8cpnv\") pod \"ovn-controller-9m9bm\" (UID: \"477501bc-7bfe-4712-b61e-e566d7e48037\") " pod="openstack-kuttl-tests/ovn-controller-9m9bm" Dec 03 11:54:36 crc kubenswrapper[5052]: I1203 11:54:36.815106 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-run\" (UniqueName: \"kubernetes.io/host-path/7f7e6c2a-8c03-435d-9b23-3ed37bb9811d-var-run\") pod \"ovn-controller-ovs-2bfws\" (UID: \"7f7e6c2a-8c03-435d-9b23-3ed37bb9811d\") " pod="openstack-kuttl-tests/ovn-controller-ovs-2bfws" Dec 03 11:54:36 crc kubenswrapper[5052]: I1203 11:54:36.815182 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-log-ovn\" (UniqueName: \"kubernetes.io/host-path/477501bc-7bfe-4712-b61e-e566d7e48037-var-log-ovn\") pod \"ovn-controller-9m9bm\" (UID: \"477501bc-7bfe-4712-b61e-e566d7e48037\") " pod="openstack-kuttl-tests/ovn-controller-9m9bm" Dec 03 11:54:36 crc kubenswrapper[5052]: I1203 11:54:36.815210 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-ovs\" (UniqueName: \"kubernetes.io/host-path/7f7e6c2a-8c03-435d-9b23-3ed37bb9811d-etc-ovs\") pod \"ovn-controller-ovs-2bfws\" (UID: \"7f7e6c2a-8c03-435d-9b23-3ed37bb9811d\") " pod="openstack-kuttl-tests/ovn-controller-ovs-2bfws" Dec 03 11:54:36 crc kubenswrapper[5052]: I1203 11:54:36.815308 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/477501bc-7bfe-4712-b61e-e566d7e48037-scripts\") pod \"ovn-controller-9m9bm\" (UID: \"477501bc-7bfe-4712-b61e-e566d7e48037\") " pod="openstack-kuttl-tests/ovn-controller-9m9bm" Dec 03 11:54:36 crc kubenswrapper[5052]: I1203 11:54:36.815337 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/7f7e6c2a-8c03-435d-9b23-3ed37bb9811d-scripts\") pod \"ovn-controller-ovs-2bfws\" (UID: \"7f7e6c2a-8c03-435d-9b23-3ed37bb9811d\") " pod="openstack-kuttl-tests/ovn-controller-ovs-2bfws" Dec 03 11:54:36 crc kubenswrapper[5052]: I1203 11:54:36.815351 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-controller-tls-certs\" (UniqueName: \"kubernetes.io/secret/477501bc-7bfe-4712-b61e-e566d7e48037-ovn-controller-tls-certs\") pod \"ovn-controller-9m9bm\" (UID: \"477501bc-7bfe-4712-b61e-e566d7e48037\") " pod="openstack-kuttl-tests/ovn-controller-9m9bm" Dec 03 11:54:36 crc kubenswrapper[5052]: I1203 11:54:36.815387 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-run\" (UniqueName: \"kubernetes.io/host-path/477501bc-7bfe-4712-b61e-e566d7e48037-var-run\") pod \"ovn-controller-9m9bm\" (UID: \"477501bc-7bfe-4712-b61e-e566d7e48037\") " pod="openstack-kuttl-tests/ovn-controller-9m9bm" Dec 03 11:54:36 crc kubenswrapper[5052]: I1203 11:54:36.815422 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-log\" (UniqueName: \"kubernetes.io/host-path/7f7e6c2a-8c03-435d-9b23-3ed37bb9811d-var-log\") pod \"ovn-controller-ovs-2bfws\" (UID: \"7f7e6c2a-8c03-435d-9b23-3ed37bb9811d\") " pod="openstack-kuttl-tests/ovn-controller-ovs-2bfws" Dec 03 11:54:36 crc kubenswrapper[5052]: I1203 11:54:36.815483 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-run-ovn\" (UniqueName: \"kubernetes.io/host-path/477501bc-7bfe-4712-b61e-e566d7e48037-var-run-ovn\") pod \"ovn-controller-9m9bm\" (UID: \"477501bc-7bfe-4712-b61e-e566d7e48037\") " pod="openstack-kuttl-tests/ovn-controller-9m9bm" Dec 03 11:54:36 crc kubenswrapper[5052]: I1203 11:54:36.917314 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/477501bc-7bfe-4712-b61e-e566d7e48037-combined-ca-bundle\") pod \"ovn-controller-9m9bm\" (UID: \"477501bc-7bfe-4712-b61e-e566d7e48037\") " pod="openstack-kuttl-tests/ovn-controller-9m9bm" Dec 03 11:54:36 crc kubenswrapper[5052]: I1203 11:54:36.917372 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"var-lib\" (UniqueName: \"kubernetes.io/host-path/7f7e6c2a-8c03-435d-9b23-3ed37bb9811d-var-lib\") pod \"ovn-controller-ovs-2bfws\" (UID: \"7f7e6c2a-8c03-435d-9b23-3ed37bb9811d\") " pod="openstack-kuttl-tests/ovn-controller-ovs-2bfws" Dec 03 11:54:36 crc kubenswrapper[5052]: I1203 11:54:36.917401 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-68v7z\" (UniqueName: \"kubernetes.io/projected/7f7e6c2a-8c03-435d-9b23-3ed37bb9811d-kube-api-access-68v7z\") pod \"ovn-controller-ovs-2bfws\" (UID: \"7f7e6c2a-8c03-435d-9b23-3ed37bb9811d\") " pod="openstack-kuttl-tests/ovn-controller-ovs-2bfws" Dec 03 11:54:36 crc kubenswrapper[5052]: I1203 11:54:36.917427 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovs-rundir\" (UniqueName: \"kubernetes.io/host-path/38e74be1-166e-49b3-9a4d-049f44e0547c-ovs-rundir\") pod \"ovn-controller-metrics-mxqpj\" (UID: \"38e74be1-166e-49b3-9a4d-049f44e0547c\") " pod="openstack-kuttl-tests/ovn-controller-metrics-mxqpj" Dec 03 11:54:36 crc kubenswrapper[5052]: I1203 11:54:36.917452 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-8cpnv\" (UniqueName: \"kubernetes.io/projected/477501bc-7bfe-4712-b61e-e566d7e48037-kube-api-access-8cpnv\") pod \"ovn-controller-9m9bm\" (UID: \"477501bc-7bfe-4712-b61e-e566d7e48037\") " pod="openstack-kuttl-tests/ovn-controller-9m9bm" Dec 03 11:54:36 crc kubenswrapper[5052]: I1203 11:54:36.917472 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-rundir\" (UniqueName: \"kubernetes.io/host-path/38e74be1-166e-49b3-9a4d-049f44e0547c-ovn-rundir\") pod \"ovn-controller-metrics-mxqpj\" (UID: \"38e74be1-166e-49b3-9a4d-049f44e0547c\") " pod="openstack-kuttl-tests/ovn-controller-metrics-mxqpj" Dec 03 11:54:36 crc kubenswrapper[5052]: I1203 11:54:36.917535 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"var-run\" (UniqueName: \"kubernetes.io/host-path/7f7e6c2a-8c03-435d-9b23-3ed37bb9811d-var-run\") pod \"ovn-controller-ovs-2bfws\" (UID: \"7f7e6c2a-8c03-435d-9b23-3ed37bb9811d\") " pod="openstack-kuttl-tests/ovn-controller-ovs-2bfws" Dec 03 11:54:36 crc kubenswrapper[5052]: I1203 11:54:36.917554 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"var-log-ovn\" (UniqueName: \"kubernetes.io/host-path/477501bc-7bfe-4712-b61e-e566d7e48037-var-log-ovn\") pod \"ovn-controller-9m9bm\" (UID: \"477501bc-7bfe-4712-b61e-e566d7e48037\") " pod="openstack-kuttl-tests/ovn-controller-9m9bm" Dec 03 11:54:36 crc kubenswrapper[5052]: I1203 11:54:36.917581 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/38e74be1-166e-49b3-9a4d-049f44e0547c-metrics-certs-tls-certs\") pod \"ovn-controller-metrics-mxqpj\" (UID: \"38e74be1-166e-49b3-9a4d-049f44e0547c\") " pod="openstack-kuttl-tests/ovn-controller-metrics-mxqpj" Dec 03 11:54:36 crc kubenswrapper[5052]: I1203 11:54:36.917603 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-ovs\" (UniqueName: \"kubernetes.io/host-path/7f7e6c2a-8c03-435d-9b23-3ed37bb9811d-etc-ovs\") pod \"ovn-controller-ovs-2bfws\" (UID: \"7f7e6c2a-8c03-435d-9b23-3ed37bb9811d\") " pod="openstack-kuttl-tests/ovn-controller-ovs-2bfws" Dec 03 11:54:36 crc kubenswrapper[5052]: I1203 11:54:36.917624 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8dhhg\" (UniqueName: \"kubernetes.io/projected/38e74be1-166e-49b3-9a4d-049f44e0547c-kube-api-access-8dhhg\") pod \"ovn-controller-metrics-mxqpj\" (UID: \"38e74be1-166e-49b3-9a4d-049f44e0547c\") " pod="openstack-kuttl-tests/ovn-controller-metrics-mxqpj" Dec 03 11:54:36 crc kubenswrapper[5052]: I1203 11:54:36.917838 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"var-run\" (UniqueName: \"kubernetes.io/host-path/7f7e6c2a-8c03-435d-9b23-3ed37bb9811d-var-run\") pod \"ovn-controller-ovs-2bfws\" (UID: \"7f7e6c2a-8c03-435d-9b23-3ed37bb9811d\") " pod="openstack-kuttl-tests/ovn-controller-ovs-2bfws" Dec 03 11:54:36 crc kubenswrapper[5052]: I1203 11:54:36.917839 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"var-log-ovn\" (UniqueName: \"kubernetes.io/host-path/477501bc-7bfe-4712-b61e-e566d7e48037-var-log-ovn\") pod \"ovn-controller-9m9bm\" (UID: \"477501bc-7bfe-4712-b61e-e566d7e48037\") " pod="openstack-kuttl-tests/ovn-controller-9m9bm" Dec 03 11:54:36 crc kubenswrapper[5052]: I1203 11:54:36.917852 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/477501bc-7bfe-4712-b61e-e566d7e48037-scripts\") pod \"ovn-controller-9m9bm\" (UID: \"477501bc-7bfe-4712-b61e-e566d7e48037\") " pod="openstack-kuttl-tests/ovn-controller-9m9bm" Dec 03 11:54:36 crc kubenswrapper[5052]: I1203 11:54:36.917856 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"var-lib\" (UniqueName: \"kubernetes.io/host-path/7f7e6c2a-8c03-435d-9b23-3ed37bb9811d-var-lib\") pod \"ovn-controller-ovs-2bfws\" (UID: \"7f7e6c2a-8c03-435d-9b23-3ed37bb9811d\") " pod="openstack-kuttl-tests/ovn-controller-ovs-2bfws" Dec 03 11:54:36 crc kubenswrapper[5052]: I1203 11:54:36.917930 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/7f7e6c2a-8c03-435d-9b23-3ed37bb9811d-scripts\") pod \"ovn-controller-ovs-2bfws\" (UID: \"7f7e6c2a-8c03-435d-9b23-3ed37bb9811d\") " pod="openstack-kuttl-tests/ovn-controller-ovs-2bfws" Dec 03 11:54:36 crc kubenswrapper[5052]: I1203 11:54:36.917957 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"etc-ovs\" (UniqueName: \"kubernetes.io/host-path/7f7e6c2a-8c03-435d-9b23-3ed37bb9811d-etc-ovs\") pod \"ovn-controller-ovs-2bfws\" (UID: \"7f7e6c2a-8c03-435d-9b23-3ed37bb9811d\") " pod="openstack-kuttl-tests/ovn-controller-ovs-2bfws" Dec 03 11:54:36 crc kubenswrapper[5052]: I1203 11:54:36.917969 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovn-controller-tls-certs\" (UniqueName: \"kubernetes.io/secret/477501bc-7bfe-4712-b61e-e566d7e48037-ovn-controller-tls-certs\") pod \"ovn-controller-9m9bm\" (UID: \"477501bc-7bfe-4712-b61e-e566d7e48037\") " pod="openstack-kuttl-tests/ovn-controller-9m9bm" Dec 03 11:54:36 crc kubenswrapper[5052]: I1203 11:54:36.918132 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"var-run\" (UniqueName: \"kubernetes.io/host-path/477501bc-7bfe-4712-b61e-e566d7e48037-var-run\") pod \"ovn-controller-9m9bm\" (UID: \"477501bc-7bfe-4712-b61e-e566d7e48037\") " pod="openstack-kuttl-tests/ovn-controller-9m9bm" Dec 03 11:54:36 crc kubenswrapper[5052]: I1203 11:54:36.918209 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"var-log\" (UniqueName: \"kubernetes.io/host-path/7f7e6c2a-8c03-435d-9b23-3ed37bb9811d-var-log\") pod \"ovn-controller-ovs-2bfws\" (UID: \"7f7e6c2a-8c03-435d-9b23-3ed37bb9811d\") " pod="openstack-kuttl-tests/ovn-controller-ovs-2bfws" Dec 03 11:54:36 crc kubenswrapper[5052]: I1203 11:54:36.918256 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"var-run\" (UniqueName: \"kubernetes.io/host-path/477501bc-7bfe-4712-b61e-e566d7e48037-var-run\") pod \"ovn-controller-9m9bm\" (UID: \"477501bc-7bfe-4712-b61e-e566d7e48037\") " pod="openstack-kuttl-tests/ovn-controller-9m9bm" Dec 03 11:54:36 crc kubenswrapper[5052]: I1203 11:54:36.918307 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"var-run-ovn\" (UniqueName: \"kubernetes.io/host-path/477501bc-7bfe-4712-b61e-e566d7e48037-var-run-ovn\") pod \"ovn-controller-9m9bm\" (UID: \"477501bc-7bfe-4712-b61e-e566d7e48037\") " pod="openstack-kuttl-tests/ovn-controller-9m9bm" Dec 03 11:54:36 crc kubenswrapper[5052]: I1203 11:54:36.918347 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"var-log\" (UniqueName: \"kubernetes.io/host-path/7f7e6c2a-8c03-435d-9b23-3ed37bb9811d-var-log\") pod \"ovn-controller-ovs-2bfws\" (UID: \"7f7e6c2a-8c03-435d-9b23-3ed37bb9811d\") " pod="openstack-kuttl-tests/ovn-controller-ovs-2bfws" Dec 03 11:54:36 crc kubenswrapper[5052]: I1203 11:54:36.918411 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"var-run-ovn\" (UniqueName: \"kubernetes.io/host-path/477501bc-7bfe-4712-b61e-e566d7e48037-var-run-ovn\") pod \"ovn-controller-9m9bm\" (UID: \"477501bc-7bfe-4712-b61e-e566d7e48037\") " pod="openstack-kuttl-tests/ovn-controller-9m9bm" Dec 03 11:54:36 crc kubenswrapper[5052]: I1203 11:54:36.918429 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/38e74be1-166e-49b3-9a4d-049f44e0547c-config\") pod \"ovn-controller-metrics-mxqpj\" (UID: \"38e74be1-166e-49b3-9a4d-049f44e0547c\") " pod="openstack-kuttl-tests/ovn-controller-metrics-mxqpj" Dec 03 11:54:36 crc kubenswrapper[5052]: I1203 11:54:36.918469 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/38e74be1-166e-49b3-9a4d-049f44e0547c-combined-ca-bundle\") pod \"ovn-controller-metrics-mxqpj\" (UID: \"38e74be1-166e-49b3-9a4d-049f44e0547c\") " pod="openstack-kuttl-tests/ovn-controller-metrics-mxqpj" Dec 03 11:54:36 crc kubenswrapper[5052]: I1203 11:54:36.919966 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/477501bc-7bfe-4712-b61e-e566d7e48037-scripts\") pod \"ovn-controller-9m9bm\" (UID: \"477501bc-7bfe-4712-b61e-e566d7e48037\") " pod="openstack-kuttl-tests/ovn-controller-9m9bm" Dec 03 11:54:36 crc kubenswrapper[5052]: I1203 11:54:36.920045 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/7f7e6c2a-8c03-435d-9b23-3ed37bb9811d-scripts\") pod \"ovn-controller-ovs-2bfws\" (UID: \"7f7e6c2a-8c03-435d-9b23-3ed37bb9811d\") " pod="openstack-kuttl-tests/ovn-controller-ovs-2bfws" Dec 03 11:54:36 crc kubenswrapper[5052]: I1203 11:54:36.923368 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/477501bc-7bfe-4712-b61e-e566d7e48037-combined-ca-bundle\") pod \"ovn-controller-9m9bm\" (UID: \"477501bc-7bfe-4712-b61e-e566d7e48037\") " pod="openstack-kuttl-tests/ovn-controller-9m9bm" Dec 03 11:54:36 crc kubenswrapper[5052]: I1203 11:54:36.923867 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovn-controller-tls-certs\" (UniqueName: \"kubernetes.io/secret/477501bc-7bfe-4712-b61e-e566d7e48037-ovn-controller-tls-certs\") pod \"ovn-controller-9m9bm\" (UID: \"477501bc-7bfe-4712-b61e-e566d7e48037\") " pod="openstack-kuttl-tests/ovn-controller-9m9bm" Dec 03 11:54:36 crc kubenswrapper[5052]: I1203 11:54:36.935498 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-8cpnv\" (UniqueName: \"kubernetes.io/projected/477501bc-7bfe-4712-b61e-e566d7e48037-kube-api-access-8cpnv\") pod \"ovn-controller-9m9bm\" (UID: \"477501bc-7bfe-4712-b61e-e566d7e48037\") " pod="openstack-kuttl-tests/ovn-controller-9m9bm" Dec 03 11:54:36 crc kubenswrapper[5052]: I1203 11:54:36.936196 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-68v7z\" (UniqueName: \"kubernetes.io/projected/7f7e6c2a-8c03-435d-9b23-3ed37bb9811d-kube-api-access-68v7z\") pod \"ovn-controller-ovs-2bfws\" (UID: \"7f7e6c2a-8c03-435d-9b23-3ed37bb9811d\") " pod="openstack-kuttl-tests/ovn-controller-ovs-2bfws" Dec 03 11:54:36 crc kubenswrapper[5052]: I1203 11:54:36.985683 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovn-controller-9m9bm" Dec 03 11:54:37 crc kubenswrapper[5052]: I1203 11:54:37.020217 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/38e74be1-166e-49b3-9a4d-049f44e0547c-metrics-certs-tls-certs\") pod \"ovn-controller-metrics-mxqpj\" (UID: \"38e74be1-166e-49b3-9a4d-049f44e0547c\") " pod="openstack-kuttl-tests/ovn-controller-metrics-mxqpj" Dec 03 11:54:37 crc kubenswrapper[5052]: I1203 11:54:37.020630 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-8dhhg\" (UniqueName: \"kubernetes.io/projected/38e74be1-166e-49b3-9a4d-049f44e0547c-kube-api-access-8dhhg\") pod \"ovn-controller-metrics-mxqpj\" (UID: \"38e74be1-166e-49b3-9a4d-049f44e0547c\") " pod="openstack-kuttl-tests/ovn-controller-metrics-mxqpj" Dec 03 11:54:37 crc kubenswrapper[5052]: I1203 11:54:37.020917 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/38e74be1-166e-49b3-9a4d-049f44e0547c-config\") pod \"ovn-controller-metrics-mxqpj\" (UID: \"38e74be1-166e-49b3-9a4d-049f44e0547c\") " pod="openstack-kuttl-tests/ovn-controller-metrics-mxqpj" Dec 03 11:54:37 crc kubenswrapper[5052]: I1203 11:54:37.020947 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/38e74be1-166e-49b3-9a4d-049f44e0547c-combined-ca-bundle\") pod \"ovn-controller-metrics-mxqpj\" (UID: \"38e74be1-166e-49b3-9a4d-049f44e0547c\") " pod="openstack-kuttl-tests/ovn-controller-metrics-mxqpj" Dec 03 11:54:37 crc kubenswrapper[5052]: I1203 11:54:37.021053 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovs-rundir\" (UniqueName: \"kubernetes.io/host-path/38e74be1-166e-49b3-9a4d-049f44e0547c-ovs-rundir\") pod \"ovn-controller-metrics-mxqpj\" (UID: \"38e74be1-166e-49b3-9a4d-049f44e0547c\") " pod="openstack-kuttl-tests/ovn-controller-metrics-mxqpj" Dec 03 11:54:37 crc kubenswrapper[5052]: I1203 11:54:37.021085 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovn-rundir\" (UniqueName: \"kubernetes.io/host-path/38e74be1-166e-49b3-9a4d-049f44e0547c-ovn-rundir\") pod \"ovn-controller-metrics-mxqpj\" (UID: \"38e74be1-166e-49b3-9a4d-049f44e0547c\") " pod="openstack-kuttl-tests/ovn-controller-metrics-mxqpj" Dec 03 11:54:37 crc kubenswrapper[5052]: I1203 11:54:37.021340 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovn-rundir\" (UniqueName: \"kubernetes.io/host-path/38e74be1-166e-49b3-9a4d-049f44e0547c-ovn-rundir\") pod \"ovn-controller-metrics-mxqpj\" (UID: \"38e74be1-166e-49b3-9a4d-049f44e0547c\") " pod="openstack-kuttl-tests/ovn-controller-metrics-mxqpj" Dec 03 11:54:37 crc kubenswrapper[5052]: I1203 11:54:37.021493 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/38e74be1-166e-49b3-9a4d-049f44e0547c-config\") pod \"ovn-controller-metrics-mxqpj\" (UID: \"38e74be1-166e-49b3-9a4d-049f44e0547c\") " pod="openstack-kuttl-tests/ovn-controller-metrics-mxqpj" Dec 03 11:54:37 crc kubenswrapper[5052]: I1203 11:54:37.021580 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovs-rundir\" (UniqueName: \"kubernetes.io/host-path/38e74be1-166e-49b3-9a4d-049f44e0547c-ovs-rundir\") pod \"ovn-controller-metrics-mxqpj\" (UID: \"38e74be1-166e-49b3-9a4d-049f44e0547c\") " pod="openstack-kuttl-tests/ovn-controller-metrics-mxqpj" Dec 03 11:54:37 crc kubenswrapper[5052]: I1203 11:54:37.026077 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/38e74be1-166e-49b3-9a4d-049f44e0547c-combined-ca-bundle\") pod \"ovn-controller-metrics-mxqpj\" (UID: \"38e74be1-166e-49b3-9a4d-049f44e0547c\") " pod="openstack-kuttl-tests/ovn-controller-metrics-mxqpj" Dec 03 11:54:37 crc kubenswrapper[5052]: I1203 11:54:37.026951 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/38e74be1-166e-49b3-9a4d-049f44e0547c-metrics-certs-tls-certs\") pod \"ovn-controller-metrics-mxqpj\" (UID: \"38e74be1-166e-49b3-9a4d-049f44e0547c\") " pod="openstack-kuttl-tests/ovn-controller-metrics-mxqpj" Dec 03 11:54:37 crc kubenswrapper[5052]: I1203 11:54:37.037087 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-8dhhg\" (UniqueName: \"kubernetes.io/projected/38e74be1-166e-49b3-9a4d-049f44e0547c-kube-api-access-8dhhg\") pod \"ovn-controller-metrics-mxqpj\" (UID: \"38e74be1-166e-49b3-9a4d-049f44e0547c\") " pod="openstack-kuttl-tests/ovn-controller-metrics-mxqpj" Dec 03 11:54:37 crc kubenswrapper[5052]: I1203 11:54:37.046739 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovn-controller-ovs-2bfws" Dec 03 11:54:37 crc kubenswrapper[5052]: I1203 11:54:37.079767 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovn-controller-metrics-mxqpj" Dec 03 11:54:37 crc kubenswrapper[5052]: I1203 11:54:37.492445 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ovn-controller-9m9bm"] Dec 03 11:54:37 crc kubenswrapper[5052]: I1203 11:54:37.620379 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ovn-controller-ovs-2bfws"] Dec 03 11:54:37 crc kubenswrapper[5052]: I1203 11:54:37.691390 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ovn-controller-metrics-mxqpj"] Dec 03 11:54:38 crc kubenswrapper[5052]: I1203 11:54:38.198441 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovn-controller-ovs-2bfws" event={"ID":"7f7e6c2a-8c03-435d-9b23-3ed37bb9811d","Type":"ContainerStarted","Data":"0ba3cd3025da56ec1901b743f3c0e26307ba5791d78061bb75be7dee0bcd180e"} Dec 03 11:54:38 crc kubenswrapper[5052]: I1203 11:54:38.199909 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovn-controller-9m9bm" event={"ID":"477501bc-7bfe-4712-b61e-e566d7e48037","Type":"ContainerStarted","Data":"300518f1414f335214df24fec878cd06f2faa15b94ba1919c6de55720ae97c5c"} Dec 03 11:54:38 crc kubenswrapper[5052]: I1203 11:54:38.201395 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovn-controller-metrics-mxqpj" event={"ID":"38e74be1-166e-49b3-9a4d-049f44e0547c","Type":"ContainerStarted","Data":"fb1248c42fdbb10898fe6eda4d8ab3743aa4fd6cf625e7cc6a5e821f76e58c40"} Dec 03 11:54:39 crc kubenswrapper[5052]: I1203 11:54:39.214889 5052 generic.go:334] "Generic (PLEG): container finished" podID="7f7e6c2a-8c03-435d-9b23-3ed37bb9811d" containerID="5329302d29b19e4f5f394764bc11bf1db401d466936911460100a0ce63b692d6" exitCode=0 Dec 03 11:54:39 crc kubenswrapper[5052]: I1203 11:54:39.214979 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovn-controller-ovs-2bfws" event={"ID":"7f7e6c2a-8c03-435d-9b23-3ed37bb9811d","Type":"ContainerDied","Data":"5329302d29b19e4f5f394764bc11bf1db401d466936911460100a0ce63b692d6"} Dec 03 11:54:39 crc kubenswrapper[5052]: I1203 11:54:39.219825 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovn-controller-9m9bm" event={"ID":"477501bc-7bfe-4712-b61e-e566d7e48037","Type":"ContainerStarted","Data":"511f73e19426246267775dc4625ae7c7af245dceb1e0d4269c332092b8770f2e"} Dec 03 11:54:39 crc kubenswrapper[5052]: I1203 11:54:39.219906 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/ovn-controller-9m9bm" Dec 03 11:54:39 crc kubenswrapper[5052]: I1203 11:54:39.226976 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovn-controller-metrics-mxqpj" event={"ID":"38e74be1-166e-49b3-9a4d-049f44e0547c","Type":"ContainerStarted","Data":"d27c93cd537043ea03f8b08722fb0c631f236714bb3ab51fee417f235f65e84b"} Dec 03 11:54:39 crc kubenswrapper[5052]: I1203 11:54:39.272473 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/ovn-controller-metrics-mxqpj" podStartSLOduration=3.272448655 podStartE2EDuration="3.272448655s" podCreationTimestamp="2025-12-03 11:54:36 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:54:39.25656625 +0000 UTC m=+2069.844808034" watchObservedRunningTime="2025-12-03 11:54:39.272448655 +0000 UTC m=+2069.860690419" Dec 03 11:54:39 crc kubenswrapper[5052]: I1203 11:54:39.306179 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/ovn-controller-9m9bm" podStartSLOduration=3.30615718 podStartE2EDuration="3.30615718s" podCreationTimestamp="2025-12-03 11:54:36 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:54:39.275829942 +0000 UTC m=+2069.864071716" watchObservedRunningTime="2025-12-03 11:54:39.30615718 +0000 UTC m=+2069.894398944" Dec 03 11:54:40 crc kubenswrapper[5052]: I1203 11:54:40.238868 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovn-controller-ovs-2bfws" event={"ID":"7f7e6c2a-8c03-435d-9b23-3ed37bb9811d","Type":"ContainerStarted","Data":"555a48fb112549936fa017dafd0bd2b3a6535396cbeaebfeacc9a34ebb837f35"} Dec 03 11:54:40 crc kubenswrapper[5052]: I1203 11:54:40.239439 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovn-controller-ovs-2bfws" event={"ID":"7f7e6c2a-8c03-435d-9b23-3ed37bb9811d","Type":"ContainerStarted","Data":"b114369f17334b3461585b8e213c8285e8ea520f731eeea65e56b8a04d08702b"} Dec 03 11:54:40 crc kubenswrapper[5052]: I1203 11:54:40.264441 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/ovn-controller-ovs-2bfws" podStartSLOduration=4.264422817 podStartE2EDuration="4.264422817s" podCreationTimestamp="2025-12-03 11:54:36 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:54:40.263432339 +0000 UTC m=+2070.851674103" watchObservedRunningTime="2025-12-03 11:54:40.264422817 +0000 UTC m=+2070.852664581" Dec 03 11:54:40 crc kubenswrapper[5052]: I1203 11:54:40.917993 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ovn-controller-ovs-2bfws"] Dec 03 11:54:40 crc kubenswrapper[5052]: I1203 11:54:40.926927 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ovn-controller-9m9bm"] Dec 03 11:54:40 crc kubenswrapper[5052]: I1203 11:54:40.934924 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ovn-controller-metrics-mxqpj"] Dec 03 11:54:41 crc kubenswrapper[5052]: I1203 11:54:41.246554 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ovn-controller-metrics-mxqpj" podUID="38e74be1-166e-49b3-9a4d-049f44e0547c" containerName="openstack-network-exporter" containerID="cri-o://d27c93cd537043ea03f8b08722fb0c631f236714bb3ab51fee417f235f65e84b" gracePeriod=30 Dec 03 11:54:41 crc kubenswrapper[5052]: I1203 11:54:41.247053 5052 kubelet_pods.go:1007] "Unable to retrieve pull secret, the image pull may not succeed." pod="openstack-kuttl-tests/ovn-controller-ovs-2bfws" secret="" err="secret \"ovncontroller-ovncontroller-dockercfg-pmp29\" not found" Dec 03 11:54:41 crc kubenswrapper[5052]: I1203 11:54:41.247280 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/ovn-controller-ovs-2bfws" Dec 03 11:54:41 crc kubenswrapper[5052]: I1203 11:54:41.247691 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/ovn-controller-ovs-2bfws" Dec 03 11:54:41 crc kubenswrapper[5052]: E1203 11:54:41.320088 5052 handlers.go:78] "Exec lifecycle hook for Container in Pod failed" err="command '/usr/share/ovn/scripts/ovn-ctl stop_controller' exited with 137: " execCommand=["/usr/share/ovn/scripts/ovn-ctl","stop_controller"] containerName="ovn-controller" pod="openstack-kuttl-tests/ovn-controller-9m9bm" message="Exiting ovn-controller (1) " Dec 03 11:54:41 crc kubenswrapper[5052]: E1203 11:54:41.320129 5052 kuberuntime_container.go:691] "PreStop hook failed" err="command '/usr/share/ovn/scripts/ovn-ctl stop_controller' exited with 137: " pod="openstack-kuttl-tests/ovn-controller-9m9bm" podUID="477501bc-7bfe-4712-b61e-e566d7e48037" containerName="ovn-controller" containerID="cri-o://511f73e19426246267775dc4625ae7c7af245dceb1e0d4269c332092b8770f2e" Dec 03 11:54:41 crc kubenswrapper[5052]: I1203 11:54:41.320175 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ovn-controller-9m9bm" podUID="477501bc-7bfe-4712-b61e-e566d7e48037" containerName="ovn-controller" containerID="cri-o://511f73e19426246267775dc4625ae7c7af245dceb1e0d4269c332092b8770f2e" gracePeriod=30 Dec 03 11:54:41 crc kubenswrapper[5052]: E1203 11:54:41.419322 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/ovncontroller-scripts: configmap "ovncontroller-scripts" not found Dec 03 11:54:41 crc kubenswrapper[5052]: E1203 11:54:41.419434 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7f7e6c2a-8c03-435d-9b23-3ed37bb9811d-scripts podName:7f7e6c2a-8c03-435d-9b23-3ed37bb9811d nodeName:}" failed. No retries permitted until 2025-12-03 11:54:41.919411707 +0000 UTC m=+2072.507653561 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "scripts" (UniqueName: "kubernetes.io/configmap/7f7e6c2a-8c03-435d-9b23-3ed37bb9811d-scripts") pod "ovn-controller-ovs-2bfws" (UID: "7f7e6c2a-8c03-435d-9b23-3ed37bb9811d") : configmap "ovncontroller-scripts" not found Dec 03 11:54:41 crc kubenswrapper[5052]: I1203 11:54:41.810789 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovn-controller-9m9bm" Dec 03 11:54:41 crc kubenswrapper[5052]: I1203 11:54:41.819294 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_ovn-controller-metrics-mxqpj_38e74be1-166e-49b3-9a4d-049f44e0547c/openstack-network-exporter/0.log" Dec 03 11:54:41 crc kubenswrapper[5052]: I1203 11:54:41.819344 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovn-controller-metrics-mxqpj" Dec 03 11:54:41 crc kubenswrapper[5052]: I1203 11:54:41.930321 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"var-run-ovn\" (UniqueName: \"kubernetes.io/host-path/477501bc-7bfe-4712-b61e-e566d7e48037-var-run-ovn\") pod \"477501bc-7bfe-4712-b61e-e566d7e48037\" (UID: \"477501bc-7bfe-4712-b61e-e566d7e48037\") " Dec 03 11:54:41 crc kubenswrapper[5052]: I1203 11:54:41.930362 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"var-run\" (UniqueName: \"kubernetes.io/host-path/477501bc-7bfe-4712-b61e-e566d7e48037-var-run\") pod \"477501bc-7bfe-4712-b61e-e566d7e48037\" (UID: \"477501bc-7bfe-4712-b61e-e566d7e48037\") " Dec 03 11:54:41 crc kubenswrapper[5052]: I1203 11:54:41.930436 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/38e74be1-166e-49b3-9a4d-049f44e0547c-combined-ca-bundle\") pod \"38e74be1-166e-49b3-9a4d-049f44e0547c\" (UID: \"38e74be1-166e-49b3-9a4d-049f44e0547c\") " Dec 03 11:54:41 crc kubenswrapper[5052]: I1203 11:54:41.930445 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/477501bc-7bfe-4712-b61e-e566d7e48037-var-run-ovn" (OuterVolumeSpecName: "var-run-ovn") pod "477501bc-7bfe-4712-b61e-e566d7e48037" (UID: "477501bc-7bfe-4712-b61e-e566d7e48037"). InnerVolumeSpecName "var-run-ovn". PluginName "kubernetes.io/host-path", VolumeGidValue "" Dec 03 11:54:41 crc kubenswrapper[5052]: I1203 11:54:41.930477 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/38e74be1-166e-49b3-9a4d-049f44e0547c-metrics-certs-tls-certs\") pod \"38e74be1-166e-49b3-9a4d-049f44e0547c\" (UID: \"38e74be1-166e-49b3-9a4d-049f44e0547c\") " Dec 03 11:54:41 crc kubenswrapper[5052]: I1203 11:54:41.930540 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovs-rundir\" (UniqueName: \"kubernetes.io/host-path/38e74be1-166e-49b3-9a4d-049f44e0547c-ovs-rundir\") pod \"38e74be1-166e-49b3-9a4d-049f44e0547c\" (UID: \"38e74be1-166e-49b3-9a4d-049f44e0547c\") " Dec 03 11:54:41 crc kubenswrapper[5052]: I1203 11:54:41.930560 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovn-rundir\" (UniqueName: \"kubernetes.io/host-path/38e74be1-166e-49b3-9a4d-049f44e0547c-ovn-rundir\") pod \"38e74be1-166e-49b3-9a4d-049f44e0547c\" (UID: \"38e74be1-166e-49b3-9a4d-049f44e0547c\") " Dec 03 11:54:41 crc kubenswrapper[5052]: I1203 11:54:41.930571 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/477501bc-7bfe-4712-b61e-e566d7e48037-var-run" (OuterVolumeSpecName: "var-run") pod "477501bc-7bfe-4712-b61e-e566d7e48037" (UID: "477501bc-7bfe-4712-b61e-e566d7e48037"). InnerVolumeSpecName "var-run". PluginName "kubernetes.io/host-path", VolumeGidValue "" Dec 03 11:54:41 crc kubenswrapper[5052]: I1203 11:54:41.930635 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/38e74be1-166e-49b3-9a4d-049f44e0547c-ovs-rundir" (OuterVolumeSpecName: "ovs-rundir") pod "38e74be1-166e-49b3-9a4d-049f44e0547c" (UID: "38e74be1-166e-49b3-9a4d-049f44e0547c"). InnerVolumeSpecName "ovs-rundir". PluginName "kubernetes.io/host-path", VolumeGidValue "" Dec 03 11:54:41 crc kubenswrapper[5052]: I1203 11:54:41.930727 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/38e74be1-166e-49b3-9a4d-049f44e0547c-ovn-rundir" (OuterVolumeSpecName: "ovn-rundir") pod "38e74be1-166e-49b3-9a4d-049f44e0547c" (UID: "38e74be1-166e-49b3-9a4d-049f44e0547c"). InnerVolumeSpecName "ovn-rundir". PluginName "kubernetes.io/host-path", VolumeGidValue "" Dec 03 11:54:41 crc kubenswrapper[5052]: I1203 11:54:41.930782 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"var-log-ovn\" (UniqueName: \"kubernetes.io/host-path/477501bc-7bfe-4712-b61e-e566d7e48037-var-log-ovn\") pod \"477501bc-7bfe-4712-b61e-e566d7e48037\" (UID: \"477501bc-7bfe-4712-b61e-e566d7e48037\") " Dec 03 11:54:41 crc kubenswrapper[5052]: I1203 11:54:41.930813 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovn-controller-tls-certs\" (UniqueName: \"kubernetes.io/secret/477501bc-7bfe-4712-b61e-e566d7e48037-ovn-controller-tls-certs\") pod \"477501bc-7bfe-4712-b61e-e566d7e48037\" (UID: \"477501bc-7bfe-4712-b61e-e566d7e48037\") " Dec 03 11:54:41 crc kubenswrapper[5052]: I1203 11:54:41.931326 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8cpnv\" (UniqueName: \"kubernetes.io/projected/477501bc-7bfe-4712-b61e-e566d7e48037-kube-api-access-8cpnv\") pod \"477501bc-7bfe-4712-b61e-e566d7e48037\" (UID: \"477501bc-7bfe-4712-b61e-e566d7e48037\") " Dec 03 11:54:41 crc kubenswrapper[5052]: I1203 11:54:41.931345 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8dhhg\" (UniqueName: \"kubernetes.io/projected/38e74be1-166e-49b3-9a4d-049f44e0547c-kube-api-access-8dhhg\") pod \"38e74be1-166e-49b3-9a4d-049f44e0547c\" (UID: \"38e74be1-166e-49b3-9a4d-049f44e0547c\") " Dec 03 11:54:41 crc kubenswrapper[5052]: I1203 11:54:41.930853 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/477501bc-7bfe-4712-b61e-e566d7e48037-var-log-ovn" (OuterVolumeSpecName: "var-log-ovn") pod "477501bc-7bfe-4712-b61e-e566d7e48037" (UID: "477501bc-7bfe-4712-b61e-e566d7e48037"). InnerVolumeSpecName "var-log-ovn". PluginName "kubernetes.io/host-path", VolumeGidValue "" Dec 03 11:54:41 crc kubenswrapper[5052]: I1203 11:54:41.931461 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/477501bc-7bfe-4712-b61e-e566d7e48037-combined-ca-bundle\") pod \"477501bc-7bfe-4712-b61e-e566d7e48037\" (UID: \"477501bc-7bfe-4712-b61e-e566d7e48037\") " Dec 03 11:54:41 crc kubenswrapper[5052]: I1203 11:54:41.931498 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/477501bc-7bfe-4712-b61e-e566d7e48037-scripts\") pod \"477501bc-7bfe-4712-b61e-e566d7e48037\" (UID: \"477501bc-7bfe-4712-b61e-e566d7e48037\") " Dec 03 11:54:41 crc kubenswrapper[5052]: I1203 11:54:41.931545 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/38e74be1-166e-49b3-9a4d-049f44e0547c-config\") pod \"38e74be1-166e-49b3-9a4d-049f44e0547c\" (UID: \"38e74be1-166e-49b3-9a4d-049f44e0547c\") " Dec 03 11:54:41 crc kubenswrapper[5052]: I1203 11:54:41.931974 5052 reconciler_common.go:293] "Volume detached for volume \"var-run-ovn\" (UniqueName: \"kubernetes.io/host-path/477501bc-7bfe-4712-b61e-e566d7e48037-var-run-ovn\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:41 crc kubenswrapper[5052]: I1203 11:54:41.931995 5052 reconciler_common.go:293] "Volume detached for volume \"var-run\" (UniqueName: \"kubernetes.io/host-path/477501bc-7bfe-4712-b61e-e566d7e48037-var-run\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:41 crc kubenswrapper[5052]: I1203 11:54:41.932004 5052 reconciler_common.go:293] "Volume detached for volume \"ovs-rundir\" (UniqueName: \"kubernetes.io/host-path/38e74be1-166e-49b3-9a4d-049f44e0547c-ovs-rundir\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:41 crc kubenswrapper[5052]: I1203 11:54:41.932014 5052 reconciler_common.go:293] "Volume detached for volume \"ovn-rundir\" (UniqueName: \"kubernetes.io/host-path/38e74be1-166e-49b3-9a4d-049f44e0547c-ovn-rundir\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:41 crc kubenswrapper[5052]: I1203 11:54:41.932022 5052 reconciler_common.go:293] "Volume detached for volume \"var-log-ovn\" (UniqueName: \"kubernetes.io/host-path/477501bc-7bfe-4712-b61e-e566d7e48037-var-log-ovn\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:41 crc kubenswrapper[5052]: E1203 11:54:41.932072 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/ovncontroller-scripts: configmap "ovncontroller-scripts" not found Dec 03 11:54:41 crc kubenswrapper[5052]: E1203 11:54:41.932113 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7f7e6c2a-8c03-435d-9b23-3ed37bb9811d-scripts podName:7f7e6c2a-8c03-435d-9b23-3ed37bb9811d nodeName:}" failed. No retries permitted until 2025-12-03 11:54:42.932098075 +0000 UTC m=+2073.520339839 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "scripts" (UniqueName: "kubernetes.io/configmap/7f7e6c2a-8c03-435d-9b23-3ed37bb9811d-scripts") pod "ovn-controller-ovs-2bfws" (UID: "7f7e6c2a-8c03-435d-9b23-3ed37bb9811d") : configmap "ovncontroller-scripts" not found Dec 03 11:54:41 crc kubenswrapper[5052]: I1203 11:54:41.933127 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/477501bc-7bfe-4712-b61e-e566d7e48037-scripts" (OuterVolumeSpecName: "scripts") pod "477501bc-7bfe-4712-b61e-e566d7e48037" (UID: "477501bc-7bfe-4712-b61e-e566d7e48037"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:54:41 crc kubenswrapper[5052]: I1203 11:54:41.934128 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/38e74be1-166e-49b3-9a4d-049f44e0547c-config" (OuterVolumeSpecName: "config") pod "38e74be1-166e-49b3-9a4d-049f44e0547c" (UID: "38e74be1-166e-49b3-9a4d-049f44e0547c"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:54:41 crc kubenswrapper[5052]: I1203 11:54:41.938696 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/38e74be1-166e-49b3-9a4d-049f44e0547c-kube-api-access-8dhhg" (OuterVolumeSpecName: "kube-api-access-8dhhg") pod "38e74be1-166e-49b3-9a4d-049f44e0547c" (UID: "38e74be1-166e-49b3-9a4d-049f44e0547c"). InnerVolumeSpecName "kube-api-access-8dhhg". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:54:41 crc kubenswrapper[5052]: I1203 11:54:41.939085 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/477501bc-7bfe-4712-b61e-e566d7e48037-kube-api-access-8cpnv" (OuterVolumeSpecName: "kube-api-access-8cpnv") pod "477501bc-7bfe-4712-b61e-e566d7e48037" (UID: "477501bc-7bfe-4712-b61e-e566d7e48037"). InnerVolumeSpecName "kube-api-access-8cpnv". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:54:41 crc kubenswrapper[5052]: I1203 11:54:41.969133 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/38e74be1-166e-49b3-9a4d-049f44e0547c-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "38e74be1-166e-49b3-9a4d-049f44e0547c" (UID: "38e74be1-166e-49b3-9a4d-049f44e0547c"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:54:41 crc kubenswrapper[5052]: I1203 11:54:41.969200 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/477501bc-7bfe-4712-b61e-e566d7e48037-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "477501bc-7bfe-4712-b61e-e566d7e48037" (UID: "477501bc-7bfe-4712-b61e-e566d7e48037"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:54:42 crc kubenswrapper[5052]: I1203 11:54:42.008805 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/38e74be1-166e-49b3-9a4d-049f44e0547c-metrics-certs-tls-certs" (OuterVolumeSpecName: "metrics-certs-tls-certs") pod "38e74be1-166e-49b3-9a4d-049f44e0547c" (UID: "38e74be1-166e-49b3-9a4d-049f44e0547c"). InnerVolumeSpecName "metrics-certs-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:54:42 crc kubenswrapper[5052]: I1203 11:54:42.021040 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/477501bc-7bfe-4712-b61e-e566d7e48037-ovn-controller-tls-certs" (OuterVolumeSpecName: "ovn-controller-tls-certs") pod "477501bc-7bfe-4712-b61e-e566d7e48037" (UID: "477501bc-7bfe-4712-b61e-e566d7e48037"). InnerVolumeSpecName "ovn-controller-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:54:42 crc kubenswrapper[5052]: I1203 11:54:42.036551 5052 reconciler_common.go:293] "Volume detached for volume \"ovn-controller-tls-certs\" (UniqueName: \"kubernetes.io/secret/477501bc-7bfe-4712-b61e-e566d7e48037-ovn-controller-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:42 crc kubenswrapper[5052]: I1203 11:54:42.037082 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-8cpnv\" (UniqueName: \"kubernetes.io/projected/477501bc-7bfe-4712-b61e-e566d7e48037-kube-api-access-8cpnv\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:42 crc kubenswrapper[5052]: I1203 11:54:42.037097 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-8dhhg\" (UniqueName: \"kubernetes.io/projected/38e74be1-166e-49b3-9a4d-049f44e0547c-kube-api-access-8dhhg\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:42 crc kubenswrapper[5052]: I1203 11:54:42.037108 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/477501bc-7bfe-4712-b61e-e566d7e48037-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:42 crc kubenswrapper[5052]: I1203 11:54:42.037119 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/477501bc-7bfe-4712-b61e-e566d7e48037-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:42 crc kubenswrapper[5052]: I1203 11:54:42.037132 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/38e74be1-166e-49b3-9a4d-049f44e0547c-config\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:42 crc kubenswrapper[5052]: I1203 11:54:42.037144 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/38e74be1-166e-49b3-9a4d-049f44e0547c-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:42 crc kubenswrapper[5052]: I1203 11:54:42.037156 5052 reconciler_common.go:293] "Volume detached for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/38e74be1-166e-49b3-9a4d-049f44e0547c-metrics-certs-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:42 crc kubenswrapper[5052]: I1203 11:54:42.257916 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_ovn-controller-metrics-mxqpj_38e74be1-166e-49b3-9a4d-049f44e0547c/openstack-network-exporter/0.log" Dec 03 11:54:42 crc kubenswrapper[5052]: I1203 11:54:42.257975 5052 generic.go:334] "Generic (PLEG): container finished" podID="38e74be1-166e-49b3-9a4d-049f44e0547c" containerID="d27c93cd537043ea03f8b08722fb0c631f236714bb3ab51fee417f235f65e84b" exitCode=2 Dec 03 11:54:42 crc kubenswrapper[5052]: I1203 11:54:42.258044 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovn-controller-metrics-mxqpj" event={"ID":"38e74be1-166e-49b3-9a4d-049f44e0547c","Type":"ContainerDied","Data":"d27c93cd537043ea03f8b08722fb0c631f236714bb3ab51fee417f235f65e84b"} Dec 03 11:54:42 crc kubenswrapper[5052]: I1203 11:54:42.258080 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovn-controller-metrics-mxqpj" Dec 03 11:54:42 crc kubenswrapper[5052]: I1203 11:54:42.258101 5052 scope.go:117] "RemoveContainer" containerID="d27c93cd537043ea03f8b08722fb0c631f236714bb3ab51fee417f235f65e84b" Dec 03 11:54:42 crc kubenswrapper[5052]: I1203 11:54:42.258087 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovn-controller-metrics-mxqpj" event={"ID":"38e74be1-166e-49b3-9a4d-049f44e0547c","Type":"ContainerDied","Data":"fb1248c42fdbb10898fe6eda4d8ab3743aa4fd6cf625e7cc6a5e821f76e58c40"} Dec 03 11:54:42 crc kubenswrapper[5052]: I1203 11:54:42.260232 5052 generic.go:334] "Generic (PLEG): container finished" podID="477501bc-7bfe-4712-b61e-e566d7e48037" containerID="511f73e19426246267775dc4625ae7c7af245dceb1e0d4269c332092b8770f2e" exitCode=0 Dec 03 11:54:42 crc kubenswrapper[5052]: I1203 11:54:42.260580 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovn-controller-9m9bm" Dec 03 11:54:42 crc kubenswrapper[5052]: I1203 11:54:42.261412 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovn-controller-9m9bm" event={"ID":"477501bc-7bfe-4712-b61e-e566d7e48037","Type":"ContainerDied","Data":"511f73e19426246267775dc4625ae7c7af245dceb1e0d4269c332092b8770f2e"} Dec 03 11:54:42 crc kubenswrapper[5052]: I1203 11:54:42.261465 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovn-controller-9m9bm" event={"ID":"477501bc-7bfe-4712-b61e-e566d7e48037","Type":"ContainerDied","Data":"300518f1414f335214df24fec878cd06f2faa15b94ba1919c6de55720ae97c5c"} Dec 03 11:54:42 crc kubenswrapper[5052]: I1203 11:54:42.295324 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ovn-controller-metrics-mxqpj"] Dec 03 11:54:42 crc kubenswrapper[5052]: I1203 11:54:42.299241 5052 scope.go:117] "RemoveContainer" containerID="d27c93cd537043ea03f8b08722fb0c631f236714bb3ab51fee417f235f65e84b" Dec 03 11:54:42 crc kubenswrapper[5052]: E1203 11:54:42.299931 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d27c93cd537043ea03f8b08722fb0c631f236714bb3ab51fee417f235f65e84b\": container with ID starting with d27c93cd537043ea03f8b08722fb0c631f236714bb3ab51fee417f235f65e84b not found: ID does not exist" containerID="d27c93cd537043ea03f8b08722fb0c631f236714bb3ab51fee417f235f65e84b" Dec 03 11:54:42 crc kubenswrapper[5052]: I1203 11:54:42.299969 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d27c93cd537043ea03f8b08722fb0c631f236714bb3ab51fee417f235f65e84b"} err="failed to get container status \"d27c93cd537043ea03f8b08722fb0c631f236714bb3ab51fee417f235f65e84b\": rpc error: code = NotFound desc = could not find container \"d27c93cd537043ea03f8b08722fb0c631f236714bb3ab51fee417f235f65e84b\": container with ID starting with d27c93cd537043ea03f8b08722fb0c631f236714bb3ab51fee417f235f65e84b not found: ID does not exist" Dec 03 11:54:42 crc kubenswrapper[5052]: I1203 11:54:42.299996 5052 scope.go:117] "RemoveContainer" containerID="511f73e19426246267775dc4625ae7c7af245dceb1e0d4269c332092b8770f2e" Dec 03 11:54:42 crc kubenswrapper[5052]: I1203 11:54:42.305916 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ovn-controller-metrics-mxqpj"] Dec 03 11:54:42 crc kubenswrapper[5052]: I1203 11:54:42.316206 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ovn-controller-9m9bm"] Dec 03 11:54:42 crc kubenswrapper[5052]: I1203 11:54:42.324558 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ovn-controller-9m9bm"] Dec 03 11:54:42 crc kubenswrapper[5052]: I1203 11:54:42.329026 5052 scope.go:117] "RemoveContainer" containerID="511f73e19426246267775dc4625ae7c7af245dceb1e0d4269c332092b8770f2e" Dec 03 11:54:42 crc kubenswrapper[5052]: E1203 11:54:42.329886 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"511f73e19426246267775dc4625ae7c7af245dceb1e0d4269c332092b8770f2e\": container with ID starting with 511f73e19426246267775dc4625ae7c7af245dceb1e0d4269c332092b8770f2e not found: ID does not exist" containerID="511f73e19426246267775dc4625ae7c7af245dceb1e0d4269c332092b8770f2e" Dec 03 11:54:42 crc kubenswrapper[5052]: I1203 11:54:42.329927 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"511f73e19426246267775dc4625ae7c7af245dceb1e0d4269c332092b8770f2e"} err="failed to get container status \"511f73e19426246267775dc4625ae7c7af245dceb1e0d4269c332092b8770f2e\": rpc error: code = NotFound desc = could not find container \"511f73e19426246267775dc4625ae7c7af245dceb1e0d4269c332092b8770f2e\": container with ID starting with 511f73e19426246267775dc4625ae7c7af245dceb1e0d4269c332092b8770f2e not found: ID does not exist" Dec 03 11:54:42 crc kubenswrapper[5052]: I1203 11:54:42.627762 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ovn-controller-ovs-2bfws" podUID="7f7e6c2a-8c03-435d-9b23-3ed37bb9811d" containerName="ovs-vswitchd" containerID="cri-o://555a48fb112549936fa017dafd0bd2b3a6535396cbeaebfeacc9a34ebb837f35" gracePeriod=30 Dec 03 11:54:42 crc kubenswrapper[5052]: E1203 11:54:42.890162 5052 handlers.go:78] "Exec lifecycle hook for Container in Pod failed" err=< Dec 03 11:54:42 crc kubenswrapper[5052]: command '/usr/local/bin/container-scripts/stop-ovsdb-server.sh' exited with 137: ++ dirname /usr/local/bin/container-scripts/stop-ovsdb-server.sh Dec 03 11:54:42 crc kubenswrapper[5052]: + source /usr/local/bin/container-scripts/functions Dec 03 11:54:42 crc kubenswrapper[5052]: ++ OVNBridge=br-int Dec 03 11:54:42 crc kubenswrapper[5052]: ++ OVNRemote=tcp:localhost:6642 Dec 03 11:54:42 crc kubenswrapper[5052]: ++ OVNEncapType=geneve Dec 03 11:54:42 crc kubenswrapper[5052]: ++ OVNAvailabilityZones= Dec 03 11:54:42 crc kubenswrapper[5052]: ++ EnableChassisAsGateway=true Dec 03 11:54:42 crc kubenswrapper[5052]: ++ PhysicalNetworks= Dec 03 11:54:42 crc kubenswrapper[5052]: ++ OVNHostName= Dec 03 11:54:42 crc kubenswrapper[5052]: ++ DB_FILE=/etc/openvswitch/conf.db Dec 03 11:54:42 crc kubenswrapper[5052]: ++ ovs_dir=/var/lib/openvswitch Dec 03 11:54:42 crc kubenswrapper[5052]: ++ FLOWS_RESTORE_SCRIPT=/var/lib/openvswitch/flows-script Dec 03 11:54:42 crc kubenswrapper[5052]: ++ FLOWS_RESTORE_DIR=/var/lib/openvswitch/saved-flows Dec 03 11:54:42 crc kubenswrapper[5052]: ++ SAFE_TO_STOP_OVSDB_SERVER_SEMAPHORE=/var/lib/openvswitch/is_safe_to_stop_ovsdb_server Dec 03 11:54:42 crc kubenswrapper[5052]: + '[' '!' -f /var/lib/openvswitch/is_safe_to_stop_ovsdb_server ']' Dec 03 11:54:42 crc kubenswrapper[5052]: + sleep 0.5 Dec 03 11:54:42 crc kubenswrapper[5052]: + '[' '!' -f /var/lib/openvswitch/is_safe_to_stop_ovsdb_server ']' Dec 03 11:54:42 crc kubenswrapper[5052]: + cleanup_ovsdb_server_semaphore Dec 03 11:54:42 crc kubenswrapper[5052]: + rm -f /var/lib/openvswitch/is_safe_to_stop_ovsdb_server Dec 03 11:54:42 crc kubenswrapper[5052]: + /usr/share/openvswitch/scripts/ovs-ctl stop --no-ovs-vswitchd Dec 03 11:54:42 crc kubenswrapper[5052]: > execCommand=["/usr/local/bin/container-scripts/stop-ovsdb-server.sh"] containerName="ovsdb-server" pod="openstack-kuttl-tests/ovn-controller-ovs-2bfws" message=< Dec 03 11:54:42 crc kubenswrapper[5052]: Exiting ovsdb-server (5) [ OK ] Dec 03 11:54:42 crc kubenswrapper[5052]: ++ dirname /usr/local/bin/container-scripts/stop-ovsdb-server.sh Dec 03 11:54:42 crc kubenswrapper[5052]: + source /usr/local/bin/container-scripts/functions Dec 03 11:54:42 crc kubenswrapper[5052]: ++ OVNBridge=br-int Dec 03 11:54:42 crc kubenswrapper[5052]: ++ OVNRemote=tcp:localhost:6642 Dec 03 11:54:42 crc kubenswrapper[5052]: ++ OVNEncapType=geneve Dec 03 11:54:42 crc kubenswrapper[5052]: ++ OVNAvailabilityZones= Dec 03 11:54:42 crc kubenswrapper[5052]: ++ EnableChassisAsGateway=true Dec 03 11:54:42 crc kubenswrapper[5052]: ++ PhysicalNetworks= Dec 03 11:54:42 crc kubenswrapper[5052]: ++ OVNHostName= Dec 03 11:54:42 crc kubenswrapper[5052]: ++ DB_FILE=/etc/openvswitch/conf.db Dec 03 11:54:42 crc kubenswrapper[5052]: ++ ovs_dir=/var/lib/openvswitch Dec 03 11:54:42 crc kubenswrapper[5052]: ++ FLOWS_RESTORE_SCRIPT=/var/lib/openvswitch/flows-script Dec 03 11:54:42 crc kubenswrapper[5052]: ++ FLOWS_RESTORE_DIR=/var/lib/openvswitch/saved-flows Dec 03 11:54:42 crc kubenswrapper[5052]: ++ SAFE_TO_STOP_OVSDB_SERVER_SEMAPHORE=/var/lib/openvswitch/is_safe_to_stop_ovsdb_server Dec 03 11:54:42 crc kubenswrapper[5052]: + '[' '!' -f /var/lib/openvswitch/is_safe_to_stop_ovsdb_server ']' Dec 03 11:54:42 crc kubenswrapper[5052]: + sleep 0.5 Dec 03 11:54:42 crc kubenswrapper[5052]: + '[' '!' -f /var/lib/openvswitch/is_safe_to_stop_ovsdb_server ']' Dec 03 11:54:42 crc kubenswrapper[5052]: + cleanup_ovsdb_server_semaphore Dec 03 11:54:42 crc kubenswrapper[5052]: + rm -f /var/lib/openvswitch/is_safe_to_stop_ovsdb_server Dec 03 11:54:42 crc kubenswrapper[5052]: + /usr/share/openvswitch/scripts/ovs-ctl stop --no-ovs-vswitchd Dec 03 11:54:42 crc kubenswrapper[5052]: > Dec 03 11:54:42 crc kubenswrapper[5052]: E1203 11:54:42.890212 5052 kuberuntime_container.go:691] "PreStop hook failed" err=< Dec 03 11:54:42 crc kubenswrapper[5052]: command '/usr/local/bin/container-scripts/stop-ovsdb-server.sh' exited with 137: ++ dirname /usr/local/bin/container-scripts/stop-ovsdb-server.sh Dec 03 11:54:42 crc kubenswrapper[5052]: + source /usr/local/bin/container-scripts/functions Dec 03 11:54:42 crc kubenswrapper[5052]: ++ OVNBridge=br-int Dec 03 11:54:42 crc kubenswrapper[5052]: ++ OVNRemote=tcp:localhost:6642 Dec 03 11:54:42 crc kubenswrapper[5052]: ++ OVNEncapType=geneve Dec 03 11:54:42 crc kubenswrapper[5052]: ++ OVNAvailabilityZones= Dec 03 11:54:42 crc kubenswrapper[5052]: ++ EnableChassisAsGateway=true Dec 03 11:54:42 crc kubenswrapper[5052]: ++ PhysicalNetworks= Dec 03 11:54:42 crc kubenswrapper[5052]: ++ OVNHostName= Dec 03 11:54:42 crc kubenswrapper[5052]: ++ DB_FILE=/etc/openvswitch/conf.db Dec 03 11:54:42 crc kubenswrapper[5052]: ++ ovs_dir=/var/lib/openvswitch Dec 03 11:54:42 crc kubenswrapper[5052]: ++ FLOWS_RESTORE_SCRIPT=/var/lib/openvswitch/flows-script Dec 03 11:54:42 crc kubenswrapper[5052]: ++ FLOWS_RESTORE_DIR=/var/lib/openvswitch/saved-flows Dec 03 11:54:42 crc kubenswrapper[5052]: ++ SAFE_TO_STOP_OVSDB_SERVER_SEMAPHORE=/var/lib/openvswitch/is_safe_to_stop_ovsdb_server Dec 03 11:54:42 crc kubenswrapper[5052]: + '[' '!' -f /var/lib/openvswitch/is_safe_to_stop_ovsdb_server ']' Dec 03 11:54:42 crc kubenswrapper[5052]: + sleep 0.5 Dec 03 11:54:42 crc kubenswrapper[5052]: + '[' '!' -f /var/lib/openvswitch/is_safe_to_stop_ovsdb_server ']' Dec 03 11:54:42 crc kubenswrapper[5052]: + cleanup_ovsdb_server_semaphore Dec 03 11:54:42 crc kubenswrapper[5052]: + rm -f /var/lib/openvswitch/is_safe_to_stop_ovsdb_server Dec 03 11:54:42 crc kubenswrapper[5052]: + /usr/share/openvswitch/scripts/ovs-ctl stop --no-ovs-vswitchd Dec 03 11:54:42 crc kubenswrapper[5052]: > pod="openstack-kuttl-tests/ovn-controller-ovs-2bfws" podUID="7f7e6c2a-8c03-435d-9b23-3ed37bb9811d" containerName="ovsdb-server" containerID="cri-o://b114369f17334b3461585b8e213c8285e8ea520f731eeea65e56b8a04d08702b" Dec 03 11:54:42 crc kubenswrapper[5052]: I1203 11:54:42.890260 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ovn-controller-ovs-2bfws" podUID="7f7e6c2a-8c03-435d-9b23-3ed37bb9811d" containerName="ovsdb-server" containerID="cri-o://b114369f17334b3461585b8e213c8285e8ea520f731eeea65e56b8a04d08702b" gracePeriod=30 Dec 03 11:54:42 crc kubenswrapper[5052]: E1203 11:54:42.954813 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/ovncontroller-scripts: configmap "ovncontroller-scripts" not found Dec 03 11:54:42 crc kubenswrapper[5052]: E1203 11:54:42.954901 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7f7e6c2a-8c03-435d-9b23-3ed37bb9811d-scripts podName:7f7e6c2a-8c03-435d-9b23-3ed37bb9811d nodeName:}" failed. No retries permitted until 2025-12-03 11:54:44.954869879 +0000 UTC m=+2075.543111643 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "scripts" (UniqueName: "kubernetes.io/configmap/7f7e6c2a-8c03-435d-9b23-3ed37bb9811d-scripts") pod "ovn-controller-ovs-2bfws" (UID: "7f7e6c2a-8c03-435d-9b23-3ed37bb9811d") : configmap "ovncontroller-scripts" not found Dec 03 11:54:43 crc kubenswrapper[5052]: I1203 11:54:43.273714 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_ovn-controller-ovs-2bfws_7f7e6c2a-8c03-435d-9b23-3ed37bb9811d/ovs-vswitchd/0.log" Dec 03 11:54:43 crc kubenswrapper[5052]: I1203 11:54:43.274826 5052 generic.go:334] "Generic (PLEG): container finished" podID="7f7e6c2a-8c03-435d-9b23-3ed37bb9811d" containerID="555a48fb112549936fa017dafd0bd2b3a6535396cbeaebfeacc9a34ebb837f35" exitCode=143 Dec 03 11:54:43 crc kubenswrapper[5052]: I1203 11:54:43.274845 5052 generic.go:334] "Generic (PLEG): container finished" podID="7f7e6c2a-8c03-435d-9b23-3ed37bb9811d" containerID="b114369f17334b3461585b8e213c8285e8ea520f731eeea65e56b8a04d08702b" exitCode=0 Dec 03 11:54:43 crc kubenswrapper[5052]: I1203 11:54:43.274893 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovn-controller-ovs-2bfws" event={"ID":"7f7e6c2a-8c03-435d-9b23-3ed37bb9811d","Type":"ContainerDied","Data":"555a48fb112549936fa017dafd0bd2b3a6535396cbeaebfeacc9a34ebb837f35"} Dec 03 11:54:43 crc kubenswrapper[5052]: I1203 11:54:43.274924 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovn-controller-ovs-2bfws" event={"ID":"7f7e6c2a-8c03-435d-9b23-3ed37bb9811d","Type":"ContainerDied","Data":"b114369f17334b3461585b8e213c8285e8ea520f731eeea65e56b8a04d08702b"} Dec 03 11:54:43 crc kubenswrapper[5052]: I1203 11:54:43.274935 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovn-controller-ovs-2bfws" event={"ID":"7f7e6c2a-8c03-435d-9b23-3ed37bb9811d","Type":"ContainerDied","Data":"0ba3cd3025da56ec1901b743f3c0e26307ba5791d78061bb75be7dee0bcd180e"} Dec 03 11:54:43 crc kubenswrapper[5052]: I1203 11:54:43.274944 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0ba3cd3025da56ec1901b743f3c0e26307ba5791d78061bb75be7dee0bcd180e" Dec 03 11:54:43 crc kubenswrapper[5052]: I1203 11:54:43.341310 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_ovn-controller-ovs-2bfws_7f7e6c2a-8c03-435d-9b23-3ed37bb9811d/ovs-vswitchd/0.log" Dec 03 11:54:43 crc kubenswrapper[5052]: I1203 11:54:43.349275 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovn-controller-ovs-2bfws" Dec 03 11:54:43 crc kubenswrapper[5052]: I1203 11:54:43.463438 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"var-lib\" (UniqueName: \"kubernetes.io/host-path/7f7e6c2a-8c03-435d-9b23-3ed37bb9811d-var-lib\") pod \"7f7e6c2a-8c03-435d-9b23-3ed37bb9811d\" (UID: \"7f7e6c2a-8c03-435d-9b23-3ed37bb9811d\") " Dec 03 11:54:43 crc kubenswrapper[5052]: I1203 11:54:43.463529 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"var-log\" (UniqueName: \"kubernetes.io/host-path/7f7e6c2a-8c03-435d-9b23-3ed37bb9811d-var-log\") pod \"7f7e6c2a-8c03-435d-9b23-3ed37bb9811d\" (UID: \"7f7e6c2a-8c03-435d-9b23-3ed37bb9811d\") " Dec 03 11:54:43 crc kubenswrapper[5052]: I1203 11:54:43.463630 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/7f7e6c2a-8c03-435d-9b23-3ed37bb9811d-scripts\") pod \"7f7e6c2a-8c03-435d-9b23-3ed37bb9811d\" (UID: \"7f7e6c2a-8c03-435d-9b23-3ed37bb9811d\") " Dec 03 11:54:43 crc kubenswrapper[5052]: I1203 11:54:43.463776 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-68v7z\" (UniqueName: \"kubernetes.io/projected/7f7e6c2a-8c03-435d-9b23-3ed37bb9811d-kube-api-access-68v7z\") pod \"7f7e6c2a-8c03-435d-9b23-3ed37bb9811d\" (UID: \"7f7e6c2a-8c03-435d-9b23-3ed37bb9811d\") " Dec 03 11:54:43 crc kubenswrapper[5052]: I1203 11:54:43.463820 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"var-run\" (UniqueName: \"kubernetes.io/host-path/7f7e6c2a-8c03-435d-9b23-3ed37bb9811d-var-run\") pod \"7f7e6c2a-8c03-435d-9b23-3ed37bb9811d\" (UID: \"7f7e6c2a-8c03-435d-9b23-3ed37bb9811d\") " Dec 03 11:54:43 crc kubenswrapper[5052]: I1203 11:54:43.463907 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"etc-ovs\" (UniqueName: \"kubernetes.io/host-path/7f7e6c2a-8c03-435d-9b23-3ed37bb9811d-etc-ovs\") pod \"7f7e6c2a-8c03-435d-9b23-3ed37bb9811d\" (UID: \"7f7e6c2a-8c03-435d-9b23-3ed37bb9811d\") " Dec 03 11:54:43 crc kubenswrapper[5052]: I1203 11:54:43.464440 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/7f7e6c2a-8c03-435d-9b23-3ed37bb9811d-etc-ovs" (OuterVolumeSpecName: "etc-ovs") pod "7f7e6c2a-8c03-435d-9b23-3ed37bb9811d" (UID: "7f7e6c2a-8c03-435d-9b23-3ed37bb9811d"). InnerVolumeSpecName "etc-ovs". PluginName "kubernetes.io/host-path", VolumeGidValue "" Dec 03 11:54:43 crc kubenswrapper[5052]: I1203 11:54:43.464484 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/7f7e6c2a-8c03-435d-9b23-3ed37bb9811d-var-lib" (OuterVolumeSpecName: "var-lib") pod "7f7e6c2a-8c03-435d-9b23-3ed37bb9811d" (UID: "7f7e6c2a-8c03-435d-9b23-3ed37bb9811d"). InnerVolumeSpecName "var-lib". PluginName "kubernetes.io/host-path", VolumeGidValue "" Dec 03 11:54:43 crc kubenswrapper[5052]: I1203 11:54:43.464507 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/7f7e6c2a-8c03-435d-9b23-3ed37bb9811d-var-log" (OuterVolumeSpecName: "var-log") pod "7f7e6c2a-8c03-435d-9b23-3ed37bb9811d" (UID: "7f7e6c2a-8c03-435d-9b23-3ed37bb9811d"). InnerVolumeSpecName "var-log". PluginName "kubernetes.io/host-path", VolumeGidValue "" Dec 03 11:54:43 crc kubenswrapper[5052]: I1203 11:54:43.465759 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7f7e6c2a-8c03-435d-9b23-3ed37bb9811d-scripts" (OuterVolumeSpecName: "scripts") pod "7f7e6c2a-8c03-435d-9b23-3ed37bb9811d" (UID: "7f7e6c2a-8c03-435d-9b23-3ed37bb9811d"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:54:43 crc kubenswrapper[5052]: I1203 11:54:43.466454 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/7f7e6c2a-8c03-435d-9b23-3ed37bb9811d-var-run" (OuterVolumeSpecName: "var-run") pod "7f7e6c2a-8c03-435d-9b23-3ed37bb9811d" (UID: "7f7e6c2a-8c03-435d-9b23-3ed37bb9811d"). InnerVolumeSpecName "var-run". PluginName "kubernetes.io/host-path", VolumeGidValue "" Dec 03 11:54:43 crc kubenswrapper[5052]: I1203 11:54:43.487542 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7f7e6c2a-8c03-435d-9b23-3ed37bb9811d-kube-api-access-68v7z" (OuterVolumeSpecName: "kube-api-access-68v7z") pod "7f7e6c2a-8c03-435d-9b23-3ed37bb9811d" (UID: "7f7e6c2a-8c03-435d-9b23-3ed37bb9811d"). InnerVolumeSpecName "kube-api-access-68v7z". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:54:43 crc kubenswrapper[5052]: I1203 11:54:43.566132 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/7f7e6c2a-8c03-435d-9b23-3ed37bb9811d-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:43 crc kubenswrapper[5052]: I1203 11:54:43.566185 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-68v7z\" (UniqueName: \"kubernetes.io/projected/7f7e6c2a-8c03-435d-9b23-3ed37bb9811d-kube-api-access-68v7z\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:43 crc kubenswrapper[5052]: I1203 11:54:43.566210 5052 reconciler_common.go:293] "Volume detached for volume \"var-run\" (UniqueName: \"kubernetes.io/host-path/7f7e6c2a-8c03-435d-9b23-3ed37bb9811d-var-run\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:43 crc kubenswrapper[5052]: I1203 11:54:43.566220 5052 reconciler_common.go:293] "Volume detached for volume \"etc-ovs\" (UniqueName: \"kubernetes.io/host-path/7f7e6c2a-8c03-435d-9b23-3ed37bb9811d-etc-ovs\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:43 crc kubenswrapper[5052]: I1203 11:54:43.566229 5052 reconciler_common.go:293] "Volume detached for volume \"var-lib\" (UniqueName: \"kubernetes.io/host-path/7f7e6c2a-8c03-435d-9b23-3ed37bb9811d-var-lib\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:43 crc kubenswrapper[5052]: I1203 11:54:43.566239 5052 reconciler_common.go:293] "Volume detached for volume \"var-log\" (UniqueName: \"kubernetes.io/host-path/7f7e6c2a-8c03-435d-9b23-3ed37bb9811d-var-log\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:43 crc kubenswrapper[5052]: I1203 11:54:43.683614 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="38e74be1-166e-49b3-9a4d-049f44e0547c" path="/var/lib/kubelet/pods/38e74be1-166e-49b3-9a4d-049f44e0547c/volumes" Dec 03 11:54:43 crc kubenswrapper[5052]: I1203 11:54:43.684311 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="477501bc-7bfe-4712-b61e-e566d7e48037" path="/var/lib/kubelet/pods/477501bc-7bfe-4712-b61e-e566d7e48037/volumes" Dec 03 11:54:44 crc kubenswrapper[5052]: I1203 11:54:44.283450 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovn-controller-ovs-2bfws" Dec 03 11:54:44 crc kubenswrapper[5052]: I1203 11:54:44.306354 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ovn-controller-ovs-2bfws"] Dec 03 11:54:44 crc kubenswrapper[5052]: I1203 11:54:44.315608 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ovn-controller-ovs-2bfws"] Dec 03 11:54:45 crc kubenswrapper[5052]: I1203 11:54:45.336345 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/rabbitmq-server-0"] Dec 03 11:54:45 crc kubenswrapper[5052]: I1203 11:54:45.468252 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/openstackclient"] Dec 03 11:54:45 crc kubenswrapper[5052]: I1203 11:54:45.468490 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/openstackclient" podUID="4c81b4dc-4f69-4447-8e5e-043aa0c7dd59" containerName="openstackclient" containerID="cri-o://d40e529ae6719c4ad0db1e286623449a547ab4530cef62b4eede6892bcc97992" gracePeriod=2 Dec 03 11:54:45 crc kubenswrapper[5052]: I1203 11:54:45.485659 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/openstackclient"] Dec 03 11:54:45 crc kubenswrapper[5052]: E1203 11:54:45.506347 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/rabbitmq-config-data: configmap "rabbitmq-config-data" not found Dec 03 11:54:45 crc kubenswrapper[5052]: E1203 11:54:45.506652 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/10e3094e-f625-4d69-b0e9-358220c0d195-config-data podName:10e3094e-f625-4d69-b0e9-358220c0d195 nodeName:}" failed. No retries permitted until 2025-12-03 11:54:46.006632981 +0000 UTC m=+2076.594874745 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "config-data" (UniqueName: "kubernetes.io/configmap/10e3094e-f625-4d69-b0e9-358220c0d195-config-data") pod "rabbitmq-server-0" (UID: "10e3094e-f625-4d69-b0e9-358220c0d195") : configmap "rabbitmq-config-data" not found Dec 03 11:54:45 crc kubenswrapper[5052]: I1203 11:54:45.830208 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7f7e6c2a-8c03-435d-9b23-3ed37bb9811d" path="/var/lib/kubelet/pods/7f7e6c2a-8c03-435d-9b23-3ed37bb9811d/volumes" Dec 03 11:54:45 crc kubenswrapper[5052]: I1203 11:54:45.831484 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ovn-northd-0"] Dec 03 11:54:45 crc kubenswrapper[5052]: I1203 11:54:45.843835 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ovn-northd-0" podUID="9d8cd78f-2670-4933-81dd-01cdc0044454" containerName="ovn-northd" containerID="cri-o://2cf5c7f704cdea9711753e85146be0ca1fd13eb56b9658c079f3f91d897ce7a3" gracePeriod=30 Dec 03 11:54:45 crc kubenswrapper[5052]: I1203 11:54:45.844261 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ovn-northd-0" podUID="9d8cd78f-2670-4933-81dd-01cdc0044454" containerName="openstack-network-exporter" containerID="cri-o://a00e41ad3b048582d984194b4e78df8f197fd9e5a5af35f2e726a5b8f13fb9fb" gracePeriod=30 Dec 03 11:54:45 crc kubenswrapper[5052]: I1203 11:54:45.870488 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/rabbitmq-cell1-server-0"] Dec 03 11:54:45 crc kubenswrapper[5052]: I1203 11:54:45.915996 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/barbican4385-account-delete-krpzg"] Dec 03 11:54:45 crc kubenswrapper[5052]: E1203 11:54:45.916525 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="7f7e6c2a-8c03-435d-9b23-3ed37bb9811d" containerName="ovsdb-server-init" Dec 03 11:54:45 crc kubenswrapper[5052]: I1203 11:54:45.916547 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f7e6c2a-8c03-435d-9b23-3ed37bb9811d" containerName="ovsdb-server-init" Dec 03 11:54:45 crc kubenswrapper[5052]: E1203 11:54:45.916564 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="38e74be1-166e-49b3-9a4d-049f44e0547c" containerName="openstack-network-exporter" Dec 03 11:54:45 crc kubenswrapper[5052]: I1203 11:54:45.916573 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="38e74be1-166e-49b3-9a4d-049f44e0547c" containerName="openstack-network-exporter" Dec 03 11:54:45 crc kubenswrapper[5052]: E1203 11:54:45.916593 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="7f7e6c2a-8c03-435d-9b23-3ed37bb9811d" containerName="ovsdb-server" Dec 03 11:54:45 crc kubenswrapper[5052]: I1203 11:54:45.916602 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f7e6c2a-8c03-435d-9b23-3ed37bb9811d" containerName="ovsdb-server" Dec 03 11:54:45 crc kubenswrapper[5052]: E1203 11:54:45.916611 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="7f7e6c2a-8c03-435d-9b23-3ed37bb9811d" containerName="ovs-vswitchd" Dec 03 11:54:45 crc kubenswrapper[5052]: I1203 11:54:45.916619 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f7e6c2a-8c03-435d-9b23-3ed37bb9811d" containerName="ovs-vswitchd" Dec 03 11:54:45 crc kubenswrapper[5052]: E1203 11:54:45.916645 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="477501bc-7bfe-4712-b61e-e566d7e48037" containerName="ovn-controller" Dec 03 11:54:45 crc kubenswrapper[5052]: I1203 11:54:45.916654 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="477501bc-7bfe-4712-b61e-e566d7e48037" containerName="ovn-controller" Dec 03 11:54:45 crc kubenswrapper[5052]: E1203 11:54:45.916682 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="4c81b4dc-4f69-4447-8e5e-043aa0c7dd59" containerName="openstackclient" Dec 03 11:54:45 crc kubenswrapper[5052]: I1203 11:54:45.916689 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="4c81b4dc-4f69-4447-8e5e-043aa0c7dd59" containerName="openstackclient" Dec 03 11:54:45 crc kubenswrapper[5052]: I1203 11:54:45.916901 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="38e74be1-166e-49b3-9a4d-049f44e0547c" containerName="openstack-network-exporter" Dec 03 11:54:45 crc kubenswrapper[5052]: I1203 11:54:45.916923 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="7f7e6c2a-8c03-435d-9b23-3ed37bb9811d" containerName="ovsdb-server" Dec 03 11:54:45 crc kubenswrapper[5052]: I1203 11:54:45.916939 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="4c81b4dc-4f69-4447-8e5e-043aa0c7dd59" containerName="openstackclient" Dec 03 11:54:45 crc kubenswrapper[5052]: I1203 11:54:45.916950 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="477501bc-7bfe-4712-b61e-e566d7e48037" containerName="ovn-controller" Dec 03 11:54:45 crc kubenswrapper[5052]: I1203 11:54:45.916964 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="7f7e6c2a-8c03-435d-9b23-3ed37bb9811d" containerName="ovs-vswitchd" Dec 03 11:54:45 crc kubenswrapper[5052]: I1203 11:54:45.917708 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican4385-account-delete-krpzg" Dec 03 11:54:45 crc kubenswrapper[5052]: I1203 11:54:45.947567 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican4385-account-delete-krpzg"] Dec 03 11:54:45 crc kubenswrapper[5052]: I1203 11:54:45.977757 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-db-sync-847xp"] Dec 03 11:54:46 crc kubenswrapper[5052]: I1203 11:54:46.007338 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-nb-0"] Dec 03 11:54:46 crc kubenswrapper[5052]: I1203 11:54:46.008065 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ovsdbserver-nb-0" podUID="b4961b81-97b8-4174-82bf-2d1d02cf8335" containerName="openstack-network-exporter" containerID="cri-o://e54d2995d89269947a87a50c5c5e626a778c66e141e2d728f3fafc46c76e5a9b" gracePeriod=300 Dec 03 11:54:46 crc kubenswrapper[5052]: I1203 11:54:46.027580 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/barbican-db-sync-847xp"] Dec 03 11:54:46 crc kubenswrapper[5052]: I1203 11:54:46.051577 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m8pj8\" (UniqueName: \"kubernetes.io/projected/383e7933-2bc4-4ead-bc34-0832091bd952-kube-api-access-m8pj8\") pod \"barbican4385-account-delete-krpzg\" (UID: \"383e7933-2bc4-4ead-bc34-0832091bd952\") " pod="openstack-kuttl-tests/barbican4385-account-delete-krpzg" Dec 03 11:54:46 crc kubenswrapper[5052]: I1203 11:54:46.051652 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/383e7933-2bc4-4ead-bc34-0832091bd952-operator-scripts\") pod \"barbican4385-account-delete-krpzg\" (UID: \"383e7933-2bc4-4ead-bc34-0832091bd952\") " pod="openstack-kuttl-tests/barbican4385-account-delete-krpzg" Dec 03 11:54:46 crc kubenswrapper[5052]: E1203 11:54:46.051877 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/rabbitmq-cell1-config-data: configmap "rabbitmq-cell1-config-data" not found Dec 03 11:54:46 crc kubenswrapper[5052]: E1203 11:54:46.051938 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a5ba4775-815a-46ef-bd2d-05dfa7513288-config-data podName:a5ba4775-815a-46ef-bd2d-05dfa7513288 nodeName:}" failed. No retries permitted until 2025-12-03 11:54:46.551917483 +0000 UTC m=+2077.140159247 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "config-data" (UniqueName: "kubernetes.io/configmap/a5ba4775-815a-46ef-bd2d-05dfa7513288-config-data") pod "rabbitmq-cell1-server-0" (UID: "a5ba4775-815a-46ef-bd2d-05dfa7513288") : configmap "rabbitmq-cell1-config-data" not found Dec 03 11:54:46 crc kubenswrapper[5052]: E1203 11:54:46.051974 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/rabbitmq-config-data: configmap "rabbitmq-config-data" not found Dec 03 11:54:46 crc kubenswrapper[5052]: E1203 11:54:46.052000 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/10e3094e-f625-4d69-b0e9-358220c0d195-config-data podName:10e3094e-f625-4d69-b0e9-358220c0d195 nodeName:}" failed. No retries permitted until 2025-12-03 11:54:47.051991915 +0000 UTC m=+2077.640233679 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "config-data" (UniqueName: "kubernetes.io/configmap/10e3094e-f625-4d69-b0e9-358220c0d195-config-data") pod "rabbitmq-server-0" (UID: "10e3094e-f625-4d69-b0e9-358220c0d195") : configmap "rabbitmq-config-data" not found Dec 03 11:54:46 crc kubenswrapper[5052]: I1203 11:54:46.053258 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/glance634c-account-delete-nzwqr"] Dec 03 11:54:46 crc kubenswrapper[5052]: I1203 11:54:46.096576 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance634c-account-delete-nzwqr"] Dec 03 11:54:46 crc kubenswrapper[5052]: I1203 11:54:46.096604 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-db-sync-zzt28"] Dec 03 11:54:46 crc kubenswrapper[5052]: I1203 11:54:46.096617 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/glance-db-sync-zzt28"] Dec 03 11:54:46 crc kubenswrapper[5052]: I1203 11:54:46.097342 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance634c-account-delete-nzwqr" Dec 03 11:54:46 crc kubenswrapper[5052]: I1203 11:54:46.106417 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/cinder6fd6-account-delete-5z97w"] Dec 03 11:54:46 crc kubenswrapper[5052]: I1203 11:54:46.109626 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder6fd6-account-delete-5z97w" Dec 03 11:54:46 crc kubenswrapper[5052]: I1203 11:54:46.135582 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder6fd6-account-delete-5z97w"] Dec 03 11:54:46 crc kubenswrapper[5052]: I1203 11:54:46.155912 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/a73e0afd-e037-41f7-b214-d4eebd4bafe0-operator-scripts\") pod \"glance634c-account-delete-nzwqr\" (UID: \"a73e0afd-e037-41f7-b214-d4eebd4bafe0\") " pod="openstack-kuttl-tests/glance634c-account-delete-nzwqr" Dec 03 11:54:46 crc kubenswrapper[5052]: I1203 11:54:46.155953 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-m8pj8\" (UniqueName: \"kubernetes.io/projected/383e7933-2bc4-4ead-bc34-0832091bd952-kube-api-access-m8pj8\") pod \"barbican4385-account-delete-krpzg\" (UID: \"383e7933-2bc4-4ead-bc34-0832091bd952\") " pod="openstack-kuttl-tests/barbican4385-account-delete-krpzg" Dec 03 11:54:46 crc kubenswrapper[5052]: I1203 11:54:46.155990 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/383e7933-2bc4-4ead-bc34-0832091bd952-operator-scripts\") pod \"barbican4385-account-delete-krpzg\" (UID: \"383e7933-2bc4-4ead-bc34-0832091bd952\") " pod="openstack-kuttl-tests/barbican4385-account-delete-krpzg" Dec 03 11:54:46 crc kubenswrapper[5052]: I1203 11:54:46.156016 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zvhll\" (UniqueName: \"kubernetes.io/projected/a73e0afd-e037-41f7-b214-d4eebd4bafe0-kube-api-access-zvhll\") pod \"glance634c-account-delete-nzwqr\" (UID: \"a73e0afd-e037-41f7-b214-d4eebd4bafe0\") " pod="openstack-kuttl-tests/glance634c-account-delete-nzwqr" Dec 03 11:54:46 crc kubenswrapper[5052]: I1203 11:54:46.157065 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/383e7933-2bc4-4ead-bc34-0832091bd952-operator-scripts\") pod \"barbican4385-account-delete-krpzg\" (UID: \"383e7933-2bc4-4ead-bc34-0832091bd952\") " pod="openstack-kuttl-tests/barbican4385-account-delete-krpzg" Dec 03 11:54:46 crc kubenswrapper[5052]: I1203 11:54:46.165640 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-sb-0"] Dec 03 11:54:46 crc kubenswrapper[5052]: I1203 11:54:46.165974 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ovsdbserver-sb-0" podUID="75dd245b-b54f-4a9a-9627-cbf338a69566" containerName="openstack-network-exporter" containerID="cri-o://ed36cfdff4484c95d4ba9ac84fbe8a541c4883ce7afbed30c9761965aace756c" gracePeriod=300 Dec 03 11:54:46 crc kubenswrapper[5052]: I1203 11:54:46.191565 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/placement2868-account-delete-gtkfr"] Dec 03 11:54:46 crc kubenswrapper[5052]: I1203 11:54:46.192679 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement2868-account-delete-gtkfr" Dec 03 11:54:46 crc kubenswrapper[5052]: I1203 11:54:46.226660 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/placement2868-account-delete-gtkfr"] Dec 03 11:54:46 crc kubenswrapper[5052]: I1203 11:54:46.258568 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zq5wj\" (UniqueName: \"kubernetes.io/projected/53d7a9a2-b6a9-42dc-9828-d8ceaa70f0a3-kube-api-access-zq5wj\") pod \"cinder6fd6-account-delete-5z97w\" (UID: \"53d7a9a2-b6a9-42dc-9828-d8ceaa70f0a3\") " pod="openstack-kuttl-tests/cinder6fd6-account-delete-5z97w" Dec 03 11:54:46 crc kubenswrapper[5052]: I1203 11:54:46.258774 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b2d4g\" (UniqueName: \"kubernetes.io/projected/0bda7b03-76fd-405e-9a69-b897331e829c-kube-api-access-b2d4g\") pod \"placement2868-account-delete-gtkfr\" (UID: \"0bda7b03-76fd-405e-9a69-b897331e829c\") " pod="openstack-kuttl-tests/placement2868-account-delete-gtkfr" Dec 03 11:54:46 crc kubenswrapper[5052]: I1203 11:54:46.258855 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/a73e0afd-e037-41f7-b214-d4eebd4bafe0-operator-scripts\") pod \"glance634c-account-delete-nzwqr\" (UID: \"a73e0afd-e037-41f7-b214-d4eebd4bafe0\") " pod="openstack-kuttl-tests/glance634c-account-delete-nzwqr" Dec 03 11:54:46 crc kubenswrapper[5052]: I1203 11:54:46.258898 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/0bda7b03-76fd-405e-9a69-b897331e829c-operator-scripts\") pod \"placement2868-account-delete-gtkfr\" (UID: \"0bda7b03-76fd-405e-9a69-b897331e829c\") " pod="openstack-kuttl-tests/placement2868-account-delete-gtkfr" Dec 03 11:54:46 crc kubenswrapper[5052]: I1203 11:54:46.258955 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/53d7a9a2-b6a9-42dc-9828-d8ceaa70f0a3-operator-scripts\") pod \"cinder6fd6-account-delete-5z97w\" (UID: \"53d7a9a2-b6a9-42dc-9828-d8ceaa70f0a3\") " pod="openstack-kuttl-tests/cinder6fd6-account-delete-5z97w" Dec 03 11:54:46 crc kubenswrapper[5052]: I1203 11:54:46.258986 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-zvhll\" (UniqueName: \"kubernetes.io/projected/a73e0afd-e037-41f7-b214-d4eebd4bafe0-kube-api-access-zvhll\") pod \"glance634c-account-delete-nzwqr\" (UID: \"a73e0afd-e037-41f7-b214-d4eebd4bafe0\") " pod="openstack-kuttl-tests/glance634c-account-delete-nzwqr" Dec 03 11:54:46 crc kubenswrapper[5052]: I1203 11:54:46.261624 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/a73e0afd-e037-41f7-b214-d4eebd4bafe0-operator-scripts\") pod \"glance634c-account-delete-nzwqr\" (UID: \"a73e0afd-e037-41f7-b214-d4eebd4bafe0\") " pod="openstack-kuttl-tests/glance634c-account-delete-nzwqr" Dec 03 11:54:46 crc kubenswrapper[5052]: I1203 11:54:46.301394 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-m8pj8\" (UniqueName: \"kubernetes.io/projected/383e7933-2bc4-4ead-bc34-0832091bd952-kube-api-access-m8pj8\") pod \"barbican4385-account-delete-krpzg\" (UID: \"383e7933-2bc4-4ead-bc34-0832091bd952\") " pod="openstack-kuttl-tests/barbican4385-account-delete-krpzg" Dec 03 11:54:46 crc kubenswrapper[5052]: I1203 11:54:46.364346 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-zvhll\" (UniqueName: \"kubernetes.io/projected/a73e0afd-e037-41f7-b214-d4eebd4bafe0-kube-api-access-zvhll\") pod \"glance634c-account-delete-nzwqr\" (UID: \"a73e0afd-e037-41f7-b214-d4eebd4bafe0\") " pod="openstack-kuttl-tests/glance634c-account-delete-nzwqr" Dec 03 11:54:46 crc kubenswrapper[5052]: I1203 11:54:46.365366 5052 generic.go:334] "Generic (PLEG): container finished" podID="b4961b81-97b8-4174-82bf-2d1d02cf8335" containerID="e54d2995d89269947a87a50c5c5e626a778c66e141e2d728f3fafc46c76e5a9b" exitCode=2 Dec 03 11:54:46 crc kubenswrapper[5052]: I1203 11:54:46.365459 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-nb-0" event={"ID":"b4961b81-97b8-4174-82bf-2d1d02cf8335","Type":"ContainerDied","Data":"e54d2995d89269947a87a50c5c5e626a778c66e141e2d728f3fafc46c76e5a9b"} Dec 03 11:54:46 crc kubenswrapper[5052]: I1203 11:54:46.381100 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-b2d4g\" (UniqueName: \"kubernetes.io/projected/0bda7b03-76fd-405e-9a69-b897331e829c-kube-api-access-b2d4g\") pod \"placement2868-account-delete-gtkfr\" (UID: \"0bda7b03-76fd-405e-9a69-b897331e829c\") " pod="openstack-kuttl-tests/placement2868-account-delete-gtkfr" Dec 03 11:54:46 crc kubenswrapper[5052]: I1203 11:54:46.381244 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/0bda7b03-76fd-405e-9a69-b897331e829c-operator-scripts\") pod \"placement2868-account-delete-gtkfr\" (UID: \"0bda7b03-76fd-405e-9a69-b897331e829c\") " pod="openstack-kuttl-tests/placement2868-account-delete-gtkfr" Dec 03 11:54:46 crc kubenswrapper[5052]: I1203 11:54:46.381313 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/53d7a9a2-b6a9-42dc-9828-d8ceaa70f0a3-operator-scripts\") pod \"cinder6fd6-account-delete-5z97w\" (UID: \"53d7a9a2-b6a9-42dc-9828-d8ceaa70f0a3\") " pod="openstack-kuttl-tests/cinder6fd6-account-delete-5z97w" Dec 03 11:54:46 crc kubenswrapper[5052]: I1203 11:54:46.381455 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-zq5wj\" (UniqueName: \"kubernetes.io/projected/53d7a9a2-b6a9-42dc-9828-d8ceaa70f0a3-kube-api-access-zq5wj\") pod \"cinder6fd6-account-delete-5z97w\" (UID: \"53d7a9a2-b6a9-42dc-9828-d8ceaa70f0a3\") " pod="openstack-kuttl-tests/cinder6fd6-account-delete-5z97w" Dec 03 11:54:46 crc kubenswrapper[5052]: I1203 11:54:46.382611 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/0bda7b03-76fd-405e-9a69-b897331e829c-operator-scripts\") pod \"placement2868-account-delete-gtkfr\" (UID: \"0bda7b03-76fd-405e-9a69-b897331e829c\") " pod="openstack-kuttl-tests/placement2868-account-delete-gtkfr" Dec 03 11:54:46 crc kubenswrapper[5052]: I1203 11:54:46.383277 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/53d7a9a2-b6a9-42dc-9828-d8ceaa70f0a3-operator-scripts\") pod \"cinder6fd6-account-delete-5z97w\" (UID: \"53d7a9a2-b6a9-42dc-9828-d8ceaa70f0a3\") " pod="openstack-kuttl-tests/cinder6fd6-account-delete-5z97w" Dec 03 11:54:46 crc kubenswrapper[5052]: I1203 11:54:46.408861 5052 generic.go:334] "Generic (PLEG): container finished" podID="9d8cd78f-2670-4933-81dd-01cdc0044454" containerID="a00e41ad3b048582d984194b4e78df8f197fd9e5a5af35f2e726a5b8f13fb9fb" exitCode=2 Dec 03 11:54:46 crc kubenswrapper[5052]: I1203 11:54:46.408935 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovn-northd-0" event={"ID":"9d8cd78f-2670-4933-81dd-01cdc0044454","Type":"ContainerDied","Data":"a00e41ad3b048582d984194b4e78df8f197fd9e5a5af35f2e726a5b8f13fb9fb"} Dec 03 11:54:46 crc kubenswrapper[5052]: I1203 11:54:46.413264 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/neutron3cf1-account-delete-zk6qw"] Dec 03 11:54:46 crc kubenswrapper[5052]: I1203 11:54:46.433001 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron3cf1-account-delete-zk6qw" Dec 03 11:54:46 crc kubenswrapper[5052]: I1203 11:54:46.442681 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron3cf1-account-delete-zk6qw"] Dec 03 11:54:46 crc kubenswrapper[5052]: I1203 11:54:46.504189 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-b2d4g\" (UniqueName: \"kubernetes.io/projected/0bda7b03-76fd-405e-9a69-b897331e829c-kube-api-access-b2d4g\") pod \"placement2868-account-delete-gtkfr\" (UID: \"0bda7b03-76fd-405e-9a69-b897331e829c\") " pod="openstack-kuttl-tests/placement2868-account-delete-gtkfr" Dec 03 11:54:46 crc kubenswrapper[5052]: I1203 11:54:46.505116 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-zq5wj\" (UniqueName: \"kubernetes.io/projected/53d7a9a2-b6a9-42dc-9828-d8ceaa70f0a3-kube-api-access-zq5wj\") pod \"cinder6fd6-account-delete-5z97w\" (UID: \"53d7a9a2-b6a9-42dc-9828-d8ceaa70f0a3\") " pod="openstack-kuttl-tests/cinder6fd6-account-delete-5z97w" Dec 03 11:54:46 crc kubenswrapper[5052]: I1203 11:54:46.537850 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/placement-db-sync-nqvdz"] Dec 03 11:54:46 crc kubenswrapper[5052]: I1203 11:54:46.588670 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/0f42494c-058e-417d-9e0d-97ce7040b288-operator-scripts\") pod \"neutron3cf1-account-delete-zk6qw\" (UID: \"0f42494c-058e-417d-9e0d-97ce7040b288\") " pod="openstack-kuttl-tests/neutron3cf1-account-delete-zk6qw" Dec 03 11:54:46 crc kubenswrapper[5052]: I1203 11:54:46.588793 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rsvm8\" (UniqueName: \"kubernetes.io/projected/0f42494c-058e-417d-9e0d-97ce7040b288-kube-api-access-rsvm8\") pod \"neutron3cf1-account-delete-zk6qw\" (UID: \"0f42494c-058e-417d-9e0d-97ce7040b288\") " pod="openstack-kuttl-tests/neutron3cf1-account-delete-zk6qw" Dec 03 11:54:46 crc kubenswrapper[5052]: E1203 11:54:46.588963 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/rabbitmq-cell1-config-data: configmap "rabbitmq-cell1-config-data" not found Dec 03 11:54:46 crc kubenswrapper[5052]: E1203 11:54:46.589017 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a5ba4775-815a-46ef-bd2d-05dfa7513288-config-data podName:a5ba4775-815a-46ef-bd2d-05dfa7513288 nodeName:}" failed. No retries permitted until 2025-12-03 11:54:47.588998541 +0000 UTC m=+2078.177240305 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "config-data" (UniqueName: "kubernetes.io/configmap/a5ba4775-815a-46ef-bd2d-05dfa7513288-config-data") pod "rabbitmq-cell1-server-0" (UID: "a5ba4775-815a-46ef-bd2d-05dfa7513288") : configmap "rabbitmq-cell1-config-data" not found Dec 03 11:54:46 crc kubenswrapper[5052]: I1203 11:54:46.597017 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/placement-db-sync-nqvdz"] Dec 03 11:54:46 crc kubenswrapper[5052]: I1203 11:54:46.648460 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder-db-sync-lgtfz"] Dec 03 11:54:46 crc kubenswrapper[5052]: I1203 11:54:46.689519 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/cinder-db-sync-lgtfz"] Dec 03 11:54:46 crc kubenswrapper[5052]: I1203 11:54:46.696533 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/0f42494c-058e-417d-9e0d-97ce7040b288-operator-scripts\") pod \"neutron3cf1-account-delete-zk6qw\" (UID: \"0f42494c-058e-417d-9e0d-97ce7040b288\") " pod="openstack-kuttl-tests/neutron3cf1-account-delete-zk6qw" Dec 03 11:54:46 crc kubenswrapper[5052]: I1203 11:54:46.696644 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-rsvm8\" (UniqueName: \"kubernetes.io/projected/0f42494c-058e-417d-9e0d-97ce7040b288-kube-api-access-rsvm8\") pod \"neutron3cf1-account-delete-zk6qw\" (UID: \"0f42494c-058e-417d-9e0d-97ce7040b288\") " pod="openstack-kuttl-tests/neutron3cf1-account-delete-zk6qw" Dec 03 11:54:46 crc kubenswrapper[5052]: I1203 11:54:46.697834 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/0f42494c-058e-417d-9e0d-97ce7040b288-operator-scripts\") pod \"neutron3cf1-account-delete-zk6qw\" (UID: \"0f42494c-058e-417d-9e0d-97ce7040b288\") " pod="openstack-kuttl-tests/neutron3cf1-account-delete-zk6qw" Dec 03 11:54:46 crc kubenswrapper[5052]: I1203 11:54:46.748296 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance634c-account-delete-nzwqr" Dec 03 11:54:46 crc kubenswrapper[5052]: I1203 11:54:46.754656 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/neutron-db-sync-ghfwv"] Dec 03 11:54:46 crc kubenswrapper[5052]: I1203 11:54:46.755675 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican4385-account-delete-krpzg" Dec 03 11:54:46 crc kubenswrapper[5052]: I1203 11:54:46.782027 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder6fd6-account-delete-5z97w" Dec 03 11:54:46 crc kubenswrapper[5052]: I1203 11:54:46.793862 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 11:54:46 crc kubenswrapper[5052]: I1203 11:54:46.794258 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-internal-api-0" podUID="d541d73d-2f95-43a8-8409-40f5f2ff54c4" containerName="glance-log" containerID="cri-o://3b60c38b2468b7d978dcdc73d52fcc2371995bf05e7490d1357efab444e714d0" gracePeriod=30 Dec 03 11:54:46 crc kubenswrapper[5052]: I1203 11:54:46.795078 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-internal-api-0" podUID="d541d73d-2f95-43a8-8409-40f5f2ff54c4" containerName="glance-httpd" containerID="cri-o://756be0d767606b67fd07bb5cf4a07ee4a5ec3d33b08c84185318683745d97e1b" gracePeriod=30 Dec 03 11:54:46 crc kubenswrapper[5052]: E1203 11:54:46.801213 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/combined-ca-bundle: secret "combined-ca-bundle" not found Dec 03 11:54:46 crc kubenswrapper[5052]: E1203 11:54:46.801480 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/88e7db65-8253-474b-b115-cd3f6edaf85c-combined-ca-bundle podName:88e7db65-8253-474b-b115-cd3f6edaf85c nodeName:}" failed. No retries permitted until 2025-12-03 11:54:47.301463384 +0000 UTC m=+2077.889705148 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "combined-ca-bundle" (UniqueName: "kubernetes.io/secret/88e7db65-8253-474b-b115-cd3f6edaf85c-combined-ca-bundle") pod "kube-state-metrics-0" (UID: "88e7db65-8253-474b-b115-cd3f6edaf85c") : secret "combined-ca-bundle" not found Dec 03 11:54:46 crc kubenswrapper[5052]: E1203 11:54:46.801953 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-kube-state-metrics-svc: secret "cert-kube-state-metrics-svc" not found Dec 03 11:54:46 crc kubenswrapper[5052]: E1203 11:54:46.801988 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/88e7db65-8253-474b-b115-cd3f6edaf85c-kube-state-metrics-tls-certs podName:88e7db65-8253-474b-b115-cd3f6edaf85c nodeName:}" failed. No retries permitted until 2025-12-03 11:54:47.301981109 +0000 UTC m=+2077.890222863 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls-certs" (UniqueName: "kubernetes.io/secret/88e7db65-8253-474b-b115-cd3f6edaf85c-kube-state-metrics-tls-certs") pod "kube-state-metrics-0" (UID: "88e7db65-8253-474b-b115-cd3f6edaf85c") : secret "cert-kube-state-metrics-svc" not found Dec 03 11:54:46 crc kubenswrapper[5052]: I1203 11:54:46.810734 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-rsvm8\" (UniqueName: \"kubernetes.io/projected/0f42494c-058e-417d-9e0d-97ce7040b288-kube-api-access-rsvm8\") pod \"neutron3cf1-account-delete-zk6qw\" (UID: \"0f42494c-058e-417d-9e0d-97ce7040b288\") " pod="openstack-kuttl-tests/neutron3cf1-account-delete-zk6qw" Dec 03 11:54:46 crc kubenswrapper[5052]: I1203 11:54:46.828837 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/neutron-db-sync-ghfwv"] Dec 03 11:54:46 crc kubenswrapper[5052]: I1203 11:54:46.882944 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ovsdbserver-nb-0" podUID="b4961b81-97b8-4174-82bf-2d1d02cf8335" containerName="ovsdbserver-nb" containerID="cri-o://434f07c085aa632ec76c3e82580e18718305175f93ff194a656a5b66cfbf2180" gracePeriod=300 Dec 03 11:54:46 crc kubenswrapper[5052]: I1203 11:54:46.902574 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/rabbitmq-server-0"] Dec 03 11:54:47 crc kubenswrapper[5052]: I1203 11:54:47.017090 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/novaapie4da-account-delete-mknr9"] Dec 03 11:54:47 crc kubenswrapper[5052]: I1203 11:54:47.027494 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/novaapie4da-account-delete-mknr9" Dec 03 11:54:47 crc kubenswrapper[5052]: I1203 11:54:47.035337 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement2868-account-delete-gtkfr" Dec 03 11:54:47 crc kubenswrapper[5052]: I1203 11:54:47.057519 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/novaapie4da-account-delete-mknr9"] Dec 03 11:54:47 crc kubenswrapper[5052]: E1203 11:54:47.099455 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: EOF, stdout: , stderr: , exit code -1" containerID="434f07c085aa632ec76c3e82580e18718305175f93ff194a656a5b66cfbf2180" cmd=["/usr/bin/pidof","ovsdb-server"] Dec 03 11:54:47 crc kubenswrapper[5052]: E1203 11:54:47.117781 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/rabbitmq-config-data: configmap "rabbitmq-config-data" not found Dec 03 11:54:47 crc kubenswrapper[5052]: E1203 11:54:47.117869 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/10e3094e-f625-4d69-b0e9-358220c0d195-config-data podName:10e3094e-f625-4d69-b0e9-358220c0d195 nodeName:}" failed. No retries permitted until 2025-12-03 11:54:49.117847623 +0000 UTC m=+2079.706089467 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "config-data" (UniqueName: "kubernetes.io/configmap/10e3094e-f625-4d69-b0e9-358220c0d195-config-data") pod "rabbitmq-server-0" (UID: "10e3094e-f625-4d69-b0e9-358220c0d195") : configmap "rabbitmq-config-data" not found Dec 03 11:54:47 crc kubenswrapper[5052]: I1203 11:54:47.123224 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vnvmh\" (UniqueName: \"kubernetes.io/projected/6087dc7e-081b-4392-adb6-205b2aec805f-kube-api-access-vnvmh\") pod \"novaapie4da-account-delete-mknr9\" (UID: \"6087dc7e-081b-4392-adb6-205b2aec805f\") " pod="openstack-kuttl-tests/novaapie4da-account-delete-mknr9" Dec 03 11:54:47 crc kubenswrapper[5052]: I1203 11:54:47.123284 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/6087dc7e-081b-4392-adb6-205b2aec805f-operator-scripts\") pod \"novaapie4da-account-delete-mknr9\" (UID: \"6087dc7e-081b-4392-adb6-205b2aec805f\") " pod="openstack-kuttl-tests/novaapie4da-account-delete-mknr9" Dec 03 11:54:47 crc kubenswrapper[5052]: I1203 11:54:47.125591 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/swift-storage-0"] Dec 03 11:54:47 crc kubenswrapper[5052]: I1203 11:54:47.126133 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="337bf4ce-ed6a-47bb-820d-6d64cd15e16e" containerName="account-server" containerID="cri-o://da24bea2b91d001778224eaa9d20a940c1cfa8aec9ca78a78f4063dc10e39b5e" gracePeriod=30 Dec 03 11:54:47 crc kubenswrapper[5052]: I1203 11:54:47.126272 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="337bf4ce-ed6a-47bb-820d-6d64cd15e16e" containerName="swift-recon-cron" containerID="cri-o://adcb7e1e3f0cfa3f2f565b2ee561330619830866aab79c1bb82dc8a5cacf862d" gracePeriod=30 Dec 03 11:54:47 crc kubenswrapper[5052]: I1203 11:54:47.126315 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="337bf4ce-ed6a-47bb-820d-6d64cd15e16e" containerName="rsync" containerID="cri-o://54a021ff30a08f0f3bd288edbc672e78a69a0e8398c93034e0304c71564937f1" gracePeriod=30 Dec 03 11:54:47 crc kubenswrapper[5052]: I1203 11:54:47.126352 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="337bf4ce-ed6a-47bb-820d-6d64cd15e16e" containerName="object-expirer" containerID="cri-o://308f757a4c5baeeb088bbd69071d883ee0fd5f6717ce1fb801584cc7308cf01e" gracePeriod=30 Dec 03 11:54:47 crc kubenswrapper[5052]: I1203 11:54:47.126388 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="337bf4ce-ed6a-47bb-820d-6d64cd15e16e" containerName="object-updater" containerID="cri-o://0bdc5789ab44d02d9702c5cf3e475db9c7a7ce6bf079ab5b06570c12673c510c" gracePeriod=30 Dec 03 11:54:47 crc kubenswrapper[5052]: I1203 11:54:47.126422 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="337bf4ce-ed6a-47bb-820d-6d64cd15e16e" containerName="object-auditor" containerID="cri-o://ef2775155138fd30b620bc0fbab9074d6f7cd4364b1cbc82c977fd6b48054772" gracePeriod=30 Dec 03 11:54:47 crc kubenswrapper[5052]: I1203 11:54:47.126478 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="337bf4ce-ed6a-47bb-820d-6d64cd15e16e" containerName="object-replicator" containerID="cri-o://bf4d57ce3d071492f7492f8995fdfdb4a50c770c3ee835e5864c7889d9aca849" gracePeriod=30 Dec 03 11:54:47 crc kubenswrapper[5052]: I1203 11:54:47.126532 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="337bf4ce-ed6a-47bb-820d-6d64cd15e16e" containerName="object-server" containerID="cri-o://6b9be92cb0d9072365edb2b4115b091dd2b7a90d8f70424fe8b36078d19bcab6" gracePeriod=30 Dec 03 11:54:47 crc kubenswrapper[5052]: I1203 11:54:47.126577 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="337bf4ce-ed6a-47bb-820d-6d64cd15e16e" containerName="container-updater" containerID="cri-o://aeee80ed3e9e151ab7e9e179efcf5113a7eda44c709ef92d44f8e5e51a28368d" gracePeriod=30 Dec 03 11:54:47 crc kubenswrapper[5052]: I1203 11:54:47.126607 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="337bf4ce-ed6a-47bb-820d-6d64cd15e16e" containerName="container-auditor" containerID="cri-o://6b146d4fbd455766a79f81d255c8552e4579557dae733283b1a6647e1871ce4a" gracePeriod=30 Dec 03 11:54:47 crc kubenswrapper[5052]: I1203 11:54:47.126638 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="337bf4ce-ed6a-47bb-820d-6d64cd15e16e" containerName="container-replicator" containerID="cri-o://bf27f4652a0cd9c65c3371e4a8f8b663bd4dc1a05420dbf79f77e2f00185a31d" gracePeriod=30 Dec 03 11:54:47 crc kubenswrapper[5052]: I1203 11:54:47.126668 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="337bf4ce-ed6a-47bb-820d-6d64cd15e16e" containerName="container-server" containerID="cri-o://cac91cb8a133117b6ac9ef92818b0c43f08504bb99f13b28c87c24ca86efd6f6" gracePeriod=30 Dec 03 11:54:47 crc kubenswrapper[5052]: I1203 11:54:47.126696 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="337bf4ce-ed6a-47bb-820d-6d64cd15e16e" containerName="account-reaper" containerID="cri-o://187fe18924b493edafcac4ae2021f9d0b6680c618116eb5301da25207a0518fb" gracePeriod=30 Dec 03 11:54:47 crc kubenswrapper[5052]: I1203 11:54:47.126724 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="337bf4ce-ed6a-47bb-820d-6d64cd15e16e" containerName="account-auditor" containerID="cri-o://36f10fc8c3ae11fad49f741b63364021479706c1dcab5d1f51a1da9b35e558c2" gracePeriod=30 Dec 03 11:54:47 crc kubenswrapper[5052]: I1203 11:54:47.126755 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="337bf4ce-ed6a-47bb-820d-6d64cd15e16e" containerName="account-replicator" containerID="cri-o://1572718ba303c5fe3c2aff8c645fa6d74e146847e7945778af6c12cfd1f31c90" gracePeriod=30 Dec 03 11:54:47 crc kubenswrapper[5052]: I1203 11:54:47.131148 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/rabbitmq-server-0" podUID="10e3094e-f625-4d69-b0e9-358220c0d195" containerName="rabbitmq" containerID="cri-o://db5a765ca282bd8e9880ef9379b02a9202ec6dc773b029eb3abdd3d2f5581f14" gracePeriod=604800 Dec 03 11:54:47 crc kubenswrapper[5052]: E1203 11:54:47.131288 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of 434f07c085aa632ec76c3e82580e18718305175f93ff194a656a5b66cfbf2180 is running failed: container process not found" containerID="434f07c085aa632ec76c3e82580e18718305175f93ff194a656a5b66cfbf2180" cmd=["/usr/bin/pidof","ovsdb-server"] Dec 03 11:54:47 crc kubenswrapper[5052]: E1203 11:54:47.150152 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of 434f07c085aa632ec76c3e82580e18718305175f93ff194a656a5b66cfbf2180 is running failed: container process not found" containerID="434f07c085aa632ec76c3e82580e18718305175f93ff194a656a5b66cfbf2180" cmd=["/usr/bin/pidof","ovsdb-server"] Dec 03 11:54:47 crc kubenswrapper[5052]: E1203 11:54:47.150237 5052 prober.go:104] "Probe errored" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of 434f07c085aa632ec76c3e82580e18718305175f93ff194a656a5b66cfbf2180 is running failed: container process not found" probeType="Readiness" pod="openstack-kuttl-tests/ovsdbserver-nb-0" podUID="b4961b81-97b8-4174-82bf-2d1d02cf8335" containerName="ovsdbserver-nb" Dec 03 11:54:47 crc kubenswrapper[5052]: I1203 11:54:47.175644 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron3cf1-account-delete-zk6qw" Dec 03 11:54:47 crc kubenswrapper[5052]: I1203 11:54:47.184394 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ovsdbserver-sb-0" podUID="75dd245b-b54f-4a9a-9627-cbf338a69566" containerName="ovsdbserver-sb" containerID="cri-o://a3ef9454b0aa12b5e49b5ada9b84ef790f5b7a8483610db37b8081939ff19a0a" gracePeriod=299 Dec 03 11:54:47 crc kubenswrapper[5052]: I1203 11:54:47.226191 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-vnvmh\" (UniqueName: \"kubernetes.io/projected/6087dc7e-081b-4392-adb6-205b2aec805f-kube-api-access-vnvmh\") pod \"novaapie4da-account-delete-mknr9\" (UID: \"6087dc7e-081b-4392-adb6-205b2aec805f\") " pod="openstack-kuttl-tests/novaapie4da-account-delete-mknr9" Dec 03 11:54:47 crc kubenswrapper[5052]: I1203 11:54:47.226249 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/6087dc7e-081b-4392-adb6-205b2aec805f-operator-scripts\") pod \"novaapie4da-account-delete-mknr9\" (UID: \"6087dc7e-081b-4392-adb6-205b2aec805f\") " pod="openstack-kuttl-tests/novaapie4da-account-delete-mknr9" Dec 03 11:54:47 crc kubenswrapper[5052]: I1203 11:54:47.227102 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/swift-ring-rebalance-g5xx8"] Dec 03 11:54:47 crc kubenswrapper[5052]: I1203 11:54:47.227249 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/6087dc7e-081b-4392-adb6-205b2aec805f-operator-scripts\") pod \"novaapie4da-account-delete-mknr9\" (UID: \"6087dc7e-081b-4392-adb6-205b2aec805f\") " pod="openstack-kuttl-tests/novaapie4da-account-delete-mknr9" Dec 03 11:54:47 crc kubenswrapper[5052]: I1203 11:54:47.282438 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-vnvmh\" (UniqueName: \"kubernetes.io/projected/6087dc7e-081b-4392-adb6-205b2aec805f-kube-api-access-vnvmh\") pod \"novaapie4da-account-delete-mknr9\" (UID: \"6087dc7e-081b-4392-adb6-205b2aec805f\") " pod="openstack-kuttl-tests/novaapie4da-account-delete-mknr9" Dec 03 11:54:47 crc kubenswrapper[5052]: I1203 11:54:47.318614 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 11:54:47 crc kubenswrapper[5052]: I1203 11:54:47.318898 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-external-api-0" podUID="f32095c7-4e6f-43c7-a16f-513e89f7c28e" containerName="glance-log" containerID="cri-o://a9bd9892b49a6bfba8e02d24efa8266231be0e4d99061688b1aeffccdabed81f" gracePeriod=30 Dec 03 11:54:47 crc kubenswrapper[5052]: I1203 11:54:47.319257 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-external-api-0" podUID="f32095c7-4e6f-43c7-a16f-513e89f7c28e" containerName="glance-httpd" containerID="cri-o://9b77bce42854e145583416efda6ad1a83fa0803e3bbc085d98e3a15ec1a50003" gracePeriod=30 Dec 03 11:54:47 crc kubenswrapper[5052]: E1203 11:54:47.328005 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/combined-ca-bundle: secret "combined-ca-bundle" not found Dec 03 11:54:47 crc kubenswrapper[5052]: E1203 11:54:47.328090 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/88e7db65-8253-474b-b115-cd3f6edaf85c-combined-ca-bundle podName:88e7db65-8253-474b-b115-cd3f6edaf85c nodeName:}" failed. No retries permitted until 2025-12-03 11:54:48.328068672 +0000 UTC m=+2078.916310436 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "combined-ca-bundle" (UniqueName: "kubernetes.io/secret/88e7db65-8253-474b-b115-cd3f6edaf85c-combined-ca-bundle") pod "kube-state-metrics-0" (UID: "88e7db65-8253-474b-b115-cd3f6edaf85c") : secret "combined-ca-bundle" not found Dec 03 11:54:47 crc kubenswrapper[5052]: E1203 11:54:47.328143 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-kube-state-metrics-svc: secret "cert-kube-state-metrics-svc" not found Dec 03 11:54:47 crc kubenswrapper[5052]: E1203 11:54:47.328170 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/88e7db65-8253-474b-b115-cd3f6edaf85c-kube-state-metrics-tls-certs podName:88e7db65-8253-474b-b115-cd3f6edaf85c nodeName:}" failed. No retries permitted until 2025-12-03 11:54:48.328161905 +0000 UTC m=+2078.916403669 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls-certs" (UniqueName: "kubernetes.io/secret/88e7db65-8253-474b-b115-cd3f6edaf85c-kube-state-metrics-tls-certs") pod "kube-state-metrics-0" (UID: "88e7db65-8253-474b-b115-cd3f6edaf85c") : secret "cert-kube-state-metrics-svc" not found Dec 03 11:54:47 crc kubenswrapper[5052]: I1203 11:54:47.347013 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/novacell0745f-account-delete-zbhjv"] Dec 03 11:54:47 crc kubenswrapper[5052]: I1203 11:54:47.348380 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/novacell0745f-account-delete-zbhjv" Dec 03 11:54:47 crc kubenswrapper[5052]: I1203 11:54:47.367477 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/swift-ring-rebalance-g5xx8"] Dec 03 11:54:47 crc kubenswrapper[5052]: I1203 11:54:47.378603 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell1-cell-mapping-ptcx4"] Dec 03 11:54:47 crc kubenswrapper[5052]: I1203 11:54:47.386079 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell1-cell-mapping-ptcx4"] Dec 03 11:54:47 crc kubenswrapper[5052]: I1203 11:54:47.396894 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/novacell0745f-account-delete-zbhjv"] Dec 03 11:54:47 crc kubenswrapper[5052]: I1203 11:54:47.407598 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell0-cell-mapping-7v46q"] Dec 03 11:54:47 crc kubenswrapper[5052]: I1203 11:54:47.425467 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell0-cell-mapping-7v46q"] Dec 03 11:54:47 crc kubenswrapper[5052]: I1203 11:54:47.430920 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/93d03e32-f318-470c-ad5d-f17a609f5003-operator-scripts\") pod \"novacell0745f-account-delete-zbhjv\" (UID: \"93d03e32-f318-470c-ad5d-f17a609f5003\") " pod="openstack-kuttl-tests/novacell0745f-account-delete-zbhjv" Dec 03 11:54:47 crc kubenswrapper[5052]: I1203 11:54:47.431083 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sxkjh\" (UniqueName: \"kubernetes.io/projected/93d03e32-f318-470c-ad5d-f17a609f5003-kube-api-access-sxkjh\") pod \"novacell0745f-account-delete-zbhjv\" (UID: \"93d03e32-f318-470c-ad5d-f17a609f5003\") " pod="openstack-kuttl-tests/novacell0745f-account-delete-zbhjv" Dec 03 11:54:47 crc kubenswrapper[5052]: I1203 11:54:47.438466 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 11:54:47 crc kubenswrapper[5052]: I1203 11:54:47.438853 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/cinder-api-0" podUID="9a4b1245-d95c-4eb6-8d2b-fdd93d98bb1e" containerName="cinder-api-log" containerID="cri-o://8c286da6c5ac86a7bafdea57cd407cfa1b12fb35b2f4dd6c013af4a83aa8fbb3" gracePeriod=30 Dec 03 11:54:47 crc kubenswrapper[5052]: I1203 11:54:47.438953 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/cinder-api-0" podUID="9a4b1245-d95c-4eb6-8d2b-fdd93d98bb1e" containerName="cinder-api" containerID="cri-o://b4835f9bc39d3950a321af04adbfa74294d65285b1d19075ec93d354b42417ef" gracePeriod=30 Dec 03 11:54:47 crc kubenswrapper[5052]: I1203 11:54:47.438959 5052 generic.go:334] "Generic (PLEG): container finished" podID="d541d73d-2f95-43a8-8409-40f5f2ff54c4" containerID="3b60c38b2468b7d978dcdc73d52fcc2371995bf05e7490d1357efab444e714d0" exitCode=143 Dec 03 11:54:47 crc kubenswrapper[5052]: I1203 11:54:47.438999 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"d541d73d-2f95-43a8-8409-40f5f2ff54c4","Type":"ContainerDied","Data":"3b60c38b2468b7d978dcdc73d52fcc2371995bf05e7490d1357efab444e714d0"} Dec 03 11:54:47 crc kubenswrapper[5052]: I1203 11:54:47.449952 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/neutron-5ff446f48b-7hc25"] Dec 03 11:54:47 crc kubenswrapper[5052]: I1203 11:54:47.450177 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/neutron-5ff446f48b-7hc25" podUID="83b9fcfd-8007-4d6b-9090-d2e536cb1bbc" containerName="neutron-api" containerID="cri-o://cd35d473482d3a4b6bc6f957931d66333a4d95cf45ac228f925b1160c222f1f6" gracePeriod=30 Dec 03 11:54:47 crc kubenswrapper[5052]: I1203 11:54:47.450552 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/neutron-5ff446f48b-7hc25" podUID="83b9fcfd-8007-4d6b-9090-d2e536cb1bbc" containerName="neutron-httpd" containerID="cri-o://d4b83b49b57a27fa3f6b26ff04edd361dfcc21c0d52243555076b619c81a9ec9" gracePeriod=30 Dec 03 11:54:47 crc kubenswrapper[5052]: I1203 11:54:47.454774 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-worker-6d7f657c65-cqxbg"] Dec 03 11:54:47 crc kubenswrapper[5052]: I1203 11:54:47.455001 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/barbican-worker-6d7f657c65-cqxbg" podUID="584deaba-f945-42a2-8c0d-e531d6f48fd8" containerName="barbican-worker-log" containerID="cri-o://2ee3c0361bbf90cf40dcd13279e44150c5f8ed97f268cf91a9701bccff60b20b" gracePeriod=30 Dec 03 11:54:47 crc kubenswrapper[5052]: I1203 11:54:47.458113 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/barbican-worker-6d7f657c65-cqxbg" podUID="584deaba-f945-42a2-8c0d-e531d6f48fd8" containerName="barbican-worker" containerID="cri-o://3e624a11d1616304ada395cbc34d76126b3b0f68d661f5c753eefaa96b499a5c" gracePeriod=30 Dec 03 11:54:47 crc kubenswrapper[5052]: I1203 11:54:47.467206 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_ovsdbserver-nb-0_b4961b81-97b8-4174-82bf-2d1d02cf8335/ovsdbserver-nb/0.log" Dec 03 11:54:47 crc kubenswrapper[5052]: I1203 11:54:47.467273 5052 generic.go:334] "Generic (PLEG): container finished" podID="b4961b81-97b8-4174-82bf-2d1d02cf8335" containerID="434f07c085aa632ec76c3e82580e18718305175f93ff194a656a5b66cfbf2180" exitCode=143 Dec 03 11:54:47 crc kubenswrapper[5052]: I1203 11:54:47.467358 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-nb-0" event={"ID":"b4961b81-97b8-4174-82bf-2d1d02cf8335","Type":"ContainerDied","Data":"434f07c085aa632ec76c3e82580e18718305175f93ff194a656a5b66cfbf2180"} Dec 03 11:54:47 crc kubenswrapper[5052]: I1203 11:54:47.479111 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-api-5758478fc6-5g46v"] Dec 03 11:54:47 crc kubenswrapper[5052]: I1203 11:54:47.479462 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/barbican-api-5758478fc6-5g46v" podUID="a18602ce-7206-4edc-a8d3-5d0754c3da02" containerName="barbican-api-log" containerID="cri-o://b8bffbbe67b941eebfcc7d809311107fe0d16c3fd77f96571d99325342cabd72" gracePeriod=30 Dec 03 11:54:47 crc kubenswrapper[5052]: I1203 11:54:47.479640 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/barbican-api-5758478fc6-5g46v" podUID="a18602ce-7206-4edc-a8d3-5d0754c3da02" containerName="barbican-api" containerID="cri-o://9684094fee34437582198b41d3a72ef6d576587c1466aadd379f8be1cedcf6af" gracePeriod=30 Dec 03 11:54:47 crc kubenswrapper[5052]: I1203 11:54:47.488765 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 11:54:47 crc kubenswrapper[5052]: I1203 11:54:47.489060 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/cinder-scheduler-0" podUID="0201d941-cd05-4d1c-9af6-f3a82a375a0f" containerName="cinder-scheduler" containerID="cri-o://24b9c22c01a17699dc23692d82c8a6d2aba256d2fe1411900dc2de65cbd33d93" gracePeriod=30 Dec 03 11:54:47 crc kubenswrapper[5052]: I1203 11:54:47.489483 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/cinder-scheduler-0" podUID="0201d941-cd05-4d1c-9af6-f3a82a375a0f" containerName="probe" containerID="cri-o://f0df1e8717d84a41bd64b8ee9a76553b85d16b020416be110d56d085ed4f7b45" gracePeriod=30 Dec 03 11:54:47 crc kubenswrapper[5052]: I1203 11:54:47.496118 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_ovn-northd-0_9d8cd78f-2670-4933-81dd-01cdc0044454/ovn-northd/0.log" Dec 03 11:54:47 crc kubenswrapper[5052]: I1203 11:54:47.496169 5052 generic.go:334] "Generic (PLEG): container finished" podID="9d8cd78f-2670-4933-81dd-01cdc0044454" containerID="2cf5c7f704cdea9711753e85146be0ca1fd13eb56b9658c079f3f91d897ce7a3" exitCode=143 Dec 03 11:54:47 crc kubenswrapper[5052]: I1203 11:54:47.496193 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovn-northd-0" event={"ID":"9d8cd78f-2670-4933-81dd-01cdc0044454","Type":"ContainerDied","Data":"2cf5c7f704cdea9711753e85146be0ca1fd13eb56b9658c079f3f91d897ce7a3"} Dec 03 11:54:47 crc kubenswrapper[5052]: I1203 11:54:47.499113 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/novaapie4da-account-delete-mknr9" Dec 03 11:54:47 crc kubenswrapper[5052]: I1203 11:54:47.512876 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/placement-5fc4755558-qz76x"] Dec 03 11:54:47 crc kubenswrapper[5052]: I1203 11:54:47.513131 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/placement-5fc4755558-qz76x" podUID="428e9203-aa9c-48d3-84c9-4dfaa806140d" containerName="placement-log" containerID="cri-o://932117c4bd5dd37889c00f1528583dfd60eee1eb15198c88c121a5439e07b994" gracePeriod=30 Dec 03 11:54:47 crc kubenswrapper[5052]: I1203 11:54:47.513248 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/placement-5fc4755558-qz76x" podUID="428e9203-aa9c-48d3-84c9-4dfaa806140d" containerName="placement-api" containerID="cri-o://f7f46b76f15d594e9e319920752ee78fe4901c945bcac1aa5273dd6bd20b6164" gracePeriod=30 Dec 03 11:54:47 crc kubenswrapper[5052]: I1203 11:54:47.523818 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-keystone-listener-7bf89d7c74-c4srn"] Dec 03 11:54:47 crc kubenswrapper[5052]: I1203 11:54:47.524084 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/barbican-keystone-listener-7bf89d7c74-c4srn" podUID="5679a95d-a498-448e-97ed-497496520506" containerName="barbican-keystone-listener-log" containerID="cri-o://b657a1c95ebe202d32286143c3e1cb3b321d2c67300afcf9cb53509b8a6aee42" gracePeriod=30 Dec 03 11:54:47 crc kubenswrapper[5052]: I1203 11:54:47.524540 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/barbican-keystone-listener-7bf89d7c74-c4srn" podUID="5679a95d-a498-448e-97ed-497496520506" containerName="barbican-keystone-listener" containerID="cri-o://3125ced637b6686a088afdde74ed21eda056777bed8dca841a4e6549c3cf70c8" gracePeriod=30 Dec 03 11:54:47 crc kubenswrapper[5052]: I1203 11:54:47.529759 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_ovsdbserver-sb-0_75dd245b-b54f-4a9a-9627-cbf338a69566/ovsdbserver-sb/0.log" Dec 03 11:54:47 crc kubenswrapper[5052]: I1203 11:54:47.529805 5052 generic.go:334] "Generic (PLEG): container finished" podID="75dd245b-b54f-4a9a-9627-cbf338a69566" containerID="ed36cfdff4484c95d4ba9ac84fbe8a541c4883ce7afbed30c9761965aace756c" exitCode=2 Dec 03 11:54:47 crc kubenswrapper[5052]: I1203 11:54:47.529924 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-sb-0" event={"ID":"75dd245b-b54f-4a9a-9627-cbf338a69566","Type":"ContainerDied","Data":"ed36cfdff4484c95d4ba9ac84fbe8a541c4883ce7afbed30c9761965aace756c"} Dec 03 11:54:47 crc kubenswrapper[5052]: I1203 11:54:47.533202 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-sxkjh\" (UniqueName: \"kubernetes.io/projected/93d03e32-f318-470c-ad5d-f17a609f5003-kube-api-access-sxkjh\") pod \"novacell0745f-account-delete-zbhjv\" (UID: \"93d03e32-f318-470c-ad5d-f17a609f5003\") " pod="openstack-kuttl-tests/novacell0745f-account-delete-zbhjv" Dec 03 11:54:47 crc kubenswrapper[5052]: I1203 11:54:47.533586 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/93d03e32-f318-470c-ad5d-f17a609f5003-operator-scripts\") pod \"novacell0745f-account-delete-zbhjv\" (UID: \"93d03e32-f318-470c-ad5d-f17a609f5003\") " pod="openstack-kuttl-tests/novacell0745f-account-delete-zbhjv" Dec 03 11:54:47 crc kubenswrapper[5052]: I1203 11:54:47.534554 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/93d03e32-f318-470c-ad5d-f17a609f5003-operator-scripts\") pod \"novacell0745f-account-delete-zbhjv\" (UID: \"93d03e32-f318-470c-ad5d-f17a609f5003\") " pod="openstack-kuttl-tests/novacell0745f-account-delete-zbhjv" Dec 03 11:54:47 crc kubenswrapper[5052]: I1203 11:54:47.542203 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 11:54:47 crc kubenswrapper[5052]: I1203 11:54:47.542679 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-metadata-0" podUID="653882ec-4522-4979-b1aa-7fad5023605c" containerName="nova-metadata-log" containerID="cri-o://cbf6b026a9b62a7464f44bbde7effeb8a0aa796789269fbcd6b9b3589fc439c9" gracePeriod=30 Dec 03 11:54:47 crc kubenswrapper[5052]: I1203 11:54:47.542861 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-metadata-0" podUID="653882ec-4522-4979-b1aa-7fad5023605c" containerName="nova-metadata-metadata" containerID="cri-o://5c59824fa390c99841f43488a426aa7880afaa22f074091daf68f834930db61d" gracePeriod=30 Dec 03 11:54:47 crc kubenswrapper[5052]: I1203 11:54:47.558312 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-z8rn9"] Dec 03 11:54:47 crc kubenswrapper[5052]: I1203 11:54:47.571256 5052 generic.go:334] "Generic (PLEG): container finished" podID="337bf4ce-ed6a-47bb-820d-6d64cd15e16e" containerID="aeee80ed3e9e151ab7e9e179efcf5113a7eda44c709ef92d44f8e5e51a28368d" exitCode=0 Dec 03 11:54:47 crc kubenswrapper[5052]: I1203 11:54:47.571298 5052 generic.go:334] "Generic (PLEG): container finished" podID="337bf4ce-ed6a-47bb-820d-6d64cd15e16e" containerID="6b146d4fbd455766a79f81d255c8552e4579557dae733283b1a6647e1871ce4a" exitCode=0 Dec 03 11:54:47 crc kubenswrapper[5052]: I1203 11:54:47.571312 5052 generic.go:334] "Generic (PLEG): container finished" podID="337bf4ce-ed6a-47bb-820d-6d64cd15e16e" containerID="bf27f4652a0cd9c65c3371e4a8f8b663bd4dc1a05420dbf79f77e2f00185a31d" exitCode=0 Dec 03 11:54:47 crc kubenswrapper[5052]: I1203 11:54:47.571321 5052 generic.go:334] "Generic (PLEG): container finished" podID="337bf4ce-ed6a-47bb-820d-6d64cd15e16e" containerID="187fe18924b493edafcac4ae2021f9d0b6680c618116eb5301da25207a0518fb" exitCode=0 Dec 03 11:54:47 crc kubenswrapper[5052]: I1203 11:54:47.571328 5052 generic.go:334] "Generic (PLEG): container finished" podID="337bf4ce-ed6a-47bb-820d-6d64cd15e16e" containerID="36f10fc8c3ae11fad49f741b63364021479706c1dcab5d1f51a1da9b35e558c2" exitCode=0 Dec 03 11:54:47 crc kubenswrapper[5052]: I1203 11:54:47.571339 5052 generic.go:334] "Generic (PLEG): container finished" podID="337bf4ce-ed6a-47bb-820d-6d64cd15e16e" containerID="1572718ba303c5fe3c2aff8c645fa6d74e146847e7945778af6c12cfd1f31c90" exitCode=0 Dec 03 11:54:47 crc kubenswrapper[5052]: I1203 11:54:47.571361 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"337bf4ce-ed6a-47bb-820d-6d64cd15e16e","Type":"ContainerDied","Data":"aeee80ed3e9e151ab7e9e179efcf5113a7eda44c709ef92d44f8e5e51a28368d"} Dec 03 11:54:47 crc kubenswrapper[5052]: I1203 11:54:47.571393 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"337bf4ce-ed6a-47bb-820d-6d64cd15e16e","Type":"ContainerDied","Data":"6b146d4fbd455766a79f81d255c8552e4579557dae733283b1a6647e1871ce4a"} Dec 03 11:54:47 crc kubenswrapper[5052]: I1203 11:54:47.571408 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"337bf4ce-ed6a-47bb-820d-6d64cd15e16e","Type":"ContainerDied","Data":"bf27f4652a0cd9c65c3371e4a8f8b663bd4dc1a05420dbf79f77e2f00185a31d"} Dec 03 11:54:47 crc kubenswrapper[5052]: I1203 11:54:47.571418 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"337bf4ce-ed6a-47bb-820d-6d64cd15e16e","Type":"ContainerDied","Data":"187fe18924b493edafcac4ae2021f9d0b6680c618116eb5301da25207a0518fb"} Dec 03 11:54:47 crc kubenswrapper[5052]: I1203 11:54:47.571430 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"337bf4ce-ed6a-47bb-820d-6d64cd15e16e","Type":"ContainerDied","Data":"36f10fc8c3ae11fad49f741b63364021479706c1dcab5d1f51a1da9b35e558c2"} Dec 03 11:54:47 crc kubenswrapper[5052]: I1203 11:54:47.571442 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"337bf4ce-ed6a-47bb-820d-6d64cd15e16e","Type":"ContainerDied","Data":"1572718ba303c5fe3c2aff8c645fa6d74e146847e7945778af6c12cfd1f31c90"} Dec 03 11:54:47 crc kubenswrapper[5052]: I1203 11:54:47.572383 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-z8rn9" Dec 03 11:54:47 crc kubenswrapper[5052]: I1203 11:54:47.577936 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/openstack-cell1-galera-0"] Dec 03 11:54:47 crc kubenswrapper[5052]: I1203 11:54:47.585039 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-sxkjh\" (UniqueName: \"kubernetes.io/projected/93d03e32-f318-470c-ad5d-f17a609f5003-kube-api-access-sxkjh\") pod \"novacell0745f-account-delete-zbhjv\" (UID: \"93d03e32-f318-470c-ad5d-f17a609f5003\") " pod="openstack-kuttl-tests/novacell0745f-account-delete-zbhjv" Dec 03 11:54:47 crc kubenswrapper[5052]: I1203 11:54:47.602731 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-z8rn9"] Dec 03 11:54:47 crc kubenswrapper[5052]: I1203 11:54:47.615794 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 11:54:47 crc kubenswrapper[5052]: I1203 11:54:47.616045 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-scheduler-0" podUID="ec16e8c3-9750-4475-a671-17b0c16826c8" containerName="nova-scheduler-scheduler" containerID="cri-o://21459936e0c2e5ac30de5ff0dac1297b633e3badacbcf9ad455e81ffb5671c96" gracePeriod=30 Dec 03 11:54:47 crc kubenswrapper[5052]: I1203 11:54:47.635714 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/97767bcf-c9a4-430f-8298-3c677150b865-config\") pod \"dnsmasq-dnsmasq-84b9f45d47-z8rn9\" (UID: \"97767bcf-c9a4-430f-8298-3c677150b865\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-z8rn9" Dec 03 11:54:47 crc kubenswrapper[5052]: I1203 11:54:47.635802 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/97767bcf-c9a4-430f-8298-3c677150b865-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-84b9f45d47-z8rn9\" (UID: \"97767bcf-c9a4-430f-8298-3c677150b865\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-z8rn9" Dec 03 11:54:47 crc kubenswrapper[5052]: I1203 11:54:47.635875 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ts94l\" (UniqueName: \"kubernetes.io/projected/97767bcf-c9a4-430f-8298-3c677150b865-kube-api-access-ts94l\") pod \"dnsmasq-dnsmasq-84b9f45d47-z8rn9\" (UID: \"97767bcf-c9a4-430f-8298-3c677150b865\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-z8rn9" Dec 03 11:54:47 crc kubenswrapper[5052]: E1203 11:54:47.636098 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/rabbitmq-cell1-config-data: configmap "rabbitmq-cell1-config-data" not found Dec 03 11:54:47 crc kubenswrapper[5052]: E1203 11:54:47.636151 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a5ba4775-815a-46ef-bd2d-05dfa7513288-config-data podName:a5ba4775-815a-46ef-bd2d-05dfa7513288 nodeName:}" failed. No retries permitted until 2025-12-03 11:54:49.636132062 +0000 UTC m=+2080.224373826 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "config-data" (UniqueName: "kubernetes.io/configmap/a5ba4775-815a-46ef-bd2d-05dfa7513288-config-data") pod "rabbitmq-cell1-server-0" (UID: "a5ba4775-815a-46ef-bd2d-05dfa7513288") : configmap "rabbitmq-cell1-config-data" not found Dec 03 11:54:47 crc kubenswrapper[5052]: I1203 11:54:47.644502 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell1-db-create-8vmt9"] Dec 03 11:54:47 crc kubenswrapper[5052]: I1203 11:54:47.659318 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell1-db-create-8vmt9"] Dec 03 11:54:47 crc kubenswrapper[5052]: I1203 11:54:47.670580 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/novacell0745f-account-delete-zbhjv" Dec 03 11:54:47 crc kubenswrapper[5052]: I1203 11:54:47.671017 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/rabbitmq-cell1-server-0"] Dec 03 11:54:47 crc kubenswrapper[5052]: I1203 11:54:47.707951 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="18c1240f-c951-40de-a0da-3b50f11f4cf1" path="/var/lib/kubelet/pods/18c1240f-c951-40de-a0da-3b50f11f4cf1/volumes" Dec 03 11:54:47 crc kubenswrapper[5052]: I1203 11:54:47.708636 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3024dd24-2f4a-454d-b7a6-d7ba158c6f33" path="/var/lib/kubelet/pods/3024dd24-2f4a-454d-b7a6-d7ba158c6f33/volumes" Dec 03 11:54:47 crc kubenswrapper[5052]: I1203 11:54:47.709224 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3a49f2d7-59af-4bc8-8d05-641b33c7a24c" path="/var/lib/kubelet/pods/3a49f2d7-59af-4bc8-8d05-641b33c7a24c/volumes" Dec 03 11:54:47 crc kubenswrapper[5052]: I1203 11:54:47.712762 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5ddb757c-c235-4a2f-b6db-0d120215edcf" path="/var/lib/kubelet/pods/5ddb757c-c235-4a2f-b6db-0d120215edcf/volumes" Dec 03 11:54:47 crc kubenswrapper[5052]: I1203 11:54:47.713753 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6500a947-382b-4269-b31b-5b518c858a6f" path="/var/lib/kubelet/pods/6500a947-382b-4269-b31b-5b518c858a6f/volumes" Dec 03 11:54:47 crc kubenswrapper[5052]: I1203 11:54:47.714332 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="97107a61-a19d-4494-bcdb-a8d8f6223cb5" path="/var/lib/kubelet/pods/97107a61-a19d-4494-bcdb-a8d8f6223cb5/volumes" Dec 03 11:54:47 crc kubenswrapper[5052]: I1203 11:54:47.714964 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a7ea647c-c7da-4001-befa-c8412db3570b" path="/var/lib/kubelet/pods/a7ea647c-c7da-4001-befa-c8412db3570b/volumes" Dec 03 11:54:47 crc kubenswrapper[5052]: I1203 11:54:47.718496 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b72434ac-6b58-49a8-aaa1-6d5aa7b920f2" path="/var/lib/kubelet/pods/b72434ac-6b58-49a8-aaa1-6d5aa7b920f2/volumes" Dec 03 11:54:47 crc kubenswrapper[5052]: I1203 11:54:47.719075 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d6eac989-1eec-4b3a-b9d7-d5a8da2c1a27" path="/var/lib/kubelet/pods/d6eac989-1eec-4b3a-b9d7-d5a8da2c1a27/volumes" Dec 03 11:54:47 crc kubenswrapper[5052]: I1203 11:54:47.719606 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 11:54:47 crc kubenswrapper[5052]: I1203 11:54:47.719649 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell1-4fec-account-create-update-x8rjw"] Dec 03 11:54:47 crc kubenswrapper[5052]: I1203 11:54:47.719660 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 11:54:47 crc kubenswrapper[5052]: I1203 11:54:47.719674 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell1-4fec-account-create-update-x8rjw"] Dec 03 11:54:47 crc kubenswrapper[5052]: I1203 11:54:47.720017 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-api-0" podUID="d3222c69-47bc-46d8-a025-cc457881cac9" containerName="nova-api-api" containerID="cri-o://13ed5d128b8c9df19939f5f214d29b30d31dc1296f2560559e840f9d3ffbfd12" gracePeriod=30 Dec 03 11:54:47 crc kubenswrapper[5052]: I1203 11:54:47.720036 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" podUID="da3a810d-9fd0-4f18-8f4a-e896b94b1431" containerName="nova-cell1-novncproxy-novncproxy" containerID="cri-o://80ffd33af4023d6b6f0f24ec3bd666459e45b4216c33dab791e932793c7535bf" gracePeriod=30 Dec 03 11:54:47 crc kubenswrapper[5052]: I1203 11:54:47.720293 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-api-0" podUID="d3222c69-47bc-46d8-a025-cc457881cac9" containerName="nova-api-log" containerID="cri-o://d7f3ac6af5a2c0d33f118ffd9646c754d018fd85708d55c58a60a5cc00574cd1" gracePeriod=30 Dec 03 11:54:47 crc kubenswrapper[5052]: I1203 11:54:47.737395 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/97767bcf-c9a4-430f-8298-3c677150b865-config\") pod \"dnsmasq-dnsmasq-84b9f45d47-z8rn9\" (UID: \"97767bcf-c9a4-430f-8298-3c677150b865\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-z8rn9" Dec 03 11:54:47 crc kubenswrapper[5052]: I1203 11:54:47.737499 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/97767bcf-c9a4-430f-8298-3c677150b865-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-84b9f45d47-z8rn9\" (UID: \"97767bcf-c9a4-430f-8298-3c677150b865\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-z8rn9" Dec 03 11:54:47 crc kubenswrapper[5052]: I1203 11:54:47.737681 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-ts94l\" (UniqueName: \"kubernetes.io/projected/97767bcf-c9a4-430f-8298-3c677150b865-kube-api-access-ts94l\") pod \"dnsmasq-dnsmasq-84b9f45d47-z8rn9\" (UID: \"97767bcf-c9a4-430f-8298-3c677150b865\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-z8rn9" Dec 03 11:54:47 crc kubenswrapper[5052]: I1203 11:54:47.748645 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/97767bcf-c9a4-430f-8298-3c677150b865-config\") pod \"dnsmasq-dnsmasq-84b9f45d47-z8rn9\" (UID: \"97767bcf-c9a4-430f-8298-3c677150b865\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-z8rn9" Dec 03 11:54:47 crc kubenswrapper[5052]: I1203 11:54:47.749545 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/97767bcf-c9a4-430f-8298-3c677150b865-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-84b9f45d47-z8rn9\" (UID: \"97767bcf-c9a4-430f-8298-3c677150b865\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-z8rn9" Dec 03 11:54:47 crc kubenswrapper[5052]: I1203 11:54:47.754150 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" podUID="a5ba4775-815a-46ef-bd2d-05dfa7513288" containerName="rabbitmq" containerID="cri-o://c25fbe13b5143561b88ab53fac63216ac379a26107913a3a532fbd7033bbeba6" gracePeriod=604800 Dec 03 11:54:47 crc kubenswrapper[5052]: I1203 11:54:47.798568 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-ts94l\" (UniqueName: \"kubernetes.io/projected/97767bcf-c9a4-430f-8298-3c677150b865-kube-api-access-ts94l\") pod \"dnsmasq-dnsmasq-84b9f45d47-z8rn9\" (UID: \"97767bcf-c9a4-430f-8298-3c677150b865\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-z8rn9" Dec 03 11:54:47 crc kubenswrapper[5052]: I1203 11:54:47.980210 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-z8rn9" Dec 03 11:54:48 crc kubenswrapper[5052]: I1203 11:54:48.028664 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/openstack-cell1-galera-0" podUID="8dd63158-2c75-455f-8f58-57a55c7e0f22" containerName="galera" containerID="cri-o://77ae7474c6477ac30175222ea524c1ac6cf15f586c4f63872104bcf7083b3836" gracePeriod=30 Dec 03 11:54:48 crc kubenswrapper[5052]: I1203 11:54:48.084723 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance634c-account-delete-nzwqr"] Dec 03 11:54:48 crc kubenswrapper[5052]: I1203 11:54:48.282269 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_ovn-northd-0_9d8cd78f-2670-4933-81dd-01cdc0044454/ovn-northd/0.log" Dec 03 11:54:48 crc kubenswrapper[5052]: I1203 11:54:48.282621 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 11:54:48 crc kubenswrapper[5052]: I1203 11:54:48.339461 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_ovsdbserver-nb-0_b4961b81-97b8-4174-82bf-2d1d02cf8335/ovsdbserver-nb/0.log" Dec 03 11:54:48 crc kubenswrapper[5052]: I1203 11:54:48.339575 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 11:54:48 crc kubenswrapper[5052]: I1203 11:54:48.378016 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/b4961b81-97b8-4174-82bf-2d1d02cf8335-config\") pod \"b4961b81-97b8-4174-82bf-2d1d02cf8335\" (UID: \"b4961b81-97b8-4174-82bf-2d1d02cf8335\") " Dec 03 11:54:48 crc kubenswrapper[5052]: I1203 11:54:48.378226 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/9d8cd78f-2670-4933-81dd-01cdc0044454-scripts\") pod \"9d8cd78f-2670-4933-81dd-01cdc0044454\" (UID: \"9d8cd78f-2670-4933-81dd-01cdc0044454\") " Dec 03 11:54:48 crc kubenswrapper[5052]: I1203 11:54:48.378335 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b4961b81-97b8-4174-82bf-2d1d02cf8335-combined-ca-bundle\") pod \"b4961b81-97b8-4174-82bf-2d1d02cf8335\" (UID: \"b4961b81-97b8-4174-82bf-2d1d02cf8335\") " Dec 03 11:54:48 crc kubenswrapper[5052]: I1203 11:54:48.378424 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/9d8cd78f-2670-4933-81dd-01cdc0044454-config\") pod \"9d8cd78f-2670-4933-81dd-01cdc0044454\" (UID: \"9d8cd78f-2670-4933-81dd-01cdc0044454\") " Dec 03 11:54:48 crc kubenswrapper[5052]: I1203 11:54:48.378501 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovn-northd-tls-certs\" (UniqueName: \"kubernetes.io/secret/9d8cd78f-2670-4933-81dd-01cdc0044454-ovn-northd-tls-certs\") pod \"9d8cd78f-2670-4933-81dd-01cdc0044454\" (UID: \"9d8cd78f-2670-4933-81dd-01cdc0044454\") " Dec 03 11:54:48 crc kubenswrapper[5052]: I1203 11:54:48.378635 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/b4961b81-97b8-4174-82bf-2d1d02cf8335-ovsdb-rundir\") pod \"b4961b81-97b8-4174-82bf-2d1d02cf8335\" (UID: \"b4961b81-97b8-4174-82bf-2d1d02cf8335\") " Dec 03 11:54:48 crc kubenswrapper[5052]: I1203 11:54:48.378721 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovndbcluster-nb-etc-ovn\" (UniqueName: \"kubernetes.io/local-volume/local-storage19-crc\") pod \"b4961b81-97b8-4174-82bf-2d1d02cf8335\" (UID: \"b4961b81-97b8-4174-82bf-2d1d02cf8335\") " Dec 03 11:54:48 crc kubenswrapper[5052]: I1203 11:54:48.378820 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/b4961b81-97b8-4174-82bf-2d1d02cf8335-metrics-certs-tls-certs\") pod \"b4961b81-97b8-4174-82bf-2d1d02cf8335\" (UID: \"b4961b81-97b8-4174-82bf-2d1d02cf8335\") " Dec 03 11:54:48 crc kubenswrapper[5052]: I1203 11:54:48.378924 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovsdbserver-nb-tls-certs\" (UniqueName: \"kubernetes.io/secret/b4961b81-97b8-4174-82bf-2d1d02cf8335-ovsdbserver-nb-tls-certs\") pod \"b4961b81-97b8-4174-82bf-2d1d02cf8335\" (UID: \"b4961b81-97b8-4174-82bf-2d1d02cf8335\") " Dec 03 11:54:48 crc kubenswrapper[5052]: I1203 11:54:48.379054 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovn-rundir\" (UniqueName: \"kubernetes.io/empty-dir/9d8cd78f-2670-4933-81dd-01cdc0044454-ovn-rundir\") pod \"9d8cd78f-2670-4933-81dd-01cdc0044454\" (UID: \"9d8cd78f-2670-4933-81dd-01cdc0044454\") " Dec 03 11:54:48 crc kubenswrapper[5052]: I1203 11:54:48.379184 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/b4961b81-97b8-4174-82bf-2d1d02cf8335-scripts\") pod \"b4961b81-97b8-4174-82bf-2d1d02cf8335\" (UID: \"b4961b81-97b8-4174-82bf-2d1d02cf8335\") " Dec 03 11:54:48 crc kubenswrapper[5052]: I1203 11:54:48.379267 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2drl2\" (UniqueName: \"kubernetes.io/projected/b4961b81-97b8-4174-82bf-2d1d02cf8335-kube-api-access-2drl2\") pod \"b4961b81-97b8-4174-82bf-2d1d02cf8335\" (UID: \"b4961b81-97b8-4174-82bf-2d1d02cf8335\") " Dec 03 11:54:48 crc kubenswrapper[5052]: I1203 11:54:48.379347 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9d8cd78f-2670-4933-81dd-01cdc0044454-combined-ca-bundle\") pod \"9d8cd78f-2670-4933-81dd-01cdc0044454\" (UID: \"9d8cd78f-2670-4933-81dd-01cdc0044454\") " Dec 03 11:54:48 crc kubenswrapper[5052]: I1203 11:54:48.379439 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/9d8cd78f-2670-4933-81dd-01cdc0044454-metrics-certs-tls-certs\") pod \"9d8cd78f-2670-4933-81dd-01cdc0044454\" (UID: \"9d8cd78f-2670-4933-81dd-01cdc0044454\") " Dec 03 11:54:48 crc kubenswrapper[5052]: I1203 11:54:48.379539 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sw2xt\" (UniqueName: \"kubernetes.io/projected/9d8cd78f-2670-4933-81dd-01cdc0044454-kube-api-access-sw2xt\") pod \"9d8cd78f-2670-4933-81dd-01cdc0044454\" (UID: \"9d8cd78f-2670-4933-81dd-01cdc0044454\") " Dec 03 11:54:48 crc kubenswrapper[5052]: I1203 11:54:48.382147 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" podUID="da3a810d-9fd0-4f18-8f4a-e896b94b1431" containerName="nova-cell1-novncproxy-novncproxy" probeResult="failure" output="Get \"https://10.217.1.123:6080/vnc_lite.html\": dial tcp 10.217.1.123:6080: connect: connection refused" Dec 03 11:54:48 crc kubenswrapper[5052]: I1203 11:54:48.383063 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b4961b81-97b8-4174-82bf-2d1d02cf8335-scripts" (OuterVolumeSpecName: "scripts") pod "b4961b81-97b8-4174-82bf-2d1d02cf8335" (UID: "b4961b81-97b8-4174-82bf-2d1d02cf8335"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:54:48 crc kubenswrapper[5052]: I1203 11:54:48.389586 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b4961b81-97b8-4174-82bf-2d1d02cf8335-config" (OuterVolumeSpecName: "config") pod "b4961b81-97b8-4174-82bf-2d1d02cf8335" (UID: "b4961b81-97b8-4174-82bf-2d1d02cf8335"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:54:48 crc kubenswrapper[5052]: I1203 11:54:48.390033 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9d8cd78f-2670-4933-81dd-01cdc0044454-scripts" (OuterVolumeSpecName: "scripts") pod "9d8cd78f-2670-4933-81dd-01cdc0044454" (UID: "9d8cd78f-2670-4933-81dd-01cdc0044454"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:54:48 crc kubenswrapper[5052]: I1203 11:54:48.397231 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder6fd6-account-delete-5z97w"] Dec 03 11:54:48 crc kubenswrapper[5052]: I1203 11:54:48.397666 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b4961b81-97b8-4174-82bf-2d1d02cf8335-ovsdb-rundir" (OuterVolumeSpecName: "ovsdb-rundir") pod "b4961b81-97b8-4174-82bf-2d1d02cf8335" (UID: "b4961b81-97b8-4174-82bf-2d1d02cf8335"). InnerVolumeSpecName "ovsdb-rundir". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:54:48 crc kubenswrapper[5052]: I1203 11:54:48.397753 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage19-crc" (OuterVolumeSpecName: "ovndbcluster-nb-etc-ovn") pod "b4961b81-97b8-4174-82bf-2d1d02cf8335" (UID: "b4961b81-97b8-4174-82bf-2d1d02cf8335"). InnerVolumeSpecName "local-storage19-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 11:54:48 crc kubenswrapper[5052]: I1203 11:54:48.404017 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d8cd78f-2670-4933-81dd-01cdc0044454-ovn-rundir" (OuterVolumeSpecName: "ovn-rundir") pod "9d8cd78f-2670-4933-81dd-01cdc0044454" (UID: "9d8cd78f-2670-4933-81dd-01cdc0044454"). InnerVolumeSpecName "ovn-rundir". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:54:48 crc kubenswrapper[5052]: I1203 11:54:48.405134 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9d8cd78f-2670-4933-81dd-01cdc0044454-config" (OuterVolumeSpecName: "config") pod "9d8cd78f-2670-4933-81dd-01cdc0044454" (UID: "9d8cd78f-2670-4933-81dd-01cdc0044454"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:54:48 crc kubenswrapper[5052]: I1203 11:54:48.427832 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican4385-account-delete-krpzg"] Dec 03 11:54:48 crc kubenswrapper[5052]: I1203 11:54:48.451647 5052 reconciler_common.go:293] "Volume detached for volume \"ovn-rundir\" (UniqueName: \"kubernetes.io/empty-dir/9d8cd78f-2670-4933-81dd-01cdc0044454-ovn-rundir\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:48 crc kubenswrapper[5052]: I1203 11:54:48.451715 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/b4961b81-97b8-4174-82bf-2d1d02cf8335-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:48 crc kubenswrapper[5052]: I1203 11:54:48.451761 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/b4961b81-97b8-4174-82bf-2d1d02cf8335-config\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:48 crc kubenswrapper[5052]: I1203 11:54:48.451778 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/9d8cd78f-2670-4933-81dd-01cdc0044454-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:48 crc kubenswrapper[5052]: I1203 11:54:48.451791 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/9d8cd78f-2670-4933-81dd-01cdc0044454-config\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:48 crc kubenswrapper[5052]: I1203 11:54:48.451802 5052 reconciler_common.go:293] "Volume detached for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/b4961b81-97b8-4174-82bf-2d1d02cf8335-ovsdb-rundir\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:48 crc kubenswrapper[5052]: I1203 11:54:48.451863 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage19-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage19-crc\") on node \"crc\" " Dec 03 11:54:48 crc kubenswrapper[5052]: E1203 11:54:48.451796 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-kube-state-metrics-svc: secret "cert-kube-state-metrics-svc" not found Dec 03 11:54:48 crc kubenswrapper[5052]: E1203 11:54:48.451954 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/combined-ca-bundle: secret "combined-ca-bundle" not found Dec 03 11:54:48 crc kubenswrapper[5052]: E1203 11:54:48.452008 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/88e7db65-8253-474b-b115-cd3f6edaf85c-kube-state-metrics-tls-certs podName:88e7db65-8253-474b-b115-cd3f6edaf85c nodeName:}" failed. No retries permitted until 2025-12-03 11:54:50.451988632 +0000 UTC m=+2081.040230396 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls-certs" (UniqueName: "kubernetes.io/secret/88e7db65-8253-474b-b115-cd3f6edaf85c-kube-state-metrics-tls-certs") pod "kube-state-metrics-0" (UID: "88e7db65-8253-474b-b115-cd3f6edaf85c") : secret "cert-kube-state-metrics-svc" not found Dec 03 11:54:48 crc kubenswrapper[5052]: E1203 11:54:48.452309 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/88e7db65-8253-474b-b115-cd3f6edaf85c-combined-ca-bundle podName:88e7db65-8253-474b-b115-cd3f6edaf85c nodeName:}" failed. No retries permitted until 2025-12-03 11:54:50.45226679 +0000 UTC m=+2081.040508554 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "combined-ca-bundle" (UniqueName: "kubernetes.io/secret/88e7db65-8253-474b-b115-cd3f6edaf85c-combined-ca-bundle") pod "kube-state-metrics-0" (UID: "88e7db65-8253-474b-b115-cd3f6edaf85c") : secret "combined-ca-bundle" not found Dec 03 11:54:48 crc kubenswrapper[5052]: I1203 11:54:48.476829 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9d8cd78f-2670-4933-81dd-01cdc0044454-kube-api-access-sw2xt" (OuterVolumeSpecName: "kube-api-access-sw2xt") pod "9d8cd78f-2670-4933-81dd-01cdc0044454" (UID: "9d8cd78f-2670-4933-81dd-01cdc0044454"). InnerVolumeSpecName "kube-api-access-sw2xt". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:54:48 crc kubenswrapper[5052]: I1203 11:54:48.498999 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b4961b81-97b8-4174-82bf-2d1d02cf8335-kube-api-access-2drl2" (OuterVolumeSpecName: "kube-api-access-2drl2") pod "b4961b81-97b8-4174-82bf-2d1d02cf8335" (UID: "b4961b81-97b8-4174-82bf-2d1d02cf8335"). InnerVolumeSpecName "kube-api-access-2drl2". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:54:48 crc kubenswrapper[5052]: I1203 11:54:48.516138 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-0"] Dec 03 11:54:48 crc kubenswrapper[5052]: I1203 11:54:48.516413 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-cell1-conductor-0" podUID="4a181d2c-7595-4394-98de-957451309c98" containerName="nova-cell1-conductor-conductor" containerID="cri-o://c3cb1469712553e8283f9907b8e296f175dfb01cae7b28331f721bbf4e80e204" gracePeriod=30 Dec 03 11:54:48 crc kubenswrapper[5052]: I1203 11:54:48.558829 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-2drl2\" (UniqueName: \"kubernetes.io/projected/b4961b81-97b8-4174-82bf-2d1d02cf8335-kube-api-access-2drl2\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:48 crc kubenswrapper[5052]: I1203 11:54:48.559352 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-sw2xt\" (UniqueName: \"kubernetes.io/projected/9d8cd78f-2670-4933-81dd-01cdc0044454-kube-api-access-sw2xt\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:48 crc kubenswrapper[5052]: I1203 11:54:48.642064 5052 generic.go:334] "Generic (PLEG): container finished" podID="f32095c7-4e6f-43c7-a16f-513e89f7c28e" containerID="a9bd9892b49a6bfba8e02d24efa8266231be0e4d99061688b1aeffccdabed81f" exitCode=143 Dec 03 11:54:48 crc kubenswrapper[5052]: I1203 11:54:48.642161 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"f32095c7-4e6f-43c7-a16f-513e89f7c28e","Type":"ContainerDied","Data":"a9bd9892b49a6bfba8e02d24efa8266231be0e4d99061688b1aeffccdabed81f"} Dec 03 11:54:48 crc kubenswrapper[5052]: I1203 11:54:48.652257 5052 generic.go:334] "Generic (PLEG): container finished" podID="428e9203-aa9c-48d3-84c9-4dfaa806140d" containerID="932117c4bd5dd37889c00f1528583dfd60eee1eb15198c88c121a5439e07b994" exitCode=143 Dec 03 11:54:48 crc kubenswrapper[5052]: I1203 11:54:48.652344 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-5fc4755558-qz76x" event={"ID":"428e9203-aa9c-48d3-84c9-4dfaa806140d","Type":"ContainerDied","Data":"932117c4bd5dd37889c00f1528583dfd60eee1eb15198c88c121a5439e07b994"} Dec 03 11:54:48 crc kubenswrapper[5052]: I1203 11:54:48.656790 5052 prober.go:107] "Probe failed" probeType="Liveness" pod="openstack-kuttl-tests/kube-state-metrics-0" podUID="88e7db65-8253-474b-b115-cd3f6edaf85c" containerName="kube-state-metrics" probeResult="failure" output="HTTP probe failed with statuscode: 503" Dec 03 11:54:48 crc kubenswrapper[5052]: I1203 11:54:48.661732 5052 generic.go:334] "Generic (PLEG): container finished" podID="9a4b1245-d95c-4eb6-8d2b-fdd93d98bb1e" containerID="8c286da6c5ac86a7bafdea57cd407cfa1b12fb35b2f4dd6c013af4a83aa8fbb3" exitCode=143 Dec 03 11:54:48 crc kubenswrapper[5052]: I1203 11:54:48.661790 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"9a4b1245-d95c-4eb6-8d2b-fdd93d98bb1e","Type":"ContainerDied","Data":"8c286da6c5ac86a7bafdea57cd407cfa1b12fb35b2f4dd6c013af4a83aa8fbb3"} Dec 03 11:54:48 crc kubenswrapper[5052]: I1203 11:54:48.664015 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-db-sync-xr6nj"] Dec 03 11:54:48 crc kubenswrapper[5052]: I1203 11:54:48.668226 5052 generic.go:334] "Generic (PLEG): container finished" podID="5679a95d-a498-448e-97ed-497496520506" containerID="b657a1c95ebe202d32286143c3e1cb3b321d2c67300afcf9cb53509b8a6aee42" exitCode=143 Dec 03 11:54:48 crc kubenswrapper[5052]: I1203 11:54:48.668279 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-keystone-listener-7bf89d7c74-c4srn" event={"ID":"5679a95d-a498-448e-97ed-497496520506","Type":"ContainerDied","Data":"b657a1c95ebe202d32286143c3e1cb3b321d2c67300afcf9cb53509b8a6aee42"} Dec 03 11:54:48 crc kubenswrapper[5052]: I1203 11:54:48.670058 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-db-sync-xr6nj"] Dec 03 11:54:48 crc kubenswrapper[5052]: I1203 11:54:48.672140 5052 generic.go:334] "Generic (PLEG): container finished" podID="4c81b4dc-4f69-4447-8e5e-043aa0c7dd59" containerID="d40e529ae6719c4ad0db1e286623449a547ab4530cef62b4eede6892bcc97992" exitCode=137 Dec 03 11:54:48 crc kubenswrapper[5052]: I1203 11:54:48.679536 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-0"] Dec 03 11:54:48 crc kubenswrapper[5052]: I1203 11:54:48.679746 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-cell0-conductor-0" podUID="4ec0d450-cd52-432e-8bb2-1ef9698821c1" containerName="nova-cell0-conductor-conductor" containerID="cri-o://c75d78e4eb183b74af6b09e658fe926d3135454064d32c91f94389213e67abcc" gracePeriod=30 Dec 03 11:54:48 crc kubenswrapper[5052]: I1203 11:54:48.681955 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican4385-account-delete-krpzg" event={"ID":"383e7933-2bc4-4ead-bc34-0832091bd952","Type":"ContainerStarted","Data":"22526e6c1f19a453ccca65f572d624a9eb9f5313506688f1a63bfc7853365f6a"} Dec 03 11:54:48 crc kubenswrapper[5052]: I1203 11:54:48.688162 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-db-sync-tttqk"] Dec 03 11:54:48 crc kubenswrapper[5052]: I1203 11:54:48.690653 5052 generic.go:334] "Generic (PLEG): container finished" podID="653882ec-4522-4979-b1aa-7fad5023605c" containerID="cbf6b026a9b62a7464f44bbde7effeb8a0aa796789269fbcd6b9b3589fc439c9" exitCode=143 Dec 03 11:54:48 crc kubenswrapper[5052]: I1203 11:54:48.690722 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"653882ec-4522-4979-b1aa-7fad5023605c","Type":"ContainerDied","Data":"cbf6b026a9b62a7464f44bbde7effeb8a0aa796789269fbcd6b9b3589fc439c9"} Dec 03 11:54:48 crc kubenswrapper[5052]: I1203 11:54:48.691281 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage19-crc" (UniqueName: "kubernetes.io/local-volume/local-storage19-crc") on node "crc" Dec 03 11:54:48 crc kubenswrapper[5052]: E1203 11:54:48.694328 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of a3ef9454b0aa12b5e49b5ada9b84ef790f5b7a8483610db37b8081939ff19a0a is running failed: container process not found" containerID="a3ef9454b0aa12b5e49b5ada9b84ef790f5b7a8483610db37b8081939ff19a0a" cmd=["/usr/bin/pidof","ovsdb-server"] Dec 03 11:54:48 crc kubenswrapper[5052]: I1203 11:54:48.697958 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-db-sync-tttqk"] Dec 03 11:54:48 crc kubenswrapper[5052]: E1203 11:54:48.698214 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of a3ef9454b0aa12b5e49b5ada9b84ef790f5b7a8483610db37b8081939ff19a0a is running failed: container process not found" containerID="a3ef9454b0aa12b5e49b5ada9b84ef790f5b7a8483610db37b8081939ff19a0a" cmd=["/usr/bin/pidof","ovsdb-server"] Dec 03 11:54:48 crc kubenswrapper[5052]: E1203 11:54:48.698969 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of a3ef9454b0aa12b5e49b5ada9b84ef790f5b7a8483610db37b8081939ff19a0a is running failed: container process not found" containerID="a3ef9454b0aa12b5e49b5ada9b84ef790f5b7a8483610db37b8081939ff19a0a" cmd=["/usr/bin/pidof","ovsdb-server"] Dec 03 11:54:48 crc kubenswrapper[5052]: E1203 11:54:48.699027 5052 prober.go:104] "Probe errored" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of a3ef9454b0aa12b5e49b5ada9b84ef790f5b7a8483610db37b8081939ff19a0a is running failed: container process not found" probeType="Readiness" pod="openstack-kuttl-tests/ovsdbserver-sb-0" podUID="75dd245b-b54f-4a9a-9627-cbf338a69566" containerName="ovsdbserver-sb" Dec 03 11:54:48 crc kubenswrapper[5052]: I1203 11:54:48.699818 5052 generic.go:334] "Generic (PLEG): container finished" podID="83b9fcfd-8007-4d6b-9090-d2e536cb1bbc" containerID="d4b83b49b57a27fa3f6b26ff04edd361dfcc21c0d52243555076b619c81a9ec9" exitCode=0 Dec 03 11:54:48 crc kubenswrapper[5052]: I1203 11:54:48.699880 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-5ff446f48b-7hc25" event={"ID":"83b9fcfd-8007-4d6b-9090-d2e536cb1bbc","Type":"ContainerDied","Data":"d4b83b49b57a27fa3f6b26ff04edd361dfcc21c0d52243555076b619c81a9ec9"} Dec 03 11:54:48 crc kubenswrapper[5052]: I1203 11:54:48.704298 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/kube-state-metrics-0"] Dec 03 11:54:48 crc kubenswrapper[5052]: I1203 11:54:48.704576 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/kube-state-metrics-0" podUID="88e7db65-8253-474b-b115-cd3f6edaf85c" containerName="kube-state-metrics" containerID="cri-o://d035b8a698baa861b1fde789196a379e9a2b04d327860b01e3e264af295b680f" gracePeriod=30 Dec 03 11:54:48 crc kubenswrapper[5052]: I1203 11:54:48.705926 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance634c-account-delete-nzwqr" event={"ID":"a73e0afd-e037-41f7-b214-d4eebd4bafe0","Type":"ContainerStarted","Data":"ec815d3878e79f333506c388e7ff97c3234e71a84254f90b253c512d8fe9a121"} Dec 03 11:54:48 crc kubenswrapper[5052]: I1203 11:54:48.717124 5052 generic.go:334] "Generic (PLEG): container finished" podID="a18602ce-7206-4edc-a8d3-5d0754c3da02" containerID="b8bffbbe67b941eebfcc7d809311107fe0d16c3fd77f96571d99325342cabd72" exitCode=143 Dec 03 11:54:48 crc kubenswrapper[5052]: I1203 11:54:48.717176 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-5758478fc6-5g46v" event={"ID":"a18602ce-7206-4edc-a8d3-5d0754c3da02","Type":"ContainerDied","Data":"b8bffbbe67b941eebfcc7d809311107fe0d16c3fd77f96571d99325342cabd72"} Dec 03 11:54:48 crc kubenswrapper[5052]: I1203 11:54:48.718591 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:54:48 crc kubenswrapper[5052]: I1203 11:54:48.718852 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="a5ff0249-173a-4793-8cbe-0aa749f34632" containerName="ceilometer-central-agent" containerID="cri-o://7bad91ec5c27b9fd97fb06baef39407cc01e86e5c940cc4ffd9ae98b217e02e2" gracePeriod=30 Dec 03 11:54:48 crc kubenswrapper[5052]: I1203 11:54:48.719281 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="a5ff0249-173a-4793-8cbe-0aa749f34632" containerName="proxy-httpd" containerID="cri-o://615dcdb302a4fa77dc8006892c9f0c5509fe1f5ef890116dd6a627f53880b9b8" gracePeriod=30 Dec 03 11:54:48 crc kubenswrapper[5052]: I1203 11:54:48.719702 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="a5ff0249-173a-4793-8cbe-0aa749f34632" containerName="sg-core" containerID="cri-o://14c4e0b558f59a7e38279d4f104332c292bd1c29af9d8ae1768b1a9ec55a8d31" gracePeriod=30 Dec 03 11:54:48 crc kubenswrapper[5052]: I1203 11:54:48.719652 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="a5ff0249-173a-4793-8cbe-0aa749f34632" containerName="ceilometer-notification-agent" containerID="cri-o://cf164870d3f804d61f70d5e02392ceff8cd44fdf830fb056d8e5f574d1ef05c1" gracePeriod=30 Dec 03 11:54:48 crc kubenswrapper[5052]: I1203 11:54:48.729388 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_ovn-northd-0_9d8cd78f-2670-4933-81dd-01cdc0044454/ovn-northd/0.log" Dec 03 11:54:48 crc kubenswrapper[5052]: I1203 11:54:48.729467 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovn-northd-0" event={"ID":"9d8cd78f-2670-4933-81dd-01cdc0044454","Type":"ContainerDied","Data":"fe5a7b41847839b2299acdb962194c6f5f4679c210cf40fde158b31f5dd06532"} Dec 03 11:54:48 crc kubenswrapper[5052]: I1203 11:54:48.729521 5052 scope.go:117] "RemoveContainer" containerID="a00e41ad3b048582d984194b4e78df8f197fd9e5a5af35f2e726a5b8f13fb9fb" Dec 03 11:54:48 crc kubenswrapper[5052]: I1203 11:54:48.729664 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 11:54:48 crc kubenswrapper[5052]: I1203 11:54:48.760152 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b4961b81-97b8-4174-82bf-2d1d02cf8335-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "b4961b81-97b8-4174-82bf-2d1d02cf8335" (UID: "b4961b81-97b8-4174-82bf-2d1d02cf8335"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:54:48 crc kubenswrapper[5052]: I1203 11:54:48.764140 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b4961b81-97b8-4174-82bf-2d1d02cf8335-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:48 crc kubenswrapper[5052]: I1203 11:54:48.764176 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage19-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage19-crc\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:48 crc kubenswrapper[5052]: I1203 11:54:48.789677 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9d8cd78f-2670-4933-81dd-01cdc0044454-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "9d8cd78f-2670-4933-81dd-01cdc0044454" (UID: "9d8cd78f-2670-4933-81dd-01cdc0044454"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:54:48 crc kubenswrapper[5052]: I1203 11:54:48.803087 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_ovsdbserver-sb-0_75dd245b-b54f-4a9a-9627-cbf338a69566/ovsdbserver-sb/0.log" Dec 03 11:54:48 crc kubenswrapper[5052]: I1203 11:54:48.803246 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 11:54:48 crc kubenswrapper[5052]: I1203 11:54:48.847725 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9d8cd78f-2670-4933-81dd-01cdc0044454-metrics-certs-tls-certs" (OuterVolumeSpecName: "metrics-certs-tls-certs") pod "9d8cd78f-2670-4933-81dd-01cdc0044454" (UID: "9d8cd78f-2670-4933-81dd-01cdc0044454"). InnerVolumeSpecName "metrics-certs-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:54:48 crc kubenswrapper[5052]: I1203 11:54:48.848640 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b4961b81-97b8-4174-82bf-2d1d02cf8335-metrics-certs-tls-certs" (OuterVolumeSpecName: "metrics-certs-tls-certs") pod "b4961b81-97b8-4174-82bf-2d1d02cf8335" (UID: "b4961b81-97b8-4174-82bf-2d1d02cf8335"). InnerVolumeSpecName "metrics-certs-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:54:48 crc kubenswrapper[5052]: I1203 11:54:48.852377 5052 generic.go:334] "Generic (PLEG): container finished" podID="337bf4ce-ed6a-47bb-820d-6d64cd15e16e" containerID="54a021ff30a08f0f3bd288edbc672e78a69a0e8398c93034e0304c71564937f1" exitCode=0 Dec 03 11:54:48 crc kubenswrapper[5052]: I1203 11:54:48.852412 5052 generic.go:334] "Generic (PLEG): container finished" podID="337bf4ce-ed6a-47bb-820d-6d64cd15e16e" containerID="308f757a4c5baeeb088bbd69071d883ee0fd5f6717ce1fb801584cc7308cf01e" exitCode=0 Dec 03 11:54:48 crc kubenswrapper[5052]: I1203 11:54:48.852419 5052 generic.go:334] "Generic (PLEG): container finished" podID="337bf4ce-ed6a-47bb-820d-6d64cd15e16e" containerID="0bdc5789ab44d02d9702c5cf3e475db9c7a7ce6bf079ab5b06570c12673c510c" exitCode=0 Dec 03 11:54:48 crc kubenswrapper[5052]: I1203 11:54:48.852427 5052 generic.go:334] "Generic (PLEG): container finished" podID="337bf4ce-ed6a-47bb-820d-6d64cd15e16e" containerID="ef2775155138fd30b620bc0fbab9074d6f7cd4364b1cbc82c977fd6b48054772" exitCode=0 Dec 03 11:54:48 crc kubenswrapper[5052]: I1203 11:54:48.852434 5052 generic.go:334] "Generic (PLEG): container finished" podID="337bf4ce-ed6a-47bb-820d-6d64cd15e16e" containerID="bf4d57ce3d071492f7492f8995fdfdb4a50c770c3ee835e5864c7889d9aca849" exitCode=0 Dec 03 11:54:48 crc kubenswrapper[5052]: I1203 11:54:48.852442 5052 generic.go:334] "Generic (PLEG): container finished" podID="337bf4ce-ed6a-47bb-820d-6d64cd15e16e" containerID="6b9be92cb0d9072365edb2b4115b091dd2b7a90d8f70424fe8b36078d19bcab6" exitCode=0 Dec 03 11:54:48 crc kubenswrapper[5052]: I1203 11:54:48.852448 5052 generic.go:334] "Generic (PLEG): container finished" podID="337bf4ce-ed6a-47bb-820d-6d64cd15e16e" containerID="cac91cb8a133117b6ac9ef92818b0c43f08504bb99f13b28c87c24ca86efd6f6" exitCode=0 Dec 03 11:54:48 crc kubenswrapper[5052]: I1203 11:54:48.852455 5052 generic.go:334] "Generic (PLEG): container finished" podID="337bf4ce-ed6a-47bb-820d-6d64cd15e16e" containerID="da24bea2b91d001778224eaa9d20a940c1cfa8aec9ca78a78f4063dc10e39b5e" exitCode=0 Dec 03 11:54:48 crc kubenswrapper[5052]: I1203 11:54:48.852483 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"337bf4ce-ed6a-47bb-820d-6d64cd15e16e","Type":"ContainerDied","Data":"54a021ff30a08f0f3bd288edbc672e78a69a0e8398c93034e0304c71564937f1"} Dec 03 11:54:48 crc kubenswrapper[5052]: I1203 11:54:48.852590 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"337bf4ce-ed6a-47bb-820d-6d64cd15e16e","Type":"ContainerDied","Data":"308f757a4c5baeeb088bbd69071d883ee0fd5f6717ce1fb801584cc7308cf01e"} Dec 03 11:54:48 crc kubenswrapper[5052]: I1203 11:54:48.852603 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"337bf4ce-ed6a-47bb-820d-6d64cd15e16e","Type":"ContainerDied","Data":"0bdc5789ab44d02d9702c5cf3e475db9c7a7ce6bf079ab5b06570c12673c510c"} Dec 03 11:54:48 crc kubenswrapper[5052]: I1203 11:54:48.852617 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"337bf4ce-ed6a-47bb-820d-6d64cd15e16e","Type":"ContainerDied","Data":"ef2775155138fd30b620bc0fbab9074d6f7cd4364b1cbc82c977fd6b48054772"} Dec 03 11:54:48 crc kubenswrapper[5052]: I1203 11:54:48.852627 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"337bf4ce-ed6a-47bb-820d-6d64cd15e16e","Type":"ContainerDied","Data":"bf4d57ce3d071492f7492f8995fdfdb4a50c770c3ee835e5864c7889d9aca849"} Dec 03 11:54:48 crc kubenswrapper[5052]: I1203 11:54:48.852637 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"337bf4ce-ed6a-47bb-820d-6d64cd15e16e","Type":"ContainerDied","Data":"6b9be92cb0d9072365edb2b4115b091dd2b7a90d8f70424fe8b36078d19bcab6"} Dec 03 11:54:48 crc kubenswrapper[5052]: I1203 11:54:48.852645 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"337bf4ce-ed6a-47bb-820d-6d64cd15e16e","Type":"ContainerDied","Data":"cac91cb8a133117b6ac9ef92818b0c43f08504bb99f13b28c87c24ca86efd6f6"} Dec 03 11:54:48 crc kubenswrapper[5052]: I1203 11:54:48.852654 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"337bf4ce-ed6a-47bb-820d-6d64cd15e16e","Type":"ContainerDied","Data":"da24bea2b91d001778224eaa9d20a940c1cfa8aec9ca78a78f4063dc10e39b5e"} Dec 03 11:54:48 crc kubenswrapper[5052]: I1203 11:54:48.858472 5052 generic.go:334] "Generic (PLEG): container finished" podID="d3222c69-47bc-46d8-a025-cc457881cac9" containerID="d7f3ac6af5a2c0d33f118ffd9646c754d018fd85708d55c58a60a5cc00574cd1" exitCode=143 Dec 03 11:54:48 crc kubenswrapper[5052]: I1203 11:54:48.858546 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"d3222c69-47bc-46d8-a025-cc457881cac9","Type":"ContainerDied","Data":"d7f3ac6af5a2c0d33f118ffd9646c754d018fd85708d55c58a60a5cc00574cd1"} Dec 03 11:54:48 crc kubenswrapper[5052]: I1203 11:54:48.858911 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9d8cd78f-2670-4933-81dd-01cdc0044454-ovn-northd-tls-certs" (OuterVolumeSpecName: "ovn-northd-tls-certs") pod "9d8cd78f-2670-4933-81dd-01cdc0044454" (UID: "9d8cd78f-2670-4933-81dd-01cdc0044454"). InnerVolumeSpecName "ovn-northd-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:54:48 crc kubenswrapper[5052]: I1203 11:54:48.864990 5052 generic.go:334] "Generic (PLEG): container finished" podID="0201d941-cd05-4d1c-9af6-f3a82a375a0f" containerID="f0df1e8717d84a41bd64b8ee9a76553b85d16b020416be110d56d085ed4f7b45" exitCode=0 Dec 03 11:54:48 crc kubenswrapper[5052]: I1203 11:54:48.865081 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"0201d941-cd05-4d1c-9af6-f3a82a375a0f","Type":"ContainerDied","Data":"f0df1e8717d84a41bd64b8ee9a76553b85d16b020416be110d56d085ed4f7b45"} Dec 03 11:54:48 crc kubenswrapper[5052]: I1203 11:54:48.866116 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9d8cd78f-2670-4933-81dd-01cdc0044454-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:48 crc kubenswrapper[5052]: I1203 11:54:48.866146 5052 reconciler_common.go:293] "Volume detached for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/9d8cd78f-2670-4933-81dd-01cdc0044454-metrics-certs-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:48 crc kubenswrapper[5052]: I1203 11:54:48.866156 5052 reconciler_common.go:293] "Volume detached for volume \"ovn-northd-tls-certs\" (UniqueName: \"kubernetes.io/secret/9d8cd78f-2670-4933-81dd-01cdc0044454-ovn-northd-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:48 crc kubenswrapper[5052]: I1203 11:54:48.866164 5052 reconciler_common.go:293] "Volume detached for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/b4961b81-97b8-4174-82bf-2d1d02cf8335-metrics-certs-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:48 crc kubenswrapper[5052]: I1203 11:54:48.868903 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron3cf1-account-delete-zk6qw"] Dec 03 11:54:48 crc kubenswrapper[5052]: I1203 11:54:48.871809 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_ovsdbserver-nb-0_b4961b81-97b8-4174-82bf-2d1d02cf8335/ovsdbserver-nb/0.log" Dec 03 11:54:48 crc kubenswrapper[5052]: I1203 11:54:48.874227 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 11:54:48 crc kubenswrapper[5052]: I1203 11:54:48.874806 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b4961b81-97b8-4174-82bf-2d1d02cf8335-ovsdbserver-nb-tls-certs" (OuterVolumeSpecName: "ovsdbserver-nb-tls-certs") pod "b4961b81-97b8-4174-82bf-2d1d02cf8335" (UID: "b4961b81-97b8-4174-82bf-2d1d02cf8335"). InnerVolumeSpecName "ovsdbserver-nb-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:54:48 crc kubenswrapper[5052]: I1203 11:54:48.874859 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-nb-0" event={"ID":"b4961b81-97b8-4174-82bf-2d1d02cf8335","Type":"ContainerDied","Data":"e519c2d639554564bd415c09d122b9a2625467e70003e8cbac73c52a20dbb731"} Dec 03 11:54:48 crc kubenswrapper[5052]: I1203 11:54:48.883685 5052 scope.go:117] "RemoveContainer" containerID="2cf5c7f704cdea9711753e85146be0ca1fd13eb56b9658c079f3f91d897ce7a3" Dec 03 11:54:48 crc kubenswrapper[5052]: I1203 11:54:48.885698 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_ovsdbserver-sb-0_75dd245b-b54f-4a9a-9627-cbf338a69566/ovsdbserver-sb/0.log" Dec 03 11:54:48 crc kubenswrapper[5052]: I1203 11:54:48.885741 5052 generic.go:334] "Generic (PLEG): container finished" podID="75dd245b-b54f-4a9a-9627-cbf338a69566" containerID="a3ef9454b0aa12b5e49b5ada9b84ef790f5b7a8483610db37b8081939ff19a0a" exitCode=143 Dec 03 11:54:48 crc kubenswrapper[5052]: I1203 11:54:48.885790 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-sb-0" event={"ID":"75dd245b-b54f-4a9a-9627-cbf338a69566","Type":"ContainerDied","Data":"a3ef9454b0aa12b5e49b5ada9b84ef790f5b7a8483610db37b8081939ff19a0a"} Dec 03 11:54:48 crc kubenswrapper[5052]: I1203 11:54:48.885862 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 11:54:48 crc kubenswrapper[5052]: I1203 11:54:48.900667 5052 generic.go:334] "Generic (PLEG): container finished" podID="da3a810d-9fd0-4f18-8f4a-e896b94b1431" containerID="80ffd33af4023d6b6f0f24ec3bd666459e45b4216c33dab791e932793c7535bf" exitCode=0 Dec 03 11:54:48 crc kubenswrapper[5052]: I1203 11:54:48.900736 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" event={"ID":"da3a810d-9fd0-4f18-8f4a-e896b94b1431","Type":"ContainerDied","Data":"80ffd33af4023d6b6f0f24ec3bd666459e45b4216c33dab791e932793c7535bf"} Dec 03 11:54:48 crc kubenswrapper[5052]: W1203 11:54:48.909852 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0f42494c_058e_417d_9e0d_97ce7040b288.slice/crio-e0edc022d786ec7711974e5ad8fa9b59a5df42dc00c4cf83fd27b41325567c72 WatchSource:0}: Error finding container e0edc022d786ec7711974e5ad8fa9b59a5df42dc00c4cf83fd27b41325567c72: Status 404 returned error can't find the container with id e0edc022d786ec7711974e5ad8fa9b59a5df42dc00c4cf83fd27b41325567c72 Dec 03 11:54:48 crc kubenswrapper[5052]: I1203 11:54:48.909979 5052 generic.go:334] "Generic (PLEG): container finished" podID="584deaba-f945-42a2-8c0d-e531d6f48fd8" containerID="2ee3c0361bbf90cf40dcd13279e44150c5f8ed97f268cf91a9701bccff60b20b" exitCode=143 Dec 03 11:54:48 crc kubenswrapper[5052]: I1203 11:54:48.910045 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-worker-6d7f657c65-cqxbg" event={"ID":"584deaba-f945-42a2-8c0d-e531d6f48fd8","Type":"ContainerDied","Data":"2ee3c0361bbf90cf40dcd13279e44150c5f8ed97f268cf91a9701bccff60b20b"} Dec 03 11:54:48 crc kubenswrapper[5052]: I1203 11:54:48.918858 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder6fd6-account-delete-5z97w" event={"ID":"53d7a9a2-b6a9-42dc-9828-d8ceaa70f0a3","Type":"ContainerStarted","Data":"98f873e1e36190437d014aac57c5d4a18331863c2203a7f032d5286ac1d56c80"} Dec 03 11:54:48 crc kubenswrapper[5052]: I1203 11:54:48.925605 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstackclient" Dec 03 11:54:48 crc kubenswrapper[5052]: I1203 11:54:48.935048 5052 scope.go:117] "RemoveContainer" containerID="e54d2995d89269947a87a50c5c5e626a778c66e141e2d728f3fafc46c76e5a9b" Dec 03 11:54:48 crc kubenswrapper[5052]: I1203 11:54:48.975695 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/75dd245b-b54f-4a9a-9627-cbf338a69566-ovsdb-rundir\") pod \"75dd245b-b54f-4a9a-9627-cbf338a69566\" (UID: \"75dd245b-b54f-4a9a-9627-cbf338a69566\") " Dec 03 11:54:48 crc kubenswrapper[5052]: I1203 11:54:48.975793 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/75dd245b-b54f-4a9a-9627-cbf338a69566-metrics-certs-tls-certs\") pod \"75dd245b-b54f-4a9a-9627-cbf338a69566\" (UID: \"75dd245b-b54f-4a9a-9627-cbf338a69566\") " Dec 03 11:54:48 crc kubenswrapper[5052]: I1203 11:54:48.975874 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovndbcluster-sb-etc-ovn\" (UniqueName: \"kubernetes.io/local-volume/local-storage01-crc\") pod \"75dd245b-b54f-4a9a-9627-cbf338a69566\" (UID: \"75dd245b-b54f-4a9a-9627-cbf338a69566\") " Dec 03 11:54:48 crc kubenswrapper[5052]: I1203 11:54:48.975904 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/75dd245b-b54f-4a9a-9627-cbf338a69566-combined-ca-bundle\") pod \"75dd245b-b54f-4a9a-9627-cbf338a69566\" (UID: \"75dd245b-b54f-4a9a-9627-cbf338a69566\") " Dec 03 11:54:48 crc kubenswrapper[5052]: I1203 11:54:48.975976 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/75dd245b-b54f-4a9a-9627-cbf338a69566-config\") pod \"75dd245b-b54f-4a9a-9627-cbf338a69566\" (UID: \"75dd245b-b54f-4a9a-9627-cbf338a69566\") " Dec 03 11:54:48 crc kubenswrapper[5052]: I1203 11:54:48.976010 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/75dd245b-b54f-4a9a-9627-cbf338a69566-scripts\") pod \"75dd245b-b54f-4a9a-9627-cbf338a69566\" (UID: \"75dd245b-b54f-4a9a-9627-cbf338a69566\") " Dec 03 11:54:48 crc kubenswrapper[5052]: I1203 11:54:48.976065 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cqprg\" (UniqueName: \"kubernetes.io/projected/75dd245b-b54f-4a9a-9627-cbf338a69566-kube-api-access-cqprg\") pod \"75dd245b-b54f-4a9a-9627-cbf338a69566\" (UID: \"75dd245b-b54f-4a9a-9627-cbf338a69566\") " Dec 03 11:54:48 crc kubenswrapper[5052]: I1203 11:54:48.976109 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovsdbserver-sb-tls-certs\" (UniqueName: \"kubernetes.io/secret/75dd245b-b54f-4a9a-9627-cbf338a69566-ovsdbserver-sb-tls-certs\") pod \"75dd245b-b54f-4a9a-9627-cbf338a69566\" (UID: \"75dd245b-b54f-4a9a-9627-cbf338a69566\") " Dec 03 11:54:48 crc kubenswrapper[5052]: I1203 11:54:48.976478 5052 reconciler_common.go:293] "Volume detached for volume \"ovsdbserver-nb-tls-certs\" (UniqueName: \"kubernetes.io/secret/b4961b81-97b8-4174-82bf-2d1d02cf8335-ovsdbserver-nb-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:48 crc kubenswrapper[5052]: I1203 11:54:48.981943 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/75dd245b-b54f-4a9a-9627-cbf338a69566-config" (OuterVolumeSpecName: "config") pod "75dd245b-b54f-4a9a-9627-cbf338a69566" (UID: "75dd245b-b54f-4a9a-9627-cbf338a69566"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:54:48 crc kubenswrapper[5052]: I1203 11:54:48.982121 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/75dd245b-b54f-4a9a-9627-cbf338a69566-ovsdb-rundir" (OuterVolumeSpecName: "ovsdb-rundir") pod "75dd245b-b54f-4a9a-9627-cbf338a69566" (UID: "75dd245b-b54f-4a9a-9627-cbf338a69566"). InnerVolumeSpecName "ovsdb-rundir". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:54:48 crc kubenswrapper[5052]: I1203 11:54:48.982388 5052 scope.go:117] "RemoveContainer" containerID="434f07c085aa632ec76c3e82580e18718305175f93ff194a656a5b66cfbf2180" Dec 03 11:54:48 crc kubenswrapper[5052]: I1203 11:54:48.983083 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/75dd245b-b54f-4a9a-9627-cbf338a69566-scripts" (OuterVolumeSpecName: "scripts") pod "75dd245b-b54f-4a9a-9627-cbf338a69566" (UID: "75dd245b-b54f-4a9a-9627-cbf338a69566"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:54:48 crc kubenswrapper[5052]: I1203 11:54:48.988605 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage01-crc" (OuterVolumeSpecName: "ovndbcluster-sb-etc-ovn") pod "75dd245b-b54f-4a9a-9627-cbf338a69566" (UID: "75dd245b-b54f-4a9a-9627-cbf338a69566"). InnerVolumeSpecName "local-storage01-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 11:54:49 crc kubenswrapper[5052]: I1203 11:54:49.051909 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/75dd245b-b54f-4a9a-9627-cbf338a69566-kube-api-access-cqprg" (OuterVolumeSpecName: "kube-api-access-cqprg") pod "75dd245b-b54f-4a9a-9627-cbf338a69566" (UID: "75dd245b-b54f-4a9a-9627-cbf338a69566"). InnerVolumeSpecName "kube-api-access-cqprg". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:54:49 crc kubenswrapper[5052]: I1203 11:54:49.078207 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"openstack-config\" (UniqueName: \"kubernetes.io/configmap/4c81b4dc-4f69-4447-8e5e-043aa0c7dd59-openstack-config\") pod \"4c81b4dc-4f69-4447-8e5e-043aa0c7dd59\" (UID: \"4c81b4dc-4f69-4447-8e5e-043aa0c7dd59\") " Dec 03 11:54:49 crc kubenswrapper[5052]: I1203 11:54:49.078247 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"openstack-config-secret\" (UniqueName: \"kubernetes.io/secret/4c81b4dc-4f69-4447-8e5e-043aa0c7dd59-openstack-config-secret\") pod \"4c81b4dc-4f69-4447-8e5e-043aa0c7dd59\" (UID: \"4c81b4dc-4f69-4447-8e5e-043aa0c7dd59\") " Dec 03 11:54:49 crc kubenswrapper[5052]: I1203 11:54:49.078295 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4c81b4dc-4f69-4447-8e5e-043aa0c7dd59-combined-ca-bundle\") pod \"4c81b4dc-4f69-4447-8e5e-043aa0c7dd59\" (UID: \"4c81b4dc-4f69-4447-8e5e-043aa0c7dd59\") " Dec 03 11:54:49 crc kubenswrapper[5052]: I1203 11:54:49.078343 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-68s8l\" (UniqueName: \"kubernetes.io/projected/4c81b4dc-4f69-4447-8e5e-043aa0c7dd59-kube-api-access-68s8l\") pod \"4c81b4dc-4f69-4447-8e5e-043aa0c7dd59\" (UID: \"4c81b4dc-4f69-4447-8e5e-043aa0c7dd59\") " Dec 03 11:54:49 crc kubenswrapper[5052]: I1203 11:54:49.078847 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/75dd245b-b54f-4a9a-9627-cbf338a69566-config\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:49 crc kubenswrapper[5052]: I1203 11:54:49.078864 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/75dd245b-b54f-4a9a-9627-cbf338a69566-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:49 crc kubenswrapper[5052]: I1203 11:54:49.078874 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-cqprg\" (UniqueName: \"kubernetes.io/projected/75dd245b-b54f-4a9a-9627-cbf338a69566-kube-api-access-cqprg\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:49 crc kubenswrapper[5052]: I1203 11:54:49.078883 5052 reconciler_common.go:293] "Volume detached for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/75dd245b-b54f-4a9a-9627-cbf338a69566-ovsdb-rundir\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:49 crc kubenswrapper[5052]: I1203 11:54:49.078904 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage01-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage01-crc\") on node \"crc\" " Dec 03 11:54:49 crc kubenswrapper[5052]: I1203 11:54:49.151571 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/swift-proxy-b9d8f996b-brs7h"] Dec 03 11:54:49 crc kubenswrapper[5052]: I1203 11:54:49.152130 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-proxy-b9d8f996b-brs7h" podUID="036bc2b0-bc50-4c91-bea5-882b9e592e4f" containerName="proxy-httpd" containerID="cri-o://b83ee0552872f35b33c41234c7f87cdaa58debb1beeb08c6e6d9b6f3a591cb00" gracePeriod=30 Dec 03 11:54:49 crc kubenswrapper[5052]: I1203 11:54:49.152244 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-proxy-b9d8f996b-brs7h" podUID="036bc2b0-bc50-4c91-bea5-882b9e592e4f" containerName="proxy-server" containerID="cri-o://47975dc763fa99fd8e031feff26f6ec865ddd0443ad75de0e4f2b3626568c51b" gracePeriod=30 Dec 03 11:54:49 crc kubenswrapper[5052]: I1203 11:54:49.169308 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4c81b4dc-4f69-4447-8e5e-043aa0c7dd59-kube-api-access-68s8l" (OuterVolumeSpecName: "kube-api-access-68s8l") pod "4c81b4dc-4f69-4447-8e5e-043aa0c7dd59" (UID: "4c81b4dc-4f69-4447-8e5e-043aa0c7dd59"). InnerVolumeSpecName "kube-api-access-68s8l". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:54:49 crc kubenswrapper[5052]: I1203 11:54:49.181027 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-68s8l\" (UniqueName: \"kubernetes.io/projected/4c81b4dc-4f69-4447-8e5e-043aa0c7dd59-kube-api-access-68s8l\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:49 crc kubenswrapper[5052]: E1203 11:54:49.181099 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/rabbitmq-config-data: configmap "rabbitmq-config-data" not found Dec 03 11:54:49 crc kubenswrapper[5052]: E1203 11:54:49.181142 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/10e3094e-f625-4d69-b0e9-358220c0d195-config-data podName:10e3094e-f625-4d69-b0e9-358220c0d195 nodeName:}" failed. No retries permitted until 2025-12-03 11:54:53.181129498 +0000 UTC m=+2083.769371262 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "config-data" (UniqueName: "kubernetes.io/configmap/10e3094e-f625-4d69-b0e9-358220c0d195-config-data") pod "rabbitmq-server-0" (UID: "10e3094e-f625-4d69-b0e9-358220c0d195") : configmap "rabbitmq-config-data" not found Dec 03 11:54:49 crc kubenswrapper[5052]: I1203 11:54:49.314902 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/75dd245b-b54f-4a9a-9627-cbf338a69566-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "75dd245b-b54f-4a9a-9627-cbf338a69566" (UID: "75dd245b-b54f-4a9a-9627-cbf338a69566"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:54:49 crc kubenswrapper[5052]: I1203 11:54:49.386496 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/75dd245b-b54f-4a9a-9627-cbf338a69566-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:49 crc kubenswrapper[5052]: I1203 11:54:49.459690 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4c81b4dc-4f69-4447-8e5e-043aa0c7dd59-openstack-config" (OuterVolumeSpecName: "openstack-config") pod "4c81b4dc-4f69-4447-8e5e-043aa0c7dd59" (UID: "4c81b4dc-4f69-4447-8e5e-043aa0c7dd59"). InnerVolumeSpecName "openstack-config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:54:49 crc kubenswrapper[5052]: I1203 11:54:49.460838 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/novaapie4da-account-delete-mknr9"] Dec 03 11:54:49 crc kubenswrapper[5052]: I1203 11:54:49.467613 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-z8rn9"] Dec 03 11:54:49 crc kubenswrapper[5052]: I1203 11:54:49.472874 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/placement2868-account-delete-gtkfr"] Dec 03 11:54:49 crc kubenswrapper[5052]: I1203 11:54:49.488803 5052 reconciler_common.go:293] "Volume detached for volume \"openstack-config\" (UniqueName: \"kubernetes.io/configmap/4c81b4dc-4f69-4447-8e5e-043aa0c7dd59-openstack-config\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:49 crc kubenswrapper[5052]: I1203 11:54:49.501381 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage01-crc" (UniqueName: "kubernetes.io/local-volume/local-storage01-crc") on node "crc" Dec 03 11:54:49 crc kubenswrapper[5052]: I1203 11:54:49.594435 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage01-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage01-crc\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:49 crc kubenswrapper[5052]: I1203 11:54:49.615864 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/novacell0745f-account-delete-zbhjv"] Dec 03 11:54:49 crc kubenswrapper[5052]: E1203 11:54:49.718167 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/rabbitmq-cell1-config-data: configmap "rabbitmq-cell1-config-data" not found Dec 03 11:54:49 crc kubenswrapper[5052]: E1203 11:54:49.718226 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a5ba4775-815a-46ef-bd2d-05dfa7513288-config-data podName:a5ba4775-815a-46ef-bd2d-05dfa7513288 nodeName:}" failed. No retries permitted until 2025-12-03 11:54:53.718201855 +0000 UTC m=+2084.306443619 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "config-data" (UniqueName: "kubernetes.io/configmap/a5ba4775-815a-46ef-bd2d-05dfa7513288-config-data") pod "rabbitmq-cell1-server-0" (UID: "a5ba4775-815a-46ef-bd2d-05dfa7513288") : configmap "rabbitmq-cell1-config-data" not found Dec 03 11:54:49 crc kubenswrapper[5052]: I1203 11:54:49.725251 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="511d01be-4413-4580-b284-d49c914d0b8a" path="/var/lib/kubelet/pods/511d01be-4413-4580-b284-d49c914d0b8a/volumes" Dec 03 11:54:49 crc kubenswrapper[5052]: I1203 11:54:49.725845 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c84c82ae-32a2-4965-9ad3-7eb90e03657c" path="/var/lib/kubelet/pods/c84c82ae-32a2-4965-9ad3-7eb90e03657c/volumes" Dec 03 11:54:49 crc kubenswrapper[5052]: I1203 11:54:49.726365 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cca0de9c-68fc-44d4-b6b2-2be05e5d9995" path="/var/lib/kubelet/pods/cca0de9c-68fc-44d4-b6b2-2be05e5d9995/volumes" Dec 03 11:54:49 crc kubenswrapper[5052]: I1203 11:54:49.742604 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4c81b4dc-4f69-4447-8e5e-043aa0c7dd59-openstack-config-secret" (OuterVolumeSpecName: "openstack-config-secret") pod "4c81b4dc-4f69-4447-8e5e-043aa0c7dd59" (UID: "4c81b4dc-4f69-4447-8e5e-043aa0c7dd59"). InnerVolumeSpecName "openstack-config-secret". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:54:49 crc kubenswrapper[5052]: I1203 11:54:49.785879 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4c81b4dc-4f69-4447-8e5e-043aa0c7dd59-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "4c81b4dc-4f69-4447-8e5e-043aa0c7dd59" (UID: "4c81b4dc-4f69-4447-8e5e-043aa0c7dd59"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:54:49 crc kubenswrapper[5052]: I1203 11:54:49.819742 5052 reconciler_common.go:293] "Volume detached for volume \"openstack-config-secret\" (UniqueName: \"kubernetes.io/secret/4c81b4dc-4f69-4447-8e5e-043aa0c7dd59-openstack-config-secret\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:49 crc kubenswrapper[5052]: I1203 11:54:49.820257 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4c81b4dc-4f69-4447-8e5e-043aa0c7dd59-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:49 crc kubenswrapper[5052]: I1203 11:54:49.842366 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/75dd245b-b54f-4a9a-9627-cbf338a69566-ovsdbserver-sb-tls-certs" (OuterVolumeSpecName: "ovsdbserver-sb-tls-certs") pod "75dd245b-b54f-4a9a-9627-cbf338a69566" (UID: "75dd245b-b54f-4a9a-9627-cbf338a69566"). InnerVolumeSpecName "ovsdbserver-sb-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:54:49 crc kubenswrapper[5052]: I1203 11:54:49.855912 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/75dd245b-b54f-4a9a-9627-cbf338a69566-metrics-certs-tls-certs" (OuterVolumeSpecName: "metrics-certs-tls-certs") pod "75dd245b-b54f-4a9a-9627-cbf338a69566" (UID: "75dd245b-b54f-4a9a-9627-cbf338a69566"). InnerVolumeSpecName "metrics-certs-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:54:49 crc kubenswrapper[5052]: I1203 11:54:49.922112 5052 reconciler_common.go:293] "Volume detached for volume \"ovsdbserver-sb-tls-certs\" (UniqueName: \"kubernetes.io/secret/75dd245b-b54f-4a9a-9627-cbf338a69566-ovsdbserver-sb-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:49 crc kubenswrapper[5052]: I1203 11:54:49.922137 5052 reconciler_common.go:293] "Volume detached for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/75dd245b-b54f-4a9a-9627-cbf338a69566-metrics-certs-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:50 crc kubenswrapper[5052]: I1203 11:54:50.004479 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_ovsdbserver-sb-0_75dd245b-b54f-4a9a-9627-cbf338a69566/ovsdbserver-sb/0.log" Dec 03 11:54:50 crc kubenswrapper[5052]: I1203 11:54:50.016157 5052 generic.go:334] "Generic (PLEG): container finished" podID="a5ff0249-173a-4793-8cbe-0aa749f34632" containerID="615dcdb302a4fa77dc8006892c9f0c5509fe1f5ef890116dd6a627f53880b9b8" exitCode=0 Dec 03 11:54:50 crc kubenswrapper[5052]: I1203 11:54:50.016198 5052 generic.go:334] "Generic (PLEG): container finished" podID="a5ff0249-173a-4793-8cbe-0aa749f34632" containerID="14c4e0b558f59a7e38279d4f104332c292bd1c29af9d8ae1768b1a9ec55a8d31" exitCode=2 Dec 03 11:54:50 crc kubenswrapper[5052]: I1203 11:54:50.016209 5052 generic.go:334] "Generic (PLEG): container finished" podID="a5ff0249-173a-4793-8cbe-0aa749f34632" containerID="7bad91ec5c27b9fd97fb06baef39407cc01e86e5c940cc4ffd9ae98b217e02e2" exitCode=0 Dec 03 11:54:50 crc kubenswrapper[5052]: I1203 11:54:50.022625 5052 generic.go:334] "Generic (PLEG): container finished" podID="584deaba-f945-42a2-8c0d-e531d6f48fd8" containerID="3e624a11d1616304ada395cbc34d76126b3b0f68d661f5c753eefaa96b499a5c" exitCode=0 Dec 03 11:54:50 crc kubenswrapper[5052]: I1203 11:54:50.052777 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/cinder6fd6-account-delete-5z97w" podStartSLOduration=5.052758954 podStartE2EDuration="5.052758954s" podCreationTimestamp="2025-12-03 11:54:45 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:54:50.05086036 +0000 UTC m=+2080.639102124" watchObservedRunningTime="2025-12-03 11:54:50.052758954 +0000 UTC m=+2080.641000728" Dec 03 11:54:50 crc kubenswrapper[5052]: I1203 11:54:50.054904 5052 generic.go:334] "Generic (PLEG): container finished" podID="383e7933-2bc4-4ead-bc34-0832091bd952" containerID="60e68a3c80e70643ed7fc58979bc6f13ffbe656be8c84f11083d76691b56fbba" exitCode=0 Dec 03 11:54:50 crc kubenswrapper[5052]: I1203 11:54:50.067575 5052 generic.go:334] "Generic (PLEG): container finished" podID="8dd63158-2c75-455f-8f58-57a55c7e0f22" containerID="77ae7474c6477ac30175222ea524c1ac6cf15f586c4f63872104bcf7083b3836" exitCode=0 Dec 03 11:54:50 crc kubenswrapper[5052]: I1203 11:54:50.083339 5052 generic.go:334] "Generic (PLEG): container finished" podID="a73e0afd-e037-41f7-b214-d4eebd4bafe0" containerID="f95fe7d911e5df1600159d8eebb7f9779559c2e85ed0e281114a0ec510f37ed4" exitCode=0 Dec 03 11:54:50 crc kubenswrapper[5052]: I1203 11:54:50.086962 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstackclient" Dec 03 11:54:50 crc kubenswrapper[5052]: I1203 11:54:50.100775 5052 generic.go:334] "Generic (PLEG): container finished" podID="036bc2b0-bc50-4c91-bea5-882b9e592e4f" containerID="47975dc763fa99fd8e031feff26f6ec865ddd0443ad75de0e4f2b3626568c51b" exitCode=0 Dec 03 11:54:50 crc kubenswrapper[5052]: I1203 11:54:50.100812 5052 generic.go:334] "Generic (PLEG): container finished" podID="036bc2b0-bc50-4c91-bea5-882b9e592e4f" containerID="b83ee0552872f35b33c41234c7f87cdaa58debb1beeb08c6e6d9b6f3a591cb00" exitCode=0 Dec 03 11:54:50 crc kubenswrapper[5052]: I1203 11:54:50.110122 5052 generic.go:334] "Generic (PLEG): container finished" podID="88e7db65-8253-474b-b115-cd3f6edaf85c" containerID="d035b8a698baa861b1fde789196a379e9a2b04d327860b01e3e264af295b680f" exitCode=2 Dec 03 11:54:50 crc kubenswrapper[5052]: E1203 11:54:50.159894 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of 77ae7474c6477ac30175222ea524c1ac6cf15f586c4f63872104bcf7083b3836 is running failed: container process not found" containerID="77ae7474c6477ac30175222ea524c1ac6cf15f586c4f63872104bcf7083b3836" cmd=["/bin/bash","/var/lib/operator-scripts/mysql_probe.sh","readiness"] Dec 03 11:54:50 crc kubenswrapper[5052]: E1203 11:54:50.160810 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of 77ae7474c6477ac30175222ea524c1ac6cf15f586c4f63872104bcf7083b3836 is running failed: container process not found" containerID="77ae7474c6477ac30175222ea524c1ac6cf15f586c4f63872104bcf7083b3836" cmd=["/bin/bash","/var/lib/operator-scripts/mysql_probe.sh","readiness"] Dec 03 11:54:50 crc kubenswrapper[5052]: E1203 11:54:50.161606 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of 77ae7474c6477ac30175222ea524c1ac6cf15f586c4f63872104bcf7083b3836 is running failed: container process not found" containerID="77ae7474c6477ac30175222ea524c1ac6cf15f586c4f63872104bcf7083b3836" cmd=["/bin/bash","/var/lib/operator-scripts/mysql_probe.sh","readiness"] Dec 03 11:54:50 crc kubenswrapper[5052]: E1203 11:54:50.161704 5052 prober.go:104] "Probe errored" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of 77ae7474c6477ac30175222ea524c1ac6cf15f586c4f63872104bcf7083b3836 is running failed: container process not found" probeType="Readiness" pod="openstack-kuttl-tests/openstack-cell1-galera-0" podUID="8dd63158-2c75-455f-8f58-57a55c7e0f22" containerName="galera" Dec 03 11:54:50 crc kubenswrapper[5052]: I1203 11:54:50.228617 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-sb-0" event={"ID":"75dd245b-b54f-4a9a-9627-cbf338a69566","Type":"ContainerDied","Data":"3ca68891bf73f4d039a81c4089ffa5c05d2005f23b02f62e0858bc1c390192eb"} Dec 03 11:54:50 crc kubenswrapper[5052]: I1203 11:54:50.228669 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"a5ff0249-173a-4793-8cbe-0aa749f34632","Type":"ContainerDied","Data":"615dcdb302a4fa77dc8006892c9f0c5509fe1f5ef890116dd6a627f53880b9b8"} Dec 03 11:54:50 crc kubenswrapper[5052]: I1203 11:54:50.228690 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"a5ff0249-173a-4793-8cbe-0aa749f34632","Type":"ContainerDied","Data":"14c4e0b558f59a7e38279d4f104332c292bd1c29af9d8ae1768b1a9ec55a8d31"} Dec 03 11:54:50 crc kubenswrapper[5052]: I1203 11:54:50.228711 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"a5ff0249-173a-4793-8cbe-0aa749f34632","Type":"ContainerDied","Data":"7bad91ec5c27b9fd97fb06baef39407cc01e86e5c940cc4ffd9ae98b217e02e2"} Dec 03 11:54:50 crc kubenswrapper[5052]: I1203 11:54:50.228722 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-worker-6d7f657c65-cqxbg" event={"ID":"584deaba-f945-42a2-8c0d-e531d6f48fd8","Type":"ContainerDied","Data":"3e624a11d1616304ada395cbc34d76126b3b0f68d661f5c753eefaa96b499a5c"} Dec 03 11:54:50 crc kubenswrapper[5052]: I1203 11:54:50.228738 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-worker-6d7f657c65-cqxbg" event={"ID":"584deaba-f945-42a2-8c0d-e531d6f48fd8","Type":"ContainerDied","Data":"945068210f2d3452572796b34ccff45ea13b851c61feca7b498419c59ada21cc"} Dec 03 11:54:50 crc kubenswrapper[5052]: I1203 11:54:50.228751 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="945068210f2d3452572796b34ccff45ea13b851c61feca7b498419c59ada21cc" Dec 03 11:54:50 crc kubenswrapper[5052]: I1203 11:54:50.228767 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/novacell0745f-account-delete-zbhjv" event={"ID":"93d03e32-f318-470c-ad5d-f17a609f5003","Type":"ContainerStarted","Data":"55c65845b9883cfa8223f22f3936649db274c92350e5b942c49cfdce2b08ea11"} Dec 03 11:54:50 crc kubenswrapper[5052]: I1203 11:54:50.228779 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron3cf1-account-delete-zk6qw" event={"ID":"0f42494c-058e-417d-9e0d-97ce7040b288","Type":"ContainerStarted","Data":"e0edc022d786ec7711974e5ad8fa9b59a5df42dc00c4cf83fd27b41325567c72"} Dec 03 11:54:50 crc kubenswrapper[5052]: I1203 11:54:50.228803 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder6fd6-account-delete-5z97w" event={"ID":"53d7a9a2-b6a9-42dc-9828-d8ceaa70f0a3","Type":"ContainerStarted","Data":"099b061b6cbe44d1e953f1c733f1f09b56e58f04c0ae2026319ceff46e418cf1"} Dec 03 11:54:50 crc kubenswrapper[5052]: I1203 11:54:50.228847 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement2868-account-delete-gtkfr" event={"ID":"0bda7b03-76fd-405e-9a69-b897331e829c","Type":"ContainerStarted","Data":"92f47638f85990fab3449257e38cffbd0de5e3c0d1848807a7762ebe630c0e12"} Dec 03 11:54:50 crc kubenswrapper[5052]: I1203 11:54:50.228920 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/novaapie4da-account-delete-mknr9" event={"ID":"6087dc7e-081b-4392-adb6-205b2aec805f","Type":"ContainerStarted","Data":"6d4798b99d8ee6cf41344eb8c16aeb07875c9c62815bb1c4306ee7ec69a52f33"} Dec 03 11:54:50 crc kubenswrapper[5052]: I1203 11:54:50.228933 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican4385-account-delete-krpzg" event={"ID":"383e7933-2bc4-4ead-bc34-0832091bd952","Type":"ContainerDied","Data":"60e68a3c80e70643ed7fc58979bc6f13ffbe656be8c84f11083d76691b56fbba"} Dec 03 11:54:50 crc kubenswrapper[5052]: I1203 11:54:50.229009 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-cell1-galera-0" event={"ID":"8dd63158-2c75-455f-8f58-57a55c7e0f22","Type":"ContainerDied","Data":"77ae7474c6477ac30175222ea524c1ac6cf15f586c4f63872104bcf7083b3836"} Dec 03 11:54:50 crc kubenswrapper[5052]: I1203 11:54:50.229027 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-cell1-galera-0" event={"ID":"8dd63158-2c75-455f-8f58-57a55c7e0f22","Type":"ContainerDied","Data":"c0b9acd24dac2bb1464d4eb45c3f207e86ea3ec833bc0cc1d710c34de7af5f71"} Dec 03 11:54:50 crc kubenswrapper[5052]: I1203 11:54:50.229079 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c0b9acd24dac2bb1464d4eb45c3f207e86ea3ec833bc0cc1d710c34de7af5f71" Dec 03 11:54:50 crc kubenswrapper[5052]: I1203 11:54:50.229105 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance634c-account-delete-nzwqr" event={"ID":"a73e0afd-e037-41f7-b214-d4eebd4bafe0","Type":"ContainerDied","Data":"f95fe7d911e5df1600159d8eebb7f9779559c2e85ed0e281114a0ec510f37ed4"} Dec 03 11:54:50 crc kubenswrapper[5052]: I1203 11:54:50.229142 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" event={"ID":"da3a810d-9fd0-4f18-8f4a-e896b94b1431","Type":"ContainerDied","Data":"3a8314004123a6281ae05824b2b1fadbf5096f8a07470680389b4ffa3aff159c"} Dec 03 11:54:50 crc kubenswrapper[5052]: I1203 11:54:50.229156 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3a8314004123a6281ae05824b2b1fadbf5096f8a07470680389b4ffa3aff159c" Dec 03 11:54:50 crc kubenswrapper[5052]: I1203 11:54:50.229167 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-proxy-b9d8f996b-brs7h" event={"ID":"036bc2b0-bc50-4c91-bea5-882b9e592e4f","Type":"ContainerDied","Data":"47975dc763fa99fd8e031feff26f6ec865ddd0443ad75de0e4f2b3626568c51b"} Dec 03 11:54:50 crc kubenswrapper[5052]: I1203 11:54:50.229180 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-proxy-b9d8f996b-brs7h" event={"ID":"036bc2b0-bc50-4c91-bea5-882b9e592e4f","Type":"ContainerDied","Data":"b83ee0552872f35b33c41234c7f87cdaa58debb1beeb08c6e6d9b6f3a591cb00"} Dec 03 11:54:50 crc kubenswrapper[5052]: I1203 11:54:50.229192 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-z8rn9" event={"ID":"97767bcf-c9a4-430f-8298-3c677150b865","Type":"ContainerStarted","Data":"0161ad705a523e0e5372991220e39f800512d1e2b1dcdd3739d66fcf5243acdf"} Dec 03 11:54:50 crc kubenswrapper[5052]: I1203 11:54:50.229232 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/kube-state-metrics-0" event={"ID":"88e7db65-8253-474b-b115-cd3f6edaf85c","Type":"ContainerDied","Data":"d035b8a698baa861b1fde789196a379e9a2b04d327860b01e3e264af295b680f"} Dec 03 11:54:50 crc kubenswrapper[5052]: I1203 11:54:50.229248 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/kube-state-metrics-0" event={"ID":"88e7db65-8253-474b-b115-cd3f6edaf85c","Type":"ContainerDied","Data":"7a7ebfc7c11351ab35dc3f7e2e09b70208c51b2ecfaedabd2d35aa5d1babb04f"} Dec 03 11:54:50 crc kubenswrapper[5052]: I1203 11:54:50.229278 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7a7ebfc7c11351ab35dc3f7e2e09b70208c51b2ecfaedabd2d35aa5d1babb04f" Dec 03 11:54:50 crc kubenswrapper[5052]: I1203 11:54:50.236062 5052 scope.go:117] "RemoveContainer" containerID="ed36cfdff4484c95d4ba9ac84fbe8a541c4883ce7afbed30c9761965aace756c" Dec 03 11:54:50 crc kubenswrapper[5052]: E1203 11:54:50.291802 5052 cadvisor_stats_provider.go:516] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod036bc2b0_bc50_4c91_bea5_882b9e592e4f.slice/crio-b83ee0552872f35b33c41234c7f87cdaa58debb1beeb08c6e6d9b6f3a591cb00.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod383e7933_2bc4_4ead_bc34_0832091bd952.slice/crio-conmon-60e68a3c80e70643ed7fc58979bc6f13ffbe656be8c84f11083d76691b56fbba.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod036bc2b0_bc50_4c91_bea5_882b9e592e4f.slice/crio-47975dc763fa99fd8e031feff26f6ec865ddd0443ad75de0e4f2b3626568c51b.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod383e7933_2bc4_4ead_bc34_0832091bd952.slice/crio-60e68a3c80e70643ed7fc58979bc6f13ffbe656be8c84f11083d76691b56fbba.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod036bc2b0_bc50_4c91_bea5_882b9e592e4f.slice/crio-conmon-47975dc763fa99fd8e031feff26f6ec865ddd0443ad75de0e4f2b3626568c51b.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod036bc2b0_bc50_4c91_bea5_882b9e592e4f.slice/crio-conmon-b83ee0552872f35b33c41234c7f87cdaa58debb1beeb08c6e6d9b6f3a591cb00.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod53d7a9a2_b6a9_42dc_9828_d8ceaa70f0a3.slice/crio-099b061b6cbe44d1e953f1c733f1f09b56e58f04c0ae2026319ceff46e418cf1.scope\": RecentStats: unable to find data in memory cache]" Dec 03 11:54:50 crc kubenswrapper[5052]: I1203 11:54:50.325550 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/memcached-0"] Dec 03 11:54:50 crc kubenswrapper[5052]: I1203 11:54:50.325778 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/memcached-0" podUID="53a152d4-c58c-40a7-b4ce-b9a9dc42337d" containerName="memcached" containerID="cri-o://c2bfc844a9ad9b1a7f5228851e78306069141a35c777c8e2d5f4055edf6d1d62" gracePeriod=30 Dec 03 11:54:50 crc kubenswrapper[5052]: I1203 11:54:50.369785 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 11:54:50 crc kubenswrapper[5052]: I1203 11:54:50.392249 5052 scope.go:117] "RemoveContainer" containerID="a3ef9454b0aa12b5e49b5ada9b84ef790f5b7a8483610db37b8081939ff19a0a" Dec 03 11:54:50 crc kubenswrapper[5052]: I1203 11:54:50.405675 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/keystone-bootstrap-hv96x"] Dec 03 11:54:50 crc kubenswrapper[5052]: I1203 11:54:50.422570 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/keystone-db-sync-l74fc"] Dec 03 11:54:50 crc kubenswrapper[5052]: I1203 11:54:50.434854 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/keystone-bootstrap-hv96x"] Dec 03 11:54:50 crc kubenswrapper[5052]: I1203 11:54:50.445319 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/keystone-db-sync-l74fc"] Dec 03 11:54:50 crc kubenswrapper[5052]: I1203 11:54:50.461640 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/keystone-94f46848f-wgzvx"] Dec 03 11:54:50 crc kubenswrapper[5052]: I1203 11:54:50.461890 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/keystone-94f46848f-wgzvx" podUID="775d2ccf-9b52-4274-99ef-2c33b18beaea" containerName="keystone-api" containerID="cri-o://92f19f59b5c7604cf76509c11bcfe787a7d812a120d0641a971265397e22c941" gracePeriod=30 Dec 03 11:54:50 crc kubenswrapper[5052]: I1203 11:54:50.495600 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/openstack-galera-0"] Dec 03 11:54:50 crc kubenswrapper[5052]: I1203 11:54:50.504470 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/keystone-db-create-fj555"] Dec 03 11:54:50 crc kubenswrapper[5052]: I1203 11:54:50.513383 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/keystone-db-create-fj555"] Dec 03 11:54:50 crc kubenswrapper[5052]: I1203 11:54:50.515856 5052 scope.go:117] "RemoveContainer" containerID="ed36cfdff4484c95d4ba9ac84fbe8a541c4883ce7afbed30c9761965aace756c" Dec 03 11:54:50 crc kubenswrapper[5052]: E1203 11:54:50.516319 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ed36cfdff4484c95d4ba9ac84fbe8a541c4883ce7afbed30c9761965aace756c\": container with ID starting with ed36cfdff4484c95d4ba9ac84fbe8a541c4883ce7afbed30c9761965aace756c not found: ID does not exist" containerID="ed36cfdff4484c95d4ba9ac84fbe8a541c4883ce7afbed30c9761965aace756c" Dec 03 11:54:50 crc kubenswrapper[5052]: I1203 11:54:50.516350 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ed36cfdff4484c95d4ba9ac84fbe8a541c4883ce7afbed30c9761965aace756c"} err="failed to get container status \"ed36cfdff4484c95d4ba9ac84fbe8a541c4883ce7afbed30c9761965aace756c\": rpc error: code = NotFound desc = could not find container \"ed36cfdff4484c95d4ba9ac84fbe8a541c4883ce7afbed30c9761965aace756c\": container with ID starting with ed36cfdff4484c95d4ba9ac84fbe8a541c4883ce7afbed30c9761965aace756c not found: ID does not exist" Dec 03 11:54:50 crc kubenswrapper[5052]: I1203 11:54:50.516373 5052 scope.go:117] "RemoveContainer" containerID="a3ef9454b0aa12b5e49b5ada9b84ef790f5b7a8483610db37b8081939ff19a0a" Dec 03 11:54:50 crc kubenswrapper[5052]: E1203 11:54:50.517186 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a3ef9454b0aa12b5e49b5ada9b84ef790f5b7a8483610db37b8081939ff19a0a\": container with ID starting with a3ef9454b0aa12b5e49b5ada9b84ef790f5b7a8483610db37b8081939ff19a0a not found: ID does not exist" containerID="a3ef9454b0aa12b5e49b5ada9b84ef790f5b7a8483610db37b8081939ff19a0a" Dec 03 11:54:50 crc kubenswrapper[5052]: I1203 11:54:50.517229 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a3ef9454b0aa12b5e49b5ada9b84ef790f5b7a8483610db37b8081939ff19a0a"} err="failed to get container status \"a3ef9454b0aa12b5e49b5ada9b84ef790f5b7a8483610db37b8081939ff19a0a\": rpc error: code = NotFound desc = could not find container \"a3ef9454b0aa12b5e49b5ada9b84ef790f5b7a8483610db37b8081939ff19a0a\": container with ID starting with a3ef9454b0aa12b5e49b5ada9b84ef790f5b7a8483610db37b8081939ff19a0a not found: ID does not exist" Dec 03 11:54:50 crc kubenswrapper[5052]: I1203 11:54:50.517255 5052 scope.go:117] "RemoveContainer" containerID="d40e529ae6719c4ad0db1e286623449a547ab4530cef62b4eede6892bcc97992" Dec 03 11:54:50 crc kubenswrapper[5052]: I1203 11:54:50.525598 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/keystone-42e8-account-create-update-7bd5x"] Dec 03 11:54:50 crc kubenswrapper[5052]: I1203 11:54:50.532412 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7jb5v\" (UniqueName: \"kubernetes.io/projected/88e7db65-8253-474b-b115-cd3f6edaf85c-kube-api-access-7jb5v\") pod \"88e7db65-8253-474b-b115-cd3f6edaf85c\" (UID: \"88e7db65-8253-474b-b115-cd3f6edaf85c\") " Dec 03 11:54:50 crc kubenswrapper[5052]: I1203 11:54:50.532488 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-state-metrics-tls-config\" (UniqueName: \"kubernetes.io/secret/88e7db65-8253-474b-b115-cd3f6edaf85c-kube-state-metrics-tls-config\") pod \"88e7db65-8253-474b-b115-cd3f6edaf85c\" (UID: \"88e7db65-8253-474b-b115-cd3f6edaf85c\") " Dec 03 11:54:50 crc kubenswrapper[5052]: I1203 11:54:50.532661 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-state-metrics-tls-certs\" (UniqueName: \"kubernetes.io/secret/88e7db65-8253-474b-b115-cd3f6edaf85c-kube-state-metrics-tls-certs\") pod \"88e7db65-8253-474b-b115-cd3f6edaf85c\" (UID: \"88e7db65-8253-474b-b115-cd3f6edaf85c\") " Dec 03 11:54:50 crc kubenswrapper[5052]: I1203 11:54:50.532703 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/88e7db65-8253-474b-b115-cd3f6edaf85c-combined-ca-bundle\") pod \"88e7db65-8253-474b-b115-cd3f6edaf85c\" (UID: \"88e7db65-8253-474b-b115-cd3f6edaf85c\") " Dec 03 11:54:50 crc kubenswrapper[5052]: I1203 11:54:50.535109 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/keystone-42e8-account-create-update-7bd5x"] Dec 03 11:54:50 crc kubenswrapper[5052]: I1203 11:54:50.539813 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 11:54:50 crc kubenswrapper[5052]: I1203 11:54:50.574003 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/88e7db65-8253-474b-b115-cd3f6edaf85c-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "88e7db65-8253-474b-b115-cd3f6edaf85c" (UID: "88e7db65-8253-474b-b115-cd3f6edaf85c"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:54:50 crc kubenswrapper[5052]: I1203 11:54:50.576692 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/88e7db65-8253-474b-b115-cd3f6edaf85c-kube-state-metrics-tls-config" (OuterVolumeSpecName: "kube-state-metrics-tls-config") pod "88e7db65-8253-474b-b115-cd3f6edaf85c" (UID: "88e7db65-8253-474b-b115-cd3f6edaf85c"). InnerVolumeSpecName "kube-state-metrics-tls-config". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:54:50 crc kubenswrapper[5052]: I1203 11:54:50.592713 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/88e7db65-8253-474b-b115-cd3f6edaf85c-kube-api-access-7jb5v" (OuterVolumeSpecName: "kube-api-access-7jb5v") pod "88e7db65-8253-474b-b115-cd3f6edaf85c" (UID: "88e7db65-8253-474b-b115-cd3f6edaf85c"). InnerVolumeSpecName "kube-api-access-7jb5v". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:54:50 crc kubenswrapper[5052]: I1203 11:54:50.613674 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/88e7db65-8253-474b-b115-cd3f6edaf85c-kube-state-metrics-tls-certs" (OuterVolumeSpecName: "kube-state-metrics-tls-certs") pod "88e7db65-8253-474b-b115-cd3f6edaf85c" (UID: "88e7db65-8253-474b-b115-cd3f6edaf85c"). InnerVolumeSpecName "kube-state-metrics-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:54:50 crc kubenswrapper[5052]: I1203 11:54:50.634214 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nm6dm\" (UniqueName: \"kubernetes.io/projected/da3a810d-9fd0-4f18-8f4a-e896b94b1431-kube-api-access-nm6dm\") pod \"da3a810d-9fd0-4f18-8f4a-e896b94b1431\" (UID: \"da3a810d-9fd0-4f18-8f4a-e896b94b1431\") " Dec 03 11:54:50 crc kubenswrapper[5052]: I1203 11:54:50.634372 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/da3a810d-9fd0-4f18-8f4a-e896b94b1431-config-data\") pod \"da3a810d-9fd0-4f18-8f4a-e896b94b1431\" (UID: \"da3a810d-9fd0-4f18-8f4a-e896b94b1431\") " Dec 03 11:54:50 crc kubenswrapper[5052]: I1203 11:54:50.634421 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"nova-novncproxy-tls-certs\" (UniqueName: \"kubernetes.io/secret/da3a810d-9fd0-4f18-8f4a-e896b94b1431-nova-novncproxy-tls-certs\") pod \"da3a810d-9fd0-4f18-8f4a-e896b94b1431\" (UID: \"da3a810d-9fd0-4f18-8f4a-e896b94b1431\") " Dec 03 11:54:50 crc kubenswrapper[5052]: I1203 11:54:50.634462 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"vencrypt-tls-certs\" (UniqueName: \"kubernetes.io/secret/da3a810d-9fd0-4f18-8f4a-e896b94b1431-vencrypt-tls-certs\") pod \"da3a810d-9fd0-4f18-8f4a-e896b94b1431\" (UID: \"da3a810d-9fd0-4f18-8f4a-e896b94b1431\") " Dec 03 11:54:50 crc kubenswrapper[5052]: I1203 11:54:50.634570 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/da3a810d-9fd0-4f18-8f4a-e896b94b1431-combined-ca-bundle\") pod \"da3a810d-9fd0-4f18-8f4a-e896b94b1431\" (UID: \"da3a810d-9fd0-4f18-8f4a-e896b94b1431\") " Dec 03 11:54:50 crc kubenswrapper[5052]: I1203 11:54:50.635563 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-7jb5v\" (UniqueName: \"kubernetes.io/projected/88e7db65-8253-474b-b115-cd3f6edaf85c-kube-api-access-7jb5v\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:50 crc kubenswrapper[5052]: I1203 11:54:50.635583 5052 reconciler_common.go:293] "Volume detached for volume \"kube-state-metrics-tls-config\" (UniqueName: \"kubernetes.io/secret/88e7db65-8253-474b-b115-cd3f6edaf85c-kube-state-metrics-tls-config\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:50 crc kubenswrapper[5052]: I1203 11:54:50.635619 5052 reconciler_common.go:293] "Volume detached for volume \"kube-state-metrics-tls-certs\" (UniqueName: \"kubernetes.io/secret/88e7db65-8253-474b-b115-cd3f6edaf85c-kube-state-metrics-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:50 crc kubenswrapper[5052]: I1203 11:54:50.635633 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/88e7db65-8253-474b-b115-cd3f6edaf85c-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:50 crc kubenswrapper[5052]: I1203 11:54:50.643106 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/da3a810d-9fd0-4f18-8f4a-e896b94b1431-kube-api-access-nm6dm" (OuterVolumeSpecName: "kube-api-access-nm6dm") pod "da3a810d-9fd0-4f18-8f4a-e896b94b1431" (UID: "da3a810d-9fd0-4f18-8f4a-e896b94b1431"). InnerVolumeSpecName "kube-api-access-nm6dm". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:54:50 crc kubenswrapper[5052]: I1203 11:54:50.647766 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/cinder-api-0" podUID="9a4b1245-d95c-4eb6-8d2b-fdd93d98bb1e" containerName="cinder-api" probeResult="failure" output="Get \"https://10.217.1.96:8776/healthcheck\": read tcp 10.217.0.2:40212->10.217.1.96:8776: read: connection reset by peer" Dec 03 11:54:50 crc kubenswrapper[5052]: I1203 11:54:50.698654 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/da3a810d-9fd0-4f18-8f4a-e896b94b1431-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "da3a810d-9fd0-4f18-8f4a-e896b94b1431" (UID: "da3a810d-9fd0-4f18-8f4a-e896b94b1431"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:54:50 crc kubenswrapper[5052]: I1203 11:54:50.740106 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/da3a810d-9fd0-4f18-8f4a-e896b94b1431-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:50 crc kubenswrapper[5052]: I1203 11:54:50.740141 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-nm6dm\" (UniqueName: \"kubernetes.io/projected/da3a810d-9fd0-4f18-8f4a-e896b94b1431-kube-api-access-nm6dm\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:50 crc kubenswrapper[5052]: I1203 11:54:50.744649 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/da3a810d-9fd0-4f18-8f4a-e896b94b1431-vencrypt-tls-certs" (OuterVolumeSpecName: "vencrypt-tls-certs") pod "da3a810d-9fd0-4f18-8f4a-e896b94b1431" (UID: "da3a810d-9fd0-4f18-8f4a-e896b94b1431"). InnerVolumeSpecName "vencrypt-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:54:50 crc kubenswrapper[5052]: I1203 11:54:50.746622 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/da3a810d-9fd0-4f18-8f4a-e896b94b1431-config-data" (OuterVolumeSpecName: "config-data") pod "da3a810d-9fd0-4f18-8f4a-e896b94b1431" (UID: "da3a810d-9fd0-4f18-8f4a-e896b94b1431"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:54:50 crc kubenswrapper[5052]: I1203 11:54:50.751268 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-db-create-rzt98"] Dec 03 11:54:50 crc kubenswrapper[5052]: I1203 11:54:50.762609 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/da3a810d-9fd0-4f18-8f4a-e896b94b1431-nova-novncproxy-tls-certs" (OuterVolumeSpecName: "nova-novncproxy-tls-certs") pod "da3a810d-9fd0-4f18-8f4a-e896b94b1431" (UID: "da3a810d-9fd0-4f18-8f4a-e896b94b1431"). InnerVolumeSpecName "nova-novncproxy-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:54:50 crc kubenswrapper[5052]: I1203 11:54:50.765138 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/barbican-db-create-rzt98"] Dec 03 11:54:50 crc kubenswrapper[5052]: I1203 11:54:50.780746 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-4385-account-create-update-52gf9"] Dec 03 11:54:50 crc kubenswrapper[5052]: I1203 11:54:50.802562 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican4385-account-delete-krpzg"] Dec 03 11:54:50 crc kubenswrapper[5052]: I1203 11:54:50.809177 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/barbican-4385-account-create-update-52gf9"] Dec 03 11:54:50 crc kubenswrapper[5052]: I1203 11:54:50.814609 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/nova-metadata-0" podUID="653882ec-4522-4979-b1aa-7fad5023605c" containerName="nova-metadata-metadata" probeResult="failure" output="Get \"https://10.217.1.135:8775/\": read tcp 10.217.0.2:34624->10.217.1.135:8775: read: connection reset by peer" Dec 03 11:54:50 crc kubenswrapper[5052]: I1203 11:54:50.814944 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/nova-metadata-0" podUID="653882ec-4522-4979-b1aa-7fad5023605c" containerName="nova-metadata-log" probeResult="failure" output="Get \"https://10.217.1.135:8775/\": read tcp 10.217.0.2:34630->10.217.1.135:8775: read: connection reset by peer" Dec 03 11:54:50 crc kubenswrapper[5052]: I1203 11:54:50.818769 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-db-create-m6rsr"] Dec 03 11:54:50 crc kubenswrapper[5052]: I1203 11:54:50.837847 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/glance-db-create-m6rsr"] Dec 03 11:54:50 crc kubenswrapper[5052]: I1203 11:54:50.841736 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/da3a810d-9fd0-4f18-8f4a-e896b94b1431-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:50 crc kubenswrapper[5052]: I1203 11:54:50.841763 5052 reconciler_common.go:293] "Volume detached for volume \"nova-novncproxy-tls-certs\" (UniqueName: \"kubernetes.io/secret/da3a810d-9fd0-4f18-8f4a-e896b94b1431-nova-novncproxy-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:50 crc kubenswrapper[5052]: I1203 11:54:50.841772 5052 reconciler_common.go:293] "Volume detached for volume \"vencrypt-tls-certs\" (UniqueName: \"kubernetes.io/secret/da3a810d-9fd0-4f18-8f4a-e896b94b1431-vencrypt-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:50 crc kubenswrapper[5052]: I1203 11:54:50.852834 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-634c-account-create-update-mmdsg"] Dec 03 11:54:50 crc kubenswrapper[5052]: I1203 11:54:50.861711 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/openstack-galera-0" podUID="f856a02f-3f21-4fd4-8158-50a232de756e" containerName="galera" containerID="cri-o://8d6e9aac3a6cab1fd7cc9acbcd622c819b6570c8ebf974e1634165fe0334ddb0" gracePeriod=30 Dec 03 11:54:50 crc kubenswrapper[5052]: I1203 11:54:50.866753 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/glance-634c-account-create-update-mmdsg"] Dec 03 11:54:50 crc kubenswrapper[5052]: I1203 11:54:50.874207 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance634c-account-delete-nzwqr"] Dec 03 11:54:50 crc kubenswrapper[5052]: I1203 11:54:50.967719 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-sb-0"] Dec 03 11:54:51 crc kubenswrapper[5052]: I1203 11:54:51.029236 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-sb-0"] Dec 03 11:54:51 crc kubenswrapper[5052]: I1203 11:54:51.053573 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-nb-0"] Dec 03 11:54:51 crc kubenswrapper[5052]: I1203 11:54:51.087139 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-nb-0"] Dec 03 11:54:51 crc kubenswrapper[5052]: I1203 11:54:51.126691 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder-db-create-kddz8"] Dec 03 11:54:51 crc kubenswrapper[5052]: I1203 11:54:51.150193 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/cinder-db-create-kddz8"] Dec 03 11:54:51 crc kubenswrapper[5052]: I1203 11:54:51.164852 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder6fd6-account-delete-5z97w"] Dec 03 11:54:51 crc kubenswrapper[5052]: I1203 11:54:51.168833 5052 generic.go:334] "Generic (PLEG): container finished" podID="53a152d4-c58c-40a7-b4ce-b9a9dc42337d" containerID="c2bfc844a9ad9b1a7f5228851e78306069141a35c777c8e2d5f4055edf6d1d62" exitCode=0 Dec 03 11:54:51 crc kubenswrapper[5052]: I1203 11:54:51.168883 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/memcached-0" event={"ID":"53a152d4-c58c-40a7-b4ce-b9a9dc42337d","Type":"ContainerDied","Data":"c2bfc844a9ad9b1a7f5228851e78306069141a35c777c8e2d5f4055edf6d1d62"} Dec 03 11:54:51 crc kubenswrapper[5052]: I1203 11:54:51.174591 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder-6fd6-account-create-update-sz9zj"] Dec 03 11:54:51 crc kubenswrapper[5052]: I1203 11:54:51.183789 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/cinder-6fd6-account-create-update-sz9zj"] Dec 03 11:54:51 crc kubenswrapper[5052]: I1203 11:54:51.189071 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/placement-db-create-8bls7"] Dec 03 11:54:51 crc kubenswrapper[5052]: I1203 11:54:51.197375 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/placement-db-create-8bls7"] Dec 03 11:54:51 crc kubenswrapper[5052]: I1203 11:54:51.222420 5052 generic.go:334] "Generic (PLEG): container finished" podID="a18602ce-7206-4edc-a8d3-5d0754c3da02" containerID="9684094fee34437582198b41d3a72ef6d576587c1466aadd379f8be1cedcf6af" exitCode=0 Dec 03 11:54:51 crc kubenswrapper[5052]: I1203 11:54:51.222608 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/placement2868-account-delete-gtkfr"] Dec 03 11:54:51 crc kubenswrapper[5052]: I1203 11:54:51.222639 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-5758478fc6-5g46v" event={"ID":"a18602ce-7206-4edc-a8d3-5d0754c3da02","Type":"ContainerDied","Data":"9684094fee34437582198b41d3a72ef6d576587c1466aadd379f8be1cedcf6af"} Dec 03 11:54:51 crc kubenswrapper[5052]: I1203 11:54:51.254951 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/placement-2868-account-create-update-nbw4x"] Dec 03 11:54:51 crc kubenswrapper[5052]: I1203 11:54:51.276809 5052 generic.go:334] "Generic (PLEG): container finished" podID="f32095c7-4e6f-43c7-a16f-513e89f7c28e" containerID="9b77bce42854e145583416efda6ad1a83fa0803e3bbc085d98e3a15ec1a50003" exitCode=0 Dec 03 11:54:51 crc kubenswrapper[5052]: I1203 11:54:51.276865 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"f32095c7-4e6f-43c7-a16f-513e89f7c28e","Type":"ContainerDied","Data":"9b77bce42854e145583416efda6ad1a83fa0803e3bbc085d98e3a15ec1a50003"} Dec 03 11:54:51 crc kubenswrapper[5052]: I1203 11:54:51.296297 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/placement-2868-account-create-update-nbw4x"] Dec 03 11:54:51 crc kubenswrapper[5052]: I1203 11:54:51.303821 5052 generic.go:334] "Generic (PLEG): container finished" podID="428e9203-aa9c-48d3-84c9-4dfaa806140d" containerID="f7f46b76f15d594e9e319920752ee78fe4901c945bcac1aa5273dd6bd20b6164" exitCode=0 Dec 03 11:54:51 crc kubenswrapper[5052]: I1203 11:54:51.303898 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-5fc4755558-qz76x" event={"ID":"428e9203-aa9c-48d3-84c9-4dfaa806140d","Type":"ContainerDied","Data":"f7f46b76f15d594e9e319920752ee78fe4901c945bcac1aa5273dd6bd20b6164"} Dec 03 11:54:51 crc kubenswrapper[5052]: I1203 11:54:51.305629 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/neutron-db-create-lfrbv"] Dec 03 11:54:51 crc kubenswrapper[5052]: I1203 11:54:51.316425 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/neutron-db-create-lfrbv"] Dec 03 11:54:51 crc kubenswrapper[5052]: I1203 11:54:51.322040 5052 generic.go:334] "Generic (PLEG): container finished" podID="5679a95d-a498-448e-97ed-497496520506" containerID="3125ced637b6686a088afdde74ed21eda056777bed8dca841a4e6549c3cf70c8" exitCode=0 Dec 03 11:54:51 crc kubenswrapper[5052]: I1203 11:54:51.322096 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-keystone-listener-7bf89d7c74-c4srn" event={"ID":"5679a95d-a498-448e-97ed-497496520506","Type":"ContainerDied","Data":"3125ced637b6686a088afdde74ed21eda056777bed8dca841a4e6549c3cf70c8"} Dec 03 11:54:51 crc kubenswrapper[5052]: I1203 11:54:51.327123 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/neutron3cf1-account-delete-zk6qw"] Dec 03 11:54:51 crc kubenswrapper[5052]: I1203 11:54:51.348815 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/neutron-3cf1-account-create-update-96n2n"] Dec 03 11:54:51 crc kubenswrapper[5052]: I1203 11:54:51.350783 5052 generic.go:334] "Generic (PLEG): container finished" podID="653882ec-4522-4979-b1aa-7fad5023605c" containerID="5c59824fa390c99841f43488a426aa7880afaa22f074091daf68f834930db61d" exitCode=0 Dec 03 11:54:51 crc kubenswrapper[5052]: I1203 11:54:51.350875 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"653882ec-4522-4979-b1aa-7fad5023605c","Type":"ContainerDied","Data":"5c59824fa390c99841f43488a426aa7880afaa22f074091daf68f834930db61d"} Dec 03 11:54:51 crc kubenswrapper[5052]: I1203 11:54:51.366942 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/neutron-3cf1-account-create-update-96n2n"] Dec 03 11:54:51 crc kubenswrapper[5052]: I1203 11:54:51.374300 5052 generic.go:334] "Generic (PLEG): container finished" podID="9a4b1245-d95c-4eb6-8d2b-fdd93d98bb1e" containerID="b4835f9bc39d3950a321af04adbfa74294d65285b1d19075ec93d354b42417ef" exitCode=0 Dec 03 11:54:51 crc kubenswrapper[5052]: I1203 11:54:51.374389 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"9a4b1245-d95c-4eb6-8d2b-fdd93d98bb1e","Type":"ContainerDied","Data":"b4835f9bc39d3950a321af04adbfa74294d65285b1d19075ec93d354b42417ef"} Dec 03 11:54:51 crc kubenswrapper[5052]: I1203 11:54:51.398629 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-proxy-b9d8f996b-brs7h" event={"ID":"036bc2b0-bc50-4c91-bea5-882b9e592e4f","Type":"ContainerDied","Data":"f0b09f86c71e6c4c0eeb418caafc02a9fd15b0cade1aeff34665731a0a629c87"} Dec 03 11:54:51 crc kubenswrapper[5052]: I1203 11:54:51.398689 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f0b09f86c71e6c4c0eeb418caafc02a9fd15b0cade1aeff34665731a0a629c87" Dec 03 11:54:51 crc kubenswrapper[5052]: I1203 11:54:51.419175 5052 generic.go:334] "Generic (PLEG): container finished" podID="53d7a9a2-b6a9-42dc-9828-d8ceaa70f0a3" containerID="099b061b6cbe44d1e953f1c733f1f09b56e58f04c0ae2026319ceff46e418cf1" exitCode=0 Dec 03 11:54:51 crc kubenswrapper[5052]: I1203 11:54:51.419848 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder6fd6-account-delete-5z97w" event={"ID":"53d7a9a2-b6a9-42dc-9828-d8ceaa70f0a3","Type":"ContainerDied","Data":"099b061b6cbe44d1e953f1c733f1f09b56e58f04c0ae2026319ceff46e418cf1"} Dec 03 11:54:51 crc kubenswrapper[5052]: I1203 11:54:51.419923 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 11:54:51 crc kubenswrapper[5052]: I1203 11:54:51.420043 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 11:54:51 crc kubenswrapper[5052]: I1203 11:54:51.701743 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0071cca3-cb80-44f5-86f6-8abc8399d732" path="/var/lib/kubelet/pods/0071cca3-cb80-44f5-86f6-8abc8399d732/volumes" Dec 03 11:54:51 crc kubenswrapper[5052]: I1203 11:54:51.704194 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="00b2551c-6bbe-444d-abb0-2267676dfb09" path="/var/lib/kubelet/pods/00b2551c-6bbe-444d-abb0-2267676dfb09/volumes" Dec 03 11:54:51 crc kubenswrapper[5052]: I1203 11:54:51.711715 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="01498d4a-9b4a-4991-9bb0-a5c77f885672" path="/var/lib/kubelet/pods/01498d4a-9b4a-4991-9bb0-a5c77f885672/volumes" Dec 03 11:54:51 crc kubenswrapper[5052]: I1203 11:54:51.713387 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3df6012c-618d-434a-bfdb-56f8f602b05d" path="/var/lib/kubelet/pods/3df6012c-618d-434a-bfdb-56f8f602b05d/volumes" Dec 03 11:54:51 crc kubenswrapper[5052]: I1203 11:54:51.714078 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3f0bc56f-8f7f-4ebf-94f2-e02dc2edd717" path="/var/lib/kubelet/pods/3f0bc56f-8f7f-4ebf-94f2-e02dc2edd717/volumes" Dec 03 11:54:51 crc kubenswrapper[5052]: I1203 11:54:51.715623 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4c81b4dc-4f69-4447-8e5e-043aa0c7dd59" path="/var/lib/kubelet/pods/4c81b4dc-4f69-4447-8e5e-043aa0c7dd59/volumes" Dec 03 11:54:51 crc kubenswrapper[5052]: I1203 11:54:51.717499 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="75dd245b-b54f-4a9a-9627-cbf338a69566" path="/var/lib/kubelet/pods/75dd245b-b54f-4a9a-9627-cbf338a69566/volumes" Dec 03 11:54:51 crc kubenswrapper[5052]: I1203 11:54:51.718272 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="760bae42-69a5-4a98-8a7d-5f1fd7143591" path="/var/lib/kubelet/pods/760bae42-69a5-4a98-8a7d-5f1fd7143591/volumes" Dec 03 11:54:51 crc kubenswrapper[5052]: I1203 11:54:51.719705 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="79fd3287-4c54-497c-9081-16fbf13bbb48" path="/var/lib/kubelet/pods/79fd3287-4c54-497c-9081-16fbf13bbb48/volumes" Dec 03 11:54:51 crc kubenswrapper[5052]: I1203 11:54:51.720394 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="84a261ec-41e5-487b-b5a1-3743d98c7a76" path="/var/lib/kubelet/pods/84a261ec-41e5-487b-b5a1-3743d98c7a76/volumes" Dec 03 11:54:51 crc kubenswrapper[5052]: I1203 11:54:51.721032 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="896b9690-42d0-4528-886d-e2e4984f1aee" path="/var/lib/kubelet/pods/896b9690-42d0-4528-886d-e2e4984f1aee/volumes" Dec 03 11:54:51 crc kubenswrapper[5052]: I1203 11:54:51.722324 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b4961b81-97b8-4174-82bf-2d1d02cf8335" path="/var/lib/kubelet/pods/b4961b81-97b8-4174-82bf-2d1d02cf8335/volumes" Dec 03 11:54:51 crc kubenswrapper[5052]: I1203 11:54:51.723852 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c86d8ada-bec6-4d35-978b-23e5cbac83ff" path="/var/lib/kubelet/pods/c86d8ada-bec6-4d35-978b-23e5cbac83ff/volumes" Dec 03 11:54:51 crc kubenswrapper[5052]: I1203 11:54:51.724448 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d815bd05-9b18-4ef8-ab32-60bee21bf881" path="/var/lib/kubelet/pods/d815bd05-9b18-4ef8-ab32-60bee21bf881/volumes" Dec 03 11:54:51 crc kubenswrapper[5052]: I1203 11:54:51.725132 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e7c839dc-afd1-4cd2-a6a7-a24a5f716f13" path="/var/lib/kubelet/pods/e7c839dc-afd1-4cd2-a6a7-a24a5f716f13/volumes" Dec 03 11:54:51 crc kubenswrapper[5052]: I1203 11:54:51.726257 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f009f642-7d9a-4739-9e45-bf734d0a0eea" path="/var/lib/kubelet/pods/f009f642-7d9a-4739-9e45-bf734d0a0eea/volumes" Dec 03 11:54:51 crc kubenswrapper[5052]: I1203 11:54:51.726912 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f5994dc8-c6d2-43c7-8035-7e0bfdc09445" path="/var/lib/kubelet/pods/f5994dc8-c6d2-43c7-8035-7e0bfdc09445/volumes" Dec 03 11:54:51 crc kubenswrapper[5052]: I1203 11:54:51.727919 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-api-db-create-4fs9m"] Dec 03 11:54:51 crc kubenswrapper[5052]: I1203 11:54:51.737677 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-api-db-create-4fs9m"] Dec 03 11:54:51 crc kubenswrapper[5052]: I1203 11:54:51.737735 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-api-e4da-account-create-update-rqlfw"] Dec 03 11:54:51 crc kubenswrapper[5052]: I1203 11:54:51.737748 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/novaapie4da-account-delete-mknr9"] Dec 03 11:54:51 crc kubenswrapper[5052]: I1203 11:54:51.742496 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-api-e4da-account-create-update-rqlfw"] Dec 03 11:54:51 crc kubenswrapper[5052]: I1203 11:54:51.787535 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-worker-6d7f657c65-cqxbg" Dec 03 11:54:51 crc kubenswrapper[5052]: I1203 11:54:51.806975 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell0-db-create-42nm2"] Dec 03 11:54:51 crc kubenswrapper[5052]: I1203 11:54:51.811410 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 11:54:51 crc kubenswrapper[5052]: I1203 11:54:51.814941 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell0-db-create-42nm2"] Dec 03 11:54:51 crc kubenswrapper[5052]: I1203 11:54:51.831678 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:54:51 crc kubenswrapper[5052]: I1203 11:54:51.831696 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-proxy-b9d8f996b-brs7h" Dec 03 11:54:51 crc kubenswrapper[5052]: I1203 11:54:51.867075 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell0-745f-account-create-update-5v2dr"] Dec 03 11:54:51 crc kubenswrapper[5052]: I1203 11:54:51.890637 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/novacell0745f-account-delete-zbhjv"] Dec 03 11:54:51 crc kubenswrapper[5052]: I1203 11:54:51.901106 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cjpxg\" (UniqueName: \"kubernetes.io/projected/584deaba-f945-42a2-8c0d-e531d6f48fd8-kube-api-access-cjpxg\") pod \"584deaba-f945-42a2-8c0d-e531d6f48fd8\" (UID: \"584deaba-f945-42a2-8c0d-e531d6f48fd8\") " Dec 03 11:54:51 crc kubenswrapper[5052]: I1203 11:54:51.901220 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/584deaba-f945-42a2-8c0d-e531d6f48fd8-combined-ca-bundle\") pod \"584deaba-f945-42a2-8c0d-e531d6f48fd8\" (UID: \"584deaba-f945-42a2-8c0d-e531d6f48fd8\") " Dec 03 11:54:51 crc kubenswrapper[5052]: I1203 11:54:51.901245 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/584deaba-f945-42a2-8c0d-e531d6f48fd8-logs\") pod \"584deaba-f945-42a2-8c0d-e531d6f48fd8\" (UID: \"584deaba-f945-42a2-8c0d-e531d6f48fd8\") " Dec 03 11:54:51 crc kubenswrapper[5052]: I1203 11:54:51.901280 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/584deaba-f945-42a2-8c0d-e531d6f48fd8-config-data-custom\") pod \"584deaba-f945-42a2-8c0d-e531d6f48fd8\" (UID: \"584deaba-f945-42a2-8c0d-e531d6f48fd8\") " Dec 03 11:54:51 crc kubenswrapper[5052]: I1203 11:54:51.901367 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/584deaba-f945-42a2-8c0d-e531d6f48fd8-config-data\") pod \"584deaba-f945-42a2-8c0d-e531d6f48fd8\" (UID: \"584deaba-f945-42a2-8c0d-e531d6f48fd8\") " Dec 03 11:54:51 crc kubenswrapper[5052]: I1203 11:54:51.904010 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/584deaba-f945-42a2-8c0d-e531d6f48fd8-logs" (OuterVolumeSpecName: "logs") pod "584deaba-f945-42a2-8c0d-e531d6f48fd8" (UID: "584deaba-f945-42a2-8c0d-e531d6f48fd8"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:54:51 crc kubenswrapper[5052]: I1203 11:54:51.910302 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/584deaba-f945-42a2-8c0d-e531d6f48fd8-kube-api-access-cjpxg" (OuterVolumeSpecName: "kube-api-access-cjpxg") pod "584deaba-f945-42a2-8c0d-e531d6f48fd8" (UID: "584deaba-f945-42a2-8c0d-e531d6f48fd8"). InnerVolumeSpecName "kube-api-access-cjpxg". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:54:51 crc kubenswrapper[5052]: I1203 11:54:51.917262 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:54:51 crc kubenswrapper[5052]: I1203 11:54:51.918671 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/584deaba-f945-42a2-8c0d-e531d6f48fd8-config-data-custom" (OuterVolumeSpecName: "config-data-custom") pod "584deaba-f945-42a2-8c0d-e531d6f48fd8" (UID: "584deaba-f945-42a2-8c0d-e531d6f48fd8"). InnerVolumeSpecName "config-data-custom". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:54:51 crc kubenswrapper[5052]: I1203 11:54:51.925286 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell0-745f-account-create-update-5v2dr"] Dec 03 11:54:51 crc kubenswrapper[5052]: I1203 11:54:51.927250 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-5fc4755558-qz76x" Dec 03 11:54:51 crc kubenswrapper[5052]: I1203 11:54:51.935849 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/kube-state-metrics-0"] Dec 03 11:54:51 crc kubenswrapper[5052]: I1203 11:54:51.940639 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/584deaba-f945-42a2-8c0d-e531d6f48fd8-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "584deaba-f945-42a2-8c0d-e531d6f48fd8" (UID: "584deaba-f945-42a2-8c0d-e531d6f48fd8"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:54:51 crc kubenswrapper[5052]: I1203 11:54:51.949572 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/kube-state-metrics-0"] Dec 03 11:54:51 crc kubenswrapper[5052]: E1203 11:54:51.965692 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of 21459936e0c2e5ac30de5ff0dac1297b633e3badacbcf9ad455e81ffb5671c96 is running failed: container process not found" containerID="21459936e0c2e5ac30de5ff0dac1297b633e3badacbcf9ad455e81ffb5671c96" cmd=["/usr/bin/pgrep","-r","DRST","nova-scheduler"] Dec 03 11:54:51 crc kubenswrapper[5052]: E1203 11:54:51.976721 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of 21459936e0c2e5ac30de5ff0dac1297b633e3badacbcf9ad455e81ffb5671c96 is running failed: container process not found" containerID="21459936e0c2e5ac30de5ff0dac1297b633e3badacbcf9ad455e81ffb5671c96" cmd=["/usr/bin/pgrep","-r","DRST","nova-scheduler"] Dec 03 11:54:52 crc kubenswrapper[5052]: E1203 11:54:51.998908 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of 21459936e0c2e5ac30de5ff0dac1297b633e3badacbcf9ad455e81ffb5671c96 is running failed: container process not found" containerID="21459936e0c2e5ac30de5ff0dac1297b633e3badacbcf9ad455e81ffb5671c96" cmd=["/usr/bin/pgrep","-r","DRST","nova-scheduler"] Dec 03 11:54:52 crc kubenswrapper[5052]: E1203 11:54:51.998994 5052 prober.go:104] "Probe errored" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of 21459936e0c2e5ac30de5ff0dac1297b633e3badacbcf9ad455e81ffb5671c96 is running failed: container process not found" probeType="Readiness" pod="openstack-kuttl-tests/nova-scheduler-0" podUID="ec16e8c3-9750-4475-a671-17b0c16826c8" containerName="nova-scheduler-scheduler" Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.002999 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/036bc2b0-bc50-4c91-bea5-882b9e592e4f-log-httpd\") pod \"036bc2b0-bc50-4c91-bea5-882b9e592e4f\" (UID: \"036bc2b0-bc50-4c91-bea5-882b9e592e4f\") " Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.003050 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/9a4b1245-d95c-4eb6-8d2b-fdd93d98bb1e-etc-machine-id\") pod \"9a4b1245-d95c-4eb6-8d2b-fdd93d98bb1e\" (UID: \"9a4b1245-d95c-4eb6-8d2b-fdd93d98bb1e\") " Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.003089 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/8dd63158-2c75-455f-8f58-57a55c7e0f22-galera-tls-certs\") pod \"8dd63158-2c75-455f-8f58-57a55c7e0f22\" (UID: \"8dd63158-2c75-455f-8f58-57a55c7e0f22\") " Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.003128 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/9a4b1245-d95c-4eb6-8d2b-fdd93d98bb1e-config-data\") pod \"9a4b1245-d95c-4eb6-8d2b-fdd93d98bb1e\" (UID: \"9a4b1245-d95c-4eb6-8d2b-fdd93d98bb1e\") " Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.003156 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/f32095c7-4e6f-43c7-a16f-513e89f7c28e-httpd-run\") pod \"f32095c7-4e6f-43c7-a16f-513e89f7c28e\" (UID: \"f32095c7-4e6f-43c7-a16f-513e89f7c28e\") " Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.003231 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/9a4b1245-d95c-4eb6-8d2b-fdd93d98bb1e-logs\") pod \"9a4b1245-d95c-4eb6-8d2b-fdd93d98bb1e\" (UID: \"9a4b1245-d95c-4eb6-8d2b-fdd93d98bb1e\") " Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.003257 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/8dd63158-2c75-455f-8f58-57a55c7e0f22-kolla-config\") pod \"8dd63158-2c75-455f-8f58-57a55c7e0f22\" (UID: \"8dd63158-2c75-455f-8f58-57a55c7e0f22\") " Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.003341 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/036bc2b0-bc50-4c91-bea5-882b9e592e4f-internal-tls-certs\") pod \"036bc2b0-bc50-4c91-bea5-882b9e592e4f\" (UID: \"036bc2b0-bc50-4c91-bea5-882b9e592e4f\") " Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.003369 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/f32095c7-4e6f-43c7-a16f-513e89f7c28e-scripts\") pod \"f32095c7-4e6f-43c7-a16f-513e89f7c28e\" (UID: \"f32095c7-4e6f-43c7-a16f-513e89f7c28e\") " Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.003392 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/036bc2b0-bc50-4c91-bea5-882b9e592e4f-public-tls-certs\") pod \"036bc2b0-bc50-4c91-bea5-882b9e592e4f\" (UID: \"036bc2b0-bc50-4c91-bea5-882b9e592e4f\") " Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.003415 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/9a4b1245-d95c-4eb6-8d2b-fdd93d98bb1e-public-tls-certs\") pod \"9a4b1245-d95c-4eb6-8d2b-fdd93d98bb1e\" (UID: \"9a4b1245-d95c-4eb6-8d2b-fdd93d98bb1e\") " Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.003436 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/8dd63158-2c75-455f-8f58-57a55c7e0f22-config-data-generated\") pod \"8dd63158-2c75-455f-8f58-57a55c7e0f22\" (UID: \"8dd63158-2c75-455f-8f58-57a55c7e0f22\") " Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.003456 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/036bc2b0-bc50-4c91-bea5-882b9e592e4f-combined-ca-bundle\") pod \"036bc2b0-bc50-4c91-bea5-882b9e592e4f\" (UID: \"036bc2b0-bc50-4c91-bea5-882b9e592e4f\") " Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.003485 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/9a4b1245-d95c-4eb6-8d2b-fdd93d98bb1e-config-data-custom\") pod \"9a4b1245-d95c-4eb6-8d2b-fdd93d98bb1e\" (UID: \"9a4b1245-d95c-4eb6-8d2b-fdd93d98bb1e\") " Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.003522 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/9a4b1245-d95c-4eb6-8d2b-fdd93d98bb1e-internal-tls-certs\") pod \"9a4b1245-d95c-4eb6-8d2b-fdd93d98bb1e\" (UID: \"9a4b1245-d95c-4eb6-8d2b-fdd93d98bb1e\") " Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.003544 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/036bc2b0-bc50-4c91-bea5-882b9e592e4f-config-data\") pod \"036bc2b0-bc50-4c91-bea5-882b9e592e4f\" (UID: \"036bc2b0-bc50-4c91-bea5-882b9e592e4f\") " Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.003566 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"mysql-db\" (UniqueName: \"kubernetes.io/local-volume/local-storage17-crc\") pod \"8dd63158-2c75-455f-8f58-57a55c7e0f22\" (UID: \"8dd63158-2c75-455f-8f58-57a55c7e0f22\") " Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.003606 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8k5pl\" (UniqueName: \"kubernetes.io/projected/9a4b1245-d95c-4eb6-8d2b-fdd93d98bb1e-kube-api-access-8k5pl\") pod \"9a4b1245-d95c-4eb6-8d2b-fdd93d98bb1e\" (UID: \"9a4b1245-d95c-4eb6-8d2b-fdd93d98bb1e\") " Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.003639 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/9a4b1245-d95c-4eb6-8d2b-fdd93d98bb1e-scripts\") pod \"9a4b1245-d95c-4eb6-8d2b-fdd93d98bb1e\" (UID: \"9a4b1245-d95c-4eb6-8d2b-fdd93d98bb1e\") " Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.003661 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/036bc2b0-bc50-4c91-bea5-882b9e592e4f-run-httpd\") pod \"036bc2b0-bc50-4c91-bea5-882b9e592e4f\" (UID: \"036bc2b0-bc50-4c91-bea5-882b9e592e4f\") " Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.003696 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/8dd63158-2c75-455f-8f58-57a55c7e0f22-operator-scripts\") pod \"8dd63158-2c75-455f-8f58-57a55c7e0f22\" (UID: \"8dd63158-2c75-455f-8f58-57a55c7e0f22\") " Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.003716 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/f32095c7-4e6f-43c7-a16f-513e89f7c28e-logs\") pod \"f32095c7-4e6f-43c7-a16f-513e89f7c28e\" (UID: \"f32095c7-4e6f-43c7-a16f-513e89f7c28e\") " Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.003735 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"glance\" (UniqueName: \"kubernetes.io/local-volume/local-storage09-crc\") pod \"f32095c7-4e6f-43c7-a16f-513e89f7c28e\" (UID: \"f32095c7-4e6f-43c7-a16f-513e89f7c28e\") " Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.003767 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/036bc2b0-bc50-4c91-bea5-882b9e592e4f-etc-swift\") pod \"036bc2b0-bc50-4c91-bea5-882b9e592e4f\" (UID: \"036bc2b0-bc50-4c91-bea5-882b9e592e4f\") " Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.003803 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8dd63158-2c75-455f-8f58-57a55c7e0f22-combined-ca-bundle\") pod \"8dd63158-2c75-455f-8f58-57a55c7e0f22\" (UID: \"8dd63158-2c75-455f-8f58-57a55c7e0f22\") " Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.003824 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f8hmk\" (UniqueName: \"kubernetes.io/projected/036bc2b0-bc50-4c91-bea5-882b9e592e4f-kube-api-access-f8hmk\") pod \"036bc2b0-bc50-4c91-bea5-882b9e592e4f\" (UID: \"036bc2b0-bc50-4c91-bea5-882b9e592e4f\") " Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.003863 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/8dd63158-2c75-455f-8f58-57a55c7e0f22-config-data-default\") pod \"8dd63158-2c75-455f-8f58-57a55c7e0f22\" (UID: \"8dd63158-2c75-455f-8f58-57a55c7e0f22\") " Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.003881 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9a4b1245-d95c-4eb6-8d2b-fdd93d98bb1e-combined-ca-bundle\") pod \"9a4b1245-d95c-4eb6-8d2b-fdd93d98bb1e\" (UID: \"9a4b1245-d95c-4eb6-8d2b-fdd93d98bb1e\") " Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.003919 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/f32095c7-4e6f-43c7-a16f-513e89f7c28e-config-data\") pod \"f32095c7-4e6f-43c7-a16f-513e89f7c28e\" (UID: \"f32095c7-4e6f-43c7-a16f-513e89f7c28e\") " Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.003948 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/f32095c7-4e6f-43c7-a16f-513e89f7c28e-public-tls-certs\") pod \"f32095c7-4e6f-43c7-a16f-513e89f7c28e\" (UID: \"f32095c7-4e6f-43c7-a16f-513e89f7c28e\") " Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.003983 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-swksq\" (UniqueName: \"kubernetes.io/projected/8dd63158-2c75-455f-8f58-57a55c7e0f22-kube-api-access-swksq\") pod \"8dd63158-2c75-455f-8f58-57a55c7e0f22\" (UID: \"8dd63158-2c75-455f-8f58-57a55c7e0f22\") " Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.004006 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f32095c7-4e6f-43c7-a16f-513e89f7c28e-combined-ca-bundle\") pod \"f32095c7-4e6f-43c7-a16f-513e89f7c28e\" (UID: \"f32095c7-4e6f-43c7-a16f-513e89f7c28e\") " Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.004031 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7l4t2\" (UniqueName: \"kubernetes.io/projected/f32095c7-4e6f-43c7-a16f-513e89f7c28e-kube-api-access-7l4t2\") pod \"f32095c7-4e6f-43c7-a16f-513e89f7c28e\" (UID: \"f32095c7-4e6f-43c7-a16f-513e89f7c28e\") " Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.004500 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-cjpxg\" (UniqueName: \"kubernetes.io/projected/584deaba-f945-42a2-8c0d-e531d6f48fd8-kube-api-access-cjpxg\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.004532 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/584deaba-f945-42a2-8c0d-e531d6f48fd8-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.004544 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/584deaba-f945-42a2-8c0d-e531d6f48fd8-logs\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.004556 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/584deaba-f945-42a2-8c0d-e531d6f48fd8-config-data-custom\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.009103 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/036bc2b0-bc50-4c91-bea5-882b9e592e4f-run-httpd" (OuterVolumeSpecName: "run-httpd") pod "036bc2b0-bc50-4c91-bea5-882b9e592e4f" (UID: "036bc2b0-bc50-4c91-bea5-882b9e592e4f"). InnerVolumeSpecName "run-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.019813 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f32095c7-4e6f-43c7-a16f-513e89f7c28e-httpd-run" (OuterVolumeSpecName: "httpd-run") pod "f32095c7-4e6f-43c7-a16f-513e89f7c28e" (UID: "f32095c7-4e6f-43c7-a16f-513e89f7c28e"). InnerVolumeSpecName "httpd-run". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.020196 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9a4b1245-d95c-4eb6-8d2b-fdd93d98bb1e-logs" (OuterVolumeSpecName: "logs") pod "9a4b1245-d95c-4eb6-8d2b-fdd93d98bb1e" (UID: "9a4b1245-d95c-4eb6-8d2b-fdd93d98bb1e"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.020436 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/9a4b1245-d95c-4eb6-8d2b-fdd93d98bb1e-etc-machine-id" (OuterVolumeSpecName: "etc-machine-id") pod "9a4b1245-d95c-4eb6-8d2b-fdd93d98bb1e" (UID: "9a4b1245-d95c-4eb6-8d2b-fdd93d98bb1e"). InnerVolumeSpecName "etc-machine-id". PluginName "kubernetes.io/host-path", VolumeGidValue "" Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.020618 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8dd63158-2c75-455f-8f58-57a55c7e0f22-kolla-config" (OuterVolumeSpecName: "kolla-config") pod "8dd63158-2c75-455f-8f58-57a55c7e0f22" (UID: "8dd63158-2c75-455f-8f58-57a55c7e0f22"). InnerVolumeSpecName "kolla-config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.021009 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/036bc2b0-bc50-4c91-bea5-882b9e592e4f-log-httpd" (OuterVolumeSpecName: "log-httpd") pod "036bc2b0-bc50-4c91-bea5-882b9e592e4f" (UID: "036bc2b0-bc50-4c91-bea5-882b9e592e4f"). InnerVolumeSpecName "log-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.029184 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8dd63158-2c75-455f-8f58-57a55c7e0f22-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "8dd63158-2c75-455f-8f58-57a55c7e0f22" (UID: "8dd63158-2c75-455f-8f58-57a55c7e0f22"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.029711 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f32095c7-4e6f-43c7-a16f-513e89f7c28e-logs" (OuterVolumeSpecName: "logs") pod "f32095c7-4e6f-43c7-a16f-513e89f7c28e" (UID: "f32095c7-4e6f-43c7-a16f-513e89f7c28e"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.033560 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8dd63158-2c75-455f-8f58-57a55c7e0f22-config-data-generated" (OuterVolumeSpecName: "config-data-generated") pod "8dd63158-2c75-455f-8f58-57a55c7e0f22" (UID: "8dd63158-2c75-455f-8f58-57a55c7e0f22"). InnerVolumeSpecName "config-data-generated". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.064876 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f32095c7-4e6f-43c7-a16f-513e89f7c28e-kube-api-access-7l4t2" (OuterVolumeSpecName: "kube-api-access-7l4t2") pod "f32095c7-4e6f-43c7-a16f-513e89f7c28e" (UID: "f32095c7-4e6f-43c7-a16f-513e89f7c28e"). InnerVolumeSpecName "kube-api-access-7l4t2". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.065273 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f32095c7-4e6f-43c7-a16f-513e89f7c28e-scripts" (OuterVolumeSpecName: "scripts") pod "f32095c7-4e6f-43c7-a16f-513e89f7c28e" (UID: "f32095c7-4e6f-43c7-a16f-513e89f7c28e"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.068914 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8dd63158-2c75-455f-8f58-57a55c7e0f22-config-data-default" (OuterVolumeSpecName: "config-data-default") pod "8dd63158-2c75-455f-8f58-57a55c7e0f22" (UID: "8dd63158-2c75-455f-8f58-57a55c7e0f22"). InnerVolumeSpecName "config-data-default". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.076476 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9a4b1245-d95c-4eb6-8d2b-fdd93d98bb1e-kube-api-access-8k5pl" (OuterVolumeSpecName: "kube-api-access-8k5pl") pod "9a4b1245-d95c-4eb6-8d2b-fdd93d98bb1e" (UID: "9a4b1245-d95c-4eb6-8d2b-fdd93d98bb1e"). InnerVolumeSpecName "kube-api-access-8k5pl". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.112057 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/428e9203-aa9c-48d3-84c9-4dfaa806140d-combined-ca-bundle\") pod \"428e9203-aa9c-48d3-84c9-4dfaa806140d\" (UID: \"428e9203-aa9c-48d3-84c9-4dfaa806140d\") " Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.112106 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sfv2r\" (UniqueName: \"kubernetes.io/projected/428e9203-aa9c-48d3-84c9-4dfaa806140d-kube-api-access-sfv2r\") pod \"428e9203-aa9c-48d3-84c9-4dfaa806140d\" (UID: \"428e9203-aa9c-48d3-84c9-4dfaa806140d\") " Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.112303 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/428e9203-aa9c-48d3-84c9-4dfaa806140d-public-tls-certs\") pod \"428e9203-aa9c-48d3-84c9-4dfaa806140d\" (UID: \"428e9203-aa9c-48d3-84c9-4dfaa806140d\") " Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.112776 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9a4b1245-d95c-4eb6-8d2b-fdd93d98bb1e-scripts" (OuterVolumeSpecName: "scripts") pod "9a4b1245-d95c-4eb6-8d2b-fdd93d98bb1e" (UID: "9a4b1245-d95c-4eb6-8d2b-fdd93d98bb1e"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.112964 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/428e9203-aa9c-48d3-84c9-4dfaa806140d-config-data\") pod \"428e9203-aa9c-48d3-84c9-4dfaa806140d\" (UID: \"428e9203-aa9c-48d3-84c9-4dfaa806140d\") " Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.112988 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/428e9203-aa9c-48d3-84c9-4dfaa806140d-scripts\") pod \"428e9203-aa9c-48d3-84c9-4dfaa806140d\" (UID: \"428e9203-aa9c-48d3-84c9-4dfaa806140d\") " Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.113012 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/9a4b1245-d95c-4eb6-8d2b-fdd93d98bb1e-scripts\") pod \"9a4b1245-d95c-4eb6-8d2b-fdd93d98bb1e\" (UID: \"9a4b1245-d95c-4eb6-8d2b-fdd93d98bb1e\") " Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.113106 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/428e9203-aa9c-48d3-84c9-4dfaa806140d-internal-tls-certs\") pod \"428e9203-aa9c-48d3-84c9-4dfaa806140d\" (UID: \"428e9203-aa9c-48d3-84c9-4dfaa806140d\") " Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.113127 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/428e9203-aa9c-48d3-84c9-4dfaa806140d-logs\") pod \"428e9203-aa9c-48d3-84c9-4dfaa806140d\" (UID: \"428e9203-aa9c-48d3-84c9-4dfaa806140d\") " Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.113745 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-8k5pl\" (UniqueName: \"kubernetes.io/projected/9a4b1245-d95c-4eb6-8d2b-fdd93d98bb1e-kube-api-access-8k5pl\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.113761 5052 reconciler_common.go:293] "Volume detached for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/036bc2b0-bc50-4c91-bea5-882b9e592e4f-run-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.113771 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/8dd63158-2c75-455f-8f58-57a55c7e0f22-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.113779 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/f32095c7-4e6f-43c7-a16f-513e89f7c28e-logs\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.113787 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/8dd63158-2c75-455f-8f58-57a55c7e0f22-config-data-default\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.113796 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-7l4t2\" (UniqueName: \"kubernetes.io/projected/f32095c7-4e6f-43c7-a16f-513e89f7c28e-kube-api-access-7l4t2\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.113805 5052 reconciler_common.go:293] "Volume detached for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/036bc2b0-bc50-4c91-bea5-882b9e592e4f-log-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.113812 5052 reconciler_common.go:293] "Volume detached for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/9a4b1245-d95c-4eb6-8d2b-fdd93d98bb1e-etc-machine-id\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.113822 5052 reconciler_common.go:293] "Volume detached for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/f32095c7-4e6f-43c7-a16f-513e89f7c28e-httpd-run\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.113830 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/9a4b1245-d95c-4eb6-8d2b-fdd93d98bb1e-logs\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.113838 5052 reconciler_common.go:293] "Volume detached for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/8dd63158-2c75-455f-8f58-57a55c7e0f22-kolla-config\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.113846 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/f32095c7-4e6f-43c7-a16f-513e89f7c28e-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.113854 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/8dd63158-2c75-455f-8f58-57a55c7e0f22-config-data-generated\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.113895 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9a4b1245-d95c-4eb6-8d2b-fdd93d98bb1e-config-data-custom" (OuterVolumeSpecName: "config-data-custom") pod "9a4b1245-d95c-4eb6-8d2b-fdd93d98bb1e" (UID: "9a4b1245-d95c-4eb6-8d2b-fdd93d98bb1e"). InnerVolumeSpecName "config-data-custom". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:54:52 crc kubenswrapper[5052]: W1203 11:54:52.114249 5052 empty_dir.go:500] Warning: Unmount skipped because path does not exist: /var/lib/kubelet/pods/9a4b1245-d95c-4eb6-8d2b-fdd93d98bb1e/volumes/kubernetes.io~secret/scripts Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.114268 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9a4b1245-d95c-4eb6-8d2b-fdd93d98bb1e-scripts" (OuterVolumeSpecName: "scripts") pod "9a4b1245-d95c-4eb6-8d2b-fdd93d98bb1e" (UID: "9a4b1245-d95c-4eb6-8d2b-fdd93d98bb1e"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.118374 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/036bc2b0-bc50-4c91-bea5-882b9e592e4f-kube-api-access-f8hmk" (OuterVolumeSpecName: "kube-api-access-f8hmk") pod "036bc2b0-bc50-4c91-bea5-882b9e592e4f" (UID: "036bc2b0-bc50-4c91-bea5-882b9e592e4f"). InnerVolumeSpecName "kube-api-access-f8hmk". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.151636 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/428e9203-aa9c-48d3-84c9-4dfaa806140d-logs" (OuterVolumeSpecName: "logs") pod "428e9203-aa9c-48d3-84c9-4dfaa806140d" (UID: "428e9203-aa9c-48d3-84c9-4dfaa806140d"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.163174 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/036bc2b0-bc50-4c91-bea5-882b9e592e4f-etc-swift" (OuterVolumeSpecName: "etc-swift") pod "036bc2b0-bc50-4c91-bea5-882b9e592e4f" (UID: "036bc2b0-bc50-4c91-bea5-882b9e592e4f"). InnerVolumeSpecName "etc-swift". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.163284 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage09-crc" (OuterVolumeSpecName: "glance") pod "f32095c7-4e6f-43c7-a16f-513e89f7c28e" (UID: "f32095c7-4e6f-43c7-a16f-513e89f7c28e"). InnerVolumeSpecName "local-storage09-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.172713 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/428e9203-aa9c-48d3-84c9-4dfaa806140d-scripts" (OuterVolumeSpecName: "scripts") pod "428e9203-aa9c-48d3-84c9-4dfaa806140d" (UID: "428e9203-aa9c-48d3-84c9-4dfaa806140d"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.188898 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/428e9203-aa9c-48d3-84c9-4dfaa806140d-kube-api-access-sfv2r" (OuterVolumeSpecName: "kube-api-access-sfv2r") pod "428e9203-aa9c-48d3-84c9-4dfaa806140d" (UID: "428e9203-aa9c-48d3-84c9-4dfaa806140d"). InnerVolumeSpecName "kube-api-access-sfv2r". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.189069 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8dd63158-2c75-455f-8f58-57a55c7e0f22-kube-api-access-swksq" (OuterVolumeSpecName: "kube-api-access-swksq") pod "8dd63158-2c75-455f-8f58-57a55c7e0f22" (UID: "8dd63158-2c75-455f-8f58-57a55c7e0f22"). InnerVolumeSpecName "kube-api-access-swksq". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.200000 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/584deaba-f945-42a2-8c0d-e531d6f48fd8-config-data" (OuterVolumeSpecName: "config-data") pod "584deaba-f945-42a2-8c0d-e531d6f48fd8" (UID: "584deaba-f945-42a2-8c0d-e531d6f48fd8"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.209653 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage17-crc" (OuterVolumeSpecName: "mysql-db") pod "8dd63158-2c75-455f-8f58-57a55c7e0f22" (UID: "8dd63158-2c75-455f-8f58-57a55c7e0f22"). InnerVolumeSpecName "local-storage17-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.219699 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/9a4b1245-d95c-4eb6-8d2b-fdd93d98bb1e-config-data-custom\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.219755 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage17-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage17-crc\") on node \"crc\" " Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.219765 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/428e9203-aa9c-48d3-84c9-4dfaa806140d-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.219774 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/9a4b1245-d95c-4eb6-8d2b-fdd93d98bb1e-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.219787 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage09-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage09-crc\") on node \"crc\" " Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.219797 5052 reconciler_common.go:293] "Volume detached for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/036bc2b0-bc50-4c91-bea5-882b9e592e4f-etc-swift\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.219806 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-f8hmk\" (UniqueName: \"kubernetes.io/projected/036bc2b0-bc50-4c91-bea5-882b9e592e4f-kube-api-access-f8hmk\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.219815 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/428e9203-aa9c-48d3-84c9-4dfaa806140d-logs\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.219824 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-sfv2r\" (UniqueName: \"kubernetes.io/projected/428e9203-aa9c-48d3-84c9-4dfaa806140d-kube-api-access-sfv2r\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.219832 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/584deaba-f945-42a2-8c0d-e531d6f48fd8-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.219840 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-swksq\" (UniqueName: \"kubernetes.io/projected/8dd63158-2c75-455f-8f58-57a55c7e0f22-kube-api-access-swksq\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.332581 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage09-crc" (UniqueName: "kubernetes.io/local-volume/local-storage09-crc") on node "crc" Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.356353 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage17-crc" (UniqueName: "kubernetes.io/local-volume/local-storage17-crc") on node "crc" Dec 03 11:54:52 crc kubenswrapper[5052]: E1203 11:54:52.358930 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="c75d78e4eb183b74af6b09e658fe926d3135454064d32c91f94389213e67abcc" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 11:54:52 crc kubenswrapper[5052]: E1203 11:54:52.365070 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="c75d78e4eb183b74af6b09e658fe926d3135454064d32c91f94389213e67abcc" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 11:54:52 crc kubenswrapper[5052]: E1203 11:54:52.366811 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="c75d78e4eb183b74af6b09e658fe926d3135454064d32c91f94389213e67abcc" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 11:54:52 crc kubenswrapper[5052]: E1203 11:54:52.366873 5052 prober.go:104] "Probe errored" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" probeType="Readiness" pod="openstack-kuttl-tests/nova-cell0-conductor-0" podUID="4ec0d450-cd52-432e-8bb2-1ef9698821c1" containerName="nova-cell0-conductor-conductor" Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.374468 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f32095c7-4e6f-43c7-a16f-513e89f7c28e-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "f32095c7-4e6f-43c7-a16f-513e89f7c28e" (UID: "f32095c7-4e6f-43c7-a16f-513e89f7c28e"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.405826 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f32095c7-4e6f-43c7-a16f-513e89f7c28e-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "f32095c7-4e6f-43c7-a16f-513e89f7c28e" (UID: "f32095c7-4e6f-43c7-a16f-513e89f7c28e"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.407689 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9a4b1245-d95c-4eb6-8d2b-fdd93d98bb1e-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "9a4b1245-d95c-4eb6-8d2b-fdd93d98bb1e" (UID: "9a4b1245-d95c-4eb6-8d2b-fdd93d98bb1e"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.415071 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9a4b1245-d95c-4eb6-8d2b-fdd93d98bb1e-config-data" (OuterVolumeSpecName: "config-data") pod "9a4b1245-d95c-4eb6-8d2b-fdd93d98bb1e" (UID: "9a4b1245-d95c-4eb6-8d2b-fdd93d98bb1e"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.421784 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9a4b1245-d95c-4eb6-8d2b-fdd93d98bb1e-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "9a4b1245-d95c-4eb6-8d2b-fdd93d98bb1e" (UID: "9a4b1245-d95c-4eb6-8d2b-fdd93d98bb1e"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.426082 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/036bc2b0-bc50-4c91-bea5-882b9e592e4f-config-data" (OuterVolumeSpecName: "config-data") pod "036bc2b0-bc50-4c91-bea5-882b9e592e4f" (UID: "036bc2b0-bc50-4c91-bea5-882b9e592e4f"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.431973 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/9a4b1245-d95c-4eb6-8d2b-fdd93d98bb1e-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.432238 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/036bc2b0-bc50-4c91-bea5-882b9e592e4f-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.432344 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage17-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage17-crc\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.432417 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage09-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage09-crc\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.432473 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9a4b1245-d95c-4eb6-8d2b-fdd93d98bb1e-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.432592 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/f32095c7-4e6f-43c7-a16f-513e89f7c28e-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.432655 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f32095c7-4e6f-43c7-a16f-513e89f7c28e-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.432715 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/9a4b1245-d95c-4eb6-8d2b-fdd93d98bb1e-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.433814 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement2868-account-delete-gtkfr" event={"ID":"0bda7b03-76fd-405e-9a69-b897331e829c","Type":"ContainerStarted","Data":"3ee6799c23947cd1ca061d572689f0e21a053ace392723fbac989b50aff781bb"} Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.434227 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/placement2868-account-delete-gtkfr" podUID="0bda7b03-76fd-405e-9a69-b897331e829c" containerName="mariadb-account-delete" containerID="cri-o://3ee6799c23947cd1ca061d572689f0e21a053ace392723fbac989b50aff781bb" gracePeriod=30 Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.466469 5052 generic.go:334] "Generic (PLEG): container finished" podID="ec16e8c3-9750-4475-a671-17b0c16826c8" containerID="21459936e0c2e5ac30de5ff0dac1297b633e3badacbcf9ad455e81ffb5671c96" exitCode=0 Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.466581 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"ec16e8c3-9750-4475-a671-17b0c16826c8","Type":"ContainerDied","Data":"21459936e0c2e5ac30de5ff0dac1297b633e3badacbcf9ad455e81ffb5671c96"} Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.532893 5052 generic.go:334] "Generic (PLEG): container finished" podID="97767bcf-c9a4-430f-8298-3c677150b865" containerID="02c1e5c317d63ae18cc826a2fda38a6f9220d39c9c3beaf5b02ece77c75ff269" exitCode=0 Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.532997 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-z8rn9" event={"ID":"97767bcf-c9a4-430f-8298-3c677150b865","Type":"ContainerDied","Data":"02c1e5c317d63ae18cc826a2fda38a6f9220d39c9c3beaf5b02ece77c75ff269"} Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.544583 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/428e9203-aa9c-48d3-84c9-4dfaa806140d-config-data" (OuterVolumeSpecName: "config-data") pod "428e9203-aa9c-48d3-84c9-4dfaa806140d" (UID: "428e9203-aa9c-48d3-84c9-4dfaa806140d"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.545716 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"653882ec-4522-4979-b1aa-7fad5023605c","Type":"ContainerDied","Data":"8cf209cc3fa5306baea03e779a78d1b18f9d6410f0d70d3533fdca6d5197279e"} Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.545772 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8cf209cc3fa5306baea03e779a78d1b18f9d6410f0d70d3533fdca6d5197279e" Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.570888 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/placement2868-account-delete-gtkfr" podStartSLOduration=6.570869092 podStartE2EDuration="6.570869092s" podCreationTimestamp="2025-12-03 11:54:46 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:54:52.466885065 +0000 UTC m=+2083.055126829" watchObservedRunningTime="2025-12-03 11:54:52.570869092 +0000 UTC m=+2083.159110856" Dec 03 11:54:52 crc kubenswrapper[5052]: E1203 11:54:52.581295 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="c3cb1469712553e8283f9907b8e296f175dfb01cae7b28331f721bbf4e80e204" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 11:54:52 crc kubenswrapper[5052]: E1203 11:54:52.583883 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="c3cb1469712553e8283f9907b8e296f175dfb01cae7b28331f721bbf4e80e204" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.584868 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-5758478fc6-5g46v" event={"ID":"a18602ce-7206-4edc-a8d3-5d0754c3da02","Type":"ContainerDied","Data":"cc433ae07b2f7176b8b5d5186d1280a020187109a3af8495da69733e3bc0551d"} Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.584912 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cc433ae07b2f7176b8b5d5186d1280a020187109a3af8495da69733e3bc0551d" Dec 03 11:54:52 crc kubenswrapper[5052]: E1203 11:54:52.585638 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="c3cb1469712553e8283f9907b8e296f175dfb01cae7b28331f721bbf4e80e204" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 11:54:52 crc kubenswrapper[5052]: E1203 11:54:52.585736 5052 prober.go:104] "Probe errored" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" probeType="Readiness" pod="openstack-kuttl-tests/nova-cell1-conductor-0" podUID="4a181d2c-7595-4394-98de-957451309c98" containerName="nova-cell1-conductor-conductor" Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.598145 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-api-5758478fc6-5g46v" Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.598428 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"9a4b1245-d95c-4eb6-8d2b-fdd93d98bb1e","Type":"ContainerDied","Data":"6164f84f9476faf7069883e2525db6c95bc01479dc7ece67cdb31494c9c1c048"} Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.599301 5052 scope.go:117] "RemoveContainer" containerID="b4835f9bc39d3950a321af04adbfa74294d65285b1d19075ec93d354b42417ef" Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.598490 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.603029 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/novacell0745f-account-delete-zbhjv" event={"ID":"93d03e32-f318-470c-ad5d-f17a609f5003","Type":"ContainerStarted","Data":"75afa46f01077a531888f3be755bcaf1e48e735b4481e7ab69d40c172fb1751f"} Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.603243 5052 kubelet_pods.go:1007] "Unable to retrieve pull secret, the image pull may not succeed." pod="openstack-kuttl-tests/novacell0745f-account-delete-zbhjv" secret="" err="secret \"galera-openstack-dockercfg-jfp47\" not found" Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.604257 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/memcached-0" Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.606711 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron3cf1-account-delete-zk6qw" event={"ID":"0f42494c-058e-417d-9e0d-97ce7040b288","Type":"ContainerStarted","Data":"39d19d353a9f922fcda81960e089ebd7d426e58421f5f1bd9b9dc1951175ebeb"} Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.606849 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/neutron3cf1-account-delete-zk6qw" podUID="0f42494c-058e-417d-9e0d-97ce7040b288" containerName="mariadb-account-delete" containerID="cri-o://39d19d353a9f922fcda81960e089ebd7d426e58421f5f1bd9b9dc1951175ebeb" gracePeriod=30 Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.609357 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/memcached-0" Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.609535 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/memcached-0" event={"ID":"53a152d4-c58c-40a7-b4ce-b9a9dc42337d","Type":"ContainerDied","Data":"7699d5b4862dda1b26c99cc5f67d2c38366ff6be793417801fb8d505bf49ce00"} Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.611342 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.612835 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"f32095c7-4e6f-43c7-a16f-513e89f7c28e","Type":"ContainerDied","Data":"c51dcbab996d3c90af5356485f15758b6ccebf38d9965e40f0c878c24fb68968"} Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.613125 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.619894 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-5fc4755558-qz76x" event={"ID":"428e9203-aa9c-48d3-84c9-4dfaa806140d","Type":"ContainerDied","Data":"08e19dd9ccd6aec98e6f2274df9787afa9e6ed5538a316cc3e37b8118cbcb222"} Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.619945 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-5fc4755558-qz76x" Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.627803 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/036bc2b0-bc50-4c91-bea5-882b9e592e4f-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "036bc2b0-bc50-4c91-bea5-882b9e592e4f" (UID: "036bc2b0-bc50-4c91-bea5-882b9e592e4f"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.635977 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/a18602ce-7206-4edc-a8d3-5d0754c3da02-internal-tls-certs\") pod \"a18602ce-7206-4edc-a8d3-5d0754c3da02\" (UID: \"a18602ce-7206-4edc-a8d3-5d0754c3da02\") " Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.636039 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/a18602ce-7206-4edc-a8d3-5d0754c3da02-public-tls-certs\") pod \"a18602ce-7206-4edc-a8d3-5d0754c3da02\" (UID: \"a18602ce-7206-4edc-a8d3-5d0754c3da02\") " Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.636201 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a18602ce-7206-4edc-a8d3-5d0754c3da02-combined-ca-bundle\") pod \"a18602ce-7206-4edc-a8d3-5d0754c3da02\" (UID: \"a18602ce-7206-4edc-a8d3-5d0754c3da02\") " Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.636300 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/a18602ce-7206-4edc-a8d3-5d0754c3da02-config-data-custom\") pod \"a18602ce-7206-4edc-a8d3-5d0754c3da02\" (UID: \"a18602ce-7206-4edc-a8d3-5d0754c3da02\") " Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.636348 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/a18602ce-7206-4edc-a8d3-5d0754c3da02-logs\") pod \"a18602ce-7206-4edc-a8d3-5d0754c3da02\" (UID: \"a18602ce-7206-4edc-a8d3-5d0754c3da02\") " Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.636399 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a18602ce-7206-4edc-a8d3-5d0754c3da02-config-data\") pod \"a18602ce-7206-4edc-a8d3-5d0754c3da02\" (UID: \"a18602ce-7206-4edc-a8d3-5d0754c3da02\") " Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.636424 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fkmkd\" (UniqueName: \"kubernetes.io/projected/a18602ce-7206-4edc-a8d3-5d0754c3da02-kube-api-access-fkmkd\") pod \"a18602ce-7206-4edc-a8d3-5d0754c3da02\" (UID: \"a18602ce-7206-4edc-a8d3-5d0754c3da02\") " Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.636927 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/428e9203-aa9c-48d3-84c9-4dfaa806140d-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.636945 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/036bc2b0-bc50-4c91-bea5-882b9e592e4f-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.637689 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a18602ce-7206-4edc-a8d3-5d0754c3da02-logs" (OuterVolumeSpecName: "logs") pod "a18602ce-7206-4edc-a8d3-5d0754c3da02" (UID: "a18602ce-7206-4edc-a8d3-5d0754c3da02"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.638657 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-keystone-listener-7bf89d7c74-c4srn" Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.646054 5052 generic.go:334] "Generic (PLEG): container finished" podID="0201d941-cd05-4d1c-9af6-f3a82a375a0f" containerID="24b9c22c01a17699dc23692d82c8a6d2aba256d2fe1411900dc2de65cbd33d93" exitCode=0 Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.646200 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"0201d941-cd05-4d1c-9af6-f3a82a375a0f","Type":"ContainerDied","Data":"24b9c22c01a17699dc23692d82c8a6d2aba256d2fe1411900dc2de65cbd33d93"} Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.647998 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.652445 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/novaapie4da-account-delete-mknr9" event={"ID":"6087dc7e-081b-4392-adb6-205b2aec805f","Type":"ContainerStarted","Data":"162808060d27104600e8d3bf34956c7d1f29ffc6f699566d17f44cd664e8fc4c"} Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.653389 5052 kubelet_pods.go:1007] "Unable to retrieve pull secret, the image pull may not succeed." pod="openstack-kuttl-tests/novaapie4da-account-delete-mknr9" secret="" err="secret \"galera-openstack-dockercfg-jfp47\" not found" Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.667158 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-keystone-listener-7bf89d7c74-c4srn" event={"ID":"5679a95d-a498-448e-97ed-497496520506","Type":"ContainerDied","Data":"53669a1fe2b33a6066722e089f916ce83215ef76bae4fcb17bcd2d37ebe966bb"} Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.667256 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-keystone-listener-7bf89d7c74-c4srn" Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.671588 5052 generic.go:334] "Generic (PLEG): container finished" podID="d541d73d-2f95-43a8-8409-40f5f2ff54c4" containerID="756be0d767606b67fd07bb5cf4a07ee4a5ec3d33b08c84185318683745d97e1b" exitCode=0 Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.671646 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"d541d73d-2f95-43a8-8409-40f5f2ff54c4","Type":"ContainerDied","Data":"756be0d767606b67fd07bb5cf4a07ee4a5ec3d33b08c84185318683745d97e1b"} Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.681572 5052 generic.go:334] "Generic (PLEG): container finished" podID="d3222c69-47bc-46d8-a025-cc457881cac9" containerID="13ed5d128b8c9df19939f5f214d29b30d31dc1296f2560559e840f9d3ffbfd12" exitCode=0 Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.681877 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.682932 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.684267 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"d3222c69-47bc-46d8-a025-cc457881cac9","Type":"ContainerDied","Data":"13ed5d128b8c9df19939f5f214d29b30d31dc1296f2560559e840f9d3ffbfd12"} Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.684374 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"d3222c69-47bc-46d8-a025-cc457881cac9","Type":"ContainerDied","Data":"468d819d57751760180e3421d5195178c6b74c7e66737c3521cc9374ec0895b0"} Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.685053 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-worker-6d7f657c65-cqxbg" Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.685620 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-proxy-b9d8f996b-brs7h" Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.707761 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a18602ce-7206-4edc-a8d3-5d0754c3da02-kube-api-access-fkmkd" (OuterVolumeSpecName: "kube-api-access-fkmkd") pod "a18602ce-7206-4edc-a8d3-5d0754c3da02" (UID: "a18602ce-7206-4edc-a8d3-5d0754c3da02"). InnerVolumeSpecName "kube-api-access-fkmkd". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.710724 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a18602ce-7206-4edc-a8d3-5d0754c3da02-config-data-custom" (OuterVolumeSpecName: "config-data-custom") pod "a18602ce-7206-4edc-a8d3-5d0754c3da02" (UID: "a18602ce-7206-4edc-a8d3-5d0754c3da02"). InnerVolumeSpecName "config-data-custom". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.714856 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/neutron3cf1-account-delete-zk6qw" podStartSLOduration=6.714840504 podStartE2EDuration="6.714840504s" podCreationTimestamp="2025-12-03 11:54:46 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:54:52.666939863 +0000 UTC m=+2083.255181637" watchObservedRunningTime="2025-12-03 11:54:52.714840504 +0000 UTC m=+2083.303082268" Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.726979 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f32095c7-4e6f-43c7-a16f-513e89f7c28e-config-data" (OuterVolumeSpecName: "config-data") pod "f32095c7-4e6f-43c7-a16f-513e89f7c28e" (UID: "f32095c7-4e6f-43c7-a16f-513e89f7c28e"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.739084 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j5gfw\" (UniqueName: \"kubernetes.io/projected/53a152d4-c58c-40a7-b4ce-b9a9dc42337d-kube-api-access-j5gfw\") pod \"53a152d4-c58c-40a7-b4ce-b9a9dc42337d\" (UID: \"53a152d4-c58c-40a7-b4ce-b9a9dc42337d\") " Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.739167 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/653882ec-4522-4979-b1aa-7fad5023605c-logs\") pod \"653882ec-4522-4979-b1aa-7fad5023605c\" (UID: \"653882ec-4522-4979-b1aa-7fad5023605c\") " Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.739297 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/5679a95d-a498-448e-97ed-497496520506-logs\") pod \"5679a95d-a498-448e-97ed-497496520506\" (UID: \"5679a95d-a498-448e-97ed-497496520506\") " Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.739314 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/d3222c69-47bc-46d8-a025-cc457881cac9-logs\") pod \"d3222c69-47bc-46d8-a025-cc457881cac9\" (UID: \"d3222c69-47bc-46d8-a025-cc457881cac9\") " Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.739342 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/d3222c69-47bc-46d8-a025-cc457881cac9-internal-tls-certs\") pod \"d3222c69-47bc-46d8-a025-cc457881cac9\" (UID: \"d3222c69-47bc-46d8-a025-cc457881cac9\") " Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.739368 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7csfc\" (UniqueName: \"kubernetes.io/projected/5679a95d-a498-448e-97ed-497496520506-kube-api-access-7csfc\") pod \"5679a95d-a498-448e-97ed-497496520506\" (UID: \"5679a95d-a498-448e-97ed-497496520506\") " Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.739384 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/653882ec-4522-4979-b1aa-7fad5023605c-config-data\") pod \"653882ec-4522-4979-b1aa-7fad5023605c\" (UID: \"653882ec-4522-4979-b1aa-7fad5023605c\") " Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.739419 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/53a152d4-c58c-40a7-b4ce-b9a9dc42337d-kolla-config\") pod \"53a152d4-c58c-40a7-b4ce-b9a9dc42337d\" (UID: \"53a152d4-c58c-40a7-b4ce-b9a9dc42337d\") " Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.739435 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/653882ec-4522-4979-b1aa-7fad5023605c-combined-ca-bundle\") pod \"653882ec-4522-4979-b1aa-7fad5023605c\" (UID: \"653882ec-4522-4979-b1aa-7fad5023605c\") " Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.739478 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d3222c69-47bc-46d8-a025-cc457881cac9-combined-ca-bundle\") pod \"d3222c69-47bc-46d8-a025-cc457881cac9\" (UID: \"d3222c69-47bc-46d8-a025-cc457881cac9\") " Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.739498 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/53a152d4-c58c-40a7-b4ce-b9a9dc42337d-combined-ca-bundle\") pod \"53a152d4-c58c-40a7-b4ce-b9a9dc42337d\" (UID: \"53a152d4-c58c-40a7-b4ce-b9a9dc42337d\") " Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.739578 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/d3222c69-47bc-46d8-a025-cc457881cac9-public-tls-certs\") pod \"d3222c69-47bc-46d8-a025-cc457881cac9\" (UID: \"d3222c69-47bc-46d8-a025-cc457881cac9\") " Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.739604 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/d3222c69-47bc-46d8-a025-cc457881cac9-config-data\") pod \"d3222c69-47bc-46d8-a025-cc457881cac9\" (UID: \"d3222c69-47bc-46d8-a025-cc457881cac9\") " Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.739654 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/5679a95d-a498-448e-97ed-497496520506-config-data-custom\") pod \"5679a95d-a498-448e-97ed-497496520506\" (UID: \"5679a95d-a498-448e-97ed-497496520506\") " Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.739693 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5679a95d-a498-448e-97ed-497496520506-combined-ca-bundle\") pod \"5679a95d-a498-448e-97ed-497496520506\" (UID: \"5679a95d-a498-448e-97ed-497496520506\") " Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.739716 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/53a152d4-c58c-40a7-b4ce-b9a9dc42337d-config-data\") pod \"53a152d4-c58c-40a7-b4ce-b9a9dc42337d\" (UID: \"53a152d4-c58c-40a7-b4ce-b9a9dc42337d\") " Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.739738 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-72gng\" (UniqueName: \"kubernetes.io/projected/653882ec-4522-4979-b1aa-7fad5023605c-kube-api-access-72gng\") pod \"653882ec-4522-4979-b1aa-7fad5023605c\" (UID: \"653882ec-4522-4979-b1aa-7fad5023605c\") " Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.739762 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/5679a95d-a498-448e-97ed-497496520506-config-data\") pod \"5679a95d-a498-448e-97ed-497496520506\" (UID: \"5679a95d-a498-448e-97ed-497496520506\") " Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.739779 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"memcached-tls-certs\" (UniqueName: \"kubernetes.io/secret/53a152d4-c58c-40a7-b4ce-b9a9dc42337d-memcached-tls-certs\") pod \"53a152d4-c58c-40a7-b4ce-b9a9dc42337d\" (UID: \"53a152d4-c58c-40a7-b4ce-b9a9dc42337d\") " Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.739803 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/653882ec-4522-4979-b1aa-7fad5023605c-nova-metadata-tls-certs\") pod \"653882ec-4522-4979-b1aa-7fad5023605c\" (UID: \"653882ec-4522-4979-b1aa-7fad5023605c\") " Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.739827 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gfqkf\" (UniqueName: \"kubernetes.io/projected/d3222c69-47bc-46d8-a025-cc457881cac9-kube-api-access-gfqkf\") pod \"d3222c69-47bc-46d8-a025-cc457881cac9\" (UID: \"d3222c69-47bc-46d8-a025-cc457881cac9\") " Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.740330 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/f32095c7-4e6f-43c7-a16f-513e89f7c28e-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.740345 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/a18602ce-7206-4edc-a8d3-5d0754c3da02-config-data-custom\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.740376 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/a18602ce-7206-4edc-a8d3-5d0754c3da02-logs\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.740386 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-fkmkd\" (UniqueName: \"kubernetes.io/projected/a18602ce-7206-4edc-a8d3-5d0754c3da02-kube-api-access-fkmkd\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:52 crc kubenswrapper[5052]: E1203 11:54:52.744768 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 11:54:52 crc kubenswrapper[5052]: E1203 11:54:52.744835 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/93d03e32-f318-470c-ad5d-f17a609f5003-operator-scripts podName:93d03e32-f318-470c-ad5d-f17a609f5003 nodeName:}" failed. No retries permitted until 2025-12-03 11:54:53.244817433 +0000 UTC m=+2083.833059197 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/93d03e32-f318-470c-ad5d-f17a609f5003-operator-scripts") pod "novacell0745f-account-delete-zbhjv" (UID: "93d03e32-f318-470c-ad5d-f17a609f5003") : configmap "openstack-scripts" not found Dec 03 11:54:52 crc kubenswrapper[5052]: E1203 11:54:52.745070 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 11:54:52 crc kubenswrapper[5052]: E1203 11:54:52.745096 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6087dc7e-081b-4392-adb6-205b2aec805f-operator-scripts podName:6087dc7e-081b-4392-adb6-205b2aec805f nodeName:}" failed. No retries permitted until 2025-12-03 11:54:53.245089711 +0000 UTC m=+2083.833331475 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/6087dc7e-081b-4392-adb6-205b2aec805f-operator-scripts") pod "novaapie4da-account-delete-mknr9" (UID: "6087dc7e-081b-4392-adb6-205b2aec805f") : configmap "openstack-scripts" not found Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.745866 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/653882ec-4522-4979-b1aa-7fad5023605c-logs" (OuterVolumeSpecName: "logs") pod "653882ec-4522-4979-b1aa-7fad5023605c" (UID: "653882ec-4522-4979-b1aa-7fad5023605c"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.748743 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d3222c69-47bc-46d8-a025-cc457881cac9-logs" (OuterVolumeSpecName: "logs") pod "d3222c69-47bc-46d8-a025-cc457881cac9" (UID: "d3222c69-47bc-46d8-a025-cc457881cac9"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.751153 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5679a95d-a498-448e-97ed-497496520506-logs" (OuterVolumeSpecName: "logs") pod "5679a95d-a498-448e-97ed-497496520506" (UID: "5679a95d-a498-448e-97ed-497496520506"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.758572 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/53a152d4-c58c-40a7-b4ce-b9a9dc42337d-kolla-config" (OuterVolumeSpecName: "kolla-config") pod "53a152d4-c58c-40a7-b4ce-b9a9dc42337d" (UID: "53a152d4-c58c-40a7-b4ce-b9a9dc42337d"). InnerVolumeSpecName "kolla-config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.760976 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/53a152d4-c58c-40a7-b4ce-b9a9dc42337d-config-data" (OuterVolumeSpecName: "config-data") pod "53a152d4-c58c-40a7-b4ce-b9a9dc42337d" (UID: "53a152d4-c58c-40a7-b4ce-b9a9dc42337d"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.769561 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/novacell0745f-account-delete-zbhjv" podStartSLOduration=6.769546401 podStartE2EDuration="6.769546401s" podCreationTimestamp="2025-12-03 11:54:46 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:54:52.68117057 +0000 UTC m=+2083.269412334" watchObservedRunningTime="2025-12-03 11:54:52.769546401 +0000 UTC m=+2083.357788165" Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.778455 5052 scope.go:117] "RemoveContainer" containerID="8c286da6c5ac86a7bafdea57cd407cfa1b12fb35b2f4dd6c013af4a83aa8fbb3" Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.782093 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance634c-account-delete-nzwqr" Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.788897 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/novaapie4da-account-delete-mknr9" podStartSLOduration=6.788882384 podStartE2EDuration="6.788882384s" podCreationTimestamp="2025-12-03 11:54:46 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:54:52.749134666 +0000 UTC m=+2083.337376430" watchObservedRunningTime="2025-12-03 11:54:52.788882384 +0000 UTC m=+2083.377124148" Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.801793 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.811710 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.839677 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5679a95d-a498-448e-97ed-497496520506-kube-api-access-7csfc" (OuterVolumeSpecName: "kube-api-access-7csfc") pod "5679a95d-a498-448e-97ed-497496520506" (UID: "5679a95d-a498-448e-97ed-497496520506"). InnerVolumeSpecName "kube-api-access-7csfc". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.839780 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5679a95d-a498-448e-97ed-497496520506-config-data-custom" (OuterVolumeSpecName: "config-data-custom") pod "5679a95d-a498-448e-97ed-497496520506" (UID: "5679a95d-a498-448e-97ed-497496520506"). InnerVolumeSpecName "config-data-custom". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.839845 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/53a152d4-c58c-40a7-b4ce-b9a9dc42337d-kube-api-access-j5gfw" (OuterVolumeSpecName: "kube-api-access-j5gfw") pod "53a152d4-c58c-40a7-b4ce-b9a9dc42337d" (UID: "53a152d4-c58c-40a7-b4ce-b9a9dc42337d"). InnerVolumeSpecName "kube-api-access-j5gfw". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.841117 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/0201d941-cd05-4d1c-9af6-f3a82a375a0f-etc-machine-id\") pod \"0201d941-cd05-4d1c-9af6-f3a82a375a0f\" (UID: \"0201d941-cd05-4d1c-9af6-f3a82a375a0f\") " Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.841245 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/a73e0afd-e037-41f7-b214-d4eebd4bafe0-operator-scripts\") pod \"a73e0afd-e037-41f7-b214-d4eebd4bafe0\" (UID: \"a73e0afd-e037-41f7-b214-d4eebd4bafe0\") " Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.841296 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/0201d941-cd05-4d1c-9af6-f3a82a375a0f-scripts\") pod \"0201d941-cd05-4d1c-9af6-f3a82a375a0f\" (UID: \"0201d941-cd05-4d1c-9af6-f3a82a375a0f\") " Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.841352 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/0201d941-cd05-4d1c-9af6-f3a82a375a0f-config-data\") pod \"0201d941-cd05-4d1c-9af6-f3a82a375a0f\" (UID: \"0201d941-cd05-4d1c-9af6-f3a82a375a0f\") " Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.841387 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zczqd\" (UniqueName: \"kubernetes.io/projected/0201d941-cd05-4d1c-9af6-f3a82a375a0f-kube-api-access-zczqd\") pod \"0201d941-cd05-4d1c-9af6-f3a82a375a0f\" (UID: \"0201d941-cd05-4d1c-9af6-f3a82a375a0f\") " Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.841410 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0201d941-cd05-4d1c-9af6-f3a82a375a0f-combined-ca-bundle\") pod \"0201d941-cd05-4d1c-9af6-f3a82a375a0f\" (UID: \"0201d941-cd05-4d1c-9af6-f3a82a375a0f\") " Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.841484 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/0201d941-cd05-4d1c-9af6-f3a82a375a0f-config-data-custom\") pod \"0201d941-cd05-4d1c-9af6-f3a82a375a0f\" (UID: \"0201d941-cd05-4d1c-9af6-f3a82a375a0f\") " Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.841499 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zvhll\" (UniqueName: \"kubernetes.io/projected/a73e0afd-e037-41f7-b214-d4eebd4bafe0-kube-api-access-zvhll\") pod \"a73e0afd-e037-41f7-b214-d4eebd4bafe0\" (UID: \"a73e0afd-e037-41f7-b214-d4eebd4bafe0\") " Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.842077 5052 reconciler_common.go:293] "Volume detached for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/53a152d4-c58c-40a7-b4ce-b9a9dc42337d-kolla-config\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.842098 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/5679a95d-a498-448e-97ed-497496520506-config-data-custom\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.842107 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/53a152d4-c58c-40a7-b4ce-b9a9dc42337d-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.842116 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-j5gfw\" (UniqueName: \"kubernetes.io/projected/53a152d4-c58c-40a7-b4ce-b9a9dc42337d-kube-api-access-j5gfw\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.842125 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/653882ec-4522-4979-b1aa-7fad5023605c-logs\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.842135 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/d3222c69-47bc-46d8-a025-cc457881cac9-logs\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.842143 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/5679a95d-a498-448e-97ed-497496520506-logs\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.842152 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-7csfc\" (UniqueName: \"kubernetes.io/projected/5679a95d-a498-448e-97ed-497496520506-kube-api-access-7csfc\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.848595 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/653882ec-4522-4979-b1aa-7fad5023605c-kube-api-access-72gng" (OuterVolumeSpecName: "kube-api-access-72gng") pod "653882ec-4522-4979-b1aa-7fad5023605c" (UID: "653882ec-4522-4979-b1aa-7fad5023605c"). InnerVolumeSpecName "kube-api-access-72gng". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.848652 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/0201d941-cd05-4d1c-9af6-f3a82a375a0f-etc-machine-id" (OuterVolumeSpecName: "etc-machine-id") pod "0201d941-cd05-4d1c-9af6-f3a82a375a0f" (UID: "0201d941-cd05-4d1c-9af6-f3a82a375a0f"). InnerVolumeSpecName "etc-machine-id". PluginName "kubernetes.io/host-path", VolumeGidValue "" Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.854006 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8dd63158-2c75-455f-8f58-57a55c7e0f22-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "8dd63158-2c75-455f-8f58-57a55c7e0f22" (UID: "8dd63158-2c75-455f-8f58-57a55c7e0f22"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.854124 5052 scope.go:117] "RemoveContainer" containerID="c2bfc844a9ad9b1a7f5228851e78306069141a35c777c8e2d5f4055edf6d1d62" Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.854278 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican4385-account-delete-krpzg" Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.860253 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d3222c69-47bc-46d8-a025-cc457881cac9-kube-api-access-gfqkf" (OuterVolumeSpecName: "kube-api-access-gfqkf") pod "d3222c69-47bc-46d8-a025-cc457881cac9" (UID: "d3222c69-47bc-46d8-a025-cc457881cac9"). InnerVolumeSpecName "kube-api-access-gfqkf". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.872277 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a73e0afd-e037-41f7-b214-d4eebd4bafe0-kube-api-access-zvhll" (OuterVolumeSpecName: "kube-api-access-zvhll") pod "a73e0afd-e037-41f7-b214-d4eebd4bafe0" (UID: "a73e0afd-e037-41f7-b214-d4eebd4bafe0"). InnerVolumeSpecName "kube-api-access-zvhll". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.872375 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0201d941-cd05-4d1c-9af6-f3a82a375a0f-scripts" (OuterVolumeSpecName: "scripts") pod "0201d941-cd05-4d1c-9af6-f3a82a375a0f" (UID: "0201d941-cd05-4d1c-9af6-f3a82a375a0f"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.872424 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0201d941-cd05-4d1c-9af6-f3a82a375a0f-config-data-custom" (OuterVolumeSpecName: "config-data-custom") pod "0201d941-cd05-4d1c-9af6-f3a82a375a0f" (UID: "0201d941-cd05-4d1c-9af6-f3a82a375a0f"). InnerVolumeSpecName "config-data-custom". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.857941 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a73e0afd-e037-41f7-b214-d4eebd4bafe0-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "a73e0afd-e037-41f7-b214-d4eebd4bafe0" (UID: "a73e0afd-e037-41f7-b214-d4eebd4bafe0"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.875708 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0201d941-cd05-4d1c-9af6-f3a82a375a0f-kube-api-access-zczqd" (OuterVolumeSpecName: "kube-api-access-zczqd") pod "0201d941-cd05-4d1c-9af6-f3a82a375a0f" (UID: "0201d941-cd05-4d1c-9af6-f3a82a375a0f"). InnerVolumeSpecName "kube-api-access-zczqd". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.878226 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d3222c69-47bc-46d8-a025-cc457881cac9-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "d3222c69-47bc-46d8-a025-cc457881cac9" (UID: "d3222c69-47bc-46d8-a025-cc457881cac9"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.896161 5052 scope.go:117] "RemoveContainer" containerID="9b77bce42854e145583416efda6ad1a83fa0803e3bbc085d98e3a15ec1a50003" Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.898634 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-worker-6d7f657c65-cqxbg"] Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.905799 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/barbican-worker-6d7f657c65-cqxbg"] Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.915694 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8dd63158-2c75-455f-8f58-57a55c7e0f22-galera-tls-certs" (OuterVolumeSpecName: "galera-tls-certs") pod "8dd63158-2c75-455f-8f58-57a55c7e0f22" (UID: "8dd63158-2c75-455f-8f58-57a55c7e0f22"). InnerVolumeSpecName "galera-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.936671 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/036bc2b0-bc50-4c91-bea5-882b9e592e4f-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "036bc2b0-bc50-4c91-bea5-882b9e592e4f" (UID: "036bc2b0-bc50-4c91-bea5-882b9e592e4f"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.942983 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/d541d73d-2f95-43a8-8409-40f5f2ff54c4-logs\") pod \"d541d73d-2f95-43a8-8409-40f5f2ff54c4\" (UID: \"d541d73d-2f95-43a8-8409-40f5f2ff54c4\") " Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.943698 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"glance\" (UniqueName: \"kubernetes.io/local-volume/local-storage02-crc\") pod \"d541d73d-2f95-43a8-8409-40f5f2ff54c4\" (UID: \"d541d73d-2f95-43a8-8409-40f5f2ff54c4\") " Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.943731 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d541d73d-2f95-43a8-8409-40f5f2ff54c4-combined-ca-bundle\") pod \"d541d73d-2f95-43a8-8409-40f5f2ff54c4\" (UID: \"d541d73d-2f95-43a8-8409-40f5f2ff54c4\") " Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.943783 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/383e7933-2bc4-4ead-bc34-0832091bd952-operator-scripts\") pod \"383e7933-2bc4-4ead-bc34-0832091bd952\" (UID: \"383e7933-2bc4-4ead-bc34-0832091bd952\") " Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.943849 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/d541d73d-2f95-43a8-8409-40f5f2ff54c4-config-data\") pod \"d541d73d-2f95-43a8-8409-40f5f2ff54c4\" (UID: \"d541d73d-2f95-43a8-8409-40f5f2ff54c4\") " Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.943929 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/d541d73d-2f95-43a8-8409-40f5f2ff54c4-internal-tls-certs\") pod \"d541d73d-2f95-43a8-8409-40f5f2ff54c4\" (UID: \"d541d73d-2f95-43a8-8409-40f5f2ff54c4\") " Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.943975 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hvhb2\" (UniqueName: \"kubernetes.io/projected/d541d73d-2f95-43a8-8409-40f5f2ff54c4-kube-api-access-hvhb2\") pod \"d541d73d-2f95-43a8-8409-40f5f2ff54c4\" (UID: \"d541d73d-2f95-43a8-8409-40f5f2ff54c4\") " Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.944056 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/d541d73d-2f95-43a8-8409-40f5f2ff54c4-scripts\") pod \"d541d73d-2f95-43a8-8409-40f5f2ff54c4\" (UID: \"d541d73d-2f95-43a8-8409-40f5f2ff54c4\") " Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.944118 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m8pj8\" (UniqueName: \"kubernetes.io/projected/383e7933-2bc4-4ead-bc34-0832091bd952-kube-api-access-m8pj8\") pod \"383e7933-2bc4-4ead-bc34-0832091bd952\" (UID: \"383e7933-2bc4-4ead-bc34-0832091bd952\") " Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.944174 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/d541d73d-2f95-43a8-8409-40f5f2ff54c4-httpd-run\") pod \"d541d73d-2f95-43a8-8409-40f5f2ff54c4\" (UID: \"d541d73d-2f95-43a8-8409-40f5f2ff54c4\") " Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.944607 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d541d73d-2f95-43a8-8409-40f5f2ff54c4-logs" (OuterVolumeSpecName: "logs") pod "d541d73d-2f95-43a8-8409-40f5f2ff54c4" (UID: "d541d73d-2f95-43a8-8409-40f5f2ff54c4"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.944966 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d541d73d-2f95-43a8-8409-40f5f2ff54c4-httpd-run" (OuterVolumeSpecName: "httpd-run") pod "d541d73d-2f95-43a8-8409-40f5f2ff54c4" (UID: "d541d73d-2f95-43a8-8409-40f5f2ff54c4"). InnerVolumeSpecName "httpd-run". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.945056 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/a73e0afd-e037-41f7-b214-d4eebd4bafe0-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.945074 5052 reconciler_common.go:293] "Volume detached for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/8dd63158-2c75-455f-8f58-57a55c7e0f22-galera-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.945086 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/0201d941-cd05-4d1c-9af6-f3a82a375a0f-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.945096 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d3222c69-47bc-46d8-a025-cc457881cac9-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.945123 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-zczqd\" (UniqueName: \"kubernetes.io/projected/0201d941-cd05-4d1c-9af6-f3a82a375a0f-kube-api-access-zczqd\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.945134 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/0201d941-cd05-4d1c-9af6-f3a82a375a0f-config-data-custom\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.945145 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-zvhll\" (UniqueName: \"kubernetes.io/projected/a73e0afd-e037-41f7-b214-d4eebd4bafe0-kube-api-access-zvhll\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.945137 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/383e7933-2bc4-4ead-bc34-0832091bd952-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "383e7933-2bc4-4ead-bc34-0832091bd952" (UID: "383e7933-2bc4-4ead-bc34-0832091bd952"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.945155 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/036bc2b0-bc50-4c91-bea5-882b9e592e4f-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.945209 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/d541d73d-2f95-43a8-8409-40f5f2ff54c4-logs\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.945226 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-72gng\" (UniqueName: \"kubernetes.io/projected/653882ec-4522-4979-b1aa-7fad5023605c-kube-api-access-72gng\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.945241 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-gfqkf\" (UniqueName: \"kubernetes.io/projected/d3222c69-47bc-46d8-a025-cc457881cac9-kube-api-access-gfqkf\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.945257 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8dd63158-2c75-455f-8f58-57a55c7e0f22-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.945270 5052 reconciler_common.go:293] "Volume detached for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/0201d941-cd05-4d1c-9af6-f3a82a375a0f-etc-machine-id\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.947754 5052 scope.go:117] "RemoveContainer" containerID="a9bd9892b49a6bfba8e02d24efa8266231be0e4d99061688b1aeffccdabed81f" Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.954690 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d541d73d-2f95-43a8-8409-40f5f2ff54c4-kube-api-access-hvhb2" (OuterVolumeSpecName: "kube-api-access-hvhb2") pod "d541d73d-2f95-43a8-8409-40f5f2ff54c4" (UID: "d541d73d-2f95-43a8-8409-40f5f2ff54c4"). InnerVolumeSpecName "kube-api-access-hvhb2". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.967073 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.972023 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/428e9203-aa9c-48d3-84c9-4dfaa806140d-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "428e9203-aa9c-48d3-84c9-4dfaa806140d" (UID: "428e9203-aa9c-48d3-84c9-4dfaa806140d"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.973087 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage02-crc" (OuterVolumeSpecName: "glance") pod "d541d73d-2f95-43a8-8409-40f5f2ff54c4" (UID: "d541d73d-2f95-43a8-8409-40f5f2ff54c4"). InnerVolumeSpecName "local-storage02-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 11:54:52 crc kubenswrapper[5052]: I1203 11:54:52.985523 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/383e7933-2bc4-4ead-bc34-0832091bd952-kube-api-access-m8pj8" (OuterVolumeSpecName: "kube-api-access-m8pj8") pod "383e7933-2bc4-4ead-bc34-0832091bd952" (UID: "383e7933-2bc4-4ead-bc34-0832091bd952"). InnerVolumeSpecName "kube-api-access-m8pj8". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:54:53 crc kubenswrapper[5052]: I1203 11:54:53.018437 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d541d73d-2f95-43a8-8409-40f5f2ff54c4-scripts" (OuterVolumeSpecName: "scripts") pod "d541d73d-2f95-43a8-8409-40f5f2ff54c4" (UID: "d541d73d-2f95-43a8-8409-40f5f2ff54c4"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:54:53 crc kubenswrapper[5052]: I1203 11:54:53.019988 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/53a152d4-c58c-40a7-b4ce-b9a9dc42337d-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "53a152d4-c58c-40a7-b4ce-b9a9dc42337d" (UID: "53a152d4-c58c-40a7-b4ce-b9a9dc42337d"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:54:53 crc kubenswrapper[5052]: I1203 11:54:53.026663 5052 scope.go:117] "RemoveContainer" containerID="f7f46b76f15d594e9e319920752ee78fe4901c945bcac1aa5273dd6bd20b6164" Dec 03 11:54:53 crc kubenswrapper[5052]: I1203 11:54:53.029816 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 11:54:53 crc kubenswrapper[5052]: I1203 11:54:53.044366 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 11:54:53 crc kubenswrapper[5052]: I1203 11:54:53.046534 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/ec16e8c3-9750-4475-a671-17b0c16826c8-config-data\") pod \"ec16e8c3-9750-4475-a671-17b0c16826c8\" (UID: \"ec16e8c3-9750-4475-a671-17b0c16826c8\") " Dec 03 11:54:53 crc kubenswrapper[5052]: I1203 11:54:53.046849 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q8jkt\" (UniqueName: \"kubernetes.io/projected/ec16e8c3-9750-4475-a671-17b0c16826c8-kube-api-access-q8jkt\") pod \"ec16e8c3-9750-4475-a671-17b0c16826c8\" (UID: \"ec16e8c3-9750-4475-a671-17b0c16826c8\") " Dec 03 11:54:53 crc kubenswrapper[5052]: I1203 11:54:53.046913 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ec16e8c3-9750-4475-a671-17b0c16826c8-combined-ca-bundle\") pod \"ec16e8c3-9750-4475-a671-17b0c16826c8\" (UID: \"ec16e8c3-9750-4475-a671-17b0c16826c8\") " Dec 03 11:54:53 crc kubenswrapper[5052]: I1203 11:54:53.047349 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-hvhb2\" (UniqueName: \"kubernetes.io/projected/d541d73d-2f95-43a8-8409-40f5f2ff54c4-kube-api-access-hvhb2\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:53 crc kubenswrapper[5052]: I1203 11:54:53.047372 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/d541d73d-2f95-43a8-8409-40f5f2ff54c4-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:53 crc kubenswrapper[5052]: I1203 11:54:53.047387 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/53a152d4-c58c-40a7-b4ce-b9a9dc42337d-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:53 crc kubenswrapper[5052]: I1203 11:54:53.047401 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-m8pj8\" (UniqueName: \"kubernetes.io/projected/383e7933-2bc4-4ead-bc34-0832091bd952-kube-api-access-m8pj8\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:53 crc kubenswrapper[5052]: I1203 11:54:53.047413 5052 reconciler_common.go:293] "Volume detached for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/d541d73d-2f95-43a8-8409-40f5f2ff54c4-httpd-run\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:53 crc kubenswrapper[5052]: I1203 11:54:53.047423 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/428e9203-aa9c-48d3-84c9-4dfaa806140d-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:53 crc kubenswrapper[5052]: I1203 11:54:53.047448 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage02-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage02-crc\") on node \"crc\" " Dec 03 11:54:53 crc kubenswrapper[5052]: I1203 11:54:53.047462 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/383e7933-2bc4-4ead-bc34-0832091bd952-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:53 crc kubenswrapper[5052]: I1203 11:54:53.063163 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9a4b1245-d95c-4eb6-8d2b-fdd93d98bb1e-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "9a4b1245-d95c-4eb6-8d2b-fdd93d98bb1e" (UID: "9a4b1245-d95c-4eb6-8d2b-fdd93d98bb1e"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:54:53 crc kubenswrapper[5052]: I1203 11:54:53.066275 5052 scope.go:117] "RemoveContainer" containerID="932117c4bd5dd37889c00f1528583dfd60eee1eb15198c88c121a5439e07b994" Dec 03 11:54:53 crc kubenswrapper[5052]: I1203 11:54:53.069263 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ec16e8c3-9750-4475-a671-17b0c16826c8-kube-api-access-q8jkt" (OuterVolumeSpecName: "kube-api-access-q8jkt") pod "ec16e8c3-9750-4475-a671-17b0c16826c8" (UID: "ec16e8c3-9750-4475-a671-17b0c16826c8"). InnerVolumeSpecName "kube-api-access-q8jkt". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:54:53 crc kubenswrapper[5052]: I1203 11:54:53.085309 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/036bc2b0-bc50-4c91-bea5-882b9e592e4f-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "036bc2b0-bc50-4c91-bea5-882b9e592e4f" (UID: "036bc2b0-bc50-4c91-bea5-882b9e592e4f"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:54:53 crc kubenswrapper[5052]: I1203 11:54:53.087654 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/openstack-cell1-galera-0"] Dec 03 11:54:53 crc kubenswrapper[5052]: I1203 11:54:53.096856 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/openstack-cell1-galera-0"] Dec 03 11:54:53 crc kubenswrapper[5052]: I1203 11:54:53.149005 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-q8jkt\" (UniqueName: \"kubernetes.io/projected/ec16e8c3-9750-4475-a671-17b0c16826c8-kube-api-access-q8jkt\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:53 crc kubenswrapper[5052]: I1203 11:54:53.149038 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/036bc2b0-bc50-4c91-bea5-882b9e592e4f-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:53 crc kubenswrapper[5052]: I1203 11:54:53.149052 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/9a4b1245-d95c-4eb6-8d2b-fdd93d98bb1e-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:53 crc kubenswrapper[5052]: I1203 11:54:53.181799 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d541d73d-2f95-43a8-8409-40f5f2ff54c4-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "d541d73d-2f95-43a8-8409-40f5f2ff54c4" (UID: "d541d73d-2f95-43a8-8409-40f5f2ff54c4"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:54:53 crc kubenswrapper[5052]: I1203 11:54:53.181912 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/428e9203-aa9c-48d3-84c9-4dfaa806140d-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "428e9203-aa9c-48d3-84c9-4dfaa806140d" (UID: "428e9203-aa9c-48d3-84c9-4dfaa806140d"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:54:53 crc kubenswrapper[5052]: I1203 11:54:53.183529 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/428e9203-aa9c-48d3-84c9-4dfaa806140d-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "428e9203-aa9c-48d3-84c9-4dfaa806140d" (UID: "428e9203-aa9c-48d3-84c9-4dfaa806140d"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:54:53 crc kubenswrapper[5052]: I1203 11:54:53.184643 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5679a95d-a498-448e-97ed-497496520506-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "5679a95d-a498-448e-97ed-497496520506" (UID: "5679a95d-a498-448e-97ed-497496520506"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:54:53 crc kubenswrapper[5052]: I1203 11:54:53.184967 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/653882ec-4522-4979-b1aa-7fad5023605c-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "653882ec-4522-4979-b1aa-7fad5023605c" (UID: "653882ec-4522-4979-b1aa-7fad5023605c"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:54:53 crc kubenswrapper[5052]: I1203 11:54:53.188343 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d3222c69-47bc-46d8-a025-cc457881cac9-config-data" (OuterVolumeSpecName: "config-data") pod "d3222c69-47bc-46d8-a025-cc457881cac9" (UID: "d3222c69-47bc-46d8-a025-cc457881cac9"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:54:53 crc kubenswrapper[5052]: I1203 11:54:53.220822 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0201d941-cd05-4d1c-9af6-f3a82a375a0f-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "0201d941-cd05-4d1c-9af6-f3a82a375a0f" (UID: "0201d941-cd05-4d1c-9af6-f3a82a375a0f"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:54:53 crc kubenswrapper[5052]: I1203 11:54:53.222252 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/653882ec-4522-4979-b1aa-7fad5023605c-config-data" (OuterVolumeSpecName: "config-data") pod "653882ec-4522-4979-b1aa-7fad5023605c" (UID: "653882ec-4522-4979-b1aa-7fad5023605c"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:54:53 crc kubenswrapper[5052]: I1203 11:54:53.250621 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/d3222c69-47bc-46d8-a025-cc457881cac9-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:53 crc kubenswrapper[5052]: I1203 11:54:53.250656 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5679a95d-a498-448e-97ed-497496520506-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:53 crc kubenswrapper[5052]: I1203 11:54:53.250669 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d541d73d-2f95-43a8-8409-40f5f2ff54c4-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:53 crc kubenswrapper[5052]: I1203 11:54:53.250679 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/428e9203-aa9c-48d3-84c9-4dfaa806140d-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:53 crc kubenswrapper[5052]: I1203 11:54:53.250692 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/428e9203-aa9c-48d3-84c9-4dfaa806140d-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:53 crc kubenswrapper[5052]: I1203 11:54:53.250702 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/653882ec-4522-4979-b1aa-7fad5023605c-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:53 crc kubenswrapper[5052]: I1203 11:54:53.250711 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/653882ec-4522-4979-b1aa-7fad5023605c-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:53 crc kubenswrapper[5052]: I1203 11:54:53.250719 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0201d941-cd05-4d1c-9af6-f3a82a375a0f-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:53 crc kubenswrapper[5052]: E1203 11:54:53.250782 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/rabbitmq-config-data: configmap "rabbitmq-config-data" not found Dec 03 11:54:53 crc kubenswrapper[5052]: E1203 11:54:53.250832 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/10e3094e-f625-4d69-b0e9-358220c0d195-config-data podName:10e3094e-f625-4d69-b0e9-358220c0d195 nodeName:}" failed. No retries permitted until 2025-12-03 11:55:01.250817041 +0000 UTC m=+2091.839058805 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "config-data" (UniqueName: "kubernetes.io/configmap/10e3094e-f625-4d69-b0e9-358220c0d195-config-data") pod "rabbitmq-server-0" (UID: "10e3094e-f625-4d69-b0e9-358220c0d195") : configmap "rabbitmq-config-data" not found Dec 03 11:54:53 crc kubenswrapper[5052]: E1203 11:54:53.250868 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 11:54:53 crc kubenswrapper[5052]: E1203 11:54:53.250890 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/93d03e32-f318-470c-ad5d-f17a609f5003-operator-scripts podName:93d03e32-f318-470c-ad5d-f17a609f5003 nodeName:}" failed. No retries permitted until 2025-12-03 11:54:54.250882912 +0000 UTC m=+2084.839124676 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/93d03e32-f318-470c-ad5d-f17a609f5003-operator-scripts") pod "novacell0745f-account-delete-zbhjv" (UID: "93d03e32-f318-470c-ad5d-f17a609f5003") : configmap "openstack-scripts" not found Dec 03 11:54:53 crc kubenswrapper[5052]: E1203 11:54:53.250912 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 11:54:53 crc kubenswrapper[5052]: E1203 11:54:53.250930 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6087dc7e-081b-4392-adb6-205b2aec805f-operator-scripts podName:6087dc7e-081b-4392-adb6-205b2aec805f nodeName:}" failed. No retries permitted until 2025-12-03 11:54:54.250924044 +0000 UTC m=+2084.839165808 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/6087dc7e-081b-4392-adb6-205b2aec805f-operator-scripts") pod "novaapie4da-account-delete-mknr9" (UID: "6087dc7e-081b-4392-adb6-205b2aec805f") : configmap "openstack-scripts" not found Dec 03 11:54:53 crc kubenswrapper[5052]: I1203 11:54:53.261388 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a18602ce-7206-4edc-a8d3-5d0754c3da02-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "a18602ce-7206-4edc-a8d3-5d0754c3da02" (UID: "a18602ce-7206-4edc-a8d3-5d0754c3da02"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:54:53 crc kubenswrapper[5052]: I1203 11:54:53.270165 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage02-crc" (UniqueName: "kubernetes.io/local-volume/local-storage02-crc") on node "crc" Dec 03 11:54:53 crc kubenswrapper[5052]: I1203 11:54:53.272149 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a18602ce-7206-4edc-a8d3-5d0754c3da02-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "a18602ce-7206-4edc-a8d3-5d0754c3da02" (UID: "a18602ce-7206-4edc-a8d3-5d0754c3da02"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:54:53 crc kubenswrapper[5052]: I1203 11:54:53.277716 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a18602ce-7206-4edc-a8d3-5d0754c3da02-config-data" (OuterVolumeSpecName: "config-data") pod "a18602ce-7206-4edc-a8d3-5d0754c3da02" (UID: "a18602ce-7206-4edc-a8d3-5d0754c3da02"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:54:53 crc kubenswrapper[5052]: I1203 11:54:53.283995 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a18602ce-7206-4edc-a8d3-5d0754c3da02-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "a18602ce-7206-4edc-a8d3-5d0754c3da02" (UID: "a18602ce-7206-4edc-a8d3-5d0754c3da02"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:54:53 crc kubenswrapper[5052]: I1203 11:54:53.308638 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5679a95d-a498-448e-97ed-497496520506-config-data" (OuterVolumeSpecName: "config-data") pod "5679a95d-a498-448e-97ed-497496520506" (UID: "5679a95d-a498-448e-97ed-497496520506"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:54:53 crc kubenswrapper[5052]: I1203 11:54:53.319277 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ec16e8c3-9750-4475-a671-17b0c16826c8-config-data" (OuterVolumeSpecName: "config-data") pod "ec16e8c3-9750-4475-a671-17b0c16826c8" (UID: "ec16e8c3-9750-4475-a671-17b0c16826c8"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:54:53 crc kubenswrapper[5052]: I1203 11:54:53.324772 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/53a152d4-c58c-40a7-b4ce-b9a9dc42337d-memcached-tls-certs" (OuterVolumeSpecName: "memcached-tls-certs") pod "53a152d4-c58c-40a7-b4ce-b9a9dc42337d" (UID: "53a152d4-c58c-40a7-b4ce-b9a9dc42337d"). InnerVolumeSpecName "memcached-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:54:53 crc kubenswrapper[5052]: I1203 11:54:53.339720 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/653882ec-4522-4979-b1aa-7fad5023605c-nova-metadata-tls-certs" (OuterVolumeSpecName: "nova-metadata-tls-certs") pod "653882ec-4522-4979-b1aa-7fad5023605c" (UID: "653882ec-4522-4979-b1aa-7fad5023605c"). InnerVolumeSpecName "nova-metadata-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:54:53 crc kubenswrapper[5052]: I1203 11:54:53.341449 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d3222c69-47bc-46d8-a025-cc457881cac9-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "d3222c69-47bc-46d8-a025-cc457881cac9" (UID: "d3222c69-47bc-46d8-a025-cc457881cac9"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:54:53 crc kubenswrapper[5052]: I1203 11:54:53.346678 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ec16e8c3-9750-4475-a671-17b0c16826c8-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "ec16e8c3-9750-4475-a671-17b0c16826c8" (UID: "ec16e8c3-9750-4475-a671-17b0c16826c8"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:54:53 crc kubenswrapper[5052]: I1203 11:54:53.352268 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage02-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage02-crc\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:53 crc kubenswrapper[5052]: I1203 11:54:53.352306 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a18602ce-7206-4edc-a8d3-5d0754c3da02-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:53 crc kubenswrapper[5052]: I1203 11:54:53.352346 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ec16e8c3-9750-4475-a671-17b0c16826c8-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:53 crc kubenswrapper[5052]: I1203 11:54:53.352362 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/d3222c69-47bc-46d8-a025-cc457881cac9-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:53 crc kubenswrapper[5052]: I1203 11:54:53.352373 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a18602ce-7206-4edc-a8d3-5d0754c3da02-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:53 crc kubenswrapper[5052]: I1203 11:54:53.352385 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/ec16e8c3-9750-4475-a671-17b0c16826c8-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:53 crc kubenswrapper[5052]: I1203 11:54:53.352396 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/a18602ce-7206-4edc-a8d3-5d0754c3da02-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:53 crc kubenswrapper[5052]: I1203 11:54:53.352432 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/a18602ce-7206-4edc-a8d3-5d0754c3da02-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:53 crc kubenswrapper[5052]: I1203 11:54:53.352445 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/5679a95d-a498-448e-97ed-497496520506-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:53 crc kubenswrapper[5052]: I1203 11:54:53.352456 5052 reconciler_common.go:293] "Volume detached for volume \"memcached-tls-certs\" (UniqueName: \"kubernetes.io/secret/53a152d4-c58c-40a7-b4ce-b9a9dc42337d-memcached-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:53 crc kubenswrapper[5052]: I1203 11:54:53.352469 5052 reconciler_common.go:293] "Volume detached for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/653882ec-4522-4979-b1aa-7fad5023605c-nova-metadata-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:53 crc kubenswrapper[5052]: I1203 11:54:53.353404 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d541d73d-2f95-43a8-8409-40f5f2ff54c4-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "d541d73d-2f95-43a8-8409-40f5f2ff54c4" (UID: "d541d73d-2f95-43a8-8409-40f5f2ff54c4"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:54:53 crc kubenswrapper[5052]: I1203 11:54:53.354936 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d3222c69-47bc-46d8-a025-cc457881cac9-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "d3222c69-47bc-46d8-a025-cc457881cac9" (UID: "d3222c69-47bc-46d8-a025-cc457881cac9"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:54:53 crc kubenswrapper[5052]: I1203 11:54:53.360503 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d541d73d-2f95-43a8-8409-40f5f2ff54c4-config-data" (OuterVolumeSpecName: "config-data") pod "d541d73d-2f95-43a8-8409-40f5f2ff54c4" (UID: "d541d73d-2f95-43a8-8409-40f5f2ff54c4"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:54:53 crc kubenswrapper[5052]: I1203 11:54:53.366477 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0201d941-cd05-4d1c-9af6-f3a82a375a0f-config-data" (OuterVolumeSpecName: "config-data") pod "0201d941-cd05-4d1c-9af6-f3a82a375a0f" (UID: "0201d941-cd05-4d1c-9af6-f3a82a375a0f"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:54:53 crc kubenswrapper[5052]: I1203 11:54:53.448004 5052 scope.go:117] "RemoveContainer" containerID="3125ced637b6686a088afdde74ed21eda056777bed8dca841a4e6549c3cf70c8" Dec 03 11:54:53 crc kubenswrapper[5052]: I1203 11:54:53.454592 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/d541d73d-2f95-43a8-8409-40f5f2ff54c4-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:53 crc kubenswrapper[5052]: I1203 11:54:53.454617 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/0201d941-cd05-4d1c-9af6-f3a82a375a0f-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:53 crc kubenswrapper[5052]: I1203 11:54:53.454632 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/d3222c69-47bc-46d8-a025-cc457881cac9-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:53 crc kubenswrapper[5052]: I1203 11:54:53.454643 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/d541d73d-2f95-43a8-8409-40f5f2ff54c4-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:53 crc kubenswrapper[5052]: I1203 11:54:53.465966 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder6fd6-account-delete-5z97w" Dec 03 11:54:53 crc kubenswrapper[5052]: I1203 11:54:53.477541 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/swift-proxy-b9d8f996b-brs7h"] Dec 03 11:54:53 crc kubenswrapper[5052]: I1203 11:54:53.484887 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/swift-proxy-b9d8f996b-brs7h"] Dec 03 11:54:53 crc kubenswrapper[5052]: I1203 11:54:53.495183 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 11:54:53 crc kubenswrapper[5052]: I1203 11:54:53.497430 5052 scope.go:117] "RemoveContainer" containerID="b657a1c95ebe202d32286143c3e1cb3b321d2c67300afcf9cb53509b8a6aee42" Dec 03 11:54:53 crc kubenswrapper[5052]: I1203 11:54:53.505364 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 11:54:53 crc kubenswrapper[5052]: I1203 11:54:53.520925 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/placement-5fc4755558-qz76x"] Dec 03 11:54:53 crc kubenswrapper[5052]: I1203 11:54:53.546346 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/placement-5fc4755558-qz76x"] Dec 03 11:54:53 crc kubenswrapper[5052]: I1203 11:54:53.555233 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zq5wj\" (UniqueName: \"kubernetes.io/projected/53d7a9a2-b6a9-42dc-9828-d8ceaa70f0a3-kube-api-access-zq5wj\") pod \"53d7a9a2-b6a9-42dc-9828-d8ceaa70f0a3\" (UID: \"53d7a9a2-b6a9-42dc-9828-d8ceaa70f0a3\") " Dec 03 11:54:53 crc kubenswrapper[5052]: I1203 11:54:53.555433 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/53d7a9a2-b6a9-42dc-9828-d8ceaa70f0a3-operator-scripts\") pod \"53d7a9a2-b6a9-42dc-9828-d8ceaa70f0a3\" (UID: \"53d7a9a2-b6a9-42dc-9828-d8ceaa70f0a3\") " Dec 03 11:54:53 crc kubenswrapper[5052]: I1203 11:54:53.556798 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/53d7a9a2-b6a9-42dc-9828-d8ceaa70f0a3-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "53d7a9a2-b6a9-42dc-9828-d8ceaa70f0a3" (UID: "53d7a9a2-b6a9-42dc-9828-d8ceaa70f0a3"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:54:53 crc kubenswrapper[5052]: I1203 11:54:53.559456 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/53d7a9a2-b6a9-42dc-9828-d8ceaa70f0a3-kube-api-access-zq5wj" (OuterVolumeSpecName: "kube-api-access-zq5wj") pod "53d7a9a2-b6a9-42dc-9828-d8ceaa70f0a3" (UID: "53d7a9a2-b6a9-42dc-9828-d8ceaa70f0a3"). InnerVolumeSpecName "kube-api-access-zq5wj". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:54:53 crc kubenswrapper[5052]: I1203 11:54:53.615396 5052 scope.go:117] "RemoveContainer" containerID="13ed5d128b8c9df19939f5f214d29b30d31dc1296f2560559e840f9d3ffbfd12" Dec 03 11:54:53 crc kubenswrapper[5052]: I1203 11:54:53.655395 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/memcached-0"] Dec 03 11:54:53 crc kubenswrapper[5052]: I1203 11:54:53.657288 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/53d7a9a2-b6a9-42dc-9828-d8ceaa70f0a3-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:53 crc kubenswrapper[5052]: I1203 11:54:53.657419 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-zq5wj\" (UniqueName: \"kubernetes.io/projected/53d7a9a2-b6a9-42dc-9828-d8ceaa70f0a3-kube-api-access-zq5wj\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:53 crc kubenswrapper[5052]: I1203 11:54:53.666145 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/memcached-0"] Dec 03 11:54:53 crc kubenswrapper[5052]: I1203 11:54:53.671758 5052 scope.go:117] "RemoveContainer" containerID="d7f3ac6af5a2c0d33f118ffd9646c754d018fd85708d55c58a60a5cc00574cd1" Dec 03 11:54:53 crc kubenswrapper[5052]: I1203 11:54:53.692318 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="036bc2b0-bc50-4c91-bea5-882b9e592e4f" path="/var/lib/kubelet/pods/036bc2b0-bc50-4c91-bea5-882b9e592e4f/volumes" Dec 03 11:54:53 crc kubenswrapper[5052]: I1203 11:54:53.693594 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="428e9203-aa9c-48d3-84c9-4dfaa806140d" path="/var/lib/kubelet/pods/428e9203-aa9c-48d3-84c9-4dfaa806140d/volumes" Dec 03 11:54:53 crc kubenswrapper[5052]: I1203 11:54:53.694366 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="53a152d4-c58c-40a7-b4ce-b9a9dc42337d" path="/var/lib/kubelet/pods/53a152d4-c58c-40a7-b4ce-b9a9dc42337d/volumes" Dec 03 11:54:53 crc kubenswrapper[5052]: I1203 11:54:53.695709 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="584deaba-f945-42a2-8c0d-e531d6f48fd8" path="/var/lib/kubelet/pods/584deaba-f945-42a2-8c0d-e531d6f48fd8/volumes" Dec 03 11:54:53 crc kubenswrapper[5052]: I1203 11:54:53.696415 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="80d855f5-afda-46c0-9d72-01dee8dd93fb" path="/var/lib/kubelet/pods/80d855f5-afda-46c0-9d72-01dee8dd93fb/volumes" Dec 03 11:54:53 crc kubenswrapper[5052]: I1203 11:54:53.697052 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="88e7db65-8253-474b-b115-cd3f6edaf85c" path="/var/lib/kubelet/pods/88e7db65-8253-474b-b115-cd3f6edaf85c/volumes" Dec 03 11:54:53 crc kubenswrapper[5052]: I1203 11:54:53.698525 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8dd63158-2c75-455f-8f58-57a55c7e0f22" path="/var/lib/kubelet/pods/8dd63158-2c75-455f-8f58-57a55c7e0f22/volumes" Dec 03 11:54:53 crc kubenswrapper[5052]: I1203 11:54:53.699262 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9a4b1245-d95c-4eb6-8d2b-fdd93d98bb1e" path="/var/lib/kubelet/pods/9a4b1245-d95c-4eb6-8d2b-fdd93d98bb1e/volumes" Dec 03 11:54:53 crc kubenswrapper[5052]: I1203 11:54:53.700148 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a4aab9e3-5790-4143-948b-e24a52e552a5" path="/var/lib/kubelet/pods/a4aab9e3-5790-4143-948b-e24a52e552a5/volumes" Dec 03 11:54:53 crc kubenswrapper[5052]: I1203 11:54:53.700411 5052 generic.go:334] "Generic (PLEG): container finished" podID="10e3094e-f625-4d69-b0e9-358220c0d195" containerID="db5a765ca282bd8e9880ef9379b02a9202ec6dc773b029eb3abdd3d2f5581f14" exitCode=0 Dec 03 11:54:53 crc kubenswrapper[5052]: I1203 11:54:53.701348 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c1c20a0e-ee8a-416e-b739-05f16a536cfa" path="/var/lib/kubelet/pods/c1c20a0e-ee8a-416e-b739-05f16a536cfa/volumes" Dec 03 11:54:53 crc kubenswrapper[5052]: I1203 11:54:53.702155 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder6fd6-account-delete-5z97w" Dec 03 11:54:53 crc kubenswrapper[5052]: I1203 11:54:53.702164 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f32095c7-4e6f-43c7-a16f-513e89f7c28e" path="/var/lib/kubelet/pods/f32095c7-4e6f-43c7-a16f-513e89f7c28e/volumes" Dec 03 11:54:53 crc kubenswrapper[5052]: I1203 11:54:53.703066 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fa84c362-f076-4b47-8d2b-9eadd3f2d10c" path="/var/lib/kubelet/pods/fa84c362-f076-4b47-8d2b-9eadd3f2d10c/volumes" Dec 03 11:54:53 crc kubenswrapper[5052]: I1203 11:54:53.704814 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance634c-account-delete-nzwqr" Dec 03 11:54:53 crc kubenswrapper[5052]: I1203 11:54:53.706003 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 11:54:53 crc kubenswrapper[5052]: I1203 11:54:53.706044 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-server-0" event={"ID":"10e3094e-f625-4d69-b0e9-358220c0d195","Type":"ContainerDied","Data":"db5a765ca282bd8e9880ef9379b02a9202ec6dc773b029eb3abdd3d2f5581f14"} Dec 03 11:54:53 crc kubenswrapper[5052]: I1203 11:54:53.706078 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 11:54:53 crc kubenswrapper[5052]: I1203 11:54:53.706107 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder6fd6-account-delete-5z97w" event={"ID":"53d7a9a2-b6a9-42dc-9828-d8ceaa70f0a3","Type":"ContainerDied","Data":"98f873e1e36190437d014aac57c5d4a18331863c2203a7f032d5286ac1d56c80"} Dec 03 11:54:53 crc kubenswrapper[5052]: I1203 11:54:53.706130 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance634c-account-delete-nzwqr" event={"ID":"a73e0afd-e037-41f7-b214-d4eebd4bafe0","Type":"ContainerDied","Data":"ec815d3878e79f333506c388e7ff97c3234e71a84254f90b253c512d8fe9a121"} Dec 03 11:54:53 crc kubenswrapper[5052]: I1203 11:54:53.706144 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ec815d3878e79f333506c388e7ff97c3234e71a84254f90b253c512d8fe9a121" Dec 03 11:54:53 crc kubenswrapper[5052]: I1203 11:54:53.706158 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-keystone-listener-7bf89d7c74-c4srn"] Dec 03 11:54:53 crc kubenswrapper[5052]: I1203 11:54:53.720103 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/barbican-keystone-listener-7bf89d7c74-c4srn"] Dec 03 11:54:53 crc kubenswrapper[5052]: I1203 11:54:53.722091 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"d541d73d-2f95-43a8-8409-40f5f2ff54c4","Type":"ContainerDied","Data":"c8dfb8aa05e13b277498c99703ea5496b118e00d18f48f2fc71e83350fd07c6e"} Dec 03 11:54:53 crc kubenswrapper[5052]: I1203 11:54:53.722212 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:54:53 crc kubenswrapper[5052]: I1203 11:54:53.734262 5052 scope.go:117] "RemoveContainer" containerID="13ed5d128b8c9df19939f5f214d29b30d31dc1296f2560559e840f9d3ffbfd12" Dec 03 11:54:53 crc kubenswrapper[5052]: E1203 11:54:53.734654 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"13ed5d128b8c9df19939f5f214d29b30d31dc1296f2560559e840f9d3ffbfd12\": container with ID starting with 13ed5d128b8c9df19939f5f214d29b30d31dc1296f2560559e840f9d3ffbfd12 not found: ID does not exist" containerID="13ed5d128b8c9df19939f5f214d29b30d31dc1296f2560559e840f9d3ffbfd12" Dec 03 11:54:53 crc kubenswrapper[5052]: I1203 11:54:53.734690 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"13ed5d128b8c9df19939f5f214d29b30d31dc1296f2560559e840f9d3ffbfd12"} err="failed to get container status \"13ed5d128b8c9df19939f5f214d29b30d31dc1296f2560559e840f9d3ffbfd12\": rpc error: code = NotFound desc = could not find container \"13ed5d128b8c9df19939f5f214d29b30d31dc1296f2560559e840f9d3ffbfd12\": container with ID starting with 13ed5d128b8c9df19939f5f214d29b30d31dc1296f2560559e840f9d3ffbfd12 not found: ID does not exist" Dec 03 11:54:53 crc kubenswrapper[5052]: I1203 11:54:53.734716 5052 scope.go:117] "RemoveContainer" containerID="d7f3ac6af5a2c0d33f118ffd9646c754d018fd85708d55c58a60a5cc00574cd1" Dec 03 11:54:53 crc kubenswrapper[5052]: E1203 11:54:53.735972 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d7f3ac6af5a2c0d33f118ffd9646c754d018fd85708d55c58a60a5cc00574cd1\": container with ID starting with d7f3ac6af5a2c0d33f118ffd9646c754d018fd85708d55c58a60a5cc00574cd1 not found: ID does not exist" containerID="d7f3ac6af5a2c0d33f118ffd9646c754d018fd85708d55c58a60a5cc00574cd1" Dec 03 11:54:53 crc kubenswrapper[5052]: I1203 11:54:53.736004 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d7f3ac6af5a2c0d33f118ffd9646c754d018fd85708d55c58a60a5cc00574cd1"} err="failed to get container status \"d7f3ac6af5a2c0d33f118ffd9646c754d018fd85708d55c58a60a5cc00574cd1\": rpc error: code = NotFound desc = could not find container \"d7f3ac6af5a2c0d33f118ffd9646c754d018fd85708d55c58a60a5cc00574cd1\": container with ID starting with d7f3ac6af5a2c0d33f118ffd9646c754d018fd85708d55c58a60a5cc00574cd1 not found: ID does not exist" Dec 03 11:54:53 crc kubenswrapper[5052]: I1203 11:54:53.736025 5052 scope.go:117] "RemoveContainer" containerID="099b061b6cbe44d1e953f1c733f1f09b56e58f04c0ae2026319ceff46e418cf1" Dec 03 11:54:53 crc kubenswrapper[5052]: I1203 11:54:53.739001 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican4385-account-delete-krpzg" event={"ID":"383e7933-2bc4-4ead-bc34-0832091bd952","Type":"ContainerDied","Data":"22526e6c1f19a453ccca65f572d624a9eb9f5313506688f1a63bfc7853365f6a"} Dec 03 11:54:53 crc kubenswrapper[5052]: I1203 11:54:53.739044 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="22526e6c1f19a453ccca65f572d624a9eb9f5313506688f1a63bfc7853365f6a" Dec 03 11:54:53 crc kubenswrapper[5052]: I1203 11:54:53.739265 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican4385-account-delete-krpzg" Dec 03 11:54:53 crc kubenswrapper[5052]: I1203 11:54:53.745233 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"0201d941-cd05-4d1c-9af6-f3a82a375a0f","Type":"ContainerDied","Data":"a172d8e41b4536cc8c4a79909bc82ba0ecbe2bd22bde8680827a3a0781216413"} Dec 03 11:54:53 crc kubenswrapper[5052]: I1203 11:54:53.745274 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:54:53 crc kubenswrapper[5052]: I1203 11:54:53.747891 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"ec16e8c3-9750-4475-a671-17b0c16826c8","Type":"ContainerDied","Data":"61c38ece388590278bd72d985a321f75db09a50bb828355913a9b0bdfa05b5a8"} Dec 03 11:54:53 crc kubenswrapper[5052]: I1203 11:54:53.747953 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:54:53 crc kubenswrapper[5052]: I1203 11:54:53.751811 5052 generic.go:334] "Generic (PLEG): container finished" podID="775d2ccf-9b52-4274-99ef-2c33b18beaea" containerID="92f19f59b5c7604cf76509c11bcfe787a7d812a120d0641a971265397e22c941" exitCode=0 Dec 03 11:54:53 crc kubenswrapper[5052]: I1203 11:54:53.751895 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:54:53 crc kubenswrapper[5052]: I1203 11:54:53.752500 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-94f46848f-wgzvx" event={"ID":"775d2ccf-9b52-4274-99ef-2c33b18beaea","Type":"ContainerDied","Data":"92f19f59b5c7604cf76509c11bcfe787a7d812a120d0641a971265397e22c941"} Dec 03 11:54:53 crc kubenswrapper[5052]: I1203 11:54:53.752545 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/novaapie4da-account-delete-mknr9" podUID="6087dc7e-081b-4392-adb6-205b2aec805f" containerName="mariadb-account-delete" containerID="cri-o://162808060d27104600e8d3bf34956c7d1f29ffc6f699566d17f44cd664e8fc4c" gracePeriod=30 Dec 03 11:54:53 crc kubenswrapper[5052]: I1203 11:54:53.753069 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/novacell0745f-account-delete-zbhjv" podUID="93d03e32-f318-470c-ad5d-f17a609f5003" containerName="mariadb-account-delete" containerID="cri-o://75afa46f01077a531888f3be755bcaf1e48e735b4481e7ab69d40c172fb1751f" gracePeriod=30 Dec 03 11:54:53 crc kubenswrapper[5052]: I1203 11:54:53.753193 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-api-5758478fc6-5g46v" Dec 03 11:54:53 crc kubenswrapper[5052]: E1203 11:54:53.759219 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/rabbitmq-cell1-config-data: configmap "rabbitmq-cell1-config-data" not found Dec 03 11:54:53 crc kubenswrapper[5052]: E1203 11:54:53.759273 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a5ba4775-815a-46ef-bd2d-05dfa7513288-config-data podName:a5ba4775-815a-46ef-bd2d-05dfa7513288 nodeName:}" failed. No retries permitted until 2025-12-03 11:55:01.759258888 +0000 UTC m=+2092.347500652 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "config-data" (UniqueName: "kubernetes.io/configmap/a5ba4775-815a-46ef-bd2d-05dfa7513288-config-data") pod "rabbitmq-cell1-server-0" (UID: "a5ba4775-815a-46ef-bd2d-05dfa7513288") : configmap "rabbitmq-cell1-config-data" not found Dec 03 11:54:53 crc kubenswrapper[5052]: I1203 11:54:53.812781 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance634c-account-delete-nzwqr"] Dec 03 11:54:53 crc kubenswrapper[5052]: I1203 11:54:53.817629 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/glance634c-account-delete-nzwqr"] Dec 03 11:54:53 crc kubenswrapper[5052]: I1203 11:54:53.830377 5052 scope.go:117] "RemoveContainer" containerID="756be0d767606b67fd07bb5cf4a07ee4a5ec3d33b08c84185318683745d97e1b" Dec 03 11:54:53 crc kubenswrapper[5052]: I1203 11:54:53.840670 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder6fd6-account-delete-5z97w"] Dec 03 11:54:53 crc kubenswrapper[5052]: I1203 11:54:53.853460 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/cinder6fd6-account-delete-5z97w"] Dec 03 11:54:53 crc kubenswrapper[5052]: I1203 11:54:53.861867 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 11:54:53 crc kubenswrapper[5052]: I1203 11:54:53.870723 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 11:54:53 crc kubenswrapper[5052]: I1203 11:54:53.878465 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican4385-account-delete-krpzg"] Dec 03 11:54:53 crc kubenswrapper[5052]: I1203 11:54:53.894771 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/barbican4385-account-delete-krpzg"] Dec 03 11:54:53 crc kubenswrapper[5052]: I1203 11:54:53.897159 5052 scope.go:117] "RemoveContainer" containerID="3b60c38b2468b7d978dcdc73d52fcc2371995bf05e7490d1357efab444e714d0" Dec 03 11:54:53 crc kubenswrapper[5052]: I1203 11:54:53.902562 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 11:54:53 crc kubenswrapper[5052]: I1203 11:54:53.920912 5052 scope.go:117] "RemoveContainer" containerID="f0df1e8717d84a41bd64b8ee9a76553b85d16b020416be110d56d085ed4f7b45" Dec 03 11:54:53 crc kubenswrapper[5052]: I1203 11:54:53.929494 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 11:54:53 crc kubenswrapper[5052]: I1203 11:54:53.934986 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-api-5758478fc6-5g46v"] Dec 03 11:54:53 crc kubenswrapper[5052]: I1203 11:54:53.943004 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/barbican-api-5758478fc6-5g46v"] Dec 03 11:54:53 crc kubenswrapper[5052]: I1203 11:54:53.949130 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 11:54:53 crc kubenswrapper[5052]: I1203 11:54:53.954462 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 11:54:53 crc kubenswrapper[5052]: I1203 11:54:53.963966 5052 scope.go:117] "RemoveContainer" containerID="24b9c22c01a17699dc23692d82c8a6d2aba256d2fe1411900dc2de65cbd33d93" Dec 03 11:54:54 crc kubenswrapper[5052]: I1203 11:54:53.997643 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 11:54:54 crc kubenswrapper[5052]: I1203 11:54:54.005945 5052 scope.go:117] "RemoveContainer" containerID="21459936e0c2e5ac30de5ff0dac1297b633e3badacbcf9ad455e81ffb5671c96" Dec 03 11:54:54 crc kubenswrapper[5052]: I1203 11:54:54.008498 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 11:54:54 crc kubenswrapper[5052]: I1203 11:54:54.185575 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 11:54:54 crc kubenswrapper[5052]: I1203 11:54:54.268606 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"rabbitmq-confd\" (UniqueName: \"kubernetes.io/projected/10e3094e-f625-4d69-b0e9-358220c0d195-rabbitmq-confd\") pod \"10e3094e-f625-4d69-b0e9-358220c0d195\" (UID: \"10e3094e-f625-4d69-b0e9-358220c0d195\") " Dec 03 11:54:54 crc kubenswrapper[5052]: I1203 11:54:54.268644 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"plugins-conf\" (UniqueName: \"kubernetes.io/configmap/10e3094e-f625-4d69-b0e9-358220c0d195-plugins-conf\") pod \"10e3094e-f625-4d69-b0e9-358220c0d195\" (UID: \"10e3094e-f625-4d69-b0e9-358220c0d195\") " Dec 03 11:54:54 crc kubenswrapper[5052]: I1203 11:54:54.268677 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"rabbitmq-erlang-cookie\" (UniqueName: \"kubernetes.io/empty-dir/10e3094e-f625-4d69-b0e9-358220c0d195-rabbitmq-erlang-cookie\") pod \"10e3094e-f625-4d69-b0e9-358220c0d195\" (UID: \"10e3094e-f625-4d69-b0e9-358220c0d195\") " Dec 03 11:54:54 crc kubenswrapper[5052]: I1203 11:54:54.268711 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xsgxx\" (UniqueName: \"kubernetes.io/projected/10e3094e-f625-4d69-b0e9-358220c0d195-kube-api-access-xsgxx\") pod \"10e3094e-f625-4d69-b0e9-358220c0d195\" (UID: \"10e3094e-f625-4d69-b0e9-358220c0d195\") " Dec 03 11:54:54 crc kubenswrapper[5052]: I1203 11:54:54.268751 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"persistence\" (UniqueName: \"kubernetes.io/local-volume/local-storage08-crc\") pod \"10e3094e-f625-4d69-b0e9-358220c0d195\" (UID: \"10e3094e-f625-4d69-b0e9-358220c0d195\") " Dec 03 11:54:54 crc kubenswrapper[5052]: I1203 11:54:54.268833 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"server-conf\" (UniqueName: \"kubernetes.io/configmap/10e3094e-f625-4d69-b0e9-358220c0d195-server-conf\") pod \"10e3094e-f625-4d69-b0e9-358220c0d195\" (UID: \"10e3094e-f625-4d69-b0e9-358220c0d195\") " Dec 03 11:54:54 crc kubenswrapper[5052]: I1203 11:54:54.268857 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"erlang-cookie-secret\" (UniqueName: \"kubernetes.io/secret/10e3094e-f625-4d69-b0e9-358220c0d195-erlang-cookie-secret\") pod \"10e3094e-f625-4d69-b0e9-358220c0d195\" (UID: \"10e3094e-f625-4d69-b0e9-358220c0d195\") " Dec 03 11:54:54 crc kubenswrapper[5052]: I1203 11:54:54.268914 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"rabbitmq-plugins\" (UniqueName: \"kubernetes.io/empty-dir/10e3094e-f625-4d69-b0e9-358220c0d195-rabbitmq-plugins\") pod \"10e3094e-f625-4d69-b0e9-358220c0d195\" (UID: \"10e3094e-f625-4d69-b0e9-358220c0d195\") " Dec 03 11:54:54 crc kubenswrapper[5052]: I1203 11:54:54.268997 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"rabbitmq-tls\" (UniqueName: \"kubernetes.io/projected/10e3094e-f625-4d69-b0e9-358220c0d195-rabbitmq-tls\") pod \"10e3094e-f625-4d69-b0e9-358220c0d195\" (UID: \"10e3094e-f625-4d69-b0e9-358220c0d195\") " Dec 03 11:54:54 crc kubenswrapper[5052]: I1203 11:54:54.269038 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"pod-info\" (UniqueName: \"kubernetes.io/downward-api/10e3094e-f625-4d69-b0e9-358220c0d195-pod-info\") pod \"10e3094e-f625-4d69-b0e9-358220c0d195\" (UID: \"10e3094e-f625-4d69-b0e9-358220c0d195\") " Dec 03 11:54:54 crc kubenswrapper[5052]: I1203 11:54:54.269077 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/10e3094e-f625-4d69-b0e9-358220c0d195-config-data\") pod \"10e3094e-f625-4d69-b0e9-358220c0d195\" (UID: \"10e3094e-f625-4d69-b0e9-358220c0d195\") " Dec 03 11:54:54 crc kubenswrapper[5052]: E1203 11:54:54.269559 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 11:54:54 crc kubenswrapper[5052]: E1203 11:54:54.269621 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6087dc7e-081b-4392-adb6-205b2aec805f-operator-scripts podName:6087dc7e-081b-4392-adb6-205b2aec805f nodeName:}" failed. No retries permitted until 2025-12-03 11:54:56.26960487 +0000 UTC m=+2086.857846654 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/6087dc7e-081b-4392-adb6-205b2aec805f-operator-scripts") pod "novaapie4da-account-delete-mknr9" (UID: "6087dc7e-081b-4392-adb6-205b2aec805f") : configmap "openstack-scripts" not found Dec 03 11:54:54 crc kubenswrapper[5052]: I1203 11:54:54.271647 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/10e3094e-f625-4d69-b0e9-358220c0d195-rabbitmq-erlang-cookie" (OuterVolumeSpecName: "rabbitmq-erlang-cookie") pod "10e3094e-f625-4d69-b0e9-358220c0d195" (UID: "10e3094e-f625-4d69-b0e9-358220c0d195"). InnerVolumeSpecName "rabbitmq-erlang-cookie". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:54:54 crc kubenswrapper[5052]: E1203 11:54:54.273072 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 11:54:54 crc kubenswrapper[5052]: E1203 11:54:54.273151 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/93d03e32-f318-470c-ad5d-f17a609f5003-operator-scripts podName:93d03e32-f318-470c-ad5d-f17a609f5003 nodeName:}" failed. No retries permitted until 2025-12-03 11:54:56.273130731 +0000 UTC m=+2086.861372495 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/93d03e32-f318-470c-ad5d-f17a609f5003-operator-scripts") pod "novacell0745f-account-delete-zbhjv" (UID: "93d03e32-f318-470c-ad5d-f17a609f5003") : configmap "openstack-scripts" not found Dec 03 11:54:54 crc kubenswrapper[5052]: I1203 11:54:54.274935 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/10e3094e-f625-4d69-b0e9-358220c0d195-erlang-cookie-secret" (OuterVolumeSpecName: "erlang-cookie-secret") pod "10e3094e-f625-4d69-b0e9-358220c0d195" (UID: "10e3094e-f625-4d69-b0e9-358220c0d195"). InnerVolumeSpecName "erlang-cookie-secret". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:54:54 crc kubenswrapper[5052]: I1203 11:54:54.275908 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/10e3094e-f625-4d69-b0e9-358220c0d195-plugins-conf" (OuterVolumeSpecName: "plugins-conf") pod "10e3094e-f625-4d69-b0e9-358220c0d195" (UID: "10e3094e-f625-4d69-b0e9-358220c0d195"). InnerVolumeSpecName "plugins-conf". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:54:54 crc kubenswrapper[5052]: I1203 11:54:54.275978 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/10e3094e-f625-4d69-b0e9-358220c0d195-rabbitmq-plugins" (OuterVolumeSpecName: "rabbitmq-plugins") pod "10e3094e-f625-4d69-b0e9-358220c0d195" (UID: "10e3094e-f625-4d69-b0e9-358220c0d195"). InnerVolumeSpecName "rabbitmq-plugins". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:54:54 crc kubenswrapper[5052]: I1203 11:54:54.275983 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/10e3094e-f625-4d69-b0e9-358220c0d195-pod-info" (OuterVolumeSpecName: "pod-info") pod "10e3094e-f625-4d69-b0e9-358220c0d195" (UID: "10e3094e-f625-4d69-b0e9-358220c0d195"). InnerVolumeSpecName "pod-info". PluginName "kubernetes.io/downward-api", VolumeGidValue "" Dec 03 11:54:54 crc kubenswrapper[5052]: I1203 11:54:54.277792 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage08-crc" (OuterVolumeSpecName: "persistence") pod "10e3094e-f625-4d69-b0e9-358220c0d195" (UID: "10e3094e-f625-4d69-b0e9-358220c0d195"). InnerVolumeSpecName "local-storage08-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 11:54:54 crc kubenswrapper[5052]: I1203 11:54:54.278035 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/10e3094e-f625-4d69-b0e9-358220c0d195-kube-api-access-xsgxx" (OuterVolumeSpecName: "kube-api-access-xsgxx") pod "10e3094e-f625-4d69-b0e9-358220c0d195" (UID: "10e3094e-f625-4d69-b0e9-358220c0d195"). InnerVolumeSpecName "kube-api-access-xsgxx". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:54:54 crc kubenswrapper[5052]: I1203 11:54:54.279141 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/10e3094e-f625-4d69-b0e9-358220c0d195-rabbitmq-tls" (OuterVolumeSpecName: "rabbitmq-tls") pod "10e3094e-f625-4d69-b0e9-358220c0d195" (UID: "10e3094e-f625-4d69-b0e9-358220c0d195"). InnerVolumeSpecName "rabbitmq-tls". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:54:54 crc kubenswrapper[5052]: I1203 11:54:54.324529 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/10e3094e-f625-4d69-b0e9-358220c0d195-server-conf" (OuterVolumeSpecName: "server-conf") pod "10e3094e-f625-4d69-b0e9-358220c0d195" (UID: "10e3094e-f625-4d69-b0e9-358220c0d195"). InnerVolumeSpecName "server-conf". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:54:54 crc kubenswrapper[5052]: I1203 11:54:54.335086 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/10e3094e-f625-4d69-b0e9-358220c0d195-config-data" (OuterVolumeSpecName: "config-data") pod "10e3094e-f625-4d69-b0e9-358220c0d195" (UID: "10e3094e-f625-4d69-b0e9-358220c0d195"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:54:54 crc kubenswrapper[5052]: I1203 11:54:54.353568 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/10e3094e-f625-4d69-b0e9-358220c0d195-rabbitmq-confd" (OuterVolumeSpecName: "rabbitmq-confd") pod "10e3094e-f625-4d69-b0e9-358220c0d195" (UID: "10e3094e-f625-4d69-b0e9-358220c0d195"). InnerVolumeSpecName "rabbitmq-confd". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:54:54 crc kubenswrapper[5052]: I1203 11:54:54.373949 5052 reconciler_common.go:293] "Volume detached for volume \"rabbitmq-tls\" (UniqueName: \"kubernetes.io/projected/10e3094e-f625-4d69-b0e9-358220c0d195-rabbitmq-tls\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:54 crc kubenswrapper[5052]: I1203 11:54:54.374006 5052 reconciler_common.go:293] "Volume detached for volume \"pod-info\" (UniqueName: \"kubernetes.io/downward-api/10e3094e-f625-4d69-b0e9-358220c0d195-pod-info\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:54 crc kubenswrapper[5052]: I1203 11:54:54.374018 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/10e3094e-f625-4d69-b0e9-358220c0d195-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:54 crc kubenswrapper[5052]: I1203 11:54:54.374027 5052 reconciler_common.go:293] "Volume detached for volume \"rabbitmq-confd\" (UniqueName: \"kubernetes.io/projected/10e3094e-f625-4d69-b0e9-358220c0d195-rabbitmq-confd\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:54 crc kubenswrapper[5052]: I1203 11:54:54.374036 5052 reconciler_common.go:293] "Volume detached for volume \"plugins-conf\" (UniqueName: \"kubernetes.io/configmap/10e3094e-f625-4d69-b0e9-358220c0d195-plugins-conf\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:54 crc kubenswrapper[5052]: I1203 11:54:54.374061 5052 reconciler_common.go:293] "Volume detached for volume \"rabbitmq-erlang-cookie\" (UniqueName: \"kubernetes.io/empty-dir/10e3094e-f625-4d69-b0e9-358220c0d195-rabbitmq-erlang-cookie\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:54 crc kubenswrapper[5052]: I1203 11:54:54.374071 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-xsgxx\" (UniqueName: \"kubernetes.io/projected/10e3094e-f625-4d69-b0e9-358220c0d195-kube-api-access-xsgxx\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:54 crc kubenswrapper[5052]: I1203 11:54:54.374106 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage08-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage08-crc\") on node \"crc\" " Dec 03 11:54:54 crc kubenswrapper[5052]: I1203 11:54:54.374146 5052 reconciler_common.go:293] "Volume detached for volume \"server-conf\" (UniqueName: \"kubernetes.io/configmap/10e3094e-f625-4d69-b0e9-358220c0d195-server-conf\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:54 crc kubenswrapper[5052]: I1203 11:54:54.374156 5052 reconciler_common.go:293] "Volume detached for volume \"erlang-cookie-secret\" (UniqueName: \"kubernetes.io/secret/10e3094e-f625-4d69-b0e9-358220c0d195-erlang-cookie-secret\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:54 crc kubenswrapper[5052]: I1203 11:54:54.374165 5052 reconciler_common.go:293] "Volume detached for volume \"rabbitmq-plugins\" (UniqueName: \"kubernetes.io/empty-dir/10e3094e-f625-4d69-b0e9-358220c0d195-rabbitmq-plugins\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:54 crc kubenswrapper[5052]: I1203 11:54:54.402043 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage08-crc" (UniqueName: "kubernetes.io/local-volume/local-storage08-crc") on node "crc" Dec 03 11:54:54 crc kubenswrapper[5052]: I1203 11:54:54.475728 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage08-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage08-crc\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:54 crc kubenswrapper[5052]: I1203 11:54:54.481364 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-94f46848f-wgzvx" Dec 03 11:54:54 crc kubenswrapper[5052]: I1203 11:54:54.570391 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 11:54:54 crc kubenswrapper[5052]: I1203 11:54:54.578341 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/775d2ccf-9b52-4274-99ef-2c33b18beaea-config-data\") pod \"775d2ccf-9b52-4274-99ef-2c33b18beaea\" (UID: \"775d2ccf-9b52-4274-99ef-2c33b18beaea\") " Dec 03 11:54:54 crc kubenswrapper[5052]: I1203 11:54:54.578468 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/775d2ccf-9b52-4274-99ef-2c33b18beaea-combined-ca-bundle\") pod \"775d2ccf-9b52-4274-99ef-2c33b18beaea\" (UID: \"775d2ccf-9b52-4274-99ef-2c33b18beaea\") " Dec 03 11:54:54 crc kubenswrapper[5052]: I1203 11:54:54.578631 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/775d2ccf-9b52-4274-99ef-2c33b18beaea-credential-keys\") pod \"775d2ccf-9b52-4274-99ef-2c33b18beaea\" (UID: \"775d2ccf-9b52-4274-99ef-2c33b18beaea\") " Dec 03 11:54:54 crc kubenswrapper[5052]: I1203 11:54:54.578697 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c69gm\" (UniqueName: \"kubernetes.io/projected/775d2ccf-9b52-4274-99ef-2c33b18beaea-kube-api-access-c69gm\") pod \"775d2ccf-9b52-4274-99ef-2c33b18beaea\" (UID: \"775d2ccf-9b52-4274-99ef-2c33b18beaea\") " Dec 03 11:54:54 crc kubenswrapper[5052]: I1203 11:54:54.578744 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/775d2ccf-9b52-4274-99ef-2c33b18beaea-public-tls-certs\") pod \"775d2ccf-9b52-4274-99ef-2c33b18beaea\" (UID: \"775d2ccf-9b52-4274-99ef-2c33b18beaea\") " Dec 03 11:54:54 crc kubenswrapper[5052]: I1203 11:54:54.578825 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/775d2ccf-9b52-4274-99ef-2c33b18beaea-fernet-keys\") pod \"775d2ccf-9b52-4274-99ef-2c33b18beaea\" (UID: \"775d2ccf-9b52-4274-99ef-2c33b18beaea\") " Dec 03 11:54:54 crc kubenswrapper[5052]: I1203 11:54:54.578888 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/775d2ccf-9b52-4274-99ef-2c33b18beaea-internal-tls-certs\") pod \"775d2ccf-9b52-4274-99ef-2c33b18beaea\" (UID: \"775d2ccf-9b52-4274-99ef-2c33b18beaea\") " Dec 03 11:54:54 crc kubenswrapper[5052]: I1203 11:54:54.578964 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/775d2ccf-9b52-4274-99ef-2c33b18beaea-scripts\") pod \"775d2ccf-9b52-4274-99ef-2c33b18beaea\" (UID: \"775d2ccf-9b52-4274-99ef-2c33b18beaea\") " Dec 03 11:54:54 crc kubenswrapper[5052]: I1203 11:54:54.584398 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/775d2ccf-9b52-4274-99ef-2c33b18beaea-scripts" (OuterVolumeSpecName: "scripts") pod "775d2ccf-9b52-4274-99ef-2c33b18beaea" (UID: "775d2ccf-9b52-4274-99ef-2c33b18beaea"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:54:54 crc kubenswrapper[5052]: I1203 11:54:54.589586 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/775d2ccf-9b52-4274-99ef-2c33b18beaea-fernet-keys" (OuterVolumeSpecName: "fernet-keys") pod "775d2ccf-9b52-4274-99ef-2c33b18beaea" (UID: "775d2ccf-9b52-4274-99ef-2c33b18beaea"). InnerVolumeSpecName "fernet-keys". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:54:54 crc kubenswrapper[5052]: I1203 11:54:54.601120 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/775d2ccf-9b52-4274-99ef-2c33b18beaea-kube-api-access-c69gm" (OuterVolumeSpecName: "kube-api-access-c69gm") pod "775d2ccf-9b52-4274-99ef-2c33b18beaea" (UID: "775d2ccf-9b52-4274-99ef-2c33b18beaea"). InnerVolumeSpecName "kube-api-access-c69gm". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:54:54 crc kubenswrapper[5052]: I1203 11:54:54.604885 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/775d2ccf-9b52-4274-99ef-2c33b18beaea-credential-keys" (OuterVolumeSpecName: "credential-keys") pod "775d2ccf-9b52-4274-99ef-2c33b18beaea" (UID: "775d2ccf-9b52-4274-99ef-2c33b18beaea"). InnerVolumeSpecName "credential-keys". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:54:54 crc kubenswrapper[5052]: I1203 11:54:54.611784 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/775d2ccf-9b52-4274-99ef-2c33b18beaea-config-data" (OuterVolumeSpecName: "config-data") pod "775d2ccf-9b52-4274-99ef-2c33b18beaea" (UID: "775d2ccf-9b52-4274-99ef-2c33b18beaea"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:54:54 crc kubenswrapper[5052]: I1203 11:54:54.642224 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/775d2ccf-9b52-4274-99ef-2c33b18beaea-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "775d2ccf-9b52-4274-99ef-2c33b18beaea" (UID: "775d2ccf-9b52-4274-99ef-2c33b18beaea"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:54:54 crc kubenswrapper[5052]: I1203 11:54:54.643626 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/775d2ccf-9b52-4274-99ef-2c33b18beaea-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "775d2ccf-9b52-4274-99ef-2c33b18beaea" (UID: "775d2ccf-9b52-4274-99ef-2c33b18beaea"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:54:54 crc kubenswrapper[5052]: I1203 11:54:54.645707 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/775d2ccf-9b52-4274-99ef-2c33b18beaea-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "775d2ccf-9b52-4274-99ef-2c33b18beaea" (UID: "775d2ccf-9b52-4274-99ef-2c33b18beaea"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:54:54 crc kubenswrapper[5052]: I1203 11:54:54.680993 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"rabbitmq-plugins\" (UniqueName: \"kubernetes.io/empty-dir/a5ba4775-815a-46ef-bd2d-05dfa7513288-rabbitmq-plugins\") pod \"a5ba4775-815a-46ef-bd2d-05dfa7513288\" (UID: \"a5ba4775-815a-46ef-bd2d-05dfa7513288\") " Dec 03 11:54:54 crc kubenswrapper[5052]: I1203 11:54:54.681031 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"erlang-cookie-secret\" (UniqueName: \"kubernetes.io/secret/a5ba4775-815a-46ef-bd2d-05dfa7513288-erlang-cookie-secret\") pod \"a5ba4775-815a-46ef-bd2d-05dfa7513288\" (UID: \"a5ba4775-815a-46ef-bd2d-05dfa7513288\") " Dec 03 11:54:54 crc kubenswrapper[5052]: I1203 11:54:54.681074 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"persistence\" (UniqueName: \"kubernetes.io/local-volume/local-storage11-crc\") pod \"a5ba4775-815a-46ef-bd2d-05dfa7513288\" (UID: \"a5ba4775-815a-46ef-bd2d-05dfa7513288\") " Dec 03 11:54:54 crc kubenswrapper[5052]: I1203 11:54:54.681122 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"plugins-conf\" (UniqueName: \"kubernetes.io/configmap/a5ba4775-815a-46ef-bd2d-05dfa7513288-plugins-conf\") pod \"a5ba4775-815a-46ef-bd2d-05dfa7513288\" (UID: \"a5ba4775-815a-46ef-bd2d-05dfa7513288\") " Dec 03 11:54:54 crc kubenswrapper[5052]: I1203 11:54:54.681177 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"server-conf\" (UniqueName: \"kubernetes.io/configmap/a5ba4775-815a-46ef-bd2d-05dfa7513288-server-conf\") pod \"a5ba4775-815a-46ef-bd2d-05dfa7513288\" (UID: \"a5ba4775-815a-46ef-bd2d-05dfa7513288\") " Dec 03 11:54:54 crc kubenswrapper[5052]: I1203 11:54:54.681225 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"rabbitmq-tls\" (UniqueName: \"kubernetes.io/projected/a5ba4775-815a-46ef-bd2d-05dfa7513288-rabbitmq-tls\") pod \"a5ba4775-815a-46ef-bd2d-05dfa7513288\" (UID: \"a5ba4775-815a-46ef-bd2d-05dfa7513288\") " Dec 03 11:54:54 crc kubenswrapper[5052]: I1203 11:54:54.681263 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"pod-info\" (UniqueName: \"kubernetes.io/downward-api/a5ba4775-815a-46ef-bd2d-05dfa7513288-pod-info\") pod \"a5ba4775-815a-46ef-bd2d-05dfa7513288\" (UID: \"a5ba4775-815a-46ef-bd2d-05dfa7513288\") " Dec 03 11:54:54 crc kubenswrapper[5052]: I1203 11:54:54.681320 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/a5ba4775-815a-46ef-bd2d-05dfa7513288-config-data\") pod \"a5ba4775-815a-46ef-bd2d-05dfa7513288\" (UID: \"a5ba4775-815a-46ef-bd2d-05dfa7513288\") " Dec 03 11:54:54 crc kubenswrapper[5052]: I1203 11:54:54.681349 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"rabbitmq-erlang-cookie\" (UniqueName: \"kubernetes.io/empty-dir/a5ba4775-815a-46ef-bd2d-05dfa7513288-rabbitmq-erlang-cookie\") pod \"a5ba4775-815a-46ef-bd2d-05dfa7513288\" (UID: \"a5ba4775-815a-46ef-bd2d-05dfa7513288\") " Dec 03 11:54:54 crc kubenswrapper[5052]: I1203 11:54:54.681374 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"rabbitmq-confd\" (UniqueName: \"kubernetes.io/projected/a5ba4775-815a-46ef-bd2d-05dfa7513288-rabbitmq-confd\") pod \"a5ba4775-815a-46ef-bd2d-05dfa7513288\" (UID: \"a5ba4775-815a-46ef-bd2d-05dfa7513288\") " Dec 03 11:54:54 crc kubenswrapper[5052]: I1203 11:54:54.681438 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xbr2g\" (UniqueName: \"kubernetes.io/projected/a5ba4775-815a-46ef-bd2d-05dfa7513288-kube-api-access-xbr2g\") pod \"a5ba4775-815a-46ef-bd2d-05dfa7513288\" (UID: \"a5ba4775-815a-46ef-bd2d-05dfa7513288\") " Dec 03 11:54:54 crc kubenswrapper[5052]: I1203 11:54:54.681752 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/775d2ccf-9b52-4274-99ef-2c33b18beaea-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:54 crc kubenswrapper[5052]: I1203 11:54:54.681768 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/775d2ccf-9b52-4274-99ef-2c33b18beaea-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:54 crc kubenswrapper[5052]: I1203 11:54:54.681778 5052 reconciler_common.go:293] "Volume detached for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/775d2ccf-9b52-4274-99ef-2c33b18beaea-credential-keys\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:54 crc kubenswrapper[5052]: I1203 11:54:54.681787 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-c69gm\" (UniqueName: \"kubernetes.io/projected/775d2ccf-9b52-4274-99ef-2c33b18beaea-kube-api-access-c69gm\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:54 crc kubenswrapper[5052]: I1203 11:54:54.681795 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/775d2ccf-9b52-4274-99ef-2c33b18beaea-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:54 crc kubenswrapper[5052]: I1203 11:54:54.681803 5052 reconciler_common.go:293] "Volume detached for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/775d2ccf-9b52-4274-99ef-2c33b18beaea-fernet-keys\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:54 crc kubenswrapper[5052]: I1203 11:54:54.681811 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/775d2ccf-9b52-4274-99ef-2c33b18beaea-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:54 crc kubenswrapper[5052]: I1203 11:54:54.681819 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/775d2ccf-9b52-4274-99ef-2c33b18beaea-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:54 crc kubenswrapper[5052]: I1203 11:54:54.681880 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5ba4775-815a-46ef-bd2d-05dfa7513288-rabbitmq-plugins" (OuterVolumeSpecName: "rabbitmq-plugins") pod "a5ba4775-815a-46ef-bd2d-05dfa7513288" (UID: "a5ba4775-815a-46ef-bd2d-05dfa7513288"). InnerVolumeSpecName "rabbitmq-plugins". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:54:54 crc kubenswrapper[5052]: I1203 11:54:54.682445 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a5ba4775-815a-46ef-bd2d-05dfa7513288-plugins-conf" (OuterVolumeSpecName: "plugins-conf") pod "a5ba4775-815a-46ef-bd2d-05dfa7513288" (UID: "a5ba4775-815a-46ef-bd2d-05dfa7513288"). InnerVolumeSpecName "plugins-conf". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:54:54 crc kubenswrapper[5052]: I1203 11:54:54.683031 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5ba4775-815a-46ef-bd2d-05dfa7513288-rabbitmq-erlang-cookie" (OuterVolumeSpecName: "rabbitmq-erlang-cookie") pod "a5ba4775-815a-46ef-bd2d-05dfa7513288" (UID: "a5ba4775-815a-46ef-bd2d-05dfa7513288"). InnerVolumeSpecName "rabbitmq-erlang-cookie". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:54:54 crc kubenswrapper[5052]: I1203 11:54:54.685974 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a5ba4775-815a-46ef-bd2d-05dfa7513288-rabbitmq-tls" (OuterVolumeSpecName: "rabbitmq-tls") pod "a5ba4775-815a-46ef-bd2d-05dfa7513288" (UID: "a5ba4775-815a-46ef-bd2d-05dfa7513288"). InnerVolumeSpecName "rabbitmq-tls". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:54:54 crc kubenswrapper[5052]: I1203 11:54:54.686356 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage11-crc" (OuterVolumeSpecName: "persistence") pod "a5ba4775-815a-46ef-bd2d-05dfa7513288" (UID: "a5ba4775-815a-46ef-bd2d-05dfa7513288"). InnerVolumeSpecName "local-storage11-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 11:54:54 crc kubenswrapper[5052]: I1203 11:54:54.686387 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 11:54:54 crc kubenswrapper[5052]: I1203 11:54:54.686575 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a5ba4775-815a-46ef-bd2d-05dfa7513288-pod-info" (OuterVolumeSpecName: "pod-info") pod "a5ba4775-815a-46ef-bd2d-05dfa7513288" (UID: "a5ba4775-815a-46ef-bd2d-05dfa7513288"). InnerVolumeSpecName "pod-info". PluginName "kubernetes.io/downward-api", VolumeGidValue "" Dec 03 11:54:54 crc kubenswrapper[5052]: I1203 11:54:54.686872 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a5ba4775-815a-46ef-bd2d-05dfa7513288-erlang-cookie-secret" (OuterVolumeSpecName: "erlang-cookie-secret") pod "a5ba4775-815a-46ef-bd2d-05dfa7513288" (UID: "a5ba4775-815a-46ef-bd2d-05dfa7513288"). InnerVolumeSpecName "erlang-cookie-secret". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:54:54 crc kubenswrapper[5052]: I1203 11:54:54.687396 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a5ba4775-815a-46ef-bd2d-05dfa7513288-kube-api-access-xbr2g" (OuterVolumeSpecName: "kube-api-access-xbr2g") pod "a5ba4775-815a-46ef-bd2d-05dfa7513288" (UID: "a5ba4775-815a-46ef-bd2d-05dfa7513288"). InnerVolumeSpecName "kube-api-access-xbr2g". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:54:54 crc kubenswrapper[5052]: I1203 11:54:54.716770 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a5ba4775-815a-46ef-bd2d-05dfa7513288-config-data" (OuterVolumeSpecName: "config-data") pod "a5ba4775-815a-46ef-bd2d-05dfa7513288" (UID: "a5ba4775-815a-46ef-bd2d-05dfa7513288"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:54:54 crc kubenswrapper[5052]: I1203 11:54:54.740502 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a5ba4775-815a-46ef-bd2d-05dfa7513288-server-conf" (OuterVolumeSpecName: "server-conf") pod "a5ba4775-815a-46ef-bd2d-05dfa7513288" (UID: "a5ba4775-815a-46ef-bd2d-05dfa7513288"). InnerVolumeSpecName "server-conf". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:54:54 crc kubenswrapper[5052]: I1203 11:54:54.763950 5052 generic.go:334] "Generic (PLEG): container finished" podID="a5ff0249-173a-4793-8cbe-0aa749f34632" containerID="cf164870d3f804d61f70d5e02392ceff8cd44fdf830fb056d8e5f574d1ef05c1" exitCode=0 Dec 03 11:54:54 crc kubenswrapper[5052]: I1203 11:54:54.764009 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"a5ff0249-173a-4793-8cbe-0aa749f34632","Type":"ContainerDied","Data":"cf164870d3f804d61f70d5e02392ceff8cd44fdf830fb056d8e5f574d1ef05c1"} Dec 03 11:54:54 crc kubenswrapper[5052]: I1203 11:54:54.765407 5052 generic.go:334] "Generic (PLEG): container finished" podID="a5ba4775-815a-46ef-bd2d-05dfa7513288" containerID="c25fbe13b5143561b88ab53fac63216ac379a26107913a3a532fbd7033bbeba6" exitCode=0 Dec 03 11:54:54 crc kubenswrapper[5052]: I1203 11:54:54.765445 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" event={"ID":"a5ba4775-815a-46ef-bd2d-05dfa7513288","Type":"ContainerDied","Data":"c25fbe13b5143561b88ab53fac63216ac379a26107913a3a532fbd7033bbeba6"} Dec 03 11:54:54 crc kubenswrapper[5052]: I1203 11:54:54.765463 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" event={"ID":"a5ba4775-815a-46ef-bd2d-05dfa7513288","Type":"ContainerDied","Data":"9cddb4e8af380ee07414ad1c06f823e0dea2c6ce693e4ed3a60ef4221d3e87e4"} Dec 03 11:54:54 crc kubenswrapper[5052]: I1203 11:54:54.765479 5052 scope.go:117] "RemoveContainer" containerID="c25fbe13b5143561b88ab53fac63216ac379a26107913a3a532fbd7033bbeba6" Dec 03 11:54:54 crc kubenswrapper[5052]: I1203 11:54:54.765640 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 11:54:54 crc kubenswrapper[5052]: I1203 11:54:54.773243 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-server-0" event={"ID":"10e3094e-f625-4d69-b0e9-358220c0d195","Type":"ContainerDied","Data":"551a3ead523383a5c92a144136bec5a30d11a6644fcc4ed6012c53179ab67346"} Dec 03 11:54:54 crc kubenswrapper[5052]: I1203 11:54:54.773500 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 11:54:54 crc kubenswrapper[5052]: I1203 11:54:54.782445 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f856a02f-3f21-4fd4-8158-50a232de756e-combined-ca-bundle\") pod \"f856a02f-3f21-4fd4-8158-50a232de756e\" (UID: \"f856a02f-3f21-4fd4-8158-50a232de756e\") " Dec 03 11:54:54 crc kubenswrapper[5052]: I1203 11:54:54.782485 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9bf6j\" (UniqueName: \"kubernetes.io/projected/f856a02f-3f21-4fd4-8158-50a232de756e-kube-api-access-9bf6j\") pod \"f856a02f-3f21-4fd4-8158-50a232de756e\" (UID: \"f856a02f-3f21-4fd4-8158-50a232de756e\") " Dec 03 11:54:54 crc kubenswrapper[5052]: I1203 11:54:54.782525 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/f856a02f-3f21-4fd4-8158-50a232de756e-galera-tls-certs\") pod \"f856a02f-3f21-4fd4-8158-50a232de756e\" (UID: \"f856a02f-3f21-4fd4-8158-50a232de756e\") " Dec 03 11:54:54 crc kubenswrapper[5052]: I1203 11:54:54.782554 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"mysql-db\" (UniqueName: \"kubernetes.io/local-volume/local-storage03-crc\") pod \"f856a02f-3f21-4fd4-8158-50a232de756e\" (UID: \"f856a02f-3f21-4fd4-8158-50a232de756e\") " Dec 03 11:54:54 crc kubenswrapper[5052]: I1203 11:54:54.782671 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/f856a02f-3f21-4fd4-8158-50a232de756e-config-data-generated\") pod \"f856a02f-3f21-4fd4-8158-50a232de756e\" (UID: \"f856a02f-3f21-4fd4-8158-50a232de756e\") " Dec 03 11:54:54 crc kubenswrapper[5052]: I1203 11:54:54.782692 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/f856a02f-3f21-4fd4-8158-50a232de756e-operator-scripts\") pod \"f856a02f-3f21-4fd4-8158-50a232de756e\" (UID: \"f856a02f-3f21-4fd4-8158-50a232de756e\") " Dec 03 11:54:54 crc kubenswrapper[5052]: I1203 11:54:54.782724 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/f856a02f-3f21-4fd4-8158-50a232de756e-kolla-config\") pod \"f856a02f-3f21-4fd4-8158-50a232de756e\" (UID: \"f856a02f-3f21-4fd4-8158-50a232de756e\") " Dec 03 11:54:54 crc kubenswrapper[5052]: I1203 11:54:54.782788 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/f856a02f-3f21-4fd4-8158-50a232de756e-config-data-default\") pod \"f856a02f-3f21-4fd4-8158-50a232de756e\" (UID: \"f856a02f-3f21-4fd4-8158-50a232de756e\") " Dec 03 11:54:54 crc kubenswrapper[5052]: I1203 11:54:54.783100 5052 reconciler_common.go:293] "Volume detached for volume \"plugins-conf\" (UniqueName: \"kubernetes.io/configmap/a5ba4775-815a-46ef-bd2d-05dfa7513288-plugins-conf\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:54 crc kubenswrapper[5052]: I1203 11:54:54.783116 5052 reconciler_common.go:293] "Volume detached for volume \"server-conf\" (UniqueName: \"kubernetes.io/configmap/a5ba4775-815a-46ef-bd2d-05dfa7513288-server-conf\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:54 crc kubenswrapper[5052]: I1203 11:54:54.783124 5052 reconciler_common.go:293] "Volume detached for volume \"rabbitmq-tls\" (UniqueName: \"kubernetes.io/projected/a5ba4775-815a-46ef-bd2d-05dfa7513288-rabbitmq-tls\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:54 crc kubenswrapper[5052]: I1203 11:54:54.783132 5052 reconciler_common.go:293] "Volume detached for volume \"pod-info\" (UniqueName: \"kubernetes.io/downward-api/a5ba4775-815a-46ef-bd2d-05dfa7513288-pod-info\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:54 crc kubenswrapper[5052]: I1203 11:54:54.783140 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/a5ba4775-815a-46ef-bd2d-05dfa7513288-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:54 crc kubenswrapper[5052]: I1203 11:54:54.783149 5052 reconciler_common.go:293] "Volume detached for volume \"rabbitmq-erlang-cookie\" (UniqueName: \"kubernetes.io/empty-dir/a5ba4775-815a-46ef-bd2d-05dfa7513288-rabbitmq-erlang-cookie\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:54 crc kubenswrapper[5052]: I1203 11:54:54.783159 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-xbr2g\" (UniqueName: \"kubernetes.io/projected/a5ba4775-815a-46ef-bd2d-05dfa7513288-kube-api-access-xbr2g\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:54 crc kubenswrapper[5052]: I1203 11:54:54.783166 5052 reconciler_common.go:293] "Volume detached for volume \"rabbitmq-plugins\" (UniqueName: \"kubernetes.io/empty-dir/a5ba4775-815a-46ef-bd2d-05dfa7513288-rabbitmq-plugins\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:54 crc kubenswrapper[5052]: I1203 11:54:54.783175 5052 reconciler_common.go:293] "Volume detached for volume \"erlang-cookie-secret\" (UniqueName: \"kubernetes.io/secret/a5ba4775-815a-46ef-bd2d-05dfa7513288-erlang-cookie-secret\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:54 crc kubenswrapper[5052]: I1203 11:54:54.783192 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage11-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage11-crc\") on node \"crc\" " Dec 03 11:54:54 crc kubenswrapper[5052]: I1203 11:54:54.785830 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f856a02f-3f21-4fd4-8158-50a232de756e-config-data-default" (OuterVolumeSpecName: "config-data-default") pod "f856a02f-3f21-4fd4-8158-50a232de756e" (UID: "f856a02f-3f21-4fd4-8158-50a232de756e"). InnerVolumeSpecName "config-data-default". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:54:54 crc kubenswrapper[5052]: I1203 11:54:54.786290 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f856a02f-3f21-4fd4-8158-50a232de756e-config-data-generated" (OuterVolumeSpecName: "config-data-generated") pod "f856a02f-3f21-4fd4-8158-50a232de756e" (UID: "f856a02f-3f21-4fd4-8158-50a232de756e"). InnerVolumeSpecName "config-data-generated". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:54:54 crc kubenswrapper[5052]: I1203 11:54:54.786885 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f856a02f-3f21-4fd4-8158-50a232de756e-kolla-config" (OuterVolumeSpecName: "kolla-config") pod "f856a02f-3f21-4fd4-8158-50a232de756e" (UID: "f856a02f-3f21-4fd4-8158-50a232de756e"). InnerVolumeSpecName "kolla-config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:54:54 crc kubenswrapper[5052]: I1203 11:54:54.787274 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f856a02f-3f21-4fd4-8158-50a232de756e-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "f856a02f-3f21-4fd4-8158-50a232de756e" (UID: "f856a02f-3f21-4fd4-8158-50a232de756e"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:54:54 crc kubenswrapper[5052]: I1203 11:54:54.788296 5052 scope.go:117] "RemoveContainer" containerID="f2cfc34275d0c16e884df462f785981d52cd8107d11e34b450fe684cd48e4f3b" Dec 03 11:54:54 crc kubenswrapper[5052]: I1203 11:54:54.789742 5052 generic.go:334] "Generic (PLEG): container finished" podID="f856a02f-3f21-4fd4-8158-50a232de756e" containerID="8d6e9aac3a6cab1fd7cc9acbcd622c819b6570c8ebf974e1634165fe0334ddb0" exitCode=0 Dec 03 11:54:54 crc kubenswrapper[5052]: I1203 11:54:54.789795 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-galera-0" event={"ID":"f856a02f-3f21-4fd4-8158-50a232de756e","Type":"ContainerDied","Data":"8d6e9aac3a6cab1fd7cc9acbcd622c819b6570c8ebf974e1634165fe0334ddb0"} Dec 03 11:54:54 crc kubenswrapper[5052]: I1203 11:54:54.789824 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-galera-0" event={"ID":"f856a02f-3f21-4fd4-8158-50a232de756e","Type":"ContainerDied","Data":"e9a78ff070e5ba47efabe3e32a44713239d9f7733bea04dd31fc687e43dc182e"} Dec 03 11:54:54 crc kubenswrapper[5052]: I1203 11:54:54.789993 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 11:54:54 crc kubenswrapper[5052]: I1203 11:54:54.802138 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f856a02f-3f21-4fd4-8158-50a232de756e-kube-api-access-9bf6j" (OuterVolumeSpecName: "kube-api-access-9bf6j") pod "f856a02f-3f21-4fd4-8158-50a232de756e" (UID: "f856a02f-3f21-4fd4-8158-50a232de756e"). InnerVolumeSpecName "kube-api-access-9bf6j". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:54:54 crc kubenswrapper[5052]: I1203 11:54:54.811889 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-z8rn9" event={"ID":"97767bcf-c9a4-430f-8298-3c677150b865","Type":"ContainerStarted","Data":"8b643ff2eaca11b08a18ec9a8c2c3b8893b19dc1a8053d47c00be7efad4ddb8e"} Dec 03 11:54:54 crc kubenswrapper[5052]: I1203 11:54:54.811941 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/rabbitmq-server-0"] Dec 03 11:54:54 crc kubenswrapper[5052]: I1203 11:54:54.811961 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-z8rn9" Dec 03 11:54:54 crc kubenswrapper[5052]: I1203 11:54:54.814850 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-94f46848f-wgzvx" event={"ID":"775d2ccf-9b52-4274-99ef-2c33b18beaea","Type":"ContainerDied","Data":"baba821dc78b24dbf946de1b85b6619340cdd58507e31c86dc21895cd0437171"} Dec 03 11:54:54 crc kubenswrapper[5052]: I1203 11:54:54.814954 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-94f46848f-wgzvx" Dec 03 11:54:54 crc kubenswrapper[5052]: I1203 11:54:54.817106 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a5ba4775-815a-46ef-bd2d-05dfa7513288-rabbitmq-confd" (OuterVolumeSpecName: "rabbitmq-confd") pod "a5ba4775-815a-46ef-bd2d-05dfa7513288" (UID: "a5ba4775-815a-46ef-bd2d-05dfa7513288"). InnerVolumeSpecName "rabbitmq-confd". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:54:54 crc kubenswrapper[5052]: I1203 11:54:54.818796 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage03-crc" (OuterVolumeSpecName: "mysql-db") pod "f856a02f-3f21-4fd4-8158-50a232de756e" (UID: "f856a02f-3f21-4fd4-8158-50a232de756e"). InnerVolumeSpecName "local-storage03-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 11:54:54 crc kubenswrapper[5052]: I1203 11:54:54.821459 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage11-crc" (UniqueName: "kubernetes.io/local-volume/local-storage11-crc") on node "crc" Dec 03 11:54:54 crc kubenswrapper[5052]: I1203 11:54:54.822011 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/rabbitmq-server-0"] Dec 03 11:54:54 crc kubenswrapper[5052]: I1203 11:54:54.836889 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f856a02f-3f21-4fd4-8158-50a232de756e-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "f856a02f-3f21-4fd4-8158-50a232de756e" (UID: "f856a02f-3f21-4fd4-8158-50a232de756e"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:54:54 crc kubenswrapper[5052]: I1203 11:54:54.839987 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-z8rn9" podStartSLOduration=7.839968801 podStartE2EDuration="7.839968801s" podCreationTimestamp="2025-12-03 11:54:47 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:54:54.837038337 +0000 UTC m=+2085.425280101" watchObservedRunningTime="2025-12-03 11:54:54.839968801 +0000 UTC m=+2085.428210565" Dec 03 11:54:54 crc kubenswrapper[5052]: I1203 11:54:54.845888 5052 scope.go:117] "RemoveContainer" containerID="c25fbe13b5143561b88ab53fac63216ac379a26107913a3a532fbd7033bbeba6" Dec 03 11:54:54 crc kubenswrapper[5052]: E1203 11:54:54.846356 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c25fbe13b5143561b88ab53fac63216ac379a26107913a3a532fbd7033bbeba6\": container with ID starting with c25fbe13b5143561b88ab53fac63216ac379a26107913a3a532fbd7033bbeba6 not found: ID does not exist" containerID="c25fbe13b5143561b88ab53fac63216ac379a26107913a3a532fbd7033bbeba6" Dec 03 11:54:54 crc kubenswrapper[5052]: I1203 11:54:54.846383 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c25fbe13b5143561b88ab53fac63216ac379a26107913a3a532fbd7033bbeba6"} err="failed to get container status \"c25fbe13b5143561b88ab53fac63216ac379a26107913a3a532fbd7033bbeba6\": rpc error: code = NotFound desc = could not find container \"c25fbe13b5143561b88ab53fac63216ac379a26107913a3a532fbd7033bbeba6\": container with ID starting with c25fbe13b5143561b88ab53fac63216ac379a26107913a3a532fbd7033bbeba6 not found: ID does not exist" Dec 03 11:54:54 crc kubenswrapper[5052]: I1203 11:54:54.846403 5052 scope.go:117] "RemoveContainer" containerID="f2cfc34275d0c16e884df462f785981d52cd8107d11e34b450fe684cd48e4f3b" Dec 03 11:54:54 crc kubenswrapper[5052]: E1203 11:54:54.846702 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f2cfc34275d0c16e884df462f785981d52cd8107d11e34b450fe684cd48e4f3b\": container with ID starting with f2cfc34275d0c16e884df462f785981d52cd8107d11e34b450fe684cd48e4f3b not found: ID does not exist" containerID="f2cfc34275d0c16e884df462f785981d52cd8107d11e34b450fe684cd48e4f3b" Dec 03 11:54:54 crc kubenswrapper[5052]: I1203 11:54:54.846720 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f2cfc34275d0c16e884df462f785981d52cd8107d11e34b450fe684cd48e4f3b"} err="failed to get container status \"f2cfc34275d0c16e884df462f785981d52cd8107d11e34b450fe684cd48e4f3b\": rpc error: code = NotFound desc = could not find container \"f2cfc34275d0c16e884df462f785981d52cd8107d11e34b450fe684cd48e4f3b\": container with ID starting with f2cfc34275d0c16e884df462f785981d52cd8107d11e34b450fe684cd48e4f3b not found: ID does not exist" Dec 03 11:54:54 crc kubenswrapper[5052]: I1203 11:54:54.846734 5052 scope.go:117] "RemoveContainer" containerID="db5a765ca282bd8e9880ef9379b02a9202ec6dc773b029eb3abdd3d2f5581f14" Dec 03 11:54:54 crc kubenswrapper[5052]: I1203 11:54:54.856864 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f856a02f-3f21-4fd4-8158-50a232de756e-galera-tls-certs" (OuterVolumeSpecName: "galera-tls-certs") pod "f856a02f-3f21-4fd4-8158-50a232de756e" (UID: "f856a02f-3f21-4fd4-8158-50a232de756e"). InnerVolumeSpecName "galera-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:54:54 crc kubenswrapper[5052]: I1203 11:54:54.888366 5052 reconciler_common.go:293] "Volume detached for volume \"rabbitmq-confd\" (UniqueName: \"kubernetes.io/projected/a5ba4775-815a-46ef-bd2d-05dfa7513288-rabbitmq-confd\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:54 crc kubenswrapper[5052]: I1203 11:54:54.888410 5052 reconciler_common.go:293] "Volume detached for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/f856a02f-3f21-4fd4-8158-50a232de756e-kolla-config\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:54 crc kubenswrapper[5052]: I1203 11:54:54.888429 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/f856a02f-3f21-4fd4-8158-50a232de756e-config-data-default\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:54 crc kubenswrapper[5052]: I1203 11:54:54.888447 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage11-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage11-crc\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:54 crc kubenswrapper[5052]: I1203 11:54:54.888464 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f856a02f-3f21-4fd4-8158-50a232de756e-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:54 crc kubenswrapper[5052]: I1203 11:54:54.888479 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-9bf6j\" (UniqueName: \"kubernetes.io/projected/f856a02f-3f21-4fd4-8158-50a232de756e-kube-api-access-9bf6j\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:54 crc kubenswrapper[5052]: I1203 11:54:54.888497 5052 reconciler_common.go:293] "Volume detached for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/f856a02f-3f21-4fd4-8158-50a232de756e-galera-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:54 crc kubenswrapper[5052]: I1203 11:54:54.888572 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage03-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage03-crc\") on node \"crc\" " Dec 03 11:54:54 crc kubenswrapper[5052]: I1203 11:54:54.888592 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/f856a02f-3f21-4fd4-8158-50a232de756e-config-data-generated\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:54 crc kubenswrapper[5052]: I1203 11:54:54.888609 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/f856a02f-3f21-4fd4-8158-50a232de756e-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:54 crc kubenswrapper[5052]: I1203 11:54:54.894632 5052 scope.go:117] "RemoveContainer" containerID="c42837bc18a40995ef2cc940533425d21d777f73ad9d7999540fb90eedd02248" Dec 03 11:54:54 crc kubenswrapper[5052]: I1203 11:54:54.908690 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/keystone-94f46848f-wgzvx"] Dec 03 11:54:54 crc kubenswrapper[5052]: I1203 11:54:54.915269 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/keystone-94f46848f-wgzvx"] Dec 03 11:54:54 crc kubenswrapper[5052]: I1203 11:54:54.915395 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage03-crc" (UniqueName: "kubernetes.io/local-volume/local-storage03-crc") on node "crc" Dec 03 11:54:54 crc kubenswrapper[5052]: I1203 11:54:54.938399 5052 scope.go:117] "RemoveContainer" containerID="8d6e9aac3a6cab1fd7cc9acbcd622c819b6570c8ebf974e1634165fe0334ddb0" Dec 03 11:54:54 crc kubenswrapper[5052]: I1203 11:54:54.987951 5052 scope.go:117] "RemoveContainer" containerID="14eb89a7a38e6b85144c32603126a266bb620c9850f97165f7900e4b5c32df21" Dec 03 11:54:54 crc kubenswrapper[5052]: I1203 11:54:54.989873 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage03-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage03-crc\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:55 crc kubenswrapper[5052]: I1203 11:54:55.050299 5052 scope.go:117] "RemoveContainer" containerID="8d6e9aac3a6cab1fd7cc9acbcd622c819b6570c8ebf974e1634165fe0334ddb0" Dec 03 11:54:55 crc kubenswrapper[5052]: E1203 11:54:55.052224 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8d6e9aac3a6cab1fd7cc9acbcd622c819b6570c8ebf974e1634165fe0334ddb0\": container with ID starting with 8d6e9aac3a6cab1fd7cc9acbcd622c819b6570c8ebf974e1634165fe0334ddb0 not found: ID does not exist" containerID="8d6e9aac3a6cab1fd7cc9acbcd622c819b6570c8ebf974e1634165fe0334ddb0" Dec 03 11:54:55 crc kubenswrapper[5052]: I1203 11:54:55.052269 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8d6e9aac3a6cab1fd7cc9acbcd622c819b6570c8ebf974e1634165fe0334ddb0"} err="failed to get container status \"8d6e9aac3a6cab1fd7cc9acbcd622c819b6570c8ebf974e1634165fe0334ddb0\": rpc error: code = NotFound desc = could not find container \"8d6e9aac3a6cab1fd7cc9acbcd622c819b6570c8ebf974e1634165fe0334ddb0\": container with ID starting with 8d6e9aac3a6cab1fd7cc9acbcd622c819b6570c8ebf974e1634165fe0334ddb0 not found: ID does not exist" Dec 03 11:54:55 crc kubenswrapper[5052]: I1203 11:54:55.052297 5052 scope.go:117] "RemoveContainer" containerID="14eb89a7a38e6b85144c32603126a266bb620c9850f97165f7900e4b5c32df21" Dec 03 11:54:55 crc kubenswrapper[5052]: E1203 11:54:55.052761 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"14eb89a7a38e6b85144c32603126a266bb620c9850f97165f7900e4b5c32df21\": container with ID starting with 14eb89a7a38e6b85144c32603126a266bb620c9850f97165f7900e4b5c32df21 not found: ID does not exist" containerID="14eb89a7a38e6b85144c32603126a266bb620c9850f97165f7900e4b5c32df21" Dec 03 11:54:55 crc kubenswrapper[5052]: I1203 11:54:55.052791 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"14eb89a7a38e6b85144c32603126a266bb620c9850f97165f7900e4b5c32df21"} err="failed to get container status \"14eb89a7a38e6b85144c32603126a266bb620c9850f97165f7900e4b5c32df21\": rpc error: code = NotFound desc = could not find container \"14eb89a7a38e6b85144c32603126a266bb620c9850f97165f7900e4b5c32df21\": container with ID starting with 14eb89a7a38e6b85144c32603126a266bb620c9850f97165f7900e4b5c32df21 not found: ID does not exist" Dec 03 11:54:55 crc kubenswrapper[5052]: I1203 11:54:55.052814 5052 scope.go:117] "RemoveContainer" containerID="92f19f59b5c7604cf76509c11bcfe787a7d812a120d0641a971265397e22c941" Dec 03 11:54:55 crc kubenswrapper[5052]: I1203 11:54:55.175703 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:54:55 crc kubenswrapper[5052]: I1203 11:54:55.191970 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/openstack-galera-0"] Dec 03 11:54:55 crc kubenswrapper[5052]: I1203 11:54:55.195802 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/neutron-5ff446f48b-7hc25" podUID="83b9fcfd-8007-4d6b-9090-d2e536cb1bbc" containerName="neutron-httpd" probeResult="failure" output="Get \"https://10.217.1.99:9696/\": dial tcp 10.217.1.99:9696: connect: connection refused" Dec 03 11:54:55 crc kubenswrapper[5052]: I1203 11:54:55.206602 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/openstack-galera-0"] Dec 03 11:54:55 crc kubenswrapper[5052]: I1203 11:54:55.212810 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/rabbitmq-cell1-server-0"] Dec 03 11:54:55 crc kubenswrapper[5052]: I1203 11:54:55.224983 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/rabbitmq-cell1-server-0"] Dec 03 11:54:55 crc kubenswrapper[5052]: I1203 11:54:55.298187 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a5ff0249-173a-4793-8cbe-0aa749f34632-combined-ca-bundle\") pod \"a5ff0249-173a-4793-8cbe-0aa749f34632\" (UID: \"a5ff0249-173a-4793-8cbe-0aa749f34632\") " Dec 03 11:54:55 crc kubenswrapper[5052]: I1203 11:54:55.298456 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/a5ff0249-173a-4793-8cbe-0aa749f34632-scripts\") pod \"a5ff0249-173a-4793-8cbe-0aa749f34632\" (UID: \"a5ff0249-173a-4793-8cbe-0aa749f34632\") " Dec 03 11:54:55 crc kubenswrapper[5052]: I1203 11:54:55.298613 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/a5ff0249-173a-4793-8cbe-0aa749f34632-sg-core-conf-yaml\") pod \"a5ff0249-173a-4793-8cbe-0aa749f34632\" (UID: \"a5ff0249-173a-4793-8cbe-0aa749f34632\") " Dec 03 11:54:55 crc kubenswrapper[5052]: I1203 11:54:55.298752 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/a5ff0249-173a-4793-8cbe-0aa749f34632-run-httpd\") pod \"a5ff0249-173a-4793-8cbe-0aa749f34632\" (UID: \"a5ff0249-173a-4793-8cbe-0aa749f34632\") " Dec 03 11:54:55 crc kubenswrapper[5052]: I1203 11:54:55.298892 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/a5ff0249-173a-4793-8cbe-0aa749f34632-log-httpd\") pod \"a5ff0249-173a-4793-8cbe-0aa749f34632\" (UID: \"a5ff0249-173a-4793-8cbe-0aa749f34632\") " Dec 03 11:54:55 crc kubenswrapper[5052]: I1203 11:54:55.299104 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4v26g\" (UniqueName: \"kubernetes.io/projected/a5ff0249-173a-4793-8cbe-0aa749f34632-kube-api-access-4v26g\") pod \"a5ff0249-173a-4793-8cbe-0aa749f34632\" (UID: \"a5ff0249-173a-4793-8cbe-0aa749f34632\") " Dec 03 11:54:55 crc kubenswrapper[5052]: I1203 11:54:55.299203 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5ff0249-173a-4793-8cbe-0aa749f34632-run-httpd" (OuterVolumeSpecName: "run-httpd") pod "a5ff0249-173a-4793-8cbe-0aa749f34632" (UID: "a5ff0249-173a-4793-8cbe-0aa749f34632"). InnerVolumeSpecName "run-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:54:55 crc kubenswrapper[5052]: I1203 11:54:55.299223 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/a5ff0249-173a-4793-8cbe-0aa749f34632-ceilometer-tls-certs\") pod \"a5ff0249-173a-4793-8cbe-0aa749f34632\" (UID: \"a5ff0249-173a-4793-8cbe-0aa749f34632\") " Dec 03 11:54:55 crc kubenswrapper[5052]: I1203 11:54:55.299403 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a5ff0249-173a-4793-8cbe-0aa749f34632-config-data\") pod \"a5ff0249-173a-4793-8cbe-0aa749f34632\" (UID: \"a5ff0249-173a-4793-8cbe-0aa749f34632\") " Dec 03 11:54:55 crc kubenswrapper[5052]: I1203 11:54:55.299557 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5ff0249-173a-4793-8cbe-0aa749f34632-log-httpd" (OuterVolumeSpecName: "log-httpd") pod "a5ff0249-173a-4793-8cbe-0aa749f34632" (UID: "a5ff0249-173a-4793-8cbe-0aa749f34632"). InnerVolumeSpecName "log-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:54:55 crc kubenswrapper[5052]: I1203 11:54:55.301055 5052 reconciler_common.go:293] "Volume detached for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/a5ff0249-173a-4793-8cbe-0aa749f34632-run-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:55 crc kubenswrapper[5052]: I1203 11:54:55.303268 5052 reconciler_common.go:293] "Volume detached for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/a5ff0249-173a-4793-8cbe-0aa749f34632-log-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:55 crc kubenswrapper[5052]: I1203 11:54:55.301472 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a5ff0249-173a-4793-8cbe-0aa749f34632-scripts" (OuterVolumeSpecName: "scripts") pod "a5ff0249-173a-4793-8cbe-0aa749f34632" (UID: "a5ff0249-173a-4793-8cbe-0aa749f34632"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:54:55 crc kubenswrapper[5052]: I1203 11:54:55.303773 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a5ff0249-173a-4793-8cbe-0aa749f34632-kube-api-access-4v26g" (OuterVolumeSpecName: "kube-api-access-4v26g") pod "a5ff0249-173a-4793-8cbe-0aa749f34632" (UID: "a5ff0249-173a-4793-8cbe-0aa749f34632"). InnerVolumeSpecName "kube-api-access-4v26g". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:54:55 crc kubenswrapper[5052]: I1203 11:54:55.325062 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a5ff0249-173a-4793-8cbe-0aa749f34632-sg-core-conf-yaml" (OuterVolumeSpecName: "sg-core-conf-yaml") pod "a5ff0249-173a-4793-8cbe-0aa749f34632" (UID: "a5ff0249-173a-4793-8cbe-0aa749f34632"). InnerVolumeSpecName "sg-core-conf-yaml". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:54:55 crc kubenswrapper[5052]: I1203 11:54:55.358507 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a5ff0249-173a-4793-8cbe-0aa749f34632-ceilometer-tls-certs" (OuterVolumeSpecName: "ceilometer-tls-certs") pod "a5ff0249-173a-4793-8cbe-0aa749f34632" (UID: "a5ff0249-173a-4793-8cbe-0aa749f34632"). InnerVolumeSpecName "ceilometer-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:54:55 crc kubenswrapper[5052]: I1203 11:54:55.383686 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a5ff0249-173a-4793-8cbe-0aa749f34632-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "a5ff0249-173a-4793-8cbe-0aa749f34632" (UID: "a5ff0249-173a-4793-8cbe-0aa749f34632"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:54:55 crc kubenswrapper[5052]: I1203 11:54:55.393401 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a5ff0249-173a-4793-8cbe-0aa749f34632-config-data" (OuterVolumeSpecName: "config-data") pod "a5ff0249-173a-4793-8cbe-0aa749f34632" (UID: "a5ff0249-173a-4793-8cbe-0aa749f34632"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:54:55 crc kubenswrapper[5052]: I1203 11:54:55.404424 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-4v26g\" (UniqueName: \"kubernetes.io/projected/a5ff0249-173a-4793-8cbe-0aa749f34632-kube-api-access-4v26g\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:55 crc kubenswrapper[5052]: I1203 11:54:55.404450 5052 reconciler_common.go:293] "Volume detached for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/a5ff0249-173a-4793-8cbe-0aa749f34632-ceilometer-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:55 crc kubenswrapper[5052]: I1203 11:54:55.404480 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a5ff0249-173a-4793-8cbe-0aa749f34632-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:55 crc kubenswrapper[5052]: I1203 11:54:55.404490 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a5ff0249-173a-4793-8cbe-0aa749f34632-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:55 crc kubenswrapper[5052]: I1203 11:54:55.404499 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/a5ff0249-173a-4793-8cbe-0aa749f34632-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:55 crc kubenswrapper[5052]: I1203 11:54:55.404524 5052 reconciler_common.go:293] "Volume detached for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/a5ff0249-173a-4793-8cbe-0aa749f34632-sg-core-conf-yaml\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:55 crc kubenswrapper[5052]: I1203 11:54:55.457968 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 11:54:55 crc kubenswrapper[5052]: I1203 11:54:55.505397 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4ec0d450-cd52-432e-8bb2-1ef9698821c1-combined-ca-bundle\") pod \"4ec0d450-cd52-432e-8bb2-1ef9698821c1\" (UID: \"4ec0d450-cd52-432e-8bb2-1ef9698821c1\") " Dec 03 11:54:55 crc kubenswrapper[5052]: I1203 11:54:55.505489 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nnk5d\" (UniqueName: \"kubernetes.io/projected/4ec0d450-cd52-432e-8bb2-1ef9698821c1-kube-api-access-nnk5d\") pod \"4ec0d450-cd52-432e-8bb2-1ef9698821c1\" (UID: \"4ec0d450-cd52-432e-8bb2-1ef9698821c1\") " Dec 03 11:54:55 crc kubenswrapper[5052]: I1203 11:54:55.505579 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/4ec0d450-cd52-432e-8bb2-1ef9698821c1-config-data\") pod \"4ec0d450-cd52-432e-8bb2-1ef9698821c1\" (UID: \"4ec0d450-cd52-432e-8bb2-1ef9698821c1\") " Dec 03 11:54:55 crc kubenswrapper[5052]: I1203 11:54:55.510809 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4ec0d450-cd52-432e-8bb2-1ef9698821c1-kube-api-access-nnk5d" (OuterVolumeSpecName: "kube-api-access-nnk5d") pod "4ec0d450-cd52-432e-8bb2-1ef9698821c1" (UID: "4ec0d450-cd52-432e-8bb2-1ef9698821c1"). InnerVolumeSpecName "kube-api-access-nnk5d". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:54:55 crc kubenswrapper[5052]: I1203 11:54:55.530626 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4ec0d450-cd52-432e-8bb2-1ef9698821c1-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "4ec0d450-cd52-432e-8bb2-1ef9698821c1" (UID: "4ec0d450-cd52-432e-8bb2-1ef9698821c1"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:54:55 crc kubenswrapper[5052]: I1203 11:54:55.532947 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4ec0d450-cd52-432e-8bb2-1ef9698821c1-config-data" (OuterVolumeSpecName: "config-data") pod "4ec0d450-cd52-432e-8bb2-1ef9698821c1" (UID: "4ec0d450-cd52-432e-8bb2-1ef9698821c1"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:54:55 crc kubenswrapper[5052]: I1203 11:54:55.608181 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/4ec0d450-cd52-432e-8bb2-1ef9698821c1-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:55 crc kubenswrapper[5052]: I1203 11:54:55.608227 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4ec0d450-cd52-432e-8bb2-1ef9698821c1-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:55 crc kubenswrapper[5052]: I1203 11:54:55.608243 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-nnk5d\" (UniqueName: \"kubernetes.io/projected/4ec0d450-cd52-432e-8bb2-1ef9698821c1-kube-api-access-nnk5d\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:55 crc kubenswrapper[5052]: I1203 11:54:55.647073 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 11:54:55 crc kubenswrapper[5052]: I1203 11:54:55.682313 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0201d941-cd05-4d1c-9af6-f3a82a375a0f" path="/var/lib/kubelet/pods/0201d941-cd05-4d1c-9af6-f3a82a375a0f/volumes" Dec 03 11:54:55 crc kubenswrapper[5052]: I1203 11:54:55.683259 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="10e3094e-f625-4d69-b0e9-358220c0d195" path="/var/lib/kubelet/pods/10e3094e-f625-4d69-b0e9-358220c0d195/volumes" Dec 03 11:54:55 crc kubenswrapper[5052]: I1203 11:54:55.683855 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="383e7933-2bc4-4ead-bc34-0832091bd952" path="/var/lib/kubelet/pods/383e7933-2bc4-4ead-bc34-0832091bd952/volumes" Dec 03 11:54:55 crc kubenswrapper[5052]: I1203 11:54:55.685078 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="53d7a9a2-b6a9-42dc-9828-d8ceaa70f0a3" path="/var/lib/kubelet/pods/53d7a9a2-b6a9-42dc-9828-d8ceaa70f0a3/volumes" Dec 03 11:54:55 crc kubenswrapper[5052]: I1203 11:54:55.685639 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5679a95d-a498-448e-97ed-497496520506" path="/var/lib/kubelet/pods/5679a95d-a498-448e-97ed-497496520506/volumes" Dec 03 11:54:55 crc kubenswrapper[5052]: I1203 11:54:55.686215 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="653882ec-4522-4979-b1aa-7fad5023605c" path="/var/lib/kubelet/pods/653882ec-4522-4979-b1aa-7fad5023605c/volumes" Dec 03 11:54:55 crc kubenswrapper[5052]: I1203 11:54:55.687304 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="775d2ccf-9b52-4274-99ef-2c33b18beaea" path="/var/lib/kubelet/pods/775d2ccf-9b52-4274-99ef-2c33b18beaea/volumes" Dec 03 11:54:55 crc kubenswrapper[5052]: I1203 11:54:55.688023 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a18602ce-7206-4edc-a8d3-5d0754c3da02" path="/var/lib/kubelet/pods/a18602ce-7206-4edc-a8d3-5d0754c3da02/volumes" Dec 03 11:54:55 crc kubenswrapper[5052]: I1203 11:54:55.690063 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a5ba4775-815a-46ef-bd2d-05dfa7513288" path="/var/lib/kubelet/pods/a5ba4775-815a-46ef-bd2d-05dfa7513288/volumes" Dec 03 11:54:55 crc kubenswrapper[5052]: I1203 11:54:55.690892 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a73e0afd-e037-41f7-b214-d4eebd4bafe0" path="/var/lib/kubelet/pods/a73e0afd-e037-41f7-b214-d4eebd4bafe0/volumes" Dec 03 11:54:55 crc kubenswrapper[5052]: I1203 11:54:55.691650 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d3222c69-47bc-46d8-a025-cc457881cac9" path="/var/lib/kubelet/pods/d3222c69-47bc-46d8-a025-cc457881cac9/volumes" Dec 03 11:54:55 crc kubenswrapper[5052]: I1203 11:54:55.693042 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d541d73d-2f95-43a8-8409-40f5f2ff54c4" path="/var/lib/kubelet/pods/d541d73d-2f95-43a8-8409-40f5f2ff54c4/volumes" Dec 03 11:54:55 crc kubenswrapper[5052]: I1203 11:54:55.693872 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ec16e8c3-9750-4475-a671-17b0c16826c8" path="/var/lib/kubelet/pods/ec16e8c3-9750-4475-a671-17b0c16826c8/volumes" Dec 03 11:54:55 crc kubenswrapper[5052]: I1203 11:54:55.694752 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f856a02f-3f21-4fd4-8158-50a232de756e" path="/var/lib/kubelet/pods/f856a02f-3f21-4fd4-8158-50a232de756e/volumes" Dec 03 11:54:55 crc kubenswrapper[5052]: I1203 11:54:55.710003 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4a181d2c-7595-4394-98de-957451309c98-combined-ca-bundle\") pod \"4a181d2c-7595-4394-98de-957451309c98\" (UID: \"4a181d2c-7595-4394-98de-957451309c98\") " Dec 03 11:54:55 crc kubenswrapper[5052]: I1203 11:54:55.710180 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5mmcz\" (UniqueName: \"kubernetes.io/projected/4a181d2c-7595-4394-98de-957451309c98-kube-api-access-5mmcz\") pod \"4a181d2c-7595-4394-98de-957451309c98\" (UID: \"4a181d2c-7595-4394-98de-957451309c98\") " Dec 03 11:54:55 crc kubenswrapper[5052]: I1203 11:54:55.710219 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/4a181d2c-7595-4394-98de-957451309c98-config-data\") pod \"4a181d2c-7595-4394-98de-957451309c98\" (UID: \"4a181d2c-7595-4394-98de-957451309c98\") " Dec 03 11:54:55 crc kubenswrapper[5052]: I1203 11:54:55.714128 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4a181d2c-7595-4394-98de-957451309c98-kube-api-access-5mmcz" (OuterVolumeSpecName: "kube-api-access-5mmcz") pod "4a181d2c-7595-4394-98de-957451309c98" (UID: "4a181d2c-7595-4394-98de-957451309c98"). InnerVolumeSpecName "kube-api-access-5mmcz". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:54:55 crc kubenswrapper[5052]: I1203 11:54:55.746627 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4a181d2c-7595-4394-98de-957451309c98-config-data" (OuterVolumeSpecName: "config-data") pod "4a181d2c-7595-4394-98de-957451309c98" (UID: "4a181d2c-7595-4394-98de-957451309c98"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:54:55 crc kubenswrapper[5052]: I1203 11:54:55.757160 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4a181d2c-7595-4394-98de-957451309c98-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "4a181d2c-7595-4394-98de-957451309c98" (UID: "4a181d2c-7595-4394-98de-957451309c98"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:54:55 crc kubenswrapper[5052]: I1203 11:54:55.812387 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-5mmcz\" (UniqueName: \"kubernetes.io/projected/4a181d2c-7595-4394-98de-957451309c98-kube-api-access-5mmcz\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:55 crc kubenswrapper[5052]: I1203 11:54:55.812416 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/4a181d2c-7595-4394-98de-957451309c98-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:55 crc kubenswrapper[5052]: I1203 11:54:55.812424 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4a181d2c-7595-4394-98de-957451309c98-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:55 crc kubenswrapper[5052]: I1203 11:54:55.826411 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"a5ff0249-173a-4793-8cbe-0aa749f34632","Type":"ContainerDied","Data":"42ce9b7cb4235ab65f20054ec3746eea9e53fe033f16c4c19190abce6617e7a5"} Dec 03 11:54:55 crc kubenswrapper[5052]: I1203 11:54:55.826445 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:54:55 crc kubenswrapper[5052]: I1203 11:54:55.826482 5052 scope.go:117] "RemoveContainer" containerID="615dcdb302a4fa77dc8006892c9f0c5509fe1f5ef890116dd6a627f53880b9b8" Dec 03 11:54:55 crc kubenswrapper[5052]: I1203 11:54:55.828788 5052 generic.go:334] "Generic (PLEG): container finished" podID="4a181d2c-7595-4394-98de-957451309c98" containerID="c3cb1469712553e8283f9907b8e296f175dfb01cae7b28331f721bbf4e80e204" exitCode=0 Dec 03 11:54:55 crc kubenswrapper[5052]: I1203 11:54:55.828848 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-conductor-0" event={"ID":"4a181d2c-7595-4394-98de-957451309c98","Type":"ContainerDied","Data":"c3cb1469712553e8283f9907b8e296f175dfb01cae7b28331f721bbf4e80e204"} Dec 03 11:54:55 crc kubenswrapper[5052]: I1203 11:54:55.828870 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-conductor-0" event={"ID":"4a181d2c-7595-4394-98de-957451309c98","Type":"ContainerDied","Data":"ee653fb33989f4aca07fcbe1bf8a3dbf2e60a62c48b8ca19f6784644b6e2efd2"} Dec 03 11:54:55 crc kubenswrapper[5052]: I1203 11:54:55.829017 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 11:54:55 crc kubenswrapper[5052]: I1203 11:54:55.833261 5052 generic.go:334] "Generic (PLEG): container finished" podID="83b9fcfd-8007-4d6b-9090-d2e536cb1bbc" containerID="cd35d473482d3a4b6bc6f957931d66333a4d95cf45ac228f925b1160c222f1f6" exitCode=0 Dec 03 11:54:55 crc kubenswrapper[5052]: I1203 11:54:55.833411 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-5ff446f48b-7hc25" event={"ID":"83b9fcfd-8007-4d6b-9090-d2e536cb1bbc","Type":"ContainerDied","Data":"cd35d473482d3a4b6bc6f957931d66333a4d95cf45ac228f925b1160c222f1f6"} Dec 03 11:54:55 crc kubenswrapper[5052]: I1203 11:54:55.836206 5052 generic.go:334] "Generic (PLEG): container finished" podID="4ec0d450-cd52-432e-8bb2-1ef9698821c1" containerID="c75d78e4eb183b74af6b09e658fe926d3135454064d32c91f94389213e67abcc" exitCode=0 Dec 03 11:54:55 crc kubenswrapper[5052]: I1203 11:54:55.836271 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-conductor-0" event={"ID":"4ec0d450-cd52-432e-8bb2-1ef9698821c1","Type":"ContainerDied","Data":"c75d78e4eb183b74af6b09e658fe926d3135454064d32c91f94389213e67abcc"} Dec 03 11:54:55 crc kubenswrapper[5052]: I1203 11:54:55.836295 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-conductor-0" event={"ID":"4ec0d450-cd52-432e-8bb2-1ef9698821c1","Type":"ContainerDied","Data":"7cfae3384dfc64e883acd3f6252258d3a8c0820ae83f7b47b7037b2129664b37"} Dec 03 11:54:55 crc kubenswrapper[5052]: I1203 11:54:55.836394 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 11:54:55 crc kubenswrapper[5052]: I1203 11:54:55.837904 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-5ff446f48b-7hc25" Dec 03 11:54:55 crc kubenswrapper[5052]: I1203 11:54:55.852781 5052 scope.go:117] "RemoveContainer" containerID="14c4e0b558f59a7e38279d4f104332c292bd1c29af9d8ae1768b1a9ec55a8d31" Dec 03 11:54:55 crc kubenswrapper[5052]: I1203 11:54:55.877926 5052 scope.go:117] "RemoveContainer" containerID="cf164870d3f804d61f70d5e02392ceff8cd44fdf830fb056d8e5f574d1ef05c1" Dec 03 11:54:55 crc kubenswrapper[5052]: I1203 11:54:55.901523 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:54:55 crc kubenswrapper[5052]: I1203 11:54:55.906996 5052 scope.go:117] "RemoveContainer" containerID="7bad91ec5c27b9fd97fb06baef39407cc01e86e5c940cc4ffd9ae98b217e02e2" Dec 03 11:54:55 crc kubenswrapper[5052]: I1203 11:54:55.913244 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/83b9fcfd-8007-4d6b-9090-d2e536cb1bbc-combined-ca-bundle\") pod \"83b9fcfd-8007-4d6b-9090-d2e536cb1bbc\" (UID: \"83b9fcfd-8007-4d6b-9090-d2e536cb1bbc\") " Dec 03 11:54:55 crc kubenswrapper[5052]: I1203 11:54:55.913325 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6lpvb\" (UniqueName: \"kubernetes.io/projected/83b9fcfd-8007-4d6b-9090-d2e536cb1bbc-kube-api-access-6lpvb\") pod \"83b9fcfd-8007-4d6b-9090-d2e536cb1bbc\" (UID: \"83b9fcfd-8007-4d6b-9090-d2e536cb1bbc\") " Dec 03 11:54:55 crc kubenswrapper[5052]: I1203 11:54:55.913352 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/83b9fcfd-8007-4d6b-9090-d2e536cb1bbc-internal-tls-certs\") pod \"83b9fcfd-8007-4d6b-9090-d2e536cb1bbc\" (UID: \"83b9fcfd-8007-4d6b-9090-d2e536cb1bbc\") " Dec 03 11:54:55 crc kubenswrapper[5052]: I1203 11:54:55.913390 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/83b9fcfd-8007-4d6b-9090-d2e536cb1bbc-ovndb-tls-certs\") pod \"83b9fcfd-8007-4d6b-9090-d2e536cb1bbc\" (UID: \"83b9fcfd-8007-4d6b-9090-d2e536cb1bbc\") " Dec 03 11:54:55 crc kubenswrapper[5052]: I1203 11:54:55.913449 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/83b9fcfd-8007-4d6b-9090-d2e536cb1bbc-config\") pod \"83b9fcfd-8007-4d6b-9090-d2e536cb1bbc\" (UID: \"83b9fcfd-8007-4d6b-9090-d2e536cb1bbc\") " Dec 03 11:54:55 crc kubenswrapper[5052]: I1203 11:54:55.913533 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/83b9fcfd-8007-4d6b-9090-d2e536cb1bbc-httpd-config\") pod \"83b9fcfd-8007-4d6b-9090-d2e536cb1bbc\" (UID: \"83b9fcfd-8007-4d6b-9090-d2e536cb1bbc\") " Dec 03 11:54:55 crc kubenswrapper[5052]: I1203 11:54:55.913601 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/83b9fcfd-8007-4d6b-9090-d2e536cb1bbc-public-tls-certs\") pod \"83b9fcfd-8007-4d6b-9090-d2e536cb1bbc\" (UID: \"83b9fcfd-8007-4d6b-9090-d2e536cb1bbc\") " Dec 03 11:54:55 crc kubenswrapper[5052]: I1203 11:54:55.921949 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:54:55 crc kubenswrapper[5052]: I1203 11:54:55.922051 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/83b9fcfd-8007-4d6b-9090-d2e536cb1bbc-kube-api-access-6lpvb" (OuterVolumeSpecName: "kube-api-access-6lpvb") pod "83b9fcfd-8007-4d6b-9090-d2e536cb1bbc" (UID: "83b9fcfd-8007-4d6b-9090-d2e536cb1bbc"). InnerVolumeSpecName "kube-api-access-6lpvb". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:54:55 crc kubenswrapper[5052]: I1203 11:54:55.924406 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/83b9fcfd-8007-4d6b-9090-d2e536cb1bbc-httpd-config" (OuterVolumeSpecName: "httpd-config") pod "83b9fcfd-8007-4d6b-9090-d2e536cb1bbc" (UID: "83b9fcfd-8007-4d6b-9090-d2e536cb1bbc"). InnerVolumeSpecName "httpd-config". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:54:55 crc kubenswrapper[5052]: I1203 11:54:55.928223 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-0"] Dec 03 11:54:55 crc kubenswrapper[5052]: I1203 11:54:55.932806 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-0"] Dec 03 11:54:55 crc kubenswrapper[5052]: I1203 11:54:55.937874 5052 scope.go:117] "RemoveContainer" containerID="c3cb1469712553e8283f9907b8e296f175dfb01cae7b28331f721bbf4e80e204" Dec 03 11:54:55 crc kubenswrapper[5052]: I1203 11:54:55.937990 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-0"] Dec 03 11:54:55 crc kubenswrapper[5052]: I1203 11:54:55.942753 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-0"] Dec 03 11:54:55 crc kubenswrapper[5052]: I1203 11:54:55.953288 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/83b9fcfd-8007-4d6b-9090-d2e536cb1bbc-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "83b9fcfd-8007-4d6b-9090-d2e536cb1bbc" (UID: "83b9fcfd-8007-4d6b-9090-d2e536cb1bbc"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:54:55 crc kubenswrapper[5052]: I1203 11:54:55.953630 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/83b9fcfd-8007-4d6b-9090-d2e536cb1bbc-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "83b9fcfd-8007-4d6b-9090-d2e536cb1bbc" (UID: "83b9fcfd-8007-4d6b-9090-d2e536cb1bbc"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:54:55 crc kubenswrapper[5052]: I1203 11:54:55.956373 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/83b9fcfd-8007-4d6b-9090-d2e536cb1bbc-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "83b9fcfd-8007-4d6b-9090-d2e536cb1bbc" (UID: "83b9fcfd-8007-4d6b-9090-d2e536cb1bbc"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:54:55 crc kubenswrapper[5052]: I1203 11:54:55.957466 5052 scope.go:117] "RemoveContainer" containerID="c3cb1469712553e8283f9907b8e296f175dfb01cae7b28331f721bbf4e80e204" Dec 03 11:54:55 crc kubenswrapper[5052]: E1203 11:54:55.961308 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c3cb1469712553e8283f9907b8e296f175dfb01cae7b28331f721bbf4e80e204\": container with ID starting with c3cb1469712553e8283f9907b8e296f175dfb01cae7b28331f721bbf4e80e204 not found: ID does not exist" containerID="c3cb1469712553e8283f9907b8e296f175dfb01cae7b28331f721bbf4e80e204" Dec 03 11:54:55 crc kubenswrapper[5052]: I1203 11:54:55.961360 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c3cb1469712553e8283f9907b8e296f175dfb01cae7b28331f721bbf4e80e204"} err="failed to get container status \"c3cb1469712553e8283f9907b8e296f175dfb01cae7b28331f721bbf4e80e204\": rpc error: code = NotFound desc = could not find container \"c3cb1469712553e8283f9907b8e296f175dfb01cae7b28331f721bbf4e80e204\": container with ID starting with c3cb1469712553e8283f9907b8e296f175dfb01cae7b28331f721bbf4e80e204 not found: ID does not exist" Dec 03 11:54:55 crc kubenswrapper[5052]: I1203 11:54:55.961388 5052 scope.go:117] "RemoveContainer" containerID="c75d78e4eb183b74af6b09e658fe926d3135454064d32c91f94389213e67abcc" Dec 03 11:54:55 crc kubenswrapper[5052]: I1203 11:54:55.963149 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/83b9fcfd-8007-4d6b-9090-d2e536cb1bbc-config" (OuterVolumeSpecName: "config") pod "83b9fcfd-8007-4d6b-9090-d2e536cb1bbc" (UID: "83b9fcfd-8007-4d6b-9090-d2e536cb1bbc"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:54:55 crc kubenswrapper[5052]: I1203 11:54:55.987565 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/83b9fcfd-8007-4d6b-9090-d2e536cb1bbc-ovndb-tls-certs" (OuterVolumeSpecName: "ovndb-tls-certs") pod "83b9fcfd-8007-4d6b-9090-d2e536cb1bbc" (UID: "83b9fcfd-8007-4d6b-9090-d2e536cb1bbc"). InnerVolumeSpecName "ovndb-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:54:55 crc kubenswrapper[5052]: I1203 11:54:55.990828 5052 scope.go:117] "RemoveContainer" containerID="c75d78e4eb183b74af6b09e658fe926d3135454064d32c91f94389213e67abcc" Dec 03 11:54:55 crc kubenswrapper[5052]: E1203 11:54:55.991255 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c75d78e4eb183b74af6b09e658fe926d3135454064d32c91f94389213e67abcc\": container with ID starting with c75d78e4eb183b74af6b09e658fe926d3135454064d32c91f94389213e67abcc not found: ID does not exist" containerID="c75d78e4eb183b74af6b09e658fe926d3135454064d32c91f94389213e67abcc" Dec 03 11:54:55 crc kubenswrapper[5052]: I1203 11:54:55.991413 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c75d78e4eb183b74af6b09e658fe926d3135454064d32c91f94389213e67abcc"} err="failed to get container status \"c75d78e4eb183b74af6b09e658fe926d3135454064d32c91f94389213e67abcc\": rpc error: code = NotFound desc = could not find container \"c75d78e4eb183b74af6b09e658fe926d3135454064d32c91f94389213e67abcc\": container with ID starting with c75d78e4eb183b74af6b09e658fe926d3135454064d32c91f94389213e67abcc not found: ID does not exist" Dec 03 11:54:56 crc kubenswrapper[5052]: I1203 11:54:56.015456 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/83b9fcfd-8007-4d6b-9090-d2e536cb1bbc-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:56 crc kubenswrapper[5052]: I1203 11:54:56.015679 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-6lpvb\" (UniqueName: \"kubernetes.io/projected/83b9fcfd-8007-4d6b-9090-d2e536cb1bbc-kube-api-access-6lpvb\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:56 crc kubenswrapper[5052]: I1203 11:54:56.015758 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/83b9fcfd-8007-4d6b-9090-d2e536cb1bbc-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:56 crc kubenswrapper[5052]: I1203 11:54:56.015831 5052 reconciler_common.go:293] "Volume detached for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/83b9fcfd-8007-4d6b-9090-d2e536cb1bbc-ovndb-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:56 crc kubenswrapper[5052]: I1203 11:54:56.015900 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/83b9fcfd-8007-4d6b-9090-d2e536cb1bbc-config\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:56 crc kubenswrapper[5052]: I1203 11:54:56.016810 5052 reconciler_common.go:293] "Volume detached for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/83b9fcfd-8007-4d6b-9090-d2e536cb1bbc-httpd-config\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:56 crc kubenswrapper[5052]: I1203 11:54:56.016910 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/83b9fcfd-8007-4d6b-9090-d2e536cb1bbc-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 11:54:56 crc kubenswrapper[5052]: E1203 11:54:56.321467 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 11:54:56 crc kubenswrapper[5052]: E1203 11:54:56.321550 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6087dc7e-081b-4392-adb6-205b2aec805f-operator-scripts podName:6087dc7e-081b-4392-adb6-205b2aec805f nodeName:}" failed. No retries permitted until 2025-12-03 11:55:00.32153536 +0000 UTC m=+2090.909777124 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/6087dc7e-081b-4392-adb6-205b2aec805f-operator-scripts") pod "novaapie4da-account-delete-mknr9" (UID: "6087dc7e-081b-4392-adb6-205b2aec805f") : configmap "openstack-scripts" not found Dec 03 11:54:56 crc kubenswrapper[5052]: E1203 11:54:56.322185 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 11:54:56 crc kubenswrapper[5052]: E1203 11:54:56.322220 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/93d03e32-f318-470c-ad5d-f17a609f5003-operator-scripts podName:93d03e32-f318-470c-ad5d-f17a609f5003 nodeName:}" failed. No retries permitted until 2025-12-03 11:55:00.32221217 +0000 UTC m=+2090.910453934 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/93d03e32-f318-470c-ad5d-f17a609f5003-operator-scripts") pod "novacell0745f-account-delete-zbhjv" (UID: "93d03e32-f318-470c-ad5d-f17a609f5003") : configmap "openstack-scripts" not found Dec 03 11:54:56 crc kubenswrapper[5052]: I1203 11:54:56.484922 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/swift-proxy-b9d8f996b-brs7h" podUID="036bc2b0-bc50-4c91-bea5-882b9e592e4f" containerName="proxy-httpd" probeResult="failure" output="Get \"https://10.217.1.101:8080/healthcheck\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Dec 03 11:54:56 crc kubenswrapper[5052]: I1203 11:54:56.484922 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/swift-proxy-b9d8f996b-brs7h" podUID="036bc2b0-bc50-4c91-bea5-882b9e592e4f" containerName="proxy-server" probeResult="failure" output="Get \"https://10.217.1.101:8080/healthcheck\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Dec 03 11:54:56 crc kubenswrapper[5052]: I1203 11:54:56.854919 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-5ff446f48b-7hc25" event={"ID":"83b9fcfd-8007-4d6b-9090-d2e536cb1bbc","Type":"ContainerDied","Data":"d9a6cf6c4902e067e4f6982221164cc67224cb6bdd2f6ecce5a98dcf499fa966"} Dec 03 11:54:56 crc kubenswrapper[5052]: I1203 11:54:56.854937 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-5ff446f48b-7hc25" Dec 03 11:54:56 crc kubenswrapper[5052]: I1203 11:54:56.855030 5052 scope.go:117] "RemoveContainer" containerID="d4b83b49b57a27fa3f6b26ff04edd361dfcc21c0d52243555076b619c81a9ec9" Dec 03 11:54:56 crc kubenswrapper[5052]: I1203 11:54:56.877770 5052 scope.go:117] "RemoveContainer" containerID="cd35d473482d3a4b6bc6f957931d66333a4d95cf45ac228f925b1160c222f1f6" Dec 03 11:54:56 crc kubenswrapper[5052]: I1203 11:54:56.882613 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/neutron-5ff446f48b-7hc25"] Dec 03 11:54:56 crc kubenswrapper[5052]: I1203 11:54:56.888618 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/neutron-5ff446f48b-7hc25"] Dec 03 11:54:56 crc kubenswrapper[5052]: I1203 11:54:56.940103 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/barbican-api-5758478fc6-5g46v" podUID="a18602ce-7206-4edc-a8d3-5d0754c3da02" containerName="barbican-api" probeResult="failure" output="Get \"https://10.217.1.95:9311/healthcheck\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Dec 03 11:54:56 crc kubenswrapper[5052]: I1203 11:54:56.940121 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/barbican-api-5758478fc6-5g46v" podUID="a18602ce-7206-4edc-a8d3-5d0754c3da02" containerName="barbican-api-log" probeResult="failure" output="Get \"https://10.217.1.95:9311/healthcheck\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Dec 03 11:54:57 crc kubenswrapper[5052]: I1203 11:54:57.683721 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4a181d2c-7595-4394-98de-957451309c98" path="/var/lib/kubelet/pods/4a181d2c-7595-4394-98de-957451309c98/volumes" Dec 03 11:54:57 crc kubenswrapper[5052]: I1203 11:54:57.684709 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4ec0d450-cd52-432e-8bb2-1ef9698821c1" path="/var/lib/kubelet/pods/4ec0d450-cd52-432e-8bb2-1ef9698821c1/volumes" Dec 03 11:54:57 crc kubenswrapper[5052]: I1203 11:54:57.685641 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="83b9fcfd-8007-4d6b-9090-d2e536cb1bbc" path="/var/lib/kubelet/pods/83b9fcfd-8007-4d6b-9090-d2e536cb1bbc/volumes" Dec 03 11:54:57 crc kubenswrapper[5052]: I1203 11:54:57.687781 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a5ff0249-173a-4793-8cbe-0aa749f34632" path="/var/lib/kubelet/pods/a5ff0249-173a-4793-8cbe-0aa749f34632/volumes" Dec 03 11:54:59 crc kubenswrapper[5052]: E1203 11:54:59.779933 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/dns-swift-storage-0: configmap "dns-swift-storage-0" not found Dec 03 11:54:59 crc kubenswrapper[5052]: E1203 11:54:59.780262 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5d0169e1-4f74-4f6e-89cc-70d2f79c34e2-dns-swift-storage-0 podName:5d0169e1-4f74-4f6e-89cc-70d2f79c34e2 nodeName:}" failed. No retries permitted until 2025-12-03 11:55:00.280239939 +0000 UTC m=+2090.868481793 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "dns-swift-storage-0" (UniqueName: "kubernetes.io/configmap/5d0169e1-4f74-4f6e-89cc-70d2f79c34e2-dns-swift-storage-0") pod "dnsmasq-dnsmasq-88667bdf5-ndf84" (UID: "5d0169e1-4f74-4f6e-89cc-70d2f79c34e2") : configmap "dns-swift-storage-0" not found Dec 03 11:55:00 crc kubenswrapper[5052]: E1203 11:55:00.286486 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/dns-swift-storage-0: configmap "dns-swift-storage-0" not found Dec 03 11:55:00 crc kubenswrapper[5052]: E1203 11:55:00.286591 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5d0169e1-4f74-4f6e-89cc-70d2f79c34e2-dns-swift-storage-0 podName:5d0169e1-4f74-4f6e-89cc-70d2f79c34e2 nodeName:}" failed. No retries permitted until 2025-12-03 11:55:01.286574126 +0000 UTC m=+2091.874815890 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "dns-swift-storage-0" (UniqueName: "kubernetes.io/configmap/5d0169e1-4f74-4f6e-89cc-70d2f79c34e2-dns-swift-storage-0") pod "dnsmasq-dnsmasq-88667bdf5-ndf84" (UID: "5d0169e1-4f74-4f6e-89cc-70d2f79c34e2") : configmap "dns-swift-storage-0" not found Dec 03 11:55:00 crc kubenswrapper[5052]: E1203 11:55:00.388349 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 11:55:00 crc kubenswrapper[5052]: E1203 11:55:00.388439 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/93d03e32-f318-470c-ad5d-f17a609f5003-operator-scripts podName:93d03e32-f318-470c-ad5d-f17a609f5003 nodeName:}" failed. No retries permitted until 2025-12-03 11:55:08.388418972 +0000 UTC m=+2098.976660736 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/93d03e32-f318-470c-ad5d-f17a609f5003-operator-scripts") pod "novacell0745f-account-delete-zbhjv" (UID: "93d03e32-f318-470c-ad5d-f17a609f5003") : configmap "openstack-scripts" not found Dec 03 11:55:00 crc kubenswrapper[5052]: E1203 11:55:00.389396 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 11:55:00 crc kubenswrapper[5052]: E1203 11:55:00.389457 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6087dc7e-081b-4392-adb6-205b2aec805f-operator-scripts podName:6087dc7e-081b-4392-adb6-205b2aec805f nodeName:}" failed. No retries permitted until 2025-12-03 11:55:08.389448912 +0000 UTC m=+2098.977690676 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/6087dc7e-081b-4392-adb6-205b2aec805f-operator-scripts") pod "novaapie4da-account-delete-mknr9" (UID: "6087dc7e-081b-4392-adb6-205b2aec805f") : configmap "openstack-scripts" not found Dec 03 11:55:01 crc kubenswrapper[5052]: E1203 11:55:01.305065 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/dns-swift-storage-0: configmap "dns-swift-storage-0" not found Dec 03 11:55:01 crc kubenswrapper[5052]: E1203 11:55:01.305404 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5d0169e1-4f74-4f6e-89cc-70d2f79c34e2-dns-swift-storage-0 podName:5d0169e1-4f74-4f6e-89cc-70d2f79c34e2 nodeName:}" failed. No retries permitted until 2025-12-03 11:55:03.305389927 +0000 UTC m=+2093.893631691 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "dns-swift-storage-0" (UniqueName: "kubernetes.io/configmap/5d0169e1-4f74-4f6e-89cc-70d2f79c34e2-dns-swift-storage-0") pod "dnsmasq-dnsmasq-88667bdf5-ndf84" (UID: "5d0169e1-4f74-4f6e-89cc-70d2f79c34e2") : configmap "dns-swift-storage-0" not found Dec 03 11:55:02 crc kubenswrapper[5052]: I1203 11:55:02.982711 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-z8rn9" Dec 03 11:55:03 crc kubenswrapper[5052]: I1203 11:55:03.041857 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-88667bdf5-ndf84"] Dec 03 11:55:03 crc kubenswrapper[5052]: I1203 11:55:03.042262 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-88667bdf5-ndf84" podUID="5d0169e1-4f74-4f6e-89cc-70d2f79c34e2" containerName="dnsmasq-dns" containerID="cri-o://5941b7d31e38a458b2b2f04a347d19ebe8ee3aa64cc1993b16459efd04d966e0" gracePeriod=10 Dec 03 11:55:03 crc kubenswrapper[5052]: E1203 11:55:03.336182 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/dns-swift-storage-0: configmap "dns-swift-storage-0" not found Dec 03 11:55:03 crc kubenswrapper[5052]: E1203 11:55:03.336269 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5d0169e1-4f74-4f6e-89cc-70d2f79c34e2-dns-swift-storage-0 podName:5d0169e1-4f74-4f6e-89cc-70d2f79c34e2 nodeName:}" failed. No retries permitted until 2025-12-03 11:55:07.336246174 +0000 UTC m=+2097.924487958 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "dns-swift-storage-0" (UniqueName: "kubernetes.io/configmap/5d0169e1-4f74-4f6e-89cc-70d2f79c34e2-dns-swift-storage-0") pod "dnsmasq-dnsmasq-88667bdf5-ndf84" (UID: "5d0169e1-4f74-4f6e-89cc-70d2f79c34e2") : configmap "dns-swift-storage-0" not found Dec 03 11:55:03 crc kubenswrapper[5052]: I1203 11:55:03.957403 5052 generic.go:334] "Generic (PLEG): container finished" podID="5d0169e1-4f74-4f6e-89cc-70d2f79c34e2" containerID="5941b7d31e38a458b2b2f04a347d19ebe8ee3aa64cc1993b16459efd04d966e0" exitCode=0 Dec 03 11:55:03 crc kubenswrapper[5052]: I1203 11:55:03.957450 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-88667bdf5-ndf84" event={"ID":"5d0169e1-4f74-4f6e-89cc-70d2f79c34e2","Type":"ContainerDied","Data":"5941b7d31e38a458b2b2f04a347d19ebe8ee3aa64cc1993b16459efd04d966e0"} Dec 03 11:55:03 crc kubenswrapper[5052]: I1203 11:55:03.957489 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-88667bdf5-ndf84" event={"ID":"5d0169e1-4f74-4f6e-89cc-70d2f79c34e2","Type":"ContainerDied","Data":"369282bf4385da505931e707f888035ed5b46cf28271b9962d446947f428c38a"} Dec 03 11:55:03 crc kubenswrapper[5052]: I1203 11:55:03.957554 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="369282bf4385da505931e707f888035ed5b46cf28271b9962d446947f428c38a" Dec 03 11:55:04 crc kubenswrapper[5052]: I1203 11:55:04.017067 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-88667bdf5-ndf84" Dec 03 11:55:04 crc kubenswrapper[5052]: I1203 11:55:04.149344 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"dns-swift-storage-0\" (UniqueName: \"kubernetes.io/configmap/5d0169e1-4f74-4f6e-89cc-70d2f79c34e2-dns-swift-storage-0\") pod \"5d0169e1-4f74-4f6e-89cc-70d2f79c34e2\" (UID: \"5d0169e1-4f74-4f6e-89cc-70d2f79c34e2\") " Dec 03 11:55:04 crc kubenswrapper[5052]: I1203 11:55:04.149398 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lqx4d\" (UniqueName: \"kubernetes.io/projected/5d0169e1-4f74-4f6e-89cc-70d2f79c34e2-kube-api-access-lqx4d\") pod \"5d0169e1-4f74-4f6e-89cc-70d2f79c34e2\" (UID: \"5d0169e1-4f74-4f6e-89cc-70d2f79c34e2\") " Dec 03 11:55:04 crc kubenswrapper[5052]: I1203 11:55:04.149459 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5d0169e1-4f74-4f6e-89cc-70d2f79c34e2-config\") pod \"5d0169e1-4f74-4f6e-89cc-70d2f79c34e2\" (UID: \"5d0169e1-4f74-4f6e-89cc-70d2f79c34e2\") " Dec 03 11:55:04 crc kubenswrapper[5052]: I1203 11:55:04.149496 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/5d0169e1-4f74-4f6e-89cc-70d2f79c34e2-dnsmasq-svc\") pod \"5d0169e1-4f74-4f6e-89cc-70d2f79c34e2\" (UID: \"5d0169e1-4f74-4f6e-89cc-70d2f79c34e2\") " Dec 03 11:55:04 crc kubenswrapper[5052]: I1203 11:55:04.159929 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5d0169e1-4f74-4f6e-89cc-70d2f79c34e2-kube-api-access-lqx4d" (OuterVolumeSpecName: "kube-api-access-lqx4d") pod "5d0169e1-4f74-4f6e-89cc-70d2f79c34e2" (UID: "5d0169e1-4f74-4f6e-89cc-70d2f79c34e2"). InnerVolumeSpecName "kube-api-access-lqx4d". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:55:04 crc kubenswrapper[5052]: I1203 11:55:04.193654 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5d0169e1-4f74-4f6e-89cc-70d2f79c34e2-dnsmasq-svc" (OuterVolumeSpecName: "dnsmasq-svc") pod "5d0169e1-4f74-4f6e-89cc-70d2f79c34e2" (UID: "5d0169e1-4f74-4f6e-89cc-70d2f79c34e2"). InnerVolumeSpecName "dnsmasq-svc". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:55:04 crc kubenswrapper[5052]: I1203 11:55:04.198704 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5d0169e1-4f74-4f6e-89cc-70d2f79c34e2-config" (OuterVolumeSpecName: "config") pod "5d0169e1-4f74-4f6e-89cc-70d2f79c34e2" (UID: "5d0169e1-4f74-4f6e-89cc-70d2f79c34e2"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:55:04 crc kubenswrapper[5052]: I1203 11:55:04.200576 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5d0169e1-4f74-4f6e-89cc-70d2f79c34e2-dns-swift-storage-0" (OuterVolumeSpecName: "dns-swift-storage-0") pod "5d0169e1-4f74-4f6e-89cc-70d2f79c34e2" (UID: "5d0169e1-4f74-4f6e-89cc-70d2f79c34e2"). InnerVolumeSpecName "dns-swift-storage-0". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:55:04 crc kubenswrapper[5052]: I1203 11:55:04.251466 5052 reconciler_common.go:293] "Volume detached for volume \"dns-swift-storage-0\" (UniqueName: \"kubernetes.io/configmap/5d0169e1-4f74-4f6e-89cc-70d2f79c34e2-dns-swift-storage-0\") on node \"crc\" DevicePath \"\"" Dec 03 11:55:04 crc kubenswrapper[5052]: I1203 11:55:04.251500 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-lqx4d\" (UniqueName: \"kubernetes.io/projected/5d0169e1-4f74-4f6e-89cc-70d2f79c34e2-kube-api-access-lqx4d\") on node \"crc\" DevicePath \"\"" Dec 03 11:55:04 crc kubenswrapper[5052]: I1203 11:55:04.251524 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5d0169e1-4f74-4f6e-89cc-70d2f79c34e2-config\") on node \"crc\" DevicePath \"\"" Dec 03 11:55:04 crc kubenswrapper[5052]: I1203 11:55:04.251533 5052 reconciler_common.go:293] "Volume detached for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/5d0169e1-4f74-4f6e-89cc-70d2f79c34e2-dnsmasq-svc\") on node \"crc\" DevicePath \"\"" Dec 03 11:55:04 crc kubenswrapper[5052]: I1203 11:55:04.968462 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-88667bdf5-ndf84" Dec 03 11:55:05 crc kubenswrapper[5052]: I1203 11:55:05.004558 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-88667bdf5-ndf84"] Dec 03 11:55:05 crc kubenswrapper[5052]: I1203 11:55:05.010066 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-88667bdf5-ndf84"] Dec 03 11:55:05 crc kubenswrapper[5052]: I1203 11:55:05.684418 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5d0169e1-4f74-4f6e-89cc-70d2f79c34e2" path="/var/lib/kubelet/pods/5d0169e1-4f74-4f6e-89cc-70d2f79c34e2/volumes" Dec 03 11:55:08 crc kubenswrapper[5052]: E1203 11:55:08.411854 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 11:55:08 crc kubenswrapper[5052]: E1203 11:55:08.411907 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 11:55:08 crc kubenswrapper[5052]: E1203 11:55:08.412098 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6087dc7e-081b-4392-adb6-205b2aec805f-operator-scripts podName:6087dc7e-081b-4392-adb6-205b2aec805f nodeName:}" failed. No retries permitted until 2025-12-03 11:55:24.412082506 +0000 UTC m=+2115.000324270 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/6087dc7e-081b-4392-adb6-205b2aec805f-operator-scripts") pod "novaapie4da-account-delete-mknr9" (UID: "6087dc7e-081b-4392-adb6-205b2aec805f") : configmap "openstack-scripts" not found Dec 03 11:55:08 crc kubenswrapper[5052]: E1203 11:55:08.412157 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/93d03e32-f318-470c-ad5d-f17a609f5003-operator-scripts podName:93d03e32-f318-470c-ad5d-f17a609f5003 nodeName:}" failed. No retries permitted until 2025-12-03 11:55:24.412131797 +0000 UTC m=+2115.000373561 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/93d03e32-f318-470c-ad5d-f17a609f5003-operator-scripts") pod "novacell0745f-account-delete-zbhjv" (UID: "93d03e32-f318-470c-ad5d-f17a609f5003") : configmap "openstack-scripts" not found Dec 03 11:55:18 crc kubenswrapper[5052]: I1203 11:55:18.009489 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-storage-0" Dec 03 11:55:18 crc kubenswrapper[5052]: I1203 11:55:18.064650 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"cache\" (UniqueName: \"kubernetes.io/empty-dir/337bf4ce-ed6a-47bb-820d-6d64cd15e16e-cache\") pod \"337bf4ce-ed6a-47bb-820d-6d64cd15e16e\" (UID: \"337bf4ce-ed6a-47bb-820d-6d64cd15e16e\") " Dec 03 11:55:18 crc kubenswrapper[5052]: I1203 11:55:18.064706 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"swift\" (UniqueName: \"kubernetes.io/local-volume/local-storage06-crc\") pod \"337bf4ce-ed6a-47bb-820d-6d64cd15e16e\" (UID: \"337bf4ce-ed6a-47bb-820d-6d64cd15e16e\") " Dec 03 11:55:18 crc kubenswrapper[5052]: I1203 11:55:18.064737 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gwmsq\" (UniqueName: \"kubernetes.io/projected/337bf4ce-ed6a-47bb-820d-6d64cd15e16e-kube-api-access-gwmsq\") pod \"337bf4ce-ed6a-47bb-820d-6d64cd15e16e\" (UID: \"337bf4ce-ed6a-47bb-820d-6d64cd15e16e\") " Dec 03 11:55:18 crc kubenswrapper[5052]: I1203 11:55:18.064790 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"lock\" (UniqueName: \"kubernetes.io/empty-dir/337bf4ce-ed6a-47bb-820d-6d64cd15e16e-lock\") pod \"337bf4ce-ed6a-47bb-820d-6d64cd15e16e\" (UID: \"337bf4ce-ed6a-47bb-820d-6d64cd15e16e\") " Dec 03 11:55:18 crc kubenswrapper[5052]: I1203 11:55:18.064812 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/337bf4ce-ed6a-47bb-820d-6d64cd15e16e-etc-swift\") pod \"337bf4ce-ed6a-47bb-820d-6d64cd15e16e\" (UID: \"337bf4ce-ed6a-47bb-820d-6d64cd15e16e\") " Dec 03 11:55:18 crc kubenswrapper[5052]: I1203 11:55:18.065440 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/337bf4ce-ed6a-47bb-820d-6d64cd15e16e-cache" (OuterVolumeSpecName: "cache") pod "337bf4ce-ed6a-47bb-820d-6d64cd15e16e" (UID: "337bf4ce-ed6a-47bb-820d-6d64cd15e16e"). InnerVolumeSpecName "cache". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:55:18 crc kubenswrapper[5052]: I1203 11:55:18.066082 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/337bf4ce-ed6a-47bb-820d-6d64cd15e16e-lock" (OuterVolumeSpecName: "lock") pod "337bf4ce-ed6a-47bb-820d-6d64cd15e16e" (UID: "337bf4ce-ed6a-47bb-820d-6d64cd15e16e"). InnerVolumeSpecName "lock". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:55:18 crc kubenswrapper[5052]: I1203 11:55:18.070317 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/337bf4ce-ed6a-47bb-820d-6d64cd15e16e-kube-api-access-gwmsq" (OuterVolumeSpecName: "kube-api-access-gwmsq") pod "337bf4ce-ed6a-47bb-820d-6d64cd15e16e" (UID: "337bf4ce-ed6a-47bb-820d-6d64cd15e16e"). InnerVolumeSpecName "kube-api-access-gwmsq". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:55:18 crc kubenswrapper[5052]: I1203 11:55:18.070522 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/337bf4ce-ed6a-47bb-820d-6d64cd15e16e-etc-swift" (OuterVolumeSpecName: "etc-swift") pod "337bf4ce-ed6a-47bb-820d-6d64cd15e16e" (UID: "337bf4ce-ed6a-47bb-820d-6d64cd15e16e"). InnerVolumeSpecName "etc-swift". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:55:18 crc kubenswrapper[5052]: I1203 11:55:18.070854 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage06-crc" (OuterVolumeSpecName: "swift") pod "337bf4ce-ed6a-47bb-820d-6d64cd15e16e" (UID: "337bf4ce-ed6a-47bb-820d-6d64cd15e16e"). InnerVolumeSpecName "local-storage06-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 11:55:18 crc kubenswrapper[5052]: I1203 11:55:18.107390 5052 generic.go:334] "Generic (PLEG): container finished" podID="337bf4ce-ed6a-47bb-820d-6d64cd15e16e" containerID="adcb7e1e3f0cfa3f2f565b2ee561330619830866aab79c1bb82dc8a5cacf862d" exitCode=137 Dec 03 11:55:18 crc kubenswrapper[5052]: I1203 11:55:18.107449 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"337bf4ce-ed6a-47bb-820d-6d64cd15e16e","Type":"ContainerDied","Data":"adcb7e1e3f0cfa3f2f565b2ee561330619830866aab79c1bb82dc8a5cacf862d"} Dec 03 11:55:18 crc kubenswrapper[5052]: I1203 11:55:18.107786 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"337bf4ce-ed6a-47bb-820d-6d64cd15e16e","Type":"ContainerDied","Data":"fc539dbf46bdf8b30ab4b7079777cd46ae1b263b2a2b5b2e7afc75cabc6b5735"} Dec 03 11:55:18 crc kubenswrapper[5052]: I1203 11:55:18.107493 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-storage-0" Dec 03 11:55:18 crc kubenswrapper[5052]: I1203 11:55:18.107852 5052 scope.go:117] "RemoveContainer" containerID="adcb7e1e3f0cfa3f2f565b2ee561330619830866aab79c1bb82dc8a5cacf862d" Dec 03 11:55:18 crc kubenswrapper[5052]: I1203 11:55:18.151463 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/swift-storage-0"] Dec 03 11:55:18 crc kubenswrapper[5052]: I1203 11:55:18.151842 5052 scope.go:117] "RemoveContainer" containerID="54a021ff30a08f0f3bd288edbc672e78a69a0e8398c93034e0304c71564937f1" Dec 03 11:55:18 crc kubenswrapper[5052]: I1203 11:55:18.156314 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/swift-storage-0"] Dec 03 11:55:18 crc kubenswrapper[5052]: I1203 11:55:18.166745 5052 reconciler_common.go:293] "Volume detached for volume \"cache\" (UniqueName: \"kubernetes.io/empty-dir/337bf4ce-ed6a-47bb-820d-6d64cd15e16e-cache\") on node \"crc\" DevicePath \"\"" Dec 03 11:55:18 crc kubenswrapper[5052]: I1203 11:55:18.166813 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage06-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage06-crc\") on node \"crc\" " Dec 03 11:55:18 crc kubenswrapper[5052]: I1203 11:55:18.166829 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-gwmsq\" (UniqueName: \"kubernetes.io/projected/337bf4ce-ed6a-47bb-820d-6d64cd15e16e-kube-api-access-gwmsq\") on node \"crc\" DevicePath \"\"" Dec 03 11:55:18 crc kubenswrapper[5052]: I1203 11:55:18.166842 5052 reconciler_common.go:293] "Volume detached for volume \"lock\" (UniqueName: \"kubernetes.io/empty-dir/337bf4ce-ed6a-47bb-820d-6d64cd15e16e-lock\") on node \"crc\" DevicePath \"\"" Dec 03 11:55:18 crc kubenswrapper[5052]: I1203 11:55:18.166855 5052 reconciler_common.go:293] "Volume detached for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/337bf4ce-ed6a-47bb-820d-6d64cd15e16e-etc-swift\") on node \"crc\" DevicePath \"\"" Dec 03 11:55:18 crc kubenswrapper[5052]: I1203 11:55:18.171223 5052 scope.go:117] "RemoveContainer" containerID="308f757a4c5baeeb088bbd69071d883ee0fd5f6717ce1fb801584cc7308cf01e" Dec 03 11:55:18 crc kubenswrapper[5052]: I1203 11:55:18.180871 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage06-crc" (UniqueName: "kubernetes.io/local-volume/local-storage06-crc") on node "crc" Dec 03 11:55:18 crc kubenswrapper[5052]: I1203 11:55:18.191023 5052 scope.go:117] "RemoveContainer" containerID="0bdc5789ab44d02d9702c5cf3e475db9c7a7ce6bf079ab5b06570c12673c510c" Dec 03 11:55:18 crc kubenswrapper[5052]: I1203 11:55:18.207655 5052 scope.go:117] "RemoveContainer" containerID="ef2775155138fd30b620bc0fbab9074d6f7cd4364b1cbc82c977fd6b48054772" Dec 03 11:55:18 crc kubenswrapper[5052]: I1203 11:55:18.223266 5052 scope.go:117] "RemoveContainer" containerID="bf4d57ce3d071492f7492f8995fdfdb4a50c770c3ee835e5864c7889d9aca849" Dec 03 11:55:18 crc kubenswrapper[5052]: I1203 11:55:18.242944 5052 scope.go:117] "RemoveContainer" containerID="6b9be92cb0d9072365edb2b4115b091dd2b7a90d8f70424fe8b36078d19bcab6" Dec 03 11:55:18 crc kubenswrapper[5052]: I1203 11:55:18.264805 5052 scope.go:117] "RemoveContainer" containerID="aeee80ed3e9e151ab7e9e179efcf5113a7eda44c709ef92d44f8e5e51a28368d" Dec 03 11:55:18 crc kubenswrapper[5052]: I1203 11:55:18.269149 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage06-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage06-crc\") on node \"crc\" DevicePath \"\"" Dec 03 11:55:18 crc kubenswrapper[5052]: I1203 11:55:18.284580 5052 scope.go:117] "RemoveContainer" containerID="6b146d4fbd455766a79f81d255c8552e4579557dae733283b1a6647e1871ce4a" Dec 03 11:55:18 crc kubenswrapper[5052]: I1203 11:55:18.304123 5052 scope.go:117] "RemoveContainer" containerID="bf27f4652a0cd9c65c3371e4a8f8b663bd4dc1a05420dbf79f77e2f00185a31d" Dec 03 11:55:18 crc kubenswrapper[5052]: I1203 11:55:18.322705 5052 scope.go:117] "RemoveContainer" containerID="cac91cb8a133117b6ac9ef92818b0c43f08504bb99f13b28c87c24ca86efd6f6" Dec 03 11:55:18 crc kubenswrapper[5052]: I1203 11:55:18.339995 5052 scope.go:117] "RemoveContainer" containerID="187fe18924b493edafcac4ae2021f9d0b6680c618116eb5301da25207a0518fb" Dec 03 11:55:18 crc kubenswrapper[5052]: I1203 11:55:18.357371 5052 scope.go:117] "RemoveContainer" containerID="36f10fc8c3ae11fad49f741b63364021479706c1dcab5d1f51a1da9b35e558c2" Dec 03 11:55:18 crc kubenswrapper[5052]: I1203 11:55:18.373894 5052 scope.go:117] "RemoveContainer" containerID="1572718ba303c5fe3c2aff8c645fa6d74e146847e7945778af6c12cfd1f31c90" Dec 03 11:55:18 crc kubenswrapper[5052]: I1203 11:55:18.390909 5052 scope.go:117] "RemoveContainer" containerID="da24bea2b91d001778224eaa9d20a940c1cfa8aec9ca78a78f4063dc10e39b5e" Dec 03 11:55:18 crc kubenswrapper[5052]: I1203 11:55:18.408716 5052 scope.go:117] "RemoveContainer" containerID="adcb7e1e3f0cfa3f2f565b2ee561330619830866aab79c1bb82dc8a5cacf862d" Dec 03 11:55:18 crc kubenswrapper[5052]: E1203 11:55:18.409140 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"adcb7e1e3f0cfa3f2f565b2ee561330619830866aab79c1bb82dc8a5cacf862d\": container with ID starting with adcb7e1e3f0cfa3f2f565b2ee561330619830866aab79c1bb82dc8a5cacf862d not found: ID does not exist" containerID="adcb7e1e3f0cfa3f2f565b2ee561330619830866aab79c1bb82dc8a5cacf862d" Dec 03 11:55:18 crc kubenswrapper[5052]: I1203 11:55:18.409173 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"adcb7e1e3f0cfa3f2f565b2ee561330619830866aab79c1bb82dc8a5cacf862d"} err="failed to get container status \"adcb7e1e3f0cfa3f2f565b2ee561330619830866aab79c1bb82dc8a5cacf862d\": rpc error: code = NotFound desc = could not find container \"adcb7e1e3f0cfa3f2f565b2ee561330619830866aab79c1bb82dc8a5cacf862d\": container with ID starting with adcb7e1e3f0cfa3f2f565b2ee561330619830866aab79c1bb82dc8a5cacf862d not found: ID does not exist" Dec 03 11:55:18 crc kubenswrapper[5052]: I1203 11:55:18.409197 5052 scope.go:117] "RemoveContainer" containerID="54a021ff30a08f0f3bd288edbc672e78a69a0e8398c93034e0304c71564937f1" Dec 03 11:55:18 crc kubenswrapper[5052]: E1203 11:55:18.409447 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"54a021ff30a08f0f3bd288edbc672e78a69a0e8398c93034e0304c71564937f1\": container with ID starting with 54a021ff30a08f0f3bd288edbc672e78a69a0e8398c93034e0304c71564937f1 not found: ID does not exist" containerID="54a021ff30a08f0f3bd288edbc672e78a69a0e8398c93034e0304c71564937f1" Dec 03 11:55:18 crc kubenswrapper[5052]: I1203 11:55:18.409488 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"54a021ff30a08f0f3bd288edbc672e78a69a0e8398c93034e0304c71564937f1"} err="failed to get container status \"54a021ff30a08f0f3bd288edbc672e78a69a0e8398c93034e0304c71564937f1\": rpc error: code = NotFound desc = could not find container \"54a021ff30a08f0f3bd288edbc672e78a69a0e8398c93034e0304c71564937f1\": container with ID starting with 54a021ff30a08f0f3bd288edbc672e78a69a0e8398c93034e0304c71564937f1 not found: ID does not exist" Dec 03 11:55:18 crc kubenswrapper[5052]: I1203 11:55:18.409535 5052 scope.go:117] "RemoveContainer" containerID="308f757a4c5baeeb088bbd69071d883ee0fd5f6717ce1fb801584cc7308cf01e" Dec 03 11:55:18 crc kubenswrapper[5052]: E1203 11:55:18.410024 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"308f757a4c5baeeb088bbd69071d883ee0fd5f6717ce1fb801584cc7308cf01e\": container with ID starting with 308f757a4c5baeeb088bbd69071d883ee0fd5f6717ce1fb801584cc7308cf01e not found: ID does not exist" containerID="308f757a4c5baeeb088bbd69071d883ee0fd5f6717ce1fb801584cc7308cf01e" Dec 03 11:55:18 crc kubenswrapper[5052]: I1203 11:55:18.410053 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"308f757a4c5baeeb088bbd69071d883ee0fd5f6717ce1fb801584cc7308cf01e"} err="failed to get container status \"308f757a4c5baeeb088bbd69071d883ee0fd5f6717ce1fb801584cc7308cf01e\": rpc error: code = NotFound desc = could not find container \"308f757a4c5baeeb088bbd69071d883ee0fd5f6717ce1fb801584cc7308cf01e\": container with ID starting with 308f757a4c5baeeb088bbd69071d883ee0fd5f6717ce1fb801584cc7308cf01e not found: ID does not exist" Dec 03 11:55:18 crc kubenswrapper[5052]: I1203 11:55:18.410066 5052 scope.go:117] "RemoveContainer" containerID="0bdc5789ab44d02d9702c5cf3e475db9c7a7ce6bf079ab5b06570c12673c510c" Dec 03 11:55:18 crc kubenswrapper[5052]: E1203 11:55:18.410250 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0bdc5789ab44d02d9702c5cf3e475db9c7a7ce6bf079ab5b06570c12673c510c\": container with ID starting with 0bdc5789ab44d02d9702c5cf3e475db9c7a7ce6bf079ab5b06570c12673c510c not found: ID does not exist" containerID="0bdc5789ab44d02d9702c5cf3e475db9c7a7ce6bf079ab5b06570c12673c510c" Dec 03 11:55:18 crc kubenswrapper[5052]: I1203 11:55:18.410273 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0bdc5789ab44d02d9702c5cf3e475db9c7a7ce6bf079ab5b06570c12673c510c"} err="failed to get container status \"0bdc5789ab44d02d9702c5cf3e475db9c7a7ce6bf079ab5b06570c12673c510c\": rpc error: code = NotFound desc = could not find container \"0bdc5789ab44d02d9702c5cf3e475db9c7a7ce6bf079ab5b06570c12673c510c\": container with ID starting with 0bdc5789ab44d02d9702c5cf3e475db9c7a7ce6bf079ab5b06570c12673c510c not found: ID does not exist" Dec 03 11:55:18 crc kubenswrapper[5052]: I1203 11:55:18.410287 5052 scope.go:117] "RemoveContainer" containerID="ef2775155138fd30b620bc0fbab9074d6f7cd4364b1cbc82c977fd6b48054772" Dec 03 11:55:18 crc kubenswrapper[5052]: E1203 11:55:18.410457 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ef2775155138fd30b620bc0fbab9074d6f7cd4364b1cbc82c977fd6b48054772\": container with ID starting with ef2775155138fd30b620bc0fbab9074d6f7cd4364b1cbc82c977fd6b48054772 not found: ID does not exist" containerID="ef2775155138fd30b620bc0fbab9074d6f7cd4364b1cbc82c977fd6b48054772" Dec 03 11:55:18 crc kubenswrapper[5052]: I1203 11:55:18.410475 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ef2775155138fd30b620bc0fbab9074d6f7cd4364b1cbc82c977fd6b48054772"} err="failed to get container status \"ef2775155138fd30b620bc0fbab9074d6f7cd4364b1cbc82c977fd6b48054772\": rpc error: code = NotFound desc = could not find container \"ef2775155138fd30b620bc0fbab9074d6f7cd4364b1cbc82c977fd6b48054772\": container with ID starting with ef2775155138fd30b620bc0fbab9074d6f7cd4364b1cbc82c977fd6b48054772 not found: ID does not exist" Dec 03 11:55:18 crc kubenswrapper[5052]: I1203 11:55:18.410486 5052 scope.go:117] "RemoveContainer" containerID="bf4d57ce3d071492f7492f8995fdfdb4a50c770c3ee835e5864c7889d9aca849" Dec 03 11:55:18 crc kubenswrapper[5052]: E1203 11:55:18.410858 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bf4d57ce3d071492f7492f8995fdfdb4a50c770c3ee835e5864c7889d9aca849\": container with ID starting with bf4d57ce3d071492f7492f8995fdfdb4a50c770c3ee835e5864c7889d9aca849 not found: ID does not exist" containerID="bf4d57ce3d071492f7492f8995fdfdb4a50c770c3ee835e5864c7889d9aca849" Dec 03 11:55:18 crc kubenswrapper[5052]: I1203 11:55:18.410904 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bf4d57ce3d071492f7492f8995fdfdb4a50c770c3ee835e5864c7889d9aca849"} err="failed to get container status \"bf4d57ce3d071492f7492f8995fdfdb4a50c770c3ee835e5864c7889d9aca849\": rpc error: code = NotFound desc = could not find container \"bf4d57ce3d071492f7492f8995fdfdb4a50c770c3ee835e5864c7889d9aca849\": container with ID starting with bf4d57ce3d071492f7492f8995fdfdb4a50c770c3ee835e5864c7889d9aca849 not found: ID does not exist" Dec 03 11:55:18 crc kubenswrapper[5052]: I1203 11:55:18.410933 5052 scope.go:117] "RemoveContainer" containerID="6b9be92cb0d9072365edb2b4115b091dd2b7a90d8f70424fe8b36078d19bcab6" Dec 03 11:55:18 crc kubenswrapper[5052]: E1203 11:55:18.411281 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6b9be92cb0d9072365edb2b4115b091dd2b7a90d8f70424fe8b36078d19bcab6\": container with ID starting with 6b9be92cb0d9072365edb2b4115b091dd2b7a90d8f70424fe8b36078d19bcab6 not found: ID does not exist" containerID="6b9be92cb0d9072365edb2b4115b091dd2b7a90d8f70424fe8b36078d19bcab6" Dec 03 11:55:18 crc kubenswrapper[5052]: I1203 11:55:18.411355 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6b9be92cb0d9072365edb2b4115b091dd2b7a90d8f70424fe8b36078d19bcab6"} err="failed to get container status \"6b9be92cb0d9072365edb2b4115b091dd2b7a90d8f70424fe8b36078d19bcab6\": rpc error: code = NotFound desc = could not find container \"6b9be92cb0d9072365edb2b4115b091dd2b7a90d8f70424fe8b36078d19bcab6\": container with ID starting with 6b9be92cb0d9072365edb2b4115b091dd2b7a90d8f70424fe8b36078d19bcab6 not found: ID does not exist" Dec 03 11:55:18 crc kubenswrapper[5052]: I1203 11:55:18.411379 5052 scope.go:117] "RemoveContainer" containerID="aeee80ed3e9e151ab7e9e179efcf5113a7eda44c709ef92d44f8e5e51a28368d" Dec 03 11:55:18 crc kubenswrapper[5052]: E1203 11:55:18.411635 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aeee80ed3e9e151ab7e9e179efcf5113a7eda44c709ef92d44f8e5e51a28368d\": container with ID starting with aeee80ed3e9e151ab7e9e179efcf5113a7eda44c709ef92d44f8e5e51a28368d not found: ID does not exist" containerID="aeee80ed3e9e151ab7e9e179efcf5113a7eda44c709ef92d44f8e5e51a28368d" Dec 03 11:55:18 crc kubenswrapper[5052]: I1203 11:55:18.411658 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aeee80ed3e9e151ab7e9e179efcf5113a7eda44c709ef92d44f8e5e51a28368d"} err="failed to get container status \"aeee80ed3e9e151ab7e9e179efcf5113a7eda44c709ef92d44f8e5e51a28368d\": rpc error: code = NotFound desc = could not find container \"aeee80ed3e9e151ab7e9e179efcf5113a7eda44c709ef92d44f8e5e51a28368d\": container with ID starting with aeee80ed3e9e151ab7e9e179efcf5113a7eda44c709ef92d44f8e5e51a28368d not found: ID does not exist" Dec 03 11:55:18 crc kubenswrapper[5052]: I1203 11:55:18.411673 5052 scope.go:117] "RemoveContainer" containerID="6b146d4fbd455766a79f81d255c8552e4579557dae733283b1a6647e1871ce4a" Dec 03 11:55:18 crc kubenswrapper[5052]: E1203 11:55:18.411881 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6b146d4fbd455766a79f81d255c8552e4579557dae733283b1a6647e1871ce4a\": container with ID starting with 6b146d4fbd455766a79f81d255c8552e4579557dae733283b1a6647e1871ce4a not found: ID does not exist" containerID="6b146d4fbd455766a79f81d255c8552e4579557dae733283b1a6647e1871ce4a" Dec 03 11:55:18 crc kubenswrapper[5052]: I1203 11:55:18.411920 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6b146d4fbd455766a79f81d255c8552e4579557dae733283b1a6647e1871ce4a"} err="failed to get container status \"6b146d4fbd455766a79f81d255c8552e4579557dae733283b1a6647e1871ce4a\": rpc error: code = NotFound desc = could not find container \"6b146d4fbd455766a79f81d255c8552e4579557dae733283b1a6647e1871ce4a\": container with ID starting with 6b146d4fbd455766a79f81d255c8552e4579557dae733283b1a6647e1871ce4a not found: ID does not exist" Dec 03 11:55:18 crc kubenswrapper[5052]: I1203 11:55:18.411946 5052 scope.go:117] "RemoveContainer" containerID="bf27f4652a0cd9c65c3371e4a8f8b663bd4dc1a05420dbf79f77e2f00185a31d" Dec 03 11:55:18 crc kubenswrapper[5052]: E1203 11:55:18.412154 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bf27f4652a0cd9c65c3371e4a8f8b663bd4dc1a05420dbf79f77e2f00185a31d\": container with ID starting with bf27f4652a0cd9c65c3371e4a8f8b663bd4dc1a05420dbf79f77e2f00185a31d not found: ID does not exist" containerID="bf27f4652a0cd9c65c3371e4a8f8b663bd4dc1a05420dbf79f77e2f00185a31d" Dec 03 11:55:18 crc kubenswrapper[5052]: I1203 11:55:18.412179 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bf27f4652a0cd9c65c3371e4a8f8b663bd4dc1a05420dbf79f77e2f00185a31d"} err="failed to get container status \"bf27f4652a0cd9c65c3371e4a8f8b663bd4dc1a05420dbf79f77e2f00185a31d\": rpc error: code = NotFound desc = could not find container \"bf27f4652a0cd9c65c3371e4a8f8b663bd4dc1a05420dbf79f77e2f00185a31d\": container with ID starting with bf27f4652a0cd9c65c3371e4a8f8b663bd4dc1a05420dbf79f77e2f00185a31d not found: ID does not exist" Dec 03 11:55:18 crc kubenswrapper[5052]: I1203 11:55:18.412194 5052 scope.go:117] "RemoveContainer" containerID="cac91cb8a133117b6ac9ef92818b0c43f08504bb99f13b28c87c24ca86efd6f6" Dec 03 11:55:18 crc kubenswrapper[5052]: E1203 11:55:18.412420 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cac91cb8a133117b6ac9ef92818b0c43f08504bb99f13b28c87c24ca86efd6f6\": container with ID starting with cac91cb8a133117b6ac9ef92818b0c43f08504bb99f13b28c87c24ca86efd6f6 not found: ID does not exist" containerID="cac91cb8a133117b6ac9ef92818b0c43f08504bb99f13b28c87c24ca86efd6f6" Dec 03 11:55:18 crc kubenswrapper[5052]: I1203 11:55:18.412461 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cac91cb8a133117b6ac9ef92818b0c43f08504bb99f13b28c87c24ca86efd6f6"} err="failed to get container status \"cac91cb8a133117b6ac9ef92818b0c43f08504bb99f13b28c87c24ca86efd6f6\": rpc error: code = NotFound desc = could not find container \"cac91cb8a133117b6ac9ef92818b0c43f08504bb99f13b28c87c24ca86efd6f6\": container with ID starting with cac91cb8a133117b6ac9ef92818b0c43f08504bb99f13b28c87c24ca86efd6f6 not found: ID does not exist" Dec 03 11:55:18 crc kubenswrapper[5052]: I1203 11:55:18.412488 5052 scope.go:117] "RemoveContainer" containerID="187fe18924b493edafcac4ae2021f9d0b6680c618116eb5301da25207a0518fb" Dec 03 11:55:18 crc kubenswrapper[5052]: E1203 11:55:18.412738 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"187fe18924b493edafcac4ae2021f9d0b6680c618116eb5301da25207a0518fb\": container with ID starting with 187fe18924b493edafcac4ae2021f9d0b6680c618116eb5301da25207a0518fb not found: ID does not exist" containerID="187fe18924b493edafcac4ae2021f9d0b6680c618116eb5301da25207a0518fb" Dec 03 11:55:18 crc kubenswrapper[5052]: I1203 11:55:18.412778 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"187fe18924b493edafcac4ae2021f9d0b6680c618116eb5301da25207a0518fb"} err="failed to get container status \"187fe18924b493edafcac4ae2021f9d0b6680c618116eb5301da25207a0518fb\": rpc error: code = NotFound desc = could not find container \"187fe18924b493edafcac4ae2021f9d0b6680c618116eb5301da25207a0518fb\": container with ID starting with 187fe18924b493edafcac4ae2021f9d0b6680c618116eb5301da25207a0518fb not found: ID does not exist" Dec 03 11:55:18 crc kubenswrapper[5052]: I1203 11:55:18.412804 5052 scope.go:117] "RemoveContainer" containerID="36f10fc8c3ae11fad49f741b63364021479706c1dcab5d1f51a1da9b35e558c2" Dec 03 11:55:18 crc kubenswrapper[5052]: E1203 11:55:18.413026 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"36f10fc8c3ae11fad49f741b63364021479706c1dcab5d1f51a1da9b35e558c2\": container with ID starting with 36f10fc8c3ae11fad49f741b63364021479706c1dcab5d1f51a1da9b35e558c2 not found: ID does not exist" containerID="36f10fc8c3ae11fad49f741b63364021479706c1dcab5d1f51a1da9b35e558c2" Dec 03 11:55:18 crc kubenswrapper[5052]: I1203 11:55:18.413063 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"36f10fc8c3ae11fad49f741b63364021479706c1dcab5d1f51a1da9b35e558c2"} err="failed to get container status \"36f10fc8c3ae11fad49f741b63364021479706c1dcab5d1f51a1da9b35e558c2\": rpc error: code = NotFound desc = could not find container \"36f10fc8c3ae11fad49f741b63364021479706c1dcab5d1f51a1da9b35e558c2\": container with ID starting with 36f10fc8c3ae11fad49f741b63364021479706c1dcab5d1f51a1da9b35e558c2 not found: ID does not exist" Dec 03 11:55:18 crc kubenswrapper[5052]: I1203 11:55:18.413081 5052 scope.go:117] "RemoveContainer" containerID="1572718ba303c5fe3c2aff8c645fa6d74e146847e7945778af6c12cfd1f31c90" Dec 03 11:55:18 crc kubenswrapper[5052]: E1203 11:55:18.413262 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1572718ba303c5fe3c2aff8c645fa6d74e146847e7945778af6c12cfd1f31c90\": container with ID starting with 1572718ba303c5fe3c2aff8c645fa6d74e146847e7945778af6c12cfd1f31c90 not found: ID does not exist" containerID="1572718ba303c5fe3c2aff8c645fa6d74e146847e7945778af6c12cfd1f31c90" Dec 03 11:55:18 crc kubenswrapper[5052]: I1203 11:55:18.413299 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1572718ba303c5fe3c2aff8c645fa6d74e146847e7945778af6c12cfd1f31c90"} err="failed to get container status \"1572718ba303c5fe3c2aff8c645fa6d74e146847e7945778af6c12cfd1f31c90\": rpc error: code = NotFound desc = could not find container \"1572718ba303c5fe3c2aff8c645fa6d74e146847e7945778af6c12cfd1f31c90\": container with ID starting with 1572718ba303c5fe3c2aff8c645fa6d74e146847e7945778af6c12cfd1f31c90 not found: ID does not exist" Dec 03 11:55:18 crc kubenswrapper[5052]: I1203 11:55:18.413315 5052 scope.go:117] "RemoveContainer" containerID="da24bea2b91d001778224eaa9d20a940c1cfa8aec9ca78a78f4063dc10e39b5e" Dec 03 11:55:18 crc kubenswrapper[5052]: E1203 11:55:18.413557 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"da24bea2b91d001778224eaa9d20a940c1cfa8aec9ca78a78f4063dc10e39b5e\": container with ID starting with da24bea2b91d001778224eaa9d20a940c1cfa8aec9ca78a78f4063dc10e39b5e not found: ID does not exist" containerID="da24bea2b91d001778224eaa9d20a940c1cfa8aec9ca78a78f4063dc10e39b5e" Dec 03 11:55:18 crc kubenswrapper[5052]: I1203 11:55:18.413593 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"da24bea2b91d001778224eaa9d20a940c1cfa8aec9ca78a78f4063dc10e39b5e"} err="failed to get container status \"da24bea2b91d001778224eaa9d20a940c1cfa8aec9ca78a78f4063dc10e39b5e\": rpc error: code = NotFound desc = could not find container \"da24bea2b91d001778224eaa9d20a940c1cfa8aec9ca78a78f4063dc10e39b5e\": container with ID starting with da24bea2b91d001778224eaa9d20a940c1cfa8aec9ca78a78f4063dc10e39b5e not found: ID does not exist" Dec 03 11:55:19 crc kubenswrapper[5052]: I1203 11:55:19.683370 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="337bf4ce-ed6a-47bb-820d-6d64cd15e16e" path="/var/lib/kubelet/pods/337bf4ce-ed6a-47bb-820d-6d64cd15e16e/volumes" Dec 03 11:55:20 crc kubenswrapper[5052]: I1203 11:55:20.219235 5052 pod_container_manager_linux.go:210] "Failed to delete cgroup paths" cgroupName=["kubepods","besteffort","pod9d8cd78f-2670-4933-81dd-01cdc0044454"] err="unable to destroy cgroup paths for cgroup [kubepods besteffort pod9d8cd78f-2670-4933-81dd-01cdc0044454] : Timed out while waiting for systemd to remove kubepods-besteffort-pod9d8cd78f_2670_4933_81dd_01cdc0044454.slice" Dec 03 11:55:20 crc kubenswrapper[5052]: E1203 11:55:20.219362 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to delete cgroup paths for [kubepods besteffort pod9d8cd78f-2670-4933-81dd-01cdc0044454] : unable to destroy cgroup paths for cgroup [kubepods besteffort pod9d8cd78f-2670-4933-81dd-01cdc0044454] : Timed out while waiting for systemd to remove kubepods-besteffort-pod9d8cd78f_2670_4933_81dd_01cdc0044454.slice" pod="openstack-kuttl-tests/ovn-northd-0" podUID="9d8cd78f-2670-4933-81dd-01cdc0044454" Dec 03 11:55:20 crc kubenswrapper[5052]: I1203 11:55:20.361000 5052 pod_container_manager_linux.go:210] "Failed to delete cgroup paths" cgroupName=["kubepods","besteffort","pod4c81b4dc-4f69-4447-8e5e-043aa0c7dd59"] err="unable to destroy cgroup paths for cgroup [kubepods besteffort pod4c81b4dc-4f69-4447-8e5e-043aa0c7dd59] : Timed out while waiting for systemd to remove kubepods-besteffort-pod4c81b4dc_4f69_4447_8e5e_043aa0c7dd59.slice" Dec 03 11:55:20 crc kubenswrapper[5052]: E1203 11:55:20.361066 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to delete cgroup paths for [kubepods besteffort pod4c81b4dc-4f69-4447-8e5e-043aa0c7dd59] : unable to destroy cgroup paths for cgroup [kubepods besteffort pod4c81b4dc-4f69-4447-8e5e-043aa0c7dd59] : Timed out while waiting for systemd to remove kubepods-besteffort-pod4c81b4dc_4f69_4447_8e5e_043aa0c7dd59.slice" pod="openstack-kuttl-tests/openstackclient" podUID="4c81b4dc-4f69-4447-8e5e-043aa0c7dd59" Dec 03 11:55:21 crc kubenswrapper[5052]: I1203 11:55:21.134811 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstackclient" Dec 03 11:55:21 crc kubenswrapper[5052]: I1203 11:55:21.134861 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 11:55:21 crc kubenswrapper[5052]: I1203 11:55:21.163003 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ovn-northd-0"] Dec 03 11:55:21 crc kubenswrapper[5052]: I1203 11:55:21.169500 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ovn-northd-0"] Dec 03 11:55:21 crc kubenswrapper[5052]: I1203 11:55:21.684631 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9d8cd78f-2670-4933-81dd-01cdc0044454" path="/var/lib/kubelet/pods/9d8cd78f-2670-4933-81dd-01cdc0044454/volumes" Dec 03 11:55:21 crc kubenswrapper[5052]: I1203 11:55:21.778133 5052 pod_container_manager_linux.go:210] "Failed to delete cgroup paths" cgroupName=["kubepods","besteffort","podda3a810d-9fd0-4f18-8f4a-e896b94b1431"] err="unable to destroy cgroup paths for cgroup [kubepods besteffort podda3a810d-9fd0-4f18-8f4a-e896b94b1431] : Timed out while waiting for systemd to remove kubepods-besteffort-podda3a810d_9fd0_4f18_8f4a_e896b94b1431.slice" Dec 03 11:55:21 crc kubenswrapper[5052]: E1203 11:55:21.778193 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to delete cgroup paths for [kubepods besteffort podda3a810d-9fd0-4f18-8f4a-e896b94b1431] : unable to destroy cgroup paths for cgroup [kubepods besteffort podda3a810d-9fd0-4f18-8f4a-e896b94b1431] : Timed out while waiting for systemd to remove kubepods-besteffort-podda3a810d_9fd0_4f18_8f4a_e896b94b1431.slice" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" podUID="da3a810d-9fd0-4f18-8f4a-e896b94b1431" Dec 03 11:55:22 crc kubenswrapper[5052]: I1203 11:55:22.153620 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 11:55:22 crc kubenswrapper[5052]: I1203 11:55:22.176317 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 11:55:22 crc kubenswrapper[5052]: I1203 11:55:22.187599 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 11:55:23 crc kubenswrapper[5052]: I1203 11:55:23.171565 5052 generic.go:334] "Generic (PLEG): container finished" podID="0bda7b03-76fd-405e-9a69-b897331e829c" containerID="3ee6799c23947cd1ca061d572689f0e21a053ace392723fbac989b50aff781bb" exitCode=137 Dec 03 11:55:23 crc kubenswrapper[5052]: I1203 11:55:23.171651 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement2868-account-delete-gtkfr" event={"ID":"0bda7b03-76fd-405e-9a69-b897331e829c","Type":"ContainerDied","Data":"3ee6799c23947cd1ca061d572689f0e21a053ace392723fbac989b50aff781bb"} Dec 03 11:55:23 crc kubenswrapper[5052]: I1203 11:55:23.177574 5052 generic.go:334] "Generic (PLEG): container finished" podID="0f42494c-058e-417d-9e0d-97ce7040b288" containerID="39d19d353a9f922fcda81960e089ebd7d426e58421f5f1bd9b9dc1951175ebeb" exitCode=137 Dec 03 11:55:23 crc kubenswrapper[5052]: I1203 11:55:23.177604 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron3cf1-account-delete-zk6qw" event={"ID":"0f42494c-058e-417d-9e0d-97ce7040b288","Type":"ContainerDied","Data":"39d19d353a9f922fcda81960e089ebd7d426e58421f5f1bd9b9dc1951175ebeb"} Dec 03 11:55:23 crc kubenswrapper[5052]: I1203 11:55:23.315084 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement2868-account-delete-gtkfr" Dec 03 11:55:23 crc kubenswrapper[5052]: I1203 11:55:23.445417 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/0bda7b03-76fd-405e-9a69-b897331e829c-operator-scripts\") pod \"0bda7b03-76fd-405e-9a69-b897331e829c\" (UID: \"0bda7b03-76fd-405e-9a69-b897331e829c\") " Dec 03 11:55:23 crc kubenswrapper[5052]: I1203 11:55:23.445483 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b2d4g\" (UniqueName: \"kubernetes.io/projected/0bda7b03-76fd-405e-9a69-b897331e829c-kube-api-access-b2d4g\") pod \"0bda7b03-76fd-405e-9a69-b897331e829c\" (UID: \"0bda7b03-76fd-405e-9a69-b897331e829c\") " Dec 03 11:55:23 crc kubenswrapper[5052]: I1203 11:55:23.446331 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0bda7b03-76fd-405e-9a69-b897331e829c-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "0bda7b03-76fd-405e-9a69-b897331e829c" (UID: "0bda7b03-76fd-405e-9a69-b897331e829c"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:55:23 crc kubenswrapper[5052]: I1203 11:55:23.451389 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0bda7b03-76fd-405e-9a69-b897331e829c-kube-api-access-b2d4g" (OuterVolumeSpecName: "kube-api-access-b2d4g") pod "0bda7b03-76fd-405e-9a69-b897331e829c" (UID: "0bda7b03-76fd-405e-9a69-b897331e829c"). InnerVolumeSpecName "kube-api-access-b2d4g". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:55:23 crc kubenswrapper[5052]: I1203 11:55:23.548407 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/0bda7b03-76fd-405e-9a69-b897331e829c-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:55:23 crc kubenswrapper[5052]: I1203 11:55:23.548922 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-b2d4g\" (UniqueName: \"kubernetes.io/projected/0bda7b03-76fd-405e-9a69-b897331e829c-kube-api-access-b2d4g\") on node \"crc\" DevicePath \"\"" Dec 03 11:55:23 crc kubenswrapper[5052]: I1203 11:55:23.596029 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron3cf1-account-delete-zk6qw" Dec 03 11:55:23 crc kubenswrapper[5052]: I1203 11:55:23.650331 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rsvm8\" (UniqueName: \"kubernetes.io/projected/0f42494c-058e-417d-9e0d-97ce7040b288-kube-api-access-rsvm8\") pod \"0f42494c-058e-417d-9e0d-97ce7040b288\" (UID: \"0f42494c-058e-417d-9e0d-97ce7040b288\") " Dec 03 11:55:23 crc kubenswrapper[5052]: I1203 11:55:23.650477 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/0f42494c-058e-417d-9e0d-97ce7040b288-operator-scripts\") pod \"0f42494c-058e-417d-9e0d-97ce7040b288\" (UID: \"0f42494c-058e-417d-9e0d-97ce7040b288\") " Dec 03 11:55:23 crc kubenswrapper[5052]: I1203 11:55:23.651098 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0f42494c-058e-417d-9e0d-97ce7040b288-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "0f42494c-058e-417d-9e0d-97ce7040b288" (UID: "0f42494c-058e-417d-9e0d-97ce7040b288"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:55:23 crc kubenswrapper[5052]: I1203 11:55:23.653885 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0f42494c-058e-417d-9e0d-97ce7040b288-kube-api-access-rsvm8" (OuterVolumeSpecName: "kube-api-access-rsvm8") pod "0f42494c-058e-417d-9e0d-97ce7040b288" (UID: "0f42494c-058e-417d-9e0d-97ce7040b288"). InnerVolumeSpecName "kube-api-access-rsvm8". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:55:23 crc kubenswrapper[5052]: I1203 11:55:23.682612 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="da3a810d-9fd0-4f18-8f4a-e896b94b1431" path="/var/lib/kubelet/pods/da3a810d-9fd0-4f18-8f4a-e896b94b1431/volumes" Dec 03 11:55:23 crc kubenswrapper[5052]: I1203 11:55:23.752413 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-rsvm8\" (UniqueName: \"kubernetes.io/projected/0f42494c-058e-417d-9e0d-97ce7040b288-kube-api-access-rsvm8\") on node \"crc\" DevicePath \"\"" Dec 03 11:55:23 crc kubenswrapper[5052]: I1203 11:55:23.752453 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/0f42494c-058e-417d-9e0d-97ce7040b288-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:55:24 crc kubenswrapper[5052]: I1203 11:55:24.188420 5052 generic.go:334] "Generic (PLEG): container finished" podID="6087dc7e-081b-4392-adb6-205b2aec805f" containerID="162808060d27104600e8d3bf34956c7d1f29ffc6f699566d17f44cd664e8fc4c" exitCode=137 Dec 03 11:55:24 crc kubenswrapper[5052]: I1203 11:55:24.188522 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/novaapie4da-account-delete-mknr9" event={"ID":"6087dc7e-081b-4392-adb6-205b2aec805f","Type":"ContainerDied","Data":"162808060d27104600e8d3bf34956c7d1f29ffc6f699566d17f44cd664e8fc4c"} Dec 03 11:55:24 crc kubenswrapper[5052]: I1203 11:55:24.191674 5052 generic.go:334] "Generic (PLEG): container finished" podID="93d03e32-f318-470c-ad5d-f17a609f5003" containerID="75afa46f01077a531888f3be755bcaf1e48e735b4481e7ab69d40c172fb1751f" exitCode=137 Dec 03 11:55:24 crc kubenswrapper[5052]: I1203 11:55:24.191778 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/novacell0745f-account-delete-zbhjv" event={"ID":"93d03e32-f318-470c-ad5d-f17a609f5003","Type":"ContainerDied","Data":"75afa46f01077a531888f3be755bcaf1e48e735b4481e7ab69d40c172fb1751f"} Dec 03 11:55:24 crc kubenswrapper[5052]: I1203 11:55:24.193325 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron3cf1-account-delete-zk6qw" event={"ID":"0f42494c-058e-417d-9e0d-97ce7040b288","Type":"ContainerDied","Data":"e0edc022d786ec7711974e5ad8fa9b59a5df42dc00c4cf83fd27b41325567c72"} Dec 03 11:55:24 crc kubenswrapper[5052]: I1203 11:55:24.193373 5052 scope.go:117] "RemoveContainer" containerID="39d19d353a9f922fcda81960e089ebd7d426e58421f5f1bd9b9dc1951175ebeb" Dec 03 11:55:24 crc kubenswrapper[5052]: I1203 11:55:24.193491 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron3cf1-account-delete-zk6qw" Dec 03 11:55:24 crc kubenswrapper[5052]: I1203 11:55:24.195240 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement2868-account-delete-gtkfr" event={"ID":"0bda7b03-76fd-405e-9a69-b897331e829c","Type":"ContainerDied","Data":"92f47638f85990fab3449257e38cffbd0de5e3c0d1848807a7762ebe630c0e12"} Dec 03 11:55:24 crc kubenswrapper[5052]: I1203 11:55:24.195293 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement2868-account-delete-gtkfr" Dec 03 11:55:24 crc kubenswrapper[5052]: I1203 11:55:24.228072 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/placement2868-account-delete-gtkfr"] Dec 03 11:55:24 crc kubenswrapper[5052]: I1203 11:55:24.236410 5052 scope.go:117] "RemoveContainer" containerID="3ee6799c23947cd1ca061d572689f0e21a053ace392723fbac989b50aff781bb" Dec 03 11:55:24 crc kubenswrapper[5052]: I1203 11:55:24.243731 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/placement2868-account-delete-gtkfr"] Dec 03 11:55:24 crc kubenswrapper[5052]: I1203 11:55:24.251462 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/neutron3cf1-account-delete-zk6qw"] Dec 03 11:55:24 crc kubenswrapper[5052]: I1203 11:55:24.257115 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/neutron3cf1-account-delete-zk6qw"] Dec 03 11:55:24 crc kubenswrapper[5052]: E1203 11:55:24.463132 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 11:55:24 crc kubenswrapper[5052]: E1203 11:55:24.463610 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/93d03e32-f318-470c-ad5d-f17a609f5003-operator-scripts podName:93d03e32-f318-470c-ad5d-f17a609f5003 nodeName:}" failed. No retries permitted until 2025-12-03 11:55:56.46359077 +0000 UTC m=+2147.051832534 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/93d03e32-f318-470c-ad5d-f17a609f5003-operator-scripts") pod "novacell0745f-account-delete-zbhjv" (UID: "93d03e32-f318-470c-ad5d-f17a609f5003") : configmap "openstack-scripts" not found Dec 03 11:55:24 crc kubenswrapper[5052]: E1203 11:55:24.463132 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 11:55:24 crc kubenswrapper[5052]: E1203 11:55:24.463690 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6087dc7e-081b-4392-adb6-205b2aec805f-operator-scripts podName:6087dc7e-081b-4392-adb6-205b2aec805f nodeName:}" failed. No retries permitted until 2025-12-03 11:55:56.463679242 +0000 UTC m=+2147.051921006 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/6087dc7e-081b-4392-adb6-205b2aec805f-operator-scripts") pod "novaapie4da-account-delete-mknr9" (UID: "6087dc7e-081b-4392-adb6-205b2aec805f") : configmap "openstack-scripts" not found Dec 03 11:55:24 crc kubenswrapper[5052]: I1203 11:55:24.594403 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/novacell0745f-account-delete-zbhjv" Dec 03 11:55:24 crc kubenswrapper[5052]: I1203 11:55:24.659170 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/novaapie4da-account-delete-mknr9" Dec 03 11:55:24 crc kubenswrapper[5052]: I1203 11:55:24.664927 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sxkjh\" (UniqueName: \"kubernetes.io/projected/93d03e32-f318-470c-ad5d-f17a609f5003-kube-api-access-sxkjh\") pod \"93d03e32-f318-470c-ad5d-f17a609f5003\" (UID: \"93d03e32-f318-470c-ad5d-f17a609f5003\") " Dec 03 11:55:24 crc kubenswrapper[5052]: I1203 11:55:24.665052 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/93d03e32-f318-470c-ad5d-f17a609f5003-operator-scripts\") pod \"93d03e32-f318-470c-ad5d-f17a609f5003\" (UID: \"93d03e32-f318-470c-ad5d-f17a609f5003\") " Dec 03 11:55:24 crc kubenswrapper[5052]: I1203 11:55:24.665737 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/93d03e32-f318-470c-ad5d-f17a609f5003-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "93d03e32-f318-470c-ad5d-f17a609f5003" (UID: "93d03e32-f318-470c-ad5d-f17a609f5003"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:55:24 crc kubenswrapper[5052]: I1203 11:55:24.675391 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/93d03e32-f318-470c-ad5d-f17a609f5003-kube-api-access-sxkjh" (OuterVolumeSpecName: "kube-api-access-sxkjh") pod "93d03e32-f318-470c-ad5d-f17a609f5003" (UID: "93d03e32-f318-470c-ad5d-f17a609f5003"). InnerVolumeSpecName "kube-api-access-sxkjh". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:55:24 crc kubenswrapper[5052]: I1203 11:55:24.765897 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/6087dc7e-081b-4392-adb6-205b2aec805f-operator-scripts\") pod \"6087dc7e-081b-4392-adb6-205b2aec805f\" (UID: \"6087dc7e-081b-4392-adb6-205b2aec805f\") " Dec 03 11:55:24 crc kubenswrapper[5052]: I1203 11:55:24.766044 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vnvmh\" (UniqueName: \"kubernetes.io/projected/6087dc7e-081b-4392-adb6-205b2aec805f-kube-api-access-vnvmh\") pod \"6087dc7e-081b-4392-adb6-205b2aec805f\" (UID: \"6087dc7e-081b-4392-adb6-205b2aec805f\") " Dec 03 11:55:24 crc kubenswrapper[5052]: I1203 11:55:24.766417 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/93d03e32-f318-470c-ad5d-f17a609f5003-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:55:24 crc kubenswrapper[5052]: I1203 11:55:24.766432 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-sxkjh\" (UniqueName: \"kubernetes.io/projected/93d03e32-f318-470c-ad5d-f17a609f5003-kube-api-access-sxkjh\") on node \"crc\" DevicePath \"\"" Dec 03 11:55:24 crc kubenswrapper[5052]: I1203 11:55:24.766628 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6087dc7e-081b-4392-adb6-205b2aec805f-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "6087dc7e-081b-4392-adb6-205b2aec805f" (UID: "6087dc7e-081b-4392-adb6-205b2aec805f"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:55:24 crc kubenswrapper[5052]: I1203 11:55:24.769394 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6087dc7e-081b-4392-adb6-205b2aec805f-kube-api-access-vnvmh" (OuterVolumeSpecName: "kube-api-access-vnvmh") pod "6087dc7e-081b-4392-adb6-205b2aec805f" (UID: "6087dc7e-081b-4392-adb6-205b2aec805f"). InnerVolumeSpecName "kube-api-access-vnvmh". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:55:24 crc kubenswrapper[5052]: I1203 11:55:24.868176 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/6087dc7e-081b-4392-adb6-205b2aec805f-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:55:24 crc kubenswrapper[5052]: I1203 11:55:24.868227 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-vnvmh\" (UniqueName: \"kubernetes.io/projected/6087dc7e-081b-4392-adb6-205b2aec805f-kube-api-access-vnvmh\") on node \"crc\" DevicePath \"\"" Dec 03 11:55:25 crc kubenswrapper[5052]: I1203 11:55:25.204999 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/novacell0745f-account-delete-zbhjv" event={"ID":"93d03e32-f318-470c-ad5d-f17a609f5003","Type":"ContainerDied","Data":"55c65845b9883cfa8223f22f3936649db274c92350e5b942c49cfdce2b08ea11"} Dec 03 11:55:25 crc kubenswrapper[5052]: I1203 11:55:25.205058 5052 scope.go:117] "RemoveContainer" containerID="75afa46f01077a531888f3be755bcaf1e48e735b4481e7ab69d40c172fb1751f" Dec 03 11:55:25 crc kubenswrapper[5052]: I1203 11:55:25.205620 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/novacell0745f-account-delete-zbhjv" Dec 03 11:55:25 crc kubenswrapper[5052]: I1203 11:55:25.209382 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/novaapie4da-account-delete-mknr9" event={"ID":"6087dc7e-081b-4392-adb6-205b2aec805f","Type":"ContainerDied","Data":"6d4798b99d8ee6cf41344eb8c16aeb07875c9c62815bb1c4306ee7ec69a52f33"} Dec 03 11:55:25 crc kubenswrapper[5052]: I1203 11:55:25.209468 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/novaapie4da-account-delete-mknr9" Dec 03 11:55:25 crc kubenswrapper[5052]: I1203 11:55:25.232462 5052 scope.go:117] "RemoveContainer" containerID="162808060d27104600e8d3bf34956c7d1f29ffc6f699566d17f44cd664e8fc4c" Dec 03 11:55:25 crc kubenswrapper[5052]: I1203 11:55:25.248848 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/novacell0745f-account-delete-zbhjv"] Dec 03 11:55:25 crc kubenswrapper[5052]: I1203 11:55:25.255793 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/novacell0745f-account-delete-zbhjv"] Dec 03 11:55:25 crc kubenswrapper[5052]: I1203 11:55:25.261778 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/novaapie4da-account-delete-mknr9"] Dec 03 11:55:25 crc kubenswrapper[5052]: I1203 11:55:25.267370 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/novaapie4da-account-delete-mknr9"] Dec 03 11:55:25 crc kubenswrapper[5052]: I1203 11:55:25.681709 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0bda7b03-76fd-405e-9a69-b897331e829c" path="/var/lib/kubelet/pods/0bda7b03-76fd-405e-9a69-b897331e829c/volumes" Dec 03 11:55:25 crc kubenswrapper[5052]: I1203 11:55:25.682243 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0f42494c-058e-417d-9e0d-97ce7040b288" path="/var/lib/kubelet/pods/0f42494c-058e-417d-9e0d-97ce7040b288/volumes" Dec 03 11:55:25 crc kubenswrapper[5052]: I1203 11:55:25.682814 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6087dc7e-081b-4392-adb6-205b2aec805f" path="/var/lib/kubelet/pods/6087dc7e-081b-4392-adb6-205b2aec805f/volumes" Dec 03 11:55:25 crc kubenswrapper[5052]: I1203 11:55:25.683346 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="93d03e32-f318-470c-ad5d-f17a609f5003" path="/var/lib/kubelet/pods/93d03e32-f318-470c-ad5d-f17a609f5003/volumes" Dec 03 11:55:32 crc kubenswrapper[5052]: I1203 11:55:32.472757 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["crc-storage/crc-storage-crc-6vsfp"] Dec 03 11:55:32 crc kubenswrapper[5052]: I1203 11:55:32.477870 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["crc-storage/crc-storage-crc-6vsfp"] Dec 03 11:55:32 crc kubenswrapper[5052]: I1203 11:55:32.628559 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["crc-storage/crc-storage-crc-zskmc"] Dec 03 11:55:32 crc kubenswrapper[5052]: E1203 11:55:32.628877 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="83b9fcfd-8007-4d6b-9090-d2e536cb1bbc" containerName="neutron-api" Dec 03 11:55:32 crc kubenswrapper[5052]: I1203 11:55:32.628897 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="83b9fcfd-8007-4d6b-9090-d2e536cb1bbc" containerName="neutron-api" Dec 03 11:55:32 crc kubenswrapper[5052]: E1203 11:55:32.628908 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="88e7db65-8253-474b-b115-cd3f6edaf85c" containerName="kube-state-metrics" Dec 03 11:55:32 crc kubenswrapper[5052]: I1203 11:55:32.628917 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="88e7db65-8253-474b-b115-cd3f6edaf85c" containerName="kube-state-metrics" Dec 03 11:55:32 crc kubenswrapper[5052]: E1203 11:55:32.628938 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="653882ec-4522-4979-b1aa-7fad5023605c" containerName="nova-metadata-metadata" Dec 03 11:55:32 crc kubenswrapper[5052]: I1203 11:55:32.628947 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="653882ec-4522-4979-b1aa-7fad5023605c" containerName="nova-metadata-metadata" Dec 03 11:55:32 crc kubenswrapper[5052]: E1203 11:55:32.628958 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="337bf4ce-ed6a-47bb-820d-6d64cd15e16e" containerName="account-reaper" Dec 03 11:55:32 crc kubenswrapper[5052]: I1203 11:55:32.628967 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="337bf4ce-ed6a-47bb-820d-6d64cd15e16e" containerName="account-reaper" Dec 03 11:55:32 crc kubenswrapper[5052]: E1203 11:55:32.628978 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="337bf4ce-ed6a-47bb-820d-6d64cd15e16e" containerName="container-server" Dec 03 11:55:32 crc kubenswrapper[5052]: I1203 11:55:32.628986 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="337bf4ce-ed6a-47bb-820d-6d64cd15e16e" containerName="container-server" Dec 03 11:55:32 crc kubenswrapper[5052]: E1203 11:55:32.629001 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="337bf4ce-ed6a-47bb-820d-6d64cd15e16e" containerName="object-replicator" Dec 03 11:55:32 crc kubenswrapper[5052]: I1203 11:55:32.629010 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="337bf4ce-ed6a-47bb-820d-6d64cd15e16e" containerName="object-replicator" Dec 03 11:55:32 crc kubenswrapper[5052]: E1203 11:55:32.629019 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="428e9203-aa9c-48d3-84c9-4dfaa806140d" containerName="placement-log" Dec 03 11:55:32 crc kubenswrapper[5052]: I1203 11:55:32.629029 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="428e9203-aa9c-48d3-84c9-4dfaa806140d" containerName="placement-log" Dec 03 11:55:32 crc kubenswrapper[5052]: E1203 11:55:32.629041 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="5d0169e1-4f74-4f6e-89cc-70d2f79c34e2" containerName="dnsmasq-dns" Dec 03 11:55:32 crc kubenswrapper[5052]: I1203 11:55:32.629049 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="5d0169e1-4f74-4f6e-89cc-70d2f79c34e2" containerName="dnsmasq-dns" Dec 03 11:55:32 crc kubenswrapper[5052]: E1203 11:55:32.629065 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="9d8cd78f-2670-4933-81dd-01cdc0044454" containerName="openstack-network-exporter" Dec 03 11:55:32 crc kubenswrapper[5052]: I1203 11:55:32.629073 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d8cd78f-2670-4933-81dd-01cdc0044454" containerName="openstack-network-exporter" Dec 03 11:55:32 crc kubenswrapper[5052]: E1203 11:55:32.629083 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="337bf4ce-ed6a-47bb-820d-6d64cd15e16e" containerName="object-server" Dec 03 11:55:32 crc kubenswrapper[5052]: I1203 11:55:32.629090 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="337bf4ce-ed6a-47bb-820d-6d64cd15e16e" containerName="object-server" Dec 03 11:55:32 crc kubenswrapper[5052]: E1203 11:55:32.629107 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="10e3094e-f625-4d69-b0e9-358220c0d195" containerName="rabbitmq" Dec 03 11:55:32 crc kubenswrapper[5052]: I1203 11:55:32.629115 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="10e3094e-f625-4d69-b0e9-358220c0d195" containerName="rabbitmq" Dec 03 11:55:32 crc kubenswrapper[5052]: E1203 11:55:32.629125 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="653882ec-4522-4979-b1aa-7fad5023605c" containerName="nova-metadata-log" Dec 03 11:55:32 crc kubenswrapper[5052]: I1203 11:55:32.629133 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="653882ec-4522-4979-b1aa-7fad5023605c" containerName="nova-metadata-log" Dec 03 11:55:32 crc kubenswrapper[5052]: E1203 11:55:32.629144 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="0f42494c-058e-417d-9e0d-97ce7040b288" containerName="mariadb-account-delete" Dec 03 11:55:32 crc kubenswrapper[5052]: I1203 11:55:32.629152 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="0f42494c-058e-417d-9e0d-97ce7040b288" containerName="mariadb-account-delete" Dec 03 11:55:32 crc kubenswrapper[5052]: E1203 11:55:32.629168 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="d541d73d-2f95-43a8-8409-40f5f2ff54c4" containerName="glance-log" Dec 03 11:55:32 crc kubenswrapper[5052]: I1203 11:55:32.629176 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="d541d73d-2f95-43a8-8409-40f5f2ff54c4" containerName="glance-log" Dec 03 11:55:32 crc kubenswrapper[5052]: E1203 11:55:32.629193 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="8dd63158-2c75-455f-8f58-57a55c7e0f22" containerName="galera" Dec 03 11:55:32 crc kubenswrapper[5052]: I1203 11:55:32.629201 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="8dd63158-2c75-455f-8f58-57a55c7e0f22" containerName="galera" Dec 03 11:55:32 crc kubenswrapper[5052]: E1203 11:55:32.629212 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="584deaba-f945-42a2-8c0d-e531d6f48fd8" containerName="barbican-worker-log" Dec 03 11:55:32 crc kubenswrapper[5052]: I1203 11:55:32.629221 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="584deaba-f945-42a2-8c0d-e531d6f48fd8" containerName="barbican-worker-log" Dec 03 11:55:32 crc kubenswrapper[5052]: E1203 11:55:32.629238 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="75dd245b-b54f-4a9a-9627-cbf338a69566" containerName="ovsdbserver-sb" Dec 03 11:55:32 crc kubenswrapper[5052]: I1203 11:55:32.629246 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="75dd245b-b54f-4a9a-9627-cbf338a69566" containerName="ovsdbserver-sb" Dec 03 11:55:32 crc kubenswrapper[5052]: E1203 11:55:32.629258 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="6087dc7e-081b-4392-adb6-205b2aec805f" containerName="mariadb-account-delete" Dec 03 11:55:32 crc kubenswrapper[5052]: I1203 11:55:32.629266 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="6087dc7e-081b-4392-adb6-205b2aec805f" containerName="mariadb-account-delete" Dec 03 11:55:32 crc kubenswrapper[5052]: E1203 11:55:32.629279 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="584deaba-f945-42a2-8c0d-e531d6f48fd8" containerName="barbican-worker" Dec 03 11:55:32 crc kubenswrapper[5052]: I1203 11:55:32.629286 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="584deaba-f945-42a2-8c0d-e531d6f48fd8" containerName="barbican-worker" Dec 03 11:55:32 crc kubenswrapper[5052]: E1203 11:55:32.629296 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="337bf4ce-ed6a-47bb-820d-6d64cd15e16e" containerName="account-server" Dec 03 11:55:32 crc kubenswrapper[5052]: I1203 11:55:32.629305 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="337bf4ce-ed6a-47bb-820d-6d64cd15e16e" containerName="account-server" Dec 03 11:55:32 crc kubenswrapper[5052]: E1203 11:55:32.629313 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="a5ff0249-173a-4793-8cbe-0aa749f34632" containerName="ceilometer-central-agent" Dec 03 11:55:32 crc kubenswrapper[5052]: I1203 11:55:32.629321 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="a5ff0249-173a-4793-8cbe-0aa749f34632" containerName="ceilometer-central-agent" Dec 03 11:55:32 crc kubenswrapper[5052]: E1203 11:55:32.629331 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="d541d73d-2f95-43a8-8409-40f5f2ff54c4" containerName="glance-httpd" Dec 03 11:55:32 crc kubenswrapper[5052]: I1203 11:55:32.629339 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="d541d73d-2f95-43a8-8409-40f5f2ff54c4" containerName="glance-httpd" Dec 03 11:55:32 crc kubenswrapper[5052]: E1203 11:55:32.629351 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="337bf4ce-ed6a-47bb-820d-6d64cd15e16e" containerName="account-replicator" Dec 03 11:55:32 crc kubenswrapper[5052]: I1203 11:55:32.629359 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="337bf4ce-ed6a-47bb-820d-6d64cd15e16e" containerName="account-replicator" Dec 03 11:55:32 crc kubenswrapper[5052]: E1203 11:55:32.629374 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="8dd63158-2c75-455f-8f58-57a55c7e0f22" containerName="mysql-bootstrap" Dec 03 11:55:32 crc kubenswrapper[5052]: I1203 11:55:32.629382 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="8dd63158-2c75-455f-8f58-57a55c7e0f22" containerName="mysql-bootstrap" Dec 03 11:55:32 crc kubenswrapper[5052]: E1203 11:55:32.629396 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="53a152d4-c58c-40a7-b4ce-b9a9dc42337d" containerName="memcached" Dec 03 11:55:32 crc kubenswrapper[5052]: I1203 11:55:32.629405 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="53a152d4-c58c-40a7-b4ce-b9a9dc42337d" containerName="memcached" Dec 03 11:55:32 crc kubenswrapper[5052]: E1203 11:55:32.629418 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="0201d941-cd05-4d1c-9af6-f3a82a375a0f" containerName="probe" Dec 03 11:55:32 crc kubenswrapper[5052]: I1203 11:55:32.629427 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="0201d941-cd05-4d1c-9af6-f3a82a375a0f" containerName="probe" Dec 03 11:55:32 crc kubenswrapper[5052]: E1203 11:55:32.629443 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="a73e0afd-e037-41f7-b214-d4eebd4bafe0" containerName="mariadb-account-delete" Dec 03 11:55:32 crc kubenswrapper[5052]: I1203 11:55:32.629451 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="a73e0afd-e037-41f7-b214-d4eebd4bafe0" containerName="mariadb-account-delete" Dec 03 11:55:32 crc kubenswrapper[5052]: E1203 11:55:32.629465 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="ec16e8c3-9750-4475-a671-17b0c16826c8" containerName="nova-scheduler-scheduler" Dec 03 11:55:32 crc kubenswrapper[5052]: I1203 11:55:32.629474 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="ec16e8c3-9750-4475-a671-17b0c16826c8" containerName="nova-scheduler-scheduler" Dec 03 11:55:32 crc kubenswrapper[5052]: E1203 11:55:32.629485 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="93d03e32-f318-470c-ad5d-f17a609f5003" containerName="mariadb-account-delete" Dec 03 11:55:32 crc kubenswrapper[5052]: I1203 11:55:32.629493 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="93d03e32-f318-470c-ad5d-f17a609f5003" containerName="mariadb-account-delete" Dec 03 11:55:32 crc kubenswrapper[5052]: E1203 11:55:32.629525 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="da3a810d-9fd0-4f18-8f4a-e896b94b1431" containerName="nova-cell1-novncproxy-novncproxy" Dec 03 11:55:32 crc kubenswrapper[5052]: I1203 11:55:32.629534 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="da3a810d-9fd0-4f18-8f4a-e896b94b1431" containerName="nova-cell1-novncproxy-novncproxy" Dec 03 11:55:32 crc kubenswrapper[5052]: E1203 11:55:32.629548 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="a5ff0249-173a-4793-8cbe-0aa749f34632" containerName="sg-core" Dec 03 11:55:32 crc kubenswrapper[5052]: I1203 11:55:32.629557 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="a5ff0249-173a-4793-8cbe-0aa749f34632" containerName="sg-core" Dec 03 11:55:32 crc kubenswrapper[5052]: E1203 11:55:32.629567 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="337bf4ce-ed6a-47bb-820d-6d64cd15e16e" containerName="container-updater" Dec 03 11:55:32 crc kubenswrapper[5052]: I1203 11:55:32.629575 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="337bf4ce-ed6a-47bb-820d-6d64cd15e16e" containerName="container-updater" Dec 03 11:55:32 crc kubenswrapper[5052]: E1203 11:55:32.629584 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="5d0169e1-4f74-4f6e-89cc-70d2f79c34e2" containerName="init" Dec 03 11:55:32 crc kubenswrapper[5052]: I1203 11:55:32.629592 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="5d0169e1-4f74-4f6e-89cc-70d2f79c34e2" containerName="init" Dec 03 11:55:32 crc kubenswrapper[5052]: E1203 11:55:32.629602 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="a18602ce-7206-4edc-a8d3-5d0754c3da02" containerName="barbican-api" Dec 03 11:55:32 crc kubenswrapper[5052]: I1203 11:55:32.629610 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="a18602ce-7206-4edc-a8d3-5d0754c3da02" containerName="barbican-api" Dec 03 11:55:32 crc kubenswrapper[5052]: E1203 11:55:32.629620 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="4ec0d450-cd52-432e-8bb2-1ef9698821c1" containerName="nova-cell0-conductor-conductor" Dec 03 11:55:32 crc kubenswrapper[5052]: I1203 11:55:32.629628 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="4ec0d450-cd52-432e-8bb2-1ef9698821c1" containerName="nova-cell0-conductor-conductor" Dec 03 11:55:32 crc kubenswrapper[5052]: E1203 11:55:32.629640 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="a5ff0249-173a-4793-8cbe-0aa749f34632" containerName="proxy-httpd" Dec 03 11:55:32 crc kubenswrapper[5052]: I1203 11:55:32.629648 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="a5ff0249-173a-4793-8cbe-0aa749f34632" containerName="proxy-httpd" Dec 03 11:55:32 crc kubenswrapper[5052]: E1203 11:55:32.629662 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="a5ba4775-815a-46ef-bd2d-05dfa7513288" containerName="setup-container" Dec 03 11:55:32 crc kubenswrapper[5052]: I1203 11:55:32.629671 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="a5ba4775-815a-46ef-bd2d-05dfa7513288" containerName="setup-container" Dec 03 11:55:32 crc kubenswrapper[5052]: E1203 11:55:32.629682 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="0201d941-cd05-4d1c-9af6-f3a82a375a0f" containerName="cinder-scheduler" Dec 03 11:55:32 crc kubenswrapper[5052]: I1203 11:55:32.629690 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="0201d941-cd05-4d1c-9af6-f3a82a375a0f" containerName="cinder-scheduler" Dec 03 11:55:32 crc kubenswrapper[5052]: E1203 11:55:32.629699 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="f32095c7-4e6f-43c7-a16f-513e89f7c28e" containerName="glance-httpd" Dec 03 11:55:32 crc kubenswrapper[5052]: I1203 11:55:32.629707 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="f32095c7-4e6f-43c7-a16f-513e89f7c28e" containerName="glance-httpd" Dec 03 11:55:32 crc kubenswrapper[5052]: E1203 11:55:32.629718 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="f856a02f-3f21-4fd4-8158-50a232de756e" containerName="mysql-bootstrap" Dec 03 11:55:32 crc kubenswrapper[5052]: I1203 11:55:32.629726 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="f856a02f-3f21-4fd4-8158-50a232de756e" containerName="mysql-bootstrap" Dec 03 11:55:32 crc kubenswrapper[5052]: E1203 11:55:32.629739 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="b4961b81-97b8-4174-82bf-2d1d02cf8335" containerName="ovsdbserver-nb" Dec 03 11:55:32 crc kubenswrapper[5052]: I1203 11:55:32.629748 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="b4961b81-97b8-4174-82bf-2d1d02cf8335" containerName="ovsdbserver-nb" Dec 03 11:55:32 crc kubenswrapper[5052]: E1203 11:55:32.629758 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="337bf4ce-ed6a-47bb-820d-6d64cd15e16e" containerName="container-replicator" Dec 03 11:55:32 crc kubenswrapper[5052]: I1203 11:55:32.629766 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="337bf4ce-ed6a-47bb-820d-6d64cd15e16e" containerName="container-replicator" Dec 03 11:55:32 crc kubenswrapper[5052]: E1203 11:55:32.629776 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="0bda7b03-76fd-405e-9a69-b897331e829c" containerName="mariadb-account-delete" Dec 03 11:55:32 crc kubenswrapper[5052]: I1203 11:55:32.629785 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="0bda7b03-76fd-405e-9a69-b897331e829c" containerName="mariadb-account-delete" Dec 03 11:55:32 crc kubenswrapper[5052]: E1203 11:55:32.629795 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="53d7a9a2-b6a9-42dc-9828-d8ceaa70f0a3" containerName="mariadb-account-delete" Dec 03 11:55:32 crc kubenswrapper[5052]: I1203 11:55:32.629803 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="53d7a9a2-b6a9-42dc-9828-d8ceaa70f0a3" containerName="mariadb-account-delete" Dec 03 11:55:32 crc kubenswrapper[5052]: E1203 11:55:32.629818 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="f856a02f-3f21-4fd4-8158-50a232de756e" containerName="galera" Dec 03 11:55:32 crc kubenswrapper[5052]: I1203 11:55:32.629829 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="f856a02f-3f21-4fd4-8158-50a232de756e" containerName="galera" Dec 03 11:55:32 crc kubenswrapper[5052]: E1203 11:55:32.629841 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="9a4b1245-d95c-4eb6-8d2b-fdd93d98bb1e" containerName="cinder-api" Dec 03 11:55:32 crc kubenswrapper[5052]: I1203 11:55:32.629850 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="9a4b1245-d95c-4eb6-8d2b-fdd93d98bb1e" containerName="cinder-api" Dec 03 11:55:32 crc kubenswrapper[5052]: E1203 11:55:32.629862 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="a5ba4775-815a-46ef-bd2d-05dfa7513288" containerName="rabbitmq" Dec 03 11:55:32 crc kubenswrapper[5052]: I1203 11:55:32.629870 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="a5ba4775-815a-46ef-bd2d-05dfa7513288" containerName="rabbitmq" Dec 03 11:55:32 crc kubenswrapper[5052]: E1203 11:55:32.629880 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="337bf4ce-ed6a-47bb-820d-6d64cd15e16e" containerName="account-auditor" Dec 03 11:55:32 crc kubenswrapper[5052]: I1203 11:55:32.629888 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="337bf4ce-ed6a-47bb-820d-6d64cd15e16e" containerName="account-auditor" Dec 03 11:55:32 crc kubenswrapper[5052]: E1203 11:55:32.629903 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="a5ff0249-173a-4793-8cbe-0aa749f34632" containerName="ceilometer-notification-agent" Dec 03 11:55:32 crc kubenswrapper[5052]: I1203 11:55:32.629911 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="a5ff0249-173a-4793-8cbe-0aa749f34632" containerName="ceilometer-notification-agent" Dec 03 11:55:32 crc kubenswrapper[5052]: E1203 11:55:32.629921 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="337bf4ce-ed6a-47bb-820d-6d64cd15e16e" containerName="object-auditor" Dec 03 11:55:32 crc kubenswrapper[5052]: I1203 11:55:32.629929 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="337bf4ce-ed6a-47bb-820d-6d64cd15e16e" containerName="object-auditor" Dec 03 11:55:32 crc kubenswrapper[5052]: E1203 11:55:32.629939 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="775d2ccf-9b52-4274-99ef-2c33b18beaea" containerName="keystone-api" Dec 03 11:55:32 crc kubenswrapper[5052]: I1203 11:55:32.629947 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="775d2ccf-9b52-4274-99ef-2c33b18beaea" containerName="keystone-api" Dec 03 11:55:32 crc kubenswrapper[5052]: E1203 11:55:32.629963 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="a18602ce-7206-4edc-a8d3-5d0754c3da02" containerName="barbican-api-log" Dec 03 11:55:32 crc kubenswrapper[5052]: I1203 11:55:32.629970 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="a18602ce-7206-4edc-a8d3-5d0754c3da02" containerName="barbican-api-log" Dec 03 11:55:32 crc kubenswrapper[5052]: E1203 11:55:32.629980 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="4a181d2c-7595-4394-98de-957451309c98" containerName="nova-cell1-conductor-conductor" Dec 03 11:55:32 crc kubenswrapper[5052]: I1203 11:55:32.629989 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="4a181d2c-7595-4394-98de-957451309c98" containerName="nova-cell1-conductor-conductor" Dec 03 11:55:32 crc kubenswrapper[5052]: E1203 11:55:32.630002 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="383e7933-2bc4-4ead-bc34-0832091bd952" containerName="mariadb-account-delete" Dec 03 11:55:32 crc kubenswrapper[5052]: I1203 11:55:32.630010 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="383e7933-2bc4-4ead-bc34-0832091bd952" containerName="mariadb-account-delete" Dec 03 11:55:32 crc kubenswrapper[5052]: E1203 11:55:32.630019 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="9d8cd78f-2670-4933-81dd-01cdc0044454" containerName="ovn-northd" Dec 03 11:55:32 crc kubenswrapper[5052]: I1203 11:55:32.630026 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d8cd78f-2670-4933-81dd-01cdc0044454" containerName="ovn-northd" Dec 03 11:55:32 crc kubenswrapper[5052]: E1203 11:55:32.630036 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="75dd245b-b54f-4a9a-9627-cbf338a69566" containerName="openstack-network-exporter" Dec 03 11:55:32 crc kubenswrapper[5052]: I1203 11:55:32.630045 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="75dd245b-b54f-4a9a-9627-cbf338a69566" containerName="openstack-network-exporter" Dec 03 11:55:32 crc kubenswrapper[5052]: E1203 11:55:32.630057 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="337bf4ce-ed6a-47bb-820d-6d64cd15e16e" containerName="container-auditor" Dec 03 11:55:32 crc kubenswrapper[5052]: I1203 11:55:32.630065 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="337bf4ce-ed6a-47bb-820d-6d64cd15e16e" containerName="container-auditor" Dec 03 11:55:32 crc kubenswrapper[5052]: E1203 11:55:32.630081 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="b4961b81-97b8-4174-82bf-2d1d02cf8335" containerName="openstack-network-exporter" Dec 03 11:55:32 crc kubenswrapper[5052]: I1203 11:55:32.630089 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="b4961b81-97b8-4174-82bf-2d1d02cf8335" containerName="openstack-network-exporter" Dec 03 11:55:32 crc kubenswrapper[5052]: E1203 11:55:32.630100 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="5679a95d-a498-448e-97ed-497496520506" containerName="barbican-keystone-listener-log" Dec 03 11:55:32 crc kubenswrapper[5052]: I1203 11:55:32.630108 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="5679a95d-a498-448e-97ed-497496520506" containerName="barbican-keystone-listener-log" Dec 03 11:55:32 crc kubenswrapper[5052]: E1203 11:55:32.630123 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="036bc2b0-bc50-4c91-bea5-882b9e592e4f" containerName="proxy-server" Dec 03 11:55:32 crc kubenswrapper[5052]: I1203 11:55:32.630131 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="036bc2b0-bc50-4c91-bea5-882b9e592e4f" containerName="proxy-server" Dec 03 11:55:32 crc kubenswrapper[5052]: E1203 11:55:32.630146 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="337bf4ce-ed6a-47bb-820d-6d64cd15e16e" containerName="object-expirer" Dec 03 11:55:32 crc kubenswrapper[5052]: I1203 11:55:32.630153 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="337bf4ce-ed6a-47bb-820d-6d64cd15e16e" containerName="object-expirer" Dec 03 11:55:32 crc kubenswrapper[5052]: E1203 11:55:32.630164 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="f32095c7-4e6f-43c7-a16f-513e89f7c28e" containerName="glance-log" Dec 03 11:55:32 crc kubenswrapper[5052]: I1203 11:55:32.630172 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="f32095c7-4e6f-43c7-a16f-513e89f7c28e" containerName="glance-log" Dec 03 11:55:32 crc kubenswrapper[5052]: E1203 11:55:32.630182 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="9a4b1245-d95c-4eb6-8d2b-fdd93d98bb1e" containerName="cinder-api-log" Dec 03 11:55:32 crc kubenswrapper[5052]: I1203 11:55:32.630190 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="9a4b1245-d95c-4eb6-8d2b-fdd93d98bb1e" containerName="cinder-api-log" Dec 03 11:55:32 crc kubenswrapper[5052]: E1203 11:55:32.630202 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="10e3094e-f625-4d69-b0e9-358220c0d195" containerName="setup-container" Dec 03 11:55:32 crc kubenswrapper[5052]: I1203 11:55:32.630210 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="10e3094e-f625-4d69-b0e9-358220c0d195" containerName="setup-container" Dec 03 11:55:32 crc kubenswrapper[5052]: E1203 11:55:32.630223 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="d3222c69-47bc-46d8-a025-cc457881cac9" containerName="nova-api-log" Dec 03 11:55:32 crc kubenswrapper[5052]: I1203 11:55:32.630231 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="d3222c69-47bc-46d8-a025-cc457881cac9" containerName="nova-api-log" Dec 03 11:55:32 crc kubenswrapper[5052]: E1203 11:55:32.630242 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="d3222c69-47bc-46d8-a025-cc457881cac9" containerName="nova-api-api" Dec 03 11:55:32 crc kubenswrapper[5052]: I1203 11:55:32.630250 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="d3222c69-47bc-46d8-a025-cc457881cac9" containerName="nova-api-api" Dec 03 11:55:32 crc kubenswrapper[5052]: E1203 11:55:32.630259 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="337bf4ce-ed6a-47bb-820d-6d64cd15e16e" containerName="rsync" Dec 03 11:55:32 crc kubenswrapper[5052]: I1203 11:55:32.630266 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="337bf4ce-ed6a-47bb-820d-6d64cd15e16e" containerName="rsync" Dec 03 11:55:32 crc kubenswrapper[5052]: E1203 11:55:32.630275 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="83b9fcfd-8007-4d6b-9090-d2e536cb1bbc" containerName="neutron-httpd" Dec 03 11:55:32 crc kubenswrapper[5052]: I1203 11:55:32.630282 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="83b9fcfd-8007-4d6b-9090-d2e536cb1bbc" containerName="neutron-httpd" Dec 03 11:55:32 crc kubenswrapper[5052]: E1203 11:55:32.630297 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="5679a95d-a498-448e-97ed-497496520506" containerName="barbican-keystone-listener" Dec 03 11:55:32 crc kubenswrapper[5052]: I1203 11:55:32.630304 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="5679a95d-a498-448e-97ed-497496520506" containerName="barbican-keystone-listener" Dec 03 11:55:32 crc kubenswrapper[5052]: E1203 11:55:32.630318 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="337bf4ce-ed6a-47bb-820d-6d64cd15e16e" containerName="object-updater" Dec 03 11:55:32 crc kubenswrapper[5052]: I1203 11:55:32.630326 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="337bf4ce-ed6a-47bb-820d-6d64cd15e16e" containerName="object-updater" Dec 03 11:55:32 crc kubenswrapper[5052]: E1203 11:55:32.630338 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="428e9203-aa9c-48d3-84c9-4dfaa806140d" containerName="placement-api" Dec 03 11:55:32 crc kubenswrapper[5052]: I1203 11:55:32.630346 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="428e9203-aa9c-48d3-84c9-4dfaa806140d" containerName="placement-api" Dec 03 11:55:32 crc kubenswrapper[5052]: E1203 11:55:32.630359 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="337bf4ce-ed6a-47bb-820d-6d64cd15e16e" containerName="swift-recon-cron" Dec 03 11:55:32 crc kubenswrapper[5052]: I1203 11:55:32.630366 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="337bf4ce-ed6a-47bb-820d-6d64cd15e16e" containerName="swift-recon-cron" Dec 03 11:55:32 crc kubenswrapper[5052]: E1203 11:55:32.630375 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="036bc2b0-bc50-4c91-bea5-882b9e592e4f" containerName="proxy-httpd" Dec 03 11:55:32 crc kubenswrapper[5052]: I1203 11:55:32.630383 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="036bc2b0-bc50-4c91-bea5-882b9e592e4f" containerName="proxy-httpd" Dec 03 11:55:32 crc kubenswrapper[5052]: I1203 11:55:32.630588 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="337bf4ce-ed6a-47bb-820d-6d64cd15e16e" containerName="account-auditor" Dec 03 11:55:32 crc kubenswrapper[5052]: I1203 11:55:32.630638 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="036bc2b0-bc50-4c91-bea5-882b9e592e4f" containerName="proxy-server" Dec 03 11:55:32 crc kubenswrapper[5052]: I1203 11:55:32.630653 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="f856a02f-3f21-4fd4-8158-50a232de756e" containerName="galera" Dec 03 11:55:32 crc kubenswrapper[5052]: I1203 11:55:32.630664 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="10e3094e-f625-4d69-b0e9-358220c0d195" containerName="rabbitmq" Dec 03 11:55:32 crc kubenswrapper[5052]: I1203 11:55:32.630674 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="584deaba-f945-42a2-8c0d-e531d6f48fd8" containerName="barbican-worker" Dec 03 11:55:32 crc kubenswrapper[5052]: I1203 11:55:32.630688 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="6087dc7e-081b-4392-adb6-205b2aec805f" containerName="mariadb-account-delete" Dec 03 11:55:32 crc kubenswrapper[5052]: I1203 11:55:32.630701 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="0bda7b03-76fd-405e-9a69-b897331e829c" containerName="mariadb-account-delete" Dec 03 11:55:32 crc kubenswrapper[5052]: I1203 11:55:32.630721 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="337bf4ce-ed6a-47bb-820d-6d64cd15e16e" containerName="object-auditor" Dec 03 11:55:32 crc kubenswrapper[5052]: I1203 11:55:32.630732 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="428e9203-aa9c-48d3-84c9-4dfaa806140d" containerName="placement-api" Dec 03 11:55:32 crc kubenswrapper[5052]: I1203 11:55:32.630744 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="0201d941-cd05-4d1c-9af6-f3a82a375a0f" containerName="probe" Dec 03 11:55:32 crc kubenswrapper[5052]: I1203 11:55:32.630758 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="83b9fcfd-8007-4d6b-9090-d2e536cb1bbc" containerName="neutron-api" Dec 03 11:55:32 crc kubenswrapper[5052]: I1203 11:55:32.630772 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="83b9fcfd-8007-4d6b-9090-d2e536cb1bbc" containerName="neutron-httpd" Dec 03 11:55:32 crc kubenswrapper[5052]: I1203 11:55:32.630791 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="75dd245b-b54f-4a9a-9627-cbf338a69566" containerName="openstack-network-exporter" Dec 03 11:55:32 crc kubenswrapper[5052]: I1203 11:55:32.630803 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="337bf4ce-ed6a-47bb-820d-6d64cd15e16e" containerName="swift-recon-cron" Dec 03 11:55:32 crc kubenswrapper[5052]: I1203 11:55:32.630814 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="f32095c7-4e6f-43c7-a16f-513e89f7c28e" containerName="glance-log" Dec 03 11:55:32 crc kubenswrapper[5052]: I1203 11:55:32.630825 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="75dd245b-b54f-4a9a-9627-cbf338a69566" containerName="ovsdbserver-sb" Dec 03 11:55:32 crc kubenswrapper[5052]: I1203 11:55:32.630839 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="337bf4ce-ed6a-47bb-820d-6d64cd15e16e" containerName="object-server" Dec 03 11:55:32 crc kubenswrapper[5052]: I1203 11:55:32.630850 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="f32095c7-4e6f-43c7-a16f-513e89f7c28e" containerName="glance-httpd" Dec 03 11:55:32 crc kubenswrapper[5052]: I1203 11:55:32.630862 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="53a152d4-c58c-40a7-b4ce-b9a9dc42337d" containerName="memcached" Dec 03 11:55:32 crc kubenswrapper[5052]: I1203 11:55:32.630872 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="0201d941-cd05-4d1c-9af6-f3a82a375a0f" containerName="cinder-scheduler" Dec 03 11:55:32 crc kubenswrapper[5052]: I1203 11:55:32.630889 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="5679a95d-a498-448e-97ed-497496520506" containerName="barbican-keystone-listener-log" Dec 03 11:55:32 crc kubenswrapper[5052]: I1203 11:55:32.630904 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="a18602ce-7206-4edc-a8d3-5d0754c3da02" containerName="barbican-api" Dec 03 11:55:32 crc kubenswrapper[5052]: I1203 11:55:32.630913 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="a73e0afd-e037-41f7-b214-d4eebd4bafe0" containerName="mariadb-account-delete" Dec 03 11:55:32 crc kubenswrapper[5052]: I1203 11:55:32.630927 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="b4961b81-97b8-4174-82bf-2d1d02cf8335" containerName="ovsdbserver-nb" Dec 03 11:55:32 crc kubenswrapper[5052]: I1203 11:55:32.630945 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="9a4b1245-d95c-4eb6-8d2b-fdd93d98bb1e" containerName="cinder-api" Dec 03 11:55:32 crc kubenswrapper[5052]: I1203 11:55:32.630960 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="337bf4ce-ed6a-47bb-820d-6d64cd15e16e" containerName="container-replicator" Dec 03 11:55:32 crc kubenswrapper[5052]: I1203 11:55:32.630973 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="428e9203-aa9c-48d3-84c9-4dfaa806140d" containerName="placement-log" Dec 03 11:55:32 crc kubenswrapper[5052]: I1203 11:55:32.630993 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="383e7933-2bc4-4ead-bc34-0832091bd952" containerName="mariadb-account-delete" Dec 03 11:55:32 crc kubenswrapper[5052]: I1203 11:55:32.631005 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="a5ff0249-173a-4793-8cbe-0aa749f34632" containerName="sg-core" Dec 03 11:55:32 crc kubenswrapper[5052]: I1203 11:55:32.631020 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="8dd63158-2c75-455f-8f58-57a55c7e0f22" containerName="galera" Dec 03 11:55:32 crc kubenswrapper[5052]: I1203 11:55:32.631034 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="0f42494c-058e-417d-9e0d-97ce7040b288" containerName="mariadb-account-delete" Dec 03 11:55:32 crc kubenswrapper[5052]: I1203 11:55:32.631053 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="337bf4ce-ed6a-47bb-820d-6d64cd15e16e" containerName="object-expirer" Dec 03 11:55:32 crc kubenswrapper[5052]: I1203 11:55:32.631065 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="a5ba4775-815a-46ef-bd2d-05dfa7513288" containerName="rabbitmq" Dec 03 11:55:32 crc kubenswrapper[5052]: I1203 11:55:32.631079 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="a5ff0249-173a-4793-8cbe-0aa749f34632" containerName="ceilometer-notification-agent" Dec 03 11:55:32 crc kubenswrapper[5052]: I1203 11:55:32.631093 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="53d7a9a2-b6a9-42dc-9828-d8ceaa70f0a3" containerName="mariadb-account-delete" Dec 03 11:55:32 crc kubenswrapper[5052]: I1203 11:55:32.631108 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="337bf4ce-ed6a-47bb-820d-6d64cd15e16e" containerName="object-updater" Dec 03 11:55:32 crc kubenswrapper[5052]: I1203 11:55:32.631125 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="337bf4ce-ed6a-47bb-820d-6d64cd15e16e" containerName="account-reaper" Dec 03 11:55:32 crc kubenswrapper[5052]: I1203 11:55:32.631137 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="93d03e32-f318-470c-ad5d-f17a609f5003" containerName="mariadb-account-delete" Dec 03 11:55:32 crc kubenswrapper[5052]: I1203 11:55:32.631156 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="653882ec-4522-4979-b1aa-7fad5023605c" containerName="nova-metadata-metadata" Dec 03 11:55:32 crc kubenswrapper[5052]: I1203 11:55:32.631171 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="88e7db65-8253-474b-b115-cd3f6edaf85c" containerName="kube-state-metrics" Dec 03 11:55:32 crc kubenswrapper[5052]: I1203 11:55:32.631186 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="9a4b1245-d95c-4eb6-8d2b-fdd93d98bb1e" containerName="cinder-api-log" Dec 03 11:55:32 crc kubenswrapper[5052]: I1203 11:55:32.631202 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="337bf4ce-ed6a-47bb-820d-6d64cd15e16e" containerName="rsync" Dec 03 11:55:32 crc kubenswrapper[5052]: I1203 11:55:32.631215 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="a5ff0249-173a-4793-8cbe-0aa749f34632" containerName="ceilometer-central-agent" Dec 03 11:55:32 crc kubenswrapper[5052]: I1203 11:55:32.631233 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="5679a95d-a498-448e-97ed-497496520506" containerName="barbican-keystone-listener" Dec 03 11:55:32 crc kubenswrapper[5052]: I1203 11:55:32.631245 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="036bc2b0-bc50-4c91-bea5-882b9e592e4f" containerName="proxy-httpd" Dec 03 11:55:32 crc kubenswrapper[5052]: I1203 11:55:32.631259 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="b4961b81-97b8-4174-82bf-2d1d02cf8335" containerName="openstack-network-exporter" Dec 03 11:55:32 crc kubenswrapper[5052]: I1203 11:55:32.631272 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="337bf4ce-ed6a-47bb-820d-6d64cd15e16e" containerName="container-updater" Dec 03 11:55:32 crc kubenswrapper[5052]: I1203 11:55:32.631289 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="5d0169e1-4f74-4f6e-89cc-70d2f79c34e2" containerName="dnsmasq-dns" Dec 03 11:55:32 crc kubenswrapper[5052]: I1203 11:55:32.631298 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="a18602ce-7206-4edc-a8d3-5d0754c3da02" containerName="barbican-api-log" Dec 03 11:55:32 crc kubenswrapper[5052]: I1203 11:55:32.631311 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="653882ec-4522-4979-b1aa-7fad5023605c" containerName="nova-metadata-log" Dec 03 11:55:32 crc kubenswrapper[5052]: I1203 11:55:32.631321 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="337bf4ce-ed6a-47bb-820d-6d64cd15e16e" containerName="account-server" Dec 03 11:55:32 crc kubenswrapper[5052]: I1203 11:55:32.631339 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="337bf4ce-ed6a-47bb-820d-6d64cd15e16e" containerName="container-auditor" Dec 03 11:55:32 crc kubenswrapper[5052]: I1203 11:55:32.631354 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="4ec0d450-cd52-432e-8bb2-1ef9698821c1" containerName="nova-cell0-conductor-conductor" Dec 03 11:55:32 crc kubenswrapper[5052]: I1203 11:55:32.631366 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="a5ff0249-173a-4793-8cbe-0aa749f34632" containerName="proxy-httpd" Dec 03 11:55:32 crc kubenswrapper[5052]: I1203 11:55:32.631380 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="337bf4ce-ed6a-47bb-820d-6d64cd15e16e" containerName="account-replicator" Dec 03 11:55:32 crc kubenswrapper[5052]: I1203 11:55:32.631394 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="584deaba-f945-42a2-8c0d-e531d6f48fd8" containerName="barbican-worker-log" Dec 03 11:55:32 crc kubenswrapper[5052]: I1203 11:55:32.631407 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="d3222c69-47bc-46d8-a025-cc457881cac9" containerName="nova-api-api" Dec 03 11:55:32 crc kubenswrapper[5052]: I1203 11:55:32.631423 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="d3222c69-47bc-46d8-a025-cc457881cac9" containerName="nova-api-log" Dec 03 11:55:32 crc kubenswrapper[5052]: I1203 11:55:32.631438 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="337bf4ce-ed6a-47bb-820d-6d64cd15e16e" containerName="container-server" Dec 03 11:55:32 crc kubenswrapper[5052]: I1203 11:55:32.631456 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="d541d73d-2f95-43a8-8409-40f5f2ff54c4" containerName="glance-httpd" Dec 03 11:55:32 crc kubenswrapper[5052]: I1203 11:55:32.631469 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="4a181d2c-7595-4394-98de-957451309c98" containerName="nova-cell1-conductor-conductor" Dec 03 11:55:32 crc kubenswrapper[5052]: I1203 11:55:32.631487 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="9d8cd78f-2670-4933-81dd-01cdc0044454" containerName="ovn-northd" Dec 03 11:55:32 crc kubenswrapper[5052]: I1203 11:55:32.631530 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="775d2ccf-9b52-4274-99ef-2c33b18beaea" containerName="keystone-api" Dec 03 11:55:32 crc kubenswrapper[5052]: I1203 11:55:32.631544 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="ec16e8c3-9750-4475-a671-17b0c16826c8" containerName="nova-scheduler-scheduler" Dec 03 11:55:32 crc kubenswrapper[5052]: I1203 11:55:32.631560 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="d541d73d-2f95-43a8-8409-40f5f2ff54c4" containerName="glance-log" Dec 03 11:55:32 crc kubenswrapper[5052]: I1203 11:55:32.631571 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="da3a810d-9fd0-4f18-8f4a-e896b94b1431" containerName="nova-cell1-novncproxy-novncproxy" Dec 03 11:55:32 crc kubenswrapper[5052]: I1203 11:55:32.631588 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="9d8cd78f-2670-4933-81dd-01cdc0044454" containerName="openstack-network-exporter" Dec 03 11:55:32 crc kubenswrapper[5052]: I1203 11:55:32.631603 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="337bf4ce-ed6a-47bb-820d-6d64cd15e16e" containerName="object-replicator" Dec 03 11:55:32 crc kubenswrapper[5052]: I1203 11:55:32.632308 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="crc-storage/crc-storage-crc-zskmc" Dec 03 11:55:32 crc kubenswrapper[5052]: I1203 11:55:32.635569 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"crc-storage"/"crc-storage" Dec 03 11:55:32 crc kubenswrapper[5052]: I1203 11:55:32.635919 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"crc-storage"/"openshift-service-ca.crt" Dec 03 11:55:32 crc kubenswrapper[5052]: I1203 11:55:32.636126 5052 reflector.go:368] Caches populated for *v1.Secret from object-"crc-storage"/"crc-storage-dockercfg-mf76g" Dec 03 11:55:32 crc kubenswrapper[5052]: I1203 11:55:32.637851 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["crc-storage/crc-storage-crc-zskmc"] Dec 03 11:55:32 crc kubenswrapper[5052]: I1203 11:55:32.639854 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"crc-storage"/"kube-root-ca.crt" Dec 03 11:55:32 crc kubenswrapper[5052]: I1203 11:55:32.682483 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s2zcf\" (UniqueName: \"kubernetes.io/projected/b8f9b6c4-1875-404a-97ca-218104f15822-kube-api-access-s2zcf\") pod \"crc-storage-crc-zskmc\" (UID: \"b8f9b6c4-1875-404a-97ca-218104f15822\") " pod="crc-storage/crc-storage-crc-zskmc" Dec 03 11:55:32 crc kubenswrapper[5052]: I1203 11:55:32.682879 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/b8f9b6c4-1875-404a-97ca-218104f15822-node-mnt\") pod \"crc-storage-crc-zskmc\" (UID: \"b8f9b6c4-1875-404a-97ca-218104f15822\") " pod="crc-storage/crc-storage-crc-zskmc" Dec 03 11:55:32 crc kubenswrapper[5052]: I1203 11:55:32.682979 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/b8f9b6c4-1875-404a-97ca-218104f15822-crc-storage\") pod \"crc-storage-crc-zskmc\" (UID: \"b8f9b6c4-1875-404a-97ca-218104f15822\") " pod="crc-storage/crc-storage-crc-zskmc" Dec 03 11:55:32 crc kubenswrapper[5052]: I1203 11:55:32.784618 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/b8f9b6c4-1875-404a-97ca-218104f15822-node-mnt\") pod \"crc-storage-crc-zskmc\" (UID: \"b8f9b6c4-1875-404a-97ca-218104f15822\") " pod="crc-storage/crc-storage-crc-zskmc" Dec 03 11:55:32 crc kubenswrapper[5052]: I1203 11:55:32.784923 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/b8f9b6c4-1875-404a-97ca-218104f15822-crc-storage\") pod \"crc-storage-crc-zskmc\" (UID: \"b8f9b6c4-1875-404a-97ca-218104f15822\") " pod="crc-storage/crc-storage-crc-zskmc" Dec 03 11:55:32 crc kubenswrapper[5052]: I1203 11:55:32.785025 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-s2zcf\" (UniqueName: \"kubernetes.io/projected/b8f9b6c4-1875-404a-97ca-218104f15822-kube-api-access-s2zcf\") pod \"crc-storage-crc-zskmc\" (UID: \"b8f9b6c4-1875-404a-97ca-218104f15822\") " pod="crc-storage/crc-storage-crc-zskmc" Dec 03 11:55:32 crc kubenswrapper[5052]: I1203 11:55:32.784871 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/b8f9b6c4-1875-404a-97ca-218104f15822-node-mnt\") pod \"crc-storage-crc-zskmc\" (UID: \"b8f9b6c4-1875-404a-97ca-218104f15822\") " pod="crc-storage/crc-storage-crc-zskmc" Dec 03 11:55:32 crc kubenswrapper[5052]: I1203 11:55:32.785807 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/b8f9b6c4-1875-404a-97ca-218104f15822-crc-storage\") pod \"crc-storage-crc-zskmc\" (UID: \"b8f9b6c4-1875-404a-97ca-218104f15822\") " pod="crc-storage/crc-storage-crc-zskmc" Dec 03 11:55:32 crc kubenswrapper[5052]: I1203 11:55:32.802190 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-s2zcf\" (UniqueName: \"kubernetes.io/projected/b8f9b6c4-1875-404a-97ca-218104f15822-kube-api-access-s2zcf\") pod \"crc-storage-crc-zskmc\" (UID: \"b8f9b6c4-1875-404a-97ca-218104f15822\") " pod="crc-storage/crc-storage-crc-zskmc" Dec 03 11:55:32 crc kubenswrapper[5052]: I1203 11:55:32.950604 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="crc-storage/crc-storage-crc-zskmc" Dec 03 11:55:33 crc kubenswrapper[5052]: I1203 11:55:33.351364 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["crc-storage/crc-storage-crc-zskmc"] Dec 03 11:55:33 crc kubenswrapper[5052]: I1203 11:55:33.683543 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f1ba93eb-7c01-4add-b32d-8523c5e7808e" path="/var/lib/kubelet/pods/f1ba93eb-7c01-4add-b32d-8523c5e7808e/volumes" Dec 03 11:55:34 crc kubenswrapper[5052]: I1203 11:55:34.290233 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="crc-storage/crc-storage-crc-zskmc" event={"ID":"b8f9b6c4-1875-404a-97ca-218104f15822","Type":"ContainerStarted","Data":"7e9a14f9f69b52e3e2033dbde0cd158690a3be283164b91ed0b5060bbfc4a374"} Dec 03 11:55:34 crc kubenswrapper[5052]: I1203 11:55:34.695073 5052 scope.go:117] "RemoveContainer" containerID="a23cab28141cdcf3f5726567816a96041bb63b99717c6ec1de9a11677f249a09" Dec 03 11:55:34 crc kubenswrapper[5052]: I1203 11:55:34.719902 5052 scope.go:117] "RemoveContainer" containerID="92f0af5182d1bd32fb01791a918d23abf78685fd72687f4722dc812f9d7e2f5e" Dec 03 11:55:34 crc kubenswrapper[5052]: I1203 11:55:34.757216 5052 scope.go:117] "RemoveContainer" containerID="783ae349abd4666b2f1a54103fcd03ff8d6cf865a685a1b5c3faed8dae76e348" Dec 03 11:55:34 crc kubenswrapper[5052]: I1203 11:55:34.837358 5052 scope.go:117] "RemoveContainer" containerID="77ae7474c6477ac30175222ea524c1ac6cf15f586c4f63872104bcf7083b3836" Dec 03 11:55:34 crc kubenswrapper[5052]: I1203 11:55:34.865633 5052 scope.go:117] "RemoveContainer" containerID="04d3797bc8a2e14f5d24f9cb2c6ee2c3f3c3bca4f308b9679823623d214051bf" Dec 03 11:55:34 crc kubenswrapper[5052]: I1203 11:55:34.894002 5052 scope.go:117] "RemoveContainer" containerID="166938713bd25e93070764dc802907a92def42903c27f9f3ee3ea93bf02238e5" Dec 03 11:55:34 crc kubenswrapper[5052]: I1203 11:55:34.919426 5052 scope.go:117] "RemoveContainer" containerID="a796623231501a155ae6a6f3667bcb4114e657680ac474de044e4ecdac197168" Dec 03 11:55:34 crc kubenswrapper[5052]: I1203 11:55:34.945984 5052 scope.go:117] "RemoveContainer" containerID="1f0ebc3a3c2e59c7963f57f2c53a6a37ed280f6b4a8ba80dc35ffc947af1383a" Dec 03 11:55:34 crc kubenswrapper[5052]: I1203 11:55:34.967767 5052 scope.go:117] "RemoveContainer" containerID="8dd2dc759600bdcc2a723346b6089dd30b5efa1d3c638543a2d02e0bfda6a7f1" Dec 03 11:55:35 crc kubenswrapper[5052]: I1203 11:55:35.010323 5052 scope.go:117] "RemoveContainer" containerID="586f73aa377f7546d2b8ba5198112bca3f096ed69864b933f12805e0f366fdea" Dec 03 11:55:35 crc kubenswrapper[5052]: I1203 11:55:35.039097 5052 scope.go:117] "RemoveContainer" containerID="3075a9e061bf97d3b130a49220cf7a765251499b5d0b7a0bd53a3647974aaaee" Dec 03 11:55:35 crc kubenswrapper[5052]: I1203 11:55:35.068837 5052 scope.go:117] "RemoveContainer" containerID="129c9172d0e6d0d175261d5f3c8438735b0e303535baa4945b26905504879c36" Dec 03 11:55:35 crc kubenswrapper[5052]: I1203 11:55:35.308300 5052 generic.go:334] "Generic (PLEG): container finished" podID="b8f9b6c4-1875-404a-97ca-218104f15822" containerID="d1f2e599d330bacf497ed2a3bd6fd79e8f02effc0561f9015a472179a4829d62" exitCode=0 Dec 03 11:55:35 crc kubenswrapper[5052]: I1203 11:55:35.308369 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="crc-storage/crc-storage-crc-zskmc" event={"ID":"b8f9b6c4-1875-404a-97ca-218104f15822","Type":"ContainerDied","Data":"d1f2e599d330bacf497ed2a3bd6fd79e8f02effc0561f9015a472179a4829d62"} Dec 03 11:55:36 crc kubenswrapper[5052]: I1203 11:55:36.304711 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openshift-marketplace/certified-operators-tbvzn"] Dec 03 11:55:36 crc kubenswrapper[5052]: I1203 11:55:36.309752 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/certified-operators-tbvzn"] Dec 03 11:55:36 crc kubenswrapper[5052]: I1203 11:55:36.310141 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/certified-operators-tbvzn" Dec 03 11:55:36 crc kubenswrapper[5052]: I1203 11:55:36.435057 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/0331d49e-36e9-4d40-940a-728eef79277f-utilities\") pod \"certified-operators-tbvzn\" (UID: \"0331d49e-36e9-4d40-940a-728eef79277f\") " pod="openshift-marketplace/certified-operators-tbvzn" Dec 03 11:55:36 crc kubenswrapper[5052]: I1203 11:55:36.435156 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v2m9m\" (UniqueName: \"kubernetes.io/projected/0331d49e-36e9-4d40-940a-728eef79277f-kube-api-access-v2m9m\") pod \"certified-operators-tbvzn\" (UID: \"0331d49e-36e9-4d40-940a-728eef79277f\") " pod="openshift-marketplace/certified-operators-tbvzn" Dec 03 11:55:36 crc kubenswrapper[5052]: I1203 11:55:36.435381 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/0331d49e-36e9-4d40-940a-728eef79277f-catalog-content\") pod \"certified-operators-tbvzn\" (UID: \"0331d49e-36e9-4d40-940a-728eef79277f\") " pod="openshift-marketplace/certified-operators-tbvzn" Dec 03 11:55:36 crc kubenswrapper[5052]: I1203 11:55:36.537123 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-v2m9m\" (UniqueName: \"kubernetes.io/projected/0331d49e-36e9-4d40-940a-728eef79277f-kube-api-access-v2m9m\") pod \"certified-operators-tbvzn\" (UID: \"0331d49e-36e9-4d40-940a-728eef79277f\") " pod="openshift-marketplace/certified-operators-tbvzn" Dec 03 11:55:36 crc kubenswrapper[5052]: I1203 11:55:36.537221 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/0331d49e-36e9-4d40-940a-728eef79277f-catalog-content\") pod \"certified-operators-tbvzn\" (UID: \"0331d49e-36e9-4d40-940a-728eef79277f\") " pod="openshift-marketplace/certified-operators-tbvzn" Dec 03 11:55:36 crc kubenswrapper[5052]: I1203 11:55:36.537265 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/0331d49e-36e9-4d40-940a-728eef79277f-utilities\") pod \"certified-operators-tbvzn\" (UID: \"0331d49e-36e9-4d40-940a-728eef79277f\") " pod="openshift-marketplace/certified-operators-tbvzn" Dec 03 11:55:36 crc kubenswrapper[5052]: I1203 11:55:36.538600 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/0331d49e-36e9-4d40-940a-728eef79277f-utilities\") pod \"certified-operators-tbvzn\" (UID: \"0331d49e-36e9-4d40-940a-728eef79277f\") " pod="openshift-marketplace/certified-operators-tbvzn" Dec 03 11:55:36 crc kubenswrapper[5052]: I1203 11:55:36.538641 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/0331d49e-36e9-4d40-940a-728eef79277f-catalog-content\") pod \"certified-operators-tbvzn\" (UID: \"0331d49e-36e9-4d40-940a-728eef79277f\") " pod="openshift-marketplace/certified-operators-tbvzn" Dec 03 11:55:36 crc kubenswrapper[5052]: I1203 11:55:36.564129 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-v2m9m\" (UniqueName: \"kubernetes.io/projected/0331d49e-36e9-4d40-940a-728eef79277f-kube-api-access-v2m9m\") pod \"certified-operators-tbvzn\" (UID: \"0331d49e-36e9-4d40-940a-728eef79277f\") " pod="openshift-marketplace/certified-operators-tbvzn" Dec 03 11:55:36 crc kubenswrapper[5052]: I1203 11:55:36.605751 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="crc-storage/crc-storage-crc-zskmc" Dec 03 11:55:36 crc kubenswrapper[5052]: I1203 11:55:36.637273 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/certified-operators-tbvzn" Dec 03 11:55:36 crc kubenswrapper[5052]: I1203 11:55:36.740003 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s2zcf\" (UniqueName: \"kubernetes.io/projected/b8f9b6c4-1875-404a-97ca-218104f15822-kube-api-access-s2zcf\") pod \"b8f9b6c4-1875-404a-97ca-218104f15822\" (UID: \"b8f9b6c4-1875-404a-97ca-218104f15822\") " Dec 03 11:55:36 crc kubenswrapper[5052]: I1203 11:55:36.740325 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/b8f9b6c4-1875-404a-97ca-218104f15822-crc-storage\") pod \"b8f9b6c4-1875-404a-97ca-218104f15822\" (UID: \"b8f9b6c4-1875-404a-97ca-218104f15822\") " Dec 03 11:55:36 crc kubenswrapper[5052]: I1203 11:55:36.740368 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/b8f9b6c4-1875-404a-97ca-218104f15822-node-mnt\") pod \"b8f9b6c4-1875-404a-97ca-218104f15822\" (UID: \"b8f9b6c4-1875-404a-97ca-218104f15822\") " Dec 03 11:55:36 crc kubenswrapper[5052]: I1203 11:55:36.741234 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/b8f9b6c4-1875-404a-97ca-218104f15822-node-mnt" (OuterVolumeSpecName: "node-mnt") pod "b8f9b6c4-1875-404a-97ca-218104f15822" (UID: "b8f9b6c4-1875-404a-97ca-218104f15822"). InnerVolumeSpecName "node-mnt". PluginName "kubernetes.io/host-path", VolumeGidValue "" Dec 03 11:55:36 crc kubenswrapper[5052]: I1203 11:55:36.773620 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b8f9b6c4-1875-404a-97ca-218104f15822-kube-api-access-s2zcf" (OuterVolumeSpecName: "kube-api-access-s2zcf") pod "b8f9b6c4-1875-404a-97ca-218104f15822" (UID: "b8f9b6c4-1875-404a-97ca-218104f15822"). InnerVolumeSpecName "kube-api-access-s2zcf". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:55:36 crc kubenswrapper[5052]: I1203 11:55:36.775987 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b8f9b6c4-1875-404a-97ca-218104f15822-crc-storage" (OuterVolumeSpecName: "crc-storage") pod "b8f9b6c4-1875-404a-97ca-218104f15822" (UID: "b8f9b6c4-1875-404a-97ca-218104f15822"). InnerVolumeSpecName "crc-storage". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:55:36 crc kubenswrapper[5052]: I1203 11:55:36.842273 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-s2zcf\" (UniqueName: \"kubernetes.io/projected/b8f9b6c4-1875-404a-97ca-218104f15822-kube-api-access-s2zcf\") on node \"crc\" DevicePath \"\"" Dec 03 11:55:36 crc kubenswrapper[5052]: I1203 11:55:36.842314 5052 reconciler_common.go:293] "Volume detached for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/b8f9b6c4-1875-404a-97ca-218104f15822-crc-storage\") on node \"crc\" DevicePath \"\"" Dec 03 11:55:36 crc kubenswrapper[5052]: I1203 11:55:36.842326 5052 reconciler_common.go:293] "Volume detached for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/b8f9b6c4-1875-404a-97ca-218104f15822-node-mnt\") on node \"crc\" DevicePath \"\"" Dec 03 11:55:37 crc kubenswrapper[5052]: I1203 11:55:37.127626 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/certified-operators-tbvzn"] Dec 03 11:55:37 crc kubenswrapper[5052]: I1203 11:55:37.325899 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="crc-storage/crc-storage-crc-zskmc" event={"ID":"b8f9b6c4-1875-404a-97ca-218104f15822","Type":"ContainerDied","Data":"7e9a14f9f69b52e3e2033dbde0cd158690a3be283164b91ed0b5060bbfc4a374"} Dec 03 11:55:37 crc kubenswrapper[5052]: I1203 11:55:37.326135 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7e9a14f9f69b52e3e2033dbde0cd158690a3be283164b91ed0b5060bbfc4a374" Dec 03 11:55:37 crc kubenswrapper[5052]: I1203 11:55:37.325962 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="crc-storage/crc-storage-crc-zskmc" Dec 03 11:55:37 crc kubenswrapper[5052]: I1203 11:55:37.328026 5052 generic.go:334] "Generic (PLEG): container finished" podID="0331d49e-36e9-4d40-940a-728eef79277f" containerID="455df8951e1e1ce222e199eaefa93aca76d4171e9f616f09d50cbb1fce6ab709" exitCode=0 Dec 03 11:55:37 crc kubenswrapper[5052]: I1203 11:55:37.328074 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/certified-operators-tbvzn" event={"ID":"0331d49e-36e9-4d40-940a-728eef79277f","Type":"ContainerDied","Data":"455df8951e1e1ce222e199eaefa93aca76d4171e9f616f09d50cbb1fce6ab709"} Dec 03 11:55:37 crc kubenswrapper[5052]: I1203 11:55:37.328107 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/certified-operators-tbvzn" event={"ID":"0331d49e-36e9-4d40-940a-728eef79277f","Type":"ContainerStarted","Data":"ff5e4025670376d1c99cec926718ade22ab7d0fd77309243cc83118373f329f3"} Dec 03 11:55:38 crc kubenswrapper[5052]: I1203 11:55:38.337951 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/certified-operators-tbvzn" event={"ID":"0331d49e-36e9-4d40-940a-728eef79277f","Type":"ContainerStarted","Data":"d0286ede6abbcc2979a2b987ebadc00f66a05874c311b19a12ba1e1cceeba0b8"} Dec 03 11:55:39 crc kubenswrapper[5052]: I1203 11:55:39.352795 5052 generic.go:334] "Generic (PLEG): container finished" podID="0331d49e-36e9-4d40-940a-728eef79277f" containerID="d0286ede6abbcc2979a2b987ebadc00f66a05874c311b19a12ba1e1cceeba0b8" exitCode=0 Dec 03 11:55:39 crc kubenswrapper[5052]: I1203 11:55:39.352845 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/certified-operators-tbvzn" event={"ID":"0331d49e-36e9-4d40-940a-728eef79277f","Type":"ContainerDied","Data":"d0286ede6abbcc2979a2b987ebadc00f66a05874c311b19a12ba1e1cceeba0b8"} Dec 03 11:55:40 crc kubenswrapper[5052]: I1203 11:55:40.253893 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["crc-storage/crc-storage-crc-zskmc"] Dec 03 11:55:40 crc kubenswrapper[5052]: I1203 11:55:40.277622 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["crc-storage/crc-storage-crc-zskmc"] Dec 03 11:55:40 crc kubenswrapper[5052]: I1203 11:55:40.363835 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/certified-operators-tbvzn" event={"ID":"0331d49e-36e9-4d40-940a-728eef79277f","Type":"ContainerStarted","Data":"d2c21cbe3b969ac991db2af4775c9c934da3798b487a14a91ac423c6546d1fb6"} Dec 03 11:55:40 crc kubenswrapper[5052]: I1203 11:55:40.379679 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-marketplace/certified-operators-tbvzn" podStartSLOduration=1.596173829 podStartE2EDuration="4.379661175s" podCreationTimestamp="2025-12-03 11:55:36 +0000 UTC" firstStartedPulling="2025-12-03 11:55:37.329102082 +0000 UTC m=+2127.917343856" lastFinishedPulling="2025-12-03 11:55:40.112589428 +0000 UTC m=+2130.700831202" observedRunningTime="2025-12-03 11:55:40.379218672 +0000 UTC m=+2130.967460446" watchObservedRunningTime="2025-12-03 11:55:40.379661175 +0000 UTC m=+2130.967902939" Dec 03 11:55:40 crc kubenswrapper[5052]: I1203 11:55:40.407982 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["crc-storage/crc-storage-crc-6vfwn"] Dec 03 11:55:40 crc kubenswrapper[5052]: E1203 11:55:40.408290 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="b8f9b6c4-1875-404a-97ca-218104f15822" containerName="storage" Dec 03 11:55:40 crc kubenswrapper[5052]: I1203 11:55:40.408308 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="b8f9b6c4-1875-404a-97ca-218104f15822" containerName="storage" Dec 03 11:55:40 crc kubenswrapper[5052]: I1203 11:55:40.408474 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="b8f9b6c4-1875-404a-97ca-218104f15822" containerName="storage" Dec 03 11:55:40 crc kubenswrapper[5052]: I1203 11:55:40.409023 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="crc-storage/crc-storage-crc-6vfwn" Dec 03 11:55:40 crc kubenswrapper[5052]: I1203 11:55:40.410853 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"crc-storage"/"crc-storage" Dec 03 11:55:40 crc kubenswrapper[5052]: I1203 11:55:40.410906 5052 reflector.go:368] Caches populated for *v1.Secret from object-"crc-storage"/"crc-storage-dockercfg-mf76g" Dec 03 11:55:40 crc kubenswrapper[5052]: I1203 11:55:40.411232 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"crc-storage"/"kube-root-ca.crt" Dec 03 11:55:40 crc kubenswrapper[5052]: I1203 11:55:40.411794 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"crc-storage"/"openshift-service-ca.crt" Dec 03 11:55:40 crc kubenswrapper[5052]: I1203 11:55:40.413951 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["crc-storage/crc-storage-crc-6vfwn"] Dec 03 11:55:40 crc kubenswrapper[5052]: I1203 11:55:40.499308 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jfdfl\" (UniqueName: \"kubernetes.io/projected/bbd55b7d-3f80-409a-877f-2bf99918eafc-kube-api-access-jfdfl\") pod \"crc-storage-crc-6vfwn\" (UID: \"bbd55b7d-3f80-409a-877f-2bf99918eafc\") " pod="crc-storage/crc-storage-crc-6vfwn" Dec 03 11:55:40 crc kubenswrapper[5052]: I1203 11:55:40.499386 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/bbd55b7d-3f80-409a-877f-2bf99918eafc-node-mnt\") pod \"crc-storage-crc-6vfwn\" (UID: \"bbd55b7d-3f80-409a-877f-2bf99918eafc\") " pod="crc-storage/crc-storage-crc-6vfwn" Dec 03 11:55:40 crc kubenswrapper[5052]: I1203 11:55:40.499414 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/bbd55b7d-3f80-409a-877f-2bf99918eafc-crc-storage\") pod \"crc-storage-crc-6vfwn\" (UID: \"bbd55b7d-3f80-409a-877f-2bf99918eafc\") " pod="crc-storage/crc-storage-crc-6vfwn" Dec 03 11:55:40 crc kubenswrapper[5052]: I1203 11:55:40.600867 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-jfdfl\" (UniqueName: \"kubernetes.io/projected/bbd55b7d-3f80-409a-877f-2bf99918eafc-kube-api-access-jfdfl\") pod \"crc-storage-crc-6vfwn\" (UID: \"bbd55b7d-3f80-409a-877f-2bf99918eafc\") " pod="crc-storage/crc-storage-crc-6vfwn" Dec 03 11:55:40 crc kubenswrapper[5052]: I1203 11:55:40.600993 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/bbd55b7d-3f80-409a-877f-2bf99918eafc-node-mnt\") pod \"crc-storage-crc-6vfwn\" (UID: \"bbd55b7d-3f80-409a-877f-2bf99918eafc\") " pod="crc-storage/crc-storage-crc-6vfwn" Dec 03 11:55:40 crc kubenswrapper[5052]: I1203 11:55:40.601031 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/bbd55b7d-3f80-409a-877f-2bf99918eafc-crc-storage\") pod \"crc-storage-crc-6vfwn\" (UID: \"bbd55b7d-3f80-409a-877f-2bf99918eafc\") " pod="crc-storage/crc-storage-crc-6vfwn" Dec 03 11:55:40 crc kubenswrapper[5052]: I1203 11:55:40.601597 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/bbd55b7d-3f80-409a-877f-2bf99918eafc-node-mnt\") pod \"crc-storage-crc-6vfwn\" (UID: \"bbd55b7d-3f80-409a-877f-2bf99918eafc\") " pod="crc-storage/crc-storage-crc-6vfwn" Dec 03 11:55:40 crc kubenswrapper[5052]: I1203 11:55:40.603641 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/bbd55b7d-3f80-409a-877f-2bf99918eafc-crc-storage\") pod \"crc-storage-crc-6vfwn\" (UID: \"bbd55b7d-3f80-409a-877f-2bf99918eafc\") " pod="crc-storage/crc-storage-crc-6vfwn" Dec 03 11:55:40 crc kubenswrapper[5052]: I1203 11:55:40.632709 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-jfdfl\" (UniqueName: \"kubernetes.io/projected/bbd55b7d-3f80-409a-877f-2bf99918eafc-kube-api-access-jfdfl\") pod \"crc-storage-crc-6vfwn\" (UID: \"bbd55b7d-3f80-409a-877f-2bf99918eafc\") " pod="crc-storage/crc-storage-crc-6vfwn" Dec 03 11:55:40 crc kubenswrapper[5052]: I1203 11:55:40.723372 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="crc-storage/crc-storage-crc-6vfwn" Dec 03 11:55:40 crc kubenswrapper[5052]: I1203 11:55:40.941358 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["crc-storage/crc-storage-crc-6vfwn"] Dec 03 11:55:40 crc kubenswrapper[5052]: W1203 11:55:40.945947 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbbd55b7d_3f80_409a_877f_2bf99918eafc.slice/crio-6a1ce748aa817e79a39a61470e910544fa1ac757abc6ca3ef8bfae1b1ab4d279 WatchSource:0}: Error finding container 6a1ce748aa817e79a39a61470e910544fa1ac757abc6ca3ef8bfae1b1ab4d279: Status 404 returned error can't find the container with id 6a1ce748aa817e79a39a61470e910544fa1ac757abc6ca3ef8bfae1b1ab4d279 Dec 03 11:55:41 crc kubenswrapper[5052]: I1203 11:55:41.381989 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="crc-storage/crc-storage-crc-6vfwn" event={"ID":"bbd55b7d-3f80-409a-877f-2bf99918eafc","Type":"ContainerStarted","Data":"6a1ce748aa817e79a39a61470e910544fa1ac757abc6ca3ef8bfae1b1ab4d279"} Dec 03 11:55:41 crc kubenswrapper[5052]: I1203 11:55:41.683483 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b8f9b6c4-1875-404a-97ca-218104f15822" path="/var/lib/kubelet/pods/b8f9b6c4-1875-404a-97ca-218104f15822/volumes" Dec 03 11:55:42 crc kubenswrapper[5052]: I1203 11:55:42.390751 5052 generic.go:334] "Generic (PLEG): container finished" podID="bbd55b7d-3f80-409a-877f-2bf99918eafc" containerID="07458cccfd7ebfac3961356761dea4f45e484fb48cbe566dff5f71534efbd7ff" exitCode=0 Dec 03 11:55:42 crc kubenswrapper[5052]: I1203 11:55:42.390867 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="crc-storage/crc-storage-crc-6vfwn" event={"ID":"bbd55b7d-3f80-409a-877f-2bf99918eafc","Type":"ContainerDied","Data":"07458cccfd7ebfac3961356761dea4f45e484fb48cbe566dff5f71534efbd7ff"} Dec 03 11:55:43 crc kubenswrapper[5052]: I1203 11:55:43.657196 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="crc-storage/crc-storage-crc-6vfwn" Dec 03 11:55:43 crc kubenswrapper[5052]: I1203 11:55:43.751098 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/bbd55b7d-3f80-409a-877f-2bf99918eafc-node-mnt\") pod \"bbd55b7d-3f80-409a-877f-2bf99918eafc\" (UID: \"bbd55b7d-3f80-409a-877f-2bf99918eafc\") " Dec 03 11:55:43 crc kubenswrapper[5052]: I1203 11:55:43.751407 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jfdfl\" (UniqueName: \"kubernetes.io/projected/bbd55b7d-3f80-409a-877f-2bf99918eafc-kube-api-access-jfdfl\") pod \"bbd55b7d-3f80-409a-877f-2bf99918eafc\" (UID: \"bbd55b7d-3f80-409a-877f-2bf99918eafc\") " Dec 03 11:55:43 crc kubenswrapper[5052]: I1203 11:55:43.751209 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/bbd55b7d-3f80-409a-877f-2bf99918eafc-node-mnt" (OuterVolumeSpecName: "node-mnt") pod "bbd55b7d-3f80-409a-877f-2bf99918eafc" (UID: "bbd55b7d-3f80-409a-877f-2bf99918eafc"). InnerVolumeSpecName "node-mnt". PluginName "kubernetes.io/host-path", VolumeGidValue "" Dec 03 11:55:43 crc kubenswrapper[5052]: I1203 11:55:43.751670 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/bbd55b7d-3f80-409a-877f-2bf99918eafc-crc-storage\") pod \"bbd55b7d-3f80-409a-877f-2bf99918eafc\" (UID: \"bbd55b7d-3f80-409a-877f-2bf99918eafc\") " Dec 03 11:55:43 crc kubenswrapper[5052]: I1203 11:55:43.752332 5052 reconciler_common.go:293] "Volume detached for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/bbd55b7d-3f80-409a-877f-2bf99918eafc-node-mnt\") on node \"crc\" DevicePath \"\"" Dec 03 11:55:43 crc kubenswrapper[5052]: I1203 11:55:43.757246 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bbd55b7d-3f80-409a-877f-2bf99918eafc-kube-api-access-jfdfl" (OuterVolumeSpecName: "kube-api-access-jfdfl") pod "bbd55b7d-3f80-409a-877f-2bf99918eafc" (UID: "bbd55b7d-3f80-409a-877f-2bf99918eafc"). InnerVolumeSpecName "kube-api-access-jfdfl". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:55:43 crc kubenswrapper[5052]: I1203 11:55:43.774413 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bbd55b7d-3f80-409a-877f-2bf99918eafc-crc-storage" (OuterVolumeSpecName: "crc-storage") pod "bbd55b7d-3f80-409a-877f-2bf99918eafc" (UID: "bbd55b7d-3f80-409a-877f-2bf99918eafc"). InnerVolumeSpecName "crc-storage". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:55:43 crc kubenswrapper[5052]: I1203 11:55:43.854196 5052 reconciler_common.go:293] "Volume detached for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/bbd55b7d-3f80-409a-877f-2bf99918eafc-crc-storage\") on node \"crc\" DevicePath \"\"" Dec 03 11:55:43 crc kubenswrapper[5052]: I1203 11:55:43.854493 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-jfdfl\" (UniqueName: \"kubernetes.io/projected/bbd55b7d-3f80-409a-877f-2bf99918eafc-kube-api-access-jfdfl\") on node \"crc\" DevicePath \"\"" Dec 03 11:55:44 crc kubenswrapper[5052]: I1203 11:55:44.410940 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="crc-storage/crc-storage-crc-6vfwn" event={"ID":"bbd55b7d-3f80-409a-877f-2bf99918eafc","Type":"ContainerDied","Data":"6a1ce748aa817e79a39a61470e910544fa1ac757abc6ca3ef8bfae1b1ab4d279"} Dec 03 11:55:44 crc kubenswrapper[5052]: I1203 11:55:44.410995 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6a1ce748aa817e79a39a61470e910544fa1ac757abc6ca3ef8bfae1b1ab4d279" Dec 03 11:55:44 crc kubenswrapper[5052]: I1203 11:55:44.410994 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="crc-storage/crc-storage-crc-6vfwn" Dec 03 11:55:46 crc kubenswrapper[5052]: I1203 11:55:46.638891 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-marketplace/certified-operators-tbvzn" Dec 03 11:55:46 crc kubenswrapper[5052]: I1203 11:55:46.639610 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openshift-marketplace/certified-operators-tbvzn" Dec 03 11:55:46 crc kubenswrapper[5052]: I1203 11:55:46.688333 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-marketplace/certified-operators-tbvzn" Dec 03 11:55:47 crc kubenswrapper[5052]: I1203 11:55:47.485673 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-marketplace/certified-operators-tbvzn" Dec 03 11:55:49 crc kubenswrapper[5052]: I1203 11:55:49.729448 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openshift-marketplace/certified-operators-tbvzn"] Dec 03 11:55:49 crc kubenswrapper[5052]: I1203 11:55:49.729883 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openshift-marketplace/certified-operators-tbvzn" podUID="0331d49e-36e9-4d40-940a-728eef79277f" containerName="registry-server" containerID="cri-o://d2c21cbe3b969ac991db2af4775c9c934da3798b487a14a91ac423c6546d1fb6" gracePeriod=2 Dec 03 11:55:50 crc kubenswrapper[5052]: I1203 11:55:50.472628 5052 generic.go:334] "Generic (PLEG): container finished" podID="0331d49e-36e9-4d40-940a-728eef79277f" containerID="d2c21cbe3b969ac991db2af4775c9c934da3798b487a14a91ac423c6546d1fb6" exitCode=0 Dec 03 11:55:50 crc kubenswrapper[5052]: I1203 11:55:50.472959 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/certified-operators-tbvzn" event={"ID":"0331d49e-36e9-4d40-940a-728eef79277f","Type":"ContainerDied","Data":"d2c21cbe3b969ac991db2af4775c9c934da3798b487a14a91ac423c6546d1fb6"} Dec 03 11:55:50 crc kubenswrapper[5052]: I1203 11:55:50.717721 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/certified-operators-tbvzn" Dec 03 11:55:50 crc kubenswrapper[5052]: I1203 11:55:50.795184 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/0331d49e-36e9-4d40-940a-728eef79277f-utilities\") pod \"0331d49e-36e9-4d40-940a-728eef79277f\" (UID: \"0331d49e-36e9-4d40-940a-728eef79277f\") " Dec 03 11:55:50 crc kubenswrapper[5052]: I1203 11:55:50.795281 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/0331d49e-36e9-4d40-940a-728eef79277f-catalog-content\") pod \"0331d49e-36e9-4d40-940a-728eef79277f\" (UID: \"0331d49e-36e9-4d40-940a-728eef79277f\") " Dec 03 11:55:50 crc kubenswrapper[5052]: I1203 11:55:50.795307 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v2m9m\" (UniqueName: \"kubernetes.io/projected/0331d49e-36e9-4d40-940a-728eef79277f-kube-api-access-v2m9m\") pod \"0331d49e-36e9-4d40-940a-728eef79277f\" (UID: \"0331d49e-36e9-4d40-940a-728eef79277f\") " Dec 03 11:55:50 crc kubenswrapper[5052]: I1203 11:55:50.797024 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0331d49e-36e9-4d40-940a-728eef79277f-utilities" (OuterVolumeSpecName: "utilities") pod "0331d49e-36e9-4d40-940a-728eef79277f" (UID: "0331d49e-36e9-4d40-940a-728eef79277f"). InnerVolumeSpecName "utilities". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:55:50 crc kubenswrapper[5052]: I1203 11:55:50.800096 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0331d49e-36e9-4d40-940a-728eef79277f-kube-api-access-v2m9m" (OuterVolumeSpecName: "kube-api-access-v2m9m") pod "0331d49e-36e9-4d40-940a-728eef79277f" (UID: "0331d49e-36e9-4d40-940a-728eef79277f"). InnerVolumeSpecName "kube-api-access-v2m9m". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:55:50 crc kubenswrapper[5052]: I1203 11:55:50.848223 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0331d49e-36e9-4d40-940a-728eef79277f-catalog-content" (OuterVolumeSpecName: "catalog-content") pod "0331d49e-36e9-4d40-940a-728eef79277f" (UID: "0331d49e-36e9-4d40-940a-728eef79277f"). InnerVolumeSpecName "catalog-content". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:55:50 crc kubenswrapper[5052]: I1203 11:55:50.896792 5052 reconciler_common.go:293] "Volume detached for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/0331d49e-36e9-4d40-940a-728eef79277f-utilities\") on node \"crc\" DevicePath \"\"" Dec 03 11:55:50 crc kubenswrapper[5052]: I1203 11:55:50.896826 5052 reconciler_common.go:293] "Volume detached for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/0331d49e-36e9-4d40-940a-728eef79277f-catalog-content\") on node \"crc\" DevicePath \"\"" Dec 03 11:55:50 crc kubenswrapper[5052]: I1203 11:55:50.896844 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-v2m9m\" (UniqueName: \"kubernetes.io/projected/0331d49e-36e9-4d40-940a-728eef79277f-kube-api-access-v2m9m\") on node \"crc\" DevicePath \"\"" Dec 03 11:55:51 crc kubenswrapper[5052]: I1203 11:55:51.481729 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/certified-operators-tbvzn" event={"ID":"0331d49e-36e9-4d40-940a-728eef79277f","Type":"ContainerDied","Data":"ff5e4025670376d1c99cec926718ade22ab7d0fd77309243cc83118373f329f3"} Dec 03 11:55:51 crc kubenswrapper[5052]: I1203 11:55:51.481799 5052 scope.go:117] "RemoveContainer" containerID="d2c21cbe3b969ac991db2af4775c9c934da3798b487a14a91ac423c6546d1fb6" Dec 03 11:55:51 crc kubenswrapper[5052]: I1203 11:55:51.481799 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/certified-operators-tbvzn" Dec 03 11:55:51 crc kubenswrapper[5052]: I1203 11:55:51.500078 5052 scope.go:117] "RemoveContainer" containerID="d0286ede6abbcc2979a2b987ebadc00f66a05874c311b19a12ba1e1cceeba0b8" Dec 03 11:55:51 crc kubenswrapper[5052]: I1203 11:55:51.510144 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openshift-marketplace/certified-operators-tbvzn"] Dec 03 11:55:51 crc kubenswrapper[5052]: I1203 11:55:51.515368 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openshift-marketplace/certified-operators-tbvzn"] Dec 03 11:55:51 crc kubenswrapper[5052]: I1203 11:55:51.519782 5052 scope.go:117] "RemoveContainer" containerID="455df8951e1e1ce222e199eaefa93aca76d4171e9f616f09d50cbb1fce6ab709" Dec 03 11:55:51 crc kubenswrapper[5052]: I1203 11:55:51.683573 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0331d49e-36e9-4d40-940a-728eef79277f" path="/var/lib/kubelet/pods/0331d49e-36e9-4d40-940a-728eef79277f/volumes" Dec 03 11:55:51 crc kubenswrapper[5052]: I1203 11:55:51.971415 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openshift-marketplace/6c372a8d094fad7255d3bbeabb4914bd2356af7b203a2d2176be1c9210msh2h"] Dec 03 11:55:51 crc kubenswrapper[5052]: E1203 11:55:51.972111 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="bbd55b7d-3f80-409a-877f-2bf99918eafc" containerName="storage" Dec 03 11:55:51 crc kubenswrapper[5052]: I1203 11:55:51.972128 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="bbd55b7d-3f80-409a-877f-2bf99918eafc" containerName="storage" Dec 03 11:55:51 crc kubenswrapper[5052]: E1203 11:55:51.972145 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="0331d49e-36e9-4d40-940a-728eef79277f" containerName="extract-content" Dec 03 11:55:51 crc kubenswrapper[5052]: I1203 11:55:51.972153 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="0331d49e-36e9-4d40-940a-728eef79277f" containerName="extract-content" Dec 03 11:55:51 crc kubenswrapper[5052]: E1203 11:55:51.972169 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="0331d49e-36e9-4d40-940a-728eef79277f" containerName="extract-utilities" Dec 03 11:55:51 crc kubenswrapper[5052]: I1203 11:55:51.972177 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="0331d49e-36e9-4d40-940a-728eef79277f" containerName="extract-utilities" Dec 03 11:55:51 crc kubenswrapper[5052]: E1203 11:55:51.972209 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="0331d49e-36e9-4d40-940a-728eef79277f" containerName="registry-server" Dec 03 11:55:51 crc kubenswrapper[5052]: I1203 11:55:51.972217 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="0331d49e-36e9-4d40-940a-728eef79277f" containerName="registry-server" Dec 03 11:55:51 crc kubenswrapper[5052]: I1203 11:55:51.972374 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="0331d49e-36e9-4d40-940a-728eef79277f" containerName="registry-server" Dec 03 11:55:51 crc kubenswrapper[5052]: I1203 11:55:51.972393 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="bbd55b7d-3f80-409a-877f-2bf99918eafc" containerName="storage" Dec 03 11:55:51 crc kubenswrapper[5052]: I1203 11:55:51.973668 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6c372a8d094fad7255d3bbeabb4914bd2356af7b203a2d2176be1c9210msh2h" Dec 03 11:55:51 crc kubenswrapper[5052]: I1203 11:55:51.975571 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-marketplace"/"default-dockercfg-vmwhc" Dec 03 11:55:51 crc kubenswrapper[5052]: I1203 11:55:51.981207 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6c372a8d094fad7255d3bbeabb4914bd2356af7b203a2d2176be1c9210msh2h"] Dec 03 11:55:52 crc kubenswrapper[5052]: I1203 11:55:52.110405 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c2bb83c5-741f-4f8d-ac20-4bee739a6f43-bundle\") pod \"6c372a8d094fad7255d3bbeabb4914bd2356af7b203a2d2176be1c9210msh2h\" (UID: \"c2bb83c5-741f-4f8d-ac20-4bee739a6f43\") " pod="openshift-marketplace/6c372a8d094fad7255d3bbeabb4914bd2356af7b203a2d2176be1c9210msh2h" Dec 03 11:55:52 crc kubenswrapper[5052]: I1203 11:55:52.110527 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n5mcx\" (UniqueName: \"kubernetes.io/projected/c2bb83c5-741f-4f8d-ac20-4bee739a6f43-kube-api-access-n5mcx\") pod \"6c372a8d094fad7255d3bbeabb4914bd2356af7b203a2d2176be1c9210msh2h\" (UID: \"c2bb83c5-741f-4f8d-ac20-4bee739a6f43\") " pod="openshift-marketplace/6c372a8d094fad7255d3bbeabb4914bd2356af7b203a2d2176be1c9210msh2h" Dec 03 11:55:52 crc kubenswrapper[5052]: I1203 11:55:52.110731 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c2bb83c5-741f-4f8d-ac20-4bee739a6f43-util\") pod \"6c372a8d094fad7255d3bbeabb4914bd2356af7b203a2d2176be1c9210msh2h\" (UID: \"c2bb83c5-741f-4f8d-ac20-4bee739a6f43\") " pod="openshift-marketplace/6c372a8d094fad7255d3bbeabb4914bd2356af7b203a2d2176be1c9210msh2h" Dec 03 11:55:52 crc kubenswrapper[5052]: I1203 11:55:52.212355 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c2bb83c5-741f-4f8d-ac20-4bee739a6f43-bundle\") pod \"6c372a8d094fad7255d3bbeabb4914bd2356af7b203a2d2176be1c9210msh2h\" (UID: \"c2bb83c5-741f-4f8d-ac20-4bee739a6f43\") " pod="openshift-marketplace/6c372a8d094fad7255d3bbeabb4914bd2356af7b203a2d2176be1c9210msh2h" Dec 03 11:55:52 crc kubenswrapper[5052]: I1203 11:55:52.212427 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-n5mcx\" (UniqueName: \"kubernetes.io/projected/c2bb83c5-741f-4f8d-ac20-4bee739a6f43-kube-api-access-n5mcx\") pod \"6c372a8d094fad7255d3bbeabb4914bd2356af7b203a2d2176be1c9210msh2h\" (UID: \"c2bb83c5-741f-4f8d-ac20-4bee739a6f43\") " pod="openshift-marketplace/6c372a8d094fad7255d3bbeabb4914bd2356af7b203a2d2176be1c9210msh2h" Dec 03 11:55:52 crc kubenswrapper[5052]: I1203 11:55:52.212462 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c2bb83c5-741f-4f8d-ac20-4bee739a6f43-util\") pod \"6c372a8d094fad7255d3bbeabb4914bd2356af7b203a2d2176be1c9210msh2h\" (UID: \"c2bb83c5-741f-4f8d-ac20-4bee739a6f43\") " pod="openshift-marketplace/6c372a8d094fad7255d3bbeabb4914bd2356af7b203a2d2176be1c9210msh2h" Dec 03 11:55:52 crc kubenswrapper[5052]: I1203 11:55:52.212999 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c2bb83c5-741f-4f8d-ac20-4bee739a6f43-util\") pod \"6c372a8d094fad7255d3bbeabb4914bd2356af7b203a2d2176be1c9210msh2h\" (UID: \"c2bb83c5-741f-4f8d-ac20-4bee739a6f43\") " pod="openshift-marketplace/6c372a8d094fad7255d3bbeabb4914bd2356af7b203a2d2176be1c9210msh2h" Dec 03 11:55:52 crc kubenswrapper[5052]: I1203 11:55:52.213073 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c2bb83c5-741f-4f8d-ac20-4bee739a6f43-bundle\") pod \"6c372a8d094fad7255d3bbeabb4914bd2356af7b203a2d2176be1c9210msh2h\" (UID: \"c2bb83c5-741f-4f8d-ac20-4bee739a6f43\") " pod="openshift-marketplace/6c372a8d094fad7255d3bbeabb4914bd2356af7b203a2d2176be1c9210msh2h" Dec 03 11:55:52 crc kubenswrapper[5052]: I1203 11:55:52.230395 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-n5mcx\" (UniqueName: \"kubernetes.io/projected/c2bb83c5-741f-4f8d-ac20-4bee739a6f43-kube-api-access-n5mcx\") pod \"6c372a8d094fad7255d3bbeabb4914bd2356af7b203a2d2176be1c9210msh2h\" (UID: \"c2bb83c5-741f-4f8d-ac20-4bee739a6f43\") " pod="openshift-marketplace/6c372a8d094fad7255d3bbeabb4914bd2356af7b203a2d2176be1c9210msh2h" Dec 03 11:55:52 crc kubenswrapper[5052]: I1203 11:55:52.294813 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6c372a8d094fad7255d3bbeabb4914bd2356af7b203a2d2176be1c9210msh2h" Dec 03 11:55:53 crc kubenswrapper[5052]: I1203 11:55:52.711333 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6c372a8d094fad7255d3bbeabb4914bd2356af7b203a2d2176be1c9210msh2h"] Dec 03 11:55:53 crc kubenswrapper[5052]: I1203 11:55:53.500898 5052 generic.go:334] "Generic (PLEG): container finished" podID="c2bb83c5-741f-4f8d-ac20-4bee739a6f43" containerID="541e56837d4b477c989027904924f301bc1b55448ee63ac7e471f7939d7e6ae4" exitCode=0 Dec 03 11:55:53 crc kubenswrapper[5052]: I1203 11:55:53.501257 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6c372a8d094fad7255d3bbeabb4914bd2356af7b203a2d2176be1c9210msh2h" event={"ID":"c2bb83c5-741f-4f8d-ac20-4bee739a6f43","Type":"ContainerDied","Data":"541e56837d4b477c989027904924f301bc1b55448ee63ac7e471f7939d7e6ae4"} Dec 03 11:55:53 crc kubenswrapper[5052]: I1203 11:55:53.501291 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6c372a8d094fad7255d3bbeabb4914bd2356af7b203a2d2176be1c9210msh2h" event={"ID":"c2bb83c5-741f-4f8d-ac20-4bee739a6f43","Type":"ContainerStarted","Data":"9b322860ed899935472ac3e58dc52fefb3ad3dcd59e400f79c9579a77f66c3de"} Dec 03 11:55:55 crc kubenswrapper[5052]: I1203 11:55:55.529576 5052 generic.go:334] "Generic (PLEG): container finished" podID="c2bb83c5-741f-4f8d-ac20-4bee739a6f43" containerID="ddc3c41ebe6dd9669040f3532a14f51c09a1da829dab6b03440fce4bddf0ee5a" exitCode=0 Dec 03 11:55:55 crc kubenswrapper[5052]: I1203 11:55:55.529685 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6c372a8d094fad7255d3bbeabb4914bd2356af7b203a2d2176be1c9210msh2h" event={"ID":"c2bb83c5-741f-4f8d-ac20-4bee739a6f43","Type":"ContainerDied","Data":"ddc3c41ebe6dd9669040f3532a14f51c09a1da829dab6b03440fce4bddf0ee5a"} Dec 03 11:55:56 crc kubenswrapper[5052]: I1203 11:55:56.542411 5052 generic.go:334] "Generic (PLEG): container finished" podID="c2bb83c5-741f-4f8d-ac20-4bee739a6f43" containerID="e4844c2d7826fe1f428114c94f142a0577efb85a114428f259ac21044da4dcdd" exitCode=0 Dec 03 11:55:56 crc kubenswrapper[5052]: I1203 11:55:56.542491 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6c372a8d094fad7255d3bbeabb4914bd2356af7b203a2d2176be1c9210msh2h" event={"ID":"c2bb83c5-741f-4f8d-ac20-4bee739a6f43","Type":"ContainerDied","Data":"e4844c2d7826fe1f428114c94f142a0577efb85a114428f259ac21044da4dcdd"} Dec 03 11:55:57 crc kubenswrapper[5052]: I1203 11:55:57.812277 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6c372a8d094fad7255d3bbeabb4914bd2356af7b203a2d2176be1c9210msh2h" Dec 03 11:55:57 crc kubenswrapper[5052]: I1203 11:55:57.891976 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c2bb83c5-741f-4f8d-ac20-4bee739a6f43-bundle\") pod \"c2bb83c5-741f-4f8d-ac20-4bee739a6f43\" (UID: \"c2bb83c5-741f-4f8d-ac20-4bee739a6f43\") " Dec 03 11:55:57 crc kubenswrapper[5052]: I1203 11:55:57.892098 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n5mcx\" (UniqueName: \"kubernetes.io/projected/c2bb83c5-741f-4f8d-ac20-4bee739a6f43-kube-api-access-n5mcx\") pod \"c2bb83c5-741f-4f8d-ac20-4bee739a6f43\" (UID: \"c2bb83c5-741f-4f8d-ac20-4bee739a6f43\") " Dec 03 11:55:57 crc kubenswrapper[5052]: I1203 11:55:57.892115 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c2bb83c5-741f-4f8d-ac20-4bee739a6f43-util\") pod \"c2bb83c5-741f-4f8d-ac20-4bee739a6f43\" (UID: \"c2bb83c5-741f-4f8d-ac20-4bee739a6f43\") " Dec 03 11:55:57 crc kubenswrapper[5052]: I1203 11:55:57.894061 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c2bb83c5-741f-4f8d-ac20-4bee739a6f43-bundle" (OuterVolumeSpecName: "bundle") pod "c2bb83c5-741f-4f8d-ac20-4bee739a6f43" (UID: "c2bb83c5-741f-4f8d-ac20-4bee739a6f43"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:55:57 crc kubenswrapper[5052]: I1203 11:55:57.899318 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c2bb83c5-741f-4f8d-ac20-4bee739a6f43-kube-api-access-n5mcx" (OuterVolumeSpecName: "kube-api-access-n5mcx") pod "c2bb83c5-741f-4f8d-ac20-4bee739a6f43" (UID: "c2bb83c5-741f-4f8d-ac20-4bee739a6f43"). InnerVolumeSpecName "kube-api-access-n5mcx". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:55:57 crc kubenswrapper[5052]: I1203 11:55:57.909083 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c2bb83c5-741f-4f8d-ac20-4bee739a6f43-util" (OuterVolumeSpecName: "util") pod "c2bb83c5-741f-4f8d-ac20-4bee739a6f43" (UID: "c2bb83c5-741f-4f8d-ac20-4bee739a6f43"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:55:57 crc kubenswrapper[5052]: I1203 11:55:57.994359 5052 reconciler_common.go:293] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c2bb83c5-741f-4f8d-ac20-4bee739a6f43-util\") on node \"crc\" DevicePath \"\"" Dec 03 11:55:57 crc kubenswrapper[5052]: I1203 11:55:57.994664 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-n5mcx\" (UniqueName: \"kubernetes.io/projected/c2bb83c5-741f-4f8d-ac20-4bee739a6f43-kube-api-access-n5mcx\") on node \"crc\" DevicePath \"\"" Dec 03 11:55:57 crc kubenswrapper[5052]: I1203 11:55:57.994724 5052 reconciler_common.go:293] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c2bb83c5-741f-4f8d-ac20-4bee739a6f43-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:55:58 crc kubenswrapper[5052]: I1203 11:55:58.564233 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6c372a8d094fad7255d3bbeabb4914bd2356af7b203a2d2176be1c9210msh2h" event={"ID":"c2bb83c5-741f-4f8d-ac20-4bee739a6f43","Type":"ContainerDied","Data":"9b322860ed899935472ac3e58dc52fefb3ad3dcd59e400f79c9579a77f66c3de"} Dec 03 11:55:58 crc kubenswrapper[5052]: I1203 11:55:58.564283 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9b322860ed899935472ac3e58dc52fefb3ad3dcd59e400f79c9579a77f66c3de" Dec 03 11:55:58 crc kubenswrapper[5052]: I1203 11:55:58.564301 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6c372a8d094fad7255d3bbeabb4914bd2356af7b203a2d2176be1c9210msh2h" Dec 03 11:56:08 crc kubenswrapper[5052]: I1203 11:56:08.386323 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openshift-operators/obo-prometheus-operator-668cf9dfbb-mddsw"] Dec 03 11:56:08 crc kubenswrapper[5052]: E1203 11:56:08.387180 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="c2bb83c5-741f-4f8d-ac20-4bee739a6f43" containerName="pull" Dec 03 11:56:08 crc kubenswrapper[5052]: I1203 11:56:08.387193 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="c2bb83c5-741f-4f8d-ac20-4bee739a6f43" containerName="pull" Dec 03 11:56:08 crc kubenswrapper[5052]: E1203 11:56:08.387215 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="c2bb83c5-741f-4f8d-ac20-4bee739a6f43" containerName="extract" Dec 03 11:56:08 crc kubenswrapper[5052]: I1203 11:56:08.387221 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="c2bb83c5-741f-4f8d-ac20-4bee739a6f43" containerName="extract" Dec 03 11:56:08 crc kubenswrapper[5052]: E1203 11:56:08.387233 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="c2bb83c5-741f-4f8d-ac20-4bee739a6f43" containerName="util" Dec 03 11:56:08 crc kubenswrapper[5052]: I1203 11:56:08.387239 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="c2bb83c5-741f-4f8d-ac20-4bee739a6f43" containerName="util" Dec 03 11:56:08 crc kubenswrapper[5052]: I1203 11:56:08.387371 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="c2bb83c5-741f-4f8d-ac20-4bee739a6f43" containerName="extract" Dec 03 11:56:08 crc kubenswrapper[5052]: I1203 11:56:08.387904 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-operators/obo-prometheus-operator-668cf9dfbb-mddsw" Dec 03 11:56:08 crc kubenswrapper[5052]: W1203 11:56:08.389983 5052 reflector.go:561] object-"openshift-operators"/"kube-root-ca.crt": failed to list *v1.ConfigMap: configmaps "kube-root-ca.crt" is forbidden: User "system:node:crc" cannot list resource "configmaps" in API group "" in the namespace "openshift-operators": no relationship found between node 'crc' and this object Dec 03 11:56:08 crc kubenswrapper[5052]: E1203 11:56:08.390031 5052 reflector.go:158] "Unhandled Error" err="object-\"openshift-operators\"/\"kube-root-ca.crt\": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps \"kube-root-ca.crt\" is forbidden: User \"system:node:crc\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-operators\": no relationship found between node 'crc' and this object" logger="UnhandledError" Dec 03 11:56:08 crc kubenswrapper[5052]: W1203 11:56:08.390081 5052 reflector.go:561] object-"openshift-operators"/"obo-prometheus-operator-dockercfg-5jwdl": failed to list *v1.Secret: secrets "obo-prometheus-operator-dockercfg-5jwdl" is forbidden: User "system:node:crc" cannot list resource "secrets" in API group "" in the namespace "openshift-operators": no relationship found between node 'crc' and this object Dec 03 11:56:08 crc kubenswrapper[5052]: E1203 11:56:08.390094 5052 reflector.go:158] "Unhandled Error" err="object-\"openshift-operators\"/\"obo-prometheus-operator-dockercfg-5jwdl\": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets \"obo-prometheus-operator-dockercfg-5jwdl\" is forbidden: User \"system:node:crc\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-operators\": no relationship found between node 'crc' and this object" logger="UnhandledError" Dec 03 11:56:08 crc kubenswrapper[5052]: I1203 11:56:08.390596 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-operators"/"openshift-service-ca.crt" Dec 03 11:56:08 crc kubenswrapper[5052]: I1203 11:56:08.415162 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-operators/obo-prometheus-operator-668cf9dfbb-mddsw"] Dec 03 11:56:08 crc kubenswrapper[5052]: I1203 11:56:08.433950 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s9nbd\" (UniqueName: \"kubernetes.io/projected/c0580db1-72b0-4b6b-9136-d45a283a060c-kube-api-access-s9nbd\") pod \"obo-prometheus-operator-668cf9dfbb-mddsw\" (UID: \"c0580db1-72b0-4b6b-9136-d45a283a060c\") " pod="openshift-operators/obo-prometheus-operator-668cf9dfbb-mddsw" Dec 03 11:56:08 crc kubenswrapper[5052]: I1203 11:56:08.454835 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openshift-operators/obo-prometheus-operator-admission-webhook-7cd69dc689-zh2bj"] Dec 03 11:56:08 crc kubenswrapper[5052]: I1203 11:56:08.455710 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-operators/obo-prometheus-operator-admission-webhook-7cd69dc689-zh2bj" Dec 03 11:56:08 crc kubenswrapper[5052]: I1203 11:56:08.459265 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-operators"/"obo-prometheus-operator-admission-webhook-dockercfg-f5dff" Dec 03 11:56:08 crc kubenswrapper[5052]: I1203 11:56:08.462844 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-operators"/"obo-prometheus-operator-admission-webhook-service-cert" Dec 03 11:56:08 crc kubenswrapper[5052]: I1203 11:56:08.479066 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-operators/obo-prometheus-operator-admission-webhook-7cd69dc689-zh2bj"] Dec 03 11:56:08 crc kubenswrapper[5052]: I1203 11:56:08.491312 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openshift-operators/obo-prometheus-operator-admission-webhook-7cd69dc689-cwkf4"] Dec 03 11:56:08 crc kubenswrapper[5052]: I1203 11:56:08.492476 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-operators/obo-prometheus-operator-admission-webhook-7cd69dc689-cwkf4" Dec 03 11:56:08 crc kubenswrapper[5052]: I1203 11:56:08.505728 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-operators/obo-prometheus-operator-admission-webhook-7cd69dc689-cwkf4"] Dec 03 11:56:08 crc kubenswrapper[5052]: I1203 11:56:08.542062 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6a2264eb-3e7a-4987-be04-9131bb690ebf-webhook-cert\") pod \"obo-prometheus-operator-admission-webhook-7cd69dc689-zh2bj\" (UID: \"6a2264eb-3e7a-4987-be04-9131bb690ebf\") " pod="openshift-operators/obo-prometheus-operator-admission-webhook-7cd69dc689-zh2bj" Dec 03 11:56:08 crc kubenswrapper[5052]: I1203 11:56:08.542134 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-s9nbd\" (UniqueName: \"kubernetes.io/projected/c0580db1-72b0-4b6b-9136-d45a283a060c-kube-api-access-s9nbd\") pod \"obo-prometheus-operator-668cf9dfbb-mddsw\" (UID: \"c0580db1-72b0-4b6b-9136-d45a283a060c\") " pod="openshift-operators/obo-prometheus-operator-668cf9dfbb-mddsw" Dec 03 11:56:08 crc kubenswrapper[5052]: I1203 11:56:08.542183 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ff11d0bc-dedc-433d-9e0e-a3fd2232919f-webhook-cert\") pod \"obo-prometheus-operator-admission-webhook-7cd69dc689-cwkf4\" (UID: \"ff11d0bc-dedc-433d-9e0e-a3fd2232919f\") " pod="openshift-operators/obo-prometheus-operator-admission-webhook-7cd69dc689-cwkf4" Dec 03 11:56:08 crc kubenswrapper[5052]: I1203 11:56:08.542203 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ff11d0bc-dedc-433d-9e0e-a3fd2232919f-apiservice-cert\") pod \"obo-prometheus-operator-admission-webhook-7cd69dc689-cwkf4\" (UID: \"ff11d0bc-dedc-433d-9e0e-a3fd2232919f\") " pod="openshift-operators/obo-prometheus-operator-admission-webhook-7cd69dc689-cwkf4" Dec 03 11:56:08 crc kubenswrapper[5052]: I1203 11:56:08.542234 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/6a2264eb-3e7a-4987-be04-9131bb690ebf-apiservice-cert\") pod \"obo-prometheus-operator-admission-webhook-7cd69dc689-zh2bj\" (UID: \"6a2264eb-3e7a-4987-be04-9131bb690ebf\") " pod="openshift-operators/obo-prometheus-operator-admission-webhook-7cd69dc689-zh2bj" Dec 03 11:56:08 crc kubenswrapper[5052]: I1203 11:56:08.643743 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ff11d0bc-dedc-433d-9e0e-a3fd2232919f-webhook-cert\") pod \"obo-prometheus-operator-admission-webhook-7cd69dc689-cwkf4\" (UID: \"ff11d0bc-dedc-433d-9e0e-a3fd2232919f\") " pod="openshift-operators/obo-prometheus-operator-admission-webhook-7cd69dc689-cwkf4" Dec 03 11:56:08 crc kubenswrapper[5052]: I1203 11:56:08.643792 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ff11d0bc-dedc-433d-9e0e-a3fd2232919f-apiservice-cert\") pod \"obo-prometheus-operator-admission-webhook-7cd69dc689-cwkf4\" (UID: \"ff11d0bc-dedc-433d-9e0e-a3fd2232919f\") " pod="openshift-operators/obo-prometheus-operator-admission-webhook-7cd69dc689-cwkf4" Dec 03 11:56:08 crc kubenswrapper[5052]: I1203 11:56:08.643829 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/6a2264eb-3e7a-4987-be04-9131bb690ebf-apiservice-cert\") pod \"obo-prometheus-operator-admission-webhook-7cd69dc689-zh2bj\" (UID: \"6a2264eb-3e7a-4987-be04-9131bb690ebf\") " pod="openshift-operators/obo-prometheus-operator-admission-webhook-7cd69dc689-zh2bj" Dec 03 11:56:08 crc kubenswrapper[5052]: I1203 11:56:08.643866 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6a2264eb-3e7a-4987-be04-9131bb690ebf-webhook-cert\") pod \"obo-prometheus-operator-admission-webhook-7cd69dc689-zh2bj\" (UID: \"6a2264eb-3e7a-4987-be04-9131bb690ebf\") " pod="openshift-operators/obo-prometheus-operator-admission-webhook-7cd69dc689-zh2bj" Dec 03 11:56:08 crc kubenswrapper[5052]: I1203 11:56:08.658358 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/6a2264eb-3e7a-4987-be04-9131bb690ebf-apiservice-cert\") pod \"obo-prometheus-operator-admission-webhook-7cd69dc689-zh2bj\" (UID: \"6a2264eb-3e7a-4987-be04-9131bb690ebf\") " pod="openshift-operators/obo-prometheus-operator-admission-webhook-7cd69dc689-zh2bj" Dec 03 11:56:08 crc kubenswrapper[5052]: I1203 11:56:08.659059 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ff11d0bc-dedc-433d-9e0e-a3fd2232919f-apiservice-cert\") pod \"obo-prometheus-operator-admission-webhook-7cd69dc689-cwkf4\" (UID: \"ff11d0bc-dedc-433d-9e0e-a3fd2232919f\") " pod="openshift-operators/obo-prometheus-operator-admission-webhook-7cd69dc689-cwkf4" Dec 03 11:56:08 crc kubenswrapper[5052]: I1203 11:56:08.674971 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6a2264eb-3e7a-4987-be04-9131bb690ebf-webhook-cert\") pod \"obo-prometheus-operator-admission-webhook-7cd69dc689-zh2bj\" (UID: \"6a2264eb-3e7a-4987-be04-9131bb690ebf\") " pod="openshift-operators/obo-prometheus-operator-admission-webhook-7cd69dc689-zh2bj" Dec 03 11:56:08 crc kubenswrapper[5052]: I1203 11:56:08.677074 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ff11d0bc-dedc-433d-9e0e-a3fd2232919f-webhook-cert\") pod \"obo-prometheus-operator-admission-webhook-7cd69dc689-cwkf4\" (UID: \"ff11d0bc-dedc-433d-9e0e-a3fd2232919f\") " pod="openshift-operators/obo-prometheus-operator-admission-webhook-7cd69dc689-cwkf4" Dec 03 11:56:08 crc kubenswrapper[5052]: I1203 11:56:08.704161 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openshift-operators/observability-operator-d8bb48f5d-swndl"] Dec 03 11:56:08 crc kubenswrapper[5052]: I1203 11:56:08.705093 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-operators/observability-operator-d8bb48f5d-swndl" Dec 03 11:56:08 crc kubenswrapper[5052]: I1203 11:56:08.708357 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-operators"/"observability-operator-tls" Dec 03 11:56:08 crc kubenswrapper[5052]: I1203 11:56:08.708697 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-operators"/"observability-operator-sa-dockercfg-5vgbj" Dec 03 11:56:08 crc kubenswrapper[5052]: I1203 11:56:08.727000 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-operators/observability-operator-d8bb48f5d-swndl"] Dec 03 11:56:08 crc kubenswrapper[5052]: I1203 11:56:08.781214 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-operators/obo-prometheus-operator-admission-webhook-7cd69dc689-zh2bj" Dec 03 11:56:08 crc kubenswrapper[5052]: I1203 11:56:08.822874 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-operators/obo-prometheus-operator-admission-webhook-7cd69dc689-cwkf4" Dec 03 11:56:08 crc kubenswrapper[5052]: I1203 11:56:08.853341 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-24tt9\" (UniqueName: \"kubernetes.io/projected/0e9b7522-0b02-484a-b9e6-545e8dabcbc7-kube-api-access-24tt9\") pod \"observability-operator-d8bb48f5d-swndl\" (UID: \"0e9b7522-0b02-484a-b9e6-545e8dabcbc7\") " pod="openshift-operators/observability-operator-d8bb48f5d-swndl" Dec 03 11:56:08 crc kubenswrapper[5052]: I1203 11:56:08.853431 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"observability-operator-tls\" (UniqueName: \"kubernetes.io/secret/0e9b7522-0b02-484a-b9e6-545e8dabcbc7-observability-operator-tls\") pod \"observability-operator-d8bb48f5d-swndl\" (UID: \"0e9b7522-0b02-484a-b9e6-545e8dabcbc7\") " pod="openshift-operators/observability-operator-d8bb48f5d-swndl" Dec 03 11:56:08 crc kubenswrapper[5052]: I1203 11:56:08.880492 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openshift-operators/perses-operator-5446b9c989-gqddj"] Dec 03 11:56:08 crc kubenswrapper[5052]: I1203 11:56:08.892707 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-operators/perses-operator-5446b9c989-gqddj" Dec 03 11:56:08 crc kubenswrapper[5052]: I1203 11:56:08.899805 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-operators"/"perses-operator-dockercfg-n928q" Dec 03 11:56:08 crc kubenswrapper[5052]: I1203 11:56:08.901323 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-operators/perses-operator-5446b9c989-gqddj"] Dec 03 11:56:08 crc kubenswrapper[5052]: I1203 11:56:08.955024 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-24tt9\" (UniqueName: \"kubernetes.io/projected/0e9b7522-0b02-484a-b9e6-545e8dabcbc7-kube-api-access-24tt9\") pod \"observability-operator-d8bb48f5d-swndl\" (UID: \"0e9b7522-0b02-484a-b9e6-545e8dabcbc7\") " pod="openshift-operators/observability-operator-d8bb48f5d-swndl" Dec 03 11:56:08 crc kubenswrapper[5052]: I1203 11:56:08.955083 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8b649975-2e0f-4406-9494-d0e0a21fc168-openshift-service-ca\") pod \"perses-operator-5446b9c989-gqddj\" (UID: \"8b649975-2e0f-4406-9494-d0e0a21fc168\") " pod="openshift-operators/perses-operator-5446b9c989-gqddj" Dec 03 11:56:08 crc kubenswrapper[5052]: I1203 11:56:08.955141 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6n4d5\" (UniqueName: \"kubernetes.io/projected/8b649975-2e0f-4406-9494-d0e0a21fc168-kube-api-access-6n4d5\") pod \"perses-operator-5446b9c989-gqddj\" (UID: \"8b649975-2e0f-4406-9494-d0e0a21fc168\") " pod="openshift-operators/perses-operator-5446b9c989-gqddj" Dec 03 11:56:08 crc kubenswrapper[5052]: I1203 11:56:08.955178 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"observability-operator-tls\" (UniqueName: \"kubernetes.io/secret/0e9b7522-0b02-484a-b9e6-545e8dabcbc7-observability-operator-tls\") pod \"observability-operator-d8bb48f5d-swndl\" (UID: \"0e9b7522-0b02-484a-b9e6-545e8dabcbc7\") " pod="openshift-operators/observability-operator-d8bb48f5d-swndl" Dec 03 11:56:08 crc kubenswrapper[5052]: I1203 11:56:08.969306 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"observability-operator-tls\" (UniqueName: \"kubernetes.io/secret/0e9b7522-0b02-484a-b9e6-545e8dabcbc7-observability-operator-tls\") pod \"observability-operator-d8bb48f5d-swndl\" (UID: \"0e9b7522-0b02-484a-b9e6-545e8dabcbc7\") " pod="openshift-operators/observability-operator-d8bb48f5d-swndl" Dec 03 11:56:09 crc kubenswrapper[5052]: I1203 11:56:09.056463 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8b649975-2e0f-4406-9494-d0e0a21fc168-openshift-service-ca\") pod \"perses-operator-5446b9c989-gqddj\" (UID: \"8b649975-2e0f-4406-9494-d0e0a21fc168\") " pod="openshift-operators/perses-operator-5446b9c989-gqddj" Dec 03 11:56:09 crc kubenswrapper[5052]: I1203 11:56:09.056548 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-6n4d5\" (UniqueName: \"kubernetes.io/projected/8b649975-2e0f-4406-9494-d0e0a21fc168-kube-api-access-6n4d5\") pod \"perses-operator-5446b9c989-gqddj\" (UID: \"8b649975-2e0f-4406-9494-d0e0a21fc168\") " pod="openshift-operators/perses-operator-5446b9c989-gqddj" Dec 03 11:56:09 crc kubenswrapper[5052]: I1203 11:56:09.058136 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8b649975-2e0f-4406-9494-d0e0a21fc168-openshift-service-ca\") pod \"perses-operator-5446b9c989-gqddj\" (UID: \"8b649975-2e0f-4406-9494-d0e0a21fc168\") " pod="openshift-operators/perses-operator-5446b9c989-gqddj" Dec 03 11:56:09 crc kubenswrapper[5052]: I1203 11:56:09.238907 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-operators"/"obo-prometheus-operator-dockercfg-5jwdl" Dec 03 11:56:09 crc kubenswrapper[5052]: I1203 11:56:09.396870 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-operators/obo-prometheus-operator-admission-webhook-7cd69dc689-zh2bj"] Dec 03 11:56:09 crc kubenswrapper[5052]: I1203 11:56:09.420794 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-operators/obo-prometheus-operator-admission-webhook-7cd69dc689-cwkf4"] Dec 03 11:56:09 crc kubenswrapper[5052]: W1203 11:56:09.425206 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff11d0bc_dedc_433d_9e0e_a3fd2232919f.slice/crio-5588923a539eb5dba55dd1631be5328ea13e30b9c39eaca4dcc68fddd0cac756 WatchSource:0}: Error finding container 5588923a539eb5dba55dd1631be5328ea13e30b9c39eaca4dcc68fddd0cac756: Status 404 returned error can't find the container with id 5588923a539eb5dba55dd1631be5328ea13e30b9c39eaca4dcc68fddd0cac756 Dec 03 11:56:09 crc kubenswrapper[5052]: I1203 11:56:09.446146 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-operators"/"kube-root-ca.crt" Dec 03 11:56:09 crc kubenswrapper[5052]: I1203 11:56:09.454136 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-s9nbd\" (UniqueName: \"kubernetes.io/projected/c0580db1-72b0-4b6b-9136-d45a283a060c-kube-api-access-s9nbd\") pod \"obo-prometheus-operator-668cf9dfbb-mddsw\" (UID: \"c0580db1-72b0-4b6b-9136-d45a283a060c\") " pod="openshift-operators/obo-prometheus-operator-668cf9dfbb-mddsw" Dec 03 11:56:09 crc kubenswrapper[5052]: I1203 11:56:09.459130 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-6n4d5\" (UniqueName: \"kubernetes.io/projected/8b649975-2e0f-4406-9494-d0e0a21fc168-kube-api-access-6n4d5\") pod \"perses-operator-5446b9c989-gqddj\" (UID: \"8b649975-2e0f-4406-9494-d0e0a21fc168\") " pod="openshift-operators/perses-operator-5446b9c989-gqddj" Dec 03 11:56:09 crc kubenswrapper[5052]: I1203 11:56:09.460307 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-24tt9\" (UniqueName: \"kubernetes.io/projected/0e9b7522-0b02-484a-b9e6-545e8dabcbc7-kube-api-access-24tt9\") pod \"observability-operator-d8bb48f5d-swndl\" (UID: \"0e9b7522-0b02-484a-b9e6-545e8dabcbc7\") " pod="openshift-operators/observability-operator-d8bb48f5d-swndl" Dec 03 11:56:09 crc kubenswrapper[5052]: I1203 11:56:09.531953 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-operators/perses-operator-5446b9c989-gqddj" Dec 03 11:56:09 crc kubenswrapper[5052]: I1203 11:56:09.610978 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-operators/obo-prometheus-operator-668cf9dfbb-mddsw" Dec 03 11:56:09 crc kubenswrapper[5052]: I1203 11:56:09.698568 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-operators"/"observability-operator-sa-dockercfg-5vgbj" Dec 03 11:56:09 crc kubenswrapper[5052]: I1203 11:56:09.706822 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-operators/observability-operator-d8bb48f5d-swndl" Dec 03 11:56:09 crc kubenswrapper[5052]: I1203 11:56:09.721433 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-operators/obo-prometheus-operator-admission-webhook-7cd69dc689-zh2bj" event={"ID":"6a2264eb-3e7a-4987-be04-9131bb690ebf","Type":"ContainerStarted","Data":"bace0f2cc1593f449f2cee0688747b11e41b6885de1762d8824d81f283b4b68e"} Dec 03 11:56:09 crc kubenswrapper[5052]: I1203 11:56:09.721479 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-operators/obo-prometheus-operator-admission-webhook-7cd69dc689-cwkf4" event={"ID":"ff11d0bc-dedc-433d-9e0e-a3fd2232919f","Type":"ContainerStarted","Data":"5588923a539eb5dba55dd1631be5328ea13e30b9c39eaca4dcc68fddd0cac756"} Dec 03 11:56:10 crc kubenswrapper[5052]: I1203 11:56:10.042669 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-operators/perses-operator-5446b9c989-gqddj"] Dec 03 11:56:10 crc kubenswrapper[5052]: I1203 11:56:10.200136 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-operators/obo-prometheus-operator-668cf9dfbb-mddsw"] Dec 03 11:56:10 crc kubenswrapper[5052]: W1203 11:56:10.213962 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0580db1_72b0_4b6b_9136_d45a283a060c.slice/crio-828720fe827aae3a1ae7927f294ecd3df3ba56261177e7902a76e8b86a8ec9b1 WatchSource:0}: Error finding container 828720fe827aae3a1ae7927f294ecd3df3ba56261177e7902a76e8b86a8ec9b1: Status 404 returned error can't find the container with id 828720fe827aae3a1ae7927f294ecd3df3ba56261177e7902a76e8b86a8ec9b1 Dec 03 11:56:10 crc kubenswrapper[5052]: I1203 11:56:10.259758 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-operators/observability-operator-d8bb48f5d-swndl"] Dec 03 11:56:10 crc kubenswrapper[5052]: W1203 11:56:10.263304 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e9b7522_0b02_484a_b9e6_545e8dabcbc7.slice/crio-830038f5e86359616767bf89e71d4e5edd058381ed251615960ba4b9c339e540 WatchSource:0}: Error finding container 830038f5e86359616767bf89e71d4e5edd058381ed251615960ba4b9c339e540: Status 404 returned error can't find the container with id 830038f5e86359616767bf89e71d4e5edd058381ed251615960ba4b9c339e540 Dec 03 11:56:10 crc kubenswrapper[5052]: I1203 11:56:10.749018 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-operators/observability-operator-d8bb48f5d-swndl" event={"ID":"0e9b7522-0b02-484a-b9e6-545e8dabcbc7","Type":"ContainerStarted","Data":"830038f5e86359616767bf89e71d4e5edd058381ed251615960ba4b9c339e540"} Dec 03 11:56:10 crc kubenswrapper[5052]: I1203 11:56:10.751671 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-operators/perses-operator-5446b9c989-gqddj" event={"ID":"8b649975-2e0f-4406-9494-d0e0a21fc168","Type":"ContainerStarted","Data":"c2433afab7a0a9e1a5d15261368797e738df80ec3c79044f78d4ae9367b60a0d"} Dec 03 11:56:10 crc kubenswrapper[5052]: I1203 11:56:10.753002 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-operators/obo-prometheus-operator-668cf9dfbb-mddsw" event={"ID":"c0580db1-72b0-4b6b-9136-d45a283a060c","Type":"ContainerStarted","Data":"828720fe827aae3a1ae7927f294ecd3df3ba56261177e7902a76e8b86a8ec9b1"} Dec 03 11:56:12 crc kubenswrapper[5052]: I1203 11:56:12.301091 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/rabbitmq-server-0"] Dec 03 11:56:12 crc kubenswrapper[5052]: I1203 11:56:12.303221 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 11:56:12 crc kubenswrapper[5052]: I1203 11:56:12.305915 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"rabbitmq-server-dockercfg-d7vcr" Dec 03 11:56:12 crc kubenswrapper[5052]: I1203 11:56:12.306176 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"rabbitmq-erlang-cookie" Dec 03 11:56:12 crc kubenswrapper[5052]: I1203 11:56:12.309874 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-rabbitmq-svc" Dec 03 11:56:12 crc kubenswrapper[5052]: I1203 11:56:12.310085 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"rabbitmq-config-data" Dec 03 11:56:12 crc kubenswrapper[5052]: I1203 11:56:12.310232 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"rabbitmq-plugins-conf" Dec 03 11:56:12 crc kubenswrapper[5052]: I1203 11:56:12.310351 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"rabbitmq-default-user" Dec 03 11:56:12 crc kubenswrapper[5052]: I1203 11:56:12.310484 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"rabbitmq-server-conf" Dec 03 11:56:12 crc kubenswrapper[5052]: I1203 11:56:12.328273 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/rabbitmq-server-0"] Dec 03 11:56:12 crc kubenswrapper[5052]: I1203 11:56:12.437002 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"server-conf\" (UniqueName: \"kubernetes.io/configmap/30578d71-fe2b-4eed-9a07-6c8bfe4a1c43-server-conf\") pod \"rabbitmq-server-0\" (UID: \"30578d71-fe2b-4eed-9a07-6c8bfe4a1c43\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 11:56:12 crc kubenswrapper[5052]: I1203 11:56:12.437057 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"rabbitmq-erlang-cookie\" (UniqueName: \"kubernetes.io/empty-dir/30578d71-fe2b-4eed-9a07-6c8bfe4a1c43-rabbitmq-erlang-cookie\") pod \"rabbitmq-server-0\" (UID: \"30578d71-fe2b-4eed-9a07-6c8bfe4a1c43\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 11:56:12 crc kubenswrapper[5052]: I1203 11:56:12.437081 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"rabbitmq-tls\" (UniqueName: \"kubernetes.io/projected/30578d71-fe2b-4eed-9a07-6c8bfe4a1c43-rabbitmq-tls\") pod \"rabbitmq-server-0\" (UID: \"30578d71-fe2b-4eed-9a07-6c8bfe4a1c43\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 11:56:12 crc kubenswrapper[5052]: I1203 11:56:12.437129 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"pod-info\" (UniqueName: \"kubernetes.io/downward-api/30578d71-fe2b-4eed-9a07-6c8bfe4a1c43-pod-info\") pod \"rabbitmq-server-0\" (UID: \"30578d71-fe2b-4eed-9a07-6c8bfe4a1c43\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 11:56:12 crc kubenswrapper[5052]: I1203 11:56:12.437150 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nt9pl\" (UniqueName: \"kubernetes.io/projected/30578d71-fe2b-4eed-9a07-6c8bfe4a1c43-kube-api-access-nt9pl\") pod \"rabbitmq-server-0\" (UID: \"30578d71-fe2b-4eed-9a07-6c8bfe4a1c43\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 11:56:12 crc kubenswrapper[5052]: I1203 11:56:12.437169 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage18-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage18-crc\") pod \"rabbitmq-server-0\" (UID: \"30578d71-fe2b-4eed-9a07-6c8bfe4a1c43\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 11:56:12 crc kubenswrapper[5052]: I1203 11:56:12.437190 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"rabbitmq-confd\" (UniqueName: \"kubernetes.io/projected/30578d71-fe2b-4eed-9a07-6c8bfe4a1c43-rabbitmq-confd\") pod \"rabbitmq-server-0\" (UID: \"30578d71-fe2b-4eed-9a07-6c8bfe4a1c43\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 11:56:12 crc kubenswrapper[5052]: I1203 11:56:12.437208 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"rabbitmq-plugins\" (UniqueName: \"kubernetes.io/empty-dir/30578d71-fe2b-4eed-9a07-6c8bfe4a1c43-rabbitmq-plugins\") pod \"rabbitmq-server-0\" (UID: \"30578d71-fe2b-4eed-9a07-6c8bfe4a1c43\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 11:56:12 crc kubenswrapper[5052]: I1203 11:56:12.437263 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-conf\" (UniqueName: \"kubernetes.io/configmap/30578d71-fe2b-4eed-9a07-6c8bfe4a1c43-plugins-conf\") pod \"rabbitmq-server-0\" (UID: \"30578d71-fe2b-4eed-9a07-6c8bfe4a1c43\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 11:56:12 crc kubenswrapper[5052]: I1203 11:56:12.437282 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"erlang-cookie-secret\" (UniqueName: \"kubernetes.io/secret/30578d71-fe2b-4eed-9a07-6c8bfe4a1c43-erlang-cookie-secret\") pod \"rabbitmq-server-0\" (UID: \"30578d71-fe2b-4eed-9a07-6c8bfe4a1c43\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 11:56:12 crc kubenswrapper[5052]: I1203 11:56:12.437314 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/30578d71-fe2b-4eed-9a07-6c8bfe4a1c43-config-data\") pod \"rabbitmq-server-0\" (UID: \"30578d71-fe2b-4eed-9a07-6c8bfe4a1c43\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 11:56:12 crc kubenswrapper[5052]: I1203 11:56:12.539041 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage18-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage18-crc\") pod \"rabbitmq-server-0\" (UID: \"30578d71-fe2b-4eed-9a07-6c8bfe4a1c43\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 11:56:12 crc kubenswrapper[5052]: I1203 11:56:12.539097 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"rabbitmq-confd\" (UniqueName: \"kubernetes.io/projected/30578d71-fe2b-4eed-9a07-6c8bfe4a1c43-rabbitmq-confd\") pod \"rabbitmq-server-0\" (UID: \"30578d71-fe2b-4eed-9a07-6c8bfe4a1c43\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 11:56:12 crc kubenswrapper[5052]: I1203 11:56:12.539155 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"rabbitmq-plugins\" (UniqueName: \"kubernetes.io/empty-dir/30578d71-fe2b-4eed-9a07-6c8bfe4a1c43-rabbitmq-plugins\") pod \"rabbitmq-server-0\" (UID: \"30578d71-fe2b-4eed-9a07-6c8bfe4a1c43\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 11:56:12 crc kubenswrapper[5052]: I1203 11:56:12.539491 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage18-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage18-crc\") pod \"rabbitmq-server-0\" (UID: \"30578d71-fe2b-4eed-9a07-6c8bfe4a1c43\") device mount path \"/mnt/openstack/pv18\"" pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 11:56:12 crc kubenswrapper[5052]: I1203 11:56:12.539770 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"rabbitmq-plugins\" (UniqueName: \"kubernetes.io/empty-dir/30578d71-fe2b-4eed-9a07-6c8bfe4a1c43-rabbitmq-plugins\") pod \"rabbitmq-server-0\" (UID: \"30578d71-fe2b-4eed-9a07-6c8bfe4a1c43\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 11:56:12 crc kubenswrapper[5052]: I1203 11:56:12.540055 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"plugins-conf\" (UniqueName: \"kubernetes.io/configmap/30578d71-fe2b-4eed-9a07-6c8bfe4a1c43-plugins-conf\") pod \"rabbitmq-server-0\" (UID: \"30578d71-fe2b-4eed-9a07-6c8bfe4a1c43\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 11:56:12 crc kubenswrapper[5052]: I1203 11:56:12.540086 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"erlang-cookie-secret\" (UniqueName: \"kubernetes.io/secret/30578d71-fe2b-4eed-9a07-6c8bfe4a1c43-erlang-cookie-secret\") pod \"rabbitmq-server-0\" (UID: \"30578d71-fe2b-4eed-9a07-6c8bfe4a1c43\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 11:56:12 crc kubenswrapper[5052]: I1203 11:56:12.540131 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/30578d71-fe2b-4eed-9a07-6c8bfe4a1c43-config-data\") pod \"rabbitmq-server-0\" (UID: \"30578d71-fe2b-4eed-9a07-6c8bfe4a1c43\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 11:56:12 crc kubenswrapper[5052]: I1203 11:56:12.540203 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"server-conf\" (UniqueName: \"kubernetes.io/configmap/30578d71-fe2b-4eed-9a07-6c8bfe4a1c43-server-conf\") pod \"rabbitmq-server-0\" (UID: \"30578d71-fe2b-4eed-9a07-6c8bfe4a1c43\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 11:56:12 crc kubenswrapper[5052]: I1203 11:56:12.540231 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"rabbitmq-erlang-cookie\" (UniqueName: \"kubernetes.io/empty-dir/30578d71-fe2b-4eed-9a07-6c8bfe4a1c43-rabbitmq-erlang-cookie\") pod \"rabbitmq-server-0\" (UID: \"30578d71-fe2b-4eed-9a07-6c8bfe4a1c43\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 11:56:12 crc kubenswrapper[5052]: I1203 11:56:12.540271 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"rabbitmq-tls\" (UniqueName: \"kubernetes.io/projected/30578d71-fe2b-4eed-9a07-6c8bfe4a1c43-rabbitmq-tls\") pod \"rabbitmq-server-0\" (UID: \"30578d71-fe2b-4eed-9a07-6c8bfe4a1c43\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 11:56:12 crc kubenswrapper[5052]: I1203 11:56:12.540329 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"pod-info\" (UniqueName: \"kubernetes.io/downward-api/30578d71-fe2b-4eed-9a07-6c8bfe4a1c43-pod-info\") pod \"rabbitmq-server-0\" (UID: \"30578d71-fe2b-4eed-9a07-6c8bfe4a1c43\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 11:56:12 crc kubenswrapper[5052]: I1203 11:56:12.540360 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-nt9pl\" (UniqueName: \"kubernetes.io/projected/30578d71-fe2b-4eed-9a07-6c8bfe4a1c43-kube-api-access-nt9pl\") pod \"rabbitmq-server-0\" (UID: \"30578d71-fe2b-4eed-9a07-6c8bfe4a1c43\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 11:56:12 crc kubenswrapper[5052]: I1203 11:56:12.541058 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"plugins-conf\" (UniqueName: \"kubernetes.io/configmap/30578d71-fe2b-4eed-9a07-6c8bfe4a1c43-plugins-conf\") pod \"rabbitmq-server-0\" (UID: \"30578d71-fe2b-4eed-9a07-6c8bfe4a1c43\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 11:56:12 crc kubenswrapper[5052]: I1203 11:56:12.541411 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/30578d71-fe2b-4eed-9a07-6c8bfe4a1c43-config-data\") pod \"rabbitmq-server-0\" (UID: \"30578d71-fe2b-4eed-9a07-6c8bfe4a1c43\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 11:56:12 crc kubenswrapper[5052]: I1203 11:56:12.542153 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"rabbitmq-erlang-cookie\" (UniqueName: \"kubernetes.io/empty-dir/30578d71-fe2b-4eed-9a07-6c8bfe4a1c43-rabbitmq-erlang-cookie\") pod \"rabbitmq-server-0\" (UID: \"30578d71-fe2b-4eed-9a07-6c8bfe4a1c43\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 11:56:12 crc kubenswrapper[5052]: I1203 11:56:12.542849 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"server-conf\" (UniqueName: \"kubernetes.io/configmap/30578d71-fe2b-4eed-9a07-6c8bfe4a1c43-server-conf\") pod \"rabbitmq-server-0\" (UID: \"30578d71-fe2b-4eed-9a07-6c8bfe4a1c43\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 11:56:12 crc kubenswrapper[5052]: I1203 11:56:12.547003 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"pod-info\" (UniqueName: \"kubernetes.io/downward-api/30578d71-fe2b-4eed-9a07-6c8bfe4a1c43-pod-info\") pod \"rabbitmq-server-0\" (UID: \"30578d71-fe2b-4eed-9a07-6c8bfe4a1c43\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 11:56:12 crc kubenswrapper[5052]: I1203 11:56:12.549639 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"erlang-cookie-secret\" (UniqueName: \"kubernetes.io/secret/30578d71-fe2b-4eed-9a07-6c8bfe4a1c43-erlang-cookie-secret\") pod \"rabbitmq-server-0\" (UID: \"30578d71-fe2b-4eed-9a07-6c8bfe4a1c43\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 11:56:12 crc kubenswrapper[5052]: I1203 11:56:12.550794 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"rabbitmq-confd\" (UniqueName: \"kubernetes.io/projected/30578d71-fe2b-4eed-9a07-6c8bfe4a1c43-rabbitmq-confd\") pod \"rabbitmq-server-0\" (UID: \"30578d71-fe2b-4eed-9a07-6c8bfe4a1c43\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 11:56:12 crc kubenswrapper[5052]: I1203 11:56:12.556493 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"rabbitmq-tls\" (UniqueName: \"kubernetes.io/projected/30578d71-fe2b-4eed-9a07-6c8bfe4a1c43-rabbitmq-tls\") pod \"rabbitmq-server-0\" (UID: \"30578d71-fe2b-4eed-9a07-6c8bfe4a1c43\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 11:56:12 crc kubenswrapper[5052]: I1203 11:56:12.579661 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage18-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage18-crc\") pod \"rabbitmq-server-0\" (UID: \"30578d71-fe2b-4eed-9a07-6c8bfe4a1c43\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 11:56:12 crc kubenswrapper[5052]: I1203 11:56:12.587368 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-nt9pl\" (UniqueName: \"kubernetes.io/projected/30578d71-fe2b-4eed-9a07-6c8bfe4a1c43-kube-api-access-nt9pl\") pod \"rabbitmq-server-0\" (UID: \"30578d71-fe2b-4eed-9a07-6c8bfe4a1c43\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 11:56:12 crc kubenswrapper[5052]: I1203 11:56:12.632857 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 11:56:13 crc kubenswrapper[5052]: I1203 11:56:13.525079 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/openstack-galera-0"] Dec 03 11:56:13 crc kubenswrapper[5052]: I1203 11:56:13.527144 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 11:56:13 crc kubenswrapper[5052]: I1203 11:56:13.531655 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"openstack-config-data" Dec 03 11:56:13 crc kubenswrapper[5052]: I1203 11:56:13.531871 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"openstack-scripts" Dec 03 11:56:13 crc kubenswrapper[5052]: I1203 11:56:13.532084 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-galera-openstack-svc" Dec 03 11:56:13 crc kubenswrapper[5052]: I1203 11:56:13.539223 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"galera-openstack-dockercfg-bq5g4" Dec 03 11:56:13 crc kubenswrapper[5052]: I1203 11:56:13.541760 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"combined-ca-bundle" Dec 03 11:56:13 crc kubenswrapper[5052]: I1203 11:56:13.555354 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/openstack-galera-0"] Dec 03 11:56:13 crc kubenswrapper[5052]: I1203 11:56:13.664281 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/75c87049-f7b9-4c6b-add4-2eec1f52deb2-config-data-default\") pod \"openstack-galera-0\" (UID: \"75c87049-f7b9-4c6b-add4-2eec1f52deb2\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 11:56:13 crc kubenswrapper[5052]: I1203 11:56:13.664627 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n5brn\" (UniqueName: \"kubernetes.io/projected/75c87049-f7b9-4c6b-add4-2eec1f52deb2-kube-api-access-n5brn\") pod \"openstack-galera-0\" (UID: \"75c87049-f7b9-4c6b-add4-2eec1f52deb2\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 11:56:13 crc kubenswrapper[5052]: I1203 11:56:13.664646 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/75c87049-f7b9-4c6b-add4-2eec1f52deb2-kolla-config\") pod \"openstack-galera-0\" (UID: \"75c87049-f7b9-4c6b-add4-2eec1f52deb2\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 11:56:13 crc kubenswrapper[5052]: I1203 11:56:13.664699 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/75c87049-f7b9-4c6b-add4-2eec1f52deb2-galera-tls-certs\") pod \"openstack-galera-0\" (UID: \"75c87049-f7b9-4c6b-add4-2eec1f52deb2\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 11:56:13 crc kubenswrapper[5052]: I1203 11:56:13.664715 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/75c87049-f7b9-4c6b-add4-2eec1f52deb2-config-data-generated\") pod \"openstack-galera-0\" (UID: \"75c87049-f7b9-4c6b-add4-2eec1f52deb2\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 11:56:13 crc kubenswrapper[5052]: I1203 11:56:13.664748 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage14-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage14-crc\") pod \"openstack-galera-0\" (UID: \"75c87049-f7b9-4c6b-add4-2eec1f52deb2\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 11:56:13 crc kubenswrapper[5052]: I1203 11:56:13.664776 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/75c87049-f7b9-4c6b-add4-2eec1f52deb2-operator-scripts\") pod \"openstack-galera-0\" (UID: \"75c87049-f7b9-4c6b-add4-2eec1f52deb2\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 11:56:13 crc kubenswrapper[5052]: I1203 11:56:13.664804 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/75c87049-f7b9-4c6b-add4-2eec1f52deb2-combined-ca-bundle\") pod \"openstack-galera-0\" (UID: \"75c87049-f7b9-4c6b-add4-2eec1f52deb2\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 11:56:13 crc kubenswrapper[5052]: I1203 11:56:13.697580 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/rabbitmq-server-0"] Dec 03 11:56:13 crc kubenswrapper[5052]: I1203 11:56:13.767469 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/75c87049-f7b9-4c6b-add4-2eec1f52deb2-galera-tls-certs\") pod \"openstack-galera-0\" (UID: \"75c87049-f7b9-4c6b-add4-2eec1f52deb2\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 11:56:13 crc kubenswrapper[5052]: I1203 11:56:13.767506 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/75c87049-f7b9-4c6b-add4-2eec1f52deb2-config-data-generated\") pod \"openstack-galera-0\" (UID: \"75c87049-f7b9-4c6b-add4-2eec1f52deb2\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 11:56:13 crc kubenswrapper[5052]: I1203 11:56:13.767553 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage14-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage14-crc\") pod \"openstack-galera-0\" (UID: \"75c87049-f7b9-4c6b-add4-2eec1f52deb2\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 11:56:13 crc kubenswrapper[5052]: I1203 11:56:13.767582 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/75c87049-f7b9-4c6b-add4-2eec1f52deb2-operator-scripts\") pod \"openstack-galera-0\" (UID: \"75c87049-f7b9-4c6b-add4-2eec1f52deb2\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 11:56:13 crc kubenswrapper[5052]: I1203 11:56:13.767613 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/75c87049-f7b9-4c6b-add4-2eec1f52deb2-combined-ca-bundle\") pod \"openstack-galera-0\" (UID: \"75c87049-f7b9-4c6b-add4-2eec1f52deb2\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 11:56:13 crc kubenswrapper[5052]: I1203 11:56:13.767670 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/75c87049-f7b9-4c6b-add4-2eec1f52deb2-config-data-default\") pod \"openstack-galera-0\" (UID: \"75c87049-f7b9-4c6b-add4-2eec1f52deb2\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 11:56:13 crc kubenswrapper[5052]: I1203 11:56:13.767689 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-n5brn\" (UniqueName: \"kubernetes.io/projected/75c87049-f7b9-4c6b-add4-2eec1f52deb2-kube-api-access-n5brn\") pod \"openstack-galera-0\" (UID: \"75c87049-f7b9-4c6b-add4-2eec1f52deb2\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 11:56:13 crc kubenswrapper[5052]: I1203 11:56:13.767704 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/75c87049-f7b9-4c6b-add4-2eec1f52deb2-kolla-config\") pod \"openstack-galera-0\" (UID: \"75c87049-f7b9-4c6b-add4-2eec1f52deb2\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 11:56:13 crc kubenswrapper[5052]: I1203 11:56:13.767952 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage14-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage14-crc\") pod \"openstack-galera-0\" (UID: \"75c87049-f7b9-4c6b-add4-2eec1f52deb2\") device mount path \"/mnt/openstack/pv14\"" pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 11:56:13 crc kubenswrapper[5052]: I1203 11:56:13.767999 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/75c87049-f7b9-4c6b-add4-2eec1f52deb2-config-data-generated\") pod \"openstack-galera-0\" (UID: \"75c87049-f7b9-4c6b-add4-2eec1f52deb2\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 11:56:13 crc kubenswrapper[5052]: I1203 11:56:13.769213 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/75c87049-f7b9-4c6b-add4-2eec1f52deb2-config-data-default\") pod \"openstack-galera-0\" (UID: \"75c87049-f7b9-4c6b-add4-2eec1f52deb2\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 11:56:13 crc kubenswrapper[5052]: I1203 11:56:13.769880 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/75c87049-f7b9-4c6b-add4-2eec1f52deb2-kolla-config\") pod \"openstack-galera-0\" (UID: \"75c87049-f7b9-4c6b-add4-2eec1f52deb2\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 11:56:13 crc kubenswrapper[5052]: I1203 11:56:13.770986 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/75c87049-f7b9-4c6b-add4-2eec1f52deb2-operator-scripts\") pod \"openstack-galera-0\" (UID: \"75c87049-f7b9-4c6b-add4-2eec1f52deb2\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 11:56:13 crc kubenswrapper[5052]: I1203 11:56:13.774849 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/75c87049-f7b9-4c6b-add4-2eec1f52deb2-combined-ca-bundle\") pod \"openstack-galera-0\" (UID: \"75c87049-f7b9-4c6b-add4-2eec1f52deb2\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 11:56:13 crc kubenswrapper[5052]: I1203 11:56:13.776049 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/75c87049-f7b9-4c6b-add4-2eec1f52deb2-galera-tls-certs\") pod \"openstack-galera-0\" (UID: \"75c87049-f7b9-4c6b-add4-2eec1f52deb2\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 11:56:13 crc kubenswrapper[5052]: I1203 11:56:13.795935 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-n5brn\" (UniqueName: \"kubernetes.io/projected/75c87049-f7b9-4c6b-add4-2eec1f52deb2-kube-api-access-n5brn\") pod \"openstack-galera-0\" (UID: \"75c87049-f7b9-4c6b-add4-2eec1f52deb2\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 11:56:13 crc kubenswrapper[5052]: I1203 11:56:13.818767 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage14-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage14-crc\") pod \"openstack-galera-0\" (UID: \"75c87049-f7b9-4c6b-add4-2eec1f52deb2\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 11:56:13 crc kubenswrapper[5052]: I1203 11:56:13.853355 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 11:56:14 crc kubenswrapper[5052]: I1203 11:56:14.000292 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/memcached-0"] Dec 03 11:56:14 crc kubenswrapper[5052]: I1203 11:56:14.001557 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/memcached-0" Dec 03 11:56:14 crc kubenswrapper[5052]: I1203 11:56:14.017537 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"memcached-memcached-dockercfg-jflt2" Dec 03 11:56:14 crc kubenswrapper[5052]: I1203 11:56:14.018660 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"memcached-config-data" Dec 03 11:56:14 crc kubenswrapper[5052]: I1203 11:56:14.020956 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-memcached-svc" Dec 03 11:56:14 crc kubenswrapper[5052]: I1203 11:56:14.024549 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/memcached-0"] Dec 03 11:56:14 crc kubenswrapper[5052]: I1203 11:56:14.079409 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"memcached-tls-certs\" (UniqueName: \"kubernetes.io/secret/5a08cda2-5952-4740-a571-7058f56c3b58-memcached-tls-certs\") pod \"memcached-0\" (UID: \"5a08cda2-5952-4740-a571-7058f56c3b58\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 11:56:14 crc kubenswrapper[5052]: I1203 11:56:14.079485 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xrrfc\" (UniqueName: \"kubernetes.io/projected/5a08cda2-5952-4740-a571-7058f56c3b58-kube-api-access-xrrfc\") pod \"memcached-0\" (UID: \"5a08cda2-5952-4740-a571-7058f56c3b58\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 11:56:14 crc kubenswrapper[5052]: I1203 11:56:14.079536 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/5a08cda2-5952-4740-a571-7058f56c3b58-config-data\") pod \"memcached-0\" (UID: \"5a08cda2-5952-4740-a571-7058f56c3b58\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 11:56:14 crc kubenswrapper[5052]: I1203 11:56:14.079568 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5a08cda2-5952-4740-a571-7058f56c3b58-combined-ca-bundle\") pod \"memcached-0\" (UID: \"5a08cda2-5952-4740-a571-7058f56c3b58\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 11:56:14 crc kubenswrapper[5052]: I1203 11:56:14.079606 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/5a08cda2-5952-4740-a571-7058f56c3b58-kolla-config\") pod \"memcached-0\" (UID: \"5a08cda2-5952-4740-a571-7058f56c3b58\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 11:56:14 crc kubenswrapper[5052]: I1203 11:56:14.180923 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"memcached-tls-certs\" (UniqueName: \"kubernetes.io/secret/5a08cda2-5952-4740-a571-7058f56c3b58-memcached-tls-certs\") pod \"memcached-0\" (UID: \"5a08cda2-5952-4740-a571-7058f56c3b58\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 11:56:14 crc kubenswrapper[5052]: I1203 11:56:14.181271 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-xrrfc\" (UniqueName: \"kubernetes.io/projected/5a08cda2-5952-4740-a571-7058f56c3b58-kube-api-access-xrrfc\") pod \"memcached-0\" (UID: \"5a08cda2-5952-4740-a571-7058f56c3b58\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 11:56:14 crc kubenswrapper[5052]: I1203 11:56:14.181301 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/5a08cda2-5952-4740-a571-7058f56c3b58-config-data\") pod \"memcached-0\" (UID: \"5a08cda2-5952-4740-a571-7058f56c3b58\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 11:56:14 crc kubenswrapper[5052]: I1203 11:56:14.181333 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5a08cda2-5952-4740-a571-7058f56c3b58-combined-ca-bundle\") pod \"memcached-0\" (UID: \"5a08cda2-5952-4740-a571-7058f56c3b58\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 11:56:14 crc kubenswrapper[5052]: I1203 11:56:14.181365 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/5a08cda2-5952-4740-a571-7058f56c3b58-kolla-config\") pod \"memcached-0\" (UID: \"5a08cda2-5952-4740-a571-7058f56c3b58\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 11:56:14 crc kubenswrapper[5052]: I1203 11:56:14.182269 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/5a08cda2-5952-4740-a571-7058f56c3b58-kolla-config\") pod \"memcached-0\" (UID: \"5a08cda2-5952-4740-a571-7058f56c3b58\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 11:56:14 crc kubenswrapper[5052]: I1203 11:56:14.182377 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/5a08cda2-5952-4740-a571-7058f56c3b58-config-data\") pod \"memcached-0\" (UID: \"5a08cda2-5952-4740-a571-7058f56c3b58\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 11:56:14 crc kubenswrapper[5052]: I1203 11:56:14.188382 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5a08cda2-5952-4740-a571-7058f56c3b58-combined-ca-bundle\") pod \"memcached-0\" (UID: \"5a08cda2-5952-4740-a571-7058f56c3b58\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 11:56:14 crc kubenswrapper[5052]: I1203 11:56:14.193169 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"memcached-tls-certs\" (UniqueName: \"kubernetes.io/secret/5a08cda2-5952-4740-a571-7058f56c3b58-memcached-tls-certs\") pod \"memcached-0\" (UID: \"5a08cda2-5952-4740-a571-7058f56c3b58\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 11:56:14 crc kubenswrapper[5052]: I1203 11:56:14.209866 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-xrrfc\" (UniqueName: \"kubernetes.io/projected/5a08cda2-5952-4740-a571-7058f56c3b58-kube-api-access-xrrfc\") pod \"memcached-0\" (UID: \"5a08cda2-5952-4740-a571-7058f56c3b58\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 11:56:14 crc kubenswrapper[5052]: I1203 11:56:14.340439 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/memcached-0" Dec 03 11:56:15 crc kubenswrapper[5052]: W1203 11:56:15.107567 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30578d71_fe2b_4eed_9a07_6c8bfe4a1c43.slice/crio-5eb284d08c25ae2aebd301c1a598b75bd5453227419b8a30ec5ec374f2e772c9 WatchSource:0}: Error finding container 5eb284d08c25ae2aebd301c1a598b75bd5453227419b8a30ec5ec374f2e772c9: Status 404 returned error can't find the container with id 5eb284d08c25ae2aebd301c1a598b75bd5453227419b8a30ec5ec374f2e772c9 Dec 03 11:56:15 crc kubenswrapper[5052]: I1203 11:56:15.813699 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-server-0" event={"ID":"30578d71-fe2b-4eed-9a07-6c8bfe4a1c43","Type":"ContainerStarted","Data":"5eb284d08c25ae2aebd301c1a598b75bd5453227419b8a30ec5ec374f2e772c9"} Dec 03 11:56:16 crc kubenswrapper[5052]: I1203 11:56:16.322566 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/kube-state-metrics-0"] Dec 03 11:56:16 crc kubenswrapper[5052]: I1203 11:56:16.323771 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 11:56:16 crc kubenswrapper[5052]: I1203 11:56:16.347495 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"telemetry-ceilometer-dockercfg-wzk88" Dec 03 11:56:16 crc kubenswrapper[5052]: I1203 11:56:16.354734 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/kube-state-metrics-0"] Dec 03 11:56:16 crc kubenswrapper[5052]: I1203 11:56:16.418562 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-grsf7\" (UniqueName: \"kubernetes.io/projected/316d245b-16aa-44a0-8aaa-4a41794e1291-kube-api-access-grsf7\") pod \"kube-state-metrics-0\" (UID: \"316d245b-16aa-44a0-8aaa-4a41794e1291\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 11:56:16 crc kubenswrapper[5052]: I1203 11:56:16.520154 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-grsf7\" (UniqueName: \"kubernetes.io/projected/316d245b-16aa-44a0-8aaa-4a41794e1291-kube-api-access-grsf7\") pod \"kube-state-metrics-0\" (UID: \"316d245b-16aa-44a0-8aaa-4a41794e1291\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 11:56:16 crc kubenswrapper[5052]: I1203 11:56:16.564952 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-grsf7\" (UniqueName: \"kubernetes.io/projected/316d245b-16aa-44a0-8aaa-4a41794e1291-kube-api-access-grsf7\") pod \"kube-state-metrics-0\" (UID: \"316d245b-16aa-44a0-8aaa-4a41794e1291\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 11:56:16 crc kubenswrapper[5052]: I1203 11:56:16.644957 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 11:56:18 crc kubenswrapper[5052]: I1203 11:56:18.060223 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/openstack-galera-0"] Dec 03 11:56:18 crc kubenswrapper[5052]: I1203 11:56:18.179938 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/memcached-0"] Dec 03 11:56:18 crc kubenswrapper[5052]: I1203 11:56:18.201922 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/kube-state-metrics-0"] Dec 03 11:56:18 crc kubenswrapper[5052]: I1203 11:56:18.872913 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-operators/perses-operator-5446b9c989-gqddj" event={"ID":"8b649975-2e0f-4406-9494-d0e0a21fc168","Type":"ContainerStarted","Data":"52cdb9b684b00eb1d503623c4691b6e2fd177690224eeb5c70f4e0b79c8446ba"} Dec 03 11:56:18 crc kubenswrapper[5052]: I1203 11:56:18.873359 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openshift-operators/perses-operator-5446b9c989-gqddj" Dec 03 11:56:18 crc kubenswrapper[5052]: I1203 11:56:18.875072 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/kube-state-metrics-0" event={"ID":"316d245b-16aa-44a0-8aaa-4a41794e1291","Type":"ContainerStarted","Data":"89649df4e0c91b3003a7c8e4c4d98473c7fd7488089f8dd1c03e7da5a297c96f"} Dec 03 11:56:18 crc kubenswrapper[5052]: I1203 11:56:18.877299 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-operators/obo-prometheus-operator-admission-webhook-7cd69dc689-zh2bj" event={"ID":"6a2264eb-3e7a-4987-be04-9131bb690ebf","Type":"ContainerStarted","Data":"f37fd5cbfad65ae67b491dbff08f38adbb3414a2309a690dfeed61144fe5f4a3"} Dec 03 11:56:18 crc kubenswrapper[5052]: I1203 11:56:18.879891 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-operators/obo-prometheus-operator-admission-webhook-7cd69dc689-cwkf4" event={"ID":"ff11d0bc-dedc-433d-9e0e-a3fd2232919f","Type":"ContainerStarted","Data":"53fc8f7ef5e51d2afbefbee44f2be37f8b543dd0cca2e72c93a8ef3319e57672"} Dec 03 11:56:18 crc kubenswrapper[5052]: I1203 11:56:18.882608 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-galera-0" event={"ID":"75c87049-f7b9-4c6b-add4-2eec1f52deb2","Type":"ContainerStarted","Data":"747f297461b54875e189e5f421860b725e3b1e8ee94217f21a72b29f666ac5e9"} Dec 03 11:56:18 crc kubenswrapper[5052]: I1203 11:56:18.882644 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-galera-0" event={"ID":"75c87049-f7b9-4c6b-add4-2eec1f52deb2","Type":"ContainerStarted","Data":"96e331e95f8752e753d096f5fc8e1b953bcea07c1a9b3abb3dfeb833853b6cf3"} Dec 03 11:56:18 crc kubenswrapper[5052]: I1203 11:56:18.884917 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-operators/obo-prometheus-operator-668cf9dfbb-mddsw" event={"ID":"c0580db1-72b0-4b6b-9136-d45a283a060c","Type":"ContainerStarted","Data":"6cb9bb7bb11078ea762482d62413240d8dba53f44ea9f5a30e1d598a20f700ff"} Dec 03 11:56:18 crc kubenswrapper[5052]: I1203 11:56:18.890413 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/memcached-0" event={"ID":"5a08cda2-5952-4740-a571-7058f56c3b58","Type":"ContainerStarted","Data":"57e4d8c014f57177dc5f916168de2b019cf9fcd67c0d39bcaf0f7357d2dbf4bc"} Dec 03 11:56:18 crc kubenswrapper[5052]: I1203 11:56:18.890467 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/memcached-0" event={"ID":"5a08cda2-5952-4740-a571-7058f56c3b58","Type":"ContainerStarted","Data":"c72d5775a273d915d2fc20ae603ad1eabc0cee48e77e8afb58283ffb6c131c59"} Dec 03 11:56:18 crc kubenswrapper[5052]: I1203 11:56:18.890639 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-operators/perses-operator-5446b9c989-gqddj" podStartSLOduration=3.473868072 podStartE2EDuration="10.890627399s" podCreationTimestamp="2025-12-03 11:56:08 +0000 UTC" firstStartedPulling="2025-12-03 11:56:10.055674866 +0000 UTC m=+2160.643916630" lastFinishedPulling="2025-12-03 11:56:17.472434183 +0000 UTC m=+2168.060675957" observedRunningTime="2025-12-03 11:56:18.890168726 +0000 UTC m=+2169.478410510" watchObservedRunningTime="2025-12-03 11:56:18.890627399 +0000 UTC m=+2169.478869163" Dec 03 11:56:18 crc kubenswrapper[5052]: I1203 11:56:18.891269 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/memcached-0" Dec 03 11:56:18 crc kubenswrapper[5052]: I1203 11:56:18.910617 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-operators/obo-prometheus-operator-admission-webhook-7cd69dc689-zh2bj" podStartSLOduration=2.892039821 podStartE2EDuration="10.910601781s" podCreationTimestamp="2025-12-03 11:56:08 +0000 UTC" firstStartedPulling="2025-12-03 11:56:09.40833289 +0000 UTC m=+2159.996574654" lastFinishedPulling="2025-12-03 11:56:17.42689485 +0000 UTC m=+2168.015136614" observedRunningTime="2025-12-03 11:56:18.908213262 +0000 UTC m=+2169.496455026" watchObservedRunningTime="2025-12-03 11:56:18.910601781 +0000 UTC m=+2169.498843545" Dec 03 11:56:18 crc kubenswrapper[5052]: I1203 11:56:18.931916 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-operators/obo-prometheus-operator-668cf9dfbb-mddsw" podStartSLOduration=3.641049475 podStartE2EDuration="10.93189208s" podCreationTimestamp="2025-12-03 11:56:08 +0000 UTC" firstStartedPulling="2025-12-03 11:56:10.216968791 +0000 UTC m=+2160.805210555" lastFinishedPulling="2025-12-03 11:56:17.507811385 +0000 UTC m=+2168.096053160" observedRunningTime="2025-12-03 11:56:18.925440985 +0000 UTC m=+2169.513682749" watchObservedRunningTime="2025-12-03 11:56:18.93189208 +0000 UTC m=+2169.520133844" Dec 03 11:56:18 crc kubenswrapper[5052]: I1203 11:56:18.984348 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-operators/obo-prometheus-operator-admission-webhook-7cd69dc689-cwkf4" podStartSLOduration=3.094092804 podStartE2EDuration="10.984330541s" podCreationTimestamp="2025-12-03 11:56:08 +0000 UTC" firstStartedPulling="2025-12-03 11:56:09.427872659 +0000 UTC m=+2160.016114423" lastFinishedPulling="2025-12-03 11:56:17.318110396 +0000 UTC m=+2167.906352160" observedRunningTime="2025-12-03 11:56:18.978309218 +0000 UTC m=+2169.566550992" watchObservedRunningTime="2025-12-03 11:56:18.984330541 +0000 UTC m=+2169.572572305" Dec 03 11:56:19 crc kubenswrapper[5052]: I1203 11:56:19.002429 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/memcached-0" podStartSLOduration=6.002404358 podStartE2EDuration="6.002404358s" podCreationTimestamp="2025-12-03 11:56:13 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:56:18.999907177 +0000 UTC m=+2169.588148941" watchObservedRunningTime="2025-12-03 11:56:19.002404358 +0000 UTC m=+2169.590646122" Dec 03 11:56:19 crc kubenswrapper[5052]: I1203 11:56:19.080408 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openshift-marketplace/community-operators-tb5qk"] Dec 03 11:56:19 crc kubenswrapper[5052]: I1203 11:56:19.082789 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/community-operators-tb5qk" Dec 03 11:56:19 crc kubenswrapper[5052]: I1203 11:56:19.100383 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/community-operators-tb5qk"] Dec 03 11:56:19 crc kubenswrapper[5052]: I1203 11:56:19.174398 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t5rrr\" (UniqueName: \"kubernetes.io/projected/478d0cb6-25de-46d5-979e-9e43bf1b7d4b-kube-api-access-t5rrr\") pod \"community-operators-tb5qk\" (UID: \"478d0cb6-25de-46d5-979e-9e43bf1b7d4b\") " pod="openshift-marketplace/community-operators-tb5qk" Dec 03 11:56:19 crc kubenswrapper[5052]: I1203 11:56:19.174821 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/478d0cb6-25de-46d5-979e-9e43bf1b7d4b-catalog-content\") pod \"community-operators-tb5qk\" (UID: \"478d0cb6-25de-46d5-979e-9e43bf1b7d4b\") " pod="openshift-marketplace/community-operators-tb5qk" Dec 03 11:56:19 crc kubenswrapper[5052]: I1203 11:56:19.174939 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/478d0cb6-25de-46d5-979e-9e43bf1b7d4b-utilities\") pod \"community-operators-tb5qk\" (UID: \"478d0cb6-25de-46d5-979e-9e43bf1b7d4b\") " pod="openshift-marketplace/community-operators-tb5qk" Dec 03 11:56:19 crc kubenswrapper[5052]: I1203 11:56:19.276781 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/478d0cb6-25de-46d5-979e-9e43bf1b7d4b-utilities\") pod \"community-operators-tb5qk\" (UID: \"478d0cb6-25de-46d5-979e-9e43bf1b7d4b\") " pod="openshift-marketplace/community-operators-tb5qk" Dec 03 11:56:19 crc kubenswrapper[5052]: I1203 11:56:19.276861 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-t5rrr\" (UniqueName: \"kubernetes.io/projected/478d0cb6-25de-46d5-979e-9e43bf1b7d4b-kube-api-access-t5rrr\") pod \"community-operators-tb5qk\" (UID: \"478d0cb6-25de-46d5-979e-9e43bf1b7d4b\") " pod="openshift-marketplace/community-operators-tb5qk" Dec 03 11:56:19 crc kubenswrapper[5052]: I1203 11:56:19.276909 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/478d0cb6-25de-46d5-979e-9e43bf1b7d4b-catalog-content\") pod \"community-operators-tb5qk\" (UID: \"478d0cb6-25de-46d5-979e-9e43bf1b7d4b\") " pod="openshift-marketplace/community-operators-tb5qk" Dec 03 11:56:19 crc kubenswrapper[5052]: I1203 11:56:19.277439 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/478d0cb6-25de-46d5-979e-9e43bf1b7d4b-utilities\") pod \"community-operators-tb5qk\" (UID: \"478d0cb6-25de-46d5-979e-9e43bf1b7d4b\") " pod="openshift-marketplace/community-operators-tb5qk" Dec 03 11:56:19 crc kubenswrapper[5052]: I1203 11:56:19.277619 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/478d0cb6-25de-46d5-979e-9e43bf1b7d4b-catalog-content\") pod \"community-operators-tb5qk\" (UID: \"478d0cb6-25de-46d5-979e-9e43bf1b7d4b\") " pod="openshift-marketplace/community-operators-tb5qk" Dec 03 11:56:19 crc kubenswrapper[5052]: I1203 11:56:19.299531 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-t5rrr\" (UniqueName: \"kubernetes.io/projected/478d0cb6-25de-46d5-979e-9e43bf1b7d4b-kube-api-access-t5rrr\") pod \"community-operators-tb5qk\" (UID: \"478d0cb6-25de-46d5-979e-9e43bf1b7d4b\") " pod="openshift-marketplace/community-operators-tb5qk" Dec 03 11:56:19 crc kubenswrapper[5052]: I1203 11:56:19.407318 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/community-operators-tb5qk" Dec 03 11:56:19 crc kubenswrapper[5052]: I1203 11:56:19.914424 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-server-0" event={"ID":"30578d71-fe2b-4eed-9a07-6c8bfe4a1c43","Type":"ContainerStarted","Data":"250de81141f60765c155818d8a4cfbffd80ff7babe5acee647422de56500c58c"} Dec 03 11:56:22 crc kubenswrapper[5052]: I1203 11:56:22.241834 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ovsdbserver-sb-0"] Dec 03 11:56:22 crc kubenswrapper[5052]: I1203 11:56:22.246843 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 11:56:22 crc kubenswrapper[5052]: I1203 11:56:22.250292 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"ovndbcluster-sb-config" Dec 03 11:56:22 crc kubenswrapper[5052]: I1203 11:56:22.250526 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ovncluster-ovndbcluster-sb-dockercfg-wbxkw" Dec 03 11:56:22 crc kubenswrapper[5052]: I1203 11:56:22.250664 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-ovndbcluster-sb-ovndbs" Dec 03 11:56:22 crc kubenswrapper[5052]: I1203 11:56:22.250784 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-ovn-metrics" Dec 03 11:56:22 crc kubenswrapper[5052]: I1203 11:56:22.251399 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"ovndbcluster-sb-scripts" Dec 03 11:56:22 crc kubenswrapper[5052]: I1203 11:56:22.257880 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-sb-0"] Dec 03 11:56:22 crc kubenswrapper[5052]: I1203 11:56:22.343012 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3cb4559f-542a-4076-82fb-c0ca88f9de94-combined-ca-bundle\") pod \"ovsdbserver-sb-0\" (UID: \"3cb4559f-542a-4076-82fb-c0ca88f9de94\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 11:56:22 crc kubenswrapper[5052]: I1203 11:56:22.343077 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovsdbserver-sb-tls-certs\" (UniqueName: \"kubernetes.io/secret/3cb4559f-542a-4076-82fb-c0ca88f9de94-ovsdbserver-sb-tls-certs\") pod \"ovsdbserver-sb-0\" (UID: \"3cb4559f-542a-4076-82fb-c0ca88f9de94\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 11:56:22 crc kubenswrapper[5052]: I1203 11:56:22.343133 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3cb4559f-542a-4076-82fb-c0ca88f9de94-config\") pod \"ovsdbserver-sb-0\" (UID: \"3cb4559f-542a-4076-82fb-c0ca88f9de94\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 11:56:22 crc kubenswrapper[5052]: I1203 11:56:22.343174 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage10-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage10-crc\") pod \"ovsdbserver-sb-0\" (UID: \"3cb4559f-542a-4076-82fb-c0ca88f9de94\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 11:56:22 crc kubenswrapper[5052]: I1203 11:56:22.343208 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/3cb4559f-542a-4076-82fb-c0ca88f9de94-scripts\") pod \"ovsdbserver-sb-0\" (UID: \"3cb4559f-542a-4076-82fb-c0ca88f9de94\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 11:56:22 crc kubenswrapper[5052]: I1203 11:56:22.343246 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/3cb4559f-542a-4076-82fb-c0ca88f9de94-metrics-certs-tls-certs\") pod \"ovsdbserver-sb-0\" (UID: \"3cb4559f-542a-4076-82fb-c0ca88f9de94\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 11:56:22 crc kubenswrapper[5052]: I1203 11:56:22.343265 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/3cb4559f-542a-4076-82fb-c0ca88f9de94-ovsdb-rundir\") pod \"ovsdbserver-sb-0\" (UID: \"3cb4559f-542a-4076-82fb-c0ca88f9de94\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 11:56:22 crc kubenswrapper[5052]: I1203 11:56:22.343299 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4vfjg\" (UniqueName: \"kubernetes.io/projected/3cb4559f-542a-4076-82fb-c0ca88f9de94-kube-api-access-4vfjg\") pod \"ovsdbserver-sb-0\" (UID: \"3cb4559f-542a-4076-82fb-c0ca88f9de94\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 11:56:22 crc kubenswrapper[5052]: I1203 11:56:22.447479 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/3cb4559f-542a-4076-82fb-c0ca88f9de94-scripts\") pod \"ovsdbserver-sb-0\" (UID: \"3cb4559f-542a-4076-82fb-c0ca88f9de94\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 11:56:22 crc kubenswrapper[5052]: I1203 11:56:22.447670 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/3cb4559f-542a-4076-82fb-c0ca88f9de94-metrics-certs-tls-certs\") pod \"ovsdbserver-sb-0\" (UID: \"3cb4559f-542a-4076-82fb-c0ca88f9de94\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 11:56:22 crc kubenswrapper[5052]: I1203 11:56:22.447703 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/3cb4559f-542a-4076-82fb-c0ca88f9de94-ovsdb-rundir\") pod \"ovsdbserver-sb-0\" (UID: \"3cb4559f-542a-4076-82fb-c0ca88f9de94\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 11:56:22 crc kubenswrapper[5052]: I1203 11:56:22.447732 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-4vfjg\" (UniqueName: \"kubernetes.io/projected/3cb4559f-542a-4076-82fb-c0ca88f9de94-kube-api-access-4vfjg\") pod \"ovsdbserver-sb-0\" (UID: \"3cb4559f-542a-4076-82fb-c0ca88f9de94\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 11:56:22 crc kubenswrapper[5052]: I1203 11:56:22.447766 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3cb4559f-542a-4076-82fb-c0ca88f9de94-combined-ca-bundle\") pod \"ovsdbserver-sb-0\" (UID: \"3cb4559f-542a-4076-82fb-c0ca88f9de94\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 11:56:22 crc kubenswrapper[5052]: I1203 11:56:22.447809 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovsdbserver-sb-tls-certs\" (UniqueName: \"kubernetes.io/secret/3cb4559f-542a-4076-82fb-c0ca88f9de94-ovsdbserver-sb-tls-certs\") pod \"ovsdbserver-sb-0\" (UID: \"3cb4559f-542a-4076-82fb-c0ca88f9de94\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 11:56:22 crc kubenswrapper[5052]: I1203 11:56:22.447858 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3cb4559f-542a-4076-82fb-c0ca88f9de94-config\") pod \"ovsdbserver-sb-0\" (UID: \"3cb4559f-542a-4076-82fb-c0ca88f9de94\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 11:56:22 crc kubenswrapper[5052]: I1203 11:56:22.447892 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage10-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage10-crc\") pod \"ovsdbserver-sb-0\" (UID: \"3cb4559f-542a-4076-82fb-c0ca88f9de94\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 11:56:22 crc kubenswrapper[5052]: I1203 11:56:22.448451 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage10-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage10-crc\") pod \"ovsdbserver-sb-0\" (UID: \"3cb4559f-542a-4076-82fb-c0ca88f9de94\") device mount path \"/mnt/openstack/pv10\"" pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 11:56:22 crc kubenswrapper[5052]: I1203 11:56:22.449945 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/3cb4559f-542a-4076-82fb-c0ca88f9de94-scripts\") pod \"ovsdbserver-sb-0\" (UID: \"3cb4559f-542a-4076-82fb-c0ca88f9de94\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 11:56:22 crc kubenswrapper[5052]: I1203 11:56:22.450985 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/3cb4559f-542a-4076-82fb-c0ca88f9de94-ovsdb-rundir\") pod \"ovsdbserver-sb-0\" (UID: \"3cb4559f-542a-4076-82fb-c0ca88f9de94\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 11:56:22 crc kubenswrapper[5052]: I1203 11:56:22.451142 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ovsdbserver-nb-0"] Dec 03 11:56:22 crc kubenswrapper[5052]: I1203 11:56:22.451429 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3cb4559f-542a-4076-82fb-c0ca88f9de94-config\") pod \"ovsdbserver-sb-0\" (UID: \"3cb4559f-542a-4076-82fb-c0ca88f9de94\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 11:56:22 crc kubenswrapper[5052]: I1203 11:56:22.456961 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 11:56:22 crc kubenswrapper[5052]: I1203 11:56:22.458385 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3cb4559f-542a-4076-82fb-c0ca88f9de94-combined-ca-bundle\") pod \"ovsdbserver-sb-0\" (UID: \"3cb4559f-542a-4076-82fb-c0ca88f9de94\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 11:56:22 crc kubenswrapper[5052]: I1203 11:56:22.458698 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ovncluster-ovndbcluster-nb-dockercfg-4gp6v" Dec 03 11:56:22 crc kubenswrapper[5052]: I1203 11:56:22.459072 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"ovndbcluster-nb-scripts" Dec 03 11:56:22 crc kubenswrapper[5052]: I1203 11:56:22.459206 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"ovndbcluster-nb-config" Dec 03 11:56:22 crc kubenswrapper[5052]: I1203 11:56:22.459268 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/3cb4559f-542a-4076-82fb-c0ca88f9de94-metrics-certs-tls-certs\") pod \"ovsdbserver-sb-0\" (UID: \"3cb4559f-542a-4076-82fb-c0ca88f9de94\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 11:56:22 crc kubenswrapper[5052]: I1203 11:56:22.459553 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-ovndbcluster-nb-ovndbs" Dec 03 11:56:22 crc kubenswrapper[5052]: I1203 11:56:22.469204 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-nb-0"] Dec 03 11:56:22 crc kubenswrapper[5052]: I1203 11:56:22.469940 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovsdbserver-sb-tls-certs\" (UniqueName: \"kubernetes.io/secret/3cb4559f-542a-4076-82fb-c0ca88f9de94-ovsdbserver-sb-tls-certs\") pod \"ovsdbserver-sb-0\" (UID: \"3cb4559f-542a-4076-82fb-c0ca88f9de94\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 11:56:22 crc kubenswrapper[5052]: I1203 11:56:22.472781 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-4vfjg\" (UniqueName: \"kubernetes.io/projected/3cb4559f-542a-4076-82fb-c0ca88f9de94-kube-api-access-4vfjg\") pod \"ovsdbserver-sb-0\" (UID: \"3cb4559f-542a-4076-82fb-c0ca88f9de94\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 11:56:22 crc kubenswrapper[5052]: I1203 11:56:22.498022 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage10-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage10-crc\") pod \"ovsdbserver-sb-0\" (UID: \"3cb4559f-542a-4076-82fb-c0ca88f9de94\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 11:56:22 crc kubenswrapper[5052]: I1203 11:56:22.550586 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/e8de8adf-e4e5-4a10-9722-9106357e67d0-metrics-certs-tls-certs\") pod \"ovsdbserver-nb-0\" (UID: \"e8de8adf-e4e5-4a10-9722-9106357e67d0\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 11:56:22 crc kubenswrapper[5052]: I1203 11:56:22.550638 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovsdbserver-nb-tls-certs\" (UniqueName: \"kubernetes.io/secret/e8de8adf-e4e5-4a10-9722-9106357e67d0-ovsdbserver-nb-tls-certs\") pod \"ovsdbserver-nb-0\" (UID: \"e8de8adf-e4e5-4a10-9722-9106357e67d0\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 11:56:22 crc kubenswrapper[5052]: I1203 11:56:22.551062 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage01-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage01-crc\") pod \"ovsdbserver-nb-0\" (UID: \"e8de8adf-e4e5-4a10-9722-9106357e67d0\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 11:56:22 crc kubenswrapper[5052]: I1203 11:56:22.551125 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6x8wn\" (UniqueName: \"kubernetes.io/projected/e8de8adf-e4e5-4a10-9722-9106357e67d0-kube-api-access-6x8wn\") pod \"ovsdbserver-nb-0\" (UID: \"e8de8adf-e4e5-4a10-9722-9106357e67d0\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 11:56:22 crc kubenswrapper[5052]: I1203 11:56:22.551250 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/e8de8adf-e4e5-4a10-9722-9106357e67d0-scripts\") pod \"ovsdbserver-nb-0\" (UID: \"e8de8adf-e4e5-4a10-9722-9106357e67d0\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 11:56:22 crc kubenswrapper[5052]: I1203 11:56:22.551945 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/e8de8adf-e4e5-4a10-9722-9106357e67d0-ovsdb-rundir\") pod \"ovsdbserver-nb-0\" (UID: \"e8de8adf-e4e5-4a10-9722-9106357e67d0\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 11:56:22 crc kubenswrapper[5052]: I1203 11:56:22.551997 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e8de8adf-e4e5-4a10-9722-9106357e67d0-config\") pod \"ovsdbserver-nb-0\" (UID: \"e8de8adf-e4e5-4a10-9722-9106357e67d0\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 11:56:22 crc kubenswrapper[5052]: I1203 11:56:22.552323 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e8de8adf-e4e5-4a10-9722-9106357e67d0-combined-ca-bundle\") pod \"ovsdbserver-nb-0\" (UID: \"e8de8adf-e4e5-4a10-9722-9106357e67d0\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 11:56:22 crc kubenswrapper[5052]: I1203 11:56:22.654352 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e8de8adf-e4e5-4a10-9722-9106357e67d0-config\") pod \"ovsdbserver-nb-0\" (UID: \"e8de8adf-e4e5-4a10-9722-9106357e67d0\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 11:56:22 crc kubenswrapper[5052]: I1203 11:56:22.654474 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e8de8adf-e4e5-4a10-9722-9106357e67d0-combined-ca-bundle\") pod \"ovsdbserver-nb-0\" (UID: \"e8de8adf-e4e5-4a10-9722-9106357e67d0\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 11:56:22 crc kubenswrapper[5052]: I1203 11:56:22.654553 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/e8de8adf-e4e5-4a10-9722-9106357e67d0-metrics-certs-tls-certs\") pod \"ovsdbserver-nb-0\" (UID: \"e8de8adf-e4e5-4a10-9722-9106357e67d0\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 11:56:22 crc kubenswrapper[5052]: I1203 11:56:22.654584 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovsdbserver-nb-tls-certs\" (UniqueName: \"kubernetes.io/secret/e8de8adf-e4e5-4a10-9722-9106357e67d0-ovsdbserver-nb-tls-certs\") pod \"ovsdbserver-nb-0\" (UID: \"e8de8adf-e4e5-4a10-9722-9106357e67d0\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 11:56:22 crc kubenswrapper[5052]: I1203 11:56:22.654625 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage01-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage01-crc\") pod \"ovsdbserver-nb-0\" (UID: \"e8de8adf-e4e5-4a10-9722-9106357e67d0\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 11:56:22 crc kubenswrapper[5052]: I1203 11:56:22.654663 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-6x8wn\" (UniqueName: \"kubernetes.io/projected/e8de8adf-e4e5-4a10-9722-9106357e67d0-kube-api-access-6x8wn\") pod \"ovsdbserver-nb-0\" (UID: \"e8de8adf-e4e5-4a10-9722-9106357e67d0\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 11:56:22 crc kubenswrapper[5052]: I1203 11:56:22.654711 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/e8de8adf-e4e5-4a10-9722-9106357e67d0-scripts\") pod \"ovsdbserver-nb-0\" (UID: \"e8de8adf-e4e5-4a10-9722-9106357e67d0\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 11:56:22 crc kubenswrapper[5052]: I1203 11:56:22.654731 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/e8de8adf-e4e5-4a10-9722-9106357e67d0-ovsdb-rundir\") pod \"ovsdbserver-nb-0\" (UID: \"e8de8adf-e4e5-4a10-9722-9106357e67d0\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 11:56:22 crc kubenswrapper[5052]: I1203 11:56:22.656259 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/e8de8adf-e4e5-4a10-9722-9106357e67d0-ovsdb-rundir\") pod \"ovsdbserver-nb-0\" (UID: \"e8de8adf-e4e5-4a10-9722-9106357e67d0\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 11:56:22 crc kubenswrapper[5052]: I1203 11:56:22.657689 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e8de8adf-e4e5-4a10-9722-9106357e67d0-config\") pod \"ovsdbserver-nb-0\" (UID: \"e8de8adf-e4e5-4a10-9722-9106357e67d0\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 11:56:22 crc kubenswrapper[5052]: I1203 11:56:22.657757 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage01-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage01-crc\") pod \"ovsdbserver-nb-0\" (UID: \"e8de8adf-e4e5-4a10-9722-9106357e67d0\") device mount path \"/mnt/openstack/pv01\"" pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 11:56:22 crc kubenswrapper[5052]: I1203 11:56:22.658740 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/e8de8adf-e4e5-4a10-9722-9106357e67d0-scripts\") pod \"ovsdbserver-nb-0\" (UID: \"e8de8adf-e4e5-4a10-9722-9106357e67d0\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 11:56:22 crc kubenswrapper[5052]: I1203 11:56:22.662006 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovsdbserver-nb-tls-certs\" (UniqueName: \"kubernetes.io/secret/e8de8adf-e4e5-4a10-9722-9106357e67d0-ovsdbserver-nb-tls-certs\") pod \"ovsdbserver-nb-0\" (UID: \"e8de8adf-e4e5-4a10-9722-9106357e67d0\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 11:56:22 crc kubenswrapper[5052]: I1203 11:56:22.662950 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e8de8adf-e4e5-4a10-9722-9106357e67d0-combined-ca-bundle\") pod \"ovsdbserver-nb-0\" (UID: \"e8de8adf-e4e5-4a10-9722-9106357e67d0\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 11:56:22 crc kubenswrapper[5052]: I1203 11:56:22.674872 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 11:56:22 crc kubenswrapper[5052]: I1203 11:56:22.676683 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/e8de8adf-e4e5-4a10-9722-9106357e67d0-metrics-certs-tls-certs\") pod \"ovsdbserver-nb-0\" (UID: \"e8de8adf-e4e5-4a10-9722-9106357e67d0\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 11:56:22 crc kubenswrapper[5052]: I1203 11:56:22.695049 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-6x8wn\" (UniqueName: \"kubernetes.io/projected/e8de8adf-e4e5-4a10-9722-9106357e67d0-kube-api-access-6x8wn\") pod \"ovsdbserver-nb-0\" (UID: \"e8de8adf-e4e5-4a10-9722-9106357e67d0\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 11:56:22 crc kubenswrapper[5052]: I1203 11:56:22.697749 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage01-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage01-crc\") pod \"ovsdbserver-nb-0\" (UID: \"e8de8adf-e4e5-4a10-9722-9106357e67d0\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 11:56:22 crc kubenswrapper[5052]: I1203 11:56:22.853308 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 11:56:23 crc kubenswrapper[5052]: I1203 11:56:23.008243 5052 generic.go:334] "Generic (PLEG): container finished" podID="75c87049-f7b9-4c6b-add4-2eec1f52deb2" containerID="747f297461b54875e189e5f421860b725e3b1e8ee94217f21a72b29f666ac5e9" exitCode=0 Dec 03 11:56:23 crc kubenswrapper[5052]: I1203 11:56:23.008533 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-galera-0" event={"ID":"75c87049-f7b9-4c6b-add4-2eec1f52deb2","Type":"ContainerDied","Data":"747f297461b54875e189e5f421860b725e3b1e8ee94217f21a72b29f666ac5e9"} Dec 03 11:56:23 crc kubenswrapper[5052]: I1203 11:56:23.495190 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-sb-0"] Dec 03 11:56:23 crc kubenswrapper[5052]: I1203 11:56:23.816069 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/community-operators-tb5qk"] Dec 03 11:56:23 crc kubenswrapper[5052]: W1203 11:56:23.818064 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod478d0cb6_25de_46d5_979e_9e43bf1b7d4b.slice/crio-8fb48f8c4cc76d9a046b2c73e73b841bd240767f9507449785c4e91d488f72ae WatchSource:0}: Error finding container 8fb48f8c4cc76d9a046b2c73e73b841bd240767f9507449785c4e91d488f72ae: Status 404 returned error can't find the container with id 8fb48f8c4cc76d9a046b2c73e73b841bd240767f9507449785c4e91d488f72ae Dec 03 11:56:23 crc kubenswrapper[5052]: I1203 11:56:23.837469 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-nb-0"] Dec 03 11:56:23 crc kubenswrapper[5052]: W1203 11:56:23.847181 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode8de8adf_e4e5_4a10_9722_9106357e67d0.slice/crio-9293b1b69f544359bada65ca1151c8ddba8749132dd536bed546abf83b768377 WatchSource:0}: Error finding container 9293b1b69f544359bada65ca1151c8ddba8749132dd536bed546abf83b768377: Status 404 returned error can't find the container with id 9293b1b69f544359bada65ca1151c8ddba8749132dd536bed546abf83b768377 Dec 03 11:56:24 crc kubenswrapper[5052]: I1203 11:56:24.049262 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-nb-0" event={"ID":"e8de8adf-e4e5-4a10-9722-9106357e67d0","Type":"ContainerStarted","Data":"9293b1b69f544359bada65ca1151c8ddba8749132dd536bed546abf83b768377"} Dec 03 11:56:24 crc kubenswrapper[5052]: I1203 11:56:24.051036 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/community-operators-tb5qk" event={"ID":"478d0cb6-25de-46d5-979e-9e43bf1b7d4b","Type":"ContainerStarted","Data":"8fb48f8c4cc76d9a046b2c73e73b841bd240767f9507449785c4e91d488f72ae"} Dec 03 11:56:24 crc kubenswrapper[5052]: I1203 11:56:24.053495 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-sb-0" event={"ID":"3cb4559f-542a-4076-82fb-c0ca88f9de94","Type":"ContainerStarted","Data":"fdc5cd84bfb15a2c4b38c80fec90fd397d2d372bb2cd5443eb73cf72b1a62b30"} Dec 03 11:56:24 crc kubenswrapper[5052]: I1203 11:56:24.053557 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-sb-0" event={"ID":"3cb4559f-542a-4076-82fb-c0ca88f9de94","Type":"ContainerStarted","Data":"d43b0348a840f00bd4fe9d9e2af0a2be437725e748e98b1e97bf216455be5575"} Dec 03 11:56:24 crc kubenswrapper[5052]: I1203 11:56:24.071636 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-galera-0" event={"ID":"75c87049-f7b9-4c6b-add4-2eec1f52deb2","Type":"ContainerStarted","Data":"f270b916e5373a4dc04617edb6ac8552ab0bed39941d9a4293aaa5264e513b53"} Dec 03 11:56:24 crc kubenswrapper[5052]: I1203 11:56:24.074813 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-operators/observability-operator-d8bb48f5d-swndl" event={"ID":"0e9b7522-0b02-484a-b9e6-545e8dabcbc7","Type":"ContainerStarted","Data":"a4e1b67b6138193bd6e775ae52334a9ff828ce00a548e80f0dbb0d1332aa5204"} Dec 03 11:56:24 crc kubenswrapper[5052]: I1203 11:56:24.075706 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openshift-operators/observability-operator-d8bb48f5d-swndl" Dec 03 11:56:24 crc kubenswrapper[5052]: I1203 11:56:24.079661 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/kube-state-metrics-0" event={"ID":"316d245b-16aa-44a0-8aaa-4a41794e1291","Type":"ContainerStarted","Data":"35db98cfc0c0e4e1307993f339292d48cb6c7695dea2f6b7b92d067b386acd94"} Dec 03 11:56:24 crc kubenswrapper[5052]: I1203 11:56:24.080220 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 11:56:24 crc kubenswrapper[5052]: I1203 11:56:24.095099 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/openstack-galera-0" podStartSLOduration=12.095082984 podStartE2EDuration="12.095082984s" podCreationTimestamp="2025-12-03 11:56:12 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:56:24.091825411 +0000 UTC m=+2174.680067275" watchObservedRunningTime="2025-12-03 11:56:24.095082984 +0000 UTC m=+2174.683324748" Dec 03 11:56:24 crc kubenswrapper[5052]: I1203 11:56:24.123695 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-operators/observability-operator-d8bb48f5d-swndl" Dec 03 11:56:24 crc kubenswrapper[5052]: I1203 11:56:24.125280 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/kube-state-metrics-0" podStartSLOduration=3.3123508 podStartE2EDuration="8.125262968s" podCreationTimestamp="2025-12-03 11:56:16 +0000 UTC" firstStartedPulling="2025-12-03 11:56:18.224042853 +0000 UTC m=+2168.812284617" lastFinishedPulling="2025-12-03 11:56:23.036955021 +0000 UTC m=+2173.625196785" observedRunningTime="2025-12-03 11:56:24.115499998 +0000 UTC m=+2174.703741762" watchObservedRunningTime="2025-12-03 11:56:24.125262968 +0000 UTC m=+2174.713504742" Dec 03 11:56:24 crc kubenswrapper[5052]: I1203 11:56:24.152934 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-operators/observability-operator-d8bb48f5d-swndl" podStartSLOduration=3.394230424 podStartE2EDuration="16.152812796s" podCreationTimestamp="2025-12-03 11:56:08 +0000 UTC" firstStartedPulling="2025-12-03 11:56:10.26513145 +0000 UTC m=+2160.853373214" lastFinishedPulling="2025-12-03 11:56:23.023713822 +0000 UTC m=+2173.611955586" observedRunningTime="2025-12-03 11:56:24.151056586 +0000 UTC m=+2174.739298350" watchObservedRunningTime="2025-12-03 11:56:24.152812796 +0000 UTC m=+2174.741054560" Dec 03 11:56:24 crc kubenswrapper[5052]: I1203 11:56:24.342737 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/memcached-0" Dec 03 11:56:25 crc kubenswrapper[5052]: I1203 11:56:25.092763 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-nb-0" event={"ID":"e8de8adf-e4e5-4a10-9722-9106357e67d0","Type":"ContainerStarted","Data":"a3deb4fed82228a5b22c0094ab0a2185263de5b8a00e34fbefb31c5da84f1d53"} Dec 03 11:56:25 crc kubenswrapper[5052]: I1203 11:56:25.095259 5052 generic.go:334] "Generic (PLEG): container finished" podID="478d0cb6-25de-46d5-979e-9e43bf1b7d4b" containerID="ef87bb53feef54bf7a1bda6533d1d5fd699d50c9e8449bc841967feab918c6e0" exitCode=0 Dec 03 11:56:25 crc kubenswrapper[5052]: I1203 11:56:25.095348 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/community-operators-tb5qk" event={"ID":"478d0cb6-25de-46d5-979e-9e43bf1b7d4b","Type":"ContainerDied","Data":"ef87bb53feef54bf7a1bda6533d1d5fd699d50c9e8449bc841967feab918c6e0"} Dec 03 11:56:25 crc kubenswrapper[5052]: I1203 11:56:25.424696 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/alertmanager-metric-storage-0"] Dec 03 11:56:25 crc kubenswrapper[5052]: I1203 11:56:25.426431 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/alertmanager-metric-storage-0" Dec 03 11:56:25 crc kubenswrapper[5052]: I1203 11:56:25.429463 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/alertmanager-metric-storage-0"] Dec 03 11:56:25 crc kubenswrapper[5052]: I1203 11:56:25.430452 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"alertmanager-metric-storage-generated" Dec 03 11:56:25 crc kubenswrapper[5052]: I1203 11:56:25.430486 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"alertmanager-metric-storage-web-config" Dec 03 11:56:25 crc kubenswrapper[5052]: I1203 11:56:25.430535 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"alertmanager-metric-storage-tls-assets-0" Dec 03 11:56:25 crc kubenswrapper[5052]: I1203 11:56:25.435041 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"alertmanager-metric-storage-cluster-tls-config" Dec 03 11:56:25 crc kubenswrapper[5052]: I1203 11:56:25.435094 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"metric-storage-alertmanager-dockercfg-db2td" Dec 03 11:56:25 crc kubenswrapper[5052]: I1203 11:56:25.544078 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7cbbb6c5-a087-400f-9beb-e740ae1ddfbd-config-out\") pod \"alertmanager-metric-storage-0\" (UID: \"7cbbb6c5-a087-400f-9beb-e740ae1ddfbd\") " pod="openstack-kuttl-tests/alertmanager-metric-storage-0" Dec 03 11:56:25 crc kubenswrapper[5052]: I1203 11:56:25.544165 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kwp54\" (UniqueName: \"kubernetes.io/projected/7cbbb6c5-a087-400f-9beb-e740ae1ddfbd-kube-api-access-kwp54\") pod \"alertmanager-metric-storage-0\" (UID: \"7cbbb6c5-a087-400f-9beb-e740ae1ddfbd\") " pod="openstack-kuttl-tests/alertmanager-metric-storage-0" Dec 03 11:56:25 crc kubenswrapper[5052]: I1203 11:56:25.544190 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/7cbbb6c5-a087-400f-9beb-e740ae1ddfbd-config-volume\") pod \"alertmanager-metric-storage-0\" (UID: \"7cbbb6c5-a087-400f-9beb-e740ae1ddfbd\") " pod="openstack-kuttl-tests/alertmanager-metric-storage-0" Dec 03 11:56:25 crc kubenswrapper[5052]: I1203 11:56:25.544222 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7cbbb6c5-a087-400f-9beb-e740ae1ddfbd-tls-assets\") pod \"alertmanager-metric-storage-0\" (UID: \"7cbbb6c5-a087-400f-9beb-e740ae1ddfbd\") " pod="openstack-kuttl-tests/alertmanager-metric-storage-0" Dec 03 11:56:25 crc kubenswrapper[5052]: I1203 11:56:25.544253 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-metric-storage-db\" (UniqueName: \"kubernetes.io/empty-dir/7cbbb6c5-a087-400f-9beb-e740ae1ddfbd-alertmanager-metric-storage-db\") pod \"alertmanager-metric-storage-0\" (UID: \"7cbbb6c5-a087-400f-9beb-e740ae1ddfbd\") " pod="openstack-kuttl-tests/alertmanager-metric-storage-0" Dec 03 11:56:25 crc kubenswrapper[5052]: I1203 11:56:25.544298 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7cbbb6c5-a087-400f-9beb-e740ae1ddfbd-web-config\") pod \"alertmanager-metric-storage-0\" (UID: \"7cbbb6c5-a087-400f-9beb-e740ae1ddfbd\") " pod="openstack-kuttl-tests/alertmanager-metric-storage-0" Dec 03 11:56:25 crc kubenswrapper[5052]: I1203 11:56:25.544344 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/7cbbb6c5-a087-400f-9beb-e740ae1ddfbd-cluster-tls-config\") pod \"alertmanager-metric-storage-0\" (UID: \"7cbbb6c5-a087-400f-9beb-e740ae1ddfbd\") " pod="openstack-kuttl-tests/alertmanager-metric-storage-0" Dec 03 11:56:25 crc kubenswrapper[5052]: I1203 11:56:25.645432 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7cbbb6c5-a087-400f-9beb-e740ae1ddfbd-web-config\") pod \"alertmanager-metric-storage-0\" (UID: \"7cbbb6c5-a087-400f-9beb-e740ae1ddfbd\") " pod="openstack-kuttl-tests/alertmanager-metric-storage-0" Dec 03 11:56:25 crc kubenswrapper[5052]: I1203 11:56:25.645743 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/7cbbb6c5-a087-400f-9beb-e740ae1ddfbd-cluster-tls-config\") pod \"alertmanager-metric-storage-0\" (UID: \"7cbbb6c5-a087-400f-9beb-e740ae1ddfbd\") " pod="openstack-kuttl-tests/alertmanager-metric-storage-0" Dec 03 11:56:25 crc kubenswrapper[5052]: I1203 11:56:25.645777 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7cbbb6c5-a087-400f-9beb-e740ae1ddfbd-config-out\") pod \"alertmanager-metric-storage-0\" (UID: \"7cbbb6c5-a087-400f-9beb-e740ae1ddfbd\") " pod="openstack-kuttl-tests/alertmanager-metric-storage-0" Dec 03 11:56:25 crc kubenswrapper[5052]: I1203 11:56:25.645833 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-kwp54\" (UniqueName: \"kubernetes.io/projected/7cbbb6c5-a087-400f-9beb-e740ae1ddfbd-kube-api-access-kwp54\") pod \"alertmanager-metric-storage-0\" (UID: \"7cbbb6c5-a087-400f-9beb-e740ae1ddfbd\") " pod="openstack-kuttl-tests/alertmanager-metric-storage-0" Dec 03 11:56:25 crc kubenswrapper[5052]: I1203 11:56:25.645856 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/7cbbb6c5-a087-400f-9beb-e740ae1ddfbd-config-volume\") pod \"alertmanager-metric-storage-0\" (UID: \"7cbbb6c5-a087-400f-9beb-e740ae1ddfbd\") " pod="openstack-kuttl-tests/alertmanager-metric-storage-0" Dec 03 11:56:25 crc kubenswrapper[5052]: I1203 11:56:25.645889 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7cbbb6c5-a087-400f-9beb-e740ae1ddfbd-tls-assets\") pod \"alertmanager-metric-storage-0\" (UID: \"7cbbb6c5-a087-400f-9beb-e740ae1ddfbd\") " pod="openstack-kuttl-tests/alertmanager-metric-storage-0" Dec 03 11:56:25 crc kubenswrapper[5052]: I1203 11:56:25.645909 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"alertmanager-metric-storage-db\" (UniqueName: \"kubernetes.io/empty-dir/7cbbb6c5-a087-400f-9beb-e740ae1ddfbd-alertmanager-metric-storage-db\") pod \"alertmanager-metric-storage-0\" (UID: \"7cbbb6c5-a087-400f-9beb-e740ae1ddfbd\") " pod="openstack-kuttl-tests/alertmanager-metric-storage-0" Dec 03 11:56:25 crc kubenswrapper[5052]: I1203 11:56:25.646323 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"alertmanager-metric-storage-db\" (UniqueName: \"kubernetes.io/empty-dir/7cbbb6c5-a087-400f-9beb-e740ae1ddfbd-alertmanager-metric-storage-db\") pod \"alertmanager-metric-storage-0\" (UID: \"7cbbb6c5-a087-400f-9beb-e740ae1ddfbd\") " pod="openstack-kuttl-tests/alertmanager-metric-storage-0" Dec 03 11:56:25 crc kubenswrapper[5052]: I1203 11:56:25.650380 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7cbbb6c5-a087-400f-9beb-e740ae1ddfbd-config-out\") pod \"alertmanager-metric-storage-0\" (UID: \"7cbbb6c5-a087-400f-9beb-e740ae1ddfbd\") " pod="openstack-kuttl-tests/alertmanager-metric-storage-0" Dec 03 11:56:25 crc kubenswrapper[5052]: I1203 11:56:25.650620 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/7cbbb6c5-a087-400f-9beb-e740ae1ddfbd-config-volume\") pod \"alertmanager-metric-storage-0\" (UID: \"7cbbb6c5-a087-400f-9beb-e740ae1ddfbd\") " pod="openstack-kuttl-tests/alertmanager-metric-storage-0" Dec 03 11:56:25 crc kubenswrapper[5052]: I1203 11:56:25.668737 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7cbbb6c5-a087-400f-9beb-e740ae1ddfbd-tls-assets\") pod \"alertmanager-metric-storage-0\" (UID: \"7cbbb6c5-a087-400f-9beb-e740ae1ddfbd\") " pod="openstack-kuttl-tests/alertmanager-metric-storage-0" Dec 03 11:56:25 crc kubenswrapper[5052]: I1203 11:56:25.668940 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/7cbbb6c5-a087-400f-9beb-e740ae1ddfbd-cluster-tls-config\") pod \"alertmanager-metric-storage-0\" (UID: \"7cbbb6c5-a087-400f-9beb-e740ae1ddfbd\") " pod="openstack-kuttl-tests/alertmanager-metric-storage-0" Dec 03 11:56:25 crc kubenswrapper[5052]: I1203 11:56:25.668972 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-kwp54\" (UniqueName: \"kubernetes.io/projected/7cbbb6c5-a087-400f-9beb-e740ae1ddfbd-kube-api-access-kwp54\") pod \"alertmanager-metric-storage-0\" (UID: \"7cbbb6c5-a087-400f-9beb-e740ae1ddfbd\") " pod="openstack-kuttl-tests/alertmanager-metric-storage-0" Dec 03 11:56:25 crc kubenswrapper[5052]: I1203 11:56:25.669095 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7cbbb6c5-a087-400f-9beb-e740ae1ddfbd-web-config\") pod \"alertmanager-metric-storage-0\" (UID: \"7cbbb6c5-a087-400f-9beb-e740ae1ddfbd\") " pod="openstack-kuttl-tests/alertmanager-metric-storage-0" Dec 03 11:56:25 crc kubenswrapper[5052]: I1203 11:56:25.755973 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/alertmanager-metric-storage-0" Dec 03 11:56:25 crc kubenswrapper[5052]: I1203 11:56:25.962094 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/prometheus-metric-storage-0"] Dec 03 11:56:25 crc kubenswrapper[5052]: I1203 11:56:25.964410 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/prometheus-metric-storage-0" Dec 03 11:56:25 crc kubenswrapper[5052]: I1203 11:56:25.968344 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"prometheus-metric-storage" Dec 03 11:56:25 crc kubenswrapper[5052]: I1203 11:56:25.969280 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"prometheus-metric-storage-tls-assets-0" Dec 03 11:56:25 crc kubenswrapper[5052]: I1203 11:56:25.969615 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"prometheus-metric-storage-web-config" Dec 03 11:56:25 crc kubenswrapper[5052]: I1203 11:56:25.969825 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"prometheus-metric-storage-thanos-prometheus-http-client-file" Dec 03 11:56:25 crc kubenswrapper[5052]: I1203 11:56:25.969946 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"metric-storage-prometheus-dockercfg-9948m" Dec 03 11:56:25 crc kubenswrapper[5052]: I1203 11:56:25.970155 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"prometheus-metric-storage-rulefiles-0" Dec 03 11:56:25 crc kubenswrapper[5052]: I1203 11:56:25.985061 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/prometheus-metric-storage-0"] Dec 03 11:56:26 crc kubenswrapper[5052]: I1203 11:56:26.055623 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage19-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage19-crc\") pod \"prometheus-metric-storage-0\" (UID: \"33918bc1-340a-4d26-a204-001b85c15f04\") " pod="openstack-kuttl-tests/prometheus-metric-storage-0" Dec 03 11:56:26 crc kubenswrapper[5052]: I1203 11:56:26.055715 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-metric-storage-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/33918bc1-340a-4d26-a204-001b85c15f04-prometheus-metric-storage-rulefiles-0\") pod \"prometheus-metric-storage-0\" (UID: \"33918bc1-340a-4d26-a204-001b85c15f04\") " pod="openstack-kuttl-tests/prometheus-metric-storage-0" Dec 03 11:56:26 crc kubenswrapper[5052]: I1203 11:56:26.055743 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9xpqc\" (UniqueName: \"kubernetes.io/projected/33918bc1-340a-4d26-a204-001b85c15f04-kube-api-access-9xpqc\") pod \"prometheus-metric-storage-0\" (UID: \"33918bc1-340a-4d26-a204-001b85c15f04\") " pod="openstack-kuttl-tests/prometheus-metric-storage-0" Dec 03 11:56:26 crc kubenswrapper[5052]: I1203 11:56:26.055783 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/33918bc1-340a-4d26-a204-001b85c15f04-config\") pod \"prometheus-metric-storage-0\" (UID: \"33918bc1-340a-4d26-a204-001b85c15f04\") " pod="openstack-kuttl-tests/prometheus-metric-storage-0" Dec 03 11:56:26 crc kubenswrapper[5052]: I1203 11:56:26.055844 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/33918bc1-340a-4d26-a204-001b85c15f04-tls-assets\") pod \"prometheus-metric-storage-0\" (UID: \"33918bc1-340a-4d26-a204-001b85c15f04\") " pod="openstack-kuttl-tests/prometheus-metric-storage-0" Dec 03 11:56:26 crc kubenswrapper[5052]: I1203 11:56:26.055876 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/33918bc1-340a-4d26-a204-001b85c15f04-thanos-prometheus-http-client-file\") pod \"prometheus-metric-storage-0\" (UID: \"33918bc1-340a-4d26-a204-001b85c15f04\") " pod="openstack-kuttl-tests/prometheus-metric-storage-0" Dec 03 11:56:26 crc kubenswrapper[5052]: I1203 11:56:26.055931 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/33918bc1-340a-4d26-a204-001b85c15f04-web-config\") pod \"prometheus-metric-storage-0\" (UID: \"33918bc1-340a-4d26-a204-001b85c15f04\") " pod="openstack-kuttl-tests/prometheus-metric-storage-0" Dec 03 11:56:26 crc kubenswrapper[5052]: I1203 11:56:26.055963 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/33918bc1-340a-4d26-a204-001b85c15f04-config-out\") pod \"prometheus-metric-storage-0\" (UID: \"33918bc1-340a-4d26-a204-001b85c15f04\") " pod="openstack-kuttl-tests/prometheus-metric-storage-0" Dec 03 11:56:26 crc kubenswrapper[5052]: I1203 11:56:26.106623 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-sb-0" event={"ID":"3cb4559f-542a-4076-82fb-c0ca88f9de94","Type":"ContainerStarted","Data":"1ed38008a53ff370520241b3fd280d78870c056da808615558e81bfec79012cf"} Dec 03 11:56:26 crc kubenswrapper[5052]: I1203 11:56:26.155734 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/ovsdbserver-sb-0" podStartSLOduration=5.155716266 podStartE2EDuration="5.155716266s" podCreationTimestamp="2025-12-03 11:56:21 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:56:26.148903231 +0000 UTC m=+2176.737144995" watchObservedRunningTime="2025-12-03 11:56:26.155716266 +0000 UTC m=+2176.743958050" Dec 03 11:56:26 crc kubenswrapper[5052]: I1203 11:56:26.156829 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage19-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage19-crc\") pod \"prometheus-metric-storage-0\" (UID: \"33918bc1-340a-4d26-a204-001b85c15f04\") " pod="openstack-kuttl-tests/prometheus-metric-storage-0" Dec 03 11:56:26 crc kubenswrapper[5052]: I1203 11:56:26.156934 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"prometheus-metric-storage-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/33918bc1-340a-4d26-a204-001b85c15f04-prometheus-metric-storage-rulefiles-0\") pod \"prometheus-metric-storage-0\" (UID: \"33918bc1-340a-4d26-a204-001b85c15f04\") " pod="openstack-kuttl-tests/prometheus-metric-storage-0" Dec 03 11:56:26 crc kubenswrapper[5052]: I1203 11:56:26.156959 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-9xpqc\" (UniqueName: \"kubernetes.io/projected/33918bc1-340a-4d26-a204-001b85c15f04-kube-api-access-9xpqc\") pod \"prometheus-metric-storage-0\" (UID: \"33918bc1-340a-4d26-a204-001b85c15f04\") " pod="openstack-kuttl-tests/prometheus-metric-storage-0" Dec 03 11:56:26 crc kubenswrapper[5052]: I1203 11:56:26.157010 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/33918bc1-340a-4d26-a204-001b85c15f04-config\") pod \"prometheus-metric-storage-0\" (UID: \"33918bc1-340a-4d26-a204-001b85c15f04\") " pod="openstack-kuttl-tests/prometheus-metric-storage-0" Dec 03 11:56:26 crc kubenswrapper[5052]: I1203 11:56:26.157094 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/33918bc1-340a-4d26-a204-001b85c15f04-tls-assets\") pod \"prometheus-metric-storage-0\" (UID: \"33918bc1-340a-4d26-a204-001b85c15f04\") " pod="openstack-kuttl-tests/prometheus-metric-storage-0" Dec 03 11:56:26 crc kubenswrapper[5052]: I1203 11:56:26.157146 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/33918bc1-340a-4d26-a204-001b85c15f04-thanos-prometheus-http-client-file\") pod \"prometheus-metric-storage-0\" (UID: \"33918bc1-340a-4d26-a204-001b85c15f04\") " pod="openstack-kuttl-tests/prometheus-metric-storage-0" Dec 03 11:56:26 crc kubenswrapper[5052]: I1203 11:56:26.157203 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/33918bc1-340a-4d26-a204-001b85c15f04-web-config\") pod \"prometheus-metric-storage-0\" (UID: \"33918bc1-340a-4d26-a204-001b85c15f04\") " pod="openstack-kuttl-tests/prometheus-metric-storage-0" Dec 03 11:56:26 crc kubenswrapper[5052]: I1203 11:56:26.157225 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/33918bc1-340a-4d26-a204-001b85c15f04-config-out\") pod \"prometheus-metric-storage-0\" (UID: \"33918bc1-340a-4d26-a204-001b85c15f04\") " pod="openstack-kuttl-tests/prometheus-metric-storage-0" Dec 03 11:56:26 crc kubenswrapper[5052]: I1203 11:56:26.160753 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/33918bc1-340a-4d26-a204-001b85c15f04-config-out\") pod \"prometheus-metric-storage-0\" (UID: \"33918bc1-340a-4d26-a204-001b85c15f04\") " pod="openstack-kuttl-tests/prometheus-metric-storage-0" Dec 03 11:56:26 crc kubenswrapper[5052]: I1203 11:56:26.161065 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage19-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage19-crc\") pod \"prometheus-metric-storage-0\" (UID: \"33918bc1-340a-4d26-a204-001b85c15f04\") device mount path \"/mnt/openstack/pv19\"" pod="openstack-kuttl-tests/prometheus-metric-storage-0" Dec 03 11:56:26 crc kubenswrapper[5052]: I1203 11:56:26.162591 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"prometheus-metric-storage-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/33918bc1-340a-4d26-a204-001b85c15f04-prometheus-metric-storage-rulefiles-0\") pod \"prometheus-metric-storage-0\" (UID: \"33918bc1-340a-4d26-a204-001b85c15f04\") " pod="openstack-kuttl-tests/prometheus-metric-storage-0" Dec 03 11:56:26 crc kubenswrapper[5052]: I1203 11:56:26.163071 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/33918bc1-340a-4d26-a204-001b85c15f04-tls-assets\") pod \"prometheus-metric-storage-0\" (UID: \"33918bc1-340a-4d26-a204-001b85c15f04\") " pod="openstack-kuttl-tests/prometheus-metric-storage-0" Dec 03 11:56:26 crc kubenswrapper[5052]: I1203 11:56:26.164742 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/33918bc1-340a-4d26-a204-001b85c15f04-config\") pod \"prometheus-metric-storage-0\" (UID: \"33918bc1-340a-4d26-a204-001b85c15f04\") " pod="openstack-kuttl-tests/prometheus-metric-storage-0" Dec 03 11:56:26 crc kubenswrapper[5052]: I1203 11:56:26.165638 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/33918bc1-340a-4d26-a204-001b85c15f04-web-config\") pod \"prometheus-metric-storage-0\" (UID: \"33918bc1-340a-4d26-a204-001b85c15f04\") " pod="openstack-kuttl-tests/prometheus-metric-storage-0" Dec 03 11:56:26 crc kubenswrapper[5052]: I1203 11:56:26.167322 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/33918bc1-340a-4d26-a204-001b85c15f04-thanos-prometheus-http-client-file\") pod \"prometheus-metric-storage-0\" (UID: \"33918bc1-340a-4d26-a204-001b85c15f04\") " pod="openstack-kuttl-tests/prometheus-metric-storage-0" Dec 03 11:56:26 crc kubenswrapper[5052]: I1203 11:56:26.184000 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-9xpqc\" (UniqueName: \"kubernetes.io/projected/33918bc1-340a-4d26-a204-001b85c15f04-kube-api-access-9xpqc\") pod \"prometheus-metric-storage-0\" (UID: \"33918bc1-340a-4d26-a204-001b85c15f04\") " pod="openstack-kuttl-tests/prometheus-metric-storage-0" Dec 03 11:56:26 crc kubenswrapper[5052]: I1203 11:56:26.203921 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage19-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage19-crc\") pod \"prometheus-metric-storage-0\" (UID: \"33918bc1-340a-4d26-a204-001b85c15f04\") " pod="openstack-kuttl-tests/prometheus-metric-storage-0" Dec 03 11:56:26 crc kubenswrapper[5052]: I1203 11:56:26.312925 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/prometheus-metric-storage-0" Dec 03 11:56:26 crc kubenswrapper[5052]: I1203 11:56:26.380751 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/alertmanager-metric-storage-0"] Dec 03 11:56:26 crc kubenswrapper[5052]: W1203 11:56:26.391088 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7cbbb6c5_a087_400f_9beb_e740ae1ddfbd.slice/crio-361730d7e651e0be896723f73ddb01ac250f74201d7ec122093f7b12ce8b7836 WatchSource:0}: Error finding container 361730d7e651e0be896723f73ddb01ac250f74201d7ec122093f7b12ce8b7836: Status 404 returned error can't find the container with id 361730d7e651e0be896723f73ddb01ac250f74201d7ec122093f7b12ce8b7836 Dec 03 11:56:26 crc kubenswrapper[5052]: I1203 11:56:26.831621 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/prometheus-metric-storage-0"] Dec 03 11:56:26 crc kubenswrapper[5052]: W1203 11:56:26.844884 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33918bc1_340a_4d26_a204_001b85c15f04.slice/crio-768873257caa2a7a191a492cee44124407a3c1d86b7d431237814573ef7b6713 WatchSource:0}: Error finding container 768873257caa2a7a191a492cee44124407a3c1d86b7d431237814573ef7b6713: Status 404 returned error can't find the container with id 768873257caa2a7a191a492cee44124407a3c1d86b7d431237814573ef7b6713 Dec 03 11:56:27 crc kubenswrapper[5052]: I1203 11:56:27.116838 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/prometheus-metric-storage-0" event={"ID":"33918bc1-340a-4d26-a204-001b85c15f04","Type":"ContainerStarted","Data":"768873257caa2a7a191a492cee44124407a3c1d86b7d431237814573ef7b6713"} Dec 03 11:56:27 crc kubenswrapper[5052]: I1203 11:56:27.118876 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/alertmanager-metric-storage-0" event={"ID":"7cbbb6c5-a087-400f-9beb-e740ae1ddfbd","Type":"ContainerStarted","Data":"361730d7e651e0be896723f73ddb01ac250f74201d7ec122093f7b12ce8b7836"} Dec 03 11:56:27 crc kubenswrapper[5052]: I1203 11:56:27.123071 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-nb-0" event={"ID":"e8de8adf-e4e5-4a10-9722-9106357e67d0","Type":"ContainerStarted","Data":"156a9ab495313477724afb77cc40733731b002a3db4ec302faefc4c513160b71"} Dec 03 11:56:27 crc kubenswrapper[5052]: I1203 11:56:27.192415 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/ovsdbserver-nb-0" podStartSLOduration=6.19224507 podStartE2EDuration="6.19224507s" podCreationTimestamp="2025-12-03 11:56:21 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:56:27.19082989 +0000 UTC m=+2177.779071664" watchObservedRunningTime="2025-12-03 11:56:27.19224507 +0000 UTC m=+2177.780486834" Dec 03 11:56:27 crc kubenswrapper[5052]: I1203 11:56:27.683360 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 11:56:27 crc kubenswrapper[5052]: I1203 11:56:27.703416 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/swift-storage-0"] Dec 03 11:56:27 crc kubenswrapper[5052]: I1203 11:56:27.710421 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-storage-0" Dec 03 11:56:27 crc kubenswrapper[5052]: I1203 11:56:27.720131 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"swift-storage-config-data" Dec 03 11:56:27 crc kubenswrapper[5052]: I1203 11:56:27.720183 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"swift-ring-files" Dec 03 11:56:27 crc kubenswrapper[5052]: I1203 11:56:27.720465 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"swift-conf" Dec 03 11:56:27 crc kubenswrapper[5052]: I1203 11:56:27.720671 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"swift-swift-dockercfg-9xlmq" Dec 03 11:56:27 crc kubenswrapper[5052]: I1203 11:56:27.737439 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/swift-storage-0"] Dec 03 11:56:27 crc kubenswrapper[5052]: I1203 11:56:27.854592 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 11:56:27 crc kubenswrapper[5052]: I1203 11:56:27.888462 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lock\" (UniqueName: \"kubernetes.io/empty-dir/7c36839d-8518-4b0e-bf63-ec94d92e3c13-lock\") pod \"swift-storage-0\" (UID: \"7c36839d-8518-4b0e-bf63-ec94d92e3c13\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 11:56:27 crc kubenswrapper[5052]: I1203 11:56:27.888550 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage20-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage20-crc\") pod \"swift-storage-0\" (UID: \"7c36839d-8518-4b0e-bf63-ec94d92e3c13\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 11:56:27 crc kubenswrapper[5052]: I1203 11:56:27.888597 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cache\" (UniqueName: \"kubernetes.io/empty-dir/7c36839d-8518-4b0e-bf63-ec94d92e3c13-cache\") pod \"swift-storage-0\" (UID: \"7c36839d-8518-4b0e-bf63-ec94d92e3c13\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 11:56:27 crc kubenswrapper[5052]: I1203 11:56:27.888648 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8zq68\" (UniqueName: \"kubernetes.io/projected/7c36839d-8518-4b0e-bf63-ec94d92e3c13-kube-api-access-8zq68\") pod \"swift-storage-0\" (UID: \"7c36839d-8518-4b0e-bf63-ec94d92e3c13\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 11:56:27 crc kubenswrapper[5052]: I1203 11:56:27.888685 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/7c36839d-8518-4b0e-bf63-ec94d92e3c13-etc-swift\") pod \"swift-storage-0\" (UID: \"7c36839d-8518-4b0e-bf63-ec94d92e3c13\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 11:56:27 crc kubenswrapper[5052]: I1203 11:56:27.989735 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"lock\" (UniqueName: \"kubernetes.io/empty-dir/7c36839d-8518-4b0e-bf63-ec94d92e3c13-lock\") pod \"swift-storage-0\" (UID: \"7c36839d-8518-4b0e-bf63-ec94d92e3c13\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 11:56:27 crc kubenswrapper[5052]: I1203 11:56:27.989827 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage20-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage20-crc\") pod \"swift-storage-0\" (UID: \"7c36839d-8518-4b0e-bf63-ec94d92e3c13\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 11:56:27 crc kubenswrapper[5052]: I1203 11:56:27.989867 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"cache\" (UniqueName: \"kubernetes.io/empty-dir/7c36839d-8518-4b0e-bf63-ec94d92e3c13-cache\") pod \"swift-storage-0\" (UID: \"7c36839d-8518-4b0e-bf63-ec94d92e3c13\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 11:56:27 crc kubenswrapper[5052]: I1203 11:56:27.989934 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-8zq68\" (UniqueName: \"kubernetes.io/projected/7c36839d-8518-4b0e-bf63-ec94d92e3c13-kube-api-access-8zq68\") pod \"swift-storage-0\" (UID: \"7c36839d-8518-4b0e-bf63-ec94d92e3c13\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 11:56:27 crc kubenswrapper[5052]: I1203 11:56:27.989981 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/7c36839d-8518-4b0e-bf63-ec94d92e3c13-etc-swift\") pod \"swift-storage-0\" (UID: \"7c36839d-8518-4b0e-bf63-ec94d92e3c13\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 11:56:27 crc kubenswrapper[5052]: E1203 11:56:27.990184 5052 projected.go:288] Couldn't get configMap openstack-kuttl-tests/swift-ring-files: configmap "swift-ring-files" not found Dec 03 11:56:27 crc kubenswrapper[5052]: E1203 11:56:27.990220 5052 projected.go:194] Error preparing data for projected volume etc-swift for pod openstack-kuttl-tests/swift-storage-0: configmap "swift-ring-files" not found Dec 03 11:56:27 crc kubenswrapper[5052]: E1203 11:56:27.990273 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7c36839d-8518-4b0e-bf63-ec94d92e3c13-etc-swift podName:7c36839d-8518-4b0e-bf63-ec94d92e3c13 nodeName:}" failed. No retries permitted until 2025-12-03 11:56:28.490249798 +0000 UTC m=+2179.078491562 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "etc-swift" (UniqueName: "kubernetes.io/projected/7c36839d-8518-4b0e-bf63-ec94d92e3c13-etc-swift") pod "swift-storage-0" (UID: "7c36839d-8518-4b0e-bf63-ec94d92e3c13") : configmap "swift-ring-files" not found Dec 03 11:56:27 crc kubenswrapper[5052]: I1203 11:56:27.990944 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"lock\" (UniqueName: \"kubernetes.io/empty-dir/7c36839d-8518-4b0e-bf63-ec94d92e3c13-lock\") pod \"swift-storage-0\" (UID: \"7c36839d-8518-4b0e-bf63-ec94d92e3c13\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 11:56:27 crc kubenswrapper[5052]: I1203 11:56:27.991243 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage20-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage20-crc\") pod \"swift-storage-0\" (UID: \"7c36839d-8518-4b0e-bf63-ec94d92e3c13\") device mount path \"/mnt/openstack/pv20\"" pod="openstack-kuttl-tests/swift-storage-0" Dec 03 11:56:27 crc kubenswrapper[5052]: I1203 11:56:27.991649 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"cache\" (UniqueName: \"kubernetes.io/empty-dir/7c36839d-8518-4b0e-bf63-ec94d92e3c13-cache\") pod \"swift-storage-0\" (UID: \"7c36839d-8518-4b0e-bf63-ec94d92e3c13\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 11:56:28 crc kubenswrapper[5052]: I1203 11:56:28.025460 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-8zq68\" (UniqueName: \"kubernetes.io/projected/7c36839d-8518-4b0e-bf63-ec94d92e3c13-kube-api-access-8zq68\") pod \"swift-storage-0\" (UID: \"7c36839d-8518-4b0e-bf63-ec94d92e3c13\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 11:56:28 crc kubenswrapper[5052]: I1203 11:56:28.036740 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage20-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage20-crc\") pod \"swift-storage-0\" (UID: \"7c36839d-8518-4b0e-bf63-ec94d92e3c13\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 11:56:28 crc kubenswrapper[5052]: I1203 11:56:28.137274 5052 generic.go:334] "Generic (PLEG): container finished" podID="478d0cb6-25de-46d5-979e-9e43bf1b7d4b" containerID="968e0cfb3ba2a3fb71f3041ec8d4c733c2efe16927bb0c2821676b869b3fa296" exitCode=0 Dec 03 11:56:28 crc kubenswrapper[5052]: I1203 11:56:28.138459 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/community-operators-tb5qk" event={"ID":"478d0cb6-25de-46d5-979e-9e43bf1b7d4b","Type":"ContainerDied","Data":"968e0cfb3ba2a3fb71f3041ec8d4c733c2efe16927bb0c2821676b869b3fa296"} Dec 03 11:56:28 crc kubenswrapper[5052]: I1203 11:56:28.172901 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/swift-ring-rebalance-mbrxc"] Dec 03 11:56:28 crc kubenswrapper[5052]: I1203 11:56:28.174121 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-ring-rebalance-mbrxc" Dec 03 11:56:28 crc kubenswrapper[5052]: I1203 11:56:28.176977 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"swift-proxy-config-data" Dec 03 11:56:28 crc kubenswrapper[5052]: I1203 11:56:28.177009 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"swift-ring-scripts" Dec 03 11:56:28 crc kubenswrapper[5052]: I1203 11:56:28.188082 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"swift-ring-config-data" Dec 03 11:56:28 crc kubenswrapper[5052]: I1203 11:56:28.192886 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/swift-ring-rebalance-mbrxc"] Dec 03 11:56:28 crc kubenswrapper[5052]: I1203 11:56:28.296693 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dispersionconf\" (UniqueName: \"kubernetes.io/secret/b25492d8-45bc-446a-a60c-b4f732484218-dispersionconf\") pod \"swift-ring-rebalance-mbrxc\" (UID: \"b25492d8-45bc-446a-a60c-b4f732484218\") " pod="openstack-kuttl-tests/swift-ring-rebalance-mbrxc" Dec 03 11:56:28 crc kubenswrapper[5052]: I1203 11:56:28.296747 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f6vds\" (UniqueName: \"kubernetes.io/projected/b25492d8-45bc-446a-a60c-b4f732484218-kube-api-access-f6vds\") pod \"swift-ring-rebalance-mbrxc\" (UID: \"b25492d8-45bc-446a-a60c-b4f732484218\") " pod="openstack-kuttl-tests/swift-ring-rebalance-mbrxc" Dec 03 11:56:28 crc kubenswrapper[5052]: I1203 11:56:28.296816 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"swiftconf\" (UniqueName: \"kubernetes.io/secret/b25492d8-45bc-446a-a60c-b4f732484218-swiftconf\") pod \"swift-ring-rebalance-mbrxc\" (UID: \"b25492d8-45bc-446a-a60c-b4f732484218\") " pod="openstack-kuttl-tests/swift-ring-rebalance-mbrxc" Dec 03 11:56:28 crc kubenswrapper[5052]: I1203 11:56:28.296945 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/b25492d8-45bc-446a-a60c-b4f732484218-scripts\") pod \"swift-ring-rebalance-mbrxc\" (UID: \"b25492d8-45bc-446a-a60c-b4f732484218\") " pod="openstack-kuttl-tests/swift-ring-rebalance-mbrxc" Dec 03 11:56:28 crc kubenswrapper[5052]: I1203 11:56:28.296977 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ring-data-devices\" (UniqueName: \"kubernetes.io/configmap/b25492d8-45bc-446a-a60c-b4f732484218-ring-data-devices\") pod \"swift-ring-rebalance-mbrxc\" (UID: \"b25492d8-45bc-446a-a60c-b4f732484218\") " pod="openstack-kuttl-tests/swift-ring-rebalance-mbrxc" Dec 03 11:56:28 crc kubenswrapper[5052]: I1203 11:56:28.297382 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/empty-dir/b25492d8-45bc-446a-a60c-b4f732484218-etc-swift\") pod \"swift-ring-rebalance-mbrxc\" (UID: \"b25492d8-45bc-446a-a60c-b4f732484218\") " pod="openstack-kuttl-tests/swift-ring-rebalance-mbrxc" Dec 03 11:56:28 crc kubenswrapper[5052]: I1203 11:56:28.297642 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b25492d8-45bc-446a-a60c-b4f732484218-combined-ca-bundle\") pod \"swift-ring-rebalance-mbrxc\" (UID: \"b25492d8-45bc-446a-a60c-b4f732484218\") " pod="openstack-kuttl-tests/swift-ring-rebalance-mbrxc" Dec 03 11:56:28 crc kubenswrapper[5052]: I1203 11:56:28.399444 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b25492d8-45bc-446a-a60c-b4f732484218-combined-ca-bundle\") pod \"swift-ring-rebalance-mbrxc\" (UID: \"b25492d8-45bc-446a-a60c-b4f732484218\") " pod="openstack-kuttl-tests/swift-ring-rebalance-mbrxc" Dec 03 11:56:28 crc kubenswrapper[5052]: I1203 11:56:28.399488 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"dispersionconf\" (UniqueName: \"kubernetes.io/secret/b25492d8-45bc-446a-a60c-b4f732484218-dispersionconf\") pod \"swift-ring-rebalance-mbrxc\" (UID: \"b25492d8-45bc-446a-a60c-b4f732484218\") " pod="openstack-kuttl-tests/swift-ring-rebalance-mbrxc" Dec 03 11:56:28 crc kubenswrapper[5052]: I1203 11:56:28.399520 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-f6vds\" (UniqueName: \"kubernetes.io/projected/b25492d8-45bc-446a-a60c-b4f732484218-kube-api-access-f6vds\") pod \"swift-ring-rebalance-mbrxc\" (UID: \"b25492d8-45bc-446a-a60c-b4f732484218\") " pod="openstack-kuttl-tests/swift-ring-rebalance-mbrxc" Dec 03 11:56:28 crc kubenswrapper[5052]: I1203 11:56:28.399546 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"swiftconf\" (UniqueName: \"kubernetes.io/secret/b25492d8-45bc-446a-a60c-b4f732484218-swiftconf\") pod \"swift-ring-rebalance-mbrxc\" (UID: \"b25492d8-45bc-446a-a60c-b4f732484218\") " pod="openstack-kuttl-tests/swift-ring-rebalance-mbrxc" Dec 03 11:56:28 crc kubenswrapper[5052]: I1203 11:56:28.399610 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/b25492d8-45bc-446a-a60c-b4f732484218-scripts\") pod \"swift-ring-rebalance-mbrxc\" (UID: \"b25492d8-45bc-446a-a60c-b4f732484218\") " pod="openstack-kuttl-tests/swift-ring-rebalance-mbrxc" Dec 03 11:56:28 crc kubenswrapper[5052]: I1203 11:56:28.399631 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ring-data-devices\" (UniqueName: \"kubernetes.io/configmap/b25492d8-45bc-446a-a60c-b4f732484218-ring-data-devices\") pod \"swift-ring-rebalance-mbrxc\" (UID: \"b25492d8-45bc-446a-a60c-b4f732484218\") " pod="openstack-kuttl-tests/swift-ring-rebalance-mbrxc" Dec 03 11:56:28 crc kubenswrapper[5052]: I1203 11:56:28.399666 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/empty-dir/b25492d8-45bc-446a-a60c-b4f732484218-etc-swift\") pod \"swift-ring-rebalance-mbrxc\" (UID: \"b25492d8-45bc-446a-a60c-b4f732484218\") " pod="openstack-kuttl-tests/swift-ring-rebalance-mbrxc" Dec 03 11:56:28 crc kubenswrapper[5052]: I1203 11:56:28.400024 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"etc-swift\" (UniqueName: \"kubernetes.io/empty-dir/b25492d8-45bc-446a-a60c-b4f732484218-etc-swift\") pod \"swift-ring-rebalance-mbrxc\" (UID: \"b25492d8-45bc-446a-a60c-b4f732484218\") " pod="openstack-kuttl-tests/swift-ring-rebalance-mbrxc" Dec 03 11:56:28 crc kubenswrapper[5052]: I1203 11:56:28.402625 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/b25492d8-45bc-446a-a60c-b4f732484218-scripts\") pod \"swift-ring-rebalance-mbrxc\" (UID: \"b25492d8-45bc-446a-a60c-b4f732484218\") " pod="openstack-kuttl-tests/swift-ring-rebalance-mbrxc" Dec 03 11:56:28 crc kubenswrapper[5052]: I1203 11:56:28.402741 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ring-data-devices\" (UniqueName: \"kubernetes.io/configmap/b25492d8-45bc-446a-a60c-b4f732484218-ring-data-devices\") pod \"swift-ring-rebalance-mbrxc\" (UID: \"b25492d8-45bc-446a-a60c-b4f732484218\") " pod="openstack-kuttl-tests/swift-ring-rebalance-mbrxc" Dec 03 11:56:28 crc kubenswrapper[5052]: I1203 11:56:28.408271 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b25492d8-45bc-446a-a60c-b4f732484218-combined-ca-bundle\") pod \"swift-ring-rebalance-mbrxc\" (UID: \"b25492d8-45bc-446a-a60c-b4f732484218\") " pod="openstack-kuttl-tests/swift-ring-rebalance-mbrxc" Dec 03 11:56:28 crc kubenswrapper[5052]: I1203 11:56:28.412325 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"swiftconf\" (UniqueName: \"kubernetes.io/secret/b25492d8-45bc-446a-a60c-b4f732484218-swiftconf\") pod \"swift-ring-rebalance-mbrxc\" (UID: \"b25492d8-45bc-446a-a60c-b4f732484218\") " pod="openstack-kuttl-tests/swift-ring-rebalance-mbrxc" Dec 03 11:56:28 crc kubenswrapper[5052]: I1203 11:56:28.412891 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"dispersionconf\" (UniqueName: \"kubernetes.io/secret/b25492d8-45bc-446a-a60c-b4f732484218-dispersionconf\") pod \"swift-ring-rebalance-mbrxc\" (UID: \"b25492d8-45bc-446a-a60c-b4f732484218\") " pod="openstack-kuttl-tests/swift-ring-rebalance-mbrxc" Dec 03 11:56:28 crc kubenswrapper[5052]: I1203 11:56:28.422487 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-f6vds\" (UniqueName: \"kubernetes.io/projected/b25492d8-45bc-446a-a60c-b4f732484218-kube-api-access-f6vds\") pod \"swift-ring-rebalance-mbrxc\" (UID: \"b25492d8-45bc-446a-a60c-b4f732484218\") " pod="openstack-kuttl-tests/swift-ring-rebalance-mbrxc" Dec 03 11:56:28 crc kubenswrapper[5052]: I1203 11:56:28.501236 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/7c36839d-8518-4b0e-bf63-ec94d92e3c13-etc-swift\") pod \"swift-storage-0\" (UID: \"7c36839d-8518-4b0e-bf63-ec94d92e3c13\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 11:56:28 crc kubenswrapper[5052]: E1203 11:56:28.501442 5052 projected.go:288] Couldn't get configMap openstack-kuttl-tests/swift-ring-files: configmap "swift-ring-files" not found Dec 03 11:56:28 crc kubenswrapper[5052]: E1203 11:56:28.501456 5052 projected.go:194] Error preparing data for projected volume etc-swift for pod openstack-kuttl-tests/swift-storage-0: configmap "swift-ring-files" not found Dec 03 11:56:28 crc kubenswrapper[5052]: E1203 11:56:28.501530 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7c36839d-8518-4b0e-bf63-ec94d92e3c13-etc-swift podName:7c36839d-8518-4b0e-bf63-ec94d92e3c13 nodeName:}" failed. No retries permitted until 2025-12-03 11:56:29.501485939 +0000 UTC m=+2180.089727703 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "etc-swift" (UniqueName: "kubernetes.io/projected/7c36839d-8518-4b0e-bf63-ec94d92e3c13-etc-swift") pod "swift-storage-0" (UID: "7c36839d-8518-4b0e-bf63-ec94d92e3c13") : configmap "swift-ring-files" not found Dec 03 11:56:28 crc kubenswrapper[5052]: I1203 11:56:28.501997 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-ring-rebalance-mbrxc" Dec 03 11:56:28 crc kubenswrapper[5052]: I1203 11:56:28.676339 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 11:56:28 crc kubenswrapper[5052]: I1203 11:56:28.719797 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 11:56:28 crc kubenswrapper[5052]: I1203 11:56:28.854873 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 11:56:28 crc kubenswrapper[5052]: I1203 11:56:28.895558 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 11:56:29 crc kubenswrapper[5052]: I1203 11:56:29.283503 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 11:56:29 crc kubenswrapper[5052]: I1203 11:56:29.293240 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 11:56:29 crc kubenswrapper[5052]: I1203 11:56:29.389264 5052 patch_prober.go:28] interesting pod/machine-config-daemon-74bj6 container/machine-config-daemon namespace/openshift-machine-config-operator: Liveness probe status=failure output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" start-of-body= Dec 03 11:56:29 crc kubenswrapper[5052]: I1203 11:56:29.389310 5052 prober.go:107] "Probe failed" probeType="Liveness" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" containerName="machine-config-daemon" probeResult="failure" output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" Dec 03 11:56:29 crc kubenswrapper[5052]: I1203 11:56:29.517433 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/7c36839d-8518-4b0e-bf63-ec94d92e3c13-etc-swift\") pod \"swift-storage-0\" (UID: \"7c36839d-8518-4b0e-bf63-ec94d92e3c13\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 11:56:29 crc kubenswrapper[5052]: E1203 11:56:29.517638 5052 projected.go:288] Couldn't get configMap openstack-kuttl-tests/swift-ring-files: configmap "swift-ring-files" not found Dec 03 11:56:29 crc kubenswrapper[5052]: E1203 11:56:29.517656 5052 projected.go:194] Error preparing data for projected volume etc-swift for pod openstack-kuttl-tests/swift-storage-0: configmap "swift-ring-files" not found Dec 03 11:56:29 crc kubenswrapper[5052]: E1203 11:56:29.517707 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7c36839d-8518-4b0e-bf63-ec94d92e3c13-etc-swift podName:7c36839d-8518-4b0e-bf63-ec94d92e3c13 nodeName:}" failed. No retries permitted until 2025-12-03 11:56:31.517692611 +0000 UTC m=+2182.105934375 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "etc-swift" (UniqueName: "kubernetes.io/projected/7c36839d-8518-4b0e-bf63-ec94d92e3c13-etc-swift") pod "swift-storage-0" (UID: "7c36839d-8518-4b0e-bf63-ec94d92e3c13") : configmap "swift-ring-files" not found Dec 03 11:56:29 crc kubenswrapper[5052]: I1203 11:56:29.538331 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-operators/perses-operator-5446b9c989-gqddj" Dec 03 11:56:29 crc kubenswrapper[5052]: I1203 11:56:29.787902 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ovn-northd-0"] Dec 03 11:56:29 crc kubenswrapper[5052]: I1203 11:56:29.789555 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 11:56:29 crc kubenswrapper[5052]: I1203 11:56:29.798965 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"ovnnorthd-scripts" Dec 03 11:56:29 crc kubenswrapper[5052]: I1203 11:56:29.799054 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-ovnnorthd-ovndbs" Dec 03 11:56:29 crc kubenswrapper[5052]: I1203 11:56:29.799540 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"ovnnorthd-config" Dec 03 11:56:29 crc kubenswrapper[5052]: I1203 11:56:29.799711 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ovnnorthd-ovnnorthd-dockercfg-tb5l5" Dec 03 11:56:29 crc kubenswrapper[5052]: I1203 11:56:29.807573 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ovn-northd-0"] Dec 03 11:56:29 crc kubenswrapper[5052]: I1203 11:56:29.822710 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-rundir\" (UniqueName: \"kubernetes.io/empty-dir/5e5ddf8c-6109-481f-beb0-0802de034912-ovn-rundir\") pod \"ovn-northd-0\" (UID: \"5e5ddf8c-6109-481f-beb0-0802de034912\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 11:56:29 crc kubenswrapper[5052]: I1203 11:56:29.822754 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/5e5ddf8c-6109-481f-beb0-0802de034912-metrics-certs-tls-certs\") pod \"ovn-northd-0\" (UID: \"5e5ddf8c-6109-481f-beb0-0802de034912\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 11:56:29 crc kubenswrapper[5052]: I1203 11:56:29.822780 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-northd-tls-certs\" (UniqueName: \"kubernetes.io/secret/5e5ddf8c-6109-481f-beb0-0802de034912-ovn-northd-tls-certs\") pod \"ovn-northd-0\" (UID: \"5e5ddf8c-6109-481f-beb0-0802de034912\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 11:56:29 crc kubenswrapper[5052]: I1203 11:56:29.822811 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/5e5ddf8c-6109-481f-beb0-0802de034912-scripts\") pod \"ovn-northd-0\" (UID: \"5e5ddf8c-6109-481f-beb0-0802de034912\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 11:56:29 crc kubenswrapper[5052]: I1203 11:56:29.823039 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5e5ddf8c-6109-481f-beb0-0802de034912-config\") pod \"ovn-northd-0\" (UID: \"5e5ddf8c-6109-481f-beb0-0802de034912\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 11:56:29 crc kubenswrapper[5052]: I1203 11:56:29.823116 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5e5ddf8c-6109-481f-beb0-0802de034912-combined-ca-bundle\") pod \"ovn-northd-0\" (UID: \"5e5ddf8c-6109-481f-beb0-0802de034912\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 11:56:29 crc kubenswrapper[5052]: I1203 11:56:29.823149 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2hf5m\" (UniqueName: \"kubernetes.io/projected/5e5ddf8c-6109-481f-beb0-0802de034912-kube-api-access-2hf5m\") pod \"ovn-northd-0\" (UID: \"5e5ddf8c-6109-481f-beb0-0802de034912\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 11:56:29 crc kubenswrapper[5052]: I1203 11:56:29.925224 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovn-rundir\" (UniqueName: \"kubernetes.io/empty-dir/5e5ddf8c-6109-481f-beb0-0802de034912-ovn-rundir\") pod \"ovn-northd-0\" (UID: \"5e5ddf8c-6109-481f-beb0-0802de034912\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 11:56:29 crc kubenswrapper[5052]: I1203 11:56:29.925299 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/5e5ddf8c-6109-481f-beb0-0802de034912-metrics-certs-tls-certs\") pod \"ovn-northd-0\" (UID: \"5e5ddf8c-6109-481f-beb0-0802de034912\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 11:56:29 crc kubenswrapper[5052]: I1203 11:56:29.925339 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovn-northd-tls-certs\" (UniqueName: \"kubernetes.io/secret/5e5ddf8c-6109-481f-beb0-0802de034912-ovn-northd-tls-certs\") pod \"ovn-northd-0\" (UID: \"5e5ddf8c-6109-481f-beb0-0802de034912\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 11:56:29 crc kubenswrapper[5052]: I1203 11:56:29.925390 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/5e5ddf8c-6109-481f-beb0-0802de034912-scripts\") pod \"ovn-northd-0\" (UID: \"5e5ddf8c-6109-481f-beb0-0802de034912\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 11:56:29 crc kubenswrapper[5052]: I1203 11:56:29.925467 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5e5ddf8c-6109-481f-beb0-0802de034912-config\") pod \"ovn-northd-0\" (UID: \"5e5ddf8c-6109-481f-beb0-0802de034912\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 11:56:29 crc kubenswrapper[5052]: I1203 11:56:29.925521 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5e5ddf8c-6109-481f-beb0-0802de034912-combined-ca-bundle\") pod \"ovn-northd-0\" (UID: \"5e5ddf8c-6109-481f-beb0-0802de034912\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 11:56:29 crc kubenswrapper[5052]: I1203 11:56:29.925550 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-2hf5m\" (UniqueName: \"kubernetes.io/projected/5e5ddf8c-6109-481f-beb0-0802de034912-kube-api-access-2hf5m\") pod \"ovn-northd-0\" (UID: \"5e5ddf8c-6109-481f-beb0-0802de034912\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 11:56:29 crc kubenswrapper[5052]: I1203 11:56:29.926104 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovn-rundir\" (UniqueName: \"kubernetes.io/empty-dir/5e5ddf8c-6109-481f-beb0-0802de034912-ovn-rundir\") pod \"ovn-northd-0\" (UID: \"5e5ddf8c-6109-481f-beb0-0802de034912\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 11:56:29 crc kubenswrapper[5052]: I1203 11:56:29.926588 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/5e5ddf8c-6109-481f-beb0-0802de034912-scripts\") pod \"ovn-northd-0\" (UID: \"5e5ddf8c-6109-481f-beb0-0802de034912\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 11:56:29 crc kubenswrapper[5052]: I1203 11:56:29.927017 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5e5ddf8c-6109-481f-beb0-0802de034912-config\") pod \"ovn-northd-0\" (UID: \"5e5ddf8c-6109-481f-beb0-0802de034912\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 11:56:29 crc kubenswrapper[5052]: I1203 11:56:29.931171 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovn-northd-tls-certs\" (UniqueName: \"kubernetes.io/secret/5e5ddf8c-6109-481f-beb0-0802de034912-ovn-northd-tls-certs\") pod \"ovn-northd-0\" (UID: \"5e5ddf8c-6109-481f-beb0-0802de034912\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 11:56:29 crc kubenswrapper[5052]: I1203 11:56:29.932165 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5e5ddf8c-6109-481f-beb0-0802de034912-combined-ca-bundle\") pod \"ovn-northd-0\" (UID: \"5e5ddf8c-6109-481f-beb0-0802de034912\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 11:56:29 crc kubenswrapper[5052]: I1203 11:56:29.948246 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/5e5ddf8c-6109-481f-beb0-0802de034912-metrics-certs-tls-certs\") pod \"ovn-northd-0\" (UID: \"5e5ddf8c-6109-481f-beb0-0802de034912\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 11:56:29 crc kubenswrapper[5052]: I1203 11:56:29.961651 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-2hf5m\" (UniqueName: \"kubernetes.io/projected/5e5ddf8c-6109-481f-beb0-0802de034912-kube-api-access-2hf5m\") pod \"ovn-northd-0\" (UID: \"5e5ddf8c-6109-481f-beb0-0802de034912\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 11:56:30 crc kubenswrapper[5052]: I1203 11:56:30.116842 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 11:56:30 crc kubenswrapper[5052]: I1203 11:56:30.506494 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/swift-ring-rebalance-mbrxc"] Dec 03 11:56:30 crc kubenswrapper[5052]: I1203 11:56:30.692920 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ovn-northd-0"] Dec 03 11:56:30 crc kubenswrapper[5052]: W1203 11:56:30.754528 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb25492d8_45bc_446a_a60c_b4f732484218.slice/crio-1f3690e37d7f7a12050418a9e645f30bee5e3dcc3b42b8ed7615e5c72e0aa105 WatchSource:0}: Error finding container 1f3690e37d7f7a12050418a9e645f30bee5e3dcc3b42b8ed7615e5c72e0aa105: Status 404 returned error can't find the container with id 1f3690e37d7f7a12050418a9e645f30bee5e3dcc3b42b8ed7615e5c72e0aa105 Dec 03 11:56:31 crc kubenswrapper[5052]: I1203 11:56:31.200468 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovn-northd-0" event={"ID":"5e5ddf8c-6109-481f-beb0-0802de034912","Type":"ContainerStarted","Data":"0ad383e8c3943932d2c7d098b04c0f60d89e57f30cd3e7c590b8a688a1852e9d"} Dec 03 11:56:31 crc kubenswrapper[5052]: I1203 11:56:31.219538 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-ring-rebalance-mbrxc" event={"ID":"b25492d8-45bc-446a-a60c-b4f732484218","Type":"ContainerStarted","Data":"1f3690e37d7f7a12050418a9e645f30bee5e3dcc3b42b8ed7615e5c72e0aa105"} Dec 03 11:56:31 crc kubenswrapper[5052]: I1203 11:56:31.604077 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/7c36839d-8518-4b0e-bf63-ec94d92e3c13-etc-swift\") pod \"swift-storage-0\" (UID: \"7c36839d-8518-4b0e-bf63-ec94d92e3c13\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 11:56:31 crc kubenswrapper[5052]: E1203 11:56:31.604319 5052 projected.go:288] Couldn't get configMap openstack-kuttl-tests/swift-ring-files: configmap "swift-ring-files" not found Dec 03 11:56:31 crc kubenswrapper[5052]: E1203 11:56:31.604656 5052 projected.go:194] Error preparing data for projected volume etc-swift for pod openstack-kuttl-tests/swift-storage-0: configmap "swift-ring-files" not found Dec 03 11:56:31 crc kubenswrapper[5052]: E1203 11:56:31.604729 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7c36839d-8518-4b0e-bf63-ec94d92e3c13-etc-swift podName:7c36839d-8518-4b0e-bf63-ec94d92e3c13 nodeName:}" failed. No retries permitted until 2025-12-03 11:56:35.604709789 +0000 UTC m=+2186.192951553 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "etc-swift" (UniqueName: "kubernetes.io/projected/7c36839d-8518-4b0e-bf63-ec94d92e3c13-etc-swift") pod "swift-storage-0" (UID: "7c36839d-8518-4b0e-bf63-ec94d92e3c13") : configmap "swift-ring-files" not found Dec 03 11:56:32 crc kubenswrapper[5052]: I1203 11:56:32.229835 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-ring-rebalance-mbrxc" event={"ID":"b25492d8-45bc-446a-a60c-b4f732484218","Type":"ContainerStarted","Data":"9d6766f724c021f51b09b306536e6d9042806dbfded59d6853ab9da78a74450c"} Dec 03 11:56:32 crc kubenswrapper[5052]: I1203 11:56:32.232607 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovn-northd-0" event={"ID":"5e5ddf8c-6109-481f-beb0-0802de034912","Type":"ContainerStarted","Data":"e4b74aed237af57c4a5ccac244bde0abf28abccef9d3f249bad7df1bbb43da9d"} Dec 03 11:56:32 crc kubenswrapper[5052]: I1203 11:56:32.235678 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/community-operators-tb5qk" event={"ID":"478d0cb6-25de-46d5-979e-9e43bf1b7d4b","Type":"ContainerStarted","Data":"4f161a0e078151ba1bcbddcbdbae8f043fc412323e5f17d9f511a1e706347388"} Dec 03 11:56:32 crc kubenswrapper[5052]: I1203 11:56:32.253465 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/swift-ring-rebalance-mbrxc" podStartSLOduration=4.253438995 podStartE2EDuration="4.253438995s" podCreationTimestamp="2025-12-03 11:56:28 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:56:32.250370767 +0000 UTC m=+2182.838612531" watchObservedRunningTime="2025-12-03 11:56:32.253438995 +0000 UTC m=+2182.841680759" Dec 03 11:56:32 crc kubenswrapper[5052]: I1203 11:56:32.295371 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-marketplace/community-operators-tb5qk" podStartSLOduration=8.261291986 podStartE2EDuration="13.295345164s" podCreationTimestamp="2025-12-03 11:56:19 +0000 UTC" firstStartedPulling="2025-12-03 11:56:26.107698022 +0000 UTC m=+2176.695939786" lastFinishedPulling="2025-12-03 11:56:31.1417512 +0000 UTC m=+2181.729992964" observedRunningTime="2025-12-03 11:56:32.288599981 +0000 UTC m=+2182.876841745" watchObservedRunningTime="2025-12-03 11:56:32.295345164 +0000 UTC m=+2182.883586928" Dec 03 11:56:33 crc kubenswrapper[5052]: I1203 11:56:33.245789 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovn-northd-0" event={"ID":"5e5ddf8c-6109-481f-beb0-0802de034912","Type":"ContainerStarted","Data":"986d340d25b89c2b20e9877dd2590330ca97a1a957d198b877ab542caeca6588"} Dec 03 11:56:33 crc kubenswrapper[5052]: I1203 11:56:33.245850 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 11:56:33 crc kubenswrapper[5052]: I1203 11:56:33.854548 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 11:56:33 crc kubenswrapper[5052]: I1203 11:56:33.855148 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 11:56:34 crc kubenswrapper[5052]: I1203 11:56:34.233492 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 11:56:34 crc kubenswrapper[5052]: I1203 11:56:34.283575 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/prometheus-metric-storage-0" event={"ID":"33918bc1-340a-4d26-a204-001b85c15f04","Type":"ContainerStarted","Data":"414b563368af8ca87e25bd40b323d2ab1710df0f803632e3f17e886ce6740020"} Dec 03 11:56:34 crc kubenswrapper[5052]: I1203 11:56:34.283679 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/ovn-northd-0" podStartSLOduration=5.283657037 podStartE2EDuration="5.283657037s" podCreationTimestamp="2025-12-03 11:56:29 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:56:33.298196244 +0000 UTC m=+2183.886438008" watchObservedRunningTime="2025-12-03 11:56:34.283657037 +0000 UTC m=+2184.871898801" Dec 03 11:56:34 crc kubenswrapper[5052]: I1203 11:56:34.448844 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 11:56:34 crc kubenswrapper[5052]: I1203 11:56:34.690215 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/glance-0d27-account-create-update-9jj4h"] Dec 03 11:56:34 crc kubenswrapper[5052]: I1203 11:56:34.691238 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-0d27-account-create-update-9jj4h" Dec 03 11:56:34 crc kubenswrapper[5052]: I1203 11:56:34.700253 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-db-secret" Dec 03 11:56:34 crc kubenswrapper[5052]: I1203 11:56:34.702254 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-0d27-account-create-update-9jj4h"] Dec 03 11:56:34 crc kubenswrapper[5052]: I1203 11:56:34.786073 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tdbh6\" (UniqueName: \"kubernetes.io/projected/0c055858-1ccd-444e-a055-1bd2a919556f-kube-api-access-tdbh6\") pod \"glance-0d27-account-create-update-9jj4h\" (UID: \"0c055858-1ccd-444e-a055-1bd2a919556f\") " pod="openstack-kuttl-tests/glance-0d27-account-create-update-9jj4h" Dec 03 11:56:34 crc kubenswrapper[5052]: I1203 11:56:34.786241 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/0c055858-1ccd-444e-a055-1bd2a919556f-operator-scripts\") pod \"glance-0d27-account-create-update-9jj4h\" (UID: \"0c055858-1ccd-444e-a055-1bd2a919556f\") " pod="openstack-kuttl-tests/glance-0d27-account-create-update-9jj4h" Dec 03 11:56:34 crc kubenswrapper[5052]: I1203 11:56:34.888164 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-tdbh6\" (UniqueName: \"kubernetes.io/projected/0c055858-1ccd-444e-a055-1bd2a919556f-kube-api-access-tdbh6\") pod \"glance-0d27-account-create-update-9jj4h\" (UID: \"0c055858-1ccd-444e-a055-1bd2a919556f\") " pod="openstack-kuttl-tests/glance-0d27-account-create-update-9jj4h" Dec 03 11:56:34 crc kubenswrapper[5052]: I1203 11:56:34.888681 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/0c055858-1ccd-444e-a055-1bd2a919556f-operator-scripts\") pod \"glance-0d27-account-create-update-9jj4h\" (UID: \"0c055858-1ccd-444e-a055-1bd2a919556f\") " pod="openstack-kuttl-tests/glance-0d27-account-create-update-9jj4h" Dec 03 11:56:34 crc kubenswrapper[5052]: I1203 11:56:34.889463 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/0c055858-1ccd-444e-a055-1bd2a919556f-operator-scripts\") pod \"glance-0d27-account-create-update-9jj4h\" (UID: \"0c055858-1ccd-444e-a055-1bd2a919556f\") " pod="openstack-kuttl-tests/glance-0d27-account-create-update-9jj4h" Dec 03 11:56:34 crc kubenswrapper[5052]: I1203 11:56:34.906574 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-tdbh6\" (UniqueName: \"kubernetes.io/projected/0c055858-1ccd-444e-a055-1bd2a919556f-kube-api-access-tdbh6\") pod \"glance-0d27-account-create-update-9jj4h\" (UID: \"0c055858-1ccd-444e-a055-1bd2a919556f\") " pod="openstack-kuttl-tests/glance-0d27-account-create-update-9jj4h" Dec 03 11:56:35 crc kubenswrapper[5052]: I1203 11:56:35.021467 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-0d27-account-create-update-9jj4h" Dec 03 11:56:35 crc kubenswrapper[5052]: I1203 11:56:35.300174 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/alertmanager-metric-storage-0" event={"ID":"7cbbb6c5-a087-400f-9beb-e740ae1ddfbd","Type":"ContainerStarted","Data":"daf20541a4af7f41da95e3513d45e8f6425015208b249e68097ecaea69b47670"} Dec 03 11:56:35 crc kubenswrapper[5052]: I1203 11:56:35.551345 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-0d27-account-create-update-9jj4h"] Dec 03 11:56:35 crc kubenswrapper[5052]: I1203 11:56:35.612979 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/7c36839d-8518-4b0e-bf63-ec94d92e3c13-etc-swift\") pod \"swift-storage-0\" (UID: \"7c36839d-8518-4b0e-bf63-ec94d92e3c13\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 11:56:35 crc kubenswrapper[5052]: E1203 11:56:35.613154 5052 projected.go:288] Couldn't get configMap openstack-kuttl-tests/swift-ring-files: configmap "swift-ring-files" not found Dec 03 11:56:35 crc kubenswrapper[5052]: E1203 11:56:35.613444 5052 projected.go:194] Error preparing data for projected volume etc-swift for pod openstack-kuttl-tests/swift-storage-0: configmap "swift-ring-files" not found Dec 03 11:56:35 crc kubenswrapper[5052]: E1203 11:56:35.613523 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7c36839d-8518-4b0e-bf63-ec94d92e3c13-etc-swift podName:7c36839d-8518-4b0e-bf63-ec94d92e3c13 nodeName:}" failed. No retries permitted until 2025-12-03 11:56:43.613483045 +0000 UTC m=+2194.201724809 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "etc-swift" (UniqueName: "kubernetes.io/projected/7c36839d-8518-4b0e-bf63-ec94d92e3c13-etc-swift") pod "swift-storage-0" (UID: "7c36839d-8518-4b0e-bf63-ec94d92e3c13") : configmap "swift-ring-files" not found Dec 03 11:56:35 crc kubenswrapper[5052]: I1203 11:56:35.701551 5052 scope.go:117] "RemoveContainer" containerID="845fffc100a142a0e49faa1925c6d2a46f3a8982adbaa217c0cfa62f30ea7ce7" Dec 03 11:56:35 crc kubenswrapper[5052]: I1203 11:56:35.748797 5052 scope.go:117] "RemoveContainer" containerID="ec1fb887baa43947f24b5957fd0b17eec560f3a84fa381dab76beddac96dcb20" Dec 03 11:56:35 crc kubenswrapper[5052]: I1203 11:56:35.778039 5052 scope.go:117] "RemoveContainer" containerID="8664cd2b37abb007f9897daeb6f6c5de1533943a513fca1505b8bf8ee642c203" Dec 03 11:56:35 crc kubenswrapper[5052]: I1203 11:56:35.805208 5052 scope.go:117] "RemoveContainer" containerID="8393f7f6989898dc3fee2162e87f42b6979e4c1290368f41d1aa48d371d31314" Dec 03 11:56:35 crc kubenswrapper[5052]: I1203 11:56:35.833965 5052 scope.go:117] "RemoveContainer" containerID="57ea218fb8f0a0038fe40dcb1ad5997273497a2c37f8b30405d5fc310990e84b" Dec 03 11:56:35 crc kubenswrapper[5052]: I1203 11:56:35.864090 5052 scope.go:117] "RemoveContainer" containerID="f66d133226f04488b65271e680367fcbcbdfdc8cbbafb3a1d5494736b7c5cfb3" Dec 03 11:56:35 crc kubenswrapper[5052]: I1203 11:56:35.889154 5052 scope.go:117] "RemoveContainer" containerID="24721104cb9088d6a7d4f0ae127c54a770da6fbf71ba161bc9d6020ff94a6ce7" Dec 03 11:56:35 crc kubenswrapper[5052]: I1203 11:56:35.961683 5052 scope.go:117] "RemoveContainer" containerID="87e11068f85c53dfc10a1bce0f37811f9e183ed6ffd052218a764a795d3ec700" Dec 03 11:56:36 crc kubenswrapper[5052]: I1203 11:56:36.008617 5052 scope.go:117] "RemoveContainer" containerID="ad2e20f86ff81ee90b61200f14867c2ba74e9e0bbf7e5ed7916c06cfc4183fad" Dec 03 11:56:36 crc kubenswrapper[5052]: I1203 11:56:36.056483 5052 scope.go:117] "RemoveContainer" containerID="aaa2d28c5e3338722d4f5eaedeeb8a30c10f86ffe40a62d737b912ad57d3b9b1" Dec 03 11:56:36 crc kubenswrapper[5052]: I1203 11:56:36.091449 5052 scope.go:117] "RemoveContainer" containerID="2d4901c4da8c5451102183e62f966e62a2dd0b19c0f466d04d0154b54dcb5f2d" Dec 03 11:56:36 crc kubenswrapper[5052]: I1203 11:56:36.110497 5052 scope.go:117] "RemoveContainer" containerID="97f15cadb1a96ebd3eddcbe674bed92f481485532cc8ed2bb6b1191062dafefa" Dec 03 11:56:36 crc kubenswrapper[5052]: I1203 11:56:36.170435 5052 scope.go:117] "RemoveContainer" containerID="5941b7d31e38a458b2b2f04a347d19ebe8ee3aa64cc1993b16459efd04d966e0" Dec 03 11:56:36 crc kubenswrapper[5052]: I1203 11:56:36.195074 5052 scope.go:117] "RemoveContainer" containerID="4ca16aced64bf17827b7e9533988c8f6aa76a1efc00408bc2b421323bbd6026b" Dec 03 11:56:36 crc kubenswrapper[5052]: I1203 11:56:36.318458 5052 generic.go:334] "Generic (PLEG): container finished" podID="0c055858-1ccd-444e-a055-1bd2a919556f" containerID="9c3e540a901f3f23370d4b704b73c51ab43a8999c92566dd189c3398da965634" exitCode=0 Dec 03 11:56:36 crc kubenswrapper[5052]: I1203 11:56:36.318536 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-0d27-account-create-update-9jj4h" event={"ID":"0c055858-1ccd-444e-a055-1bd2a919556f","Type":"ContainerDied","Data":"9c3e540a901f3f23370d4b704b73c51ab43a8999c92566dd189c3398da965634"} Dec 03 11:56:36 crc kubenswrapper[5052]: I1203 11:56:36.318568 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-0d27-account-create-update-9jj4h" event={"ID":"0c055858-1ccd-444e-a055-1bd2a919556f","Type":"ContainerStarted","Data":"f045d9727012c4d56dbbb5e8163932ca5382a57e05fcd9876694a8fee24a85a7"} Dec 03 11:56:36 crc kubenswrapper[5052]: I1203 11:56:36.608017 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/watcher-db-create-sr6dr"] Dec 03 11:56:36 crc kubenswrapper[5052]: I1203 11:56:36.609405 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/watcher-db-create-sr6dr" Dec 03 11:56:36 crc kubenswrapper[5052]: I1203 11:56:36.620764 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/watcher-db-create-sr6dr"] Dec 03 11:56:36 crc kubenswrapper[5052]: I1203 11:56:36.632682 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/87e193c2-c4a3-4370-b683-b151324dbcec-operator-scripts\") pod \"watcher-db-create-sr6dr\" (UID: \"87e193c2-c4a3-4370-b683-b151324dbcec\") " pod="openstack-kuttl-tests/watcher-db-create-sr6dr" Dec 03 11:56:36 crc kubenswrapper[5052]: I1203 11:56:36.632895 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kqm45\" (UniqueName: \"kubernetes.io/projected/87e193c2-c4a3-4370-b683-b151324dbcec-kube-api-access-kqm45\") pod \"watcher-db-create-sr6dr\" (UID: \"87e193c2-c4a3-4370-b683-b151324dbcec\") " pod="openstack-kuttl-tests/watcher-db-create-sr6dr" Dec 03 11:56:36 crc kubenswrapper[5052]: I1203 11:56:36.704045 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 11:56:36 crc kubenswrapper[5052]: I1203 11:56:36.735606 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/watcher-36ab-account-create-update-spl42"] Dec 03 11:56:36 crc kubenswrapper[5052]: I1203 11:56:36.736879 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/87e193c2-c4a3-4370-b683-b151324dbcec-operator-scripts\") pod \"watcher-db-create-sr6dr\" (UID: \"87e193c2-c4a3-4370-b683-b151324dbcec\") " pod="openstack-kuttl-tests/watcher-db-create-sr6dr" Dec 03 11:56:36 crc kubenswrapper[5052]: I1203 11:56:36.736965 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-kqm45\" (UniqueName: \"kubernetes.io/projected/87e193c2-c4a3-4370-b683-b151324dbcec-kube-api-access-kqm45\") pod \"watcher-db-create-sr6dr\" (UID: \"87e193c2-c4a3-4370-b683-b151324dbcec\") " pod="openstack-kuttl-tests/watcher-db-create-sr6dr" Dec 03 11:56:36 crc kubenswrapper[5052]: I1203 11:56:36.738144 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/87e193c2-c4a3-4370-b683-b151324dbcec-operator-scripts\") pod \"watcher-db-create-sr6dr\" (UID: \"87e193c2-c4a3-4370-b683-b151324dbcec\") " pod="openstack-kuttl-tests/watcher-db-create-sr6dr" Dec 03 11:56:36 crc kubenswrapper[5052]: I1203 11:56:36.743981 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/watcher-36ab-account-create-update-spl42" Dec 03 11:56:36 crc kubenswrapper[5052]: I1203 11:56:36.747368 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"watcher-db-secret" Dec 03 11:56:36 crc kubenswrapper[5052]: I1203 11:56:36.764991 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-kqm45\" (UniqueName: \"kubernetes.io/projected/87e193c2-c4a3-4370-b683-b151324dbcec-kube-api-access-kqm45\") pod \"watcher-db-create-sr6dr\" (UID: \"87e193c2-c4a3-4370-b683-b151324dbcec\") " pod="openstack-kuttl-tests/watcher-db-create-sr6dr" Dec 03 11:56:36 crc kubenswrapper[5052]: I1203 11:56:36.765070 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/watcher-36ab-account-create-update-spl42"] Dec 03 11:56:36 crc kubenswrapper[5052]: I1203 11:56:36.838786 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/f0bbecd8-194a-4067-8786-1da410979cbf-operator-scripts\") pod \"watcher-36ab-account-create-update-spl42\" (UID: \"f0bbecd8-194a-4067-8786-1da410979cbf\") " pod="openstack-kuttl-tests/watcher-36ab-account-create-update-spl42" Dec 03 11:56:36 crc kubenswrapper[5052]: I1203 11:56:36.838835 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g4g9k\" (UniqueName: \"kubernetes.io/projected/f0bbecd8-194a-4067-8786-1da410979cbf-kube-api-access-g4g9k\") pod \"watcher-36ab-account-create-update-spl42\" (UID: \"f0bbecd8-194a-4067-8786-1da410979cbf\") " pod="openstack-kuttl-tests/watcher-36ab-account-create-update-spl42" Dec 03 11:56:36 crc kubenswrapper[5052]: I1203 11:56:36.928948 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/watcher-db-create-sr6dr" Dec 03 11:56:36 crc kubenswrapper[5052]: I1203 11:56:36.940593 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/f0bbecd8-194a-4067-8786-1da410979cbf-operator-scripts\") pod \"watcher-36ab-account-create-update-spl42\" (UID: \"f0bbecd8-194a-4067-8786-1da410979cbf\") " pod="openstack-kuttl-tests/watcher-36ab-account-create-update-spl42" Dec 03 11:56:36 crc kubenswrapper[5052]: I1203 11:56:36.940645 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-g4g9k\" (UniqueName: \"kubernetes.io/projected/f0bbecd8-194a-4067-8786-1da410979cbf-kube-api-access-g4g9k\") pod \"watcher-36ab-account-create-update-spl42\" (UID: \"f0bbecd8-194a-4067-8786-1da410979cbf\") " pod="openstack-kuttl-tests/watcher-36ab-account-create-update-spl42" Dec 03 11:56:36 crc kubenswrapper[5052]: I1203 11:56:36.941821 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/f0bbecd8-194a-4067-8786-1da410979cbf-operator-scripts\") pod \"watcher-36ab-account-create-update-spl42\" (UID: \"f0bbecd8-194a-4067-8786-1da410979cbf\") " pod="openstack-kuttl-tests/watcher-36ab-account-create-update-spl42" Dec 03 11:56:36 crc kubenswrapper[5052]: I1203 11:56:36.960075 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-g4g9k\" (UniqueName: \"kubernetes.io/projected/f0bbecd8-194a-4067-8786-1da410979cbf-kube-api-access-g4g9k\") pod \"watcher-36ab-account-create-update-spl42\" (UID: \"f0bbecd8-194a-4067-8786-1da410979cbf\") " pod="openstack-kuttl-tests/watcher-36ab-account-create-update-spl42" Dec 03 11:56:37 crc kubenswrapper[5052]: I1203 11:56:37.073110 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/watcher-36ab-account-create-update-spl42" Dec 03 11:56:37 crc kubenswrapper[5052]: I1203 11:56:37.455856 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/watcher-db-create-sr6dr"] Dec 03 11:56:37 crc kubenswrapper[5052]: I1203 11:56:37.589683 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/watcher-36ab-account-create-update-spl42"] Dec 03 11:56:37 crc kubenswrapper[5052]: I1203 11:56:37.707953 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-0d27-account-create-update-9jj4h" Dec 03 11:56:37 crc kubenswrapper[5052]: I1203 11:56:37.767489 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tdbh6\" (UniqueName: \"kubernetes.io/projected/0c055858-1ccd-444e-a055-1bd2a919556f-kube-api-access-tdbh6\") pod \"0c055858-1ccd-444e-a055-1bd2a919556f\" (UID: \"0c055858-1ccd-444e-a055-1bd2a919556f\") " Dec 03 11:56:37 crc kubenswrapper[5052]: I1203 11:56:37.767680 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/0c055858-1ccd-444e-a055-1bd2a919556f-operator-scripts\") pod \"0c055858-1ccd-444e-a055-1bd2a919556f\" (UID: \"0c055858-1ccd-444e-a055-1bd2a919556f\") " Dec 03 11:56:37 crc kubenswrapper[5052]: I1203 11:56:37.769409 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0c055858-1ccd-444e-a055-1bd2a919556f-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "0c055858-1ccd-444e-a055-1bd2a919556f" (UID: "0c055858-1ccd-444e-a055-1bd2a919556f"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:56:37 crc kubenswrapper[5052]: I1203 11:56:37.775457 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0c055858-1ccd-444e-a055-1bd2a919556f-kube-api-access-tdbh6" (OuterVolumeSpecName: "kube-api-access-tdbh6") pod "0c055858-1ccd-444e-a055-1bd2a919556f" (UID: "0c055858-1ccd-444e-a055-1bd2a919556f"). InnerVolumeSpecName "kube-api-access-tdbh6". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:56:37 crc kubenswrapper[5052]: I1203 11:56:37.869181 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-tdbh6\" (UniqueName: \"kubernetes.io/projected/0c055858-1ccd-444e-a055-1bd2a919556f-kube-api-access-tdbh6\") on node \"crc\" DevicePath \"\"" Dec 03 11:56:37 crc kubenswrapper[5052]: I1203 11:56:37.869212 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/0c055858-1ccd-444e-a055-1bd2a919556f-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:56:38 crc kubenswrapper[5052]: I1203 11:56:38.363584 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-0d27-account-create-update-9jj4h" event={"ID":"0c055858-1ccd-444e-a055-1bd2a919556f","Type":"ContainerDied","Data":"f045d9727012c4d56dbbb5e8163932ca5382a57e05fcd9876694a8fee24a85a7"} Dec 03 11:56:38 crc kubenswrapper[5052]: I1203 11:56:38.363633 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f045d9727012c4d56dbbb5e8163932ca5382a57e05fcd9876694a8fee24a85a7" Dec 03 11:56:38 crc kubenswrapper[5052]: I1203 11:56:38.363610 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-0d27-account-create-update-9jj4h" Dec 03 11:56:38 crc kubenswrapper[5052]: I1203 11:56:38.365947 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/watcher-36ab-account-create-update-spl42" event={"ID":"f0bbecd8-194a-4067-8786-1da410979cbf","Type":"ContainerStarted","Data":"0a02427bd50c50cbb993f7fe4b5961d674d53b51ca7dbe5de9fca001e0d11572"} Dec 03 11:56:38 crc kubenswrapper[5052]: I1203 11:56:38.365994 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/watcher-36ab-account-create-update-spl42" event={"ID":"f0bbecd8-194a-4067-8786-1da410979cbf","Type":"ContainerStarted","Data":"9a22f0ed63a534e068c03f83b60c9de6cfc7a00450e0ba3fb23f40cd1af55cfd"} Dec 03 11:56:38 crc kubenswrapper[5052]: I1203 11:56:38.368209 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/watcher-db-create-sr6dr" event={"ID":"87e193c2-c4a3-4370-b683-b151324dbcec","Type":"ContainerStarted","Data":"4fae91e98a69ef557495570d6dcbe9cef23173f11867aae8ecb37fecd4959d8b"} Dec 03 11:56:38 crc kubenswrapper[5052]: I1203 11:56:38.368305 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/watcher-db-create-sr6dr" event={"ID":"87e193c2-c4a3-4370-b683-b151324dbcec","Type":"ContainerStarted","Data":"e0babef662057c18e4e8cc71ee06deef8f95d2e28f8732d47425e24c3c12ec20"} Dec 03 11:56:38 crc kubenswrapper[5052]: I1203 11:56:38.395954 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/watcher-36ab-account-create-update-spl42" podStartSLOduration=2.395927904 podStartE2EDuration="2.395927904s" podCreationTimestamp="2025-12-03 11:56:36 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:56:38.385911798 +0000 UTC m=+2188.974153562" watchObservedRunningTime="2025-12-03 11:56:38.395927904 +0000 UTC m=+2188.984169668" Dec 03 11:56:38 crc kubenswrapper[5052]: I1203 11:56:38.411811 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/watcher-db-create-sr6dr" podStartSLOduration=2.411794368 podStartE2EDuration="2.411794368s" podCreationTimestamp="2025-12-03 11:56:36 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:56:38.407701561 +0000 UTC m=+2188.995943325" watchObservedRunningTime="2025-12-03 11:56:38.411794368 +0000 UTC m=+2189.000036132" Dec 03 11:56:39 crc kubenswrapper[5052]: I1203 11:56:39.408129 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-marketplace/community-operators-tb5qk" Dec 03 11:56:39 crc kubenswrapper[5052]: I1203 11:56:39.408545 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openshift-marketplace/community-operators-tb5qk" Dec 03 11:56:39 crc kubenswrapper[5052]: I1203 11:56:39.454542 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-marketplace/community-operators-tb5qk" Dec 03 11:56:40 crc kubenswrapper[5052]: I1203 11:56:40.409290 5052 generic.go:334] "Generic (PLEG): container finished" podID="f0bbecd8-194a-4067-8786-1da410979cbf" containerID="0a02427bd50c50cbb993f7fe4b5961d674d53b51ca7dbe5de9fca001e0d11572" exitCode=0 Dec 03 11:56:40 crc kubenswrapper[5052]: I1203 11:56:40.409762 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/watcher-36ab-account-create-update-spl42" event={"ID":"f0bbecd8-194a-4067-8786-1da410979cbf","Type":"ContainerDied","Data":"0a02427bd50c50cbb993f7fe4b5961d674d53b51ca7dbe5de9fca001e0d11572"} Dec 03 11:56:40 crc kubenswrapper[5052]: I1203 11:56:40.411535 5052 generic.go:334] "Generic (PLEG): container finished" podID="87e193c2-c4a3-4370-b683-b151324dbcec" containerID="4fae91e98a69ef557495570d6dcbe9cef23173f11867aae8ecb37fecd4959d8b" exitCode=0 Dec 03 11:56:40 crc kubenswrapper[5052]: I1203 11:56:40.411583 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/watcher-db-create-sr6dr" event={"ID":"87e193c2-c4a3-4370-b683-b151324dbcec","Type":"ContainerDied","Data":"4fae91e98a69ef557495570d6dcbe9cef23173f11867aae8ecb37fecd4959d8b"} Dec 03 11:56:40 crc kubenswrapper[5052]: I1203 11:56:40.412764 5052 generic.go:334] "Generic (PLEG): container finished" podID="33918bc1-340a-4d26-a204-001b85c15f04" containerID="414b563368af8ca87e25bd40b323d2ab1710df0f803632e3f17e886ce6740020" exitCode=0 Dec 03 11:56:40 crc kubenswrapper[5052]: I1203 11:56:40.413094 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/prometheus-metric-storage-0" event={"ID":"33918bc1-340a-4d26-a204-001b85c15f04","Type":"ContainerDied","Data":"414b563368af8ca87e25bd40b323d2ab1710df0f803632e3f17e886ce6740020"} Dec 03 11:56:40 crc kubenswrapper[5052]: I1203 11:56:40.465076 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-marketplace/community-operators-tb5qk" Dec 03 11:56:41 crc kubenswrapper[5052]: I1203 11:56:41.434602 5052 generic.go:334] "Generic (PLEG): container finished" podID="7cbbb6c5-a087-400f-9beb-e740ae1ddfbd" containerID="daf20541a4af7f41da95e3513d45e8f6425015208b249e68097ecaea69b47670" exitCode=0 Dec 03 11:56:41 crc kubenswrapper[5052]: I1203 11:56:41.434694 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/alertmanager-metric-storage-0" event={"ID":"7cbbb6c5-a087-400f-9beb-e740ae1ddfbd","Type":"ContainerDied","Data":"daf20541a4af7f41da95e3513d45e8f6425015208b249e68097ecaea69b47670"} Dec 03 11:56:41 crc kubenswrapper[5052]: I1203 11:56:41.874793 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openshift-marketplace/community-operators-tb5qk"] Dec 03 11:56:41 crc kubenswrapper[5052]: I1203 11:56:41.889455 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/watcher-db-create-sr6dr" Dec 03 11:56:41 crc kubenswrapper[5052]: I1203 11:56:41.893779 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/watcher-36ab-account-create-update-spl42" Dec 03 11:56:42 crc kubenswrapper[5052]: I1203 11:56:42.038183 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kqm45\" (UniqueName: \"kubernetes.io/projected/87e193c2-c4a3-4370-b683-b151324dbcec-kube-api-access-kqm45\") pod \"87e193c2-c4a3-4370-b683-b151324dbcec\" (UID: \"87e193c2-c4a3-4370-b683-b151324dbcec\") " Dec 03 11:56:42 crc kubenswrapper[5052]: I1203 11:56:42.038240 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/87e193c2-c4a3-4370-b683-b151324dbcec-operator-scripts\") pod \"87e193c2-c4a3-4370-b683-b151324dbcec\" (UID: \"87e193c2-c4a3-4370-b683-b151324dbcec\") " Dec 03 11:56:42 crc kubenswrapper[5052]: I1203 11:56:42.038263 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g4g9k\" (UniqueName: \"kubernetes.io/projected/f0bbecd8-194a-4067-8786-1da410979cbf-kube-api-access-g4g9k\") pod \"f0bbecd8-194a-4067-8786-1da410979cbf\" (UID: \"f0bbecd8-194a-4067-8786-1da410979cbf\") " Dec 03 11:56:42 crc kubenswrapper[5052]: I1203 11:56:42.038301 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/f0bbecd8-194a-4067-8786-1da410979cbf-operator-scripts\") pod \"f0bbecd8-194a-4067-8786-1da410979cbf\" (UID: \"f0bbecd8-194a-4067-8786-1da410979cbf\") " Dec 03 11:56:42 crc kubenswrapper[5052]: I1203 11:56:42.039087 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f0bbecd8-194a-4067-8786-1da410979cbf-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "f0bbecd8-194a-4067-8786-1da410979cbf" (UID: "f0bbecd8-194a-4067-8786-1da410979cbf"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:56:42 crc kubenswrapper[5052]: I1203 11:56:42.040121 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/87e193c2-c4a3-4370-b683-b151324dbcec-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "87e193c2-c4a3-4370-b683-b151324dbcec" (UID: "87e193c2-c4a3-4370-b683-b151324dbcec"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:56:42 crc kubenswrapper[5052]: I1203 11:56:42.044092 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f0bbecd8-194a-4067-8786-1da410979cbf-kube-api-access-g4g9k" (OuterVolumeSpecName: "kube-api-access-g4g9k") pod "f0bbecd8-194a-4067-8786-1da410979cbf" (UID: "f0bbecd8-194a-4067-8786-1da410979cbf"). InnerVolumeSpecName "kube-api-access-g4g9k". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:56:42 crc kubenswrapper[5052]: I1203 11:56:42.045822 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/87e193c2-c4a3-4370-b683-b151324dbcec-kube-api-access-kqm45" (OuterVolumeSpecName: "kube-api-access-kqm45") pod "87e193c2-c4a3-4370-b683-b151324dbcec" (UID: "87e193c2-c4a3-4370-b683-b151324dbcec"). InnerVolumeSpecName "kube-api-access-kqm45". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:56:42 crc kubenswrapper[5052]: I1203 11:56:42.139945 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-kqm45\" (UniqueName: \"kubernetes.io/projected/87e193c2-c4a3-4370-b683-b151324dbcec-kube-api-access-kqm45\") on node \"crc\" DevicePath \"\"" Dec 03 11:56:42 crc kubenswrapper[5052]: I1203 11:56:42.139980 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/87e193c2-c4a3-4370-b683-b151324dbcec-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:56:42 crc kubenswrapper[5052]: I1203 11:56:42.139990 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-g4g9k\" (UniqueName: \"kubernetes.io/projected/f0bbecd8-194a-4067-8786-1da410979cbf-kube-api-access-g4g9k\") on node \"crc\" DevicePath \"\"" Dec 03 11:56:42 crc kubenswrapper[5052]: I1203 11:56:42.140000 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/f0bbecd8-194a-4067-8786-1da410979cbf-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:56:42 crc kubenswrapper[5052]: I1203 11:56:42.451683 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/watcher-36ab-account-create-update-spl42" event={"ID":"f0bbecd8-194a-4067-8786-1da410979cbf","Type":"ContainerDied","Data":"9a22f0ed63a534e068c03f83b60c9de6cfc7a00450e0ba3fb23f40cd1af55cfd"} Dec 03 11:56:42 crc kubenswrapper[5052]: I1203 11:56:42.452009 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9a22f0ed63a534e068c03f83b60c9de6cfc7a00450e0ba3fb23f40cd1af55cfd" Dec 03 11:56:42 crc kubenswrapper[5052]: I1203 11:56:42.451721 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/watcher-36ab-account-create-update-spl42" Dec 03 11:56:42 crc kubenswrapper[5052]: I1203 11:56:42.466170 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/watcher-db-create-sr6dr" Dec 03 11:56:42 crc kubenswrapper[5052]: I1203 11:56:42.466217 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/watcher-db-create-sr6dr" event={"ID":"87e193c2-c4a3-4370-b683-b151324dbcec","Type":"ContainerDied","Data":"e0babef662057c18e4e8cc71ee06deef8f95d2e28f8732d47425e24c3c12ec20"} Dec 03 11:56:42 crc kubenswrapper[5052]: I1203 11:56:42.466250 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e0babef662057c18e4e8cc71ee06deef8f95d2e28f8732d47425e24c3c12ec20" Dec 03 11:56:42 crc kubenswrapper[5052]: I1203 11:56:42.466324 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openshift-marketplace/community-operators-tb5qk" podUID="478d0cb6-25de-46d5-979e-9e43bf1b7d4b" containerName="registry-server" containerID="cri-o://4f161a0e078151ba1bcbddcbdbae8f043fc412323e5f17d9f511a1e706347388" gracePeriod=2 Dec 03 11:56:42 crc kubenswrapper[5052]: E1203 11:56:42.756422 5052 kubelet_node_status.go:756] "Failed to set some node status fields" err="failed to validate nodeIP: route ip+net: no such network interface" node="crc" Dec 03 11:56:42 crc kubenswrapper[5052]: I1203 11:56:42.947349 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/community-operators-tb5qk" Dec 03 11:56:43 crc kubenswrapper[5052]: I1203 11:56:43.053018 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t5rrr\" (UniqueName: \"kubernetes.io/projected/478d0cb6-25de-46d5-979e-9e43bf1b7d4b-kube-api-access-t5rrr\") pod \"478d0cb6-25de-46d5-979e-9e43bf1b7d4b\" (UID: \"478d0cb6-25de-46d5-979e-9e43bf1b7d4b\") " Dec 03 11:56:43 crc kubenswrapper[5052]: I1203 11:56:43.053069 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/478d0cb6-25de-46d5-979e-9e43bf1b7d4b-catalog-content\") pod \"478d0cb6-25de-46d5-979e-9e43bf1b7d4b\" (UID: \"478d0cb6-25de-46d5-979e-9e43bf1b7d4b\") " Dec 03 11:56:43 crc kubenswrapper[5052]: I1203 11:56:43.053122 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/478d0cb6-25de-46d5-979e-9e43bf1b7d4b-utilities\") pod \"478d0cb6-25de-46d5-979e-9e43bf1b7d4b\" (UID: \"478d0cb6-25de-46d5-979e-9e43bf1b7d4b\") " Dec 03 11:56:43 crc kubenswrapper[5052]: I1203 11:56:43.055253 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/478d0cb6-25de-46d5-979e-9e43bf1b7d4b-utilities" (OuterVolumeSpecName: "utilities") pod "478d0cb6-25de-46d5-979e-9e43bf1b7d4b" (UID: "478d0cb6-25de-46d5-979e-9e43bf1b7d4b"). InnerVolumeSpecName "utilities". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:56:43 crc kubenswrapper[5052]: I1203 11:56:43.065760 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/478d0cb6-25de-46d5-979e-9e43bf1b7d4b-kube-api-access-t5rrr" (OuterVolumeSpecName: "kube-api-access-t5rrr") pod "478d0cb6-25de-46d5-979e-9e43bf1b7d4b" (UID: "478d0cb6-25de-46d5-979e-9e43bf1b7d4b"). InnerVolumeSpecName "kube-api-access-t5rrr". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:56:43 crc kubenswrapper[5052]: I1203 11:56:43.121003 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/478d0cb6-25de-46d5-979e-9e43bf1b7d4b-catalog-content" (OuterVolumeSpecName: "catalog-content") pod "478d0cb6-25de-46d5-979e-9e43bf1b7d4b" (UID: "478d0cb6-25de-46d5-979e-9e43bf1b7d4b"). InnerVolumeSpecName "catalog-content". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:56:43 crc kubenswrapper[5052]: I1203 11:56:43.156490 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-t5rrr\" (UniqueName: \"kubernetes.io/projected/478d0cb6-25de-46d5-979e-9e43bf1b7d4b-kube-api-access-t5rrr\") on node \"crc\" DevicePath \"\"" Dec 03 11:56:43 crc kubenswrapper[5052]: I1203 11:56:43.156562 5052 reconciler_common.go:293] "Volume detached for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/478d0cb6-25de-46d5-979e-9e43bf1b7d4b-catalog-content\") on node \"crc\" DevicePath \"\"" Dec 03 11:56:43 crc kubenswrapper[5052]: I1203 11:56:43.156576 5052 reconciler_common.go:293] "Volume detached for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/478d0cb6-25de-46d5-979e-9e43bf1b7d4b-utilities\") on node \"crc\" DevicePath \"\"" Dec 03 11:56:43 crc kubenswrapper[5052]: I1203 11:56:43.481797 5052 generic.go:334] "Generic (PLEG): container finished" podID="b25492d8-45bc-446a-a60c-b4f732484218" containerID="9d6766f724c021f51b09b306536e6d9042806dbfded59d6853ab9da78a74450c" exitCode=0 Dec 03 11:56:43 crc kubenswrapper[5052]: I1203 11:56:43.481892 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-ring-rebalance-mbrxc" event={"ID":"b25492d8-45bc-446a-a60c-b4f732484218","Type":"ContainerDied","Data":"9d6766f724c021f51b09b306536e6d9042806dbfded59d6853ab9da78a74450c"} Dec 03 11:56:43 crc kubenswrapper[5052]: I1203 11:56:43.495634 5052 generic.go:334] "Generic (PLEG): container finished" podID="478d0cb6-25de-46d5-979e-9e43bf1b7d4b" containerID="4f161a0e078151ba1bcbddcbdbae8f043fc412323e5f17d9f511a1e706347388" exitCode=0 Dec 03 11:56:43 crc kubenswrapper[5052]: I1203 11:56:43.495679 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/community-operators-tb5qk" event={"ID":"478d0cb6-25de-46d5-979e-9e43bf1b7d4b","Type":"ContainerDied","Data":"4f161a0e078151ba1bcbddcbdbae8f043fc412323e5f17d9f511a1e706347388"} Dec 03 11:56:43 crc kubenswrapper[5052]: I1203 11:56:43.495711 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/community-operators-tb5qk" event={"ID":"478d0cb6-25de-46d5-979e-9e43bf1b7d4b","Type":"ContainerDied","Data":"8fb48f8c4cc76d9a046b2c73e73b841bd240767f9507449785c4e91d488f72ae"} Dec 03 11:56:43 crc kubenswrapper[5052]: I1203 11:56:43.495731 5052 scope.go:117] "RemoveContainer" containerID="4f161a0e078151ba1bcbddcbdbae8f043fc412323e5f17d9f511a1e706347388" Dec 03 11:56:43 crc kubenswrapper[5052]: I1203 11:56:43.495770 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/community-operators-tb5qk" Dec 03 11:56:43 crc kubenswrapper[5052]: I1203 11:56:43.548593 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openshift-marketplace/community-operators-tb5qk"] Dec 03 11:56:43 crc kubenswrapper[5052]: I1203 11:56:43.559705 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openshift-marketplace/community-operators-tb5qk"] Dec 03 11:56:43 crc kubenswrapper[5052]: I1203 11:56:43.670650 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/7c36839d-8518-4b0e-bf63-ec94d92e3c13-etc-swift\") pod \"swift-storage-0\" (UID: \"7c36839d-8518-4b0e-bf63-ec94d92e3c13\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 11:56:43 crc kubenswrapper[5052]: I1203 11:56:43.692787 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/7c36839d-8518-4b0e-bf63-ec94d92e3c13-etc-swift\") pod \"swift-storage-0\" (UID: \"7c36839d-8518-4b0e-bf63-ec94d92e3c13\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 11:56:43 crc kubenswrapper[5052]: I1203 11:56:43.698698 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="478d0cb6-25de-46d5-979e-9e43bf1b7d4b" path="/var/lib/kubelet/pods/478d0cb6-25de-46d5-979e-9e43bf1b7d4b/volumes" Dec 03 11:56:43 crc kubenswrapper[5052]: I1203 11:56:43.828839 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/keystone-db-create-d5l4l"] Dec 03 11:56:43 crc kubenswrapper[5052]: E1203 11:56:43.829343 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="478d0cb6-25de-46d5-979e-9e43bf1b7d4b" containerName="registry-server" Dec 03 11:56:43 crc kubenswrapper[5052]: I1203 11:56:43.829364 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="478d0cb6-25de-46d5-979e-9e43bf1b7d4b" containerName="registry-server" Dec 03 11:56:43 crc kubenswrapper[5052]: E1203 11:56:43.829384 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="478d0cb6-25de-46d5-979e-9e43bf1b7d4b" containerName="extract-content" Dec 03 11:56:43 crc kubenswrapper[5052]: I1203 11:56:43.829396 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="478d0cb6-25de-46d5-979e-9e43bf1b7d4b" containerName="extract-content" Dec 03 11:56:43 crc kubenswrapper[5052]: E1203 11:56:43.829410 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="f0bbecd8-194a-4067-8786-1da410979cbf" containerName="mariadb-account-create-update" Dec 03 11:56:43 crc kubenswrapper[5052]: I1203 11:56:43.829422 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="f0bbecd8-194a-4067-8786-1da410979cbf" containerName="mariadb-account-create-update" Dec 03 11:56:43 crc kubenswrapper[5052]: E1203 11:56:43.829455 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="478d0cb6-25de-46d5-979e-9e43bf1b7d4b" containerName="extract-utilities" Dec 03 11:56:43 crc kubenswrapper[5052]: I1203 11:56:43.829467 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="478d0cb6-25de-46d5-979e-9e43bf1b7d4b" containerName="extract-utilities" Dec 03 11:56:43 crc kubenswrapper[5052]: E1203 11:56:43.829492 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="0c055858-1ccd-444e-a055-1bd2a919556f" containerName="mariadb-account-create-update" Dec 03 11:56:43 crc kubenswrapper[5052]: I1203 11:56:43.829503 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="0c055858-1ccd-444e-a055-1bd2a919556f" containerName="mariadb-account-create-update" Dec 03 11:56:43 crc kubenswrapper[5052]: E1203 11:56:43.829551 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="87e193c2-c4a3-4370-b683-b151324dbcec" containerName="mariadb-database-create" Dec 03 11:56:43 crc kubenswrapper[5052]: I1203 11:56:43.829562 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="87e193c2-c4a3-4370-b683-b151324dbcec" containerName="mariadb-database-create" Dec 03 11:56:43 crc kubenswrapper[5052]: I1203 11:56:43.829833 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="87e193c2-c4a3-4370-b683-b151324dbcec" containerName="mariadb-database-create" Dec 03 11:56:43 crc kubenswrapper[5052]: I1203 11:56:43.829867 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="f0bbecd8-194a-4067-8786-1da410979cbf" containerName="mariadb-account-create-update" Dec 03 11:56:43 crc kubenswrapper[5052]: I1203 11:56:43.829881 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="478d0cb6-25de-46d5-979e-9e43bf1b7d4b" containerName="registry-server" Dec 03 11:56:43 crc kubenswrapper[5052]: I1203 11:56:43.829900 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="0c055858-1ccd-444e-a055-1bd2a919556f" containerName="mariadb-account-create-update" Dec 03 11:56:43 crc kubenswrapper[5052]: I1203 11:56:43.831007 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-db-create-d5l4l" Dec 03 11:56:43 crc kubenswrapper[5052]: I1203 11:56:43.839024 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-db-create-d5l4l"] Dec 03 11:56:43 crc kubenswrapper[5052]: I1203 11:56:43.906431 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/keystone-0297-account-create-update-txbhd"] Dec 03 11:56:43 crc kubenswrapper[5052]: I1203 11:56:43.907572 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-0297-account-create-update-txbhd" Dec 03 11:56:43 crc kubenswrapper[5052]: I1203 11:56:43.910103 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone-db-secret" Dec 03 11:56:43 crc kubenswrapper[5052]: I1203 11:56:43.915656 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-0297-account-create-update-txbhd"] Dec 03 11:56:43 crc kubenswrapper[5052]: I1203 11:56:43.936035 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-storage-0" Dec 03 11:56:43 crc kubenswrapper[5052]: I1203 11:56:43.974770 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/534e7f12-9e45-4fd6-8bd4-dc712add1e1a-operator-scripts\") pod \"keystone-db-create-d5l4l\" (UID: \"534e7f12-9e45-4fd6-8bd4-dc712add1e1a\") " pod="openstack-kuttl-tests/keystone-db-create-d5l4l" Dec 03 11:56:43 crc kubenswrapper[5052]: I1203 11:56:43.974835 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bjz8v\" (UniqueName: \"kubernetes.io/projected/534e7f12-9e45-4fd6-8bd4-dc712add1e1a-kube-api-access-bjz8v\") pod \"keystone-db-create-d5l4l\" (UID: \"534e7f12-9e45-4fd6-8bd4-dc712add1e1a\") " pod="openstack-kuttl-tests/keystone-db-create-d5l4l" Dec 03 11:56:43 crc kubenswrapper[5052]: I1203 11:56:43.974941 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/6eda40cd-e0c7-4f9c-922a-e66672da99ed-operator-scripts\") pod \"keystone-0297-account-create-update-txbhd\" (UID: \"6eda40cd-e0c7-4f9c-922a-e66672da99ed\") " pod="openstack-kuttl-tests/keystone-0297-account-create-update-txbhd" Dec 03 11:56:43 crc kubenswrapper[5052]: I1203 11:56:43.975002 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2v7ls\" (UniqueName: \"kubernetes.io/projected/6eda40cd-e0c7-4f9c-922a-e66672da99ed-kube-api-access-2v7ls\") pod \"keystone-0297-account-create-update-txbhd\" (UID: \"6eda40cd-e0c7-4f9c-922a-e66672da99ed\") " pod="openstack-kuttl-tests/keystone-0297-account-create-update-txbhd" Dec 03 11:56:44 crc kubenswrapper[5052]: I1203 11:56:44.076974 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/534e7f12-9e45-4fd6-8bd4-dc712add1e1a-operator-scripts\") pod \"keystone-db-create-d5l4l\" (UID: \"534e7f12-9e45-4fd6-8bd4-dc712add1e1a\") " pod="openstack-kuttl-tests/keystone-db-create-d5l4l" Dec 03 11:56:44 crc kubenswrapper[5052]: I1203 11:56:44.077041 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-bjz8v\" (UniqueName: \"kubernetes.io/projected/534e7f12-9e45-4fd6-8bd4-dc712add1e1a-kube-api-access-bjz8v\") pod \"keystone-db-create-d5l4l\" (UID: \"534e7f12-9e45-4fd6-8bd4-dc712add1e1a\") " pod="openstack-kuttl-tests/keystone-db-create-d5l4l" Dec 03 11:56:44 crc kubenswrapper[5052]: I1203 11:56:44.077116 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/6eda40cd-e0c7-4f9c-922a-e66672da99ed-operator-scripts\") pod \"keystone-0297-account-create-update-txbhd\" (UID: \"6eda40cd-e0c7-4f9c-922a-e66672da99ed\") " pod="openstack-kuttl-tests/keystone-0297-account-create-update-txbhd" Dec 03 11:56:44 crc kubenswrapper[5052]: I1203 11:56:44.077147 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-2v7ls\" (UniqueName: \"kubernetes.io/projected/6eda40cd-e0c7-4f9c-922a-e66672da99ed-kube-api-access-2v7ls\") pod \"keystone-0297-account-create-update-txbhd\" (UID: \"6eda40cd-e0c7-4f9c-922a-e66672da99ed\") " pod="openstack-kuttl-tests/keystone-0297-account-create-update-txbhd" Dec 03 11:56:44 crc kubenswrapper[5052]: I1203 11:56:44.078004 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/534e7f12-9e45-4fd6-8bd4-dc712add1e1a-operator-scripts\") pod \"keystone-db-create-d5l4l\" (UID: \"534e7f12-9e45-4fd6-8bd4-dc712add1e1a\") " pod="openstack-kuttl-tests/keystone-db-create-d5l4l" Dec 03 11:56:44 crc kubenswrapper[5052]: I1203 11:56:44.079417 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/6eda40cd-e0c7-4f9c-922a-e66672da99ed-operator-scripts\") pod \"keystone-0297-account-create-update-txbhd\" (UID: \"6eda40cd-e0c7-4f9c-922a-e66672da99ed\") " pod="openstack-kuttl-tests/keystone-0297-account-create-update-txbhd" Dec 03 11:56:44 crc kubenswrapper[5052]: I1203 11:56:44.097999 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-2v7ls\" (UniqueName: \"kubernetes.io/projected/6eda40cd-e0c7-4f9c-922a-e66672da99ed-kube-api-access-2v7ls\") pod \"keystone-0297-account-create-update-txbhd\" (UID: \"6eda40cd-e0c7-4f9c-922a-e66672da99ed\") " pod="openstack-kuttl-tests/keystone-0297-account-create-update-txbhd" Dec 03 11:56:44 crc kubenswrapper[5052]: I1203 11:56:44.098172 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-bjz8v\" (UniqueName: \"kubernetes.io/projected/534e7f12-9e45-4fd6-8bd4-dc712add1e1a-kube-api-access-bjz8v\") pod \"keystone-db-create-d5l4l\" (UID: \"534e7f12-9e45-4fd6-8bd4-dc712add1e1a\") " pod="openstack-kuttl-tests/keystone-db-create-d5l4l" Dec 03 11:56:44 crc kubenswrapper[5052]: I1203 11:56:44.154640 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-db-create-d5l4l" Dec 03 11:56:44 crc kubenswrapper[5052]: I1203 11:56:44.206779 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/placement-db-create-c7b84"] Dec 03 11:56:44 crc kubenswrapper[5052]: I1203 11:56:44.208044 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/placement-db-create-c7b84"] Dec 03 11:56:44 crc kubenswrapper[5052]: I1203 11:56:44.208142 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-db-create-c7b84" Dec 03 11:56:44 crc kubenswrapper[5052]: I1203 11:56:44.252422 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-0297-account-create-update-txbhd" Dec 03 11:56:44 crc kubenswrapper[5052]: I1203 11:56:44.310658 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/placement-e203-account-create-update-kq6qb"] Dec 03 11:56:44 crc kubenswrapper[5052]: I1203 11:56:44.311896 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-e203-account-create-update-kq6qb" Dec 03 11:56:44 crc kubenswrapper[5052]: I1203 11:56:44.313878 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"placement-db-secret" Dec 03 11:56:44 crc kubenswrapper[5052]: I1203 11:56:44.315439 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/placement-e203-account-create-update-kq6qb"] Dec 03 11:56:44 crc kubenswrapper[5052]: I1203 11:56:44.382186 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/e043d529-47c3-4741-85a3-bdb55c638a97-operator-scripts\") pod \"placement-e203-account-create-update-kq6qb\" (UID: \"e043d529-47c3-4741-85a3-bdb55c638a97\") " pod="openstack-kuttl-tests/placement-e203-account-create-update-kq6qb" Dec 03 11:56:44 crc kubenswrapper[5052]: I1203 11:56:44.382558 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bqkkh\" (UniqueName: \"kubernetes.io/projected/fb1f8e74-cde9-4196-8b2d-5d302e69de58-kube-api-access-bqkkh\") pod \"placement-db-create-c7b84\" (UID: \"fb1f8e74-cde9-4196-8b2d-5d302e69de58\") " pod="openstack-kuttl-tests/placement-db-create-c7b84" Dec 03 11:56:44 crc kubenswrapper[5052]: I1203 11:56:44.382599 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/fb1f8e74-cde9-4196-8b2d-5d302e69de58-operator-scripts\") pod \"placement-db-create-c7b84\" (UID: \"fb1f8e74-cde9-4196-8b2d-5d302e69de58\") " pod="openstack-kuttl-tests/placement-db-create-c7b84" Dec 03 11:56:44 crc kubenswrapper[5052]: I1203 11:56:44.382747 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z7k7h\" (UniqueName: \"kubernetes.io/projected/e043d529-47c3-4741-85a3-bdb55c638a97-kube-api-access-z7k7h\") pod \"placement-e203-account-create-update-kq6qb\" (UID: \"e043d529-47c3-4741-85a3-bdb55c638a97\") " pod="openstack-kuttl-tests/placement-e203-account-create-update-kq6qb" Dec 03 11:56:44 crc kubenswrapper[5052]: I1203 11:56:44.438928 5052 scope.go:117] "RemoveContainer" containerID="968e0cfb3ba2a3fb71f3041ec8d4c733c2efe16927bb0c2821676b869b3fa296" Dec 03 11:56:44 crc kubenswrapper[5052]: I1203 11:56:44.485085 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-z7k7h\" (UniqueName: \"kubernetes.io/projected/e043d529-47c3-4741-85a3-bdb55c638a97-kube-api-access-z7k7h\") pod \"placement-e203-account-create-update-kq6qb\" (UID: \"e043d529-47c3-4741-85a3-bdb55c638a97\") " pod="openstack-kuttl-tests/placement-e203-account-create-update-kq6qb" Dec 03 11:56:44 crc kubenswrapper[5052]: I1203 11:56:44.485148 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/glance-db-create-nlxff"] Dec 03 11:56:44 crc kubenswrapper[5052]: I1203 11:56:44.485849 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/e043d529-47c3-4741-85a3-bdb55c638a97-operator-scripts\") pod \"placement-e203-account-create-update-kq6qb\" (UID: \"e043d529-47c3-4741-85a3-bdb55c638a97\") " pod="openstack-kuttl-tests/placement-e203-account-create-update-kq6qb" Dec 03 11:56:44 crc kubenswrapper[5052]: I1203 11:56:44.485156 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/e043d529-47c3-4741-85a3-bdb55c638a97-operator-scripts\") pod \"placement-e203-account-create-update-kq6qb\" (UID: \"e043d529-47c3-4741-85a3-bdb55c638a97\") " pod="openstack-kuttl-tests/placement-e203-account-create-update-kq6qb" Dec 03 11:56:44 crc kubenswrapper[5052]: I1203 11:56:44.486258 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-db-create-nlxff" Dec 03 11:56:44 crc kubenswrapper[5052]: I1203 11:56:44.486433 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-bqkkh\" (UniqueName: \"kubernetes.io/projected/fb1f8e74-cde9-4196-8b2d-5d302e69de58-kube-api-access-bqkkh\") pod \"placement-db-create-c7b84\" (UID: \"fb1f8e74-cde9-4196-8b2d-5d302e69de58\") " pod="openstack-kuttl-tests/placement-db-create-c7b84" Dec 03 11:56:44 crc kubenswrapper[5052]: I1203 11:56:44.486545 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/fb1f8e74-cde9-4196-8b2d-5d302e69de58-operator-scripts\") pod \"placement-db-create-c7b84\" (UID: \"fb1f8e74-cde9-4196-8b2d-5d302e69de58\") " pod="openstack-kuttl-tests/placement-db-create-c7b84" Dec 03 11:56:44 crc kubenswrapper[5052]: I1203 11:56:44.487355 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/fb1f8e74-cde9-4196-8b2d-5d302e69de58-operator-scripts\") pod \"placement-db-create-c7b84\" (UID: \"fb1f8e74-cde9-4196-8b2d-5d302e69de58\") " pod="openstack-kuttl-tests/placement-db-create-c7b84" Dec 03 11:56:44 crc kubenswrapper[5052]: I1203 11:56:44.502842 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-bqkkh\" (UniqueName: \"kubernetes.io/projected/fb1f8e74-cde9-4196-8b2d-5d302e69de58-kube-api-access-bqkkh\") pod \"placement-db-create-c7b84\" (UID: \"fb1f8e74-cde9-4196-8b2d-5d302e69de58\") " pod="openstack-kuttl-tests/placement-db-create-c7b84" Dec 03 11:56:44 crc kubenswrapper[5052]: I1203 11:56:44.515593 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-z7k7h\" (UniqueName: \"kubernetes.io/projected/e043d529-47c3-4741-85a3-bdb55c638a97-kube-api-access-z7k7h\") pod \"placement-e203-account-create-update-kq6qb\" (UID: \"e043d529-47c3-4741-85a3-bdb55c638a97\") " pod="openstack-kuttl-tests/placement-e203-account-create-update-kq6qb" Dec 03 11:56:44 crc kubenswrapper[5052]: I1203 11:56:44.519019 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-db-create-nlxff"] Dec 03 11:56:44 crc kubenswrapper[5052]: I1203 11:56:44.569652 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-db-create-c7b84" Dec 03 11:56:44 crc kubenswrapper[5052]: I1203 11:56:44.587282 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p5tvq\" (UniqueName: \"kubernetes.io/projected/fce46734-b4c7-4e69-9010-d23e2d5f372f-kube-api-access-p5tvq\") pod \"glance-db-create-nlxff\" (UID: \"fce46734-b4c7-4e69-9010-d23e2d5f372f\") " pod="openstack-kuttl-tests/glance-db-create-nlxff" Dec 03 11:56:44 crc kubenswrapper[5052]: I1203 11:56:44.587327 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/fce46734-b4c7-4e69-9010-d23e2d5f372f-operator-scripts\") pod \"glance-db-create-nlxff\" (UID: \"fce46734-b4c7-4e69-9010-d23e2d5f372f\") " pod="openstack-kuttl-tests/glance-db-create-nlxff" Dec 03 11:56:44 crc kubenswrapper[5052]: I1203 11:56:44.626543 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-e203-account-create-update-kq6qb" Dec 03 11:56:44 crc kubenswrapper[5052]: I1203 11:56:44.628007 5052 scope.go:117] "RemoveContainer" containerID="ef87bb53feef54bf7a1bda6533d1d5fd699d50c9e8449bc841967feab918c6e0" Dec 03 11:56:44 crc kubenswrapper[5052]: I1203 11:56:44.681594 5052 scope.go:117] "RemoveContainer" containerID="4f161a0e078151ba1bcbddcbdbae8f043fc412323e5f17d9f511a1e706347388" Dec 03 11:56:44 crc kubenswrapper[5052]: E1203 11:56:44.683135 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4f161a0e078151ba1bcbddcbdbae8f043fc412323e5f17d9f511a1e706347388\": container with ID starting with 4f161a0e078151ba1bcbddcbdbae8f043fc412323e5f17d9f511a1e706347388 not found: ID does not exist" containerID="4f161a0e078151ba1bcbddcbdbae8f043fc412323e5f17d9f511a1e706347388" Dec 03 11:56:44 crc kubenswrapper[5052]: I1203 11:56:44.683185 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4f161a0e078151ba1bcbddcbdbae8f043fc412323e5f17d9f511a1e706347388"} err="failed to get container status \"4f161a0e078151ba1bcbddcbdbae8f043fc412323e5f17d9f511a1e706347388\": rpc error: code = NotFound desc = could not find container \"4f161a0e078151ba1bcbddcbdbae8f043fc412323e5f17d9f511a1e706347388\": container with ID starting with 4f161a0e078151ba1bcbddcbdbae8f043fc412323e5f17d9f511a1e706347388 not found: ID does not exist" Dec 03 11:56:44 crc kubenswrapper[5052]: I1203 11:56:44.683219 5052 scope.go:117] "RemoveContainer" containerID="968e0cfb3ba2a3fb71f3041ec8d4c733c2efe16927bb0c2821676b869b3fa296" Dec 03 11:56:44 crc kubenswrapper[5052]: E1203 11:56:44.683832 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"968e0cfb3ba2a3fb71f3041ec8d4c733c2efe16927bb0c2821676b869b3fa296\": container with ID starting with 968e0cfb3ba2a3fb71f3041ec8d4c733c2efe16927bb0c2821676b869b3fa296 not found: ID does not exist" containerID="968e0cfb3ba2a3fb71f3041ec8d4c733c2efe16927bb0c2821676b869b3fa296" Dec 03 11:56:44 crc kubenswrapper[5052]: I1203 11:56:44.683883 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"968e0cfb3ba2a3fb71f3041ec8d4c733c2efe16927bb0c2821676b869b3fa296"} err="failed to get container status \"968e0cfb3ba2a3fb71f3041ec8d4c733c2efe16927bb0c2821676b869b3fa296\": rpc error: code = NotFound desc = could not find container \"968e0cfb3ba2a3fb71f3041ec8d4c733c2efe16927bb0c2821676b869b3fa296\": container with ID starting with 968e0cfb3ba2a3fb71f3041ec8d4c733c2efe16927bb0c2821676b869b3fa296 not found: ID does not exist" Dec 03 11:56:44 crc kubenswrapper[5052]: I1203 11:56:44.683900 5052 scope.go:117] "RemoveContainer" containerID="ef87bb53feef54bf7a1bda6533d1d5fd699d50c9e8449bc841967feab918c6e0" Dec 03 11:56:44 crc kubenswrapper[5052]: E1203 11:56:44.685125 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ef87bb53feef54bf7a1bda6533d1d5fd699d50c9e8449bc841967feab918c6e0\": container with ID starting with ef87bb53feef54bf7a1bda6533d1d5fd699d50c9e8449bc841967feab918c6e0 not found: ID does not exist" containerID="ef87bb53feef54bf7a1bda6533d1d5fd699d50c9e8449bc841967feab918c6e0" Dec 03 11:56:44 crc kubenswrapper[5052]: I1203 11:56:44.685169 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ef87bb53feef54bf7a1bda6533d1d5fd699d50c9e8449bc841967feab918c6e0"} err="failed to get container status \"ef87bb53feef54bf7a1bda6533d1d5fd699d50c9e8449bc841967feab918c6e0\": rpc error: code = NotFound desc = could not find container \"ef87bb53feef54bf7a1bda6533d1d5fd699d50c9e8449bc841967feab918c6e0\": container with ID starting with ef87bb53feef54bf7a1bda6533d1d5fd699d50c9e8449bc841967feab918c6e0 not found: ID does not exist" Dec 03 11:56:44 crc kubenswrapper[5052]: I1203 11:56:44.689024 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-p5tvq\" (UniqueName: \"kubernetes.io/projected/fce46734-b4c7-4e69-9010-d23e2d5f372f-kube-api-access-p5tvq\") pod \"glance-db-create-nlxff\" (UID: \"fce46734-b4c7-4e69-9010-d23e2d5f372f\") " pod="openstack-kuttl-tests/glance-db-create-nlxff" Dec 03 11:56:44 crc kubenswrapper[5052]: I1203 11:56:44.689067 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/fce46734-b4c7-4e69-9010-d23e2d5f372f-operator-scripts\") pod \"glance-db-create-nlxff\" (UID: \"fce46734-b4c7-4e69-9010-d23e2d5f372f\") " pod="openstack-kuttl-tests/glance-db-create-nlxff" Dec 03 11:56:44 crc kubenswrapper[5052]: I1203 11:56:44.689720 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/fce46734-b4c7-4e69-9010-d23e2d5f372f-operator-scripts\") pod \"glance-db-create-nlxff\" (UID: \"fce46734-b4c7-4e69-9010-d23e2d5f372f\") " pod="openstack-kuttl-tests/glance-db-create-nlxff" Dec 03 11:56:44 crc kubenswrapper[5052]: I1203 11:56:44.716936 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-p5tvq\" (UniqueName: \"kubernetes.io/projected/fce46734-b4c7-4e69-9010-d23e2d5f372f-kube-api-access-p5tvq\") pod \"glance-db-create-nlxff\" (UID: \"fce46734-b4c7-4e69-9010-d23e2d5f372f\") " pod="openstack-kuttl-tests/glance-db-create-nlxff" Dec 03 11:56:44 crc kubenswrapper[5052]: I1203 11:56:44.801117 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-db-create-nlxff" Dec 03 11:56:44 crc kubenswrapper[5052]: I1203 11:56:44.967638 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-ring-rebalance-mbrxc" Dec 03 11:56:45 crc kubenswrapper[5052]: I1203 11:56:45.101713 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/empty-dir/b25492d8-45bc-446a-a60c-b4f732484218-etc-swift\") pod \"b25492d8-45bc-446a-a60c-b4f732484218\" (UID: \"b25492d8-45bc-446a-a60c-b4f732484218\") " Dec 03 11:56:45 crc kubenswrapper[5052]: I1203 11:56:45.101856 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"dispersionconf\" (UniqueName: \"kubernetes.io/secret/b25492d8-45bc-446a-a60c-b4f732484218-dispersionconf\") pod \"b25492d8-45bc-446a-a60c-b4f732484218\" (UID: \"b25492d8-45bc-446a-a60c-b4f732484218\") " Dec 03 11:56:45 crc kubenswrapper[5052]: I1203 11:56:45.101970 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f6vds\" (UniqueName: \"kubernetes.io/projected/b25492d8-45bc-446a-a60c-b4f732484218-kube-api-access-f6vds\") pod \"b25492d8-45bc-446a-a60c-b4f732484218\" (UID: \"b25492d8-45bc-446a-a60c-b4f732484218\") " Dec 03 11:56:45 crc kubenswrapper[5052]: I1203 11:56:45.102068 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/b25492d8-45bc-446a-a60c-b4f732484218-scripts\") pod \"b25492d8-45bc-446a-a60c-b4f732484218\" (UID: \"b25492d8-45bc-446a-a60c-b4f732484218\") " Dec 03 11:56:45 crc kubenswrapper[5052]: I1203 11:56:45.102088 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b25492d8-45bc-446a-a60c-b4f732484218-combined-ca-bundle\") pod \"b25492d8-45bc-446a-a60c-b4f732484218\" (UID: \"b25492d8-45bc-446a-a60c-b4f732484218\") " Dec 03 11:56:45 crc kubenswrapper[5052]: I1203 11:56:45.102115 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ring-data-devices\" (UniqueName: \"kubernetes.io/configmap/b25492d8-45bc-446a-a60c-b4f732484218-ring-data-devices\") pod \"b25492d8-45bc-446a-a60c-b4f732484218\" (UID: \"b25492d8-45bc-446a-a60c-b4f732484218\") " Dec 03 11:56:45 crc kubenswrapper[5052]: I1203 11:56:45.102167 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"swiftconf\" (UniqueName: \"kubernetes.io/secret/b25492d8-45bc-446a-a60c-b4f732484218-swiftconf\") pod \"b25492d8-45bc-446a-a60c-b4f732484218\" (UID: \"b25492d8-45bc-446a-a60c-b4f732484218\") " Dec 03 11:56:45 crc kubenswrapper[5052]: I1203 11:56:45.105819 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b25492d8-45bc-446a-a60c-b4f732484218-etc-swift" (OuterVolumeSpecName: "etc-swift") pod "b25492d8-45bc-446a-a60c-b4f732484218" (UID: "b25492d8-45bc-446a-a60c-b4f732484218"). InnerVolumeSpecName "etc-swift". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:56:45 crc kubenswrapper[5052]: I1203 11:56:45.107103 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b25492d8-45bc-446a-a60c-b4f732484218-ring-data-devices" (OuterVolumeSpecName: "ring-data-devices") pod "b25492d8-45bc-446a-a60c-b4f732484218" (UID: "b25492d8-45bc-446a-a60c-b4f732484218"). InnerVolumeSpecName "ring-data-devices". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:56:45 crc kubenswrapper[5052]: I1203 11:56:45.108569 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b25492d8-45bc-446a-a60c-b4f732484218-kube-api-access-f6vds" (OuterVolumeSpecName: "kube-api-access-f6vds") pod "b25492d8-45bc-446a-a60c-b4f732484218" (UID: "b25492d8-45bc-446a-a60c-b4f732484218"). InnerVolumeSpecName "kube-api-access-f6vds". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:56:45 crc kubenswrapper[5052]: I1203 11:56:45.116327 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b25492d8-45bc-446a-a60c-b4f732484218-dispersionconf" (OuterVolumeSpecName: "dispersionconf") pod "b25492d8-45bc-446a-a60c-b4f732484218" (UID: "b25492d8-45bc-446a-a60c-b4f732484218"). InnerVolumeSpecName "dispersionconf". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:56:45 crc kubenswrapper[5052]: I1203 11:56:45.136099 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b25492d8-45bc-446a-a60c-b4f732484218-scripts" (OuterVolumeSpecName: "scripts") pod "b25492d8-45bc-446a-a60c-b4f732484218" (UID: "b25492d8-45bc-446a-a60c-b4f732484218"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:56:45 crc kubenswrapper[5052]: I1203 11:56:45.142017 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b25492d8-45bc-446a-a60c-b4f732484218-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "b25492d8-45bc-446a-a60c-b4f732484218" (UID: "b25492d8-45bc-446a-a60c-b4f732484218"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:56:45 crc kubenswrapper[5052]: I1203 11:56:45.159691 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/swift-storage-0"] Dec 03 11:56:45 crc kubenswrapper[5052]: I1203 11:56:45.170198 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b25492d8-45bc-446a-a60c-b4f732484218-swiftconf" (OuterVolumeSpecName: "swiftconf") pod "b25492d8-45bc-446a-a60c-b4f732484218" (UID: "b25492d8-45bc-446a-a60c-b4f732484218"). InnerVolumeSpecName "swiftconf". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:56:45 crc kubenswrapper[5052]: I1203 11:56:45.203838 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/b25492d8-45bc-446a-a60c-b4f732484218-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:56:45 crc kubenswrapper[5052]: I1203 11:56:45.203902 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b25492d8-45bc-446a-a60c-b4f732484218-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:56:45 crc kubenswrapper[5052]: I1203 11:56:45.203918 5052 reconciler_common.go:293] "Volume detached for volume \"ring-data-devices\" (UniqueName: \"kubernetes.io/configmap/b25492d8-45bc-446a-a60c-b4f732484218-ring-data-devices\") on node \"crc\" DevicePath \"\"" Dec 03 11:56:45 crc kubenswrapper[5052]: I1203 11:56:45.203930 5052 reconciler_common.go:293] "Volume detached for volume \"swiftconf\" (UniqueName: \"kubernetes.io/secret/b25492d8-45bc-446a-a60c-b4f732484218-swiftconf\") on node \"crc\" DevicePath \"\"" Dec 03 11:56:45 crc kubenswrapper[5052]: I1203 11:56:45.203941 5052 reconciler_common.go:293] "Volume detached for volume \"etc-swift\" (UniqueName: \"kubernetes.io/empty-dir/b25492d8-45bc-446a-a60c-b4f732484218-etc-swift\") on node \"crc\" DevicePath \"\"" Dec 03 11:56:45 crc kubenswrapper[5052]: I1203 11:56:45.203980 5052 reconciler_common.go:293] "Volume detached for volume \"dispersionconf\" (UniqueName: \"kubernetes.io/secret/b25492d8-45bc-446a-a60c-b4f732484218-dispersionconf\") on node \"crc\" DevicePath \"\"" Dec 03 11:56:45 crc kubenswrapper[5052]: I1203 11:56:45.203997 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-f6vds\" (UniqueName: \"kubernetes.io/projected/b25492d8-45bc-446a-a60c-b4f732484218-kube-api-access-f6vds\") on node \"crc\" DevicePath \"\"" Dec 03 11:56:45 crc kubenswrapper[5052]: I1203 11:56:45.207232 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 11:56:45 crc kubenswrapper[5052]: I1203 11:56:45.281685 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-0297-account-create-update-txbhd"] Dec 03 11:56:45 crc kubenswrapper[5052]: I1203 11:56:45.369651 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/placement-db-create-c7b84"] Dec 03 11:56:45 crc kubenswrapper[5052]: I1203 11:56:45.385014 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-db-create-d5l4l"] Dec 03 11:56:45 crc kubenswrapper[5052]: I1203 11:56:45.393113 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/placement-e203-account-create-update-kq6qb"] Dec 03 11:56:45 crc kubenswrapper[5052]: I1203 11:56:45.535595 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-db-create-d5l4l" event={"ID":"534e7f12-9e45-4fd6-8bd4-dc712add1e1a","Type":"ContainerStarted","Data":"8f87ffc06ad2819c6885e1edc838be1deaeb9279486cd8a9255b0105da92876f"} Dec 03 11:56:45 crc kubenswrapper[5052]: I1203 11:56:45.543901 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-0297-account-create-update-txbhd" event={"ID":"6eda40cd-e0c7-4f9c-922a-e66672da99ed","Type":"ContainerStarted","Data":"ab64b91a36179e8cec6eeaec8a32abe2c8a2a7d82d49a3bdc9944e6ab3fcf130"} Dec 03 11:56:45 crc kubenswrapper[5052]: I1203 11:56:45.548764 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-db-create-c7b84" event={"ID":"fb1f8e74-cde9-4196-8b2d-5d302e69de58","Type":"ContainerStarted","Data":"80b6c31049ee52acef77c4eab6f959ebea031cdd61c41e455881e0b4f54a9957"} Dec 03 11:56:45 crc kubenswrapper[5052]: I1203 11:56:45.550935 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-e203-account-create-update-kq6qb" event={"ID":"e043d529-47c3-4741-85a3-bdb55c638a97","Type":"ContainerStarted","Data":"86ee952692a75d4114e6347c0290c0cf4dc66df29b253899c50000a8f725e058"} Dec 03 11:56:45 crc kubenswrapper[5052]: I1203 11:56:45.552688 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-db-create-nlxff"] Dec 03 11:56:45 crc kubenswrapper[5052]: I1203 11:56:45.565314 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-ring-rebalance-mbrxc" event={"ID":"b25492d8-45bc-446a-a60c-b4f732484218","Type":"ContainerDied","Data":"1f3690e37d7f7a12050418a9e645f30bee5e3dcc3b42b8ed7615e5c72e0aa105"} Dec 03 11:56:45 crc kubenswrapper[5052]: I1203 11:56:45.565378 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1f3690e37d7f7a12050418a9e645f30bee5e3dcc3b42b8ed7615e5c72e0aa105" Dec 03 11:56:45 crc kubenswrapper[5052]: I1203 11:56:45.565807 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-ring-rebalance-mbrxc" Dec 03 11:56:45 crc kubenswrapper[5052]: I1203 11:56:45.586483 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"7c36839d-8518-4b0e-bf63-ec94d92e3c13","Type":"ContainerStarted","Data":"a47f522eb33440e9d86ad8e2ec20e36a501400f9a7394734425605e03d2dc2ea"} Dec 03 11:56:46 crc kubenswrapper[5052]: I1203 11:56:46.639873 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-db-create-nlxff" event={"ID":"fce46734-b4c7-4e69-9010-d23e2d5f372f","Type":"ContainerStarted","Data":"324fde5777eff4b5588948210f2107fc67b80463e3177dbcc4c61e7e419dc4cd"} Dec 03 11:56:46 crc kubenswrapper[5052]: I1203 11:56:46.639937 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-db-create-nlxff" event={"ID":"fce46734-b4c7-4e69-9010-d23e2d5f372f","Type":"ContainerStarted","Data":"c07fe01c1741acd6b9de7eb56a59d5a5d3784adc8df765ec47b12de0168b2f65"} Dec 03 11:56:46 crc kubenswrapper[5052]: I1203 11:56:46.672840 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-db-create-d5l4l" event={"ID":"534e7f12-9e45-4fd6-8bd4-dc712add1e1a","Type":"ContainerStarted","Data":"cd5d0e91d5279fe56400b9ddb8c67759af038b8df5fd0c0db561a366628bc6a3"} Dec 03 11:56:46 crc kubenswrapper[5052]: I1203 11:56:46.686983 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/glance-db-create-nlxff" podStartSLOduration=2.686956972 podStartE2EDuration="2.686956972s" podCreationTimestamp="2025-12-03 11:56:44 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:56:46.676496773 +0000 UTC m=+2197.264738547" watchObservedRunningTime="2025-12-03 11:56:46.686956972 +0000 UTC m=+2197.275198736" Dec 03 11:56:46 crc kubenswrapper[5052]: I1203 11:56:46.700815 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-0297-account-create-update-txbhd" event={"ID":"6eda40cd-e0c7-4f9c-922a-e66672da99ed","Type":"ContainerStarted","Data":"5e5c3ad7b8915e802d7144bd906adc382a388dd664d9902320496f0eaa0909bc"} Dec 03 11:56:46 crc kubenswrapper[5052]: I1203 11:56:46.742363 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-db-create-c7b84" event={"ID":"fb1f8e74-cde9-4196-8b2d-5d302e69de58","Type":"ContainerStarted","Data":"9b319e148e0126893b069cd2f5fdc7170727f3a457ff8ccf29646edcca07668d"} Dec 03 11:56:46 crc kubenswrapper[5052]: I1203 11:56:46.755580 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/keystone-db-create-d5l4l" podStartSLOduration=3.755560415 podStartE2EDuration="3.755560415s" podCreationTimestamp="2025-12-03 11:56:43 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:56:46.710821175 +0000 UTC m=+2197.299062939" watchObservedRunningTime="2025-12-03 11:56:46.755560415 +0000 UTC m=+2197.343802179" Dec 03 11:56:46 crc kubenswrapper[5052]: I1203 11:56:46.763553 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/alertmanager-metric-storage-0" event={"ID":"7cbbb6c5-a087-400f-9beb-e740ae1ddfbd","Type":"ContainerStarted","Data":"ae36686b468c95feb23e421e5898d4b9d5542290763ef55cdda58e32e8d7665b"} Dec 03 11:56:46 crc kubenswrapper[5052]: I1203 11:56:46.793179 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-e203-account-create-update-kq6qb" event={"ID":"e043d529-47c3-4741-85a3-bdb55c638a97","Type":"ContainerStarted","Data":"021eba600b1eeccb1d30827210c0e0365f75e3571270f84c0206d112c79c35af"} Dec 03 11:56:46 crc kubenswrapper[5052]: I1203 11:56:46.793953 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/keystone-0297-account-create-update-txbhd" podStartSLOduration=3.793929753 podStartE2EDuration="3.793929753s" podCreationTimestamp="2025-12-03 11:56:43 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:56:46.737997313 +0000 UTC m=+2197.326239097" watchObservedRunningTime="2025-12-03 11:56:46.793929753 +0000 UTC m=+2197.382171517" Dec 03 11:56:46 crc kubenswrapper[5052]: I1203 11:56:46.816482 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/placement-db-create-c7b84" podStartSLOduration=2.816464008 podStartE2EDuration="2.816464008s" podCreationTimestamp="2025-12-03 11:56:44 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:56:46.784153324 +0000 UTC m=+2197.372395098" watchObservedRunningTime="2025-12-03 11:56:46.816464008 +0000 UTC m=+2197.404705792" Dec 03 11:56:46 crc kubenswrapper[5052]: I1203 11:56:46.817290 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"7c36839d-8518-4b0e-bf63-ec94d92e3c13","Type":"ContainerStarted","Data":"25010c622994b6f9bf074625c83c6f4b20f430bc358fcaedc8fb64c94ab3dc3a"} Dec 03 11:56:46 crc kubenswrapper[5052]: I1203 11:56:46.855005 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/placement-e203-account-create-update-kq6qb" podStartSLOduration=2.854989611 podStartE2EDuration="2.854989611s" podCreationTimestamp="2025-12-03 11:56:44 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:56:46.828033279 +0000 UTC m=+2197.416275063" watchObservedRunningTime="2025-12-03 11:56:46.854989611 +0000 UTC m=+2197.443231385" Dec 03 11:56:47 crc kubenswrapper[5052]: I1203 11:56:47.828311 5052 generic.go:334] "Generic (PLEG): container finished" podID="534e7f12-9e45-4fd6-8bd4-dc712add1e1a" containerID="cd5d0e91d5279fe56400b9ddb8c67759af038b8df5fd0c0db561a366628bc6a3" exitCode=0 Dec 03 11:56:47 crc kubenswrapper[5052]: I1203 11:56:47.828396 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-db-create-d5l4l" event={"ID":"534e7f12-9e45-4fd6-8bd4-dc712add1e1a","Type":"ContainerDied","Data":"cd5d0e91d5279fe56400b9ddb8c67759af038b8df5fd0c0db561a366628bc6a3"} Dec 03 11:56:47 crc kubenswrapper[5052]: I1203 11:56:47.830419 5052 generic.go:334] "Generic (PLEG): container finished" podID="6eda40cd-e0c7-4f9c-922a-e66672da99ed" containerID="5e5c3ad7b8915e802d7144bd906adc382a388dd664d9902320496f0eaa0909bc" exitCode=0 Dec 03 11:56:47 crc kubenswrapper[5052]: I1203 11:56:47.830477 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-0297-account-create-update-txbhd" event={"ID":"6eda40cd-e0c7-4f9c-922a-e66672da99ed","Type":"ContainerDied","Data":"5e5c3ad7b8915e802d7144bd906adc382a388dd664d9902320496f0eaa0909bc"} Dec 03 11:56:47 crc kubenswrapper[5052]: I1203 11:56:47.832640 5052 generic.go:334] "Generic (PLEG): container finished" podID="fb1f8e74-cde9-4196-8b2d-5d302e69de58" containerID="9b319e148e0126893b069cd2f5fdc7170727f3a457ff8ccf29646edcca07668d" exitCode=0 Dec 03 11:56:47 crc kubenswrapper[5052]: I1203 11:56:47.832702 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-db-create-c7b84" event={"ID":"fb1f8e74-cde9-4196-8b2d-5d302e69de58","Type":"ContainerDied","Data":"9b319e148e0126893b069cd2f5fdc7170727f3a457ff8ccf29646edcca07668d"} Dec 03 11:56:47 crc kubenswrapper[5052]: I1203 11:56:47.834696 5052 generic.go:334] "Generic (PLEG): container finished" podID="e043d529-47c3-4741-85a3-bdb55c638a97" containerID="021eba600b1eeccb1d30827210c0e0365f75e3571270f84c0206d112c79c35af" exitCode=0 Dec 03 11:56:47 crc kubenswrapper[5052]: I1203 11:56:47.834736 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-e203-account-create-update-kq6qb" event={"ID":"e043d529-47c3-4741-85a3-bdb55c638a97","Type":"ContainerDied","Data":"021eba600b1eeccb1d30827210c0e0365f75e3571270f84c0206d112c79c35af"} Dec 03 11:56:47 crc kubenswrapper[5052]: I1203 11:56:47.836216 5052 generic.go:334] "Generic (PLEG): container finished" podID="fce46734-b4c7-4e69-9010-d23e2d5f372f" containerID="324fde5777eff4b5588948210f2107fc67b80463e3177dbcc4c61e7e419dc4cd" exitCode=0 Dec 03 11:56:47 crc kubenswrapper[5052]: I1203 11:56:47.836244 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-db-create-nlxff" event={"ID":"fce46734-b4c7-4e69-9010-d23e2d5f372f","Type":"ContainerDied","Data":"324fde5777eff4b5588948210f2107fc67b80463e3177dbcc4c61e7e419dc4cd"} Dec 03 11:56:49 crc kubenswrapper[5052]: I1203 11:56:49.686554 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-db-create-d5l4l" Dec 03 11:56:49 crc kubenswrapper[5052]: I1203 11:56:49.727581 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-db-create-c7b84" Dec 03 11:56:49 crc kubenswrapper[5052]: I1203 11:56:49.730600 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-db-create-nlxff" Dec 03 11:56:49 crc kubenswrapper[5052]: I1203 11:56:49.741393 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-e203-account-create-update-kq6qb" Dec 03 11:56:49 crc kubenswrapper[5052]: I1203 11:56:49.806812 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/534e7f12-9e45-4fd6-8bd4-dc712add1e1a-operator-scripts\") pod \"534e7f12-9e45-4fd6-8bd4-dc712add1e1a\" (UID: \"534e7f12-9e45-4fd6-8bd4-dc712add1e1a\") " Dec 03 11:56:49 crc kubenswrapper[5052]: I1203 11:56:49.807267 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bjz8v\" (UniqueName: \"kubernetes.io/projected/534e7f12-9e45-4fd6-8bd4-dc712add1e1a-kube-api-access-bjz8v\") pod \"534e7f12-9e45-4fd6-8bd4-dc712add1e1a\" (UID: \"534e7f12-9e45-4fd6-8bd4-dc712add1e1a\") " Dec 03 11:56:49 crc kubenswrapper[5052]: I1203 11:56:49.807481 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-0297-account-create-update-txbhd" Dec 03 11:56:49 crc kubenswrapper[5052]: I1203 11:56:49.808861 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/534e7f12-9e45-4fd6-8bd4-dc712add1e1a-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "534e7f12-9e45-4fd6-8bd4-dc712add1e1a" (UID: "534e7f12-9e45-4fd6-8bd4-dc712add1e1a"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:56:49 crc kubenswrapper[5052]: I1203 11:56:49.811759 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/534e7f12-9e45-4fd6-8bd4-dc712add1e1a-kube-api-access-bjz8v" (OuterVolumeSpecName: "kube-api-access-bjz8v") pod "534e7f12-9e45-4fd6-8bd4-dc712add1e1a" (UID: "534e7f12-9e45-4fd6-8bd4-dc712add1e1a"). InnerVolumeSpecName "kube-api-access-bjz8v". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:56:49 crc kubenswrapper[5052]: I1203 11:56:49.864065 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-db-create-nlxff" event={"ID":"fce46734-b4c7-4e69-9010-d23e2d5f372f","Type":"ContainerDied","Data":"c07fe01c1741acd6b9de7eb56a59d5a5d3784adc8df765ec47b12de0168b2f65"} Dec 03 11:56:49 crc kubenswrapper[5052]: I1203 11:56:49.864106 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c07fe01c1741acd6b9de7eb56a59d5a5d3784adc8df765ec47b12de0168b2f65" Dec 03 11:56:49 crc kubenswrapper[5052]: I1203 11:56:49.864168 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-db-create-nlxff" Dec 03 11:56:49 crc kubenswrapper[5052]: I1203 11:56:49.869327 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-db-create-d5l4l" event={"ID":"534e7f12-9e45-4fd6-8bd4-dc712add1e1a","Type":"ContainerDied","Data":"8f87ffc06ad2819c6885e1edc838be1deaeb9279486cd8a9255b0105da92876f"} Dec 03 11:56:49 crc kubenswrapper[5052]: I1203 11:56:49.869370 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8f87ffc06ad2819c6885e1edc838be1deaeb9279486cd8a9255b0105da92876f" Dec 03 11:56:49 crc kubenswrapper[5052]: I1203 11:56:49.869339 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-db-create-d5l4l" Dec 03 11:56:49 crc kubenswrapper[5052]: I1203 11:56:49.874370 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-0297-account-create-update-txbhd" event={"ID":"6eda40cd-e0c7-4f9c-922a-e66672da99ed","Type":"ContainerDied","Data":"ab64b91a36179e8cec6eeaec8a32abe2c8a2a7d82d49a3bdc9944e6ab3fcf130"} Dec 03 11:56:49 crc kubenswrapper[5052]: I1203 11:56:49.874417 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ab64b91a36179e8cec6eeaec8a32abe2c8a2a7d82d49a3bdc9944e6ab3fcf130" Dec 03 11:56:49 crc kubenswrapper[5052]: I1203 11:56:49.874497 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-0297-account-create-update-txbhd" Dec 03 11:56:49 crc kubenswrapper[5052]: I1203 11:56:49.895357 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-db-create-c7b84" Dec 03 11:56:49 crc kubenswrapper[5052]: I1203 11:56:49.895871 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-db-create-c7b84" event={"ID":"fb1f8e74-cde9-4196-8b2d-5d302e69de58","Type":"ContainerDied","Data":"80b6c31049ee52acef77c4eab6f959ebea031cdd61c41e455881e0b4f54a9957"} Dec 03 11:56:49 crc kubenswrapper[5052]: I1203 11:56:49.895920 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="80b6c31049ee52acef77c4eab6f959ebea031cdd61c41e455881e0b4f54a9957" Dec 03 11:56:49 crc kubenswrapper[5052]: I1203 11:56:49.897383 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-e203-account-create-update-kq6qb" event={"ID":"e043d529-47c3-4741-85a3-bdb55c638a97","Type":"ContainerDied","Data":"86ee952692a75d4114e6347c0290c0cf4dc66df29b253899c50000a8f725e058"} Dec 03 11:56:49 crc kubenswrapper[5052]: I1203 11:56:49.897404 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="86ee952692a75d4114e6347c0290c0cf4dc66df29b253899c50000a8f725e058" Dec 03 11:56:49 crc kubenswrapper[5052]: I1203 11:56:49.897461 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-e203-account-create-update-kq6qb" Dec 03 11:56:49 crc kubenswrapper[5052]: I1203 11:56:49.899113 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"7c36839d-8518-4b0e-bf63-ec94d92e3c13","Type":"ContainerStarted","Data":"109cd7db5617ce6f1e5aaea63acd2470d8ce489ee50ccc8c8d13a1422834ac45"} Dec 03 11:56:49 crc kubenswrapper[5052]: I1203 11:56:49.912798 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/fb1f8e74-cde9-4196-8b2d-5d302e69de58-operator-scripts\") pod \"fb1f8e74-cde9-4196-8b2d-5d302e69de58\" (UID: \"fb1f8e74-cde9-4196-8b2d-5d302e69de58\") " Dec 03 11:56:49 crc kubenswrapper[5052]: I1203 11:56:49.912834 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/e043d529-47c3-4741-85a3-bdb55c638a97-operator-scripts\") pod \"e043d529-47c3-4741-85a3-bdb55c638a97\" (UID: \"e043d529-47c3-4741-85a3-bdb55c638a97\") " Dec 03 11:56:49 crc kubenswrapper[5052]: I1203 11:56:49.912870 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/6eda40cd-e0c7-4f9c-922a-e66672da99ed-operator-scripts\") pod \"6eda40cd-e0c7-4f9c-922a-e66672da99ed\" (UID: \"6eda40cd-e0c7-4f9c-922a-e66672da99ed\") " Dec 03 11:56:49 crc kubenswrapper[5052]: I1203 11:56:49.912896 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/fce46734-b4c7-4e69-9010-d23e2d5f372f-operator-scripts\") pod \"fce46734-b4c7-4e69-9010-d23e2d5f372f\" (UID: \"fce46734-b4c7-4e69-9010-d23e2d5f372f\") " Dec 03 11:56:49 crc kubenswrapper[5052]: I1203 11:56:49.912931 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bqkkh\" (UniqueName: \"kubernetes.io/projected/fb1f8e74-cde9-4196-8b2d-5d302e69de58-kube-api-access-bqkkh\") pod \"fb1f8e74-cde9-4196-8b2d-5d302e69de58\" (UID: \"fb1f8e74-cde9-4196-8b2d-5d302e69de58\") " Dec 03 11:56:49 crc kubenswrapper[5052]: I1203 11:56:49.912959 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2v7ls\" (UniqueName: \"kubernetes.io/projected/6eda40cd-e0c7-4f9c-922a-e66672da99ed-kube-api-access-2v7ls\") pod \"6eda40cd-e0c7-4f9c-922a-e66672da99ed\" (UID: \"6eda40cd-e0c7-4f9c-922a-e66672da99ed\") " Dec 03 11:56:49 crc kubenswrapper[5052]: I1203 11:56:49.913080 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p5tvq\" (UniqueName: \"kubernetes.io/projected/fce46734-b4c7-4e69-9010-d23e2d5f372f-kube-api-access-p5tvq\") pod \"fce46734-b4c7-4e69-9010-d23e2d5f372f\" (UID: \"fce46734-b4c7-4e69-9010-d23e2d5f372f\") " Dec 03 11:56:49 crc kubenswrapper[5052]: I1203 11:56:49.913115 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z7k7h\" (UniqueName: \"kubernetes.io/projected/e043d529-47c3-4741-85a3-bdb55c638a97-kube-api-access-z7k7h\") pod \"e043d529-47c3-4741-85a3-bdb55c638a97\" (UID: \"e043d529-47c3-4741-85a3-bdb55c638a97\") " Dec 03 11:56:49 crc kubenswrapper[5052]: I1203 11:56:49.913443 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-bjz8v\" (UniqueName: \"kubernetes.io/projected/534e7f12-9e45-4fd6-8bd4-dc712add1e1a-kube-api-access-bjz8v\") on node \"crc\" DevicePath \"\"" Dec 03 11:56:49 crc kubenswrapper[5052]: I1203 11:56:49.913455 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/534e7f12-9e45-4fd6-8bd4-dc712add1e1a-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:56:49 crc kubenswrapper[5052]: I1203 11:56:49.913955 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fce46734-b4c7-4e69-9010-d23e2d5f372f-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "fce46734-b4c7-4e69-9010-d23e2d5f372f" (UID: "fce46734-b4c7-4e69-9010-d23e2d5f372f"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:56:49 crc kubenswrapper[5052]: I1203 11:56:49.913970 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6eda40cd-e0c7-4f9c-922a-e66672da99ed-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "6eda40cd-e0c7-4f9c-922a-e66672da99ed" (UID: "6eda40cd-e0c7-4f9c-922a-e66672da99ed"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:56:49 crc kubenswrapper[5052]: I1203 11:56:49.914085 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fb1f8e74-cde9-4196-8b2d-5d302e69de58-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "fb1f8e74-cde9-4196-8b2d-5d302e69de58" (UID: "fb1f8e74-cde9-4196-8b2d-5d302e69de58"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:56:49 crc kubenswrapper[5052]: I1203 11:56:49.914494 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e043d529-47c3-4741-85a3-bdb55c638a97-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "e043d529-47c3-4741-85a3-bdb55c638a97" (UID: "e043d529-47c3-4741-85a3-bdb55c638a97"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:56:49 crc kubenswrapper[5052]: I1203 11:56:49.921390 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fce46734-b4c7-4e69-9010-d23e2d5f372f-kube-api-access-p5tvq" (OuterVolumeSpecName: "kube-api-access-p5tvq") pod "fce46734-b4c7-4e69-9010-d23e2d5f372f" (UID: "fce46734-b4c7-4e69-9010-d23e2d5f372f"). InnerVolumeSpecName "kube-api-access-p5tvq". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:56:49 crc kubenswrapper[5052]: I1203 11:56:49.921593 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6eda40cd-e0c7-4f9c-922a-e66672da99ed-kube-api-access-2v7ls" (OuterVolumeSpecName: "kube-api-access-2v7ls") pod "6eda40cd-e0c7-4f9c-922a-e66672da99ed" (UID: "6eda40cd-e0c7-4f9c-922a-e66672da99ed"). InnerVolumeSpecName "kube-api-access-2v7ls". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:56:49 crc kubenswrapper[5052]: I1203 11:56:49.921681 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e043d529-47c3-4741-85a3-bdb55c638a97-kube-api-access-z7k7h" (OuterVolumeSpecName: "kube-api-access-z7k7h") pod "e043d529-47c3-4741-85a3-bdb55c638a97" (UID: "e043d529-47c3-4741-85a3-bdb55c638a97"). InnerVolumeSpecName "kube-api-access-z7k7h". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:56:49 crc kubenswrapper[5052]: I1203 11:56:49.921753 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fb1f8e74-cde9-4196-8b2d-5d302e69de58-kube-api-access-bqkkh" (OuterVolumeSpecName: "kube-api-access-bqkkh") pod "fb1f8e74-cde9-4196-8b2d-5d302e69de58" (UID: "fb1f8e74-cde9-4196-8b2d-5d302e69de58"). InnerVolumeSpecName "kube-api-access-bqkkh". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:56:50 crc kubenswrapper[5052]: I1203 11:56:50.014773 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/fb1f8e74-cde9-4196-8b2d-5d302e69de58-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:56:50 crc kubenswrapper[5052]: I1203 11:56:50.014829 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/e043d529-47c3-4741-85a3-bdb55c638a97-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:56:50 crc kubenswrapper[5052]: I1203 11:56:50.014847 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/6eda40cd-e0c7-4f9c-922a-e66672da99ed-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:56:50 crc kubenswrapper[5052]: I1203 11:56:50.014858 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/fce46734-b4c7-4e69-9010-d23e2d5f372f-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:56:50 crc kubenswrapper[5052]: I1203 11:56:50.014872 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-bqkkh\" (UniqueName: \"kubernetes.io/projected/fb1f8e74-cde9-4196-8b2d-5d302e69de58-kube-api-access-bqkkh\") on node \"crc\" DevicePath \"\"" Dec 03 11:56:50 crc kubenswrapper[5052]: I1203 11:56:50.014884 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-2v7ls\" (UniqueName: \"kubernetes.io/projected/6eda40cd-e0c7-4f9c-922a-e66672da99ed-kube-api-access-2v7ls\") on node \"crc\" DevicePath \"\"" Dec 03 11:56:50 crc kubenswrapper[5052]: I1203 11:56:50.014895 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-p5tvq\" (UniqueName: \"kubernetes.io/projected/fce46734-b4c7-4e69-9010-d23e2d5f372f-kube-api-access-p5tvq\") on node \"crc\" DevicePath \"\"" Dec 03 11:56:50 crc kubenswrapper[5052]: I1203 11:56:50.014906 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-z7k7h\" (UniqueName: \"kubernetes.io/projected/e043d529-47c3-4741-85a3-bdb55c638a97-kube-api-access-z7k7h\") on node \"crc\" DevicePath \"\"" Dec 03 11:56:50 crc kubenswrapper[5052]: I1203 11:56:50.908432 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/prometheus-metric-storage-0" event={"ID":"33918bc1-340a-4d26-a204-001b85c15f04","Type":"ContainerStarted","Data":"a39079dcf889f7fbf32bc70b596f4c1f6eab751f6901536f1d0bda478008c680"} Dec 03 11:56:50 crc kubenswrapper[5052]: I1203 11:56:50.911130 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"7c36839d-8518-4b0e-bf63-ec94d92e3c13","Type":"ContainerStarted","Data":"2cbddf37ed81f13769bd22ea5da02802984cddbc9008db7adf62cbd5f35a8c05"} Dec 03 11:56:50 crc kubenswrapper[5052]: I1203 11:56:50.911186 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"7c36839d-8518-4b0e-bf63-ec94d92e3c13","Type":"ContainerStarted","Data":"11cecc9603bd65df20d6a71c4f44da191cdbb3a4f878b9e3695671d86d19bd72"} Dec 03 11:56:50 crc kubenswrapper[5052]: I1203 11:56:50.911200 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"7c36839d-8518-4b0e-bf63-ec94d92e3c13","Type":"ContainerStarted","Data":"682960d7b8798b4f6d377b7a7a2fe296dcddb554fb6aa11cf50780880e5ef868"} Dec 03 11:56:51 crc kubenswrapper[5052]: I1203 11:56:51.926024 5052 generic.go:334] "Generic (PLEG): container finished" podID="30578d71-fe2b-4eed-9a07-6c8bfe4a1c43" containerID="250de81141f60765c155818d8a4cfbffd80ff7babe5acee647422de56500c58c" exitCode=0 Dec 03 11:56:51 crc kubenswrapper[5052]: I1203 11:56:51.926432 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-server-0" event={"ID":"30578d71-fe2b-4eed-9a07-6c8bfe4a1c43","Type":"ContainerDied","Data":"250de81141f60765c155818d8a4cfbffd80ff7babe5acee647422de56500c58c"} Dec 03 11:56:51 crc kubenswrapper[5052]: I1203 11:56:51.934080 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"7c36839d-8518-4b0e-bf63-ec94d92e3c13","Type":"ContainerStarted","Data":"4066fa8ec273d30d5e0da93dde5867442a7c42b658e83fd69bdb6853a4596664"} Dec 03 11:56:51 crc kubenswrapper[5052]: I1203 11:56:51.934121 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"7c36839d-8518-4b0e-bf63-ec94d92e3c13","Type":"ContainerStarted","Data":"1baab415bc319f90ebf14e16f12a6406c7324f83d7e1edd6fce3f36ba0d17106"} Dec 03 11:56:51 crc kubenswrapper[5052]: I1203 11:56:51.934135 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"7c36839d-8518-4b0e-bf63-ec94d92e3c13","Type":"ContainerStarted","Data":"8bf39629dcf79a88a7eec59f36c04fffa01173de57cefa2317a3fcbe70912276"} Dec 03 11:56:51 crc kubenswrapper[5052]: I1203 11:56:51.934143 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"7c36839d-8518-4b0e-bf63-ec94d92e3c13","Type":"ContainerStarted","Data":"0fc7226ddfe9193d0d31fe7fe4a9dcc3ec094ff416d17f3fbe045f601f3a147d"} Dec 03 11:56:52 crc kubenswrapper[5052]: I1203 11:56:52.944849 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-server-0" event={"ID":"30578d71-fe2b-4eed-9a07-6c8bfe4a1c43","Type":"ContainerStarted","Data":"d9ddf262949f1f7939bc5d4f4522859474860ed8b5edac12e5534c72d138210a"} Dec 03 11:56:52 crc kubenswrapper[5052]: I1203 11:56:52.945359 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 11:56:52 crc kubenswrapper[5052]: I1203 11:56:52.947451 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/prometheus-metric-storage-0" event={"ID":"33918bc1-340a-4d26-a204-001b85c15f04","Type":"ContainerStarted","Data":"4efdd771545fd00295d9e81c501f88a8052264f58cb92b45ecd0fedadf7e279c"} Dec 03 11:56:52 crc kubenswrapper[5052]: I1203 11:56:52.955245 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"7c36839d-8518-4b0e-bf63-ec94d92e3c13","Type":"ContainerStarted","Data":"fcf4f78f80f00487ca12528cc832dae0efc274f54e76db4429646d7410f72e60"} Dec 03 11:56:52 crc kubenswrapper[5052]: I1203 11:56:52.955293 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"7c36839d-8518-4b0e-bf63-ec94d92e3c13","Type":"ContainerStarted","Data":"ac3482f5fa4de80aab2650bc04e9d5264e54ea6c82aaf1d84113fd1b78ebc9e2"} Dec 03 11:56:52 crc kubenswrapper[5052]: I1203 11:56:52.955302 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"7c36839d-8518-4b0e-bf63-ec94d92e3c13","Type":"ContainerStarted","Data":"5f57677d7d2dc23e3fd4fa6e81bdcbe886b2bfe28287b6105f234e2a357ac510"} Dec 03 11:56:52 crc kubenswrapper[5052]: I1203 11:56:52.955311 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"7c36839d-8518-4b0e-bf63-ec94d92e3c13","Type":"ContainerStarted","Data":"99e3b71ca2f828cfb4d8d9e730eaa81b9c4c624aca8e895c30c5e6fdf97517ac"} Dec 03 11:56:52 crc kubenswrapper[5052]: I1203 11:56:52.959062 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/alertmanager-metric-storage-0" event={"ID":"7cbbb6c5-a087-400f-9beb-e740ae1ddfbd","Type":"ContainerStarted","Data":"079d8a9a5ad9c1f5d88409c04f6a13ba44666a756c6ea8ff7bd4d25cb9a1528e"} Dec 03 11:56:52 crc kubenswrapper[5052]: I1203 11:56:52.959321 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/alertmanager-metric-storage-0" Dec 03 11:56:52 crc kubenswrapper[5052]: I1203 11:56:52.961355 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/alertmanager-metric-storage-0" Dec 03 11:56:52 crc kubenswrapper[5052]: I1203 11:56:52.980257 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/rabbitmq-server-0" podStartSLOduration=41.980234487 podStartE2EDuration="41.980234487s" podCreationTimestamp="2025-12-03 11:56:11 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:56:52.978981151 +0000 UTC m=+2203.567222925" watchObservedRunningTime="2025-12-03 11:56:52.980234487 +0000 UTC m=+2203.568476251" Dec 03 11:56:53 crc kubenswrapper[5052]: I1203 11:56:53.010816 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/alertmanager-metric-storage-0" podStartSLOduration=9.752218286 podStartE2EDuration="28.010795201s" podCreationTimestamp="2025-12-03 11:56:25 +0000 UTC" firstStartedPulling="2025-12-03 11:56:26.393166302 +0000 UTC m=+2176.981408066" lastFinishedPulling="2025-12-03 11:56:44.651743217 +0000 UTC m=+2195.239984981" observedRunningTime="2025-12-03 11:56:53.006613462 +0000 UTC m=+2203.594855246" watchObservedRunningTime="2025-12-03 11:56:53.010795201 +0000 UTC m=+2203.599036965" Dec 03 11:56:53 crc kubenswrapper[5052]: I1203 11:56:53.975106 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"7c36839d-8518-4b0e-bf63-ec94d92e3c13","Type":"ContainerStarted","Data":"eac74c2a0f154d6d834ce77005c3de792b1ffb15c4c79891dc165810f2f43317"} Dec 03 11:56:53 crc kubenswrapper[5052]: I1203 11:56:53.975418 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"7c36839d-8518-4b0e-bf63-ec94d92e3c13","Type":"ContainerStarted","Data":"52442374ae2d8fc1cfd375a7ed3be0f18021f36b0334d1758b377586b846921d"} Dec 03 11:56:54 crc kubenswrapper[5052]: I1203 11:56:54.180772 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/swift-storage-0" podStartSLOduration=28.180751103 podStartE2EDuration="28.180751103s" podCreationTimestamp="2025-12-03 11:56:26 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:56:54.021069284 +0000 UTC m=+2204.609311058" watchObservedRunningTime="2025-12-03 11:56:54.180751103 +0000 UTC m=+2204.768992877" Dec 03 11:56:54 crc kubenswrapper[5052]: I1203 11:56:54.189817 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-b6ccb55c5-4rjkw"] Dec 03 11:56:54 crc kubenswrapper[5052]: E1203 11:56:54.190206 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="6eda40cd-e0c7-4f9c-922a-e66672da99ed" containerName="mariadb-account-create-update" Dec 03 11:56:54 crc kubenswrapper[5052]: I1203 11:56:54.190229 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="6eda40cd-e0c7-4f9c-922a-e66672da99ed" containerName="mariadb-account-create-update" Dec 03 11:56:54 crc kubenswrapper[5052]: E1203 11:56:54.190245 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="fce46734-b4c7-4e69-9010-d23e2d5f372f" containerName="mariadb-database-create" Dec 03 11:56:54 crc kubenswrapper[5052]: I1203 11:56:54.190254 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="fce46734-b4c7-4e69-9010-d23e2d5f372f" containerName="mariadb-database-create" Dec 03 11:56:54 crc kubenswrapper[5052]: E1203 11:56:54.190287 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="534e7f12-9e45-4fd6-8bd4-dc712add1e1a" containerName="mariadb-database-create" Dec 03 11:56:54 crc kubenswrapper[5052]: I1203 11:56:54.190298 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="534e7f12-9e45-4fd6-8bd4-dc712add1e1a" containerName="mariadb-database-create" Dec 03 11:56:54 crc kubenswrapper[5052]: E1203 11:56:54.190317 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="e043d529-47c3-4741-85a3-bdb55c638a97" containerName="mariadb-account-create-update" Dec 03 11:56:54 crc kubenswrapper[5052]: I1203 11:56:54.190326 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="e043d529-47c3-4741-85a3-bdb55c638a97" containerName="mariadb-account-create-update" Dec 03 11:56:54 crc kubenswrapper[5052]: E1203 11:56:54.190343 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="b25492d8-45bc-446a-a60c-b4f732484218" containerName="swift-ring-rebalance" Dec 03 11:56:54 crc kubenswrapper[5052]: I1203 11:56:54.190351 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="b25492d8-45bc-446a-a60c-b4f732484218" containerName="swift-ring-rebalance" Dec 03 11:56:54 crc kubenswrapper[5052]: E1203 11:56:54.190372 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="fb1f8e74-cde9-4196-8b2d-5d302e69de58" containerName="mariadb-database-create" Dec 03 11:56:54 crc kubenswrapper[5052]: I1203 11:56:54.190379 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb1f8e74-cde9-4196-8b2d-5d302e69de58" containerName="mariadb-database-create" Dec 03 11:56:54 crc kubenswrapper[5052]: I1203 11:56:54.190596 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="b25492d8-45bc-446a-a60c-b4f732484218" containerName="swift-ring-rebalance" Dec 03 11:56:54 crc kubenswrapper[5052]: I1203 11:56:54.190630 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="fb1f8e74-cde9-4196-8b2d-5d302e69de58" containerName="mariadb-database-create" Dec 03 11:56:54 crc kubenswrapper[5052]: I1203 11:56:54.190649 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="e043d529-47c3-4741-85a3-bdb55c638a97" containerName="mariadb-account-create-update" Dec 03 11:56:54 crc kubenswrapper[5052]: I1203 11:56:54.190669 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="fce46734-b4c7-4e69-9010-d23e2d5f372f" containerName="mariadb-database-create" Dec 03 11:56:54 crc kubenswrapper[5052]: I1203 11:56:54.190693 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="6eda40cd-e0c7-4f9c-922a-e66672da99ed" containerName="mariadb-account-create-update" Dec 03 11:56:54 crc kubenswrapper[5052]: I1203 11:56:54.190711 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="534e7f12-9e45-4fd6-8bd4-dc712add1e1a" containerName="mariadb-database-create" Dec 03 11:56:54 crc kubenswrapper[5052]: I1203 11:56:54.191886 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-b6ccb55c5-4rjkw" Dec 03 11:56:54 crc kubenswrapper[5052]: I1203 11:56:54.195021 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"dns-swift-storage-0" Dec 03 11:56:54 crc kubenswrapper[5052]: I1203 11:56:54.201930 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-b6ccb55c5-4rjkw"] Dec 03 11:56:54 crc kubenswrapper[5052]: I1203 11:56:54.300918 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/10e87ef1-40ac-4691-a6ef-5f91352e85e7-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-b6ccb55c5-4rjkw\" (UID: \"10e87ef1-40ac-4691-a6ef-5f91352e85e7\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-b6ccb55c5-4rjkw" Dec 03 11:56:54 crc kubenswrapper[5052]: I1203 11:56:54.300987 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dns-swift-storage-0\" (UniqueName: \"kubernetes.io/configmap/10e87ef1-40ac-4691-a6ef-5f91352e85e7-dns-swift-storage-0\") pod \"dnsmasq-dnsmasq-b6ccb55c5-4rjkw\" (UID: \"10e87ef1-40ac-4691-a6ef-5f91352e85e7\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-b6ccb55c5-4rjkw" Dec 03 11:56:54 crc kubenswrapper[5052]: I1203 11:56:54.301023 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2z4h9\" (UniqueName: \"kubernetes.io/projected/10e87ef1-40ac-4691-a6ef-5f91352e85e7-kube-api-access-2z4h9\") pod \"dnsmasq-dnsmasq-b6ccb55c5-4rjkw\" (UID: \"10e87ef1-40ac-4691-a6ef-5f91352e85e7\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-b6ccb55c5-4rjkw" Dec 03 11:56:54 crc kubenswrapper[5052]: I1203 11:56:54.301184 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/10e87ef1-40ac-4691-a6ef-5f91352e85e7-config\") pod \"dnsmasq-dnsmasq-b6ccb55c5-4rjkw\" (UID: \"10e87ef1-40ac-4691-a6ef-5f91352e85e7\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-b6ccb55c5-4rjkw" Dec 03 11:56:54 crc kubenswrapper[5052]: I1203 11:56:54.410650 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/10e87ef1-40ac-4691-a6ef-5f91352e85e7-config\") pod \"dnsmasq-dnsmasq-b6ccb55c5-4rjkw\" (UID: \"10e87ef1-40ac-4691-a6ef-5f91352e85e7\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-b6ccb55c5-4rjkw" Dec 03 11:56:54 crc kubenswrapper[5052]: I1203 11:56:54.410756 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/10e87ef1-40ac-4691-a6ef-5f91352e85e7-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-b6ccb55c5-4rjkw\" (UID: \"10e87ef1-40ac-4691-a6ef-5f91352e85e7\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-b6ccb55c5-4rjkw" Dec 03 11:56:54 crc kubenswrapper[5052]: I1203 11:56:54.410792 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"dns-swift-storage-0\" (UniqueName: \"kubernetes.io/configmap/10e87ef1-40ac-4691-a6ef-5f91352e85e7-dns-swift-storage-0\") pod \"dnsmasq-dnsmasq-b6ccb55c5-4rjkw\" (UID: \"10e87ef1-40ac-4691-a6ef-5f91352e85e7\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-b6ccb55c5-4rjkw" Dec 03 11:56:54 crc kubenswrapper[5052]: I1203 11:56:54.410835 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-2z4h9\" (UniqueName: \"kubernetes.io/projected/10e87ef1-40ac-4691-a6ef-5f91352e85e7-kube-api-access-2z4h9\") pod \"dnsmasq-dnsmasq-b6ccb55c5-4rjkw\" (UID: \"10e87ef1-40ac-4691-a6ef-5f91352e85e7\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-b6ccb55c5-4rjkw" Dec 03 11:56:54 crc kubenswrapper[5052]: I1203 11:56:54.411547 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/10e87ef1-40ac-4691-a6ef-5f91352e85e7-config\") pod \"dnsmasq-dnsmasq-b6ccb55c5-4rjkw\" (UID: \"10e87ef1-40ac-4691-a6ef-5f91352e85e7\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-b6ccb55c5-4rjkw" Dec 03 11:56:54 crc kubenswrapper[5052]: I1203 11:56:54.411639 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"dns-swift-storage-0\" (UniqueName: \"kubernetes.io/configmap/10e87ef1-40ac-4691-a6ef-5f91352e85e7-dns-swift-storage-0\") pod \"dnsmasq-dnsmasq-b6ccb55c5-4rjkw\" (UID: \"10e87ef1-40ac-4691-a6ef-5f91352e85e7\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-b6ccb55c5-4rjkw" Dec 03 11:56:54 crc kubenswrapper[5052]: I1203 11:56:54.412196 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/10e87ef1-40ac-4691-a6ef-5f91352e85e7-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-b6ccb55c5-4rjkw\" (UID: \"10e87ef1-40ac-4691-a6ef-5f91352e85e7\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-b6ccb55c5-4rjkw" Dec 03 11:56:54 crc kubenswrapper[5052]: I1203 11:56:54.446004 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-2z4h9\" (UniqueName: \"kubernetes.io/projected/10e87ef1-40ac-4691-a6ef-5f91352e85e7-kube-api-access-2z4h9\") pod \"dnsmasq-dnsmasq-b6ccb55c5-4rjkw\" (UID: \"10e87ef1-40ac-4691-a6ef-5f91352e85e7\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-b6ccb55c5-4rjkw" Dec 03 11:56:54 crc kubenswrapper[5052]: I1203 11:56:54.553817 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-b6ccb55c5-4rjkw" Dec 03 11:56:54 crc kubenswrapper[5052]: I1203 11:56:54.693352 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/glance-db-sync-kfnjj"] Dec 03 11:56:54 crc kubenswrapper[5052]: I1203 11:56:54.694583 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-db-sync-kfnjj" Dec 03 11:56:54 crc kubenswrapper[5052]: I1203 11:56:54.702173 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-config-data" Dec 03 11:56:54 crc kubenswrapper[5052]: I1203 11:56:54.706284 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-glance-dockercfg-8nxm6" Dec 03 11:56:54 crc kubenswrapper[5052]: I1203 11:56:54.716690 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-db-sync-kfnjj"] Dec 03 11:56:54 crc kubenswrapper[5052]: I1203 11:56:54.816813 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/a626fb80-9d64-44c4-94e7-e0dde9638ffc-db-sync-config-data\") pod \"glance-db-sync-kfnjj\" (UID: \"a626fb80-9d64-44c4-94e7-e0dde9638ffc\") " pod="openstack-kuttl-tests/glance-db-sync-kfnjj" Dec 03 11:56:54 crc kubenswrapper[5052]: I1203 11:56:54.816857 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a626fb80-9d64-44c4-94e7-e0dde9638ffc-config-data\") pod \"glance-db-sync-kfnjj\" (UID: \"a626fb80-9d64-44c4-94e7-e0dde9638ffc\") " pod="openstack-kuttl-tests/glance-db-sync-kfnjj" Dec 03 11:56:54 crc kubenswrapper[5052]: I1203 11:56:54.816893 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a626fb80-9d64-44c4-94e7-e0dde9638ffc-combined-ca-bundle\") pod \"glance-db-sync-kfnjj\" (UID: \"a626fb80-9d64-44c4-94e7-e0dde9638ffc\") " pod="openstack-kuttl-tests/glance-db-sync-kfnjj" Dec 03 11:56:54 crc kubenswrapper[5052]: I1203 11:56:54.816926 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hgg7z\" (UniqueName: \"kubernetes.io/projected/a626fb80-9d64-44c4-94e7-e0dde9638ffc-kube-api-access-hgg7z\") pod \"glance-db-sync-kfnjj\" (UID: \"a626fb80-9d64-44c4-94e7-e0dde9638ffc\") " pod="openstack-kuttl-tests/glance-db-sync-kfnjj" Dec 03 11:56:54 crc kubenswrapper[5052]: I1203 11:56:54.918388 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/a626fb80-9d64-44c4-94e7-e0dde9638ffc-db-sync-config-data\") pod \"glance-db-sync-kfnjj\" (UID: \"a626fb80-9d64-44c4-94e7-e0dde9638ffc\") " pod="openstack-kuttl-tests/glance-db-sync-kfnjj" Dec 03 11:56:54 crc kubenswrapper[5052]: I1203 11:56:54.918439 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a626fb80-9d64-44c4-94e7-e0dde9638ffc-config-data\") pod \"glance-db-sync-kfnjj\" (UID: \"a626fb80-9d64-44c4-94e7-e0dde9638ffc\") " pod="openstack-kuttl-tests/glance-db-sync-kfnjj" Dec 03 11:56:54 crc kubenswrapper[5052]: I1203 11:56:54.918581 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a626fb80-9d64-44c4-94e7-e0dde9638ffc-combined-ca-bundle\") pod \"glance-db-sync-kfnjj\" (UID: \"a626fb80-9d64-44c4-94e7-e0dde9638ffc\") " pod="openstack-kuttl-tests/glance-db-sync-kfnjj" Dec 03 11:56:54 crc kubenswrapper[5052]: I1203 11:56:54.918664 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-hgg7z\" (UniqueName: \"kubernetes.io/projected/a626fb80-9d64-44c4-94e7-e0dde9638ffc-kube-api-access-hgg7z\") pod \"glance-db-sync-kfnjj\" (UID: \"a626fb80-9d64-44c4-94e7-e0dde9638ffc\") " pod="openstack-kuttl-tests/glance-db-sync-kfnjj" Dec 03 11:56:54 crc kubenswrapper[5052]: I1203 11:56:54.924373 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a626fb80-9d64-44c4-94e7-e0dde9638ffc-config-data\") pod \"glance-db-sync-kfnjj\" (UID: \"a626fb80-9d64-44c4-94e7-e0dde9638ffc\") " pod="openstack-kuttl-tests/glance-db-sync-kfnjj" Dec 03 11:56:54 crc kubenswrapper[5052]: I1203 11:56:54.925456 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/a626fb80-9d64-44c4-94e7-e0dde9638ffc-db-sync-config-data\") pod \"glance-db-sync-kfnjj\" (UID: \"a626fb80-9d64-44c4-94e7-e0dde9638ffc\") " pod="openstack-kuttl-tests/glance-db-sync-kfnjj" Dec 03 11:56:54 crc kubenswrapper[5052]: I1203 11:56:54.938720 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a626fb80-9d64-44c4-94e7-e0dde9638ffc-combined-ca-bundle\") pod \"glance-db-sync-kfnjj\" (UID: \"a626fb80-9d64-44c4-94e7-e0dde9638ffc\") " pod="openstack-kuttl-tests/glance-db-sync-kfnjj" Dec 03 11:56:54 crc kubenswrapper[5052]: I1203 11:56:54.943100 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-hgg7z\" (UniqueName: \"kubernetes.io/projected/a626fb80-9d64-44c4-94e7-e0dde9638ffc-kube-api-access-hgg7z\") pod \"glance-db-sync-kfnjj\" (UID: \"a626fb80-9d64-44c4-94e7-e0dde9638ffc\") " pod="openstack-kuttl-tests/glance-db-sync-kfnjj" Dec 03 11:56:55 crc kubenswrapper[5052]: I1203 11:56:55.015998 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-db-sync-kfnjj" Dec 03 11:56:56 crc kubenswrapper[5052]: I1203 11:56:56.207627 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-b6ccb55c5-4rjkw"] Dec 03 11:56:56 crc kubenswrapper[5052]: I1203 11:56:56.481055 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-db-sync-kfnjj"] Dec 03 11:56:56 crc kubenswrapper[5052]: W1203 11:56:56.486475 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda626fb80_9d64_44c4_94e7_e0dde9638ffc.slice/crio-3997ea686cf55000226517c5cd86449112f31b0e7c3d79426a62c193e75c84ae WatchSource:0}: Error finding container 3997ea686cf55000226517c5cd86449112f31b0e7c3d79426a62c193e75c84ae: Status 404 returned error can't find the container with id 3997ea686cf55000226517c5cd86449112f31b0e7c3d79426a62c193e75c84ae Dec 03 11:56:57 crc kubenswrapper[5052]: I1203 11:56:57.000711 5052 generic.go:334] "Generic (PLEG): container finished" podID="10e87ef1-40ac-4691-a6ef-5f91352e85e7" containerID="23eaf5d19434696e2ee9e76a1ffe87942aaa033ee35702b7b274a64f72ffa07f" exitCode=0 Dec 03 11:56:57 crc kubenswrapper[5052]: I1203 11:56:57.000794 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-b6ccb55c5-4rjkw" event={"ID":"10e87ef1-40ac-4691-a6ef-5f91352e85e7","Type":"ContainerDied","Data":"23eaf5d19434696e2ee9e76a1ffe87942aaa033ee35702b7b274a64f72ffa07f"} Dec 03 11:56:57 crc kubenswrapper[5052]: I1203 11:56:57.000822 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-b6ccb55c5-4rjkw" event={"ID":"10e87ef1-40ac-4691-a6ef-5f91352e85e7","Type":"ContainerStarted","Data":"f7eba0d2677b6143975774d22746fe1ae4a84b5d06485c4469e16c1463c80800"} Dec 03 11:56:57 crc kubenswrapper[5052]: I1203 11:56:57.002286 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-db-sync-kfnjj" event={"ID":"a626fb80-9d64-44c4-94e7-e0dde9638ffc","Type":"ContainerStarted","Data":"3997ea686cf55000226517c5cd86449112f31b0e7c3d79426a62c193e75c84ae"} Dec 03 11:56:57 crc kubenswrapper[5052]: I1203 11:56:57.005535 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/prometheus-metric-storage-0" event={"ID":"33918bc1-340a-4d26-a204-001b85c15f04","Type":"ContainerStarted","Data":"338f13d17f6b49a2ab7b8705345e66ade7dfcb2716d85396b3217b2553ae2ae2"} Dec 03 11:56:57 crc kubenswrapper[5052]: I1203 11:56:57.070392 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/prometheus-metric-storage-0" podStartSLOduration=3.905160483 podStartE2EDuration="33.070366301s" podCreationTimestamp="2025-12-03 11:56:24 +0000 UTC" firstStartedPulling="2025-12-03 11:56:26.847160904 +0000 UTC m=+2177.435402668" lastFinishedPulling="2025-12-03 11:56:56.012366722 +0000 UTC m=+2206.600608486" observedRunningTime="2025-12-03 11:56:57.06162005 +0000 UTC m=+2207.649861824" watchObservedRunningTime="2025-12-03 11:56:57.070366301 +0000 UTC m=+2207.658608065" Dec 03 11:56:58 crc kubenswrapper[5052]: I1203 11:56:58.015116 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-b6ccb55c5-4rjkw" event={"ID":"10e87ef1-40ac-4691-a6ef-5f91352e85e7","Type":"ContainerStarted","Data":"b8cd94166a15570855e1df2776b78818e8ab0111af82a51c3cc7ddaffca1dc22"} Dec 03 11:56:58 crc kubenswrapper[5052]: I1203 11:56:58.015781 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-b6ccb55c5-4rjkw" Dec 03 11:56:58 crc kubenswrapper[5052]: I1203 11:56:58.016721 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-db-sync-kfnjj" event={"ID":"a626fb80-9d64-44c4-94e7-e0dde9638ffc","Type":"ContainerStarted","Data":"608e4a1a02f619ff83a69532ecf951dbd52836921fd0e850e8efab96f9af411b"} Dec 03 11:56:58 crc kubenswrapper[5052]: I1203 11:56:58.044024 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-b6ccb55c5-4rjkw" podStartSLOduration=4.044003884 podStartE2EDuration="4.044003884s" podCreationTimestamp="2025-12-03 11:56:54 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:56:58.042707347 +0000 UTC m=+2208.630949221" watchObservedRunningTime="2025-12-03 11:56:58.044003884 +0000 UTC m=+2208.632245648" Dec 03 11:56:58 crc kubenswrapper[5052]: I1203 11:56:58.059647 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/glance-db-sync-kfnjj" podStartSLOduration=4.059624461 podStartE2EDuration="4.059624461s" podCreationTimestamp="2025-12-03 11:56:54 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:56:58.056925284 +0000 UTC m=+2208.645167048" watchObservedRunningTime="2025-12-03 11:56:58.059624461 +0000 UTC m=+2208.647866225" Dec 03 11:56:59 crc kubenswrapper[5052]: I1203 11:56:59.379814 5052 patch_prober.go:28] interesting pod/machine-config-daemon-74bj6 container/machine-config-daemon namespace/openshift-machine-config-operator: Liveness probe status=failure output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" start-of-body= Dec 03 11:56:59 crc kubenswrapper[5052]: I1203 11:56:59.379880 5052 prober.go:107] "Probe failed" probeType="Liveness" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" containerName="machine-config-daemon" probeResult="failure" output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" Dec 03 11:57:01 crc kubenswrapper[5052]: I1203 11:57:01.313608 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/prometheus-metric-storage-0" Dec 03 11:57:02 crc kubenswrapper[5052]: I1203 11:57:02.638576 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 11:57:02 crc kubenswrapper[5052]: I1203 11:57:02.947088 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/watcher-db-sync-chlm4"] Dec 03 11:57:02 crc kubenswrapper[5052]: I1203 11:57:02.948802 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/watcher-db-sync-chlm4" Dec 03 11:57:02 crc kubenswrapper[5052]: I1203 11:57:02.951077 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"watcher-watcher-dockercfg-q8qv2" Dec 03 11:57:02 crc kubenswrapper[5052]: I1203 11:57:02.963929 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/watcher-db-sync-chlm4"] Dec 03 11:57:02 crc kubenswrapper[5052]: I1203 11:57:02.964821 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"watcher-config-data" Dec 03 11:57:02 crc kubenswrapper[5052]: I1203 11:57:02.975222 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/cinder-db-create-gx2gn"] Dec 03 11:57:02 crc kubenswrapper[5052]: I1203 11:57:02.982581 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-db-create-gx2gn" Dec 03 11:57:03 crc kubenswrapper[5052]: I1203 11:57:03.037204 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-db-create-gx2gn"] Dec 03 11:57:03 crc kubenswrapper[5052]: I1203 11:57:03.058409 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-29fml\" (UniqueName: \"kubernetes.io/projected/b3146921-cc78-401f-a918-fa8b91814f10-kube-api-access-29fml\") pod \"watcher-db-sync-chlm4\" (UID: \"b3146921-cc78-401f-a918-fa8b91814f10\") " pod="openstack-kuttl-tests/watcher-db-sync-chlm4" Dec 03 11:57:03 crc kubenswrapper[5052]: I1203 11:57:03.058474 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-scblb\" (UniqueName: \"kubernetes.io/projected/ace50fc0-2d6c-4014-b576-0da48b39b4a5-kube-api-access-scblb\") pod \"cinder-db-create-gx2gn\" (UID: \"ace50fc0-2d6c-4014-b576-0da48b39b4a5\") " pod="openstack-kuttl-tests/cinder-db-create-gx2gn" Dec 03 11:57:03 crc kubenswrapper[5052]: I1203 11:57:03.058564 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/b3146921-cc78-401f-a918-fa8b91814f10-config-data\") pod \"watcher-db-sync-chlm4\" (UID: \"b3146921-cc78-401f-a918-fa8b91814f10\") " pod="openstack-kuttl-tests/watcher-db-sync-chlm4" Dec 03 11:57:03 crc kubenswrapper[5052]: I1203 11:57:03.058607 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b3146921-cc78-401f-a918-fa8b91814f10-combined-ca-bundle\") pod \"watcher-db-sync-chlm4\" (UID: \"b3146921-cc78-401f-a918-fa8b91814f10\") " pod="openstack-kuttl-tests/watcher-db-sync-chlm4" Dec 03 11:57:03 crc kubenswrapper[5052]: I1203 11:57:03.058651 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/ace50fc0-2d6c-4014-b576-0da48b39b4a5-operator-scripts\") pod \"cinder-db-create-gx2gn\" (UID: \"ace50fc0-2d6c-4014-b576-0da48b39b4a5\") " pod="openstack-kuttl-tests/cinder-db-create-gx2gn" Dec 03 11:57:03 crc kubenswrapper[5052]: I1203 11:57:03.058667 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/b3146921-cc78-401f-a918-fa8b91814f10-db-sync-config-data\") pod \"watcher-db-sync-chlm4\" (UID: \"b3146921-cc78-401f-a918-fa8b91814f10\") " pod="openstack-kuttl-tests/watcher-db-sync-chlm4" Dec 03 11:57:03 crc kubenswrapper[5052]: I1203 11:57:03.086071 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/barbican-db-create-tl9wz"] Dec 03 11:57:03 crc kubenswrapper[5052]: I1203 11:57:03.087205 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-db-create-tl9wz" Dec 03 11:57:03 crc kubenswrapper[5052]: I1203 11:57:03.096232 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-db-create-tl9wz"] Dec 03 11:57:03 crc kubenswrapper[5052]: I1203 11:57:03.164378 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/barbican-4aeb-account-create-update-hn4jf"] Dec 03 11:57:03 crc kubenswrapper[5052]: I1203 11:57:03.170030 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-29fml\" (UniqueName: \"kubernetes.io/projected/b3146921-cc78-401f-a918-fa8b91814f10-kube-api-access-29fml\") pod \"watcher-db-sync-chlm4\" (UID: \"b3146921-cc78-401f-a918-fa8b91814f10\") " pod="openstack-kuttl-tests/watcher-db-sync-chlm4" Dec 03 11:57:03 crc kubenswrapper[5052]: I1203 11:57:03.170090 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-scblb\" (UniqueName: \"kubernetes.io/projected/ace50fc0-2d6c-4014-b576-0da48b39b4a5-kube-api-access-scblb\") pod \"cinder-db-create-gx2gn\" (UID: \"ace50fc0-2d6c-4014-b576-0da48b39b4a5\") " pod="openstack-kuttl-tests/cinder-db-create-gx2gn" Dec 03 11:57:03 crc kubenswrapper[5052]: I1203 11:57:03.170292 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-clzxx\" (UniqueName: \"kubernetes.io/projected/7b231c94-d92f-49e9-8ae1-93b2cc125936-kube-api-access-clzxx\") pod \"barbican-db-create-tl9wz\" (UID: \"7b231c94-d92f-49e9-8ae1-93b2cc125936\") " pod="openstack-kuttl-tests/barbican-db-create-tl9wz" Dec 03 11:57:03 crc kubenswrapper[5052]: I1203 11:57:03.170368 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/b3146921-cc78-401f-a918-fa8b91814f10-config-data\") pod \"watcher-db-sync-chlm4\" (UID: \"b3146921-cc78-401f-a918-fa8b91814f10\") " pod="openstack-kuttl-tests/watcher-db-sync-chlm4" Dec 03 11:57:03 crc kubenswrapper[5052]: I1203 11:57:03.171296 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/7b231c94-d92f-49e9-8ae1-93b2cc125936-operator-scripts\") pod \"barbican-db-create-tl9wz\" (UID: \"7b231c94-d92f-49e9-8ae1-93b2cc125936\") " pod="openstack-kuttl-tests/barbican-db-create-tl9wz" Dec 03 11:57:03 crc kubenswrapper[5052]: I1203 11:57:03.171329 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b3146921-cc78-401f-a918-fa8b91814f10-combined-ca-bundle\") pod \"watcher-db-sync-chlm4\" (UID: \"b3146921-cc78-401f-a918-fa8b91814f10\") " pod="openstack-kuttl-tests/watcher-db-sync-chlm4" Dec 03 11:57:03 crc kubenswrapper[5052]: I1203 11:57:03.171395 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/ace50fc0-2d6c-4014-b576-0da48b39b4a5-operator-scripts\") pod \"cinder-db-create-gx2gn\" (UID: \"ace50fc0-2d6c-4014-b576-0da48b39b4a5\") " pod="openstack-kuttl-tests/cinder-db-create-gx2gn" Dec 03 11:57:03 crc kubenswrapper[5052]: I1203 11:57:03.171413 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/b3146921-cc78-401f-a918-fa8b91814f10-db-sync-config-data\") pod \"watcher-db-sync-chlm4\" (UID: \"b3146921-cc78-401f-a918-fa8b91814f10\") " pod="openstack-kuttl-tests/watcher-db-sync-chlm4" Dec 03 11:57:03 crc kubenswrapper[5052]: I1203 11:57:03.171736 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-4aeb-account-create-update-hn4jf" Dec 03 11:57:03 crc kubenswrapper[5052]: I1203 11:57:03.173070 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/ace50fc0-2d6c-4014-b576-0da48b39b4a5-operator-scripts\") pod \"cinder-db-create-gx2gn\" (UID: \"ace50fc0-2d6c-4014-b576-0da48b39b4a5\") " pod="openstack-kuttl-tests/cinder-db-create-gx2gn" Dec 03 11:57:03 crc kubenswrapper[5052]: I1203 11:57:03.174384 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"barbican-db-secret" Dec 03 11:57:03 crc kubenswrapper[5052]: I1203 11:57:03.175810 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/b3146921-cc78-401f-a918-fa8b91814f10-db-sync-config-data\") pod \"watcher-db-sync-chlm4\" (UID: \"b3146921-cc78-401f-a918-fa8b91814f10\") " pod="openstack-kuttl-tests/watcher-db-sync-chlm4" Dec 03 11:57:03 crc kubenswrapper[5052]: I1203 11:57:03.175929 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/b3146921-cc78-401f-a918-fa8b91814f10-config-data\") pod \"watcher-db-sync-chlm4\" (UID: \"b3146921-cc78-401f-a918-fa8b91814f10\") " pod="openstack-kuttl-tests/watcher-db-sync-chlm4" Dec 03 11:57:03 crc kubenswrapper[5052]: I1203 11:57:03.176159 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b3146921-cc78-401f-a918-fa8b91814f10-combined-ca-bundle\") pod \"watcher-db-sync-chlm4\" (UID: \"b3146921-cc78-401f-a918-fa8b91814f10\") " pod="openstack-kuttl-tests/watcher-db-sync-chlm4" Dec 03 11:57:03 crc kubenswrapper[5052]: I1203 11:57:03.202039 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-scblb\" (UniqueName: \"kubernetes.io/projected/ace50fc0-2d6c-4014-b576-0da48b39b4a5-kube-api-access-scblb\") pod \"cinder-db-create-gx2gn\" (UID: \"ace50fc0-2d6c-4014-b576-0da48b39b4a5\") " pod="openstack-kuttl-tests/cinder-db-create-gx2gn" Dec 03 11:57:03 crc kubenswrapper[5052]: I1203 11:57:03.205719 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-29fml\" (UniqueName: \"kubernetes.io/projected/b3146921-cc78-401f-a918-fa8b91814f10-kube-api-access-29fml\") pod \"watcher-db-sync-chlm4\" (UID: \"b3146921-cc78-401f-a918-fa8b91814f10\") " pod="openstack-kuttl-tests/watcher-db-sync-chlm4" Dec 03 11:57:03 crc kubenswrapper[5052]: I1203 11:57:03.213283 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-4aeb-account-create-update-hn4jf"] Dec 03 11:57:03 crc kubenswrapper[5052]: I1203 11:57:03.247939 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/keystone-db-sync-2jm7f"] Dec 03 11:57:03 crc kubenswrapper[5052]: I1203 11:57:03.249254 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-db-sync-2jm7f" Dec 03 11:57:03 crc kubenswrapper[5052]: I1203 11:57:03.251298 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone-config-data" Dec 03 11:57:03 crc kubenswrapper[5052]: I1203 11:57:03.253332 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone-keystone-dockercfg-csztk" Dec 03 11:57:03 crc kubenswrapper[5052]: I1203 11:57:03.253607 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone" Dec 03 11:57:03 crc kubenswrapper[5052]: I1203 11:57:03.253724 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone-scripts" Dec 03 11:57:03 crc kubenswrapper[5052]: I1203 11:57:03.273542 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/638e6ada-710e-43ce-9ac8-0569396886b3-operator-scripts\") pod \"barbican-4aeb-account-create-update-hn4jf\" (UID: \"638e6ada-710e-43ce-9ac8-0569396886b3\") " pod="openstack-kuttl-tests/barbican-4aeb-account-create-update-hn4jf" Dec 03 11:57:03 crc kubenswrapper[5052]: I1203 11:57:03.273667 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-clzxx\" (UniqueName: \"kubernetes.io/projected/7b231c94-d92f-49e9-8ae1-93b2cc125936-kube-api-access-clzxx\") pod \"barbican-db-create-tl9wz\" (UID: \"7b231c94-d92f-49e9-8ae1-93b2cc125936\") " pod="openstack-kuttl-tests/barbican-db-create-tl9wz" Dec 03 11:57:03 crc kubenswrapper[5052]: I1203 11:57:03.273713 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jhmqr\" (UniqueName: \"kubernetes.io/projected/638e6ada-710e-43ce-9ac8-0569396886b3-kube-api-access-jhmqr\") pod \"barbican-4aeb-account-create-update-hn4jf\" (UID: \"638e6ada-710e-43ce-9ac8-0569396886b3\") " pod="openstack-kuttl-tests/barbican-4aeb-account-create-update-hn4jf" Dec 03 11:57:03 crc kubenswrapper[5052]: I1203 11:57:03.273789 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/7b231c94-d92f-49e9-8ae1-93b2cc125936-operator-scripts\") pod \"barbican-db-create-tl9wz\" (UID: \"7b231c94-d92f-49e9-8ae1-93b2cc125936\") " pod="openstack-kuttl-tests/barbican-db-create-tl9wz" Dec 03 11:57:03 crc kubenswrapper[5052]: I1203 11:57:03.274789 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/watcher-db-sync-chlm4" Dec 03 11:57:03 crc kubenswrapper[5052]: I1203 11:57:03.274937 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/7b231c94-d92f-49e9-8ae1-93b2cc125936-operator-scripts\") pod \"barbican-db-create-tl9wz\" (UID: \"7b231c94-d92f-49e9-8ae1-93b2cc125936\") " pod="openstack-kuttl-tests/barbican-db-create-tl9wz" Dec 03 11:57:03 crc kubenswrapper[5052]: I1203 11:57:03.277736 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/neutron-db-create-2s4jv"] Dec 03 11:57:03 crc kubenswrapper[5052]: I1203 11:57:03.280861 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-db-create-2s4jv" Dec 03 11:57:03 crc kubenswrapper[5052]: I1203 11:57:03.308182 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-db-create-2s4jv"] Dec 03 11:57:03 crc kubenswrapper[5052]: I1203 11:57:03.314215 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-clzxx\" (UniqueName: \"kubernetes.io/projected/7b231c94-d92f-49e9-8ae1-93b2cc125936-kube-api-access-clzxx\") pod \"barbican-db-create-tl9wz\" (UID: \"7b231c94-d92f-49e9-8ae1-93b2cc125936\") " pod="openstack-kuttl-tests/barbican-db-create-tl9wz" Dec 03 11:57:03 crc kubenswrapper[5052]: I1203 11:57:03.321680 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-db-create-gx2gn" Dec 03 11:57:03 crc kubenswrapper[5052]: I1203 11:57:03.330787 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/cinder-aaa7-account-create-update-z472g"] Dec 03 11:57:03 crc kubenswrapper[5052]: I1203 11:57:03.341379 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-aaa7-account-create-update-z472g" Dec 03 11:57:03 crc kubenswrapper[5052]: I1203 11:57:03.354076 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cinder-db-secret" Dec 03 11:57:03 crc kubenswrapper[5052]: I1203 11:57:03.364665 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-aaa7-account-create-update-z472g"] Dec 03 11:57:03 crc kubenswrapper[5052]: I1203 11:57:03.377354 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/842a3b9b-c6e6-4032-926a-95cc59d990f3-operator-scripts\") pod \"neutron-db-create-2s4jv\" (UID: \"842a3b9b-c6e6-4032-926a-95cc59d990f3\") " pod="openstack-kuttl-tests/neutron-db-create-2s4jv" Dec 03 11:57:03 crc kubenswrapper[5052]: I1203 11:57:03.377463 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/7b1f895c-b513-4cc1-bf2a-72b84f1cd594-config-data\") pod \"keystone-db-sync-2jm7f\" (UID: \"7b1f895c-b513-4cc1-bf2a-72b84f1cd594\") " pod="openstack-kuttl-tests/keystone-db-sync-2jm7f" Dec 03 11:57:03 crc kubenswrapper[5052]: I1203 11:57:03.377503 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rqdbr\" (UniqueName: \"kubernetes.io/projected/7b1f895c-b513-4cc1-bf2a-72b84f1cd594-kube-api-access-rqdbr\") pod \"keystone-db-sync-2jm7f\" (UID: \"7b1f895c-b513-4cc1-bf2a-72b84f1cd594\") " pod="openstack-kuttl-tests/keystone-db-sync-2jm7f" Dec 03 11:57:03 crc kubenswrapper[5052]: I1203 11:57:03.377564 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f4c8m\" (UniqueName: \"kubernetes.io/projected/842a3b9b-c6e6-4032-926a-95cc59d990f3-kube-api-access-f4c8m\") pod \"neutron-db-create-2s4jv\" (UID: \"842a3b9b-c6e6-4032-926a-95cc59d990f3\") " pod="openstack-kuttl-tests/neutron-db-create-2s4jv" Dec 03 11:57:03 crc kubenswrapper[5052]: I1203 11:57:03.377596 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/71879aa1-8e5d-4f20-9f73-65637988af5c-operator-scripts\") pod \"cinder-aaa7-account-create-update-z472g\" (UID: \"71879aa1-8e5d-4f20-9f73-65637988af5c\") " pod="openstack-kuttl-tests/cinder-aaa7-account-create-update-z472g" Dec 03 11:57:03 crc kubenswrapper[5052]: I1203 11:57:03.377640 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/638e6ada-710e-43ce-9ac8-0569396886b3-operator-scripts\") pod \"barbican-4aeb-account-create-update-hn4jf\" (UID: \"638e6ada-710e-43ce-9ac8-0569396886b3\") " pod="openstack-kuttl-tests/barbican-4aeb-account-create-update-hn4jf" Dec 03 11:57:03 crc kubenswrapper[5052]: I1203 11:57:03.377743 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/7b1f895c-b513-4cc1-bf2a-72b84f1cd594-combined-ca-bundle\") pod \"keystone-db-sync-2jm7f\" (UID: \"7b1f895c-b513-4cc1-bf2a-72b84f1cd594\") " pod="openstack-kuttl-tests/keystone-db-sync-2jm7f" Dec 03 11:57:03 crc kubenswrapper[5052]: I1203 11:57:03.377799 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x28f8\" (UniqueName: \"kubernetes.io/projected/71879aa1-8e5d-4f20-9f73-65637988af5c-kube-api-access-x28f8\") pod \"cinder-aaa7-account-create-update-z472g\" (UID: \"71879aa1-8e5d-4f20-9f73-65637988af5c\") " pod="openstack-kuttl-tests/cinder-aaa7-account-create-update-z472g" Dec 03 11:57:03 crc kubenswrapper[5052]: I1203 11:57:03.377850 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-jhmqr\" (UniqueName: \"kubernetes.io/projected/638e6ada-710e-43ce-9ac8-0569396886b3-kube-api-access-jhmqr\") pod \"barbican-4aeb-account-create-update-hn4jf\" (UID: \"638e6ada-710e-43ce-9ac8-0569396886b3\") " pod="openstack-kuttl-tests/barbican-4aeb-account-create-update-hn4jf" Dec 03 11:57:03 crc kubenswrapper[5052]: I1203 11:57:03.379268 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/638e6ada-710e-43ce-9ac8-0569396886b3-operator-scripts\") pod \"barbican-4aeb-account-create-update-hn4jf\" (UID: \"638e6ada-710e-43ce-9ac8-0569396886b3\") " pod="openstack-kuttl-tests/barbican-4aeb-account-create-update-hn4jf" Dec 03 11:57:03 crc kubenswrapper[5052]: I1203 11:57:03.379734 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-db-sync-2jm7f"] Dec 03 11:57:03 crc kubenswrapper[5052]: I1203 11:57:03.402349 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-jhmqr\" (UniqueName: \"kubernetes.io/projected/638e6ada-710e-43ce-9ac8-0569396886b3-kube-api-access-jhmqr\") pod \"barbican-4aeb-account-create-update-hn4jf\" (UID: \"638e6ada-710e-43ce-9ac8-0569396886b3\") " pod="openstack-kuttl-tests/barbican-4aeb-account-create-update-hn4jf" Dec 03 11:57:03 crc kubenswrapper[5052]: I1203 11:57:03.406259 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-db-create-tl9wz" Dec 03 11:57:03 crc kubenswrapper[5052]: I1203 11:57:03.479799 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-x28f8\" (UniqueName: \"kubernetes.io/projected/71879aa1-8e5d-4f20-9f73-65637988af5c-kube-api-access-x28f8\") pod \"cinder-aaa7-account-create-update-z472g\" (UID: \"71879aa1-8e5d-4f20-9f73-65637988af5c\") " pod="openstack-kuttl-tests/cinder-aaa7-account-create-update-z472g" Dec 03 11:57:03 crc kubenswrapper[5052]: I1203 11:57:03.480189 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/842a3b9b-c6e6-4032-926a-95cc59d990f3-operator-scripts\") pod \"neutron-db-create-2s4jv\" (UID: \"842a3b9b-c6e6-4032-926a-95cc59d990f3\") " pod="openstack-kuttl-tests/neutron-db-create-2s4jv" Dec 03 11:57:03 crc kubenswrapper[5052]: I1203 11:57:03.480277 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/7b1f895c-b513-4cc1-bf2a-72b84f1cd594-config-data\") pod \"keystone-db-sync-2jm7f\" (UID: \"7b1f895c-b513-4cc1-bf2a-72b84f1cd594\") " pod="openstack-kuttl-tests/keystone-db-sync-2jm7f" Dec 03 11:57:03 crc kubenswrapper[5052]: I1203 11:57:03.480314 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-rqdbr\" (UniqueName: \"kubernetes.io/projected/7b1f895c-b513-4cc1-bf2a-72b84f1cd594-kube-api-access-rqdbr\") pod \"keystone-db-sync-2jm7f\" (UID: \"7b1f895c-b513-4cc1-bf2a-72b84f1cd594\") " pod="openstack-kuttl-tests/keystone-db-sync-2jm7f" Dec 03 11:57:03 crc kubenswrapper[5052]: I1203 11:57:03.480337 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-f4c8m\" (UniqueName: \"kubernetes.io/projected/842a3b9b-c6e6-4032-926a-95cc59d990f3-kube-api-access-f4c8m\") pod \"neutron-db-create-2s4jv\" (UID: \"842a3b9b-c6e6-4032-926a-95cc59d990f3\") " pod="openstack-kuttl-tests/neutron-db-create-2s4jv" Dec 03 11:57:03 crc kubenswrapper[5052]: I1203 11:57:03.480365 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/71879aa1-8e5d-4f20-9f73-65637988af5c-operator-scripts\") pod \"cinder-aaa7-account-create-update-z472g\" (UID: \"71879aa1-8e5d-4f20-9f73-65637988af5c\") " pod="openstack-kuttl-tests/cinder-aaa7-account-create-update-z472g" Dec 03 11:57:03 crc kubenswrapper[5052]: I1203 11:57:03.480485 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/7b1f895c-b513-4cc1-bf2a-72b84f1cd594-combined-ca-bundle\") pod \"keystone-db-sync-2jm7f\" (UID: \"7b1f895c-b513-4cc1-bf2a-72b84f1cd594\") " pod="openstack-kuttl-tests/keystone-db-sync-2jm7f" Dec 03 11:57:03 crc kubenswrapper[5052]: I1203 11:57:03.484115 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/71879aa1-8e5d-4f20-9f73-65637988af5c-operator-scripts\") pod \"cinder-aaa7-account-create-update-z472g\" (UID: \"71879aa1-8e5d-4f20-9f73-65637988af5c\") " pod="openstack-kuttl-tests/cinder-aaa7-account-create-update-z472g" Dec 03 11:57:03 crc kubenswrapper[5052]: I1203 11:57:03.484195 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/842a3b9b-c6e6-4032-926a-95cc59d990f3-operator-scripts\") pod \"neutron-db-create-2s4jv\" (UID: \"842a3b9b-c6e6-4032-926a-95cc59d990f3\") " pod="openstack-kuttl-tests/neutron-db-create-2s4jv" Dec 03 11:57:03 crc kubenswrapper[5052]: I1203 11:57:03.485781 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/7b1f895c-b513-4cc1-bf2a-72b84f1cd594-combined-ca-bundle\") pod \"keystone-db-sync-2jm7f\" (UID: \"7b1f895c-b513-4cc1-bf2a-72b84f1cd594\") " pod="openstack-kuttl-tests/keystone-db-sync-2jm7f" Dec 03 11:57:03 crc kubenswrapper[5052]: I1203 11:57:03.486308 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/7b1f895c-b513-4cc1-bf2a-72b84f1cd594-config-data\") pod \"keystone-db-sync-2jm7f\" (UID: \"7b1f895c-b513-4cc1-bf2a-72b84f1cd594\") " pod="openstack-kuttl-tests/keystone-db-sync-2jm7f" Dec 03 11:57:03 crc kubenswrapper[5052]: I1203 11:57:03.517261 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/neutron-0ea0-account-create-update-p5z4j"] Dec 03 11:57:03 crc kubenswrapper[5052]: I1203 11:57:03.525118 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-0ea0-account-create-update-p5z4j" Dec 03 11:57:03 crc kubenswrapper[5052]: I1203 11:57:03.526955 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-rqdbr\" (UniqueName: \"kubernetes.io/projected/7b1f895c-b513-4cc1-bf2a-72b84f1cd594-kube-api-access-rqdbr\") pod \"keystone-db-sync-2jm7f\" (UID: \"7b1f895c-b513-4cc1-bf2a-72b84f1cd594\") " pod="openstack-kuttl-tests/keystone-db-sync-2jm7f" Dec 03 11:57:03 crc kubenswrapper[5052]: I1203 11:57:03.527158 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-f4c8m\" (UniqueName: \"kubernetes.io/projected/842a3b9b-c6e6-4032-926a-95cc59d990f3-kube-api-access-f4c8m\") pod \"neutron-db-create-2s4jv\" (UID: \"842a3b9b-c6e6-4032-926a-95cc59d990f3\") " pod="openstack-kuttl-tests/neutron-db-create-2s4jv" Dec 03 11:57:03 crc kubenswrapper[5052]: I1203 11:57:03.529537 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-x28f8\" (UniqueName: \"kubernetes.io/projected/71879aa1-8e5d-4f20-9f73-65637988af5c-kube-api-access-x28f8\") pod \"cinder-aaa7-account-create-update-z472g\" (UID: \"71879aa1-8e5d-4f20-9f73-65637988af5c\") " pod="openstack-kuttl-tests/cinder-aaa7-account-create-update-z472g" Dec 03 11:57:03 crc kubenswrapper[5052]: I1203 11:57:03.536028 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"neutron-db-secret" Dec 03 11:57:03 crc kubenswrapper[5052]: I1203 11:57:03.550754 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-0ea0-account-create-update-p5z4j"] Dec 03 11:57:03 crc kubenswrapper[5052]: I1203 11:57:03.592883 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-4aeb-account-create-update-hn4jf" Dec 03 11:57:03 crc kubenswrapper[5052]: I1203 11:57:03.593483 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-75dkf\" (UniqueName: \"kubernetes.io/projected/29f56a0c-db26-42fe-934f-0958d7e934cf-kube-api-access-75dkf\") pod \"neutron-0ea0-account-create-update-p5z4j\" (UID: \"29f56a0c-db26-42fe-934f-0958d7e934cf\") " pod="openstack-kuttl-tests/neutron-0ea0-account-create-update-p5z4j" Dec 03 11:57:03 crc kubenswrapper[5052]: I1203 11:57:03.593563 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/29f56a0c-db26-42fe-934f-0958d7e934cf-operator-scripts\") pod \"neutron-0ea0-account-create-update-p5z4j\" (UID: \"29f56a0c-db26-42fe-934f-0958d7e934cf\") " pod="openstack-kuttl-tests/neutron-0ea0-account-create-update-p5z4j" Dec 03 11:57:03 crc kubenswrapper[5052]: I1203 11:57:03.598988 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-db-sync-2jm7f" Dec 03 11:57:03 crc kubenswrapper[5052]: I1203 11:57:03.694694 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-75dkf\" (UniqueName: \"kubernetes.io/projected/29f56a0c-db26-42fe-934f-0958d7e934cf-kube-api-access-75dkf\") pod \"neutron-0ea0-account-create-update-p5z4j\" (UID: \"29f56a0c-db26-42fe-934f-0958d7e934cf\") " pod="openstack-kuttl-tests/neutron-0ea0-account-create-update-p5z4j" Dec 03 11:57:03 crc kubenswrapper[5052]: I1203 11:57:03.695039 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/29f56a0c-db26-42fe-934f-0958d7e934cf-operator-scripts\") pod \"neutron-0ea0-account-create-update-p5z4j\" (UID: \"29f56a0c-db26-42fe-934f-0958d7e934cf\") " pod="openstack-kuttl-tests/neutron-0ea0-account-create-update-p5z4j" Dec 03 11:57:03 crc kubenswrapper[5052]: I1203 11:57:03.697207 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/29f56a0c-db26-42fe-934f-0958d7e934cf-operator-scripts\") pod \"neutron-0ea0-account-create-update-p5z4j\" (UID: \"29f56a0c-db26-42fe-934f-0958d7e934cf\") " pod="openstack-kuttl-tests/neutron-0ea0-account-create-update-p5z4j" Dec 03 11:57:03 crc kubenswrapper[5052]: I1203 11:57:03.708133 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/watcher-db-sync-chlm4"] Dec 03 11:57:03 crc kubenswrapper[5052]: I1203 11:57:03.720319 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-75dkf\" (UniqueName: \"kubernetes.io/projected/29f56a0c-db26-42fe-934f-0958d7e934cf-kube-api-access-75dkf\") pod \"neutron-0ea0-account-create-update-p5z4j\" (UID: \"29f56a0c-db26-42fe-934f-0958d7e934cf\") " pod="openstack-kuttl-tests/neutron-0ea0-account-create-update-p5z4j" Dec 03 11:57:03 crc kubenswrapper[5052]: I1203 11:57:03.747435 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-db-create-2s4jv" Dec 03 11:57:03 crc kubenswrapper[5052]: I1203 11:57:03.766470 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-aaa7-account-create-update-z472g" Dec 03 11:57:03 crc kubenswrapper[5052]: I1203 11:57:03.875463 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-0ea0-account-create-update-p5z4j" Dec 03 11:57:03 crc kubenswrapper[5052]: I1203 11:57:03.883345 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-db-create-gx2gn"] Dec 03 11:57:03 crc kubenswrapper[5052]: I1203 11:57:03.940387 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-db-create-tl9wz"] Dec 03 11:57:03 crc kubenswrapper[5052]: W1203 11:57:03.964174 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podace50fc0_2d6c_4014_b576_0da48b39b4a5.slice/crio-c5b0f956a5c8802e28e94670a9bf5dfb3b66882b4a88e53a0259f4801d754e98 WatchSource:0}: Error finding container c5b0f956a5c8802e28e94670a9bf5dfb3b66882b4a88e53a0259f4801d754e98: Status 404 returned error can't find the container with id c5b0f956a5c8802e28e94670a9bf5dfb3b66882b4a88e53a0259f4801d754e98 Dec 03 11:57:04 crc kubenswrapper[5052]: I1203 11:57:04.113709 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-db-create-tl9wz" event={"ID":"7b231c94-d92f-49e9-8ae1-93b2cc125936","Type":"ContainerStarted","Data":"78d5c8a6e24707e201c49ef3917664d82af240f38b2fd10d8d4a352dbea95230"} Dec 03 11:57:04 crc kubenswrapper[5052]: I1203 11:57:04.114863 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-db-create-gx2gn" event={"ID":"ace50fc0-2d6c-4014-b576-0da48b39b4a5","Type":"ContainerStarted","Data":"c5b0f956a5c8802e28e94670a9bf5dfb3b66882b4a88e53a0259f4801d754e98"} Dec 03 11:57:04 crc kubenswrapper[5052]: I1203 11:57:04.115954 5052 generic.go:334] "Generic (PLEG): container finished" podID="a626fb80-9d64-44c4-94e7-e0dde9638ffc" containerID="608e4a1a02f619ff83a69532ecf951dbd52836921fd0e850e8efab96f9af411b" exitCode=0 Dec 03 11:57:04 crc kubenswrapper[5052]: I1203 11:57:04.115993 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-db-sync-kfnjj" event={"ID":"a626fb80-9d64-44c4-94e7-e0dde9638ffc","Type":"ContainerDied","Data":"608e4a1a02f619ff83a69532ecf951dbd52836921fd0e850e8efab96f9af411b"} Dec 03 11:57:04 crc kubenswrapper[5052]: I1203 11:57:04.117234 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/watcher-db-sync-chlm4" event={"ID":"b3146921-cc78-401f-a918-fa8b91814f10","Type":"ContainerStarted","Data":"4ca08618f8e64cb2936c7d44418631a3bca5e400de6062df7154882a95613b8e"} Dec 03 11:57:04 crc kubenswrapper[5052]: I1203 11:57:04.290376 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-4aeb-account-create-update-hn4jf"] Dec 03 11:57:04 crc kubenswrapper[5052]: W1203 11:57:04.326701 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod638e6ada_710e_43ce_9ac8_0569396886b3.slice/crio-9c42b13ddb2f6eddf1379fd89876c2508a592db1c24c07c86d53caec0546a8fb WatchSource:0}: Error finding container 9c42b13ddb2f6eddf1379fd89876c2508a592db1c24c07c86d53caec0546a8fb: Status 404 returned error can't find the container with id 9c42b13ddb2f6eddf1379fd89876c2508a592db1c24c07c86d53caec0546a8fb Dec 03 11:57:04 crc kubenswrapper[5052]: I1203 11:57:04.485897 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-db-sync-2jm7f"] Dec 03 11:57:04 crc kubenswrapper[5052]: I1203 11:57:04.573353 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-b6ccb55c5-4rjkw" Dec 03 11:57:04 crc kubenswrapper[5052]: I1203 11:57:04.587842 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-db-create-2s4jv"] Dec 03 11:57:04 crc kubenswrapper[5052]: I1203 11:57:04.710070 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-z8rn9"] Dec 03 11:57:04 crc kubenswrapper[5052]: I1203 11:57:04.710633 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-z8rn9" podUID="97767bcf-c9a4-430f-8298-3c677150b865" containerName="dnsmasq-dns" containerID="cri-o://8b643ff2eaca11b08a18ec9a8c2c3b8893b19dc1a8053d47c00be7efad4ddb8e" gracePeriod=10 Dec 03 11:57:04 crc kubenswrapper[5052]: I1203 11:57:04.774684 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-0ea0-account-create-update-p5z4j"] Dec 03 11:57:04 crc kubenswrapper[5052]: I1203 11:57:04.805313 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-aaa7-account-create-update-z472g"] Dec 03 11:57:04 crc kubenswrapper[5052]: W1203 11:57:04.876647 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod71879aa1_8e5d_4f20_9f73_65637988af5c.slice/crio-3ef5a6572bb75433b676bcd6b7c984755905f0eee85f7879a1ff34c8d0219ea9 WatchSource:0}: Error finding container 3ef5a6572bb75433b676bcd6b7c984755905f0eee85f7879a1ff34c8d0219ea9: Status 404 returned error can't find the container with id 3ef5a6572bb75433b676bcd6b7c984755905f0eee85f7879a1ff34c8d0219ea9 Dec 03 11:57:05 crc kubenswrapper[5052]: I1203 11:57:05.137252 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-db-create-2s4jv" event={"ID":"842a3b9b-c6e6-4032-926a-95cc59d990f3","Type":"ContainerStarted","Data":"3878c20bec17655742d669ac45a8e056745cb2364c66b143955a59eac4734e5f"} Dec 03 11:57:05 crc kubenswrapper[5052]: I1203 11:57:05.137999 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-db-create-2s4jv" event={"ID":"842a3b9b-c6e6-4032-926a-95cc59d990f3","Type":"ContainerStarted","Data":"e62c99dbbb60c6b584cd058b659c5da602a9eac45c5bbbd591c1df493f081123"} Dec 03 11:57:05 crc kubenswrapper[5052]: I1203 11:57:05.143911 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-db-sync-2jm7f" event={"ID":"7b1f895c-b513-4cc1-bf2a-72b84f1cd594","Type":"ContainerStarted","Data":"1b244ec0990cc16d9706217cfd2f7c20024ec92450402cd800184fce52721118"} Dec 03 11:57:05 crc kubenswrapper[5052]: I1203 11:57:05.143971 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-db-sync-2jm7f" event={"ID":"7b1f895c-b513-4cc1-bf2a-72b84f1cd594","Type":"ContainerStarted","Data":"f02d748830a6e7c9c0193d884e9741d2ba034ee1a115f9832f33fcd42de54f17"} Dec 03 11:57:05 crc kubenswrapper[5052]: I1203 11:57:05.148793 5052 generic.go:334] "Generic (PLEG): container finished" podID="7b231c94-d92f-49e9-8ae1-93b2cc125936" containerID="fe84882a80ccaeef2f8d940fc38370f099f9c488aea123cf93e4406cf6c98a68" exitCode=0 Dec 03 11:57:05 crc kubenswrapper[5052]: I1203 11:57:05.148848 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-db-create-tl9wz" event={"ID":"7b231c94-d92f-49e9-8ae1-93b2cc125936","Type":"ContainerDied","Data":"fe84882a80ccaeef2f8d940fc38370f099f9c488aea123cf93e4406cf6c98a68"} Dec 03 11:57:05 crc kubenswrapper[5052]: I1203 11:57:05.156143 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-0ea0-account-create-update-p5z4j" event={"ID":"29f56a0c-db26-42fe-934f-0958d7e934cf","Type":"ContainerStarted","Data":"751ae0e4435180af9fb225e29b5dbf5806750e9bf1e3dc5b8822572113aba4a4"} Dec 03 11:57:05 crc kubenswrapper[5052]: I1203 11:57:05.157185 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/neutron-db-create-2s4jv" podStartSLOduration=2.157164753 podStartE2EDuration="2.157164753s" podCreationTimestamp="2025-12-03 11:57:03 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:57:05.151186622 +0000 UTC m=+2215.739428396" watchObservedRunningTime="2025-12-03 11:57:05.157164753 +0000 UTC m=+2215.745406517" Dec 03 11:57:05 crc kubenswrapper[5052]: I1203 11:57:05.184134 5052 generic.go:334] "Generic (PLEG): container finished" podID="ace50fc0-2d6c-4014-b576-0da48b39b4a5" containerID="b98e34ad22699ba1f43e04c09cce28fdffc0b0ce4215afae3d6add511b2c1de7" exitCode=0 Dec 03 11:57:05 crc kubenswrapper[5052]: I1203 11:57:05.184201 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-db-create-gx2gn" event={"ID":"ace50fc0-2d6c-4014-b576-0da48b39b4a5","Type":"ContainerDied","Data":"b98e34ad22699ba1f43e04c09cce28fdffc0b0ce4215afae3d6add511b2c1de7"} Dec 03 11:57:05 crc kubenswrapper[5052]: I1203 11:57:05.187365 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-z8rn9" Dec 03 11:57:05 crc kubenswrapper[5052]: I1203 11:57:05.188025 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-aaa7-account-create-update-z472g" event={"ID":"71879aa1-8e5d-4f20-9f73-65637988af5c","Type":"ContainerStarted","Data":"3ef5a6572bb75433b676bcd6b7c984755905f0eee85f7879a1ff34c8d0219ea9"} Dec 03 11:57:05 crc kubenswrapper[5052]: I1203 11:57:05.191836 5052 generic.go:334] "Generic (PLEG): container finished" podID="97767bcf-c9a4-430f-8298-3c677150b865" containerID="8b643ff2eaca11b08a18ec9a8c2c3b8893b19dc1a8053d47c00be7efad4ddb8e" exitCode=0 Dec 03 11:57:05 crc kubenswrapper[5052]: I1203 11:57:05.191909 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-z8rn9" event={"ID":"97767bcf-c9a4-430f-8298-3c677150b865","Type":"ContainerDied","Data":"8b643ff2eaca11b08a18ec9a8c2c3b8893b19dc1a8053d47c00be7efad4ddb8e"} Dec 03 11:57:05 crc kubenswrapper[5052]: I1203 11:57:05.191945 5052 scope.go:117] "RemoveContainer" containerID="8b643ff2eaca11b08a18ec9a8c2c3b8893b19dc1a8053d47c00be7efad4ddb8e" Dec 03 11:57:05 crc kubenswrapper[5052]: I1203 11:57:05.192104 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-z8rn9" Dec 03 11:57:05 crc kubenswrapper[5052]: I1203 11:57:05.196755 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-4aeb-account-create-update-hn4jf" event={"ID":"638e6ada-710e-43ce-9ac8-0569396886b3","Type":"ContainerStarted","Data":"2e50a1ee31cb9fb17d19ff08d0f8c676ef19837034c0538001c15fc2e39b0bac"} Dec 03 11:57:05 crc kubenswrapper[5052]: I1203 11:57:05.196813 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-4aeb-account-create-update-hn4jf" event={"ID":"638e6ada-710e-43ce-9ac8-0569396886b3","Type":"ContainerStarted","Data":"9c42b13ddb2f6eddf1379fd89876c2508a592db1c24c07c86d53caec0546a8fb"} Dec 03 11:57:05 crc kubenswrapper[5052]: I1203 11:57:05.208604 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/keystone-db-sync-2jm7f" podStartSLOduration=2.208575134 podStartE2EDuration="2.208575134s" podCreationTimestamp="2025-12-03 11:57:03 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:57:05.203356375 +0000 UTC m=+2215.791598159" watchObservedRunningTime="2025-12-03 11:57:05.208575134 +0000 UTC m=+2215.796816908" Dec 03 11:57:05 crc kubenswrapper[5052]: I1203 11:57:05.257159 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/barbican-4aeb-account-create-update-hn4jf" podStartSLOduration=2.257137114 podStartE2EDuration="2.257137114s" podCreationTimestamp="2025-12-03 11:57:03 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:57:05.218481658 +0000 UTC m=+2215.806723422" watchObservedRunningTime="2025-12-03 11:57:05.257137114 +0000 UTC m=+2215.845378878" Dec 03 11:57:05 crc kubenswrapper[5052]: I1203 11:57:05.258047 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/97767bcf-c9a4-430f-8298-3c677150b865-config\") pod \"97767bcf-c9a4-430f-8298-3c677150b865\" (UID: \"97767bcf-c9a4-430f-8298-3c677150b865\") " Dec 03 11:57:05 crc kubenswrapper[5052]: I1203 11:57:05.258181 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ts94l\" (UniqueName: \"kubernetes.io/projected/97767bcf-c9a4-430f-8298-3c677150b865-kube-api-access-ts94l\") pod \"97767bcf-c9a4-430f-8298-3c677150b865\" (UID: \"97767bcf-c9a4-430f-8298-3c677150b865\") " Dec 03 11:57:05 crc kubenswrapper[5052]: I1203 11:57:05.258224 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/97767bcf-c9a4-430f-8298-3c677150b865-dnsmasq-svc\") pod \"97767bcf-c9a4-430f-8298-3c677150b865\" (UID: \"97767bcf-c9a4-430f-8298-3c677150b865\") " Dec 03 11:57:05 crc kubenswrapper[5052]: I1203 11:57:05.283896 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/97767bcf-c9a4-430f-8298-3c677150b865-kube-api-access-ts94l" (OuterVolumeSpecName: "kube-api-access-ts94l") pod "97767bcf-c9a4-430f-8298-3c677150b865" (UID: "97767bcf-c9a4-430f-8298-3c677150b865"). InnerVolumeSpecName "kube-api-access-ts94l". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:57:05 crc kubenswrapper[5052]: I1203 11:57:05.340991 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/97767bcf-c9a4-430f-8298-3c677150b865-config" (OuterVolumeSpecName: "config") pod "97767bcf-c9a4-430f-8298-3c677150b865" (UID: "97767bcf-c9a4-430f-8298-3c677150b865"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:57:05 crc kubenswrapper[5052]: I1203 11:57:05.361943 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/97767bcf-c9a4-430f-8298-3c677150b865-dnsmasq-svc" (OuterVolumeSpecName: "dnsmasq-svc") pod "97767bcf-c9a4-430f-8298-3c677150b865" (UID: "97767bcf-c9a4-430f-8298-3c677150b865"). InnerVolumeSpecName "dnsmasq-svc". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:57:05 crc kubenswrapper[5052]: I1203 11:57:05.362313 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-ts94l\" (UniqueName: \"kubernetes.io/projected/97767bcf-c9a4-430f-8298-3c677150b865-kube-api-access-ts94l\") on node \"crc\" DevicePath \"\"" Dec 03 11:57:05 crc kubenswrapper[5052]: I1203 11:57:05.362342 5052 reconciler_common.go:293] "Volume detached for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/97767bcf-c9a4-430f-8298-3c677150b865-dnsmasq-svc\") on node \"crc\" DevicePath \"\"" Dec 03 11:57:05 crc kubenswrapper[5052]: I1203 11:57:05.362355 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/97767bcf-c9a4-430f-8298-3c677150b865-config\") on node \"crc\" DevicePath \"\"" Dec 03 11:57:05 crc kubenswrapper[5052]: I1203 11:57:05.401421 5052 scope.go:117] "RemoveContainer" containerID="02c1e5c317d63ae18cc826a2fda38a6f9220d39c9c3beaf5b02ece77c75ff269" Dec 03 11:57:05 crc kubenswrapper[5052]: I1203 11:57:05.551374 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-z8rn9"] Dec 03 11:57:05 crc kubenswrapper[5052]: I1203 11:57:05.556947 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-db-sync-kfnjj" Dec 03 11:57:05 crc kubenswrapper[5052]: I1203 11:57:05.569282 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-z8rn9"] Dec 03 11:57:05 crc kubenswrapper[5052]: I1203 11:57:05.665955 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/a626fb80-9d64-44c4-94e7-e0dde9638ffc-db-sync-config-data\") pod \"a626fb80-9d64-44c4-94e7-e0dde9638ffc\" (UID: \"a626fb80-9d64-44c4-94e7-e0dde9638ffc\") " Dec 03 11:57:05 crc kubenswrapper[5052]: I1203 11:57:05.666098 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hgg7z\" (UniqueName: \"kubernetes.io/projected/a626fb80-9d64-44c4-94e7-e0dde9638ffc-kube-api-access-hgg7z\") pod \"a626fb80-9d64-44c4-94e7-e0dde9638ffc\" (UID: \"a626fb80-9d64-44c4-94e7-e0dde9638ffc\") " Dec 03 11:57:05 crc kubenswrapper[5052]: I1203 11:57:05.666145 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a626fb80-9d64-44c4-94e7-e0dde9638ffc-config-data\") pod \"a626fb80-9d64-44c4-94e7-e0dde9638ffc\" (UID: \"a626fb80-9d64-44c4-94e7-e0dde9638ffc\") " Dec 03 11:57:05 crc kubenswrapper[5052]: I1203 11:57:05.666212 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a626fb80-9d64-44c4-94e7-e0dde9638ffc-combined-ca-bundle\") pod \"a626fb80-9d64-44c4-94e7-e0dde9638ffc\" (UID: \"a626fb80-9d64-44c4-94e7-e0dde9638ffc\") " Dec 03 11:57:05 crc kubenswrapper[5052]: I1203 11:57:05.672742 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a626fb80-9d64-44c4-94e7-e0dde9638ffc-kube-api-access-hgg7z" (OuterVolumeSpecName: "kube-api-access-hgg7z") pod "a626fb80-9d64-44c4-94e7-e0dde9638ffc" (UID: "a626fb80-9d64-44c4-94e7-e0dde9638ffc"). InnerVolumeSpecName "kube-api-access-hgg7z". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:57:05 crc kubenswrapper[5052]: I1203 11:57:05.673938 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a626fb80-9d64-44c4-94e7-e0dde9638ffc-db-sync-config-data" (OuterVolumeSpecName: "db-sync-config-data") pod "a626fb80-9d64-44c4-94e7-e0dde9638ffc" (UID: "a626fb80-9d64-44c4-94e7-e0dde9638ffc"). InnerVolumeSpecName "db-sync-config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:57:05 crc kubenswrapper[5052]: I1203 11:57:05.688986 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="97767bcf-c9a4-430f-8298-3c677150b865" path="/var/lib/kubelet/pods/97767bcf-c9a4-430f-8298-3c677150b865/volumes" Dec 03 11:57:05 crc kubenswrapper[5052]: I1203 11:57:05.693490 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a626fb80-9d64-44c4-94e7-e0dde9638ffc-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "a626fb80-9d64-44c4-94e7-e0dde9638ffc" (UID: "a626fb80-9d64-44c4-94e7-e0dde9638ffc"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:57:05 crc kubenswrapper[5052]: I1203 11:57:05.716887 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a626fb80-9d64-44c4-94e7-e0dde9638ffc-config-data" (OuterVolumeSpecName: "config-data") pod "a626fb80-9d64-44c4-94e7-e0dde9638ffc" (UID: "a626fb80-9d64-44c4-94e7-e0dde9638ffc"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:57:05 crc kubenswrapper[5052]: I1203 11:57:05.768034 5052 reconciler_common.go:293] "Volume detached for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/a626fb80-9d64-44c4-94e7-e0dde9638ffc-db-sync-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:57:05 crc kubenswrapper[5052]: I1203 11:57:05.768077 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-hgg7z\" (UniqueName: \"kubernetes.io/projected/a626fb80-9d64-44c4-94e7-e0dde9638ffc-kube-api-access-hgg7z\") on node \"crc\" DevicePath \"\"" Dec 03 11:57:05 crc kubenswrapper[5052]: I1203 11:57:05.768093 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a626fb80-9d64-44c4-94e7-e0dde9638ffc-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:57:05 crc kubenswrapper[5052]: I1203 11:57:05.768103 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a626fb80-9d64-44c4-94e7-e0dde9638ffc-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:57:06 crc kubenswrapper[5052]: I1203 11:57:06.220393 5052 generic.go:334] "Generic (PLEG): container finished" podID="29f56a0c-db26-42fe-934f-0958d7e934cf" containerID="640c47d1ca72473f70926ab28ebc111db3c659c0cb4011d19f805e98ea5f46b1" exitCode=0 Dec 03 11:57:06 crc kubenswrapper[5052]: I1203 11:57:06.221344 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-0ea0-account-create-update-p5z4j" event={"ID":"29f56a0c-db26-42fe-934f-0958d7e934cf","Type":"ContainerDied","Data":"640c47d1ca72473f70926ab28ebc111db3c659c0cb4011d19f805e98ea5f46b1"} Dec 03 11:57:06 crc kubenswrapper[5052]: I1203 11:57:06.222802 5052 generic.go:334] "Generic (PLEG): container finished" podID="71879aa1-8e5d-4f20-9f73-65637988af5c" containerID="d284b92928b94c054ca5728e80c48dca72344c183bd279f1d273ea56c61bafed" exitCode=0 Dec 03 11:57:06 crc kubenswrapper[5052]: I1203 11:57:06.222871 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-aaa7-account-create-update-z472g" event={"ID":"71879aa1-8e5d-4f20-9f73-65637988af5c","Type":"ContainerDied","Data":"d284b92928b94c054ca5728e80c48dca72344c183bd279f1d273ea56c61bafed"} Dec 03 11:57:06 crc kubenswrapper[5052]: I1203 11:57:06.226734 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-db-sync-kfnjj" event={"ID":"a626fb80-9d64-44c4-94e7-e0dde9638ffc","Type":"ContainerDied","Data":"3997ea686cf55000226517c5cd86449112f31b0e7c3d79426a62c193e75c84ae"} Dec 03 11:57:06 crc kubenswrapper[5052]: I1203 11:57:06.226775 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3997ea686cf55000226517c5cd86449112f31b0e7c3d79426a62c193e75c84ae" Dec 03 11:57:06 crc kubenswrapper[5052]: I1203 11:57:06.226827 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-db-sync-kfnjj" Dec 03 11:57:06 crc kubenswrapper[5052]: I1203 11:57:06.245743 5052 generic.go:334] "Generic (PLEG): container finished" podID="638e6ada-710e-43ce-9ac8-0569396886b3" containerID="2e50a1ee31cb9fb17d19ff08d0f8c676ef19837034c0538001c15fc2e39b0bac" exitCode=0 Dec 03 11:57:06 crc kubenswrapper[5052]: I1203 11:57:06.245822 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-4aeb-account-create-update-hn4jf" event={"ID":"638e6ada-710e-43ce-9ac8-0569396886b3","Type":"ContainerDied","Data":"2e50a1ee31cb9fb17d19ff08d0f8c676ef19837034c0538001c15fc2e39b0bac"} Dec 03 11:57:06 crc kubenswrapper[5052]: I1203 11:57:06.249292 5052 generic.go:334] "Generic (PLEG): container finished" podID="842a3b9b-c6e6-4032-926a-95cc59d990f3" containerID="3878c20bec17655742d669ac45a8e056745cb2364c66b143955a59eac4734e5f" exitCode=0 Dec 03 11:57:06 crc kubenswrapper[5052]: I1203 11:57:06.249379 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-db-create-2s4jv" event={"ID":"842a3b9b-c6e6-4032-926a-95cc59d990f3","Type":"ContainerDied","Data":"3878c20bec17655742d669ac45a8e056745cb2364c66b143955a59eac4734e5f"} Dec 03 11:57:06 crc kubenswrapper[5052]: I1203 11:57:06.671259 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-db-create-gx2gn" Dec 03 11:57:06 crc kubenswrapper[5052]: I1203 11:57:06.754998 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-db-create-tl9wz" Dec 03 11:57:06 crc kubenswrapper[5052]: I1203 11:57:06.791629 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/ace50fc0-2d6c-4014-b576-0da48b39b4a5-operator-scripts\") pod \"ace50fc0-2d6c-4014-b576-0da48b39b4a5\" (UID: \"ace50fc0-2d6c-4014-b576-0da48b39b4a5\") " Dec 03 11:57:06 crc kubenswrapper[5052]: I1203 11:57:06.791694 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-scblb\" (UniqueName: \"kubernetes.io/projected/ace50fc0-2d6c-4014-b576-0da48b39b4a5-kube-api-access-scblb\") pod \"ace50fc0-2d6c-4014-b576-0da48b39b4a5\" (UID: \"ace50fc0-2d6c-4014-b576-0da48b39b4a5\") " Dec 03 11:57:06 crc kubenswrapper[5052]: I1203 11:57:06.792576 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ace50fc0-2d6c-4014-b576-0da48b39b4a5-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "ace50fc0-2d6c-4014-b576-0da48b39b4a5" (UID: "ace50fc0-2d6c-4014-b576-0da48b39b4a5"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:57:06 crc kubenswrapper[5052]: I1203 11:57:06.797746 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ace50fc0-2d6c-4014-b576-0da48b39b4a5-kube-api-access-scblb" (OuterVolumeSpecName: "kube-api-access-scblb") pod "ace50fc0-2d6c-4014-b576-0da48b39b4a5" (UID: "ace50fc0-2d6c-4014-b576-0da48b39b4a5"). InnerVolumeSpecName "kube-api-access-scblb". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:57:06 crc kubenswrapper[5052]: I1203 11:57:06.894050 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-clzxx\" (UniqueName: \"kubernetes.io/projected/7b231c94-d92f-49e9-8ae1-93b2cc125936-kube-api-access-clzxx\") pod \"7b231c94-d92f-49e9-8ae1-93b2cc125936\" (UID: \"7b231c94-d92f-49e9-8ae1-93b2cc125936\") " Dec 03 11:57:06 crc kubenswrapper[5052]: I1203 11:57:06.894251 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/7b231c94-d92f-49e9-8ae1-93b2cc125936-operator-scripts\") pod \"7b231c94-d92f-49e9-8ae1-93b2cc125936\" (UID: \"7b231c94-d92f-49e9-8ae1-93b2cc125936\") " Dec 03 11:57:06 crc kubenswrapper[5052]: I1203 11:57:06.894476 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/ace50fc0-2d6c-4014-b576-0da48b39b4a5-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:57:06 crc kubenswrapper[5052]: I1203 11:57:06.894492 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-scblb\" (UniqueName: \"kubernetes.io/projected/ace50fc0-2d6c-4014-b576-0da48b39b4a5-kube-api-access-scblb\") on node \"crc\" DevicePath \"\"" Dec 03 11:57:06 crc kubenswrapper[5052]: I1203 11:57:06.894884 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7b231c94-d92f-49e9-8ae1-93b2cc125936-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "7b231c94-d92f-49e9-8ae1-93b2cc125936" (UID: "7b231c94-d92f-49e9-8ae1-93b2cc125936"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:57:06 crc kubenswrapper[5052]: I1203 11:57:06.899664 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7b231c94-d92f-49e9-8ae1-93b2cc125936-kube-api-access-clzxx" (OuterVolumeSpecName: "kube-api-access-clzxx") pod "7b231c94-d92f-49e9-8ae1-93b2cc125936" (UID: "7b231c94-d92f-49e9-8ae1-93b2cc125936"). InnerVolumeSpecName "kube-api-access-clzxx". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:57:06 crc kubenswrapper[5052]: I1203 11:57:06.995602 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-clzxx\" (UniqueName: \"kubernetes.io/projected/7b231c94-d92f-49e9-8ae1-93b2cc125936-kube-api-access-clzxx\") on node \"crc\" DevicePath \"\"" Dec 03 11:57:06 crc kubenswrapper[5052]: I1203 11:57:06.995639 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/7b231c94-d92f-49e9-8ae1-93b2cc125936-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:57:07 crc kubenswrapper[5052]: I1203 11:57:07.263726 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-db-create-tl9wz" event={"ID":"7b231c94-d92f-49e9-8ae1-93b2cc125936","Type":"ContainerDied","Data":"78d5c8a6e24707e201c49ef3917664d82af240f38b2fd10d8d4a352dbea95230"} Dec 03 11:57:07 crc kubenswrapper[5052]: I1203 11:57:07.263814 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="78d5c8a6e24707e201c49ef3917664d82af240f38b2fd10d8d4a352dbea95230" Dec 03 11:57:07 crc kubenswrapper[5052]: I1203 11:57:07.264078 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-db-create-tl9wz" Dec 03 11:57:07 crc kubenswrapper[5052]: I1203 11:57:07.265821 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-db-create-gx2gn" Dec 03 11:57:07 crc kubenswrapper[5052]: I1203 11:57:07.265803 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-db-create-gx2gn" event={"ID":"ace50fc0-2d6c-4014-b576-0da48b39b4a5","Type":"ContainerDied","Data":"c5b0f956a5c8802e28e94670a9bf5dfb3b66882b4a88e53a0259f4801d754e98"} Dec 03 11:57:07 crc kubenswrapper[5052]: I1203 11:57:07.265869 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c5b0f956a5c8802e28e94670a9bf5dfb3b66882b4a88e53a0259f4801d754e98" Dec 03 11:57:07 crc kubenswrapper[5052]: I1203 11:57:07.645804 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-0ea0-account-create-update-p5z4j" Dec 03 11:57:07 crc kubenswrapper[5052]: I1203 11:57:07.715788 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-75dkf\" (UniqueName: \"kubernetes.io/projected/29f56a0c-db26-42fe-934f-0958d7e934cf-kube-api-access-75dkf\") pod \"29f56a0c-db26-42fe-934f-0958d7e934cf\" (UID: \"29f56a0c-db26-42fe-934f-0958d7e934cf\") " Dec 03 11:57:07 crc kubenswrapper[5052]: I1203 11:57:07.716255 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/29f56a0c-db26-42fe-934f-0958d7e934cf-operator-scripts\") pod \"29f56a0c-db26-42fe-934f-0958d7e934cf\" (UID: \"29f56a0c-db26-42fe-934f-0958d7e934cf\") " Dec 03 11:57:07 crc kubenswrapper[5052]: I1203 11:57:07.731957 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/29f56a0c-db26-42fe-934f-0958d7e934cf-kube-api-access-75dkf" (OuterVolumeSpecName: "kube-api-access-75dkf") pod "29f56a0c-db26-42fe-934f-0958d7e934cf" (UID: "29f56a0c-db26-42fe-934f-0958d7e934cf"). InnerVolumeSpecName "kube-api-access-75dkf". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:57:07 crc kubenswrapper[5052]: I1203 11:57:07.734050 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/29f56a0c-db26-42fe-934f-0958d7e934cf-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "29f56a0c-db26-42fe-934f-0958d7e934cf" (UID: "29f56a0c-db26-42fe-934f-0958d7e934cf"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:57:07 crc kubenswrapper[5052]: I1203 11:57:07.817738 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-75dkf\" (UniqueName: \"kubernetes.io/projected/29f56a0c-db26-42fe-934f-0958d7e934cf-kube-api-access-75dkf\") on node \"crc\" DevicePath \"\"" Dec 03 11:57:07 crc kubenswrapper[5052]: I1203 11:57:07.817768 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/29f56a0c-db26-42fe-934f-0958d7e934cf-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:57:08 crc kubenswrapper[5052]: I1203 11:57:08.283225 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-0ea0-account-create-update-p5z4j" event={"ID":"29f56a0c-db26-42fe-934f-0958d7e934cf","Type":"ContainerDied","Data":"751ae0e4435180af9fb225e29b5dbf5806750e9bf1e3dc5b8822572113aba4a4"} Dec 03 11:57:08 crc kubenswrapper[5052]: I1203 11:57:08.283271 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-0ea0-account-create-update-p5z4j" Dec 03 11:57:08 crc kubenswrapper[5052]: I1203 11:57:08.283280 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="751ae0e4435180af9fb225e29b5dbf5806750e9bf1e3dc5b8822572113aba4a4" Dec 03 11:57:09 crc kubenswrapper[5052]: I1203 11:57:09.291835 5052 generic.go:334] "Generic (PLEG): container finished" podID="7b1f895c-b513-4cc1-bf2a-72b84f1cd594" containerID="1b244ec0990cc16d9706217cfd2f7c20024ec92450402cd800184fce52721118" exitCode=0 Dec 03 11:57:09 crc kubenswrapper[5052]: I1203 11:57:09.292036 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-db-sync-2jm7f" event={"ID":"7b1f895c-b513-4cc1-bf2a-72b84f1cd594","Type":"ContainerDied","Data":"1b244ec0990cc16d9706217cfd2f7c20024ec92450402cd800184fce52721118"} Dec 03 11:57:11 crc kubenswrapper[5052]: I1203 11:57:11.313737 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/prometheus-metric-storage-0" Dec 03 11:57:11 crc kubenswrapper[5052]: I1203 11:57:11.316690 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/prometheus-metric-storage-0" Dec 03 11:57:12 crc kubenswrapper[5052]: I1203 11:57:12.319815 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/prometheus-metric-storage-0" Dec 03 11:57:15 crc kubenswrapper[5052]: I1203 11:57:15.176419 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/prometheus-metric-storage-0"] Dec 03 11:57:15 crc kubenswrapper[5052]: I1203 11:57:15.177085 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/prometheus-metric-storage-0" podUID="33918bc1-340a-4d26-a204-001b85c15f04" containerName="thanos-sidecar" containerID="cri-o://338f13d17f6b49a2ab7b8705345e66ade7dfcb2716d85396b3217b2553ae2ae2" gracePeriod=600 Dec 03 11:57:15 crc kubenswrapper[5052]: I1203 11:57:15.177377 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/prometheus-metric-storage-0" podUID="33918bc1-340a-4d26-a204-001b85c15f04" containerName="config-reloader" containerID="cri-o://4efdd771545fd00295d9e81c501f88a8052264f58cb92b45ecd0fedadf7e279c" gracePeriod=600 Dec 03 11:57:15 crc kubenswrapper[5052]: I1203 11:57:15.177586 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/prometheus-metric-storage-0" podUID="33918bc1-340a-4d26-a204-001b85c15f04" containerName="prometheus" containerID="cri-o://a39079dcf889f7fbf32bc70b596f4c1f6eab751f6901536f1d0bda478008c680" gracePeriod=600 Dec 03 11:57:16 crc kubenswrapper[5052]: I1203 11:57:16.315741 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/prometheus-metric-storage-0" podUID="33918bc1-340a-4d26-a204-001b85c15f04" containerName="prometheus" probeResult="failure" output="HTTP probe failed with statuscode: 503" Dec 03 11:57:17 crc kubenswrapper[5052]: I1203 11:57:17.054900 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-4aeb-account-create-update-hn4jf" Dec 03 11:57:17 crc kubenswrapper[5052]: I1203 11:57:17.059715 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-aaa7-account-create-update-z472g" Dec 03 11:57:17 crc kubenswrapper[5052]: I1203 11:57:17.068150 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-db-create-2s4jv" Dec 03 11:57:17 crc kubenswrapper[5052]: I1203 11:57:17.098712 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-db-sync-2jm7f" Dec 03 11:57:17 crc kubenswrapper[5052]: I1203 11:57:17.202195 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jhmqr\" (UniqueName: \"kubernetes.io/projected/638e6ada-710e-43ce-9ac8-0569396886b3-kube-api-access-jhmqr\") pod \"638e6ada-710e-43ce-9ac8-0569396886b3\" (UID: \"638e6ada-710e-43ce-9ac8-0569396886b3\") " Dec 03 11:57:17 crc kubenswrapper[5052]: I1203 11:57:17.202306 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/7b1f895c-b513-4cc1-bf2a-72b84f1cd594-config-data\") pod \"7b1f895c-b513-4cc1-bf2a-72b84f1cd594\" (UID: \"7b1f895c-b513-4cc1-bf2a-72b84f1cd594\") " Dec 03 11:57:17 crc kubenswrapper[5052]: I1203 11:57:17.202378 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rqdbr\" (UniqueName: \"kubernetes.io/projected/7b1f895c-b513-4cc1-bf2a-72b84f1cd594-kube-api-access-rqdbr\") pod \"7b1f895c-b513-4cc1-bf2a-72b84f1cd594\" (UID: \"7b1f895c-b513-4cc1-bf2a-72b84f1cd594\") " Dec 03 11:57:17 crc kubenswrapper[5052]: I1203 11:57:17.202407 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f4c8m\" (UniqueName: \"kubernetes.io/projected/842a3b9b-c6e6-4032-926a-95cc59d990f3-kube-api-access-f4c8m\") pod \"842a3b9b-c6e6-4032-926a-95cc59d990f3\" (UID: \"842a3b9b-c6e6-4032-926a-95cc59d990f3\") " Dec 03 11:57:17 crc kubenswrapper[5052]: I1203 11:57:17.202424 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x28f8\" (UniqueName: \"kubernetes.io/projected/71879aa1-8e5d-4f20-9f73-65637988af5c-kube-api-access-x28f8\") pod \"71879aa1-8e5d-4f20-9f73-65637988af5c\" (UID: \"71879aa1-8e5d-4f20-9f73-65637988af5c\") " Dec 03 11:57:17 crc kubenswrapper[5052]: I1203 11:57:17.203785 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/7b1f895c-b513-4cc1-bf2a-72b84f1cd594-combined-ca-bundle\") pod \"7b1f895c-b513-4cc1-bf2a-72b84f1cd594\" (UID: \"7b1f895c-b513-4cc1-bf2a-72b84f1cd594\") " Dec 03 11:57:17 crc kubenswrapper[5052]: I1203 11:57:17.203848 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/638e6ada-710e-43ce-9ac8-0569396886b3-operator-scripts\") pod \"638e6ada-710e-43ce-9ac8-0569396886b3\" (UID: \"638e6ada-710e-43ce-9ac8-0569396886b3\") " Dec 03 11:57:17 crc kubenswrapper[5052]: I1203 11:57:17.203879 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/71879aa1-8e5d-4f20-9f73-65637988af5c-operator-scripts\") pod \"71879aa1-8e5d-4f20-9f73-65637988af5c\" (UID: \"71879aa1-8e5d-4f20-9f73-65637988af5c\") " Dec 03 11:57:17 crc kubenswrapper[5052]: I1203 11:57:17.203902 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/842a3b9b-c6e6-4032-926a-95cc59d990f3-operator-scripts\") pod \"842a3b9b-c6e6-4032-926a-95cc59d990f3\" (UID: \"842a3b9b-c6e6-4032-926a-95cc59d990f3\") " Dec 03 11:57:17 crc kubenswrapper[5052]: I1203 11:57:17.204871 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/842a3b9b-c6e6-4032-926a-95cc59d990f3-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "842a3b9b-c6e6-4032-926a-95cc59d990f3" (UID: "842a3b9b-c6e6-4032-926a-95cc59d990f3"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:57:17 crc kubenswrapper[5052]: I1203 11:57:17.204936 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/638e6ada-710e-43ce-9ac8-0569396886b3-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "638e6ada-710e-43ce-9ac8-0569396886b3" (UID: "638e6ada-710e-43ce-9ac8-0569396886b3"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:57:17 crc kubenswrapper[5052]: I1203 11:57:17.204864 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/71879aa1-8e5d-4f20-9f73-65637988af5c-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "71879aa1-8e5d-4f20-9f73-65637988af5c" (UID: "71879aa1-8e5d-4f20-9f73-65637988af5c"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:57:17 crc kubenswrapper[5052]: I1203 11:57:17.223546 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7b1f895c-b513-4cc1-bf2a-72b84f1cd594-kube-api-access-rqdbr" (OuterVolumeSpecName: "kube-api-access-rqdbr") pod "7b1f895c-b513-4cc1-bf2a-72b84f1cd594" (UID: "7b1f895c-b513-4cc1-bf2a-72b84f1cd594"). InnerVolumeSpecName "kube-api-access-rqdbr". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:57:17 crc kubenswrapper[5052]: I1203 11:57:17.223608 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/638e6ada-710e-43ce-9ac8-0569396886b3-kube-api-access-jhmqr" (OuterVolumeSpecName: "kube-api-access-jhmqr") pod "638e6ada-710e-43ce-9ac8-0569396886b3" (UID: "638e6ada-710e-43ce-9ac8-0569396886b3"). InnerVolumeSpecName "kube-api-access-jhmqr". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:57:17 crc kubenswrapper[5052]: I1203 11:57:17.223637 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/842a3b9b-c6e6-4032-926a-95cc59d990f3-kube-api-access-f4c8m" (OuterVolumeSpecName: "kube-api-access-f4c8m") pod "842a3b9b-c6e6-4032-926a-95cc59d990f3" (UID: "842a3b9b-c6e6-4032-926a-95cc59d990f3"). InnerVolumeSpecName "kube-api-access-f4c8m". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:57:17 crc kubenswrapper[5052]: I1203 11:57:17.223653 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/71879aa1-8e5d-4f20-9f73-65637988af5c-kube-api-access-x28f8" (OuterVolumeSpecName: "kube-api-access-x28f8") pod "71879aa1-8e5d-4f20-9f73-65637988af5c" (UID: "71879aa1-8e5d-4f20-9f73-65637988af5c"). InnerVolumeSpecName "kube-api-access-x28f8". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:57:17 crc kubenswrapper[5052]: I1203 11:57:17.228875 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7b1f895c-b513-4cc1-bf2a-72b84f1cd594-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "7b1f895c-b513-4cc1-bf2a-72b84f1cd594" (UID: "7b1f895c-b513-4cc1-bf2a-72b84f1cd594"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:57:17 crc kubenswrapper[5052]: I1203 11:57:17.260527 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7b1f895c-b513-4cc1-bf2a-72b84f1cd594-config-data" (OuterVolumeSpecName: "config-data") pod "7b1f895c-b513-4cc1-bf2a-72b84f1cd594" (UID: "7b1f895c-b513-4cc1-bf2a-72b84f1cd594"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:57:17 crc kubenswrapper[5052]: I1203 11:57:17.305507 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/71879aa1-8e5d-4f20-9f73-65637988af5c-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:57:17 crc kubenswrapper[5052]: I1203 11:57:17.305553 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/842a3b9b-c6e6-4032-926a-95cc59d990f3-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:57:17 crc kubenswrapper[5052]: I1203 11:57:17.305566 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-jhmqr\" (UniqueName: \"kubernetes.io/projected/638e6ada-710e-43ce-9ac8-0569396886b3-kube-api-access-jhmqr\") on node \"crc\" DevicePath \"\"" Dec 03 11:57:17 crc kubenswrapper[5052]: I1203 11:57:17.305575 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/7b1f895c-b513-4cc1-bf2a-72b84f1cd594-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:57:17 crc kubenswrapper[5052]: I1203 11:57:17.305584 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-rqdbr\" (UniqueName: \"kubernetes.io/projected/7b1f895c-b513-4cc1-bf2a-72b84f1cd594-kube-api-access-rqdbr\") on node \"crc\" DevicePath \"\"" Dec 03 11:57:17 crc kubenswrapper[5052]: I1203 11:57:17.305593 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-f4c8m\" (UniqueName: \"kubernetes.io/projected/842a3b9b-c6e6-4032-926a-95cc59d990f3-kube-api-access-f4c8m\") on node \"crc\" DevicePath \"\"" Dec 03 11:57:17 crc kubenswrapper[5052]: I1203 11:57:17.305601 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-x28f8\" (UniqueName: \"kubernetes.io/projected/71879aa1-8e5d-4f20-9f73-65637988af5c-kube-api-access-x28f8\") on node \"crc\" DevicePath \"\"" Dec 03 11:57:17 crc kubenswrapper[5052]: I1203 11:57:17.305609 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/7b1f895c-b513-4cc1-bf2a-72b84f1cd594-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:57:17 crc kubenswrapper[5052]: I1203 11:57:17.305617 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/638e6ada-710e-43ce-9ac8-0569396886b3-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:57:17 crc kubenswrapper[5052]: I1203 11:57:17.366012 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-4aeb-account-create-update-hn4jf" event={"ID":"638e6ada-710e-43ce-9ac8-0569396886b3","Type":"ContainerDied","Data":"9c42b13ddb2f6eddf1379fd89876c2508a592db1c24c07c86d53caec0546a8fb"} Dec 03 11:57:17 crc kubenswrapper[5052]: I1203 11:57:17.366074 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9c42b13ddb2f6eddf1379fd89876c2508a592db1c24c07c86d53caec0546a8fb" Dec 03 11:57:17 crc kubenswrapper[5052]: I1203 11:57:17.366150 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-4aeb-account-create-update-hn4jf" Dec 03 11:57:17 crc kubenswrapper[5052]: I1203 11:57:17.374036 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-db-create-2s4jv" Dec 03 11:57:17 crc kubenswrapper[5052]: I1203 11:57:17.374041 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-db-create-2s4jv" event={"ID":"842a3b9b-c6e6-4032-926a-95cc59d990f3","Type":"ContainerDied","Data":"e62c99dbbb60c6b584cd058b659c5da602a9eac45c5bbbd591c1df493f081123"} Dec 03 11:57:17 crc kubenswrapper[5052]: I1203 11:57:17.374154 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e62c99dbbb60c6b584cd058b659c5da602a9eac45c5bbbd591c1df493f081123" Dec 03 11:57:17 crc kubenswrapper[5052]: I1203 11:57:17.375436 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-db-sync-2jm7f" event={"ID":"7b1f895c-b513-4cc1-bf2a-72b84f1cd594","Type":"ContainerDied","Data":"f02d748830a6e7c9c0193d884e9741d2ba034ee1a115f9832f33fcd42de54f17"} Dec 03 11:57:17 crc kubenswrapper[5052]: I1203 11:57:17.375476 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f02d748830a6e7c9c0193d884e9741d2ba034ee1a115f9832f33fcd42de54f17" Dec 03 11:57:17 crc kubenswrapper[5052]: I1203 11:57:17.375552 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-db-sync-2jm7f" Dec 03 11:57:17 crc kubenswrapper[5052]: I1203 11:57:17.383245 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-aaa7-account-create-update-z472g" event={"ID":"71879aa1-8e5d-4f20-9f73-65637988af5c","Type":"ContainerDied","Data":"3ef5a6572bb75433b676bcd6b7c984755905f0eee85f7879a1ff34c8d0219ea9"} Dec 03 11:57:17 crc kubenswrapper[5052]: I1203 11:57:17.383292 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3ef5a6572bb75433b676bcd6b7c984755905f0eee85f7879a1ff34c8d0219ea9" Dec 03 11:57:17 crc kubenswrapper[5052]: I1203 11:57:17.383356 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-aaa7-account-create-update-z472g" Dec 03 11:57:17 crc kubenswrapper[5052]: I1203 11:57:17.391584 5052 generic.go:334] "Generic (PLEG): container finished" podID="33918bc1-340a-4d26-a204-001b85c15f04" containerID="338f13d17f6b49a2ab7b8705345e66ade7dfcb2716d85396b3217b2553ae2ae2" exitCode=0 Dec 03 11:57:17 crc kubenswrapper[5052]: I1203 11:57:17.391619 5052 generic.go:334] "Generic (PLEG): container finished" podID="33918bc1-340a-4d26-a204-001b85c15f04" containerID="4efdd771545fd00295d9e81c501f88a8052264f58cb92b45ecd0fedadf7e279c" exitCode=0 Dec 03 11:57:17 crc kubenswrapper[5052]: I1203 11:57:17.391632 5052 generic.go:334] "Generic (PLEG): container finished" podID="33918bc1-340a-4d26-a204-001b85c15f04" containerID="a39079dcf889f7fbf32bc70b596f4c1f6eab751f6901536f1d0bda478008c680" exitCode=0 Dec 03 11:57:17 crc kubenswrapper[5052]: I1203 11:57:17.391654 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/prometheus-metric-storage-0" event={"ID":"33918bc1-340a-4d26-a204-001b85c15f04","Type":"ContainerDied","Data":"338f13d17f6b49a2ab7b8705345e66ade7dfcb2716d85396b3217b2553ae2ae2"} Dec 03 11:57:17 crc kubenswrapper[5052]: I1203 11:57:17.391698 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/prometheus-metric-storage-0" event={"ID":"33918bc1-340a-4d26-a204-001b85c15f04","Type":"ContainerDied","Data":"4efdd771545fd00295d9e81c501f88a8052264f58cb92b45ecd0fedadf7e279c"} Dec 03 11:57:17 crc kubenswrapper[5052]: I1203 11:57:17.391712 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/prometheus-metric-storage-0" event={"ID":"33918bc1-340a-4d26-a204-001b85c15f04","Type":"ContainerDied","Data":"a39079dcf889f7fbf32bc70b596f4c1f6eab751f6901536f1d0bda478008c680"} Dec 03 11:57:18 crc kubenswrapper[5052]: I1203 11:57:18.361577 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/keystone-bootstrap-kdr6v"] Dec 03 11:57:18 crc kubenswrapper[5052]: E1203 11:57:18.362214 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="7b1f895c-b513-4cc1-bf2a-72b84f1cd594" containerName="keystone-db-sync" Dec 03 11:57:18 crc kubenswrapper[5052]: I1203 11:57:18.362231 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="7b1f895c-b513-4cc1-bf2a-72b84f1cd594" containerName="keystone-db-sync" Dec 03 11:57:18 crc kubenswrapper[5052]: E1203 11:57:18.362248 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="ace50fc0-2d6c-4014-b576-0da48b39b4a5" containerName="mariadb-database-create" Dec 03 11:57:18 crc kubenswrapper[5052]: I1203 11:57:18.362254 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="ace50fc0-2d6c-4014-b576-0da48b39b4a5" containerName="mariadb-database-create" Dec 03 11:57:18 crc kubenswrapper[5052]: E1203 11:57:18.362265 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="97767bcf-c9a4-430f-8298-3c677150b865" containerName="init" Dec 03 11:57:18 crc kubenswrapper[5052]: I1203 11:57:18.362271 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="97767bcf-c9a4-430f-8298-3c677150b865" containerName="init" Dec 03 11:57:18 crc kubenswrapper[5052]: E1203 11:57:18.362280 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="29f56a0c-db26-42fe-934f-0958d7e934cf" containerName="mariadb-account-create-update" Dec 03 11:57:18 crc kubenswrapper[5052]: I1203 11:57:18.362286 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="29f56a0c-db26-42fe-934f-0958d7e934cf" containerName="mariadb-account-create-update" Dec 03 11:57:18 crc kubenswrapper[5052]: E1203 11:57:18.362298 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="a626fb80-9d64-44c4-94e7-e0dde9638ffc" containerName="glance-db-sync" Dec 03 11:57:18 crc kubenswrapper[5052]: I1203 11:57:18.362304 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="a626fb80-9d64-44c4-94e7-e0dde9638ffc" containerName="glance-db-sync" Dec 03 11:57:18 crc kubenswrapper[5052]: E1203 11:57:18.362312 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="842a3b9b-c6e6-4032-926a-95cc59d990f3" containerName="mariadb-database-create" Dec 03 11:57:18 crc kubenswrapper[5052]: I1203 11:57:18.362318 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="842a3b9b-c6e6-4032-926a-95cc59d990f3" containerName="mariadb-database-create" Dec 03 11:57:18 crc kubenswrapper[5052]: E1203 11:57:18.362336 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="71879aa1-8e5d-4f20-9f73-65637988af5c" containerName="mariadb-account-create-update" Dec 03 11:57:18 crc kubenswrapper[5052]: I1203 11:57:18.362341 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="71879aa1-8e5d-4f20-9f73-65637988af5c" containerName="mariadb-account-create-update" Dec 03 11:57:18 crc kubenswrapper[5052]: E1203 11:57:18.362357 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="97767bcf-c9a4-430f-8298-3c677150b865" containerName="dnsmasq-dns" Dec 03 11:57:18 crc kubenswrapper[5052]: I1203 11:57:18.362362 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="97767bcf-c9a4-430f-8298-3c677150b865" containerName="dnsmasq-dns" Dec 03 11:57:18 crc kubenswrapper[5052]: E1203 11:57:18.362375 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="7b231c94-d92f-49e9-8ae1-93b2cc125936" containerName="mariadb-database-create" Dec 03 11:57:18 crc kubenswrapper[5052]: I1203 11:57:18.362380 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="7b231c94-d92f-49e9-8ae1-93b2cc125936" containerName="mariadb-database-create" Dec 03 11:57:18 crc kubenswrapper[5052]: E1203 11:57:18.362394 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="638e6ada-710e-43ce-9ac8-0569396886b3" containerName="mariadb-account-create-update" Dec 03 11:57:18 crc kubenswrapper[5052]: I1203 11:57:18.362400 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="638e6ada-710e-43ce-9ac8-0569396886b3" containerName="mariadb-account-create-update" Dec 03 11:57:18 crc kubenswrapper[5052]: I1203 11:57:18.362561 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="7b231c94-d92f-49e9-8ae1-93b2cc125936" containerName="mariadb-database-create" Dec 03 11:57:18 crc kubenswrapper[5052]: I1203 11:57:18.362576 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="638e6ada-710e-43ce-9ac8-0569396886b3" containerName="mariadb-account-create-update" Dec 03 11:57:18 crc kubenswrapper[5052]: I1203 11:57:18.362583 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="ace50fc0-2d6c-4014-b576-0da48b39b4a5" containerName="mariadb-database-create" Dec 03 11:57:18 crc kubenswrapper[5052]: I1203 11:57:18.362597 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="842a3b9b-c6e6-4032-926a-95cc59d990f3" containerName="mariadb-database-create" Dec 03 11:57:18 crc kubenswrapper[5052]: I1203 11:57:18.362604 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="7b1f895c-b513-4cc1-bf2a-72b84f1cd594" containerName="keystone-db-sync" Dec 03 11:57:18 crc kubenswrapper[5052]: I1203 11:57:18.362615 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="a626fb80-9d64-44c4-94e7-e0dde9638ffc" containerName="glance-db-sync" Dec 03 11:57:18 crc kubenswrapper[5052]: I1203 11:57:18.362621 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="29f56a0c-db26-42fe-934f-0958d7e934cf" containerName="mariadb-account-create-update" Dec 03 11:57:18 crc kubenswrapper[5052]: I1203 11:57:18.362631 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="97767bcf-c9a4-430f-8298-3c677150b865" containerName="dnsmasq-dns" Dec 03 11:57:18 crc kubenswrapper[5052]: I1203 11:57:18.362643 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="71879aa1-8e5d-4f20-9f73-65637988af5c" containerName="mariadb-account-create-update" Dec 03 11:57:18 crc kubenswrapper[5052]: I1203 11:57:18.363222 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-bootstrap-kdr6v" Dec 03 11:57:18 crc kubenswrapper[5052]: I1203 11:57:18.366366 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-bootstrap-kdr6v"] Dec 03 11:57:18 crc kubenswrapper[5052]: I1203 11:57:18.368672 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone-config-data" Dec 03 11:57:18 crc kubenswrapper[5052]: I1203 11:57:18.369307 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone" Dec 03 11:57:18 crc kubenswrapper[5052]: I1203 11:57:18.369422 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone-scripts" Dec 03 11:57:18 crc kubenswrapper[5052]: I1203 11:57:18.369487 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone-keystone-dockercfg-csztk" Dec 03 11:57:18 crc kubenswrapper[5052]: I1203 11:57:18.369549 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"osp-secret" Dec 03 11:57:18 crc kubenswrapper[5052]: I1203 11:57:18.388164 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9gqn4\" (UniqueName: \"kubernetes.io/projected/b70c4b01-42e4-4990-be48-a2d762e66cd7-kube-api-access-9gqn4\") pod \"keystone-bootstrap-kdr6v\" (UID: \"b70c4b01-42e4-4990-be48-a2d762e66cd7\") " pod="openstack-kuttl-tests/keystone-bootstrap-kdr6v" Dec 03 11:57:18 crc kubenswrapper[5052]: I1203 11:57:18.388216 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/b70c4b01-42e4-4990-be48-a2d762e66cd7-scripts\") pod \"keystone-bootstrap-kdr6v\" (UID: \"b70c4b01-42e4-4990-be48-a2d762e66cd7\") " pod="openstack-kuttl-tests/keystone-bootstrap-kdr6v" Dec 03 11:57:18 crc kubenswrapper[5052]: I1203 11:57:18.388240 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b70c4b01-42e4-4990-be48-a2d762e66cd7-combined-ca-bundle\") pod \"keystone-bootstrap-kdr6v\" (UID: \"b70c4b01-42e4-4990-be48-a2d762e66cd7\") " pod="openstack-kuttl-tests/keystone-bootstrap-kdr6v" Dec 03 11:57:18 crc kubenswrapper[5052]: I1203 11:57:18.388258 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/b70c4b01-42e4-4990-be48-a2d762e66cd7-fernet-keys\") pod \"keystone-bootstrap-kdr6v\" (UID: \"b70c4b01-42e4-4990-be48-a2d762e66cd7\") " pod="openstack-kuttl-tests/keystone-bootstrap-kdr6v" Dec 03 11:57:18 crc kubenswrapper[5052]: I1203 11:57:18.388278 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/b70c4b01-42e4-4990-be48-a2d762e66cd7-config-data\") pod \"keystone-bootstrap-kdr6v\" (UID: \"b70c4b01-42e4-4990-be48-a2d762e66cd7\") " pod="openstack-kuttl-tests/keystone-bootstrap-kdr6v" Dec 03 11:57:18 crc kubenswrapper[5052]: I1203 11:57:18.388318 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/b70c4b01-42e4-4990-be48-a2d762e66cd7-credential-keys\") pod \"keystone-bootstrap-kdr6v\" (UID: \"b70c4b01-42e4-4990-be48-a2d762e66cd7\") " pod="openstack-kuttl-tests/keystone-bootstrap-kdr6v" Dec 03 11:57:18 crc kubenswrapper[5052]: I1203 11:57:18.489623 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-9gqn4\" (UniqueName: \"kubernetes.io/projected/b70c4b01-42e4-4990-be48-a2d762e66cd7-kube-api-access-9gqn4\") pod \"keystone-bootstrap-kdr6v\" (UID: \"b70c4b01-42e4-4990-be48-a2d762e66cd7\") " pod="openstack-kuttl-tests/keystone-bootstrap-kdr6v" Dec 03 11:57:18 crc kubenswrapper[5052]: I1203 11:57:18.489689 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/b70c4b01-42e4-4990-be48-a2d762e66cd7-scripts\") pod \"keystone-bootstrap-kdr6v\" (UID: \"b70c4b01-42e4-4990-be48-a2d762e66cd7\") " pod="openstack-kuttl-tests/keystone-bootstrap-kdr6v" Dec 03 11:57:18 crc kubenswrapper[5052]: I1203 11:57:18.489716 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b70c4b01-42e4-4990-be48-a2d762e66cd7-combined-ca-bundle\") pod \"keystone-bootstrap-kdr6v\" (UID: \"b70c4b01-42e4-4990-be48-a2d762e66cd7\") " pod="openstack-kuttl-tests/keystone-bootstrap-kdr6v" Dec 03 11:57:18 crc kubenswrapper[5052]: I1203 11:57:18.489753 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/b70c4b01-42e4-4990-be48-a2d762e66cd7-fernet-keys\") pod \"keystone-bootstrap-kdr6v\" (UID: \"b70c4b01-42e4-4990-be48-a2d762e66cd7\") " pod="openstack-kuttl-tests/keystone-bootstrap-kdr6v" Dec 03 11:57:18 crc kubenswrapper[5052]: I1203 11:57:18.490409 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/b70c4b01-42e4-4990-be48-a2d762e66cd7-config-data\") pod \"keystone-bootstrap-kdr6v\" (UID: \"b70c4b01-42e4-4990-be48-a2d762e66cd7\") " pod="openstack-kuttl-tests/keystone-bootstrap-kdr6v" Dec 03 11:57:18 crc kubenswrapper[5052]: I1203 11:57:18.490486 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/b70c4b01-42e4-4990-be48-a2d762e66cd7-credential-keys\") pod \"keystone-bootstrap-kdr6v\" (UID: \"b70c4b01-42e4-4990-be48-a2d762e66cd7\") " pod="openstack-kuttl-tests/keystone-bootstrap-kdr6v" Dec 03 11:57:18 crc kubenswrapper[5052]: I1203 11:57:18.498566 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/b70c4b01-42e4-4990-be48-a2d762e66cd7-fernet-keys\") pod \"keystone-bootstrap-kdr6v\" (UID: \"b70c4b01-42e4-4990-be48-a2d762e66cd7\") " pod="openstack-kuttl-tests/keystone-bootstrap-kdr6v" Dec 03 11:57:18 crc kubenswrapper[5052]: I1203 11:57:18.499160 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/b70c4b01-42e4-4990-be48-a2d762e66cd7-credential-keys\") pod \"keystone-bootstrap-kdr6v\" (UID: \"b70c4b01-42e4-4990-be48-a2d762e66cd7\") " pod="openstack-kuttl-tests/keystone-bootstrap-kdr6v" Dec 03 11:57:18 crc kubenswrapper[5052]: I1203 11:57:18.502287 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/b70c4b01-42e4-4990-be48-a2d762e66cd7-scripts\") pod \"keystone-bootstrap-kdr6v\" (UID: \"b70c4b01-42e4-4990-be48-a2d762e66cd7\") " pod="openstack-kuttl-tests/keystone-bootstrap-kdr6v" Dec 03 11:57:18 crc kubenswrapper[5052]: I1203 11:57:18.507119 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b70c4b01-42e4-4990-be48-a2d762e66cd7-combined-ca-bundle\") pod \"keystone-bootstrap-kdr6v\" (UID: \"b70c4b01-42e4-4990-be48-a2d762e66cd7\") " pod="openstack-kuttl-tests/keystone-bootstrap-kdr6v" Dec 03 11:57:18 crc kubenswrapper[5052]: I1203 11:57:18.516173 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/b70c4b01-42e4-4990-be48-a2d762e66cd7-config-data\") pod \"keystone-bootstrap-kdr6v\" (UID: \"b70c4b01-42e4-4990-be48-a2d762e66cd7\") " pod="openstack-kuttl-tests/keystone-bootstrap-kdr6v" Dec 03 11:57:18 crc kubenswrapper[5052]: I1203 11:57:18.521815 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-9gqn4\" (UniqueName: \"kubernetes.io/projected/b70c4b01-42e4-4990-be48-a2d762e66cd7-kube-api-access-9gqn4\") pod \"keystone-bootstrap-kdr6v\" (UID: \"b70c4b01-42e4-4990-be48-a2d762e66cd7\") " pod="openstack-kuttl-tests/keystone-bootstrap-kdr6v" Dec 03 11:57:18 crc kubenswrapper[5052]: I1203 11:57:18.620442 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:57:18 crc kubenswrapper[5052]: I1203 11:57:18.622763 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:57:18 crc kubenswrapper[5052]: I1203 11:57:18.625969 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-config-data" Dec 03 11:57:18 crc kubenswrapper[5052]: I1203 11:57:18.626362 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-scripts" Dec 03 11:57:18 crc kubenswrapper[5052]: I1203 11:57:18.642116 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:57:18 crc kubenswrapper[5052]: I1203 11:57:18.661809 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/placement-db-sync-8tdfb"] Dec 03 11:57:18 crc kubenswrapper[5052]: I1203 11:57:18.690267 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/barbican-db-sync-xs2lk"] Dec 03 11:57:18 crc kubenswrapper[5052]: I1203 11:57:18.692395 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-db-sync-xs2lk" Dec 03 11:57:18 crc kubenswrapper[5052]: I1203 11:57:18.693297 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-db-sync-8tdfb" Dec 03 11:57:18 crc kubenswrapper[5052]: I1203 11:57:18.696698 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"placement-placement-dockercfg-mtqj9" Dec 03 11:57:18 crc kubenswrapper[5052]: I1203 11:57:18.697017 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"barbican-config-data" Dec 03 11:57:18 crc kubenswrapper[5052]: I1203 11:57:18.697270 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"placement-config-data" Dec 03 11:57:18 crc kubenswrapper[5052]: I1203 11:57:18.697463 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"barbican-barbican-dockercfg-x9zph" Dec 03 11:57:18 crc kubenswrapper[5052]: I1203 11:57:18.700012 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"placement-scripts" Dec 03 11:57:18 crc kubenswrapper[5052]: I1203 11:57:18.703450 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-bootstrap-kdr6v" Dec 03 11:57:18 crc kubenswrapper[5052]: I1203 11:57:18.708471 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/placement-db-sync-8tdfb"] Dec 03 11:57:18 crc kubenswrapper[5052]: I1203 11:57:18.738752 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-db-sync-xs2lk"] Dec 03 11:57:18 crc kubenswrapper[5052]: I1203 11:57:18.785688 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/neutron-db-sync-xvd7n"] Dec 03 11:57:18 crc kubenswrapper[5052]: I1203 11:57:18.786984 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-db-sync-xvd7n" Dec 03 11:57:18 crc kubenswrapper[5052]: I1203 11:57:18.791114 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"neutron-config" Dec 03 11:57:18 crc kubenswrapper[5052]: I1203 11:57:18.791460 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"neutron-neutron-dockercfg-qpfvs" Dec 03 11:57:18 crc kubenswrapper[5052]: I1203 11:57:18.792407 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"neutron-httpd-config" Dec 03 11:57:18 crc kubenswrapper[5052]: I1203 11:57:18.794890 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-db-sync-xvd7n"] Dec 03 11:57:18 crc kubenswrapper[5052]: I1203 11:57:18.795862 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/14cb8e46-418d-4c42-ae97-a384a05b432f-scripts\") pod \"placement-db-sync-8tdfb\" (UID: \"14cb8e46-418d-4c42-ae97-a384a05b432f\") " pod="openstack-kuttl-tests/placement-db-sync-8tdfb" Dec 03 11:57:18 crc kubenswrapper[5052]: I1203 11:57:18.796050 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/14cb8e46-418d-4c42-ae97-a384a05b432f-combined-ca-bundle\") pod \"placement-db-sync-8tdfb\" (UID: \"14cb8e46-418d-4c42-ae97-a384a05b432f\") " pod="openstack-kuttl-tests/placement-db-sync-8tdfb" Dec 03 11:57:18 crc kubenswrapper[5052]: I1203 11:57:18.796084 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/bac97bd6-a939-41e2-b577-05d6fb87dcb5-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"bac97bd6-a939-41e2-b577-05d6fb87dcb5\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:57:18 crc kubenswrapper[5052]: I1203 11:57:18.796109 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/bac97bd6-a939-41e2-b577-05d6fb87dcb5-scripts\") pod \"ceilometer-0\" (UID: \"bac97bd6-a939-41e2-b577-05d6fb87dcb5\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:57:18 crc kubenswrapper[5052]: I1203 11:57:18.796160 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/bac97bd6-a939-41e2-b577-05d6fb87dcb5-config-data\") pod \"ceilometer-0\" (UID: \"bac97bd6-a939-41e2-b577-05d6fb87dcb5\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:57:18 crc kubenswrapper[5052]: I1203 11:57:18.796194 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8jrcb\" (UniqueName: \"kubernetes.io/projected/bac97bd6-a939-41e2-b577-05d6fb87dcb5-kube-api-access-8jrcb\") pod \"ceilometer-0\" (UID: \"bac97bd6-a939-41e2-b577-05d6fb87dcb5\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:57:18 crc kubenswrapper[5052]: I1203 11:57:18.796247 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pz5h5\" (UniqueName: \"kubernetes.io/projected/a50cd1a3-6eb7-4712-bb60-648250b396d6-kube-api-access-pz5h5\") pod \"barbican-db-sync-xs2lk\" (UID: \"a50cd1a3-6eb7-4712-bb60-648250b396d6\") " pod="openstack-kuttl-tests/barbican-db-sync-xs2lk" Dec 03 11:57:18 crc kubenswrapper[5052]: I1203 11:57:18.796278 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/14cb8e46-418d-4c42-ae97-a384a05b432f-config-data\") pod \"placement-db-sync-8tdfb\" (UID: \"14cb8e46-418d-4c42-ae97-a384a05b432f\") " pod="openstack-kuttl-tests/placement-db-sync-8tdfb" Dec 03 11:57:18 crc kubenswrapper[5052]: I1203 11:57:18.796316 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5x5ll\" (UniqueName: \"kubernetes.io/projected/14cb8e46-418d-4c42-ae97-a384a05b432f-kube-api-access-5x5ll\") pod \"placement-db-sync-8tdfb\" (UID: \"14cb8e46-418d-4c42-ae97-a384a05b432f\") " pod="openstack-kuttl-tests/placement-db-sync-8tdfb" Dec 03 11:57:18 crc kubenswrapper[5052]: I1203 11:57:18.796339 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/bac97bd6-a939-41e2-b577-05d6fb87dcb5-log-httpd\") pod \"ceilometer-0\" (UID: \"bac97bd6-a939-41e2-b577-05d6fb87dcb5\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:57:18 crc kubenswrapper[5052]: I1203 11:57:18.796365 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/bac97bd6-a939-41e2-b577-05d6fb87dcb5-run-httpd\") pod \"ceilometer-0\" (UID: \"bac97bd6-a939-41e2-b577-05d6fb87dcb5\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:57:18 crc kubenswrapper[5052]: I1203 11:57:18.796405 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/bac97bd6-a939-41e2-b577-05d6fb87dcb5-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"bac97bd6-a939-41e2-b577-05d6fb87dcb5\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:57:18 crc kubenswrapper[5052]: I1203 11:57:18.796435 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a50cd1a3-6eb7-4712-bb60-648250b396d6-combined-ca-bundle\") pod \"barbican-db-sync-xs2lk\" (UID: \"a50cd1a3-6eb7-4712-bb60-648250b396d6\") " pod="openstack-kuttl-tests/barbican-db-sync-xs2lk" Dec 03 11:57:18 crc kubenswrapper[5052]: I1203 11:57:18.796469 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/14cb8e46-418d-4c42-ae97-a384a05b432f-logs\") pod \"placement-db-sync-8tdfb\" (UID: \"14cb8e46-418d-4c42-ae97-a384a05b432f\") " pod="openstack-kuttl-tests/placement-db-sync-8tdfb" Dec 03 11:57:18 crc kubenswrapper[5052]: I1203 11:57:18.796662 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/a50cd1a3-6eb7-4712-bb60-648250b396d6-db-sync-config-data\") pod \"barbican-db-sync-xs2lk\" (UID: \"a50cd1a3-6eb7-4712-bb60-648250b396d6\") " pod="openstack-kuttl-tests/barbican-db-sync-xs2lk" Dec 03 11:57:18 crc kubenswrapper[5052]: I1203 11:57:18.825793 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/cinder-db-sync-76kbf"] Dec 03 11:57:18 crc kubenswrapper[5052]: I1203 11:57:18.827084 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-db-sync-76kbf" Dec 03 11:57:18 crc kubenswrapper[5052]: I1203 11:57:18.831073 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cinder-scripts" Dec 03 11:57:18 crc kubenswrapper[5052]: I1203 11:57:18.831186 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cinder-config-data" Dec 03 11:57:18 crc kubenswrapper[5052]: I1203 11:57:18.831394 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cinder-cinder-dockercfg-vq5vw" Dec 03 11:57:18 crc kubenswrapper[5052]: I1203 11:57:18.842846 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-db-sync-76kbf"] Dec 03 11:57:18 crc kubenswrapper[5052]: I1203 11:57:18.898312 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/bac97bd6-a939-41e2-b577-05d6fb87dcb5-scripts\") pod \"ceilometer-0\" (UID: \"bac97bd6-a939-41e2-b577-05d6fb87dcb5\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:57:18 crc kubenswrapper[5052]: I1203 11:57:18.898637 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/bac97bd6-a939-41e2-b577-05d6fb87dcb5-config-data\") pod \"ceilometer-0\" (UID: \"bac97bd6-a939-41e2-b577-05d6fb87dcb5\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:57:18 crc kubenswrapper[5052]: I1203 11:57:18.898766 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-8jrcb\" (UniqueName: \"kubernetes.io/projected/bac97bd6-a939-41e2-b577-05d6fb87dcb5-kube-api-access-8jrcb\") pod \"ceilometer-0\" (UID: \"bac97bd6-a939-41e2-b577-05d6fb87dcb5\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:57:18 crc kubenswrapper[5052]: I1203 11:57:18.899569 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-pz5h5\" (UniqueName: \"kubernetes.io/projected/a50cd1a3-6eb7-4712-bb60-648250b396d6-kube-api-access-pz5h5\") pod \"barbican-db-sync-xs2lk\" (UID: \"a50cd1a3-6eb7-4712-bb60-648250b396d6\") " pod="openstack-kuttl-tests/barbican-db-sync-xs2lk" Dec 03 11:57:18 crc kubenswrapper[5052]: I1203 11:57:18.899699 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/14cb8e46-418d-4c42-ae97-a384a05b432f-config-data\") pod \"placement-db-sync-8tdfb\" (UID: \"14cb8e46-418d-4c42-ae97-a384a05b432f\") " pod="openstack-kuttl-tests/placement-db-sync-8tdfb" Dec 03 11:57:18 crc kubenswrapper[5052]: I1203 11:57:18.900350 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-5x5ll\" (UniqueName: \"kubernetes.io/projected/14cb8e46-418d-4c42-ae97-a384a05b432f-kube-api-access-5x5ll\") pod \"placement-db-sync-8tdfb\" (UID: \"14cb8e46-418d-4c42-ae97-a384a05b432f\") " pod="openstack-kuttl-tests/placement-db-sync-8tdfb" Dec 03 11:57:18 crc kubenswrapper[5052]: I1203 11:57:18.900478 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/bac97bd6-a939-41e2-b577-05d6fb87dcb5-log-httpd\") pod \"ceilometer-0\" (UID: \"bac97bd6-a939-41e2-b577-05d6fb87dcb5\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:57:18 crc kubenswrapper[5052]: I1203 11:57:18.900666 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/bac97bd6-a939-41e2-b577-05d6fb87dcb5-run-httpd\") pod \"ceilometer-0\" (UID: \"bac97bd6-a939-41e2-b577-05d6fb87dcb5\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:57:18 crc kubenswrapper[5052]: I1203 11:57:18.900794 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/bac97bd6-a939-41e2-b577-05d6fb87dcb5-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"bac97bd6-a939-41e2-b577-05d6fb87dcb5\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:57:18 crc kubenswrapper[5052]: I1203 11:57:18.900925 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a50cd1a3-6eb7-4712-bb60-648250b396d6-combined-ca-bundle\") pod \"barbican-db-sync-xs2lk\" (UID: \"a50cd1a3-6eb7-4712-bb60-648250b396d6\") " pod="openstack-kuttl-tests/barbican-db-sync-xs2lk" Dec 03 11:57:18 crc kubenswrapper[5052]: I1203 11:57:18.901033 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/14cb8e46-418d-4c42-ae97-a384a05b432f-logs\") pod \"placement-db-sync-8tdfb\" (UID: \"14cb8e46-418d-4c42-ae97-a384a05b432f\") " pod="openstack-kuttl-tests/placement-db-sync-8tdfb" Dec 03 11:57:18 crc kubenswrapper[5052]: I1203 11:57:18.901142 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/a50cd1a3-6eb7-4712-bb60-648250b396d6-db-sync-config-data\") pod \"barbican-db-sync-xs2lk\" (UID: \"a50cd1a3-6eb7-4712-bb60-648250b396d6\") " pod="openstack-kuttl-tests/barbican-db-sync-xs2lk" Dec 03 11:57:18 crc kubenswrapper[5052]: I1203 11:57:18.901322 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/14cb8e46-418d-4c42-ae97-a384a05b432f-scripts\") pod \"placement-db-sync-8tdfb\" (UID: \"14cb8e46-418d-4c42-ae97-a384a05b432f\") " pod="openstack-kuttl-tests/placement-db-sync-8tdfb" Dec 03 11:57:18 crc kubenswrapper[5052]: I1203 11:57:18.901472 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c7d7v\" (UniqueName: \"kubernetes.io/projected/28c32800-797f-4560-ac08-55c9ca254a79-kube-api-access-c7d7v\") pod \"neutron-db-sync-xvd7n\" (UID: \"28c32800-797f-4560-ac08-55c9ca254a79\") " pod="openstack-kuttl-tests/neutron-db-sync-xvd7n" Dec 03 11:57:18 crc kubenswrapper[5052]: I1203 11:57:18.901585 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/28c32800-797f-4560-ac08-55c9ca254a79-config\") pod \"neutron-db-sync-xvd7n\" (UID: \"28c32800-797f-4560-ac08-55c9ca254a79\") " pod="openstack-kuttl-tests/neutron-db-sync-xvd7n" Dec 03 11:57:18 crc kubenswrapper[5052]: I1203 11:57:18.901747 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/28c32800-797f-4560-ac08-55c9ca254a79-combined-ca-bundle\") pod \"neutron-db-sync-xvd7n\" (UID: \"28c32800-797f-4560-ac08-55c9ca254a79\") " pod="openstack-kuttl-tests/neutron-db-sync-xvd7n" Dec 03 11:57:18 crc kubenswrapper[5052]: I1203 11:57:18.901890 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/14cb8e46-418d-4c42-ae97-a384a05b432f-combined-ca-bundle\") pod \"placement-db-sync-8tdfb\" (UID: \"14cb8e46-418d-4c42-ae97-a384a05b432f\") " pod="openstack-kuttl-tests/placement-db-sync-8tdfb" Dec 03 11:57:18 crc kubenswrapper[5052]: I1203 11:57:18.902000 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/bac97bd6-a939-41e2-b577-05d6fb87dcb5-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"bac97bd6-a939-41e2-b577-05d6fb87dcb5\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:57:18 crc kubenswrapper[5052]: I1203 11:57:18.901966 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/14cb8e46-418d-4c42-ae97-a384a05b432f-logs\") pod \"placement-db-sync-8tdfb\" (UID: \"14cb8e46-418d-4c42-ae97-a384a05b432f\") " pod="openstack-kuttl-tests/placement-db-sync-8tdfb" Dec 03 11:57:18 crc kubenswrapper[5052]: I1203 11:57:18.904004 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/bac97bd6-a939-41e2-b577-05d6fb87dcb5-log-httpd\") pod \"ceilometer-0\" (UID: \"bac97bd6-a939-41e2-b577-05d6fb87dcb5\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:57:18 crc kubenswrapper[5052]: I1203 11:57:18.905616 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/bac97bd6-a939-41e2-b577-05d6fb87dcb5-run-httpd\") pod \"ceilometer-0\" (UID: \"bac97bd6-a939-41e2-b577-05d6fb87dcb5\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:57:18 crc kubenswrapper[5052]: I1203 11:57:18.909142 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/bac97bd6-a939-41e2-b577-05d6fb87dcb5-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"bac97bd6-a939-41e2-b577-05d6fb87dcb5\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:57:18 crc kubenswrapper[5052]: I1203 11:57:18.909188 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/14cb8e46-418d-4c42-ae97-a384a05b432f-scripts\") pod \"placement-db-sync-8tdfb\" (UID: \"14cb8e46-418d-4c42-ae97-a384a05b432f\") " pod="openstack-kuttl-tests/placement-db-sync-8tdfb" Dec 03 11:57:18 crc kubenswrapper[5052]: I1203 11:57:18.909690 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/14cb8e46-418d-4c42-ae97-a384a05b432f-config-data\") pod \"placement-db-sync-8tdfb\" (UID: \"14cb8e46-418d-4c42-ae97-a384a05b432f\") " pod="openstack-kuttl-tests/placement-db-sync-8tdfb" Dec 03 11:57:18 crc kubenswrapper[5052]: I1203 11:57:18.910438 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/bac97bd6-a939-41e2-b577-05d6fb87dcb5-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"bac97bd6-a939-41e2-b577-05d6fb87dcb5\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:57:18 crc kubenswrapper[5052]: I1203 11:57:18.910618 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/bac97bd6-a939-41e2-b577-05d6fb87dcb5-config-data\") pod \"ceilometer-0\" (UID: \"bac97bd6-a939-41e2-b577-05d6fb87dcb5\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:57:18 crc kubenswrapper[5052]: I1203 11:57:18.914459 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/14cb8e46-418d-4c42-ae97-a384a05b432f-combined-ca-bundle\") pod \"placement-db-sync-8tdfb\" (UID: \"14cb8e46-418d-4c42-ae97-a384a05b432f\") " pod="openstack-kuttl-tests/placement-db-sync-8tdfb" Dec 03 11:57:18 crc kubenswrapper[5052]: I1203 11:57:18.916476 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/bac97bd6-a939-41e2-b577-05d6fb87dcb5-scripts\") pod \"ceilometer-0\" (UID: \"bac97bd6-a939-41e2-b577-05d6fb87dcb5\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:57:18 crc kubenswrapper[5052]: I1203 11:57:18.918118 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/a50cd1a3-6eb7-4712-bb60-648250b396d6-db-sync-config-data\") pod \"barbican-db-sync-xs2lk\" (UID: \"a50cd1a3-6eb7-4712-bb60-648250b396d6\") " pod="openstack-kuttl-tests/barbican-db-sync-xs2lk" Dec 03 11:57:18 crc kubenswrapper[5052]: I1203 11:57:18.919160 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a50cd1a3-6eb7-4712-bb60-648250b396d6-combined-ca-bundle\") pod \"barbican-db-sync-xs2lk\" (UID: \"a50cd1a3-6eb7-4712-bb60-648250b396d6\") " pod="openstack-kuttl-tests/barbican-db-sync-xs2lk" Dec 03 11:57:18 crc kubenswrapper[5052]: I1203 11:57:18.919422 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-8jrcb\" (UniqueName: \"kubernetes.io/projected/bac97bd6-a939-41e2-b577-05d6fb87dcb5-kube-api-access-8jrcb\") pod \"ceilometer-0\" (UID: \"bac97bd6-a939-41e2-b577-05d6fb87dcb5\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:57:18 crc kubenswrapper[5052]: I1203 11:57:18.921870 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-pz5h5\" (UniqueName: \"kubernetes.io/projected/a50cd1a3-6eb7-4712-bb60-648250b396d6-kube-api-access-pz5h5\") pod \"barbican-db-sync-xs2lk\" (UID: \"a50cd1a3-6eb7-4712-bb60-648250b396d6\") " pod="openstack-kuttl-tests/barbican-db-sync-xs2lk" Dec 03 11:57:18 crc kubenswrapper[5052]: I1203 11:57:18.934452 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-5x5ll\" (UniqueName: \"kubernetes.io/projected/14cb8e46-418d-4c42-ae97-a384a05b432f-kube-api-access-5x5ll\") pod \"placement-db-sync-8tdfb\" (UID: \"14cb8e46-418d-4c42-ae97-a384a05b432f\") " pod="openstack-kuttl-tests/placement-db-sync-8tdfb" Dec 03 11:57:18 crc kubenswrapper[5052]: I1203 11:57:18.947019 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:57:19 crc kubenswrapper[5052]: I1203 11:57:19.004005 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-c7d7v\" (UniqueName: \"kubernetes.io/projected/28c32800-797f-4560-ac08-55c9ca254a79-kube-api-access-c7d7v\") pod \"neutron-db-sync-xvd7n\" (UID: \"28c32800-797f-4560-ac08-55c9ca254a79\") " pod="openstack-kuttl-tests/neutron-db-sync-xvd7n" Dec 03 11:57:19 crc kubenswrapper[5052]: I1203 11:57:19.004069 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/28c32800-797f-4560-ac08-55c9ca254a79-config\") pod \"neutron-db-sync-xvd7n\" (UID: \"28c32800-797f-4560-ac08-55c9ca254a79\") " pod="openstack-kuttl-tests/neutron-db-sync-xvd7n" Dec 03 11:57:19 crc kubenswrapper[5052]: I1203 11:57:19.004094 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/28c32800-797f-4560-ac08-55c9ca254a79-combined-ca-bundle\") pod \"neutron-db-sync-xvd7n\" (UID: \"28c32800-797f-4560-ac08-55c9ca254a79\") " pod="openstack-kuttl-tests/neutron-db-sync-xvd7n" Dec 03 11:57:19 crc kubenswrapper[5052]: I1203 11:57:19.004164 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/16630eec-cd03-42b8-95aa-d9a960c2858c-combined-ca-bundle\") pod \"cinder-db-sync-76kbf\" (UID: \"16630eec-cd03-42b8-95aa-d9a960c2858c\") " pod="openstack-kuttl-tests/cinder-db-sync-76kbf" Dec 03 11:57:19 crc kubenswrapper[5052]: I1203 11:57:19.004221 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/16630eec-cd03-42b8-95aa-d9a960c2858c-config-data\") pod \"cinder-db-sync-76kbf\" (UID: \"16630eec-cd03-42b8-95aa-d9a960c2858c\") " pod="openstack-kuttl-tests/cinder-db-sync-76kbf" Dec 03 11:57:19 crc kubenswrapper[5052]: I1203 11:57:19.004243 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x4z68\" (UniqueName: \"kubernetes.io/projected/16630eec-cd03-42b8-95aa-d9a960c2858c-kube-api-access-x4z68\") pod \"cinder-db-sync-76kbf\" (UID: \"16630eec-cd03-42b8-95aa-d9a960c2858c\") " pod="openstack-kuttl-tests/cinder-db-sync-76kbf" Dec 03 11:57:19 crc kubenswrapper[5052]: I1203 11:57:19.004285 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/16630eec-cd03-42b8-95aa-d9a960c2858c-etc-machine-id\") pod \"cinder-db-sync-76kbf\" (UID: \"16630eec-cd03-42b8-95aa-d9a960c2858c\") " pod="openstack-kuttl-tests/cinder-db-sync-76kbf" Dec 03 11:57:19 crc kubenswrapper[5052]: I1203 11:57:19.004317 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/16630eec-cd03-42b8-95aa-d9a960c2858c-db-sync-config-data\") pod \"cinder-db-sync-76kbf\" (UID: \"16630eec-cd03-42b8-95aa-d9a960c2858c\") " pod="openstack-kuttl-tests/cinder-db-sync-76kbf" Dec 03 11:57:19 crc kubenswrapper[5052]: I1203 11:57:19.004374 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/16630eec-cd03-42b8-95aa-d9a960c2858c-scripts\") pod \"cinder-db-sync-76kbf\" (UID: \"16630eec-cd03-42b8-95aa-d9a960c2858c\") " pod="openstack-kuttl-tests/cinder-db-sync-76kbf" Dec 03 11:57:19 crc kubenswrapper[5052]: I1203 11:57:19.007848 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/28c32800-797f-4560-ac08-55c9ca254a79-combined-ca-bundle\") pod \"neutron-db-sync-xvd7n\" (UID: \"28c32800-797f-4560-ac08-55c9ca254a79\") " pod="openstack-kuttl-tests/neutron-db-sync-xvd7n" Dec 03 11:57:19 crc kubenswrapper[5052]: I1203 11:57:19.009166 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/28c32800-797f-4560-ac08-55c9ca254a79-config\") pod \"neutron-db-sync-xvd7n\" (UID: \"28c32800-797f-4560-ac08-55c9ca254a79\") " pod="openstack-kuttl-tests/neutron-db-sync-xvd7n" Dec 03 11:57:19 crc kubenswrapper[5052]: I1203 11:57:19.020740 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-c7d7v\" (UniqueName: \"kubernetes.io/projected/28c32800-797f-4560-ac08-55c9ca254a79-kube-api-access-c7d7v\") pod \"neutron-db-sync-xvd7n\" (UID: \"28c32800-797f-4560-ac08-55c9ca254a79\") " pod="openstack-kuttl-tests/neutron-db-sync-xvd7n" Dec 03 11:57:19 crc kubenswrapper[5052]: I1203 11:57:19.028884 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-db-sync-xs2lk" Dec 03 11:57:19 crc kubenswrapper[5052]: I1203 11:57:19.052979 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-db-sync-8tdfb" Dec 03 11:57:19 crc kubenswrapper[5052]: I1203 11:57:19.106121 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/16630eec-cd03-42b8-95aa-d9a960c2858c-scripts\") pod \"cinder-db-sync-76kbf\" (UID: \"16630eec-cd03-42b8-95aa-d9a960c2858c\") " pod="openstack-kuttl-tests/cinder-db-sync-76kbf" Dec 03 11:57:19 crc kubenswrapper[5052]: I1203 11:57:19.106264 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/16630eec-cd03-42b8-95aa-d9a960c2858c-combined-ca-bundle\") pod \"cinder-db-sync-76kbf\" (UID: \"16630eec-cd03-42b8-95aa-d9a960c2858c\") " pod="openstack-kuttl-tests/cinder-db-sync-76kbf" Dec 03 11:57:19 crc kubenswrapper[5052]: I1203 11:57:19.106336 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/16630eec-cd03-42b8-95aa-d9a960c2858c-config-data\") pod \"cinder-db-sync-76kbf\" (UID: \"16630eec-cd03-42b8-95aa-d9a960c2858c\") " pod="openstack-kuttl-tests/cinder-db-sync-76kbf" Dec 03 11:57:19 crc kubenswrapper[5052]: I1203 11:57:19.106267 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-db-sync-xvd7n" Dec 03 11:57:19 crc kubenswrapper[5052]: I1203 11:57:19.106731 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-x4z68\" (UniqueName: \"kubernetes.io/projected/16630eec-cd03-42b8-95aa-d9a960c2858c-kube-api-access-x4z68\") pod \"cinder-db-sync-76kbf\" (UID: \"16630eec-cd03-42b8-95aa-d9a960c2858c\") " pod="openstack-kuttl-tests/cinder-db-sync-76kbf" Dec 03 11:57:19 crc kubenswrapper[5052]: I1203 11:57:19.106804 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/16630eec-cd03-42b8-95aa-d9a960c2858c-etc-machine-id\") pod \"cinder-db-sync-76kbf\" (UID: \"16630eec-cd03-42b8-95aa-d9a960c2858c\") " pod="openstack-kuttl-tests/cinder-db-sync-76kbf" Dec 03 11:57:19 crc kubenswrapper[5052]: I1203 11:57:19.107079 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/16630eec-cd03-42b8-95aa-d9a960c2858c-db-sync-config-data\") pod \"cinder-db-sync-76kbf\" (UID: \"16630eec-cd03-42b8-95aa-d9a960c2858c\") " pod="openstack-kuttl-tests/cinder-db-sync-76kbf" Dec 03 11:57:19 crc kubenswrapper[5052]: I1203 11:57:19.107129 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/16630eec-cd03-42b8-95aa-d9a960c2858c-etc-machine-id\") pod \"cinder-db-sync-76kbf\" (UID: \"16630eec-cd03-42b8-95aa-d9a960c2858c\") " pod="openstack-kuttl-tests/cinder-db-sync-76kbf" Dec 03 11:57:19 crc kubenswrapper[5052]: I1203 11:57:19.110941 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/16630eec-cd03-42b8-95aa-d9a960c2858c-db-sync-config-data\") pod \"cinder-db-sync-76kbf\" (UID: \"16630eec-cd03-42b8-95aa-d9a960c2858c\") " pod="openstack-kuttl-tests/cinder-db-sync-76kbf" Dec 03 11:57:19 crc kubenswrapper[5052]: I1203 11:57:19.111438 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/16630eec-cd03-42b8-95aa-d9a960c2858c-config-data\") pod \"cinder-db-sync-76kbf\" (UID: \"16630eec-cd03-42b8-95aa-d9a960c2858c\") " pod="openstack-kuttl-tests/cinder-db-sync-76kbf" Dec 03 11:57:19 crc kubenswrapper[5052]: I1203 11:57:19.111465 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/16630eec-cd03-42b8-95aa-d9a960c2858c-combined-ca-bundle\") pod \"cinder-db-sync-76kbf\" (UID: \"16630eec-cd03-42b8-95aa-d9a960c2858c\") " pod="openstack-kuttl-tests/cinder-db-sync-76kbf" Dec 03 11:57:19 crc kubenswrapper[5052]: I1203 11:57:19.111622 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/16630eec-cd03-42b8-95aa-d9a960c2858c-scripts\") pod \"cinder-db-sync-76kbf\" (UID: \"16630eec-cd03-42b8-95aa-d9a960c2858c\") " pod="openstack-kuttl-tests/cinder-db-sync-76kbf" Dec 03 11:57:19 crc kubenswrapper[5052]: I1203 11:57:19.124486 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-x4z68\" (UniqueName: \"kubernetes.io/projected/16630eec-cd03-42b8-95aa-d9a960c2858c-kube-api-access-x4z68\") pod \"cinder-db-sync-76kbf\" (UID: \"16630eec-cd03-42b8-95aa-d9a960c2858c\") " pod="openstack-kuttl-tests/cinder-db-sync-76kbf" Dec 03 11:57:19 crc kubenswrapper[5052]: I1203 11:57:19.151211 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-db-sync-76kbf" Dec 03 11:57:19 crc kubenswrapper[5052]: I1203 11:57:19.423841 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 11:57:19 crc kubenswrapper[5052]: I1203 11:57:19.428524 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:57:19 crc kubenswrapper[5052]: I1203 11:57:19.430328 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-default-external-config-data" Dec 03 11:57:19 crc kubenswrapper[5052]: I1203 11:57:19.431607 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-scripts" Dec 03 11:57:19 crc kubenswrapper[5052]: I1203 11:57:19.431733 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-glance-default-public-svc" Dec 03 11:57:19 crc kubenswrapper[5052]: I1203 11:57:19.431616 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-glance-dockercfg-8nxm6" Dec 03 11:57:19 crc kubenswrapper[5052]: I1203 11:57:19.438209 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 11:57:19 crc kubenswrapper[5052]: I1203 11:57:19.509247 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 11:57:19 crc kubenswrapper[5052]: I1203 11:57:19.510814 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:57:19 crc kubenswrapper[5052]: I1203 11:57:19.514664 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-default-internal-config-data" Dec 03 11:57:19 crc kubenswrapper[5052]: I1203 11:57:19.516469 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-glance-default-internal-svc" Dec 03 11:57:19 crc kubenswrapper[5052]: I1203 11:57:19.530736 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 11:57:19 crc kubenswrapper[5052]: I1203 11:57:19.623310 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rc4pw\" (UniqueName: \"kubernetes.io/projected/bf275c0c-27e5-462b-883e-9483cfe91bcd-kube-api-access-rc4pw\") pod \"glance-default-external-api-0\" (UID: \"bf275c0c-27e5-462b-883e-9483cfe91bcd\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:57:19 crc kubenswrapper[5052]: I1203 11:57:19.623374 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/32c3a6f2-948d-47a6-ad0f-2784e2623e8c-logs\") pod \"glance-default-internal-api-0\" (UID: \"32c3a6f2-948d-47a6-ad0f-2784e2623e8c\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:57:19 crc kubenswrapper[5052]: I1203 11:57:19.623441 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/32c3a6f2-948d-47a6-ad0f-2784e2623e8c-scripts\") pod \"glance-default-internal-api-0\" (UID: \"32c3a6f2-948d-47a6-ad0f-2784e2623e8c\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:57:19 crc kubenswrapper[5052]: I1203 11:57:19.623481 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/bf275c0c-27e5-462b-883e-9483cfe91bcd-combined-ca-bundle\") pod \"glance-default-external-api-0\" (UID: \"bf275c0c-27e5-462b-883e-9483cfe91bcd\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:57:19 crc kubenswrapper[5052]: I1203 11:57:19.623501 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/32c3a6f2-948d-47a6-ad0f-2784e2623e8c-httpd-run\") pod \"glance-default-internal-api-0\" (UID: \"32c3a6f2-948d-47a6-ad0f-2784e2623e8c\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:57:19 crc kubenswrapper[5052]: I1203 11:57:19.623536 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/bf275c0c-27e5-462b-883e-9483cfe91bcd-httpd-run\") pod \"glance-default-external-api-0\" (UID: \"bf275c0c-27e5-462b-883e-9483cfe91bcd\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:57:19 crc kubenswrapper[5052]: I1203 11:57:19.623559 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/bf275c0c-27e5-462b-883e-9483cfe91bcd-public-tls-certs\") pod \"glance-default-external-api-0\" (UID: \"bf275c0c-27e5-462b-883e-9483cfe91bcd\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:57:19 crc kubenswrapper[5052]: I1203 11:57:19.623594 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/32c3a6f2-948d-47a6-ad0f-2784e2623e8c-internal-tls-certs\") pod \"glance-default-internal-api-0\" (UID: \"32c3a6f2-948d-47a6-ad0f-2784e2623e8c\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:57:19 crc kubenswrapper[5052]: I1203 11:57:19.623645 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/bf275c0c-27e5-462b-883e-9483cfe91bcd-config-data\") pod \"glance-default-external-api-0\" (UID: \"bf275c0c-27e5-462b-883e-9483cfe91bcd\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:57:19 crc kubenswrapper[5052]: I1203 11:57:19.623670 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/bf275c0c-27e5-462b-883e-9483cfe91bcd-scripts\") pod \"glance-default-external-api-0\" (UID: \"bf275c0c-27e5-462b-883e-9483cfe91bcd\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:57:19 crc kubenswrapper[5052]: I1203 11:57:19.623729 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/32c3a6f2-948d-47a6-ad0f-2784e2623e8c-config-data\") pod \"glance-default-internal-api-0\" (UID: \"32c3a6f2-948d-47a6-ad0f-2784e2623e8c\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:57:19 crc kubenswrapper[5052]: I1203 11:57:19.623849 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage16-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage16-crc\") pod \"glance-default-external-api-0\" (UID: \"bf275c0c-27e5-462b-883e-9483cfe91bcd\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:57:19 crc kubenswrapper[5052]: I1203 11:57:19.623902 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/32c3a6f2-948d-47a6-ad0f-2784e2623e8c-combined-ca-bundle\") pod \"glance-default-internal-api-0\" (UID: \"32c3a6f2-948d-47a6-ad0f-2784e2623e8c\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:57:19 crc kubenswrapper[5052]: I1203 11:57:19.624030 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/bf275c0c-27e5-462b-883e-9483cfe91bcd-logs\") pod \"glance-default-external-api-0\" (UID: \"bf275c0c-27e5-462b-883e-9483cfe91bcd\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:57:19 crc kubenswrapper[5052]: I1203 11:57:19.624101 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage15-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage15-crc\") pod \"glance-default-internal-api-0\" (UID: \"32c3a6f2-948d-47a6-ad0f-2784e2623e8c\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:57:19 crc kubenswrapper[5052]: I1203 11:57:19.624120 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7nzs8\" (UniqueName: \"kubernetes.io/projected/32c3a6f2-948d-47a6-ad0f-2784e2623e8c-kube-api-access-7nzs8\") pod \"glance-default-internal-api-0\" (UID: \"32c3a6f2-948d-47a6-ad0f-2784e2623e8c\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:57:19 crc kubenswrapper[5052]: I1203 11:57:19.725442 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage15-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage15-crc\") pod \"glance-default-internal-api-0\" (UID: \"32c3a6f2-948d-47a6-ad0f-2784e2623e8c\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:57:19 crc kubenswrapper[5052]: I1203 11:57:19.725489 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-7nzs8\" (UniqueName: \"kubernetes.io/projected/32c3a6f2-948d-47a6-ad0f-2784e2623e8c-kube-api-access-7nzs8\") pod \"glance-default-internal-api-0\" (UID: \"32c3a6f2-948d-47a6-ad0f-2784e2623e8c\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:57:19 crc kubenswrapper[5052]: I1203 11:57:19.725564 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-rc4pw\" (UniqueName: \"kubernetes.io/projected/bf275c0c-27e5-462b-883e-9483cfe91bcd-kube-api-access-rc4pw\") pod \"glance-default-external-api-0\" (UID: \"bf275c0c-27e5-462b-883e-9483cfe91bcd\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:57:19 crc kubenswrapper[5052]: I1203 11:57:19.725610 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/32c3a6f2-948d-47a6-ad0f-2784e2623e8c-logs\") pod \"glance-default-internal-api-0\" (UID: \"32c3a6f2-948d-47a6-ad0f-2784e2623e8c\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:57:19 crc kubenswrapper[5052]: I1203 11:57:19.725644 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/32c3a6f2-948d-47a6-ad0f-2784e2623e8c-scripts\") pod \"glance-default-internal-api-0\" (UID: \"32c3a6f2-948d-47a6-ad0f-2784e2623e8c\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:57:19 crc kubenswrapper[5052]: I1203 11:57:19.725676 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/bf275c0c-27e5-462b-883e-9483cfe91bcd-combined-ca-bundle\") pod \"glance-default-external-api-0\" (UID: \"bf275c0c-27e5-462b-883e-9483cfe91bcd\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:57:19 crc kubenswrapper[5052]: I1203 11:57:19.725702 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/32c3a6f2-948d-47a6-ad0f-2784e2623e8c-httpd-run\") pod \"glance-default-internal-api-0\" (UID: \"32c3a6f2-948d-47a6-ad0f-2784e2623e8c\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:57:19 crc kubenswrapper[5052]: I1203 11:57:19.725729 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/bf275c0c-27e5-462b-883e-9483cfe91bcd-httpd-run\") pod \"glance-default-external-api-0\" (UID: \"bf275c0c-27e5-462b-883e-9483cfe91bcd\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:57:19 crc kubenswrapper[5052]: I1203 11:57:19.725753 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/bf275c0c-27e5-462b-883e-9483cfe91bcd-public-tls-certs\") pod \"glance-default-external-api-0\" (UID: \"bf275c0c-27e5-462b-883e-9483cfe91bcd\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:57:19 crc kubenswrapper[5052]: I1203 11:57:19.725780 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/32c3a6f2-948d-47a6-ad0f-2784e2623e8c-internal-tls-certs\") pod \"glance-default-internal-api-0\" (UID: \"32c3a6f2-948d-47a6-ad0f-2784e2623e8c\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:57:19 crc kubenswrapper[5052]: I1203 11:57:19.725805 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/bf275c0c-27e5-462b-883e-9483cfe91bcd-config-data\") pod \"glance-default-external-api-0\" (UID: \"bf275c0c-27e5-462b-883e-9483cfe91bcd\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:57:19 crc kubenswrapper[5052]: I1203 11:57:19.725831 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/bf275c0c-27e5-462b-883e-9483cfe91bcd-scripts\") pod \"glance-default-external-api-0\" (UID: \"bf275c0c-27e5-462b-883e-9483cfe91bcd\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:57:19 crc kubenswrapper[5052]: I1203 11:57:19.725854 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/32c3a6f2-948d-47a6-ad0f-2784e2623e8c-config-data\") pod \"glance-default-internal-api-0\" (UID: \"32c3a6f2-948d-47a6-ad0f-2784e2623e8c\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:57:19 crc kubenswrapper[5052]: I1203 11:57:19.725881 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage16-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage16-crc\") pod \"glance-default-external-api-0\" (UID: \"bf275c0c-27e5-462b-883e-9483cfe91bcd\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:57:19 crc kubenswrapper[5052]: I1203 11:57:19.725914 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/32c3a6f2-948d-47a6-ad0f-2784e2623e8c-combined-ca-bundle\") pod \"glance-default-internal-api-0\" (UID: \"32c3a6f2-948d-47a6-ad0f-2784e2623e8c\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:57:19 crc kubenswrapper[5052]: I1203 11:57:19.725972 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/bf275c0c-27e5-462b-883e-9483cfe91bcd-logs\") pod \"glance-default-external-api-0\" (UID: \"bf275c0c-27e5-462b-883e-9483cfe91bcd\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:57:19 crc kubenswrapper[5052]: I1203 11:57:19.726583 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/bf275c0c-27e5-462b-883e-9483cfe91bcd-logs\") pod \"glance-default-external-api-0\" (UID: \"bf275c0c-27e5-462b-883e-9483cfe91bcd\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:57:19 crc kubenswrapper[5052]: I1203 11:57:19.727696 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/32c3a6f2-948d-47a6-ad0f-2784e2623e8c-httpd-run\") pod \"glance-default-internal-api-0\" (UID: \"32c3a6f2-948d-47a6-ad0f-2784e2623e8c\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:57:19 crc kubenswrapper[5052]: I1203 11:57:19.727783 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/32c3a6f2-948d-47a6-ad0f-2784e2623e8c-logs\") pod \"glance-default-internal-api-0\" (UID: \"32c3a6f2-948d-47a6-ad0f-2784e2623e8c\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:57:19 crc kubenswrapper[5052]: I1203 11:57:19.728480 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage15-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage15-crc\") pod \"glance-default-internal-api-0\" (UID: \"32c3a6f2-948d-47a6-ad0f-2784e2623e8c\") device mount path \"/mnt/openstack/pv15\"" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:57:19 crc kubenswrapper[5052]: I1203 11:57:19.729689 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/bf275c0c-27e5-462b-883e-9483cfe91bcd-httpd-run\") pod \"glance-default-external-api-0\" (UID: \"bf275c0c-27e5-462b-883e-9483cfe91bcd\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:57:19 crc kubenswrapper[5052]: I1203 11:57:19.729851 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage16-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage16-crc\") pod \"glance-default-external-api-0\" (UID: \"bf275c0c-27e5-462b-883e-9483cfe91bcd\") device mount path \"/mnt/openstack/pv16\"" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:57:19 crc kubenswrapper[5052]: I1203 11:57:19.737705 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/32c3a6f2-948d-47a6-ad0f-2784e2623e8c-config-data\") pod \"glance-default-internal-api-0\" (UID: \"32c3a6f2-948d-47a6-ad0f-2784e2623e8c\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:57:19 crc kubenswrapper[5052]: I1203 11:57:19.738329 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/bf275c0c-27e5-462b-883e-9483cfe91bcd-scripts\") pod \"glance-default-external-api-0\" (UID: \"bf275c0c-27e5-462b-883e-9483cfe91bcd\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:57:19 crc kubenswrapper[5052]: I1203 11:57:19.738865 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/32c3a6f2-948d-47a6-ad0f-2784e2623e8c-scripts\") pod \"glance-default-internal-api-0\" (UID: \"32c3a6f2-948d-47a6-ad0f-2784e2623e8c\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:57:19 crc kubenswrapper[5052]: I1203 11:57:19.740247 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/bf275c0c-27e5-462b-883e-9483cfe91bcd-combined-ca-bundle\") pod \"glance-default-external-api-0\" (UID: \"bf275c0c-27e5-462b-883e-9483cfe91bcd\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:57:19 crc kubenswrapper[5052]: I1203 11:57:19.744723 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/bf275c0c-27e5-462b-883e-9483cfe91bcd-config-data\") pod \"glance-default-external-api-0\" (UID: \"bf275c0c-27e5-462b-883e-9483cfe91bcd\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:57:19 crc kubenswrapper[5052]: I1203 11:57:19.745296 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/bf275c0c-27e5-462b-883e-9483cfe91bcd-public-tls-certs\") pod \"glance-default-external-api-0\" (UID: \"bf275c0c-27e5-462b-883e-9483cfe91bcd\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:57:19 crc kubenswrapper[5052]: I1203 11:57:19.746595 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/32c3a6f2-948d-47a6-ad0f-2784e2623e8c-internal-tls-certs\") pod \"glance-default-internal-api-0\" (UID: \"32c3a6f2-948d-47a6-ad0f-2784e2623e8c\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:57:19 crc kubenswrapper[5052]: I1203 11:57:19.754086 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-7nzs8\" (UniqueName: \"kubernetes.io/projected/32c3a6f2-948d-47a6-ad0f-2784e2623e8c-kube-api-access-7nzs8\") pod \"glance-default-internal-api-0\" (UID: \"32c3a6f2-948d-47a6-ad0f-2784e2623e8c\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:57:19 crc kubenswrapper[5052]: I1203 11:57:19.756629 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/32c3a6f2-948d-47a6-ad0f-2784e2623e8c-combined-ca-bundle\") pod \"glance-default-internal-api-0\" (UID: \"32c3a6f2-948d-47a6-ad0f-2784e2623e8c\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:57:19 crc kubenswrapper[5052]: I1203 11:57:19.757866 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-rc4pw\" (UniqueName: \"kubernetes.io/projected/bf275c0c-27e5-462b-883e-9483cfe91bcd-kube-api-access-rc4pw\") pod \"glance-default-external-api-0\" (UID: \"bf275c0c-27e5-462b-883e-9483cfe91bcd\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:57:19 crc kubenswrapper[5052]: I1203 11:57:19.771822 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage15-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage15-crc\") pod \"glance-default-internal-api-0\" (UID: \"32c3a6f2-948d-47a6-ad0f-2784e2623e8c\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:57:19 crc kubenswrapper[5052]: I1203 11:57:19.782890 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage16-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage16-crc\") pod \"glance-default-external-api-0\" (UID: \"bf275c0c-27e5-462b-883e-9483cfe91bcd\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:57:19 crc kubenswrapper[5052]: I1203 11:57:19.874989 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:57:20 crc kubenswrapper[5052]: I1203 11:57:20.050446 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:57:20 crc kubenswrapper[5052]: I1203 11:57:20.262129 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 11:57:20 crc kubenswrapper[5052]: I1203 11:57:20.303309 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:57:20 crc kubenswrapper[5052]: I1203 11:57:20.370649 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 11:57:24 crc kubenswrapper[5052]: I1203 11:57:24.313400 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/prometheus-metric-storage-0" podUID="33918bc1-340a-4d26-a204-001b85c15f04" containerName="prometheus" probeResult="failure" output="Get \"http://10.217.1.166:9090/-/ready\": dial tcp 10.217.1.166:9090: i/o timeout (Client.Timeout exceeded while awaiting headers)" Dec 03 11:57:26 crc kubenswrapper[5052]: I1203 11:57:26.185799 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/prometheus-metric-storage-0" Dec 03 11:57:26 crc kubenswrapper[5052]: I1203 11:57:26.302980 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"prometheus-metric-storage-db\" (UniqueName: \"kubernetes.io/local-volume/local-storage19-crc\") pod \"33918bc1-340a-4d26-a204-001b85c15f04\" (UID: \"33918bc1-340a-4d26-a204-001b85c15f04\") " Dec 03 11:57:26 crc kubenswrapper[5052]: I1203 11:57:26.303059 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9xpqc\" (UniqueName: \"kubernetes.io/projected/33918bc1-340a-4d26-a204-001b85c15f04-kube-api-access-9xpqc\") pod \"33918bc1-340a-4d26-a204-001b85c15f04\" (UID: \"33918bc1-340a-4d26-a204-001b85c15f04\") " Dec 03 11:57:26 crc kubenswrapper[5052]: I1203 11:57:26.303088 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"prometheus-metric-storage-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/33918bc1-340a-4d26-a204-001b85c15f04-prometheus-metric-storage-rulefiles-0\") pod \"33918bc1-340a-4d26-a204-001b85c15f04\" (UID: \"33918bc1-340a-4d26-a204-001b85c15f04\") " Dec 03 11:57:26 crc kubenswrapper[5052]: I1203 11:57:26.303130 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/33918bc1-340a-4d26-a204-001b85c15f04-web-config\") pod \"33918bc1-340a-4d26-a204-001b85c15f04\" (UID: \"33918bc1-340a-4d26-a204-001b85c15f04\") " Dec 03 11:57:26 crc kubenswrapper[5052]: I1203 11:57:26.303238 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/33918bc1-340a-4d26-a204-001b85c15f04-tls-assets\") pod \"33918bc1-340a-4d26-a204-001b85c15f04\" (UID: \"33918bc1-340a-4d26-a204-001b85c15f04\") " Dec 03 11:57:26 crc kubenswrapper[5052]: I1203 11:57:26.303265 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/33918bc1-340a-4d26-a204-001b85c15f04-config\") pod \"33918bc1-340a-4d26-a204-001b85c15f04\" (UID: \"33918bc1-340a-4d26-a204-001b85c15f04\") " Dec 03 11:57:26 crc kubenswrapper[5052]: I1203 11:57:26.303328 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/33918bc1-340a-4d26-a204-001b85c15f04-thanos-prometheus-http-client-file\") pod \"33918bc1-340a-4d26-a204-001b85c15f04\" (UID: \"33918bc1-340a-4d26-a204-001b85c15f04\") " Dec 03 11:57:26 crc kubenswrapper[5052]: I1203 11:57:26.303356 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/33918bc1-340a-4d26-a204-001b85c15f04-config-out\") pod \"33918bc1-340a-4d26-a204-001b85c15f04\" (UID: \"33918bc1-340a-4d26-a204-001b85c15f04\") " Dec 03 11:57:26 crc kubenswrapper[5052]: I1203 11:57:26.305458 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/33918bc1-340a-4d26-a204-001b85c15f04-prometheus-metric-storage-rulefiles-0" (OuterVolumeSpecName: "prometheus-metric-storage-rulefiles-0") pod "33918bc1-340a-4d26-a204-001b85c15f04" (UID: "33918bc1-340a-4d26-a204-001b85c15f04"). InnerVolumeSpecName "prometheus-metric-storage-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:57:26 crc kubenswrapper[5052]: I1203 11:57:26.312098 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33918bc1-340a-4d26-a204-001b85c15f04-config-out" (OuterVolumeSpecName: "config-out") pod "33918bc1-340a-4d26-a204-001b85c15f04" (UID: "33918bc1-340a-4d26-a204-001b85c15f04"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:57:26 crc kubenswrapper[5052]: I1203 11:57:26.312141 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/33918bc1-340a-4d26-a204-001b85c15f04-config" (OuterVolumeSpecName: "config") pod "33918bc1-340a-4d26-a204-001b85c15f04" (UID: "33918bc1-340a-4d26-a204-001b85c15f04"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:57:26 crc kubenswrapper[5052]: I1203 11:57:26.313335 5052 reconciler_common.go:293] "Volume detached for volume \"prometheus-metric-storage-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/33918bc1-340a-4d26-a204-001b85c15f04-prometheus-metric-storage-rulefiles-0\") on node \"crc\" DevicePath \"\"" Dec 03 11:57:26 crc kubenswrapper[5052]: I1203 11:57:26.313374 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/33918bc1-340a-4d26-a204-001b85c15f04-config\") on node \"crc\" DevicePath \"\"" Dec 03 11:57:26 crc kubenswrapper[5052]: I1203 11:57:26.313388 5052 reconciler_common.go:293] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/33918bc1-340a-4d26-a204-001b85c15f04-config-out\") on node \"crc\" DevicePath \"\"" Dec 03 11:57:26 crc kubenswrapper[5052]: I1203 11:57:26.314236 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/33918bc1-340a-4d26-a204-001b85c15f04-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "33918bc1-340a-4d26-a204-001b85c15f04" (UID: "33918bc1-340a-4d26-a204-001b85c15f04"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:57:26 crc kubenswrapper[5052]: I1203 11:57:26.315197 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage19-crc" (OuterVolumeSpecName: "prometheus-metric-storage-db") pod "33918bc1-340a-4d26-a204-001b85c15f04" (UID: "33918bc1-340a-4d26-a204-001b85c15f04"). InnerVolumeSpecName "local-storage19-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 11:57:26 crc kubenswrapper[5052]: I1203 11:57:26.315763 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/33918bc1-340a-4d26-a204-001b85c15f04-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "33918bc1-340a-4d26-a204-001b85c15f04" (UID: "33918bc1-340a-4d26-a204-001b85c15f04"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:57:26 crc kubenswrapper[5052]: I1203 11:57:26.320916 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/33918bc1-340a-4d26-a204-001b85c15f04-kube-api-access-9xpqc" (OuterVolumeSpecName: "kube-api-access-9xpqc") pod "33918bc1-340a-4d26-a204-001b85c15f04" (UID: "33918bc1-340a-4d26-a204-001b85c15f04"). InnerVolumeSpecName "kube-api-access-9xpqc". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:57:26 crc kubenswrapper[5052]: I1203 11:57:26.341069 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/33918bc1-340a-4d26-a204-001b85c15f04-web-config" (OuterVolumeSpecName: "web-config") pod "33918bc1-340a-4d26-a204-001b85c15f04" (UID: "33918bc1-340a-4d26-a204-001b85c15f04"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:57:26 crc kubenswrapper[5052]: I1203 11:57:26.416682 5052 reconciler_common.go:293] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/33918bc1-340a-4d26-a204-001b85c15f04-tls-assets\") on node \"crc\" DevicePath \"\"" Dec 03 11:57:26 crc kubenswrapper[5052]: I1203 11:57:26.417172 5052 reconciler_common.go:293] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/33918bc1-340a-4d26-a204-001b85c15f04-thanos-prometheus-http-client-file\") on node \"crc\" DevicePath \"\"" Dec 03 11:57:26 crc kubenswrapper[5052]: I1203 11:57:26.417211 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage19-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage19-crc\") on node \"crc\" " Dec 03 11:57:26 crc kubenswrapper[5052]: I1203 11:57:26.417226 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-9xpqc\" (UniqueName: \"kubernetes.io/projected/33918bc1-340a-4d26-a204-001b85c15f04-kube-api-access-9xpqc\") on node \"crc\" DevicePath \"\"" Dec 03 11:57:26 crc kubenswrapper[5052]: I1203 11:57:26.417239 5052 reconciler_common.go:293] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/33918bc1-340a-4d26-a204-001b85c15f04-web-config\") on node \"crc\" DevicePath \"\"" Dec 03 11:57:26 crc kubenswrapper[5052]: E1203 11:57:26.416913 5052 log.go:32] "PullImage from image service failed" err="rpc error: code = Canceled desc = copying config: context canceled" image="quay.io/podified-master-centos9/openstack-watcher-api@sha256:75f08e61323e7039d3cf5cdf81ac5d416caa4f7a0de7ce33992cfa2686f5080e" Dec 03 11:57:26 crc kubenswrapper[5052]: E1203 11:57:26.418334 5052 kuberuntime_manager.go:1274] "Unhandled Error" err="container &Container{Name:watcher-db-sync,Image:quay.io/podified-master-centos9/openstack-watcher-api@sha256:75f08e61323e7039d3cf5cdf81ac5d416caa4f7a0de7ce33992cfa2686f5080e,Command:[/bin/bash],Args:[-c /usr/local/bin/kolla_set_configs && /usr/local/bin/kolla_start],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{EnvVar{Name:KOLLA_BOOTSTRAP,Value:TRUE,ValueFrom:nil,},EnvVar{Name:KOLLA_CONFIG_STRATEGY,Value:COPY_ALWAYS,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:config-data,ReadOnly:true,MountPath:/var/lib/config-data/default,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:config-data,ReadOnly:true,MountPath:/etc/my.cnf,SubPath:my.cnf,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:db-sync-config-data,ReadOnly:true,MountPath:/etc/watcher/watcher.conf.d,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:config-data,ReadOnly:true,MountPath:/var/lib/kolla/config_files/config.json,SubPath:watcher-dbsync-config.json,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:combined-ca-bundle,ReadOnly:true,MountPath:/etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem,SubPath:tls-ca-bundle.pem,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-29fml,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:nil,ReadinessProbe:nil,Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[MKNOD],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*0,RunAsNonRoot:nil,ReadOnlyRootFilesystem:nil,AllowPrivilegeEscalation:nil,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod watcher-db-sync-chlm4_openstack-kuttl-tests(b3146921-cc78-401f-a918-fa8b91814f10): ErrImagePull: rpc error: code = Canceled desc = copying config: context canceled" logger="UnhandledError" Dec 03 11:57:26 crc kubenswrapper[5052]: E1203 11:57:26.421854 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"watcher-db-sync\" with ErrImagePull: \"rpc error: code = Canceled desc = copying config: context canceled\"" pod="openstack-kuttl-tests/watcher-db-sync-chlm4" podUID="b3146921-cc78-401f-a918-fa8b91814f10" Dec 03 11:57:26 crc kubenswrapper[5052]: I1203 11:57:26.465689 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage19-crc" (UniqueName: "kubernetes.io/local-volume/local-storage19-crc") on node "crc" Dec 03 11:57:26 crc kubenswrapper[5052]: I1203 11:57:26.520630 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage19-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage19-crc\") on node \"crc\" DevicePath \"\"" Dec 03 11:57:26 crc kubenswrapper[5052]: I1203 11:57:26.537987 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/prometheus-metric-storage-0" Dec 03 11:57:26 crc kubenswrapper[5052]: I1203 11:57:26.549670 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/prometheus-metric-storage-0" event={"ID":"33918bc1-340a-4d26-a204-001b85c15f04","Type":"ContainerDied","Data":"768873257caa2a7a191a492cee44124407a3c1d86b7d431237814573ef7b6713"} Dec 03 11:57:26 crc kubenswrapper[5052]: I1203 11:57:26.549773 5052 scope.go:117] "RemoveContainer" containerID="338f13d17f6b49a2ab7b8705345e66ade7dfcb2716d85396b3217b2553ae2ae2" Dec 03 11:57:26 crc kubenswrapper[5052]: E1203 11:57:26.550913 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"watcher-db-sync\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/podified-master-centos9/openstack-watcher-api@sha256:75f08e61323e7039d3cf5cdf81ac5d416caa4f7a0de7ce33992cfa2686f5080e\\\"\"" pod="openstack-kuttl-tests/watcher-db-sync-chlm4" podUID="b3146921-cc78-401f-a918-fa8b91814f10" Dec 03 11:57:26 crc kubenswrapper[5052]: I1203 11:57:26.597560 5052 scope.go:117] "RemoveContainer" containerID="4efdd771545fd00295d9e81c501f88a8052264f58cb92b45ecd0fedadf7e279c" Dec 03 11:57:26 crc kubenswrapper[5052]: I1203 11:57:26.615721 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/prometheus-metric-storage-0"] Dec 03 11:57:26 crc kubenswrapper[5052]: I1203 11:57:26.632569 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/prometheus-metric-storage-0"] Dec 03 11:57:26 crc kubenswrapper[5052]: I1203 11:57:26.643307 5052 scope.go:117] "RemoveContainer" containerID="a39079dcf889f7fbf32bc70b596f4c1f6eab751f6901536f1d0bda478008c680" Dec 03 11:57:26 crc kubenswrapper[5052]: I1203 11:57:26.669328 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/prometheus-metric-storage-0"] Dec 03 11:57:26 crc kubenswrapper[5052]: E1203 11:57:26.669882 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="33918bc1-340a-4d26-a204-001b85c15f04" containerName="config-reloader" Dec 03 11:57:26 crc kubenswrapper[5052]: I1203 11:57:26.669900 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="33918bc1-340a-4d26-a204-001b85c15f04" containerName="config-reloader" Dec 03 11:57:26 crc kubenswrapper[5052]: E1203 11:57:26.669925 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="33918bc1-340a-4d26-a204-001b85c15f04" containerName="prometheus" Dec 03 11:57:26 crc kubenswrapper[5052]: I1203 11:57:26.669933 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="33918bc1-340a-4d26-a204-001b85c15f04" containerName="prometheus" Dec 03 11:57:26 crc kubenswrapper[5052]: E1203 11:57:26.669959 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="33918bc1-340a-4d26-a204-001b85c15f04" containerName="thanos-sidecar" Dec 03 11:57:26 crc kubenswrapper[5052]: I1203 11:57:26.669966 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="33918bc1-340a-4d26-a204-001b85c15f04" containerName="thanos-sidecar" Dec 03 11:57:26 crc kubenswrapper[5052]: E1203 11:57:26.669976 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="33918bc1-340a-4d26-a204-001b85c15f04" containerName="init-config-reloader" Dec 03 11:57:26 crc kubenswrapper[5052]: I1203 11:57:26.669984 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="33918bc1-340a-4d26-a204-001b85c15f04" containerName="init-config-reloader" Dec 03 11:57:26 crc kubenswrapper[5052]: I1203 11:57:26.670194 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="33918bc1-340a-4d26-a204-001b85c15f04" containerName="config-reloader" Dec 03 11:57:26 crc kubenswrapper[5052]: I1203 11:57:26.670221 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="33918bc1-340a-4d26-a204-001b85c15f04" containerName="thanos-sidecar" Dec 03 11:57:26 crc kubenswrapper[5052]: I1203 11:57:26.670241 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="33918bc1-340a-4d26-a204-001b85c15f04" containerName="prometheus" Dec 03 11:57:26 crc kubenswrapper[5052]: I1203 11:57:26.674172 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/prometheus-metric-storage-0" Dec 03 11:57:26 crc kubenswrapper[5052]: I1203 11:57:26.678243 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"prometheus-metric-storage-web-config" Dec 03 11:57:26 crc kubenswrapper[5052]: I1203 11:57:26.682452 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"prometheus-metric-storage" Dec 03 11:57:26 crc kubenswrapper[5052]: I1203 11:57:26.686097 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"prometheus-metric-storage-thanos-prometheus-http-client-file" Dec 03 11:57:26 crc kubenswrapper[5052]: I1203 11:57:26.689989 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/prometheus-metric-storage-0"] Dec 03 11:57:26 crc kubenswrapper[5052]: I1203 11:57:26.690701 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"prometheus-metric-storage-rulefiles-0" Dec 03 11:57:26 crc kubenswrapper[5052]: I1203 11:57:26.690911 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-metric-storage-prometheus-svc" Dec 03 11:57:26 crc kubenswrapper[5052]: I1203 11:57:26.691111 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"metric-storage-prometheus-dockercfg-9948m" Dec 03 11:57:26 crc kubenswrapper[5052]: I1203 11:57:26.699456 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"prometheus-metric-storage-tls-assets-0" Dec 03 11:57:26 crc kubenswrapper[5052]: I1203 11:57:26.718975 5052 scope.go:117] "RemoveContainer" containerID="414b563368af8ca87e25bd40b323d2ab1710df0f803632e3f17e886ce6740020" Dec 03 11:57:26 crc kubenswrapper[5052]: I1203 11:57:26.774781 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-db-sync-xvd7n"] Dec 03 11:57:26 crc kubenswrapper[5052]: I1203 11:57:26.826921 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/01bc82de-61dc-477e-b4cf-e3b0d32c99ba-config\") pod \"prometheus-metric-storage-0\" (UID: \"01bc82de-61dc-477e-b4cf-e3b0d32c99ba\") " pod="openstack-kuttl-tests/prometheus-metric-storage-0" Dec 03 11:57:26 crc kubenswrapper[5052]: I1203 11:57:26.827020 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/01bc82de-61dc-477e-b4cf-e3b0d32c99ba-tls-assets\") pod \"prometheus-metric-storage-0\" (UID: \"01bc82de-61dc-477e-b4cf-e3b0d32c99ba\") " pod="openstack-kuttl-tests/prometheus-metric-storage-0" Dec 03 11:57:26 crc kubenswrapper[5052]: I1203 11:57:26.827051 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/01bc82de-61dc-477e-b4cf-e3b0d32c99ba-thanos-prometheus-http-client-file\") pod \"prometheus-metric-storage-0\" (UID: \"01bc82de-61dc-477e-b4cf-e3b0d32c99ba\") " pod="openstack-kuttl-tests/prometheus-metric-storage-0" Dec 03 11:57:26 crc kubenswrapper[5052]: I1203 11:57:26.827125 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config-tls-secret-key-cert-metric-storage-promethe-dc638c2d\" (UniqueName: \"kubernetes.io/secret/01bc82de-61dc-477e-b4cf-e3b0d32c99ba-web-config-tls-secret-key-cert-metric-storage-promethe-dc638c2d\") pod \"prometheus-metric-storage-0\" (UID: \"01bc82de-61dc-477e-b4cf-e3b0d32c99ba\") " pod="openstack-kuttl-tests/prometheus-metric-storage-0" Dec 03 11:57:26 crc kubenswrapper[5052]: I1203 11:57:26.827163 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-metric-storage-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/01bc82de-61dc-477e-b4cf-e3b0d32c99ba-prometheus-metric-storage-rulefiles-0\") pod \"prometheus-metric-storage-0\" (UID: \"01bc82de-61dc-477e-b4cf-e3b0d32c99ba\") " pod="openstack-kuttl-tests/prometheus-metric-storage-0" Dec 03 11:57:26 crc kubenswrapper[5052]: I1203 11:57:26.827312 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/01bc82de-61dc-477e-b4cf-e3b0d32c99ba-secret-combined-ca-bundle\") pod \"prometheus-metric-storage-0\" (UID: \"01bc82de-61dc-477e-b4cf-e3b0d32c99ba\") " pod="openstack-kuttl-tests/prometheus-metric-storage-0" Dec 03 11:57:26 crc kubenswrapper[5052]: I1203 11:57:26.827358 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/01bc82de-61dc-477e-b4cf-e3b0d32c99ba-config-out\") pod \"prometheus-metric-storage-0\" (UID: \"01bc82de-61dc-477e-b4cf-e3b0d32c99ba\") " pod="openstack-kuttl-tests/prometheus-metric-storage-0" Dec 03 11:57:26 crc kubenswrapper[5052]: I1203 11:57:26.827434 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/01bc82de-61dc-477e-b4cf-e3b0d32c99ba-web-config\") pod \"prometheus-metric-storage-0\" (UID: \"01bc82de-61dc-477e-b4cf-e3b0d32c99ba\") " pod="openstack-kuttl-tests/prometheus-metric-storage-0" Dec 03 11:57:26 crc kubenswrapper[5052]: I1203 11:57:26.827470 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kj8bk\" (UniqueName: \"kubernetes.io/projected/01bc82de-61dc-477e-b4cf-e3b0d32c99ba-kube-api-access-kj8bk\") pod \"prometheus-metric-storage-0\" (UID: \"01bc82de-61dc-477e-b4cf-e3b0d32c99ba\") " pod="openstack-kuttl-tests/prometheus-metric-storage-0" Dec 03 11:57:26 crc kubenswrapper[5052]: I1203 11:57:26.827501 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage19-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage19-crc\") pod \"prometheus-metric-storage-0\" (UID: \"01bc82de-61dc-477e-b4cf-e3b0d32c99ba\") " pod="openstack-kuttl-tests/prometheus-metric-storage-0" Dec 03 11:57:26 crc kubenswrapper[5052]: I1203 11:57:26.827790 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config-tls-secret-cert-cert-metric-storage-prometh-dc638c2d\" (UniqueName: \"kubernetes.io/secret/01bc82de-61dc-477e-b4cf-e3b0d32c99ba-web-config-tls-secret-cert-cert-metric-storage-prometh-dc638c2d\") pod \"prometheus-metric-storage-0\" (UID: \"01bc82de-61dc-477e-b4cf-e3b0d32c99ba\") " pod="openstack-kuttl-tests/prometheus-metric-storage-0" Dec 03 11:57:26 crc kubenswrapper[5052]: I1203 11:57:26.929300 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"secret-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/01bc82de-61dc-477e-b4cf-e3b0d32c99ba-secret-combined-ca-bundle\") pod \"prometheus-metric-storage-0\" (UID: \"01bc82de-61dc-477e-b4cf-e3b0d32c99ba\") " pod="openstack-kuttl-tests/prometheus-metric-storage-0" Dec 03 11:57:26 crc kubenswrapper[5052]: I1203 11:57:26.930367 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/01bc82de-61dc-477e-b4cf-e3b0d32c99ba-config-out\") pod \"prometheus-metric-storage-0\" (UID: \"01bc82de-61dc-477e-b4cf-e3b0d32c99ba\") " pod="openstack-kuttl-tests/prometheus-metric-storage-0" Dec 03 11:57:26 crc kubenswrapper[5052]: I1203 11:57:26.930424 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/01bc82de-61dc-477e-b4cf-e3b0d32c99ba-web-config\") pod \"prometheus-metric-storage-0\" (UID: \"01bc82de-61dc-477e-b4cf-e3b0d32c99ba\") " pod="openstack-kuttl-tests/prometheus-metric-storage-0" Dec 03 11:57:26 crc kubenswrapper[5052]: I1203 11:57:26.930442 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-kj8bk\" (UniqueName: \"kubernetes.io/projected/01bc82de-61dc-477e-b4cf-e3b0d32c99ba-kube-api-access-kj8bk\") pod \"prometheus-metric-storage-0\" (UID: \"01bc82de-61dc-477e-b4cf-e3b0d32c99ba\") " pod="openstack-kuttl-tests/prometheus-metric-storage-0" Dec 03 11:57:26 crc kubenswrapper[5052]: I1203 11:57:26.930521 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage19-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage19-crc\") pod \"prometheus-metric-storage-0\" (UID: \"01bc82de-61dc-477e-b4cf-e3b0d32c99ba\") " pod="openstack-kuttl-tests/prometheus-metric-storage-0" Dec 03 11:57:26 crc kubenswrapper[5052]: I1203 11:57:26.930556 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"web-config-tls-secret-cert-cert-metric-storage-prometh-dc638c2d\" (UniqueName: \"kubernetes.io/secret/01bc82de-61dc-477e-b4cf-e3b0d32c99ba-web-config-tls-secret-cert-cert-metric-storage-prometh-dc638c2d\") pod \"prometheus-metric-storage-0\" (UID: \"01bc82de-61dc-477e-b4cf-e3b0d32c99ba\") " pod="openstack-kuttl-tests/prometheus-metric-storage-0" Dec 03 11:57:26 crc kubenswrapper[5052]: I1203 11:57:26.930583 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/01bc82de-61dc-477e-b4cf-e3b0d32c99ba-config\") pod \"prometheus-metric-storage-0\" (UID: \"01bc82de-61dc-477e-b4cf-e3b0d32c99ba\") " pod="openstack-kuttl-tests/prometheus-metric-storage-0" Dec 03 11:57:26 crc kubenswrapper[5052]: I1203 11:57:26.930659 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/01bc82de-61dc-477e-b4cf-e3b0d32c99ba-tls-assets\") pod \"prometheus-metric-storage-0\" (UID: \"01bc82de-61dc-477e-b4cf-e3b0d32c99ba\") " pod="openstack-kuttl-tests/prometheus-metric-storage-0" Dec 03 11:57:26 crc kubenswrapper[5052]: I1203 11:57:26.930684 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/01bc82de-61dc-477e-b4cf-e3b0d32c99ba-thanos-prometheus-http-client-file\") pod \"prometheus-metric-storage-0\" (UID: \"01bc82de-61dc-477e-b4cf-e3b0d32c99ba\") " pod="openstack-kuttl-tests/prometheus-metric-storage-0" Dec 03 11:57:26 crc kubenswrapper[5052]: I1203 11:57:26.930719 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"web-config-tls-secret-key-cert-metric-storage-promethe-dc638c2d\" (UniqueName: \"kubernetes.io/secret/01bc82de-61dc-477e-b4cf-e3b0d32c99ba-web-config-tls-secret-key-cert-metric-storage-promethe-dc638c2d\") pod \"prometheus-metric-storage-0\" (UID: \"01bc82de-61dc-477e-b4cf-e3b0d32c99ba\") " pod="openstack-kuttl-tests/prometheus-metric-storage-0" Dec 03 11:57:26 crc kubenswrapper[5052]: I1203 11:57:26.930746 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"prometheus-metric-storage-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/01bc82de-61dc-477e-b4cf-e3b0d32c99ba-prometheus-metric-storage-rulefiles-0\") pod \"prometheus-metric-storage-0\" (UID: \"01bc82de-61dc-477e-b4cf-e3b0d32c99ba\") " pod="openstack-kuttl-tests/prometheus-metric-storage-0" Dec 03 11:57:26 crc kubenswrapper[5052]: I1203 11:57:26.931404 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"prometheus-metric-storage-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/01bc82de-61dc-477e-b4cf-e3b0d32c99ba-prometheus-metric-storage-rulefiles-0\") pod \"prometheus-metric-storage-0\" (UID: \"01bc82de-61dc-477e-b4cf-e3b0d32c99ba\") " pod="openstack-kuttl-tests/prometheus-metric-storage-0" Dec 03 11:57:26 crc kubenswrapper[5052]: I1203 11:57:26.938758 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/01bc82de-61dc-477e-b4cf-e3b0d32c99ba-tls-assets\") pod \"prometheus-metric-storage-0\" (UID: \"01bc82de-61dc-477e-b4cf-e3b0d32c99ba\") " pod="openstack-kuttl-tests/prometheus-metric-storage-0" Dec 03 11:57:26 crc kubenswrapper[5052]: I1203 11:57:26.938748 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"web-config-tls-secret-key-cert-metric-storage-promethe-dc638c2d\" (UniqueName: \"kubernetes.io/secret/01bc82de-61dc-477e-b4cf-e3b0d32c99ba-web-config-tls-secret-key-cert-metric-storage-promethe-dc638c2d\") pod \"prometheus-metric-storage-0\" (UID: \"01bc82de-61dc-477e-b4cf-e3b0d32c99ba\") " pod="openstack-kuttl-tests/prometheus-metric-storage-0" Dec 03 11:57:26 crc kubenswrapper[5052]: I1203 11:57:26.938870 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage19-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage19-crc\") pod \"prometheus-metric-storage-0\" (UID: \"01bc82de-61dc-477e-b4cf-e3b0d32c99ba\") device mount path \"/mnt/openstack/pv19\"" pod="openstack-kuttl-tests/prometheus-metric-storage-0" Dec 03 11:57:26 crc kubenswrapper[5052]: I1203 11:57:26.940216 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"web-config-tls-secret-cert-cert-metric-storage-prometh-dc638c2d\" (UniqueName: \"kubernetes.io/secret/01bc82de-61dc-477e-b4cf-e3b0d32c99ba-web-config-tls-secret-cert-cert-metric-storage-prometh-dc638c2d\") pod \"prometheus-metric-storage-0\" (UID: \"01bc82de-61dc-477e-b4cf-e3b0d32c99ba\") " pod="openstack-kuttl-tests/prometheus-metric-storage-0" Dec 03 11:57:26 crc kubenswrapper[5052]: I1203 11:57:26.948359 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/01bc82de-61dc-477e-b4cf-e3b0d32c99ba-thanos-prometheus-http-client-file\") pod \"prometheus-metric-storage-0\" (UID: \"01bc82de-61dc-477e-b4cf-e3b0d32c99ba\") " pod="openstack-kuttl-tests/prometheus-metric-storage-0" Dec 03 11:57:26 crc kubenswrapper[5052]: I1203 11:57:26.949336 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/01bc82de-61dc-477e-b4cf-e3b0d32c99ba-config\") pod \"prometheus-metric-storage-0\" (UID: \"01bc82de-61dc-477e-b4cf-e3b0d32c99ba\") " pod="openstack-kuttl-tests/prometheus-metric-storage-0" Dec 03 11:57:26 crc kubenswrapper[5052]: I1203 11:57:26.950195 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"secret-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/01bc82de-61dc-477e-b4cf-e3b0d32c99ba-secret-combined-ca-bundle\") pod \"prometheus-metric-storage-0\" (UID: \"01bc82de-61dc-477e-b4cf-e3b0d32c99ba\") " pod="openstack-kuttl-tests/prometheus-metric-storage-0" Dec 03 11:57:26 crc kubenswrapper[5052]: I1203 11:57:26.950733 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/01bc82de-61dc-477e-b4cf-e3b0d32c99ba-web-config\") pod \"prometheus-metric-storage-0\" (UID: \"01bc82de-61dc-477e-b4cf-e3b0d32c99ba\") " pod="openstack-kuttl-tests/prometheus-metric-storage-0" Dec 03 11:57:26 crc kubenswrapper[5052]: I1203 11:57:26.971061 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/01bc82de-61dc-477e-b4cf-e3b0d32c99ba-config-out\") pod \"prometheus-metric-storage-0\" (UID: \"01bc82de-61dc-477e-b4cf-e3b0d32c99ba\") " pod="openstack-kuttl-tests/prometheus-metric-storage-0" Dec 03 11:57:26 crc kubenswrapper[5052]: I1203 11:57:26.971391 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-kj8bk\" (UniqueName: \"kubernetes.io/projected/01bc82de-61dc-477e-b4cf-e3b0d32c99ba-kube-api-access-kj8bk\") pod \"prometheus-metric-storage-0\" (UID: \"01bc82de-61dc-477e-b4cf-e3b0d32c99ba\") " pod="openstack-kuttl-tests/prometheus-metric-storage-0" Dec 03 11:57:26 crc kubenswrapper[5052]: I1203 11:57:26.986950 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage19-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage19-crc\") pod \"prometheus-metric-storage-0\" (UID: \"01bc82de-61dc-477e-b4cf-e3b0d32c99ba\") " pod="openstack-kuttl-tests/prometheus-metric-storage-0" Dec 03 11:57:27 crc kubenswrapper[5052]: W1203 11:57:27.010314 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb70c4b01_42e4_4990_be48_a2d762e66cd7.slice/crio-4cc5bcf5e3eae342e3769376a8d544310344b7c571bd4994114a57bc6c9f2fcc WatchSource:0}: Error finding container 4cc5bcf5e3eae342e3769376a8d544310344b7c571bd4994114a57bc6c9f2fcc: Status 404 returned error can't find the container with id 4cc5bcf5e3eae342e3769376a8d544310344b7c571bd4994114a57bc6c9f2fcc Dec 03 11:57:27 crc kubenswrapper[5052]: I1203 11:57:27.017099 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-bootstrap-kdr6v"] Dec 03 11:57:27 crc kubenswrapper[5052]: I1203 11:57:27.017871 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/prometheus-metric-storage-0" Dec 03 11:57:27 crc kubenswrapper[5052]: W1203 11:57:27.018528 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbac97bd6_a939_41e2_b577_05d6fb87dcb5.slice/crio-82e14a469f5c2e516506acdec3d3743f7e118069caea0f770e4955fe7d6047ad WatchSource:0}: Error finding container 82e14a469f5c2e516506acdec3d3743f7e118069caea0f770e4955fe7d6047ad: Status 404 returned error can't find the container with id 82e14a469f5c2e516506acdec3d3743f7e118069caea0f770e4955fe7d6047ad Dec 03 11:57:27 crc kubenswrapper[5052]: I1203 11:57:27.027421 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/placement-db-sync-8tdfb"] Dec 03 11:57:27 crc kubenswrapper[5052]: I1203 11:57:27.035693 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:57:27 crc kubenswrapper[5052]: I1203 11:57:27.207006 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-db-sync-xs2lk"] Dec 03 11:57:27 crc kubenswrapper[5052]: I1203 11:57:27.282752 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 11:57:27 crc kubenswrapper[5052]: W1203 11:57:27.294831 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbf275c0c_27e5_462b_883e_9483cfe91bcd.slice/crio-131bdbef6449ae8940b0ed097e6d0a58c61746393c4bbc7dc14f1b111e3e4451 WatchSource:0}: Error finding container 131bdbef6449ae8940b0ed097e6d0a58c61746393c4bbc7dc14f1b111e3e4451: Status 404 returned error can't find the container with id 131bdbef6449ae8940b0ed097e6d0a58c61746393c4bbc7dc14f1b111e3e4451 Dec 03 11:57:27 crc kubenswrapper[5052]: I1203 11:57:27.304792 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 11:57:27 crc kubenswrapper[5052]: I1203 11:57:27.312033 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-db-sync-76kbf"] Dec 03 11:57:27 crc kubenswrapper[5052]: I1203 11:57:27.348285 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/prometheus-metric-storage-0"] Dec 03 11:57:27 crc kubenswrapper[5052]: W1203 11:57:27.351304 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod16630eec_cd03_42b8_95aa_d9a960c2858c.slice/crio-eaaf88ceaaee19b616e26866996ae2dadbb489e5568807f6dc1d1492084fb00e WatchSource:0}: Error finding container eaaf88ceaaee19b616e26866996ae2dadbb489e5568807f6dc1d1492084fb00e: Status 404 returned error can't find the container with id eaaf88ceaaee19b616e26866996ae2dadbb489e5568807f6dc1d1492084fb00e Dec 03 11:57:27 crc kubenswrapper[5052]: W1203 11:57:27.384020 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod01bc82de_61dc_477e_b4cf_e3b0d32c99ba.slice/crio-49c771312739dd2b8423503d8d97fb0470cb22b362440cae06abd17cdefa0629 WatchSource:0}: Error finding container 49c771312739dd2b8423503d8d97fb0470cb22b362440cae06abd17cdefa0629: Status 404 returned error can't find the container with id 49c771312739dd2b8423503d8d97fb0470cb22b362440cae06abd17cdefa0629 Dec 03 11:57:27 crc kubenswrapper[5052]: I1203 11:57:27.548493 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"bac97bd6-a939-41e2-b577-05d6fb87dcb5","Type":"ContainerStarted","Data":"82e14a469f5c2e516506acdec3d3743f7e118069caea0f770e4955fe7d6047ad"} Dec 03 11:57:27 crc kubenswrapper[5052]: I1203 11:57:27.551808 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-db-sync-xvd7n" event={"ID":"28c32800-797f-4560-ac08-55c9ca254a79","Type":"ContainerStarted","Data":"efba1971cde8e2d60c21027b0213c476cc57031f7a94af481671b4c6bb25e4fe"} Dec 03 11:57:27 crc kubenswrapper[5052]: I1203 11:57:27.551841 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-db-sync-xvd7n" event={"ID":"28c32800-797f-4560-ac08-55c9ca254a79","Type":"ContainerStarted","Data":"6a9c59a33f09852cb3d612699609c4e996fb88af55a03d65e3743c6765b8cd7b"} Dec 03 11:57:27 crc kubenswrapper[5052]: I1203 11:57:27.553544 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-db-sync-76kbf" event={"ID":"16630eec-cd03-42b8-95aa-d9a960c2858c","Type":"ContainerStarted","Data":"eaaf88ceaaee19b616e26866996ae2dadbb489e5568807f6dc1d1492084fb00e"} Dec 03 11:57:27 crc kubenswrapper[5052]: I1203 11:57:27.555856 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-bootstrap-kdr6v" event={"ID":"b70c4b01-42e4-4990-be48-a2d762e66cd7","Type":"ContainerStarted","Data":"4cc5bcf5e3eae342e3769376a8d544310344b7c571bd4994114a57bc6c9f2fcc"} Dec 03 11:57:27 crc kubenswrapper[5052]: I1203 11:57:27.563331 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"bf275c0c-27e5-462b-883e-9483cfe91bcd","Type":"ContainerStarted","Data":"131bdbef6449ae8940b0ed097e6d0a58c61746393c4bbc7dc14f1b111e3e4451"} Dec 03 11:57:27 crc kubenswrapper[5052]: I1203 11:57:27.568986 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/neutron-db-sync-xvd7n" podStartSLOduration=9.568971258 podStartE2EDuration="9.568971258s" podCreationTimestamp="2025-12-03 11:57:18 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:57:27.568460843 +0000 UTC m=+2238.156702607" watchObservedRunningTime="2025-12-03 11:57:27.568971258 +0000 UTC m=+2238.157213022" Dec 03 11:57:27 crc kubenswrapper[5052]: I1203 11:57:27.576048 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-db-sync-8tdfb" event={"ID":"14cb8e46-418d-4c42-ae97-a384a05b432f","Type":"ContainerStarted","Data":"52ea8ee3cb12bfc11fc6b5e5c5cc9c2498e7e2589c7072d86a8f6b9fdb81c4a1"} Dec 03 11:57:27 crc kubenswrapper[5052]: I1203 11:57:27.596463 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"32c3a6f2-948d-47a6-ad0f-2784e2623e8c","Type":"ContainerStarted","Data":"aff493502bb952fcc13fba9e95e99a0605bebb7c164576dbf24992e81bd483b9"} Dec 03 11:57:27 crc kubenswrapper[5052]: I1203 11:57:27.599808 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/prometheus-metric-storage-0" event={"ID":"01bc82de-61dc-477e-b4cf-e3b0d32c99ba","Type":"ContainerStarted","Data":"49c771312739dd2b8423503d8d97fb0470cb22b362440cae06abd17cdefa0629"} Dec 03 11:57:27 crc kubenswrapper[5052]: I1203 11:57:27.601724 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-db-sync-xs2lk" event={"ID":"a50cd1a3-6eb7-4712-bb60-648250b396d6","Type":"ContainerStarted","Data":"37a42c9d3db76f0dfccd0a96b80c798307fcfb938d7f490f56ab9c77c423a50f"} Dec 03 11:57:27 crc kubenswrapper[5052]: I1203 11:57:27.601776 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-db-sync-xs2lk" event={"ID":"a50cd1a3-6eb7-4712-bb60-648250b396d6","Type":"ContainerStarted","Data":"584317d8f9b53a44d8099c03595a73e25695880f52f568ed574db59b7a5db678"} Dec 03 11:57:27 crc kubenswrapper[5052]: I1203 11:57:27.692749 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="33918bc1-340a-4d26-a204-001b85c15f04" path="/var/lib/kubelet/pods/33918bc1-340a-4d26-a204-001b85c15f04/volumes" Dec 03 11:57:28 crc kubenswrapper[5052]: I1203 11:57:28.622035 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"bf275c0c-27e5-462b-883e-9483cfe91bcd","Type":"ContainerStarted","Data":"bf55314ea998abd4f91d7fde30baf699d2e4eb5a8a6691ce81452f4e56a42948"} Dec 03 11:57:28 crc kubenswrapper[5052]: I1203 11:57:28.627022 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-db-sync-8tdfb" event={"ID":"14cb8e46-418d-4c42-ae97-a384a05b432f","Type":"ContainerStarted","Data":"56106c72b8ee52c2322893ce9cab31cbca3baecdb6d70605f34d8b51f2c8f37b"} Dec 03 11:57:28 crc kubenswrapper[5052]: I1203 11:57:28.631551 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"32c3a6f2-948d-47a6-ad0f-2784e2623e8c","Type":"ContainerStarted","Data":"6d3253a75ecb252b680cf02b7ecf60d0b0bc66b2f0bd8ea511d43639a9734fb7"} Dec 03 11:57:28 crc kubenswrapper[5052]: I1203 11:57:28.634410 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-db-sync-76kbf" event={"ID":"16630eec-cd03-42b8-95aa-d9a960c2858c","Type":"ContainerStarted","Data":"9b3146eaddf7ac59c648a8a038db1a5905188e2a8b2f4dc2de4f7459b0197c3c"} Dec 03 11:57:28 crc kubenswrapper[5052]: I1203 11:57:28.641094 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-bootstrap-kdr6v" event={"ID":"b70c4b01-42e4-4990-be48-a2d762e66cd7","Type":"ContainerStarted","Data":"3e630cf9c031c51d9e57dbf5f4f68a125ad4930a9ae93db53fda22a443e086f2"} Dec 03 11:57:28 crc kubenswrapper[5052]: I1203 11:57:28.664651 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/placement-db-sync-8tdfb" podStartSLOduration=10.664624674 podStartE2EDuration="10.664624674s" podCreationTimestamp="2025-12-03 11:57:18 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:57:28.652285131 +0000 UTC m=+2239.240526895" watchObservedRunningTime="2025-12-03 11:57:28.664624674 +0000 UTC m=+2239.252866438" Dec 03 11:57:28 crc kubenswrapper[5052]: I1203 11:57:28.671339 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/keystone-bootstrap-kdr6v" podStartSLOduration=10.671321756 podStartE2EDuration="10.671321756s" podCreationTimestamp="2025-12-03 11:57:18 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:57:28.668017592 +0000 UTC m=+2239.256259356" watchObservedRunningTime="2025-12-03 11:57:28.671321756 +0000 UTC m=+2239.259563520" Dec 03 11:57:28 crc kubenswrapper[5052]: I1203 11:57:28.684536 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/cinder-db-sync-76kbf" podStartSLOduration=10.684521844 podStartE2EDuration="10.684521844s" podCreationTimestamp="2025-12-03 11:57:18 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:57:28.683293429 +0000 UTC m=+2239.271535193" watchObservedRunningTime="2025-12-03 11:57:28.684521844 +0000 UTC m=+2239.272763608" Dec 03 11:57:28 crc kubenswrapper[5052]: I1203 11:57:28.717756 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/barbican-db-sync-xs2lk" podStartSLOduration=10.717738204 podStartE2EDuration="10.717738204s" podCreationTimestamp="2025-12-03 11:57:18 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:57:28.707312926 +0000 UTC m=+2239.295554690" watchObservedRunningTime="2025-12-03 11:57:28.717738204 +0000 UTC m=+2239.305979968" Dec 03 11:57:29 crc kubenswrapper[5052]: I1203 11:57:29.379765 5052 patch_prober.go:28] interesting pod/machine-config-daemon-74bj6 container/machine-config-daemon namespace/openshift-machine-config-operator: Liveness probe status=failure output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" start-of-body= Dec 03 11:57:29 crc kubenswrapper[5052]: I1203 11:57:29.380070 5052 prober.go:107] "Probe failed" probeType="Liveness" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" containerName="machine-config-daemon" probeResult="failure" output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" Dec 03 11:57:29 crc kubenswrapper[5052]: I1203 11:57:29.380113 5052 kubelet.go:2542] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" Dec 03 11:57:29 crc kubenswrapper[5052]: I1203 11:57:29.380820 5052 kuberuntime_manager.go:1027] "Message for Container of pod" containerName="machine-config-daemon" containerStatusID={"Type":"cri-o","ID":"d224717c6d9ca8135fd510e03a6c40c9f2b1a56dd8b4a3b685af3e2dfa855982"} pod="openshift-machine-config-operator/machine-config-daemon-74bj6" containerMessage="Container machine-config-daemon failed liveness probe, will be restarted" Dec 03 11:57:29 crc kubenswrapper[5052]: I1203 11:57:29.380872 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" containerName="machine-config-daemon" containerID="cri-o://d224717c6d9ca8135fd510e03a6c40c9f2b1a56dd8b4a3b685af3e2dfa855982" gracePeriod=600 Dec 03 11:57:29 crc kubenswrapper[5052]: I1203 11:57:29.651329 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"bac97bd6-a939-41e2-b577-05d6fb87dcb5","Type":"ContainerStarted","Data":"eb431f6c1818b9852af36eea4a58126f109d053175b87db72d750e6bc3098387"} Dec 03 11:57:29 crc kubenswrapper[5052]: I1203 11:57:29.654352 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"32c3a6f2-948d-47a6-ad0f-2784e2623e8c","Type":"ContainerStarted","Data":"6e7a121aafc72c053dc5ac79b3667dcfcefa4aa263daac5c76499dc9cf210cf7"} Dec 03 11:57:29 crc kubenswrapper[5052]: I1203 11:57:29.656899 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"bf275c0c-27e5-462b-883e-9483cfe91bcd","Type":"ContainerStarted","Data":"8a2eec599442bf1bc410ee4b4d585262201c84834e013c4700b3283ebadb883c"} Dec 03 11:57:30 crc kubenswrapper[5052]: I1203 11:57:30.668935 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/prometheus-metric-storage-0" event={"ID":"01bc82de-61dc-477e-b4cf-e3b0d32c99ba","Type":"ContainerStarted","Data":"be71930393f0fbaabf4c9dd57b05ba54c75c478e04e607ad33f67424c249c0f3"} Dec 03 11:57:30 crc kubenswrapper[5052]: I1203 11:57:30.672137 5052 generic.go:334] "Generic (PLEG): container finished" podID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" containerID="d224717c6d9ca8135fd510e03a6c40c9f2b1a56dd8b4a3b685af3e2dfa855982" exitCode=0 Dec 03 11:57:30 crc kubenswrapper[5052]: I1203 11:57:30.672219 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" event={"ID":"10c237bc-72e7-4da3-9f2c-b1688395bf6e","Type":"ContainerDied","Data":"d224717c6d9ca8135fd510e03a6c40c9f2b1a56dd8b4a3b685af3e2dfa855982"} Dec 03 11:57:30 crc kubenswrapper[5052]: I1203 11:57:30.672259 5052 scope.go:117] "RemoveContainer" containerID="cc4f296896bbf8aa4d2c78b7e4045d3222cac9b2a6ed232c44ec9d7458bb3338" Dec 03 11:57:30 crc kubenswrapper[5052]: I1203 11:57:30.672277 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-external-api-0" podUID="bf275c0c-27e5-462b-883e-9483cfe91bcd" containerName="glance-log" containerID="cri-o://bf55314ea998abd4f91d7fde30baf699d2e4eb5a8a6691ce81452f4e56a42948" gracePeriod=30 Dec 03 11:57:30 crc kubenswrapper[5052]: I1203 11:57:30.672342 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-external-api-0" podUID="bf275c0c-27e5-462b-883e-9483cfe91bcd" containerName="glance-httpd" containerID="cri-o://8a2eec599442bf1bc410ee4b4d585262201c84834e013c4700b3283ebadb883c" gracePeriod=30 Dec 03 11:57:30 crc kubenswrapper[5052]: I1203 11:57:30.672584 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-internal-api-0" podUID="32c3a6f2-948d-47a6-ad0f-2784e2623e8c" containerName="glance-log" containerID="cri-o://6d3253a75ecb252b680cf02b7ecf60d0b0bc66b2f0bd8ea511d43639a9734fb7" gracePeriod=30 Dec 03 11:57:30 crc kubenswrapper[5052]: I1203 11:57:30.672630 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-internal-api-0" podUID="32c3a6f2-948d-47a6-ad0f-2784e2623e8c" containerName="glance-httpd" containerID="cri-o://6e7a121aafc72c053dc5ac79b3667dcfcefa4aa263daac5c76499dc9cf210cf7" gracePeriod=30 Dec 03 11:57:30 crc kubenswrapper[5052]: I1203 11:57:30.703676 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/glance-default-external-api-0" podStartSLOduration=12.703651468 podStartE2EDuration="12.703651468s" podCreationTimestamp="2025-12-03 11:57:18 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:57:30.690334227 +0000 UTC m=+2241.278575991" watchObservedRunningTime="2025-12-03 11:57:30.703651468 +0000 UTC m=+2241.291893242" Dec 03 11:57:30 crc kubenswrapper[5052]: E1203 11:57:30.714446 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 11:57:31 crc kubenswrapper[5052]: I1203 11:57:31.686751 5052 generic.go:334] "Generic (PLEG): container finished" podID="bf275c0c-27e5-462b-883e-9483cfe91bcd" containerID="8a2eec599442bf1bc410ee4b4d585262201c84834e013c4700b3283ebadb883c" exitCode=0 Dec 03 11:57:31 crc kubenswrapper[5052]: I1203 11:57:31.686990 5052 generic.go:334] "Generic (PLEG): container finished" podID="bf275c0c-27e5-462b-883e-9483cfe91bcd" containerID="bf55314ea998abd4f91d7fde30baf699d2e4eb5a8a6691ce81452f4e56a42948" exitCode=143 Dec 03 11:57:31 crc kubenswrapper[5052]: I1203 11:57:31.686944 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"bf275c0c-27e5-462b-883e-9483cfe91bcd","Type":"ContainerDied","Data":"8a2eec599442bf1bc410ee4b4d585262201c84834e013c4700b3283ebadb883c"} Dec 03 11:57:31 crc kubenswrapper[5052]: I1203 11:57:31.687061 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"bf275c0c-27e5-462b-883e-9483cfe91bcd","Type":"ContainerDied","Data":"bf55314ea998abd4f91d7fde30baf699d2e4eb5a8a6691ce81452f4e56a42948"} Dec 03 11:57:31 crc kubenswrapper[5052]: I1203 11:57:31.689665 5052 generic.go:334] "Generic (PLEG): container finished" podID="32c3a6f2-948d-47a6-ad0f-2784e2623e8c" containerID="6e7a121aafc72c053dc5ac79b3667dcfcefa4aa263daac5c76499dc9cf210cf7" exitCode=0 Dec 03 11:57:31 crc kubenswrapper[5052]: I1203 11:57:31.689692 5052 generic.go:334] "Generic (PLEG): container finished" podID="32c3a6f2-948d-47a6-ad0f-2784e2623e8c" containerID="6d3253a75ecb252b680cf02b7ecf60d0b0bc66b2f0bd8ea511d43639a9734fb7" exitCode=143 Dec 03 11:57:31 crc kubenswrapper[5052]: I1203 11:57:31.689744 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"32c3a6f2-948d-47a6-ad0f-2784e2623e8c","Type":"ContainerDied","Data":"6e7a121aafc72c053dc5ac79b3667dcfcefa4aa263daac5c76499dc9cf210cf7"} Dec 03 11:57:31 crc kubenswrapper[5052]: I1203 11:57:31.689799 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"32c3a6f2-948d-47a6-ad0f-2784e2623e8c","Type":"ContainerDied","Data":"6d3253a75ecb252b680cf02b7ecf60d0b0bc66b2f0bd8ea511d43639a9734fb7"} Dec 03 11:57:31 crc kubenswrapper[5052]: I1203 11:57:31.697827 5052 scope.go:117] "RemoveContainer" containerID="d224717c6d9ca8135fd510e03a6c40c9f2b1a56dd8b4a3b685af3e2dfa855982" Dec 03 11:57:31 crc kubenswrapper[5052]: E1203 11:57:31.698092 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 11:57:31 crc kubenswrapper[5052]: I1203 11:57:31.740203 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/glance-default-internal-api-0" podStartSLOduration=13.740180102 podStartE2EDuration="13.740180102s" podCreationTimestamp="2025-12-03 11:57:18 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:57:30.725951726 +0000 UTC m=+2241.314193500" watchObservedRunningTime="2025-12-03 11:57:31.740180102 +0000 UTC m=+2242.328421866" Dec 03 11:57:33 crc kubenswrapper[5052]: I1203 11:57:33.209992 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:57:33 crc kubenswrapper[5052]: I1203 11:57:33.216771 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:57:33 crc kubenswrapper[5052]: I1203 11:57:33.363502 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/bf275c0c-27e5-462b-883e-9483cfe91bcd-logs\") pod \"bf275c0c-27e5-462b-883e-9483cfe91bcd\" (UID: \"bf275c0c-27e5-462b-883e-9483cfe91bcd\") " Dec 03 11:57:33 crc kubenswrapper[5052]: I1203 11:57:33.363573 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/32c3a6f2-948d-47a6-ad0f-2784e2623e8c-config-data\") pod \"32c3a6f2-948d-47a6-ad0f-2784e2623e8c\" (UID: \"32c3a6f2-948d-47a6-ad0f-2784e2623e8c\") " Dec 03 11:57:33 crc kubenswrapper[5052]: I1203 11:57:33.363605 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7nzs8\" (UniqueName: \"kubernetes.io/projected/32c3a6f2-948d-47a6-ad0f-2784e2623e8c-kube-api-access-7nzs8\") pod \"32c3a6f2-948d-47a6-ad0f-2784e2623e8c\" (UID: \"32c3a6f2-948d-47a6-ad0f-2784e2623e8c\") " Dec 03 11:57:33 crc kubenswrapper[5052]: I1203 11:57:33.363630 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rc4pw\" (UniqueName: \"kubernetes.io/projected/bf275c0c-27e5-462b-883e-9483cfe91bcd-kube-api-access-rc4pw\") pod \"bf275c0c-27e5-462b-883e-9483cfe91bcd\" (UID: \"bf275c0c-27e5-462b-883e-9483cfe91bcd\") " Dec 03 11:57:33 crc kubenswrapper[5052]: I1203 11:57:33.363682 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"glance\" (UniqueName: \"kubernetes.io/local-volume/local-storage16-crc\") pod \"bf275c0c-27e5-462b-883e-9483cfe91bcd\" (UID: \"bf275c0c-27e5-462b-883e-9483cfe91bcd\") " Dec 03 11:57:33 crc kubenswrapper[5052]: I1203 11:57:33.363718 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/32c3a6f2-948d-47a6-ad0f-2784e2623e8c-internal-tls-certs\") pod \"32c3a6f2-948d-47a6-ad0f-2784e2623e8c\" (UID: \"32c3a6f2-948d-47a6-ad0f-2784e2623e8c\") " Dec 03 11:57:33 crc kubenswrapper[5052]: I1203 11:57:33.363744 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/bf275c0c-27e5-462b-883e-9483cfe91bcd-public-tls-certs\") pod \"bf275c0c-27e5-462b-883e-9483cfe91bcd\" (UID: \"bf275c0c-27e5-462b-883e-9483cfe91bcd\") " Dec 03 11:57:33 crc kubenswrapper[5052]: I1203 11:57:33.363802 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/bf275c0c-27e5-462b-883e-9483cfe91bcd-combined-ca-bundle\") pod \"bf275c0c-27e5-462b-883e-9483cfe91bcd\" (UID: \"bf275c0c-27e5-462b-883e-9483cfe91bcd\") " Dec 03 11:57:33 crc kubenswrapper[5052]: I1203 11:57:33.363829 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"glance\" (UniqueName: \"kubernetes.io/local-volume/local-storage15-crc\") pod \"32c3a6f2-948d-47a6-ad0f-2784e2623e8c\" (UID: \"32c3a6f2-948d-47a6-ad0f-2784e2623e8c\") " Dec 03 11:57:33 crc kubenswrapper[5052]: I1203 11:57:33.363872 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/bf275c0c-27e5-462b-883e-9483cfe91bcd-scripts\") pod \"bf275c0c-27e5-462b-883e-9483cfe91bcd\" (UID: \"bf275c0c-27e5-462b-883e-9483cfe91bcd\") " Dec 03 11:57:33 crc kubenswrapper[5052]: I1203 11:57:33.363908 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/bf275c0c-27e5-462b-883e-9483cfe91bcd-httpd-run\") pod \"bf275c0c-27e5-462b-883e-9483cfe91bcd\" (UID: \"bf275c0c-27e5-462b-883e-9483cfe91bcd\") " Dec 03 11:57:33 crc kubenswrapper[5052]: I1203 11:57:33.363957 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/32c3a6f2-948d-47a6-ad0f-2784e2623e8c-scripts\") pod \"32c3a6f2-948d-47a6-ad0f-2784e2623e8c\" (UID: \"32c3a6f2-948d-47a6-ad0f-2784e2623e8c\") " Dec 03 11:57:33 crc kubenswrapper[5052]: I1203 11:57:33.363979 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/32c3a6f2-948d-47a6-ad0f-2784e2623e8c-combined-ca-bundle\") pod \"32c3a6f2-948d-47a6-ad0f-2784e2623e8c\" (UID: \"32c3a6f2-948d-47a6-ad0f-2784e2623e8c\") " Dec 03 11:57:33 crc kubenswrapper[5052]: I1203 11:57:33.364025 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/32c3a6f2-948d-47a6-ad0f-2784e2623e8c-httpd-run\") pod \"32c3a6f2-948d-47a6-ad0f-2784e2623e8c\" (UID: \"32c3a6f2-948d-47a6-ad0f-2784e2623e8c\") " Dec 03 11:57:33 crc kubenswrapper[5052]: I1203 11:57:33.364046 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/bf275c0c-27e5-462b-883e-9483cfe91bcd-config-data\") pod \"bf275c0c-27e5-462b-883e-9483cfe91bcd\" (UID: \"bf275c0c-27e5-462b-883e-9483cfe91bcd\") " Dec 03 11:57:33 crc kubenswrapper[5052]: I1203 11:57:33.364082 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/32c3a6f2-948d-47a6-ad0f-2784e2623e8c-logs\") pod \"32c3a6f2-948d-47a6-ad0f-2784e2623e8c\" (UID: \"32c3a6f2-948d-47a6-ad0f-2784e2623e8c\") " Dec 03 11:57:33 crc kubenswrapper[5052]: I1203 11:57:33.365257 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/32c3a6f2-948d-47a6-ad0f-2784e2623e8c-logs" (OuterVolumeSpecName: "logs") pod "32c3a6f2-948d-47a6-ad0f-2784e2623e8c" (UID: "32c3a6f2-948d-47a6-ad0f-2784e2623e8c"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:57:33 crc kubenswrapper[5052]: I1203 11:57:33.368253 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bf275c0c-27e5-462b-883e-9483cfe91bcd-httpd-run" (OuterVolumeSpecName: "httpd-run") pod "bf275c0c-27e5-462b-883e-9483cfe91bcd" (UID: "bf275c0c-27e5-462b-883e-9483cfe91bcd"). InnerVolumeSpecName "httpd-run". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:57:33 crc kubenswrapper[5052]: I1203 11:57:33.368603 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bf275c0c-27e5-462b-883e-9483cfe91bcd-logs" (OuterVolumeSpecName: "logs") pod "bf275c0c-27e5-462b-883e-9483cfe91bcd" (UID: "bf275c0c-27e5-462b-883e-9483cfe91bcd"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:57:33 crc kubenswrapper[5052]: I1203 11:57:33.368617 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bf275c0c-27e5-462b-883e-9483cfe91bcd-kube-api-access-rc4pw" (OuterVolumeSpecName: "kube-api-access-rc4pw") pod "bf275c0c-27e5-462b-883e-9483cfe91bcd" (UID: "bf275c0c-27e5-462b-883e-9483cfe91bcd"). InnerVolumeSpecName "kube-api-access-rc4pw". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:57:33 crc kubenswrapper[5052]: I1203 11:57:33.368864 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/32c3a6f2-948d-47a6-ad0f-2784e2623e8c-httpd-run" (OuterVolumeSpecName: "httpd-run") pod "32c3a6f2-948d-47a6-ad0f-2784e2623e8c" (UID: "32c3a6f2-948d-47a6-ad0f-2784e2623e8c"). InnerVolumeSpecName "httpd-run". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:57:33 crc kubenswrapper[5052]: I1203 11:57:33.370313 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/32c3a6f2-948d-47a6-ad0f-2784e2623e8c-scripts" (OuterVolumeSpecName: "scripts") pod "32c3a6f2-948d-47a6-ad0f-2784e2623e8c" (UID: "32c3a6f2-948d-47a6-ad0f-2784e2623e8c"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:57:33 crc kubenswrapper[5052]: I1203 11:57:33.370531 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bf275c0c-27e5-462b-883e-9483cfe91bcd-scripts" (OuterVolumeSpecName: "scripts") pod "bf275c0c-27e5-462b-883e-9483cfe91bcd" (UID: "bf275c0c-27e5-462b-883e-9483cfe91bcd"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:57:33 crc kubenswrapper[5052]: I1203 11:57:33.370984 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage16-crc" (OuterVolumeSpecName: "glance") pod "bf275c0c-27e5-462b-883e-9483cfe91bcd" (UID: "bf275c0c-27e5-462b-883e-9483cfe91bcd"). InnerVolumeSpecName "local-storage16-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 11:57:33 crc kubenswrapper[5052]: I1203 11:57:33.372654 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/32c3a6f2-948d-47a6-ad0f-2784e2623e8c-kube-api-access-7nzs8" (OuterVolumeSpecName: "kube-api-access-7nzs8") pod "32c3a6f2-948d-47a6-ad0f-2784e2623e8c" (UID: "32c3a6f2-948d-47a6-ad0f-2784e2623e8c"). InnerVolumeSpecName "kube-api-access-7nzs8". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:57:33 crc kubenswrapper[5052]: I1203 11:57:33.379885 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage15-crc" (OuterVolumeSpecName: "glance") pod "32c3a6f2-948d-47a6-ad0f-2784e2623e8c" (UID: "32c3a6f2-948d-47a6-ad0f-2784e2623e8c"). InnerVolumeSpecName "local-storage15-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 11:57:33 crc kubenswrapper[5052]: I1203 11:57:33.403834 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bf275c0c-27e5-462b-883e-9483cfe91bcd-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "bf275c0c-27e5-462b-883e-9483cfe91bcd" (UID: "bf275c0c-27e5-462b-883e-9483cfe91bcd"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:57:33 crc kubenswrapper[5052]: I1203 11:57:33.411278 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/32c3a6f2-948d-47a6-ad0f-2784e2623e8c-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "32c3a6f2-948d-47a6-ad0f-2784e2623e8c" (UID: "32c3a6f2-948d-47a6-ad0f-2784e2623e8c"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:57:33 crc kubenswrapper[5052]: I1203 11:57:33.420639 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bf275c0c-27e5-462b-883e-9483cfe91bcd-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "bf275c0c-27e5-462b-883e-9483cfe91bcd" (UID: "bf275c0c-27e5-462b-883e-9483cfe91bcd"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:57:33 crc kubenswrapper[5052]: I1203 11:57:33.431366 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/32c3a6f2-948d-47a6-ad0f-2784e2623e8c-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "32c3a6f2-948d-47a6-ad0f-2784e2623e8c" (UID: "32c3a6f2-948d-47a6-ad0f-2784e2623e8c"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:57:33 crc kubenswrapper[5052]: I1203 11:57:33.439767 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/32c3a6f2-948d-47a6-ad0f-2784e2623e8c-config-data" (OuterVolumeSpecName: "config-data") pod "32c3a6f2-948d-47a6-ad0f-2784e2623e8c" (UID: "32c3a6f2-948d-47a6-ad0f-2784e2623e8c"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:57:33 crc kubenswrapper[5052]: I1203 11:57:33.439946 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bf275c0c-27e5-462b-883e-9483cfe91bcd-config-data" (OuterVolumeSpecName: "config-data") pod "bf275c0c-27e5-462b-883e-9483cfe91bcd" (UID: "bf275c0c-27e5-462b-883e-9483cfe91bcd"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:57:33 crc kubenswrapper[5052]: I1203 11:57:33.465703 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/bf275c0c-27e5-462b-883e-9483cfe91bcd-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:57:33 crc kubenswrapper[5052]: I1203 11:57:33.465768 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage15-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage15-crc\") on node \"crc\" " Dec 03 11:57:33 crc kubenswrapper[5052]: I1203 11:57:33.465779 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/bf275c0c-27e5-462b-883e-9483cfe91bcd-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:57:33 crc kubenswrapper[5052]: I1203 11:57:33.465788 5052 reconciler_common.go:293] "Volume detached for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/bf275c0c-27e5-462b-883e-9483cfe91bcd-httpd-run\") on node \"crc\" DevicePath \"\"" Dec 03 11:57:33 crc kubenswrapper[5052]: I1203 11:57:33.465796 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/32c3a6f2-948d-47a6-ad0f-2784e2623e8c-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:57:33 crc kubenswrapper[5052]: I1203 11:57:33.465804 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/32c3a6f2-948d-47a6-ad0f-2784e2623e8c-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:57:33 crc kubenswrapper[5052]: I1203 11:57:33.465812 5052 reconciler_common.go:293] "Volume detached for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/32c3a6f2-948d-47a6-ad0f-2784e2623e8c-httpd-run\") on node \"crc\" DevicePath \"\"" Dec 03 11:57:33 crc kubenswrapper[5052]: I1203 11:57:33.465820 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/bf275c0c-27e5-462b-883e-9483cfe91bcd-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:57:33 crc kubenswrapper[5052]: I1203 11:57:33.465827 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/32c3a6f2-948d-47a6-ad0f-2784e2623e8c-logs\") on node \"crc\" DevicePath \"\"" Dec 03 11:57:33 crc kubenswrapper[5052]: I1203 11:57:33.465834 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/bf275c0c-27e5-462b-883e-9483cfe91bcd-logs\") on node \"crc\" DevicePath \"\"" Dec 03 11:57:33 crc kubenswrapper[5052]: I1203 11:57:33.465844 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/32c3a6f2-948d-47a6-ad0f-2784e2623e8c-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:57:33 crc kubenswrapper[5052]: I1203 11:57:33.465852 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-7nzs8\" (UniqueName: \"kubernetes.io/projected/32c3a6f2-948d-47a6-ad0f-2784e2623e8c-kube-api-access-7nzs8\") on node \"crc\" DevicePath \"\"" Dec 03 11:57:33 crc kubenswrapper[5052]: I1203 11:57:33.465862 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-rc4pw\" (UniqueName: \"kubernetes.io/projected/bf275c0c-27e5-462b-883e-9483cfe91bcd-kube-api-access-rc4pw\") on node \"crc\" DevicePath \"\"" Dec 03 11:57:33 crc kubenswrapper[5052]: I1203 11:57:33.465877 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage16-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage16-crc\") on node \"crc\" " Dec 03 11:57:33 crc kubenswrapper[5052]: I1203 11:57:33.465885 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/32c3a6f2-948d-47a6-ad0f-2784e2623e8c-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 11:57:33 crc kubenswrapper[5052]: I1203 11:57:33.465894 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/bf275c0c-27e5-462b-883e-9483cfe91bcd-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 11:57:33 crc kubenswrapper[5052]: I1203 11:57:33.483015 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage15-crc" (UniqueName: "kubernetes.io/local-volume/local-storage15-crc") on node "crc" Dec 03 11:57:33 crc kubenswrapper[5052]: I1203 11:57:33.485606 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage16-crc" (UniqueName: "kubernetes.io/local-volume/local-storage16-crc") on node "crc" Dec 03 11:57:33 crc kubenswrapper[5052]: I1203 11:57:33.567719 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage16-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage16-crc\") on node \"crc\" DevicePath \"\"" Dec 03 11:57:33 crc kubenswrapper[5052]: I1203 11:57:33.567746 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage15-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage15-crc\") on node \"crc\" DevicePath \"\"" Dec 03 11:57:33 crc kubenswrapper[5052]: I1203 11:57:33.720119 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"bf275c0c-27e5-462b-883e-9483cfe91bcd","Type":"ContainerDied","Data":"131bdbef6449ae8940b0ed097e6d0a58c61746393c4bbc7dc14f1b111e3e4451"} Dec 03 11:57:33 crc kubenswrapper[5052]: I1203 11:57:33.720133 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:57:33 crc kubenswrapper[5052]: I1203 11:57:33.720193 5052 scope.go:117] "RemoveContainer" containerID="8a2eec599442bf1bc410ee4b4d585262201c84834e013c4700b3283ebadb883c" Dec 03 11:57:33 crc kubenswrapper[5052]: I1203 11:57:33.725049 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"32c3a6f2-948d-47a6-ad0f-2784e2623e8c","Type":"ContainerDied","Data":"aff493502bb952fcc13fba9e95e99a0605bebb7c164576dbf24992e81bd483b9"} Dec 03 11:57:33 crc kubenswrapper[5052]: I1203 11:57:33.725161 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:57:33 crc kubenswrapper[5052]: I1203 11:57:33.769728 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 11:57:33 crc kubenswrapper[5052]: I1203 11:57:33.797842 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 11:57:33 crc kubenswrapper[5052]: I1203 11:57:33.808305 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 11:57:33 crc kubenswrapper[5052]: I1203 11:57:33.832014 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 11:57:33 crc kubenswrapper[5052]: I1203 11:57:33.839786 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 11:57:33 crc kubenswrapper[5052]: E1203 11:57:33.841154 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="32c3a6f2-948d-47a6-ad0f-2784e2623e8c" containerName="glance-log" Dec 03 11:57:33 crc kubenswrapper[5052]: I1203 11:57:33.841184 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="32c3a6f2-948d-47a6-ad0f-2784e2623e8c" containerName="glance-log" Dec 03 11:57:33 crc kubenswrapper[5052]: E1203 11:57:33.841204 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="bf275c0c-27e5-462b-883e-9483cfe91bcd" containerName="glance-httpd" Dec 03 11:57:33 crc kubenswrapper[5052]: I1203 11:57:33.841213 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="bf275c0c-27e5-462b-883e-9483cfe91bcd" containerName="glance-httpd" Dec 03 11:57:33 crc kubenswrapper[5052]: E1203 11:57:33.841241 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="32c3a6f2-948d-47a6-ad0f-2784e2623e8c" containerName="glance-httpd" Dec 03 11:57:33 crc kubenswrapper[5052]: I1203 11:57:33.841248 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="32c3a6f2-948d-47a6-ad0f-2784e2623e8c" containerName="glance-httpd" Dec 03 11:57:33 crc kubenswrapper[5052]: E1203 11:57:33.841261 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="bf275c0c-27e5-462b-883e-9483cfe91bcd" containerName="glance-log" Dec 03 11:57:33 crc kubenswrapper[5052]: I1203 11:57:33.841268 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="bf275c0c-27e5-462b-883e-9483cfe91bcd" containerName="glance-log" Dec 03 11:57:33 crc kubenswrapper[5052]: I1203 11:57:33.841484 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="32c3a6f2-948d-47a6-ad0f-2784e2623e8c" containerName="glance-log" Dec 03 11:57:33 crc kubenswrapper[5052]: I1203 11:57:33.841496 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="bf275c0c-27e5-462b-883e-9483cfe91bcd" containerName="glance-httpd" Dec 03 11:57:33 crc kubenswrapper[5052]: I1203 11:57:33.841528 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="32c3a6f2-948d-47a6-ad0f-2784e2623e8c" containerName="glance-httpd" Dec 03 11:57:33 crc kubenswrapper[5052]: I1203 11:57:33.841537 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="bf275c0c-27e5-462b-883e-9483cfe91bcd" containerName="glance-log" Dec 03 11:57:33 crc kubenswrapper[5052]: I1203 11:57:33.842732 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:57:33 crc kubenswrapper[5052]: I1203 11:57:33.849067 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-scripts" Dec 03 11:57:33 crc kubenswrapper[5052]: I1203 11:57:33.849299 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-glance-dockercfg-8nxm6" Dec 03 11:57:33 crc kubenswrapper[5052]: I1203 11:57:33.849354 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-glance-default-public-svc" Dec 03 11:57:33 crc kubenswrapper[5052]: I1203 11:57:33.849669 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-default-external-config-data" Dec 03 11:57:33 crc kubenswrapper[5052]: I1203 11:57:33.861919 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 11:57:33 crc kubenswrapper[5052]: I1203 11:57:33.863735 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:57:33 crc kubenswrapper[5052]: I1203 11:57:33.871419 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-default-internal-config-data" Dec 03 11:57:33 crc kubenswrapper[5052]: I1203 11:57:33.871665 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-glance-default-internal-svc" Dec 03 11:57:33 crc kubenswrapper[5052]: I1203 11:57:33.872040 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 11:57:33 crc kubenswrapper[5052]: I1203 11:57:33.877100 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 11:57:33 crc kubenswrapper[5052]: I1203 11:57:33.943322 5052 scope.go:117] "RemoveContainer" containerID="bf55314ea998abd4f91d7fde30baf699d2e4eb5a8a6691ce81452f4e56a42948" Dec 03 11:57:33 crc kubenswrapper[5052]: I1203 11:57:33.975403 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/0d55422b-84bf-4455-9112-94de20fb5b6d-internal-tls-certs\") pod \"glance-default-internal-api-0\" (UID: \"0d55422b-84bf-4455-9112-94de20fb5b6d\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:57:33 crc kubenswrapper[5052]: I1203 11:57:33.975479 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/9fe1f02f-d2cf-4220-925c-6ecdcd099048-config-data\") pod \"glance-default-external-api-0\" (UID: \"9fe1f02f-d2cf-4220-925c-6ecdcd099048\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:57:33 crc kubenswrapper[5052]: I1203 11:57:33.975504 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/0d55422b-84bf-4455-9112-94de20fb5b6d-logs\") pod \"glance-default-internal-api-0\" (UID: \"0d55422b-84bf-4455-9112-94de20fb5b6d\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:57:33 crc kubenswrapper[5052]: I1203 11:57:33.975621 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/9fe1f02f-d2cf-4220-925c-6ecdcd099048-public-tls-certs\") pod \"glance-default-external-api-0\" (UID: \"9fe1f02f-d2cf-4220-925c-6ecdcd099048\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:57:33 crc kubenswrapper[5052]: I1203 11:57:33.975648 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/9fe1f02f-d2cf-4220-925c-6ecdcd099048-logs\") pod \"glance-default-external-api-0\" (UID: \"9fe1f02f-d2cf-4220-925c-6ecdcd099048\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:57:33 crc kubenswrapper[5052]: I1203 11:57:33.975683 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/9fe1f02f-d2cf-4220-925c-6ecdcd099048-scripts\") pod \"glance-default-external-api-0\" (UID: \"9fe1f02f-d2cf-4220-925c-6ecdcd099048\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:57:33 crc kubenswrapper[5052]: I1203 11:57:33.975714 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage16-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage16-crc\") pod \"glance-default-external-api-0\" (UID: \"9fe1f02f-d2cf-4220-925c-6ecdcd099048\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:57:33 crc kubenswrapper[5052]: I1203 11:57:33.975746 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/0d55422b-84bf-4455-9112-94de20fb5b6d-scripts\") pod \"glance-default-internal-api-0\" (UID: \"0d55422b-84bf-4455-9112-94de20fb5b6d\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:57:33 crc kubenswrapper[5052]: I1203 11:57:33.975774 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x829v\" (UniqueName: \"kubernetes.io/projected/9fe1f02f-d2cf-4220-925c-6ecdcd099048-kube-api-access-x829v\") pod \"glance-default-external-api-0\" (UID: \"9fe1f02f-d2cf-4220-925c-6ecdcd099048\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:57:33 crc kubenswrapper[5052]: I1203 11:57:33.975795 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0d55422b-84bf-4455-9112-94de20fb5b6d-combined-ca-bundle\") pod \"glance-default-internal-api-0\" (UID: \"0d55422b-84bf-4455-9112-94de20fb5b6d\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:57:33 crc kubenswrapper[5052]: I1203 11:57:33.975819 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/9fe1f02f-d2cf-4220-925c-6ecdcd099048-httpd-run\") pod \"glance-default-external-api-0\" (UID: \"9fe1f02f-d2cf-4220-925c-6ecdcd099048\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:57:33 crc kubenswrapper[5052]: I1203 11:57:33.975837 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fvnwr\" (UniqueName: \"kubernetes.io/projected/0d55422b-84bf-4455-9112-94de20fb5b6d-kube-api-access-fvnwr\") pod \"glance-default-internal-api-0\" (UID: \"0d55422b-84bf-4455-9112-94de20fb5b6d\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:57:33 crc kubenswrapper[5052]: I1203 11:57:33.975856 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/0d55422b-84bf-4455-9112-94de20fb5b6d-config-data\") pod \"glance-default-internal-api-0\" (UID: \"0d55422b-84bf-4455-9112-94de20fb5b6d\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:57:33 crc kubenswrapper[5052]: I1203 11:57:33.975881 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/0d55422b-84bf-4455-9112-94de20fb5b6d-httpd-run\") pod \"glance-default-internal-api-0\" (UID: \"0d55422b-84bf-4455-9112-94de20fb5b6d\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:57:33 crc kubenswrapper[5052]: I1203 11:57:33.975902 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage15-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage15-crc\") pod \"glance-default-internal-api-0\" (UID: \"0d55422b-84bf-4455-9112-94de20fb5b6d\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:57:33 crc kubenswrapper[5052]: I1203 11:57:33.975919 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9fe1f02f-d2cf-4220-925c-6ecdcd099048-combined-ca-bundle\") pod \"glance-default-external-api-0\" (UID: \"9fe1f02f-d2cf-4220-925c-6ecdcd099048\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:57:33 crc kubenswrapper[5052]: I1203 11:57:33.988699 5052 scope.go:117] "RemoveContainer" containerID="6e7a121aafc72c053dc5ac79b3667dcfcefa4aa263daac5c76499dc9cf210cf7" Dec 03 11:57:34 crc kubenswrapper[5052]: I1203 11:57:34.056224 5052 scope.go:117] "RemoveContainer" containerID="6d3253a75ecb252b680cf02b7ecf60d0b0bc66b2f0bd8ea511d43639a9734fb7" Dec 03 11:57:34 crc kubenswrapper[5052]: I1203 11:57:34.077367 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/9fe1f02f-d2cf-4220-925c-6ecdcd099048-public-tls-certs\") pod \"glance-default-external-api-0\" (UID: \"9fe1f02f-d2cf-4220-925c-6ecdcd099048\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:57:34 crc kubenswrapper[5052]: I1203 11:57:34.077410 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/9fe1f02f-d2cf-4220-925c-6ecdcd099048-logs\") pod \"glance-default-external-api-0\" (UID: \"9fe1f02f-d2cf-4220-925c-6ecdcd099048\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:57:34 crc kubenswrapper[5052]: I1203 11:57:34.077436 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/9fe1f02f-d2cf-4220-925c-6ecdcd099048-scripts\") pod \"glance-default-external-api-0\" (UID: \"9fe1f02f-d2cf-4220-925c-6ecdcd099048\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:57:34 crc kubenswrapper[5052]: I1203 11:57:34.077457 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage16-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage16-crc\") pod \"glance-default-external-api-0\" (UID: \"9fe1f02f-d2cf-4220-925c-6ecdcd099048\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:57:34 crc kubenswrapper[5052]: I1203 11:57:34.077477 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/0d55422b-84bf-4455-9112-94de20fb5b6d-scripts\") pod \"glance-default-internal-api-0\" (UID: \"0d55422b-84bf-4455-9112-94de20fb5b6d\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:57:34 crc kubenswrapper[5052]: I1203 11:57:34.077495 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-x829v\" (UniqueName: \"kubernetes.io/projected/9fe1f02f-d2cf-4220-925c-6ecdcd099048-kube-api-access-x829v\") pod \"glance-default-external-api-0\" (UID: \"9fe1f02f-d2cf-4220-925c-6ecdcd099048\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:57:34 crc kubenswrapper[5052]: I1203 11:57:34.077526 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0d55422b-84bf-4455-9112-94de20fb5b6d-combined-ca-bundle\") pod \"glance-default-internal-api-0\" (UID: \"0d55422b-84bf-4455-9112-94de20fb5b6d\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:57:34 crc kubenswrapper[5052]: I1203 11:57:34.077553 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/9fe1f02f-d2cf-4220-925c-6ecdcd099048-httpd-run\") pod \"glance-default-external-api-0\" (UID: \"9fe1f02f-d2cf-4220-925c-6ecdcd099048\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:57:34 crc kubenswrapper[5052]: I1203 11:57:34.077569 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-fvnwr\" (UniqueName: \"kubernetes.io/projected/0d55422b-84bf-4455-9112-94de20fb5b6d-kube-api-access-fvnwr\") pod \"glance-default-internal-api-0\" (UID: \"0d55422b-84bf-4455-9112-94de20fb5b6d\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:57:34 crc kubenswrapper[5052]: I1203 11:57:34.077586 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/0d55422b-84bf-4455-9112-94de20fb5b6d-config-data\") pod \"glance-default-internal-api-0\" (UID: \"0d55422b-84bf-4455-9112-94de20fb5b6d\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:57:34 crc kubenswrapper[5052]: I1203 11:57:34.077609 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/0d55422b-84bf-4455-9112-94de20fb5b6d-httpd-run\") pod \"glance-default-internal-api-0\" (UID: \"0d55422b-84bf-4455-9112-94de20fb5b6d\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:57:34 crc kubenswrapper[5052]: I1203 11:57:34.077629 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage15-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage15-crc\") pod \"glance-default-internal-api-0\" (UID: \"0d55422b-84bf-4455-9112-94de20fb5b6d\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:57:34 crc kubenswrapper[5052]: I1203 11:57:34.077645 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9fe1f02f-d2cf-4220-925c-6ecdcd099048-combined-ca-bundle\") pod \"glance-default-external-api-0\" (UID: \"9fe1f02f-d2cf-4220-925c-6ecdcd099048\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:57:34 crc kubenswrapper[5052]: I1203 11:57:34.077672 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/0d55422b-84bf-4455-9112-94de20fb5b6d-internal-tls-certs\") pod \"glance-default-internal-api-0\" (UID: \"0d55422b-84bf-4455-9112-94de20fb5b6d\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:57:34 crc kubenswrapper[5052]: I1203 11:57:34.077713 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/9fe1f02f-d2cf-4220-925c-6ecdcd099048-config-data\") pod \"glance-default-external-api-0\" (UID: \"9fe1f02f-d2cf-4220-925c-6ecdcd099048\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:57:34 crc kubenswrapper[5052]: I1203 11:57:34.077731 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/0d55422b-84bf-4455-9112-94de20fb5b6d-logs\") pod \"glance-default-internal-api-0\" (UID: \"0d55422b-84bf-4455-9112-94de20fb5b6d\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:57:34 crc kubenswrapper[5052]: I1203 11:57:34.078181 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/0d55422b-84bf-4455-9112-94de20fb5b6d-logs\") pod \"glance-default-internal-api-0\" (UID: \"0d55422b-84bf-4455-9112-94de20fb5b6d\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:57:34 crc kubenswrapper[5052]: I1203 11:57:34.080088 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/9fe1f02f-d2cf-4220-925c-6ecdcd099048-httpd-run\") pod \"glance-default-external-api-0\" (UID: \"9fe1f02f-d2cf-4220-925c-6ecdcd099048\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:57:34 crc kubenswrapper[5052]: I1203 11:57:34.080373 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/9fe1f02f-d2cf-4220-925c-6ecdcd099048-logs\") pod \"glance-default-external-api-0\" (UID: \"9fe1f02f-d2cf-4220-925c-6ecdcd099048\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:57:34 crc kubenswrapper[5052]: I1203 11:57:34.083942 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage15-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage15-crc\") pod \"glance-default-internal-api-0\" (UID: \"0d55422b-84bf-4455-9112-94de20fb5b6d\") device mount path \"/mnt/openstack/pv15\"" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:57:34 crc kubenswrapper[5052]: I1203 11:57:34.084973 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage16-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage16-crc\") pod \"glance-default-external-api-0\" (UID: \"9fe1f02f-d2cf-4220-925c-6ecdcd099048\") device mount path \"/mnt/openstack/pv16\"" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:57:34 crc kubenswrapper[5052]: I1203 11:57:34.085781 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/9fe1f02f-d2cf-4220-925c-6ecdcd099048-public-tls-certs\") pod \"glance-default-external-api-0\" (UID: \"9fe1f02f-d2cf-4220-925c-6ecdcd099048\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:57:34 crc kubenswrapper[5052]: I1203 11:57:34.086107 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/0d55422b-84bf-4455-9112-94de20fb5b6d-httpd-run\") pod \"glance-default-internal-api-0\" (UID: \"0d55422b-84bf-4455-9112-94de20fb5b6d\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:57:34 crc kubenswrapper[5052]: I1203 11:57:34.088007 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/0d55422b-84bf-4455-9112-94de20fb5b6d-scripts\") pod \"glance-default-internal-api-0\" (UID: \"0d55422b-84bf-4455-9112-94de20fb5b6d\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:57:34 crc kubenswrapper[5052]: I1203 11:57:34.089580 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9fe1f02f-d2cf-4220-925c-6ecdcd099048-combined-ca-bundle\") pod \"glance-default-external-api-0\" (UID: \"9fe1f02f-d2cf-4220-925c-6ecdcd099048\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:57:34 crc kubenswrapper[5052]: I1203 11:57:34.089654 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/0d55422b-84bf-4455-9112-94de20fb5b6d-internal-tls-certs\") pod \"glance-default-internal-api-0\" (UID: \"0d55422b-84bf-4455-9112-94de20fb5b6d\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:57:34 crc kubenswrapper[5052]: I1203 11:57:34.089758 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/9fe1f02f-d2cf-4220-925c-6ecdcd099048-scripts\") pod \"glance-default-external-api-0\" (UID: \"9fe1f02f-d2cf-4220-925c-6ecdcd099048\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:57:34 crc kubenswrapper[5052]: I1203 11:57:34.091163 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/9fe1f02f-d2cf-4220-925c-6ecdcd099048-config-data\") pod \"glance-default-external-api-0\" (UID: \"9fe1f02f-d2cf-4220-925c-6ecdcd099048\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:57:34 crc kubenswrapper[5052]: I1203 11:57:34.106949 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-fvnwr\" (UniqueName: \"kubernetes.io/projected/0d55422b-84bf-4455-9112-94de20fb5b6d-kube-api-access-fvnwr\") pod \"glance-default-internal-api-0\" (UID: \"0d55422b-84bf-4455-9112-94de20fb5b6d\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:57:34 crc kubenswrapper[5052]: I1203 11:57:34.109386 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0d55422b-84bf-4455-9112-94de20fb5b6d-combined-ca-bundle\") pod \"glance-default-internal-api-0\" (UID: \"0d55422b-84bf-4455-9112-94de20fb5b6d\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:57:34 crc kubenswrapper[5052]: I1203 11:57:34.109571 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/0d55422b-84bf-4455-9112-94de20fb5b6d-config-data\") pod \"glance-default-internal-api-0\" (UID: \"0d55422b-84bf-4455-9112-94de20fb5b6d\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:57:34 crc kubenswrapper[5052]: I1203 11:57:34.110692 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-x829v\" (UniqueName: \"kubernetes.io/projected/9fe1f02f-d2cf-4220-925c-6ecdcd099048-kube-api-access-x829v\") pod \"glance-default-external-api-0\" (UID: \"9fe1f02f-d2cf-4220-925c-6ecdcd099048\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:57:34 crc kubenswrapper[5052]: I1203 11:57:34.133044 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage16-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage16-crc\") pod \"glance-default-external-api-0\" (UID: \"9fe1f02f-d2cf-4220-925c-6ecdcd099048\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:57:34 crc kubenswrapper[5052]: I1203 11:57:34.133764 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage15-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage15-crc\") pod \"glance-default-internal-api-0\" (UID: \"0d55422b-84bf-4455-9112-94de20fb5b6d\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:57:34 crc kubenswrapper[5052]: I1203 11:57:34.165265 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:57:34 crc kubenswrapper[5052]: I1203 11:57:34.185423 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:57:34 crc kubenswrapper[5052]: I1203 11:57:34.438453 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 11:57:34 crc kubenswrapper[5052]: I1203 11:57:34.691606 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 11:57:34 crc kubenswrapper[5052]: W1203 11:57:34.699734 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0d55422b_84bf_4455_9112_94de20fb5b6d.slice/crio-8212e962531fdda3bed98c5640eba93c6a37f64fcef5844606b8b236e6aadac4 WatchSource:0}: Error finding container 8212e962531fdda3bed98c5640eba93c6a37f64fcef5844606b8b236e6aadac4: Status 404 returned error can't find the container with id 8212e962531fdda3bed98c5640eba93c6a37f64fcef5844606b8b236e6aadac4 Dec 03 11:57:34 crc kubenswrapper[5052]: I1203 11:57:34.738672 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"0d55422b-84bf-4455-9112-94de20fb5b6d","Type":"ContainerStarted","Data":"8212e962531fdda3bed98c5640eba93c6a37f64fcef5844606b8b236e6aadac4"} Dec 03 11:57:34 crc kubenswrapper[5052]: I1203 11:57:34.740922 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"9fe1f02f-d2cf-4220-925c-6ecdcd099048","Type":"ContainerStarted","Data":"ff8d4a99acf44da56b0cdd06d13bdea02536f2f50d2a62e30c7dfc1dacef5465"} Dec 03 11:57:34 crc kubenswrapper[5052]: I1203 11:57:34.747336 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"bac97bd6-a939-41e2-b577-05d6fb87dcb5","Type":"ContainerStarted","Data":"6348d1c5e4280b87678fd42b2fa52df5ff03d4842c5ebe3f001352fb55f604d6"} Dec 03 11:57:35 crc kubenswrapper[5052]: I1203 11:57:35.702900 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="32c3a6f2-948d-47a6-ad0f-2784e2623e8c" path="/var/lib/kubelet/pods/32c3a6f2-948d-47a6-ad0f-2784e2623e8c/volumes" Dec 03 11:57:35 crc kubenswrapper[5052]: I1203 11:57:35.704568 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bf275c0c-27e5-462b-883e-9483cfe91bcd" path="/var/lib/kubelet/pods/bf275c0c-27e5-462b-883e-9483cfe91bcd/volumes" Dec 03 11:57:35 crc kubenswrapper[5052]: I1203 11:57:35.762360 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"9fe1f02f-d2cf-4220-925c-6ecdcd099048","Type":"ContainerStarted","Data":"0d2b603ec774a9e5e57e3ee6ad57ea900ac923cb148ec977d0759fb33a4ba2e8"} Dec 03 11:57:35 crc kubenswrapper[5052]: I1203 11:57:35.762399 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"9fe1f02f-d2cf-4220-925c-6ecdcd099048","Type":"ContainerStarted","Data":"cb18e4b2f1c1bd57f2ceaa59ede059a92c607d787ad9e92d233d8bbb0cff36be"} Dec 03 11:57:35 crc kubenswrapper[5052]: I1203 11:57:35.765160 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"0d55422b-84bf-4455-9112-94de20fb5b6d","Type":"ContainerStarted","Data":"f3557803e708885b7be19d0c0a7d6e82313f419fd68f6cf32092dc379a16b9af"} Dec 03 11:57:36 crc kubenswrapper[5052]: I1203 11:57:36.576067 5052 scope.go:117] "RemoveContainer" containerID="2ee3c0361bbf90cf40dcd13279e44150c5f8ed97f268cf91a9701bccff60b20b" Dec 03 11:57:36 crc kubenswrapper[5052]: I1203 11:57:36.774745 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"bac97bd6-a939-41e2-b577-05d6fb87dcb5","Type":"ContainerStarted","Data":"b866b6d846a4e51e96dee71c028075183b02f4011789bce203172a26d1256e1b"} Dec 03 11:57:36 crc kubenswrapper[5052]: I1203 11:57:36.802390 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/glance-default-external-api-0" podStartSLOduration=3.8023724960000003 podStartE2EDuration="3.802372496s" podCreationTimestamp="2025-12-03 11:57:33 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:57:36.797808395 +0000 UTC m=+2247.386050169" watchObservedRunningTime="2025-12-03 11:57:36.802372496 +0000 UTC m=+2247.390614270" Dec 03 11:57:37 crc kubenswrapper[5052]: I1203 11:57:37.409587 5052 scope.go:117] "RemoveContainer" containerID="3e624a11d1616304ada395cbc34d76126b3b0f68d661f5c753eefaa96b499a5c" Dec 03 11:57:37 crc kubenswrapper[5052]: I1203 11:57:37.558999 5052 scope.go:117] "RemoveContainer" containerID="9684094fee34437582198b41d3a72ef6d576587c1466aadd379f8be1cedcf6af" Dec 03 11:57:37 crc kubenswrapper[5052]: I1203 11:57:37.728626 5052 scope.go:117] "RemoveContainer" containerID="b8bffbbe67b941eebfcc7d809311107fe0d16c3fd77f96571d99325342cabd72" Dec 03 11:57:37 crc kubenswrapper[5052]: I1203 11:57:37.758014 5052 scope.go:117] "RemoveContainer" containerID="9393f4a1875bd93dade4f4a250ae2161d3bae5bab0dfeec9d201f006f152194d" Dec 03 11:57:40 crc kubenswrapper[5052]: I1203 11:57:40.865905 5052 generic.go:334] "Generic (PLEG): container finished" podID="01bc82de-61dc-477e-b4cf-e3b0d32c99ba" containerID="be71930393f0fbaabf4c9dd57b05ba54c75c478e04e607ad33f67424c249c0f3" exitCode=0 Dec 03 11:57:40 crc kubenswrapper[5052]: I1203 11:57:40.866283 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/prometheus-metric-storage-0" event={"ID":"01bc82de-61dc-477e-b4cf-e3b0d32c99ba","Type":"ContainerDied","Data":"be71930393f0fbaabf4c9dd57b05ba54c75c478e04e607ad33f67424c249c0f3"} Dec 03 11:57:44 crc kubenswrapper[5052]: I1203 11:57:44.166232 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:57:44 crc kubenswrapper[5052]: I1203 11:57:44.166632 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:57:44 crc kubenswrapper[5052]: I1203 11:57:44.201596 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:57:44 crc kubenswrapper[5052]: I1203 11:57:44.208534 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:57:44 crc kubenswrapper[5052]: I1203 11:57:44.673355 5052 scope.go:117] "RemoveContainer" containerID="d224717c6d9ca8135fd510e03a6c40c9f2b1a56dd8b4a3b685af3e2dfa855982" Dec 03 11:57:44 crc kubenswrapper[5052]: E1203 11:57:44.673792 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 11:57:44 crc kubenswrapper[5052]: I1203 11:57:44.913390 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:57:44 crc kubenswrapper[5052]: I1203 11:57:44.913923 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:57:46 crc kubenswrapper[5052]: I1203 11:57:46.909699 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:57:46 crc kubenswrapper[5052]: I1203 11:57:46.911228 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:57:47 crc kubenswrapper[5052]: I1203 11:57:47.949534 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"0d55422b-84bf-4455-9112-94de20fb5b6d","Type":"ContainerStarted","Data":"2bf75f1bec647492a24e9e2e2a4b240c1dcac0144cf5a6f394ba143a1374064f"} Dec 03 11:57:47 crc kubenswrapper[5052]: I1203 11:57:47.986136 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/glance-default-internal-api-0" podStartSLOduration=14.986115368 podStartE2EDuration="14.986115368s" podCreationTimestamp="2025-12-03 11:57:33 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:57:47.983652938 +0000 UTC m=+2258.571894702" watchObservedRunningTime="2025-12-03 11:57:47.986115368 +0000 UTC m=+2258.574357132" Dec 03 11:57:48 crc kubenswrapper[5052]: I1203 11:57:48.960368 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/prometheus-metric-storage-0" event={"ID":"01bc82de-61dc-477e-b4cf-e3b0d32c99ba","Type":"ContainerStarted","Data":"599972ea725b2448de0f384ff58d4263ae4c6939731529af770910368c748187"} Dec 03 11:57:48 crc kubenswrapper[5052]: I1203 11:57:48.962717 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/watcher-db-sync-chlm4" event={"ID":"b3146921-cc78-401f-a918-fa8b91814f10","Type":"ContainerStarted","Data":"cfcf65482c408d6744121d3337ff364f65dfd4df7db4e0d1ca7a10b2ec5a6849"} Dec 03 11:57:48 crc kubenswrapper[5052]: I1203 11:57:48.966711 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="bac97bd6-a939-41e2-b577-05d6fb87dcb5" containerName="ceilometer-central-agent" containerID="cri-o://eb431f6c1818b9852af36eea4a58126f109d053175b87db72d750e6bc3098387" gracePeriod=30 Dec 03 11:57:48 crc kubenswrapper[5052]: I1203 11:57:48.966955 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"bac97bd6-a939-41e2-b577-05d6fb87dcb5","Type":"ContainerStarted","Data":"ecc0df50483ef40efaa5b0f8429d41c8eeeb4b9aa07d84f10b24c739cfcc7770"} Dec 03 11:57:48 crc kubenswrapper[5052]: I1203 11:57:48.966991 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:57:48 crc kubenswrapper[5052]: I1203 11:57:48.967030 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="bac97bd6-a939-41e2-b577-05d6fb87dcb5" containerName="proxy-httpd" containerID="cri-o://ecc0df50483ef40efaa5b0f8429d41c8eeeb4b9aa07d84f10b24c739cfcc7770" gracePeriod=30 Dec 03 11:57:48 crc kubenswrapper[5052]: I1203 11:57:48.967074 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="bac97bd6-a939-41e2-b577-05d6fb87dcb5" containerName="sg-core" containerID="cri-o://b866b6d846a4e51e96dee71c028075183b02f4011789bce203172a26d1256e1b" gracePeriod=30 Dec 03 11:57:48 crc kubenswrapper[5052]: I1203 11:57:48.967123 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="bac97bd6-a939-41e2-b577-05d6fb87dcb5" containerName="ceilometer-notification-agent" containerID="cri-o://6348d1c5e4280b87678fd42b2fa52df5ff03d4842c5ebe3f001352fb55f604d6" gracePeriod=30 Dec 03 11:57:48 crc kubenswrapper[5052]: I1203 11:57:48.983850 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/watcher-db-sync-chlm4" podStartSLOduration=2.885068537 podStartE2EDuration="46.983831152s" podCreationTimestamp="2025-12-03 11:57:02 +0000 UTC" firstStartedPulling="2025-12-03 11:57:03.734824588 +0000 UTC m=+2214.323066352" lastFinishedPulling="2025-12-03 11:57:47.833587203 +0000 UTC m=+2258.421828967" observedRunningTime="2025-12-03 11:57:48.981612778 +0000 UTC m=+2259.569854542" watchObservedRunningTime="2025-12-03 11:57:48.983831152 +0000 UTC m=+2259.572072916" Dec 03 11:57:49 crc kubenswrapper[5052]: I1203 11:57:49.007847 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/ceilometer-0" podStartSLOduration=10.211735883 podStartE2EDuration="31.007824728s" podCreationTimestamp="2025-12-03 11:57:18 +0000 UTC" firstStartedPulling="2025-12-03 11:57:27.020614645 +0000 UTC m=+2237.608856409" lastFinishedPulling="2025-12-03 11:57:47.81670349 +0000 UTC m=+2258.404945254" observedRunningTime="2025-12-03 11:57:49.005957475 +0000 UTC m=+2259.594199249" watchObservedRunningTime="2025-12-03 11:57:49.007824728 +0000 UTC m=+2259.596066492" Dec 03 11:57:49 crc kubenswrapper[5052]: I1203 11:57:49.985671 5052 generic.go:334] "Generic (PLEG): container finished" podID="bac97bd6-a939-41e2-b577-05d6fb87dcb5" containerID="ecc0df50483ef40efaa5b0f8429d41c8eeeb4b9aa07d84f10b24c739cfcc7770" exitCode=0 Dec 03 11:57:49 crc kubenswrapper[5052]: I1203 11:57:49.985929 5052 generic.go:334] "Generic (PLEG): container finished" podID="bac97bd6-a939-41e2-b577-05d6fb87dcb5" containerID="b866b6d846a4e51e96dee71c028075183b02f4011789bce203172a26d1256e1b" exitCode=2 Dec 03 11:57:49 crc kubenswrapper[5052]: I1203 11:57:49.985937 5052 generic.go:334] "Generic (PLEG): container finished" podID="bac97bd6-a939-41e2-b577-05d6fb87dcb5" containerID="6348d1c5e4280b87678fd42b2fa52df5ff03d4842c5ebe3f001352fb55f604d6" exitCode=0 Dec 03 11:57:49 crc kubenswrapper[5052]: I1203 11:57:49.985948 5052 generic.go:334] "Generic (PLEG): container finished" podID="bac97bd6-a939-41e2-b577-05d6fb87dcb5" containerID="eb431f6c1818b9852af36eea4a58126f109d053175b87db72d750e6bc3098387" exitCode=0 Dec 03 11:57:49 crc kubenswrapper[5052]: I1203 11:57:49.985758 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"bac97bd6-a939-41e2-b577-05d6fb87dcb5","Type":"ContainerDied","Data":"ecc0df50483ef40efaa5b0f8429d41c8eeeb4b9aa07d84f10b24c739cfcc7770"} Dec 03 11:57:49 crc kubenswrapper[5052]: I1203 11:57:49.985979 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"bac97bd6-a939-41e2-b577-05d6fb87dcb5","Type":"ContainerDied","Data":"b866b6d846a4e51e96dee71c028075183b02f4011789bce203172a26d1256e1b"} Dec 03 11:57:49 crc kubenswrapper[5052]: I1203 11:57:49.985990 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"bac97bd6-a939-41e2-b577-05d6fb87dcb5","Type":"ContainerDied","Data":"6348d1c5e4280b87678fd42b2fa52df5ff03d4842c5ebe3f001352fb55f604d6"} Dec 03 11:57:49 crc kubenswrapper[5052]: I1203 11:57:49.985999 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"bac97bd6-a939-41e2-b577-05d6fb87dcb5","Type":"ContainerDied","Data":"eb431f6c1818b9852af36eea4a58126f109d053175b87db72d750e6bc3098387"} Dec 03 11:57:50 crc kubenswrapper[5052]: I1203 11:57:50.310368 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:57:50 crc kubenswrapper[5052]: I1203 11:57:50.391875 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/bac97bd6-a939-41e2-b577-05d6fb87dcb5-run-httpd\") pod \"bac97bd6-a939-41e2-b577-05d6fb87dcb5\" (UID: \"bac97bd6-a939-41e2-b577-05d6fb87dcb5\") " Dec 03 11:57:50 crc kubenswrapper[5052]: I1203 11:57:50.391940 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8jrcb\" (UniqueName: \"kubernetes.io/projected/bac97bd6-a939-41e2-b577-05d6fb87dcb5-kube-api-access-8jrcb\") pod \"bac97bd6-a939-41e2-b577-05d6fb87dcb5\" (UID: \"bac97bd6-a939-41e2-b577-05d6fb87dcb5\") " Dec 03 11:57:50 crc kubenswrapper[5052]: I1203 11:57:50.391967 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/bac97bd6-a939-41e2-b577-05d6fb87dcb5-combined-ca-bundle\") pod \"bac97bd6-a939-41e2-b577-05d6fb87dcb5\" (UID: \"bac97bd6-a939-41e2-b577-05d6fb87dcb5\") " Dec 03 11:57:50 crc kubenswrapper[5052]: I1203 11:57:50.391990 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/bac97bd6-a939-41e2-b577-05d6fb87dcb5-sg-core-conf-yaml\") pod \"bac97bd6-a939-41e2-b577-05d6fb87dcb5\" (UID: \"bac97bd6-a939-41e2-b577-05d6fb87dcb5\") " Dec 03 11:57:50 crc kubenswrapper[5052]: I1203 11:57:50.392028 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/bac97bd6-a939-41e2-b577-05d6fb87dcb5-config-data\") pod \"bac97bd6-a939-41e2-b577-05d6fb87dcb5\" (UID: \"bac97bd6-a939-41e2-b577-05d6fb87dcb5\") " Dec 03 11:57:50 crc kubenswrapper[5052]: I1203 11:57:50.392099 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/bac97bd6-a939-41e2-b577-05d6fb87dcb5-scripts\") pod \"bac97bd6-a939-41e2-b577-05d6fb87dcb5\" (UID: \"bac97bd6-a939-41e2-b577-05d6fb87dcb5\") " Dec 03 11:57:50 crc kubenswrapper[5052]: I1203 11:57:50.392157 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/bac97bd6-a939-41e2-b577-05d6fb87dcb5-log-httpd\") pod \"bac97bd6-a939-41e2-b577-05d6fb87dcb5\" (UID: \"bac97bd6-a939-41e2-b577-05d6fb87dcb5\") " Dec 03 11:57:50 crc kubenswrapper[5052]: I1203 11:57:50.392328 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bac97bd6-a939-41e2-b577-05d6fb87dcb5-run-httpd" (OuterVolumeSpecName: "run-httpd") pod "bac97bd6-a939-41e2-b577-05d6fb87dcb5" (UID: "bac97bd6-a939-41e2-b577-05d6fb87dcb5"). InnerVolumeSpecName "run-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:57:50 crc kubenswrapper[5052]: I1203 11:57:50.392729 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bac97bd6-a939-41e2-b577-05d6fb87dcb5-log-httpd" (OuterVolumeSpecName: "log-httpd") pod "bac97bd6-a939-41e2-b577-05d6fb87dcb5" (UID: "bac97bd6-a939-41e2-b577-05d6fb87dcb5"). InnerVolumeSpecName "log-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:57:50 crc kubenswrapper[5052]: I1203 11:57:50.392815 5052 reconciler_common.go:293] "Volume detached for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/bac97bd6-a939-41e2-b577-05d6fb87dcb5-run-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 11:57:50 crc kubenswrapper[5052]: I1203 11:57:50.398863 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bac97bd6-a939-41e2-b577-05d6fb87dcb5-kube-api-access-8jrcb" (OuterVolumeSpecName: "kube-api-access-8jrcb") pod "bac97bd6-a939-41e2-b577-05d6fb87dcb5" (UID: "bac97bd6-a939-41e2-b577-05d6fb87dcb5"). InnerVolumeSpecName "kube-api-access-8jrcb". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:57:50 crc kubenswrapper[5052]: I1203 11:57:50.398981 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bac97bd6-a939-41e2-b577-05d6fb87dcb5-scripts" (OuterVolumeSpecName: "scripts") pod "bac97bd6-a939-41e2-b577-05d6fb87dcb5" (UID: "bac97bd6-a939-41e2-b577-05d6fb87dcb5"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:57:50 crc kubenswrapper[5052]: I1203 11:57:50.420264 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bac97bd6-a939-41e2-b577-05d6fb87dcb5-sg-core-conf-yaml" (OuterVolumeSpecName: "sg-core-conf-yaml") pod "bac97bd6-a939-41e2-b577-05d6fb87dcb5" (UID: "bac97bd6-a939-41e2-b577-05d6fb87dcb5"). InnerVolumeSpecName "sg-core-conf-yaml". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:57:50 crc kubenswrapper[5052]: I1203 11:57:50.480093 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bac97bd6-a939-41e2-b577-05d6fb87dcb5-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "bac97bd6-a939-41e2-b577-05d6fb87dcb5" (UID: "bac97bd6-a939-41e2-b577-05d6fb87dcb5"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:57:50 crc kubenswrapper[5052]: I1203 11:57:50.489561 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bac97bd6-a939-41e2-b577-05d6fb87dcb5-config-data" (OuterVolumeSpecName: "config-data") pod "bac97bd6-a939-41e2-b577-05d6fb87dcb5" (UID: "bac97bd6-a939-41e2-b577-05d6fb87dcb5"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:57:50 crc kubenswrapper[5052]: I1203 11:57:50.494624 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-8jrcb\" (UniqueName: \"kubernetes.io/projected/bac97bd6-a939-41e2-b577-05d6fb87dcb5-kube-api-access-8jrcb\") on node \"crc\" DevicePath \"\"" Dec 03 11:57:50 crc kubenswrapper[5052]: I1203 11:57:50.494664 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/bac97bd6-a939-41e2-b577-05d6fb87dcb5-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:57:50 crc kubenswrapper[5052]: I1203 11:57:50.494676 5052 reconciler_common.go:293] "Volume detached for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/bac97bd6-a939-41e2-b577-05d6fb87dcb5-sg-core-conf-yaml\") on node \"crc\" DevicePath \"\"" Dec 03 11:57:50 crc kubenswrapper[5052]: I1203 11:57:50.494689 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/bac97bd6-a939-41e2-b577-05d6fb87dcb5-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:57:50 crc kubenswrapper[5052]: I1203 11:57:50.494700 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/bac97bd6-a939-41e2-b577-05d6fb87dcb5-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:57:50 crc kubenswrapper[5052]: I1203 11:57:50.494710 5052 reconciler_common.go:293] "Volume detached for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/bac97bd6-a939-41e2-b577-05d6fb87dcb5-log-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 11:57:50 crc kubenswrapper[5052]: I1203 11:57:50.999762 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/prometheus-metric-storage-0" event={"ID":"01bc82de-61dc-477e-b4cf-e3b0d32c99ba","Type":"ContainerStarted","Data":"b2b8f2e1a2b0959c9d345484247345b165016e4ab99777de279ec024458c9da1"} Dec 03 11:57:50 crc kubenswrapper[5052]: I1203 11:57:50.999810 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/prometheus-metric-storage-0" event={"ID":"01bc82de-61dc-477e-b4cf-e3b0d32c99ba","Type":"ContainerStarted","Data":"3845a093071f0f8ca351aba36d4c3c56dd2d469ba1bfbea7b327c64e55ea871c"} Dec 03 11:57:51 crc kubenswrapper[5052]: I1203 11:57:51.003233 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"bac97bd6-a939-41e2-b577-05d6fb87dcb5","Type":"ContainerDied","Data":"82e14a469f5c2e516506acdec3d3743f7e118069caea0f770e4955fe7d6047ad"} Dec 03 11:57:51 crc kubenswrapper[5052]: I1203 11:57:51.003285 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:57:51 crc kubenswrapper[5052]: I1203 11:57:51.003295 5052 scope.go:117] "RemoveContainer" containerID="ecc0df50483ef40efaa5b0f8429d41c8eeeb4b9aa07d84f10b24c739cfcc7770" Dec 03 11:57:51 crc kubenswrapper[5052]: I1203 11:57:51.029773 5052 scope.go:117] "RemoveContainer" containerID="b866b6d846a4e51e96dee71c028075183b02f4011789bce203172a26d1256e1b" Dec 03 11:57:51 crc kubenswrapper[5052]: I1203 11:57:51.079166 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/prometheus-metric-storage-0" podStartSLOduration=25.079143677 podStartE2EDuration="25.079143677s" podCreationTimestamp="2025-12-03 11:57:26 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:57:51.062616564 +0000 UTC m=+2261.650858348" watchObservedRunningTime="2025-12-03 11:57:51.079143677 +0000 UTC m=+2261.667385451" Dec 03 11:57:51 crc kubenswrapper[5052]: I1203 11:57:51.082550 5052 scope.go:117] "RemoveContainer" containerID="6348d1c5e4280b87678fd42b2fa52df5ff03d4842c5ebe3f001352fb55f604d6" Dec 03 11:57:51 crc kubenswrapper[5052]: I1203 11:57:51.104655 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:57:51 crc kubenswrapper[5052]: I1203 11:57:51.114047 5052 scope.go:117] "RemoveContainer" containerID="eb431f6c1818b9852af36eea4a58126f109d053175b87db72d750e6bc3098387" Dec 03 11:57:51 crc kubenswrapper[5052]: I1203 11:57:51.114993 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:57:51 crc kubenswrapper[5052]: I1203 11:57:51.133313 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:57:51 crc kubenswrapper[5052]: E1203 11:57:51.133790 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="bac97bd6-a939-41e2-b577-05d6fb87dcb5" containerName="ceilometer-notification-agent" Dec 03 11:57:51 crc kubenswrapper[5052]: I1203 11:57:51.133810 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="bac97bd6-a939-41e2-b577-05d6fb87dcb5" containerName="ceilometer-notification-agent" Dec 03 11:57:51 crc kubenswrapper[5052]: E1203 11:57:51.133846 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="bac97bd6-a939-41e2-b577-05d6fb87dcb5" containerName="sg-core" Dec 03 11:57:51 crc kubenswrapper[5052]: I1203 11:57:51.133853 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="bac97bd6-a939-41e2-b577-05d6fb87dcb5" containerName="sg-core" Dec 03 11:57:51 crc kubenswrapper[5052]: E1203 11:57:51.133866 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="bac97bd6-a939-41e2-b577-05d6fb87dcb5" containerName="proxy-httpd" Dec 03 11:57:51 crc kubenswrapper[5052]: I1203 11:57:51.133872 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="bac97bd6-a939-41e2-b577-05d6fb87dcb5" containerName="proxy-httpd" Dec 03 11:57:51 crc kubenswrapper[5052]: E1203 11:57:51.133884 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="bac97bd6-a939-41e2-b577-05d6fb87dcb5" containerName="ceilometer-central-agent" Dec 03 11:57:51 crc kubenswrapper[5052]: I1203 11:57:51.133889 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="bac97bd6-a939-41e2-b577-05d6fb87dcb5" containerName="ceilometer-central-agent" Dec 03 11:57:51 crc kubenswrapper[5052]: I1203 11:57:51.134045 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="bac97bd6-a939-41e2-b577-05d6fb87dcb5" containerName="ceilometer-notification-agent" Dec 03 11:57:51 crc kubenswrapper[5052]: I1203 11:57:51.134063 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="bac97bd6-a939-41e2-b577-05d6fb87dcb5" containerName="proxy-httpd" Dec 03 11:57:51 crc kubenswrapper[5052]: I1203 11:57:51.134079 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="bac97bd6-a939-41e2-b577-05d6fb87dcb5" containerName="ceilometer-central-agent" Dec 03 11:57:51 crc kubenswrapper[5052]: I1203 11:57:51.134088 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="bac97bd6-a939-41e2-b577-05d6fb87dcb5" containerName="sg-core" Dec 03 11:57:51 crc kubenswrapper[5052]: I1203 11:57:51.135633 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:57:51 crc kubenswrapper[5052]: I1203 11:57:51.141196 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-config-data" Dec 03 11:57:51 crc kubenswrapper[5052]: I1203 11:57:51.141397 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-scripts" Dec 03 11:57:51 crc kubenswrapper[5052]: I1203 11:57:51.146600 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:57:51 crc kubenswrapper[5052]: I1203 11:57:51.208297 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/7724320e-d88f-40eb-8200-3732d97944fb-scripts\") pod \"ceilometer-0\" (UID: \"7724320e-d88f-40eb-8200-3732d97944fb\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:57:51 crc kubenswrapper[5052]: I1203 11:57:51.208346 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/7724320e-d88f-40eb-8200-3732d97944fb-log-httpd\") pod \"ceilometer-0\" (UID: \"7724320e-d88f-40eb-8200-3732d97944fb\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:57:51 crc kubenswrapper[5052]: I1203 11:57:51.208396 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/7724320e-d88f-40eb-8200-3732d97944fb-config-data\") pod \"ceilometer-0\" (UID: \"7724320e-d88f-40eb-8200-3732d97944fb\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:57:51 crc kubenswrapper[5052]: I1203 11:57:51.208420 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/7724320e-d88f-40eb-8200-3732d97944fb-run-httpd\") pod \"ceilometer-0\" (UID: \"7724320e-d88f-40eb-8200-3732d97944fb\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:57:51 crc kubenswrapper[5052]: I1203 11:57:51.208443 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pg7v6\" (UniqueName: \"kubernetes.io/projected/7724320e-d88f-40eb-8200-3732d97944fb-kube-api-access-pg7v6\") pod \"ceilometer-0\" (UID: \"7724320e-d88f-40eb-8200-3732d97944fb\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:57:51 crc kubenswrapper[5052]: I1203 11:57:51.208643 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/7724320e-d88f-40eb-8200-3732d97944fb-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"7724320e-d88f-40eb-8200-3732d97944fb\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:57:51 crc kubenswrapper[5052]: I1203 11:57:51.208719 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/7724320e-d88f-40eb-8200-3732d97944fb-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"7724320e-d88f-40eb-8200-3732d97944fb\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:57:51 crc kubenswrapper[5052]: I1203 11:57:51.310552 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/7724320e-d88f-40eb-8200-3732d97944fb-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"7724320e-d88f-40eb-8200-3732d97944fb\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:57:51 crc kubenswrapper[5052]: I1203 11:57:51.310679 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/7724320e-d88f-40eb-8200-3732d97944fb-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"7724320e-d88f-40eb-8200-3732d97944fb\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:57:51 crc kubenswrapper[5052]: I1203 11:57:51.310727 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/7724320e-d88f-40eb-8200-3732d97944fb-scripts\") pod \"ceilometer-0\" (UID: \"7724320e-d88f-40eb-8200-3732d97944fb\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:57:51 crc kubenswrapper[5052]: I1203 11:57:51.310759 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/7724320e-d88f-40eb-8200-3732d97944fb-log-httpd\") pod \"ceilometer-0\" (UID: \"7724320e-d88f-40eb-8200-3732d97944fb\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:57:51 crc kubenswrapper[5052]: I1203 11:57:51.310818 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/7724320e-d88f-40eb-8200-3732d97944fb-config-data\") pod \"ceilometer-0\" (UID: \"7724320e-d88f-40eb-8200-3732d97944fb\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:57:51 crc kubenswrapper[5052]: I1203 11:57:51.310851 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/7724320e-d88f-40eb-8200-3732d97944fb-run-httpd\") pod \"ceilometer-0\" (UID: \"7724320e-d88f-40eb-8200-3732d97944fb\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:57:51 crc kubenswrapper[5052]: I1203 11:57:51.310909 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-pg7v6\" (UniqueName: \"kubernetes.io/projected/7724320e-d88f-40eb-8200-3732d97944fb-kube-api-access-pg7v6\") pod \"ceilometer-0\" (UID: \"7724320e-d88f-40eb-8200-3732d97944fb\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:57:51 crc kubenswrapper[5052]: I1203 11:57:51.311398 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/7724320e-d88f-40eb-8200-3732d97944fb-run-httpd\") pod \"ceilometer-0\" (UID: \"7724320e-d88f-40eb-8200-3732d97944fb\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:57:51 crc kubenswrapper[5052]: I1203 11:57:51.311822 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/7724320e-d88f-40eb-8200-3732d97944fb-log-httpd\") pod \"ceilometer-0\" (UID: \"7724320e-d88f-40eb-8200-3732d97944fb\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:57:51 crc kubenswrapper[5052]: I1203 11:57:51.315849 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/7724320e-d88f-40eb-8200-3732d97944fb-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"7724320e-d88f-40eb-8200-3732d97944fb\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:57:51 crc kubenswrapper[5052]: I1203 11:57:51.316806 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/7724320e-d88f-40eb-8200-3732d97944fb-scripts\") pod \"ceilometer-0\" (UID: \"7724320e-d88f-40eb-8200-3732d97944fb\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:57:51 crc kubenswrapper[5052]: I1203 11:57:51.316994 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/7724320e-d88f-40eb-8200-3732d97944fb-config-data\") pod \"ceilometer-0\" (UID: \"7724320e-d88f-40eb-8200-3732d97944fb\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:57:51 crc kubenswrapper[5052]: I1203 11:57:51.318446 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/7724320e-d88f-40eb-8200-3732d97944fb-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"7724320e-d88f-40eb-8200-3732d97944fb\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:57:51 crc kubenswrapper[5052]: I1203 11:57:51.328237 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-pg7v6\" (UniqueName: \"kubernetes.io/projected/7724320e-d88f-40eb-8200-3732d97944fb-kube-api-access-pg7v6\") pod \"ceilometer-0\" (UID: \"7724320e-d88f-40eb-8200-3732d97944fb\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:57:51 crc kubenswrapper[5052]: I1203 11:57:51.504637 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:57:51 crc kubenswrapper[5052]: I1203 11:57:51.684182 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bac97bd6-a939-41e2-b577-05d6fb87dcb5" path="/var/lib/kubelet/pods/bac97bd6-a939-41e2-b577-05d6fb87dcb5/volumes" Dec 03 11:57:52 crc kubenswrapper[5052]: I1203 11:57:52.018216 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/prometheus-metric-storage-0" Dec 03 11:57:52 crc kubenswrapper[5052]: I1203 11:57:52.059918 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:57:53 crc kubenswrapper[5052]: I1203 11:57:53.043348 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"7724320e-d88f-40eb-8200-3732d97944fb","Type":"ContainerStarted","Data":"80c5885e88721179b4914941e8f2b1fd1771f258e1787f873289a0db7a3e98d8"} Dec 03 11:57:53 crc kubenswrapper[5052]: I1203 11:57:53.043694 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"7724320e-d88f-40eb-8200-3732d97944fb","Type":"ContainerStarted","Data":"24684a3db7378d1ddf53fbf9b0b30b8cd80c753998289e7f290b173da1226c64"} Dec 03 11:57:54 crc kubenswrapper[5052]: I1203 11:57:54.055433 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"7724320e-d88f-40eb-8200-3732d97944fb","Type":"ContainerStarted","Data":"a5a9a0fc1f9628172fb97240315512de6c963cb035df3695470c5c34431a3c9e"} Dec 03 11:57:54 crc kubenswrapper[5052]: I1203 11:57:54.057443 5052 generic.go:334] "Generic (PLEG): container finished" podID="b70c4b01-42e4-4990-be48-a2d762e66cd7" containerID="3e630cf9c031c51d9e57dbf5f4f68a125ad4930a9ae93db53fda22a443e086f2" exitCode=0 Dec 03 11:57:54 crc kubenswrapper[5052]: I1203 11:57:54.057483 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-bootstrap-kdr6v" event={"ID":"b70c4b01-42e4-4990-be48-a2d762e66cd7","Type":"ContainerDied","Data":"3e630cf9c031c51d9e57dbf5f4f68a125ad4930a9ae93db53fda22a443e086f2"} Dec 03 11:57:54 crc kubenswrapper[5052]: I1203 11:57:54.185894 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:57:54 crc kubenswrapper[5052]: I1203 11:57:54.185960 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:57:54 crc kubenswrapper[5052]: I1203 11:57:54.218304 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:57:54 crc kubenswrapper[5052]: I1203 11:57:54.230457 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:57:55 crc kubenswrapper[5052]: I1203 11:57:55.068223 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"7724320e-d88f-40eb-8200-3732d97944fb","Type":"ContainerStarted","Data":"62d906ae5951f750faecdac5f48bd2e3f15dd18ff4b65d6d9234602103045354"} Dec 03 11:57:55 crc kubenswrapper[5052]: I1203 11:57:55.073610 5052 generic.go:334] "Generic (PLEG): container finished" podID="b3146921-cc78-401f-a918-fa8b91814f10" containerID="cfcf65482c408d6744121d3337ff364f65dfd4df7db4e0d1ca7a10b2ec5a6849" exitCode=0 Dec 03 11:57:55 crc kubenswrapper[5052]: I1203 11:57:55.073724 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/watcher-db-sync-chlm4" event={"ID":"b3146921-cc78-401f-a918-fa8b91814f10","Type":"ContainerDied","Data":"cfcf65482c408d6744121d3337ff364f65dfd4df7db4e0d1ca7a10b2ec5a6849"} Dec 03 11:57:55 crc kubenswrapper[5052]: I1203 11:57:55.078289 5052 generic.go:334] "Generic (PLEG): container finished" podID="14cb8e46-418d-4c42-ae97-a384a05b432f" containerID="56106c72b8ee52c2322893ce9cab31cbca3baecdb6d70605f34d8b51f2c8f37b" exitCode=0 Dec 03 11:57:55 crc kubenswrapper[5052]: I1203 11:57:55.078338 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-db-sync-8tdfb" event={"ID":"14cb8e46-418d-4c42-ae97-a384a05b432f","Type":"ContainerDied","Data":"56106c72b8ee52c2322893ce9cab31cbca3baecdb6d70605f34d8b51f2c8f37b"} Dec 03 11:57:55 crc kubenswrapper[5052]: I1203 11:57:55.078786 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:57:55 crc kubenswrapper[5052]: I1203 11:57:55.079109 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:57:55 crc kubenswrapper[5052]: I1203 11:57:55.430606 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-bootstrap-kdr6v" Dec 03 11:57:55 crc kubenswrapper[5052]: I1203 11:57:55.551933 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/b70c4b01-42e4-4990-be48-a2d762e66cd7-config-data\") pod \"b70c4b01-42e4-4990-be48-a2d762e66cd7\" (UID: \"b70c4b01-42e4-4990-be48-a2d762e66cd7\") " Dec 03 11:57:55 crc kubenswrapper[5052]: I1203 11:57:55.552055 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9gqn4\" (UniqueName: \"kubernetes.io/projected/b70c4b01-42e4-4990-be48-a2d762e66cd7-kube-api-access-9gqn4\") pod \"b70c4b01-42e4-4990-be48-a2d762e66cd7\" (UID: \"b70c4b01-42e4-4990-be48-a2d762e66cd7\") " Dec 03 11:57:55 crc kubenswrapper[5052]: I1203 11:57:55.552131 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b70c4b01-42e4-4990-be48-a2d762e66cd7-combined-ca-bundle\") pod \"b70c4b01-42e4-4990-be48-a2d762e66cd7\" (UID: \"b70c4b01-42e4-4990-be48-a2d762e66cd7\") " Dec 03 11:57:55 crc kubenswrapper[5052]: I1203 11:57:55.552166 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/b70c4b01-42e4-4990-be48-a2d762e66cd7-credential-keys\") pod \"b70c4b01-42e4-4990-be48-a2d762e66cd7\" (UID: \"b70c4b01-42e4-4990-be48-a2d762e66cd7\") " Dec 03 11:57:55 crc kubenswrapper[5052]: I1203 11:57:55.552200 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/b70c4b01-42e4-4990-be48-a2d762e66cd7-scripts\") pod \"b70c4b01-42e4-4990-be48-a2d762e66cd7\" (UID: \"b70c4b01-42e4-4990-be48-a2d762e66cd7\") " Dec 03 11:57:55 crc kubenswrapper[5052]: I1203 11:57:55.552303 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/b70c4b01-42e4-4990-be48-a2d762e66cd7-fernet-keys\") pod \"b70c4b01-42e4-4990-be48-a2d762e66cd7\" (UID: \"b70c4b01-42e4-4990-be48-a2d762e66cd7\") " Dec 03 11:57:55 crc kubenswrapper[5052]: I1203 11:57:55.558936 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b70c4b01-42e4-4990-be48-a2d762e66cd7-credential-keys" (OuterVolumeSpecName: "credential-keys") pod "b70c4b01-42e4-4990-be48-a2d762e66cd7" (UID: "b70c4b01-42e4-4990-be48-a2d762e66cd7"). InnerVolumeSpecName "credential-keys". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:57:55 crc kubenswrapper[5052]: I1203 11:57:55.560013 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b70c4b01-42e4-4990-be48-a2d762e66cd7-scripts" (OuterVolumeSpecName: "scripts") pod "b70c4b01-42e4-4990-be48-a2d762e66cd7" (UID: "b70c4b01-42e4-4990-be48-a2d762e66cd7"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:57:55 crc kubenswrapper[5052]: I1203 11:57:55.570378 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b70c4b01-42e4-4990-be48-a2d762e66cd7-fernet-keys" (OuterVolumeSpecName: "fernet-keys") pod "b70c4b01-42e4-4990-be48-a2d762e66cd7" (UID: "b70c4b01-42e4-4990-be48-a2d762e66cd7"). InnerVolumeSpecName "fernet-keys". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:57:55 crc kubenswrapper[5052]: I1203 11:57:55.576665 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b70c4b01-42e4-4990-be48-a2d762e66cd7-kube-api-access-9gqn4" (OuterVolumeSpecName: "kube-api-access-9gqn4") pod "b70c4b01-42e4-4990-be48-a2d762e66cd7" (UID: "b70c4b01-42e4-4990-be48-a2d762e66cd7"). InnerVolumeSpecName "kube-api-access-9gqn4". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:57:55 crc kubenswrapper[5052]: I1203 11:57:55.587745 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b70c4b01-42e4-4990-be48-a2d762e66cd7-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "b70c4b01-42e4-4990-be48-a2d762e66cd7" (UID: "b70c4b01-42e4-4990-be48-a2d762e66cd7"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:57:55 crc kubenswrapper[5052]: I1203 11:57:55.588713 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b70c4b01-42e4-4990-be48-a2d762e66cd7-config-data" (OuterVolumeSpecName: "config-data") pod "b70c4b01-42e4-4990-be48-a2d762e66cd7" (UID: "b70c4b01-42e4-4990-be48-a2d762e66cd7"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:57:55 crc kubenswrapper[5052]: I1203 11:57:55.654542 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/b70c4b01-42e4-4990-be48-a2d762e66cd7-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:57:55 crc kubenswrapper[5052]: I1203 11:57:55.654598 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-9gqn4\" (UniqueName: \"kubernetes.io/projected/b70c4b01-42e4-4990-be48-a2d762e66cd7-kube-api-access-9gqn4\") on node \"crc\" DevicePath \"\"" Dec 03 11:57:55 crc kubenswrapper[5052]: I1203 11:57:55.654611 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b70c4b01-42e4-4990-be48-a2d762e66cd7-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:57:55 crc kubenswrapper[5052]: I1203 11:57:55.654620 5052 reconciler_common.go:293] "Volume detached for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/b70c4b01-42e4-4990-be48-a2d762e66cd7-credential-keys\") on node \"crc\" DevicePath \"\"" Dec 03 11:57:55 crc kubenswrapper[5052]: I1203 11:57:55.654628 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/b70c4b01-42e4-4990-be48-a2d762e66cd7-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:57:55 crc kubenswrapper[5052]: I1203 11:57:55.654637 5052 reconciler_common.go:293] "Volume detached for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/b70c4b01-42e4-4990-be48-a2d762e66cd7-fernet-keys\") on node \"crc\" DevicePath \"\"" Dec 03 11:57:56 crc kubenswrapper[5052]: I1203 11:57:56.088889 5052 generic.go:334] "Generic (PLEG): container finished" podID="a50cd1a3-6eb7-4712-bb60-648250b396d6" containerID="37a42c9d3db76f0dfccd0a96b80c798307fcfb938d7f490f56ab9c77c423a50f" exitCode=0 Dec 03 11:57:56 crc kubenswrapper[5052]: I1203 11:57:56.088960 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-db-sync-xs2lk" event={"ID":"a50cd1a3-6eb7-4712-bb60-648250b396d6","Type":"ContainerDied","Data":"37a42c9d3db76f0dfccd0a96b80c798307fcfb938d7f490f56ab9c77c423a50f"} Dec 03 11:57:56 crc kubenswrapper[5052]: I1203 11:57:56.091091 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-bootstrap-kdr6v" event={"ID":"b70c4b01-42e4-4990-be48-a2d762e66cd7","Type":"ContainerDied","Data":"4cc5bcf5e3eae342e3769376a8d544310344b7c571bd4994114a57bc6c9f2fcc"} Dec 03 11:57:56 crc kubenswrapper[5052]: I1203 11:57:56.091124 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4cc5bcf5e3eae342e3769376a8d544310344b7c571bd4994114a57bc6c9f2fcc" Dec 03 11:57:56 crc kubenswrapper[5052]: I1203 11:57:56.091124 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-bootstrap-kdr6v" Dec 03 11:57:56 crc kubenswrapper[5052]: I1203 11:57:56.093925 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"7724320e-d88f-40eb-8200-3732d97944fb","Type":"ContainerStarted","Data":"d3f7788549fe5083a4ffcb34e6dd06fab76a2e6bd9a093c7c5982ff1fb50728b"} Dec 03 11:57:56 crc kubenswrapper[5052]: I1203 11:57:56.236183 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/ceilometer-0" podStartSLOduration=1.533629893 podStartE2EDuration="5.236157924s" podCreationTimestamp="2025-12-03 11:57:51 +0000 UTC" firstStartedPulling="2025-12-03 11:57:52.065043042 +0000 UTC m=+2262.653284806" lastFinishedPulling="2025-12-03 11:57:55.767571083 +0000 UTC m=+2266.355812837" observedRunningTime="2025-12-03 11:57:56.189009184 +0000 UTC m=+2266.777250948" watchObservedRunningTime="2025-12-03 11:57:56.236157924 +0000 UTC m=+2266.824399688" Dec 03 11:57:56 crc kubenswrapper[5052]: I1203 11:57:56.241764 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/keystone-bootstrap-kdr6v"] Dec 03 11:57:56 crc kubenswrapper[5052]: I1203 11:57:56.250593 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/keystone-bootstrap-kdr6v"] Dec 03 11:57:56 crc kubenswrapper[5052]: I1203 11:57:56.432442 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/keystone-bootstrap-jhbzq"] Dec 03 11:57:56 crc kubenswrapper[5052]: E1203 11:57:56.433226 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="b70c4b01-42e4-4990-be48-a2d762e66cd7" containerName="keystone-bootstrap" Dec 03 11:57:56 crc kubenswrapper[5052]: I1203 11:57:56.433243 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="b70c4b01-42e4-4990-be48-a2d762e66cd7" containerName="keystone-bootstrap" Dec 03 11:57:56 crc kubenswrapper[5052]: I1203 11:57:56.433542 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="b70c4b01-42e4-4990-be48-a2d762e66cd7" containerName="keystone-bootstrap" Dec 03 11:57:56 crc kubenswrapper[5052]: I1203 11:57:56.435225 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-bootstrap-jhbzq" Dec 03 11:57:56 crc kubenswrapper[5052]: I1203 11:57:56.447056 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"osp-secret" Dec 03 11:57:56 crc kubenswrapper[5052]: I1203 11:57:56.447354 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone-keystone-dockercfg-csztk" Dec 03 11:57:56 crc kubenswrapper[5052]: I1203 11:57:56.447662 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone-config-data" Dec 03 11:57:56 crc kubenswrapper[5052]: I1203 11:57:56.447800 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone-scripts" Dec 03 11:57:56 crc kubenswrapper[5052]: I1203 11:57:56.447927 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone" Dec 03 11:57:56 crc kubenswrapper[5052]: I1203 11:57:56.471705 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-db-sync-8tdfb" Dec 03 11:57:56 crc kubenswrapper[5052]: I1203 11:57:56.489274 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-bootstrap-jhbzq"] Dec 03 11:57:56 crc kubenswrapper[5052]: I1203 11:57:56.575400 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/14cb8e46-418d-4c42-ae97-a384a05b432f-combined-ca-bundle\") pod \"14cb8e46-418d-4c42-ae97-a384a05b432f\" (UID: \"14cb8e46-418d-4c42-ae97-a384a05b432f\") " Dec 03 11:57:56 crc kubenswrapper[5052]: I1203 11:57:56.575609 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5x5ll\" (UniqueName: \"kubernetes.io/projected/14cb8e46-418d-4c42-ae97-a384a05b432f-kube-api-access-5x5ll\") pod \"14cb8e46-418d-4c42-ae97-a384a05b432f\" (UID: \"14cb8e46-418d-4c42-ae97-a384a05b432f\") " Dec 03 11:57:56 crc kubenswrapper[5052]: I1203 11:57:56.575711 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/14cb8e46-418d-4c42-ae97-a384a05b432f-config-data\") pod \"14cb8e46-418d-4c42-ae97-a384a05b432f\" (UID: \"14cb8e46-418d-4c42-ae97-a384a05b432f\") " Dec 03 11:57:56 crc kubenswrapper[5052]: I1203 11:57:56.576732 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/14cb8e46-418d-4c42-ae97-a384a05b432f-scripts\") pod \"14cb8e46-418d-4c42-ae97-a384a05b432f\" (UID: \"14cb8e46-418d-4c42-ae97-a384a05b432f\") " Dec 03 11:57:56 crc kubenswrapper[5052]: I1203 11:57:56.576790 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/14cb8e46-418d-4c42-ae97-a384a05b432f-logs\") pod \"14cb8e46-418d-4c42-ae97-a384a05b432f\" (UID: \"14cb8e46-418d-4c42-ae97-a384a05b432f\") " Dec 03 11:57:56 crc kubenswrapper[5052]: I1203 11:57:56.577056 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/c856f6ae-8216-4bd5-a9dd-1d87b546854e-config-data\") pod \"keystone-bootstrap-jhbzq\" (UID: \"c856f6ae-8216-4bd5-a9dd-1d87b546854e\") " pod="openstack-kuttl-tests/keystone-bootstrap-jhbzq" Dec 03 11:57:56 crc kubenswrapper[5052]: I1203 11:57:56.577105 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/c856f6ae-8216-4bd5-a9dd-1d87b546854e-credential-keys\") pod \"keystone-bootstrap-jhbzq\" (UID: \"c856f6ae-8216-4bd5-a9dd-1d87b546854e\") " pod="openstack-kuttl-tests/keystone-bootstrap-jhbzq" Dec 03 11:57:56 crc kubenswrapper[5052]: I1203 11:57:56.577138 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/c856f6ae-8216-4bd5-a9dd-1d87b546854e-fernet-keys\") pod \"keystone-bootstrap-jhbzq\" (UID: \"c856f6ae-8216-4bd5-a9dd-1d87b546854e\") " pod="openstack-kuttl-tests/keystone-bootstrap-jhbzq" Dec 03 11:57:56 crc kubenswrapper[5052]: I1203 11:57:56.577166 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v65wg\" (UniqueName: \"kubernetes.io/projected/c856f6ae-8216-4bd5-a9dd-1d87b546854e-kube-api-access-v65wg\") pod \"keystone-bootstrap-jhbzq\" (UID: \"c856f6ae-8216-4bd5-a9dd-1d87b546854e\") " pod="openstack-kuttl-tests/keystone-bootstrap-jhbzq" Dec 03 11:57:56 crc kubenswrapper[5052]: I1203 11:57:56.577231 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/c856f6ae-8216-4bd5-a9dd-1d87b546854e-scripts\") pod \"keystone-bootstrap-jhbzq\" (UID: \"c856f6ae-8216-4bd5-a9dd-1d87b546854e\") " pod="openstack-kuttl-tests/keystone-bootstrap-jhbzq" Dec 03 11:57:56 crc kubenswrapper[5052]: I1203 11:57:56.577266 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c856f6ae-8216-4bd5-a9dd-1d87b546854e-combined-ca-bundle\") pod \"keystone-bootstrap-jhbzq\" (UID: \"c856f6ae-8216-4bd5-a9dd-1d87b546854e\") " pod="openstack-kuttl-tests/keystone-bootstrap-jhbzq" Dec 03 11:57:56 crc kubenswrapper[5052]: I1203 11:57:56.578576 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/14cb8e46-418d-4c42-ae97-a384a05b432f-logs" (OuterVolumeSpecName: "logs") pod "14cb8e46-418d-4c42-ae97-a384a05b432f" (UID: "14cb8e46-418d-4c42-ae97-a384a05b432f"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:57:56 crc kubenswrapper[5052]: I1203 11:57:56.582972 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/14cb8e46-418d-4c42-ae97-a384a05b432f-kube-api-access-5x5ll" (OuterVolumeSpecName: "kube-api-access-5x5ll") pod "14cb8e46-418d-4c42-ae97-a384a05b432f" (UID: "14cb8e46-418d-4c42-ae97-a384a05b432f"). InnerVolumeSpecName "kube-api-access-5x5ll". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:57:56 crc kubenswrapper[5052]: I1203 11:57:56.583970 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/14cb8e46-418d-4c42-ae97-a384a05b432f-scripts" (OuterVolumeSpecName: "scripts") pod "14cb8e46-418d-4c42-ae97-a384a05b432f" (UID: "14cb8e46-418d-4c42-ae97-a384a05b432f"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:57:56 crc kubenswrapper[5052]: I1203 11:57:56.622633 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/14cb8e46-418d-4c42-ae97-a384a05b432f-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "14cb8e46-418d-4c42-ae97-a384a05b432f" (UID: "14cb8e46-418d-4c42-ae97-a384a05b432f"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:57:56 crc kubenswrapper[5052]: I1203 11:57:56.679882 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/c856f6ae-8216-4bd5-a9dd-1d87b546854e-config-data\") pod \"keystone-bootstrap-jhbzq\" (UID: \"c856f6ae-8216-4bd5-a9dd-1d87b546854e\") " pod="openstack-kuttl-tests/keystone-bootstrap-jhbzq" Dec 03 11:57:56 crc kubenswrapper[5052]: I1203 11:57:56.679958 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/c856f6ae-8216-4bd5-a9dd-1d87b546854e-credential-keys\") pod \"keystone-bootstrap-jhbzq\" (UID: \"c856f6ae-8216-4bd5-a9dd-1d87b546854e\") " pod="openstack-kuttl-tests/keystone-bootstrap-jhbzq" Dec 03 11:57:56 crc kubenswrapper[5052]: I1203 11:57:56.679987 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/c856f6ae-8216-4bd5-a9dd-1d87b546854e-fernet-keys\") pod \"keystone-bootstrap-jhbzq\" (UID: \"c856f6ae-8216-4bd5-a9dd-1d87b546854e\") " pod="openstack-kuttl-tests/keystone-bootstrap-jhbzq" Dec 03 11:57:56 crc kubenswrapper[5052]: I1203 11:57:56.680027 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-v65wg\" (UniqueName: \"kubernetes.io/projected/c856f6ae-8216-4bd5-a9dd-1d87b546854e-kube-api-access-v65wg\") pod \"keystone-bootstrap-jhbzq\" (UID: \"c856f6ae-8216-4bd5-a9dd-1d87b546854e\") " pod="openstack-kuttl-tests/keystone-bootstrap-jhbzq" Dec 03 11:57:56 crc kubenswrapper[5052]: I1203 11:57:56.680141 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/c856f6ae-8216-4bd5-a9dd-1d87b546854e-scripts\") pod \"keystone-bootstrap-jhbzq\" (UID: \"c856f6ae-8216-4bd5-a9dd-1d87b546854e\") " pod="openstack-kuttl-tests/keystone-bootstrap-jhbzq" Dec 03 11:57:56 crc kubenswrapper[5052]: I1203 11:57:56.680196 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c856f6ae-8216-4bd5-a9dd-1d87b546854e-combined-ca-bundle\") pod \"keystone-bootstrap-jhbzq\" (UID: \"c856f6ae-8216-4bd5-a9dd-1d87b546854e\") " pod="openstack-kuttl-tests/keystone-bootstrap-jhbzq" Dec 03 11:57:56 crc kubenswrapper[5052]: I1203 11:57:56.680381 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/14cb8e46-418d-4c42-ae97-a384a05b432f-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:57:56 crc kubenswrapper[5052]: I1203 11:57:56.680392 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/14cb8e46-418d-4c42-ae97-a384a05b432f-logs\") on node \"crc\" DevicePath \"\"" Dec 03 11:57:56 crc kubenswrapper[5052]: I1203 11:57:56.680422 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/14cb8e46-418d-4c42-ae97-a384a05b432f-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:57:56 crc kubenswrapper[5052]: I1203 11:57:56.680446 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-5x5ll\" (UniqueName: \"kubernetes.io/projected/14cb8e46-418d-4c42-ae97-a384a05b432f-kube-api-access-5x5ll\") on node \"crc\" DevicePath \"\"" Dec 03 11:57:56 crc kubenswrapper[5052]: I1203 11:57:56.685835 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/c856f6ae-8216-4bd5-a9dd-1d87b546854e-config-data\") pod \"keystone-bootstrap-jhbzq\" (UID: \"c856f6ae-8216-4bd5-a9dd-1d87b546854e\") " pod="openstack-kuttl-tests/keystone-bootstrap-jhbzq" Dec 03 11:57:56 crc kubenswrapper[5052]: I1203 11:57:56.687037 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/c856f6ae-8216-4bd5-a9dd-1d87b546854e-scripts\") pod \"keystone-bootstrap-jhbzq\" (UID: \"c856f6ae-8216-4bd5-a9dd-1d87b546854e\") " pod="openstack-kuttl-tests/keystone-bootstrap-jhbzq" Dec 03 11:57:56 crc kubenswrapper[5052]: I1203 11:57:56.688203 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/c856f6ae-8216-4bd5-a9dd-1d87b546854e-credential-keys\") pod \"keystone-bootstrap-jhbzq\" (UID: \"c856f6ae-8216-4bd5-a9dd-1d87b546854e\") " pod="openstack-kuttl-tests/keystone-bootstrap-jhbzq" Dec 03 11:57:56 crc kubenswrapper[5052]: I1203 11:57:56.691049 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/c856f6ae-8216-4bd5-a9dd-1d87b546854e-fernet-keys\") pod \"keystone-bootstrap-jhbzq\" (UID: \"c856f6ae-8216-4bd5-a9dd-1d87b546854e\") " pod="openstack-kuttl-tests/keystone-bootstrap-jhbzq" Dec 03 11:57:56 crc kubenswrapper[5052]: I1203 11:57:56.691141 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c856f6ae-8216-4bd5-a9dd-1d87b546854e-combined-ca-bundle\") pod \"keystone-bootstrap-jhbzq\" (UID: \"c856f6ae-8216-4bd5-a9dd-1d87b546854e\") " pod="openstack-kuttl-tests/keystone-bootstrap-jhbzq" Dec 03 11:57:56 crc kubenswrapper[5052]: I1203 11:57:56.703338 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-v65wg\" (UniqueName: \"kubernetes.io/projected/c856f6ae-8216-4bd5-a9dd-1d87b546854e-kube-api-access-v65wg\") pod \"keystone-bootstrap-jhbzq\" (UID: \"c856f6ae-8216-4bd5-a9dd-1d87b546854e\") " pod="openstack-kuttl-tests/keystone-bootstrap-jhbzq" Dec 03 11:57:56 crc kubenswrapper[5052]: I1203 11:57:56.712135 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/14cb8e46-418d-4c42-ae97-a384a05b432f-config-data" (OuterVolumeSpecName: "config-data") pod "14cb8e46-418d-4c42-ae97-a384a05b432f" (UID: "14cb8e46-418d-4c42-ae97-a384a05b432f"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:57:56 crc kubenswrapper[5052]: I1203 11:57:56.764707 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-bootstrap-jhbzq" Dec 03 11:57:56 crc kubenswrapper[5052]: I1203 11:57:56.781442 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/watcher-db-sync-chlm4" Dec 03 11:57:56 crc kubenswrapper[5052]: I1203 11:57:56.781576 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/14cb8e46-418d-4c42-ae97-a384a05b432f-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:57:56 crc kubenswrapper[5052]: I1203 11:57:56.883271 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-29fml\" (UniqueName: \"kubernetes.io/projected/b3146921-cc78-401f-a918-fa8b91814f10-kube-api-access-29fml\") pod \"b3146921-cc78-401f-a918-fa8b91814f10\" (UID: \"b3146921-cc78-401f-a918-fa8b91814f10\") " Dec 03 11:57:56 crc kubenswrapper[5052]: I1203 11:57:56.883862 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b3146921-cc78-401f-a918-fa8b91814f10-combined-ca-bundle\") pod \"b3146921-cc78-401f-a918-fa8b91814f10\" (UID: \"b3146921-cc78-401f-a918-fa8b91814f10\") " Dec 03 11:57:56 crc kubenswrapper[5052]: I1203 11:57:56.883913 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/b3146921-cc78-401f-a918-fa8b91814f10-config-data\") pod \"b3146921-cc78-401f-a918-fa8b91814f10\" (UID: \"b3146921-cc78-401f-a918-fa8b91814f10\") " Dec 03 11:57:56 crc kubenswrapper[5052]: I1203 11:57:56.883961 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/b3146921-cc78-401f-a918-fa8b91814f10-db-sync-config-data\") pod \"b3146921-cc78-401f-a918-fa8b91814f10\" (UID: \"b3146921-cc78-401f-a918-fa8b91814f10\") " Dec 03 11:57:56 crc kubenswrapper[5052]: I1203 11:57:56.888197 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b3146921-cc78-401f-a918-fa8b91814f10-db-sync-config-data" (OuterVolumeSpecName: "db-sync-config-data") pod "b3146921-cc78-401f-a918-fa8b91814f10" (UID: "b3146921-cc78-401f-a918-fa8b91814f10"). InnerVolumeSpecName "db-sync-config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:57:56 crc kubenswrapper[5052]: I1203 11:57:56.888495 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b3146921-cc78-401f-a918-fa8b91814f10-kube-api-access-29fml" (OuterVolumeSpecName: "kube-api-access-29fml") pod "b3146921-cc78-401f-a918-fa8b91814f10" (UID: "b3146921-cc78-401f-a918-fa8b91814f10"). InnerVolumeSpecName "kube-api-access-29fml". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:57:56 crc kubenswrapper[5052]: I1203 11:57:56.925455 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b3146921-cc78-401f-a918-fa8b91814f10-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "b3146921-cc78-401f-a918-fa8b91814f10" (UID: "b3146921-cc78-401f-a918-fa8b91814f10"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:57:56 crc kubenswrapper[5052]: I1203 11:57:56.964844 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b3146921-cc78-401f-a918-fa8b91814f10-config-data" (OuterVolumeSpecName: "config-data") pod "b3146921-cc78-401f-a918-fa8b91814f10" (UID: "b3146921-cc78-401f-a918-fa8b91814f10"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:57:56 crc kubenswrapper[5052]: I1203 11:57:56.986089 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b3146921-cc78-401f-a918-fa8b91814f10-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:57:56 crc kubenswrapper[5052]: I1203 11:57:56.986124 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/b3146921-cc78-401f-a918-fa8b91814f10-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:57:56 crc kubenswrapper[5052]: I1203 11:57:56.986133 5052 reconciler_common.go:293] "Volume detached for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/b3146921-cc78-401f-a918-fa8b91814f10-db-sync-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:57:56 crc kubenswrapper[5052]: I1203 11:57:56.986144 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-29fml\" (UniqueName: \"kubernetes.io/projected/b3146921-cc78-401f-a918-fa8b91814f10-kube-api-access-29fml\") on node \"crc\" DevicePath \"\"" Dec 03 11:57:57 crc kubenswrapper[5052]: I1203 11:57:57.019176 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/prometheus-metric-storage-0" Dec 03 11:57:57 crc kubenswrapper[5052]: I1203 11:57:57.049124 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/prometheus-metric-storage-0" Dec 03 11:57:57 crc kubenswrapper[5052]: I1203 11:57:57.108179 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/watcher-db-sync-chlm4" event={"ID":"b3146921-cc78-401f-a918-fa8b91814f10","Type":"ContainerDied","Data":"4ca08618f8e64cb2936c7d44418631a3bca5e400de6062df7154882a95613b8e"} Dec 03 11:57:57 crc kubenswrapper[5052]: I1203 11:57:57.108225 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4ca08618f8e64cb2936c7d44418631a3bca5e400de6062df7154882a95613b8e" Dec 03 11:57:57 crc kubenswrapper[5052]: I1203 11:57:57.108297 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/watcher-db-sync-chlm4" Dec 03 11:57:57 crc kubenswrapper[5052]: I1203 11:57:57.112172 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-db-sync-8tdfb" event={"ID":"14cb8e46-418d-4c42-ae97-a384a05b432f","Type":"ContainerDied","Data":"52ea8ee3cb12bfc11fc6b5e5c5cc9c2498e7e2589c7072d86a8f6b9fdb81c4a1"} Dec 03 11:57:57 crc kubenswrapper[5052]: I1203 11:57:57.112217 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="52ea8ee3cb12bfc11fc6b5e5c5cc9c2498e7e2589c7072d86a8f6b9fdb81c4a1" Dec 03 11:57:57 crc kubenswrapper[5052]: I1203 11:57:57.112281 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-db-sync-8tdfb" Dec 03 11:57:57 crc kubenswrapper[5052]: I1203 11:57:57.113272 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:57:57 crc kubenswrapper[5052]: I1203 11:57:57.118862 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/prometheus-metric-storage-0" Dec 03 11:57:57 crc kubenswrapper[5052]: I1203 11:57:57.256715 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-bootstrap-jhbzq"] Dec 03 11:57:57 crc kubenswrapper[5052]: I1203 11:57:57.318976 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/placement-7d9769b856-tzkn2"] Dec 03 11:57:57 crc kubenswrapper[5052]: E1203 11:57:57.319416 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="b3146921-cc78-401f-a918-fa8b91814f10" containerName="watcher-db-sync" Dec 03 11:57:57 crc kubenswrapper[5052]: I1203 11:57:57.319441 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="b3146921-cc78-401f-a918-fa8b91814f10" containerName="watcher-db-sync" Dec 03 11:57:57 crc kubenswrapper[5052]: E1203 11:57:57.319493 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="14cb8e46-418d-4c42-ae97-a384a05b432f" containerName="placement-db-sync" Dec 03 11:57:57 crc kubenswrapper[5052]: I1203 11:57:57.319504 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="14cb8e46-418d-4c42-ae97-a384a05b432f" containerName="placement-db-sync" Dec 03 11:57:57 crc kubenswrapper[5052]: I1203 11:57:57.319727 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="b3146921-cc78-401f-a918-fa8b91814f10" containerName="watcher-db-sync" Dec 03 11:57:57 crc kubenswrapper[5052]: I1203 11:57:57.319757 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="14cb8e46-418d-4c42-ae97-a384a05b432f" containerName="placement-db-sync" Dec 03 11:57:57 crc kubenswrapper[5052]: I1203 11:57:57.325005 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-7d9769b856-tzkn2" Dec 03 11:57:57 crc kubenswrapper[5052]: I1203 11:57:57.337158 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"placement-scripts" Dec 03 11:57:57 crc kubenswrapper[5052]: I1203 11:57:57.340103 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"placement-placement-dockercfg-mtqj9" Dec 03 11:57:57 crc kubenswrapper[5052]: I1203 11:57:57.340421 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-placement-internal-svc" Dec 03 11:57:57 crc kubenswrapper[5052]: I1203 11:57:57.340605 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-placement-public-svc" Dec 03 11:57:57 crc kubenswrapper[5052]: I1203 11:57:57.340775 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"placement-config-data" Dec 03 11:57:57 crc kubenswrapper[5052]: I1203 11:57:57.360541 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/placement-7d9769b856-tzkn2"] Dec 03 11:57:57 crc kubenswrapper[5052]: I1203 11:57:57.407884 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/60ca40b3-79c7-49cc-9100-0a37c47e4462-config-data\") pod \"placement-7d9769b856-tzkn2\" (UID: \"60ca40b3-79c7-49cc-9100-0a37c47e4462\") " pod="openstack-kuttl-tests/placement-7d9769b856-tzkn2" Dec 03 11:57:57 crc kubenswrapper[5052]: I1203 11:57:57.407944 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/60ca40b3-79c7-49cc-9100-0a37c47e4462-public-tls-certs\") pod \"placement-7d9769b856-tzkn2\" (UID: \"60ca40b3-79c7-49cc-9100-0a37c47e4462\") " pod="openstack-kuttl-tests/placement-7d9769b856-tzkn2" Dec 03 11:57:57 crc kubenswrapper[5052]: I1203 11:57:57.407996 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/60ca40b3-79c7-49cc-9100-0a37c47e4462-combined-ca-bundle\") pod \"placement-7d9769b856-tzkn2\" (UID: \"60ca40b3-79c7-49cc-9100-0a37c47e4462\") " pod="openstack-kuttl-tests/placement-7d9769b856-tzkn2" Dec 03 11:57:57 crc kubenswrapper[5052]: I1203 11:57:57.408040 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/60ca40b3-79c7-49cc-9100-0a37c47e4462-internal-tls-certs\") pod \"placement-7d9769b856-tzkn2\" (UID: \"60ca40b3-79c7-49cc-9100-0a37c47e4462\") " pod="openstack-kuttl-tests/placement-7d9769b856-tzkn2" Dec 03 11:57:57 crc kubenswrapper[5052]: I1203 11:57:57.408086 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/60ca40b3-79c7-49cc-9100-0a37c47e4462-logs\") pod \"placement-7d9769b856-tzkn2\" (UID: \"60ca40b3-79c7-49cc-9100-0a37c47e4462\") " pod="openstack-kuttl-tests/placement-7d9769b856-tzkn2" Dec 03 11:57:57 crc kubenswrapper[5052]: I1203 11:57:57.408110 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8mmxl\" (UniqueName: \"kubernetes.io/projected/60ca40b3-79c7-49cc-9100-0a37c47e4462-kube-api-access-8mmxl\") pod \"placement-7d9769b856-tzkn2\" (UID: \"60ca40b3-79c7-49cc-9100-0a37c47e4462\") " pod="openstack-kuttl-tests/placement-7d9769b856-tzkn2" Dec 03 11:57:57 crc kubenswrapper[5052]: I1203 11:57:57.408155 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/60ca40b3-79c7-49cc-9100-0a37c47e4462-scripts\") pod \"placement-7d9769b856-tzkn2\" (UID: \"60ca40b3-79c7-49cc-9100-0a37c47e4462\") " pod="openstack-kuttl-tests/placement-7d9769b856-tzkn2" Dec 03 11:57:57 crc kubenswrapper[5052]: I1203 11:57:57.460534 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/watcher-decision-engine-0"] Dec 03 11:57:57 crc kubenswrapper[5052]: I1203 11:57:57.469110 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/watcher-decision-engine-0" Dec 03 11:57:57 crc kubenswrapper[5052]: I1203 11:57:57.475915 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"watcher-decision-engine-config-data" Dec 03 11:57:57 crc kubenswrapper[5052]: I1203 11:57:57.476149 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"watcher-watcher-dockercfg-q8qv2" Dec 03 11:57:57 crc kubenswrapper[5052]: I1203 11:57:57.491378 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/watcher-decision-engine-0"] Dec 03 11:57:57 crc kubenswrapper[5052]: I1203 11:57:57.515919 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"custom-prometheus-ca\" (UniqueName: \"kubernetes.io/secret/5893db31-2978-45eb-8407-2e4d178954ce-custom-prometheus-ca\") pod \"watcher-decision-engine-0\" (UID: \"5893db31-2978-45eb-8407-2e4d178954ce\") " pod="openstack-kuttl-tests/watcher-decision-engine-0" Dec 03 11:57:57 crc kubenswrapper[5052]: I1203 11:57:57.516000 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qcmpr\" (UniqueName: \"kubernetes.io/projected/5893db31-2978-45eb-8407-2e4d178954ce-kube-api-access-qcmpr\") pod \"watcher-decision-engine-0\" (UID: \"5893db31-2978-45eb-8407-2e4d178954ce\") " pod="openstack-kuttl-tests/watcher-decision-engine-0" Dec 03 11:57:57 crc kubenswrapper[5052]: I1203 11:57:57.516032 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5893db31-2978-45eb-8407-2e4d178954ce-combined-ca-bundle\") pod \"watcher-decision-engine-0\" (UID: \"5893db31-2978-45eb-8407-2e4d178954ce\") " pod="openstack-kuttl-tests/watcher-decision-engine-0" Dec 03 11:57:57 crc kubenswrapper[5052]: I1203 11:57:57.516105 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/60ca40b3-79c7-49cc-9100-0a37c47e4462-config-data\") pod \"placement-7d9769b856-tzkn2\" (UID: \"60ca40b3-79c7-49cc-9100-0a37c47e4462\") " pod="openstack-kuttl-tests/placement-7d9769b856-tzkn2" Dec 03 11:57:57 crc kubenswrapper[5052]: I1203 11:57:57.516156 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/60ca40b3-79c7-49cc-9100-0a37c47e4462-public-tls-certs\") pod \"placement-7d9769b856-tzkn2\" (UID: \"60ca40b3-79c7-49cc-9100-0a37c47e4462\") " pod="openstack-kuttl-tests/placement-7d9769b856-tzkn2" Dec 03 11:57:57 crc kubenswrapper[5052]: I1203 11:57:57.516184 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/5893db31-2978-45eb-8407-2e4d178954ce-logs\") pod \"watcher-decision-engine-0\" (UID: \"5893db31-2978-45eb-8407-2e4d178954ce\") " pod="openstack-kuttl-tests/watcher-decision-engine-0" Dec 03 11:57:57 crc kubenswrapper[5052]: I1203 11:57:57.516262 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/60ca40b3-79c7-49cc-9100-0a37c47e4462-combined-ca-bundle\") pod \"placement-7d9769b856-tzkn2\" (UID: \"60ca40b3-79c7-49cc-9100-0a37c47e4462\") " pod="openstack-kuttl-tests/placement-7d9769b856-tzkn2" Dec 03 11:57:57 crc kubenswrapper[5052]: I1203 11:57:57.516318 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/5893db31-2978-45eb-8407-2e4d178954ce-config-data\") pod \"watcher-decision-engine-0\" (UID: \"5893db31-2978-45eb-8407-2e4d178954ce\") " pod="openstack-kuttl-tests/watcher-decision-engine-0" Dec 03 11:57:57 crc kubenswrapper[5052]: I1203 11:57:57.516390 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/60ca40b3-79c7-49cc-9100-0a37c47e4462-internal-tls-certs\") pod \"placement-7d9769b856-tzkn2\" (UID: \"60ca40b3-79c7-49cc-9100-0a37c47e4462\") " pod="openstack-kuttl-tests/placement-7d9769b856-tzkn2" Dec 03 11:57:57 crc kubenswrapper[5052]: I1203 11:57:57.516461 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/60ca40b3-79c7-49cc-9100-0a37c47e4462-logs\") pod \"placement-7d9769b856-tzkn2\" (UID: \"60ca40b3-79c7-49cc-9100-0a37c47e4462\") " pod="openstack-kuttl-tests/placement-7d9769b856-tzkn2" Dec 03 11:57:57 crc kubenswrapper[5052]: I1203 11:57:57.516494 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-8mmxl\" (UniqueName: \"kubernetes.io/projected/60ca40b3-79c7-49cc-9100-0a37c47e4462-kube-api-access-8mmxl\") pod \"placement-7d9769b856-tzkn2\" (UID: \"60ca40b3-79c7-49cc-9100-0a37c47e4462\") " pod="openstack-kuttl-tests/placement-7d9769b856-tzkn2" Dec 03 11:57:57 crc kubenswrapper[5052]: I1203 11:57:57.516574 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/60ca40b3-79c7-49cc-9100-0a37c47e4462-scripts\") pod \"placement-7d9769b856-tzkn2\" (UID: \"60ca40b3-79c7-49cc-9100-0a37c47e4462\") " pod="openstack-kuttl-tests/placement-7d9769b856-tzkn2" Dec 03 11:57:57 crc kubenswrapper[5052]: I1203 11:57:57.528932 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/60ca40b3-79c7-49cc-9100-0a37c47e4462-logs\") pod \"placement-7d9769b856-tzkn2\" (UID: \"60ca40b3-79c7-49cc-9100-0a37c47e4462\") " pod="openstack-kuttl-tests/placement-7d9769b856-tzkn2" Dec 03 11:57:57 crc kubenswrapper[5052]: I1203 11:57:57.552748 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/60ca40b3-79c7-49cc-9100-0a37c47e4462-config-data\") pod \"placement-7d9769b856-tzkn2\" (UID: \"60ca40b3-79c7-49cc-9100-0a37c47e4462\") " pod="openstack-kuttl-tests/placement-7d9769b856-tzkn2" Dec 03 11:57:57 crc kubenswrapper[5052]: I1203 11:57:57.552819 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/60ca40b3-79c7-49cc-9100-0a37c47e4462-scripts\") pod \"placement-7d9769b856-tzkn2\" (UID: \"60ca40b3-79c7-49cc-9100-0a37c47e4462\") " pod="openstack-kuttl-tests/placement-7d9769b856-tzkn2" Dec 03 11:57:57 crc kubenswrapper[5052]: I1203 11:57:57.553839 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/60ca40b3-79c7-49cc-9100-0a37c47e4462-public-tls-certs\") pod \"placement-7d9769b856-tzkn2\" (UID: \"60ca40b3-79c7-49cc-9100-0a37c47e4462\") " pod="openstack-kuttl-tests/placement-7d9769b856-tzkn2" Dec 03 11:57:57 crc kubenswrapper[5052]: I1203 11:57:57.557168 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-8mmxl\" (UniqueName: \"kubernetes.io/projected/60ca40b3-79c7-49cc-9100-0a37c47e4462-kube-api-access-8mmxl\") pod \"placement-7d9769b856-tzkn2\" (UID: \"60ca40b3-79c7-49cc-9100-0a37c47e4462\") " pod="openstack-kuttl-tests/placement-7d9769b856-tzkn2" Dec 03 11:57:57 crc kubenswrapper[5052]: I1203 11:57:57.558020 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/watcher-applier-0"] Dec 03 11:57:57 crc kubenswrapper[5052]: I1203 11:57:57.558403 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/60ca40b3-79c7-49cc-9100-0a37c47e4462-internal-tls-certs\") pod \"placement-7d9769b856-tzkn2\" (UID: \"60ca40b3-79c7-49cc-9100-0a37c47e4462\") " pod="openstack-kuttl-tests/placement-7d9769b856-tzkn2" Dec 03 11:57:57 crc kubenswrapper[5052]: I1203 11:57:57.567244 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/watcher-applier-0"] Dec 03 11:57:57 crc kubenswrapper[5052]: I1203 11:57:57.567350 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/watcher-applier-0" Dec 03 11:57:57 crc kubenswrapper[5052]: I1203 11:57:57.570291 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/60ca40b3-79c7-49cc-9100-0a37c47e4462-combined-ca-bundle\") pod \"placement-7d9769b856-tzkn2\" (UID: \"60ca40b3-79c7-49cc-9100-0a37c47e4462\") " pod="openstack-kuttl-tests/placement-7d9769b856-tzkn2" Dec 03 11:57:57 crc kubenswrapper[5052]: I1203 11:57:57.572203 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"watcher-applier-config-data" Dec 03 11:57:57 crc kubenswrapper[5052]: I1203 11:57:57.622496 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vn6ws\" (UniqueName: \"kubernetes.io/projected/192fa797-c8b8-42d3-a689-e6a7dbc77282-kube-api-access-vn6ws\") pod \"watcher-applier-0\" (UID: \"192fa797-c8b8-42d3-a689-e6a7dbc77282\") " pod="openstack-kuttl-tests/watcher-applier-0" Dec 03 11:57:57 crc kubenswrapper[5052]: I1203 11:57:57.622694 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/192fa797-c8b8-42d3-a689-e6a7dbc77282-combined-ca-bundle\") pod \"watcher-applier-0\" (UID: \"192fa797-c8b8-42d3-a689-e6a7dbc77282\") " pod="openstack-kuttl-tests/watcher-applier-0" Dec 03 11:57:57 crc kubenswrapper[5052]: I1203 11:57:57.622744 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/5893db31-2978-45eb-8407-2e4d178954ce-config-data\") pod \"watcher-decision-engine-0\" (UID: \"5893db31-2978-45eb-8407-2e4d178954ce\") " pod="openstack-kuttl-tests/watcher-decision-engine-0" Dec 03 11:57:57 crc kubenswrapper[5052]: I1203 11:57:57.622867 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/192fa797-c8b8-42d3-a689-e6a7dbc77282-config-data\") pod \"watcher-applier-0\" (UID: \"192fa797-c8b8-42d3-a689-e6a7dbc77282\") " pod="openstack-kuttl-tests/watcher-applier-0" Dec 03 11:57:57 crc kubenswrapper[5052]: I1203 11:57:57.622981 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"custom-prometheus-ca\" (UniqueName: \"kubernetes.io/secret/5893db31-2978-45eb-8407-2e4d178954ce-custom-prometheus-ca\") pod \"watcher-decision-engine-0\" (UID: \"5893db31-2978-45eb-8407-2e4d178954ce\") " pod="openstack-kuttl-tests/watcher-decision-engine-0" Dec 03 11:57:57 crc kubenswrapper[5052]: I1203 11:57:57.623016 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-qcmpr\" (UniqueName: \"kubernetes.io/projected/5893db31-2978-45eb-8407-2e4d178954ce-kube-api-access-qcmpr\") pod \"watcher-decision-engine-0\" (UID: \"5893db31-2978-45eb-8407-2e4d178954ce\") " pod="openstack-kuttl-tests/watcher-decision-engine-0" Dec 03 11:57:57 crc kubenswrapper[5052]: I1203 11:57:57.623043 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5893db31-2978-45eb-8407-2e4d178954ce-combined-ca-bundle\") pod \"watcher-decision-engine-0\" (UID: \"5893db31-2978-45eb-8407-2e4d178954ce\") " pod="openstack-kuttl-tests/watcher-decision-engine-0" Dec 03 11:57:57 crc kubenswrapper[5052]: I1203 11:57:57.623091 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/192fa797-c8b8-42d3-a689-e6a7dbc77282-logs\") pod \"watcher-applier-0\" (UID: \"192fa797-c8b8-42d3-a689-e6a7dbc77282\") " pod="openstack-kuttl-tests/watcher-applier-0" Dec 03 11:57:57 crc kubenswrapper[5052]: I1203 11:57:57.623136 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/5893db31-2978-45eb-8407-2e4d178954ce-logs\") pod \"watcher-decision-engine-0\" (UID: \"5893db31-2978-45eb-8407-2e4d178954ce\") " pod="openstack-kuttl-tests/watcher-decision-engine-0" Dec 03 11:57:57 crc kubenswrapper[5052]: I1203 11:57:57.623503 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/5893db31-2978-45eb-8407-2e4d178954ce-logs\") pod \"watcher-decision-engine-0\" (UID: \"5893db31-2978-45eb-8407-2e4d178954ce\") " pod="openstack-kuttl-tests/watcher-decision-engine-0" Dec 03 11:57:57 crc kubenswrapper[5052]: I1203 11:57:57.632345 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/5893db31-2978-45eb-8407-2e4d178954ce-config-data\") pod \"watcher-decision-engine-0\" (UID: \"5893db31-2978-45eb-8407-2e4d178954ce\") " pod="openstack-kuttl-tests/watcher-decision-engine-0" Dec 03 11:57:57 crc kubenswrapper[5052]: I1203 11:57:57.636034 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/watcher-api-0"] Dec 03 11:57:57 crc kubenswrapper[5052]: I1203 11:57:57.637517 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/watcher-api-0" Dec 03 11:57:57 crc kubenswrapper[5052]: I1203 11:57:57.638743 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"custom-prometheus-ca\" (UniqueName: \"kubernetes.io/secret/5893db31-2978-45eb-8407-2e4d178954ce-custom-prometheus-ca\") pod \"watcher-decision-engine-0\" (UID: \"5893db31-2978-45eb-8407-2e4d178954ce\") " pod="openstack-kuttl-tests/watcher-decision-engine-0" Dec 03 11:57:57 crc kubenswrapper[5052]: I1203 11:57:57.638972 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5893db31-2978-45eb-8407-2e4d178954ce-combined-ca-bundle\") pod \"watcher-decision-engine-0\" (UID: \"5893db31-2978-45eb-8407-2e4d178954ce\") " pod="openstack-kuttl-tests/watcher-decision-engine-0" Dec 03 11:57:57 crc kubenswrapper[5052]: I1203 11:57:57.640191 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"watcher-api-config-data" Dec 03 11:57:57 crc kubenswrapper[5052]: I1203 11:57:57.650645 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/watcher-api-0"] Dec 03 11:57:57 crc kubenswrapper[5052]: I1203 11:57:57.651028 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-7d9769b856-tzkn2" Dec 03 11:57:57 crc kubenswrapper[5052]: I1203 11:57:57.674087 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-qcmpr\" (UniqueName: \"kubernetes.io/projected/5893db31-2978-45eb-8407-2e4d178954ce-kube-api-access-qcmpr\") pod \"watcher-decision-engine-0\" (UID: \"5893db31-2978-45eb-8407-2e4d178954ce\") " pod="openstack-kuttl-tests/watcher-decision-engine-0" Dec 03 11:57:57 crc kubenswrapper[5052]: I1203 11:57:57.680681 5052 scope.go:117] "RemoveContainer" containerID="d224717c6d9ca8135fd510e03a6c40c9f2b1a56dd8b4a3b685af3e2dfa855982" Dec 03 11:57:57 crc kubenswrapper[5052]: E1203 11:57:57.680922 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 11:57:57 crc kubenswrapper[5052]: I1203 11:57:57.698818 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b70c4b01-42e4-4990-be48-a2d762e66cd7" path="/var/lib/kubelet/pods/b70c4b01-42e4-4990-be48-a2d762e66cd7/volumes" Dec 03 11:57:57 crc kubenswrapper[5052]: I1203 11:57:57.724691 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/192fa797-c8b8-42d3-a689-e6a7dbc77282-combined-ca-bundle\") pod \"watcher-applier-0\" (UID: \"192fa797-c8b8-42d3-a689-e6a7dbc77282\") " pod="openstack-kuttl-tests/watcher-applier-0" Dec 03 11:57:57 crc kubenswrapper[5052]: I1203 11:57:57.724827 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"custom-prometheus-ca\" (UniqueName: \"kubernetes.io/secret/dc7d33d8-fd6f-466a-9ea2-2a643e314d53-custom-prometheus-ca\") pod \"watcher-api-0\" (UID: \"dc7d33d8-fd6f-466a-9ea2-2a643e314d53\") " pod="openstack-kuttl-tests/watcher-api-0" Dec 03 11:57:57 crc kubenswrapper[5052]: I1203 11:57:57.724853 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qqjgg\" (UniqueName: \"kubernetes.io/projected/dc7d33d8-fd6f-466a-9ea2-2a643e314d53-kube-api-access-qqjgg\") pod \"watcher-api-0\" (UID: \"dc7d33d8-fd6f-466a-9ea2-2a643e314d53\") " pod="openstack-kuttl-tests/watcher-api-0" Dec 03 11:57:57 crc kubenswrapper[5052]: I1203 11:57:57.724912 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/192fa797-c8b8-42d3-a689-e6a7dbc77282-config-data\") pod \"watcher-applier-0\" (UID: \"192fa797-c8b8-42d3-a689-e6a7dbc77282\") " pod="openstack-kuttl-tests/watcher-applier-0" Dec 03 11:57:57 crc kubenswrapper[5052]: I1203 11:57:57.724958 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/dc7d33d8-fd6f-466a-9ea2-2a643e314d53-config-data\") pod \"watcher-api-0\" (UID: \"dc7d33d8-fd6f-466a-9ea2-2a643e314d53\") " pod="openstack-kuttl-tests/watcher-api-0" Dec 03 11:57:57 crc kubenswrapper[5052]: I1203 11:57:57.725057 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/dc7d33d8-fd6f-466a-9ea2-2a643e314d53-combined-ca-bundle\") pod \"watcher-api-0\" (UID: \"dc7d33d8-fd6f-466a-9ea2-2a643e314d53\") " pod="openstack-kuttl-tests/watcher-api-0" Dec 03 11:57:57 crc kubenswrapper[5052]: I1203 11:57:57.725138 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/192fa797-c8b8-42d3-a689-e6a7dbc77282-logs\") pod \"watcher-applier-0\" (UID: \"192fa797-c8b8-42d3-a689-e6a7dbc77282\") " pod="openstack-kuttl-tests/watcher-applier-0" Dec 03 11:57:57 crc kubenswrapper[5052]: I1203 11:57:57.725182 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/dc7d33d8-fd6f-466a-9ea2-2a643e314d53-logs\") pod \"watcher-api-0\" (UID: \"dc7d33d8-fd6f-466a-9ea2-2a643e314d53\") " pod="openstack-kuttl-tests/watcher-api-0" Dec 03 11:57:57 crc kubenswrapper[5052]: I1203 11:57:57.725227 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-vn6ws\" (UniqueName: \"kubernetes.io/projected/192fa797-c8b8-42d3-a689-e6a7dbc77282-kube-api-access-vn6ws\") pod \"watcher-applier-0\" (UID: \"192fa797-c8b8-42d3-a689-e6a7dbc77282\") " pod="openstack-kuttl-tests/watcher-applier-0" Dec 03 11:57:57 crc kubenswrapper[5052]: I1203 11:57:57.726827 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/192fa797-c8b8-42d3-a689-e6a7dbc77282-logs\") pod \"watcher-applier-0\" (UID: \"192fa797-c8b8-42d3-a689-e6a7dbc77282\") " pod="openstack-kuttl-tests/watcher-applier-0" Dec 03 11:57:57 crc kubenswrapper[5052]: I1203 11:57:57.731308 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/192fa797-c8b8-42d3-a689-e6a7dbc77282-combined-ca-bundle\") pod \"watcher-applier-0\" (UID: \"192fa797-c8b8-42d3-a689-e6a7dbc77282\") " pod="openstack-kuttl-tests/watcher-applier-0" Dec 03 11:57:57 crc kubenswrapper[5052]: I1203 11:57:57.732838 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/192fa797-c8b8-42d3-a689-e6a7dbc77282-config-data\") pod \"watcher-applier-0\" (UID: \"192fa797-c8b8-42d3-a689-e6a7dbc77282\") " pod="openstack-kuttl-tests/watcher-applier-0" Dec 03 11:57:57 crc kubenswrapper[5052]: I1203 11:57:57.746163 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-vn6ws\" (UniqueName: \"kubernetes.io/projected/192fa797-c8b8-42d3-a689-e6a7dbc77282-kube-api-access-vn6ws\") pod \"watcher-applier-0\" (UID: \"192fa797-c8b8-42d3-a689-e6a7dbc77282\") " pod="openstack-kuttl-tests/watcher-applier-0" Dec 03 11:57:57 crc kubenswrapper[5052]: I1203 11:57:57.827290 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/dc7d33d8-fd6f-466a-9ea2-2a643e314d53-logs\") pod \"watcher-api-0\" (UID: \"dc7d33d8-fd6f-466a-9ea2-2a643e314d53\") " pod="openstack-kuttl-tests/watcher-api-0" Dec 03 11:57:57 crc kubenswrapper[5052]: I1203 11:57:57.827437 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"custom-prometheus-ca\" (UniqueName: \"kubernetes.io/secret/dc7d33d8-fd6f-466a-9ea2-2a643e314d53-custom-prometheus-ca\") pod \"watcher-api-0\" (UID: \"dc7d33d8-fd6f-466a-9ea2-2a643e314d53\") " pod="openstack-kuttl-tests/watcher-api-0" Dec 03 11:57:57 crc kubenswrapper[5052]: I1203 11:57:57.827463 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-qqjgg\" (UniqueName: \"kubernetes.io/projected/dc7d33d8-fd6f-466a-9ea2-2a643e314d53-kube-api-access-qqjgg\") pod \"watcher-api-0\" (UID: \"dc7d33d8-fd6f-466a-9ea2-2a643e314d53\") " pod="openstack-kuttl-tests/watcher-api-0" Dec 03 11:57:57 crc kubenswrapper[5052]: I1203 11:57:57.827535 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/dc7d33d8-fd6f-466a-9ea2-2a643e314d53-config-data\") pod \"watcher-api-0\" (UID: \"dc7d33d8-fd6f-466a-9ea2-2a643e314d53\") " pod="openstack-kuttl-tests/watcher-api-0" Dec 03 11:57:57 crc kubenswrapper[5052]: I1203 11:57:57.827612 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/dc7d33d8-fd6f-466a-9ea2-2a643e314d53-combined-ca-bundle\") pod \"watcher-api-0\" (UID: \"dc7d33d8-fd6f-466a-9ea2-2a643e314d53\") " pod="openstack-kuttl-tests/watcher-api-0" Dec 03 11:57:57 crc kubenswrapper[5052]: I1203 11:57:57.828355 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/dc7d33d8-fd6f-466a-9ea2-2a643e314d53-logs\") pod \"watcher-api-0\" (UID: \"dc7d33d8-fd6f-466a-9ea2-2a643e314d53\") " pod="openstack-kuttl-tests/watcher-api-0" Dec 03 11:57:57 crc kubenswrapper[5052]: I1203 11:57:57.837189 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/dc7d33d8-fd6f-466a-9ea2-2a643e314d53-combined-ca-bundle\") pod \"watcher-api-0\" (UID: \"dc7d33d8-fd6f-466a-9ea2-2a643e314d53\") " pod="openstack-kuttl-tests/watcher-api-0" Dec 03 11:57:57 crc kubenswrapper[5052]: I1203 11:57:57.846284 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/dc7d33d8-fd6f-466a-9ea2-2a643e314d53-config-data\") pod \"watcher-api-0\" (UID: \"dc7d33d8-fd6f-466a-9ea2-2a643e314d53\") " pod="openstack-kuttl-tests/watcher-api-0" Dec 03 11:57:57 crc kubenswrapper[5052]: I1203 11:57:57.847428 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"custom-prometheus-ca\" (UniqueName: \"kubernetes.io/secret/dc7d33d8-fd6f-466a-9ea2-2a643e314d53-custom-prometheus-ca\") pod \"watcher-api-0\" (UID: \"dc7d33d8-fd6f-466a-9ea2-2a643e314d53\") " pod="openstack-kuttl-tests/watcher-api-0" Dec 03 11:57:57 crc kubenswrapper[5052]: I1203 11:57:57.852010 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-qqjgg\" (UniqueName: \"kubernetes.io/projected/dc7d33d8-fd6f-466a-9ea2-2a643e314d53-kube-api-access-qqjgg\") pod \"watcher-api-0\" (UID: \"dc7d33d8-fd6f-466a-9ea2-2a643e314d53\") " pod="openstack-kuttl-tests/watcher-api-0" Dec 03 11:57:57 crc kubenswrapper[5052]: I1203 11:57:57.852813 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/watcher-decision-engine-0" Dec 03 11:57:57 crc kubenswrapper[5052]: I1203 11:57:57.892386 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/watcher-applier-0" Dec 03 11:57:57 crc kubenswrapper[5052]: I1203 11:57:57.906052 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-db-sync-xs2lk" Dec 03 11:57:57 crc kubenswrapper[5052]: I1203 11:57:57.908613 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/watcher-api-0" Dec 03 11:57:57 crc kubenswrapper[5052]: I1203 11:57:57.928371 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pz5h5\" (UniqueName: \"kubernetes.io/projected/a50cd1a3-6eb7-4712-bb60-648250b396d6-kube-api-access-pz5h5\") pod \"a50cd1a3-6eb7-4712-bb60-648250b396d6\" (UID: \"a50cd1a3-6eb7-4712-bb60-648250b396d6\") " Dec 03 11:57:57 crc kubenswrapper[5052]: I1203 11:57:57.928467 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a50cd1a3-6eb7-4712-bb60-648250b396d6-combined-ca-bundle\") pod \"a50cd1a3-6eb7-4712-bb60-648250b396d6\" (UID: \"a50cd1a3-6eb7-4712-bb60-648250b396d6\") " Dec 03 11:57:57 crc kubenswrapper[5052]: I1203 11:57:57.928504 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/a50cd1a3-6eb7-4712-bb60-648250b396d6-db-sync-config-data\") pod \"a50cd1a3-6eb7-4712-bb60-648250b396d6\" (UID: \"a50cd1a3-6eb7-4712-bb60-648250b396d6\") " Dec 03 11:57:57 crc kubenswrapper[5052]: I1203 11:57:57.937700 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:57:57 crc kubenswrapper[5052]: I1203 11:57:57.937817 5052 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Dec 03 11:57:57 crc kubenswrapper[5052]: I1203 11:57:57.944308 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a50cd1a3-6eb7-4712-bb60-648250b396d6-kube-api-access-pz5h5" (OuterVolumeSpecName: "kube-api-access-pz5h5") pod "a50cd1a3-6eb7-4712-bb60-648250b396d6" (UID: "a50cd1a3-6eb7-4712-bb60-648250b396d6"). InnerVolumeSpecName "kube-api-access-pz5h5". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:57:57 crc kubenswrapper[5052]: I1203 11:57:57.944333 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a50cd1a3-6eb7-4712-bb60-648250b396d6-db-sync-config-data" (OuterVolumeSpecName: "db-sync-config-data") pod "a50cd1a3-6eb7-4712-bb60-648250b396d6" (UID: "a50cd1a3-6eb7-4712-bb60-648250b396d6"). InnerVolumeSpecName "db-sync-config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:57:57 crc kubenswrapper[5052]: I1203 11:57:57.978206 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a50cd1a3-6eb7-4712-bb60-648250b396d6-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "a50cd1a3-6eb7-4712-bb60-648250b396d6" (UID: "a50cd1a3-6eb7-4712-bb60-648250b396d6"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:57:58 crc kubenswrapper[5052]: I1203 11:57:58.002192 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/placement-7d9769b856-tzkn2"] Dec 03 11:57:58 crc kubenswrapper[5052]: W1203 11:57:58.030706 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod60ca40b3_79c7_49cc_9100_0a37c47e4462.slice/crio-f81f8b7a9a04ac8ec5ef925547ed0d5543d363bc13231b720dca75d6ff169fbc WatchSource:0}: Error finding container f81f8b7a9a04ac8ec5ef925547ed0d5543d363bc13231b720dca75d6ff169fbc: Status 404 returned error can't find the container with id f81f8b7a9a04ac8ec5ef925547ed0d5543d363bc13231b720dca75d6ff169fbc Dec 03 11:57:58 crc kubenswrapper[5052]: I1203 11:57:58.031817 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-pz5h5\" (UniqueName: \"kubernetes.io/projected/a50cd1a3-6eb7-4712-bb60-648250b396d6-kube-api-access-pz5h5\") on node \"crc\" DevicePath \"\"" Dec 03 11:57:58 crc kubenswrapper[5052]: I1203 11:57:58.031835 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a50cd1a3-6eb7-4712-bb60-648250b396d6-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:57:58 crc kubenswrapper[5052]: I1203 11:57:58.031845 5052 reconciler_common.go:293] "Volume detached for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/a50cd1a3-6eb7-4712-bb60-648250b396d6-db-sync-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:57:58 crc kubenswrapper[5052]: I1203 11:57:58.144333 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-7d9769b856-tzkn2" event={"ID":"60ca40b3-79c7-49cc-9100-0a37c47e4462","Type":"ContainerStarted","Data":"f81f8b7a9a04ac8ec5ef925547ed0d5543d363bc13231b720dca75d6ff169fbc"} Dec 03 11:57:58 crc kubenswrapper[5052]: I1203 11:57:58.154457 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-bootstrap-jhbzq" event={"ID":"c856f6ae-8216-4bd5-a9dd-1d87b546854e","Type":"ContainerStarted","Data":"cc2a1588ce159bc7153ef7c3f5b7ed3bf02b793e16744abb26c49b8e3ea42995"} Dec 03 11:57:58 crc kubenswrapper[5052]: I1203 11:57:58.158703 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-db-sync-xs2lk" event={"ID":"a50cd1a3-6eb7-4712-bb60-648250b396d6","Type":"ContainerDied","Data":"584317d8f9b53a44d8099c03595a73e25695880f52f568ed574db59b7a5db678"} Dec 03 11:57:58 crc kubenswrapper[5052]: I1203 11:57:58.158748 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="584317d8f9b53a44d8099c03595a73e25695880f52f568ed574db59b7a5db678" Dec 03 11:57:58 crc kubenswrapper[5052]: I1203 11:57:58.158751 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-db-sync-xs2lk" Dec 03 11:57:58 crc kubenswrapper[5052]: I1203 11:57:58.418447 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/barbican-keystone-listener-6774dbcdb8-wzwf7"] Dec 03 11:57:58 crc kubenswrapper[5052]: E1203 11:57:58.418925 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="a50cd1a3-6eb7-4712-bb60-648250b396d6" containerName="barbican-db-sync" Dec 03 11:57:58 crc kubenswrapper[5052]: I1203 11:57:58.418950 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="a50cd1a3-6eb7-4712-bb60-648250b396d6" containerName="barbican-db-sync" Dec 03 11:57:58 crc kubenswrapper[5052]: I1203 11:57:58.428164 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="a50cd1a3-6eb7-4712-bb60-648250b396d6" containerName="barbican-db-sync" Dec 03 11:57:58 crc kubenswrapper[5052]: I1203 11:57:58.431474 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-keystone-listener-6774dbcdb8-wzwf7" Dec 03 11:57:58 crc kubenswrapper[5052]: I1203 11:57:58.452757 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"barbican-keystone-listener-config-data" Dec 03 11:57:58 crc kubenswrapper[5052]: I1203 11:57:58.453058 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"barbican-barbican-dockercfg-x9zph" Dec 03 11:57:58 crc kubenswrapper[5052]: I1203 11:57:58.453394 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"barbican-config-data" Dec 03 11:57:58 crc kubenswrapper[5052]: I1203 11:57:58.484922 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2pzd9\" (UniqueName: \"kubernetes.io/projected/e09397b4-fabb-4e9d-8695-669bf9f49c77-kube-api-access-2pzd9\") pod \"barbican-keystone-listener-6774dbcdb8-wzwf7\" (UID: \"e09397b4-fabb-4e9d-8695-669bf9f49c77\") " pod="openstack-kuttl-tests/barbican-keystone-listener-6774dbcdb8-wzwf7" Dec 03 11:57:58 crc kubenswrapper[5052]: I1203 11:57:58.485059 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/e09397b4-fabb-4e9d-8695-669bf9f49c77-config-data-custom\") pod \"barbican-keystone-listener-6774dbcdb8-wzwf7\" (UID: \"e09397b4-fabb-4e9d-8695-669bf9f49c77\") " pod="openstack-kuttl-tests/barbican-keystone-listener-6774dbcdb8-wzwf7" Dec 03 11:57:58 crc kubenswrapper[5052]: I1203 11:57:58.485271 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e09397b4-fabb-4e9d-8695-669bf9f49c77-combined-ca-bundle\") pod \"barbican-keystone-listener-6774dbcdb8-wzwf7\" (UID: \"e09397b4-fabb-4e9d-8695-669bf9f49c77\") " pod="openstack-kuttl-tests/barbican-keystone-listener-6774dbcdb8-wzwf7" Dec 03 11:57:58 crc kubenswrapper[5052]: I1203 11:57:58.485403 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/e09397b4-fabb-4e9d-8695-669bf9f49c77-logs\") pod \"barbican-keystone-listener-6774dbcdb8-wzwf7\" (UID: \"e09397b4-fabb-4e9d-8695-669bf9f49c77\") " pod="openstack-kuttl-tests/barbican-keystone-listener-6774dbcdb8-wzwf7" Dec 03 11:57:58 crc kubenswrapper[5052]: I1203 11:57:58.485442 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/e09397b4-fabb-4e9d-8695-669bf9f49c77-config-data\") pod \"barbican-keystone-listener-6774dbcdb8-wzwf7\" (UID: \"e09397b4-fabb-4e9d-8695-669bf9f49c77\") " pod="openstack-kuttl-tests/barbican-keystone-listener-6774dbcdb8-wzwf7" Dec 03 11:57:58 crc kubenswrapper[5052]: I1203 11:57:58.505472 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/barbican-worker-ff9447cb9-nmz9j"] Dec 03 11:57:58 crc kubenswrapper[5052]: I1203 11:57:58.509380 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-worker-ff9447cb9-nmz9j" Dec 03 11:57:58 crc kubenswrapper[5052]: I1203 11:57:58.521885 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"barbican-worker-config-data" Dec 03 11:57:58 crc kubenswrapper[5052]: I1203 11:57:58.589578 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-keystone-listener-6774dbcdb8-wzwf7"] Dec 03 11:57:58 crc kubenswrapper[5052]: I1203 11:57:58.590643 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-2pzd9\" (UniqueName: \"kubernetes.io/projected/e09397b4-fabb-4e9d-8695-669bf9f49c77-kube-api-access-2pzd9\") pod \"barbican-keystone-listener-6774dbcdb8-wzwf7\" (UID: \"e09397b4-fabb-4e9d-8695-669bf9f49c77\") " pod="openstack-kuttl-tests/barbican-keystone-listener-6774dbcdb8-wzwf7" Dec 03 11:57:58 crc kubenswrapper[5052]: I1203 11:57:58.590736 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/e09397b4-fabb-4e9d-8695-669bf9f49c77-config-data-custom\") pod \"barbican-keystone-listener-6774dbcdb8-wzwf7\" (UID: \"e09397b4-fabb-4e9d-8695-669bf9f49c77\") " pod="openstack-kuttl-tests/barbican-keystone-listener-6774dbcdb8-wzwf7" Dec 03 11:57:58 crc kubenswrapper[5052]: I1203 11:57:58.590842 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e09397b4-fabb-4e9d-8695-669bf9f49c77-combined-ca-bundle\") pod \"barbican-keystone-listener-6774dbcdb8-wzwf7\" (UID: \"e09397b4-fabb-4e9d-8695-669bf9f49c77\") " pod="openstack-kuttl-tests/barbican-keystone-listener-6774dbcdb8-wzwf7" Dec 03 11:57:58 crc kubenswrapper[5052]: I1203 11:57:58.590926 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/e09397b4-fabb-4e9d-8695-669bf9f49c77-logs\") pod \"barbican-keystone-listener-6774dbcdb8-wzwf7\" (UID: \"e09397b4-fabb-4e9d-8695-669bf9f49c77\") " pod="openstack-kuttl-tests/barbican-keystone-listener-6774dbcdb8-wzwf7" Dec 03 11:57:58 crc kubenswrapper[5052]: I1203 11:57:58.590963 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/e09397b4-fabb-4e9d-8695-669bf9f49c77-config-data\") pod \"barbican-keystone-listener-6774dbcdb8-wzwf7\" (UID: \"e09397b4-fabb-4e9d-8695-669bf9f49c77\") " pod="openstack-kuttl-tests/barbican-keystone-listener-6774dbcdb8-wzwf7" Dec 03 11:57:58 crc kubenswrapper[5052]: I1203 11:57:58.594490 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/e09397b4-fabb-4e9d-8695-669bf9f49c77-logs\") pod \"barbican-keystone-listener-6774dbcdb8-wzwf7\" (UID: \"e09397b4-fabb-4e9d-8695-669bf9f49c77\") " pod="openstack-kuttl-tests/barbican-keystone-listener-6774dbcdb8-wzwf7" Dec 03 11:57:58 crc kubenswrapper[5052]: I1203 11:57:58.617152 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/e09397b4-fabb-4e9d-8695-669bf9f49c77-config-data-custom\") pod \"barbican-keystone-listener-6774dbcdb8-wzwf7\" (UID: \"e09397b4-fabb-4e9d-8695-669bf9f49c77\") " pod="openstack-kuttl-tests/barbican-keystone-listener-6774dbcdb8-wzwf7" Dec 03 11:57:58 crc kubenswrapper[5052]: I1203 11:57:58.617228 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-worker-ff9447cb9-nmz9j"] Dec 03 11:57:58 crc kubenswrapper[5052]: I1203 11:57:58.623785 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/e09397b4-fabb-4e9d-8695-669bf9f49c77-config-data\") pod \"barbican-keystone-listener-6774dbcdb8-wzwf7\" (UID: \"e09397b4-fabb-4e9d-8695-669bf9f49c77\") " pod="openstack-kuttl-tests/barbican-keystone-listener-6774dbcdb8-wzwf7" Dec 03 11:57:58 crc kubenswrapper[5052]: I1203 11:57:58.631728 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-2pzd9\" (UniqueName: \"kubernetes.io/projected/e09397b4-fabb-4e9d-8695-669bf9f49c77-kube-api-access-2pzd9\") pod \"barbican-keystone-listener-6774dbcdb8-wzwf7\" (UID: \"e09397b4-fabb-4e9d-8695-669bf9f49c77\") " pod="openstack-kuttl-tests/barbican-keystone-listener-6774dbcdb8-wzwf7" Dec 03 11:57:58 crc kubenswrapper[5052]: I1203 11:57:58.642233 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e09397b4-fabb-4e9d-8695-669bf9f49c77-combined-ca-bundle\") pod \"barbican-keystone-listener-6774dbcdb8-wzwf7\" (UID: \"e09397b4-fabb-4e9d-8695-669bf9f49c77\") " pod="openstack-kuttl-tests/barbican-keystone-listener-6774dbcdb8-wzwf7" Dec 03 11:57:58 crc kubenswrapper[5052]: I1203 11:57:58.647747 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:57:58 crc kubenswrapper[5052]: I1203 11:57:58.664357 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/watcher-decision-engine-0"] Dec 03 11:57:58 crc kubenswrapper[5052]: I1203 11:57:58.694318 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/60f75996-dceb-4640-8fca-96fdcc03ebb9-logs\") pod \"barbican-worker-ff9447cb9-nmz9j\" (UID: \"60f75996-dceb-4640-8fca-96fdcc03ebb9\") " pod="openstack-kuttl-tests/barbican-worker-ff9447cb9-nmz9j" Dec 03 11:57:58 crc kubenswrapper[5052]: I1203 11:57:58.694404 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ljw82\" (UniqueName: \"kubernetes.io/projected/60f75996-dceb-4640-8fca-96fdcc03ebb9-kube-api-access-ljw82\") pod \"barbican-worker-ff9447cb9-nmz9j\" (UID: \"60f75996-dceb-4640-8fca-96fdcc03ebb9\") " pod="openstack-kuttl-tests/barbican-worker-ff9447cb9-nmz9j" Dec 03 11:57:58 crc kubenswrapper[5052]: I1203 11:57:58.694439 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/60f75996-dceb-4640-8fca-96fdcc03ebb9-config-data\") pod \"barbican-worker-ff9447cb9-nmz9j\" (UID: \"60f75996-dceb-4640-8fca-96fdcc03ebb9\") " pod="openstack-kuttl-tests/barbican-worker-ff9447cb9-nmz9j" Dec 03 11:57:58 crc kubenswrapper[5052]: I1203 11:57:58.694456 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/60f75996-dceb-4640-8fca-96fdcc03ebb9-combined-ca-bundle\") pod \"barbican-worker-ff9447cb9-nmz9j\" (UID: \"60f75996-dceb-4640-8fca-96fdcc03ebb9\") " pod="openstack-kuttl-tests/barbican-worker-ff9447cb9-nmz9j" Dec 03 11:57:58 crc kubenswrapper[5052]: I1203 11:57:58.694541 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/60f75996-dceb-4640-8fca-96fdcc03ebb9-config-data-custom\") pod \"barbican-worker-ff9447cb9-nmz9j\" (UID: \"60f75996-dceb-4640-8fca-96fdcc03ebb9\") " pod="openstack-kuttl-tests/barbican-worker-ff9447cb9-nmz9j" Dec 03 11:57:58 crc kubenswrapper[5052]: I1203 11:57:58.721939 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/barbican-api-8784ffcfb-p694c"] Dec 03 11:57:58 crc kubenswrapper[5052]: I1203 11:57:58.724465 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-api-8784ffcfb-p694c" Dec 03 11:57:58 crc kubenswrapper[5052]: I1203 11:57:58.737299 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"barbican-api-config-data" Dec 03 11:57:58 crc kubenswrapper[5052]: I1203 11:57:58.737567 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-api-8784ffcfb-p694c"] Dec 03 11:57:58 crc kubenswrapper[5052]: I1203 11:57:58.752613 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/watcher-api-0"] Dec 03 11:57:58 crc kubenswrapper[5052]: I1203 11:57:58.760919 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/watcher-applier-0"] Dec 03 11:57:58 crc kubenswrapper[5052]: I1203 11:57:58.794956 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-keystone-listener-6774dbcdb8-wzwf7" Dec 03 11:57:58 crc kubenswrapper[5052]: I1203 11:57:58.796265 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/60f75996-dceb-4640-8fca-96fdcc03ebb9-logs\") pod \"barbican-worker-ff9447cb9-nmz9j\" (UID: \"60f75996-dceb-4640-8fca-96fdcc03ebb9\") " pod="openstack-kuttl-tests/barbican-worker-ff9447cb9-nmz9j" Dec 03 11:57:58 crc kubenswrapper[5052]: I1203 11:57:58.796379 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-ljw82\" (UniqueName: \"kubernetes.io/projected/60f75996-dceb-4640-8fca-96fdcc03ebb9-kube-api-access-ljw82\") pod \"barbican-worker-ff9447cb9-nmz9j\" (UID: \"60f75996-dceb-4640-8fca-96fdcc03ebb9\") " pod="openstack-kuttl-tests/barbican-worker-ff9447cb9-nmz9j" Dec 03 11:57:58 crc kubenswrapper[5052]: I1203 11:57:58.796435 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/60f75996-dceb-4640-8fca-96fdcc03ebb9-config-data\") pod \"barbican-worker-ff9447cb9-nmz9j\" (UID: \"60f75996-dceb-4640-8fca-96fdcc03ebb9\") " pod="openstack-kuttl-tests/barbican-worker-ff9447cb9-nmz9j" Dec 03 11:57:58 crc kubenswrapper[5052]: I1203 11:57:58.796451 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/60f75996-dceb-4640-8fca-96fdcc03ebb9-combined-ca-bundle\") pod \"barbican-worker-ff9447cb9-nmz9j\" (UID: \"60f75996-dceb-4640-8fca-96fdcc03ebb9\") " pod="openstack-kuttl-tests/barbican-worker-ff9447cb9-nmz9j" Dec 03 11:57:58 crc kubenswrapper[5052]: I1203 11:57:58.796577 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/60f75996-dceb-4640-8fca-96fdcc03ebb9-config-data-custom\") pod \"barbican-worker-ff9447cb9-nmz9j\" (UID: \"60f75996-dceb-4640-8fca-96fdcc03ebb9\") " pod="openstack-kuttl-tests/barbican-worker-ff9447cb9-nmz9j" Dec 03 11:57:58 crc kubenswrapper[5052]: I1203 11:57:58.798867 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/60f75996-dceb-4640-8fca-96fdcc03ebb9-logs\") pod \"barbican-worker-ff9447cb9-nmz9j\" (UID: \"60f75996-dceb-4640-8fca-96fdcc03ebb9\") " pod="openstack-kuttl-tests/barbican-worker-ff9447cb9-nmz9j" Dec 03 11:57:58 crc kubenswrapper[5052]: I1203 11:57:58.802631 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/60f75996-dceb-4640-8fca-96fdcc03ebb9-combined-ca-bundle\") pod \"barbican-worker-ff9447cb9-nmz9j\" (UID: \"60f75996-dceb-4640-8fca-96fdcc03ebb9\") " pod="openstack-kuttl-tests/barbican-worker-ff9447cb9-nmz9j" Dec 03 11:57:58 crc kubenswrapper[5052]: I1203 11:57:58.802682 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/60f75996-dceb-4640-8fca-96fdcc03ebb9-config-data-custom\") pod \"barbican-worker-ff9447cb9-nmz9j\" (UID: \"60f75996-dceb-4640-8fca-96fdcc03ebb9\") " pod="openstack-kuttl-tests/barbican-worker-ff9447cb9-nmz9j" Dec 03 11:57:58 crc kubenswrapper[5052]: I1203 11:57:58.803999 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/60f75996-dceb-4640-8fca-96fdcc03ebb9-config-data\") pod \"barbican-worker-ff9447cb9-nmz9j\" (UID: \"60f75996-dceb-4640-8fca-96fdcc03ebb9\") " pod="openstack-kuttl-tests/barbican-worker-ff9447cb9-nmz9j" Dec 03 11:57:58 crc kubenswrapper[5052]: I1203 11:57:58.813891 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-ljw82\" (UniqueName: \"kubernetes.io/projected/60f75996-dceb-4640-8fca-96fdcc03ebb9-kube-api-access-ljw82\") pod \"barbican-worker-ff9447cb9-nmz9j\" (UID: \"60f75996-dceb-4640-8fca-96fdcc03ebb9\") " pod="openstack-kuttl-tests/barbican-worker-ff9447cb9-nmz9j" Dec 03 11:57:58 crc kubenswrapper[5052]: I1203 11:57:58.898661 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/1f1f2013-0dae-4527-8734-ac23d8776153-config-data\") pod \"barbican-api-8784ffcfb-p694c\" (UID: \"1f1f2013-0dae-4527-8734-ac23d8776153\") " pod="openstack-kuttl-tests/barbican-api-8784ffcfb-p694c" Dec 03 11:57:58 crc kubenswrapper[5052]: I1203 11:57:58.899001 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6t85d\" (UniqueName: \"kubernetes.io/projected/1f1f2013-0dae-4527-8734-ac23d8776153-kube-api-access-6t85d\") pod \"barbican-api-8784ffcfb-p694c\" (UID: \"1f1f2013-0dae-4527-8734-ac23d8776153\") " pod="openstack-kuttl-tests/barbican-api-8784ffcfb-p694c" Dec 03 11:57:58 crc kubenswrapper[5052]: I1203 11:57:58.899040 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/1f1f2013-0dae-4527-8734-ac23d8776153-config-data-custom\") pod \"barbican-api-8784ffcfb-p694c\" (UID: \"1f1f2013-0dae-4527-8734-ac23d8776153\") " pod="openstack-kuttl-tests/barbican-api-8784ffcfb-p694c" Dec 03 11:57:58 crc kubenswrapper[5052]: I1203 11:57:58.899054 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/1f1f2013-0dae-4527-8734-ac23d8776153-logs\") pod \"barbican-api-8784ffcfb-p694c\" (UID: \"1f1f2013-0dae-4527-8734-ac23d8776153\") " pod="openstack-kuttl-tests/barbican-api-8784ffcfb-p694c" Dec 03 11:57:58 crc kubenswrapper[5052]: I1203 11:57:58.899090 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1f1f2013-0dae-4527-8734-ac23d8776153-combined-ca-bundle\") pod \"barbican-api-8784ffcfb-p694c\" (UID: \"1f1f2013-0dae-4527-8734-ac23d8776153\") " pod="openstack-kuttl-tests/barbican-api-8784ffcfb-p694c" Dec 03 11:57:59 crc kubenswrapper[5052]: I1203 11:57:59.001106 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/1f1f2013-0dae-4527-8734-ac23d8776153-config-data-custom\") pod \"barbican-api-8784ffcfb-p694c\" (UID: \"1f1f2013-0dae-4527-8734-ac23d8776153\") " pod="openstack-kuttl-tests/barbican-api-8784ffcfb-p694c" Dec 03 11:57:59 crc kubenswrapper[5052]: I1203 11:57:59.001150 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/1f1f2013-0dae-4527-8734-ac23d8776153-logs\") pod \"barbican-api-8784ffcfb-p694c\" (UID: \"1f1f2013-0dae-4527-8734-ac23d8776153\") " pod="openstack-kuttl-tests/barbican-api-8784ffcfb-p694c" Dec 03 11:57:59 crc kubenswrapper[5052]: I1203 11:57:59.001207 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1f1f2013-0dae-4527-8734-ac23d8776153-combined-ca-bundle\") pod \"barbican-api-8784ffcfb-p694c\" (UID: \"1f1f2013-0dae-4527-8734-ac23d8776153\") " pod="openstack-kuttl-tests/barbican-api-8784ffcfb-p694c" Dec 03 11:57:59 crc kubenswrapper[5052]: I1203 11:57:59.001293 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/1f1f2013-0dae-4527-8734-ac23d8776153-config-data\") pod \"barbican-api-8784ffcfb-p694c\" (UID: \"1f1f2013-0dae-4527-8734-ac23d8776153\") " pod="openstack-kuttl-tests/barbican-api-8784ffcfb-p694c" Dec 03 11:57:59 crc kubenswrapper[5052]: I1203 11:57:59.001382 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-6t85d\" (UniqueName: \"kubernetes.io/projected/1f1f2013-0dae-4527-8734-ac23d8776153-kube-api-access-6t85d\") pod \"barbican-api-8784ffcfb-p694c\" (UID: \"1f1f2013-0dae-4527-8734-ac23d8776153\") " pod="openstack-kuttl-tests/barbican-api-8784ffcfb-p694c" Dec 03 11:57:59 crc kubenswrapper[5052]: I1203 11:57:59.002574 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/1f1f2013-0dae-4527-8734-ac23d8776153-logs\") pod \"barbican-api-8784ffcfb-p694c\" (UID: \"1f1f2013-0dae-4527-8734-ac23d8776153\") " pod="openstack-kuttl-tests/barbican-api-8784ffcfb-p694c" Dec 03 11:57:59 crc kubenswrapper[5052]: I1203 11:57:59.011102 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/1f1f2013-0dae-4527-8734-ac23d8776153-config-data-custom\") pod \"barbican-api-8784ffcfb-p694c\" (UID: \"1f1f2013-0dae-4527-8734-ac23d8776153\") " pod="openstack-kuttl-tests/barbican-api-8784ffcfb-p694c" Dec 03 11:57:59 crc kubenswrapper[5052]: I1203 11:57:59.012330 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1f1f2013-0dae-4527-8734-ac23d8776153-combined-ca-bundle\") pod \"barbican-api-8784ffcfb-p694c\" (UID: \"1f1f2013-0dae-4527-8734-ac23d8776153\") " pod="openstack-kuttl-tests/barbican-api-8784ffcfb-p694c" Dec 03 11:57:59 crc kubenswrapper[5052]: I1203 11:57:59.012798 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/1f1f2013-0dae-4527-8734-ac23d8776153-config-data\") pod \"barbican-api-8784ffcfb-p694c\" (UID: \"1f1f2013-0dae-4527-8734-ac23d8776153\") " pod="openstack-kuttl-tests/barbican-api-8784ffcfb-p694c" Dec 03 11:57:59 crc kubenswrapper[5052]: I1203 11:57:59.030797 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-6t85d\" (UniqueName: \"kubernetes.io/projected/1f1f2013-0dae-4527-8734-ac23d8776153-kube-api-access-6t85d\") pod \"barbican-api-8784ffcfb-p694c\" (UID: \"1f1f2013-0dae-4527-8734-ac23d8776153\") " pod="openstack-kuttl-tests/barbican-api-8784ffcfb-p694c" Dec 03 11:57:59 crc kubenswrapper[5052]: I1203 11:57:59.092907 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-worker-ff9447cb9-nmz9j" Dec 03 11:57:59 crc kubenswrapper[5052]: I1203 11:57:59.133954 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-api-8784ffcfb-p694c" Dec 03 11:57:59 crc kubenswrapper[5052]: I1203 11:57:59.228783 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/watcher-api-0" event={"ID":"dc7d33d8-fd6f-466a-9ea2-2a643e314d53","Type":"ContainerStarted","Data":"180e4cefdddfd8cd4d063d16f01531609d3814b7cd2413970bcfd19e05b9b2dd"} Dec 03 11:57:59 crc kubenswrapper[5052]: I1203 11:57:59.228831 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/watcher-api-0" event={"ID":"dc7d33d8-fd6f-466a-9ea2-2a643e314d53","Type":"ContainerStarted","Data":"bd56774d357cd15024dcad0db5097905485f7ba868ebff38e623740f43909232"} Dec 03 11:57:59 crc kubenswrapper[5052]: I1203 11:57:59.259260 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-bootstrap-jhbzq" event={"ID":"c856f6ae-8216-4bd5-a9dd-1d87b546854e","Type":"ContainerStarted","Data":"a77396915b8338d90eddac5eb06104115079d2d8327f9130ff83c33c77a22246"} Dec 03 11:57:59 crc kubenswrapper[5052]: I1203 11:57:59.274793 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/watcher-applier-0" event={"ID":"192fa797-c8b8-42d3-a689-e6a7dbc77282","Type":"ContainerStarted","Data":"b35373cfc82c42a1e9a5c087fd12f5e905f199a9f619266cede30f1f7dd75eaf"} Dec 03 11:57:59 crc kubenswrapper[5052]: I1203 11:57:59.290444 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/keystone-bootstrap-jhbzq" podStartSLOduration=3.290421352 podStartE2EDuration="3.290421352s" podCreationTimestamp="2025-12-03 11:57:56 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:57:59.286886271 +0000 UTC m=+2269.875128035" watchObservedRunningTime="2025-12-03 11:57:59.290421352 +0000 UTC m=+2269.878663126" Dec 03 11:57:59 crc kubenswrapper[5052]: I1203 11:57:59.299335 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-7d9769b856-tzkn2" event={"ID":"60ca40b3-79c7-49cc-9100-0a37c47e4462","Type":"ContainerStarted","Data":"eac316d032f9e574c9c5551e5d62e39ad5141fe5bd8033f5ce21da3b31105028"} Dec 03 11:57:59 crc kubenswrapper[5052]: I1203 11:57:59.299372 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-7d9769b856-tzkn2" event={"ID":"60ca40b3-79c7-49cc-9100-0a37c47e4462","Type":"ContainerStarted","Data":"70d329d677837ae32ed6ea2cf9d76bcf62eb95f82373e7dfdeacdfcbca06156a"} Dec 03 11:57:59 crc kubenswrapper[5052]: I1203 11:57:59.300376 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/placement-7d9769b856-tzkn2" Dec 03 11:57:59 crc kubenswrapper[5052]: I1203 11:57:59.300399 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/placement-7d9769b856-tzkn2" Dec 03 11:57:59 crc kubenswrapper[5052]: I1203 11:57:59.301495 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/watcher-decision-engine-0" event={"ID":"5893db31-2978-45eb-8407-2e4d178954ce","Type":"ContainerStarted","Data":"451ff2577699cff2494d08d669aa519a79d76d38c7b97681ac3e06727f31805b"} Dec 03 11:57:59 crc kubenswrapper[5052]: I1203 11:57:59.431261 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/placement-7d9769b856-tzkn2" podStartSLOduration=2.431238292 podStartE2EDuration="2.431238292s" podCreationTimestamp="2025-12-03 11:57:57 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:57:59.369250028 +0000 UTC m=+2269.957491792" watchObservedRunningTime="2025-12-03 11:57:59.431238292 +0000 UTC m=+2270.019480066" Dec 03 11:57:59 crc kubenswrapper[5052]: I1203 11:57:59.576025 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-keystone-listener-6774dbcdb8-wzwf7"] Dec 03 11:57:59 crc kubenswrapper[5052]: I1203 11:57:59.882728 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-worker-ff9447cb9-nmz9j"] Dec 03 11:58:00 crc kubenswrapper[5052]: I1203 11:58:00.048130 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-api-8784ffcfb-p694c"] Dec 03 11:58:00 crc kubenswrapper[5052]: I1203 11:58:00.334151 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-8784ffcfb-p694c" event={"ID":"1f1f2013-0dae-4527-8734-ac23d8776153","Type":"ContainerStarted","Data":"b9bb37cfd7841690059bca55c66cbaa4d860650c9846d9af571aa4067c855fec"} Dec 03 11:58:00 crc kubenswrapper[5052]: I1203 11:58:00.334481 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-8784ffcfb-p694c" event={"ID":"1f1f2013-0dae-4527-8734-ac23d8776153","Type":"ContainerStarted","Data":"308d3e381c0f505066fb57e757f4b0584c5d0cc0fb3865f3f6b45a7cb1c64d74"} Dec 03 11:58:00 crc kubenswrapper[5052]: I1203 11:58:00.353399 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/watcher-api-0" event={"ID":"dc7d33d8-fd6f-466a-9ea2-2a643e314d53","Type":"ContainerStarted","Data":"69e769ca022be16554b1387944766e2b196a0fc86b6db305eaf0473441b4f1ff"} Dec 03 11:58:00 crc kubenswrapper[5052]: I1203 11:58:00.355087 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/watcher-api-0" Dec 03 11:58:00 crc kubenswrapper[5052]: I1203 11:58:00.362636 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-worker-ff9447cb9-nmz9j" event={"ID":"60f75996-dceb-4640-8fca-96fdcc03ebb9","Type":"ContainerStarted","Data":"3a24f581129d9e74e6cdc6ff9baf1728f121946baea46179b88abf5f28003c5f"} Dec 03 11:58:00 crc kubenswrapper[5052]: I1203 11:58:00.362685 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-worker-ff9447cb9-nmz9j" event={"ID":"60f75996-dceb-4640-8fca-96fdcc03ebb9","Type":"ContainerStarted","Data":"45f8d5c2251f538926e8b98dd1b8e9f03d54f5816e6f1edc0b901925bbb7b98e"} Dec 03 11:58:00 crc kubenswrapper[5052]: I1203 11:58:00.388044 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-keystone-listener-6774dbcdb8-wzwf7" event={"ID":"e09397b4-fabb-4e9d-8695-669bf9f49c77","Type":"ContainerStarted","Data":"f80007b6133c4b4853d49d9ef314cd82a9852973ff7a6cf37381a6ca1b1ec967"} Dec 03 11:58:00 crc kubenswrapper[5052]: I1203 11:58:00.388103 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-keystone-listener-6774dbcdb8-wzwf7" event={"ID":"e09397b4-fabb-4e9d-8695-669bf9f49c77","Type":"ContainerStarted","Data":"32933642a3f541b1b3569edbbb3a29c32da560890311d2484c68ee57f99463a4"} Dec 03 11:58:00 crc kubenswrapper[5052]: I1203 11:58:00.388117 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-keystone-listener-6774dbcdb8-wzwf7" event={"ID":"e09397b4-fabb-4e9d-8695-669bf9f49c77","Type":"ContainerStarted","Data":"d555975059be3bb86bfde48d46336b16ab9e981a5f964ae14484b856f1372aa1"} Dec 03 11:58:00 crc kubenswrapper[5052]: I1203 11:58:00.410073 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/watcher-api-0" podStartSLOduration=3.410040255 podStartE2EDuration="3.410040255s" podCreationTimestamp="2025-12-03 11:57:57 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:58:00.382643021 +0000 UTC m=+2270.970884795" watchObservedRunningTime="2025-12-03 11:58:00.410040255 +0000 UTC m=+2270.998282019" Dec 03 11:58:00 crc kubenswrapper[5052]: I1203 11:58:00.466018 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/barbican-keystone-listener-6774dbcdb8-wzwf7" podStartSLOduration=2.465992576 podStartE2EDuration="2.465992576s" podCreationTimestamp="2025-12-03 11:57:58 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:58:00.43398225 +0000 UTC m=+2271.022224024" watchObservedRunningTime="2025-12-03 11:58:00.465992576 +0000 UTC m=+2271.054234330" Dec 03 11:58:01 crc kubenswrapper[5052]: I1203 11:58:01.399977 5052 generic.go:334] "Generic (PLEG): container finished" podID="16630eec-cd03-42b8-95aa-d9a960c2858c" containerID="9b3146eaddf7ac59c648a8a038db1a5905188e2a8b2f4dc2de4f7459b0197c3c" exitCode=0 Dec 03 11:58:01 crc kubenswrapper[5052]: I1203 11:58:01.400063 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-db-sync-76kbf" event={"ID":"16630eec-cd03-42b8-95aa-d9a960c2858c","Type":"ContainerDied","Data":"9b3146eaddf7ac59c648a8a038db1a5905188e2a8b2f4dc2de4f7459b0197c3c"} Dec 03 11:58:01 crc kubenswrapper[5052]: I1203 11:58:01.404605 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-worker-ff9447cb9-nmz9j" event={"ID":"60f75996-dceb-4640-8fca-96fdcc03ebb9","Type":"ContainerStarted","Data":"5f92af45a915348c684ef49149ebdedefc7b5163a55e465615daf0ad05fa17b8"} Dec 03 11:58:01 crc kubenswrapper[5052]: I1203 11:58:01.413292 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-8784ffcfb-p694c" event={"ID":"1f1f2013-0dae-4527-8734-ac23d8776153","Type":"ContainerStarted","Data":"f7bb5981c82ecfa1fdd697c4703c407c5dfc77e905fb89792ccd44bfd8b55602"} Dec 03 11:58:01 crc kubenswrapper[5052]: I1203 11:58:01.452624 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/barbican-worker-ff9447cb9-nmz9j" podStartSLOduration=3.452601182 podStartE2EDuration="3.452601182s" podCreationTimestamp="2025-12-03 11:57:58 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:58:01.442271666 +0000 UTC m=+2272.030513440" watchObservedRunningTime="2025-12-03 11:58:01.452601182 +0000 UTC m=+2272.040842956" Dec 03 11:58:01 crc kubenswrapper[5052]: I1203 11:58:01.484774 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/barbican-api-8784ffcfb-p694c" podStartSLOduration=3.484733231 podStartE2EDuration="3.484733231s" podCreationTimestamp="2025-12-03 11:57:58 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:58:01.473768557 +0000 UTC m=+2272.062010321" watchObservedRunningTime="2025-12-03 11:58:01.484733231 +0000 UTC m=+2272.072974995" Dec 03 11:58:02 crc kubenswrapper[5052]: I1203 11:58:02.423572 5052 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Dec 03 11:58:02 crc kubenswrapper[5052]: I1203 11:58:02.423874 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/barbican-api-8784ffcfb-p694c" Dec 03 11:58:02 crc kubenswrapper[5052]: I1203 11:58:02.423910 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/barbican-api-8784ffcfb-p694c" Dec 03 11:58:02 crc kubenswrapper[5052]: I1203 11:58:02.494423 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/barbican-api-d6bc8866d-xrhs6"] Dec 03 11:58:02 crc kubenswrapper[5052]: I1203 11:58:02.501103 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-api-d6bc8866d-xrhs6" Dec 03 11:58:02 crc kubenswrapper[5052]: I1203 11:58:02.508415 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-barbican-public-svc" Dec 03 11:58:02 crc kubenswrapper[5052]: I1203 11:58:02.508663 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-barbican-internal-svc" Dec 03 11:58:02 crc kubenswrapper[5052]: I1203 11:58:02.518783 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-api-d6bc8866d-xrhs6"] Dec 03 11:58:02 crc kubenswrapper[5052]: I1203 11:58:02.599448 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/5785b60f-14d2-43c5-95e2-68741c6729f3-config-data\") pod \"barbican-api-d6bc8866d-xrhs6\" (UID: \"5785b60f-14d2-43c5-95e2-68741c6729f3\") " pod="openstack-kuttl-tests/barbican-api-d6bc8866d-xrhs6" Dec 03 11:58:02 crc kubenswrapper[5052]: I1203 11:58:02.599520 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5785b60f-14d2-43c5-95e2-68741c6729f3-combined-ca-bundle\") pod \"barbican-api-d6bc8866d-xrhs6\" (UID: \"5785b60f-14d2-43c5-95e2-68741c6729f3\") " pod="openstack-kuttl-tests/barbican-api-d6bc8866d-xrhs6" Dec 03 11:58:02 crc kubenswrapper[5052]: I1203 11:58:02.599707 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/5785b60f-14d2-43c5-95e2-68741c6729f3-public-tls-certs\") pod \"barbican-api-d6bc8866d-xrhs6\" (UID: \"5785b60f-14d2-43c5-95e2-68741c6729f3\") " pod="openstack-kuttl-tests/barbican-api-d6bc8866d-xrhs6" Dec 03 11:58:02 crc kubenswrapper[5052]: I1203 11:58:02.599781 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/5785b60f-14d2-43c5-95e2-68741c6729f3-logs\") pod \"barbican-api-d6bc8866d-xrhs6\" (UID: \"5785b60f-14d2-43c5-95e2-68741c6729f3\") " pod="openstack-kuttl-tests/barbican-api-d6bc8866d-xrhs6" Dec 03 11:58:02 crc kubenswrapper[5052]: I1203 11:58:02.599928 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wphmx\" (UniqueName: \"kubernetes.io/projected/5785b60f-14d2-43c5-95e2-68741c6729f3-kube-api-access-wphmx\") pod \"barbican-api-d6bc8866d-xrhs6\" (UID: \"5785b60f-14d2-43c5-95e2-68741c6729f3\") " pod="openstack-kuttl-tests/barbican-api-d6bc8866d-xrhs6" Dec 03 11:58:02 crc kubenswrapper[5052]: I1203 11:58:02.599961 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/5785b60f-14d2-43c5-95e2-68741c6729f3-config-data-custom\") pod \"barbican-api-d6bc8866d-xrhs6\" (UID: \"5785b60f-14d2-43c5-95e2-68741c6729f3\") " pod="openstack-kuttl-tests/barbican-api-d6bc8866d-xrhs6" Dec 03 11:58:02 crc kubenswrapper[5052]: I1203 11:58:02.600005 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/5785b60f-14d2-43c5-95e2-68741c6729f3-internal-tls-certs\") pod \"barbican-api-d6bc8866d-xrhs6\" (UID: \"5785b60f-14d2-43c5-95e2-68741c6729f3\") " pod="openstack-kuttl-tests/barbican-api-d6bc8866d-xrhs6" Dec 03 11:58:02 crc kubenswrapper[5052]: I1203 11:58:02.702421 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-wphmx\" (UniqueName: \"kubernetes.io/projected/5785b60f-14d2-43c5-95e2-68741c6729f3-kube-api-access-wphmx\") pod \"barbican-api-d6bc8866d-xrhs6\" (UID: \"5785b60f-14d2-43c5-95e2-68741c6729f3\") " pod="openstack-kuttl-tests/barbican-api-d6bc8866d-xrhs6" Dec 03 11:58:02 crc kubenswrapper[5052]: I1203 11:58:02.702473 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/5785b60f-14d2-43c5-95e2-68741c6729f3-config-data-custom\") pod \"barbican-api-d6bc8866d-xrhs6\" (UID: \"5785b60f-14d2-43c5-95e2-68741c6729f3\") " pod="openstack-kuttl-tests/barbican-api-d6bc8866d-xrhs6" Dec 03 11:58:02 crc kubenswrapper[5052]: I1203 11:58:02.702543 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/5785b60f-14d2-43c5-95e2-68741c6729f3-internal-tls-certs\") pod \"barbican-api-d6bc8866d-xrhs6\" (UID: \"5785b60f-14d2-43c5-95e2-68741c6729f3\") " pod="openstack-kuttl-tests/barbican-api-d6bc8866d-xrhs6" Dec 03 11:58:02 crc kubenswrapper[5052]: I1203 11:58:02.702594 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/5785b60f-14d2-43c5-95e2-68741c6729f3-config-data\") pod \"barbican-api-d6bc8866d-xrhs6\" (UID: \"5785b60f-14d2-43c5-95e2-68741c6729f3\") " pod="openstack-kuttl-tests/barbican-api-d6bc8866d-xrhs6" Dec 03 11:58:02 crc kubenswrapper[5052]: I1203 11:58:02.702627 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5785b60f-14d2-43c5-95e2-68741c6729f3-combined-ca-bundle\") pod \"barbican-api-d6bc8866d-xrhs6\" (UID: \"5785b60f-14d2-43c5-95e2-68741c6729f3\") " pod="openstack-kuttl-tests/barbican-api-d6bc8866d-xrhs6" Dec 03 11:58:02 crc kubenswrapper[5052]: I1203 11:58:02.702688 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/5785b60f-14d2-43c5-95e2-68741c6729f3-public-tls-certs\") pod \"barbican-api-d6bc8866d-xrhs6\" (UID: \"5785b60f-14d2-43c5-95e2-68741c6729f3\") " pod="openstack-kuttl-tests/barbican-api-d6bc8866d-xrhs6" Dec 03 11:58:02 crc kubenswrapper[5052]: I1203 11:58:02.702712 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/5785b60f-14d2-43c5-95e2-68741c6729f3-logs\") pod \"barbican-api-d6bc8866d-xrhs6\" (UID: \"5785b60f-14d2-43c5-95e2-68741c6729f3\") " pod="openstack-kuttl-tests/barbican-api-d6bc8866d-xrhs6" Dec 03 11:58:02 crc kubenswrapper[5052]: I1203 11:58:02.703139 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/5785b60f-14d2-43c5-95e2-68741c6729f3-logs\") pod \"barbican-api-d6bc8866d-xrhs6\" (UID: \"5785b60f-14d2-43c5-95e2-68741c6729f3\") " pod="openstack-kuttl-tests/barbican-api-d6bc8866d-xrhs6" Dec 03 11:58:02 crc kubenswrapper[5052]: I1203 11:58:02.723622 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/5785b60f-14d2-43c5-95e2-68741c6729f3-internal-tls-certs\") pod \"barbican-api-d6bc8866d-xrhs6\" (UID: \"5785b60f-14d2-43c5-95e2-68741c6729f3\") " pod="openstack-kuttl-tests/barbican-api-d6bc8866d-xrhs6" Dec 03 11:58:02 crc kubenswrapper[5052]: I1203 11:58:02.726149 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5785b60f-14d2-43c5-95e2-68741c6729f3-combined-ca-bundle\") pod \"barbican-api-d6bc8866d-xrhs6\" (UID: \"5785b60f-14d2-43c5-95e2-68741c6729f3\") " pod="openstack-kuttl-tests/barbican-api-d6bc8866d-xrhs6" Dec 03 11:58:02 crc kubenswrapper[5052]: I1203 11:58:02.750748 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/5785b60f-14d2-43c5-95e2-68741c6729f3-public-tls-certs\") pod \"barbican-api-d6bc8866d-xrhs6\" (UID: \"5785b60f-14d2-43c5-95e2-68741c6729f3\") " pod="openstack-kuttl-tests/barbican-api-d6bc8866d-xrhs6" Dec 03 11:58:02 crc kubenswrapper[5052]: I1203 11:58:02.756988 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/5785b60f-14d2-43c5-95e2-68741c6729f3-config-data\") pod \"barbican-api-d6bc8866d-xrhs6\" (UID: \"5785b60f-14d2-43c5-95e2-68741c6729f3\") " pod="openstack-kuttl-tests/barbican-api-d6bc8866d-xrhs6" Dec 03 11:58:02 crc kubenswrapper[5052]: I1203 11:58:02.758697 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-wphmx\" (UniqueName: \"kubernetes.io/projected/5785b60f-14d2-43c5-95e2-68741c6729f3-kube-api-access-wphmx\") pod \"barbican-api-d6bc8866d-xrhs6\" (UID: \"5785b60f-14d2-43c5-95e2-68741c6729f3\") " pod="openstack-kuttl-tests/barbican-api-d6bc8866d-xrhs6" Dec 03 11:58:02 crc kubenswrapper[5052]: I1203 11:58:02.777159 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/5785b60f-14d2-43c5-95e2-68741c6729f3-config-data-custom\") pod \"barbican-api-d6bc8866d-xrhs6\" (UID: \"5785b60f-14d2-43c5-95e2-68741c6729f3\") " pod="openstack-kuttl-tests/barbican-api-d6bc8866d-xrhs6" Dec 03 11:58:02 crc kubenswrapper[5052]: I1203 11:58:02.822949 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-api-d6bc8866d-xrhs6" Dec 03 11:58:02 crc kubenswrapper[5052]: I1203 11:58:02.911172 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/watcher-api-0" Dec 03 11:58:03 crc kubenswrapper[5052]: I1203 11:58:03.365590 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/watcher-api-0" Dec 03 11:58:04 crc kubenswrapper[5052]: I1203 11:58:04.031291 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-db-sync-76kbf" Dec 03 11:58:04 crc kubenswrapper[5052]: I1203 11:58:04.129296 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/16630eec-cd03-42b8-95aa-d9a960c2858c-config-data\") pod \"16630eec-cd03-42b8-95aa-d9a960c2858c\" (UID: \"16630eec-cd03-42b8-95aa-d9a960c2858c\") " Dec 03 11:58:04 crc kubenswrapper[5052]: I1203 11:58:04.129544 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/16630eec-cd03-42b8-95aa-d9a960c2858c-db-sync-config-data\") pod \"16630eec-cd03-42b8-95aa-d9a960c2858c\" (UID: \"16630eec-cd03-42b8-95aa-d9a960c2858c\") " Dec 03 11:58:04 crc kubenswrapper[5052]: I1203 11:58:04.129584 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/16630eec-cd03-42b8-95aa-d9a960c2858c-etc-machine-id\") pod \"16630eec-cd03-42b8-95aa-d9a960c2858c\" (UID: \"16630eec-cd03-42b8-95aa-d9a960c2858c\") " Dec 03 11:58:04 crc kubenswrapper[5052]: I1203 11:58:04.129663 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x4z68\" (UniqueName: \"kubernetes.io/projected/16630eec-cd03-42b8-95aa-d9a960c2858c-kube-api-access-x4z68\") pod \"16630eec-cd03-42b8-95aa-d9a960c2858c\" (UID: \"16630eec-cd03-42b8-95aa-d9a960c2858c\") " Dec 03 11:58:04 crc kubenswrapper[5052]: I1203 11:58:04.129810 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/16630eec-cd03-42b8-95aa-d9a960c2858c-scripts\") pod \"16630eec-cd03-42b8-95aa-d9a960c2858c\" (UID: \"16630eec-cd03-42b8-95aa-d9a960c2858c\") " Dec 03 11:58:04 crc kubenswrapper[5052]: I1203 11:58:04.129826 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/16630eec-cd03-42b8-95aa-d9a960c2858c-etc-machine-id" (OuterVolumeSpecName: "etc-machine-id") pod "16630eec-cd03-42b8-95aa-d9a960c2858c" (UID: "16630eec-cd03-42b8-95aa-d9a960c2858c"). InnerVolumeSpecName "etc-machine-id". PluginName "kubernetes.io/host-path", VolumeGidValue "" Dec 03 11:58:04 crc kubenswrapper[5052]: I1203 11:58:04.129900 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/16630eec-cd03-42b8-95aa-d9a960c2858c-combined-ca-bundle\") pod \"16630eec-cd03-42b8-95aa-d9a960c2858c\" (UID: \"16630eec-cd03-42b8-95aa-d9a960c2858c\") " Dec 03 11:58:04 crc kubenswrapper[5052]: I1203 11:58:04.134190 5052 reconciler_common.go:293] "Volume detached for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/16630eec-cd03-42b8-95aa-d9a960c2858c-etc-machine-id\") on node \"crc\" DevicePath \"\"" Dec 03 11:58:04 crc kubenswrapper[5052]: I1203 11:58:04.134548 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/16630eec-cd03-42b8-95aa-d9a960c2858c-db-sync-config-data" (OuterVolumeSpecName: "db-sync-config-data") pod "16630eec-cd03-42b8-95aa-d9a960c2858c" (UID: "16630eec-cd03-42b8-95aa-d9a960c2858c"). InnerVolumeSpecName "db-sync-config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:58:04 crc kubenswrapper[5052]: I1203 11:58:04.134804 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/16630eec-cd03-42b8-95aa-d9a960c2858c-kube-api-access-x4z68" (OuterVolumeSpecName: "kube-api-access-x4z68") pod "16630eec-cd03-42b8-95aa-d9a960c2858c" (UID: "16630eec-cd03-42b8-95aa-d9a960c2858c"). InnerVolumeSpecName "kube-api-access-x4z68". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:58:04 crc kubenswrapper[5052]: I1203 11:58:04.142756 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/16630eec-cd03-42b8-95aa-d9a960c2858c-scripts" (OuterVolumeSpecName: "scripts") pod "16630eec-cd03-42b8-95aa-d9a960c2858c" (UID: "16630eec-cd03-42b8-95aa-d9a960c2858c"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:58:04 crc kubenswrapper[5052]: I1203 11:58:04.150006 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-api-d6bc8866d-xrhs6"] Dec 03 11:58:04 crc kubenswrapper[5052]: I1203 11:58:04.170761 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/16630eec-cd03-42b8-95aa-d9a960c2858c-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "16630eec-cd03-42b8-95aa-d9a960c2858c" (UID: "16630eec-cd03-42b8-95aa-d9a960c2858c"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:58:04 crc kubenswrapper[5052]: I1203 11:58:04.188633 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/16630eec-cd03-42b8-95aa-d9a960c2858c-config-data" (OuterVolumeSpecName: "config-data") pod "16630eec-cd03-42b8-95aa-d9a960c2858c" (UID: "16630eec-cd03-42b8-95aa-d9a960c2858c"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:58:04 crc kubenswrapper[5052]: I1203 11:58:04.235668 5052 reconciler_common.go:293] "Volume detached for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/16630eec-cd03-42b8-95aa-d9a960c2858c-db-sync-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:58:04 crc kubenswrapper[5052]: I1203 11:58:04.235705 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-x4z68\" (UniqueName: \"kubernetes.io/projected/16630eec-cd03-42b8-95aa-d9a960c2858c-kube-api-access-x4z68\") on node \"crc\" DevicePath \"\"" Dec 03 11:58:04 crc kubenswrapper[5052]: I1203 11:58:04.235722 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/16630eec-cd03-42b8-95aa-d9a960c2858c-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:58:04 crc kubenswrapper[5052]: I1203 11:58:04.235741 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/16630eec-cd03-42b8-95aa-d9a960c2858c-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:58:04 crc kubenswrapper[5052]: I1203 11:58:04.235753 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/16630eec-cd03-42b8-95aa-d9a960c2858c-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:58:04 crc kubenswrapper[5052]: I1203 11:58:04.446200 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/watcher-applier-0" event={"ID":"192fa797-c8b8-42d3-a689-e6a7dbc77282","Type":"ContainerStarted","Data":"0df8ce5588ea9142955dd14165fc02005b40b2952f5607fe472ebbf7177b2be8"} Dec 03 11:58:04 crc kubenswrapper[5052]: I1203 11:58:04.448609 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/watcher-decision-engine-0" event={"ID":"5893db31-2978-45eb-8407-2e4d178954ce","Type":"ContainerStarted","Data":"d7086739b41b7d360cbe4227c0a28ecffd42edb2ee30fb5fcc39d0c67f27ead1"} Dec 03 11:58:04 crc kubenswrapper[5052]: I1203 11:58:04.452126 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-d6bc8866d-xrhs6" event={"ID":"5785b60f-14d2-43c5-95e2-68741c6729f3","Type":"ContainerStarted","Data":"9828c63a7f8943e7325b0971d75b7f4ef89e7d95366cacf404cd7f47ac39d61f"} Dec 03 11:58:04 crc kubenswrapper[5052]: I1203 11:58:04.452172 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-d6bc8866d-xrhs6" event={"ID":"5785b60f-14d2-43c5-95e2-68741c6729f3","Type":"ContainerStarted","Data":"3138ba2a51029b0a2134fdcaa1df4d9d80072f97bc869d869f4cae9a03c70434"} Dec 03 11:58:04 crc kubenswrapper[5052]: I1203 11:58:04.454159 5052 generic.go:334] "Generic (PLEG): container finished" podID="c856f6ae-8216-4bd5-a9dd-1d87b546854e" containerID="a77396915b8338d90eddac5eb06104115079d2d8327f9130ff83c33c77a22246" exitCode=0 Dec 03 11:58:04 crc kubenswrapper[5052]: I1203 11:58:04.454223 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-bootstrap-jhbzq" event={"ID":"c856f6ae-8216-4bd5-a9dd-1d87b546854e","Type":"ContainerDied","Data":"a77396915b8338d90eddac5eb06104115079d2d8327f9130ff83c33c77a22246"} Dec 03 11:58:04 crc kubenswrapper[5052]: I1203 11:58:04.456258 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-db-sync-76kbf" Dec 03 11:58:04 crc kubenswrapper[5052]: I1203 11:58:04.456311 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-db-sync-76kbf" event={"ID":"16630eec-cd03-42b8-95aa-d9a960c2858c","Type":"ContainerDied","Data":"eaaf88ceaaee19b616e26866996ae2dadbb489e5568807f6dc1d1492084fb00e"} Dec 03 11:58:04 crc kubenswrapper[5052]: I1203 11:58:04.456346 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="eaaf88ceaaee19b616e26866996ae2dadbb489e5568807f6dc1d1492084fb00e" Dec 03 11:58:04 crc kubenswrapper[5052]: I1203 11:58:04.492718 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/watcher-decision-engine-0" podStartSLOduration=2.339837227 podStartE2EDuration="7.492701165s" podCreationTimestamp="2025-12-03 11:57:57 +0000 UTC" firstStartedPulling="2025-12-03 11:57:58.569792569 +0000 UTC m=+2269.158034343" lastFinishedPulling="2025-12-03 11:58:03.722656517 +0000 UTC m=+2274.310898281" observedRunningTime="2025-12-03 11:58:04.490325687 +0000 UTC m=+2275.078567481" watchObservedRunningTime="2025-12-03 11:58:04.492701165 +0000 UTC m=+2275.080942929" Dec 03 11:58:04 crc kubenswrapper[5052]: I1203 11:58:04.496248 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/watcher-applier-0" podStartSLOduration=2.437501202 podStartE2EDuration="7.496237006s" podCreationTimestamp="2025-12-03 11:57:57 +0000 UTC" firstStartedPulling="2025-12-03 11:57:58.663438859 +0000 UTC m=+2269.251680623" lastFinishedPulling="2025-12-03 11:58:03.722174653 +0000 UTC m=+2274.310416427" observedRunningTime="2025-12-03 11:58:04.468032619 +0000 UTC m=+2275.056274393" watchObservedRunningTime="2025-12-03 11:58:04.496237006 +0000 UTC m=+2275.084478770" Dec 03 11:58:05 crc kubenswrapper[5052]: I1203 11:58:05.289013 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 11:58:05 crc kubenswrapper[5052]: E1203 11:58:05.289837 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="16630eec-cd03-42b8-95aa-d9a960c2858c" containerName="cinder-db-sync" Dec 03 11:58:05 crc kubenswrapper[5052]: I1203 11:58:05.289856 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="16630eec-cd03-42b8-95aa-d9a960c2858c" containerName="cinder-db-sync" Dec 03 11:58:05 crc kubenswrapper[5052]: I1203 11:58:05.290097 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="16630eec-cd03-42b8-95aa-d9a960c2858c" containerName="cinder-db-sync" Dec 03 11:58:05 crc kubenswrapper[5052]: I1203 11:58:05.291363 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:58:05 crc kubenswrapper[5052]: I1203 11:58:05.295816 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cinder-scheduler-config-data" Dec 03 11:58:05 crc kubenswrapper[5052]: I1203 11:58:05.295995 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cinder-cinder-dockercfg-vq5vw" Dec 03 11:58:05 crc kubenswrapper[5052]: I1203 11:58:05.296074 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cinder-scripts" Dec 03 11:58:05 crc kubenswrapper[5052]: I1203 11:58:05.296824 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cinder-config-data" Dec 03 11:58:05 crc kubenswrapper[5052]: I1203 11:58:05.300091 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 11:58:05 crc kubenswrapper[5052]: I1203 11:58:05.354561 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f8ecfb0e-6d4e-4380-8fd3-5f39dd783d14-combined-ca-bundle\") pod \"cinder-scheduler-0\" (UID: \"f8ecfb0e-6d4e-4380-8fd3-5f39dd783d14\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:58:05 crc kubenswrapper[5052]: I1203 11:58:05.354673 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/f8ecfb0e-6d4e-4380-8fd3-5f39dd783d14-scripts\") pod \"cinder-scheduler-0\" (UID: \"f8ecfb0e-6d4e-4380-8fd3-5f39dd783d14\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:58:05 crc kubenswrapper[5052]: I1203 11:58:05.354716 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rzv9m\" (UniqueName: \"kubernetes.io/projected/f8ecfb0e-6d4e-4380-8fd3-5f39dd783d14-kube-api-access-rzv9m\") pod \"cinder-scheduler-0\" (UID: \"f8ecfb0e-6d4e-4380-8fd3-5f39dd783d14\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:58:05 crc kubenswrapper[5052]: I1203 11:58:05.354778 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/f8ecfb0e-6d4e-4380-8fd3-5f39dd783d14-etc-machine-id\") pod \"cinder-scheduler-0\" (UID: \"f8ecfb0e-6d4e-4380-8fd3-5f39dd783d14\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:58:05 crc kubenswrapper[5052]: I1203 11:58:05.354829 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/f8ecfb0e-6d4e-4380-8fd3-5f39dd783d14-config-data\") pod \"cinder-scheduler-0\" (UID: \"f8ecfb0e-6d4e-4380-8fd3-5f39dd783d14\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:58:05 crc kubenswrapper[5052]: I1203 11:58:05.354987 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/f8ecfb0e-6d4e-4380-8fd3-5f39dd783d14-config-data-custom\") pod \"cinder-scheduler-0\" (UID: \"f8ecfb0e-6d4e-4380-8fd3-5f39dd783d14\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:58:05 crc kubenswrapper[5052]: I1203 11:58:05.362040 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 11:58:05 crc kubenswrapper[5052]: I1203 11:58:05.363868 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:58:05 crc kubenswrapper[5052]: I1203 11:58:05.368404 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cinder-api-config-data" Dec 03 11:58:05 crc kubenswrapper[5052]: I1203 11:58:05.387126 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 11:58:05 crc kubenswrapper[5052]: I1203 11:58:05.456705 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f8ecfb0e-6d4e-4380-8fd3-5f39dd783d14-combined-ca-bundle\") pod \"cinder-scheduler-0\" (UID: \"f8ecfb0e-6d4e-4380-8fd3-5f39dd783d14\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:58:05 crc kubenswrapper[5052]: I1203 11:58:05.456958 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/f8ecfb0e-6d4e-4380-8fd3-5f39dd783d14-scripts\") pod \"cinder-scheduler-0\" (UID: \"f8ecfb0e-6d4e-4380-8fd3-5f39dd783d14\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:58:05 crc kubenswrapper[5052]: I1203 11:58:05.457087 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-rzv9m\" (UniqueName: \"kubernetes.io/projected/f8ecfb0e-6d4e-4380-8fd3-5f39dd783d14-kube-api-access-rzv9m\") pod \"cinder-scheduler-0\" (UID: \"f8ecfb0e-6d4e-4380-8fd3-5f39dd783d14\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:58:05 crc kubenswrapper[5052]: I1203 11:58:05.457181 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9bd9b8fa-d754-4e26-b66c-f06555f245ee-combined-ca-bundle\") pod \"cinder-api-0\" (UID: \"9bd9b8fa-d754-4e26-b66c-f06555f245ee\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:58:05 crc kubenswrapper[5052]: I1203 11:58:05.457385 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/9bd9b8fa-d754-4e26-b66c-f06555f245ee-logs\") pod \"cinder-api-0\" (UID: \"9bd9b8fa-d754-4e26-b66c-f06555f245ee\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:58:05 crc kubenswrapper[5052]: I1203 11:58:05.457447 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/9bd9b8fa-d754-4e26-b66c-f06555f245ee-config-data-custom\") pod \"cinder-api-0\" (UID: \"9bd9b8fa-d754-4e26-b66c-f06555f245ee\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:58:05 crc kubenswrapper[5052]: I1203 11:58:05.457557 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xjhf8\" (UniqueName: \"kubernetes.io/projected/9bd9b8fa-d754-4e26-b66c-f06555f245ee-kube-api-access-xjhf8\") pod \"cinder-api-0\" (UID: \"9bd9b8fa-d754-4e26-b66c-f06555f245ee\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:58:05 crc kubenswrapper[5052]: I1203 11:58:05.457604 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/f8ecfb0e-6d4e-4380-8fd3-5f39dd783d14-etc-machine-id\") pod \"cinder-scheduler-0\" (UID: \"f8ecfb0e-6d4e-4380-8fd3-5f39dd783d14\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:58:05 crc kubenswrapper[5052]: I1203 11:58:05.457674 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/f8ecfb0e-6d4e-4380-8fd3-5f39dd783d14-etc-machine-id\") pod \"cinder-scheduler-0\" (UID: \"f8ecfb0e-6d4e-4380-8fd3-5f39dd783d14\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:58:05 crc kubenswrapper[5052]: I1203 11:58:05.457734 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/f8ecfb0e-6d4e-4380-8fd3-5f39dd783d14-config-data\") pod \"cinder-scheduler-0\" (UID: \"f8ecfb0e-6d4e-4380-8fd3-5f39dd783d14\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:58:05 crc kubenswrapper[5052]: I1203 11:58:05.457916 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/9bd9b8fa-d754-4e26-b66c-f06555f245ee-etc-machine-id\") pod \"cinder-api-0\" (UID: \"9bd9b8fa-d754-4e26-b66c-f06555f245ee\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:58:05 crc kubenswrapper[5052]: I1203 11:58:05.457958 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/9bd9b8fa-d754-4e26-b66c-f06555f245ee-scripts\") pod \"cinder-api-0\" (UID: \"9bd9b8fa-d754-4e26-b66c-f06555f245ee\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:58:05 crc kubenswrapper[5052]: I1203 11:58:05.458020 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/9bd9b8fa-d754-4e26-b66c-f06555f245ee-config-data\") pod \"cinder-api-0\" (UID: \"9bd9b8fa-d754-4e26-b66c-f06555f245ee\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:58:05 crc kubenswrapper[5052]: I1203 11:58:05.458068 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/f8ecfb0e-6d4e-4380-8fd3-5f39dd783d14-config-data-custom\") pod \"cinder-scheduler-0\" (UID: \"f8ecfb0e-6d4e-4380-8fd3-5f39dd783d14\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:58:05 crc kubenswrapper[5052]: I1203 11:58:05.462274 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/f8ecfb0e-6d4e-4380-8fd3-5f39dd783d14-scripts\") pod \"cinder-scheduler-0\" (UID: \"f8ecfb0e-6d4e-4380-8fd3-5f39dd783d14\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:58:05 crc kubenswrapper[5052]: I1203 11:58:05.462371 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/f8ecfb0e-6d4e-4380-8fd3-5f39dd783d14-config-data-custom\") pod \"cinder-scheduler-0\" (UID: \"f8ecfb0e-6d4e-4380-8fd3-5f39dd783d14\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:58:05 crc kubenswrapper[5052]: I1203 11:58:05.463374 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/f8ecfb0e-6d4e-4380-8fd3-5f39dd783d14-config-data\") pod \"cinder-scheduler-0\" (UID: \"f8ecfb0e-6d4e-4380-8fd3-5f39dd783d14\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:58:05 crc kubenswrapper[5052]: I1203 11:58:05.473944 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f8ecfb0e-6d4e-4380-8fd3-5f39dd783d14-combined-ca-bundle\") pod \"cinder-scheduler-0\" (UID: \"f8ecfb0e-6d4e-4380-8fd3-5f39dd783d14\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:58:05 crc kubenswrapper[5052]: I1203 11:58:05.489094 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-d6bc8866d-xrhs6" event={"ID":"5785b60f-14d2-43c5-95e2-68741c6729f3","Type":"ContainerStarted","Data":"a7bae4cc8cec979afcf2817d0be9cd0e1558b2c480f44fd3bf85f63c239d1a5c"} Dec 03 11:58:05 crc kubenswrapper[5052]: I1203 11:58:05.489688 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/barbican-api-d6bc8866d-xrhs6" Dec 03 11:58:05 crc kubenswrapper[5052]: I1203 11:58:05.490260 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-rzv9m\" (UniqueName: \"kubernetes.io/projected/f8ecfb0e-6d4e-4380-8fd3-5f39dd783d14-kube-api-access-rzv9m\") pod \"cinder-scheduler-0\" (UID: \"f8ecfb0e-6d4e-4380-8fd3-5f39dd783d14\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:58:05 crc kubenswrapper[5052]: I1203 11:58:05.533329 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/barbican-api-d6bc8866d-xrhs6" podStartSLOduration=3.533307636 podStartE2EDuration="3.533307636s" podCreationTimestamp="2025-12-03 11:58:02 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:58:05.519911492 +0000 UTC m=+2276.108153256" watchObservedRunningTime="2025-12-03 11:58:05.533307636 +0000 UTC m=+2276.121549400" Dec 03 11:58:05 crc kubenswrapper[5052]: I1203 11:58:05.559714 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/9bd9b8fa-d754-4e26-b66c-f06555f245ee-etc-machine-id\") pod \"cinder-api-0\" (UID: \"9bd9b8fa-d754-4e26-b66c-f06555f245ee\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:58:05 crc kubenswrapper[5052]: I1203 11:58:05.559987 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/9bd9b8fa-d754-4e26-b66c-f06555f245ee-scripts\") pod \"cinder-api-0\" (UID: \"9bd9b8fa-d754-4e26-b66c-f06555f245ee\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:58:05 crc kubenswrapper[5052]: I1203 11:58:05.560094 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/9bd9b8fa-d754-4e26-b66c-f06555f245ee-config-data\") pod \"cinder-api-0\" (UID: \"9bd9b8fa-d754-4e26-b66c-f06555f245ee\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:58:05 crc kubenswrapper[5052]: I1203 11:58:05.560237 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9bd9b8fa-d754-4e26-b66c-f06555f245ee-combined-ca-bundle\") pod \"cinder-api-0\" (UID: \"9bd9b8fa-d754-4e26-b66c-f06555f245ee\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:58:05 crc kubenswrapper[5052]: I1203 11:58:05.560336 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/9bd9b8fa-d754-4e26-b66c-f06555f245ee-logs\") pod \"cinder-api-0\" (UID: \"9bd9b8fa-d754-4e26-b66c-f06555f245ee\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:58:05 crc kubenswrapper[5052]: I1203 11:58:05.560404 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/9bd9b8fa-d754-4e26-b66c-f06555f245ee-config-data-custom\") pod \"cinder-api-0\" (UID: \"9bd9b8fa-d754-4e26-b66c-f06555f245ee\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:58:05 crc kubenswrapper[5052]: I1203 11:58:05.560483 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-xjhf8\" (UniqueName: \"kubernetes.io/projected/9bd9b8fa-d754-4e26-b66c-f06555f245ee-kube-api-access-xjhf8\") pod \"cinder-api-0\" (UID: \"9bd9b8fa-d754-4e26-b66c-f06555f245ee\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:58:05 crc kubenswrapper[5052]: I1203 11:58:05.561632 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/9bd9b8fa-d754-4e26-b66c-f06555f245ee-etc-machine-id\") pod \"cinder-api-0\" (UID: \"9bd9b8fa-d754-4e26-b66c-f06555f245ee\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:58:05 crc kubenswrapper[5052]: I1203 11:58:05.562061 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/9bd9b8fa-d754-4e26-b66c-f06555f245ee-logs\") pod \"cinder-api-0\" (UID: \"9bd9b8fa-d754-4e26-b66c-f06555f245ee\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:58:05 crc kubenswrapper[5052]: I1203 11:58:05.567356 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9bd9b8fa-d754-4e26-b66c-f06555f245ee-combined-ca-bundle\") pod \"cinder-api-0\" (UID: \"9bd9b8fa-d754-4e26-b66c-f06555f245ee\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:58:05 crc kubenswrapper[5052]: I1203 11:58:05.568247 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/9bd9b8fa-d754-4e26-b66c-f06555f245ee-config-data-custom\") pod \"cinder-api-0\" (UID: \"9bd9b8fa-d754-4e26-b66c-f06555f245ee\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:58:05 crc kubenswrapper[5052]: I1203 11:58:05.594547 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/9bd9b8fa-d754-4e26-b66c-f06555f245ee-config-data\") pod \"cinder-api-0\" (UID: \"9bd9b8fa-d754-4e26-b66c-f06555f245ee\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:58:05 crc kubenswrapper[5052]: I1203 11:58:05.594900 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/9bd9b8fa-d754-4e26-b66c-f06555f245ee-scripts\") pod \"cinder-api-0\" (UID: \"9bd9b8fa-d754-4e26-b66c-f06555f245ee\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:58:05 crc kubenswrapper[5052]: I1203 11:58:05.595138 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-xjhf8\" (UniqueName: \"kubernetes.io/projected/9bd9b8fa-d754-4e26-b66c-f06555f245ee-kube-api-access-xjhf8\") pod \"cinder-api-0\" (UID: \"9bd9b8fa-d754-4e26-b66c-f06555f245ee\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:58:05 crc kubenswrapper[5052]: I1203 11:58:05.617937 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:58:05 crc kubenswrapper[5052]: I1203 11:58:05.694168 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:58:06 crc kubenswrapper[5052]: I1203 11:58:06.069013 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-bootstrap-jhbzq" Dec 03 11:58:06 crc kubenswrapper[5052]: I1203 11:58:06.191275 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/c856f6ae-8216-4bd5-a9dd-1d87b546854e-fernet-keys\") pod \"c856f6ae-8216-4bd5-a9dd-1d87b546854e\" (UID: \"c856f6ae-8216-4bd5-a9dd-1d87b546854e\") " Dec 03 11:58:06 crc kubenswrapper[5052]: I1203 11:58:06.191318 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c856f6ae-8216-4bd5-a9dd-1d87b546854e-combined-ca-bundle\") pod \"c856f6ae-8216-4bd5-a9dd-1d87b546854e\" (UID: \"c856f6ae-8216-4bd5-a9dd-1d87b546854e\") " Dec 03 11:58:06 crc kubenswrapper[5052]: I1203 11:58:06.191338 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v65wg\" (UniqueName: \"kubernetes.io/projected/c856f6ae-8216-4bd5-a9dd-1d87b546854e-kube-api-access-v65wg\") pod \"c856f6ae-8216-4bd5-a9dd-1d87b546854e\" (UID: \"c856f6ae-8216-4bd5-a9dd-1d87b546854e\") " Dec 03 11:58:06 crc kubenswrapper[5052]: I1203 11:58:06.191425 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/c856f6ae-8216-4bd5-a9dd-1d87b546854e-config-data\") pod \"c856f6ae-8216-4bd5-a9dd-1d87b546854e\" (UID: \"c856f6ae-8216-4bd5-a9dd-1d87b546854e\") " Dec 03 11:58:06 crc kubenswrapper[5052]: I1203 11:58:06.191447 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/c856f6ae-8216-4bd5-a9dd-1d87b546854e-credential-keys\") pod \"c856f6ae-8216-4bd5-a9dd-1d87b546854e\" (UID: \"c856f6ae-8216-4bd5-a9dd-1d87b546854e\") " Dec 03 11:58:06 crc kubenswrapper[5052]: I1203 11:58:06.191540 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/c856f6ae-8216-4bd5-a9dd-1d87b546854e-scripts\") pod \"c856f6ae-8216-4bd5-a9dd-1d87b546854e\" (UID: \"c856f6ae-8216-4bd5-a9dd-1d87b546854e\") " Dec 03 11:58:06 crc kubenswrapper[5052]: I1203 11:58:06.201116 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c856f6ae-8216-4bd5-a9dd-1d87b546854e-scripts" (OuterVolumeSpecName: "scripts") pod "c856f6ae-8216-4bd5-a9dd-1d87b546854e" (UID: "c856f6ae-8216-4bd5-a9dd-1d87b546854e"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:58:06 crc kubenswrapper[5052]: I1203 11:58:06.201290 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c856f6ae-8216-4bd5-a9dd-1d87b546854e-credential-keys" (OuterVolumeSpecName: "credential-keys") pod "c856f6ae-8216-4bd5-a9dd-1d87b546854e" (UID: "c856f6ae-8216-4bd5-a9dd-1d87b546854e"). InnerVolumeSpecName "credential-keys". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:58:06 crc kubenswrapper[5052]: I1203 11:58:06.207126 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c856f6ae-8216-4bd5-a9dd-1d87b546854e-fernet-keys" (OuterVolumeSpecName: "fernet-keys") pod "c856f6ae-8216-4bd5-a9dd-1d87b546854e" (UID: "c856f6ae-8216-4bd5-a9dd-1d87b546854e"). InnerVolumeSpecName "fernet-keys". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:58:06 crc kubenswrapper[5052]: I1203 11:58:06.213807 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c856f6ae-8216-4bd5-a9dd-1d87b546854e-kube-api-access-v65wg" (OuterVolumeSpecName: "kube-api-access-v65wg") pod "c856f6ae-8216-4bd5-a9dd-1d87b546854e" (UID: "c856f6ae-8216-4bd5-a9dd-1d87b546854e"). InnerVolumeSpecName "kube-api-access-v65wg". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:58:06 crc kubenswrapper[5052]: I1203 11:58:06.238065 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c856f6ae-8216-4bd5-a9dd-1d87b546854e-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "c856f6ae-8216-4bd5-a9dd-1d87b546854e" (UID: "c856f6ae-8216-4bd5-a9dd-1d87b546854e"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:58:06 crc kubenswrapper[5052]: I1203 11:58:06.255448 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c856f6ae-8216-4bd5-a9dd-1d87b546854e-config-data" (OuterVolumeSpecName: "config-data") pod "c856f6ae-8216-4bd5-a9dd-1d87b546854e" (UID: "c856f6ae-8216-4bd5-a9dd-1d87b546854e"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:58:06 crc kubenswrapper[5052]: I1203 11:58:06.285810 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 11:58:06 crc kubenswrapper[5052]: I1203 11:58:06.294346 5052 reconciler_common.go:293] "Volume detached for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/c856f6ae-8216-4bd5-a9dd-1d87b546854e-fernet-keys\") on node \"crc\" DevicePath \"\"" Dec 03 11:58:06 crc kubenswrapper[5052]: I1203 11:58:06.294375 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-v65wg\" (UniqueName: \"kubernetes.io/projected/c856f6ae-8216-4bd5-a9dd-1d87b546854e-kube-api-access-v65wg\") on node \"crc\" DevicePath \"\"" Dec 03 11:58:06 crc kubenswrapper[5052]: I1203 11:58:06.294388 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c856f6ae-8216-4bd5-a9dd-1d87b546854e-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:58:06 crc kubenswrapper[5052]: I1203 11:58:06.294402 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/c856f6ae-8216-4bd5-a9dd-1d87b546854e-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:58:06 crc kubenswrapper[5052]: I1203 11:58:06.294413 5052 reconciler_common.go:293] "Volume detached for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/c856f6ae-8216-4bd5-a9dd-1d87b546854e-credential-keys\") on node \"crc\" DevicePath \"\"" Dec 03 11:58:06 crc kubenswrapper[5052]: I1203 11:58:06.294424 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/c856f6ae-8216-4bd5-a9dd-1d87b546854e-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:58:06 crc kubenswrapper[5052]: I1203 11:58:06.454950 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 11:58:06 crc kubenswrapper[5052]: W1203 11:58:06.477807 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf8ecfb0e_6d4e_4380_8fd3_5f39dd783d14.slice/crio-c248b5c6f06ff1b7b8ce6af52071bd8e90fbffc7f19054833fa49f6acc52c28d WatchSource:0}: Error finding container c248b5c6f06ff1b7b8ce6af52071bd8e90fbffc7f19054833fa49f6acc52c28d: Status 404 returned error can't find the container with id c248b5c6f06ff1b7b8ce6af52071bd8e90fbffc7f19054833fa49f6acc52c28d Dec 03 11:58:06 crc kubenswrapper[5052]: I1203 11:58:06.520813 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"9bd9b8fa-d754-4e26-b66c-f06555f245ee","Type":"ContainerStarted","Data":"3e8125429b69c318d2d290f5a4b930874ff02d827635dedcbc5482f41978954f"} Dec 03 11:58:06 crc kubenswrapper[5052]: I1203 11:58:06.535744 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"f8ecfb0e-6d4e-4380-8fd3-5f39dd783d14","Type":"ContainerStarted","Data":"c248b5c6f06ff1b7b8ce6af52071bd8e90fbffc7f19054833fa49f6acc52c28d"} Dec 03 11:58:06 crc kubenswrapper[5052]: I1203 11:58:06.572710 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-bootstrap-jhbzq" Dec 03 11:58:06 crc kubenswrapper[5052]: I1203 11:58:06.574715 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-bootstrap-jhbzq" event={"ID":"c856f6ae-8216-4bd5-a9dd-1d87b546854e","Type":"ContainerDied","Data":"cc2a1588ce159bc7153ef7c3f5b7ed3bf02b793e16744abb26c49b8e3ea42995"} Dec 03 11:58:06 crc kubenswrapper[5052]: I1203 11:58:06.574758 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cc2a1588ce159bc7153ef7c3f5b7ed3bf02b793e16744abb26c49b8e3ea42995" Dec 03 11:58:06 crc kubenswrapper[5052]: I1203 11:58:06.574784 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/barbican-api-d6bc8866d-xrhs6" Dec 03 11:58:06 crc kubenswrapper[5052]: I1203 11:58:06.730879 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/keystone-bd5c68479-wvv6d"] Dec 03 11:58:06 crc kubenswrapper[5052]: E1203 11:58:06.731679 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="c856f6ae-8216-4bd5-a9dd-1d87b546854e" containerName="keystone-bootstrap" Dec 03 11:58:06 crc kubenswrapper[5052]: I1203 11:58:06.731698 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="c856f6ae-8216-4bd5-a9dd-1d87b546854e" containerName="keystone-bootstrap" Dec 03 11:58:06 crc kubenswrapper[5052]: I1203 11:58:06.731911 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="c856f6ae-8216-4bd5-a9dd-1d87b546854e" containerName="keystone-bootstrap" Dec 03 11:58:06 crc kubenswrapper[5052]: I1203 11:58:06.732707 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-bd5c68479-wvv6d" Dec 03 11:58:06 crc kubenswrapper[5052]: I1203 11:58:06.735612 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone-scripts" Dec 03 11:58:06 crc kubenswrapper[5052]: I1203 11:58:06.735928 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-keystone-public-svc" Dec 03 11:58:06 crc kubenswrapper[5052]: I1203 11:58:06.736183 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone-keystone-dockercfg-csztk" Dec 03 11:58:06 crc kubenswrapper[5052]: I1203 11:58:06.736426 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone-config-data" Dec 03 11:58:06 crc kubenswrapper[5052]: I1203 11:58:06.736646 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone" Dec 03 11:58:06 crc kubenswrapper[5052]: I1203 11:58:06.736874 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-keystone-internal-svc" Dec 03 11:58:06 crc kubenswrapper[5052]: I1203 11:58:06.760404 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-bd5c68479-wvv6d"] Dec 03 11:58:06 crc kubenswrapper[5052]: I1203 11:58:06.811556 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/0432c361-154a-46a9-b675-04267061f13b-credential-keys\") pod \"keystone-bd5c68479-wvv6d\" (UID: \"0432c361-154a-46a9-b675-04267061f13b\") " pod="openstack-kuttl-tests/keystone-bd5c68479-wvv6d" Dec 03 11:58:06 crc kubenswrapper[5052]: I1203 11:58:06.811617 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/0432c361-154a-46a9-b675-04267061f13b-public-tls-certs\") pod \"keystone-bd5c68479-wvv6d\" (UID: \"0432c361-154a-46a9-b675-04267061f13b\") " pod="openstack-kuttl-tests/keystone-bd5c68479-wvv6d" Dec 03 11:58:06 crc kubenswrapper[5052]: I1203 11:58:06.811683 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/0432c361-154a-46a9-b675-04267061f13b-config-data\") pod \"keystone-bd5c68479-wvv6d\" (UID: \"0432c361-154a-46a9-b675-04267061f13b\") " pod="openstack-kuttl-tests/keystone-bd5c68479-wvv6d" Dec 03 11:58:06 crc kubenswrapper[5052]: I1203 11:58:06.811733 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/0432c361-154a-46a9-b675-04267061f13b-internal-tls-certs\") pod \"keystone-bd5c68479-wvv6d\" (UID: \"0432c361-154a-46a9-b675-04267061f13b\") " pod="openstack-kuttl-tests/keystone-bd5c68479-wvv6d" Dec 03 11:58:06 crc kubenswrapper[5052]: I1203 11:58:06.811803 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/0432c361-154a-46a9-b675-04267061f13b-scripts\") pod \"keystone-bd5c68479-wvv6d\" (UID: \"0432c361-154a-46a9-b675-04267061f13b\") " pod="openstack-kuttl-tests/keystone-bd5c68479-wvv6d" Dec 03 11:58:06 crc kubenswrapper[5052]: I1203 11:58:06.811836 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0432c361-154a-46a9-b675-04267061f13b-combined-ca-bundle\") pod \"keystone-bd5c68479-wvv6d\" (UID: \"0432c361-154a-46a9-b675-04267061f13b\") " pod="openstack-kuttl-tests/keystone-bd5c68479-wvv6d" Dec 03 11:58:06 crc kubenswrapper[5052]: I1203 11:58:06.811863 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/0432c361-154a-46a9-b675-04267061f13b-fernet-keys\") pod \"keystone-bd5c68479-wvv6d\" (UID: \"0432c361-154a-46a9-b675-04267061f13b\") " pod="openstack-kuttl-tests/keystone-bd5c68479-wvv6d" Dec 03 11:58:06 crc kubenswrapper[5052]: I1203 11:58:06.811910 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qfflm\" (UniqueName: \"kubernetes.io/projected/0432c361-154a-46a9-b675-04267061f13b-kube-api-access-qfflm\") pod \"keystone-bd5c68479-wvv6d\" (UID: \"0432c361-154a-46a9-b675-04267061f13b\") " pod="openstack-kuttl-tests/keystone-bd5c68479-wvv6d" Dec 03 11:58:06 crc kubenswrapper[5052]: I1203 11:58:06.913013 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/0432c361-154a-46a9-b675-04267061f13b-config-data\") pod \"keystone-bd5c68479-wvv6d\" (UID: \"0432c361-154a-46a9-b675-04267061f13b\") " pod="openstack-kuttl-tests/keystone-bd5c68479-wvv6d" Dec 03 11:58:06 crc kubenswrapper[5052]: I1203 11:58:06.913108 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/0432c361-154a-46a9-b675-04267061f13b-internal-tls-certs\") pod \"keystone-bd5c68479-wvv6d\" (UID: \"0432c361-154a-46a9-b675-04267061f13b\") " pod="openstack-kuttl-tests/keystone-bd5c68479-wvv6d" Dec 03 11:58:06 crc kubenswrapper[5052]: I1203 11:58:06.913192 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/0432c361-154a-46a9-b675-04267061f13b-scripts\") pod \"keystone-bd5c68479-wvv6d\" (UID: \"0432c361-154a-46a9-b675-04267061f13b\") " pod="openstack-kuttl-tests/keystone-bd5c68479-wvv6d" Dec 03 11:58:06 crc kubenswrapper[5052]: I1203 11:58:06.913228 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0432c361-154a-46a9-b675-04267061f13b-combined-ca-bundle\") pod \"keystone-bd5c68479-wvv6d\" (UID: \"0432c361-154a-46a9-b675-04267061f13b\") " pod="openstack-kuttl-tests/keystone-bd5c68479-wvv6d" Dec 03 11:58:06 crc kubenswrapper[5052]: I1203 11:58:06.913258 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/0432c361-154a-46a9-b675-04267061f13b-fernet-keys\") pod \"keystone-bd5c68479-wvv6d\" (UID: \"0432c361-154a-46a9-b675-04267061f13b\") " pod="openstack-kuttl-tests/keystone-bd5c68479-wvv6d" Dec 03 11:58:06 crc kubenswrapper[5052]: I1203 11:58:06.913314 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-qfflm\" (UniqueName: \"kubernetes.io/projected/0432c361-154a-46a9-b675-04267061f13b-kube-api-access-qfflm\") pod \"keystone-bd5c68479-wvv6d\" (UID: \"0432c361-154a-46a9-b675-04267061f13b\") " pod="openstack-kuttl-tests/keystone-bd5c68479-wvv6d" Dec 03 11:58:06 crc kubenswrapper[5052]: I1203 11:58:06.913366 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/0432c361-154a-46a9-b675-04267061f13b-credential-keys\") pod \"keystone-bd5c68479-wvv6d\" (UID: \"0432c361-154a-46a9-b675-04267061f13b\") " pod="openstack-kuttl-tests/keystone-bd5c68479-wvv6d" Dec 03 11:58:06 crc kubenswrapper[5052]: I1203 11:58:06.913401 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/0432c361-154a-46a9-b675-04267061f13b-public-tls-certs\") pod \"keystone-bd5c68479-wvv6d\" (UID: \"0432c361-154a-46a9-b675-04267061f13b\") " pod="openstack-kuttl-tests/keystone-bd5c68479-wvv6d" Dec 03 11:58:06 crc kubenswrapper[5052]: I1203 11:58:06.919548 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/0432c361-154a-46a9-b675-04267061f13b-public-tls-certs\") pod \"keystone-bd5c68479-wvv6d\" (UID: \"0432c361-154a-46a9-b675-04267061f13b\") " pod="openstack-kuttl-tests/keystone-bd5c68479-wvv6d" Dec 03 11:58:06 crc kubenswrapper[5052]: I1203 11:58:06.921250 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/0432c361-154a-46a9-b675-04267061f13b-config-data\") pod \"keystone-bd5c68479-wvv6d\" (UID: \"0432c361-154a-46a9-b675-04267061f13b\") " pod="openstack-kuttl-tests/keystone-bd5c68479-wvv6d" Dec 03 11:58:06 crc kubenswrapper[5052]: I1203 11:58:06.923187 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0432c361-154a-46a9-b675-04267061f13b-combined-ca-bundle\") pod \"keystone-bd5c68479-wvv6d\" (UID: \"0432c361-154a-46a9-b675-04267061f13b\") " pod="openstack-kuttl-tests/keystone-bd5c68479-wvv6d" Dec 03 11:58:06 crc kubenswrapper[5052]: I1203 11:58:06.937917 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/0432c361-154a-46a9-b675-04267061f13b-scripts\") pod \"keystone-bd5c68479-wvv6d\" (UID: \"0432c361-154a-46a9-b675-04267061f13b\") " pod="openstack-kuttl-tests/keystone-bd5c68479-wvv6d" Dec 03 11:58:06 crc kubenswrapper[5052]: I1203 11:58:06.940993 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/0432c361-154a-46a9-b675-04267061f13b-fernet-keys\") pod \"keystone-bd5c68479-wvv6d\" (UID: \"0432c361-154a-46a9-b675-04267061f13b\") " pod="openstack-kuttl-tests/keystone-bd5c68479-wvv6d" Dec 03 11:58:06 crc kubenswrapper[5052]: I1203 11:58:06.944044 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/0432c361-154a-46a9-b675-04267061f13b-credential-keys\") pod \"keystone-bd5c68479-wvv6d\" (UID: \"0432c361-154a-46a9-b675-04267061f13b\") " pod="openstack-kuttl-tests/keystone-bd5c68479-wvv6d" Dec 03 11:58:06 crc kubenswrapper[5052]: I1203 11:58:06.944400 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-qfflm\" (UniqueName: \"kubernetes.io/projected/0432c361-154a-46a9-b675-04267061f13b-kube-api-access-qfflm\") pod \"keystone-bd5c68479-wvv6d\" (UID: \"0432c361-154a-46a9-b675-04267061f13b\") " pod="openstack-kuttl-tests/keystone-bd5c68479-wvv6d" Dec 03 11:58:06 crc kubenswrapper[5052]: I1203 11:58:06.949028 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/0432c361-154a-46a9-b675-04267061f13b-internal-tls-certs\") pod \"keystone-bd5c68479-wvv6d\" (UID: \"0432c361-154a-46a9-b675-04267061f13b\") " pod="openstack-kuttl-tests/keystone-bd5c68479-wvv6d" Dec 03 11:58:07 crc kubenswrapper[5052]: I1203 11:58:07.120846 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-bd5c68479-wvv6d" Dec 03 11:58:07 crc kubenswrapper[5052]: I1203 11:58:07.615914 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-bd5c68479-wvv6d"] Dec 03 11:58:07 crc kubenswrapper[5052]: W1203 11:58:07.618824 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0432c361_154a_46a9_b675_04267061f13b.slice/crio-92690517c27d1488acd9a690534bf92b583ecc54d1e454272977568c2c3b275c WatchSource:0}: Error finding container 92690517c27d1488acd9a690534bf92b583ecc54d1e454272977568c2c3b275c: Status 404 returned error can't find the container with id 92690517c27d1488acd9a690534bf92b583ecc54d1e454272977568c2c3b275c Dec 03 11:58:07 crc kubenswrapper[5052]: I1203 11:58:07.853681 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/watcher-decision-engine-0" Dec 03 11:58:07 crc kubenswrapper[5052]: I1203 11:58:07.893622 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/watcher-applier-0" Dec 03 11:58:07 crc kubenswrapper[5052]: I1203 11:58:07.894877 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/watcher-applier-0" Dec 03 11:58:07 crc kubenswrapper[5052]: I1203 11:58:07.910557 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/watcher-api-0" Dec 03 11:58:07 crc kubenswrapper[5052]: I1203 11:58:07.930984 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/watcher-api-0" Dec 03 11:58:08 crc kubenswrapper[5052]: I1203 11:58:08.001319 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/watcher-applier-0" Dec 03 11:58:08 crc kubenswrapper[5052]: I1203 11:58:08.036954 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/watcher-decision-engine-0" Dec 03 11:58:08 crc kubenswrapper[5052]: I1203 11:58:08.570684 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 11:58:08 crc kubenswrapper[5052]: I1203 11:58:08.591323 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"9bd9b8fa-d754-4e26-b66c-f06555f245ee","Type":"ContainerStarted","Data":"c1a0a57a71d63e1b1ab94dd8f9155ec9b8bad4a9698cffb1c4303ae5aebc4290"} Dec 03 11:58:08 crc kubenswrapper[5052]: I1203 11:58:08.593040 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-bd5c68479-wvv6d" event={"ID":"0432c361-154a-46a9-b675-04267061f13b","Type":"ContainerStarted","Data":"92690517c27d1488acd9a690534bf92b583ecc54d1e454272977568c2c3b275c"} Dec 03 11:58:08 crc kubenswrapper[5052]: I1203 11:58:08.595115 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"f8ecfb0e-6d4e-4380-8fd3-5f39dd783d14","Type":"ContainerStarted","Data":"d28fc0f27461b3eab6210c295d762fe75878c134e2f0a020bb1fc3bc995fe637"} Dec 03 11:58:08 crc kubenswrapper[5052]: I1203 11:58:08.595755 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/watcher-decision-engine-0" Dec 03 11:58:08 crc kubenswrapper[5052]: I1203 11:58:08.606543 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/watcher-api-0" Dec 03 11:58:08 crc kubenswrapper[5052]: I1203 11:58:08.662657 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/watcher-decision-engine-0" Dec 03 11:58:08 crc kubenswrapper[5052]: I1203 11:58:08.669888 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/watcher-applier-0" Dec 03 11:58:08 crc kubenswrapper[5052]: I1203 11:58:08.963236 5052 prober.go:107] "Probe failed" probeType="Liveness" pod="openstack-kuttl-tests/barbican-api-8784ffcfb-p694c" podUID="1f1f2013-0dae-4527-8734-ac23d8776153" containerName="barbican-api" probeResult="failure" output="HTTP probe failed with statuscode: 500" Dec 03 11:58:09 crc kubenswrapper[5052]: I1203 11:58:09.663756 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"9bd9b8fa-d754-4e26-b66c-f06555f245ee","Type":"ContainerStarted","Data":"623445068bdffcb936976141d3dc1401dcc9d9e2694b56d9d30853c1ce9c631e"} Dec 03 11:58:09 crc kubenswrapper[5052]: I1203 11:58:09.664215 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/cinder-api-0" podUID="9bd9b8fa-d754-4e26-b66c-f06555f245ee" containerName="cinder-api-log" containerID="cri-o://c1a0a57a71d63e1b1ab94dd8f9155ec9b8bad4a9698cffb1c4303ae5aebc4290" gracePeriod=30 Dec 03 11:58:09 crc kubenswrapper[5052]: I1203 11:58:09.664483 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:58:09 crc kubenswrapper[5052]: I1203 11:58:09.664750 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/cinder-api-0" podUID="9bd9b8fa-d754-4e26-b66c-f06555f245ee" containerName="cinder-api" containerID="cri-o://623445068bdffcb936976141d3dc1401dcc9d9e2694b56d9d30853c1ce9c631e" gracePeriod=30 Dec 03 11:58:09 crc kubenswrapper[5052]: I1203 11:58:09.711140 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/keystone-bd5c68479-wvv6d" Dec 03 11:58:09 crc kubenswrapper[5052]: I1203 11:58:09.711367 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-bd5c68479-wvv6d" event={"ID":"0432c361-154a-46a9-b675-04267061f13b","Type":"ContainerStarted","Data":"13a3f64e26fc4afd5ac59ca4b43b427f4f0bb964e269ef968f7456ea48b99a59"} Dec 03 11:58:09 crc kubenswrapper[5052]: I1203 11:58:09.711380 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"f8ecfb0e-6d4e-4380-8fd3-5f39dd783d14","Type":"ContainerStarted","Data":"553ad6270fd0c6958ea79310e06358db01aa11f895cfe33c1f2419ec507fcbc5"} Dec 03 11:58:09 crc kubenswrapper[5052]: I1203 11:58:09.720059 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/cinder-api-0" podStartSLOduration=4.720036283 podStartE2EDuration="4.720036283s" podCreationTimestamp="2025-12-03 11:58:05 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:58:09.707257868 +0000 UTC m=+2280.295499632" watchObservedRunningTime="2025-12-03 11:58:09.720036283 +0000 UTC m=+2280.308278047" Dec 03 11:58:09 crc kubenswrapper[5052]: I1203 11:58:09.730536 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/keystone-bd5c68479-wvv6d" podStartSLOduration=3.730502133 podStartE2EDuration="3.730502133s" podCreationTimestamp="2025-12-03 11:58:06 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:58:09.728493505 +0000 UTC m=+2280.316735269" watchObservedRunningTime="2025-12-03 11:58:09.730502133 +0000 UTC m=+2280.318743887" Dec 03 11:58:09 crc kubenswrapper[5052]: I1203 11:58:09.780476 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/cinder-scheduler-0" podStartSLOduration=4.780456793 podStartE2EDuration="4.780456793s" podCreationTimestamp="2025-12-03 11:58:05 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:58:09.77130073 +0000 UTC m=+2280.359542494" watchObservedRunningTime="2025-12-03 11:58:09.780456793 +0000 UTC m=+2280.368698557" Dec 03 11:58:10 crc kubenswrapper[5052]: I1203 11:58:10.626804 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:58:10 crc kubenswrapper[5052]: I1203 11:58:10.672777 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:58:10 crc kubenswrapper[5052]: I1203 11:58:10.758106 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/9bd9b8fa-d754-4e26-b66c-f06555f245ee-logs\") pod \"9bd9b8fa-d754-4e26-b66c-f06555f245ee\" (UID: \"9bd9b8fa-d754-4e26-b66c-f06555f245ee\") " Dec 03 11:58:10 crc kubenswrapper[5052]: I1203 11:58:10.758179 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/9bd9b8fa-d754-4e26-b66c-f06555f245ee-config-data\") pod \"9bd9b8fa-d754-4e26-b66c-f06555f245ee\" (UID: \"9bd9b8fa-d754-4e26-b66c-f06555f245ee\") " Dec 03 11:58:10 crc kubenswrapper[5052]: I1203 11:58:10.758208 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/9bd9b8fa-d754-4e26-b66c-f06555f245ee-config-data-custom\") pod \"9bd9b8fa-d754-4e26-b66c-f06555f245ee\" (UID: \"9bd9b8fa-d754-4e26-b66c-f06555f245ee\") " Dec 03 11:58:10 crc kubenswrapper[5052]: I1203 11:58:10.758300 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/9bd9b8fa-d754-4e26-b66c-f06555f245ee-scripts\") pod \"9bd9b8fa-d754-4e26-b66c-f06555f245ee\" (UID: \"9bd9b8fa-d754-4e26-b66c-f06555f245ee\") " Dec 03 11:58:10 crc kubenswrapper[5052]: I1203 11:58:10.758364 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/9bd9b8fa-d754-4e26-b66c-f06555f245ee-etc-machine-id\") pod \"9bd9b8fa-d754-4e26-b66c-f06555f245ee\" (UID: \"9bd9b8fa-d754-4e26-b66c-f06555f245ee\") " Dec 03 11:58:10 crc kubenswrapper[5052]: I1203 11:58:10.758410 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9bd9b8fa-d754-4e26-b66c-f06555f245ee-combined-ca-bundle\") pod \"9bd9b8fa-d754-4e26-b66c-f06555f245ee\" (UID: \"9bd9b8fa-d754-4e26-b66c-f06555f245ee\") " Dec 03 11:58:10 crc kubenswrapper[5052]: I1203 11:58:10.758476 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xjhf8\" (UniqueName: \"kubernetes.io/projected/9bd9b8fa-d754-4e26-b66c-f06555f245ee-kube-api-access-xjhf8\") pod \"9bd9b8fa-d754-4e26-b66c-f06555f245ee\" (UID: \"9bd9b8fa-d754-4e26-b66c-f06555f245ee\") " Dec 03 11:58:10 crc kubenswrapper[5052]: I1203 11:58:10.764165 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/9bd9b8fa-d754-4e26-b66c-f06555f245ee-etc-machine-id" (OuterVolumeSpecName: "etc-machine-id") pod "9bd9b8fa-d754-4e26-b66c-f06555f245ee" (UID: "9bd9b8fa-d754-4e26-b66c-f06555f245ee"). InnerVolumeSpecName "etc-machine-id". PluginName "kubernetes.io/host-path", VolumeGidValue "" Dec 03 11:58:10 crc kubenswrapper[5052]: I1203 11:58:10.769380 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9bd9b8fa-d754-4e26-b66c-f06555f245ee-logs" (OuterVolumeSpecName: "logs") pod "9bd9b8fa-d754-4e26-b66c-f06555f245ee" (UID: "9bd9b8fa-d754-4e26-b66c-f06555f245ee"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:58:10 crc kubenswrapper[5052]: I1203 11:58:10.786028 5052 generic.go:334] "Generic (PLEG): container finished" podID="9bd9b8fa-d754-4e26-b66c-f06555f245ee" containerID="623445068bdffcb936976141d3dc1401dcc9d9e2694b56d9d30853c1ce9c631e" exitCode=0 Dec 03 11:58:10 crc kubenswrapper[5052]: I1203 11:58:10.786521 5052 generic.go:334] "Generic (PLEG): container finished" podID="9bd9b8fa-d754-4e26-b66c-f06555f245ee" containerID="c1a0a57a71d63e1b1ab94dd8f9155ec9b8bad4a9698cffb1c4303ae5aebc4290" exitCode=143 Dec 03 11:58:10 crc kubenswrapper[5052]: I1203 11:58:10.791593 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9bd9b8fa-d754-4e26-b66c-f06555f245ee-scripts" (OuterVolumeSpecName: "scripts") pod "9bd9b8fa-d754-4e26-b66c-f06555f245ee" (UID: "9bd9b8fa-d754-4e26-b66c-f06555f245ee"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:58:10 crc kubenswrapper[5052]: I1203 11:58:10.795161 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9bd9b8fa-d754-4e26-b66c-f06555f245ee-kube-api-access-xjhf8" (OuterVolumeSpecName: "kube-api-access-xjhf8") pod "9bd9b8fa-d754-4e26-b66c-f06555f245ee" (UID: "9bd9b8fa-d754-4e26-b66c-f06555f245ee"). InnerVolumeSpecName "kube-api-access-xjhf8". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:58:10 crc kubenswrapper[5052]: I1203 11:58:10.798875 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"9bd9b8fa-d754-4e26-b66c-f06555f245ee","Type":"ContainerDied","Data":"623445068bdffcb936976141d3dc1401dcc9d9e2694b56d9d30853c1ce9c631e"} Dec 03 11:58:10 crc kubenswrapper[5052]: I1203 11:58:10.803280 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"9bd9b8fa-d754-4e26-b66c-f06555f245ee","Type":"ContainerDied","Data":"c1a0a57a71d63e1b1ab94dd8f9155ec9b8bad4a9698cffb1c4303ae5aebc4290"} Dec 03 11:58:10 crc kubenswrapper[5052]: I1203 11:58:10.803357 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"9bd9b8fa-d754-4e26-b66c-f06555f245ee","Type":"ContainerDied","Data":"3e8125429b69c318d2d290f5a4b930874ff02d827635dedcbc5482f41978954f"} Dec 03 11:58:10 crc kubenswrapper[5052]: I1203 11:58:10.803431 5052 scope.go:117] "RemoveContainer" containerID="623445068bdffcb936976141d3dc1401dcc9d9e2694b56d9d30853c1ce9c631e" Dec 03 11:58:10 crc kubenswrapper[5052]: I1203 11:58:10.798860 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:58:10 crc kubenswrapper[5052]: I1203 11:58:10.808274 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9bd9b8fa-d754-4e26-b66c-f06555f245ee-config-data-custom" (OuterVolumeSpecName: "config-data-custom") pod "9bd9b8fa-d754-4e26-b66c-f06555f245ee" (UID: "9bd9b8fa-d754-4e26-b66c-f06555f245ee"). InnerVolumeSpecName "config-data-custom". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:58:10 crc kubenswrapper[5052]: I1203 11:58:10.846942 5052 scope.go:117] "RemoveContainer" containerID="c1a0a57a71d63e1b1ab94dd8f9155ec9b8bad4a9698cffb1c4303ae5aebc4290" Dec 03 11:58:10 crc kubenswrapper[5052]: I1203 11:58:10.859148 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9bd9b8fa-d754-4e26-b66c-f06555f245ee-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "9bd9b8fa-d754-4e26-b66c-f06555f245ee" (UID: "9bd9b8fa-d754-4e26-b66c-f06555f245ee"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:58:10 crc kubenswrapper[5052]: I1203 11:58:10.871635 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/9bd9b8fa-d754-4e26-b66c-f06555f245ee-config-data-custom\") on node \"crc\" DevicePath \"\"" Dec 03 11:58:10 crc kubenswrapper[5052]: I1203 11:58:10.872458 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/9bd9b8fa-d754-4e26-b66c-f06555f245ee-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:58:10 crc kubenswrapper[5052]: I1203 11:58:10.872620 5052 reconciler_common.go:293] "Volume detached for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/9bd9b8fa-d754-4e26-b66c-f06555f245ee-etc-machine-id\") on node \"crc\" DevicePath \"\"" Dec 03 11:58:10 crc kubenswrapper[5052]: I1203 11:58:10.872730 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9bd9b8fa-d754-4e26-b66c-f06555f245ee-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:58:10 crc kubenswrapper[5052]: I1203 11:58:10.872925 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-xjhf8\" (UniqueName: \"kubernetes.io/projected/9bd9b8fa-d754-4e26-b66c-f06555f245ee-kube-api-access-xjhf8\") on node \"crc\" DevicePath \"\"" Dec 03 11:58:10 crc kubenswrapper[5052]: I1203 11:58:10.873308 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/9bd9b8fa-d754-4e26-b66c-f06555f245ee-logs\") on node \"crc\" DevicePath \"\"" Dec 03 11:58:10 crc kubenswrapper[5052]: I1203 11:58:10.888831 5052 scope.go:117] "RemoveContainer" containerID="623445068bdffcb936976141d3dc1401dcc9d9e2694b56d9d30853c1ce9c631e" Dec 03 11:58:10 crc kubenswrapper[5052]: E1203 11:58:10.889501 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"623445068bdffcb936976141d3dc1401dcc9d9e2694b56d9d30853c1ce9c631e\": container with ID starting with 623445068bdffcb936976141d3dc1401dcc9d9e2694b56d9d30853c1ce9c631e not found: ID does not exist" containerID="623445068bdffcb936976141d3dc1401dcc9d9e2694b56d9d30853c1ce9c631e" Dec 03 11:58:10 crc kubenswrapper[5052]: I1203 11:58:10.889562 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"623445068bdffcb936976141d3dc1401dcc9d9e2694b56d9d30853c1ce9c631e"} err="failed to get container status \"623445068bdffcb936976141d3dc1401dcc9d9e2694b56d9d30853c1ce9c631e\": rpc error: code = NotFound desc = could not find container \"623445068bdffcb936976141d3dc1401dcc9d9e2694b56d9d30853c1ce9c631e\": container with ID starting with 623445068bdffcb936976141d3dc1401dcc9d9e2694b56d9d30853c1ce9c631e not found: ID does not exist" Dec 03 11:58:10 crc kubenswrapper[5052]: I1203 11:58:10.889617 5052 scope.go:117] "RemoveContainer" containerID="c1a0a57a71d63e1b1ab94dd8f9155ec9b8bad4a9698cffb1c4303ae5aebc4290" Dec 03 11:58:10 crc kubenswrapper[5052]: E1203 11:58:10.890715 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c1a0a57a71d63e1b1ab94dd8f9155ec9b8bad4a9698cffb1c4303ae5aebc4290\": container with ID starting with c1a0a57a71d63e1b1ab94dd8f9155ec9b8bad4a9698cffb1c4303ae5aebc4290 not found: ID does not exist" containerID="c1a0a57a71d63e1b1ab94dd8f9155ec9b8bad4a9698cffb1c4303ae5aebc4290" Dec 03 11:58:10 crc kubenswrapper[5052]: I1203 11:58:10.890769 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c1a0a57a71d63e1b1ab94dd8f9155ec9b8bad4a9698cffb1c4303ae5aebc4290"} err="failed to get container status \"c1a0a57a71d63e1b1ab94dd8f9155ec9b8bad4a9698cffb1c4303ae5aebc4290\": rpc error: code = NotFound desc = could not find container \"c1a0a57a71d63e1b1ab94dd8f9155ec9b8bad4a9698cffb1c4303ae5aebc4290\": container with ID starting with c1a0a57a71d63e1b1ab94dd8f9155ec9b8bad4a9698cffb1c4303ae5aebc4290 not found: ID does not exist" Dec 03 11:58:10 crc kubenswrapper[5052]: I1203 11:58:10.890784 5052 scope.go:117] "RemoveContainer" containerID="623445068bdffcb936976141d3dc1401dcc9d9e2694b56d9d30853c1ce9c631e" Dec 03 11:58:10 crc kubenswrapper[5052]: I1203 11:58:10.893004 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"623445068bdffcb936976141d3dc1401dcc9d9e2694b56d9d30853c1ce9c631e"} err="failed to get container status \"623445068bdffcb936976141d3dc1401dcc9d9e2694b56d9d30853c1ce9c631e\": rpc error: code = NotFound desc = could not find container \"623445068bdffcb936976141d3dc1401dcc9d9e2694b56d9d30853c1ce9c631e\": container with ID starting with 623445068bdffcb936976141d3dc1401dcc9d9e2694b56d9d30853c1ce9c631e not found: ID does not exist" Dec 03 11:58:10 crc kubenswrapper[5052]: I1203 11:58:10.893031 5052 scope.go:117] "RemoveContainer" containerID="c1a0a57a71d63e1b1ab94dd8f9155ec9b8bad4a9698cffb1c4303ae5aebc4290" Dec 03 11:58:10 crc kubenswrapper[5052]: I1203 11:58:10.893900 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c1a0a57a71d63e1b1ab94dd8f9155ec9b8bad4a9698cffb1c4303ae5aebc4290"} err="failed to get container status \"c1a0a57a71d63e1b1ab94dd8f9155ec9b8bad4a9698cffb1c4303ae5aebc4290\": rpc error: code = NotFound desc = could not find container \"c1a0a57a71d63e1b1ab94dd8f9155ec9b8bad4a9698cffb1c4303ae5aebc4290\": container with ID starting with c1a0a57a71d63e1b1ab94dd8f9155ec9b8bad4a9698cffb1c4303ae5aebc4290 not found: ID does not exist" Dec 03 11:58:10 crc kubenswrapper[5052]: I1203 11:58:10.972989 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9bd9b8fa-d754-4e26-b66c-f06555f245ee-config-data" (OuterVolumeSpecName: "config-data") pod "9bd9b8fa-d754-4e26-b66c-f06555f245ee" (UID: "9bd9b8fa-d754-4e26-b66c-f06555f245ee"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:58:10 crc kubenswrapper[5052]: I1203 11:58:10.976216 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/9bd9b8fa-d754-4e26-b66c-f06555f245ee-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:58:11 crc kubenswrapper[5052]: I1203 11:58:11.157403 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 11:58:11 crc kubenswrapper[5052]: I1203 11:58:11.175529 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 11:58:11 crc kubenswrapper[5052]: I1203 11:58:11.186672 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 11:58:11 crc kubenswrapper[5052]: E1203 11:58:11.187090 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="9bd9b8fa-d754-4e26-b66c-f06555f245ee" containerName="cinder-api-log" Dec 03 11:58:11 crc kubenswrapper[5052]: I1203 11:58:11.187111 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="9bd9b8fa-d754-4e26-b66c-f06555f245ee" containerName="cinder-api-log" Dec 03 11:58:11 crc kubenswrapper[5052]: E1203 11:58:11.187133 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="9bd9b8fa-d754-4e26-b66c-f06555f245ee" containerName="cinder-api" Dec 03 11:58:11 crc kubenswrapper[5052]: I1203 11:58:11.187141 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="9bd9b8fa-d754-4e26-b66c-f06555f245ee" containerName="cinder-api" Dec 03 11:58:11 crc kubenswrapper[5052]: I1203 11:58:11.187406 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="9bd9b8fa-d754-4e26-b66c-f06555f245ee" containerName="cinder-api-log" Dec 03 11:58:11 crc kubenswrapper[5052]: I1203 11:58:11.187445 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="9bd9b8fa-d754-4e26-b66c-f06555f245ee" containerName="cinder-api" Dec 03 11:58:11 crc kubenswrapper[5052]: I1203 11:58:11.188743 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:58:11 crc kubenswrapper[5052]: I1203 11:58:11.191745 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-cinder-internal-svc" Dec 03 11:58:11 crc kubenswrapper[5052]: I1203 11:58:11.192443 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cinder-api-config-data" Dec 03 11:58:11 crc kubenswrapper[5052]: I1203 11:58:11.192539 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-cinder-public-svc" Dec 03 11:58:11 crc kubenswrapper[5052]: I1203 11:58:11.202944 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 11:58:11 crc kubenswrapper[5052]: I1203 11:58:11.280917 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/0291d0e7-9abf-48dd-8d5e-c329f989b700-public-tls-certs\") pod \"cinder-api-0\" (UID: \"0291d0e7-9abf-48dd-8d5e-c329f989b700\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:58:11 crc kubenswrapper[5052]: I1203 11:58:11.280970 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/0291d0e7-9abf-48dd-8d5e-c329f989b700-internal-tls-certs\") pod \"cinder-api-0\" (UID: \"0291d0e7-9abf-48dd-8d5e-c329f989b700\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:58:11 crc kubenswrapper[5052]: I1203 11:58:11.281043 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/0291d0e7-9abf-48dd-8d5e-c329f989b700-logs\") pod \"cinder-api-0\" (UID: \"0291d0e7-9abf-48dd-8d5e-c329f989b700\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:58:11 crc kubenswrapper[5052]: I1203 11:58:11.281073 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/0291d0e7-9abf-48dd-8d5e-c329f989b700-etc-machine-id\") pod \"cinder-api-0\" (UID: \"0291d0e7-9abf-48dd-8d5e-c329f989b700\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:58:11 crc kubenswrapper[5052]: I1203 11:58:11.281110 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/0291d0e7-9abf-48dd-8d5e-c329f989b700-config-data\") pod \"cinder-api-0\" (UID: \"0291d0e7-9abf-48dd-8d5e-c329f989b700\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:58:11 crc kubenswrapper[5052]: I1203 11:58:11.281184 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/0291d0e7-9abf-48dd-8d5e-c329f989b700-scripts\") pod \"cinder-api-0\" (UID: \"0291d0e7-9abf-48dd-8d5e-c329f989b700\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:58:11 crc kubenswrapper[5052]: I1203 11:58:11.281251 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0291d0e7-9abf-48dd-8d5e-c329f989b700-combined-ca-bundle\") pod \"cinder-api-0\" (UID: \"0291d0e7-9abf-48dd-8d5e-c329f989b700\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:58:11 crc kubenswrapper[5052]: I1203 11:58:11.281361 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nphb7\" (UniqueName: \"kubernetes.io/projected/0291d0e7-9abf-48dd-8d5e-c329f989b700-kube-api-access-nphb7\") pod \"cinder-api-0\" (UID: \"0291d0e7-9abf-48dd-8d5e-c329f989b700\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:58:11 crc kubenswrapper[5052]: I1203 11:58:11.281460 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/0291d0e7-9abf-48dd-8d5e-c329f989b700-config-data-custom\") pod \"cinder-api-0\" (UID: \"0291d0e7-9abf-48dd-8d5e-c329f989b700\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:58:11 crc kubenswrapper[5052]: I1203 11:58:11.383319 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-nphb7\" (UniqueName: \"kubernetes.io/projected/0291d0e7-9abf-48dd-8d5e-c329f989b700-kube-api-access-nphb7\") pod \"cinder-api-0\" (UID: \"0291d0e7-9abf-48dd-8d5e-c329f989b700\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:58:11 crc kubenswrapper[5052]: I1203 11:58:11.383413 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/0291d0e7-9abf-48dd-8d5e-c329f989b700-config-data-custom\") pod \"cinder-api-0\" (UID: \"0291d0e7-9abf-48dd-8d5e-c329f989b700\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:58:11 crc kubenswrapper[5052]: I1203 11:58:11.383478 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/0291d0e7-9abf-48dd-8d5e-c329f989b700-public-tls-certs\") pod \"cinder-api-0\" (UID: \"0291d0e7-9abf-48dd-8d5e-c329f989b700\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:58:11 crc kubenswrapper[5052]: I1203 11:58:11.383500 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/0291d0e7-9abf-48dd-8d5e-c329f989b700-internal-tls-certs\") pod \"cinder-api-0\" (UID: \"0291d0e7-9abf-48dd-8d5e-c329f989b700\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:58:11 crc kubenswrapper[5052]: I1203 11:58:11.383540 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/0291d0e7-9abf-48dd-8d5e-c329f989b700-logs\") pod \"cinder-api-0\" (UID: \"0291d0e7-9abf-48dd-8d5e-c329f989b700\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:58:11 crc kubenswrapper[5052]: I1203 11:58:11.383561 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/0291d0e7-9abf-48dd-8d5e-c329f989b700-etc-machine-id\") pod \"cinder-api-0\" (UID: \"0291d0e7-9abf-48dd-8d5e-c329f989b700\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:58:11 crc kubenswrapper[5052]: I1203 11:58:11.383585 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/0291d0e7-9abf-48dd-8d5e-c329f989b700-config-data\") pod \"cinder-api-0\" (UID: \"0291d0e7-9abf-48dd-8d5e-c329f989b700\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:58:11 crc kubenswrapper[5052]: I1203 11:58:11.383630 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/0291d0e7-9abf-48dd-8d5e-c329f989b700-scripts\") pod \"cinder-api-0\" (UID: \"0291d0e7-9abf-48dd-8d5e-c329f989b700\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:58:11 crc kubenswrapper[5052]: I1203 11:58:11.383656 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0291d0e7-9abf-48dd-8d5e-c329f989b700-combined-ca-bundle\") pod \"cinder-api-0\" (UID: \"0291d0e7-9abf-48dd-8d5e-c329f989b700\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:58:11 crc kubenswrapper[5052]: I1203 11:58:11.383955 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/0291d0e7-9abf-48dd-8d5e-c329f989b700-etc-machine-id\") pod \"cinder-api-0\" (UID: \"0291d0e7-9abf-48dd-8d5e-c329f989b700\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:58:11 crc kubenswrapper[5052]: I1203 11:58:11.384017 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/0291d0e7-9abf-48dd-8d5e-c329f989b700-logs\") pod \"cinder-api-0\" (UID: \"0291d0e7-9abf-48dd-8d5e-c329f989b700\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:58:11 crc kubenswrapper[5052]: I1203 11:58:11.387827 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0291d0e7-9abf-48dd-8d5e-c329f989b700-combined-ca-bundle\") pod \"cinder-api-0\" (UID: \"0291d0e7-9abf-48dd-8d5e-c329f989b700\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:58:11 crc kubenswrapper[5052]: I1203 11:58:11.388525 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/0291d0e7-9abf-48dd-8d5e-c329f989b700-internal-tls-certs\") pod \"cinder-api-0\" (UID: \"0291d0e7-9abf-48dd-8d5e-c329f989b700\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:58:11 crc kubenswrapper[5052]: I1203 11:58:11.392062 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/0291d0e7-9abf-48dd-8d5e-c329f989b700-config-data-custom\") pod \"cinder-api-0\" (UID: \"0291d0e7-9abf-48dd-8d5e-c329f989b700\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:58:11 crc kubenswrapper[5052]: I1203 11:58:11.392316 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/0291d0e7-9abf-48dd-8d5e-c329f989b700-public-tls-certs\") pod \"cinder-api-0\" (UID: \"0291d0e7-9abf-48dd-8d5e-c329f989b700\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:58:11 crc kubenswrapper[5052]: I1203 11:58:11.392614 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/0291d0e7-9abf-48dd-8d5e-c329f989b700-config-data\") pod \"cinder-api-0\" (UID: \"0291d0e7-9abf-48dd-8d5e-c329f989b700\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:58:11 crc kubenswrapper[5052]: I1203 11:58:11.395266 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/0291d0e7-9abf-48dd-8d5e-c329f989b700-scripts\") pod \"cinder-api-0\" (UID: \"0291d0e7-9abf-48dd-8d5e-c329f989b700\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:58:11 crc kubenswrapper[5052]: I1203 11:58:11.398912 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-nphb7\" (UniqueName: \"kubernetes.io/projected/0291d0e7-9abf-48dd-8d5e-c329f989b700-kube-api-access-nphb7\") pod \"cinder-api-0\" (UID: \"0291d0e7-9abf-48dd-8d5e-c329f989b700\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:58:11 crc kubenswrapper[5052]: I1203 11:58:11.528938 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:58:11 crc kubenswrapper[5052]: I1203 11:58:11.695982 5052 scope.go:117] "RemoveContainer" containerID="d224717c6d9ca8135fd510e03a6c40c9f2b1a56dd8b4a3b685af3e2dfa855982" Dec 03 11:58:11 crc kubenswrapper[5052]: E1203 11:58:11.696680 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 11:58:11 crc kubenswrapper[5052]: I1203 11:58:11.716585 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9bd9b8fa-d754-4e26-b66c-f06555f245ee" path="/var/lib/kubelet/pods/9bd9b8fa-d754-4e26-b66c-f06555f245ee/volumes" Dec 03 11:58:11 crc kubenswrapper[5052]: I1203 11:58:11.885088 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/barbican-api-8784ffcfb-p694c" Dec 03 11:58:12 crc kubenswrapper[5052]: I1203 11:58:12.057343 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/watcher-api-0"] Dec 03 11:58:12 crc kubenswrapper[5052]: I1203 11:58:12.057817 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/watcher-api-0" podUID="dc7d33d8-fd6f-466a-9ea2-2a643e314d53" containerName="watcher-api-log" containerID="cri-o://180e4cefdddfd8cd4d063d16f01531609d3814b7cd2413970bcfd19e05b9b2dd" gracePeriod=30 Dec 03 11:58:12 crc kubenswrapper[5052]: I1203 11:58:12.058003 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/watcher-api-0" podUID="dc7d33d8-fd6f-466a-9ea2-2a643e314d53" containerName="watcher-api" containerID="cri-o://69e769ca022be16554b1387944766e2b196a0fc86b6db305eaf0473441b4f1ff" gracePeriod=30 Dec 03 11:58:12 crc kubenswrapper[5052]: I1203 11:58:12.075712 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 11:58:12 crc kubenswrapper[5052]: I1203 11:58:12.528101 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/barbican-api-8784ffcfb-p694c" Dec 03 11:58:12 crc kubenswrapper[5052]: I1203 11:58:12.842487 5052 generic.go:334] "Generic (PLEG): container finished" podID="dc7d33d8-fd6f-466a-9ea2-2a643e314d53" containerID="180e4cefdddfd8cd4d063d16f01531609d3814b7cd2413970bcfd19e05b9b2dd" exitCode=143 Dec 03 11:58:12 crc kubenswrapper[5052]: I1203 11:58:12.843191 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/watcher-api-0" event={"ID":"dc7d33d8-fd6f-466a-9ea2-2a643e314d53","Type":"ContainerDied","Data":"180e4cefdddfd8cd4d063d16f01531609d3814b7cd2413970bcfd19e05b9b2dd"} Dec 03 11:58:12 crc kubenswrapper[5052]: I1203 11:58:12.860704 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"0291d0e7-9abf-48dd-8d5e-c329f989b700","Type":"ContainerStarted","Data":"dc0867961d380294a4a7f4bc8f4216631327967552db175aba11e79e252a54a4"} Dec 03 11:58:13 crc kubenswrapper[5052]: I1203 11:58:13.889245 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"0291d0e7-9abf-48dd-8d5e-c329f989b700","Type":"ContainerStarted","Data":"f877a20aa72fb38f5d985123667f657d88840787bd73214d01bf7253e2f1bf64"} Dec 03 11:58:14 crc kubenswrapper[5052]: I1203 11:58:14.757830 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/barbican-api-d6bc8866d-xrhs6" Dec 03 11:58:15 crc kubenswrapper[5052]: I1203 11:58:15.217071 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/watcher-api-0" podUID="dc7d33d8-fd6f-466a-9ea2-2a643e314d53" containerName="watcher-api" probeResult="failure" output="Get \"http://10.217.1.204:9322/\": read tcp 10.217.0.2:35468->10.217.1.204:9322: read: connection reset by peer" Dec 03 11:58:15 crc kubenswrapper[5052]: I1203 11:58:15.217663 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/watcher-api-0" podUID="dc7d33d8-fd6f-466a-9ea2-2a643e314d53" containerName="watcher-api-log" probeResult="failure" output="Get \"http://10.217.1.204:9322/\": read tcp 10.217.0.2:35482->10.217.1.204:9322: read: connection reset by peer" Dec 03 11:58:15 crc kubenswrapper[5052]: I1203 11:58:15.705319 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/barbican-api-d6bc8866d-xrhs6" Dec 03 11:58:15 crc kubenswrapper[5052]: I1203 11:58:15.805741 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/watcher-api-0" Dec 03 11:58:15 crc kubenswrapper[5052]: I1203 11:58:15.808990 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-api-8784ffcfb-p694c"] Dec 03 11:58:15 crc kubenswrapper[5052]: I1203 11:58:15.809822 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/barbican-api-8784ffcfb-p694c" podUID="1f1f2013-0dae-4527-8734-ac23d8776153" containerName="barbican-api-log" containerID="cri-o://b9bb37cfd7841690059bca55c66cbaa4d860650c9846d9af571aa4067c855fec" gracePeriod=30 Dec 03 11:58:15 crc kubenswrapper[5052]: I1203 11:58:15.809974 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/barbican-api-8784ffcfb-p694c" podUID="1f1f2013-0dae-4527-8734-ac23d8776153" containerName="barbican-api" containerID="cri-o://f7bb5981c82ecfa1fdd697c4703c407c5dfc77e905fb89792ccd44bfd8b55602" gracePeriod=30 Dec 03 11:58:15 crc kubenswrapper[5052]: I1203 11:58:15.875220 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qqjgg\" (UniqueName: \"kubernetes.io/projected/dc7d33d8-fd6f-466a-9ea2-2a643e314d53-kube-api-access-qqjgg\") pod \"dc7d33d8-fd6f-466a-9ea2-2a643e314d53\" (UID: \"dc7d33d8-fd6f-466a-9ea2-2a643e314d53\") " Dec 03 11:58:15 crc kubenswrapper[5052]: I1203 11:58:15.875352 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/dc7d33d8-fd6f-466a-9ea2-2a643e314d53-combined-ca-bundle\") pod \"dc7d33d8-fd6f-466a-9ea2-2a643e314d53\" (UID: \"dc7d33d8-fd6f-466a-9ea2-2a643e314d53\") " Dec 03 11:58:15 crc kubenswrapper[5052]: I1203 11:58:15.875420 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/dc7d33d8-fd6f-466a-9ea2-2a643e314d53-config-data\") pod \"dc7d33d8-fd6f-466a-9ea2-2a643e314d53\" (UID: \"dc7d33d8-fd6f-466a-9ea2-2a643e314d53\") " Dec 03 11:58:15 crc kubenswrapper[5052]: I1203 11:58:15.875496 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"custom-prometheus-ca\" (UniqueName: \"kubernetes.io/secret/dc7d33d8-fd6f-466a-9ea2-2a643e314d53-custom-prometheus-ca\") pod \"dc7d33d8-fd6f-466a-9ea2-2a643e314d53\" (UID: \"dc7d33d8-fd6f-466a-9ea2-2a643e314d53\") " Dec 03 11:58:15 crc kubenswrapper[5052]: I1203 11:58:15.875594 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/dc7d33d8-fd6f-466a-9ea2-2a643e314d53-logs\") pod \"dc7d33d8-fd6f-466a-9ea2-2a643e314d53\" (UID: \"dc7d33d8-fd6f-466a-9ea2-2a643e314d53\") " Dec 03 11:58:15 crc kubenswrapper[5052]: I1203 11:58:15.876265 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc7d33d8-fd6f-466a-9ea2-2a643e314d53-logs" (OuterVolumeSpecName: "logs") pod "dc7d33d8-fd6f-466a-9ea2-2a643e314d53" (UID: "dc7d33d8-fd6f-466a-9ea2-2a643e314d53"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:58:15 crc kubenswrapper[5052]: I1203 11:58:15.912096 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dc7d33d8-fd6f-466a-9ea2-2a643e314d53-kube-api-access-qqjgg" (OuterVolumeSpecName: "kube-api-access-qqjgg") pod "dc7d33d8-fd6f-466a-9ea2-2a643e314d53" (UID: "dc7d33d8-fd6f-466a-9ea2-2a643e314d53"). InnerVolumeSpecName "kube-api-access-qqjgg". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:58:15 crc kubenswrapper[5052]: I1203 11:58:15.929147 5052 generic.go:334] "Generic (PLEG): container finished" podID="dc7d33d8-fd6f-466a-9ea2-2a643e314d53" containerID="69e769ca022be16554b1387944766e2b196a0fc86b6db305eaf0473441b4f1ff" exitCode=0 Dec 03 11:58:15 crc kubenswrapper[5052]: I1203 11:58:15.929239 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/watcher-api-0" event={"ID":"dc7d33d8-fd6f-466a-9ea2-2a643e314d53","Type":"ContainerDied","Data":"69e769ca022be16554b1387944766e2b196a0fc86b6db305eaf0473441b4f1ff"} Dec 03 11:58:15 crc kubenswrapper[5052]: I1203 11:58:15.929268 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/watcher-api-0" event={"ID":"dc7d33d8-fd6f-466a-9ea2-2a643e314d53","Type":"ContainerDied","Data":"bd56774d357cd15024dcad0db5097905485f7ba868ebff38e623740f43909232"} Dec 03 11:58:15 crc kubenswrapper[5052]: I1203 11:58:15.929285 5052 scope.go:117] "RemoveContainer" containerID="69e769ca022be16554b1387944766e2b196a0fc86b6db305eaf0473441b4f1ff" Dec 03 11:58:15 crc kubenswrapper[5052]: I1203 11:58:15.929426 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/watcher-api-0" Dec 03 11:58:15 crc kubenswrapper[5052]: I1203 11:58:15.942925 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dc7d33d8-fd6f-466a-9ea2-2a643e314d53-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "dc7d33d8-fd6f-466a-9ea2-2a643e314d53" (UID: "dc7d33d8-fd6f-466a-9ea2-2a643e314d53"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:58:15 crc kubenswrapper[5052]: I1203 11:58:15.950107 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:58:15 crc kubenswrapper[5052]: I1203 11:58:15.960757 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"0291d0e7-9abf-48dd-8d5e-c329f989b700","Type":"ContainerStarted","Data":"8ddd0225baa5c9e8b247c70bd8287b4bcb735d619e3bc97d77620dc899e06c7e"} Dec 03 11:58:15 crc kubenswrapper[5052]: I1203 11:58:15.972630 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/cinder-api-0" podStartSLOduration=4.972610744 podStartE2EDuration="4.972610744s" podCreationTimestamp="2025-12-03 11:58:11 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:58:15.967599221 +0000 UTC m=+2286.555840985" watchObservedRunningTime="2025-12-03 11:58:15.972610744 +0000 UTC m=+2286.560852508" Dec 03 11:58:15 crc kubenswrapper[5052]: I1203 11:58:15.977403 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/dc7d33d8-fd6f-466a-9ea2-2a643e314d53-logs\") on node \"crc\" DevicePath \"\"" Dec 03 11:58:15 crc kubenswrapper[5052]: I1203 11:58:15.977440 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-qqjgg\" (UniqueName: \"kubernetes.io/projected/dc7d33d8-fd6f-466a-9ea2-2a643e314d53-kube-api-access-qqjgg\") on node \"crc\" DevicePath \"\"" Dec 03 11:58:15 crc kubenswrapper[5052]: I1203 11:58:15.977451 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/dc7d33d8-fd6f-466a-9ea2-2a643e314d53-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:58:15 crc kubenswrapper[5052]: I1203 11:58:15.981941 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dc7d33d8-fd6f-466a-9ea2-2a643e314d53-custom-prometheus-ca" (OuterVolumeSpecName: "custom-prometheus-ca") pod "dc7d33d8-fd6f-466a-9ea2-2a643e314d53" (UID: "dc7d33d8-fd6f-466a-9ea2-2a643e314d53"). InnerVolumeSpecName "custom-prometheus-ca". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:58:15 crc kubenswrapper[5052]: I1203 11:58:15.985659 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dc7d33d8-fd6f-466a-9ea2-2a643e314d53-config-data" (OuterVolumeSpecName: "config-data") pod "dc7d33d8-fd6f-466a-9ea2-2a643e314d53" (UID: "dc7d33d8-fd6f-466a-9ea2-2a643e314d53"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:58:16 crc kubenswrapper[5052]: I1203 11:58:16.049444 5052 scope.go:117] "RemoveContainer" containerID="180e4cefdddfd8cd4d063d16f01531609d3814b7cd2413970bcfd19e05b9b2dd" Dec 03 11:58:16 crc kubenswrapper[5052]: I1203 11:58:16.072764 5052 scope.go:117] "RemoveContainer" containerID="69e769ca022be16554b1387944766e2b196a0fc86b6db305eaf0473441b4f1ff" Dec 03 11:58:16 crc kubenswrapper[5052]: E1203 11:58:16.073221 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"69e769ca022be16554b1387944766e2b196a0fc86b6db305eaf0473441b4f1ff\": container with ID starting with 69e769ca022be16554b1387944766e2b196a0fc86b6db305eaf0473441b4f1ff not found: ID does not exist" containerID="69e769ca022be16554b1387944766e2b196a0fc86b6db305eaf0473441b4f1ff" Dec 03 11:58:16 crc kubenswrapper[5052]: I1203 11:58:16.073255 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"69e769ca022be16554b1387944766e2b196a0fc86b6db305eaf0473441b4f1ff"} err="failed to get container status \"69e769ca022be16554b1387944766e2b196a0fc86b6db305eaf0473441b4f1ff\": rpc error: code = NotFound desc = could not find container \"69e769ca022be16554b1387944766e2b196a0fc86b6db305eaf0473441b4f1ff\": container with ID starting with 69e769ca022be16554b1387944766e2b196a0fc86b6db305eaf0473441b4f1ff not found: ID does not exist" Dec 03 11:58:16 crc kubenswrapper[5052]: I1203 11:58:16.073276 5052 scope.go:117] "RemoveContainer" containerID="180e4cefdddfd8cd4d063d16f01531609d3814b7cd2413970bcfd19e05b9b2dd" Dec 03 11:58:16 crc kubenswrapper[5052]: E1203 11:58:16.073623 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"180e4cefdddfd8cd4d063d16f01531609d3814b7cd2413970bcfd19e05b9b2dd\": container with ID starting with 180e4cefdddfd8cd4d063d16f01531609d3814b7cd2413970bcfd19e05b9b2dd not found: ID does not exist" containerID="180e4cefdddfd8cd4d063d16f01531609d3814b7cd2413970bcfd19e05b9b2dd" Dec 03 11:58:16 crc kubenswrapper[5052]: I1203 11:58:16.073682 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"180e4cefdddfd8cd4d063d16f01531609d3814b7cd2413970bcfd19e05b9b2dd"} err="failed to get container status \"180e4cefdddfd8cd4d063d16f01531609d3814b7cd2413970bcfd19e05b9b2dd\": rpc error: code = NotFound desc = could not find container \"180e4cefdddfd8cd4d063d16f01531609d3814b7cd2413970bcfd19e05b9b2dd\": container with ID starting with 180e4cefdddfd8cd4d063d16f01531609d3814b7cd2413970bcfd19e05b9b2dd not found: ID does not exist" Dec 03 11:58:16 crc kubenswrapper[5052]: I1203 11:58:16.082246 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/dc7d33d8-fd6f-466a-9ea2-2a643e314d53-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:58:16 crc kubenswrapper[5052]: I1203 11:58:16.082289 5052 reconciler_common.go:293] "Volume detached for volume \"custom-prometheus-ca\" (UniqueName: \"kubernetes.io/secret/dc7d33d8-fd6f-466a-9ea2-2a643e314d53-custom-prometheus-ca\") on node \"crc\" DevicePath \"\"" Dec 03 11:58:16 crc kubenswrapper[5052]: I1203 11:58:16.209409 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:58:16 crc kubenswrapper[5052]: I1203 11:58:16.256312 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 11:58:16 crc kubenswrapper[5052]: I1203 11:58:16.269495 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/watcher-api-0"] Dec 03 11:58:16 crc kubenswrapper[5052]: I1203 11:58:16.278613 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/watcher-api-0"] Dec 03 11:58:16 crc kubenswrapper[5052]: I1203 11:58:16.291560 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/watcher-api-0"] Dec 03 11:58:16 crc kubenswrapper[5052]: E1203 11:58:16.292022 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="dc7d33d8-fd6f-466a-9ea2-2a643e314d53" containerName="watcher-api-log" Dec 03 11:58:16 crc kubenswrapper[5052]: I1203 11:58:16.292040 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="dc7d33d8-fd6f-466a-9ea2-2a643e314d53" containerName="watcher-api-log" Dec 03 11:58:16 crc kubenswrapper[5052]: E1203 11:58:16.292071 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="dc7d33d8-fd6f-466a-9ea2-2a643e314d53" containerName="watcher-api" Dec 03 11:58:16 crc kubenswrapper[5052]: I1203 11:58:16.292076 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="dc7d33d8-fd6f-466a-9ea2-2a643e314d53" containerName="watcher-api" Dec 03 11:58:16 crc kubenswrapper[5052]: I1203 11:58:16.292249 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="dc7d33d8-fd6f-466a-9ea2-2a643e314d53" containerName="watcher-api" Dec 03 11:58:16 crc kubenswrapper[5052]: I1203 11:58:16.292275 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="dc7d33d8-fd6f-466a-9ea2-2a643e314d53" containerName="watcher-api-log" Dec 03 11:58:16 crc kubenswrapper[5052]: I1203 11:58:16.293906 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/watcher-api-0" Dec 03 11:58:16 crc kubenswrapper[5052]: I1203 11:58:16.295730 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-watcher-public-svc" Dec 03 11:58:16 crc kubenswrapper[5052]: I1203 11:58:16.296544 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-watcher-internal-svc" Dec 03 11:58:16 crc kubenswrapper[5052]: I1203 11:58:16.297816 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"watcher-api-config-data" Dec 03 11:58:16 crc kubenswrapper[5052]: I1203 11:58:16.312772 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/watcher-api-0"] Dec 03 11:58:16 crc kubenswrapper[5052]: I1203 11:58:16.387818 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2d5669ca-8e5e-49b4-95e9-116eb1195030-combined-ca-bundle\") pod \"watcher-api-0\" (UID: \"2d5669ca-8e5e-49b4-95e9-116eb1195030\") " pod="openstack-kuttl-tests/watcher-api-0" Dec 03 11:58:16 crc kubenswrapper[5052]: I1203 11:58:16.387918 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/2d5669ca-8e5e-49b4-95e9-116eb1195030-public-tls-certs\") pod \"watcher-api-0\" (UID: \"2d5669ca-8e5e-49b4-95e9-116eb1195030\") " pod="openstack-kuttl-tests/watcher-api-0" Dec 03 11:58:16 crc kubenswrapper[5052]: I1203 11:58:16.387959 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7ndkx\" (UniqueName: \"kubernetes.io/projected/2d5669ca-8e5e-49b4-95e9-116eb1195030-kube-api-access-7ndkx\") pod \"watcher-api-0\" (UID: \"2d5669ca-8e5e-49b4-95e9-116eb1195030\") " pod="openstack-kuttl-tests/watcher-api-0" Dec 03 11:58:16 crc kubenswrapper[5052]: I1203 11:58:16.387994 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/2d5669ca-8e5e-49b4-95e9-116eb1195030-internal-tls-certs\") pod \"watcher-api-0\" (UID: \"2d5669ca-8e5e-49b4-95e9-116eb1195030\") " pod="openstack-kuttl-tests/watcher-api-0" Dec 03 11:58:16 crc kubenswrapper[5052]: I1203 11:58:16.388027 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"custom-prometheus-ca\" (UniqueName: \"kubernetes.io/secret/2d5669ca-8e5e-49b4-95e9-116eb1195030-custom-prometheus-ca\") pod \"watcher-api-0\" (UID: \"2d5669ca-8e5e-49b4-95e9-116eb1195030\") " pod="openstack-kuttl-tests/watcher-api-0" Dec 03 11:58:16 crc kubenswrapper[5052]: I1203 11:58:16.388052 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/2d5669ca-8e5e-49b4-95e9-116eb1195030-logs\") pod \"watcher-api-0\" (UID: \"2d5669ca-8e5e-49b4-95e9-116eb1195030\") " pod="openstack-kuttl-tests/watcher-api-0" Dec 03 11:58:16 crc kubenswrapper[5052]: I1203 11:58:16.388141 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/2d5669ca-8e5e-49b4-95e9-116eb1195030-config-data\") pod \"watcher-api-0\" (UID: \"2d5669ca-8e5e-49b4-95e9-116eb1195030\") " pod="openstack-kuttl-tests/watcher-api-0" Dec 03 11:58:16 crc kubenswrapper[5052]: I1203 11:58:16.490352 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/2d5669ca-8e5e-49b4-95e9-116eb1195030-public-tls-certs\") pod \"watcher-api-0\" (UID: \"2d5669ca-8e5e-49b4-95e9-116eb1195030\") " pod="openstack-kuttl-tests/watcher-api-0" Dec 03 11:58:16 crc kubenswrapper[5052]: I1203 11:58:16.490421 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-7ndkx\" (UniqueName: \"kubernetes.io/projected/2d5669ca-8e5e-49b4-95e9-116eb1195030-kube-api-access-7ndkx\") pod \"watcher-api-0\" (UID: \"2d5669ca-8e5e-49b4-95e9-116eb1195030\") " pod="openstack-kuttl-tests/watcher-api-0" Dec 03 11:58:16 crc kubenswrapper[5052]: I1203 11:58:16.490463 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/2d5669ca-8e5e-49b4-95e9-116eb1195030-internal-tls-certs\") pod \"watcher-api-0\" (UID: \"2d5669ca-8e5e-49b4-95e9-116eb1195030\") " pod="openstack-kuttl-tests/watcher-api-0" Dec 03 11:58:16 crc kubenswrapper[5052]: I1203 11:58:16.490502 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"custom-prometheus-ca\" (UniqueName: \"kubernetes.io/secret/2d5669ca-8e5e-49b4-95e9-116eb1195030-custom-prometheus-ca\") pod \"watcher-api-0\" (UID: \"2d5669ca-8e5e-49b4-95e9-116eb1195030\") " pod="openstack-kuttl-tests/watcher-api-0" Dec 03 11:58:16 crc kubenswrapper[5052]: I1203 11:58:16.490545 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/2d5669ca-8e5e-49b4-95e9-116eb1195030-logs\") pod \"watcher-api-0\" (UID: \"2d5669ca-8e5e-49b4-95e9-116eb1195030\") " pod="openstack-kuttl-tests/watcher-api-0" Dec 03 11:58:16 crc kubenswrapper[5052]: I1203 11:58:16.490643 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/2d5669ca-8e5e-49b4-95e9-116eb1195030-config-data\") pod \"watcher-api-0\" (UID: \"2d5669ca-8e5e-49b4-95e9-116eb1195030\") " pod="openstack-kuttl-tests/watcher-api-0" Dec 03 11:58:16 crc kubenswrapper[5052]: I1203 11:58:16.490687 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2d5669ca-8e5e-49b4-95e9-116eb1195030-combined-ca-bundle\") pod \"watcher-api-0\" (UID: \"2d5669ca-8e5e-49b4-95e9-116eb1195030\") " pod="openstack-kuttl-tests/watcher-api-0" Dec 03 11:58:16 crc kubenswrapper[5052]: I1203 11:58:16.495092 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/2d5669ca-8e5e-49b4-95e9-116eb1195030-logs\") pod \"watcher-api-0\" (UID: \"2d5669ca-8e5e-49b4-95e9-116eb1195030\") " pod="openstack-kuttl-tests/watcher-api-0" Dec 03 11:58:16 crc kubenswrapper[5052]: I1203 11:58:16.495311 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/2d5669ca-8e5e-49b4-95e9-116eb1195030-public-tls-certs\") pod \"watcher-api-0\" (UID: \"2d5669ca-8e5e-49b4-95e9-116eb1195030\") " pod="openstack-kuttl-tests/watcher-api-0" Dec 03 11:58:16 crc kubenswrapper[5052]: I1203 11:58:16.496078 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2d5669ca-8e5e-49b4-95e9-116eb1195030-combined-ca-bundle\") pod \"watcher-api-0\" (UID: \"2d5669ca-8e5e-49b4-95e9-116eb1195030\") " pod="openstack-kuttl-tests/watcher-api-0" Dec 03 11:58:16 crc kubenswrapper[5052]: I1203 11:58:16.500387 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"custom-prometheus-ca\" (UniqueName: \"kubernetes.io/secret/2d5669ca-8e5e-49b4-95e9-116eb1195030-custom-prometheus-ca\") pod \"watcher-api-0\" (UID: \"2d5669ca-8e5e-49b4-95e9-116eb1195030\") " pod="openstack-kuttl-tests/watcher-api-0" Dec 03 11:58:16 crc kubenswrapper[5052]: I1203 11:58:16.500393 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/2d5669ca-8e5e-49b4-95e9-116eb1195030-config-data\") pod \"watcher-api-0\" (UID: \"2d5669ca-8e5e-49b4-95e9-116eb1195030\") " pod="openstack-kuttl-tests/watcher-api-0" Dec 03 11:58:16 crc kubenswrapper[5052]: I1203 11:58:16.501286 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/2d5669ca-8e5e-49b4-95e9-116eb1195030-internal-tls-certs\") pod \"watcher-api-0\" (UID: \"2d5669ca-8e5e-49b4-95e9-116eb1195030\") " pod="openstack-kuttl-tests/watcher-api-0" Dec 03 11:58:16 crc kubenswrapper[5052]: I1203 11:58:16.515306 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-7ndkx\" (UniqueName: \"kubernetes.io/projected/2d5669ca-8e5e-49b4-95e9-116eb1195030-kube-api-access-7ndkx\") pod \"watcher-api-0\" (UID: \"2d5669ca-8e5e-49b4-95e9-116eb1195030\") " pod="openstack-kuttl-tests/watcher-api-0" Dec 03 11:58:16 crc kubenswrapper[5052]: I1203 11:58:16.615307 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/watcher-api-0" Dec 03 11:58:16 crc kubenswrapper[5052]: I1203 11:58:16.981810 5052 generic.go:334] "Generic (PLEG): container finished" podID="1f1f2013-0dae-4527-8734-ac23d8776153" containerID="b9bb37cfd7841690059bca55c66cbaa4d860650c9846d9af571aa4067c855fec" exitCode=143 Dec 03 11:58:16 crc kubenswrapper[5052]: I1203 11:58:16.981913 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-8784ffcfb-p694c" event={"ID":"1f1f2013-0dae-4527-8734-ac23d8776153","Type":"ContainerDied","Data":"b9bb37cfd7841690059bca55c66cbaa4d860650c9846d9af571aa4067c855fec"} Dec 03 11:58:16 crc kubenswrapper[5052]: I1203 11:58:16.982412 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/cinder-scheduler-0" podUID="f8ecfb0e-6d4e-4380-8fd3-5f39dd783d14" containerName="cinder-scheduler" containerID="cri-o://d28fc0f27461b3eab6210c295d762fe75878c134e2f0a020bb1fc3bc995fe637" gracePeriod=30 Dec 03 11:58:16 crc kubenswrapper[5052]: I1203 11:58:16.982942 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/cinder-scheduler-0" podUID="f8ecfb0e-6d4e-4380-8fd3-5f39dd783d14" containerName="probe" containerID="cri-o://553ad6270fd0c6958ea79310e06358db01aa11f895cfe33c1f2419ec507fcbc5" gracePeriod=30 Dec 03 11:58:17 crc kubenswrapper[5052]: W1203 11:58:17.257342 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2d5669ca_8e5e_49b4_95e9_116eb1195030.slice/crio-e99242523c7a011129425601f63e8df5ef6b39ce41e606227c6fdf6aeba438e3 WatchSource:0}: Error finding container e99242523c7a011129425601f63e8df5ef6b39ce41e606227c6fdf6aeba438e3: Status 404 returned error can't find the container with id e99242523c7a011129425601f63e8df5ef6b39ce41e606227c6fdf6aeba438e3 Dec 03 11:58:17 crc kubenswrapper[5052]: I1203 11:58:17.268636 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/watcher-api-0"] Dec 03 11:58:17 crc kubenswrapper[5052]: E1203 11:58:17.376280 5052 cadvisor_stats_provider.go:516] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/system.slice/rpm-ostreed.service\": RecentStats: unable to find data in memory cache]" Dec 03 11:58:17 crc kubenswrapper[5052]: I1203 11:58:17.682665 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dc7d33d8-fd6f-466a-9ea2-2a643e314d53" path="/var/lib/kubelet/pods/dc7d33d8-fd6f-466a-9ea2-2a643e314d53/volumes" Dec 03 11:58:17 crc kubenswrapper[5052]: I1203 11:58:17.992267 5052 generic.go:334] "Generic (PLEG): container finished" podID="f8ecfb0e-6d4e-4380-8fd3-5f39dd783d14" containerID="553ad6270fd0c6958ea79310e06358db01aa11f895cfe33c1f2419ec507fcbc5" exitCode=0 Dec 03 11:58:17 crc kubenswrapper[5052]: I1203 11:58:17.992322 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"f8ecfb0e-6d4e-4380-8fd3-5f39dd783d14","Type":"ContainerDied","Data":"553ad6270fd0c6958ea79310e06358db01aa11f895cfe33c1f2419ec507fcbc5"} Dec 03 11:58:17 crc kubenswrapper[5052]: I1203 11:58:17.994860 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/watcher-api-0" event={"ID":"2d5669ca-8e5e-49b4-95e9-116eb1195030","Type":"ContainerStarted","Data":"677fccb8442f04cfdcccbb93e98725dc995271445330b661ed4f10796a9c7c91"} Dec 03 11:58:17 crc kubenswrapper[5052]: I1203 11:58:17.994888 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/watcher-api-0" event={"ID":"2d5669ca-8e5e-49b4-95e9-116eb1195030","Type":"ContainerStarted","Data":"e99242523c7a011129425601f63e8df5ef6b39ce41e606227c6fdf6aeba438e3"} Dec 03 11:58:18 crc kubenswrapper[5052]: I1203 11:58:18.625078 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:58:18 crc kubenswrapper[5052]: I1203 11:58:18.733630 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/f8ecfb0e-6d4e-4380-8fd3-5f39dd783d14-config-data\") pod \"f8ecfb0e-6d4e-4380-8fd3-5f39dd783d14\" (UID: \"f8ecfb0e-6d4e-4380-8fd3-5f39dd783d14\") " Dec 03 11:58:18 crc kubenswrapper[5052]: I1203 11:58:18.733689 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rzv9m\" (UniqueName: \"kubernetes.io/projected/f8ecfb0e-6d4e-4380-8fd3-5f39dd783d14-kube-api-access-rzv9m\") pod \"f8ecfb0e-6d4e-4380-8fd3-5f39dd783d14\" (UID: \"f8ecfb0e-6d4e-4380-8fd3-5f39dd783d14\") " Dec 03 11:58:18 crc kubenswrapper[5052]: I1203 11:58:18.733775 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/f8ecfb0e-6d4e-4380-8fd3-5f39dd783d14-etc-machine-id\") pod \"f8ecfb0e-6d4e-4380-8fd3-5f39dd783d14\" (UID: \"f8ecfb0e-6d4e-4380-8fd3-5f39dd783d14\") " Dec 03 11:58:18 crc kubenswrapper[5052]: I1203 11:58:18.733811 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f8ecfb0e-6d4e-4380-8fd3-5f39dd783d14-combined-ca-bundle\") pod \"f8ecfb0e-6d4e-4380-8fd3-5f39dd783d14\" (UID: \"f8ecfb0e-6d4e-4380-8fd3-5f39dd783d14\") " Dec 03 11:58:18 crc kubenswrapper[5052]: I1203 11:58:18.733839 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/f8ecfb0e-6d4e-4380-8fd3-5f39dd783d14-config-data-custom\") pod \"f8ecfb0e-6d4e-4380-8fd3-5f39dd783d14\" (UID: \"f8ecfb0e-6d4e-4380-8fd3-5f39dd783d14\") " Dec 03 11:58:18 crc kubenswrapper[5052]: I1203 11:58:18.733921 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/f8ecfb0e-6d4e-4380-8fd3-5f39dd783d14-scripts\") pod \"f8ecfb0e-6d4e-4380-8fd3-5f39dd783d14\" (UID: \"f8ecfb0e-6d4e-4380-8fd3-5f39dd783d14\") " Dec 03 11:58:18 crc kubenswrapper[5052]: I1203 11:58:18.733936 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/f8ecfb0e-6d4e-4380-8fd3-5f39dd783d14-etc-machine-id" (OuterVolumeSpecName: "etc-machine-id") pod "f8ecfb0e-6d4e-4380-8fd3-5f39dd783d14" (UID: "f8ecfb0e-6d4e-4380-8fd3-5f39dd783d14"). InnerVolumeSpecName "etc-machine-id". PluginName "kubernetes.io/host-path", VolumeGidValue "" Dec 03 11:58:18 crc kubenswrapper[5052]: I1203 11:58:18.735265 5052 reconciler_common.go:293] "Volume detached for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/f8ecfb0e-6d4e-4380-8fd3-5f39dd783d14-etc-machine-id\") on node \"crc\" DevicePath \"\"" Dec 03 11:58:18 crc kubenswrapper[5052]: I1203 11:58:18.738802 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f8ecfb0e-6d4e-4380-8fd3-5f39dd783d14-config-data-custom" (OuterVolumeSpecName: "config-data-custom") pod "f8ecfb0e-6d4e-4380-8fd3-5f39dd783d14" (UID: "f8ecfb0e-6d4e-4380-8fd3-5f39dd783d14"). InnerVolumeSpecName "config-data-custom". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:58:18 crc kubenswrapper[5052]: I1203 11:58:18.739578 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f8ecfb0e-6d4e-4380-8fd3-5f39dd783d14-kube-api-access-rzv9m" (OuterVolumeSpecName: "kube-api-access-rzv9m") pod "f8ecfb0e-6d4e-4380-8fd3-5f39dd783d14" (UID: "f8ecfb0e-6d4e-4380-8fd3-5f39dd783d14"). InnerVolumeSpecName "kube-api-access-rzv9m". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:58:18 crc kubenswrapper[5052]: I1203 11:58:18.741868 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f8ecfb0e-6d4e-4380-8fd3-5f39dd783d14-scripts" (OuterVolumeSpecName: "scripts") pod "f8ecfb0e-6d4e-4380-8fd3-5f39dd783d14" (UID: "f8ecfb0e-6d4e-4380-8fd3-5f39dd783d14"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:58:18 crc kubenswrapper[5052]: I1203 11:58:18.796538 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f8ecfb0e-6d4e-4380-8fd3-5f39dd783d14-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "f8ecfb0e-6d4e-4380-8fd3-5f39dd783d14" (UID: "f8ecfb0e-6d4e-4380-8fd3-5f39dd783d14"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:58:18 crc kubenswrapper[5052]: I1203 11:58:18.837863 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-rzv9m\" (UniqueName: \"kubernetes.io/projected/f8ecfb0e-6d4e-4380-8fd3-5f39dd783d14-kube-api-access-rzv9m\") on node \"crc\" DevicePath \"\"" Dec 03 11:58:18 crc kubenswrapper[5052]: I1203 11:58:18.837916 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f8ecfb0e-6d4e-4380-8fd3-5f39dd783d14-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:58:18 crc kubenswrapper[5052]: I1203 11:58:18.837936 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/f8ecfb0e-6d4e-4380-8fd3-5f39dd783d14-config-data-custom\") on node \"crc\" DevicePath \"\"" Dec 03 11:58:18 crc kubenswrapper[5052]: I1203 11:58:18.837957 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/f8ecfb0e-6d4e-4380-8fd3-5f39dd783d14-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:58:18 crc kubenswrapper[5052]: I1203 11:58:18.861689 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f8ecfb0e-6d4e-4380-8fd3-5f39dd783d14-config-data" (OuterVolumeSpecName: "config-data") pod "f8ecfb0e-6d4e-4380-8fd3-5f39dd783d14" (UID: "f8ecfb0e-6d4e-4380-8fd3-5f39dd783d14"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:58:18 crc kubenswrapper[5052]: I1203 11:58:18.939181 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/f8ecfb0e-6d4e-4380-8fd3-5f39dd783d14-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:58:19 crc kubenswrapper[5052]: I1203 11:58:19.006684 5052 generic.go:334] "Generic (PLEG): container finished" podID="f8ecfb0e-6d4e-4380-8fd3-5f39dd783d14" containerID="d28fc0f27461b3eab6210c295d762fe75878c134e2f0a020bb1fc3bc995fe637" exitCode=0 Dec 03 11:58:19 crc kubenswrapper[5052]: I1203 11:58:19.006761 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:58:19 crc kubenswrapper[5052]: I1203 11:58:19.006782 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"f8ecfb0e-6d4e-4380-8fd3-5f39dd783d14","Type":"ContainerDied","Data":"d28fc0f27461b3eab6210c295d762fe75878c134e2f0a020bb1fc3bc995fe637"} Dec 03 11:58:19 crc kubenswrapper[5052]: I1203 11:58:19.008060 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"f8ecfb0e-6d4e-4380-8fd3-5f39dd783d14","Type":"ContainerDied","Data":"c248b5c6f06ff1b7b8ce6af52071bd8e90fbffc7f19054833fa49f6acc52c28d"} Dec 03 11:58:19 crc kubenswrapper[5052]: I1203 11:58:19.008090 5052 scope.go:117] "RemoveContainer" containerID="553ad6270fd0c6958ea79310e06358db01aa11f895cfe33c1f2419ec507fcbc5" Dec 03 11:58:19 crc kubenswrapper[5052]: I1203 11:58:19.010364 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/watcher-api-0" event={"ID":"2d5669ca-8e5e-49b4-95e9-116eb1195030","Type":"ContainerStarted","Data":"7fa07f1e6f3c12445900ff78caa499630cedc8446b5a3e6c5c977fa8f1760705"} Dec 03 11:58:19 crc kubenswrapper[5052]: I1203 11:58:19.010621 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/watcher-api-0" Dec 03 11:58:19 crc kubenswrapper[5052]: I1203 11:58:19.042476 5052 scope.go:117] "RemoveContainer" containerID="d28fc0f27461b3eab6210c295d762fe75878c134e2f0a020bb1fc3bc995fe637" Dec 03 11:58:19 crc kubenswrapper[5052]: I1203 11:58:19.058418 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/watcher-api-0" podStartSLOduration=3.058397335 podStartE2EDuration="3.058397335s" podCreationTimestamp="2025-12-03 11:58:16 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:58:19.042838029 +0000 UTC m=+2289.631079793" watchObservedRunningTime="2025-12-03 11:58:19.058397335 +0000 UTC m=+2289.646639119" Dec 03 11:58:19 crc kubenswrapper[5052]: I1203 11:58:19.078014 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 11:58:19 crc kubenswrapper[5052]: I1203 11:58:19.089485 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 11:58:19 crc kubenswrapper[5052]: I1203 11:58:19.091082 5052 scope.go:117] "RemoveContainer" containerID="553ad6270fd0c6958ea79310e06358db01aa11f895cfe33c1f2419ec507fcbc5" Dec 03 11:58:19 crc kubenswrapper[5052]: E1203 11:58:19.091816 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"553ad6270fd0c6958ea79310e06358db01aa11f895cfe33c1f2419ec507fcbc5\": container with ID starting with 553ad6270fd0c6958ea79310e06358db01aa11f895cfe33c1f2419ec507fcbc5 not found: ID does not exist" containerID="553ad6270fd0c6958ea79310e06358db01aa11f895cfe33c1f2419ec507fcbc5" Dec 03 11:58:19 crc kubenswrapper[5052]: I1203 11:58:19.091869 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"553ad6270fd0c6958ea79310e06358db01aa11f895cfe33c1f2419ec507fcbc5"} err="failed to get container status \"553ad6270fd0c6958ea79310e06358db01aa11f895cfe33c1f2419ec507fcbc5\": rpc error: code = NotFound desc = could not find container \"553ad6270fd0c6958ea79310e06358db01aa11f895cfe33c1f2419ec507fcbc5\": container with ID starting with 553ad6270fd0c6958ea79310e06358db01aa11f895cfe33c1f2419ec507fcbc5 not found: ID does not exist" Dec 03 11:58:19 crc kubenswrapper[5052]: I1203 11:58:19.091904 5052 scope.go:117] "RemoveContainer" containerID="d28fc0f27461b3eab6210c295d762fe75878c134e2f0a020bb1fc3bc995fe637" Dec 03 11:58:19 crc kubenswrapper[5052]: E1203 11:58:19.092548 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d28fc0f27461b3eab6210c295d762fe75878c134e2f0a020bb1fc3bc995fe637\": container with ID starting with d28fc0f27461b3eab6210c295d762fe75878c134e2f0a020bb1fc3bc995fe637 not found: ID does not exist" containerID="d28fc0f27461b3eab6210c295d762fe75878c134e2f0a020bb1fc3bc995fe637" Dec 03 11:58:19 crc kubenswrapper[5052]: I1203 11:58:19.092577 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d28fc0f27461b3eab6210c295d762fe75878c134e2f0a020bb1fc3bc995fe637"} err="failed to get container status \"d28fc0f27461b3eab6210c295d762fe75878c134e2f0a020bb1fc3bc995fe637\": rpc error: code = NotFound desc = could not find container \"d28fc0f27461b3eab6210c295d762fe75878c134e2f0a020bb1fc3bc995fe637\": container with ID starting with d28fc0f27461b3eab6210c295d762fe75878c134e2f0a020bb1fc3bc995fe637 not found: ID does not exist" Dec 03 11:58:19 crc kubenswrapper[5052]: I1203 11:58:19.098621 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 11:58:19 crc kubenswrapper[5052]: E1203 11:58:19.099099 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="f8ecfb0e-6d4e-4380-8fd3-5f39dd783d14" containerName="cinder-scheduler" Dec 03 11:58:19 crc kubenswrapper[5052]: I1203 11:58:19.099130 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="f8ecfb0e-6d4e-4380-8fd3-5f39dd783d14" containerName="cinder-scheduler" Dec 03 11:58:19 crc kubenswrapper[5052]: E1203 11:58:19.099176 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="f8ecfb0e-6d4e-4380-8fd3-5f39dd783d14" containerName="probe" Dec 03 11:58:19 crc kubenswrapper[5052]: I1203 11:58:19.099185 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="f8ecfb0e-6d4e-4380-8fd3-5f39dd783d14" containerName="probe" Dec 03 11:58:19 crc kubenswrapper[5052]: I1203 11:58:19.099404 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="f8ecfb0e-6d4e-4380-8fd3-5f39dd783d14" containerName="probe" Dec 03 11:58:19 crc kubenswrapper[5052]: I1203 11:58:19.099465 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="f8ecfb0e-6d4e-4380-8fd3-5f39dd783d14" containerName="cinder-scheduler" Dec 03 11:58:19 crc kubenswrapper[5052]: I1203 11:58:19.100739 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:58:19 crc kubenswrapper[5052]: I1203 11:58:19.103762 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cinder-scheduler-config-data" Dec 03 11:58:19 crc kubenswrapper[5052]: I1203 11:58:19.111128 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 11:58:19 crc kubenswrapper[5052]: I1203 11:58:19.138163 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/barbican-api-8784ffcfb-p694c" podUID="1f1f2013-0dae-4527-8734-ac23d8776153" containerName="barbican-api" probeResult="failure" output="Get \"http://10.217.1.207:9311/healthcheck\": dial tcp 10.217.1.207:9311: connect: connection refused" Dec 03 11:58:19 crc kubenswrapper[5052]: I1203 11:58:19.138197 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/barbican-api-8784ffcfb-p694c" podUID="1f1f2013-0dae-4527-8734-ac23d8776153" containerName="barbican-api-log" probeResult="failure" output="Get \"http://10.217.1.207:9311/healthcheck\": dial tcp 10.217.1.207:9311: connect: connection refused" Dec 03 11:58:19 crc kubenswrapper[5052]: I1203 11:58:19.145329 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/2d2fda50-7118-43f2-8290-36d97b66de86-config-data-custom\") pod \"cinder-scheduler-0\" (UID: \"2d2fda50-7118-43f2-8290-36d97b66de86\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:58:19 crc kubenswrapper[5052]: I1203 11:58:19.145388 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2d2fda50-7118-43f2-8290-36d97b66de86-combined-ca-bundle\") pod \"cinder-scheduler-0\" (UID: \"2d2fda50-7118-43f2-8290-36d97b66de86\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:58:19 crc kubenswrapper[5052]: I1203 11:58:19.145449 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/2d2fda50-7118-43f2-8290-36d97b66de86-etc-machine-id\") pod \"cinder-scheduler-0\" (UID: \"2d2fda50-7118-43f2-8290-36d97b66de86\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:58:19 crc kubenswrapper[5052]: I1203 11:58:19.145853 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/2d2fda50-7118-43f2-8290-36d97b66de86-scripts\") pod \"cinder-scheduler-0\" (UID: \"2d2fda50-7118-43f2-8290-36d97b66de86\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:58:19 crc kubenswrapper[5052]: I1203 11:58:19.146016 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fqxvv\" (UniqueName: \"kubernetes.io/projected/2d2fda50-7118-43f2-8290-36d97b66de86-kube-api-access-fqxvv\") pod \"cinder-scheduler-0\" (UID: \"2d2fda50-7118-43f2-8290-36d97b66de86\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:58:19 crc kubenswrapper[5052]: I1203 11:58:19.146074 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/2d2fda50-7118-43f2-8290-36d97b66de86-config-data\") pod \"cinder-scheduler-0\" (UID: \"2d2fda50-7118-43f2-8290-36d97b66de86\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:58:19 crc kubenswrapper[5052]: I1203 11:58:19.247087 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-fqxvv\" (UniqueName: \"kubernetes.io/projected/2d2fda50-7118-43f2-8290-36d97b66de86-kube-api-access-fqxvv\") pod \"cinder-scheduler-0\" (UID: \"2d2fda50-7118-43f2-8290-36d97b66de86\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:58:19 crc kubenswrapper[5052]: I1203 11:58:19.247134 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/2d2fda50-7118-43f2-8290-36d97b66de86-config-data\") pod \"cinder-scheduler-0\" (UID: \"2d2fda50-7118-43f2-8290-36d97b66de86\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:58:19 crc kubenswrapper[5052]: I1203 11:58:19.247197 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/2d2fda50-7118-43f2-8290-36d97b66de86-config-data-custom\") pod \"cinder-scheduler-0\" (UID: \"2d2fda50-7118-43f2-8290-36d97b66de86\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:58:19 crc kubenswrapper[5052]: I1203 11:58:19.247225 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2d2fda50-7118-43f2-8290-36d97b66de86-combined-ca-bundle\") pod \"cinder-scheduler-0\" (UID: \"2d2fda50-7118-43f2-8290-36d97b66de86\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:58:19 crc kubenswrapper[5052]: I1203 11:58:19.247265 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/2d2fda50-7118-43f2-8290-36d97b66de86-etc-machine-id\") pod \"cinder-scheduler-0\" (UID: \"2d2fda50-7118-43f2-8290-36d97b66de86\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:58:19 crc kubenswrapper[5052]: I1203 11:58:19.247307 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/2d2fda50-7118-43f2-8290-36d97b66de86-scripts\") pod \"cinder-scheduler-0\" (UID: \"2d2fda50-7118-43f2-8290-36d97b66de86\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:58:19 crc kubenswrapper[5052]: I1203 11:58:19.249483 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/2d2fda50-7118-43f2-8290-36d97b66de86-etc-machine-id\") pod \"cinder-scheduler-0\" (UID: \"2d2fda50-7118-43f2-8290-36d97b66de86\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:58:19 crc kubenswrapper[5052]: I1203 11:58:19.250426 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2d2fda50-7118-43f2-8290-36d97b66de86-combined-ca-bundle\") pod \"cinder-scheduler-0\" (UID: \"2d2fda50-7118-43f2-8290-36d97b66de86\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:58:19 crc kubenswrapper[5052]: I1203 11:58:19.251178 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/2d2fda50-7118-43f2-8290-36d97b66de86-config-data\") pod \"cinder-scheduler-0\" (UID: \"2d2fda50-7118-43f2-8290-36d97b66de86\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:58:19 crc kubenswrapper[5052]: I1203 11:58:19.251849 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/2d2fda50-7118-43f2-8290-36d97b66de86-scripts\") pod \"cinder-scheduler-0\" (UID: \"2d2fda50-7118-43f2-8290-36d97b66de86\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:58:19 crc kubenswrapper[5052]: I1203 11:58:19.254971 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/2d2fda50-7118-43f2-8290-36d97b66de86-config-data-custom\") pod \"cinder-scheduler-0\" (UID: \"2d2fda50-7118-43f2-8290-36d97b66de86\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:58:19 crc kubenswrapper[5052]: I1203 11:58:19.264211 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-fqxvv\" (UniqueName: \"kubernetes.io/projected/2d2fda50-7118-43f2-8290-36d97b66de86-kube-api-access-fqxvv\") pod \"cinder-scheduler-0\" (UID: \"2d2fda50-7118-43f2-8290-36d97b66de86\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:58:19 crc kubenswrapper[5052]: I1203 11:58:19.442456 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:58:19 crc kubenswrapper[5052]: I1203 11:58:19.699568 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f8ecfb0e-6d4e-4380-8fd3-5f39dd783d14" path="/var/lib/kubelet/pods/f8ecfb0e-6d4e-4380-8fd3-5f39dd783d14/volumes" Dec 03 11:58:19 crc kubenswrapper[5052]: I1203 11:58:19.773808 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-api-8784ffcfb-p694c" Dec 03 11:58:19 crc kubenswrapper[5052]: I1203 11:58:19.869001 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1f1f2013-0dae-4527-8734-ac23d8776153-combined-ca-bundle\") pod \"1f1f2013-0dae-4527-8734-ac23d8776153\" (UID: \"1f1f2013-0dae-4527-8734-ac23d8776153\") " Dec 03 11:58:19 crc kubenswrapper[5052]: I1203 11:58:19.869165 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/1f1f2013-0dae-4527-8734-ac23d8776153-logs\") pod \"1f1f2013-0dae-4527-8734-ac23d8776153\" (UID: \"1f1f2013-0dae-4527-8734-ac23d8776153\") " Dec 03 11:58:19 crc kubenswrapper[5052]: I1203 11:58:19.869225 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/1f1f2013-0dae-4527-8734-ac23d8776153-config-data\") pod \"1f1f2013-0dae-4527-8734-ac23d8776153\" (UID: \"1f1f2013-0dae-4527-8734-ac23d8776153\") " Dec 03 11:58:19 crc kubenswrapper[5052]: I1203 11:58:19.869284 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6t85d\" (UniqueName: \"kubernetes.io/projected/1f1f2013-0dae-4527-8734-ac23d8776153-kube-api-access-6t85d\") pod \"1f1f2013-0dae-4527-8734-ac23d8776153\" (UID: \"1f1f2013-0dae-4527-8734-ac23d8776153\") " Dec 03 11:58:19 crc kubenswrapper[5052]: I1203 11:58:19.869312 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/1f1f2013-0dae-4527-8734-ac23d8776153-config-data-custom\") pod \"1f1f2013-0dae-4527-8734-ac23d8776153\" (UID: \"1f1f2013-0dae-4527-8734-ac23d8776153\") " Dec 03 11:58:19 crc kubenswrapper[5052]: I1203 11:58:19.870301 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f1f2013-0dae-4527-8734-ac23d8776153-logs" (OuterVolumeSpecName: "logs") pod "1f1f2013-0dae-4527-8734-ac23d8776153" (UID: "1f1f2013-0dae-4527-8734-ac23d8776153"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:58:19 crc kubenswrapper[5052]: I1203 11:58:19.874800 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1f1f2013-0dae-4527-8734-ac23d8776153-kube-api-access-6t85d" (OuterVolumeSpecName: "kube-api-access-6t85d") pod "1f1f2013-0dae-4527-8734-ac23d8776153" (UID: "1f1f2013-0dae-4527-8734-ac23d8776153"). InnerVolumeSpecName "kube-api-access-6t85d". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:58:19 crc kubenswrapper[5052]: I1203 11:58:19.874788 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1f1f2013-0dae-4527-8734-ac23d8776153-config-data-custom" (OuterVolumeSpecName: "config-data-custom") pod "1f1f2013-0dae-4527-8734-ac23d8776153" (UID: "1f1f2013-0dae-4527-8734-ac23d8776153"). InnerVolumeSpecName "config-data-custom". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:58:19 crc kubenswrapper[5052]: I1203 11:58:19.894992 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1f1f2013-0dae-4527-8734-ac23d8776153-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "1f1f2013-0dae-4527-8734-ac23d8776153" (UID: "1f1f2013-0dae-4527-8734-ac23d8776153"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:58:19 crc kubenswrapper[5052]: I1203 11:58:19.924288 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1f1f2013-0dae-4527-8734-ac23d8776153-config-data" (OuterVolumeSpecName: "config-data") pod "1f1f2013-0dae-4527-8734-ac23d8776153" (UID: "1f1f2013-0dae-4527-8734-ac23d8776153"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:58:19 crc kubenswrapper[5052]: I1203 11:58:19.972093 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-6t85d\" (UniqueName: \"kubernetes.io/projected/1f1f2013-0dae-4527-8734-ac23d8776153-kube-api-access-6t85d\") on node \"crc\" DevicePath \"\"" Dec 03 11:58:19 crc kubenswrapper[5052]: I1203 11:58:19.972138 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/1f1f2013-0dae-4527-8734-ac23d8776153-config-data-custom\") on node \"crc\" DevicePath \"\"" Dec 03 11:58:19 crc kubenswrapper[5052]: I1203 11:58:19.972151 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1f1f2013-0dae-4527-8734-ac23d8776153-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:58:19 crc kubenswrapper[5052]: I1203 11:58:19.972164 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/1f1f2013-0dae-4527-8734-ac23d8776153-logs\") on node \"crc\" DevicePath \"\"" Dec 03 11:58:19 crc kubenswrapper[5052]: I1203 11:58:19.972177 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/1f1f2013-0dae-4527-8734-ac23d8776153-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:58:20 crc kubenswrapper[5052]: I1203 11:58:19.999992 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 11:58:20 crc kubenswrapper[5052]: I1203 11:58:20.023501 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"2d2fda50-7118-43f2-8290-36d97b66de86","Type":"ContainerStarted","Data":"9442fbe11e649cdb482a44ed1cc5ef0f6c2f30c2feeae4bab8dc6c03ec92704c"} Dec 03 11:58:20 crc kubenswrapper[5052]: I1203 11:58:20.028252 5052 generic.go:334] "Generic (PLEG): container finished" podID="1f1f2013-0dae-4527-8734-ac23d8776153" containerID="f7bb5981c82ecfa1fdd697c4703c407c5dfc77e905fb89792ccd44bfd8b55602" exitCode=0 Dec 03 11:58:20 crc kubenswrapper[5052]: I1203 11:58:20.029159 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-api-8784ffcfb-p694c" Dec 03 11:58:20 crc kubenswrapper[5052]: I1203 11:58:20.030618 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-8784ffcfb-p694c" event={"ID":"1f1f2013-0dae-4527-8734-ac23d8776153","Type":"ContainerDied","Data":"f7bb5981c82ecfa1fdd697c4703c407c5dfc77e905fb89792ccd44bfd8b55602"} Dec 03 11:58:20 crc kubenswrapper[5052]: I1203 11:58:20.031140 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-8784ffcfb-p694c" event={"ID":"1f1f2013-0dae-4527-8734-ac23d8776153","Type":"ContainerDied","Data":"308d3e381c0f505066fb57e757f4b0584c5d0cc0fb3865f3f6b45a7cb1c64d74"} Dec 03 11:58:20 crc kubenswrapper[5052]: I1203 11:58:20.031218 5052 scope.go:117] "RemoveContainer" containerID="f7bb5981c82ecfa1fdd697c4703c407c5dfc77e905fb89792ccd44bfd8b55602" Dec 03 11:58:20 crc kubenswrapper[5052]: I1203 11:58:20.068042 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-api-8784ffcfb-p694c"] Dec 03 11:58:20 crc kubenswrapper[5052]: I1203 11:58:20.072644 5052 scope.go:117] "RemoveContainer" containerID="b9bb37cfd7841690059bca55c66cbaa4d860650c9846d9af571aa4067c855fec" Dec 03 11:58:20 crc kubenswrapper[5052]: I1203 11:58:20.081615 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/barbican-api-8784ffcfb-p694c"] Dec 03 11:58:20 crc kubenswrapper[5052]: I1203 11:58:20.092952 5052 scope.go:117] "RemoveContainer" containerID="f7bb5981c82ecfa1fdd697c4703c407c5dfc77e905fb89792ccd44bfd8b55602" Dec 03 11:58:20 crc kubenswrapper[5052]: E1203 11:58:20.094015 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f7bb5981c82ecfa1fdd697c4703c407c5dfc77e905fb89792ccd44bfd8b55602\": container with ID starting with f7bb5981c82ecfa1fdd697c4703c407c5dfc77e905fb89792ccd44bfd8b55602 not found: ID does not exist" containerID="f7bb5981c82ecfa1fdd697c4703c407c5dfc77e905fb89792ccd44bfd8b55602" Dec 03 11:58:20 crc kubenswrapper[5052]: I1203 11:58:20.094059 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f7bb5981c82ecfa1fdd697c4703c407c5dfc77e905fb89792ccd44bfd8b55602"} err="failed to get container status \"f7bb5981c82ecfa1fdd697c4703c407c5dfc77e905fb89792ccd44bfd8b55602\": rpc error: code = NotFound desc = could not find container \"f7bb5981c82ecfa1fdd697c4703c407c5dfc77e905fb89792ccd44bfd8b55602\": container with ID starting with f7bb5981c82ecfa1fdd697c4703c407c5dfc77e905fb89792ccd44bfd8b55602 not found: ID does not exist" Dec 03 11:58:20 crc kubenswrapper[5052]: I1203 11:58:20.094086 5052 scope.go:117] "RemoveContainer" containerID="b9bb37cfd7841690059bca55c66cbaa4d860650c9846d9af571aa4067c855fec" Dec 03 11:58:20 crc kubenswrapper[5052]: E1203 11:58:20.094643 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b9bb37cfd7841690059bca55c66cbaa4d860650c9846d9af571aa4067c855fec\": container with ID starting with b9bb37cfd7841690059bca55c66cbaa4d860650c9846d9af571aa4067c855fec not found: ID does not exist" containerID="b9bb37cfd7841690059bca55c66cbaa4d860650c9846d9af571aa4067c855fec" Dec 03 11:58:20 crc kubenswrapper[5052]: I1203 11:58:20.094688 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b9bb37cfd7841690059bca55c66cbaa4d860650c9846d9af571aa4067c855fec"} err="failed to get container status \"b9bb37cfd7841690059bca55c66cbaa4d860650c9846d9af571aa4067c855fec\": rpc error: code = NotFound desc = could not find container \"b9bb37cfd7841690059bca55c66cbaa4d860650c9846d9af571aa4067c855fec\": container with ID starting with b9bb37cfd7841690059bca55c66cbaa4d860650c9846d9af571aa4067c855fec not found: ID does not exist" Dec 03 11:58:21 crc kubenswrapper[5052]: I1203 11:58:21.043361 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"2d2fda50-7118-43f2-8290-36d97b66de86","Type":"ContainerStarted","Data":"541d5fc878d9121d9e872cb4361abda24d819f62b553c1995d69dbf6c3ab83a1"} Dec 03 11:58:21 crc kubenswrapper[5052]: I1203 11:58:21.509733 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:58:21 crc kubenswrapper[5052]: I1203 11:58:21.616890 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/watcher-api-0" Dec 03 11:58:21 crc kubenswrapper[5052]: I1203 11:58:21.616996 5052 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Dec 03 11:58:21 crc kubenswrapper[5052]: I1203 11:58:21.694274 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1f1f2013-0dae-4527-8734-ac23d8776153" path="/var/lib/kubelet/pods/1f1f2013-0dae-4527-8734-ac23d8776153/volumes" Dec 03 11:58:21 crc kubenswrapper[5052]: I1203 11:58:21.859038 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/watcher-api-0" Dec 03 11:58:23 crc kubenswrapper[5052]: I1203 11:58:23.066428 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"2d2fda50-7118-43f2-8290-36d97b66de86","Type":"ContainerStarted","Data":"365e155e111e9968f8e4c88e8465be33bb14e93a08d5db3a41bced9b4391e4fd"} Dec 03 11:58:23 crc kubenswrapper[5052]: I1203 11:58:23.089934 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/cinder-scheduler-0" podStartSLOduration=4.089915171 podStartE2EDuration="4.089915171s" podCreationTimestamp="2025-12-03 11:58:19 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:58:23.085894706 +0000 UTC m=+2293.674136470" watchObservedRunningTime="2025-12-03 11:58:23.089915171 +0000 UTC m=+2293.678156935" Dec 03 11:58:23 crc kubenswrapper[5052]: I1203 11:58:23.816110 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 11:58:24 crc kubenswrapper[5052]: I1203 11:58:24.442790 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:58:26 crc kubenswrapper[5052]: I1203 11:58:26.616064 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/watcher-api-0" Dec 03 11:58:26 crc kubenswrapper[5052]: I1203 11:58:26.626450 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/watcher-api-0" Dec 03 11:58:26 crc kubenswrapper[5052]: I1203 11:58:26.673144 5052 scope.go:117] "RemoveContainer" containerID="d224717c6d9ca8135fd510e03a6c40c9f2b1a56dd8b4a3b685af3e2dfa855982" Dec 03 11:58:26 crc kubenswrapper[5052]: E1203 11:58:26.673353 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 11:58:27 crc kubenswrapper[5052]: I1203 11:58:27.114881 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/watcher-api-0" Dec 03 11:58:27 crc kubenswrapper[5052]: E1203 11:58:27.629279 5052 cadvisor_stats_provider.go:516] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/system.slice/rpm-ostreed.service\": RecentStats: unable to find data in memory cache]" Dec 03 11:58:29 crc kubenswrapper[5052]: I1203 11:58:29.668314 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 11:58:29 crc kubenswrapper[5052]: I1203 11:58:29.734192 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/placement-7d9769b856-tzkn2" Dec 03 11:58:29 crc kubenswrapper[5052]: I1203 11:58:29.742987 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/placement-7d9769b856-tzkn2" Dec 03 11:58:35 crc kubenswrapper[5052]: I1203 11:58:35.181840 5052 generic.go:334] "Generic (PLEG): container finished" podID="28c32800-797f-4560-ac08-55c9ca254a79" containerID="efba1971cde8e2d60c21027b0213c476cc57031f7a94af481671b4c6bb25e4fe" exitCode=0 Dec 03 11:58:35 crc kubenswrapper[5052]: I1203 11:58:35.181929 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-db-sync-xvd7n" event={"ID":"28c32800-797f-4560-ac08-55c9ca254a79","Type":"ContainerDied","Data":"efba1971cde8e2d60c21027b0213c476cc57031f7a94af481671b4c6bb25e4fe"} Dec 03 11:58:36 crc kubenswrapper[5052]: I1203 11:58:36.541578 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-db-sync-xvd7n" Dec 03 11:58:36 crc kubenswrapper[5052]: I1203 11:58:36.625780 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/28c32800-797f-4560-ac08-55c9ca254a79-config\") pod \"28c32800-797f-4560-ac08-55c9ca254a79\" (UID: \"28c32800-797f-4560-ac08-55c9ca254a79\") " Dec 03 11:58:36 crc kubenswrapper[5052]: I1203 11:58:36.625958 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c7d7v\" (UniqueName: \"kubernetes.io/projected/28c32800-797f-4560-ac08-55c9ca254a79-kube-api-access-c7d7v\") pod \"28c32800-797f-4560-ac08-55c9ca254a79\" (UID: \"28c32800-797f-4560-ac08-55c9ca254a79\") " Dec 03 11:58:36 crc kubenswrapper[5052]: I1203 11:58:36.626026 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/28c32800-797f-4560-ac08-55c9ca254a79-combined-ca-bundle\") pod \"28c32800-797f-4560-ac08-55c9ca254a79\" (UID: \"28c32800-797f-4560-ac08-55c9ca254a79\") " Dec 03 11:58:36 crc kubenswrapper[5052]: I1203 11:58:36.631388 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/28c32800-797f-4560-ac08-55c9ca254a79-kube-api-access-c7d7v" (OuterVolumeSpecName: "kube-api-access-c7d7v") pod "28c32800-797f-4560-ac08-55c9ca254a79" (UID: "28c32800-797f-4560-ac08-55c9ca254a79"). InnerVolumeSpecName "kube-api-access-c7d7v". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:58:36 crc kubenswrapper[5052]: I1203 11:58:36.653494 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/28c32800-797f-4560-ac08-55c9ca254a79-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "28c32800-797f-4560-ac08-55c9ca254a79" (UID: "28c32800-797f-4560-ac08-55c9ca254a79"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:58:36 crc kubenswrapper[5052]: I1203 11:58:36.654878 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/28c32800-797f-4560-ac08-55c9ca254a79-config" (OuterVolumeSpecName: "config") pod "28c32800-797f-4560-ac08-55c9ca254a79" (UID: "28c32800-797f-4560-ac08-55c9ca254a79"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:58:36 crc kubenswrapper[5052]: I1203 11:58:36.728924 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-c7d7v\" (UniqueName: \"kubernetes.io/projected/28c32800-797f-4560-ac08-55c9ca254a79-kube-api-access-c7d7v\") on node \"crc\" DevicePath \"\"" Dec 03 11:58:36 crc kubenswrapper[5052]: I1203 11:58:36.728970 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/28c32800-797f-4560-ac08-55c9ca254a79-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:58:36 crc kubenswrapper[5052]: I1203 11:58:36.728987 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/28c32800-797f-4560-ac08-55c9ca254a79-config\") on node \"crc\" DevicePath \"\"" Dec 03 11:58:37 crc kubenswrapper[5052]: I1203 11:58:37.200952 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-db-sync-xvd7n" event={"ID":"28c32800-797f-4560-ac08-55c9ca254a79","Type":"ContainerDied","Data":"6a9c59a33f09852cb3d612699609c4e996fb88af55a03d65e3743c6765b8cd7b"} Dec 03 11:58:37 crc kubenswrapper[5052]: I1203 11:58:37.200991 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6a9c59a33f09852cb3d612699609c4e996fb88af55a03d65e3743c6765b8cd7b" Dec 03 11:58:37 crc kubenswrapper[5052]: I1203 11:58:37.201044 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-db-sync-xvd7n" Dec 03 11:58:37 crc kubenswrapper[5052]: I1203 11:58:37.397468 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/neutron-8966cf85c-7cmmp"] Dec 03 11:58:37 crc kubenswrapper[5052]: E1203 11:58:37.397860 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="1f1f2013-0dae-4527-8734-ac23d8776153" containerName="barbican-api-log" Dec 03 11:58:37 crc kubenswrapper[5052]: I1203 11:58:37.397876 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f1f2013-0dae-4527-8734-ac23d8776153" containerName="barbican-api-log" Dec 03 11:58:37 crc kubenswrapper[5052]: E1203 11:58:37.397889 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="1f1f2013-0dae-4527-8734-ac23d8776153" containerName="barbican-api" Dec 03 11:58:37 crc kubenswrapper[5052]: I1203 11:58:37.397897 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f1f2013-0dae-4527-8734-ac23d8776153" containerName="barbican-api" Dec 03 11:58:37 crc kubenswrapper[5052]: E1203 11:58:37.397915 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="28c32800-797f-4560-ac08-55c9ca254a79" containerName="neutron-db-sync" Dec 03 11:58:37 crc kubenswrapper[5052]: I1203 11:58:37.397922 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="28c32800-797f-4560-ac08-55c9ca254a79" containerName="neutron-db-sync" Dec 03 11:58:37 crc kubenswrapper[5052]: I1203 11:58:37.398093 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="28c32800-797f-4560-ac08-55c9ca254a79" containerName="neutron-db-sync" Dec 03 11:58:37 crc kubenswrapper[5052]: I1203 11:58:37.398103 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="1f1f2013-0dae-4527-8734-ac23d8776153" containerName="barbican-api" Dec 03 11:58:37 crc kubenswrapper[5052]: I1203 11:58:37.398110 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="1f1f2013-0dae-4527-8734-ac23d8776153" containerName="barbican-api-log" Dec 03 11:58:37 crc kubenswrapper[5052]: I1203 11:58:37.399061 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-8966cf85c-7cmmp" Dec 03 11:58:37 crc kubenswrapper[5052]: I1203 11:58:37.401371 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"neutron-httpd-config" Dec 03 11:58:37 crc kubenswrapper[5052]: I1203 11:58:37.401718 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"neutron-neutron-dockercfg-qpfvs" Dec 03 11:58:37 crc kubenswrapper[5052]: I1203 11:58:37.401727 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"neutron-config" Dec 03 11:58:37 crc kubenswrapper[5052]: I1203 11:58:37.405487 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-neutron-ovndbs" Dec 03 11:58:37 crc kubenswrapper[5052]: I1203 11:58:37.413188 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-8966cf85c-7cmmp"] Dec 03 11:58:37 crc kubenswrapper[5052]: I1203 11:58:37.547669 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/6bb40332-be9f-4633-8cab-72971768cbba-config\") pod \"neutron-8966cf85c-7cmmp\" (UID: \"6bb40332-be9f-4633-8cab-72971768cbba\") " pod="openstack-kuttl-tests/neutron-8966cf85c-7cmmp" Dec 03 11:58:37 crc kubenswrapper[5052]: I1203 11:58:37.548433 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tcvfl\" (UniqueName: \"kubernetes.io/projected/6bb40332-be9f-4633-8cab-72971768cbba-kube-api-access-tcvfl\") pod \"neutron-8966cf85c-7cmmp\" (UID: \"6bb40332-be9f-4633-8cab-72971768cbba\") " pod="openstack-kuttl-tests/neutron-8966cf85c-7cmmp" Dec 03 11:58:37 crc kubenswrapper[5052]: I1203 11:58:37.548589 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6bb40332-be9f-4633-8cab-72971768cbba-combined-ca-bundle\") pod \"neutron-8966cf85c-7cmmp\" (UID: \"6bb40332-be9f-4633-8cab-72971768cbba\") " pod="openstack-kuttl-tests/neutron-8966cf85c-7cmmp" Dec 03 11:58:37 crc kubenswrapper[5052]: I1203 11:58:37.548730 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/6bb40332-be9f-4633-8cab-72971768cbba-ovndb-tls-certs\") pod \"neutron-8966cf85c-7cmmp\" (UID: \"6bb40332-be9f-4633-8cab-72971768cbba\") " pod="openstack-kuttl-tests/neutron-8966cf85c-7cmmp" Dec 03 11:58:37 crc kubenswrapper[5052]: I1203 11:58:37.548834 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/6bb40332-be9f-4633-8cab-72971768cbba-httpd-config\") pod \"neutron-8966cf85c-7cmmp\" (UID: \"6bb40332-be9f-4633-8cab-72971768cbba\") " pod="openstack-kuttl-tests/neutron-8966cf85c-7cmmp" Dec 03 11:58:37 crc kubenswrapper[5052]: I1203 11:58:37.650528 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/6bb40332-be9f-4633-8cab-72971768cbba-config\") pod \"neutron-8966cf85c-7cmmp\" (UID: \"6bb40332-be9f-4633-8cab-72971768cbba\") " pod="openstack-kuttl-tests/neutron-8966cf85c-7cmmp" Dec 03 11:58:37 crc kubenswrapper[5052]: I1203 11:58:37.650589 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-tcvfl\" (UniqueName: \"kubernetes.io/projected/6bb40332-be9f-4633-8cab-72971768cbba-kube-api-access-tcvfl\") pod \"neutron-8966cf85c-7cmmp\" (UID: \"6bb40332-be9f-4633-8cab-72971768cbba\") " pod="openstack-kuttl-tests/neutron-8966cf85c-7cmmp" Dec 03 11:58:37 crc kubenswrapper[5052]: I1203 11:58:37.650628 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6bb40332-be9f-4633-8cab-72971768cbba-combined-ca-bundle\") pod \"neutron-8966cf85c-7cmmp\" (UID: \"6bb40332-be9f-4633-8cab-72971768cbba\") " pod="openstack-kuttl-tests/neutron-8966cf85c-7cmmp" Dec 03 11:58:37 crc kubenswrapper[5052]: I1203 11:58:37.650655 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/6bb40332-be9f-4633-8cab-72971768cbba-ovndb-tls-certs\") pod \"neutron-8966cf85c-7cmmp\" (UID: \"6bb40332-be9f-4633-8cab-72971768cbba\") " pod="openstack-kuttl-tests/neutron-8966cf85c-7cmmp" Dec 03 11:58:37 crc kubenswrapper[5052]: I1203 11:58:37.650670 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/6bb40332-be9f-4633-8cab-72971768cbba-httpd-config\") pod \"neutron-8966cf85c-7cmmp\" (UID: \"6bb40332-be9f-4633-8cab-72971768cbba\") " pod="openstack-kuttl-tests/neutron-8966cf85c-7cmmp" Dec 03 11:58:37 crc kubenswrapper[5052]: I1203 11:58:37.655032 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/6bb40332-be9f-4633-8cab-72971768cbba-httpd-config\") pod \"neutron-8966cf85c-7cmmp\" (UID: \"6bb40332-be9f-4633-8cab-72971768cbba\") " pod="openstack-kuttl-tests/neutron-8966cf85c-7cmmp" Dec 03 11:58:37 crc kubenswrapper[5052]: I1203 11:58:37.655592 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/6bb40332-be9f-4633-8cab-72971768cbba-config\") pod \"neutron-8966cf85c-7cmmp\" (UID: \"6bb40332-be9f-4633-8cab-72971768cbba\") " pod="openstack-kuttl-tests/neutron-8966cf85c-7cmmp" Dec 03 11:58:37 crc kubenswrapper[5052]: I1203 11:58:37.657352 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6bb40332-be9f-4633-8cab-72971768cbba-combined-ca-bundle\") pod \"neutron-8966cf85c-7cmmp\" (UID: \"6bb40332-be9f-4633-8cab-72971768cbba\") " pod="openstack-kuttl-tests/neutron-8966cf85c-7cmmp" Dec 03 11:58:37 crc kubenswrapper[5052]: I1203 11:58:37.657487 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/6bb40332-be9f-4633-8cab-72971768cbba-ovndb-tls-certs\") pod \"neutron-8966cf85c-7cmmp\" (UID: \"6bb40332-be9f-4633-8cab-72971768cbba\") " pod="openstack-kuttl-tests/neutron-8966cf85c-7cmmp" Dec 03 11:58:37 crc kubenswrapper[5052]: I1203 11:58:37.671406 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-tcvfl\" (UniqueName: \"kubernetes.io/projected/6bb40332-be9f-4633-8cab-72971768cbba-kube-api-access-tcvfl\") pod \"neutron-8966cf85c-7cmmp\" (UID: \"6bb40332-be9f-4633-8cab-72971768cbba\") " pod="openstack-kuttl-tests/neutron-8966cf85c-7cmmp" Dec 03 11:58:37 crc kubenswrapper[5052]: I1203 11:58:37.675447 5052 scope.go:117] "RemoveContainer" containerID="d224717c6d9ca8135fd510e03a6c40c9f2b1a56dd8b4a3b685af3e2dfa855982" Dec 03 11:58:37 crc kubenswrapper[5052]: E1203 11:58:37.676132 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 11:58:37 crc kubenswrapper[5052]: I1203 11:58:37.723988 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-8966cf85c-7cmmp" Dec 03 11:58:37 crc kubenswrapper[5052]: E1203 11:58:37.907711 5052 cadvisor_stats_provider.go:516] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/system.slice/rpm-ostreed.service\": RecentStats: unable to find data in memory cache]" Dec 03 11:58:38 crc kubenswrapper[5052]: W1203 11:58:38.212177 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6bb40332_be9f_4633_8cab_72971768cbba.slice/crio-33f5bc6ff8e55b7e1eda38ad9f955f861f54fdf563eb86d08bc4e61568bd4b98 WatchSource:0}: Error finding container 33f5bc6ff8e55b7e1eda38ad9f955f861f54fdf563eb86d08bc4e61568bd4b98: Status 404 returned error can't find the container with id 33f5bc6ff8e55b7e1eda38ad9f955f861f54fdf563eb86d08bc4e61568bd4b98 Dec 03 11:58:38 crc kubenswrapper[5052]: I1203 11:58:38.212450 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-8966cf85c-7cmmp"] Dec 03 11:58:39 crc kubenswrapper[5052]: I1203 11:58:39.226833 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-8966cf85c-7cmmp" event={"ID":"6bb40332-be9f-4633-8cab-72971768cbba","Type":"ContainerStarted","Data":"33f5bc6ff8e55b7e1eda38ad9f955f861f54fdf563eb86d08bc4e61568bd4b98"} Dec 03 11:58:39 crc kubenswrapper[5052]: I1203 11:58:39.321772 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/keystone-bd5c68479-wvv6d" Dec 03 11:58:40 crc kubenswrapper[5052]: I1203 11:58:40.238876 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-8966cf85c-7cmmp" event={"ID":"6bb40332-be9f-4633-8cab-72971768cbba","Type":"ContainerStarted","Data":"072116c6057db13f831617dbb142496bb56bb8d9006e2da94d6128552d93ae62"} Dec 03 11:58:40 crc kubenswrapper[5052]: I1203 11:58:40.239238 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-8966cf85c-7cmmp" event={"ID":"6bb40332-be9f-4633-8cab-72971768cbba","Type":"ContainerStarted","Data":"ec5e6c8abdbedd4339a7b82a497d3d7b854810af9a4c0415133053de14bb98fb"} Dec 03 11:58:40 crc kubenswrapper[5052]: I1203 11:58:40.239647 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/neutron-8966cf85c-7cmmp" Dec 03 11:58:40 crc kubenswrapper[5052]: I1203 11:58:40.266064 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/neutron-8966cf85c-7cmmp" podStartSLOduration=3.266044888 podStartE2EDuration="3.266044888s" podCreationTimestamp="2025-12-03 11:58:37 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:58:40.255490786 +0000 UTC m=+2310.843732550" watchObservedRunningTime="2025-12-03 11:58:40.266044888 +0000 UTC m=+2310.854286662" Dec 03 11:58:40 crc kubenswrapper[5052]: I1203 11:58:40.310448 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/neutron-7f59bd7c8d-4jwhw"] Dec 03 11:58:40 crc kubenswrapper[5052]: I1203 11:58:40.311990 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-7f59bd7c8d-4jwhw" Dec 03 11:58:40 crc kubenswrapper[5052]: I1203 11:58:40.317342 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-neutron-public-svc" Dec 03 11:58:40 crc kubenswrapper[5052]: I1203 11:58:40.317626 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-neutron-internal-svc" Dec 03 11:58:40 crc kubenswrapper[5052]: I1203 11:58:40.325584 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-7f59bd7c8d-4jwhw"] Dec 03 11:58:40 crc kubenswrapper[5052]: I1203 11:58:40.400064 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/73f90998-d47e-407c-b964-12065411b4e2-config\") pod \"neutron-7f59bd7c8d-4jwhw\" (UID: \"73f90998-d47e-407c-b964-12065411b4e2\") " pod="openstack-kuttl-tests/neutron-7f59bd7c8d-4jwhw" Dec 03 11:58:40 crc kubenswrapper[5052]: I1203 11:58:40.400121 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/73f90998-d47e-407c-b964-12065411b4e2-public-tls-certs\") pod \"neutron-7f59bd7c8d-4jwhw\" (UID: \"73f90998-d47e-407c-b964-12065411b4e2\") " pod="openstack-kuttl-tests/neutron-7f59bd7c8d-4jwhw" Dec 03 11:58:40 crc kubenswrapper[5052]: I1203 11:58:40.400156 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/73f90998-d47e-407c-b964-12065411b4e2-combined-ca-bundle\") pod \"neutron-7f59bd7c8d-4jwhw\" (UID: \"73f90998-d47e-407c-b964-12065411b4e2\") " pod="openstack-kuttl-tests/neutron-7f59bd7c8d-4jwhw" Dec 03 11:58:40 crc kubenswrapper[5052]: I1203 11:58:40.400256 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/73f90998-d47e-407c-b964-12065411b4e2-httpd-config\") pod \"neutron-7f59bd7c8d-4jwhw\" (UID: \"73f90998-d47e-407c-b964-12065411b4e2\") " pod="openstack-kuttl-tests/neutron-7f59bd7c8d-4jwhw" Dec 03 11:58:40 crc kubenswrapper[5052]: I1203 11:58:40.400352 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/73f90998-d47e-407c-b964-12065411b4e2-ovndb-tls-certs\") pod \"neutron-7f59bd7c8d-4jwhw\" (UID: \"73f90998-d47e-407c-b964-12065411b4e2\") " pod="openstack-kuttl-tests/neutron-7f59bd7c8d-4jwhw" Dec 03 11:58:40 crc kubenswrapper[5052]: I1203 11:58:40.400378 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k29nl\" (UniqueName: \"kubernetes.io/projected/73f90998-d47e-407c-b964-12065411b4e2-kube-api-access-k29nl\") pod \"neutron-7f59bd7c8d-4jwhw\" (UID: \"73f90998-d47e-407c-b964-12065411b4e2\") " pod="openstack-kuttl-tests/neutron-7f59bd7c8d-4jwhw" Dec 03 11:58:40 crc kubenswrapper[5052]: I1203 11:58:40.400536 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/73f90998-d47e-407c-b964-12065411b4e2-internal-tls-certs\") pod \"neutron-7f59bd7c8d-4jwhw\" (UID: \"73f90998-d47e-407c-b964-12065411b4e2\") " pod="openstack-kuttl-tests/neutron-7f59bd7c8d-4jwhw" Dec 03 11:58:40 crc kubenswrapper[5052]: I1203 11:58:40.502220 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/73f90998-d47e-407c-b964-12065411b4e2-config\") pod \"neutron-7f59bd7c8d-4jwhw\" (UID: \"73f90998-d47e-407c-b964-12065411b4e2\") " pod="openstack-kuttl-tests/neutron-7f59bd7c8d-4jwhw" Dec 03 11:58:40 crc kubenswrapper[5052]: I1203 11:58:40.502272 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/73f90998-d47e-407c-b964-12065411b4e2-public-tls-certs\") pod \"neutron-7f59bd7c8d-4jwhw\" (UID: \"73f90998-d47e-407c-b964-12065411b4e2\") " pod="openstack-kuttl-tests/neutron-7f59bd7c8d-4jwhw" Dec 03 11:58:40 crc kubenswrapper[5052]: I1203 11:58:40.502303 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/73f90998-d47e-407c-b964-12065411b4e2-combined-ca-bundle\") pod \"neutron-7f59bd7c8d-4jwhw\" (UID: \"73f90998-d47e-407c-b964-12065411b4e2\") " pod="openstack-kuttl-tests/neutron-7f59bd7c8d-4jwhw" Dec 03 11:58:40 crc kubenswrapper[5052]: I1203 11:58:40.502320 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/73f90998-d47e-407c-b964-12065411b4e2-httpd-config\") pod \"neutron-7f59bd7c8d-4jwhw\" (UID: \"73f90998-d47e-407c-b964-12065411b4e2\") " pod="openstack-kuttl-tests/neutron-7f59bd7c8d-4jwhw" Dec 03 11:58:40 crc kubenswrapper[5052]: I1203 11:58:40.502354 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/73f90998-d47e-407c-b964-12065411b4e2-ovndb-tls-certs\") pod \"neutron-7f59bd7c8d-4jwhw\" (UID: \"73f90998-d47e-407c-b964-12065411b4e2\") " pod="openstack-kuttl-tests/neutron-7f59bd7c8d-4jwhw" Dec 03 11:58:40 crc kubenswrapper[5052]: I1203 11:58:40.502370 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-k29nl\" (UniqueName: \"kubernetes.io/projected/73f90998-d47e-407c-b964-12065411b4e2-kube-api-access-k29nl\") pod \"neutron-7f59bd7c8d-4jwhw\" (UID: \"73f90998-d47e-407c-b964-12065411b4e2\") " pod="openstack-kuttl-tests/neutron-7f59bd7c8d-4jwhw" Dec 03 11:58:40 crc kubenswrapper[5052]: I1203 11:58:40.502429 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/73f90998-d47e-407c-b964-12065411b4e2-internal-tls-certs\") pod \"neutron-7f59bd7c8d-4jwhw\" (UID: \"73f90998-d47e-407c-b964-12065411b4e2\") " pod="openstack-kuttl-tests/neutron-7f59bd7c8d-4jwhw" Dec 03 11:58:40 crc kubenswrapper[5052]: I1203 11:58:40.508222 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/73f90998-d47e-407c-b964-12065411b4e2-internal-tls-certs\") pod \"neutron-7f59bd7c8d-4jwhw\" (UID: \"73f90998-d47e-407c-b964-12065411b4e2\") " pod="openstack-kuttl-tests/neutron-7f59bd7c8d-4jwhw" Dec 03 11:58:40 crc kubenswrapper[5052]: I1203 11:58:40.509387 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/73f90998-d47e-407c-b964-12065411b4e2-ovndb-tls-certs\") pod \"neutron-7f59bd7c8d-4jwhw\" (UID: \"73f90998-d47e-407c-b964-12065411b4e2\") " pod="openstack-kuttl-tests/neutron-7f59bd7c8d-4jwhw" Dec 03 11:58:40 crc kubenswrapper[5052]: I1203 11:58:40.512539 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/73f90998-d47e-407c-b964-12065411b4e2-combined-ca-bundle\") pod \"neutron-7f59bd7c8d-4jwhw\" (UID: \"73f90998-d47e-407c-b964-12065411b4e2\") " pod="openstack-kuttl-tests/neutron-7f59bd7c8d-4jwhw" Dec 03 11:58:40 crc kubenswrapper[5052]: I1203 11:58:40.513180 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/73f90998-d47e-407c-b964-12065411b4e2-httpd-config\") pod \"neutron-7f59bd7c8d-4jwhw\" (UID: \"73f90998-d47e-407c-b964-12065411b4e2\") " pod="openstack-kuttl-tests/neutron-7f59bd7c8d-4jwhw" Dec 03 11:58:40 crc kubenswrapper[5052]: I1203 11:58:40.518260 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/73f90998-d47e-407c-b964-12065411b4e2-public-tls-certs\") pod \"neutron-7f59bd7c8d-4jwhw\" (UID: \"73f90998-d47e-407c-b964-12065411b4e2\") " pod="openstack-kuttl-tests/neutron-7f59bd7c8d-4jwhw" Dec 03 11:58:40 crc kubenswrapper[5052]: I1203 11:58:40.529704 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/73f90998-d47e-407c-b964-12065411b4e2-config\") pod \"neutron-7f59bd7c8d-4jwhw\" (UID: \"73f90998-d47e-407c-b964-12065411b4e2\") " pod="openstack-kuttl-tests/neutron-7f59bd7c8d-4jwhw" Dec 03 11:58:40 crc kubenswrapper[5052]: I1203 11:58:40.538354 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-k29nl\" (UniqueName: \"kubernetes.io/projected/73f90998-d47e-407c-b964-12065411b4e2-kube-api-access-k29nl\") pod \"neutron-7f59bd7c8d-4jwhw\" (UID: \"73f90998-d47e-407c-b964-12065411b4e2\") " pod="openstack-kuttl-tests/neutron-7f59bd7c8d-4jwhw" Dec 03 11:58:40 crc kubenswrapper[5052]: I1203 11:58:40.628498 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-7f59bd7c8d-4jwhw" Dec 03 11:58:40 crc kubenswrapper[5052]: I1203 11:58:40.750088 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/openstackclient"] Dec 03 11:58:40 crc kubenswrapper[5052]: I1203 11:58:40.751284 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstackclient" Dec 03 11:58:40 crc kubenswrapper[5052]: I1203 11:58:40.754283 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"openstack-config-secret" Dec 03 11:58:40 crc kubenswrapper[5052]: I1203 11:58:40.754310 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"openstack-config" Dec 03 11:58:40 crc kubenswrapper[5052]: I1203 11:58:40.754361 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"openstackclient-openstackclient-dockercfg-qzk77" Dec 03 11:58:40 crc kubenswrapper[5052]: I1203 11:58:40.785721 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/openstackclient"] Dec 03 11:58:40 crc kubenswrapper[5052]: I1203 11:58:40.808427 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hz667\" (UniqueName: \"kubernetes.io/projected/05236591-6a13-4ff8-bc2f-2016649fa3fc-kube-api-access-hz667\") pod \"openstackclient\" (UID: \"05236591-6a13-4ff8-bc2f-2016649fa3fc\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 11:58:40 crc kubenswrapper[5052]: I1203 11:58:40.808964 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openstack-config-secret\" (UniqueName: \"kubernetes.io/secret/05236591-6a13-4ff8-bc2f-2016649fa3fc-openstack-config-secret\") pod \"openstackclient\" (UID: \"05236591-6a13-4ff8-bc2f-2016649fa3fc\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 11:58:40 crc kubenswrapper[5052]: I1203 11:58:40.809012 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openstack-config\" (UniqueName: \"kubernetes.io/configmap/05236591-6a13-4ff8-bc2f-2016649fa3fc-openstack-config\") pod \"openstackclient\" (UID: \"05236591-6a13-4ff8-bc2f-2016649fa3fc\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 11:58:40 crc kubenswrapper[5052]: I1203 11:58:40.809042 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/05236591-6a13-4ff8-bc2f-2016649fa3fc-combined-ca-bundle\") pod \"openstackclient\" (UID: \"05236591-6a13-4ff8-bc2f-2016649fa3fc\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 11:58:40 crc kubenswrapper[5052]: I1203 11:58:40.911062 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"openstack-config-secret\" (UniqueName: \"kubernetes.io/secret/05236591-6a13-4ff8-bc2f-2016649fa3fc-openstack-config-secret\") pod \"openstackclient\" (UID: \"05236591-6a13-4ff8-bc2f-2016649fa3fc\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 11:58:40 crc kubenswrapper[5052]: I1203 11:58:40.911131 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"openstack-config\" (UniqueName: \"kubernetes.io/configmap/05236591-6a13-4ff8-bc2f-2016649fa3fc-openstack-config\") pod \"openstackclient\" (UID: \"05236591-6a13-4ff8-bc2f-2016649fa3fc\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 11:58:40 crc kubenswrapper[5052]: I1203 11:58:40.911164 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/05236591-6a13-4ff8-bc2f-2016649fa3fc-combined-ca-bundle\") pod \"openstackclient\" (UID: \"05236591-6a13-4ff8-bc2f-2016649fa3fc\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 11:58:40 crc kubenswrapper[5052]: I1203 11:58:40.911229 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-hz667\" (UniqueName: \"kubernetes.io/projected/05236591-6a13-4ff8-bc2f-2016649fa3fc-kube-api-access-hz667\") pod \"openstackclient\" (UID: \"05236591-6a13-4ff8-bc2f-2016649fa3fc\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 11:58:40 crc kubenswrapper[5052]: I1203 11:58:40.912363 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"openstack-config\" (UniqueName: \"kubernetes.io/configmap/05236591-6a13-4ff8-bc2f-2016649fa3fc-openstack-config\") pod \"openstackclient\" (UID: \"05236591-6a13-4ff8-bc2f-2016649fa3fc\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 11:58:40 crc kubenswrapper[5052]: I1203 11:58:40.921022 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"openstack-config-secret\" (UniqueName: \"kubernetes.io/secret/05236591-6a13-4ff8-bc2f-2016649fa3fc-openstack-config-secret\") pod \"openstackclient\" (UID: \"05236591-6a13-4ff8-bc2f-2016649fa3fc\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 11:58:40 crc kubenswrapper[5052]: I1203 11:58:40.931496 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/05236591-6a13-4ff8-bc2f-2016649fa3fc-combined-ca-bundle\") pod \"openstackclient\" (UID: \"05236591-6a13-4ff8-bc2f-2016649fa3fc\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 11:58:40 crc kubenswrapper[5052]: I1203 11:58:40.946805 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-hz667\" (UniqueName: \"kubernetes.io/projected/05236591-6a13-4ff8-bc2f-2016649fa3fc-kube-api-access-hz667\") pod \"openstackclient\" (UID: \"05236591-6a13-4ff8-bc2f-2016649fa3fc\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 11:58:41 crc kubenswrapper[5052]: I1203 11:58:41.096191 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstackclient" Dec 03 11:58:41 crc kubenswrapper[5052]: I1203 11:58:41.183316 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-7f59bd7c8d-4jwhw"] Dec 03 11:58:41 crc kubenswrapper[5052]: I1203 11:58:41.259413 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-7f59bd7c8d-4jwhw" event={"ID":"73f90998-d47e-407c-b964-12065411b4e2","Type":"ContainerStarted","Data":"f08a3072fc2ec221cf35e3fa74281c89a30ce3c6cebef66afd6b1a373a9bfbbc"} Dec 03 11:58:41 crc kubenswrapper[5052]: I1203 11:58:41.592782 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/openstackclient"] Dec 03 11:58:42 crc kubenswrapper[5052]: I1203 11:58:42.271128 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstackclient" event={"ID":"05236591-6a13-4ff8-bc2f-2016649fa3fc","Type":"ContainerStarted","Data":"f9982d5e0a6d516106ed549ea5250927607d0748ff6fe35b4feae30164c85aa3"} Dec 03 11:58:42 crc kubenswrapper[5052]: I1203 11:58:42.271429 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstackclient" event={"ID":"05236591-6a13-4ff8-bc2f-2016649fa3fc","Type":"ContainerStarted","Data":"0bd547f7d1add0ad740d3fbbaadbb8e4bdc5956b2340bb46b827236dfdb14353"} Dec 03 11:58:42 crc kubenswrapper[5052]: I1203 11:58:42.273276 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-7f59bd7c8d-4jwhw" event={"ID":"73f90998-d47e-407c-b964-12065411b4e2","Type":"ContainerStarted","Data":"841d973c00ea29cca586a4e7b4174c1d2c994db38756378837d3f5c9d0346486"} Dec 03 11:58:42 crc kubenswrapper[5052]: I1203 11:58:42.273299 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-7f59bd7c8d-4jwhw" event={"ID":"73f90998-d47e-407c-b964-12065411b4e2","Type":"ContainerStarted","Data":"be99515097973c2c0916717764695e4147e9b00df227c2974f21e0db3fe72a0f"} Dec 03 11:58:43 crc kubenswrapper[5052]: I1203 11:58:43.302663 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/neutron-7f59bd7c8d-4jwhw" Dec 03 11:58:43 crc kubenswrapper[5052]: I1203 11:58:43.345619 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/openstackclient" podStartSLOduration=3.34559932 podStartE2EDuration="3.34559932s" podCreationTimestamp="2025-12-03 11:58:40 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:58:43.322483069 +0000 UTC m=+2313.910724833" watchObservedRunningTime="2025-12-03 11:58:43.34559932 +0000 UTC m=+2313.933841084" Dec 03 11:58:43 crc kubenswrapper[5052]: I1203 11:58:43.365765 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/neutron-7f59bd7c8d-4jwhw" podStartSLOduration=3.3657472569999998 podStartE2EDuration="3.365747257s" podCreationTimestamp="2025-12-03 11:58:40 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:58:43.351264863 +0000 UTC m=+2313.939506627" watchObservedRunningTime="2025-12-03 11:58:43.365747257 +0000 UTC m=+2313.953989021" Dec 03 11:58:44 crc kubenswrapper[5052]: I1203 11:58:44.966667 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openshift-marketplace/redhat-marketplace-nzgnj"] Dec 03 11:58:44 crc kubenswrapper[5052]: I1203 11:58:44.968875 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/redhat-marketplace-nzgnj" Dec 03 11:58:44 crc kubenswrapper[5052]: I1203 11:58:44.978659 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/redhat-marketplace-nzgnj"] Dec 03 11:58:45 crc kubenswrapper[5052]: I1203 11:58:45.094363 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jh2nt\" (UniqueName: \"kubernetes.io/projected/8746e6ab-2863-45c2-b516-3b2bdfed2fb9-kube-api-access-jh2nt\") pod \"redhat-marketplace-nzgnj\" (UID: \"8746e6ab-2863-45c2-b516-3b2bdfed2fb9\") " pod="openshift-marketplace/redhat-marketplace-nzgnj" Dec 03 11:58:45 crc kubenswrapper[5052]: I1203 11:58:45.094443 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/8746e6ab-2863-45c2-b516-3b2bdfed2fb9-utilities\") pod \"redhat-marketplace-nzgnj\" (UID: \"8746e6ab-2863-45c2-b516-3b2bdfed2fb9\") " pod="openshift-marketplace/redhat-marketplace-nzgnj" Dec 03 11:58:45 crc kubenswrapper[5052]: I1203 11:58:45.094576 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/8746e6ab-2863-45c2-b516-3b2bdfed2fb9-catalog-content\") pod \"redhat-marketplace-nzgnj\" (UID: \"8746e6ab-2863-45c2-b516-3b2bdfed2fb9\") " pod="openshift-marketplace/redhat-marketplace-nzgnj" Dec 03 11:58:45 crc kubenswrapper[5052]: I1203 11:58:45.196527 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/8746e6ab-2863-45c2-b516-3b2bdfed2fb9-catalog-content\") pod \"redhat-marketplace-nzgnj\" (UID: \"8746e6ab-2863-45c2-b516-3b2bdfed2fb9\") " pod="openshift-marketplace/redhat-marketplace-nzgnj" Dec 03 11:58:45 crc kubenswrapper[5052]: I1203 11:58:45.196679 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-jh2nt\" (UniqueName: \"kubernetes.io/projected/8746e6ab-2863-45c2-b516-3b2bdfed2fb9-kube-api-access-jh2nt\") pod \"redhat-marketplace-nzgnj\" (UID: \"8746e6ab-2863-45c2-b516-3b2bdfed2fb9\") " pod="openshift-marketplace/redhat-marketplace-nzgnj" Dec 03 11:58:45 crc kubenswrapper[5052]: I1203 11:58:45.196747 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/8746e6ab-2863-45c2-b516-3b2bdfed2fb9-utilities\") pod \"redhat-marketplace-nzgnj\" (UID: \"8746e6ab-2863-45c2-b516-3b2bdfed2fb9\") " pod="openshift-marketplace/redhat-marketplace-nzgnj" Dec 03 11:58:45 crc kubenswrapper[5052]: I1203 11:58:45.197143 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/8746e6ab-2863-45c2-b516-3b2bdfed2fb9-catalog-content\") pod \"redhat-marketplace-nzgnj\" (UID: \"8746e6ab-2863-45c2-b516-3b2bdfed2fb9\") " pod="openshift-marketplace/redhat-marketplace-nzgnj" Dec 03 11:58:45 crc kubenswrapper[5052]: I1203 11:58:45.197251 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/8746e6ab-2863-45c2-b516-3b2bdfed2fb9-utilities\") pod \"redhat-marketplace-nzgnj\" (UID: \"8746e6ab-2863-45c2-b516-3b2bdfed2fb9\") " pod="openshift-marketplace/redhat-marketplace-nzgnj" Dec 03 11:58:45 crc kubenswrapper[5052]: I1203 11:58:45.237583 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-jh2nt\" (UniqueName: \"kubernetes.io/projected/8746e6ab-2863-45c2-b516-3b2bdfed2fb9-kube-api-access-jh2nt\") pod \"redhat-marketplace-nzgnj\" (UID: \"8746e6ab-2863-45c2-b516-3b2bdfed2fb9\") " pod="openshift-marketplace/redhat-marketplace-nzgnj" Dec 03 11:58:45 crc kubenswrapper[5052]: I1203 11:58:45.299459 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/redhat-marketplace-nzgnj" Dec 03 11:58:45 crc kubenswrapper[5052]: I1203 11:58:45.958876 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/redhat-marketplace-nzgnj"] Dec 03 11:58:46 crc kubenswrapper[5052]: I1203 11:58:46.341892 5052 generic.go:334] "Generic (PLEG): container finished" podID="8746e6ab-2863-45c2-b516-3b2bdfed2fb9" containerID="46ea7b758ebfe4d32a01510cd603559d92b9393d6d064ce81e1362b1d7108e83" exitCode=0 Dec 03 11:58:46 crc kubenswrapper[5052]: I1203 11:58:46.341968 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-marketplace-nzgnj" event={"ID":"8746e6ab-2863-45c2-b516-3b2bdfed2fb9","Type":"ContainerDied","Data":"46ea7b758ebfe4d32a01510cd603559d92b9393d6d064ce81e1362b1d7108e83"} Dec 03 11:58:46 crc kubenswrapper[5052]: I1203 11:58:46.342214 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-marketplace-nzgnj" event={"ID":"8746e6ab-2863-45c2-b516-3b2bdfed2fb9","Type":"ContainerStarted","Data":"d4d256451268e7b6c5904e85ccdc211c5cdbf1217c77e5742ab007a1e5c01ec7"} Dec 03 11:58:46 crc kubenswrapper[5052]: I1203 11:58:46.789503 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/swift-proxy-6555cb554-n2dp6"] Dec 03 11:58:46 crc kubenswrapper[5052]: I1203 11:58:46.791270 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-proxy-6555cb554-n2dp6" Dec 03 11:58:46 crc kubenswrapper[5052]: I1203 11:58:46.793208 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-swift-internal-svc" Dec 03 11:58:46 crc kubenswrapper[5052]: I1203 11:58:46.793613 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"swift-proxy-config-data" Dec 03 11:58:46 crc kubenswrapper[5052]: I1203 11:58:46.794577 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-swift-public-svc" Dec 03 11:58:46 crc kubenswrapper[5052]: I1203 11:58:46.820362 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/swift-proxy-6555cb554-n2dp6"] Dec 03 11:58:46 crc kubenswrapper[5052]: I1203 11:58:46.936520 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2mmxb\" (UniqueName: \"kubernetes.io/projected/8c848c79-8799-4e32-a635-958ce73341c4-kube-api-access-2mmxb\") pod \"swift-proxy-6555cb554-n2dp6\" (UID: \"8c848c79-8799-4e32-a635-958ce73341c4\") " pod="openstack-kuttl-tests/swift-proxy-6555cb554-n2dp6" Dec 03 11:58:46 crc kubenswrapper[5052]: I1203 11:58:46.936567 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8c848c79-8799-4e32-a635-958ce73341c4-combined-ca-bundle\") pod \"swift-proxy-6555cb554-n2dp6\" (UID: \"8c848c79-8799-4e32-a635-958ce73341c4\") " pod="openstack-kuttl-tests/swift-proxy-6555cb554-n2dp6" Dec 03 11:58:46 crc kubenswrapper[5052]: I1203 11:58:46.936613 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/8c848c79-8799-4e32-a635-958ce73341c4-etc-swift\") pod \"swift-proxy-6555cb554-n2dp6\" (UID: \"8c848c79-8799-4e32-a635-958ce73341c4\") " pod="openstack-kuttl-tests/swift-proxy-6555cb554-n2dp6" Dec 03 11:58:46 crc kubenswrapper[5052]: I1203 11:58:46.936640 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/8c848c79-8799-4e32-a635-958ce73341c4-public-tls-certs\") pod \"swift-proxy-6555cb554-n2dp6\" (UID: \"8c848c79-8799-4e32-a635-958ce73341c4\") " pod="openstack-kuttl-tests/swift-proxy-6555cb554-n2dp6" Dec 03 11:58:46 crc kubenswrapper[5052]: I1203 11:58:46.936657 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/8c848c79-8799-4e32-a635-958ce73341c4-config-data\") pod \"swift-proxy-6555cb554-n2dp6\" (UID: \"8c848c79-8799-4e32-a635-958ce73341c4\") " pod="openstack-kuttl-tests/swift-proxy-6555cb554-n2dp6" Dec 03 11:58:46 crc kubenswrapper[5052]: I1203 11:58:46.936690 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/8c848c79-8799-4e32-a635-958ce73341c4-log-httpd\") pod \"swift-proxy-6555cb554-n2dp6\" (UID: \"8c848c79-8799-4e32-a635-958ce73341c4\") " pod="openstack-kuttl-tests/swift-proxy-6555cb554-n2dp6" Dec 03 11:58:46 crc kubenswrapper[5052]: I1203 11:58:46.936716 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/8c848c79-8799-4e32-a635-958ce73341c4-run-httpd\") pod \"swift-proxy-6555cb554-n2dp6\" (UID: \"8c848c79-8799-4e32-a635-958ce73341c4\") " pod="openstack-kuttl-tests/swift-proxy-6555cb554-n2dp6" Dec 03 11:58:46 crc kubenswrapper[5052]: I1203 11:58:46.936744 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/8c848c79-8799-4e32-a635-958ce73341c4-internal-tls-certs\") pod \"swift-proxy-6555cb554-n2dp6\" (UID: \"8c848c79-8799-4e32-a635-958ce73341c4\") " pod="openstack-kuttl-tests/swift-proxy-6555cb554-n2dp6" Dec 03 11:58:47 crc kubenswrapper[5052]: I1203 11:58:47.039017 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/8c848c79-8799-4e32-a635-958ce73341c4-etc-swift\") pod \"swift-proxy-6555cb554-n2dp6\" (UID: \"8c848c79-8799-4e32-a635-958ce73341c4\") " pod="openstack-kuttl-tests/swift-proxy-6555cb554-n2dp6" Dec 03 11:58:47 crc kubenswrapper[5052]: I1203 11:58:47.039113 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/8c848c79-8799-4e32-a635-958ce73341c4-public-tls-certs\") pod \"swift-proxy-6555cb554-n2dp6\" (UID: \"8c848c79-8799-4e32-a635-958ce73341c4\") " pod="openstack-kuttl-tests/swift-proxy-6555cb554-n2dp6" Dec 03 11:58:47 crc kubenswrapper[5052]: I1203 11:58:47.039141 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/8c848c79-8799-4e32-a635-958ce73341c4-config-data\") pod \"swift-proxy-6555cb554-n2dp6\" (UID: \"8c848c79-8799-4e32-a635-958ce73341c4\") " pod="openstack-kuttl-tests/swift-proxy-6555cb554-n2dp6" Dec 03 11:58:47 crc kubenswrapper[5052]: I1203 11:58:47.039219 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/8c848c79-8799-4e32-a635-958ce73341c4-log-httpd\") pod \"swift-proxy-6555cb554-n2dp6\" (UID: \"8c848c79-8799-4e32-a635-958ce73341c4\") " pod="openstack-kuttl-tests/swift-proxy-6555cb554-n2dp6" Dec 03 11:58:47 crc kubenswrapper[5052]: I1203 11:58:47.039288 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/8c848c79-8799-4e32-a635-958ce73341c4-run-httpd\") pod \"swift-proxy-6555cb554-n2dp6\" (UID: \"8c848c79-8799-4e32-a635-958ce73341c4\") " pod="openstack-kuttl-tests/swift-proxy-6555cb554-n2dp6" Dec 03 11:58:47 crc kubenswrapper[5052]: I1203 11:58:47.039355 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/8c848c79-8799-4e32-a635-958ce73341c4-internal-tls-certs\") pod \"swift-proxy-6555cb554-n2dp6\" (UID: \"8c848c79-8799-4e32-a635-958ce73341c4\") " pod="openstack-kuttl-tests/swift-proxy-6555cb554-n2dp6" Dec 03 11:58:47 crc kubenswrapper[5052]: I1203 11:58:47.039452 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-2mmxb\" (UniqueName: \"kubernetes.io/projected/8c848c79-8799-4e32-a635-958ce73341c4-kube-api-access-2mmxb\") pod \"swift-proxy-6555cb554-n2dp6\" (UID: \"8c848c79-8799-4e32-a635-958ce73341c4\") " pod="openstack-kuttl-tests/swift-proxy-6555cb554-n2dp6" Dec 03 11:58:47 crc kubenswrapper[5052]: I1203 11:58:47.039495 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8c848c79-8799-4e32-a635-958ce73341c4-combined-ca-bundle\") pod \"swift-proxy-6555cb554-n2dp6\" (UID: \"8c848c79-8799-4e32-a635-958ce73341c4\") " pod="openstack-kuttl-tests/swift-proxy-6555cb554-n2dp6" Dec 03 11:58:47 crc kubenswrapper[5052]: I1203 11:58:47.040986 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/8c848c79-8799-4e32-a635-958ce73341c4-log-httpd\") pod \"swift-proxy-6555cb554-n2dp6\" (UID: \"8c848c79-8799-4e32-a635-958ce73341c4\") " pod="openstack-kuttl-tests/swift-proxy-6555cb554-n2dp6" Dec 03 11:58:47 crc kubenswrapper[5052]: I1203 11:58:47.044871 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/8c848c79-8799-4e32-a635-958ce73341c4-run-httpd\") pod \"swift-proxy-6555cb554-n2dp6\" (UID: \"8c848c79-8799-4e32-a635-958ce73341c4\") " pod="openstack-kuttl-tests/swift-proxy-6555cb554-n2dp6" Dec 03 11:58:47 crc kubenswrapper[5052]: I1203 11:58:47.048198 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/8c848c79-8799-4e32-a635-958ce73341c4-etc-swift\") pod \"swift-proxy-6555cb554-n2dp6\" (UID: \"8c848c79-8799-4e32-a635-958ce73341c4\") " pod="openstack-kuttl-tests/swift-proxy-6555cb554-n2dp6" Dec 03 11:58:47 crc kubenswrapper[5052]: I1203 11:58:47.056089 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/8c848c79-8799-4e32-a635-958ce73341c4-internal-tls-certs\") pod \"swift-proxy-6555cb554-n2dp6\" (UID: \"8c848c79-8799-4e32-a635-958ce73341c4\") " pod="openstack-kuttl-tests/swift-proxy-6555cb554-n2dp6" Dec 03 11:58:47 crc kubenswrapper[5052]: I1203 11:58:47.057713 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/8c848c79-8799-4e32-a635-958ce73341c4-config-data\") pod \"swift-proxy-6555cb554-n2dp6\" (UID: \"8c848c79-8799-4e32-a635-958ce73341c4\") " pod="openstack-kuttl-tests/swift-proxy-6555cb554-n2dp6" Dec 03 11:58:47 crc kubenswrapper[5052]: I1203 11:58:47.061288 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8c848c79-8799-4e32-a635-958ce73341c4-combined-ca-bundle\") pod \"swift-proxy-6555cb554-n2dp6\" (UID: \"8c848c79-8799-4e32-a635-958ce73341c4\") " pod="openstack-kuttl-tests/swift-proxy-6555cb554-n2dp6" Dec 03 11:58:47 crc kubenswrapper[5052]: I1203 11:58:47.071248 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/8c848c79-8799-4e32-a635-958ce73341c4-public-tls-certs\") pod \"swift-proxy-6555cb554-n2dp6\" (UID: \"8c848c79-8799-4e32-a635-958ce73341c4\") " pod="openstack-kuttl-tests/swift-proxy-6555cb554-n2dp6" Dec 03 11:58:47 crc kubenswrapper[5052]: I1203 11:58:47.107383 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-2mmxb\" (UniqueName: \"kubernetes.io/projected/8c848c79-8799-4e32-a635-958ce73341c4-kube-api-access-2mmxb\") pod \"swift-proxy-6555cb554-n2dp6\" (UID: \"8c848c79-8799-4e32-a635-958ce73341c4\") " pod="openstack-kuttl-tests/swift-proxy-6555cb554-n2dp6" Dec 03 11:58:47 crc kubenswrapper[5052]: I1203 11:58:47.110020 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-proxy-6555cb554-n2dp6" Dec 03 11:58:47 crc kubenswrapper[5052]: I1203 11:58:47.358482 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:58:47 crc kubenswrapper[5052]: I1203 11:58:47.359084 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="7724320e-d88f-40eb-8200-3732d97944fb" containerName="ceilometer-central-agent" containerID="cri-o://80c5885e88721179b4914941e8f2b1fd1771f258e1787f873289a0db7a3e98d8" gracePeriod=30 Dec 03 11:58:47 crc kubenswrapper[5052]: I1203 11:58:47.359619 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="7724320e-d88f-40eb-8200-3732d97944fb" containerName="proxy-httpd" containerID="cri-o://d3f7788549fe5083a4ffcb34e6dd06fab76a2e6bd9a093c7c5982ff1fb50728b" gracePeriod=30 Dec 03 11:58:47 crc kubenswrapper[5052]: I1203 11:58:47.359662 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="7724320e-d88f-40eb-8200-3732d97944fb" containerName="sg-core" containerID="cri-o://62d906ae5951f750faecdac5f48bd2e3f15dd18ff4b65d6d9234602103045354" gracePeriod=30 Dec 03 11:58:47 crc kubenswrapper[5052]: I1203 11:58:47.359691 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="7724320e-d88f-40eb-8200-3732d97944fb" containerName="ceilometer-notification-agent" containerID="cri-o://a5a9a0fc1f9628172fb97240315512de6c963cb035df3695470c5c34431a3c9e" gracePeriod=30 Dec 03 11:58:47 crc kubenswrapper[5052]: I1203 11:58:47.363626 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-marketplace-nzgnj" event={"ID":"8746e6ab-2863-45c2-b516-3b2bdfed2fb9","Type":"ContainerStarted","Data":"d52b345183e10e412ba8c94c5e5e15175c4339e5af63d381f6b95e790c2b3e85"} Dec 03 11:58:47 crc kubenswrapper[5052]: I1203 11:58:47.488847 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/swift-proxy-6555cb554-n2dp6"] Dec 03 11:58:47 crc kubenswrapper[5052]: W1203 11:58:47.492358 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8c848c79_8799_4e32_a635_958ce73341c4.slice/crio-7d4f5d001749900396d79238af55e15dacdbf7971ffed62de68ae484370385a2 WatchSource:0}: Error finding container 7d4f5d001749900396d79238af55e15dacdbf7971ffed62de68ae484370385a2: Status 404 returned error can't find the container with id 7d4f5d001749900396d79238af55e15dacdbf7971ffed62de68ae484370385a2 Dec 03 11:58:47 crc kubenswrapper[5052]: I1203 11:58:47.939919 5052 scope.go:117] "RemoveContainer" containerID="160567c413c48ad58e663b87e889640ffaea914541b8bd2d0d0fe0e8fa1833b4" Dec 03 11:58:47 crc kubenswrapper[5052]: I1203 11:58:47.963035 5052 scope.go:117] "RemoveContainer" containerID="47975dc763fa99fd8e031feff26f6ec865ddd0443ad75de0e4f2b3626568c51b" Dec 03 11:58:47 crc kubenswrapper[5052]: I1203 11:58:47.991851 5052 scope.go:117] "RemoveContainer" containerID="b83ee0552872f35b33c41234c7f87cdaa58debb1beeb08c6e6d9b6f3a591cb00" Dec 03 11:58:48 crc kubenswrapper[5052]: I1203 11:58:48.015281 5052 scope.go:117] "RemoveContainer" containerID="665790873f6c55ad2c188d468337affcedf0bb7b24e2eac8edb9cafae8234e8f" Dec 03 11:58:48 crc kubenswrapper[5052]: I1203 11:58:48.066441 5052 scope.go:117] "RemoveContainer" containerID="5b169bb512dbd1668acdc5b85816f5b2f20666e2a7728f2199a7f78b7ccc7e33" Dec 03 11:58:48 crc kubenswrapper[5052]: I1203 11:58:48.121796 5052 scope.go:117] "RemoveContainer" containerID="39d9f6dd82140e8dbcb6a630404644ad9185d3873550446e8d89fa09e6c54eb8" Dec 03 11:58:48 crc kubenswrapper[5052]: E1203 11:58:48.203440 5052 cadvisor_stats_provider.go:516] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/system.slice/rpm-ostreed.service\": RecentStats: unable to find data in memory cache]" Dec 03 11:58:48 crc kubenswrapper[5052]: I1203 11:58:48.350069 5052 scope.go:117] "RemoveContainer" containerID="caa4105376a4fd91a83a73b83c341fe59f8dc60a5bc1121168be275047db84c0" Dec 03 11:58:48 crc kubenswrapper[5052]: I1203 11:58:48.380965 5052 scope.go:117] "RemoveContainer" containerID="b297067c85cdfb444ef47e977436dd7c0943a8de9f725b758f5f37ef9fb27e07" Dec 03 11:58:48 crc kubenswrapper[5052]: I1203 11:58:48.390441 5052 generic.go:334] "Generic (PLEG): container finished" podID="7724320e-d88f-40eb-8200-3732d97944fb" containerID="d3f7788549fe5083a4ffcb34e6dd06fab76a2e6bd9a093c7c5982ff1fb50728b" exitCode=0 Dec 03 11:58:48 crc kubenswrapper[5052]: I1203 11:58:48.390468 5052 generic.go:334] "Generic (PLEG): container finished" podID="7724320e-d88f-40eb-8200-3732d97944fb" containerID="62d906ae5951f750faecdac5f48bd2e3f15dd18ff4b65d6d9234602103045354" exitCode=2 Dec 03 11:58:48 crc kubenswrapper[5052]: I1203 11:58:48.390478 5052 generic.go:334] "Generic (PLEG): container finished" podID="7724320e-d88f-40eb-8200-3732d97944fb" containerID="80c5885e88721179b4914941e8f2b1fd1771f258e1787f873289a0db7a3e98d8" exitCode=0 Dec 03 11:58:48 crc kubenswrapper[5052]: I1203 11:58:48.390532 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"7724320e-d88f-40eb-8200-3732d97944fb","Type":"ContainerDied","Data":"d3f7788549fe5083a4ffcb34e6dd06fab76a2e6bd9a093c7c5982ff1fb50728b"} Dec 03 11:58:48 crc kubenswrapper[5052]: I1203 11:58:48.390574 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"7724320e-d88f-40eb-8200-3732d97944fb","Type":"ContainerDied","Data":"62d906ae5951f750faecdac5f48bd2e3f15dd18ff4b65d6d9234602103045354"} Dec 03 11:58:48 crc kubenswrapper[5052]: I1203 11:58:48.390593 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"7724320e-d88f-40eb-8200-3732d97944fb","Type":"ContainerDied","Data":"80c5885e88721179b4914941e8f2b1fd1771f258e1787f873289a0db7a3e98d8"} Dec 03 11:58:48 crc kubenswrapper[5052]: I1203 11:58:48.396623 5052 generic.go:334] "Generic (PLEG): container finished" podID="8746e6ab-2863-45c2-b516-3b2bdfed2fb9" containerID="d52b345183e10e412ba8c94c5e5e15175c4339e5af63d381f6b95e790c2b3e85" exitCode=0 Dec 03 11:58:48 crc kubenswrapper[5052]: I1203 11:58:48.396791 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-marketplace-nzgnj" event={"ID":"8746e6ab-2863-45c2-b516-3b2bdfed2fb9","Type":"ContainerDied","Data":"d52b345183e10e412ba8c94c5e5e15175c4339e5af63d381f6b95e790c2b3e85"} Dec 03 11:58:48 crc kubenswrapper[5052]: I1203 11:58:48.401034 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-proxy-6555cb554-n2dp6" event={"ID":"8c848c79-8799-4e32-a635-958ce73341c4","Type":"ContainerStarted","Data":"ac84406b1507746af79e8f6d313cc521e0407948593d4c617a53957cf7d38341"} Dec 03 11:58:48 crc kubenswrapper[5052]: I1203 11:58:48.401082 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-proxy-6555cb554-n2dp6" event={"ID":"8c848c79-8799-4e32-a635-958ce73341c4","Type":"ContainerStarted","Data":"77bff56b6eb8629741796ab9f7892ba83978c309def196be2b0d6b39a27f09ec"} Dec 03 11:58:48 crc kubenswrapper[5052]: I1203 11:58:48.401096 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-proxy-6555cb554-n2dp6" event={"ID":"8c848c79-8799-4e32-a635-958ce73341c4","Type":"ContainerStarted","Data":"7d4f5d001749900396d79238af55e15dacdbf7971ffed62de68ae484370385a2"} Dec 03 11:58:48 crc kubenswrapper[5052]: I1203 11:58:48.401268 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/swift-proxy-6555cb554-n2dp6" Dec 03 11:58:48 crc kubenswrapper[5052]: I1203 11:58:48.401340 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/swift-proxy-6555cb554-n2dp6" Dec 03 11:58:48 crc kubenswrapper[5052]: I1203 11:58:48.430867 5052 scope.go:117] "RemoveContainer" containerID="89d88ba4bc869ab32cad85adfbac99355488669b3081a20686a3e517b6deb8f8" Dec 03 11:58:48 crc kubenswrapper[5052]: I1203 11:58:48.452141 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/swift-proxy-6555cb554-n2dp6" podStartSLOduration=2.4521218830000002 podStartE2EDuration="2.452121883s" podCreationTimestamp="2025-12-03 11:58:46 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:58:48.449451326 +0000 UTC m=+2319.037693090" watchObservedRunningTime="2025-12-03 11:58:48.452121883 +0000 UTC m=+2319.040363647" Dec 03 11:58:48 crc kubenswrapper[5052]: I1203 11:58:48.509156 5052 scope.go:117] "RemoveContainer" containerID="ca10bcd783453e6f7e39b4abea8415b2fde88b6f5ba39461841a0649464dfe87" Dec 03 11:58:48 crc kubenswrapper[5052]: I1203 11:58:48.534636 5052 scope.go:117] "RemoveContainer" containerID="7458d0a52664f876e9b1ff1ffa9b479a1d19950d3701f07e57764be8b8c11e99" Dec 03 11:58:48 crc kubenswrapper[5052]: I1203 11:58:48.631818 5052 scope.go:117] "RemoveContainer" containerID="12163794070d86f13c99de7a2ca7f5dc22e8548c4c16239dc6f3c9916233ac1d" Dec 03 11:58:48 crc kubenswrapper[5052]: I1203 11:58:48.672906 5052 scope.go:117] "RemoveContainer" containerID="6abb759f1844b571cc5be4dcbd64ddf7b2892093b015739e89b0a35c6e169927" Dec 03 11:58:48 crc kubenswrapper[5052]: I1203 11:58:48.673761 5052 scope.go:117] "RemoveContainer" containerID="d224717c6d9ca8135fd510e03a6c40c9f2b1a56dd8b4a3b685af3e2dfa855982" Dec 03 11:58:48 crc kubenswrapper[5052]: E1203 11:58:48.674024 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 11:58:49 crc kubenswrapper[5052]: I1203 11:58:49.423455 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-marketplace-nzgnj" event={"ID":"8746e6ab-2863-45c2-b516-3b2bdfed2fb9","Type":"ContainerStarted","Data":"2de3285792422077503aa990bb4ca5abad10c82f4b20745ad58f5159cc75f2c1"} Dec 03 11:58:49 crc kubenswrapper[5052]: I1203 11:58:49.455253 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-marketplace/redhat-marketplace-nzgnj" podStartSLOduration=2.837478133 podStartE2EDuration="5.455228499s" podCreationTimestamp="2025-12-03 11:58:44 +0000 UTC" firstStartedPulling="2025-12-03 11:58:46.343616299 +0000 UTC m=+2316.931858063" lastFinishedPulling="2025-12-03 11:58:48.961366675 +0000 UTC m=+2319.549608429" observedRunningTime="2025-12-03 11:58:49.448311171 +0000 UTC m=+2320.036552935" watchObservedRunningTime="2025-12-03 11:58:49.455228499 +0000 UTC m=+2320.043470273" Dec 03 11:58:50 crc kubenswrapper[5052]: I1203 11:58:50.203007 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:58:50 crc kubenswrapper[5052]: I1203 11:58:50.304660 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/7724320e-d88f-40eb-8200-3732d97944fb-log-httpd\") pod \"7724320e-d88f-40eb-8200-3732d97944fb\" (UID: \"7724320e-d88f-40eb-8200-3732d97944fb\") " Dec 03 11:58:50 crc kubenswrapper[5052]: I1203 11:58:50.304770 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/7724320e-d88f-40eb-8200-3732d97944fb-config-data\") pod \"7724320e-d88f-40eb-8200-3732d97944fb\" (UID: \"7724320e-d88f-40eb-8200-3732d97944fb\") " Dec 03 11:58:50 crc kubenswrapper[5052]: I1203 11:58:50.304826 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/7724320e-d88f-40eb-8200-3732d97944fb-run-httpd\") pod \"7724320e-d88f-40eb-8200-3732d97944fb\" (UID: \"7724320e-d88f-40eb-8200-3732d97944fb\") " Dec 03 11:58:50 crc kubenswrapper[5052]: I1203 11:58:50.304907 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/7724320e-d88f-40eb-8200-3732d97944fb-scripts\") pod \"7724320e-d88f-40eb-8200-3732d97944fb\" (UID: \"7724320e-d88f-40eb-8200-3732d97944fb\") " Dec 03 11:58:50 crc kubenswrapper[5052]: I1203 11:58:50.304971 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/7724320e-d88f-40eb-8200-3732d97944fb-combined-ca-bundle\") pod \"7724320e-d88f-40eb-8200-3732d97944fb\" (UID: \"7724320e-d88f-40eb-8200-3732d97944fb\") " Dec 03 11:58:50 crc kubenswrapper[5052]: I1203 11:58:50.305013 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pg7v6\" (UniqueName: \"kubernetes.io/projected/7724320e-d88f-40eb-8200-3732d97944fb-kube-api-access-pg7v6\") pod \"7724320e-d88f-40eb-8200-3732d97944fb\" (UID: \"7724320e-d88f-40eb-8200-3732d97944fb\") " Dec 03 11:58:50 crc kubenswrapper[5052]: I1203 11:58:50.305061 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/7724320e-d88f-40eb-8200-3732d97944fb-sg-core-conf-yaml\") pod \"7724320e-d88f-40eb-8200-3732d97944fb\" (UID: \"7724320e-d88f-40eb-8200-3732d97944fb\") " Dec 03 11:58:50 crc kubenswrapper[5052]: I1203 11:58:50.305096 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7724320e-d88f-40eb-8200-3732d97944fb-log-httpd" (OuterVolumeSpecName: "log-httpd") pod "7724320e-d88f-40eb-8200-3732d97944fb" (UID: "7724320e-d88f-40eb-8200-3732d97944fb"). InnerVolumeSpecName "log-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:58:50 crc kubenswrapper[5052]: I1203 11:58:50.305426 5052 reconciler_common.go:293] "Volume detached for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/7724320e-d88f-40eb-8200-3732d97944fb-log-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 11:58:50 crc kubenswrapper[5052]: I1203 11:58:50.305623 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7724320e-d88f-40eb-8200-3732d97944fb-run-httpd" (OuterVolumeSpecName: "run-httpd") pod "7724320e-d88f-40eb-8200-3732d97944fb" (UID: "7724320e-d88f-40eb-8200-3732d97944fb"). InnerVolumeSpecName "run-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:58:50 crc kubenswrapper[5052]: I1203 11:58:50.310176 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7724320e-d88f-40eb-8200-3732d97944fb-kube-api-access-pg7v6" (OuterVolumeSpecName: "kube-api-access-pg7v6") pod "7724320e-d88f-40eb-8200-3732d97944fb" (UID: "7724320e-d88f-40eb-8200-3732d97944fb"). InnerVolumeSpecName "kube-api-access-pg7v6". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:58:50 crc kubenswrapper[5052]: I1203 11:58:50.310333 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7724320e-d88f-40eb-8200-3732d97944fb-scripts" (OuterVolumeSpecName: "scripts") pod "7724320e-d88f-40eb-8200-3732d97944fb" (UID: "7724320e-d88f-40eb-8200-3732d97944fb"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:58:50 crc kubenswrapper[5052]: I1203 11:58:50.343194 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7724320e-d88f-40eb-8200-3732d97944fb-sg-core-conf-yaml" (OuterVolumeSpecName: "sg-core-conf-yaml") pod "7724320e-d88f-40eb-8200-3732d97944fb" (UID: "7724320e-d88f-40eb-8200-3732d97944fb"). InnerVolumeSpecName "sg-core-conf-yaml". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:58:50 crc kubenswrapper[5052]: I1203 11:58:50.406562 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7724320e-d88f-40eb-8200-3732d97944fb-config-data" (OuterVolumeSpecName: "config-data") pod "7724320e-d88f-40eb-8200-3732d97944fb" (UID: "7724320e-d88f-40eb-8200-3732d97944fb"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:58:50 crc kubenswrapper[5052]: I1203 11:58:50.407061 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/7724320e-d88f-40eb-8200-3732d97944fb-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:58:50 crc kubenswrapper[5052]: I1203 11:58:50.407084 5052 reconciler_common.go:293] "Volume detached for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/7724320e-d88f-40eb-8200-3732d97944fb-run-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 11:58:50 crc kubenswrapper[5052]: I1203 11:58:50.407097 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/7724320e-d88f-40eb-8200-3732d97944fb-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:58:50 crc kubenswrapper[5052]: I1203 11:58:50.407108 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-pg7v6\" (UniqueName: \"kubernetes.io/projected/7724320e-d88f-40eb-8200-3732d97944fb-kube-api-access-pg7v6\") on node \"crc\" DevicePath \"\"" Dec 03 11:58:50 crc kubenswrapper[5052]: I1203 11:58:50.407121 5052 reconciler_common.go:293] "Volume detached for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/7724320e-d88f-40eb-8200-3732d97944fb-sg-core-conf-yaml\") on node \"crc\" DevicePath \"\"" Dec 03 11:58:50 crc kubenswrapper[5052]: I1203 11:58:50.416620 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7724320e-d88f-40eb-8200-3732d97944fb-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "7724320e-d88f-40eb-8200-3732d97944fb" (UID: "7724320e-d88f-40eb-8200-3732d97944fb"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:58:50 crc kubenswrapper[5052]: I1203 11:58:50.433739 5052 generic.go:334] "Generic (PLEG): container finished" podID="7724320e-d88f-40eb-8200-3732d97944fb" containerID="a5a9a0fc1f9628172fb97240315512de6c963cb035df3695470c5c34431a3c9e" exitCode=0 Dec 03 11:58:50 crc kubenswrapper[5052]: I1203 11:58:50.435853 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:58:50 crc kubenswrapper[5052]: I1203 11:58:50.436569 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"7724320e-d88f-40eb-8200-3732d97944fb","Type":"ContainerDied","Data":"a5a9a0fc1f9628172fb97240315512de6c963cb035df3695470c5c34431a3c9e"} Dec 03 11:58:50 crc kubenswrapper[5052]: I1203 11:58:50.436628 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"7724320e-d88f-40eb-8200-3732d97944fb","Type":"ContainerDied","Data":"24684a3db7378d1ddf53fbf9b0b30b8cd80c753998289e7f290b173da1226c64"} Dec 03 11:58:50 crc kubenswrapper[5052]: I1203 11:58:50.436648 5052 scope.go:117] "RemoveContainer" containerID="d3f7788549fe5083a4ffcb34e6dd06fab76a2e6bd9a093c7c5982ff1fb50728b" Dec 03 11:58:50 crc kubenswrapper[5052]: I1203 11:58:50.476850 5052 scope.go:117] "RemoveContainer" containerID="62d906ae5951f750faecdac5f48bd2e3f15dd18ff4b65d6d9234602103045354" Dec 03 11:58:50 crc kubenswrapper[5052]: I1203 11:58:50.494690 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:58:50 crc kubenswrapper[5052]: I1203 11:58:50.500885 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:58:50 crc kubenswrapper[5052]: I1203 11:58:50.510078 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/7724320e-d88f-40eb-8200-3732d97944fb-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:58:50 crc kubenswrapper[5052]: I1203 11:58:50.522391 5052 scope.go:117] "RemoveContainer" containerID="a5a9a0fc1f9628172fb97240315512de6c963cb035df3695470c5c34431a3c9e" Dec 03 11:58:50 crc kubenswrapper[5052]: I1203 11:58:50.533081 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:58:50 crc kubenswrapper[5052]: E1203 11:58:50.533656 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="7724320e-d88f-40eb-8200-3732d97944fb" containerName="ceilometer-notification-agent" Dec 03 11:58:50 crc kubenswrapper[5052]: I1203 11:58:50.533683 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="7724320e-d88f-40eb-8200-3732d97944fb" containerName="ceilometer-notification-agent" Dec 03 11:58:50 crc kubenswrapper[5052]: E1203 11:58:50.533701 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="7724320e-d88f-40eb-8200-3732d97944fb" containerName="proxy-httpd" Dec 03 11:58:50 crc kubenswrapper[5052]: I1203 11:58:50.533712 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="7724320e-d88f-40eb-8200-3732d97944fb" containerName="proxy-httpd" Dec 03 11:58:50 crc kubenswrapper[5052]: E1203 11:58:50.533729 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="7724320e-d88f-40eb-8200-3732d97944fb" containerName="ceilometer-central-agent" Dec 03 11:58:50 crc kubenswrapper[5052]: I1203 11:58:50.533737 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="7724320e-d88f-40eb-8200-3732d97944fb" containerName="ceilometer-central-agent" Dec 03 11:58:50 crc kubenswrapper[5052]: E1203 11:58:50.533759 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="7724320e-d88f-40eb-8200-3732d97944fb" containerName="sg-core" Dec 03 11:58:50 crc kubenswrapper[5052]: I1203 11:58:50.533766 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="7724320e-d88f-40eb-8200-3732d97944fb" containerName="sg-core" Dec 03 11:58:50 crc kubenswrapper[5052]: I1203 11:58:50.534028 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="7724320e-d88f-40eb-8200-3732d97944fb" containerName="proxy-httpd" Dec 03 11:58:50 crc kubenswrapper[5052]: I1203 11:58:50.534087 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="7724320e-d88f-40eb-8200-3732d97944fb" containerName="ceilometer-notification-agent" Dec 03 11:58:50 crc kubenswrapper[5052]: I1203 11:58:50.534097 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="7724320e-d88f-40eb-8200-3732d97944fb" containerName="ceilometer-central-agent" Dec 03 11:58:50 crc kubenswrapper[5052]: I1203 11:58:50.534111 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="7724320e-d88f-40eb-8200-3732d97944fb" containerName="sg-core" Dec 03 11:58:50 crc kubenswrapper[5052]: I1203 11:58:50.537456 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:58:50 crc kubenswrapper[5052]: I1203 11:58:50.541116 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-scripts" Dec 03 11:58:50 crc kubenswrapper[5052]: I1203 11:58:50.541347 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-config-data" Dec 03 11:58:50 crc kubenswrapper[5052]: I1203 11:58:50.547982 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:58:50 crc kubenswrapper[5052]: I1203 11:58:50.603818 5052 scope.go:117] "RemoveContainer" containerID="80c5885e88721179b4914941e8f2b1fd1771f258e1787f873289a0db7a3e98d8" Dec 03 11:58:50 crc kubenswrapper[5052]: I1203 11:58:50.611704 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/977a3b90-7dc4-4e79-9117-6218dccc8f5e-scripts\") pod \"ceilometer-0\" (UID: \"977a3b90-7dc4-4e79-9117-6218dccc8f5e\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:58:50 crc kubenswrapper[5052]: I1203 11:58:50.613807 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/977a3b90-7dc4-4e79-9117-6218dccc8f5e-config-data\") pod \"ceilometer-0\" (UID: \"977a3b90-7dc4-4e79-9117-6218dccc8f5e\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:58:50 crc kubenswrapper[5052]: I1203 11:58:50.613889 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/977a3b90-7dc4-4e79-9117-6218dccc8f5e-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"977a3b90-7dc4-4e79-9117-6218dccc8f5e\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:58:50 crc kubenswrapper[5052]: I1203 11:58:50.613933 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/977a3b90-7dc4-4e79-9117-6218dccc8f5e-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"977a3b90-7dc4-4e79-9117-6218dccc8f5e\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:58:50 crc kubenswrapper[5052]: I1203 11:58:50.613979 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fmcv7\" (UniqueName: \"kubernetes.io/projected/977a3b90-7dc4-4e79-9117-6218dccc8f5e-kube-api-access-fmcv7\") pod \"ceilometer-0\" (UID: \"977a3b90-7dc4-4e79-9117-6218dccc8f5e\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:58:50 crc kubenswrapper[5052]: I1203 11:58:50.614053 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/977a3b90-7dc4-4e79-9117-6218dccc8f5e-log-httpd\") pod \"ceilometer-0\" (UID: \"977a3b90-7dc4-4e79-9117-6218dccc8f5e\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:58:50 crc kubenswrapper[5052]: I1203 11:58:50.614090 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/977a3b90-7dc4-4e79-9117-6218dccc8f5e-run-httpd\") pod \"ceilometer-0\" (UID: \"977a3b90-7dc4-4e79-9117-6218dccc8f5e\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:58:50 crc kubenswrapper[5052]: I1203 11:58:50.649206 5052 scope.go:117] "RemoveContainer" containerID="d3f7788549fe5083a4ffcb34e6dd06fab76a2e6bd9a093c7c5982ff1fb50728b" Dec 03 11:58:50 crc kubenswrapper[5052]: E1203 11:58:50.651058 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d3f7788549fe5083a4ffcb34e6dd06fab76a2e6bd9a093c7c5982ff1fb50728b\": container with ID starting with d3f7788549fe5083a4ffcb34e6dd06fab76a2e6bd9a093c7c5982ff1fb50728b not found: ID does not exist" containerID="d3f7788549fe5083a4ffcb34e6dd06fab76a2e6bd9a093c7c5982ff1fb50728b" Dec 03 11:58:50 crc kubenswrapper[5052]: I1203 11:58:50.651127 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d3f7788549fe5083a4ffcb34e6dd06fab76a2e6bd9a093c7c5982ff1fb50728b"} err="failed to get container status \"d3f7788549fe5083a4ffcb34e6dd06fab76a2e6bd9a093c7c5982ff1fb50728b\": rpc error: code = NotFound desc = could not find container \"d3f7788549fe5083a4ffcb34e6dd06fab76a2e6bd9a093c7c5982ff1fb50728b\": container with ID starting with d3f7788549fe5083a4ffcb34e6dd06fab76a2e6bd9a093c7c5982ff1fb50728b not found: ID does not exist" Dec 03 11:58:50 crc kubenswrapper[5052]: I1203 11:58:50.651161 5052 scope.go:117] "RemoveContainer" containerID="62d906ae5951f750faecdac5f48bd2e3f15dd18ff4b65d6d9234602103045354" Dec 03 11:58:50 crc kubenswrapper[5052]: E1203 11:58:50.651778 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"62d906ae5951f750faecdac5f48bd2e3f15dd18ff4b65d6d9234602103045354\": container with ID starting with 62d906ae5951f750faecdac5f48bd2e3f15dd18ff4b65d6d9234602103045354 not found: ID does not exist" containerID="62d906ae5951f750faecdac5f48bd2e3f15dd18ff4b65d6d9234602103045354" Dec 03 11:58:50 crc kubenswrapper[5052]: I1203 11:58:50.651893 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"62d906ae5951f750faecdac5f48bd2e3f15dd18ff4b65d6d9234602103045354"} err="failed to get container status \"62d906ae5951f750faecdac5f48bd2e3f15dd18ff4b65d6d9234602103045354\": rpc error: code = NotFound desc = could not find container \"62d906ae5951f750faecdac5f48bd2e3f15dd18ff4b65d6d9234602103045354\": container with ID starting with 62d906ae5951f750faecdac5f48bd2e3f15dd18ff4b65d6d9234602103045354 not found: ID does not exist" Dec 03 11:58:50 crc kubenswrapper[5052]: I1203 11:58:50.651989 5052 scope.go:117] "RemoveContainer" containerID="a5a9a0fc1f9628172fb97240315512de6c963cb035df3695470c5c34431a3c9e" Dec 03 11:58:50 crc kubenswrapper[5052]: E1203 11:58:50.652382 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a5a9a0fc1f9628172fb97240315512de6c963cb035df3695470c5c34431a3c9e\": container with ID starting with a5a9a0fc1f9628172fb97240315512de6c963cb035df3695470c5c34431a3c9e not found: ID does not exist" containerID="a5a9a0fc1f9628172fb97240315512de6c963cb035df3695470c5c34431a3c9e" Dec 03 11:58:50 crc kubenswrapper[5052]: I1203 11:58:50.652417 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a5a9a0fc1f9628172fb97240315512de6c963cb035df3695470c5c34431a3c9e"} err="failed to get container status \"a5a9a0fc1f9628172fb97240315512de6c963cb035df3695470c5c34431a3c9e\": rpc error: code = NotFound desc = could not find container \"a5a9a0fc1f9628172fb97240315512de6c963cb035df3695470c5c34431a3c9e\": container with ID starting with a5a9a0fc1f9628172fb97240315512de6c963cb035df3695470c5c34431a3c9e not found: ID does not exist" Dec 03 11:58:50 crc kubenswrapper[5052]: I1203 11:58:50.652438 5052 scope.go:117] "RemoveContainer" containerID="80c5885e88721179b4914941e8f2b1fd1771f258e1787f873289a0db7a3e98d8" Dec 03 11:58:50 crc kubenswrapper[5052]: E1203 11:58:50.652767 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"80c5885e88721179b4914941e8f2b1fd1771f258e1787f873289a0db7a3e98d8\": container with ID starting with 80c5885e88721179b4914941e8f2b1fd1771f258e1787f873289a0db7a3e98d8 not found: ID does not exist" containerID="80c5885e88721179b4914941e8f2b1fd1771f258e1787f873289a0db7a3e98d8" Dec 03 11:58:50 crc kubenswrapper[5052]: I1203 11:58:50.652835 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"80c5885e88721179b4914941e8f2b1fd1771f258e1787f873289a0db7a3e98d8"} err="failed to get container status \"80c5885e88721179b4914941e8f2b1fd1771f258e1787f873289a0db7a3e98d8\": rpc error: code = NotFound desc = could not find container \"80c5885e88721179b4914941e8f2b1fd1771f258e1787f873289a0db7a3e98d8\": container with ID starting with 80c5885e88721179b4914941e8f2b1fd1771f258e1787f873289a0db7a3e98d8 not found: ID does not exist" Dec 03 11:58:50 crc kubenswrapper[5052]: I1203 11:58:50.716365 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/977a3b90-7dc4-4e79-9117-6218dccc8f5e-config-data\") pod \"ceilometer-0\" (UID: \"977a3b90-7dc4-4e79-9117-6218dccc8f5e\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:58:50 crc kubenswrapper[5052]: I1203 11:58:50.716426 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/977a3b90-7dc4-4e79-9117-6218dccc8f5e-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"977a3b90-7dc4-4e79-9117-6218dccc8f5e\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:58:50 crc kubenswrapper[5052]: I1203 11:58:50.716458 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/977a3b90-7dc4-4e79-9117-6218dccc8f5e-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"977a3b90-7dc4-4e79-9117-6218dccc8f5e\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:58:50 crc kubenswrapper[5052]: I1203 11:58:50.716487 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-fmcv7\" (UniqueName: \"kubernetes.io/projected/977a3b90-7dc4-4e79-9117-6218dccc8f5e-kube-api-access-fmcv7\") pod \"ceilometer-0\" (UID: \"977a3b90-7dc4-4e79-9117-6218dccc8f5e\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:58:50 crc kubenswrapper[5052]: I1203 11:58:50.716541 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/977a3b90-7dc4-4e79-9117-6218dccc8f5e-log-httpd\") pod \"ceilometer-0\" (UID: \"977a3b90-7dc4-4e79-9117-6218dccc8f5e\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:58:50 crc kubenswrapper[5052]: I1203 11:58:50.716567 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/977a3b90-7dc4-4e79-9117-6218dccc8f5e-run-httpd\") pod \"ceilometer-0\" (UID: \"977a3b90-7dc4-4e79-9117-6218dccc8f5e\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:58:50 crc kubenswrapper[5052]: I1203 11:58:50.716688 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/977a3b90-7dc4-4e79-9117-6218dccc8f5e-scripts\") pod \"ceilometer-0\" (UID: \"977a3b90-7dc4-4e79-9117-6218dccc8f5e\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:58:50 crc kubenswrapper[5052]: I1203 11:58:50.717101 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/977a3b90-7dc4-4e79-9117-6218dccc8f5e-log-httpd\") pod \"ceilometer-0\" (UID: \"977a3b90-7dc4-4e79-9117-6218dccc8f5e\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:58:50 crc kubenswrapper[5052]: I1203 11:58:50.717122 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/977a3b90-7dc4-4e79-9117-6218dccc8f5e-run-httpd\") pod \"ceilometer-0\" (UID: \"977a3b90-7dc4-4e79-9117-6218dccc8f5e\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:58:50 crc kubenswrapper[5052]: I1203 11:58:50.721920 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/977a3b90-7dc4-4e79-9117-6218dccc8f5e-scripts\") pod \"ceilometer-0\" (UID: \"977a3b90-7dc4-4e79-9117-6218dccc8f5e\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:58:50 crc kubenswrapper[5052]: I1203 11:58:50.723349 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/977a3b90-7dc4-4e79-9117-6218dccc8f5e-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"977a3b90-7dc4-4e79-9117-6218dccc8f5e\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:58:50 crc kubenswrapper[5052]: I1203 11:58:50.723722 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/977a3b90-7dc4-4e79-9117-6218dccc8f5e-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"977a3b90-7dc4-4e79-9117-6218dccc8f5e\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:58:50 crc kubenswrapper[5052]: I1203 11:58:50.724410 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/977a3b90-7dc4-4e79-9117-6218dccc8f5e-config-data\") pod \"ceilometer-0\" (UID: \"977a3b90-7dc4-4e79-9117-6218dccc8f5e\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:58:50 crc kubenswrapper[5052]: I1203 11:58:50.735200 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-fmcv7\" (UniqueName: \"kubernetes.io/projected/977a3b90-7dc4-4e79-9117-6218dccc8f5e-kube-api-access-fmcv7\") pod \"ceilometer-0\" (UID: \"977a3b90-7dc4-4e79-9117-6218dccc8f5e\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:58:50 crc kubenswrapper[5052]: I1203 11:58:50.895706 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:58:51 crc kubenswrapper[5052]: I1203 11:58:51.359689 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:58:51 crc kubenswrapper[5052]: W1203 11:58:51.361788 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod977a3b90_7dc4_4e79_9117_6218dccc8f5e.slice/crio-89aecbd57886bf6651f4421f3e61f1ba3e578f9809d5ba0f946fa80cb9b6d27f WatchSource:0}: Error finding container 89aecbd57886bf6651f4421f3e61f1ba3e578f9809d5ba0f946fa80cb9b6d27f: Status 404 returned error can't find the container with id 89aecbd57886bf6651f4421f3e61f1ba3e578f9809d5ba0f946fa80cb9b6d27f Dec 03 11:58:51 crc kubenswrapper[5052]: I1203 11:58:51.446475 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"977a3b90-7dc4-4e79-9117-6218dccc8f5e","Type":"ContainerStarted","Data":"89aecbd57886bf6651f4421f3e61f1ba3e578f9809d5ba0f946fa80cb9b6d27f"} Dec 03 11:58:51 crc kubenswrapper[5052]: I1203 11:58:51.685488 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7724320e-d88f-40eb-8200-3732d97944fb" path="/var/lib/kubelet/pods/7724320e-d88f-40eb-8200-3732d97944fb/volumes" Dec 03 11:58:52 crc kubenswrapper[5052]: I1203 11:58:52.125916 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/swift-proxy-6555cb554-n2dp6" Dec 03 11:58:52 crc kubenswrapper[5052]: I1203 11:58:52.128799 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/swift-proxy-6555cb554-n2dp6" Dec 03 11:58:52 crc kubenswrapper[5052]: I1203 11:58:52.455345 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"977a3b90-7dc4-4e79-9117-6218dccc8f5e","Type":"ContainerStarted","Data":"add40057fd805113d3fbd97ea999b349355c3d1189310f1971b04d3b3e9da055"} Dec 03 11:58:52 crc kubenswrapper[5052]: I1203 11:58:52.648744 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:58:53 crc kubenswrapper[5052]: I1203 11:58:53.466122 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"977a3b90-7dc4-4e79-9117-6218dccc8f5e","Type":"ContainerStarted","Data":"1fd1a3840b23019c4246810ff2fe1c0d61d03f5a8efb0d7bffa142c536cd7db9"} Dec 03 11:58:54 crc kubenswrapper[5052]: I1203 11:58:53.999751 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/watcher-decision-engine-0"] Dec 03 11:58:54 crc kubenswrapper[5052]: I1203 11:58:54.000317 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/watcher-decision-engine-0" podUID="5893db31-2978-45eb-8407-2e4d178954ce" containerName="watcher-decision-engine" containerID="cri-o://d7086739b41b7d360cbe4227c0a28ecffd42edb2ee30fb5fcc39d0c67f27ead1" gracePeriod=30 Dec 03 11:58:54 crc kubenswrapper[5052]: I1203 11:58:54.476963 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"977a3b90-7dc4-4e79-9117-6218dccc8f5e","Type":"ContainerStarted","Data":"087b4fae22b4d3f9c083241ea4ff6b9b34dd8e9a9eb57381f29a4eeac60c24fb"} Dec 03 11:58:55 crc kubenswrapper[5052]: I1203 11:58:55.300144 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openshift-marketplace/redhat-marketplace-nzgnj" Dec 03 11:58:55 crc kubenswrapper[5052]: I1203 11:58:55.300333 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-marketplace/redhat-marketplace-nzgnj" Dec 03 11:58:55 crc kubenswrapper[5052]: I1203 11:58:55.358942 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-marketplace/redhat-marketplace-nzgnj" Dec 03 11:58:55 crc kubenswrapper[5052]: I1203 11:58:55.551426 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-marketplace/redhat-marketplace-nzgnj" Dec 03 11:58:55 crc kubenswrapper[5052]: I1203 11:58:55.610288 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openshift-marketplace/redhat-marketplace-nzgnj"] Dec 03 11:58:55 crc kubenswrapper[5052]: I1203 11:58:55.885058 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 11:58:55 crc kubenswrapper[5052]: I1203 11:58:55.885616 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-external-api-0" podUID="9fe1f02f-d2cf-4220-925c-6ecdcd099048" containerName="glance-log" containerID="cri-o://cb18e4b2f1c1bd57f2ceaa59ede059a92c607d787ad9e92d233d8bbb0cff36be" gracePeriod=30 Dec 03 11:58:55 crc kubenswrapper[5052]: I1203 11:58:55.885695 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-external-api-0" podUID="9fe1f02f-d2cf-4220-925c-6ecdcd099048" containerName="glance-httpd" containerID="cri-o://0d2b603ec774a9e5e57e3ee6ad57ea900ac923cb148ec977d0759fb33a4ba2e8" gracePeriod=30 Dec 03 11:58:56 crc kubenswrapper[5052]: I1203 11:58:56.501706 5052 generic.go:334] "Generic (PLEG): container finished" podID="9fe1f02f-d2cf-4220-925c-6ecdcd099048" containerID="cb18e4b2f1c1bd57f2ceaa59ede059a92c607d787ad9e92d233d8bbb0cff36be" exitCode=143 Dec 03 11:58:56 crc kubenswrapper[5052]: I1203 11:58:56.501791 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"9fe1f02f-d2cf-4220-925c-6ecdcd099048","Type":"ContainerDied","Data":"cb18e4b2f1c1bd57f2ceaa59ede059a92c607d787ad9e92d233d8bbb0cff36be"} Dec 03 11:58:56 crc kubenswrapper[5052]: I1203 11:58:56.504576 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"977a3b90-7dc4-4e79-9117-6218dccc8f5e","Type":"ContainerStarted","Data":"950e6e25edb77f3fc2134f5dd19d895d398dd8572d4e0481d5c706ccc6c62d32"} Dec 03 11:58:56 crc kubenswrapper[5052]: I1203 11:58:56.504805 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="977a3b90-7dc4-4e79-9117-6218dccc8f5e" containerName="ceilometer-central-agent" containerID="cri-o://add40057fd805113d3fbd97ea999b349355c3d1189310f1971b04d3b3e9da055" gracePeriod=30 Dec 03 11:58:56 crc kubenswrapper[5052]: I1203 11:58:56.504825 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="977a3b90-7dc4-4e79-9117-6218dccc8f5e" containerName="proxy-httpd" containerID="cri-o://950e6e25edb77f3fc2134f5dd19d895d398dd8572d4e0481d5c706ccc6c62d32" gracePeriod=30 Dec 03 11:58:56 crc kubenswrapper[5052]: I1203 11:58:56.504888 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="977a3b90-7dc4-4e79-9117-6218dccc8f5e" containerName="sg-core" containerID="cri-o://087b4fae22b4d3f9c083241ea4ff6b9b34dd8e9a9eb57381f29a4eeac60c24fb" gracePeriod=30 Dec 03 11:58:56 crc kubenswrapper[5052]: I1203 11:58:56.504924 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="977a3b90-7dc4-4e79-9117-6218dccc8f5e" containerName="ceilometer-notification-agent" containerID="cri-o://1fd1a3840b23019c4246810ff2fe1c0d61d03f5a8efb0d7bffa142c536cd7db9" gracePeriod=30 Dec 03 11:58:56 crc kubenswrapper[5052]: I1203 11:58:56.533479 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/ceilometer-0" podStartSLOduration=2.4967718740000002 podStartE2EDuration="6.533456758s" podCreationTimestamp="2025-12-03 11:58:50 +0000 UTC" firstStartedPulling="2025-12-03 11:58:51.364554962 +0000 UTC m=+2321.952796736" lastFinishedPulling="2025-12-03 11:58:55.401239856 +0000 UTC m=+2325.989481620" observedRunningTime="2025-12-03 11:58:56.530631337 +0000 UTC m=+2327.118873101" watchObservedRunningTime="2025-12-03 11:58:56.533456758 +0000 UTC m=+2327.121698522" Dec 03 11:58:57 crc kubenswrapper[5052]: I1203 11:58:57.174442 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 11:58:57 crc kubenswrapper[5052]: I1203 11:58:57.175024 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-internal-api-0" podUID="0d55422b-84bf-4455-9112-94de20fb5b6d" containerName="glance-log" containerID="cri-o://f3557803e708885b7be19d0c0a7d6e82313f419fd68f6cf32092dc379a16b9af" gracePeriod=30 Dec 03 11:58:57 crc kubenswrapper[5052]: I1203 11:58:57.175158 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-internal-api-0" podUID="0d55422b-84bf-4455-9112-94de20fb5b6d" containerName="glance-httpd" containerID="cri-o://2bf75f1bec647492a24e9e2e2a4b240c1dcac0144cf5a6f394ba143a1374064f" gracePeriod=30 Dec 03 11:58:57 crc kubenswrapper[5052]: I1203 11:58:57.337138 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:58:57 crc kubenswrapper[5052]: I1203 11:58:57.472304 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/977a3b90-7dc4-4e79-9117-6218dccc8f5e-run-httpd\") pod \"977a3b90-7dc4-4e79-9117-6218dccc8f5e\" (UID: \"977a3b90-7dc4-4e79-9117-6218dccc8f5e\") " Dec 03 11:58:57 crc kubenswrapper[5052]: I1203 11:58:57.472417 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/977a3b90-7dc4-4e79-9117-6218dccc8f5e-sg-core-conf-yaml\") pod \"977a3b90-7dc4-4e79-9117-6218dccc8f5e\" (UID: \"977a3b90-7dc4-4e79-9117-6218dccc8f5e\") " Dec 03 11:58:57 crc kubenswrapper[5052]: I1203 11:58:57.472467 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/977a3b90-7dc4-4e79-9117-6218dccc8f5e-config-data\") pod \"977a3b90-7dc4-4e79-9117-6218dccc8f5e\" (UID: \"977a3b90-7dc4-4e79-9117-6218dccc8f5e\") " Dec 03 11:58:57 crc kubenswrapper[5052]: I1203 11:58:57.472532 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fmcv7\" (UniqueName: \"kubernetes.io/projected/977a3b90-7dc4-4e79-9117-6218dccc8f5e-kube-api-access-fmcv7\") pod \"977a3b90-7dc4-4e79-9117-6218dccc8f5e\" (UID: \"977a3b90-7dc4-4e79-9117-6218dccc8f5e\") " Dec 03 11:58:57 crc kubenswrapper[5052]: I1203 11:58:57.472794 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/977a3b90-7dc4-4e79-9117-6218dccc8f5e-run-httpd" (OuterVolumeSpecName: "run-httpd") pod "977a3b90-7dc4-4e79-9117-6218dccc8f5e" (UID: "977a3b90-7dc4-4e79-9117-6218dccc8f5e"). InnerVolumeSpecName "run-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:58:57 crc kubenswrapper[5052]: I1203 11:58:57.472620 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/977a3b90-7dc4-4e79-9117-6218dccc8f5e-scripts\") pod \"977a3b90-7dc4-4e79-9117-6218dccc8f5e\" (UID: \"977a3b90-7dc4-4e79-9117-6218dccc8f5e\") " Dec 03 11:58:57 crc kubenswrapper[5052]: I1203 11:58:57.473384 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/977a3b90-7dc4-4e79-9117-6218dccc8f5e-combined-ca-bundle\") pod \"977a3b90-7dc4-4e79-9117-6218dccc8f5e\" (UID: \"977a3b90-7dc4-4e79-9117-6218dccc8f5e\") " Dec 03 11:58:57 crc kubenswrapper[5052]: I1203 11:58:57.473407 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/977a3b90-7dc4-4e79-9117-6218dccc8f5e-log-httpd\") pod \"977a3b90-7dc4-4e79-9117-6218dccc8f5e\" (UID: \"977a3b90-7dc4-4e79-9117-6218dccc8f5e\") " Dec 03 11:58:57 crc kubenswrapper[5052]: I1203 11:58:57.474180 5052 reconciler_common.go:293] "Volume detached for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/977a3b90-7dc4-4e79-9117-6218dccc8f5e-run-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 11:58:57 crc kubenswrapper[5052]: I1203 11:58:57.474551 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/977a3b90-7dc4-4e79-9117-6218dccc8f5e-log-httpd" (OuterVolumeSpecName: "log-httpd") pod "977a3b90-7dc4-4e79-9117-6218dccc8f5e" (UID: "977a3b90-7dc4-4e79-9117-6218dccc8f5e"). InnerVolumeSpecName "log-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:58:57 crc kubenswrapper[5052]: I1203 11:58:57.478182 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/977a3b90-7dc4-4e79-9117-6218dccc8f5e-kube-api-access-fmcv7" (OuterVolumeSpecName: "kube-api-access-fmcv7") pod "977a3b90-7dc4-4e79-9117-6218dccc8f5e" (UID: "977a3b90-7dc4-4e79-9117-6218dccc8f5e"). InnerVolumeSpecName "kube-api-access-fmcv7". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:58:57 crc kubenswrapper[5052]: I1203 11:58:57.485329 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/977a3b90-7dc4-4e79-9117-6218dccc8f5e-scripts" (OuterVolumeSpecName: "scripts") pod "977a3b90-7dc4-4e79-9117-6218dccc8f5e" (UID: "977a3b90-7dc4-4e79-9117-6218dccc8f5e"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:58:57 crc kubenswrapper[5052]: I1203 11:58:57.507697 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/977a3b90-7dc4-4e79-9117-6218dccc8f5e-sg-core-conf-yaml" (OuterVolumeSpecName: "sg-core-conf-yaml") pod "977a3b90-7dc4-4e79-9117-6218dccc8f5e" (UID: "977a3b90-7dc4-4e79-9117-6218dccc8f5e"). InnerVolumeSpecName "sg-core-conf-yaml". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:58:57 crc kubenswrapper[5052]: I1203 11:58:57.519874 5052 generic.go:334] "Generic (PLEG): container finished" podID="0d55422b-84bf-4455-9112-94de20fb5b6d" containerID="f3557803e708885b7be19d0c0a7d6e82313f419fd68f6cf32092dc379a16b9af" exitCode=143 Dec 03 11:58:57 crc kubenswrapper[5052]: I1203 11:58:57.519937 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"0d55422b-84bf-4455-9112-94de20fb5b6d","Type":"ContainerDied","Data":"f3557803e708885b7be19d0c0a7d6e82313f419fd68f6cf32092dc379a16b9af"} Dec 03 11:58:57 crc kubenswrapper[5052]: I1203 11:58:57.522536 5052 generic.go:334] "Generic (PLEG): container finished" podID="977a3b90-7dc4-4e79-9117-6218dccc8f5e" containerID="950e6e25edb77f3fc2134f5dd19d895d398dd8572d4e0481d5c706ccc6c62d32" exitCode=0 Dec 03 11:58:57 crc kubenswrapper[5052]: I1203 11:58:57.522552 5052 generic.go:334] "Generic (PLEG): container finished" podID="977a3b90-7dc4-4e79-9117-6218dccc8f5e" containerID="087b4fae22b4d3f9c083241ea4ff6b9b34dd8e9a9eb57381f29a4eeac60c24fb" exitCode=2 Dec 03 11:58:57 crc kubenswrapper[5052]: I1203 11:58:57.522560 5052 generic.go:334] "Generic (PLEG): container finished" podID="977a3b90-7dc4-4e79-9117-6218dccc8f5e" containerID="1fd1a3840b23019c4246810ff2fe1c0d61d03f5a8efb0d7bffa142c536cd7db9" exitCode=0 Dec 03 11:58:57 crc kubenswrapper[5052]: I1203 11:58:57.522567 5052 generic.go:334] "Generic (PLEG): container finished" podID="977a3b90-7dc4-4e79-9117-6218dccc8f5e" containerID="add40057fd805113d3fbd97ea999b349355c3d1189310f1971b04d3b3e9da055" exitCode=0 Dec 03 11:58:57 crc kubenswrapper[5052]: I1203 11:58:57.522745 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openshift-marketplace/redhat-marketplace-nzgnj" podUID="8746e6ab-2863-45c2-b516-3b2bdfed2fb9" containerName="registry-server" containerID="cri-o://2de3285792422077503aa990bb4ca5abad10c82f4b20745ad58f5159cc75f2c1" gracePeriod=2 Dec 03 11:58:57 crc kubenswrapper[5052]: I1203 11:58:57.523030 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:58:57 crc kubenswrapper[5052]: I1203 11:58:57.523444 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"977a3b90-7dc4-4e79-9117-6218dccc8f5e","Type":"ContainerDied","Data":"950e6e25edb77f3fc2134f5dd19d895d398dd8572d4e0481d5c706ccc6c62d32"} Dec 03 11:58:57 crc kubenswrapper[5052]: I1203 11:58:57.523467 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"977a3b90-7dc4-4e79-9117-6218dccc8f5e","Type":"ContainerDied","Data":"087b4fae22b4d3f9c083241ea4ff6b9b34dd8e9a9eb57381f29a4eeac60c24fb"} Dec 03 11:58:57 crc kubenswrapper[5052]: I1203 11:58:57.523477 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"977a3b90-7dc4-4e79-9117-6218dccc8f5e","Type":"ContainerDied","Data":"1fd1a3840b23019c4246810ff2fe1c0d61d03f5a8efb0d7bffa142c536cd7db9"} Dec 03 11:58:57 crc kubenswrapper[5052]: I1203 11:58:57.523487 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"977a3b90-7dc4-4e79-9117-6218dccc8f5e","Type":"ContainerDied","Data":"add40057fd805113d3fbd97ea999b349355c3d1189310f1971b04d3b3e9da055"} Dec 03 11:58:57 crc kubenswrapper[5052]: I1203 11:58:57.523496 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"977a3b90-7dc4-4e79-9117-6218dccc8f5e","Type":"ContainerDied","Data":"89aecbd57886bf6651f4421f3e61f1ba3e578f9809d5ba0f946fa80cb9b6d27f"} Dec 03 11:58:57 crc kubenswrapper[5052]: I1203 11:58:57.523525 5052 scope.go:117] "RemoveContainer" containerID="950e6e25edb77f3fc2134f5dd19d895d398dd8572d4e0481d5c706ccc6c62d32" Dec 03 11:58:57 crc kubenswrapper[5052]: I1203 11:58:57.551669 5052 scope.go:117] "RemoveContainer" containerID="087b4fae22b4d3f9c083241ea4ff6b9b34dd8e9a9eb57381f29a4eeac60c24fb" Dec 03 11:58:57 crc kubenswrapper[5052]: I1203 11:58:57.576797 5052 reconciler_common.go:293] "Volume detached for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/977a3b90-7dc4-4e79-9117-6218dccc8f5e-sg-core-conf-yaml\") on node \"crc\" DevicePath \"\"" Dec 03 11:58:57 crc kubenswrapper[5052]: I1203 11:58:57.576830 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-fmcv7\" (UniqueName: \"kubernetes.io/projected/977a3b90-7dc4-4e79-9117-6218dccc8f5e-kube-api-access-fmcv7\") on node \"crc\" DevicePath \"\"" Dec 03 11:58:57 crc kubenswrapper[5052]: I1203 11:58:57.576842 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/977a3b90-7dc4-4e79-9117-6218dccc8f5e-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:58:57 crc kubenswrapper[5052]: I1203 11:58:57.576852 5052 reconciler_common.go:293] "Volume detached for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/977a3b90-7dc4-4e79-9117-6218dccc8f5e-log-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 11:58:57 crc kubenswrapper[5052]: I1203 11:58:57.586479 5052 scope.go:117] "RemoveContainer" containerID="1fd1a3840b23019c4246810ff2fe1c0d61d03f5a8efb0d7bffa142c536cd7db9" Dec 03 11:58:57 crc kubenswrapper[5052]: I1203 11:58:57.589188 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/977a3b90-7dc4-4e79-9117-6218dccc8f5e-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "977a3b90-7dc4-4e79-9117-6218dccc8f5e" (UID: "977a3b90-7dc4-4e79-9117-6218dccc8f5e"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:58:57 crc kubenswrapper[5052]: I1203 11:58:57.593733 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/977a3b90-7dc4-4e79-9117-6218dccc8f5e-config-data" (OuterVolumeSpecName: "config-data") pod "977a3b90-7dc4-4e79-9117-6218dccc8f5e" (UID: "977a3b90-7dc4-4e79-9117-6218dccc8f5e"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:58:57 crc kubenswrapper[5052]: I1203 11:58:57.678328 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/977a3b90-7dc4-4e79-9117-6218dccc8f5e-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:58:57 crc kubenswrapper[5052]: I1203 11:58:57.678357 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/977a3b90-7dc4-4e79-9117-6218dccc8f5e-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:58:57 crc kubenswrapper[5052]: E1203 11:58:57.857483 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of d7086739b41b7d360cbe4227c0a28ecffd42edb2ee30fb5fcc39d0c67f27ead1 is running failed: container process not found" containerID="d7086739b41b7d360cbe4227c0a28ecffd42edb2ee30fb5fcc39d0c67f27ead1" cmd=["/usr/bin/pgrep","-f","-r","DRST","watcher-decision-engine"] Dec 03 11:58:57 crc kubenswrapper[5052]: E1203 11:58:57.858626 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of d7086739b41b7d360cbe4227c0a28ecffd42edb2ee30fb5fcc39d0c67f27ead1 is running failed: container process not found" containerID="d7086739b41b7d360cbe4227c0a28ecffd42edb2ee30fb5fcc39d0c67f27ead1" cmd=["/usr/bin/pgrep","-f","-r","DRST","watcher-decision-engine"] Dec 03 11:58:57 crc kubenswrapper[5052]: E1203 11:58:57.858900 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of d7086739b41b7d360cbe4227c0a28ecffd42edb2ee30fb5fcc39d0c67f27ead1 is running failed: container process not found" containerID="d7086739b41b7d360cbe4227c0a28ecffd42edb2ee30fb5fcc39d0c67f27ead1" cmd=["/usr/bin/pgrep","-f","-r","DRST","watcher-decision-engine"] Dec 03 11:58:57 crc kubenswrapper[5052]: E1203 11:58:57.858926 5052 prober.go:104] "Probe errored" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of d7086739b41b7d360cbe4227c0a28ecffd42edb2ee30fb5fcc39d0c67f27ead1 is running failed: container process not found" probeType="Readiness" pod="openstack-kuttl-tests/watcher-decision-engine-0" podUID="5893db31-2978-45eb-8407-2e4d178954ce" containerName="watcher-decision-engine" Dec 03 11:58:57 crc kubenswrapper[5052]: I1203 11:58:57.870100 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:58:57 crc kubenswrapper[5052]: I1203 11:58:57.878746 5052 scope.go:117] "RemoveContainer" containerID="add40057fd805113d3fbd97ea999b349355c3d1189310f1971b04d3b3e9da055" Dec 03 11:58:57 crc kubenswrapper[5052]: I1203 11:58:57.893186 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:58:57 crc kubenswrapper[5052]: I1203 11:58:57.916385 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:58:57 crc kubenswrapper[5052]: E1203 11:58:57.916988 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="977a3b90-7dc4-4e79-9117-6218dccc8f5e" containerName="sg-core" Dec 03 11:58:57 crc kubenswrapper[5052]: I1203 11:58:57.917008 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="977a3b90-7dc4-4e79-9117-6218dccc8f5e" containerName="sg-core" Dec 03 11:58:57 crc kubenswrapper[5052]: E1203 11:58:57.917019 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="977a3b90-7dc4-4e79-9117-6218dccc8f5e" containerName="proxy-httpd" Dec 03 11:58:57 crc kubenswrapper[5052]: I1203 11:58:57.917027 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="977a3b90-7dc4-4e79-9117-6218dccc8f5e" containerName="proxy-httpd" Dec 03 11:58:57 crc kubenswrapper[5052]: E1203 11:58:57.917052 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="977a3b90-7dc4-4e79-9117-6218dccc8f5e" containerName="ceilometer-central-agent" Dec 03 11:58:57 crc kubenswrapper[5052]: I1203 11:58:57.917059 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="977a3b90-7dc4-4e79-9117-6218dccc8f5e" containerName="ceilometer-central-agent" Dec 03 11:58:57 crc kubenswrapper[5052]: E1203 11:58:57.917103 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="977a3b90-7dc4-4e79-9117-6218dccc8f5e" containerName="ceilometer-notification-agent" Dec 03 11:58:57 crc kubenswrapper[5052]: I1203 11:58:57.917112 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="977a3b90-7dc4-4e79-9117-6218dccc8f5e" containerName="ceilometer-notification-agent" Dec 03 11:58:57 crc kubenswrapper[5052]: I1203 11:58:57.917316 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="977a3b90-7dc4-4e79-9117-6218dccc8f5e" containerName="ceilometer-central-agent" Dec 03 11:58:57 crc kubenswrapper[5052]: I1203 11:58:57.917340 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="977a3b90-7dc4-4e79-9117-6218dccc8f5e" containerName="sg-core" Dec 03 11:58:57 crc kubenswrapper[5052]: I1203 11:58:57.917364 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="977a3b90-7dc4-4e79-9117-6218dccc8f5e" containerName="ceilometer-notification-agent" Dec 03 11:58:57 crc kubenswrapper[5052]: I1203 11:58:57.917382 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="977a3b90-7dc4-4e79-9117-6218dccc8f5e" containerName="proxy-httpd" Dec 03 11:58:57 crc kubenswrapper[5052]: I1203 11:58:57.920304 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:58:57 crc kubenswrapper[5052]: I1203 11:58:57.924239 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-config-data" Dec 03 11:58:57 crc kubenswrapper[5052]: I1203 11:58:57.924261 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-scripts" Dec 03 11:58:57 crc kubenswrapper[5052]: I1203 11:58:57.929872 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:58:57 crc kubenswrapper[5052]: I1203 11:58:57.931483 5052 scope.go:117] "RemoveContainer" containerID="950e6e25edb77f3fc2134f5dd19d895d398dd8572d4e0481d5c706ccc6c62d32" Dec 03 11:58:57 crc kubenswrapper[5052]: E1203 11:58:57.940654 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"950e6e25edb77f3fc2134f5dd19d895d398dd8572d4e0481d5c706ccc6c62d32\": container with ID starting with 950e6e25edb77f3fc2134f5dd19d895d398dd8572d4e0481d5c706ccc6c62d32 not found: ID does not exist" containerID="950e6e25edb77f3fc2134f5dd19d895d398dd8572d4e0481d5c706ccc6c62d32" Dec 03 11:58:57 crc kubenswrapper[5052]: I1203 11:58:57.940692 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"950e6e25edb77f3fc2134f5dd19d895d398dd8572d4e0481d5c706ccc6c62d32"} err="failed to get container status \"950e6e25edb77f3fc2134f5dd19d895d398dd8572d4e0481d5c706ccc6c62d32\": rpc error: code = NotFound desc = could not find container \"950e6e25edb77f3fc2134f5dd19d895d398dd8572d4e0481d5c706ccc6c62d32\": container with ID starting with 950e6e25edb77f3fc2134f5dd19d895d398dd8572d4e0481d5c706ccc6c62d32 not found: ID does not exist" Dec 03 11:58:57 crc kubenswrapper[5052]: I1203 11:58:57.940717 5052 scope.go:117] "RemoveContainer" containerID="087b4fae22b4d3f9c083241ea4ff6b9b34dd8e9a9eb57381f29a4eeac60c24fb" Dec 03 11:58:57 crc kubenswrapper[5052]: E1203 11:58:57.940990 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"087b4fae22b4d3f9c083241ea4ff6b9b34dd8e9a9eb57381f29a4eeac60c24fb\": container with ID starting with 087b4fae22b4d3f9c083241ea4ff6b9b34dd8e9a9eb57381f29a4eeac60c24fb not found: ID does not exist" containerID="087b4fae22b4d3f9c083241ea4ff6b9b34dd8e9a9eb57381f29a4eeac60c24fb" Dec 03 11:58:57 crc kubenswrapper[5052]: I1203 11:58:57.941009 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"087b4fae22b4d3f9c083241ea4ff6b9b34dd8e9a9eb57381f29a4eeac60c24fb"} err="failed to get container status \"087b4fae22b4d3f9c083241ea4ff6b9b34dd8e9a9eb57381f29a4eeac60c24fb\": rpc error: code = NotFound desc = could not find container \"087b4fae22b4d3f9c083241ea4ff6b9b34dd8e9a9eb57381f29a4eeac60c24fb\": container with ID starting with 087b4fae22b4d3f9c083241ea4ff6b9b34dd8e9a9eb57381f29a4eeac60c24fb not found: ID does not exist" Dec 03 11:58:57 crc kubenswrapper[5052]: I1203 11:58:57.941037 5052 scope.go:117] "RemoveContainer" containerID="1fd1a3840b23019c4246810ff2fe1c0d61d03f5a8efb0d7bffa142c536cd7db9" Dec 03 11:58:57 crc kubenswrapper[5052]: E1203 11:58:57.941394 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1fd1a3840b23019c4246810ff2fe1c0d61d03f5a8efb0d7bffa142c536cd7db9\": container with ID starting with 1fd1a3840b23019c4246810ff2fe1c0d61d03f5a8efb0d7bffa142c536cd7db9 not found: ID does not exist" containerID="1fd1a3840b23019c4246810ff2fe1c0d61d03f5a8efb0d7bffa142c536cd7db9" Dec 03 11:58:57 crc kubenswrapper[5052]: I1203 11:58:57.941427 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1fd1a3840b23019c4246810ff2fe1c0d61d03f5a8efb0d7bffa142c536cd7db9"} err="failed to get container status \"1fd1a3840b23019c4246810ff2fe1c0d61d03f5a8efb0d7bffa142c536cd7db9\": rpc error: code = NotFound desc = could not find container \"1fd1a3840b23019c4246810ff2fe1c0d61d03f5a8efb0d7bffa142c536cd7db9\": container with ID starting with 1fd1a3840b23019c4246810ff2fe1c0d61d03f5a8efb0d7bffa142c536cd7db9 not found: ID does not exist" Dec 03 11:58:57 crc kubenswrapper[5052]: I1203 11:58:57.941441 5052 scope.go:117] "RemoveContainer" containerID="add40057fd805113d3fbd97ea999b349355c3d1189310f1971b04d3b3e9da055" Dec 03 11:58:57 crc kubenswrapper[5052]: E1203 11:58:57.942317 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"add40057fd805113d3fbd97ea999b349355c3d1189310f1971b04d3b3e9da055\": container with ID starting with add40057fd805113d3fbd97ea999b349355c3d1189310f1971b04d3b3e9da055 not found: ID does not exist" containerID="add40057fd805113d3fbd97ea999b349355c3d1189310f1971b04d3b3e9da055" Dec 03 11:58:57 crc kubenswrapper[5052]: I1203 11:58:57.942361 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"add40057fd805113d3fbd97ea999b349355c3d1189310f1971b04d3b3e9da055"} err="failed to get container status \"add40057fd805113d3fbd97ea999b349355c3d1189310f1971b04d3b3e9da055\": rpc error: code = NotFound desc = could not find container \"add40057fd805113d3fbd97ea999b349355c3d1189310f1971b04d3b3e9da055\": container with ID starting with add40057fd805113d3fbd97ea999b349355c3d1189310f1971b04d3b3e9da055 not found: ID does not exist" Dec 03 11:58:57 crc kubenswrapper[5052]: I1203 11:58:57.942385 5052 scope.go:117] "RemoveContainer" containerID="950e6e25edb77f3fc2134f5dd19d895d398dd8572d4e0481d5c706ccc6c62d32" Dec 03 11:58:57 crc kubenswrapper[5052]: I1203 11:58:57.945960 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"950e6e25edb77f3fc2134f5dd19d895d398dd8572d4e0481d5c706ccc6c62d32"} err="failed to get container status \"950e6e25edb77f3fc2134f5dd19d895d398dd8572d4e0481d5c706ccc6c62d32\": rpc error: code = NotFound desc = could not find container \"950e6e25edb77f3fc2134f5dd19d895d398dd8572d4e0481d5c706ccc6c62d32\": container with ID starting with 950e6e25edb77f3fc2134f5dd19d895d398dd8572d4e0481d5c706ccc6c62d32 not found: ID does not exist" Dec 03 11:58:57 crc kubenswrapper[5052]: I1203 11:58:57.945985 5052 scope.go:117] "RemoveContainer" containerID="087b4fae22b4d3f9c083241ea4ff6b9b34dd8e9a9eb57381f29a4eeac60c24fb" Dec 03 11:58:57 crc kubenswrapper[5052]: I1203 11:58:57.950079 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"087b4fae22b4d3f9c083241ea4ff6b9b34dd8e9a9eb57381f29a4eeac60c24fb"} err="failed to get container status \"087b4fae22b4d3f9c083241ea4ff6b9b34dd8e9a9eb57381f29a4eeac60c24fb\": rpc error: code = NotFound desc = could not find container \"087b4fae22b4d3f9c083241ea4ff6b9b34dd8e9a9eb57381f29a4eeac60c24fb\": container with ID starting with 087b4fae22b4d3f9c083241ea4ff6b9b34dd8e9a9eb57381f29a4eeac60c24fb not found: ID does not exist" Dec 03 11:58:57 crc kubenswrapper[5052]: I1203 11:58:57.950103 5052 scope.go:117] "RemoveContainer" containerID="1fd1a3840b23019c4246810ff2fe1c0d61d03f5a8efb0d7bffa142c536cd7db9" Dec 03 11:58:57 crc kubenswrapper[5052]: I1203 11:58:57.950299 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1fd1a3840b23019c4246810ff2fe1c0d61d03f5a8efb0d7bffa142c536cd7db9"} err="failed to get container status \"1fd1a3840b23019c4246810ff2fe1c0d61d03f5a8efb0d7bffa142c536cd7db9\": rpc error: code = NotFound desc = could not find container \"1fd1a3840b23019c4246810ff2fe1c0d61d03f5a8efb0d7bffa142c536cd7db9\": container with ID starting with 1fd1a3840b23019c4246810ff2fe1c0d61d03f5a8efb0d7bffa142c536cd7db9 not found: ID does not exist" Dec 03 11:58:57 crc kubenswrapper[5052]: I1203 11:58:57.950316 5052 scope.go:117] "RemoveContainer" containerID="add40057fd805113d3fbd97ea999b349355c3d1189310f1971b04d3b3e9da055" Dec 03 11:58:57 crc kubenswrapper[5052]: I1203 11:58:57.950504 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"add40057fd805113d3fbd97ea999b349355c3d1189310f1971b04d3b3e9da055"} err="failed to get container status \"add40057fd805113d3fbd97ea999b349355c3d1189310f1971b04d3b3e9da055\": rpc error: code = NotFound desc = could not find container \"add40057fd805113d3fbd97ea999b349355c3d1189310f1971b04d3b3e9da055\": container with ID starting with add40057fd805113d3fbd97ea999b349355c3d1189310f1971b04d3b3e9da055 not found: ID does not exist" Dec 03 11:58:57 crc kubenswrapper[5052]: I1203 11:58:57.950531 5052 scope.go:117] "RemoveContainer" containerID="950e6e25edb77f3fc2134f5dd19d895d398dd8572d4e0481d5c706ccc6c62d32" Dec 03 11:58:57 crc kubenswrapper[5052]: I1203 11:58:57.950759 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"950e6e25edb77f3fc2134f5dd19d895d398dd8572d4e0481d5c706ccc6c62d32"} err="failed to get container status \"950e6e25edb77f3fc2134f5dd19d895d398dd8572d4e0481d5c706ccc6c62d32\": rpc error: code = NotFound desc = could not find container \"950e6e25edb77f3fc2134f5dd19d895d398dd8572d4e0481d5c706ccc6c62d32\": container with ID starting with 950e6e25edb77f3fc2134f5dd19d895d398dd8572d4e0481d5c706ccc6c62d32 not found: ID does not exist" Dec 03 11:58:57 crc kubenswrapper[5052]: I1203 11:58:57.950778 5052 scope.go:117] "RemoveContainer" containerID="087b4fae22b4d3f9c083241ea4ff6b9b34dd8e9a9eb57381f29a4eeac60c24fb" Dec 03 11:58:57 crc kubenswrapper[5052]: I1203 11:58:57.951333 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"087b4fae22b4d3f9c083241ea4ff6b9b34dd8e9a9eb57381f29a4eeac60c24fb"} err="failed to get container status \"087b4fae22b4d3f9c083241ea4ff6b9b34dd8e9a9eb57381f29a4eeac60c24fb\": rpc error: code = NotFound desc = could not find container \"087b4fae22b4d3f9c083241ea4ff6b9b34dd8e9a9eb57381f29a4eeac60c24fb\": container with ID starting with 087b4fae22b4d3f9c083241ea4ff6b9b34dd8e9a9eb57381f29a4eeac60c24fb not found: ID does not exist" Dec 03 11:58:57 crc kubenswrapper[5052]: I1203 11:58:57.951360 5052 scope.go:117] "RemoveContainer" containerID="1fd1a3840b23019c4246810ff2fe1c0d61d03f5a8efb0d7bffa142c536cd7db9" Dec 03 11:58:57 crc kubenswrapper[5052]: I1203 11:58:57.952445 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1fd1a3840b23019c4246810ff2fe1c0d61d03f5a8efb0d7bffa142c536cd7db9"} err="failed to get container status \"1fd1a3840b23019c4246810ff2fe1c0d61d03f5a8efb0d7bffa142c536cd7db9\": rpc error: code = NotFound desc = could not find container \"1fd1a3840b23019c4246810ff2fe1c0d61d03f5a8efb0d7bffa142c536cd7db9\": container with ID starting with 1fd1a3840b23019c4246810ff2fe1c0d61d03f5a8efb0d7bffa142c536cd7db9 not found: ID does not exist" Dec 03 11:58:57 crc kubenswrapper[5052]: I1203 11:58:57.952466 5052 scope.go:117] "RemoveContainer" containerID="add40057fd805113d3fbd97ea999b349355c3d1189310f1971b04d3b3e9da055" Dec 03 11:58:57 crc kubenswrapper[5052]: I1203 11:58:57.954640 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"add40057fd805113d3fbd97ea999b349355c3d1189310f1971b04d3b3e9da055"} err="failed to get container status \"add40057fd805113d3fbd97ea999b349355c3d1189310f1971b04d3b3e9da055\": rpc error: code = NotFound desc = could not find container \"add40057fd805113d3fbd97ea999b349355c3d1189310f1971b04d3b3e9da055\": container with ID starting with add40057fd805113d3fbd97ea999b349355c3d1189310f1971b04d3b3e9da055 not found: ID does not exist" Dec 03 11:58:57 crc kubenswrapper[5052]: I1203 11:58:57.954662 5052 scope.go:117] "RemoveContainer" containerID="950e6e25edb77f3fc2134f5dd19d895d398dd8572d4e0481d5c706ccc6c62d32" Dec 03 11:58:57 crc kubenswrapper[5052]: I1203 11:58:57.958482 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"950e6e25edb77f3fc2134f5dd19d895d398dd8572d4e0481d5c706ccc6c62d32"} err="failed to get container status \"950e6e25edb77f3fc2134f5dd19d895d398dd8572d4e0481d5c706ccc6c62d32\": rpc error: code = NotFound desc = could not find container \"950e6e25edb77f3fc2134f5dd19d895d398dd8572d4e0481d5c706ccc6c62d32\": container with ID starting with 950e6e25edb77f3fc2134f5dd19d895d398dd8572d4e0481d5c706ccc6c62d32 not found: ID does not exist" Dec 03 11:58:57 crc kubenswrapper[5052]: I1203 11:58:57.958536 5052 scope.go:117] "RemoveContainer" containerID="087b4fae22b4d3f9c083241ea4ff6b9b34dd8e9a9eb57381f29a4eeac60c24fb" Dec 03 11:58:57 crc kubenswrapper[5052]: I1203 11:58:57.959080 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"087b4fae22b4d3f9c083241ea4ff6b9b34dd8e9a9eb57381f29a4eeac60c24fb"} err="failed to get container status \"087b4fae22b4d3f9c083241ea4ff6b9b34dd8e9a9eb57381f29a4eeac60c24fb\": rpc error: code = NotFound desc = could not find container \"087b4fae22b4d3f9c083241ea4ff6b9b34dd8e9a9eb57381f29a4eeac60c24fb\": container with ID starting with 087b4fae22b4d3f9c083241ea4ff6b9b34dd8e9a9eb57381f29a4eeac60c24fb not found: ID does not exist" Dec 03 11:58:57 crc kubenswrapper[5052]: I1203 11:58:57.959103 5052 scope.go:117] "RemoveContainer" containerID="1fd1a3840b23019c4246810ff2fe1c0d61d03f5a8efb0d7bffa142c536cd7db9" Dec 03 11:58:57 crc kubenswrapper[5052]: I1203 11:58:57.960389 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1fd1a3840b23019c4246810ff2fe1c0d61d03f5a8efb0d7bffa142c536cd7db9"} err="failed to get container status \"1fd1a3840b23019c4246810ff2fe1c0d61d03f5a8efb0d7bffa142c536cd7db9\": rpc error: code = NotFound desc = could not find container \"1fd1a3840b23019c4246810ff2fe1c0d61d03f5a8efb0d7bffa142c536cd7db9\": container with ID starting with 1fd1a3840b23019c4246810ff2fe1c0d61d03f5a8efb0d7bffa142c536cd7db9 not found: ID does not exist" Dec 03 11:58:57 crc kubenswrapper[5052]: I1203 11:58:57.960414 5052 scope.go:117] "RemoveContainer" containerID="add40057fd805113d3fbd97ea999b349355c3d1189310f1971b04d3b3e9da055" Dec 03 11:58:57 crc kubenswrapper[5052]: I1203 11:58:57.960800 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"add40057fd805113d3fbd97ea999b349355c3d1189310f1971b04d3b3e9da055"} err="failed to get container status \"add40057fd805113d3fbd97ea999b349355c3d1189310f1971b04d3b3e9da055\": rpc error: code = NotFound desc = could not find container \"add40057fd805113d3fbd97ea999b349355c3d1189310f1971b04d3b3e9da055\": container with ID starting with add40057fd805113d3fbd97ea999b349355c3d1189310f1971b04d3b3e9da055 not found: ID does not exist" Dec 03 11:58:58 crc kubenswrapper[5052]: I1203 11:58:58.054399 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:58:58 crc kubenswrapper[5052]: E1203 11:58:58.055848 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[combined-ca-bundle config-data kube-api-access-469bh log-httpd run-httpd scripts sg-core-conf-yaml], unattached volumes=[], failed to process volumes=[]: context canceled" pod="openstack-kuttl-tests/ceilometer-0" podUID="97230225-f283-4927-a24c-78fece560796" Dec 03 11:58:58 crc kubenswrapper[5052]: I1203 11:58:58.085967 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/97230225-f283-4927-a24c-78fece560796-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"97230225-f283-4927-a24c-78fece560796\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:58:58 crc kubenswrapper[5052]: I1203 11:58:58.086134 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/97230225-f283-4927-a24c-78fece560796-run-httpd\") pod \"ceilometer-0\" (UID: \"97230225-f283-4927-a24c-78fece560796\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:58:58 crc kubenswrapper[5052]: I1203 11:58:58.086195 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/97230225-f283-4927-a24c-78fece560796-config-data\") pod \"ceilometer-0\" (UID: \"97230225-f283-4927-a24c-78fece560796\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:58:58 crc kubenswrapper[5052]: I1203 11:58:58.086227 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/97230225-f283-4927-a24c-78fece560796-log-httpd\") pod \"ceilometer-0\" (UID: \"97230225-f283-4927-a24c-78fece560796\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:58:58 crc kubenswrapper[5052]: I1203 11:58:58.086263 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/97230225-f283-4927-a24c-78fece560796-scripts\") pod \"ceilometer-0\" (UID: \"97230225-f283-4927-a24c-78fece560796\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:58:58 crc kubenswrapper[5052]: I1203 11:58:58.086329 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-469bh\" (UniqueName: \"kubernetes.io/projected/97230225-f283-4927-a24c-78fece560796-kube-api-access-469bh\") pod \"ceilometer-0\" (UID: \"97230225-f283-4927-a24c-78fece560796\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:58:58 crc kubenswrapper[5052]: I1203 11:58:58.086353 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/97230225-f283-4927-a24c-78fece560796-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"97230225-f283-4927-a24c-78fece560796\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:58:58 crc kubenswrapper[5052]: I1203 11:58:58.122284 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/redhat-marketplace-nzgnj" Dec 03 11:58:58 crc kubenswrapper[5052]: I1203 11:58:58.188023 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-469bh\" (UniqueName: \"kubernetes.io/projected/97230225-f283-4927-a24c-78fece560796-kube-api-access-469bh\") pod \"ceilometer-0\" (UID: \"97230225-f283-4927-a24c-78fece560796\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:58:58 crc kubenswrapper[5052]: I1203 11:58:58.188380 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/97230225-f283-4927-a24c-78fece560796-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"97230225-f283-4927-a24c-78fece560796\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:58:58 crc kubenswrapper[5052]: I1203 11:58:58.188409 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/97230225-f283-4927-a24c-78fece560796-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"97230225-f283-4927-a24c-78fece560796\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:58:58 crc kubenswrapper[5052]: I1203 11:58:58.188556 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/97230225-f283-4927-a24c-78fece560796-run-httpd\") pod \"ceilometer-0\" (UID: \"97230225-f283-4927-a24c-78fece560796\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:58:58 crc kubenswrapper[5052]: I1203 11:58:58.188614 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/97230225-f283-4927-a24c-78fece560796-config-data\") pod \"ceilometer-0\" (UID: \"97230225-f283-4927-a24c-78fece560796\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:58:58 crc kubenswrapper[5052]: I1203 11:58:58.188644 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/97230225-f283-4927-a24c-78fece560796-log-httpd\") pod \"ceilometer-0\" (UID: \"97230225-f283-4927-a24c-78fece560796\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:58:58 crc kubenswrapper[5052]: I1203 11:58:58.188678 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/97230225-f283-4927-a24c-78fece560796-scripts\") pod \"ceilometer-0\" (UID: \"97230225-f283-4927-a24c-78fece560796\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:58:58 crc kubenswrapper[5052]: I1203 11:58:58.194039 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/97230225-f283-4927-a24c-78fece560796-log-httpd\") pod \"ceilometer-0\" (UID: \"97230225-f283-4927-a24c-78fece560796\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:58:58 crc kubenswrapper[5052]: I1203 11:58:58.194157 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/97230225-f283-4927-a24c-78fece560796-run-httpd\") pod \"ceilometer-0\" (UID: \"97230225-f283-4927-a24c-78fece560796\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:58:58 crc kubenswrapper[5052]: I1203 11:58:58.196164 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/97230225-f283-4927-a24c-78fece560796-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"97230225-f283-4927-a24c-78fece560796\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:58:58 crc kubenswrapper[5052]: I1203 11:58:58.197395 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/97230225-f283-4927-a24c-78fece560796-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"97230225-f283-4927-a24c-78fece560796\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:58:58 crc kubenswrapper[5052]: I1203 11:58:58.198176 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/97230225-f283-4927-a24c-78fece560796-config-data\") pod \"ceilometer-0\" (UID: \"97230225-f283-4927-a24c-78fece560796\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:58:58 crc kubenswrapper[5052]: I1203 11:58:58.217642 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-469bh\" (UniqueName: \"kubernetes.io/projected/97230225-f283-4927-a24c-78fece560796-kube-api-access-469bh\") pod \"ceilometer-0\" (UID: \"97230225-f283-4927-a24c-78fece560796\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:58:58 crc kubenswrapper[5052]: I1203 11:58:58.224443 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/97230225-f283-4927-a24c-78fece560796-scripts\") pod \"ceilometer-0\" (UID: \"97230225-f283-4927-a24c-78fece560796\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:58:58 crc kubenswrapper[5052]: I1203 11:58:58.295520 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/8746e6ab-2863-45c2-b516-3b2bdfed2fb9-utilities\") pod \"8746e6ab-2863-45c2-b516-3b2bdfed2fb9\" (UID: \"8746e6ab-2863-45c2-b516-3b2bdfed2fb9\") " Dec 03 11:58:58 crc kubenswrapper[5052]: I1203 11:58:58.295605 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jh2nt\" (UniqueName: \"kubernetes.io/projected/8746e6ab-2863-45c2-b516-3b2bdfed2fb9-kube-api-access-jh2nt\") pod \"8746e6ab-2863-45c2-b516-3b2bdfed2fb9\" (UID: \"8746e6ab-2863-45c2-b516-3b2bdfed2fb9\") " Dec 03 11:58:58 crc kubenswrapper[5052]: I1203 11:58:58.295691 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/8746e6ab-2863-45c2-b516-3b2bdfed2fb9-catalog-content\") pod \"8746e6ab-2863-45c2-b516-3b2bdfed2fb9\" (UID: \"8746e6ab-2863-45c2-b516-3b2bdfed2fb9\") " Dec 03 11:58:58 crc kubenswrapper[5052]: I1203 11:58:58.296659 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8746e6ab-2863-45c2-b516-3b2bdfed2fb9-utilities" (OuterVolumeSpecName: "utilities") pod "8746e6ab-2863-45c2-b516-3b2bdfed2fb9" (UID: "8746e6ab-2863-45c2-b516-3b2bdfed2fb9"). InnerVolumeSpecName "utilities". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:58:58 crc kubenswrapper[5052]: I1203 11:58:58.307842 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8746e6ab-2863-45c2-b516-3b2bdfed2fb9-kube-api-access-jh2nt" (OuterVolumeSpecName: "kube-api-access-jh2nt") pod "8746e6ab-2863-45c2-b516-3b2bdfed2fb9" (UID: "8746e6ab-2863-45c2-b516-3b2bdfed2fb9"). InnerVolumeSpecName "kube-api-access-jh2nt". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:58:58 crc kubenswrapper[5052]: I1203 11:58:58.331584 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8746e6ab-2863-45c2-b516-3b2bdfed2fb9-catalog-content" (OuterVolumeSpecName: "catalog-content") pod "8746e6ab-2863-45c2-b516-3b2bdfed2fb9" (UID: "8746e6ab-2863-45c2-b516-3b2bdfed2fb9"). InnerVolumeSpecName "catalog-content". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:58:58 crc kubenswrapper[5052]: I1203 11:58:58.397550 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-jh2nt\" (UniqueName: \"kubernetes.io/projected/8746e6ab-2863-45c2-b516-3b2bdfed2fb9-kube-api-access-jh2nt\") on node \"crc\" DevicePath \"\"" Dec 03 11:58:58 crc kubenswrapper[5052]: I1203 11:58:58.397583 5052 reconciler_common.go:293] "Volume detached for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/8746e6ab-2863-45c2-b516-3b2bdfed2fb9-catalog-content\") on node \"crc\" DevicePath \"\"" Dec 03 11:58:58 crc kubenswrapper[5052]: I1203 11:58:58.397592 5052 reconciler_common.go:293] "Volume detached for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/8746e6ab-2863-45c2-b516-3b2bdfed2fb9-utilities\") on node \"crc\" DevicePath \"\"" Dec 03 11:58:58 crc kubenswrapper[5052]: I1203 11:58:58.424073 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/watcher-decision-engine-0" Dec 03 11:58:58 crc kubenswrapper[5052]: E1203 11:58:58.516027 5052 cadvisor_stats_provider.go:516] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/system.slice/rpm-ostreed.service\": RecentStats: unable to find data in memory cache]" Dec 03 11:58:58 crc kubenswrapper[5052]: I1203 11:58:58.536675 5052 generic.go:334] "Generic (PLEG): container finished" podID="8746e6ab-2863-45c2-b516-3b2bdfed2fb9" containerID="2de3285792422077503aa990bb4ca5abad10c82f4b20745ad58f5159cc75f2c1" exitCode=0 Dec 03 11:58:58 crc kubenswrapper[5052]: I1203 11:58:58.536748 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/redhat-marketplace-nzgnj" Dec 03 11:58:58 crc kubenswrapper[5052]: I1203 11:58:58.536736 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-marketplace-nzgnj" event={"ID":"8746e6ab-2863-45c2-b516-3b2bdfed2fb9","Type":"ContainerDied","Data":"2de3285792422077503aa990bb4ca5abad10c82f4b20745ad58f5159cc75f2c1"} Dec 03 11:58:58 crc kubenswrapper[5052]: I1203 11:58:58.537158 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-marketplace-nzgnj" event={"ID":"8746e6ab-2863-45c2-b516-3b2bdfed2fb9","Type":"ContainerDied","Data":"d4d256451268e7b6c5904e85ccdc211c5cdbf1217c77e5742ab007a1e5c01ec7"} Dec 03 11:58:58 crc kubenswrapper[5052]: I1203 11:58:58.537181 5052 scope.go:117] "RemoveContainer" containerID="2de3285792422077503aa990bb4ca5abad10c82f4b20745ad58f5159cc75f2c1" Dec 03 11:58:58 crc kubenswrapper[5052]: I1203 11:58:58.541294 5052 generic.go:334] "Generic (PLEG): container finished" podID="5893db31-2978-45eb-8407-2e4d178954ce" containerID="d7086739b41b7d360cbe4227c0a28ecffd42edb2ee30fb5fcc39d0c67f27ead1" exitCode=0 Dec 03 11:58:58 crc kubenswrapper[5052]: I1203 11:58:58.541375 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:58:58 crc kubenswrapper[5052]: I1203 11:58:58.542055 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/watcher-decision-engine-0" Dec 03 11:58:58 crc kubenswrapper[5052]: I1203 11:58:58.542497 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/watcher-decision-engine-0" event={"ID":"5893db31-2978-45eb-8407-2e4d178954ce","Type":"ContainerDied","Data":"d7086739b41b7d360cbe4227c0a28ecffd42edb2ee30fb5fcc39d0c67f27ead1"} Dec 03 11:58:58 crc kubenswrapper[5052]: I1203 11:58:58.542545 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/watcher-decision-engine-0" event={"ID":"5893db31-2978-45eb-8407-2e4d178954ce","Type":"ContainerDied","Data":"451ff2577699cff2494d08d669aa519a79d76d38c7b97681ac3e06727f31805b"} Dec 03 11:58:58 crc kubenswrapper[5052]: I1203 11:58:58.551296 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:58:58 crc kubenswrapper[5052]: I1203 11:58:58.561721 5052 scope.go:117] "RemoveContainer" containerID="d52b345183e10e412ba8c94c5e5e15175c4339e5af63d381f6b95e790c2b3e85" Dec 03 11:58:58 crc kubenswrapper[5052]: I1203 11:58:58.581037 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openshift-marketplace/redhat-marketplace-nzgnj"] Dec 03 11:58:58 crc kubenswrapper[5052]: I1203 11:58:58.589299 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openshift-marketplace/redhat-marketplace-nzgnj"] Dec 03 11:58:58 crc kubenswrapper[5052]: I1203 11:58:58.599921 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"custom-prometheus-ca\" (UniqueName: \"kubernetes.io/secret/5893db31-2978-45eb-8407-2e4d178954ce-custom-prometheus-ca\") pod \"5893db31-2978-45eb-8407-2e4d178954ce\" (UID: \"5893db31-2978-45eb-8407-2e4d178954ce\") " Dec 03 11:58:58 crc kubenswrapper[5052]: I1203 11:58:58.599975 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5893db31-2978-45eb-8407-2e4d178954ce-combined-ca-bundle\") pod \"5893db31-2978-45eb-8407-2e4d178954ce\" (UID: \"5893db31-2978-45eb-8407-2e4d178954ce\") " Dec 03 11:58:58 crc kubenswrapper[5052]: I1203 11:58:58.600037 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/5893db31-2978-45eb-8407-2e4d178954ce-config-data\") pod \"5893db31-2978-45eb-8407-2e4d178954ce\" (UID: \"5893db31-2978-45eb-8407-2e4d178954ce\") " Dec 03 11:58:58 crc kubenswrapper[5052]: I1203 11:58:58.600177 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/5893db31-2978-45eb-8407-2e4d178954ce-logs\") pod \"5893db31-2978-45eb-8407-2e4d178954ce\" (UID: \"5893db31-2978-45eb-8407-2e4d178954ce\") " Dec 03 11:58:58 crc kubenswrapper[5052]: I1203 11:58:58.600251 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qcmpr\" (UniqueName: \"kubernetes.io/projected/5893db31-2978-45eb-8407-2e4d178954ce-kube-api-access-qcmpr\") pod \"5893db31-2978-45eb-8407-2e4d178954ce\" (UID: \"5893db31-2978-45eb-8407-2e4d178954ce\") " Dec 03 11:58:58 crc kubenswrapper[5052]: I1203 11:58:58.600505 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5893db31-2978-45eb-8407-2e4d178954ce-logs" (OuterVolumeSpecName: "logs") pod "5893db31-2978-45eb-8407-2e4d178954ce" (UID: "5893db31-2978-45eb-8407-2e4d178954ce"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:58:58 crc kubenswrapper[5052]: I1203 11:58:58.600913 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/5893db31-2978-45eb-8407-2e4d178954ce-logs\") on node \"crc\" DevicePath \"\"" Dec 03 11:58:58 crc kubenswrapper[5052]: I1203 11:58:58.604001 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5893db31-2978-45eb-8407-2e4d178954ce-kube-api-access-qcmpr" (OuterVolumeSpecName: "kube-api-access-qcmpr") pod "5893db31-2978-45eb-8407-2e4d178954ce" (UID: "5893db31-2978-45eb-8407-2e4d178954ce"). InnerVolumeSpecName "kube-api-access-qcmpr". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:58:58 crc kubenswrapper[5052]: I1203 11:58:58.610444 5052 scope.go:117] "RemoveContainer" containerID="46ea7b758ebfe4d32a01510cd603559d92b9393d6d064ce81e1362b1d7108e83" Dec 03 11:58:58 crc kubenswrapper[5052]: I1203 11:58:58.640799 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5893db31-2978-45eb-8407-2e4d178954ce-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "5893db31-2978-45eb-8407-2e4d178954ce" (UID: "5893db31-2978-45eb-8407-2e4d178954ce"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:58:58 crc kubenswrapper[5052]: I1203 11:58:58.651626 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5893db31-2978-45eb-8407-2e4d178954ce-custom-prometheus-ca" (OuterVolumeSpecName: "custom-prometheus-ca") pod "5893db31-2978-45eb-8407-2e4d178954ce" (UID: "5893db31-2978-45eb-8407-2e4d178954ce"). InnerVolumeSpecName "custom-prometheus-ca". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:58:58 crc kubenswrapper[5052]: I1203 11:58:58.662996 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5893db31-2978-45eb-8407-2e4d178954ce-config-data" (OuterVolumeSpecName: "config-data") pod "5893db31-2978-45eb-8407-2e4d178954ce" (UID: "5893db31-2978-45eb-8407-2e4d178954ce"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:58:58 crc kubenswrapper[5052]: I1203 11:58:58.701569 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-469bh\" (UniqueName: \"kubernetes.io/projected/97230225-f283-4927-a24c-78fece560796-kube-api-access-469bh\") pod \"97230225-f283-4927-a24c-78fece560796\" (UID: \"97230225-f283-4927-a24c-78fece560796\") " Dec 03 11:58:58 crc kubenswrapper[5052]: I1203 11:58:58.701671 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/97230225-f283-4927-a24c-78fece560796-config-data\") pod \"97230225-f283-4927-a24c-78fece560796\" (UID: \"97230225-f283-4927-a24c-78fece560796\") " Dec 03 11:58:58 crc kubenswrapper[5052]: I1203 11:58:58.701701 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/97230225-f283-4927-a24c-78fece560796-run-httpd\") pod \"97230225-f283-4927-a24c-78fece560796\" (UID: \"97230225-f283-4927-a24c-78fece560796\") " Dec 03 11:58:58 crc kubenswrapper[5052]: I1203 11:58:58.701740 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/97230225-f283-4927-a24c-78fece560796-log-httpd\") pod \"97230225-f283-4927-a24c-78fece560796\" (UID: \"97230225-f283-4927-a24c-78fece560796\") " Dec 03 11:58:58 crc kubenswrapper[5052]: I1203 11:58:58.701822 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/97230225-f283-4927-a24c-78fece560796-scripts\") pod \"97230225-f283-4927-a24c-78fece560796\" (UID: \"97230225-f283-4927-a24c-78fece560796\") " Dec 03 11:58:58 crc kubenswrapper[5052]: I1203 11:58:58.701936 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/97230225-f283-4927-a24c-78fece560796-sg-core-conf-yaml\") pod \"97230225-f283-4927-a24c-78fece560796\" (UID: \"97230225-f283-4927-a24c-78fece560796\") " Dec 03 11:58:58 crc kubenswrapper[5052]: I1203 11:58:58.701966 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/97230225-f283-4927-a24c-78fece560796-combined-ca-bundle\") pod \"97230225-f283-4927-a24c-78fece560796\" (UID: \"97230225-f283-4927-a24c-78fece560796\") " Dec 03 11:58:58 crc kubenswrapper[5052]: I1203 11:58:58.702255 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/97230225-f283-4927-a24c-78fece560796-log-httpd" (OuterVolumeSpecName: "log-httpd") pod "97230225-f283-4927-a24c-78fece560796" (UID: "97230225-f283-4927-a24c-78fece560796"). InnerVolumeSpecName "log-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:58:58 crc kubenswrapper[5052]: I1203 11:58:58.702277 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/97230225-f283-4927-a24c-78fece560796-run-httpd" (OuterVolumeSpecName: "run-httpd") pod "97230225-f283-4927-a24c-78fece560796" (UID: "97230225-f283-4927-a24c-78fece560796"). InnerVolumeSpecName "run-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:58:58 crc kubenswrapper[5052]: I1203 11:58:58.705766 5052 reconciler_common.go:293] "Volume detached for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/97230225-f283-4927-a24c-78fece560796-log-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 11:58:58 crc kubenswrapper[5052]: I1203 11:58:58.705815 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-qcmpr\" (UniqueName: \"kubernetes.io/projected/5893db31-2978-45eb-8407-2e4d178954ce-kube-api-access-qcmpr\") on node \"crc\" DevicePath \"\"" Dec 03 11:58:58 crc kubenswrapper[5052]: I1203 11:58:58.705842 5052 reconciler_common.go:293] "Volume detached for volume \"custom-prometheus-ca\" (UniqueName: \"kubernetes.io/secret/5893db31-2978-45eb-8407-2e4d178954ce-custom-prometheus-ca\") on node \"crc\" DevicePath \"\"" Dec 03 11:58:58 crc kubenswrapper[5052]: I1203 11:58:58.705853 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5893db31-2978-45eb-8407-2e4d178954ce-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:58:58 crc kubenswrapper[5052]: I1203 11:58:58.705864 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/5893db31-2978-45eb-8407-2e4d178954ce-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:58:58 crc kubenswrapper[5052]: I1203 11:58:58.708627 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/97230225-f283-4927-a24c-78fece560796-scripts" (OuterVolumeSpecName: "scripts") pod "97230225-f283-4927-a24c-78fece560796" (UID: "97230225-f283-4927-a24c-78fece560796"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:58:58 crc kubenswrapper[5052]: I1203 11:58:58.713600 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/97230225-f283-4927-a24c-78fece560796-sg-core-conf-yaml" (OuterVolumeSpecName: "sg-core-conf-yaml") pod "97230225-f283-4927-a24c-78fece560796" (UID: "97230225-f283-4927-a24c-78fece560796"). InnerVolumeSpecName "sg-core-conf-yaml". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:58:58 crc kubenswrapper[5052]: I1203 11:58:58.714271 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/97230225-f283-4927-a24c-78fece560796-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "97230225-f283-4927-a24c-78fece560796" (UID: "97230225-f283-4927-a24c-78fece560796"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:58:58 crc kubenswrapper[5052]: I1203 11:58:58.717107 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/97230225-f283-4927-a24c-78fece560796-config-data" (OuterVolumeSpecName: "config-data") pod "97230225-f283-4927-a24c-78fece560796" (UID: "97230225-f283-4927-a24c-78fece560796"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:58:58 crc kubenswrapper[5052]: I1203 11:58:58.720537 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/97230225-f283-4927-a24c-78fece560796-kube-api-access-469bh" (OuterVolumeSpecName: "kube-api-access-469bh") pod "97230225-f283-4927-a24c-78fece560796" (UID: "97230225-f283-4927-a24c-78fece560796"). InnerVolumeSpecName "kube-api-access-469bh". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:58:58 crc kubenswrapper[5052]: I1203 11:58:58.797782 5052 scope.go:117] "RemoveContainer" containerID="2de3285792422077503aa990bb4ca5abad10c82f4b20745ad58f5159cc75f2c1" Dec 03 11:58:58 crc kubenswrapper[5052]: E1203 11:58:58.798275 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2de3285792422077503aa990bb4ca5abad10c82f4b20745ad58f5159cc75f2c1\": container with ID starting with 2de3285792422077503aa990bb4ca5abad10c82f4b20745ad58f5159cc75f2c1 not found: ID does not exist" containerID="2de3285792422077503aa990bb4ca5abad10c82f4b20745ad58f5159cc75f2c1" Dec 03 11:58:58 crc kubenswrapper[5052]: I1203 11:58:58.798331 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2de3285792422077503aa990bb4ca5abad10c82f4b20745ad58f5159cc75f2c1"} err="failed to get container status \"2de3285792422077503aa990bb4ca5abad10c82f4b20745ad58f5159cc75f2c1\": rpc error: code = NotFound desc = could not find container \"2de3285792422077503aa990bb4ca5abad10c82f4b20745ad58f5159cc75f2c1\": container with ID starting with 2de3285792422077503aa990bb4ca5abad10c82f4b20745ad58f5159cc75f2c1 not found: ID does not exist" Dec 03 11:58:58 crc kubenswrapper[5052]: I1203 11:58:58.798361 5052 scope.go:117] "RemoveContainer" containerID="d52b345183e10e412ba8c94c5e5e15175c4339e5af63d381f6b95e790c2b3e85" Dec 03 11:58:58 crc kubenswrapper[5052]: E1203 11:58:58.798783 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d52b345183e10e412ba8c94c5e5e15175c4339e5af63d381f6b95e790c2b3e85\": container with ID starting with d52b345183e10e412ba8c94c5e5e15175c4339e5af63d381f6b95e790c2b3e85 not found: ID does not exist" containerID="d52b345183e10e412ba8c94c5e5e15175c4339e5af63d381f6b95e790c2b3e85" Dec 03 11:58:58 crc kubenswrapper[5052]: I1203 11:58:58.798810 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d52b345183e10e412ba8c94c5e5e15175c4339e5af63d381f6b95e790c2b3e85"} err="failed to get container status \"d52b345183e10e412ba8c94c5e5e15175c4339e5af63d381f6b95e790c2b3e85\": rpc error: code = NotFound desc = could not find container \"d52b345183e10e412ba8c94c5e5e15175c4339e5af63d381f6b95e790c2b3e85\": container with ID starting with d52b345183e10e412ba8c94c5e5e15175c4339e5af63d381f6b95e790c2b3e85 not found: ID does not exist" Dec 03 11:58:58 crc kubenswrapper[5052]: I1203 11:58:58.798825 5052 scope.go:117] "RemoveContainer" containerID="46ea7b758ebfe4d32a01510cd603559d92b9393d6d064ce81e1362b1d7108e83" Dec 03 11:58:58 crc kubenswrapper[5052]: E1203 11:58:58.799111 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"46ea7b758ebfe4d32a01510cd603559d92b9393d6d064ce81e1362b1d7108e83\": container with ID starting with 46ea7b758ebfe4d32a01510cd603559d92b9393d6d064ce81e1362b1d7108e83 not found: ID does not exist" containerID="46ea7b758ebfe4d32a01510cd603559d92b9393d6d064ce81e1362b1d7108e83" Dec 03 11:58:58 crc kubenswrapper[5052]: I1203 11:58:58.799150 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"46ea7b758ebfe4d32a01510cd603559d92b9393d6d064ce81e1362b1d7108e83"} err="failed to get container status \"46ea7b758ebfe4d32a01510cd603559d92b9393d6d064ce81e1362b1d7108e83\": rpc error: code = NotFound desc = could not find container \"46ea7b758ebfe4d32a01510cd603559d92b9393d6d064ce81e1362b1d7108e83\": container with ID starting with 46ea7b758ebfe4d32a01510cd603559d92b9393d6d064ce81e1362b1d7108e83 not found: ID does not exist" Dec 03 11:58:58 crc kubenswrapper[5052]: I1203 11:58:58.799169 5052 scope.go:117] "RemoveContainer" containerID="d7086739b41b7d360cbe4227c0a28ecffd42edb2ee30fb5fcc39d0c67f27ead1" Dec 03 11:58:58 crc kubenswrapper[5052]: I1203 11:58:58.807877 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/97230225-f283-4927-a24c-78fece560796-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:58:58 crc kubenswrapper[5052]: I1203 11:58:58.807903 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-469bh\" (UniqueName: \"kubernetes.io/projected/97230225-f283-4927-a24c-78fece560796-kube-api-access-469bh\") on node \"crc\" DevicePath \"\"" Dec 03 11:58:58 crc kubenswrapper[5052]: I1203 11:58:58.807912 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/97230225-f283-4927-a24c-78fece560796-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:58:58 crc kubenswrapper[5052]: I1203 11:58:58.807923 5052 reconciler_common.go:293] "Volume detached for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/97230225-f283-4927-a24c-78fece560796-run-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 11:58:58 crc kubenswrapper[5052]: I1203 11:58:58.807931 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/97230225-f283-4927-a24c-78fece560796-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:58:58 crc kubenswrapper[5052]: I1203 11:58:58.807940 5052 reconciler_common.go:293] "Volume detached for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/97230225-f283-4927-a24c-78fece560796-sg-core-conf-yaml\") on node \"crc\" DevicePath \"\"" Dec 03 11:58:58 crc kubenswrapper[5052]: I1203 11:58:58.837397 5052 scope.go:117] "RemoveContainer" containerID="d7086739b41b7d360cbe4227c0a28ecffd42edb2ee30fb5fcc39d0c67f27ead1" Dec 03 11:58:58 crc kubenswrapper[5052]: E1203 11:58:58.837877 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d7086739b41b7d360cbe4227c0a28ecffd42edb2ee30fb5fcc39d0c67f27ead1\": container with ID starting with d7086739b41b7d360cbe4227c0a28ecffd42edb2ee30fb5fcc39d0c67f27ead1 not found: ID does not exist" containerID="d7086739b41b7d360cbe4227c0a28ecffd42edb2ee30fb5fcc39d0c67f27ead1" Dec 03 11:58:58 crc kubenswrapper[5052]: I1203 11:58:58.837913 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d7086739b41b7d360cbe4227c0a28ecffd42edb2ee30fb5fcc39d0c67f27ead1"} err="failed to get container status \"d7086739b41b7d360cbe4227c0a28ecffd42edb2ee30fb5fcc39d0c67f27ead1\": rpc error: code = NotFound desc = could not find container \"d7086739b41b7d360cbe4227c0a28ecffd42edb2ee30fb5fcc39d0c67f27ead1\": container with ID starting with d7086739b41b7d360cbe4227c0a28ecffd42edb2ee30fb5fcc39d0c67f27ead1 not found: ID does not exist" Dec 03 11:58:58 crc kubenswrapper[5052]: I1203 11:58:58.901239 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/watcher-decision-engine-0"] Dec 03 11:58:58 crc kubenswrapper[5052]: I1203 11:58:58.909200 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/watcher-decision-engine-0"] Dec 03 11:58:58 crc kubenswrapper[5052]: I1203 11:58:58.936857 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/watcher-decision-engine-0"] Dec 03 11:58:58 crc kubenswrapper[5052]: E1203 11:58:58.937542 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="8746e6ab-2863-45c2-b516-3b2bdfed2fb9" containerName="extract-utilities" Dec 03 11:58:58 crc kubenswrapper[5052]: I1203 11:58:58.937640 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="8746e6ab-2863-45c2-b516-3b2bdfed2fb9" containerName="extract-utilities" Dec 03 11:58:58 crc kubenswrapper[5052]: E1203 11:58:58.937723 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="8746e6ab-2863-45c2-b516-3b2bdfed2fb9" containerName="extract-content" Dec 03 11:58:58 crc kubenswrapper[5052]: I1203 11:58:58.937783 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="8746e6ab-2863-45c2-b516-3b2bdfed2fb9" containerName="extract-content" Dec 03 11:58:58 crc kubenswrapper[5052]: E1203 11:58:58.937844 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="5893db31-2978-45eb-8407-2e4d178954ce" containerName="watcher-decision-engine" Dec 03 11:58:58 crc kubenswrapper[5052]: I1203 11:58:58.937900 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="5893db31-2978-45eb-8407-2e4d178954ce" containerName="watcher-decision-engine" Dec 03 11:58:58 crc kubenswrapper[5052]: E1203 11:58:58.937971 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="8746e6ab-2863-45c2-b516-3b2bdfed2fb9" containerName="registry-server" Dec 03 11:58:58 crc kubenswrapper[5052]: I1203 11:58:58.938025 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="8746e6ab-2863-45c2-b516-3b2bdfed2fb9" containerName="registry-server" Dec 03 11:58:58 crc kubenswrapper[5052]: I1203 11:58:58.938284 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="5893db31-2978-45eb-8407-2e4d178954ce" containerName="watcher-decision-engine" Dec 03 11:58:58 crc kubenswrapper[5052]: I1203 11:58:58.938376 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="8746e6ab-2863-45c2-b516-3b2bdfed2fb9" containerName="registry-server" Dec 03 11:58:58 crc kubenswrapper[5052]: I1203 11:58:58.939152 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/watcher-decision-engine-0" Dec 03 11:58:58 crc kubenswrapper[5052]: I1203 11:58:58.941448 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"watcher-decision-engine-config-data" Dec 03 11:58:58 crc kubenswrapper[5052]: I1203 11:58:58.946381 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/watcher-decision-engine-0"] Dec 03 11:58:59 crc kubenswrapper[5052]: I1203 11:58:59.012022 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/e9985bb3-f162-4793-9491-2e4f46644071-logs\") pod \"watcher-decision-engine-0\" (UID: \"e9985bb3-f162-4793-9491-2e4f46644071\") " pod="openstack-kuttl-tests/watcher-decision-engine-0" Dec 03 11:58:59 crc kubenswrapper[5052]: I1203 11:58:59.012161 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/e9985bb3-f162-4793-9491-2e4f46644071-config-data\") pod \"watcher-decision-engine-0\" (UID: \"e9985bb3-f162-4793-9491-2e4f46644071\") " pod="openstack-kuttl-tests/watcher-decision-engine-0" Dec 03 11:58:59 crc kubenswrapper[5052]: I1203 11:58:59.012286 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ngtzp\" (UniqueName: \"kubernetes.io/projected/e9985bb3-f162-4793-9491-2e4f46644071-kube-api-access-ngtzp\") pod \"watcher-decision-engine-0\" (UID: \"e9985bb3-f162-4793-9491-2e4f46644071\") " pod="openstack-kuttl-tests/watcher-decision-engine-0" Dec 03 11:58:59 crc kubenswrapper[5052]: I1203 11:58:59.012412 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e9985bb3-f162-4793-9491-2e4f46644071-combined-ca-bundle\") pod \"watcher-decision-engine-0\" (UID: \"e9985bb3-f162-4793-9491-2e4f46644071\") " pod="openstack-kuttl-tests/watcher-decision-engine-0" Dec 03 11:58:59 crc kubenswrapper[5052]: I1203 11:58:59.012495 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"custom-prometheus-ca\" (UniqueName: \"kubernetes.io/secret/e9985bb3-f162-4793-9491-2e4f46644071-custom-prometheus-ca\") pod \"watcher-decision-engine-0\" (UID: \"e9985bb3-f162-4793-9491-2e4f46644071\") " pod="openstack-kuttl-tests/watcher-decision-engine-0" Dec 03 11:58:59 crc kubenswrapper[5052]: I1203 11:58:59.113640 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/e9985bb3-f162-4793-9491-2e4f46644071-config-data\") pod \"watcher-decision-engine-0\" (UID: \"e9985bb3-f162-4793-9491-2e4f46644071\") " pod="openstack-kuttl-tests/watcher-decision-engine-0" Dec 03 11:58:59 crc kubenswrapper[5052]: I1203 11:58:59.113721 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-ngtzp\" (UniqueName: \"kubernetes.io/projected/e9985bb3-f162-4793-9491-2e4f46644071-kube-api-access-ngtzp\") pod \"watcher-decision-engine-0\" (UID: \"e9985bb3-f162-4793-9491-2e4f46644071\") " pod="openstack-kuttl-tests/watcher-decision-engine-0" Dec 03 11:58:59 crc kubenswrapper[5052]: I1203 11:58:59.113780 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e9985bb3-f162-4793-9491-2e4f46644071-combined-ca-bundle\") pod \"watcher-decision-engine-0\" (UID: \"e9985bb3-f162-4793-9491-2e4f46644071\") " pod="openstack-kuttl-tests/watcher-decision-engine-0" Dec 03 11:58:59 crc kubenswrapper[5052]: I1203 11:58:59.113828 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"custom-prometheus-ca\" (UniqueName: \"kubernetes.io/secret/e9985bb3-f162-4793-9491-2e4f46644071-custom-prometheus-ca\") pod \"watcher-decision-engine-0\" (UID: \"e9985bb3-f162-4793-9491-2e4f46644071\") " pod="openstack-kuttl-tests/watcher-decision-engine-0" Dec 03 11:58:59 crc kubenswrapper[5052]: I1203 11:58:59.113919 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/e9985bb3-f162-4793-9491-2e4f46644071-logs\") pod \"watcher-decision-engine-0\" (UID: \"e9985bb3-f162-4793-9491-2e4f46644071\") " pod="openstack-kuttl-tests/watcher-decision-engine-0" Dec 03 11:58:59 crc kubenswrapper[5052]: I1203 11:58:59.114291 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/e9985bb3-f162-4793-9491-2e4f46644071-logs\") pod \"watcher-decision-engine-0\" (UID: \"e9985bb3-f162-4793-9491-2e4f46644071\") " pod="openstack-kuttl-tests/watcher-decision-engine-0" Dec 03 11:58:59 crc kubenswrapper[5052]: I1203 11:58:59.119111 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e9985bb3-f162-4793-9491-2e4f46644071-combined-ca-bundle\") pod \"watcher-decision-engine-0\" (UID: \"e9985bb3-f162-4793-9491-2e4f46644071\") " pod="openstack-kuttl-tests/watcher-decision-engine-0" Dec 03 11:58:59 crc kubenswrapper[5052]: I1203 11:58:59.120130 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"custom-prometheus-ca\" (UniqueName: \"kubernetes.io/secret/e9985bb3-f162-4793-9491-2e4f46644071-custom-prometheus-ca\") pod \"watcher-decision-engine-0\" (UID: \"e9985bb3-f162-4793-9491-2e4f46644071\") " pod="openstack-kuttl-tests/watcher-decision-engine-0" Dec 03 11:58:59 crc kubenswrapper[5052]: I1203 11:58:59.122547 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/e9985bb3-f162-4793-9491-2e4f46644071-config-data\") pod \"watcher-decision-engine-0\" (UID: \"e9985bb3-f162-4793-9491-2e4f46644071\") " pod="openstack-kuttl-tests/watcher-decision-engine-0" Dec 03 11:58:59 crc kubenswrapper[5052]: I1203 11:58:59.130969 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-ngtzp\" (UniqueName: \"kubernetes.io/projected/e9985bb3-f162-4793-9491-2e4f46644071-kube-api-access-ngtzp\") pod \"watcher-decision-engine-0\" (UID: \"e9985bb3-f162-4793-9491-2e4f46644071\") " pod="openstack-kuttl-tests/watcher-decision-engine-0" Dec 03 11:58:59 crc kubenswrapper[5052]: I1203 11:58:59.254880 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/watcher-decision-engine-0" Dec 03 11:58:59 crc kubenswrapper[5052]: I1203 11:58:59.433630 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:58:59 crc kubenswrapper[5052]: I1203 11:58:59.528815 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/9fe1f02f-d2cf-4220-925c-6ecdcd099048-logs\") pod \"9fe1f02f-d2cf-4220-925c-6ecdcd099048\" (UID: \"9fe1f02f-d2cf-4220-925c-6ecdcd099048\") " Dec 03 11:58:59 crc kubenswrapper[5052]: I1203 11:58:59.528872 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"glance\" (UniqueName: \"kubernetes.io/local-volume/local-storage16-crc\") pod \"9fe1f02f-d2cf-4220-925c-6ecdcd099048\" (UID: \"9fe1f02f-d2cf-4220-925c-6ecdcd099048\") " Dec 03 11:58:59 crc kubenswrapper[5052]: I1203 11:58:59.528979 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/9fe1f02f-d2cf-4220-925c-6ecdcd099048-httpd-run\") pod \"9fe1f02f-d2cf-4220-925c-6ecdcd099048\" (UID: \"9fe1f02f-d2cf-4220-925c-6ecdcd099048\") " Dec 03 11:58:59 crc kubenswrapper[5052]: I1203 11:58:59.529069 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x829v\" (UniqueName: \"kubernetes.io/projected/9fe1f02f-d2cf-4220-925c-6ecdcd099048-kube-api-access-x829v\") pod \"9fe1f02f-d2cf-4220-925c-6ecdcd099048\" (UID: \"9fe1f02f-d2cf-4220-925c-6ecdcd099048\") " Dec 03 11:58:59 crc kubenswrapper[5052]: I1203 11:58:59.529106 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9fe1f02f-d2cf-4220-925c-6ecdcd099048-combined-ca-bundle\") pod \"9fe1f02f-d2cf-4220-925c-6ecdcd099048\" (UID: \"9fe1f02f-d2cf-4220-925c-6ecdcd099048\") " Dec 03 11:58:59 crc kubenswrapper[5052]: I1203 11:58:59.529164 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/9fe1f02f-d2cf-4220-925c-6ecdcd099048-public-tls-certs\") pod \"9fe1f02f-d2cf-4220-925c-6ecdcd099048\" (UID: \"9fe1f02f-d2cf-4220-925c-6ecdcd099048\") " Dec 03 11:58:59 crc kubenswrapper[5052]: I1203 11:58:59.529206 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/9fe1f02f-d2cf-4220-925c-6ecdcd099048-config-data\") pod \"9fe1f02f-d2cf-4220-925c-6ecdcd099048\" (UID: \"9fe1f02f-d2cf-4220-925c-6ecdcd099048\") " Dec 03 11:58:59 crc kubenswrapper[5052]: I1203 11:58:59.529230 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/9fe1f02f-d2cf-4220-925c-6ecdcd099048-scripts\") pod \"9fe1f02f-d2cf-4220-925c-6ecdcd099048\" (UID: \"9fe1f02f-d2cf-4220-925c-6ecdcd099048\") " Dec 03 11:58:59 crc kubenswrapper[5052]: I1203 11:58:59.532075 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9fe1f02f-d2cf-4220-925c-6ecdcd099048-httpd-run" (OuterVolumeSpecName: "httpd-run") pod "9fe1f02f-d2cf-4220-925c-6ecdcd099048" (UID: "9fe1f02f-d2cf-4220-925c-6ecdcd099048"). InnerVolumeSpecName "httpd-run". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:58:59 crc kubenswrapper[5052]: I1203 11:58:59.532469 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9fe1f02f-d2cf-4220-925c-6ecdcd099048-logs" (OuterVolumeSpecName: "logs") pod "9fe1f02f-d2cf-4220-925c-6ecdcd099048" (UID: "9fe1f02f-d2cf-4220-925c-6ecdcd099048"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:58:59 crc kubenswrapper[5052]: I1203 11:58:59.535719 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9fe1f02f-d2cf-4220-925c-6ecdcd099048-scripts" (OuterVolumeSpecName: "scripts") pod "9fe1f02f-d2cf-4220-925c-6ecdcd099048" (UID: "9fe1f02f-d2cf-4220-925c-6ecdcd099048"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:58:59 crc kubenswrapper[5052]: I1203 11:58:59.537355 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage16-crc" (OuterVolumeSpecName: "glance") pod "9fe1f02f-d2cf-4220-925c-6ecdcd099048" (UID: "9fe1f02f-d2cf-4220-925c-6ecdcd099048"). InnerVolumeSpecName "local-storage16-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 11:58:59 crc kubenswrapper[5052]: I1203 11:58:59.546026 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9fe1f02f-d2cf-4220-925c-6ecdcd099048-kube-api-access-x829v" (OuterVolumeSpecName: "kube-api-access-x829v") pod "9fe1f02f-d2cf-4220-925c-6ecdcd099048" (UID: "9fe1f02f-d2cf-4220-925c-6ecdcd099048"). InnerVolumeSpecName "kube-api-access-x829v". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:58:59 crc kubenswrapper[5052]: I1203 11:58:59.574730 5052 generic.go:334] "Generic (PLEG): container finished" podID="9fe1f02f-d2cf-4220-925c-6ecdcd099048" containerID="0d2b603ec774a9e5e57e3ee6ad57ea900ac923cb148ec977d0759fb33a4ba2e8" exitCode=0 Dec 03 11:58:59 crc kubenswrapper[5052]: I1203 11:58:59.574822 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"9fe1f02f-d2cf-4220-925c-6ecdcd099048","Type":"ContainerDied","Data":"0d2b603ec774a9e5e57e3ee6ad57ea900ac923cb148ec977d0759fb33a4ba2e8"} Dec 03 11:58:59 crc kubenswrapper[5052]: I1203 11:58:59.574855 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"9fe1f02f-d2cf-4220-925c-6ecdcd099048","Type":"ContainerDied","Data":"ff8d4a99acf44da56b0cdd06d13bdea02536f2f50d2a62e30c7dfc1dacef5465"} Dec 03 11:58:59 crc kubenswrapper[5052]: I1203 11:58:59.574873 5052 scope.go:117] "RemoveContainer" containerID="0d2b603ec774a9e5e57e3ee6ad57ea900ac923cb148ec977d0759fb33a4ba2e8" Dec 03 11:58:59 crc kubenswrapper[5052]: I1203 11:58:59.574904 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:58:59 crc kubenswrapper[5052]: I1203 11:58:59.584648 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:58:59 crc kubenswrapper[5052]: I1203 11:58:59.588957 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9fe1f02f-d2cf-4220-925c-6ecdcd099048-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "9fe1f02f-d2cf-4220-925c-6ecdcd099048" (UID: "9fe1f02f-d2cf-4220-925c-6ecdcd099048"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:58:59 crc kubenswrapper[5052]: I1203 11:58:59.631479 5052 reconciler_common.go:293] "Volume detached for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/9fe1f02f-d2cf-4220-925c-6ecdcd099048-httpd-run\") on node \"crc\" DevicePath \"\"" Dec 03 11:58:59 crc kubenswrapper[5052]: I1203 11:58:59.631536 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-x829v\" (UniqueName: \"kubernetes.io/projected/9fe1f02f-d2cf-4220-925c-6ecdcd099048-kube-api-access-x829v\") on node \"crc\" DevicePath \"\"" Dec 03 11:58:59 crc kubenswrapper[5052]: I1203 11:58:59.631551 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9fe1f02f-d2cf-4220-925c-6ecdcd099048-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:58:59 crc kubenswrapper[5052]: I1203 11:58:59.631563 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/9fe1f02f-d2cf-4220-925c-6ecdcd099048-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:58:59 crc kubenswrapper[5052]: I1203 11:58:59.631574 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/9fe1f02f-d2cf-4220-925c-6ecdcd099048-logs\") on node \"crc\" DevicePath \"\"" Dec 03 11:58:59 crc kubenswrapper[5052]: I1203 11:58:59.631601 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage16-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage16-crc\") on node \"crc\" " Dec 03 11:58:59 crc kubenswrapper[5052]: I1203 11:58:59.637757 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9fe1f02f-d2cf-4220-925c-6ecdcd099048-config-data" (OuterVolumeSpecName: "config-data") pod "9fe1f02f-d2cf-4220-925c-6ecdcd099048" (UID: "9fe1f02f-d2cf-4220-925c-6ecdcd099048"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:58:59 crc kubenswrapper[5052]: I1203 11:58:59.637780 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9fe1f02f-d2cf-4220-925c-6ecdcd099048-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "9fe1f02f-d2cf-4220-925c-6ecdcd099048" (UID: "9fe1f02f-d2cf-4220-925c-6ecdcd099048"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:58:59 crc kubenswrapper[5052]: I1203 11:58:59.642596 5052 scope.go:117] "RemoveContainer" containerID="cb18e4b2f1c1bd57f2ceaa59ede059a92c607d787ad9e92d233d8bbb0cff36be" Dec 03 11:58:59 crc kubenswrapper[5052]: I1203 11:58:59.656197 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage16-crc" (UniqueName: "kubernetes.io/local-volume/local-storage16-crc") on node "crc" Dec 03 11:58:59 crc kubenswrapper[5052]: I1203 11:58:59.687639 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5893db31-2978-45eb-8407-2e4d178954ce" path="/var/lib/kubelet/pods/5893db31-2978-45eb-8407-2e4d178954ce/volumes" Dec 03 11:58:59 crc kubenswrapper[5052]: I1203 11:58:59.690493 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8746e6ab-2863-45c2-b516-3b2bdfed2fb9" path="/var/lib/kubelet/pods/8746e6ab-2863-45c2-b516-3b2bdfed2fb9/volumes" Dec 03 11:58:59 crc kubenswrapper[5052]: I1203 11:58:59.691378 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="977a3b90-7dc4-4e79-9117-6218dccc8f5e" path="/var/lib/kubelet/pods/977a3b90-7dc4-4e79-9117-6218dccc8f5e/volumes" Dec 03 11:58:59 crc kubenswrapper[5052]: I1203 11:58:59.734212 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage16-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage16-crc\") on node \"crc\" DevicePath \"\"" Dec 03 11:58:59 crc kubenswrapper[5052]: I1203 11:58:59.734807 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/9fe1f02f-d2cf-4220-925c-6ecdcd099048-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 11:58:59 crc kubenswrapper[5052]: I1203 11:58:59.734822 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/9fe1f02f-d2cf-4220-925c-6ecdcd099048-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:58:59 crc kubenswrapper[5052]: I1203 11:58:59.767569 5052 scope.go:117] "RemoveContainer" containerID="0d2b603ec774a9e5e57e3ee6ad57ea900ac923cb148ec977d0759fb33a4ba2e8" Dec 03 11:58:59 crc kubenswrapper[5052]: E1203 11:58:59.768335 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0d2b603ec774a9e5e57e3ee6ad57ea900ac923cb148ec977d0759fb33a4ba2e8\": container with ID starting with 0d2b603ec774a9e5e57e3ee6ad57ea900ac923cb148ec977d0759fb33a4ba2e8 not found: ID does not exist" containerID="0d2b603ec774a9e5e57e3ee6ad57ea900ac923cb148ec977d0759fb33a4ba2e8" Dec 03 11:58:59 crc kubenswrapper[5052]: I1203 11:58:59.768372 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0d2b603ec774a9e5e57e3ee6ad57ea900ac923cb148ec977d0759fb33a4ba2e8"} err="failed to get container status \"0d2b603ec774a9e5e57e3ee6ad57ea900ac923cb148ec977d0759fb33a4ba2e8\": rpc error: code = NotFound desc = could not find container \"0d2b603ec774a9e5e57e3ee6ad57ea900ac923cb148ec977d0759fb33a4ba2e8\": container with ID starting with 0d2b603ec774a9e5e57e3ee6ad57ea900ac923cb148ec977d0759fb33a4ba2e8 not found: ID does not exist" Dec 03 11:58:59 crc kubenswrapper[5052]: I1203 11:58:59.768400 5052 scope.go:117] "RemoveContainer" containerID="cb18e4b2f1c1bd57f2ceaa59ede059a92c607d787ad9e92d233d8bbb0cff36be" Dec 03 11:58:59 crc kubenswrapper[5052]: E1203 11:58:59.769937 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cb18e4b2f1c1bd57f2ceaa59ede059a92c607d787ad9e92d233d8bbb0cff36be\": container with ID starting with cb18e4b2f1c1bd57f2ceaa59ede059a92c607d787ad9e92d233d8bbb0cff36be not found: ID does not exist" containerID="cb18e4b2f1c1bd57f2ceaa59ede059a92c607d787ad9e92d233d8bbb0cff36be" Dec 03 11:58:59 crc kubenswrapper[5052]: I1203 11:58:59.769972 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cb18e4b2f1c1bd57f2ceaa59ede059a92c607d787ad9e92d233d8bbb0cff36be"} err="failed to get container status \"cb18e4b2f1c1bd57f2ceaa59ede059a92c607d787ad9e92d233d8bbb0cff36be\": rpc error: code = NotFound desc = could not find container \"cb18e4b2f1c1bd57f2ceaa59ede059a92c607d787ad9e92d233d8bbb0cff36be\": container with ID starting with cb18e4b2f1c1bd57f2ceaa59ede059a92c607d787ad9e92d233d8bbb0cff36be not found: ID does not exist" Dec 03 11:58:59 crc kubenswrapper[5052]: I1203 11:58:59.777755 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/watcher-decision-engine-0"] Dec 03 11:58:59 crc kubenswrapper[5052]: I1203 11:58:59.801782 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:58:59 crc kubenswrapper[5052]: I1203 11:58:59.808285 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:58:59 crc kubenswrapper[5052]: I1203 11:58:59.868738 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:58:59 crc kubenswrapper[5052]: E1203 11:58:59.869479 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="9fe1f02f-d2cf-4220-925c-6ecdcd099048" containerName="glance-httpd" Dec 03 11:58:59 crc kubenswrapper[5052]: I1203 11:58:59.869617 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="9fe1f02f-d2cf-4220-925c-6ecdcd099048" containerName="glance-httpd" Dec 03 11:58:59 crc kubenswrapper[5052]: E1203 11:58:59.869781 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="9fe1f02f-d2cf-4220-925c-6ecdcd099048" containerName="glance-log" Dec 03 11:58:59 crc kubenswrapper[5052]: I1203 11:58:59.869917 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="9fe1f02f-d2cf-4220-925c-6ecdcd099048" containerName="glance-log" Dec 03 11:58:59 crc kubenswrapper[5052]: I1203 11:58:59.870328 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="9fe1f02f-d2cf-4220-925c-6ecdcd099048" containerName="glance-log" Dec 03 11:58:59 crc kubenswrapper[5052]: I1203 11:58:59.870464 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="9fe1f02f-d2cf-4220-925c-6ecdcd099048" containerName="glance-httpd" Dec 03 11:58:59 crc kubenswrapper[5052]: I1203 11:58:59.873059 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:58:59 crc kubenswrapper[5052]: I1203 11:58:59.881015 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-scripts" Dec 03 11:58:59 crc kubenswrapper[5052]: I1203 11:58:59.881226 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-config-data" Dec 03 11:58:59 crc kubenswrapper[5052]: I1203 11:58:59.902912 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:58:59 crc kubenswrapper[5052]: I1203 11:58:59.983851 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 11:59:00 crc kubenswrapper[5052]: I1203 11:59:00.017118 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 11:59:00 crc kubenswrapper[5052]: I1203 11:59:00.033033 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 11:59:00 crc kubenswrapper[5052]: I1203 11:59:00.038847 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:59:00 crc kubenswrapper[5052]: I1203 11:59:00.040714 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/e8e69500-b31c-4293-8b70-e20b3fb0a92c-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"e8e69500-b31c-4293-8b70-e20b3fb0a92c\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:59:00 crc kubenswrapper[5052]: I1203 11:59:00.040803 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/e8e69500-b31c-4293-8b70-e20b3fb0a92c-config-data\") pod \"ceilometer-0\" (UID: \"e8e69500-b31c-4293-8b70-e20b3fb0a92c\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:59:00 crc kubenswrapper[5052]: I1203 11:59:00.040871 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jsk62\" (UniqueName: \"kubernetes.io/projected/e8e69500-b31c-4293-8b70-e20b3fb0a92c-kube-api-access-jsk62\") pod \"ceilometer-0\" (UID: \"e8e69500-b31c-4293-8b70-e20b3fb0a92c\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:59:00 crc kubenswrapper[5052]: I1203 11:59:00.040967 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/e8e69500-b31c-4293-8b70-e20b3fb0a92c-scripts\") pod \"ceilometer-0\" (UID: \"e8e69500-b31c-4293-8b70-e20b3fb0a92c\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:59:00 crc kubenswrapper[5052]: I1203 11:59:00.041013 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/e8e69500-b31c-4293-8b70-e20b3fb0a92c-run-httpd\") pod \"ceilometer-0\" (UID: \"e8e69500-b31c-4293-8b70-e20b3fb0a92c\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:59:00 crc kubenswrapper[5052]: I1203 11:59:00.041047 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e8e69500-b31c-4293-8b70-e20b3fb0a92c-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"e8e69500-b31c-4293-8b70-e20b3fb0a92c\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:59:00 crc kubenswrapper[5052]: I1203 11:59:00.041108 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/e8e69500-b31c-4293-8b70-e20b3fb0a92c-log-httpd\") pod \"ceilometer-0\" (UID: \"e8e69500-b31c-4293-8b70-e20b3fb0a92c\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:59:00 crc kubenswrapper[5052]: I1203 11:59:00.048049 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-glance-default-public-svc" Dec 03 11:59:00 crc kubenswrapper[5052]: I1203 11:59:00.048256 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-default-external-config-data" Dec 03 11:59:00 crc kubenswrapper[5052]: I1203 11:59:00.051661 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 11:59:00 crc kubenswrapper[5052]: I1203 11:59:00.142806 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/e8e69500-b31c-4293-8b70-e20b3fb0a92c-config-data\") pod \"ceilometer-0\" (UID: \"e8e69500-b31c-4293-8b70-e20b3fb0a92c\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:59:00 crc kubenswrapper[5052]: I1203 11:59:00.142872 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-jsk62\" (UniqueName: \"kubernetes.io/projected/e8e69500-b31c-4293-8b70-e20b3fb0a92c-kube-api-access-jsk62\") pod \"ceilometer-0\" (UID: \"e8e69500-b31c-4293-8b70-e20b3fb0a92c\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:59:00 crc kubenswrapper[5052]: I1203 11:59:00.142903 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/51a9dfff-37f2-4fbe-b2d1-039f20bb4560-httpd-run\") pod \"glance-default-external-api-0\" (UID: \"51a9dfff-37f2-4fbe-b2d1-039f20bb4560\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:59:00 crc kubenswrapper[5052]: I1203 11:59:00.142938 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage16-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage16-crc\") pod \"glance-default-external-api-0\" (UID: \"51a9dfff-37f2-4fbe-b2d1-039f20bb4560\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:59:00 crc kubenswrapper[5052]: I1203 11:59:00.142960 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/e8e69500-b31c-4293-8b70-e20b3fb0a92c-scripts\") pod \"ceilometer-0\" (UID: \"e8e69500-b31c-4293-8b70-e20b3fb0a92c\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:59:00 crc kubenswrapper[5052]: I1203 11:59:00.142982 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/e8e69500-b31c-4293-8b70-e20b3fb0a92c-run-httpd\") pod \"ceilometer-0\" (UID: \"e8e69500-b31c-4293-8b70-e20b3fb0a92c\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:59:00 crc kubenswrapper[5052]: I1203 11:59:00.143007 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e8e69500-b31c-4293-8b70-e20b3fb0a92c-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"e8e69500-b31c-4293-8b70-e20b3fb0a92c\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:59:00 crc kubenswrapper[5052]: I1203 11:59:00.143031 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/e8e69500-b31c-4293-8b70-e20b3fb0a92c-log-httpd\") pod \"ceilometer-0\" (UID: \"e8e69500-b31c-4293-8b70-e20b3fb0a92c\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:59:00 crc kubenswrapper[5052]: I1203 11:59:00.143066 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/51a9dfff-37f2-4fbe-b2d1-039f20bb4560-scripts\") pod \"glance-default-external-api-0\" (UID: \"51a9dfff-37f2-4fbe-b2d1-039f20bb4560\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:59:00 crc kubenswrapper[5052]: I1203 11:59:00.143096 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qmrtr\" (UniqueName: \"kubernetes.io/projected/51a9dfff-37f2-4fbe-b2d1-039f20bb4560-kube-api-access-qmrtr\") pod \"glance-default-external-api-0\" (UID: \"51a9dfff-37f2-4fbe-b2d1-039f20bb4560\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:59:00 crc kubenswrapper[5052]: I1203 11:59:00.143118 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/51a9dfff-37f2-4fbe-b2d1-039f20bb4560-config-data\") pod \"glance-default-external-api-0\" (UID: \"51a9dfff-37f2-4fbe-b2d1-039f20bb4560\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:59:00 crc kubenswrapper[5052]: I1203 11:59:00.143140 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/51a9dfff-37f2-4fbe-b2d1-039f20bb4560-logs\") pod \"glance-default-external-api-0\" (UID: \"51a9dfff-37f2-4fbe-b2d1-039f20bb4560\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:59:00 crc kubenswrapper[5052]: I1203 11:59:00.143168 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/e8e69500-b31c-4293-8b70-e20b3fb0a92c-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"e8e69500-b31c-4293-8b70-e20b3fb0a92c\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:59:00 crc kubenswrapper[5052]: I1203 11:59:00.143185 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/51a9dfff-37f2-4fbe-b2d1-039f20bb4560-combined-ca-bundle\") pod \"glance-default-external-api-0\" (UID: \"51a9dfff-37f2-4fbe-b2d1-039f20bb4560\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:59:00 crc kubenswrapper[5052]: I1203 11:59:00.143245 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/51a9dfff-37f2-4fbe-b2d1-039f20bb4560-public-tls-certs\") pod \"glance-default-external-api-0\" (UID: \"51a9dfff-37f2-4fbe-b2d1-039f20bb4560\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:59:00 crc kubenswrapper[5052]: I1203 11:59:00.144048 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/e8e69500-b31c-4293-8b70-e20b3fb0a92c-log-httpd\") pod \"ceilometer-0\" (UID: \"e8e69500-b31c-4293-8b70-e20b3fb0a92c\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:59:00 crc kubenswrapper[5052]: I1203 11:59:00.144652 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/e8e69500-b31c-4293-8b70-e20b3fb0a92c-run-httpd\") pod \"ceilometer-0\" (UID: \"e8e69500-b31c-4293-8b70-e20b3fb0a92c\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:59:00 crc kubenswrapper[5052]: I1203 11:59:00.166490 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/e8e69500-b31c-4293-8b70-e20b3fb0a92c-scripts\") pod \"ceilometer-0\" (UID: \"e8e69500-b31c-4293-8b70-e20b3fb0a92c\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:59:00 crc kubenswrapper[5052]: I1203 11:59:00.166769 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e8e69500-b31c-4293-8b70-e20b3fb0a92c-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"e8e69500-b31c-4293-8b70-e20b3fb0a92c\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:59:00 crc kubenswrapper[5052]: I1203 11:59:00.166854 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/e8e69500-b31c-4293-8b70-e20b3fb0a92c-config-data\") pod \"ceilometer-0\" (UID: \"e8e69500-b31c-4293-8b70-e20b3fb0a92c\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:59:00 crc kubenswrapper[5052]: I1203 11:59:00.167284 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/e8e69500-b31c-4293-8b70-e20b3fb0a92c-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"e8e69500-b31c-4293-8b70-e20b3fb0a92c\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:59:00 crc kubenswrapper[5052]: I1203 11:59:00.182183 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-jsk62\" (UniqueName: \"kubernetes.io/projected/e8e69500-b31c-4293-8b70-e20b3fb0a92c-kube-api-access-jsk62\") pod \"ceilometer-0\" (UID: \"e8e69500-b31c-4293-8b70-e20b3fb0a92c\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:59:00 crc kubenswrapper[5052]: I1203 11:59:00.236029 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:59:00 crc kubenswrapper[5052]: I1203 11:59:00.245354 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/51a9dfff-37f2-4fbe-b2d1-039f20bb4560-httpd-run\") pod \"glance-default-external-api-0\" (UID: \"51a9dfff-37f2-4fbe-b2d1-039f20bb4560\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:59:00 crc kubenswrapper[5052]: I1203 11:59:00.245424 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage16-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage16-crc\") pod \"glance-default-external-api-0\" (UID: \"51a9dfff-37f2-4fbe-b2d1-039f20bb4560\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:59:00 crc kubenswrapper[5052]: I1203 11:59:00.245491 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/51a9dfff-37f2-4fbe-b2d1-039f20bb4560-scripts\") pod \"glance-default-external-api-0\" (UID: \"51a9dfff-37f2-4fbe-b2d1-039f20bb4560\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:59:00 crc kubenswrapper[5052]: I1203 11:59:00.245548 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/51a9dfff-37f2-4fbe-b2d1-039f20bb4560-config-data\") pod \"glance-default-external-api-0\" (UID: \"51a9dfff-37f2-4fbe-b2d1-039f20bb4560\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:59:00 crc kubenswrapper[5052]: I1203 11:59:00.245569 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-qmrtr\" (UniqueName: \"kubernetes.io/projected/51a9dfff-37f2-4fbe-b2d1-039f20bb4560-kube-api-access-qmrtr\") pod \"glance-default-external-api-0\" (UID: \"51a9dfff-37f2-4fbe-b2d1-039f20bb4560\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:59:00 crc kubenswrapper[5052]: I1203 11:59:00.245593 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/51a9dfff-37f2-4fbe-b2d1-039f20bb4560-logs\") pod \"glance-default-external-api-0\" (UID: \"51a9dfff-37f2-4fbe-b2d1-039f20bb4560\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:59:00 crc kubenswrapper[5052]: I1203 11:59:00.245618 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/51a9dfff-37f2-4fbe-b2d1-039f20bb4560-combined-ca-bundle\") pod \"glance-default-external-api-0\" (UID: \"51a9dfff-37f2-4fbe-b2d1-039f20bb4560\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:59:00 crc kubenswrapper[5052]: I1203 11:59:00.245648 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/51a9dfff-37f2-4fbe-b2d1-039f20bb4560-public-tls-certs\") pod \"glance-default-external-api-0\" (UID: \"51a9dfff-37f2-4fbe-b2d1-039f20bb4560\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:59:00 crc kubenswrapper[5052]: I1203 11:59:00.245974 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/51a9dfff-37f2-4fbe-b2d1-039f20bb4560-httpd-run\") pod \"glance-default-external-api-0\" (UID: \"51a9dfff-37f2-4fbe-b2d1-039f20bb4560\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:59:00 crc kubenswrapper[5052]: I1203 11:59:00.246021 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage16-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage16-crc\") pod \"glance-default-external-api-0\" (UID: \"51a9dfff-37f2-4fbe-b2d1-039f20bb4560\") device mount path \"/mnt/openstack/pv16\"" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:59:00 crc kubenswrapper[5052]: I1203 11:59:00.246252 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/51a9dfff-37f2-4fbe-b2d1-039f20bb4560-logs\") pod \"glance-default-external-api-0\" (UID: \"51a9dfff-37f2-4fbe-b2d1-039f20bb4560\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:59:00 crc kubenswrapper[5052]: I1203 11:59:00.252891 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/51a9dfff-37f2-4fbe-b2d1-039f20bb4560-config-data\") pod \"glance-default-external-api-0\" (UID: \"51a9dfff-37f2-4fbe-b2d1-039f20bb4560\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:59:00 crc kubenswrapper[5052]: I1203 11:59:00.253581 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/51a9dfff-37f2-4fbe-b2d1-039f20bb4560-scripts\") pod \"glance-default-external-api-0\" (UID: \"51a9dfff-37f2-4fbe-b2d1-039f20bb4560\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:59:00 crc kubenswrapper[5052]: I1203 11:59:00.258835 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/51a9dfff-37f2-4fbe-b2d1-039f20bb4560-combined-ca-bundle\") pod \"glance-default-external-api-0\" (UID: \"51a9dfff-37f2-4fbe-b2d1-039f20bb4560\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:59:00 crc kubenswrapper[5052]: I1203 11:59:00.259091 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/51a9dfff-37f2-4fbe-b2d1-039f20bb4560-public-tls-certs\") pod \"glance-default-external-api-0\" (UID: \"51a9dfff-37f2-4fbe-b2d1-039f20bb4560\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:59:00 crc kubenswrapper[5052]: I1203 11:59:00.268327 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-qmrtr\" (UniqueName: \"kubernetes.io/projected/51a9dfff-37f2-4fbe-b2d1-039f20bb4560-kube-api-access-qmrtr\") pod \"glance-default-external-api-0\" (UID: \"51a9dfff-37f2-4fbe-b2d1-039f20bb4560\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:59:00 crc kubenswrapper[5052]: I1203 11:59:00.289890 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage16-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage16-crc\") pod \"glance-default-external-api-0\" (UID: \"51a9dfff-37f2-4fbe-b2d1-039f20bb4560\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:59:00 crc kubenswrapper[5052]: I1203 11:59:00.374803 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:59:00 crc kubenswrapper[5052]: I1203 11:59:00.620330 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/watcher-decision-engine-0" event={"ID":"e9985bb3-f162-4793-9491-2e4f46644071","Type":"ContainerStarted","Data":"1a61e630d65199a5ceab549e6965a4dfa5b829b5db1743b8843226f45bd3e896"} Dec 03 11:59:00 crc kubenswrapper[5052]: I1203 11:59:00.620396 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/watcher-decision-engine-0" event={"ID":"e9985bb3-f162-4793-9491-2e4f46644071","Type":"ContainerStarted","Data":"d312a50e3e98ec7eec06a0fabc9c4b327c92d31256027ba7ee537bcf312f3337"} Dec 03 11:59:00 crc kubenswrapper[5052]: I1203 11:59:00.648627 5052 generic.go:334] "Generic (PLEG): container finished" podID="0d55422b-84bf-4455-9112-94de20fb5b6d" containerID="2bf75f1bec647492a24e9e2e2a4b240c1dcac0144cf5a6f394ba143a1374064f" exitCode=0 Dec 03 11:59:00 crc kubenswrapper[5052]: I1203 11:59:00.648690 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"0d55422b-84bf-4455-9112-94de20fb5b6d","Type":"ContainerDied","Data":"2bf75f1bec647492a24e9e2e2a4b240c1dcac0144cf5a6f394ba143a1374064f"} Dec 03 11:59:00 crc kubenswrapper[5052]: I1203 11:59:00.651996 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/watcher-decision-engine-0" podStartSLOduration=2.6519744149999998 podStartE2EDuration="2.651974415s" podCreationTimestamp="2025-12-03 11:58:58 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:59:00.64763215 +0000 UTC m=+2331.235873914" watchObservedRunningTime="2025-12-03 11:59:00.651974415 +0000 UTC m=+2331.240216179" Dec 03 11:59:00 crc kubenswrapper[5052]: I1203 11:59:00.673121 5052 scope.go:117] "RemoveContainer" containerID="d224717c6d9ca8135fd510e03a6c40c9f2b1a56dd8b4a3b685af3e2dfa855982" Dec 03 11:59:00 crc kubenswrapper[5052]: E1203 11:59:00.673411 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 11:59:00 crc kubenswrapper[5052]: I1203 11:59:00.774809 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:59:00 crc kubenswrapper[5052]: I1203 11:59:00.796499 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:59:00 crc kubenswrapper[5052]: I1203 11:59:00.962535 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:59:01 crc kubenswrapper[5052]: I1203 11:59:01.060093 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 11:59:01 crc kubenswrapper[5052]: I1203 11:59:01.082767 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"glance\" (UniqueName: \"kubernetes.io/local-volume/local-storage15-crc\") pod \"0d55422b-84bf-4455-9112-94de20fb5b6d\" (UID: \"0d55422b-84bf-4455-9112-94de20fb5b6d\") " Dec 03 11:59:01 crc kubenswrapper[5052]: I1203 11:59:01.083332 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/0d55422b-84bf-4455-9112-94de20fb5b6d-scripts\") pod \"0d55422b-84bf-4455-9112-94de20fb5b6d\" (UID: \"0d55422b-84bf-4455-9112-94de20fb5b6d\") " Dec 03 11:59:01 crc kubenswrapper[5052]: I1203 11:59:01.083387 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fvnwr\" (UniqueName: \"kubernetes.io/projected/0d55422b-84bf-4455-9112-94de20fb5b6d-kube-api-access-fvnwr\") pod \"0d55422b-84bf-4455-9112-94de20fb5b6d\" (UID: \"0d55422b-84bf-4455-9112-94de20fb5b6d\") " Dec 03 11:59:01 crc kubenswrapper[5052]: I1203 11:59:01.083408 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0d55422b-84bf-4455-9112-94de20fb5b6d-combined-ca-bundle\") pod \"0d55422b-84bf-4455-9112-94de20fb5b6d\" (UID: \"0d55422b-84bf-4455-9112-94de20fb5b6d\") " Dec 03 11:59:01 crc kubenswrapper[5052]: I1203 11:59:01.083481 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/0d55422b-84bf-4455-9112-94de20fb5b6d-httpd-run\") pod \"0d55422b-84bf-4455-9112-94de20fb5b6d\" (UID: \"0d55422b-84bf-4455-9112-94de20fb5b6d\") " Dec 03 11:59:01 crc kubenswrapper[5052]: I1203 11:59:01.083614 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/0d55422b-84bf-4455-9112-94de20fb5b6d-config-data\") pod \"0d55422b-84bf-4455-9112-94de20fb5b6d\" (UID: \"0d55422b-84bf-4455-9112-94de20fb5b6d\") " Dec 03 11:59:01 crc kubenswrapper[5052]: I1203 11:59:01.083654 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/0d55422b-84bf-4455-9112-94de20fb5b6d-logs\") pod \"0d55422b-84bf-4455-9112-94de20fb5b6d\" (UID: \"0d55422b-84bf-4455-9112-94de20fb5b6d\") " Dec 03 11:59:01 crc kubenswrapper[5052]: I1203 11:59:01.085429 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d55422b-84bf-4455-9112-94de20fb5b6d-httpd-run" (OuterVolumeSpecName: "httpd-run") pod "0d55422b-84bf-4455-9112-94de20fb5b6d" (UID: "0d55422b-84bf-4455-9112-94de20fb5b6d"). InnerVolumeSpecName "httpd-run". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:59:01 crc kubenswrapper[5052]: I1203 11:59:01.086764 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/0d55422b-84bf-4455-9112-94de20fb5b6d-internal-tls-certs\") pod \"0d55422b-84bf-4455-9112-94de20fb5b6d\" (UID: \"0d55422b-84bf-4455-9112-94de20fb5b6d\") " Dec 03 11:59:01 crc kubenswrapper[5052]: I1203 11:59:01.087863 5052 reconciler_common.go:293] "Volume detached for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/0d55422b-84bf-4455-9112-94de20fb5b6d-httpd-run\") on node \"crc\" DevicePath \"\"" Dec 03 11:59:01 crc kubenswrapper[5052]: I1203 11:59:01.091365 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage15-crc" (OuterVolumeSpecName: "glance") pod "0d55422b-84bf-4455-9112-94de20fb5b6d" (UID: "0d55422b-84bf-4455-9112-94de20fb5b6d"). InnerVolumeSpecName "local-storage15-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 11:59:01 crc kubenswrapper[5052]: I1203 11:59:01.091635 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0d55422b-84bf-4455-9112-94de20fb5b6d-scripts" (OuterVolumeSpecName: "scripts") pod "0d55422b-84bf-4455-9112-94de20fb5b6d" (UID: "0d55422b-84bf-4455-9112-94de20fb5b6d"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:59:01 crc kubenswrapper[5052]: I1203 11:59:01.091828 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d55422b-84bf-4455-9112-94de20fb5b6d-logs" (OuterVolumeSpecName: "logs") pod "0d55422b-84bf-4455-9112-94de20fb5b6d" (UID: "0d55422b-84bf-4455-9112-94de20fb5b6d"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:59:01 crc kubenswrapper[5052]: I1203 11:59:01.104687 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0d55422b-84bf-4455-9112-94de20fb5b6d-kube-api-access-fvnwr" (OuterVolumeSpecName: "kube-api-access-fvnwr") pod "0d55422b-84bf-4455-9112-94de20fb5b6d" (UID: "0d55422b-84bf-4455-9112-94de20fb5b6d"). InnerVolumeSpecName "kube-api-access-fvnwr". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:59:01 crc kubenswrapper[5052]: I1203 11:59:01.141752 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0d55422b-84bf-4455-9112-94de20fb5b6d-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "0d55422b-84bf-4455-9112-94de20fb5b6d" (UID: "0d55422b-84bf-4455-9112-94de20fb5b6d"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:59:01 crc kubenswrapper[5052]: I1203 11:59:01.144203 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0d55422b-84bf-4455-9112-94de20fb5b6d-config-data" (OuterVolumeSpecName: "config-data") pod "0d55422b-84bf-4455-9112-94de20fb5b6d" (UID: "0d55422b-84bf-4455-9112-94de20fb5b6d"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:59:01 crc kubenswrapper[5052]: I1203 11:59:01.179823 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0d55422b-84bf-4455-9112-94de20fb5b6d-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "0d55422b-84bf-4455-9112-94de20fb5b6d" (UID: "0d55422b-84bf-4455-9112-94de20fb5b6d"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:59:01 crc kubenswrapper[5052]: I1203 11:59:01.189424 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/0d55422b-84bf-4455-9112-94de20fb5b6d-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:59:01 crc kubenswrapper[5052]: I1203 11:59:01.189461 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-fvnwr\" (UniqueName: \"kubernetes.io/projected/0d55422b-84bf-4455-9112-94de20fb5b6d-kube-api-access-fvnwr\") on node \"crc\" DevicePath \"\"" Dec 03 11:59:01 crc kubenswrapper[5052]: I1203 11:59:01.189474 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0d55422b-84bf-4455-9112-94de20fb5b6d-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:59:01 crc kubenswrapper[5052]: I1203 11:59:01.189487 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/0d55422b-84bf-4455-9112-94de20fb5b6d-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:59:01 crc kubenswrapper[5052]: I1203 11:59:01.189498 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/0d55422b-84bf-4455-9112-94de20fb5b6d-logs\") on node \"crc\" DevicePath \"\"" Dec 03 11:59:01 crc kubenswrapper[5052]: I1203 11:59:01.189525 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/0d55422b-84bf-4455-9112-94de20fb5b6d-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 11:59:01 crc kubenswrapper[5052]: I1203 11:59:01.189556 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage15-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage15-crc\") on node \"crc\" " Dec 03 11:59:01 crc kubenswrapper[5052]: I1203 11:59:01.223025 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage15-crc" (UniqueName: "kubernetes.io/local-volume/local-storage15-crc") on node "crc" Dec 03 11:59:01 crc kubenswrapper[5052]: I1203 11:59:01.291096 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage15-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage15-crc\") on node \"crc\" DevicePath \"\"" Dec 03 11:59:01 crc kubenswrapper[5052]: I1203 11:59:01.658842 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"e8e69500-b31c-4293-8b70-e20b3fb0a92c","Type":"ContainerStarted","Data":"44de97a9e74496203efa484b712873c64b1a3400a74757ddf430d7d82c83f0b7"} Dec 03 11:59:01 crc kubenswrapper[5052]: I1203 11:59:01.659094 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"e8e69500-b31c-4293-8b70-e20b3fb0a92c","Type":"ContainerStarted","Data":"9f5998f5c9a2b271bacf7425c810412451fff65af895183f6904084852dcbd2b"} Dec 03 11:59:01 crc kubenswrapper[5052]: I1203 11:59:01.661710 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"0d55422b-84bf-4455-9112-94de20fb5b6d","Type":"ContainerDied","Data":"8212e962531fdda3bed98c5640eba93c6a37f64fcef5844606b8b236e6aadac4"} Dec 03 11:59:01 crc kubenswrapper[5052]: I1203 11:59:01.661756 5052 scope.go:117] "RemoveContainer" containerID="2bf75f1bec647492a24e9e2e2a4b240c1dcac0144cf5a6f394ba143a1374064f" Dec 03 11:59:01 crc kubenswrapper[5052]: I1203 11:59:01.661904 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:59:01 crc kubenswrapper[5052]: I1203 11:59:01.668930 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"51a9dfff-37f2-4fbe-b2d1-039f20bb4560","Type":"ContainerStarted","Data":"78d5d43de3a6f7f3e9852c4596ef7b083a06e8b8627e91c62a5f728ddf8c08f0"} Dec 03 11:59:01 crc kubenswrapper[5052]: I1203 11:59:01.691069 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="97230225-f283-4927-a24c-78fece560796" path="/var/lib/kubelet/pods/97230225-f283-4927-a24c-78fece560796/volumes" Dec 03 11:59:01 crc kubenswrapper[5052]: I1203 11:59:01.691526 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9fe1f02f-d2cf-4220-925c-6ecdcd099048" path="/var/lib/kubelet/pods/9fe1f02f-d2cf-4220-925c-6ecdcd099048/volumes" Dec 03 11:59:01 crc kubenswrapper[5052]: I1203 11:59:01.716813 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 11:59:01 crc kubenswrapper[5052]: I1203 11:59:01.725697 5052 scope.go:117] "RemoveContainer" containerID="f3557803e708885b7be19d0c0a7d6e82313f419fd68f6cf32092dc379a16b9af" Dec 03 11:59:01 crc kubenswrapper[5052]: I1203 11:59:01.735718 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 11:59:01 crc kubenswrapper[5052]: I1203 11:59:01.757116 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 11:59:01 crc kubenswrapper[5052]: E1203 11:59:01.757544 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="0d55422b-84bf-4455-9112-94de20fb5b6d" containerName="glance-log" Dec 03 11:59:01 crc kubenswrapper[5052]: I1203 11:59:01.757560 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="0d55422b-84bf-4455-9112-94de20fb5b6d" containerName="glance-log" Dec 03 11:59:01 crc kubenswrapper[5052]: E1203 11:59:01.757575 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="0d55422b-84bf-4455-9112-94de20fb5b6d" containerName="glance-httpd" Dec 03 11:59:01 crc kubenswrapper[5052]: I1203 11:59:01.757583 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="0d55422b-84bf-4455-9112-94de20fb5b6d" containerName="glance-httpd" Dec 03 11:59:01 crc kubenswrapper[5052]: I1203 11:59:01.757748 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="0d55422b-84bf-4455-9112-94de20fb5b6d" containerName="glance-log" Dec 03 11:59:01 crc kubenswrapper[5052]: I1203 11:59:01.757778 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="0d55422b-84bf-4455-9112-94de20fb5b6d" containerName="glance-httpd" Dec 03 11:59:01 crc kubenswrapper[5052]: I1203 11:59:01.762136 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:59:01 crc kubenswrapper[5052]: I1203 11:59:01.772415 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 11:59:01 crc kubenswrapper[5052]: I1203 11:59:01.808242 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-glance-default-internal-svc" Dec 03 11:59:01 crc kubenswrapper[5052]: I1203 11:59:01.808430 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-default-internal-config-data" Dec 03 11:59:01 crc kubenswrapper[5052]: I1203 11:59:01.911336 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/6de7dc2a-f19c-4cc3-9965-9be822dc6270-internal-tls-certs\") pod \"glance-default-internal-api-0\" (UID: \"6de7dc2a-f19c-4cc3-9965-9be822dc6270\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:59:01 crc kubenswrapper[5052]: I1203 11:59:01.911382 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/6de7dc2a-f19c-4cc3-9965-9be822dc6270-httpd-run\") pod \"glance-default-internal-api-0\" (UID: \"6de7dc2a-f19c-4cc3-9965-9be822dc6270\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:59:01 crc kubenswrapper[5052]: I1203 11:59:01.911412 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c4j2x\" (UniqueName: \"kubernetes.io/projected/6de7dc2a-f19c-4cc3-9965-9be822dc6270-kube-api-access-c4j2x\") pod \"glance-default-internal-api-0\" (UID: \"6de7dc2a-f19c-4cc3-9965-9be822dc6270\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:59:01 crc kubenswrapper[5052]: I1203 11:59:01.911569 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/6de7dc2a-f19c-4cc3-9965-9be822dc6270-scripts\") pod \"glance-default-internal-api-0\" (UID: \"6de7dc2a-f19c-4cc3-9965-9be822dc6270\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:59:01 crc kubenswrapper[5052]: I1203 11:59:01.911675 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6de7dc2a-f19c-4cc3-9965-9be822dc6270-combined-ca-bundle\") pod \"glance-default-internal-api-0\" (UID: \"6de7dc2a-f19c-4cc3-9965-9be822dc6270\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:59:01 crc kubenswrapper[5052]: I1203 11:59:01.911721 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6de7dc2a-f19c-4cc3-9965-9be822dc6270-config-data\") pod \"glance-default-internal-api-0\" (UID: \"6de7dc2a-f19c-4cc3-9965-9be822dc6270\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:59:01 crc kubenswrapper[5052]: I1203 11:59:01.911819 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/6de7dc2a-f19c-4cc3-9965-9be822dc6270-logs\") pod \"glance-default-internal-api-0\" (UID: \"6de7dc2a-f19c-4cc3-9965-9be822dc6270\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:59:01 crc kubenswrapper[5052]: I1203 11:59:01.911864 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage15-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage15-crc\") pod \"glance-default-internal-api-0\" (UID: \"6de7dc2a-f19c-4cc3-9965-9be822dc6270\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:59:02 crc kubenswrapper[5052]: I1203 11:59:02.013432 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6de7dc2a-f19c-4cc3-9965-9be822dc6270-config-data\") pod \"glance-default-internal-api-0\" (UID: \"6de7dc2a-f19c-4cc3-9965-9be822dc6270\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:59:02 crc kubenswrapper[5052]: I1203 11:59:02.013559 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/6de7dc2a-f19c-4cc3-9965-9be822dc6270-logs\") pod \"glance-default-internal-api-0\" (UID: \"6de7dc2a-f19c-4cc3-9965-9be822dc6270\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:59:02 crc kubenswrapper[5052]: I1203 11:59:02.013601 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage15-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage15-crc\") pod \"glance-default-internal-api-0\" (UID: \"6de7dc2a-f19c-4cc3-9965-9be822dc6270\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:59:02 crc kubenswrapper[5052]: I1203 11:59:02.013665 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/6de7dc2a-f19c-4cc3-9965-9be822dc6270-internal-tls-certs\") pod \"glance-default-internal-api-0\" (UID: \"6de7dc2a-f19c-4cc3-9965-9be822dc6270\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:59:02 crc kubenswrapper[5052]: I1203 11:59:02.013699 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/6de7dc2a-f19c-4cc3-9965-9be822dc6270-httpd-run\") pod \"glance-default-internal-api-0\" (UID: \"6de7dc2a-f19c-4cc3-9965-9be822dc6270\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:59:02 crc kubenswrapper[5052]: I1203 11:59:02.013737 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-c4j2x\" (UniqueName: \"kubernetes.io/projected/6de7dc2a-f19c-4cc3-9965-9be822dc6270-kube-api-access-c4j2x\") pod \"glance-default-internal-api-0\" (UID: \"6de7dc2a-f19c-4cc3-9965-9be822dc6270\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:59:02 crc kubenswrapper[5052]: I1203 11:59:02.013803 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/6de7dc2a-f19c-4cc3-9965-9be822dc6270-scripts\") pod \"glance-default-internal-api-0\" (UID: \"6de7dc2a-f19c-4cc3-9965-9be822dc6270\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:59:02 crc kubenswrapper[5052]: I1203 11:59:02.013864 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6de7dc2a-f19c-4cc3-9965-9be822dc6270-combined-ca-bundle\") pod \"glance-default-internal-api-0\" (UID: \"6de7dc2a-f19c-4cc3-9965-9be822dc6270\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:59:02 crc kubenswrapper[5052]: I1203 11:59:02.015116 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/6de7dc2a-f19c-4cc3-9965-9be822dc6270-logs\") pod \"glance-default-internal-api-0\" (UID: \"6de7dc2a-f19c-4cc3-9965-9be822dc6270\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:59:02 crc kubenswrapper[5052]: I1203 11:59:02.018430 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6de7dc2a-f19c-4cc3-9965-9be822dc6270-combined-ca-bundle\") pod \"glance-default-internal-api-0\" (UID: \"6de7dc2a-f19c-4cc3-9965-9be822dc6270\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:59:02 crc kubenswrapper[5052]: I1203 11:59:02.018915 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/6de7dc2a-f19c-4cc3-9965-9be822dc6270-httpd-run\") pod \"glance-default-internal-api-0\" (UID: \"6de7dc2a-f19c-4cc3-9965-9be822dc6270\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:59:02 crc kubenswrapper[5052]: I1203 11:59:02.019658 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage15-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage15-crc\") pod \"glance-default-internal-api-0\" (UID: \"6de7dc2a-f19c-4cc3-9965-9be822dc6270\") device mount path \"/mnt/openstack/pv15\"" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:59:02 crc kubenswrapper[5052]: I1203 11:59:02.019741 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6de7dc2a-f19c-4cc3-9965-9be822dc6270-config-data\") pod \"glance-default-internal-api-0\" (UID: \"6de7dc2a-f19c-4cc3-9965-9be822dc6270\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:59:02 crc kubenswrapper[5052]: I1203 11:59:02.023436 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/6de7dc2a-f19c-4cc3-9965-9be822dc6270-internal-tls-certs\") pod \"glance-default-internal-api-0\" (UID: \"6de7dc2a-f19c-4cc3-9965-9be822dc6270\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:59:02 crc kubenswrapper[5052]: I1203 11:59:02.029263 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/6de7dc2a-f19c-4cc3-9965-9be822dc6270-scripts\") pod \"glance-default-internal-api-0\" (UID: \"6de7dc2a-f19c-4cc3-9965-9be822dc6270\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:59:02 crc kubenswrapper[5052]: I1203 11:59:02.043793 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-c4j2x\" (UniqueName: \"kubernetes.io/projected/6de7dc2a-f19c-4cc3-9965-9be822dc6270-kube-api-access-c4j2x\") pod \"glance-default-internal-api-0\" (UID: \"6de7dc2a-f19c-4cc3-9965-9be822dc6270\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:59:02 crc kubenswrapper[5052]: I1203 11:59:02.055095 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage15-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage15-crc\") pod \"glance-default-internal-api-0\" (UID: \"6de7dc2a-f19c-4cc3-9965-9be822dc6270\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:59:02 crc kubenswrapper[5052]: I1203 11:59:02.143986 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:59:02 crc kubenswrapper[5052]: I1203 11:59:02.617307 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 11:59:02 crc kubenswrapper[5052]: W1203 11:59:02.619765 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6de7dc2a_f19c_4cc3_9965_9be822dc6270.slice/crio-2957fee30e0d0f082a9b76698b455e2162249f7b4cff38dcfa9f76793e9ba6d2 WatchSource:0}: Error finding container 2957fee30e0d0f082a9b76698b455e2162249f7b4cff38dcfa9f76793e9ba6d2: Status 404 returned error can't find the container with id 2957fee30e0d0f082a9b76698b455e2162249f7b4cff38dcfa9f76793e9ba6d2 Dec 03 11:59:02 crc kubenswrapper[5052]: I1203 11:59:02.712098 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"6de7dc2a-f19c-4cc3-9965-9be822dc6270","Type":"ContainerStarted","Data":"2957fee30e0d0f082a9b76698b455e2162249f7b4cff38dcfa9f76793e9ba6d2"} Dec 03 11:59:02 crc kubenswrapper[5052]: I1203 11:59:02.721085 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"51a9dfff-37f2-4fbe-b2d1-039f20bb4560","Type":"ContainerStarted","Data":"7f62ed7588549b9a67859ad18997d03e123b56ebc6c553d299ea1f897cb73ce2"} Dec 03 11:59:03 crc kubenswrapper[5052]: I1203 11:59:03.212757 5052 provider.go:102] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Dec 03 11:59:03 crc kubenswrapper[5052]: I1203 11:59:03.685186 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0d55422b-84bf-4455-9112-94de20fb5b6d" path="/var/lib/kubelet/pods/0d55422b-84bf-4455-9112-94de20fb5b6d/volumes" Dec 03 11:59:03 crc kubenswrapper[5052]: I1203 11:59:03.730816 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"51a9dfff-37f2-4fbe-b2d1-039f20bb4560","Type":"ContainerStarted","Data":"87bc47eac41558bc881491bf1b0295423263bce8a03bd87436868096e25a318d"} Dec 03 11:59:03 crc kubenswrapper[5052]: I1203 11:59:03.733305 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"e8e69500-b31c-4293-8b70-e20b3fb0a92c","Type":"ContainerStarted","Data":"cf52355e58c670eaf6324032b2f4213330743eac60b123e414c885d69d5b0420"} Dec 03 11:59:03 crc kubenswrapper[5052]: I1203 11:59:03.734726 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"6de7dc2a-f19c-4cc3-9965-9be822dc6270","Type":"ContainerStarted","Data":"220ad427ac37b8c14a8f2bf882e72550b22b51dfe6dbd65908d5362fb9eed124"} Dec 03 11:59:03 crc kubenswrapper[5052]: I1203 11:59:03.760093 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/glance-default-external-api-0" podStartSLOduration=4.760075194 podStartE2EDuration="4.760075194s" podCreationTimestamp="2025-12-03 11:58:59 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:59:03.751031405 +0000 UTC m=+2334.339273169" watchObservedRunningTime="2025-12-03 11:59:03.760075194 +0000 UTC m=+2334.348316958" Dec 03 11:59:04 crc kubenswrapper[5052]: I1203 11:59:04.746867 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"6de7dc2a-f19c-4cc3-9965-9be822dc6270","Type":"ContainerStarted","Data":"f562bb4258333809d9cdbdea6ecb675c5df538644d028c44e7b8954cc6a364c8"} Dec 03 11:59:04 crc kubenswrapper[5052]: I1203 11:59:04.750135 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"e8e69500-b31c-4293-8b70-e20b3fb0a92c","Type":"ContainerStarted","Data":"428e88e71fdc36c3129ac1afc53bfd80adfb34cacd60ce16e1c866e0ce51fbf5"} Dec 03 11:59:04 crc kubenswrapper[5052]: I1203 11:59:04.775202 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/glance-default-internal-api-0" podStartSLOduration=3.775175945 podStartE2EDuration="3.775175945s" podCreationTimestamp="2025-12-03 11:59:01 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:59:04.769812691 +0000 UTC m=+2335.358054465" watchObservedRunningTime="2025-12-03 11:59:04.775175945 +0000 UTC m=+2335.363417709" Dec 03 11:59:05 crc kubenswrapper[5052]: I1203 11:59:05.764944 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"e8e69500-b31c-4293-8b70-e20b3fb0a92c","Type":"ContainerStarted","Data":"f4628a3ce73272161479e35a9010da2fc0ac5c370bbbaa08d8f4d1dbeee1708f"} Dec 03 11:59:05 crc kubenswrapper[5052]: I1203 11:59:05.765378 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="e8e69500-b31c-4293-8b70-e20b3fb0a92c" containerName="ceilometer-central-agent" containerID="cri-o://44de97a9e74496203efa484b712873c64b1a3400a74757ddf430d7d82c83f0b7" gracePeriod=30 Dec 03 11:59:05 crc kubenswrapper[5052]: I1203 11:59:05.765827 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="e8e69500-b31c-4293-8b70-e20b3fb0a92c" containerName="proxy-httpd" containerID="cri-o://f4628a3ce73272161479e35a9010da2fc0ac5c370bbbaa08d8f4d1dbeee1708f" gracePeriod=30 Dec 03 11:59:05 crc kubenswrapper[5052]: I1203 11:59:05.765892 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="e8e69500-b31c-4293-8b70-e20b3fb0a92c" containerName="sg-core" containerID="cri-o://428e88e71fdc36c3129ac1afc53bfd80adfb34cacd60ce16e1c866e0ce51fbf5" gracePeriod=30 Dec 03 11:59:05 crc kubenswrapper[5052]: I1203 11:59:05.765961 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="e8e69500-b31c-4293-8b70-e20b3fb0a92c" containerName="ceilometer-notification-agent" containerID="cri-o://cf52355e58c670eaf6324032b2f4213330743eac60b123e414c885d69d5b0420" gracePeriod=30 Dec 03 11:59:05 crc kubenswrapper[5052]: I1203 11:59:05.791348 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/ceilometer-0" podStartSLOduration=2.4536765689999998 podStartE2EDuration="6.791331066s" podCreationTimestamp="2025-12-03 11:58:59 +0000 UTC" firstStartedPulling="2025-12-03 11:59:00.802680518 +0000 UTC m=+2331.390922282" lastFinishedPulling="2025-12-03 11:59:05.140335015 +0000 UTC m=+2335.728576779" observedRunningTime="2025-12-03 11:59:05.785939992 +0000 UTC m=+2336.374181756" watchObservedRunningTime="2025-12-03 11:59:05.791331066 +0000 UTC m=+2336.379572830" Dec 03 11:59:06 crc kubenswrapper[5052]: I1203 11:59:06.776990 5052 generic.go:334] "Generic (PLEG): container finished" podID="e8e69500-b31c-4293-8b70-e20b3fb0a92c" containerID="f4628a3ce73272161479e35a9010da2fc0ac5c370bbbaa08d8f4d1dbeee1708f" exitCode=0 Dec 03 11:59:06 crc kubenswrapper[5052]: I1203 11:59:06.777023 5052 generic.go:334] "Generic (PLEG): container finished" podID="e8e69500-b31c-4293-8b70-e20b3fb0a92c" containerID="428e88e71fdc36c3129ac1afc53bfd80adfb34cacd60ce16e1c866e0ce51fbf5" exitCode=2 Dec 03 11:59:06 crc kubenswrapper[5052]: I1203 11:59:06.777055 5052 generic.go:334] "Generic (PLEG): container finished" podID="e8e69500-b31c-4293-8b70-e20b3fb0a92c" containerID="cf52355e58c670eaf6324032b2f4213330743eac60b123e414c885d69d5b0420" exitCode=0 Dec 03 11:59:06 crc kubenswrapper[5052]: I1203 11:59:06.777075 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"e8e69500-b31c-4293-8b70-e20b3fb0a92c","Type":"ContainerDied","Data":"f4628a3ce73272161479e35a9010da2fc0ac5c370bbbaa08d8f4d1dbeee1708f"} Dec 03 11:59:06 crc kubenswrapper[5052]: I1203 11:59:06.777102 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"e8e69500-b31c-4293-8b70-e20b3fb0a92c","Type":"ContainerDied","Data":"428e88e71fdc36c3129ac1afc53bfd80adfb34cacd60ce16e1c866e0ce51fbf5"} Dec 03 11:59:06 crc kubenswrapper[5052]: I1203 11:59:06.777111 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"e8e69500-b31c-4293-8b70-e20b3fb0a92c","Type":"ContainerDied","Data":"cf52355e58c670eaf6324032b2f4213330743eac60b123e414c885d69d5b0420"} Dec 03 11:59:07 crc kubenswrapper[5052]: I1203 11:59:07.735375 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/neutron-8966cf85c-7cmmp" Dec 03 11:59:08 crc kubenswrapper[5052]: E1203 11:59:08.760583 5052 cadvisor_stats_provider.go:516] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/system.slice/rpm-ostreed.service\": RecentStats: unable to find data in memory cache]" Dec 03 11:59:09 crc kubenswrapper[5052]: I1203 11:59:09.256467 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/watcher-decision-engine-0" Dec 03 11:59:09 crc kubenswrapper[5052]: I1203 11:59:09.298744 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/watcher-decision-engine-0" Dec 03 11:59:09 crc kubenswrapper[5052]: I1203 11:59:09.820701 5052 generic.go:334] "Generic (PLEG): container finished" podID="e8e69500-b31c-4293-8b70-e20b3fb0a92c" containerID="44de97a9e74496203efa484b712873c64b1a3400a74757ddf430d7d82c83f0b7" exitCode=0 Dec 03 11:59:09 crc kubenswrapper[5052]: I1203 11:59:09.821843 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"e8e69500-b31c-4293-8b70-e20b3fb0a92c","Type":"ContainerDied","Data":"44de97a9e74496203efa484b712873c64b1a3400a74757ddf430d7d82c83f0b7"} Dec 03 11:59:09 crc kubenswrapper[5052]: I1203 11:59:09.821883 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/watcher-decision-engine-0" Dec 03 11:59:09 crc kubenswrapper[5052]: I1203 11:59:09.852140 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/watcher-decision-engine-0" Dec 03 11:59:10 crc kubenswrapper[5052]: I1203 11:59:10.115847 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:59:10 crc kubenswrapper[5052]: I1203 11:59:10.178730 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/e8e69500-b31c-4293-8b70-e20b3fb0a92c-config-data\") pod \"e8e69500-b31c-4293-8b70-e20b3fb0a92c\" (UID: \"e8e69500-b31c-4293-8b70-e20b3fb0a92c\") " Dec 03 11:59:10 crc kubenswrapper[5052]: I1203 11:59:10.178814 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e8e69500-b31c-4293-8b70-e20b3fb0a92c-combined-ca-bundle\") pod \"e8e69500-b31c-4293-8b70-e20b3fb0a92c\" (UID: \"e8e69500-b31c-4293-8b70-e20b3fb0a92c\") " Dec 03 11:59:10 crc kubenswrapper[5052]: I1203 11:59:10.178873 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/e8e69500-b31c-4293-8b70-e20b3fb0a92c-log-httpd\") pod \"e8e69500-b31c-4293-8b70-e20b3fb0a92c\" (UID: \"e8e69500-b31c-4293-8b70-e20b3fb0a92c\") " Dec 03 11:59:10 crc kubenswrapper[5052]: I1203 11:59:10.178898 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jsk62\" (UniqueName: \"kubernetes.io/projected/e8e69500-b31c-4293-8b70-e20b3fb0a92c-kube-api-access-jsk62\") pod \"e8e69500-b31c-4293-8b70-e20b3fb0a92c\" (UID: \"e8e69500-b31c-4293-8b70-e20b3fb0a92c\") " Dec 03 11:59:10 crc kubenswrapper[5052]: I1203 11:59:10.178920 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/e8e69500-b31c-4293-8b70-e20b3fb0a92c-run-httpd\") pod \"e8e69500-b31c-4293-8b70-e20b3fb0a92c\" (UID: \"e8e69500-b31c-4293-8b70-e20b3fb0a92c\") " Dec 03 11:59:10 crc kubenswrapper[5052]: I1203 11:59:10.178938 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/e8e69500-b31c-4293-8b70-e20b3fb0a92c-scripts\") pod \"e8e69500-b31c-4293-8b70-e20b3fb0a92c\" (UID: \"e8e69500-b31c-4293-8b70-e20b3fb0a92c\") " Dec 03 11:59:10 crc kubenswrapper[5052]: I1203 11:59:10.179047 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/e8e69500-b31c-4293-8b70-e20b3fb0a92c-sg-core-conf-yaml\") pod \"e8e69500-b31c-4293-8b70-e20b3fb0a92c\" (UID: \"e8e69500-b31c-4293-8b70-e20b3fb0a92c\") " Dec 03 11:59:10 crc kubenswrapper[5052]: I1203 11:59:10.180735 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e8e69500-b31c-4293-8b70-e20b3fb0a92c-run-httpd" (OuterVolumeSpecName: "run-httpd") pod "e8e69500-b31c-4293-8b70-e20b3fb0a92c" (UID: "e8e69500-b31c-4293-8b70-e20b3fb0a92c"). InnerVolumeSpecName "run-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:59:10 crc kubenswrapper[5052]: I1203 11:59:10.181006 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e8e69500-b31c-4293-8b70-e20b3fb0a92c-log-httpd" (OuterVolumeSpecName: "log-httpd") pod "e8e69500-b31c-4293-8b70-e20b3fb0a92c" (UID: "e8e69500-b31c-4293-8b70-e20b3fb0a92c"). InnerVolumeSpecName "log-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:59:10 crc kubenswrapper[5052]: I1203 11:59:10.186634 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e8e69500-b31c-4293-8b70-e20b3fb0a92c-scripts" (OuterVolumeSpecName: "scripts") pod "e8e69500-b31c-4293-8b70-e20b3fb0a92c" (UID: "e8e69500-b31c-4293-8b70-e20b3fb0a92c"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:59:10 crc kubenswrapper[5052]: I1203 11:59:10.195281 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e8e69500-b31c-4293-8b70-e20b3fb0a92c-kube-api-access-jsk62" (OuterVolumeSpecName: "kube-api-access-jsk62") pod "e8e69500-b31c-4293-8b70-e20b3fb0a92c" (UID: "e8e69500-b31c-4293-8b70-e20b3fb0a92c"). InnerVolumeSpecName "kube-api-access-jsk62". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:59:10 crc kubenswrapper[5052]: I1203 11:59:10.208405 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e8e69500-b31c-4293-8b70-e20b3fb0a92c-sg-core-conf-yaml" (OuterVolumeSpecName: "sg-core-conf-yaml") pod "e8e69500-b31c-4293-8b70-e20b3fb0a92c" (UID: "e8e69500-b31c-4293-8b70-e20b3fb0a92c"). InnerVolumeSpecName "sg-core-conf-yaml". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:59:10 crc kubenswrapper[5052]: I1203 11:59:10.280900 5052 reconciler_common.go:293] "Volume detached for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/e8e69500-b31c-4293-8b70-e20b3fb0a92c-sg-core-conf-yaml\") on node \"crc\" DevicePath \"\"" Dec 03 11:59:10 crc kubenswrapper[5052]: I1203 11:59:10.280948 5052 reconciler_common.go:293] "Volume detached for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/e8e69500-b31c-4293-8b70-e20b3fb0a92c-log-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 11:59:10 crc kubenswrapper[5052]: I1203 11:59:10.280962 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-jsk62\" (UniqueName: \"kubernetes.io/projected/e8e69500-b31c-4293-8b70-e20b3fb0a92c-kube-api-access-jsk62\") on node \"crc\" DevicePath \"\"" Dec 03 11:59:10 crc kubenswrapper[5052]: I1203 11:59:10.280975 5052 reconciler_common.go:293] "Volume detached for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/e8e69500-b31c-4293-8b70-e20b3fb0a92c-run-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 11:59:10 crc kubenswrapper[5052]: I1203 11:59:10.280987 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/e8e69500-b31c-4293-8b70-e20b3fb0a92c-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:59:10 crc kubenswrapper[5052]: I1203 11:59:10.285691 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e8e69500-b31c-4293-8b70-e20b3fb0a92c-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "e8e69500-b31c-4293-8b70-e20b3fb0a92c" (UID: "e8e69500-b31c-4293-8b70-e20b3fb0a92c"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:59:10 crc kubenswrapper[5052]: I1203 11:59:10.309348 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e8e69500-b31c-4293-8b70-e20b3fb0a92c-config-data" (OuterVolumeSpecName: "config-data") pod "e8e69500-b31c-4293-8b70-e20b3fb0a92c" (UID: "e8e69500-b31c-4293-8b70-e20b3fb0a92c"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:59:10 crc kubenswrapper[5052]: I1203 11:59:10.376291 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:59:10 crc kubenswrapper[5052]: I1203 11:59:10.376343 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:59:10 crc kubenswrapper[5052]: I1203 11:59:10.382727 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/e8e69500-b31c-4293-8b70-e20b3fb0a92c-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:59:10 crc kubenswrapper[5052]: I1203 11:59:10.382767 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e8e69500-b31c-4293-8b70-e20b3fb0a92c-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:59:10 crc kubenswrapper[5052]: I1203 11:59:10.407684 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:59:10 crc kubenswrapper[5052]: I1203 11:59:10.419116 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:59:10 crc kubenswrapper[5052]: I1203 11:59:10.652757 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/neutron-7f59bd7c8d-4jwhw" Dec 03 11:59:10 crc kubenswrapper[5052]: I1203 11:59:10.729022 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/neutron-8966cf85c-7cmmp"] Dec 03 11:59:10 crc kubenswrapper[5052]: I1203 11:59:10.729237 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/neutron-8966cf85c-7cmmp" podUID="6bb40332-be9f-4633-8cab-72971768cbba" containerName="neutron-api" containerID="cri-o://ec5e6c8abdbedd4339a7b82a497d3d7b854810af9a4c0415133053de14bb98fb" gracePeriod=30 Dec 03 11:59:10 crc kubenswrapper[5052]: I1203 11:59:10.729640 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/neutron-8966cf85c-7cmmp" podUID="6bb40332-be9f-4633-8cab-72971768cbba" containerName="neutron-httpd" containerID="cri-o://072116c6057db13f831617dbb142496bb56bb8d9006e2da94d6128552d93ae62" gracePeriod=30 Dec 03 11:59:10 crc kubenswrapper[5052]: I1203 11:59:10.835595 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:59:10 crc kubenswrapper[5052]: I1203 11:59:10.839881 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"e8e69500-b31c-4293-8b70-e20b3fb0a92c","Type":"ContainerDied","Data":"9f5998f5c9a2b271bacf7425c810412451fff65af895183f6904084852dcbd2b"} Dec 03 11:59:10 crc kubenswrapper[5052]: I1203 11:59:10.839958 5052 scope.go:117] "RemoveContainer" containerID="f4628a3ce73272161479e35a9010da2fc0ac5c370bbbaa08d8f4d1dbeee1708f" Dec 03 11:59:10 crc kubenswrapper[5052]: I1203 11:59:10.840424 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:59:10 crc kubenswrapper[5052]: I1203 11:59:10.840450 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:59:10 crc kubenswrapper[5052]: I1203 11:59:10.878792 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:59:10 crc kubenswrapper[5052]: I1203 11:59:10.884855 5052 scope.go:117] "RemoveContainer" containerID="428e88e71fdc36c3129ac1afc53bfd80adfb34cacd60ce16e1c866e0ce51fbf5" Dec 03 11:59:10 crc kubenswrapper[5052]: I1203 11:59:10.887982 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:59:10 crc kubenswrapper[5052]: I1203 11:59:10.918804 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:59:10 crc kubenswrapper[5052]: E1203 11:59:10.919710 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="e8e69500-b31c-4293-8b70-e20b3fb0a92c" containerName="ceilometer-central-agent" Dec 03 11:59:10 crc kubenswrapper[5052]: I1203 11:59:10.919732 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="e8e69500-b31c-4293-8b70-e20b3fb0a92c" containerName="ceilometer-central-agent" Dec 03 11:59:10 crc kubenswrapper[5052]: E1203 11:59:10.919746 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="e8e69500-b31c-4293-8b70-e20b3fb0a92c" containerName="sg-core" Dec 03 11:59:10 crc kubenswrapper[5052]: I1203 11:59:10.919754 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="e8e69500-b31c-4293-8b70-e20b3fb0a92c" containerName="sg-core" Dec 03 11:59:10 crc kubenswrapper[5052]: E1203 11:59:10.919796 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="e8e69500-b31c-4293-8b70-e20b3fb0a92c" containerName="proxy-httpd" Dec 03 11:59:10 crc kubenswrapper[5052]: I1203 11:59:10.919804 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="e8e69500-b31c-4293-8b70-e20b3fb0a92c" containerName="proxy-httpd" Dec 03 11:59:10 crc kubenswrapper[5052]: E1203 11:59:10.919824 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="e8e69500-b31c-4293-8b70-e20b3fb0a92c" containerName="ceilometer-notification-agent" Dec 03 11:59:10 crc kubenswrapper[5052]: I1203 11:59:10.919831 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="e8e69500-b31c-4293-8b70-e20b3fb0a92c" containerName="ceilometer-notification-agent" Dec 03 11:59:10 crc kubenswrapper[5052]: I1203 11:59:10.920047 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="e8e69500-b31c-4293-8b70-e20b3fb0a92c" containerName="ceilometer-notification-agent" Dec 03 11:59:10 crc kubenswrapper[5052]: I1203 11:59:10.920065 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="e8e69500-b31c-4293-8b70-e20b3fb0a92c" containerName="proxy-httpd" Dec 03 11:59:10 crc kubenswrapper[5052]: I1203 11:59:10.920082 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="e8e69500-b31c-4293-8b70-e20b3fb0a92c" containerName="ceilometer-central-agent" Dec 03 11:59:10 crc kubenswrapper[5052]: I1203 11:59:10.920090 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="e8e69500-b31c-4293-8b70-e20b3fb0a92c" containerName="sg-core" Dec 03 11:59:10 crc kubenswrapper[5052]: I1203 11:59:10.922309 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:59:10 crc kubenswrapper[5052]: I1203 11:59:10.925378 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-config-data" Dec 03 11:59:10 crc kubenswrapper[5052]: I1203 11:59:10.925786 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-scripts" Dec 03 11:59:10 crc kubenswrapper[5052]: I1203 11:59:10.946088 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:59:10 crc kubenswrapper[5052]: I1203 11:59:10.963974 5052 scope.go:117] "RemoveContainer" containerID="cf52355e58c670eaf6324032b2f4213330743eac60b123e414c885d69d5b0420" Dec 03 11:59:10 crc kubenswrapper[5052]: I1203 11:59:10.997124 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/366cc2b7-5223-45c0-b9dc-15e8975e60a7-log-httpd\") pod \"ceilometer-0\" (UID: \"366cc2b7-5223-45c0-b9dc-15e8975e60a7\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:59:10 crc kubenswrapper[5052]: I1203 11:59:10.997497 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/366cc2b7-5223-45c0-b9dc-15e8975e60a7-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"366cc2b7-5223-45c0-b9dc-15e8975e60a7\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:59:10 crc kubenswrapper[5052]: I1203 11:59:10.998056 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/366cc2b7-5223-45c0-b9dc-15e8975e60a7-run-httpd\") pod \"ceilometer-0\" (UID: \"366cc2b7-5223-45c0-b9dc-15e8975e60a7\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:59:10 crc kubenswrapper[5052]: I1203 11:59:10.998207 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6mznl\" (UniqueName: \"kubernetes.io/projected/366cc2b7-5223-45c0-b9dc-15e8975e60a7-kube-api-access-6mznl\") pod \"ceilometer-0\" (UID: \"366cc2b7-5223-45c0-b9dc-15e8975e60a7\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:59:10 crc kubenswrapper[5052]: I1203 11:59:10.998298 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/366cc2b7-5223-45c0-b9dc-15e8975e60a7-scripts\") pod \"ceilometer-0\" (UID: \"366cc2b7-5223-45c0-b9dc-15e8975e60a7\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:59:10 crc kubenswrapper[5052]: I1203 11:59:10.998404 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/366cc2b7-5223-45c0-b9dc-15e8975e60a7-config-data\") pod \"ceilometer-0\" (UID: \"366cc2b7-5223-45c0-b9dc-15e8975e60a7\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:59:10 crc kubenswrapper[5052]: I1203 11:59:10.998581 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/366cc2b7-5223-45c0-b9dc-15e8975e60a7-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"366cc2b7-5223-45c0-b9dc-15e8975e60a7\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:59:11 crc kubenswrapper[5052]: I1203 11:59:11.027371 5052 scope.go:117] "RemoveContainer" containerID="44de97a9e74496203efa484b712873c64b1a3400a74757ddf430d7d82c83f0b7" Dec 03 11:59:11 crc kubenswrapper[5052]: I1203 11:59:11.100011 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/366cc2b7-5223-45c0-b9dc-15e8975e60a7-log-httpd\") pod \"ceilometer-0\" (UID: \"366cc2b7-5223-45c0-b9dc-15e8975e60a7\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:59:11 crc kubenswrapper[5052]: I1203 11:59:11.100065 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/366cc2b7-5223-45c0-b9dc-15e8975e60a7-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"366cc2b7-5223-45c0-b9dc-15e8975e60a7\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:59:11 crc kubenswrapper[5052]: I1203 11:59:11.100094 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/366cc2b7-5223-45c0-b9dc-15e8975e60a7-run-httpd\") pod \"ceilometer-0\" (UID: \"366cc2b7-5223-45c0-b9dc-15e8975e60a7\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:59:11 crc kubenswrapper[5052]: I1203 11:59:11.100135 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-6mznl\" (UniqueName: \"kubernetes.io/projected/366cc2b7-5223-45c0-b9dc-15e8975e60a7-kube-api-access-6mznl\") pod \"ceilometer-0\" (UID: \"366cc2b7-5223-45c0-b9dc-15e8975e60a7\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:59:11 crc kubenswrapper[5052]: I1203 11:59:11.100158 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/366cc2b7-5223-45c0-b9dc-15e8975e60a7-scripts\") pod \"ceilometer-0\" (UID: \"366cc2b7-5223-45c0-b9dc-15e8975e60a7\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:59:11 crc kubenswrapper[5052]: I1203 11:59:11.100202 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/366cc2b7-5223-45c0-b9dc-15e8975e60a7-config-data\") pod \"ceilometer-0\" (UID: \"366cc2b7-5223-45c0-b9dc-15e8975e60a7\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:59:11 crc kubenswrapper[5052]: I1203 11:59:11.100260 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/366cc2b7-5223-45c0-b9dc-15e8975e60a7-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"366cc2b7-5223-45c0-b9dc-15e8975e60a7\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:59:11 crc kubenswrapper[5052]: I1203 11:59:11.100573 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/366cc2b7-5223-45c0-b9dc-15e8975e60a7-run-httpd\") pod \"ceilometer-0\" (UID: \"366cc2b7-5223-45c0-b9dc-15e8975e60a7\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:59:11 crc kubenswrapper[5052]: I1203 11:59:11.100588 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/366cc2b7-5223-45c0-b9dc-15e8975e60a7-log-httpd\") pod \"ceilometer-0\" (UID: \"366cc2b7-5223-45c0-b9dc-15e8975e60a7\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:59:11 crc kubenswrapper[5052]: I1203 11:59:11.104777 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/366cc2b7-5223-45c0-b9dc-15e8975e60a7-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"366cc2b7-5223-45c0-b9dc-15e8975e60a7\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:59:11 crc kubenswrapper[5052]: I1203 11:59:11.105040 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/366cc2b7-5223-45c0-b9dc-15e8975e60a7-config-data\") pod \"ceilometer-0\" (UID: \"366cc2b7-5223-45c0-b9dc-15e8975e60a7\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:59:11 crc kubenswrapper[5052]: I1203 11:59:11.106435 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/366cc2b7-5223-45c0-b9dc-15e8975e60a7-scripts\") pod \"ceilometer-0\" (UID: \"366cc2b7-5223-45c0-b9dc-15e8975e60a7\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:59:11 crc kubenswrapper[5052]: I1203 11:59:11.110350 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/366cc2b7-5223-45c0-b9dc-15e8975e60a7-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"366cc2b7-5223-45c0-b9dc-15e8975e60a7\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:59:11 crc kubenswrapper[5052]: I1203 11:59:11.122383 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-6mznl\" (UniqueName: \"kubernetes.io/projected/366cc2b7-5223-45c0-b9dc-15e8975e60a7-kube-api-access-6mznl\") pod \"ceilometer-0\" (UID: \"366cc2b7-5223-45c0-b9dc-15e8975e60a7\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:59:11 crc kubenswrapper[5052]: I1203 11:59:11.239850 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:59:11 crc kubenswrapper[5052]: I1203 11:59:11.684651 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e8e69500-b31c-4293-8b70-e20b3fb0a92c" path="/var/lib/kubelet/pods/e8e69500-b31c-4293-8b70-e20b3fb0a92c/volumes" Dec 03 11:59:11 crc kubenswrapper[5052]: I1203 11:59:11.717659 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:59:11 crc kubenswrapper[5052]: W1203 11:59:11.718900 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod366cc2b7_5223_45c0_b9dc_15e8975e60a7.slice/crio-b0b2b155c6fb3e1cb498acd403cfbbd8bf6cfde001e38128450ebb976d678715 WatchSource:0}: Error finding container b0b2b155c6fb3e1cb498acd403cfbbd8bf6cfde001e38128450ebb976d678715: Status 404 returned error can't find the container with id b0b2b155c6fb3e1cb498acd403cfbbd8bf6cfde001e38128450ebb976d678715 Dec 03 11:59:11 crc kubenswrapper[5052]: I1203 11:59:11.845072 5052 generic.go:334] "Generic (PLEG): container finished" podID="6bb40332-be9f-4633-8cab-72971768cbba" containerID="072116c6057db13f831617dbb142496bb56bb8d9006e2da94d6128552d93ae62" exitCode=0 Dec 03 11:59:11 crc kubenswrapper[5052]: I1203 11:59:11.845078 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-8966cf85c-7cmmp" event={"ID":"6bb40332-be9f-4633-8cab-72971768cbba","Type":"ContainerDied","Data":"072116c6057db13f831617dbb142496bb56bb8d9006e2da94d6128552d93ae62"} Dec 03 11:59:11 crc kubenswrapper[5052]: I1203 11:59:11.849044 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"366cc2b7-5223-45c0-b9dc-15e8975e60a7","Type":"ContainerStarted","Data":"b0b2b155c6fb3e1cb498acd403cfbbd8bf6cfde001e38128450ebb976d678715"} Dec 03 11:59:12 crc kubenswrapper[5052]: I1203 11:59:12.145288 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:59:12 crc kubenswrapper[5052]: I1203 11:59:12.146273 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:59:12 crc kubenswrapper[5052]: I1203 11:59:12.180654 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:59:12 crc kubenswrapper[5052]: I1203 11:59:12.195657 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:59:12 crc kubenswrapper[5052]: I1203 11:59:12.673715 5052 scope.go:117] "RemoveContainer" containerID="d224717c6d9ca8135fd510e03a6c40c9f2b1a56dd8b4a3b685af3e2dfa855982" Dec 03 11:59:12 crc kubenswrapper[5052]: E1203 11:59:12.674052 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 11:59:12 crc kubenswrapper[5052]: I1203 11:59:12.859571 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:59:12 crc kubenswrapper[5052]: I1203 11:59:12.859623 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:59:13 crc kubenswrapper[5052]: I1203 11:59:13.214821 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:59:13 crc kubenswrapper[5052]: I1203 11:59:13.215287 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 11:59:13 crc kubenswrapper[5052]: I1203 11:59:13.875250 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"366cc2b7-5223-45c0-b9dc-15e8975e60a7","Type":"ContainerStarted","Data":"351abba6c79b072d578f848106cf4ca4c9b9e1097e08602304846997d8bdebfb"} Dec 03 11:59:14 crc kubenswrapper[5052]: I1203 11:59:14.808971 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-api-db-create-5rwbf"] Dec 03 11:59:14 crc kubenswrapper[5052]: I1203 11:59:14.810469 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-db-create-5rwbf" Dec 03 11:59:14 crc kubenswrapper[5052]: I1203 11:59:14.828977 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-db-create-5rwbf"] Dec 03 11:59:14 crc kubenswrapper[5052]: I1203 11:59:14.890051 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"366cc2b7-5223-45c0-b9dc-15e8975e60a7","Type":"ContainerStarted","Data":"b4ff98aed209f866dc33b6fd55f0c211be929bf140a7fedb2c2d387b427f84e0"} Dec 03 11:59:14 crc kubenswrapper[5052]: I1203 11:59:14.928670 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-api-f429-account-create-update-9r5b7"] Dec 03 11:59:14 crc kubenswrapper[5052]: I1203 11:59:14.930165 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-f429-account-create-update-9r5b7" Dec 03 11:59:14 crc kubenswrapper[5052]: I1203 11:59:14.935221 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-api-db-secret" Dec 03 11:59:14 crc kubenswrapper[5052]: I1203 11:59:14.937866 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell0-db-create-62hhq"] Dec 03 11:59:14 crc kubenswrapper[5052]: I1203 11:59:14.939599 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-db-create-62hhq" Dec 03 11:59:14 crc kubenswrapper[5052]: I1203 11:59:14.956169 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-f429-account-create-update-9r5b7"] Dec 03 11:59:14 crc kubenswrapper[5052]: I1203 11:59:14.973495 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell0-db-create-62hhq"] Dec 03 11:59:14 crc kubenswrapper[5052]: I1203 11:59:14.981722 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/4e7bcb77-fa54-4f5b-bb6f-77c12d876b11-operator-scripts\") pod \"nova-api-db-create-5rwbf\" (UID: \"4e7bcb77-fa54-4f5b-bb6f-77c12d876b11\") " pod="openstack-kuttl-tests/nova-api-db-create-5rwbf" Dec 03 11:59:14 crc kubenswrapper[5052]: I1203 11:59:14.982126 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x4c76\" (UniqueName: \"kubernetes.io/projected/4e7bcb77-fa54-4f5b-bb6f-77c12d876b11-kube-api-access-x4c76\") pod \"nova-api-db-create-5rwbf\" (UID: \"4e7bcb77-fa54-4f5b-bb6f-77c12d876b11\") " pod="openstack-kuttl-tests/nova-api-db-create-5rwbf" Dec 03 11:59:15 crc kubenswrapper[5052]: I1203 11:59:15.029541 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell1-db-create-d4vgs"] Dec 03 11:59:15 crc kubenswrapper[5052]: I1203 11:59:15.031195 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-db-create-d4vgs" Dec 03 11:59:15 crc kubenswrapper[5052]: I1203 11:59:15.060261 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-db-create-d4vgs"] Dec 03 11:59:15 crc kubenswrapper[5052]: I1203 11:59:15.086696 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/4e7bcb77-fa54-4f5b-bb6f-77c12d876b11-operator-scripts\") pod \"nova-api-db-create-5rwbf\" (UID: \"4e7bcb77-fa54-4f5b-bb6f-77c12d876b11\") " pod="openstack-kuttl-tests/nova-api-db-create-5rwbf" Dec 03 11:59:15 crc kubenswrapper[5052]: I1203 11:59:15.086786 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p255z\" (UniqueName: \"kubernetes.io/projected/30aeb633-e745-42cd-8f1b-cf60ac77d7ba-kube-api-access-p255z\") pod \"nova-api-f429-account-create-update-9r5b7\" (UID: \"30aeb633-e745-42cd-8f1b-cf60ac77d7ba\") " pod="openstack-kuttl-tests/nova-api-f429-account-create-update-9r5b7" Dec 03 11:59:15 crc kubenswrapper[5052]: I1203 11:59:15.086819 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/30aeb633-e745-42cd-8f1b-cf60ac77d7ba-operator-scripts\") pod \"nova-api-f429-account-create-update-9r5b7\" (UID: \"30aeb633-e745-42cd-8f1b-cf60ac77d7ba\") " pod="openstack-kuttl-tests/nova-api-f429-account-create-update-9r5b7" Dec 03 11:59:15 crc kubenswrapper[5052]: I1203 11:59:15.086843 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-x4c76\" (UniqueName: \"kubernetes.io/projected/4e7bcb77-fa54-4f5b-bb6f-77c12d876b11-kube-api-access-x4c76\") pod \"nova-api-db-create-5rwbf\" (UID: \"4e7bcb77-fa54-4f5b-bb6f-77c12d876b11\") " pod="openstack-kuttl-tests/nova-api-db-create-5rwbf" Dec 03 11:59:15 crc kubenswrapper[5052]: I1203 11:59:15.086898 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pfckh\" (UniqueName: \"kubernetes.io/projected/10051460-11e9-4e59-93c6-669a4d43d908-kube-api-access-pfckh\") pod \"nova-cell0-db-create-62hhq\" (UID: \"10051460-11e9-4e59-93c6-669a4d43d908\") " pod="openstack-kuttl-tests/nova-cell0-db-create-62hhq" Dec 03 11:59:15 crc kubenswrapper[5052]: I1203 11:59:15.086938 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/10051460-11e9-4e59-93c6-669a4d43d908-operator-scripts\") pod \"nova-cell0-db-create-62hhq\" (UID: \"10051460-11e9-4e59-93c6-669a4d43d908\") " pod="openstack-kuttl-tests/nova-cell0-db-create-62hhq" Dec 03 11:59:15 crc kubenswrapper[5052]: I1203 11:59:15.087638 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/4e7bcb77-fa54-4f5b-bb6f-77c12d876b11-operator-scripts\") pod \"nova-api-db-create-5rwbf\" (UID: \"4e7bcb77-fa54-4f5b-bb6f-77c12d876b11\") " pod="openstack-kuttl-tests/nova-api-db-create-5rwbf" Dec 03 11:59:15 crc kubenswrapper[5052]: I1203 11:59:15.115898 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-x4c76\" (UniqueName: \"kubernetes.io/projected/4e7bcb77-fa54-4f5b-bb6f-77c12d876b11-kube-api-access-x4c76\") pod \"nova-api-db-create-5rwbf\" (UID: \"4e7bcb77-fa54-4f5b-bb6f-77c12d876b11\") " pod="openstack-kuttl-tests/nova-api-db-create-5rwbf" Dec 03 11:59:15 crc kubenswrapper[5052]: I1203 11:59:15.131926 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-db-create-5rwbf" Dec 03 11:59:15 crc kubenswrapper[5052]: I1203 11:59:15.149729 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell0-6be2-account-create-update-2hj8c"] Dec 03 11:59:15 crc kubenswrapper[5052]: I1203 11:59:15.151163 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-6be2-account-create-update-2hj8c" Dec 03 11:59:15 crc kubenswrapper[5052]: I1203 11:59:15.154850 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell0-db-secret" Dec 03 11:59:15 crc kubenswrapper[5052]: I1203 11:59:15.176445 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell0-6be2-account-create-update-2hj8c"] Dec 03 11:59:15 crc kubenswrapper[5052]: I1203 11:59:15.188447 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/2ddd27d6-ad35-4919-8fe5-17100335e14a-operator-scripts\") pod \"nova-cell1-db-create-d4vgs\" (UID: \"2ddd27d6-ad35-4919-8fe5-17100335e14a\") " pod="openstack-kuttl-tests/nova-cell1-db-create-d4vgs" Dec 03 11:59:15 crc kubenswrapper[5052]: I1203 11:59:15.188671 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-pfckh\" (UniqueName: \"kubernetes.io/projected/10051460-11e9-4e59-93c6-669a4d43d908-kube-api-access-pfckh\") pod \"nova-cell0-db-create-62hhq\" (UID: \"10051460-11e9-4e59-93c6-669a4d43d908\") " pod="openstack-kuttl-tests/nova-cell0-db-create-62hhq" Dec 03 11:59:15 crc kubenswrapper[5052]: I1203 11:59:15.188801 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vgbqj\" (UniqueName: \"kubernetes.io/projected/2ddd27d6-ad35-4919-8fe5-17100335e14a-kube-api-access-vgbqj\") pod \"nova-cell1-db-create-d4vgs\" (UID: \"2ddd27d6-ad35-4919-8fe5-17100335e14a\") " pod="openstack-kuttl-tests/nova-cell1-db-create-d4vgs" Dec 03 11:59:15 crc kubenswrapper[5052]: I1203 11:59:15.188937 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/10051460-11e9-4e59-93c6-669a4d43d908-operator-scripts\") pod \"nova-cell0-db-create-62hhq\" (UID: \"10051460-11e9-4e59-93c6-669a4d43d908\") " pod="openstack-kuttl-tests/nova-cell0-db-create-62hhq" Dec 03 11:59:15 crc kubenswrapper[5052]: I1203 11:59:15.189160 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-p255z\" (UniqueName: \"kubernetes.io/projected/30aeb633-e745-42cd-8f1b-cf60ac77d7ba-kube-api-access-p255z\") pod \"nova-api-f429-account-create-update-9r5b7\" (UID: \"30aeb633-e745-42cd-8f1b-cf60ac77d7ba\") " pod="openstack-kuttl-tests/nova-api-f429-account-create-update-9r5b7" Dec 03 11:59:15 crc kubenswrapper[5052]: I1203 11:59:15.189278 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/30aeb633-e745-42cd-8f1b-cf60ac77d7ba-operator-scripts\") pod \"nova-api-f429-account-create-update-9r5b7\" (UID: \"30aeb633-e745-42cd-8f1b-cf60ac77d7ba\") " pod="openstack-kuttl-tests/nova-api-f429-account-create-update-9r5b7" Dec 03 11:59:15 crc kubenswrapper[5052]: I1203 11:59:15.190235 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/30aeb633-e745-42cd-8f1b-cf60ac77d7ba-operator-scripts\") pod \"nova-api-f429-account-create-update-9r5b7\" (UID: \"30aeb633-e745-42cd-8f1b-cf60ac77d7ba\") " pod="openstack-kuttl-tests/nova-api-f429-account-create-update-9r5b7" Dec 03 11:59:15 crc kubenswrapper[5052]: I1203 11:59:15.190235 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/10051460-11e9-4e59-93c6-669a4d43d908-operator-scripts\") pod \"nova-cell0-db-create-62hhq\" (UID: \"10051460-11e9-4e59-93c6-669a4d43d908\") " pod="openstack-kuttl-tests/nova-cell0-db-create-62hhq" Dec 03 11:59:15 crc kubenswrapper[5052]: I1203 11:59:15.218737 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-pfckh\" (UniqueName: \"kubernetes.io/projected/10051460-11e9-4e59-93c6-669a4d43d908-kube-api-access-pfckh\") pod \"nova-cell0-db-create-62hhq\" (UID: \"10051460-11e9-4e59-93c6-669a4d43d908\") " pod="openstack-kuttl-tests/nova-cell0-db-create-62hhq" Dec 03 11:59:15 crc kubenswrapper[5052]: I1203 11:59:15.226150 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-p255z\" (UniqueName: \"kubernetes.io/projected/30aeb633-e745-42cd-8f1b-cf60ac77d7ba-kube-api-access-p255z\") pod \"nova-api-f429-account-create-update-9r5b7\" (UID: \"30aeb633-e745-42cd-8f1b-cf60ac77d7ba\") " pod="openstack-kuttl-tests/nova-api-f429-account-create-update-9r5b7" Dec 03 11:59:15 crc kubenswrapper[5052]: I1203 11:59:15.249137 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-f429-account-create-update-9r5b7" Dec 03 11:59:15 crc kubenswrapper[5052]: I1203 11:59:15.264378 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-db-create-62hhq" Dec 03 11:59:15 crc kubenswrapper[5052]: I1203 11:59:15.291106 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t764p\" (UniqueName: \"kubernetes.io/projected/15926354-9368-4283-8452-89eed0479195-kube-api-access-t764p\") pod \"nova-cell0-6be2-account-create-update-2hj8c\" (UID: \"15926354-9368-4283-8452-89eed0479195\") " pod="openstack-kuttl-tests/nova-cell0-6be2-account-create-update-2hj8c" Dec 03 11:59:15 crc kubenswrapper[5052]: I1203 11:59:15.291250 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/2ddd27d6-ad35-4919-8fe5-17100335e14a-operator-scripts\") pod \"nova-cell1-db-create-d4vgs\" (UID: \"2ddd27d6-ad35-4919-8fe5-17100335e14a\") " pod="openstack-kuttl-tests/nova-cell1-db-create-d4vgs" Dec 03 11:59:15 crc kubenswrapper[5052]: I1203 11:59:15.291315 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/15926354-9368-4283-8452-89eed0479195-operator-scripts\") pod \"nova-cell0-6be2-account-create-update-2hj8c\" (UID: \"15926354-9368-4283-8452-89eed0479195\") " pod="openstack-kuttl-tests/nova-cell0-6be2-account-create-update-2hj8c" Dec 03 11:59:15 crc kubenswrapper[5052]: I1203 11:59:15.291352 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-vgbqj\" (UniqueName: \"kubernetes.io/projected/2ddd27d6-ad35-4919-8fe5-17100335e14a-kube-api-access-vgbqj\") pod \"nova-cell1-db-create-d4vgs\" (UID: \"2ddd27d6-ad35-4919-8fe5-17100335e14a\") " pod="openstack-kuttl-tests/nova-cell1-db-create-d4vgs" Dec 03 11:59:15 crc kubenswrapper[5052]: I1203 11:59:15.292779 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/2ddd27d6-ad35-4919-8fe5-17100335e14a-operator-scripts\") pod \"nova-cell1-db-create-d4vgs\" (UID: \"2ddd27d6-ad35-4919-8fe5-17100335e14a\") " pod="openstack-kuttl-tests/nova-cell1-db-create-d4vgs" Dec 03 11:59:15 crc kubenswrapper[5052]: I1203 11:59:15.319651 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-vgbqj\" (UniqueName: \"kubernetes.io/projected/2ddd27d6-ad35-4919-8fe5-17100335e14a-kube-api-access-vgbqj\") pod \"nova-cell1-db-create-d4vgs\" (UID: \"2ddd27d6-ad35-4919-8fe5-17100335e14a\") " pod="openstack-kuttl-tests/nova-cell1-db-create-d4vgs" Dec 03 11:59:15 crc kubenswrapper[5052]: I1203 11:59:15.345421 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell1-84d8-account-create-update-d7f29"] Dec 03 11:59:15 crc kubenswrapper[5052]: I1203 11:59:15.347038 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-84d8-account-create-update-d7f29" Dec 03 11:59:15 crc kubenswrapper[5052]: I1203 11:59:15.347179 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-db-create-d4vgs" Dec 03 11:59:15 crc kubenswrapper[5052]: I1203 11:59:15.364004 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell1-db-secret" Dec 03 11:59:15 crc kubenswrapper[5052]: I1203 11:59:15.393143 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-84d8-account-create-update-d7f29"] Dec 03 11:59:15 crc kubenswrapper[5052]: I1203 11:59:15.411134 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-t764p\" (UniqueName: \"kubernetes.io/projected/15926354-9368-4283-8452-89eed0479195-kube-api-access-t764p\") pod \"nova-cell0-6be2-account-create-update-2hj8c\" (UID: \"15926354-9368-4283-8452-89eed0479195\") " pod="openstack-kuttl-tests/nova-cell0-6be2-account-create-update-2hj8c" Dec 03 11:59:15 crc kubenswrapper[5052]: I1203 11:59:15.411310 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/15926354-9368-4283-8452-89eed0479195-operator-scripts\") pod \"nova-cell0-6be2-account-create-update-2hj8c\" (UID: \"15926354-9368-4283-8452-89eed0479195\") " pod="openstack-kuttl-tests/nova-cell0-6be2-account-create-update-2hj8c" Dec 03 11:59:15 crc kubenswrapper[5052]: I1203 11:59:15.412243 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/15926354-9368-4283-8452-89eed0479195-operator-scripts\") pod \"nova-cell0-6be2-account-create-update-2hj8c\" (UID: \"15926354-9368-4283-8452-89eed0479195\") " pod="openstack-kuttl-tests/nova-cell0-6be2-account-create-update-2hj8c" Dec 03 11:59:15 crc kubenswrapper[5052]: I1203 11:59:15.412413 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:59:15 crc kubenswrapper[5052]: I1203 11:59:15.427121 5052 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Dec 03 11:59:15 crc kubenswrapper[5052]: I1203 11:59:15.450230 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-t764p\" (UniqueName: \"kubernetes.io/projected/15926354-9368-4283-8452-89eed0479195-kube-api-access-t764p\") pod \"nova-cell0-6be2-account-create-update-2hj8c\" (UID: \"15926354-9368-4283-8452-89eed0479195\") " pod="openstack-kuttl-tests/nova-cell0-6be2-account-create-update-2hj8c" Dec 03 11:59:15 crc kubenswrapper[5052]: I1203 11:59:15.456047 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-6be2-account-create-update-2hj8c" Dec 03 11:59:15 crc kubenswrapper[5052]: I1203 11:59:15.513346 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/5e3ad34f-c054-4dfd-a21e-f6a5d91d820f-operator-scripts\") pod \"nova-cell1-84d8-account-create-update-d7f29\" (UID: \"5e3ad34f-c054-4dfd-a21e-f6a5d91d820f\") " pod="openstack-kuttl-tests/nova-cell1-84d8-account-create-update-d7f29" Dec 03 11:59:15 crc kubenswrapper[5052]: I1203 11:59:15.513414 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qzxbz\" (UniqueName: \"kubernetes.io/projected/5e3ad34f-c054-4dfd-a21e-f6a5d91d820f-kube-api-access-qzxbz\") pod \"nova-cell1-84d8-account-create-update-d7f29\" (UID: \"5e3ad34f-c054-4dfd-a21e-f6a5d91d820f\") " pod="openstack-kuttl-tests/nova-cell1-84d8-account-create-update-d7f29" Dec 03 11:59:15 crc kubenswrapper[5052]: I1203 11:59:15.617021 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/5e3ad34f-c054-4dfd-a21e-f6a5d91d820f-operator-scripts\") pod \"nova-cell1-84d8-account-create-update-d7f29\" (UID: \"5e3ad34f-c054-4dfd-a21e-f6a5d91d820f\") " pod="openstack-kuttl-tests/nova-cell1-84d8-account-create-update-d7f29" Dec 03 11:59:15 crc kubenswrapper[5052]: I1203 11:59:15.617315 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-qzxbz\" (UniqueName: \"kubernetes.io/projected/5e3ad34f-c054-4dfd-a21e-f6a5d91d820f-kube-api-access-qzxbz\") pod \"nova-cell1-84d8-account-create-update-d7f29\" (UID: \"5e3ad34f-c054-4dfd-a21e-f6a5d91d820f\") " pod="openstack-kuttl-tests/nova-cell1-84d8-account-create-update-d7f29" Dec 03 11:59:15 crc kubenswrapper[5052]: I1203 11:59:15.621049 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/5e3ad34f-c054-4dfd-a21e-f6a5d91d820f-operator-scripts\") pod \"nova-cell1-84d8-account-create-update-d7f29\" (UID: \"5e3ad34f-c054-4dfd-a21e-f6a5d91d820f\") " pod="openstack-kuttl-tests/nova-cell1-84d8-account-create-update-d7f29" Dec 03 11:59:15 crc kubenswrapper[5052]: I1203 11:59:15.659078 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-qzxbz\" (UniqueName: \"kubernetes.io/projected/5e3ad34f-c054-4dfd-a21e-f6a5d91d820f-kube-api-access-qzxbz\") pod \"nova-cell1-84d8-account-create-update-d7f29\" (UID: \"5e3ad34f-c054-4dfd-a21e-f6a5d91d820f\") " pod="openstack-kuttl-tests/nova-cell1-84d8-account-create-update-d7f29" Dec 03 11:59:15 crc kubenswrapper[5052]: I1203 11:59:15.772985 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-84d8-account-create-update-d7f29" Dec 03 11:59:15 crc kubenswrapper[5052]: I1203 11:59:15.799086 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 11:59:15 crc kubenswrapper[5052]: I1203 11:59:15.930830 5052 generic.go:334] "Generic (PLEG): container finished" podID="6bb40332-be9f-4633-8cab-72971768cbba" containerID="ec5e6c8abdbedd4339a7b82a497d3d7b854810af9a4c0415133053de14bb98fb" exitCode=0 Dec 03 11:59:15 crc kubenswrapper[5052]: I1203 11:59:15.932199 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-8966cf85c-7cmmp" event={"ID":"6bb40332-be9f-4633-8cab-72971768cbba","Type":"ContainerDied","Data":"ec5e6c8abdbedd4339a7b82a497d3d7b854810af9a4c0415133053de14bb98fb"} Dec 03 11:59:16 crc kubenswrapper[5052]: I1203 11:59:16.005874 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:59:16 crc kubenswrapper[5052]: I1203 11:59:16.103533 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell0-db-create-62hhq"] Dec 03 11:59:16 crc kubenswrapper[5052]: W1203 11:59:16.110067 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod10051460_11e9_4e59_93c6_669a4d43d908.slice/crio-f2865875fdf65085c824bee5c9cd637f0f41be0e052fc8a986dd838264df08d6 WatchSource:0}: Error finding container f2865875fdf65085c824bee5c9cd637f0f41be0e052fc8a986dd838264df08d6: Status 404 returned error can't find the container with id f2865875fdf65085c824bee5c9cd637f0f41be0e052fc8a986dd838264df08d6 Dec 03 11:59:16 crc kubenswrapper[5052]: W1203 11:59:16.178653 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod30aeb633_e745_42cd_8f1b_cf60ac77d7ba.slice/crio-b2e5565915b164e5a6cf65000cb08b68089eff058cdba921c00f3cfad6115b83 WatchSource:0}: Error finding container b2e5565915b164e5a6cf65000cb08b68089eff058cdba921c00f3cfad6115b83: Status 404 returned error can't find the container with id b2e5565915b164e5a6cf65000cb08b68089eff058cdba921c00f3cfad6115b83 Dec 03 11:59:16 crc kubenswrapper[5052]: I1203 11:59:16.193583 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-f429-account-create-update-9r5b7"] Dec 03 11:59:16 crc kubenswrapper[5052]: I1203 11:59:16.214951 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-db-create-5rwbf"] Dec 03 11:59:16 crc kubenswrapper[5052]: I1203 11:59:16.380551 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-db-create-d4vgs"] Dec 03 11:59:16 crc kubenswrapper[5052]: I1203 11:59:16.493162 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell0-6be2-account-create-update-2hj8c"] Dec 03 11:59:16 crc kubenswrapper[5052]: W1203 11:59:16.497972 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod15926354_9368_4283_8452_89eed0479195.slice/crio-b1ab7cf1f01eb5992430c6d26e1119771c6d163e8ec6422cd5cce5c5be22c837 WatchSource:0}: Error finding container b1ab7cf1f01eb5992430c6d26e1119771c6d163e8ec6422cd5cce5c5be22c837: Status 404 returned error can't find the container with id b1ab7cf1f01eb5992430c6d26e1119771c6d163e8ec6422cd5cce5c5be22c837 Dec 03 11:59:16 crc kubenswrapper[5052]: W1203 11:59:16.554600 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5e3ad34f_c054_4dfd_a21e_f6a5d91d820f.slice/crio-62fd8accdf0d496522c0f98ee6165a82dada061160892381139770fa80ec87d5 WatchSource:0}: Error finding container 62fd8accdf0d496522c0f98ee6165a82dada061160892381139770fa80ec87d5: Status 404 returned error can't find the container with id 62fd8accdf0d496522c0f98ee6165a82dada061160892381139770fa80ec87d5 Dec 03 11:59:16 crc kubenswrapper[5052]: I1203 11:59:16.559347 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-84d8-account-create-update-d7f29"] Dec 03 11:59:16 crc kubenswrapper[5052]: I1203 11:59:16.959705 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-db-create-62hhq" event={"ID":"10051460-11e9-4e59-93c6-669a4d43d908","Type":"ContainerStarted","Data":"0bb871ee79387ffeef2e7d444c482d183bd44015eb4fc256b2f431a4e4cdb6e7"} Dec 03 11:59:16 crc kubenswrapper[5052]: I1203 11:59:16.959751 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-db-create-62hhq" event={"ID":"10051460-11e9-4e59-93c6-669a4d43d908","Type":"ContainerStarted","Data":"f2865875fdf65085c824bee5c9cd637f0f41be0e052fc8a986dd838264df08d6"} Dec 03 11:59:16 crc kubenswrapper[5052]: I1203 11:59:16.968869 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"366cc2b7-5223-45c0-b9dc-15e8975e60a7","Type":"ContainerStarted","Data":"74c0f8f8c372aba09bba156d15cd1c5d79941edbe0f664f0bb3a335749b39608"} Dec 03 11:59:16 crc kubenswrapper[5052]: I1203 11:59:16.977587 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-db-create-d4vgs" event={"ID":"2ddd27d6-ad35-4919-8fe5-17100335e14a","Type":"ContainerStarted","Data":"d2f8589601d9421722403f9ed2193c9aac86c207654cb38b976b628ea82a5c12"} Dec 03 11:59:16 crc kubenswrapper[5052]: I1203 11:59:16.988008 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-db-create-5rwbf" event={"ID":"4e7bcb77-fa54-4f5b-bb6f-77c12d876b11","Type":"ContainerStarted","Data":"02340be78947f8f3ca41eedbebdaa47f237470f1f23212c0d023f165bb78b32c"} Dec 03 11:59:16 crc kubenswrapper[5052]: I1203 11:59:16.988067 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-db-create-5rwbf" event={"ID":"4e7bcb77-fa54-4f5b-bb6f-77c12d876b11","Type":"ContainerStarted","Data":"69c6387332f5863375dc33449d991426290c3adcd0bd8ed0849e3a49d53f4b14"} Dec 03 11:59:17 crc kubenswrapper[5052]: I1203 11:59:17.002849 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-6be2-account-create-update-2hj8c" event={"ID":"15926354-9368-4283-8452-89eed0479195","Type":"ContainerStarted","Data":"c88549135746d4c71800918fd34260a35faff14ed50e30eb9b79cd16e17209d2"} Dec 03 11:59:17 crc kubenswrapper[5052]: I1203 11:59:17.002912 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-6be2-account-create-update-2hj8c" event={"ID":"15926354-9368-4283-8452-89eed0479195","Type":"ContainerStarted","Data":"b1ab7cf1f01eb5992430c6d26e1119771c6d163e8ec6422cd5cce5c5be22c837"} Dec 03 11:59:17 crc kubenswrapper[5052]: I1203 11:59:17.015982 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-cell1-db-create-d4vgs" podStartSLOduration=2.01455786 podStartE2EDuration="2.01455786s" podCreationTimestamp="2025-12-03 11:59:15 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:59:16.998769068 +0000 UTC m=+2347.587010832" watchObservedRunningTime="2025-12-03 11:59:17.01455786 +0000 UTC m=+2347.602799624" Dec 03 11:59:17 crc kubenswrapper[5052]: I1203 11:59:17.021820 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-f429-account-create-update-9r5b7" event={"ID":"30aeb633-e745-42cd-8f1b-cf60ac77d7ba","Type":"ContainerStarted","Data":"be4c08c873da291db8d549e72617dd42955fe1f6b45313231d60e035c2c44cc1"} Dec 03 11:59:17 crc kubenswrapper[5052]: I1203 11:59:17.021886 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-f429-account-create-update-9r5b7" event={"ID":"30aeb633-e745-42cd-8f1b-cf60ac77d7ba","Type":"ContainerStarted","Data":"b2e5565915b164e5a6cf65000cb08b68089eff058cdba921c00f3cfad6115b83"} Dec 03 11:59:17 crc kubenswrapper[5052]: I1203 11:59:17.032875 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-api-db-create-5rwbf" podStartSLOduration=3.032850143 podStartE2EDuration="3.032850143s" podCreationTimestamp="2025-12-03 11:59:14 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:59:17.018015249 +0000 UTC m=+2347.606257013" watchObservedRunningTime="2025-12-03 11:59:17.032850143 +0000 UTC m=+2347.621091917" Dec 03 11:59:17 crc kubenswrapper[5052]: I1203 11:59:17.045726 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-84d8-account-create-update-d7f29" event={"ID":"5e3ad34f-c054-4dfd-a21e-f6a5d91d820f","Type":"ContainerStarted","Data":"62fd8accdf0d496522c0f98ee6165a82dada061160892381139770fa80ec87d5"} Dec 03 11:59:17 crc kubenswrapper[5052]: I1203 11:59:17.056328 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-cell0-6be2-account-create-update-2hj8c" podStartSLOduration=2.056307375 podStartE2EDuration="2.056307375s" podCreationTimestamp="2025-12-03 11:59:15 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:59:17.045933008 +0000 UTC m=+2347.634174772" watchObservedRunningTime="2025-12-03 11:59:17.056307375 +0000 UTC m=+2347.644549129" Dec 03 11:59:17 crc kubenswrapper[5052]: I1203 11:59:17.064353 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-api-f429-account-create-update-9r5b7" podStartSLOduration=3.064331534 podStartE2EDuration="3.064331534s" podCreationTimestamp="2025-12-03 11:59:14 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:59:17.06280168 +0000 UTC m=+2347.651043444" watchObservedRunningTime="2025-12-03 11:59:17.064331534 +0000 UTC m=+2347.652573298" Dec 03 11:59:17 crc kubenswrapper[5052]: I1203 11:59:17.092643 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-cell1-84d8-account-create-update-d7f29" podStartSLOduration=2.092621344 podStartE2EDuration="2.092621344s" podCreationTimestamp="2025-12-03 11:59:15 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:59:17.092442189 +0000 UTC m=+2347.680683953" watchObservedRunningTime="2025-12-03 11:59:17.092621344 +0000 UTC m=+2347.680863108" Dec 03 11:59:17 crc kubenswrapper[5052]: I1203 11:59:17.510044 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-8966cf85c-7cmmp" Dec 03 11:59:17 crc kubenswrapper[5052]: I1203 11:59:17.687199 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6bb40332-be9f-4633-8cab-72971768cbba-combined-ca-bundle\") pod \"6bb40332-be9f-4633-8cab-72971768cbba\" (UID: \"6bb40332-be9f-4633-8cab-72971768cbba\") " Dec 03 11:59:17 crc kubenswrapper[5052]: I1203 11:59:17.687367 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/6bb40332-be9f-4633-8cab-72971768cbba-httpd-config\") pod \"6bb40332-be9f-4633-8cab-72971768cbba\" (UID: \"6bb40332-be9f-4633-8cab-72971768cbba\") " Dec 03 11:59:17 crc kubenswrapper[5052]: I1203 11:59:17.687395 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/6bb40332-be9f-4633-8cab-72971768cbba-config\") pod \"6bb40332-be9f-4633-8cab-72971768cbba\" (UID: \"6bb40332-be9f-4633-8cab-72971768cbba\") " Dec 03 11:59:17 crc kubenswrapper[5052]: I1203 11:59:17.687419 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/6bb40332-be9f-4633-8cab-72971768cbba-ovndb-tls-certs\") pod \"6bb40332-be9f-4633-8cab-72971768cbba\" (UID: \"6bb40332-be9f-4633-8cab-72971768cbba\") " Dec 03 11:59:17 crc kubenswrapper[5052]: I1203 11:59:17.687500 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tcvfl\" (UniqueName: \"kubernetes.io/projected/6bb40332-be9f-4633-8cab-72971768cbba-kube-api-access-tcvfl\") pod \"6bb40332-be9f-4633-8cab-72971768cbba\" (UID: \"6bb40332-be9f-4633-8cab-72971768cbba\") " Dec 03 11:59:17 crc kubenswrapper[5052]: I1203 11:59:17.704682 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6bb40332-be9f-4633-8cab-72971768cbba-kube-api-access-tcvfl" (OuterVolumeSpecName: "kube-api-access-tcvfl") pod "6bb40332-be9f-4633-8cab-72971768cbba" (UID: "6bb40332-be9f-4633-8cab-72971768cbba"). InnerVolumeSpecName "kube-api-access-tcvfl". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:59:17 crc kubenswrapper[5052]: I1203 11:59:17.716870 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6bb40332-be9f-4633-8cab-72971768cbba-httpd-config" (OuterVolumeSpecName: "httpd-config") pod "6bb40332-be9f-4633-8cab-72971768cbba" (UID: "6bb40332-be9f-4633-8cab-72971768cbba"). InnerVolumeSpecName "httpd-config". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:59:17 crc kubenswrapper[5052]: I1203 11:59:17.744845 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6bb40332-be9f-4633-8cab-72971768cbba-config" (OuterVolumeSpecName: "config") pod "6bb40332-be9f-4633-8cab-72971768cbba" (UID: "6bb40332-be9f-4633-8cab-72971768cbba"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:59:17 crc kubenswrapper[5052]: I1203 11:59:17.766740 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6bb40332-be9f-4633-8cab-72971768cbba-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "6bb40332-be9f-4633-8cab-72971768cbba" (UID: "6bb40332-be9f-4633-8cab-72971768cbba"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:59:17 crc kubenswrapper[5052]: I1203 11:59:17.790665 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6bb40332-be9f-4633-8cab-72971768cbba-ovndb-tls-certs" (OuterVolumeSpecName: "ovndb-tls-certs") pod "6bb40332-be9f-4633-8cab-72971768cbba" (UID: "6bb40332-be9f-4633-8cab-72971768cbba"). InnerVolumeSpecName "ovndb-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:59:17 crc kubenswrapper[5052]: I1203 11:59:17.791320 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/6bb40332-be9f-4633-8cab-72971768cbba-ovndb-tls-certs\") pod \"6bb40332-be9f-4633-8cab-72971768cbba\" (UID: \"6bb40332-be9f-4633-8cab-72971768cbba\") " Dec 03 11:59:17 crc kubenswrapper[5052]: I1203 11:59:17.791722 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6bb40332-be9f-4633-8cab-72971768cbba-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:59:17 crc kubenswrapper[5052]: I1203 11:59:17.791739 5052 reconciler_common.go:293] "Volume detached for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/6bb40332-be9f-4633-8cab-72971768cbba-httpd-config\") on node \"crc\" DevicePath \"\"" Dec 03 11:59:17 crc kubenswrapper[5052]: I1203 11:59:17.791752 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/6bb40332-be9f-4633-8cab-72971768cbba-config\") on node \"crc\" DevicePath \"\"" Dec 03 11:59:17 crc kubenswrapper[5052]: I1203 11:59:17.791789 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-tcvfl\" (UniqueName: \"kubernetes.io/projected/6bb40332-be9f-4633-8cab-72971768cbba-kube-api-access-tcvfl\") on node \"crc\" DevicePath \"\"" Dec 03 11:59:17 crc kubenswrapper[5052]: W1203 11:59:17.791906 5052 empty_dir.go:500] Warning: Unmount skipped because path does not exist: /var/lib/kubelet/pods/6bb40332-be9f-4633-8cab-72971768cbba/volumes/kubernetes.io~secret/ovndb-tls-certs Dec 03 11:59:17 crc kubenswrapper[5052]: I1203 11:59:17.791921 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6bb40332-be9f-4633-8cab-72971768cbba-ovndb-tls-certs" (OuterVolumeSpecName: "ovndb-tls-certs") pod "6bb40332-be9f-4633-8cab-72971768cbba" (UID: "6bb40332-be9f-4633-8cab-72971768cbba"). InnerVolumeSpecName "ovndb-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:59:17 crc kubenswrapper[5052]: I1203 11:59:17.896772 5052 reconciler_common.go:293] "Volume detached for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/6bb40332-be9f-4633-8cab-72971768cbba-ovndb-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 11:59:18 crc kubenswrapper[5052]: I1203 11:59:18.058846 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-84d8-account-create-update-d7f29" event={"ID":"5e3ad34f-c054-4dfd-a21e-f6a5d91d820f","Type":"ContainerStarted","Data":"159a5df23aba4e3c962924f0a774119d3e466b040473b579f0a0904498430999"} Dec 03 11:59:18 crc kubenswrapper[5052]: I1203 11:59:18.063041 5052 generic.go:334] "Generic (PLEG): container finished" podID="10051460-11e9-4e59-93c6-669a4d43d908" containerID="0bb871ee79387ffeef2e7d444c482d183bd44015eb4fc256b2f431a4e4cdb6e7" exitCode=0 Dec 03 11:59:18 crc kubenswrapper[5052]: I1203 11:59:18.063204 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-db-create-62hhq" event={"ID":"10051460-11e9-4e59-93c6-669a4d43d908","Type":"ContainerDied","Data":"0bb871ee79387ffeef2e7d444c482d183bd44015eb4fc256b2f431a4e4cdb6e7"} Dec 03 11:59:18 crc kubenswrapper[5052]: I1203 11:59:18.072002 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-db-create-d4vgs" event={"ID":"2ddd27d6-ad35-4919-8fe5-17100335e14a","Type":"ContainerStarted","Data":"fcfc59f028f6c29ffcc2a22439423717d9f8cacc46e9ad92735c695c4aae8643"} Dec 03 11:59:18 crc kubenswrapper[5052]: I1203 11:59:18.089577 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-8966cf85c-7cmmp" Dec 03 11:59:18 crc kubenswrapper[5052]: I1203 11:59:18.095523 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-8966cf85c-7cmmp" event={"ID":"6bb40332-be9f-4633-8cab-72971768cbba","Type":"ContainerDied","Data":"33f5bc6ff8e55b7e1eda38ad9f955f861f54fdf563eb86d08bc4e61568bd4b98"} Dec 03 11:59:18 crc kubenswrapper[5052]: I1203 11:59:18.095623 5052 scope.go:117] "RemoveContainer" containerID="072116c6057db13f831617dbb142496bb56bb8d9006e2da94d6128552d93ae62" Dec 03 11:59:18 crc kubenswrapper[5052]: I1203 11:59:18.124362 5052 scope.go:117] "RemoveContainer" containerID="ec5e6c8abdbedd4339a7b82a497d3d7b854810af9a4c0415133053de14bb98fb" Dec 03 11:59:18 crc kubenswrapper[5052]: I1203 11:59:18.138850 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/neutron-8966cf85c-7cmmp"] Dec 03 11:59:18 crc kubenswrapper[5052]: I1203 11:59:18.151679 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/neutron-8966cf85c-7cmmp"] Dec 03 11:59:19 crc kubenswrapper[5052]: I1203 11:59:19.101128 5052 generic.go:334] "Generic (PLEG): container finished" podID="4e7bcb77-fa54-4f5b-bb6f-77c12d876b11" containerID="02340be78947f8f3ca41eedbebdaa47f237470f1f23212c0d023f165bb78b32c" exitCode=0 Dec 03 11:59:19 crc kubenswrapper[5052]: I1203 11:59:19.101187 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-db-create-5rwbf" event={"ID":"4e7bcb77-fa54-4f5b-bb6f-77c12d876b11","Type":"ContainerDied","Data":"02340be78947f8f3ca41eedbebdaa47f237470f1f23212c0d023f165bb78b32c"} Dec 03 11:59:19 crc kubenswrapper[5052]: I1203 11:59:19.104086 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"366cc2b7-5223-45c0-b9dc-15e8975e60a7","Type":"ContainerStarted","Data":"551dcbe22aae72ff2844b88c792e5be1d69a760de8cfc7a0c16f3f458aee81e0"} Dec 03 11:59:19 crc kubenswrapper[5052]: I1203 11:59:19.105454 5052 generic.go:334] "Generic (PLEG): container finished" podID="2ddd27d6-ad35-4919-8fe5-17100335e14a" containerID="fcfc59f028f6c29ffcc2a22439423717d9f8cacc46e9ad92735c695c4aae8643" exitCode=0 Dec 03 11:59:19 crc kubenswrapper[5052]: I1203 11:59:19.105545 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-db-create-d4vgs" event={"ID":"2ddd27d6-ad35-4919-8fe5-17100335e14a","Type":"ContainerDied","Data":"fcfc59f028f6c29ffcc2a22439423717d9f8cacc46e9ad92735c695c4aae8643"} Dec 03 11:59:19 crc kubenswrapper[5052]: I1203 11:59:19.683672 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6bb40332-be9f-4633-8cab-72971768cbba" path="/var/lib/kubelet/pods/6bb40332-be9f-4633-8cab-72971768cbba/volumes" Dec 03 11:59:19 crc kubenswrapper[5052]: I1203 11:59:19.727440 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-db-create-62hhq" Dec 03 11:59:19 crc kubenswrapper[5052]: I1203 11:59:19.841002 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pfckh\" (UniqueName: \"kubernetes.io/projected/10051460-11e9-4e59-93c6-669a4d43d908-kube-api-access-pfckh\") pod \"10051460-11e9-4e59-93c6-669a4d43d908\" (UID: \"10051460-11e9-4e59-93c6-669a4d43d908\") " Dec 03 11:59:19 crc kubenswrapper[5052]: I1203 11:59:19.841216 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/10051460-11e9-4e59-93c6-669a4d43d908-operator-scripts\") pod \"10051460-11e9-4e59-93c6-669a4d43d908\" (UID: \"10051460-11e9-4e59-93c6-669a4d43d908\") " Dec 03 11:59:19 crc kubenswrapper[5052]: I1203 11:59:19.841785 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/10051460-11e9-4e59-93c6-669a4d43d908-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "10051460-11e9-4e59-93c6-669a4d43d908" (UID: "10051460-11e9-4e59-93c6-669a4d43d908"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:59:19 crc kubenswrapper[5052]: I1203 11:59:19.859829 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/10051460-11e9-4e59-93c6-669a4d43d908-kube-api-access-pfckh" (OuterVolumeSpecName: "kube-api-access-pfckh") pod "10051460-11e9-4e59-93c6-669a4d43d908" (UID: "10051460-11e9-4e59-93c6-669a4d43d908"). InnerVolumeSpecName "kube-api-access-pfckh". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:59:19 crc kubenswrapper[5052]: I1203 11:59:19.943688 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-pfckh\" (UniqueName: \"kubernetes.io/projected/10051460-11e9-4e59-93c6-669a4d43d908-kube-api-access-pfckh\") on node \"crc\" DevicePath \"\"" Dec 03 11:59:19 crc kubenswrapper[5052]: I1203 11:59:19.943724 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/10051460-11e9-4e59-93c6-669a4d43d908-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:59:20 crc kubenswrapper[5052]: I1203 11:59:20.117233 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-f429-account-create-update-9r5b7" event={"ID":"30aeb633-e745-42cd-8f1b-cf60ac77d7ba","Type":"ContainerDied","Data":"be4c08c873da291db8d549e72617dd42955fe1f6b45313231d60e035c2c44cc1"} Dec 03 11:59:20 crc kubenswrapper[5052]: I1203 11:59:20.117186 5052 generic.go:334] "Generic (PLEG): container finished" podID="30aeb633-e745-42cd-8f1b-cf60ac77d7ba" containerID="be4c08c873da291db8d549e72617dd42955fe1f6b45313231d60e035c2c44cc1" exitCode=0 Dec 03 11:59:20 crc kubenswrapper[5052]: I1203 11:59:20.121777 5052 generic.go:334] "Generic (PLEG): container finished" podID="5e3ad34f-c054-4dfd-a21e-f6a5d91d820f" containerID="159a5df23aba4e3c962924f0a774119d3e466b040473b579f0a0904498430999" exitCode=0 Dec 03 11:59:20 crc kubenswrapper[5052]: I1203 11:59:20.121844 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-84d8-account-create-update-d7f29" event={"ID":"5e3ad34f-c054-4dfd-a21e-f6a5d91d820f","Type":"ContainerDied","Data":"159a5df23aba4e3c962924f0a774119d3e466b040473b579f0a0904498430999"} Dec 03 11:59:20 crc kubenswrapper[5052]: I1203 11:59:20.152370 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-db-create-62hhq" event={"ID":"10051460-11e9-4e59-93c6-669a4d43d908","Type":"ContainerDied","Data":"f2865875fdf65085c824bee5c9cd637f0f41be0e052fc8a986dd838264df08d6"} Dec 03 11:59:20 crc kubenswrapper[5052]: I1203 11:59:20.152421 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f2865875fdf65085c824bee5c9cd637f0f41be0e052fc8a986dd838264df08d6" Dec 03 11:59:20 crc kubenswrapper[5052]: I1203 11:59:20.152574 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-db-create-62hhq" Dec 03 11:59:20 crc kubenswrapper[5052]: I1203 11:59:20.174189 5052 generic.go:334] "Generic (PLEG): container finished" podID="15926354-9368-4283-8452-89eed0479195" containerID="c88549135746d4c71800918fd34260a35faff14ed50e30eb9b79cd16e17209d2" exitCode=0 Dec 03 11:59:20 crc kubenswrapper[5052]: I1203 11:59:20.174356 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="366cc2b7-5223-45c0-b9dc-15e8975e60a7" containerName="ceilometer-central-agent" containerID="cri-o://351abba6c79b072d578f848106cf4ca4c9b9e1097e08602304846997d8bdebfb" gracePeriod=30 Dec 03 11:59:20 crc kubenswrapper[5052]: I1203 11:59:20.174757 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-6be2-account-create-update-2hj8c" event={"ID":"15926354-9368-4283-8452-89eed0479195","Type":"ContainerDied","Data":"c88549135746d4c71800918fd34260a35faff14ed50e30eb9b79cd16e17209d2"} Dec 03 11:59:20 crc kubenswrapper[5052]: I1203 11:59:20.175113 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:59:20 crc kubenswrapper[5052]: I1203 11:59:20.175167 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="366cc2b7-5223-45c0-b9dc-15e8975e60a7" containerName="proxy-httpd" containerID="cri-o://551dcbe22aae72ff2844b88c792e5be1d69a760de8cfc7a0c16f3f458aee81e0" gracePeriod=30 Dec 03 11:59:20 crc kubenswrapper[5052]: I1203 11:59:20.175219 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="366cc2b7-5223-45c0-b9dc-15e8975e60a7" containerName="sg-core" containerID="cri-o://74c0f8f8c372aba09bba156d15cd1c5d79941edbe0f664f0bb3a335749b39608" gracePeriod=30 Dec 03 11:59:20 crc kubenswrapper[5052]: I1203 11:59:20.175272 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="366cc2b7-5223-45c0-b9dc-15e8975e60a7" containerName="ceilometer-notification-agent" containerID="cri-o://b4ff98aed209f866dc33b6fd55f0c211be929bf140a7fedb2c2d387b427f84e0" gracePeriod=30 Dec 03 11:59:20 crc kubenswrapper[5052]: I1203 11:59:20.215014 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/ceilometer-0" podStartSLOduration=3.562417555 podStartE2EDuration="10.214994582s" podCreationTimestamp="2025-12-03 11:59:10 +0000 UTC" firstStartedPulling="2025-12-03 11:59:11.721253112 +0000 UTC m=+2342.309494876" lastFinishedPulling="2025-12-03 11:59:18.373830139 +0000 UTC m=+2348.962071903" observedRunningTime="2025-12-03 11:59:20.213821558 +0000 UTC m=+2350.802063322" watchObservedRunningTime="2025-12-03 11:59:20.214994582 +0000 UTC m=+2350.803236346" Dec 03 11:59:20 crc kubenswrapper[5052]: I1203 11:59:20.643278 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-db-create-d4vgs" Dec 03 11:59:20 crc kubenswrapper[5052]: I1203 11:59:20.759833 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/2ddd27d6-ad35-4919-8fe5-17100335e14a-operator-scripts\") pod \"2ddd27d6-ad35-4919-8fe5-17100335e14a\" (UID: \"2ddd27d6-ad35-4919-8fe5-17100335e14a\") " Dec 03 11:59:20 crc kubenswrapper[5052]: I1203 11:59:20.759984 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vgbqj\" (UniqueName: \"kubernetes.io/projected/2ddd27d6-ad35-4919-8fe5-17100335e14a-kube-api-access-vgbqj\") pod \"2ddd27d6-ad35-4919-8fe5-17100335e14a\" (UID: \"2ddd27d6-ad35-4919-8fe5-17100335e14a\") " Dec 03 11:59:20 crc kubenswrapper[5052]: I1203 11:59:20.761478 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2ddd27d6-ad35-4919-8fe5-17100335e14a-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "2ddd27d6-ad35-4919-8fe5-17100335e14a" (UID: "2ddd27d6-ad35-4919-8fe5-17100335e14a"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:59:21 crc kubenswrapper[5052]: I1203 11:59:21.615590 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2ddd27d6-ad35-4919-8fe5-17100335e14a-kube-api-access-vgbqj" (OuterVolumeSpecName: "kube-api-access-vgbqj") pod "2ddd27d6-ad35-4919-8fe5-17100335e14a" (UID: "2ddd27d6-ad35-4919-8fe5-17100335e14a"). InnerVolumeSpecName "kube-api-access-vgbqj". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:59:21 crc kubenswrapper[5052]: I1203 11:59:21.617563 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/2ddd27d6-ad35-4919-8fe5-17100335e14a-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:59:21 crc kubenswrapper[5052]: I1203 11:59:21.617595 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-vgbqj\" (UniqueName: \"kubernetes.io/projected/2ddd27d6-ad35-4919-8fe5-17100335e14a-kube-api-access-vgbqj\") on node \"crc\" DevicePath \"\"" Dec 03 11:59:21 crc kubenswrapper[5052]: I1203 11:59:21.665821 5052 generic.go:334] "Generic (PLEG): container finished" podID="366cc2b7-5223-45c0-b9dc-15e8975e60a7" containerID="551dcbe22aae72ff2844b88c792e5be1d69a760de8cfc7a0c16f3f458aee81e0" exitCode=0 Dec 03 11:59:21 crc kubenswrapper[5052]: I1203 11:59:21.666093 5052 generic.go:334] "Generic (PLEG): container finished" podID="366cc2b7-5223-45c0-b9dc-15e8975e60a7" containerID="74c0f8f8c372aba09bba156d15cd1c5d79941edbe0f664f0bb3a335749b39608" exitCode=2 Dec 03 11:59:21 crc kubenswrapper[5052]: I1203 11:59:21.666131 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"366cc2b7-5223-45c0-b9dc-15e8975e60a7","Type":"ContainerDied","Data":"551dcbe22aae72ff2844b88c792e5be1d69a760de8cfc7a0c16f3f458aee81e0"} Dec 03 11:59:21 crc kubenswrapper[5052]: I1203 11:59:21.666162 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"366cc2b7-5223-45c0-b9dc-15e8975e60a7","Type":"ContainerDied","Data":"74c0f8f8c372aba09bba156d15cd1c5d79941edbe0f664f0bb3a335749b39608"} Dec 03 11:59:21 crc kubenswrapper[5052]: I1203 11:59:21.668216 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-db-create-d4vgs" Dec 03 11:59:21 crc kubenswrapper[5052]: I1203 11:59:21.668728 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-db-create-d4vgs" event={"ID":"2ddd27d6-ad35-4919-8fe5-17100335e14a","Type":"ContainerDied","Data":"d2f8589601d9421722403f9ed2193c9aac86c207654cb38b976b628ea82a5c12"} Dec 03 11:59:21 crc kubenswrapper[5052]: I1203 11:59:21.668782 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d2f8589601d9421722403f9ed2193c9aac86c207654cb38b976b628ea82a5c12" Dec 03 11:59:21 crc kubenswrapper[5052]: I1203 11:59:21.848154 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-db-create-5rwbf" Dec 03 11:59:21 crc kubenswrapper[5052]: I1203 11:59:21.921221 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/4e7bcb77-fa54-4f5b-bb6f-77c12d876b11-operator-scripts\") pod \"4e7bcb77-fa54-4f5b-bb6f-77c12d876b11\" (UID: \"4e7bcb77-fa54-4f5b-bb6f-77c12d876b11\") " Dec 03 11:59:21 crc kubenswrapper[5052]: I1203 11:59:21.921394 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x4c76\" (UniqueName: \"kubernetes.io/projected/4e7bcb77-fa54-4f5b-bb6f-77c12d876b11-kube-api-access-x4c76\") pod \"4e7bcb77-fa54-4f5b-bb6f-77c12d876b11\" (UID: \"4e7bcb77-fa54-4f5b-bb6f-77c12d876b11\") " Dec 03 11:59:21 crc kubenswrapper[5052]: I1203 11:59:21.923989 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4e7bcb77-fa54-4f5b-bb6f-77c12d876b11-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "4e7bcb77-fa54-4f5b-bb6f-77c12d876b11" (UID: "4e7bcb77-fa54-4f5b-bb6f-77c12d876b11"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:59:21 crc kubenswrapper[5052]: I1203 11:59:21.946949 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4e7bcb77-fa54-4f5b-bb6f-77c12d876b11-kube-api-access-x4c76" (OuterVolumeSpecName: "kube-api-access-x4c76") pod "4e7bcb77-fa54-4f5b-bb6f-77c12d876b11" (UID: "4e7bcb77-fa54-4f5b-bb6f-77c12d876b11"). InnerVolumeSpecName "kube-api-access-x4c76". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:59:22 crc kubenswrapper[5052]: I1203 11:59:22.022996 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/4e7bcb77-fa54-4f5b-bb6f-77c12d876b11-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:59:22 crc kubenswrapper[5052]: I1203 11:59:22.023029 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-x4c76\" (UniqueName: \"kubernetes.io/projected/4e7bcb77-fa54-4f5b-bb6f-77c12d876b11-kube-api-access-x4c76\") on node \"crc\" DevicePath \"\"" Dec 03 11:59:22 crc kubenswrapper[5052]: I1203 11:59:22.291314 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-f429-account-create-update-9r5b7" Dec 03 11:59:22 crc kubenswrapper[5052]: I1203 11:59:22.298397 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-6be2-account-create-update-2hj8c" Dec 03 11:59:22 crc kubenswrapper[5052]: I1203 11:59:22.330982 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-84d8-account-create-update-d7f29" Dec 03 11:59:22 crc kubenswrapper[5052]: I1203 11:59:22.343802 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t764p\" (UniqueName: \"kubernetes.io/projected/15926354-9368-4283-8452-89eed0479195-kube-api-access-t764p\") pod \"15926354-9368-4283-8452-89eed0479195\" (UID: \"15926354-9368-4283-8452-89eed0479195\") " Dec 03 11:59:22 crc kubenswrapper[5052]: I1203 11:59:22.343921 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/30aeb633-e745-42cd-8f1b-cf60ac77d7ba-operator-scripts\") pod \"30aeb633-e745-42cd-8f1b-cf60ac77d7ba\" (UID: \"30aeb633-e745-42cd-8f1b-cf60ac77d7ba\") " Dec 03 11:59:22 crc kubenswrapper[5052]: I1203 11:59:22.344058 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p255z\" (UniqueName: \"kubernetes.io/projected/30aeb633-e745-42cd-8f1b-cf60ac77d7ba-kube-api-access-p255z\") pod \"30aeb633-e745-42cd-8f1b-cf60ac77d7ba\" (UID: \"30aeb633-e745-42cd-8f1b-cf60ac77d7ba\") " Dec 03 11:59:22 crc kubenswrapper[5052]: I1203 11:59:22.344178 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/15926354-9368-4283-8452-89eed0479195-operator-scripts\") pod \"15926354-9368-4283-8452-89eed0479195\" (UID: \"15926354-9368-4283-8452-89eed0479195\") " Dec 03 11:59:22 crc kubenswrapper[5052]: I1203 11:59:22.344811 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/30aeb633-e745-42cd-8f1b-cf60ac77d7ba-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "30aeb633-e745-42cd-8f1b-cf60ac77d7ba" (UID: "30aeb633-e745-42cd-8f1b-cf60ac77d7ba"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:59:22 crc kubenswrapper[5052]: I1203 11:59:22.345436 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/30aeb633-e745-42cd-8f1b-cf60ac77d7ba-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:59:22 crc kubenswrapper[5052]: I1203 11:59:22.345970 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/15926354-9368-4283-8452-89eed0479195-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "15926354-9368-4283-8452-89eed0479195" (UID: "15926354-9368-4283-8452-89eed0479195"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:59:22 crc kubenswrapper[5052]: I1203 11:59:22.348468 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/15926354-9368-4283-8452-89eed0479195-kube-api-access-t764p" (OuterVolumeSpecName: "kube-api-access-t764p") pod "15926354-9368-4283-8452-89eed0479195" (UID: "15926354-9368-4283-8452-89eed0479195"). InnerVolumeSpecName "kube-api-access-t764p". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:59:22 crc kubenswrapper[5052]: I1203 11:59:22.352456 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/30aeb633-e745-42cd-8f1b-cf60ac77d7ba-kube-api-access-p255z" (OuterVolumeSpecName: "kube-api-access-p255z") pod "30aeb633-e745-42cd-8f1b-cf60ac77d7ba" (UID: "30aeb633-e745-42cd-8f1b-cf60ac77d7ba"). InnerVolumeSpecName "kube-api-access-p255z". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:59:22 crc kubenswrapper[5052]: I1203 11:59:22.447387 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/5e3ad34f-c054-4dfd-a21e-f6a5d91d820f-operator-scripts\") pod \"5e3ad34f-c054-4dfd-a21e-f6a5d91d820f\" (UID: \"5e3ad34f-c054-4dfd-a21e-f6a5d91d820f\") " Dec 03 11:59:22 crc kubenswrapper[5052]: I1203 11:59:22.447665 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qzxbz\" (UniqueName: \"kubernetes.io/projected/5e3ad34f-c054-4dfd-a21e-f6a5d91d820f-kube-api-access-qzxbz\") pod \"5e3ad34f-c054-4dfd-a21e-f6a5d91d820f\" (UID: \"5e3ad34f-c054-4dfd-a21e-f6a5d91d820f\") " Dec 03 11:59:22 crc kubenswrapper[5052]: I1203 11:59:22.448131 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-t764p\" (UniqueName: \"kubernetes.io/projected/15926354-9368-4283-8452-89eed0479195-kube-api-access-t764p\") on node \"crc\" DevicePath \"\"" Dec 03 11:59:22 crc kubenswrapper[5052]: I1203 11:59:22.448153 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-p255z\" (UniqueName: \"kubernetes.io/projected/30aeb633-e745-42cd-8f1b-cf60ac77d7ba-kube-api-access-p255z\") on node \"crc\" DevicePath \"\"" Dec 03 11:59:22 crc kubenswrapper[5052]: I1203 11:59:22.448165 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/15926354-9368-4283-8452-89eed0479195-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:59:22 crc kubenswrapper[5052]: I1203 11:59:22.448337 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5e3ad34f-c054-4dfd-a21e-f6a5d91d820f-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "5e3ad34f-c054-4dfd-a21e-f6a5d91d820f" (UID: "5e3ad34f-c054-4dfd-a21e-f6a5d91d820f"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 11:59:22 crc kubenswrapper[5052]: I1203 11:59:22.451115 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5e3ad34f-c054-4dfd-a21e-f6a5d91d820f-kube-api-access-qzxbz" (OuterVolumeSpecName: "kube-api-access-qzxbz") pod "5e3ad34f-c054-4dfd-a21e-f6a5d91d820f" (UID: "5e3ad34f-c054-4dfd-a21e-f6a5d91d820f"). InnerVolumeSpecName "kube-api-access-qzxbz". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:59:22 crc kubenswrapper[5052]: I1203 11:59:22.550105 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/5e3ad34f-c054-4dfd-a21e-f6a5d91d820f-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:59:22 crc kubenswrapper[5052]: I1203 11:59:22.550153 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-qzxbz\" (UniqueName: \"kubernetes.io/projected/5e3ad34f-c054-4dfd-a21e-f6a5d91d820f-kube-api-access-qzxbz\") on node \"crc\" DevicePath \"\"" Dec 03 11:59:22 crc kubenswrapper[5052]: I1203 11:59:22.682531 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-f429-account-create-update-9r5b7" event={"ID":"30aeb633-e745-42cd-8f1b-cf60ac77d7ba","Type":"ContainerDied","Data":"b2e5565915b164e5a6cf65000cb08b68089eff058cdba921c00f3cfad6115b83"} Dec 03 11:59:22 crc kubenswrapper[5052]: I1203 11:59:22.682572 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b2e5565915b164e5a6cf65000cb08b68089eff058cdba921c00f3cfad6115b83" Dec 03 11:59:22 crc kubenswrapper[5052]: I1203 11:59:22.682595 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-f429-account-create-update-9r5b7" Dec 03 11:59:22 crc kubenswrapper[5052]: I1203 11:59:22.684871 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-84d8-account-create-update-d7f29" event={"ID":"5e3ad34f-c054-4dfd-a21e-f6a5d91d820f","Type":"ContainerDied","Data":"62fd8accdf0d496522c0f98ee6165a82dada061160892381139770fa80ec87d5"} Dec 03 11:59:22 crc kubenswrapper[5052]: I1203 11:59:22.684910 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="62fd8accdf0d496522c0f98ee6165a82dada061160892381139770fa80ec87d5" Dec 03 11:59:22 crc kubenswrapper[5052]: I1203 11:59:22.685018 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-84d8-account-create-update-d7f29" Dec 03 11:59:22 crc kubenswrapper[5052]: I1203 11:59:22.686844 5052 generic.go:334] "Generic (PLEG): container finished" podID="366cc2b7-5223-45c0-b9dc-15e8975e60a7" containerID="b4ff98aed209f866dc33b6fd55f0c211be929bf140a7fedb2c2d387b427f84e0" exitCode=0 Dec 03 11:59:22 crc kubenswrapper[5052]: I1203 11:59:22.686888 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"366cc2b7-5223-45c0-b9dc-15e8975e60a7","Type":"ContainerDied","Data":"b4ff98aed209f866dc33b6fd55f0c211be929bf140a7fedb2c2d387b427f84e0"} Dec 03 11:59:22 crc kubenswrapper[5052]: I1203 11:59:22.691039 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-db-create-5rwbf" Dec 03 11:59:22 crc kubenswrapper[5052]: I1203 11:59:22.691090 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-db-create-5rwbf" event={"ID":"4e7bcb77-fa54-4f5b-bb6f-77c12d876b11","Type":"ContainerDied","Data":"69c6387332f5863375dc33449d991426290c3adcd0bd8ed0849e3a49d53f4b14"} Dec 03 11:59:22 crc kubenswrapper[5052]: I1203 11:59:22.691138 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="69c6387332f5863375dc33449d991426290c3adcd0bd8ed0849e3a49d53f4b14" Dec 03 11:59:22 crc kubenswrapper[5052]: I1203 11:59:22.699886 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-6be2-account-create-update-2hj8c" event={"ID":"15926354-9368-4283-8452-89eed0479195","Type":"ContainerDied","Data":"b1ab7cf1f01eb5992430c6d26e1119771c6d163e8ec6422cd5cce5c5be22c837"} Dec 03 11:59:22 crc kubenswrapper[5052]: I1203 11:59:22.699928 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b1ab7cf1f01eb5992430c6d26e1119771c6d163e8ec6422cd5cce5c5be22c837" Dec 03 11:59:22 crc kubenswrapper[5052]: I1203 11:59:22.699986 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-6be2-account-create-update-2hj8c" Dec 03 11:59:23 crc kubenswrapper[5052]: I1203 11:59:23.673909 5052 scope.go:117] "RemoveContainer" containerID="d224717c6d9ca8135fd510e03a6c40c9f2b1a56dd8b4a3b685af3e2dfa855982" Dec 03 11:59:23 crc kubenswrapper[5052]: E1203 11:59:23.674115 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 11:59:23 crc kubenswrapper[5052]: I1203 11:59:23.743350 5052 generic.go:334] "Generic (PLEG): container finished" podID="366cc2b7-5223-45c0-b9dc-15e8975e60a7" containerID="351abba6c79b072d578f848106cf4ca4c9b9e1097e08602304846997d8bdebfb" exitCode=0 Dec 03 11:59:23 crc kubenswrapper[5052]: I1203 11:59:23.744151 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"366cc2b7-5223-45c0-b9dc-15e8975e60a7","Type":"ContainerDied","Data":"351abba6c79b072d578f848106cf4ca4c9b9e1097e08602304846997d8bdebfb"} Dec 03 11:59:24 crc kubenswrapper[5052]: I1203 11:59:24.055947 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:59:24 crc kubenswrapper[5052]: I1203 11:59:24.190392 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/366cc2b7-5223-45c0-b9dc-15e8975e60a7-log-httpd\") pod \"366cc2b7-5223-45c0-b9dc-15e8975e60a7\" (UID: \"366cc2b7-5223-45c0-b9dc-15e8975e60a7\") " Dec 03 11:59:24 crc kubenswrapper[5052]: I1203 11:59:24.190449 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/366cc2b7-5223-45c0-b9dc-15e8975e60a7-run-httpd\") pod \"366cc2b7-5223-45c0-b9dc-15e8975e60a7\" (UID: \"366cc2b7-5223-45c0-b9dc-15e8975e60a7\") " Dec 03 11:59:24 crc kubenswrapper[5052]: I1203 11:59:24.190479 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6mznl\" (UniqueName: \"kubernetes.io/projected/366cc2b7-5223-45c0-b9dc-15e8975e60a7-kube-api-access-6mznl\") pod \"366cc2b7-5223-45c0-b9dc-15e8975e60a7\" (UID: \"366cc2b7-5223-45c0-b9dc-15e8975e60a7\") " Dec 03 11:59:24 crc kubenswrapper[5052]: I1203 11:59:24.190549 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/366cc2b7-5223-45c0-b9dc-15e8975e60a7-scripts\") pod \"366cc2b7-5223-45c0-b9dc-15e8975e60a7\" (UID: \"366cc2b7-5223-45c0-b9dc-15e8975e60a7\") " Dec 03 11:59:24 crc kubenswrapper[5052]: I1203 11:59:24.190679 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/366cc2b7-5223-45c0-b9dc-15e8975e60a7-combined-ca-bundle\") pod \"366cc2b7-5223-45c0-b9dc-15e8975e60a7\" (UID: \"366cc2b7-5223-45c0-b9dc-15e8975e60a7\") " Dec 03 11:59:24 crc kubenswrapper[5052]: I1203 11:59:24.190735 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/366cc2b7-5223-45c0-b9dc-15e8975e60a7-config-data\") pod \"366cc2b7-5223-45c0-b9dc-15e8975e60a7\" (UID: \"366cc2b7-5223-45c0-b9dc-15e8975e60a7\") " Dec 03 11:59:24 crc kubenswrapper[5052]: I1203 11:59:24.190826 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/366cc2b7-5223-45c0-b9dc-15e8975e60a7-sg-core-conf-yaml\") pod \"366cc2b7-5223-45c0-b9dc-15e8975e60a7\" (UID: \"366cc2b7-5223-45c0-b9dc-15e8975e60a7\") " Dec 03 11:59:24 crc kubenswrapper[5052]: I1203 11:59:24.190983 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/366cc2b7-5223-45c0-b9dc-15e8975e60a7-run-httpd" (OuterVolumeSpecName: "run-httpd") pod "366cc2b7-5223-45c0-b9dc-15e8975e60a7" (UID: "366cc2b7-5223-45c0-b9dc-15e8975e60a7"). InnerVolumeSpecName "run-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:59:24 crc kubenswrapper[5052]: I1203 11:59:24.191138 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/366cc2b7-5223-45c0-b9dc-15e8975e60a7-log-httpd" (OuterVolumeSpecName: "log-httpd") pod "366cc2b7-5223-45c0-b9dc-15e8975e60a7" (UID: "366cc2b7-5223-45c0-b9dc-15e8975e60a7"). InnerVolumeSpecName "log-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:59:24 crc kubenswrapper[5052]: I1203 11:59:24.191768 5052 reconciler_common.go:293] "Volume detached for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/366cc2b7-5223-45c0-b9dc-15e8975e60a7-log-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 11:59:24 crc kubenswrapper[5052]: I1203 11:59:24.191797 5052 reconciler_common.go:293] "Volume detached for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/366cc2b7-5223-45c0-b9dc-15e8975e60a7-run-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 11:59:24 crc kubenswrapper[5052]: I1203 11:59:24.196303 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/366cc2b7-5223-45c0-b9dc-15e8975e60a7-kube-api-access-6mznl" (OuterVolumeSpecName: "kube-api-access-6mznl") pod "366cc2b7-5223-45c0-b9dc-15e8975e60a7" (UID: "366cc2b7-5223-45c0-b9dc-15e8975e60a7"). InnerVolumeSpecName "kube-api-access-6mznl". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:59:24 crc kubenswrapper[5052]: I1203 11:59:24.200241 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/366cc2b7-5223-45c0-b9dc-15e8975e60a7-scripts" (OuterVolumeSpecName: "scripts") pod "366cc2b7-5223-45c0-b9dc-15e8975e60a7" (UID: "366cc2b7-5223-45c0-b9dc-15e8975e60a7"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:59:24 crc kubenswrapper[5052]: I1203 11:59:24.228736 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/366cc2b7-5223-45c0-b9dc-15e8975e60a7-sg-core-conf-yaml" (OuterVolumeSpecName: "sg-core-conf-yaml") pod "366cc2b7-5223-45c0-b9dc-15e8975e60a7" (UID: "366cc2b7-5223-45c0-b9dc-15e8975e60a7"). InnerVolumeSpecName "sg-core-conf-yaml". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:59:24 crc kubenswrapper[5052]: I1203 11:59:24.285275 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/366cc2b7-5223-45c0-b9dc-15e8975e60a7-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "366cc2b7-5223-45c0-b9dc-15e8975e60a7" (UID: "366cc2b7-5223-45c0-b9dc-15e8975e60a7"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:59:24 crc kubenswrapper[5052]: I1203 11:59:24.293478 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/366cc2b7-5223-45c0-b9dc-15e8975e60a7-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:59:24 crc kubenswrapper[5052]: I1203 11:59:24.293529 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/366cc2b7-5223-45c0-b9dc-15e8975e60a7-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:59:24 crc kubenswrapper[5052]: I1203 11:59:24.293541 5052 reconciler_common.go:293] "Volume detached for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/366cc2b7-5223-45c0-b9dc-15e8975e60a7-sg-core-conf-yaml\") on node \"crc\" DevicePath \"\"" Dec 03 11:59:24 crc kubenswrapper[5052]: I1203 11:59:24.293551 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-6mznl\" (UniqueName: \"kubernetes.io/projected/366cc2b7-5223-45c0-b9dc-15e8975e60a7-kube-api-access-6mznl\") on node \"crc\" DevicePath \"\"" Dec 03 11:59:24 crc kubenswrapper[5052]: I1203 11:59:24.301046 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/366cc2b7-5223-45c0-b9dc-15e8975e60a7-config-data" (OuterVolumeSpecName: "config-data") pod "366cc2b7-5223-45c0-b9dc-15e8975e60a7" (UID: "366cc2b7-5223-45c0-b9dc-15e8975e60a7"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:59:24 crc kubenswrapper[5052]: I1203 11:59:24.395803 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/366cc2b7-5223-45c0-b9dc-15e8975e60a7-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:59:24 crc kubenswrapper[5052]: I1203 11:59:24.760726 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"366cc2b7-5223-45c0-b9dc-15e8975e60a7","Type":"ContainerDied","Data":"b0b2b155c6fb3e1cb498acd403cfbbd8bf6cfde001e38128450ebb976d678715"} Dec 03 11:59:24 crc kubenswrapper[5052]: I1203 11:59:24.760785 5052 scope.go:117] "RemoveContainer" containerID="551dcbe22aae72ff2844b88c792e5be1d69a760de8cfc7a0c16f3f458aee81e0" Dec 03 11:59:24 crc kubenswrapper[5052]: I1203 11:59:24.760809 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:59:24 crc kubenswrapper[5052]: I1203 11:59:24.797584 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:59:24 crc kubenswrapper[5052]: I1203 11:59:24.797595 5052 scope.go:117] "RemoveContainer" containerID="74c0f8f8c372aba09bba156d15cd1c5d79941edbe0f664f0bb3a335749b39608" Dec 03 11:59:24 crc kubenswrapper[5052]: I1203 11:59:24.802900 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:59:24 crc kubenswrapper[5052]: I1203 11:59:24.829467 5052 scope.go:117] "RemoveContainer" containerID="b4ff98aed209f866dc33b6fd55f0c211be929bf140a7fedb2c2d387b427f84e0" Dec 03 11:59:24 crc kubenswrapper[5052]: I1203 11:59:24.831036 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:59:24 crc kubenswrapper[5052]: E1203 11:59:24.831412 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="10051460-11e9-4e59-93c6-669a4d43d908" containerName="mariadb-database-create" Dec 03 11:59:24 crc kubenswrapper[5052]: I1203 11:59:24.831429 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="10051460-11e9-4e59-93c6-669a4d43d908" containerName="mariadb-database-create" Dec 03 11:59:24 crc kubenswrapper[5052]: E1203 11:59:24.831447 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="366cc2b7-5223-45c0-b9dc-15e8975e60a7" containerName="sg-core" Dec 03 11:59:24 crc kubenswrapper[5052]: I1203 11:59:24.831455 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="366cc2b7-5223-45c0-b9dc-15e8975e60a7" containerName="sg-core" Dec 03 11:59:24 crc kubenswrapper[5052]: E1203 11:59:24.831470 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="5e3ad34f-c054-4dfd-a21e-f6a5d91d820f" containerName="mariadb-account-create-update" Dec 03 11:59:24 crc kubenswrapper[5052]: I1203 11:59:24.831479 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="5e3ad34f-c054-4dfd-a21e-f6a5d91d820f" containerName="mariadb-account-create-update" Dec 03 11:59:24 crc kubenswrapper[5052]: E1203 11:59:24.831488 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="15926354-9368-4283-8452-89eed0479195" containerName="mariadb-account-create-update" Dec 03 11:59:24 crc kubenswrapper[5052]: I1203 11:59:24.831497 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="15926354-9368-4283-8452-89eed0479195" containerName="mariadb-account-create-update" Dec 03 11:59:24 crc kubenswrapper[5052]: E1203 11:59:24.831638 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="6bb40332-be9f-4633-8cab-72971768cbba" containerName="neutron-api" Dec 03 11:59:24 crc kubenswrapper[5052]: I1203 11:59:24.831647 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="6bb40332-be9f-4633-8cab-72971768cbba" containerName="neutron-api" Dec 03 11:59:24 crc kubenswrapper[5052]: E1203 11:59:24.831665 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="6bb40332-be9f-4633-8cab-72971768cbba" containerName="neutron-httpd" Dec 03 11:59:24 crc kubenswrapper[5052]: I1203 11:59:24.831671 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="6bb40332-be9f-4633-8cab-72971768cbba" containerName="neutron-httpd" Dec 03 11:59:24 crc kubenswrapper[5052]: E1203 11:59:24.831689 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="366cc2b7-5223-45c0-b9dc-15e8975e60a7" containerName="proxy-httpd" Dec 03 11:59:24 crc kubenswrapper[5052]: I1203 11:59:24.831694 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="366cc2b7-5223-45c0-b9dc-15e8975e60a7" containerName="proxy-httpd" Dec 03 11:59:24 crc kubenswrapper[5052]: E1203 11:59:24.831706 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="30aeb633-e745-42cd-8f1b-cf60ac77d7ba" containerName="mariadb-account-create-update" Dec 03 11:59:24 crc kubenswrapper[5052]: I1203 11:59:24.831713 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="30aeb633-e745-42cd-8f1b-cf60ac77d7ba" containerName="mariadb-account-create-update" Dec 03 11:59:24 crc kubenswrapper[5052]: E1203 11:59:24.831727 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="366cc2b7-5223-45c0-b9dc-15e8975e60a7" containerName="ceilometer-notification-agent" Dec 03 11:59:24 crc kubenswrapper[5052]: I1203 11:59:24.831733 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="366cc2b7-5223-45c0-b9dc-15e8975e60a7" containerName="ceilometer-notification-agent" Dec 03 11:59:24 crc kubenswrapper[5052]: E1203 11:59:24.831745 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="4e7bcb77-fa54-4f5b-bb6f-77c12d876b11" containerName="mariadb-database-create" Dec 03 11:59:24 crc kubenswrapper[5052]: I1203 11:59:24.831750 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e7bcb77-fa54-4f5b-bb6f-77c12d876b11" containerName="mariadb-database-create" Dec 03 11:59:24 crc kubenswrapper[5052]: E1203 11:59:24.831764 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="366cc2b7-5223-45c0-b9dc-15e8975e60a7" containerName="ceilometer-central-agent" Dec 03 11:59:24 crc kubenswrapper[5052]: I1203 11:59:24.831769 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="366cc2b7-5223-45c0-b9dc-15e8975e60a7" containerName="ceilometer-central-agent" Dec 03 11:59:24 crc kubenswrapper[5052]: E1203 11:59:24.831791 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="2ddd27d6-ad35-4919-8fe5-17100335e14a" containerName="mariadb-database-create" Dec 03 11:59:24 crc kubenswrapper[5052]: I1203 11:59:24.831798 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="2ddd27d6-ad35-4919-8fe5-17100335e14a" containerName="mariadb-database-create" Dec 03 11:59:24 crc kubenswrapper[5052]: I1203 11:59:24.831967 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="366cc2b7-5223-45c0-b9dc-15e8975e60a7" containerName="ceilometer-central-agent" Dec 03 11:59:24 crc kubenswrapper[5052]: I1203 11:59:24.831976 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="30aeb633-e745-42cd-8f1b-cf60ac77d7ba" containerName="mariadb-account-create-update" Dec 03 11:59:24 crc kubenswrapper[5052]: I1203 11:59:24.831990 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="10051460-11e9-4e59-93c6-669a4d43d908" containerName="mariadb-database-create" Dec 03 11:59:24 crc kubenswrapper[5052]: I1203 11:59:24.832000 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="5e3ad34f-c054-4dfd-a21e-f6a5d91d820f" containerName="mariadb-account-create-update" Dec 03 11:59:24 crc kubenswrapper[5052]: I1203 11:59:24.832008 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="4e7bcb77-fa54-4f5b-bb6f-77c12d876b11" containerName="mariadb-database-create" Dec 03 11:59:24 crc kubenswrapper[5052]: I1203 11:59:24.832016 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="2ddd27d6-ad35-4919-8fe5-17100335e14a" containerName="mariadb-database-create" Dec 03 11:59:24 crc kubenswrapper[5052]: I1203 11:59:24.832028 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="6bb40332-be9f-4633-8cab-72971768cbba" containerName="neutron-api" Dec 03 11:59:24 crc kubenswrapper[5052]: I1203 11:59:24.832036 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="366cc2b7-5223-45c0-b9dc-15e8975e60a7" containerName="ceilometer-notification-agent" Dec 03 11:59:24 crc kubenswrapper[5052]: I1203 11:59:24.832049 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="6bb40332-be9f-4633-8cab-72971768cbba" containerName="neutron-httpd" Dec 03 11:59:24 crc kubenswrapper[5052]: I1203 11:59:24.832064 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="366cc2b7-5223-45c0-b9dc-15e8975e60a7" containerName="proxy-httpd" Dec 03 11:59:24 crc kubenswrapper[5052]: I1203 11:59:24.832073 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="15926354-9368-4283-8452-89eed0479195" containerName="mariadb-account-create-update" Dec 03 11:59:24 crc kubenswrapper[5052]: I1203 11:59:24.832080 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="366cc2b7-5223-45c0-b9dc-15e8975e60a7" containerName="sg-core" Dec 03 11:59:24 crc kubenswrapper[5052]: I1203 11:59:24.834336 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:59:24 crc kubenswrapper[5052]: I1203 11:59:24.837371 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-config-data" Dec 03 11:59:24 crc kubenswrapper[5052]: I1203 11:59:24.837770 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-scripts" Dec 03 11:59:24 crc kubenswrapper[5052]: I1203 11:59:24.839576 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:59:24 crc kubenswrapper[5052]: I1203 11:59:24.859996 5052 scope.go:117] "RemoveContainer" containerID="351abba6c79b072d578f848106cf4ca4c9b9e1097e08602304846997d8bdebfb" Dec 03 11:59:24 crc kubenswrapper[5052]: I1203 11:59:24.905779 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/23026a47-055f-4596-8e25-03dfbd7765e4-config-data\") pod \"ceilometer-0\" (UID: \"23026a47-055f-4596-8e25-03dfbd7765e4\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:59:24 crc kubenswrapper[5052]: I1203 11:59:24.905836 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/23026a47-055f-4596-8e25-03dfbd7765e4-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"23026a47-055f-4596-8e25-03dfbd7765e4\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:59:24 crc kubenswrapper[5052]: I1203 11:59:24.906113 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l87ts\" (UniqueName: \"kubernetes.io/projected/23026a47-055f-4596-8e25-03dfbd7765e4-kube-api-access-l87ts\") pod \"ceilometer-0\" (UID: \"23026a47-055f-4596-8e25-03dfbd7765e4\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:59:24 crc kubenswrapper[5052]: I1203 11:59:24.906164 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/23026a47-055f-4596-8e25-03dfbd7765e4-log-httpd\") pod \"ceilometer-0\" (UID: \"23026a47-055f-4596-8e25-03dfbd7765e4\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:59:24 crc kubenswrapper[5052]: I1203 11:59:24.906279 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/23026a47-055f-4596-8e25-03dfbd7765e4-run-httpd\") pod \"ceilometer-0\" (UID: \"23026a47-055f-4596-8e25-03dfbd7765e4\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:59:24 crc kubenswrapper[5052]: I1203 11:59:24.906317 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/23026a47-055f-4596-8e25-03dfbd7765e4-scripts\") pod \"ceilometer-0\" (UID: \"23026a47-055f-4596-8e25-03dfbd7765e4\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:59:24 crc kubenswrapper[5052]: I1203 11:59:24.906479 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/23026a47-055f-4596-8e25-03dfbd7765e4-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"23026a47-055f-4596-8e25-03dfbd7765e4\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:59:25 crc kubenswrapper[5052]: I1203 11:59:25.008067 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-l87ts\" (UniqueName: \"kubernetes.io/projected/23026a47-055f-4596-8e25-03dfbd7765e4-kube-api-access-l87ts\") pod \"ceilometer-0\" (UID: \"23026a47-055f-4596-8e25-03dfbd7765e4\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:59:25 crc kubenswrapper[5052]: I1203 11:59:25.008121 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/23026a47-055f-4596-8e25-03dfbd7765e4-log-httpd\") pod \"ceilometer-0\" (UID: \"23026a47-055f-4596-8e25-03dfbd7765e4\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:59:25 crc kubenswrapper[5052]: I1203 11:59:25.008168 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/23026a47-055f-4596-8e25-03dfbd7765e4-run-httpd\") pod \"ceilometer-0\" (UID: \"23026a47-055f-4596-8e25-03dfbd7765e4\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:59:25 crc kubenswrapper[5052]: I1203 11:59:25.008192 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/23026a47-055f-4596-8e25-03dfbd7765e4-scripts\") pod \"ceilometer-0\" (UID: \"23026a47-055f-4596-8e25-03dfbd7765e4\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:59:25 crc kubenswrapper[5052]: I1203 11:59:25.008248 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/23026a47-055f-4596-8e25-03dfbd7765e4-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"23026a47-055f-4596-8e25-03dfbd7765e4\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:59:25 crc kubenswrapper[5052]: I1203 11:59:25.008295 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/23026a47-055f-4596-8e25-03dfbd7765e4-config-data\") pod \"ceilometer-0\" (UID: \"23026a47-055f-4596-8e25-03dfbd7765e4\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:59:25 crc kubenswrapper[5052]: I1203 11:59:25.008614 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/23026a47-055f-4596-8e25-03dfbd7765e4-run-httpd\") pod \"ceilometer-0\" (UID: \"23026a47-055f-4596-8e25-03dfbd7765e4\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:59:25 crc kubenswrapper[5052]: I1203 11:59:25.008653 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/23026a47-055f-4596-8e25-03dfbd7765e4-log-httpd\") pod \"ceilometer-0\" (UID: \"23026a47-055f-4596-8e25-03dfbd7765e4\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:59:25 crc kubenswrapper[5052]: I1203 11:59:25.008830 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/23026a47-055f-4596-8e25-03dfbd7765e4-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"23026a47-055f-4596-8e25-03dfbd7765e4\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:59:25 crc kubenswrapper[5052]: I1203 11:59:25.012488 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/23026a47-055f-4596-8e25-03dfbd7765e4-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"23026a47-055f-4596-8e25-03dfbd7765e4\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:59:25 crc kubenswrapper[5052]: I1203 11:59:25.014943 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/23026a47-055f-4596-8e25-03dfbd7765e4-scripts\") pod \"ceilometer-0\" (UID: \"23026a47-055f-4596-8e25-03dfbd7765e4\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:59:25 crc kubenswrapper[5052]: I1203 11:59:25.015158 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/23026a47-055f-4596-8e25-03dfbd7765e4-config-data\") pod \"ceilometer-0\" (UID: \"23026a47-055f-4596-8e25-03dfbd7765e4\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:59:25 crc kubenswrapper[5052]: I1203 11:59:25.015564 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/23026a47-055f-4596-8e25-03dfbd7765e4-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"23026a47-055f-4596-8e25-03dfbd7765e4\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:59:25 crc kubenswrapper[5052]: I1203 11:59:25.027733 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-l87ts\" (UniqueName: \"kubernetes.io/projected/23026a47-055f-4596-8e25-03dfbd7765e4-kube-api-access-l87ts\") pod \"ceilometer-0\" (UID: \"23026a47-055f-4596-8e25-03dfbd7765e4\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:59:25 crc kubenswrapper[5052]: I1203 11:59:25.152229 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:59:25 crc kubenswrapper[5052]: I1203 11:59:25.422807 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-db-sync-nljtg"] Dec 03 11:59:25 crc kubenswrapper[5052]: I1203 11:59:25.424742 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-nljtg" Dec 03 11:59:25 crc kubenswrapper[5052]: I1203 11:59:25.427159 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-nova-dockercfg-zwdn9" Dec 03 11:59:25 crc kubenswrapper[5052]: I1203 11:59:25.427334 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell0-conductor-scripts" Dec 03 11:59:25 crc kubenswrapper[5052]: I1203 11:59:25.430027 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell0-conductor-config-data" Dec 03 11:59:25 crc kubenswrapper[5052]: I1203 11:59:25.435131 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-db-sync-nljtg"] Dec 03 11:59:25 crc kubenswrapper[5052]: I1203 11:59:25.517944 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/820f9385-7816-4259-987f-15eef2e78099-scripts\") pod \"nova-cell0-conductor-db-sync-nljtg\" (UID: \"820f9385-7816-4259-987f-15eef2e78099\") " pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-nljtg" Dec 03 11:59:25 crc kubenswrapper[5052]: I1203 11:59:25.518084 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bhkx6\" (UniqueName: \"kubernetes.io/projected/820f9385-7816-4259-987f-15eef2e78099-kube-api-access-bhkx6\") pod \"nova-cell0-conductor-db-sync-nljtg\" (UID: \"820f9385-7816-4259-987f-15eef2e78099\") " pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-nljtg" Dec 03 11:59:25 crc kubenswrapper[5052]: I1203 11:59:25.518133 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/820f9385-7816-4259-987f-15eef2e78099-combined-ca-bundle\") pod \"nova-cell0-conductor-db-sync-nljtg\" (UID: \"820f9385-7816-4259-987f-15eef2e78099\") " pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-nljtg" Dec 03 11:59:25 crc kubenswrapper[5052]: I1203 11:59:25.518377 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/820f9385-7816-4259-987f-15eef2e78099-config-data\") pod \"nova-cell0-conductor-db-sync-nljtg\" (UID: \"820f9385-7816-4259-987f-15eef2e78099\") " pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-nljtg" Dec 03 11:59:25 crc kubenswrapper[5052]: I1203 11:59:25.620353 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/820f9385-7816-4259-987f-15eef2e78099-config-data\") pod \"nova-cell0-conductor-db-sync-nljtg\" (UID: \"820f9385-7816-4259-987f-15eef2e78099\") " pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-nljtg" Dec 03 11:59:25 crc kubenswrapper[5052]: I1203 11:59:25.620449 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/820f9385-7816-4259-987f-15eef2e78099-scripts\") pod \"nova-cell0-conductor-db-sync-nljtg\" (UID: \"820f9385-7816-4259-987f-15eef2e78099\") " pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-nljtg" Dec 03 11:59:25 crc kubenswrapper[5052]: I1203 11:59:25.620526 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-bhkx6\" (UniqueName: \"kubernetes.io/projected/820f9385-7816-4259-987f-15eef2e78099-kube-api-access-bhkx6\") pod \"nova-cell0-conductor-db-sync-nljtg\" (UID: \"820f9385-7816-4259-987f-15eef2e78099\") " pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-nljtg" Dec 03 11:59:25 crc kubenswrapper[5052]: I1203 11:59:25.620571 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/820f9385-7816-4259-987f-15eef2e78099-combined-ca-bundle\") pod \"nova-cell0-conductor-db-sync-nljtg\" (UID: \"820f9385-7816-4259-987f-15eef2e78099\") " pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-nljtg" Dec 03 11:59:25 crc kubenswrapper[5052]: I1203 11:59:25.632727 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/820f9385-7816-4259-987f-15eef2e78099-config-data\") pod \"nova-cell0-conductor-db-sync-nljtg\" (UID: \"820f9385-7816-4259-987f-15eef2e78099\") " pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-nljtg" Dec 03 11:59:25 crc kubenswrapper[5052]: I1203 11:59:25.633537 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/820f9385-7816-4259-987f-15eef2e78099-combined-ca-bundle\") pod \"nova-cell0-conductor-db-sync-nljtg\" (UID: \"820f9385-7816-4259-987f-15eef2e78099\") " pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-nljtg" Dec 03 11:59:25 crc kubenswrapper[5052]: I1203 11:59:25.633880 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/820f9385-7816-4259-987f-15eef2e78099-scripts\") pod \"nova-cell0-conductor-db-sync-nljtg\" (UID: \"820f9385-7816-4259-987f-15eef2e78099\") " pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-nljtg" Dec 03 11:59:25 crc kubenswrapper[5052]: I1203 11:59:25.640993 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-bhkx6\" (UniqueName: \"kubernetes.io/projected/820f9385-7816-4259-987f-15eef2e78099-kube-api-access-bhkx6\") pod \"nova-cell0-conductor-db-sync-nljtg\" (UID: \"820f9385-7816-4259-987f-15eef2e78099\") " pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-nljtg" Dec 03 11:59:25 crc kubenswrapper[5052]: I1203 11:59:25.684366 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="366cc2b7-5223-45c0-b9dc-15e8975e60a7" path="/var/lib/kubelet/pods/366cc2b7-5223-45c0-b9dc-15e8975e60a7/volumes" Dec 03 11:59:25 crc kubenswrapper[5052]: I1203 11:59:25.718257 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:59:25 crc kubenswrapper[5052]: I1203 11:59:25.745213 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-nljtg" Dec 03 11:59:25 crc kubenswrapper[5052]: I1203 11:59:25.771908 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"23026a47-055f-4596-8e25-03dfbd7765e4","Type":"ContainerStarted","Data":"7f043a7498d72acf4491cc6f58eb62909f60b19130837936092120fddf686c96"} Dec 03 11:59:26 crc kubenswrapper[5052]: I1203 11:59:26.232691 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-db-sync-nljtg"] Dec 03 11:59:26 crc kubenswrapper[5052]: W1203 11:59:26.235873 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod820f9385_7816_4259_987f_15eef2e78099.slice/crio-12483860bbb96968ef2ae1cdf1d847f67aa9f19683b97080b5df16fd4ef4119c WatchSource:0}: Error finding container 12483860bbb96968ef2ae1cdf1d847f67aa9f19683b97080b5df16fd4ef4119c: Status 404 returned error can't find the container with id 12483860bbb96968ef2ae1cdf1d847f67aa9f19683b97080b5df16fd4ef4119c Dec 03 11:59:26 crc kubenswrapper[5052]: I1203 11:59:26.783364 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-nljtg" event={"ID":"820f9385-7816-4259-987f-15eef2e78099","Type":"ContainerStarted","Data":"ae2dd9c7d6d2cf6767ca2ef1b07d9dc82f31acfcfdbfe9432084c6de9851a9f0"} Dec 03 11:59:26 crc kubenswrapper[5052]: I1203 11:59:26.783695 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-nljtg" event={"ID":"820f9385-7816-4259-987f-15eef2e78099","Type":"ContainerStarted","Data":"12483860bbb96968ef2ae1cdf1d847f67aa9f19683b97080b5df16fd4ef4119c"} Dec 03 11:59:26 crc kubenswrapper[5052]: I1203 11:59:26.785248 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"23026a47-055f-4596-8e25-03dfbd7765e4","Type":"ContainerStarted","Data":"715956d36abed25c5075c9bdd118f9c4b5e07adf089b0e30a1fb3a171e6d463c"} Dec 03 11:59:26 crc kubenswrapper[5052]: I1203 11:59:26.802811 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-nljtg" podStartSLOduration=1.802795155 podStartE2EDuration="1.802795155s" podCreationTimestamp="2025-12-03 11:59:25 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:59:26.798548974 +0000 UTC m=+2357.386790738" watchObservedRunningTime="2025-12-03 11:59:26.802795155 +0000 UTC m=+2357.391036919" Dec 03 11:59:27 crc kubenswrapper[5052]: I1203 11:59:27.800196 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"23026a47-055f-4596-8e25-03dfbd7765e4","Type":"ContainerStarted","Data":"7a059e1f453d9043b751ae5f5f2d609e9a943a78ec08c26d04dafa23dd2ed753"} Dec 03 11:59:28 crc kubenswrapper[5052]: I1203 11:59:28.810671 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"23026a47-055f-4596-8e25-03dfbd7765e4","Type":"ContainerStarted","Data":"bff805d09e466c6e21a67bec1309612d4fa585685f819fb9edbfd91df7e9548d"} Dec 03 11:59:30 crc kubenswrapper[5052]: I1203 11:59:30.839849 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"23026a47-055f-4596-8e25-03dfbd7765e4","Type":"ContainerStarted","Data":"288d258a1bd6e87ebfe9f5f08dcae443f876b9972488a68c4760a4636e7320ff"} Dec 03 11:59:30 crc kubenswrapper[5052]: I1203 11:59:30.841988 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:59:30 crc kubenswrapper[5052]: I1203 11:59:30.864060 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/ceilometer-0" podStartSLOduration=2.662589652 podStartE2EDuration="6.864042472s" podCreationTimestamp="2025-12-03 11:59:24 +0000 UTC" firstStartedPulling="2025-12-03 11:59:25.730378924 +0000 UTC m=+2356.318620688" lastFinishedPulling="2025-12-03 11:59:29.931831734 +0000 UTC m=+2360.520073508" observedRunningTime="2025-12-03 11:59:30.861060677 +0000 UTC m=+2361.449302441" watchObservedRunningTime="2025-12-03 11:59:30.864042472 +0000 UTC m=+2361.452284256" Dec 03 11:59:32 crc kubenswrapper[5052]: I1203 11:59:32.687309 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:59:33 crc kubenswrapper[5052]: I1203 11:59:33.869298 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="23026a47-055f-4596-8e25-03dfbd7765e4" containerName="ceilometer-central-agent" containerID="cri-o://715956d36abed25c5075c9bdd118f9c4b5e07adf089b0e30a1fb3a171e6d463c" gracePeriod=30 Dec 03 11:59:33 crc kubenswrapper[5052]: I1203 11:59:33.869966 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="23026a47-055f-4596-8e25-03dfbd7765e4" containerName="proxy-httpd" containerID="cri-o://288d258a1bd6e87ebfe9f5f08dcae443f876b9972488a68c4760a4636e7320ff" gracePeriod=30 Dec 03 11:59:33 crc kubenswrapper[5052]: I1203 11:59:33.870031 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="23026a47-055f-4596-8e25-03dfbd7765e4" containerName="sg-core" containerID="cri-o://bff805d09e466c6e21a67bec1309612d4fa585685f819fb9edbfd91df7e9548d" gracePeriod=30 Dec 03 11:59:33 crc kubenswrapper[5052]: I1203 11:59:33.870086 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="23026a47-055f-4596-8e25-03dfbd7765e4" containerName="ceilometer-notification-agent" containerID="cri-o://7a059e1f453d9043b751ae5f5f2d609e9a943a78ec08c26d04dafa23dd2ed753" gracePeriod=30 Dec 03 11:59:34 crc kubenswrapper[5052]: I1203 11:59:34.883100 5052 generic.go:334] "Generic (PLEG): container finished" podID="23026a47-055f-4596-8e25-03dfbd7765e4" containerID="288d258a1bd6e87ebfe9f5f08dcae443f876b9972488a68c4760a4636e7320ff" exitCode=0 Dec 03 11:59:34 crc kubenswrapper[5052]: I1203 11:59:34.883410 5052 generic.go:334] "Generic (PLEG): container finished" podID="23026a47-055f-4596-8e25-03dfbd7765e4" containerID="bff805d09e466c6e21a67bec1309612d4fa585685f819fb9edbfd91df7e9548d" exitCode=2 Dec 03 11:59:34 crc kubenswrapper[5052]: I1203 11:59:34.883420 5052 generic.go:334] "Generic (PLEG): container finished" podID="23026a47-055f-4596-8e25-03dfbd7765e4" containerID="7a059e1f453d9043b751ae5f5f2d609e9a943a78ec08c26d04dafa23dd2ed753" exitCode=0 Dec 03 11:59:34 crc kubenswrapper[5052]: I1203 11:59:34.883427 5052 generic.go:334] "Generic (PLEG): container finished" podID="23026a47-055f-4596-8e25-03dfbd7765e4" containerID="715956d36abed25c5075c9bdd118f9c4b5e07adf089b0e30a1fb3a171e6d463c" exitCode=0 Dec 03 11:59:34 crc kubenswrapper[5052]: I1203 11:59:34.883303 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"23026a47-055f-4596-8e25-03dfbd7765e4","Type":"ContainerDied","Data":"288d258a1bd6e87ebfe9f5f08dcae443f876b9972488a68c4760a4636e7320ff"} Dec 03 11:59:34 crc kubenswrapper[5052]: I1203 11:59:34.883464 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"23026a47-055f-4596-8e25-03dfbd7765e4","Type":"ContainerDied","Data":"bff805d09e466c6e21a67bec1309612d4fa585685f819fb9edbfd91df7e9548d"} Dec 03 11:59:34 crc kubenswrapper[5052]: I1203 11:59:34.883477 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"23026a47-055f-4596-8e25-03dfbd7765e4","Type":"ContainerDied","Data":"7a059e1f453d9043b751ae5f5f2d609e9a943a78ec08c26d04dafa23dd2ed753"} Dec 03 11:59:34 crc kubenswrapper[5052]: I1203 11:59:34.883487 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"23026a47-055f-4596-8e25-03dfbd7765e4","Type":"ContainerDied","Data":"715956d36abed25c5075c9bdd118f9c4b5e07adf089b0e30a1fb3a171e6d463c"} Dec 03 11:59:35 crc kubenswrapper[5052]: I1203 11:59:35.411278 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:59:35 crc kubenswrapper[5052]: I1203 11:59:35.539331 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l87ts\" (UniqueName: \"kubernetes.io/projected/23026a47-055f-4596-8e25-03dfbd7765e4-kube-api-access-l87ts\") pod \"23026a47-055f-4596-8e25-03dfbd7765e4\" (UID: \"23026a47-055f-4596-8e25-03dfbd7765e4\") " Dec 03 11:59:35 crc kubenswrapper[5052]: I1203 11:59:35.539436 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/23026a47-055f-4596-8e25-03dfbd7765e4-combined-ca-bundle\") pod \"23026a47-055f-4596-8e25-03dfbd7765e4\" (UID: \"23026a47-055f-4596-8e25-03dfbd7765e4\") " Dec 03 11:59:35 crc kubenswrapper[5052]: I1203 11:59:35.539465 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/23026a47-055f-4596-8e25-03dfbd7765e4-sg-core-conf-yaml\") pod \"23026a47-055f-4596-8e25-03dfbd7765e4\" (UID: \"23026a47-055f-4596-8e25-03dfbd7765e4\") " Dec 03 11:59:35 crc kubenswrapper[5052]: I1203 11:59:35.539625 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/23026a47-055f-4596-8e25-03dfbd7765e4-log-httpd\") pod \"23026a47-055f-4596-8e25-03dfbd7765e4\" (UID: \"23026a47-055f-4596-8e25-03dfbd7765e4\") " Dec 03 11:59:35 crc kubenswrapper[5052]: I1203 11:59:35.539718 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/23026a47-055f-4596-8e25-03dfbd7765e4-run-httpd\") pod \"23026a47-055f-4596-8e25-03dfbd7765e4\" (UID: \"23026a47-055f-4596-8e25-03dfbd7765e4\") " Dec 03 11:59:35 crc kubenswrapper[5052]: I1203 11:59:35.539748 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/23026a47-055f-4596-8e25-03dfbd7765e4-config-data\") pod \"23026a47-055f-4596-8e25-03dfbd7765e4\" (UID: \"23026a47-055f-4596-8e25-03dfbd7765e4\") " Dec 03 11:59:35 crc kubenswrapper[5052]: I1203 11:59:35.539809 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/23026a47-055f-4596-8e25-03dfbd7765e4-scripts\") pod \"23026a47-055f-4596-8e25-03dfbd7765e4\" (UID: \"23026a47-055f-4596-8e25-03dfbd7765e4\") " Dec 03 11:59:35 crc kubenswrapper[5052]: I1203 11:59:35.540399 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/23026a47-055f-4596-8e25-03dfbd7765e4-log-httpd" (OuterVolumeSpecName: "log-httpd") pod "23026a47-055f-4596-8e25-03dfbd7765e4" (UID: "23026a47-055f-4596-8e25-03dfbd7765e4"). InnerVolumeSpecName "log-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:59:35 crc kubenswrapper[5052]: I1203 11:59:35.541261 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/23026a47-055f-4596-8e25-03dfbd7765e4-run-httpd" (OuterVolumeSpecName: "run-httpd") pod "23026a47-055f-4596-8e25-03dfbd7765e4" (UID: "23026a47-055f-4596-8e25-03dfbd7765e4"). InnerVolumeSpecName "run-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:59:35 crc kubenswrapper[5052]: I1203 11:59:35.545627 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/23026a47-055f-4596-8e25-03dfbd7765e4-kube-api-access-l87ts" (OuterVolumeSpecName: "kube-api-access-l87ts") pod "23026a47-055f-4596-8e25-03dfbd7765e4" (UID: "23026a47-055f-4596-8e25-03dfbd7765e4"). InnerVolumeSpecName "kube-api-access-l87ts". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:59:35 crc kubenswrapper[5052]: I1203 11:59:35.546257 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/23026a47-055f-4596-8e25-03dfbd7765e4-scripts" (OuterVolumeSpecName: "scripts") pod "23026a47-055f-4596-8e25-03dfbd7765e4" (UID: "23026a47-055f-4596-8e25-03dfbd7765e4"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:59:35 crc kubenswrapper[5052]: I1203 11:59:35.565984 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/23026a47-055f-4596-8e25-03dfbd7765e4-sg-core-conf-yaml" (OuterVolumeSpecName: "sg-core-conf-yaml") pod "23026a47-055f-4596-8e25-03dfbd7765e4" (UID: "23026a47-055f-4596-8e25-03dfbd7765e4"). InnerVolumeSpecName "sg-core-conf-yaml". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:59:35 crc kubenswrapper[5052]: I1203 11:59:35.634844 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/23026a47-055f-4596-8e25-03dfbd7765e4-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "23026a47-055f-4596-8e25-03dfbd7765e4" (UID: "23026a47-055f-4596-8e25-03dfbd7765e4"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:59:35 crc kubenswrapper[5052]: I1203 11:59:35.641787 5052 reconciler_common.go:293] "Volume detached for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/23026a47-055f-4596-8e25-03dfbd7765e4-log-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 11:59:35 crc kubenswrapper[5052]: I1203 11:59:35.641825 5052 reconciler_common.go:293] "Volume detached for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/23026a47-055f-4596-8e25-03dfbd7765e4-run-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 11:59:35 crc kubenswrapper[5052]: I1203 11:59:35.641835 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/23026a47-055f-4596-8e25-03dfbd7765e4-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:59:35 crc kubenswrapper[5052]: I1203 11:59:35.641846 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-l87ts\" (UniqueName: \"kubernetes.io/projected/23026a47-055f-4596-8e25-03dfbd7765e4-kube-api-access-l87ts\") on node \"crc\" DevicePath \"\"" Dec 03 11:59:35 crc kubenswrapper[5052]: I1203 11:59:35.641856 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/23026a47-055f-4596-8e25-03dfbd7765e4-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:59:35 crc kubenswrapper[5052]: I1203 11:59:35.641865 5052 reconciler_common.go:293] "Volume detached for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/23026a47-055f-4596-8e25-03dfbd7765e4-sg-core-conf-yaml\") on node \"crc\" DevicePath \"\"" Dec 03 11:59:35 crc kubenswrapper[5052]: I1203 11:59:35.668591 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/23026a47-055f-4596-8e25-03dfbd7765e4-config-data" (OuterVolumeSpecName: "config-data") pod "23026a47-055f-4596-8e25-03dfbd7765e4" (UID: "23026a47-055f-4596-8e25-03dfbd7765e4"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:59:35 crc kubenswrapper[5052]: I1203 11:59:35.675385 5052 scope.go:117] "RemoveContainer" containerID="d224717c6d9ca8135fd510e03a6c40c9f2b1a56dd8b4a3b685af3e2dfa855982" Dec 03 11:59:35 crc kubenswrapper[5052]: E1203 11:59:35.675753 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 11:59:35 crc kubenswrapper[5052]: I1203 11:59:35.744006 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/23026a47-055f-4596-8e25-03dfbd7765e4-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:59:35 crc kubenswrapper[5052]: I1203 11:59:35.897914 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"23026a47-055f-4596-8e25-03dfbd7765e4","Type":"ContainerDied","Data":"7f043a7498d72acf4491cc6f58eb62909f60b19130837936092120fddf686c96"} Dec 03 11:59:35 crc kubenswrapper[5052]: I1203 11:59:35.897956 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:59:35 crc kubenswrapper[5052]: I1203 11:59:35.897976 5052 scope.go:117] "RemoveContainer" containerID="288d258a1bd6e87ebfe9f5f08dcae443f876b9972488a68c4760a4636e7320ff" Dec 03 11:59:35 crc kubenswrapper[5052]: I1203 11:59:35.932195 5052 scope.go:117] "RemoveContainer" containerID="bff805d09e466c6e21a67bec1309612d4fa585685f819fb9edbfd91df7e9548d" Dec 03 11:59:35 crc kubenswrapper[5052]: I1203 11:59:35.937728 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:59:35 crc kubenswrapper[5052]: I1203 11:59:35.955850 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:59:35 crc kubenswrapper[5052]: I1203 11:59:35.968424 5052 scope.go:117] "RemoveContainer" containerID="7a059e1f453d9043b751ae5f5f2d609e9a943a78ec08c26d04dafa23dd2ed753" Dec 03 11:59:35 crc kubenswrapper[5052]: I1203 11:59:35.973586 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:59:35 crc kubenswrapper[5052]: E1203 11:59:35.974136 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="23026a47-055f-4596-8e25-03dfbd7765e4" containerName="ceilometer-central-agent" Dec 03 11:59:35 crc kubenswrapper[5052]: I1203 11:59:35.974159 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="23026a47-055f-4596-8e25-03dfbd7765e4" containerName="ceilometer-central-agent" Dec 03 11:59:35 crc kubenswrapper[5052]: E1203 11:59:35.974182 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="23026a47-055f-4596-8e25-03dfbd7765e4" containerName="proxy-httpd" Dec 03 11:59:35 crc kubenswrapper[5052]: I1203 11:59:35.974193 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="23026a47-055f-4596-8e25-03dfbd7765e4" containerName="proxy-httpd" Dec 03 11:59:35 crc kubenswrapper[5052]: E1203 11:59:35.974210 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="23026a47-055f-4596-8e25-03dfbd7765e4" containerName="ceilometer-notification-agent" Dec 03 11:59:35 crc kubenswrapper[5052]: I1203 11:59:35.974220 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="23026a47-055f-4596-8e25-03dfbd7765e4" containerName="ceilometer-notification-agent" Dec 03 11:59:35 crc kubenswrapper[5052]: E1203 11:59:35.974261 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="23026a47-055f-4596-8e25-03dfbd7765e4" containerName="sg-core" Dec 03 11:59:35 crc kubenswrapper[5052]: I1203 11:59:35.974270 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="23026a47-055f-4596-8e25-03dfbd7765e4" containerName="sg-core" Dec 03 11:59:35 crc kubenswrapper[5052]: I1203 11:59:35.974487 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="23026a47-055f-4596-8e25-03dfbd7765e4" containerName="sg-core" Dec 03 11:59:35 crc kubenswrapper[5052]: I1203 11:59:35.974526 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="23026a47-055f-4596-8e25-03dfbd7765e4" containerName="proxy-httpd" Dec 03 11:59:35 crc kubenswrapper[5052]: I1203 11:59:35.974549 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="23026a47-055f-4596-8e25-03dfbd7765e4" containerName="ceilometer-notification-agent" Dec 03 11:59:35 crc kubenswrapper[5052]: I1203 11:59:35.974570 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="23026a47-055f-4596-8e25-03dfbd7765e4" containerName="ceilometer-central-agent" Dec 03 11:59:35 crc kubenswrapper[5052]: I1203 11:59:35.979343 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:59:35 crc kubenswrapper[5052]: I1203 11:59:35.983780 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-scripts" Dec 03 11:59:35 crc kubenswrapper[5052]: I1203 11:59:35.984727 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-config-data" Dec 03 11:59:35 crc kubenswrapper[5052]: I1203 11:59:35.995103 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:59:36 crc kubenswrapper[5052]: I1203 11:59:36.013209 5052 scope.go:117] "RemoveContainer" containerID="715956d36abed25c5075c9bdd118f9c4b5e07adf089b0e30a1fb3a171e6d463c" Dec 03 11:59:36 crc kubenswrapper[5052]: I1203 11:59:36.050267 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/b2a6ebec-b059-4e69-a8b3-6e8834d0e7d4-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"b2a6ebec-b059-4e69-a8b3-6e8834d0e7d4\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:59:36 crc kubenswrapper[5052]: I1203 11:59:36.050838 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bkqhk\" (UniqueName: \"kubernetes.io/projected/b2a6ebec-b059-4e69-a8b3-6e8834d0e7d4-kube-api-access-bkqhk\") pod \"ceilometer-0\" (UID: \"b2a6ebec-b059-4e69-a8b3-6e8834d0e7d4\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:59:36 crc kubenswrapper[5052]: I1203 11:59:36.051037 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b2a6ebec-b059-4e69-a8b3-6e8834d0e7d4-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"b2a6ebec-b059-4e69-a8b3-6e8834d0e7d4\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:59:36 crc kubenswrapper[5052]: I1203 11:59:36.051156 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/b2a6ebec-b059-4e69-a8b3-6e8834d0e7d4-scripts\") pod \"ceilometer-0\" (UID: \"b2a6ebec-b059-4e69-a8b3-6e8834d0e7d4\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:59:36 crc kubenswrapper[5052]: I1203 11:59:36.051298 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/b2a6ebec-b059-4e69-a8b3-6e8834d0e7d4-run-httpd\") pod \"ceilometer-0\" (UID: \"b2a6ebec-b059-4e69-a8b3-6e8834d0e7d4\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:59:36 crc kubenswrapper[5052]: I1203 11:59:36.051446 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/b2a6ebec-b059-4e69-a8b3-6e8834d0e7d4-log-httpd\") pod \"ceilometer-0\" (UID: \"b2a6ebec-b059-4e69-a8b3-6e8834d0e7d4\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:59:36 crc kubenswrapper[5052]: I1203 11:59:36.051581 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/b2a6ebec-b059-4e69-a8b3-6e8834d0e7d4-config-data\") pod \"ceilometer-0\" (UID: \"b2a6ebec-b059-4e69-a8b3-6e8834d0e7d4\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:59:36 crc kubenswrapper[5052]: I1203 11:59:36.153083 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b2a6ebec-b059-4e69-a8b3-6e8834d0e7d4-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"b2a6ebec-b059-4e69-a8b3-6e8834d0e7d4\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:59:36 crc kubenswrapper[5052]: I1203 11:59:36.153142 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/b2a6ebec-b059-4e69-a8b3-6e8834d0e7d4-scripts\") pod \"ceilometer-0\" (UID: \"b2a6ebec-b059-4e69-a8b3-6e8834d0e7d4\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:59:36 crc kubenswrapper[5052]: I1203 11:59:36.153198 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/b2a6ebec-b059-4e69-a8b3-6e8834d0e7d4-run-httpd\") pod \"ceilometer-0\" (UID: \"b2a6ebec-b059-4e69-a8b3-6e8834d0e7d4\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:59:36 crc kubenswrapper[5052]: I1203 11:59:36.153244 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/b2a6ebec-b059-4e69-a8b3-6e8834d0e7d4-log-httpd\") pod \"ceilometer-0\" (UID: \"b2a6ebec-b059-4e69-a8b3-6e8834d0e7d4\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:59:36 crc kubenswrapper[5052]: I1203 11:59:36.153288 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/b2a6ebec-b059-4e69-a8b3-6e8834d0e7d4-config-data\") pod \"ceilometer-0\" (UID: \"b2a6ebec-b059-4e69-a8b3-6e8834d0e7d4\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:59:36 crc kubenswrapper[5052]: I1203 11:59:36.153340 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/b2a6ebec-b059-4e69-a8b3-6e8834d0e7d4-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"b2a6ebec-b059-4e69-a8b3-6e8834d0e7d4\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:59:36 crc kubenswrapper[5052]: I1203 11:59:36.153388 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-bkqhk\" (UniqueName: \"kubernetes.io/projected/b2a6ebec-b059-4e69-a8b3-6e8834d0e7d4-kube-api-access-bkqhk\") pod \"ceilometer-0\" (UID: \"b2a6ebec-b059-4e69-a8b3-6e8834d0e7d4\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:59:36 crc kubenswrapper[5052]: I1203 11:59:36.153785 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/b2a6ebec-b059-4e69-a8b3-6e8834d0e7d4-run-httpd\") pod \"ceilometer-0\" (UID: \"b2a6ebec-b059-4e69-a8b3-6e8834d0e7d4\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:59:36 crc kubenswrapper[5052]: I1203 11:59:36.153814 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/b2a6ebec-b059-4e69-a8b3-6e8834d0e7d4-log-httpd\") pod \"ceilometer-0\" (UID: \"b2a6ebec-b059-4e69-a8b3-6e8834d0e7d4\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:59:36 crc kubenswrapper[5052]: I1203 11:59:36.157892 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/b2a6ebec-b059-4e69-a8b3-6e8834d0e7d4-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"b2a6ebec-b059-4e69-a8b3-6e8834d0e7d4\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:59:36 crc kubenswrapper[5052]: I1203 11:59:36.158187 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/b2a6ebec-b059-4e69-a8b3-6e8834d0e7d4-config-data\") pod \"ceilometer-0\" (UID: \"b2a6ebec-b059-4e69-a8b3-6e8834d0e7d4\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:59:36 crc kubenswrapper[5052]: I1203 11:59:36.158380 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b2a6ebec-b059-4e69-a8b3-6e8834d0e7d4-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"b2a6ebec-b059-4e69-a8b3-6e8834d0e7d4\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:59:36 crc kubenswrapper[5052]: I1203 11:59:36.163591 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/b2a6ebec-b059-4e69-a8b3-6e8834d0e7d4-scripts\") pod \"ceilometer-0\" (UID: \"b2a6ebec-b059-4e69-a8b3-6e8834d0e7d4\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:59:36 crc kubenswrapper[5052]: I1203 11:59:36.172265 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-bkqhk\" (UniqueName: \"kubernetes.io/projected/b2a6ebec-b059-4e69-a8b3-6e8834d0e7d4-kube-api-access-bkqhk\") pod \"ceilometer-0\" (UID: \"b2a6ebec-b059-4e69-a8b3-6e8834d0e7d4\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:59:36 crc kubenswrapper[5052]: I1203 11:59:36.313345 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:59:36 crc kubenswrapper[5052]: I1203 11:59:36.761260 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 11:59:36 crc kubenswrapper[5052]: I1203 11:59:36.907935 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"b2a6ebec-b059-4e69-a8b3-6e8834d0e7d4","Type":"ContainerStarted","Data":"f060bcdf30dd614f6d54a5afeed8290132a9c6f0617546e94ac296834031acdc"} Dec 03 11:59:37 crc kubenswrapper[5052]: I1203 11:59:37.689250 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="23026a47-055f-4596-8e25-03dfbd7765e4" path="/var/lib/kubelet/pods/23026a47-055f-4596-8e25-03dfbd7765e4/volumes" Dec 03 11:59:37 crc kubenswrapper[5052]: I1203 11:59:37.923128 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"b2a6ebec-b059-4e69-a8b3-6e8834d0e7d4","Type":"ContainerStarted","Data":"86af6e57ea77f80fbae559523013c161c284dc908293fced86d0ca28d47bfa88"} Dec 03 11:59:38 crc kubenswrapper[5052]: I1203 11:59:38.935640 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"b2a6ebec-b059-4e69-a8b3-6e8834d0e7d4","Type":"ContainerStarted","Data":"13b86d68aa550db22118a91df9a6d966c050dc8fb26964acf4c3f503015a6bfd"} Dec 03 11:59:39 crc kubenswrapper[5052]: I1203 11:59:39.946666 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"b2a6ebec-b059-4e69-a8b3-6e8834d0e7d4","Type":"ContainerStarted","Data":"82080fd1b638a84bacd26b6d91ce32afe1b5d62e8dd19694f376960bcc7f7a12"} Dec 03 11:59:41 crc kubenswrapper[5052]: I1203 11:59:41.967793 5052 generic.go:334] "Generic (PLEG): container finished" podID="820f9385-7816-4259-987f-15eef2e78099" containerID="ae2dd9c7d6d2cf6767ca2ef1b07d9dc82f31acfcfdbfe9432084c6de9851a9f0" exitCode=0 Dec 03 11:59:41 crc kubenswrapper[5052]: I1203 11:59:41.967863 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-nljtg" event={"ID":"820f9385-7816-4259-987f-15eef2e78099","Type":"ContainerDied","Data":"ae2dd9c7d6d2cf6767ca2ef1b07d9dc82f31acfcfdbfe9432084c6de9851a9f0"} Dec 03 11:59:41 crc kubenswrapper[5052]: I1203 11:59:41.975092 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"b2a6ebec-b059-4e69-a8b3-6e8834d0e7d4","Type":"ContainerStarted","Data":"8dbc153f9b45d32a17f2e4f6457f2cd39bf8f659e8f3ca16ed4576efd8c7aca2"} Dec 03 11:59:41 crc kubenswrapper[5052]: I1203 11:59:41.975327 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 11:59:43 crc kubenswrapper[5052]: I1203 11:59:43.323101 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-nljtg" Dec 03 11:59:43 crc kubenswrapper[5052]: I1203 11:59:43.341018 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/ceilometer-0" podStartSLOduration=3.915244506 podStartE2EDuration="8.340998645s" podCreationTimestamp="2025-12-03 11:59:35 +0000 UTC" firstStartedPulling="2025-12-03 11:59:36.762744865 +0000 UTC m=+2367.350986629" lastFinishedPulling="2025-12-03 11:59:41.188499004 +0000 UTC m=+2371.776740768" observedRunningTime="2025-12-03 11:59:42.018291881 +0000 UTC m=+2372.606533635" watchObservedRunningTime="2025-12-03 11:59:43.340998645 +0000 UTC m=+2373.929240409" Dec 03 11:59:43 crc kubenswrapper[5052]: I1203 11:59:43.396283 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bhkx6\" (UniqueName: \"kubernetes.io/projected/820f9385-7816-4259-987f-15eef2e78099-kube-api-access-bhkx6\") pod \"820f9385-7816-4259-987f-15eef2e78099\" (UID: \"820f9385-7816-4259-987f-15eef2e78099\") " Dec 03 11:59:43 crc kubenswrapper[5052]: I1203 11:59:43.396392 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/820f9385-7816-4259-987f-15eef2e78099-combined-ca-bundle\") pod \"820f9385-7816-4259-987f-15eef2e78099\" (UID: \"820f9385-7816-4259-987f-15eef2e78099\") " Dec 03 11:59:43 crc kubenswrapper[5052]: I1203 11:59:43.396450 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/820f9385-7816-4259-987f-15eef2e78099-config-data\") pod \"820f9385-7816-4259-987f-15eef2e78099\" (UID: \"820f9385-7816-4259-987f-15eef2e78099\") " Dec 03 11:59:43 crc kubenswrapper[5052]: I1203 11:59:43.396559 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/820f9385-7816-4259-987f-15eef2e78099-scripts\") pod \"820f9385-7816-4259-987f-15eef2e78099\" (UID: \"820f9385-7816-4259-987f-15eef2e78099\") " Dec 03 11:59:43 crc kubenswrapper[5052]: I1203 11:59:43.402993 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/820f9385-7816-4259-987f-15eef2e78099-scripts" (OuterVolumeSpecName: "scripts") pod "820f9385-7816-4259-987f-15eef2e78099" (UID: "820f9385-7816-4259-987f-15eef2e78099"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:59:43 crc kubenswrapper[5052]: I1203 11:59:43.404079 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/820f9385-7816-4259-987f-15eef2e78099-kube-api-access-bhkx6" (OuterVolumeSpecName: "kube-api-access-bhkx6") pod "820f9385-7816-4259-987f-15eef2e78099" (UID: "820f9385-7816-4259-987f-15eef2e78099"). InnerVolumeSpecName "kube-api-access-bhkx6". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:59:43 crc kubenswrapper[5052]: I1203 11:59:43.426666 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/820f9385-7816-4259-987f-15eef2e78099-config-data" (OuterVolumeSpecName: "config-data") pod "820f9385-7816-4259-987f-15eef2e78099" (UID: "820f9385-7816-4259-987f-15eef2e78099"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:59:43 crc kubenswrapper[5052]: I1203 11:59:43.429573 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/820f9385-7816-4259-987f-15eef2e78099-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "820f9385-7816-4259-987f-15eef2e78099" (UID: "820f9385-7816-4259-987f-15eef2e78099"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:59:43 crc kubenswrapper[5052]: I1203 11:59:43.498400 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/820f9385-7816-4259-987f-15eef2e78099-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:59:43 crc kubenswrapper[5052]: I1203 11:59:43.498437 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/820f9385-7816-4259-987f-15eef2e78099-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:59:43 crc kubenswrapper[5052]: I1203 11:59:43.498445 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/820f9385-7816-4259-987f-15eef2e78099-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:59:43 crc kubenswrapper[5052]: I1203 11:59:43.498454 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-bhkx6\" (UniqueName: \"kubernetes.io/projected/820f9385-7816-4259-987f-15eef2e78099-kube-api-access-bhkx6\") on node \"crc\" DevicePath \"\"" Dec 03 11:59:44 crc kubenswrapper[5052]: I1203 11:59:44.006238 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-nljtg" event={"ID":"820f9385-7816-4259-987f-15eef2e78099","Type":"ContainerDied","Data":"12483860bbb96968ef2ae1cdf1d847f67aa9f19683b97080b5df16fd4ef4119c"} Dec 03 11:59:44 crc kubenswrapper[5052]: I1203 11:59:44.006324 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="12483860bbb96968ef2ae1cdf1d847f67aa9f19683b97080b5df16fd4ef4119c" Dec 03 11:59:44 crc kubenswrapper[5052]: I1203 11:59:44.006439 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-nljtg" Dec 03 11:59:44 crc kubenswrapper[5052]: I1203 11:59:44.562809 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell0-cell-mapping-fc2zk"] Dec 03 11:59:44 crc kubenswrapper[5052]: E1203 11:59:44.563955 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="820f9385-7816-4259-987f-15eef2e78099" containerName="nova-cell0-conductor-db-sync" Dec 03 11:59:44 crc kubenswrapper[5052]: I1203 11:59:44.563992 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="820f9385-7816-4259-987f-15eef2e78099" containerName="nova-cell0-conductor-db-sync" Dec 03 11:59:44 crc kubenswrapper[5052]: I1203 11:59:44.564461 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="820f9385-7816-4259-987f-15eef2e78099" containerName="nova-cell0-conductor-db-sync" Dec 03 11:59:44 crc kubenswrapper[5052]: I1203 11:59:44.565711 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-cell-mapping-fc2zk" Dec 03 11:59:44 crc kubenswrapper[5052]: I1203 11:59:44.568192 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-nova-dockercfg-zwdn9" Dec 03 11:59:44 crc kubenswrapper[5052]: I1203 11:59:44.569110 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell0-manage-scripts" Dec 03 11:59:44 crc kubenswrapper[5052]: I1203 11:59:44.570955 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell0-manage-config-data" Dec 03 11:59:44 crc kubenswrapper[5052]: I1203 11:59:44.574328 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell0-cell-mapping-fc2zk"] Dec 03 11:59:44 crc kubenswrapper[5052]: I1203 11:59:44.623248 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/03284adf-04cd-4ca4-8b31-7ae483082bba-scripts\") pod \"nova-cell0-cell-mapping-fc2zk\" (UID: \"03284adf-04cd-4ca4-8b31-7ae483082bba\") " pod="openstack-kuttl-tests/nova-cell0-cell-mapping-fc2zk" Dec 03 11:59:44 crc kubenswrapper[5052]: I1203 11:59:44.623656 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/03284adf-04cd-4ca4-8b31-7ae483082bba-combined-ca-bundle\") pod \"nova-cell0-cell-mapping-fc2zk\" (UID: \"03284adf-04cd-4ca4-8b31-7ae483082bba\") " pod="openstack-kuttl-tests/nova-cell0-cell-mapping-fc2zk" Dec 03 11:59:44 crc kubenswrapper[5052]: I1203 11:59:44.623815 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jrvgv\" (UniqueName: \"kubernetes.io/projected/03284adf-04cd-4ca4-8b31-7ae483082bba-kube-api-access-jrvgv\") pod \"nova-cell0-cell-mapping-fc2zk\" (UID: \"03284adf-04cd-4ca4-8b31-7ae483082bba\") " pod="openstack-kuttl-tests/nova-cell0-cell-mapping-fc2zk" Dec 03 11:59:44 crc kubenswrapper[5052]: I1203 11:59:44.624063 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/03284adf-04cd-4ca4-8b31-7ae483082bba-config-data\") pod \"nova-cell0-cell-mapping-fc2zk\" (UID: \"03284adf-04cd-4ca4-8b31-7ae483082bba\") " pod="openstack-kuttl-tests/nova-cell0-cell-mapping-fc2zk" Dec 03 11:59:44 crc kubenswrapper[5052]: I1203 11:59:44.725769 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/03284adf-04cd-4ca4-8b31-7ae483082bba-config-data\") pod \"nova-cell0-cell-mapping-fc2zk\" (UID: \"03284adf-04cd-4ca4-8b31-7ae483082bba\") " pod="openstack-kuttl-tests/nova-cell0-cell-mapping-fc2zk" Dec 03 11:59:44 crc kubenswrapper[5052]: I1203 11:59:44.725919 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/03284adf-04cd-4ca4-8b31-7ae483082bba-scripts\") pod \"nova-cell0-cell-mapping-fc2zk\" (UID: \"03284adf-04cd-4ca4-8b31-7ae483082bba\") " pod="openstack-kuttl-tests/nova-cell0-cell-mapping-fc2zk" Dec 03 11:59:44 crc kubenswrapper[5052]: I1203 11:59:44.726009 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/03284adf-04cd-4ca4-8b31-7ae483082bba-combined-ca-bundle\") pod \"nova-cell0-cell-mapping-fc2zk\" (UID: \"03284adf-04cd-4ca4-8b31-7ae483082bba\") " pod="openstack-kuttl-tests/nova-cell0-cell-mapping-fc2zk" Dec 03 11:59:44 crc kubenswrapper[5052]: I1203 11:59:44.726033 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-jrvgv\" (UniqueName: \"kubernetes.io/projected/03284adf-04cd-4ca4-8b31-7ae483082bba-kube-api-access-jrvgv\") pod \"nova-cell0-cell-mapping-fc2zk\" (UID: \"03284adf-04cd-4ca4-8b31-7ae483082bba\") " pod="openstack-kuttl-tests/nova-cell0-cell-mapping-fc2zk" Dec 03 11:59:44 crc kubenswrapper[5052]: I1203 11:59:44.741399 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/03284adf-04cd-4ca4-8b31-7ae483082bba-config-data\") pod \"nova-cell0-cell-mapping-fc2zk\" (UID: \"03284adf-04cd-4ca4-8b31-7ae483082bba\") " pod="openstack-kuttl-tests/nova-cell0-cell-mapping-fc2zk" Dec 03 11:59:44 crc kubenswrapper[5052]: I1203 11:59:44.744992 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/03284adf-04cd-4ca4-8b31-7ae483082bba-scripts\") pod \"nova-cell0-cell-mapping-fc2zk\" (UID: \"03284adf-04cd-4ca4-8b31-7ae483082bba\") " pod="openstack-kuttl-tests/nova-cell0-cell-mapping-fc2zk" Dec 03 11:59:44 crc kubenswrapper[5052]: I1203 11:59:44.749049 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 11:59:44 crc kubenswrapper[5052]: I1203 11:59:44.750787 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:59:44 crc kubenswrapper[5052]: I1203 11:59:44.759346 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-api-config-data" Dec 03 11:59:44 crc kubenswrapper[5052]: I1203 11:59:44.760054 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/03284adf-04cd-4ca4-8b31-7ae483082bba-combined-ca-bundle\") pod \"nova-cell0-cell-mapping-fc2zk\" (UID: \"03284adf-04cd-4ca4-8b31-7ae483082bba\") " pod="openstack-kuttl-tests/nova-cell0-cell-mapping-fc2zk" Dec 03 11:59:44 crc kubenswrapper[5052]: I1203 11:59:44.784286 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 11:59:44 crc kubenswrapper[5052]: I1203 11:59:44.790325 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-jrvgv\" (UniqueName: \"kubernetes.io/projected/03284adf-04cd-4ca4-8b31-7ae483082bba-kube-api-access-jrvgv\") pod \"nova-cell0-cell-mapping-fc2zk\" (UID: \"03284adf-04cd-4ca4-8b31-7ae483082bba\") " pod="openstack-kuttl-tests/nova-cell0-cell-mapping-fc2zk" Dec 03 11:59:44 crc kubenswrapper[5052]: I1203 11:59:44.810869 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 11:59:44 crc kubenswrapper[5052]: I1203 11:59:44.811036 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:59:44 crc kubenswrapper[5052]: I1203 11:59:44.812915 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-metadata-config-data" Dec 03 11:59:44 crc kubenswrapper[5052]: I1203 11:59:44.827485 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/531ff22d-8820-443e-b1ac-5d1335c0691a-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"531ff22d-8820-443e-b1ac-5d1335c0691a\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:59:44 crc kubenswrapper[5052]: I1203 11:59:44.827575 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/ede50db6-6b4c-4dd5-9f79-cd76d5de6a99-config-data\") pod \"nova-api-0\" (UID: \"ede50db6-6b4c-4dd5-9f79-cd76d5de6a99\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:59:44 crc kubenswrapper[5052]: I1203 11:59:44.827609 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q6pbq\" (UniqueName: \"kubernetes.io/projected/ede50db6-6b4c-4dd5-9f79-cd76d5de6a99-kube-api-access-q6pbq\") pod \"nova-api-0\" (UID: \"ede50db6-6b4c-4dd5-9f79-cd76d5de6a99\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:59:44 crc kubenswrapper[5052]: I1203 11:59:44.827666 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8dktw\" (UniqueName: \"kubernetes.io/projected/531ff22d-8820-443e-b1ac-5d1335c0691a-kube-api-access-8dktw\") pod \"nova-metadata-0\" (UID: \"531ff22d-8820-443e-b1ac-5d1335c0691a\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:59:44 crc kubenswrapper[5052]: I1203 11:59:44.827689 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/531ff22d-8820-443e-b1ac-5d1335c0691a-logs\") pod \"nova-metadata-0\" (UID: \"531ff22d-8820-443e-b1ac-5d1335c0691a\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:59:44 crc kubenswrapper[5052]: I1203 11:59:44.827719 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/531ff22d-8820-443e-b1ac-5d1335c0691a-config-data\") pod \"nova-metadata-0\" (UID: \"531ff22d-8820-443e-b1ac-5d1335c0691a\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:59:44 crc kubenswrapper[5052]: I1203 11:59:44.827737 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ede50db6-6b4c-4dd5-9f79-cd76d5de6a99-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"ede50db6-6b4c-4dd5-9f79-cd76d5de6a99\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:59:44 crc kubenswrapper[5052]: I1203 11:59:44.827779 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/ede50db6-6b4c-4dd5-9f79-cd76d5de6a99-logs\") pod \"nova-api-0\" (UID: \"ede50db6-6b4c-4dd5-9f79-cd76d5de6a99\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:59:44 crc kubenswrapper[5052]: I1203 11:59:44.831594 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 11:59:44 crc kubenswrapper[5052]: I1203 11:59:44.879760 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 11:59:44 crc kubenswrapper[5052]: I1203 11:59:44.881150 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 11:59:44 crc kubenswrapper[5052]: I1203 11:59:44.883291 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell1-novncproxy-config-data" Dec 03 11:59:44 crc kubenswrapper[5052]: I1203 11:59:44.893011 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-cell-mapping-fc2zk" Dec 03 11:59:44 crc kubenswrapper[5052]: I1203 11:59:44.916669 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 11:59:44 crc kubenswrapper[5052]: I1203 11:59:44.930179 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/531ff22d-8820-443e-b1ac-5d1335c0691a-config-data\") pod \"nova-metadata-0\" (UID: \"531ff22d-8820-443e-b1ac-5d1335c0691a\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:59:44 crc kubenswrapper[5052]: I1203 11:59:44.930221 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ede50db6-6b4c-4dd5-9f79-cd76d5de6a99-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"ede50db6-6b4c-4dd5-9f79-cd76d5de6a99\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:59:44 crc kubenswrapper[5052]: I1203 11:59:44.930275 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/ede50db6-6b4c-4dd5-9f79-cd76d5de6a99-logs\") pod \"nova-api-0\" (UID: \"ede50db6-6b4c-4dd5-9f79-cd76d5de6a99\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:59:44 crc kubenswrapper[5052]: I1203 11:59:44.930302 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/531ff22d-8820-443e-b1ac-5d1335c0691a-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"531ff22d-8820-443e-b1ac-5d1335c0691a\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:59:44 crc kubenswrapper[5052]: I1203 11:59:44.930326 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/ede50db6-6b4c-4dd5-9f79-cd76d5de6a99-config-data\") pod \"nova-api-0\" (UID: \"ede50db6-6b4c-4dd5-9f79-cd76d5de6a99\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:59:44 crc kubenswrapper[5052]: I1203 11:59:44.930356 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/1f41370a-19d0-4c2d-b942-26562f752813-config-data\") pod \"nova-cell1-novncproxy-0\" (UID: \"1f41370a-19d0-4c2d-b942-26562f752813\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 11:59:44 crc kubenswrapper[5052]: I1203 11:59:44.930387 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-q6pbq\" (UniqueName: \"kubernetes.io/projected/ede50db6-6b4c-4dd5-9f79-cd76d5de6a99-kube-api-access-q6pbq\") pod \"nova-api-0\" (UID: \"ede50db6-6b4c-4dd5-9f79-cd76d5de6a99\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:59:44 crc kubenswrapper[5052]: I1203 11:59:44.930405 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-brl7s\" (UniqueName: \"kubernetes.io/projected/1f41370a-19d0-4c2d-b942-26562f752813-kube-api-access-brl7s\") pod \"nova-cell1-novncproxy-0\" (UID: \"1f41370a-19d0-4c2d-b942-26562f752813\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 11:59:44 crc kubenswrapper[5052]: I1203 11:59:44.930445 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1f41370a-19d0-4c2d-b942-26562f752813-combined-ca-bundle\") pod \"nova-cell1-novncproxy-0\" (UID: \"1f41370a-19d0-4c2d-b942-26562f752813\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 11:59:44 crc kubenswrapper[5052]: I1203 11:59:44.930481 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-8dktw\" (UniqueName: \"kubernetes.io/projected/531ff22d-8820-443e-b1ac-5d1335c0691a-kube-api-access-8dktw\") pod \"nova-metadata-0\" (UID: \"531ff22d-8820-443e-b1ac-5d1335c0691a\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:59:44 crc kubenswrapper[5052]: I1203 11:59:44.930523 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/531ff22d-8820-443e-b1ac-5d1335c0691a-logs\") pod \"nova-metadata-0\" (UID: \"531ff22d-8820-443e-b1ac-5d1335c0691a\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:59:44 crc kubenswrapper[5052]: I1203 11:59:44.930924 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/531ff22d-8820-443e-b1ac-5d1335c0691a-logs\") pod \"nova-metadata-0\" (UID: \"531ff22d-8820-443e-b1ac-5d1335c0691a\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:59:44 crc kubenswrapper[5052]: I1203 11:59:44.935786 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/ede50db6-6b4c-4dd5-9f79-cd76d5de6a99-logs\") pod \"nova-api-0\" (UID: \"ede50db6-6b4c-4dd5-9f79-cd76d5de6a99\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:59:44 crc kubenswrapper[5052]: I1203 11:59:44.944207 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/531ff22d-8820-443e-b1ac-5d1335c0691a-config-data\") pod \"nova-metadata-0\" (UID: \"531ff22d-8820-443e-b1ac-5d1335c0691a\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:59:44 crc kubenswrapper[5052]: I1203 11:59:44.946774 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 11:59:44 crc kubenswrapper[5052]: I1203 11:59:44.954268 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/531ff22d-8820-443e-b1ac-5d1335c0691a-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"531ff22d-8820-443e-b1ac-5d1335c0691a\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:59:44 crc kubenswrapper[5052]: I1203 11:59:44.958851 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:59:44 crc kubenswrapper[5052]: I1203 11:59:44.970041 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-scheduler-config-data" Dec 03 11:59:44 crc kubenswrapper[5052]: I1203 11:59:44.970215 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ede50db6-6b4c-4dd5-9f79-cd76d5de6a99-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"ede50db6-6b4c-4dd5-9f79-cd76d5de6a99\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:59:44 crc kubenswrapper[5052]: I1203 11:59:44.970286 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 11:59:44 crc kubenswrapper[5052]: I1203 11:59:44.970941 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/ede50db6-6b4c-4dd5-9f79-cd76d5de6a99-config-data\") pod \"nova-api-0\" (UID: \"ede50db6-6b4c-4dd5-9f79-cd76d5de6a99\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:59:44 crc kubenswrapper[5052]: I1203 11:59:44.971526 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-q6pbq\" (UniqueName: \"kubernetes.io/projected/ede50db6-6b4c-4dd5-9f79-cd76d5de6a99-kube-api-access-q6pbq\") pod \"nova-api-0\" (UID: \"ede50db6-6b4c-4dd5-9f79-cd76d5de6a99\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:59:44 crc kubenswrapper[5052]: I1203 11:59:44.971890 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-8dktw\" (UniqueName: \"kubernetes.io/projected/531ff22d-8820-443e-b1ac-5d1335c0691a-kube-api-access-8dktw\") pod \"nova-metadata-0\" (UID: \"531ff22d-8820-443e-b1ac-5d1335c0691a\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:59:45 crc kubenswrapper[5052]: I1203 11:59:45.032792 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-brl7s\" (UniqueName: \"kubernetes.io/projected/1f41370a-19d0-4c2d-b942-26562f752813-kube-api-access-brl7s\") pod \"nova-cell1-novncproxy-0\" (UID: \"1f41370a-19d0-4c2d-b942-26562f752813\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 11:59:45 crc kubenswrapper[5052]: I1203 11:59:45.036250 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1f41370a-19d0-4c2d-b942-26562f752813-combined-ca-bundle\") pod \"nova-cell1-novncproxy-0\" (UID: \"1f41370a-19d0-4c2d-b942-26562f752813\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 11:59:45 crc kubenswrapper[5052]: I1203 11:59:45.039351 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/79b9410f-a5f6-48ab-a697-0aa515754ab0-combined-ca-bundle\") pod \"nova-scheduler-0\" (UID: \"79b9410f-a5f6-48ab-a697-0aa515754ab0\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:59:45 crc kubenswrapper[5052]: I1203 11:59:45.039527 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/79b9410f-a5f6-48ab-a697-0aa515754ab0-config-data\") pod \"nova-scheduler-0\" (UID: \"79b9410f-a5f6-48ab-a697-0aa515754ab0\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:59:45 crc kubenswrapper[5052]: I1203 11:59:45.039740 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9fkmg\" (UniqueName: \"kubernetes.io/projected/79b9410f-a5f6-48ab-a697-0aa515754ab0-kube-api-access-9fkmg\") pod \"nova-scheduler-0\" (UID: \"79b9410f-a5f6-48ab-a697-0aa515754ab0\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:59:45 crc kubenswrapper[5052]: I1203 11:59:45.039905 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/1f41370a-19d0-4c2d-b942-26562f752813-config-data\") pod \"nova-cell1-novncproxy-0\" (UID: \"1f41370a-19d0-4c2d-b942-26562f752813\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 11:59:45 crc kubenswrapper[5052]: I1203 11:59:45.046401 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1f41370a-19d0-4c2d-b942-26562f752813-combined-ca-bundle\") pod \"nova-cell1-novncproxy-0\" (UID: \"1f41370a-19d0-4c2d-b942-26562f752813\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 11:59:45 crc kubenswrapper[5052]: I1203 11:59:45.049295 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/1f41370a-19d0-4c2d-b942-26562f752813-config-data\") pod \"nova-cell1-novncproxy-0\" (UID: \"1f41370a-19d0-4c2d-b942-26562f752813\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 11:59:45 crc kubenswrapper[5052]: I1203 11:59:45.057043 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-brl7s\" (UniqueName: \"kubernetes.io/projected/1f41370a-19d0-4c2d-b942-26562f752813-kube-api-access-brl7s\") pod \"nova-cell1-novncproxy-0\" (UID: \"1f41370a-19d0-4c2d-b942-26562f752813\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 11:59:45 crc kubenswrapper[5052]: I1203 11:59:45.141853 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/79b9410f-a5f6-48ab-a697-0aa515754ab0-config-data\") pod \"nova-scheduler-0\" (UID: \"79b9410f-a5f6-48ab-a697-0aa515754ab0\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:59:45 crc kubenswrapper[5052]: I1203 11:59:45.141927 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-9fkmg\" (UniqueName: \"kubernetes.io/projected/79b9410f-a5f6-48ab-a697-0aa515754ab0-kube-api-access-9fkmg\") pod \"nova-scheduler-0\" (UID: \"79b9410f-a5f6-48ab-a697-0aa515754ab0\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:59:45 crc kubenswrapper[5052]: I1203 11:59:45.142055 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/79b9410f-a5f6-48ab-a697-0aa515754ab0-combined-ca-bundle\") pod \"nova-scheduler-0\" (UID: \"79b9410f-a5f6-48ab-a697-0aa515754ab0\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:59:45 crc kubenswrapper[5052]: I1203 11:59:45.146057 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/79b9410f-a5f6-48ab-a697-0aa515754ab0-combined-ca-bundle\") pod \"nova-scheduler-0\" (UID: \"79b9410f-a5f6-48ab-a697-0aa515754ab0\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:59:45 crc kubenswrapper[5052]: I1203 11:59:45.154464 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/79b9410f-a5f6-48ab-a697-0aa515754ab0-config-data\") pod \"nova-scheduler-0\" (UID: \"79b9410f-a5f6-48ab-a697-0aa515754ab0\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:59:45 crc kubenswrapper[5052]: I1203 11:59:45.161184 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-9fkmg\" (UniqueName: \"kubernetes.io/projected/79b9410f-a5f6-48ab-a697-0aa515754ab0-kube-api-access-9fkmg\") pod \"nova-scheduler-0\" (UID: \"79b9410f-a5f6-48ab-a697-0aa515754ab0\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:59:45 crc kubenswrapper[5052]: I1203 11:59:45.181026 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:59:45 crc kubenswrapper[5052]: I1203 11:59:45.198356 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:59:45 crc kubenswrapper[5052]: I1203 11:59:45.212120 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 11:59:46 crc kubenswrapper[5052]: I1203 11:59:45.365777 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:59:46 crc kubenswrapper[5052]: I1203 11:59:45.437304 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell0-cell-mapping-fc2zk"] Dec 03 11:59:46 crc kubenswrapper[5052]: I1203 11:59:45.783962 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-db-sync-dlzcc"] Dec 03 11:59:46 crc kubenswrapper[5052]: I1203 11:59:45.785437 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-dlzcc" Dec 03 11:59:46 crc kubenswrapper[5052]: I1203 11:59:45.787731 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell1-conductor-config-data" Dec 03 11:59:46 crc kubenswrapper[5052]: I1203 11:59:45.788134 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell1-conductor-scripts" Dec 03 11:59:46 crc kubenswrapper[5052]: I1203 11:59:45.803173 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-db-sync-dlzcc"] Dec 03 11:59:46 crc kubenswrapper[5052]: I1203 11:59:45.869336 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/091d6975-4400-4dde-b61c-bd635c1ed1f7-combined-ca-bundle\") pod \"nova-cell1-conductor-db-sync-dlzcc\" (UID: \"091d6975-4400-4dde-b61c-bd635c1ed1f7\") " pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-dlzcc" Dec 03 11:59:46 crc kubenswrapper[5052]: I1203 11:59:45.869388 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h8smg\" (UniqueName: \"kubernetes.io/projected/091d6975-4400-4dde-b61c-bd635c1ed1f7-kube-api-access-h8smg\") pod \"nova-cell1-conductor-db-sync-dlzcc\" (UID: \"091d6975-4400-4dde-b61c-bd635c1ed1f7\") " pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-dlzcc" Dec 03 11:59:46 crc kubenswrapper[5052]: I1203 11:59:45.869701 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/091d6975-4400-4dde-b61c-bd635c1ed1f7-scripts\") pod \"nova-cell1-conductor-db-sync-dlzcc\" (UID: \"091d6975-4400-4dde-b61c-bd635c1ed1f7\") " pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-dlzcc" Dec 03 11:59:46 crc kubenswrapper[5052]: I1203 11:59:45.869860 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/091d6975-4400-4dde-b61c-bd635c1ed1f7-config-data\") pod \"nova-cell1-conductor-db-sync-dlzcc\" (UID: \"091d6975-4400-4dde-b61c-bd635c1ed1f7\") " pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-dlzcc" Dec 03 11:59:46 crc kubenswrapper[5052]: I1203 11:59:45.971612 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/091d6975-4400-4dde-b61c-bd635c1ed1f7-scripts\") pod \"nova-cell1-conductor-db-sync-dlzcc\" (UID: \"091d6975-4400-4dde-b61c-bd635c1ed1f7\") " pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-dlzcc" Dec 03 11:59:46 crc kubenswrapper[5052]: I1203 11:59:45.971742 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/091d6975-4400-4dde-b61c-bd635c1ed1f7-config-data\") pod \"nova-cell1-conductor-db-sync-dlzcc\" (UID: \"091d6975-4400-4dde-b61c-bd635c1ed1f7\") " pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-dlzcc" Dec 03 11:59:46 crc kubenswrapper[5052]: I1203 11:59:45.972714 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/091d6975-4400-4dde-b61c-bd635c1ed1f7-combined-ca-bundle\") pod \"nova-cell1-conductor-db-sync-dlzcc\" (UID: \"091d6975-4400-4dde-b61c-bd635c1ed1f7\") " pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-dlzcc" Dec 03 11:59:46 crc kubenswrapper[5052]: I1203 11:59:45.972777 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-h8smg\" (UniqueName: \"kubernetes.io/projected/091d6975-4400-4dde-b61c-bd635c1ed1f7-kube-api-access-h8smg\") pod \"nova-cell1-conductor-db-sync-dlzcc\" (UID: \"091d6975-4400-4dde-b61c-bd635c1ed1f7\") " pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-dlzcc" Dec 03 11:59:46 crc kubenswrapper[5052]: I1203 11:59:45.979958 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/091d6975-4400-4dde-b61c-bd635c1ed1f7-config-data\") pod \"nova-cell1-conductor-db-sync-dlzcc\" (UID: \"091d6975-4400-4dde-b61c-bd635c1ed1f7\") " pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-dlzcc" Dec 03 11:59:46 crc kubenswrapper[5052]: I1203 11:59:45.986343 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/091d6975-4400-4dde-b61c-bd635c1ed1f7-combined-ca-bundle\") pod \"nova-cell1-conductor-db-sync-dlzcc\" (UID: \"091d6975-4400-4dde-b61c-bd635c1ed1f7\") " pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-dlzcc" Dec 03 11:59:46 crc kubenswrapper[5052]: I1203 11:59:45.986871 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/091d6975-4400-4dde-b61c-bd635c1ed1f7-scripts\") pod \"nova-cell1-conductor-db-sync-dlzcc\" (UID: \"091d6975-4400-4dde-b61c-bd635c1ed1f7\") " pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-dlzcc" Dec 03 11:59:46 crc kubenswrapper[5052]: I1203 11:59:46.026051 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-h8smg\" (UniqueName: \"kubernetes.io/projected/091d6975-4400-4dde-b61c-bd635c1ed1f7-kube-api-access-h8smg\") pod \"nova-cell1-conductor-db-sync-dlzcc\" (UID: \"091d6975-4400-4dde-b61c-bd635c1ed1f7\") " pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-dlzcc" Dec 03 11:59:46 crc kubenswrapper[5052]: I1203 11:59:46.054704 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-cell-mapping-fc2zk" event={"ID":"03284adf-04cd-4ca4-8b31-7ae483082bba","Type":"ContainerStarted","Data":"47bcb775089e195a3a2981bacc0958e1cef8109d2909c2918e326e5e17c0b9ba"} Dec 03 11:59:46 crc kubenswrapper[5052]: I1203 11:59:46.054748 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-cell-mapping-fc2zk" event={"ID":"03284adf-04cd-4ca4-8b31-7ae483082bba","Type":"ContainerStarted","Data":"ee23ccb16f1df386f6b6e8f669d6d2e731fa79440f20879f2c801342d6192f9c"} Dec 03 11:59:46 crc kubenswrapper[5052]: I1203 11:59:46.127769 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-dlzcc" Dec 03 11:59:46 crc kubenswrapper[5052]: I1203 11:59:46.673226 5052 scope.go:117] "RemoveContainer" containerID="d224717c6d9ca8135fd510e03a6c40c9f2b1a56dd8b4a3b685af3e2dfa855982" Dec 03 11:59:46 crc kubenswrapper[5052]: E1203 11:59:46.674169 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 11:59:46 crc kubenswrapper[5052]: I1203 11:59:46.855562 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 11:59:46 crc kubenswrapper[5052]: I1203 11:59:46.870181 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 11:59:46 crc kubenswrapper[5052]: I1203 11:59:46.879259 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 11:59:46 crc kubenswrapper[5052]: W1203 11:59:46.882103 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod091d6975_4400_4dde_b61c_bd635c1ed1f7.slice/crio-0ec1b94324a2f7fb2b943fbb47c241b8c62b1996dcf799fe6c895658681aaf74 WatchSource:0}: Error finding container 0ec1b94324a2f7fb2b943fbb47c241b8c62b1996dcf799fe6c895658681aaf74: Status 404 returned error can't find the container with id 0ec1b94324a2f7fb2b943fbb47c241b8c62b1996dcf799fe6c895658681aaf74 Dec 03 11:59:46 crc kubenswrapper[5052]: I1203 11:59:46.892257 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 11:59:46 crc kubenswrapper[5052]: I1203 11:59:46.901363 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-db-sync-dlzcc"] Dec 03 11:59:47 crc kubenswrapper[5052]: I1203 11:59:47.070088 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"79b9410f-a5f6-48ab-a697-0aa515754ab0","Type":"ContainerStarted","Data":"77ad9d745342d82cd0811540a1915d09073d8ab4d0f16bc8fda9937af750b125"} Dec 03 11:59:47 crc kubenswrapper[5052]: I1203 11:59:47.071716 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"531ff22d-8820-443e-b1ac-5d1335c0691a","Type":"ContainerStarted","Data":"57c4d66aa9003cf5971b72d1aae392ee8c61f2a53e76c7a9b3b230b8aa70ea0b"} Dec 03 11:59:47 crc kubenswrapper[5052]: I1203 11:59:47.074726 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"ede50db6-6b4c-4dd5-9f79-cd76d5de6a99","Type":"ContainerStarted","Data":"250ec4648768ae29c7948b385e6a9b18073cced52ba1d2f4bd23c6ac91ac9c52"} Dec 03 11:59:47 crc kubenswrapper[5052]: I1203 11:59:47.076544 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" event={"ID":"1f41370a-19d0-4c2d-b942-26562f752813","Type":"ContainerStarted","Data":"8f837453929db711888ca07110672b6c1204a13b9fb39db4c3b0031692905192"} Dec 03 11:59:47 crc kubenswrapper[5052]: I1203 11:59:47.078693 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-dlzcc" event={"ID":"091d6975-4400-4dde-b61c-bd635c1ed1f7","Type":"ContainerStarted","Data":"0ec1b94324a2f7fb2b943fbb47c241b8c62b1996dcf799fe6c895658681aaf74"} Dec 03 11:59:47 crc kubenswrapper[5052]: I1203 11:59:47.113415 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-cell0-cell-mapping-fc2zk" podStartSLOduration=3.113396726 podStartE2EDuration="3.113396726s" podCreationTimestamp="2025-12-03 11:59:44 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:59:47.103715309 +0000 UTC m=+2377.691957073" watchObservedRunningTime="2025-12-03 11:59:47.113396726 +0000 UTC m=+2377.701638490" Dec 03 11:59:47 crc kubenswrapper[5052]: I1203 11:59:47.956062 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 11:59:47 crc kubenswrapper[5052]: I1203 11:59:47.985372 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 11:59:48 crc kubenswrapper[5052]: I1203 11:59:48.090200 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"531ff22d-8820-443e-b1ac-5d1335c0691a","Type":"ContainerStarted","Data":"1611bf0c695b712a78e2e976758bbf53ac09cd6c6e80e60bf641a357ab0e4829"} Dec 03 11:59:48 crc kubenswrapper[5052]: I1203 11:59:48.090258 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"531ff22d-8820-443e-b1ac-5d1335c0691a","Type":"ContainerStarted","Data":"64f7ba72e9ec36e3bb0845adfbd3ad9b79764b3dd7e25377cb0ba9a336fc852e"} Dec 03 11:59:48 crc kubenswrapper[5052]: I1203 11:59:48.092565 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"ede50db6-6b4c-4dd5-9f79-cd76d5de6a99","Type":"ContainerStarted","Data":"2ac74a9091c3ed67a00d2b336d665cba78befa7258777b544feace96771b2c48"} Dec 03 11:59:48 crc kubenswrapper[5052]: I1203 11:59:48.092663 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"ede50db6-6b4c-4dd5-9f79-cd76d5de6a99","Type":"ContainerStarted","Data":"41d12baf2450bd08a8a04af49248da7d52b30dcec65dec130a07bbe2795bb68c"} Dec 03 11:59:48 crc kubenswrapper[5052]: I1203 11:59:48.094343 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" event={"ID":"1f41370a-19d0-4c2d-b942-26562f752813","Type":"ContainerStarted","Data":"2f0b1bc5e2c1b087f5be9d90327ea9a97c57020ec2b1922d1d41714021fc9db7"} Dec 03 11:59:48 crc kubenswrapper[5052]: I1203 11:59:48.095982 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-dlzcc" event={"ID":"091d6975-4400-4dde-b61c-bd635c1ed1f7","Type":"ContainerStarted","Data":"a7260079f4bde2479c2fcef41cd28875413e615dbb2c2031c2b3435fe94b4524"} Dec 03 11:59:48 crc kubenswrapper[5052]: I1203 11:59:48.097586 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"79b9410f-a5f6-48ab-a697-0aa515754ab0","Type":"ContainerStarted","Data":"40df4e21c3e08adbee7e844764da6ada1d8397a746cbb932f76f824d48838795"} Dec 03 11:59:48 crc kubenswrapper[5052]: I1203 11:59:48.113148 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-metadata-0" podStartSLOduration=4.1131325069999995 podStartE2EDuration="4.113132507s" podCreationTimestamp="2025-12-03 11:59:44 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:59:48.111264763 +0000 UTC m=+2378.699506527" watchObservedRunningTime="2025-12-03 11:59:48.113132507 +0000 UTC m=+2378.701374261" Dec 03 11:59:48 crc kubenswrapper[5052]: I1203 11:59:48.142269 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" podStartSLOduration=4.14224905 podStartE2EDuration="4.14224905s" podCreationTimestamp="2025-12-03 11:59:44 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:59:48.134586531 +0000 UTC m=+2378.722828295" watchObservedRunningTime="2025-12-03 11:59:48.14224905 +0000 UTC m=+2378.730490814" Dec 03 11:59:48 crc kubenswrapper[5052]: I1203 11:59:48.156026 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-api-0" podStartSLOduration=4.156010354 podStartE2EDuration="4.156010354s" podCreationTimestamp="2025-12-03 11:59:44 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:59:48.150485896 +0000 UTC m=+2378.738727660" watchObservedRunningTime="2025-12-03 11:59:48.156010354 +0000 UTC m=+2378.744252118" Dec 03 11:59:48 crc kubenswrapper[5052]: I1203 11:59:48.189318 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-dlzcc" podStartSLOduration=3.189300027 podStartE2EDuration="3.189300027s" podCreationTimestamp="2025-12-03 11:59:45 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:59:48.179891947 +0000 UTC m=+2378.768133711" watchObservedRunningTime="2025-12-03 11:59:48.189300027 +0000 UTC m=+2378.777541791" Dec 03 11:59:48 crc kubenswrapper[5052]: I1203 11:59:48.208097 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-scheduler-0" podStartSLOduration=4.208077204 podStartE2EDuration="4.208077204s" podCreationTimestamp="2025-12-03 11:59:44 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:59:48.198330735 +0000 UTC m=+2378.786572499" watchObservedRunningTime="2025-12-03 11:59:48.208077204 +0000 UTC m=+2378.796318968" Dec 03 11:59:48 crc kubenswrapper[5052]: I1203 11:59:48.977013 5052 scope.go:117] "RemoveContainer" containerID="f681937d4e3600c07f0e7a5f6674a949192d35f7d97ddf1a5a4f6b78e7c5ca57" Dec 03 11:59:49 crc kubenswrapper[5052]: I1203 11:59:49.117190 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" podUID="1f41370a-19d0-4c2d-b942-26562f752813" containerName="nova-cell1-novncproxy-novncproxy" containerID="cri-o://2f0b1bc5e2c1b087f5be9d90327ea9a97c57020ec2b1922d1d41714021fc9db7" gracePeriod=30 Dec 03 11:59:49 crc kubenswrapper[5052]: I1203 11:59:49.117723 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-metadata-0" podUID="531ff22d-8820-443e-b1ac-5d1335c0691a" containerName="nova-metadata-log" containerID="cri-o://64f7ba72e9ec36e3bb0845adfbd3ad9b79764b3dd7e25377cb0ba9a336fc852e" gracePeriod=30 Dec 03 11:59:49 crc kubenswrapper[5052]: I1203 11:59:49.117855 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-metadata-0" podUID="531ff22d-8820-443e-b1ac-5d1335c0691a" containerName="nova-metadata-metadata" containerID="cri-o://1611bf0c695b712a78e2e976758bbf53ac09cd6c6e80e60bf641a357ab0e4829" gracePeriod=30 Dec 03 11:59:49 crc kubenswrapper[5052]: I1203 11:59:49.559923 5052 scope.go:117] "RemoveContainer" containerID="45b01a8f4fda9b4f932ead3e127a5cc89388e307cc160b1e9e3f45111d4ef6b0" Dec 03 11:59:49 crc kubenswrapper[5052]: I1203 11:59:49.695258 5052 scope.go:117] "RemoveContainer" containerID="cbf6b026a9b62a7464f44bbde7effeb8a0aa796789269fbcd6b9b3589fc439c9" Dec 03 11:59:49 crc kubenswrapper[5052]: I1203 11:59:49.726360 5052 scope.go:117] "RemoveContainer" containerID="47ff9ffb2ee04fc839b0023d8e10c46ffdf46d84f9e4a70a36307e82f3f234da" Dec 03 11:59:49 crc kubenswrapper[5052]: I1203 11:59:49.768527 5052 scope.go:117] "RemoveContainer" containerID="70062d4b63028d8ee6f2b890b4bb61d128ba1af7288be9bc6c1fe530ecb428dc" Dec 03 11:59:49 crc kubenswrapper[5052]: I1203 11:59:49.831099 5052 scope.go:117] "RemoveContainer" containerID="80ffd33af4023d6b6f0f24ec3bd666459e45b4216c33dab791e932793c7535bf" Dec 03 11:59:49 crc kubenswrapper[5052]: I1203 11:59:49.854829 5052 scope.go:117] "RemoveContainer" containerID="974df8d52afeb94ca7f6376aa32c93e7908382a0a0e609c4d64eb521bfbc0a41" Dec 03 11:59:49 crc kubenswrapper[5052]: I1203 11:59:49.935851 5052 scope.go:117] "RemoveContainer" containerID="5c59824fa390c99841f43488a426aa7880afaa22f074091daf68f834930db61d" Dec 03 11:59:49 crc kubenswrapper[5052]: I1203 11:59:49.959996 5052 scope.go:117] "RemoveContainer" containerID="d035b8a698baa861b1fde789196a379e9a2b04d327860b01e3e264af295b680f" Dec 03 11:59:50 crc kubenswrapper[5052]: I1203 11:59:50.142098 5052 generic.go:334] "Generic (PLEG): container finished" podID="531ff22d-8820-443e-b1ac-5d1335c0691a" containerID="1611bf0c695b712a78e2e976758bbf53ac09cd6c6e80e60bf641a357ab0e4829" exitCode=0 Dec 03 11:59:50 crc kubenswrapper[5052]: I1203 11:59:50.142472 5052 generic.go:334] "Generic (PLEG): container finished" podID="531ff22d-8820-443e-b1ac-5d1335c0691a" containerID="64f7ba72e9ec36e3bb0845adfbd3ad9b79764b3dd7e25377cb0ba9a336fc852e" exitCode=143 Dec 03 11:59:50 crc kubenswrapper[5052]: I1203 11:59:50.142177 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"531ff22d-8820-443e-b1ac-5d1335c0691a","Type":"ContainerDied","Data":"1611bf0c695b712a78e2e976758bbf53ac09cd6c6e80e60bf641a357ab0e4829"} Dec 03 11:59:50 crc kubenswrapper[5052]: I1203 11:59:50.142636 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"531ff22d-8820-443e-b1ac-5d1335c0691a","Type":"ContainerDied","Data":"64f7ba72e9ec36e3bb0845adfbd3ad9b79764b3dd7e25377cb0ba9a336fc852e"} Dec 03 11:59:50 crc kubenswrapper[5052]: I1203 11:59:50.199725 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:59:50 crc kubenswrapper[5052]: I1203 11:59:50.199782 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:59:50 crc kubenswrapper[5052]: I1203 11:59:50.213252 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 11:59:50 crc kubenswrapper[5052]: I1203 11:59:50.366848 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:59:50 crc kubenswrapper[5052]: I1203 11:59:50.452295 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:59:50 crc kubenswrapper[5052]: I1203 11:59:50.577295 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/531ff22d-8820-443e-b1ac-5d1335c0691a-logs\") pod \"531ff22d-8820-443e-b1ac-5d1335c0691a\" (UID: \"531ff22d-8820-443e-b1ac-5d1335c0691a\") " Dec 03 11:59:50 crc kubenswrapper[5052]: I1203 11:59:50.577490 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/531ff22d-8820-443e-b1ac-5d1335c0691a-config-data\") pod \"531ff22d-8820-443e-b1ac-5d1335c0691a\" (UID: \"531ff22d-8820-443e-b1ac-5d1335c0691a\") " Dec 03 11:59:50 crc kubenswrapper[5052]: I1203 11:59:50.578234 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/531ff22d-8820-443e-b1ac-5d1335c0691a-logs" (OuterVolumeSpecName: "logs") pod "531ff22d-8820-443e-b1ac-5d1335c0691a" (UID: "531ff22d-8820-443e-b1ac-5d1335c0691a"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:59:50 crc kubenswrapper[5052]: I1203 11:59:50.578479 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8dktw\" (UniqueName: \"kubernetes.io/projected/531ff22d-8820-443e-b1ac-5d1335c0691a-kube-api-access-8dktw\") pod \"531ff22d-8820-443e-b1ac-5d1335c0691a\" (UID: \"531ff22d-8820-443e-b1ac-5d1335c0691a\") " Dec 03 11:59:50 crc kubenswrapper[5052]: I1203 11:59:50.578591 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/531ff22d-8820-443e-b1ac-5d1335c0691a-combined-ca-bundle\") pod \"531ff22d-8820-443e-b1ac-5d1335c0691a\" (UID: \"531ff22d-8820-443e-b1ac-5d1335c0691a\") " Dec 03 11:59:50 crc kubenswrapper[5052]: I1203 11:59:50.579080 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/531ff22d-8820-443e-b1ac-5d1335c0691a-logs\") on node \"crc\" DevicePath \"\"" Dec 03 11:59:50 crc kubenswrapper[5052]: I1203 11:59:50.584158 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/531ff22d-8820-443e-b1ac-5d1335c0691a-kube-api-access-8dktw" (OuterVolumeSpecName: "kube-api-access-8dktw") pod "531ff22d-8820-443e-b1ac-5d1335c0691a" (UID: "531ff22d-8820-443e-b1ac-5d1335c0691a"). InnerVolumeSpecName "kube-api-access-8dktw". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:59:50 crc kubenswrapper[5052]: I1203 11:59:50.604776 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/531ff22d-8820-443e-b1ac-5d1335c0691a-config-data" (OuterVolumeSpecName: "config-data") pod "531ff22d-8820-443e-b1ac-5d1335c0691a" (UID: "531ff22d-8820-443e-b1ac-5d1335c0691a"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:59:50 crc kubenswrapper[5052]: I1203 11:59:50.610661 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/531ff22d-8820-443e-b1ac-5d1335c0691a-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "531ff22d-8820-443e-b1ac-5d1335c0691a" (UID: "531ff22d-8820-443e-b1ac-5d1335c0691a"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:59:50 crc kubenswrapper[5052]: I1203 11:59:50.680438 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/531ff22d-8820-443e-b1ac-5d1335c0691a-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:59:50 crc kubenswrapper[5052]: I1203 11:59:50.680485 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/531ff22d-8820-443e-b1ac-5d1335c0691a-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:59:50 crc kubenswrapper[5052]: I1203 11:59:50.680495 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-8dktw\" (UniqueName: \"kubernetes.io/projected/531ff22d-8820-443e-b1ac-5d1335c0691a-kube-api-access-8dktw\") on node \"crc\" DevicePath \"\"" Dec 03 11:59:51 crc kubenswrapper[5052]: I1203 11:59:51.164629 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"531ff22d-8820-443e-b1ac-5d1335c0691a","Type":"ContainerDied","Data":"57c4d66aa9003cf5971b72d1aae392ee8c61f2a53e76c7a9b3b230b8aa70ea0b"} Dec 03 11:59:51 crc kubenswrapper[5052]: I1203 11:59:51.164674 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:59:51 crc kubenswrapper[5052]: I1203 11:59:51.164732 5052 scope.go:117] "RemoveContainer" containerID="1611bf0c695b712a78e2e976758bbf53ac09cd6c6e80e60bf641a357ab0e4829" Dec 03 11:59:51 crc kubenswrapper[5052]: I1203 11:59:51.193434 5052 scope.go:117] "RemoveContainer" containerID="64f7ba72e9ec36e3bb0845adfbd3ad9b79764b3dd7e25377cb0ba9a336fc852e" Dec 03 11:59:51 crc kubenswrapper[5052]: I1203 11:59:51.218102 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 11:59:51 crc kubenswrapper[5052]: I1203 11:59:51.236739 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 11:59:51 crc kubenswrapper[5052]: I1203 11:59:51.251973 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 11:59:51 crc kubenswrapper[5052]: E1203 11:59:51.252432 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="531ff22d-8820-443e-b1ac-5d1335c0691a" containerName="nova-metadata-metadata" Dec 03 11:59:51 crc kubenswrapper[5052]: I1203 11:59:51.252450 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="531ff22d-8820-443e-b1ac-5d1335c0691a" containerName="nova-metadata-metadata" Dec 03 11:59:51 crc kubenswrapper[5052]: E1203 11:59:51.252465 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="531ff22d-8820-443e-b1ac-5d1335c0691a" containerName="nova-metadata-log" Dec 03 11:59:51 crc kubenswrapper[5052]: I1203 11:59:51.252472 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="531ff22d-8820-443e-b1ac-5d1335c0691a" containerName="nova-metadata-log" Dec 03 11:59:51 crc kubenswrapper[5052]: I1203 11:59:51.252763 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="531ff22d-8820-443e-b1ac-5d1335c0691a" containerName="nova-metadata-log" Dec 03 11:59:51 crc kubenswrapper[5052]: I1203 11:59:51.252782 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="531ff22d-8820-443e-b1ac-5d1335c0691a" containerName="nova-metadata-metadata" Dec 03 11:59:51 crc kubenswrapper[5052]: I1203 11:59:51.253842 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:59:51 crc kubenswrapper[5052]: I1203 11:59:51.268284 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-metadata-config-data" Dec 03 11:59:51 crc kubenswrapper[5052]: I1203 11:59:51.268630 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-nova-metadata-internal-svc" Dec 03 11:59:51 crc kubenswrapper[5052]: I1203 11:59:51.286495 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 11:59:51 crc kubenswrapper[5052]: I1203 11:59:51.392545 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/3cd98ccc-9e96-4f06-abc5-ef6342585d62-nova-metadata-tls-certs\") pod \"nova-metadata-0\" (UID: \"3cd98ccc-9e96-4f06-abc5-ef6342585d62\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:59:51 crc kubenswrapper[5052]: I1203 11:59:51.392656 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/3cd98ccc-9e96-4f06-abc5-ef6342585d62-logs\") pod \"nova-metadata-0\" (UID: \"3cd98ccc-9e96-4f06-abc5-ef6342585d62\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:59:51 crc kubenswrapper[5052]: I1203 11:59:51.392718 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3cd98ccc-9e96-4f06-abc5-ef6342585d62-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"3cd98ccc-9e96-4f06-abc5-ef6342585d62\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:59:51 crc kubenswrapper[5052]: I1203 11:59:51.392809 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/3cd98ccc-9e96-4f06-abc5-ef6342585d62-config-data\") pod \"nova-metadata-0\" (UID: \"3cd98ccc-9e96-4f06-abc5-ef6342585d62\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:59:51 crc kubenswrapper[5052]: I1203 11:59:51.392975 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hw7zc\" (UniqueName: \"kubernetes.io/projected/3cd98ccc-9e96-4f06-abc5-ef6342585d62-kube-api-access-hw7zc\") pod \"nova-metadata-0\" (UID: \"3cd98ccc-9e96-4f06-abc5-ef6342585d62\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:59:51 crc kubenswrapper[5052]: I1203 11:59:51.494810 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/3cd98ccc-9e96-4f06-abc5-ef6342585d62-nova-metadata-tls-certs\") pod \"nova-metadata-0\" (UID: \"3cd98ccc-9e96-4f06-abc5-ef6342585d62\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:59:51 crc kubenswrapper[5052]: I1203 11:59:51.494871 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/3cd98ccc-9e96-4f06-abc5-ef6342585d62-logs\") pod \"nova-metadata-0\" (UID: \"3cd98ccc-9e96-4f06-abc5-ef6342585d62\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:59:51 crc kubenswrapper[5052]: I1203 11:59:51.494905 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3cd98ccc-9e96-4f06-abc5-ef6342585d62-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"3cd98ccc-9e96-4f06-abc5-ef6342585d62\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:59:51 crc kubenswrapper[5052]: I1203 11:59:51.494955 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/3cd98ccc-9e96-4f06-abc5-ef6342585d62-config-data\") pod \"nova-metadata-0\" (UID: \"3cd98ccc-9e96-4f06-abc5-ef6342585d62\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:59:51 crc kubenswrapper[5052]: I1203 11:59:51.495046 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-hw7zc\" (UniqueName: \"kubernetes.io/projected/3cd98ccc-9e96-4f06-abc5-ef6342585d62-kube-api-access-hw7zc\") pod \"nova-metadata-0\" (UID: \"3cd98ccc-9e96-4f06-abc5-ef6342585d62\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:59:51 crc kubenswrapper[5052]: I1203 11:59:51.495445 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/3cd98ccc-9e96-4f06-abc5-ef6342585d62-logs\") pod \"nova-metadata-0\" (UID: \"3cd98ccc-9e96-4f06-abc5-ef6342585d62\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:59:51 crc kubenswrapper[5052]: I1203 11:59:51.499555 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/3cd98ccc-9e96-4f06-abc5-ef6342585d62-nova-metadata-tls-certs\") pod \"nova-metadata-0\" (UID: \"3cd98ccc-9e96-4f06-abc5-ef6342585d62\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:59:51 crc kubenswrapper[5052]: I1203 11:59:51.499585 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/3cd98ccc-9e96-4f06-abc5-ef6342585d62-config-data\") pod \"nova-metadata-0\" (UID: \"3cd98ccc-9e96-4f06-abc5-ef6342585d62\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:59:51 crc kubenswrapper[5052]: I1203 11:59:51.516656 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3cd98ccc-9e96-4f06-abc5-ef6342585d62-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"3cd98ccc-9e96-4f06-abc5-ef6342585d62\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:59:51 crc kubenswrapper[5052]: I1203 11:59:51.522705 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-hw7zc\" (UniqueName: \"kubernetes.io/projected/3cd98ccc-9e96-4f06-abc5-ef6342585d62-kube-api-access-hw7zc\") pod \"nova-metadata-0\" (UID: \"3cd98ccc-9e96-4f06-abc5-ef6342585d62\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:59:51 crc kubenswrapper[5052]: I1203 11:59:51.587929 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:59:51 crc kubenswrapper[5052]: I1203 11:59:51.742692 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="531ff22d-8820-443e-b1ac-5d1335c0691a" path="/var/lib/kubelet/pods/531ff22d-8820-443e-b1ac-5d1335c0691a/volumes" Dec 03 11:59:51 crc kubenswrapper[5052]: I1203 11:59:51.775623 5052 pod_container_manager_linux.go:210] "Failed to delete cgroup paths" cgroupName=["kubepods","besteffort","pod2ddd27d6-ad35-4919-8fe5-17100335e14a"] err="unable to destroy cgroup paths for cgroup [kubepods besteffort pod2ddd27d6-ad35-4919-8fe5-17100335e14a] : Timed out while waiting for systemd to remove kubepods-besteffort-pod2ddd27d6_ad35_4919_8fe5_17100335e14a.slice" Dec 03 11:59:51 crc kubenswrapper[5052]: E1203 11:59:51.775666 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to delete cgroup paths for [kubepods besteffort pod2ddd27d6-ad35-4919-8fe5-17100335e14a] : unable to destroy cgroup paths for cgroup [kubepods besteffort pod2ddd27d6-ad35-4919-8fe5-17100335e14a] : Timed out while waiting for systemd to remove kubepods-besteffort-pod2ddd27d6_ad35_4919_8fe5_17100335e14a.slice" pod="openstack-kuttl-tests/nova-cell1-db-create-d4vgs" podUID="2ddd27d6-ad35-4919-8fe5-17100335e14a" Dec 03 11:59:52 crc kubenswrapper[5052]: I1203 11:59:52.101479 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 11:59:52 crc kubenswrapper[5052]: I1203 11:59:52.178492 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-db-create-d4vgs" Dec 03 11:59:52 crc kubenswrapper[5052]: I1203 11:59:52.178490 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"3cd98ccc-9e96-4f06-abc5-ef6342585d62","Type":"ContainerStarted","Data":"054ca984a2231b9f8b59b25c1e50671e4be6b5df8f7689c86aa911207d074b83"} Dec 03 11:59:53 crc kubenswrapper[5052]: I1203 11:59:53.189074 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"3cd98ccc-9e96-4f06-abc5-ef6342585d62","Type":"ContainerStarted","Data":"d4ef4097dfbeeeade7c6933eda2f2f879f07022db9f47964fc70b63943ac8259"} Dec 03 11:59:53 crc kubenswrapper[5052]: I1203 11:59:53.189449 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"3cd98ccc-9e96-4f06-abc5-ef6342585d62","Type":"ContainerStarted","Data":"fb631af9ed30c3aa9e9d9c6d3aa2a169ddef995774a2bf394ecb585b3bb5d3df"} Dec 03 11:59:53 crc kubenswrapper[5052]: I1203 11:59:53.209953 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-metadata-0" podStartSLOduration=2.20993346 podStartE2EDuration="2.20993346s" podCreationTimestamp="2025-12-03 11:59:51 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:59:53.205561144 +0000 UTC m=+2383.793802908" watchObservedRunningTime="2025-12-03 11:59:53.20993346 +0000 UTC m=+2383.798175224" Dec 03 11:59:54 crc kubenswrapper[5052]: I1203 11:59:54.200579 5052 generic.go:334] "Generic (PLEG): container finished" podID="03284adf-04cd-4ca4-8b31-7ae483082bba" containerID="47bcb775089e195a3a2981bacc0958e1cef8109d2909c2918e326e5e17c0b9ba" exitCode=0 Dec 03 11:59:54 crc kubenswrapper[5052]: I1203 11:59:54.200669 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-cell-mapping-fc2zk" event={"ID":"03284adf-04cd-4ca4-8b31-7ae483082bba","Type":"ContainerDied","Data":"47bcb775089e195a3a2981bacc0958e1cef8109d2909c2918e326e5e17c0b9ba"} Dec 03 11:59:55 crc kubenswrapper[5052]: I1203 11:59:55.182400 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:59:55 crc kubenswrapper[5052]: I1203 11:59:55.182474 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-api-0" Dec 03 11:59:55 crc kubenswrapper[5052]: I1203 11:59:55.213100 5052 generic.go:334] "Generic (PLEG): container finished" podID="091d6975-4400-4dde-b61c-bd635c1ed1f7" containerID="a7260079f4bde2479c2fcef41cd28875413e615dbb2c2031c2b3435fe94b4524" exitCode=0 Dec 03 11:59:55 crc kubenswrapper[5052]: I1203 11:59:55.213184 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-dlzcc" event={"ID":"091d6975-4400-4dde-b61c-bd635c1ed1f7","Type":"ContainerDied","Data":"a7260079f4bde2479c2fcef41cd28875413e615dbb2c2031c2b3435fe94b4524"} Dec 03 11:59:55 crc kubenswrapper[5052]: I1203 11:59:55.367031 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:59:55 crc kubenswrapper[5052]: I1203 11:59:55.413570 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:59:55 crc kubenswrapper[5052]: I1203 11:59:55.592609 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-cell-mapping-fc2zk" Dec 03 11:59:55 crc kubenswrapper[5052]: I1203 11:59:55.778780 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/03284adf-04cd-4ca4-8b31-7ae483082bba-scripts\") pod \"03284adf-04cd-4ca4-8b31-7ae483082bba\" (UID: \"03284adf-04cd-4ca4-8b31-7ae483082bba\") " Dec 03 11:59:55 crc kubenswrapper[5052]: I1203 11:59:55.778923 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/03284adf-04cd-4ca4-8b31-7ae483082bba-combined-ca-bundle\") pod \"03284adf-04cd-4ca4-8b31-7ae483082bba\" (UID: \"03284adf-04cd-4ca4-8b31-7ae483082bba\") " Dec 03 11:59:55 crc kubenswrapper[5052]: I1203 11:59:55.778978 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/03284adf-04cd-4ca4-8b31-7ae483082bba-config-data\") pod \"03284adf-04cd-4ca4-8b31-7ae483082bba\" (UID: \"03284adf-04cd-4ca4-8b31-7ae483082bba\") " Dec 03 11:59:55 crc kubenswrapper[5052]: I1203 11:59:55.779008 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jrvgv\" (UniqueName: \"kubernetes.io/projected/03284adf-04cd-4ca4-8b31-7ae483082bba-kube-api-access-jrvgv\") pod \"03284adf-04cd-4ca4-8b31-7ae483082bba\" (UID: \"03284adf-04cd-4ca4-8b31-7ae483082bba\") " Dec 03 11:59:55 crc kubenswrapper[5052]: I1203 11:59:55.799659 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/03284adf-04cd-4ca4-8b31-7ae483082bba-kube-api-access-jrvgv" (OuterVolumeSpecName: "kube-api-access-jrvgv") pod "03284adf-04cd-4ca4-8b31-7ae483082bba" (UID: "03284adf-04cd-4ca4-8b31-7ae483082bba"). InnerVolumeSpecName "kube-api-access-jrvgv". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:59:55 crc kubenswrapper[5052]: I1203 11:59:55.799773 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/03284adf-04cd-4ca4-8b31-7ae483082bba-scripts" (OuterVolumeSpecName: "scripts") pod "03284adf-04cd-4ca4-8b31-7ae483082bba" (UID: "03284adf-04cd-4ca4-8b31-7ae483082bba"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:59:55 crc kubenswrapper[5052]: I1203 11:59:55.806017 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/03284adf-04cd-4ca4-8b31-7ae483082bba-config-data" (OuterVolumeSpecName: "config-data") pod "03284adf-04cd-4ca4-8b31-7ae483082bba" (UID: "03284adf-04cd-4ca4-8b31-7ae483082bba"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:59:55 crc kubenswrapper[5052]: I1203 11:59:55.806478 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/03284adf-04cd-4ca4-8b31-7ae483082bba-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "03284adf-04cd-4ca4-8b31-7ae483082bba" (UID: "03284adf-04cd-4ca4-8b31-7ae483082bba"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:59:55 crc kubenswrapper[5052]: I1203 11:59:55.881102 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/03284adf-04cd-4ca4-8b31-7ae483082bba-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:59:55 crc kubenswrapper[5052]: I1203 11:59:55.881154 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-jrvgv\" (UniqueName: \"kubernetes.io/projected/03284adf-04cd-4ca4-8b31-7ae483082bba-kube-api-access-jrvgv\") on node \"crc\" DevicePath \"\"" Dec 03 11:59:55 crc kubenswrapper[5052]: I1203 11:59:55.881170 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/03284adf-04cd-4ca4-8b31-7ae483082bba-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:59:55 crc kubenswrapper[5052]: I1203 11:59:55.881182 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/03284adf-04cd-4ca4-8b31-7ae483082bba-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:59:56 crc kubenswrapper[5052]: I1203 11:59:56.229919 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-cell-mapping-fc2zk" Dec 03 11:59:56 crc kubenswrapper[5052]: I1203 11:59:56.229822 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-cell-mapping-fc2zk" event={"ID":"03284adf-04cd-4ca4-8b31-7ae483082bba","Type":"ContainerDied","Data":"ee23ccb16f1df386f6b6e8f669d6d2e731fa79440f20879f2c801342d6192f9c"} Dec 03 11:59:56 crc kubenswrapper[5052]: I1203 11:59:56.229985 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ee23ccb16f1df386f6b6e8f669d6d2e731fa79440f20879f2c801342d6192f9c" Dec 03 11:59:56 crc kubenswrapper[5052]: I1203 11:59:56.265646 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-api-0" podUID="ede50db6-6b4c-4dd5-9f79-cd76d5de6a99" containerName="nova-api-api" probeResult="failure" output="Get \"http://10.217.1.237:8774/\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Dec 03 11:59:56 crc kubenswrapper[5052]: I1203 11:59:56.265864 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-api-0" podUID="ede50db6-6b4c-4dd5-9f79-cd76d5de6a99" containerName="nova-api-log" probeResult="failure" output="Get \"http://10.217.1.237:8774/\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Dec 03 11:59:56 crc kubenswrapper[5052]: I1203 11:59:56.267747 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 11:59:56 crc kubenswrapper[5052]: I1203 11:59:56.421169 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 11:59:56 crc kubenswrapper[5052]: I1203 11:59:56.421406 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-api-0" podUID="ede50db6-6b4c-4dd5-9f79-cd76d5de6a99" containerName="nova-api-log" containerID="cri-o://41d12baf2450bd08a8a04af49248da7d52b30dcec65dec130a07bbe2795bb68c" gracePeriod=30 Dec 03 11:59:56 crc kubenswrapper[5052]: I1203 11:59:56.421547 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-api-0" podUID="ede50db6-6b4c-4dd5-9f79-cd76d5de6a99" containerName="nova-api-api" containerID="cri-o://2ac74a9091c3ed67a00d2b336d665cba78befa7258777b544feace96771b2c48" gracePeriod=30 Dec 03 11:59:56 crc kubenswrapper[5052]: I1203 11:59:56.474492 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 11:59:56 crc kubenswrapper[5052]: I1203 11:59:56.474787 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-metadata-0" podUID="3cd98ccc-9e96-4f06-abc5-ef6342585d62" containerName="nova-metadata-log" containerID="cri-o://fb631af9ed30c3aa9e9d9c6d3aa2a169ddef995774a2bf394ecb585b3bb5d3df" gracePeriod=30 Dec 03 11:59:56 crc kubenswrapper[5052]: I1203 11:59:56.475499 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-metadata-0" podUID="3cd98ccc-9e96-4f06-abc5-ef6342585d62" containerName="nova-metadata-metadata" containerID="cri-o://d4ef4097dfbeeeade7c6933eda2f2f879f07022db9f47964fc70b63943ac8259" gracePeriod=30 Dec 03 11:59:56 crc kubenswrapper[5052]: I1203 11:59:56.588243 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:59:56 crc kubenswrapper[5052]: I1203 11:59:56.592642 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:59:56 crc kubenswrapper[5052]: I1203 11:59:56.757413 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-dlzcc" Dec 03 11:59:56 crc kubenswrapper[5052]: I1203 11:59:56.825219 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 11:59:56 crc kubenswrapper[5052]: I1203 11:59:56.901240 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h8smg\" (UniqueName: \"kubernetes.io/projected/091d6975-4400-4dde-b61c-bd635c1ed1f7-kube-api-access-h8smg\") pod \"091d6975-4400-4dde-b61c-bd635c1ed1f7\" (UID: \"091d6975-4400-4dde-b61c-bd635c1ed1f7\") " Dec 03 11:59:56 crc kubenswrapper[5052]: I1203 11:59:56.901350 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/091d6975-4400-4dde-b61c-bd635c1ed1f7-config-data\") pod \"091d6975-4400-4dde-b61c-bd635c1ed1f7\" (UID: \"091d6975-4400-4dde-b61c-bd635c1ed1f7\") " Dec 03 11:59:56 crc kubenswrapper[5052]: I1203 11:59:56.901487 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/091d6975-4400-4dde-b61c-bd635c1ed1f7-combined-ca-bundle\") pod \"091d6975-4400-4dde-b61c-bd635c1ed1f7\" (UID: \"091d6975-4400-4dde-b61c-bd635c1ed1f7\") " Dec 03 11:59:56 crc kubenswrapper[5052]: I1203 11:59:56.901743 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/091d6975-4400-4dde-b61c-bd635c1ed1f7-scripts\") pod \"091d6975-4400-4dde-b61c-bd635c1ed1f7\" (UID: \"091d6975-4400-4dde-b61c-bd635c1ed1f7\") " Dec 03 11:59:56 crc kubenswrapper[5052]: I1203 11:59:56.907081 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/091d6975-4400-4dde-b61c-bd635c1ed1f7-kube-api-access-h8smg" (OuterVolumeSpecName: "kube-api-access-h8smg") pod "091d6975-4400-4dde-b61c-bd635c1ed1f7" (UID: "091d6975-4400-4dde-b61c-bd635c1ed1f7"). InnerVolumeSpecName "kube-api-access-h8smg". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:59:56 crc kubenswrapper[5052]: I1203 11:59:56.907273 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/091d6975-4400-4dde-b61c-bd635c1ed1f7-scripts" (OuterVolumeSpecName: "scripts") pod "091d6975-4400-4dde-b61c-bd635c1ed1f7" (UID: "091d6975-4400-4dde-b61c-bd635c1ed1f7"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:59:56 crc kubenswrapper[5052]: I1203 11:59:56.934987 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/091d6975-4400-4dde-b61c-bd635c1ed1f7-config-data" (OuterVolumeSpecName: "config-data") pod "091d6975-4400-4dde-b61c-bd635c1ed1f7" (UID: "091d6975-4400-4dde-b61c-bd635c1ed1f7"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:59:56 crc kubenswrapper[5052]: I1203 11:59:56.943079 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/091d6975-4400-4dde-b61c-bd635c1ed1f7-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "091d6975-4400-4dde-b61c-bd635c1ed1f7" (UID: "091d6975-4400-4dde-b61c-bd635c1ed1f7"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:59:57 crc kubenswrapper[5052]: I1203 11:59:57.004201 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-h8smg\" (UniqueName: \"kubernetes.io/projected/091d6975-4400-4dde-b61c-bd635c1ed1f7-kube-api-access-h8smg\") on node \"crc\" DevicePath \"\"" Dec 03 11:59:57 crc kubenswrapper[5052]: I1203 11:59:57.004231 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/091d6975-4400-4dde-b61c-bd635c1ed1f7-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:59:57 crc kubenswrapper[5052]: I1203 11:59:57.004242 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/091d6975-4400-4dde-b61c-bd635c1ed1f7-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:59:57 crc kubenswrapper[5052]: I1203 11:59:57.004251 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/091d6975-4400-4dde-b61c-bd635c1ed1f7-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 11:59:57 crc kubenswrapper[5052]: I1203 11:59:57.240409 5052 generic.go:334] "Generic (PLEG): container finished" podID="3cd98ccc-9e96-4f06-abc5-ef6342585d62" containerID="d4ef4097dfbeeeade7c6933eda2f2f879f07022db9f47964fc70b63943ac8259" exitCode=0 Dec 03 11:59:57 crc kubenswrapper[5052]: I1203 11:59:57.240446 5052 generic.go:334] "Generic (PLEG): container finished" podID="3cd98ccc-9e96-4f06-abc5-ef6342585d62" containerID="fb631af9ed30c3aa9e9d9c6d3aa2a169ddef995774a2bf394ecb585b3bb5d3df" exitCode=143 Dec 03 11:59:57 crc kubenswrapper[5052]: I1203 11:59:57.240503 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"3cd98ccc-9e96-4f06-abc5-ef6342585d62","Type":"ContainerDied","Data":"d4ef4097dfbeeeade7c6933eda2f2f879f07022db9f47964fc70b63943ac8259"} Dec 03 11:59:57 crc kubenswrapper[5052]: I1203 11:59:57.240557 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"3cd98ccc-9e96-4f06-abc5-ef6342585d62","Type":"ContainerDied","Data":"fb631af9ed30c3aa9e9d9c6d3aa2a169ddef995774a2bf394ecb585b3bb5d3df"} Dec 03 11:59:57 crc kubenswrapper[5052]: I1203 11:59:57.242501 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-dlzcc" event={"ID":"091d6975-4400-4dde-b61c-bd635c1ed1f7","Type":"ContainerDied","Data":"0ec1b94324a2f7fb2b943fbb47c241b8c62b1996dcf799fe6c895658681aaf74"} Dec 03 11:59:57 crc kubenswrapper[5052]: I1203 11:59:57.242576 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0ec1b94324a2f7fb2b943fbb47c241b8c62b1996dcf799fe6c895658681aaf74" Dec 03 11:59:57 crc kubenswrapper[5052]: I1203 11:59:57.242531 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-dlzcc" Dec 03 11:59:57 crc kubenswrapper[5052]: I1203 11:59:57.246882 5052 generic.go:334] "Generic (PLEG): container finished" podID="ede50db6-6b4c-4dd5-9f79-cd76d5de6a99" containerID="41d12baf2450bd08a8a04af49248da7d52b30dcec65dec130a07bbe2795bb68c" exitCode=143 Dec 03 11:59:57 crc kubenswrapper[5052]: I1203 11:59:57.246977 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"ede50db6-6b4c-4dd5-9f79-cd76d5de6a99","Type":"ContainerDied","Data":"41d12baf2450bd08a8a04af49248da7d52b30dcec65dec130a07bbe2795bb68c"} Dec 03 11:59:57 crc kubenswrapper[5052]: I1203 11:59:57.320406 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-0"] Dec 03 11:59:57 crc kubenswrapper[5052]: E1203 11:59:57.320841 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="091d6975-4400-4dde-b61c-bd635c1ed1f7" containerName="nova-cell1-conductor-db-sync" Dec 03 11:59:57 crc kubenswrapper[5052]: I1203 11:59:57.320858 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="091d6975-4400-4dde-b61c-bd635c1ed1f7" containerName="nova-cell1-conductor-db-sync" Dec 03 11:59:57 crc kubenswrapper[5052]: E1203 11:59:57.320883 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="03284adf-04cd-4ca4-8b31-7ae483082bba" containerName="nova-manage" Dec 03 11:59:57 crc kubenswrapper[5052]: I1203 11:59:57.320889 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="03284adf-04cd-4ca4-8b31-7ae483082bba" containerName="nova-manage" Dec 03 11:59:57 crc kubenswrapper[5052]: I1203 11:59:57.321079 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="091d6975-4400-4dde-b61c-bd635c1ed1f7" containerName="nova-cell1-conductor-db-sync" Dec 03 11:59:57 crc kubenswrapper[5052]: I1203 11:59:57.321095 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="03284adf-04cd-4ca4-8b31-7ae483082bba" containerName="nova-manage" Dec 03 11:59:57 crc kubenswrapper[5052]: I1203 11:59:57.321754 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 11:59:57 crc kubenswrapper[5052]: I1203 11:59:57.329083 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell1-conductor-config-data" Dec 03 11:59:57 crc kubenswrapper[5052]: I1203 11:59:57.349853 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-0"] Dec 03 11:59:57 crc kubenswrapper[5052]: I1203 11:59:57.413243 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/0a480cd2-586d-4213-beff-424b3d201f8e-config-data\") pod \"nova-cell1-conductor-0\" (UID: \"0a480cd2-586d-4213-beff-424b3d201f8e\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 11:59:57 crc kubenswrapper[5052]: I1203 11:59:57.413337 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vnhjc\" (UniqueName: \"kubernetes.io/projected/0a480cd2-586d-4213-beff-424b3d201f8e-kube-api-access-vnhjc\") pod \"nova-cell1-conductor-0\" (UID: \"0a480cd2-586d-4213-beff-424b3d201f8e\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 11:59:57 crc kubenswrapper[5052]: I1203 11:59:57.413408 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0a480cd2-586d-4213-beff-424b3d201f8e-combined-ca-bundle\") pod \"nova-cell1-conductor-0\" (UID: \"0a480cd2-586d-4213-beff-424b3d201f8e\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 11:59:57 crc kubenswrapper[5052]: I1203 11:59:57.515465 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-vnhjc\" (UniqueName: \"kubernetes.io/projected/0a480cd2-586d-4213-beff-424b3d201f8e-kube-api-access-vnhjc\") pod \"nova-cell1-conductor-0\" (UID: \"0a480cd2-586d-4213-beff-424b3d201f8e\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 11:59:57 crc kubenswrapper[5052]: I1203 11:59:57.515580 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0a480cd2-586d-4213-beff-424b3d201f8e-combined-ca-bundle\") pod \"nova-cell1-conductor-0\" (UID: \"0a480cd2-586d-4213-beff-424b3d201f8e\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 11:59:57 crc kubenswrapper[5052]: I1203 11:59:57.515753 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/0a480cd2-586d-4213-beff-424b3d201f8e-config-data\") pod \"nova-cell1-conductor-0\" (UID: \"0a480cd2-586d-4213-beff-424b3d201f8e\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 11:59:57 crc kubenswrapper[5052]: I1203 11:59:57.521952 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/0a480cd2-586d-4213-beff-424b3d201f8e-config-data\") pod \"nova-cell1-conductor-0\" (UID: \"0a480cd2-586d-4213-beff-424b3d201f8e\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 11:59:57 crc kubenswrapper[5052]: I1203 11:59:57.525238 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0a480cd2-586d-4213-beff-424b3d201f8e-combined-ca-bundle\") pod \"nova-cell1-conductor-0\" (UID: \"0a480cd2-586d-4213-beff-424b3d201f8e\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 11:59:57 crc kubenswrapper[5052]: I1203 11:59:57.569152 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-vnhjc\" (UniqueName: \"kubernetes.io/projected/0a480cd2-586d-4213-beff-424b3d201f8e-kube-api-access-vnhjc\") pod \"nova-cell1-conductor-0\" (UID: \"0a480cd2-586d-4213-beff-424b3d201f8e\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 11:59:57 crc kubenswrapper[5052]: I1203 11:59:57.646924 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 11:59:57 crc kubenswrapper[5052]: I1203 11:59:57.743470 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:59:57 crc kubenswrapper[5052]: I1203 11:59:57.824395 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/3cd98ccc-9e96-4f06-abc5-ef6342585d62-logs\") pod \"3cd98ccc-9e96-4f06-abc5-ef6342585d62\" (UID: \"3cd98ccc-9e96-4f06-abc5-ef6342585d62\") " Dec 03 11:59:57 crc kubenswrapper[5052]: I1203 11:59:57.824865 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3cd98ccc-9e96-4f06-abc5-ef6342585d62-combined-ca-bundle\") pod \"3cd98ccc-9e96-4f06-abc5-ef6342585d62\" (UID: \"3cd98ccc-9e96-4f06-abc5-ef6342585d62\") " Dec 03 11:59:57 crc kubenswrapper[5052]: I1203 11:59:57.824931 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hw7zc\" (UniqueName: \"kubernetes.io/projected/3cd98ccc-9e96-4f06-abc5-ef6342585d62-kube-api-access-hw7zc\") pod \"3cd98ccc-9e96-4f06-abc5-ef6342585d62\" (UID: \"3cd98ccc-9e96-4f06-abc5-ef6342585d62\") " Dec 03 11:59:57 crc kubenswrapper[5052]: I1203 11:59:57.825005 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/3cd98ccc-9e96-4f06-abc5-ef6342585d62-nova-metadata-tls-certs\") pod \"3cd98ccc-9e96-4f06-abc5-ef6342585d62\" (UID: \"3cd98ccc-9e96-4f06-abc5-ef6342585d62\") " Dec 03 11:59:57 crc kubenswrapper[5052]: I1203 11:59:57.825269 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/3cd98ccc-9e96-4f06-abc5-ef6342585d62-config-data\") pod \"3cd98ccc-9e96-4f06-abc5-ef6342585d62\" (UID: \"3cd98ccc-9e96-4f06-abc5-ef6342585d62\") " Dec 03 11:59:57 crc kubenswrapper[5052]: I1203 11:59:57.829323 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3cd98ccc-9e96-4f06-abc5-ef6342585d62-logs" (OuterVolumeSpecName: "logs") pod "3cd98ccc-9e96-4f06-abc5-ef6342585d62" (UID: "3cd98ccc-9e96-4f06-abc5-ef6342585d62"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 11:59:57 crc kubenswrapper[5052]: I1203 11:59:57.831343 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3cd98ccc-9e96-4f06-abc5-ef6342585d62-kube-api-access-hw7zc" (OuterVolumeSpecName: "kube-api-access-hw7zc") pod "3cd98ccc-9e96-4f06-abc5-ef6342585d62" (UID: "3cd98ccc-9e96-4f06-abc5-ef6342585d62"). InnerVolumeSpecName "kube-api-access-hw7zc". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 11:59:57 crc kubenswrapper[5052]: I1203 11:59:57.875927 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3cd98ccc-9e96-4f06-abc5-ef6342585d62-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "3cd98ccc-9e96-4f06-abc5-ef6342585d62" (UID: "3cd98ccc-9e96-4f06-abc5-ef6342585d62"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:59:57 crc kubenswrapper[5052]: I1203 11:59:57.902669 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3cd98ccc-9e96-4f06-abc5-ef6342585d62-config-data" (OuterVolumeSpecName: "config-data") pod "3cd98ccc-9e96-4f06-abc5-ef6342585d62" (UID: "3cd98ccc-9e96-4f06-abc5-ef6342585d62"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:59:57 crc kubenswrapper[5052]: I1203 11:59:57.916366 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3cd98ccc-9e96-4f06-abc5-ef6342585d62-nova-metadata-tls-certs" (OuterVolumeSpecName: "nova-metadata-tls-certs") pod "3cd98ccc-9e96-4f06-abc5-ef6342585d62" (UID: "3cd98ccc-9e96-4f06-abc5-ef6342585d62"). InnerVolumeSpecName "nova-metadata-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 11:59:57 crc kubenswrapper[5052]: I1203 11:59:57.929968 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/3cd98ccc-9e96-4f06-abc5-ef6342585d62-logs\") on node \"crc\" DevicePath \"\"" Dec 03 11:59:57 crc kubenswrapper[5052]: I1203 11:59:57.930012 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3cd98ccc-9e96-4f06-abc5-ef6342585d62-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 11:59:57 crc kubenswrapper[5052]: I1203 11:59:57.930028 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-hw7zc\" (UniqueName: \"kubernetes.io/projected/3cd98ccc-9e96-4f06-abc5-ef6342585d62-kube-api-access-hw7zc\") on node \"crc\" DevicePath \"\"" Dec 03 11:59:57 crc kubenswrapper[5052]: I1203 11:59:57.930043 5052 reconciler_common.go:293] "Volume detached for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/3cd98ccc-9e96-4f06-abc5-ef6342585d62-nova-metadata-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 11:59:57 crc kubenswrapper[5052]: I1203 11:59:57.930054 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/3cd98ccc-9e96-4f06-abc5-ef6342585d62-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 11:59:58 crc kubenswrapper[5052]: W1203 11:59:58.123534 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0a480cd2_586d_4213_beff_424b3d201f8e.slice/crio-6ac2bb981783b44c6199342e5aed0c278d04b5124604075498466b014b0e1e07 WatchSource:0}: Error finding container 6ac2bb981783b44c6199342e5aed0c278d04b5124604075498466b014b0e1e07: Status 404 returned error can't find the container with id 6ac2bb981783b44c6199342e5aed0c278d04b5124604075498466b014b0e1e07 Dec 03 11:59:58 crc kubenswrapper[5052]: I1203 11:59:58.129446 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-0"] Dec 03 11:59:58 crc kubenswrapper[5052]: I1203 11:59:58.258398 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-conductor-0" event={"ID":"0a480cd2-586d-4213-beff-424b3d201f8e","Type":"ContainerStarted","Data":"6ac2bb981783b44c6199342e5aed0c278d04b5124604075498466b014b0e1e07"} Dec 03 11:59:58 crc kubenswrapper[5052]: I1203 11:59:58.261607 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"3cd98ccc-9e96-4f06-abc5-ef6342585d62","Type":"ContainerDied","Data":"054ca984a2231b9f8b59b25c1e50671e4be6b5df8f7689c86aa911207d074b83"} Dec 03 11:59:58 crc kubenswrapper[5052]: I1203 11:59:58.261667 5052 scope.go:117] "RemoveContainer" containerID="d4ef4097dfbeeeade7c6933eda2f2f879f07022db9f47964fc70b63943ac8259" Dec 03 11:59:58 crc kubenswrapper[5052]: I1203 11:59:58.261679 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:59:58 crc kubenswrapper[5052]: I1203 11:59:58.261730 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-scheduler-0" podUID="79b9410f-a5f6-48ab-a697-0aa515754ab0" containerName="nova-scheduler-scheduler" containerID="cri-o://40df4e21c3e08adbee7e844764da6ada1d8397a746cbb932f76f824d48838795" gracePeriod=30 Dec 03 11:59:58 crc kubenswrapper[5052]: I1203 11:59:58.291971 5052 scope.go:117] "RemoveContainer" containerID="fb631af9ed30c3aa9e9d9c6d3aa2a169ddef995774a2bf394ecb585b3bb5d3df" Dec 03 11:59:58 crc kubenswrapper[5052]: I1203 11:59:58.303073 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 11:59:58 crc kubenswrapper[5052]: I1203 11:59:58.316757 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 11:59:58 crc kubenswrapper[5052]: I1203 11:59:58.326644 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 11:59:58 crc kubenswrapper[5052]: E1203 11:59:58.327282 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="3cd98ccc-9e96-4f06-abc5-ef6342585d62" containerName="nova-metadata-log" Dec 03 11:59:58 crc kubenswrapper[5052]: I1203 11:59:58.327345 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="3cd98ccc-9e96-4f06-abc5-ef6342585d62" containerName="nova-metadata-log" Dec 03 11:59:58 crc kubenswrapper[5052]: E1203 11:59:58.327406 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="3cd98ccc-9e96-4f06-abc5-ef6342585d62" containerName="nova-metadata-metadata" Dec 03 11:59:58 crc kubenswrapper[5052]: I1203 11:59:58.327451 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="3cd98ccc-9e96-4f06-abc5-ef6342585d62" containerName="nova-metadata-metadata" Dec 03 11:59:58 crc kubenswrapper[5052]: I1203 11:59:58.327786 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="3cd98ccc-9e96-4f06-abc5-ef6342585d62" containerName="nova-metadata-metadata" Dec 03 11:59:58 crc kubenswrapper[5052]: I1203 11:59:58.327850 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="3cd98ccc-9e96-4f06-abc5-ef6342585d62" containerName="nova-metadata-log" Dec 03 11:59:58 crc kubenswrapper[5052]: I1203 11:59:58.328896 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:59:58 crc kubenswrapper[5052]: I1203 11:59:58.331056 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-nova-metadata-internal-svc" Dec 03 11:59:58 crc kubenswrapper[5052]: I1203 11:59:58.332632 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-metadata-config-data" Dec 03 11:59:58 crc kubenswrapper[5052]: I1203 11:59:58.337111 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 11:59:58 crc kubenswrapper[5052]: I1203 11:59:58.437388 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/57aac579-21a0-4c70-937e-1f8cda2fdcf8-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"57aac579-21a0-4c70-937e-1f8cda2fdcf8\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:59:58 crc kubenswrapper[5052]: I1203 11:59:58.437728 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/57aac579-21a0-4c70-937e-1f8cda2fdcf8-nova-metadata-tls-certs\") pod \"nova-metadata-0\" (UID: \"57aac579-21a0-4c70-937e-1f8cda2fdcf8\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:59:58 crc kubenswrapper[5052]: I1203 11:59:58.437759 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/57aac579-21a0-4c70-937e-1f8cda2fdcf8-config-data\") pod \"nova-metadata-0\" (UID: \"57aac579-21a0-4c70-937e-1f8cda2fdcf8\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:59:58 crc kubenswrapper[5052]: I1203 11:59:58.437878 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jmpmp\" (UniqueName: \"kubernetes.io/projected/57aac579-21a0-4c70-937e-1f8cda2fdcf8-kube-api-access-jmpmp\") pod \"nova-metadata-0\" (UID: \"57aac579-21a0-4c70-937e-1f8cda2fdcf8\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:59:58 crc kubenswrapper[5052]: I1203 11:59:58.438065 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/57aac579-21a0-4c70-937e-1f8cda2fdcf8-logs\") pod \"nova-metadata-0\" (UID: \"57aac579-21a0-4c70-937e-1f8cda2fdcf8\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:59:58 crc kubenswrapper[5052]: I1203 11:59:58.539310 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/57aac579-21a0-4c70-937e-1f8cda2fdcf8-logs\") pod \"nova-metadata-0\" (UID: \"57aac579-21a0-4c70-937e-1f8cda2fdcf8\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:59:58 crc kubenswrapper[5052]: I1203 11:59:58.539394 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/57aac579-21a0-4c70-937e-1f8cda2fdcf8-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"57aac579-21a0-4c70-937e-1f8cda2fdcf8\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:59:58 crc kubenswrapper[5052]: I1203 11:59:58.539447 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/57aac579-21a0-4c70-937e-1f8cda2fdcf8-nova-metadata-tls-certs\") pod \"nova-metadata-0\" (UID: \"57aac579-21a0-4c70-937e-1f8cda2fdcf8\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:59:58 crc kubenswrapper[5052]: I1203 11:59:58.539473 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/57aac579-21a0-4c70-937e-1f8cda2fdcf8-config-data\") pod \"nova-metadata-0\" (UID: \"57aac579-21a0-4c70-937e-1f8cda2fdcf8\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:59:58 crc kubenswrapper[5052]: I1203 11:59:58.539577 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-jmpmp\" (UniqueName: \"kubernetes.io/projected/57aac579-21a0-4c70-937e-1f8cda2fdcf8-kube-api-access-jmpmp\") pod \"nova-metadata-0\" (UID: \"57aac579-21a0-4c70-937e-1f8cda2fdcf8\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:59:58 crc kubenswrapper[5052]: I1203 11:59:58.539839 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/57aac579-21a0-4c70-937e-1f8cda2fdcf8-logs\") pod \"nova-metadata-0\" (UID: \"57aac579-21a0-4c70-937e-1f8cda2fdcf8\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:59:58 crc kubenswrapper[5052]: I1203 11:59:58.544902 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/57aac579-21a0-4c70-937e-1f8cda2fdcf8-config-data\") pod \"nova-metadata-0\" (UID: \"57aac579-21a0-4c70-937e-1f8cda2fdcf8\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:59:58 crc kubenswrapper[5052]: I1203 11:59:58.546495 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/57aac579-21a0-4c70-937e-1f8cda2fdcf8-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"57aac579-21a0-4c70-937e-1f8cda2fdcf8\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:59:58 crc kubenswrapper[5052]: I1203 11:59:58.554951 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/57aac579-21a0-4c70-937e-1f8cda2fdcf8-nova-metadata-tls-certs\") pod \"nova-metadata-0\" (UID: \"57aac579-21a0-4c70-937e-1f8cda2fdcf8\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:59:58 crc kubenswrapper[5052]: I1203 11:59:58.559069 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-jmpmp\" (UniqueName: \"kubernetes.io/projected/57aac579-21a0-4c70-937e-1f8cda2fdcf8-kube-api-access-jmpmp\") pod \"nova-metadata-0\" (UID: \"57aac579-21a0-4c70-937e-1f8cda2fdcf8\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:59:58 crc kubenswrapper[5052]: I1203 11:59:58.653738 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 11:59:59 crc kubenswrapper[5052]: I1203 11:59:59.155058 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 11:59:59 crc kubenswrapper[5052]: I1203 11:59:59.277311 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"57aac579-21a0-4c70-937e-1f8cda2fdcf8","Type":"ContainerStarted","Data":"6a62f38d2d9751d6ac145d3ea02a1610e3fb7fcd4e7e20de15f29e60aa3d5945"} Dec 03 11:59:59 crc kubenswrapper[5052]: I1203 11:59:59.279407 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-conductor-0" event={"ID":"0a480cd2-586d-4213-beff-424b3d201f8e","Type":"ContainerStarted","Data":"84a4a97816b15f9f8b42611faefec10e993e636d5d1d408805fe13fa7df1d13e"} Dec 03 11:59:59 crc kubenswrapper[5052]: I1203 11:59:59.280019 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 11:59:59 crc kubenswrapper[5052]: I1203 11:59:59.354659 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-cell1-conductor-0" podStartSLOduration=2.354636713 podStartE2EDuration="2.354636713s" podCreationTimestamp="2025-12-03 11:59:57 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 11:59:59.353012117 +0000 UTC m=+2389.941253881" watchObservedRunningTime="2025-12-03 11:59:59.354636713 +0000 UTC m=+2389.942878477" Dec 03 11:59:59 crc kubenswrapper[5052]: I1203 11:59:59.681997 5052 scope.go:117] "RemoveContainer" containerID="d224717c6d9ca8135fd510e03a6c40c9f2b1a56dd8b4a3b685af3e2dfa855982" Dec 03 11:59:59 crc kubenswrapper[5052]: E1203 11:59:59.682991 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 11:59:59 crc kubenswrapper[5052]: I1203 11:59:59.696154 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3cd98ccc-9e96-4f06-abc5-ef6342585d62" path="/var/lib/kubelet/pods/3cd98ccc-9e96-4f06-abc5-ef6342585d62/volumes" Dec 03 12:00:00 crc kubenswrapper[5052]: I1203 12:00:00.037157 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:00:00 crc kubenswrapper[5052]: I1203 12:00:00.147669 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openshift-operator-lifecycle-manager/collect-profiles-29412720-28m24"] Dec 03 12:00:00 crc kubenswrapper[5052]: E1203 12:00:00.148496 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="79b9410f-a5f6-48ab-a697-0aa515754ab0" containerName="nova-scheduler-scheduler" Dec 03 12:00:00 crc kubenswrapper[5052]: I1203 12:00:00.148540 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="79b9410f-a5f6-48ab-a697-0aa515754ab0" containerName="nova-scheduler-scheduler" Dec 03 12:00:00 crc kubenswrapper[5052]: I1203 12:00:00.148768 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="79b9410f-a5f6-48ab-a697-0aa515754ab0" containerName="nova-scheduler-scheduler" Dec 03 12:00:00 crc kubenswrapper[5052]: I1203 12:00:00.149616 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-operator-lifecycle-manager/collect-profiles-29412720-28m24" Dec 03 12:00:00 crc kubenswrapper[5052]: I1203 12:00:00.152942 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-operator-lifecycle-manager"/"collect-profiles-config" Dec 03 12:00:00 crc kubenswrapper[5052]: I1203 12:00:00.153191 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-operator-lifecycle-manager"/"collect-profiles-dockercfg-kzf4t" Dec 03 12:00:00 crc kubenswrapper[5052]: I1203 12:00:00.159054 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-operator-lifecycle-manager/collect-profiles-29412720-28m24"] Dec 03 12:00:00 crc kubenswrapper[5052]: I1203 12:00:00.175367 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9fkmg\" (UniqueName: \"kubernetes.io/projected/79b9410f-a5f6-48ab-a697-0aa515754ab0-kube-api-access-9fkmg\") pod \"79b9410f-a5f6-48ab-a697-0aa515754ab0\" (UID: \"79b9410f-a5f6-48ab-a697-0aa515754ab0\") " Dec 03 12:00:00 crc kubenswrapper[5052]: I1203 12:00:00.175457 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/79b9410f-a5f6-48ab-a697-0aa515754ab0-config-data\") pod \"79b9410f-a5f6-48ab-a697-0aa515754ab0\" (UID: \"79b9410f-a5f6-48ab-a697-0aa515754ab0\") " Dec 03 12:00:00 crc kubenswrapper[5052]: I1203 12:00:00.175593 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/79b9410f-a5f6-48ab-a697-0aa515754ab0-combined-ca-bundle\") pod \"79b9410f-a5f6-48ab-a697-0aa515754ab0\" (UID: \"79b9410f-a5f6-48ab-a697-0aa515754ab0\") " Dec 03 12:00:00 crc kubenswrapper[5052]: I1203 12:00:00.181278 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/79b9410f-a5f6-48ab-a697-0aa515754ab0-kube-api-access-9fkmg" (OuterVolumeSpecName: "kube-api-access-9fkmg") pod "79b9410f-a5f6-48ab-a697-0aa515754ab0" (UID: "79b9410f-a5f6-48ab-a697-0aa515754ab0"). InnerVolumeSpecName "kube-api-access-9fkmg". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:00:00 crc kubenswrapper[5052]: I1203 12:00:00.206597 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/79b9410f-a5f6-48ab-a697-0aa515754ab0-config-data" (OuterVolumeSpecName: "config-data") pod "79b9410f-a5f6-48ab-a697-0aa515754ab0" (UID: "79b9410f-a5f6-48ab-a697-0aa515754ab0"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:00:00 crc kubenswrapper[5052]: I1203 12:00:00.207748 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/79b9410f-a5f6-48ab-a697-0aa515754ab0-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "79b9410f-a5f6-48ab-a697-0aa515754ab0" (UID: "79b9410f-a5f6-48ab-a697-0aa515754ab0"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:00:00 crc kubenswrapper[5052]: I1203 12:00:00.277491 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f268de6e-4938-4114-b436-9515d64d3f60-config-volume\") pod \"collect-profiles-29412720-28m24\" (UID: \"f268de6e-4938-4114-b436-9515d64d3f60\") " pod="openshift-operator-lifecycle-manager/collect-profiles-29412720-28m24" Dec 03 12:00:00 crc kubenswrapper[5052]: I1203 12:00:00.277614 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-drkg4\" (UniqueName: \"kubernetes.io/projected/f268de6e-4938-4114-b436-9515d64d3f60-kube-api-access-drkg4\") pod \"collect-profiles-29412720-28m24\" (UID: \"f268de6e-4938-4114-b436-9515d64d3f60\") " pod="openshift-operator-lifecycle-manager/collect-profiles-29412720-28m24" Dec 03 12:00:00 crc kubenswrapper[5052]: I1203 12:00:00.277645 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-volume\" (UniqueName: \"kubernetes.io/secret/f268de6e-4938-4114-b436-9515d64d3f60-secret-volume\") pod \"collect-profiles-29412720-28m24\" (UID: \"f268de6e-4938-4114-b436-9515d64d3f60\") " pod="openshift-operator-lifecycle-manager/collect-profiles-29412720-28m24" Dec 03 12:00:00 crc kubenswrapper[5052]: I1203 12:00:00.277761 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-9fkmg\" (UniqueName: \"kubernetes.io/projected/79b9410f-a5f6-48ab-a697-0aa515754ab0-kube-api-access-9fkmg\") on node \"crc\" DevicePath \"\"" Dec 03 12:00:00 crc kubenswrapper[5052]: I1203 12:00:00.277776 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/79b9410f-a5f6-48ab-a697-0aa515754ab0-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:00:00 crc kubenswrapper[5052]: I1203 12:00:00.277786 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/79b9410f-a5f6-48ab-a697-0aa515754ab0-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:00:00 crc kubenswrapper[5052]: I1203 12:00:00.294229 5052 generic.go:334] "Generic (PLEG): container finished" podID="79b9410f-a5f6-48ab-a697-0aa515754ab0" containerID="40df4e21c3e08adbee7e844764da6ada1d8397a746cbb932f76f824d48838795" exitCode=0 Dec 03 12:00:00 crc kubenswrapper[5052]: I1203 12:00:00.294310 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"79b9410f-a5f6-48ab-a697-0aa515754ab0","Type":"ContainerDied","Data":"40df4e21c3e08adbee7e844764da6ada1d8397a746cbb932f76f824d48838795"} Dec 03 12:00:00 crc kubenswrapper[5052]: I1203 12:00:00.294345 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"79b9410f-a5f6-48ab-a697-0aa515754ab0","Type":"ContainerDied","Data":"77ad9d745342d82cd0811540a1915d09073d8ab4d0f16bc8fda9937af750b125"} Dec 03 12:00:00 crc kubenswrapper[5052]: I1203 12:00:00.294369 5052 scope.go:117] "RemoveContainer" containerID="40df4e21c3e08adbee7e844764da6ada1d8397a746cbb932f76f824d48838795" Dec 03 12:00:00 crc kubenswrapper[5052]: I1203 12:00:00.294539 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:00:00 crc kubenswrapper[5052]: I1203 12:00:00.304841 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"57aac579-21a0-4c70-937e-1f8cda2fdcf8","Type":"ContainerStarted","Data":"57b8a2a187f58262450a502c6a73481dc9325be51ee5640a105953473b7e6d2d"} Dec 03 12:00:00 crc kubenswrapper[5052]: I1203 12:00:00.304912 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"57aac579-21a0-4c70-937e-1f8cda2fdcf8","Type":"ContainerStarted","Data":"108b04271be86cc94040113d925492ae24fbcfeb479bddcda4881f05086c7288"} Dec 03 12:00:00 crc kubenswrapper[5052]: I1203 12:00:00.330265 5052 scope.go:117] "RemoveContainer" containerID="40df4e21c3e08adbee7e844764da6ada1d8397a746cbb932f76f824d48838795" Dec 03 12:00:00 crc kubenswrapper[5052]: E1203 12:00:00.330906 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"40df4e21c3e08adbee7e844764da6ada1d8397a746cbb932f76f824d48838795\": container with ID starting with 40df4e21c3e08adbee7e844764da6ada1d8397a746cbb932f76f824d48838795 not found: ID does not exist" containerID="40df4e21c3e08adbee7e844764da6ada1d8397a746cbb932f76f824d48838795" Dec 03 12:00:00 crc kubenswrapper[5052]: I1203 12:00:00.330962 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"40df4e21c3e08adbee7e844764da6ada1d8397a746cbb932f76f824d48838795"} err="failed to get container status \"40df4e21c3e08adbee7e844764da6ada1d8397a746cbb932f76f824d48838795\": rpc error: code = NotFound desc = could not find container \"40df4e21c3e08adbee7e844764da6ada1d8397a746cbb932f76f824d48838795\": container with ID starting with 40df4e21c3e08adbee7e844764da6ada1d8397a746cbb932f76f824d48838795 not found: ID does not exist" Dec 03 12:00:00 crc kubenswrapper[5052]: I1203 12:00:00.337896 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-metadata-0" podStartSLOduration=2.337878351 podStartE2EDuration="2.337878351s" podCreationTimestamp="2025-12-03 11:59:58 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:00:00.322213933 +0000 UTC m=+2390.910455707" watchObservedRunningTime="2025-12-03 12:00:00.337878351 +0000 UTC m=+2390.926120115" Dec 03 12:00:00 crc kubenswrapper[5052]: I1203 12:00:00.360895 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:00:00 crc kubenswrapper[5052]: I1203 12:00:00.378052 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:00:00 crc kubenswrapper[5052]: I1203 12:00:00.378891 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-drkg4\" (UniqueName: \"kubernetes.io/projected/f268de6e-4938-4114-b436-9515d64d3f60-kube-api-access-drkg4\") pod \"collect-profiles-29412720-28m24\" (UID: \"f268de6e-4938-4114-b436-9515d64d3f60\") " pod="openshift-operator-lifecycle-manager/collect-profiles-29412720-28m24" Dec 03 12:00:00 crc kubenswrapper[5052]: I1203 12:00:00.378984 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"secret-volume\" (UniqueName: \"kubernetes.io/secret/f268de6e-4938-4114-b436-9515d64d3f60-secret-volume\") pod \"collect-profiles-29412720-28m24\" (UID: \"f268de6e-4938-4114-b436-9515d64d3f60\") " pod="openshift-operator-lifecycle-manager/collect-profiles-29412720-28m24" Dec 03 12:00:00 crc kubenswrapper[5052]: I1203 12:00:00.379184 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f268de6e-4938-4114-b436-9515d64d3f60-config-volume\") pod \"collect-profiles-29412720-28m24\" (UID: \"f268de6e-4938-4114-b436-9515d64d3f60\") " pod="openshift-operator-lifecycle-manager/collect-profiles-29412720-28m24" Dec 03 12:00:00 crc kubenswrapper[5052]: I1203 12:00:00.380221 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f268de6e-4938-4114-b436-9515d64d3f60-config-volume\") pod \"collect-profiles-29412720-28m24\" (UID: \"f268de6e-4938-4114-b436-9515d64d3f60\") " pod="openshift-operator-lifecycle-manager/collect-profiles-29412720-28m24" Dec 03 12:00:00 crc kubenswrapper[5052]: I1203 12:00:00.384175 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"secret-volume\" (UniqueName: \"kubernetes.io/secret/f268de6e-4938-4114-b436-9515d64d3f60-secret-volume\") pod \"collect-profiles-29412720-28m24\" (UID: \"f268de6e-4938-4114-b436-9515d64d3f60\") " pod="openshift-operator-lifecycle-manager/collect-profiles-29412720-28m24" Dec 03 12:00:00 crc kubenswrapper[5052]: I1203 12:00:00.392772 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:00:00 crc kubenswrapper[5052]: I1203 12:00:00.394217 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:00:00 crc kubenswrapper[5052]: I1203 12:00:00.397845 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-scheduler-config-data" Dec 03 12:00:00 crc kubenswrapper[5052]: I1203 12:00:00.402283 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-drkg4\" (UniqueName: \"kubernetes.io/projected/f268de6e-4938-4114-b436-9515d64d3f60-kube-api-access-drkg4\") pod \"collect-profiles-29412720-28m24\" (UID: \"f268de6e-4938-4114-b436-9515d64d3f60\") " pod="openshift-operator-lifecycle-manager/collect-profiles-29412720-28m24" Dec 03 12:00:00 crc kubenswrapper[5052]: I1203 12:00:00.426112 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:00:00 crc kubenswrapper[5052]: I1203 12:00:00.471860 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-operator-lifecycle-manager/collect-profiles-29412720-28m24" Dec 03 12:00:00 crc kubenswrapper[5052]: I1203 12:00:00.480776 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m465p\" (UniqueName: \"kubernetes.io/projected/25079157-2add-4870-a7a9-e002013bdec0-kube-api-access-m465p\") pod \"nova-scheduler-0\" (UID: \"25079157-2add-4870-a7a9-e002013bdec0\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:00:00 crc kubenswrapper[5052]: I1203 12:00:00.480841 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/25079157-2add-4870-a7a9-e002013bdec0-combined-ca-bundle\") pod \"nova-scheduler-0\" (UID: \"25079157-2add-4870-a7a9-e002013bdec0\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:00:00 crc kubenswrapper[5052]: I1203 12:00:00.480951 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/25079157-2add-4870-a7a9-e002013bdec0-config-data\") pod \"nova-scheduler-0\" (UID: \"25079157-2add-4870-a7a9-e002013bdec0\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:00:00 crc kubenswrapper[5052]: I1203 12:00:00.583416 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/25079157-2add-4870-a7a9-e002013bdec0-config-data\") pod \"nova-scheduler-0\" (UID: \"25079157-2add-4870-a7a9-e002013bdec0\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:00:00 crc kubenswrapper[5052]: I1203 12:00:00.583786 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-m465p\" (UniqueName: \"kubernetes.io/projected/25079157-2add-4870-a7a9-e002013bdec0-kube-api-access-m465p\") pod \"nova-scheduler-0\" (UID: \"25079157-2add-4870-a7a9-e002013bdec0\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:00:00 crc kubenswrapper[5052]: I1203 12:00:00.583827 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/25079157-2add-4870-a7a9-e002013bdec0-combined-ca-bundle\") pod \"nova-scheduler-0\" (UID: \"25079157-2add-4870-a7a9-e002013bdec0\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:00:00 crc kubenswrapper[5052]: I1203 12:00:00.588618 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/25079157-2add-4870-a7a9-e002013bdec0-config-data\") pod \"nova-scheduler-0\" (UID: \"25079157-2add-4870-a7a9-e002013bdec0\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:00:00 crc kubenswrapper[5052]: I1203 12:00:00.593913 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/25079157-2add-4870-a7a9-e002013bdec0-combined-ca-bundle\") pod \"nova-scheduler-0\" (UID: \"25079157-2add-4870-a7a9-e002013bdec0\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:00:00 crc kubenswrapper[5052]: I1203 12:00:00.614601 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-m465p\" (UniqueName: \"kubernetes.io/projected/25079157-2add-4870-a7a9-e002013bdec0-kube-api-access-m465p\") pod \"nova-scheduler-0\" (UID: \"25079157-2add-4870-a7a9-e002013bdec0\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:00:00 crc kubenswrapper[5052]: I1203 12:00:00.713188 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:00:00 crc kubenswrapper[5052]: I1203 12:00:00.967117 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-operator-lifecycle-manager/collect-profiles-29412720-28m24"] Dec 03 12:00:00 crc kubenswrapper[5052]: W1203 12:00:00.969821 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf268de6e_4938_4114_b436_9515d64d3f60.slice/crio-bdc7cd76d172942b35eef82631c98755a1160df4db3a2c612d1a37a753b3ed60 WatchSource:0}: Error finding container bdc7cd76d172942b35eef82631c98755a1160df4db3a2c612d1a37a753b3ed60: Status 404 returned error can't find the container with id bdc7cd76d172942b35eef82631c98755a1160df4db3a2c612d1a37a753b3ed60 Dec 03 12:00:01 crc kubenswrapper[5052]: W1203 12:00:01.178575 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod25079157_2add_4870_a7a9_e002013bdec0.slice/crio-c2989297dc81507d947678a3fff0076b7fe50fed3d9cb397e30e5bf635771739 WatchSource:0}: Error finding container c2989297dc81507d947678a3fff0076b7fe50fed3d9cb397e30e5bf635771739: Status 404 returned error can't find the container with id c2989297dc81507d947678a3fff0076b7fe50fed3d9cb397e30e5bf635771739 Dec 03 12:00:01 crc kubenswrapper[5052]: I1203 12:00:01.178924 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:00:01 crc kubenswrapper[5052]: I1203 12:00:01.312428 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-operator-lifecycle-manager/collect-profiles-29412720-28m24" event={"ID":"f268de6e-4938-4114-b436-9515d64d3f60","Type":"ContainerStarted","Data":"d09c826d0d22b153c916799836e2f1426871c707d4364227e89e6fd55c6f616d"} Dec 03 12:00:01 crc kubenswrapper[5052]: I1203 12:00:01.312472 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-operator-lifecycle-manager/collect-profiles-29412720-28m24" event={"ID":"f268de6e-4938-4114-b436-9515d64d3f60","Type":"ContainerStarted","Data":"bdc7cd76d172942b35eef82631c98755a1160df4db3a2c612d1a37a753b3ed60"} Dec 03 12:00:01 crc kubenswrapper[5052]: I1203 12:00:01.314652 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"25079157-2add-4870-a7a9-e002013bdec0","Type":"ContainerStarted","Data":"c2989297dc81507d947678a3fff0076b7fe50fed3d9cb397e30e5bf635771739"} Dec 03 12:00:01 crc kubenswrapper[5052]: I1203 12:00:01.328242 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-operator-lifecycle-manager/collect-profiles-29412720-28m24" podStartSLOduration=1.328221904 podStartE2EDuration="1.328221904s" podCreationTimestamp="2025-12-03 12:00:00 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:00:01.327576586 +0000 UTC m=+2391.915818350" watchObservedRunningTime="2025-12-03 12:00:01.328221904 +0000 UTC m=+2391.916463668" Dec 03 12:00:01 crc kubenswrapper[5052]: I1203 12:00:01.684272 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="79b9410f-a5f6-48ab-a697-0aa515754ab0" path="/var/lib/kubelet/pods/79b9410f-a5f6-48ab-a697-0aa515754ab0/volumes" Dec 03 12:00:02 crc kubenswrapper[5052]: I1203 12:00:02.171407 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:00:02 crc kubenswrapper[5052]: I1203 12:00:02.321065 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/ede50db6-6b4c-4dd5-9f79-cd76d5de6a99-logs\") pod \"ede50db6-6b4c-4dd5-9f79-cd76d5de6a99\" (UID: \"ede50db6-6b4c-4dd5-9f79-cd76d5de6a99\") " Dec 03 12:00:02 crc kubenswrapper[5052]: I1203 12:00:02.321142 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ede50db6-6b4c-4dd5-9f79-cd76d5de6a99-combined-ca-bundle\") pod \"ede50db6-6b4c-4dd5-9f79-cd76d5de6a99\" (UID: \"ede50db6-6b4c-4dd5-9f79-cd76d5de6a99\") " Dec 03 12:00:02 crc kubenswrapper[5052]: I1203 12:00:02.321270 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/ede50db6-6b4c-4dd5-9f79-cd76d5de6a99-config-data\") pod \"ede50db6-6b4c-4dd5-9f79-cd76d5de6a99\" (UID: \"ede50db6-6b4c-4dd5-9f79-cd76d5de6a99\") " Dec 03 12:00:02 crc kubenswrapper[5052]: I1203 12:00:02.321301 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q6pbq\" (UniqueName: \"kubernetes.io/projected/ede50db6-6b4c-4dd5-9f79-cd76d5de6a99-kube-api-access-q6pbq\") pod \"ede50db6-6b4c-4dd5-9f79-cd76d5de6a99\" (UID: \"ede50db6-6b4c-4dd5-9f79-cd76d5de6a99\") " Dec 03 12:00:02 crc kubenswrapper[5052]: I1203 12:00:02.321937 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ede50db6-6b4c-4dd5-9f79-cd76d5de6a99-logs" (OuterVolumeSpecName: "logs") pod "ede50db6-6b4c-4dd5-9f79-cd76d5de6a99" (UID: "ede50db6-6b4c-4dd5-9f79-cd76d5de6a99"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:00:02 crc kubenswrapper[5052]: I1203 12:00:02.328258 5052 generic.go:334] "Generic (PLEG): container finished" podID="ede50db6-6b4c-4dd5-9f79-cd76d5de6a99" containerID="2ac74a9091c3ed67a00d2b336d665cba78befa7258777b544feace96771b2c48" exitCode=0 Dec 03 12:00:02 crc kubenswrapper[5052]: I1203 12:00:02.328339 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:00:02 crc kubenswrapper[5052]: I1203 12:00:02.328357 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"ede50db6-6b4c-4dd5-9f79-cd76d5de6a99","Type":"ContainerDied","Data":"2ac74a9091c3ed67a00d2b336d665cba78befa7258777b544feace96771b2c48"} Dec 03 12:00:02 crc kubenswrapper[5052]: I1203 12:00:02.329098 5052 scope.go:117] "RemoveContainer" containerID="2ac74a9091c3ed67a00d2b336d665cba78befa7258777b544feace96771b2c48" Dec 03 12:00:02 crc kubenswrapper[5052]: I1203 12:00:02.329210 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"ede50db6-6b4c-4dd5-9f79-cd76d5de6a99","Type":"ContainerDied","Data":"250ec4648768ae29c7948b385e6a9b18073cced52ba1d2f4bd23c6ac91ac9c52"} Dec 03 12:00:02 crc kubenswrapper[5052]: I1203 12:00:02.330391 5052 generic.go:334] "Generic (PLEG): container finished" podID="f268de6e-4938-4114-b436-9515d64d3f60" containerID="d09c826d0d22b153c916799836e2f1426871c707d4364227e89e6fd55c6f616d" exitCode=0 Dec 03 12:00:02 crc kubenswrapper[5052]: I1203 12:00:02.330449 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-operator-lifecycle-manager/collect-profiles-29412720-28m24" event={"ID":"f268de6e-4938-4114-b436-9515d64d3f60","Type":"ContainerDied","Data":"d09c826d0d22b153c916799836e2f1426871c707d4364227e89e6fd55c6f616d"} Dec 03 12:00:02 crc kubenswrapper[5052]: I1203 12:00:02.336416 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"25079157-2add-4870-a7a9-e002013bdec0","Type":"ContainerStarted","Data":"bd545383bb0ebe1c5a551c19ee05eb022d006fd7d6d08e21e1ca450105aa434e"} Dec 03 12:00:02 crc kubenswrapper[5052]: I1203 12:00:02.347694 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ede50db6-6b4c-4dd5-9f79-cd76d5de6a99-kube-api-access-q6pbq" (OuterVolumeSpecName: "kube-api-access-q6pbq") pod "ede50db6-6b4c-4dd5-9f79-cd76d5de6a99" (UID: "ede50db6-6b4c-4dd5-9f79-cd76d5de6a99"). InnerVolumeSpecName "kube-api-access-q6pbq". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:00:02 crc kubenswrapper[5052]: I1203 12:00:02.359686 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ede50db6-6b4c-4dd5-9f79-cd76d5de6a99-config-data" (OuterVolumeSpecName: "config-data") pod "ede50db6-6b4c-4dd5-9f79-cd76d5de6a99" (UID: "ede50db6-6b4c-4dd5-9f79-cd76d5de6a99"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:00:02 crc kubenswrapper[5052]: I1203 12:00:02.371650 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ede50db6-6b4c-4dd5-9f79-cd76d5de6a99-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "ede50db6-6b4c-4dd5-9f79-cd76d5de6a99" (UID: "ede50db6-6b4c-4dd5-9f79-cd76d5de6a99"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:00:02 crc kubenswrapper[5052]: I1203 12:00:02.386009 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-scheduler-0" podStartSLOduration=2.385990076 podStartE2EDuration="2.385990076s" podCreationTimestamp="2025-12-03 12:00:00 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:00:02.368492875 +0000 UTC m=+2392.956734649" watchObservedRunningTime="2025-12-03 12:00:02.385990076 +0000 UTC m=+2392.974231840" Dec 03 12:00:02 crc kubenswrapper[5052]: I1203 12:00:02.399093 5052 scope.go:117] "RemoveContainer" containerID="41d12baf2450bd08a8a04af49248da7d52b30dcec65dec130a07bbe2795bb68c" Dec 03 12:00:02 crc kubenswrapper[5052]: I1203 12:00:02.417315 5052 scope.go:117] "RemoveContainer" containerID="2ac74a9091c3ed67a00d2b336d665cba78befa7258777b544feace96771b2c48" Dec 03 12:00:02 crc kubenswrapper[5052]: E1203 12:00:02.417682 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2ac74a9091c3ed67a00d2b336d665cba78befa7258777b544feace96771b2c48\": container with ID starting with 2ac74a9091c3ed67a00d2b336d665cba78befa7258777b544feace96771b2c48 not found: ID does not exist" containerID="2ac74a9091c3ed67a00d2b336d665cba78befa7258777b544feace96771b2c48" Dec 03 12:00:02 crc kubenswrapper[5052]: I1203 12:00:02.417731 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2ac74a9091c3ed67a00d2b336d665cba78befa7258777b544feace96771b2c48"} err="failed to get container status \"2ac74a9091c3ed67a00d2b336d665cba78befa7258777b544feace96771b2c48\": rpc error: code = NotFound desc = could not find container \"2ac74a9091c3ed67a00d2b336d665cba78befa7258777b544feace96771b2c48\": container with ID starting with 2ac74a9091c3ed67a00d2b336d665cba78befa7258777b544feace96771b2c48 not found: ID does not exist" Dec 03 12:00:02 crc kubenswrapper[5052]: I1203 12:00:02.417758 5052 scope.go:117] "RemoveContainer" containerID="41d12baf2450bd08a8a04af49248da7d52b30dcec65dec130a07bbe2795bb68c" Dec 03 12:00:02 crc kubenswrapper[5052]: E1203 12:00:02.417969 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"41d12baf2450bd08a8a04af49248da7d52b30dcec65dec130a07bbe2795bb68c\": container with ID starting with 41d12baf2450bd08a8a04af49248da7d52b30dcec65dec130a07bbe2795bb68c not found: ID does not exist" containerID="41d12baf2450bd08a8a04af49248da7d52b30dcec65dec130a07bbe2795bb68c" Dec 03 12:00:02 crc kubenswrapper[5052]: I1203 12:00:02.417992 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"41d12baf2450bd08a8a04af49248da7d52b30dcec65dec130a07bbe2795bb68c"} err="failed to get container status \"41d12baf2450bd08a8a04af49248da7d52b30dcec65dec130a07bbe2795bb68c\": rpc error: code = NotFound desc = could not find container \"41d12baf2450bd08a8a04af49248da7d52b30dcec65dec130a07bbe2795bb68c\": container with ID starting with 41d12baf2450bd08a8a04af49248da7d52b30dcec65dec130a07bbe2795bb68c not found: ID does not exist" Dec 03 12:00:02 crc kubenswrapper[5052]: I1203 12:00:02.424244 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/ede50db6-6b4c-4dd5-9f79-cd76d5de6a99-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:00:02 crc kubenswrapper[5052]: I1203 12:00:02.424305 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ede50db6-6b4c-4dd5-9f79-cd76d5de6a99-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:00:02 crc kubenswrapper[5052]: I1203 12:00:02.424319 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/ede50db6-6b4c-4dd5-9f79-cd76d5de6a99-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:00:02 crc kubenswrapper[5052]: I1203 12:00:02.424332 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-q6pbq\" (UniqueName: \"kubernetes.io/projected/ede50db6-6b4c-4dd5-9f79-cd76d5de6a99-kube-api-access-q6pbq\") on node \"crc\" DevicePath \"\"" Dec 03 12:00:02 crc kubenswrapper[5052]: I1203 12:00:02.677407 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:00:02 crc kubenswrapper[5052]: I1203 12:00:02.704161 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:00:02 crc kubenswrapper[5052]: I1203 12:00:02.733774 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:00:02 crc kubenswrapper[5052]: E1203 12:00:02.734378 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="ede50db6-6b4c-4dd5-9f79-cd76d5de6a99" containerName="nova-api-api" Dec 03 12:00:02 crc kubenswrapper[5052]: I1203 12:00:02.734393 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="ede50db6-6b4c-4dd5-9f79-cd76d5de6a99" containerName="nova-api-api" Dec 03 12:00:02 crc kubenswrapper[5052]: E1203 12:00:02.734454 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="ede50db6-6b4c-4dd5-9f79-cd76d5de6a99" containerName="nova-api-log" Dec 03 12:00:02 crc kubenswrapper[5052]: I1203 12:00:02.734462 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="ede50db6-6b4c-4dd5-9f79-cd76d5de6a99" containerName="nova-api-log" Dec 03 12:00:02 crc kubenswrapper[5052]: I1203 12:00:02.734721 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="ede50db6-6b4c-4dd5-9f79-cd76d5de6a99" containerName="nova-api-log" Dec 03 12:00:02 crc kubenswrapper[5052]: I1203 12:00:02.734738 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="ede50db6-6b4c-4dd5-9f79-cd76d5de6a99" containerName="nova-api-api" Dec 03 12:00:02 crc kubenswrapper[5052]: I1203 12:00:02.739293 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:00:02 crc kubenswrapper[5052]: I1203 12:00:02.743069 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-api-config-data" Dec 03 12:00:02 crc kubenswrapper[5052]: I1203 12:00:02.761451 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:00:02 crc kubenswrapper[5052]: I1203 12:00:02.833822 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dmtxq\" (UniqueName: \"kubernetes.io/projected/eebaba62-8aa5-447a-bcc2-590bb40de88f-kube-api-access-dmtxq\") pod \"nova-api-0\" (UID: \"eebaba62-8aa5-447a-bcc2-590bb40de88f\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:00:02 crc kubenswrapper[5052]: I1203 12:00:02.833880 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/eebaba62-8aa5-447a-bcc2-590bb40de88f-logs\") pod \"nova-api-0\" (UID: \"eebaba62-8aa5-447a-bcc2-590bb40de88f\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:00:02 crc kubenswrapper[5052]: I1203 12:00:02.833993 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/eebaba62-8aa5-447a-bcc2-590bb40de88f-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"eebaba62-8aa5-447a-bcc2-590bb40de88f\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:00:02 crc kubenswrapper[5052]: I1203 12:00:02.834100 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/eebaba62-8aa5-447a-bcc2-590bb40de88f-config-data\") pod \"nova-api-0\" (UID: \"eebaba62-8aa5-447a-bcc2-590bb40de88f\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:00:02 crc kubenswrapper[5052]: I1203 12:00:02.936267 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-dmtxq\" (UniqueName: \"kubernetes.io/projected/eebaba62-8aa5-447a-bcc2-590bb40de88f-kube-api-access-dmtxq\") pod \"nova-api-0\" (UID: \"eebaba62-8aa5-447a-bcc2-590bb40de88f\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:00:02 crc kubenswrapper[5052]: I1203 12:00:02.936324 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/eebaba62-8aa5-447a-bcc2-590bb40de88f-logs\") pod \"nova-api-0\" (UID: \"eebaba62-8aa5-447a-bcc2-590bb40de88f\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:00:02 crc kubenswrapper[5052]: I1203 12:00:02.936383 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/eebaba62-8aa5-447a-bcc2-590bb40de88f-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"eebaba62-8aa5-447a-bcc2-590bb40de88f\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:00:02 crc kubenswrapper[5052]: I1203 12:00:02.936457 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/eebaba62-8aa5-447a-bcc2-590bb40de88f-config-data\") pod \"nova-api-0\" (UID: \"eebaba62-8aa5-447a-bcc2-590bb40de88f\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:00:02 crc kubenswrapper[5052]: I1203 12:00:02.937512 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/eebaba62-8aa5-447a-bcc2-590bb40de88f-logs\") pod \"nova-api-0\" (UID: \"eebaba62-8aa5-447a-bcc2-590bb40de88f\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:00:02 crc kubenswrapper[5052]: I1203 12:00:02.940387 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/eebaba62-8aa5-447a-bcc2-590bb40de88f-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"eebaba62-8aa5-447a-bcc2-590bb40de88f\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:00:02 crc kubenswrapper[5052]: I1203 12:00:02.956803 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/eebaba62-8aa5-447a-bcc2-590bb40de88f-config-data\") pod \"nova-api-0\" (UID: \"eebaba62-8aa5-447a-bcc2-590bb40de88f\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:00:02 crc kubenswrapper[5052]: I1203 12:00:02.968139 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-dmtxq\" (UniqueName: \"kubernetes.io/projected/eebaba62-8aa5-447a-bcc2-590bb40de88f-kube-api-access-dmtxq\") pod \"nova-api-0\" (UID: \"eebaba62-8aa5-447a-bcc2-590bb40de88f\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:00:03 crc kubenswrapper[5052]: I1203 12:00:03.069359 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:00:03 crc kubenswrapper[5052]: I1203 12:00:03.574273 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:00:03 crc kubenswrapper[5052]: I1203 12:00:03.653786 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:00:03 crc kubenswrapper[5052]: I1203 12:00:03.654716 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:00:03 crc kubenswrapper[5052]: I1203 12:00:03.685302 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ede50db6-6b4c-4dd5-9f79-cd76d5de6a99" path="/var/lib/kubelet/pods/ede50db6-6b4c-4dd5-9f79-cd76d5de6a99/volumes" Dec 03 12:00:03 crc kubenswrapper[5052]: I1203 12:00:03.748353 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-operator-lifecycle-manager/collect-profiles-29412720-28m24" Dec 03 12:00:03 crc kubenswrapper[5052]: I1203 12:00:03.856255 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"secret-volume\" (UniqueName: \"kubernetes.io/secret/f268de6e-4938-4114-b436-9515d64d3f60-secret-volume\") pod \"f268de6e-4938-4114-b436-9515d64d3f60\" (UID: \"f268de6e-4938-4114-b436-9515d64d3f60\") " Dec 03 12:00:03 crc kubenswrapper[5052]: I1203 12:00:03.856418 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f268de6e-4938-4114-b436-9515d64d3f60-config-volume\") pod \"f268de6e-4938-4114-b436-9515d64d3f60\" (UID: \"f268de6e-4938-4114-b436-9515d64d3f60\") " Dec 03 12:00:03 crc kubenswrapper[5052]: I1203 12:00:03.856490 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-drkg4\" (UniqueName: \"kubernetes.io/projected/f268de6e-4938-4114-b436-9515d64d3f60-kube-api-access-drkg4\") pod \"f268de6e-4938-4114-b436-9515d64d3f60\" (UID: \"f268de6e-4938-4114-b436-9515d64d3f60\") " Dec 03 12:00:03 crc kubenswrapper[5052]: I1203 12:00:03.859462 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f268de6e-4938-4114-b436-9515d64d3f60-config-volume" (OuterVolumeSpecName: "config-volume") pod "f268de6e-4938-4114-b436-9515d64d3f60" (UID: "f268de6e-4938-4114-b436-9515d64d3f60"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:00:03 crc kubenswrapper[5052]: I1203 12:00:03.862421 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f268de6e-4938-4114-b436-9515d64d3f60-secret-volume" (OuterVolumeSpecName: "secret-volume") pod "f268de6e-4938-4114-b436-9515d64d3f60" (UID: "f268de6e-4938-4114-b436-9515d64d3f60"). InnerVolumeSpecName "secret-volume". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:00:03 crc kubenswrapper[5052]: I1203 12:00:03.862582 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f268de6e-4938-4114-b436-9515d64d3f60-kube-api-access-drkg4" (OuterVolumeSpecName: "kube-api-access-drkg4") pod "f268de6e-4938-4114-b436-9515d64d3f60" (UID: "f268de6e-4938-4114-b436-9515d64d3f60"). InnerVolumeSpecName "kube-api-access-drkg4". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:00:03 crc kubenswrapper[5052]: I1203 12:00:03.958749 5052 reconciler_common.go:293] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f268de6e-4938-4114-b436-9515d64d3f60-config-volume\") on node \"crc\" DevicePath \"\"" Dec 03 12:00:03 crc kubenswrapper[5052]: I1203 12:00:03.959075 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-drkg4\" (UniqueName: \"kubernetes.io/projected/f268de6e-4938-4114-b436-9515d64d3f60-kube-api-access-drkg4\") on node \"crc\" DevicePath \"\"" Dec 03 12:00:03 crc kubenswrapper[5052]: I1203 12:00:03.959088 5052 reconciler_common.go:293] "Volume detached for volume \"secret-volume\" (UniqueName: \"kubernetes.io/secret/f268de6e-4938-4114-b436-9515d64d3f60-secret-volume\") on node \"crc\" DevicePath \"\"" Dec 03 12:00:04 crc kubenswrapper[5052]: I1203 12:00:04.380134 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-operator-lifecycle-manager/collect-profiles-29412720-28m24" Dec 03 12:00:04 crc kubenswrapper[5052]: I1203 12:00:04.380137 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-operator-lifecycle-manager/collect-profiles-29412720-28m24" event={"ID":"f268de6e-4938-4114-b436-9515d64d3f60","Type":"ContainerDied","Data":"bdc7cd76d172942b35eef82631c98755a1160df4db3a2c612d1a37a753b3ed60"} Dec 03 12:00:04 crc kubenswrapper[5052]: I1203 12:00:04.380273 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bdc7cd76d172942b35eef82631c98755a1160df4db3a2c612d1a37a753b3ed60" Dec 03 12:00:04 crc kubenswrapper[5052]: I1203 12:00:04.381957 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"eebaba62-8aa5-447a-bcc2-590bb40de88f","Type":"ContainerStarted","Data":"1a49b9d7137fb6a41e5ffeefb32672f648c89fd732896a81025268cb006acfaf"} Dec 03 12:00:04 crc kubenswrapper[5052]: I1203 12:00:04.382006 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"eebaba62-8aa5-447a-bcc2-590bb40de88f","Type":"ContainerStarted","Data":"69ee8e1bd36c85abc2f41d4ac965fe34ba9a46a96372eb56d8bb396f2b5d9337"} Dec 03 12:00:04 crc kubenswrapper[5052]: I1203 12:00:04.382023 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"eebaba62-8aa5-447a-bcc2-590bb40de88f","Type":"ContainerStarted","Data":"0da99586eb02a317f52fc0a1ac125a104da1eee129069c6fd6c62850608eceb6"} Dec 03 12:00:04 crc kubenswrapper[5052]: I1203 12:00:04.455384 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-api-0" podStartSLOduration=2.455360558 podStartE2EDuration="2.455360558s" podCreationTimestamp="2025-12-03 12:00:02 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:00:04.428314504 +0000 UTC m=+2395.016556278" watchObservedRunningTime="2025-12-03 12:00:04.455360558 +0000 UTC m=+2395.043602332" Dec 03 12:00:04 crc kubenswrapper[5052]: I1203 12:00:04.473757 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openshift-operator-lifecycle-manager/collect-profiles-29412675-hzd4n"] Dec 03 12:00:04 crc kubenswrapper[5052]: I1203 12:00:04.486438 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openshift-operator-lifecycle-manager/collect-profiles-29412675-hzd4n"] Dec 03 12:00:05 crc kubenswrapper[5052]: I1203 12:00:05.686738 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e43f14fc-f3eb-4f58-b202-cdfc641ac08a" path="/var/lib/kubelet/pods/e43f14fc-f3eb-4f58-b202-cdfc641ac08a/volumes" Dec 03 12:00:05 crc kubenswrapper[5052]: I1203 12:00:05.719256 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:00:06 crc kubenswrapper[5052]: I1203 12:00:06.318409 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:00:07 crc kubenswrapper[5052]: I1203 12:00:07.696642 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:00:08 crc kubenswrapper[5052]: I1203 12:00:08.654099 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:00:08 crc kubenswrapper[5052]: I1203 12:00:08.654494 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:00:09 crc kubenswrapper[5052]: I1203 12:00:09.666736 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-metadata-0" podUID="57aac579-21a0-4c70-937e-1f8cda2fdcf8" containerName="nova-metadata-metadata" probeResult="failure" output="Get \"https://10.217.1.244:8775/\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Dec 03 12:00:09 crc kubenswrapper[5052]: I1203 12:00:09.666739 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-metadata-0" podUID="57aac579-21a0-4c70-937e-1f8cda2fdcf8" containerName="nova-metadata-log" probeResult="failure" output="Get \"https://10.217.1.244:8775/\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Dec 03 12:00:09 crc kubenswrapper[5052]: I1203 12:00:09.977039 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/kube-state-metrics-0"] Dec 03 12:00:09 crc kubenswrapper[5052]: I1203 12:00:09.977570 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/kube-state-metrics-0" podUID="316d245b-16aa-44a0-8aaa-4a41794e1291" containerName="kube-state-metrics" containerID="cri-o://35db98cfc0c0e4e1307993f339292d48cb6c7695dea2f6b7b92d067b386acd94" gracePeriod=30 Dec 03 12:00:10 crc kubenswrapper[5052]: I1203 12:00:10.442572 5052 generic.go:334] "Generic (PLEG): container finished" podID="316d245b-16aa-44a0-8aaa-4a41794e1291" containerID="35db98cfc0c0e4e1307993f339292d48cb6c7695dea2f6b7b92d067b386acd94" exitCode=2 Dec 03 12:00:10 crc kubenswrapper[5052]: I1203 12:00:10.442683 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/kube-state-metrics-0" event={"ID":"316d245b-16aa-44a0-8aaa-4a41794e1291","Type":"ContainerDied","Data":"35db98cfc0c0e4e1307993f339292d48cb6c7695dea2f6b7b92d067b386acd94"} Dec 03 12:00:10 crc kubenswrapper[5052]: I1203 12:00:10.442973 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/kube-state-metrics-0" event={"ID":"316d245b-16aa-44a0-8aaa-4a41794e1291","Type":"ContainerDied","Data":"89649df4e0c91b3003a7c8e4c4d98473c7fd7488089f8dd1c03e7da5a297c96f"} Dec 03 12:00:10 crc kubenswrapper[5052]: I1203 12:00:10.442994 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="89649df4e0c91b3003a7c8e4c4d98473c7fd7488089f8dd1c03e7da5a297c96f" Dec 03 12:00:10 crc kubenswrapper[5052]: I1203 12:00:10.515462 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:00:10 crc kubenswrapper[5052]: I1203 12:00:10.590095 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-grsf7\" (UniqueName: \"kubernetes.io/projected/316d245b-16aa-44a0-8aaa-4a41794e1291-kube-api-access-grsf7\") pod \"316d245b-16aa-44a0-8aaa-4a41794e1291\" (UID: \"316d245b-16aa-44a0-8aaa-4a41794e1291\") " Dec 03 12:00:10 crc kubenswrapper[5052]: I1203 12:00:10.602732 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/316d245b-16aa-44a0-8aaa-4a41794e1291-kube-api-access-grsf7" (OuterVolumeSpecName: "kube-api-access-grsf7") pod "316d245b-16aa-44a0-8aaa-4a41794e1291" (UID: "316d245b-16aa-44a0-8aaa-4a41794e1291"). InnerVolumeSpecName "kube-api-access-grsf7". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:00:10 crc kubenswrapper[5052]: I1203 12:00:10.693105 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-grsf7\" (UniqueName: \"kubernetes.io/projected/316d245b-16aa-44a0-8aaa-4a41794e1291-kube-api-access-grsf7\") on node \"crc\" DevicePath \"\"" Dec 03 12:00:10 crc kubenswrapper[5052]: I1203 12:00:10.719198 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:00:10 crc kubenswrapper[5052]: I1203 12:00:10.749605 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:00:11 crc kubenswrapper[5052]: I1203 12:00:11.451893 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:00:11 crc kubenswrapper[5052]: I1203 12:00:11.486367 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/kube-state-metrics-0"] Dec 03 12:00:11 crc kubenswrapper[5052]: I1203 12:00:11.486543 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:00:11 crc kubenswrapper[5052]: I1203 12:00:11.503981 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/kube-state-metrics-0"] Dec 03 12:00:11 crc kubenswrapper[5052]: I1203 12:00:11.513830 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/kube-state-metrics-0"] Dec 03 12:00:11 crc kubenswrapper[5052]: E1203 12:00:11.514273 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="316d245b-16aa-44a0-8aaa-4a41794e1291" containerName="kube-state-metrics" Dec 03 12:00:11 crc kubenswrapper[5052]: I1203 12:00:11.514295 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="316d245b-16aa-44a0-8aaa-4a41794e1291" containerName="kube-state-metrics" Dec 03 12:00:11 crc kubenswrapper[5052]: E1203 12:00:11.514315 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="f268de6e-4938-4114-b436-9515d64d3f60" containerName="collect-profiles" Dec 03 12:00:11 crc kubenswrapper[5052]: I1203 12:00:11.514321 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="f268de6e-4938-4114-b436-9515d64d3f60" containerName="collect-profiles" Dec 03 12:00:11 crc kubenswrapper[5052]: I1203 12:00:11.514497 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="316d245b-16aa-44a0-8aaa-4a41794e1291" containerName="kube-state-metrics" Dec 03 12:00:11 crc kubenswrapper[5052]: I1203 12:00:11.514554 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="f268de6e-4938-4114-b436-9515d64d3f60" containerName="collect-profiles" Dec 03 12:00:11 crc kubenswrapper[5052]: I1203 12:00:11.515320 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:00:11 crc kubenswrapper[5052]: I1203 12:00:11.517502 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-kube-state-metrics-svc" Dec 03 12:00:11 crc kubenswrapper[5052]: I1203 12:00:11.517561 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"kube-state-metrics-tls-config" Dec 03 12:00:11 crc kubenswrapper[5052]: I1203 12:00:11.533901 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/kube-state-metrics-0"] Dec 03 12:00:11 crc kubenswrapper[5052]: I1203 12:00:11.612598 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls-config\" (UniqueName: \"kubernetes.io/secret/d03bf1c0-7353-4836-a767-9a5b2295e8dc-kube-state-metrics-tls-config\") pod \"kube-state-metrics-0\" (UID: \"d03bf1c0-7353-4836-a767-9a5b2295e8dc\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:00:11 crc kubenswrapper[5052]: I1203 12:00:11.612677 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qw5db\" (UniqueName: \"kubernetes.io/projected/d03bf1c0-7353-4836-a767-9a5b2295e8dc-kube-api-access-qw5db\") pod \"kube-state-metrics-0\" (UID: \"d03bf1c0-7353-4836-a767-9a5b2295e8dc\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:00:11 crc kubenswrapper[5052]: I1203 12:00:11.612744 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d03bf1c0-7353-4836-a767-9a5b2295e8dc-combined-ca-bundle\") pod \"kube-state-metrics-0\" (UID: \"d03bf1c0-7353-4836-a767-9a5b2295e8dc\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:00:11 crc kubenswrapper[5052]: I1203 12:00:11.612782 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls-certs\" (UniqueName: \"kubernetes.io/secret/d03bf1c0-7353-4836-a767-9a5b2295e8dc-kube-state-metrics-tls-certs\") pod \"kube-state-metrics-0\" (UID: \"d03bf1c0-7353-4836-a767-9a5b2295e8dc\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:00:11 crc kubenswrapper[5052]: I1203 12:00:11.673431 5052 scope.go:117] "RemoveContainer" containerID="d224717c6d9ca8135fd510e03a6c40c9f2b1a56dd8b4a3b685af3e2dfa855982" Dec 03 12:00:11 crc kubenswrapper[5052]: E1203 12:00:11.673696 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 12:00:11 crc kubenswrapper[5052]: I1203 12:00:11.686814 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="316d245b-16aa-44a0-8aaa-4a41794e1291" path="/var/lib/kubelet/pods/316d245b-16aa-44a0-8aaa-4a41794e1291/volumes" Dec 03 12:00:11 crc kubenswrapper[5052]: I1203 12:00:11.715251 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls-config\" (UniqueName: \"kubernetes.io/secret/d03bf1c0-7353-4836-a767-9a5b2295e8dc-kube-state-metrics-tls-config\") pod \"kube-state-metrics-0\" (UID: \"d03bf1c0-7353-4836-a767-9a5b2295e8dc\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:00:11 crc kubenswrapper[5052]: I1203 12:00:11.716263 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-qw5db\" (UniqueName: \"kubernetes.io/projected/d03bf1c0-7353-4836-a767-9a5b2295e8dc-kube-api-access-qw5db\") pod \"kube-state-metrics-0\" (UID: \"d03bf1c0-7353-4836-a767-9a5b2295e8dc\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:00:11 crc kubenswrapper[5052]: I1203 12:00:11.716531 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d03bf1c0-7353-4836-a767-9a5b2295e8dc-combined-ca-bundle\") pod \"kube-state-metrics-0\" (UID: \"d03bf1c0-7353-4836-a767-9a5b2295e8dc\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:00:11 crc kubenswrapper[5052]: I1203 12:00:11.716655 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls-certs\" (UniqueName: \"kubernetes.io/secret/d03bf1c0-7353-4836-a767-9a5b2295e8dc-kube-state-metrics-tls-certs\") pod \"kube-state-metrics-0\" (UID: \"d03bf1c0-7353-4836-a767-9a5b2295e8dc\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:00:11 crc kubenswrapper[5052]: I1203 12:00:11.719630 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls-config\" (UniqueName: \"kubernetes.io/secret/d03bf1c0-7353-4836-a767-9a5b2295e8dc-kube-state-metrics-tls-config\") pod \"kube-state-metrics-0\" (UID: \"d03bf1c0-7353-4836-a767-9a5b2295e8dc\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:00:11 crc kubenswrapper[5052]: I1203 12:00:11.721216 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls-certs\" (UniqueName: \"kubernetes.io/secret/d03bf1c0-7353-4836-a767-9a5b2295e8dc-kube-state-metrics-tls-certs\") pod \"kube-state-metrics-0\" (UID: \"d03bf1c0-7353-4836-a767-9a5b2295e8dc\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:00:11 crc kubenswrapper[5052]: I1203 12:00:11.732357 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d03bf1c0-7353-4836-a767-9a5b2295e8dc-combined-ca-bundle\") pod \"kube-state-metrics-0\" (UID: \"d03bf1c0-7353-4836-a767-9a5b2295e8dc\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:00:11 crc kubenswrapper[5052]: I1203 12:00:11.736758 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-qw5db\" (UniqueName: \"kubernetes.io/projected/d03bf1c0-7353-4836-a767-9a5b2295e8dc-kube-api-access-qw5db\") pod \"kube-state-metrics-0\" (UID: \"d03bf1c0-7353-4836-a767-9a5b2295e8dc\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:00:11 crc kubenswrapper[5052]: I1203 12:00:11.849938 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:00:11 crc kubenswrapper[5052]: I1203 12:00:11.900858 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:00:11 crc kubenswrapper[5052]: I1203 12:00:11.901139 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="b2a6ebec-b059-4e69-a8b3-6e8834d0e7d4" containerName="ceilometer-central-agent" containerID="cri-o://86af6e57ea77f80fbae559523013c161c284dc908293fced86d0ca28d47bfa88" gracePeriod=30 Dec 03 12:00:11 crc kubenswrapper[5052]: I1203 12:00:11.901448 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="b2a6ebec-b059-4e69-a8b3-6e8834d0e7d4" containerName="ceilometer-notification-agent" containerID="cri-o://13b86d68aa550db22118a91df9a6d966c050dc8fb26964acf4c3f503015a6bfd" gracePeriod=30 Dec 03 12:00:11 crc kubenswrapper[5052]: I1203 12:00:11.901436 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="b2a6ebec-b059-4e69-a8b3-6e8834d0e7d4" containerName="proxy-httpd" containerID="cri-o://8dbc153f9b45d32a17f2e4f6457f2cd39bf8f659e8f3ca16ed4576efd8c7aca2" gracePeriod=30 Dec 03 12:00:11 crc kubenswrapper[5052]: I1203 12:00:11.901576 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="b2a6ebec-b059-4e69-a8b3-6e8834d0e7d4" containerName="sg-core" containerID="cri-o://82080fd1b638a84bacd26b6d91ce32afe1b5d62e8dd19694f376960bcc7f7a12" gracePeriod=30 Dec 03 12:00:12 crc kubenswrapper[5052]: I1203 12:00:12.340153 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/kube-state-metrics-0"] Dec 03 12:00:12 crc kubenswrapper[5052]: I1203 12:00:12.464864 5052 generic.go:334] "Generic (PLEG): container finished" podID="b2a6ebec-b059-4e69-a8b3-6e8834d0e7d4" containerID="8dbc153f9b45d32a17f2e4f6457f2cd39bf8f659e8f3ca16ed4576efd8c7aca2" exitCode=0 Dec 03 12:00:12 crc kubenswrapper[5052]: I1203 12:00:12.464903 5052 generic.go:334] "Generic (PLEG): container finished" podID="b2a6ebec-b059-4e69-a8b3-6e8834d0e7d4" containerID="82080fd1b638a84bacd26b6d91ce32afe1b5d62e8dd19694f376960bcc7f7a12" exitCode=2 Dec 03 12:00:12 crc kubenswrapper[5052]: I1203 12:00:12.464937 5052 generic.go:334] "Generic (PLEG): container finished" podID="b2a6ebec-b059-4e69-a8b3-6e8834d0e7d4" containerID="86af6e57ea77f80fbae559523013c161c284dc908293fced86d0ca28d47bfa88" exitCode=0 Dec 03 12:00:12 crc kubenswrapper[5052]: I1203 12:00:12.464929 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"b2a6ebec-b059-4e69-a8b3-6e8834d0e7d4","Type":"ContainerDied","Data":"8dbc153f9b45d32a17f2e4f6457f2cd39bf8f659e8f3ca16ed4576efd8c7aca2"} Dec 03 12:00:12 crc kubenswrapper[5052]: I1203 12:00:12.464979 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"b2a6ebec-b059-4e69-a8b3-6e8834d0e7d4","Type":"ContainerDied","Data":"82080fd1b638a84bacd26b6d91ce32afe1b5d62e8dd19694f376960bcc7f7a12"} Dec 03 12:00:12 crc kubenswrapper[5052]: I1203 12:00:12.464992 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"b2a6ebec-b059-4e69-a8b3-6e8834d0e7d4","Type":"ContainerDied","Data":"86af6e57ea77f80fbae559523013c161c284dc908293fced86d0ca28d47bfa88"} Dec 03 12:00:12 crc kubenswrapper[5052]: I1203 12:00:12.466991 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/kube-state-metrics-0" event={"ID":"d03bf1c0-7353-4836-a767-9a5b2295e8dc","Type":"ContainerStarted","Data":"5200bd8a659d963e7653504e706b0695a96a1bbd01693588e9d6f0596b8e6de1"} Dec 03 12:00:13 crc kubenswrapper[5052]: I1203 12:00:13.070945 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:00:13 crc kubenswrapper[5052]: I1203 12:00:13.071274 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:00:13 crc kubenswrapper[5052]: I1203 12:00:13.478984 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/kube-state-metrics-0" event={"ID":"d03bf1c0-7353-4836-a767-9a5b2295e8dc","Type":"ContainerStarted","Data":"3a4d32fe7507dccb3b5a703d41f75ff7c20cb477aef43d8d0062e58cf58c09bf"} Dec 03 12:00:13 crc kubenswrapper[5052]: I1203 12:00:13.479246 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:00:13 crc kubenswrapper[5052]: I1203 12:00:13.495216 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/kube-state-metrics-0" podStartSLOduration=2.135400608 podStartE2EDuration="2.495186814s" podCreationTimestamp="2025-12-03 12:00:11 +0000 UTC" firstStartedPulling="2025-12-03 12:00:12.344750094 +0000 UTC m=+2402.932991858" lastFinishedPulling="2025-12-03 12:00:12.7045363 +0000 UTC m=+2403.292778064" observedRunningTime="2025-12-03 12:00:13.49117838 +0000 UTC m=+2404.079420144" watchObservedRunningTime="2025-12-03 12:00:13.495186814 +0000 UTC m=+2404.083428618" Dec 03 12:00:14 crc kubenswrapper[5052]: I1203 12:00:14.153730 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-api-0" podUID="eebaba62-8aa5-447a-bcc2-590bb40de88f" containerName="nova-api-log" probeResult="failure" output="Get \"http://10.217.1.247:8774/\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Dec 03 12:00:14 crc kubenswrapper[5052]: I1203 12:00:14.153884 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-api-0" podUID="eebaba62-8aa5-447a-bcc2-590bb40de88f" containerName="nova-api-api" probeResult="failure" output="Get \"http://10.217.1.247:8774/\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Dec 03 12:00:16 crc kubenswrapper[5052]: I1203 12:00:16.327910 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:00:16 crc kubenswrapper[5052]: I1203 12:00:16.395367 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/b2a6ebec-b059-4e69-a8b3-6e8834d0e7d4-log-httpd\") pod \"b2a6ebec-b059-4e69-a8b3-6e8834d0e7d4\" (UID: \"b2a6ebec-b059-4e69-a8b3-6e8834d0e7d4\") " Dec 03 12:00:16 crc kubenswrapper[5052]: I1203 12:00:16.395450 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bkqhk\" (UniqueName: \"kubernetes.io/projected/b2a6ebec-b059-4e69-a8b3-6e8834d0e7d4-kube-api-access-bkqhk\") pod \"b2a6ebec-b059-4e69-a8b3-6e8834d0e7d4\" (UID: \"b2a6ebec-b059-4e69-a8b3-6e8834d0e7d4\") " Dec 03 12:00:16 crc kubenswrapper[5052]: I1203 12:00:16.395523 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/b2a6ebec-b059-4e69-a8b3-6e8834d0e7d4-run-httpd\") pod \"b2a6ebec-b059-4e69-a8b3-6e8834d0e7d4\" (UID: \"b2a6ebec-b059-4e69-a8b3-6e8834d0e7d4\") " Dec 03 12:00:16 crc kubenswrapper[5052]: I1203 12:00:16.395596 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/b2a6ebec-b059-4e69-a8b3-6e8834d0e7d4-sg-core-conf-yaml\") pod \"b2a6ebec-b059-4e69-a8b3-6e8834d0e7d4\" (UID: \"b2a6ebec-b059-4e69-a8b3-6e8834d0e7d4\") " Dec 03 12:00:16 crc kubenswrapper[5052]: I1203 12:00:16.395618 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b2a6ebec-b059-4e69-a8b3-6e8834d0e7d4-combined-ca-bundle\") pod \"b2a6ebec-b059-4e69-a8b3-6e8834d0e7d4\" (UID: \"b2a6ebec-b059-4e69-a8b3-6e8834d0e7d4\") " Dec 03 12:00:16 crc kubenswrapper[5052]: I1203 12:00:16.395737 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/b2a6ebec-b059-4e69-a8b3-6e8834d0e7d4-scripts\") pod \"b2a6ebec-b059-4e69-a8b3-6e8834d0e7d4\" (UID: \"b2a6ebec-b059-4e69-a8b3-6e8834d0e7d4\") " Dec 03 12:00:16 crc kubenswrapper[5052]: I1203 12:00:16.396484 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/b2a6ebec-b059-4e69-a8b3-6e8834d0e7d4-config-data\") pod \"b2a6ebec-b059-4e69-a8b3-6e8834d0e7d4\" (UID: \"b2a6ebec-b059-4e69-a8b3-6e8834d0e7d4\") " Dec 03 12:00:16 crc kubenswrapper[5052]: I1203 12:00:16.396028 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b2a6ebec-b059-4e69-a8b3-6e8834d0e7d4-run-httpd" (OuterVolumeSpecName: "run-httpd") pod "b2a6ebec-b059-4e69-a8b3-6e8834d0e7d4" (UID: "b2a6ebec-b059-4e69-a8b3-6e8834d0e7d4"). InnerVolumeSpecName "run-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:00:16 crc kubenswrapper[5052]: I1203 12:00:16.396648 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b2a6ebec-b059-4e69-a8b3-6e8834d0e7d4-log-httpd" (OuterVolumeSpecName: "log-httpd") pod "b2a6ebec-b059-4e69-a8b3-6e8834d0e7d4" (UID: "b2a6ebec-b059-4e69-a8b3-6e8834d0e7d4"). InnerVolumeSpecName "log-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:00:16 crc kubenswrapper[5052]: I1203 12:00:16.396986 5052 reconciler_common.go:293] "Volume detached for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/b2a6ebec-b059-4e69-a8b3-6e8834d0e7d4-log-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:00:16 crc kubenswrapper[5052]: I1203 12:00:16.397004 5052 reconciler_common.go:293] "Volume detached for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/b2a6ebec-b059-4e69-a8b3-6e8834d0e7d4-run-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:00:16 crc kubenswrapper[5052]: I1203 12:00:16.403089 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b2a6ebec-b059-4e69-a8b3-6e8834d0e7d4-kube-api-access-bkqhk" (OuterVolumeSpecName: "kube-api-access-bkqhk") pod "b2a6ebec-b059-4e69-a8b3-6e8834d0e7d4" (UID: "b2a6ebec-b059-4e69-a8b3-6e8834d0e7d4"). InnerVolumeSpecName "kube-api-access-bkqhk". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:00:16 crc kubenswrapper[5052]: I1203 12:00:16.418944 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b2a6ebec-b059-4e69-a8b3-6e8834d0e7d4-scripts" (OuterVolumeSpecName: "scripts") pod "b2a6ebec-b059-4e69-a8b3-6e8834d0e7d4" (UID: "b2a6ebec-b059-4e69-a8b3-6e8834d0e7d4"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:00:16 crc kubenswrapper[5052]: I1203 12:00:16.426599 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b2a6ebec-b059-4e69-a8b3-6e8834d0e7d4-sg-core-conf-yaml" (OuterVolumeSpecName: "sg-core-conf-yaml") pod "b2a6ebec-b059-4e69-a8b3-6e8834d0e7d4" (UID: "b2a6ebec-b059-4e69-a8b3-6e8834d0e7d4"). InnerVolumeSpecName "sg-core-conf-yaml". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:00:16 crc kubenswrapper[5052]: I1203 12:00:16.483459 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b2a6ebec-b059-4e69-a8b3-6e8834d0e7d4-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "b2a6ebec-b059-4e69-a8b3-6e8834d0e7d4" (UID: "b2a6ebec-b059-4e69-a8b3-6e8834d0e7d4"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:00:16 crc kubenswrapper[5052]: I1203 12:00:16.494292 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b2a6ebec-b059-4e69-a8b3-6e8834d0e7d4-config-data" (OuterVolumeSpecName: "config-data") pod "b2a6ebec-b059-4e69-a8b3-6e8834d0e7d4" (UID: "b2a6ebec-b059-4e69-a8b3-6e8834d0e7d4"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:00:16 crc kubenswrapper[5052]: I1203 12:00:16.498082 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-bkqhk\" (UniqueName: \"kubernetes.io/projected/b2a6ebec-b059-4e69-a8b3-6e8834d0e7d4-kube-api-access-bkqhk\") on node \"crc\" DevicePath \"\"" Dec 03 12:00:16 crc kubenswrapper[5052]: I1203 12:00:16.498106 5052 reconciler_common.go:293] "Volume detached for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/b2a6ebec-b059-4e69-a8b3-6e8834d0e7d4-sg-core-conf-yaml\") on node \"crc\" DevicePath \"\"" Dec 03 12:00:16 crc kubenswrapper[5052]: I1203 12:00:16.498114 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b2a6ebec-b059-4e69-a8b3-6e8834d0e7d4-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:00:16 crc kubenswrapper[5052]: I1203 12:00:16.498125 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/b2a6ebec-b059-4e69-a8b3-6e8834d0e7d4-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:00:16 crc kubenswrapper[5052]: I1203 12:00:16.498134 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/b2a6ebec-b059-4e69-a8b3-6e8834d0e7d4-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:00:16 crc kubenswrapper[5052]: I1203 12:00:16.505450 5052 generic.go:334] "Generic (PLEG): container finished" podID="b2a6ebec-b059-4e69-a8b3-6e8834d0e7d4" containerID="13b86d68aa550db22118a91df9a6d966c050dc8fb26964acf4c3f503015a6bfd" exitCode=0 Dec 03 12:00:16 crc kubenswrapper[5052]: I1203 12:00:16.505494 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"b2a6ebec-b059-4e69-a8b3-6e8834d0e7d4","Type":"ContainerDied","Data":"13b86d68aa550db22118a91df9a6d966c050dc8fb26964acf4c3f503015a6bfd"} Dec 03 12:00:16 crc kubenswrapper[5052]: I1203 12:00:16.505546 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"b2a6ebec-b059-4e69-a8b3-6e8834d0e7d4","Type":"ContainerDied","Data":"f060bcdf30dd614f6d54a5afeed8290132a9c6f0617546e94ac296834031acdc"} Dec 03 12:00:16 crc kubenswrapper[5052]: I1203 12:00:16.505571 5052 scope.go:117] "RemoveContainer" containerID="8dbc153f9b45d32a17f2e4f6457f2cd39bf8f659e8f3ca16ed4576efd8c7aca2" Dec 03 12:00:16 crc kubenswrapper[5052]: I1203 12:00:16.505756 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:00:16 crc kubenswrapper[5052]: I1203 12:00:16.567177 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:00:16 crc kubenswrapper[5052]: I1203 12:00:16.570711 5052 scope.go:117] "RemoveContainer" containerID="82080fd1b638a84bacd26b6d91ce32afe1b5d62e8dd19694f376960bcc7f7a12" Dec 03 12:00:16 crc kubenswrapper[5052]: I1203 12:00:16.576054 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:00:16 crc kubenswrapper[5052]: I1203 12:00:16.602199 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:00:16 crc kubenswrapper[5052]: E1203 12:00:16.602652 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="b2a6ebec-b059-4e69-a8b3-6e8834d0e7d4" containerName="ceilometer-central-agent" Dec 03 12:00:16 crc kubenswrapper[5052]: I1203 12:00:16.602669 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="b2a6ebec-b059-4e69-a8b3-6e8834d0e7d4" containerName="ceilometer-central-agent" Dec 03 12:00:16 crc kubenswrapper[5052]: E1203 12:00:16.602700 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="b2a6ebec-b059-4e69-a8b3-6e8834d0e7d4" containerName="proxy-httpd" Dec 03 12:00:16 crc kubenswrapper[5052]: I1203 12:00:16.602707 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="b2a6ebec-b059-4e69-a8b3-6e8834d0e7d4" containerName="proxy-httpd" Dec 03 12:00:16 crc kubenswrapper[5052]: E1203 12:00:16.602723 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="b2a6ebec-b059-4e69-a8b3-6e8834d0e7d4" containerName="ceilometer-notification-agent" Dec 03 12:00:16 crc kubenswrapper[5052]: I1203 12:00:16.602731 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="b2a6ebec-b059-4e69-a8b3-6e8834d0e7d4" containerName="ceilometer-notification-agent" Dec 03 12:00:16 crc kubenswrapper[5052]: E1203 12:00:16.602751 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="b2a6ebec-b059-4e69-a8b3-6e8834d0e7d4" containerName="sg-core" Dec 03 12:00:16 crc kubenswrapper[5052]: I1203 12:00:16.602757 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="b2a6ebec-b059-4e69-a8b3-6e8834d0e7d4" containerName="sg-core" Dec 03 12:00:16 crc kubenswrapper[5052]: I1203 12:00:16.603030 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="b2a6ebec-b059-4e69-a8b3-6e8834d0e7d4" containerName="sg-core" Dec 03 12:00:16 crc kubenswrapper[5052]: I1203 12:00:16.603053 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="b2a6ebec-b059-4e69-a8b3-6e8834d0e7d4" containerName="ceilometer-central-agent" Dec 03 12:00:16 crc kubenswrapper[5052]: I1203 12:00:16.603068 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="b2a6ebec-b059-4e69-a8b3-6e8834d0e7d4" containerName="ceilometer-notification-agent" Dec 03 12:00:16 crc kubenswrapper[5052]: I1203 12:00:16.603095 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="b2a6ebec-b059-4e69-a8b3-6e8834d0e7d4" containerName="proxy-httpd" Dec 03 12:00:16 crc kubenswrapper[5052]: I1203 12:00:16.605141 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:00:16 crc kubenswrapper[5052]: I1203 12:00:16.605997 5052 scope.go:117] "RemoveContainer" containerID="13b86d68aa550db22118a91df9a6d966c050dc8fb26964acf4c3f503015a6bfd" Dec 03 12:00:16 crc kubenswrapper[5052]: I1203 12:00:16.607495 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-scripts" Dec 03 12:00:16 crc kubenswrapper[5052]: I1203 12:00:16.607701 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-config-data" Dec 03 12:00:16 crc kubenswrapper[5052]: I1203 12:00:16.607709 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-ceilometer-internal-svc" Dec 03 12:00:16 crc kubenswrapper[5052]: I1203 12:00:16.627255 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:00:16 crc kubenswrapper[5052]: I1203 12:00:16.637568 5052 scope.go:117] "RemoveContainer" containerID="86af6e57ea77f80fbae559523013c161c284dc908293fced86d0ca28d47bfa88" Dec 03 12:00:16 crc kubenswrapper[5052]: I1203 12:00:16.658431 5052 scope.go:117] "RemoveContainer" containerID="8dbc153f9b45d32a17f2e4f6457f2cd39bf8f659e8f3ca16ed4576efd8c7aca2" Dec 03 12:00:16 crc kubenswrapper[5052]: E1203 12:00:16.658907 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8dbc153f9b45d32a17f2e4f6457f2cd39bf8f659e8f3ca16ed4576efd8c7aca2\": container with ID starting with 8dbc153f9b45d32a17f2e4f6457f2cd39bf8f659e8f3ca16ed4576efd8c7aca2 not found: ID does not exist" containerID="8dbc153f9b45d32a17f2e4f6457f2cd39bf8f659e8f3ca16ed4576efd8c7aca2" Dec 03 12:00:16 crc kubenswrapper[5052]: I1203 12:00:16.658941 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8dbc153f9b45d32a17f2e4f6457f2cd39bf8f659e8f3ca16ed4576efd8c7aca2"} err="failed to get container status \"8dbc153f9b45d32a17f2e4f6457f2cd39bf8f659e8f3ca16ed4576efd8c7aca2\": rpc error: code = NotFound desc = could not find container \"8dbc153f9b45d32a17f2e4f6457f2cd39bf8f659e8f3ca16ed4576efd8c7aca2\": container with ID starting with 8dbc153f9b45d32a17f2e4f6457f2cd39bf8f659e8f3ca16ed4576efd8c7aca2 not found: ID does not exist" Dec 03 12:00:16 crc kubenswrapper[5052]: I1203 12:00:16.658961 5052 scope.go:117] "RemoveContainer" containerID="82080fd1b638a84bacd26b6d91ce32afe1b5d62e8dd19694f376960bcc7f7a12" Dec 03 12:00:16 crc kubenswrapper[5052]: E1203 12:00:16.659378 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"82080fd1b638a84bacd26b6d91ce32afe1b5d62e8dd19694f376960bcc7f7a12\": container with ID starting with 82080fd1b638a84bacd26b6d91ce32afe1b5d62e8dd19694f376960bcc7f7a12 not found: ID does not exist" containerID="82080fd1b638a84bacd26b6d91ce32afe1b5d62e8dd19694f376960bcc7f7a12" Dec 03 12:00:16 crc kubenswrapper[5052]: I1203 12:00:16.659402 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"82080fd1b638a84bacd26b6d91ce32afe1b5d62e8dd19694f376960bcc7f7a12"} err="failed to get container status \"82080fd1b638a84bacd26b6d91ce32afe1b5d62e8dd19694f376960bcc7f7a12\": rpc error: code = NotFound desc = could not find container \"82080fd1b638a84bacd26b6d91ce32afe1b5d62e8dd19694f376960bcc7f7a12\": container with ID starting with 82080fd1b638a84bacd26b6d91ce32afe1b5d62e8dd19694f376960bcc7f7a12 not found: ID does not exist" Dec 03 12:00:16 crc kubenswrapper[5052]: I1203 12:00:16.659414 5052 scope.go:117] "RemoveContainer" containerID="13b86d68aa550db22118a91df9a6d966c050dc8fb26964acf4c3f503015a6bfd" Dec 03 12:00:16 crc kubenswrapper[5052]: E1203 12:00:16.659816 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"13b86d68aa550db22118a91df9a6d966c050dc8fb26964acf4c3f503015a6bfd\": container with ID starting with 13b86d68aa550db22118a91df9a6d966c050dc8fb26964acf4c3f503015a6bfd not found: ID does not exist" containerID="13b86d68aa550db22118a91df9a6d966c050dc8fb26964acf4c3f503015a6bfd" Dec 03 12:00:16 crc kubenswrapper[5052]: I1203 12:00:16.659837 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"13b86d68aa550db22118a91df9a6d966c050dc8fb26964acf4c3f503015a6bfd"} err="failed to get container status \"13b86d68aa550db22118a91df9a6d966c050dc8fb26964acf4c3f503015a6bfd\": rpc error: code = NotFound desc = could not find container \"13b86d68aa550db22118a91df9a6d966c050dc8fb26964acf4c3f503015a6bfd\": container with ID starting with 13b86d68aa550db22118a91df9a6d966c050dc8fb26964acf4c3f503015a6bfd not found: ID does not exist" Dec 03 12:00:16 crc kubenswrapper[5052]: I1203 12:00:16.659851 5052 scope.go:117] "RemoveContainer" containerID="86af6e57ea77f80fbae559523013c161c284dc908293fced86d0ca28d47bfa88" Dec 03 12:00:16 crc kubenswrapper[5052]: E1203 12:00:16.660132 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"86af6e57ea77f80fbae559523013c161c284dc908293fced86d0ca28d47bfa88\": container with ID starting with 86af6e57ea77f80fbae559523013c161c284dc908293fced86d0ca28d47bfa88 not found: ID does not exist" containerID="86af6e57ea77f80fbae559523013c161c284dc908293fced86d0ca28d47bfa88" Dec 03 12:00:16 crc kubenswrapper[5052]: I1203 12:00:16.660155 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"86af6e57ea77f80fbae559523013c161c284dc908293fced86d0ca28d47bfa88"} err="failed to get container status \"86af6e57ea77f80fbae559523013c161c284dc908293fced86d0ca28d47bfa88\": rpc error: code = NotFound desc = could not find container \"86af6e57ea77f80fbae559523013c161c284dc908293fced86d0ca28d47bfa88\": container with ID starting with 86af6e57ea77f80fbae559523013c161c284dc908293fced86d0ca28d47bfa88 not found: ID does not exist" Dec 03 12:00:16 crc kubenswrapper[5052]: I1203 12:00:16.701462 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/900bf7c0-8907-4d20-9d2c-9c6716cf7d4c-config-data\") pod \"ceilometer-0\" (UID: \"900bf7c0-8907-4d20-9d2c-9c6716cf7d4c\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:00:16 crc kubenswrapper[5052]: I1203 12:00:16.701598 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/900bf7c0-8907-4d20-9d2c-9c6716cf7d4c-log-httpd\") pod \"ceilometer-0\" (UID: \"900bf7c0-8907-4d20-9d2c-9c6716cf7d4c\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:00:16 crc kubenswrapper[5052]: I1203 12:00:16.701629 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/900bf7c0-8907-4d20-9d2c-9c6716cf7d4c-ceilometer-tls-certs\") pod \"ceilometer-0\" (UID: \"900bf7c0-8907-4d20-9d2c-9c6716cf7d4c\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:00:16 crc kubenswrapper[5052]: I1203 12:00:16.701733 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2gldt\" (UniqueName: \"kubernetes.io/projected/900bf7c0-8907-4d20-9d2c-9c6716cf7d4c-kube-api-access-2gldt\") pod \"ceilometer-0\" (UID: \"900bf7c0-8907-4d20-9d2c-9c6716cf7d4c\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:00:16 crc kubenswrapper[5052]: I1203 12:00:16.701844 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/900bf7c0-8907-4d20-9d2c-9c6716cf7d4c-run-httpd\") pod \"ceilometer-0\" (UID: \"900bf7c0-8907-4d20-9d2c-9c6716cf7d4c\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:00:16 crc kubenswrapper[5052]: I1203 12:00:16.701922 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/900bf7c0-8907-4d20-9d2c-9c6716cf7d4c-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"900bf7c0-8907-4d20-9d2c-9c6716cf7d4c\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:00:16 crc kubenswrapper[5052]: I1203 12:00:16.701978 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/900bf7c0-8907-4d20-9d2c-9c6716cf7d4c-scripts\") pod \"ceilometer-0\" (UID: \"900bf7c0-8907-4d20-9d2c-9c6716cf7d4c\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:00:16 crc kubenswrapper[5052]: I1203 12:00:16.702405 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/900bf7c0-8907-4d20-9d2c-9c6716cf7d4c-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"900bf7c0-8907-4d20-9d2c-9c6716cf7d4c\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:00:16 crc kubenswrapper[5052]: I1203 12:00:16.803908 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/900bf7c0-8907-4d20-9d2c-9c6716cf7d4c-config-data\") pod \"ceilometer-0\" (UID: \"900bf7c0-8907-4d20-9d2c-9c6716cf7d4c\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:00:16 crc kubenswrapper[5052]: I1203 12:00:16.804010 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/900bf7c0-8907-4d20-9d2c-9c6716cf7d4c-log-httpd\") pod \"ceilometer-0\" (UID: \"900bf7c0-8907-4d20-9d2c-9c6716cf7d4c\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:00:16 crc kubenswrapper[5052]: I1203 12:00:16.804048 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/900bf7c0-8907-4d20-9d2c-9c6716cf7d4c-ceilometer-tls-certs\") pod \"ceilometer-0\" (UID: \"900bf7c0-8907-4d20-9d2c-9c6716cf7d4c\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:00:16 crc kubenswrapper[5052]: I1203 12:00:16.804089 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-2gldt\" (UniqueName: \"kubernetes.io/projected/900bf7c0-8907-4d20-9d2c-9c6716cf7d4c-kube-api-access-2gldt\") pod \"ceilometer-0\" (UID: \"900bf7c0-8907-4d20-9d2c-9c6716cf7d4c\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:00:16 crc kubenswrapper[5052]: I1203 12:00:16.804116 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/900bf7c0-8907-4d20-9d2c-9c6716cf7d4c-run-httpd\") pod \"ceilometer-0\" (UID: \"900bf7c0-8907-4d20-9d2c-9c6716cf7d4c\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:00:16 crc kubenswrapper[5052]: I1203 12:00:16.804146 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/900bf7c0-8907-4d20-9d2c-9c6716cf7d4c-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"900bf7c0-8907-4d20-9d2c-9c6716cf7d4c\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:00:16 crc kubenswrapper[5052]: I1203 12:00:16.804167 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/900bf7c0-8907-4d20-9d2c-9c6716cf7d4c-scripts\") pod \"ceilometer-0\" (UID: \"900bf7c0-8907-4d20-9d2c-9c6716cf7d4c\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:00:16 crc kubenswrapper[5052]: I1203 12:00:16.804180 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/900bf7c0-8907-4d20-9d2c-9c6716cf7d4c-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"900bf7c0-8907-4d20-9d2c-9c6716cf7d4c\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:00:16 crc kubenswrapper[5052]: I1203 12:00:16.804878 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/900bf7c0-8907-4d20-9d2c-9c6716cf7d4c-log-httpd\") pod \"ceilometer-0\" (UID: \"900bf7c0-8907-4d20-9d2c-9c6716cf7d4c\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:00:16 crc kubenswrapper[5052]: I1203 12:00:16.805108 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/900bf7c0-8907-4d20-9d2c-9c6716cf7d4c-run-httpd\") pod \"ceilometer-0\" (UID: \"900bf7c0-8907-4d20-9d2c-9c6716cf7d4c\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:00:16 crc kubenswrapper[5052]: I1203 12:00:16.809347 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/900bf7c0-8907-4d20-9d2c-9c6716cf7d4c-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"900bf7c0-8907-4d20-9d2c-9c6716cf7d4c\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:00:16 crc kubenswrapper[5052]: I1203 12:00:16.809367 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/900bf7c0-8907-4d20-9d2c-9c6716cf7d4c-ceilometer-tls-certs\") pod \"ceilometer-0\" (UID: \"900bf7c0-8907-4d20-9d2c-9c6716cf7d4c\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:00:16 crc kubenswrapper[5052]: I1203 12:00:16.809554 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/900bf7c0-8907-4d20-9d2c-9c6716cf7d4c-scripts\") pod \"ceilometer-0\" (UID: \"900bf7c0-8907-4d20-9d2c-9c6716cf7d4c\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:00:16 crc kubenswrapper[5052]: I1203 12:00:16.809573 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/900bf7c0-8907-4d20-9d2c-9c6716cf7d4c-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"900bf7c0-8907-4d20-9d2c-9c6716cf7d4c\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:00:16 crc kubenswrapper[5052]: I1203 12:00:16.810260 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/900bf7c0-8907-4d20-9d2c-9c6716cf7d4c-config-data\") pod \"ceilometer-0\" (UID: \"900bf7c0-8907-4d20-9d2c-9c6716cf7d4c\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:00:16 crc kubenswrapper[5052]: I1203 12:00:16.820609 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-2gldt\" (UniqueName: \"kubernetes.io/projected/900bf7c0-8907-4d20-9d2c-9c6716cf7d4c-kube-api-access-2gldt\") pod \"ceilometer-0\" (UID: \"900bf7c0-8907-4d20-9d2c-9c6716cf7d4c\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:00:16 crc kubenswrapper[5052]: I1203 12:00:16.922493 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:00:17 crc kubenswrapper[5052]: I1203 12:00:17.415464 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:00:17 crc kubenswrapper[5052]: W1203 12:00:17.417729 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod900bf7c0_8907_4d20_9d2c_9c6716cf7d4c.slice/crio-cce5e575166559fcedc44682c8b032408b60771e5a3483bc8da1f364906ffe48 WatchSource:0}: Error finding container cce5e575166559fcedc44682c8b032408b60771e5a3483bc8da1f364906ffe48: Status 404 returned error can't find the container with id cce5e575166559fcedc44682c8b032408b60771e5a3483bc8da1f364906ffe48 Dec 03 12:00:17 crc kubenswrapper[5052]: I1203 12:00:17.518982 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"900bf7c0-8907-4d20-9d2c-9c6716cf7d4c","Type":"ContainerStarted","Data":"cce5e575166559fcedc44682c8b032408b60771e5a3483bc8da1f364906ffe48"} Dec 03 12:00:17 crc kubenswrapper[5052]: I1203 12:00:17.686274 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b2a6ebec-b059-4e69-a8b3-6e8834d0e7d4" path="/var/lib/kubelet/pods/b2a6ebec-b059-4e69-a8b3-6e8834d0e7d4/volumes" Dec 03 12:00:18 crc kubenswrapper[5052]: I1203 12:00:18.530780 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"900bf7c0-8907-4d20-9d2c-9c6716cf7d4c","Type":"ContainerStarted","Data":"5a46c706b716d58c0e633d615cd546a53bde37fdd9dc2b1a4dcca2b69b88cc8b"} Dec 03 12:00:18 crc kubenswrapper[5052]: I1203 12:00:18.662735 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:00:18 crc kubenswrapper[5052]: I1203 12:00:18.662799 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:00:18 crc kubenswrapper[5052]: I1203 12:00:18.672447 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:00:18 crc kubenswrapper[5052]: I1203 12:00:18.673644 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:00:19 crc kubenswrapper[5052]: I1203 12:00:19.541428 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"900bf7c0-8907-4d20-9d2c-9c6716cf7d4c","Type":"ContainerStarted","Data":"052af835c664bb69b95dc6b37b6f6a9a9acfd9bbf7fd2efa3efaf8768515cba4"} Dec 03 12:00:19 crc kubenswrapper[5052]: I1203 12:00:19.542998 5052 generic.go:334] "Generic (PLEG): container finished" podID="1f41370a-19d0-4c2d-b942-26562f752813" containerID="2f0b1bc5e2c1b087f5be9d90327ea9a97c57020ec2b1922d1d41714021fc9db7" exitCode=137 Dec 03 12:00:19 crc kubenswrapper[5052]: I1203 12:00:19.543097 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" event={"ID":"1f41370a-19d0-4c2d-b942-26562f752813","Type":"ContainerDied","Data":"2f0b1bc5e2c1b087f5be9d90327ea9a97c57020ec2b1922d1d41714021fc9db7"} Dec 03 12:00:19 crc kubenswrapper[5052]: I1203 12:00:19.543202 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" event={"ID":"1f41370a-19d0-4c2d-b942-26562f752813","Type":"ContainerDied","Data":"8f837453929db711888ca07110672b6c1204a13b9fb39db4c3b0031692905192"} Dec 03 12:00:19 crc kubenswrapper[5052]: I1203 12:00:19.543301 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8f837453929db711888ca07110672b6c1204a13b9fb39db4c3b0031692905192" Dec 03 12:00:19 crc kubenswrapper[5052]: I1203 12:00:19.554299 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:00:19 crc kubenswrapper[5052]: I1203 12:00:19.573708 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-brl7s\" (UniqueName: \"kubernetes.io/projected/1f41370a-19d0-4c2d-b942-26562f752813-kube-api-access-brl7s\") pod \"1f41370a-19d0-4c2d-b942-26562f752813\" (UID: \"1f41370a-19d0-4c2d-b942-26562f752813\") " Dec 03 12:00:19 crc kubenswrapper[5052]: I1203 12:00:19.573820 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1f41370a-19d0-4c2d-b942-26562f752813-combined-ca-bundle\") pod \"1f41370a-19d0-4c2d-b942-26562f752813\" (UID: \"1f41370a-19d0-4c2d-b942-26562f752813\") " Dec 03 12:00:19 crc kubenswrapper[5052]: I1203 12:00:19.573868 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/1f41370a-19d0-4c2d-b942-26562f752813-config-data\") pod \"1f41370a-19d0-4c2d-b942-26562f752813\" (UID: \"1f41370a-19d0-4c2d-b942-26562f752813\") " Dec 03 12:00:19 crc kubenswrapper[5052]: I1203 12:00:19.579738 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1f41370a-19d0-4c2d-b942-26562f752813-kube-api-access-brl7s" (OuterVolumeSpecName: "kube-api-access-brl7s") pod "1f41370a-19d0-4c2d-b942-26562f752813" (UID: "1f41370a-19d0-4c2d-b942-26562f752813"). InnerVolumeSpecName "kube-api-access-brl7s". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:00:19 crc kubenswrapper[5052]: I1203 12:00:19.609747 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1f41370a-19d0-4c2d-b942-26562f752813-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "1f41370a-19d0-4c2d-b942-26562f752813" (UID: "1f41370a-19d0-4c2d-b942-26562f752813"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:00:19 crc kubenswrapper[5052]: I1203 12:00:19.617499 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1f41370a-19d0-4c2d-b942-26562f752813-config-data" (OuterVolumeSpecName: "config-data") pod "1f41370a-19d0-4c2d-b942-26562f752813" (UID: "1f41370a-19d0-4c2d-b942-26562f752813"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:00:19 crc kubenswrapper[5052]: I1203 12:00:19.676154 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-brl7s\" (UniqueName: \"kubernetes.io/projected/1f41370a-19d0-4c2d-b942-26562f752813-kube-api-access-brl7s\") on node \"crc\" DevicePath \"\"" Dec 03 12:00:19 crc kubenswrapper[5052]: I1203 12:00:19.676186 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1f41370a-19d0-4c2d-b942-26562f752813-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:00:19 crc kubenswrapper[5052]: I1203 12:00:19.676199 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/1f41370a-19d0-4c2d-b942-26562f752813-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:00:20 crc kubenswrapper[5052]: I1203 12:00:20.564563 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:00:20 crc kubenswrapper[5052]: I1203 12:00:20.564820 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"900bf7c0-8907-4d20-9d2c-9c6716cf7d4c","Type":"ContainerStarted","Data":"e7cc78dd31d4e2ecd3f97d9d229e97e11d244874476d9cf5fa8c317f1ce47ae9"} Dec 03 12:00:20 crc kubenswrapper[5052]: I1203 12:00:20.604963 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 12:00:20 crc kubenswrapper[5052]: I1203 12:00:20.621439 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 12:00:20 crc kubenswrapper[5052]: I1203 12:00:20.634604 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 12:00:20 crc kubenswrapper[5052]: E1203 12:00:20.638584 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="1f41370a-19d0-4c2d-b942-26562f752813" containerName="nova-cell1-novncproxy-novncproxy" Dec 03 12:00:20 crc kubenswrapper[5052]: I1203 12:00:20.638626 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f41370a-19d0-4c2d-b942-26562f752813" containerName="nova-cell1-novncproxy-novncproxy" Dec 03 12:00:20 crc kubenswrapper[5052]: I1203 12:00:20.638983 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="1f41370a-19d0-4c2d-b942-26562f752813" containerName="nova-cell1-novncproxy-novncproxy" Dec 03 12:00:20 crc kubenswrapper[5052]: I1203 12:00:20.640307 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:00:20 crc kubenswrapper[5052]: I1203 12:00:20.644055 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell1-novncproxy-config-data" Dec 03 12:00:20 crc kubenswrapper[5052]: I1203 12:00:20.644573 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-nova-novncproxy-cell1-public-svc" Dec 03 12:00:20 crc kubenswrapper[5052]: I1203 12:00:20.644891 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-nova-novncproxy-cell1-vencrypt" Dec 03 12:00:20 crc kubenswrapper[5052]: I1203 12:00:20.648363 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 12:00:20 crc kubenswrapper[5052]: I1203 12:00:20.710709 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/914c0f8f-e5d1-4c63-b985-0bcd07f75374-combined-ca-bundle\") pod \"nova-cell1-novncproxy-0\" (UID: \"914c0f8f-e5d1-4c63-b985-0bcd07f75374\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:00:20 crc kubenswrapper[5052]: I1203 12:00:20.710836 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7t9cz\" (UniqueName: \"kubernetes.io/projected/914c0f8f-e5d1-4c63-b985-0bcd07f75374-kube-api-access-7t9cz\") pod \"nova-cell1-novncproxy-0\" (UID: \"914c0f8f-e5d1-4c63-b985-0bcd07f75374\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:00:20 crc kubenswrapper[5052]: I1203 12:00:20.710906 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/914c0f8f-e5d1-4c63-b985-0bcd07f75374-config-data\") pod \"nova-cell1-novncproxy-0\" (UID: \"914c0f8f-e5d1-4c63-b985-0bcd07f75374\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:00:20 crc kubenswrapper[5052]: I1203 12:00:20.710975 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nova-novncproxy-tls-certs\" (UniqueName: \"kubernetes.io/secret/914c0f8f-e5d1-4c63-b985-0bcd07f75374-nova-novncproxy-tls-certs\") pod \"nova-cell1-novncproxy-0\" (UID: \"914c0f8f-e5d1-4c63-b985-0bcd07f75374\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:00:20 crc kubenswrapper[5052]: I1203 12:00:20.711158 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"vencrypt-tls-certs\" (UniqueName: \"kubernetes.io/secret/914c0f8f-e5d1-4c63-b985-0bcd07f75374-vencrypt-tls-certs\") pod \"nova-cell1-novncproxy-0\" (UID: \"914c0f8f-e5d1-4c63-b985-0bcd07f75374\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:00:20 crc kubenswrapper[5052]: I1203 12:00:20.812441 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/914c0f8f-e5d1-4c63-b985-0bcd07f75374-combined-ca-bundle\") pod \"nova-cell1-novncproxy-0\" (UID: \"914c0f8f-e5d1-4c63-b985-0bcd07f75374\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:00:20 crc kubenswrapper[5052]: I1203 12:00:20.812526 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-7t9cz\" (UniqueName: \"kubernetes.io/projected/914c0f8f-e5d1-4c63-b985-0bcd07f75374-kube-api-access-7t9cz\") pod \"nova-cell1-novncproxy-0\" (UID: \"914c0f8f-e5d1-4c63-b985-0bcd07f75374\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:00:20 crc kubenswrapper[5052]: I1203 12:00:20.812555 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/914c0f8f-e5d1-4c63-b985-0bcd07f75374-config-data\") pod \"nova-cell1-novncproxy-0\" (UID: \"914c0f8f-e5d1-4c63-b985-0bcd07f75374\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:00:20 crc kubenswrapper[5052]: I1203 12:00:20.812581 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"nova-novncproxy-tls-certs\" (UniqueName: \"kubernetes.io/secret/914c0f8f-e5d1-4c63-b985-0bcd07f75374-nova-novncproxy-tls-certs\") pod \"nova-cell1-novncproxy-0\" (UID: \"914c0f8f-e5d1-4c63-b985-0bcd07f75374\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:00:20 crc kubenswrapper[5052]: I1203 12:00:20.812636 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"vencrypt-tls-certs\" (UniqueName: \"kubernetes.io/secret/914c0f8f-e5d1-4c63-b985-0bcd07f75374-vencrypt-tls-certs\") pod \"nova-cell1-novncproxy-0\" (UID: \"914c0f8f-e5d1-4c63-b985-0bcd07f75374\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:00:20 crc kubenswrapper[5052]: I1203 12:00:20.819336 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"vencrypt-tls-certs\" (UniqueName: \"kubernetes.io/secret/914c0f8f-e5d1-4c63-b985-0bcd07f75374-vencrypt-tls-certs\") pod \"nova-cell1-novncproxy-0\" (UID: \"914c0f8f-e5d1-4c63-b985-0bcd07f75374\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:00:20 crc kubenswrapper[5052]: I1203 12:00:20.823793 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"nova-novncproxy-tls-certs\" (UniqueName: \"kubernetes.io/secret/914c0f8f-e5d1-4c63-b985-0bcd07f75374-nova-novncproxy-tls-certs\") pod \"nova-cell1-novncproxy-0\" (UID: \"914c0f8f-e5d1-4c63-b985-0bcd07f75374\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:00:20 crc kubenswrapper[5052]: I1203 12:00:20.827394 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/914c0f8f-e5d1-4c63-b985-0bcd07f75374-config-data\") pod \"nova-cell1-novncproxy-0\" (UID: \"914c0f8f-e5d1-4c63-b985-0bcd07f75374\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:00:20 crc kubenswrapper[5052]: I1203 12:00:20.828181 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/914c0f8f-e5d1-4c63-b985-0bcd07f75374-combined-ca-bundle\") pod \"nova-cell1-novncproxy-0\" (UID: \"914c0f8f-e5d1-4c63-b985-0bcd07f75374\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:00:20 crc kubenswrapper[5052]: I1203 12:00:20.841120 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-7t9cz\" (UniqueName: \"kubernetes.io/projected/914c0f8f-e5d1-4c63-b985-0bcd07f75374-kube-api-access-7t9cz\") pod \"nova-cell1-novncproxy-0\" (UID: \"914c0f8f-e5d1-4c63-b985-0bcd07f75374\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:00:20 crc kubenswrapper[5052]: I1203 12:00:20.967312 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:00:22 crc kubenswrapper[5052]: I1203 12:00:22.046024 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1f41370a-19d0-4c2d-b942-26562f752813" path="/var/lib/kubelet/pods/1f41370a-19d0-4c2d-b942-26562f752813/volumes" Dec 03 12:00:23 crc kubenswrapper[5052]: I1203 12:00:23.033178 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:00:23 crc kubenswrapper[5052]: I1203 12:00:23.084105 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:00:23 crc kubenswrapper[5052]: I1203 12:00:23.084167 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:00:23 crc kubenswrapper[5052]: I1203 12:00:23.085462 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:00:23 crc kubenswrapper[5052]: I1203 12:00:23.085482 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:00:23 crc kubenswrapper[5052]: I1203 12:00:23.089776 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:00:23 crc kubenswrapper[5052]: I1203 12:00:23.096857 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:00:23 crc kubenswrapper[5052]: W1203 12:00:23.372138 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod914c0f8f_e5d1_4c63_b985_0bcd07f75374.slice/crio-ddc1a19aa77d2a741435f4f1d516057d357cd30b098acd5e03d21402ad882813 WatchSource:0}: Error finding container ddc1a19aa77d2a741435f4f1d516057d357cd30b098acd5e03d21402ad882813: Status 404 returned error can't find the container with id ddc1a19aa77d2a741435f4f1d516057d357cd30b098acd5e03d21402ad882813 Dec 03 12:00:23 crc kubenswrapper[5052]: I1203 12:00:23.374671 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 12:00:23 crc kubenswrapper[5052]: I1203 12:00:23.593492 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"900bf7c0-8907-4d20-9d2c-9c6716cf7d4c","Type":"ContainerStarted","Data":"9aa5536e427e473cee4877086b09688ac33285338ec0866fa111e98760a53d9d"} Dec 03 12:00:23 crc kubenswrapper[5052]: I1203 12:00:23.594224 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:00:23 crc kubenswrapper[5052]: I1203 12:00:23.595795 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" event={"ID":"914c0f8f-e5d1-4c63-b985-0bcd07f75374","Type":"ContainerStarted","Data":"ddc1a19aa77d2a741435f4f1d516057d357cd30b098acd5e03d21402ad882813"} Dec 03 12:00:23 crc kubenswrapper[5052]: I1203 12:00:23.623961 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/ceilometer-0" podStartSLOduration=1.994155267 podStartE2EDuration="7.623934118s" podCreationTimestamp="2025-12-03 12:00:16 +0000 UTC" firstStartedPulling="2025-12-03 12:00:17.420095455 +0000 UTC m=+2408.008337219" lastFinishedPulling="2025-12-03 12:00:23.049874306 +0000 UTC m=+2413.638116070" observedRunningTime="2025-12-03 12:00:23.617733501 +0000 UTC m=+2414.205975265" watchObservedRunningTime="2025-12-03 12:00:23.623934118 +0000 UTC m=+2414.212175882" Dec 03 12:00:24 crc kubenswrapper[5052]: I1203 12:00:24.607073 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" event={"ID":"914c0f8f-e5d1-4c63-b985-0bcd07f75374","Type":"ContainerStarted","Data":"2d2f9846f42dc135f23ee0343741d94b75b793c15135ba895cc30047a367cc1e"} Dec 03 12:00:24 crc kubenswrapper[5052]: I1203 12:00:24.635795 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" podStartSLOduration=4.635775527 podStartE2EDuration="4.635775527s" podCreationTimestamp="2025-12-03 12:00:20 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:00:24.626580564 +0000 UTC m=+2415.214822328" watchObservedRunningTime="2025-12-03 12:00:24.635775527 +0000 UTC m=+2415.224017291" Dec 03 12:00:25 crc kubenswrapper[5052]: I1203 12:00:25.019059 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:00:25 crc kubenswrapper[5052]: I1203 12:00:25.616031 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="900bf7c0-8907-4d20-9d2c-9c6716cf7d4c" containerName="sg-core" containerID="cri-o://e7cc78dd31d4e2ecd3f97d9d229e97e11d244874476d9cf5fa8c317f1ce47ae9" gracePeriod=30 Dec 03 12:00:25 crc kubenswrapper[5052]: I1203 12:00:25.616031 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="900bf7c0-8907-4d20-9d2c-9c6716cf7d4c" containerName="ceilometer-notification-agent" containerID="cri-o://052af835c664bb69b95dc6b37b6f6a9a9acfd9bbf7fd2efa3efaf8768515cba4" gracePeriod=30 Dec 03 12:00:25 crc kubenswrapper[5052]: I1203 12:00:25.616068 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="900bf7c0-8907-4d20-9d2c-9c6716cf7d4c" containerName="proxy-httpd" containerID="cri-o://9aa5536e427e473cee4877086b09688ac33285338ec0866fa111e98760a53d9d" gracePeriod=30 Dec 03 12:00:25 crc kubenswrapper[5052]: I1203 12:00:25.616163 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="900bf7c0-8907-4d20-9d2c-9c6716cf7d4c" containerName="ceilometer-central-agent" containerID="cri-o://5a46c706b716d58c0e633d615cd546a53bde37fdd9dc2b1a4dcca2b69b88cc8b" gracePeriod=30 Dec 03 12:00:25 crc kubenswrapper[5052]: I1203 12:00:25.678500 5052 scope.go:117] "RemoveContainer" containerID="d224717c6d9ca8135fd510e03a6c40c9f2b1a56dd8b4a3b685af3e2dfa855982" Dec 03 12:00:25 crc kubenswrapper[5052]: E1203 12:00:25.678876 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 12:00:25 crc kubenswrapper[5052]: I1203 12:00:25.963843 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:00:25 crc kubenswrapper[5052]: I1203 12:00:25.964086 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-api-0" podUID="eebaba62-8aa5-447a-bcc2-590bb40de88f" containerName="nova-api-log" containerID="cri-o://69ee8e1bd36c85abc2f41d4ac965fe34ba9a46a96372eb56d8bb396f2b5d9337" gracePeriod=30 Dec 03 12:00:25 crc kubenswrapper[5052]: I1203 12:00:25.964208 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-api-0" podUID="eebaba62-8aa5-447a-bcc2-590bb40de88f" containerName="nova-api-api" containerID="cri-o://1a49b9d7137fb6a41e5ffeefb32672f648c89fd732896a81025268cb006acfaf" gracePeriod=30 Dec 03 12:00:25 crc kubenswrapper[5052]: I1203 12:00:25.967860 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:00:26 crc kubenswrapper[5052]: I1203 12:00:26.628963 5052 generic.go:334] "Generic (PLEG): container finished" podID="900bf7c0-8907-4d20-9d2c-9c6716cf7d4c" containerID="9aa5536e427e473cee4877086b09688ac33285338ec0866fa111e98760a53d9d" exitCode=0 Dec 03 12:00:26 crc kubenswrapper[5052]: I1203 12:00:26.628998 5052 generic.go:334] "Generic (PLEG): container finished" podID="900bf7c0-8907-4d20-9d2c-9c6716cf7d4c" containerID="e7cc78dd31d4e2ecd3f97d9d229e97e11d244874476d9cf5fa8c317f1ce47ae9" exitCode=2 Dec 03 12:00:26 crc kubenswrapper[5052]: I1203 12:00:26.629008 5052 generic.go:334] "Generic (PLEG): container finished" podID="900bf7c0-8907-4d20-9d2c-9c6716cf7d4c" containerID="052af835c664bb69b95dc6b37b6f6a9a9acfd9bbf7fd2efa3efaf8768515cba4" exitCode=0 Dec 03 12:00:26 crc kubenswrapper[5052]: I1203 12:00:26.629024 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"900bf7c0-8907-4d20-9d2c-9c6716cf7d4c","Type":"ContainerDied","Data":"9aa5536e427e473cee4877086b09688ac33285338ec0866fa111e98760a53d9d"} Dec 03 12:00:26 crc kubenswrapper[5052]: I1203 12:00:26.629083 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"900bf7c0-8907-4d20-9d2c-9c6716cf7d4c","Type":"ContainerDied","Data":"e7cc78dd31d4e2ecd3f97d9d229e97e11d244874476d9cf5fa8c317f1ce47ae9"} Dec 03 12:00:26 crc kubenswrapper[5052]: I1203 12:00:26.629099 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"900bf7c0-8907-4d20-9d2c-9c6716cf7d4c","Type":"ContainerDied","Data":"052af835c664bb69b95dc6b37b6f6a9a9acfd9bbf7fd2efa3efaf8768515cba4"} Dec 03 12:00:26 crc kubenswrapper[5052]: I1203 12:00:26.631290 5052 generic.go:334] "Generic (PLEG): container finished" podID="eebaba62-8aa5-447a-bcc2-590bb40de88f" containerID="69ee8e1bd36c85abc2f41d4ac965fe34ba9a46a96372eb56d8bb396f2b5d9337" exitCode=143 Dec 03 12:00:26 crc kubenswrapper[5052]: I1203 12:00:26.631390 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"eebaba62-8aa5-447a-bcc2-590bb40de88f","Type":"ContainerDied","Data":"69ee8e1bd36c85abc2f41d4ac965fe34ba9a46a96372eb56d8bb396f2b5d9337"} Dec 03 12:00:27 crc kubenswrapper[5052]: I1203 12:00:27.644608 5052 generic.go:334] "Generic (PLEG): container finished" podID="900bf7c0-8907-4d20-9d2c-9c6716cf7d4c" containerID="5a46c706b716d58c0e633d615cd546a53bde37fdd9dc2b1a4dcca2b69b88cc8b" exitCode=0 Dec 03 12:00:27 crc kubenswrapper[5052]: I1203 12:00:27.644683 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"900bf7c0-8907-4d20-9d2c-9c6716cf7d4c","Type":"ContainerDied","Data":"5a46c706b716d58c0e633d615cd546a53bde37fdd9dc2b1a4dcca2b69b88cc8b"} Dec 03 12:00:27 crc kubenswrapper[5052]: I1203 12:00:27.766072 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:00:27 crc kubenswrapper[5052]: I1203 12:00:27.847347 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/900bf7c0-8907-4d20-9d2c-9c6716cf7d4c-config-data\") pod \"900bf7c0-8907-4d20-9d2c-9c6716cf7d4c\" (UID: \"900bf7c0-8907-4d20-9d2c-9c6716cf7d4c\") " Dec 03 12:00:27 crc kubenswrapper[5052]: I1203 12:00:27.847449 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/900bf7c0-8907-4d20-9d2c-9c6716cf7d4c-run-httpd\") pod \"900bf7c0-8907-4d20-9d2c-9c6716cf7d4c\" (UID: \"900bf7c0-8907-4d20-9d2c-9c6716cf7d4c\") " Dec 03 12:00:27 crc kubenswrapper[5052]: I1203 12:00:27.847478 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2gldt\" (UniqueName: \"kubernetes.io/projected/900bf7c0-8907-4d20-9d2c-9c6716cf7d4c-kube-api-access-2gldt\") pod \"900bf7c0-8907-4d20-9d2c-9c6716cf7d4c\" (UID: \"900bf7c0-8907-4d20-9d2c-9c6716cf7d4c\") " Dec 03 12:00:27 crc kubenswrapper[5052]: I1203 12:00:27.847611 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/900bf7c0-8907-4d20-9d2c-9c6716cf7d4c-scripts\") pod \"900bf7c0-8907-4d20-9d2c-9c6716cf7d4c\" (UID: \"900bf7c0-8907-4d20-9d2c-9c6716cf7d4c\") " Dec 03 12:00:27 crc kubenswrapper[5052]: I1203 12:00:27.847640 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/900bf7c0-8907-4d20-9d2c-9c6716cf7d4c-log-httpd\") pod \"900bf7c0-8907-4d20-9d2c-9c6716cf7d4c\" (UID: \"900bf7c0-8907-4d20-9d2c-9c6716cf7d4c\") " Dec 03 12:00:27 crc kubenswrapper[5052]: I1203 12:00:27.847673 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/900bf7c0-8907-4d20-9d2c-9c6716cf7d4c-sg-core-conf-yaml\") pod \"900bf7c0-8907-4d20-9d2c-9c6716cf7d4c\" (UID: \"900bf7c0-8907-4d20-9d2c-9c6716cf7d4c\") " Dec 03 12:00:27 crc kubenswrapper[5052]: I1203 12:00:27.847704 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/900bf7c0-8907-4d20-9d2c-9c6716cf7d4c-ceilometer-tls-certs\") pod \"900bf7c0-8907-4d20-9d2c-9c6716cf7d4c\" (UID: \"900bf7c0-8907-4d20-9d2c-9c6716cf7d4c\") " Dec 03 12:00:27 crc kubenswrapper[5052]: I1203 12:00:27.847753 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/900bf7c0-8907-4d20-9d2c-9c6716cf7d4c-combined-ca-bundle\") pod \"900bf7c0-8907-4d20-9d2c-9c6716cf7d4c\" (UID: \"900bf7c0-8907-4d20-9d2c-9c6716cf7d4c\") " Dec 03 12:00:27 crc kubenswrapper[5052]: I1203 12:00:27.848396 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/900bf7c0-8907-4d20-9d2c-9c6716cf7d4c-run-httpd" (OuterVolumeSpecName: "run-httpd") pod "900bf7c0-8907-4d20-9d2c-9c6716cf7d4c" (UID: "900bf7c0-8907-4d20-9d2c-9c6716cf7d4c"). InnerVolumeSpecName "run-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:00:27 crc kubenswrapper[5052]: I1203 12:00:27.848476 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/900bf7c0-8907-4d20-9d2c-9c6716cf7d4c-log-httpd" (OuterVolumeSpecName: "log-httpd") pod "900bf7c0-8907-4d20-9d2c-9c6716cf7d4c" (UID: "900bf7c0-8907-4d20-9d2c-9c6716cf7d4c"). InnerVolumeSpecName "log-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:00:27 crc kubenswrapper[5052]: I1203 12:00:27.853273 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/900bf7c0-8907-4d20-9d2c-9c6716cf7d4c-scripts" (OuterVolumeSpecName: "scripts") pod "900bf7c0-8907-4d20-9d2c-9c6716cf7d4c" (UID: "900bf7c0-8907-4d20-9d2c-9c6716cf7d4c"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:00:27 crc kubenswrapper[5052]: I1203 12:00:27.854276 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/900bf7c0-8907-4d20-9d2c-9c6716cf7d4c-kube-api-access-2gldt" (OuterVolumeSpecName: "kube-api-access-2gldt") pod "900bf7c0-8907-4d20-9d2c-9c6716cf7d4c" (UID: "900bf7c0-8907-4d20-9d2c-9c6716cf7d4c"). InnerVolumeSpecName "kube-api-access-2gldt". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:00:27 crc kubenswrapper[5052]: I1203 12:00:27.878178 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/900bf7c0-8907-4d20-9d2c-9c6716cf7d4c-sg-core-conf-yaml" (OuterVolumeSpecName: "sg-core-conf-yaml") pod "900bf7c0-8907-4d20-9d2c-9c6716cf7d4c" (UID: "900bf7c0-8907-4d20-9d2c-9c6716cf7d4c"). InnerVolumeSpecName "sg-core-conf-yaml". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:00:27 crc kubenswrapper[5052]: I1203 12:00:27.931356 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/900bf7c0-8907-4d20-9d2c-9c6716cf7d4c-ceilometer-tls-certs" (OuterVolumeSpecName: "ceilometer-tls-certs") pod "900bf7c0-8907-4d20-9d2c-9c6716cf7d4c" (UID: "900bf7c0-8907-4d20-9d2c-9c6716cf7d4c"). InnerVolumeSpecName "ceilometer-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:00:27 crc kubenswrapper[5052]: I1203 12:00:27.945929 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/900bf7c0-8907-4d20-9d2c-9c6716cf7d4c-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "900bf7c0-8907-4d20-9d2c-9c6716cf7d4c" (UID: "900bf7c0-8907-4d20-9d2c-9c6716cf7d4c"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:00:27 crc kubenswrapper[5052]: I1203 12:00:27.950135 5052 reconciler_common.go:293] "Volume detached for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/900bf7c0-8907-4d20-9d2c-9c6716cf7d4c-run-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:00:27 crc kubenswrapper[5052]: I1203 12:00:27.950253 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-2gldt\" (UniqueName: \"kubernetes.io/projected/900bf7c0-8907-4d20-9d2c-9c6716cf7d4c-kube-api-access-2gldt\") on node \"crc\" DevicePath \"\"" Dec 03 12:00:27 crc kubenswrapper[5052]: I1203 12:00:27.950325 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/900bf7c0-8907-4d20-9d2c-9c6716cf7d4c-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:00:27 crc kubenswrapper[5052]: I1203 12:00:27.950378 5052 reconciler_common.go:293] "Volume detached for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/900bf7c0-8907-4d20-9d2c-9c6716cf7d4c-log-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:00:27 crc kubenswrapper[5052]: I1203 12:00:27.950433 5052 reconciler_common.go:293] "Volume detached for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/900bf7c0-8907-4d20-9d2c-9c6716cf7d4c-sg-core-conf-yaml\") on node \"crc\" DevicePath \"\"" Dec 03 12:00:27 crc kubenswrapper[5052]: I1203 12:00:27.950491 5052 reconciler_common.go:293] "Volume detached for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/900bf7c0-8907-4d20-9d2c-9c6716cf7d4c-ceilometer-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:00:27 crc kubenswrapper[5052]: I1203 12:00:27.950580 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/900bf7c0-8907-4d20-9d2c-9c6716cf7d4c-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:00:27 crc kubenswrapper[5052]: I1203 12:00:27.953251 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/900bf7c0-8907-4d20-9d2c-9c6716cf7d4c-config-data" (OuterVolumeSpecName: "config-data") pod "900bf7c0-8907-4d20-9d2c-9c6716cf7d4c" (UID: "900bf7c0-8907-4d20-9d2c-9c6716cf7d4c"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:00:28 crc kubenswrapper[5052]: I1203 12:00:28.052696 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/900bf7c0-8907-4d20-9d2c-9c6716cf7d4c-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:00:28 crc kubenswrapper[5052]: I1203 12:00:28.658291 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"900bf7c0-8907-4d20-9d2c-9c6716cf7d4c","Type":"ContainerDied","Data":"cce5e575166559fcedc44682c8b032408b60771e5a3483bc8da1f364906ffe48"} Dec 03 12:00:28 crc kubenswrapper[5052]: I1203 12:00:28.658351 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:00:28 crc kubenswrapper[5052]: I1203 12:00:28.659393 5052 scope.go:117] "RemoveContainer" containerID="9aa5536e427e473cee4877086b09688ac33285338ec0866fa111e98760a53d9d" Dec 03 12:00:28 crc kubenswrapper[5052]: I1203 12:00:28.692703 5052 scope.go:117] "RemoveContainer" containerID="e7cc78dd31d4e2ecd3f97d9d229e97e11d244874476d9cf5fa8c317f1ce47ae9" Dec 03 12:00:28 crc kubenswrapper[5052]: I1203 12:00:28.694585 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:00:28 crc kubenswrapper[5052]: I1203 12:00:28.706243 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:00:28 crc kubenswrapper[5052]: I1203 12:00:28.724820 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:00:28 crc kubenswrapper[5052]: E1203 12:00:28.725585 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="900bf7c0-8907-4d20-9d2c-9c6716cf7d4c" containerName="ceilometer-notification-agent" Dec 03 12:00:28 crc kubenswrapper[5052]: I1203 12:00:28.725769 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="900bf7c0-8907-4d20-9d2c-9c6716cf7d4c" containerName="ceilometer-notification-agent" Dec 03 12:00:28 crc kubenswrapper[5052]: E1203 12:00:28.725843 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="900bf7c0-8907-4d20-9d2c-9c6716cf7d4c" containerName="sg-core" Dec 03 12:00:28 crc kubenswrapper[5052]: I1203 12:00:28.725904 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="900bf7c0-8907-4d20-9d2c-9c6716cf7d4c" containerName="sg-core" Dec 03 12:00:28 crc kubenswrapper[5052]: E1203 12:00:28.725970 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="900bf7c0-8907-4d20-9d2c-9c6716cf7d4c" containerName="ceilometer-central-agent" Dec 03 12:00:28 crc kubenswrapper[5052]: I1203 12:00:28.726029 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="900bf7c0-8907-4d20-9d2c-9c6716cf7d4c" containerName="ceilometer-central-agent" Dec 03 12:00:28 crc kubenswrapper[5052]: E1203 12:00:28.726125 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="900bf7c0-8907-4d20-9d2c-9c6716cf7d4c" containerName="proxy-httpd" Dec 03 12:00:28 crc kubenswrapper[5052]: I1203 12:00:28.726184 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="900bf7c0-8907-4d20-9d2c-9c6716cf7d4c" containerName="proxy-httpd" Dec 03 12:00:28 crc kubenswrapper[5052]: I1203 12:00:28.726578 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="900bf7c0-8907-4d20-9d2c-9c6716cf7d4c" containerName="ceilometer-central-agent" Dec 03 12:00:28 crc kubenswrapper[5052]: I1203 12:00:28.726670 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="900bf7c0-8907-4d20-9d2c-9c6716cf7d4c" containerName="ceilometer-notification-agent" Dec 03 12:00:28 crc kubenswrapper[5052]: I1203 12:00:28.726742 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="900bf7c0-8907-4d20-9d2c-9c6716cf7d4c" containerName="sg-core" Dec 03 12:00:28 crc kubenswrapper[5052]: I1203 12:00:28.726815 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="900bf7c0-8907-4d20-9d2c-9c6716cf7d4c" containerName="proxy-httpd" Dec 03 12:00:28 crc kubenswrapper[5052]: I1203 12:00:28.730778 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:00:28 crc kubenswrapper[5052]: I1203 12:00:28.733438 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-scripts" Dec 03 12:00:28 crc kubenswrapper[5052]: I1203 12:00:28.733751 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-ceilometer-internal-svc" Dec 03 12:00:28 crc kubenswrapper[5052]: I1203 12:00:28.733831 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-config-data" Dec 03 12:00:28 crc kubenswrapper[5052]: I1203 12:00:28.734916 5052 scope.go:117] "RemoveContainer" containerID="052af835c664bb69b95dc6b37b6f6a9a9acfd9bbf7fd2efa3efaf8768515cba4" Dec 03 12:00:28 crc kubenswrapper[5052]: I1203 12:00:28.752473 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:00:28 crc kubenswrapper[5052]: I1203 12:00:28.786679 5052 scope.go:117] "RemoveContainer" containerID="5a46c706b716d58c0e633d615cd546a53bde37fdd9dc2b1a4dcca2b69b88cc8b" Dec 03 12:00:28 crc kubenswrapper[5052]: I1203 12:00:28.870352 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/da4dfbea-20a2-4638-bbbe-ed4bfee0ad49-run-httpd\") pod \"ceilometer-0\" (UID: \"da4dfbea-20a2-4638-bbbe-ed4bfee0ad49\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:00:28 crc kubenswrapper[5052]: I1203 12:00:28.870405 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/da4dfbea-20a2-4638-bbbe-ed4bfee0ad49-ceilometer-tls-certs\") pod \"ceilometer-0\" (UID: \"da4dfbea-20a2-4638-bbbe-ed4bfee0ad49\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:00:28 crc kubenswrapper[5052]: I1203 12:00:28.870438 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kd4bh\" (UniqueName: \"kubernetes.io/projected/da4dfbea-20a2-4638-bbbe-ed4bfee0ad49-kube-api-access-kd4bh\") pod \"ceilometer-0\" (UID: \"da4dfbea-20a2-4638-bbbe-ed4bfee0ad49\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:00:28 crc kubenswrapper[5052]: I1203 12:00:28.870555 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/da4dfbea-20a2-4638-bbbe-ed4bfee0ad49-scripts\") pod \"ceilometer-0\" (UID: \"da4dfbea-20a2-4638-bbbe-ed4bfee0ad49\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:00:28 crc kubenswrapper[5052]: I1203 12:00:28.870589 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/da4dfbea-20a2-4638-bbbe-ed4bfee0ad49-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"da4dfbea-20a2-4638-bbbe-ed4bfee0ad49\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:00:28 crc kubenswrapper[5052]: I1203 12:00:28.870633 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/da4dfbea-20a2-4638-bbbe-ed4bfee0ad49-config-data\") pod \"ceilometer-0\" (UID: \"da4dfbea-20a2-4638-bbbe-ed4bfee0ad49\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:00:28 crc kubenswrapper[5052]: I1203 12:00:28.870655 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/da4dfbea-20a2-4638-bbbe-ed4bfee0ad49-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"da4dfbea-20a2-4638-bbbe-ed4bfee0ad49\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:00:28 crc kubenswrapper[5052]: I1203 12:00:28.870702 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/da4dfbea-20a2-4638-bbbe-ed4bfee0ad49-log-httpd\") pod \"ceilometer-0\" (UID: \"da4dfbea-20a2-4638-bbbe-ed4bfee0ad49\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:00:28 crc kubenswrapper[5052]: I1203 12:00:28.972397 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/da4dfbea-20a2-4638-bbbe-ed4bfee0ad49-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"da4dfbea-20a2-4638-bbbe-ed4bfee0ad49\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:00:28 crc kubenswrapper[5052]: I1203 12:00:28.972851 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/da4dfbea-20a2-4638-bbbe-ed4bfee0ad49-config-data\") pod \"ceilometer-0\" (UID: \"da4dfbea-20a2-4638-bbbe-ed4bfee0ad49\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:00:28 crc kubenswrapper[5052]: I1203 12:00:28.973023 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/da4dfbea-20a2-4638-bbbe-ed4bfee0ad49-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"da4dfbea-20a2-4638-bbbe-ed4bfee0ad49\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:00:28 crc kubenswrapper[5052]: I1203 12:00:28.973175 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/da4dfbea-20a2-4638-bbbe-ed4bfee0ad49-log-httpd\") pod \"ceilometer-0\" (UID: \"da4dfbea-20a2-4638-bbbe-ed4bfee0ad49\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:00:28 crc kubenswrapper[5052]: I1203 12:00:28.973433 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/da4dfbea-20a2-4638-bbbe-ed4bfee0ad49-run-httpd\") pod \"ceilometer-0\" (UID: \"da4dfbea-20a2-4638-bbbe-ed4bfee0ad49\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:00:28 crc kubenswrapper[5052]: I1203 12:00:28.973623 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/da4dfbea-20a2-4638-bbbe-ed4bfee0ad49-ceilometer-tls-certs\") pod \"ceilometer-0\" (UID: \"da4dfbea-20a2-4638-bbbe-ed4bfee0ad49\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:00:28 crc kubenswrapper[5052]: I1203 12:00:28.973795 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/da4dfbea-20a2-4638-bbbe-ed4bfee0ad49-run-httpd\") pod \"ceilometer-0\" (UID: \"da4dfbea-20a2-4638-bbbe-ed4bfee0ad49\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:00:28 crc kubenswrapper[5052]: I1203 12:00:28.973872 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/da4dfbea-20a2-4638-bbbe-ed4bfee0ad49-log-httpd\") pod \"ceilometer-0\" (UID: \"da4dfbea-20a2-4638-bbbe-ed4bfee0ad49\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:00:28 crc kubenswrapper[5052]: I1203 12:00:28.973783 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-kd4bh\" (UniqueName: \"kubernetes.io/projected/da4dfbea-20a2-4638-bbbe-ed4bfee0ad49-kube-api-access-kd4bh\") pod \"ceilometer-0\" (UID: \"da4dfbea-20a2-4638-bbbe-ed4bfee0ad49\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:00:28 crc kubenswrapper[5052]: I1203 12:00:28.976082 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/da4dfbea-20a2-4638-bbbe-ed4bfee0ad49-scripts\") pod \"ceilometer-0\" (UID: \"da4dfbea-20a2-4638-bbbe-ed4bfee0ad49\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:00:28 crc kubenswrapper[5052]: I1203 12:00:28.978423 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/da4dfbea-20a2-4638-bbbe-ed4bfee0ad49-ceilometer-tls-certs\") pod \"ceilometer-0\" (UID: \"da4dfbea-20a2-4638-bbbe-ed4bfee0ad49\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:00:28 crc kubenswrapper[5052]: I1203 12:00:28.978435 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/da4dfbea-20a2-4638-bbbe-ed4bfee0ad49-config-data\") pod \"ceilometer-0\" (UID: \"da4dfbea-20a2-4638-bbbe-ed4bfee0ad49\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:00:28 crc kubenswrapper[5052]: I1203 12:00:28.978446 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/da4dfbea-20a2-4638-bbbe-ed4bfee0ad49-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"da4dfbea-20a2-4638-bbbe-ed4bfee0ad49\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:00:28 crc kubenswrapper[5052]: I1203 12:00:28.979282 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/da4dfbea-20a2-4638-bbbe-ed4bfee0ad49-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"da4dfbea-20a2-4638-bbbe-ed4bfee0ad49\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:00:28 crc kubenswrapper[5052]: I1203 12:00:28.989434 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/da4dfbea-20a2-4638-bbbe-ed4bfee0ad49-scripts\") pod \"ceilometer-0\" (UID: \"da4dfbea-20a2-4638-bbbe-ed4bfee0ad49\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:00:28 crc kubenswrapper[5052]: I1203 12:00:28.991368 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-kd4bh\" (UniqueName: \"kubernetes.io/projected/da4dfbea-20a2-4638-bbbe-ed4bfee0ad49-kube-api-access-kd4bh\") pod \"ceilometer-0\" (UID: \"da4dfbea-20a2-4638-bbbe-ed4bfee0ad49\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:00:29 crc kubenswrapper[5052]: I1203 12:00:29.052842 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:00:29 crc kubenswrapper[5052]: I1203 12:00:29.503855 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:00:29 crc kubenswrapper[5052]: W1203 12:00:29.509215 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podda4dfbea_20a2_4638_bbbe_ed4bfee0ad49.slice/crio-9328c605b4db437b147c5c1a4f4757786be4e46886d96fefe2e7797e925c5676 WatchSource:0}: Error finding container 9328c605b4db437b147c5c1a4f4757786be4e46886d96fefe2e7797e925c5676: Status 404 returned error can't find the container with id 9328c605b4db437b147c5c1a4f4757786be4e46886d96fefe2e7797e925c5676 Dec 03 12:00:29 crc kubenswrapper[5052]: I1203 12:00:29.606866 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:00:29 crc kubenswrapper[5052]: I1203 12:00:29.670343 5052 generic.go:334] "Generic (PLEG): container finished" podID="eebaba62-8aa5-447a-bcc2-590bb40de88f" containerID="1a49b9d7137fb6a41e5ffeefb32672f648c89fd732896a81025268cb006acfaf" exitCode=0 Dec 03 12:00:29 crc kubenswrapper[5052]: I1203 12:00:29.670424 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:00:29 crc kubenswrapper[5052]: I1203 12:00:29.670438 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"eebaba62-8aa5-447a-bcc2-590bb40de88f","Type":"ContainerDied","Data":"1a49b9d7137fb6a41e5ffeefb32672f648c89fd732896a81025268cb006acfaf"} Dec 03 12:00:29 crc kubenswrapper[5052]: I1203 12:00:29.670472 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"eebaba62-8aa5-447a-bcc2-590bb40de88f","Type":"ContainerDied","Data":"0da99586eb02a317f52fc0a1ac125a104da1eee129069c6fd6c62850608eceb6"} Dec 03 12:00:29 crc kubenswrapper[5052]: I1203 12:00:29.670495 5052 scope.go:117] "RemoveContainer" containerID="1a49b9d7137fb6a41e5ffeefb32672f648c89fd732896a81025268cb006acfaf" Dec 03 12:00:29 crc kubenswrapper[5052]: I1203 12:00:29.690759 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="900bf7c0-8907-4d20-9d2c-9c6716cf7d4c" path="/var/lib/kubelet/pods/900bf7c0-8907-4d20-9d2c-9c6716cf7d4c/volumes" Dec 03 12:00:29 crc kubenswrapper[5052]: I1203 12:00:29.693044 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/eebaba62-8aa5-447a-bcc2-590bb40de88f-combined-ca-bundle\") pod \"eebaba62-8aa5-447a-bcc2-590bb40de88f\" (UID: \"eebaba62-8aa5-447a-bcc2-590bb40de88f\") " Dec 03 12:00:29 crc kubenswrapper[5052]: I1203 12:00:29.693247 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/eebaba62-8aa5-447a-bcc2-590bb40de88f-logs\") pod \"eebaba62-8aa5-447a-bcc2-590bb40de88f\" (UID: \"eebaba62-8aa5-447a-bcc2-590bb40de88f\") " Dec 03 12:00:29 crc kubenswrapper[5052]: I1203 12:00:29.693282 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/eebaba62-8aa5-447a-bcc2-590bb40de88f-config-data\") pod \"eebaba62-8aa5-447a-bcc2-590bb40de88f\" (UID: \"eebaba62-8aa5-447a-bcc2-590bb40de88f\") " Dec 03 12:00:29 crc kubenswrapper[5052]: I1203 12:00:29.693325 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dmtxq\" (UniqueName: \"kubernetes.io/projected/eebaba62-8aa5-447a-bcc2-590bb40de88f-kube-api-access-dmtxq\") pod \"eebaba62-8aa5-447a-bcc2-590bb40de88f\" (UID: \"eebaba62-8aa5-447a-bcc2-590bb40de88f\") " Dec 03 12:00:29 crc kubenswrapper[5052]: I1203 12:00:29.694133 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eebaba62-8aa5-447a-bcc2-590bb40de88f-logs" (OuterVolumeSpecName: "logs") pod "eebaba62-8aa5-447a-bcc2-590bb40de88f" (UID: "eebaba62-8aa5-447a-bcc2-590bb40de88f"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:00:29 crc kubenswrapper[5052]: I1203 12:00:29.695533 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/eebaba62-8aa5-447a-bcc2-590bb40de88f-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:00:29 crc kubenswrapper[5052]: I1203 12:00:29.698819 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eebaba62-8aa5-447a-bcc2-590bb40de88f-kube-api-access-dmtxq" (OuterVolumeSpecName: "kube-api-access-dmtxq") pod "eebaba62-8aa5-447a-bcc2-590bb40de88f" (UID: "eebaba62-8aa5-447a-bcc2-590bb40de88f"). InnerVolumeSpecName "kube-api-access-dmtxq". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:00:29 crc kubenswrapper[5052]: I1203 12:00:29.701883 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"da4dfbea-20a2-4638-bbbe-ed4bfee0ad49","Type":"ContainerStarted","Data":"9328c605b4db437b147c5c1a4f4757786be4e46886d96fefe2e7797e925c5676"} Dec 03 12:00:29 crc kubenswrapper[5052]: I1203 12:00:29.702302 5052 scope.go:117] "RemoveContainer" containerID="69ee8e1bd36c85abc2f41d4ac965fe34ba9a46a96372eb56d8bb396f2b5d9337" Dec 03 12:00:29 crc kubenswrapper[5052]: I1203 12:00:29.731936 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eebaba62-8aa5-447a-bcc2-590bb40de88f-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "eebaba62-8aa5-447a-bcc2-590bb40de88f" (UID: "eebaba62-8aa5-447a-bcc2-590bb40de88f"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:00:29 crc kubenswrapper[5052]: I1203 12:00:29.753947 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eebaba62-8aa5-447a-bcc2-590bb40de88f-config-data" (OuterVolumeSpecName: "config-data") pod "eebaba62-8aa5-447a-bcc2-590bb40de88f" (UID: "eebaba62-8aa5-447a-bcc2-590bb40de88f"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:00:29 crc kubenswrapper[5052]: I1203 12:00:29.797567 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/eebaba62-8aa5-447a-bcc2-590bb40de88f-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:00:29 crc kubenswrapper[5052]: I1203 12:00:29.797775 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/eebaba62-8aa5-447a-bcc2-590bb40de88f-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:00:29 crc kubenswrapper[5052]: I1203 12:00:29.797859 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-dmtxq\" (UniqueName: \"kubernetes.io/projected/eebaba62-8aa5-447a-bcc2-590bb40de88f-kube-api-access-dmtxq\") on node \"crc\" DevicePath \"\"" Dec 03 12:00:29 crc kubenswrapper[5052]: I1203 12:00:29.800638 5052 scope.go:117] "RemoveContainer" containerID="1a49b9d7137fb6a41e5ffeefb32672f648c89fd732896a81025268cb006acfaf" Dec 03 12:00:29 crc kubenswrapper[5052]: E1203 12:00:29.801099 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1a49b9d7137fb6a41e5ffeefb32672f648c89fd732896a81025268cb006acfaf\": container with ID starting with 1a49b9d7137fb6a41e5ffeefb32672f648c89fd732896a81025268cb006acfaf not found: ID does not exist" containerID="1a49b9d7137fb6a41e5ffeefb32672f648c89fd732896a81025268cb006acfaf" Dec 03 12:00:29 crc kubenswrapper[5052]: I1203 12:00:29.801146 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1a49b9d7137fb6a41e5ffeefb32672f648c89fd732896a81025268cb006acfaf"} err="failed to get container status \"1a49b9d7137fb6a41e5ffeefb32672f648c89fd732896a81025268cb006acfaf\": rpc error: code = NotFound desc = could not find container \"1a49b9d7137fb6a41e5ffeefb32672f648c89fd732896a81025268cb006acfaf\": container with ID starting with 1a49b9d7137fb6a41e5ffeefb32672f648c89fd732896a81025268cb006acfaf not found: ID does not exist" Dec 03 12:00:29 crc kubenswrapper[5052]: I1203 12:00:29.801173 5052 scope.go:117] "RemoveContainer" containerID="69ee8e1bd36c85abc2f41d4ac965fe34ba9a46a96372eb56d8bb396f2b5d9337" Dec 03 12:00:29 crc kubenswrapper[5052]: E1203 12:00:29.801582 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"69ee8e1bd36c85abc2f41d4ac965fe34ba9a46a96372eb56d8bb396f2b5d9337\": container with ID starting with 69ee8e1bd36c85abc2f41d4ac965fe34ba9a46a96372eb56d8bb396f2b5d9337 not found: ID does not exist" containerID="69ee8e1bd36c85abc2f41d4ac965fe34ba9a46a96372eb56d8bb396f2b5d9337" Dec 03 12:00:29 crc kubenswrapper[5052]: I1203 12:00:29.801620 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"69ee8e1bd36c85abc2f41d4ac965fe34ba9a46a96372eb56d8bb396f2b5d9337"} err="failed to get container status \"69ee8e1bd36c85abc2f41d4ac965fe34ba9a46a96372eb56d8bb396f2b5d9337\": rpc error: code = NotFound desc = could not find container \"69ee8e1bd36c85abc2f41d4ac965fe34ba9a46a96372eb56d8bb396f2b5d9337\": container with ID starting with 69ee8e1bd36c85abc2f41d4ac965fe34ba9a46a96372eb56d8bb396f2b5d9337 not found: ID does not exist" Dec 03 12:00:30 crc kubenswrapper[5052]: I1203 12:00:30.010123 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:00:30 crc kubenswrapper[5052]: I1203 12:00:30.025346 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:00:30 crc kubenswrapper[5052]: I1203 12:00:30.038109 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:00:30 crc kubenswrapper[5052]: E1203 12:00:30.038698 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="eebaba62-8aa5-447a-bcc2-590bb40de88f" containerName="nova-api-api" Dec 03 12:00:30 crc kubenswrapper[5052]: I1203 12:00:30.038722 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="eebaba62-8aa5-447a-bcc2-590bb40de88f" containerName="nova-api-api" Dec 03 12:00:30 crc kubenswrapper[5052]: E1203 12:00:30.038753 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="eebaba62-8aa5-447a-bcc2-590bb40de88f" containerName="nova-api-log" Dec 03 12:00:30 crc kubenswrapper[5052]: I1203 12:00:30.038763 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="eebaba62-8aa5-447a-bcc2-590bb40de88f" containerName="nova-api-log" Dec 03 12:00:30 crc kubenswrapper[5052]: I1203 12:00:30.039025 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="eebaba62-8aa5-447a-bcc2-590bb40de88f" containerName="nova-api-log" Dec 03 12:00:30 crc kubenswrapper[5052]: I1203 12:00:30.039052 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="eebaba62-8aa5-447a-bcc2-590bb40de88f" containerName="nova-api-api" Dec 03 12:00:30 crc kubenswrapper[5052]: I1203 12:00:30.040538 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:00:30 crc kubenswrapper[5052]: I1203 12:00:30.043215 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-nova-internal-svc" Dec 03 12:00:30 crc kubenswrapper[5052]: I1203 12:00:30.043303 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-nova-public-svc" Dec 03 12:00:30 crc kubenswrapper[5052]: I1203 12:00:30.043220 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-api-config-data" Dec 03 12:00:30 crc kubenswrapper[5052]: I1203 12:00:30.048916 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:00:30 crc kubenswrapper[5052]: I1203 12:00:30.103377 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pqpqh\" (UniqueName: \"kubernetes.io/projected/d7d3fec6-50e1-43f1-a7af-0adff12da7e9-kube-api-access-pqpqh\") pod \"nova-api-0\" (UID: \"d7d3fec6-50e1-43f1-a7af-0adff12da7e9\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:00:30 crc kubenswrapper[5052]: I1203 12:00:30.103451 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/d7d3fec6-50e1-43f1-a7af-0adff12da7e9-config-data\") pod \"nova-api-0\" (UID: \"d7d3fec6-50e1-43f1-a7af-0adff12da7e9\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:00:30 crc kubenswrapper[5052]: I1203 12:00:30.103481 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d7d3fec6-50e1-43f1-a7af-0adff12da7e9-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"d7d3fec6-50e1-43f1-a7af-0adff12da7e9\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:00:30 crc kubenswrapper[5052]: I1203 12:00:30.103688 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/d7d3fec6-50e1-43f1-a7af-0adff12da7e9-logs\") pod \"nova-api-0\" (UID: \"d7d3fec6-50e1-43f1-a7af-0adff12da7e9\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:00:30 crc kubenswrapper[5052]: I1203 12:00:30.103903 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/d7d3fec6-50e1-43f1-a7af-0adff12da7e9-internal-tls-certs\") pod \"nova-api-0\" (UID: \"d7d3fec6-50e1-43f1-a7af-0adff12da7e9\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:00:30 crc kubenswrapper[5052]: I1203 12:00:30.103933 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/d7d3fec6-50e1-43f1-a7af-0adff12da7e9-public-tls-certs\") pod \"nova-api-0\" (UID: \"d7d3fec6-50e1-43f1-a7af-0adff12da7e9\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:00:30 crc kubenswrapper[5052]: I1203 12:00:30.205788 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/d7d3fec6-50e1-43f1-a7af-0adff12da7e9-internal-tls-certs\") pod \"nova-api-0\" (UID: \"d7d3fec6-50e1-43f1-a7af-0adff12da7e9\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:00:30 crc kubenswrapper[5052]: I1203 12:00:30.205832 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/d7d3fec6-50e1-43f1-a7af-0adff12da7e9-public-tls-certs\") pod \"nova-api-0\" (UID: \"d7d3fec6-50e1-43f1-a7af-0adff12da7e9\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:00:30 crc kubenswrapper[5052]: I1203 12:00:30.205892 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-pqpqh\" (UniqueName: \"kubernetes.io/projected/d7d3fec6-50e1-43f1-a7af-0adff12da7e9-kube-api-access-pqpqh\") pod \"nova-api-0\" (UID: \"d7d3fec6-50e1-43f1-a7af-0adff12da7e9\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:00:30 crc kubenswrapper[5052]: I1203 12:00:30.205951 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/d7d3fec6-50e1-43f1-a7af-0adff12da7e9-config-data\") pod \"nova-api-0\" (UID: \"d7d3fec6-50e1-43f1-a7af-0adff12da7e9\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:00:30 crc kubenswrapper[5052]: I1203 12:00:30.205969 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d7d3fec6-50e1-43f1-a7af-0adff12da7e9-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"d7d3fec6-50e1-43f1-a7af-0adff12da7e9\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:00:30 crc kubenswrapper[5052]: I1203 12:00:30.206009 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/d7d3fec6-50e1-43f1-a7af-0adff12da7e9-logs\") pod \"nova-api-0\" (UID: \"d7d3fec6-50e1-43f1-a7af-0adff12da7e9\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:00:30 crc kubenswrapper[5052]: I1203 12:00:30.206452 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/d7d3fec6-50e1-43f1-a7af-0adff12da7e9-logs\") pod \"nova-api-0\" (UID: \"d7d3fec6-50e1-43f1-a7af-0adff12da7e9\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:00:30 crc kubenswrapper[5052]: I1203 12:00:30.210313 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/d7d3fec6-50e1-43f1-a7af-0adff12da7e9-internal-tls-certs\") pod \"nova-api-0\" (UID: \"d7d3fec6-50e1-43f1-a7af-0adff12da7e9\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:00:30 crc kubenswrapper[5052]: I1203 12:00:30.210462 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/d7d3fec6-50e1-43f1-a7af-0adff12da7e9-public-tls-certs\") pod \"nova-api-0\" (UID: \"d7d3fec6-50e1-43f1-a7af-0adff12da7e9\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:00:30 crc kubenswrapper[5052]: I1203 12:00:30.210607 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d7d3fec6-50e1-43f1-a7af-0adff12da7e9-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"d7d3fec6-50e1-43f1-a7af-0adff12da7e9\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:00:30 crc kubenswrapper[5052]: I1203 12:00:30.210957 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/d7d3fec6-50e1-43f1-a7af-0adff12da7e9-config-data\") pod \"nova-api-0\" (UID: \"d7d3fec6-50e1-43f1-a7af-0adff12da7e9\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:00:30 crc kubenswrapper[5052]: I1203 12:00:30.225282 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-pqpqh\" (UniqueName: \"kubernetes.io/projected/d7d3fec6-50e1-43f1-a7af-0adff12da7e9-kube-api-access-pqpqh\") pod \"nova-api-0\" (UID: \"d7d3fec6-50e1-43f1-a7af-0adff12da7e9\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:00:30 crc kubenswrapper[5052]: I1203 12:00:30.355828 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:00:30 crc kubenswrapper[5052]: I1203 12:00:30.853901 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:00:30 crc kubenswrapper[5052]: W1203 12:00:30.853992 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd7d3fec6_50e1_43f1_a7af_0adff12da7e9.slice/crio-4a46493492cb7f7a53a30e71b5f893e458ab8416cfd466b547793df14cdf5ff2 WatchSource:0}: Error finding container 4a46493492cb7f7a53a30e71b5f893e458ab8416cfd466b547793df14cdf5ff2: Status 404 returned error can't find the container with id 4a46493492cb7f7a53a30e71b5f893e458ab8416cfd466b547793df14cdf5ff2 Dec 03 12:00:30 crc kubenswrapper[5052]: I1203 12:00:30.968098 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:00:30 crc kubenswrapper[5052]: I1203 12:00:30.989682 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:00:31 crc kubenswrapper[5052]: I1203 12:00:31.685502 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eebaba62-8aa5-447a-bcc2-590bb40de88f" path="/var/lib/kubelet/pods/eebaba62-8aa5-447a-bcc2-590bb40de88f/volumes" Dec 03 12:00:31 crc kubenswrapper[5052]: I1203 12:00:31.703785 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"da4dfbea-20a2-4638-bbbe-ed4bfee0ad49","Type":"ContainerStarted","Data":"1658de9b62f64e764364ab2fb2ba523e5ca2b9210d45094169d4d8ec956ac253"} Dec 03 12:00:31 crc kubenswrapper[5052]: I1203 12:00:31.708524 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"d7d3fec6-50e1-43f1-a7af-0adff12da7e9","Type":"ContainerStarted","Data":"838e7d0ebd87ff7d17a6dd5cd30ef663084b15cce3ffd0de1f2432223a49a2fc"} Dec 03 12:00:31 crc kubenswrapper[5052]: I1203 12:00:31.708651 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"d7d3fec6-50e1-43f1-a7af-0adff12da7e9","Type":"ContainerStarted","Data":"1eed0cb7f6903cfa393e439901bd3eafddc510b9a3ec89193350cf065af37020"} Dec 03 12:00:31 crc kubenswrapper[5052]: I1203 12:00:31.709004 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"d7d3fec6-50e1-43f1-a7af-0adff12da7e9","Type":"ContainerStarted","Data":"4a46493492cb7f7a53a30e71b5f893e458ab8416cfd466b547793df14cdf5ff2"} Dec 03 12:00:31 crc kubenswrapper[5052]: I1203 12:00:31.728266 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:00:31 crc kubenswrapper[5052]: I1203 12:00:31.763232 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-api-0" podStartSLOduration=1.763210265 podStartE2EDuration="1.763210265s" podCreationTimestamp="2025-12-03 12:00:30 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:00:31.72734815 +0000 UTC m=+2422.315589904" watchObservedRunningTime="2025-12-03 12:00:31.763210265 +0000 UTC m=+2422.351452029" Dec 03 12:00:31 crc kubenswrapper[5052]: I1203 12:00:31.893578 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell1-cell-mapping-wsbvz"] Dec 03 12:00:31 crc kubenswrapper[5052]: I1203 12:00:31.895404 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-cell-mapping-wsbvz" Dec 03 12:00:31 crc kubenswrapper[5052]: I1203 12:00:31.898207 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell1-manage-config-data" Dec 03 12:00:31 crc kubenswrapper[5052]: I1203 12:00:31.898805 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell1-manage-scripts" Dec 03 12:00:31 crc kubenswrapper[5052]: I1203 12:00:31.900940 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-cell-mapping-wsbvz"] Dec 03 12:00:31 crc kubenswrapper[5052]: I1203 12:00:31.948913 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tmzgt\" (UniqueName: \"kubernetes.io/projected/158aaec8-feb0-4a6e-84ff-b137b74f0ef6-kube-api-access-tmzgt\") pod \"nova-cell1-cell-mapping-wsbvz\" (UID: \"158aaec8-feb0-4a6e-84ff-b137b74f0ef6\") " pod="openstack-kuttl-tests/nova-cell1-cell-mapping-wsbvz" Dec 03 12:00:31 crc kubenswrapper[5052]: I1203 12:00:31.949035 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/158aaec8-feb0-4a6e-84ff-b137b74f0ef6-combined-ca-bundle\") pod \"nova-cell1-cell-mapping-wsbvz\" (UID: \"158aaec8-feb0-4a6e-84ff-b137b74f0ef6\") " pod="openstack-kuttl-tests/nova-cell1-cell-mapping-wsbvz" Dec 03 12:00:31 crc kubenswrapper[5052]: I1203 12:00:31.949108 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/158aaec8-feb0-4a6e-84ff-b137b74f0ef6-scripts\") pod \"nova-cell1-cell-mapping-wsbvz\" (UID: \"158aaec8-feb0-4a6e-84ff-b137b74f0ef6\") " pod="openstack-kuttl-tests/nova-cell1-cell-mapping-wsbvz" Dec 03 12:00:31 crc kubenswrapper[5052]: I1203 12:00:31.949133 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/158aaec8-feb0-4a6e-84ff-b137b74f0ef6-config-data\") pod \"nova-cell1-cell-mapping-wsbvz\" (UID: \"158aaec8-feb0-4a6e-84ff-b137b74f0ef6\") " pod="openstack-kuttl-tests/nova-cell1-cell-mapping-wsbvz" Dec 03 12:00:32 crc kubenswrapper[5052]: I1203 12:00:32.050427 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/158aaec8-feb0-4a6e-84ff-b137b74f0ef6-scripts\") pod \"nova-cell1-cell-mapping-wsbvz\" (UID: \"158aaec8-feb0-4a6e-84ff-b137b74f0ef6\") " pod="openstack-kuttl-tests/nova-cell1-cell-mapping-wsbvz" Dec 03 12:00:32 crc kubenswrapper[5052]: I1203 12:00:32.050488 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/158aaec8-feb0-4a6e-84ff-b137b74f0ef6-config-data\") pod \"nova-cell1-cell-mapping-wsbvz\" (UID: \"158aaec8-feb0-4a6e-84ff-b137b74f0ef6\") " pod="openstack-kuttl-tests/nova-cell1-cell-mapping-wsbvz" Dec 03 12:00:32 crc kubenswrapper[5052]: I1203 12:00:32.050583 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-tmzgt\" (UniqueName: \"kubernetes.io/projected/158aaec8-feb0-4a6e-84ff-b137b74f0ef6-kube-api-access-tmzgt\") pod \"nova-cell1-cell-mapping-wsbvz\" (UID: \"158aaec8-feb0-4a6e-84ff-b137b74f0ef6\") " pod="openstack-kuttl-tests/nova-cell1-cell-mapping-wsbvz" Dec 03 12:00:32 crc kubenswrapper[5052]: I1203 12:00:32.050716 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/158aaec8-feb0-4a6e-84ff-b137b74f0ef6-combined-ca-bundle\") pod \"nova-cell1-cell-mapping-wsbvz\" (UID: \"158aaec8-feb0-4a6e-84ff-b137b74f0ef6\") " pod="openstack-kuttl-tests/nova-cell1-cell-mapping-wsbvz" Dec 03 12:00:32 crc kubenswrapper[5052]: I1203 12:00:32.055926 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/158aaec8-feb0-4a6e-84ff-b137b74f0ef6-scripts\") pod \"nova-cell1-cell-mapping-wsbvz\" (UID: \"158aaec8-feb0-4a6e-84ff-b137b74f0ef6\") " pod="openstack-kuttl-tests/nova-cell1-cell-mapping-wsbvz" Dec 03 12:00:32 crc kubenswrapper[5052]: I1203 12:00:32.057143 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/158aaec8-feb0-4a6e-84ff-b137b74f0ef6-combined-ca-bundle\") pod \"nova-cell1-cell-mapping-wsbvz\" (UID: \"158aaec8-feb0-4a6e-84ff-b137b74f0ef6\") " pod="openstack-kuttl-tests/nova-cell1-cell-mapping-wsbvz" Dec 03 12:00:32 crc kubenswrapper[5052]: I1203 12:00:32.059758 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/158aaec8-feb0-4a6e-84ff-b137b74f0ef6-config-data\") pod \"nova-cell1-cell-mapping-wsbvz\" (UID: \"158aaec8-feb0-4a6e-84ff-b137b74f0ef6\") " pod="openstack-kuttl-tests/nova-cell1-cell-mapping-wsbvz" Dec 03 12:00:32 crc kubenswrapper[5052]: I1203 12:00:32.077800 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-tmzgt\" (UniqueName: \"kubernetes.io/projected/158aaec8-feb0-4a6e-84ff-b137b74f0ef6-kube-api-access-tmzgt\") pod \"nova-cell1-cell-mapping-wsbvz\" (UID: \"158aaec8-feb0-4a6e-84ff-b137b74f0ef6\") " pod="openstack-kuttl-tests/nova-cell1-cell-mapping-wsbvz" Dec 03 12:00:32 crc kubenswrapper[5052]: I1203 12:00:32.216027 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-cell-mapping-wsbvz" Dec 03 12:00:32 crc kubenswrapper[5052]: I1203 12:00:32.674379 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-cell-mapping-wsbvz"] Dec 03 12:00:32 crc kubenswrapper[5052]: I1203 12:00:32.735000 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"da4dfbea-20a2-4638-bbbe-ed4bfee0ad49","Type":"ContainerStarted","Data":"ef9cd2fbef68f2e1fe845f78b334b0ff9a4fcb0c22c21ce8f7ab13794d397239"} Dec 03 12:00:32 crc kubenswrapper[5052]: I1203 12:00:32.735042 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"da4dfbea-20a2-4638-bbbe-ed4bfee0ad49","Type":"ContainerStarted","Data":"3f16de4b1d15886789bdc38843cbe1fb5e5180679e89cb7fb19837deadc091ae"} Dec 03 12:00:32 crc kubenswrapper[5052]: I1203 12:00:32.754484 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-cell-mapping-wsbvz" event={"ID":"158aaec8-feb0-4a6e-84ff-b137b74f0ef6","Type":"ContainerStarted","Data":"4b51c9acef67f528582ab01dc34e6ca398f05c994d8213abf64288617b9b752e"} Dec 03 12:00:33 crc kubenswrapper[5052]: I1203 12:00:33.787075 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-cell-mapping-wsbvz" event={"ID":"158aaec8-feb0-4a6e-84ff-b137b74f0ef6","Type":"ContainerStarted","Data":"39232820b6e0e754d5a5084f941da2ffd8fab9858a3574cd08f2a2ef9dce5a18"} Dec 03 12:00:33 crc kubenswrapper[5052]: I1203 12:00:33.806761 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-cell1-cell-mapping-wsbvz" podStartSLOduration=2.806736298 podStartE2EDuration="2.806736298s" podCreationTimestamp="2025-12-03 12:00:31 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:00:33.801474158 +0000 UTC m=+2424.389715932" watchObservedRunningTime="2025-12-03 12:00:33.806736298 +0000 UTC m=+2424.394978062" Dec 03 12:00:34 crc kubenswrapper[5052]: I1203 12:00:34.799325 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"da4dfbea-20a2-4638-bbbe-ed4bfee0ad49","Type":"ContainerStarted","Data":"903649a39fdac1f3614d17974bbdb299dee9a076158995226565f8b3dd246b89"} Dec 03 12:00:34 crc kubenswrapper[5052]: I1203 12:00:34.799640 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:00:34 crc kubenswrapper[5052]: I1203 12:00:34.831919 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/ceilometer-0" podStartSLOduration=2.485376313 podStartE2EDuration="6.831902687s" podCreationTimestamp="2025-12-03 12:00:28 +0000 UTC" firstStartedPulling="2025-12-03 12:00:29.515066672 +0000 UTC m=+2420.103308436" lastFinishedPulling="2025-12-03 12:00:33.861593046 +0000 UTC m=+2424.449834810" observedRunningTime="2025-12-03 12:00:34.817120045 +0000 UTC m=+2425.405361819" watchObservedRunningTime="2025-12-03 12:00:34.831902687 +0000 UTC m=+2425.420144441" Dec 03 12:00:36 crc kubenswrapper[5052]: I1203 12:00:36.672932 5052 scope.go:117] "RemoveContainer" containerID="d224717c6d9ca8135fd510e03a6c40c9f2b1a56dd8b4a3b685af3e2dfa855982" Dec 03 12:00:36 crc kubenswrapper[5052]: E1203 12:00:36.673392 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 12:00:38 crc kubenswrapper[5052]: I1203 12:00:38.836464 5052 generic.go:334] "Generic (PLEG): container finished" podID="158aaec8-feb0-4a6e-84ff-b137b74f0ef6" containerID="39232820b6e0e754d5a5084f941da2ffd8fab9858a3574cd08f2a2ef9dce5a18" exitCode=0 Dec 03 12:00:38 crc kubenswrapper[5052]: I1203 12:00:38.837317 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-cell-mapping-wsbvz" event={"ID":"158aaec8-feb0-4a6e-84ff-b137b74f0ef6","Type":"ContainerDied","Data":"39232820b6e0e754d5a5084f941da2ffd8fab9858a3574cd08f2a2ef9dce5a18"} Dec 03 12:00:40 crc kubenswrapper[5052]: I1203 12:00:40.246723 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-cell-mapping-wsbvz" Dec 03 12:00:40 crc kubenswrapper[5052]: I1203 12:00:40.356343 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:00:40 crc kubenswrapper[5052]: I1203 12:00:40.356417 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:00:40 crc kubenswrapper[5052]: I1203 12:00:40.424896 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tmzgt\" (UniqueName: \"kubernetes.io/projected/158aaec8-feb0-4a6e-84ff-b137b74f0ef6-kube-api-access-tmzgt\") pod \"158aaec8-feb0-4a6e-84ff-b137b74f0ef6\" (UID: \"158aaec8-feb0-4a6e-84ff-b137b74f0ef6\") " Dec 03 12:00:40 crc kubenswrapper[5052]: I1203 12:00:40.424985 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/158aaec8-feb0-4a6e-84ff-b137b74f0ef6-scripts\") pod \"158aaec8-feb0-4a6e-84ff-b137b74f0ef6\" (UID: \"158aaec8-feb0-4a6e-84ff-b137b74f0ef6\") " Dec 03 12:00:40 crc kubenswrapper[5052]: I1203 12:00:40.425063 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/158aaec8-feb0-4a6e-84ff-b137b74f0ef6-combined-ca-bundle\") pod \"158aaec8-feb0-4a6e-84ff-b137b74f0ef6\" (UID: \"158aaec8-feb0-4a6e-84ff-b137b74f0ef6\") " Dec 03 12:00:40 crc kubenswrapper[5052]: I1203 12:00:40.425118 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/158aaec8-feb0-4a6e-84ff-b137b74f0ef6-config-data\") pod \"158aaec8-feb0-4a6e-84ff-b137b74f0ef6\" (UID: \"158aaec8-feb0-4a6e-84ff-b137b74f0ef6\") " Dec 03 12:00:40 crc kubenswrapper[5052]: I1203 12:00:40.431216 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/158aaec8-feb0-4a6e-84ff-b137b74f0ef6-kube-api-access-tmzgt" (OuterVolumeSpecName: "kube-api-access-tmzgt") pod "158aaec8-feb0-4a6e-84ff-b137b74f0ef6" (UID: "158aaec8-feb0-4a6e-84ff-b137b74f0ef6"). InnerVolumeSpecName "kube-api-access-tmzgt". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:00:40 crc kubenswrapper[5052]: I1203 12:00:40.434629 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/158aaec8-feb0-4a6e-84ff-b137b74f0ef6-scripts" (OuterVolumeSpecName: "scripts") pod "158aaec8-feb0-4a6e-84ff-b137b74f0ef6" (UID: "158aaec8-feb0-4a6e-84ff-b137b74f0ef6"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:00:40 crc kubenswrapper[5052]: I1203 12:00:40.462764 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/158aaec8-feb0-4a6e-84ff-b137b74f0ef6-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "158aaec8-feb0-4a6e-84ff-b137b74f0ef6" (UID: "158aaec8-feb0-4a6e-84ff-b137b74f0ef6"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:00:40 crc kubenswrapper[5052]: I1203 12:00:40.463662 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/158aaec8-feb0-4a6e-84ff-b137b74f0ef6-config-data" (OuterVolumeSpecName: "config-data") pod "158aaec8-feb0-4a6e-84ff-b137b74f0ef6" (UID: "158aaec8-feb0-4a6e-84ff-b137b74f0ef6"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:00:40 crc kubenswrapper[5052]: I1203 12:00:40.528643 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-tmzgt\" (UniqueName: \"kubernetes.io/projected/158aaec8-feb0-4a6e-84ff-b137b74f0ef6-kube-api-access-tmzgt\") on node \"crc\" DevicePath \"\"" Dec 03 12:00:40 crc kubenswrapper[5052]: I1203 12:00:40.528670 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/158aaec8-feb0-4a6e-84ff-b137b74f0ef6-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:00:40 crc kubenswrapper[5052]: I1203 12:00:40.528679 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/158aaec8-feb0-4a6e-84ff-b137b74f0ef6-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:00:40 crc kubenswrapper[5052]: I1203 12:00:40.528689 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/158aaec8-feb0-4a6e-84ff-b137b74f0ef6-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:00:40 crc kubenswrapper[5052]: I1203 12:00:40.861105 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-cell-mapping-wsbvz" event={"ID":"158aaec8-feb0-4a6e-84ff-b137b74f0ef6","Type":"ContainerDied","Data":"4b51c9acef67f528582ab01dc34e6ca398f05c994d8213abf64288617b9b752e"} Dec 03 12:00:40 crc kubenswrapper[5052]: I1203 12:00:40.861148 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4b51c9acef67f528582ab01dc34e6ca398f05c994d8213abf64288617b9b752e" Dec 03 12:00:40 crc kubenswrapper[5052]: I1203 12:00:40.861199 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-cell-mapping-wsbvz" Dec 03 12:00:41 crc kubenswrapper[5052]: I1203 12:00:41.050132 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:00:41 crc kubenswrapper[5052]: I1203 12:00:41.050347 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-api-0" podUID="d7d3fec6-50e1-43f1-a7af-0adff12da7e9" containerName="nova-api-log" containerID="cri-o://1eed0cb7f6903cfa393e439901bd3eafddc510b9a3ec89193350cf065af37020" gracePeriod=30 Dec 03 12:00:41 crc kubenswrapper[5052]: I1203 12:00:41.050816 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-api-0" podUID="d7d3fec6-50e1-43f1-a7af-0adff12da7e9" containerName="nova-api-api" containerID="cri-o://838e7d0ebd87ff7d17a6dd5cd30ef663084b15cce3ffd0de1f2432223a49a2fc" gracePeriod=30 Dec 03 12:00:41 crc kubenswrapper[5052]: I1203 12:00:41.066560 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-api-0" podUID="d7d3fec6-50e1-43f1-a7af-0adff12da7e9" containerName="nova-api-log" probeResult="failure" output="Get \"https://10.217.1.252:8774/\": EOF" Dec 03 12:00:41 crc kubenswrapper[5052]: I1203 12:00:41.072915 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-api-0" podUID="d7d3fec6-50e1-43f1-a7af-0adff12da7e9" containerName="nova-api-api" probeResult="failure" output="Get \"https://10.217.1.252:8774/\": EOF" Dec 03 12:00:41 crc kubenswrapper[5052]: I1203 12:00:41.098619 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:00:41 crc kubenswrapper[5052]: I1203 12:00:41.099163 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-metadata-0" podUID="57aac579-21a0-4c70-937e-1f8cda2fdcf8" containerName="nova-metadata-log" containerID="cri-o://108b04271be86cc94040113d925492ae24fbcfeb479bddcda4881f05086c7288" gracePeriod=30 Dec 03 12:00:41 crc kubenswrapper[5052]: I1203 12:00:41.099437 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-metadata-0" podUID="57aac579-21a0-4c70-937e-1f8cda2fdcf8" containerName="nova-metadata-metadata" containerID="cri-o://57b8a2a187f58262450a502c6a73481dc9325be51ee5640a105953473b7e6d2d" gracePeriod=30 Dec 03 12:00:41 crc kubenswrapper[5052]: I1203 12:00:41.114043 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:00:41 crc kubenswrapper[5052]: I1203 12:00:41.114965 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-scheduler-0" podUID="25079157-2add-4870-a7a9-e002013bdec0" containerName="nova-scheduler-scheduler" containerID="cri-o://bd545383bb0ebe1c5a551c19ee05eb022d006fd7d6d08e21e1ca450105aa434e" gracePeriod=30 Dec 03 12:00:41 crc kubenswrapper[5052]: E1203 12:00:41.117882 5052 cadvisor_stats_provider.go:516] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod158aaec8_feb0_4a6e_84ff_b137b74f0ef6.slice\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod158aaec8_feb0_4a6e_84ff_b137b74f0ef6.slice/crio-4b51c9acef67f528582ab01dc34e6ca398f05c994d8213abf64288617b9b752e\": RecentStats: unable to find data in memory cache]" Dec 03 12:00:41 crc kubenswrapper[5052]: I1203 12:00:41.874772 5052 generic.go:334] "Generic (PLEG): container finished" podID="d7d3fec6-50e1-43f1-a7af-0adff12da7e9" containerID="1eed0cb7f6903cfa393e439901bd3eafddc510b9a3ec89193350cf065af37020" exitCode=143 Dec 03 12:00:41 crc kubenswrapper[5052]: I1203 12:00:41.874839 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"d7d3fec6-50e1-43f1-a7af-0adff12da7e9","Type":"ContainerDied","Data":"1eed0cb7f6903cfa393e439901bd3eafddc510b9a3ec89193350cf065af37020"} Dec 03 12:00:41 crc kubenswrapper[5052]: I1203 12:00:41.878403 5052 generic.go:334] "Generic (PLEG): container finished" podID="57aac579-21a0-4c70-937e-1f8cda2fdcf8" containerID="108b04271be86cc94040113d925492ae24fbcfeb479bddcda4881f05086c7288" exitCode=143 Dec 03 12:00:41 crc kubenswrapper[5052]: I1203 12:00:41.878461 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"57aac579-21a0-4c70-937e-1f8cda2fdcf8","Type":"ContainerDied","Data":"108b04271be86cc94040113d925492ae24fbcfeb479bddcda4881f05086c7288"} Dec 03 12:00:44 crc kubenswrapper[5052]: I1203 12:00:44.240051 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/nova-metadata-0" podUID="57aac579-21a0-4c70-937e-1f8cda2fdcf8" containerName="nova-metadata-metadata" probeResult="failure" output="Get \"https://10.217.1.244:8775/\": read tcp 10.217.0.2:57586->10.217.1.244:8775: read: connection reset by peer" Dec 03 12:00:44 crc kubenswrapper[5052]: I1203 12:00:44.240073 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/nova-metadata-0" podUID="57aac579-21a0-4c70-937e-1f8cda2fdcf8" containerName="nova-metadata-log" probeResult="failure" output="Get \"https://10.217.1.244:8775/\": read tcp 10.217.0.2:57590->10.217.1.244:8775: read: connection reset by peer" Dec 03 12:00:44 crc kubenswrapper[5052]: I1203 12:00:44.911344 5052 generic.go:334] "Generic (PLEG): container finished" podID="25079157-2add-4870-a7a9-e002013bdec0" containerID="bd545383bb0ebe1c5a551c19ee05eb022d006fd7d6d08e21e1ca450105aa434e" exitCode=0 Dec 03 12:00:44 crc kubenswrapper[5052]: I1203 12:00:44.911441 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"25079157-2add-4870-a7a9-e002013bdec0","Type":"ContainerDied","Data":"bd545383bb0ebe1c5a551c19ee05eb022d006fd7d6d08e21e1ca450105aa434e"} Dec 03 12:00:44 crc kubenswrapper[5052]: I1203 12:00:44.916046 5052 generic.go:334] "Generic (PLEG): container finished" podID="57aac579-21a0-4c70-937e-1f8cda2fdcf8" containerID="57b8a2a187f58262450a502c6a73481dc9325be51ee5640a105953473b7e6d2d" exitCode=0 Dec 03 12:00:44 crc kubenswrapper[5052]: I1203 12:00:44.916086 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"57aac579-21a0-4c70-937e-1f8cda2fdcf8","Type":"ContainerDied","Data":"57b8a2a187f58262450a502c6a73481dc9325be51ee5640a105953473b7e6d2d"} Dec 03 12:00:45 crc kubenswrapper[5052]: I1203 12:00:45.100166 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:00:45 crc kubenswrapper[5052]: I1203 12:00:45.166220 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:00:45 crc kubenswrapper[5052]: I1203 12:00:45.236799 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m465p\" (UniqueName: \"kubernetes.io/projected/25079157-2add-4870-a7a9-e002013bdec0-kube-api-access-m465p\") pod \"25079157-2add-4870-a7a9-e002013bdec0\" (UID: \"25079157-2add-4870-a7a9-e002013bdec0\") " Dec 03 12:00:45 crc kubenswrapper[5052]: I1203 12:00:45.237161 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/25079157-2add-4870-a7a9-e002013bdec0-combined-ca-bundle\") pod \"25079157-2add-4870-a7a9-e002013bdec0\" (UID: \"25079157-2add-4870-a7a9-e002013bdec0\") " Dec 03 12:00:45 crc kubenswrapper[5052]: I1203 12:00:45.237205 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/25079157-2add-4870-a7a9-e002013bdec0-config-data\") pod \"25079157-2add-4870-a7a9-e002013bdec0\" (UID: \"25079157-2add-4870-a7a9-e002013bdec0\") " Dec 03 12:00:45 crc kubenswrapper[5052]: I1203 12:00:45.242080 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/25079157-2add-4870-a7a9-e002013bdec0-kube-api-access-m465p" (OuterVolumeSpecName: "kube-api-access-m465p") pod "25079157-2add-4870-a7a9-e002013bdec0" (UID: "25079157-2add-4870-a7a9-e002013bdec0"). InnerVolumeSpecName "kube-api-access-m465p". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:00:45 crc kubenswrapper[5052]: I1203 12:00:45.266660 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/25079157-2add-4870-a7a9-e002013bdec0-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "25079157-2add-4870-a7a9-e002013bdec0" (UID: "25079157-2add-4870-a7a9-e002013bdec0"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:00:45 crc kubenswrapper[5052]: I1203 12:00:45.285791 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/25079157-2add-4870-a7a9-e002013bdec0-config-data" (OuterVolumeSpecName: "config-data") pod "25079157-2add-4870-a7a9-e002013bdec0" (UID: "25079157-2add-4870-a7a9-e002013bdec0"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:00:45 crc kubenswrapper[5052]: I1203 12:00:45.339157 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/57aac579-21a0-4c70-937e-1f8cda2fdcf8-logs\") pod \"57aac579-21a0-4c70-937e-1f8cda2fdcf8\" (UID: \"57aac579-21a0-4c70-937e-1f8cda2fdcf8\") " Dec 03 12:00:45 crc kubenswrapper[5052]: I1203 12:00:45.339255 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jmpmp\" (UniqueName: \"kubernetes.io/projected/57aac579-21a0-4c70-937e-1f8cda2fdcf8-kube-api-access-jmpmp\") pod \"57aac579-21a0-4c70-937e-1f8cda2fdcf8\" (UID: \"57aac579-21a0-4c70-937e-1f8cda2fdcf8\") " Dec 03 12:00:45 crc kubenswrapper[5052]: I1203 12:00:45.339377 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/57aac579-21a0-4c70-937e-1f8cda2fdcf8-combined-ca-bundle\") pod \"57aac579-21a0-4c70-937e-1f8cda2fdcf8\" (UID: \"57aac579-21a0-4c70-937e-1f8cda2fdcf8\") " Dec 03 12:00:45 crc kubenswrapper[5052]: I1203 12:00:45.339417 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/57aac579-21a0-4c70-937e-1f8cda2fdcf8-nova-metadata-tls-certs\") pod \"57aac579-21a0-4c70-937e-1f8cda2fdcf8\" (UID: \"57aac579-21a0-4c70-937e-1f8cda2fdcf8\") " Dec 03 12:00:45 crc kubenswrapper[5052]: I1203 12:00:45.339459 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/57aac579-21a0-4c70-937e-1f8cda2fdcf8-config-data\") pod \"57aac579-21a0-4c70-937e-1f8cda2fdcf8\" (UID: \"57aac579-21a0-4c70-937e-1f8cda2fdcf8\") " Dec 03 12:00:45 crc kubenswrapper[5052]: I1203 12:00:45.339995 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-m465p\" (UniqueName: \"kubernetes.io/projected/25079157-2add-4870-a7a9-e002013bdec0-kube-api-access-m465p\") on node \"crc\" DevicePath \"\"" Dec 03 12:00:45 crc kubenswrapper[5052]: I1203 12:00:45.340021 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/25079157-2add-4870-a7a9-e002013bdec0-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:00:45 crc kubenswrapper[5052]: I1203 12:00:45.340035 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/25079157-2add-4870-a7a9-e002013bdec0-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:00:45 crc kubenswrapper[5052]: I1203 12:00:45.340171 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/57aac579-21a0-4c70-937e-1f8cda2fdcf8-logs" (OuterVolumeSpecName: "logs") pod "57aac579-21a0-4c70-937e-1f8cda2fdcf8" (UID: "57aac579-21a0-4c70-937e-1f8cda2fdcf8"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:00:45 crc kubenswrapper[5052]: I1203 12:00:45.342775 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/57aac579-21a0-4c70-937e-1f8cda2fdcf8-kube-api-access-jmpmp" (OuterVolumeSpecName: "kube-api-access-jmpmp") pod "57aac579-21a0-4c70-937e-1f8cda2fdcf8" (UID: "57aac579-21a0-4c70-937e-1f8cda2fdcf8"). InnerVolumeSpecName "kube-api-access-jmpmp". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:00:45 crc kubenswrapper[5052]: I1203 12:00:45.367102 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/57aac579-21a0-4c70-937e-1f8cda2fdcf8-config-data" (OuterVolumeSpecName: "config-data") pod "57aac579-21a0-4c70-937e-1f8cda2fdcf8" (UID: "57aac579-21a0-4c70-937e-1f8cda2fdcf8"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:00:45 crc kubenswrapper[5052]: I1203 12:00:45.369835 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/57aac579-21a0-4c70-937e-1f8cda2fdcf8-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "57aac579-21a0-4c70-937e-1f8cda2fdcf8" (UID: "57aac579-21a0-4c70-937e-1f8cda2fdcf8"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:00:45 crc kubenswrapper[5052]: I1203 12:00:45.400822 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/57aac579-21a0-4c70-937e-1f8cda2fdcf8-nova-metadata-tls-certs" (OuterVolumeSpecName: "nova-metadata-tls-certs") pod "57aac579-21a0-4c70-937e-1f8cda2fdcf8" (UID: "57aac579-21a0-4c70-937e-1f8cda2fdcf8"). InnerVolumeSpecName "nova-metadata-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:00:45 crc kubenswrapper[5052]: I1203 12:00:45.442098 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/57aac579-21a0-4c70-937e-1f8cda2fdcf8-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:00:45 crc kubenswrapper[5052]: I1203 12:00:45.442128 5052 reconciler_common.go:293] "Volume detached for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/57aac579-21a0-4c70-937e-1f8cda2fdcf8-nova-metadata-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:00:45 crc kubenswrapper[5052]: I1203 12:00:45.442138 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/57aac579-21a0-4c70-937e-1f8cda2fdcf8-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:00:45 crc kubenswrapper[5052]: I1203 12:00:45.442148 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/57aac579-21a0-4c70-937e-1f8cda2fdcf8-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:00:45 crc kubenswrapper[5052]: I1203 12:00:45.442157 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-jmpmp\" (UniqueName: \"kubernetes.io/projected/57aac579-21a0-4c70-937e-1f8cda2fdcf8-kube-api-access-jmpmp\") on node \"crc\" DevicePath \"\"" Dec 03 12:00:45 crc kubenswrapper[5052]: I1203 12:00:45.926294 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"25079157-2add-4870-a7a9-e002013bdec0","Type":"ContainerDied","Data":"c2989297dc81507d947678a3fff0076b7fe50fed3d9cb397e30e5bf635771739"} Dec 03 12:00:45 crc kubenswrapper[5052]: I1203 12:00:45.926329 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:00:45 crc kubenswrapper[5052]: I1203 12:00:45.926347 5052 scope.go:117] "RemoveContainer" containerID="bd545383bb0ebe1c5a551c19ee05eb022d006fd7d6d08e21e1ca450105aa434e" Dec 03 12:00:45 crc kubenswrapper[5052]: I1203 12:00:45.928644 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"57aac579-21a0-4c70-937e-1f8cda2fdcf8","Type":"ContainerDied","Data":"6a62f38d2d9751d6ac145d3ea02a1610e3fb7fcd4e7e20de15f29e60aa3d5945"} Dec 03 12:00:45 crc kubenswrapper[5052]: I1203 12:00:45.928688 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:00:45 crc kubenswrapper[5052]: I1203 12:00:45.960847 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:00:45 crc kubenswrapper[5052]: I1203 12:00:45.969328 5052 scope.go:117] "RemoveContainer" containerID="57b8a2a187f58262450a502c6a73481dc9325be51ee5640a105953473b7e6d2d" Dec 03 12:00:45 crc kubenswrapper[5052]: I1203 12:00:45.972925 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:00:45 crc kubenswrapper[5052]: I1203 12:00:45.983244 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:00:45 crc kubenswrapper[5052]: I1203 12:00:45.996519 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:00:46 crc kubenswrapper[5052]: I1203 12:00:46.003525 5052 scope.go:117] "RemoveContainer" containerID="108b04271be86cc94040113d925492ae24fbcfeb479bddcda4881f05086c7288" Dec 03 12:00:46 crc kubenswrapper[5052]: I1203 12:00:46.009061 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:00:46 crc kubenswrapper[5052]: E1203 12:00:46.009579 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="25079157-2add-4870-a7a9-e002013bdec0" containerName="nova-scheduler-scheduler" Dec 03 12:00:46 crc kubenswrapper[5052]: I1203 12:00:46.009601 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="25079157-2add-4870-a7a9-e002013bdec0" containerName="nova-scheduler-scheduler" Dec 03 12:00:46 crc kubenswrapper[5052]: E1203 12:00:46.009622 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="57aac579-21a0-4c70-937e-1f8cda2fdcf8" containerName="nova-metadata-log" Dec 03 12:00:46 crc kubenswrapper[5052]: I1203 12:00:46.009631 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="57aac579-21a0-4c70-937e-1f8cda2fdcf8" containerName="nova-metadata-log" Dec 03 12:00:46 crc kubenswrapper[5052]: E1203 12:00:46.009652 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="158aaec8-feb0-4a6e-84ff-b137b74f0ef6" containerName="nova-manage" Dec 03 12:00:46 crc kubenswrapper[5052]: I1203 12:00:46.009660 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="158aaec8-feb0-4a6e-84ff-b137b74f0ef6" containerName="nova-manage" Dec 03 12:00:46 crc kubenswrapper[5052]: E1203 12:00:46.009684 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="57aac579-21a0-4c70-937e-1f8cda2fdcf8" containerName="nova-metadata-metadata" Dec 03 12:00:46 crc kubenswrapper[5052]: I1203 12:00:46.009691 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="57aac579-21a0-4c70-937e-1f8cda2fdcf8" containerName="nova-metadata-metadata" Dec 03 12:00:46 crc kubenswrapper[5052]: I1203 12:00:46.009917 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="158aaec8-feb0-4a6e-84ff-b137b74f0ef6" containerName="nova-manage" Dec 03 12:00:46 crc kubenswrapper[5052]: I1203 12:00:46.009955 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="57aac579-21a0-4c70-937e-1f8cda2fdcf8" containerName="nova-metadata-metadata" Dec 03 12:00:46 crc kubenswrapper[5052]: I1203 12:00:46.009966 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="57aac579-21a0-4c70-937e-1f8cda2fdcf8" containerName="nova-metadata-log" Dec 03 12:00:46 crc kubenswrapper[5052]: I1203 12:00:46.009980 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="25079157-2add-4870-a7a9-e002013bdec0" containerName="nova-scheduler-scheduler" Dec 03 12:00:46 crc kubenswrapper[5052]: I1203 12:00:46.010810 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:00:46 crc kubenswrapper[5052]: I1203 12:00:46.015607 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-scheduler-config-data" Dec 03 12:00:46 crc kubenswrapper[5052]: I1203 12:00:46.019035 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:00:46 crc kubenswrapper[5052]: I1203 12:00:46.028312 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:00:46 crc kubenswrapper[5052]: I1203 12:00:46.032420 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:00:46 crc kubenswrapper[5052]: I1203 12:00:46.036182 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-metadata-config-data" Dec 03 12:00:46 crc kubenswrapper[5052]: I1203 12:00:46.036559 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-nova-metadata-internal-svc" Dec 03 12:00:46 crc kubenswrapper[5052]: I1203 12:00:46.036693 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:00:46 crc kubenswrapper[5052]: I1203 12:00:46.153287 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/98d3c14e-7549-441a-9532-f3d75833cdf8-nova-metadata-tls-certs\") pod \"nova-metadata-0\" (UID: \"98d3c14e-7549-441a-9532-f3d75833cdf8\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:00:46 crc kubenswrapper[5052]: I1203 12:00:46.153376 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/98d3c14e-7549-441a-9532-f3d75833cdf8-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"98d3c14e-7549-441a-9532-f3d75833cdf8\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:00:46 crc kubenswrapper[5052]: I1203 12:00:46.153438 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/98d3c14e-7549-441a-9532-f3d75833cdf8-config-data\") pod \"nova-metadata-0\" (UID: \"98d3c14e-7549-441a-9532-f3d75833cdf8\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:00:46 crc kubenswrapper[5052]: I1203 12:00:46.153463 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/73c1b1fb-5062-4c33-be17-72324f222396-config-data\") pod \"nova-scheduler-0\" (UID: \"73c1b1fb-5062-4c33-be17-72324f222396\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:00:46 crc kubenswrapper[5052]: I1203 12:00:46.153490 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/73c1b1fb-5062-4c33-be17-72324f222396-combined-ca-bundle\") pod \"nova-scheduler-0\" (UID: \"73c1b1fb-5062-4c33-be17-72324f222396\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:00:46 crc kubenswrapper[5052]: I1203 12:00:46.153538 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h786s\" (UniqueName: \"kubernetes.io/projected/98d3c14e-7549-441a-9532-f3d75833cdf8-kube-api-access-h786s\") pod \"nova-metadata-0\" (UID: \"98d3c14e-7549-441a-9532-f3d75833cdf8\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:00:46 crc kubenswrapper[5052]: I1203 12:00:46.153678 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r6pw2\" (UniqueName: \"kubernetes.io/projected/73c1b1fb-5062-4c33-be17-72324f222396-kube-api-access-r6pw2\") pod \"nova-scheduler-0\" (UID: \"73c1b1fb-5062-4c33-be17-72324f222396\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:00:46 crc kubenswrapper[5052]: I1203 12:00:46.153749 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/98d3c14e-7549-441a-9532-f3d75833cdf8-logs\") pod \"nova-metadata-0\" (UID: \"98d3c14e-7549-441a-9532-f3d75833cdf8\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:00:46 crc kubenswrapper[5052]: I1203 12:00:46.255148 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/98d3c14e-7549-441a-9532-f3d75833cdf8-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"98d3c14e-7549-441a-9532-f3d75833cdf8\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:00:46 crc kubenswrapper[5052]: I1203 12:00:46.255248 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/98d3c14e-7549-441a-9532-f3d75833cdf8-config-data\") pod \"nova-metadata-0\" (UID: \"98d3c14e-7549-441a-9532-f3d75833cdf8\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:00:46 crc kubenswrapper[5052]: I1203 12:00:46.255283 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/73c1b1fb-5062-4c33-be17-72324f222396-config-data\") pod \"nova-scheduler-0\" (UID: \"73c1b1fb-5062-4c33-be17-72324f222396\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:00:46 crc kubenswrapper[5052]: I1203 12:00:46.255324 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/73c1b1fb-5062-4c33-be17-72324f222396-combined-ca-bundle\") pod \"nova-scheduler-0\" (UID: \"73c1b1fb-5062-4c33-be17-72324f222396\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:00:46 crc kubenswrapper[5052]: I1203 12:00:46.255393 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-h786s\" (UniqueName: \"kubernetes.io/projected/98d3c14e-7549-441a-9532-f3d75833cdf8-kube-api-access-h786s\") pod \"nova-metadata-0\" (UID: \"98d3c14e-7549-441a-9532-f3d75833cdf8\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:00:46 crc kubenswrapper[5052]: I1203 12:00:46.255474 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-r6pw2\" (UniqueName: \"kubernetes.io/projected/73c1b1fb-5062-4c33-be17-72324f222396-kube-api-access-r6pw2\") pod \"nova-scheduler-0\" (UID: \"73c1b1fb-5062-4c33-be17-72324f222396\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:00:46 crc kubenswrapper[5052]: I1203 12:00:46.255559 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/98d3c14e-7549-441a-9532-f3d75833cdf8-logs\") pod \"nova-metadata-0\" (UID: \"98d3c14e-7549-441a-9532-f3d75833cdf8\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:00:46 crc kubenswrapper[5052]: I1203 12:00:46.255618 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/98d3c14e-7549-441a-9532-f3d75833cdf8-nova-metadata-tls-certs\") pod \"nova-metadata-0\" (UID: \"98d3c14e-7549-441a-9532-f3d75833cdf8\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:00:46 crc kubenswrapper[5052]: I1203 12:00:46.256565 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/98d3c14e-7549-441a-9532-f3d75833cdf8-logs\") pod \"nova-metadata-0\" (UID: \"98d3c14e-7549-441a-9532-f3d75833cdf8\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:00:46 crc kubenswrapper[5052]: I1203 12:00:46.260452 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/98d3c14e-7549-441a-9532-f3d75833cdf8-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"98d3c14e-7549-441a-9532-f3d75833cdf8\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:00:46 crc kubenswrapper[5052]: I1203 12:00:46.262224 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/73c1b1fb-5062-4c33-be17-72324f222396-config-data\") pod \"nova-scheduler-0\" (UID: \"73c1b1fb-5062-4c33-be17-72324f222396\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:00:46 crc kubenswrapper[5052]: I1203 12:00:46.262356 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/98d3c14e-7549-441a-9532-f3d75833cdf8-config-data\") pod \"nova-metadata-0\" (UID: \"98d3c14e-7549-441a-9532-f3d75833cdf8\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:00:46 crc kubenswrapper[5052]: I1203 12:00:46.262560 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/98d3c14e-7549-441a-9532-f3d75833cdf8-nova-metadata-tls-certs\") pod \"nova-metadata-0\" (UID: \"98d3c14e-7549-441a-9532-f3d75833cdf8\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:00:46 crc kubenswrapper[5052]: I1203 12:00:46.264760 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/73c1b1fb-5062-4c33-be17-72324f222396-combined-ca-bundle\") pod \"nova-scheduler-0\" (UID: \"73c1b1fb-5062-4c33-be17-72324f222396\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:00:46 crc kubenswrapper[5052]: I1203 12:00:46.274404 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-r6pw2\" (UniqueName: \"kubernetes.io/projected/73c1b1fb-5062-4c33-be17-72324f222396-kube-api-access-r6pw2\") pod \"nova-scheduler-0\" (UID: \"73c1b1fb-5062-4c33-be17-72324f222396\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:00:46 crc kubenswrapper[5052]: I1203 12:00:46.276286 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-h786s\" (UniqueName: \"kubernetes.io/projected/98d3c14e-7549-441a-9532-f3d75833cdf8-kube-api-access-h786s\") pod \"nova-metadata-0\" (UID: \"98d3c14e-7549-441a-9532-f3d75833cdf8\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:00:46 crc kubenswrapper[5052]: I1203 12:00:46.338725 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:00:46 crc kubenswrapper[5052]: I1203 12:00:46.353358 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:00:46 crc kubenswrapper[5052]: I1203 12:00:46.800895 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:00:46 crc kubenswrapper[5052]: I1203 12:00:46.882198 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:00:46 crc kubenswrapper[5052]: W1203 12:00:46.884592 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod98d3c14e_7549_441a_9532_f3d75833cdf8.slice/crio-fd4e9a275e8eb105d9f1d75c42811cdf01c0b028b69193cbfde45831f5ab9c5a WatchSource:0}: Error finding container fd4e9a275e8eb105d9f1d75c42811cdf01c0b028b69193cbfde45831f5ab9c5a: Status 404 returned error can't find the container with id fd4e9a275e8eb105d9f1d75c42811cdf01c0b028b69193cbfde45831f5ab9c5a Dec 03 12:00:46 crc kubenswrapper[5052]: I1203 12:00:46.892745 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:00:46 crc kubenswrapper[5052]: I1203 12:00:46.947070 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"98d3c14e-7549-441a-9532-f3d75833cdf8","Type":"ContainerStarted","Data":"fd4e9a275e8eb105d9f1d75c42811cdf01c0b028b69193cbfde45831f5ab9c5a"} Dec 03 12:00:46 crc kubenswrapper[5052]: I1203 12:00:46.950645 5052 generic.go:334] "Generic (PLEG): container finished" podID="d7d3fec6-50e1-43f1-a7af-0adff12da7e9" containerID="838e7d0ebd87ff7d17a6dd5cd30ef663084b15cce3ffd0de1f2432223a49a2fc" exitCode=0 Dec 03 12:00:46 crc kubenswrapper[5052]: I1203 12:00:46.950690 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:00:46 crc kubenswrapper[5052]: I1203 12:00:46.950747 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"d7d3fec6-50e1-43f1-a7af-0adff12da7e9","Type":"ContainerDied","Data":"838e7d0ebd87ff7d17a6dd5cd30ef663084b15cce3ffd0de1f2432223a49a2fc"} Dec 03 12:00:46 crc kubenswrapper[5052]: I1203 12:00:46.950783 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"d7d3fec6-50e1-43f1-a7af-0adff12da7e9","Type":"ContainerDied","Data":"4a46493492cb7f7a53a30e71b5f893e458ab8416cfd466b547793df14cdf5ff2"} Dec 03 12:00:46 crc kubenswrapper[5052]: I1203 12:00:46.950801 5052 scope.go:117] "RemoveContainer" containerID="838e7d0ebd87ff7d17a6dd5cd30ef663084b15cce3ffd0de1f2432223a49a2fc" Dec 03 12:00:46 crc kubenswrapper[5052]: I1203 12:00:46.955056 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"73c1b1fb-5062-4c33-be17-72324f222396","Type":"ContainerStarted","Data":"b6417a09a2777892975f9717fe655663cdf13e6c02a46df1a2143449ae46e532"} Dec 03 12:00:47 crc kubenswrapper[5052]: I1203 12:00:47.014938 5052 scope.go:117] "RemoveContainer" containerID="1eed0cb7f6903cfa393e439901bd3eafddc510b9a3ec89193350cf065af37020" Dec 03 12:00:47 crc kubenswrapper[5052]: I1203 12:00:47.045331 5052 scope.go:117] "RemoveContainer" containerID="838e7d0ebd87ff7d17a6dd5cd30ef663084b15cce3ffd0de1f2432223a49a2fc" Dec 03 12:00:47 crc kubenswrapper[5052]: E1203 12:00:47.046001 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"838e7d0ebd87ff7d17a6dd5cd30ef663084b15cce3ffd0de1f2432223a49a2fc\": container with ID starting with 838e7d0ebd87ff7d17a6dd5cd30ef663084b15cce3ffd0de1f2432223a49a2fc not found: ID does not exist" containerID="838e7d0ebd87ff7d17a6dd5cd30ef663084b15cce3ffd0de1f2432223a49a2fc" Dec 03 12:00:47 crc kubenswrapper[5052]: I1203 12:00:47.046073 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"838e7d0ebd87ff7d17a6dd5cd30ef663084b15cce3ffd0de1f2432223a49a2fc"} err="failed to get container status \"838e7d0ebd87ff7d17a6dd5cd30ef663084b15cce3ffd0de1f2432223a49a2fc\": rpc error: code = NotFound desc = could not find container \"838e7d0ebd87ff7d17a6dd5cd30ef663084b15cce3ffd0de1f2432223a49a2fc\": container with ID starting with 838e7d0ebd87ff7d17a6dd5cd30ef663084b15cce3ffd0de1f2432223a49a2fc not found: ID does not exist" Dec 03 12:00:47 crc kubenswrapper[5052]: I1203 12:00:47.046124 5052 scope.go:117] "RemoveContainer" containerID="1eed0cb7f6903cfa393e439901bd3eafddc510b9a3ec89193350cf065af37020" Dec 03 12:00:47 crc kubenswrapper[5052]: E1203 12:00:47.046604 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1eed0cb7f6903cfa393e439901bd3eafddc510b9a3ec89193350cf065af37020\": container with ID starting with 1eed0cb7f6903cfa393e439901bd3eafddc510b9a3ec89193350cf065af37020 not found: ID does not exist" containerID="1eed0cb7f6903cfa393e439901bd3eafddc510b9a3ec89193350cf065af37020" Dec 03 12:00:47 crc kubenswrapper[5052]: I1203 12:00:47.046670 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1eed0cb7f6903cfa393e439901bd3eafddc510b9a3ec89193350cf065af37020"} err="failed to get container status \"1eed0cb7f6903cfa393e439901bd3eafddc510b9a3ec89193350cf065af37020\": rpc error: code = NotFound desc = could not find container \"1eed0cb7f6903cfa393e439901bd3eafddc510b9a3ec89193350cf065af37020\": container with ID starting with 1eed0cb7f6903cfa393e439901bd3eafddc510b9a3ec89193350cf065af37020 not found: ID does not exist" Dec 03 12:00:47 crc kubenswrapper[5052]: I1203 12:00:47.072713 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pqpqh\" (UniqueName: \"kubernetes.io/projected/d7d3fec6-50e1-43f1-a7af-0adff12da7e9-kube-api-access-pqpqh\") pod \"d7d3fec6-50e1-43f1-a7af-0adff12da7e9\" (UID: \"d7d3fec6-50e1-43f1-a7af-0adff12da7e9\") " Dec 03 12:00:47 crc kubenswrapper[5052]: I1203 12:00:47.072873 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/d7d3fec6-50e1-43f1-a7af-0adff12da7e9-logs\") pod \"d7d3fec6-50e1-43f1-a7af-0adff12da7e9\" (UID: \"d7d3fec6-50e1-43f1-a7af-0adff12da7e9\") " Dec 03 12:00:47 crc kubenswrapper[5052]: I1203 12:00:47.072925 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d7d3fec6-50e1-43f1-a7af-0adff12da7e9-combined-ca-bundle\") pod \"d7d3fec6-50e1-43f1-a7af-0adff12da7e9\" (UID: \"d7d3fec6-50e1-43f1-a7af-0adff12da7e9\") " Dec 03 12:00:47 crc kubenswrapper[5052]: I1203 12:00:47.072974 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/d7d3fec6-50e1-43f1-a7af-0adff12da7e9-internal-tls-certs\") pod \"d7d3fec6-50e1-43f1-a7af-0adff12da7e9\" (UID: \"d7d3fec6-50e1-43f1-a7af-0adff12da7e9\") " Dec 03 12:00:47 crc kubenswrapper[5052]: I1203 12:00:47.073030 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/d7d3fec6-50e1-43f1-a7af-0adff12da7e9-config-data\") pod \"d7d3fec6-50e1-43f1-a7af-0adff12da7e9\" (UID: \"d7d3fec6-50e1-43f1-a7af-0adff12da7e9\") " Dec 03 12:00:47 crc kubenswrapper[5052]: I1203 12:00:47.073086 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/d7d3fec6-50e1-43f1-a7af-0adff12da7e9-public-tls-certs\") pod \"d7d3fec6-50e1-43f1-a7af-0adff12da7e9\" (UID: \"d7d3fec6-50e1-43f1-a7af-0adff12da7e9\") " Dec 03 12:00:47 crc kubenswrapper[5052]: I1203 12:00:47.073703 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7d3fec6-50e1-43f1-a7af-0adff12da7e9-logs" (OuterVolumeSpecName: "logs") pod "d7d3fec6-50e1-43f1-a7af-0adff12da7e9" (UID: "d7d3fec6-50e1-43f1-a7af-0adff12da7e9"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:00:47 crc kubenswrapper[5052]: I1203 12:00:47.078333 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d7d3fec6-50e1-43f1-a7af-0adff12da7e9-kube-api-access-pqpqh" (OuterVolumeSpecName: "kube-api-access-pqpqh") pod "d7d3fec6-50e1-43f1-a7af-0adff12da7e9" (UID: "d7d3fec6-50e1-43f1-a7af-0adff12da7e9"). InnerVolumeSpecName "kube-api-access-pqpqh". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:00:47 crc kubenswrapper[5052]: I1203 12:00:47.104354 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d7d3fec6-50e1-43f1-a7af-0adff12da7e9-config-data" (OuterVolumeSpecName: "config-data") pod "d7d3fec6-50e1-43f1-a7af-0adff12da7e9" (UID: "d7d3fec6-50e1-43f1-a7af-0adff12da7e9"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:00:47 crc kubenswrapper[5052]: I1203 12:00:47.105257 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d7d3fec6-50e1-43f1-a7af-0adff12da7e9-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "d7d3fec6-50e1-43f1-a7af-0adff12da7e9" (UID: "d7d3fec6-50e1-43f1-a7af-0adff12da7e9"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:00:47 crc kubenswrapper[5052]: E1203 12:00:47.129306 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d7d3fec6-50e1-43f1-a7af-0adff12da7e9-internal-tls-certs podName:d7d3fec6-50e1-43f1-a7af-0adff12da7e9 nodeName:}" failed. No retries permitted until 2025-12-03 12:00:47.629274901 +0000 UTC m=+2438.217516675 (durationBeforeRetry 500ms). Error: error cleaning subPath mounts for volume "internal-tls-certs" (UniqueName: "kubernetes.io/secret/d7d3fec6-50e1-43f1-a7af-0adff12da7e9-internal-tls-certs") pod "d7d3fec6-50e1-43f1-a7af-0adff12da7e9" (UID: "d7d3fec6-50e1-43f1-a7af-0adff12da7e9") : error deleting /var/lib/kubelet/pods/d7d3fec6-50e1-43f1-a7af-0adff12da7e9/volume-subpaths: remove /var/lib/kubelet/pods/d7d3fec6-50e1-43f1-a7af-0adff12da7e9/volume-subpaths: no such file or directory Dec 03 12:00:47 crc kubenswrapper[5052]: I1203 12:00:47.132445 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d7d3fec6-50e1-43f1-a7af-0adff12da7e9-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "d7d3fec6-50e1-43f1-a7af-0adff12da7e9" (UID: "d7d3fec6-50e1-43f1-a7af-0adff12da7e9"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:00:47 crc kubenswrapper[5052]: I1203 12:00:47.175578 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/d7d3fec6-50e1-43f1-a7af-0adff12da7e9-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:00:47 crc kubenswrapper[5052]: I1203 12:00:47.175656 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d7d3fec6-50e1-43f1-a7af-0adff12da7e9-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:00:47 crc kubenswrapper[5052]: I1203 12:00:47.175674 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/d7d3fec6-50e1-43f1-a7af-0adff12da7e9-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:00:47 crc kubenswrapper[5052]: I1203 12:00:47.175686 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/d7d3fec6-50e1-43f1-a7af-0adff12da7e9-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:00:47 crc kubenswrapper[5052]: I1203 12:00:47.175725 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-pqpqh\" (UniqueName: \"kubernetes.io/projected/d7d3fec6-50e1-43f1-a7af-0adff12da7e9-kube-api-access-pqpqh\") on node \"crc\" DevicePath \"\"" Dec 03 12:00:47 crc kubenswrapper[5052]: I1203 12:00:47.683810 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/d7d3fec6-50e1-43f1-a7af-0adff12da7e9-internal-tls-certs\") pod \"d7d3fec6-50e1-43f1-a7af-0adff12da7e9\" (UID: \"d7d3fec6-50e1-43f1-a7af-0adff12da7e9\") " Dec 03 12:00:47 crc kubenswrapper[5052]: I1203 12:00:47.684432 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="25079157-2add-4870-a7a9-e002013bdec0" path="/var/lib/kubelet/pods/25079157-2add-4870-a7a9-e002013bdec0/volumes" Dec 03 12:00:47 crc kubenswrapper[5052]: I1203 12:00:47.686810 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="57aac579-21a0-4c70-937e-1f8cda2fdcf8" path="/var/lib/kubelet/pods/57aac579-21a0-4c70-937e-1f8cda2fdcf8/volumes" Dec 03 12:00:47 crc kubenswrapper[5052]: I1203 12:00:47.687175 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d7d3fec6-50e1-43f1-a7af-0adff12da7e9-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "d7d3fec6-50e1-43f1-a7af-0adff12da7e9" (UID: "d7d3fec6-50e1-43f1-a7af-0adff12da7e9"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:00:47 crc kubenswrapper[5052]: I1203 12:00:47.787694 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/d7d3fec6-50e1-43f1-a7af-0adff12da7e9-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:00:47 crc kubenswrapper[5052]: I1203 12:00:47.879929 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:00:47 crc kubenswrapper[5052]: I1203 12:00:47.896611 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:00:47 crc kubenswrapper[5052]: I1203 12:00:47.917817 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:00:47 crc kubenswrapper[5052]: E1203 12:00:47.918374 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="d7d3fec6-50e1-43f1-a7af-0adff12da7e9" containerName="nova-api-api" Dec 03 12:00:47 crc kubenswrapper[5052]: I1203 12:00:47.918393 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="d7d3fec6-50e1-43f1-a7af-0adff12da7e9" containerName="nova-api-api" Dec 03 12:00:47 crc kubenswrapper[5052]: E1203 12:00:47.918442 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="d7d3fec6-50e1-43f1-a7af-0adff12da7e9" containerName="nova-api-log" Dec 03 12:00:47 crc kubenswrapper[5052]: I1203 12:00:47.918450 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="d7d3fec6-50e1-43f1-a7af-0adff12da7e9" containerName="nova-api-log" Dec 03 12:00:47 crc kubenswrapper[5052]: I1203 12:00:47.918729 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="d7d3fec6-50e1-43f1-a7af-0adff12da7e9" containerName="nova-api-api" Dec 03 12:00:47 crc kubenswrapper[5052]: I1203 12:00:47.918759 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="d7d3fec6-50e1-43f1-a7af-0adff12da7e9" containerName="nova-api-log" Dec 03 12:00:47 crc kubenswrapper[5052]: I1203 12:00:47.920093 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:00:47 crc kubenswrapper[5052]: I1203 12:00:47.923353 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-api-config-data" Dec 03 12:00:47 crc kubenswrapper[5052]: I1203 12:00:47.923618 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-nova-internal-svc" Dec 03 12:00:47 crc kubenswrapper[5052]: I1203 12:00:47.925906 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-nova-public-svc" Dec 03 12:00:47 crc kubenswrapper[5052]: I1203 12:00:47.928184 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:00:47 crc kubenswrapper[5052]: I1203 12:00:47.968016 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"73c1b1fb-5062-4c33-be17-72324f222396","Type":"ContainerStarted","Data":"fa90af8ae17f19b8e657ed6b90a3d32798eac71027682bd29946af1de563200d"} Dec 03 12:00:47 crc kubenswrapper[5052]: I1203 12:00:47.969715 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"98d3c14e-7549-441a-9532-f3d75833cdf8","Type":"ContainerStarted","Data":"1c18a9d55257352dbf809914f4dca8da241a548437c4cbcf603a968646bbaaf6"} Dec 03 12:00:47 crc kubenswrapper[5052]: I1203 12:00:47.969738 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"98d3c14e-7549-441a-9532-f3d75833cdf8","Type":"ContainerStarted","Data":"710c4f7c04a361b712543bf316631f84336d70d531c95a80fd6746ea4334640d"} Dec 03 12:00:47 crc kubenswrapper[5052]: I1203 12:00:47.988842 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-scheduler-0" podStartSLOduration=2.9888234049999998 podStartE2EDuration="2.988823405s" podCreationTimestamp="2025-12-03 12:00:45 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:00:47.981799624 +0000 UTC m=+2438.570041388" watchObservedRunningTime="2025-12-03 12:00:47.988823405 +0000 UTC m=+2438.577065169" Dec 03 12:00:48 crc kubenswrapper[5052]: I1203 12:00:48.006209 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-metadata-0" podStartSLOduration=3.006190231 podStartE2EDuration="3.006190231s" podCreationTimestamp="2025-12-03 12:00:45 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:00:48.002368932 +0000 UTC m=+2438.590610706" watchObservedRunningTime="2025-12-03 12:00:48.006190231 +0000 UTC m=+2438.594431995" Dec 03 12:00:48 crc kubenswrapper[5052]: I1203 12:00:48.093252 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1103e975-cc9a-435b-adc9-5c6492e9c629-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"1103e975-cc9a-435b-adc9-5c6492e9c629\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:00:48 crc kubenswrapper[5052]: I1203 12:00:48.093294 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8brzf\" (UniqueName: \"kubernetes.io/projected/1103e975-cc9a-435b-adc9-5c6492e9c629-kube-api-access-8brzf\") pod \"nova-api-0\" (UID: \"1103e975-cc9a-435b-adc9-5c6492e9c629\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:00:48 crc kubenswrapper[5052]: I1203 12:00:48.093380 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/1103e975-cc9a-435b-adc9-5c6492e9c629-public-tls-certs\") pod \"nova-api-0\" (UID: \"1103e975-cc9a-435b-adc9-5c6492e9c629\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:00:48 crc kubenswrapper[5052]: I1203 12:00:48.093418 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/1103e975-cc9a-435b-adc9-5c6492e9c629-logs\") pod \"nova-api-0\" (UID: \"1103e975-cc9a-435b-adc9-5c6492e9c629\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:00:48 crc kubenswrapper[5052]: I1203 12:00:48.093999 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/1103e975-cc9a-435b-adc9-5c6492e9c629-config-data\") pod \"nova-api-0\" (UID: \"1103e975-cc9a-435b-adc9-5c6492e9c629\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:00:48 crc kubenswrapper[5052]: I1203 12:00:48.094080 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/1103e975-cc9a-435b-adc9-5c6492e9c629-internal-tls-certs\") pod \"nova-api-0\" (UID: \"1103e975-cc9a-435b-adc9-5c6492e9c629\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:00:48 crc kubenswrapper[5052]: I1203 12:00:48.196133 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/1103e975-cc9a-435b-adc9-5c6492e9c629-public-tls-certs\") pod \"nova-api-0\" (UID: \"1103e975-cc9a-435b-adc9-5c6492e9c629\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:00:48 crc kubenswrapper[5052]: I1203 12:00:48.196461 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/1103e975-cc9a-435b-adc9-5c6492e9c629-logs\") pod \"nova-api-0\" (UID: \"1103e975-cc9a-435b-adc9-5c6492e9c629\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:00:48 crc kubenswrapper[5052]: I1203 12:00:48.196608 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/1103e975-cc9a-435b-adc9-5c6492e9c629-config-data\") pod \"nova-api-0\" (UID: \"1103e975-cc9a-435b-adc9-5c6492e9c629\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:00:48 crc kubenswrapper[5052]: I1203 12:00:48.196716 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/1103e975-cc9a-435b-adc9-5c6492e9c629-internal-tls-certs\") pod \"nova-api-0\" (UID: \"1103e975-cc9a-435b-adc9-5c6492e9c629\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:00:48 crc kubenswrapper[5052]: I1203 12:00:48.196817 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1103e975-cc9a-435b-adc9-5c6492e9c629-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"1103e975-cc9a-435b-adc9-5c6492e9c629\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:00:48 crc kubenswrapper[5052]: I1203 12:00:48.196891 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-8brzf\" (UniqueName: \"kubernetes.io/projected/1103e975-cc9a-435b-adc9-5c6492e9c629-kube-api-access-8brzf\") pod \"nova-api-0\" (UID: \"1103e975-cc9a-435b-adc9-5c6492e9c629\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:00:48 crc kubenswrapper[5052]: I1203 12:00:48.196972 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/1103e975-cc9a-435b-adc9-5c6492e9c629-logs\") pod \"nova-api-0\" (UID: \"1103e975-cc9a-435b-adc9-5c6492e9c629\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:00:48 crc kubenswrapper[5052]: I1203 12:00:48.200261 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1103e975-cc9a-435b-adc9-5c6492e9c629-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"1103e975-cc9a-435b-adc9-5c6492e9c629\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:00:48 crc kubenswrapper[5052]: I1203 12:00:48.200367 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/1103e975-cc9a-435b-adc9-5c6492e9c629-internal-tls-certs\") pod \"nova-api-0\" (UID: \"1103e975-cc9a-435b-adc9-5c6492e9c629\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:00:48 crc kubenswrapper[5052]: I1203 12:00:48.200400 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/1103e975-cc9a-435b-adc9-5c6492e9c629-config-data\") pod \"nova-api-0\" (UID: \"1103e975-cc9a-435b-adc9-5c6492e9c629\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:00:48 crc kubenswrapper[5052]: I1203 12:00:48.201304 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/1103e975-cc9a-435b-adc9-5c6492e9c629-public-tls-certs\") pod \"nova-api-0\" (UID: \"1103e975-cc9a-435b-adc9-5c6492e9c629\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:00:48 crc kubenswrapper[5052]: I1203 12:00:48.224537 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-8brzf\" (UniqueName: \"kubernetes.io/projected/1103e975-cc9a-435b-adc9-5c6492e9c629-kube-api-access-8brzf\") pod \"nova-api-0\" (UID: \"1103e975-cc9a-435b-adc9-5c6492e9c629\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:00:48 crc kubenswrapper[5052]: I1203 12:00:48.246765 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:00:48 crc kubenswrapper[5052]: I1203 12:00:48.687151 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:00:48 crc kubenswrapper[5052]: I1203 12:00:48.990058 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"1103e975-cc9a-435b-adc9-5c6492e9c629","Type":"ContainerStarted","Data":"be465bb3373c78a7e0d0341f23e780fbf057154f4f4e5cd558d38b99cc20e088"} Dec 03 12:00:48 crc kubenswrapper[5052]: I1203 12:00:48.990384 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"1103e975-cc9a-435b-adc9-5c6492e9c629","Type":"ContainerStarted","Data":"5ae0d6b38d9ba8996463de010c21292bbc70e163c1d484bf8a332a74fedfbc7b"} Dec 03 12:00:49 crc kubenswrapper[5052]: I1203 12:00:49.686763 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d7d3fec6-50e1-43f1-a7af-0adff12da7e9" path="/var/lib/kubelet/pods/d7d3fec6-50e1-43f1-a7af-0adff12da7e9/volumes" Dec 03 12:00:50 crc kubenswrapper[5052]: I1203 12:00:50.001298 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"1103e975-cc9a-435b-adc9-5c6492e9c629","Type":"ContainerStarted","Data":"aebafbbccc45118d70ddef8e3f76333d5b32b3641a77534948d68d7de2fc9fd0"} Dec 03 12:00:50 crc kubenswrapper[5052]: I1203 12:00:50.033219 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-api-0" podStartSLOduration=3.033197662 podStartE2EDuration="3.033197662s" podCreationTimestamp="2025-12-03 12:00:47 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:00:50.026378607 +0000 UTC m=+2440.614620391" watchObservedRunningTime="2025-12-03 12:00:50.033197662 +0000 UTC m=+2440.621439446" Dec 03 12:00:50 crc kubenswrapper[5052]: I1203 12:00:50.419921 5052 scope.go:117] "RemoveContainer" containerID="555a48fb112549936fa017dafd0bd2b3a6535396cbeaebfeacc9a34ebb837f35" Dec 03 12:00:50 crc kubenswrapper[5052]: I1203 12:00:50.448914 5052 scope.go:117] "RemoveContainer" containerID="f95fe7d911e5df1600159d8eebb7f9779559c2e85ed0e281114a0ec510f37ed4" Dec 03 12:00:50 crc kubenswrapper[5052]: I1203 12:00:50.475606 5052 scope.go:117] "RemoveContainer" containerID="5329302d29b19e4f5f394764bc11bf1db401d466936911460100a0ce63b692d6" Dec 03 12:00:50 crc kubenswrapper[5052]: I1203 12:00:50.530499 5052 scope.go:117] "RemoveContainer" containerID="e2f05729fcded9f0fd3a94445c561f0dc54d8da986e5b0a038ffa712510d5162" Dec 03 12:00:50 crc kubenswrapper[5052]: I1203 12:00:50.569909 5052 scope.go:117] "RemoveContainer" containerID="60e68a3c80e70643ed7fc58979bc6f13ffbe656be8c84f11083d76691b56fbba" Dec 03 12:00:50 crc kubenswrapper[5052]: I1203 12:00:50.610062 5052 scope.go:117] "RemoveContainer" containerID="b114369f17334b3461585b8e213c8285e8ea520f731eeea65e56b8a04d08702b" Dec 03 12:00:51 crc kubenswrapper[5052]: I1203 12:00:51.338932 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:00:51 crc kubenswrapper[5052]: I1203 12:00:51.353459 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:00:51 crc kubenswrapper[5052]: I1203 12:00:51.353559 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:00:51 crc kubenswrapper[5052]: I1203 12:00:51.673422 5052 scope.go:117] "RemoveContainer" containerID="d224717c6d9ca8135fd510e03a6c40c9f2b1a56dd8b4a3b685af3e2dfa855982" Dec 03 12:00:51 crc kubenswrapper[5052]: E1203 12:00:51.673691 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 12:00:56 crc kubenswrapper[5052]: I1203 12:00:56.339959 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:00:56 crc kubenswrapper[5052]: I1203 12:00:56.354474 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:00:56 crc kubenswrapper[5052]: I1203 12:00:56.355192 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:00:56 crc kubenswrapper[5052]: I1203 12:00:56.370534 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:00:57 crc kubenswrapper[5052]: I1203 12:00:57.088843 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:00:57 crc kubenswrapper[5052]: I1203 12:00:57.365748 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-metadata-0" podUID="98d3c14e-7549-441a-9532-f3d75833cdf8" containerName="nova-metadata-log" probeResult="failure" output="Get \"https://10.217.0.12:8775/\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Dec 03 12:00:57 crc kubenswrapper[5052]: I1203 12:00:57.365774 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-metadata-0" podUID="98d3c14e-7549-441a-9532-f3d75833cdf8" containerName="nova-metadata-metadata" probeResult="failure" output="Get \"https://10.217.0.12:8775/\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Dec 03 12:00:58 crc kubenswrapper[5052]: I1203 12:00:58.247123 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:00:58 crc kubenswrapper[5052]: I1203 12:00:58.247478 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:00:59 crc kubenswrapper[5052]: I1203 12:00:59.061896 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:00:59 crc kubenswrapper[5052]: I1203 12:00:59.262829 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-api-0" podUID="1103e975-cc9a-435b-adc9-5c6492e9c629" containerName="nova-api-api" probeResult="failure" output="Get \"https://10.217.0.16:8774/\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Dec 03 12:00:59 crc kubenswrapper[5052]: I1203 12:00:59.263349 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-api-0" podUID="1103e975-cc9a-435b-adc9-5c6492e9c629" containerName="nova-api-log" probeResult="failure" output="Get \"https://10.217.0.16:8774/\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Dec 03 12:01:00 crc kubenswrapper[5052]: I1203 12:01:00.161993 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/keystone-cron-29412721-2k46m"] Dec 03 12:01:00 crc kubenswrapper[5052]: I1203 12:01:00.164062 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-cron-29412721-2k46m" Dec 03 12:01:00 crc kubenswrapper[5052]: I1203 12:01:00.176574 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-cron-29412721-2k46m"] Dec 03 12:01:00 crc kubenswrapper[5052]: I1203 12:01:00.330226 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/27904d79-b16e-4909-bf71-3a9f8f325597-config-data\") pod \"keystone-cron-29412721-2k46m\" (UID: \"27904d79-b16e-4909-bf71-3a9f8f325597\") " pod="openstack-kuttl-tests/keystone-cron-29412721-2k46m" Dec 03 12:01:00 crc kubenswrapper[5052]: I1203 12:01:00.330358 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5l7xf\" (UniqueName: \"kubernetes.io/projected/27904d79-b16e-4909-bf71-3a9f8f325597-kube-api-access-5l7xf\") pod \"keystone-cron-29412721-2k46m\" (UID: \"27904d79-b16e-4909-bf71-3a9f8f325597\") " pod="openstack-kuttl-tests/keystone-cron-29412721-2k46m" Dec 03 12:01:00 crc kubenswrapper[5052]: I1203 12:01:00.330438 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/27904d79-b16e-4909-bf71-3a9f8f325597-combined-ca-bundle\") pod \"keystone-cron-29412721-2k46m\" (UID: \"27904d79-b16e-4909-bf71-3a9f8f325597\") " pod="openstack-kuttl-tests/keystone-cron-29412721-2k46m" Dec 03 12:01:00 crc kubenswrapper[5052]: I1203 12:01:00.330462 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/27904d79-b16e-4909-bf71-3a9f8f325597-fernet-keys\") pod \"keystone-cron-29412721-2k46m\" (UID: \"27904d79-b16e-4909-bf71-3a9f8f325597\") " pod="openstack-kuttl-tests/keystone-cron-29412721-2k46m" Dec 03 12:01:00 crc kubenswrapper[5052]: I1203 12:01:00.432538 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-5l7xf\" (UniqueName: \"kubernetes.io/projected/27904d79-b16e-4909-bf71-3a9f8f325597-kube-api-access-5l7xf\") pod \"keystone-cron-29412721-2k46m\" (UID: \"27904d79-b16e-4909-bf71-3a9f8f325597\") " pod="openstack-kuttl-tests/keystone-cron-29412721-2k46m" Dec 03 12:01:00 crc kubenswrapper[5052]: I1203 12:01:00.432653 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/27904d79-b16e-4909-bf71-3a9f8f325597-combined-ca-bundle\") pod \"keystone-cron-29412721-2k46m\" (UID: \"27904d79-b16e-4909-bf71-3a9f8f325597\") " pod="openstack-kuttl-tests/keystone-cron-29412721-2k46m" Dec 03 12:01:00 crc kubenswrapper[5052]: I1203 12:01:00.432688 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/27904d79-b16e-4909-bf71-3a9f8f325597-fernet-keys\") pod \"keystone-cron-29412721-2k46m\" (UID: \"27904d79-b16e-4909-bf71-3a9f8f325597\") " pod="openstack-kuttl-tests/keystone-cron-29412721-2k46m" Dec 03 12:01:00 crc kubenswrapper[5052]: I1203 12:01:00.432762 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/27904d79-b16e-4909-bf71-3a9f8f325597-config-data\") pod \"keystone-cron-29412721-2k46m\" (UID: \"27904d79-b16e-4909-bf71-3a9f8f325597\") " pod="openstack-kuttl-tests/keystone-cron-29412721-2k46m" Dec 03 12:01:00 crc kubenswrapper[5052]: I1203 12:01:00.451335 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/27904d79-b16e-4909-bf71-3a9f8f325597-combined-ca-bundle\") pod \"keystone-cron-29412721-2k46m\" (UID: \"27904d79-b16e-4909-bf71-3a9f8f325597\") " pod="openstack-kuttl-tests/keystone-cron-29412721-2k46m" Dec 03 12:01:00 crc kubenswrapper[5052]: I1203 12:01:00.451440 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/27904d79-b16e-4909-bf71-3a9f8f325597-config-data\") pod \"keystone-cron-29412721-2k46m\" (UID: \"27904d79-b16e-4909-bf71-3a9f8f325597\") " pod="openstack-kuttl-tests/keystone-cron-29412721-2k46m" Dec 03 12:01:00 crc kubenswrapper[5052]: I1203 12:01:00.452638 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/27904d79-b16e-4909-bf71-3a9f8f325597-fernet-keys\") pod \"keystone-cron-29412721-2k46m\" (UID: \"27904d79-b16e-4909-bf71-3a9f8f325597\") " pod="openstack-kuttl-tests/keystone-cron-29412721-2k46m" Dec 03 12:01:00 crc kubenswrapper[5052]: I1203 12:01:00.465199 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-5l7xf\" (UniqueName: \"kubernetes.io/projected/27904d79-b16e-4909-bf71-3a9f8f325597-kube-api-access-5l7xf\") pod \"keystone-cron-29412721-2k46m\" (UID: \"27904d79-b16e-4909-bf71-3a9f8f325597\") " pod="openstack-kuttl-tests/keystone-cron-29412721-2k46m" Dec 03 12:01:00 crc kubenswrapper[5052]: I1203 12:01:00.498325 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-cron-29412721-2k46m" Dec 03 12:01:00 crc kubenswrapper[5052]: I1203 12:01:00.995723 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-cron-29412721-2k46m"] Dec 03 12:01:01 crc kubenswrapper[5052]: I1203 12:01:01.101432 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-cron-29412721-2k46m" event={"ID":"27904d79-b16e-4909-bf71-3a9f8f325597","Type":"ContainerStarted","Data":"d0a0a22c4200325dc7e22483f3c041fd292bdd0dd25bc7d3ce64f177f4c3b839"} Dec 03 12:01:02 crc kubenswrapper[5052]: I1203 12:01:02.115847 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-cron-29412721-2k46m" event={"ID":"27904d79-b16e-4909-bf71-3a9f8f325597","Type":"ContainerStarted","Data":"cb6c2c93752f2f8c0a07df86d1815e82e87a24854f9d5ff8d5cb21edecd1b88a"} Dec 03 12:01:02 crc kubenswrapper[5052]: I1203 12:01:02.136657 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/keystone-cron-29412721-2k46m" podStartSLOduration=2.1366404 podStartE2EDuration="2.1366404s" podCreationTimestamp="2025-12-03 12:01:00 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:01:02.133379077 +0000 UTC m=+2452.721620841" watchObservedRunningTime="2025-12-03 12:01:02.1366404 +0000 UTC m=+2452.724882174" Dec 03 12:01:04 crc kubenswrapper[5052]: I1203 12:01:04.143729 5052 generic.go:334] "Generic (PLEG): container finished" podID="27904d79-b16e-4909-bf71-3a9f8f325597" containerID="cb6c2c93752f2f8c0a07df86d1815e82e87a24854f9d5ff8d5cb21edecd1b88a" exitCode=0 Dec 03 12:01:04 crc kubenswrapper[5052]: I1203 12:01:04.143848 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-cron-29412721-2k46m" event={"ID":"27904d79-b16e-4909-bf71-3a9f8f325597","Type":"ContainerDied","Data":"cb6c2c93752f2f8c0a07df86d1815e82e87a24854f9d5ff8d5cb21edecd1b88a"} Dec 03 12:01:05 crc kubenswrapper[5052]: I1203 12:01:05.499825 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-cron-29412721-2k46m" Dec 03 12:01:05 crc kubenswrapper[5052]: I1203 12:01:05.561629 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5l7xf\" (UniqueName: \"kubernetes.io/projected/27904d79-b16e-4909-bf71-3a9f8f325597-kube-api-access-5l7xf\") pod \"27904d79-b16e-4909-bf71-3a9f8f325597\" (UID: \"27904d79-b16e-4909-bf71-3a9f8f325597\") " Dec 03 12:01:05 crc kubenswrapper[5052]: I1203 12:01:05.561731 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/27904d79-b16e-4909-bf71-3a9f8f325597-config-data\") pod \"27904d79-b16e-4909-bf71-3a9f8f325597\" (UID: \"27904d79-b16e-4909-bf71-3a9f8f325597\") " Dec 03 12:01:05 crc kubenswrapper[5052]: I1203 12:01:05.561799 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/27904d79-b16e-4909-bf71-3a9f8f325597-combined-ca-bundle\") pod \"27904d79-b16e-4909-bf71-3a9f8f325597\" (UID: \"27904d79-b16e-4909-bf71-3a9f8f325597\") " Dec 03 12:01:05 crc kubenswrapper[5052]: I1203 12:01:05.561900 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/27904d79-b16e-4909-bf71-3a9f8f325597-fernet-keys\") pod \"27904d79-b16e-4909-bf71-3a9f8f325597\" (UID: \"27904d79-b16e-4909-bf71-3a9f8f325597\") " Dec 03 12:01:05 crc kubenswrapper[5052]: I1203 12:01:05.567686 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/27904d79-b16e-4909-bf71-3a9f8f325597-kube-api-access-5l7xf" (OuterVolumeSpecName: "kube-api-access-5l7xf") pod "27904d79-b16e-4909-bf71-3a9f8f325597" (UID: "27904d79-b16e-4909-bf71-3a9f8f325597"). InnerVolumeSpecName "kube-api-access-5l7xf". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:01:05 crc kubenswrapper[5052]: I1203 12:01:05.567678 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/27904d79-b16e-4909-bf71-3a9f8f325597-fernet-keys" (OuterVolumeSpecName: "fernet-keys") pod "27904d79-b16e-4909-bf71-3a9f8f325597" (UID: "27904d79-b16e-4909-bf71-3a9f8f325597"). InnerVolumeSpecName "fernet-keys". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:01:05 crc kubenswrapper[5052]: I1203 12:01:05.591996 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/27904d79-b16e-4909-bf71-3a9f8f325597-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "27904d79-b16e-4909-bf71-3a9f8f325597" (UID: "27904d79-b16e-4909-bf71-3a9f8f325597"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:01:05 crc kubenswrapper[5052]: I1203 12:01:05.614889 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/27904d79-b16e-4909-bf71-3a9f8f325597-config-data" (OuterVolumeSpecName: "config-data") pod "27904d79-b16e-4909-bf71-3a9f8f325597" (UID: "27904d79-b16e-4909-bf71-3a9f8f325597"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:01:05 crc kubenswrapper[5052]: I1203 12:01:05.663954 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/27904d79-b16e-4909-bf71-3a9f8f325597-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:05 crc kubenswrapper[5052]: I1203 12:01:05.663983 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/27904d79-b16e-4909-bf71-3a9f8f325597-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:05 crc kubenswrapper[5052]: I1203 12:01:05.663997 5052 reconciler_common.go:293] "Volume detached for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/27904d79-b16e-4909-bf71-3a9f8f325597-fernet-keys\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:05 crc kubenswrapper[5052]: I1203 12:01:05.664006 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-5l7xf\" (UniqueName: \"kubernetes.io/projected/27904d79-b16e-4909-bf71-3a9f8f325597-kube-api-access-5l7xf\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:06 crc kubenswrapper[5052]: I1203 12:01:06.162344 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-cron-29412721-2k46m" event={"ID":"27904d79-b16e-4909-bf71-3a9f8f325597","Type":"ContainerDied","Data":"d0a0a22c4200325dc7e22483f3c041fd292bdd0dd25bc7d3ce64f177f4c3b839"} Dec 03 12:01:06 crc kubenswrapper[5052]: I1203 12:01:06.162669 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d0a0a22c4200325dc7e22483f3c041fd292bdd0dd25bc7d3ce64f177f4c3b839" Dec 03 12:01:06 crc kubenswrapper[5052]: I1203 12:01:06.162419 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-cron-29412721-2k46m" Dec 03 12:01:06 crc kubenswrapper[5052]: I1203 12:01:06.358673 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:01:06 crc kubenswrapper[5052]: I1203 12:01:06.359089 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:01:06 crc kubenswrapper[5052]: I1203 12:01:06.365792 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:01:06 crc kubenswrapper[5052]: I1203 12:01:06.690287 5052 scope.go:117] "RemoveContainer" containerID="d224717c6d9ca8135fd510e03a6c40c9f2b1a56dd8b4a3b685af3e2dfa855982" Dec 03 12:01:06 crc kubenswrapper[5052]: E1203 12:01:06.691068 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 12:01:07 crc kubenswrapper[5052]: I1203 12:01:07.181089 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:01:08 crc kubenswrapper[5052]: I1203 12:01:08.256266 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:01:08 crc kubenswrapper[5052]: I1203 12:01:08.257669 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:01:08 crc kubenswrapper[5052]: I1203 12:01:08.260552 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:01:08 crc kubenswrapper[5052]: I1203 12:01:08.265660 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:01:09 crc kubenswrapper[5052]: I1203 12:01:09.189496 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:01:09 crc kubenswrapper[5052]: I1203 12:01:09.196242 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:01:11 crc kubenswrapper[5052]: I1203 12:01:11.195388 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/openstackclient"] Dec 03 12:01:11 crc kubenswrapper[5052]: I1203 12:01:11.195916 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/openstackclient" podUID="05236591-6a13-4ff8-bc2f-2016649fa3fc" containerName="openstackclient" containerID="cri-o://f9982d5e0a6d516106ed549ea5250927607d0748ff6fe35b4feae30164c85aa3" gracePeriod=2 Dec 03 12:01:11 crc kubenswrapper[5052]: I1203 12:01:11.211222 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/openstackclient"] Dec 03 12:01:11 crc kubenswrapper[5052]: E1203 12:01:11.292959 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/combined-ca-bundle: secret "combined-ca-bundle" not found Dec 03 12:01:11 crc kubenswrapper[5052]: E1203 12:01:11.293043 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1103e975-cc9a-435b-adc9-5c6492e9c629-combined-ca-bundle podName:1103e975-cc9a-435b-adc9-5c6492e9c629 nodeName:}" failed. No retries permitted until 2025-12-03 12:01:11.793019675 +0000 UTC m=+2462.381261439 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "combined-ca-bundle" (UniqueName: "kubernetes.io/secret/1103e975-cc9a-435b-adc9-5c6492e9c629-combined-ca-bundle") pod "nova-api-0" (UID: "1103e975-cc9a-435b-adc9-5c6492e9c629") : secret "combined-ca-bundle" not found Dec 03 12:01:11 crc kubenswrapper[5052]: I1203 12:01:11.548034 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:01:11 crc kubenswrapper[5052]: I1203 12:01:11.548354 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="da4dfbea-20a2-4638-bbbe-ed4bfee0ad49" containerName="ceilometer-central-agent" containerID="cri-o://1658de9b62f64e764364ab2fb2ba523e5ca2b9210d45094169d4d8ec956ac253" gracePeriod=30 Dec 03 12:01:11 crc kubenswrapper[5052]: I1203 12:01:11.548992 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="da4dfbea-20a2-4638-bbbe-ed4bfee0ad49" containerName="proxy-httpd" containerID="cri-o://903649a39fdac1f3614d17974bbdb299dee9a076158995226565f8b3dd246b89" gracePeriod=30 Dec 03 12:01:11 crc kubenswrapper[5052]: I1203 12:01:11.549063 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="da4dfbea-20a2-4638-bbbe-ed4bfee0ad49" containerName="sg-core" containerID="cri-o://ef9cd2fbef68f2e1fe845f78b334b0ff9a4fcb0c22c21ce8f7ab13794d397239" gracePeriod=30 Dec 03 12:01:11 crc kubenswrapper[5052]: I1203 12:01:11.549112 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="da4dfbea-20a2-4638-bbbe-ed4bfee0ad49" containerName="ceilometer-notification-agent" containerID="cri-o://3f16de4b1d15886789bdc38843cbe1fb5e5180679e89cb7fb19837deadc091ae" gracePeriod=30 Dec 03 12:01:11 crc kubenswrapper[5052]: I1203 12:01:11.651222 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/cinderaaa7-account-delete-rdrqp"] Dec 03 12:01:11 crc kubenswrapper[5052]: E1203 12:01:11.651702 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="27904d79-b16e-4909-bf71-3a9f8f325597" containerName="keystone-cron" Dec 03 12:01:11 crc kubenswrapper[5052]: I1203 12:01:11.651721 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="27904d79-b16e-4909-bf71-3a9f8f325597" containerName="keystone-cron" Dec 03 12:01:11 crc kubenswrapper[5052]: E1203 12:01:11.651752 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="05236591-6a13-4ff8-bc2f-2016649fa3fc" containerName="openstackclient" Dec 03 12:01:11 crc kubenswrapper[5052]: I1203 12:01:11.651758 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="05236591-6a13-4ff8-bc2f-2016649fa3fc" containerName="openstackclient" Dec 03 12:01:11 crc kubenswrapper[5052]: I1203 12:01:11.652012 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="27904d79-b16e-4909-bf71-3a9f8f325597" containerName="keystone-cron" Dec 03 12:01:11 crc kubenswrapper[5052]: I1203 12:01:11.652047 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="05236591-6a13-4ff8-bc2f-2016649fa3fc" containerName="openstackclient" Dec 03 12:01:11 crc kubenswrapper[5052]: I1203 12:01:11.652967 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinderaaa7-account-delete-rdrqp" Dec 03 12:01:11 crc kubenswrapper[5052]: I1203 12:01:11.673741 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinderaaa7-account-delete-rdrqp"] Dec 03 12:01:11 crc kubenswrapper[5052]: I1203 12:01:11.700414 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r5pbr\" (UniqueName: \"kubernetes.io/projected/492ec2b7-0c28-4fe2-b2be-4e9e58423f23-kube-api-access-r5pbr\") pod \"cinderaaa7-account-delete-rdrqp\" (UID: \"492ec2b7-0c28-4fe2-b2be-4e9e58423f23\") " pod="openstack-kuttl-tests/cinderaaa7-account-delete-rdrqp" Dec 03 12:01:11 crc kubenswrapper[5052]: I1203 12:01:11.700474 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/492ec2b7-0c28-4fe2-b2be-4e9e58423f23-operator-scripts\") pod \"cinderaaa7-account-delete-rdrqp\" (UID: \"492ec2b7-0c28-4fe2-b2be-4e9e58423f23\") " pod="openstack-kuttl-tests/cinderaaa7-account-delete-rdrqp" Dec 03 12:01:11 crc kubenswrapper[5052]: I1203 12:01:11.808431 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-r5pbr\" (UniqueName: \"kubernetes.io/projected/492ec2b7-0c28-4fe2-b2be-4e9e58423f23-kube-api-access-r5pbr\") pod \"cinderaaa7-account-delete-rdrqp\" (UID: \"492ec2b7-0c28-4fe2-b2be-4e9e58423f23\") " pod="openstack-kuttl-tests/cinderaaa7-account-delete-rdrqp" Dec 03 12:01:11 crc kubenswrapper[5052]: I1203 12:01:11.808491 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/492ec2b7-0c28-4fe2-b2be-4e9e58423f23-operator-scripts\") pod \"cinderaaa7-account-delete-rdrqp\" (UID: \"492ec2b7-0c28-4fe2-b2be-4e9e58423f23\") " pod="openstack-kuttl-tests/cinderaaa7-account-delete-rdrqp" Dec 03 12:01:11 crc kubenswrapper[5052]: I1203 12:01:11.809599 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/492ec2b7-0c28-4fe2-b2be-4e9e58423f23-operator-scripts\") pod \"cinderaaa7-account-delete-rdrqp\" (UID: \"492ec2b7-0c28-4fe2-b2be-4e9e58423f23\") " pod="openstack-kuttl-tests/cinderaaa7-account-delete-rdrqp" Dec 03 12:01:11 crc kubenswrapper[5052]: E1203 12:01:11.809722 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/combined-ca-bundle: secret "combined-ca-bundle" not found Dec 03 12:01:11 crc kubenswrapper[5052]: E1203 12:01:11.809770 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1103e975-cc9a-435b-adc9-5c6492e9c629-combined-ca-bundle podName:1103e975-cc9a-435b-adc9-5c6492e9c629 nodeName:}" failed. No retries permitted until 2025-12-03 12:01:12.809756319 +0000 UTC m=+2463.397998083 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "combined-ca-bundle" (UniqueName: "kubernetes.io/secret/1103e975-cc9a-435b-adc9-5c6492e9c629-combined-ca-bundle") pod "nova-api-0" (UID: "1103e975-cc9a-435b-adc9-5c6492e9c629") : secret "combined-ca-bundle" not found Dec 03 12:01:11 crc kubenswrapper[5052]: I1203 12:01:11.814850 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/glance0d27-account-delete-brqv4"] Dec 03 12:01:11 crc kubenswrapper[5052]: I1203 12:01:11.816459 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance0d27-account-delete-brqv4" Dec 03 12:01:11 crc kubenswrapper[5052]: I1203 12:01:11.891552 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance0d27-account-delete-brqv4"] Dec 03 12:01:11 crc kubenswrapper[5052]: I1203 12:01:11.910896 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lfdbn\" (UniqueName: \"kubernetes.io/projected/1a97695f-322c-4c4c-834e-ecb2f27c2e97-kube-api-access-lfdbn\") pod \"glance0d27-account-delete-brqv4\" (UID: \"1a97695f-322c-4c4c-834e-ecb2f27c2e97\") " pod="openstack-kuttl-tests/glance0d27-account-delete-brqv4" Dec 03 12:01:11 crc kubenswrapper[5052]: I1203 12:01:11.911029 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/1a97695f-322c-4c4c-834e-ecb2f27c2e97-operator-scripts\") pod \"glance0d27-account-delete-brqv4\" (UID: \"1a97695f-322c-4c4c-834e-ecb2f27c2e97\") " pod="openstack-kuttl-tests/glance0d27-account-delete-brqv4" Dec 03 12:01:11 crc kubenswrapper[5052]: I1203 12:01:11.940357 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-r5pbr\" (UniqueName: \"kubernetes.io/projected/492ec2b7-0c28-4fe2-b2be-4e9e58423f23-kube-api-access-r5pbr\") pod \"cinderaaa7-account-delete-rdrqp\" (UID: \"492ec2b7-0c28-4fe2-b2be-4e9e58423f23\") " pod="openstack-kuttl-tests/cinderaaa7-account-delete-rdrqp" Dec 03 12:01:12 crc kubenswrapper[5052]: I1203 12:01:12.009421 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinderaaa7-account-delete-rdrqp" Dec 03 12:01:12 crc kubenswrapper[5052]: I1203 12:01:12.013841 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/1a97695f-322c-4c4c-834e-ecb2f27c2e97-operator-scripts\") pod \"glance0d27-account-delete-brqv4\" (UID: \"1a97695f-322c-4c4c-834e-ecb2f27c2e97\") " pod="openstack-kuttl-tests/glance0d27-account-delete-brqv4" Dec 03 12:01:12 crc kubenswrapper[5052]: I1203 12:01:12.013974 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-lfdbn\" (UniqueName: \"kubernetes.io/projected/1a97695f-322c-4c4c-834e-ecb2f27c2e97-kube-api-access-lfdbn\") pod \"glance0d27-account-delete-brqv4\" (UID: \"1a97695f-322c-4c4c-834e-ecb2f27c2e97\") " pod="openstack-kuttl-tests/glance0d27-account-delete-brqv4" Dec 03 12:01:12 crc kubenswrapper[5052]: I1203 12:01:12.015081 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/1a97695f-322c-4c4c-834e-ecb2f27c2e97-operator-scripts\") pod \"glance0d27-account-delete-brqv4\" (UID: \"1a97695f-322c-4c4c-834e-ecb2f27c2e97\") " pod="openstack-kuttl-tests/glance0d27-account-delete-brqv4" Dec 03 12:01:12 crc kubenswrapper[5052]: I1203 12:01:12.043938 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/rabbitmq-server-0"] Dec 03 12:01:12 crc kubenswrapper[5052]: E1203 12:01:12.121256 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/rabbitmq-config-data: configmap "rabbitmq-config-data" not found Dec 03 12:01:12 crc kubenswrapper[5052]: E1203 12:01:12.121989 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/30578d71-fe2b-4eed-9a07-6c8bfe4a1c43-config-data podName:30578d71-fe2b-4eed-9a07-6c8bfe4a1c43 nodeName:}" failed. No retries permitted until 2025-12-03 12:01:12.621967384 +0000 UTC m=+2463.210209238 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "config-data" (UniqueName: "kubernetes.io/configmap/30578d71-fe2b-4eed-9a07-6c8bfe4a1c43-config-data") pod "rabbitmq-server-0" (UID: "30578d71-fe2b-4eed-9a07-6c8bfe4a1c43") : configmap "rabbitmq-config-data" not found Dec 03 12:01:12 crc kubenswrapper[5052]: I1203 12:01:12.127547 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/watcher36ab-account-delete-cvjb7"] Dec 03 12:01:12 crc kubenswrapper[5052]: I1203 12:01:12.133241 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/watcher36ab-account-delete-cvjb7" Dec 03 12:01:12 crc kubenswrapper[5052]: I1203 12:01:12.145209 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-lfdbn\" (UniqueName: \"kubernetes.io/projected/1a97695f-322c-4c4c-834e-ecb2f27c2e97-kube-api-access-lfdbn\") pod \"glance0d27-account-delete-brqv4\" (UID: \"1a97695f-322c-4c4c-834e-ecb2f27c2e97\") " pod="openstack-kuttl-tests/glance0d27-account-delete-brqv4" Dec 03 12:01:12 crc kubenswrapper[5052]: I1203 12:01:12.193491 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/watcher36ab-account-delete-cvjb7"] Dec 03 12:01:12 crc kubenswrapper[5052]: I1203 12:01:12.203372 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance0d27-account-delete-brqv4" Dec 03 12:01:12 crc kubenswrapper[5052]: I1203 12:01:12.250689 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder-db-sync-76kbf"] Dec 03 12:01:12 crc kubenswrapper[5052]: I1203 12:01:12.262676 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/cinder-db-sync-76kbf"] Dec 03 12:01:12 crc kubenswrapper[5052]: I1203 12:01:12.311428 5052 generic.go:334] "Generic (PLEG): container finished" podID="da4dfbea-20a2-4638-bbbe-ed4bfee0ad49" containerID="ef9cd2fbef68f2e1fe845f78b334b0ff9a4fcb0c22c21ce8f7ab13794d397239" exitCode=2 Dec 03 12:01:12 crc kubenswrapper[5052]: I1203 12:01:12.311474 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"da4dfbea-20a2-4638-bbbe-ed4bfee0ad49","Type":"ContainerDied","Data":"ef9cd2fbef68f2e1fe845f78b334b0ff9a4fcb0c22c21ce8f7ab13794d397239"} Dec 03 12:01:12 crc kubenswrapper[5052]: I1203 12:01:12.326494 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-spkfb\" (UniqueName: \"kubernetes.io/projected/b2a6be10-e7df-494c-be94-a8ede55e2868-kube-api-access-spkfb\") pod \"watcher36ab-account-delete-cvjb7\" (UID: \"b2a6be10-e7df-494c-be94-a8ede55e2868\") " pod="openstack-kuttl-tests/watcher36ab-account-delete-cvjb7" Dec 03 12:01:12 crc kubenswrapper[5052]: I1203 12:01:12.326595 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/b2a6be10-e7df-494c-be94-a8ede55e2868-operator-scripts\") pod \"watcher36ab-account-delete-cvjb7\" (UID: \"b2a6be10-e7df-494c-be94-a8ede55e2868\") " pod="openstack-kuttl-tests/watcher36ab-account-delete-cvjb7" Dec 03 12:01:12 crc kubenswrapper[5052]: I1203 12:01:12.334039 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ovn-northd-0"] Dec 03 12:01:12 crc kubenswrapper[5052]: I1203 12:01:12.336219 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ovn-northd-0" podUID="5e5ddf8c-6109-481f-beb0-0802de034912" containerName="ovn-northd" containerID="cri-o://e4b74aed237af57c4a5ccac244bde0abf28abccef9d3f249bad7df1bbb43da9d" gracePeriod=30 Dec 03 12:01:12 crc kubenswrapper[5052]: I1203 12:01:12.336740 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ovn-northd-0" podUID="5e5ddf8c-6109-481f-beb0-0802de034912" containerName="openstack-network-exporter" containerID="cri-o://986d340d25b89c2b20e9877dd2590330ca97a1a957d198b877ab542caeca6588" gracePeriod=30 Dec 03 12:01:12 crc kubenswrapper[5052]: I1203 12:01:12.421105 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/placemente203-account-delete-sdpql"] Dec 03 12:01:12 crc kubenswrapper[5052]: I1203 12:01:12.424546 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placemente203-account-delete-sdpql" Dec 03 12:01:12 crc kubenswrapper[5052]: I1203 12:01:12.430093 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-spkfb\" (UniqueName: \"kubernetes.io/projected/b2a6be10-e7df-494c-be94-a8ede55e2868-kube-api-access-spkfb\") pod \"watcher36ab-account-delete-cvjb7\" (UID: \"b2a6be10-e7df-494c-be94-a8ede55e2868\") " pod="openstack-kuttl-tests/watcher36ab-account-delete-cvjb7" Dec 03 12:01:12 crc kubenswrapper[5052]: I1203 12:01:12.439918 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/b2a6be10-e7df-494c-be94-a8ede55e2868-operator-scripts\") pod \"watcher36ab-account-delete-cvjb7\" (UID: \"b2a6be10-e7df-494c-be94-a8ede55e2868\") " pod="openstack-kuttl-tests/watcher36ab-account-delete-cvjb7" Dec 03 12:01:12 crc kubenswrapper[5052]: I1203 12:01:12.458631 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/b2a6be10-e7df-494c-be94-a8ede55e2868-operator-scripts\") pod \"watcher36ab-account-delete-cvjb7\" (UID: \"b2a6be10-e7df-494c-be94-a8ede55e2868\") " pod="openstack-kuttl-tests/watcher36ab-account-delete-cvjb7" Dec 03 12:01:12 crc kubenswrapper[5052]: I1203 12:01:12.463678 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/placemente203-account-delete-sdpql"] Dec 03 12:01:12 crc kubenswrapper[5052]: I1203 12:01:12.494687 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/barbican4aeb-account-delete-bvhdj"] Dec 03 12:01:12 crc kubenswrapper[5052]: I1203 12:01:12.496150 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican4aeb-account-delete-bvhdj" Dec 03 12:01:12 crc kubenswrapper[5052]: I1203 12:01:12.521335 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-spkfb\" (UniqueName: \"kubernetes.io/projected/b2a6be10-e7df-494c-be94-a8ede55e2868-kube-api-access-spkfb\") pod \"watcher36ab-account-delete-cvjb7\" (UID: \"b2a6be10-e7df-494c-be94-a8ede55e2868\") " pod="openstack-kuttl-tests/watcher36ab-account-delete-cvjb7" Dec 03 12:01:12 crc kubenswrapper[5052]: I1203 12:01:12.535853 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-db-sync-kfnjj"] Dec 03 12:01:12 crc kubenswrapper[5052]: I1203 12:01:12.542463 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/36225798-caeb-4514-9f51-c8a07fb6da9b-operator-scripts\") pod \"placemente203-account-delete-sdpql\" (UID: \"36225798-caeb-4514-9f51-c8a07fb6da9b\") " pod="openstack-kuttl-tests/placemente203-account-delete-sdpql" Dec 03 12:01:12 crc kubenswrapper[5052]: I1203 12:01:12.542622 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-28x5p\" (UniqueName: \"kubernetes.io/projected/36225798-caeb-4514-9f51-c8a07fb6da9b-kube-api-access-28x5p\") pod \"placemente203-account-delete-sdpql\" (UID: \"36225798-caeb-4514-9f51-c8a07fb6da9b\") " pod="openstack-kuttl-tests/placemente203-account-delete-sdpql" Dec 03 12:01:12 crc kubenswrapper[5052]: I1203 12:01:12.599194 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/watcher36ab-account-delete-cvjb7" Dec 03 12:01:12 crc kubenswrapper[5052]: I1203 12:01:12.683359 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6md2l\" (UniqueName: \"kubernetes.io/projected/d420a2c2-9b7b-400b-88a7-839a93cce573-kube-api-access-6md2l\") pod \"barbican4aeb-account-delete-bvhdj\" (UID: \"d420a2c2-9b7b-400b-88a7-839a93cce573\") " pod="openstack-kuttl-tests/barbican4aeb-account-delete-bvhdj" Dec 03 12:01:12 crc kubenswrapper[5052]: I1203 12:01:12.683423 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-28x5p\" (UniqueName: \"kubernetes.io/projected/36225798-caeb-4514-9f51-c8a07fb6da9b-kube-api-access-28x5p\") pod \"placemente203-account-delete-sdpql\" (UID: \"36225798-caeb-4514-9f51-c8a07fb6da9b\") " pod="openstack-kuttl-tests/placemente203-account-delete-sdpql" Dec 03 12:01:12 crc kubenswrapper[5052]: I1203 12:01:12.683640 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/36225798-caeb-4514-9f51-c8a07fb6da9b-operator-scripts\") pod \"placemente203-account-delete-sdpql\" (UID: \"36225798-caeb-4514-9f51-c8a07fb6da9b\") " pod="openstack-kuttl-tests/placemente203-account-delete-sdpql" Dec 03 12:01:12 crc kubenswrapper[5052]: I1203 12:01:12.683693 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/d420a2c2-9b7b-400b-88a7-839a93cce573-operator-scripts\") pod \"barbican4aeb-account-delete-bvhdj\" (UID: \"d420a2c2-9b7b-400b-88a7-839a93cce573\") " pod="openstack-kuttl-tests/barbican4aeb-account-delete-bvhdj" Dec 03 12:01:12 crc kubenswrapper[5052]: E1203 12:01:12.684067 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/rabbitmq-config-data: configmap "rabbitmq-config-data" not found Dec 03 12:01:12 crc kubenswrapper[5052]: E1203 12:01:12.684132 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/30578d71-fe2b-4eed-9a07-6c8bfe4a1c43-config-data podName:30578d71-fe2b-4eed-9a07-6c8bfe4a1c43 nodeName:}" failed. No retries permitted until 2025-12-03 12:01:13.684117015 +0000 UTC m=+2464.272358779 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "config-data" (UniqueName: "kubernetes.io/configmap/30578d71-fe2b-4eed-9a07-6c8bfe4a1c43-config-data") pod "rabbitmq-server-0" (UID: "30578d71-fe2b-4eed-9a07-6c8bfe4a1c43") : configmap "rabbitmq-config-data" not found Dec 03 12:01:12 crc kubenswrapper[5052]: I1203 12:01:12.808681 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/36225798-caeb-4514-9f51-c8a07fb6da9b-operator-scripts\") pod \"placemente203-account-delete-sdpql\" (UID: \"36225798-caeb-4514-9f51-c8a07fb6da9b\") " pod="openstack-kuttl-tests/placemente203-account-delete-sdpql" Dec 03 12:01:12 crc kubenswrapper[5052]: I1203 12:01:12.816006 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/glance-db-sync-kfnjj"] Dec 03 12:01:12 crc kubenswrapper[5052]: I1203 12:01:12.821057 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-6md2l\" (UniqueName: \"kubernetes.io/projected/d420a2c2-9b7b-400b-88a7-839a93cce573-kube-api-access-6md2l\") pod \"barbican4aeb-account-delete-bvhdj\" (UID: \"d420a2c2-9b7b-400b-88a7-839a93cce573\") " pod="openstack-kuttl-tests/barbican4aeb-account-delete-bvhdj" Dec 03 12:01:12 crc kubenswrapper[5052]: I1203 12:01:12.821191 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/d420a2c2-9b7b-400b-88a7-839a93cce573-operator-scripts\") pod \"barbican4aeb-account-delete-bvhdj\" (UID: \"d420a2c2-9b7b-400b-88a7-839a93cce573\") " pod="openstack-kuttl-tests/barbican4aeb-account-delete-bvhdj" Dec 03 12:01:12 crc kubenswrapper[5052]: E1203 12:01:12.821448 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/combined-ca-bundle: secret "combined-ca-bundle" not found Dec 03 12:01:12 crc kubenswrapper[5052]: E1203 12:01:12.821499 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1103e975-cc9a-435b-adc9-5c6492e9c629-combined-ca-bundle podName:1103e975-cc9a-435b-adc9-5c6492e9c629 nodeName:}" failed. No retries permitted until 2025-12-03 12:01:14.821483383 +0000 UTC m=+2465.409725147 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "combined-ca-bundle" (UniqueName: "kubernetes.io/secret/1103e975-cc9a-435b-adc9-5c6492e9c629-combined-ca-bundle") pod "nova-api-0" (UID: "1103e975-cc9a-435b-adc9-5c6492e9c629") : secret "combined-ca-bundle" not found Dec 03 12:01:12 crc kubenswrapper[5052]: I1203 12:01:12.822236 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/d420a2c2-9b7b-400b-88a7-839a93cce573-operator-scripts\") pod \"barbican4aeb-account-delete-bvhdj\" (UID: \"d420a2c2-9b7b-400b-88a7-839a93cce573\") " pod="openstack-kuttl-tests/barbican4aeb-account-delete-bvhdj" Dec 03 12:01:12 crc kubenswrapper[5052]: I1203 12:01:12.854857 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-28x5p\" (UniqueName: \"kubernetes.io/projected/36225798-caeb-4514-9f51-c8a07fb6da9b-kube-api-access-28x5p\") pod \"placemente203-account-delete-sdpql\" (UID: \"36225798-caeb-4514-9f51-c8a07fb6da9b\") " pod="openstack-kuttl-tests/placemente203-account-delete-sdpql" Dec 03 12:01:12 crc kubenswrapper[5052]: I1203 12:01:12.876073 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican4aeb-account-delete-bvhdj"] Dec 03 12:01:12 crc kubenswrapper[5052]: I1203 12:01:12.916165 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-6md2l\" (UniqueName: \"kubernetes.io/projected/d420a2c2-9b7b-400b-88a7-839a93cce573-kube-api-access-6md2l\") pod \"barbican4aeb-account-delete-bvhdj\" (UID: \"d420a2c2-9b7b-400b-88a7-839a93cce573\") " pod="openstack-kuttl-tests/barbican4aeb-account-delete-bvhdj" Dec 03 12:01:12 crc kubenswrapper[5052]: I1203 12:01:12.939448 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican4aeb-account-delete-bvhdj" Dec 03 12:01:12 crc kubenswrapper[5052]: I1203 12:01:12.953034 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/kube-state-metrics-0"] Dec 03 12:01:12 crc kubenswrapper[5052]: I1203 12:01:12.954435 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/kube-state-metrics-0" podUID="d03bf1c0-7353-4836-a767-9a5b2295e8dc" containerName="kube-state-metrics" containerID="cri-o://3a4d32fe7507dccb3b5a703d41f75ff7c20cb477aef43d8d0062e58cf58c09bf" gracePeriod=30 Dec 03 12:01:12 crc kubenswrapper[5052]: E1203 12:01:12.991636 5052 cadvisor_stats_provider.go:516] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podda4dfbea_20a2_4638_bbbe_ed4bfee0ad49.slice/crio-conmon-ef9cd2fbef68f2e1fe845f78b334b0ff9a4fcb0c22c21ce8f7ab13794d397239.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podda4dfbea_20a2_4638_bbbe_ed4bfee0ad49.slice/crio-ef9cd2fbef68f2e1fe845f78b334b0ff9a4fcb0c22c21ce8f7ab13794d397239.scope\": RecentStats: unable to find data in memory cache]" Dec 03 12:01:13 crc kubenswrapper[5052]: I1203 12:01:13.006989 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-sb-0"] Dec 03 12:01:13 crc kubenswrapper[5052]: I1203 12:01:13.007944 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ovsdbserver-sb-0" podUID="3cb4559f-542a-4076-82fb-c0ca88f9de94" containerName="openstack-network-exporter" containerID="cri-o://1ed38008a53ff370520241b3fd280d78870c056da808615558e81bfec79012cf" gracePeriod=300 Dec 03 12:01:13 crc kubenswrapper[5052]: I1203 12:01:13.115673 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ovsdbserver-sb-0" podUID="3cb4559f-542a-4076-82fb-c0ca88f9de94" containerName="ovsdbserver-sb" containerID="cri-o://fdc5cd84bfb15a2c4b38c80fec90fd397d2d372bb2cd5443eb73cf72b1a62b30" gracePeriod=300 Dec 03 12:01:13 crc kubenswrapper[5052]: I1203 12:01:13.123957 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/watcher-db-sync-chlm4"] Dec 03 12:01:13 crc kubenswrapper[5052]: I1203 12:01:13.149266 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/watcher-db-sync-chlm4"] Dec 03 12:01:13 crc kubenswrapper[5052]: I1203 12:01:13.227305 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/neutron0ea0-account-delete-lvffs"] Dec 03 12:01:13 crc kubenswrapper[5052]: I1203 12:01:13.229276 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron0ea0-account-delete-lvffs" Dec 03 12:01:13 crc kubenswrapper[5052]: I1203 12:01:13.253855 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron0ea0-account-delete-lvffs"] Dec 03 12:01:13 crc kubenswrapper[5052]: I1203 12:01:13.271238 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-nb-0"] Dec 03 12:01:13 crc kubenswrapper[5052]: I1203 12:01:13.272033 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ovsdbserver-nb-0" podUID="e8de8adf-e4e5-4a10-9722-9106357e67d0" containerName="openstack-network-exporter" containerID="cri-o://156a9ab495313477724afb77cc40733731b002a3db4ec302faefc4c513160b71" gracePeriod=300 Dec 03 12:01:13 crc kubenswrapper[5052]: I1203 12:01:13.324329 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/placement-db-sync-8tdfb"] Dec 03 12:01:13 crc kubenswrapper[5052]: I1203 12:01:13.324371 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-db-sync-xs2lk"] Dec 03 12:01:13 crc kubenswrapper[5052]: I1203 12:01:13.353792 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/c17bae4f-1c1b-4da2-9bb5-c84d6fdfa184-operator-scripts\") pod \"neutron0ea0-account-delete-lvffs\" (UID: \"c17bae4f-1c1b-4da2-9bb5-c84d6fdfa184\") " pod="openstack-kuttl-tests/neutron0ea0-account-delete-lvffs" Dec 03 12:01:13 crc kubenswrapper[5052]: I1203 12:01:13.354312 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j76fd\" (UniqueName: \"kubernetes.io/projected/c17bae4f-1c1b-4da2-9bb5-c84d6fdfa184-kube-api-access-j76fd\") pod \"neutron0ea0-account-delete-lvffs\" (UID: \"c17bae4f-1c1b-4da2-9bb5-c84d6fdfa184\") " pod="openstack-kuttl-tests/neutron0ea0-account-delete-lvffs" Dec 03 12:01:13 crc kubenswrapper[5052]: I1203 12:01:13.356658 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/placement-db-sync-8tdfb"] Dec 03 12:01:13 crc kubenswrapper[5052]: I1203 12:01:13.385879 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/barbican-db-sync-xs2lk"] Dec 03 12:01:13 crc kubenswrapper[5052]: I1203 12:01:13.389857 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ovsdbserver-nb-0" podUID="e8de8adf-e4e5-4a10-9722-9106357e67d0" containerName="ovsdbserver-nb" containerID="cri-o://a3deb4fed82228a5b22c0094ab0a2185263de5b8a00e34fbefb31c5da84f1d53" gracePeriod=300 Dec 03 12:01:13 crc kubenswrapper[5052]: I1203 12:01:13.404460 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_ovsdbserver-sb-0_3cb4559f-542a-4076-82fb-c0ca88f9de94/ovsdbserver-sb/0.log" Dec 03 12:01:13 crc kubenswrapper[5052]: I1203 12:01:13.408686 5052 generic.go:334] "Generic (PLEG): container finished" podID="3cb4559f-542a-4076-82fb-c0ca88f9de94" containerID="1ed38008a53ff370520241b3fd280d78870c056da808615558e81bfec79012cf" exitCode=2 Dec 03 12:01:13 crc kubenswrapper[5052]: I1203 12:01:13.409100 5052 generic.go:334] "Generic (PLEG): container finished" podID="3cb4559f-542a-4076-82fb-c0ca88f9de94" containerID="fdc5cd84bfb15a2c4b38c80fec90fd397d2d372bb2cd5443eb73cf72b1a62b30" exitCode=143 Dec 03 12:01:13 crc kubenswrapper[5052]: I1203 12:01:13.408956 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-sb-0" event={"ID":"3cb4559f-542a-4076-82fb-c0ca88f9de94","Type":"ContainerDied","Data":"1ed38008a53ff370520241b3fd280d78870c056da808615558e81bfec79012cf"} Dec 03 12:01:13 crc kubenswrapper[5052]: I1203 12:01:13.414270 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-sb-0" event={"ID":"3cb4559f-542a-4076-82fb-c0ca88f9de94","Type":"ContainerDied","Data":"fdc5cd84bfb15a2c4b38c80fec90fd397d2d372bb2cd5443eb73cf72b1a62b30"} Dec 03 12:01:13 crc kubenswrapper[5052]: I1203 12:01:13.416022 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/swift-ring-rebalance-mbrxc"] Dec 03 12:01:13 crc kubenswrapper[5052]: I1203 12:01:13.414413 5052 generic.go:334] "Generic (PLEG): container finished" podID="05236591-6a13-4ff8-bc2f-2016649fa3fc" containerID="f9982d5e0a6d516106ed549ea5250927607d0748ff6fe35b4feae30164c85aa3" exitCode=137 Dec 03 12:01:13 crc kubenswrapper[5052]: I1203 12:01:13.458223 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/c17bae4f-1c1b-4da2-9bb5-c84d6fdfa184-operator-scripts\") pod \"neutron0ea0-account-delete-lvffs\" (UID: \"c17bae4f-1c1b-4da2-9bb5-c84d6fdfa184\") " pod="openstack-kuttl-tests/neutron0ea0-account-delete-lvffs" Dec 03 12:01:13 crc kubenswrapper[5052]: I1203 12:01:13.459218 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-j76fd\" (UniqueName: \"kubernetes.io/projected/c17bae4f-1c1b-4da2-9bb5-c84d6fdfa184-kube-api-access-j76fd\") pod \"neutron0ea0-account-delete-lvffs\" (UID: \"c17bae4f-1c1b-4da2-9bb5-c84d6fdfa184\") " pod="openstack-kuttl-tests/neutron0ea0-account-delete-lvffs" Dec 03 12:01:13 crc kubenswrapper[5052]: I1203 12:01:13.460260 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/c17bae4f-1c1b-4da2-9bb5-c84d6fdfa184-operator-scripts\") pod \"neutron0ea0-account-delete-lvffs\" (UID: \"c17bae4f-1c1b-4da2-9bb5-c84d6fdfa184\") " pod="openstack-kuttl-tests/neutron0ea0-account-delete-lvffs" Dec 03 12:01:13 crc kubenswrapper[5052]: I1203 12:01:13.468578 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/swift-ring-rebalance-mbrxc"] Dec 03 12:01:13 crc kubenswrapper[5052]: I1203 12:01:13.473139 5052 generic.go:334] "Generic (PLEG): container finished" podID="da4dfbea-20a2-4638-bbbe-ed4bfee0ad49" containerID="903649a39fdac1f3614d17974bbdb299dee9a076158995226565f8b3dd246b89" exitCode=0 Dec 03 12:01:13 crc kubenswrapper[5052]: I1203 12:01:13.473241 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"da4dfbea-20a2-4638-bbbe-ed4bfee0ad49","Type":"ContainerDied","Data":"903649a39fdac1f3614d17974bbdb299dee9a076158995226565f8b3dd246b89"} Dec 03 12:01:13 crc kubenswrapper[5052]: I1203 12:01:13.491627 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-j76fd\" (UniqueName: \"kubernetes.io/projected/c17bae4f-1c1b-4da2-9bb5-c84d6fdfa184-kube-api-access-j76fd\") pod \"neutron0ea0-account-delete-lvffs\" (UID: \"c17bae4f-1c1b-4da2-9bb5-c84d6fdfa184\") " pod="openstack-kuttl-tests/neutron0ea0-account-delete-lvffs" Dec 03 12:01:13 crc kubenswrapper[5052]: I1203 12:01:13.497580 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_ovn-northd-0_5e5ddf8c-6109-481f-beb0-0802de034912/ovn-northd/0.log" Dec 03 12:01:13 crc kubenswrapper[5052]: I1203 12:01:13.497622 5052 generic.go:334] "Generic (PLEG): container finished" podID="5e5ddf8c-6109-481f-beb0-0802de034912" containerID="986d340d25b89c2b20e9877dd2590330ca97a1a957d198b877ab542caeca6588" exitCode=2 Dec 03 12:01:13 crc kubenswrapper[5052]: I1203 12:01:13.497637 5052 generic.go:334] "Generic (PLEG): container finished" podID="5e5ddf8c-6109-481f-beb0-0802de034912" containerID="e4b74aed237af57c4a5ccac244bde0abf28abccef9d3f249bad7df1bbb43da9d" exitCode=143 Dec 03 12:01:13 crc kubenswrapper[5052]: I1203 12:01:13.497708 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovn-northd-0" event={"ID":"5e5ddf8c-6109-481f-beb0-0802de034912","Type":"ContainerDied","Data":"986d340d25b89c2b20e9877dd2590330ca97a1a957d198b877ab542caeca6588"} Dec 03 12:01:13 crc kubenswrapper[5052]: I1203 12:01:13.497734 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovn-northd-0" event={"ID":"5e5ddf8c-6109-481f-beb0-0802de034912","Type":"ContainerDied","Data":"e4b74aed237af57c4a5ccac244bde0abf28abccef9d3f249bad7df1bbb43da9d"} Dec 03 12:01:13 crc kubenswrapper[5052]: I1203 12:01:13.499021 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placemente203-account-delete-sdpql" Dec 03 12:01:13 crc kubenswrapper[5052]: I1203 12:01:13.502067 5052 generic.go:334] "Generic (PLEG): container finished" podID="d03bf1c0-7353-4836-a767-9a5b2295e8dc" containerID="3a4d32fe7507dccb3b5a703d41f75ff7c20cb477aef43d8d0062e58cf58c09bf" exitCode=2 Dec 03 12:01:13 crc kubenswrapper[5052]: I1203 12:01:13.502101 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/kube-state-metrics-0" event={"ID":"d03bf1c0-7353-4836-a767-9a5b2295e8dc","Type":"ContainerDied","Data":"3a4d32fe7507dccb3b5a703d41f75ff7c20cb477aef43d8d0062e58cf58c09bf"} Dec 03 12:01:13 crc kubenswrapper[5052]: I1203 12:01:13.517713 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron0ea0-account-delete-lvffs" Dec 03 12:01:13 crc kubenswrapper[5052]: I1203 12:01:13.522455 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/novacell06be2-account-delete-7m8cr"] Dec 03 12:01:13 crc kubenswrapper[5052]: I1203 12:01:13.523969 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/novacell06be2-account-delete-7m8cr" Dec 03 12:01:13 crc kubenswrapper[5052]: I1203 12:01:13.566363 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/novacell06be2-account-delete-7m8cr"] Dec 03 12:01:13 crc kubenswrapper[5052]: I1203 12:01:13.587628 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/novaapif429-account-delete-qwz7z"] Dec 03 12:01:13 crc kubenswrapper[5052]: I1203 12:01:13.589043 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/novaapif429-account-delete-qwz7z" Dec 03 12:01:13 crc kubenswrapper[5052]: I1203 12:01:13.621106 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/neutron-db-sync-xvd7n"] Dec 03 12:01:13 crc kubenswrapper[5052]: I1203 12:01:13.635083 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/neutron-db-sync-xvd7n"] Dec 03 12:01:13 crc kubenswrapper[5052]: I1203 12:01:13.651847 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/novaapif429-account-delete-qwz7z"] Dec 03 12:01:13 crc kubenswrapper[5052]: I1203 12:01:13.667946 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9qndr\" (UniqueName: \"kubernetes.io/projected/6343f4a5-892f-4118-a666-e6601d2df818-kube-api-access-9qndr\") pod \"novacell06be2-account-delete-7m8cr\" (UID: \"6343f4a5-892f-4118-a666-e6601d2df818\") " pod="openstack-kuttl-tests/novacell06be2-account-delete-7m8cr" Dec 03 12:01:13 crc kubenswrapper[5052]: I1203 12:01:13.671614 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/swift-storage-0"] Dec 03 12:01:13 crc kubenswrapper[5052]: I1203 12:01:13.672185 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="7c36839d-8518-4b0e-bf63-ec94d92e3c13" containerName="account-server" containerID="cri-o://25010c622994b6f9bf074625c83c6f4b20f430bc358fcaedc8fb64c94ab3dc3a" gracePeriod=30 Dec 03 12:01:13 crc kubenswrapper[5052]: I1203 12:01:13.672303 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="7c36839d-8518-4b0e-bf63-ec94d92e3c13" containerName="object-replicator" containerID="cri-o://99e3b71ca2f828cfb4d8d9e730eaa81b9c4c624aca8e895c30c5e6fdf97517ac" gracePeriod=30 Dec 03 12:01:13 crc kubenswrapper[5052]: I1203 12:01:13.672323 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="7c36839d-8518-4b0e-bf63-ec94d92e3c13" containerName="object-server" containerID="cri-o://4066fa8ec273d30d5e0da93dde5867442a7c42b658e83fd69bdb6853a4596664" gracePeriod=30 Dec 03 12:01:13 crc kubenswrapper[5052]: I1203 12:01:13.672372 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="7c36839d-8518-4b0e-bf63-ec94d92e3c13" containerName="container-updater" containerID="cri-o://1baab415bc319f90ebf14e16f12a6406c7324f83d7e1edd6fce3f36ba0d17106" gracePeriod=30 Dec 03 12:01:13 crc kubenswrapper[5052]: I1203 12:01:13.672417 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="7c36839d-8518-4b0e-bf63-ec94d92e3c13" containerName="container-auditor" containerID="cri-o://8bf39629dcf79a88a7eec59f36c04fffa01173de57cefa2317a3fcbe70912276" gracePeriod=30 Dec 03 12:01:13 crc kubenswrapper[5052]: I1203 12:01:13.672455 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="7c36839d-8518-4b0e-bf63-ec94d92e3c13" containerName="container-replicator" containerID="cri-o://0fc7226ddfe9193d0d31fe7fe4a9dcc3ec094ff416d17f3fbe045f601f3a147d" gracePeriod=30 Dec 03 12:01:13 crc kubenswrapper[5052]: I1203 12:01:13.672520 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="7c36839d-8518-4b0e-bf63-ec94d92e3c13" containerName="container-server" containerID="cri-o://2cbddf37ed81f13769bd22ea5da02802984cddbc9008db7adf62cbd5f35a8c05" gracePeriod=30 Dec 03 12:01:13 crc kubenswrapper[5052]: I1203 12:01:13.672569 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="7c36839d-8518-4b0e-bf63-ec94d92e3c13" containerName="account-reaper" containerID="cri-o://11cecc9603bd65df20d6a71c4f44da191cdbb3a4f878b9e3695671d86d19bd72" gracePeriod=30 Dec 03 12:01:13 crc kubenswrapper[5052]: I1203 12:01:13.672615 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="7c36839d-8518-4b0e-bf63-ec94d92e3c13" containerName="account-auditor" containerID="cri-o://682960d7b8798b4f6d377b7a7a2fe296dcddb554fb6aa11cf50780880e5ef868" gracePeriod=30 Dec 03 12:01:13 crc kubenswrapper[5052]: I1203 12:01:13.672684 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="7c36839d-8518-4b0e-bf63-ec94d92e3c13" containerName="rsync" containerID="cri-o://52442374ae2d8fc1cfd375a7ed3be0f18021f36b0334d1758b377586b846921d" gracePeriod=30 Dec 03 12:01:13 crc kubenswrapper[5052]: I1203 12:01:13.672775 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="7c36839d-8518-4b0e-bf63-ec94d92e3c13" containerName="swift-recon-cron" containerID="cri-o://eac74c2a0f154d6d834ce77005c3de792b1ffb15c4c79891dc165810f2f43317" gracePeriod=30 Dec 03 12:01:13 crc kubenswrapper[5052]: I1203 12:01:13.672859 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="7c36839d-8518-4b0e-bf63-ec94d92e3c13" containerName="account-replicator" containerID="cri-o://109cd7db5617ce6f1e5aaea63acd2470d8ce489ee50ccc8c8d13a1422834ac45" gracePeriod=30 Dec 03 12:01:13 crc kubenswrapper[5052]: I1203 12:01:13.672935 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="7c36839d-8518-4b0e-bf63-ec94d92e3c13" containerName="object-updater" containerID="cri-o://ac3482f5fa4de80aab2650bc04e9d5264e54ea6c82aaf1d84113fd1b78ebc9e2" gracePeriod=30 Dec 03 12:01:13 crc kubenswrapper[5052]: I1203 12:01:13.672973 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="7c36839d-8518-4b0e-bf63-ec94d92e3c13" containerName="object-expirer" containerID="cri-o://fcf4f78f80f00487ca12528cc832dae0efc274f54e76db4429646d7410f72e60" gracePeriod=30 Dec 03 12:01:13 crc kubenswrapper[5052]: I1203 12:01:13.673023 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="7c36839d-8518-4b0e-bf63-ec94d92e3c13" containerName="object-auditor" containerID="cri-o://5f57677d7d2dc23e3fd4fa6e81bdcbe886b2bfe28287b6105f234e2a357ac510" gracePeriod=30 Dec 03 12:01:13 crc kubenswrapper[5052]: I1203 12:01:13.673502 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/6343f4a5-892f-4118-a666-e6601d2df818-operator-scripts\") pod \"novacell06be2-account-delete-7m8cr\" (UID: \"6343f4a5-892f-4118-a666-e6601d2df818\") " pod="openstack-kuttl-tests/novacell06be2-account-delete-7m8cr" Dec 03 12:01:13 crc kubenswrapper[5052]: I1203 12:01:13.716682 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="14cb8e46-418d-4c42-ae97-a384a05b432f" path="/var/lib/kubelet/pods/14cb8e46-418d-4c42-ae97-a384a05b432f/volumes" Dec 03 12:01:13 crc kubenswrapper[5052]: I1203 12:01:13.717915 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="16630eec-cd03-42b8-95aa-d9a960c2858c" path="/var/lib/kubelet/pods/16630eec-cd03-42b8-95aa-d9a960c2858c/volumes" Dec 03 12:01:13 crc kubenswrapper[5052]: I1203 12:01:13.718984 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="28c32800-797f-4560-ac08-55c9ca254a79" path="/var/lib/kubelet/pods/28c32800-797f-4560-ac08-55c9ca254a79/volumes" Dec 03 12:01:13 crc kubenswrapper[5052]: I1203 12:01:13.722273 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a50cd1a3-6eb7-4712-bb60-648250b396d6" path="/var/lib/kubelet/pods/a50cd1a3-6eb7-4712-bb60-648250b396d6/volumes" Dec 03 12:01:13 crc kubenswrapper[5052]: I1203 12:01:13.722890 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a626fb80-9d64-44c4-94e7-e0dde9638ffc" path="/var/lib/kubelet/pods/a626fb80-9d64-44c4-94e7-e0dde9638ffc/volumes" Dec 03 12:01:13 crc kubenswrapper[5052]: I1203 12:01:13.723776 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b25492d8-45bc-446a-a60c-b4f732484218" path="/var/lib/kubelet/pods/b25492d8-45bc-446a-a60c-b4f732484218/volumes" Dec 03 12:01:13 crc kubenswrapper[5052]: I1203 12:01:13.763236 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b3146921-cc78-401f-a918-fa8b91814f10" path="/var/lib/kubelet/pods/b3146921-cc78-401f-a918-fa8b91814f10/volumes" Dec 03 12:01:13 crc kubenswrapper[5052]: I1203 12:01:13.770966 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/novacell184d8-account-delete-4nmn9"] Dec 03 12:01:13 crc kubenswrapper[5052]: I1203 12:01:13.778300 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/eaeff688-fe65-403f-a01a-dfff69db9fb9-operator-scripts\") pod \"novaapif429-account-delete-qwz7z\" (UID: \"eaeff688-fe65-403f-a01a-dfff69db9fb9\") " pod="openstack-kuttl-tests/novaapif429-account-delete-qwz7z" Dec 03 12:01:13 crc kubenswrapper[5052]: I1203 12:01:13.778371 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/6343f4a5-892f-4118-a666-e6601d2df818-operator-scripts\") pod \"novacell06be2-account-delete-7m8cr\" (UID: \"6343f4a5-892f-4118-a666-e6601d2df818\") " pod="openstack-kuttl-tests/novacell06be2-account-delete-7m8cr" Dec 03 12:01:13 crc kubenswrapper[5052]: E1203 12:01:13.778865 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/rabbitmq-config-data: configmap "rabbitmq-config-data" not found Dec 03 12:01:13 crc kubenswrapper[5052]: E1203 12:01:13.778952 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/30578d71-fe2b-4eed-9a07-6c8bfe4a1c43-config-data podName:30578d71-fe2b-4eed-9a07-6c8bfe4a1c43 nodeName:}" failed. No retries permitted until 2025-12-03 12:01:15.778929026 +0000 UTC m=+2466.367170870 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "config-data" (UniqueName: "kubernetes.io/configmap/30578d71-fe2b-4eed-9a07-6c8bfe4a1c43-config-data") pod "rabbitmq-server-0" (UID: "30578d71-fe2b-4eed-9a07-6c8bfe4a1c43") : configmap "rabbitmq-config-data" not found Dec 03 12:01:13 crc kubenswrapper[5052]: I1203 12:01:13.779061 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-9qndr\" (UniqueName: \"kubernetes.io/projected/6343f4a5-892f-4118-a666-e6601d2df818-kube-api-access-9qndr\") pod \"novacell06be2-account-delete-7m8cr\" (UID: \"6343f4a5-892f-4118-a666-e6601d2df818\") " pod="openstack-kuttl-tests/novacell06be2-account-delete-7m8cr" Dec 03 12:01:13 crc kubenswrapper[5052]: I1203 12:01:13.779137 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mmqmq\" (UniqueName: \"kubernetes.io/projected/eaeff688-fe65-403f-a01a-dfff69db9fb9-kube-api-access-mmqmq\") pod \"novaapif429-account-delete-qwz7z\" (UID: \"eaeff688-fe65-403f-a01a-dfff69db9fb9\") " pod="openstack-kuttl-tests/novaapif429-account-delete-qwz7z" Dec 03 12:01:13 crc kubenswrapper[5052]: I1203 12:01:13.779558 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/6343f4a5-892f-4118-a666-e6601d2df818-operator-scripts\") pod \"novacell06be2-account-delete-7m8cr\" (UID: \"6343f4a5-892f-4118-a666-e6601d2df818\") " pod="openstack-kuttl-tests/novacell06be2-account-delete-7m8cr" Dec 03 12:01:13 crc kubenswrapper[5052]: I1203 12:01:13.779895 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 12:01:13 crc kubenswrapper[5052]: I1203 12:01:13.779939 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 12:01:13 crc kubenswrapper[5052]: I1203 12:01:13.779957 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/novacell184d8-account-delete-4nmn9"] Dec 03 12:01:13 crc kubenswrapper[5052]: I1203 12:01:13.779971 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-wfzg9"] Dec 03 12:01:13 crc kubenswrapper[5052]: I1203 12:01:13.785500 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/cinder-api-0" podUID="0291d0e7-9abf-48dd-8d5e-c329f989b700" containerName="cinder-api-log" containerID="cri-o://f877a20aa72fb38f5d985123667f657d88840787bd73214d01bf7253e2f1bf64" gracePeriod=30 Dec 03 12:01:13 crc kubenswrapper[5052]: I1203 12:01:13.785767 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/cinder-api-0" podUID="0291d0e7-9abf-48dd-8d5e-c329f989b700" containerName="cinder-api" containerID="cri-o://8ddd0225baa5c9e8b247c70bd8287b4bcb735d619e3bc97d77620dc899e06c7e" gracePeriod=30 Dec 03 12:01:13 crc kubenswrapper[5052]: I1203 12:01:13.786289 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/cinder-scheduler-0" podUID="2d2fda50-7118-43f2-8290-36d97b66de86" containerName="cinder-scheduler" containerID="cri-o://541d5fc878d9121d9e872cb4361abda24d819f62b553c1995d69dbf6c3ab83a1" gracePeriod=30 Dec 03 12:01:13 crc kubenswrapper[5052]: I1203 12:01:13.786786 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/novacell184d8-account-delete-4nmn9" Dec 03 12:01:13 crc kubenswrapper[5052]: I1203 12:01:13.786954 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/cinder-scheduler-0" podUID="2d2fda50-7118-43f2-8290-36d97b66de86" containerName="probe" containerID="cri-o://365e155e111e9968f8e4c88e8465be33bb14e93a08d5db3a41bced9b4391e4fd" gracePeriod=30 Dec 03 12:01:13 crc kubenswrapper[5052]: I1203 12:01:13.790434 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-wfzg9" Dec 03 12:01:13 crc kubenswrapper[5052]: I1203 12:01:13.801486 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-9qndr\" (UniqueName: \"kubernetes.io/projected/6343f4a5-892f-4118-a666-e6601d2df818-kube-api-access-9qndr\") pod \"novacell06be2-account-delete-7m8cr\" (UID: \"6343f4a5-892f-4118-a666-e6601d2df818\") " pod="openstack-kuttl-tests/novacell06be2-account-delete-7m8cr" Dec 03 12:01:13 crc kubenswrapper[5052]: I1203 12:01:13.868668 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-wfzg9"] Dec 03 12:01:13 crc kubenswrapper[5052]: I1203 12:01:13.874264 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/novacell06be2-account-delete-7m8cr" Dec 03 12:01:13 crc kubenswrapper[5052]: I1203 12:01:13.899860 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/cf90ea08-b172-4549-9522-92be7a5b5623-operator-scripts\") pod \"novacell184d8-account-delete-4nmn9\" (UID: \"cf90ea08-b172-4549-9522-92be7a5b5623\") " pod="openstack-kuttl-tests/novacell184d8-account-delete-4nmn9" Dec 03 12:01:13 crc kubenswrapper[5052]: I1203 12:01:13.911352 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-mmqmq\" (UniqueName: \"kubernetes.io/projected/eaeff688-fe65-403f-a01a-dfff69db9fb9-kube-api-access-mmqmq\") pod \"novaapif429-account-delete-qwz7z\" (UID: \"eaeff688-fe65-403f-a01a-dfff69db9fb9\") " pod="openstack-kuttl-tests/novaapif429-account-delete-qwz7z" Dec 03 12:01:13 crc kubenswrapper[5052]: I1203 12:01:13.911674 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/eaeff688-fe65-403f-a01a-dfff69db9fb9-operator-scripts\") pod \"novaapif429-account-delete-qwz7z\" (UID: \"eaeff688-fe65-403f-a01a-dfff69db9fb9\") " pod="openstack-kuttl-tests/novaapif429-account-delete-qwz7z" Dec 03 12:01:13 crc kubenswrapper[5052]: I1203 12:01:13.911805 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-228cq\" (UniqueName: \"kubernetes.io/projected/cf90ea08-b172-4549-9522-92be7a5b5623-kube-api-access-228cq\") pod \"novacell184d8-account-delete-4nmn9\" (UID: \"cf90ea08-b172-4549-9522-92be7a5b5623\") " pod="openstack-kuttl-tests/novacell184d8-account-delete-4nmn9" Dec 03 12:01:13 crc kubenswrapper[5052]: I1203 12:01:13.913355 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/eaeff688-fe65-403f-a01a-dfff69db9fb9-operator-scripts\") pod \"novaapif429-account-delete-qwz7z\" (UID: \"eaeff688-fe65-403f-a01a-dfff69db9fb9\") " pod="openstack-kuttl-tests/novaapif429-account-delete-qwz7z" Dec 03 12:01:13 crc kubenswrapper[5052]: I1203 12:01:13.913451 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:01:13 crc kubenswrapper[5052]: I1203 12:01:13.914295 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-external-api-0" podUID="51a9dfff-37f2-4fbe-b2d1-039f20bb4560" containerName="glance-log" containerID="cri-o://7f62ed7588549b9a67859ad18997d03e123b56ebc6c553d299ea1f897cb73ce2" gracePeriod=30 Dec 03 12:01:13 crc kubenswrapper[5052]: I1203 12:01:13.914945 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-external-api-0" podUID="51a9dfff-37f2-4fbe-b2d1-039f20bb4560" containerName="glance-httpd" containerID="cri-o://87bc47eac41558bc881491bf1b0295423263bce8a03bd87436868096e25a318d" gracePeriod=30 Dec 03 12:01:13 crc kubenswrapper[5052]: I1203 12:01:13.956701 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-mmqmq\" (UniqueName: \"kubernetes.io/projected/eaeff688-fe65-403f-a01a-dfff69db9fb9-kube-api-access-mmqmq\") pod \"novaapif429-account-delete-qwz7z\" (UID: \"eaeff688-fe65-403f-a01a-dfff69db9fb9\") " pod="openstack-kuttl-tests/novaapif429-account-delete-qwz7z" Dec 03 12:01:13 crc kubenswrapper[5052]: I1203 12:01:13.990920 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:01:13 crc kubenswrapper[5052]: I1203 12:01:13.991349 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-internal-api-0" podUID="6de7dc2a-f19c-4cc3-9965-9be822dc6270" containerName="glance-log" containerID="cri-o://220ad427ac37b8c14a8f2bf882e72550b22b51dfe6dbd65908d5362fb9eed124" gracePeriod=30 Dec 03 12:01:13 crc kubenswrapper[5052]: I1203 12:01:13.992661 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-internal-api-0" podUID="6de7dc2a-f19c-4cc3-9965-9be822dc6270" containerName="glance-httpd" containerID="cri-o://f562bb4258333809d9cdbdea6ecb675c5df538644d028c44e7b8954cc6a364c8" gracePeriod=30 Dec 03 12:01:14 crc kubenswrapper[5052]: I1203 12:01:14.015726 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/5105e4c1-412b-4c11-92db-2354ebcfc575-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-84b9f45d47-wfzg9\" (UID: \"5105e4c1-412b-4c11-92db-2354ebcfc575\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-wfzg9" Dec 03 12:01:14 crc kubenswrapper[5052]: I1203 12:01:14.015832 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-228cq\" (UniqueName: \"kubernetes.io/projected/cf90ea08-b172-4549-9522-92be7a5b5623-kube-api-access-228cq\") pod \"novacell184d8-account-delete-4nmn9\" (UID: \"cf90ea08-b172-4549-9522-92be7a5b5623\") " pod="openstack-kuttl-tests/novacell184d8-account-delete-4nmn9" Dec 03 12:01:14 crc kubenswrapper[5052]: I1203 12:01:14.015873 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5105e4c1-412b-4c11-92db-2354ebcfc575-config\") pod \"dnsmasq-dnsmasq-84b9f45d47-wfzg9\" (UID: \"5105e4c1-412b-4c11-92db-2354ebcfc575\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-wfzg9" Dec 03 12:01:14 crc kubenswrapper[5052]: I1203 12:01:14.015907 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vwfzr\" (UniqueName: \"kubernetes.io/projected/5105e4c1-412b-4c11-92db-2354ebcfc575-kube-api-access-vwfzr\") pod \"dnsmasq-dnsmasq-84b9f45d47-wfzg9\" (UID: \"5105e4c1-412b-4c11-92db-2354ebcfc575\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-wfzg9" Dec 03 12:01:14 crc kubenswrapper[5052]: I1203 12:01:14.016167 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/cf90ea08-b172-4549-9522-92be7a5b5623-operator-scripts\") pod \"novacell184d8-account-delete-4nmn9\" (UID: \"cf90ea08-b172-4549-9522-92be7a5b5623\") " pod="openstack-kuttl-tests/novacell184d8-account-delete-4nmn9" Dec 03 12:01:14 crc kubenswrapper[5052]: I1203 12:01:14.017326 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/cf90ea08-b172-4549-9522-92be7a5b5623-operator-scripts\") pod \"novacell184d8-account-delete-4nmn9\" (UID: \"cf90ea08-b172-4549-9522-92be7a5b5623\") " pod="openstack-kuttl-tests/novacell184d8-account-delete-4nmn9" Dec 03 12:01:14 crc kubenswrapper[5052]: I1203 12:01:14.062756 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-228cq\" (UniqueName: \"kubernetes.io/projected/cf90ea08-b172-4549-9522-92be7a5b5623-kube-api-access-228cq\") pod \"novacell184d8-account-delete-4nmn9\" (UID: \"cf90ea08-b172-4549-9522-92be7a5b5623\") " pod="openstack-kuttl-tests/novacell184d8-account-delete-4nmn9" Dec 03 12:01:14 crc kubenswrapper[5052]: I1203 12:01:14.074136 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/watcher-decision-engine-0"] Dec 03 12:01:14 crc kubenswrapper[5052]: I1203 12:01:14.074361 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/watcher-decision-engine-0" podUID="e9985bb3-f162-4793-9491-2e4f46644071" containerName="watcher-decision-engine" containerID="cri-o://1a61e630d65199a5ceab549e6965a4dfa5b829b5db1743b8843226f45bd3e896" gracePeriod=30 Dec 03 12:01:14 crc kubenswrapper[5052]: I1203 12:01:14.091870 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell1-cell-mapping-wsbvz"] Dec 03 12:01:14 crc kubenswrapper[5052]: I1203 12:01:14.101530 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell0-cell-mapping-fc2zk"] Dec 03 12:01:14 crc kubenswrapper[5052]: I1203 12:01:14.118916 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/5105e4c1-412b-4c11-92db-2354ebcfc575-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-84b9f45d47-wfzg9\" (UID: \"5105e4c1-412b-4c11-92db-2354ebcfc575\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-wfzg9" Dec 03 12:01:14 crc kubenswrapper[5052]: I1203 12:01:14.119367 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5105e4c1-412b-4c11-92db-2354ebcfc575-config\") pod \"dnsmasq-dnsmasq-84b9f45d47-wfzg9\" (UID: \"5105e4c1-412b-4c11-92db-2354ebcfc575\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-wfzg9" Dec 03 12:01:14 crc kubenswrapper[5052]: I1203 12:01:14.119435 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-vwfzr\" (UniqueName: \"kubernetes.io/projected/5105e4c1-412b-4c11-92db-2354ebcfc575-kube-api-access-vwfzr\") pod \"dnsmasq-dnsmasq-84b9f45d47-wfzg9\" (UID: \"5105e4c1-412b-4c11-92db-2354ebcfc575\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-wfzg9" Dec 03 12:01:14 crc kubenswrapper[5052]: I1203 12:01:14.123031 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5105e4c1-412b-4c11-92db-2354ebcfc575-config\") pod \"dnsmasq-dnsmasq-84b9f45d47-wfzg9\" (UID: \"5105e4c1-412b-4c11-92db-2354ebcfc575\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-wfzg9" Dec 03 12:01:14 crc kubenswrapper[5052]: I1203 12:01:14.123686 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/5105e4c1-412b-4c11-92db-2354ebcfc575-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-84b9f45d47-wfzg9\" (UID: \"5105e4c1-412b-4c11-92db-2354ebcfc575\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-wfzg9" Dec 03 12:01:14 crc kubenswrapper[5052]: I1203 12:01:14.142576 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell1-cell-mapping-wsbvz"] Dec 03 12:01:14 crc kubenswrapper[5052]: I1203 12:01:14.151851 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell0-cell-mapping-fc2zk"] Dec 03 12:01:14 crc kubenswrapper[5052]: I1203 12:01:14.162642 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinderaaa7-account-delete-rdrqp"] Dec 03 12:01:14 crc kubenswrapper[5052]: I1203 12:01:14.168387 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-vwfzr\" (UniqueName: \"kubernetes.io/projected/5105e4c1-412b-4c11-92db-2354ebcfc575-kube-api-access-vwfzr\") pod \"dnsmasq-dnsmasq-84b9f45d47-wfzg9\" (UID: \"5105e4c1-412b-4c11-92db-2354ebcfc575\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-wfzg9" Dec 03 12:01:14 crc kubenswrapper[5052]: I1203 12:01:14.173496 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/watcher-applier-0"] Dec 03 12:01:14 crc kubenswrapper[5052]: I1203 12:01:14.173751 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/watcher-applier-0" podUID="192fa797-c8b8-42d3-a689-e6a7dbc77282" containerName="watcher-applier" containerID="cri-o://0df8ce5588ea9142955dd14165fc02005b40b2952f5607fe472ebbf7177b2be8" gracePeriod=30 Dec 03 12:01:14 crc kubenswrapper[5052]: I1203 12:01:14.188563 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/placement-7d9769b856-tzkn2"] Dec 03 12:01:14 crc kubenswrapper[5052]: I1203 12:01:14.188906 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/placement-7d9769b856-tzkn2" podUID="60ca40b3-79c7-49cc-9100-0a37c47e4462" containerName="placement-log" containerID="cri-o://70d329d677837ae32ed6ea2cf9d76bcf62eb95f82373e7dfdeacdfcbca06156a" gracePeriod=30 Dec 03 12:01:14 crc kubenswrapper[5052]: I1203 12:01:14.189074 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/placement-7d9769b856-tzkn2" podUID="60ca40b3-79c7-49cc-9100-0a37c47e4462" containerName="placement-api" containerID="cri-o://eac316d032f9e574c9c5551e5d62e39ad5141fe5bd8033f5ce21da3b31105028" gracePeriod=30 Dec 03 12:01:14 crc kubenswrapper[5052]: I1203 12:01:14.201139 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/watcher-api-0"] Dec 03 12:01:14 crc kubenswrapper[5052]: I1203 12:01:14.201421 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/watcher-api-0" podUID="2d5669ca-8e5e-49b4-95e9-116eb1195030" containerName="watcher-api-log" containerID="cri-o://677fccb8442f04cfdcccbb93e98725dc995271445330b661ed4f10796a9c7c91" gracePeriod=30 Dec 03 12:01:14 crc kubenswrapper[5052]: I1203 12:01:14.201856 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/watcher-api-0" podUID="2d5669ca-8e5e-49b4-95e9-116eb1195030" containerName="watcher-api" containerID="cri-o://7fa07f1e6f3c12445900ff78caa499630cedc8446b5a3e6c5c977fa8f1760705" gracePeriod=30 Dec 03 12:01:14 crc kubenswrapper[5052]: I1203 12:01:14.211593 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/neutron-7f59bd7c8d-4jwhw"] Dec 03 12:01:14 crc kubenswrapper[5052]: I1203 12:01:14.211860 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/neutron-7f59bd7c8d-4jwhw" podUID="73f90998-d47e-407c-b964-12065411b4e2" containerName="neutron-api" containerID="cri-o://be99515097973c2c0916717764695e4147e9b00df227c2974f21e0db3fe72a0f" gracePeriod=30 Dec 03 12:01:14 crc kubenswrapper[5052]: I1203 12:01:14.211940 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/neutron-7f59bd7c8d-4jwhw" podUID="73f90998-d47e-407c-b964-12065411b4e2" containerName="neutron-httpd" containerID="cri-o://841d973c00ea29cca586a4e7b4174c1d2c994db38756378837d3f5c9d0346486" gracePeriod=30 Dec 03 12:01:14 crc kubenswrapper[5052]: I1203 12:01:14.221780 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/novaapif429-account-delete-qwz7z" Dec 03 12:01:14 crc kubenswrapper[5052]: I1203 12:01:14.225752 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/swift-proxy-6555cb554-n2dp6"] Dec 03 12:01:14 crc kubenswrapper[5052]: I1203 12:01:14.226062 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-proxy-6555cb554-n2dp6" podUID="8c848c79-8799-4e32-a635-958ce73341c4" containerName="proxy-httpd" containerID="cri-o://77bff56b6eb8629741796ab9f7892ba83978c309def196be2b0d6b39a27f09ec" gracePeriod=30 Dec 03 12:01:14 crc kubenswrapper[5052]: I1203 12:01:14.226769 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-proxy-6555cb554-n2dp6" podUID="8c848c79-8799-4e32-a635-958ce73341c4" containerName="proxy-server" containerID="cri-o://ac84406b1507746af79e8f6d313cc521e0407948593d4c617a53957cf7d38341" gracePeriod=30 Dec 03 12:01:14 crc kubenswrapper[5052]: I1203 12:01:14.251325 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance0d27-account-delete-brqv4"] Dec 03 12:01:14 crc kubenswrapper[5052]: I1203 12:01:14.275307 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/novacell184d8-account-delete-4nmn9" Dec 03 12:01:14 crc kubenswrapper[5052]: I1203 12:01:14.297583 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/rabbitmq-server-0"] Dec 03 12:01:14 crc kubenswrapper[5052]: I1203 12:01:14.313569 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/watcher36ab-account-delete-cvjb7"] Dec 03 12:01:14 crc kubenswrapper[5052]: I1203 12:01:14.323619 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-worker-ff9447cb9-nmz9j"] Dec 03 12:01:14 crc kubenswrapper[5052]: I1203 12:01:14.323878 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/barbican-worker-ff9447cb9-nmz9j" podUID="60f75996-dceb-4640-8fca-96fdcc03ebb9" containerName="barbican-worker-log" containerID="cri-o://3a24f581129d9e74e6cdc6ff9baf1728f121946baea46179b88abf5f28003c5f" gracePeriod=30 Dec 03 12:01:14 crc kubenswrapper[5052]: I1203 12:01:14.324268 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/barbican-worker-ff9447cb9-nmz9j" podUID="60f75996-dceb-4640-8fca-96fdcc03ebb9" containerName="barbican-worker" containerID="cri-o://5f92af45a915348c684ef49149ebdedefc7b5163a55e465615daf0ad05fa17b8" gracePeriod=30 Dec 03 12:01:14 crc kubenswrapper[5052]: I1203 12:01:14.338826 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-api-d6bc8866d-xrhs6"] Dec 03 12:01:14 crc kubenswrapper[5052]: I1203 12:01:14.339166 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/barbican-api-d6bc8866d-xrhs6" podUID="5785b60f-14d2-43c5-95e2-68741c6729f3" containerName="barbican-api-log" containerID="cri-o://9828c63a7f8943e7325b0971d75b7f4ef89e7d95366cacf404cd7f47ac39d61f" gracePeriod=30 Dec 03 12:01:14 crc kubenswrapper[5052]: I1203 12:01:14.339639 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/barbican-api-d6bc8866d-xrhs6" podUID="5785b60f-14d2-43c5-95e2-68741c6729f3" containerName="barbican-api" containerID="cri-o://a7bae4cc8cec979afcf2817d0be9cd0e1558b2c480f44fd3bf85f63c239d1a5c" gracePeriod=30 Dec 03 12:01:14 crc kubenswrapper[5052]: I1203 12:01:14.350891 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-keystone-listener-6774dbcdb8-wzwf7"] Dec 03 12:01:14 crc kubenswrapper[5052]: I1203 12:01:14.351183 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/barbican-keystone-listener-6774dbcdb8-wzwf7" podUID="e09397b4-fabb-4e9d-8695-669bf9f49c77" containerName="barbican-keystone-listener-log" containerID="cri-o://32933642a3f541b1b3569edbbb3a29c32da560890311d2484c68ee57f99463a4" gracePeriod=30 Dec 03 12:01:14 crc kubenswrapper[5052]: I1203 12:01:14.351691 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/barbican-keystone-listener-6774dbcdb8-wzwf7" podUID="e09397b4-fabb-4e9d-8695-669bf9f49c77" containerName="barbican-keystone-listener" containerID="cri-o://f80007b6133c4b4853d49d9ef314cd82a9852973ff7a6cf37381a6ca1b1ec967" gracePeriod=30 Dec 03 12:01:14 crc kubenswrapper[5052]: I1203 12:01:14.375705 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:01:14 crc kubenswrapper[5052]: I1203 12:01:14.375980 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-metadata-0" podUID="98d3c14e-7549-441a-9532-f3d75833cdf8" containerName="nova-metadata-log" containerID="cri-o://710c4f7c04a361b712543bf316631f84336d70d531c95a80fd6746ea4334640d" gracePeriod=30 Dec 03 12:01:14 crc kubenswrapper[5052]: I1203 12:01:14.376493 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-metadata-0" podUID="98d3c14e-7549-441a-9532-f3d75833cdf8" containerName="nova-metadata-metadata" containerID="cri-o://1c18a9d55257352dbf809914f4dca8da241a548437c4cbcf603a968646bbaaf6" gracePeriod=30 Dec 03 12:01:14 crc kubenswrapper[5052]: I1203 12:01:14.437945 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:01:14 crc kubenswrapper[5052]: I1203 12:01:14.438224 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-scheduler-0" podUID="73c1b1fb-5062-4c33-be17-72324f222396" containerName="nova-scheduler-scheduler" containerID="cri-o://fa90af8ae17f19b8e657ed6b90a3d32798eac71027682bd29946af1de563200d" gracePeriod=30 Dec 03 12:01:14 crc kubenswrapper[5052]: I1203 12:01:14.466227 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:01:14 crc kubenswrapper[5052]: I1203 12:01:14.467869 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-api-0" podUID="1103e975-cc9a-435b-adc9-5c6492e9c629" containerName="nova-api-api" containerID="cri-o://aebafbbccc45118d70ddef8e3f76333d5b32b3641a77534948d68d7de2fc9fd0" gracePeriod=30 Dec 03 12:01:14 crc kubenswrapper[5052]: I1203 12:01:14.467498 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-api-0" podUID="1103e975-cc9a-435b-adc9-5c6492e9c629" containerName="nova-api-log" containerID="cri-o://be465bb3373c78a7e0d0341f23e780fbf057154f4f4e5cd558d38b99cc20e088" gracePeriod=30 Dec 03 12:01:14 crc kubenswrapper[5052]: I1203 12:01:14.477145 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/prometheus-metric-storage-0"] Dec 03 12:01:14 crc kubenswrapper[5052]: I1203 12:01:14.477705 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/prometheus-metric-storage-0" podUID="01bc82de-61dc-477e-b4cf-e3b0d32c99ba" containerName="prometheus" containerID="cri-o://599972ea725b2448de0f384ff58d4263ae4c6939731529af770910368c748187" gracePeriod=600 Dec 03 12:01:14 crc kubenswrapper[5052]: I1203 12:01:14.478241 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/prometheus-metric-storage-0" podUID="01bc82de-61dc-477e-b4cf-e3b0d32c99ba" containerName="thanos-sidecar" containerID="cri-o://b2b8f2e1a2b0959c9d345484247345b165016e4ab99777de279ec024458c9da1" gracePeriod=600 Dec 03 12:01:14 crc kubenswrapper[5052]: I1203 12:01:14.478308 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/prometheus-metric-storage-0" podUID="01bc82de-61dc-477e-b4cf-e3b0d32c99ba" containerName="config-reloader" containerID="cri-o://3845a093071f0f8ca351aba36d4c3c56dd2d469ba1bfbea7b327c64e55ea871c" gracePeriod=600 Dec 03 12:01:14 crc kubenswrapper[5052]: I1203 12:01:14.498586 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/alertmanager-metric-storage-0"] Dec 03 12:01:14 crc kubenswrapper[5052]: I1203 12:01:14.499939 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/alertmanager-metric-storage-0" podUID="7cbbb6c5-a087-400f-9beb-e740ae1ddfbd" containerName="alertmanager" containerID="cri-o://ae36686b468c95feb23e421e5898d4b9d5542290763ef55cdda58e32e8d7665b" gracePeriod=120 Dec 03 12:01:14 crc kubenswrapper[5052]: I1203 12:01:14.500098 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/alertmanager-metric-storage-0" podUID="7cbbb6c5-a087-400f-9beb-e740ae1ddfbd" containerName="config-reloader" containerID="cri-o://079d8a9a5ad9c1f5d88409c04f6a13ba44666a756c6ea8ff7bd4d25cb9a1528e" gracePeriod=120 Dec 03 12:01:14 crc kubenswrapper[5052]: I1203 12:01:14.507311 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/rabbitmq-server-0" podUID="30578d71-fe2b-4eed-9a07-6c8bfe4a1c43" containerName="rabbitmq" containerID="cri-o://d9ddf262949f1f7939bc5d4f4522859474860ed8b5edac12e5534c72d138210a" gracePeriod=604800 Dec 03 12:01:14 crc kubenswrapper[5052]: I1203 12:01:14.508746 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 12:01:14 crc kubenswrapper[5052]: I1203 12:01:14.513568 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" podUID="914c0f8f-e5d1-4c63-b985-0bcd07f75374" containerName="nova-cell1-novncproxy-novncproxy" containerID="cri-o://2d2f9846f42dc135f23ee0343741d94b75b793c15135ba895cc30047a367cc1e" gracePeriod=30 Dec 03 12:01:14 crc kubenswrapper[5052]: I1203 12:01:14.523972 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-db-sync-nljtg"] Dec 03 12:01:14 crc kubenswrapper[5052]: I1203 12:01:14.525840 5052 generic.go:334] "Generic (PLEG): container finished" podID="73f90998-d47e-407c-b964-12065411b4e2" containerID="841d973c00ea29cca586a4e7b4174c1d2c994db38756378837d3f5c9d0346486" exitCode=0 Dec 03 12:01:14 crc kubenswrapper[5052]: I1203 12:01:14.525950 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-7f59bd7c8d-4jwhw" event={"ID":"73f90998-d47e-407c-b964-12065411b4e2","Type":"ContainerDied","Data":"841d973c00ea29cca586a4e7b4174c1d2c994db38756378837d3f5c9d0346486"} Dec 03 12:01:14 crc kubenswrapper[5052]: I1203 12:01:14.532061 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_ovsdbserver-nb-0_e8de8adf-e4e5-4a10-9722-9106357e67d0/ovsdbserver-nb/0.log" Dec 03 12:01:14 crc kubenswrapper[5052]: I1203 12:01:14.532111 5052 generic.go:334] "Generic (PLEG): container finished" podID="e8de8adf-e4e5-4a10-9722-9106357e67d0" containerID="156a9ab495313477724afb77cc40733731b002a3db4ec302faefc4c513160b71" exitCode=2 Dec 03 12:01:14 crc kubenswrapper[5052]: I1203 12:01:14.532128 5052 generic.go:334] "Generic (PLEG): container finished" podID="e8de8adf-e4e5-4a10-9722-9106357e67d0" containerID="a3deb4fed82228a5b22c0094ab0a2185263de5b8a00e34fbefb31c5da84f1d53" exitCode=143 Dec 03 12:01:14 crc kubenswrapper[5052]: I1203 12:01:14.532199 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-nb-0" event={"ID":"e8de8adf-e4e5-4a10-9722-9106357e67d0","Type":"ContainerDied","Data":"156a9ab495313477724afb77cc40733731b002a3db4ec302faefc4c513160b71"} Dec 03 12:01:14 crc kubenswrapper[5052]: I1203 12:01:14.532241 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-nb-0" event={"ID":"e8de8adf-e4e5-4a10-9722-9106357e67d0","Type":"ContainerDied","Data":"a3deb4fed82228a5b22c0094ab0a2185263de5b8a00e34fbefb31c5da84f1d53"} Dec 03 12:01:14 crc kubenswrapper[5052]: I1203 12:01:14.533817 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-db-sync-nljtg"] Dec 03 12:01:14 crc kubenswrapper[5052]: I1203 12:01:14.559301 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-0"] Dec 03 12:01:14 crc kubenswrapper[5052]: I1203 12:01:14.559477 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-cell1-conductor-0" podUID="0a480cd2-586d-4213-beff-424b3d201f8e" containerName="nova-cell1-conductor-conductor" containerID="cri-o://84a4a97816b15f9f8b42611faefec10e993e636d5d1d408805fe13fa7df1d13e" gracePeriod=30 Dec 03 12:01:14 crc kubenswrapper[5052]: I1203 12:01:14.579746 5052 generic.go:334] "Generic (PLEG): container finished" podID="7c36839d-8518-4b0e-bf63-ec94d92e3c13" containerID="52442374ae2d8fc1cfd375a7ed3be0f18021f36b0334d1758b377586b846921d" exitCode=0 Dec 03 12:01:14 crc kubenswrapper[5052]: I1203 12:01:14.579769 5052 generic.go:334] "Generic (PLEG): container finished" podID="7c36839d-8518-4b0e-bf63-ec94d92e3c13" containerID="fcf4f78f80f00487ca12528cc832dae0efc274f54e76db4429646d7410f72e60" exitCode=0 Dec 03 12:01:14 crc kubenswrapper[5052]: I1203 12:01:14.579778 5052 generic.go:334] "Generic (PLEG): container finished" podID="7c36839d-8518-4b0e-bf63-ec94d92e3c13" containerID="ac3482f5fa4de80aab2650bc04e9d5264e54ea6c82aaf1d84113fd1b78ebc9e2" exitCode=0 Dec 03 12:01:14 crc kubenswrapper[5052]: I1203 12:01:14.579788 5052 generic.go:334] "Generic (PLEG): container finished" podID="7c36839d-8518-4b0e-bf63-ec94d92e3c13" containerID="5f57677d7d2dc23e3fd4fa6e81bdcbe886b2bfe28287b6105f234e2a357ac510" exitCode=0 Dec 03 12:01:14 crc kubenswrapper[5052]: I1203 12:01:14.579795 5052 generic.go:334] "Generic (PLEG): container finished" podID="7c36839d-8518-4b0e-bf63-ec94d92e3c13" containerID="99e3b71ca2f828cfb4d8d9e730eaa81b9c4c624aca8e895c30c5e6fdf97517ac" exitCode=0 Dec 03 12:01:14 crc kubenswrapper[5052]: I1203 12:01:14.579802 5052 generic.go:334] "Generic (PLEG): container finished" podID="7c36839d-8518-4b0e-bf63-ec94d92e3c13" containerID="4066fa8ec273d30d5e0da93dde5867442a7c42b658e83fd69bdb6853a4596664" exitCode=0 Dec 03 12:01:14 crc kubenswrapper[5052]: I1203 12:01:14.579808 5052 generic.go:334] "Generic (PLEG): container finished" podID="7c36839d-8518-4b0e-bf63-ec94d92e3c13" containerID="1baab415bc319f90ebf14e16f12a6406c7324f83d7e1edd6fce3f36ba0d17106" exitCode=0 Dec 03 12:01:14 crc kubenswrapper[5052]: I1203 12:01:14.579795 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"7c36839d-8518-4b0e-bf63-ec94d92e3c13","Type":"ContainerDied","Data":"52442374ae2d8fc1cfd375a7ed3be0f18021f36b0334d1758b377586b846921d"} Dec 03 12:01:14 crc kubenswrapper[5052]: I1203 12:01:14.579855 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"7c36839d-8518-4b0e-bf63-ec94d92e3c13","Type":"ContainerDied","Data":"fcf4f78f80f00487ca12528cc832dae0efc274f54e76db4429646d7410f72e60"} Dec 03 12:01:14 crc kubenswrapper[5052]: I1203 12:01:14.579882 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"7c36839d-8518-4b0e-bf63-ec94d92e3c13","Type":"ContainerDied","Data":"ac3482f5fa4de80aab2650bc04e9d5264e54ea6c82aaf1d84113fd1b78ebc9e2"} Dec 03 12:01:14 crc kubenswrapper[5052]: I1203 12:01:14.579897 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"7c36839d-8518-4b0e-bf63-ec94d92e3c13","Type":"ContainerDied","Data":"5f57677d7d2dc23e3fd4fa6e81bdcbe886b2bfe28287b6105f234e2a357ac510"} Dec 03 12:01:14 crc kubenswrapper[5052]: I1203 12:01:14.580168 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"7c36839d-8518-4b0e-bf63-ec94d92e3c13","Type":"ContainerDied","Data":"99e3b71ca2f828cfb4d8d9e730eaa81b9c4c624aca8e895c30c5e6fdf97517ac"} Dec 03 12:01:14 crc kubenswrapper[5052]: I1203 12:01:14.580187 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"7c36839d-8518-4b0e-bf63-ec94d92e3c13","Type":"ContainerDied","Data":"4066fa8ec273d30d5e0da93dde5867442a7c42b658e83fd69bdb6853a4596664"} Dec 03 12:01:14 crc kubenswrapper[5052]: I1203 12:01:14.579816 5052 generic.go:334] "Generic (PLEG): container finished" podID="7c36839d-8518-4b0e-bf63-ec94d92e3c13" containerID="8bf39629dcf79a88a7eec59f36c04fffa01173de57cefa2317a3fcbe70912276" exitCode=0 Dec 03 12:01:14 crc kubenswrapper[5052]: I1203 12:01:14.580199 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"7c36839d-8518-4b0e-bf63-ec94d92e3c13","Type":"ContainerDied","Data":"1baab415bc319f90ebf14e16f12a6406c7324f83d7e1edd6fce3f36ba0d17106"} Dec 03 12:01:14 crc kubenswrapper[5052]: I1203 12:01:14.580264 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"7c36839d-8518-4b0e-bf63-ec94d92e3c13","Type":"ContainerDied","Data":"8bf39629dcf79a88a7eec59f36c04fffa01173de57cefa2317a3fcbe70912276"} Dec 03 12:01:14 crc kubenswrapper[5052]: I1203 12:01:14.580279 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"7c36839d-8518-4b0e-bf63-ec94d92e3c13","Type":"ContainerDied","Data":"0fc7226ddfe9193d0d31fe7fe4a9dcc3ec094ff416d17f3fbe045f601f3a147d"} Dec 03 12:01:14 crc kubenswrapper[5052]: I1203 12:01:14.580204 5052 generic.go:334] "Generic (PLEG): container finished" podID="7c36839d-8518-4b0e-bf63-ec94d92e3c13" containerID="0fc7226ddfe9193d0d31fe7fe4a9dcc3ec094ff416d17f3fbe045f601f3a147d" exitCode=0 Dec 03 12:01:14 crc kubenswrapper[5052]: I1203 12:01:14.580306 5052 generic.go:334] "Generic (PLEG): container finished" podID="7c36839d-8518-4b0e-bf63-ec94d92e3c13" containerID="2cbddf37ed81f13769bd22ea5da02802984cddbc9008db7adf62cbd5f35a8c05" exitCode=0 Dec 03 12:01:14 crc kubenswrapper[5052]: I1203 12:01:14.580324 5052 generic.go:334] "Generic (PLEG): container finished" podID="7c36839d-8518-4b0e-bf63-ec94d92e3c13" containerID="11cecc9603bd65df20d6a71c4f44da191cdbb3a4f878b9e3695671d86d19bd72" exitCode=0 Dec 03 12:01:14 crc kubenswrapper[5052]: I1203 12:01:14.580336 5052 generic.go:334] "Generic (PLEG): container finished" podID="7c36839d-8518-4b0e-bf63-ec94d92e3c13" containerID="682960d7b8798b4f6d377b7a7a2fe296dcddb554fb6aa11cf50780880e5ef868" exitCode=0 Dec 03 12:01:14 crc kubenswrapper[5052]: I1203 12:01:14.580356 5052 generic.go:334] "Generic (PLEG): container finished" podID="7c36839d-8518-4b0e-bf63-ec94d92e3c13" containerID="109cd7db5617ce6f1e5aaea63acd2470d8ce489ee50ccc8c8d13a1422834ac45" exitCode=0 Dec 03 12:01:14 crc kubenswrapper[5052]: I1203 12:01:14.580366 5052 generic.go:334] "Generic (PLEG): container finished" podID="7c36839d-8518-4b0e-bf63-ec94d92e3c13" containerID="25010c622994b6f9bf074625c83c6f4b20f430bc358fcaedc8fb64c94ab3dc3a" exitCode=0 Dec 03 12:01:14 crc kubenswrapper[5052]: I1203 12:01:14.580330 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"7c36839d-8518-4b0e-bf63-ec94d92e3c13","Type":"ContainerDied","Data":"2cbddf37ed81f13769bd22ea5da02802984cddbc9008db7adf62cbd5f35a8c05"} Dec 03 12:01:14 crc kubenswrapper[5052]: I1203 12:01:14.580434 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"7c36839d-8518-4b0e-bf63-ec94d92e3c13","Type":"ContainerDied","Data":"11cecc9603bd65df20d6a71c4f44da191cdbb3a4f878b9e3695671d86d19bd72"} Dec 03 12:01:14 crc kubenswrapper[5052]: I1203 12:01:14.580445 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"7c36839d-8518-4b0e-bf63-ec94d92e3c13","Type":"ContainerDied","Data":"682960d7b8798b4f6d377b7a7a2fe296dcddb554fb6aa11cf50780880e5ef868"} Dec 03 12:01:14 crc kubenswrapper[5052]: I1203 12:01:14.580456 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"7c36839d-8518-4b0e-bf63-ec94d92e3c13","Type":"ContainerDied","Data":"109cd7db5617ce6f1e5aaea63acd2470d8ce489ee50ccc8c8d13a1422834ac45"} Dec 03 12:01:14 crc kubenswrapper[5052]: I1203 12:01:14.580465 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"7c36839d-8518-4b0e-bf63-ec94d92e3c13","Type":"ContainerDied","Data":"25010c622994b6f9bf074625c83c6f4b20f430bc358fcaedc8fb64c94ab3dc3a"} Dec 03 12:01:14 crc kubenswrapper[5052]: I1203 12:01:14.586746 5052 generic.go:334] "Generic (PLEG): container finished" podID="0291d0e7-9abf-48dd-8d5e-c329f989b700" containerID="f877a20aa72fb38f5d985123667f657d88840787bd73214d01bf7253e2f1bf64" exitCode=143 Dec 03 12:01:14 crc kubenswrapper[5052]: I1203 12:01:14.586801 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"0291d0e7-9abf-48dd-8d5e-c329f989b700","Type":"ContainerDied","Data":"f877a20aa72fb38f5d985123667f657d88840787bd73214d01bf7253e2f1bf64"} Dec 03 12:01:14 crc kubenswrapper[5052]: I1203 12:01:14.589003 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0bd547f7d1add0ad740d3fbbaadbb8e4bdc5956b2340bb46b827236dfdb14353" Dec 03 12:01:14 crc kubenswrapper[5052]: I1203 12:01:14.594797 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-db-sync-dlzcc"] Dec 03 12:01:14 crc kubenswrapper[5052]: I1203 12:01:14.596603 5052 generic.go:334] "Generic (PLEG): container finished" podID="e09397b4-fabb-4e9d-8695-669bf9f49c77" containerID="32933642a3f541b1b3569edbbb3a29c32da560890311d2484c68ee57f99463a4" exitCode=143 Dec 03 12:01:14 crc kubenswrapper[5052]: I1203 12:01:14.596681 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-keystone-listener-6774dbcdb8-wzwf7" event={"ID":"e09397b4-fabb-4e9d-8695-669bf9f49c77","Type":"ContainerDied","Data":"32933642a3f541b1b3569edbbb3a29c32da560890311d2484c68ee57f99463a4"} Dec 03 12:01:14 crc kubenswrapper[5052]: I1203 12:01:14.606184 5052 generic.go:334] "Generic (PLEG): container finished" podID="8c848c79-8799-4e32-a635-958ce73341c4" containerID="77bff56b6eb8629741796ab9f7892ba83978c309def196be2b0d6b39a27f09ec" exitCode=0 Dec 03 12:01:14 crc kubenswrapper[5052]: I1203 12:01:14.606298 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-proxy-6555cb554-n2dp6" event={"ID":"8c848c79-8799-4e32-a635-958ce73341c4","Type":"ContainerDied","Data":"77bff56b6eb8629741796ab9f7892ba83978c309def196be2b0d6b39a27f09ec"} Dec 03 12:01:14 crc kubenswrapper[5052]: I1203 12:01:14.608558 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-db-sync-dlzcc"] Dec 03 12:01:14 crc kubenswrapper[5052]: I1203 12:01:14.632322 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/kube-state-metrics-0" event={"ID":"d03bf1c0-7353-4836-a767-9a5b2295e8dc","Type":"ContainerDied","Data":"5200bd8a659d963e7653504e706b0695a96a1bbd01693588e9d6f0596b8e6de1"} Dec 03 12:01:14 crc kubenswrapper[5052]: I1203 12:01:14.632365 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5200bd8a659d963e7653504e706b0695a96a1bbd01693588e9d6f0596b8e6de1" Dec 03 12:01:14 crc kubenswrapper[5052]: I1203 12:01:14.634259 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinderaaa7-account-delete-rdrqp" event={"ID":"492ec2b7-0c28-4fe2-b2be-4e9e58423f23","Type":"ContainerStarted","Data":"015cccbd4c1596494b8217997ac4a87be227da3e9b2b7729dd71f4bc53db28b6"} Dec 03 12:01:14 crc kubenswrapper[5052]: I1203 12:01:14.634294 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinderaaa7-account-delete-rdrqp" event={"ID":"492ec2b7-0c28-4fe2-b2be-4e9e58423f23","Type":"ContainerStarted","Data":"44f47cb05e42046d5daee7d260ecc89c1c45fb1139d010bcbf92cec26ea65196"} Dec 03 12:01:14 crc kubenswrapper[5052]: I1203 12:01:14.657422 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican4aeb-account-delete-bvhdj"] Dec 03 12:01:14 crc kubenswrapper[5052]: I1203 12:01:14.665198 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/cinderaaa7-account-delete-rdrqp" podStartSLOduration=3.665170613 podStartE2EDuration="3.665170613s" podCreationTimestamp="2025-12-03 12:01:11 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:01:14.654494808 +0000 UTC m=+2465.242736572" watchObservedRunningTime="2025-12-03 12:01:14.665170613 +0000 UTC m=+2465.253412377" Dec 03 12:01:14 crc kubenswrapper[5052]: I1203 12:01:14.674699 5052 generic.go:334] "Generic (PLEG): container finished" podID="da4dfbea-20a2-4638-bbbe-ed4bfee0ad49" containerID="1658de9b62f64e764364ab2fb2ba523e5ca2b9210d45094169d4d8ec956ac253" exitCode=0 Dec 03 12:01:14 crc kubenswrapper[5052]: I1203 12:01:14.674835 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"da4dfbea-20a2-4638-bbbe-ed4bfee0ad49","Type":"ContainerDied","Data":"1658de9b62f64e764364ab2fb2ba523e5ca2b9210d45094169d4d8ec956ac253"} Dec 03 12:01:14 crc kubenswrapper[5052]: I1203 12:01:14.682061 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_ovn-northd-0_5e5ddf8c-6109-481f-beb0-0802de034912/ovn-northd/0.log" Dec 03 12:01:14 crc kubenswrapper[5052]: I1203 12:01:14.682132 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovn-northd-0" event={"ID":"5e5ddf8c-6109-481f-beb0-0802de034912","Type":"ContainerDied","Data":"0ad383e8c3943932d2c7d098b04c0f60d89e57f30cd3e7c590b8a688a1852e9d"} Dec 03 12:01:14 crc kubenswrapper[5052]: I1203 12:01:14.682160 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0ad383e8c3943932d2c7d098b04c0f60d89e57f30cd3e7c590b8a688a1852e9d" Dec 03 12:01:14 crc kubenswrapper[5052]: I1203 12:01:14.684182 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance0d27-account-delete-brqv4" event={"ID":"1a97695f-322c-4c4c-834e-ecb2f27c2e97","Type":"ContainerStarted","Data":"1028d16ccbe5cc7f8fc751b118eb9f0f6a94f366ef81a2b38d3bfb04f93ca1fc"} Dec 03 12:01:14 crc kubenswrapper[5052]: I1203 12:01:14.705428 5052 generic.go:334] "Generic (PLEG): container finished" podID="60ca40b3-79c7-49cc-9100-0a37c47e4462" containerID="70d329d677837ae32ed6ea2cf9d76bcf62eb95f82373e7dfdeacdfcbca06156a" exitCode=143 Dec 03 12:01:14 crc kubenswrapper[5052]: I1203 12:01:14.705577 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-7d9769b856-tzkn2" event={"ID":"60ca40b3-79c7-49cc-9100-0a37c47e4462","Type":"ContainerDied","Data":"70d329d677837ae32ed6ea2cf9d76bcf62eb95f82373e7dfdeacdfcbca06156a"} Dec 03 12:01:14 crc kubenswrapper[5052]: I1203 12:01:14.714488 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_ovsdbserver-sb-0_3cb4559f-542a-4076-82fb-c0ca88f9de94/ovsdbserver-sb/0.log" Dec 03 12:01:14 crc kubenswrapper[5052]: I1203 12:01:14.714595 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-sb-0" event={"ID":"3cb4559f-542a-4076-82fb-c0ca88f9de94","Type":"ContainerDied","Data":"d43b0348a840f00bd4fe9d9e2af0a2be437725e748e98b1e97bf216455be5575"} Dec 03 12:01:14 crc kubenswrapper[5052]: I1203 12:01:14.714622 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d43b0348a840f00bd4fe9d9e2af0a2be437725e748e98b1e97bf216455be5575" Dec 03 12:01:14 crc kubenswrapper[5052]: I1203 12:01:14.719613 5052 generic.go:334] "Generic (PLEG): container finished" podID="2d5669ca-8e5e-49b4-95e9-116eb1195030" containerID="677fccb8442f04cfdcccbb93e98725dc995271445330b661ed4f10796a9c7c91" exitCode=143 Dec 03 12:01:14 crc kubenswrapper[5052]: I1203 12:01:14.719667 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/watcher-api-0" event={"ID":"2d5669ca-8e5e-49b4-95e9-116eb1195030","Type":"ContainerDied","Data":"677fccb8442f04cfdcccbb93e98725dc995271445330b661ed4f10796a9c7c91"} Dec 03 12:01:14 crc kubenswrapper[5052]: I1203 12:01:14.728099 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron0ea0-account-delete-lvffs"] Dec 03 12:01:14 crc kubenswrapper[5052]: I1203 12:01:14.737669 5052 generic.go:334] "Generic (PLEG): container finished" podID="6de7dc2a-f19c-4cc3-9965-9be822dc6270" containerID="220ad427ac37b8c14a8f2bf882e72550b22b51dfe6dbd65908d5362fb9eed124" exitCode=143 Dec 03 12:01:14 crc kubenswrapper[5052]: I1203 12:01:14.737715 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"6de7dc2a-f19c-4cc3-9965-9be822dc6270","Type":"ContainerDied","Data":"220ad427ac37b8c14a8f2bf882e72550b22b51dfe6dbd65908d5362fb9eed124"} Dec 03 12:01:14 crc kubenswrapper[5052]: I1203 12:01:14.750946 5052 generic.go:334] "Generic (PLEG): container finished" podID="51a9dfff-37f2-4fbe-b2d1-039f20bb4560" containerID="7f62ed7588549b9a67859ad18997d03e123b56ebc6c553d299ea1f897cb73ce2" exitCode=143 Dec 03 12:01:14 crc kubenswrapper[5052]: I1203 12:01:14.751036 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"51a9dfff-37f2-4fbe-b2d1-039f20bb4560","Type":"ContainerDied","Data":"7f62ed7588549b9a67859ad18997d03e123b56ebc6c553d299ea1f897cb73ce2"} Dec 03 12:01:14 crc kubenswrapper[5052]: I1203 12:01:14.757805 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/placemente203-account-delete-sdpql"] Dec 03 12:01:14 crc kubenswrapper[5052]: I1203 12:01:14.765836 5052 generic.go:334] "Generic (PLEG): container finished" podID="5785b60f-14d2-43c5-95e2-68741c6729f3" containerID="9828c63a7f8943e7325b0971d75b7f4ef89e7d95366cacf404cd7f47ac39d61f" exitCode=143 Dec 03 12:01:14 crc kubenswrapper[5052]: I1203 12:01:14.765935 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-d6bc8866d-xrhs6" event={"ID":"5785b60f-14d2-43c5-95e2-68741c6729f3","Type":"ContainerDied","Data":"9828c63a7f8943e7325b0971d75b7f4ef89e7d95366cacf404cd7f47ac39d61f"} Dec 03 12:01:14 crc kubenswrapper[5052]: I1203 12:01:14.768191 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/watcher36ab-account-delete-cvjb7" event={"ID":"b2a6be10-e7df-494c-be94-a8ede55e2868","Type":"ContainerStarted","Data":"b55a7c9c0998b40f49c2aa6b63362830feb7cfe0c74271745c31355e9b24b4e3"} Dec 03 12:01:14 crc kubenswrapper[5052]: I1203 12:01:14.770830 5052 generic.go:334] "Generic (PLEG): container finished" podID="60f75996-dceb-4640-8fca-96fdcc03ebb9" containerID="3a24f581129d9e74e6cdc6ff9baf1728f121946baea46179b88abf5f28003c5f" exitCode=143 Dec 03 12:01:14 crc kubenswrapper[5052]: I1203 12:01:14.770882 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-worker-ff9447cb9-nmz9j" event={"ID":"60f75996-dceb-4640-8fca-96fdcc03ebb9","Type":"ContainerDied","Data":"3a24f581129d9e74e6cdc6ff9baf1728f121946baea46179b88abf5f28003c5f"} Dec 03 12:01:14 crc kubenswrapper[5052]: E1203 12:01:14.869822 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/combined-ca-bundle: secret "combined-ca-bundle" not found Dec 03 12:01:14 crc kubenswrapper[5052]: E1203 12:01:14.870085 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1103e975-cc9a-435b-adc9-5c6492e9c629-combined-ca-bundle podName:1103e975-cc9a-435b-adc9-5c6492e9c629 nodeName:}" failed. No retries permitted until 2025-12-03 12:01:18.870070501 +0000 UTC m=+2469.458312265 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "combined-ca-bundle" (UniqueName: "kubernetes.io/secret/1103e975-cc9a-435b-adc9-5c6492e9c629-combined-ca-bundle") pod "nova-api-0" (UID: "1103e975-cc9a-435b-adc9-5c6492e9c629") : secret "combined-ca-bundle" not found Dec 03 12:01:14 crc kubenswrapper[5052]: I1203 12:01:14.895280 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstackclient" Dec 03 12:01:14 crc kubenswrapper[5052]: I1203 12:01:14.895759 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-wfzg9" Dec 03 12:01:14 crc kubenswrapper[5052]: I1203 12:01:14.991340 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:01:15 crc kubenswrapper[5052]: I1203 12:01:15.099176 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hz667\" (UniqueName: \"kubernetes.io/projected/05236591-6a13-4ff8-bc2f-2016649fa3fc-kube-api-access-hz667\") pod \"05236591-6a13-4ff8-bc2f-2016649fa3fc\" (UID: \"05236591-6a13-4ff8-bc2f-2016649fa3fc\") " Dec 03 12:01:15 crc kubenswrapper[5052]: I1203 12:01:15.099288 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"openstack-config\" (UniqueName: \"kubernetes.io/configmap/05236591-6a13-4ff8-bc2f-2016649fa3fc-openstack-config\") pod \"05236591-6a13-4ff8-bc2f-2016649fa3fc\" (UID: \"05236591-6a13-4ff8-bc2f-2016649fa3fc\") " Dec 03 12:01:15 crc kubenswrapper[5052]: I1203 12:01:15.099404 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qw5db\" (UniqueName: \"kubernetes.io/projected/d03bf1c0-7353-4836-a767-9a5b2295e8dc-kube-api-access-qw5db\") pod \"d03bf1c0-7353-4836-a767-9a5b2295e8dc\" (UID: \"d03bf1c0-7353-4836-a767-9a5b2295e8dc\") " Dec 03 12:01:15 crc kubenswrapper[5052]: I1203 12:01:15.099488 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-state-metrics-tls-certs\" (UniqueName: \"kubernetes.io/secret/d03bf1c0-7353-4836-a767-9a5b2295e8dc-kube-state-metrics-tls-certs\") pod \"d03bf1c0-7353-4836-a767-9a5b2295e8dc\" (UID: \"d03bf1c0-7353-4836-a767-9a5b2295e8dc\") " Dec 03 12:01:15 crc kubenswrapper[5052]: I1203 12:01:15.099666 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"openstack-config-secret\" (UniqueName: \"kubernetes.io/secret/05236591-6a13-4ff8-bc2f-2016649fa3fc-openstack-config-secret\") pod \"05236591-6a13-4ff8-bc2f-2016649fa3fc\" (UID: \"05236591-6a13-4ff8-bc2f-2016649fa3fc\") " Dec 03 12:01:15 crc kubenswrapper[5052]: I1203 12:01:15.099728 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d03bf1c0-7353-4836-a767-9a5b2295e8dc-combined-ca-bundle\") pod \"d03bf1c0-7353-4836-a767-9a5b2295e8dc\" (UID: \"d03bf1c0-7353-4836-a767-9a5b2295e8dc\") " Dec 03 12:01:15 crc kubenswrapper[5052]: I1203 12:01:15.099931 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/05236591-6a13-4ff8-bc2f-2016649fa3fc-combined-ca-bundle\") pod \"05236591-6a13-4ff8-bc2f-2016649fa3fc\" (UID: \"05236591-6a13-4ff8-bc2f-2016649fa3fc\") " Dec 03 12:01:15 crc kubenswrapper[5052]: I1203 12:01:15.100042 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-state-metrics-tls-config\" (UniqueName: \"kubernetes.io/secret/d03bf1c0-7353-4836-a767-9a5b2295e8dc-kube-state-metrics-tls-config\") pod \"d03bf1c0-7353-4836-a767-9a5b2295e8dc\" (UID: \"d03bf1c0-7353-4836-a767-9a5b2295e8dc\") " Dec 03 12:01:15 crc kubenswrapper[5052]: I1203 12:01:15.117992 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/05236591-6a13-4ff8-bc2f-2016649fa3fc-kube-api-access-hz667" (OuterVolumeSpecName: "kube-api-access-hz667") pod "05236591-6a13-4ff8-bc2f-2016649fa3fc" (UID: "05236591-6a13-4ff8-bc2f-2016649fa3fc"). InnerVolumeSpecName "kube-api-access-hz667". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:01:15 crc kubenswrapper[5052]: E1203 12:01:15.120993 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of e4b74aed237af57c4a5ccac244bde0abf28abccef9d3f249bad7df1bbb43da9d is running failed: container process not found" containerID="e4b74aed237af57c4a5ccac244bde0abf28abccef9d3f249bad7df1bbb43da9d" cmd=["/usr/local/bin/container-scripts/status_check.sh"] Dec 03 12:01:15 crc kubenswrapper[5052]: E1203 12:01:15.121459 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of e4b74aed237af57c4a5ccac244bde0abf28abccef9d3f249bad7df1bbb43da9d is running failed: container process not found" containerID="e4b74aed237af57c4a5ccac244bde0abf28abccef9d3f249bad7df1bbb43da9d" cmd=["/usr/local/bin/container-scripts/status_check.sh"] Dec 03 12:01:15 crc kubenswrapper[5052]: E1203 12:01:15.121745 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of e4b74aed237af57c4a5ccac244bde0abf28abccef9d3f249bad7df1bbb43da9d is running failed: container process not found" containerID="e4b74aed237af57c4a5ccac244bde0abf28abccef9d3f249bad7df1bbb43da9d" cmd=["/usr/local/bin/container-scripts/status_check.sh"] Dec 03 12:01:15 crc kubenswrapper[5052]: E1203 12:01:15.121771 5052 prober.go:104] "Probe errored" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of e4b74aed237af57c4a5ccac244bde0abf28abccef9d3f249bad7df1bbb43da9d is running failed: container process not found" probeType="Readiness" pod="openstack-kuttl-tests/ovn-northd-0" podUID="5e5ddf8c-6109-481f-beb0-0802de034912" containerName="ovn-northd" Dec 03 12:01:15 crc kubenswrapper[5052]: I1203 12:01:15.175601 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/05236591-6a13-4ff8-bc2f-2016649fa3fc-openstack-config" (OuterVolumeSpecName: "openstack-config") pod "05236591-6a13-4ff8-bc2f-2016649fa3fc" (UID: "05236591-6a13-4ff8-bc2f-2016649fa3fc"). InnerVolumeSpecName "openstack-config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:01:15 crc kubenswrapper[5052]: I1203 12:01:15.188829 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d03bf1c0-7353-4836-a767-9a5b2295e8dc-kube-api-access-qw5db" (OuterVolumeSpecName: "kube-api-access-qw5db") pod "d03bf1c0-7353-4836-a767-9a5b2295e8dc" (UID: "d03bf1c0-7353-4836-a767-9a5b2295e8dc"). InnerVolumeSpecName "kube-api-access-qw5db". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:01:15 crc kubenswrapper[5052]: I1203 12:01:15.192921 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d03bf1c0-7353-4836-a767-9a5b2295e8dc-kube-state-metrics-tls-config" (OuterVolumeSpecName: "kube-state-metrics-tls-config") pod "d03bf1c0-7353-4836-a767-9a5b2295e8dc" (UID: "d03bf1c0-7353-4836-a767-9a5b2295e8dc"). InnerVolumeSpecName "kube-state-metrics-tls-config". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:01:15 crc kubenswrapper[5052]: I1203 12:01:15.214250 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/05236591-6a13-4ff8-bc2f-2016649fa3fc-openstack-config-secret" (OuterVolumeSpecName: "openstack-config-secret") pod "05236591-6a13-4ff8-bc2f-2016649fa3fc" (UID: "05236591-6a13-4ff8-bc2f-2016649fa3fc"). InnerVolumeSpecName "openstack-config-secret". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:01:15 crc kubenswrapper[5052]: I1203 12:01:15.215957 5052 reconciler_common.go:293] "Volume detached for volume \"kube-state-metrics-tls-config\" (UniqueName: \"kubernetes.io/secret/d03bf1c0-7353-4836-a767-9a5b2295e8dc-kube-state-metrics-tls-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:15 crc kubenswrapper[5052]: I1203 12:01:15.216000 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-hz667\" (UniqueName: \"kubernetes.io/projected/05236591-6a13-4ff8-bc2f-2016649fa3fc-kube-api-access-hz667\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:15 crc kubenswrapper[5052]: I1203 12:01:15.216010 5052 reconciler_common.go:293] "Volume detached for volume \"openstack-config\" (UniqueName: \"kubernetes.io/configmap/05236591-6a13-4ff8-bc2f-2016649fa3fc-openstack-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:15 crc kubenswrapper[5052]: I1203 12:01:15.216019 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-qw5db\" (UniqueName: \"kubernetes.io/projected/d03bf1c0-7353-4836-a767-9a5b2295e8dc-kube-api-access-qw5db\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:15 crc kubenswrapper[5052]: I1203 12:01:15.216028 5052 reconciler_common.go:293] "Volume detached for volume \"openstack-config-secret\" (UniqueName: \"kubernetes.io/secret/05236591-6a13-4ff8-bc2f-2016649fa3fc-openstack-config-secret\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:15 crc kubenswrapper[5052]: I1203 12:01:15.225046 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d03bf1c0-7353-4836-a767-9a5b2295e8dc-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "d03bf1c0-7353-4836-a767-9a5b2295e8dc" (UID: "d03bf1c0-7353-4836-a767-9a5b2295e8dc"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:01:15 crc kubenswrapper[5052]: I1203 12:01:15.231940 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/05236591-6a13-4ff8-bc2f-2016649fa3fc-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "05236591-6a13-4ff8-bc2f-2016649fa3fc" (UID: "05236591-6a13-4ff8-bc2f-2016649fa3fc"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:01:15 crc kubenswrapper[5052]: I1203 12:01:15.297219 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d03bf1c0-7353-4836-a767-9a5b2295e8dc-kube-state-metrics-tls-certs" (OuterVolumeSpecName: "kube-state-metrics-tls-certs") pod "d03bf1c0-7353-4836-a767-9a5b2295e8dc" (UID: "d03bf1c0-7353-4836-a767-9a5b2295e8dc"). InnerVolumeSpecName "kube-state-metrics-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:01:15 crc kubenswrapper[5052]: I1203 12:01:15.320179 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d03bf1c0-7353-4836-a767-9a5b2295e8dc-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:15 crc kubenswrapper[5052]: I1203 12:01:15.320473 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/05236591-6a13-4ff8-bc2f-2016649fa3fc-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:15 crc kubenswrapper[5052]: I1203 12:01:15.320594 5052 reconciler_common.go:293] "Volume detached for volume \"kube-state-metrics-tls-certs\" (UniqueName: \"kubernetes.io/secret/d03bf1c0-7353-4836-a767-9a5b2295e8dc-kube-state-metrics-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:15 crc kubenswrapper[5052]: I1203 12:01:15.326283 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/novaapif429-account-delete-qwz7z"] Dec 03 12:01:15 crc kubenswrapper[5052]: I1203 12:01:15.482203 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/novacell06be2-account-delete-7m8cr"] Dec 03 12:01:15 crc kubenswrapper[5052]: I1203 12:01:15.493764 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/novacell184d8-account-delete-4nmn9"] Dec 03 12:01:15 crc kubenswrapper[5052]: W1203 12:01:15.534871 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6343f4a5_892f_4118_a666_e6601d2df818.slice/crio-8273a87287f1d179a43891b8da0c2941ee49856e411b0cf9a44a8684b07521ed WatchSource:0}: Error finding container 8273a87287f1d179a43891b8da0c2941ee49856e411b0cf9a44a8684b07521ed: Status 404 returned error can't find the container with id 8273a87287f1d179a43891b8da0c2941ee49856e411b0cf9a44a8684b07521ed Dec 03 12:01:15 crc kubenswrapper[5052]: I1203 12:01:15.588127 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_ovn-northd-0_5e5ddf8c-6109-481f-beb0-0802de034912/ovn-northd/0.log" Dec 03 12:01:15 crc kubenswrapper[5052]: I1203 12:01:15.589097 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:01:15 crc kubenswrapper[5052]: I1203 12:01:15.634031 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_ovsdbserver-sb-0_3cb4559f-542a-4076-82fb-c0ca88f9de94/ovsdbserver-sb/0.log" Dec 03 12:01:15 crc kubenswrapper[5052]: I1203 12:01:15.634124 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:01:15 crc kubenswrapper[5052]: I1203 12:01:15.669323 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_ovsdbserver-nb-0_e8de8adf-e4e5-4a10-9722-9106357e67d0/ovsdbserver-nb/0.log" Dec 03 12:01:15 crc kubenswrapper[5052]: I1203 12:01:15.669412 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:01:15 crc kubenswrapper[5052]: I1203 12:01:15.701059 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="03284adf-04cd-4ca4-8b31-7ae483082bba" path="/var/lib/kubelet/pods/03284adf-04cd-4ca4-8b31-7ae483082bba/volumes" Dec 03 12:01:15 crc kubenswrapper[5052]: I1203 12:01:15.701653 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="05236591-6a13-4ff8-bc2f-2016649fa3fc" path="/var/lib/kubelet/pods/05236591-6a13-4ff8-bc2f-2016649fa3fc/volumes" Dec 03 12:01:15 crc kubenswrapper[5052]: I1203 12:01:15.702152 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="091d6975-4400-4dde-b61c-bd635c1ed1f7" path="/var/lib/kubelet/pods/091d6975-4400-4dde-b61c-bd635c1ed1f7/volumes" Dec 03 12:01:15 crc kubenswrapper[5052]: I1203 12:01:15.709328 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="158aaec8-feb0-4a6e-84ff-b137b74f0ef6" path="/var/lib/kubelet/pods/158aaec8-feb0-4a6e-84ff-b137b74f0ef6/volumes" Dec 03 12:01:15 crc kubenswrapper[5052]: I1203 12:01:15.710105 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="820f9385-7816-4259-987f-15eef2e78099" path="/var/lib/kubelet/pods/820f9385-7816-4259-987f-15eef2e78099/volumes" Dec 03 12:01:15 crc kubenswrapper[5052]: I1203 12:01:15.734262 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5e5ddf8c-6109-481f-beb0-0802de034912-combined-ca-bundle\") pod \"5e5ddf8c-6109-481f-beb0-0802de034912\" (UID: \"5e5ddf8c-6109-481f-beb0-0802de034912\") " Dec 03 12:01:15 crc kubenswrapper[5052]: I1203 12:01:15.734397 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/3cb4559f-542a-4076-82fb-c0ca88f9de94-metrics-certs-tls-certs\") pod \"3cb4559f-542a-4076-82fb-c0ca88f9de94\" (UID: \"3cb4559f-542a-4076-82fb-c0ca88f9de94\") " Dec 03 12:01:15 crc kubenswrapper[5052]: I1203 12:01:15.734451 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3cb4559f-542a-4076-82fb-c0ca88f9de94-combined-ca-bundle\") pod \"3cb4559f-542a-4076-82fb-c0ca88f9de94\" (UID: \"3cb4559f-542a-4076-82fb-c0ca88f9de94\") " Dec 03 12:01:15 crc kubenswrapper[5052]: I1203 12:01:15.734476 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovn-northd-tls-certs\" (UniqueName: \"kubernetes.io/secret/5e5ddf8c-6109-481f-beb0-0802de034912-ovn-northd-tls-certs\") pod \"5e5ddf8c-6109-481f-beb0-0802de034912\" (UID: \"5e5ddf8c-6109-481f-beb0-0802de034912\") " Dec 03 12:01:15 crc kubenswrapper[5052]: I1203 12:01:15.734551 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/5e5ddf8c-6109-481f-beb0-0802de034912-scripts\") pod \"5e5ddf8c-6109-481f-beb0-0802de034912\" (UID: \"5e5ddf8c-6109-481f-beb0-0802de034912\") " Dec 03 12:01:15 crc kubenswrapper[5052]: I1203 12:01:15.734622 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovndbcluster-sb-etc-ovn\" (UniqueName: \"kubernetes.io/local-volume/local-storage10-crc\") pod \"3cb4559f-542a-4076-82fb-c0ca88f9de94\" (UID: \"3cb4559f-542a-4076-82fb-c0ca88f9de94\") " Dec 03 12:01:15 crc kubenswrapper[5052]: I1203 12:01:15.734663 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4vfjg\" (UniqueName: \"kubernetes.io/projected/3cb4559f-542a-4076-82fb-c0ca88f9de94-kube-api-access-4vfjg\") pod \"3cb4559f-542a-4076-82fb-c0ca88f9de94\" (UID: \"3cb4559f-542a-4076-82fb-c0ca88f9de94\") " Dec 03 12:01:15 crc kubenswrapper[5052]: I1203 12:01:15.734716 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovsdbserver-sb-tls-certs\" (UniqueName: \"kubernetes.io/secret/3cb4559f-542a-4076-82fb-c0ca88f9de94-ovsdbserver-sb-tls-certs\") pod \"3cb4559f-542a-4076-82fb-c0ca88f9de94\" (UID: \"3cb4559f-542a-4076-82fb-c0ca88f9de94\") " Dec 03 12:01:15 crc kubenswrapper[5052]: I1203 12:01:15.734762 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/3cb4559f-542a-4076-82fb-c0ca88f9de94-ovsdb-rundir\") pod \"3cb4559f-542a-4076-82fb-c0ca88f9de94\" (UID: \"3cb4559f-542a-4076-82fb-c0ca88f9de94\") " Dec 03 12:01:15 crc kubenswrapper[5052]: I1203 12:01:15.734779 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovn-rundir\" (UniqueName: \"kubernetes.io/empty-dir/5e5ddf8c-6109-481f-beb0-0802de034912-ovn-rundir\") pod \"5e5ddf8c-6109-481f-beb0-0802de034912\" (UID: \"5e5ddf8c-6109-481f-beb0-0802de034912\") " Dec 03 12:01:15 crc kubenswrapper[5052]: I1203 12:01:15.734804 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3cb4559f-542a-4076-82fb-c0ca88f9de94-config\") pod \"3cb4559f-542a-4076-82fb-c0ca88f9de94\" (UID: \"3cb4559f-542a-4076-82fb-c0ca88f9de94\") " Dec 03 12:01:15 crc kubenswrapper[5052]: I1203 12:01:15.734979 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/3cb4559f-542a-4076-82fb-c0ca88f9de94-scripts\") pod \"3cb4559f-542a-4076-82fb-c0ca88f9de94\" (UID: \"3cb4559f-542a-4076-82fb-c0ca88f9de94\") " Dec 03 12:01:15 crc kubenswrapper[5052]: I1203 12:01:15.735007 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5e5ddf8c-6109-481f-beb0-0802de034912-config\") pod \"5e5ddf8c-6109-481f-beb0-0802de034912\" (UID: \"5e5ddf8c-6109-481f-beb0-0802de034912\") " Dec 03 12:01:15 crc kubenswrapper[5052]: I1203 12:01:15.735132 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/5e5ddf8c-6109-481f-beb0-0802de034912-metrics-certs-tls-certs\") pod \"5e5ddf8c-6109-481f-beb0-0802de034912\" (UID: \"5e5ddf8c-6109-481f-beb0-0802de034912\") " Dec 03 12:01:15 crc kubenswrapper[5052]: I1203 12:01:15.735185 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2hf5m\" (UniqueName: \"kubernetes.io/projected/5e5ddf8c-6109-481f-beb0-0802de034912-kube-api-access-2hf5m\") pod \"5e5ddf8c-6109-481f-beb0-0802de034912\" (UID: \"5e5ddf8c-6109-481f-beb0-0802de034912\") " Dec 03 12:01:15 crc kubenswrapper[5052]: I1203 12:01:15.737387 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3cb4559f-542a-4076-82fb-c0ca88f9de94-scripts" (OuterVolumeSpecName: "scripts") pod "3cb4559f-542a-4076-82fb-c0ca88f9de94" (UID: "3cb4559f-542a-4076-82fb-c0ca88f9de94"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:01:15 crc kubenswrapper[5052]: I1203 12:01:15.737378 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3cb4559f-542a-4076-82fb-c0ca88f9de94-config" (OuterVolumeSpecName: "config") pod "3cb4559f-542a-4076-82fb-c0ca88f9de94" (UID: "3cb4559f-542a-4076-82fb-c0ca88f9de94"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:01:15 crc kubenswrapper[5052]: I1203 12:01:15.737701 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3cb4559f-542a-4076-82fb-c0ca88f9de94-ovsdb-rundir" (OuterVolumeSpecName: "ovsdb-rundir") pod "3cb4559f-542a-4076-82fb-c0ca88f9de94" (UID: "3cb4559f-542a-4076-82fb-c0ca88f9de94"). InnerVolumeSpecName "ovsdb-rundir". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:01:15 crc kubenswrapper[5052]: I1203 12:01:15.737865 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e5ddf8c-6109-481f-beb0-0802de034912-ovn-rundir" (OuterVolumeSpecName: "ovn-rundir") pod "5e5ddf8c-6109-481f-beb0-0802de034912" (UID: "5e5ddf8c-6109-481f-beb0-0802de034912"). InnerVolumeSpecName "ovn-rundir". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:01:15 crc kubenswrapper[5052]: I1203 12:01:15.737934 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5e5ddf8c-6109-481f-beb0-0802de034912-config" (OuterVolumeSpecName: "config") pod "5e5ddf8c-6109-481f-beb0-0802de034912" (UID: "5e5ddf8c-6109-481f-beb0-0802de034912"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:01:15 crc kubenswrapper[5052]: I1203 12:01:15.740618 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5e5ddf8c-6109-481f-beb0-0802de034912-scripts" (OuterVolumeSpecName: "scripts") pod "5e5ddf8c-6109-481f-beb0-0802de034912" (UID: "5e5ddf8c-6109-481f-beb0-0802de034912"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:01:15 crc kubenswrapper[5052]: I1203 12:01:15.774397 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage10-crc" (OuterVolumeSpecName: "ovndbcluster-sb-etc-ovn") pod "3cb4559f-542a-4076-82fb-c0ca88f9de94" (UID: "3cb4559f-542a-4076-82fb-c0ca88f9de94"). InnerVolumeSpecName "local-storage10-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 12:01:15 crc kubenswrapper[5052]: I1203 12:01:15.806406 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3cb4559f-542a-4076-82fb-c0ca88f9de94-kube-api-access-4vfjg" (OuterVolumeSpecName: "kube-api-access-4vfjg") pod "3cb4559f-542a-4076-82fb-c0ca88f9de94" (UID: "3cb4559f-542a-4076-82fb-c0ca88f9de94"). InnerVolumeSpecName "kube-api-access-4vfjg". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:01:15 crc kubenswrapper[5052]: I1203 12:01:15.807034 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5e5ddf8c-6109-481f-beb0-0802de034912-kube-api-access-2hf5m" (OuterVolumeSpecName: "kube-api-access-2hf5m") pod "5e5ddf8c-6109-481f-beb0-0802de034912" (UID: "5e5ddf8c-6109-481f-beb0-0802de034912"). InnerVolumeSpecName "kube-api-access-2hf5m". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:01:15 crc kubenswrapper[5052]: I1203 12:01:15.821811 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/novacell184d8-account-delete-4nmn9" event={"ID":"cf90ea08-b172-4549-9522-92be7a5b5623","Type":"ContainerStarted","Data":"44a5bf19277ea6d430a875328ef65cac7328db3c64a6c1ad83804395f34122c3"} Dec 03 12:01:15 crc kubenswrapper[5052]: I1203 12:01:15.822588 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:01:15 crc kubenswrapper[5052]: I1203 12:01:15.836742 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6x8wn\" (UniqueName: \"kubernetes.io/projected/e8de8adf-e4e5-4a10-9722-9106357e67d0-kube-api-access-6x8wn\") pod \"e8de8adf-e4e5-4a10-9722-9106357e67d0\" (UID: \"e8de8adf-e4e5-4a10-9722-9106357e67d0\") " Dec 03 12:01:15 crc kubenswrapper[5052]: I1203 12:01:15.836819 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e8de8adf-e4e5-4a10-9722-9106357e67d0-combined-ca-bundle\") pod \"e8de8adf-e4e5-4a10-9722-9106357e67d0\" (UID: \"e8de8adf-e4e5-4a10-9722-9106357e67d0\") " Dec 03 12:01:15 crc kubenswrapper[5052]: I1203 12:01:15.836863 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/e8de8adf-e4e5-4a10-9722-9106357e67d0-ovsdb-rundir\") pod \"e8de8adf-e4e5-4a10-9722-9106357e67d0\" (UID: \"e8de8adf-e4e5-4a10-9722-9106357e67d0\") " Dec 03 12:01:15 crc kubenswrapper[5052]: I1203 12:01:15.836964 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/e8de8adf-e4e5-4a10-9722-9106357e67d0-scripts\") pod \"e8de8adf-e4e5-4a10-9722-9106357e67d0\" (UID: \"e8de8adf-e4e5-4a10-9722-9106357e67d0\") " Dec 03 12:01:15 crc kubenswrapper[5052]: I1203 12:01:15.836997 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovndbcluster-nb-etc-ovn\" (UniqueName: \"kubernetes.io/local-volume/local-storage01-crc\") pod \"e8de8adf-e4e5-4a10-9722-9106357e67d0\" (UID: \"e8de8adf-e4e5-4a10-9722-9106357e67d0\") " Dec 03 12:01:15 crc kubenswrapper[5052]: I1203 12:01:15.837021 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e8de8adf-e4e5-4a10-9722-9106357e67d0-config\") pod \"e8de8adf-e4e5-4a10-9722-9106357e67d0\" (UID: \"e8de8adf-e4e5-4a10-9722-9106357e67d0\") " Dec 03 12:01:15 crc kubenswrapper[5052]: I1203 12:01:15.837091 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovsdbserver-nb-tls-certs\" (UniqueName: \"kubernetes.io/secret/e8de8adf-e4e5-4a10-9722-9106357e67d0-ovsdbserver-nb-tls-certs\") pod \"e8de8adf-e4e5-4a10-9722-9106357e67d0\" (UID: \"e8de8adf-e4e5-4a10-9722-9106357e67d0\") " Dec 03 12:01:15 crc kubenswrapper[5052]: I1203 12:01:15.839543 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:01:15 crc kubenswrapper[5052]: I1203 12:01:15.839683 5052 generic.go:334] "Generic (PLEG): container finished" podID="98d3c14e-7549-441a-9532-f3d75833cdf8" containerID="710c4f7c04a361b712543bf316631f84336d70d531c95a80fd6746ea4334640d" exitCode=143 Dec 03 12:01:15 crc kubenswrapper[5052]: I1203 12:01:15.839778 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"98d3c14e-7549-441a-9532-f3d75833cdf8","Type":"ContainerDied","Data":"710c4f7c04a361b712543bf316631f84336d70d531c95a80fd6746ea4334640d"} Dec 03 12:01:15 crc kubenswrapper[5052]: I1203 12:01:15.840003 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e8de8adf-e4e5-4a10-9722-9106357e67d0-scripts" (OuterVolumeSpecName: "scripts") pod "e8de8adf-e4e5-4a10-9722-9106357e67d0" (UID: "e8de8adf-e4e5-4a10-9722-9106357e67d0"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:01:15 crc kubenswrapper[5052]: I1203 12:01:15.842711 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/e8de8adf-e4e5-4a10-9722-9106357e67d0-metrics-certs-tls-certs\") pod \"e8de8adf-e4e5-4a10-9722-9106357e67d0\" (UID: \"e8de8adf-e4e5-4a10-9722-9106357e67d0\") " Dec 03 12:01:15 crc kubenswrapper[5052]: I1203 12:01:15.843237 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e8de8adf-e4e5-4a10-9722-9106357e67d0-ovsdb-rundir" (OuterVolumeSpecName: "ovsdb-rundir") pod "e8de8adf-e4e5-4a10-9722-9106357e67d0" (UID: "e8de8adf-e4e5-4a10-9722-9106357e67d0"). InnerVolumeSpecName "ovsdb-rundir". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:01:15 crc kubenswrapper[5052]: I1203 12:01:15.843701 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-4vfjg\" (UniqueName: \"kubernetes.io/projected/3cb4559f-542a-4076-82fb-c0ca88f9de94-kube-api-access-4vfjg\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:15 crc kubenswrapper[5052]: I1203 12:01:15.843729 5052 reconciler_common.go:293] "Volume detached for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/e8de8adf-e4e5-4a10-9722-9106357e67d0-ovsdb-rundir\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:15 crc kubenswrapper[5052]: I1203 12:01:15.843742 5052 reconciler_common.go:293] "Volume detached for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/3cb4559f-542a-4076-82fb-c0ca88f9de94-ovsdb-rundir\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:15 crc kubenswrapper[5052]: I1203 12:01:15.843753 5052 reconciler_common.go:293] "Volume detached for volume \"ovn-rundir\" (UniqueName: \"kubernetes.io/empty-dir/5e5ddf8c-6109-481f-beb0-0802de034912-ovn-rundir\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:15 crc kubenswrapper[5052]: I1203 12:01:15.843764 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3cb4559f-542a-4076-82fb-c0ca88f9de94-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:15 crc kubenswrapper[5052]: I1203 12:01:15.843775 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/e8de8adf-e4e5-4a10-9722-9106357e67d0-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:15 crc kubenswrapper[5052]: I1203 12:01:15.843785 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/3cb4559f-542a-4076-82fb-c0ca88f9de94-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:15 crc kubenswrapper[5052]: I1203 12:01:15.843797 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5e5ddf8c-6109-481f-beb0-0802de034912-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:15 crc kubenswrapper[5052]: E1203 12:01:15.848218 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/rabbitmq-config-data: configmap "rabbitmq-config-data" not found Dec 03 12:01:15 crc kubenswrapper[5052]: E1203 12:01:15.848289 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/30578d71-fe2b-4eed-9a07-6c8bfe4a1c43-config-data podName:30578d71-fe2b-4eed-9a07-6c8bfe4a1c43 nodeName:}" failed. No retries permitted until 2025-12-03 12:01:19.848266046 +0000 UTC m=+2470.436507910 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "config-data" (UniqueName: "kubernetes.io/configmap/30578d71-fe2b-4eed-9a07-6c8bfe4a1c43-config-data") pod "rabbitmq-server-0" (UID: "30578d71-fe2b-4eed-9a07-6c8bfe4a1c43") : configmap "rabbitmq-config-data" not found Dec 03 12:01:15 crc kubenswrapper[5052]: I1203 12:01:15.850446 5052 generic.go:334] "Generic (PLEG): container finished" podID="2d2fda50-7118-43f2-8290-36d97b66de86" containerID="365e155e111e9968f8e4c88e8465be33bb14e93a08d5db3a41bced9b4391e4fd" exitCode=0 Dec 03 12:01:15 crc kubenswrapper[5052]: I1203 12:01:15.850613 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"2d2fda50-7118-43f2-8290-36d97b66de86","Type":"ContainerDied","Data":"365e155e111e9968f8e4c88e8465be33bb14e93a08d5db3a41bced9b4391e4fd"} Dec 03 12:01:15 crc kubenswrapper[5052]: I1203 12:01:15.851266 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e8de8adf-e4e5-4a10-9722-9106357e67d0-config" (OuterVolumeSpecName: "config") pod "e8de8adf-e4e5-4a10-9722-9106357e67d0" (UID: "e8de8adf-e4e5-4a10-9722-9106357e67d0"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:01:15 crc kubenswrapper[5052]: I1203 12:01:15.852990 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-2hf5m\" (UniqueName: \"kubernetes.io/projected/5e5ddf8c-6109-481f-beb0-0802de034912-kube-api-access-2hf5m\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:15 crc kubenswrapper[5052]: I1203 12:01:15.853048 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/5e5ddf8c-6109-481f-beb0-0802de034912-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:15 crc kubenswrapper[5052]: I1203 12:01:15.853084 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage10-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage10-crc\") on node \"crc\" " Dec 03 12:01:15 crc kubenswrapper[5052]: I1203 12:01:15.875003 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e8de8adf-e4e5-4a10-9722-9106357e67d0-kube-api-access-6x8wn" (OuterVolumeSpecName: "kube-api-access-6x8wn") pod "e8de8adf-e4e5-4a10-9722-9106357e67d0" (UID: "e8de8adf-e4e5-4a10-9722-9106357e67d0"). InnerVolumeSpecName "kube-api-access-6x8wn". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:01:15 crc kubenswrapper[5052]: I1203 12:01:15.902797 5052 generic.go:334] "Generic (PLEG): container finished" podID="01bc82de-61dc-477e-b4cf-e3b0d32c99ba" containerID="b2b8f2e1a2b0959c9d345484247345b165016e4ab99777de279ec024458c9da1" exitCode=0 Dec 03 12:01:15 crc kubenswrapper[5052]: I1203 12:01:15.902825 5052 generic.go:334] "Generic (PLEG): container finished" podID="01bc82de-61dc-477e-b4cf-e3b0d32c99ba" containerID="3845a093071f0f8ca351aba36d4c3c56dd2d469ba1bfbea7b327c64e55ea871c" exitCode=0 Dec 03 12:01:15 crc kubenswrapper[5052]: I1203 12:01:15.902834 5052 generic.go:334] "Generic (PLEG): container finished" podID="01bc82de-61dc-477e-b4cf-e3b0d32c99ba" containerID="599972ea725b2448de0f384ff58d4263ae4c6939731529af770910368c748187" exitCode=0 Dec 03 12:01:15 crc kubenswrapper[5052]: I1203 12:01:15.902903 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/prometheus-metric-storage-0" event={"ID":"01bc82de-61dc-477e-b4cf-e3b0d32c99ba","Type":"ContainerDied","Data":"b2b8f2e1a2b0959c9d345484247345b165016e4ab99777de279ec024458c9da1"} Dec 03 12:01:15 crc kubenswrapper[5052]: I1203 12:01:15.902933 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/prometheus-metric-storage-0" event={"ID":"01bc82de-61dc-477e-b4cf-e3b0d32c99ba","Type":"ContainerDied","Data":"3845a093071f0f8ca351aba36d4c3c56dd2d469ba1bfbea7b327c64e55ea871c"} Dec 03 12:01:15 crc kubenswrapper[5052]: I1203 12:01:15.902942 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/prometheus-metric-storage-0" event={"ID":"01bc82de-61dc-477e-b4cf-e3b0d32c99ba","Type":"ContainerDied","Data":"599972ea725b2448de0f384ff58d4263ae4c6939731529af770910368c748187"} Dec 03 12:01:15 crc kubenswrapper[5052]: I1203 12:01:15.905559 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/novacell06be2-account-delete-7m8cr" event={"ID":"6343f4a5-892f-4118-a666-e6601d2df818","Type":"ContainerStarted","Data":"8273a87287f1d179a43891b8da0c2941ee49856e411b0cf9a44a8684b07521ed"} Dec 03 12:01:15 crc kubenswrapper[5052]: I1203 12:01:15.918398 5052 generic.go:334] "Generic (PLEG): container finished" podID="7cbbb6c5-a087-400f-9beb-e740ae1ddfbd" containerID="079d8a9a5ad9c1f5d88409c04f6a13ba44666a756c6ea8ff7bd4d25cb9a1528e" exitCode=0 Dec 03 12:01:15 crc kubenswrapper[5052]: I1203 12:01:15.918429 5052 generic.go:334] "Generic (PLEG): container finished" podID="7cbbb6c5-a087-400f-9beb-e740ae1ddfbd" containerID="ae36686b468c95feb23e421e5898d4b9d5542290763ef55cdda58e32e8d7665b" exitCode=0 Dec 03 12:01:15 crc kubenswrapper[5052]: I1203 12:01:15.918520 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/alertmanager-metric-storage-0" event={"ID":"7cbbb6c5-a087-400f-9beb-e740ae1ddfbd","Type":"ContainerDied","Data":"079d8a9a5ad9c1f5d88409c04f6a13ba44666a756c6ea8ff7bd4d25cb9a1528e"} Dec 03 12:01:15 crc kubenswrapper[5052]: I1203 12:01:15.918549 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/alertmanager-metric-storage-0" event={"ID":"7cbbb6c5-a087-400f-9beb-e740ae1ddfbd","Type":"ContainerDied","Data":"ae36686b468c95feb23e421e5898d4b9d5542290763ef55cdda58e32e8d7665b"} Dec 03 12:01:15 crc kubenswrapper[5052]: I1203 12:01:15.930222 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/novaapif429-account-delete-qwz7z" event={"ID":"eaeff688-fe65-403f-a01a-dfff69db9fb9","Type":"ContainerStarted","Data":"7634d97b9cc32e4677f109f4e33648a1b55579ab8f2afab402440c5c075e7a32"} Dec 03 12:01:15 crc kubenswrapper[5052]: I1203 12:01:15.937345 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage01-crc" (OuterVolumeSpecName: "ovndbcluster-nb-etc-ovn") pod "e8de8adf-e4e5-4a10-9722-9106357e67d0" (UID: "e8de8adf-e4e5-4a10-9722-9106357e67d0"). InnerVolumeSpecName "local-storage01-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 12:01:15 crc kubenswrapper[5052]: I1203 12:01:15.942895 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_ovsdbserver-nb-0_e8de8adf-e4e5-4a10-9722-9106357e67d0/ovsdbserver-nb/0.log" Dec 03 12:01:15 crc kubenswrapper[5052]: I1203 12:01:15.942993 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-nb-0" event={"ID":"e8de8adf-e4e5-4a10-9722-9106357e67d0","Type":"ContainerDied","Data":"9293b1b69f544359bada65ca1151c8ddba8749132dd536bed546abf83b768377"} Dec 03 12:01:15 crc kubenswrapper[5052]: I1203 12:01:15.943036 5052 scope.go:117] "RemoveContainer" containerID="156a9ab495313477724afb77cc40733731b002a3db4ec302faefc4c513160b71" Dec 03 12:01:15 crc kubenswrapper[5052]: I1203 12:01:15.943208 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:01:15 crc kubenswrapper[5052]: I1203 12:01:15.949174 5052 generic.go:334] "Generic (PLEG): container finished" podID="914c0f8f-e5d1-4c63-b985-0bcd07f75374" containerID="2d2f9846f42dc135f23ee0343741d94b75b793c15135ba895cc30047a367cc1e" exitCode=0 Dec 03 12:01:15 crc kubenswrapper[5052]: I1203 12:01:15.949236 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" event={"ID":"914c0f8f-e5d1-4c63-b985-0bcd07f75374","Type":"ContainerDied","Data":"2d2f9846f42dc135f23ee0343741d94b75b793c15135ba895cc30047a367cc1e"} Dec 03 12:01:15 crc kubenswrapper[5052]: I1203 12:01:15.949263 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" event={"ID":"914c0f8f-e5d1-4c63-b985-0bcd07f75374","Type":"ContainerDied","Data":"ddc1a19aa77d2a741435f4f1d516057d357cd30b098acd5e03d21402ad882813"} Dec 03 12:01:15 crc kubenswrapper[5052]: I1203 12:01:15.951351 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:01:15 crc kubenswrapper[5052]: I1203 12:01:15.952538 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/novaapif429-account-delete-qwz7z" podStartSLOduration=3.952505816 podStartE2EDuration="3.952505816s" podCreationTimestamp="2025-12-03 12:01:12 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:01:15.949933193 +0000 UTC m=+2466.538174967" watchObservedRunningTime="2025-12-03 12:01:15.952505816 +0000 UTC m=+2466.540747580" Dec 03 12:01:15 crc kubenswrapper[5052]: I1203 12:01:15.953060 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron0ea0-account-delete-lvffs" event={"ID":"c17bae4f-1c1b-4da2-9bb5-c84d6fdfa184","Type":"ContainerStarted","Data":"3ed5d3b7ccddd21aa86bde44d59c298853aa183e114f61bd8e87fa66be37af06"} Dec 03 12:01:15 crc kubenswrapper[5052]: I1203 12:01:15.955803 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/da4dfbea-20a2-4638-bbbe-ed4bfee0ad49-run-httpd\") pod \"da4dfbea-20a2-4638-bbbe-ed4bfee0ad49\" (UID: \"da4dfbea-20a2-4638-bbbe-ed4bfee0ad49\") " Dec 03 12:01:15 crc kubenswrapper[5052]: I1203 12:01:15.955853 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/da4dfbea-20a2-4638-bbbe-ed4bfee0ad49-config-data\") pod \"da4dfbea-20a2-4638-bbbe-ed4bfee0ad49\" (UID: \"da4dfbea-20a2-4638-bbbe-ed4bfee0ad49\") " Dec 03 12:01:15 crc kubenswrapper[5052]: I1203 12:01:15.955976 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"nova-novncproxy-tls-certs\" (UniqueName: \"kubernetes.io/secret/914c0f8f-e5d1-4c63-b985-0bcd07f75374-nova-novncproxy-tls-certs\") pod \"914c0f8f-e5d1-4c63-b985-0bcd07f75374\" (UID: \"914c0f8f-e5d1-4c63-b985-0bcd07f75374\") " Dec 03 12:01:15 crc kubenswrapper[5052]: I1203 12:01:15.956030 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kd4bh\" (UniqueName: \"kubernetes.io/projected/da4dfbea-20a2-4638-bbbe-ed4bfee0ad49-kube-api-access-kd4bh\") pod \"da4dfbea-20a2-4638-bbbe-ed4bfee0ad49\" (UID: \"da4dfbea-20a2-4638-bbbe-ed4bfee0ad49\") " Dec 03 12:01:15 crc kubenswrapper[5052]: I1203 12:01:15.956057 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/914c0f8f-e5d1-4c63-b985-0bcd07f75374-config-data\") pod \"914c0f8f-e5d1-4c63-b985-0bcd07f75374\" (UID: \"914c0f8f-e5d1-4c63-b985-0bcd07f75374\") " Dec 03 12:01:15 crc kubenswrapper[5052]: I1203 12:01:15.956100 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/914c0f8f-e5d1-4c63-b985-0bcd07f75374-combined-ca-bundle\") pod \"914c0f8f-e5d1-4c63-b985-0bcd07f75374\" (UID: \"914c0f8f-e5d1-4c63-b985-0bcd07f75374\") " Dec 03 12:01:15 crc kubenswrapper[5052]: I1203 12:01:15.956165 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/da4dfbea-20a2-4638-bbbe-ed4bfee0ad49-log-httpd\") pod \"da4dfbea-20a2-4638-bbbe-ed4bfee0ad49\" (UID: \"da4dfbea-20a2-4638-bbbe-ed4bfee0ad49\") " Dec 03 12:01:15 crc kubenswrapper[5052]: I1203 12:01:15.956204 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"vencrypt-tls-certs\" (UniqueName: \"kubernetes.io/secret/914c0f8f-e5d1-4c63-b985-0bcd07f75374-vencrypt-tls-certs\") pod \"914c0f8f-e5d1-4c63-b985-0bcd07f75374\" (UID: \"914c0f8f-e5d1-4c63-b985-0bcd07f75374\") " Dec 03 12:01:15 crc kubenswrapper[5052]: I1203 12:01:15.956240 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7t9cz\" (UniqueName: \"kubernetes.io/projected/914c0f8f-e5d1-4c63-b985-0bcd07f75374-kube-api-access-7t9cz\") pod \"914c0f8f-e5d1-4c63-b985-0bcd07f75374\" (UID: \"914c0f8f-e5d1-4c63-b985-0bcd07f75374\") " Dec 03 12:01:15 crc kubenswrapper[5052]: I1203 12:01:15.956299 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/da4dfbea-20a2-4638-bbbe-ed4bfee0ad49-sg-core-conf-yaml\") pod \"da4dfbea-20a2-4638-bbbe-ed4bfee0ad49\" (UID: \"da4dfbea-20a2-4638-bbbe-ed4bfee0ad49\") " Dec 03 12:01:15 crc kubenswrapper[5052]: I1203 12:01:15.956539 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/da4dfbea-20a2-4638-bbbe-ed4bfee0ad49-ceilometer-tls-certs\") pod \"da4dfbea-20a2-4638-bbbe-ed4bfee0ad49\" (UID: \"da4dfbea-20a2-4638-bbbe-ed4bfee0ad49\") " Dec 03 12:01:15 crc kubenswrapper[5052]: I1203 12:01:15.956583 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/da4dfbea-20a2-4638-bbbe-ed4bfee0ad49-combined-ca-bundle\") pod \"da4dfbea-20a2-4638-bbbe-ed4bfee0ad49\" (UID: \"da4dfbea-20a2-4638-bbbe-ed4bfee0ad49\") " Dec 03 12:01:15 crc kubenswrapper[5052]: I1203 12:01:15.956631 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/da4dfbea-20a2-4638-bbbe-ed4bfee0ad49-scripts\") pod \"da4dfbea-20a2-4638-bbbe-ed4bfee0ad49\" (UID: \"da4dfbea-20a2-4638-bbbe-ed4bfee0ad49\") " Dec 03 12:01:15 crc kubenswrapper[5052]: I1203 12:01:15.960635 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-6x8wn\" (UniqueName: \"kubernetes.io/projected/e8de8adf-e4e5-4a10-9722-9106357e67d0-kube-api-access-6x8wn\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:15 crc kubenswrapper[5052]: I1203 12:01:15.960669 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage01-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage01-crc\") on node \"crc\" " Dec 03 12:01:15 crc kubenswrapper[5052]: I1203 12:01:15.960680 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e8de8adf-e4e5-4a10-9722-9106357e67d0-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:15 crc kubenswrapper[5052]: I1203 12:01:15.961529 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/da4dfbea-20a2-4638-bbbe-ed4bfee0ad49-run-httpd" (OuterVolumeSpecName: "run-httpd") pod "da4dfbea-20a2-4638-bbbe-ed4bfee0ad49" (UID: "da4dfbea-20a2-4638-bbbe-ed4bfee0ad49"). InnerVolumeSpecName "run-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:01:15 crc kubenswrapper[5052]: I1203 12:01:15.964386 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/da4dfbea-20a2-4638-bbbe-ed4bfee0ad49-log-httpd" (OuterVolumeSpecName: "log-httpd") pod "da4dfbea-20a2-4638-bbbe-ed4bfee0ad49" (UID: "da4dfbea-20a2-4638-bbbe-ed4bfee0ad49"). InnerVolumeSpecName "log-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:01:15 crc kubenswrapper[5052]: I1203 12:01:15.977010 5052 scope.go:117] "RemoveContainer" containerID="a3deb4fed82228a5b22c0094ab0a2185263de5b8a00e34fbefb31c5da84f1d53" Dec 03 12:01:15 crc kubenswrapper[5052]: I1203 12:01:15.982482 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/da4dfbea-20a2-4638-bbbe-ed4bfee0ad49-scripts" (OuterVolumeSpecName: "scripts") pod "da4dfbea-20a2-4638-bbbe-ed4bfee0ad49" (UID: "da4dfbea-20a2-4638-bbbe-ed4bfee0ad49"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:01:15 crc kubenswrapper[5052]: I1203 12:01:15.983556 5052 generic.go:334] "Generic (PLEG): container finished" podID="1103e975-cc9a-435b-adc9-5c6492e9c629" containerID="be465bb3373c78a7e0d0341f23e780fbf057154f4f4e5cd558d38b99cc20e088" exitCode=143 Dec 03 12:01:15 crc kubenswrapper[5052]: I1203 12:01:15.983602 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"1103e975-cc9a-435b-adc9-5c6492e9c629","Type":"ContainerDied","Data":"be465bb3373c78a7e0d0341f23e780fbf057154f4f4e5cd558d38b99cc20e088"} Dec 03 12:01:15 crc kubenswrapper[5052]: I1203 12:01:15.983756 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/914c0f8f-e5d1-4c63-b985-0bcd07f75374-kube-api-access-7t9cz" (OuterVolumeSpecName: "kube-api-access-7t9cz") pod "914c0f8f-e5d1-4c63-b985-0bcd07f75374" (UID: "914c0f8f-e5d1-4c63-b985-0bcd07f75374"). InnerVolumeSpecName "kube-api-access-7t9cz". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:01:16 crc kubenswrapper[5052]: I1203 12:01:16.005856 5052 generic.go:334] "Generic (PLEG): container finished" podID="192fa797-c8b8-42d3-a689-e6a7dbc77282" containerID="0df8ce5588ea9142955dd14165fc02005b40b2952f5607fe472ebbf7177b2be8" exitCode=0 Dec 03 12:01:16 crc kubenswrapper[5052]: I1203 12:01:16.005979 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/watcher-applier-0" event={"ID":"192fa797-c8b8-42d3-a689-e6a7dbc77282","Type":"ContainerDied","Data":"0df8ce5588ea9142955dd14165fc02005b40b2952f5607fe472ebbf7177b2be8"} Dec 03 12:01:16 crc kubenswrapper[5052]: I1203 12:01:16.053688 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:01:16 crc kubenswrapper[5052]: I1203 12:01:16.053224 5052 generic.go:334] "Generic (PLEG): container finished" podID="da4dfbea-20a2-4638-bbbe-ed4bfee0ad49" containerID="3f16de4b1d15886789bdc38843cbe1fb5e5180679e89cb7fb19837deadc091ae" exitCode=0 Dec 03 12:01:16 crc kubenswrapper[5052]: I1203 12:01:16.057677 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"da4dfbea-20a2-4638-bbbe-ed4bfee0ad49","Type":"ContainerDied","Data":"3f16de4b1d15886789bdc38843cbe1fb5e5180679e89cb7fb19837deadc091ae"} Dec 03 12:01:16 crc kubenswrapper[5052]: I1203 12:01:16.062838 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/memcached-0"] Dec 03 12:01:16 crc kubenswrapper[5052]: I1203 12:01:16.063064 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/memcached-0" podUID="5a08cda2-5952-4740-a571-7058f56c3b58" containerName="memcached" containerID="cri-o://57e4d8c014f57177dc5f916168de2b019cf9fcd67c0d39bcaf0f7357d2dbf4bc" gracePeriod=30 Dec 03 12:01:16 crc kubenswrapper[5052]: I1203 12:01:16.070281 5052 generic.go:334] "Generic (PLEG): container finished" podID="492ec2b7-0c28-4fe2-b2be-4e9e58423f23" containerID="015cccbd4c1596494b8217997ac4a87be227da3e9b2b7729dd71f4bc53db28b6" exitCode=0 Dec 03 12:01:16 crc kubenswrapper[5052]: I1203 12:01:16.070396 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinderaaa7-account-delete-rdrqp" event={"ID":"492ec2b7-0c28-4fe2-b2be-4e9e58423f23","Type":"ContainerDied","Data":"015cccbd4c1596494b8217997ac4a87be227da3e9b2b7729dd71f4bc53db28b6"} Dec 03 12:01:16 crc kubenswrapper[5052]: I1203 12:01:16.071119 5052 reconciler_common.go:293] "Volume detached for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/da4dfbea-20a2-4638-bbbe-ed4bfee0ad49-log-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:16 crc kubenswrapper[5052]: I1203 12:01:16.071147 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-7t9cz\" (UniqueName: \"kubernetes.io/projected/914c0f8f-e5d1-4c63-b985-0bcd07f75374-kube-api-access-7t9cz\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:16 crc kubenswrapper[5052]: I1203 12:01:16.071160 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/da4dfbea-20a2-4638-bbbe-ed4bfee0ad49-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:16 crc kubenswrapper[5052]: I1203 12:01:16.071172 5052 reconciler_common.go:293] "Volume detached for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/da4dfbea-20a2-4638-bbbe-ed4bfee0ad49-run-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:16 crc kubenswrapper[5052]: I1203 12:01:16.083490 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/da4dfbea-20a2-4638-bbbe-ed4bfee0ad49-kube-api-access-kd4bh" (OuterVolumeSpecName: "kube-api-access-kd4bh") pod "da4dfbea-20a2-4638-bbbe-ed4bfee0ad49" (UID: "da4dfbea-20a2-4638-bbbe-ed4bfee0ad49"). InnerVolumeSpecName "kube-api-access-kd4bh". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:01:16 crc kubenswrapper[5052]: I1203 12:01:16.099011 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican4aeb-account-delete-bvhdj" event={"ID":"d420a2c2-9b7b-400b-88a7-839a93cce573","Type":"ContainerStarted","Data":"1c87d91694a155411ce2f1edf96cdd4dce990021b797e016cf7712dfb5d7780c"} Dec 03 12:01:16 crc kubenswrapper[5052]: I1203 12:01:16.102224 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/neutron0ea0-account-delete-lvffs" podStartSLOduration=4.102173285 podStartE2EDuration="4.102173285s" podCreationTimestamp="2025-12-03 12:01:12 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:01:16.0099998 +0000 UTC m=+2466.598241564" watchObservedRunningTime="2025-12-03 12:01:16.102173285 +0000 UTC m=+2466.690415059" Dec 03 12:01:16 crc kubenswrapper[5052]: I1203 12:01:16.111817 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placemente203-account-delete-sdpql" event={"ID":"36225798-caeb-4514-9f51-c8a07fb6da9b","Type":"ContainerStarted","Data":"f4465faaa5fa11eb658b595d91c5bb333d670fc1540b89c1e7c2d98cb5ec5e6d"} Dec 03 12:01:16 crc kubenswrapper[5052]: I1203 12:01:16.121966 5052 generic.go:334] "Generic (PLEG): container finished" podID="8c848c79-8799-4e32-a635-958ce73341c4" containerID="ac84406b1507746af79e8f6d313cc521e0407948593d4c617a53957cf7d38341" exitCode=0 Dec 03 12:01:16 crc kubenswrapper[5052]: I1203 12:01:16.122005 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-proxy-6555cb554-n2dp6" event={"ID":"8c848c79-8799-4e32-a635-958ce73341c4","Type":"ContainerDied","Data":"ac84406b1507746af79e8f6d313cc521e0407948593d4c617a53957cf7d38341"} Dec 03 12:01:16 crc kubenswrapper[5052]: I1203 12:01:16.122115 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstackclient" Dec 03 12:01:16 crc kubenswrapper[5052]: I1203 12:01:16.122136 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:01:16 crc kubenswrapper[5052]: I1203 12:01:16.122184 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:01:16 crc kubenswrapper[5052]: I1203 12:01:16.122365 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:01:16 crc kubenswrapper[5052]: I1203 12:01:16.167847 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/keystone-db-sync-2jm7f"] Dec 03 12:01:16 crc kubenswrapper[5052]: I1203 12:01:16.176923 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-kd4bh\" (UniqueName: \"kubernetes.io/projected/da4dfbea-20a2-4638-bbbe-ed4bfee0ad49-kube-api-access-kd4bh\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:16 crc kubenswrapper[5052]: I1203 12:01:16.188660 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/keystone-bootstrap-jhbzq"] Dec 03 12:01:16 crc kubenswrapper[5052]: I1203 12:01:16.201085 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/keystone-db-sync-2jm7f"] Dec 03 12:01:16 crc kubenswrapper[5052]: I1203 12:01:16.214183 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/keystone-bootstrap-jhbzq"] Dec 03 12:01:16 crc kubenswrapper[5052]: I1203 12:01:16.223961 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/keystone-bd5c68479-wvv6d"] Dec 03 12:01:16 crc kubenswrapper[5052]: I1203 12:01:16.224402 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/keystone-bd5c68479-wvv6d" podUID="0432c361-154a-46a9-b675-04267061f13b" containerName="keystone-api" containerID="cri-o://13a3f64e26fc4afd5ac59ca4b43b427f4f0bb964e269ef968f7456ea48b99a59" gracePeriod=30 Dec 03 12:01:16 crc kubenswrapper[5052]: I1203 12:01:16.235234 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/keystone-cron-29412721-2k46m"] Dec 03 12:01:16 crc kubenswrapper[5052]: I1203 12:01:16.243885 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/keystone-cron-29412721-2k46m"] Dec 03 12:01:16 crc kubenswrapper[5052]: I1203 12:01:16.255832 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/openstack-galera-0"] Dec 03 12:01:16 crc kubenswrapper[5052]: I1203 12:01:16.277436 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-wfzg9"] Dec 03 12:01:16 crc kubenswrapper[5052]: I1203 12:01:16.287670 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/keystone-db-create-d5l4l"] Dec 03 12:01:16 crc kubenswrapper[5052]: I1203 12:01:16.296187 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/keystone-db-create-d5l4l"] Dec 03 12:01:16 crc kubenswrapper[5052]: I1203 12:01:16.306232 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/keystone-0297-account-create-update-txbhd"] Dec 03 12:01:16 crc kubenswrapper[5052]: I1203 12:01:16.320394 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/keystone-0297-account-create-update-txbhd"] Dec 03 12:01:16 crc kubenswrapper[5052]: E1203 12:01:16.348923 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="fa90af8ae17f19b8e657ed6b90a3d32798eac71027682bd29946af1de563200d" cmd=["/usr/bin/pgrep","-r","DRST","nova-scheduler"] Dec 03 12:01:16 crc kubenswrapper[5052]: E1203 12:01:16.354733 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="fa90af8ae17f19b8e657ed6b90a3d32798eac71027682bd29946af1de563200d" cmd=["/usr/bin/pgrep","-r","DRST","nova-scheduler"] Dec 03 12:01:16 crc kubenswrapper[5052]: E1203 12:01:16.355908 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="fa90af8ae17f19b8e657ed6b90a3d32798eac71027682bd29946af1de563200d" cmd=["/usr/bin/pgrep","-r","DRST","nova-scheduler"] Dec 03 12:01:16 crc kubenswrapper[5052]: E1203 12:01:16.355985 5052 prober.go:104] "Probe errored" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" probeType="Readiness" pod="openstack-kuttl-tests/nova-scheduler-0" podUID="73c1b1fb-5062-4c33-be17-72324f222396" containerName="nova-scheduler-scheduler" Dec 03 12:01:16 crc kubenswrapper[5052]: I1203 12:01:16.419437 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage01-crc" (UniqueName: "kubernetes.io/local-volume/local-storage01-crc") on node "crc" Dec 03 12:01:16 crc kubenswrapper[5052]: I1203 12:01:16.455866 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage10-crc" (UniqueName: "kubernetes.io/local-volume/local-storage10-crc") on node "crc" Dec 03 12:01:16 crc kubenswrapper[5052]: I1203 12:01:16.485341 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage01-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage01-crc\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:16 crc kubenswrapper[5052]: I1203 12:01:16.485372 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage10-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage10-crc\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:16 crc kubenswrapper[5052]: I1203 12:01:16.501752 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3cb4559f-542a-4076-82fb-c0ca88f9de94-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "3cb4559f-542a-4076-82fb-c0ca88f9de94" (UID: "3cb4559f-542a-4076-82fb-c0ca88f9de94"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:01:16 crc kubenswrapper[5052]: I1203 12:01:16.504009 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5e5ddf8c-6109-481f-beb0-0802de034912-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "5e5ddf8c-6109-481f-beb0-0802de034912" (UID: "5e5ddf8c-6109-481f-beb0-0802de034912"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:01:16 crc kubenswrapper[5052]: I1203 12:01:16.512459 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e8de8adf-e4e5-4a10-9722-9106357e67d0-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "e8de8adf-e4e5-4a10-9722-9106357e67d0" (UID: "e8de8adf-e4e5-4a10-9722-9106357e67d0"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:01:16 crc kubenswrapper[5052]: I1203 12:01:16.518158 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/914c0f8f-e5d1-4c63-b985-0bcd07f75374-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "914c0f8f-e5d1-4c63-b985-0bcd07f75374" (UID: "914c0f8f-e5d1-4c63-b985-0bcd07f75374"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:01:16 crc kubenswrapper[5052]: I1203 12:01:16.523003 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/da4dfbea-20a2-4638-bbbe-ed4bfee0ad49-sg-core-conf-yaml" (OuterVolumeSpecName: "sg-core-conf-yaml") pod "da4dfbea-20a2-4638-bbbe-ed4bfee0ad49" (UID: "da4dfbea-20a2-4638-bbbe-ed4bfee0ad49"). InnerVolumeSpecName "sg-core-conf-yaml". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:01:16 crc kubenswrapper[5052]: I1203 12:01:16.536287 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/914c0f8f-e5d1-4c63-b985-0bcd07f75374-config-data" (OuterVolumeSpecName: "config-data") pod "914c0f8f-e5d1-4c63-b985-0bcd07f75374" (UID: "914c0f8f-e5d1-4c63-b985-0bcd07f75374"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:01:16 crc kubenswrapper[5052]: I1203 12:01:16.555047 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/da4dfbea-20a2-4638-bbbe-ed4bfee0ad49-ceilometer-tls-certs" (OuterVolumeSpecName: "ceilometer-tls-certs") pod "da4dfbea-20a2-4638-bbbe-ed4bfee0ad49" (UID: "da4dfbea-20a2-4638-bbbe-ed4bfee0ad49"). InnerVolumeSpecName "ceilometer-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:01:16 crc kubenswrapper[5052]: I1203 12:01:16.566081 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/914c0f8f-e5d1-4c63-b985-0bcd07f75374-nova-novncproxy-tls-certs" (OuterVolumeSpecName: "nova-novncproxy-tls-certs") pod "914c0f8f-e5d1-4c63-b985-0bcd07f75374" (UID: "914c0f8f-e5d1-4c63-b985-0bcd07f75374"). InnerVolumeSpecName "nova-novncproxy-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:01:16 crc kubenswrapper[5052]: I1203 12:01:16.586570 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5e5ddf8c-6109-481f-beb0-0802de034912-ovn-northd-tls-certs" (OuterVolumeSpecName: "ovn-northd-tls-certs") pod "5e5ddf8c-6109-481f-beb0-0802de034912" (UID: "5e5ddf8c-6109-481f-beb0-0802de034912"). InnerVolumeSpecName "ovn-northd-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:01:16 crc kubenswrapper[5052]: I1203 12:01:16.590580 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/914c0f8f-e5d1-4c63-b985-0bcd07f75374-vencrypt-tls-certs" (OuterVolumeSpecName: "vencrypt-tls-certs") pod "914c0f8f-e5d1-4c63-b985-0bcd07f75374" (UID: "914c0f8f-e5d1-4c63-b985-0bcd07f75374"). InnerVolumeSpecName "vencrypt-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:01:16 crc kubenswrapper[5052]: I1203 12:01:16.591994 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovn-northd-tls-certs\" (UniqueName: \"kubernetes.io/secret/5e5ddf8c-6109-481f-beb0-0802de034912-ovn-northd-tls-certs\") pod \"5e5ddf8c-6109-481f-beb0-0802de034912\" (UID: \"5e5ddf8c-6109-481f-beb0-0802de034912\") " Dec 03 12:01:16 crc kubenswrapper[5052]: W1203 12:01:16.592151 5052 empty_dir.go:500] Warning: Unmount skipped because path does not exist: /var/lib/kubelet/pods/5e5ddf8c-6109-481f-beb0-0802de034912/volumes/kubernetes.io~secret/ovn-northd-tls-certs Dec 03 12:01:16 crc kubenswrapper[5052]: I1203 12:01:16.592177 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5e5ddf8c-6109-481f-beb0-0802de034912-ovn-northd-tls-certs" (OuterVolumeSpecName: "ovn-northd-tls-certs") pod "5e5ddf8c-6109-481f-beb0-0802de034912" (UID: "5e5ddf8c-6109-481f-beb0-0802de034912"). InnerVolumeSpecName "ovn-northd-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:01:16 crc kubenswrapper[5052]: I1203 12:01:16.624194 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e8de8adf-e4e5-4a10-9722-9106357e67d0-metrics-certs-tls-certs" (OuterVolumeSpecName: "metrics-certs-tls-certs") pod "e8de8adf-e4e5-4a10-9722-9106357e67d0" (UID: "e8de8adf-e4e5-4a10-9722-9106357e67d0"). InnerVolumeSpecName "metrics-certs-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:01:16 crc kubenswrapper[5052]: I1203 12:01:16.625293 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e8de8adf-e4e5-4a10-9722-9106357e67d0-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:16 crc kubenswrapper[5052]: I1203 12:01:16.625343 5052 reconciler_common.go:293] "Volume detached for volume \"nova-novncproxy-tls-certs\" (UniqueName: \"kubernetes.io/secret/914c0f8f-e5d1-4c63-b985-0bcd07f75374-nova-novncproxy-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:16 crc kubenswrapper[5052]: I1203 12:01:16.625354 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/914c0f8f-e5d1-4c63-b985-0bcd07f75374-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:16 crc kubenswrapper[5052]: I1203 12:01:16.625366 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/914c0f8f-e5d1-4c63-b985-0bcd07f75374-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:16 crc kubenswrapper[5052]: I1203 12:01:16.625375 5052 reconciler_common.go:293] "Volume detached for volume \"vencrypt-tls-certs\" (UniqueName: \"kubernetes.io/secret/914c0f8f-e5d1-4c63-b985-0bcd07f75374-vencrypt-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:16 crc kubenswrapper[5052]: I1203 12:01:16.625382 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5e5ddf8c-6109-481f-beb0-0802de034912-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:16 crc kubenswrapper[5052]: I1203 12:01:16.625392 5052 reconciler_common.go:293] "Volume detached for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/da4dfbea-20a2-4638-bbbe-ed4bfee0ad49-sg-core-conf-yaml\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:16 crc kubenswrapper[5052]: I1203 12:01:16.625419 5052 reconciler_common.go:293] "Volume detached for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/da4dfbea-20a2-4638-bbbe-ed4bfee0ad49-ceilometer-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:16 crc kubenswrapper[5052]: I1203 12:01:16.625427 5052 reconciler_common.go:293] "Volume detached for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/e8de8adf-e4e5-4a10-9722-9106357e67d0-metrics-certs-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:16 crc kubenswrapper[5052]: I1203 12:01:16.625435 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3cb4559f-542a-4076-82fb-c0ca88f9de94-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:16 crc kubenswrapper[5052]: I1203 12:01:16.625444 5052 reconciler_common.go:293] "Volume detached for volume \"ovn-northd-tls-certs\" (UniqueName: \"kubernetes.io/secret/5e5ddf8c-6109-481f-beb0-0802de034912-ovn-northd-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:16 crc kubenswrapper[5052]: I1203 12:01:16.626269 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder-db-create-gx2gn"] Dec 03 12:01:16 crc kubenswrapper[5052]: I1203 12:01:16.638645 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3cb4559f-542a-4076-82fb-c0ca88f9de94-ovsdbserver-sb-tls-certs" (OuterVolumeSpecName: "ovsdbserver-sb-tls-certs") pod "3cb4559f-542a-4076-82fb-c0ca88f9de94" (UID: "3cb4559f-542a-4076-82fb-c0ca88f9de94"). InnerVolumeSpecName "ovsdbserver-sb-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:01:16 crc kubenswrapper[5052]: I1203 12:01:16.640701 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/cinder-db-create-gx2gn"] Dec 03 12:01:16 crc kubenswrapper[5052]: I1203 12:01:16.647681 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5e5ddf8c-6109-481f-beb0-0802de034912-metrics-certs-tls-certs" (OuterVolumeSpecName: "metrics-certs-tls-certs") pod "5e5ddf8c-6109-481f-beb0-0802de034912" (UID: "5e5ddf8c-6109-481f-beb0-0802de034912"). InnerVolumeSpecName "metrics-certs-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:01:16 crc kubenswrapper[5052]: I1203 12:01:16.650034 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder-aaa7-account-create-update-z472g"] Dec 03 12:01:16 crc kubenswrapper[5052]: I1203 12:01:16.654364 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/da4dfbea-20a2-4638-bbbe-ed4bfee0ad49-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "da4dfbea-20a2-4638-bbbe-ed4bfee0ad49" (UID: "da4dfbea-20a2-4638-bbbe-ed4bfee0ad49"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:01:16 crc kubenswrapper[5052]: I1203 12:01:16.659865 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinderaaa7-account-delete-rdrqp"] Dec 03 12:01:16 crc kubenswrapper[5052]: I1203 12:01:16.670501 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/cinder-aaa7-account-create-update-z472g"] Dec 03 12:01:16 crc kubenswrapper[5052]: I1203 12:01:16.674098 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3cb4559f-542a-4076-82fb-c0ca88f9de94-metrics-certs-tls-certs" (OuterVolumeSpecName: "metrics-certs-tls-certs") pod "3cb4559f-542a-4076-82fb-c0ca88f9de94" (UID: "3cb4559f-542a-4076-82fb-c0ca88f9de94"). InnerVolumeSpecName "metrics-certs-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:01:16 crc kubenswrapper[5052]: I1203 12:01:16.678813 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/da4dfbea-20a2-4638-bbbe-ed4bfee0ad49-config-data" (OuterVolumeSpecName: "config-data") pod "da4dfbea-20a2-4638-bbbe-ed4bfee0ad49" (UID: "da4dfbea-20a2-4638-bbbe-ed4bfee0ad49"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:01:16 crc kubenswrapper[5052]: I1203 12:01:16.680291 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e8de8adf-e4e5-4a10-9722-9106357e67d0-ovsdbserver-nb-tls-certs" (OuterVolumeSpecName: "ovsdbserver-nb-tls-certs") pod "e8de8adf-e4e5-4a10-9722-9106357e67d0" (UID: "e8de8adf-e4e5-4a10-9722-9106357e67d0"). InnerVolumeSpecName "ovsdbserver-nb-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:01:16 crc kubenswrapper[5052]: I1203 12:01:16.681166 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/openstack-galera-0" podUID="75c87049-f7b9-4c6b-add4-2eec1f52deb2" containerName="galera" containerID="cri-o://f270b916e5373a4dc04617edb6ac8552ab0bed39941d9a4293aaa5264e513b53" gracePeriod=30 Dec 03 12:01:16 crc kubenswrapper[5052]: I1203 12:01:16.729017 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-db-create-nlxff"] Dec 03 12:01:16 crc kubenswrapper[5052]: I1203 12:01:16.729198 5052 reconciler_common.go:293] "Volume detached for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/3cb4559f-542a-4076-82fb-c0ca88f9de94-metrics-certs-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:16 crc kubenswrapper[5052]: I1203 12:01:16.729227 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/da4dfbea-20a2-4638-bbbe-ed4bfee0ad49-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:16 crc kubenswrapper[5052]: I1203 12:01:16.729239 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/da4dfbea-20a2-4638-bbbe-ed4bfee0ad49-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:16 crc kubenswrapper[5052]: I1203 12:01:16.729251 5052 reconciler_common.go:293] "Volume detached for volume \"ovsdbserver-sb-tls-certs\" (UniqueName: \"kubernetes.io/secret/3cb4559f-542a-4076-82fb-c0ca88f9de94-ovsdbserver-sb-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:16 crc kubenswrapper[5052]: I1203 12:01:16.729262 5052 reconciler_common.go:293] "Volume detached for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/5e5ddf8c-6109-481f-beb0-0802de034912-metrics-certs-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:16 crc kubenswrapper[5052]: I1203 12:01:16.729272 5052 reconciler_common.go:293] "Volume detached for volume \"ovsdbserver-nb-tls-certs\" (UniqueName: \"kubernetes.io/secret/e8de8adf-e4e5-4a10-9722-9106357e67d0-ovsdbserver-nb-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:16 crc kubenswrapper[5052]: I1203 12:01:16.746572 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/glance-db-create-nlxff"] Dec 03 12:01:16 crc kubenswrapper[5052]: I1203 12:01:16.754027 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-0d27-account-create-update-9jj4h"] Dec 03 12:01:16 crc kubenswrapper[5052]: I1203 12:01:16.774279 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance0d27-account-delete-brqv4"] Dec 03 12:01:16 crc kubenswrapper[5052]: I1203 12:01:16.783578 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/glance-0d27-account-create-update-9jj4h"] Dec 03 12:01:16 crc kubenswrapper[5052]: I1203 12:01:16.804982 5052 scope.go:117] "RemoveContainer" containerID="2d2f9846f42dc135f23ee0343741d94b75b793c15135ba895cc30047a367cc1e" Dec 03 12:01:16 crc kubenswrapper[5052]: I1203 12:01:16.952460 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/watcher-applier-0" Dec 03 12:01:16 crc kubenswrapper[5052]: I1203 12:01:16.957463 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/alertmanager-metric-storage-0" Dec 03 12:01:16 crc kubenswrapper[5052]: I1203 12:01:16.959558 5052 scope.go:117] "RemoveContainer" containerID="2d2f9846f42dc135f23ee0343741d94b75b793c15135ba895cc30047a367cc1e" Dec 03 12:01:16 crc kubenswrapper[5052]: E1203 12:01:16.962201 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2d2f9846f42dc135f23ee0343741d94b75b793c15135ba895cc30047a367cc1e\": container with ID starting with 2d2f9846f42dc135f23ee0343741d94b75b793c15135ba895cc30047a367cc1e not found: ID does not exist" containerID="2d2f9846f42dc135f23ee0343741d94b75b793c15135ba895cc30047a367cc1e" Dec 03 12:01:16 crc kubenswrapper[5052]: I1203 12:01:16.962237 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d2f9846f42dc135f23ee0343741d94b75b793c15135ba895cc30047a367cc1e"} err="failed to get container status \"2d2f9846f42dc135f23ee0343741d94b75b793c15135ba895cc30047a367cc1e\": rpc error: code = NotFound desc = could not find container \"2d2f9846f42dc135f23ee0343741d94b75b793c15135ba895cc30047a367cc1e\": container with ID starting with 2d2f9846f42dc135f23ee0343741d94b75b793c15135ba895cc30047a367cc1e not found: ID does not exist" Dec 03 12:01:16 crc kubenswrapper[5052]: I1203 12:01:16.962258 5052 scope.go:117] "RemoveContainer" containerID="903649a39fdac1f3614d17974bbdb299dee9a076158995226565f8b3dd246b89" Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.037418 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/7cbbb6c5-a087-400f-9beb-e740ae1ddfbd-cluster-tls-config\") pod \"7cbbb6c5-a087-400f-9beb-e740ae1ddfbd\" (UID: \"7cbbb6c5-a087-400f-9beb-e740ae1ddfbd\") " Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.037460 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vn6ws\" (UniqueName: \"kubernetes.io/projected/192fa797-c8b8-42d3-a689-e6a7dbc77282-kube-api-access-vn6ws\") pod \"192fa797-c8b8-42d3-a689-e6a7dbc77282\" (UID: \"192fa797-c8b8-42d3-a689-e6a7dbc77282\") " Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.037490 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7cbbb6c5-a087-400f-9beb-e740ae1ddfbd-tls-assets\") pod \"7cbbb6c5-a087-400f-9beb-e740ae1ddfbd\" (UID: \"7cbbb6c5-a087-400f-9beb-e740ae1ddfbd\") " Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.037556 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/192fa797-c8b8-42d3-a689-e6a7dbc77282-config-data\") pod \"192fa797-c8b8-42d3-a689-e6a7dbc77282\" (UID: \"192fa797-c8b8-42d3-a689-e6a7dbc77282\") " Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.037611 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kwp54\" (UniqueName: \"kubernetes.io/projected/7cbbb6c5-a087-400f-9beb-e740ae1ddfbd-kube-api-access-kwp54\") pod \"7cbbb6c5-a087-400f-9beb-e740ae1ddfbd\" (UID: \"7cbbb6c5-a087-400f-9beb-e740ae1ddfbd\") " Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.037640 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/192fa797-c8b8-42d3-a689-e6a7dbc77282-logs\") pod \"192fa797-c8b8-42d3-a689-e6a7dbc77282\" (UID: \"192fa797-c8b8-42d3-a689-e6a7dbc77282\") " Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.037682 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"alertmanager-metric-storage-db\" (UniqueName: \"kubernetes.io/empty-dir/7cbbb6c5-a087-400f-9beb-e740ae1ddfbd-alertmanager-metric-storage-db\") pod \"7cbbb6c5-a087-400f-9beb-e740ae1ddfbd\" (UID: \"7cbbb6c5-a087-400f-9beb-e740ae1ddfbd\") " Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.037743 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/192fa797-c8b8-42d3-a689-e6a7dbc77282-combined-ca-bundle\") pod \"192fa797-c8b8-42d3-a689-e6a7dbc77282\" (UID: \"192fa797-c8b8-42d3-a689-e6a7dbc77282\") " Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.037816 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7cbbb6c5-a087-400f-9beb-e740ae1ddfbd-web-config\") pod \"7cbbb6c5-a087-400f-9beb-e740ae1ddfbd\" (UID: \"7cbbb6c5-a087-400f-9beb-e740ae1ddfbd\") " Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.037889 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7cbbb6c5-a087-400f-9beb-e740ae1ddfbd-config-out\") pod \"7cbbb6c5-a087-400f-9beb-e740ae1ddfbd\" (UID: \"7cbbb6c5-a087-400f-9beb-e740ae1ddfbd\") " Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.037924 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/7cbbb6c5-a087-400f-9beb-e740ae1ddfbd-config-volume\") pod \"7cbbb6c5-a087-400f-9beb-e740ae1ddfbd\" (UID: \"7cbbb6c5-a087-400f-9beb-e740ae1ddfbd\") " Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.040397 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/prometheus-metric-storage-0" Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.040580 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7cbbb6c5-a087-400f-9beb-e740ae1ddfbd-alertmanager-metric-storage-db" (OuterVolumeSpecName: "alertmanager-metric-storage-db") pod "7cbbb6c5-a087-400f-9beb-e740ae1ddfbd" (UID: "7cbbb6c5-a087-400f-9beb-e740ae1ddfbd"). InnerVolumeSpecName "alertmanager-metric-storage-db". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.051995 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/192fa797-c8b8-42d3-a689-e6a7dbc77282-kube-api-access-vn6ws" (OuterVolumeSpecName: "kube-api-access-vn6ws") pod "192fa797-c8b8-42d3-a689-e6a7dbc77282" (UID: "192fa797-c8b8-42d3-a689-e6a7dbc77282"). InnerVolumeSpecName "kube-api-access-vn6ws". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.052363 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/192fa797-c8b8-42d3-a689-e6a7dbc77282-logs" (OuterVolumeSpecName: "logs") pod "192fa797-c8b8-42d3-a689-e6a7dbc77282" (UID: "192fa797-c8b8-42d3-a689-e6a7dbc77282"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.054006 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7cbbb6c5-a087-400f-9beb-e740ae1ddfbd-config-out" (OuterVolumeSpecName: "config-out") pod "7cbbb6c5-a087-400f-9beb-e740ae1ddfbd" (UID: "7cbbb6c5-a087-400f-9beb-e740ae1ddfbd"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.062361 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7cbbb6c5-a087-400f-9beb-e740ae1ddfbd-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "7cbbb6c5-a087-400f-9beb-e740ae1ddfbd" (UID: "7cbbb6c5-a087-400f-9beb-e740ae1ddfbd"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.065545 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7cbbb6c5-a087-400f-9beb-e740ae1ddfbd-kube-api-access-kwp54" (OuterVolumeSpecName: "kube-api-access-kwp54") pod "7cbbb6c5-a087-400f-9beb-e740ae1ddfbd" (UID: "7cbbb6c5-a087-400f-9beb-e740ae1ddfbd"). InnerVolumeSpecName "kube-api-access-kwp54". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.070574 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7cbbb6c5-a087-400f-9beb-e740ae1ddfbd-config-volume" (OuterVolumeSpecName: "config-volume") pod "7cbbb6c5-a087-400f-9beb-e740ae1ddfbd" (UID: "7cbbb6c5-a087-400f-9beb-e740ae1ddfbd"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.078196 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/cinder-api-0" podUID="0291d0e7-9abf-48dd-8d5e-c329f989b700" containerName="cinder-api" probeResult="failure" output="Get \"https://10.217.1.212:8776/healthcheck\": read tcp 10.217.0.2:42754->10.217.1.212:8776: read: connection reset by peer" Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.079942 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-proxy-6555cb554-n2dp6" Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.094991 5052 scope.go:117] "RemoveContainer" containerID="ef9cd2fbef68f2e1fe845f78b334b0ff9a4fcb0c22c21ce8f7ab13794d397239" Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.097733 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-sb-0"] Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.109538 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-sb-0"] Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.139944 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.140831 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/8c848c79-8799-4e32-a635-958ce73341c4-run-httpd\") pod \"8c848c79-8799-4e32-a635-958ce73341c4\" (UID: \"8c848c79-8799-4e32-a635-958ce73341c4\") " Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.140952 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/8c848c79-8799-4e32-a635-958ce73341c4-log-httpd\") pod \"8c848c79-8799-4e32-a635-958ce73341c4\" (UID: \"8c848c79-8799-4e32-a635-958ce73341c4\") " Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.141031 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/01bc82de-61dc-477e-b4cf-e3b0d32c99ba-config\") pod \"01bc82de-61dc-477e-b4cf-e3b0d32c99ba\" (UID: \"01bc82de-61dc-477e-b4cf-e3b0d32c99ba\") " Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.141110 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/01bc82de-61dc-477e-b4cf-e3b0d32c99ba-thanos-prometheus-http-client-file\") pod \"01bc82de-61dc-477e-b4cf-e3b0d32c99ba\" (UID: \"01bc82de-61dc-477e-b4cf-e3b0d32c99ba\") " Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.142132 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"prometheus-metric-storage-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/01bc82de-61dc-477e-b4cf-e3b0d32c99ba-prometheus-metric-storage-rulefiles-0\") pod \"01bc82de-61dc-477e-b4cf-e3b0d32c99ba\" (UID: \"01bc82de-61dc-477e-b4cf-e3b0d32c99ba\") " Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.142269 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/8c848c79-8799-4e32-a635-958ce73341c4-public-tls-certs\") pod \"8c848c79-8799-4e32-a635-958ce73341c4\" (UID: \"8c848c79-8799-4e32-a635-958ce73341c4\") " Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.142361 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/01bc82de-61dc-477e-b4cf-e3b0d32c99ba-web-config\") pod \"01bc82de-61dc-477e-b4cf-e3b0d32c99ba\" (UID: \"01bc82de-61dc-477e-b4cf-e3b0d32c99ba\") " Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.142466 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"prometheus-metric-storage-db\" (UniqueName: \"kubernetes.io/local-volume/local-storage19-crc\") pod \"01bc82de-61dc-477e-b4cf-e3b0d32c99ba\" (UID: \"01bc82de-61dc-477e-b4cf-e3b0d32c99ba\") " Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.142585 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"web-config-tls-secret-key-cert-metric-storage-promethe-dc638c2d\" (UniqueName: \"kubernetes.io/secret/01bc82de-61dc-477e-b4cf-e3b0d32c99ba-web-config-tls-secret-key-cert-metric-storage-promethe-dc638c2d\") pod \"01bc82de-61dc-477e-b4cf-e3b0d32c99ba\" (UID: \"01bc82de-61dc-477e-b4cf-e3b0d32c99ba\") " Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.142677 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/8c848c79-8799-4e32-a635-958ce73341c4-etc-swift\") pod \"8c848c79-8799-4e32-a635-958ce73341c4\" (UID: \"8c848c79-8799-4e32-a635-958ce73341c4\") " Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.142767 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8c848c79-8799-4e32-a635-958ce73341c4-combined-ca-bundle\") pod \"8c848c79-8799-4e32-a635-958ce73341c4\" (UID: \"8c848c79-8799-4e32-a635-958ce73341c4\") " Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.142942 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/8c848c79-8799-4e32-a635-958ce73341c4-internal-tls-certs\") pod \"8c848c79-8799-4e32-a635-958ce73341c4\" (UID: \"8c848c79-8799-4e32-a635-958ce73341c4\") " Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.143044 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/01bc82de-61dc-477e-b4cf-e3b0d32c99ba-config-out\") pod \"01bc82de-61dc-477e-b4cf-e3b0d32c99ba\" (UID: \"01bc82de-61dc-477e-b4cf-e3b0d32c99ba\") " Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.143135 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/8c848c79-8799-4e32-a635-958ce73341c4-config-data\") pod \"8c848c79-8799-4e32-a635-958ce73341c4\" (UID: \"8c848c79-8799-4e32-a635-958ce73341c4\") " Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.143247 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"secret-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/01bc82de-61dc-477e-b4cf-e3b0d32c99ba-secret-combined-ca-bundle\") pod \"01bc82de-61dc-477e-b4cf-e3b0d32c99ba\" (UID: \"01bc82de-61dc-477e-b4cf-e3b0d32c99ba\") " Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.143328 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2mmxb\" (UniqueName: \"kubernetes.io/projected/8c848c79-8799-4e32-a635-958ce73341c4-kube-api-access-2mmxb\") pod \"8c848c79-8799-4e32-a635-958ce73341c4\" (UID: \"8c848c79-8799-4e32-a635-958ce73341c4\") " Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.143405 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"web-config-tls-secret-cert-cert-metric-storage-prometh-dc638c2d\" (UniqueName: \"kubernetes.io/secret/01bc82de-61dc-477e-b4cf-e3b0d32c99ba-web-config-tls-secret-cert-cert-metric-storage-prometh-dc638c2d\") pod \"01bc82de-61dc-477e-b4cf-e3b0d32c99ba\" (UID: \"01bc82de-61dc-477e-b4cf-e3b0d32c99ba\") " Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.143526 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kj8bk\" (UniqueName: \"kubernetes.io/projected/01bc82de-61dc-477e-b4cf-e3b0d32c99ba-kube-api-access-kj8bk\") pod \"01bc82de-61dc-477e-b4cf-e3b0d32c99ba\" (UID: \"01bc82de-61dc-477e-b4cf-e3b0d32c99ba\") " Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.143630 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/01bc82de-61dc-477e-b4cf-e3b0d32c99ba-tls-assets\") pod \"01bc82de-61dc-477e-b4cf-e3b0d32c99ba\" (UID: \"01bc82de-61dc-477e-b4cf-e3b0d32c99ba\") " Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.144301 5052 reconciler_common.go:293] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7cbbb6c5-a087-400f-9beb-e740ae1ddfbd-config-out\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.144389 5052 reconciler_common.go:293] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/7cbbb6c5-a087-400f-9beb-e740ae1ddfbd-config-volume\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.145143 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-vn6ws\" (UniqueName: \"kubernetes.io/projected/192fa797-c8b8-42d3-a689-e6a7dbc77282-kube-api-access-vn6ws\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.145228 5052 reconciler_common.go:293] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7cbbb6c5-a087-400f-9beb-e740ae1ddfbd-tls-assets\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.145338 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-kwp54\" (UniqueName: \"kubernetes.io/projected/7cbbb6c5-a087-400f-9beb-e740ae1ddfbd-kube-api-access-kwp54\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.145396 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/192fa797-c8b8-42d3-a689-e6a7dbc77282-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.147313 5052 reconciler_common.go:293] "Volume detached for volume \"alertmanager-metric-storage-db\" (UniqueName: \"kubernetes.io/empty-dir/7cbbb6c5-a087-400f-9beb-e740ae1ddfbd-alertmanager-metric-storage-db\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.149405 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8c848c79-8799-4e32-a635-958ce73341c4-log-httpd" (OuterVolumeSpecName: "log-httpd") pod "8c848c79-8799-4e32-a635-958ce73341c4" (UID: "8c848c79-8799-4e32-a635-958ce73341c4"). InnerVolumeSpecName "log-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.150463 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8c848c79-8799-4e32-a635-958ce73341c4-run-httpd" (OuterVolumeSpecName: "run-httpd") pod "8c848c79-8799-4e32-a635-958ce73341c4" (UID: "8c848c79-8799-4e32-a635-958ce73341c4"). InnerVolumeSpecName "run-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.168645 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/01bc82de-61dc-477e-b4cf-e3b0d32c99ba-prometheus-metric-storage-rulefiles-0" (OuterVolumeSpecName: "prometheus-metric-storage-rulefiles-0") pod "01bc82de-61dc-477e-b4cf-e3b0d32c99ba" (UID: "01bc82de-61dc-477e-b4cf-e3b0d32c99ba"). InnerVolumeSpecName "prometheus-metric-storage-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.170006 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.175710 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican4aeb-account-delete-bvhdj" event={"ID":"d420a2c2-9b7b-400b-88a7-839a93cce573","Type":"ContainerStarted","Data":"428b572eb3f0325bc4882ff8b1b7b1e0f00b59900d1b774323cd1ac0a5bb910c"} Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.176422 5052 kubelet_pods.go:1007] "Unable to retrieve pull secret, the image pull may not succeed." pod="openstack-kuttl-tests/barbican4aeb-account-delete-bvhdj" secret="" err="secret \"galera-openstack-dockercfg-bq5g4\" not found" Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.184784 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ovn-northd-0"] Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.192439 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/novacell06be2-account-delete-7m8cr" event={"ID":"6343f4a5-892f-4118-a666-e6601d2df818","Type":"ContainerStarted","Data":"e4bacb322ed2aac7c9485b5f2c64807ad6675335614edb5a4ddbdabca0c1fa1c"} Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.193283 5052 kubelet_pods.go:1007] "Unable to retrieve pull secret, the image pull may not succeed." pod="openstack-kuttl-tests/novacell06be2-account-delete-7m8cr" secret="" err="secret \"galera-openstack-dockercfg-bq5g4\" not found" Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.200541 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ovn-northd-0"] Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.202551 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/01bc82de-61dc-477e-b4cf-e3b0d32c99ba-config-out" (OuterVolumeSpecName: "config-out") pod "01bc82de-61dc-477e-b4cf-e3b0d32c99ba" (UID: "01bc82de-61dc-477e-b4cf-e3b0d32c99ba"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.202736 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/01bc82de-61dc-477e-b4cf-e3b0d32c99ba-web-config-tls-secret-key-cert-metric-storage-promethe-dc638c2d" (OuterVolumeSpecName: "web-config-tls-secret-key-cert-metric-storage-promethe-dc638c2d") pod "01bc82de-61dc-477e-b4cf-e3b0d32c99ba" (UID: "01bc82de-61dc-477e-b4cf-e3b0d32c99ba"). InnerVolumeSpecName "web-config-tls-secret-key-cert-metric-storage-promethe-dc638c2d". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.203578 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/01bc82de-61dc-477e-b4cf-e3b0d32c99ba-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "01bc82de-61dc-477e-b4cf-e3b0d32c99ba" (UID: "01bc82de-61dc-477e-b4cf-e3b0d32c99ba"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.215153 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/01bc82de-61dc-477e-b4cf-e3b0d32c99ba-config" (OuterVolumeSpecName: "config") pod "01bc82de-61dc-477e-b4cf-e3b0d32c99ba" (UID: "01bc82de-61dc-477e-b4cf-e3b0d32c99ba"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.215206 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/01bc82de-61dc-477e-b4cf-e3b0d32c99ba-web-config-tls-secret-cert-cert-metric-storage-prometh-dc638c2d" (OuterVolumeSpecName: "web-config-tls-secret-cert-cert-metric-storage-prometh-dc638c2d") pod "01bc82de-61dc-477e-b4cf-e3b0d32c99ba" (UID: "01bc82de-61dc-477e-b4cf-e3b0d32c99ba"). InnerVolumeSpecName "web-config-tls-secret-cert-cert-metric-storage-prometh-dc638c2d". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.215173 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/01bc82de-61dc-477e-b4cf-e3b0d32c99ba-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "01bc82de-61dc-477e-b4cf-e3b0d32c99ba" (UID: "01bc82de-61dc-477e-b4cf-e3b0d32c99ba"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.215678 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-nb-0"] Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.216351 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/01bc82de-61dc-477e-b4cf-e3b0d32c99ba-kube-api-access-kj8bk" (OuterVolumeSpecName: "kube-api-access-kj8bk") pod "01bc82de-61dc-477e-b4cf-e3b0d32c99ba" (UID: "01bc82de-61dc-477e-b4cf-e3b0d32c99ba"). InnerVolumeSpecName "kube-api-access-kj8bk". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.219685 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/01bc82de-61dc-477e-b4cf-e3b0d32c99ba-secret-combined-ca-bundle" (OuterVolumeSpecName: "secret-combined-ca-bundle") pod "01bc82de-61dc-477e-b4cf-e3b0d32c99ba" (UID: "01bc82de-61dc-477e-b4cf-e3b0d32c99ba"). InnerVolumeSpecName "secret-combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.221104 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/alertmanager-metric-storage-0" event={"ID":"7cbbb6c5-a087-400f-9beb-e740ae1ddfbd","Type":"ContainerDied","Data":"361730d7e651e0be896723f73ddb01ac250f74201d7ec122093f7b12ce8b7836"} Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.221239 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/alertmanager-metric-storage-0" Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.225286 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8c848c79-8799-4e32-a635-958ce73341c4-kube-api-access-2mmxb" (OuterVolumeSpecName: "kube-api-access-2mmxb") pod "8c848c79-8799-4e32-a635-958ce73341c4" (UID: "8c848c79-8799-4e32-a635-958ce73341c4"). InnerVolumeSpecName "kube-api-access-2mmxb". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.225375 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8c848c79-8799-4e32-a635-958ce73341c4-etc-swift" (OuterVolumeSpecName: "etc-swift") pod "8c848c79-8799-4e32-a635-958ce73341c4" (UID: "8c848c79-8799-4e32-a635-958ce73341c4"). InnerVolumeSpecName "etc-swift". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.228058 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-nb-0"] Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.237922 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/kube-state-metrics-0"] Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.250952 5052 reconciler_common.go:293] "Volume detached for volume \"secret-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/01bc82de-61dc-477e-b4cf-e3b0d32c99ba-secret-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.251011 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-2mmxb\" (UniqueName: \"kubernetes.io/projected/8c848c79-8799-4e32-a635-958ce73341c4-kube-api-access-2mmxb\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.251024 5052 reconciler_common.go:293] "Volume detached for volume \"web-config-tls-secret-cert-cert-metric-storage-prometh-dc638c2d\" (UniqueName: \"kubernetes.io/secret/01bc82de-61dc-477e-b4cf-e3b0d32c99ba-web-config-tls-secret-cert-cert-metric-storage-prometh-dc638c2d\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.251037 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-kj8bk\" (UniqueName: \"kubernetes.io/projected/01bc82de-61dc-477e-b4cf-e3b0d32c99ba-kube-api-access-kj8bk\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.251051 5052 reconciler_common.go:293] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/01bc82de-61dc-477e-b4cf-e3b0d32c99ba-tls-assets\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.251063 5052 reconciler_common.go:293] "Volume detached for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/8c848c79-8799-4e32-a635-958ce73341c4-run-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.251075 5052 reconciler_common.go:293] "Volume detached for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/8c848c79-8799-4e32-a635-958ce73341c4-log-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.251084 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/01bc82de-61dc-477e-b4cf-e3b0d32c99ba-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.251095 5052 reconciler_common.go:293] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/01bc82de-61dc-477e-b4cf-e3b0d32c99ba-thanos-prometheus-http-client-file\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.251105 5052 reconciler_common.go:293] "Volume detached for volume \"prometheus-metric-storage-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/01bc82de-61dc-477e-b4cf-e3b0d32c99ba-prometheus-metric-storage-rulefiles-0\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.251116 5052 reconciler_common.go:293] "Volume detached for volume \"web-config-tls-secret-key-cert-metric-storage-promethe-dc638c2d\" (UniqueName: \"kubernetes.io/secret/01bc82de-61dc-477e-b4cf-e3b0d32c99ba-web-config-tls-secret-key-cert-metric-storage-promethe-dc638c2d\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.251127 5052 reconciler_common.go:293] "Volume detached for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/8c848c79-8799-4e32-a635-958ce73341c4-etc-swift\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.251137 5052 reconciler_common.go:293] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/01bc82de-61dc-477e-b4cf-e3b0d32c99ba-config-out\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:17 crc kubenswrapper[5052]: E1203 12:01:17.251233 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:01:17 crc kubenswrapper[5052]: E1203 12:01:17.251303 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6343f4a5-892f-4118-a666-e6601d2df818-operator-scripts podName:6343f4a5-892f-4118-a666-e6601d2df818 nodeName:}" failed. No retries permitted until 2025-12-03 12:01:17.751280478 +0000 UTC m=+2468.339522242 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/6343f4a5-892f-4118-a666-e6601d2df818-operator-scripts") pod "novacell06be2-account-delete-7m8cr" (UID: "6343f4a5-892f-4118-a666-e6601d2df818") : configmap "openstack-scripts" not found Dec 03 12:01:17 crc kubenswrapper[5052]: E1203 12:01:17.251586 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:01:17 crc kubenswrapper[5052]: E1203 12:01:17.251616 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d420a2c2-9b7b-400b-88a7-839a93cce573-operator-scripts podName:d420a2c2-9b7b-400b-88a7-839a93cce573 nodeName:}" failed. No retries permitted until 2025-12-03 12:01:17.751608097 +0000 UTC m=+2468.339849861 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/d420a2c2-9b7b-400b-88a7-839a93cce573-operator-scripts") pod "barbican4aeb-account-delete-bvhdj" (UID: "d420a2c2-9b7b-400b-88a7-839a93cce573") : configmap "openstack-scripts" not found Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.251647 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/kube-state-metrics-0"] Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.267555 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placemente203-account-delete-sdpql" event={"ID":"36225798-caeb-4514-9f51-c8a07fb6da9b","Type":"ContainerStarted","Data":"63357c27c7708913d6e53569543c75fd7424796b936ce997d25a360f5d1d6f99"} Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.268286 5052 kubelet_pods.go:1007] "Unable to retrieve pull secret, the image pull may not succeed." pod="openstack-kuttl-tests/placemente203-account-delete-sdpql" secret="" err="secret \"galera-openstack-dockercfg-bq5g4\" not found" Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.275540 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/watcher-db-create-sr6dr"] Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.276045 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance0d27-account-delete-brqv4" event={"ID":"1a97695f-322c-4c4c-834e-ecb2f27c2e97","Type":"ContainerStarted","Data":"8a30f3c7e814f9afe6fc81acd574630714d7b92fdabac257b238341c59e0cc48"} Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.276602 5052 kubelet_pods.go:1007] "Unable to retrieve pull secret, the image pull may not succeed." pod="openstack-kuttl-tests/glance0d27-account-delete-brqv4" secret="" err="secret \"galera-openstack-dockercfg-bq5g4\" not found" Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.283976 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/novaapif429-account-delete-qwz7z" event={"ID":"eaeff688-fe65-403f-a01a-dfff69db9fb9","Type":"ContainerStarted","Data":"f7eae34142d8abb346f6280e3bbdc08126ce436adb89c7086b7aa2dead8ef908"} Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.284563 5052 kubelet_pods.go:1007] "Unable to retrieve pull secret, the image pull may not succeed." pod="openstack-kuttl-tests/novaapif429-account-delete-qwz7z" secret="" err="secret \"galera-openstack-dockercfg-bq5g4\" not found" Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.349113 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage19-crc" (OuterVolumeSpecName: "prometheus-metric-storage-db") pod "01bc82de-61dc-477e-b4cf-e3b0d32c99ba" (UID: "01bc82de-61dc-477e-b4cf-e3b0d32c99ba"). InnerVolumeSpecName "local-storage19-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.349202 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/watcher-db-create-sr6dr"] Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.351347 5052 generic.go:334] "Generic (PLEG): container finished" podID="0291d0e7-9abf-48dd-8d5e-c329f989b700" containerID="8ddd0225baa5c9e8b247c70bd8287b4bcb735d619e3bc97d77620dc899e06c7e" exitCode=0 Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.352808 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"0291d0e7-9abf-48dd-8d5e-c329f989b700","Type":"ContainerDied","Data":"8ddd0225baa5c9e8b247c70bd8287b4bcb735d619e3bc97d77620dc899e06c7e"} Dec 03 12:01:17 crc kubenswrapper[5052]: E1203 12:01:17.375194 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:01:17 crc kubenswrapper[5052]: E1203 12:01:17.375287 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1a97695f-322c-4c4c-834e-ecb2f27c2e97-operator-scripts podName:1a97695f-322c-4c4c-834e-ecb2f27c2e97 nodeName:}" failed. No retries permitted until 2025-12-03 12:01:17.875265763 +0000 UTC m=+2468.463507527 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/1a97695f-322c-4c4c-834e-ecb2f27c2e97-operator-scripts") pod "glance0d27-account-delete-brqv4" (UID: "1a97695f-322c-4c4c-834e-ecb2f27c2e97") : configmap "openstack-scripts" not found Dec 03 12:01:17 crc kubenswrapper[5052]: E1203 12:01:17.375620 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:01:17 crc kubenswrapper[5052]: E1203 12:01:17.375675 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/36225798-caeb-4514-9f51-c8a07fb6da9b-operator-scripts podName:36225798-caeb-4514-9f51-c8a07fb6da9b nodeName:}" failed. No retries permitted until 2025-12-03 12:01:17.875658544 +0000 UTC m=+2468.463900308 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/36225798-caeb-4514-9f51-c8a07fb6da9b-operator-scripts") pod "placemente203-account-delete-sdpql" (UID: "36225798-caeb-4514-9f51-c8a07fb6da9b") : configmap "openstack-scripts" not found Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.375766 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage19-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage19-crc\") on node \"crc\" " Dec 03 12:01:17 crc kubenswrapper[5052]: E1203 12:01:17.376313 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:01:17 crc kubenswrapper[5052]: E1203 12:01:17.379761 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/eaeff688-fe65-403f-a01a-dfff69db9fb9-operator-scripts podName:eaeff688-fe65-403f-a01a-dfff69db9fb9 nodeName:}" failed. No retries permitted until 2025-12-03 12:01:17.876469837 +0000 UTC m=+2468.464711601 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/eaeff688-fe65-403f-a01a-dfff69db9fb9-operator-scripts") pod "novaapif429-account-delete-qwz7z" (UID: "eaeff688-fe65-403f-a01a-dfff69db9fb9") : configmap "openstack-scripts" not found Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.396946 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron0ea0-account-delete-lvffs" event={"ID":"c17bae4f-1c1b-4da2-9bb5-c84d6fdfa184","Type":"ContainerStarted","Data":"65e23cdd1d480da8bdf0b2780f64b3f291ae9fc3c647fcf6b00e0dd8e2199e7e"} Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.397805 5052 kubelet_pods.go:1007] "Unable to retrieve pull secret, the image pull may not succeed." pod="openstack-kuttl-tests/neutron0ea0-account-delete-lvffs" secret="" err="secret \"galera-openstack-dockercfg-bq5g4\" not found" Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.409266 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/prometheus-metric-storage-0" event={"ID":"01bc82de-61dc-477e-b4cf-e3b0d32c99ba","Type":"ContainerDied","Data":"49c771312739dd2b8423503d8d97fb0470cb22b362440cae06abd17cdefa0629"} Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.409669 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/prometheus-metric-storage-0" Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.416313 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/watcher-applier-0" event={"ID":"192fa797-c8b8-42d3-a689-e6a7dbc77282","Type":"ContainerDied","Data":"b35373cfc82c42a1e9a5c087fd12f5e905f199a9f619266cede30f1f7dd75eaf"} Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.416469 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/watcher-applier-0" Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.422253 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.423772 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"da4dfbea-20a2-4638-bbbe-ed4bfee0ad49","Type":"ContainerDied","Data":"9328c605b4db437b147c5c1a4f4757786be4e46886d96fefe2e7797e925c5676"} Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.427003 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/192fa797-c8b8-42d3-a689-e6a7dbc77282-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "192fa797-c8b8-42d3-a689-e6a7dbc77282" (UID: "192fa797-c8b8-42d3-a689-e6a7dbc77282"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.438358 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-wfzg9" event={"ID":"5105e4c1-412b-4c11-92db-2354ebcfc575","Type":"ContainerStarted","Data":"6206e2066b39d8bea7f976a4de9746fa94b987163d6375b6584d3022506471ca"} Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.445188 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8c848c79-8799-4e32-a635-958ce73341c4-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "8c848c79-8799-4e32-a635-958ce73341c4" (UID: "8c848c79-8799-4e32-a635-958ce73341c4"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.455477 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage19-crc" (UniqueName: "kubernetes.io/local-volume/local-storage19-crc") on node "crc" Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.458059 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8c848c79-8799-4e32-a635-958ce73341c4-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "8c848c79-8799-4e32-a635-958ce73341c4" (UID: "8c848c79-8799-4e32-a635-958ce73341c4"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.462762 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-proxy-6555cb554-n2dp6" event={"ID":"8c848c79-8799-4e32-a635-958ce73341c4","Type":"ContainerDied","Data":"7d4f5d001749900396d79238af55e15dacdbf7971ffed62de68ae484370385a2"} Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.463003 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-proxy-6555cb554-n2dp6" Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.473951 5052 kubelet_pods.go:1007] "Unable to retrieve pull secret, the image pull may not succeed." pod="openstack-kuttl-tests/watcher36ab-account-delete-cvjb7" secret="" err="secret \"galera-openstack-dockercfg-bq5g4\" not found" Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.474446 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/watcher36ab-account-delete-cvjb7" event={"ID":"b2a6be10-e7df-494c-be94-a8ede55e2868","Type":"ContainerStarted","Data":"0a5a8979c8ed65f7542409f1cb80cacdb93b2cadbc0a475c4d889d76b585b861"} Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.480225 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/192fa797-c8b8-42d3-a689-e6a7dbc77282-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.480251 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/8c848c79-8799-4e32-a635-958ce73341c4-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.480260 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage19-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage19-crc\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.480275 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/8c848c79-8799-4e32-a635-958ce73341c4-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:17 crc kubenswrapper[5052]: E1203 12:01:17.480635 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:01:17 crc kubenswrapper[5052]: E1203 12:01:17.480698 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c17bae4f-1c1b-4da2-9bb5-c84d6fdfa184-operator-scripts podName:c17bae4f-1c1b-4da2-9bb5-c84d6fdfa184 nodeName:}" failed. No retries permitted until 2025-12-03 12:01:17.980680366 +0000 UTC m=+2468.568922120 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/c17bae4f-1c1b-4da2-9bb5-c84d6fdfa184-operator-scripts") pod "neutron0ea0-account-delete-lvffs" (UID: "c17bae4f-1c1b-4da2-9bb5-c84d6fdfa184") : configmap "openstack-scripts" not found Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.499293 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/192fa797-c8b8-42d3-a689-e6a7dbc77282-config-data" (OuterVolumeSpecName: "config-data") pod "192fa797-c8b8-42d3-a689-e6a7dbc77282" (UID: "192fa797-c8b8-42d3-a689-e6a7dbc77282"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.503494 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7cbbb6c5-a087-400f-9beb-e740ae1ddfbd-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "7cbbb6c5-a087-400f-9beb-e740ae1ddfbd" (UID: "7cbbb6c5-a087-400f-9beb-e740ae1ddfbd"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.528206 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/watcher-36ab-account-create-update-spl42"] Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.543892 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/watcher36ab-account-delete-cvjb7"] Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.550779 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.558741 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/watcher-36ab-account-create-update-spl42"] Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.561576 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7cbbb6c5-a087-400f-9beb-e740ae1ddfbd-web-config" (OuterVolumeSpecName: "web-config") pod "7cbbb6c5-a087-400f-9beb-e740ae1ddfbd" (UID: "7cbbb6c5-a087-400f-9beb-e740ae1ddfbd"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.570585 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/barbican4aeb-account-delete-bvhdj" podStartSLOduration=5.5705701560000005 podStartE2EDuration="5.570570156s" podCreationTimestamp="2025-12-03 12:01:12 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:01:17.242739754 +0000 UTC m=+2467.830981518" watchObservedRunningTime="2025-12-03 12:01:17.570570156 +0000 UTC m=+2468.158811920" Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.577664 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/01bc82de-61dc-477e-b4cf-e3b0d32c99ba-web-config" (OuterVolumeSpecName: "web-config") pod "01bc82de-61dc-477e-b4cf-e3b0d32c99ba" (UID: "01bc82de-61dc-477e-b4cf-e3b0d32c99ba"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.583757 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/nova-metadata-0" podUID="98d3c14e-7549-441a-9532-f3d75833cdf8" containerName="nova-metadata-log" probeResult="failure" output="Get \"https://10.217.0.12:8775/\": read tcp 10.217.0.2:44166->10.217.0.12:8775: read: connection reset by peer" Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.584016 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/nova-metadata-0" podUID="98d3c14e-7549-441a-9532-f3d75833cdf8" containerName="nova-metadata-metadata" probeResult="failure" output="Get \"https://10.217.0.12:8775/\": read tcp 10.217.0.2:44152->10.217.0.12:8775: read: connection reset by peer" Dec 03 12:01:17 crc kubenswrapper[5052]: E1203 12:01:17.584199 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:01:17 crc kubenswrapper[5052]: E1203 12:01:17.584502 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b2a6be10-e7df-494c-be94-a8ede55e2868-operator-scripts podName:b2a6be10-e7df-494c-be94-a8ede55e2868 nodeName:}" failed. No retries permitted until 2025-12-03 12:01:18.084479984 +0000 UTC m=+2468.672721748 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/b2a6be10-e7df-494c-be94-a8ede55e2868-operator-scripts") pod "watcher36ab-account-delete-cvjb7" (UID: "b2a6be10-e7df-494c-be94-a8ede55e2868") : configmap "openstack-scripts" not found Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.584597 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/192fa797-c8b8-42d3-a689-e6a7dbc77282-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.584640 5052 reconciler_common.go:293] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/01bc82de-61dc-477e-b4cf-e3b0d32c99ba-web-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.584652 5052 reconciler_common.go:293] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7cbbb6c5-a087-400f-9beb-e740ae1ddfbd-web-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.584661 5052 reconciler_common.go:293] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/7cbbb6c5-a087-400f-9beb-e740ae1ddfbd-cluster-tls-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.642183 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/watcher-api-0" podUID="2d5669ca-8e5e-49b4-95e9-116eb1195030" containerName="watcher-api-log" probeResult="failure" output="Get \"https://10.217.1.213:9322/\": read tcp 10.217.0.2:46198->10.217.1.213:9322: read: connection reset by peer" Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.642622 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8c848c79-8799-4e32-a635-958ce73341c4-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "8c848c79-8799-4e32-a635-958ce73341c4" (UID: "8c848c79-8799-4e32-a635-958ce73341c4"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.642728 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/watcher-api-0" podUID="2d5669ca-8e5e-49b4-95e9-116eb1195030" containerName="watcher-api" probeResult="failure" output="Get \"https://10.217.1.213:9322/\": read tcp 10.217.0.2:46190->10.217.1.213:9322: read: connection reset by peer" Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.642774 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/placement-db-create-c7b84"] Dec 03 12:01:17 crc kubenswrapper[5052]: E1203 12:01:17.652001 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="84a4a97816b15f9f8b42611faefec10e993e636d5d1d408805fe13fa7df1d13e" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.665802 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/placement-db-create-c7b84"] Dec 03 12:01:17 crc kubenswrapper[5052]: E1203 12:01:17.670378 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="84a4a97816b15f9f8b42611faefec10e993e636d5d1d408805fe13fa7df1d13e" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:01:17 crc kubenswrapper[5052]: E1203 12:01:17.680817 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="84a4a97816b15f9f8b42611faefec10e993e636d5d1d408805fe13fa7df1d13e" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:01:17 crc kubenswrapper[5052]: E1203 12:01:17.681031 5052 prober.go:104] "Probe errored" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" probeType="Readiness" pod="openstack-kuttl-tests/nova-cell1-conductor-0" podUID="0a480cd2-586d-4213-beff-424b3d201f8e" containerName="nova-cell1-conductor-conductor" Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.682524 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/novacell06be2-account-delete-7m8cr" podStartSLOduration=5.682490046 podStartE2EDuration="5.682490046s" podCreationTimestamp="2025-12-03 12:01:12 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:01:17.274407499 +0000 UTC m=+2467.862649283" watchObservedRunningTime="2025-12-03 12:01:17.682490046 +0000 UTC m=+2468.270731810" Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.686716 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0c055858-1ccd-444e-a055-1bd2a919556f" path="/var/lib/kubelet/pods/0c055858-1ccd-444e-a055-1bd2a919556f/volumes" Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.687062 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8c848c79-8799-4e32-a635-958ce73341c4-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.687656 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="27904d79-b16e-4909-bf71-3a9f8f325597" path="/var/lib/kubelet/pods/27904d79-b16e-4909-bf71-3a9f8f325597/volumes" Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.690591 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3cb4559f-542a-4076-82fb-c0ca88f9de94" path="/var/lib/kubelet/pods/3cb4559f-542a-4076-82fb-c0ca88f9de94/volumes" Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.691248 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="534e7f12-9e45-4fd6-8bd4-dc712add1e1a" path="/var/lib/kubelet/pods/534e7f12-9e45-4fd6-8bd4-dc712add1e1a/volumes" Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.692320 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5e5ddf8c-6109-481f-beb0-0802de034912" path="/var/lib/kubelet/pods/5e5ddf8c-6109-481f-beb0-0802de034912/volumes" Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.692930 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6eda40cd-e0c7-4f9c-922a-e66672da99ed" path="/var/lib/kubelet/pods/6eda40cd-e0c7-4f9c-922a-e66672da99ed/volumes" Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.693467 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="71879aa1-8e5d-4f20-9f73-65637988af5c" path="/var/lib/kubelet/pods/71879aa1-8e5d-4f20-9f73-65637988af5c/volumes" Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.694664 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7b1f895c-b513-4cc1-bf2a-72b84f1cd594" path="/var/lib/kubelet/pods/7b1f895c-b513-4cc1-bf2a-72b84f1cd594/volumes" Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.698947 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="87e193c2-c4a3-4370-b683-b151324dbcec" path="/var/lib/kubelet/pods/87e193c2-c4a3-4370-b683-b151324dbcec/volumes" Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.699933 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="914c0f8f-e5d1-4c63-b985-0bcd07f75374" path="/var/lib/kubelet/pods/914c0f8f-e5d1-4c63-b985-0bcd07f75374/volumes" Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.700660 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ace50fc0-2d6c-4014-b576-0da48b39b4a5" path="/var/lib/kubelet/pods/ace50fc0-2d6c-4014-b576-0da48b39b4a5/volumes" Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.701216 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c856f6ae-8216-4bd5-a9dd-1d87b546854e" path="/var/lib/kubelet/pods/c856f6ae-8216-4bd5-a9dd-1d87b546854e/volumes" Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.712044 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d03bf1c0-7353-4836-a767-9a5b2295e8dc" path="/var/lib/kubelet/pods/d03bf1c0-7353-4836-a767-9a5b2295e8dc/volumes" Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.713660 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="da4dfbea-20a2-4638-bbbe-ed4bfee0ad49" path="/var/lib/kubelet/pods/da4dfbea-20a2-4638-bbbe-ed4bfee0ad49/volumes" Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.714977 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e8de8adf-e4e5-4a10-9722-9106357e67d0" path="/var/lib/kubelet/pods/e8de8adf-e4e5-4a10-9722-9106357e67d0/volumes" Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.715602 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f0bbecd8-194a-4067-8786-1da410979cbf" path="/var/lib/kubelet/pods/f0bbecd8-194a-4067-8786-1da410979cbf/volumes" Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.716144 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fb1f8e74-cde9-4196-8b2d-5d302e69de58" path="/var/lib/kubelet/pods/fb1f8e74-cde9-4196-8b2d-5d302e69de58/volumes" Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.718814 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fce46734-b4c7-4e69-9010-d23e2d5f372f" path="/var/lib/kubelet/pods/fce46734-b4c7-4e69-9010-d23e2d5f372f/volumes" Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.759684 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8c848c79-8799-4e32-a635-958ce73341c4-config-data" (OuterVolumeSpecName: "config-data") pod "8c848c79-8799-4e32-a635-958ce73341c4" (UID: "8c848c79-8799-4e32-a635-958ce73341c4"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.788443 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/8c848c79-8799-4e32-a635-958ce73341c4-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:17 crc kubenswrapper[5052]: E1203 12:01:17.790152 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:01:17 crc kubenswrapper[5052]: E1203 12:01:17.790445 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d420a2c2-9b7b-400b-88a7-839a93cce573-operator-scripts podName:d420a2c2-9b7b-400b-88a7-839a93cce573 nodeName:}" failed. No retries permitted until 2025-12-03 12:01:18.790428442 +0000 UTC m=+2469.378670206 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/d420a2c2-9b7b-400b-88a7-839a93cce573-operator-scripts") pod "barbican4aeb-account-delete-bvhdj" (UID: "d420a2c2-9b7b-400b-88a7-839a93cce573") : configmap "openstack-scripts" not found Dec 03 12:01:17 crc kubenswrapper[5052]: E1203 12:01:17.790397 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:01:17 crc kubenswrapper[5052]: E1203 12:01:17.790484 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6343f4a5-892f-4118-a666-e6601d2df818-operator-scripts podName:6343f4a5-892f-4118-a666-e6601d2df818 nodeName:}" failed. No retries permitted until 2025-12-03 12:01:18.790478363 +0000 UTC m=+2469.378720127 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/6343f4a5-892f-4118-a666-e6601d2df818-operator-scripts") pod "novacell06be2-account-delete-7m8cr" (UID: "6343f4a5-892f-4118-a666-e6601d2df818") : configmap "openstack-scripts" not found Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.828325 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/barbican-api-d6bc8866d-xrhs6" podUID="5785b60f-14d2-43c5-95e2-68741c6729f3" containerName="barbican-api-log" probeResult="failure" output="Get \"https://10.217.1.208:9311/healthcheck\": dial tcp 10.217.1.208:9311: connect: connection refused" Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.828714 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/barbican-api-d6bc8866d-xrhs6" podUID="5785b60f-14d2-43c5-95e2-68741c6729f3" containerName="barbican-api" probeResult="failure" output="Get \"https://10.217.1.208:9311/healthcheck\": dial tcp 10.217.1.208:9311: connect: connection refused" Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.866905 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/placemente203-account-delete-sdpql" podStartSLOduration=5.866885188 podStartE2EDuration="5.866885188s" podCreationTimestamp="2025-12-03 12:01:12 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:01:17.310569263 +0000 UTC m=+2467.898811037" watchObservedRunningTime="2025-12-03 12:01:17.866885188 +0000 UTC m=+2468.455126952" Dec 03 12:01:17 crc kubenswrapper[5052]: I1203 12:01:17.884304 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/glance0d27-account-delete-brqv4" podStartSLOduration=6.884283805 podStartE2EDuration="6.884283805s" podCreationTimestamp="2025-12-03 12:01:11 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:01:17.360101419 +0000 UTC m=+2467.948343183" watchObservedRunningTime="2025-12-03 12:01:17.884283805 +0000 UTC m=+2468.472525569" Dec 03 12:01:17 crc kubenswrapper[5052]: E1203 12:01:17.892017 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:01:17 crc kubenswrapper[5052]: E1203 12:01:17.892097 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1a97695f-322c-4c4c-834e-ecb2f27c2e97-operator-scripts podName:1a97695f-322c-4c4c-834e-ecb2f27c2e97 nodeName:}" failed. No retries permitted until 2025-12-03 12:01:18.892078348 +0000 UTC m=+2469.480320112 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/1a97695f-322c-4c4c-834e-ecb2f27c2e97-operator-scripts") pod "glance0d27-account-delete-brqv4" (UID: "1a97695f-322c-4c4c-834e-ecb2f27c2e97") : configmap "openstack-scripts" not found Dec 03 12:01:17 crc kubenswrapper[5052]: E1203 12:01:17.892646 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:01:17 crc kubenswrapper[5052]: E1203 12:01:17.892747 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/eaeff688-fe65-403f-a01a-dfff69db9fb9-operator-scripts podName:eaeff688-fe65-403f-a01a-dfff69db9fb9 nodeName:}" failed. No retries permitted until 2025-12-03 12:01:18.892718976 +0000 UTC m=+2469.480960740 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/eaeff688-fe65-403f-a01a-dfff69db9fb9-operator-scripts") pod "novaapif429-account-delete-qwz7z" (UID: "eaeff688-fe65-403f-a01a-dfff69db9fb9") : configmap "openstack-scripts" not found Dec 03 12:01:17 crc kubenswrapper[5052]: E1203 12:01:17.892768 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:01:17 crc kubenswrapper[5052]: E1203 12:01:17.892803 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/36225798-caeb-4514-9f51-c8a07fb6da9b-operator-scripts podName:36225798-caeb-4514-9f51-c8a07fb6da9b nodeName:}" failed. No retries permitted until 2025-12-03 12:01:18.892792359 +0000 UTC m=+2469.481034183 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/36225798-caeb-4514-9f51-c8a07fb6da9b-operator-scripts") pod "placemente203-account-delete-sdpql" (UID: "36225798-caeb-4514-9f51-c8a07fb6da9b") : configmap "openstack-scripts" not found Dec 03 12:01:18 crc kubenswrapper[5052]: E1203 12:01:18.004091 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:01:18 crc kubenswrapper[5052]: E1203 12:01:18.004259 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c17bae4f-1c1b-4da2-9bb5-c84d6fdfa184-operator-scripts podName:c17bae4f-1c1b-4da2-9bb5-c84d6fdfa184 nodeName:}" failed. No retries permitted until 2025-12-03 12:01:19.004216544 +0000 UTC m=+2469.592458308 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/c17bae4f-1c1b-4da2-9bb5-c84d6fdfa184-operator-scripts") pod "neutron0ea0-account-delete-lvffs" (UID: "c17bae4f-1c1b-4da2-9bb5-c84d6fdfa184") : configmap "openstack-scripts" not found Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.021007 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/watcher36ab-account-delete-cvjb7" podStartSLOduration=6.020992664 podStartE2EDuration="6.020992664s" podCreationTimestamp="2025-12-03 12:01:12 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:01:17.499203376 +0000 UTC m=+2468.087445140" watchObservedRunningTime="2025-12-03 12:01:18.020992664 +0000 UTC m=+2468.609234428" Dec 03 12:01:18 crc kubenswrapper[5052]: E1203 12:01:18.105590 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:01:18 crc kubenswrapper[5052]: E1203 12:01:18.105655 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b2a6be10-e7df-494c-be94-a8ede55e2868-operator-scripts podName:b2a6be10-e7df-494c-be94-a8ede55e2868 nodeName:}" failed. No retries permitted until 2025-12-03 12:01:19.105641714 +0000 UTC m=+2469.693883478 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/b2a6be10-e7df-494c-be94-a8ede55e2868-operator-scripts") pod "watcher36ab-account-delete-cvjb7" (UID: "b2a6be10-e7df-494c-be94-a8ede55e2868") : configmap "openstack-scripts" not found Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.250920 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/nova-api-0" podUID="1103e975-cc9a-435b-adc9-5c6492e9c629" containerName="nova-api-log" probeResult="failure" output="Get \"https://10.217.0.16:8774/\": dial tcp 10.217.0.16:8774: connect: connection refused" Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.250969 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/nova-api-0" podUID="1103e975-cc9a-435b-adc9-5c6492e9c629" containerName="nova-api-api" probeResult="failure" output="Get \"https://10.217.0.16:8774/\": dial tcp 10.217.0.16:8774: connect: connection refused" Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.492291 5052 generic.go:334] "Generic (PLEG): container finished" podID="e09397b4-fabb-4e9d-8695-669bf9f49c77" containerID="f80007b6133c4b4853d49d9ef314cd82a9852973ff7a6cf37381a6ca1b1ec967" exitCode=0 Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.497676 5052 generic.go:334] "Generic (PLEG): container finished" podID="1103e975-cc9a-435b-adc9-5c6492e9c629" containerID="aebafbbccc45118d70ddef8e3f76333d5b32b3641a77534948d68d7de2fc9fd0" exitCode=0 Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.501689 5052 generic.go:334] "Generic (PLEG): container finished" podID="98d3c14e-7549-441a-9532-f3d75833cdf8" containerID="1c18a9d55257352dbf809914f4dca8da241a548437c4cbcf603a968646bbaaf6" exitCode=0 Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.505886 5052 generic.go:334] "Generic (PLEG): container finished" podID="2d5669ca-8e5e-49b4-95e9-116eb1195030" containerID="7fa07f1e6f3c12445900ff78caa499630cedc8446b5a3e6c5c977fa8f1760705" exitCode=0 Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.509661 5052 kubelet_pods.go:1007] "Unable to retrieve pull secret, the image pull may not succeed." pod="openstack-kuttl-tests/novacell184d8-account-delete-4nmn9" secret="" err="secret \"galera-openstack-dockercfg-bq5g4\" not found" Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.510278 5052 generic.go:334] "Generic (PLEG): container finished" podID="0a480cd2-586d-4213-beff-424b3d201f8e" containerID="84a4a97816b15f9f8b42611faefec10e993e636d5d1d408805fe13fa7df1d13e" exitCode=0 Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.512275 5052 generic.go:334] "Generic (PLEG): container finished" podID="60f75996-dceb-4640-8fca-96fdcc03ebb9" containerID="5f92af45a915348c684ef49149ebdedefc7b5163a55e465615daf0ad05fa17b8" exitCode=0 Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.517700 5052 generic.go:334] "Generic (PLEG): container finished" podID="5105e4c1-412b-4c11-92db-2354ebcfc575" containerID="99454548152a276442d6ff5c809528ace0ce757a6eb67b07cc45504cedad329a" exitCode=0 Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.526850 5052 generic.go:334] "Generic (PLEG): container finished" podID="5785b60f-14d2-43c5-95e2-68741c6729f3" containerID="a7bae4cc8cec979afcf2817d0be9cd0e1558b2c480f44fd3bf85f63c239d1a5c" exitCode=0 Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.561687 5052 generic.go:334] "Generic (PLEG): container finished" podID="6de7dc2a-f19c-4cc3-9965-9be822dc6270" containerID="f562bb4258333809d9cdbdea6ecb675c5df538644d028c44e7b8954cc6a364c8" exitCode=0 Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.568455 5052 generic.go:334] "Generic (PLEG): container finished" podID="5a08cda2-5952-4740-a571-7058f56c3b58" containerID="57e4d8c014f57177dc5f916168de2b019cf9fcd67c0d39bcaf0f7357d2dbf4bc" exitCode=0 Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.572108 5052 generic.go:334] "Generic (PLEG): container finished" podID="51a9dfff-37f2-4fbe-b2d1-039f20bb4560" containerID="87bc47eac41558bc881491bf1b0295423263bce8a03bd87436868096e25a318d" exitCode=0 Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.588829 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/novacell184d8-account-delete-4nmn9" podStartSLOduration=6.588810618 podStartE2EDuration="6.588810618s" podCreationTimestamp="2025-12-03 12:01:12 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:01:18.547757544 +0000 UTC m=+2469.135999308" watchObservedRunningTime="2025-12-03 12:01:18.588810618 +0000 UTC m=+2469.177052382" Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.594056 5052 generic.go:334] "Generic (PLEG): container finished" podID="60ca40b3-79c7-49cc-9100-0a37c47e4462" containerID="eac316d032f9e574c9c5551e5d62e39ad5141fe5bd8033f5ce21da3b31105028" exitCode=0 Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.595059 5052 kubelet_pods.go:1007] "Unable to retrieve pull secret, the image pull may not succeed." pod="openstack-kuttl-tests/placemente203-account-delete-sdpql" secret="" err="secret \"galera-openstack-dockercfg-bq5g4\" not found" Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.595778 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance0d27-account-delete-brqv4" podUID="1a97695f-322c-4c4c-834e-ecb2f27c2e97" containerName="mariadb-account-delete" containerID="cri-o://8a30f3c7e814f9afe6fc81acd574630714d7b92fdabac257b238341c59e0cc48" gracePeriod=30 Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.596163 5052 kubelet_pods.go:1007] "Unable to retrieve pull secret, the image pull may not succeed." pod="openstack-kuttl-tests/barbican4aeb-account-delete-bvhdj" secret="" err="secret \"galera-openstack-dockercfg-bq5g4\" not found" Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.596656 5052 kubelet_pods.go:1007] "Unable to retrieve pull secret, the image pull may not succeed." pod="openstack-kuttl-tests/novacell06be2-account-delete-7m8cr" secret="" err="secret \"galera-openstack-dockercfg-bq5g4\" not found" Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.596883 5052 kubelet_pods.go:1007] "Unable to retrieve pull secret, the image pull may not succeed." pod="openstack-kuttl-tests/watcher36ab-account-delete-cvjb7" secret="" err="secret \"galera-openstack-dockercfg-bq5g4\" not found" Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.597306 5052 kubelet_pods.go:1007] "Unable to retrieve pull secret, the image pull may not succeed." pod="openstack-kuttl-tests/neutron0ea0-account-delete-lvffs" secret="" err="secret \"galera-openstack-dockercfg-bq5g4\" not found" Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.597460 5052 kubelet_pods.go:1007] "Unable to retrieve pull secret, the image pull may not succeed." pod="openstack-kuttl-tests/novaapif429-account-delete-qwz7z" secret="" err="secret \"galera-openstack-dockercfg-bq5g4\" not found" Dec 03 12:01:18 crc kubenswrapper[5052]: E1203 12:01:18.621017 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:01:18 crc kubenswrapper[5052]: E1203 12:01:18.621104 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/cf90ea08-b172-4549-9522-92be7a5b5623-operator-scripts podName:cf90ea08-b172-4549-9522-92be7a5b5623 nodeName:}" failed. No retries permitted until 2025-12-03 12:01:19.12108486 +0000 UTC m=+2469.709326624 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/cf90ea08-b172-4549-9522-92be7a5b5623-operator-scripts") pod "novacell184d8-account-delete-4nmn9" (UID: "cf90ea08-b172-4549-9522-92be7a5b5623") : configmap "openstack-scripts" not found Dec 03 12:01:18 crc kubenswrapper[5052]: E1203 12:01:18.722264 5052 kubelet.go:2526] "Housekeeping took longer than expected" err="housekeeping took too long" expected="1s" actual="1.05s" Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.722376 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-keystone-listener-6774dbcdb8-wzwf7" event={"ID":"e09397b4-fabb-4e9d-8695-669bf9f49c77","Type":"ContainerDied","Data":"f80007b6133c4b4853d49d9ef314cd82a9852973ff7a6cf37381a6ca1b1ec967"} Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.722405 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"1103e975-cc9a-435b-adc9-5c6492e9c629","Type":"ContainerDied","Data":"aebafbbccc45118d70ddef8e3f76333d5b32b3641a77534948d68d7de2fc9fd0"} Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.722423 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"98d3c14e-7549-441a-9532-f3d75833cdf8","Type":"ContainerDied","Data":"1c18a9d55257352dbf809914f4dca8da241a548437c4cbcf603a968646bbaaf6"} Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.722438 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/watcher-api-0" event={"ID":"2d5669ca-8e5e-49b4-95e9-116eb1195030","Type":"ContainerDied","Data":"7fa07f1e6f3c12445900ff78caa499630cedc8446b5a3e6c5c977fa8f1760705"} Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.722451 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/novacell184d8-account-delete-4nmn9" event={"ID":"cf90ea08-b172-4549-9522-92be7a5b5623","Type":"ContainerStarted","Data":"5fa05d1c2c7ef877a6224f4d7efc7e1e0e45168ddefbd565972822108fe67233"} Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.722463 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-conductor-0" event={"ID":"0a480cd2-586d-4213-beff-424b3d201f8e","Type":"ContainerDied","Data":"84a4a97816b15f9f8b42611faefec10e993e636d5d1d408805fe13fa7df1d13e"} Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.722478 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/placement-e203-account-create-update-kq6qb"] Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.722493 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/placemente203-account-delete-sdpql"] Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.722525 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/placement-e203-account-create-update-kq6qb"] Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.722543 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-worker-ff9447cb9-nmz9j" event={"ID":"60f75996-dceb-4640-8fca-96fdcc03ebb9","Type":"ContainerDied","Data":"5f92af45a915348c684ef49149ebdedefc7b5163a55e465615daf0ad05fa17b8"} Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.722565 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-db-create-tl9wz"] Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.722577 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/barbican-db-create-tl9wz"] Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.722614 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican4aeb-account-delete-bvhdj"] Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.722630 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-wfzg9" event={"ID":"5105e4c1-412b-4c11-92db-2354ebcfc575","Type":"ContainerDied","Data":"99454548152a276442d6ff5c809528ace0ce757a6eb67b07cc45504cedad329a"} Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.722643 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-d6bc8866d-xrhs6" event={"ID":"5785b60f-14d2-43c5-95e2-68741c6729f3","Type":"ContainerDied","Data":"a7bae4cc8cec979afcf2817d0be9cd0e1558b2c480f44fd3bf85f63c239d1a5c"} Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.722657 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-4aeb-account-create-update-hn4jf"] Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.722671 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/barbican-4aeb-account-create-update-hn4jf"] Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.722684 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"0291d0e7-9abf-48dd-8d5e-c329f989b700","Type":"ContainerDied","Data":"dc0867961d380294a4a7f4bc8f4216631327967552db175aba11e79e252a54a4"} Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.722697 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="dc0867961d380294a4a7f4bc8f4216631327967552db175aba11e79e252a54a4" Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.722714 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"6de7dc2a-f19c-4cc3-9965-9be822dc6270","Type":"ContainerDied","Data":"f562bb4258333809d9cdbdea6ecb675c5df538644d028c44e7b8954cc6a364c8"} Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.722728 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/neutron-db-create-2s4jv"] Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.722743 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/neutron-db-create-2s4jv"] Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.722755 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"6de7dc2a-f19c-4cc3-9965-9be822dc6270","Type":"ContainerDied","Data":"2957fee30e0d0f082a9b76698b455e2162249f7b4cff38dcfa9f76793e9ba6d2"} Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.722764 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2957fee30e0d0f082a9b76698b455e2162249f7b4cff38dcfa9f76793e9ba6d2" Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.722774 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/neutron-0ea0-account-create-update-p5z4j"] Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.722785 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/memcached-0" event={"ID":"5a08cda2-5952-4740-a571-7058f56c3b58","Type":"ContainerDied","Data":"57e4d8c014f57177dc5f916168de2b019cf9fcd67c0d39bcaf0f7357d2dbf4bc"} Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.722798 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"51a9dfff-37f2-4fbe-b2d1-039f20bb4560","Type":"ContainerDied","Data":"87bc47eac41558bc881491bf1b0295423263bce8a03bd87436868096e25a318d"} Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.722815 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/neutron0ea0-account-delete-lvffs"] Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.722831 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/neutron-0ea0-account-create-update-p5z4j"] Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.722842 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell0-db-create-62hhq"] Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.722854 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell0-db-create-62hhq"] Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.722865 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"51a9dfff-37f2-4fbe-b2d1-039f20bb4560","Type":"ContainerDied","Data":"78d5d43de3a6f7f3e9852c4596ef7b083a06e8b8627e91c62a5f728ddf8c08f0"} Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.722874 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="78d5d43de3a6f7f3e9852c4596ef7b083a06e8b8627e91c62a5f728ddf8c08f0" Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.722884 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-7d9769b856-tzkn2" event={"ID":"60ca40b3-79c7-49cc-9100-0a37c47e4462","Type":"ContainerDied","Data":"eac316d032f9e574c9c5551e5d62e39ad5141fe5bd8033f5ce21da3b31105028"} Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.722897 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell0-6be2-account-create-update-2hj8c"] Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.722908 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-7d9769b856-tzkn2" event={"ID":"60ca40b3-79c7-49cc-9100-0a37c47e4462","Type":"ContainerDied","Data":"f81f8b7a9a04ac8ec5ef925547ed0d5543d363bc13231b720dca75d6ff169fbc"} Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.722916 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f81f8b7a9a04ac8ec5ef925547ed0d5543d363bc13231b720dca75d6ff169fbc" Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.722925 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/novacell06be2-account-delete-7m8cr"] Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.722939 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell0-6be2-account-create-update-2hj8c"] Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.722951 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-api-db-create-5rwbf"] Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.722962 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-api-db-create-5rwbf"] Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.722973 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-api-f429-account-create-update-9r5b7"] Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.722984 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/novaapif429-account-delete-qwz7z"] Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.722998 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-api-f429-account-create-update-9r5b7"] Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.723010 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell1-db-create-d4vgs"] Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.723022 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell1-db-create-d4vgs"] Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.723034 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/novacell184d8-account-delete-4nmn9"] Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.723057 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell1-84d8-account-create-update-d7f29"] Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.723069 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell1-84d8-account-create-update-d7f29"] Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.743521 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.745659 5052 scope.go:117] "RemoveContainer" containerID="3f16de4b1d15886789bdc38843cbe1fb5e5180679e89cb7fb19837deadc091ae" Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.757242 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/alertmanager-metric-storage-0" podUID="7cbbb6c5-a087-400f-9beb-e740ae1ddfbd" containerName="alertmanager" probeResult="failure" output="Get \"http://10.217.1.165:9093/-/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.781332 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.788939 5052 scope.go:117] "RemoveContainer" containerID="1658de9b62f64e764364ab2fb2ba523e5ca2b9210d45094169d4d8ec956ac253" Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.824100 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/0291d0e7-9abf-48dd-8d5e-c329f989b700-internal-tls-certs\") pod \"0291d0e7-9abf-48dd-8d5e-c329f989b700\" (UID: \"0291d0e7-9abf-48dd-8d5e-c329f989b700\") " Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.824130 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/6de7dc2a-f19c-4cc3-9965-9be822dc6270-logs\") pod \"6de7dc2a-f19c-4cc3-9965-9be822dc6270\" (UID: \"6de7dc2a-f19c-4cc3-9965-9be822dc6270\") " Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.824182 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"glance\" (UniqueName: \"kubernetes.io/local-volume/local-storage15-crc\") pod \"6de7dc2a-f19c-4cc3-9965-9be822dc6270\" (UID: \"6de7dc2a-f19c-4cc3-9965-9be822dc6270\") " Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.824222 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/0291d0e7-9abf-48dd-8d5e-c329f989b700-config-data\") pod \"0291d0e7-9abf-48dd-8d5e-c329f989b700\" (UID: \"0291d0e7-9abf-48dd-8d5e-c329f989b700\") " Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.824250 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/0291d0e7-9abf-48dd-8d5e-c329f989b700-logs\") pod \"0291d0e7-9abf-48dd-8d5e-c329f989b700\" (UID: \"0291d0e7-9abf-48dd-8d5e-c329f989b700\") " Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.824282 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/0291d0e7-9abf-48dd-8d5e-c329f989b700-config-data-custom\") pod \"0291d0e7-9abf-48dd-8d5e-c329f989b700\" (UID: \"0291d0e7-9abf-48dd-8d5e-c329f989b700\") " Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.824317 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nphb7\" (UniqueName: \"kubernetes.io/projected/0291d0e7-9abf-48dd-8d5e-c329f989b700-kube-api-access-nphb7\") pod \"0291d0e7-9abf-48dd-8d5e-c329f989b700\" (UID: \"0291d0e7-9abf-48dd-8d5e-c329f989b700\") " Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.824334 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/0291d0e7-9abf-48dd-8d5e-c329f989b700-public-tls-certs\") pod \"0291d0e7-9abf-48dd-8d5e-c329f989b700\" (UID: \"0291d0e7-9abf-48dd-8d5e-c329f989b700\") " Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.824349 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/6de7dc2a-f19c-4cc3-9965-9be822dc6270-internal-tls-certs\") pod \"6de7dc2a-f19c-4cc3-9965-9be822dc6270\" (UID: \"6de7dc2a-f19c-4cc3-9965-9be822dc6270\") " Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.824381 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6de7dc2a-f19c-4cc3-9965-9be822dc6270-config-data\") pod \"6de7dc2a-f19c-4cc3-9965-9be822dc6270\" (UID: \"6de7dc2a-f19c-4cc3-9965-9be822dc6270\") " Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.824413 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c4j2x\" (UniqueName: \"kubernetes.io/projected/6de7dc2a-f19c-4cc3-9965-9be822dc6270-kube-api-access-c4j2x\") pod \"6de7dc2a-f19c-4cc3-9965-9be822dc6270\" (UID: \"6de7dc2a-f19c-4cc3-9965-9be822dc6270\") " Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.824435 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6de7dc2a-f19c-4cc3-9965-9be822dc6270-combined-ca-bundle\") pod \"6de7dc2a-f19c-4cc3-9965-9be822dc6270\" (UID: \"6de7dc2a-f19c-4cc3-9965-9be822dc6270\") " Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.824470 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/6de7dc2a-f19c-4cc3-9965-9be822dc6270-scripts\") pod \"6de7dc2a-f19c-4cc3-9965-9be822dc6270\" (UID: \"6de7dc2a-f19c-4cc3-9965-9be822dc6270\") " Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.824503 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0291d0e7-9abf-48dd-8d5e-c329f989b700-combined-ca-bundle\") pod \"0291d0e7-9abf-48dd-8d5e-c329f989b700\" (UID: \"0291d0e7-9abf-48dd-8d5e-c329f989b700\") " Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.824617 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/0291d0e7-9abf-48dd-8d5e-c329f989b700-scripts\") pod \"0291d0e7-9abf-48dd-8d5e-c329f989b700\" (UID: \"0291d0e7-9abf-48dd-8d5e-c329f989b700\") " Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.824646 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/6de7dc2a-f19c-4cc3-9965-9be822dc6270-httpd-run\") pod \"6de7dc2a-f19c-4cc3-9965-9be822dc6270\" (UID: \"6de7dc2a-f19c-4cc3-9965-9be822dc6270\") " Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.824666 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/0291d0e7-9abf-48dd-8d5e-c329f989b700-etc-machine-id\") pod \"0291d0e7-9abf-48dd-8d5e-c329f989b700\" (UID: \"0291d0e7-9abf-48dd-8d5e-c329f989b700\") " Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.824706 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-7d9769b856-tzkn2" Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.825225 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0291d0e7-9abf-48dd-8d5e-c329f989b700-logs" (OuterVolumeSpecName: "logs") pod "0291d0e7-9abf-48dd-8d5e-c329f989b700" (UID: "0291d0e7-9abf-48dd-8d5e-c329f989b700"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.826584 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/0291d0e7-9abf-48dd-8d5e-c329f989b700-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:18 crc kubenswrapper[5052]: E1203 12:01:18.830720 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:01:18 crc kubenswrapper[5052]: E1203 12:01:18.830815 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d420a2c2-9b7b-400b-88a7-839a93cce573-operator-scripts podName:d420a2c2-9b7b-400b-88a7-839a93cce573 nodeName:}" failed. No retries permitted until 2025-12-03 12:01:20.830794146 +0000 UTC m=+2471.419036020 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/d420a2c2-9b7b-400b-88a7-839a93cce573-operator-scripts") pod "barbican4aeb-account-delete-bvhdj" (UID: "d420a2c2-9b7b-400b-88a7-839a93cce573") : configmap "openstack-scripts" not found Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.832805 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6de7dc2a-f19c-4cc3-9965-9be822dc6270-httpd-run" (OuterVolumeSpecName: "httpd-run") pod "6de7dc2a-f19c-4cc3-9965-9be822dc6270" (UID: "6de7dc2a-f19c-4cc3-9965-9be822dc6270"). InnerVolumeSpecName "httpd-run". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.832882 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/0291d0e7-9abf-48dd-8d5e-c329f989b700-etc-machine-id" (OuterVolumeSpecName: "etc-machine-id") pod "0291d0e7-9abf-48dd-8d5e-c329f989b700" (UID: "0291d0e7-9abf-48dd-8d5e-c329f989b700"). InnerVolumeSpecName "etc-machine-id". PluginName "kubernetes.io/host-path", VolumeGidValue "" Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.834093 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage15-crc" (OuterVolumeSpecName: "glance") pod "6de7dc2a-f19c-4cc3-9965-9be822dc6270" (UID: "6de7dc2a-f19c-4cc3-9965-9be822dc6270"). InnerVolumeSpecName "local-storage15-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 12:01:18 crc kubenswrapper[5052]: E1203 12:01:18.835449 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:01:18 crc kubenswrapper[5052]: E1203 12:01:18.835554 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6343f4a5-892f-4118-a666-e6601d2df818-operator-scripts podName:6343f4a5-892f-4118-a666-e6601d2df818 nodeName:}" failed. No retries permitted until 2025-12-03 12:01:20.83548813 +0000 UTC m=+2471.423729974 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/6343f4a5-892f-4118-a666-e6601d2df818-operator-scripts") pod "novacell06be2-account-delete-7m8cr" (UID: "6343f4a5-892f-4118-a666-e6601d2df818") : configmap "openstack-scripts" not found Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.849338 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6de7dc2a-f19c-4cc3-9965-9be822dc6270-logs" (OuterVolumeSpecName: "logs") pod "6de7dc2a-f19c-4cc3-9965-9be822dc6270" (UID: "6de7dc2a-f19c-4cc3-9965-9be822dc6270"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.853980 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0291d0e7-9abf-48dd-8d5e-c329f989b700-kube-api-access-nphb7" (OuterVolumeSpecName: "kube-api-access-nphb7") pod "0291d0e7-9abf-48dd-8d5e-c329f989b700" (UID: "0291d0e7-9abf-48dd-8d5e-c329f989b700"). InnerVolumeSpecName "kube-api-access-nphb7". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.858186 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.870432 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6de7dc2a-f19c-4cc3-9965-9be822dc6270-kube-api-access-c4j2x" (OuterVolumeSpecName: "kube-api-access-c4j2x") pod "6de7dc2a-f19c-4cc3-9965-9be822dc6270" (UID: "6de7dc2a-f19c-4cc3-9965-9be822dc6270"). InnerVolumeSpecName "kube-api-access-c4j2x". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.873199 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-api-d6bc8866d-xrhs6" Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.874794 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/swift-proxy-6555cb554-n2dp6"] Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.880005 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0291d0e7-9abf-48dd-8d5e-c329f989b700-scripts" (OuterVolumeSpecName: "scripts") pod "0291d0e7-9abf-48dd-8d5e-c329f989b700" (UID: "0291d0e7-9abf-48dd-8d5e-c329f989b700"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.881783 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6de7dc2a-f19c-4cc3-9965-9be822dc6270-scripts" (OuterVolumeSpecName: "scripts") pod "6de7dc2a-f19c-4cc3-9965-9be822dc6270" (UID: "6de7dc2a-f19c-4cc3-9965-9be822dc6270"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.885627 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinderaaa7-account-delete-rdrqp" Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.897419 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0291d0e7-9abf-48dd-8d5e-c329f989b700-config-data-custom" (OuterVolumeSpecName: "config-data-custom") pod "0291d0e7-9abf-48dd-8d5e-c329f989b700" (UID: "0291d0e7-9abf-48dd-8d5e-c329f989b700"). InnerVolumeSpecName "config-data-custom". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.909329 5052 scope.go:117] "RemoveContainer" containerID="079d8a9a5ad9c1f5d88409c04f6a13ba44666a756c6ea8ff7bd4d25cb9a1528e" Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.909665 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/memcached-0" Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.910554 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/swift-proxy-6555cb554-n2dp6"] Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.917583 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.924307 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/alertmanager-metric-storage-0"] Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.927652 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/5785b60f-14d2-43c5-95e2-68741c6729f3-logs\") pod \"5785b60f-14d2-43c5-95e2-68741c6729f3\" (UID: \"5785b60f-14d2-43c5-95e2-68741c6729f3\") " Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.927690 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/51a9dfff-37f2-4fbe-b2d1-039f20bb4560-scripts\") pod \"51a9dfff-37f2-4fbe-b2d1-039f20bb4560\" (UID: \"51a9dfff-37f2-4fbe-b2d1-039f20bb4560\") " Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.927714 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8mmxl\" (UniqueName: \"kubernetes.io/projected/60ca40b3-79c7-49cc-9100-0a37c47e4462-kube-api-access-8mmxl\") pod \"60ca40b3-79c7-49cc-9100-0a37c47e4462\" (UID: \"60ca40b3-79c7-49cc-9100-0a37c47e4462\") " Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.927743 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/5785b60f-14d2-43c5-95e2-68741c6729f3-internal-tls-certs\") pod \"5785b60f-14d2-43c5-95e2-68741c6729f3\" (UID: \"5785b60f-14d2-43c5-95e2-68741c6729f3\") " Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.927766 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/60ca40b3-79c7-49cc-9100-0a37c47e4462-logs\") pod \"60ca40b3-79c7-49cc-9100-0a37c47e4462\" (UID: \"60ca40b3-79c7-49cc-9100-0a37c47e4462\") " Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.927785 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"glance\" (UniqueName: \"kubernetes.io/local-volume/local-storage16-crc\") pod \"51a9dfff-37f2-4fbe-b2d1-039f20bb4560\" (UID: \"51a9dfff-37f2-4fbe-b2d1-039f20bb4560\") " Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.927803 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/60ca40b3-79c7-49cc-9100-0a37c47e4462-public-tls-certs\") pod \"60ca40b3-79c7-49cc-9100-0a37c47e4462\" (UID: \"60ca40b3-79c7-49cc-9100-0a37c47e4462\") " Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.927850 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/492ec2b7-0c28-4fe2-b2be-4e9e58423f23-operator-scripts\") pod \"492ec2b7-0c28-4fe2-b2be-4e9e58423f23\" (UID: \"492ec2b7-0c28-4fe2-b2be-4e9e58423f23\") " Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.927871 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qmrtr\" (UniqueName: \"kubernetes.io/projected/51a9dfff-37f2-4fbe-b2d1-039f20bb4560-kube-api-access-qmrtr\") pod \"51a9dfff-37f2-4fbe-b2d1-039f20bb4560\" (UID: \"51a9dfff-37f2-4fbe-b2d1-039f20bb4560\") " Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.927908 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/51a9dfff-37f2-4fbe-b2d1-039f20bb4560-combined-ca-bundle\") pod \"51a9dfff-37f2-4fbe-b2d1-039f20bb4560\" (UID: \"51a9dfff-37f2-4fbe-b2d1-039f20bb4560\") " Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.927926 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r5pbr\" (UniqueName: \"kubernetes.io/projected/492ec2b7-0c28-4fe2-b2be-4e9e58423f23-kube-api-access-r5pbr\") pod \"492ec2b7-0c28-4fe2-b2be-4e9e58423f23\" (UID: \"492ec2b7-0c28-4fe2-b2be-4e9e58423f23\") " Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.927949 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/51a9dfff-37f2-4fbe-b2d1-039f20bb4560-httpd-run\") pod \"51a9dfff-37f2-4fbe-b2d1-039f20bb4560\" (UID: \"51a9dfff-37f2-4fbe-b2d1-039f20bb4560\") " Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.927975 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/51a9dfff-37f2-4fbe-b2d1-039f20bb4560-public-tls-certs\") pod \"51a9dfff-37f2-4fbe-b2d1-039f20bb4560\" (UID: \"51a9dfff-37f2-4fbe-b2d1-039f20bb4560\") " Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.927996 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/5785b60f-14d2-43c5-95e2-68741c6729f3-config-data\") pod \"5785b60f-14d2-43c5-95e2-68741c6729f3\" (UID: \"5785b60f-14d2-43c5-95e2-68741c6729f3\") " Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.928016 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5785b60f-14d2-43c5-95e2-68741c6729f3-combined-ca-bundle\") pod \"5785b60f-14d2-43c5-95e2-68741c6729f3\" (UID: \"5785b60f-14d2-43c5-95e2-68741c6729f3\") " Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.928035 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/60ca40b3-79c7-49cc-9100-0a37c47e4462-combined-ca-bundle\") pod \"60ca40b3-79c7-49cc-9100-0a37c47e4462\" (UID: \"60ca40b3-79c7-49cc-9100-0a37c47e4462\") " Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.928050 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/60ca40b3-79c7-49cc-9100-0a37c47e4462-internal-tls-certs\") pod \"60ca40b3-79c7-49cc-9100-0a37c47e4462\" (UID: \"60ca40b3-79c7-49cc-9100-0a37c47e4462\") " Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.928101 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/60ca40b3-79c7-49cc-9100-0a37c47e4462-scripts\") pod \"60ca40b3-79c7-49cc-9100-0a37c47e4462\" (UID: \"60ca40b3-79c7-49cc-9100-0a37c47e4462\") " Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.928118 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/51a9dfff-37f2-4fbe-b2d1-039f20bb4560-logs\") pod \"51a9dfff-37f2-4fbe-b2d1-039f20bb4560\" (UID: \"51a9dfff-37f2-4fbe-b2d1-039f20bb4560\") " Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.928152 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/60ca40b3-79c7-49cc-9100-0a37c47e4462-config-data\") pod \"60ca40b3-79c7-49cc-9100-0a37c47e4462\" (UID: \"60ca40b3-79c7-49cc-9100-0a37c47e4462\") " Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.928205 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wphmx\" (UniqueName: \"kubernetes.io/projected/5785b60f-14d2-43c5-95e2-68741c6729f3-kube-api-access-wphmx\") pod \"5785b60f-14d2-43c5-95e2-68741c6729f3\" (UID: \"5785b60f-14d2-43c5-95e2-68741c6729f3\") " Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.928436 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/51a9dfff-37f2-4fbe-b2d1-039f20bb4560-config-data\") pod \"51a9dfff-37f2-4fbe-b2d1-039f20bb4560\" (UID: \"51a9dfff-37f2-4fbe-b2d1-039f20bb4560\") " Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.928464 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/5785b60f-14d2-43c5-95e2-68741c6729f3-public-tls-certs\") pod \"5785b60f-14d2-43c5-95e2-68741c6729f3\" (UID: \"5785b60f-14d2-43c5-95e2-68741c6729f3\") " Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.928499 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/5785b60f-14d2-43c5-95e2-68741c6729f3-config-data-custom\") pod \"5785b60f-14d2-43c5-95e2-68741c6729f3\" (UID: \"5785b60f-14d2-43c5-95e2-68741c6729f3\") " Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.928635 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5785b60f-14d2-43c5-95e2-68741c6729f3-logs" (OuterVolumeSpecName: "logs") pod "5785b60f-14d2-43c5-95e2-68741c6729f3" (UID: "5785b60f-14d2-43c5-95e2-68741c6729f3"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.929404 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage15-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage15-crc\") on node \"crc\" " Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.929425 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/0291d0e7-9abf-48dd-8d5e-c329f989b700-config-data-custom\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.929434 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-nphb7\" (UniqueName: \"kubernetes.io/projected/0291d0e7-9abf-48dd-8d5e-c329f989b700-kube-api-access-nphb7\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.929444 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-c4j2x\" (UniqueName: \"kubernetes.io/projected/6de7dc2a-f19c-4cc3-9965-9be822dc6270-kube-api-access-c4j2x\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.929452 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/6de7dc2a-f19c-4cc3-9965-9be822dc6270-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.929460 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/5785b60f-14d2-43c5-95e2-68741c6729f3-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.929467 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/0291d0e7-9abf-48dd-8d5e-c329f989b700-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.929476 5052 reconciler_common.go:293] "Volume detached for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/6de7dc2a-f19c-4cc3-9965-9be822dc6270-httpd-run\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.929483 5052 reconciler_common.go:293] "Volume detached for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/0291d0e7-9abf-48dd-8d5e-c329f989b700-etc-machine-id\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.929492 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/6de7dc2a-f19c-4cc3-9965-9be822dc6270-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.932282 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/51a9dfff-37f2-4fbe-b2d1-039f20bb4560-httpd-run" (OuterVolumeSpecName: "httpd-run") pod "51a9dfff-37f2-4fbe-b2d1-039f20bb4560" (UID: "51a9dfff-37f2-4fbe-b2d1-039f20bb4560"). InnerVolumeSpecName "httpd-run". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:01:18 crc kubenswrapper[5052]: E1203 12:01:18.932410 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/combined-ca-bundle: secret "combined-ca-bundle" not found Dec 03 12:01:18 crc kubenswrapper[5052]: E1203 12:01:18.932569 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1103e975-cc9a-435b-adc9-5c6492e9c629-combined-ca-bundle podName:1103e975-cc9a-435b-adc9-5c6492e9c629 nodeName:}" failed. No retries permitted until 2025-12-03 12:01:26.932551575 +0000 UTC m=+2477.520793339 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "combined-ca-bundle" (UniqueName: "kubernetes.io/secret/1103e975-cc9a-435b-adc9-5c6492e9c629-combined-ca-bundle") pod "nova-api-0" (UID: "1103e975-cc9a-435b-adc9-5c6492e9c629") : secret "combined-ca-bundle" not found Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.939378 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/51a9dfff-37f2-4fbe-b2d1-039f20bb4560-logs" (OuterVolumeSpecName: "logs") pod "51a9dfff-37f2-4fbe-b2d1-039f20bb4560" (UID: "51a9dfff-37f2-4fbe-b2d1-039f20bb4560"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:01:18 crc kubenswrapper[5052]: E1203 12:01:18.942994 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:01:18 crc kubenswrapper[5052]: E1203 12:01:18.943064 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/eaeff688-fe65-403f-a01a-dfff69db9fb9-operator-scripts podName:eaeff688-fe65-403f-a01a-dfff69db9fb9 nodeName:}" failed. No retries permitted until 2025-12-03 12:01:20.943044145 +0000 UTC m=+2471.531285969 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/eaeff688-fe65-403f-a01a-dfff69db9fb9-operator-scripts") pod "novaapif429-account-delete-qwz7z" (UID: "eaeff688-fe65-403f-a01a-dfff69db9fb9") : configmap "openstack-scripts" not found Dec 03 12:01:18 crc kubenswrapper[5052]: E1203 12:01:18.949645 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:01:18 crc kubenswrapper[5052]: E1203 12:01:18.949736 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1a97695f-322c-4c4c-834e-ecb2f27c2e97-operator-scripts podName:1a97695f-322c-4c4c-834e-ecb2f27c2e97 nodeName:}" failed. No retries permitted until 2025-12-03 12:01:20.949713656 +0000 UTC m=+2471.537955420 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/1a97695f-322c-4c4c-834e-ecb2f27c2e97-operator-scripts") pod "glance0d27-account-delete-brqv4" (UID: "1a97695f-322c-4c4c-834e-ecb2f27c2e97") : configmap "openstack-scripts" not found Dec 03 12:01:18 crc kubenswrapper[5052]: E1203 12:01:18.949823 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:01:18 crc kubenswrapper[5052]: E1203 12:01:18.949857 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/36225798-caeb-4514-9f51-c8a07fb6da9b-operator-scripts podName:36225798-caeb-4514-9f51-c8a07fb6da9b nodeName:}" failed. No retries permitted until 2025-12-03 12:01:20.949847899 +0000 UTC m=+2471.538089743 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/36225798-caeb-4514-9f51-c8a07fb6da9b-operator-scripts") pod "placemente203-account-delete-sdpql" (UID: "36225798-caeb-4514-9f51-c8a07fb6da9b") : configmap "openstack-scripts" not found Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.958001 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/watcher-api-0" Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.959640 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/alertmanager-metric-storage-0"] Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.959770 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage16-crc" (OuterVolumeSpecName: "glance") pod "51a9dfff-37f2-4fbe-b2d1-039f20bb4560" (UID: "51a9dfff-37f2-4fbe-b2d1-039f20bb4560"). InnerVolumeSpecName "local-storage16-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.960169 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60ca40b3-79c7-49cc-9100-0a37c47e4462-logs" (OuterVolumeSpecName: "logs") pod "60ca40b3-79c7-49cc-9100-0a37c47e4462" (UID: "60ca40b3-79c7-49cc-9100-0a37c47e4462"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.960804 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/492ec2b7-0c28-4fe2-b2be-4e9e58423f23-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "492ec2b7-0c28-4fe2-b2be-4e9e58423f23" (UID: "492ec2b7-0c28-4fe2-b2be-4e9e58423f23"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.962174 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/60ca40b3-79c7-49cc-9100-0a37c47e4462-scripts" (OuterVolumeSpecName: "scripts") pod "60ca40b3-79c7-49cc-9100-0a37c47e4462" (UID: "60ca40b3-79c7-49cc-9100-0a37c47e4462"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.962731 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.975559 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/prometheus-metric-storage-0"] Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.978181 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/492ec2b7-0c28-4fe2-b2be-4e9e58423f23-kube-api-access-r5pbr" (OuterVolumeSpecName: "kube-api-access-r5pbr") pod "492ec2b7-0c28-4fe2-b2be-4e9e58423f23" (UID: "492ec2b7-0c28-4fe2-b2be-4e9e58423f23"). InnerVolumeSpecName "kube-api-access-r5pbr". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.980005 5052 scope.go:117] "RemoveContainer" containerID="ae36686b468c95feb23e421e5898d4b9d5542290763ef55cdda58e32e8d7665b" Dec 03 12:01:18 crc kubenswrapper[5052]: I1203 12:01:18.983562 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/prometheus-metric-storage-0"] Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.001159 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/51a9dfff-37f2-4fbe-b2d1-039f20bb4560-scripts" (OuterVolumeSpecName: "scripts") pod "51a9dfff-37f2-4fbe-b2d1-039f20bb4560" (UID: "51a9dfff-37f2-4fbe-b2d1-039f20bb4560"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.018755 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/60ca40b3-79c7-49cc-9100-0a37c47e4462-kube-api-access-8mmxl" (OuterVolumeSpecName: "kube-api-access-8mmxl") pod "60ca40b3-79c7-49cc-9100-0a37c47e4462" (UID: "60ca40b3-79c7-49cc-9100-0a37c47e4462"). InnerVolumeSpecName "kube-api-access-8mmxl". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.022264 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/51a9dfff-37f2-4fbe-b2d1-039f20bb4560-kube-api-access-qmrtr" (OuterVolumeSpecName: "kube-api-access-qmrtr") pod "51a9dfff-37f2-4fbe-b2d1-039f20bb4560" (UID: "51a9dfff-37f2-4fbe-b2d1-039f20bb4560"). InnerVolumeSpecName "kube-api-access-qmrtr". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.022274 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5785b60f-14d2-43c5-95e2-68741c6729f3-config-data-custom" (OuterVolumeSpecName: "config-data-custom") pod "5785b60f-14d2-43c5-95e2-68741c6729f3" (UID: "5785b60f-14d2-43c5-95e2-68741c6729f3"). InnerVolumeSpecName "config-data-custom". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.022352 5052 scope.go:117] "RemoveContainer" containerID="daf20541a4af7f41da95e3513d45e8f6425015208b249e68097ecaea69b47670" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.026873 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5785b60f-14d2-43c5-95e2-68741c6729f3-kube-api-access-wphmx" (OuterVolumeSpecName: "kube-api-access-wphmx") pod "5785b60f-14d2-43c5-95e2-68741c6729f3" (UID: "5785b60f-14d2-43c5-95e2-68741c6729f3"). InnerVolumeSpecName "kube-api-access-wphmx". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.032077 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/98d3c14e-7549-441a-9532-f3d75833cdf8-combined-ca-bundle\") pod \"98d3c14e-7549-441a-9532-f3d75833cdf8\" (UID: \"98d3c14e-7549-441a-9532-f3d75833cdf8\") " Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.032159 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h786s\" (UniqueName: \"kubernetes.io/projected/98d3c14e-7549-441a-9532-f3d75833cdf8-kube-api-access-h786s\") pod \"98d3c14e-7549-441a-9532-f3d75833cdf8\" (UID: \"98d3c14e-7549-441a-9532-f3d75833cdf8\") " Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.032205 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/1103e975-cc9a-435b-adc9-5c6492e9c629-config-data\") pod \"1103e975-cc9a-435b-adc9-5c6492e9c629\" (UID: \"1103e975-cc9a-435b-adc9-5c6492e9c629\") " Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.032235 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/5a08cda2-5952-4740-a571-7058f56c3b58-config-data\") pod \"5a08cda2-5952-4740-a571-7058f56c3b58\" (UID: \"5a08cda2-5952-4740-a571-7058f56c3b58\") " Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.032321 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/2d5669ca-8e5e-49b4-95e9-116eb1195030-config-data\") pod \"2d5669ca-8e5e-49b4-95e9-116eb1195030\" (UID: \"2d5669ca-8e5e-49b4-95e9-116eb1195030\") " Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.032388 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/1103e975-cc9a-435b-adc9-5c6492e9c629-logs\") pod \"1103e975-cc9a-435b-adc9-5c6492e9c629\" (UID: \"1103e975-cc9a-435b-adc9-5c6492e9c629\") " Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.032423 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7ndkx\" (UniqueName: \"kubernetes.io/projected/2d5669ca-8e5e-49b4-95e9-116eb1195030-kube-api-access-7ndkx\") pod \"2d5669ca-8e5e-49b4-95e9-116eb1195030\" (UID: \"2d5669ca-8e5e-49b4-95e9-116eb1195030\") " Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.032454 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xrrfc\" (UniqueName: \"kubernetes.io/projected/5a08cda2-5952-4740-a571-7058f56c3b58-kube-api-access-xrrfc\") pod \"5a08cda2-5952-4740-a571-7058f56c3b58\" (UID: \"5a08cda2-5952-4740-a571-7058f56c3b58\") " Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.032475 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/2d5669ca-8e5e-49b4-95e9-116eb1195030-logs\") pod \"2d5669ca-8e5e-49b4-95e9-116eb1195030\" (UID: \"2d5669ca-8e5e-49b4-95e9-116eb1195030\") " Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.032525 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/98d3c14e-7549-441a-9532-f3d75833cdf8-nova-metadata-tls-certs\") pod \"98d3c14e-7549-441a-9532-f3d75833cdf8\" (UID: \"98d3c14e-7549-441a-9532-f3d75833cdf8\") " Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.032556 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1103e975-cc9a-435b-adc9-5c6492e9c629-combined-ca-bundle\") pod \"1103e975-cc9a-435b-adc9-5c6492e9c629\" (UID: \"1103e975-cc9a-435b-adc9-5c6492e9c629\") " Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.032598 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/98d3c14e-7549-441a-9532-f3d75833cdf8-logs\") pod \"98d3c14e-7549-441a-9532-f3d75833cdf8\" (UID: \"98d3c14e-7549-441a-9532-f3d75833cdf8\") " Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.032632 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2d5669ca-8e5e-49b4-95e9-116eb1195030-combined-ca-bundle\") pod \"2d5669ca-8e5e-49b4-95e9-116eb1195030\" (UID: \"2d5669ca-8e5e-49b4-95e9-116eb1195030\") " Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.032661 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/1103e975-cc9a-435b-adc9-5c6492e9c629-public-tls-certs\") pod \"1103e975-cc9a-435b-adc9-5c6492e9c629\" (UID: \"1103e975-cc9a-435b-adc9-5c6492e9c629\") " Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.032737 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5a08cda2-5952-4740-a571-7058f56c3b58-combined-ca-bundle\") pod \"5a08cda2-5952-4740-a571-7058f56c3b58\" (UID: \"5a08cda2-5952-4740-a571-7058f56c3b58\") " Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.032835 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"custom-prometheus-ca\" (UniqueName: \"kubernetes.io/secret/2d5669ca-8e5e-49b4-95e9-116eb1195030-custom-prometheus-ca\") pod \"2d5669ca-8e5e-49b4-95e9-116eb1195030\" (UID: \"2d5669ca-8e5e-49b4-95e9-116eb1195030\") " Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.032931 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/1103e975-cc9a-435b-adc9-5c6492e9c629-internal-tls-certs\") pod \"1103e975-cc9a-435b-adc9-5c6492e9c629\" (UID: \"1103e975-cc9a-435b-adc9-5c6492e9c629\") " Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.032996 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"memcached-tls-certs\" (UniqueName: \"kubernetes.io/secret/5a08cda2-5952-4740-a571-7058f56c3b58-memcached-tls-certs\") pod \"5a08cda2-5952-4740-a571-7058f56c3b58\" (UID: \"5a08cda2-5952-4740-a571-7058f56c3b58\") " Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.033093 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/98d3c14e-7549-441a-9532-f3d75833cdf8-config-data\") pod \"98d3c14e-7549-441a-9532-f3d75833cdf8\" (UID: \"98d3c14e-7549-441a-9532-f3d75833cdf8\") " Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.033157 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/2d5669ca-8e5e-49b4-95e9-116eb1195030-internal-tls-certs\") pod \"2d5669ca-8e5e-49b4-95e9-116eb1195030\" (UID: \"2d5669ca-8e5e-49b4-95e9-116eb1195030\") " Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.033198 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8brzf\" (UniqueName: \"kubernetes.io/projected/1103e975-cc9a-435b-adc9-5c6492e9c629-kube-api-access-8brzf\") pod \"1103e975-cc9a-435b-adc9-5c6492e9c629\" (UID: \"1103e975-cc9a-435b-adc9-5c6492e9c629\") " Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.033262 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/2d5669ca-8e5e-49b4-95e9-116eb1195030-public-tls-certs\") pod \"2d5669ca-8e5e-49b4-95e9-116eb1195030\" (UID: \"2d5669ca-8e5e-49b4-95e9-116eb1195030\") " Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.033360 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/5a08cda2-5952-4740-a571-7058f56c3b58-kolla-config\") pod \"5a08cda2-5952-4740-a571-7058f56c3b58\" (UID: \"5a08cda2-5952-4740-a571-7058f56c3b58\") " Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.034073 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/51a9dfff-37f2-4fbe-b2d1-039f20bb4560-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.034095 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-8mmxl\" (UniqueName: \"kubernetes.io/projected/60ca40b3-79c7-49cc-9100-0a37c47e4462-kube-api-access-8mmxl\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.034111 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/60ca40b3-79c7-49cc-9100-0a37c47e4462-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.034164 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage16-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage16-crc\") on node \"crc\" " Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.034182 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/492ec2b7-0c28-4fe2-b2be-4e9e58423f23-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.034196 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-qmrtr\" (UniqueName: \"kubernetes.io/projected/51a9dfff-37f2-4fbe-b2d1-039f20bb4560-kube-api-access-qmrtr\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.034235 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-r5pbr\" (UniqueName: \"kubernetes.io/projected/492ec2b7-0c28-4fe2-b2be-4e9e58423f23-kube-api-access-r5pbr\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.034249 5052 reconciler_common.go:293] "Volume detached for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/51a9dfff-37f2-4fbe-b2d1-039f20bb4560-httpd-run\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.034262 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/60ca40b3-79c7-49cc-9100-0a37c47e4462-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.034273 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/51a9dfff-37f2-4fbe-b2d1-039f20bb4560-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.034285 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-wphmx\" (UniqueName: \"kubernetes.io/projected/5785b60f-14d2-43c5-95e2-68741c6729f3-kube-api-access-wphmx\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.034322 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/5785b60f-14d2-43c5-95e2-68741c6729f3-config-data-custom\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:19 crc kubenswrapper[5052]: E1203 12:01:19.040610 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:01:19 crc kubenswrapper[5052]: E1203 12:01:19.040709 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c17bae4f-1c1b-4da2-9bb5-c84d6fdfa184-operator-scripts podName:c17bae4f-1c1b-4da2-9bb5-c84d6fdfa184 nodeName:}" failed. No retries permitted until 2025-12-03 12:01:21.040685685 +0000 UTC m=+2471.628927449 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/c17bae4f-1c1b-4da2-9bb5-c84d6fdfa184-operator-scripts") pod "neutron0ea0-account-delete-lvffs" (UID: "c17bae4f-1c1b-4da2-9bb5-c84d6fdfa184") : configmap "openstack-scripts" not found Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.042543 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2d5669ca-8e5e-49b4-95e9-116eb1195030-logs" (OuterVolumeSpecName: "logs") pod "2d5669ca-8e5e-49b4-95e9-116eb1195030" (UID: "2d5669ca-8e5e-49b4-95e9-116eb1195030"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.042831 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5a08cda2-5952-4740-a571-7058f56c3b58-kolla-config" (OuterVolumeSpecName: "kolla-config") pod "5a08cda2-5952-4740-a571-7058f56c3b58" (UID: "5a08cda2-5952-4740-a571-7058f56c3b58"). InnerVolumeSpecName "kolla-config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.042894 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5a08cda2-5952-4740-a571-7058f56c3b58-config-data" (OuterVolumeSpecName: "config-data") pod "5a08cda2-5952-4740-a571-7058f56c3b58" (UID: "5a08cda2-5952-4740-a571-7058f56c3b58"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.054606 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/98d3c14e-7549-441a-9532-f3d75833cdf8-logs" (OuterVolumeSpecName: "logs") pod "98d3c14e-7549-441a-9532-f3d75833cdf8" (UID: "98d3c14e-7549-441a-9532-f3d75833cdf8"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.058060 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1103e975-cc9a-435b-adc9-5c6492e9c629-logs" (OuterVolumeSpecName: "logs") pod "1103e975-cc9a-435b-adc9-5c6492e9c629" (UID: "1103e975-cc9a-435b-adc9-5c6492e9c629"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.058421 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/98d3c14e-7549-441a-9532-f3d75833cdf8-kube-api-access-h786s" (OuterVolumeSpecName: "kube-api-access-h786s") pod "98d3c14e-7549-441a-9532-f3d75833cdf8" (UID: "98d3c14e-7549-441a-9532-f3d75833cdf8"). InnerVolumeSpecName "kube-api-access-h786s". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.059494 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-worker-ff9447cb9-nmz9j" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.064295 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0291d0e7-9abf-48dd-8d5e-c329f989b700-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "0291d0e7-9abf-48dd-8d5e-c329f989b700" (UID: "0291d0e7-9abf-48dd-8d5e-c329f989b700"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.068323 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1103e975-cc9a-435b-adc9-5c6492e9c629-kube-api-access-8brzf" (OuterVolumeSpecName: "kube-api-access-8brzf") pod "1103e975-cc9a-435b-adc9-5c6492e9c629" (UID: "1103e975-cc9a-435b-adc9-5c6492e9c629"). InnerVolumeSpecName "kube-api-access-8brzf". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.068417 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2d5669ca-8e5e-49b4-95e9-116eb1195030-kube-api-access-7ndkx" (OuterVolumeSpecName: "kube-api-access-7ndkx") pod "2d5669ca-8e5e-49b4-95e9-116eb1195030" (UID: "2d5669ca-8e5e-49b4-95e9-116eb1195030"). InnerVolumeSpecName "kube-api-access-7ndkx". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.075981 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5a08cda2-5952-4740-a571-7058f56c3b58-kube-api-access-xrrfc" (OuterVolumeSpecName: "kube-api-access-xrrfc") pod "5a08cda2-5952-4740-a571-7058f56c3b58" (UID: "5a08cda2-5952-4740-a571-7058f56c3b58"). InnerVolumeSpecName "kube-api-access-xrrfc". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.077418 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6de7dc2a-f19c-4cc3-9965-9be822dc6270-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "6de7dc2a-f19c-4cc3-9965-9be822dc6270" (UID: "6de7dc2a-f19c-4cc3-9965-9be822dc6270"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.110802 5052 scope.go:117] "RemoveContainer" containerID="b2b8f2e1a2b0959c9d345484247345b165016e4ab99777de279ec024458c9da1" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.135340 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/60f75996-dceb-4640-8fca-96fdcc03ebb9-config-data-custom\") pod \"60f75996-dceb-4640-8fca-96fdcc03ebb9\" (UID: \"60f75996-dceb-4640-8fca-96fdcc03ebb9\") " Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.135382 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/60f75996-dceb-4640-8fca-96fdcc03ebb9-combined-ca-bundle\") pod \"60f75996-dceb-4640-8fca-96fdcc03ebb9\" (UID: \"60f75996-dceb-4640-8fca-96fdcc03ebb9\") " Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.135525 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/60f75996-dceb-4640-8fca-96fdcc03ebb9-config-data\") pod \"60f75996-dceb-4640-8fca-96fdcc03ebb9\" (UID: \"60f75996-dceb-4640-8fca-96fdcc03ebb9\") " Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.135692 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/60f75996-dceb-4640-8fca-96fdcc03ebb9-logs\") pod \"60f75996-dceb-4640-8fca-96fdcc03ebb9\" (UID: \"60f75996-dceb-4640-8fca-96fdcc03ebb9\") " Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.135723 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ljw82\" (UniqueName: \"kubernetes.io/projected/60f75996-dceb-4640-8fca-96fdcc03ebb9-kube-api-access-ljw82\") pod \"60f75996-dceb-4640-8fca-96fdcc03ebb9\" (UID: \"60f75996-dceb-4640-8fca-96fdcc03ebb9\") " Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.136149 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/1103e975-cc9a-435b-adc9-5c6492e9c629-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.136171 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-7ndkx\" (UniqueName: \"kubernetes.io/projected/2d5669ca-8e5e-49b4-95e9-116eb1195030-kube-api-access-7ndkx\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.136183 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-xrrfc\" (UniqueName: \"kubernetes.io/projected/5a08cda2-5952-4740-a571-7058f56c3b58-kube-api-access-xrrfc\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.136193 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/2d5669ca-8e5e-49b4-95e9-116eb1195030-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.136201 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/98d3c14e-7549-441a-9532-f3d75833cdf8-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.136211 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-8brzf\" (UniqueName: \"kubernetes.io/projected/1103e975-cc9a-435b-adc9-5c6492e9c629-kube-api-access-8brzf\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.136223 5052 reconciler_common.go:293] "Volume detached for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/5a08cda2-5952-4740-a571-7058f56c3b58-kolla-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.136233 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6de7dc2a-f19c-4cc3-9965-9be822dc6270-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.136243 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-h786s\" (UniqueName: \"kubernetes.io/projected/98d3c14e-7549-441a-9532-f3d75833cdf8-kube-api-access-h786s\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.136252 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/5a08cda2-5952-4740-a571-7058f56c3b58-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.136261 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0291d0e7-9abf-48dd-8d5e-c329f989b700-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:19 crc kubenswrapper[5052]: E1203 12:01:19.136313 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:01:19 crc kubenswrapper[5052]: E1203 12:01:19.136361 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/cf90ea08-b172-4549-9522-92be7a5b5623-operator-scripts podName:cf90ea08-b172-4549-9522-92be7a5b5623 nodeName:}" failed. No retries permitted until 2025-12-03 12:01:20.13634576 +0000 UTC m=+2470.724587524 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/cf90ea08-b172-4549-9522-92be7a5b5623-operator-scripts") pod "novacell184d8-account-delete-4nmn9" (UID: "cf90ea08-b172-4549-9522-92be7a5b5623") : configmap "openstack-scripts" not found Dec 03 12:01:19 crc kubenswrapper[5052]: E1203 12:01:19.136629 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:01:19 crc kubenswrapper[5052]: E1203 12:01:19.136723 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b2a6be10-e7df-494c-be94-a8ede55e2868-operator-scripts podName:b2a6be10-e7df-494c-be94-a8ede55e2868 nodeName:}" failed. No retries permitted until 2025-12-03 12:01:21.13669803 +0000 UTC m=+2471.724939824 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/b2a6be10-e7df-494c-be94-a8ede55e2868-operator-scripts") pod "watcher36ab-account-delete-cvjb7" (UID: "b2a6be10-e7df-494c-be94-a8ede55e2868") : configmap "openstack-scripts" not found Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.136792 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60f75996-dceb-4640-8fca-96fdcc03ebb9-logs" (OuterVolumeSpecName: "logs") pod "60f75996-dceb-4640-8fca-96fdcc03ebb9" (UID: "60f75996-dceb-4640-8fca-96fdcc03ebb9"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.148480 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage15-crc" (UniqueName: "kubernetes.io/local-volume/local-storage15-crc") on node "crc" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.154810 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/60f75996-dceb-4640-8fca-96fdcc03ebb9-kube-api-access-ljw82" (OuterVolumeSpecName: "kube-api-access-ljw82") pod "60f75996-dceb-4640-8fca-96fdcc03ebb9" (UID: "60f75996-dceb-4640-8fca-96fdcc03ebb9"). InnerVolumeSpecName "kube-api-access-ljw82". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.158753 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/60f75996-dceb-4640-8fca-96fdcc03ebb9-config-data-custom" (OuterVolumeSpecName: "config-data-custom") pod "60f75996-dceb-4640-8fca-96fdcc03ebb9" (UID: "60f75996-dceb-4640-8fca-96fdcc03ebb9"). InnerVolumeSpecName "config-data-custom". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.199502 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2d5669ca-8e5e-49b4-95e9-116eb1195030-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "2d5669ca-8e5e-49b4-95e9-116eb1195030" (UID: "2d5669ca-8e5e-49b4-95e9-116eb1195030"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.238777 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/60f75996-dceb-4640-8fca-96fdcc03ebb9-config-data-custom\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.238805 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2d5669ca-8e5e-49b4-95e9-116eb1195030-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.238833 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage15-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage15-crc\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.238843 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/60f75996-dceb-4640-8fca-96fdcc03ebb9-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.238851 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-ljw82\" (UniqueName: \"kubernetes.io/projected/60f75996-dceb-4640-8fca-96fdcc03ebb9-kube-api-access-ljw82\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.246849 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1103e975-cc9a-435b-adc9-5c6492e9c629-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "1103e975-cc9a-435b-adc9-5c6492e9c629" (UID: "1103e975-cc9a-435b-adc9-5c6492e9c629"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.247893 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0291d0e7-9abf-48dd-8d5e-c329f989b700-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "0291d0e7-9abf-48dd-8d5e-c329f989b700" (UID: "0291d0e7-9abf-48dd-8d5e-c329f989b700"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.253945 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage16-crc" (UniqueName: "kubernetes.io/local-volume/local-storage16-crc") on node "crc" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.259651 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/60f75996-dceb-4640-8fca-96fdcc03ebb9-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "60f75996-dceb-4640-8fca-96fdcc03ebb9" (UID: "60f75996-dceb-4640-8fca-96fdcc03ebb9"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.260129 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/51a9dfff-37f2-4fbe-b2d1-039f20bb4560-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "51a9dfff-37f2-4fbe-b2d1-039f20bb4560" (UID: "51a9dfff-37f2-4fbe-b2d1-039f20bb4560"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.274299 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6de7dc2a-f19c-4cc3-9965-9be822dc6270-config-data" (OuterVolumeSpecName: "config-data") pod "6de7dc2a-f19c-4cc3-9965-9be822dc6270" (UID: "6de7dc2a-f19c-4cc3-9965-9be822dc6270"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.274863 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5785b60f-14d2-43c5-95e2-68741c6729f3-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "5785b60f-14d2-43c5-95e2-68741c6729f3" (UID: "5785b60f-14d2-43c5-95e2-68741c6729f3"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.297908 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/98d3c14e-7549-441a-9532-f3d75833cdf8-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "98d3c14e-7549-441a-9532-f3d75833cdf8" (UID: "98d3c14e-7549-441a-9532-f3d75833cdf8"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.299873 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1103e975-cc9a-435b-adc9-5c6492e9c629-config-data" (OuterVolumeSpecName: "config-data") pod "1103e975-cc9a-435b-adc9-5c6492e9c629" (UID: "1103e975-cc9a-435b-adc9-5c6492e9c629"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.310634 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0291d0e7-9abf-48dd-8d5e-c329f989b700-config-data" (OuterVolumeSpecName: "config-data") pod "0291d0e7-9abf-48dd-8d5e-c329f989b700" (UID: "0291d0e7-9abf-48dd-8d5e-c329f989b700"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.345930 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage16-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage16-crc\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.345964 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/60f75996-dceb-4640-8fca-96fdcc03ebb9-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.345975 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1103e975-cc9a-435b-adc9-5c6492e9c629-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.345991 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/0291d0e7-9abf-48dd-8d5e-c329f989b700-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.346003 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/51a9dfff-37f2-4fbe-b2d1-039f20bb4560-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.346014 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/0291d0e7-9abf-48dd-8d5e-c329f989b700-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.346024 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5785b60f-14d2-43c5-95e2-68741c6729f3-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.346032 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6de7dc2a-f19c-4cc3-9965-9be822dc6270-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.346042 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/98d3c14e-7549-441a-9532-f3d75833cdf8-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.346050 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/1103e975-cc9a-435b-adc9-5c6492e9c629-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.349268 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5785b60f-14d2-43c5-95e2-68741c6729f3-config-data" (OuterVolumeSpecName: "config-data") pod "5785b60f-14d2-43c5-95e2-68741c6729f3" (UID: "5785b60f-14d2-43c5-95e2-68741c6729f3"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.355600 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1103e975-cc9a-435b-adc9-5c6492e9c629-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "1103e975-cc9a-435b-adc9-5c6492e9c629" (UID: "1103e975-cc9a-435b-adc9-5c6492e9c629"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.363817 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2d5669ca-8e5e-49b4-95e9-116eb1195030-config-data" (OuterVolumeSpecName: "config-data") pod "2d5669ca-8e5e-49b4-95e9-116eb1195030" (UID: "2d5669ca-8e5e-49b4-95e9-116eb1195030"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.365470 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2d5669ca-8e5e-49b4-95e9-116eb1195030-custom-prometheus-ca" (OuterVolumeSpecName: "custom-prometheus-ca") pod "2d5669ca-8e5e-49b4-95e9-116eb1195030" (UID: "2d5669ca-8e5e-49b4-95e9-116eb1195030"). InnerVolumeSpecName "custom-prometheus-ca". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.374723 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5a08cda2-5952-4740-a571-7058f56c3b58-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "5a08cda2-5952-4740-a571-7058f56c3b58" (UID: "5a08cda2-5952-4740-a571-7058f56c3b58"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.384961 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/51a9dfff-37f2-4fbe-b2d1-039f20bb4560-config-data" (OuterVolumeSpecName: "config-data") pod "51a9dfff-37f2-4fbe-b2d1-039f20bb4560" (UID: "51a9dfff-37f2-4fbe-b2d1-039f20bb4560"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.390315 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5a08cda2-5952-4740-a571-7058f56c3b58-memcached-tls-certs" (OuterVolumeSpecName: "memcached-tls-certs") pod "5a08cda2-5952-4740-a571-7058f56c3b58" (UID: "5a08cda2-5952-4740-a571-7058f56c3b58"). InnerVolumeSpecName "memcached-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.392045 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5785b60f-14d2-43c5-95e2-68741c6729f3-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "5785b60f-14d2-43c5-95e2-68741c6729f3" (UID: "5785b60f-14d2-43c5-95e2-68741c6729f3"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.403733 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/51a9dfff-37f2-4fbe-b2d1-039f20bb4560-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "51a9dfff-37f2-4fbe-b2d1-039f20bb4560" (UID: "51a9dfff-37f2-4fbe-b2d1-039f20bb4560"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.409675 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1103e975-cc9a-435b-adc9-5c6492e9c629-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "1103e975-cc9a-435b-adc9-5c6492e9c629" (UID: "1103e975-cc9a-435b-adc9-5c6492e9c629"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.411217 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/98d3c14e-7549-441a-9532-f3d75833cdf8-config-data" (OuterVolumeSpecName: "config-data") pod "98d3c14e-7549-441a-9532-f3d75833cdf8" (UID: "98d3c14e-7549-441a-9532-f3d75833cdf8"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.415826 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0291d0e7-9abf-48dd-8d5e-c329f989b700-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "0291d0e7-9abf-48dd-8d5e-c329f989b700" (UID: "0291d0e7-9abf-48dd-8d5e-c329f989b700"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.417390 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2d5669ca-8e5e-49b4-95e9-116eb1195030-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "2d5669ca-8e5e-49b4-95e9-116eb1195030" (UID: "2d5669ca-8e5e-49b4-95e9-116eb1195030"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.417398 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/60f75996-dceb-4640-8fca-96fdcc03ebb9-config-data" (OuterVolumeSpecName: "config-data") pod "60f75996-dceb-4640-8fca-96fdcc03ebb9" (UID: "60f75996-dceb-4640-8fca-96fdcc03ebb9"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.419537 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6de7dc2a-f19c-4cc3-9965-9be822dc6270-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "6de7dc2a-f19c-4cc3-9965-9be822dc6270" (UID: "6de7dc2a-f19c-4cc3-9965-9be822dc6270"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.422582 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2d5669ca-8e5e-49b4-95e9-116eb1195030-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "2d5669ca-8e5e-49b4-95e9-116eb1195030" (UID: "2d5669ca-8e5e-49b4-95e9-116eb1195030"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.425005 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5785b60f-14d2-43c5-95e2-68741c6729f3-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "5785b60f-14d2-43c5-95e2-68741c6729f3" (UID: "5785b60f-14d2-43c5-95e2-68741c6729f3"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.435336 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/60ca40b3-79c7-49cc-9100-0a37c47e4462-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "60ca40b3-79c7-49cc-9100-0a37c47e4462" (UID: "60ca40b3-79c7-49cc-9100-0a37c47e4462"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.438059 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/98d3c14e-7549-441a-9532-f3d75833cdf8-nova-metadata-tls-certs" (OuterVolumeSpecName: "nova-metadata-tls-certs") pod "98d3c14e-7549-441a-9532-f3d75833cdf8" (UID: "98d3c14e-7549-441a-9532-f3d75833cdf8"). InnerVolumeSpecName "nova-metadata-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.440082 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/60ca40b3-79c7-49cc-9100-0a37c47e4462-config-data" (OuterVolumeSpecName: "config-data") pod "60ca40b3-79c7-49cc-9100-0a37c47e4462" (UID: "60ca40b3-79c7-49cc-9100-0a37c47e4462"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.448285 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/51a9dfff-37f2-4fbe-b2d1-039f20bb4560-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.448325 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5a08cda2-5952-4740-a571-7058f56c3b58-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.448338 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/60f75996-dceb-4640-8fca-96fdcc03ebb9-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.448350 5052 reconciler_common.go:293] "Volume detached for volume \"custom-prometheus-ca\" (UniqueName: \"kubernetes.io/secret/2d5669ca-8e5e-49b4-95e9-116eb1195030-custom-prometheus-ca\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.448361 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/5785b60f-14d2-43c5-95e2-68741c6729f3-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.448372 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/60ca40b3-79c7-49cc-9100-0a37c47e4462-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.448385 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/1103e975-cc9a-435b-adc9-5c6492e9c629-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.448395 5052 reconciler_common.go:293] "Volume detached for volume \"memcached-tls-certs\" (UniqueName: \"kubernetes.io/secret/5a08cda2-5952-4740-a571-7058f56c3b58-memcached-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.448405 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/98d3c14e-7549-441a-9532-f3d75833cdf8-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.448417 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/0291d0e7-9abf-48dd-8d5e-c329f989b700-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.448426 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/2d5669ca-8e5e-49b4-95e9-116eb1195030-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.448438 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/6de7dc2a-f19c-4cc3-9965-9be822dc6270-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.448448 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/2d5669ca-8e5e-49b4-95e9-116eb1195030-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.448458 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/60ca40b3-79c7-49cc-9100-0a37c47e4462-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.448495 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/51a9dfff-37f2-4fbe-b2d1-039f20bb4560-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.448523 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/5785b60f-14d2-43c5-95e2-68741c6729f3-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.448536 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/2d5669ca-8e5e-49b4-95e9-116eb1195030-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.448547 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/5785b60f-14d2-43c5-95e2-68741c6729f3-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.448558 5052 reconciler_common.go:293] "Volume detached for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/98d3c14e-7549-441a-9532-f3d75833cdf8-nova-metadata-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.448570 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/1103e975-cc9a-435b-adc9-5c6492e9c629-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.464730 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/60ca40b3-79c7-49cc-9100-0a37c47e4462-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "60ca40b3-79c7-49cc-9100-0a37c47e4462" (UID: "60ca40b3-79c7-49cc-9100-0a37c47e4462"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.466523 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/60ca40b3-79c7-49cc-9100-0a37c47e4462-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "60ca40b3-79c7-49cc-9100-0a37c47e4462" (UID: "60ca40b3-79c7-49cc-9100-0a37c47e4462"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.549667 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/60ca40b3-79c7-49cc-9100-0a37c47e4462-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.549813 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/60ca40b3-79c7-49cc-9100-0a37c47e4462-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.649722 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"98d3c14e-7549-441a-9532-f3d75833cdf8","Type":"ContainerDied","Data":"fd4e9a275e8eb105d9f1d75c42811cdf01c0b028b69193cbfde45831f5ab9c5a"} Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.649860 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.655724 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-wfzg9" event={"ID":"5105e4c1-412b-4c11-92db-2354ebcfc575","Type":"ContainerStarted","Data":"58bc187a6aa5d3d79e23978554fc5a538422bd91d084cb518c12bb4a46bb1617"} Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.655911 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-wfzg9" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.659901 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-d6bc8866d-xrhs6" event={"ID":"5785b60f-14d2-43c5-95e2-68741c6729f3","Type":"ContainerDied","Data":"3138ba2a51029b0a2134fdcaa1df4d9d80072f97bc869d869f4cae9a03c70434"} Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.659916 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-api-d6bc8866d-xrhs6" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.666050 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-conductor-0" event={"ID":"0a480cd2-586d-4213-beff-424b3d201f8e","Type":"ContainerDied","Data":"6ac2bb981783b44c6199342e5aed0c278d04b5124604075498466b014b0e1e07"} Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.666083 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6ac2bb981783b44c6199342e5aed0c278d04b5124604075498466b014b0e1e07" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.668362 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinderaaa7-account-delete-rdrqp" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.668356 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinderaaa7-account-delete-rdrqp" event={"ID":"492ec2b7-0c28-4fe2-b2be-4e9e58423f23","Type":"ContainerDied","Data":"44f47cb05e42046d5daee7d260ecc89c1c45fb1139d010bcbf92cec26ea65196"} Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.668415 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="44f47cb05e42046d5daee7d260ecc89c1c45fb1139d010bcbf92cec26ea65196" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.677239 5052 generic.go:334] "Generic (PLEG): container finished" podID="0432c361-154a-46a9-b675-04267061f13b" containerID="13a3f64e26fc4afd5ac59ca4b43b427f4f0bb964e269ef968f7456ea48b99a59" exitCode=0 Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.677830 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-wfzg9" podStartSLOduration=7.677810581 podStartE2EDuration="7.677810581s" podCreationTimestamp="2025-12-03 12:01:12 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:01:19.675975728 +0000 UTC m=+2470.264217512" watchObservedRunningTime="2025-12-03 12:01:19.677810581 +0000 UTC m=+2470.266052345" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.688376 5052 scope.go:117] "RemoveContainer" containerID="d224717c6d9ca8135fd510e03a6c40c9f2b1a56dd8b4a3b685af3e2dfa855982" Dec 03 12:01:19 crc kubenswrapper[5052]: E1203 12:01:19.688612 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.689756 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-keystone-listener-6774dbcdb8-wzwf7" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.690850 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.703891 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.704011 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/watcher-api-0" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.704709 5052 scope.go:117] "RemoveContainer" containerID="3845a093071f0f8ca351aba36d4c3c56dd2d469ba1bfbea7b327c64e55ea871c" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.706258 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="01bc82de-61dc-477e-b4cf-e3b0d32c99ba" path="/var/lib/kubelet/pods/01bc82de-61dc-477e-b4cf-e3b0d32c99ba/volumes" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.706981 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="10051460-11e9-4e59-93c6-669a4d43d908" path="/var/lib/kubelet/pods/10051460-11e9-4e59-93c6-669a4d43d908/volumes" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.707439 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="15926354-9368-4283-8452-89eed0479195" path="/var/lib/kubelet/pods/15926354-9368-4283-8452-89eed0479195/volumes" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.708976 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="29f56a0c-db26-42fe-934f-0958d7e934cf" path="/var/lib/kubelet/pods/29f56a0c-db26-42fe-934f-0958d7e934cf/volumes" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.711001 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2ddd27d6-ad35-4919-8fe5-17100335e14a" path="/var/lib/kubelet/pods/2ddd27d6-ad35-4919-8fe5-17100335e14a/volumes" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.711712 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="30aeb633-e745-42cd-8f1b-cf60ac77d7ba" path="/var/lib/kubelet/pods/30aeb633-e745-42cd-8f1b-cf60ac77d7ba/volumes" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.712234 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4e7bcb77-fa54-4f5b-bb6f-77c12d876b11" path="/var/lib/kubelet/pods/4e7bcb77-fa54-4f5b-bb6f-77c12d876b11/volumes" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.713058 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5e3ad34f-c054-4dfd-a21e-f6a5d91d820f" path="/var/lib/kubelet/pods/5e3ad34f-c054-4dfd-a21e-f6a5d91d820f/volumes" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.713549 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="638e6ada-710e-43ce-9ac8-0569396886b3" path="/var/lib/kubelet/pods/638e6ada-710e-43ce-9ac8-0569396886b3/volumes" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.714084 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7b231c94-d92f-49e9-8ae1-93b2cc125936" path="/var/lib/kubelet/pods/7b231c94-d92f-49e9-8ae1-93b2cc125936/volumes" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.721254 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7cbbb6c5-a087-400f-9beb-e740ae1ddfbd" path="/var/lib/kubelet/pods/7cbbb6c5-a087-400f-9beb-e740ae1ddfbd/volumes" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.725631 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="842a3b9b-c6e6-4032-926a-95cc59d990f3" path="/var/lib/kubelet/pods/842a3b9b-c6e6-4032-926a-95cc59d990f3/volumes" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.727175 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8c848c79-8799-4e32-a635-958ce73341c4" path="/var/lib/kubelet/pods/8c848c79-8799-4e32-a635-958ce73341c4/volumes" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.728483 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e043d529-47c3-4741-85a3-bdb55c638a97" path="/var/lib/kubelet/pods/e043d529-47c3-4741-85a3-bdb55c638a97/volumes" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.739552 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-bd5c68479-wvv6d" event={"ID":"0432c361-154a-46a9-b675-04267061f13b","Type":"ContainerDied","Data":"13a3f64e26fc4afd5ac59ca4b43b427f4f0bb964e269ef968f7456ea48b99a59"} Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.739605 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-keystone-listener-6774dbcdb8-wzwf7" event={"ID":"e09397b4-fabb-4e9d-8695-669bf9f49c77","Type":"ContainerDied","Data":"d555975059be3bb86bfde48d46336b16ab9e981a5f964ae14484b856f1372aa1"} Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.739626 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"1103e975-cc9a-435b-adc9-5c6492e9c629","Type":"ContainerDied","Data":"5ae0d6b38d9ba8996463de010c21292bbc70e163c1d484bf8a332a74fedfbc7b"} Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.739645 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/watcher-api-0" event={"ID":"2d5669ca-8e5e-49b4-95e9-116eb1195030","Type":"ContainerDied","Data":"e99242523c7a011129425601f63e8df5ef6b39ce41e606227c6fdf6aeba438e3"} Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.745253 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/memcached-0" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.745250 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/memcached-0" event={"ID":"5a08cda2-5952-4740-a571-7058f56c3b58","Type":"ContainerDied","Data":"c72d5775a273d915d2fc20ae603ad1eabc0cee48e77e8afb58283ffb6c131c59"} Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.755015 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e09397b4-fabb-4e9d-8695-669bf9f49c77-combined-ca-bundle\") pod \"e09397b4-fabb-4e9d-8695-669bf9f49c77\" (UID: \"e09397b4-fabb-4e9d-8695-669bf9f49c77\") " Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.755093 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vnhjc\" (UniqueName: \"kubernetes.io/projected/0a480cd2-586d-4213-beff-424b3d201f8e-kube-api-access-vnhjc\") pod \"0a480cd2-586d-4213-beff-424b3d201f8e\" (UID: \"0a480cd2-586d-4213-beff-424b3d201f8e\") " Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.755359 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/0a480cd2-586d-4213-beff-424b3d201f8e-config-data\") pod \"0a480cd2-586d-4213-beff-424b3d201f8e\" (UID: \"0a480cd2-586d-4213-beff-424b3d201f8e\") " Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.755430 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0a480cd2-586d-4213-beff-424b3d201f8e-combined-ca-bundle\") pod \"0a480cd2-586d-4213-beff-424b3d201f8e\" (UID: \"0a480cd2-586d-4213-beff-424b3d201f8e\") " Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.755480 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/e09397b4-fabb-4e9d-8695-669bf9f49c77-config-data-custom\") pod \"e09397b4-fabb-4e9d-8695-669bf9f49c77\" (UID: \"e09397b4-fabb-4e9d-8695-669bf9f49c77\") " Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.755517 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/e09397b4-fabb-4e9d-8695-669bf9f49c77-config-data\") pod \"e09397b4-fabb-4e9d-8695-669bf9f49c77\" (UID: \"e09397b4-fabb-4e9d-8695-669bf9f49c77\") " Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.755546 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2pzd9\" (UniqueName: \"kubernetes.io/projected/e09397b4-fabb-4e9d-8695-669bf9f49c77-kube-api-access-2pzd9\") pod \"e09397b4-fabb-4e9d-8695-669bf9f49c77\" (UID: \"e09397b4-fabb-4e9d-8695-669bf9f49c77\") " Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.755574 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/e09397b4-fabb-4e9d-8695-669bf9f49c77-logs\") pod \"e09397b4-fabb-4e9d-8695-669bf9f49c77\" (UID: \"e09397b4-fabb-4e9d-8695-669bf9f49c77\") " Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.758463 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e09397b4-fabb-4e9d-8695-669bf9f49c77-logs" (OuterVolumeSpecName: "logs") pod "e09397b4-fabb-4e9d-8695-669bf9f49c77" (UID: "e09397b4-fabb-4e9d-8695-669bf9f49c77"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.761426 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0a480cd2-586d-4213-beff-424b3d201f8e-kube-api-access-vnhjc" (OuterVolumeSpecName: "kube-api-access-vnhjc") pod "0a480cd2-586d-4213-beff-424b3d201f8e" (UID: "0a480cd2-586d-4213-beff-424b3d201f8e"). InnerVolumeSpecName "kube-api-access-vnhjc". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.766210 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e09397b4-fabb-4e9d-8695-669bf9f49c77-kube-api-access-2pzd9" (OuterVolumeSpecName: "kube-api-access-2pzd9") pod "e09397b4-fabb-4e9d-8695-669bf9f49c77" (UID: "e09397b4-fabb-4e9d-8695-669bf9f49c77"). InnerVolumeSpecName "kube-api-access-2pzd9". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.779777 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/novacell06be2-account-delete-7m8cr" podUID="6343f4a5-892f-4118-a666-e6601d2df818" containerName="mariadb-account-delete" containerID="cri-o://e4bacb322ed2aac7c9485b5f2c64807ad6675335614edb5a4ddbdabca0c1fa1c" gracePeriod=30 Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.779922 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-worker-ff9447cb9-nmz9j" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.781309 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.782283 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-worker-ff9447cb9-nmz9j" event={"ID":"60f75996-dceb-4640-8fca-96fdcc03ebb9","Type":"ContainerDied","Data":"45f8d5c2251f538926e8b98dd1b8e9f03d54f5816e6f1edc0b901925bbb7b98e"} Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.782669 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.782740 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/novaapif429-account-delete-qwz7z" podUID="eaeff688-fe65-403f-a01a-dfff69db9fb9" containerName="mariadb-account-delete" containerID="cri-o://f7eae34142d8abb346f6280e3bbdc08126ce436adb89c7086b7aa2dead8ef908" gracePeriod=30 Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.782412 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/barbican4aeb-account-delete-bvhdj" podUID="d420a2c2-9b7b-400b-88a7-839a93cce573" containerName="mariadb-account-delete" containerID="cri-o://428b572eb3f0325bc4882ff8b1b7b1e0f00b59900d1b774323cd1ac0a5bb910c" gracePeriod=30 Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.783174 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/novacell184d8-account-delete-4nmn9" podUID="cf90ea08-b172-4549-9522-92be7a5b5623" containerName="mariadb-account-delete" containerID="cri-o://5fa05d1c2c7ef877a6224f4d7efc7e1e0e45168ddefbd565972822108fe67233" gracePeriod=30 Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.783436 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-7d9769b856-tzkn2" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.789316 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.779663 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/watcher36ab-account-delete-cvjb7" podUID="b2a6be10-e7df-494c-be94-a8ede55e2868" containerName="mariadb-account-delete" containerID="cri-o://0a5a8979c8ed65f7542409f1cb80cacdb93b2cadbc0a475c4d889d76b585b861" gracePeriod=30 Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.791385 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/neutron0ea0-account-delete-lvffs" podUID="c17bae4f-1c1b-4da2-9bb5-c84d6fdfa184" containerName="mariadb-account-delete" containerID="cri-o://65e23cdd1d480da8bdf0b2780f64b3f291ae9fc3c647fcf6b00e0dd8e2199e7e" gracePeriod=30 Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.797319 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/placemente203-account-delete-sdpql" podUID="36225798-caeb-4514-9f51-c8a07fb6da9b" containerName="mariadb-account-delete" containerID="cri-o://63357c27c7708913d6e53569543c75fd7424796b936ce997d25a360f5d1d6f99" gracePeriod=30 Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.797462 5052 scope.go:117] "RemoveContainer" containerID="599972ea725b2448de0f384ff58d4263ae4c6939731529af770910368c748187" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.798062 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e09397b4-fabb-4e9d-8695-669bf9f49c77-config-data-custom" (OuterVolumeSpecName: "config-data-custom") pod "e09397b4-fabb-4e9d-8695-669bf9f49c77" (UID: "e09397b4-fabb-4e9d-8695-669bf9f49c77"). InnerVolumeSpecName "config-data-custom". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.800497 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinderaaa7-account-delete-rdrqp"] Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.801636 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e09397b4-fabb-4e9d-8695-669bf9f49c77-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "e09397b4-fabb-4e9d-8695-669bf9f49c77" (UID: "e09397b4-fabb-4e9d-8695-669bf9f49c77"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.811165 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/cinderaaa7-account-delete-rdrqp"] Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.822842 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.826157 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0a480cd2-586d-4213-beff-424b3d201f8e-config-data" (OuterVolumeSpecName: "config-data") pod "0a480cd2-586d-4213-beff-424b3d201f8e" (UID: "0a480cd2-586d-4213-beff-424b3d201f8e"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.827425 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0a480cd2-586d-4213-beff-424b3d201f8e-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "0a480cd2-586d-4213-beff-424b3d201f8e" (UID: "0a480cd2-586d-4213-beff-424b3d201f8e"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.842773 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.851997 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-api-d6bc8866d-xrhs6"] Dec 03 12:01:19 crc kubenswrapper[5052]: E1203 12:01:19.865965 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/rabbitmq-config-data: configmap "rabbitmq-config-data" not found Dec 03 12:01:19 crc kubenswrapper[5052]: E1203 12:01:19.866083 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/30578d71-fe2b-4eed-9a07-6c8bfe4a1c43-config-data podName:30578d71-fe2b-4eed-9a07-6c8bfe4a1c43 nodeName:}" failed. No retries permitted until 2025-12-03 12:01:27.866047722 +0000 UTC m=+2478.454289486 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "config-data" (UniqueName: "kubernetes.io/configmap/30578d71-fe2b-4eed-9a07-6c8bfe4a1c43-config-data") pod "rabbitmq-server-0" (UID: "30578d71-fe2b-4eed-9a07-6c8bfe4a1c43") : configmap "rabbitmq-config-data" not found Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.871607 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/barbican-api-d6bc8866d-xrhs6"] Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.877690 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e09397b4-fabb-4e9d-8695-669bf9f49c77-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.877723 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-vnhjc\" (UniqueName: \"kubernetes.io/projected/0a480cd2-586d-4213-beff-424b3d201f8e-kube-api-access-vnhjc\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.877737 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/0a480cd2-586d-4213-beff-424b3d201f8e-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.877768 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0a480cd2-586d-4213-beff-424b3d201f8e-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.877784 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/e09397b4-fabb-4e9d-8695-669bf9f49c77-config-data-custom\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.877795 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-2pzd9\" (UniqueName: \"kubernetes.io/projected/e09397b4-fabb-4e9d-8695-669bf9f49c77-kube-api-access-2pzd9\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.877807 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/e09397b4-fabb-4e9d-8695-669bf9f49c77-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.881758 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.885475 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e09397b4-fabb-4e9d-8695-669bf9f49c77-config-data" (OuterVolumeSpecName: "config-data") pod "e09397b4-fabb-4e9d-8695-669bf9f49c77" (UID: "e09397b4-fabb-4e9d-8695-669bf9f49c77"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.886151 5052 scope.go:117] "RemoveContainer" containerID="be71930393f0fbaabf4c9dd57b05ba54c75c478e04e607ad33f67424c249c0f3" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.891148 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-bd5c68479-wvv6d" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.904418 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.934145 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.938237 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/watcher-api-0"] Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.942006 5052 scope.go:117] "RemoveContainer" containerID="0df8ce5588ea9142955dd14165fc02005b40b2952f5607fe472ebbf7177b2be8" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.946746 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/watcher-api-0"] Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.961186 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/memcached-0"] Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.973202 5052 scope.go:117] "RemoveContainer" containerID="ef9cd2fbef68f2e1fe845f78b334b0ff9a4fcb0c22c21ce8f7ab13794d397239" Dec 03 12:01:19 crc kubenswrapper[5052]: E1203 12:01:19.973885 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ef9cd2fbef68f2e1fe845f78b334b0ff9a4fcb0c22c21ce8f7ab13794d397239\": container with ID starting with ef9cd2fbef68f2e1fe845f78b334b0ff9a4fcb0c22c21ce8f7ab13794d397239 not found: ID does not exist" containerID="ef9cd2fbef68f2e1fe845f78b334b0ff9a4fcb0c22c21ce8f7ab13794d397239" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.973955 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ef9cd2fbef68f2e1fe845f78b334b0ff9a4fcb0c22c21ce8f7ab13794d397239"} err="failed to get container status \"ef9cd2fbef68f2e1fe845f78b334b0ff9a4fcb0c22c21ce8f7ab13794d397239\": rpc error: code = NotFound desc = could not find container \"ef9cd2fbef68f2e1fe845f78b334b0ff9a4fcb0c22c21ce8f7ab13794d397239\": container with ID starting with ef9cd2fbef68f2e1fe845f78b334b0ff9a4fcb0c22c21ce8f7ab13794d397239 not found: ID does not exist" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.973988 5052 scope.go:117] "RemoveContainer" containerID="3f16de4b1d15886789bdc38843cbe1fb5e5180679e89cb7fb19837deadc091ae" Dec 03 12:01:19 crc kubenswrapper[5052]: E1203 12:01:19.974488 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3f16de4b1d15886789bdc38843cbe1fb5e5180679e89cb7fb19837deadc091ae\": container with ID starting with 3f16de4b1d15886789bdc38843cbe1fb5e5180679e89cb7fb19837deadc091ae not found: ID does not exist" containerID="3f16de4b1d15886789bdc38843cbe1fb5e5180679e89cb7fb19837deadc091ae" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.974882 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3f16de4b1d15886789bdc38843cbe1fb5e5180679e89cb7fb19837deadc091ae"} err="failed to get container status \"3f16de4b1d15886789bdc38843cbe1fb5e5180679e89cb7fb19837deadc091ae\": rpc error: code = NotFound desc = could not find container \"3f16de4b1d15886789bdc38843cbe1fb5e5180679e89cb7fb19837deadc091ae\": container with ID starting with 3f16de4b1d15886789bdc38843cbe1fb5e5180679e89cb7fb19837deadc091ae not found: ID does not exist" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.974898 5052 scope.go:117] "RemoveContainer" containerID="1658de9b62f64e764364ab2fb2ba523e5ca2b9210d45094169d4d8ec956ac253" Dec 03 12:01:19 crc kubenswrapper[5052]: E1203 12:01:19.975145 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1658de9b62f64e764364ab2fb2ba523e5ca2b9210d45094169d4d8ec956ac253\": container with ID starting with 1658de9b62f64e764364ab2fb2ba523e5ca2b9210d45094169d4d8ec956ac253 not found: ID does not exist" containerID="1658de9b62f64e764364ab2fb2ba523e5ca2b9210d45094169d4d8ec956ac253" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.975161 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1658de9b62f64e764364ab2fb2ba523e5ca2b9210d45094169d4d8ec956ac253"} err="failed to get container status \"1658de9b62f64e764364ab2fb2ba523e5ca2b9210d45094169d4d8ec956ac253\": rpc error: code = NotFound desc = could not find container \"1658de9b62f64e764364ab2fb2ba523e5ca2b9210d45094169d4d8ec956ac253\": container with ID starting with 1658de9b62f64e764364ab2fb2ba523e5ca2b9210d45094169d4d8ec956ac253 not found: ID does not exist" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.975195 5052 scope.go:117] "RemoveContainer" containerID="ac84406b1507746af79e8f6d313cc521e0407948593d4c617a53957cf7d38341" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.978324 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qfflm\" (UniqueName: \"kubernetes.io/projected/0432c361-154a-46a9-b675-04267061f13b-kube-api-access-qfflm\") pod \"0432c361-154a-46a9-b675-04267061f13b\" (UID: \"0432c361-154a-46a9-b675-04267061f13b\") " Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.978411 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/0432c361-154a-46a9-b675-04267061f13b-public-tls-certs\") pod \"0432c361-154a-46a9-b675-04267061f13b\" (UID: \"0432c361-154a-46a9-b675-04267061f13b\") " Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.978450 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0432c361-154a-46a9-b675-04267061f13b-combined-ca-bundle\") pod \"0432c361-154a-46a9-b675-04267061f13b\" (UID: \"0432c361-154a-46a9-b675-04267061f13b\") " Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.978473 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/0432c361-154a-46a9-b675-04267061f13b-credential-keys\") pod \"0432c361-154a-46a9-b675-04267061f13b\" (UID: \"0432c361-154a-46a9-b675-04267061f13b\") " Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.978526 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/0432c361-154a-46a9-b675-04267061f13b-scripts\") pod \"0432c361-154a-46a9-b675-04267061f13b\" (UID: \"0432c361-154a-46a9-b675-04267061f13b\") " Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.978558 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/0432c361-154a-46a9-b675-04267061f13b-config-data\") pod \"0432c361-154a-46a9-b675-04267061f13b\" (UID: \"0432c361-154a-46a9-b675-04267061f13b\") " Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.978658 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/0432c361-154a-46a9-b675-04267061f13b-fernet-keys\") pod \"0432c361-154a-46a9-b675-04267061f13b\" (UID: \"0432c361-154a-46a9-b675-04267061f13b\") " Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.978699 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/0432c361-154a-46a9-b675-04267061f13b-internal-tls-certs\") pod \"0432c361-154a-46a9-b675-04267061f13b\" (UID: \"0432c361-154a-46a9-b675-04267061f13b\") " Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.980861 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/e09397b4-fabb-4e9d-8695-669bf9f49c77-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.984575 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0432c361-154a-46a9-b675-04267061f13b-credential-keys" (OuterVolumeSpecName: "credential-keys") pod "0432c361-154a-46a9-b675-04267061f13b" (UID: "0432c361-154a-46a9-b675-04267061f13b"). InnerVolumeSpecName "credential-keys". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.984646 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/memcached-0"] Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.990684 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0432c361-154a-46a9-b675-04267061f13b-fernet-keys" (OuterVolumeSpecName: "fernet-keys") pod "0432c361-154a-46a9-b675-04267061f13b" (UID: "0432c361-154a-46a9-b675-04267061f13b"). InnerVolumeSpecName "fernet-keys". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.990690 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0432c361-154a-46a9-b675-04267061f13b-kube-api-access-qfflm" (OuterVolumeSpecName: "kube-api-access-qfflm") pod "0432c361-154a-46a9-b675-04267061f13b" (UID: "0432c361-154a-46a9-b675-04267061f13b"). InnerVolumeSpecName "kube-api-access-qfflm". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.993544 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0432c361-154a-46a9-b675-04267061f13b-scripts" (OuterVolumeSpecName: "scripts") pod "0432c361-154a-46a9-b675-04267061f13b" (UID: "0432c361-154a-46a9-b675-04267061f13b"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:01:19 crc kubenswrapper[5052]: I1203 12:01:19.993889 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/placement-7d9769b856-tzkn2"] Dec 03 12:01:20 crc kubenswrapper[5052]: I1203 12:01:20.001325 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/placement-7d9769b856-tzkn2"] Dec 03 12:01:20 crc kubenswrapper[5052]: I1203 12:01:20.009705 5052 scope.go:117] "RemoveContainer" containerID="77bff56b6eb8629741796ab9f7892ba83978c309def196be2b0d6b39a27f09ec" Dec 03 12:01:20 crc kubenswrapper[5052]: I1203 12:01:20.011361 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:01:20 crc kubenswrapper[5052]: I1203 12:01:20.012223 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0432c361-154a-46a9-b675-04267061f13b-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "0432c361-154a-46a9-b675-04267061f13b" (UID: "0432c361-154a-46a9-b675-04267061f13b"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:01:20 crc kubenswrapper[5052]: I1203 12:01:20.025692 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:01:20 crc kubenswrapper[5052]: I1203 12:01:20.031222 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0432c361-154a-46a9-b675-04267061f13b-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "0432c361-154a-46a9-b675-04267061f13b" (UID: "0432c361-154a-46a9-b675-04267061f13b"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:01:20 crc kubenswrapper[5052]: I1203 12:01:20.033996 5052 scope.go:117] "RemoveContainer" containerID="1c18a9d55257352dbf809914f4dca8da241a548437c4cbcf603a968646bbaaf6" Dec 03 12:01:20 crc kubenswrapper[5052]: I1203 12:01:20.048645 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0432c361-154a-46a9-b675-04267061f13b-config-data" (OuterVolumeSpecName: "config-data") pod "0432c361-154a-46a9-b675-04267061f13b" (UID: "0432c361-154a-46a9-b675-04267061f13b"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:01:20 crc kubenswrapper[5052]: I1203 12:01:20.048760 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 12:01:20 crc kubenswrapper[5052]: I1203 12:01:20.058435 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 12:01:20 crc kubenswrapper[5052]: I1203 12:01:20.061986 5052 scope.go:117] "RemoveContainer" containerID="710c4f7c04a361b712543bf316631f84336d70d531c95a80fd6746ea4334640d" Dec 03 12:01:20 crc kubenswrapper[5052]: I1203 12:01:20.065529 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:01:20 crc kubenswrapper[5052]: I1203 12:01:20.075490 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:01:20 crc kubenswrapper[5052]: I1203 12:01:20.077482 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0432c361-154a-46a9-b675-04267061f13b-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "0432c361-154a-46a9-b675-04267061f13b" (UID: "0432c361-154a-46a9-b675-04267061f13b"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:01:20 crc kubenswrapper[5052]: I1203 12:01:20.080989 5052 scope.go:117] "RemoveContainer" containerID="a7bae4cc8cec979afcf2817d0be9cd0e1558b2c480f44fd3bf85f63c239d1a5c" Dec 03 12:01:20 crc kubenswrapper[5052]: I1203 12:01:20.083008 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/75c87049-f7b9-4c6b-add4-2eec1f52deb2-config-data-default\") pod \"75c87049-f7b9-4c6b-add4-2eec1f52deb2\" (UID: \"75c87049-f7b9-4c6b-add4-2eec1f52deb2\") " Dec 03 12:01:20 crc kubenswrapper[5052]: I1203 12:01:20.083094 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/75c87049-f7b9-4c6b-add4-2eec1f52deb2-operator-scripts\") pod \"75c87049-f7b9-4c6b-add4-2eec1f52deb2\" (UID: \"75c87049-f7b9-4c6b-add4-2eec1f52deb2\") " Dec 03 12:01:20 crc kubenswrapper[5052]: I1203 12:01:20.083145 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/75c87049-f7b9-4c6b-add4-2eec1f52deb2-galera-tls-certs\") pod \"75c87049-f7b9-4c6b-add4-2eec1f52deb2\" (UID: \"75c87049-f7b9-4c6b-add4-2eec1f52deb2\") " Dec 03 12:01:20 crc kubenswrapper[5052]: I1203 12:01:20.083285 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/75c87049-f7b9-4c6b-add4-2eec1f52deb2-combined-ca-bundle\") pod \"75c87049-f7b9-4c6b-add4-2eec1f52deb2\" (UID: \"75c87049-f7b9-4c6b-add4-2eec1f52deb2\") " Dec 03 12:01:20 crc kubenswrapper[5052]: I1203 12:01:20.083665 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/75c87049-f7b9-4c6b-add4-2eec1f52deb2-config-data-generated\") pod \"75c87049-f7b9-4c6b-add4-2eec1f52deb2\" (UID: \"75c87049-f7b9-4c6b-add4-2eec1f52deb2\") " Dec 03 12:01:20 crc kubenswrapper[5052]: I1203 12:01:20.083693 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/75c87049-f7b9-4c6b-add4-2eec1f52deb2-kolla-config\") pod \"75c87049-f7b9-4c6b-add4-2eec1f52deb2\" (UID: \"75c87049-f7b9-4c6b-add4-2eec1f52deb2\") " Dec 03 12:01:20 crc kubenswrapper[5052]: I1203 12:01:20.083750 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n5brn\" (UniqueName: \"kubernetes.io/projected/75c87049-f7b9-4c6b-add4-2eec1f52deb2-kube-api-access-n5brn\") pod \"75c87049-f7b9-4c6b-add4-2eec1f52deb2\" (UID: \"75c87049-f7b9-4c6b-add4-2eec1f52deb2\") " Dec 03 12:01:20 crc kubenswrapper[5052]: I1203 12:01:20.083810 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"mysql-db\" (UniqueName: \"kubernetes.io/local-volume/local-storage14-crc\") pod \"75c87049-f7b9-4c6b-add4-2eec1f52deb2\" (UID: \"75c87049-f7b9-4c6b-add4-2eec1f52deb2\") " Dec 03 12:01:20 crc kubenswrapper[5052]: I1203 12:01:20.084235 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/0432c361-154a-46a9-b675-04267061f13b-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:20 crc kubenswrapper[5052]: I1203 12:01:20.084255 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0432c361-154a-46a9-b675-04267061f13b-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:20 crc kubenswrapper[5052]: I1203 12:01:20.084265 5052 reconciler_common.go:293] "Volume detached for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/0432c361-154a-46a9-b675-04267061f13b-credential-keys\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:20 crc kubenswrapper[5052]: I1203 12:01:20.084274 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/0432c361-154a-46a9-b675-04267061f13b-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:20 crc kubenswrapper[5052]: I1203 12:01:20.084309 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/0432c361-154a-46a9-b675-04267061f13b-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:20 crc kubenswrapper[5052]: I1203 12:01:20.084318 5052 reconciler_common.go:293] "Volume detached for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/0432c361-154a-46a9-b675-04267061f13b-fernet-keys\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:20 crc kubenswrapper[5052]: I1203 12:01:20.084325 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/0432c361-154a-46a9-b675-04267061f13b-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:20 crc kubenswrapper[5052]: I1203 12:01:20.084333 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-qfflm\" (UniqueName: \"kubernetes.io/projected/0432c361-154a-46a9-b675-04267061f13b-kube-api-access-qfflm\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:20 crc kubenswrapper[5052]: I1203 12:01:20.084361 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/75c87049-f7b9-4c6b-add4-2eec1f52deb2-config-data-default" (OuterVolumeSpecName: "config-data-default") pod "75c87049-f7b9-4c6b-add4-2eec1f52deb2" (UID: "75c87049-f7b9-4c6b-add4-2eec1f52deb2"). InnerVolumeSpecName "config-data-default". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:01:20 crc kubenswrapper[5052]: I1203 12:01:20.084393 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/75c87049-f7b9-4c6b-add4-2eec1f52deb2-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "75c87049-f7b9-4c6b-add4-2eec1f52deb2" (UID: "75c87049-f7b9-4c6b-add4-2eec1f52deb2"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:01:20 crc kubenswrapper[5052]: I1203 12:01:20.084240 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/75c87049-f7b9-4c6b-add4-2eec1f52deb2-config-data-generated" (OuterVolumeSpecName: "config-data-generated") pod "75c87049-f7b9-4c6b-add4-2eec1f52deb2" (UID: "75c87049-f7b9-4c6b-add4-2eec1f52deb2"). InnerVolumeSpecName "config-data-generated". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:01:20 crc kubenswrapper[5052]: I1203 12:01:20.085336 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/75c87049-f7b9-4c6b-add4-2eec1f52deb2-kolla-config" (OuterVolumeSpecName: "kolla-config") pod "75c87049-f7b9-4c6b-add4-2eec1f52deb2" (UID: "75c87049-f7b9-4c6b-add4-2eec1f52deb2"). InnerVolumeSpecName "kolla-config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:01:20 crc kubenswrapper[5052]: I1203 12:01:20.088083 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/75c87049-f7b9-4c6b-add4-2eec1f52deb2-kube-api-access-n5brn" (OuterVolumeSpecName: "kube-api-access-n5brn") pod "75c87049-f7b9-4c6b-add4-2eec1f52deb2" (UID: "75c87049-f7b9-4c6b-add4-2eec1f52deb2"). InnerVolumeSpecName "kube-api-access-n5brn". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:01:20 crc kubenswrapper[5052]: I1203 12:01:20.095002 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage14-crc" (OuterVolumeSpecName: "mysql-db") pod "75c87049-f7b9-4c6b-add4-2eec1f52deb2" (UID: "75c87049-f7b9-4c6b-add4-2eec1f52deb2"). InnerVolumeSpecName "local-storage14-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 12:01:20 crc kubenswrapper[5052]: I1203 12:01:20.096844 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-worker-ff9447cb9-nmz9j"] Dec 03 12:01:20 crc kubenswrapper[5052]: I1203 12:01:20.104805 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/75c87049-f7b9-4c6b-add4-2eec1f52deb2-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "75c87049-f7b9-4c6b-add4-2eec1f52deb2" (UID: "75c87049-f7b9-4c6b-add4-2eec1f52deb2"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:01:20 crc kubenswrapper[5052]: I1203 12:01:20.105025 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/barbican-worker-ff9447cb9-nmz9j"] Dec 03 12:01:20 crc kubenswrapper[5052]: I1203 12:01:20.122462 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/75c87049-f7b9-4c6b-add4-2eec1f52deb2-galera-tls-certs" (OuterVolumeSpecName: "galera-tls-certs") pod "75c87049-f7b9-4c6b-add4-2eec1f52deb2" (UID: "75c87049-f7b9-4c6b-add4-2eec1f52deb2"). InnerVolumeSpecName "galera-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:01:20 crc kubenswrapper[5052]: I1203 12:01:20.185890 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-n5brn\" (UniqueName: \"kubernetes.io/projected/75c87049-f7b9-4c6b-add4-2eec1f52deb2-kube-api-access-n5brn\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:20 crc kubenswrapper[5052]: I1203 12:01:20.185933 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage14-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage14-crc\") on node \"crc\" " Dec 03 12:01:20 crc kubenswrapper[5052]: I1203 12:01:20.185944 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/75c87049-f7b9-4c6b-add4-2eec1f52deb2-config-data-default\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:20 crc kubenswrapper[5052]: I1203 12:01:20.185953 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/75c87049-f7b9-4c6b-add4-2eec1f52deb2-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:20 crc kubenswrapper[5052]: I1203 12:01:20.185962 5052 reconciler_common.go:293] "Volume detached for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/75c87049-f7b9-4c6b-add4-2eec1f52deb2-galera-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:20 crc kubenswrapper[5052]: I1203 12:01:20.185970 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/75c87049-f7b9-4c6b-add4-2eec1f52deb2-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:20 crc kubenswrapper[5052]: I1203 12:01:20.185978 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/75c87049-f7b9-4c6b-add4-2eec1f52deb2-config-data-generated\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:20 crc kubenswrapper[5052]: I1203 12:01:20.185987 5052 reconciler_common.go:293] "Volume detached for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/75c87049-f7b9-4c6b-add4-2eec1f52deb2-kolla-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:20 crc kubenswrapper[5052]: E1203 12:01:20.185967 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:01:20 crc kubenswrapper[5052]: E1203 12:01:20.186052 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/cf90ea08-b172-4549-9522-92be7a5b5623-operator-scripts podName:cf90ea08-b172-4549-9522-92be7a5b5623 nodeName:}" failed. No retries permitted until 2025-12-03 12:01:22.186036491 +0000 UTC m=+2472.774278255 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/cf90ea08-b172-4549-9522-92be7a5b5623-operator-scripts") pod "novacell184d8-account-delete-4nmn9" (UID: "cf90ea08-b172-4549-9522-92be7a5b5623") : configmap "openstack-scripts" not found Dec 03 12:01:20 crc kubenswrapper[5052]: I1203 12:01:20.209239 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage14-crc" (UniqueName: "kubernetes.io/local-volume/local-storage14-crc") on node "crc" Dec 03 12:01:20 crc kubenswrapper[5052]: I1203 12:01:20.231256 5052 scope.go:117] "RemoveContainer" containerID="9828c63a7f8943e7325b0971d75b7f4ef89e7d95366cacf404cd7f47ac39d61f" Dec 03 12:01:20 crc kubenswrapper[5052]: I1203 12:01:20.263180 5052 scope.go:117] "RemoveContainer" containerID="f80007b6133c4b4853d49d9ef314cd82a9852973ff7a6cf37381a6ca1b1ec967" Dec 03 12:01:20 crc kubenswrapper[5052]: I1203 12:01:20.287179 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage14-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage14-crc\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:20 crc kubenswrapper[5052]: I1203 12:01:20.287311 5052 scope.go:117] "RemoveContainer" containerID="32933642a3f541b1b3569edbbb3a29c32da560890311d2484c68ee57f99463a4" Dec 03 12:01:20 crc kubenswrapper[5052]: I1203 12:01:20.310396 5052 scope.go:117] "RemoveContainer" containerID="aebafbbccc45118d70ddef8e3f76333d5b32b3641a77534948d68d7de2fc9fd0" Dec 03 12:01:20 crc kubenswrapper[5052]: I1203 12:01:20.340202 5052 scope.go:117] "RemoveContainer" containerID="be465bb3373c78a7e0d0341f23e780fbf057154f4f4e5cd558d38b99cc20e088" Dec 03 12:01:20 crc kubenswrapper[5052]: I1203 12:01:20.360445 5052 scope.go:117] "RemoveContainer" containerID="7fa07f1e6f3c12445900ff78caa499630cedc8446b5a3e6c5c977fa8f1760705" Dec 03 12:01:20 crc kubenswrapper[5052]: I1203 12:01:20.385147 5052 scope.go:117] "RemoveContainer" containerID="677fccb8442f04cfdcccbb93e98725dc995271445330b661ed4f10796a9c7c91" Dec 03 12:01:20 crc kubenswrapper[5052]: I1203 12:01:20.404018 5052 scope.go:117] "RemoveContainer" containerID="57e4d8c014f57177dc5f916168de2b019cf9fcd67c0d39bcaf0f7357d2dbf4bc" Dec 03 12:01:20 crc kubenswrapper[5052]: I1203 12:01:20.424530 5052 scope.go:117] "RemoveContainer" containerID="5f92af45a915348c684ef49149ebdedefc7b5163a55e465615daf0ad05fa17b8" Dec 03 12:01:20 crc kubenswrapper[5052]: I1203 12:01:20.442349 5052 scope.go:117] "RemoveContainer" containerID="3a24f581129d9e74e6cdc6ff9baf1728f121946baea46179b88abf5f28003c5f" Dec 03 12:01:20 crc kubenswrapper[5052]: I1203 12:01:20.810708 5052 generic.go:334] "Generic (PLEG): container finished" podID="75c87049-f7b9-4c6b-add4-2eec1f52deb2" containerID="f270b916e5373a4dc04617edb6ac8552ab0bed39941d9a4293aaa5264e513b53" exitCode=0 Dec 03 12:01:20 crc kubenswrapper[5052]: I1203 12:01:20.810779 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-galera-0" event={"ID":"75c87049-f7b9-4c6b-add4-2eec1f52deb2","Type":"ContainerDied","Data":"f270b916e5373a4dc04617edb6ac8552ab0bed39941d9a4293aaa5264e513b53"} Dec 03 12:01:20 crc kubenswrapper[5052]: I1203 12:01:20.810810 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-galera-0" event={"ID":"75c87049-f7b9-4c6b-add4-2eec1f52deb2","Type":"ContainerDied","Data":"96e331e95f8752e753d096f5fc8e1b953bcea07c1a9b3abb3dfeb833853b6cf3"} Dec 03 12:01:20 crc kubenswrapper[5052]: I1203 12:01:20.810836 5052 scope.go:117] "RemoveContainer" containerID="f270b916e5373a4dc04617edb6ac8552ab0bed39941d9a4293aaa5264e513b53" Dec 03 12:01:20 crc kubenswrapper[5052]: I1203 12:01:20.811025 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:01:20 crc kubenswrapper[5052]: I1203 12:01:20.830029 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-bd5c68479-wvv6d" event={"ID":"0432c361-154a-46a9-b675-04267061f13b","Type":"ContainerDied","Data":"92690517c27d1488acd9a690534bf92b583ecc54d1e454272977568c2c3b275c"} Dec 03 12:01:20 crc kubenswrapper[5052]: I1203 12:01:20.830160 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-bd5c68479-wvv6d" Dec 03 12:01:20 crc kubenswrapper[5052]: I1203 12:01:20.832293 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-keystone-listener-6774dbcdb8-wzwf7" Dec 03 12:01:20 crc kubenswrapper[5052]: I1203 12:01:20.857037 5052 generic.go:334] "Generic (PLEG): container finished" podID="30578d71-fe2b-4eed-9a07-6c8bfe4a1c43" containerID="d9ddf262949f1f7939bc5d4f4522859474860ed8b5edac12e5534c72d138210a" exitCode=0 Dec 03 12:01:20 crc kubenswrapper[5052]: I1203 12:01:20.857148 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:01:20 crc kubenswrapper[5052]: I1203 12:01:20.857385 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-server-0" event={"ID":"30578d71-fe2b-4eed-9a07-6c8bfe4a1c43","Type":"ContainerDied","Data":"d9ddf262949f1f7939bc5d4f4522859474860ed8b5edac12e5534c72d138210a"} Dec 03 12:01:20 crc kubenswrapper[5052]: I1203 12:01:20.869106 5052 scope.go:117] "RemoveContainer" containerID="747f297461b54875e189e5f421860b725e3b1e8ee94217f21a72b29f666ac5e9" Dec 03 12:01:20 crc kubenswrapper[5052]: I1203 12:01:20.871450 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/openstack-galera-0"] Dec 03 12:01:20 crc kubenswrapper[5052]: I1203 12:01:20.885069 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/openstack-galera-0"] Dec 03 12:01:20 crc kubenswrapper[5052]: I1203 12:01:20.895926 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/keystone-bd5c68479-wvv6d"] Dec 03 12:01:20 crc kubenswrapper[5052]: E1203 12:01:20.897438 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:01:20 crc kubenswrapper[5052]: E1203 12:01:20.897502 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d420a2c2-9b7b-400b-88a7-839a93cce573-operator-scripts podName:d420a2c2-9b7b-400b-88a7-839a93cce573 nodeName:}" failed. No retries permitted until 2025-12-03 12:01:24.897484401 +0000 UTC m=+2475.485726165 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/d420a2c2-9b7b-400b-88a7-839a93cce573-operator-scripts") pod "barbican4aeb-account-delete-bvhdj" (UID: "d420a2c2-9b7b-400b-88a7-839a93cce573") : configmap "openstack-scripts" not found Dec 03 12:01:20 crc kubenswrapper[5052]: E1203 12:01:20.898075 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:01:20 crc kubenswrapper[5052]: E1203 12:01:20.898116 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6343f4a5-892f-4118-a666-e6601d2df818-operator-scripts podName:6343f4a5-892f-4118-a666-e6601d2df818 nodeName:}" failed. No retries permitted until 2025-12-03 12:01:24.898101008 +0000 UTC m=+2475.486342772 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/6343f4a5-892f-4118-a666-e6601d2df818-operator-scripts") pod "novacell06be2-account-delete-7m8cr" (UID: "6343f4a5-892f-4118-a666-e6601d2df818") : configmap "openstack-scripts" not found Dec 03 12:01:20 crc kubenswrapper[5052]: I1203 12:01:20.911712 5052 scope.go:117] "RemoveContainer" containerID="f270b916e5373a4dc04617edb6ac8552ab0bed39941d9a4293aaa5264e513b53" Dec 03 12:01:20 crc kubenswrapper[5052]: I1203 12:01:20.915182 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/keystone-bd5c68479-wvv6d"] Dec 03 12:01:20 crc kubenswrapper[5052]: E1203 12:01:20.916791 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f270b916e5373a4dc04617edb6ac8552ab0bed39941d9a4293aaa5264e513b53\": container with ID starting with f270b916e5373a4dc04617edb6ac8552ab0bed39941d9a4293aaa5264e513b53 not found: ID does not exist" containerID="f270b916e5373a4dc04617edb6ac8552ab0bed39941d9a4293aaa5264e513b53" Dec 03 12:01:20 crc kubenswrapper[5052]: I1203 12:01:20.916832 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f270b916e5373a4dc04617edb6ac8552ab0bed39941d9a4293aaa5264e513b53"} err="failed to get container status \"f270b916e5373a4dc04617edb6ac8552ab0bed39941d9a4293aaa5264e513b53\": rpc error: code = NotFound desc = could not find container \"f270b916e5373a4dc04617edb6ac8552ab0bed39941d9a4293aaa5264e513b53\": container with ID starting with f270b916e5373a4dc04617edb6ac8552ab0bed39941d9a4293aaa5264e513b53 not found: ID does not exist" Dec 03 12:01:20 crc kubenswrapper[5052]: I1203 12:01:20.916855 5052 scope.go:117] "RemoveContainer" containerID="747f297461b54875e189e5f421860b725e3b1e8ee94217f21a72b29f666ac5e9" Dec 03 12:01:20 crc kubenswrapper[5052]: E1203 12:01:20.917911 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"747f297461b54875e189e5f421860b725e3b1e8ee94217f21a72b29f666ac5e9\": container with ID starting with 747f297461b54875e189e5f421860b725e3b1e8ee94217f21a72b29f666ac5e9 not found: ID does not exist" containerID="747f297461b54875e189e5f421860b725e3b1e8ee94217f21a72b29f666ac5e9" Dec 03 12:01:20 crc kubenswrapper[5052]: I1203 12:01:20.917993 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"747f297461b54875e189e5f421860b725e3b1e8ee94217f21a72b29f666ac5e9"} err="failed to get container status \"747f297461b54875e189e5f421860b725e3b1e8ee94217f21a72b29f666ac5e9\": rpc error: code = NotFound desc = could not find container \"747f297461b54875e189e5f421860b725e3b1e8ee94217f21a72b29f666ac5e9\": container with ID starting with 747f297461b54875e189e5f421860b725e3b1e8ee94217f21a72b29f666ac5e9 not found: ID does not exist" Dec 03 12:01:20 crc kubenswrapper[5052]: I1203 12:01:20.918057 5052 scope.go:117] "RemoveContainer" containerID="13a3f64e26fc4afd5ac59ca4b43b427f4f0bb964e269ef968f7456ea48b99a59" Dec 03 12:01:20 crc kubenswrapper[5052]: I1203 12:01:20.924530 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-keystone-listener-6774dbcdb8-wzwf7"] Dec 03 12:01:20 crc kubenswrapper[5052]: I1203 12:01:20.933649 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/barbican-keystone-listener-6774dbcdb8-wzwf7"] Dec 03 12:01:20 crc kubenswrapper[5052]: I1203 12:01:20.938987 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-0"] Dec 03 12:01:20 crc kubenswrapper[5052]: I1203 12:01:20.945943 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-0"] Dec 03 12:01:20 crc kubenswrapper[5052]: E1203 12:01:20.999176 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:01:20 crc kubenswrapper[5052]: E1203 12:01:20.999244 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1a97695f-322c-4c4c-834e-ecb2f27c2e97-operator-scripts podName:1a97695f-322c-4c4c-834e-ecb2f27c2e97 nodeName:}" failed. No retries permitted until 2025-12-03 12:01:24.99922932 +0000 UTC m=+2475.587471084 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/1a97695f-322c-4c4c-834e-ecb2f27c2e97-operator-scripts") pod "glance0d27-account-delete-brqv4" (UID: "1a97695f-322c-4c4c-834e-ecb2f27c2e97") : configmap "openstack-scripts" not found Dec 03 12:01:20 crc kubenswrapper[5052]: E1203 12:01:20.999622 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:01:20 crc kubenswrapper[5052]: E1203 12:01:20.999658 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/36225798-caeb-4514-9f51-c8a07fb6da9b-operator-scripts podName:36225798-caeb-4514-9f51-c8a07fb6da9b nodeName:}" failed. No retries permitted until 2025-12-03 12:01:24.999648882 +0000 UTC m=+2475.587890646 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/36225798-caeb-4514-9f51-c8a07fb6da9b-operator-scripts") pod "placemente203-account-delete-sdpql" (UID: "36225798-caeb-4514-9f51-c8a07fb6da9b") : configmap "openstack-scripts" not found Dec 03 12:01:20 crc kubenswrapper[5052]: E1203 12:01:20.999704 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:01:20 crc kubenswrapper[5052]: E1203 12:01:20.999722 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/eaeff688-fe65-403f-a01a-dfff69db9fb9-operator-scripts podName:eaeff688-fe65-403f-a01a-dfff69db9fb9 nodeName:}" failed. No retries permitted until 2025-12-03 12:01:24.999717234 +0000 UTC m=+2475.587958998 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/eaeff688-fe65-403f-a01a-dfff69db9fb9-operator-scripts") pod "novaapif429-account-delete-qwz7z" (UID: "eaeff688-fe65-403f-a01a-dfff69db9fb9") : configmap "openstack-scripts" not found Dec 03 12:01:21 crc kubenswrapper[5052]: I1203 12:01:21.058803 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:01:21 crc kubenswrapper[5052]: I1203 12:01:21.100235 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"plugins-conf\" (UniqueName: \"kubernetes.io/configmap/30578d71-fe2b-4eed-9a07-6c8bfe4a1c43-plugins-conf\") pod \"30578d71-fe2b-4eed-9a07-6c8bfe4a1c43\" (UID: \"30578d71-fe2b-4eed-9a07-6c8bfe4a1c43\") " Dec 03 12:01:21 crc kubenswrapper[5052]: I1203 12:01:21.100287 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"persistence\" (UniqueName: \"kubernetes.io/local-volume/local-storage18-crc\") pod \"30578d71-fe2b-4eed-9a07-6c8bfe4a1c43\" (UID: \"30578d71-fe2b-4eed-9a07-6c8bfe4a1c43\") " Dec 03 12:01:21 crc kubenswrapper[5052]: I1203 12:01:21.100319 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"pod-info\" (UniqueName: \"kubernetes.io/downward-api/30578d71-fe2b-4eed-9a07-6c8bfe4a1c43-pod-info\") pod \"30578d71-fe2b-4eed-9a07-6c8bfe4a1c43\" (UID: \"30578d71-fe2b-4eed-9a07-6c8bfe4a1c43\") " Dec 03 12:01:21 crc kubenswrapper[5052]: I1203 12:01:21.100349 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"rabbitmq-plugins\" (UniqueName: \"kubernetes.io/empty-dir/30578d71-fe2b-4eed-9a07-6c8bfe4a1c43-rabbitmq-plugins\") pod \"30578d71-fe2b-4eed-9a07-6c8bfe4a1c43\" (UID: \"30578d71-fe2b-4eed-9a07-6c8bfe4a1c43\") " Dec 03 12:01:21 crc kubenswrapper[5052]: I1203 12:01:21.100387 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"server-conf\" (UniqueName: \"kubernetes.io/configmap/30578d71-fe2b-4eed-9a07-6c8bfe4a1c43-server-conf\") pod \"30578d71-fe2b-4eed-9a07-6c8bfe4a1c43\" (UID: \"30578d71-fe2b-4eed-9a07-6c8bfe4a1c43\") " Dec 03 12:01:21 crc kubenswrapper[5052]: I1203 12:01:21.100466 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"rabbitmq-erlang-cookie\" (UniqueName: \"kubernetes.io/empty-dir/30578d71-fe2b-4eed-9a07-6c8bfe4a1c43-rabbitmq-erlang-cookie\") pod \"30578d71-fe2b-4eed-9a07-6c8bfe4a1c43\" (UID: \"30578d71-fe2b-4eed-9a07-6c8bfe4a1c43\") " Dec 03 12:01:21 crc kubenswrapper[5052]: I1203 12:01:21.100547 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"rabbitmq-confd\" (UniqueName: \"kubernetes.io/projected/30578d71-fe2b-4eed-9a07-6c8bfe4a1c43-rabbitmq-confd\") pod \"30578d71-fe2b-4eed-9a07-6c8bfe4a1c43\" (UID: \"30578d71-fe2b-4eed-9a07-6c8bfe4a1c43\") " Dec 03 12:01:21 crc kubenswrapper[5052]: I1203 12:01:21.100587 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/30578d71-fe2b-4eed-9a07-6c8bfe4a1c43-config-data\") pod \"30578d71-fe2b-4eed-9a07-6c8bfe4a1c43\" (UID: \"30578d71-fe2b-4eed-9a07-6c8bfe4a1c43\") " Dec 03 12:01:21 crc kubenswrapper[5052]: I1203 12:01:21.100640 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"rabbitmq-tls\" (UniqueName: \"kubernetes.io/projected/30578d71-fe2b-4eed-9a07-6c8bfe4a1c43-rabbitmq-tls\") pod \"30578d71-fe2b-4eed-9a07-6c8bfe4a1c43\" (UID: \"30578d71-fe2b-4eed-9a07-6c8bfe4a1c43\") " Dec 03 12:01:21 crc kubenswrapper[5052]: I1203 12:01:21.100681 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"erlang-cookie-secret\" (UniqueName: \"kubernetes.io/secret/30578d71-fe2b-4eed-9a07-6c8bfe4a1c43-erlang-cookie-secret\") pod \"30578d71-fe2b-4eed-9a07-6c8bfe4a1c43\" (UID: \"30578d71-fe2b-4eed-9a07-6c8bfe4a1c43\") " Dec 03 12:01:21 crc kubenswrapper[5052]: I1203 12:01:21.100714 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nt9pl\" (UniqueName: \"kubernetes.io/projected/30578d71-fe2b-4eed-9a07-6c8bfe4a1c43-kube-api-access-nt9pl\") pod \"30578d71-fe2b-4eed-9a07-6c8bfe4a1c43\" (UID: \"30578d71-fe2b-4eed-9a07-6c8bfe4a1c43\") " Dec 03 12:01:21 crc kubenswrapper[5052]: E1203 12:01:21.101163 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:01:21 crc kubenswrapper[5052]: E1203 12:01:21.101231 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c17bae4f-1c1b-4da2-9bb5-c84d6fdfa184-operator-scripts podName:c17bae4f-1c1b-4da2-9bb5-c84d6fdfa184 nodeName:}" failed. No retries permitted until 2025-12-03 12:01:25.101214675 +0000 UTC m=+2475.689456449 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/c17bae4f-1c1b-4da2-9bb5-c84d6fdfa184-operator-scripts") pod "neutron0ea0-account-delete-lvffs" (UID: "c17bae4f-1c1b-4da2-9bb5-c84d6fdfa184") : configmap "openstack-scripts" not found Dec 03 12:01:21 crc kubenswrapper[5052]: I1203 12:01:21.101330 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/30578d71-fe2b-4eed-9a07-6c8bfe4a1c43-plugins-conf" (OuterVolumeSpecName: "plugins-conf") pod "30578d71-fe2b-4eed-9a07-6c8bfe4a1c43" (UID: "30578d71-fe2b-4eed-9a07-6c8bfe4a1c43"). InnerVolumeSpecName "plugins-conf". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:01:21 crc kubenswrapper[5052]: I1203 12:01:21.101386 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/30578d71-fe2b-4eed-9a07-6c8bfe4a1c43-rabbitmq-erlang-cookie" (OuterVolumeSpecName: "rabbitmq-erlang-cookie") pod "30578d71-fe2b-4eed-9a07-6c8bfe4a1c43" (UID: "30578d71-fe2b-4eed-9a07-6c8bfe4a1c43"). InnerVolumeSpecName "rabbitmq-erlang-cookie". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:01:21 crc kubenswrapper[5052]: I1203 12:01:21.101821 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/30578d71-fe2b-4eed-9a07-6c8bfe4a1c43-rabbitmq-plugins" (OuterVolumeSpecName: "rabbitmq-plugins") pod "30578d71-fe2b-4eed-9a07-6c8bfe4a1c43" (UID: "30578d71-fe2b-4eed-9a07-6c8bfe4a1c43"). InnerVolumeSpecName "rabbitmq-plugins". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:01:21 crc kubenswrapper[5052]: I1203 12:01:21.103262 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage18-crc" (OuterVolumeSpecName: "persistence") pod "30578d71-fe2b-4eed-9a07-6c8bfe4a1c43" (UID: "30578d71-fe2b-4eed-9a07-6c8bfe4a1c43"). InnerVolumeSpecName "local-storage18-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 12:01:21 crc kubenswrapper[5052]: I1203 12:01:21.106039 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/30578d71-fe2b-4eed-9a07-6c8bfe4a1c43-rabbitmq-tls" (OuterVolumeSpecName: "rabbitmq-tls") pod "30578d71-fe2b-4eed-9a07-6c8bfe4a1c43" (UID: "30578d71-fe2b-4eed-9a07-6c8bfe4a1c43"). InnerVolumeSpecName "rabbitmq-tls". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:01:21 crc kubenswrapper[5052]: I1203 12:01:21.106145 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/30578d71-fe2b-4eed-9a07-6c8bfe4a1c43-pod-info" (OuterVolumeSpecName: "pod-info") pod "30578d71-fe2b-4eed-9a07-6c8bfe4a1c43" (UID: "30578d71-fe2b-4eed-9a07-6c8bfe4a1c43"). InnerVolumeSpecName "pod-info". PluginName "kubernetes.io/downward-api", VolumeGidValue "" Dec 03 12:01:21 crc kubenswrapper[5052]: I1203 12:01:21.106967 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/30578d71-fe2b-4eed-9a07-6c8bfe4a1c43-erlang-cookie-secret" (OuterVolumeSpecName: "erlang-cookie-secret") pod "30578d71-fe2b-4eed-9a07-6c8bfe4a1c43" (UID: "30578d71-fe2b-4eed-9a07-6c8bfe4a1c43"). InnerVolumeSpecName "erlang-cookie-secret". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:01:21 crc kubenswrapper[5052]: I1203 12:01:21.112358 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/30578d71-fe2b-4eed-9a07-6c8bfe4a1c43-kube-api-access-nt9pl" (OuterVolumeSpecName: "kube-api-access-nt9pl") pod "30578d71-fe2b-4eed-9a07-6c8bfe4a1c43" (UID: "30578d71-fe2b-4eed-9a07-6c8bfe4a1c43"). InnerVolumeSpecName "kube-api-access-nt9pl". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:01:21 crc kubenswrapper[5052]: I1203 12:01:21.128171 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/30578d71-fe2b-4eed-9a07-6c8bfe4a1c43-config-data" (OuterVolumeSpecName: "config-data") pod "30578d71-fe2b-4eed-9a07-6c8bfe4a1c43" (UID: "30578d71-fe2b-4eed-9a07-6c8bfe4a1c43"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:01:21 crc kubenswrapper[5052]: I1203 12:01:21.142895 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/30578d71-fe2b-4eed-9a07-6c8bfe4a1c43-server-conf" (OuterVolumeSpecName: "server-conf") pod "30578d71-fe2b-4eed-9a07-6c8bfe4a1c43" (UID: "30578d71-fe2b-4eed-9a07-6c8bfe4a1c43"). InnerVolumeSpecName "server-conf". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:01:21 crc kubenswrapper[5052]: I1203 12:01:21.197636 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/30578d71-fe2b-4eed-9a07-6c8bfe4a1c43-rabbitmq-confd" (OuterVolumeSpecName: "rabbitmq-confd") pod "30578d71-fe2b-4eed-9a07-6c8bfe4a1c43" (UID: "30578d71-fe2b-4eed-9a07-6c8bfe4a1c43"). InnerVolumeSpecName "rabbitmq-confd". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:01:21 crc kubenswrapper[5052]: I1203 12:01:21.202870 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-nt9pl\" (UniqueName: \"kubernetes.io/projected/30578d71-fe2b-4eed-9a07-6c8bfe4a1c43-kube-api-access-nt9pl\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:21 crc kubenswrapper[5052]: I1203 12:01:21.202900 5052 reconciler_common.go:293] "Volume detached for volume \"plugins-conf\" (UniqueName: \"kubernetes.io/configmap/30578d71-fe2b-4eed-9a07-6c8bfe4a1c43-plugins-conf\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:21 crc kubenswrapper[5052]: I1203 12:01:21.202924 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage18-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage18-crc\") on node \"crc\" " Dec 03 12:01:21 crc kubenswrapper[5052]: I1203 12:01:21.202935 5052 reconciler_common.go:293] "Volume detached for volume \"pod-info\" (UniqueName: \"kubernetes.io/downward-api/30578d71-fe2b-4eed-9a07-6c8bfe4a1c43-pod-info\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:21 crc kubenswrapper[5052]: I1203 12:01:21.202943 5052 reconciler_common.go:293] "Volume detached for volume \"rabbitmq-plugins\" (UniqueName: \"kubernetes.io/empty-dir/30578d71-fe2b-4eed-9a07-6c8bfe4a1c43-rabbitmq-plugins\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:21 crc kubenswrapper[5052]: I1203 12:01:21.202952 5052 reconciler_common.go:293] "Volume detached for volume \"server-conf\" (UniqueName: \"kubernetes.io/configmap/30578d71-fe2b-4eed-9a07-6c8bfe4a1c43-server-conf\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:21 crc kubenswrapper[5052]: I1203 12:01:21.202961 5052 reconciler_common.go:293] "Volume detached for volume \"rabbitmq-erlang-cookie\" (UniqueName: \"kubernetes.io/empty-dir/30578d71-fe2b-4eed-9a07-6c8bfe4a1c43-rabbitmq-erlang-cookie\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:21 crc kubenswrapper[5052]: I1203 12:01:21.202970 5052 reconciler_common.go:293] "Volume detached for volume \"rabbitmq-confd\" (UniqueName: \"kubernetes.io/projected/30578d71-fe2b-4eed-9a07-6c8bfe4a1c43-rabbitmq-confd\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:21 crc kubenswrapper[5052]: I1203 12:01:21.202978 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/30578d71-fe2b-4eed-9a07-6c8bfe4a1c43-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:21 crc kubenswrapper[5052]: I1203 12:01:21.202986 5052 reconciler_common.go:293] "Volume detached for volume \"rabbitmq-tls\" (UniqueName: \"kubernetes.io/projected/30578d71-fe2b-4eed-9a07-6c8bfe4a1c43-rabbitmq-tls\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:21 crc kubenswrapper[5052]: I1203 12:01:21.202993 5052 reconciler_common.go:293] "Volume detached for volume \"erlang-cookie-secret\" (UniqueName: \"kubernetes.io/secret/30578d71-fe2b-4eed-9a07-6c8bfe4a1c43-erlang-cookie-secret\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:21 crc kubenswrapper[5052]: E1203 12:01:21.202923 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:01:21 crc kubenswrapper[5052]: E1203 12:01:21.203076 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b2a6be10-e7df-494c-be94-a8ede55e2868-operator-scripts podName:b2a6be10-e7df-494c-be94-a8ede55e2868 nodeName:}" failed. No retries permitted until 2025-12-03 12:01:25.203029346 +0000 UTC m=+2475.791271110 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/b2a6be10-e7df-494c-be94-a8ede55e2868-operator-scripts") pod "watcher36ab-account-delete-cvjb7" (UID: "b2a6be10-e7df-494c-be94-a8ede55e2868") : configmap "openstack-scripts" not found Dec 03 12:01:21 crc kubenswrapper[5052]: I1203 12:01:21.218126 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage18-crc" (UniqueName: "kubernetes.io/local-volume/local-storage18-crc") on node "crc" Dec 03 12:01:21 crc kubenswrapper[5052]: I1203 12:01:21.304990 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage18-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage18-crc\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:21 crc kubenswrapper[5052]: E1203 12:01:21.341952 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="fa90af8ae17f19b8e657ed6b90a3d32798eac71027682bd29946af1de563200d" cmd=["/usr/bin/pgrep","-r","DRST","nova-scheduler"] Dec 03 12:01:21 crc kubenswrapper[5052]: E1203 12:01:21.343623 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="fa90af8ae17f19b8e657ed6b90a3d32798eac71027682bd29946af1de563200d" cmd=["/usr/bin/pgrep","-r","DRST","nova-scheduler"] Dec 03 12:01:21 crc kubenswrapper[5052]: E1203 12:01:21.345023 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="fa90af8ae17f19b8e657ed6b90a3d32798eac71027682bd29946af1de563200d" cmd=["/usr/bin/pgrep","-r","DRST","nova-scheduler"] Dec 03 12:01:21 crc kubenswrapper[5052]: E1203 12:01:21.345122 5052 prober.go:104] "Probe errored" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" probeType="Readiness" pod="openstack-kuttl-tests/nova-scheduler-0" podUID="73c1b1fb-5062-4c33-be17-72324f222396" containerName="nova-scheduler-scheduler" Dec 03 12:01:21 crc kubenswrapper[5052]: I1203 12:01:21.686183 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0291d0e7-9abf-48dd-8d5e-c329f989b700" path="/var/lib/kubelet/pods/0291d0e7-9abf-48dd-8d5e-c329f989b700/volumes" Dec 03 12:01:21 crc kubenswrapper[5052]: I1203 12:01:21.687802 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0432c361-154a-46a9-b675-04267061f13b" path="/var/lib/kubelet/pods/0432c361-154a-46a9-b675-04267061f13b/volumes" Dec 03 12:01:21 crc kubenswrapper[5052]: I1203 12:01:21.688571 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0a480cd2-586d-4213-beff-424b3d201f8e" path="/var/lib/kubelet/pods/0a480cd2-586d-4213-beff-424b3d201f8e/volumes" Dec 03 12:01:21 crc kubenswrapper[5052]: I1203 12:01:21.689955 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1103e975-cc9a-435b-adc9-5c6492e9c629" path="/var/lib/kubelet/pods/1103e975-cc9a-435b-adc9-5c6492e9c629/volumes" Dec 03 12:01:21 crc kubenswrapper[5052]: I1203 12:01:21.690668 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2d5669ca-8e5e-49b4-95e9-116eb1195030" path="/var/lib/kubelet/pods/2d5669ca-8e5e-49b4-95e9-116eb1195030/volumes" Dec 03 12:01:21 crc kubenswrapper[5052]: I1203 12:01:21.691302 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="492ec2b7-0c28-4fe2-b2be-4e9e58423f23" path="/var/lib/kubelet/pods/492ec2b7-0c28-4fe2-b2be-4e9e58423f23/volumes" Dec 03 12:01:21 crc kubenswrapper[5052]: I1203 12:01:21.692575 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="51a9dfff-37f2-4fbe-b2d1-039f20bb4560" path="/var/lib/kubelet/pods/51a9dfff-37f2-4fbe-b2d1-039f20bb4560/volumes" Dec 03 12:01:21 crc kubenswrapper[5052]: I1203 12:01:21.693678 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5785b60f-14d2-43c5-95e2-68741c6729f3" path="/var/lib/kubelet/pods/5785b60f-14d2-43c5-95e2-68741c6729f3/volumes" Dec 03 12:01:21 crc kubenswrapper[5052]: I1203 12:01:21.694377 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5a08cda2-5952-4740-a571-7058f56c3b58" path="/var/lib/kubelet/pods/5a08cda2-5952-4740-a571-7058f56c3b58/volumes" Dec 03 12:01:21 crc kubenswrapper[5052]: I1203 12:01:21.695444 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="60ca40b3-79c7-49cc-9100-0a37c47e4462" path="/var/lib/kubelet/pods/60ca40b3-79c7-49cc-9100-0a37c47e4462/volumes" Dec 03 12:01:21 crc kubenswrapper[5052]: I1203 12:01:21.696110 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="60f75996-dceb-4640-8fca-96fdcc03ebb9" path="/var/lib/kubelet/pods/60f75996-dceb-4640-8fca-96fdcc03ebb9/volumes" Dec 03 12:01:21 crc kubenswrapper[5052]: I1203 12:01:21.697271 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6de7dc2a-f19c-4cc3-9965-9be822dc6270" path="/var/lib/kubelet/pods/6de7dc2a-f19c-4cc3-9965-9be822dc6270/volumes" Dec 03 12:01:21 crc kubenswrapper[5052]: I1203 12:01:21.698230 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="75c87049-f7b9-4c6b-add4-2eec1f52deb2" path="/var/lib/kubelet/pods/75c87049-f7b9-4c6b-add4-2eec1f52deb2/volumes" Dec 03 12:01:21 crc kubenswrapper[5052]: I1203 12:01:21.698966 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="98d3c14e-7549-441a-9532-f3d75833cdf8" path="/var/lib/kubelet/pods/98d3c14e-7549-441a-9532-f3d75833cdf8/volumes" Dec 03 12:01:21 crc kubenswrapper[5052]: I1203 12:01:21.700090 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e09397b4-fabb-4e9d-8695-669bf9f49c77" path="/var/lib/kubelet/pods/e09397b4-fabb-4e9d-8695-669bf9f49c77/volumes" Dec 03 12:01:21 crc kubenswrapper[5052]: I1203 12:01:21.869312 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-server-0" event={"ID":"30578d71-fe2b-4eed-9a07-6c8bfe4a1c43","Type":"ContainerDied","Data":"5eb284d08c25ae2aebd301c1a598b75bd5453227419b8a30ec5ec374f2e772c9"} Dec 03 12:01:21 crc kubenswrapper[5052]: I1203 12:01:21.869372 5052 scope.go:117] "RemoveContainer" containerID="d9ddf262949f1f7939bc5d4f4522859474860ed8b5edac12e5534c72d138210a" Dec 03 12:01:21 crc kubenswrapper[5052]: I1203 12:01:21.869490 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:01:21 crc kubenswrapper[5052]: I1203 12:01:21.895660 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/rabbitmq-server-0"] Dec 03 12:01:21 crc kubenswrapper[5052]: I1203 12:01:21.901412 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/rabbitmq-server-0"] Dec 03 12:01:21 crc kubenswrapper[5052]: I1203 12:01:21.940060 5052 scope.go:117] "RemoveContainer" containerID="250de81141f60765c155818d8a4cfbffd80ff7babe5acee647422de56500c58c" Dec 03 12:01:22 crc kubenswrapper[5052]: E1203 12:01:22.221240 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:01:22 crc kubenswrapper[5052]: E1203 12:01:22.221332 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/cf90ea08-b172-4549-9522-92be7a5b5623-operator-scripts podName:cf90ea08-b172-4549-9522-92be7a5b5623 nodeName:}" failed. No retries permitted until 2025-12-03 12:01:26.221312519 +0000 UTC m=+2476.809554283 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/cf90ea08-b172-4549-9522-92be7a5b5623-operator-scripts") pod "novacell184d8-account-delete-4nmn9" (UID: "cf90ea08-b172-4549-9522-92be7a5b5623") : configmap "openstack-scripts" not found Dec 03 12:01:22 crc kubenswrapper[5052]: I1203 12:01:22.889147 5052 generic.go:334] "Generic (PLEG): container finished" podID="73c1b1fb-5062-4c33-be17-72324f222396" containerID="fa90af8ae17f19b8e657ed6b90a3d32798eac71027682bd29946af1de563200d" exitCode=0 Dec 03 12:01:22 crc kubenswrapper[5052]: I1203 12:01:22.889214 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"73c1b1fb-5062-4c33-be17-72324f222396","Type":"ContainerDied","Data":"fa90af8ae17f19b8e657ed6b90a3d32798eac71027682bd29946af1de563200d"} Dec 03 12:01:22 crc kubenswrapper[5052]: I1203 12:01:22.891218 5052 generic.go:334] "Generic (PLEG): container finished" podID="2d2fda50-7118-43f2-8290-36d97b66de86" containerID="541d5fc878d9121d9e872cb4361abda24d819f62b553c1995d69dbf6c3ab83a1" exitCode=0 Dec 03 12:01:22 crc kubenswrapper[5052]: I1203 12:01:22.891300 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"2d2fda50-7118-43f2-8290-36d97b66de86","Type":"ContainerDied","Data":"541d5fc878d9121d9e872cb4361abda24d819f62b553c1995d69dbf6c3ab83a1"} Dec 03 12:01:23 crc kubenswrapper[5052]: I1203 12:01:23.247549 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:01:23 crc kubenswrapper[5052]: I1203 12:01:23.337473 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2d2fda50-7118-43f2-8290-36d97b66de86-combined-ca-bundle\") pod \"2d2fda50-7118-43f2-8290-36d97b66de86\" (UID: \"2d2fda50-7118-43f2-8290-36d97b66de86\") " Dec 03 12:01:23 crc kubenswrapper[5052]: I1203 12:01:23.337585 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/2d2fda50-7118-43f2-8290-36d97b66de86-config-data-custom\") pod \"2d2fda50-7118-43f2-8290-36d97b66de86\" (UID: \"2d2fda50-7118-43f2-8290-36d97b66de86\") " Dec 03 12:01:23 crc kubenswrapper[5052]: I1203 12:01:23.337629 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fqxvv\" (UniqueName: \"kubernetes.io/projected/2d2fda50-7118-43f2-8290-36d97b66de86-kube-api-access-fqxvv\") pod \"2d2fda50-7118-43f2-8290-36d97b66de86\" (UID: \"2d2fda50-7118-43f2-8290-36d97b66de86\") " Dec 03 12:01:23 crc kubenswrapper[5052]: I1203 12:01:23.337658 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/2d2fda50-7118-43f2-8290-36d97b66de86-etc-machine-id\") pod \"2d2fda50-7118-43f2-8290-36d97b66de86\" (UID: \"2d2fda50-7118-43f2-8290-36d97b66de86\") " Dec 03 12:01:23 crc kubenswrapper[5052]: I1203 12:01:23.337705 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/2d2fda50-7118-43f2-8290-36d97b66de86-config-data\") pod \"2d2fda50-7118-43f2-8290-36d97b66de86\" (UID: \"2d2fda50-7118-43f2-8290-36d97b66de86\") " Dec 03 12:01:23 crc kubenswrapper[5052]: I1203 12:01:23.337750 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/2d2fda50-7118-43f2-8290-36d97b66de86-scripts\") pod \"2d2fda50-7118-43f2-8290-36d97b66de86\" (UID: \"2d2fda50-7118-43f2-8290-36d97b66de86\") " Dec 03 12:01:23 crc kubenswrapper[5052]: I1203 12:01:23.337832 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/2d2fda50-7118-43f2-8290-36d97b66de86-etc-machine-id" (OuterVolumeSpecName: "etc-machine-id") pod "2d2fda50-7118-43f2-8290-36d97b66de86" (UID: "2d2fda50-7118-43f2-8290-36d97b66de86"). InnerVolumeSpecName "etc-machine-id". PluginName "kubernetes.io/host-path", VolumeGidValue "" Dec 03 12:01:23 crc kubenswrapper[5052]: I1203 12:01:23.338107 5052 reconciler_common.go:293] "Volume detached for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/2d2fda50-7118-43f2-8290-36d97b66de86-etc-machine-id\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:23 crc kubenswrapper[5052]: I1203 12:01:23.351599 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2d2fda50-7118-43f2-8290-36d97b66de86-config-data-custom" (OuterVolumeSpecName: "config-data-custom") pod "2d2fda50-7118-43f2-8290-36d97b66de86" (UID: "2d2fda50-7118-43f2-8290-36d97b66de86"). InnerVolumeSpecName "config-data-custom". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:01:23 crc kubenswrapper[5052]: I1203 12:01:23.351614 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2d2fda50-7118-43f2-8290-36d97b66de86-kube-api-access-fqxvv" (OuterVolumeSpecName: "kube-api-access-fqxvv") pod "2d2fda50-7118-43f2-8290-36d97b66de86" (UID: "2d2fda50-7118-43f2-8290-36d97b66de86"). InnerVolumeSpecName "kube-api-access-fqxvv". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:01:23 crc kubenswrapper[5052]: I1203 12:01:23.353229 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2d2fda50-7118-43f2-8290-36d97b66de86-scripts" (OuterVolumeSpecName: "scripts") pod "2d2fda50-7118-43f2-8290-36d97b66de86" (UID: "2d2fda50-7118-43f2-8290-36d97b66de86"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:01:23 crc kubenswrapper[5052]: I1203 12:01:23.401731 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2d2fda50-7118-43f2-8290-36d97b66de86-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "2d2fda50-7118-43f2-8290-36d97b66de86" (UID: "2d2fda50-7118-43f2-8290-36d97b66de86"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:01:23 crc kubenswrapper[5052]: I1203 12:01:23.421081 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2d2fda50-7118-43f2-8290-36d97b66de86-config-data" (OuterVolumeSpecName: "config-data") pod "2d2fda50-7118-43f2-8290-36d97b66de86" (UID: "2d2fda50-7118-43f2-8290-36d97b66de86"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:01:23 crc kubenswrapper[5052]: I1203 12:01:23.439794 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/2d2fda50-7118-43f2-8290-36d97b66de86-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:23 crc kubenswrapper[5052]: I1203 12:01:23.439826 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2d2fda50-7118-43f2-8290-36d97b66de86-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:23 crc kubenswrapper[5052]: I1203 12:01:23.439838 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/2d2fda50-7118-43f2-8290-36d97b66de86-config-data-custom\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:23 crc kubenswrapper[5052]: I1203 12:01:23.439850 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-fqxvv\" (UniqueName: \"kubernetes.io/projected/2d2fda50-7118-43f2-8290-36d97b66de86-kube-api-access-fqxvv\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:23 crc kubenswrapper[5052]: I1203 12:01:23.439861 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/2d2fda50-7118-43f2-8290-36d97b66de86-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:23 crc kubenswrapper[5052]: I1203 12:01:23.632071 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:01:23 crc kubenswrapper[5052]: I1203 12:01:23.685647 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="30578d71-fe2b-4eed-9a07-6c8bfe4a1c43" path="/var/lib/kubelet/pods/30578d71-fe2b-4eed-9a07-6c8bfe4a1c43/volumes" Dec 03 12:01:23 crc kubenswrapper[5052]: I1203 12:01:23.745567 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r6pw2\" (UniqueName: \"kubernetes.io/projected/73c1b1fb-5062-4c33-be17-72324f222396-kube-api-access-r6pw2\") pod \"73c1b1fb-5062-4c33-be17-72324f222396\" (UID: \"73c1b1fb-5062-4c33-be17-72324f222396\") " Dec 03 12:01:23 crc kubenswrapper[5052]: I1203 12:01:23.745647 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/73c1b1fb-5062-4c33-be17-72324f222396-combined-ca-bundle\") pod \"73c1b1fb-5062-4c33-be17-72324f222396\" (UID: \"73c1b1fb-5062-4c33-be17-72324f222396\") " Dec 03 12:01:23 crc kubenswrapper[5052]: I1203 12:01:23.745731 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/73c1b1fb-5062-4c33-be17-72324f222396-config-data\") pod \"73c1b1fb-5062-4c33-be17-72324f222396\" (UID: \"73c1b1fb-5062-4c33-be17-72324f222396\") " Dec 03 12:01:23 crc kubenswrapper[5052]: I1203 12:01:23.749466 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/73c1b1fb-5062-4c33-be17-72324f222396-kube-api-access-r6pw2" (OuterVolumeSpecName: "kube-api-access-r6pw2") pod "73c1b1fb-5062-4c33-be17-72324f222396" (UID: "73c1b1fb-5062-4c33-be17-72324f222396"). InnerVolumeSpecName "kube-api-access-r6pw2". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:01:23 crc kubenswrapper[5052]: I1203 12:01:23.770056 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/73c1b1fb-5062-4c33-be17-72324f222396-config-data" (OuterVolumeSpecName: "config-data") pod "73c1b1fb-5062-4c33-be17-72324f222396" (UID: "73c1b1fb-5062-4c33-be17-72324f222396"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:01:23 crc kubenswrapper[5052]: I1203 12:01:23.770617 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/73c1b1fb-5062-4c33-be17-72324f222396-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "73c1b1fb-5062-4c33-be17-72324f222396" (UID: "73c1b1fb-5062-4c33-be17-72324f222396"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:01:23 crc kubenswrapper[5052]: I1203 12:01:23.848113 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-r6pw2\" (UniqueName: \"kubernetes.io/projected/73c1b1fb-5062-4c33-be17-72324f222396-kube-api-access-r6pw2\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:23 crc kubenswrapper[5052]: I1203 12:01:23.848151 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/73c1b1fb-5062-4c33-be17-72324f222396-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:23 crc kubenswrapper[5052]: I1203 12:01:23.848160 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/73c1b1fb-5062-4c33-be17-72324f222396-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:23 crc kubenswrapper[5052]: I1203 12:01:23.904364 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:01:23 crc kubenswrapper[5052]: I1203 12:01:23.904652 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"2d2fda50-7118-43f2-8290-36d97b66de86","Type":"ContainerDied","Data":"9442fbe11e649cdb482a44ed1cc5ef0f6c2f30c2feeae4bab8dc6c03ec92704c"} Dec 03 12:01:23 crc kubenswrapper[5052]: I1203 12:01:23.904706 5052 scope.go:117] "RemoveContainer" containerID="365e155e111e9968f8e4c88e8465be33bb14e93a08d5db3a41bced9b4391e4fd" Dec 03 12:01:23 crc kubenswrapper[5052]: I1203 12:01:23.907324 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"73c1b1fb-5062-4c33-be17-72324f222396","Type":"ContainerDied","Data":"b6417a09a2777892975f9717fe655663cdf13e6c02a46df1a2143449ae46e532"} Dec 03 12:01:23 crc kubenswrapper[5052]: I1203 12:01:23.907441 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:01:23 crc kubenswrapper[5052]: I1203 12:01:23.938869 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 12:01:23 crc kubenswrapper[5052]: I1203 12:01:23.955095 5052 scope.go:117] "RemoveContainer" containerID="541d5fc878d9121d9e872cb4361abda24d819f62b553c1995d69dbf6c3ab83a1" Dec 03 12:01:23 crc kubenswrapper[5052]: I1203 12:01:23.955258 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 12:01:23 crc kubenswrapper[5052]: I1203 12:01:23.955355 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:01:23 crc kubenswrapper[5052]: I1203 12:01:23.966286 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:01:23 crc kubenswrapper[5052]: I1203 12:01:23.986064 5052 scope.go:117] "RemoveContainer" containerID="fa90af8ae17f19b8e657ed6b90a3d32798eac71027682bd29946af1de563200d" Dec 03 12:01:24 crc kubenswrapper[5052]: I1203 12:01:24.897687 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-wfzg9" Dec 03 12:01:24 crc kubenswrapper[5052]: I1203 12:01:24.954760 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-b6ccb55c5-4rjkw"] Dec 03 12:01:24 crc kubenswrapper[5052]: E1203 12:01:24.984320 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:01:24 crc kubenswrapper[5052]: E1203 12:01:24.984636 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6343f4a5-892f-4118-a666-e6601d2df818-operator-scripts podName:6343f4a5-892f-4118-a666-e6601d2df818 nodeName:}" failed. No retries permitted until 2025-12-03 12:01:32.984610679 +0000 UTC m=+2483.572852443 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/6343f4a5-892f-4118-a666-e6601d2df818-operator-scripts") pod "novacell06be2-account-delete-7m8cr" (UID: "6343f4a5-892f-4118-a666-e6601d2df818") : configmap "openstack-scripts" not found Dec 03 12:01:24 crc kubenswrapper[5052]: E1203 12:01:24.984996 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:01:24 crc kubenswrapper[5052]: E1203 12:01:24.985132 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d420a2c2-9b7b-400b-88a7-839a93cce573-operator-scripts podName:d420a2c2-9b7b-400b-88a7-839a93cce573 nodeName:}" failed. No retries permitted until 2025-12-03 12:01:32.985070112 +0000 UTC m=+2483.573311876 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/d420a2c2-9b7b-400b-88a7-839a93cce573-operator-scripts") pod "barbican4aeb-account-delete-bvhdj" (UID: "d420a2c2-9b7b-400b-88a7-839a93cce573") : configmap "openstack-scripts" not found Dec 03 12:01:24 crc kubenswrapper[5052]: I1203 12:01:24.994830 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-b6ccb55c5-4rjkw" podUID="10e87ef1-40ac-4691-a6ef-5f91352e85e7" containerName="dnsmasq-dns" containerID="cri-o://b8cd94166a15570855e1df2776b78818e8ab0111af82a51c3cc7ddaffca1dc22" gracePeriod=10 Dec 03 12:01:25 crc kubenswrapper[5052]: E1203 12:01:25.086121 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:01:25 crc kubenswrapper[5052]: E1203 12:01:25.086174 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:01:25 crc kubenswrapper[5052]: E1203 12:01:25.086220 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/36225798-caeb-4514-9f51-c8a07fb6da9b-operator-scripts podName:36225798-caeb-4514-9f51-c8a07fb6da9b nodeName:}" failed. No retries permitted until 2025-12-03 12:01:33.086198264 +0000 UTC m=+2483.674440028 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/36225798-caeb-4514-9f51-c8a07fb6da9b-operator-scripts") pod "placemente203-account-delete-sdpql" (UID: "36225798-caeb-4514-9f51-c8a07fb6da9b") : configmap "openstack-scripts" not found Dec 03 12:01:25 crc kubenswrapper[5052]: E1203 12:01:25.086262 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/eaeff688-fe65-403f-a01a-dfff69db9fb9-operator-scripts podName:eaeff688-fe65-403f-a01a-dfff69db9fb9 nodeName:}" failed. No retries permitted until 2025-12-03 12:01:33.086238145 +0000 UTC m=+2483.674479989 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/eaeff688-fe65-403f-a01a-dfff69db9fb9-operator-scripts") pod "novaapif429-account-delete-qwz7z" (UID: "eaeff688-fe65-403f-a01a-dfff69db9fb9") : configmap "openstack-scripts" not found Dec 03 12:01:25 crc kubenswrapper[5052]: E1203 12:01:25.086136 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:01:25 crc kubenswrapper[5052]: E1203 12:01:25.086326 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1a97695f-322c-4c4c-834e-ecb2f27c2e97-operator-scripts podName:1a97695f-322c-4c4c-834e-ecb2f27c2e97 nodeName:}" failed. No retries permitted until 2025-12-03 12:01:33.086311817 +0000 UTC m=+2483.674553681 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/1a97695f-322c-4c4c-834e-ecb2f27c2e97-operator-scripts") pod "glance0d27-account-delete-brqv4" (UID: "1a97695f-322c-4c4c-834e-ecb2f27c2e97") : configmap "openstack-scripts" not found Dec 03 12:01:25 crc kubenswrapper[5052]: E1203 12:01:25.187458 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:01:25 crc kubenswrapper[5052]: E1203 12:01:25.187574 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c17bae4f-1c1b-4da2-9bb5-c84d6fdfa184-operator-scripts podName:c17bae4f-1c1b-4da2-9bb5-c84d6fdfa184 nodeName:}" failed. No retries permitted until 2025-12-03 12:01:33.187553931 +0000 UTC m=+2483.775795705 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/c17bae4f-1c1b-4da2-9bb5-c84d6fdfa184-operator-scripts") pod "neutron0ea0-account-delete-lvffs" (UID: "c17bae4f-1c1b-4da2-9bb5-c84d6fdfa184") : configmap "openstack-scripts" not found Dec 03 12:01:25 crc kubenswrapper[5052]: E1203 12:01:25.289274 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:01:25 crc kubenswrapper[5052]: E1203 12:01:25.289385 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b2a6be10-e7df-494c-be94-a8ede55e2868-operator-scripts podName:b2a6be10-e7df-494c-be94-a8ede55e2868 nodeName:}" failed. No retries permitted until 2025-12-03 12:01:33.289360692 +0000 UTC m=+2483.877602486 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/b2a6be10-e7df-494c-be94-a8ede55e2868-operator-scripts") pod "watcher36ab-account-delete-cvjb7" (UID: "b2a6be10-e7df-494c-be94-a8ede55e2868") : configmap "openstack-scripts" not found Dec 03 12:01:25 crc kubenswrapper[5052]: I1203 12:01:25.683822 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2d2fda50-7118-43f2-8290-36d97b66de86" path="/var/lib/kubelet/pods/2d2fda50-7118-43f2-8290-36d97b66de86/volumes" Dec 03 12:01:25 crc kubenswrapper[5052]: I1203 12:01:25.684885 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="73c1b1fb-5062-4c33-be17-72324f222396" path="/var/lib/kubelet/pods/73c1b1fb-5062-4c33-be17-72324f222396/volumes" Dec 03 12:01:25 crc kubenswrapper[5052]: I1203 12:01:25.929562 5052 generic.go:334] "Generic (PLEG): container finished" podID="10e87ef1-40ac-4691-a6ef-5f91352e85e7" containerID="b8cd94166a15570855e1df2776b78818e8ab0111af82a51c3cc7ddaffca1dc22" exitCode=0 Dec 03 12:01:25 crc kubenswrapper[5052]: I1203 12:01:25.929603 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-b6ccb55c5-4rjkw" event={"ID":"10e87ef1-40ac-4691-a6ef-5f91352e85e7","Type":"ContainerDied","Data":"b8cd94166a15570855e1df2776b78818e8ab0111af82a51c3cc7ddaffca1dc22"} Dec 03 12:01:26 crc kubenswrapper[5052]: I1203 12:01:26.137940 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-b6ccb55c5-4rjkw" Dec 03 12:01:26 crc kubenswrapper[5052]: I1203 12:01:26.210685 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/10e87ef1-40ac-4691-a6ef-5f91352e85e7-config\") pod \"10e87ef1-40ac-4691-a6ef-5f91352e85e7\" (UID: \"10e87ef1-40ac-4691-a6ef-5f91352e85e7\") " Dec 03 12:01:26 crc kubenswrapper[5052]: I1203 12:01:26.210770 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2z4h9\" (UniqueName: \"kubernetes.io/projected/10e87ef1-40ac-4691-a6ef-5f91352e85e7-kube-api-access-2z4h9\") pod \"10e87ef1-40ac-4691-a6ef-5f91352e85e7\" (UID: \"10e87ef1-40ac-4691-a6ef-5f91352e85e7\") " Dec 03 12:01:26 crc kubenswrapper[5052]: I1203 12:01:26.210844 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"dns-swift-storage-0\" (UniqueName: \"kubernetes.io/configmap/10e87ef1-40ac-4691-a6ef-5f91352e85e7-dns-swift-storage-0\") pod \"10e87ef1-40ac-4691-a6ef-5f91352e85e7\" (UID: \"10e87ef1-40ac-4691-a6ef-5f91352e85e7\") " Dec 03 12:01:26 crc kubenswrapper[5052]: I1203 12:01:26.210877 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/10e87ef1-40ac-4691-a6ef-5f91352e85e7-dnsmasq-svc\") pod \"10e87ef1-40ac-4691-a6ef-5f91352e85e7\" (UID: \"10e87ef1-40ac-4691-a6ef-5f91352e85e7\") " Dec 03 12:01:26 crc kubenswrapper[5052]: I1203 12:01:26.216286 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/10e87ef1-40ac-4691-a6ef-5f91352e85e7-kube-api-access-2z4h9" (OuterVolumeSpecName: "kube-api-access-2z4h9") pod "10e87ef1-40ac-4691-a6ef-5f91352e85e7" (UID: "10e87ef1-40ac-4691-a6ef-5f91352e85e7"). InnerVolumeSpecName "kube-api-access-2z4h9". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:01:26 crc kubenswrapper[5052]: I1203 12:01:26.253020 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/10e87ef1-40ac-4691-a6ef-5f91352e85e7-dns-swift-storage-0" (OuterVolumeSpecName: "dns-swift-storage-0") pod "10e87ef1-40ac-4691-a6ef-5f91352e85e7" (UID: "10e87ef1-40ac-4691-a6ef-5f91352e85e7"). InnerVolumeSpecName "dns-swift-storage-0". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:01:26 crc kubenswrapper[5052]: I1203 12:01:26.258443 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/10e87ef1-40ac-4691-a6ef-5f91352e85e7-config" (OuterVolumeSpecName: "config") pod "10e87ef1-40ac-4691-a6ef-5f91352e85e7" (UID: "10e87ef1-40ac-4691-a6ef-5f91352e85e7"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:01:26 crc kubenswrapper[5052]: I1203 12:01:26.261475 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/10e87ef1-40ac-4691-a6ef-5f91352e85e7-dnsmasq-svc" (OuterVolumeSpecName: "dnsmasq-svc") pod "10e87ef1-40ac-4691-a6ef-5f91352e85e7" (UID: "10e87ef1-40ac-4691-a6ef-5f91352e85e7"). InnerVolumeSpecName "dnsmasq-svc". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:01:26 crc kubenswrapper[5052]: I1203 12:01:26.313677 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/10e87ef1-40ac-4691-a6ef-5f91352e85e7-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:26 crc kubenswrapper[5052]: I1203 12:01:26.313731 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-2z4h9\" (UniqueName: \"kubernetes.io/projected/10e87ef1-40ac-4691-a6ef-5f91352e85e7-kube-api-access-2z4h9\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:26 crc kubenswrapper[5052]: I1203 12:01:26.313748 5052 reconciler_common.go:293] "Volume detached for volume \"dns-swift-storage-0\" (UniqueName: \"kubernetes.io/configmap/10e87ef1-40ac-4691-a6ef-5f91352e85e7-dns-swift-storage-0\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:26 crc kubenswrapper[5052]: I1203 12:01:26.313761 5052 reconciler_common.go:293] "Volume detached for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/10e87ef1-40ac-4691-a6ef-5f91352e85e7-dnsmasq-svc\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:26 crc kubenswrapper[5052]: E1203 12:01:26.313795 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:01:26 crc kubenswrapper[5052]: E1203 12:01:26.313926 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/cf90ea08-b172-4549-9522-92be7a5b5623-operator-scripts podName:cf90ea08-b172-4549-9522-92be7a5b5623 nodeName:}" failed. No retries permitted until 2025-12-03 12:01:34.313897802 +0000 UTC m=+2484.902139756 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/cf90ea08-b172-4549-9522-92be7a5b5623-operator-scripts") pod "novacell184d8-account-delete-4nmn9" (UID: "cf90ea08-b172-4549-9522-92be7a5b5623") : configmap "openstack-scripts" not found Dec 03 12:01:26 crc kubenswrapper[5052]: I1203 12:01:26.942985 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-b6ccb55c5-4rjkw" event={"ID":"10e87ef1-40ac-4691-a6ef-5f91352e85e7","Type":"ContainerDied","Data":"f7eba0d2677b6143975774d22746fe1ae4a84b5d06485c4469e16c1463c80800"} Dec 03 12:01:26 crc kubenswrapper[5052]: I1203 12:01:26.943054 5052 scope.go:117] "RemoveContainer" containerID="b8cd94166a15570855e1df2776b78818e8ab0111af82a51c3cc7ddaffca1dc22" Dec 03 12:01:26 crc kubenswrapper[5052]: I1203 12:01:26.943189 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-b6ccb55c5-4rjkw" Dec 03 12:01:26 crc kubenswrapper[5052]: I1203 12:01:26.979705 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-b6ccb55c5-4rjkw"] Dec 03 12:01:26 crc kubenswrapper[5052]: I1203 12:01:26.979866 5052 scope.go:117] "RemoveContainer" containerID="23eaf5d19434696e2ee9e76a1ffe87942aaa033ee35702b7b274a64f72ffa07f" Dec 03 12:01:26 crc kubenswrapper[5052]: I1203 12:01:26.986072 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-b6ccb55c5-4rjkw"] Dec 03 12:01:27 crc kubenswrapper[5052]: I1203 12:01:27.685977 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="10e87ef1-40ac-4691-a6ef-5f91352e85e7" path="/var/lib/kubelet/pods/10e87ef1-40ac-4691-a6ef-5f91352e85e7/volumes" Dec 03 12:01:29 crc kubenswrapper[5052]: E1203 12:01:29.256432 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of 1a61e630d65199a5ceab549e6965a4dfa5b829b5db1743b8843226f45bd3e896 is running failed: container process not found" containerID="1a61e630d65199a5ceab549e6965a4dfa5b829b5db1743b8843226f45bd3e896" cmd=["/usr/bin/pgrep","-f","-r","DRST","watcher-decision-engine"] Dec 03 12:01:29 crc kubenswrapper[5052]: E1203 12:01:29.257647 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of 1a61e630d65199a5ceab549e6965a4dfa5b829b5db1743b8843226f45bd3e896 is running failed: container process not found" containerID="1a61e630d65199a5ceab549e6965a4dfa5b829b5db1743b8843226f45bd3e896" cmd=["/usr/bin/pgrep","-f","-r","DRST","watcher-decision-engine"] Dec 03 12:01:29 crc kubenswrapper[5052]: E1203 12:01:29.258314 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of 1a61e630d65199a5ceab549e6965a4dfa5b829b5db1743b8843226f45bd3e896 is running failed: container process not found" containerID="1a61e630d65199a5ceab549e6965a4dfa5b829b5db1743b8843226f45bd3e896" cmd=["/usr/bin/pgrep","-f","-r","DRST","watcher-decision-engine"] Dec 03 12:01:29 crc kubenswrapper[5052]: E1203 12:01:29.258367 5052 prober.go:104] "Probe errored" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of 1a61e630d65199a5ceab549e6965a4dfa5b829b5db1743b8843226f45bd3e896 is running failed: container process not found" probeType="Readiness" pod="openstack-kuttl-tests/watcher-decision-engine-0" podUID="e9985bb3-f162-4793-9491-2e4f46644071" containerName="watcher-decision-engine" Dec 03 12:01:29 crc kubenswrapper[5052]: I1203 12:01:29.917099 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/watcher-decision-engine-0" Dec 03 12:01:29 crc kubenswrapper[5052]: I1203 12:01:29.983770 5052 generic.go:334] "Generic (PLEG): container finished" podID="e9985bb3-f162-4793-9491-2e4f46644071" containerID="1a61e630d65199a5ceab549e6965a4dfa5b829b5db1743b8843226f45bd3e896" exitCode=0 Dec 03 12:01:29 crc kubenswrapper[5052]: I1203 12:01:29.983816 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/watcher-decision-engine-0" event={"ID":"e9985bb3-f162-4793-9491-2e4f46644071","Type":"ContainerDied","Data":"1a61e630d65199a5ceab549e6965a4dfa5b829b5db1743b8843226f45bd3e896"} Dec 03 12:01:29 crc kubenswrapper[5052]: I1203 12:01:29.983844 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/watcher-decision-engine-0" event={"ID":"e9985bb3-f162-4793-9491-2e4f46644071","Type":"ContainerDied","Data":"d312a50e3e98ec7eec06a0fabc9c4b327c92d31256027ba7ee537bcf312f3337"} Dec 03 12:01:29 crc kubenswrapper[5052]: I1203 12:01:29.983860 5052 scope.go:117] "RemoveContainer" containerID="1a61e630d65199a5ceab549e6965a4dfa5b829b5db1743b8843226f45bd3e896" Dec 03 12:01:29 crc kubenswrapper[5052]: I1203 12:01:29.983988 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/watcher-decision-engine-0" Dec 03 12:01:30 crc kubenswrapper[5052]: I1203 12:01:30.008181 5052 scope.go:117] "RemoveContainer" containerID="1a61e630d65199a5ceab549e6965a4dfa5b829b5db1743b8843226f45bd3e896" Dec 03 12:01:30 crc kubenswrapper[5052]: E1203 12:01:30.008762 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1a61e630d65199a5ceab549e6965a4dfa5b829b5db1743b8843226f45bd3e896\": container with ID starting with 1a61e630d65199a5ceab549e6965a4dfa5b829b5db1743b8843226f45bd3e896 not found: ID does not exist" containerID="1a61e630d65199a5ceab549e6965a4dfa5b829b5db1743b8843226f45bd3e896" Dec 03 12:01:30 crc kubenswrapper[5052]: I1203 12:01:30.008815 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1a61e630d65199a5ceab549e6965a4dfa5b829b5db1743b8843226f45bd3e896"} err="failed to get container status \"1a61e630d65199a5ceab549e6965a4dfa5b829b5db1743b8843226f45bd3e896\": rpc error: code = NotFound desc = could not find container \"1a61e630d65199a5ceab549e6965a4dfa5b829b5db1743b8843226f45bd3e896\": container with ID starting with 1a61e630d65199a5ceab549e6965a4dfa5b829b5db1743b8843226f45bd3e896 not found: ID does not exist" Dec 03 12:01:30 crc kubenswrapper[5052]: I1203 12:01:30.082479 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e9985bb3-f162-4793-9491-2e4f46644071-combined-ca-bundle\") pod \"e9985bb3-f162-4793-9491-2e4f46644071\" (UID: \"e9985bb3-f162-4793-9491-2e4f46644071\") " Dec 03 12:01:30 crc kubenswrapper[5052]: I1203 12:01:30.082544 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/e9985bb3-f162-4793-9491-2e4f46644071-logs\") pod \"e9985bb3-f162-4793-9491-2e4f46644071\" (UID: \"e9985bb3-f162-4793-9491-2e4f46644071\") " Dec 03 12:01:30 crc kubenswrapper[5052]: I1203 12:01:30.082583 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ngtzp\" (UniqueName: \"kubernetes.io/projected/e9985bb3-f162-4793-9491-2e4f46644071-kube-api-access-ngtzp\") pod \"e9985bb3-f162-4793-9491-2e4f46644071\" (UID: \"e9985bb3-f162-4793-9491-2e4f46644071\") " Dec 03 12:01:30 crc kubenswrapper[5052]: I1203 12:01:30.082644 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"custom-prometheus-ca\" (UniqueName: \"kubernetes.io/secret/e9985bb3-f162-4793-9491-2e4f46644071-custom-prometheus-ca\") pod \"e9985bb3-f162-4793-9491-2e4f46644071\" (UID: \"e9985bb3-f162-4793-9491-2e4f46644071\") " Dec 03 12:01:30 crc kubenswrapper[5052]: I1203 12:01:30.082666 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/e9985bb3-f162-4793-9491-2e4f46644071-config-data\") pod \"e9985bb3-f162-4793-9491-2e4f46644071\" (UID: \"e9985bb3-f162-4793-9491-2e4f46644071\") " Dec 03 12:01:30 crc kubenswrapper[5052]: I1203 12:01:30.083578 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9985bb3-f162-4793-9491-2e4f46644071-logs" (OuterVolumeSpecName: "logs") pod "e9985bb3-f162-4793-9491-2e4f46644071" (UID: "e9985bb3-f162-4793-9491-2e4f46644071"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:01:30 crc kubenswrapper[5052]: I1203 12:01:30.093500 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e9985bb3-f162-4793-9491-2e4f46644071-kube-api-access-ngtzp" (OuterVolumeSpecName: "kube-api-access-ngtzp") pod "e9985bb3-f162-4793-9491-2e4f46644071" (UID: "e9985bb3-f162-4793-9491-2e4f46644071"). InnerVolumeSpecName "kube-api-access-ngtzp". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:01:30 crc kubenswrapper[5052]: I1203 12:01:30.105040 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e9985bb3-f162-4793-9491-2e4f46644071-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "e9985bb3-f162-4793-9491-2e4f46644071" (UID: "e9985bb3-f162-4793-9491-2e4f46644071"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:01:30 crc kubenswrapper[5052]: I1203 12:01:30.108421 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e9985bb3-f162-4793-9491-2e4f46644071-custom-prometheus-ca" (OuterVolumeSpecName: "custom-prometheus-ca") pod "e9985bb3-f162-4793-9491-2e4f46644071" (UID: "e9985bb3-f162-4793-9491-2e4f46644071"). InnerVolumeSpecName "custom-prometheus-ca". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:01:30 crc kubenswrapper[5052]: I1203 12:01:30.122135 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e9985bb3-f162-4793-9491-2e4f46644071-config-data" (OuterVolumeSpecName: "config-data") pod "e9985bb3-f162-4793-9491-2e4f46644071" (UID: "e9985bb3-f162-4793-9491-2e4f46644071"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:01:30 crc kubenswrapper[5052]: I1203 12:01:30.184450 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-ngtzp\" (UniqueName: \"kubernetes.io/projected/e9985bb3-f162-4793-9491-2e4f46644071-kube-api-access-ngtzp\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:30 crc kubenswrapper[5052]: I1203 12:01:30.184504 5052 reconciler_common.go:293] "Volume detached for volume \"custom-prometheus-ca\" (UniqueName: \"kubernetes.io/secret/e9985bb3-f162-4793-9491-2e4f46644071-custom-prometheus-ca\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:30 crc kubenswrapper[5052]: I1203 12:01:30.184548 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/e9985bb3-f162-4793-9491-2e4f46644071-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:30 crc kubenswrapper[5052]: I1203 12:01:30.184569 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e9985bb3-f162-4793-9491-2e4f46644071-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:30 crc kubenswrapper[5052]: I1203 12:01:30.184587 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/e9985bb3-f162-4793-9491-2e4f46644071-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:30 crc kubenswrapper[5052]: I1203 12:01:30.322867 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/watcher-decision-engine-0"] Dec 03 12:01:30 crc kubenswrapper[5052]: I1203 12:01:30.333565 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/watcher-decision-engine-0"] Dec 03 12:01:31 crc kubenswrapper[5052]: I1203 12:01:31.676890 5052 scope.go:117] "RemoveContainer" containerID="d224717c6d9ca8135fd510e03a6c40c9f2b1a56dd8b4a3b685af3e2dfa855982" Dec 03 12:01:31 crc kubenswrapper[5052]: E1203 12:01:31.677390 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 12:01:31 crc kubenswrapper[5052]: I1203 12:01:31.682224 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e9985bb3-f162-4793-9491-2e4f46644071" path="/var/lib/kubelet/pods/e9985bb3-f162-4793-9491-2e4f46644071/volumes" Dec 03 12:01:33 crc kubenswrapper[5052]: E1203 12:01:33.033205 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:01:33 crc kubenswrapper[5052]: E1203 12:01:33.033283 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:01:33 crc kubenswrapper[5052]: E1203 12:01:33.033314 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d420a2c2-9b7b-400b-88a7-839a93cce573-operator-scripts podName:d420a2c2-9b7b-400b-88a7-839a93cce573 nodeName:}" failed. No retries permitted until 2025-12-03 12:01:49.033276886 +0000 UTC m=+2499.621518650 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/d420a2c2-9b7b-400b-88a7-839a93cce573-operator-scripts") pod "barbican4aeb-account-delete-bvhdj" (UID: "d420a2c2-9b7b-400b-88a7-839a93cce573") : configmap "openstack-scripts" not found Dec 03 12:01:33 crc kubenswrapper[5052]: E1203 12:01:33.033336 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6343f4a5-892f-4118-a666-e6601d2df818-operator-scripts podName:6343f4a5-892f-4118-a666-e6601d2df818 nodeName:}" failed. No retries permitted until 2025-12-03 12:01:49.033328287 +0000 UTC m=+2499.621570051 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/6343f4a5-892f-4118-a666-e6601d2df818-operator-scripts") pod "novacell06be2-account-delete-7m8cr" (UID: "6343f4a5-892f-4118-a666-e6601d2df818") : configmap "openstack-scripts" not found Dec 03 12:01:33 crc kubenswrapper[5052]: E1203 12:01:33.135421 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:01:33 crc kubenswrapper[5052]: E1203 12:01:33.135540 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1a97695f-322c-4c4c-834e-ecb2f27c2e97-operator-scripts podName:1a97695f-322c-4c4c-834e-ecb2f27c2e97 nodeName:}" failed. No retries permitted until 2025-12-03 12:01:49.135503188 +0000 UTC m=+2499.723744952 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/1a97695f-322c-4c4c-834e-ecb2f27c2e97-operator-scripts") pod "glance0d27-account-delete-brqv4" (UID: "1a97695f-322c-4c4c-834e-ecb2f27c2e97") : configmap "openstack-scripts" not found Dec 03 12:01:33 crc kubenswrapper[5052]: E1203 12:01:33.135535 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:01:33 crc kubenswrapper[5052]: E1203 12:01:33.135679 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/36225798-caeb-4514-9f51-c8a07fb6da9b-operator-scripts podName:36225798-caeb-4514-9f51-c8a07fb6da9b nodeName:}" failed. No retries permitted until 2025-12-03 12:01:49.135648122 +0000 UTC m=+2499.723889896 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/36225798-caeb-4514-9f51-c8a07fb6da9b-operator-scripts") pod "placemente203-account-delete-sdpql" (UID: "36225798-caeb-4514-9f51-c8a07fb6da9b") : configmap "openstack-scripts" not found Dec 03 12:01:33 crc kubenswrapper[5052]: E1203 12:01:33.135551 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:01:33 crc kubenswrapper[5052]: E1203 12:01:33.135746 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/eaeff688-fe65-403f-a01a-dfff69db9fb9-operator-scripts podName:eaeff688-fe65-403f-a01a-dfff69db9fb9 nodeName:}" failed. No retries permitted until 2025-12-03 12:01:49.135734955 +0000 UTC m=+2499.723976739 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/eaeff688-fe65-403f-a01a-dfff69db9fb9-operator-scripts") pod "novaapif429-account-delete-qwz7z" (UID: "eaeff688-fe65-403f-a01a-dfff69db9fb9") : configmap "openstack-scripts" not found Dec 03 12:01:33 crc kubenswrapper[5052]: E1203 12:01:33.236535 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:01:33 crc kubenswrapper[5052]: E1203 12:01:33.236614 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c17bae4f-1c1b-4da2-9bb5-c84d6fdfa184-operator-scripts podName:c17bae4f-1c1b-4da2-9bb5-c84d6fdfa184 nodeName:}" failed. No retries permitted until 2025-12-03 12:01:49.236600219 +0000 UTC m=+2499.824841983 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/c17bae4f-1c1b-4da2-9bb5-c84d6fdfa184-operator-scripts") pod "neutron0ea0-account-delete-lvffs" (UID: "c17bae4f-1c1b-4da2-9bb5-c84d6fdfa184") : configmap "openstack-scripts" not found Dec 03 12:01:33 crc kubenswrapper[5052]: E1203 12:01:33.337532 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:01:33 crc kubenswrapper[5052]: E1203 12:01:33.337657 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b2a6be10-e7df-494c-be94-a8ede55e2868-operator-scripts podName:b2a6be10-e7df-494c-be94-a8ede55e2868 nodeName:}" failed. No retries permitted until 2025-12-03 12:01:49.337625746 +0000 UTC m=+2499.925867550 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/b2a6be10-e7df-494c-be94-a8ede55e2868-operator-scripts") pod "watcher36ab-account-delete-cvjb7" (UID: "b2a6be10-e7df-494c-be94-a8ede55e2868") : configmap "openstack-scripts" not found Dec 03 12:01:34 crc kubenswrapper[5052]: E1203 12:01:34.355043 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:01:34 crc kubenswrapper[5052]: E1203 12:01:34.355573 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/cf90ea08-b172-4549-9522-92be7a5b5623-operator-scripts podName:cf90ea08-b172-4549-9522-92be7a5b5623 nodeName:}" failed. No retries permitted until 2025-12-03 12:01:50.355538708 +0000 UTC m=+2500.943780502 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/cf90ea08-b172-4549-9522-92be7a5b5623-operator-scripts") pod "novacell184d8-account-delete-4nmn9" (UID: "cf90ea08-b172-4549-9522-92be7a5b5623") : configmap "openstack-scripts" not found Dec 03 12:01:36 crc kubenswrapper[5052]: I1203 12:01:36.053450 5052 generic.go:334] "Generic (PLEG): container finished" podID="73f90998-d47e-407c-b964-12065411b4e2" containerID="be99515097973c2c0916717764695e4147e9b00df227c2974f21e0db3fe72a0f" exitCode=0 Dec 03 12:01:36 crc kubenswrapper[5052]: I1203 12:01:36.053549 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-7f59bd7c8d-4jwhw" event={"ID":"73f90998-d47e-407c-b964-12065411b4e2","Type":"ContainerDied","Data":"be99515097973c2c0916717764695e4147e9b00df227c2974f21e0db3fe72a0f"} Dec 03 12:01:36 crc kubenswrapper[5052]: I1203 12:01:36.123250 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-7f59bd7c8d-4jwhw" Dec 03 12:01:36 crc kubenswrapper[5052]: I1203 12:01:36.289228 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/73f90998-d47e-407c-b964-12065411b4e2-ovndb-tls-certs\") pod \"73f90998-d47e-407c-b964-12065411b4e2\" (UID: \"73f90998-d47e-407c-b964-12065411b4e2\") " Dec 03 12:01:36 crc kubenswrapper[5052]: I1203 12:01:36.289473 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/73f90998-d47e-407c-b964-12065411b4e2-combined-ca-bundle\") pod \"73f90998-d47e-407c-b964-12065411b4e2\" (UID: \"73f90998-d47e-407c-b964-12065411b4e2\") " Dec 03 12:01:36 crc kubenswrapper[5052]: I1203 12:01:36.289539 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/73f90998-d47e-407c-b964-12065411b4e2-config\") pod \"73f90998-d47e-407c-b964-12065411b4e2\" (UID: \"73f90998-d47e-407c-b964-12065411b4e2\") " Dec 03 12:01:36 crc kubenswrapper[5052]: I1203 12:01:36.289627 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/73f90998-d47e-407c-b964-12065411b4e2-httpd-config\") pod \"73f90998-d47e-407c-b964-12065411b4e2\" (UID: \"73f90998-d47e-407c-b964-12065411b4e2\") " Dec 03 12:01:36 crc kubenswrapper[5052]: I1203 12:01:36.289651 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/73f90998-d47e-407c-b964-12065411b4e2-public-tls-certs\") pod \"73f90998-d47e-407c-b964-12065411b4e2\" (UID: \"73f90998-d47e-407c-b964-12065411b4e2\") " Dec 03 12:01:36 crc kubenswrapper[5052]: I1203 12:01:36.289681 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k29nl\" (UniqueName: \"kubernetes.io/projected/73f90998-d47e-407c-b964-12065411b4e2-kube-api-access-k29nl\") pod \"73f90998-d47e-407c-b964-12065411b4e2\" (UID: \"73f90998-d47e-407c-b964-12065411b4e2\") " Dec 03 12:01:36 crc kubenswrapper[5052]: I1203 12:01:36.289742 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/73f90998-d47e-407c-b964-12065411b4e2-internal-tls-certs\") pod \"73f90998-d47e-407c-b964-12065411b4e2\" (UID: \"73f90998-d47e-407c-b964-12065411b4e2\") " Dec 03 12:01:36 crc kubenswrapper[5052]: I1203 12:01:36.301717 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/73f90998-d47e-407c-b964-12065411b4e2-kube-api-access-k29nl" (OuterVolumeSpecName: "kube-api-access-k29nl") pod "73f90998-d47e-407c-b964-12065411b4e2" (UID: "73f90998-d47e-407c-b964-12065411b4e2"). InnerVolumeSpecName "kube-api-access-k29nl". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:01:36 crc kubenswrapper[5052]: I1203 12:01:36.302731 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/73f90998-d47e-407c-b964-12065411b4e2-httpd-config" (OuterVolumeSpecName: "httpd-config") pod "73f90998-d47e-407c-b964-12065411b4e2" (UID: "73f90998-d47e-407c-b964-12065411b4e2"). InnerVolumeSpecName "httpd-config". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:01:36 crc kubenswrapper[5052]: I1203 12:01:36.330877 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/73f90998-d47e-407c-b964-12065411b4e2-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "73f90998-d47e-407c-b964-12065411b4e2" (UID: "73f90998-d47e-407c-b964-12065411b4e2"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:01:36 crc kubenswrapper[5052]: I1203 12:01:36.334397 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/73f90998-d47e-407c-b964-12065411b4e2-config" (OuterVolumeSpecName: "config") pod "73f90998-d47e-407c-b964-12065411b4e2" (UID: "73f90998-d47e-407c-b964-12065411b4e2"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:01:36 crc kubenswrapper[5052]: I1203 12:01:36.347415 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/73f90998-d47e-407c-b964-12065411b4e2-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "73f90998-d47e-407c-b964-12065411b4e2" (UID: "73f90998-d47e-407c-b964-12065411b4e2"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:01:36 crc kubenswrapper[5052]: I1203 12:01:36.348320 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/73f90998-d47e-407c-b964-12065411b4e2-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "73f90998-d47e-407c-b964-12065411b4e2" (UID: "73f90998-d47e-407c-b964-12065411b4e2"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:01:36 crc kubenswrapper[5052]: I1203 12:01:36.349612 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/73f90998-d47e-407c-b964-12065411b4e2-ovndb-tls-certs" (OuterVolumeSpecName: "ovndb-tls-certs") pod "73f90998-d47e-407c-b964-12065411b4e2" (UID: "73f90998-d47e-407c-b964-12065411b4e2"). InnerVolumeSpecName "ovndb-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:01:36 crc kubenswrapper[5052]: I1203 12:01:36.392322 5052 reconciler_common.go:293] "Volume detached for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/73f90998-d47e-407c-b964-12065411b4e2-httpd-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:36 crc kubenswrapper[5052]: I1203 12:01:36.392384 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/73f90998-d47e-407c-b964-12065411b4e2-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:36 crc kubenswrapper[5052]: I1203 12:01:36.392400 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-k29nl\" (UniqueName: \"kubernetes.io/projected/73f90998-d47e-407c-b964-12065411b4e2-kube-api-access-k29nl\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:36 crc kubenswrapper[5052]: I1203 12:01:36.392412 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/73f90998-d47e-407c-b964-12065411b4e2-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:36 crc kubenswrapper[5052]: I1203 12:01:36.392425 5052 reconciler_common.go:293] "Volume detached for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/73f90998-d47e-407c-b964-12065411b4e2-ovndb-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:36 crc kubenswrapper[5052]: I1203 12:01:36.392436 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/73f90998-d47e-407c-b964-12065411b4e2-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:36 crc kubenswrapper[5052]: I1203 12:01:36.392448 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/73f90998-d47e-407c-b964-12065411b4e2-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:37 crc kubenswrapper[5052]: I1203 12:01:37.072165 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-7f59bd7c8d-4jwhw" event={"ID":"73f90998-d47e-407c-b964-12065411b4e2","Type":"ContainerDied","Data":"f08a3072fc2ec221cf35e3fa74281c89a30ce3c6cebef66afd6b1a373a9bfbbc"} Dec 03 12:01:37 crc kubenswrapper[5052]: I1203 12:01:37.072251 5052 scope.go:117] "RemoveContainer" containerID="841d973c00ea29cca586a4e7b4174c1d2c994db38756378837d3f5c9d0346486" Dec 03 12:01:37 crc kubenswrapper[5052]: I1203 12:01:37.072296 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-7f59bd7c8d-4jwhw" Dec 03 12:01:37 crc kubenswrapper[5052]: I1203 12:01:37.115870 5052 scope.go:117] "RemoveContainer" containerID="be99515097973c2c0916717764695e4147e9b00df227c2974f21e0db3fe72a0f" Dec 03 12:01:37 crc kubenswrapper[5052]: I1203 12:01:37.120985 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/neutron-7f59bd7c8d-4jwhw"] Dec 03 12:01:37 crc kubenswrapper[5052]: I1203 12:01:37.129300 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/neutron-7f59bd7c8d-4jwhw"] Dec 03 12:01:37 crc kubenswrapper[5052]: I1203 12:01:37.682201 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="73f90998-d47e-407c-b964-12065411b4e2" path="/var/lib/kubelet/pods/73f90998-d47e-407c-b964-12065411b4e2/volumes" Dec 03 12:01:43 crc kubenswrapper[5052]: I1203 12:01:43.673275 5052 scope.go:117] "RemoveContainer" containerID="d224717c6d9ca8135fd510e03a6c40c9f2b1a56dd8b4a3b685af3e2dfa855982" Dec 03 12:01:43 crc kubenswrapper[5052]: E1203 12:01:43.673829 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 12:01:44 crc kubenswrapper[5052]: I1203 12:01:44.089889 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:01:44 crc kubenswrapper[5052]: I1203 12:01:44.159190 5052 generic.go:334] "Generic (PLEG): container finished" podID="7c36839d-8518-4b0e-bf63-ec94d92e3c13" containerID="eac74c2a0f154d6d834ce77005c3de792b1ffb15c4c79891dc165810f2f43317" exitCode=137 Dec 03 12:01:44 crc kubenswrapper[5052]: I1203 12:01:44.159239 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"7c36839d-8518-4b0e-bf63-ec94d92e3c13","Type":"ContainerDied","Data":"eac74c2a0f154d6d834ce77005c3de792b1ffb15c4c79891dc165810f2f43317"} Dec 03 12:01:44 crc kubenswrapper[5052]: I1203 12:01:44.159267 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"7c36839d-8518-4b0e-bf63-ec94d92e3c13","Type":"ContainerDied","Data":"a47f522eb33440e9d86ad8e2ec20e36a501400f9a7394734425605e03d2dc2ea"} Dec 03 12:01:44 crc kubenswrapper[5052]: I1203 12:01:44.159285 5052 scope.go:117] "RemoveContainer" containerID="eac74c2a0f154d6d834ce77005c3de792b1ffb15c4c79891dc165810f2f43317" Dec 03 12:01:44 crc kubenswrapper[5052]: I1203 12:01:44.159296 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:01:44 crc kubenswrapper[5052]: I1203 12:01:44.181203 5052 scope.go:117] "RemoveContainer" containerID="52442374ae2d8fc1cfd375a7ed3be0f18021f36b0334d1758b377586b846921d" Dec 03 12:01:44 crc kubenswrapper[5052]: I1203 12:01:44.199937 5052 scope.go:117] "RemoveContainer" containerID="fcf4f78f80f00487ca12528cc832dae0efc274f54e76db4429646d7410f72e60" Dec 03 12:01:44 crc kubenswrapper[5052]: I1203 12:01:44.222000 5052 scope.go:117] "RemoveContainer" containerID="ac3482f5fa4de80aab2650bc04e9d5264e54ea6c82aaf1d84113fd1b78ebc9e2" Dec 03 12:01:44 crc kubenswrapper[5052]: I1203 12:01:44.234916 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"swift\" (UniqueName: \"kubernetes.io/local-volume/local-storage20-crc\") pod \"7c36839d-8518-4b0e-bf63-ec94d92e3c13\" (UID: \"7c36839d-8518-4b0e-bf63-ec94d92e3c13\") " Dec 03 12:01:44 crc kubenswrapper[5052]: I1203 12:01:44.235010 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/7c36839d-8518-4b0e-bf63-ec94d92e3c13-etc-swift\") pod \"7c36839d-8518-4b0e-bf63-ec94d92e3c13\" (UID: \"7c36839d-8518-4b0e-bf63-ec94d92e3c13\") " Dec 03 12:01:44 crc kubenswrapper[5052]: I1203 12:01:44.235110 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"lock\" (UniqueName: \"kubernetes.io/empty-dir/7c36839d-8518-4b0e-bf63-ec94d92e3c13-lock\") pod \"7c36839d-8518-4b0e-bf63-ec94d92e3c13\" (UID: \"7c36839d-8518-4b0e-bf63-ec94d92e3c13\") " Dec 03 12:01:44 crc kubenswrapper[5052]: I1203 12:01:44.235207 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"cache\" (UniqueName: \"kubernetes.io/empty-dir/7c36839d-8518-4b0e-bf63-ec94d92e3c13-cache\") pod \"7c36839d-8518-4b0e-bf63-ec94d92e3c13\" (UID: \"7c36839d-8518-4b0e-bf63-ec94d92e3c13\") " Dec 03 12:01:44 crc kubenswrapper[5052]: I1203 12:01:44.235236 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8zq68\" (UniqueName: \"kubernetes.io/projected/7c36839d-8518-4b0e-bf63-ec94d92e3c13-kube-api-access-8zq68\") pod \"7c36839d-8518-4b0e-bf63-ec94d92e3c13\" (UID: \"7c36839d-8518-4b0e-bf63-ec94d92e3c13\") " Dec 03 12:01:44 crc kubenswrapper[5052]: I1203 12:01:44.235887 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7c36839d-8518-4b0e-bf63-ec94d92e3c13-lock" (OuterVolumeSpecName: "lock") pod "7c36839d-8518-4b0e-bf63-ec94d92e3c13" (UID: "7c36839d-8518-4b0e-bf63-ec94d92e3c13"). InnerVolumeSpecName "lock". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:01:44 crc kubenswrapper[5052]: I1203 12:01:44.236088 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7c36839d-8518-4b0e-bf63-ec94d92e3c13-cache" (OuterVolumeSpecName: "cache") pod "7c36839d-8518-4b0e-bf63-ec94d92e3c13" (UID: "7c36839d-8518-4b0e-bf63-ec94d92e3c13"). InnerVolumeSpecName "cache". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:01:44 crc kubenswrapper[5052]: I1203 12:01:44.240287 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage20-crc" (OuterVolumeSpecName: "swift") pod "7c36839d-8518-4b0e-bf63-ec94d92e3c13" (UID: "7c36839d-8518-4b0e-bf63-ec94d92e3c13"). InnerVolumeSpecName "local-storage20-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 12:01:44 crc kubenswrapper[5052]: I1203 12:01:44.241059 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7c36839d-8518-4b0e-bf63-ec94d92e3c13-kube-api-access-8zq68" (OuterVolumeSpecName: "kube-api-access-8zq68") pod "7c36839d-8518-4b0e-bf63-ec94d92e3c13" (UID: "7c36839d-8518-4b0e-bf63-ec94d92e3c13"). InnerVolumeSpecName "kube-api-access-8zq68". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:01:44 crc kubenswrapper[5052]: I1203 12:01:44.242238 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7c36839d-8518-4b0e-bf63-ec94d92e3c13-etc-swift" (OuterVolumeSpecName: "etc-swift") pod "7c36839d-8518-4b0e-bf63-ec94d92e3c13" (UID: "7c36839d-8518-4b0e-bf63-ec94d92e3c13"). InnerVolumeSpecName "etc-swift". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:01:44 crc kubenswrapper[5052]: I1203 12:01:44.250887 5052 scope.go:117] "RemoveContainer" containerID="5f57677d7d2dc23e3fd4fa6e81bdcbe886b2bfe28287b6105f234e2a357ac510" Dec 03 12:01:44 crc kubenswrapper[5052]: I1203 12:01:44.337691 5052 reconciler_common.go:293] "Volume detached for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/7c36839d-8518-4b0e-bf63-ec94d92e3c13-etc-swift\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:44 crc kubenswrapper[5052]: I1203 12:01:44.337741 5052 reconciler_common.go:293] "Volume detached for volume \"lock\" (UniqueName: \"kubernetes.io/empty-dir/7c36839d-8518-4b0e-bf63-ec94d92e3c13-lock\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:44 crc kubenswrapper[5052]: I1203 12:01:44.337754 5052 reconciler_common.go:293] "Volume detached for volume \"cache\" (UniqueName: \"kubernetes.io/empty-dir/7c36839d-8518-4b0e-bf63-ec94d92e3c13-cache\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:44 crc kubenswrapper[5052]: I1203 12:01:44.337766 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-8zq68\" (UniqueName: \"kubernetes.io/projected/7c36839d-8518-4b0e-bf63-ec94d92e3c13-kube-api-access-8zq68\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:44 crc kubenswrapper[5052]: I1203 12:01:44.337806 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage20-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage20-crc\") on node \"crc\" " Dec 03 12:01:44 crc kubenswrapper[5052]: I1203 12:01:44.340192 5052 scope.go:117] "RemoveContainer" containerID="99e3b71ca2f828cfb4d8d9e730eaa81b9c4c624aca8e895c30c5e6fdf97517ac" Dec 03 12:01:44 crc kubenswrapper[5052]: I1203 12:01:44.353575 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage20-crc" (UniqueName: "kubernetes.io/local-volume/local-storage20-crc") on node "crc" Dec 03 12:01:44 crc kubenswrapper[5052]: I1203 12:01:44.359864 5052 scope.go:117] "RemoveContainer" containerID="4066fa8ec273d30d5e0da93dde5867442a7c42b658e83fd69bdb6853a4596664" Dec 03 12:01:44 crc kubenswrapper[5052]: I1203 12:01:44.378124 5052 scope.go:117] "RemoveContainer" containerID="1baab415bc319f90ebf14e16f12a6406c7324f83d7e1edd6fce3f36ba0d17106" Dec 03 12:01:44 crc kubenswrapper[5052]: I1203 12:01:44.397552 5052 scope.go:117] "RemoveContainer" containerID="8bf39629dcf79a88a7eec59f36c04fffa01173de57cefa2317a3fcbe70912276" Dec 03 12:01:44 crc kubenswrapper[5052]: I1203 12:01:44.421761 5052 scope.go:117] "RemoveContainer" containerID="0fc7226ddfe9193d0d31fe7fe4a9dcc3ec094ff416d17f3fbe045f601f3a147d" Dec 03 12:01:44 crc kubenswrapper[5052]: I1203 12:01:44.438795 5052 scope.go:117] "RemoveContainer" containerID="2cbddf37ed81f13769bd22ea5da02802984cddbc9008db7adf62cbd5f35a8c05" Dec 03 12:01:44 crc kubenswrapper[5052]: I1203 12:01:44.439998 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage20-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage20-crc\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:44 crc kubenswrapper[5052]: I1203 12:01:44.455647 5052 scope.go:117] "RemoveContainer" containerID="11cecc9603bd65df20d6a71c4f44da191cdbb3a4f878b9e3695671d86d19bd72" Dec 03 12:01:44 crc kubenswrapper[5052]: I1203 12:01:44.473949 5052 scope.go:117] "RemoveContainer" containerID="682960d7b8798b4f6d377b7a7a2fe296dcddb554fb6aa11cf50780880e5ef868" Dec 03 12:01:44 crc kubenswrapper[5052]: I1203 12:01:44.497552 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/swift-storage-0"] Dec 03 12:01:44 crc kubenswrapper[5052]: I1203 12:01:44.503303 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/swift-storage-0"] Dec 03 12:01:44 crc kubenswrapper[5052]: I1203 12:01:44.516186 5052 scope.go:117] "RemoveContainer" containerID="109cd7db5617ce6f1e5aaea63acd2470d8ce489ee50ccc8c8d13a1422834ac45" Dec 03 12:01:44 crc kubenswrapper[5052]: I1203 12:01:44.537189 5052 scope.go:117] "RemoveContainer" containerID="25010c622994b6f9bf074625c83c6f4b20f430bc358fcaedc8fb64c94ab3dc3a" Dec 03 12:01:44 crc kubenswrapper[5052]: I1203 12:01:44.558687 5052 scope.go:117] "RemoveContainer" containerID="eac74c2a0f154d6d834ce77005c3de792b1ffb15c4c79891dc165810f2f43317" Dec 03 12:01:44 crc kubenswrapper[5052]: E1203 12:01:44.559128 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eac74c2a0f154d6d834ce77005c3de792b1ffb15c4c79891dc165810f2f43317\": container with ID starting with eac74c2a0f154d6d834ce77005c3de792b1ffb15c4c79891dc165810f2f43317 not found: ID does not exist" containerID="eac74c2a0f154d6d834ce77005c3de792b1ffb15c4c79891dc165810f2f43317" Dec 03 12:01:44 crc kubenswrapper[5052]: I1203 12:01:44.559181 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eac74c2a0f154d6d834ce77005c3de792b1ffb15c4c79891dc165810f2f43317"} err="failed to get container status \"eac74c2a0f154d6d834ce77005c3de792b1ffb15c4c79891dc165810f2f43317\": rpc error: code = NotFound desc = could not find container \"eac74c2a0f154d6d834ce77005c3de792b1ffb15c4c79891dc165810f2f43317\": container with ID starting with eac74c2a0f154d6d834ce77005c3de792b1ffb15c4c79891dc165810f2f43317 not found: ID does not exist" Dec 03 12:01:44 crc kubenswrapper[5052]: I1203 12:01:44.559207 5052 scope.go:117] "RemoveContainer" containerID="52442374ae2d8fc1cfd375a7ed3be0f18021f36b0334d1758b377586b846921d" Dec 03 12:01:44 crc kubenswrapper[5052]: E1203 12:01:44.559668 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"52442374ae2d8fc1cfd375a7ed3be0f18021f36b0334d1758b377586b846921d\": container with ID starting with 52442374ae2d8fc1cfd375a7ed3be0f18021f36b0334d1758b377586b846921d not found: ID does not exist" containerID="52442374ae2d8fc1cfd375a7ed3be0f18021f36b0334d1758b377586b846921d" Dec 03 12:01:44 crc kubenswrapper[5052]: I1203 12:01:44.559735 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"52442374ae2d8fc1cfd375a7ed3be0f18021f36b0334d1758b377586b846921d"} err="failed to get container status \"52442374ae2d8fc1cfd375a7ed3be0f18021f36b0334d1758b377586b846921d\": rpc error: code = NotFound desc = could not find container \"52442374ae2d8fc1cfd375a7ed3be0f18021f36b0334d1758b377586b846921d\": container with ID starting with 52442374ae2d8fc1cfd375a7ed3be0f18021f36b0334d1758b377586b846921d not found: ID does not exist" Dec 03 12:01:44 crc kubenswrapper[5052]: I1203 12:01:44.559763 5052 scope.go:117] "RemoveContainer" containerID="fcf4f78f80f00487ca12528cc832dae0efc274f54e76db4429646d7410f72e60" Dec 03 12:01:44 crc kubenswrapper[5052]: E1203 12:01:44.560125 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fcf4f78f80f00487ca12528cc832dae0efc274f54e76db4429646d7410f72e60\": container with ID starting with fcf4f78f80f00487ca12528cc832dae0efc274f54e76db4429646d7410f72e60 not found: ID does not exist" containerID="fcf4f78f80f00487ca12528cc832dae0efc274f54e76db4429646d7410f72e60" Dec 03 12:01:44 crc kubenswrapper[5052]: I1203 12:01:44.560187 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fcf4f78f80f00487ca12528cc832dae0efc274f54e76db4429646d7410f72e60"} err="failed to get container status \"fcf4f78f80f00487ca12528cc832dae0efc274f54e76db4429646d7410f72e60\": rpc error: code = NotFound desc = could not find container \"fcf4f78f80f00487ca12528cc832dae0efc274f54e76db4429646d7410f72e60\": container with ID starting with fcf4f78f80f00487ca12528cc832dae0efc274f54e76db4429646d7410f72e60 not found: ID does not exist" Dec 03 12:01:44 crc kubenswrapper[5052]: I1203 12:01:44.560207 5052 scope.go:117] "RemoveContainer" containerID="ac3482f5fa4de80aab2650bc04e9d5264e54ea6c82aaf1d84113fd1b78ebc9e2" Dec 03 12:01:44 crc kubenswrapper[5052]: E1203 12:01:44.560583 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ac3482f5fa4de80aab2650bc04e9d5264e54ea6c82aaf1d84113fd1b78ebc9e2\": container with ID starting with ac3482f5fa4de80aab2650bc04e9d5264e54ea6c82aaf1d84113fd1b78ebc9e2 not found: ID does not exist" containerID="ac3482f5fa4de80aab2650bc04e9d5264e54ea6c82aaf1d84113fd1b78ebc9e2" Dec 03 12:01:44 crc kubenswrapper[5052]: I1203 12:01:44.560611 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ac3482f5fa4de80aab2650bc04e9d5264e54ea6c82aaf1d84113fd1b78ebc9e2"} err="failed to get container status \"ac3482f5fa4de80aab2650bc04e9d5264e54ea6c82aaf1d84113fd1b78ebc9e2\": rpc error: code = NotFound desc = could not find container \"ac3482f5fa4de80aab2650bc04e9d5264e54ea6c82aaf1d84113fd1b78ebc9e2\": container with ID starting with ac3482f5fa4de80aab2650bc04e9d5264e54ea6c82aaf1d84113fd1b78ebc9e2 not found: ID does not exist" Dec 03 12:01:44 crc kubenswrapper[5052]: I1203 12:01:44.560628 5052 scope.go:117] "RemoveContainer" containerID="5f57677d7d2dc23e3fd4fa6e81bdcbe886b2bfe28287b6105f234e2a357ac510" Dec 03 12:01:44 crc kubenswrapper[5052]: E1203 12:01:44.560941 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5f57677d7d2dc23e3fd4fa6e81bdcbe886b2bfe28287b6105f234e2a357ac510\": container with ID starting with 5f57677d7d2dc23e3fd4fa6e81bdcbe886b2bfe28287b6105f234e2a357ac510 not found: ID does not exist" containerID="5f57677d7d2dc23e3fd4fa6e81bdcbe886b2bfe28287b6105f234e2a357ac510" Dec 03 12:01:44 crc kubenswrapper[5052]: I1203 12:01:44.560960 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5f57677d7d2dc23e3fd4fa6e81bdcbe886b2bfe28287b6105f234e2a357ac510"} err="failed to get container status \"5f57677d7d2dc23e3fd4fa6e81bdcbe886b2bfe28287b6105f234e2a357ac510\": rpc error: code = NotFound desc = could not find container \"5f57677d7d2dc23e3fd4fa6e81bdcbe886b2bfe28287b6105f234e2a357ac510\": container with ID starting with 5f57677d7d2dc23e3fd4fa6e81bdcbe886b2bfe28287b6105f234e2a357ac510 not found: ID does not exist" Dec 03 12:01:44 crc kubenswrapper[5052]: I1203 12:01:44.560990 5052 scope.go:117] "RemoveContainer" containerID="99e3b71ca2f828cfb4d8d9e730eaa81b9c4c624aca8e895c30c5e6fdf97517ac" Dec 03 12:01:44 crc kubenswrapper[5052]: E1203 12:01:44.561245 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"99e3b71ca2f828cfb4d8d9e730eaa81b9c4c624aca8e895c30c5e6fdf97517ac\": container with ID starting with 99e3b71ca2f828cfb4d8d9e730eaa81b9c4c624aca8e895c30c5e6fdf97517ac not found: ID does not exist" containerID="99e3b71ca2f828cfb4d8d9e730eaa81b9c4c624aca8e895c30c5e6fdf97517ac" Dec 03 12:01:44 crc kubenswrapper[5052]: I1203 12:01:44.561263 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"99e3b71ca2f828cfb4d8d9e730eaa81b9c4c624aca8e895c30c5e6fdf97517ac"} err="failed to get container status \"99e3b71ca2f828cfb4d8d9e730eaa81b9c4c624aca8e895c30c5e6fdf97517ac\": rpc error: code = NotFound desc = could not find container \"99e3b71ca2f828cfb4d8d9e730eaa81b9c4c624aca8e895c30c5e6fdf97517ac\": container with ID starting with 99e3b71ca2f828cfb4d8d9e730eaa81b9c4c624aca8e895c30c5e6fdf97517ac not found: ID does not exist" Dec 03 12:01:44 crc kubenswrapper[5052]: I1203 12:01:44.561294 5052 scope.go:117] "RemoveContainer" containerID="4066fa8ec273d30d5e0da93dde5867442a7c42b658e83fd69bdb6853a4596664" Dec 03 12:01:44 crc kubenswrapper[5052]: E1203 12:01:44.561554 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4066fa8ec273d30d5e0da93dde5867442a7c42b658e83fd69bdb6853a4596664\": container with ID starting with 4066fa8ec273d30d5e0da93dde5867442a7c42b658e83fd69bdb6853a4596664 not found: ID does not exist" containerID="4066fa8ec273d30d5e0da93dde5867442a7c42b658e83fd69bdb6853a4596664" Dec 03 12:01:44 crc kubenswrapper[5052]: I1203 12:01:44.562225 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4066fa8ec273d30d5e0da93dde5867442a7c42b658e83fd69bdb6853a4596664"} err="failed to get container status \"4066fa8ec273d30d5e0da93dde5867442a7c42b658e83fd69bdb6853a4596664\": rpc error: code = NotFound desc = could not find container \"4066fa8ec273d30d5e0da93dde5867442a7c42b658e83fd69bdb6853a4596664\": container with ID starting with 4066fa8ec273d30d5e0da93dde5867442a7c42b658e83fd69bdb6853a4596664 not found: ID does not exist" Dec 03 12:01:44 crc kubenswrapper[5052]: I1203 12:01:44.562246 5052 scope.go:117] "RemoveContainer" containerID="1baab415bc319f90ebf14e16f12a6406c7324f83d7e1edd6fce3f36ba0d17106" Dec 03 12:01:44 crc kubenswrapper[5052]: E1203 12:01:44.564025 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1baab415bc319f90ebf14e16f12a6406c7324f83d7e1edd6fce3f36ba0d17106\": container with ID starting with 1baab415bc319f90ebf14e16f12a6406c7324f83d7e1edd6fce3f36ba0d17106 not found: ID does not exist" containerID="1baab415bc319f90ebf14e16f12a6406c7324f83d7e1edd6fce3f36ba0d17106" Dec 03 12:01:44 crc kubenswrapper[5052]: I1203 12:01:44.564066 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1baab415bc319f90ebf14e16f12a6406c7324f83d7e1edd6fce3f36ba0d17106"} err="failed to get container status \"1baab415bc319f90ebf14e16f12a6406c7324f83d7e1edd6fce3f36ba0d17106\": rpc error: code = NotFound desc = could not find container \"1baab415bc319f90ebf14e16f12a6406c7324f83d7e1edd6fce3f36ba0d17106\": container with ID starting with 1baab415bc319f90ebf14e16f12a6406c7324f83d7e1edd6fce3f36ba0d17106 not found: ID does not exist" Dec 03 12:01:44 crc kubenswrapper[5052]: I1203 12:01:44.564126 5052 scope.go:117] "RemoveContainer" containerID="8bf39629dcf79a88a7eec59f36c04fffa01173de57cefa2317a3fcbe70912276" Dec 03 12:01:44 crc kubenswrapper[5052]: E1203 12:01:44.564624 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8bf39629dcf79a88a7eec59f36c04fffa01173de57cefa2317a3fcbe70912276\": container with ID starting with 8bf39629dcf79a88a7eec59f36c04fffa01173de57cefa2317a3fcbe70912276 not found: ID does not exist" containerID="8bf39629dcf79a88a7eec59f36c04fffa01173de57cefa2317a3fcbe70912276" Dec 03 12:01:44 crc kubenswrapper[5052]: I1203 12:01:44.564659 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8bf39629dcf79a88a7eec59f36c04fffa01173de57cefa2317a3fcbe70912276"} err="failed to get container status \"8bf39629dcf79a88a7eec59f36c04fffa01173de57cefa2317a3fcbe70912276\": rpc error: code = NotFound desc = could not find container \"8bf39629dcf79a88a7eec59f36c04fffa01173de57cefa2317a3fcbe70912276\": container with ID starting with 8bf39629dcf79a88a7eec59f36c04fffa01173de57cefa2317a3fcbe70912276 not found: ID does not exist" Dec 03 12:01:44 crc kubenswrapper[5052]: I1203 12:01:44.564842 5052 scope.go:117] "RemoveContainer" containerID="0fc7226ddfe9193d0d31fe7fe4a9dcc3ec094ff416d17f3fbe045f601f3a147d" Dec 03 12:01:44 crc kubenswrapper[5052]: E1203 12:01:44.565256 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0fc7226ddfe9193d0d31fe7fe4a9dcc3ec094ff416d17f3fbe045f601f3a147d\": container with ID starting with 0fc7226ddfe9193d0d31fe7fe4a9dcc3ec094ff416d17f3fbe045f601f3a147d not found: ID does not exist" containerID="0fc7226ddfe9193d0d31fe7fe4a9dcc3ec094ff416d17f3fbe045f601f3a147d" Dec 03 12:01:44 crc kubenswrapper[5052]: I1203 12:01:44.565328 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0fc7226ddfe9193d0d31fe7fe4a9dcc3ec094ff416d17f3fbe045f601f3a147d"} err="failed to get container status \"0fc7226ddfe9193d0d31fe7fe4a9dcc3ec094ff416d17f3fbe045f601f3a147d\": rpc error: code = NotFound desc = could not find container \"0fc7226ddfe9193d0d31fe7fe4a9dcc3ec094ff416d17f3fbe045f601f3a147d\": container with ID starting with 0fc7226ddfe9193d0d31fe7fe4a9dcc3ec094ff416d17f3fbe045f601f3a147d not found: ID does not exist" Dec 03 12:01:44 crc kubenswrapper[5052]: I1203 12:01:44.565367 5052 scope.go:117] "RemoveContainer" containerID="2cbddf37ed81f13769bd22ea5da02802984cddbc9008db7adf62cbd5f35a8c05" Dec 03 12:01:44 crc kubenswrapper[5052]: E1203 12:01:44.565717 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2cbddf37ed81f13769bd22ea5da02802984cddbc9008db7adf62cbd5f35a8c05\": container with ID starting with 2cbddf37ed81f13769bd22ea5da02802984cddbc9008db7adf62cbd5f35a8c05 not found: ID does not exist" containerID="2cbddf37ed81f13769bd22ea5da02802984cddbc9008db7adf62cbd5f35a8c05" Dec 03 12:01:44 crc kubenswrapper[5052]: I1203 12:01:44.565752 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2cbddf37ed81f13769bd22ea5da02802984cddbc9008db7adf62cbd5f35a8c05"} err="failed to get container status \"2cbddf37ed81f13769bd22ea5da02802984cddbc9008db7adf62cbd5f35a8c05\": rpc error: code = NotFound desc = could not find container \"2cbddf37ed81f13769bd22ea5da02802984cddbc9008db7adf62cbd5f35a8c05\": container with ID starting with 2cbddf37ed81f13769bd22ea5da02802984cddbc9008db7adf62cbd5f35a8c05 not found: ID does not exist" Dec 03 12:01:44 crc kubenswrapper[5052]: I1203 12:01:44.565771 5052 scope.go:117] "RemoveContainer" containerID="11cecc9603bd65df20d6a71c4f44da191cdbb3a4f878b9e3695671d86d19bd72" Dec 03 12:01:44 crc kubenswrapper[5052]: E1203 12:01:44.566129 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"11cecc9603bd65df20d6a71c4f44da191cdbb3a4f878b9e3695671d86d19bd72\": container with ID starting with 11cecc9603bd65df20d6a71c4f44da191cdbb3a4f878b9e3695671d86d19bd72 not found: ID does not exist" containerID="11cecc9603bd65df20d6a71c4f44da191cdbb3a4f878b9e3695671d86d19bd72" Dec 03 12:01:44 crc kubenswrapper[5052]: I1203 12:01:44.566148 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"11cecc9603bd65df20d6a71c4f44da191cdbb3a4f878b9e3695671d86d19bd72"} err="failed to get container status \"11cecc9603bd65df20d6a71c4f44da191cdbb3a4f878b9e3695671d86d19bd72\": rpc error: code = NotFound desc = could not find container \"11cecc9603bd65df20d6a71c4f44da191cdbb3a4f878b9e3695671d86d19bd72\": container with ID starting with 11cecc9603bd65df20d6a71c4f44da191cdbb3a4f878b9e3695671d86d19bd72 not found: ID does not exist" Dec 03 12:01:44 crc kubenswrapper[5052]: I1203 12:01:44.566160 5052 scope.go:117] "RemoveContainer" containerID="682960d7b8798b4f6d377b7a7a2fe296dcddb554fb6aa11cf50780880e5ef868" Dec 03 12:01:44 crc kubenswrapper[5052]: E1203 12:01:44.566533 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"682960d7b8798b4f6d377b7a7a2fe296dcddb554fb6aa11cf50780880e5ef868\": container with ID starting with 682960d7b8798b4f6d377b7a7a2fe296dcddb554fb6aa11cf50780880e5ef868 not found: ID does not exist" containerID="682960d7b8798b4f6d377b7a7a2fe296dcddb554fb6aa11cf50780880e5ef868" Dec 03 12:01:44 crc kubenswrapper[5052]: I1203 12:01:44.566552 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"682960d7b8798b4f6d377b7a7a2fe296dcddb554fb6aa11cf50780880e5ef868"} err="failed to get container status \"682960d7b8798b4f6d377b7a7a2fe296dcddb554fb6aa11cf50780880e5ef868\": rpc error: code = NotFound desc = could not find container \"682960d7b8798b4f6d377b7a7a2fe296dcddb554fb6aa11cf50780880e5ef868\": container with ID starting with 682960d7b8798b4f6d377b7a7a2fe296dcddb554fb6aa11cf50780880e5ef868 not found: ID does not exist" Dec 03 12:01:44 crc kubenswrapper[5052]: I1203 12:01:44.566565 5052 scope.go:117] "RemoveContainer" containerID="109cd7db5617ce6f1e5aaea63acd2470d8ce489ee50ccc8c8d13a1422834ac45" Dec 03 12:01:44 crc kubenswrapper[5052]: E1203 12:01:44.567148 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"109cd7db5617ce6f1e5aaea63acd2470d8ce489ee50ccc8c8d13a1422834ac45\": container with ID starting with 109cd7db5617ce6f1e5aaea63acd2470d8ce489ee50ccc8c8d13a1422834ac45 not found: ID does not exist" containerID="109cd7db5617ce6f1e5aaea63acd2470d8ce489ee50ccc8c8d13a1422834ac45" Dec 03 12:01:44 crc kubenswrapper[5052]: I1203 12:01:44.567177 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"109cd7db5617ce6f1e5aaea63acd2470d8ce489ee50ccc8c8d13a1422834ac45"} err="failed to get container status \"109cd7db5617ce6f1e5aaea63acd2470d8ce489ee50ccc8c8d13a1422834ac45\": rpc error: code = NotFound desc = could not find container \"109cd7db5617ce6f1e5aaea63acd2470d8ce489ee50ccc8c8d13a1422834ac45\": container with ID starting with 109cd7db5617ce6f1e5aaea63acd2470d8ce489ee50ccc8c8d13a1422834ac45 not found: ID does not exist" Dec 03 12:01:44 crc kubenswrapper[5052]: I1203 12:01:44.567192 5052 scope.go:117] "RemoveContainer" containerID="25010c622994b6f9bf074625c83c6f4b20f430bc358fcaedc8fb64c94ab3dc3a" Dec 03 12:01:44 crc kubenswrapper[5052]: E1203 12:01:44.567594 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"25010c622994b6f9bf074625c83c6f4b20f430bc358fcaedc8fb64c94ab3dc3a\": container with ID starting with 25010c622994b6f9bf074625c83c6f4b20f430bc358fcaedc8fb64c94ab3dc3a not found: ID does not exist" containerID="25010c622994b6f9bf074625c83c6f4b20f430bc358fcaedc8fb64c94ab3dc3a" Dec 03 12:01:44 crc kubenswrapper[5052]: I1203 12:01:44.567630 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"25010c622994b6f9bf074625c83c6f4b20f430bc358fcaedc8fb64c94ab3dc3a"} err="failed to get container status \"25010c622994b6f9bf074625c83c6f4b20f430bc358fcaedc8fb64c94ab3dc3a\": rpc error: code = NotFound desc = could not find container \"25010c622994b6f9bf074625c83c6f4b20f430bc358fcaedc8fb64c94ab3dc3a\": container with ID starting with 25010c622994b6f9bf074625c83c6f4b20f430bc358fcaedc8fb64c94ab3dc3a not found: ID does not exist" Dec 03 12:01:45 crc kubenswrapper[5052]: I1203 12:01:45.685138 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7c36839d-8518-4b0e-bf63-ec94d92e3c13" path="/var/lib/kubelet/pods/7c36839d-8518-4b0e-bf63-ec94d92e3c13/volumes" Dec 03 12:01:48 crc kubenswrapper[5052]: I1203 12:01:48.722371 5052 pod_container_manager_linux.go:210] "Failed to delete cgroup paths" cgroupName=["kubepods","besteffort","pod192fa797-c8b8-42d3-a689-e6a7dbc77282"] err="unable to destroy cgroup paths for cgroup [kubepods besteffort pod192fa797-c8b8-42d3-a689-e6a7dbc77282] : Timed out while waiting for systemd to remove kubepods-besteffort-pod192fa797_c8b8_42d3_a689_e6a7dbc77282.slice" Dec 03 12:01:48 crc kubenswrapper[5052]: E1203 12:01:48.722980 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to delete cgroup paths for [kubepods besteffort pod192fa797-c8b8-42d3-a689-e6a7dbc77282] : unable to destroy cgroup paths for cgroup [kubepods besteffort pod192fa797-c8b8-42d3-a689-e6a7dbc77282] : Timed out while waiting for systemd to remove kubepods-besteffort-pod192fa797_c8b8_42d3_a689_e6a7dbc77282.slice" pod="openstack-kuttl-tests/watcher-applier-0" podUID="192fa797-c8b8-42d3-a689-e6a7dbc77282" Dec 03 12:01:48 crc kubenswrapper[5052]: I1203 12:01:48.981221 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance0d27-account-delete-brqv4" Dec 03 12:01:49 crc kubenswrapper[5052]: I1203 12:01:49.116743 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lfdbn\" (UniqueName: \"kubernetes.io/projected/1a97695f-322c-4c4c-834e-ecb2f27c2e97-kube-api-access-lfdbn\") pod \"1a97695f-322c-4c4c-834e-ecb2f27c2e97\" (UID: \"1a97695f-322c-4c4c-834e-ecb2f27c2e97\") " Dec 03 12:01:49 crc kubenswrapper[5052]: I1203 12:01:49.116870 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/1a97695f-322c-4c4c-834e-ecb2f27c2e97-operator-scripts\") pod \"1a97695f-322c-4c4c-834e-ecb2f27c2e97\" (UID: \"1a97695f-322c-4c4c-834e-ecb2f27c2e97\") " Dec 03 12:01:49 crc kubenswrapper[5052]: I1203 12:01:49.117359 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1a97695f-322c-4c4c-834e-ecb2f27c2e97-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "1a97695f-322c-4c4c-834e-ecb2f27c2e97" (UID: "1a97695f-322c-4c4c-834e-ecb2f27c2e97"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:01:49 crc kubenswrapper[5052]: E1203 12:01:49.117438 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:01:49 crc kubenswrapper[5052]: E1203 12:01:49.117464 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:01:49 crc kubenswrapper[5052]: E1203 12:01:49.117490 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d420a2c2-9b7b-400b-88a7-839a93cce573-operator-scripts podName:d420a2c2-9b7b-400b-88a7-839a93cce573 nodeName:}" failed. No retries permitted until 2025-12-03 12:02:21.117473672 +0000 UTC m=+2531.705715436 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/d420a2c2-9b7b-400b-88a7-839a93cce573-operator-scripts") pod "barbican4aeb-account-delete-bvhdj" (UID: "d420a2c2-9b7b-400b-88a7-839a93cce573") : configmap "openstack-scripts" not found Dec 03 12:01:49 crc kubenswrapper[5052]: E1203 12:01:49.117628 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6343f4a5-892f-4118-a666-e6601d2df818-operator-scripts podName:6343f4a5-892f-4118-a666-e6601d2df818 nodeName:}" failed. No retries permitted until 2025-12-03 12:02:21.117608726 +0000 UTC m=+2531.705850490 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/6343f4a5-892f-4118-a666-e6601d2df818-operator-scripts") pod "novacell06be2-account-delete-7m8cr" (UID: "6343f4a5-892f-4118-a666-e6601d2df818") : configmap "openstack-scripts" not found Dec 03 12:01:49 crc kubenswrapper[5052]: I1203 12:01:49.134722 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1a97695f-322c-4c4c-834e-ecb2f27c2e97-kube-api-access-lfdbn" (OuterVolumeSpecName: "kube-api-access-lfdbn") pod "1a97695f-322c-4c4c-834e-ecb2f27c2e97" (UID: "1a97695f-322c-4c4c-834e-ecb2f27c2e97"). InnerVolumeSpecName "kube-api-access-lfdbn". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:01:49 crc kubenswrapper[5052]: I1203 12:01:49.220652 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-lfdbn\" (UniqueName: \"kubernetes.io/projected/1a97695f-322c-4c4c-834e-ecb2f27c2e97-kube-api-access-lfdbn\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:49 crc kubenswrapper[5052]: I1203 12:01:49.221007 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/1a97695f-322c-4c4c-834e-ecb2f27c2e97-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:49 crc kubenswrapper[5052]: E1203 12:01:49.220732 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:01:49 crc kubenswrapper[5052]: E1203 12:01:49.220962 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:01:49 crc kubenswrapper[5052]: E1203 12:01:49.221178 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/eaeff688-fe65-403f-a01a-dfff69db9fb9-operator-scripts podName:eaeff688-fe65-403f-a01a-dfff69db9fb9 nodeName:}" failed. No retries permitted until 2025-12-03 12:02:21.221132555 +0000 UTC m=+2531.809374319 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/eaeff688-fe65-403f-a01a-dfff69db9fb9-operator-scripts") pod "novaapif429-account-delete-qwz7z" (UID: "eaeff688-fe65-403f-a01a-dfff69db9fb9") : configmap "openstack-scripts" not found Dec 03 12:01:49 crc kubenswrapper[5052]: E1203 12:01:49.221406 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/36225798-caeb-4514-9f51-c8a07fb6da9b-operator-scripts podName:36225798-caeb-4514-9f51-c8a07fb6da9b nodeName:}" failed. No retries permitted until 2025-12-03 12:02:21.221323431 +0000 UTC m=+2531.809565195 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/36225798-caeb-4514-9f51-c8a07fb6da9b-operator-scripts") pod "placemente203-account-delete-sdpql" (UID: "36225798-caeb-4514-9f51-c8a07fb6da9b") : configmap "openstack-scripts" not found Dec 03 12:01:49 crc kubenswrapper[5052]: I1203 12:01:49.239874 5052 generic.go:334] "Generic (PLEG): container finished" podID="1a97695f-322c-4c4c-834e-ecb2f27c2e97" containerID="8a30f3c7e814f9afe6fc81acd574630714d7b92fdabac257b238341c59e0cc48" exitCode=137 Dec 03 12:01:49 crc kubenswrapper[5052]: I1203 12:01:49.239938 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance0d27-account-delete-brqv4" event={"ID":"1a97695f-322c-4c4c-834e-ecb2f27c2e97","Type":"ContainerDied","Data":"8a30f3c7e814f9afe6fc81acd574630714d7b92fdabac257b238341c59e0cc48"} Dec 03 12:01:49 crc kubenswrapper[5052]: I1203 12:01:49.239986 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance0d27-account-delete-brqv4" Dec 03 12:01:49 crc kubenswrapper[5052]: I1203 12:01:49.240010 5052 scope.go:117] "RemoveContainer" containerID="8a30f3c7e814f9afe6fc81acd574630714d7b92fdabac257b238341c59e0cc48" Dec 03 12:01:49 crc kubenswrapper[5052]: I1203 12:01:49.239996 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance0d27-account-delete-brqv4" event={"ID":"1a97695f-322c-4c4c-834e-ecb2f27c2e97","Type":"ContainerDied","Data":"1028d16ccbe5cc7f8fc751b118eb9f0f6a94f366ef81a2b38d3bfb04f93ca1fc"} Dec 03 12:01:49 crc kubenswrapper[5052]: I1203 12:01:49.240012 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/watcher-applier-0" Dec 03 12:01:49 crc kubenswrapper[5052]: I1203 12:01:49.269080 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/watcher-applier-0"] Dec 03 12:01:49 crc kubenswrapper[5052]: I1203 12:01:49.279647 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/watcher-applier-0"] Dec 03 12:01:49 crc kubenswrapper[5052]: I1203 12:01:49.285078 5052 scope.go:117] "RemoveContainer" containerID="8a30f3c7e814f9afe6fc81acd574630714d7b92fdabac257b238341c59e0cc48" Dec 03 12:01:49 crc kubenswrapper[5052]: E1203 12:01:49.285423 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8a30f3c7e814f9afe6fc81acd574630714d7b92fdabac257b238341c59e0cc48\": container with ID starting with 8a30f3c7e814f9afe6fc81acd574630714d7b92fdabac257b238341c59e0cc48 not found: ID does not exist" containerID="8a30f3c7e814f9afe6fc81acd574630714d7b92fdabac257b238341c59e0cc48" Dec 03 12:01:49 crc kubenswrapper[5052]: I1203 12:01:49.285464 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8a30f3c7e814f9afe6fc81acd574630714d7b92fdabac257b238341c59e0cc48"} err="failed to get container status \"8a30f3c7e814f9afe6fc81acd574630714d7b92fdabac257b238341c59e0cc48\": rpc error: code = NotFound desc = could not find container \"8a30f3c7e814f9afe6fc81acd574630714d7b92fdabac257b238341c59e0cc48\": container with ID starting with 8a30f3c7e814f9afe6fc81acd574630714d7b92fdabac257b238341c59e0cc48 not found: ID does not exist" Dec 03 12:01:49 crc kubenswrapper[5052]: I1203 12:01:49.286220 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance0d27-account-delete-brqv4"] Dec 03 12:01:49 crc kubenswrapper[5052]: I1203 12:01:49.291667 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/glance0d27-account-delete-brqv4"] Dec 03 12:01:49 crc kubenswrapper[5052]: E1203 12:01:49.322919 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:01:49 crc kubenswrapper[5052]: E1203 12:01:49.322984 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c17bae4f-1c1b-4da2-9bb5-c84d6fdfa184-operator-scripts podName:c17bae4f-1c1b-4da2-9bb5-c84d6fdfa184 nodeName:}" failed. No retries permitted until 2025-12-03 12:02:21.322970137 +0000 UTC m=+2531.911211901 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/c17bae4f-1c1b-4da2-9bb5-c84d6fdfa184-operator-scripts") pod "neutron0ea0-account-delete-lvffs" (UID: "c17bae4f-1c1b-4da2-9bb5-c84d6fdfa184") : configmap "openstack-scripts" not found Dec 03 12:01:49 crc kubenswrapper[5052]: E1203 12:01:49.424741 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:01:49 crc kubenswrapper[5052]: E1203 12:01:49.424857 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b2a6be10-e7df-494c-be94-a8ede55e2868-operator-scripts podName:b2a6be10-e7df-494c-be94-a8ede55e2868 nodeName:}" failed. No retries permitted until 2025-12-03 12:02:21.424831879 +0000 UTC m=+2532.013073653 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/b2a6be10-e7df-494c-be94-a8ede55e2868-operator-scripts") pod "watcher36ab-account-delete-cvjb7" (UID: "b2a6be10-e7df-494c-be94-a8ede55e2868") : configmap "openstack-scripts" not found Dec 03 12:01:49 crc kubenswrapper[5052]: I1203 12:01:49.684343 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="192fa797-c8b8-42d3-a689-e6a7dbc77282" path="/var/lib/kubelet/pods/192fa797-c8b8-42d3-a689-e6a7dbc77282/volumes" Dec 03 12:01:49 crc kubenswrapper[5052]: I1203 12:01:49.684968 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1a97695f-322c-4c4c-834e-ecb2f27c2e97" path="/var/lib/kubelet/pods/1a97695f-322c-4c4c-834e-ecb2f27c2e97/volumes" Dec 03 12:01:50 crc kubenswrapper[5052]: I1203 12:01:50.252421 5052 generic.go:334] "Generic (PLEG): container finished" podID="c17bae4f-1c1b-4da2-9bb5-c84d6fdfa184" containerID="65e23cdd1d480da8bdf0b2780f64b3f291ae9fc3c647fcf6b00e0dd8e2199e7e" exitCode=137 Dec 03 12:01:50 crc kubenswrapper[5052]: I1203 12:01:50.252625 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron0ea0-account-delete-lvffs" event={"ID":"c17bae4f-1c1b-4da2-9bb5-c84d6fdfa184","Type":"ContainerDied","Data":"65e23cdd1d480da8bdf0b2780f64b3f291ae9fc3c647fcf6b00e0dd8e2199e7e"} Dec 03 12:01:50 crc kubenswrapper[5052]: I1203 12:01:50.252798 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron0ea0-account-delete-lvffs" event={"ID":"c17bae4f-1c1b-4da2-9bb5-c84d6fdfa184","Type":"ContainerDied","Data":"3ed5d3b7ccddd21aa86bde44d59c298853aa183e114f61bd8e87fa66be37af06"} Dec 03 12:01:50 crc kubenswrapper[5052]: I1203 12:01:50.252810 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3ed5d3b7ccddd21aa86bde44d59c298853aa183e114f61bd8e87fa66be37af06" Dec 03 12:01:50 crc kubenswrapper[5052]: I1203 12:01:50.259286 5052 generic.go:334] "Generic (PLEG): container finished" podID="6343f4a5-892f-4118-a666-e6601d2df818" containerID="e4bacb322ed2aac7c9485b5f2c64807ad6675335614edb5a4ddbdabca0c1fa1c" exitCode=137 Dec 03 12:01:50 crc kubenswrapper[5052]: I1203 12:01:50.259348 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/novacell06be2-account-delete-7m8cr" event={"ID":"6343f4a5-892f-4118-a666-e6601d2df818","Type":"ContainerDied","Data":"e4bacb322ed2aac7c9485b5f2c64807ad6675335614edb5a4ddbdabca0c1fa1c"} Dec 03 12:01:50 crc kubenswrapper[5052]: I1203 12:01:50.261759 5052 generic.go:334] "Generic (PLEG): container finished" podID="d420a2c2-9b7b-400b-88a7-839a93cce573" containerID="428b572eb3f0325bc4882ff8b1b7b1e0f00b59900d1b774323cd1ac0a5bb910c" exitCode=137 Dec 03 12:01:50 crc kubenswrapper[5052]: I1203 12:01:50.261836 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican4aeb-account-delete-bvhdj" event={"ID":"d420a2c2-9b7b-400b-88a7-839a93cce573","Type":"ContainerDied","Data":"428b572eb3f0325bc4882ff8b1b7b1e0f00b59900d1b774323cd1ac0a5bb910c"} Dec 03 12:01:50 crc kubenswrapper[5052]: I1203 12:01:50.261869 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican4aeb-account-delete-bvhdj" event={"ID":"d420a2c2-9b7b-400b-88a7-839a93cce573","Type":"ContainerDied","Data":"1c87d91694a155411ce2f1edf96cdd4dce990021b797e016cf7712dfb5d7780c"} Dec 03 12:01:50 crc kubenswrapper[5052]: I1203 12:01:50.261881 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1c87d91694a155411ce2f1edf96cdd4dce990021b797e016cf7712dfb5d7780c" Dec 03 12:01:50 crc kubenswrapper[5052]: I1203 12:01:50.263359 5052 generic.go:334] "Generic (PLEG): container finished" podID="36225798-caeb-4514-9f51-c8a07fb6da9b" containerID="63357c27c7708913d6e53569543c75fd7424796b936ce997d25a360f5d1d6f99" exitCode=137 Dec 03 12:01:50 crc kubenswrapper[5052]: I1203 12:01:50.263435 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placemente203-account-delete-sdpql" event={"ID":"36225798-caeb-4514-9f51-c8a07fb6da9b","Type":"ContainerDied","Data":"63357c27c7708913d6e53569543c75fd7424796b936ce997d25a360f5d1d6f99"} Dec 03 12:01:50 crc kubenswrapper[5052]: I1203 12:01:50.263465 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placemente203-account-delete-sdpql" event={"ID":"36225798-caeb-4514-9f51-c8a07fb6da9b","Type":"ContainerDied","Data":"f4465faaa5fa11eb658b595d91c5bb333d670fc1540b89c1e7c2d98cb5ec5e6d"} Dec 03 12:01:50 crc kubenswrapper[5052]: I1203 12:01:50.263481 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f4465faaa5fa11eb658b595d91c5bb333d670fc1540b89c1e7c2d98cb5ec5e6d" Dec 03 12:01:50 crc kubenswrapper[5052]: I1203 12:01:50.265021 5052 generic.go:334] "Generic (PLEG): container finished" podID="eaeff688-fe65-403f-a01a-dfff69db9fb9" containerID="f7eae34142d8abb346f6280e3bbdc08126ce436adb89c7086b7aa2dead8ef908" exitCode=137 Dec 03 12:01:50 crc kubenswrapper[5052]: I1203 12:01:50.265068 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/novaapif429-account-delete-qwz7z" event={"ID":"eaeff688-fe65-403f-a01a-dfff69db9fb9","Type":"ContainerDied","Data":"f7eae34142d8abb346f6280e3bbdc08126ce436adb89c7086b7aa2dead8ef908"} Dec 03 12:01:50 crc kubenswrapper[5052]: I1203 12:01:50.265085 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/novaapif429-account-delete-qwz7z" event={"ID":"eaeff688-fe65-403f-a01a-dfff69db9fb9","Type":"ContainerDied","Data":"7634d97b9cc32e4677f109f4e33648a1b55579ab8f2afab402440c5c075e7a32"} Dec 03 12:01:50 crc kubenswrapper[5052]: I1203 12:01:50.265094 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7634d97b9cc32e4677f109f4e33648a1b55579ab8f2afab402440c5c075e7a32" Dec 03 12:01:50 crc kubenswrapper[5052]: I1203 12:01:50.267596 5052 generic.go:334] "Generic (PLEG): container finished" podID="b2a6be10-e7df-494c-be94-a8ede55e2868" containerID="0a5a8979c8ed65f7542409f1cb80cacdb93b2cadbc0a475c4d889d76b585b861" exitCode=137 Dec 03 12:01:50 crc kubenswrapper[5052]: I1203 12:01:50.267649 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/watcher36ab-account-delete-cvjb7" event={"ID":"b2a6be10-e7df-494c-be94-a8ede55e2868","Type":"ContainerDied","Data":"0a5a8979c8ed65f7542409f1cb80cacdb93b2cadbc0a475c4d889d76b585b861"} Dec 03 12:01:50 crc kubenswrapper[5052]: I1203 12:01:50.269133 5052 generic.go:334] "Generic (PLEG): container finished" podID="cf90ea08-b172-4549-9522-92be7a5b5623" containerID="5fa05d1c2c7ef877a6224f4d7efc7e1e0e45168ddefbd565972822108fe67233" exitCode=137 Dec 03 12:01:50 crc kubenswrapper[5052]: I1203 12:01:50.269156 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/novacell184d8-account-delete-4nmn9" event={"ID":"cf90ea08-b172-4549-9522-92be7a5b5623","Type":"ContainerDied","Data":"5fa05d1c2c7ef877a6224f4d7efc7e1e0e45168ddefbd565972822108fe67233"} Dec 03 12:01:50 crc kubenswrapper[5052]: I1203 12:01:50.364087 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican4aeb-account-delete-bvhdj" Dec 03 12:01:50 crc kubenswrapper[5052]: I1203 12:01:50.370874 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placemente203-account-delete-sdpql" Dec 03 12:01:50 crc kubenswrapper[5052]: I1203 12:01:50.386922 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/novaapif429-account-delete-qwz7z" Dec 03 12:01:50 crc kubenswrapper[5052]: I1203 12:01:50.392241 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron0ea0-account-delete-lvffs" Dec 03 12:01:50 crc kubenswrapper[5052]: I1203 12:01:50.408952 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/novacell06be2-account-delete-7m8cr" Dec 03 12:01:50 crc kubenswrapper[5052]: I1203 12:01:50.420823 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/watcher36ab-account-delete-cvjb7" Dec 03 12:01:50 crc kubenswrapper[5052]: I1203 12:01:50.437888 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/novacell184d8-account-delete-4nmn9" Dec 03 12:01:50 crc kubenswrapper[5052]: E1203 12:01:50.442169 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:01:50 crc kubenswrapper[5052]: E1203 12:01:50.442233 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/cf90ea08-b172-4549-9522-92be7a5b5623-operator-scripts podName:cf90ea08-b172-4549-9522-92be7a5b5623 nodeName:}" failed. No retries permitted until 2025-12-03 12:02:22.442216496 +0000 UTC m=+2533.030458260 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/cf90ea08-b172-4549-9522-92be7a5b5623-operator-scripts") pod "novacell184d8-account-delete-4nmn9" (UID: "cf90ea08-b172-4549-9522-92be7a5b5623") : configmap "openstack-scripts" not found Dec 03 12:01:50 crc kubenswrapper[5052]: I1203 12:01:50.542802 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/d420a2c2-9b7b-400b-88a7-839a93cce573-operator-scripts\") pod \"d420a2c2-9b7b-400b-88a7-839a93cce573\" (UID: \"d420a2c2-9b7b-400b-88a7-839a93cce573\") " Dec 03 12:01:50 crc kubenswrapper[5052]: I1203 12:01:50.542866 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/cf90ea08-b172-4549-9522-92be7a5b5623-operator-scripts\") pod \"cf90ea08-b172-4549-9522-92be7a5b5623\" (UID: \"cf90ea08-b172-4549-9522-92be7a5b5623\") " Dec 03 12:01:50 crc kubenswrapper[5052]: I1203 12:01:50.542887 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-28x5p\" (UniqueName: \"kubernetes.io/projected/36225798-caeb-4514-9f51-c8a07fb6da9b-kube-api-access-28x5p\") pod \"36225798-caeb-4514-9f51-c8a07fb6da9b\" (UID: \"36225798-caeb-4514-9f51-c8a07fb6da9b\") " Dec 03 12:01:50 crc kubenswrapper[5052]: I1203 12:01:50.542933 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mmqmq\" (UniqueName: \"kubernetes.io/projected/eaeff688-fe65-403f-a01a-dfff69db9fb9-kube-api-access-mmqmq\") pod \"eaeff688-fe65-403f-a01a-dfff69db9fb9\" (UID: \"eaeff688-fe65-403f-a01a-dfff69db9fb9\") " Dec 03 12:01:50 crc kubenswrapper[5052]: I1203 12:01:50.542960 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-spkfb\" (UniqueName: \"kubernetes.io/projected/b2a6be10-e7df-494c-be94-a8ede55e2868-kube-api-access-spkfb\") pod \"b2a6be10-e7df-494c-be94-a8ede55e2868\" (UID: \"b2a6be10-e7df-494c-be94-a8ede55e2868\") " Dec 03 12:01:50 crc kubenswrapper[5052]: I1203 12:01:50.543001 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/eaeff688-fe65-403f-a01a-dfff69db9fb9-operator-scripts\") pod \"eaeff688-fe65-403f-a01a-dfff69db9fb9\" (UID: \"eaeff688-fe65-403f-a01a-dfff69db9fb9\") " Dec 03 12:01:50 crc kubenswrapper[5052]: I1203 12:01:50.543041 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9qndr\" (UniqueName: \"kubernetes.io/projected/6343f4a5-892f-4118-a666-e6601d2df818-kube-api-access-9qndr\") pod \"6343f4a5-892f-4118-a666-e6601d2df818\" (UID: \"6343f4a5-892f-4118-a666-e6601d2df818\") " Dec 03 12:01:50 crc kubenswrapper[5052]: I1203 12:01:50.543064 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/c17bae4f-1c1b-4da2-9bb5-c84d6fdfa184-operator-scripts\") pod \"c17bae4f-1c1b-4da2-9bb5-c84d6fdfa184\" (UID: \"c17bae4f-1c1b-4da2-9bb5-c84d6fdfa184\") " Dec 03 12:01:50 crc kubenswrapper[5052]: I1203 12:01:50.543083 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/b2a6be10-e7df-494c-be94-a8ede55e2868-operator-scripts\") pod \"b2a6be10-e7df-494c-be94-a8ede55e2868\" (UID: \"b2a6be10-e7df-494c-be94-a8ede55e2868\") " Dec 03 12:01:50 crc kubenswrapper[5052]: I1203 12:01:50.543103 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/6343f4a5-892f-4118-a666-e6601d2df818-operator-scripts\") pod \"6343f4a5-892f-4118-a666-e6601d2df818\" (UID: \"6343f4a5-892f-4118-a666-e6601d2df818\") " Dec 03 12:01:50 crc kubenswrapper[5052]: I1203 12:01:50.543121 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j76fd\" (UniqueName: \"kubernetes.io/projected/c17bae4f-1c1b-4da2-9bb5-c84d6fdfa184-kube-api-access-j76fd\") pod \"c17bae4f-1c1b-4da2-9bb5-c84d6fdfa184\" (UID: \"c17bae4f-1c1b-4da2-9bb5-c84d6fdfa184\") " Dec 03 12:01:50 crc kubenswrapper[5052]: I1203 12:01:50.543155 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/36225798-caeb-4514-9f51-c8a07fb6da9b-operator-scripts\") pod \"36225798-caeb-4514-9f51-c8a07fb6da9b\" (UID: \"36225798-caeb-4514-9f51-c8a07fb6da9b\") " Dec 03 12:01:50 crc kubenswrapper[5052]: I1203 12:01:50.543175 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6md2l\" (UniqueName: \"kubernetes.io/projected/d420a2c2-9b7b-400b-88a7-839a93cce573-kube-api-access-6md2l\") pod \"d420a2c2-9b7b-400b-88a7-839a93cce573\" (UID: \"d420a2c2-9b7b-400b-88a7-839a93cce573\") " Dec 03 12:01:50 crc kubenswrapper[5052]: I1203 12:01:50.543194 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-228cq\" (UniqueName: \"kubernetes.io/projected/cf90ea08-b172-4549-9522-92be7a5b5623-kube-api-access-228cq\") pod \"cf90ea08-b172-4549-9522-92be7a5b5623\" (UID: \"cf90ea08-b172-4549-9522-92be7a5b5623\") " Dec 03 12:01:50 crc kubenswrapper[5052]: I1203 12:01:50.544930 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eaeff688-fe65-403f-a01a-dfff69db9fb9-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "eaeff688-fe65-403f-a01a-dfff69db9fb9" (UID: "eaeff688-fe65-403f-a01a-dfff69db9fb9"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:01:50 crc kubenswrapper[5052]: I1203 12:01:50.544962 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c17bae4f-1c1b-4da2-9bb5-c84d6fdfa184-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "c17bae4f-1c1b-4da2-9bb5-c84d6fdfa184" (UID: "c17bae4f-1c1b-4da2-9bb5-c84d6fdfa184"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:01:50 crc kubenswrapper[5052]: I1203 12:01:50.544943 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cf90ea08-b172-4549-9522-92be7a5b5623-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "cf90ea08-b172-4549-9522-92be7a5b5623" (UID: "cf90ea08-b172-4549-9522-92be7a5b5623"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:01:50 crc kubenswrapper[5052]: I1203 12:01:50.545146 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d420a2c2-9b7b-400b-88a7-839a93cce573-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "d420a2c2-9b7b-400b-88a7-839a93cce573" (UID: "d420a2c2-9b7b-400b-88a7-839a93cce573"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:01:50 crc kubenswrapper[5052]: I1203 12:01:50.545339 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6343f4a5-892f-4118-a666-e6601d2df818-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "6343f4a5-892f-4118-a666-e6601d2df818" (UID: "6343f4a5-892f-4118-a666-e6601d2df818"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:01:50 crc kubenswrapper[5052]: I1203 12:01:50.563195 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b2a6be10-e7df-494c-be94-a8ede55e2868-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "b2a6be10-e7df-494c-be94-a8ede55e2868" (UID: "b2a6be10-e7df-494c-be94-a8ede55e2868"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:01:50 crc kubenswrapper[5052]: I1203 12:01:50.563801 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/36225798-caeb-4514-9f51-c8a07fb6da9b-kube-api-access-28x5p" (OuterVolumeSpecName: "kube-api-access-28x5p") pod "36225798-caeb-4514-9f51-c8a07fb6da9b" (UID: "36225798-caeb-4514-9f51-c8a07fb6da9b"). InnerVolumeSpecName "kube-api-access-28x5p". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:01:50 crc kubenswrapper[5052]: I1203 12:01:50.563841 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cf90ea08-b172-4549-9522-92be7a5b5623-kube-api-access-228cq" (OuterVolumeSpecName: "kube-api-access-228cq") pod "cf90ea08-b172-4549-9522-92be7a5b5623" (UID: "cf90ea08-b172-4549-9522-92be7a5b5623"). InnerVolumeSpecName "kube-api-access-228cq". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:01:50 crc kubenswrapper[5052]: I1203 12:01:50.563957 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d420a2c2-9b7b-400b-88a7-839a93cce573-kube-api-access-6md2l" (OuterVolumeSpecName: "kube-api-access-6md2l") pod "d420a2c2-9b7b-400b-88a7-839a93cce573" (UID: "d420a2c2-9b7b-400b-88a7-839a93cce573"). InnerVolumeSpecName "kube-api-access-6md2l". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:01:50 crc kubenswrapper[5052]: I1203 12:01:50.563991 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/36225798-caeb-4514-9f51-c8a07fb6da9b-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "36225798-caeb-4514-9f51-c8a07fb6da9b" (UID: "36225798-caeb-4514-9f51-c8a07fb6da9b"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:01:50 crc kubenswrapper[5052]: I1203 12:01:50.564157 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b2a6be10-e7df-494c-be94-a8ede55e2868-kube-api-access-spkfb" (OuterVolumeSpecName: "kube-api-access-spkfb") pod "b2a6be10-e7df-494c-be94-a8ede55e2868" (UID: "b2a6be10-e7df-494c-be94-a8ede55e2868"). InnerVolumeSpecName "kube-api-access-spkfb". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:01:50 crc kubenswrapper[5052]: I1203 12:01:50.564973 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6343f4a5-892f-4118-a666-e6601d2df818-kube-api-access-9qndr" (OuterVolumeSpecName: "kube-api-access-9qndr") pod "6343f4a5-892f-4118-a666-e6601d2df818" (UID: "6343f4a5-892f-4118-a666-e6601d2df818"). InnerVolumeSpecName "kube-api-access-9qndr". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:01:50 crc kubenswrapper[5052]: I1203 12:01:50.565085 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c17bae4f-1c1b-4da2-9bb5-c84d6fdfa184-kube-api-access-j76fd" (OuterVolumeSpecName: "kube-api-access-j76fd") pod "c17bae4f-1c1b-4da2-9bb5-c84d6fdfa184" (UID: "c17bae4f-1c1b-4da2-9bb5-c84d6fdfa184"). InnerVolumeSpecName "kube-api-access-j76fd". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:01:50 crc kubenswrapper[5052]: I1203 12:01:50.565734 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eaeff688-fe65-403f-a01a-dfff69db9fb9-kube-api-access-mmqmq" (OuterVolumeSpecName: "kube-api-access-mmqmq") pod "eaeff688-fe65-403f-a01a-dfff69db9fb9" (UID: "eaeff688-fe65-403f-a01a-dfff69db9fb9"). InnerVolumeSpecName "kube-api-access-mmqmq". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:01:50 crc kubenswrapper[5052]: I1203 12:01:50.644313 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/eaeff688-fe65-403f-a01a-dfff69db9fb9-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:50 crc kubenswrapper[5052]: I1203 12:01:50.644348 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-9qndr\" (UniqueName: \"kubernetes.io/projected/6343f4a5-892f-4118-a666-e6601d2df818-kube-api-access-9qndr\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:50 crc kubenswrapper[5052]: I1203 12:01:50.644361 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/c17bae4f-1c1b-4da2-9bb5-c84d6fdfa184-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:50 crc kubenswrapper[5052]: I1203 12:01:50.644371 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/b2a6be10-e7df-494c-be94-a8ede55e2868-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:50 crc kubenswrapper[5052]: I1203 12:01:50.644381 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/6343f4a5-892f-4118-a666-e6601d2df818-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:50 crc kubenswrapper[5052]: I1203 12:01:50.644392 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-j76fd\" (UniqueName: \"kubernetes.io/projected/c17bae4f-1c1b-4da2-9bb5-c84d6fdfa184-kube-api-access-j76fd\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:50 crc kubenswrapper[5052]: I1203 12:01:50.644402 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/36225798-caeb-4514-9f51-c8a07fb6da9b-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:50 crc kubenswrapper[5052]: I1203 12:01:50.644412 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-6md2l\" (UniqueName: \"kubernetes.io/projected/d420a2c2-9b7b-400b-88a7-839a93cce573-kube-api-access-6md2l\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:50 crc kubenswrapper[5052]: I1203 12:01:50.644423 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-228cq\" (UniqueName: \"kubernetes.io/projected/cf90ea08-b172-4549-9522-92be7a5b5623-kube-api-access-228cq\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:50 crc kubenswrapper[5052]: I1203 12:01:50.644433 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/d420a2c2-9b7b-400b-88a7-839a93cce573-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:50 crc kubenswrapper[5052]: I1203 12:01:50.644443 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/cf90ea08-b172-4549-9522-92be7a5b5623-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:50 crc kubenswrapper[5052]: I1203 12:01:50.644453 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-28x5p\" (UniqueName: \"kubernetes.io/projected/36225798-caeb-4514-9f51-c8a07fb6da9b-kube-api-access-28x5p\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:50 crc kubenswrapper[5052]: I1203 12:01:50.644467 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-mmqmq\" (UniqueName: \"kubernetes.io/projected/eaeff688-fe65-403f-a01a-dfff69db9fb9-kube-api-access-mmqmq\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:50 crc kubenswrapper[5052]: I1203 12:01:50.644477 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-spkfb\" (UniqueName: \"kubernetes.io/projected/b2a6be10-e7df-494c-be94-a8ede55e2868-kube-api-access-spkfb\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:50 crc kubenswrapper[5052]: I1203 12:01:50.894816 5052 scope.go:117] "RemoveContainer" containerID="d1f2e599d330bacf497ed2a3bd6fd79e8f02effc0561f9015a472179a4829d62" Dec 03 12:01:51 crc kubenswrapper[5052]: I1203 12:01:51.281045 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/watcher36ab-account-delete-cvjb7" Dec 03 12:01:51 crc kubenswrapper[5052]: I1203 12:01:51.281063 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/watcher36ab-account-delete-cvjb7" event={"ID":"b2a6be10-e7df-494c-be94-a8ede55e2868","Type":"ContainerDied","Data":"b55a7c9c0998b40f49c2aa6b63362830feb7cfe0c74271745c31355e9b24b4e3"} Dec 03 12:01:51 crc kubenswrapper[5052]: I1203 12:01:51.281449 5052 scope.go:117] "RemoveContainer" containerID="0a5a8979c8ed65f7542409f1cb80cacdb93b2cadbc0a475c4d889d76b585b861" Dec 03 12:01:51 crc kubenswrapper[5052]: I1203 12:01:51.283482 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/novacell184d8-account-delete-4nmn9" Dec 03 12:01:51 crc kubenswrapper[5052]: I1203 12:01:51.283775 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/novacell184d8-account-delete-4nmn9" event={"ID":"cf90ea08-b172-4549-9522-92be7a5b5623","Type":"ContainerDied","Data":"44a5bf19277ea6d430a875328ef65cac7328db3c64a6c1ad83804395f34122c3"} Dec 03 12:01:51 crc kubenswrapper[5052]: I1203 12:01:51.290816 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/novacell06be2-account-delete-7m8cr" Dec 03 12:01:51 crc kubenswrapper[5052]: I1203 12:01:51.290825 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/novacell06be2-account-delete-7m8cr" event={"ID":"6343f4a5-892f-4118-a666-e6601d2df818","Type":"ContainerDied","Data":"8273a87287f1d179a43891b8da0c2941ee49856e411b0cf9a44a8684b07521ed"} Dec 03 12:01:51 crc kubenswrapper[5052]: I1203 12:01:51.290822 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placemente203-account-delete-sdpql" Dec 03 12:01:51 crc kubenswrapper[5052]: I1203 12:01:51.290851 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron0ea0-account-delete-lvffs" Dec 03 12:01:51 crc kubenswrapper[5052]: I1203 12:01:51.291007 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican4aeb-account-delete-bvhdj" Dec 03 12:01:51 crc kubenswrapper[5052]: I1203 12:01:51.292100 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/novaapif429-account-delete-qwz7z" Dec 03 12:01:51 crc kubenswrapper[5052]: I1203 12:01:51.306231 5052 scope.go:117] "RemoveContainer" containerID="5fa05d1c2c7ef877a6224f4d7efc7e1e0e45168ddefbd565972822108fe67233" Dec 03 12:01:51 crc kubenswrapper[5052]: I1203 12:01:51.333689 5052 scope.go:117] "RemoveContainer" containerID="e4bacb322ed2aac7c9485b5f2c64807ad6675335614edb5a4ddbdabca0c1fa1c" Dec 03 12:01:51 crc kubenswrapper[5052]: I1203 12:01:51.336358 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/novacell184d8-account-delete-4nmn9"] Dec 03 12:01:51 crc kubenswrapper[5052]: I1203 12:01:51.344323 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/novacell184d8-account-delete-4nmn9"] Dec 03 12:01:51 crc kubenswrapper[5052]: I1203 12:01:51.354439 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/watcher36ab-account-delete-cvjb7"] Dec 03 12:01:51 crc kubenswrapper[5052]: E1203 12:01:51.358916 5052 kuberuntime_gc.go:389] "Failed to remove container log dead symlink" err="remove /var/log/containers/novacell06be2-account-delete-7m8cr_openstack-kuttl-tests_mariadb-account-delete-e4bacb322ed2aac7c9485b5f2c64807ad6675335614edb5a4ddbdabca0c1fa1c.log: no such file or directory" path="/var/log/containers/novacell06be2-account-delete-7m8cr_openstack-kuttl-tests_mariadb-account-delete-e4bacb322ed2aac7c9485b5f2c64807ad6675335614edb5a4ddbdabca0c1fa1c.log" Dec 03 12:01:51 crc kubenswrapper[5052]: I1203 12:01:51.369216 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/watcher36ab-account-delete-cvjb7"] Dec 03 12:01:51 crc kubenswrapper[5052]: I1203 12:01:51.379574 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/novaapif429-account-delete-qwz7z"] Dec 03 12:01:51 crc kubenswrapper[5052]: I1203 12:01:51.386948 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/novaapif429-account-delete-qwz7z"] Dec 03 12:01:51 crc kubenswrapper[5052]: I1203 12:01:51.395036 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican4aeb-account-delete-bvhdj"] Dec 03 12:01:51 crc kubenswrapper[5052]: I1203 12:01:51.403350 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/barbican4aeb-account-delete-bvhdj"] Dec 03 12:01:51 crc kubenswrapper[5052]: I1203 12:01:51.410986 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/placemente203-account-delete-sdpql"] Dec 03 12:01:51 crc kubenswrapper[5052]: I1203 12:01:51.416455 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/placemente203-account-delete-sdpql"] Dec 03 12:01:51 crc kubenswrapper[5052]: I1203 12:01:51.426499 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/neutron0ea0-account-delete-lvffs"] Dec 03 12:01:51 crc kubenswrapper[5052]: I1203 12:01:51.440091 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/neutron0ea0-account-delete-lvffs"] Dec 03 12:01:51 crc kubenswrapper[5052]: I1203 12:01:51.447431 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/novacell06be2-account-delete-7m8cr"] Dec 03 12:01:51 crc kubenswrapper[5052]: I1203 12:01:51.453783 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/novacell06be2-account-delete-7m8cr"] Dec 03 12:01:51 crc kubenswrapper[5052]: I1203 12:01:51.685111 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="36225798-caeb-4514-9f51-c8a07fb6da9b" path="/var/lib/kubelet/pods/36225798-caeb-4514-9f51-c8a07fb6da9b/volumes" Dec 03 12:01:51 crc kubenswrapper[5052]: I1203 12:01:51.685746 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6343f4a5-892f-4118-a666-e6601d2df818" path="/var/lib/kubelet/pods/6343f4a5-892f-4118-a666-e6601d2df818/volumes" Dec 03 12:01:51 crc kubenswrapper[5052]: I1203 12:01:51.686290 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b2a6be10-e7df-494c-be94-a8ede55e2868" path="/var/lib/kubelet/pods/b2a6be10-e7df-494c-be94-a8ede55e2868/volumes" Dec 03 12:01:51 crc kubenswrapper[5052]: I1203 12:01:51.686899 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c17bae4f-1c1b-4da2-9bb5-c84d6fdfa184" path="/var/lib/kubelet/pods/c17bae4f-1c1b-4da2-9bb5-c84d6fdfa184/volumes" Dec 03 12:01:51 crc kubenswrapper[5052]: I1203 12:01:51.688642 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cf90ea08-b172-4549-9522-92be7a5b5623" path="/var/lib/kubelet/pods/cf90ea08-b172-4549-9522-92be7a5b5623/volumes" Dec 03 12:01:51 crc kubenswrapper[5052]: I1203 12:01:51.689175 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d420a2c2-9b7b-400b-88a7-839a93cce573" path="/var/lib/kubelet/pods/d420a2c2-9b7b-400b-88a7-839a93cce573/volumes" Dec 03 12:01:51 crc kubenswrapper[5052]: I1203 12:01:51.689676 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eaeff688-fe65-403f-a01a-dfff69db9fb9" path="/var/lib/kubelet/pods/eaeff688-fe65-403f-a01a-dfff69db9fb9/volumes" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.177716 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["crc-storage/crc-storage-crc-6vfwn"] Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.184178 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["crc-storage/crc-storage-crc-6vfwn"] Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.312170 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["crc-storage/crc-storage-crc-tgd7h"] Dec 03 12:01:54 crc kubenswrapper[5052]: E1203 12:01:54.312491 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="7c36839d-8518-4b0e-bf63-ec94d92e3c13" containerName="account-reaper" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.312520 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c36839d-8518-4b0e-bf63-ec94d92e3c13" containerName="account-reaper" Dec 03 12:01:54 crc kubenswrapper[5052]: E1203 12:01:54.312534 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="5785b60f-14d2-43c5-95e2-68741c6729f3" containerName="barbican-api" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.312542 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="5785b60f-14d2-43c5-95e2-68741c6729f3" containerName="barbican-api" Dec 03 12:01:54 crc kubenswrapper[5052]: E1203 12:01:54.312561 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="98d3c14e-7549-441a-9532-f3d75833cdf8" containerName="nova-metadata-log" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.312573 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="98d3c14e-7549-441a-9532-f3d75833cdf8" containerName="nova-metadata-log" Dec 03 12:01:54 crc kubenswrapper[5052]: E1203 12:01:54.312583 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="73f90998-d47e-407c-b964-12065411b4e2" containerName="neutron-httpd" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.312590 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="73f90998-d47e-407c-b964-12065411b4e2" containerName="neutron-httpd" Dec 03 12:01:54 crc kubenswrapper[5052]: E1203 12:01:54.312604 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="da4dfbea-20a2-4638-bbbe-ed4bfee0ad49" containerName="sg-core" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.312612 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="da4dfbea-20a2-4638-bbbe-ed4bfee0ad49" containerName="sg-core" Dec 03 12:01:54 crc kubenswrapper[5052]: E1203 12:01:54.312624 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="1103e975-cc9a-435b-adc9-5c6492e9c629" containerName="nova-api-log" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.312632 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="1103e975-cc9a-435b-adc9-5c6492e9c629" containerName="nova-api-log" Dec 03 12:01:54 crc kubenswrapper[5052]: E1203 12:01:54.312648 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="30578d71-fe2b-4eed-9a07-6c8bfe4a1c43" containerName="rabbitmq" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.312654 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="30578d71-fe2b-4eed-9a07-6c8bfe4a1c43" containerName="rabbitmq" Dec 03 12:01:54 crc kubenswrapper[5052]: E1203 12:01:54.312662 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="0291d0e7-9abf-48dd-8d5e-c329f989b700" containerName="cinder-api-log" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.312669 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="0291d0e7-9abf-48dd-8d5e-c329f989b700" containerName="cinder-api-log" Dec 03 12:01:54 crc kubenswrapper[5052]: E1203 12:01:54.312681 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="5e5ddf8c-6109-481f-beb0-0802de034912" containerName="openstack-network-exporter" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.312688 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="5e5ddf8c-6109-481f-beb0-0802de034912" containerName="openstack-network-exporter" Dec 03 12:01:54 crc kubenswrapper[5052]: E1203 12:01:54.312695 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="7c36839d-8518-4b0e-bf63-ec94d92e3c13" containerName="object-server" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.312701 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c36839d-8518-4b0e-bf63-ec94d92e3c13" containerName="object-server" Dec 03 12:01:54 crc kubenswrapper[5052]: E1203 12:01:54.312708 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="da4dfbea-20a2-4638-bbbe-ed4bfee0ad49" containerName="proxy-httpd" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.312713 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="da4dfbea-20a2-4638-bbbe-ed4bfee0ad49" containerName="proxy-httpd" Dec 03 12:01:54 crc kubenswrapper[5052]: E1203 12:01:54.312719 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="7c36839d-8518-4b0e-bf63-ec94d92e3c13" containerName="container-updater" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.312725 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c36839d-8518-4b0e-bf63-ec94d92e3c13" containerName="container-updater" Dec 03 12:01:54 crc kubenswrapper[5052]: E1203 12:01:54.312735 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="01bc82de-61dc-477e-b4cf-e3b0d32c99ba" containerName="prometheus" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.312741 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="01bc82de-61dc-477e-b4cf-e3b0d32c99ba" containerName="prometheus" Dec 03 12:01:54 crc kubenswrapper[5052]: E1203 12:01:54.312753 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="eaeff688-fe65-403f-a01a-dfff69db9fb9" containerName="mariadb-account-delete" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.312759 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="eaeff688-fe65-403f-a01a-dfff69db9fb9" containerName="mariadb-account-delete" Dec 03 12:01:54 crc kubenswrapper[5052]: E1203 12:01:54.312768 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="7c36839d-8518-4b0e-bf63-ec94d92e3c13" containerName="object-updater" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.312774 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c36839d-8518-4b0e-bf63-ec94d92e3c13" containerName="object-updater" Dec 03 12:01:54 crc kubenswrapper[5052]: E1203 12:01:54.312784 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="7c36839d-8518-4b0e-bf63-ec94d92e3c13" containerName="object-expirer" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.312790 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c36839d-8518-4b0e-bf63-ec94d92e3c13" containerName="object-expirer" Dec 03 12:01:54 crc kubenswrapper[5052]: E1203 12:01:54.312799 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="01bc82de-61dc-477e-b4cf-e3b0d32c99ba" containerName="init-config-reloader" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.312805 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="01bc82de-61dc-477e-b4cf-e3b0d32c99ba" containerName="init-config-reloader" Dec 03 12:01:54 crc kubenswrapper[5052]: E1203 12:01:54.312812 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="cf90ea08-b172-4549-9522-92be7a5b5623" containerName="mariadb-account-delete" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.312818 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="cf90ea08-b172-4549-9522-92be7a5b5623" containerName="mariadb-account-delete" Dec 03 12:01:54 crc kubenswrapper[5052]: E1203 12:01:54.312827 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="0291d0e7-9abf-48dd-8d5e-c329f989b700" containerName="cinder-api" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.312834 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="0291d0e7-9abf-48dd-8d5e-c329f989b700" containerName="cinder-api" Dec 03 12:01:54 crc kubenswrapper[5052]: E1203 12:01:54.312844 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="192fa797-c8b8-42d3-a689-e6a7dbc77282" containerName="watcher-applier" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.312850 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="192fa797-c8b8-42d3-a689-e6a7dbc77282" containerName="watcher-applier" Dec 03 12:01:54 crc kubenswrapper[5052]: E1203 12:01:54.312857 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="75c87049-f7b9-4c6b-add4-2eec1f52deb2" containerName="galera" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.312863 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="75c87049-f7b9-4c6b-add4-2eec1f52deb2" containerName="galera" Dec 03 12:01:54 crc kubenswrapper[5052]: E1203 12:01:54.312873 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="7cbbb6c5-a087-400f-9beb-e740ae1ddfbd" containerName="init-config-reloader" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.312879 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="7cbbb6c5-a087-400f-9beb-e740ae1ddfbd" containerName="init-config-reloader" Dec 03 12:01:54 crc kubenswrapper[5052]: E1203 12:01:54.312887 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="7c36839d-8518-4b0e-bf63-ec94d92e3c13" containerName="object-replicator" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.312894 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c36839d-8518-4b0e-bf63-ec94d92e3c13" containerName="object-replicator" Dec 03 12:01:54 crc kubenswrapper[5052]: E1203 12:01:54.312902 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="01bc82de-61dc-477e-b4cf-e3b0d32c99ba" containerName="thanos-sidecar" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.312908 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="01bc82de-61dc-477e-b4cf-e3b0d32c99ba" containerName="thanos-sidecar" Dec 03 12:01:54 crc kubenswrapper[5052]: E1203 12:01:54.312917 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="da4dfbea-20a2-4638-bbbe-ed4bfee0ad49" containerName="ceilometer-notification-agent" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.312923 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="da4dfbea-20a2-4638-bbbe-ed4bfee0ad49" containerName="ceilometer-notification-agent" Dec 03 12:01:54 crc kubenswrapper[5052]: E1203 12:01:54.312930 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="7c36839d-8518-4b0e-bf63-ec94d92e3c13" containerName="rsync" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.312936 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c36839d-8518-4b0e-bf63-ec94d92e3c13" containerName="rsync" Dec 03 12:01:54 crc kubenswrapper[5052]: E1203 12:01:54.312943 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="01bc82de-61dc-477e-b4cf-e3b0d32c99ba" containerName="config-reloader" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.312949 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="01bc82de-61dc-477e-b4cf-e3b0d32c99ba" containerName="config-reloader" Dec 03 12:01:54 crc kubenswrapper[5052]: E1203 12:01:54.312958 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="8c848c79-8799-4e32-a635-958ce73341c4" containerName="proxy-httpd" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.312963 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="8c848c79-8799-4e32-a635-958ce73341c4" containerName="proxy-httpd" Dec 03 12:01:54 crc kubenswrapper[5052]: E1203 12:01:54.312970 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="7cbbb6c5-a087-400f-9beb-e740ae1ddfbd" containerName="config-reloader" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.312976 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="7cbbb6c5-a087-400f-9beb-e740ae1ddfbd" containerName="config-reloader" Dec 03 12:01:54 crc kubenswrapper[5052]: E1203 12:01:54.312987 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="c17bae4f-1c1b-4da2-9bb5-c84d6fdfa184" containerName="mariadb-account-delete" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.312992 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="c17bae4f-1c1b-4da2-9bb5-c84d6fdfa184" containerName="mariadb-account-delete" Dec 03 12:01:54 crc kubenswrapper[5052]: E1203 12:01:54.312999 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="5785b60f-14d2-43c5-95e2-68741c6729f3" containerName="barbican-api-log" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.313006 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="5785b60f-14d2-43c5-95e2-68741c6729f3" containerName="barbican-api-log" Dec 03 12:01:54 crc kubenswrapper[5052]: E1203 12:01:54.313015 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="1103e975-cc9a-435b-adc9-5c6492e9c629" containerName="nova-api-api" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.313020 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="1103e975-cc9a-435b-adc9-5c6492e9c629" containerName="nova-api-api" Dec 03 12:01:54 crc kubenswrapper[5052]: E1203 12:01:54.313027 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="51a9dfff-37f2-4fbe-b2d1-039f20bb4560" containerName="glance-log" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.313032 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="51a9dfff-37f2-4fbe-b2d1-039f20bb4560" containerName="glance-log" Dec 03 12:01:54 crc kubenswrapper[5052]: E1203 12:01:54.313041 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="75c87049-f7b9-4c6b-add4-2eec1f52deb2" containerName="mysql-bootstrap" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.313048 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="75c87049-f7b9-4c6b-add4-2eec1f52deb2" containerName="mysql-bootstrap" Dec 03 12:01:54 crc kubenswrapper[5052]: E1203 12:01:54.313054 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="2d2fda50-7118-43f2-8290-36d97b66de86" containerName="probe" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.313060 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d2fda50-7118-43f2-8290-36d97b66de86" containerName="probe" Dec 03 12:01:54 crc kubenswrapper[5052]: E1203 12:01:54.313068 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="98d3c14e-7549-441a-9532-f3d75833cdf8" containerName="nova-metadata-metadata" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.313073 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="98d3c14e-7549-441a-9532-f3d75833cdf8" containerName="nova-metadata-metadata" Dec 03 12:01:54 crc kubenswrapper[5052]: E1203 12:01:54.313084 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="5a08cda2-5952-4740-a571-7058f56c3b58" containerName="memcached" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.313089 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="5a08cda2-5952-4740-a571-7058f56c3b58" containerName="memcached" Dec 03 12:01:54 crc kubenswrapper[5052]: E1203 12:01:54.313099 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="d03bf1c0-7353-4836-a767-9a5b2295e8dc" containerName="kube-state-metrics" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.313105 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="d03bf1c0-7353-4836-a767-9a5b2295e8dc" containerName="kube-state-metrics" Dec 03 12:01:54 crc kubenswrapper[5052]: E1203 12:01:54.313113 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="51a9dfff-37f2-4fbe-b2d1-039f20bb4560" containerName="glance-httpd" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.313118 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="51a9dfff-37f2-4fbe-b2d1-039f20bb4560" containerName="glance-httpd" Dec 03 12:01:54 crc kubenswrapper[5052]: E1203 12:01:54.313128 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="7c36839d-8518-4b0e-bf63-ec94d92e3c13" containerName="account-auditor" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.313134 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c36839d-8518-4b0e-bf63-ec94d92e3c13" containerName="account-auditor" Dec 03 12:01:54 crc kubenswrapper[5052]: E1203 12:01:54.313144 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="30578d71-fe2b-4eed-9a07-6c8bfe4a1c43" containerName="setup-container" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.313150 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="30578d71-fe2b-4eed-9a07-6c8bfe4a1c43" containerName="setup-container" Dec 03 12:01:54 crc kubenswrapper[5052]: E1203 12:01:54.313158 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="60ca40b3-79c7-49cc-9100-0a37c47e4462" containerName="placement-api" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.313164 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="60ca40b3-79c7-49cc-9100-0a37c47e4462" containerName="placement-api" Dec 03 12:01:54 crc kubenswrapper[5052]: E1203 12:01:54.313174 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="10e87ef1-40ac-4691-a6ef-5f91352e85e7" containerName="dnsmasq-dns" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.313179 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="10e87ef1-40ac-4691-a6ef-5f91352e85e7" containerName="dnsmasq-dns" Dec 03 12:01:54 crc kubenswrapper[5052]: E1203 12:01:54.313189 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="3cb4559f-542a-4076-82fb-c0ca88f9de94" containerName="ovsdbserver-sb" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.313194 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="3cb4559f-542a-4076-82fb-c0ca88f9de94" containerName="ovsdbserver-sb" Dec 03 12:01:54 crc kubenswrapper[5052]: E1203 12:01:54.313201 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="492ec2b7-0c28-4fe2-b2be-4e9e58423f23" containerName="mariadb-account-delete" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.313209 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="492ec2b7-0c28-4fe2-b2be-4e9e58423f23" containerName="mariadb-account-delete" Dec 03 12:01:54 crc kubenswrapper[5052]: E1203 12:01:54.313216 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="60ca40b3-79c7-49cc-9100-0a37c47e4462" containerName="placement-log" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.313221 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="60ca40b3-79c7-49cc-9100-0a37c47e4462" containerName="placement-log" Dec 03 12:01:54 crc kubenswrapper[5052]: E1203 12:01:54.313229 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="7cbbb6c5-a087-400f-9beb-e740ae1ddfbd" containerName="alertmanager" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.313234 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="7cbbb6c5-a087-400f-9beb-e740ae1ddfbd" containerName="alertmanager" Dec 03 12:01:54 crc kubenswrapper[5052]: E1203 12:01:54.313241 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="2d5669ca-8e5e-49b4-95e9-116eb1195030" containerName="watcher-api-log" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.313247 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d5669ca-8e5e-49b4-95e9-116eb1195030" containerName="watcher-api-log" Dec 03 12:01:54 crc kubenswrapper[5052]: E1203 12:01:54.313255 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="e09397b4-fabb-4e9d-8695-669bf9f49c77" containerName="barbican-keystone-listener" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.313261 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="e09397b4-fabb-4e9d-8695-669bf9f49c77" containerName="barbican-keystone-listener" Dec 03 12:01:54 crc kubenswrapper[5052]: E1203 12:01:54.313270 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="da4dfbea-20a2-4638-bbbe-ed4bfee0ad49" containerName="ceilometer-central-agent" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.313276 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="da4dfbea-20a2-4638-bbbe-ed4bfee0ad49" containerName="ceilometer-central-agent" Dec 03 12:01:54 crc kubenswrapper[5052]: E1203 12:01:54.313283 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="914c0f8f-e5d1-4c63-b985-0bcd07f75374" containerName="nova-cell1-novncproxy-novncproxy" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.313288 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="914c0f8f-e5d1-4c63-b985-0bcd07f75374" containerName="nova-cell1-novncproxy-novncproxy" Dec 03 12:01:54 crc kubenswrapper[5052]: E1203 12:01:54.313298 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="2d2fda50-7118-43f2-8290-36d97b66de86" containerName="cinder-scheduler" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.313304 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d2fda50-7118-43f2-8290-36d97b66de86" containerName="cinder-scheduler" Dec 03 12:01:54 crc kubenswrapper[5052]: E1203 12:01:54.313311 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="0a480cd2-586d-4213-beff-424b3d201f8e" containerName="nova-cell1-conductor-conductor" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.313316 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a480cd2-586d-4213-beff-424b3d201f8e" containerName="nova-cell1-conductor-conductor" Dec 03 12:01:54 crc kubenswrapper[5052]: E1203 12:01:54.313326 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="60f75996-dceb-4640-8fca-96fdcc03ebb9" containerName="barbican-worker" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.313331 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="60f75996-dceb-4640-8fca-96fdcc03ebb9" containerName="barbican-worker" Dec 03 12:01:54 crc kubenswrapper[5052]: E1203 12:01:54.313339 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="2d5669ca-8e5e-49b4-95e9-116eb1195030" containerName="watcher-api" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.313345 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d5669ca-8e5e-49b4-95e9-116eb1195030" containerName="watcher-api" Dec 03 12:01:54 crc kubenswrapper[5052]: E1203 12:01:54.313352 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="7c36839d-8518-4b0e-bf63-ec94d92e3c13" containerName="container-auditor" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.313357 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c36839d-8518-4b0e-bf63-ec94d92e3c13" containerName="container-auditor" Dec 03 12:01:54 crc kubenswrapper[5052]: E1203 12:01:54.313366 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="73c1b1fb-5062-4c33-be17-72324f222396" containerName="nova-scheduler-scheduler" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.313372 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="73c1b1fb-5062-4c33-be17-72324f222396" containerName="nova-scheduler-scheduler" Dec 03 12:01:54 crc kubenswrapper[5052]: E1203 12:01:54.313382 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="e8de8adf-e4e5-4a10-9722-9106357e67d0" containerName="ovsdbserver-nb" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.313387 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="e8de8adf-e4e5-4a10-9722-9106357e67d0" containerName="ovsdbserver-nb" Dec 03 12:01:54 crc kubenswrapper[5052]: E1203 12:01:54.313395 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="3cb4559f-542a-4076-82fb-c0ca88f9de94" containerName="openstack-network-exporter" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.313402 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="3cb4559f-542a-4076-82fb-c0ca88f9de94" containerName="openstack-network-exporter" Dec 03 12:01:54 crc kubenswrapper[5052]: E1203 12:01:54.313410 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="7c36839d-8518-4b0e-bf63-ec94d92e3c13" containerName="account-replicator" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.313417 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c36839d-8518-4b0e-bf63-ec94d92e3c13" containerName="account-replicator" Dec 03 12:01:54 crc kubenswrapper[5052]: E1203 12:01:54.313426 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="e09397b4-fabb-4e9d-8695-669bf9f49c77" containerName="barbican-keystone-listener-log" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.313431 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="e09397b4-fabb-4e9d-8695-669bf9f49c77" containerName="barbican-keystone-listener-log" Dec 03 12:01:54 crc kubenswrapper[5052]: E1203 12:01:54.313440 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="6de7dc2a-f19c-4cc3-9965-9be822dc6270" containerName="glance-httpd" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.313446 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="6de7dc2a-f19c-4cc3-9965-9be822dc6270" containerName="glance-httpd" Dec 03 12:01:54 crc kubenswrapper[5052]: E1203 12:01:54.313454 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="5e5ddf8c-6109-481f-beb0-0802de034912" containerName="ovn-northd" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.313461 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="5e5ddf8c-6109-481f-beb0-0802de034912" containerName="ovn-northd" Dec 03 12:01:54 crc kubenswrapper[5052]: E1203 12:01:54.313470 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="7c36839d-8518-4b0e-bf63-ec94d92e3c13" containerName="account-server" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.313476 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c36839d-8518-4b0e-bf63-ec94d92e3c13" containerName="account-server" Dec 03 12:01:54 crc kubenswrapper[5052]: E1203 12:01:54.313485 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="7c36839d-8518-4b0e-bf63-ec94d92e3c13" containerName="container-replicator" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.313491 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c36839d-8518-4b0e-bf63-ec94d92e3c13" containerName="container-replicator" Dec 03 12:01:54 crc kubenswrapper[5052]: E1203 12:01:54.313498 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="e8de8adf-e4e5-4a10-9722-9106357e67d0" containerName="openstack-network-exporter" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.313504 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="e8de8adf-e4e5-4a10-9722-9106357e67d0" containerName="openstack-network-exporter" Dec 03 12:01:54 crc kubenswrapper[5052]: E1203 12:01:54.313533 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="7c36839d-8518-4b0e-bf63-ec94d92e3c13" containerName="object-auditor" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.313538 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c36839d-8518-4b0e-bf63-ec94d92e3c13" containerName="object-auditor" Dec 03 12:01:54 crc kubenswrapper[5052]: E1203 12:01:54.313546 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="7c36839d-8518-4b0e-bf63-ec94d92e3c13" containerName="swift-recon-cron" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.313551 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c36839d-8518-4b0e-bf63-ec94d92e3c13" containerName="swift-recon-cron" Dec 03 12:01:54 crc kubenswrapper[5052]: E1203 12:01:54.313561 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="10e87ef1-40ac-4691-a6ef-5f91352e85e7" containerName="init" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.313567 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="10e87ef1-40ac-4691-a6ef-5f91352e85e7" containerName="init" Dec 03 12:01:54 crc kubenswrapper[5052]: E1203 12:01:54.313572 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="7c36839d-8518-4b0e-bf63-ec94d92e3c13" containerName="container-server" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.313578 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c36839d-8518-4b0e-bf63-ec94d92e3c13" containerName="container-server" Dec 03 12:01:54 crc kubenswrapper[5052]: E1203 12:01:54.313585 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="36225798-caeb-4514-9f51-c8a07fb6da9b" containerName="mariadb-account-delete" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.313591 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="36225798-caeb-4514-9f51-c8a07fb6da9b" containerName="mariadb-account-delete" Dec 03 12:01:54 crc kubenswrapper[5052]: E1203 12:01:54.313599 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="e9985bb3-f162-4793-9491-2e4f46644071" containerName="watcher-decision-engine" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.313605 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="e9985bb3-f162-4793-9491-2e4f46644071" containerName="watcher-decision-engine" Dec 03 12:01:54 crc kubenswrapper[5052]: E1203 12:01:54.313613 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="6de7dc2a-f19c-4cc3-9965-9be822dc6270" containerName="glance-log" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.313618 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="6de7dc2a-f19c-4cc3-9965-9be822dc6270" containerName="glance-log" Dec 03 12:01:54 crc kubenswrapper[5052]: E1203 12:01:54.313629 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="d420a2c2-9b7b-400b-88a7-839a93cce573" containerName="mariadb-account-delete" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.313634 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="d420a2c2-9b7b-400b-88a7-839a93cce573" containerName="mariadb-account-delete" Dec 03 12:01:54 crc kubenswrapper[5052]: E1203 12:01:54.313643 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="b2a6be10-e7df-494c-be94-a8ede55e2868" containerName="mariadb-account-delete" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.313648 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="b2a6be10-e7df-494c-be94-a8ede55e2868" containerName="mariadb-account-delete" Dec 03 12:01:54 crc kubenswrapper[5052]: E1203 12:01:54.313655 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="1a97695f-322c-4c4c-834e-ecb2f27c2e97" containerName="mariadb-account-delete" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.313662 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="1a97695f-322c-4c4c-834e-ecb2f27c2e97" containerName="mariadb-account-delete" Dec 03 12:01:54 crc kubenswrapper[5052]: E1203 12:01:54.313672 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="0432c361-154a-46a9-b675-04267061f13b" containerName="keystone-api" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.313678 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="0432c361-154a-46a9-b675-04267061f13b" containerName="keystone-api" Dec 03 12:01:54 crc kubenswrapper[5052]: E1203 12:01:54.313686 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="60f75996-dceb-4640-8fca-96fdcc03ebb9" containerName="barbican-worker-log" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.313692 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="60f75996-dceb-4640-8fca-96fdcc03ebb9" containerName="barbican-worker-log" Dec 03 12:01:54 crc kubenswrapper[5052]: E1203 12:01:54.313698 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="73f90998-d47e-407c-b964-12065411b4e2" containerName="neutron-api" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.313704 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="73f90998-d47e-407c-b964-12065411b4e2" containerName="neutron-api" Dec 03 12:01:54 crc kubenswrapper[5052]: E1203 12:01:54.313712 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="8c848c79-8799-4e32-a635-958ce73341c4" containerName="proxy-server" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.313717 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="8c848c79-8799-4e32-a635-958ce73341c4" containerName="proxy-server" Dec 03 12:01:54 crc kubenswrapper[5052]: E1203 12:01:54.313724 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="6343f4a5-892f-4118-a666-e6601d2df818" containerName="mariadb-account-delete" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.313730 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="6343f4a5-892f-4118-a666-e6601d2df818" containerName="mariadb-account-delete" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.313867 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="10e87ef1-40ac-4691-a6ef-5f91352e85e7" containerName="dnsmasq-dns" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.313881 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="7c36839d-8518-4b0e-bf63-ec94d92e3c13" containerName="account-replicator" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.313890 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="e09397b4-fabb-4e9d-8695-669bf9f49c77" containerName="barbican-keystone-listener-log" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.313896 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="8c848c79-8799-4e32-a635-958ce73341c4" containerName="proxy-server" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.313906 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="7c36839d-8518-4b0e-bf63-ec94d92e3c13" containerName="container-auditor" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.313915 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="b2a6be10-e7df-494c-be94-a8ede55e2868" containerName="mariadb-account-delete" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.313921 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="e9985bb3-f162-4793-9491-2e4f46644071" containerName="watcher-decision-engine" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.313932 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="7c36839d-8518-4b0e-bf63-ec94d92e3c13" containerName="account-auditor" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.313939 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="3cb4559f-542a-4076-82fb-c0ca88f9de94" containerName="openstack-network-exporter" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.313946 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="7c36839d-8518-4b0e-bf63-ec94d92e3c13" containerName="container-replicator" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.313955 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="7c36839d-8518-4b0e-bf63-ec94d92e3c13" containerName="object-auditor" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.313962 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="d420a2c2-9b7b-400b-88a7-839a93cce573" containerName="mariadb-account-delete" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.313968 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="0291d0e7-9abf-48dd-8d5e-c329f989b700" containerName="cinder-api" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.313979 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="1a97695f-322c-4c4c-834e-ecb2f27c2e97" containerName="mariadb-account-delete" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.313989 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="5e5ddf8c-6109-481f-beb0-0802de034912" containerName="openstack-network-exporter" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.313998 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="eaeff688-fe65-403f-a01a-dfff69db9fb9" containerName="mariadb-account-delete" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.314005 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="7c36839d-8518-4b0e-bf63-ec94d92e3c13" containerName="swift-recon-cron" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.314015 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="d03bf1c0-7353-4836-a767-9a5b2295e8dc" containerName="kube-state-metrics" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.314023 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="01bc82de-61dc-477e-b4cf-e3b0d32c99ba" containerName="thanos-sidecar" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.314033 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="2d2fda50-7118-43f2-8290-36d97b66de86" containerName="probe" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.314040 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="3cb4559f-542a-4076-82fb-c0ca88f9de94" containerName="ovsdbserver-sb" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.314048 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="73f90998-d47e-407c-b964-12065411b4e2" containerName="neutron-api" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.314054 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="7c36839d-8518-4b0e-bf63-ec94d92e3c13" containerName="container-updater" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.314060 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="5785b60f-14d2-43c5-95e2-68741c6729f3" containerName="barbican-api" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.314068 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="2d5669ca-8e5e-49b4-95e9-116eb1195030" containerName="watcher-api-log" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.314076 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="e8de8adf-e4e5-4a10-9722-9106357e67d0" containerName="openstack-network-exporter" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.314084 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="30578d71-fe2b-4eed-9a07-6c8bfe4a1c43" containerName="rabbitmq" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.314094 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="192fa797-c8b8-42d3-a689-e6a7dbc77282" containerName="watcher-applier" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.314104 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="51a9dfff-37f2-4fbe-b2d1-039f20bb4560" containerName="glance-httpd" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.314111 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="7c36839d-8518-4b0e-bf63-ec94d92e3c13" containerName="container-server" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.314118 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="da4dfbea-20a2-4638-bbbe-ed4bfee0ad49" containerName="proxy-httpd" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.314124 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="e8de8adf-e4e5-4a10-9722-9106357e67d0" containerName="ovsdbserver-nb" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.314131 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="c17bae4f-1c1b-4da2-9bb5-c84d6fdfa184" containerName="mariadb-account-delete" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.314140 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="6de7dc2a-f19c-4cc3-9965-9be822dc6270" containerName="glance-httpd" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.314149 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="da4dfbea-20a2-4638-bbbe-ed4bfee0ad49" containerName="ceilometer-central-agent" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.314157 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="0291d0e7-9abf-48dd-8d5e-c329f989b700" containerName="cinder-api-log" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.314164 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="5e5ddf8c-6109-481f-beb0-0802de034912" containerName="ovn-northd" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.314173 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="01bc82de-61dc-477e-b4cf-e3b0d32c99ba" containerName="config-reloader" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.314182 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="7c36839d-8518-4b0e-bf63-ec94d92e3c13" containerName="account-server" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.314188 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="2d5669ca-8e5e-49b4-95e9-116eb1195030" containerName="watcher-api" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.314194 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="0a480cd2-586d-4213-beff-424b3d201f8e" containerName="nova-cell1-conductor-conductor" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.314201 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="da4dfbea-20a2-4638-bbbe-ed4bfee0ad49" containerName="ceilometer-notification-agent" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.314210 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="60ca40b3-79c7-49cc-9100-0a37c47e4462" containerName="placement-log" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.314215 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="51a9dfff-37f2-4fbe-b2d1-039f20bb4560" containerName="glance-log" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.314223 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="1103e975-cc9a-435b-adc9-5c6492e9c629" containerName="nova-api-log" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.314231 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="36225798-caeb-4514-9f51-c8a07fb6da9b" containerName="mariadb-account-delete" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.314237 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="7c36839d-8518-4b0e-bf63-ec94d92e3c13" containerName="account-reaper" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.314244 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="7cbbb6c5-a087-400f-9beb-e740ae1ddfbd" containerName="alertmanager" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.314252 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="60ca40b3-79c7-49cc-9100-0a37c47e4462" containerName="placement-api" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.314261 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="73c1b1fb-5062-4c33-be17-72324f222396" containerName="nova-scheduler-scheduler" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.314267 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="75c87049-f7b9-4c6b-add4-2eec1f52deb2" containerName="galera" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.314275 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="7cbbb6c5-a087-400f-9beb-e740ae1ddfbd" containerName="config-reloader" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.314282 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="1103e975-cc9a-435b-adc9-5c6492e9c629" containerName="nova-api-api" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.314291 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="da4dfbea-20a2-4638-bbbe-ed4bfee0ad49" containerName="sg-core" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.314300 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="98d3c14e-7549-441a-9532-f3d75833cdf8" containerName="nova-metadata-log" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.314308 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="7c36839d-8518-4b0e-bf63-ec94d92e3c13" containerName="object-updater" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.314318 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="2d2fda50-7118-43f2-8290-36d97b66de86" containerName="cinder-scheduler" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.314326 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="5a08cda2-5952-4740-a571-7058f56c3b58" containerName="memcached" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.314332 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="60f75996-dceb-4640-8fca-96fdcc03ebb9" containerName="barbican-worker" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.314338 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="6de7dc2a-f19c-4cc3-9965-9be822dc6270" containerName="glance-log" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.314344 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="7c36839d-8518-4b0e-bf63-ec94d92e3c13" containerName="object-server" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.314352 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="5785b60f-14d2-43c5-95e2-68741c6729f3" containerName="barbican-api-log" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.314362 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="01bc82de-61dc-477e-b4cf-e3b0d32c99ba" containerName="prometheus" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.314371 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="8c848c79-8799-4e32-a635-958ce73341c4" containerName="proxy-httpd" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.314384 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="cf90ea08-b172-4549-9522-92be7a5b5623" containerName="mariadb-account-delete" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.314393 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="6343f4a5-892f-4118-a666-e6601d2df818" containerName="mariadb-account-delete" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.314403 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="7c36839d-8518-4b0e-bf63-ec94d92e3c13" containerName="rsync" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.314415 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="98d3c14e-7549-441a-9532-f3d75833cdf8" containerName="nova-metadata-metadata" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.314423 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="7c36839d-8518-4b0e-bf63-ec94d92e3c13" containerName="object-replicator" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.314436 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="0432c361-154a-46a9-b675-04267061f13b" containerName="keystone-api" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.314445 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="914c0f8f-e5d1-4c63-b985-0bcd07f75374" containerName="nova-cell1-novncproxy-novncproxy" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.314452 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="e09397b4-fabb-4e9d-8695-669bf9f49c77" containerName="barbican-keystone-listener" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.314462 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="60f75996-dceb-4640-8fca-96fdcc03ebb9" containerName="barbican-worker-log" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.314473 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="73f90998-d47e-407c-b964-12065411b4e2" containerName="neutron-httpd" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.314482 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="7c36839d-8518-4b0e-bf63-ec94d92e3c13" containerName="object-expirer" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.314491 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="492ec2b7-0c28-4fe2-b2be-4e9e58423f23" containerName="mariadb-account-delete" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.315259 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="crc-storage/crc-storage-crc-tgd7h" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.318608 5052 reflector.go:368] Caches populated for *v1.Secret from object-"crc-storage"/"crc-storage-dockercfg-mf76g" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.319453 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"crc-storage"/"kube-root-ca.crt" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.319655 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"crc-storage"/"openshift-service-ca.crt" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.319770 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"crc-storage"/"crc-storage" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.321914 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["crc-storage/crc-storage-crc-tgd7h"] Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.401115 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/dda41e3b-b187-42e8-af67-b477f29a4c25-node-mnt\") pod \"crc-storage-crc-tgd7h\" (UID: \"dda41e3b-b187-42e8-af67-b477f29a4c25\") " pod="crc-storage/crc-storage-crc-tgd7h" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.401164 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/dda41e3b-b187-42e8-af67-b477f29a4c25-crc-storage\") pod \"crc-storage-crc-tgd7h\" (UID: \"dda41e3b-b187-42e8-af67-b477f29a4c25\") " pod="crc-storage/crc-storage-crc-tgd7h" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.401244 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hglt2\" (UniqueName: \"kubernetes.io/projected/dda41e3b-b187-42e8-af67-b477f29a4c25-kube-api-access-hglt2\") pod \"crc-storage-crc-tgd7h\" (UID: \"dda41e3b-b187-42e8-af67-b477f29a4c25\") " pod="crc-storage/crc-storage-crc-tgd7h" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.502405 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-hglt2\" (UniqueName: \"kubernetes.io/projected/dda41e3b-b187-42e8-af67-b477f29a4c25-kube-api-access-hglt2\") pod \"crc-storage-crc-tgd7h\" (UID: \"dda41e3b-b187-42e8-af67-b477f29a4c25\") " pod="crc-storage/crc-storage-crc-tgd7h" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.502491 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/dda41e3b-b187-42e8-af67-b477f29a4c25-node-mnt\") pod \"crc-storage-crc-tgd7h\" (UID: \"dda41e3b-b187-42e8-af67-b477f29a4c25\") " pod="crc-storage/crc-storage-crc-tgd7h" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.502546 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/dda41e3b-b187-42e8-af67-b477f29a4c25-crc-storage\") pod \"crc-storage-crc-tgd7h\" (UID: \"dda41e3b-b187-42e8-af67-b477f29a4c25\") " pod="crc-storage/crc-storage-crc-tgd7h" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.502809 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/dda41e3b-b187-42e8-af67-b477f29a4c25-node-mnt\") pod \"crc-storage-crc-tgd7h\" (UID: \"dda41e3b-b187-42e8-af67-b477f29a4c25\") " pod="crc-storage/crc-storage-crc-tgd7h" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.503309 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/dda41e3b-b187-42e8-af67-b477f29a4c25-crc-storage\") pod \"crc-storage-crc-tgd7h\" (UID: \"dda41e3b-b187-42e8-af67-b477f29a4c25\") " pod="crc-storage/crc-storage-crc-tgd7h" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.526947 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-hglt2\" (UniqueName: \"kubernetes.io/projected/dda41e3b-b187-42e8-af67-b477f29a4c25-kube-api-access-hglt2\") pod \"crc-storage-crc-tgd7h\" (UID: \"dda41e3b-b187-42e8-af67-b477f29a4c25\") " pod="crc-storage/crc-storage-crc-tgd7h" Dec 03 12:01:54 crc kubenswrapper[5052]: I1203 12:01:54.634254 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="crc-storage/crc-storage-crc-tgd7h" Dec 03 12:01:55 crc kubenswrapper[5052]: I1203 12:01:55.083166 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["crc-storage/crc-storage-crc-tgd7h"] Dec 03 12:01:55 crc kubenswrapper[5052]: I1203 12:01:55.334494 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="crc-storage/crc-storage-crc-tgd7h" event={"ID":"dda41e3b-b187-42e8-af67-b477f29a4c25","Type":"ContainerStarted","Data":"662fdeb4f08ea1a0751a936fb73ac3b43406e840daff617af09a7aff144b7dea"} Dec 03 12:01:55 crc kubenswrapper[5052]: I1203 12:01:55.683059 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bbd55b7d-3f80-409a-877f-2bf99918eafc" path="/var/lib/kubelet/pods/bbd55b7d-3f80-409a-877f-2bf99918eafc/volumes" Dec 03 12:01:56 crc kubenswrapper[5052]: I1203 12:01:56.345320 5052 generic.go:334] "Generic (PLEG): container finished" podID="dda41e3b-b187-42e8-af67-b477f29a4c25" containerID="1b59ddf16b90baa2fbbd6151a5b6d8907aeddafbb8a0245c485597f1baf52aef" exitCode=0 Dec 03 12:01:56 crc kubenswrapper[5052]: I1203 12:01:56.345703 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="crc-storage/crc-storage-crc-tgd7h" event={"ID":"dda41e3b-b187-42e8-af67-b477f29a4c25","Type":"ContainerDied","Data":"1b59ddf16b90baa2fbbd6151a5b6d8907aeddafbb8a0245c485597f1baf52aef"} Dec 03 12:01:56 crc kubenswrapper[5052]: I1203 12:01:56.672931 5052 scope.go:117] "RemoveContainer" containerID="d224717c6d9ca8135fd510e03a6c40c9f2b1a56dd8b4a3b685af3e2dfa855982" Dec 03 12:01:56 crc kubenswrapper[5052]: E1203 12:01:56.673205 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 12:01:57 crc kubenswrapper[5052]: I1203 12:01:57.640902 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="crc-storage/crc-storage-crc-tgd7h" Dec 03 12:01:57 crc kubenswrapper[5052]: I1203 12:01:57.751130 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/dda41e3b-b187-42e8-af67-b477f29a4c25-crc-storage\") pod \"dda41e3b-b187-42e8-af67-b477f29a4c25\" (UID: \"dda41e3b-b187-42e8-af67-b477f29a4c25\") " Dec 03 12:01:57 crc kubenswrapper[5052]: I1203 12:01:57.751274 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hglt2\" (UniqueName: \"kubernetes.io/projected/dda41e3b-b187-42e8-af67-b477f29a4c25-kube-api-access-hglt2\") pod \"dda41e3b-b187-42e8-af67-b477f29a4c25\" (UID: \"dda41e3b-b187-42e8-af67-b477f29a4c25\") " Dec 03 12:01:57 crc kubenswrapper[5052]: I1203 12:01:57.751440 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/dda41e3b-b187-42e8-af67-b477f29a4c25-node-mnt\") pod \"dda41e3b-b187-42e8-af67-b477f29a4c25\" (UID: \"dda41e3b-b187-42e8-af67-b477f29a4c25\") " Dec 03 12:01:57 crc kubenswrapper[5052]: I1203 12:01:57.751684 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/dda41e3b-b187-42e8-af67-b477f29a4c25-node-mnt" (OuterVolumeSpecName: "node-mnt") pod "dda41e3b-b187-42e8-af67-b477f29a4c25" (UID: "dda41e3b-b187-42e8-af67-b477f29a4c25"). InnerVolumeSpecName "node-mnt". PluginName "kubernetes.io/host-path", VolumeGidValue "" Dec 03 12:01:57 crc kubenswrapper[5052]: I1203 12:01:57.753028 5052 reconciler_common.go:293] "Volume detached for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/dda41e3b-b187-42e8-af67-b477f29a4c25-node-mnt\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:57 crc kubenswrapper[5052]: I1203 12:01:57.764671 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dda41e3b-b187-42e8-af67-b477f29a4c25-kube-api-access-hglt2" (OuterVolumeSpecName: "kube-api-access-hglt2") pod "dda41e3b-b187-42e8-af67-b477f29a4c25" (UID: "dda41e3b-b187-42e8-af67-b477f29a4c25"). InnerVolumeSpecName "kube-api-access-hglt2". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:01:57 crc kubenswrapper[5052]: I1203 12:01:57.772185 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dda41e3b-b187-42e8-af67-b477f29a4c25-crc-storage" (OuterVolumeSpecName: "crc-storage") pod "dda41e3b-b187-42e8-af67-b477f29a4c25" (UID: "dda41e3b-b187-42e8-af67-b477f29a4c25"). InnerVolumeSpecName "crc-storage". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:01:57 crc kubenswrapper[5052]: I1203 12:01:57.854826 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-hglt2\" (UniqueName: \"kubernetes.io/projected/dda41e3b-b187-42e8-af67-b477f29a4c25-kube-api-access-hglt2\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:57 crc kubenswrapper[5052]: I1203 12:01:57.854867 5052 reconciler_common.go:293] "Volume detached for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/dda41e3b-b187-42e8-af67-b477f29a4c25-crc-storage\") on node \"crc\" DevicePath \"\"" Dec 03 12:01:58 crc kubenswrapper[5052]: I1203 12:01:58.366076 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="crc-storage/crc-storage-crc-tgd7h" event={"ID":"dda41e3b-b187-42e8-af67-b477f29a4c25","Type":"ContainerDied","Data":"662fdeb4f08ea1a0751a936fb73ac3b43406e840daff617af09a7aff144b7dea"} Dec 03 12:01:58 crc kubenswrapper[5052]: I1203 12:01:58.366131 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="662fdeb4f08ea1a0751a936fb73ac3b43406e840daff617af09a7aff144b7dea" Dec 03 12:01:58 crc kubenswrapper[5052]: I1203 12:01:58.366167 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="crc-storage/crc-storage-crc-tgd7h" Dec 03 12:02:01 crc kubenswrapper[5052]: I1203 12:02:01.148175 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["crc-storage/crc-storage-crc-tgd7h"] Dec 03 12:02:01 crc kubenswrapper[5052]: I1203 12:02:01.157137 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["crc-storage/crc-storage-crc-tgd7h"] Dec 03 12:02:01 crc kubenswrapper[5052]: I1203 12:02:01.289040 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["crc-storage/crc-storage-crc-zhb4j"] Dec 03 12:02:01 crc kubenswrapper[5052]: E1203 12:02:01.289470 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="dda41e3b-b187-42e8-af67-b477f29a4c25" containerName="storage" Dec 03 12:02:01 crc kubenswrapper[5052]: I1203 12:02:01.289485 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="dda41e3b-b187-42e8-af67-b477f29a4c25" containerName="storage" Dec 03 12:02:01 crc kubenswrapper[5052]: I1203 12:02:01.289696 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="dda41e3b-b187-42e8-af67-b477f29a4c25" containerName="storage" Dec 03 12:02:01 crc kubenswrapper[5052]: I1203 12:02:01.290390 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="crc-storage/crc-storage-crc-zhb4j" Dec 03 12:02:01 crc kubenswrapper[5052]: I1203 12:02:01.293559 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"crc-storage"/"crc-storage" Dec 03 12:02:01 crc kubenswrapper[5052]: I1203 12:02:01.293758 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"crc-storage"/"kube-root-ca.crt" Dec 03 12:02:01 crc kubenswrapper[5052]: I1203 12:02:01.293770 5052 reflector.go:368] Caches populated for *v1.Secret from object-"crc-storage"/"crc-storage-dockercfg-mf76g" Dec 03 12:02:01 crc kubenswrapper[5052]: I1203 12:02:01.293909 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"crc-storage"/"openshift-service-ca.crt" Dec 03 12:02:01 crc kubenswrapper[5052]: I1203 12:02:01.297205 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["crc-storage/crc-storage-crc-zhb4j"] Dec 03 12:02:01 crc kubenswrapper[5052]: I1203 12:02:01.407266 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/cb802b7f-5630-4a20-b23f-7d3f1501f7a8-crc-storage\") pod \"crc-storage-crc-zhb4j\" (UID: \"cb802b7f-5630-4a20-b23f-7d3f1501f7a8\") " pod="crc-storage/crc-storage-crc-zhb4j" Dec 03 12:02:01 crc kubenswrapper[5052]: I1203 12:02:01.407341 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5nfdx\" (UniqueName: \"kubernetes.io/projected/cb802b7f-5630-4a20-b23f-7d3f1501f7a8-kube-api-access-5nfdx\") pod \"crc-storage-crc-zhb4j\" (UID: \"cb802b7f-5630-4a20-b23f-7d3f1501f7a8\") " pod="crc-storage/crc-storage-crc-zhb4j" Dec 03 12:02:01 crc kubenswrapper[5052]: I1203 12:02:01.407498 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/cb802b7f-5630-4a20-b23f-7d3f1501f7a8-node-mnt\") pod \"crc-storage-crc-zhb4j\" (UID: \"cb802b7f-5630-4a20-b23f-7d3f1501f7a8\") " pod="crc-storage/crc-storage-crc-zhb4j" Dec 03 12:02:01 crc kubenswrapper[5052]: I1203 12:02:01.509163 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-5nfdx\" (UniqueName: \"kubernetes.io/projected/cb802b7f-5630-4a20-b23f-7d3f1501f7a8-kube-api-access-5nfdx\") pod \"crc-storage-crc-zhb4j\" (UID: \"cb802b7f-5630-4a20-b23f-7d3f1501f7a8\") " pod="crc-storage/crc-storage-crc-zhb4j" Dec 03 12:02:01 crc kubenswrapper[5052]: I1203 12:02:01.509294 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/cb802b7f-5630-4a20-b23f-7d3f1501f7a8-node-mnt\") pod \"crc-storage-crc-zhb4j\" (UID: \"cb802b7f-5630-4a20-b23f-7d3f1501f7a8\") " pod="crc-storage/crc-storage-crc-zhb4j" Dec 03 12:02:01 crc kubenswrapper[5052]: I1203 12:02:01.509421 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/cb802b7f-5630-4a20-b23f-7d3f1501f7a8-crc-storage\") pod \"crc-storage-crc-zhb4j\" (UID: \"cb802b7f-5630-4a20-b23f-7d3f1501f7a8\") " pod="crc-storage/crc-storage-crc-zhb4j" Dec 03 12:02:01 crc kubenswrapper[5052]: I1203 12:02:01.509805 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/cb802b7f-5630-4a20-b23f-7d3f1501f7a8-node-mnt\") pod \"crc-storage-crc-zhb4j\" (UID: \"cb802b7f-5630-4a20-b23f-7d3f1501f7a8\") " pod="crc-storage/crc-storage-crc-zhb4j" Dec 03 12:02:01 crc kubenswrapper[5052]: I1203 12:02:01.510946 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/cb802b7f-5630-4a20-b23f-7d3f1501f7a8-crc-storage\") pod \"crc-storage-crc-zhb4j\" (UID: \"cb802b7f-5630-4a20-b23f-7d3f1501f7a8\") " pod="crc-storage/crc-storage-crc-zhb4j" Dec 03 12:02:01 crc kubenswrapper[5052]: I1203 12:02:01.530037 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-5nfdx\" (UniqueName: \"kubernetes.io/projected/cb802b7f-5630-4a20-b23f-7d3f1501f7a8-kube-api-access-5nfdx\") pod \"crc-storage-crc-zhb4j\" (UID: \"cb802b7f-5630-4a20-b23f-7d3f1501f7a8\") " pod="crc-storage/crc-storage-crc-zhb4j" Dec 03 12:02:01 crc kubenswrapper[5052]: I1203 12:02:01.615821 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="crc-storage/crc-storage-crc-zhb4j" Dec 03 12:02:01 crc kubenswrapper[5052]: I1203 12:02:01.686755 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dda41e3b-b187-42e8-af67-b477f29a4c25" path="/var/lib/kubelet/pods/dda41e3b-b187-42e8-af67-b477f29a4c25/volumes" Dec 03 12:02:02 crc kubenswrapper[5052]: I1203 12:02:02.020121 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["crc-storage/crc-storage-crc-zhb4j"] Dec 03 12:02:02 crc kubenswrapper[5052]: I1203 12:02:02.405999 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="crc-storage/crc-storage-crc-zhb4j" event={"ID":"cb802b7f-5630-4a20-b23f-7d3f1501f7a8","Type":"ContainerStarted","Data":"6861f19fee68533b7a1f37bdf77d909b3e4025ac6ff95854185fceefbdb934a2"} Dec 03 12:02:04 crc kubenswrapper[5052]: I1203 12:02:04.431572 5052 generic.go:334] "Generic (PLEG): container finished" podID="cb802b7f-5630-4a20-b23f-7d3f1501f7a8" containerID="987e8c8d1466de648f02bf3d76979423cfcca9f6520f4198385ea09c6ae74d6e" exitCode=0 Dec 03 12:02:04 crc kubenswrapper[5052]: I1203 12:02:04.431672 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="crc-storage/crc-storage-crc-zhb4j" event={"ID":"cb802b7f-5630-4a20-b23f-7d3f1501f7a8","Type":"ContainerDied","Data":"987e8c8d1466de648f02bf3d76979423cfcca9f6520f4198385ea09c6ae74d6e"} Dec 03 12:02:05 crc kubenswrapper[5052]: I1203 12:02:05.708749 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="crc-storage/crc-storage-crc-zhb4j" Dec 03 12:02:05 crc kubenswrapper[5052]: I1203 12:02:05.791680 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/cb802b7f-5630-4a20-b23f-7d3f1501f7a8-crc-storage\") pod \"cb802b7f-5630-4a20-b23f-7d3f1501f7a8\" (UID: \"cb802b7f-5630-4a20-b23f-7d3f1501f7a8\") " Dec 03 12:02:05 crc kubenswrapper[5052]: I1203 12:02:05.792013 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5nfdx\" (UniqueName: \"kubernetes.io/projected/cb802b7f-5630-4a20-b23f-7d3f1501f7a8-kube-api-access-5nfdx\") pod \"cb802b7f-5630-4a20-b23f-7d3f1501f7a8\" (UID: \"cb802b7f-5630-4a20-b23f-7d3f1501f7a8\") " Dec 03 12:02:05 crc kubenswrapper[5052]: I1203 12:02:05.792177 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/cb802b7f-5630-4a20-b23f-7d3f1501f7a8-node-mnt\") pod \"cb802b7f-5630-4a20-b23f-7d3f1501f7a8\" (UID: \"cb802b7f-5630-4a20-b23f-7d3f1501f7a8\") " Dec 03 12:02:05 crc kubenswrapper[5052]: I1203 12:02:05.792224 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/cb802b7f-5630-4a20-b23f-7d3f1501f7a8-node-mnt" (OuterVolumeSpecName: "node-mnt") pod "cb802b7f-5630-4a20-b23f-7d3f1501f7a8" (UID: "cb802b7f-5630-4a20-b23f-7d3f1501f7a8"). InnerVolumeSpecName "node-mnt". PluginName "kubernetes.io/host-path", VolumeGidValue "" Dec 03 12:02:05 crc kubenswrapper[5052]: I1203 12:02:05.792551 5052 reconciler_common.go:293] "Volume detached for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/cb802b7f-5630-4a20-b23f-7d3f1501f7a8-node-mnt\") on node \"crc\" DevicePath \"\"" Dec 03 12:02:05 crc kubenswrapper[5052]: I1203 12:02:05.796903 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cb802b7f-5630-4a20-b23f-7d3f1501f7a8-kube-api-access-5nfdx" (OuterVolumeSpecName: "kube-api-access-5nfdx") pod "cb802b7f-5630-4a20-b23f-7d3f1501f7a8" (UID: "cb802b7f-5630-4a20-b23f-7d3f1501f7a8"). InnerVolumeSpecName "kube-api-access-5nfdx". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:02:05 crc kubenswrapper[5052]: I1203 12:02:05.812430 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cb802b7f-5630-4a20-b23f-7d3f1501f7a8-crc-storage" (OuterVolumeSpecName: "crc-storage") pod "cb802b7f-5630-4a20-b23f-7d3f1501f7a8" (UID: "cb802b7f-5630-4a20-b23f-7d3f1501f7a8"). InnerVolumeSpecName "crc-storage". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:02:05 crc kubenswrapper[5052]: I1203 12:02:05.893646 5052 reconciler_common.go:293] "Volume detached for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/cb802b7f-5630-4a20-b23f-7d3f1501f7a8-crc-storage\") on node \"crc\" DevicePath \"\"" Dec 03 12:02:05 crc kubenswrapper[5052]: I1203 12:02:05.893941 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-5nfdx\" (UniqueName: \"kubernetes.io/projected/cb802b7f-5630-4a20-b23f-7d3f1501f7a8-kube-api-access-5nfdx\") on node \"crc\" DevicePath \"\"" Dec 03 12:02:06 crc kubenswrapper[5052]: I1203 12:02:06.458204 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="crc-storage/crc-storage-crc-zhb4j" event={"ID":"cb802b7f-5630-4a20-b23f-7d3f1501f7a8","Type":"ContainerDied","Data":"6861f19fee68533b7a1f37bdf77d909b3e4025ac6ff95854185fceefbdb934a2"} Dec 03 12:02:06 crc kubenswrapper[5052]: I1203 12:02:06.458262 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6861f19fee68533b7a1f37bdf77d909b3e4025ac6ff95854185fceefbdb934a2" Dec 03 12:02:06 crc kubenswrapper[5052]: I1203 12:02:06.458314 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="crc-storage/crc-storage-crc-zhb4j" Dec 03 12:02:10 crc kubenswrapper[5052]: I1203 12:02:10.672992 5052 scope.go:117] "RemoveContainer" containerID="d224717c6d9ca8135fd510e03a6c40c9f2b1a56dd8b4a3b685af3e2dfa855982" Dec 03 12:02:10 crc kubenswrapper[5052]: E1203 12:02:10.674580 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 12:02:21 crc kubenswrapper[5052]: I1203 12:02:21.535400 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/rabbitmq-server-0"] Dec 03 12:02:21 crc kubenswrapper[5052]: E1203 12:02:21.536633 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="cb802b7f-5630-4a20-b23f-7d3f1501f7a8" containerName="storage" Dec 03 12:02:21 crc kubenswrapper[5052]: I1203 12:02:21.536650 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="cb802b7f-5630-4a20-b23f-7d3f1501f7a8" containerName="storage" Dec 03 12:02:21 crc kubenswrapper[5052]: I1203 12:02:21.536834 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="cb802b7f-5630-4a20-b23f-7d3f1501f7a8" containerName="storage" Dec 03 12:02:21 crc kubenswrapper[5052]: I1203 12:02:21.735646 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:02:21 crc kubenswrapper[5052]: I1203 12:02:21.741681 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"rabbitmq-server-dockercfg-t8cdr" Dec 03 12:02:21 crc kubenswrapper[5052]: I1203 12:02:21.754341 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"rabbitmq-config-data" Dec 03 12:02:21 crc kubenswrapper[5052]: I1203 12:02:21.754655 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-rabbitmq-svc" Dec 03 12:02:21 crc kubenswrapper[5052]: I1203 12:02:21.754860 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"rabbitmq-default-user" Dec 03 12:02:21 crc kubenswrapper[5052]: I1203 12:02:21.755052 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"rabbitmq-erlang-cookie" Dec 03 12:02:21 crc kubenswrapper[5052]: I1203 12:02:21.755283 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"rabbitmq-server-conf" Dec 03 12:02:21 crc kubenswrapper[5052]: I1203 12:02:21.789181 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"rabbitmq-plugins-conf" Dec 03 12:02:21 crc kubenswrapper[5052]: I1203 12:02:21.805880 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/rabbitmq-server-0"] Dec 03 12:02:21 crc kubenswrapper[5052]: I1203 12:02:21.808558 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/rabbitmq-server-2"] Dec 03 12:02:21 crc kubenswrapper[5052]: I1203 12:02:21.810257 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/rabbitmq-server-2" Dec 03 12:02:21 crc kubenswrapper[5052]: I1203 12:02:21.819565 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/rabbitmq-server-1"] Dec 03 12:02:21 crc kubenswrapper[5052]: I1203 12:02:21.820893 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/rabbitmq-server-1" Dec 03 12:02:21 crc kubenswrapper[5052]: I1203 12:02:21.840661 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/rabbitmq-server-1"] Dec 03 12:02:21 crc kubenswrapper[5052]: I1203 12:02:21.859629 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/rabbitmq-server-2"] Dec 03 12:02:21 crc kubenswrapper[5052]: I1203 12:02:21.936802 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"rabbitmq-erlang-cookie\" (UniqueName: \"kubernetes.io/empty-dir/2397d896-0d11-4656-a336-eac8ec9fe7f9-rabbitmq-erlang-cookie\") pod \"rabbitmq-server-0\" (UID: \"2397d896-0d11-4656-a336-eac8ec9fe7f9\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:02:21 crc kubenswrapper[5052]: I1203 12:02:21.936844 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"pod-info\" (UniqueName: \"kubernetes.io/downward-api/2397d896-0d11-4656-a336-eac8ec9fe7f9-pod-info\") pod \"rabbitmq-server-0\" (UID: \"2397d896-0d11-4656-a336-eac8ec9fe7f9\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:02:21 crc kubenswrapper[5052]: I1203 12:02:21.936861 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"erlang-cookie-secret\" (UniqueName: \"kubernetes.io/secret/7901ce85-4e6f-4e8b-86a6-f50a85803637-erlang-cookie-secret\") pod \"rabbitmq-server-1\" (UID: \"7901ce85-4e6f-4e8b-86a6-f50a85803637\") " pod="openstack-kuttl-tests/rabbitmq-server-1" Dec 03 12:02:21 crc kubenswrapper[5052]: I1203 12:02:21.936877 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage05-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage05-crc\") pod \"rabbitmq-server-2\" (UID: \"7b4f6078-41a3-433d-acaf-f2d67391ab6e\") " pod="openstack-kuttl-tests/rabbitmq-server-2" Dec 03 12:02:21 crc kubenswrapper[5052]: I1203 12:02:21.936901 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/2397d896-0d11-4656-a336-eac8ec9fe7f9-config-data\") pod \"rabbitmq-server-0\" (UID: \"2397d896-0d11-4656-a336-eac8ec9fe7f9\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:02:21 crc kubenswrapper[5052]: I1203 12:02:21.936917 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"pod-info\" (UniqueName: \"kubernetes.io/downward-api/7901ce85-4e6f-4e8b-86a6-f50a85803637-pod-info\") pod \"rabbitmq-server-1\" (UID: \"7901ce85-4e6f-4e8b-86a6-f50a85803637\") " pod="openstack-kuttl-tests/rabbitmq-server-1" Dec 03 12:02:21 crc kubenswrapper[5052]: I1203 12:02:21.936934 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"rabbitmq-plugins\" (UniqueName: \"kubernetes.io/empty-dir/2397d896-0d11-4656-a336-eac8ec9fe7f9-rabbitmq-plugins\") pod \"rabbitmq-server-0\" (UID: \"2397d896-0d11-4656-a336-eac8ec9fe7f9\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:02:21 crc kubenswrapper[5052]: I1203 12:02:21.936947 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"rabbitmq-tls\" (UniqueName: \"kubernetes.io/projected/7901ce85-4e6f-4e8b-86a6-f50a85803637-rabbitmq-tls\") pod \"rabbitmq-server-1\" (UID: \"7901ce85-4e6f-4e8b-86a6-f50a85803637\") " pod="openstack-kuttl-tests/rabbitmq-server-1" Dec 03 12:02:21 crc kubenswrapper[5052]: I1203 12:02:21.936967 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-conf\" (UniqueName: \"kubernetes.io/configmap/7901ce85-4e6f-4e8b-86a6-f50a85803637-plugins-conf\") pod \"rabbitmq-server-1\" (UID: \"7901ce85-4e6f-4e8b-86a6-f50a85803637\") " pod="openstack-kuttl-tests/rabbitmq-server-1" Dec 03 12:02:21 crc kubenswrapper[5052]: I1203 12:02:21.936987 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"rabbitmq-confd\" (UniqueName: \"kubernetes.io/projected/2397d896-0d11-4656-a336-eac8ec9fe7f9-rabbitmq-confd\") pod \"rabbitmq-server-0\" (UID: \"2397d896-0d11-4656-a336-eac8ec9fe7f9\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:02:21 crc kubenswrapper[5052]: I1203 12:02:21.937002 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4v29h\" (UniqueName: \"kubernetes.io/projected/7901ce85-4e6f-4e8b-86a6-f50a85803637-kube-api-access-4v29h\") pod \"rabbitmq-server-1\" (UID: \"7901ce85-4e6f-4e8b-86a6-f50a85803637\") " pod="openstack-kuttl-tests/rabbitmq-server-1" Dec 03 12:02:21 crc kubenswrapper[5052]: I1203 12:02:21.937027 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"rabbitmq-tls\" (UniqueName: \"kubernetes.io/projected/7b4f6078-41a3-433d-acaf-f2d67391ab6e-rabbitmq-tls\") pod \"rabbitmq-server-2\" (UID: \"7b4f6078-41a3-433d-acaf-f2d67391ab6e\") " pod="openstack-kuttl-tests/rabbitmq-server-2" Dec 03 12:02:21 crc kubenswrapper[5052]: I1203 12:02:21.937053 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"server-conf\" (UniqueName: \"kubernetes.io/configmap/7b4f6078-41a3-433d-acaf-f2d67391ab6e-server-conf\") pod \"rabbitmq-server-2\" (UID: \"7b4f6078-41a3-433d-acaf-f2d67391ab6e\") " pod="openstack-kuttl-tests/rabbitmq-server-2" Dec 03 12:02:21 crc kubenswrapper[5052]: I1203 12:02:21.937075 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8zh7k\" (UniqueName: \"kubernetes.io/projected/7b4f6078-41a3-433d-acaf-f2d67391ab6e-kube-api-access-8zh7k\") pod \"rabbitmq-server-2\" (UID: \"7b4f6078-41a3-433d-acaf-f2d67391ab6e\") " pod="openstack-kuttl-tests/rabbitmq-server-2" Dec 03 12:02:21 crc kubenswrapper[5052]: I1203 12:02:21.937102 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"rabbitmq-tls\" (UniqueName: \"kubernetes.io/projected/2397d896-0d11-4656-a336-eac8ec9fe7f9-rabbitmq-tls\") pod \"rabbitmq-server-0\" (UID: \"2397d896-0d11-4656-a336-eac8ec9fe7f9\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:02:21 crc kubenswrapper[5052]: I1203 12:02:21.937116 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"rabbitmq-confd\" (UniqueName: \"kubernetes.io/projected/7901ce85-4e6f-4e8b-86a6-f50a85803637-rabbitmq-confd\") pod \"rabbitmq-server-1\" (UID: \"7901ce85-4e6f-4e8b-86a6-f50a85803637\") " pod="openstack-kuttl-tests/rabbitmq-server-1" Dec 03 12:02:21 crc kubenswrapper[5052]: I1203 12:02:21.937130 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"server-conf\" (UniqueName: \"kubernetes.io/configmap/2397d896-0d11-4656-a336-eac8ec9fe7f9-server-conf\") pod \"rabbitmq-server-0\" (UID: \"2397d896-0d11-4656-a336-eac8ec9fe7f9\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:02:21 crc kubenswrapper[5052]: I1203 12:02:21.937144 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"rabbitmq-confd\" (UniqueName: \"kubernetes.io/projected/7b4f6078-41a3-433d-acaf-f2d67391ab6e-rabbitmq-confd\") pod \"rabbitmq-server-2\" (UID: \"7b4f6078-41a3-433d-acaf-f2d67391ab6e\") " pod="openstack-kuttl-tests/rabbitmq-server-2" Dec 03 12:02:21 crc kubenswrapper[5052]: I1203 12:02:21.937200 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage02-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage02-crc\") pod \"rabbitmq-server-1\" (UID: \"7901ce85-4e6f-4e8b-86a6-f50a85803637\") " pod="openstack-kuttl-tests/rabbitmq-server-1" Dec 03 12:02:21 crc kubenswrapper[5052]: I1203 12:02:21.937216 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"rabbitmq-plugins\" (UniqueName: \"kubernetes.io/empty-dir/7b4f6078-41a3-433d-acaf-f2d67391ab6e-rabbitmq-plugins\") pod \"rabbitmq-server-2\" (UID: \"7b4f6078-41a3-433d-acaf-f2d67391ab6e\") " pod="openstack-kuttl-tests/rabbitmq-server-2" Dec 03 12:02:21 crc kubenswrapper[5052]: I1203 12:02:21.937231 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-conf\" (UniqueName: \"kubernetes.io/configmap/2397d896-0d11-4656-a336-eac8ec9fe7f9-plugins-conf\") pod \"rabbitmq-server-0\" (UID: \"2397d896-0d11-4656-a336-eac8ec9fe7f9\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:02:21 crc kubenswrapper[5052]: I1203 12:02:21.937244 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/7b4f6078-41a3-433d-acaf-f2d67391ab6e-config-data\") pod \"rabbitmq-server-2\" (UID: \"7b4f6078-41a3-433d-acaf-f2d67391ab6e\") " pod="openstack-kuttl-tests/rabbitmq-server-2" Dec 03 12:02:21 crc kubenswrapper[5052]: I1203 12:02:21.937265 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"rabbitmq-erlang-cookie\" (UniqueName: \"kubernetes.io/empty-dir/7901ce85-4e6f-4e8b-86a6-f50a85803637-rabbitmq-erlang-cookie\") pod \"rabbitmq-server-1\" (UID: \"7901ce85-4e6f-4e8b-86a6-f50a85803637\") " pod="openstack-kuttl-tests/rabbitmq-server-1" Dec 03 12:02:21 crc kubenswrapper[5052]: I1203 12:02:21.937281 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/7901ce85-4e6f-4e8b-86a6-f50a85803637-config-data\") pod \"rabbitmq-server-1\" (UID: \"7901ce85-4e6f-4e8b-86a6-f50a85803637\") " pod="openstack-kuttl-tests/rabbitmq-server-1" Dec 03 12:02:21 crc kubenswrapper[5052]: I1203 12:02:21.937298 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dfcpx\" (UniqueName: \"kubernetes.io/projected/2397d896-0d11-4656-a336-eac8ec9fe7f9-kube-api-access-dfcpx\") pod \"rabbitmq-server-0\" (UID: \"2397d896-0d11-4656-a336-eac8ec9fe7f9\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:02:21 crc kubenswrapper[5052]: I1203 12:02:21.937321 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"pod-info\" (UniqueName: \"kubernetes.io/downward-api/7b4f6078-41a3-433d-acaf-f2d67391ab6e-pod-info\") pod \"rabbitmq-server-2\" (UID: \"7b4f6078-41a3-433d-acaf-f2d67391ab6e\") " pod="openstack-kuttl-tests/rabbitmq-server-2" Dec 03 12:02:21 crc kubenswrapper[5052]: I1203 12:02:21.937337 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"rabbitmq-erlang-cookie\" (UniqueName: \"kubernetes.io/empty-dir/7b4f6078-41a3-433d-acaf-f2d67391ab6e-rabbitmq-erlang-cookie\") pod \"rabbitmq-server-2\" (UID: \"7b4f6078-41a3-433d-acaf-f2d67391ab6e\") " pod="openstack-kuttl-tests/rabbitmq-server-2" Dec 03 12:02:21 crc kubenswrapper[5052]: I1203 12:02:21.937353 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage15-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage15-crc\") pod \"rabbitmq-server-0\" (UID: \"2397d896-0d11-4656-a336-eac8ec9fe7f9\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:02:21 crc kubenswrapper[5052]: I1203 12:02:21.937371 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"erlang-cookie-secret\" (UniqueName: \"kubernetes.io/secret/7b4f6078-41a3-433d-acaf-f2d67391ab6e-erlang-cookie-secret\") pod \"rabbitmq-server-2\" (UID: \"7b4f6078-41a3-433d-acaf-f2d67391ab6e\") " pod="openstack-kuttl-tests/rabbitmq-server-2" Dec 03 12:02:21 crc kubenswrapper[5052]: I1203 12:02:21.937386 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"server-conf\" (UniqueName: \"kubernetes.io/configmap/7901ce85-4e6f-4e8b-86a6-f50a85803637-server-conf\") pod \"rabbitmq-server-1\" (UID: \"7901ce85-4e6f-4e8b-86a6-f50a85803637\") " pod="openstack-kuttl-tests/rabbitmq-server-1" Dec 03 12:02:21 crc kubenswrapper[5052]: I1203 12:02:21.937404 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-conf\" (UniqueName: \"kubernetes.io/configmap/7b4f6078-41a3-433d-acaf-f2d67391ab6e-plugins-conf\") pod \"rabbitmq-server-2\" (UID: \"7b4f6078-41a3-433d-acaf-f2d67391ab6e\") " pod="openstack-kuttl-tests/rabbitmq-server-2" Dec 03 12:02:21 crc kubenswrapper[5052]: I1203 12:02:21.937435 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"rabbitmq-plugins\" (UniqueName: \"kubernetes.io/empty-dir/7901ce85-4e6f-4e8b-86a6-f50a85803637-rabbitmq-plugins\") pod \"rabbitmq-server-1\" (UID: \"7901ce85-4e6f-4e8b-86a6-f50a85803637\") " pod="openstack-kuttl-tests/rabbitmq-server-1" Dec 03 12:02:21 crc kubenswrapper[5052]: I1203 12:02:21.937451 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"erlang-cookie-secret\" (UniqueName: \"kubernetes.io/secret/2397d896-0d11-4656-a336-eac8ec9fe7f9-erlang-cookie-secret\") pod \"rabbitmq-server-0\" (UID: \"2397d896-0d11-4656-a336-eac8ec9fe7f9\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:02:22 crc kubenswrapper[5052]: I1203 12:02:22.039212 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"rabbitmq-tls\" (UniqueName: \"kubernetes.io/projected/2397d896-0d11-4656-a336-eac8ec9fe7f9-rabbitmq-tls\") pod \"rabbitmq-server-0\" (UID: \"2397d896-0d11-4656-a336-eac8ec9fe7f9\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:02:22 crc kubenswrapper[5052]: I1203 12:02:22.039256 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"rabbitmq-confd\" (UniqueName: \"kubernetes.io/projected/7901ce85-4e6f-4e8b-86a6-f50a85803637-rabbitmq-confd\") pod \"rabbitmq-server-1\" (UID: \"7901ce85-4e6f-4e8b-86a6-f50a85803637\") " pod="openstack-kuttl-tests/rabbitmq-server-1" Dec 03 12:02:22 crc kubenswrapper[5052]: I1203 12:02:22.039274 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"server-conf\" (UniqueName: \"kubernetes.io/configmap/2397d896-0d11-4656-a336-eac8ec9fe7f9-server-conf\") pod \"rabbitmq-server-0\" (UID: \"2397d896-0d11-4656-a336-eac8ec9fe7f9\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:02:22 crc kubenswrapper[5052]: I1203 12:02:22.039295 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"rabbitmq-confd\" (UniqueName: \"kubernetes.io/projected/7b4f6078-41a3-433d-acaf-f2d67391ab6e-rabbitmq-confd\") pod \"rabbitmq-server-2\" (UID: \"7b4f6078-41a3-433d-acaf-f2d67391ab6e\") " pod="openstack-kuttl-tests/rabbitmq-server-2" Dec 03 12:02:22 crc kubenswrapper[5052]: I1203 12:02:22.039318 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage02-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage02-crc\") pod \"rabbitmq-server-1\" (UID: \"7901ce85-4e6f-4e8b-86a6-f50a85803637\") " pod="openstack-kuttl-tests/rabbitmq-server-1" Dec 03 12:02:22 crc kubenswrapper[5052]: I1203 12:02:22.039333 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"rabbitmq-plugins\" (UniqueName: \"kubernetes.io/empty-dir/7b4f6078-41a3-433d-acaf-f2d67391ab6e-rabbitmq-plugins\") pod \"rabbitmq-server-2\" (UID: \"7b4f6078-41a3-433d-acaf-f2d67391ab6e\") " pod="openstack-kuttl-tests/rabbitmq-server-2" Dec 03 12:02:22 crc kubenswrapper[5052]: I1203 12:02:22.039352 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"plugins-conf\" (UniqueName: \"kubernetes.io/configmap/2397d896-0d11-4656-a336-eac8ec9fe7f9-plugins-conf\") pod \"rabbitmq-server-0\" (UID: \"2397d896-0d11-4656-a336-eac8ec9fe7f9\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:02:22 crc kubenswrapper[5052]: I1203 12:02:22.039367 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/7b4f6078-41a3-433d-acaf-f2d67391ab6e-config-data\") pod \"rabbitmq-server-2\" (UID: \"7b4f6078-41a3-433d-acaf-f2d67391ab6e\") " pod="openstack-kuttl-tests/rabbitmq-server-2" Dec 03 12:02:22 crc kubenswrapper[5052]: I1203 12:02:22.039390 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"rabbitmq-erlang-cookie\" (UniqueName: \"kubernetes.io/empty-dir/7901ce85-4e6f-4e8b-86a6-f50a85803637-rabbitmq-erlang-cookie\") pod \"rabbitmq-server-1\" (UID: \"7901ce85-4e6f-4e8b-86a6-f50a85803637\") " pod="openstack-kuttl-tests/rabbitmq-server-1" Dec 03 12:02:22 crc kubenswrapper[5052]: I1203 12:02:22.039410 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/7901ce85-4e6f-4e8b-86a6-f50a85803637-config-data\") pod \"rabbitmq-server-1\" (UID: \"7901ce85-4e6f-4e8b-86a6-f50a85803637\") " pod="openstack-kuttl-tests/rabbitmq-server-1" Dec 03 12:02:22 crc kubenswrapper[5052]: I1203 12:02:22.039435 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-dfcpx\" (UniqueName: \"kubernetes.io/projected/2397d896-0d11-4656-a336-eac8ec9fe7f9-kube-api-access-dfcpx\") pod \"rabbitmq-server-0\" (UID: \"2397d896-0d11-4656-a336-eac8ec9fe7f9\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:02:22 crc kubenswrapper[5052]: I1203 12:02:22.039463 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"pod-info\" (UniqueName: \"kubernetes.io/downward-api/7b4f6078-41a3-433d-acaf-f2d67391ab6e-pod-info\") pod \"rabbitmq-server-2\" (UID: \"7b4f6078-41a3-433d-acaf-f2d67391ab6e\") " pod="openstack-kuttl-tests/rabbitmq-server-2" Dec 03 12:02:22 crc kubenswrapper[5052]: I1203 12:02:22.039481 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"rabbitmq-erlang-cookie\" (UniqueName: \"kubernetes.io/empty-dir/7b4f6078-41a3-433d-acaf-f2d67391ab6e-rabbitmq-erlang-cookie\") pod \"rabbitmq-server-2\" (UID: \"7b4f6078-41a3-433d-acaf-f2d67391ab6e\") " pod="openstack-kuttl-tests/rabbitmq-server-2" Dec 03 12:02:22 crc kubenswrapper[5052]: I1203 12:02:22.039498 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage15-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage15-crc\") pod \"rabbitmq-server-0\" (UID: \"2397d896-0d11-4656-a336-eac8ec9fe7f9\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:02:22 crc kubenswrapper[5052]: I1203 12:02:22.039539 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"erlang-cookie-secret\" (UniqueName: \"kubernetes.io/secret/7b4f6078-41a3-433d-acaf-f2d67391ab6e-erlang-cookie-secret\") pod \"rabbitmq-server-2\" (UID: \"7b4f6078-41a3-433d-acaf-f2d67391ab6e\") " pod="openstack-kuttl-tests/rabbitmq-server-2" Dec 03 12:02:22 crc kubenswrapper[5052]: I1203 12:02:22.039559 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"server-conf\" (UniqueName: \"kubernetes.io/configmap/7901ce85-4e6f-4e8b-86a6-f50a85803637-server-conf\") pod \"rabbitmq-server-1\" (UID: \"7901ce85-4e6f-4e8b-86a6-f50a85803637\") " pod="openstack-kuttl-tests/rabbitmq-server-1" Dec 03 12:02:22 crc kubenswrapper[5052]: I1203 12:02:22.039580 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"plugins-conf\" (UniqueName: \"kubernetes.io/configmap/7b4f6078-41a3-433d-acaf-f2d67391ab6e-plugins-conf\") pod \"rabbitmq-server-2\" (UID: \"7b4f6078-41a3-433d-acaf-f2d67391ab6e\") " pod="openstack-kuttl-tests/rabbitmq-server-2" Dec 03 12:02:22 crc kubenswrapper[5052]: I1203 12:02:22.039608 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"rabbitmq-plugins\" (UniqueName: \"kubernetes.io/empty-dir/7901ce85-4e6f-4e8b-86a6-f50a85803637-rabbitmq-plugins\") pod \"rabbitmq-server-1\" (UID: \"7901ce85-4e6f-4e8b-86a6-f50a85803637\") " pod="openstack-kuttl-tests/rabbitmq-server-1" Dec 03 12:02:22 crc kubenswrapper[5052]: I1203 12:02:22.039624 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"erlang-cookie-secret\" (UniqueName: \"kubernetes.io/secret/2397d896-0d11-4656-a336-eac8ec9fe7f9-erlang-cookie-secret\") pod \"rabbitmq-server-0\" (UID: \"2397d896-0d11-4656-a336-eac8ec9fe7f9\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:02:22 crc kubenswrapper[5052]: I1203 12:02:22.039645 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"rabbitmq-erlang-cookie\" (UniqueName: \"kubernetes.io/empty-dir/2397d896-0d11-4656-a336-eac8ec9fe7f9-rabbitmq-erlang-cookie\") pod \"rabbitmq-server-0\" (UID: \"2397d896-0d11-4656-a336-eac8ec9fe7f9\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:02:22 crc kubenswrapper[5052]: I1203 12:02:22.039662 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"pod-info\" (UniqueName: \"kubernetes.io/downward-api/2397d896-0d11-4656-a336-eac8ec9fe7f9-pod-info\") pod \"rabbitmq-server-0\" (UID: \"2397d896-0d11-4656-a336-eac8ec9fe7f9\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:02:22 crc kubenswrapper[5052]: I1203 12:02:22.039676 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"erlang-cookie-secret\" (UniqueName: \"kubernetes.io/secret/7901ce85-4e6f-4e8b-86a6-f50a85803637-erlang-cookie-secret\") pod \"rabbitmq-server-1\" (UID: \"7901ce85-4e6f-4e8b-86a6-f50a85803637\") " pod="openstack-kuttl-tests/rabbitmq-server-1" Dec 03 12:02:22 crc kubenswrapper[5052]: I1203 12:02:22.039690 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage05-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage05-crc\") pod \"rabbitmq-server-2\" (UID: \"7b4f6078-41a3-433d-acaf-f2d67391ab6e\") " pod="openstack-kuttl-tests/rabbitmq-server-2" Dec 03 12:02:22 crc kubenswrapper[5052]: I1203 12:02:22.039713 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/2397d896-0d11-4656-a336-eac8ec9fe7f9-config-data\") pod \"rabbitmq-server-0\" (UID: \"2397d896-0d11-4656-a336-eac8ec9fe7f9\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:02:22 crc kubenswrapper[5052]: I1203 12:02:22.039730 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"pod-info\" (UniqueName: \"kubernetes.io/downward-api/7901ce85-4e6f-4e8b-86a6-f50a85803637-pod-info\") pod \"rabbitmq-server-1\" (UID: \"7901ce85-4e6f-4e8b-86a6-f50a85803637\") " pod="openstack-kuttl-tests/rabbitmq-server-1" Dec 03 12:02:22 crc kubenswrapper[5052]: I1203 12:02:22.039749 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"rabbitmq-plugins\" (UniqueName: \"kubernetes.io/empty-dir/2397d896-0d11-4656-a336-eac8ec9fe7f9-rabbitmq-plugins\") pod \"rabbitmq-server-0\" (UID: \"2397d896-0d11-4656-a336-eac8ec9fe7f9\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:02:22 crc kubenswrapper[5052]: I1203 12:02:22.039762 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"rabbitmq-tls\" (UniqueName: \"kubernetes.io/projected/7901ce85-4e6f-4e8b-86a6-f50a85803637-rabbitmq-tls\") pod \"rabbitmq-server-1\" (UID: \"7901ce85-4e6f-4e8b-86a6-f50a85803637\") " pod="openstack-kuttl-tests/rabbitmq-server-1" Dec 03 12:02:22 crc kubenswrapper[5052]: I1203 12:02:22.039785 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"plugins-conf\" (UniqueName: \"kubernetes.io/configmap/7901ce85-4e6f-4e8b-86a6-f50a85803637-plugins-conf\") pod \"rabbitmq-server-1\" (UID: \"7901ce85-4e6f-4e8b-86a6-f50a85803637\") " pod="openstack-kuttl-tests/rabbitmq-server-1" Dec 03 12:02:22 crc kubenswrapper[5052]: I1203 12:02:22.039805 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"rabbitmq-confd\" (UniqueName: \"kubernetes.io/projected/2397d896-0d11-4656-a336-eac8ec9fe7f9-rabbitmq-confd\") pod \"rabbitmq-server-0\" (UID: \"2397d896-0d11-4656-a336-eac8ec9fe7f9\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:02:22 crc kubenswrapper[5052]: I1203 12:02:22.039820 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-4v29h\" (UniqueName: \"kubernetes.io/projected/7901ce85-4e6f-4e8b-86a6-f50a85803637-kube-api-access-4v29h\") pod \"rabbitmq-server-1\" (UID: \"7901ce85-4e6f-4e8b-86a6-f50a85803637\") " pod="openstack-kuttl-tests/rabbitmq-server-1" Dec 03 12:02:22 crc kubenswrapper[5052]: I1203 12:02:22.039837 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"server-conf\" (UniqueName: \"kubernetes.io/configmap/7b4f6078-41a3-433d-acaf-f2d67391ab6e-server-conf\") pod \"rabbitmq-server-2\" (UID: \"7b4f6078-41a3-433d-acaf-f2d67391ab6e\") " pod="openstack-kuttl-tests/rabbitmq-server-2" Dec 03 12:02:22 crc kubenswrapper[5052]: I1203 12:02:22.039852 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"rabbitmq-tls\" (UniqueName: \"kubernetes.io/projected/7b4f6078-41a3-433d-acaf-f2d67391ab6e-rabbitmq-tls\") pod \"rabbitmq-server-2\" (UID: \"7b4f6078-41a3-433d-acaf-f2d67391ab6e\") " pod="openstack-kuttl-tests/rabbitmq-server-2" Dec 03 12:02:22 crc kubenswrapper[5052]: I1203 12:02:22.039869 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-8zh7k\" (UniqueName: \"kubernetes.io/projected/7b4f6078-41a3-433d-acaf-f2d67391ab6e-kube-api-access-8zh7k\") pod \"rabbitmq-server-2\" (UID: \"7b4f6078-41a3-433d-acaf-f2d67391ab6e\") " pod="openstack-kuttl-tests/rabbitmq-server-2" Dec 03 12:02:22 crc kubenswrapper[5052]: I1203 12:02:22.040720 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"server-conf\" (UniqueName: \"kubernetes.io/configmap/2397d896-0d11-4656-a336-eac8ec9fe7f9-server-conf\") pod \"rabbitmq-server-0\" (UID: \"2397d896-0d11-4656-a336-eac8ec9fe7f9\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:02:22 crc kubenswrapper[5052]: I1203 12:02:22.040893 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage15-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage15-crc\") pod \"rabbitmq-server-0\" (UID: \"2397d896-0d11-4656-a336-eac8ec9fe7f9\") device mount path \"/mnt/openstack/pv15\"" pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:02:22 crc kubenswrapper[5052]: I1203 12:02:22.041029 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage02-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage02-crc\") pod \"rabbitmq-server-1\" (UID: \"7901ce85-4e6f-4e8b-86a6-f50a85803637\") device mount path \"/mnt/openstack/pv02\"" pod="openstack-kuttl-tests/rabbitmq-server-1" Dec 03 12:02:22 crc kubenswrapper[5052]: I1203 12:02:22.041231 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"rabbitmq-plugins\" (UniqueName: \"kubernetes.io/empty-dir/7901ce85-4e6f-4e8b-86a6-f50a85803637-rabbitmq-plugins\") pod \"rabbitmq-server-1\" (UID: \"7901ce85-4e6f-4e8b-86a6-f50a85803637\") " pod="openstack-kuttl-tests/rabbitmq-server-1" Dec 03 12:02:22 crc kubenswrapper[5052]: I1203 12:02:22.045968 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"rabbitmq-confd\" (UniqueName: \"kubernetes.io/projected/7b4f6078-41a3-433d-acaf-f2d67391ab6e-rabbitmq-confd\") pod \"rabbitmq-server-2\" (UID: \"7b4f6078-41a3-433d-acaf-f2d67391ab6e\") " pod="openstack-kuttl-tests/rabbitmq-server-2" Dec 03 12:02:22 crc kubenswrapper[5052]: I1203 12:02:22.046055 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"rabbitmq-confd\" (UniqueName: \"kubernetes.io/projected/7901ce85-4e6f-4e8b-86a6-f50a85803637-rabbitmq-confd\") pod \"rabbitmq-server-1\" (UID: \"7901ce85-4e6f-4e8b-86a6-f50a85803637\") " pod="openstack-kuttl-tests/rabbitmq-server-1" Dec 03 12:02:22 crc kubenswrapper[5052]: I1203 12:02:22.046591 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"rabbitmq-plugins\" (UniqueName: \"kubernetes.io/empty-dir/2397d896-0d11-4656-a336-eac8ec9fe7f9-rabbitmq-plugins\") pod \"rabbitmq-server-0\" (UID: \"2397d896-0d11-4656-a336-eac8ec9fe7f9\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:02:22 crc kubenswrapper[5052]: I1203 12:02:22.049474 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"pod-info\" (UniqueName: \"kubernetes.io/downward-api/7901ce85-4e6f-4e8b-86a6-f50a85803637-pod-info\") pod \"rabbitmq-server-1\" (UID: \"7901ce85-4e6f-4e8b-86a6-f50a85803637\") " pod="openstack-kuttl-tests/rabbitmq-server-1" Dec 03 12:02:22 crc kubenswrapper[5052]: I1203 12:02:22.050299 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"rabbitmq-tls\" (UniqueName: \"kubernetes.io/projected/2397d896-0d11-4656-a336-eac8ec9fe7f9-rabbitmq-tls\") pod \"rabbitmq-server-0\" (UID: \"2397d896-0d11-4656-a336-eac8ec9fe7f9\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:02:22 crc kubenswrapper[5052]: I1203 12:02:22.050743 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"pod-info\" (UniqueName: \"kubernetes.io/downward-api/7b4f6078-41a3-433d-acaf-f2d67391ab6e-pod-info\") pod \"rabbitmq-server-2\" (UID: \"7b4f6078-41a3-433d-acaf-f2d67391ab6e\") " pod="openstack-kuttl-tests/rabbitmq-server-2" Dec 03 12:02:22 crc kubenswrapper[5052]: I1203 12:02:22.051010 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"erlang-cookie-secret\" (UniqueName: \"kubernetes.io/secret/7b4f6078-41a3-433d-acaf-f2d67391ab6e-erlang-cookie-secret\") pod \"rabbitmq-server-2\" (UID: \"7b4f6078-41a3-433d-acaf-f2d67391ab6e\") " pod="openstack-kuttl-tests/rabbitmq-server-2" Dec 03 12:02:22 crc kubenswrapper[5052]: I1203 12:02:22.051552 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/7b4f6078-41a3-433d-acaf-f2d67391ab6e-config-data\") pod \"rabbitmq-server-2\" (UID: \"7b4f6078-41a3-433d-acaf-f2d67391ab6e\") " pod="openstack-kuttl-tests/rabbitmq-server-2" Dec 03 12:02:22 crc kubenswrapper[5052]: I1203 12:02:22.051635 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"rabbitmq-confd\" (UniqueName: \"kubernetes.io/projected/2397d896-0d11-4656-a336-eac8ec9fe7f9-rabbitmq-confd\") pod \"rabbitmq-server-0\" (UID: \"2397d896-0d11-4656-a336-eac8ec9fe7f9\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:02:22 crc kubenswrapper[5052]: I1203 12:02:22.051643 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"plugins-conf\" (UniqueName: \"kubernetes.io/configmap/7901ce85-4e6f-4e8b-86a6-f50a85803637-plugins-conf\") pod \"rabbitmq-server-1\" (UID: \"7901ce85-4e6f-4e8b-86a6-f50a85803637\") " pod="openstack-kuttl-tests/rabbitmq-server-1" Dec 03 12:02:22 crc kubenswrapper[5052]: I1203 12:02:22.052050 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage05-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage05-crc\") pod \"rabbitmq-server-2\" (UID: \"7b4f6078-41a3-433d-acaf-f2d67391ab6e\") device mount path \"/mnt/openstack/pv05\"" pod="openstack-kuttl-tests/rabbitmq-server-2" Dec 03 12:02:22 crc kubenswrapper[5052]: I1203 12:02:22.052111 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"rabbitmq-plugins\" (UniqueName: \"kubernetes.io/empty-dir/7b4f6078-41a3-433d-acaf-f2d67391ab6e-rabbitmq-plugins\") pod \"rabbitmq-server-2\" (UID: \"7b4f6078-41a3-433d-acaf-f2d67391ab6e\") " pod="openstack-kuttl-tests/rabbitmq-server-2" Dec 03 12:02:22 crc kubenswrapper[5052]: I1203 12:02:22.052143 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"plugins-conf\" (UniqueName: \"kubernetes.io/configmap/2397d896-0d11-4656-a336-eac8ec9fe7f9-plugins-conf\") pod \"rabbitmq-server-0\" (UID: \"2397d896-0d11-4656-a336-eac8ec9fe7f9\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:02:22 crc kubenswrapper[5052]: I1203 12:02:22.052371 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"rabbitmq-erlang-cookie\" (UniqueName: \"kubernetes.io/empty-dir/7901ce85-4e6f-4e8b-86a6-f50a85803637-rabbitmq-erlang-cookie\") pod \"rabbitmq-server-1\" (UID: \"7901ce85-4e6f-4e8b-86a6-f50a85803637\") " pod="openstack-kuttl-tests/rabbitmq-server-1" Dec 03 12:02:22 crc kubenswrapper[5052]: I1203 12:02:22.052846 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"rabbitmq-erlang-cookie\" (UniqueName: \"kubernetes.io/empty-dir/7b4f6078-41a3-433d-acaf-f2d67391ab6e-rabbitmq-erlang-cookie\") pod \"rabbitmq-server-2\" (UID: \"7b4f6078-41a3-433d-acaf-f2d67391ab6e\") " pod="openstack-kuttl-tests/rabbitmq-server-2" Dec 03 12:02:22 crc kubenswrapper[5052]: I1203 12:02:22.052856 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"plugins-conf\" (UniqueName: \"kubernetes.io/configmap/7b4f6078-41a3-433d-acaf-f2d67391ab6e-plugins-conf\") pod \"rabbitmq-server-2\" (UID: \"7b4f6078-41a3-433d-acaf-f2d67391ab6e\") " pod="openstack-kuttl-tests/rabbitmq-server-2" Dec 03 12:02:22 crc kubenswrapper[5052]: I1203 12:02:22.053425 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"server-conf\" (UniqueName: \"kubernetes.io/configmap/7901ce85-4e6f-4e8b-86a6-f50a85803637-server-conf\") pod \"rabbitmq-server-1\" (UID: \"7901ce85-4e6f-4e8b-86a6-f50a85803637\") " pod="openstack-kuttl-tests/rabbitmq-server-1" Dec 03 12:02:22 crc kubenswrapper[5052]: I1203 12:02:22.053938 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"rabbitmq-erlang-cookie\" (UniqueName: \"kubernetes.io/empty-dir/2397d896-0d11-4656-a336-eac8ec9fe7f9-rabbitmq-erlang-cookie\") pod \"rabbitmq-server-0\" (UID: \"2397d896-0d11-4656-a336-eac8ec9fe7f9\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:02:22 crc kubenswrapper[5052]: I1203 12:02:22.054155 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/2397d896-0d11-4656-a336-eac8ec9fe7f9-config-data\") pod \"rabbitmq-server-0\" (UID: \"2397d896-0d11-4656-a336-eac8ec9fe7f9\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:02:22 crc kubenswrapper[5052]: I1203 12:02:22.054623 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"rabbitmq-tls\" (UniqueName: \"kubernetes.io/projected/7901ce85-4e6f-4e8b-86a6-f50a85803637-rabbitmq-tls\") pod \"rabbitmq-server-1\" (UID: \"7901ce85-4e6f-4e8b-86a6-f50a85803637\") " pod="openstack-kuttl-tests/rabbitmq-server-1" Dec 03 12:02:22 crc kubenswrapper[5052]: I1203 12:02:22.054699 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/7901ce85-4e6f-4e8b-86a6-f50a85803637-config-data\") pod \"rabbitmq-server-1\" (UID: \"7901ce85-4e6f-4e8b-86a6-f50a85803637\") " pod="openstack-kuttl-tests/rabbitmq-server-1" Dec 03 12:02:22 crc kubenswrapper[5052]: I1203 12:02:22.055951 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"server-conf\" (UniqueName: \"kubernetes.io/configmap/7b4f6078-41a3-433d-acaf-f2d67391ab6e-server-conf\") pod \"rabbitmq-server-2\" (UID: \"7b4f6078-41a3-433d-acaf-f2d67391ab6e\") " pod="openstack-kuttl-tests/rabbitmq-server-2" Dec 03 12:02:22 crc kubenswrapper[5052]: I1203 12:02:22.057925 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"erlang-cookie-secret\" (UniqueName: \"kubernetes.io/secret/7901ce85-4e6f-4e8b-86a6-f50a85803637-erlang-cookie-secret\") pod \"rabbitmq-server-1\" (UID: \"7901ce85-4e6f-4e8b-86a6-f50a85803637\") " pod="openstack-kuttl-tests/rabbitmq-server-1" Dec 03 12:02:22 crc kubenswrapper[5052]: I1203 12:02:22.061725 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-dfcpx\" (UniqueName: \"kubernetes.io/projected/2397d896-0d11-4656-a336-eac8ec9fe7f9-kube-api-access-dfcpx\") pod \"rabbitmq-server-0\" (UID: \"2397d896-0d11-4656-a336-eac8ec9fe7f9\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:02:22 crc kubenswrapper[5052]: I1203 12:02:22.062167 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"rabbitmq-tls\" (UniqueName: \"kubernetes.io/projected/7b4f6078-41a3-433d-acaf-f2d67391ab6e-rabbitmq-tls\") pod \"rabbitmq-server-2\" (UID: \"7b4f6078-41a3-433d-acaf-f2d67391ab6e\") " pod="openstack-kuttl-tests/rabbitmq-server-2" Dec 03 12:02:22 crc kubenswrapper[5052]: I1203 12:02:22.063738 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"pod-info\" (UniqueName: \"kubernetes.io/downward-api/2397d896-0d11-4656-a336-eac8ec9fe7f9-pod-info\") pod \"rabbitmq-server-0\" (UID: \"2397d896-0d11-4656-a336-eac8ec9fe7f9\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:02:22 crc kubenswrapper[5052]: I1203 12:02:22.063984 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"erlang-cookie-secret\" (UniqueName: \"kubernetes.io/secret/2397d896-0d11-4656-a336-eac8ec9fe7f9-erlang-cookie-secret\") pod \"rabbitmq-server-0\" (UID: \"2397d896-0d11-4656-a336-eac8ec9fe7f9\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:02:22 crc kubenswrapper[5052]: I1203 12:02:22.067242 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-8zh7k\" (UniqueName: \"kubernetes.io/projected/7b4f6078-41a3-433d-acaf-f2d67391ab6e-kube-api-access-8zh7k\") pod \"rabbitmq-server-2\" (UID: \"7b4f6078-41a3-433d-acaf-f2d67391ab6e\") " pod="openstack-kuttl-tests/rabbitmq-server-2" Dec 03 12:02:22 crc kubenswrapper[5052]: I1203 12:02:22.070660 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-4v29h\" (UniqueName: \"kubernetes.io/projected/7901ce85-4e6f-4e8b-86a6-f50a85803637-kube-api-access-4v29h\") pod \"rabbitmq-server-1\" (UID: \"7901ce85-4e6f-4e8b-86a6-f50a85803637\") " pod="openstack-kuttl-tests/rabbitmq-server-1" Dec 03 12:02:22 crc kubenswrapper[5052]: I1203 12:02:22.073869 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage15-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage15-crc\") pod \"rabbitmq-server-0\" (UID: \"2397d896-0d11-4656-a336-eac8ec9fe7f9\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:02:22 crc kubenswrapper[5052]: I1203 12:02:22.076290 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage02-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage02-crc\") pod \"rabbitmq-server-1\" (UID: \"7901ce85-4e6f-4e8b-86a6-f50a85803637\") " pod="openstack-kuttl-tests/rabbitmq-server-1" Dec 03 12:02:22 crc kubenswrapper[5052]: I1203 12:02:22.081191 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage05-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage05-crc\") pod \"rabbitmq-server-2\" (UID: \"7b4f6078-41a3-433d-acaf-f2d67391ab6e\") " pod="openstack-kuttl-tests/rabbitmq-server-2" Dec 03 12:02:22 crc kubenswrapper[5052]: I1203 12:02:22.087235 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:02:22 crc kubenswrapper[5052]: I1203 12:02:22.133582 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/rabbitmq-server-2" Dec 03 12:02:22 crc kubenswrapper[5052]: I1203 12:02:22.144494 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/rabbitmq-server-1" Dec 03 12:02:22 crc kubenswrapper[5052]: I1203 12:02:22.549080 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/rabbitmq-server-0"] Dec 03 12:02:22 crc kubenswrapper[5052]: I1203 12:02:22.618112 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/rabbitmq-server-2"] Dec 03 12:02:22 crc kubenswrapper[5052]: W1203 12:02:22.618252 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b4f6078_41a3_433d_acaf_f2d67391ab6e.slice/crio-0ecd86b4dce73c50418f75d596549890a4e7015146f1f050b2673ef00e329423 WatchSource:0}: Error finding container 0ecd86b4dce73c50418f75d596549890a4e7015146f1f050b2673ef00e329423: Status 404 returned error can't find the container with id 0ecd86b4dce73c50418f75d596549890a4e7015146f1f050b2673ef00e329423 Dec 03 12:02:22 crc kubenswrapper[5052]: I1203 12:02:22.655877 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/rabbitmq-server-1"] Dec 03 12:02:22 crc kubenswrapper[5052]: W1203 12:02:22.665559 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7901ce85_4e6f_4e8b_86a6_f50a85803637.slice/crio-9217e8f13f1d2136b88ccaf97e116bdf86a8e5417a0e7f7f09c3e798776eee44 WatchSource:0}: Error finding container 9217e8f13f1d2136b88ccaf97e116bdf86a8e5417a0e7f7f09c3e798776eee44: Status 404 returned error can't find the container with id 9217e8f13f1d2136b88ccaf97e116bdf86a8e5417a0e7f7f09c3e798776eee44 Dec 03 12:02:22 crc kubenswrapper[5052]: I1203 12:02:22.772998 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/rabbitmq-cell1-server-1"] Dec 03 12:02:22 crc kubenswrapper[5052]: I1203 12:02:22.776051 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/rabbitmq-cell1-server-1" Dec 03 12:02:22 crc kubenswrapper[5052]: I1203 12:02:22.782052 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"rabbitmq-cell1-erlang-cookie" Dec 03 12:02:22 crc kubenswrapper[5052]: I1203 12:02:22.782142 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"rabbitmq-cell1-plugins-conf" Dec 03 12:02:22 crc kubenswrapper[5052]: I1203 12:02:22.782052 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"rabbitmq-cell1-default-user" Dec 03 12:02:22 crc kubenswrapper[5052]: I1203 12:02:22.782175 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-rabbitmq-cell1-svc" Dec 03 12:02:22 crc kubenswrapper[5052]: I1203 12:02:22.782387 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"rabbitmq-cell1-config-data" Dec 03 12:02:22 crc kubenswrapper[5052]: I1203 12:02:22.782441 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"rabbitmq-cell1-server-dockercfg-7h74m" Dec 03 12:02:22 crc kubenswrapper[5052]: I1203 12:02:22.782497 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"rabbitmq-cell1-server-conf" Dec 03 12:02:22 crc kubenswrapper[5052]: I1203 12:02:22.786104 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/rabbitmq-cell1-server-2"] Dec 03 12:02:22 crc kubenswrapper[5052]: I1203 12:02:22.788040 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/rabbitmq-cell1-server-2" Dec 03 12:02:22 crc kubenswrapper[5052]: I1203 12:02:22.795176 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/rabbitmq-cell1-server-0"] Dec 03 12:02:22 crc kubenswrapper[5052]: I1203 12:02:22.796851 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:02:22 crc kubenswrapper[5052]: I1203 12:02:22.809009 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-server-1" event={"ID":"7901ce85-4e6f-4e8b-86a6-f50a85803637","Type":"ContainerStarted","Data":"9217e8f13f1d2136b88ccaf97e116bdf86a8e5417a0e7f7f09c3e798776eee44"} Dec 03 12:02:22 crc kubenswrapper[5052]: I1203 12:02:22.809319 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/rabbitmq-cell1-server-1"] Dec 03 12:02:22 crc kubenswrapper[5052]: I1203 12:02:22.816336 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-server-2" event={"ID":"7b4f6078-41a3-433d-acaf-f2d67391ab6e","Type":"ContainerStarted","Data":"0ecd86b4dce73c50418f75d596549890a4e7015146f1f050b2673ef00e329423"} Dec 03 12:02:22 crc kubenswrapper[5052]: I1203 12:02:22.818978 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-server-0" event={"ID":"2397d896-0d11-4656-a336-eac8ec9fe7f9","Type":"ContainerStarted","Data":"33f0c07e2aab29bf325981bed639c4e20ae10db667a2bc8a838a6c8012d98f79"} Dec 03 12:02:22 crc kubenswrapper[5052]: I1203 12:02:22.844100 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/rabbitmq-cell1-server-0"] Dec 03 12:02:22 crc kubenswrapper[5052]: I1203 12:02:22.851626 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4sns5\" (UniqueName: \"kubernetes.io/projected/f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3-kube-api-access-4sns5\") pod \"rabbitmq-cell1-server-1\" (UID: \"f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-1" Dec 03 12:02:22 crc kubenswrapper[5052]: I1203 12:02:22.851773 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"rabbitmq-erlang-cookie\" (UniqueName: \"kubernetes.io/empty-dir/f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3-rabbitmq-erlang-cookie\") pod \"rabbitmq-cell1-server-1\" (UID: \"f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-1" Dec 03 12:02:22 crc kubenswrapper[5052]: I1203 12:02:22.851816 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"pod-info\" (UniqueName: \"kubernetes.io/downward-api/f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3-pod-info\") pod \"rabbitmq-cell1-server-1\" (UID: \"f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-1" Dec 03 12:02:22 crc kubenswrapper[5052]: I1203 12:02:22.851871 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"rabbitmq-tls\" (UniqueName: \"kubernetes.io/projected/f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3-rabbitmq-tls\") pod \"rabbitmq-cell1-server-1\" (UID: \"f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-1" Dec 03 12:02:22 crc kubenswrapper[5052]: I1203 12:02:22.851944 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"server-conf\" (UniqueName: \"kubernetes.io/configmap/f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3-server-conf\") pod \"rabbitmq-cell1-server-1\" (UID: \"f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-1" Dec 03 12:02:22 crc kubenswrapper[5052]: I1203 12:02:22.852001 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"rabbitmq-confd\" (UniqueName: \"kubernetes.io/projected/f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3-rabbitmq-confd\") pod \"rabbitmq-cell1-server-1\" (UID: \"f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-1" Dec 03 12:02:22 crc kubenswrapper[5052]: I1203 12:02:22.852029 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"erlang-cookie-secret\" (UniqueName: \"kubernetes.io/secret/f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3-erlang-cookie-secret\") pod \"rabbitmq-cell1-server-1\" (UID: \"f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-1" Dec 03 12:02:22 crc kubenswrapper[5052]: I1203 12:02:22.852065 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"rabbitmq-plugins\" (UniqueName: \"kubernetes.io/empty-dir/f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3-rabbitmq-plugins\") pod \"rabbitmq-cell1-server-1\" (UID: \"f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-1" Dec 03 12:02:22 crc kubenswrapper[5052]: I1203 12:02:22.852098 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage16-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage16-crc\") pod \"rabbitmq-cell1-server-1\" (UID: \"f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-1" Dec 03 12:02:22 crc kubenswrapper[5052]: I1203 12:02:22.852126 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-conf\" (UniqueName: \"kubernetes.io/configmap/f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3-plugins-conf\") pod \"rabbitmq-cell1-server-1\" (UID: \"f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-1" Dec 03 12:02:22 crc kubenswrapper[5052]: I1203 12:02:22.852153 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3-config-data\") pod \"rabbitmq-cell1-server-1\" (UID: \"f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-1" Dec 03 12:02:22 crc kubenswrapper[5052]: I1203 12:02:22.855068 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/rabbitmq-cell1-server-2"] Dec 03 12:02:22 crc kubenswrapper[5052]: I1203 12:02:22.953342 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"rabbitmq-plugins\" (UniqueName: \"kubernetes.io/empty-dir/185fdd2a-cd57-45e5-9986-bebd15e5761f-rabbitmq-plugins\") pod \"rabbitmq-cell1-server-0\" (UID: \"185fdd2a-cd57-45e5-9986-bebd15e5761f\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:02:22 crc kubenswrapper[5052]: I1203 12:02:22.953712 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"rabbitmq-tls\" (UniqueName: \"kubernetes.io/projected/f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3-rabbitmq-tls\") pod \"rabbitmq-cell1-server-1\" (UID: \"f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-1" Dec 03 12:02:22 crc kubenswrapper[5052]: I1203 12:02:22.953838 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"rabbitmq-tls\" (UniqueName: \"kubernetes.io/projected/185fdd2a-cd57-45e5-9986-bebd15e5761f-rabbitmq-tls\") pod \"rabbitmq-cell1-server-0\" (UID: \"185fdd2a-cd57-45e5-9986-bebd15e5761f\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:02:22 crc kubenswrapper[5052]: I1203 12:02:22.953928 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"server-conf\" (UniqueName: \"kubernetes.io/configmap/185fdd2a-cd57-45e5-9986-bebd15e5761f-server-conf\") pod \"rabbitmq-cell1-server-0\" (UID: \"185fdd2a-cd57-45e5-9986-bebd15e5761f\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:02:22 crc kubenswrapper[5052]: I1203 12:02:22.954008 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2qnm9\" (UniqueName: \"kubernetes.io/projected/06bfd2db-abe0-4b4d-a110-94a6a1e02211-kube-api-access-2qnm9\") pod \"rabbitmq-cell1-server-2\" (UID: \"06bfd2db-abe0-4b4d-a110-94a6a1e02211\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-2" Dec 03 12:02:22 crc kubenswrapper[5052]: I1203 12:02:22.954105 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"server-conf\" (UniqueName: \"kubernetes.io/configmap/f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3-server-conf\") pod \"rabbitmq-cell1-server-1\" (UID: \"f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-1" Dec 03 12:02:22 crc kubenswrapper[5052]: I1203 12:02:22.954188 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"rabbitmq-confd\" (UniqueName: \"kubernetes.io/projected/185fdd2a-cd57-45e5-9986-bebd15e5761f-rabbitmq-confd\") pod \"rabbitmq-cell1-server-0\" (UID: \"185fdd2a-cd57-45e5-9986-bebd15e5761f\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:02:22 crc kubenswrapper[5052]: I1203 12:02:22.954269 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/06bfd2db-abe0-4b4d-a110-94a6a1e02211-config-data\") pod \"rabbitmq-cell1-server-2\" (UID: \"06bfd2db-abe0-4b4d-a110-94a6a1e02211\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-2" Dec 03 12:02:22 crc kubenswrapper[5052]: I1203 12:02:22.954340 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"rabbitmq-erlang-cookie\" (UniqueName: \"kubernetes.io/empty-dir/06bfd2db-abe0-4b4d-a110-94a6a1e02211-rabbitmq-erlang-cookie\") pod \"rabbitmq-cell1-server-2\" (UID: \"06bfd2db-abe0-4b4d-a110-94a6a1e02211\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-2" Dec 03 12:02:22 crc kubenswrapper[5052]: I1203 12:02:22.954414 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"rabbitmq-plugins\" (UniqueName: \"kubernetes.io/empty-dir/06bfd2db-abe0-4b4d-a110-94a6a1e02211-rabbitmq-plugins\") pod \"rabbitmq-cell1-server-2\" (UID: \"06bfd2db-abe0-4b4d-a110-94a6a1e02211\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-2" Dec 03 12:02:22 crc kubenswrapper[5052]: I1203 12:02:22.954492 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/185fdd2a-cd57-45e5-9986-bebd15e5761f-config-data\") pod \"rabbitmq-cell1-server-0\" (UID: \"185fdd2a-cd57-45e5-9986-bebd15e5761f\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:02:22 crc kubenswrapper[5052]: I1203 12:02:22.954601 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"rabbitmq-tls\" (UniqueName: \"kubernetes.io/projected/06bfd2db-abe0-4b4d-a110-94a6a1e02211-rabbitmq-tls\") pod \"rabbitmq-cell1-server-2\" (UID: \"06bfd2db-abe0-4b4d-a110-94a6a1e02211\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-2" Dec 03 12:02:22 crc kubenswrapper[5052]: I1203 12:02:22.954770 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"rabbitmq-confd\" (UniqueName: \"kubernetes.io/projected/f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3-rabbitmq-confd\") pod \"rabbitmq-cell1-server-1\" (UID: \"f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-1" Dec 03 12:02:22 crc kubenswrapper[5052]: I1203 12:02:22.954851 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-conf\" (UniqueName: \"kubernetes.io/configmap/06bfd2db-abe0-4b4d-a110-94a6a1e02211-plugins-conf\") pod \"rabbitmq-cell1-server-2\" (UID: \"06bfd2db-abe0-4b4d-a110-94a6a1e02211\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-2" Dec 03 12:02:22 crc kubenswrapper[5052]: I1203 12:02:22.954920 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"erlang-cookie-secret\" (UniqueName: \"kubernetes.io/secret/f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3-erlang-cookie-secret\") pod \"rabbitmq-cell1-server-1\" (UID: \"f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-1" Dec 03 12:02:22 crc kubenswrapper[5052]: I1203 12:02:22.954990 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"rabbitmq-plugins\" (UniqueName: \"kubernetes.io/empty-dir/f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3-rabbitmq-plugins\") pod \"rabbitmq-cell1-server-1\" (UID: \"f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-1" Dec 03 12:02:22 crc kubenswrapper[5052]: I1203 12:02:22.955077 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage16-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage16-crc\") pod \"rabbitmq-cell1-server-1\" (UID: \"f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-1" Dec 03 12:02:22 crc kubenswrapper[5052]: I1203 12:02:22.955158 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"plugins-conf\" (UniqueName: \"kubernetes.io/configmap/f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3-plugins-conf\") pod \"rabbitmq-cell1-server-1\" (UID: \"f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-1" Dec 03 12:02:22 crc kubenswrapper[5052]: I1203 12:02:22.955268 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"rabbitmq-erlang-cookie\" (UniqueName: \"kubernetes.io/empty-dir/185fdd2a-cd57-45e5-9986-bebd15e5761f-rabbitmq-erlang-cookie\") pod \"rabbitmq-cell1-server-0\" (UID: \"185fdd2a-cd57-45e5-9986-bebd15e5761f\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:02:22 crc kubenswrapper[5052]: I1203 12:02:22.955411 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3-config-data\") pod \"rabbitmq-cell1-server-1\" (UID: \"f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-1" Dec 03 12:02:22 crc kubenswrapper[5052]: I1203 12:02:22.955528 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p77nm\" (UniqueName: \"kubernetes.io/projected/185fdd2a-cd57-45e5-9986-bebd15e5761f-kube-api-access-p77nm\") pod \"rabbitmq-cell1-server-0\" (UID: \"185fdd2a-cd57-45e5-9986-bebd15e5761f\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:02:22 crc kubenswrapper[5052]: I1203 12:02:22.955654 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"pod-info\" (UniqueName: \"kubernetes.io/downward-api/185fdd2a-cd57-45e5-9986-bebd15e5761f-pod-info\") pod \"rabbitmq-cell1-server-0\" (UID: \"185fdd2a-cd57-45e5-9986-bebd15e5761f\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:02:22 crc kubenswrapper[5052]: I1203 12:02:22.955757 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"pod-info\" (UniqueName: \"kubernetes.io/downward-api/06bfd2db-abe0-4b4d-a110-94a6a1e02211-pod-info\") pod \"rabbitmq-cell1-server-2\" (UID: \"06bfd2db-abe0-4b4d-a110-94a6a1e02211\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-2" Dec 03 12:02:22 crc kubenswrapper[5052]: I1203 12:02:22.955798 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"rabbitmq-plugins\" (UniqueName: \"kubernetes.io/empty-dir/f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3-rabbitmq-plugins\") pod \"rabbitmq-cell1-server-1\" (UID: \"f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-1" Dec 03 12:02:22 crc kubenswrapper[5052]: I1203 12:02:22.955580 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage16-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage16-crc\") pod \"rabbitmq-cell1-server-1\" (UID: \"f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3\") device mount path \"/mnt/openstack/pv16\"" pod="openstack-kuttl-tests/rabbitmq-cell1-server-1" Dec 03 12:02:22 crc kubenswrapper[5052]: I1203 12:02:22.955464 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"server-conf\" (UniqueName: \"kubernetes.io/configmap/f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3-server-conf\") pod \"rabbitmq-cell1-server-1\" (UID: \"f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-1" Dec 03 12:02:22 crc kubenswrapper[5052]: I1203 12:02:22.955858 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"server-conf\" (UniqueName: \"kubernetes.io/configmap/06bfd2db-abe0-4b4d-a110-94a6a1e02211-server-conf\") pod \"rabbitmq-cell1-server-2\" (UID: \"06bfd2db-abe0-4b4d-a110-94a6a1e02211\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-2" Dec 03 12:02:22 crc kubenswrapper[5052]: I1203 12:02:22.956210 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3-config-data\") pod \"rabbitmq-cell1-server-1\" (UID: \"f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-1" Dec 03 12:02:22 crc kubenswrapper[5052]: I1203 12:02:22.956030 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"plugins-conf\" (UniqueName: \"kubernetes.io/configmap/f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3-plugins-conf\") pod \"rabbitmq-cell1-server-1\" (UID: \"f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-1" Dec 03 12:02:22 crc kubenswrapper[5052]: I1203 12:02:22.956351 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage20-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage20-crc\") pod \"rabbitmq-cell1-server-2\" (UID: \"06bfd2db-abe0-4b4d-a110-94a6a1e02211\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-2" Dec 03 12:02:22 crc kubenswrapper[5052]: I1203 12:02:22.956445 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-4sns5\" (UniqueName: \"kubernetes.io/projected/f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3-kube-api-access-4sns5\") pod \"rabbitmq-cell1-server-1\" (UID: \"f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-1" Dec 03 12:02:22 crc kubenswrapper[5052]: I1203 12:02:22.956625 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage18-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage18-crc\") pod \"rabbitmq-cell1-server-0\" (UID: \"185fdd2a-cd57-45e5-9986-bebd15e5761f\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:02:22 crc kubenswrapper[5052]: I1203 12:02:22.956778 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"rabbitmq-confd\" (UniqueName: \"kubernetes.io/projected/06bfd2db-abe0-4b4d-a110-94a6a1e02211-rabbitmq-confd\") pod \"rabbitmq-cell1-server-2\" (UID: \"06bfd2db-abe0-4b4d-a110-94a6a1e02211\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-2" Dec 03 12:02:22 crc kubenswrapper[5052]: I1203 12:02:22.956900 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"rabbitmq-erlang-cookie\" (UniqueName: \"kubernetes.io/empty-dir/f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3-rabbitmq-erlang-cookie\") pod \"rabbitmq-cell1-server-1\" (UID: \"f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-1" Dec 03 12:02:22 crc kubenswrapper[5052]: I1203 12:02:22.957003 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"erlang-cookie-secret\" (UniqueName: \"kubernetes.io/secret/06bfd2db-abe0-4b4d-a110-94a6a1e02211-erlang-cookie-secret\") pod \"rabbitmq-cell1-server-2\" (UID: \"06bfd2db-abe0-4b4d-a110-94a6a1e02211\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-2" Dec 03 12:02:22 crc kubenswrapper[5052]: I1203 12:02:22.956942 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"rabbitmq-tls\" (UniqueName: \"kubernetes.io/projected/f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3-rabbitmq-tls\") pod \"rabbitmq-cell1-server-1\" (UID: \"f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-1" Dec 03 12:02:22 crc kubenswrapper[5052]: I1203 12:02:22.957095 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"pod-info\" (UniqueName: \"kubernetes.io/downward-api/f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3-pod-info\") pod \"rabbitmq-cell1-server-1\" (UID: \"f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-1" Dec 03 12:02:22 crc kubenswrapper[5052]: I1203 12:02:22.957247 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-conf\" (UniqueName: \"kubernetes.io/configmap/185fdd2a-cd57-45e5-9986-bebd15e5761f-plugins-conf\") pod \"rabbitmq-cell1-server-0\" (UID: \"185fdd2a-cd57-45e5-9986-bebd15e5761f\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:02:22 crc kubenswrapper[5052]: I1203 12:02:22.957344 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"erlang-cookie-secret\" (UniqueName: \"kubernetes.io/secret/185fdd2a-cd57-45e5-9986-bebd15e5761f-erlang-cookie-secret\") pod \"rabbitmq-cell1-server-0\" (UID: \"185fdd2a-cd57-45e5-9986-bebd15e5761f\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:02:22 crc kubenswrapper[5052]: I1203 12:02:22.957197 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"rabbitmq-erlang-cookie\" (UniqueName: \"kubernetes.io/empty-dir/f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3-rabbitmq-erlang-cookie\") pod \"rabbitmq-cell1-server-1\" (UID: \"f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-1" Dec 03 12:02:22 crc kubenswrapper[5052]: I1203 12:02:22.957963 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"erlang-cookie-secret\" (UniqueName: \"kubernetes.io/secret/f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3-erlang-cookie-secret\") pod \"rabbitmq-cell1-server-1\" (UID: \"f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-1" Dec 03 12:02:22 crc kubenswrapper[5052]: I1203 12:02:22.958449 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"rabbitmq-confd\" (UniqueName: \"kubernetes.io/projected/f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3-rabbitmq-confd\") pod \"rabbitmq-cell1-server-1\" (UID: \"f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-1" Dec 03 12:02:22 crc kubenswrapper[5052]: I1203 12:02:22.961244 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"pod-info\" (UniqueName: \"kubernetes.io/downward-api/f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3-pod-info\") pod \"rabbitmq-cell1-server-1\" (UID: \"f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-1" Dec 03 12:02:22 crc kubenswrapper[5052]: I1203 12:02:22.971132 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-4sns5\" (UniqueName: \"kubernetes.io/projected/f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3-kube-api-access-4sns5\") pod \"rabbitmq-cell1-server-1\" (UID: \"f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-1" Dec 03 12:02:22 crc kubenswrapper[5052]: I1203 12:02:22.976999 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage16-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage16-crc\") pod \"rabbitmq-cell1-server-1\" (UID: \"f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-1" Dec 03 12:02:23 crc kubenswrapper[5052]: I1203 12:02:23.059481 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-2qnm9\" (UniqueName: \"kubernetes.io/projected/06bfd2db-abe0-4b4d-a110-94a6a1e02211-kube-api-access-2qnm9\") pod \"rabbitmq-cell1-server-2\" (UID: \"06bfd2db-abe0-4b4d-a110-94a6a1e02211\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-2" Dec 03 12:02:23 crc kubenswrapper[5052]: I1203 12:02:23.059581 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"rabbitmq-confd\" (UniqueName: \"kubernetes.io/projected/185fdd2a-cd57-45e5-9986-bebd15e5761f-rabbitmq-confd\") pod \"rabbitmq-cell1-server-0\" (UID: \"185fdd2a-cd57-45e5-9986-bebd15e5761f\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:02:23 crc kubenswrapper[5052]: I1203 12:02:23.059615 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/06bfd2db-abe0-4b4d-a110-94a6a1e02211-config-data\") pod \"rabbitmq-cell1-server-2\" (UID: \"06bfd2db-abe0-4b4d-a110-94a6a1e02211\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-2" Dec 03 12:02:23 crc kubenswrapper[5052]: I1203 12:02:23.059646 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"rabbitmq-erlang-cookie\" (UniqueName: \"kubernetes.io/empty-dir/06bfd2db-abe0-4b4d-a110-94a6a1e02211-rabbitmq-erlang-cookie\") pod \"rabbitmq-cell1-server-2\" (UID: \"06bfd2db-abe0-4b4d-a110-94a6a1e02211\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-2" Dec 03 12:02:23 crc kubenswrapper[5052]: I1203 12:02:23.059671 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"rabbitmq-plugins\" (UniqueName: \"kubernetes.io/empty-dir/06bfd2db-abe0-4b4d-a110-94a6a1e02211-rabbitmq-plugins\") pod \"rabbitmq-cell1-server-2\" (UID: \"06bfd2db-abe0-4b4d-a110-94a6a1e02211\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-2" Dec 03 12:02:23 crc kubenswrapper[5052]: I1203 12:02:23.059705 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/185fdd2a-cd57-45e5-9986-bebd15e5761f-config-data\") pod \"rabbitmq-cell1-server-0\" (UID: \"185fdd2a-cd57-45e5-9986-bebd15e5761f\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:02:23 crc kubenswrapper[5052]: I1203 12:02:23.059734 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"rabbitmq-tls\" (UniqueName: \"kubernetes.io/projected/06bfd2db-abe0-4b4d-a110-94a6a1e02211-rabbitmq-tls\") pod \"rabbitmq-cell1-server-2\" (UID: \"06bfd2db-abe0-4b4d-a110-94a6a1e02211\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-2" Dec 03 12:02:23 crc kubenswrapper[5052]: I1203 12:02:23.059762 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"plugins-conf\" (UniqueName: \"kubernetes.io/configmap/06bfd2db-abe0-4b4d-a110-94a6a1e02211-plugins-conf\") pod \"rabbitmq-cell1-server-2\" (UID: \"06bfd2db-abe0-4b4d-a110-94a6a1e02211\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-2" Dec 03 12:02:23 crc kubenswrapper[5052]: I1203 12:02:23.059802 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"rabbitmq-erlang-cookie\" (UniqueName: \"kubernetes.io/empty-dir/185fdd2a-cd57-45e5-9986-bebd15e5761f-rabbitmq-erlang-cookie\") pod \"rabbitmq-cell1-server-0\" (UID: \"185fdd2a-cd57-45e5-9986-bebd15e5761f\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:02:23 crc kubenswrapper[5052]: I1203 12:02:23.059829 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-p77nm\" (UniqueName: \"kubernetes.io/projected/185fdd2a-cd57-45e5-9986-bebd15e5761f-kube-api-access-p77nm\") pod \"rabbitmq-cell1-server-0\" (UID: \"185fdd2a-cd57-45e5-9986-bebd15e5761f\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:02:23 crc kubenswrapper[5052]: I1203 12:02:23.059859 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"pod-info\" (UniqueName: \"kubernetes.io/downward-api/185fdd2a-cd57-45e5-9986-bebd15e5761f-pod-info\") pod \"rabbitmq-cell1-server-0\" (UID: \"185fdd2a-cd57-45e5-9986-bebd15e5761f\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:02:23 crc kubenswrapper[5052]: I1203 12:02:23.059885 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"pod-info\" (UniqueName: \"kubernetes.io/downward-api/06bfd2db-abe0-4b4d-a110-94a6a1e02211-pod-info\") pod \"rabbitmq-cell1-server-2\" (UID: \"06bfd2db-abe0-4b4d-a110-94a6a1e02211\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-2" Dec 03 12:02:23 crc kubenswrapper[5052]: I1203 12:02:23.059910 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"server-conf\" (UniqueName: \"kubernetes.io/configmap/06bfd2db-abe0-4b4d-a110-94a6a1e02211-server-conf\") pod \"rabbitmq-cell1-server-2\" (UID: \"06bfd2db-abe0-4b4d-a110-94a6a1e02211\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-2" Dec 03 12:02:23 crc kubenswrapper[5052]: I1203 12:02:23.059937 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage20-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage20-crc\") pod \"rabbitmq-cell1-server-2\" (UID: \"06bfd2db-abe0-4b4d-a110-94a6a1e02211\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-2" Dec 03 12:02:23 crc kubenswrapper[5052]: I1203 12:02:23.059976 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage18-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage18-crc\") pod \"rabbitmq-cell1-server-0\" (UID: \"185fdd2a-cd57-45e5-9986-bebd15e5761f\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:02:23 crc kubenswrapper[5052]: I1203 12:02:23.060004 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"rabbitmq-confd\" (UniqueName: \"kubernetes.io/projected/06bfd2db-abe0-4b4d-a110-94a6a1e02211-rabbitmq-confd\") pod \"rabbitmq-cell1-server-2\" (UID: \"06bfd2db-abe0-4b4d-a110-94a6a1e02211\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-2" Dec 03 12:02:23 crc kubenswrapper[5052]: I1203 12:02:23.060042 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"erlang-cookie-secret\" (UniqueName: \"kubernetes.io/secret/06bfd2db-abe0-4b4d-a110-94a6a1e02211-erlang-cookie-secret\") pod \"rabbitmq-cell1-server-2\" (UID: \"06bfd2db-abe0-4b4d-a110-94a6a1e02211\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-2" Dec 03 12:02:23 crc kubenswrapper[5052]: I1203 12:02:23.060065 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"plugins-conf\" (UniqueName: \"kubernetes.io/configmap/185fdd2a-cd57-45e5-9986-bebd15e5761f-plugins-conf\") pod \"rabbitmq-cell1-server-0\" (UID: \"185fdd2a-cd57-45e5-9986-bebd15e5761f\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:02:23 crc kubenswrapper[5052]: I1203 12:02:23.060088 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"erlang-cookie-secret\" (UniqueName: \"kubernetes.io/secret/185fdd2a-cd57-45e5-9986-bebd15e5761f-erlang-cookie-secret\") pod \"rabbitmq-cell1-server-0\" (UID: \"185fdd2a-cd57-45e5-9986-bebd15e5761f\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:02:23 crc kubenswrapper[5052]: I1203 12:02:23.060118 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"rabbitmq-plugins\" (UniqueName: \"kubernetes.io/empty-dir/185fdd2a-cd57-45e5-9986-bebd15e5761f-rabbitmq-plugins\") pod \"rabbitmq-cell1-server-0\" (UID: \"185fdd2a-cd57-45e5-9986-bebd15e5761f\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:02:23 crc kubenswrapper[5052]: I1203 12:02:23.060149 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"rabbitmq-tls\" (UniqueName: \"kubernetes.io/projected/185fdd2a-cd57-45e5-9986-bebd15e5761f-rabbitmq-tls\") pod \"rabbitmq-cell1-server-0\" (UID: \"185fdd2a-cd57-45e5-9986-bebd15e5761f\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:02:23 crc kubenswrapper[5052]: I1203 12:02:23.060180 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"server-conf\" (UniqueName: \"kubernetes.io/configmap/185fdd2a-cd57-45e5-9986-bebd15e5761f-server-conf\") pod \"rabbitmq-cell1-server-0\" (UID: \"185fdd2a-cd57-45e5-9986-bebd15e5761f\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:02:23 crc kubenswrapper[5052]: I1203 12:02:23.062140 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"server-conf\" (UniqueName: \"kubernetes.io/configmap/185fdd2a-cd57-45e5-9986-bebd15e5761f-server-conf\") pod \"rabbitmq-cell1-server-0\" (UID: \"185fdd2a-cd57-45e5-9986-bebd15e5761f\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:02:23 crc kubenswrapper[5052]: I1203 12:02:23.065193 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"plugins-conf\" (UniqueName: \"kubernetes.io/configmap/06bfd2db-abe0-4b4d-a110-94a6a1e02211-plugins-conf\") pod \"rabbitmq-cell1-server-2\" (UID: \"06bfd2db-abe0-4b4d-a110-94a6a1e02211\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-2" Dec 03 12:02:23 crc kubenswrapper[5052]: I1203 12:02:23.065478 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"rabbitmq-erlang-cookie\" (UniqueName: \"kubernetes.io/empty-dir/06bfd2db-abe0-4b4d-a110-94a6a1e02211-rabbitmq-erlang-cookie\") pod \"rabbitmq-cell1-server-2\" (UID: \"06bfd2db-abe0-4b4d-a110-94a6a1e02211\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-2" Dec 03 12:02:23 crc kubenswrapper[5052]: I1203 12:02:23.066063 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/06bfd2db-abe0-4b4d-a110-94a6a1e02211-config-data\") pod \"rabbitmq-cell1-server-2\" (UID: \"06bfd2db-abe0-4b4d-a110-94a6a1e02211\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-2" Dec 03 12:02:23 crc kubenswrapper[5052]: I1203 12:02:23.066242 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"plugins-conf\" (UniqueName: \"kubernetes.io/configmap/185fdd2a-cd57-45e5-9986-bebd15e5761f-plugins-conf\") pod \"rabbitmq-cell1-server-0\" (UID: \"185fdd2a-cd57-45e5-9986-bebd15e5761f\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:02:23 crc kubenswrapper[5052]: I1203 12:02:23.066313 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"rabbitmq-plugins\" (UniqueName: \"kubernetes.io/empty-dir/06bfd2db-abe0-4b4d-a110-94a6a1e02211-rabbitmq-plugins\") pod \"rabbitmq-cell1-server-2\" (UID: \"06bfd2db-abe0-4b4d-a110-94a6a1e02211\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-2" Dec 03 12:02:23 crc kubenswrapper[5052]: I1203 12:02:23.066602 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"rabbitmq-plugins\" (UniqueName: \"kubernetes.io/empty-dir/185fdd2a-cd57-45e5-9986-bebd15e5761f-rabbitmq-plugins\") pod \"rabbitmq-cell1-server-0\" (UID: \"185fdd2a-cd57-45e5-9986-bebd15e5761f\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:02:23 crc kubenswrapper[5052]: I1203 12:02:23.066657 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage20-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage20-crc\") pod \"rabbitmq-cell1-server-2\" (UID: \"06bfd2db-abe0-4b4d-a110-94a6a1e02211\") device mount path \"/mnt/openstack/pv20\"" pod="openstack-kuttl-tests/rabbitmq-cell1-server-2" Dec 03 12:02:23 crc kubenswrapper[5052]: I1203 12:02:23.066734 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage18-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage18-crc\") pod \"rabbitmq-cell1-server-0\" (UID: \"185fdd2a-cd57-45e5-9986-bebd15e5761f\") device mount path \"/mnt/openstack/pv18\"" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:02:23 crc kubenswrapper[5052]: I1203 12:02:23.073820 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"rabbitmq-tls\" (UniqueName: \"kubernetes.io/projected/06bfd2db-abe0-4b4d-a110-94a6a1e02211-rabbitmq-tls\") pod \"rabbitmq-cell1-server-2\" (UID: \"06bfd2db-abe0-4b4d-a110-94a6a1e02211\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-2" Dec 03 12:02:23 crc kubenswrapper[5052]: I1203 12:02:23.067598 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"server-conf\" (UniqueName: \"kubernetes.io/configmap/06bfd2db-abe0-4b4d-a110-94a6a1e02211-server-conf\") pod \"rabbitmq-cell1-server-2\" (UID: \"06bfd2db-abe0-4b4d-a110-94a6a1e02211\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-2" Dec 03 12:02:23 crc kubenswrapper[5052]: I1203 12:02:23.067951 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"rabbitmq-erlang-cookie\" (UniqueName: \"kubernetes.io/empty-dir/185fdd2a-cd57-45e5-9986-bebd15e5761f-rabbitmq-erlang-cookie\") pod \"rabbitmq-cell1-server-0\" (UID: \"185fdd2a-cd57-45e5-9986-bebd15e5761f\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:02:23 crc kubenswrapper[5052]: I1203 12:02:23.071476 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"rabbitmq-confd\" (UniqueName: \"kubernetes.io/projected/185fdd2a-cd57-45e5-9986-bebd15e5761f-rabbitmq-confd\") pod \"rabbitmq-cell1-server-0\" (UID: \"185fdd2a-cd57-45e5-9986-bebd15e5761f\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:02:23 crc kubenswrapper[5052]: I1203 12:02:23.071883 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"erlang-cookie-secret\" (UniqueName: \"kubernetes.io/secret/185fdd2a-cd57-45e5-9986-bebd15e5761f-erlang-cookie-secret\") pod \"rabbitmq-cell1-server-0\" (UID: \"185fdd2a-cd57-45e5-9986-bebd15e5761f\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:02:23 crc kubenswrapper[5052]: I1203 12:02:23.073141 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"pod-info\" (UniqueName: \"kubernetes.io/downward-api/185fdd2a-cd57-45e5-9986-bebd15e5761f-pod-info\") pod \"rabbitmq-cell1-server-0\" (UID: \"185fdd2a-cd57-45e5-9986-bebd15e5761f\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:02:23 crc kubenswrapper[5052]: I1203 12:02:23.073186 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"erlang-cookie-secret\" (UniqueName: \"kubernetes.io/secret/06bfd2db-abe0-4b4d-a110-94a6a1e02211-erlang-cookie-secret\") pod \"rabbitmq-cell1-server-2\" (UID: \"06bfd2db-abe0-4b4d-a110-94a6a1e02211\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-2" Dec 03 12:02:23 crc kubenswrapper[5052]: I1203 12:02:23.067112 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/185fdd2a-cd57-45e5-9986-bebd15e5761f-config-data\") pod \"rabbitmq-cell1-server-0\" (UID: \"185fdd2a-cd57-45e5-9986-bebd15e5761f\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:02:23 crc kubenswrapper[5052]: I1203 12:02:23.074288 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"rabbitmq-tls\" (UniqueName: \"kubernetes.io/projected/185fdd2a-cd57-45e5-9986-bebd15e5761f-rabbitmq-tls\") pod \"rabbitmq-cell1-server-0\" (UID: \"185fdd2a-cd57-45e5-9986-bebd15e5761f\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:02:23 crc kubenswrapper[5052]: I1203 12:02:23.074629 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"pod-info\" (UniqueName: \"kubernetes.io/downward-api/06bfd2db-abe0-4b4d-a110-94a6a1e02211-pod-info\") pod \"rabbitmq-cell1-server-2\" (UID: \"06bfd2db-abe0-4b4d-a110-94a6a1e02211\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-2" Dec 03 12:02:23 crc kubenswrapper[5052]: I1203 12:02:23.074918 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"rabbitmq-confd\" (UniqueName: \"kubernetes.io/projected/06bfd2db-abe0-4b4d-a110-94a6a1e02211-rabbitmq-confd\") pod \"rabbitmq-cell1-server-2\" (UID: \"06bfd2db-abe0-4b4d-a110-94a6a1e02211\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-2" Dec 03 12:02:23 crc kubenswrapper[5052]: I1203 12:02:23.092482 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-p77nm\" (UniqueName: \"kubernetes.io/projected/185fdd2a-cd57-45e5-9986-bebd15e5761f-kube-api-access-p77nm\") pod \"rabbitmq-cell1-server-0\" (UID: \"185fdd2a-cd57-45e5-9986-bebd15e5761f\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:02:23 crc kubenswrapper[5052]: I1203 12:02:23.093238 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-2qnm9\" (UniqueName: \"kubernetes.io/projected/06bfd2db-abe0-4b4d-a110-94a6a1e02211-kube-api-access-2qnm9\") pod \"rabbitmq-cell1-server-2\" (UID: \"06bfd2db-abe0-4b4d-a110-94a6a1e02211\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-2" Dec 03 12:02:23 crc kubenswrapper[5052]: I1203 12:02:23.108648 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/rabbitmq-cell1-server-1" Dec 03 12:02:23 crc kubenswrapper[5052]: I1203 12:02:23.133703 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage20-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage20-crc\") pod \"rabbitmq-cell1-server-2\" (UID: \"06bfd2db-abe0-4b4d-a110-94a6a1e02211\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-2" Dec 03 12:02:23 crc kubenswrapper[5052]: I1203 12:02:23.237493 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage18-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage18-crc\") pod \"rabbitmq-cell1-server-0\" (UID: \"185fdd2a-cd57-45e5-9986-bebd15e5761f\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:02:23 crc kubenswrapper[5052]: I1203 12:02:23.422235 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/rabbitmq-cell1-server-2" Dec 03 12:02:23 crc kubenswrapper[5052]: I1203 12:02:23.434742 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:02:23 crc kubenswrapper[5052]: I1203 12:02:23.664087 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/rabbitmq-cell1-server-1"] Dec 03 12:02:23 crc kubenswrapper[5052]: I1203 12:02:23.678138 5052 scope.go:117] "RemoveContainer" containerID="d224717c6d9ca8135fd510e03a6c40c9f2b1a56dd8b4a3b685af3e2dfa855982" Dec 03 12:02:23 crc kubenswrapper[5052]: E1203 12:02:23.678414 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 12:02:23 crc kubenswrapper[5052]: W1203 12:02:23.726115 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7d2a2ad_3910_425b_8bfe_e7fdc5e0f1e3.slice/crio-ae046ba315453cf7f27015c994cf66e9fd7e0b45ca2564c433fa8b4244e15245 WatchSource:0}: Error finding container ae046ba315453cf7f27015c994cf66e9fd7e0b45ca2564c433fa8b4244e15245: Status 404 returned error can't find the container with id ae046ba315453cf7f27015c994cf66e9fd7e0b45ca2564c433fa8b4244e15245 Dec 03 12:02:23 crc kubenswrapper[5052]: I1203 12:02:23.831862 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-cell1-server-1" event={"ID":"f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3","Type":"ContainerStarted","Data":"ae046ba315453cf7f27015c994cf66e9fd7e0b45ca2564c433fa8b4244e15245"} Dec 03 12:02:23 crc kubenswrapper[5052]: W1203 12:02:23.897408 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06bfd2db_abe0_4b4d_a110_94a6a1e02211.slice/crio-0191429cd34cf2c22173ad9222165ff9d2df01c0d470d59a5a082e087cd7c9b1 WatchSource:0}: Error finding container 0191429cd34cf2c22173ad9222165ff9d2df01c0d470d59a5a082e087cd7c9b1: Status 404 returned error can't find the container with id 0191429cd34cf2c22173ad9222165ff9d2df01c0d470d59a5a082e087cd7c9b1 Dec 03 12:02:23 crc kubenswrapper[5052]: I1203 12:02:23.908411 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/rabbitmq-cell1-server-2"] Dec 03 12:02:23 crc kubenswrapper[5052]: I1203 12:02:23.952467 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/rabbitmq-cell1-server-0"] Dec 03 12:02:24 crc kubenswrapper[5052]: I1203 12:02:24.175612 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/openstack-galera-0"] Dec 03 12:02:24 crc kubenswrapper[5052]: I1203 12:02:24.177580 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:02:24 crc kubenswrapper[5052]: I1203 12:02:24.179804 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-galera-openstack-svc" Dec 03 12:02:24 crc kubenswrapper[5052]: I1203 12:02:24.180024 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"openstack-config-data" Dec 03 12:02:24 crc kubenswrapper[5052]: I1203 12:02:24.180771 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"galera-openstack-dockercfg-8287r" Dec 03 12:02:24 crc kubenswrapper[5052]: I1203 12:02:24.183081 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"openstack-scripts" Dec 03 12:02:24 crc kubenswrapper[5052]: I1203 12:02:24.187475 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"combined-ca-bundle" Dec 03 12:02:24 crc kubenswrapper[5052]: I1203 12:02:24.193172 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/openstack-galera-1"] Dec 03 12:02:24 crc kubenswrapper[5052]: I1203 12:02:24.194706 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstack-galera-1" Dec 03 12:02:24 crc kubenswrapper[5052]: I1203 12:02:24.200251 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/openstack-galera-2"] Dec 03 12:02:24 crc kubenswrapper[5052]: I1203 12:02:24.202460 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstack-galera-2" Dec 03 12:02:24 crc kubenswrapper[5052]: I1203 12:02:24.207068 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/openstack-galera-0"] Dec 03 12:02:24 crc kubenswrapper[5052]: I1203 12:02:24.217247 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/openstack-galera-1"] Dec 03 12:02:24 crc kubenswrapper[5052]: I1203 12:02:24.228242 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/openstack-galera-2"] Dec 03 12:02:24 crc kubenswrapper[5052]: I1203 12:02:24.281849 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage06-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage06-crc\") pod \"openstack-galera-1\" (UID: \"6b92f88e-1325-4c3d-8d4e-4089a9033775\") " pod="openstack-kuttl-tests/openstack-galera-1" Dec 03 12:02:24 crc kubenswrapper[5052]: I1203 12:02:24.281900 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/143f4f1d-4f91-4ffa-a277-d6e551d19838-config-data-default\") pod \"openstack-galera-2\" (UID: \"143f4f1d-4f91-4ffa-a277-d6e551d19838\") " pod="openstack-kuttl-tests/openstack-galera-2" Dec 03 12:02:24 crc kubenswrapper[5052]: I1203 12:02:24.281960 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/143f4f1d-4f91-4ffa-a277-d6e551d19838-kolla-config\") pod \"openstack-galera-2\" (UID: \"143f4f1d-4f91-4ffa-a277-d6e551d19838\") " pod="openstack-kuttl-tests/openstack-galera-2" Dec 03 12:02:24 crc kubenswrapper[5052]: I1203 12:02:24.281982 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-897ts\" (UniqueName: \"kubernetes.io/projected/143f4f1d-4f91-4ffa-a277-d6e551d19838-kube-api-access-897ts\") pod \"openstack-galera-2\" (UID: \"143f4f1d-4f91-4ffa-a277-d6e551d19838\") " pod="openstack-kuttl-tests/openstack-galera-2" Dec 03 12:02:24 crc kubenswrapper[5052]: I1203 12:02:24.282039 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage01-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage01-crc\") pod \"openstack-galera-0\" (UID: \"d66ed4ec-fd97-4a62-82cb-cee3fe877524\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:02:24 crc kubenswrapper[5052]: I1203 12:02:24.282110 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bwb74\" (UniqueName: \"kubernetes.io/projected/d66ed4ec-fd97-4a62-82cb-cee3fe877524-kube-api-access-bwb74\") pod \"openstack-galera-0\" (UID: \"d66ed4ec-fd97-4a62-82cb-cee3fe877524\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:02:24 crc kubenswrapper[5052]: I1203 12:02:24.282187 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/6b92f88e-1325-4c3d-8d4e-4089a9033775-galera-tls-certs\") pod \"openstack-galera-1\" (UID: \"6b92f88e-1325-4c3d-8d4e-4089a9033775\") " pod="openstack-kuttl-tests/openstack-galera-1" Dec 03 12:02:24 crc kubenswrapper[5052]: I1203 12:02:24.282229 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d29gn\" (UniqueName: \"kubernetes.io/projected/6b92f88e-1325-4c3d-8d4e-4089a9033775-kube-api-access-d29gn\") pod \"openstack-galera-1\" (UID: \"6b92f88e-1325-4c3d-8d4e-4089a9033775\") " pod="openstack-kuttl-tests/openstack-galera-1" Dec 03 12:02:24 crc kubenswrapper[5052]: I1203 12:02:24.282274 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/143f4f1d-4f91-4ffa-a277-d6e551d19838-config-data-generated\") pod \"openstack-galera-2\" (UID: \"143f4f1d-4f91-4ffa-a277-d6e551d19838\") " pod="openstack-kuttl-tests/openstack-galera-2" Dec 03 12:02:24 crc kubenswrapper[5052]: I1203 12:02:24.282302 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6b92f88e-1325-4c3d-8d4e-4089a9033775-combined-ca-bundle\") pod \"openstack-galera-1\" (UID: \"6b92f88e-1325-4c3d-8d4e-4089a9033775\") " pod="openstack-kuttl-tests/openstack-galera-1" Dec 03 12:02:24 crc kubenswrapper[5052]: I1203 12:02:24.282349 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/d66ed4ec-fd97-4a62-82cb-cee3fe877524-galera-tls-certs\") pod \"openstack-galera-0\" (UID: \"d66ed4ec-fd97-4a62-82cb-cee3fe877524\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:02:24 crc kubenswrapper[5052]: I1203 12:02:24.282436 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/6b92f88e-1325-4c3d-8d4e-4089a9033775-operator-scripts\") pod \"openstack-galera-1\" (UID: \"6b92f88e-1325-4c3d-8d4e-4089a9033775\") " pod="openstack-kuttl-tests/openstack-galera-1" Dec 03 12:02:24 crc kubenswrapper[5052]: I1203 12:02:24.282789 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/143f4f1d-4f91-4ffa-a277-d6e551d19838-galera-tls-certs\") pod \"openstack-galera-2\" (UID: \"143f4f1d-4f91-4ffa-a277-d6e551d19838\") " pod="openstack-kuttl-tests/openstack-galera-2" Dec 03 12:02:24 crc kubenswrapper[5052]: I1203 12:02:24.282964 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage08-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage08-crc\") pod \"openstack-galera-2\" (UID: \"143f4f1d-4f91-4ffa-a277-d6e551d19838\") " pod="openstack-kuttl-tests/openstack-galera-2" Dec 03 12:02:24 crc kubenswrapper[5052]: I1203 12:02:24.283018 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/143f4f1d-4f91-4ffa-a277-d6e551d19838-operator-scripts\") pod \"openstack-galera-2\" (UID: \"143f4f1d-4f91-4ffa-a277-d6e551d19838\") " pod="openstack-kuttl-tests/openstack-galera-2" Dec 03 12:02:24 crc kubenswrapper[5052]: I1203 12:02:24.283087 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/6b92f88e-1325-4c3d-8d4e-4089a9033775-kolla-config\") pod \"openstack-galera-1\" (UID: \"6b92f88e-1325-4c3d-8d4e-4089a9033775\") " pod="openstack-kuttl-tests/openstack-galera-1" Dec 03 12:02:24 crc kubenswrapper[5052]: I1203 12:02:24.283160 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/d66ed4ec-fd97-4a62-82cb-cee3fe877524-config-data-default\") pod \"openstack-galera-0\" (UID: \"d66ed4ec-fd97-4a62-82cb-cee3fe877524\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:02:24 crc kubenswrapper[5052]: I1203 12:02:24.283200 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d66ed4ec-fd97-4a62-82cb-cee3fe877524-combined-ca-bundle\") pod \"openstack-galera-0\" (UID: \"d66ed4ec-fd97-4a62-82cb-cee3fe877524\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:02:24 crc kubenswrapper[5052]: I1203 12:02:24.283242 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/6b92f88e-1325-4c3d-8d4e-4089a9033775-config-data-generated\") pod \"openstack-galera-1\" (UID: \"6b92f88e-1325-4c3d-8d4e-4089a9033775\") " pod="openstack-kuttl-tests/openstack-galera-1" Dec 03 12:02:24 crc kubenswrapper[5052]: I1203 12:02:24.283283 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/d66ed4ec-fd97-4a62-82cb-cee3fe877524-config-data-generated\") pod \"openstack-galera-0\" (UID: \"d66ed4ec-fd97-4a62-82cb-cee3fe877524\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:02:24 crc kubenswrapper[5052]: I1203 12:02:24.283309 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/6b92f88e-1325-4c3d-8d4e-4089a9033775-config-data-default\") pod \"openstack-galera-1\" (UID: \"6b92f88e-1325-4c3d-8d4e-4089a9033775\") " pod="openstack-kuttl-tests/openstack-galera-1" Dec 03 12:02:24 crc kubenswrapper[5052]: I1203 12:02:24.283326 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/143f4f1d-4f91-4ffa-a277-d6e551d19838-combined-ca-bundle\") pod \"openstack-galera-2\" (UID: \"143f4f1d-4f91-4ffa-a277-d6e551d19838\") " pod="openstack-kuttl-tests/openstack-galera-2" Dec 03 12:02:24 crc kubenswrapper[5052]: I1203 12:02:24.283365 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/d66ed4ec-fd97-4a62-82cb-cee3fe877524-kolla-config\") pod \"openstack-galera-0\" (UID: \"d66ed4ec-fd97-4a62-82cb-cee3fe877524\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:02:24 crc kubenswrapper[5052]: I1203 12:02:24.283392 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/d66ed4ec-fd97-4a62-82cb-cee3fe877524-operator-scripts\") pod \"openstack-galera-0\" (UID: \"d66ed4ec-fd97-4a62-82cb-cee3fe877524\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:02:24 crc kubenswrapper[5052]: I1203 12:02:24.384581 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/6b92f88e-1325-4c3d-8d4e-4089a9033775-kolla-config\") pod \"openstack-galera-1\" (UID: \"6b92f88e-1325-4c3d-8d4e-4089a9033775\") " pod="openstack-kuttl-tests/openstack-galera-1" Dec 03 12:02:24 crc kubenswrapper[5052]: I1203 12:02:24.384967 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/d66ed4ec-fd97-4a62-82cb-cee3fe877524-config-data-default\") pod \"openstack-galera-0\" (UID: \"d66ed4ec-fd97-4a62-82cb-cee3fe877524\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:02:24 crc kubenswrapper[5052]: I1203 12:02:24.385094 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d66ed4ec-fd97-4a62-82cb-cee3fe877524-combined-ca-bundle\") pod \"openstack-galera-0\" (UID: \"d66ed4ec-fd97-4a62-82cb-cee3fe877524\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:02:24 crc kubenswrapper[5052]: I1203 12:02:24.385217 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/6b92f88e-1325-4c3d-8d4e-4089a9033775-config-data-generated\") pod \"openstack-galera-1\" (UID: \"6b92f88e-1325-4c3d-8d4e-4089a9033775\") " pod="openstack-kuttl-tests/openstack-galera-1" Dec 03 12:02:24 crc kubenswrapper[5052]: I1203 12:02:24.385365 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/d66ed4ec-fd97-4a62-82cb-cee3fe877524-config-data-generated\") pod \"openstack-galera-0\" (UID: \"d66ed4ec-fd97-4a62-82cb-cee3fe877524\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:02:24 crc kubenswrapper[5052]: I1203 12:02:24.385634 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/6b92f88e-1325-4c3d-8d4e-4089a9033775-kolla-config\") pod \"openstack-galera-1\" (UID: \"6b92f88e-1325-4c3d-8d4e-4089a9033775\") " pod="openstack-kuttl-tests/openstack-galera-1" Dec 03 12:02:24 crc kubenswrapper[5052]: I1203 12:02:24.385735 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/6b92f88e-1325-4c3d-8d4e-4089a9033775-config-data-generated\") pod \"openstack-galera-1\" (UID: \"6b92f88e-1325-4c3d-8d4e-4089a9033775\") " pod="openstack-kuttl-tests/openstack-galera-1" Dec 03 12:02:24 crc kubenswrapper[5052]: I1203 12:02:24.386078 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/d66ed4ec-fd97-4a62-82cb-cee3fe877524-config-data-generated\") pod \"openstack-galera-0\" (UID: \"d66ed4ec-fd97-4a62-82cb-cee3fe877524\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:02:24 crc kubenswrapper[5052]: I1203 12:02:24.386200 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/6b92f88e-1325-4c3d-8d4e-4089a9033775-config-data-default\") pod \"openstack-galera-1\" (UID: \"6b92f88e-1325-4c3d-8d4e-4089a9033775\") " pod="openstack-kuttl-tests/openstack-galera-1" Dec 03 12:02:24 crc kubenswrapper[5052]: I1203 12:02:24.386300 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/143f4f1d-4f91-4ffa-a277-d6e551d19838-combined-ca-bundle\") pod \"openstack-galera-2\" (UID: \"143f4f1d-4f91-4ffa-a277-d6e551d19838\") " pod="openstack-kuttl-tests/openstack-galera-2" Dec 03 12:02:24 crc kubenswrapper[5052]: I1203 12:02:24.386410 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/d66ed4ec-fd97-4a62-82cb-cee3fe877524-kolla-config\") pod \"openstack-galera-0\" (UID: \"d66ed4ec-fd97-4a62-82cb-cee3fe877524\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:02:24 crc kubenswrapper[5052]: I1203 12:02:24.386535 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/6b92f88e-1325-4c3d-8d4e-4089a9033775-config-data-default\") pod \"openstack-galera-1\" (UID: \"6b92f88e-1325-4c3d-8d4e-4089a9033775\") " pod="openstack-kuttl-tests/openstack-galera-1" Dec 03 12:02:24 crc kubenswrapper[5052]: I1203 12:02:24.386545 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/d66ed4ec-fd97-4a62-82cb-cee3fe877524-operator-scripts\") pod \"openstack-galera-0\" (UID: \"d66ed4ec-fd97-4a62-82cb-cee3fe877524\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:02:24 crc kubenswrapper[5052]: I1203 12:02:24.386736 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage06-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage06-crc\") pod \"openstack-galera-1\" (UID: \"6b92f88e-1325-4c3d-8d4e-4089a9033775\") " pod="openstack-kuttl-tests/openstack-galera-1" Dec 03 12:02:24 crc kubenswrapper[5052]: I1203 12:02:24.386840 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/143f4f1d-4f91-4ffa-a277-d6e551d19838-config-data-default\") pod \"openstack-galera-2\" (UID: \"143f4f1d-4f91-4ffa-a277-d6e551d19838\") " pod="openstack-kuttl-tests/openstack-galera-2" Dec 03 12:02:24 crc kubenswrapper[5052]: I1203 12:02:24.386969 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-897ts\" (UniqueName: \"kubernetes.io/projected/143f4f1d-4f91-4ffa-a277-d6e551d19838-kube-api-access-897ts\") pod \"openstack-galera-2\" (UID: \"143f4f1d-4f91-4ffa-a277-d6e551d19838\") " pod="openstack-kuttl-tests/openstack-galera-2" Dec 03 12:02:24 crc kubenswrapper[5052]: I1203 12:02:24.387065 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/d66ed4ec-fd97-4a62-82cb-cee3fe877524-kolla-config\") pod \"openstack-galera-0\" (UID: \"d66ed4ec-fd97-4a62-82cb-cee3fe877524\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:02:24 crc kubenswrapper[5052]: I1203 12:02:24.387077 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/143f4f1d-4f91-4ffa-a277-d6e551d19838-kolla-config\") pod \"openstack-galera-2\" (UID: \"143f4f1d-4f91-4ffa-a277-d6e551d19838\") " pod="openstack-kuttl-tests/openstack-galera-2" Dec 03 12:02:24 crc kubenswrapper[5052]: I1203 12:02:24.387271 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage06-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage06-crc\") pod \"openstack-galera-1\" (UID: \"6b92f88e-1325-4c3d-8d4e-4089a9033775\") device mount path \"/mnt/openstack/pv06\"" pod="openstack-kuttl-tests/openstack-galera-1" Dec 03 12:02:24 crc kubenswrapper[5052]: I1203 12:02:24.387423 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage01-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage01-crc\") pod \"openstack-galera-0\" (UID: \"d66ed4ec-fd97-4a62-82cb-cee3fe877524\") device mount path \"/mnt/openstack/pv01\"" pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:02:24 crc kubenswrapper[5052]: I1203 12:02:24.389255 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/143f4f1d-4f91-4ffa-a277-d6e551d19838-kolla-config\") pod \"openstack-galera-2\" (UID: \"143f4f1d-4f91-4ffa-a277-d6e551d19838\") " pod="openstack-kuttl-tests/openstack-galera-2" Dec 03 12:02:24 crc kubenswrapper[5052]: I1203 12:02:24.387276 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage01-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage01-crc\") pod \"openstack-galera-0\" (UID: \"d66ed4ec-fd97-4a62-82cb-cee3fe877524\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:02:24 crc kubenswrapper[5052]: I1203 12:02:24.389370 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-bwb74\" (UniqueName: \"kubernetes.io/projected/d66ed4ec-fd97-4a62-82cb-cee3fe877524-kube-api-access-bwb74\") pod \"openstack-galera-0\" (UID: \"d66ed4ec-fd97-4a62-82cb-cee3fe877524\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:02:24 crc kubenswrapper[5052]: I1203 12:02:24.389443 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/6b92f88e-1325-4c3d-8d4e-4089a9033775-galera-tls-certs\") pod \"openstack-galera-1\" (UID: \"6b92f88e-1325-4c3d-8d4e-4089a9033775\") " pod="openstack-kuttl-tests/openstack-galera-1" Dec 03 12:02:24 crc kubenswrapper[5052]: I1203 12:02:24.389493 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-d29gn\" (UniqueName: \"kubernetes.io/projected/6b92f88e-1325-4c3d-8d4e-4089a9033775-kube-api-access-d29gn\") pod \"openstack-galera-1\" (UID: \"6b92f88e-1325-4c3d-8d4e-4089a9033775\") " pod="openstack-kuttl-tests/openstack-galera-1" Dec 03 12:02:24 crc kubenswrapper[5052]: I1203 12:02:24.389536 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/143f4f1d-4f91-4ffa-a277-d6e551d19838-config-data-generated\") pod \"openstack-galera-2\" (UID: \"143f4f1d-4f91-4ffa-a277-d6e551d19838\") " pod="openstack-kuttl-tests/openstack-galera-2" Dec 03 12:02:24 crc kubenswrapper[5052]: I1203 12:02:24.389567 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/143f4f1d-4f91-4ffa-a277-d6e551d19838-config-data-default\") pod \"openstack-galera-2\" (UID: \"143f4f1d-4f91-4ffa-a277-d6e551d19838\") " pod="openstack-kuttl-tests/openstack-galera-2" Dec 03 12:02:24 crc kubenswrapper[5052]: I1203 12:02:24.389574 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6b92f88e-1325-4c3d-8d4e-4089a9033775-combined-ca-bundle\") pod \"openstack-galera-1\" (UID: \"6b92f88e-1325-4c3d-8d4e-4089a9033775\") " pod="openstack-kuttl-tests/openstack-galera-1" Dec 03 12:02:24 crc kubenswrapper[5052]: I1203 12:02:24.389620 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/d66ed4ec-fd97-4a62-82cb-cee3fe877524-galera-tls-certs\") pod \"openstack-galera-0\" (UID: \"d66ed4ec-fd97-4a62-82cb-cee3fe877524\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:02:24 crc kubenswrapper[5052]: I1203 12:02:24.389687 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/6b92f88e-1325-4c3d-8d4e-4089a9033775-operator-scripts\") pod \"openstack-galera-1\" (UID: \"6b92f88e-1325-4c3d-8d4e-4089a9033775\") " pod="openstack-kuttl-tests/openstack-galera-1" Dec 03 12:02:24 crc kubenswrapper[5052]: I1203 12:02:24.389722 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/143f4f1d-4f91-4ffa-a277-d6e551d19838-galera-tls-certs\") pod \"openstack-galera-2\" (UID: \"143f4f1d-4f91-4ffa-a277-d6e551d19838\") " pod="openstack-kuttl-tests/openstack-galera-2" Dec 03 12:02:24 crc kubenswrapper[5052]: I1203 12:02:24.389752 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage08-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage08-crc\") pod \"openstack-galera-2\" (UID: \"143f4f1d-4f91-4ffa-a277-d6e551d19838\") " pod="openstack-kuttl-tests/openstack-galera-2" Dec 03 12:02:24 crc kubenswrapper[5052]: I1203 12:02:24.389795 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/143f4f1d-4f91-4ffa-a277-d6e551d19838-operator-scripts\") pod \"openstack-galera-2\" (UID: \"143f4f1d-4f91-4ffa-a277-d6e551d19838\") " pod="openstack-kuttl-tests/openstack-galera-2" Dec 03 12:02:24 crc kubenswrapper[5052]: I1203 12:02:24.389831 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/d66ed4ec-fd97-4a62-82cb-cee3fe877524-operator-scripts\") pod \"openstack-galera-0\" (UID: \"d66ed4ec-fd97-4a62-82cb-cee3fe877524\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:02:24 crc kubenswrapper[5052]: I1203 12:02:24.389938 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage08-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage08-crc\") pod \"openstack-galera-2\" (UID: \"143f4f1d-4f91-4ffa-a277-d6e551d19838\") device mount path \"/mnt/openstack/pv08\"" pod="openstack-kuttl-tests/openstack-galera-2" Dec 03 12:02:24 crc kubenswrapper[5052]: I1203 12:02:24.391111 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/6b92f88e-1325-4c3d-8d4e-4089a9033775-operator-scripts\") pod \"openstack-galera-1\" (UID: \"6b92f88e-1325-4c3d-8d4e-4089a9033775\") " pod="openstack-kuttl-tests/openstack-galera-1" Dec 03 12:02:24 crc kubenswrapper[5052]: I1203 12:02:24.391318 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/143f4f1d-4f91-4ffa-a277-d6e551d19838-config-data-generated\") pod \"openstack-galera-2\" (UID: \"143f4f1d-4f91-4ffa-a277-d6e551d19838\") " pod="openstack-kuttl-tests/openstack-galera-2" Dec 03 12:02:24 crc kubenswrapper[5052]: I1203 12:02:24.391428 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/d66ed4ec-fd97-4a62-82cb-cee3fe877524-config-data-default\") pod \"openstack-galera-0\" (UID: \"d66ed4ec-fd97-4a62-82cb-cee3fe877524\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:02:24 crc kubenswrapper[5052]: I1203 12:02:24.392526 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/143f4f1d-4f91-4ffa-a277-d6e551d19838-operator-scripts\") pod \"openstack-galera-2\" (UID: \"143f4f1d-4f91-4ffa-a277-d6e551d19838\") " pod="openstack-kuttl-tests/openstack-galera-2" Dec 03 12:02:24 crc kubenswrapper[5052]: I1203 12:02:24.393141 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d66ed4ec-fd97-4a62-82cb-cee3fe877524-combined-ca-bundle\") pod \"openstack-galera-0\" (UID: \"d66ed4ec-fd97-4a62-82cb-cee3fe877524\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:02:24 crc kubenswrapper[5052]: I1203 12:02:24.394473 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/143f4f1d-4f91-4ffa-a277-d6e551d19838-combined-ca-bundle\") pod \"openstack-galera-2\" (UID: \"143f4f1d-4f91-4ffa-a277-d6e551d19838\") " pod="openstack-kuttl-tests/openstack-galera-2" Dec 03 12:02:24 crc kubenswrapper[5052]: I1203 12:02:24.401148 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/6b92f88e-1325-4c3d-8d4e-4089a9033775-galera-tls-certs\") pod \"openstack-galera-1\" (UID: \"6b92f88e-1325-4c3d-8d4e-4089a9033775\") " pod="openstack-kuttl-tests/openstack-galera-1" Dec 03 12:02:24 crc kubenswrapper[5052]: I1203 12:02:24.406447 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/d66ed4ec-fd97-4a62-82cb-cee3fe877524-galera-tls-certs\") pod \"openstack-galera-0\" (UID: \"d66ed4ec-fd97-4a62-82cb-cee3fe877524\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:02:24 crc kubenswrapper[5052]: I1203 12:02:24.409180 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/143f4f1d-4f91-4ffa-a277-d6e551d19838-galera-tls-certs\") pod \"openstack-galera-2\" (UID: \"143f4f1d-4f91-4ffa-a277-d6e551d19838\") " pod="openstack-kuttl-tests/openstack-galera-2" Dec 03 12:02:24 crc kubenswrapper[5052]: I1203 12:02:24.415902 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-bwb74\" (UniqueName: \"kubernetes.io/projected/d66ed4ec-fd97-4a62-82cb-cee3fe877524-kube-api-access-bwb74\") pod \"openstack-galera-0\" (UID: \"d66ed4ec-fd97-4a62-82cb-cee3fe877524\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:02:24 crc kubenswrapper[5052]: I1203 12:02:24.417404 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-897ts\" (UniqueName: \"kubernetes.io/projected/143f4f1d-4f91-4ffa-a277-d6e551d19838-kube-api-access-897ts\") pod \"openstack-galera-2\" (UID: \"143f4f1d-4f91-4ffa-a277-d6e551d19838\") " pod="openstack-kuttl-tests/openstack-galera-2" Dec 03 12:02:24 crc kubenswrapper[5052]: I1203 12:02:24.418395 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-d29gn\" (UniqueName: \"kubernetes.io/projected/6b92f88e-1325-4c3d-8d4e-4089a9033775-kube-api-access-d29gn\") pod \"openstack-galera-1\" (UID: \"6b92f88e-1325-4c3d-8d4e-4089a9033775\") " pod="openstack-kuttl-tests/openstack-galera-1" Dec 03 12:02:24 crc kubenswrapper[5052]: I1203 12:02:24.419915 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6b92f88e-1325-4c3d-8d4e-4089a9033775-combined-ca-bundle\") pod \"openstack-galera-1\" (UID: \"6b92f88e-1325-4c3d-8d4e-4089a9033775\") " pod="openstack-kuttl-tests/openstack-galera-1" Dec 03 12:02:24 crc kubenswrapper[5052]: I1203 12:02:24.422837 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage06-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage06-crc\") pod \"openstack-galera-1\" (UID: \"6b92f88e-1325-4c3d-8d4e-4089a9033775\") " pod="openstack-kuttl-tests/openstack-galera-1" Dec 03 12:02:24 crc kubenswrapper[5052]: I1203 12:02:24.430634 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage08-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage08-crc\") pod \"openstack-galera-2\" (UID: \"143f4f1d-4f91-4ffa-a277-d6e551d19838\") " pod="openstack-kuttl-tests/openstack-galera-2" Dec 03 12:02:24 crc kubenswrapper[5052]: I1203 12:02:24.521000 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstack-galera-1" Dec 03 12:02:24 crc kubenswrapper[5052]: I1203 12:02:24.533179 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstack-galera-2" Dec 03 12:02:24 crc kubenswrapper[5052]: I1203 12:02:24.558586 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage01-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage01-crc\") pod \"openstack-galera-0\" (UID: \"d66ed4ec-fd97-4a62-82cb-cee3fe877524\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:02:24 crc kubenswrapper[5052]: I1203 12:02:24.811483 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:02:24 crc kubenswrapper[5052]: I1203 12:02:24.844063 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-server-0" event={"ID":"2397d896-0d11-4656-a336-eac8ec9fe7f9","Type":"ContainerStarted","Data":"c716ca61940f2dfaed65a15c3a09f0b9191853485d4f681077e718bcb0869b41"} Dec 03 12:02:24 crc kubenswrapper[5052]: I1203 12:02:24.846425 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-server-1" event={"ID":"7901ce85-4e6f-4e8b-86a6-f50a85803637","Type":"ContainerStarted","Data":"ccd68041ae4063c32d99edce20b9aa0bf23f148287d9fcd6406b6a5e57a96ff9"} Dec 03 12:02:24 crc kubenswrapper[5052]: I1203 12:02:24.848172 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" event={"ID":"185fdd2a-cd57-45e5-9986-bebd15e5761f","Type":"ContainerStarted","Data":"a2993efc1065f6ca99a9c6fd96b8a330dba2947b6fec8ceb1edc381ae2fcfce5"} Dec 03 12:02:24 crc kubenswrapper[5052]: I1203 12:02:24.849647 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-cell1-server-2" event={"ID":"06bfd2db-abe0-4b4d-a110-94a6a1e02211","Type":"ContainerStarted","Data":"0191429cd34cf2c22173ad9222165ff9d2df01c0d470d59a5a082e087cd7c9b1"} Dec 03 12:02:24 crc kubenswrapper[5052]: I1203 12:02:24.851325 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-server-2" event={"ID":"7b4f6078-41a3-433d-acaf-f2d67391ab6e","Type":"ContainerStarted","Data":"142af50f0ab6083a28092a109ed34ea8ab8faf77ec5102e5c37130c564ddc949"} Dec 03 12:02:25 crc kubenswrapper[5052]: I1203 12:02:25.076424 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/openstack-galera-2"] Dec 03 12:02:25 crc kubenswrapper[5052]: I1203 12:02:25.125355 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/openstack-galera-1"] Dec 03 12:02:25 crc kubenswrapper[5052]: W1203 12:02:25.130631 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod143f4f1d_4f91_4ffa_a277_d6e551d19838.slice/crio-734f7beb409866abb41ccc1b21dd823ca31dcff8c3df4adc426e457918b2ba55 WatchSource:0}: Error finding container 734f7beb409866abb41ccc1b21dd823ca31dcff8c3df4adc426e457918b2ba55: Status 404 returned error can't find the container with id 734f7beb409866abb41ccc1b21dd823ca31dcff8c3df4adc426e457918b2ba55 Dec 03 12:02:25 crc kubenswrapper[5052]: W1203 12:02:25.130976 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6b92f88e_1325_4c3d_8d4e_4089a9033775.slice/crio-01c655f248edf79273060fb5f909b338b166fc31d6c5a9a9785381a3b0ba8dac WatchSource:0}: Error finding container 01c655f248edf79273060fb5f909b338b166fc31d6c5a9a9785381a3b0ba8dac: Status 404 returned error can't find the container with id 01c655f248edf79273060fb5f909b338b166fc31d6c5a9a9785381a3b0ba8dac Dec 03 12:02:25 crc kubenswrapper[5052]: W1203 12:02:25.335130 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd66ed4ec_fd97_4a62_82cb_cee3fe877524.slice/crio-893e1b658eab674cf7bf9866c1d9bbba9e853171a4415f27f1f45dfbb657af35 WatchSource:0}: Error finding container 893e1b658eab674cf7bf9866c1d9bbba9e853171a4415f27f1f45dfbb657af35: Status 404 returned error can't find the container with id 893e1b658eab674cf7bf9866c1d9bbba9e853171a4415f27f1f45dfbb657af35 Dec 03 12:02:25 crc kubenswrapper[5052]: I1203 12:02:25.335490 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/openstack-galera-0"] Dec 03 12:02:25 crc kubenswrapper[5052]: I1203 12:02:25.559372 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/openstack-cell1-galera-0"] Dec 03 12:02:25 crc kubenswrapper[5052]: I1203 12:02:25.561222 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:02:25 crc kubenswrapper[5052]: I1203 12:02:25.564062 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"galera-openstack-cell1-dockercfg-5hbmx" Dec 03 12:02:25 crc kubenswrapper[5052]: I1203 12:02:25.564202 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"openstack-cell1-scripts" Dec 03 12:02:25 crc kubenswrapper[5052]: I1203 12:02:25.564239 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-galera-openstack-cell1-svc" Dec 03 12:02:25 crc kubenswrapper[5052]: I1203 12:02:25.564145 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"openstack-cell1-config-data" Dec 03 12:02:25 crc kubenswrapper[5052]: I1203 12:02:25.575747 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/openstack-cell1-galera-0"] Dec 03 12:02:25 crc kubenswrapper[5052]: I1203 12:02:25.583656 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/openstack-cell1-galera-1"] Dec 03 12:02:25 crc kubenswrapper[5052]: I1203 12:02:25.584956 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstack-cell1-galera-1" Dec 03 12:02:25 crc kubenswrapper[5052]: I1203 12:02:25.596187 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/openstack-cell1-galera-2"] Dec 03 12:02:25 crc kubenswrapper[5052]: I1203 12:02:25.597806 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstack-cell1-galera-2" Dec 03 12:02:25 crc kubenswrapper[5052]: I1203 12:02:25.613213 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/b34bb4d0-7906-42e8-a9aa-bd1a4203a312-operator-scripts\") pod \"openstack-cell1-galera-1\" (UID: \"b34bb4d0-7906-42e8-a9aa-bd1a4203a312\") " pod="openstack-kuttl-tests/openstack-cell1-galera-1" Dec 03 12:02:25 crc kubenswrapper[5052]: I1203 12:02:25.613250 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b34bb4d0-7906-42e8-a9aa-bd1a4203a312-combined-ca-bundle\") pod \"openstack-cell1-galera-1\" (UID: \"b34bb4d0-7906-42e8-a9aa-bd1a4203a312\") " pod="openstack-kuttl-tests/openstack-cell1-galera-1" Dec 03 12:02:25 crc kubenswrapper[5052]: I1203 12:02:25.613277 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/b34bb4d0-7906-42e8-a9aa-bd1a4203a312-kolla-config\") pod \"openstack-cell1-galera-1\" (UID: \"b34bb4d0-7906-42e8-a9aa-bd1a4203a312\") " pod="openstack-kuttl-tests/openstack-cell1-galera-1" Dec 03 12:02:25 crc kubenswrapper[5052]: I1203 12:02:25.613315 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kmr95\" (UniqueName: \"kubernetes.io/projected/b34bb4d0-7906-42e8-a9aa-bd1a4203a312-kube-api-access-kmr95\") pod \"openstack-cell1-galera-1\" (UID: \"b34bb4d0-7906-42e8-a9aa-bd1a4203a312\") " pod="openstack-kuttl-tests/openstack-cell1-galera-1" Dec 03 12:02:25 crc kubenswrapper[5052]: I1203 12:02:25.613351 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/b34bb4d0-7906-42e8-a9aa-bd1a4203a312-galera-tls-certs\") pod \"openstack-cell1-galera-1\" (UID: \"b34bb4d0-7906-42e8-a9aa-bd1a4203a312\") " pod="openstack-kuttl-tests/openstack-cell1-galera-1" Dec 03 12:02:25 crc kubenswrapper[5052]: I1203 12:02:25.613379 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage07-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage07-crc\") pod \"openstack-cell1-galera-1\" (UID: \"b34bb4d0-7906-42e8-a9aa-bd1a4203a312\") " pod="openstack-kuttl-tests/openstack-cell1-galera-1" Dec 03 12:02:25 crc kubenswrapper[5052]: I1203 12:02:25.613419 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/b34bb4d0-7906-42e8-a9aa-bd1a4203a312-config-data-generated\") pod \"openstack-cell1-galera-1\" (UID: \"b34bb4d0-7906-42e8-a9aa-bd1a4203a312\") " pod="openstack-kuttl-tests/openstack-cell1-galera-1" Dec 03 12:02:25 crc kubenswrapper[5052]: I1203 12:02:25.613449 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/b34bb4d0-7906-42e8-a9aa-bd1a4203a312-config-data-default\") pod \"openstack-cell1-galera-1\" (UID: \"b34bb4d0-7906-42e8-a9aa-bd1a4203a312\") " pod="openstack-kuttl-tests/openstack-cell1-galera-1" Dec 03 12:02:25 crc kubenswrapper[5052]: I1203 12:02:25.641552 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/openstack-cell1-galera-1"] Dec 03 12:02:25 crc kubenswrapper[5052]: I1203 12:02:25.650685 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/openstack-cell1-galera-2"] Dec 03 12:02:25 crc kubenswrapper[5052]: I1203 12:02:25.714777 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/3b8ce5b7-67a1-48ef-892f-7408010a84f0-config-data-generated\") pod \"openstack-cell1-galera-0\" (UID: \"3b8ce5b7-67a1-48ef-892f-7408010a84f0\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:02:25 crc kubenswrapper[5052]: I1203 12:02:25.714841 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-kmr95\" (UniqueName: \"kubernetes.io/projected/b34bb4d0-7906-42e8-a9aa-bd1a4203a312-kube-api-access-kmr95\") pod \"openstack-cell1-galera-1\" (UID: \"b34bb4d0-7906-42e8-a9aa-bd1a4203a312\") " pod="openstack-kuttl-tests/openstack-cell1-galera-1" Dec 03 12:02:25 crc kubenswrapper[5052]: I1203 12:02:25.714863 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3b8ce5b7-67a1-48ef-892f-7408010a84f0-combined-ca-bundle\") pod \"openstack-cell1-galera-0\" (UID: \"3b8ce5b7-67a1-48ef-892f-7408010a84f0\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:02:25 crc kubenswrapper[5052]: I1203 12:02:25.714917 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/b34bb4d0-7906-42e8-a9aa-bd1a4203a312-galera-tls-certs\") pod \"openstack-cell1-galera-1\" (UID: \"b34bb4d0-7906-42e8-a9aa-bd1a4203a312\") " pod="openstack-kuttl-tests/openstack-cell1-galera-1" Dec 03 12:02:25 crc kubenswrapper[5052]: I1203 12:02:25.714948 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/3b8ce5b7-67a1-48ef-892f-7408010a84f0-kolla-config\") pod \"openstack-cell1-galera-0\" (UID: \"3b8ce5b7-67a1-48ef-892f-7408010a84f0\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:02:25 crc kubenswrapper[5052]: I1203 12:02:25.714979 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage07-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage07-crc\") pod \"openstack-cell1-galera-1\" (UID: \"b34bb4d0-7906-42e8-a9aa-bd1a4203a312\") " pod="openstack-kuttl-tests/openstack-cell1-galera-1" Dec 03 12:02:25 crc kubenswrapper[5052]: I1203 12:02:25.715008 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/3b8ce5b7-67a1-48ef-892f-7408010a84f0-config-data-default\") pod \"openstack-cell1-galera-0\" (UID: \"3b8ce5b7-67a1-48ef-892f-7408010a84f0\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:02:25 crc kubenswrapper[5052]: I1203 12:02:25.715038 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/3b8ce5b7-67a1-48ef-892f-7408010a84f0-operator-scripts\") pod \"openstack-cell1-galera-0\" (UID: \"3b8ce5b7-67a1-48ef-892f-7408010a84f0\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:02:25 crc kubenswrapper[5052]: I1203 12:02:25.715075 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/b34bb4d0-7906-42e8-a9aa-bd1a4203a312-config-data-generated\") pod \"openstack-cell1-galera-1\" (UID: \"b34bb4d0-7906-42e8-a9aa-bd1a4203a312\") " pod="openstack-kuttl-tests/openstack-cell1-galera-1" Dec 03 12:02:25 crc kubenswrapper[5052]: I1203 12:02:25.715108 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/b34bb4d0-7906-42e8-a9aa-bd1a4203a312-config-data-default\") pod \"openstack-cell1-galera-1\" (UID: \"b34bb4d0-7906-42e8-a9aa-bd1a4203a312\") " pod="openstack-kuttl-tests/openstack-cell1-galera-1" Dec 03 12:02:25 crc kubenswrapper[5052]: I1203 12:02:25.715175 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/b34bb4d0-7906-42e8-a9aa-bd1a4203a312-operator-scripts\") pod \"openstack-cell1-galera-1\" (UID: \"b34bb4d0-7906-42e8-a9aa-bd1a4203a312\") " pod="openstack-kuttl-tests/openstack-cell1-galera-1" Dec 03 12:02:25 crc kubenswrapper[5052]: I1203 12:02:25.715199 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b34bb4d0-7906-42e8-a9aa-bd1a4203a312-combined-ca-bundle\") pod \"openstack-cell1-galera-1\" (UID: \"b34bb4d0-7906-42e8-a9aa-bd1a4203a312\") " pod="openstack-kuttl-tests/openstack-cell1-galera-1" Dec 03 12:02:25 crc kubenswrapper[5052]: I1203 12:02:25.715222 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/b34bb4d0-7906-42e8-a9aa-bd1a4203a312-kolla-config\") pod \"openstack-cell1-galera-1\" (UID: \"b34bb4d0-7906-42e8-a9aa-bd1a4203a312\") " pod="openstack-kuttl-tests/openstack-cell1-galera-1" Dec 03 12:02:25 crc kubenswrapper[5052]: I1203 12:02:25.715244 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/3b8ce5b7-67a1-48ef-892f-7408010a84f0-galera-tls-certs\") pod \"openstack-cell1-galera-0\" (UID: \"3b8ce5b7-67a1-48ef-892f-7408010a84f0\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:02:25 crc kubenswrapper[5052]: I1203 12:02:25.715278 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8n58j\" (UniqueName: \"kubernetes.io/projected/3b8ce5b7-67a1-48ef-892f-7408010a84f0-kube-api-access-8n58j\") pod \"openstack-cell1-galera-0\" (UID: \"3b8ce5b7-67a1-48ef-892f-7408010a84f0\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:02:25 crc kubenswrapper[5052]: I1203 12:02:25.715302 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage09-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage09-crc\") pod \"openstack-cell1-galera-0\" (UID: \"3b8ce5b7-67a1-48ef-892f-7408010a84f0\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:02:25 crc kubenswrapper[5052]: I1203 12:02:25.716877 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/b34bb4d0-7906-42e8-a9aa-bd1a4203a312-config-data-default\") pod \"openstack-cell1-galera-1\" (UID: \"b34bb4d0-7906-42e8-a9aa-bd1a4203a312\") " pod="openstack-kuttl-tests/openstack-cell1-galera-1" Dec 03 12:02:25 crc kubenswrapper[5052]: I1203 12:02:25.716884 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/b34bb4d0-7906-42e8-a9aa-bd1a4203a312-kolla-config\") pod \"openstack-cell1-galera-1\" (UID: \"b34bb4d0-7906-42e8-a9aa-bd1a4203a312\") " pod="openstack-kuttl-tests/openstack-cell1-galera-1" Dec 03 12:02:25 crc kubenswrapper[5052]: I1203 12:02:25.717089 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage07-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage07-crc\") pod \"openstack-cell1-galera-1\" (UID: \"b34bb4d0-7906-42e8-a9aa-bd1a4203a312\") device mount path \"/mnt/openstack/pv07\"" pod="openstack-kuttl-tests/openstack-cell1-galera-1" Dec 03 12:02:25 crc kubenswrapper[5052]: I1203 12:02:25.717131 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/b34bb4d0-7906-42e8-a9aa-bd1a4203a312-config-data-generated\") pod \"openstack-cell1-galera-1\" (UID: \"b34bb4d0-7906-42e8-a9aa-bd1a4203a312\") " pod="openstack-kuttl-tests/openstack-cell1-galera-1" Dec 03 12:02:25 crc kubenswrapper[5052]: I1203 12:02:25.717636 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/b34bb4d0-7906-42e8-a9aa-bd1a4203a312-operator-scripts\") pod \"openstack-cell1-galera-1\" (UID: \"b34bb4d0-7906-42e8-a9aa-bd1a4203a312\") " pod="openstack-kuttl-tests/openstack-cell1-galera-1" Dec 03 12:02:25 crc kubenswrapper[5052]: I1203 12:02:25.723720 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b34bb4d0-7906-42e8-a9aa-bd1a4203a312-combined-ca-bundle\") pod \"openstack-cell1-galera-1\" (UID: \"b34bb4d0-7906-42e8-a9aa-bd1a4203a312\") " pod="openstack-kuttl-tests/openstack-cell1-galera-1" Dec 03 12:02:25 crc kubenswrapper[5052]: I1203 12:02:25.727054 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/b34bb4d0-7906-42e8-a9aa-bd1a4203a312-galera-tls-certs\") pod \"openstack-cell1-galera-1\" (UID: \"b34bb4d0-7906-42e8-a9aa-bd1a4203a312\") " pod="openstack-kuttl-tests/openstack-cell1-galera-1" Dec 03 12:02:25 crc kubenswrapper[5052]: I1203 12:02:25.744370 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-kmr95\" (UniqueName: \"kubernetes.io/projected/b34bb4d0-7906-42e8-a9aa-bd1a4203a312-kube-api-access-kmr95\") pod \"openstack-cell1-galera-1\" (UID: \"b34bb4d0-7906-42e8-a9aa-bd1a4203a312\") " pod="openstack-kuttl-tests/openstack-cell1-galera-1" Dec 03 12:02:25 crc kubenswrapper[5052]: I1203 12:02:25.751169 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage07-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage07-crc\") pod \"openstack-cell1-galera-1\" (UID: \"b34bb4d0-7906-42e8-a9aa-bd1a4203a312\") " pod="openstack-kuttl-tests/openstack-cell1-galera-1" Dec 03 12:02:25 crc kubenswrapper[5052]: I1203 12:02:25.817251 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/6ac95bd3-125c-4966-b9a7-4eb4a7842c67-operator-scripts\") pod \"openstack-cell1-galera-2\" (UID: \"6ac95bd3-125c-4966-b9a7-4eb4a7842c67\") " pod="openstack-kuttl-tests/openstack-cell1-galera-2" Dec 03 12:02:25 crc kubenswrapper[5052]: I1203 12:02:25.817302 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/6ac95bd3-125c-4966-b9a7-4eb4a7842c67-config-data-default\") pod \"openstack-cell1-galera-2\" (UID: \"6ac95bd3-125c-4966-b9a7-4eb4a7842c67\") " pod="openstack-kuttl-tests/openstack-cell1-galera-2" Dec 03 12:02:25 crc kubenswrapper[5052]: I1203 12:02:25.817328 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/6ac95bd3-125c-4966-b9a7-4eb4a7842c67-kolla-config\") pod \"openstack-cell1-galera-2\" (UID: \"6ac95bd3-125c-4966-b9a7-4eb4a7842c67\") " pod="openstack-kuttl-tests/openstack-cell1-galera-2" Dec 03 12:02:25 crc kubenswrapper[5052]: I1203 12:02:25.817355 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/3b8ce5b7-67a1-48ef-892f-7408010a84f0-galera-tls-certs\") pod \"openstack-cell1-galera-0\" (UID: \"3b8ce5b7-67a1-48ef-892f-7408010a84f0\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:02:25 crc kubenswrapper[5052]: I1203 12:02:25.817397 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-8n58j\" (UniqueName: \"kubernetes.io/projected/3b8ce5b7-67a1-48ef-892f-7408010a84f0-kube-api-access-8n58j\") pod \"openstack-cell1-galera-0\" (UID: \"3b8ce5b7-67a1-48ef-892f-7408010a84f0\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:02:25 crc kubenswrapper[5052]: I1203 12:02:25.817419 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage09-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage09-crc\") pod \"openstack-cell1-galera-0\" (UID: \"3b8ce5b7-67a1-48ef-892f-7408010a84f0\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:02:25 crc kubenswrapper[5052]: I1203 12:02:25.817442 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/3b8ce5b7-67a1-48ef-892f-7408010a84f0-config-data-generated\") pod \"openstack-cell1-galera-0\" (UID: \"3b8ce5b7-67a1-48ef-892f-7408010a84f0\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:02:25 crc kubenswrapper[5052]: I1203 12:02:25.817459 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3b8ce5b7-67a1-48ef-892f-7408010a84f0-combined-ca-bundle\") pod \"openstack-cell1-galera-0\" (UID: \"3b8ce5b7-67a1-48ef-892f-7408010a84f0\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:02:25 crc kubenswrapper[5052]: I1203 12:02:25.817485 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/6ac95bd3-125c-4966-b9a7-4eb4a7842c67-config-data-generated\") pod \"openstack-cell1-galera-2\" (UID: \"6ac95bd3-125c-4966-b9a7-4eb4a7842c67\") " pod="openstack-kuttl-tests/openstack-cell1-galera-2" Dec 03 12:02:25 crc kubenswrapper[5052]: I1203 12:02:25.817504 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hqgfz\" (UniqueName: \"kubernetes.io/projected/6ac95bd3-125c-4966-b9a7-4eb4a7842c67-kube-api-access-hqgfz\") pod \"openstack-cell1-galera-2\" (UID: \"6ac95bd3-125c-4966-b9a7-4eb4a7842c67\") " pod="openstack-kuttl-tests/openstack-cell1-galera-2" Dec 03 12:02:25 crc kubenswrapper[5052]: I1203 12:02:25.817545 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6ac95bd3-125c-4966-b9a7-4eb4a7842c67-combined-ca-bundle\") pod \"openstack-cell1-galera-2\" (UID: \"6ac95bd3-125c-4966-b9a7-4eb4a7842c67\") " pod="openstack-kuttl-tests/openstack-cell1-galera-2" Dec 03 12:02:25 crc kubenswrapper[5052]: I1203 12:02:25.817576 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage14-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage14-crc\") pod \"openstack-cell1-galera-2\" (UID: \"6ac95bd3-125c-4966-b9a7-4eb4a7842c67\") " pod="openstack-kuttl-tests/openstack-cell1-galera-2" Dec 03 12:02:25 crc kubenswrapper[5052]: I1203 12:02:25.817597 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/3b8ce5b7-67a1-48ef-892f-7408010a84f0-kolla-config\") pod \"openstack-cell1-galera-0\" (UID: \"3b8ce5b7-67a1-48ef-892f-7408010a84f0\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:02:25 crc kubenswrapper[5052]: I1203 12:02:25.817615 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/6ac95bd3-125c-4966-b9a7-4eb4a7842c67-galera-tls-certs\") pod \"openstack-cell1-galera-2\" (UID: \"6ac95bd3-125c-4966-b9a7-4eb4a7842c67\") " pod="openstack-kuttl-tests/openstack-cell1-galera-2" Dec 03 12:02:25 crc kubenswrapper[5052]: I1203 12:02:25.817652 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/3b8ce5b7-67a1-48ef-892f-7408010a84f0-config-data-default\") pod \"openstack-cell1-galera-0\" (UID: \"3b8ce5b7-67a1-48ef-892f-7408010a84f0\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:02:25 crc kubenswrapper[5052]: I1203 12:02:25.817702 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/3b8ce5b7-67a1-48ef-892f-7408010a84f0-operator-scripts\") pod \"openstack-cell1-galera-0\" (UID: \"3b8ce5b7-67a1-48ef-892f-7408010a84f0\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:02:25 crc kubenswrapper[5052]: I1203 12:02:25.819574 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage09-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage09-crc\") pod \"openstack-cell1-galera-0\" (UID: \"3b8ce5b7-67a1-48ef-892f-7408010a84f0\") device mount path \"/mnt/openstack/pv09\"" pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:02:25 crc kubenswrapper[5052]: I1203 12:02:25.820009 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/3b8ce5b7-67a1-48ef-892f-7408010a84f0-config-data-generated\") pod \"openstack-cell1-galera-0\" (UID: \"3b8ce5b7-67a1-48ef-892f-7408010a84f0\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:02:25 crc kubenswrapper[5052]: I1203 12:02:25.820242 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/3b8ce5b7-67a1-48ef-892f-7408010a84f0-kolla-config\") pod \"openstack-cell1-galera-0\" (UID: \"3b8ce5b7-67a1-48ef-892f-7408010a84f0\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:02:25 crc kubenswrapper[5052]: I1203 12:02:25.820649 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/3b8ce5b7-67a1-48ef-892f-7408010a84f0-config-data-default\") pod \"openstack-cell1-galera-0\" (UID: \"3b8ce5b7-67a1-48ef-892f-7408010a84f0\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:02:25 crc kubenswrapper[5052]: I1203 12:02:25.821160 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/3b8ce5b7-67a1-48ef-892f-7408010a84f0-operator-scripts\") pod \"openstack-cell1-galera-0\" (UID: \"3b8ce5b7-67a1-48ef-892f-7408010a84f0\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:02:25 crc kubenswrapper[5052]: I1203 12:02:25.822845 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/3b8ce5b7-67a1-48ef-892f-7408010a84f0-galera-tls-certs\") pod \"openstack-cell1-galera-0\" (UID: \"3b8ce5b7-67a1-48ef-892f-7408010a84f0\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:02:25 crc kubenswrapper[5052]: I1203 12:02:25.825026 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3b8ce5b7-67a1-48ef-892f-7408010a84f0-combined-ca-bundle\") pod \"openstack-cell1-galera-0\" (UID: \"3b8ce5b7-67a1-48ef-892f-7408010a84f0\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:02:25 crc kubenswrapper[5052]: I1203 12:02:25.847160 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-8n58j\" (UniqueName: \"kubernetes.io/projected/3b8ce5b7-67a1-48ef-892f-7408010a84f0-kube-api-access-8n58j\") pod \"openstack-cell1-galera-0\" (UID: \"3b8ce5b7-67a1-48ef-892f-7408010a84f0\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:02:25 crc kubenswrapper[5052]: I1203 12:02:25.850117 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage09-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage09-crc\") pod \"openstack-cell1-galera-0\" (UID: \"3b8ce5b7-67a1-48ef-892f-7408010a84f0\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:02:25 crc kubenswrapper[5052]: I1203 12:02:25.861555 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-cell1-server-1" event={"ID":"f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3","Type":"ContainerStarted","Data":"0790cbdcd312b0f701867eb108daefe7d9c2065afd80108a7c514db8349d75a3"} Dec 03 12:02:25 crc kubenswrapper[5052]: I1203 12:02:25.862955 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" event={"ID":"185fdd2a-cd57-45e5-9986-bebd15e5761f","Type":"ContainerStarted","Data":"6b72ea83c326e823c684041451c505add9ed54cb24251b5b99e7e2a2086b0b19"} Dec 03 12:02:25 crc kubenswrapper[5052]: I1203 12:02:25.864481 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-galera-2" event={"ID":"143f4f1d-4f91-4ffa-a277-d6e551d19838","Type":"ContainerStarted","Data":"218bda99a56c19c275c25f001fccbc4fcd8937679bb8bced56c2b578dae2ff3a"} Dec 03 12:02:25 crc kubenswrapper[5052]: I1203 12:02:25.864537 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-galera-2" event={"ID":"143f4f1d-4f91-4ffa-a277-d6e551d19838","Type":"ContainerStarted","Data":"734f7beb409866abb41ccc1b21dd823ca31dcff8c3df4adc426e457918b2ba55"} Dec 03 12:02:25 crc kubenswrapper[5052]: I1203 12:02:25.866458 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-galera-0" event={"ID":"d66ed4ec-fd97-4a62-82cb-cee3fe877524","Type":"ContainerStarted","Data":"c379302f8861ae5abdb296b0309b5345e5b9fa7962d9bc64cf7cb65a109171d6"} Dec 03 12:02:25 crc kubenswrapper[5052]: I1203 12:02:25.866536 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-galera-0" event={"ID":"d66ed4ec-fd97-4a62-82cb-cee3fe877524","Type":"ContainerStarted","Data":"893e1b658eab674cf7bf9866c1d9bbba9e853171a4415f27f1f45dfbb657af35"} Dec 03 12:02:25 crc kubenswrapper[5052]: I1203 12:02:25.868029 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-cell1-server-2" event={"ID":"06bfd2db-abe0-4b4d-a110-94a6a1e02211","Type":"ContainerStarted","Data":"9889df1b1220aa2bd33fc61e1b06bb56a5427700764018be0e9dc52ae30425fd"} Dec 03 12:02:25 crc kubenswrapper[5052]: I1203 12:02:25.869634 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-galera-1" event={"ID":"6b92f88e-1325-4c3d-8d4e-4089a9033775","Type":"ContainerStarted","Data":"797aefb21d4fd0eef8fa64544950d707d51b6474ea9063aae71828835bd51da9"} Dec 03 12:02:25 crc kubenswrapper[5052]: I1203 12:02:25.869696 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-galera-1" event={"ID":"6b92f88e-1325-4c3d-8d4e-4089a9033775","Type":"ContainerStarted","Data":"01c655f248edf79273060fb5f909b338b166fc31d6c5a9a9785381a3b0ba8dac"} Dec 03 12:02:25 crc kubenswrapper[5052]: I1203 12:02:25.875742 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:02:25 crc kubenswrapper[5052]: I1203 12:02:25.885106 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/memcached-0"] Dec 03 12:02:25 crc kubenswrapper[5052]: I1203 12:02:25.887711 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/memcached-0" Dec 03 12:02:25 crc kubenswrapper[5052]: I1203 12:02:25.891069 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"memcached-config-data" Dec 03 12:02:25 crc kubenswrapper[5052]: I1203 12:02:25.891463 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"memcached-memcached-dockercfg-mbgzj" Dec 03 12:02:25 crc kubenswrapper[5052]: I1203 12:02:25.896136 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-memcached-svc" Dec 03 12:02:25 crc kubenswrapper[5052]: I1203 12:02:25.909854 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstack-cell1-galera-1" Dec 03 12:02:25 crc kubenswrapper[5052]: I1203 12:02:25.912226 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/memcached-0"] Dec 03 12:02:25 crc kubenswrapper[5052]: I1203 12:02:25.920462 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/6ac95bd3-125c-4966-b9a7-4eb4a7842c67-config-data-generated\") pod \"openstack-cell1-galera-2\" (UID: \"6ac95bd3-125c-4966-b9a7-4eb4a7842c67\") " pod="openstack-kuttl-tests/openstack-cell1-galera-2" Dec 03 12:02:25 crc kubenswrapper[5052]: I1203 12:02:25.920498 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-hqgfz\" (UniqueName: \"kubernetes.io/projected/6ac95bd3-125c-4966-b9a7-4eb4a7842c67-kube-api-access-hqgfz\") pod \"openstack-cell1-galera-2\" (UID: \"6ac95bd3-125c-4966-b9a7-4eb4a7842c67\") " pod="openstack-kuttl-tests/openstack-cell1-galera-2" Dec 03 12:02:25 crc kubenswrapper[5052]: I1203 12:02:25.920545 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6ac95bd3-125c-4966-b9a7-4eb4a7842c67-combined-ca-bundle\") pod \"openstack-cell1-galera-2\" (UID: \"6ac95bd3-125c-4966-b9a7-4eb4a7842c67\") " pod="openstack-kuttl-tests/openstack-cell1-galera-2" Dec 03 12:02:25 crc kubenswrapper[5052]: I1203 12:02:25.920578 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage14-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage14-crc\") pod \"openstack-cell1-galera-2\" (UID: \"6ac95bd3-125c-4966-b9a7-4eb4a7842c67\") " pod="openstack-kuttl-tests/openstack-cell1-galera-2" Dec 03 12:02:25 crc kubenswrapper[5052]: I1203 12:02:25.920616 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/6ac95bd3-125c-4966-b9a7-4eb4a7842c67-galera-tls-certs\") pod \"openstack-cell1-galera-2\" (UID: \"6ac95bd3-125c-4966-b9a7-4eb4a7842c67\") " pod="openstack-kuttl-tests/openstack-cell1-galera-2" Dec 03 12:02:25 crc kubenswrapper[5052]: I1203 12:02:25.920716 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/6ac95bd3-125c-4966-b9a7-4eb4a7842c67-operator-scripts\") pod \"openstack-cell1-galera-2\" (UID: \"6ac95bd3-125c-4966-b9a7-4eb4a7842c67\") " pod="openstack-kuttl-tests/openstack-cell1-galera-2" Dec 03 12:02:25 crc kubenswrapper[5052]: I1203 12:02:25.920735 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/6ac95bd3-125c-4966-b9a7-4eb4a7842c67-config-data-default\") pod \"openstack-cell1-galera-2\" (UID: \"6ac95bd3-125c-4966-b9a7-4eb4a7842c67\") " pod="openstack-kuttl-tests/openstack-cell1-galera-2" Dec 03 12:02:25 crc kubenswrapper[5052]: I1203 12:02:25.920770 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/6ac95bd3-125c-4966-b9a7-4eb4a7842c67-kolla-config\") pod \"openstack-cell1-galera-2\" (UID: \"6ac95bd3-125c-4966-b9a7-4eb4a7842c67\") " pod="openstack-kuttl-tests/openstack-cell1-galera-2" Dec 03 12:02:25 crc kubenswrapper[5052]: I1203 12:02:25.921955 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/6ac95bd3-125c-4966-b9a7-4eb4a7842c67-kolla-config\") pod \"openstack-cell1-galera-2\" (UID: \"6ac95bd3-125c-4966-b9a7-4eb4a7842c67\") " pod="openstack-kuttl-tests/openstack-cell1-galera-2" Dec 03 12:02:25 crc kubenswrapper[5052]: I1203 12:02:25.922227 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage14-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage14-crc\") pod \"openstack-cell1-galera-2\" (UID: \"6ac95bd3-125c-4966-b9a7-4eb4a7842c67\") device mount path \"/mnt/openstack/pv14\"" pod="openstack-kuttl-tests/openstack-cell1-galera-2" Dec 03 12:02:25 crc kubenswrapper[5052]: I1203 12:02:25.924152 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/6ac95bd3-125c-4966-b9a7-4eb4a7842c67-config-data-generated\") pod \"openstack-cell1-galera-2\" (UID: \"6ac95bd3-125c-4966-b9a7-4eb4a7842c67\") " pod="openstack-kuttl-tests/openstack-cell1-galera-2" Dec 03 12:02:25 crc kubenswrapper[5052]: I1203 12:02:25.924558 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/6ac95bd3-125c-4966-b9a7-4eb4a7842c67-config-data-default\") pod \"openstack-cell1-galera-2\" (UID: \"6ac95bd3-125c-4966-b9a7-4eb4a7842c67\") " pod="openstack-kuttl-tests/openstack-cell1-galera-2" Dec 03 12:02:25 crc kubenswrapper[5052]: I1203 12:02:25.924811 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/6ac95bd3-125c-4966-b9a7-4eb4a7842c67-operator-scripts\") pod \"openstack-cell1-galera-2\" (UID: \"6ac95bd3-125c-4966-b9a7-4eb4a7842c67\") " pod="openstack-kuttl-tests/openstack-cell1-galera-2" Dec 03 12:02:25 crc kubenswrapper[5052]: I1203 12:02:25.935378 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/6ac95bd3-125c-4966-b9a7-4eb4a7842c67-galera-tls-certs\") pod \"openstack-cell1-galera-2\" (UID: \"6ac95bd3-125c-4966-b9a7-4eb4a7842c67\") " pod="openstack-kuttl-tests/openstack-cell1-galera-2" Dec 03 12:02:25 crc kubenswrapper[5052]: I1203 12:02:25.947564 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6ac95bd3-125c-4966-b9a7-4eb4a7842c67-combined-ca-bundle\") pod \"openstack-cell1-galera-2\" (UID: \"6ac95bd3-125c-4966-b9a7-4eb4a7842c67\") " pod="openstack-kuttl-tests/openstack-cell1-galera-2" Dec 03 12:02:25 crc kubenswrapper[5052]: I1203 12:02:25.962201 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-hqgfz\" (UniqueName: \"kubernetes.io/projected/6ac95bd3-125c-4966-b9a7-4eb4a7842c67-kube-api-access-hqgfz\") pod \"openstack-cell1-galera-2\" (UID: \"6ac95bd3-125c-4966-b9a7-4eb4a7842c67\") " pod="openstack-kuttl-tests/openstack-cell1-galera-2" Dec 03 12:02:26 crc kubenswrapper[5052]: I1203 12:02:26.006668 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage14-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage14-crc\") pod \"openstack-cell1-galera-2\" (UID: \"6ac95bd3-125c-4966-b9a7-4eb4a7842c67\") " pod="openstack-kuttl-tests/openstack-cell1-galera-2" Dec 03 12:02:26 crc kubenswrapper[5052]: I1203 12:02:26.137649 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/bbdcaf66-8813-4bed-9311-552fef62ac2d-combined-ca-bundle\") pod \"memcached-0\" (UID: \"bbdcaf66-8813-4bed-9311-552fef62ac2d\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:02:26 crc kubenswrapper[5052]: I1203 12:02:26.151347 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/bbdcaf66-8813-4bed-9311-552fef62ac2d-kolla-config\") pod \"memcached-0\" (UID: \"bbdcaf66-8813-4bed-9311-552fef62ac2d\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:02:26 crc kubenswrapper[5052]: I1203 12:02:26.151409 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"memcached-tls-certs\" (UniqueName: \"kubernetes.io/secret/bbdcaf66-8813-4bed-9311-552fef62ac2d-memcached-tls-certs\") pod \"memcached-0\" (UID: \"bbdcaf66-8813-4bed-9311-552fef62ac2d\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:02:26 crc kubenswrapper[5052]: I1203 12:02:26.151529 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p47k8\" (UniqueName: \"kubernetes.io/projected/bbdcaf66-8813-4bed-9311-552fef62ac2d-kube-api-access-p47k8\") pod \"memcached-0\" (UID: \"bbdcaf66-8813-4bed-9311-552fef62ac2d\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:02:26 crc kubenswrapper[5052]: I1203 12:02:26.151827 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/bbdcaf66-8813-4bed-9311-552fef62ac2d-config-data\") pod \"memcached-0\" (UID: \"bbdcaf66-8813-4bed-9311-552fef62ac2d\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:02:26 crc kubenswrapper[5052]: I1203 12:02:26.225393 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstack-cell1-galera-2" Dec 03 12:02:26 crc kubenswrapper[5052]: I1203 12:02:26.255755 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/bbdcaf66-8813-4bed-9311-552fef62ac2d-combined-ca-bundle\") pod \"memcached-0\" (UID: \"bbdcaf66-8813-4bed-9311-552fef62ac2d\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:02:26 crc kubenswrapper[5052]: I1203 12:02:26.256141 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/bbdcaf66-8813-4bed-9311-552fef62ac2d-kolla-config\") pod \"memcached-0\" (UID: \"bbdcaf66-8813-4bed-9311-552fef62ac2d\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:02:26 crc kubenswrapper[5052]: I1203 12:02:26.256235 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"memcached-tls-certs\" (UniqueName: \"kubernetes.io/secret/bbdcaf66-8813-4bed-9311-552fef62ac2d-memcached-tls-certs\") pod \"memcached-0\" (UID: \"bbdcaf66-8813-4bed-9311-552fef62ac2d\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:02:26 crc kubenswrapper[5052]: I1203 12:02:26.256329 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-p47k8\" (UniqueName: \"kubernetes.io/projected/bbdcaf66-8813-4bed-9311-552fef62ac2d-kube-api-access-p47k8\") pod \"memcached-0\" (UID: \"bbdcaf66-8813-4bed-9311-552fef62ac2d\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:02:26 crc kubenswrapper[5052]: I1203 12:02:26.256416 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/bbdcaf66-8813-4bed-9311-552fef62ac2d-config-data\") pod \"memcached-0\" (UID: \"bbdcaf66-8813-4bed-9311-552fef62ac2d\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:02:26 crc kubenswrapper[5052]: I1203 12:02:26.257188 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/bbdcaf66-8813-4bed-9311-552fef62ac2d-config-data\") pod \"memcached-0\" (UID: \"bbdcaf66-8813-4bed-9311-552fef62ac2d\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:02:26 crc kubenswrapper[5052]: I1203 12:02:26.259615 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/bbdcaf66-8813-4bed-9311-552fef62ac2d-kolla-config\") pod \"memcached-0\" (UID: \"bbdcaf66-8813-4bed-9311-552fef62ac2d\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:02:26 crc kubenswrapper[5052]: I1203 12:02:26.270349 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/bbdcaf66-8813-4bed-9311-552fef62ac2d-combined-ca-bundle\") pod \"memcached-0\" (UID: \"bbdcaf66-8813-4bed-9311-552fef62ac2d\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:02:26 crc kubenswrapper[5052]: I1203 12:02:26.284413 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"memcached-tls-certs\" (UniqueName: \"kubernetes.io/secret/bbdcaf66-8813-4bed-9311-552fef62ac2d-memcached-tls-certs\") pod \"memcached-0\" (UID: \"bbdcaf66-8813-4bed-9311-552fef62ac2d\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:02:26 crc kubenswrapper[5052]: I1203 12:02:26.324956 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-p47k8\" (UniqueName: \"kubernetes.io/projected/bbdcaf66-8813-4bed-9311-552fef62ac2d-kube-api-access-p47k8\") pod \"memcached-0\" (UID: \"bbdcaf66-8813-4bed-9311-552fef62ac2d\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:02:26 crc kubenswrapper[5052]: I1203 12:02:26.513531 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/memcached-0" Dec 03 12:02:26 crc kubenswrapper[5052]: I1203 12:02:26.620374 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/openstack-cell1-galera-2"] Dec 03 12:02:26 crc kubenswrapper[5052]: W1203 12:02:26.627278 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6ac95bd3_125c_4966_b9a7_4eb4a7842c67.slice/crio-5d0618ff0b0c564d02f8c132d2bdf496a4eb2cd52871397cdc06d0a57306a948 WatchSource:0}: Error finding container 5d0618ff0b0c564d02f8c132d2bdf496a4eb2cd52871397cdc06d0a57306a948: Status 404 returned error can't find the container with id 5d0618ff0b0c564d02f8c132d2bdf496a4eb2cd52871397cdc06d0a57306a948 Dec 03 12:02:26 crc kubenswrapper[5052]: I1203 12:02:26.714725 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/openstack-cell1-galera-0"] Dec 03 12:02:26 crc kubenswrapper[5052]: I1203 12:02:26.725374 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/openstack-cell1-galera-1"] Dec 03 12:02:26 crc kubenswrapper[5052]: I1203 12:02:26.883994 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-cell1-galera-2" event={"ID":"6ac95bd3-125c-4966-b9a7-4eb4a7842c67","Type":"ContainerStarted","Data":"6f25f4851b71de7e5136ec08e6aa9f8b2530fdd3a88edfd5b32d7294b08cdcfa"} Dec 03 12:02:26 crc kubenswrapper[5052]: I1203 12:02:26.884045 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-cell1-galera-2" event={"ID":"6ac95bd3-125c-4966-b9a7-4eb4a7842c67","Type":"ContainerStarted","Data":"5d0618ff0b0c564d02f8c132d2bdf496a4eb2cd52871397cdc06d0a57306a948"} Dec 03 12:02:26 crc kubenswrapper[5052]: I1203 12:02:26.887522 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-cell1-galera-1" event={"ID":"b34bb4d0-7906-42e8-a9aa-bd1a4203a312","Type":"ContainerStarted","Data":"69e7ac97fd0fcb77437d78b9a5a2c861a52f2027179a08c9a2487f6d93b9a0da"} Dec 03 12:02:26 crc kubenswrapper[5052]: I1203 12:02:26.889241 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-cell1-galera-0" event={"ID":"3b8ce5b7-67a1-48ef-892f-7408010a84f0","Type":"ContainerStarted","Data":"96fc8571648dfbfa0bf3c3f62c3ad7b38970d612280158cac29836794efa1ab8"} Dec 03 12:02:26 crc kubenswrapper[5052]: W1203 12:02:26.956940 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbbdcaf66_8813_4bed_9311_552fef62ac2d.slice/crio-9461f788efd8441e720cf3cc20734008a6a4bac027929586fefd7e176e64de95 WatchSource:0}: Error finding container 9461f788efd8441e720cf3cc20734008a6a4bac027929586fefd7e176e64de95: Status 404 returned error can't find the container with id 9461f788efd8441e720cf3cc20734008a6a4bac027929586fefd7e176e64de95 Dec 03 12:02:26 crc kubenswrapper[5052]: I1203 12:02:26.956950 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/memcached-0"] Dec 03 12:02:27 crc kubenswrapper[5052]: I1203 12:02:27.516659 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/kube-state-metrics-0"] Dec 03 12:02:27 crc kubenswrapper[5052]: I1203 12:02:27.518352 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:02:27 crc kubenswrapper[5052]: I1203 12:02:27.524432 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"telemetry-ceilometer-dockercfg-897fb" Dec 03 12:02:27 crc kubenswrapper[5052]: I1203 12:02:27.541747 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/kube-state-metrics-0"] Dec 03 12:02:27 crc kubenswrapper[5052]: I1203 12:02:27.583959 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-97fxw\" (UniqueName: \"kubernetes.io/projected/8de0e655-f56e-40a1-b154-74ff024542ed-kube-api-access-97fxw\") pod \"kube-state-metrics-0\" (UID: \"8de0e655-f56e-40a1-b154-74ff024542ed\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:02:27 crc kubenswrapper[5052]: I1203 12:02:27.686220 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-97fxw\" (UniqueName: \"kubernetes.io/projected/8de0e655-f56e-40a1-b154-74ff024542ed-kube-api-access-97fxw\") pod \"kube-state-metrics-0\" (UID: \"8de0e655-f56e-40a1-b154-74ff024542ed\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:02:27 crc kubenswrapper[5052]: I1203 12:02:27.710869 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-97fxw\" (UniqueName: \"kubernetes.io/projected/8de0e655-f56e-40a1-b154-74ff024542ed-kube-api-access-97fxw\") pod \"kube-state-metrics-0\" (UID: \"8de0e655-f56e-40a1-b154-74ff024542ed\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:02:27 crc kubenswrapper[5052]: I1203 12:02:27.840869 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:02:27 crc kubenswrapper[5052]: I1203 12:02:27.905718 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-cell1-galera-0" event={"ID":"3b8ce5b7-67a1-48ef-892f-7408010a84f0","Type":"ContainerStarted","Data":"fc6a07b3ee2f8d37a641173f6de8b77d45fd5bc8c9701900d684dde2e6532818"} Dec 03 12:02:27 crc kubenswrapper[5052]: I1203 12:02:27.910609 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/memcached-0" event={"ID":"bbdcaf66-8813-4bed-9311-552fef62ac2d","Type":"ContainerStarted","Data":"e6787f6e4b03711dd124764348c5ed1eaf82f566af649919f63f24cc48189367"} Dec 03 12:02:27 crc kubenswrapper[5052]: I1203 12:02:27.910797 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/memcached-0" event={"ID":"bbdcaf66-8813-4bed-9311-552fef62ac2d","Type":"ContainerStarted","Data":"9461f788efd8441e720cf3cc20734008a6a4bac027929586fefd7e176e64de95"} Dec 03 12:02:27 crc kubenswrapper[5052]: I1203 12:02:27.911424 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/memcached-0" Dec 03 12:02:27 crc kubenswrapper[5052]: I1203 12:02:27.914655 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-cell1-galera-1" event={"ID":"b34bb4d0-7906-42e8-a9aa-bd1a4203a312","Type":"ContainerStarted","Data":"cfa78e68ee04705ba5099d5662bbfe0cc1eb410ee1f31655537d62c888b13793"} Dec 03 12:02:28 crc kubenswrapper[5052]: I1203 12:02:27.999130 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/memcached-0" podStartSLOduration=2.99910493 podStartE2EDuration="2.99910493s" podCreationTimestamp="2025-12-03 12:02:25 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:02:27.990603897 +0000 UTC m=+2538.578845671" watchObservedRunningTime="2025-12-03 12:02:27.99910493 +0000 UTC m=+2538.587346694" Dec 03 12:02:28 crc kubenswrapper[5052]: I1203 12:02:28.293666 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/kube-state-metrics-0"] Dec 03 12:02:28 crc kubenswrapper[5052]: I1203 12:02:28.924048 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/kube-state-metrics-0" event={"ID":"8de0e655-f56e-40a1-b154-74ff024542ed","Type":"ContainerStarted","Data":"f9d45e2c0b71c2af7b0a9dbe237c4df2a93ed8ebafd343943bf8443e3208c099"} Dec 03 12:02:29 crc kubenswrapper[5052]: I1203 12:02:29.937647 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/kube-state-metrics-0" event={"ID":"8de0e655-f56e-40a1-b154-74ff024542ed","Type":"ContainerStarted","Data":"0572f095ac1d9fd1aaaca356eeaaa88ff3c268ecc5d27416e4297c1a0bd46b82"} Dec 03 12:02:29 crc kubenswrapper[5052]: I1203 12:02:29.956239 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/kube-state-metrics-0" podStartSLOduration=1.7079696659999999 podStartE2EDuration="2.956215393s" podCreationTimestamp="2025-12-03 12:02:27 +0000 UTC" firstStartedPulling="2025-12-03 12:02:28.302543315 +0000 UTC m=+2538.890785079" lastFinishedPulling="2025-12-03 12:02:29.550789042 +0000 UTC m=+2540.139030806" observedRunningTime="2025-12-03 12:02:29.953961658 +0000 UTC m=+2540.542203422" watchObservedRunningTime="2025-12-03 12:02:29.956215393 +0000 UTC m=+2540.544457157" Dec 03 12:02:30 crc kubenswrapper[5052]: I1203 12:02:30.946670 5052 generic.go:334] "Generic (PLEG): container finished" podID="d66ed4ec-fd97-4a62-82cb-cee3fe877524" containerID="c379302f8861ae5abdb296b0309b5345e5b9fa7962d9bc64cf7cb65a109171d6" exitCode=0 Dec 03 12:02:30 crc kubenswrapper[5052]: I1203 12:02:30.946753 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-galera-0" event={"ID":"d66ed4ec-fd97-4a62-82cb-cee3fe877524","Type":"ContainerDied","Data":"c379302f8861ae5abdb296b0309b5345e5b9fa7962d9bc64cf7cb65a109171d6"} Dec 03 12:02:30 crc kubenswrapper[5052]: I1203 12:02:30.948895 5052 generic.go:334] "Generic (PLEG): container finished" podID="6b92f88e-1325-4c3d-8d4e-4089a9033775" containerID="797aefb21d4fd0eef8fa64544950d707d51b6474ea9063aae71828835bd51da9" exitCode=0 Dec 03 12:02:30 crc kubenswrapper[5052]: I1203 12:02:30.948997 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-galera-1" event={"ID":"6b92f88e-1325-4c3d-8d4e-4089a9033775","Type":"ContainerDied","Data":"797aefb21d4fd0eef8fa64544950d707d51b6474ea9063aae71828835bd51da9"} Dec 03 12:02:30 crc kubenswrapper[5052]: I1203 12:02:30.950764 5052 generic.go:334] "Generic (PLEG): container finished" podID="143f4f1d-4f91-4ffa-a277-d6e551d19838" containerID="218bda99a56c19c275c25f001fccbc4fcd8937679bb8bced56c2b578dae2ff3a" exitCode=0 Dec 03 12:02:30 crc kubenswrapper[5052]: I1203 12:02:30.950845 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-galera-2" event={"ID":"143f4f1d-4f91-4ffa-a277-d6e551d19838","Type":"ContainerDied","Data":"218bda99a56c19c275c25f001fccbc4fcd8937679bb8bced56c2b578dae2ff3a"} Dec 03 12:02:30 crc kubenswrapper[5052]: I1203 12:02:30.951023 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:02:31 crc kubenswrapper[5052]: I1203 12:02:31.789366 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ovsdbserver-nb-0"] Dec 03 12:02:31 crc kubenswrapper[5052]: I1203 12:02:31.795208 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:02:31 crc kubenswrapper[5052]: I1203 12:02:31.797399 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-ovn-metrics" Dec 03 12:02:31 crc kubenswrapper[5052]: I1203 12:02:31.797629 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-ovndbcluster-nb-ovndbs" Dec 03 12:02:31 crc kubenswrapper[5052]: I1203 12:02:31.798215 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ovncluster-ovndbcluster-nb-dockercfg-rzbdg" Dec 03 12:02:31 crc kubenswrapper[5052]: I1203 12:02:31.799089 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"ovndbcluster-nb-config" Dec 03 12:02:31 crc kubenswrapper[5052]: I1203 12:02:31.799655 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"ovndbcluster-nb-scripts" Dec 03 12:02:31 crc kubenswrapper[5052]: I1203 12:02:31.807991 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-nb-0"] Dec 03 12:02:31 crc kubenswrapper[5052]: I1203 12:02:31.966284 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-galera-2" event={"ID":"143f4f1d-4f91-4ffa-a277-d6e551d19838","Type":"ContainerStarted","Data":"a154e1968b2288216f60d8cd5f0368f47e36bcdef4f248bc7568b9b7174e4331"} Dec 03 12:02:31 crc kubenswrapper[5052]: I1203 12:02:31.971991 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-galera-0" event={"ID":"d66ed4ec-fd97-4a62-82cb-cee3fe877524","Type":"ContainerStarted","Data":"7e2247eb34fdd68938be37e473102ca45e588792a2cc0096e976b082bdaee0f9"} Dec 03 12:02:31 crc kubenswrapper[5052]: I1203 12:02:31.975075 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-galera-1" event={"ID":"6b92f88e-1325-4c3d-8d4e-4089a9033775","Type":"ContainerStarted","Data":"527038c3f4433583ba7d8bdd06bfde009abf48c5c09978d5bdd3df6912110859"} Dec 03 12:02:31 crc kubenswrapper[5052]: I1203 12:02:31.983215 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/327a4d19-96ba-4b5e-a19f-56ba15a0cc82-config\") pod \"ovsdbserver-nb-0\" (UID: \"327a4d19-96ba-4b5e-a19f-56ba15a0cc82\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:02:31 crc kubenswrapper[5052]: I1203 12:02:31.983365 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage19-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage19-crc\") pod \"ovsdbserver-nb-0\" (UID: \"327a4d19-96ba-4b5e-a19f-56ba15a0cc82\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:02:31 crc kubenswrapper[5052]: I1203 12:02:31.983427 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/327a4d19-96ba-4b5e-a19f-56ba15a0cc82-combined-ca-bundle\") pod \"ovsdbserver-nb-0\" (UID: \"327a4d19-96ba-4b5e-a19f-56ba15a0cc82\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:02:31 crc kubenswrapper[5052]: I1203 12:02:31.983499 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/327a4d19-96ba-4b5e-a19f-56ba15a0cc82-scripts\") pod \"ovsdbserver-nb-0\" (UID: \"327a4d19-96ba-4b5e-a19f-56ba15a0cc82\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:02:31 crc kubenswrapper[5052]: I1203 12:02:31.983640 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/327a4d19-96ba-4b5e-a19f-56ba15a0cc82-ovsdb-rundir\") pod \"ovsdbserver-nb-0\" (UID: \"327a4d19-96ba-4b5e-a19f-56ba15a0cc82\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:02:31 crc kubenswrapper[5052]: I1203 12:02:31.983716 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j7hd4\" (UniqueName: \"kubernetes.io/projected/327a4d19-96ba-4b5e-a19f-56ba15a0cc82-kube-api-access-j7hd4\") pod \"ovsdbserver-nb-0\" (UID: \"327a4d19-96ba-4b5e-a19f-56ba15a0cc82\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:02:31 crc kubenswrapper[5052]: I1203 12:02:31.983814 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovsdbserver-nb-tls-certs\" (UniqueName: \"kubernetes.io/secret/327a4d19-96ba-4b5e-a19f-56ba15a0cc82-ovsdbserver-nb-tls-certs\") pod \"ovsdbserver-nb-0\" (UID: \"327a4d19-96ba-4b5e-a19f-56ba15a0cc82\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:02:31 crc kubenswrapper[5052]: I1203 12:02:31.983897 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/327a4d19-96ba-4b5e-a19f-56ba15a0cc82-metrics-certs-tls-certs\") pod \"ovsdbserver-nb-0\" (UID: \"327a4d19-96ba-4b5e-a19f-56ba15a0cc82\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:02:31 crc kubenswrapper[5052]: I1203 12:02:31.996396 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/openstack-galera-2" podStartSLOduration=8.996360819 podStartE2EDuration="8.996360819s" podCreationTimestamp="2025-12-03 12:02:23 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:02:31.989918805 +0000 UTC m=+2542.578160579" watchObservedRunningTime="2025-12-03 12:02:31.996360819 +0000 UTC m=+2542.584602633" Dec 03 12:02:32 crc kubenswrapper[5052]: I1203 12:02:32.021976 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/openstack-galera-1" podStartSLOduration=9.021956141 podStartE2EDuration="9.021956141s" podCreationTimestamp="2025-12-03 12:02:23 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:02:32.01075363 +0000 UTC m=+2542.598995424" watchObservedRunningTime="2025-12-03 12:02:32.021956141 +0000 UTC m=+2542.610197915" Dec 03 12:02:32 crc kubenswrapper[5052]: I1203 12:02:32.044427 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/openstack-galera-0" podStartSLOduration=9.044374181 podStartE2EDuration="9.044374181s" podCreationTimestamp="2025-12-03 12:02:23 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:02:32.042825987 +0000 UTC m=+2542.631067771" watchObservedRunningTime="2025-12-03 12:02:32.044374181 +0000 UTC m=+2542.632615945" Dec 03 12:02:32 crc kubenswrapper[5052]: I1203 12:02:32.085976 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/327a4d19-96ba-4b5e-a19f-56ba15a0cc82-config\") pod \"ovsdbserver-nb-0\" (UID: \"327a4d19-96ba-4b5e-a19f-56ba15a0cc82\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:02:32 crc kubenswrapper[5052]: I1203 12:02:32.086063 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage19-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage19-crc\") pod \"ovsdbserver-nb-0\" (UID: \"327a4d19-96ba-4b5e-a19f-56ba15a0cc82\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:02:32 crc kubenswrapper[5052]: I1203 12:02:32.086079 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/327a4d19-96ba-4b5e-a19f-56ba15a0cc82-combined-ca-bundle\") pod \"ovsdbserver-nb-0\" (UID: \"327a4d19-96ba-4b5e-a19f-56ba15a0cc82\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:02:32 crc kubenswrapper[5052]: I1203 12:02:32.086115 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/327a4d19-96ba-4b5e-a19f-56ba15a0cc82-scripts\") pod \"ovsdbserver-nb-0\" (UID: \"327a4d19-96ba-4b5e-a19f-56ba15a0cc82\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:02:32 crc kubenswrapper[5052]: I1203 12:02:32.086267 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/327a4d19-96ba-4b5e-a19f-56ba15a0cc82-ovsdb-rundir\") pod \"ovsdbserver-nb-0\" (UID: \"327a4d19-96ba-4b5e-a19f-56ba15a0cc82\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:02:32 crc kubenswrapper[5052]: I1203 12:02:32.086298 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-j7hd4\" (UniqueName: \"kubernetes.io/projected/327a4d19-96ba-4b5e-a19f-56ba15a0cc82-kube-api-access-j7hd4\") pod \"ovsdbserver-nb-0\" (UID: \"327a4d19-96ba-4b5e-a19f-56ba15a0cc82\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:02:32 crc kubenswrapper[5052]: I1203 12:02:32.086363 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovsdbserver-nb-tls-certs\" (UniqueName: \"kubernetes.io/secret/327a4d19-96ba-4b5e-a19f-56ba15a0cc82-ovsdbserver-nb-tls-certs\") pod \"ovsdbserver-nb-0\" (UID: \"327a4d19-96ba-4b5e-a19f-56ba15a0cc82\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:02:32 crc kubenswrapper[5052]: I1203 12:02:32.086397 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/327a4d19-96ba-4b5e-a19f-56ba15a0cc82-metrics-certs-tls-certs\") pod \"ovsdbserver-nb-0\" (UID: \"327a4d19-96ba-4b5e-a19f-56ba15a0cc82\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:02:32 crc kubenswrapper[5052]: I1203 12:02:32.088665 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/327a4d19-96ba-4b5e-a19f-56ba15a0cc82-config\") pod \"ovsdbserver-nb-0\" (UID: \"327a4d19-96ba-4b5e-a19f-56ba15a0cc82\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:02:32 crc kubenswrapper[5052]: I1203 12:02:32.088870 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/327a4d19-96ba-4b5e-a19f-56ba15a0cc82-scripts\") pod \"ovsdbserver-nb-0\" (UID: \"327a4d19-96ba-4b5e-a19f-56ba15a0cc82\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:02:32 crc kubenswrapper[5052]: I1203 12:02:32.089244 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/327a4d19-96ba-4b5e-a19f-56ba15a0cc82-ovsdb-rundir\") pod \"ovsdbserver-nb-0\" (UID: \"327a4d19-96ba-4b5e-a19f-56ba15a0cc82\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:02:32 crc kubenswrapper[5052]: I1203 12:02:32.090060 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage19-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage19-crc\") pod \"ovsdbserver-nb-0\" (UID: \"327a4d19-96ba-4b5e-a19f-56ba15a0cc82\") device mount path \"/mnt/openstack/pv19\"" pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:02:32 crc kubenswrapper[5052]: I1203 12:02:32.093487 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/327a4d19-96ba-4b5e-a19f-56ba15a0cc82-metrics-certs-tls-certs\") pod \"ovsdbserver-nb-0\" (UID: \"327a4d19-96ba-4b5e-a19f-56ba15a0cc82\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:02:32 crc kubenswrapper[5052]: I1203 12:02:32.095249 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/327a4d19-96ba-4b5e-a19f-56ba15a0cc82-combined-ca-bundle\") pod \"ovsdbserver-nb-0\" (UID: \"327a4d19-96ba-4b5e-a19f-56ba15a0cc82\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:02:32 crc kubenswrapper[5052]: I1203 12:02:32.096716 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovsdbserver-nb-tls-certs\" (UniqueName: \"kubernetes.io/secret/327a4d19-96ba-4b5e-a19f-56ba15a0cc82-ovsdbserver-nb-tls-certs\") pod \"ovsdbserver-nb-0\" (UID: \"327a4d19-96ba-4b5e-a19f-56ba15a0cc82\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:02:32 crc kubenswrapper[5052]: I1203 12:02:32.109572 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-j7hd4\" (UniqueName: \"kubernetes.io/projected/327a4d19-96ba-4b5e-a19f-56ba15a0cc82-kube-api-access-j7hd4\") pod \"ovsdbserver-nb-0\" (UID: \"327a4d19-96ba-4b5e-a19f-56ba15a0cc82\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:02:32 crc kubenswrapper[5052]: I1203 12:02:32.119263 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage19-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage19-crc\") pod \"ovsdbserver-nb-0\" (UID: \"327a4d19-96ba-4b5e-a19f-56ba15a0cc82\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:02:32 crc kubenswrapper[5052]: I1203 12:02:32.417935 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:02:32 crc kubenswrapper[5052]: I1203 12:02:32.914764 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-nb-0"] Dec 03 12:02:32 crc kubenswrapper[5052]: W1203 12:02:32.916241 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod327a4d19_96ba_4b5e_a19f_56ba15a0cc82.slice/crio-487b73a468df70bd41429eee51087d37654668b064aa17d30ffaa4ebbede1a52 WatchSource:0}: Error finding container 487b73a468df70bd41429eee51087d37654668b064aa17d30ffaa4ebbede1a52: Status 404 returned error can't find the container with id 487b73a468df70bd41429eee51087d37654668b064aa17d30ffaa4ebbede1a52 Dec 03 12:02:32 crc kubenswrapper[5052]: I1203 12:02:32.983645 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-nb-0" event={"ID":"327a4d19-96ba-4b5e-a19f-56ba15a0cc82","Type":"ContainerStarted","Data":"487b73a468df70bd41429eee51087d37654668b064aa17d30ffaa4ebbede1a52"} Dec 03 12:02:33 crc kubenswrapper[5052]: I1203 12:02:33.683287 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ovsdbserver-sb-0"] Dec 03 12:02:33 crc kubenswrapper[5052]: I1203 12:02:33.684998 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:02:33 crc kubenswrapper[5052]: I1203 12:02:33.688312 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-ovndbcluster-sb-ovndbs" Dec 03 12:02:33 crc kubenswrapper[5052]: I1203 12:02:33.689059 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ovncluster-ovndbcluster-sb-dockercfg-sfmsw" Dec 03 12:02:33 crc kubenswrapper[5052]: I1203 12:02:33.689241 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"ovndbcluster-sb-config" Dec 03 12:02:33 crc kubenswrapper[5052]: I1203 12:02:33.689366 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"ovndbcluster-sb-scripts" Dec 03 12:02:33 crc kubenswrapper[5052]: I1203 12:02:33.691666 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-sb-0"] Dec 03 12:02:33 crc kubenswrapper[5052]: I1203 12:02:33.817484 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage17-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage17-crc\") pod \"ovsdbserver-sb-0\" (UID: \"78dad0d3-18e5-4900-932c-56ede3279dc0\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:02:33 crc kubenswrapper[5052]: I1203 12:02:33.817558 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/78dad0d3-18e5-4900-932c-56ede3279dc0-metrics-certs-tls-certs\") pod \"ovsdbserver-sb-0\" (UID: \"78dad0d3-18e5-4900-932c-56ede3279dc0\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:02:33 crc kubenswrapper[5052]: I1203 12:02:33.817603 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jp6lk\" (UniqueName: \"kubernetes.io/projected/78dad0d3-18e5-4900-932c-56ede3279dc0-kube-api-access-jp6lk\") pod \"ovsdbserver-sb-0\" (UID: \"78dad0d3-18e5-4900-932c-56ede3279dc0\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:02:33 crc kubenswrapper[5052]: I1203 12:02:33.817819 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/78dad0d3-18e5-4900-932c-56ede3279dc0-ovsdb-rundir\") pod \"ovsdbserver-sb-0\" (UID: \"78dad0d3-18e5-4900-932c-56ede3279dc0\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:02:33 crc kubenswrapper[5052]: I1203 12:02:33.818001 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/78dad0d3-18e5-4900-932c-56ede3279dc0-config\") pod \"ovsdbserver-sb-0\" (UID: \"78dad0d3-18e5-4900-932c-56ede3279dc0\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:02:33 crc kubenswrapper[5052]: I1203 12:02:33.818076 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/78dad0d3-18e5-4900-932c-56ede3279dc0-combined-ca-bundle\") pod \"ovsdbserver-sb-0\" (UID: \"78dad0d3-18e5-4900-932c-56ede3279dc0\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:02:33 crc kubenswrapper[5052]: I1203 12:02:33.818127 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovsdbserver-sb-tls-certs\" (UniqueName: \"kubernetes.io/secret/78dad0d3-18e5-4900-932c-56ede3279dc0-ovsdbserver-sb-tls-certs\") pod \"ovsdbserver-sb-0\" (UID: \"78dad0d3-18e5-4900-932c-56ede3279dc0\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:02:33 crc kubenswrapper[5052]: I1203 12:02:33.819689 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/78dad0d3-18e5-4900-932c-56ede3279dc0-scripts\") pod \"ovsdbserver-sb-0\" (UID: \"78dad0d3-18e5-4900-932c-56ede3279dc0\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:02:33 crc kubenswrapper[5052]: I1203 12:02:33.922611 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/78dad0d3-18e5-4900-932c-56ede3279dc0-ovsdb-rundir\") pod \"ovsdbserver-sb-0\" (UID: \"78dad0d3-18e5-4900-932c-56ede3279dc0\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:02:33 crc kubenswrapper[5052]: I1203 12:02:33.922671 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/78dad0d3-18e5-4900-932c-56ede3279dc0-config\") pod \"ovsdbserver-sb-0\" (UID: \"78dad0d3-18e5-4900-932c-56ede3279dc0\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:02:33 crc kubenswrapper[5052]: I1203 12:02:33.922693 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/78dad0d3-18e5-4900-932c-56ede3279dc0-combined-ca-bundle\") pod \"ovsdbserver-sb-0\" (UID: \"78dad0d3-18e5-4900-932c-56ede3279dc0\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:02:33 crc kubenswrapper[5052]: I1203 12:02:33.922711 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovsdbserver-sb-tls-certs\" (UniqueName: \"kubernetes.io/secret/78dad0d3-18e5-4900-932c-56ede3279dc0-ovsdbserver-sb-tls-certs\") pod \"ovsdbserver-sb-0\" (UID: \"78dad0d3-18e5-4900-932c-56ede3279dc0\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:02:33 crc kubenswrapper[5052]: I1203 12:02:33.922790 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/78dad0d3-18e5-4900-932c-56ede3279dc0-scripts\") pod \"ovsdbserver-sb-0\" (UID: \"78dad0d3-18e5-4900-932c-56ede3279dc0\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:02:33 crc kubenswrapper[5052]: I1203 12:02:33.922839 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage17-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage17-crc\") pod \"ovsdbserver-sb-0\" (UID: \"78dad0d3-18e5-4900-932c-56ede3279dc0\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:02:33 crc kubenswrapper[5052]: I1203 12:02:33.922857 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/78dad0d3-18e5-4900-932c-56ede3279dc0-metrics-certs-tls-certs\") pod \"ovsdbserver-sb-0\" (UID: \"78dad0d3-18e5-4900-932c-56ede3279dc0\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:02:33 crc kubenswrapper[5052]: I1203 12:02:33.922880 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-jp6lk\" (UniqueName: \"kubernetes.io/projected/78dad0d3-18e5-4900-932c-56ede3279dc0-kube-api-access-jp6lk\") pod \"ovsdbserver-sb-0\" (UID: \"78dad0d3-18e5-4900-932c-56ede3279dc0\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:02:33 crc kubenswrapper[5052]: I1203 12:02:33.923302 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage17-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage17-crc\") pod \"ovsdbserver-sb-0\" (UID: \"78dad0d3-18e5-4900-932c-56ede3279dc0\") device mount path \"/mnt/openstack/pv17\"" pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:02:33 crc kubenswrapper[5052]: I1203 12:02:33.923320 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/78dad0d3-18e5-4900-932c-56ede3279dc0-ovsdb-rundir\") pod \"ovsdbserver-sb-0\" (UID: \"78dad0d3-18e5-4900-932c-56ede3279dc0\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:02:33 crc kubenswrapper[5052]: I1203 12:02:33.923802 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/78dad0d3-18e5-4900-932c-56ede3279dc0-config\") pod \"ovsdbserver-sb-0\" (UID: \"78dad0d3-18e5-4900-932c-56ede3279dc0\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:02:33 crc kubenswrapper[5052]: I1203 12:02:33.924336 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/78dad0d3-18e5-4900-932c-56ede3279dc0-scripts\") pod \"ovsdbserver-sb-0\" (UID: \"78dad0d3-18e5-4900-932c-56ede3279dc0\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:02:33 crc kubenswrapper[5052]: I1203 12:02:33.927072 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovsdbserver-sb-tls-certs\" (UniqueName: \"kubernetes.io/secret/78dad0d3-18e5-4900-932c-56ede3279dc0-ovsdbserver-sb-tls-certs\") pod \"ovsdbserver-sb-0\" (UID: \"78dad0d3-18e5-4900-932c-56ede3279dc0\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:02:33 crc kubenswrapper[5052]: I1203 12:02:33.927471 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/78dad0d3-18e5-4900-932c-56ede3279dc0-combined-ca-bundle\") pod \"ovsdbserver-sb-0\" (UID: \"78dad0d3-18e5-4900-932c-56ede3279dc0\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:02:33 crc kubenswrapper[5052]: I1203 12:02:33.928272 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/78dad0d3-18e5-4900-932c-56ede3279dc0-metrics-certs-tls-certs\") pod \"ovsdbserver-sb-0\" (UID: \"78dad0d3-18e5-4900-932c-56ede3279dc0\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:02:33 crc kubenswrapper[5052]: I1203 12:02:33.941934 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-jp6lk\" (UniqueName: \"kubernetes.io/projected/78dad0d3-18e5-4900-932c-56ede3279dc0-kube-api-access-jp6lk\") pod \"ovsdbserver-sb-0\" (UID: \"78dad0d3-18e5-4900-932c-56ede3279dc0\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:02:33 crc kubenswrapper[5052]: I1203 12:02:33.966975 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage17-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage17-crc\") pod \"ovsdbserver-sb-0\" (UID: \"78dad0d3-18e5-4900-932c-56ede3279dc0\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:02:33 crc kubenswrapper[5052]: I1203 12:02:33.993603 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-nb-0" event={"ID":"327a4d19-96ba-4b5e-a19f-56ba15a0cc82","Type":"ContainerStarted","Data":"023f6e55b1e44cead794ab88e34aa43141c551b1a2bce50e249cb4eeec4d2009"} Dec 03 12:02:33 crc kubenswrapper[5052]: I1203 12:02:33.993658 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-nb-0" event={"ID":"327a4d19-96ba-4b5e-a19f-56ba15a0cc82","Type":"ContainerStarted","Data":"04c71fa1d03c17ad50a53dfb4585c12716ea5f19b448fb01a9fbd6d86935e208"} Dec 03 12:02:34 crc kubenswrapper[5052]: I1203 12:02:34.011945 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:02:34 crc kubenswrapper[5052]: I1203 12:02:34.016605 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/ovsdbserver-nb-0" podStartSLOduration=4.016587076 podStartE2EDuration="4.016587076s" podCreationTimestamp="2025-12-03 12:02:30 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:02:34.011628524 +0000 UTC m=+2544.599870308" watchObservedRunningTime="2025-12-03 12:02:34.016587076 +0000 UTC m=+2544.604828840" Dec 03 12:02:34 crc kubenswrapper[5052]: I1203 12:02:34.527934 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/openstack-galera-1" Dec 03 12:02:34 crc kubenswrapper[5052]: I1203 12:02:34.528395 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/openstack-galera-1" Dec 03 12:02:34 crc kubenswrapper[5052]: I1203 12:02:34.533929 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/openstack-galera-2" Dec 03 12:02:34 crc kubenswrapper[5052]: I1203 12:02:34.534141 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/openstack-galera-2" Dec 03 12:02:34 crc kubenswrapper[5052]: I1203 12:02:34.563123 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-sb-0"] Dec 03 12:02:34 crc kubenswrapper[5052]: E1203 12:02:34.707126 5052 cadvisor_stats_provider.go:516] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6ac95bd3_125c_4966_b9a7_4eb4a7842c67.slice/crio-conmon-6f25f4851b71de7e5136ec08e6aa9f8b2530fdd3a88edfd5b32d7294b08cdcfa.scope\": RecentStats: unable to find data in memory cache]" Dec 03 12:02:34 crc kubenswrapper[5052]: I1203 12:02:34.812201 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:02:34 crc kubenswrapper[5052]: I1203 12:02:34.812437 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:02:35 crc kubenswrapper[5052]: I1203 12:02:35.002941 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-sb-0" event={"ID":"78dad0d3-18e5-4900-932c-56ede3279dc0","Type":"ContainerStarted","Data":"0cc27cb717dc9537f83778189f76fdddfa28ca3f3a6e4b31159ccfb03fb8e97a"} Dec 03 12:02:35 crc kubenswrapper[5052]: I1203 12:02:35.004848 5052 generic.go:334] "Generic (PLEG): container finished" podID="6ac95bd3-125c-4966-b9a7-4eb4a7842c67" containerID="6f25f4851b71de7e5136ec08e6aa9f8b2530fdd3a88edfd5b32d7294b08cdcfa" exitCode=0 Dec 03 12:02:35 crc kubenswrapper[5052]: I1203 12:02:35.004939 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-cell1-galera-2" event={"ID":"6ac95bd3-125c-4966-b9a7-4eb4a7842c67","Type":"ContainerDied","Data":"6f25f4851b71de7e5136ec08e6aa9f8b2530fdd3a88edfd5b32d7294b08cdcfa"} Dec 03 12:02:35 crc kubenswrapper[5052]: I1203 12:02:35.419052 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:02:36 crc kubenswrapper[5052]: I1203 12:02:36.016636 5052 generic.go:334] "Generic (PLEG): container finished" podID="3b8ce5b7-67a1-48ef-892f-7408010a84f0" containerID="fc6a07b3ee2f8d37a641173f6de8b77d45fd5bc8c9701900d684dde2e6532818" exitCode=0 Dec 03 12:02:36 crc kubenswrapper[5052]: I1203 12:02:36.016743 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-cell1-galera-0" event={"ID":"3b8ce5b7-67a1-48ef-892f-7408010a84f0","Type":"ContainerDied","Data":"fc6a07b3ee2f8d37a641173f6de8b77d45fd5bc8c9701900d684dde2e6532818"} Dec 03 12:02:36 crc kubenswrapper[5052]: I1203 12:02:36.020867 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-cell1-galera-2" event={"ID":"6ac95bd3-125c-4966-b9a7-4eb4a7842c67","Type":"ContainerStarted","Data":"edf48c4037d4be3adfc21b5d97a50d68b5261315fe92282773dd8556852ce51a"} Dec 03 12:02:36 crc kubenswrapper[5052]: I1203 12:02:36.024296 5052 generic.go:334] "Generic (PLEG): container finished" podID="b34bb4d0-7906-42e8-a9aa-bd1a4203a312" containerID="cfa78e68ee04705ba5099d5662bbfe0cc1eb410ee1f31655537d62c888b13793" exitCode=0 Dec 03 12:02:36 crc kubenswrapper[5052]: I1203 12:02:36.024362 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-cell1-galera-1" event={"ID":"b34bb4d0-7906-42e8-a9aa-bd1a4203a312","Type":"ContainerDied","Data":"cfa78e68ee04705ba5099d5662bbfe0cc1eb410ee1f31655537d62c888b13793"} Dec 03 12:02:36 crc kubenswrapper[5052]: I1203 12:02:36.028253 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-sb-0" event={"ID":"78dad0d3-18e5-4900-932c-56ede3279dc0","Type":"ContainerStarted","Data":"b746c813f9a2abaab7eee298828559f29ccd2f1796875dbbe947a37faa740dfb"} Dec 03 12:02:36 crc kubenswrapper[5052]: I1203 12:02:36.028307 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-sb-0" event={"ID":"78dad0d3-18e5-4900-932c-56ede3279dc0","Type":"ContainerStarted","Data":"3fa9a29a156d24f947438d4dd3ac36264903dbd4e777dddc9352f1aab01266e5"} Dec 03 12:02:36 crc kubenswrapper[5052]: I1203 12:02:36.086021 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/openstack-cell1-galera-2" podStartSLOduration=12.085999148 podStartE2EDuration="12.085999148s" podCreationTimestamp="2025-12-03 12:02:24 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:02:36.075090616 +0000 UTC m=+2546.663332390" watchObservedRunningTime="2025-12-03 12:02:36.085999148 +0000 UTC m=+2546.674240932" Dec 03 12:02:36 crc kubenswrapper[5052]: I1203 12:02:36.226310 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/openstack-cell1-galera-2" Dec 03 12:02:36 crc kubenswrapper[5052]: I1203 12:02:36.226374 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/openstack-cell1-galera-2" Dec 03 12:02:36 crc kubenswrapper[5052]: I1203 12:02:36.514873 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/memcached-0" Dec 03 12:02:36 crc kubenswrapper[5052]: I1203 12:02:36.541150 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/ovsdbserver-sb-0" podStartSLOduration=4.541128539 podStartE2EDuration="4.541128539s" podCreationTimestamp="2025-12-03 12:02:32 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:02:36.102038816 +0000 UTC m=+2546.690280580" watchObservedRunningTime="2025-12-03 12:02:36.541128539 +0000 UTC m=+2547.129370303" Dec 03 12:02:37 crc kubenswrapper[5052]: I1203 12:02:37.012109 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:02:37 crc kubenswrapper[5052]: I1203 12:02:37.039299 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-cell1-galera-0" event={"ID":"3b8ce5b7-67a1-48ef-892f-7408010a84f0","Type":"ContainerStarted","Data":"5a5421259d7095f5c7b35c6d5fcacfbf43056034387141eb0830f2281a204092"} Dec 03 12:02:37 crc kubenswrapper[5052]: I1203 12:02:37.043795 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-cell1-galera-1" event={"ID":"b34bb4d0-7906-42e8-a9aa-bd1a4203a312","Type":"ContainerStarted","Data":"c55564d5848c1a333ba2db20a9c5ecdbf152ba4ebaa847c6e28f6ed69c3a99ed"} Dec 03 12:02:37 crc kubenswrapper[5052]: I1203 12:02:37.062346 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/openstack-cell1-galera-0" podStartSLOduration=13.06233038 podStartE2EDuration="13.06233038s" podCreationTimestamp="2025-12-03 12:02:24 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:02:37.057584805 +0000 UTC m=+2547.645826579" watchObservedRunningTime="2025-12-03 12:02:37.06233038 +0000 UTC m=+2547.650572144" Dec 03 12:02:37 crc kubenswrapper[5052]: I1203 12:02:37.082588 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/openstack-cell1-galera-1" podStartSLOduration=13.082566709 podStartE2EDuration="13.082566709s" podCreationTimestamp="2025-12-03 12:02:24 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:02:37.076209787 +0000 UTC m=+2547.664451561" watchObservedRunningTime="2025-12-03 12:02:37.082566709 +0000 UTC m=+2547.670808473" Dec 03 12:02:37 crc kubenswrapper[5052]: I1203 12:02:37.418947 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:02:37 crc kubenswrapper[5052]: I1203 12:02:37.673024 5052 scope.go:117] "RemoveContainer" containerID="d224717c6d9ca8135fd510e03a6c40c9f2b1a56dd8b4a3b685af3e2dfa855982" Dec 03 12:02:37 crc kubenswrapper[5052]: I1203 12:02:37.874206 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:02:38 crc kubenswrapper[5052]: I1203 12:02:38.055192 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" event={"ID":"10c237bc-72e7-4da3-9f2c-b1688395bf6e","Type":"ContainerStarted","Data":"fcd7a4f50449fc80828465d45d26c57ddd80c8ab4f967dd642dae3c503a27345"} Dec 03 12:02:38 crc kubenswrapper[5052]: I1203 12:02:38.464388 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:02:38 crc kubenswrapper[5052]: I1203 12:02:38.523558 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:02:38 crc kubenswrapper[5052]: I1203 12:02:38.877626 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/swift-storage-0"] Dec 03 12:02:38 crc kubenswrapper[5052]: I1203 12:02:38.883930 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:02:38 crc kubenswrapper[5052]: I1203 12:02:38.886363 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"swift-conf" Dec 03 12:02:38 crc kubenswrapper[5052]: I1203 12:02:38.886978 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"swift-storage-config-data" Dec 03 12:02:38 crc kubenswrapper[5052]: I1203 12:02:38.887264 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"swift-swift-dockercfg-r9nhm" Dec 03 12:02:38 crc kubenswrapper[5052]: I1203 12:02:38.888091 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"swift-ring-files" Dec 03 12:02:38 crc kubenswrapper[5052]: I1203 12:02:38.905289 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/swift-storage-0"] Dec 03 12:02:38 crc kubenswrapper[5052]: I1203 12:02:38.924524 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage04-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage04-crc\") pod \"swift-storage-0\" (UID: \"95bf5f72-82f3-4e88-a576-01dae857541f\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:02:38 crc kubenswrapper[5052]: I1203 12:02:38.924606 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lock\" (UniqueName: \"kubernetes.io/empty-dir/95bf5f72-82f3-4e88-a576-01dae857541f-lock\") pod \"swift-storage-0\" (UID: \"95bf5f72-82f3-4e88-a576-01dae857541f\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:02:38 crc kubenswrapper[5052]: I1203 12:02:38.924644 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cache\" (UniqueName: \"kubernetes.io/empty-dir/95bf5f72-82f3-4e88-a576-01dae857541f-cache\") pod \"swift-storage-0\" (UID: \"95bf5f72-82f3-4e88-a576-01dae857541f\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:02:38 crc kubenswrapper[5052]: I1203 12:02:38.924710 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/95bf5f72-82f3-4e88-a576-01dae857541f-etc-swift\") pod \"swift-storage-0\" (UID: \"95bf5f72-82f3-4e88-a576-01dae857541f\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:02:38 crc kubenswrapper[5052]: I1203 12:02:38.924756 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-98m8t\" (UniqueName: \"kubernetes.io/projected/95bf5f72-82f3-4e88-a576-01dae857541f-kube-api-access-98m8t\") pod \"swift-storage-0\" (UID: \"95bf5f72-82f3-4e88-a576-01dae857541f\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:02:39 crc kubenswrapper[5052]: I1203 12:02:39.012616 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:02:39 crc kubenswrapper[5052]: I1203 12:02:39.025851 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"lock\" (UniqueName: \"kubernetes.io/empty-dir/95bf5f72-82f3-4e88-a576-01dae857541f-lock\") pod \"swift-storage-0\" (UID: \"95bf5f72-82f3-4e88-a576-01dae857541f\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:02:39 crc kubenswrapper[5052]: I1203 12:02:39.025918 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"cache\" (UniqueName: \"kubernetes.io/empty-dir/95bf5f72-82f3-4e88-a576-01dae857541f-cache\") pod \"swift-storage-0\" (UID: \"95bf5f72-82f3-4e88-a576-01dae857541f\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:02:39 crc kubenswrapper[5052]: I1203 12:02:39.025987 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/95bf5f72-82f3-4e88-a576-01dae857541f-etc-swift\") pod \"swift-storage-0\" (UID: \"95bf5f72-82f3-4e88-a576-01dae857541f\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:02:39 crc kubenswrapper[5052]: I1203 12:02:39.026022 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-98m8t\" (UniqueName: \"kubernetes.io/projected/95bf5f72-82f3-4e88-a576-01dae857541f-kube-api-access-98m8t\") pod \"swift-storage-0\" (UID: \"95bf5f72-82f3-4e88-a576-01dae857541f\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:02:39 crc kubenswrapper[5052]: I1203 12:02:39.026069 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage04-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage04-crc\") pod \"swift-storage-0\" (UID: \"95bf5f72-82f3-4e88-a576-01dae857541f\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:02:39 crc kubenswrapper[5052]: E1203 12:02:39.026156 5052 projected.go:288] Couldn't get configMap openstack-kuttl-tests/swift-ring-files: configmap "swift-ring-files" not found Dec 03 12:02:39 crc kubenswrapper[5052]: E1203 12:02:39.026189 5052 projected.go:194] Error preparing data for projected volume etc-swift for pod openstack-kuttl-tests/swift-storage-0: configmap "swift-ring-files" not found Dec 03 12:02:39 crc kubenswrapper[5052]: E1203 12:02:39.026253 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/95bf5f72-82f3-4e88-a576-01dae857541f-etc-swift podName:95bf5f72-82f3-4e88-a576-01dae857541f nodeName:}" failed. No retries permitted until 2025-12-03 12:02:39.526229146 +0000 UTC m=+2550.114470910 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "etc-swift" (UniqueName: "kubernetes.io/projected/95bf5f72-82f3-4e88-a576-01dae857541f-etc-swift") pod "swift-storage-0" (UID: "95bf5f72-82f3-4e88-a576-01dae857541f") : configmap "swift-ring-files" not found Dec 03 12:02:39 crc kubenswrapper[5052]: I1203 12:02:39.026373 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage04-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage04-crc\") pod \"swift-storage-0\" (UID: \"95bf5f72-82f3-4e88-a576-01dae857541f\") device mount path \"/mnt/openstack/pv04\"" pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:02:39 crc kubenswrapper[5052]: I1203 12:02:39.026638 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"cache\" (UniqueName: \"kubernetes.io/empty-dir/95bf5f72-82f3-4e88-a576-01dae857541f-cache\") pod \"swift-storage-0\" (UID: \"95bf5f72-82f3-4e88-a576-01dae857541f\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:02:39 crc kubenswrapper[5052]: I1203 12:02:39.026659 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"lock\" (UniqueName: \"kubernetes.io/empty-dir/95bf5f72-82f3-4e88-a576-01dae857541f-lock\") pod \"swift-storage-0\" (UID: \"95bf5f72-82f3-4e88-a576-01dae857541f\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:02:39 crc kubenswrapper[5052]: I1203 12:02:39.050354 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage04-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage04-crc\") pod \"swift-storage-0\" (UID: \"95bf5f72-82f3-4e88-a576-01dae857541f\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:02:39 crc kubenswrapper[5052]: I1203 12:02:39.050810 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-98m8t\" (UniqueName: \"kubernetes.io/projected/95bf5f72-82f3-4e88-a576-01dae857541f-kube-api-access-98m8t\") pod \"swift-storage-0\" (UID: \"95bf5f72-82f3-4e88-a576-01dae857541f\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:02:39 crc kubenswrapper[5052]: I1203 12:02:39.195002 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/swift-ring-rebalance-z4x9q"] Dec 03 12:02:39 crc kubenswrapper[5052]: I1203 12:02:39.196305 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-ring-rebalance-z4x9q" Dec 03 12:02:39 crc kubenswrapper[5052]: I1203 12:02:39.197725 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"swift-proxy-config-data" Dec 03 12:02:39 crc kubenswrapper[5052]: I1203 12:02:39.199120 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"swift-ring-scripts" Dec 03 12:02:39 crc kubenswrapper[5052]: I1203 12:02:39.204006 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"swift-ring-config-data" Dec 03 12:02:39 crc kubenswrapper[5052]: I1203 12:02:39.241766 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/swift-ring-rebalance-g9qd6"] Dec 03 12:02:39 crc kubenswrapper[5052]: I1203 12:02:39.242933 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-ring-rebalance-g9qd6" Dec 03 12:02:39 crc kubenswrapper[5052]: I1203 12:02:39.263182 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/swift-ring-rebalance-z4x9q"] Dec 03 12:02:39 crc kubenswrapper[5052]: I1203 12:02:39.263265 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/swift-ring-rebalance-g9qd6"] Dec 03 12:02:39 crc kubenswrapper[5052]: E1203 12:02:39.264849 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[combined-ca-bundle dispersionconf etc-swift kube-api-access-rclt2 ring-data-devices scripts swiftconf], unattached volumes=[], failed to process volumes=[]: context canceled" pod="openstack-kuttl-tests/swift-ring-rebalance-z4x9q" podUID="9ba25edf-3f86-4a57-9f39-e217216916d3" Dec 03 12:02:39 crc kubenswrapper[5052]: I1203 12:02:39.308096 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/swift-ring-rebalance-z4x9q"] Dec 03 12:02:39 crc kubenswrapper[5052]: I1203 12:02:39.331483 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/empty-dir/9ba25edf-3f86-4a57-9f39-e217216916d3-etc-swift\") pod \"swift-ring-rebalance-z4x9q\" (UID: \"9ba25edf-3f86-4a57-9f39-e217216916d3\") " pod="openstack-kuttl-tests/swift-ring-rebalance-z4x9q" Dec 03 12:02:39 crc kubenswrapper[5052]: I1203 12:02:39.331627 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/9ba25edf-3f86-4a57-9f39-e217216916d3-scripts\") pod \"swift-ring-rebalance-z4x9q\" (UID: \"9ba25edf-3f86-4a57-9f39-e217216916d3\") " pod="openstack-kuttl-tests/swift-ring-rebalance-z4x9q" Dec 03 12:02:39 crc kubenswrapper[5052]: I1203 12:02:39.331652 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ring-data-devices\" (UniqueName: \"kubernetes.io/configmap/9ba25edf-3f86-4a57-9f39-e217216916d3-ring-data-devices\") pod \"swift-ring-rebalance-z4x9q\" (UID: \"9ba25edf-3f86-4a57-9f39-e217216916d3\") " pod="openstack-kuttl-tests/swift-ring-rebalance-z4x9q" Dec 03 12:02:39 crc kubenswrapper[5052]: I1203 12:02:39.331819 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9ba25edf-3f86-4a57-9f39-e217216916d3-combined-ca-bundle\") pod \"swift-ring-rebalance-z4x9q\" (UID: \"9ba25edf-3f86-4a57-9f39-e217216916d3\") " pod="openstack-kuttl-tests/swift-ring-rebalance-z4x9q" Dec 03 12:02:39 crc kubenswrapper[5052]: I1203 12:02:39.331868 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rclt2\" (UniqueName: \"kubernetes.io/projected/9ba25edf-3f86-4a57-9f39-e217216916d3-kube-api-access-rclt2\") pod \"swift-ring-rebalance-z4x9q\" (UID: \"9ba25edf-3f86-4a57-9f39-e217216916d3\") " pod="openstack-kuttl-tests/swift-ring-rebalance-z4x9q" Dec 03 12:02:39 crc kubenswrapper[5052]: I1203 12:02:39.331888 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"swiftconf\" (UniqueName: \"kubernetes.io/secret/9ba25edf-3f86-4a57-9f39-e217216916d3-swiftconf\") pod \"swift-ring-rebalance-z4x9q\" (UID: \"9ba25edf-3f86-4a57-9f39-e217216916d3\") " pod="openstack-kuttl-tests/swift-ring-rebalance-z4x9q" Dec 03 12:02:39 crc kubenswrapper[5052]: I1203 12:02:39.331951 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dispersionconf\" (UniqueName: \"kubernetes.io/secret/9ba25edf-3f86-4a57-9f39-e217216916d3-dispersionconf\") pod \"swift-ring-rebalance-z4x9q\" (UID: \"9ba25edf-3f86-4a57-9f39-e217216916d3\") " pod="openstack-kuttl-tests/swift-ring-rebalance-z4x9q" Dec 03 12:02:39 crc kubenswrapper[5052]: I1203 12:02:39.433835 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/empty-dir/9ba25edf-3f86-4a57-9f39-e217216916d3-etc-swift\") pod \"swift-ring-rebalance-z4x9q\" (UID: \"9ba25edf-3f86-4a57-9f39-e217216916d3\") " pod="openstack-kuttl-tests/swift-ring-rebalance-z4x9q" Dec 03 12:02:39 crc kubenswrapper[5052]: I1203 12:02:39.433916 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/907d9f47-ab0b-44c8-9aa6-3d135e1c312d-scripts\") pod \"swift-ring-rebalance-g9qd6\" (UID: \"907d9f47-ab0b-44c8-9aa6-3d135e1c312d\") " pod="openstack-kuttl-tests/swift-ring-rebalance-g9qd6" Dec 03 12:02:39 crc kubenswrapper[5052]: I1203 12:02:39.433998 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ring-data-devices\" (UniqueName: \"kubernetes.io/configmap/9ba25edf-3f86-4a57-9f39-e217216916d3-ring-data-devices\") pod \"swift-ring-rebalance-z4x9q\" (UID: \"9ba25edf-3f86-4a57-9f39-e217216916d3\") " pod="openstack-kuttl-tests/swift-ring-rebalance-z4x9q" Dec 03 12:02:39 crc kubenswrapper[5052]: I1203 12:02:39.434023 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/9ba25edf-3f86-4a57-9f39-e217216916d3-scripts\") pod \"swift-ring-rebalance-z4x9q\" (UID: \"9ba25edf-3f86-4a57-9f39-e217216916d3\") " pod="openstack-kuttl-tests/swift-ring-rebalance-z4x9q" Dec 03 12:02:39 crc kubenswrapper[5052]: I1203 12:02:39.434050 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ring-data-devices\" (UniqueName: \"kubernetes.io/configmap/907d9f47-ab0b-44c8-9aa6-3d135e1c312d-ring-data-devices\") pod \"swift-ring-rebalance-g9qd6\" (UID: \"907d9f47-ab0b-44c8-9aa6-3d135e1c312d\") " pod="openstack-kuttl-tests/swift-ring-rebalance-g9qd6" Dec 03 12:02:39 crc kubenswrapper[5052]: I1203 12:02:39.434087 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dispersionconf\" (UniqueName: \"kubernetes.io/secret/907d9f47-ab0b-44c8-9aa6-3d135e1c312d-dispersionconf\") pod \"swift-ring-rebalance-g9qd6\" (UID: \"907d9f47-ab0b-44c8-9aa6-3d135e1c312d\") " pod="openstack-kuttl-tests/swift-ring-rebalance-g9qd6" Dec 03 12:02:39 crc kubenswrapper[5052]: I1203 12:02:39.434135 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/empty-dir/907d9f47-ab0b-44c8-9aa6-3d135e1c312d-etc-swift\") pod \"swift-ring-rebalance-g9qd6\" (UID: \"907d9f47-ab0b-44c8-9aa6-3d135e1c312d\") " pod="openstack-kuttl-tests/swift-ring-rebalance-g9qd6" Dec 03 12:02:39 crc kubenswrapper[5052]: I1203 12:02:39.434182 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jzbfh\" (UniqueName: \"kubernetes.io/projected/907d9f47-ab0b-44c8-9aa6-3d135e1c312d-kube-api-access-jzbfh\") pod \"swift-ring-rebalance-g9qd6\" (UID: \"907d9f47-ab0b-44c8-9aa6-3d135e1c312d\") " pod="openstack-kuttl-tests/swift-ring-rebalance-g9qd6" Dec 03 12:02:39 crc kubenswrapper[5052]: I1203 12:02:39.434211 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9ba25edf-3f86-4a57-9f39-e217216916d3-combined-ca-bundle\") pod \"swift-ring-rebalance-z4x9q\" (UID: \"9ba25edf-3f86-4a57-9f39-e217216916d3\") " pod="openstack-kuttl-tests/swift-ring-rebalance-z4x9q" Dec 03 12:02:39 crc kubenswrapper[5052]: I1203 12:02:39.434232 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-rclt2\" (UniqueName: \"kubernetes.io/projected/9ba25edf-3f86-4a57-9f39-e217216916d3-kube-api-access-rclt2\") pod \"swift-ring-rebalance-z4x9q\" (UID: \"9ba25edf-3f86-4a57-9f39-e217216916d3\") " pod="openstack-kuttl-tests/swift-ring-rebalance-z4x9q" Dec 03 12:02:39 crc kubenswrapper[5052]: I1203 12:02:39.434255 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"swiftconf\" (UniqueName: \"kubernetes.io/secret/9ba25edf-3f86-4a57-9f39-e217216916d3-swiftconf\") pod \"swift-ring-rebalance-z4x9q\" (UID: \"9ba25edf-3f86-4a57-9f39-e217216916d3\") " pod="openstack-kuttl-tests/swift-ring-rebalance-z4x9q" Dec 03 12:02:39 crc kubenswrapper[5052]: I1203 12:02:39.434283 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"swiftconf\" (UniqueName: \"kubernetes.io/secret/907d9f47-ab0b-44c8-9aa6-3d135e1c312d-swiftconf\") pod \"swift-ring-rebalance-g9qd6\" (UID: \"907d9f47-ab0b-44c8-9aa6-3d135e1c312d\") " pod="openstack-kuttl-tests/swift-ring-rebalance-g9qd6" Dec 03 12:02:39 crc kubenswrapper[5052]: I1203 12:02:39.434335 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"dispersionconf\" (UniqueName: \"kubernetes.io/secret/9ba25edf-3f86-4a57-9f39-e217216916d3-dispersionconf\") pod \"swift-ring-rebalance-z4x9q\" (UID: \"9ba25edf-3f86-4a57-9f39-e217216916d3\") " pod="openstack-kuttl-tests/swift-ring-rebalance-z4x9q" Dec 03 12:02:39 crc kubenswrapper[5052]: I1203 12:02:39.434368 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/907d9f47-ab0b-44c8-9aa6-3d135e1c312d-combined-ca-bundle\") pod \"swift-ring-rebalance-g9qd6\" (UID: \"907d9f47-ab0b-44c8-9aa6-3d135e1c312d\") " pod="openstack-kuttl-tests/swift-ring-rebalance-g9qd6" Dec 03 12:02:39 crc kubenswrapper[5052]: I1203 12:02:39.434545 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"etc-swift\" (UniqueName: \"kubernetes.io/empty-dir/9ba25edf-3f86-4a57-9f39-e217216916d3-etc-swift\") pod \"swift-ring-rebalance-z4x9q\" (UID: \"9ba25edf-3f86-4a57-9f39-e217216916d3\") " pod="openstack-kuttl-tests/swift-ring-rebalance-z4x9q" Dec 03 12:02:39 crc kubenswrapper[5052]: I1203 12:02:39.435234 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ring-data-devices\" (UniqueName: \"kubernetes.io/configmap/9ba25edf-3f86-4a57-9f39-e217216916d3-ring-data-devices\") pod \"swift-ring-rebalance-z4x9q\" (UID: \"9ba25edf-3f86-4a57-9f39-e217216916d3\") " pod="openstack-kuttl-tests/swift-ring-rebalance-z4x9q" Dec 03 12:02:39 crc kubenswrapper[5052]: I1203 12:02:39.435888 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/9ba25edf-3f86-4a57-9f39-e217216916d3-scripts\") pod \"swift-ring-rebalance-z4x9q\" (UID: \"9ba25edf-3f86-4a57-9f39-e217216916d3\") " pod="openstack-kuttl-tests/swift-ring-rebalance-z4x9q" Dec 03 12:02:39 crc kubenswrapper[5052]: I1203 12:02:39.439501 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"swiftconf\" (UniqueName: \"kubernetes.io/secret/9ba25edf-3f86-4a57-9f39-e217216916d3-swiftconf\") pod \"swift-ring-rebalance-z4x9q\" (UID: \"9ba25edf-3f86-4a57-9f39-e217216916d3\") " pod="openstack-kuttl-tests/swift-ring-rebalance-z4x9q" Dec 03 12:02:39 crc kubenswrapper[5052]: I1203 12:02:39.440783 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"dispersionconf\" (UniqueName: \"kubernetes.io/secret/9ba25edf-3f86-4a57-9f39-e217216916d3-dispersionconf\") pod \"swift-ring-rebalance-z4x9q\" (UID: \"9ba25edf-3f86-4a57-9f39-e217216916d3\") " pod="openstack-kuttl-tests/swift-ring-rebalance-z4x9q" Dec 03 12:02:39 crc kubenswrapper[5052]: I1203 12:02:39.447248 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9ba25edf-3f86-4a57-9f39-e217216916d3-combined-ca-bundle\") pod \"swift-ring-rebalance-z4x9q\" (UID: \"9ba25edf-3f86-4a57-9f39-e217216916d3\") " pod="openstack-kuttl-tests/swift-ring-rebalance-z4x9q" Dec 03 12:02:39 crc kubenswrapper[5052]: I1203 12:02:39.452311 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-rclt2\" (UniqueName: \"kubernetes.io/projected/9ba25edf-3f86-4a57-9f39-e217216916d3-kube-api-access-rclt2\") pod \"swift-ring-rebalance-z4x9q\" (UID: \"9ba25edf-3f86-4a57-9f39-e217216916d3\") " pod="openstack-kuttl-tests/swift-ring-rebalance-z4x9q" Dec 03 12:02:39 crc kubenswrapper[5052]: E1203 12:02:39.503625 5052 upgradeaware.go:427] Error proxying data from client to backend: readfrom tcp 38.129.56.252:51184->38.129.56.252:34849: write tcp 38.129.56.252:51184->38.129.56.252:34849: write: broken pipe Dec 03 12:02:39 crc kubenswrapper[5052]: I1203 12:02:39.536273 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"swiftconf\" (UniqueName: \"kubernetes.io/secret/907d9f47-ab0b-44c8-9aa6-3d135e1c312d-swiftconf\") pod \"swift-ring-rebalance-g9qd6\" (UID: \"907d9f47-ab0b-44c8-9aa6-3d135e1c312d\") " pod="openstack-kuttl-tests/swift-ring-rebalance-g9qd6" Dec 03 12:02:39 crc kubenswrapper[5052]: I1203 12:02:39.536354 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/907d9f47-ab0b-44c8-9aa6-3d135e1c312d-combined-ca-bundle\") pod \"swift-ring-rebalance-g9qd6\" (UID: \"907d9f47-ab0b-44c8-9aa6-3d135e1c312d\") " pod="openstack-kuttl-tests/swift-ring-rebalance-g9qd6" Dec 03 12:02:39 crc kubenswrapper[5052]: I1203 12:02:39.536420 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/907d9f47-ab0b-44c8-9aa6-3d135e1c312d-scripts\") pod \"swift-ring-rebalance-g9qd6\" (UID: \"907d9f47-ab0b-44c8-9aa6-3d135e1c312d\") " pod="openstack-kuttl-tests/swift-ring-rebalance-g9qd6" Dec 03 12:02:39 crc kubenswrapper[5052]: I1203 12:02:39.537697 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ring-data-devices\" (UniqueName: \"kubernetes.io/configmap/907d9f47-ab0b-44c8-9aa6-3d135e1c312d-ring-data-devices\") pod \"swift-ring-rebalance-g9qd6\" (UID: \"907d9f47-ab0b-44c8-9aa6-3d135e1c312d\") " pod="openstack-kuttl-tests/swift-ring-rebalance-g9qd6" Dec 03 12:02:39 crc kubenswrapper[5052]: I1203 12:02:39.537744 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"dispersionconf\" (UniqueName: \"kubernetes.io/secret/907d9f47-ab0b-44c8-9aa6-3d135e1c312d-dispersionconf\") pod \"swift-ring-rebalance-g9qd6\" (UID: \"907d9f47-ab0b-44c8-9aa6-3d135e1c312d\") " pod="openstack-kuttl-tests/swift-ring-rebalance-g9qd6" Dec 03 12:02:39 crc kubenswrapper[5052]: I1203 12:02:39.537774 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/95bf5f72-82f3-4e88-a576-01dae857541f-etc-swift\") pod \"swift-storage-0\" (UID: \"95bf5f72-82f3-4e88-a576-01dae857541f\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:02:39 crc kubenswrapper[5052]: I1203 12:02:39.537791 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/empty-dir/907d9f47-ab0b-44c8-9aa6-3d135e1c312d-etc-swift\") pod \"swift-ring-rebalance-g9qd6\" (UID: \"907d9f47-ab0b-44c8-9aa6-3d135e1c312d\") " pod="openstack-kuttl-tests/swift-ring-rebalance-g9qd6" Dec 03 12:02:39 crc kubenswrapper[5052]: I1203 12:02:39.537834 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-jzbfh\" (UniqueName: \"kubernetes.io/projected/907d9f47-ab0b-44c8-9aa6-3d135e1c312d-kube-api-access-jzbfh\") pod \"swift-ring-rebalance-g9qd6\" (UID: \"907d9f47-ab0b-44c8-9aa6-3d135e1c312d\") " pod="openstack-kuttl-tests/swift-ring-rebalance-g9qd6" Dec 03 12:02:39 crc kubenswrapper[5052]: I1203 12:02:39.537695 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/907d9f47-ab0b-44c8-9aa6-3d135e1c312d-scripts\") pod \"swift-ring-rebalance-g9qd6\" (UID: \"907d9f47-ab0b-44c8-9aa6-3d135e1c312d\") " pod="openstack-kuttl-tests/swift-ring-rebalance-g9qd6" Dec 03 12:02:39 crc kubenswrapper[5052]: E1203 12:02:39.538249 5052 projected.go:288] Couldn't get configMap openstack-kuttl-tests/swift-ring-files: configmap "swift-ring-files" not found Dec 03 12:02:39 crc kubenswrapper[5052]: E1203 12:02:39.538266 5052 projected.go:194] Error preparing data for projected volume etc-swift for pod openstack-kuttl-tests/swift-storage-0: configmap "swift-ring-files" not found Dec 03 12:02:39 crc kubenswrapper[5052]: E1203 12:02:39.538299 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/95bf5f72-82f3-4e88-a576-01dae857541f-etc-swift podName:95bf5f72-82f3-4e88-a576-01dae857541f nodeName:}" failed. No retries permitted until 2025-12-03 12:02:40.538287715 +0000 UTC m=+2551.126529479 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "etc-swift" (UniqueName: "kubernetes.io/projected/95bf5f72-82f3-4e88-a576-01dae857541f-etc-swift") pod "swift-storage-0" (UID: "95bf5f72-82f3-4e88-a576-01dae857541f") : configmap "swift-ring-files" not found Dec 03 12:02:39 crc kubenswrapper[5052]: I1203 12:02:39.538662 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"etc-swift\" (UniqueName: \"kubernetes.io/empty-dir/907d9f47-ab0b-44c8-9aa6-3d135e1c312d-etc-swift\") pod \"swift-ring-rebalance-g9qd6\" (UID: \"907d9f47-ab0b-44c8-9aa6-3d135e1c312d\") " pod="openstack-kuttl-tests/swift-ring-rebalance-g9qd6" Dec 03 12:02:39 crc kubenswrapper[5052]: I1203 12:02:39.539041 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ring-data-devices\" (UniqueName: \"kubernetes.io/configmap/907d9f47-ab0b-44c8-9aa6-3d135e1c312d-ring-data-devices\") pod \"swift-ring-rebalance-g9qd6\" (UID: \"907d9f47-ab0b-44c8-9aa6-3d135e1c312d\") " pod="openstack-kuttl-tests/swift-ring-rebalance-g9qd6" Dec 03 12:02:39 crc kubenswrapper[5052]: I1203 12:02:39.540788 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"swiftconf\" (UniqueName: \"kubernetes.io/secret/907d9f47-ab0b-44c8-9aa6-3d135e1c312d-swiftconf\") pod \"swift-ring-rebalance-g9qd6\" (UID: \"907d9f47-ab0b-44c8-9aa6-3d135e1c312d\") " pod="openstack-kuttl-tests/swift-ring-rebalance-g9qd6" Dec 03 12:02:39 crc kubenswrapper[5052]: I1203 12:02:39.541016 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/907d9f47-ab0b-44c8-9aa6-3d135e1c312d-combined-ca-bundle\") pod \"swift-ring-rebalance-g9qd6\" (UID: \"907d9f47-ab0b-44c8-9aa6-3d135e1c312d\") " pod="openstack-kuttl-tests/swift-ring-rebalance-g9qd6" Dec 03 12:02:39 crc kubenswrapper[5052]: I1203 12:02:39.541076 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"dispersionconf\" (UniqueName: \"kubernetes.io/secret/907d9f47-ab0b-44c8-9aa6-3d135e1c312d-dispersionconf\") pod \"swift-ring-rebalance-g9qd6\" (UID: \"907d9f47-ab0b-44c8-9aa6-3d135e1c312d\") " pod="openstack-kuttl-tests/swift-ring-rebalance-g9qd6" Dec 03 12:02:39 crc kubenswrapper[5052]: I1203 12:02:39.558386 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-jzbfh\" (UniqueName: \"kubernetes.io/projected/907d9f47-ab0b-44c8-9aa6-3d135e1c312d-kube-api-access-jzbfh\") pod \"swift-ring-rebalance-g9qd6\" (UID: \"907d9f47-ab0b-44c8-9aa6-3d135e1c312d\") " pod="openstack-kuttl-tests/swift-ring-rebalance-g9qd6" Dec 03 12:02:39 crc kubenswrapper[5052]: I1203 12:02:39.601143 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-ring-rebalance-g9qd6" Dec 03 12:02:40 crc kubenswrapper[5052]: I1203 12:02:40.063152 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:02:40 crc kubenswrapper[5052]: I1203 12:02:40.072357 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-ring-rebalance-z4x9q" Dec 03 12:02:40 crc kubenswrapper[5052]: I1203 12:02:40.075130 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/swift-ring-rebalance-g9qd6"] Dec 03 12:02:40 crc kubenswrapper[5052]: W1203 12:02:40.089430 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod907d9f47_ab0b_44c8_9aa6_3d135e1c312d.slice/crio-fa317abab0006965a51cf1e8e792f7cae1e6364335412d963cd5261a3acf7124 WatchSource:0}: Error finding container fa317abab0006965a51cf1e8e792f7cae1e6364335412d963cd5261a3acf7124: Status 404 returned error can't find the container with id fa317abab0006965a51cf1e8e792f7cae1e6364335412d963cd5261a3acf7124 Dec 03 12:02:40 crc kubenswrapper[5052]: I1203 12:02:40.123174 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:02:40 crc kubenswrapper[5052]: I1203 12:02:40.299463 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ovn-northd-0"] Dec 03 12:02:40 crc kubenswrapper[5052]: I1203 12:02:40.301301 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:02:40 crc kubenswrapper[5052]: I1203 12:02:40.304001 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ovnnorthd-ovnnorthd-dockercfg-bztw6" Dec 03 12:02:40 crc kubenswrapper[5052]: I1203 12:02:40.304135 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"ovnnorthd-config" Dec 03 12:02:40 crc kubenswrapper[5052]: I1203 12:02:40.304266 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-ovnnorthd-ovndbs" Dec 03 12:02:40 crc kubenswrapper[5052]: I1203 12:02:40.305207 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"ovnnorthd-scripts" Dec 03 12:02:40 crc kubenswrapper[5052]: I1203 12:02:40.314691 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-ring-rebalance-z4x9q" Dec 03 12:02:40 crc kubenswrapper[5052]: I1203 12:02:40.323568 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ovn-northd-0"] Dec 03 12:02:40 crc kubenswrapper[5052]: I1203 12:02:40.456575 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"swiftconf\" (UniqueName: \"kubernetes.io/secret/9ba25edf-3f86-4a57-9f39-e217216916d3-swiftconf\") pod \"9ba25edf-3f86-4a57-9f39-e217216916d3\" (UID: \"9ba25edf-3f86-4a57-9f39-e217216916d3\") " Dec 03 12:02:40 crc kubenswrapper[5052]: I1203 12:02:40.456703 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rclt2\" (UniqueName: \"kubernetes.io/projected/9ba25edf-3f86-4a57-9f39-e217216916d3-kube-api-access-rclt2\") pod \"9ba25edf-3f86-4a57-9f39-e217216916d3\" (UID: \"9ba25edf-3f86-4a57-9f39-e217216916d3\") " Dec 03 12:02:40 crc kubenswrapper[5052]: I1203 12:02:40.456735 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ring-data-devices\" (UniqueName: \"kubernetes.io/configmap/9ba25edf-3f86-4a57-9f39-e217216916d3-ring-data-devices\") pod \"9ba25edf-3f86-4a57-9f39-e217216916d3\" (UID: \"9ba25edf-3f86-4a57-9f39-e217216916d3\") " Dec 03 12:02:40 crc kubenswrapper[5052]: I1203 12:02:40.456772 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"dispersionconf\" (UniqueName: \"kubernetes.io/secret/9ba25edf-3f86-4a57-9f39-e217216916d3-dispersionconf\") pod \"9ba25edf-3f86-4a57-9f39-e217216916d3\" (UID: \"9ba25edf-3f86-4a57-9f39-e217216916d3\") " Dec 03 12:02:40 crc kubenswrapper[5052]: I1203 12:02:40.456827 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9ba25edf-3f86-4a57-9f39-e217216916d3-combined-ca-bundle\") pod \"9ba25edf-3f86-4a57-9f39-e217216916d3\" (UID: \"9ba25edf-3f86-4a57-9f39-e217216916d3\") " Dec 03 12:02:40 crc kubenswrapper[5052]: I1203 12:02:40.456915 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/9ba25edf-3f86-4a57-9f39-e217216916d3-scripts\") pod \"9ba25edf-3f86-4a57-9f39-e217216916d3\" (UID: \"9ba25edf-3f86-4a57-9f39-e217216916d3\") " Dec 03 12:02:40 crc kubenswrapper[5052]: I1203 12:02:40.456964 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/empty-dir/9ba25edf-3f86-4a57-9f39-e217216916d3-etc-swift\") pod \"9ba25edf-3f86-4a57-9f39-e217216916d3\" (UID: \"9ba25edf-3f86-4a57-9f39-e217216916d3\") " Dec 03 12:02:40 crc kubenswrapper[5052]: I1203 12:02:40.457252 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-northd-tls-certs\" (UniqueName: \"kubernetes.io/secret/fad6f453-acf5-4f13-94d6-7d46e300f28c-ovn-northd-tls-certs\") pod \"ovn-northd-0\" (UID: \"fad6f453-acf5-4f13-94d6-7d46e300f28c\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:02:40 crc kubenswrapper[5052]: I1203 12:02:40.457291 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/fad6f453-acf5-4f13-94d6-7d46e300f28c-scripts\") pod \"ovn-northd-0\" (UID: \"fad6f453-acf5-4f13-94d6-7d46e300f28c\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:02:40 crc kubenswrapper[5052]: I1203 12:02:40.457327 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tmd66\" (UniqueName: \"kubernetes.io/projected/fad6f453-acf5-4f13-94d6-7d46e300f28c-kube-api-access-tmd66\") pod \"ovn-northd-0\" (UID: \"fad6f453-acf5-4f13-94d6-7d46e300f28c\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:02:40 crc kubenswrapper[5052]: I1203 12:02:40.457388 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/fad6f453-acf5-4f13-94d6-7d46e300f28c-metrics-certs-tls-certs\") pod \"ovn-northd-0\" (UID: \"fad6f453-acf5-4f13-94d6-7d46e300f28c\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:02:40 crc kubenswrapper[5052]: I1203 12:02:40.457411 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-rundir\" (UniqueName: \"kubernetes.io/empty-dir/fad6f453-acf5-4f13-94d6-7d46e300f28c-ovn-rundir\") pod \"ovn-northd-0\" (UID: \"fad6f453-acf5-4f13-94d6-7d46e300f28c\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:02:40 crc kubenswrapper[5052]: I1203 12:02:40.457427 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/fad6f453-acf5-4f13-94d6-7d46e300f28c-config\") pod \"ovn-northd-0\" (UID: \"fad6f453-acf5-4f13-94d6-7d46e300f28c\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:02:40 crc kubenswrapper[5052]: I1203 12:02:40.457456 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/fad6f453-acf5-4f13-94d6-7d46e300f28c-combined-ca-bundle\") pod \"ovn-northd-0\" (UID: \"fad6f453-acf5-4f13-94d6-7d46e300f28c\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:02:40 crc kubenswrapper[5052]: I1203 12:02:40.457494 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9ba25edf-3f86-4a57-9f39-e217216916d3-ring-data-devices" (OuterVolumeSpecName: "ring-data-devices") pod "9ba25edf-3f86-4a57-9f39-e217216916d3" (UID: "9ba25edf-3f86-4a57-9f39-e217216916d3"). InnerVolumeSpecName "ring-data-devices". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:02:40 crc kubenswrapper[5052]: I1203 12:02:40.457545 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9ba25edf-3f86-4a57-9f39-e217216916d3-scripts" (OuterVolumeSpecName: "scripts") pod "9ba25edf-3f86-4a57-9f39-e217216916d3" (UID: "9ba25edf-3f86-4a57-9f39-e217216916d3"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:02:40 crc kubenswrapper[5052]: I1203 12:02:40.457772 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ba25edf-3f86-4a57-9f39-e217216916d3-etc-swift" (OuterVolumeSpecName: "etc-swift") pod "9ba25edf-3f86-4a57-9f39-e217216916d3" (UID: "9ba25edf-3f86-4a57-9f39-e217216916d3"). InnerVolumeSpecName "etc-swift". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:02:40 crc kubenswrapper[5052]: I1203 12:02:40.462157 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9ba25edf-3f86-4a57-9f39-e217216916d3-kube-api-access-rclt2" (OuterVolumeSpecName: "kube-api-access-rclt2") pod "9ba25edf-3f86-4a57-9f39-e217216916d3" (UID: "9ba25edf-3f86-4a57-9f39-e217216916d3"). InnerVolumeSpecName "kube-api-access-rclt2". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:02:40 crc kubenswrapper[5052]: I1203 12:02:40.463143 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9ba25edf-3f86-4a57-9f39-e217216916d3-swiftconf" (OuterVolumeSpecName: "swiftconf") pod "9ba25edf-3f86-4a57-9f39-e217216916d3" (UID: "9ba25edf-3f86-4a57-9f39-e217216916d3"). InnerVolumeSpecName "swiftconf". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:02:40 crc kubenswrapper[5052]: I1203 12:02:40.463280 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9ba25edf-3f86-4a57-9f39-e217216916d3-dispersionconf" (OuterVolumeSpecName: "dispersionconf") pod "9ba25edf-3f86-4a57-9f39-e217216916d3" (UID: "9ba25edf-3f86-4a57-9f39-e217216916d3"). InnerVolumeSpecName "dispersionconf". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:02:40 crc kubenswrapper[5052]: I1203 12:02:40.465758 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9ba25edf-3f86-4a57-9f39-e217216916d3-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "9ba25edf-3f86-4a57-9f39-e217216916d3" (UID: "9ba25edf-3f86-4a57-9f39-e217216916d3"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:02:40 crc kubenswrapper[5052]: I1203 12:02:40.560038 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/fad6f453-acf5-4f13-94d6-7d46e300f28c-metrics-certs-tls-certs\") pod \"ovn-northd-0\" (UID: \"fad6f453-acf5-4f13-94d6-7d46e300f28c\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:02:40 crc kubenswrapper[5052]: I1203 12:02:40.560367 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovn-rundir\" (UniqueName: \"kubernetes.io/empty-dir/fad6f453-acf5-4f13-94d6-7d46e300f28c-ovn-rundir\") pod \"ovn-northd-0\" (UID: \"fad6f453-acf5-4f13-94d6-7d46e300f28c\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:02:40 crc kubenswrapper[5052]: I1203 12:02:40.560528 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/fad6f453-acf5-4f13-94d6-7d46e300f28c-config\") pod \"ovn-northd-0\" (UID: \"fad6f453-acf5-4f13-94d6-7d46e300f28c\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:02:40 crc kubenswrapper[5052]: I1203 12:02:40.560689 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/fad6f453-acf5-4f13-94d6-7d46e300f28c-combined-ca-bundle\") pod \"ovn-northd-0\" (UID: \"fad6f453-acf5-4f13-94d6-7d46e300f28c\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:02:40 crc kubenswrapper[5052]: I1203 12:02:40.560872 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovn-northd-tls-certs\" (UniqueName: \"kubernetes.io/secret/fad6f453-acf5-4f13-94d6-7d46e300f28c-ovn-northd-tls-certs\") pod \"ovn-northd-0\" (UID: \"fad6f453-acf5-4f13-94d6-7d46e300f28c\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:02:40 crc kubenswrapper[5052]: I1203 12:02:40.560960 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/95bf5f72-82f3-4e88-a576-01dae857541f-etc-swift\") pod \"swift-storage-0\" (UID: \"95bf5f72-82f3-4e88-a576-01dae857541f\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:02:40 crc kubenswrapper[5052]: I1203 12:02:40.561037 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/fad6f453-acf5-4f13-94d6-7d46e300f28c-scripts\") pod \"ovn-northd-0\" (UID: \"fad6f453-acf5-4f13-94d6-7d46e300f28c\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:02:40 crc kubenswrapper[5052]: I1203 12:02:40.561149 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-tmd66\" (UniqueName: \"kubernetes.io/projected/fad6f453-acf5-4f13-94d6-7d46e300f28c-kube-api-access-tmd66\") pod \"ovn-northd-0\" (UID: \"fad6f453-acf5-4f13-94d6-7d46e300f28c\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:02:40 crc kubenswrapper[5052]: I1203 12:02:40.561285 5052 reconciler_common.go:293] "Volume detached for volume \"ring-data-devices\" (UniqueName: \"kubernetes.io/configmap/9ba25edf-3f86-4a57-9f39-e217216916d3-ring-data-devices\") on node \"crc\" DevicePath \"\"" Dec 03 12:02:40 crc kubenswrapper[5052]: I1203 12:02:40.561361 5052 reconciler_common.go:293] "Volume detached for volume \"dispersionconf\" (UniqueName: \"kubernetes.io/secret/9ba25edf-3f86-4a57-9f39-e217216916d3-dispersionconf\") on node \"crc\" DevicePath \"\"" Dec 03 12:02:40 crc kubenswrapper[5052]: I1203 12:02:40.561435 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9ba25edf-3f86-4a57-9f39-e217216916d3-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:02:40 crc kubenswrapper[5052]: I1203 12:02:40.561552 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/9ba25edf-3f86-4a57-9f39-e217216916d3-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:02:40 crc kubenswrapper[5052]: I1203 12:02:40.561655 5052 reconciler_common.go:293] "Volume detached for volume \"etc-swift\" (UniqueName: \"kubernetes.io/empty-dir/9ba25edf-3f86-4a57-9f39-e217216916d3-etc-swift\") on node \"crc\" DevicePath \"\"" Dec 03 12:02:40 crc kubenswrapper[5052]: I1203 12:02:40.561738 5052 reconciler_common.go:293] "Volume detached for volume \"swiftconf\" (UniqueName: \"kubernetes.io/secret/9ba25edf-3f86-4a57-9f39-e217216916d3-swiftconf\") on node \"crc\" DevicePath \"\"" Dec 03 12:02:40 crc kubenswrapper[5052]: E1203 12:02:40.561164 5052 projected.go:288] Couldn't get configMap openstack-kuttl-tests/swift-ring-files: configmap "swift-ring-files" not found Dec 03 12:02:40 crc kubenswrapper[5052]: E1203 12:02:40.561842 5052 projected.go:194] Error preparing data for projected volume etc-swift for pod openstack-kuttl-tests/swift-storage-0: configmap "swift-ring-files" not found Dec 03 12:02:40 crc kubenswrapper[5052]: I1203 12:02:40.561654 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/fad6f453-acf5-4f13-94d6-7d46e300f28c-scripts\") pod \"ovn-northd-0\" (UID: \"fad6f453-acf5-4f13-94d6-7d46e300f28c\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:02:40 crc kubenswrapper[5052]: E1203 12:02:40.561900 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/95bf5f72-82f3-4e88-a576-01dae857541f-etc-swift podName:95bf5f72-82f3-4e88-a576-01dae857541f nodeName:}" failed. No retries permitted until 2025-12-03 12:02:42.561880379 +0000 UTC m=+2553.150122213 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "etc-swift" (UniqueName: "kubernetes.io/projected/95bf5f72-82f3-4e88-a576-01dae857541f-etc-swift") pod "swift-storage-0" (UID: "95bf5f72-82f3-4e88-a576-01dae857541f") : configmap "swift-ring-files" not found Dec 03 12:02:40 crc kubenswrapper[5052]: I1203 12:02:40.561316 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/fad6f453-acf5-4f13-94d6-7d46e300f28c-config\") pod \"ovn-northd-0\" (UID: \"fad6f453-acf5-4f13-94d6-7d46e300f28c\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:02:40 crc kubenswrapper[5052]: I1203 12:02:40.561960 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovn-rundir\" (UniqueName: \"kubernetes.io/empty-dir/fad6f453-acf5-4f13-94d6-7d46e300f28c-ovn-rundir\") pod \"ovn-northd-0\" (UID: \"fad6f453-acf5-4f13-94d6-7d46e300f28c\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:02:40 crc kubenswrapper[5052]: I1203 12:02:40.561813 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-rclt2\" (UniqueName: \"kubernetes.io/projected/9ba25edf-3f86-4a57-9f39-e217216916d3-kube-api-access-rclt2\") on node \"crc\" DevicePath \"\"" Dec 03 12:02:40 crc kubenswrapper[5052]: I1203 12:02:40.565352 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovn-northd-tls-certs\" (UniqueName: \"kubernetes.io/secret/fad6f453-acf5-4f13-94d6-7d46e300f28c-ovn-northd-tls-certs\") pod \"ovn-northd-0\" (UID: \"fad6f453-acf5-4f13-94d6-7d46e300f28c\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:02:40 crc kubenswrapper[5052]: I1203 12:02:40.565379 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/fad6f453-acf5-4f13-94d6-7d46e300f28c-combined-ca-bundle\") pod \"ovn-northd-0\" (UID: \"fad6f453-acf5-4f13-94d6-7d46e300f28c\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:02:40 crc kubenswrapper[5052]: I1203 12:02:40.566317 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/fad6f453-acf5-4f13-94d6-7d46e300f28c-metrics-certs-tls-certs\") pod \"ovn-northd-0\" (UID: \"fad6f453-acf5-4f13-94d6-7d46e300f28c\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:02:40 crc kubenswrapper[5052]: I1203 12:02:40.577294 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-tmd66\" (UniqueName: \"kubernetes.io/projected/fad6f453-acf5-4f13-94d6-7d46e300f28c-kube-api-access-tmd66\") pod \"ovn-northd-0\" (UID: \"fad6f453-acf5-4f13-94d6-7d46e300f28c\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:02:40 crc kubenswrapper[5052]: I1203 12:02:40.650115 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:02:41 crc kubenswrapper[5052]: I1203 12:02:41.097906 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-ring-rebalance-g9qd6" event={"ID":"907d9f47-ab0b-44c8-9aa6-3d135e1c312d","Type":"ContainerStarted","Data":"feac02bd195cd4010d3557ca2d9bccbb1a2e3e5d4cdedd3139212667bfa80b4b"} Dec 03 12:02:41 crc kubenswrapper[5052]: I1203 12:02:41.098408 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-ring-rebalance-g9qd6" event={"ID":"907d9f47-ab0b-44c8-9aa6-3d135e1c312d","Type":"ContainerStarted","Data":"fa317abab0006965a51cf1e8e792f7cae1e6364335412d963cd5261a3acf7124"} Dec 03 12:02:41 crc kubenswrapper[5052]: I1203 12:02:41.097930 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-ring-rebalance-z4x9q" Dec 03 12:02:41 crc kubenswrapper[5052]: I1203 12:02:41.123693 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ovn-northd-0"] Dec 03 12:02:41 crc kubenswrapper[5052]: I1203 12:02:41.130655 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/swift-ring-rebalance-g9qd6" podStartSLOduration=2.13063434 podStartE2EDuration="2.13063434s" podCreationTimestamp="2025-12-03 12:02:39 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:02:41.116255299 +0000 UTC m=+2551.704497083" watchObservedRunningTime="2025-12-03 12:02:41.13063434 +0000 UTC m=+2551.718876114" Dec 03 12:02:41 crc kubenswrapper[5052]: I1203 12:02:41.168216 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/swift-ring-rebalance-z4x9q"] Dec 03 12:02:41 crc kubenswrapper[5052]: I1203 12:02:41.177973 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/swift-ring-rebalance-z4x9q"] Dec 03 12:02:41 crc kubenswrapper[5052]: I1203 12:02:41.429047 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/openstack-galera-2" Dec 03 12:02:41 crc kubenswrapper[5052]: I1203 12:02:41.551249 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/openstack-galera-2" Dec 03 12:02:41 crc kubenswrapper[5052]: I1203 12:02:41.684877 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9ba25edf-3f86-4a57-9f39-e217216916d3" path="/var/lib/kubelet/pods/9ba25edf-3f86-4a57-9f39-e217216916d3/volumes" Dec 03 12:02:42 crc kubenswrapper[5052]: I1203 12:02:42.108347 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovn-northd-0" event={"ID":"fad6f453-acf5-4f13-94d6-7d46e300f28c","Type":"ContainerStarted","Data":"c15f114c02ee16b234e664feb08dc9f03835f614029770d390cfbeb8fa520a9a"} Dec 03 12:02:42 crc kubenswrapper[5052]: I1203 12:02:42.108400 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovn-northd-0" event={"ID":"fad6f453-acf5-4f13-94d6-7d46e300f28c","Type":"ContainerStarted","Data":"f82e335d0814000e829c2020ade84de1617dea07a919392e9736c05912d0b4bf"} Dec 03 12:02:42 crc kubenswrapper[5052]: I1203 12:02:42.108415 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovn-northd-0" event={"ID":"fad6f453-acf5-4f13-94d6-7d46e300f28c","Type":"ContainerStarted","Data":"1ebd09bbdb8baf71a090c3b4929e61b584b38934f5432877c609354b4da540cb"} Dec 03 12:02:42 crc kubenswrapper[5052]: I1203 12:02:42.138277 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/ovn-northd-0" podStartSLOduration=2.138255706 podStartE2EDuration="2.138255706s" podCreationTimestamp="2025-12-03 12:02:40 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:02:42.135130197 +0000 UTC m=+2552.723371961" watchObservedRunningTime="2025-12-03 12:02:42.138255706 +0000 UTC m=+2552.726497470" Dec 03 12:02:42 crc kubenswrapper[5052]: I1203 12:02:42.602563 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/95bf5f72-82f3-4e88-a576-01dae857541f-etc-swift\") pod \"swift-storage-0\" (UID: \"95bf5f72-82f3-4e88-a576-01dae857541f\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:02:42 crc kubenswrapper[5052]: E1203 12:02:42.602771 5052 projected.go:288] Couldn't get configMap openstack-kuttl-tests/swift-ring-files: configmap "swift-ring-files" not found Dec 03 12:02:42 crc kubenswrapper[5052]: E1203 12:02:42.602818 5052 projected.go:194] Error preparing data for projected volume etc-swift for pod openstack-kuttl-tests/swift-storage-0: configmap "swift-ring-files" not found Dec 03 12:02:42 crc kubenswrapper[5052]: E1203 12:02:42.602864 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/95bf5f72-82f3-4e88-a576-01dae857541f-etc-swift podName:95bf5f72-82f3-4e88-a576-01dae857541f nodeName:}" failed. No retries permitted until 2025-12-03 12:02:46.602850129 +0000 UTC m=+2557.191091893 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "etc-swift" (UniqueName: "kubernetes.io/projected/95bf5f72-82f3-4e88-a576-01dae857541f-etc-swift") pod "swift-storage-0" (UID: "95bf5f72-82f3-4e88-a576-01dae857541f") : configmap "swift-ring-files" not found Dec 03 12:02:43 crc kubenswrapper[5052]: I1203 12:02:43.116691 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:02:44 crc kubenswrapper[5052]: E1203 12:02:44.396617 5052 upgradeaware.go:427] Error proxying data from client to backend: readfrom tcp 38.129.56.252:36990->38.129.56.252:34849: write tcp 38.129.56.252:36990->38.129.56.252:34849: write: broken pipe Dec 03 12:02:45 crc kubenswrapper[5052]: I1203 12:02:45.839429 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/keystone-db-create-zgx94"] Dec 03 12:02:45 crc kubenswrapper[5052]: I1203 12:02:45.841135 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-db-create-zgx94" Dec 03 12:02:45 crc kubenswrapper[5052]: I1203 12:02:45.868626 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-db-create-zgx94"] Dec 03 12:02:45 crc kubenswrapper[5052]: I1203 12:02:45.881609 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:02:45 crc kubenswrapper[5052]: I1203 12:02:45.883803 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:02:45 crc kubenswrapper[5052]: I1203 12:02:45.900583 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/keystone-25ff-account-create-update-hcnv4"] Dec 03 12:02:45 crc kubenswrapper[5052]: I1203 12:02:45.902042 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-25ff-account-create-update-hcnv4" Dec 03 12:02:45 crc kubenswrapper[5052]: I1203 12:02:45.905250 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone-db-secret" Dec 03 12:02:45 crc kubenswrapper[5052]: I1203 12:02:45.910629 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/openstack-cell1-galera-1" Dec 03 12:02:45 crc kubenswrapper[5052]: I1203 12:02:45.911132 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/openstack-cell1-galera-1" Dec 03 12:02:45 crc kubenswrapper[5052]: I1203 12:02:45.950054 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-25ff-account-create-update-hcnv4"] Dec 03 12:02:45 crc kubenswrapper[5052]: I1203 12:02:45.962931 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qcfnk\" (UniqueName: \"kubernetes.io/projected/1e7cf8cd-09e8-4bfe-b8d3-9e509d4eae43-kube-api-access-qcfnk\") pod \"keystone-db-create-zgx94\" (UID: \"1e7cf8cd-09e8-4bfe-b8d3-9e509d4eae43\") " pod="openstack-kuttl-tests/keystone-db-create-zgx94" Dec 03 12:02:45 crc kubenswrapper[5052]: I1203 12:02:45.963024 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/1e7cf8cd-09e8-4bfe-b8d3-9e509d4eae43-operator-scripts\") pod \"keystone-db-create-zgx94\" (UID: \"1e7cf8cd-09e8-4bfe-b8d3-9e509d4eae43\") " pod="openstack-kuttl-tests/keystone-db-create-zgx94" Dec 03 12:02:45 crc kubenswrapper[5052]: I1203 12:02:45.963076 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/66b581e0-3a48-46c9-b1a7-384b87d2b7f4-operator-scripts\") pod \"keystone-25ff-account-create-update-hcnv4\" (UID: \"66b581e0-3a48-46c9-b1a7-384b87d2b7f4\") " pod="openstack-kuttl-tests/keystone-25ff-account-create-update-hcnv4" Dec 03 12:02:45 crc kubenswrapper[5052]: I1203 12:02:45.963100 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4tshg\" (UniqueName: \"kubernetes.io/projected/66b581e0-3a48-46c9-b1a7-384b87d2b7f4-kube-api-access-4tshg\") pod \"keystone-25ff-account-create-update-hcnv4\" (UID: \"66b581e0-3a48-46c9-b1a7-384b87d2b7f4\") " pod="openstack-kuttl-tests/keystone-25ff-account-create-update-hcnv4" Dec 03 12:02:46 crc kubenswrapper[5052]: I1203 12:02:46.067642 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-qcfnk\" (UniqueName: \"kubernetes.io/projected/1e7cf8cd-09e8-4bfe-b8d3-9e509d4eae43-kube-api-access-qcfnk\") pod \"keystone-db-create-zgx94\" (UID: \"1e7cf8cd-09e8-4bfe-b8d3-9e509d4eae43\") " pod="openstack-kuttl-tests/keystone-db-create-zgx94" Dec 03 12:02:46 crc kubenswrapper[5052]: I1203 12:02:46.067907 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/1e7cf8cd-09e8-4bfe-b8d3-9e509d4eae43-operator-scripts\") pod \"keystone-db-create-zgx94\" (UID: \"1e7cf8cd-09e8-4bfe-b8d3-9e509d4eae43\") " pod="openstack-kuttl-tests/keystone-db-create-zgx94" Dec 03 12:02:46 crc kubenswrapper[5052]: I1203 12:02:46.068057 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/66b581e0-3a48-46c9-b1a7-384b87d2b7f4-operator-scripts\") pod \"keystone-25ff-account-create-update-hcnv4\" (UID: \"66b581e0-3a48-46c9-b1a7-384b87d2b7f4\") " pod="openstack-kuttl-tests/keystone-25ff-account-create-update-hcnv4" Dec 03 12:02:46 crc kubenswrapper[5052]: I1203 12:02:46.068163 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-4tshg\" (UniqueName: \"kubernetes.io/projected/66b581e0-3a48-46c9-b1a7-384b87d2b7f4-kube-api-access-4tshg\") pod \"keystone-25ff-account-create-update-hcnv4\" (UID: \"66b581e0-3a48-46c9-b1a7-384b87d2b7f4\") " pod="openstack-kuttl-tests/keystone-25ff-account-create-update-hcnv4" Dec 03 12:02:46 crc kubenswrapper[5052]: I1203 12:02:46.069260 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/1e7cf8cd-09e8-4bfe-b8d3-9e509d4eae43-operator-scripts\") pod \"keystone-db-create-zgx94\" (UID: \"1e7cf8cd-09e8-4bfe-b8d3-9e509d4eae43\") " pod="openstack-kuttl-tests/keystone-db-create-zgx94" Dec 03 12:02:46 crc kubenswrapper[5052]: I1203 12:02:46.070137 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/66b581e0-3a48-46c9-b1a7-384b87d2b7f4-operator-scripts\") pod \"keystone-25ff-account-create-update-hcnv4\" (UID: \"66b581e0-3a48-46c9-b1a7-384b87d2b7f4\") " pod="openstack-kuttl-tests/keystone-25ff-account-create-update-hcnv4" Dec 03 12:02:46 crc kubenswrapper[5052]: I1203 12:02:46.107813 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-4tshg\" (UniqueName: \"kubernetes.io/projected/66b581e0-3a48-46c9-b1a7-384b87d2b7f4-kube-api-access-4tshg\") pod \"keystone-25ff-account-create-update-hcnv4\" (UID: \"66b581e0-3a48-46c9-b1a7-384b87d2b7f4\") " pod="openstack-kuttl-tests/keystone-25ff-account-create-update-hcnv4" Dec 03 12:02:46 crc kubenswrapper[5052]: I1203 12:02:46.114152 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-qcfnk\" (UniqueName: \"kubernetes.io/projected/1e7cf8cd-09e8-4bfe-b8d3-9e509d4eae43-kube-api-access-qcfnk\") pod \"keystone-db-create-zgx94\" (UID: \"1e7cf8cd-09e8-4bfe-b8d3-9e509d4eae43\") " pod="openstack-kuttl-tests/keystone-db-create-zgx94" Dec 03 12:02:46 crc kubenswrapper[5052]: I1203 12:02:46.184693 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-db-create-zgx94" Dec 03 12:02:46 crc kubenswrapper[5052]: I1203 12:02:46.283230 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-25ff-account-create-update-hcnv4" Dec 03 12:02:46 crc kubenswrapper[5052]: I1203 12:02:46.297228 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/placement-db-create-cmps4"] Dec 03 12:02:46 crc kubenswrapper[5052]: I1203 12:02:46.308295 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-db-create-cmps4" Dec 03 12:02:46 crc kubenswrapper[5052]: I1203 12:02:46.352744 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/placement-db-create-cmps4"] Dec 03 12:02:46 crc kubenswrapper[5052]: I1203 12:02:46.388918 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/79e77d8b-2481-4cbf-875f-cb626cf89463-operator-scripts\") pod \"placement-db-create-cmps4\" (UID: \"79e77d8b-2481-4cbf-875f-cb626cf89463\") " pod="openstack-kuttl-tests/placement-db-create-cmps4" Dec 03 12:02:46 crc kubenswrapper[5052]: I1203 12:02:46.389046 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zz7r7\" (UniqueName: \"kubernetes.io/projected/79e77d8b-2481-4cbf-875f-cb626cf89463-kube-api-access-zz7r7\") pod \"placement-db-create-cmps4\" (UID: \"79e77d8b-2481-4cbf-875f-cb626cf89463\") " pod="openstack-kuttl-tests/placement-db-create-cmps4" Dec 03 12:02:46 crc kubenswrapper[5052]: I1203 12:02:46.435438 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/placement-3c40-account-create-update-dtmv4"] Dec 03 12:02:46 crc kubenswrapper[5052]: I1203 12:02:46.439169 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-3c40-account-create-update-dtmv4" Dec 03 12:02:46 crc kubenswrapper[5052]: I1203 12:02:46.443622 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"placement-db-secret" Dec 03 12:02:46 crc kubenswrapper[5052]: I1203 12:02:46.449994 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/placement-3c40-account-create-update-dtmv4"] Dec 03 12:02:46 crc kubenswrapper[5052]: I1203 12:02:46.496179 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/79e77d8b-2481-4cbf-875f-cb626cf89463-operator-scripts\") pod \"placement-db-create-cmps4\" (UID: \"79e77d8b-2481-4cbf-875f-cb626cf89463\") " pod="openstack-kuttl-tests/placement-db-create-cmps4" Dec 03 12:02:46 crc kubenswrapper[5052]: I1203 12:02:46.496649 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-zz7r7\" (UniqueName: \"kubernetes.io/projected/79e77d8b-2481-4cbf-875f-cb626cf89463-kube-api-access-zz7r7\") pod \"placement-db-create-cmps4\" (UID: \"79e77d8b-2481-4cbf-875f-cb626cf89463\") " pod="openstack-kuttl-tests/placement-db-create-cmps4" Dec 03 12:02:46 crc kubenswrapper[5052]: I1203 12:02:46.496683 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hs9lw\" (UniqueName: \"kubernetes.io/projected/2a25df1a-4f92-4ed6-9c34-2d6b0212593b-kube-api-access-hs9lw\") pod \"placement-3c40-account-create-update-dtmv4\" (UID: \"2a25df1a-4f92-4ed6-9c34-2d6b0212593b\") " pod="openstack-kuttl-tests/placement-3c40-account-create-update-dtmv4" Dec 03 12:02:46 crc kubenswrapper[5052]: I1203 12:02:46.496843 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/2a25df1a-4f92-4ed6-9c34-2d6b0212593b-operator-scripts\") pod \"placement-3c40-account-create-update-dtmv4\" (UID: \"2a25df1a-4f92-4ed6-9c34-2d6b0212593b\") " pod="openstack-kuttl-tests/placement-3c40-account-create-update-dtmv4" Dec 03 12:02:46 crc kubenswrapper[5052]: I1203 12:02:46.498023 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/79e77d8b-2481-4cbf-875f-cb626cf89463-operator-scripts\") pod \"placement-db-create-cmps4\" (UID: \"79e77d8b-2481-4cbf-875f-cb626cf89463\") " pod="openstack-kuttl-tests/placement-db-create-cmps4" Dec 03 12:02:46 crc kubenswrapper[5052]: I1203 12:02:46.526529 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-zz7r7\" (UniqueName: \"kubernetes.io/projected/79e77d8b-2481-4cbf-875f-cb626cf89463-kube-api-access-zz7r7\") pod \"placement-db-create-cmps4\" (UID: \"79e77d8b-2481-4cbf-875f-cb626cf89463\") " pod="openstack-kuttl-tests/placement-db-create-cmps4" Dec 03 12:02:46 crc kubenswrapper[5052]: I1203 12:02:46.599106 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/2a25df1a-4f92-4ed6-9c34-2d6b0212593b-operator-scripts\") pod \"placement-3c40-account-create-update-dtmv4\" (UID: \"2a25df1a-4f92-4ed6-9c34-2d6b0212593b\") " pod="openstack-kuttl-tests/placement-3c40-account-create-update-dtmv4" Dec 03 12:02:46 crc kubenswrapper[5052]: I1203 12:02:46.599300 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-hs9lw\" (UniqueName: \"kubernetes.io/projected/2a25df1a-4f92-4ed6-9c34-2d6b0212593b-kube-api-access-hs9lw\") pod \"placement-3c40-account-create-update-dtmv4\" (UID: \"2a25df1a-4f92-4ed6-9c34-2d6b0212593b\") " pod="openstack-kuttl-tests/placement-3c40-account-create-update-dtmv4" Dec 03 12:02:46 crc kubenswrapper[5052]: I1203 12:02:46.599990 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/2a25df1a-4f92-4ed6-9c34-2d6b0212593b-operator-scripts\") pod \"placement-3c40-account-create-update-dtmv4\" (UID: \"2a25df1a-4f92-4ed6-9c34-2d6b0212593b\") " pod="openstack-kuttl-tests/placement-3c40-account-create-update-dtmv4" Dec 03 12:02:46 crc kubenswrapper[5052]: I1203 12:02:46.618697 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-hs9lw\" (UniqueName: \"kubernetes.io/projected/2a25df1a-4f92-4ed6-9c34-2d6b0212593b-kube-api-access-hs9lw\") pod \"placement-3c40-account-create-update-dtmv4\" (UID: \"2a25df1a-4f92-4ed6-9c34-2d6b0212593b\") " pod="openstack-kuttl-tests/placement-3c40-account-create-update-dtmv4" Dec 03 12:02:46 crc kubenswrapper[5052]: I1203 12:02:46.631853 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-db-create-cmps4" Dec 03 12:02:46 crc kubenswrapper[5052]: I1203 12:02:46.657210 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/glance-db-create-jpmhr"] Dec 03 12:02:46 crc kubenswrapper[5052]: I1203 12:02:46.658666 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-db-create-jpmhr" Dec 03 12:02:46 crc kubenswrapper[5052]: I1203 12:02:46.681418 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-db-create-jpmhr"] Dec 03 12:02:46 crc kubenswrapper[5052]: I1203 12:02:46.701753 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/95bf5f72-82f3-4e88-a576-01dae857541f-etc-swift\") pod \"swift-storage-0\" (UID: \"95bf5f72-82f3-4e88-a576-01dae857541f\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:02:46 crc kubenswrapper[5052]: I1203 12:02:46.701894 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zdzvg\" (UniqueName: \"kubernetes.io/projected/246ced2f-39a7-496b-93cd-9f8ab898e2bb-kube-api-access-zdzvg\") pod \"glance-db-create-jpmhr\" (UID: \"246ced2f-39a7-496b-93cd-9f8ab898e2bb\") " pod="openstack-kuttl-tests/glance-db-create-jpmhr" Dec 03 12:02:46 crc kubenswrapper[5052]: I1203 12:02:46.701970 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/246ced2f-39a7-496b-93cd-9f8ab898e2bb-operator-scripts\") pod \"glance-db-create-jpmhr\" (UID: \"246ced2f-39a7-496b-93cd-9f8ab898e2bb\") " pod="openstack-kuttl-tests/glance-db-create-jpmhr" Dec 03 12:02:46 crc kubenswrapper[5052]: E1203 12:02:46.702170 5052 projected.go:288] Couldn't get configMap openstack-kuttl-tests/swift-ring-files: configmap "swift-ring-files" not found Dec 03 12:02:46 crc kubenswrapper[5052]: E1203 12:02:46.702187 5052 projected.go:194] Error preparing data for projected volume etc-swift for pod openstack-kuttl-tests/swift-storage-0: configmap "swift-ring-files" not found Dec 03 12:02:46 crc kubenswrapper[5052]: E1203 12:02:46.702232 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/95bf5f72-82f3-4e88-a576-01dae857541f-etc-swift podName:95bf5f72-82f3-4e88-a576-01dae857541f nodeName:}" failed. No retries permitted until 2025-12-03 12:02:54.702214487 +0000 UTC m=+2565.290456251 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "etc-swift" (UniqueName: "kubernetes.io/projected/95bf5f72-82f3-4e88-a576-01dae857541f-etc-swift") pod "swift-storage-0" (UID: "95bf5f72-82f3-4e88-a576-01dae857541f") : configmap "swift-ring-files" not found Dec 03 12:02:46 crc kubenswrapper[5052]: I1203 12:02:46.713607 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-db-create-zgx94"] Dec 03 12:02:46 crc kubenswrapper[5052]: I1203 12:02:46.757909 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/glance-8fcf-account-create-update-r5h2k"] Dec 03 12:02:46 crc kubenswrapper[5052]: I1203 12:02:46.759154 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-8fcf-account-create-update-r5h2k" Dec 03 12:02:46 crc kubenswrapper[5052]: I1203 12:02:46.770056 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-8fcf-account-create-update-r5h2k"] Dec 03 12:02:46 crc kubenswrapper[5052]: I1203 12:02:46.770330 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-db-secret" Dec 03 12:02:46 crc kubenswrapper[5052]: I1203 12:02:46.771605 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-3c40-account-create-update-dtmv4" Dec 03 12:02:46 crc kubenswrapper[5052]: I1203 12:02:46.803194 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/246ced2f-39a7-496b-93cd-9f8ab898e2bb-operator-scripts\") pod \"glance-db-create-jpmhr\" (UID: \"246ced2f-39a7-496b-93cd-9f8ab898e2bb\") " pod="openstack-kuttl-tests/glance-db-create-jpmhr" Dec 03 12:02:46 crc kubenswrapper[5052]: I1203 12:02:46.803362 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/c665a3eb-31e2-4ac4-99c8-02c8d262d9c9-operator-scripts\") pod \"glance-8fcf-account-create-update-r5h2k\" (UID: \"c665a3eb-31e2-4ac4-99c8-02c8d262d9c9\") " pod="openstack-kuttl-tests/glance-8fcf-account-create-update-r5h2k" Dec 03 12:02:46 crc kubenswrapper[5052]: I1203 12:02:46.803487 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-zdzvg\" (UniqueName: \"kubernetes.io/projected/246ced2f-39a7-496b-93cd-9f8ab898e2bb-kube-api-access-zdzvg\") pod \"glance-db-create-jpmhr\" (UID: \"246ced2f-39a7-496b-93cd-9f8ab898e2bb\") " pod="openstack-kuttl-tests/glance-db-create-jpmhr" Dec 03 12:02:46 crc kubenswrapper[5052]: I1203 12:02:46.803571 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2nm89\" (UniqueName: \"kubernetes.io/projected/c665a3eb-31e2-4ac4-99c8-02c8d262d9c9-kube-api-access-2nm89\") pod \"glance-8fcf-account-create-update-r5h2k\" (UID: \"c665a3eb-31e2-4ac4-99c8-02c8d262d9c9\") " pod="openstack-kuttl-tests/glance-8fcf-account-create-update-r5h2k" Dec 03 12:02:46 crc kubenswrapper[5052]: I1203 12:02:46.804159 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/246ced2f-39a7-496b-93cd-9f8ab898e2bb-operator-scripts\") pod \"glance-db-create-jpmhr\" (UID: \"246ced2f-39a7-496b-93cd-9f8ab898e2bb\") " pod="openstack-kuttl-tests/glance-db-create-jpmhr" Dec 03 12:02:46 crc kubenswrapper[5052]: I1203 12:02:46.827659 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-zdzvg\" (UniqueName: \"kubernetes.io/projected/246ced2f-39a7-496b-93cd-9f8ab898e2bb-kube-api-access-zdzvg\") pod \"glance-db-create-jpmhr\" (UID: \"246ced2f-39a7-496b-93cd-9f8ab898e2bb\") " pod="openstack-kuttl-tests/glance-db-create-jpmhr" Dec 03 12:02:46 crc kubenswrapper[5052]: I1203 12:02:46.849878 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-25ff-account-create-update-hcnv4"] Dec 03 12:02:46 crc kubenswrapper[5052]: I1203 12:02:46.884936 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/openstack-cell1-galera-2" Dec 03 12:02:46 crc kubenswrapper[5052]: I1203 12:02:46.907032 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/c665a3eb-31e2-4ac4-99c8-02c8d262d9c9-operator-scripts\") pod \"glance-8fcf-account-create-update-r5h2k\" (UID: \"c665a3eb-31e2-4ac4-99c8-02c8d262d9c9\") " pod="openstack-kuttl-tests/glance-8fcf-account-create-update-r5h2k" Dec 03 12:02:46 crc kubenswrapper[5052]: I1203 12:02:46.907130 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-2nm89\" (UniqueName: \"kubernetes.io/projected/c665a3eb-31e2-4ac4-99c8-02c8d262d9c9-kube-api-access-2nm89\") pod \"glance-8fcf-account-create-update-r5h2k\" (UID: \"c665a3eb-31e2-4ac4-99c8-02c8d262d9c9\") " pod="openstack-kuttl-tests/glance-8fcf-account-create-update-r5h2k" Dec 03 12:02:46 crc kubenswrapper[5052]: I1203 12:02:46.909351 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/c665a3eb-31e2-4ac4-99c8-02c8d262d9c9-operator-scripts\") pod \"glance-8fcf-account-create-update-r5h2k\" (UID: \"c665a3eb-31e2-4ac4-99c8-02c8d262d9c9\") " pod="openstack-kuttl-tests/glance-8fcf-account-create-update-r5h2k" Dec 03 12:02:46 crc kubenswrapper[5052]: I1203 12:02:46.934457 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-2nm89\" (UniqueName: \"kubernetes.io/projected/c665a3eb-31e2-4ac4-99c8-02c8d262d9c9-kube-api-access-2nm89\") pod \"glance-8fcf-account-create-update-r5h2k\" (UID: \"c665a3eb-31e2-4ac4-99c8-02c8d262d9c9\") " pod="openstack-kuttl-tests/glance-8fcf-account-create-update-r5h2k" Dec 03 12:02:46 crc kubenswrapper[5052]: I1203 12:02:46.991120 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-db-create-jpmhr" Dec 03 12:02:47 crc kubenswrapper[5052]: I1203 12:02:47.039194 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/openstack-cell1-galera-2" Dec 03 12:02:47 crc kubenswrapper[5052]: I1203 12:02:47.102932 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-8fcf-account-create-update-r5h2k" Dec 03 12:02:47 crc kubenswrapper[5052]: I1203 12:02:47.215416 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-db-create-zgx94" event={"ID":"1e7cf8cd-09e8-4bfe-b8d3-9e509d4eae43","Type":"ContainerStarted","Data":"340f29d9a49e735b2a09dd64a1f01405a241208a475c2a1a778de21c81848ea0"} Dec 03 12:02:47 crc kubenswrapper[5052]: I1203 12:02:47.215475 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-db-create-zgx94" event={"ID":"1e7cf8cd-09e8-4bfe-b8d3-9e509d4eae43","Type":"ContainerStarted","Data":"799ef3fcae8d7d1e8a0366c282c6eca5b8e225a9ed0a295bf91a15c08248e7c2"} Dec 03 12:02:47 crc kubenswrapper[5052]: I1203 12:02:47.216862 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/placement-db-create-cmps4"] Dec 03 12:02:47 crc kubenswrapper[5052]: I1203 12:02:47.225296 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-25ff-account-create-update-hcnv4" event={"ID":"66b581e0-3a48-46c9-b1a7-384b87d2b7f4","Type":"ContainerStarted","Data":"1f0a6242e855eb2de1f01d290d9a30281da2eebae9925041a603e7661cfaf07c"} Dec 03 12:02:47 crc kubenswrapper[5052]: I1203 12:02:47.225354 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-25ff-account-create-update-hcnv4" event={"ID":"66b581e0-3a48-46c9-b1a7-384b87d2b7f4","Type":"ContainerStarted","Data":"60bcb606f3eddf72a3fdce4512ad4f88f2a9e9204ffb6db42790b45d244bb86f"} Dec 03 12:02:47 crc kubenswrapper[5052]: I1203 12:02:47.251559 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/keystone-db-create-zgx94" podStartSLOduration=2.251534102 podStartE2EDuration="2.251534102s" podCreationTimestamp="2025-12-03 12:02:45 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:02:47.246892079 +0000 UTC m=+2557.835133843" watchObservedRunningTime="2025-12-03 12:02:47.251534102 +0000 UTC m=+2557.839775866" Dec 03 12:02:47 crc kubenswrapper[5052]: I1203 12:02:47.282383 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/keystone-25ff-account-create-update-hcnv4" podStartSLOduration=2.282360613 podStartE2EDuration="2.282360613s" podCreationTimestamp="2025-12-03 12:02:45 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:02:47.271884054 +0000 UTC m=+2557.860125828" watchObservedRunningTime="2025-12-03 12:02:47.282360613 +0000 UTC m=+2557.870602377" Dec 03 12:02:47 crc kubenswrapper[5052]: I1203 12:02:47.325685 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/placement-3c40-account-create-update-dtmv4"] Dec 03 12:02:47 crc kubenswrapper[5052]: I1203 12:02:47.607488 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-db-create-jpmhr"] Dec 03 12:02:47 crc kubenswrapper[5052]: W1203 12:02:47.611237 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod246ced2f_39a7_496b_93cd_9f8ab898e2bb.slice/crio-06066766dc7585bde20d8034f678b543b12c6fd8aba4a578b04e6b51216d7f01 WatchSource:0}: Error finding container 06066766dc7585bde20d8034f678b543b12c6fd8aba4a578b04e6b51216d7f01: Status 404 returned error can't find the container with id 06066766dc7585bde20d8034f678b543b12c6fd8aba4a578b04e6b51216d7f01 Dec 03 12:02:47 crc kubenswrapper[5052]: I1203 12:02:47.711792 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-8fcf-account-create-update-r5h2k"] Dec 03 12:02:47 crc kubenswrapper[5052]: W1203 12:02:47.717017 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc665a3eb_31e2_4ac4_99c8_02c8d262d9c9.slice/crio-a64fdcde8efe21863ed651f6a7b647da1c77c31ae3d2bdfd2036d58cd2d78bbb WatchSource:0}: Error finding container a64fdcde8efe21863ed651f6a7b647da1c77c31ae3d2bdfd2036d58cd2d78bbb: Status 404 returned error can't find the container with id a64fdcde8efe21863ed651f6a7b647da1c77c31ae3d2bdfd2036d58cd2d78bbb Dec 03 12:02:48 crc kubenswrapper[5052]: I1203 12:02:48.237311 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-3c40-account-create-update-dtmv4" event={"ID":"2a25df1a-4f92-4ed6-9c34-2d6b0212593b","Type":"ContainerStarted","Data":"22c5907c0a73443041c9628057637dc9e498c4fbefa6b6e588b40e69b81c8b9c"} Dec 03 12:02:48 crc kubenswrapper[5052]: I1203 12:02:48.237367 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-3c40-account-create-update-dtmv4" event={"ID":"2a25df1a-4f92-4ed6-9c34-2d6b0212593b","Type":"ContainerStarted","Data":"ed0036318714f166c45a6b5987a46fc1bd7581b9c81691d0c1cfff54e30bebec"} Dec 03 12:02:48 crc kubenswrapper[5052]: I1203 12:02:48.241222 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-db-create-jpmhr" event={"ID":"246ced2f-39a7-496b-93cd-9f8ab898e2bb","Type":"ContainerStarted","Data":"71a1c1292c165a451da8f37d64d511b73b4e6bbaa158240dcc9e02122099a881"} Dec 03 12:02:48 crc kubenswrapper[5052]: I1203 12:02:48.241273 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-db-create-jpmhr" event={"ID":"246ced2f-39a7-496b-93cd-9f8ab898e2bb","Type":"ContainerStarted","Data":"06066766dc7585bde20d8034f678b543b12c6fd8aba4a578b04e6b51216d7f01"} Dec 03 12:02:48 crc kubenswrapper[5052]: I1203 12:02:48.244727 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-8fcf-account-create-update-r5h2k" event={"ID":"c665a3eb-31e2-4ac4-99c8-02c8d262d9c9","Type":"ContainerStarted","Data":"cf02feda51ed3db039ca069c4ae4a593ecba062d0a9b67f4c7d1a608b509c46f"} Dec 03 12:02:48 crc kubenswrapper[5052]: I1203 12:02:48.244778 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-8fcf-account-create-update-r5h2k" event={"ID":"c665a3eb-31e2-4ac4-99c8-02c8d262d9c9","Type":"ContainerStarted","Data":"a64fdcde8efe21863ed651f6a7b647da1c77c31ae3d2bdfd2036d58cd2d78bbb"} Dec 03 12:02:48 crc kubenswrapper[5052]: I1203 12:02:48.248743 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-db-create-cmps4" event={"ID":"79e77d8b-2481-4cbf-875f-cb626cf89463","Type":"ContainerStarted","Data":"368138b9d13c9ac7ee6c0551f792b75b2fa0c005de037787049a4e19da105713"} Dec 03 12:02:48 crc kubenswrapper[5052]: I1203 12:02:48.248800 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-db-create-cmps4" event={"ID":"79e77d8b-2481-4cbf-875f-cb626cf89463","Type":"ContainerStarted","Data":"cedf0d01ede14ccb8b89391a9b4e0cfbf4c62158f012617b23fecbdb4af0d236"} Dec 03 12:02:48 crc kubenswrapper[5052]: I1203 12:02:48.262611 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/placement-3c40-account-create-update-dtmv4" podStartSLOduration=2.262591258 podStartE2EDuration="2.262591258s" podCreationTimestamp="2025-12-03 12:02:46 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:02:48.259448708 +0000 UTC m=+2558.847690472" watchObservedRunningTime="2025-12-03 12:02:48.262591258 +0000 UTC m=+2558.850833022" Dec 03 12:02:48 crc kubenswrapper[5052]: I1203 12:02:48.285169 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/glance-8fcf-account-create-update-r5h2k" podStartSLOduration=2.285146063 podStartE2EDuration="2.285146063s" podCreationTimestamp="2025-12-03 12:02:46 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:02:48.274851758 +0000 UTC m=+2558.863093522" watchObservedRunningTime="2025-12-03 12:02:48.285146063 +0000 UTC m=+2558.873387827" Dec 03 12:02:48 crc kubenswrapper[5052]: I1203 12:02:48.296315 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/placement-db-create-cmps4" podStartSLOduration=2.296290761 podStartE2EDuration="2.296290761s" podCreationTimestamp="2025-12-03 12:02:46 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:02:48.286657596 +0000 UTC m=+2558.874899370" watchObservedRunningTime="2025-12-03 12:02:48.296290761 +0000 UTC m=+2558.884532525" Dec 03 12:02:48 crc kubenswrapper[5052]: I1203 12:02:48.299985 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/glance-db-create-jpmhr" podStartSLOduration=2.299969796 podStartE2EDuration="2.299969796s" podCreationTimestamp="2025-12-03 12:02:46 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:02:48.298419112 +0000 UTC m=+2558.886660876" watchObservedRunningTime="2025-12-03 12:02:48.299969796 +0000 UTC m=+2558.888211550" Dec 03 12:02:49 crc kubenswrapper[5052]: I1203 12:02:49.261668 5052 generic.go:334] "Generic (PLEG): container finished" podID="907d9f47-ab0b-44c8-9aa6-3d135e1c312d" containerID="feac02bd195cd4010d3557ca2d9bccbb1a2e3e5d4cdedd3139212667bfa80b4b" exitCode=0 Dec 03 12:02:49 crc kubenswrapper[5052]: I1203 12:02:49.261806 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-ring-rebalance-g9qd6" event={"ID":"907d9f47-ab0b-44c8-9aa6-3d135e1c312d","Type":"ContainerDied","Data":"feac02bd195cd4010d3557ca2d9bccbb1a2e3e5d4cdedd3139212667bfa80b4b"} Dec 03 12:02:50 crc kubenswrapper[5052]: I1203 12:02:50.272715 5052 generic.go:334] "Generic (PLEG): container finished" podID="246ced2f-39a7-496b-93cd-9f8ab898e2bb" containerID="71a1c1292c165a451da8f37d64d511b73b4e6bbaa158240dcc9e02122099a881" exitCode=0 Dec 03 12:02:50 crc kubenswrapper[5052]: I1203 12:02:50.272785 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-db-create-jpmhr" event={"ID":"246ced2f-39a7-496b-93cd-9f8ab898e2bb","Type":"ContainerDied","Data":"71a1c1292c165a451da8f37d64d511b73b4e6bbaa158240dcc9e02122099a881"} Dec 03 12:02:50 crc kubenswrapper[5052]: I1203 12:02:50.275293 5052 generic.go:334] "Generic (PLEG): container finished" podID="c665a3eb-31e2-4ac4-99c8-02c8d262d9c9" containerID="cf02feda51ed3db039ca069c4ae4a593ecba062d0a9b67f4c7d1a608b509c46f" exitCode=0 Dec 03 12:02:50 crc kubenswrapper[5052]: I1203 12:02:50.275360 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-8fcf-account-create-update-r5h2k" event={"ID":"c665a3eb-31e2-4ac4-99c8-02c8d262d9c9","Type":"ContainerDied","Data":"cf02feda51ed3db039ca069c4ae4a593ecba062d0a9b67f4c7d1a608b509c46f"} Dec 03 12:02:50 crc kubenswrapper[5052]: I1203 12:02:50.281078 5052 generic.go:334] "Generic (PLEG): container finished" podID="1e7cf8cd-09e8-4bfe-b8d3-9e509d4eae43" containerID="340f29d9a49e735b2a09dd64a1f01405a241208a475c2a1a778de21c81848ea0" exitCode=0 Dec 03 12:02:50 crc kubenswrapper[5052]: I1203 12:02:50.281155 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-db-create-zgx94" event={"ID":"1e7cf8cd-09e8-4bfe-b8d3-9e509d4eae43","Type":"ContainerDied","Data":"340f29d9a49e735b2a09dd64a1f01405a241208a475c2a1a778de21c81848ea0"} Dec 03 12:02:50 crc kubenswrapper[5052]: I1203 12:02:50.290872 5052 generic.go:334] "Generic (PLEG): container finished" podID="66b581e0-3a48-46c9-b1a7-384b87d2b7f4" containerID="1f0a6242e855eb2de1f01d290d9a30281da2eebae9925041a603e7661cfaf07c" exitCode=0 Dec 03 12:02:50 crc kubenswrapper[5052]: I1203 12:02:50.290965 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-25ff-account-create-update-hcnv4" event={"ID":"66b581e0-3a48-46c9-b1a7-384b87d2b7f4","Type":"ContainerDied","Data":"1f0a6242e855eb2de1f01d290d9a30281da2eebae9925041a603e7661cfaf07c"} Dec 03 12:02:50 crc kubenswrapper[5052]: I1203 12:02:50.296675 5052 generic.go:334] "Generic (PLEG): container finished" podID="79e77d8b-2481-4cbf-875f-cb626cf89463" containerID="368138b9d13c9ac7ee6c0551f792b75b2fa0c005de037787049a4e19da105713" exitCode=0 Dec 03 12:02:50 crc kubenswrapper[5052]: I1203 12:02:50.296795 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-db-create-cmps4" event={"ID":"79e77d8b-2481-4cbf-875f-cb626cf89463","Type":"ContainerDied","Data":"368138b9d13c9ac7ee6c0551f792b75b2fa0c005de037787049a4e19da105713"} Dec 03 12:02:50 crc kubenswrapper[5052]: I1203 12:02:50.299175 5052 generic.go:334] "Generic (PLEG): container finished" podID="2a25df1a-4f92-4ed6-9c34-2d6b0212593b" containerID="22c5907c0a73443041c9628057637dc9e498c4fbefa6b6e588b40e69b81c8b9c" exitCode=0 Dec 03 12:02:50 crc kubenswrapper[5052]: I1203 12:02:50.299399 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-3c40-account-create-update-dtmv4" event={"ID":"2a25df1a-4f92-4ed6-9c34-2d6b0212593b","Type":"ContainerDied","Data":"22c5907c0a73443041c9628057637dc9e498c4fbefa6b6e588b40e69b81c8b9c"} Dec 03 12:02:50 crc kubenswrapper[5052]: I1203 12:02:50.693496 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-ring-rebalance-g9qd6" Dec 03 12:02:50 crc kubenswrapper[5052]: I1203 12:02:50.799544 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jzbfh\" (UniqueName: \"kubernetes.io/projected/907d9f47-ab0b-44c8-9aa6-3d135e1c312d-kube-api-access-jzbfh\") pod \"907d9f47-ab0b-44c8-9aa6-3d135e1c312d\" (UID: \"907d9f47-ab0b-44c8-9aa6-3d135e1c312d\") " Dec 03 12:02:50 crc kubenswrapper[5052]: I1203 12:02:50.799609 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/907d9f47-ab0b-44c8-9aa6-3d135e1c312d-scripts\") pod \"907d9f47-ab0b-44c8-9aa6-3d135e1c312d\" (UID: \"907d9f47-ab0b-44c8-9aa6-3d135e1c312d\") " Dec 03 12:02:50 crc kubenswrapper[5052]: I1203 12:02:50.799667 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ring-data-devices\" (UniqueName: \"kubernetes.io/configmap/907d9f47-ab0b-44c8-9aa6-3d135e1c312d-ring-data-devices\") pod \"907d9f47-ab0b-44c8-9aa6-3d135e1c312d\" (UID: \"907d9f47-ab0b-44c8-9aa6-3d135e1c312d\") " Dec 03 12:02:50 crc kubenswrapper[5052]: I1203 12:02:50.800346 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/empty-dir/907d9f47-ab0b-44c8-9aa6-3d135e1c312d-etc-swift\") pod \"907d9f47-ab0b-44c8-9aa6-3d135e1c312d\" (UID: \"907d9f47-ab0b-44c8-9aa6-3d135e1c312d\") " Dec 03 12:02:50 crc kubenswrapper[5052]: I1203 12:02:50.800375 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/907d9f47-ab0b-44c8-9aa6-3d135e1c312d-combined-ca-bundle\") pod \"907d9f47-ab0b-44c8-9aa6-3d135e1c312d\" (UID: \"907d9f47-ab0b-44c8-9aa6-3d135e1c312d\") " Dec 03 12:02:50 crc kubenswrapper[5052]: I1203 12:02:50.800390 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/907d9f47-ab0b-44c8-9aa6-3d135e1c312d-ring-data-devices" (OuterVolumeSpecName: "ring-data-devices") pod "907d9f47-ab0b-44c8-9aa6-3d135e1c312d" (UID: "907d9f47-ab0b-44c8-9aa6-3d135e1c312d"). InnerVolumeSpecName "ring-data-devices". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:02:50 crc kubenswrapper[5052]: I1203 12:02:50.800437 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"dispersionconf\" (UniqueName: \"kubernetes.io/secret/907d9f47-ab0b-44c8-9aa6-3d135e1c312d-dispersionconf\") pod \"907d9f47-ab0b-44c8-9aa6-3d135e1c312d\" (UID: \"907d9f47-ab0b-44c8-9aa6-3d135e1c312d\") " Dec 03 12:02:50 crc kubenswrapper[5052]: I1203 12:02:50.800485 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"swiftconf\" (UniqueName: \"kubernetes.io/secret/907d9f47-ab0b-44c8-9aa6-3d135e1c312d-swiftconf\") pod \"907d9f47-ab0b-44c8-9aa6-3d135e1c312d\" (UID: \"907d9f47-ab0b-44c8-9aa6-3d135e1c312d\") " Dec 03 12:02:50 crc kubenswrapper[5052]: I1203 12:02:50.800818 5052 reconciler_common.go:293] "Volume detached for volume \"ring-data-devices\" (UniqueName: \"kubernetes.io/configmap/907d9f47-ab0b-44c8-9aa6-3d135e1c312d-ring-data-devices\") on node \"crc\" DevicePath \"\"" Dec 03 12:02:50 crc kubenswrapper[5052]: I1203 12:02:50.801337 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/907d9f47-ab0b-44c8-9aa6-3d135e1c312d-etc-swift" (OuterVolumeSpecName: "etc-swift") pod "907d9f47-ab0b-44c8-9aa6-3d135e1c312d" (UID: "907d9f47-ab0b-44c8-9aa6-3d135e1c312d"). InnerVolumeSpecName "etc-swift". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:02:50 crc kubenswrapper[5052]: I1203 12:02:50.805494 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/907d9f47-ab0b-44c8-9aa6-3d135e1c312d-kube-api-access-jzbfh" (OuterVolumeSpecName: "kube-api-access-jzbfh") pod "907d9f47-ab0b-44c8-9aa6-3d135e1c312d" (UID: "907d9f47-ab0b-44c8-9aa6-3d135e1c312d"). InnerVolumeSpecName "kube-api-access-jzbfh". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:02:50 crc kubenswrapper[5052]: I1203 12:02:50.812882 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/907d9f47-ab0b-44c8-9aa6-3d135e1c312d-dispersionconf" (OuterVolumeSpecName: "dispersionconf") pod "907d9f47-ab0b-44c8-9aa6-3d135e1c312d" (UID: "907d9f47-ab0b-44c8-9aa6-3d135e1c312d"). InnerVolumeSpecName "dispersionconf". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:02:50 crc kubenswrapper[5052]: I1203 12:02:50.832411 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/907d9f47-ab0b-44c8-9aa6-3d135e1c312d-scripts" (OuterVolumeSpecName: "scripts") pod "907d9f47-ab0b-44c8-9aa6-3d135e1c312d" (UID: "907d9f47-ab0b-44c8-9aa6-3d135e1c312d"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:02:50 crc kubenswrapper[5052]: I1203 12:02:50.840030 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/907d9f47-ab0b-44c8-9aa6-3d135e1c312d-swiftconf" (OuterVolumeSpecName: "swiftconf") pod "907d9f47-ab0b-44c8-9aa6-3d135e1c312d" (UID: "907d9f47-ab0b-44c8-9aa6-3d135e1c312d"). InnerVolumeSpecName "swiftconf". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:02:50 crc kubenswrapper[5052]: I1203 12:02:50.841484 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/907d9f47-ab0b-44c8-9aa6-3d135e1c312d-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "907d9f47-ab0b-44c8-9aa6-3d135e1c312d" (UID: "907d9f47-ab0b-44c8-9aa6-3d135e1c312d"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:02:50 crc kubenswrapper[5052]: I1203 12:02:50.903435 5052 reconciler_common.go:293] "Volume detached for volume \"etc-swift\" (UniqueName: \"kubernetes.io/empty-dir/907d9f47-ab0b-44c8-9aa6-3d135e1c312d-etc-swift\") on node \"crc\" DevicePath \"\"" Dec 03 12:02:50 crc kubenswrapper[5052]: I1203 12:02:50.903486 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/907d9f47-ab0b-44c8-9aa6-3d135e1c312d-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:02:50 crc kubenswrapper[5052]: I1203 12:02:50.903547 5052 reconciler_common.go:293] "Volume detached for volume \"dispersionconf\" (UniqueName: \"kubernetes.io/secret/907d9f47-ab0b-44c8-9aa6-3d135e1c312d-dispersionconf\") on node \"crc\" DevicePath \"\"" Dec 03 12:02:50 crc kubenswrapper[5052]: I1203 12:02:50.903563 5052 reconciler_common.go:293] "Volume detached for volume \"swiftconf\" (UniqueName: \"kubernetes.io/secret/907d9f47-ab0b-44c8-9aa6-3d135e1c312d-swiftconf\") on node \"crc\" DevicePath \"\"" Dec 03 12:02:50 crc kubenswrapper[5052]: I1203 12:02:50.903578 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-jzbfh\" (UniqueName: \"kubernetes.io/projected/907d9f47-ab0b-44c8-9aa6-3d135e1c312d-kube-api-access-jzbfh\") on node \"crc\" DevicePath \"\"" Dec 03 12:02:50 crc kubenswrapper[5052]: I1203 12:02:50.903591 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/907d9f47-ab0b-44c8-9aa6-3d135e1c312d-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:02:51 crc kubenswrapper[5052]: I1203 12:02:51.311581 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-ring-rebalance-g9qd6" Dec 03 12:02:51 crc kubenswrapper[5052]: I1203 12:02:51.312526 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-ring-rebalance-g9qd6" event={"ID":"907d9f47-ab0b-44c8-9aa6-3d135e1c312d","Type":"ContainerDied","Data":"fa317abab0006965a51cf1e8e792f7cae1e6364335412d963cd5261a3acf7124"} Dec 03 12:02:51 crc kubenswrapper[5052]: I1203 12:02:51.312567 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fa317abab0006965a51cf1e8e792f7cae1e6364335412d963cd5261a3acf7124" Dec 03 12:02:51 crc kubenswrapper[5052]: I1203 12:02:51.380719 5052 scope.go:117] "RemoveContainer" containerID="07458cccfd7ebfac3961356761dea4f45e484fb48cbe566dff5f71534efbd7ff" Dec 03 12:02:51 crc kubenswrapper[5052]: I1203 12:02:51.446758 5052 scope.go:117] "RemoveContainer" containerID="cd5d0e91d5279fe56400b9ddb8c67759af038b8df5fd0c0db561a366628bc6a3" Dec 03 12:02:51 crc kubenswrapper[5052]: I1203 12:02:51.544427 5052 scope.go:117] "RemoveContainer" containerID="9d6766f724c021f51b09b306536e6d9042806dbfded59d6853ab9da78a74450c" Dec 03 12:02:51 crc kubenswrapper[5052]: I1203 12:02:51.588565 5052 scope.go:117] "RemoveContainer" containerID="35db98cfc0c0e4e1307993f339292d48cb6c7695dea2f6b7b92d067b386acd94" Dec 03 12:02:51 crc kubenswrapper[5052]: I1203 12:02:51.681818 5052 scope.go:117] "RemoveContainer" containerID="021eba600b1eeccb1d30827210c0e0365f75e3571270f84c0206d112c79c35af" Dec 03 12:02:51 crc kubenswrapper[5052]: I1203 12:02:51.724571 5052 scope.go:117] "RemoveContainer" containerID="9b319e148e0126893b069cd2f5fdc7170727f3a457ff8ccf29646edcca07668d" Dec 03 12:02:51 crc kubenswrapper[5052]: I1203 12:02:51.835174 5052 scope.go:117] "RemoveContainer" containerID="0a02427bd50c50cbb993f7fe4b5961d674d53b51ca7dbe5de9fca001e0d11572" Dec 03 12:02:51 crc kubenswrapper[5052]: I1203 12:02:51.840598 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-db-create-cmps4" Dec 03 12:02:51 crc kubenswrapper[5052]: I1203 12:02:51.881826 5052 scope.go:117] "RemoveContainer" containerID="4fae91e98a69ef557495570d6dcbe9cef23173f11867aae8ecb37fecd4959d8b" Dec 03 12:02:51 crc kubenswrapper[5052]: I1203 12:02:51.944061 5052 scope.go:117] "RemoveContainer" containerID="986d340d25b89c2b20e9877dd2590330ca97a1a957d198b877ab542caeca6588" Dec 03 12:02:51 crc kubenswrapper[5052]: I1203 12:02:51.968658 5052 scope.go:117] "RemoveContainer" containerID="5e5c3ad7b8915e802d7144bd906adc382a388dd664d9902320496f0eaa0909bc" Dec 03 12:02:52 crc kubenswrapper[5052]: I1203 12:02:52.011179 5052 scope.go:117] "RemoveContainer" containerID="324fde5777eff4b5588948210f2107fc67b80463e3177dbcc4c61e7e419dc4cd" Dec 03 12:02:52 crc kubenswrapper[5052]: I1203 12:02:52.038187 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zz7r7\" (UniqueName: \"kubernetes.io/projected/79e77d8b-2481-4cbf-875f-cb626cf89463-kube-api-access-zz7r7\") pod \"79e77d8b-2481-4cbf-875f-cb626cf89463\" (UID: \"79e77d8b-2481-4cbf-875f-cb626cf89463\") " Dec 03 12:02:52 crc kubenswrapper[5052]: I1203 12:02:52.038293 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/79e77d8b-2481-4cbf-875f-cb626cf89463-operator-scripts\") pod \"79e77d8b-2481-4cbf-875f-cb626cf89463\" (UID: \"79e77d8b-2481-4cbf-875f-cb626cf89463\") " Dec 03 12:02:52 crc kubenswrapper[5052]: I1203 12:02:52.041291 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/79e77d8b-2481-4cbf-875f-cb626cf89463-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "79e77d8b-2481-4cbf-875f-cb626cf89463" (UID: "79e77d8b-2481-4cbf-875f-cb626cf89463"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:02:52 crc kubenswrapper[5052]: I1203 12:02:52.061913 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/79e77d8b-2481-4cbf-875f-cb626cf89463-kube-api-access-zz7r7" (OuterVolumeSpecName: "kube-api-access-zz7r7") pod "79e77d8b-2481-4cbf-875f-cb626cf89463" (UID: "79e77d8b-2481-4cbf-875f-cb626cf89463"). InnerVolumeSpecName "kube-api-access-zz7r7". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:02:52 crc kubenswrapper[5052]: I1203 12:02:52.061919 5052 scope.go:117] "RemoveContainer" containerID="e4b74aed237af57c4a5ccac244bde0abf28abccef9d3f249bad7df1bbb43da9d" Dec 03 12:02:52 crc kubenswrapper[5052]: I1203 12:02:52.095052 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-db-create-zgx94" Dec 03 12:02:52 crc kubenswrapper[5052]: I1203 12:02:52.128172 5052 scope.go:117] "RemoveContainer" containerID="1ed38008a53ff370520241b3fd280d78870c056da808615558e81bfec79012cf" Dec 03 12:02:52 crc kubenswrapper[5052]: I1203 12:02:52.141071 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-zz7r7\" (UniqueName: \"kubernetes.io/projected/79e77d8b-2481-4cbf-875f-cb626cf89463-kube-api-access-zz7r7\") on node \"crc\" DevicePath \"\"" Dec 03 12:02:52 crc kubenswrapper[5052]: I1203 12:02:52.141098 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/79e77d8b-2481-4cbf-875f-cb626cf89463-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:02:52 crc kubenswrapper[5052]: I1203 12:02:52.158564 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-25ff-account-create-update-hcnv4" Dec 03 12:02:52 crc kubenswrapper[5052]: I1203 12:02:52.166028 5052 scope.go:117] "RemoveContainer" containerID="9c3e540a901f3f23370d4b704b73c51ab43a8999c92566dd189c3398da965634" Dec 03 12:02:52 crc kubenswrapper[5052]: I1203 12:02:52.166398 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-8fcf-account-create-update-r5h2k" Dec 03 12:02:52 crc kubenswrapper[5052]: I1203 12:02:52.197853 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-db-create-jpmhr" Dec 03 12:02:52 crc kubenswrapper[5052]: I1203 12:02:52.200926 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-3c40-account-create-update-dtmv4" Dec 03 12:02:52 crc kubenswrapper[5052]: I1203 12:02:52.242984 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qcfnk\" (UniqueName: \"kubernetes.io/projected/1e7cf8cd-09e8-4bfe-b8d3-9e509d4eae43-kube-api-access-qcfnk\") pod \"1e7cf8cd-09e8-4bfe-b8d3-9e509d4eae43\" (UID: \"1e7cf8cd-09e8-4bfe-b8d3-9e509d4eae43\") " Dec 03 12:02:52 crc kubenswrapper[5052]: I1203 12:02:52.243073 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/1e7cf8cd-09e8-4bfe-b8d3-9e509d4eae43-operator-scripts\") pod \"1e7cf8cd-09e8-4bfe-b8d3-9e509d4eae43\" (UID: \"1e7cf8cd-09e8-4bfe-b8d3-9e509d4eae43\") " Dec 03 12:02:52 crc kubenswrapper[5052]: I1203 12:02:52.244590 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1e7cf8cd-09e8-4bfe-b8d3-9e509d4eae43-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "1e7cf8cd-09e8-4bfe-b8d3-9e509d4eae43" (UID: "1e7cf8cd-09e8-4bfe-b8d3-9e509d4eae43"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:02:52 crc kubenswrapper[5052]: I1203 12:02:52.247430 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1e7cf8cd-09e8-4bfe-b8d3-9e509d4eae43-kube-api-access-qcfnk" (OuterVolumeSpecName: "kube-api-access-qcfnk") pod "1e7cf8cd-09e8-4bfe-b8d3-9e509d4eae43" (UID: "1e7cf8cd-09e8-4bfe-b8d3-9e509d4eae43"). InnerVolumeSpecName "kube-api-access-qcfnk". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:02:52 crc kubenswrapper[5052]: I1203 12:02:52.259021 5052 scope.go:117] "RemoveContainer" containerID="fdc5cd84bfb15a2c4b38c80fec90fd397d2d372bb2cd5443eb73cf72b1a62b30" Dec 03 12:02:52 crc kubenswrapper[5052]: I1203 12:02:52.346311 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4tshg\" (UniqueName: \"kubernetes.io/projected/66b581e0-3a48-46c9-b1a7-384b87d2b7f4-kube-api-access-4tshg\") pod \"66b581e0-3a48-46c9-b1a7-384b87d2b7f4\" (UID: \"66b581e0-3a48-46c9-b1a7-384b87d2b7f4\") " Dec 03 12:02:52 crc kubenswrapper[5052]: I1203 12:02:52.346456 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2nm89\" (UniqueName: \"kubernetes.io/projected/c665a3eb-31e2-4ac4-99c8-02c8d262d9c9-kube-api-access-2nm89\") pod \"c665a3eb-31e2-4ac4-99c8-02c8d262d9c9\" (UID: \"c665a3eb-31e2-4ac4-99c8-02c8d262d9c9\") " Dec 03 12:02:52 crc kubenswrapper[5052]: I1203 12:02:52.346868 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/c665a3eb-31e2-4ac4-99c8-02c8d262d9c9-operator-scripts\") pod \"c665a3eb-31e2-4ac4-99c8-02c8d262d9c9\" (UID: \"c665a3eb-31e2-4ac4-99c8-02c8d262d9c9\") " Dec 03 12:02:52 crc kubenswrapper[5052]: I1203 12:02:52.346920 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zdzvg\" (UniqueName: \"kubernetes.io/projected/246ced2f-39a7-496b-93cd-9f8ab898e2bb-kube-api-access-zdzvg\") pod \"246ced2f-39a7-496b-93cd-9f8ab898e2bb\" (UID: \"246ced2f-39a7-496b-93cd-9f8ab898e2bb\") " Dec 03 12:02:52 crc kubenswrapper[5052]: I1203 12:02:52.346955 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/246ced2f-39a7-496b-93cd-9f8ab898e2bb-operator-scripts\") pod \"246ced2f-39a7-496b-93cd-9f8ab898e2bb\" (UID: \"246ced2f-39a7-496b-93cd-9f8ab898e2bb\") " Dec 03 12:02:52 crc kubenswrapper[5052]: I1203 12:02:52.347343 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hs9lw\" (UniqueName: \"kubernetes.io/projected/2a25df1a-4f92-4ed6-9c34-2d6b0212593b-kube-api-access-hs9lw\") pod \"2a25df1a-4f92-4ed6-9c34-2d6b0212593b\" (UID: \"2a25df1a-4f92-4ed6-9c34-2d6b0212593b\") " Dec 03 12:02:52 crc kubenswrapper[5052]: I1203 12:02:52.347531 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/2a25df1a-4f92-4ed6-9c34-2d6b0212593b-operator-scripts\") pod \"2a25df1a-4f92-4ed6-9c34-2d6b0212593b\" (UID: \"2a25df1a-4f92-4ed6-9c34-2d6b0212593b\") " Dec 03 12:02:52 crc kubenswrapper[5052]: I1203 12:02:52.348226 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/66b581e0-3a48-46c9-b1a7-384b87d2b7f4-operator-scripts\") pod \"66b581e0-3a48-46c9-b1a7-384b87d2b7f4\" (UID: \"66b581e0-3a48-46c9-b1a7-384b87d2b7f4\") " Dec 03 12:02:52 crc kubenswrapper[5052]: I1203 12:02:52.348850 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2a25df1a-4f92-4ed6-9c34-2d6b0212593b-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "2a25df1a-4f92-4ed6-9c34-2d6b0212593b" (UID: "2a25df1a-4f92-4ed6-9c34-2d6b0212593b"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:02:52 crc kubenswrapper[5052]: I1203 12:02:52.349022 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/66b581e0-3a48-46c9-b1a7-384b87d2b7f4-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "66b581e0-3a48-46c9-b1a7-384b87d2b7f4" (UID: "66b581e0-3a48-46c9-b1a7-384b87d2b7f4"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:02:52 crc kubenswrapper[5052]: I1203 12:02:52.350150 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/246ced2f-39a7-496b-93cd-9f8ab898e2bb-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "246ced2f-39a7-496b-93cd-9f8ab898e2bb" (UID: "246ced2f-39a7-496b-93cd-9f8ab898e2bb"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:02:52 crc kubenswrapper[5052]: I1203 12:02:52.374001 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c665a3eb-31e2-4ac4-99c8-02c8d262d9c9-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "c665a3eb-31e2-4ac4-99c8-02c8d262d9c9" (UID: "c665a3eb-31e2-4ac4-99c8-02c8d262d9c9"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:02:52 crc kubenswrapper[5052]: I1203 12:02:52.374321 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/66b581e0-3a48-46c9-b1a7-384b87d2b7f4-kube-api-access-4tshg" (OuterVolumeSpecName: "kube-api-access-4tshg") pod "66b581e0-3a48-46c9-b1a7-384b87d2b7f4" (UID: "66b581e0-3a48-46c9-b1a7-384b87d2b7f4"). InnerVolumeSpecName "kube-api-access-4tshg". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:02:52 crc kubenswrapper[5052]: I1203 12:02:52.374343 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/246ced2f-39a7-496b-93cd-9f8ab898e2bb-kube-api-access-zdzvg" (OuterVolumeSpecName: "kube-api-access-zdzvg") pod "246ced2f-39a7-496b-93cd-9f8ab898e2bb" (UID: "246ced2f-39a7-496b-93cd-9f8ab898e2bb"). InnerVolumeSpecName "kube-api-access-zdzvg". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:02:52 crc kubenswrapper[5052]: I1203 12:02:52.374373 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c665a3eb-31e2-4ac4-99c8-02c8d262d9c9-kube-api-access-2nm89" (OuterVolumeSpecName: "kube-api-access-2nm89") pod "c665a3eb-31e2-4ac4-99c8-02c8d262d9c9" (UID: "c665a3eb-31e2-4ac4-99c8-02c8d262d9c9"). InnerVolumeSpecName "kube-api-access-2nm89". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:02:52 crc kubenswrapper[5052]: I1203 12:02:52.374484 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2a25df1a-4f92-4ed6-9c34-2d6b0212593b-kube-api-access-hs9lw" (OuterVolumeSpecName: "kube-api-access-hs9lw") pod "2a25df1a-4f92-4ed6-9c34-2d6b0212593b" (UID: "2a25df1a-4f92-4ed6-9c34-2d6b0212593b"). InnerVolumeSpecName "kube-api-access-hs9lw". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:02:52 crc kubenswrapper[5052]: I1203 12:02:52.375655 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-hs9lw\" (UniqueName: \"kubernetes.io/projected/2a25df1a-4f92-4ed6-9c34-2d6b0212593b-kube-api-access-hs9lw\") on node \"crc\" DevicePath \"\"" Dec 03 12:02:52 crc kubenswrapper[5052]: I1203 12:02:52.375683 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/2a25df1a-4f92-4ed6-9c34-2d6b0212593b-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:02:52 crc kubenswrapper[5052]: I1203 12:02:52.375703 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/66b581e0-3a48-46c9-b1a7-384b87d2b7f4-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:02:52 crc kubenswrapper[5052]: I1203 12:02:52.375717 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-4tshg\" (UniqueName: \"kubernetes.io/projected/66b581e0-3a48-46c9-b1a7-384b87d2b7f4-kube-api-access-4tshg\") on node \"crc\" DevicePath \"\"" Dec 03 12:02:52 crc kubenswrapper[5052]: I1203 12:02:52.375728 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-qcfnk\" (UniqueName: \"kubernetes.io/projected/1e7cf8cd-09e8-4bfe-b8d3-9e509d4eae43-kube-api-access-qcfnk\") on node \"crc\" DevicePath \"\"" Dec 03 12:02:52 crc kubenswrapper[5052]: I1203 12:02:52.375739 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-2nm89\" (UniqueName: \"kubernetes.io/projected/c665a3eb-31e2-4ac4-99c8-02c8d262d9c9-kube-api-access-2nm89\") on node \"crc\" DevicePath \"\"" Dec 03 12:02:52 crc kubenswrapper[5052]: I1203 12:02:52.375750 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/1e7cf8cd-09e8-4bfe-b8d3-9e509d4eae43-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:02:52 crc kubenswrapper[5052]: I1203 12:02:52.375768 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/c665a3eb-31e2-4ac4-99c8-02c8d262d9c9-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:02:52 crc kubenswrapper[5052]: I1203 12:02:52.375783 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-zdzvg\" (UniqueName: \"kubernetes.io/projected/246ced2f-39a7-496b-93cd-9f8ab898e2bb-kube-api-access-zdzvg\") on node \"crc\" DevicePath \"\"" Dec 03 12:02:52 crc kubenswrapper[5052]: I1203 12:02:52.375794 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/246ced2f-39a7-496b-93cd-9f8ab898e2bb-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:02:52 crc kubenswrapper[5052]: I1203 12:02:52.379556 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-db-create-jpmhr" event={"ID":"246ced2f-39a7-496b-93cd-9f8ab898e2bb","Type":"ContainerDied","Data":"06066766dc7585bde20d8034f678b543b12c6fd8aba4a578b04e6b51216d7f01"} Dec 03 12:02:52 crc kubenswrapper[5052]: I1203 12:02:52.379598 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="06066766dc7585bde20d8034f678b543b12c6fd8aba4a578b04e6b51216d7f01" Dec 03 12:02:52 crc kubenswrapper[5052]: I1203 12:02:52.379668 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-db-create-jpmhr" Dec 03 12:02:52 crc kubenswrapper[5052]: I1203 12:02:52.405360 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-8fcf-account-create-update-r5h2k" event={"ID":"c665a3eb-31e2-4ac4-99c8-02c8d262d9c9","Type":"ContainerDied","Data":"a64fdcde8efe21863ed651f6a7b647da1c77c31ae3d2bdfd2036d58cd2d78bbb"} Dec 03 12:02:52 crc kubenswrapper[5052]: I1203 12:02:52.405445 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a64fdcde8efe21863ed651f6a7b647da1c77c31ae3d2bdfd2036d58cd2d78bbb" Dec 03 12:02:52 crc kubenswrapper[5052]: I1203 12:02:52.405566 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-8fcf-account-create-update-r5h2k" Dec 03 12:02:52 crc kubenswrapper[5052]: I1203 12:02:52.413018 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-25ff-account-create-update-hcnv4" event={"ID":"66b581e0-3a48-46c9-b1a7-384b87d2b7f4","Type":"ContainerDied","Data":"60bcb606f3eddf72a3fdce4512ad4f88f2a9e9204ffb6db42790b45d244bb86f"} Dec 03 12:02:52 crc kubenswrapper[5052]: I1203 12:02:52.413121 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="60bcb606f3eddf72a3fdce4512ad4f88f2a9e9204ffb6db42790b45d244bb86f" Dec 03 12:02:52 crc kubenswrapper[5052]: I1203 12:02:52.413265 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-25ff-account-create-update-hcnv4" Dec 03 12:02:52 crc kubenswrapper[5052]: I1203 12:02:52.421317 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-db-create-cmps4" event={"ID":"79e77d8b-2481-4cbf-875f-cb626cf89463","Type":"ContainerDied","Data":"cedf0d01ede14ccb8b89391a9b4e0cfbf4c62158f012617b23fecbdb4af0d236"} Dec 03 12:02:52 crc kubenswrapper[5052]: I1203 12:02:52.421389 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cedf0d01ede14ccb8b89391a9b4e0cfbf4c62158f012617b23fecbdb4af0d236" Dec 03 12:02:52 crc kubenswrapper[5052]: I1203 12:02:52.421355 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-db-create-cmps4" Dec 03 12:02:52 crc kubenswrapper[5052]: I1203 12:02:52.448425 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-db-create-zgx94" Dec 03 12:02:52 crc kubenswrapper[5052]: I1203 12:02:52.448505 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-db-create-zgx94" event={"ID":"1e7cf8cd-09e8-4bfe-b8d3-9e509d4eae43","Type":"ContainerDied","Data":"799ef3fcae8d7d1e8a0366c282c6eca5b8e225a9ed0a295bf91a15c08248e7c2"} Dec 03 12:02:52 crc kubenswrapper[5052]: I1203 12:02:52.448577 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="799ef3fcae8d7d1e8a0366c282c6eca5b8e225a9ed0a295bf91a15c08248e7c2" Dec 03 12:02:52 crc kubenswrapper[5052]: I1203 12:02:52.462708 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-3c40-account-create-update-dtmv4" event={"ID":"2a25df1a-4f92-4ed6-9c34-2d6b0212593b","Type":"ContainerDied","Data":"ed0036318714f166c45a6b5987a46fc1bd7581b9c81691d0c1cfff54e30bebec"} Dec 03 12:02:52 crc kubenswrapper[5052]: I1203 12:02:52.462920 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ed0036318714f166c45a6b5987a46fc1bd7581b9c81691d0c1cfff54e30bebec" Dec 03 12:02:52 crc kubenswrapper[5052]: I1203 12:02:52.462814 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-3c40-account-create-update-dtmv4" Dec 03 12:02:54 crc kubenswrapper[5052]: I1203 12:02:54.610689 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/openstack-galera-2" podUID="143f4f1d-4f91-4ffa-a277-d6e551d19838" containerName="galera" probeResult="failure" output=< Dec 03 12:02:54 crc kubenswrapper[5052]: wsrep_local_state_comment (Donor/Desynced) differs from Synced Dec 03 12:02:54 crc kubenswrapper[5052]: > Dec 03 12:02:54 crc kubenswrapper[5052]: I1203 12:02:54.718786 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/95bf5f72-82f3-4e88-a576-01dae857541f-etc-swift\") pod \"swift-storage-0\" (UID: \"95bf5f72-82f3-4e88-a576-01dae857541f\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:02:54 crc kubenswrapper[5052]: I1203 12:02:54.725228 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/95bf5f72-82f3-4e88-a576-01dae857541f-etc-swift\") pod \"swift-storage-0\" (UID: \"95bf5f72-82f3-4e88-a576-01dae857541f\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:02:54 crc kubenswrapper[5052]: I1203 12:02:54.814535 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:02:55 crc kubenswrapper[5052]: I1203 12:02:55.298898 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/swift-storage-0"] Dec 03 12:02:55 crc kubenswrapper[5052]: W1203 12:02:55.303733 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod95bf5f72_82f3_4e88_a576_01dae857541f.slice/crio-5230c658f0b920f5e8fe62e2ad61cb5ee6000feba05d7cc0ea6bcde57579cfe1 WatchSource:0}: Error finding container 5230c658f0b920f5e8fe62e2ad61cb5ee6000feba05d7cc0ea6bcde57579cfe1: Status 404 returned error can't find the container with id 5230c658f0b920f5e8fe62e2ad61cb5ee6000feba05d7cc0ea6bcde57579cfe1 Dec 03 12:02:55 crc kubenswrapper[5052]: I1203 12:02:55.510124 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"95bf5f72-82f3-4e88-a576-01dae857541f","Type":"ContainerStarted","Data":"5230c658f0b920f5e8fe62e2ad61cb5ee6000feba05d7cc0ea6bcde57579cfe1"} Dec 03 12:02:55 crc kubenswrapper[5052]: I1203 12:02:55.732267 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:02:56 crc kubenswrapper[5052]: I1203 12:02:56.362793 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/openstack-cell1-galera-2" podUID="6ac95bd3-125c-4966-b9a7-4eb4a7842c67" containerName="galera" probeResult="failure" output=< Dec 03 12:02:56 crc kubenswrapper[5052]: wsrep_local_state_comment (Donor/Desynced) differs from Synced Dec 03 12:02:56 crc kubenswrapper[5052]: > Dec 03 12:02:56 crc kubenswrapper[5052]: I1203 12:02:56.569832 5052 generic.go:334] "Generic (PLEG): container finished" podID="7b4f6078-41a3-433d-acaf-f2d67391ab6e" containerID="142af50f0ab6083a28092a109ed34ea8ab8faf77ec5102e5c37130c564ddc949" exitCode=0 Dec 03 12:02:56 crc kubenswrapper[5052]: I1203 12:02:56.569932 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-server-2" event={"ID":"7b4f6078-41a3-433d-acaf-f2d67391ab6e","Type":"ContainerDied","Data":"142af50f0ab6083a28092a109ed34ea8ab8faf77ec5102e5c37130c564ddc949"} Dec 03 12:02:56 crc kubenswrapper[5052]: I1203 12:02:56.593562 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"95bf5f72-82f3-4e88-a576-01dae857541f","Type":"ContainerStarted","Data":"3604bfccbdf55c7f8cc4aa8921d57387c338bcccd177ab45652bc037ec8fdad6"} Dec 03 12:02:56 crc kubenswrapper[5052]: I1203 12:02:56.593922 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"95bf5f72-82f3-4e88-a576-01dae857541f","Type":"ContainerStarted","Data":"003e4620ad1989f1fa65349bb9193c7be42568b4061355bee57a6b9ae2b3d5a2"} Dec 03 12:02:56 crc kubenswrapper[5052]: I1203 12:02:56.596310 5052 generic.go:334] "Generic (PLEG): container finished" podID="2397d896-0d11-4656-a336-eac8ec9fe7f9" containerID="c716ca61940f2dfaed65a15c3a09f0b9191853485d4f681077e718bcb0869b41" exitCode=0 Dec 03 12:02:56 crc kubenswrapper[5052]: I1203 12:02:56.596371 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-server-0" event={"ID":"2397d896-0d11-4656-a336-eac8ec9fe7f9","Type":"ContainerDied","Data":"c716ca61940f2dfaed65a15c3a09f0b9191853485d4f681077e718bcb0869b41"} Dec 03 12:02:56 crc kubenswrapper[5052]: I1203 12:02:56.618144 5052 generic.go:334] "Generic (PLEG): container finished" podID="7901ce85-4e6f-4e8b-86a6-f50a85803637" containerID="ccd68041ae4063c32d99edce20b9aa0bf23f148287d9fcd6406b6a5e57a96ff9" exitCode=0 Dec 03 12:02:56 crc kubenswrapper[5052]: I1203 12:02:56.618199 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-server-1" event={"ID":"7901ce85-4e6f-4e8b-86a6-f50a85803637","Type":"ContainerDied","Data":"ccd68041ae4063c32d99edce20b9aa0bf23f148287d9fcd6406b6a5e57a96ff9"} Dec 03 12:02:57 crc kubenswrapper[5052]: I1203 12:02:57.042990 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/glance-db-sync-2lq57"] Dec 03 12:02:57 crc kubenswrapper[5052]: E1203 12:02:57.043809 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="2a25df1a-4f92-4ed6-9c34-2d6b0212593b" containerName="mariadb-account-create-update" Dec 03 12:02:57 crc kubenswrapper[5052]: I1203 12:02:57.043828 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="2a25df1a-4f92-4ed6-9c34-2d6b0212593b" containerName="mariadb-account-create-update" Dec 03 12:02:57 crc kubenswrapper[5052]: E1203 12:02:57.043845 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="907d9f47-ab0b-44c8-9aa6-3d135e1c312d" containerName="swift-ring-rebalance" Dec 03 12:02:57 crc kubenswrapper[5052]: I1203 12:02:57.043853 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="907d9f47-ab0b-44c8-9aa6-3d135e1c312d" containerName="swift-ring-rebalance" Dec 03 12:02:57 crc kubenswrapper[5052]: E1203 12:02:57.043869 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="1e7cf8cd-09e8-4bfe-b8d3-9e509d4eae43" containerName="mariadb-database-create" Dec 03 12:02:57 crc kubenswrapper[5052]: I1203 12:02:57.043879 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="1e7cf8cd-09e8-4bfe-b8d3-9e509d4eae43" containerName="mariadb-database-create" Dec 03 12:02:57 crc kubenswrapper[5052]: E1203 12:02:57.043904 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="c665a3eb-31e2-4ac4-99c8-02c8d262d9c9" containerName="mariadb-account-create-update" Dec 03 12:02:57 crc kubenswrapper[5052]: I1203 12:02:57.043913 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="c665a3eb-31e2-4ac4-99c8-02c8d262d9c9" containerName="mariadb-account-create-update" Dec 03 12:02:57 crc kubenswrapper[5052]: E1203 12:02:57.043931 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="66b581e0-3a48-46c9-b1a7-384b87d2b7f4" containerName="mariadb-account-create-update" Dec 03 12:02:57 crc kubenswrapper[5052]: I1203 12:02:57.043938 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="66b581e0-3a48-46c9-b1a7-384b87d2b7f4" containerName="mariadb-account-create-update" Dec 03 12:02:57 crc kubenswrapper[5052]: E1203 12:02:57.043964 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="246ced2f-39a7-496b-93cd-9f8ab898e2bb" containerName="mariadb-database-create" Dec 03 12:02:57 crc kubenswrapper[5052]: I1203 12:02:57.043971 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="246ced2f-39a7-496b-93cd-9f8ab898e2bb" containerName="mariadb-database-create" Dec 03 12:02:57 crc kubenswrapper[5052]: E1203 12:02:57.043985 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="79e77d8b-2481-4cbf-875f-cb626cf89463" containerName="mariadb-database-create" Dec 03 12:02:57 crc kubenswrapper[5052]: I1203 12:02:57.043994 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="79e77d8b-2481-4cbf-875f-cb626cf89463" containerName="mariadb-database-create" Dec 03 12:02:57 crc kubenswrapper[5052]: I1203 12:02:57.044200 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="66b581e0-3a48-46c9-b1a7-384b87d2b7f4" containerName="mariadb-account-create-update" Dec 03 12:02:57 crc kubenswrapper[5052]: I1203 12:02:57.044221 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="2a25df1a-4f92-4ed6-9c34-2d6b0212593b" containerName="mariadb-account-create-update" Dec 03 12:02:57 crc kubenswrapper[5052]: I1203 12:02:57.044231 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="c665a3eb-31e2-4ac4-99c8-02c8d262d9c9" containerName="mariadb-account-create-update" Dec 03 12:02:57 crc kubenswrapper[5052]: I1203 12:02:57.044240 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="1e7cf8cd-09e8-4bfe-b8d3-9e509d4eae43" containerName="mariadb-database-create" Dec 03 12:02:57 crc kubenswrapper[5052]: I1203 12:02:57.044254 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="246ced2f-39a7-496b-93cd-9f8ab898e2bb" containerName="mariadb-database-create" Dec 03 12:02:57 crc kubenswrapper[5052]: I1203 12:02:57.044265 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="907d9f47-ab0b-44c8-9aa6-3d135e1c312d" containerName="swift-ring-rebalance" Dec 03 12:02:57 crc kubenswrapper[5052]: I1203 12:02:57.044283 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="79e77d8b-2481-4cbf-875f-cb626cf89463" containerName="mariadb-database-create" Dec 03 12:02:57 crc kubenswrapper[5052]: I1203 12:02:57.048337 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-db-sync-2lq57" Dec 03 12:02:57 crc kubenswrapper[5052]: I1203 12:02:57.052961 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-glance-dockercfg-x6d26" Dec 03 12:02:57 crc kubenswrapper[5052]: I1203 12:02:57.053219 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-config-data" Dec 03 12:02:57 crc kubenswrapper[5052]: I1203 12:02:57.063024 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-db-sync-2lq57"] Dec 03 12:02:57 crc kubenswrapper[5052]: I1203 12:02:57.170605 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/df00a025-ee23-44aa-8970-6f1232146b98-db-sync-config-data\") pod \"glance-db-sync-2lq57\" (UID: \"df00a025-ee23-44aa-8970-6f1232146b98\") " pod="openstack-kuttl-tests/glance-db-sync-2lq57" Dec 03 12:02:57 crc kubenswrapper[5052]: I1203 12:02:57.170667 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/df00a025-ee23-44aa-8970-6f1232146b98-config-data\") pod \"glance-db-sync-2lq57\" (UID: \"df00a025-ee23-44aa-8970-6f1232146b98\") " pod="openstack-kuttl-tests/glance-db-sync-2lq57" Dec 03 12:02:57 crc kubenswrapper[5052]: I1203 12:02:57.170740 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bbwbf\" (UniqueName: \"kubernetes.io/projected/df00a025-ee23-44aa-8970-6f1232146b98-kube-api-access-bbwbf\") pod \"glance-db-sync-2lq57\" (UID: \"df00a025-ee23-44aa-8970-6f1232146b98\") " pod="openstack-kuttl-tests/glance-db-sync-2lq57" Dec 03 12:02:57 crc kubenswrapper[5052]: I1203 12:02:57.170782 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/df00a025-ee23-44aa-8970-6f1232146b98-combined-ca-bundle\") pod \"glance-db-sync-2lq57\" (UID: \"df00a025-ee23-44aa-8970-6f1232146b98\") " pod="openstack-kuttl-tests/glance-db-sync-2lq57" Dec 03 12:02:57 crc kubenswrapper[5052]: E1203 12:02:57.192706 5052 upgradeaware.go:427] Error proxying data from client to backend: readfrom tcp 38.129.56.252:34682->38.129.56.252:34849: write tcp 38.129.56.252:34682->38.129.56.252:34849: write: broken pipe Dec 03 12:02:57 crc kubenswrapper[5052]: I1203 12:02:57.273322 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-bbwbf\" (UniqueName: \"kubernetes.io/projected/df00a025-ee23-44aa-8970-6f1232146b98-kube-api-access-bbwbf\") pod \"glance-db-sync-2lq57\" (UID: \"df00a025-ee23-44aa-8970-6f1232146b98\") " pod="openstack-kuttl-tests/glance-db-sync-2lq57" Dec 03 12:02:57 crc kubenswrapper[5052]: I1203 12:02:57.273450 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/df00a025-ee23-44aa-8970-6f1232146b98-combined-ca-bundle\") pod \"glance-db-sync-2lq57\" (UID: \"df00a025-ee23-44aa-8970-6f1232146b98\") " pod="openstack-kuttl-tests/glance-db-sync-2lq57" Dec 03 12:02:57 crc kubenswrapper[5052]: I1203 12:02:57.273576 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/df00a025-ee23-44aa-8970-6f1232146b98-db-sync-config-data\") pod \"glance-db-sync-2lq57\" (UID: \"df00a025-ee23-44aa-8970-6f1232146b98\") " pod="openstack-kuttl-tests/glance-db-sync-2lq57" Dec 03 12:02:57 crc kubenswrapper[5052]: I1203 12:02:57.273644 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/df00a025-ee23-44aa-8970-6f1232146b98-config-data\") pod \"glance-db-sync-2lq57\" (UID: \"df00a025-ee23-44aa-8970-6f1232146b98\") " pod="openstack-kuttl-tests/glance-db-sync-2lq57" Dec 03 12:02:57 crc kubenswrapper[5052]: I1203 12:02:57.284650 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/df00a025-ee23-44aa-8970-6f1232146b98-config-data\") pod \"glance-db-sync-2lq57\" (UID: \"df00a025-ee23-44aa-8970-6f1232146b98\") " pod="openstack-kuttl-tests/glance-db-sync-2lq57" Dec 03 12:02:57 crc kubenswrapper[5052]: I1203 12:02:57.284661 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/df00a025-ee23-44aa-8970-6f1232146b98-combined-ca-bundle\") pod \"glance-db-sync-2lq57\" (UID: \"df00a025-ee23-44aa-8970-6f1232146b98\") " pod="openstack-kuttl-tests/glance-db-sync-2lq57" Dec 03 12:02:57 crc kubenswrapper[5052]: I1203 12:02:57.284672 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/df00a025-ee23-44aa-8970-6f1232146b98-db-sync-config-data\") pod \"glance-db-sync-2lq57\" (UID: \"df00a025-ee23-44aa-8970-6f1232146b98\") " pod="openstack-kuttl-tests/glance-db-sync-2lq57" Dec 03 12:02:57 crc kubenswrapper[5052]: I1203 12:02:57.321380 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-bbwbf\" (UniqueName: \"kubernetes.io/projected/df00a025-ee23-44aa-8970-6f1232146b98-kube-api-access-bbwbf\") pod \"glance-db-sync-2lq57\" (UID: \"df00a025-ee23-44aa-8970-6f1232146b98\") " pod="openstack-kuttl-tests/glance-db-sync-2lq57" Dec 03 12:02:57 crc kubenswrapper[5052]: I1203 12:02:57.432000 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-db-sync-2lq57" Dec 03 12:02:57 crc kubenswrapper[5052]: I1203 12:02:57.656110 5052 generic.go:334] "Generic (PLEG): container finished" podID="f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3" containerID="0790cbdcd312b0f701867eb108daefe7d9c2065afd80108a7c514db8349d75a3" exitCode=0 Dec 03 12:02:57 crc kubenswrapper[5052]: I1203 12:02:57.656153 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-cell1-server-1" event={"ID":"f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3","Type":"ContainerDied","Data":"0790cbdcd312b0f701867eb108daefe7d9c2065afd80108a7c514db8349d75a3"} Dec 03 12:02:57 crc kubenswrapper[5052]: I1203 12:02:57.668101 5052 generic.go:334] "Generic (PLEG): container finished" podID="185fdd2a-cd57-45e5-9986-bebd15e5761f" containerID="6b72ea83c326e823c684041451c505add9ed54cb24251b5b99e7e2a2086b0b19" exitCode=0 Dec 03 12:02:57 crc kubenswrapper[5052]: I1203 12:02:57.668212 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" event={"ID":"185fdd2a-cd57-45e5-9986-bebd15e5761f","Type":"ContainerDied","Data":"6b72ea83c326e823c684041451c505add9ed54cb24251b5b99e7e2a2086b0b19"} Dec 03 12:02:57 crc kubenswrapper[5052]: I1203 12:02:57.681674 5052 generic.go:334] "Generic (PLEG): container finished" podID="06bfd2db-abe0-4b4d-a110-94a6a1e02211" containerID="9889df1b1220aa2bd33fc61e1b06bb56a5427700764018be0e9dc52ae30425fd" exitCode=0 Dec 03 12:02:57 crc kubenswrapper[5052]: I1203 12:02:57.761247 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-cell1-server-2" event={"ID":"06bfd2db-abe0-4b4d-a110-94a6a1e02211","Type":"ContainerDied","Data":"9889df1b1220aa2bd33fc61e1b06bb56a5427700764018be0e9dc52ae30425fd"} Dec 03 12:02:57 crc kubenswrapper[5052]: I1203 12:02:57.761313 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-server-2" event={"ID":"7b4f6078-41a3-433d-acaf-f2d67391ab6e","Type":"ContainerStarted","Data":"aad64a682ded16a9f4098098d689931d1934e194e45eecdc993e62379504781b"} Dec 03 12:02:57 crc kubenswrapper[5052]: I1203 12:02:57.761637 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/rabbitmq-server-2" Dec 03 12:02:57 crc kubenswrapper[5052]: I1203 12:02:57.790847 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"95bf5f72-82f3-4e88-a576-01dae857541f","Type":"ContainerStarted","Data":"2732548b8572203d895c3e589eef6680d882e9fcbe498714de0778d4c35813ec"} Dec 03 12:02:57 crc kubenswrapper[5052]: I1203 12:02:57.790904 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"95bf5f72-82f3-4e88-a576-01dae857541f","Type":"ContainerStarted","Data":"6a7d071e07e4e8b8e192642aeae3ccfa4e50a1bfae563ab609f77db309009d6d"} Dec 03 12:02:57 crc kubenswrapper[5052]: I1203 12:02:57.796969 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/rabbitmq-server-2" podStartSLOduration=37.796950459 podStartE2EDuration="37.796950459s" podCreationTimestamp="2025-12-03 12:02:20 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:02:57.79140794 +0000 UTC m=+2568.379649714" watchObservedRunningTime="2025-12-03 12:02:57.796950459 +0000 UTC m=+2568.385192223" Dec 03 12:02:57 crc kubenswrapper[5052]: I1203 12:02:57.801104 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-server-0" event={"ID":"2397d896-0d11-4656-a336-eac8ec9fe7f9","Type":"ContainerStarted","Data":"16dd1c84a3d482ce5adc97f8fdc99d27080aa0bd2d9fba51e986dc52caa0daf6"} Dec 03 12:02:57 crc kubenswrapper[5052]: I1203 12:02:57.801833 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:02:57 crc kubenswrapper[5052]: I1203 12:02:57.818051 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-server-1" event={"ID":"7901ce85-4e6f-4e8b-86a6-f50a85803637","Type":"ContainerStarted","Data":"88ec697999833a01b97e7879439a3382717780668a2e4ef174d64567bf8d897a"} Dec 03 12:02:57 crc kubenswrapper[5052]: I1203 12:02:57.819125 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/rabbitmq-server-1" Dec 03 12:02:57 crc kubenswrapper[5052]: I1203 12:02:57.894179 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/rabbitmq-server-0" podStartSLOduration=37.894160968 podStartE2EDuration="37.894160968s" podCreationTimestamp="2025-12-03 12:02:20 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:02:57.87990034 +0000 UTC m=+2568.468142114" watchObservedRunningTime="2025-12-03 12:02:57.894160968 +0000 UTC m=+2568.482402722" Dec 03 12:02:57 crc kubenswrapper[5052]: I1203 12:02:57.926074 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/rabbitmq-server-1" podStartSLOduration=37.92605494 podStartE2EDuration="37.92605494s" podCreationTimestamp="2025-12-03 12:02:20 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:02:57.925225126 +0000 UTC m=+2568.513466890" watchObservedRunningTime="2025-12-03 12:02:57.92605494 +0000 UTC m=+2568.514296704" Dec 03 12:02:58 crc kubenswrapper[5052]: W1203 12:02:58.105210 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddf00a025_ee23_44aa_8970_6f1232146b98.slice/crio-dd6f9358d32d7d27d99b2bffacd2b349544849910647e13432c8299105871ea4 WatchSource:0}: Error finding container dd6f9358d32d7d27d99b2bffacd2b349544849910647e13432c8299105871ea4: Status 404 returned error can't find the container with id dd6f9358d32d7d27d99b2bffacd2b349544849910647e13432c8299105871ea4 Dec 03 12:02:58 crc kubenswrapper[5052]: I1203 12:02:58.115061 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-db-sync-2lq57"] Dec 03 12:02:58 crc kubenswrapper[5052]: I1203 12:02:58.543471 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/openstack-cell1-galera-1" Dec 03 12:02:58 crc kubenswrapper[5052]: I1203 12:02:58.735332 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/openstack-cell1-galera-1" Dec 03 12:02:58 crc kubenswrapper[5052]: I1203 12:02:58.837937 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-db-sync-2lq57" event={"ID":"df00a025-ee23-44aa-8970-6f1232146b98","Type":"ContainerStarted","Data":"dd6f9358d32d7d27d99b2bffacd2b349544849910647e13432c8299105871ea4"} Dec 03 12:02:58 crc kubenswrapper[5052]: I1203 12:02:58.844458 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-cell1-server-2" event={"ID":"06bfd2db-abe0-4b4d-a110-94a6a1e02211","Type":"ContainerStarted","Data":"61952ac0175a2f78cbd3919ba15907c05a42be11756520e4a28f961c6f3c039e"} Dec 03 12:02:58 crc kubenswrapper[5052]: I1203 12:02:58.844792 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/rabbitmq-cell1-server-2" Dec 03 12:02:58 crc kubenswrapper[5052]: I1203 12:02:58.857994 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"95bf5f72-82f3-4e88-a576-01dae857541f","Type":"ContainerStarted","Data":"2577c8659d7bc84c157b82cc64e9d927df3b8f1b4fe022b955ef7fe05c5dadbb"} Dec 03 12:02:58 crc kubenswrapper[5052]: I1203 12:02:58.858061 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"95bf5f72-82f3-4e88-a576-01dae857541f","Type":"ContainerStarted","Data":"a4050b9976060a4f925ffaa2cb889b98f3b107441d8ca49ad092c8be65814639"} Dec 03 12:02:58 crc kubenswrapper[5052]: I1203 12:02:58.858079 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"95bf5f72-82f3-4e88-a576-01dae857541f","Type":"ContainerStarted","Data":"ae9aaced3b606148dcf3e097c33141f00f123ef8589615b163bff1470454ae80"} Dec 03 12:02:58 crc kubenswrapper[5052]: I1203 12:02:58.858092 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"95bf5f72-82f3-4e88-a576-01dae857541f","Type":"ContainerStarted","Data":"9fa20306dd177e25e71d2c3ec088ca38dc2879ff9f7569b37d3b1ab695d20bed"} Dec 03 12:02:58 crc kubenswrapper[5052]: I1203 12:02:58.861916 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-cell1-server-1" event={"ID":"f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3","Type":"ContainerStarted","Data":"ae74ed54ba186a0e9a780dc48252d0e6409d74b51697be203e1eefe16b5793fe"} Dec 03 12:02:58 crc kubenswrapper[5052]: I1203 12:02:58.862250 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/rabbitmq-cell1-server-1" Dec 03 12:02:58 crc kubenswrapper[5052]: I1203 12:02:58.869838 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" event={"ID":"185fdd2a-cd57-45e5-9986-bebd15e5761f","Type":"ContainerStarted","Data":"c2f60fc531710d84b2e54ecd132f7312264860d279c9cbb5f54ba76be0eb79aa"} Dec 03 12:02:58 crc kubenswrapper[5052]: I1203 12:02:58.870363 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:02:58 crc kubenswrapper[5052]: I1203 12:02:58.876051 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/rabbitmq-cell1-server-2" podStartSLOduration=37.876024439 podStartE2EDuration="37.876024439s" podCreationTimestamp="2025-12-03 12:02:21 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:02:58.872534369 +0000 UTC m=+2569.460776153" watchObservedRunningTime="2025-12-03 12:02:58.876024439 +0000 UTC m=+2569.464266203" Dec 03 12:02:58 crc kubenswrapper[5052]: I1203 12:02:58.910140 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/rabbitmq-cell1-server-1" podStartSLOduration=37.910117984 podStartE2EDuration="37.910117984s" podCreationTimestamp="2025-12-03 12:02:21 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:02:58.906044047 +0000 UTC m=+2569.494285821" watchObservedRunningTime="2025-12-03 12:02:58.910117984 +0000 UTC m=+2569.498359748" Dec 03 12:02:59 crc kubenswrapper[5052]: I1203 12:02:59.036211 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/openstack-galera-1" Dec 03 12:02:59 crc kubenswrapper[5052]: I1203 12:02:59.076004 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" podStartSLOduration=38.075983906 podStartE2EDuration="38.075983906s" podCreationTimestamp="2025-12-03 12:02:21 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:02:58.951342522 +0000 UTC m=+2569.539584286" watchObservedRunningTime="2025-12-03 12:02:59.075983906 +0000 UTC m=+2569.664225670" Dec 03 12:02:59 crc kubenswrapper[5052]: I1203 12:02:59.202904 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/openstack-galera-1" Dec 03 12:02:59 crc kubenswrapper[5052]: I1203 12:02:59.891030 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"95bf5f72-82f3-4e88-a576-01dae857541f","Type":"ContainerStarted","Data":"b1f5592550eedafe273eda6ddd62bcf2957cdd483d534a5369391fc101829850"} Dec 03 12:02:59 crc kubenswrapper[5052]: I1203 12:02:59.891427 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"95bf5f72-82f3-4e88-a576-01dae857541f","Type":"ContainerStarted","Data":"ff53c665fe25113ca5552ca3edb72dc98d7ffacf8d96d01a610c8292d0ca0504"} Dec 03 12:02:59 crc kubenswrapper[5052]: I1203 12:02:59.891448 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"95bf5f72-82f3-4e88-a576-01dae857541f","Type":"ContainerStarted","Data":"da24ecbec4ad70b7a0db3646cffcca7dc56b37efffe6e261de63720e146b2b8e"} Dec 03 12:02:59 crc kubenswrapper[5052]: I1203 12:02:59.897412 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-db-sync-2lq57" event={"ID":"df00a025-ee23-44aa-8970-6f1232146b98","Type":"ContainerStarted","Data":"68e08571183dbfc32af1f09b3eda2efa53f0247f79e3a38eb83e5b3438697496"} Dec 03 12:02:59 crc kubenswrapper[5052]: I1203 12:02:59.945280 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/glance-db-sync-2lq57" podStartSLOduration=2.945243977 podStartE2EDuration="2.945243977s" podCreationTimestamp="2025-12-03 12:02:57 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:02:59.934897711 +0000 UTC m=+2570.523139495" watchObservedRunningTime="2025-12-03 12:02:59.945243977 +0000 UTC m=+2570.533485741" Dec 03 12:03:00 crc kubenswrapper[5052]: I1203 12:03:00.913555 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"95bf5f72-82f3-4e88-a576-01dae857541f","Type":"ContainerStarted","Data":"2d047aaa650efd207aa517746b6cde98bb8bae986bd03e7b545eda7fd36f1023"} Dec 03 12:03:00 crc kubenswrapper[5052]: I1203 12:03:00.913888 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"95bf5f72-82f3-4e88-a576-01dae857541f","Type":"ContainerStarted","Data":"c6d65f1e5859c5423b2bed9ad8130035fdbcdb06ce5b34713168ebc78916b853"} Dec 03 12:03:01 crc kubenswrapper[5052]: I1203 12:03:01.932478 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"95bf5f72-82f3-4e88-a576-01dae857541f","Type":"ContainerStarted","Data":"c0e56624ca96c43f72b2c3cc860c510ab86560649ca040b145fbe1cdc7988153"} Dec 03 12:03:01 crc kubenswrapper[5052]: I1203 12:03:01.933854 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"95bf5f72-82f3-4e88-a576-01dae857541f","Type":"ContainerStarted","Data":"4a4d3e25edd055d3f3503821db30046cfd815c08421228eb4d5e76a3f4e38a23"} Dec 03 12:03:01 crc kubenswrapper[5052]: I1203 12:03:01.991694 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/swift-storage-0" podStartSLOduration=24.991674054 podStartE2EDuration="24.991674054s" podCreationTimestamp="2025-12-03 12:02:37 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:03:01.982288875 +0000 UTC m=+2572.570530649" watchObservedRunningTime="2025-12-03 12:03:01.991674054 +0000 UTC m=+2572.579915818" Dec 03 12:03:02 crc kubenswrapper[5052]: I1203 12:03:02.261890 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-85f8bc6fb5-b86k2"] Dec 03 12:03:02 crc kubenswrapper[5052]: I1203 12:03:02.263688 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-85f8bc6fb5-b86k2" Dec 03 12:03:02 crc kubenswrapper[5052]: I1203 12:03:02.280690 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-85f8bc6fb5-b86k2"] Dec 03 12:03:02 crc kubenswrapper[5052]: I1203 12:03:02.299375 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"dns-swift-storage-0" Dec 03 12:03:02 crc kubenswrapper[5052]: I1203 12:03:02.357008 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/77077fe5-d1ff-49c4-b049-3a0f2cee23b0-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-85f8bc6fb5-b86k2\" (UID: \"77077fe5-d1ff-49c4-b049-3a0f2cee23b0\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-85f8bc6fb5-b86k2" Dec 03 12:03:02 crc kubenswrapper[5052]: I1203 12:03:02.357050 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dns-swift-storage-0\" (UniqueName: \"kubernetes.io/configmap/77077fe5-d1ff-49c4-b049-3a0f2cee23b0-dns-swift-storage-0\") pod \"dnsmasq-dnsmasq-85f8bc6fb5-b86k2\" (UID: \"77077fe5-d1ff-49c4-b049-3a0f2cee23b0\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-85f8bc6fb5-b86k2" Dec 03 12:03:02 crc kubenswrapper[5052]: I1203 12:03:02.357124 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jckkg\" (UniqueName: \"kubernetes.io/projected/77077fe5-d1ff-49c4-b049-3a0f2cee23b0-kube-api-access-jckkg\") pod \"dnsmasq-dnsmasq-85f8bc6fb5-b86k2\" (UID: \"77077fe5-d1ff-49c4-b049-3a0f2cee23b0\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-85f8bc6fb5-b86k2" Dec 03 12:03:02 crc kubenswrapper[5052]: I1203 12:03:02.357190 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/77077fe5-d1ff-49c4-b049-3a0f2cee23b0-config\") pod \"dnsmasq-dnsmasq-85f8bc6fb5-b86k2\" (UID: \"77077fe5-d1ff-49c4-b049-3a0f2cee23b0\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-85f8bc6fb5-b86k2" Dec 03 12:03:02 crc kubenswrapper[5052]: I1203 12:03:02.458620 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-jckkg\" (UniqueName: \"kubernetes.io/projected/77077fe5-d1ff-49c4-b049-3a0f2cee23b0-kube-api-access-jckkg\") pod \"dnsmasq-dnsmasq-85f8bc6fb5-b86k2\" (UID: \"77077fe5-d1ff-49c4-b049-3a0f2cee23b0\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-85f8bc6fb5-b86k2" Dec 03 12:03:02 crc kubenswrapper[5052]: I1203 12:03:02.458696 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/77077fe5-d1ff-49c4-b049-3a0f2cee23b0-config\") pod \"dnsmasq-dnsmasq-85f8bc6fb5-b86k2\" (UID: \"77077fe5-d1ff-49c4-b049-3a0f2cee23b0\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-85f8bc6fb5-b86k2" Dec 03 12:03:02 crc kubenswrapper[5052]: I1203 12:03:02.458753 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/77077fe5-d1ff-49c4-b049-3a0f2cee23b0-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-85f8bc6fb5-b86k2\" (UID: \"77077fe5-d1ff-49c4-b049-3a0f2cee23b0\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-85f8bc6fb5-b86k2" Dec 03 12:03:02 crc kubenswrapper[5052]: I1203 12:03:02.458773 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"dns-swift-storage-0\" (UniqueName: \"kubernetes.io/configmap/77077fe5-d1ff-49c4-b049-3a0f2cee23b0-dns-swift-storage-0\") pod \"dnsmasq-dnsmasq-85f8bc6fb5-b86k2\" (UID: \"77077fe5-d1ff-49c4-b049-3a0f2cee23b0\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-85f8bc6fb5-b86k2" Dec 03 12:03:02 crc kubenswrapper[5052]: I1203 12:03:02.459792 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"dns-swift-storage-0\" (UniqueName: \"kubernetes.io/configmap/77077fe5-d1ff-49c4-b049-3a0f2cee23b0-dns-swift-storage-0\") pod \"dnsmasq-dnsmasq-85f8bc6fb5-b86k2\" (UID: \"77077fe5-d1ff-49c4-b049-3a0f2cee23b0\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-85f8bc6fb5-b86k2" Dec 03 12:03:02 crc kubenswrapper[5052]: I1203 12:03:02.460649 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/77077fe5-d1ff-49c4-b049-3a0f2cee23b0-config\") pod \"dnsmasq-dnsmasq-85f8bc6fb5-b86k2\" (UID: \"77077fe5-d1ff-49c4-b049-3a0f2cee23b0\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-85f8bc6fb5-b86k2" Dec 03 12:03:02 crc kubenswrapper[5052]: I1203 12:03:02.461258 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/77077fe5-d1ff-49c4-b049-3a0f2cee23b0-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-85f8bc6fb5-b86k2\" (UID: \"77077fe5-d1ff-49c4-b049-3a0f2cee23b0\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-85f8bc6fb5-b86k2" Dec 03 12:03:02 crc kubenswrapper[5052]: I1203 12:03:02.487693 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-jckkg\" (UniqueName: \"kubernetes.io/projected/77077fe5-d1ff-49c4-b049-3a0f2cee23b0-kube-api-access-jckkg\") pod \"dnsmasq-dnsmasq-85f8bc6fb5-b86k2\" (UID: \"77077fe5-d1ff-49c4-b049-3a0f2cee23b0\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-85f8bc6fb5-b86k2" Dec 03 12:03:02 crc kubenswrapper[5052]: I1203 12:03:02.617716 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-85f8bc6fb5-b86k2" Dec 03 12:03:03 crc kubenswrapper[5052]: I1203 12:03:03.171048 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-85f8bc6fb5-b86k2"] Dec 03 12:03:03 crc kubenswrapper[5052]: I1203 12:03:03.953498 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-85f8bc6fb5-b86k2" event={"ID":"77077fe5-d1ff-49c4-b049-3a0f2cee23b0","Type":"ContainerStarted","Data":"eec0587de577f0935369e48dc238153fc5b64bb2a1e2911d9a07e70aa47e4cf1"} Dec 03 12:03:04 crc kubenswrapper[5052]: I1203 12:03:04.824041 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:03:04 crc kubenswrapper[5052]: I1203 12:03:04.965576 5052 generic.go:334] "Generic (PLEG): container finished" podID="77077fe5-d1ff-49c4-b049-3a0f2cee23b0" containerID="b3fb4c5fa05d10cd90154cc20449cb5477be079f2a3ceaf70a240ebedfabea19" exitCode=0 Dec 03 12:03:04 crc kubenswrapper[5052]: I1203 12:03:04.965624 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-85f8bc6fb5-b86k2" event={"ID":"77077fe5-d1ff-49c4-b049-3a0f2cee23b0","Type":"ContainerDied","Data":"b3fb4c5fa05d10cd90154cc20449cb5477be079f2a3ceaf70a240ebedfabea19"} Dec 03 12:03:05 crc kubenswrapper[5052]: I1203 12:03:05.248683 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:03:05 crc kubenswrapper[5052]: I1203 12:03:05.976962 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-85f8bc6fb5-b86k2" event={"ID":"77077fe5-d1ff-49c4-b049-3a0f2cee23b0","Type":"ContainerStarted","Data":"bc3186646511337244c91e6de61ef79f4b96f53697165b0fe223dd0c36cfeadf"} Dec 03 12:03:05 crc kubenswrapper[5052]: I1203 12:03:05.978700 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-85f8bc6fb5-b86k2" Dec 03 12:03:06 crc kubenswrapper[5052]: I1203 12:03:06.006492 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-85f8bc6fb5-b86k2" podStartSLOduration=4.006471104 podStartE2EDuration="4.006471104s" podCreationTimestamp="2025-12-03 12:03:02 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:03:06.002146641 +0000 UTC m=+2576.590388415" watchObservedRunningTime="2025-12-03 12:03:06.006471104 +0000 UTC m=+2576.594712868" Dec 03 12:03:06 crc kubenswrapper[5052]: I1203 12:03:06.881776 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:03:07 crc kubenswrapper[5052]: I1203 12:03:07.007614 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:03:12 crc kubenswrapper[5052]: I1203 12:03:12.089606 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/rabbitmq-server-0" podUID="2397d896-0d11-4656-a336-eac8ec9fe7f9" containerName="rabbitmq" probeResult="failure" output="dial tcp 10.217.0.100:5671: connect: connection refused" Dec 03 12:03:12 crc kubenswrapper[5052]: I1203 12:03:12.136200 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/rabbitmq-server-2" podUID="7b4f6078-41a3-433d-acaf-f2d67391ab6e" containerName="rabbitmq" probeResult="failure" output="dial tcp 10.217.0.101:5671: connect: connection refused" Dec 03 12:03:12 crc kubenswrapper[5052]: I1203 12:03:12.146374 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/rabbitmq-server-1" podUID="7901ce85-4e6f-4e8b-86a6-f50a85803637" containerName="rabbitmq" probeResult="failure" output="dial tcp 10.217.0.102:5671: connect: connection refused" Dec 03 12:03:12 crc kubenswrapper[5052]: I1203 12:03:12.619932 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-85f8bc6fb5-b86k2" Dec 03 12:03:12 crc kubenswrapper[5052]: I1203 12:03:12.684559 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-wfzg9"] Dec 03 12:03:12 crc kubenswrapper[5052]: I1203 12:03:12.684857 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-wfzg9" podUID="5105e4c1-412b-4c11-92db-2354ebcfc575" containerName="dnsmasq-dns" containerID="cri-o://58bc187a6aa5d3d79e23978554fc5a538422bd91d084cb518c12bb4a46bb1617" gracePeriod=10 Dec 03 12:03:13 crc kubenswrapper[5052]: I1203 12:03:13.041897 5052 generic.go:334] "Generic (PLEG): container finished" podID="5105e4c1-412b-4c11-92db-2354ebcfc575" containerID="58bc187a6aa5d3d79e23978554fc5a538422bd91d084cb518c12bb4a46bb1617" exitCode=0 Dec 03 12:03:13 crc kubenswrapper[5052]: I1203 12:03:13.042169 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-wfzg9" event={"ID":"5105e4c1-412b-4c11-92db-2354ebcfc575","Type":"ContainerDied","Data":"58bc187a6aa5d3d79e23978554fc5a538422bd91d084cb518c12bb4a46bb1617"} Dec 03 12:03:13 crc kubenswrapper[5052]: I1203 12:03:13.114442 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/rabbitmq-cell1-server-1" podUID="f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3" containerName="rabbitmq" probeResult="failure" output="dial tcp 10.217.0.103:5671: connect: connection refused" Dec 03 12:03:13 crc kubenswrapper[5052]: I1203 12:03:13.425547 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/rabbitmq-cell1-server-2" podUID="06bfd2db-abe0-4b4d-a110-94a6a1e02211" containerName="rabbitmq" probeResult="failure" output="dial tcp 10.217.0.105:5671: connect: connection refused" Dec 03 12:03:13 crc kubenswrapper[5052]: I1203 12:03:13.443056 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" podUID="185fdd2a-cd57-45e5-9986-bebd15e5761f" containerName="rabbitmq" probeResult="failure" output="dial tcp 10.217.0.104:5671: connect: connection refused" Dec 03 12:03:13 crc kubenswrapper[5052]: I1203 12:03:13.733189 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-wfzg9" Dec 03 12:03:13 crc kubenswrapper[5052]: I1203 12:03:13.786818 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/5105e4c1-412b-4c11-92db-2354ebcfc575-dnsmasq-svc\") pod \"5105e4c1-412b-4c11-92db-2354ebcfc575\" (UID: \"5105e4c1-412b-4c11-92db-2354ebcfc575\") " Dec 03 12:03:13 crc kubenswrapper[5052]: I1203 12:03:13.786947 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5105e4c1-412b-4c11-92db-2354ebcfc575-config\") pod \"5105e4c1-412b-4c11-92db-2354ebcfc575\" (UID: \"5105e4c1-412b-4c11-92db-2354ebcfc575\") " Dec 03 12:03:13 crc kubenswrapper[5052]: I1203 12:03:13.787151 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vwfzr\" (UniqueName: \"kubernetes.io/projected/5105e4c1-412b-4c11-92db-2354ebcfc575-kube-api-access-vwfzr\") pod \"5105e4c1-412b-4c11-92db-2354ebcfc575\" (UID: \"5105e4c1-412b-4c11-92db-2354ebcfc575\") " Dec 03 12:03:13 crc kubenswrapper[5052]: I1203 12:03:13.813756 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5105e4c1-412b-4c11-92db-2354ebcfc575-kube-api-access-vwfzr" (OuterVolumeSpecName: "kube-api-access-vwfzr") pod "5105e4c1-412b-4c11-92db-2354ebcfc575" (UID: "5105e4c1-412b-4c11-92db-2354ebcfc575"). InnerVolumeSpecName "kube-api-access-vwfzr". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:03:13 crc kubenswrapper[5052]: I1203 12:03:13.841429 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5105e4c1-412b-4c11-92db-2354ebcfc575-config" (OuterVolumeSpecName: "config") pod "5105e4c1-412b-4c11-92db-2354ebcfc575" (UID: "5105e4c1-412b-4c11-92db-2354ebcfc575"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:03:13 crc kubenswrapper[5052]: I1203 12:03:13.843582 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5105e4c1-412b-4c11-92db-2354ebcfc575-dnsmasq-svc" (OuterVolumeSpecName: "dnsmasq-svc") pod "5105e4c1-412b-4c11-92db-2354ebcfc575" (UID: "5105e4c1-412b-4c11-92db-2354ebcfc575"). InnerVolumeSpecName "dnsmasq-svc". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:03:13 crc kubenswrapper[5052]: I1203 12:03:13.889613 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-vwfzr\" (UniqueName: \"kubernetes.io/projected/5105e4c1-412b-4c11-92db-2354ebcfc575-kube-api-access-vwfzr\") on node \"crc\" DevicePath \"\"" Dec 03 12:03:13 crc kubenswrapper[5052]: I1203 12:03:13.889645 5052 reconciler_common.go:293] "Volume detached for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/5105e4c1-412b-4c11-92db-2354ebcfc575-dnsmasq-svc\") on node \"crc\" DevicePath \"\"" Dec 03 12:03:13 crc kubenswrapper[5052]: I1203 12:03:13.889656 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5105e4c1-412b-4c11-92db-2354ebcfc575-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:03:14 crc kubenswrapper[5052]: I1203 12:03:14.054870 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-wfzg9" event={"ID":"5105e4c1-412b-4c11-92db-2354ebcfc575","Type":"ContainerDied","Data":"6206e2066b39d8bea7f976a4de9746fa94b987163d6375b6584d3022506471ca"} Dec 03 12:03:14 crc kubenswrapper[5052]: I1203 12:03:14.054944 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-wfzg9" Dec 03 12:03:14 crc kubenswrapper[5052]: I1203 12:03:14.055536 5052 scope.go:117] "RemoveContainer" containerID="58bc187a6aa5d3d79e23978554fc5a538422bd91d084cb518c12bb4a46bb1617" Dec 03 12:03:14 crc kubenswrapper[5052]: I1203 12:03:14.088990 5052 scope.go:117] "RemoveContainer" containerID="99454548152a276442d6ff5c809528ace0ce757a6eb67b07cc45504cedad329a" Dec 03 12:03:14 crc kubenswrapper[5052]: I1203 12:03:14.090057 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-wfzg9"] Dec 03 12:03:14 crc kubenswrapper[5052]: I1203 12:03:14.100029 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-wfzg9"] Dec 03 12:03:15 crc kubenswrapper[5052]: I1203 12:03:15.711585 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5105e4c1-412b-4c11-92db-2354ebcfc575" path="/var/lib/kubelet/pods/5105e4c1-412b-4c11-92db-2354ebcfc575/volumes" Dec 03 12:03:22 crc kubenswrapper[5052]: I1203 12:03:22.088428 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/rabbitmq-server-0" podUID="2397d896-0d11-4656-a336-eac8ec9fe7f9" containerName="rabbitmq" probeResult="failure" output="dial tcp 10.217.0.100:5671: connect: connection refused" Dec 03 12:03:22 crc kubenswrapper[5052]: I1203 12:03:22.135032 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/rabbitmq-server-2" podUID="7b4f6078-41a3-433d-acaf-f2d67391ab6e" containerName="rabbitmq" probeResult="failure" output="dial tcp 10.217.0.101:5671: connect: connection refused" Dec 03 12:03:22 crc kubenswrapper[5052]: I1203 12:03:22.146457 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/rabbitmq-server-1" Dec 03 12:03:23 crc kubenswrapper[5052]: I1203 12:03:23.113542 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/rabbitmq-cell1-server-1" podUID="f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3" containerName="rabbitmq" probeResult="failure" output="dial tcp 10.217.0.103:5671: connect: connection refused" Dec 03 12:03:23 crc kubenswrapper[5052]: I1203 12:03:23.426846 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/rabbitmq-cell1-server-2" Dec 03 12:03:23 crc kubenswrapper[5052]: I1203 12:03:23.437348 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" podUID="185fdd2a-cd57-45e5-9986-bebd15e5761f" containerName="rabbitmq" probeResult="failure" output="dial tcp 10.217.0.104:5671: connect: connection refused" Dec 03 12:03:32 crc kubenswrapper[5052]: I1203 12:03:32.088391 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/rabbitmq-server-0" podUID="2397d896-0d11-4656-a336-eac8ec9fe7f9" containerName="rabbitmq" probeResult="failure" output="dial tcp 10.217.0.100:5671: connect: connection refused" Dec 03 12:03:32 crc kubenswrapper[5052]: I1203 12:03:32.134777 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/rabbitmq-server-2" podUID="7b4f6078-41a3-433d-acaf-f2d67391ab6e" containerName="rabbitmq" probeResult="failure" output="dial tcp 10.217.0.101:5671: connect: connection refused" Dec 03 12:03:33 crc kubenswrapper[5052]: I1203 12:03:33.113483 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/rabbitmq-cell1-server-1" podUID="f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3" containerName="rabbitmq" probeResult="failure" output="dial tcp 10.217.0.103:5671: connect: connection refused" Dec 03 12:03:33 crc kubenswrapper[5052]: I1203 12:03:33.435842 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" podUID="185fdd2a-cd57-45e5-9986-bebd15e5761f" containerName="rabbitmq" probeResult="failure" output="dial tcp 10.217.0.104:5671: connect: connection refused" Dec 03 12:03:42 crc kubenswrapper[5052]: I1203 12:03:42.089807 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:03:42 crc kubenswrapper[5052]: I1203 12:03:42.134573 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/rabbitmq-server-2" podUID="7b4f6078-41a3-433d-acaf-f2d67391ab6e" containerName="rabbitmq" probeResult="failure" output="dial tcp 10.217.0.101:5671: connect: connection refused" Dec 03 12:03:43 crc kubenswrapper[5052]: I1203 12:03:43.114215 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/rabbitmq-cell1-server-1" Dec 03 12:03:43 crc kubenswrapper[5052]: I1203 12:03:43.436589 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:03:52 crc kubenswrapper[5052]: I1203 12:03:52.135760 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/rabbitmq-server-2" podUID="7b4f6078-41a3-433d-acaf-f2d67391ab6e" containerName="rabbitmq" probeResult="failure" output="dial tcp 10.217.0.101:5671: connect: connection refused" Dec 03 12:03:52 crc kubenswrapper[5052]: I1203 12:03:52.724178 5052 scope.go:117] "RemoveContainer" containerID="b98e34ad22699ba1f43e04c09cce28fdffc0b0ce4215afae3d6add511b2c1de7" Dec 03 12:03:53 crc kubenswrapper[5052]: I1203 12:03:53.171063 5052 scope.go:117] "RemoveContainer" containerID="efba1971cde8e2d60c21027b0213c476cc57031f7a94af481671b4c6bb25e4fe" Dec 03 12:03:53 crc kubenswrapper[5052]: I1203 12:03:53.213904 5052 scope.go:117] "RemoveContainer" containerID="1b244ec0990cc16d9706217cfd2f7c20024ec92450402cd800184fce52721118" Dec 03 12:03:53 crc kubenswrapper[5052]: I1203 12:03:53.254602 5052 scope.go:117] "RemoveContainer" containerID="cfcf65482c408d6744121d3337ff364f65dfd4df7db4e0d1ca7a10b2ec5a6849" Dec 03 12:03:53 crc kubenswrapper[5052]: I1203 12:03:53.308577 5052 scope.go:117] "RemoveContainer" containerID="2e50a1ee31cb9fb17d19ff08d0f8c676ef19837034c0538001c15fc2e39b0bac" Dec 03 12:03:53 crc kubenswrapper[5052]: I1203 12:03:53.338371 5052 scope.go:117] "RemoveContainer" containerID="608e4a1a02f619ff83a69532ecf951dbd52836921fd0e850e8efab96f9af411b" Dec 03 12:03:53 crc kubenswrapper[5052]: I1203 12:03:53.389555 5052 scope.go:117] "RemoveContainer" containerID="3e630cf9c031c51d9e57dbf5f4f68a125ad4930a9ae93db53fda22a443e086f2" Dec 03 12:03:53 crc kubenswrapper[5052]: I1203 12:03:53.550449 5052 scope.go:117] "RemoveContainer" containerID="d284b92928b94c054ca5728e80c48dca72344c183bd279f1d273ea56c61bafed" Dec 03 12:03:53 crc kubenswrapper[5052]: I1203 12:03:53.574570 5052 scope.go:117] "RemoveContainer" containerID="640c47d1ca72473f70926ab28ebc111db3c659c0cb4011d19f805e98ea5f46b1" Dec 03 12:03:53 crc kubenswrapper[5052]: I1203 12:03:53.615544 5052 scope.go:117] "RemoveContainer" containerID="56106c72b8ee52c2322893ce9cab31cbca3baecdb6d70605f34d8b51f2c8f37b" Dec 03 12:03:53 crc kubenswrapper[5052]: I1203 12:03:53.640236 5052 scope.go:117] "RemoveContainer" containerID="fe84882a80ccaeef2f8d940fc38370f099f9c488aea123cf93e4406cf6c98a68" Dec 03 12:03:53 crc kubenswrapper[5052]: I1203 12:03:53.699298 5052 scope.go:117] "RemoveContainer" containerID="9b3146eaddf7ac59c648a8a038db1a5905188e2a8b2f4dc2de4f7459b0197c3c" Dec 03 12:03:53 crc kubenswrapper[5052]: I1203 12:03:53.740815 5052 scope.go:117] "RemoveContainer" containerID="37a42c9d3db76f0dfccd0a96b80c798307fcfb938d7f490f56ab9c77c423a50f" Dec 03 12:03:53 crc kubenswrapper[5052]: I1203 12:03:53.815184 5052 scope.go:117] "RemoveContainer" containerID="3878c20bec17655742d669ac45a8e056745cb2364c66b143955a59eac4734e5f" Dec 03 12:04:02 crc kubenswrapper[5052]: I1203 12:04:02.134598 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/rabbitmq-server-2" podUID="7b4f6078-41a3-433d-acaf-f2d67391ab6e" containerName="rabbitmq" probeResult="failure" output="dial tcp 10.217.0.101:5671: connect: connection refused" Dec 03 12:04:10 crc kubenswrapper[5052]: I1203 12:04:10.675716 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/rabbitmq-server-2" Dec 03 12:04:10 crc kubenswrapper[5052]: I1203 12:04:10.997336 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/cinder-db-create-gsdcc"] Dec 03 12:04:10 crc kubenswrapper[5052]: E1203 12:04:10.997795 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="5105e4c1-412b-4c11-92db-2354ebcfc575" containerName="dnsmasq-dns" Dec 03 12:04:10 crc kubenswrapper[5052]: I1203 12:04:10.997816 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="5105e4c1-412b-4c11-92db-2354ebcfc575" containerName="dnsmasq-dns" Dec 03 12:04:10 crc kubenswrapper[5052]: E1203 12:04:10.997851 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="5105e4c1-412b-4c11-92db-2354ebcfc575" containerName="init" Dec 03 12:04:10 crc kubenswrapper[5052]: I1203 12:04:10.997857 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="5105e4c1-412b-4c11-92db-2354ebcfc575" containerName="init" Dec 03 12:04:10 crc kubenswrapper[5052]: I1203 12:04:10.998042 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="5105e4c1-412b-4c11-92db-2354ebcfc575" containerName="dnsmasq-dns" Dec 03 12:04:10 crc kubenswrapper[5052]: I1203 12:04:10.998841 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-db-create-gsdcc" Dec 03 12:04:11 crc kubenswrapper[5052]: I1203 12:04:11.021322 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-db-create-gsdcc"] Dec 03 12:04:11 crc kubenswrapper[5052]: I1203 12:04:11.097327 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/barbican-db-create-z645k"] Dec 03 12:04:11 crc kubenswrapper[5052]: I1203 12:04:11.098928 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-db-create-z645k" Dec 03 12:04:11 crc kubenswrapper[5052]: I1203 12:04:11.100650 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jj7kv\" (UniqueName: \"kubernetes.io/projected/2b7332a9-078d-4457-b136-899c4cdfaaae-kube-api-access-jj7kv\") pod \"cinder-db-create-gsdcc\" (UID: \"2b7332a9-078d-4457-b136-899c4cdfaaae\") " pod="openstack-kuttl-tests/cinder-db-create-gsdcc" Dec 03 12:04:11 crc kubenswrapper[5052]: I1203 12:04:11.100760 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/2b7332a9-078d-4457-b136-899c4cdfaaae-operator-scripts\") pod \"cinder-db-create-gsdcc\" (UID: \"2b7332a9-078d-4457-b136-899c4cdfaaae\") " pod="openstack-kuttl-tests/cinder-db-create-gsdcc" Dec 03 12:04:11 crc kubenswrapper[5052]: I1203 12:04:11.120848 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-db-create-z645k"] Dec 03 12:04:11 crc kubenswrapper[5052]: I1203 12:04:11.198176 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/neutron-db-create-z9ftc"] Dec 03 12:04:11 crc kubenswrapper[5052]: I1203 12:04:11.199564 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-db-create-z9ftc" Dec 03 12:04:11 crc kubenswrapper[5052]: I1203 12:04:11.202486 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/67b28447-93fa-45c3-8c94-9dba5eba8f94-operator-scripts\") pod \"barbican-db-create-z645k\" (UID: \"67b28447-93fa-45c3-8c94-9dba5eba8f94\") " pod="openstack-kuttl-tests/barbican-db-create-z645k" Dec 03 12:04:11 crc kubenswrapper[5052]: I1203 12:04:11.202569 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/2b7332a9-078d-4457-b136-899c4cdfaaae-operator-scripts\") pod \"cinder-db-create-gsdcc\" (UID: \"2b7332a9-078d-4457-b136-899c4cdfaaae\") " pod="openstack-kuttl-tests/cinder-db-create-gsdcc" Dec 03 12:04:11 crc kubenswrapper[5052]: I1203 12:04:11.202654 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8j2pm\" (UniqueName: \"kubernetes.io/projected/67b28447-93fa-45c3-8c94-9dba5eba8f94-kube-api-access-8j2pm\") pod \"barbican-db-create-z645k\" (UID: \"67b28447-93fa-45c3-8c94-9dba5eba8f94\") " pod="openstack-kuttl-tests/barbican-db-create-z645k" Dec 03 12:04:11 crc kubenswrapper[5052]: I1203 12:04:11.202676 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-jj7kv\" (UniqueName: \"kubernetes.io/projected/2b7332a9-078d-4457-b136-899c4cdfaaae-kube-api-access-jj7kv\") pod \"cinder-db-create-gsdcc\" (UID: \"2b7332a9-078d-4457-b136-899c4cdfaaae\") " pod="openstack-kuttl-tests/cinder-db-create-gsdcc" Dec 03 12:04:11 crc kubenswrapper[5052]: I1203 12:04:11.203374 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/2b7332a9-078d-4457-b136-899c4cdfaaae-operator-scripts\") pod \"cinder-db-create-gsdcc\" (UID: \"2b7332a9-078d-4457-b136-899c4cdfaaae\") " pod="openstack-kuttl-tests/cinder-db-create-gsdcc" Dec 03 12:04:11 crc kubenswrapper[5052]: I1203 12:04:11.210601 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/cinder-e6fc-account-create-update-j5n92"] Dec 03 12:04:11 crc kubenswrapper[5052]: I1203 12:04:11.211921 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-e6fc-account-create-update-j5n92" Dec 03 12:04:11 crc kubenswrapper[5052]: I1203 12:04:11.213702 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cinder-db-secret" Dec 03 12:04:11 crc kubenswrapper[5052]: I1203 12:04:11.222169 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-db-create-z9ftc"] Dec 03 12:04:11 crc kubenswrapper[5052]: I1203 12:04:11.231270 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-e6fc-account-create-update-j5n92"] Dec 03 12:04:11 crc kubenswrapper[5052]: I1203 12:04:11.263309 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-jj7kv\" (UniqueName: \"kubernetes.io/projected/2b7332a9-078d-4457-b136-899c4cdfaaae-kube-api-access-jj7kv\") pod \"cinder-db-create-gsdcc\" (UID: \"2b7332a9-078d-4457-b136-899c4cdfaaae\") " pod="openstack-kuttl-tests/cinder-db-create-gsdcc" Dec 03 12:04:11 crc kubenswrapper[5052]: I1203 12:04:11.286968 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/keystone-db-sync-8xzwc"] Dec 03 12:04:11 crc kubenswrapper[5052]: I1203 12:04:11.288731 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-db-sync-8xzwc" Dec 03 12:04:11 crc kubenswrapper[5052]: I1203 12:04:11.303146 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone-config-data" Dec 03 12:04:11 crc kubenswrapper[5052]: I1203 12:04:11.303372 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone-scripts" Dec 03 12:04:11 crc kubenswrapper[5052]: I1203 12:04:11.303636 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone" Dec 03 12:04:11 crc kubenswrapper[5052]: I1203 12:04:11.303645 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone-keystone-dockercfg-m92zx" Dec 03 12:04:11 crc kubenswrapper[5052]: I1203 12:04:11.304479 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-8j2pm\" (UniqueName: \"kubernetes.io/projected/67b28447-93fa-45c3-8c94-9dba5eba8f94-kube-api-access-8j2pm\") pod \"barbican-db-create-z645k\" (UID: \"67b28447-93fa-45c3-8c94-9dba5eba8f94\") " pod="openstack-kuttl-tests/barbican-db-create-z645k" Dec 03 12:04:11 crc kubenswrapper[5052]: I1203 12:04:11.304557 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/46eb04b8-4d02-4ffe-a0f6-bb4b236a4956-config-data\") pod \"keystone-db-sync-8xzwc\" (UID: \"46eb04b8-4d02-4ffe-a0f6-bb4b236a4956\") " pod="openstack-kuttl-tests/keystone-db-sync-8xzwc" Dec 03 12:04:11 crc kubenswrapper[5052]: I1203 12:04:11.304631 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/8ba21481-e9f8-41f3-80e6-c96f860b7771-operator-scripts\") pod \"neutron-db-create-z9ftc\" (UID: \"8ba21481-e9f8-41f3-80e6-c96f860b7771\") " pod="openstack-kuttl-tests/neutron-db-create-z9ftc" Dec 03 12:04:11 crc kubenswrapper[5052]: I1203 12:04:11.304669 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pz4cb\" (UniqueName: \"kubernetes.io/projected/46eb04b8-4d02-4ffe-a0f6-bb4b236a4956-kube-api-access-pz4cb\") pod \"keystone-db-sync-8xzwc\" (UID: \"46eb04b8-4d02-4ffe-a0f6-bb4b236a4956\") " pod="openstack-kuttl-tests/keystone-db-sync-8xzwc" Dec 03 12:04:11 crc kubenswrapper[5052]: I1203 12:04:11.304691 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vsvcb\" (UniqueName: \"kubernetes.io/projected/8ba21481-e9f8-41f3-80e6-c96f860b7771-kube-api-access-vsvcb\") pod \"neutron-db-create-z9ftc\" (UID: \"8ba21481-e9f8-41f3-80e6-c96f860b7771\") " pod="openstack-kuttl-tests/neutron-db-create-z9ftc" Dec 03 12:04:11 crc kubenswrapper[5052]: I1203 12:04:11.304719 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/67b28447-93fa-45c3-8c94-9dba5eba8f94-operator-scripts\") pod \"barbican-db-create-z645k\" (UID: \"67b28447-93fa-45c3-8c94-9dba5eba8f94\") " pod="openstack-kuttl-tests/barbican-db-create-z645k" Dec 03 12:04:11 crc kubenswrapper[5052]: I1203 12:04:11.304749 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/59c3e3b8-e2a6-4384-87f0-d1d7cbb4d738-operator-scripts\") pod \"cinder-e6fc-account-create-update-j5n92\" (UID: \"59c3e3b8-e2a6-4384-87f0-d1d7cbb4d738\") " pod="openstack-kuttl-tests/cinder-e6fc-account-create-update-j5n92" Dec 03 12:04:11 crc kubenswrapper[5052]: I1203 12:04:11.304783 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/46eb04b8-4d02-4ffe-a0f6-bb4b236a4956-combined-ca-bundle\") pod \"keystone-db-sync-8xzwc\" (UID: \"46eb04b8-4d02-4ffe-a0f6-bb4b236a4956\") " pod="openstack-kuttl-tests/keystone-db-sync-8xzwc" Dec 03 12:04:11 crc kubenswrapper[5052]: I1203 12:04:11.304806 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5dmxz\" (UniqueName: \"kubernetes.io/projected/59c3e3b8-e2a6-4384-87f0-d1d7cbb4d738-kube-api-access-5dmxz\") pod \"cinder-e6fc-account-create-update-j5n92\" (UID: \"59c3e3b8-e2a6-4384-87f0-d1d7cbb4d738\") " pod="openstack-kuttl-tests/cinder-e6fc-account-create-update-j5n92" Dec 03 12:04:11 crc kubenswrapper[5052]: I1203 12:04:11.305824 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/67b28447-93fa-45c3-8c94-9dba5eba8f94-operator-scripts\") pod \"barbican-db-create-z645k\" (UID: \"67b28447-93fa-45c3-8c94-9dba5eba8f94\") " pod="openstack-kuttl-tests/barbican-db-create-z645k" Dec 03 12:04:11 crc kubenswrapper[5052]: I1203 12:04:11.315470 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-db-create-gsdcc" Dec 03 12:04:11 crc kubenswrapper[5052]: I1203 12:04:11.320559 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/barbican-5c55-account-create-update-vz6m5"] Dec 03 12:04:11 crc kubenswrapper[5052]: I1203 12:04:11.321758 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-5c55-account-create-update-vz6m5" Dec 03 12:04:11 crc kubenswrapper[5052]: I1203 12:04:11.326309 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"barbican-db-secret" Dec 03 12:04:11 crc kubenswrapper[5052]: I1203 12:04:11.344201 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-db-sync-8xzwc"] Dec 03 12:04:11 crc kubenswrapper[5052]: I1203 12:04:11.344669 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-8j2pm\" (UniqueName: \"kubernetes.io/projected/67b28447-93fa-45c3-8c94-9dba5eba8f94-kube-api-access-8j2pm\") pod \"barbican-db-create-z645k\" (UID: \"67b28447-93fa-45c3-8c94-9dba5eba8f94\") " pod="openstack-kuttl-tests/barbican-db-create-z645k" Dec 03 12:04:11 crc kubenswrapper[5052]: I1203 12:04:11.378712 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-5c55-account-create-update-vz6m5"] Dec 03 12:04:11 crc kubenswrapper[5052]: I1203 12:04:11.408414 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/8ba21481-e9f8-41f3-80e6-c96f860b7771-operator-scripts\") pod \"neutron-db-create-z9ftc\" (UID: \"8ba21481-e9f8-41f3-80e6-c96f860b7771\") " pod="openstack-kuttl-tests/neutron-db-create-z9ftc" Dec 03 12:04:11 crc kubenswrapper[5052]: I1203 12:04:11.408490 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-pz4cb\" (UniqueName: \"kubernetes.io/projected/46eb04b8-4d02-4ffe-a0f6-bb4b236a4956-kube-api-access-pz4cb\") pod \"keystone-db-sync-8xzwc\" (UID: \"46eb04b8-4d02-4ffe-a0f6-bb4b236a4956\") " pod="openstack-kuttl-tests/keystone-db-sync-8xzwc" Dec 03 12:04:11 crc kubenswrapper[5052]: I1203 12:04:11.408535 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-vsvcb\" (UniqueName: \"kubernetes.io/projected/8ba21481-e9f8-41f3-80e6-c96f860b7771-kube-api-access-vsvcb\") pod \"neutron-db-create-z9ftc\" (UID: \"8ba21481-e9f8-41f3-80e6-c96f860b7771\") " pod="openstack-kuttl-tests/neutron-db-create-z9ftc" Dec 03 12:04:11 crc kubenswrapper[5052]: I1203 12:04:11.408572 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/59c3e3b8-e2a6-4384-87f0-d1d7cbb4d738-operator-scripts\") pod \"cinder-e6fc-account-create-update-j5n92\" (UID: \"59c3e3b8-e2a6-4384-87f0-d1d7cbb4d738\") " pod="openstack-kuttl-tests/cinder-e6fc-account-create-update-j5n92" Dec 03 12:04:11 crc kubenswrapper[5052]: I1203 12:04:11.408607 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/46eb04b8-4d02-4ffe-a0f6-bb4b236a4956-combined-ca-bundle\") pod \"keystone-db-sync-8xzwc\" (UID: \"46eb04b8-4d02-4ffe-a0f6-bb4b236a4956\") " pod="openstack-kuttl-tests/keystone-db-sync-8xzwc" Dec 03 12:04:11 crc kubenswrapper[5052]: I1203 12:04:11.408628 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-5dmxz\" (UniqueName: \"kubernetes.io/projected/59c3e3b8-e2a6-4384-87f0-d1d7cbb4d738-kube-api-access-5dmxz\") pod \"cinder-e6fc-account-create-update-j5n92\" (UID: \"59c3e3b8-e2a6-4384-87f0-d1d7cbb4d738\") " pod="openstack-kuttl-tests/cinder-e6fc-account-create-update-j5n92" Dec 03 12:04:11 crc kubenswrapper[5052]: I1203 12:04:11.408679 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/90d21e8c-a52c-4109-811b-7d113748cea7-operator-scripts\") pod \"barbican-5c55-account-create-update-vz6m5\" (UID: \"90d21e8c-a52c-4109-811b-7d113748cea7\") " pod="openstack-kuttl-tests/barbican-5c55-account-create-update-vz6m5" Dec 03 12:04:11 crc kubenswrapper[5052]: I1203 12:04:11.408738 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/46eb04b8-4d02-4ffe-a0f6-bb4b236a4956-config-data\") pod \"keystone-db-sync-8xzwc\" (UID: \"46eb04b8-4d02-4ffe-a0f6-bb4b236a4956\") " pod="openstack-kuttl-tests/keystone-db-sync-8xzwc" Dec 03 12:04:11 crc kubenswrapper[5052]: I1203 12:04:11.408766 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ml6wb\" (UniqueName: \"kubernetes.io/projected/90d21e8c-a52c-4109-811b-7d113748cea7-kube-api-access-ml6wb\") pod \"barbican-5c55-account-create-update-vz6m5\" (UID: \"90d21e8c-a52c-4109-811b-7d113748cea7\") " pod="openstack-kuttl-tests/barbican-5c55-account-create-update-vz6m5" Dec 03 12:04:11 crc kubenswrapper[5052]: I1203 12:04:11.409661 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/8ba21481-e9f8-41f3-80e6-c96f860b7771-operator-scripts\") pod \"neutron-db-create-z9ftc\" (UID: \"8ba21481-e9f8-41f3-80e6-c96f860b7771\") " pod="openstack-kuttl-tests/neutron-db-create-z9ftc" Dec 03 12:04:11 crc kubenswrapper[5052]: I1203 12:04:11.411393 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/59c3e3b8-e2a6-4384-87f0-d1d7cbb4d738-operator-scripts\") pod \"cinder-e6fc-account-create-update-j5n92\" (UID: \"59c3e3b8-e2a6-4384-87f0-d1d7cbb4d738\") " pod="openstack-kuttl-tests/cinder-e6fc-account-create-update-j5n92" Dec 03 12:04:11 crc kubenswrapper[5052]: I1203 12:04:11.416353 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/46eb04b8-4d02-4ffe-a0f6-bb4b236a4956-config-data\") pod \"keystone-db-sync-8xzwc\" (UID: \"46eb04b8-4d02-4ffe-a0f6-bb4b236a4956\") " pod="openstack-kuttl-tests/keystone-db-sync-8xzwc" Dec 03 12:04:11 crc kubenswrapper[5052]: I1203 12:04:11.420768 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/46eb04b8-4d02-4ffe-a0f6-bb4b236a4956-combined-ca-bundle\") pod \"keystone-db-sync-8xzwc\" (UID: \"46eb04b8-4d02-4ffe-a0f6-bb4b236a4956\") " pod="openstack-kuttl-tests/keystone-db-sync-8xzwc" Dec 03 12:04:11 crc kubenswrapper[5052]: I1203 12:04:11.422990 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-db-create-z645k" Dec 03 12:04:11 crc kubenswrapper[5052]: I1203 12:04:11.424789 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/neutron-d5c4-account-create-update-jngcd"] Dec 03 12:04:11 crc kubenswrapper[5052]: I1203 12:04:11.429039 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-d5c4-account-create-update-jngcd" Dec 03 12:04:11 crc kubenswrapper[5052]: I1203 12:04:11.438642 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"neutron-db-secret" Dec 03 12:04:11 crc kubenswrapper[5052]: I1203 12:04:11.445358 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-vsvcb\" (UniqueName: \"kubernetes.io/projected/8ba21481-e9f8-41f3-80e6-c96f860b7771-kube-api-access-vsvcb\") pod \"neutron-db-create-z9ftc\" (UID: \"8ba21481-e9f8-41f3-80e6-c96f860b7771\") " pod="openstack-kuttl-tests/neutron-db-create-z9ftc" Dec 03 12:04:11 crc kubenswrapper[5052]: I1203 12:04:11.454959 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-pz4cb\" (UniqueName: \"kubernetes.io/projected/46eb04b8-4d02-4ffe-a0f6-bb4b236a4956-kube-api-access-pz4cb\") pod \"keystone-db-sync-8xzwc\" (UID: \"46eb04b8-4d02-4ffe-a0f6-bb4b236a4956\") " pod="openstack-kuttl-tests/keystone-db-sync-8xzwc" Dec 03 12:04:11 crc kubenswrapper[5052]: I1203 12:04:11.455067 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-d5c4-account-create-update-jngcd"] Dec 03 12:04:11 crc kubenswrapper[5052]: I1203 12:04:11.458166 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-5dmxz\" (UniqueName: \"kubernetes.io/projected/59c3e3b8-e2a6-4384-87f0-d1d7cbb4d738-kube-api-access-5dmxz\") pod \"cinder-e6fc-account-create-update-j5n92\" (UID: \"59c3e3b8-e2a6-4384-87f0-d1d7cbb4d738\") " pod="openstack-kuttl-tests/cinder-e6fc-account-create-update-j5n92" Dec 03 12:04:11 crc kubenswrapper[5052]: I1203 12:04:11.515687 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/77f55a4f-32ec-4c57-82bc-ff0dd0aac965-operator-scripts\") pod \"neutron-d5c4-account-create-update-jngcd\" (UID: \"77f55a4f-32ec-4c57-82bc-ff0dd0aac965\") " pod="openstack-kuttl-tests/neutron-d5c4-account-create-update-jngcd" Dec 03 12:04:11 crc kubenswrapper[5052]: I1203 12:04:11.515845 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/90d21e8c-a52c-4109-811b-7d113748cea7-operator-scripts\") pod \"barbican-5c55-account-create-update-vz6m5\" (UID: \"90d21e8c-a52c-4109-811b-7d113748cea7\") " pod="openstack-kuttl-tests/barbican-5c55-account-create-update-vz6m5" Dec 03 12:04:11 crc kubenswrapper[5052]: I1203 12:04:11.515947 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-ml6wb\" (UniqueName: \"kubernetes.io/projected/90d21e8c-a52c-4109-811b-7d113748cea7-kube-api-access-ml6wb\") pod \"barbican-5c55-account-create-update-vz6m5\" (UID: \"90d21e8c-a52c-4109-811b-7d113748cea7\") " pod="openstack-kuttl-tests/barbican-5c55-account-create-update-vz6m5" Dec 03 12:04:11 crc kubenswrapper[5052]: I1203 12:04:11.515991 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2x74n\" (UniqueName: \"kubernetes.io/projected/77f55a4f-32ec-4c57-82bc-ff0dd0aac965-kube-api-access-2x74n\") pod \"neutron-d5c4-account-create-update-jngcd\" (UID: \"77f55a4f-32ec-4c57-82bc-ff0dd0aac965\") " pod="openstack-kuttl-tests/neutron-d5c4-account-create-update-jngcd" Dec 03 12:04:11 crc kubenswrapper[5052]: I1203 12:04:11.517111 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/90d21e8c-a52c-4109-811b-7d113748cea7-operator-scripts\") pod \"barbican-5c55-account-create-update-vz6m5\" (UID: \"90d21e8c-a52c-4109-811b-7d113748cea7\") " pod="openstack-kuttl-tests/barbican-5c55-account-create-update-vz6m5" Dec 03 12:04:11 crc kubenswrapper[5052]: I1203 12:04:11.532574 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-db-create-z9ftc" Dec 03 12:04:11 crc kubenswrapper[5052]: I1203 12:04:11.533397 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-e6fc-account-create-update-j5n92" Dec 03 12:04:11 crc kubenswrapper[5052]: I1203 12:04:11.545205 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-ml6wb\" (UniqueName: \"kubernetes.io/projected/90d21e8c-a52c-4109-811b-7d113748cea7-kube-api-access-ml6wb\") pod \"barbican-5c55-account-create-update-vz6m5\" (UID: \"90d21e8c-a52c-4109-811b-7d113748cea7\") " pod="openstack-kuttl-tests/barbican-5c55-account-create-update-vz6m5" Dec 03 12:04:11 crc kubenswrapper[5052]: I1203 12:04:11.615039 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-db-sync-8xzwc" Dec 03 12:04:11 crc kubenswrapper[5052]: I1203 12:04:11.617156 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-2x74n\" (UniqueName: \"kubernetes.io/projected/77f55a4f-32ec-4c57-82bc-ff0dd0aac965-kube-api-access-2x74n\") pod \"neutron-d5c4-account-create-update-jngcd\" (UID: \"77f55a4f-32ec-4c57-82bc-ff0dd0aac965\") " pod="openstack-kuttl-tests/neutron-d5c4-account-create-update-jngcd" Dec 03 12:04:11 crc kubenswrapper[5052]: I1203 12:04:11.617217 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/77f55a4f-32ec-4c57-82bc-ff0dd0aac965-operator-scripts\") pod \"neutron-d5c4-account-create-update-jngcd\" (UID: \"77f55a4f-32ec-4c57-82bc-ff0dd0aac965\") " pod="openstack-kuttl-tests/neutron-d5c4-account-create-update-jngcd" Dec 03 12:04:11 crc kubenswrapper[5052]: I1203 12:04:11.618285 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/77f55a4f-32ec-4c57-82bc-ff0dd0aac965-operator-scripts\") pod \"neutron-d5c4-account-create-update-jngcd\" (UID: \"77f55a4f-32ec-4c57-82bc-ff0dd0aac965\") " pod="openstack-kuttl-tests/neutron-d5c4-account-create-update-jngcd" Dec 03 12:04:11 crc kubenswrapper[5052]: I1203 12:04:11.643851 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-5c55-account-create-update-vz6m5" Dec 03 12:04:11 crc kubenswrapper[5052]: I1203 12:04:11.649506 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-2x74n\" (UniqueName: \"kubernetes.io/projected/77f55a4f-32ec-4c57-82bc-ff0dd0aac965-kube-api-access-2x74n\") pod \"neutron-d5c4-account-create-update-jngcd\" (UID: \"77f55a4f-32ec-4c57-82bc-ff0dd0aac965\") " pod="openstack-kuttl-tests/neutron-d5c4-account-create-update-jngcd" Dec 03 12:04:11 crc kubenswrapper[5052]: I1203 12:04:11.847661 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-d5c4-account-create-update-jngcd" Dec 03 12:04:12 crc kubenswrapper[5052]: I1203 12:04:12.075233 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-db-create-gsdcc"] Dec 03 12:04:12 crc kubenswrapper[5052]: I1203 12:04:12.288600 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-e6fc-account-create-update-j5n92"] Dec 03 12:04:12 crc kubenswrapper[5052]: I1203 12:04:12.295955 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-db-create-z9ftc"] Dec 03 12:04:12 crc kubenswrapper[5052]: W1203 12:04:12.339164 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod59c3e3b8_e2a6_4384_87f0_d1d7cbb4d738.slice/crio-66c0bda54d1831815304717388c80d27f14555120426066e46689f5639f2491a WatchSource:0}: Error finding container 66c0bda54d1831815304717388c80d27f14555120426066e46689f5639f2491a: Status 404 returned error can't find the container with id 66c0bda54d1831815304717388c80d27f14555120426066e46689f5639f2491a Dec 03 12:04:12 crc kubenswrapper[5052]: I1203 12:04:12.521903 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-db-sync-8xzwc"] Dec 03 12:04:12 crc kubenswrapper[5052]: W1203 12:04:12.531808 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod46eb04b8_4d02_4ffe_a0f6_bb4b236a4956.slice/crio-4f6bae0943404ccbad1eee7c5a65da3bc08d987dc7902f023ddf13bccbb85534 WatchSource:0}: Error finding container 4f6bae0943404ccbad1eee7c5a65da3bc08d987dc7902f023ddf13bccbb85534: Status 404 returned error can't find the container with id 4f6bae0943404ccbad1eee7c5a65da3bc08d987dc7902f023ddf13bccbb85534 Dec 03 12:04:12 crc kubenswrapper[5052]: I1203 12:04:12.690648 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-db-create-z9ftc" event={"ID":"8ba21481-e9f8-41f3-80e6-c96f860b7771","Type":"ContainerStarted","Data":"395e549f4f47dece407c316b9859ed7f9cd68f2549210dcc7dbc45082f1c56d8"} Dec 03 12:04:12 crc kubenswrapper[5052]: I1203 12:04:12.695687 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-db-create-z645k"] Dec 03 12:04:12 crc kubenswrapper[5052]: I1203 12:04:12.696337 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-db-sync-8xzwc" event={"ID":"46eb04b8-4d02-4ffe-a0f6-bb4b236a4956","Type":"ContainerStarted","Data":"4f6bae0943404ccbad1eee7c5a65da3bc08d987dc7902f023ddf13bccbb85534"} Dec 03 12:04:12 crc kubenswrapper[5052]: W1203 12:04:12.697093 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod67b28447_93fa_45c3_8c94_9dba5eba8f94.slice/crio-367a868f73216961f3caeacd0d3b47301abe415aa571b9afd765ca5c8abc736e WatchSource:0}: Error finding container 367a868f73216961f3caeacd0d3b47301abe415aa571b9afd765ca5c8abc736e: Status 404 returned error can't find the container with id 367a868f73216961f3caeacd0d3b47301abe415aa571b9afd765ca5c8abc736e Dec 03 12:04:12 crc kubenswrapper[5052]: I1203 12:04:12.698307 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-e6fc-account-create-update-j5n92" event={"ID":"59c3e3b8-e2a6-4384-87f0-d1d7cbb4d738","Type":"ContainerStarted","Data":"66c0bda54d1831815304717388c80d27f14555120426066e46689f5639f2491a"} Dec 03 12:04:12 crc kubenswrapper[5052]: I1203 12:04:12.701500 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-db-create-gsdcc" event={"ID":"2b7332a9-078d-4457-b136-899c4cdfaaae","Type":"ContainerStarted","Data":"d56d69a932a4844dfd6d420c8d3e39f3e14070df95fe2a5c2fbf44ba6ea2a2ba"} Dec 03 12:04:12 crc kubenswrapper[5052]: I1203 12:04:12.701561 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-db-create-gsdcc" event={"ID":"2b7332a9-078d-4457-b136-899c4cdfaaae","Type":"ContainerStarted","Data":"d2ce090a6b0545ccfbb5697b2b3238a92670ac2bbe65c93f7b3e48d21e486ee5"} Dec 03 12:04:12 crc kubenswrapper[5052]: I1203 12:04:12.745257 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-5c55-account-create-update-vz6m5"] Dec 03 12:04:12 crc kubenswrapper[5052]: W1203 12:04:12.753983 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod90d21e8c_a52c_4109_811b_7d113748cea7.slice/crio-506716d6f6889e22ff10d6ec8314b85e3f2bd4b4ae624ebb68f506ce204584df WatchSource:0}: Error finding container 506716d6f6889e22ff10d6ec8314b85e3f2bd4b4ae624ebb68f506ce204584df: Status 404 returned error can't find the container with id 506716d6f6889e22ff10d6ec8314b85e3f2bd4b4ae624ebb68f506ce204584df Dec 03 12:04:12 crc kubenswrapper[5052]: I1203 12:04:12.865057 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-d5c4-account-create-update-jngcd"] Dec 03 12:04:13 crc kubenswrapper[5052]: I1203 12:04:13.717576 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-5c55-account-create-update-vz6m5" event={"ID":"90d21e8c-a52c-4109-811b-7d113748cea7","Type":"ContainerStarted","Data":"c5b31ce4de482ce67274e759f4b4c8f79eb6d33839525a4f9726eab7cd834275"} Dec 03 12:04:13 crc kubenswrapper[5052]: I1203 12:04:13.717636 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-5c55-account-create-update-vz6m5" event={"ID":"90d21e8c-a52c-4109-811b-7d113748cea7","Type":"ContainerStarted","Data":"506716d6f6889e22ff10d6ec8314b85e3f2bd4b4ae624ebb68f506ce204584df"} Dec 03 12:04:13 crc kubenswrapper[5052]: I1203 12:04:13.719973 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-db-create-z9ftc" event={"ID":"8ba21481-e9f8-41f3-80e6-c96f860b7771","Type":"ContainerStarted","Data":"13208c18005d83a4860b544d6bc809d62861e10156063a4cca40ac62076a90b2"} Dec 03 12:04:13 crc kubenswrapper[5052]: I1203 12:04:13.721621 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-db-sync-8xzwc" event={"ID":"46eb04b8-4d02-4ffe-a0f6-bb4b236a4956","Type":"ContainerStarted","Data":"19256df03b425ac2ffa4ab3adf5ae27b797e8391dbfdc95ffe913a963e62c47d"} Dec 03 12:04:13 crc kubenswrapper[5052]: I1203 12:04:13.729026 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-e6fc-account-create-update-j5n92" event={"ID":"59c3e3b8-e2a6-4384-87f0-d1d7cbb4d738","Type":"ContainerStarted","Data":"a370d0d01033a8b135a4295ba72e03bb44219e4dfe9ea35c01614ac7a79ce4b1"} Dec 03 12:04:13 crc kubenswrapper[5052]: I1203 12:04:13.730608 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-db-create-z645k" event={"ID":"67b28447-93fa-45c3-8c94-9dba5eba8f94","Type":"ContainerStarted","Data":"445784d8ad0c8332dc5eca6c089a13cfe1fa7b4f744c5922a315d621eed9a21d"} Dec 03 12:04:13 crc kubenswrapper[5052]: I1203 12:04:13.730637 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-db-create-z645k" event={"ID":"67b28447-93fa-45c3-8c94-9dba5eba8f94","Type":"ContainerStarted","Data":"367a868f73216961f3caeacd0d3b47301abe415aa571b9afd765ca5c8abc736e"} Dec 03 12:04:13 crc kubenswrapper[5052]: I1203 12:04:13.734043 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-d5c4-account-create-update-jngcd" event={"ID":"77f55a4f-32ec-4c57-82bc-ff0dd0aac965","Type":"ContainerStarted","Data":"1faba2664a45d58a12ee28b49eef362ded8c4da1f52ad06d789532a9d315d90a"} Dec 03 12:04:13 crc kubenswrapper[5052]: I1203 12:04:13.734098 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-d5c4-account-create-update-jngcd" event={"ID":"77f55a4f-32ec-4c57-82bc-ff0dd0aac965","Type":"ContainerStarted","Data":"7b0ce5d1136ba2370134ee03980757cf80ace8a45876a9175a31272395597c9a"} Dec 03 12:04:13 crc kubenswrapper[5052]: I1203 12:04:13.741152 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/barbican-5c55-account-create-update-vz6m5" podStartSLOduration=2.741129701 podStartE2EDuration="2.741129701s" podCreationTimestamp="2025-12-03 12:04:11 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:04:13.733824172 +0000 UTC m=+2644.322065956" watchObservedRunningTime="2025-12-03 12:04:13.741129701 +0000 UTC m=+2644.329371495" Dec 03 12:04:13 crc kubenswrapper[5052]: I1203 12:04:13.760382 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/keystone-db-sync-8xzwc" podStartSLOduration=2.760360461 podStartE2EDuration="2.760360461s" podCreationTimestamp="2025-12-03 12:04:11 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:04:13.751668553 +0000 UTC m=+2644.339910367" watchObservedRunningTime="2025-12-03 12:04:13.760360461 +0000 UTC m=+2644.348602225" Dec 03 12:04:13 crc kubenswrapper[5052]: I1203 12:04:13.768841 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/neutron-db-create-z9ftc" podStartSLOduration=2.768824053 podStartE2EDuration="2.768824053s" podCreationTimestamp="2025-12-03 12:04:11 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:04:13.7669783 +0000 UTC m=+2644.355220064" watchObservedRunningTime="2025-12-03 12:04:13.768824053 +0000 UTC m=+2644.357065817" Dec 03 12:04:13 crc kubenswrapper[5052]: I1203 12:04:13.785379 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/cinder-db-create-gsdcc" podStartSLOduration=3.785357276 podStartE2EDuration="3.785357276s" podCreationTimestamp="2025-12-03 12:04:10 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:04:13.77992847 +0000 UTC m=+2644.368170234" watchObservedRunningTime="2025-12-03 12:04:13.785357276 +0000 UTC m=+2644.373599040" Dec 03 12:04:14 crc kubenswrapper[5052]: I1203 12:04:14.762525 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/barbican-db-create-z645k" podStartSLOduration=3.7624902110000003 podStartE2EDuration="3.762490211s" podCreationTimestamp="2025-12-03 12:04:11 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:04:14.76143494 +0000 UTC m=+2645.349676724" watchObservedRunningTime="2025-12-03 12:04:14.762490211 +0000 UTC m=+2645.350731975" Dec 03 12:04:14 crc kubenswrapper[5052]: I1203 12:04:14.797925 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/cinder-e6fc-account-create-update-j5n92" podStartSLOduration=3.797900123 podStartE2EDuration="3.797900123s" podCreationTimestamp="2025-12-03 12:04:11 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:04:14.775921245 +0000 UTC m=+2645.364163029" watchObservedRunningTime="2025-12-03 12:04:14.797900123 +0000 UTC m=+2645.386141887" Dec 03 12:04:14 crc kubenswrapper[5052]: I1203 12:04:14.811079 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/neutron-d5c4-account-create-update-jngcd" podStartSLOduration=3.8110617490000003 podStartE2EDuration="3.811061749s" podCreationTimestamp="2025-12-03 12:04:11 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:04:14.801853326 +0000 UTC m=+2645.390095100" watchObservedRunningTime="2025-12-03 12:04:14.811061749 +0000 UTC m=+2645.399303513" Dec 03 12:04:16 crc kubenswrapper[5052]: I1203 12:04:16.971734 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openshift-marketplace/redhat-operators-jjj57"] Dec 03 12:04:16 crc kubenswrapper[5052]: I1203 12:04:16.975262 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/redhat-operators-jjj57" Dec 03 12:04:17 crc kubenswrapper[5052]: I1203 12:04:17.006043 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/redhat-operators-jjj57"] Dec 03 12:04:17 crc kubenswrapper[5052]: I1203 12:04:17.146278 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/ede81f70-0e50-47b2-85b7-d30ec858e2e4-utilities\") pod \"redhat-operators-jjj57\" (UID: \"ede81f70-0e50-47b2-85b7-d30ec858e2e4\") " pod="openshift-marketplace/redhat-operators-jjj57" Dec 03 12:04:17 crc kubenswrapper[5052]: I1203 12:04:17.146504 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lbpxf\" (UniqueName: \"kubernetes.io/projected/ede81f70-0e50-47b2-85b7-d30ec858e2e4-kube-api-access-lbpxf\") pod \"redhat-operators-jjj57\" (UID: \"ede81f70-0e50-47b2-85b7-d30ec858e2e4\") " pod="openshift-marketplace/redhat-operators-jjj57" Dec 03 12:04:17 crc kubenswrapper[5052]: I1203 12:04:17.146735 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/ede81f70-0e50-47b2-85b7-d30ec858e2e4-catalog-content\") pod \"redhat-operators-jjj57\" (UID: \"ede81f70-0e50-47b2-85b7-d30ec858e2e4\") " pod="openshift-marketplace/redhat-operators-jjj57" Dec 03 12:04:17 crc kubenswrapper[5052]: I1203 12:04:17.248759 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/ede81f70-0e50-47b2-85b7-d30ec858e2e4-utilities\") pod \"redhat-operators-jjj57\" (UID: \"ede81f70-0e50-47b2-85b7-d30ec858e2e4\") " pod="openshift-marketplace/redhat-operators-jjj57" Dec 03 12:04:17 crc kubenswrapper[5052]: I1203 12:04:17.248869 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-lbpxf\" (UniqueName: \"kubernetes.io/projected/ede81f70-0e50-47b2-85b7-d30ec858e2e4-kube-api-access-lbpxf\") pod \"redhat-operators-jjj57\" (UID: \"ede81f70-0e50-47b2-85b7-d30ec858e2e4\") " pod="openshift-marketplace/redhat-operators-jjj57" Dec 03 12:04:17 crc kubenswrapper[5052]: I1203 12:04:17.248961 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/ede81f70-0e50-47b2-85b7-d30ec858e2e4-catalog-content\") pod \"redhat-operators-jjj57\" (UID: \"ede81f70-0e50-47b2-85b7-d30ec858e2e4\") " pod="openshift-marketplace/redhat-operators-jjj57" Dec 03 12:04:17 crc kubenswrapper[5052]: I1203 12:04:17.249305 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/ede81f70-0e50-47b2-85b7-d30ec858e2e4-utilities\") pod \"redhat-operators-jjj57\" (UID: \"ede81f70-0e50-47b2-85b7-d30ec858e2e4\") " pod="openshift-marketplace/redhat-operators-jjj57" Dec 03 12:04:17 crc kubenswrapper[5052]: I1203 12:04:17.249440 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/ede81f70-0e50-47b2-85b7-d30ec858e2e4-catalog-content\") pod \"redhat-operators-jjj57\" (UID: \"ede81f70-0e50-47b2-85b7-d30ec858e2e4\") " pod="openshift-marketplace/redhat-operators-jjj57" Dec 03 12:04:17 crc kubenswrapper[5052]: I1203 12:04:17.271492 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-lbpxf\" (UniqueName: \"kubernetes.io/projected/ede81f70-0e50-47b2-85b7-d30ec858e2e4-kube-api-access-lbpxf\") pod \"redhat-operators-jjj57\" (UID: \"ede81f70-0e50-47b2-85b7-d30ec858e2e4\") " pod="openshift-marketplace/redhat-operators-jjj57" Dec 03 12:04:17 crc kubenswrapper[5052]: I1203 12:04:17.370146 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/redhat-operators-jjj57" Dec 03 12:04:17 crc kubenswrapper[5052]: I1203 12:04:17.775675 5052 generic.go:334] "Generic (PLEG): container finished" podID="2b7332a9-078d-4457-b136-899c4cdfaaae" containerID="d56d69a932a4844dfd6d420c8d3e39f3e14070df95fe2a5c2fbf44ba6ea2a2ba" exitCode=0 Dec 03 12:04:17 crc kubenswrapper[5052]: I1203 12:04:17.775933 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-db-create-gsdcc" event={"ID":"2b7332a9-078d-4457-b136-899c4cdfaaae","Type":"ContainerDied","Data":"d56d69a932a4844dfd6d420c8d3e39f3e14070df95fe2a5c2fbf44ba6ea2a2ba"} Dec 03 12:04:17 crc kubenswrapper[5052]: I1203 12:04:17.780418 5052 generic.go:334] "Generic (PLEG): container finished" podID="8ba21481-e9f8-41f3-80e6-c96f860b7771" containerID="13208c18005d83a4860b544d6bc809d62861e10156063a4cca40ac62076a90b2" exitCode=0 Dec 03 12:04:17 crc kubenswrapper[5052]: I1203 12:04:17.780454 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-db-create-z9ftc" event={"ID":"8ba21481-e9f8-41f3-80e6-c96f860b7771","Type":"ContainerDied","Data":"13208c18005d83a4860b544d6bc809d62861e10156063a4cca40ac62076a90b2"} Dec 03 12:04:18 crc kubenswrapper[5052]: I1203 12:04:18.005992 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/redhat-operators-jjj57"] Dec 03 12:04:18 crc kubenswrapper[5052]: I1203 12:04:18.790356 5052 generic.go:334] "Generic (PLEG): container finished" podID="67b28447-93fa-45c3-8c94-9dba5eba8f94" containerID="445784d8ad0c8332dc5eca6c089a13cfe1fa7b4f744c5922a315d621eed9a21d" exitCode=0 Dec 03 12:04:18 crc kubenswrapper[5052]: I1203 12:04:18.790423 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-db-create-z645k" event={"ID":"67b28447-93fa-45c3-8c94-9dba5eba8f94","Type":"ContainerDied","Data":"445784d8ad0c8332dc5eca6c089a13cfe1fa7b4f744c5922a315d621eed9a21d"} Dec 03 12:04:18 crc kubenswrapper[5052]: I1203 12:04:18.792987 5052 generic.go:334] "Generic (PLEG): container finished" podID="ede81f70-0e50-47b2-85b7-d30ec858e2e4" containerID="5fe6a1c2c920a2c8fd804ec100332c776ee457aab74f8d3a733fd1848b5a20d7" exitCode=0 Dec 03 12:04:18 crc kubenswrapper[5052]: I1203 12:04:18.793138 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-operators-jjj57" event={"ID":"ede81f70-0e50-47b2-85b7-d30ec858e2e4","Type":"ContainerDied","Data":"5fe6a1c2c920a2c8fd804ec100332c776ee457aab74f8d3a733fd1848b5a20d7"} Dec 03 12:04:18 crc kubenswrapper[5052]: I1203 12:04:18.793256 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-operators-jjj57" event={"ID":"ede81f70-0e50-47b2-85b7-d30ec858e2e4","Type":"ContainerStarted","Data":"b050828e580762e184f99bf748b8e3628e6af1887a0356967f23e28b640904d0"} Dec 03 12:04:18 crc kubenswrapper[5052]: I1203 12:04:18.796231 5052 provider.go:102] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Dec 03 12:04:19 crc kubenswrapper[5052]: I1203 12:04:19.242705 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-db-create-z9ftc" Dec 03 12:04:19 crc kubenswrapper[5052]: I1203 12:04:19.251192 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-db-create-gsdcc" Dec 03 12:04:19 crc kubenswrapper[5052]: I1203 12:04:19.402385 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jj7kv\" (UniqueName: \"kubernetes.io/projected/2b7332a9-078d-4457-b136-899c4cdfaaae-kube-api-access-jj7kv\") pod \"2b7332a9-078d-4457-b136-899c4cdfaaae\" (UID: \"2b7332a9-078d-4457-b136-899c4cdfaaae\") " Dec 03 12:04:19 crc kubenswrapper[5052]: I1203 12:04:19.402490 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/8ba21481-e9f8-41f3-80e6-c96f860b7771-operator-scripts\") pod \"8ba21481-e9f8-41f3-80e6-c96f860b7771\" (UID: \"8ba21481-e9f8-41f3-80e6-c96f860b7771\") " Dec 03 12:04:19 crc kubenswrapper[5052]: I1203 12:04:19.402582 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vsvcb\" (UniqueName: \"kubernetes.io/projected/8ba21481-e9f8-41f3-80e6-c96f860b7771-kube-api-access-vsvcb\") pod \"8ba21481-e9f8-41f3-80e6-c96f860b7771\" (UID: \"8ba21481-e9f8-41f3-80e6-c96f860b7771\") " Dec 03 12:04:19 crc kubenswrapper[5052]: I1203 12:04:19.402647 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/2b7332a9-078d-4457-b136-899c4cdfaaae-operator-scripts\") pod \"2b7332a9-078d-4457-b136-899c4cdfaaae\" (UID: \"2b7332a9-078d-4457-b136-899c4cdfaaae\") " Dec 03 12:04:19 crc kubenswrapper[5052]: I1203 12:04:19.403046 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8ba21481-e9f8-41f3-80e6-c96f860b7771-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "8ba21481-e9f8-41f3-80e6-c96f860b7771" (UID: "8ba21481-e9f8-41f3-80e6-c96f860b7771"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:04:19 crc kubenswrapper[5052]: I1203 12:04:19.403585 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2b7332a9-078d-4457-b136-899c4cdfaaae-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "2b7332a9-078d-4457-b136-899c4cdfaaae" (UID: "2b7332a9-078d-4457-b136-899c4cdfaaae"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:04:19 crc kubenswrapper[5052]: I1203 12:04:19.414091 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2b7332a9-078d-4457-b136-899c4cdfaaae-kube-api-access-jj7kv" (OuterVolumeSpecName: "kube-api-access-jj7kv") pod "2b7332a9-078d-4457-b136-899c4cdfaaae" (UID: "2b7332a9-078d-4457-b136-899c4cdfaaae"). InnerVolumeSpecName "kube-api-access-jj7kv". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:04:19 crc kubenswrapper[5052]: I1203 12:04:19.414578 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8ba21481-e9f8-41f3-80e6-c96f860b7771-kube-api-access-vsvcb" (OuterVolumeSpecName: "kube-api-access-vsvcb") pod "8ba21481-e9f8-41f3-80e6-c96f860b7771" (UID: "8ba21481-e9f8-41f3-80e6-c96f860b7771"). InnerVolumeSpecName "kube-api-access-vsvcb". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:04:19 crc kubenswrapper[5052]: I1203 12:04:19.504348 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-vsvcb\" (UniqueName: \"kubernetes.io/projected/8ba21481-e9f8-41f3-80e6-c96f860b7771-kube-api-access-vsvcb\") on node \"crc\" DevicePath \"\"" Dec 03 12:04:19 crc kubenswrapper[5052]: I1203 12:04:19.504392 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/2b7332a9-078d-4457-b136-899c4cdfaaae-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:04:19 crc kubenswrapper[5052]: I1203 12:04:19.504405 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-jj7kv\" (UniqueName: \"kubernetes.io/projected/2b7332a9-078d-4457-b136-899c4cdfaaae-kube-api-access-jj7kv\") on node \"crc\" DevicePath \"\"" Dec 03 12:04:19 crc kubenswrapper[5052]: I1203 12:04:19.504418 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/8ba21481-e9f8-41f3-80e6-c96f860b7771-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:04:19 crc kubenswrapper[5052]: I1203 12:04:19.804738 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-db-create-z9ftc" Dec 03 12:04:19 crc kubenswrapper[5052]: I1203 12:04:19.804713 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-db-create-z9ftc" event={"ID":"8ba21481-e9f8-41f3-80e6-c96f860b7771","Type":"ContainerDied","Data":"395e549f4f47dece407c316b9859ed7f9cd68f2549210dcc7dbc45082f1c56d8"} Dec 03 12:04:19 crc kubenswrapper[5052]: I1203 12:04:19.804860 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="395e549f4f47dece407c316b9859ed7f9cd68f2549210dcc7dbc45082f1c56d8" Dec 03 12:04:19 crc kubenswrapper[5052]: I1203 12:04:19.818240 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-db-create-gsdcc" Dec 03 12:04:19 crc kubenswrapper[5052]: I1203 12:04:19.818335 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-db-create-gsdcc" event={"ID":"2b7332a9-078d-4457-b136-899c4cdfaaae","Type":"ContainerDied","Data":"d2ce090a6b0545ccfbb5697b2b3238a92670ac2bbe65c93f7b3e48d21e486ee5"} Dec 03 12:04:19 crc kubenswrapper[5052]: I1203 12:04:19.818371 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d2ce090a6b0545ccfbb5697b2b3238a92670ac2bbe65c93f7b3e48d21e486ee5" Dec 03 12:04:20 crc kubenswrapper[5052]: I1203 12:04:20.191980 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-db-create-z645k" Dec 03 12:04:20 crc kubenswrapper[5052]: I1203 12:04:20.319319 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/67b28447-93fa-45c3-8c94-9dba5eba8f94-operator-scripts\") pod \"67b28447-93fa-45c3-8c94-9dba5eba8f94\" (UID: \"67b28447-93fa-45c3-8c94-9dba5eba8f94\") " Dec 03 12:04:20 crc kubenswrapper[5052]: I1203 12:04:20.319428 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8j2pm\" (UniqueName: \"kubernetes.io/projected/67b28447-93fa-45c3-8c94-9dba5eba8f94-kube-api-access-8j2pm\") pod \"67b28447-93fa-45c3-8c94-9dba5eba8f94\" (UID: \"67b28447-93fa-45c3-8c94-9dba5eba8f94\") " Dec 03 12:04:20 crc kubenswrapper[5052]: I1203 12:04:20.320229 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/67b28447-93fa-45c3-8c94-9dba5eba8f94-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "67b28447-93fa-45c3-8c94-9dba5eba8f94" (UID: "67b28447-93fa-45c3-8c94-9dba5eba8f94"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:04:20 crc kubenswrapper[5052]: I1203 12:04:20.321088 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/67b28447-93fa-45c3-8c94-9dba5eba8f94-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:04:20 crc kubenswrapper[5052]: I1203 12:04:20.328027 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/67b28447-93fa-45c3-8c94-9dba5eba8f94-kube-api-access-8j2pm" (OuterVolumeSpecName: "kube-api-access-8j2pm") pod "67b28447-93fa-45c3-8c94-9dba5eba8f94" (UID: "67b28447-93fa-45c3-8c94-9dba5eba8f94"). InnerVolumeSpecName "kube-api-access-8j2pm". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:04:20 crc kubenswrapper[5052]: I1203 12:04:20.423348 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-8j2pm\" (UniqueName: \"kubernetes.io/projected/67b28447-93fa-45c3-8c94-9dba5eba8f94-kube-api-access-8j2pm\") on node \"crc\" DevicePath \"\"" Dec 03 12:04:20 crc kubenswrapper[5052]: I1203 12:04:20.830636 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-db-create-z645k" event={"ID":"67b28447-93fa-45c3-8c94-9dba5eba8f94","Type":"ContainerDied","Data":"367a868f73216961f3caeacd0d3b47301abe415aa571b9afd765ca5c8abc736e"} Dec 03 12:04:20 crc kubenswrapper[5052]: I1203 12:04:20.830687 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="367a868f73216961f3caeacd0d3b47301abe415aa571b9afd765ca5c8abc736e" Dec 03 12:04:20 crc kubenswrapper[5052]: I1203 12:04:20.830718 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-db-create-z645k" Dec 03 12:04:21 crc kubenswrapper[5052]: I1203 12:04:21.840832 5052 generic.go:334] "Generic (PLEG): container finished" podID="59c3e3b8-e2a6-4384-87f0-d1d7cbb4d738" containerID="a370d0d01033a8b135a4295ba72e03bb44219e4dfe9ea35c01614ac7a79ce4b1" exitCode=0 Dec 03 12:04:21 crc kubenswrapper[5052]: I1203 12:04:21.840922 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-e6fc-account-create-update-j5n92" event={"ID":"59c3e3b8-e2a6-4384-87f0-d1d7cbb4d738","Type":"ContainerDied","Data":"a370d0d01033a8b135a4295ba72e03bb44219e4dfe9ea35c01614ac7a79ce4b1"} Dec 03 12:04:21 crc kubenswrapper[5052]: I1203 12:04:21.843792 5052 generic.go:334] "Generic (PLEG): container finished" podID="ede81f70-0e50-47b2-85b7-d30ec858e2e4" containerID="286a3f1bf5f4923c58cec63c2cdb346e707904aba0bb87a984d018cbba3b63b2" exitCode=0 Dec 03 12:04:21 crc kubenswrapper[5052]: I1203 12:04:21.843820 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-operators-jjj57" event={"ID":"ede81f70-0e50-47b2-85b7-d30ec858e2e4","Type":"ContainerDied","Data":"286a3f1bf5f4923c58cec63c2cdb346e707904aba0bb87a984d018cbba3b63b2"} Dec 03 12:04:21 crc kubenswrapper[5052]: I1203 12:04:21.845228 5052 generic.go:334] "Generic (PLEG): container finished" podID="77f55a4f-32ec-4c57-82bc-ff0dd0aac965" containerID="1faba2664a45d58a12ee28b49eef362ded8c4da1f52ad06d789532a9d315d90a" exitCode=0 Dec 03 12:04:21 crc kubenswrapper[5052]: I1203 12:04:21.845270 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-d5c4-account-create-update-jngcd" event={"ID":"77f55a4f-32ec-4c57-82bc-ff0dd0aac965","Type":"ContainerDied","Data":"1faba2664a45d58a12ee28b49eef362ded8c4da1f52ad06d789532a9d315d90a"} Dec 03 12:04:21 crc kubenswrapper[5052]: I1203 12:04:21.848994 5052 generic.go:334] "Generic (PLEG): container finished" podID="90d21e8c-a52c-4109-811b-7d113748cea7" containerID="c5b31ce4de482ce67274e759f4b4c8f79eb6d33839525a4f9726eab7cd834275" exitCode=0 Dec 03 12:04:21 crc kubenswrapper[5052]: I1203 12:04:21.849055 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-5c55-account-create-update-vz6m5" event={"ID":"90d21e8c-a52c-4109-811b-7d113748cea7","Type":"ContainerDied","Data":"c5b31ce4de482ce67274e759f4b4c8f79eb6d33839525a4f9726eab7cd834275"} Dec 03 12:04:23 crc kubenswrapper[5052]: I1203 12:04:23.407895 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-e6fc-account-create-update-j5n92" Dec 03 12:04:23 crc kubenswrapper[5052]: I1203 12:04:23.426777 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-5c55-account-create-update-vz6m5" Dec 03 12:04:23 crc kubenswrapper[5052]: I1203 12:04:23.440673 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-d5c4-account-create-update-jngcd" Dec 03 12:04:23 crc kubenswrapper[5052]: I1203 12:04:23.445155 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/90d21e8c-a52c-4109-811b-7d113748cea7-operator-scripts\") pod \"90d21e8c-a52c-4109-811b-7d113748cea7\" (UID: \"90d21e8c-a52c-4109-811b-7d113748cea7\") " Dec 03 12:04:23 crc kubenswrapper[5052]: I1203 12:04:23.445228 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/77f55a4f-32ec-4c57-82bc-ff0dd0aac965-operator-scripts\") pod \"77f55a4f-32ec-4c57-82bc-ff0dd0aac965\" (UID: \"77f55a4f-32ec-4c57-82bc-ff0dd0aac965\") " Dec 03 12:04:23 crc kubenswrapper[5052]: I1203 12:04:23.445310 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5dmxz\" (UniqueName: \"kubernetes.io/projected/59c3e3b8-e2a6-4384-87f0-d1d7cbb4d738-kube-api-access-5dmxz\") pod \"59c3e3b8-e2a6-4384-87f0-d1d7cbb4d738\" (UID: \"59c3e3b8-e2a6-4384-87f0-d1d7cbb4d738\") " Dec 03 12:04:23 crc kubenswrapper[5052]: I1203 12:04:23.445363 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ml6wb\" (UniqueName: \"kubernetes.io/projected/90d21e8c-a52c-4109-811b-7d113748cea7-kube-api-access-ml6wb\") pod \"90d21e8c-a52c-4109-811b-7d113748cea7\" (UID: \"90d21e8c-a52c-4109-811b-7d113748cea7\") " Dec 03 12:04:23 crc kubenswrapper[5052]: I1203 12:04:23.445385 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/59c3e3b8-e2a6-4384-87f0-d1d7cbb4d738-operator-scripts\") pod \"59c3e3b8-e2a6-4384-87f0-d1d7cbb4d738\" (UID: \"59c3e3b8-e2a6-4384-87f0-d1d7cbb4d738\") " Dec 03 12:04:23 crc kubenswrapper[5052]: I1203 12:04:23.445410 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2x74n\" (UniqueName: \"kubernetes.io/projected/77f55a4f-32ec-4c57-82bc-ff0dd0aac965-kube-api-access-2x74n\") pod \"77f55a4f-32ec-4c57-82bc-ff0dd0aac965\" (UID: \"77f55a4f-32ec-4c57-82bc-ff0dd0aac965\") " Dec 03 12:04:23 crc kubenswrapper[5052]: I1203 12:04:23.448580 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/77f55a4f-32ec-4c57-82bc-ff0dd0aac965-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "77f55a4f-32ec-4c57-82bc-ff0dd0aac965" (UID: "77f55a4f-32ec-4c57-82bc-ff0dd0aac965"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:04:23 crc kubenswrapper[5052]: I1203 12:04:23.448916 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/59c3e3b8-e2a6-4384-87f0-d1d7cbb4d738-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "59c3e3b8-e2a6-4384-87f0-d1d7cbb4d738" (UID: "59c3e3b8-e2a6-4384-87f0-d1d7cbb4d738"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:04:23 crc kubenswrapper[5052]: I1203 12:04:23.452811 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/90d21e8c-a52c-4109-811b-7d113748cea7-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "90d21e8c-a52c-4109-811b-7d113748cea7" (UID: "90d21e8c-a52c-4109-811b-7d113748cea7"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:04:23 crc kubenswrapper[5052]: I1203 12:04:23.459429 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/59c3e3b8-e2a6-4384-87f0-d1d7cbb4d738-kube-api-access-5dmxz" (OuterVolumeSpecName: "kube-api-access-5dmxz") pod "59c3e3b8-e2a6-4384-87f0-d1d7cbb4d738" (UID: "59c3e3b8-e2a6-4384-87f0-d1d7cbb4d738"). InnerVolumeSpecName "kube-api-access-5dmxz". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:04:23 crc kubenswrapper[5052]: I1203 12:04:23.462012 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/77f55a4f-32ec-4c57-82bc-ff0dd0aac965-kube-api-access-2x74n" (OuterVolumeSpecName: "kube-api-access-2x74n") pod "77f55a4f-32ec-4c57-82bc-ff0dd0aac965" (UID: "77f55a4f-32ec-4c57-82bc-ff0dd0aac965"). InnerVolumeSpecName "kube-api-access-2x74n". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:04:23 crc kubenswrapper[5052]: I1203 12:04:23.485436 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/90d21e8c-a52c-4109-811b-7d113748cea7-kube-api-access-ml6wb" (OuterVolumeSpecName: "kube-api-access-ml6wb") pod "90d21e8c-a52c-4109-811b-7d113748cea7" (UID: "90d21e8c-a52c-4109-811b-7d113748cea7"). InnerVolumeSpecName "kube-api-access-ml6wb". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:04:23 crc kubenswrapper[5052]: I1203 12:04:23.546861 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-ml6wb\" (UniqueName: \"kubernetes.io/projected/90d21e8c-a52c-4109-811b-7d113748cea7-kube-api-access-ml6wb\") on node \"crc\" DevicePath \"\"" Dec 03 12:04:23 crc kubenswrapper[5052]: I1203 12:04:23.546896 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/59c3e3b8-e2a6-4384-87f0-d1d7cbb4d738-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:04:23 crc kubenswrapper[5052]: I1203 12:04:23.546911 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-2x74n\" (UniqueName: \"kubernetes.io/projected/77f55a4f-32ec-4c57-82bc-ff0dd0aac965-kube-api-access-2x74n\") on node \"crc\" DevicePath \"\"" Dec 03 12:04:23 crc kubenswrapper[5052]: I1203 12:04:23.546924 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/90d21e8c-a52c-4109-811b-7d113748cea7-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:04:23 crc kubenswrapper[5052]: I1203 12:04:23.546935 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/77f55a4f-32ec-4c57-82bc-ff0dd0aac965-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:04:23 crc kubenswrapper[5052]: I1203 12:04:23.546945 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-5dmxz\" (UniqueName: \"kubernetes.io/projected/59c3e3b8-e2a6-4384-87f0-d1d7cbb4d738-kube-api-access-5dmxz\") on node \"crc\" DevicePath \"\"" Dec 03 12:04:23 crc kubenswrapper[5052]: I1203 12:04:23.869052 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-d5c4-account-create-update-jngcd" event={"ID":"77f55a4f-32ec-4c57-82bc-ff0dd0aac965","Type":"ContainerDied","Data":"7b0ce5d1136ba2370134ee03980757cf80ace8a45876a9175a31272395597c9a"} Dec 03 12:04:23 crc kubenswrapper[5052]: I1203 12:04:23.869108 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7b0ce5d1136ba2370134ee03980757cf80ace8a45876a9175a31272395597c9a" Dec 03 12:04:23 crc kubenswrapper[5052]: I1203 12:04:23.869179 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-d5c4-account-create-update-jngcd" Dec 03 12:04:23 crc kubenswrapper[5052]: I1203 12:04:23.871715 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-5c55-account-create-update-vz6m5" event={"ID":"90d21e8c-a52c-4109-811b-7d113748cea7","Type":"ContainerDied","Data":"506716d6f6889e22ff10d6ec8314b85e3f2bd4b4ae624ebb68f506ce204584df"} Dec 03 12:04:23 crc kubenswrapper[5052]: I1203 12:04:23.871753 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="506716d6f6889e22ff10d6ec8314b85e3f2bd4b4ae624ebb68f506ce204584df" Dec 03 12:04:23 crc kubenswrapper[5052]: I1203 12:04:23.871810 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-5c55-account-create-update-vz6m5" Dec 03 12:04:23 crc kubenswrapper[5052]: I1203 12:04:23.876628 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-e6fc-account-create-update-j5n92" Dec 03 12:04:23 crc kubenswrapper[5052]: I1203 12:04:23.876653 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-e6fc-account-create-update-j5n92" event={"ID":"59c3e3b8-e2a6-4384-87f0-d1d7cbb4d738","Type":"ContainerDied","Data":"66c0bda54d1831815304717388c80d27f14555120426066e46689f5639f2491a"} Dec 03 12:04:23 crc kubenswrapper[5052]: I1203 12:04:23.876686 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="66c0bda54d1831815304717388c80d27f14555120426066e46689f5639f2491a" Dec 03 12:04:23 crc kubenswrapper[5052]: I1203 12:04:23.881161 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-operators-jjj57" event={"ID":"ede81f70-0e50-47b2-85b7-d30ec858e2e4","Type":"ContainerStarted","Data":"23730e89625cbefdb76c39b2c82e8573ab145d63d3aee524c618c90e90b9af8f"} Dec 03 12:04:23 crc kubenswrapper[5052]: I1203 12:04:23.936221 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-marketplace/redhat-operators-jjj57" podStartSLOduration=3.407627083 podStartE2EDuration="7.936200569s" podCreationTimestamp="2025-12-03 12:04:16 +0000 UTC" firstStartedPulling="2025-12-03 12:04:18.795930264 +0000 UTC m=+2649.384172028" lastFinishedPulling="2025-12-03 12:04:23.32450376 +0000 UTC m=+2653.912745514" observedRunningTime="2025-12-03 12:04:23.923818045 +0000 UTC m=+2654.512059809" watchObservedRunningTime="2025-12-03 12:04:23.936200569 +0000 UTC m=+2654.524442333" Dec 03 12:04:25 crc kubenswrapper[5052]: I1203 12:04:25.914610 5052 generic.go:334] "Generic (PLEG): container finished" podID="df00a025-ee23-44aa-8970-6f1232146b98" containerID="68e08571183dbfc32af1f09b3eda2efa53f0247f79e3a38eb83e5b3438697496" exitCode=0 Dec 03 12:04:25 crc kubenswrapper[5052]: I1203 12:04:25.915044 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-db-sync-2lq57" event={"ID":"df00a025-ee23-44aa-8970-6f1232146b98","Type":"ContainerDied","Data":"68e08571183dbfc32af1f09b3eda2efa53f0247f79e3a38eb83e5b3438697496"} Dec 03 12:04:26 crc kubenswrapper[5052]: I1203 12:04:26.927820 5052 generic.go:334] "Generic (PLEG): container finished" podID="46eb04b8-4d02-4ffe-a0f6-bb4b236a4956" containerID="19256df03b425ac2ffa4ab3adf5ae27b797e8391dbfdc95ffe913a963e62c47d" exitCode=0 Dec 03 12:04:26 crc kubenswrapper[5052]: I1203 12:04:26.928204 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-db-sync-8xzwc" event={"ID":"46eb04b8-4d02-4ffe-a0f6-bb4b236a4956","Type":"ContainerDied","Data":"19256df03b425ac2ffa4ab3adf5ae27b797e8391dbfdc95ffe913a963e62c47d"} Dec 03 12:04:27 crc kubenswrapper[5052]: I1203 12:04:27.362968 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-db-sync-2lq57" Dec 03 12:04:27 crc kubenswrapper[5052]: I1203 12:04:27.370845 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-marketplace/redhat-operators-jjj57" Dec 03 12:04:27 crc kubenswrapper[5052]: I1203 12:04:27.372569 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openshift-marketplace/redhat-operators-jjj57" Dec 03 12:04:27 crc kubenswrapper[5052]: I1203 12:04:27.522857 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/df00a025-ee23-44aa-8970-6f1232146b98-config-data\") pod \"df00a025-ee23-44aa-8970-6f1232146b98\" (UID: \"df00a025-ee23-44aa-8970-6f1232146b98\") " Dec 03 12:04:27 crc kubenswrapper[5052]: I1203 12:04:27.523060 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/df00a025-ee23-44aa-8970-6f1232146b98-combined-ca-bundle\") pod \"df00a025-ee23-44aa-8970-6f1232146b98\" (UID: \"df00a025-ee23-44aa-8970-6f1232146b98\") " Dec 03 12:04:27 crc kubenswrapper[5052]: I1203 12:04:27.523116 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bbwbf\" (UniqueName: \"kubernetes.io/projected/df00a025-ee23-44aa-8970-6f1232146b98-kube-api-access-bbwbf\") pod \"df00a025-ee23-44aa-8970-6f1232146b98\" (UID: \"df00a025-ee23-44aa-8970-6f1232146b98\") " Dec 03 12:04:27 crc kubenswrapper[5052]: I1203 12:04:27.523179 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/df00a025-ee23-44aa-8970-6f1232146b98-db-sync-config-data\") pod \"df00a025-ee23-44aa-8970-6f1232146b98\" (UID: \"df00a025-ee23-44aa-8970-6f1232146b98\") " Dec 03 12:04:27 crc kubenswrapper[5052]: I1203 12:04:27.531870 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/df00a025-ee23-44aa-8970-6f1232146b98-db-sync-config-data" (OuterVolumeSpecName: "db-sync-config-data") pod "df00a025-ee23-44aa-8970-6f1232146b98" (UID: "df00a025-ee23-44aa-8970-6f1232146b98"). InnerVolumeSpecName "db-sync-config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:04:27 crc kubenswrapper[5052]: I1203 12:04:27.532060 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/df00a025-ee23-44aa-8970-6f1232146b98-kube-api-access-bbwbf" (OuterVolumeSpecName: "kube-api-access-bbwbf") pod "df00a025-ee23-44aa-8970-6f1232146b98" (UID: "df00a025-ee23-44aa-8970-6f1232146b98"). InnerVolumeSpecName "kube-api-access-bbwbf". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:04:27 crc kubenswrapper[5052]: I1203 12:04:27.559266 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/df00a025-ee23-44aa-8970-6f1232146b98-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "df00a025-ee23-44aa-8970-6f1232146b98" (UID: "df00a025-ee23-44aa-8970-6f1232146b98"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:04:27 crc kubenswrapper[5052]: I1203 12:04:27.585611 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/df00a025-ee23-44aa-8970-6f1232146b98-config-data" (OuterVolumeSpecName: "config-data") pod "df00a025-ee23-44aa-8970-6f1232146b98" (UID: "df00a025-ee23-44aa-8970-6f1232146b98"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:04:27 crc kubenswrapper[5052]: I1203 12:04:27.625387 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/df00a025-ee23-44aa-8970-6f1232146b98-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:04:27 crc kubenswrapper[5052]: I1203 12:04:27.625419 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/df00a025-ee23-44aa-8970-6f1232146b98-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:04:27 crc kubenswrapper[5052]: I1203 12:04:27.625432 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-bbwbf\" (UniqueName: \"kubernetes.io/projected/df00a025-ee23-44aa-8970-6f1232146b98-kube-api-access-bbwbf\") on node \"crc\" DevicePath \"\"" Dec 03 12:04:27 crc kubenswrapper[5052]: I1203 12:04:27.625443 5052 reconciler_common.go:293] "Volume detached for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/df00a025-ee23-44aa-8970-6f1232146b98-db-sync-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:04:27 crc kubenswrapper[5052]: I1203 12:04:27.937453 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-db-sync-2lq57" event={"ID":"df00a025-ee23-44aa-8970-6f1232146b98","Type":"ContainerDied","Data":"dd6f9358d32d7d27d99b2bffacd2b349544849910647e13432c8299105871ea4"} Dec 03 12:04:27 crc kubenswrapper[5052]: I1203 12:04:27.937779 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="dd6f9358d32d7d27d99b2bffacd2b349544849910647e13432c8299105871ea4" Dec 03 12:04:27 crc kubenswrapper[5052]: I1203 12:04:27.937539 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-db-sync-2lq57" Dec 03 12:04:28 crc kubenswrapper[5052]: I1203 12:04:28.429380 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-db-sync-8xzwc" Dec 03 12:04:28 crc kubenswrapper[5052]: I1203 12:04:28.458437 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openshift-marketplace/redhat-operators-jjj57" podUID="ede81f70-0e50-47b2-85b7-d30ec858e2e4" containerName="registry-server" probeResult="failure" output=< Dec 03 12:04:28 crc kubenswrapper[5052]: timeout: failed to connect service ":50051" within 1s Dec 03 12:04:28 crc kubenswrapper[5052]: > Dec 03 12:04:28 crc kubenswrapper[5052]: I1203 12:04:28.540660 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/46eb04b8-4d02-4ffe-a0f6-bb4b236a4956-combined-ca-bundle\") pod \"46eb04b8-4d02-4ffe-a0f6-bb4b236a4956\" (UID: \"46eb04b8-4d02-4ffe-a0f6-bb4b236a4956\") " Dec 03 12:04:28 crc kubenswrapper[5052]: I1203 12:04:28.540735 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pz4cb\" (UniqueName: \"kubernetes.io/projected/46eb04b8-4d02-4ffe-a0f6-bb4b236a4956-kube-api-access-pz4cb\") pod \"46eb04b8-4d02-4ffe-a0f6-bb4b236a4956\" (UID: \"46eb04b8-4d02-4ffe-a0f6-bb4b236a4956\") " Dec 03 12:04:28 crc kubenswrapper[5052]: I1203 12:04:28.540840 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/46eb04b8-4d02-4ffe-a0f6-bb4b236a4956-config-data\") pod \"46eb04b8-4d02-4ffe-a0f6-bb4b236a4956\" (UID: \"46eb04b8-4d02-4ffe-a0f6-bb4b236a4956\") " Dec 03 12:04:28 crc kubenswrapper[5052]: I1203 12:04:28.562723 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/46eb04b8-4d02-4ffe-a0f6-bb4b236a4956-kube-api-access-pz4cb" (OuterVolumeSpecName: "kube-api-access-pz4cb") pod "46eb04b8-4d02-4ffe-a0f6-bb4b236a4956" (UID: "46eb04b8-4d02-4ffe-a0f6-bb4b236a4956"). InnerVolumeSpecName "kube-api-access-pz4cb". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:04:28 crc kubenswrapper[5052]: I1203 12:04:28.616438 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/46eb04b8-4d02-4ffe-a0f6-bb4b236a4956-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "46eb04b8-4d02-4ffe-a0f6-bb4b236a4956" (UID: "46eb04b8-4d02-4ffe-a0f6-bb4b236a4956"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:04:28 crc kubenswrapper[5052]: I1203 12:04:28.643021 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-pz4cb\" (UniqueName: \"kubernetes.io/projected/46eb04b8-4d02-4ffe-a0f6-bb4b236a4956-kube-api-access-pz4cb\") on node \"crc\" DevicePath \"\"" Dec 03 12:04:28 crc kubenswrapper[5052]: I1203 12:04:28.643060 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/46eb04b8-4d02-4ffe-a0f6-bb4b236a4956-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:04:28 crc kubenswrapper[5052]: I1203 12:04:28.678338 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/46eb04b8-4d02-4ffe-a0f6-bb4b236a4956-config-data" (OuterVolumeSpecName: "config-data") pod "46eb04b8-4d02-4ffe-a0f6-bb4b236a4956" (UID: "46eb04b8-4d02-4ffe-a0f6-bb4b236a4956"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:04:28 crc kubenswrapper[5052]: I1203 12:04:28.745174 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/46eb04b8-4d02-4ffe-a0f6-bb4b236a4956-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:04:28 crc kubenswrapper[5052]: I1203 12:04:28.970441 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-db-sync-8xzwc" event={"ID":"46eb04b8-4d02-4ffe-a0f6-bb4b236a4956","Type":"ContainerDied","Data":"4f6bae0943404ccbad1eee7c5a65da3bc08d987dc7902f023ddf13bccbb85534"} Dec 03 12:04:28 crc kubenswrapper[5052]: I1203 12:04:28.970532 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4f6bae0943404ccbad1eee7c5a65da3bc08d987dc7902f023ddf13bccbb85534" Dec 03 12:04:28 crc kubenswrapper[5052]: I1203 12:04:28.970681 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-db-sync-8xzwc" Dec 03 12:04:29 crc kubenswrapper[5052]: I1203 12:04:29.131709 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/keystone-bootstrap-wbwms"] Dec 03 12:04:29 crc kubenswrapper[5052]: E1203 12:04:29.132084 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="90d21e8c-a52c-4109-811b-7d113748cea7" containerName="mariadb-account-create-update" Dec 03 12:04:29 crc kubenswrapper[5052]: I1203 12:04:29.132098 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="90d21e8c-a52c-4109-811b-7d113748cea7" containerName="mariadb-account-create-update" Dec 03 12:04:29 crc kubenswrapper[5052]: E1203 12:04:29.132111 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="df00a025-ee23-44aa-8970-6f1232146b98" containerName="glance-db-sync" Dec 03 12:04:29 crc kubenswrapper[5052]: I1203 12:04:29.132116 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="df00a025-ee23-44aa-8970-6f1232146b98" containerName="glance-db-sync" Dec 03 12:04:29 crc kubenswrapper[5052]: E1203 12:04:29.132130 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="67b28447-93fa-45c3-8c94-9dba5eba8f94" containerName="mariadb-database-create" Dec 03 12:04:29 crc kubenswrapper[5052]: I1203 12:04:29.132136 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="67b28447-93fa-45c3-8c94-9dba5eba8f94" containerName="mariadb-database-create" Dec 03 12:04:29 crc kubenswrapper[5052]: E1203 12:04:29.132151 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="8ba21481-e9f8-41f3-80e6-c96f860b7771" containerName="mariadb-database-create" Dec 03 12:04:29 crc kubenswrapper[5052]: I1203 12:04:29.132157 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="8ba21481-e9f8-41f3-80e6-c96f860b7771" containerName="mariadb-database-create" Dec 03 12:04:29 crc kubenswrapper[5052]: E1203 12:04:29.132163 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="59c3e3b8-e2a6-4384-87f0-d1d7cbb4d738" containerName="mariadb-account-create-update" Dec 03 12:04:29 crc kubenswrapper[5052]: I1203 12:04:29.132168 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="59c3e3b8-e2a6-4384-87f0-d1d7cbb4d738" containerName="mariadb-account-create-update" Dec 03 12:04:29 crc kubenswrapper[5052]: E1203 12:04:29.132178 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="2b7332a9-078d-4457-b136-899c4cdfaaae" containerName="mariadb-database-create" Dec 03 12:04:29 crc kubenswrapper[5052]: I1203 12:04:29.132184 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="2b7332a9-078d-4457-b136-899c4cdfaaae" containerName="mariadb-database-create" Dec 03 12:04:29 crc kubenswrapper[5052]: E1203 12:04:29.132196 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="77f55a4f-32ec-4c57-82bc-ff0dd0aac965" containerName="mariadb-account-create-update" Dec 03 12:04:29 crc kubenswrapper[5052]: I1203 12:04:29.132202 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="77f55a4f-32ec-4c57-82bc-ff0dd0aac965" containerName="mariadb-account-create-update" Dec 03 12:04:29 crc kubenswrapper[5052]: E1203 12:04:29.132209 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="46eb04b8-4d02-4ffe-a0f6-bb4b236a4956" containerName="keystone-db-sync" Dec 03 12:04:29 crc kubenswrapper[5052]: I1203 12:04:29.132214 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="46eb04b8-4d02-4ffe-a0f6-bb4b236a4956" containerName="keystone-db-sync" Dec 03 12:04:29 crc kubenswrapper[5052]: I1203 12:04:29.136046 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="46eb04b8-4d02-4ffe-a0f6-bb4b236a4956" containerName="keystone-db-sync" Dec 03 12:04:29 crc kubenswrapper[5052]: I1203 12:04:29.136090 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="59c3e3b8-e2a6-4384-87f0-d1d7cbb4d738" containerName="mariadb-account-create-update" Dec 03 12:04:29 crc kubenswrapper[5052]: I1203 12:04:29.136111 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="8ba21481-e9f8-41f3-80e6-c96f860b7771" containerName="mariadb-database-create" Dec 03 12:04:29 crc kubenswrapper[5052]: I1203 12:04:29.136125 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="67b28447-93fa-45c3-8c94-9dba5eba8f94" containerName="mariadb-database-create" Dec 03 12:04:29 crc kubenswrapper[5052]: I1203 12:04:29.136146 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="2b7332a9-078d-4457-b136-899c4cdfaaae" containerName="mariadb-database-create" Dec 03 12:04:29 crc kubenswrapper[5052]: I1203 12:04:29.136159 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="77f55a4f-32ec-4c57-82bc-ff0dd0aac965" containerName="mariadb-account-create-update" Dec 03 12:04:29 crc kubenswrapper[5052]: I1203 12:04:29.136186 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="90d21e8c-a52c-4109-811b-7d113748cea7" containerName="mariadb-account-create-update" Dec 03 12:04:29 crc kubenswrapper[5052]: I1203 12:04:29.136203 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="df00a025-ee23-44aa-8970-6f1232146b98" containerName="glance-db-sync" Dec 03 12:04:29 crc kubenswrapper[5052]: I1203 12:04:29.136921 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-bootstrap-wbwms" Dec 03 12:04:29 crc kubenswrapper[5052]: I1203 12:04:29.141639 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone-scripts" Dec 03 12:04:29 crc kubenswrapper[5052]: I1203 12:04:29.142016 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone" Dec 03 12:04:29 crc kubenswrapper[5052]: I1203 12:04:29.142212 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone-config-data" Dec 03 12:04:29 crc kubenswrapper[5052]: I1203 12:04:29.142330 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"osp-secret" Dec 03 12:04:29 crc kubenswrapper[5052]: I1203 12:04:29.143598 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone-keystone-dockercfg-m92zx" Dec 03 12:04:29 crc kubenswrapper[5052]: I1203 12:04:29.152306 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-bootstrap-wbwms"] Dec 03 12:04:29 crc kubenswrapper[5052]: I1203 12:04:29.278936 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3d2ba247-e370-4d62-a34e-5b05174363fb-combined-ca-bundle\") pod \"keystone-bootstrap-wbwms\" (UID: \"3d2ba247-e370-4d62-a34e-5b05174363fb\") " pod="openstack-kuttl-tests/keystone-bootstrap-wbwms" Dec 03 12:04:29 crc kubenswrapper[5052]: I1203 12:04:29.278985 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/3d2ba247-e370-4d62-a34e-5b05174363fb-config-data\") pod \"keystone-bootstrap-wbwms\" (UID: \"3d2ba247-e370-4d62-a34e-5b05174363fb\") " pod="openstack-kuttl-tests/keystone-bootstrap-wbwms" Dec 03 12:04:29 crc kubenswrapper[5052]: I1203 12:04:29.279028 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bvbl6\" (UniqueName: \"kubernetes.io/projected/3d2ba247-e370-4d62-a34e-5b05174363fb-kube-api-access-bvbl6\") pod \"keystone-bootstrap-wbwms\" (UID: \"3d2ba247-e370-4d62-a34e-5b05174363fb\") " pod="openstack-kuttl-tests/keystone-bootstrap-wbwms" Dec 03 12:04:29 crc kubenswrapper[5052]: I1203 12:04:29.279056 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/3d2ba247-e370-4d62-a34e-5b05174363fb-scripts\") pod \"keystone-bootstrap-wbwms\" (UID: \"3d2ba247-e370-4d62-a34e-5b05174363fb\") " pod="openstack-kuttl-tests/keystone-bootstrap-wbwms" Dec 03 12:04:29 crc kubenswrapper[5052]: I1203 12:04:29.279174 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/3d2ba247-e370-4d62-a34e-5b05174363fb-fernet-keys\") pod \"keystone-bootstrap-wbwms\" (UID: \"3d2ba247-e370-4d62-a34e-5b05174363fb\") " pod="openstack-kuttl-tests/keystone-bootstrap-wbwms" Dec 03 12:04:29 crc kubenswrapper[5052]: I1203 12:04:29.279200 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/3d2ba247-e370-4d62-a34e-5b05174363fb-credential-keys\") pod \"keystone-bootstrap-wbwms\" (UID: \"3d2ba247-e370-4d62-a34e-5b05174363fb\") " pod="openstack-kuttl-tests/keystone-bootstrap-wbwms" Dec 03 12:04:29 crc kubenswrapper[5052]: I1203 12:04:29.380488 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/3d2ba247-e370-4d62-a34e-5b05174363fb-fernet-keys\") pod \"keystone-bootstrap-wbwms\" (UID: \"3d2ba247-e370-4d62-a34e-5b05174363fb\") " pod="openstack-kuttl-tests/keystone-bootstrap-wbwms" Dec 03 12:04:29 crc kubenswrapper[5052]: I1203 12:04:29.380550 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/3d2ba247-e370-4d62-a34e-5b05174363fb-credential-keys\") pod \"keystone-bootstrap-wbwms\" (UID: \"3d2ba247-e370-4d62-a34e-5b05174363fb\") " pod="openstack-kuttl-tests/keystone-bootstrap-wbwms" Dec 03 12:04:29 crc kubenswrapper[5052]: I1203 12:04:29.380603 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3d2ba247-e370-4d62-a34e-5b05174363fb-combined-ca-bundle\") pod \"keystone-bootstrap-wbwms\" (UID: \"3d2ba247-e370-4d62-a34e-5b05174363fb\") " pod="openstack-kuttl-tests/keystone-bootstrap-wbwms" Dec 03 12:04:29 crc kubenswrapper[5052]: I1203 12:04:29.380622 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/3d2ba247-e370-4d62-a34e-5b05174363fb-config-data\") pod \"keystone-bootstrap-wbwms\" (UID: \"3d2ba247-e370-4d62-a34e-5b05174363fb\") " pod="openstack-kuttl-tests/keystone-bootstrap-wbwms" Dec 03 12:04:29 crc kubenswrapper[5052]: I1203 12:04:29.380660 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-bvbl6\" (UniqueName: \"kubernetes.io/projected/3d2ba247-e370-4d62-a34e-5b05174363fb-kube-api-access-bvbl6\") pod \"keystone-bootstrap-wbwms\" (UID: \"3d2ba247-e370-4d62-a34e-5b05174363fb\") " pod="openstack-kuttl-tests/keystone-bootstrap-wbwms" Dec 03 12:04:29 crc kubenswrapper[5052]: I1203 12:04:29.380689 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/3d2ba247-e370-4d62-a34e-5b05174363fb-scripts\") pod \"keystone-bootstrap-wbwms\" (UID: \"3d2ba247-e370-4d62-a34e-5b05174363fb\") " pod="openstack-kuttl-tests/keystone-bootstrap-wbwms" Dec 03 12:04:29 crc kubenswrapper[5052]: I1203 12:04:29.394239 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/3d2ba247-e370-4d62-a34e-5b05174363fb-credential-keys\") pod \"keystone-bootstrap-wbwms\" (UID: \"3d2ba247-e370-4d62-a34e-5b05174363fb\") " pod="openstack-kuttl-tests/keystone-bootstrap-wbwms" Dec 03 12:04:29 crc kubenswrapper[5052]: I1203 12:04:29.394562 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/3d2ba247-e370-4d62-a34e-5b05174363fb-config-data\") pod \"keystone-bootstrap-wbwms\" (UID: \"3d2ba247-e370-4d62-a34e-5b05174363fb\") " pod="openstack-kuttl-tests/keystone-bootstrap-wbwms" Dec 03 12:04:29 crc kubenswrapper[5052]: I1203 12:04:29.403180 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/3d2ba247-e370-4d62-a34e-5b05174363fb-scripts\") pod \"keystone-bootstrap-wbwms\" (UID: \"3d2ba247-e370-4d62-a34e-5b05174363fb\") " pod="openstack-kuttl-tests/keystone-bootstrap-wbwms" Dec 03 12:04:29 crc kubenswrapper[5052]: I1203 12:04:29.410718 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/3d2ba247-e370-4d62-a34e-5b05174363fb-fernet-keys\") pod \"keystone-bootstrap-wbwms\" (UID: \"3d2ba247-e370-4d62-a34e-5b05174363fb\") " pod="openstack-kuttl-tests/keystone-bootstrap-wbwms" Dec 03 12:04:29 crc kubenswrapper[5052]: I1203 12:04:29.419553 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3d2ba247-e370-4d62-a34e-5b05174363fb-combined-ca-bundle\") pod \"keystone-bootstrap-wbwms\" (UID: \"3d2ba247-e370-4d62-a34e-5b05174363fb\") " pod="openstack-kuttl-tests/keystone-bootstrap-wbwms" Dec 03 12:04:29 crc kubenswrapper[5052]: I1203 12:04:29.428150 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-bvbl6\" (UniqueName: \"kubernetes.io/projected/3d2ba247-e370-4d62-a34e-5b05174363fb-kube-api-access-bvbl6\") pod \"keystone-bootstrap-wbwms\" (UID: \"3d2ba247-e370-4d62-a34e-5b05174363fb\") " pod="openstack-kuttl-tests/keystone-bootstrap-wbwms" Dec 03 12:04:29 crc kubenswrapper[5052]: I1203 12:04:29.453754 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/neutron-db-sync-lrrtz"] Dec 03 12:04:29 crc kubenswrapper[5052]: I1203 12:04:29.455403 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-db-sync-lrrtz" Dec 03 12:04:29 crc kubenswrapper[5052]: I1203 12:04:29.458979 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"neutron-httpd-config" Dec 03 12:04:29 crc kubenswrapper[5052]: I1203 12:04:29.459194 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"neutron-config" Dec 03 12:04:29 crc kubenswrapper[5052]: I1203 12:04:29.459438 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"neutron-neutron-dockercfg-6l9fz" Dec 03 12:04:29 crc kubenswrapper[5052]: I1203 12:04:29.464989 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-bootstrap-wbwms" Dec 03 12:04:29 crc kubenswrapper[5052]: I1203 12:04:29.501039 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:04:29 crc kubenswrapper[5052]: I1203 12:04:29.503789 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:04:29 crc kubenswrapper[5052]: I1203 12:04:29.544103 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-config-data" Dec 03 12:04:29 crc kubenswrapper[5052]: I1203 12:04:29.544492 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-scripts" Dec 03 12:04:29 crc kubenswrapper[5052]: I1203 12:04:29.574136 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/placement-db-sync-852kv"] Dec 03 12:04:29 crc kubenswrapper[5052]: I1203 12:04:29.576146 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-db-sync-852kv" Dec 03 12:04:29 crc kubenswrapper[5052]: I1203 12:04:29.581989 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-db-sync-lrrtz"] Dec 03 12:04:29 crc kubenswrapper[5052]: I1203 12:04:29.588211 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"placement-config-data" Dec 03 12:04:29 crc kubenswrapper[5052]: I1203 12:04:29.588248 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"placement-placement-dockercfg-rmdxz" Dec 03 12:04:29 crc kubenswrapper[5052]: I1203 12:04:29.589298 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"placement-scripts" Dec 03 12:04:29 crc kubenswrapper[5052]: I1203 12:04:29.589970 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d9b7a317-ab7f-4c6a-8f91-b66b409c8980-combined-ca-bundle\") pod \"neutron-db-sync-lrrtz\" (UID: \"d9b7a317-ab7f-4c6a-8f91-b66b409c8980\") " pod="openstack-kuttl-tests/neutron-db-sync-lrrtz" Dec 03 12:04:29 crc kubenswrapper[5052]: I1203 12:04:29.590070 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r5cw2\" (UniqueName: \"kubernetes.io/projected/d9b7a317-ab7f-4c6a-8f91-b66b409c8980-kube-api-access-r5cw2\") pod \"neutron-db-sync-lrrtz\" (UID: \"d9b7a317-ab7f-4c6a-8f91-b66b409c8980\") " pod="openstack-kuttl-tests/neutron-db-sync-lrrtz" Dec 03 12:04:29 crc kubenswrapper[5052]: I1203 12:04:29.590123 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/d9b7a317-ab7f-4c6a-8f91-b66b409c8980-config\") pod \"neutron-db-sync-lrrtz\" (UID: \"d9b7a317-ab7f-4c6a-8f91-b66b409c8980\") " pod="openstack-kuttl-tests/neutron-db-sync-lrrtz" Dec 03 12:04:29 crc kubenswrapper[5052]: I1203 12:04:29.591865 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:04:29 crc kubenswrapper[5052]: I1203 12:04:29.617537 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/cinder-db-sync-jhtdm"] Dec 03 12:04:29 crc kubenswrapper[5052]: I1203 12:04:29.619879 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-db-sync-jhtdm" Dec 03 12:04:29 crc kubenswrapper[5052]: I1203 12:04:29.622130 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cinder-config-data" Dec 03 12:04:29 crc kubenswrapper[5052]: I1203 12:04:29.622526 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cinder-cinder-dockercfg-qpb27" Dec 03 12:04:29 crc kubenswrapper[5052]: I1203 12:04:29.623552 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cinder-scripts" Dec 03 12:04:29 crc kubenswrapper[5052]: I1203 12:04:29.646648 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-db-sync-jhtdm"] Dec 03 12:04:29 crc kubenswrapper[5052]: I1203 12:04:29.664630 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/placement-db-sync-852kv"] Dec 03 12:04:29 crc kubenswrapper[5052]: I1203 12:04:29.692123 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/ab039824-fe2a-44ba-ae0f-302e5c4ea614-scripts\") pod \"placement-db-sync-852kv\" (UID: \"ab039824-fe2a-44ba-ae0f-302e5c4ea614\") " pod="openstack-kuttl-tests/placement-db-sync-852kv" Dec 03 12:04:29 crc kubenswrapper[5052]: I1203 12:04:29.692161 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/356db4d6-f512-4a47-a6ed-a288a212227d-scripts\") pod \"ceilometer-0\" (UID: \"356db4d6-f512-4a47-a6ed-a288a212227d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:04:29 crc kubenswrapper[5052]: I1203 12:04:29.692181 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ab039824-fe2a-44ba-ae0f-302e5c4ea614-combined-ca-bundle\") pod \"placement-db-sync-852kv\" (UID: \"ab039824-fe2a-44ba-ae0f-302e5c4ea614\") " pod="openstack-kuttl-tests/placement-db-sync-852kv" Dec 03 12:04:29 crc kubenswrapper[5052]: I1203 12:04:29.692210 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-r5cw2\" (UniqueName: \"kubernetes.io/projected/d9b7a317-ab7f-4c6a-8f91-b66b409c8980-kube-api-access-r5cw2\") pod \"neutron-db-sync-lrrtz\" (UID: \"d9b7a317-ab7f-4c6a-8f91-b66b409c8980\") " pod="openstack-kuttl-tests/neutron-db-sync-lrrtz" Dec 03 12:04:29 crc kubenswrapper[5052]: I1203 12:04:29.692239 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/ab039824-fe2a-44ba-ae0f-302e5c4ea614-logs\") pod \"placement-db-sync-852kv\" (UID: \"ab039824-fe2a-44ba-ae0f-302e5c4ea614\") " pod="openstack-kuttl-tests/placement-db-sync-852kv" Dec 03 12:04:29 crc kubenswrapper[5052]: I1203 12:04:29.692272 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/356db4d6-f512-4a47-a6ed-a288a212227d-config-data\") pod \"ceilometer-0\" (UID: \"356db4d6-f512-4a47-a6ed-a288a212227d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:04:29 crc kubenswrapper[5052]: I1203 12:04:29.692290 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pd5ht\" (UniqueName: \"kubernetes.io/projected/356db4d6-f512-4a47-a6ed-a288a212227d-kube-api-access-pd5ht\") pod \"ceilometer-0\" (UID: \"356db4d6-f512-4a47-a6ed-a288a212227d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:04:29 crc kubenswrapper[5052]: I1203 12:04:29.692310 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/d9b7a317-ab7f-4c6a-8f91-b66b409c8980-config\") pod \"neutron-db-sync-lrrtz\" (UID: \"d9b7a317-ab7f-4c6a-8f91-b66b409c8980\") " pod="openstack-kuttl-tests/neutron-db-sync-lrrtz" Dec 03 12:04:29 crc kubenswrapper[5052]: I1203 12:04:29.692326 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/356db4d6-f512-4a47-a6ed-a288a212227d-log-httpd\") pod \"ceilometer-0\" (UID: \"356db4d6-f512-4a47-a6ed-a288a212227d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:04:29 crc kubenswrapper[5052]: I1203 12:04:29.692342 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/356db4d6-f512-4a47-a6ed-a288a212227d-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"356db4d6-f512-4a47-a6ed-a288a212227d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:04:29 crc kubenswrapper[5052]: I1203 12:04:29.692361 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/356db4d6-f512-4a47-a6ed-a288a212227d-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"356db4d6-f512-4a47-a6ed-a288a212227d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:04:29 crc kubenswrapper[5052]: I1203 12:04:29.692392 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/ab039824-fe2a-44ba-ae0f-302e5c4ea614-config-data\") pod \"placement-db-sync-852kv\" (UID: \"ab039824-fe2a-44ba-ae0f-302e5c4ea614\") " pod="openstack-kuttl-tests/placement-db-sync-852kv" Dec 03 12:04:29 crc kubenswrapper[5052]: I1203 12:04:29.692411 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/356db4d6-f512-4a47-a6ed-a288a212227d-run-httpd\") pod \"ceilometer-0\" (UID: \"356db4d6-f512-4a47-a6ed-a288a212227d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:04:29 crc kubenswrapper[5052]: I1203 12:04:29.692433 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d9b7a317-ab7f-4c6a-8f91-b66b409c8980-combined-ca-bundle\") pod \"neutron-db-sync-lrrtz\" (UID: \"d9b7a317-ab7f-4c6a-8f91-b66b409c8980\") " pod="openstack-kuttl-tests/neutron-db-sync-lrrtz" Dec 03 12:04:29 crc kubenswrapper[5052]: I1203 12:04:29.692471 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kf6cd\" (UniqueName: \"kubernetes.io/projected/ab039824-fe2a-44ba-ae0f-302e5c4ea614-kube-api-access-kf6cd\") pod \"placement-db-sync-852kv\" (UID: \"ab039824-fe2a-44ba-ae0f-302e5c4ea614\") " pod="openstack-kuttl-tests/placement-db-sync-852kv" Dec 03 12:04:29 crc kubenswrapper[5052]: I1203 12:04:29.707540 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d9b7a317-ab7f-4c6a-8f91-b66b409c8980-combined-ca-bundle\") pod \"neutron-db-sync-lrrtz\" (UID: \"d9b7a317-ab7f-4c6a-8f91-b66b409c8980\") " pod="openstack-kuttl-tests/neutron-db-sync-lrrtz" Dec 03 12:04:29 crc kubenswrapper[5052]: I1203 12:04:29.710297 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/d9b7a317-ab7f-4c6a-8f91-b66b409c8980-config\") pod \"neutron-db-sync-lrrtz\" (UID: \"d9b7a317-ab7f-4c6a-8f91-b66b409c8980\") " pod="openstack-kuttl-tests/neutron-db-sync-lrrtz" Dec 03 12:04:29 crc kubenswrapper[5052]: I1203 12:04:29.712741 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/barbican-db-sync-4z2hl"] Dec 03 12:04:29 crc kubenswrapper[5052]: I1203 12:04:29.714024 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-r5cw2\" (UniqueName: \"kubernetes.io/projected/d9b7a317-ab7f-4c6a-8f91-b66b409c8980-kube-api-access-r5cw2\") pod \"neutron-db-sync-lrrtz\" (UID: \"d9b7a317-ab7f-4c6a-8f91-b66b409c8980\") " pod="openstack-kuttl-tests/neutron-db-sync-lrrtz" Dec 03 12:04:29 crc kubenswrapper[5052]: I1203 12:04:29.714133 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-db-sync-4z2hl" Dec 03 12:04:29 crc kubenswrapper[5052]: I1203 12:04:29.714831 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-db-sync-4z2hl"] Dec 03 12:04:29 crc kubenswrapper[5052]: I1203 12:04:29.717911 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"barbican-config-data" Dec 03 12:04:29 crc kubenswrapper[5052]: I1203 12:04:29.718226 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"barbican-barbican-dockercfg-7l5nk" Dec 03 12:04:29 crc kubenswrapper[5052]: I1203 12:04:29.794387 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/356db4d6-f512-4a47-a6ed-a288a212227d-run-httpd\") pod \"ceilometer-0\" (UID: \"356db4d6-f512-4a47-a6ed-a288a212227d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:04:29 crc kubenswrapper[5052]: I1203 12:04:29.795027 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/0c8be033-f83e-4d64-b271-f4c97bfdc108-scripts\") pod \"cinder-db-sync-jhtdm\" (UID: \"0c8be033-f83e-4d64-b271-f4c97bfdc108\") " pod="openstack-kuttl-tests/cinder-db-sync-jhtdm" Dec 03 12:04:29 crc kubenswrapper[5052]: I1203 12:04:29.795096 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-kf6cd\" (UniqueName: \"kubernetes.io/projected/ab039824-fe2a-44ba-ae0f-302e5c4ea614-kube-api-access-kf6cd\") pod \"placement-db-sync-852kv\" (UID: \"ab039824-fe2a-44ba-ae0f-302e5c4ea614\") " pod="openstack-kuttl-tests/placement-db-sync-852kv" Dec 03 12:04:29 crc kubenswrapper[5052]: I1203 12:04:29.795104 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/356db4d6-f512-4a47-a6ed-a288a212227d-run-httpd\") pod \"ceilometer-0\" (UID: \"356db4d6-f512-4a47-a6ed-a288a212227d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:04:29 crc kubenswrapper[5052]: I1203 12:04:29.795167 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/0c8be033-f83e-4d64-b271-f4c97bfdc108-config-data\") pod \"cinder-db-sync-jhtdm\" (UID: \"0c8be033-f83e-4d64-b271-f4c97bfdc108\") " pod="openstack-kuttl-tests/cinder-db-sync-jhtdm" Dec 03 12:04:29 crc kubenswrapper[5052]: I1203 12:04:29.795198 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0c8be033-f83e-4d64-b271-f4c97bfdc108-combined-ca-bundle\") pod \"cinder-db-sync-jhtdm\" (UID: \"0c8be033-f83e-4d64-b271-f4c97bfdc108\") " pod="openstack-kuttl-tests/cinder-db-sync-jhtdm" Dec 03 12:04:29 crc kubenswrapper[5052]: I1203 12:04:29.795234 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/0c8be033-f83e-4d64-b271-f4c97bfdc108-db-sync-config-data\") pod \"cinder-db-sync-jhtdm\" (UID: \"0c8be033-f83e-4d64-b271-f4c97bfdc108\") " pod="openstack-kuttl-tests/cinder-db-sync-jhtdm" Dec 03 12:04:29 crc kubenswrapper[5052]: I1203 12:04:29.795261 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/ab039824-fe2a-44ba-ae0f-302e5c4ea614-scripts\") pod \"placement-db-sync-852kv\" (UID: \"ab039824-fe2a-44ba-ae0f-302e5c4ea614\") " pod="openstack-kuttl-tests/placement-db-sync-852kv" Dec 03 12:04:29 crc kubenswrapper[5052]: I1203 12:04:29.795284 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/356db4d6-f512-4a47-a6ed-a288a212227d-scripts\") pod \"ceilometer-0\" (UID: \"356db4d6-f512-4a47-a6ed-a288a212227d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:04:29 crc kubenswrapper[5052]: I1203 12:04:29.795307 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ab039824-fe2a-44ba-ae0f-302e5c4ea614-combined-ca-bundle\") pod \"placement-db-sync-852kv\" (UID: \"ab039824-fe2a-44ba-ae0f-302e5c4ea614\") " pod="openstack-kuttl-tests/placement-db-sync-852kv" Dec 03 12:04:29 crc kubenswrapper[5052]: I1203 12:04:29.795362 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/0c8be033-f83e-4d64-b271-f4c97bfdc108-etc-machine-id\") pod \"cinder-db-sync-jhtdm\" (UID: \"0c8be033-f83e-4d64-b271-f4c97bfdc108\") " pod="openstack-kuttl-tests/cinder-db-sync-jhtdm" Dec 03 12:04:29 crc kubenswrapper[5052]: I1203 12:04:29.795395 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/ab039824-fe2a-44ba-ae0f-302e5c4ea614-logs\") pod \"placement-db-sync-852kv\" (UID: \"ab039824-fe2a-44ba-ae0f-302e5c4ea614\") " pod="openstack-kuttl-tests/placement-db-sync-852kv" Dec 03 12:04:29 crc kubenswrapper[5052]: I1203 12:04:29.795421 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-57tzg\" (UniqueName: \"kubernetes.io/projected/0c8be033-f83e-4d64-b271-f4c97bfdc108-kube-api-access-57tzg\") pod \"cinder-db-sync-jhtdm\" (UID: \"0c8be033-f83e-4d64-b271-f4c97bfdc108\") " pod="openstack-kuttl-tests/cinder-db-sync-jhtdm" Dec 03 12:04:29 crc kubenswrapper[5052]: I1203 12:04:29.795451 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/356db4d6-f512-4a47-a6ed-a288a212227d-config-data\") pod \"ceilometer-0\" (UID: \"356db4d6-f512-4a47-a6ed-a288a212227d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:04:29 crc kubenswrapper[5052]: I1203 12:04:29.795469 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-pd5ht\" (UniqueName: \"kubernetes.io/projected/356db4d6-f512-4a47-a6ed-a288a212227d-kube-api-access-pd5ht\") pod \"ceilometer-0\" (UID: \"356db4d6-f512-4a47-a6ed-a288a212227d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:04:29 crc kubenswrapper[5052]: I1203 12:04:29.795499 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/356db4d6-f512-4a47-a6ed-a288a212227d-log-httpd\") pod \"ceilometer-0\" (UID: \"356db4d6-f512-4a47-a6ed-a288a212227d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:04:29 crc kubenswrapper[5052]: I1203 12:04:29.795533 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/356db4d6-f512-4a47-a6ed-a288a212227d-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"356db4d6-f512-4a47-a6ed-a288a212227d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:04:29 crc kubenswrapper[5052]: I1203 12:04:29.795552 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/356db4d6-f512-4a47-a6ed-a288a212227d-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"356db4d6-f512-4a47-a6ed-a288a212227d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:04:29 crc kubenswrapper[5052]: I1203 12:04:29.795751 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/ab039824-fe2a-44ba-ae0f-302e5c4ea614-config-data\") pod \"placement-db-sync-852kv\" (UID: \"ab039824-fe2a-44ba-ae0f-302e5c4ea614\") " pod="openstack-kuttl-tests/placement-db-sync-852kv" Dec 03 12:04:29 crc kubenswrapper[5052]: I1203 12:04:29.797919 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/ab039824-fe2a-44ba-ae0f-302e5c4ea614-logs\") pod \"placement-db-sync-852kv\" (UID: \"ab039824-fe2a-44ba-ae0f-302e5c4ea614\") " pod="openstack-kuttl-tests/placement-db-sync-852kv" Dec 03 12:04:29 crc kubenswrapper[5052]: I1203 12:04:29.798984 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/356db4d6-f512-4a47-a6ed-a288a212227d-log-httpd\") pod \"ceilometer-0\" (UID: \"356db4d6-f512-4a47-a6ed-a288a212227d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:04:29 crc kubenswrapper[5052]: I1203 12:04:29.801337 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ab039824-fe2a-44ba-ae0f-302e5c4ea614-combined-ca-bundle\") pod \"placement-db-sync-852kv\" (UID: \"ab039824-fe2a-44ba-ae0f-302e5c4ea614\") " pod="openstack-kuttl-tests/placement-db-sync-852kv" Dec 03 12:04:29 crc kubenswrapper[5052]: I1203 12:04:29.803458 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/356db4d6-f512-4a47-a6ed-a288a212227d-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"356db4d6-f512-4a47-a6ed-a288a212227d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:04:29 crc kubenswrapper[5052]: I1203 12:04:29.803903 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/356db4d6-f512-4a47-a6ed-a288a212227d-config-data\") pod \"ceilometer-0\" (UID: \"356db4d6-f512-4a47-a6ed-a288a212227d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:04:29 crc kubenswrapper[5052]: I1203 12:04:29.805223 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/356db4d6-f512-4a47-a6ed-a288a212227d-scripts\") pod \"ceilometer-0\" (UID: \"356db4d6-f512-4a47-a6ed-a288a212227d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:04:29 crc kubenswrapper[5052]: I1203 12:04:29.805824 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/ab039824-fe2a-44ba-ae0f-302e5c4ea614-scripts\") pod \"placement-db-sync-852kv\" (UID: \"ab039824-fe2a-44ba-ae0f-302e5c4ea614\") " pod="openstack-kuttl-tests/placement-db-sync-852kv" Dec 03 12:04:29 crc kubenswrapper[5052]: I1203 12:04:29.814280 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/ab039824-fe2a-44ba-ae0f-302e5c4ea614-config-data\") pod \"placement-db-sync-852kv\" (UID: \"ab039824-fe2a-44ba-ae0f-302e5c4ea614\") " pod="openstack-kuttl-tests/placement-db-sync-852kv" Dec 03 12:04:29 crc kubenswrapper[5052]: I1203 12:04:29.815991 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/356db4d6-f512-4a47-a6ed-a288a212227d-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"356db4d6-f512-4a47-a6ed-a288a212227d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:04:29 crc kubenswrapper[5052]: I1203 12:04:29.819833 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-kf6cd\" (UniqueName: \"kubernetes.io/projected/ab039824-fe2a-44ba-ae0f-302e5c4ea614-kube-api-access-kf6cd\") pod \"placement-db-sync-852kv\" (UID: \"ab039824-fe2a-44ba-ae0f-302e5c4ea614\") " pod="openstack-kuttl-tests/placement-db-sync-852kv" Dec 03 12:04:29 crc kubenswrapper[5052]: I1203 12:04:29.820186 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-pd5ht\" (UniqueName: \"kubernetes.io/projected/356db4d6-f512-4a47-a6ed-a288a212227d-kube-api-access-pd5ht\") pod \"ceilometer-0\" (UID: \"356db4d6-f512-4a47-a6ed-a288a212227d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:04:29 crc kubenswrapper[5052]: I1203 12:04:29.897179 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/0c8be033-f83e-4d64-b271-f4c97bfdc108-etc-machine-id\") pod \"cinder-db-sync-jhtdm\" (UID: \"0c8be033-f83e-4d64-b271-f4c97bfdc108\") " pod="openstack-kuttl-tests/cinder-db-sync-jhtdm" Dec 03 12:04:29 crc kubenswrapper[5052]: I1203 12:04:29.897228 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-57tzg\" (UniqueName: \"kubernetes.io/projected/0c8be033-f83e-4d64-b271-f4c97bfdc108-kube-api-access-57tzg\") pod \"cinder-db-sync-jhtdm\" (UID: \"0c8be033-f83e-4d64-b271-f4c97bfdc108\") " pod="openstack-kuttl-tests/cinder-db-sync-jhtdm" Dec 03 12:04:29 crc kubenswrapper[5052]: I1203 12:04:29.897262 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/f3c81a5a-a6cc-406f-95b9-186ae52a3ccb-db-sync-config-data\") pod \"barbican-db-sync-4z2hl\" (UID: \"f3c81a5a-a6cc-406f-95b9-186ae52a3ccb\") " pod="openstack-kuttl-tests/barbican-db-sync-4z2hl" Dec 03 12:04:29 crc kubenswrapper[5052]: I1203 12:04:29.897312 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f3c81a5a-a6cc-406f-95b9-186ae52a3ccb-combined-ca-bundle\") pod \"barbican-db-sync-4z2hl\" (UID: \"f3c81a5a-a6cc-406f-95b9-186ae52a3ccb\") " pod="openstack-kuttl-tests/barbican-db-sync-4z2hl" Dec 03 12:04:29 crc kubenswrapper[5052]: I1203 12:04:29.897342 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/0c8be033-f83e-4d64-b271-f4c97bfdc108-scripts\") pod \"cinder-db-sync-jhtdm\" (UID: \"0c8be033-f83e-4d64-b271-f4c97bfdc108\") " pod="openstack-kuttl-tests/cinder-db-sync-jhtdm" Dec 03 12:04:29 crc kubenswrapper[5052]: I1203 12:04:29.897368 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hxq27\" (UniqueName: \"kubernetes.io/projected/f3c81a5a-a6cc-406f-95b9-186ae52a3ccb-kube-api-access-hxq27\") pod \"barbican-db-sync-4z2hl\" (UID: \"f3c81a5a-a6cc-406f-95b9-186ae52a3ccb\") " pod="openstack-kuttl-tests/barbican-db-sync-4z2hl" Dec 03 12:04:29 crc kubenswrapper[5052]: I1203 12:04:29.897404 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/0c8be033-f83e-4d64-b271-f4c97bfdc108-config-data\") pod \"cinder-db-sync-jhtdm\" (UID: \"0c8be033-f83e-4d64-b271-f4c97bfdc108\") " pod="openstack-kuttl-tests/cinder-db-sync-jhtdm" Dec 03 12:04:29 crc kubenswrapper[5052]: I1203 12:04:29.897426 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0c8be033-f83e-4d64-b271-f4c97bfdc108-combined-ca-bundle\") pod \"cinder-db-sync-jhtdm\" (UID: \"0c8be033-f83e-4d64-b271-f4c97bfdc108\") " pod="openstack-kuttl-tests/cinder-db-sync-jhtdm" Dec 03 12:04:29 crc kubenswrapper[5052]: I1203 12:04:29.897455 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/0c8be033-f83e-4d64-b271-f4c97bfdc108-db-sync-config-data\") pod \"cinder-db-sync-jhtdm\" (UID: \"0c8be033-f83e-4d64-b271-f4c97bfdc108\") " pod="openstack-kuttl-tests/cinder-db-sync-jhtdm" Dec 03 12:04:29 crc kubenswrapper[5052]: I1203 12:04:29.898745 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/0c8be033-f83e-4d64-b271-f4c97bfdc108-etc-machine-id\") pod \"cinder-db-sync-jhtdm\" (UID: \"0c8be033-f83e-4d64-b271-f4c97bfdc108\") " pod="openstack-kuttl-tests/cinder-db-sync-jhtdm" Dec 03 12:04:29 crc kubenswrapper[5052]: I1203 12:04:29.901762 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/0c8be033-f83e-4d64-b271-f4c97bfdc108-config-data\") pod \"cinder-db-sync-jhtdm\" (UID: \"0c8be033-f83e-4d64-b271-f4c97bfdc108\") " pod="openstack-kuttl-tests/cinder-db-sync-jhtdm" Dec 03 12:04:29 crc kubenswrapper[5052]: I1203 12:04:29.901942 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/0c8be033-f83e-4d64-b271-f4c97bfdc108-scripts\") pod \"cinder-db-sync-jhtdm\" (UID: \"0c8be033-f83e-4d64-b271-f4c97bfdc108\") " pod="openstack-kuttl-tests/cinder-db-sync-jhtdm" Dec 03 12:04:29 crc kubenswrapper[5052]: I1203 12:04:29.908077 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/0c8be033-f83e-4d64-b271-f4c97bfdc108-db-sync-config-data\") pod \"cinder-db-sync-jhtdm\" (UID: \"0c8be033-f83e-4d64-b271-f4c97bfdc108\") " pod="openstack-kuttl-tests/cinder-db-sync-jhtdm" Dec 03 12:04:29 crc kubenswrapper[5052]: I1203 12:04:29.911598 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0c8be033-f83e-4d64-b271-f4c97bfdc108-combined-ca-bundle\") pod \"cinder-db-sync-jhtdm\" (UID: \"0c8be033-f83e-4d64-b271-f4c97bfdc108\") " pod="openstack-kuttl-tests/cinder-db-sync-jhtdm" Dec 03 12:04:29 crc kubenswrapper[5052]: I1203 12:04:29.916248 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-57tzg\" (UniqueName: \"kubernetes.io/projected/0c8be033-f83e-4d64-b271-f4c97bfdc108-kube-api-access-57tzg\") pod \"cinder-db-sync-jhtdm\" (UID: \"0c8be033-f83e-4d64-b271-f4c97bfdc108\") " pod="openstack-kuttl-tests/cinder-db-sync-jhtdm" Dec 03 12:04:29 crc kubenswrapper[5052]: I1203 12:04:29.952708 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-db-sync-lrrtz" Dec 03 12:04:29 crc kubenswrapper[5052]: I1203 12:04:29.976147 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:04:29 crc kubenswrapper[5052]: I1203 12:04:29.998834 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/f3c81a5a-a6cc-406f-95b9-186ae52a3ccb-db-sync-config-data\") pod \"barbican-db-sync-4z2hl\" (UID: \"f3c81a5a-a6cc-406f-95b9-186ae52a3ccb\") " pod="openstack-kuttl-tests/barbican-db-sync-4z2hl" Dec 03 12:04:29 crc kubenswrapper[5052]: I1203 12:04:29.998925 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f3c81a5a-a6cc-406f-95b9-186ae52a3ccb-combined-ca-bundle\") pod \"barbican-db-sync-4z2hl\" (UID: \"f3c81a5a-a6cc-406f-95b9-186ae52a3ccb\") " pod="openstack-kuttl-tests/barbican-db-sync-4z2hl" Dec 03 12:04:29 crc kubenswrapper[5052]: I1203 12:04:29.998976 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-hxq27\" (UniqueName: \"kubernetes.io/projected/f3c81a5a-a6cc-406f-95b9-186ae52a3ccb-kube-api-access-hxq27\") pod \"barbican-db-sync-4z2hl\" (UID: \"f3c81a5a-a6cc-406f-95b9-186ae52a3ccb\") " pod="openstack-kuttl-tests/barbican-db-sync-4z2hl" Dec 03 12:04:30 crc kubenswrapper[5052]: I1203 12:04:30.006223 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-db-sync-852kv" Dec 03 12:04:30 crc kubenswrapper[5052]: I1203 12:04:30.015973 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-hxq27\" (UniqueName: \"kubernetes.io/projected/f3c81a5a-a6cc-406f-95b9-186ae52a3ccb-kube-api-access-hxq27\") pod \"barbican-db-sync-4z2hl\" (UID: \"f3c81a5a-a6cc-406f-95b9-186ae52a3ccb\") " pod="openstack-kuttl-tests/barbican-db-sync-4z2hl" Dec 03 12:04:30 crc kubenswrapper[5052]: I1203 12:04:30.021071 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/f3c81a5a-a6cc-406f-95b9-186ae52a3ccb-db-sync-config-data\") pod \"barbican-db-sync-4z2hl\" (UID: \"f3c81a5a-a6cc-406f-95b9-186ae52a3ccb\") " pod="openstack-kuttl-tests/barbican-db-sync-4z2hl" Dec 03 12:04:30 crc kubenswrapper[5052]: I1203 12:04:30.022652 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f3c81a5a-a6cc-406f-95b9-186ae52a3ccb-combined-ca-bundle\") pod \"barbican-db-sync-4z2hl\" (UID: \"f3c81a5a-a6cc-406f-95b9-186ae52a3ccb\") " pod="openstack-kuttl-tests/barbican-db-sync-4z2hl" Dec 03 12:04:30 crc kubenswrapper[5052]: I1203 12:04:30.073371 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-db-sync-jhtdm" Dec 03 12:04:30 crc kubenswrapper[5052]: I1203 12:04:30.090105 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-db-sync-4z2hl" Dec 03 12:04:30 crc kubenswrapper[5052]: I1203 12:04:30.145995 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-bootstrap-wbwms"] Dec 03 12:04:30 crc kubenswrapper[5052]: I1203 12:04:30.276900 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:04:30 crc kubenswrapper[5052]: I1203 12:04:30.279803 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:04:30 crc kubenswrapper[5052]: I1203 12:04:30.282260 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-glance-dockercfg-x6d26" Dec 03 12:04:30 crc kubenswrapper[5052]: I1203 12:04:30.290951 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-scripts" Dec 03 12:04:30 crc kubenswrapper[5052]: I1203 12:04:30.291210 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-default-external-config-data" Dec 03 12:04:30 crc kubenswrapper[5052]: I1203 12:04:30.292441 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:04:30 crc kubenswrapper[5052]: I1203 12:04:30.396084 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:04:30 crc kubenswrapper[5052]: I1203 12:04:30.397966 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:04:30 crc kubenswrapper[5052]: I1203 12:04:30.404318 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-default-internal-config-data" Dec 03 12:04:30 crc kubenswrapper[5052]: I1203 12:04:30.426254 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:04:30 crc kubenswrapper[5052]: I1203 12:04:30.431377 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fb4ng\" (UniqueName: \"kubernetes.io/projected/ef916b08-df1b-45fe-a024-4d7ece37ca35-kube-api-access-fb4ng\") pod \"glance-default-external-api-0\" (UID: \"ef916b08-df1b-45fe-a024-4d7ece37ca35\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:04:30 crc kubenswrapper[5052]: I1203 12:04:30.431699 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/ef916b08-df1b-45fe-a024-4d7ece37ca35-config-data\") pod \"glance-default-external-api-0\" (UID: \"ef916b08-df1b-45fe-a024-4d7ece37ca35\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:04:30 crc kubenswrapper[5052]: I1203 12:04:30.431842 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/ef916b08-df1b-45fe-a024-4d7ece37ca35-logs\") pod \"glance-default-external-api-0\" (UID: \"ef916b08-df1b-45fe-a024-4d7ece37ca35\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:04:30 crc kubenswrapper[5052]: I1203 12:04:30.431953 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/ef916b08-df1b-45fe-a024-4d7ece37ca35-httpd-run\") pod \"glance-default-external-api-0\" (UID: \"ef916b08-df1b-45fe-a024-4d7ece37ca35\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:04:30 crc kubenswrapper[5052]: I1203 12:04:30.431995 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ef916b08-df1b-45fe-a024-4d7ece37ca35-combined-ca-bundle\") pod \"glance-default-external-api-0\" (UID: \"ef916b08-df1b-45fe-a024-4d7ece37ca35\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:04:30 crc kubenswrapper[5052]: I1203 12:04:30.432033 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/ef916b08-df1b-45fe-a024-4d7ece37ca35-scripts\") pod \"glance-default-external-api-0\" (UID: \"ef916b08-df1b-45fe-a024-4d7ece37ca35\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:04:30 crc kubenswrapper[5052]: I1203 12:04:30.432105 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage11-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage11-crc\") pod \"glance-default-external-api-0\" (UID: \"ef916b08-df1b-45fe-a024-4d7ece37ca35\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:04:30 crc kubenswrapper[5052]: I1203 12:04:30.534813 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/ef916b08-df1b-45fe-a024-4d7ece37ca35-logs\") pod \"glance-default-external-api-0\" (UID: \"ef916b08-df1b-45fe-a024-4d7ece37ca35\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:04:30 crc kubenswrapper[5052]: I1203 12:04:30.534895 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/ef916b08-df1b-45fe-a024-4d7ece37ca35-httpd-run\") pod \"glance-default-external-api-0\" (UID: \"ef916b08-df1b-45fe-a024-4d7ece37ca35\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:04:30 crc kubenswrapper[5052]: I1203 12:04:30.534919 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ef916b08-df1b-45fe-a024-4d7ece37ca35-combined-ca-bundle\") pod \"glance-default-external-api-0\" (UID: \"ef916b08-df1b-45fe-a024-4d7ece37ca35\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:04:30 crc kubenswrapper[5052]: I1203 12:04:30.534940 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/ef916b08-df1b-45fe-a024-4d7ece37ca35-scripts\") pod \"glance-default-external-api-0\" (UID: \"ef916b08-df1b-45fe-a024-4d7ece37ca35\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:04:30 crc kubenswrapper[5052]: I1203 12:04:30.534975 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage11-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage11-crc\") pod \"glance-default-external-api-0\" (UID: \"ef916b08-df1b-45fe-a024-4d7ece37ca35\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:04:30 crc kubenswrapper[5052]: I1203 12:04:30.535025 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/937cceb6-0e2b-4ee2-8489-97ac82f339be-scripts\") pod \"glance-default-internal-api-0\" (UID: \"937cceb6-0e2b-4ee2-8489-97ac82f339be\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:04:30 crc kubenswrapper[5052]: I1203 12:04:30.535046 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-fb4ng\" (UniqueName: \"kubernetes.io/projected/ef916b08-df1b-45fe-a024-4d7ece37ca35-kube-api-access-fb4ng\") pod \"glance-default-external-api-0\" (UID: \"ef916b08-df1b-45fe-a024-4d7ece37ca35\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:04:30 crc kubenswrapper[5052]: I1203 12:04:30.535100 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mspt8\" (UniqueName: \"kubernetes.io/projected/937cceb6-0e2b-4ee2-8489-97ac82f339be-kube-api-access-mspt8\") pod \"glance-default-internal-api-0\" (UID: \"937cceb6-0e2b-4ee2-8489-97ac82f339be\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:04:30 crc kubenswrapper[5052]: I1203 12:04:30.535120 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/937cceb6-0e2b-4ee2-8489-97ac82f339be-httpd-run\") pod \"glance-default-internal-api-0\" (UID: \"937cceb6-0e2b-4ee2-8489-97ac82f339be\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:04:30 crc kubenswrapper[5052]: I1203 12:04:30.535158 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage13-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage13-crc\") pod \"glance-default-internal-api-0\" (UID: \"937cceb6-0e2b-4ee2-8489-97ac82f339be\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:04:30 crc kubenswrapper[5052]: I1203 12:04:30.535177 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/937cceb6-0e2b-4ee2-8489-97ac82f339be-combined-ca-bundle\") pod \"glance-default-internal-api-0\" (UID: \"937cceb6-0e2b-4ee2-8489-97ac82f339be\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:04:30 crc kubenswrapper[5052]: I1203 12:04:30.535219 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/937cceb6-0e2b-4ee2-8489-97ac82f339be-logs\") pod \"glance-default-internal-api-0\" (UID: \"937cceb6-0e2b-4ee2-8489-97ac82f339be\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:04:30 crc kubenswrapper[5052]: I1203 12:04:30.535257 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/ef916b08-df1b-45fe-a024-4d7ece37ca35-config-data\") pod \"glance-default-external-api-0\" (UID: \"ef916b08-df1b-45fe-a024-4d7ece37ca35\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:04:30 crc kubenswrapper[5052]: I1203 12:04:30.535298 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/937cceb6-0e2b-4ee2-8489-97ac82f339be-config-data\") pod \"glance-default-internal-api-0\" (UID: \"937cceb6-0e2b-4ee2-8489-97ac82f339be\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:04:30 crc kubenswrapper[5052]: I1203 12:04:30.535337 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/ef916b08-df1b-45fe-a024-4d7ece37ca35-logs\") pod \"glance-default-external-api-0\" (UID: \"ef916b08-df1b-45fe-a024-4d7ece37ca35\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:04:30 crc kubenswrapper[5052]: I1203 12:04:30.535609 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/ef916b08-df1b-45fe-a024-4d7ece37ca35-httpd-run\") pod \"glance-default-external-api-0\" (UID: \"ef916b08-df1b-45fe-a024-4d7ece37ca35\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:04:30 crc kubenswrapper[5052]: I1203 12:04:30.540904 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ef916b08-df1b-45fe-a024-4d7ece37ca35-combined-ca-bundle\") pod \"glance-default-external-api-0\" (UID: \"ef916b08-df1b-45fe-a024-4d7ece37ca35\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:04:30 crc kubenswrapper[5052]: I1203 12:04:30.541237 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage11-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage11-crc\") pod \"glance-default-external-api-0\" (UID: \"ef916b08-df1b-45fe-a024-4d7ece37ca35\") device mount path \"/mnt/openstack/pv11\"" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:04:30 crc kubenswrapper[5052]: I1203 12:04:30.546834 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/ef916b08-df1b-45fe-a024-4d7ece37ca35-scripts\") pod \"glance-default-external-api-0\" (UID: \"ef916b08-df1b-45fe-a024-4d7ece37ca35\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:04:30 crc kubenswrapper[5052]: I1203 12:04:30.547717 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/ef916b08-df1b-45fe-a024-4d7ece37ca35-config-data\") pod \"glance-default-external-api-0\" (UID: \"ef916b08-df1b-45fe-a024-4d7ece37ca35\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:04:30 crc kubenswrapper[5052]: I1203 12:04:30.555673 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:04:30 crc kubenswrapper[5052]: I1203 12:04:30.574526 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-fb4ng\" (UniqueName: \"kubernetes.io/projected/ef916b08-df1b-45fe-a024-4d7ece37ca35-kube-api-access-fb4ng\") pod \"glance-default-external-api-0\" (UID: \"ef916b08-df1b-45fe-a024-4d7ece37ca35\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:04:30 crc kubenswrapper[5052]: I1203 12:04:30.607865 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage11-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage11-crc\") pod \"glance-default-external-api-0\" (UID: \"ef916b08-df1b-45fe-a024-4d7ece37ca35\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:04:30 crc kubenswrapper[5052]: I1203 12:04:30.629488 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:04:30 crc kubenswrapper[5052]: I1203 12:04:30.637045 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/937cceb6-0e2b-4ee2-8489-97ac82f339be-scripts\") pod \"glance-default-internal-api-0\" (UID: \"937cceb6-0e2b-4ee2-8489-97ac82f339be\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:04:30 crc kubenswrapper[5052]: I1203 12:04:30.637104 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-mspt8\" (UniqueName: \"kubernetes.io/projected/937cceb6-0e2b-4ee2-8489-97ac82f339be-kube-api-access-mspt8\") pod \"glance-default-internal-api-0\" (UID: \"937cceb6-0e2b-4ee2-8489-97ac82f339be\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:04:30 crc kubenswrapper[5052]: I1203 12:04:30.637122 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/937cceb6-0e2b-4ee2-8489-97ac82f339be-httpd-run\") pod \"glance-default-internal-api-0\" (UID: \"937cceb6-0e2b-4ee2-8489-97ac82f339be\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:04:30 crc kubenswrapper[5052]: I1203 12:04:30.637155 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage13-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage13-crc\") pod \"glance-default-internal-api-0\" (UID: \"937cceb6-0e2b-4ee2-8489-97ac82f339be\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:04:30 crc kubenswrapper[5052]: I1203 12:04:30.637175 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/937cceb6-0e2b-4ee2-8489-97ac82f339be-combined-ca-bundle\") pod \"glance-default-internal-api-0\" (UID: \"937cceb6-0e2b-4ee2-8489-97ac82f339be\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:04:30 crc kubenswrapper[5052]: I1203 12:04:30.637208 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/937cceb6-0e2b-4ee2-8489-97ac82f339be-logs\") pod \"glance-default-internal-api-0\" (UID: \"937cceb6-0e2b-4ee2-8489-97ac82f339be\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:04:30 crc kubenswrapper[5052]: I1203 12:04:30.637250 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/937cceb6-0e2b-4ee2-8489-97ac82f339be-config-data\") pod \"glance-default-internal-api-0\" (UID: \"937cceb6-0e2b-4ee2-8489-97ac82f339be\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:04:30 crc kubenswrapper[5052]: I1203 12:04:30.640801 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage13-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage13-crc\") pod \"glance-default-internal-api-0\" (UID: \"937cceb6-0e2b-4ee2-8489-97ac82f339be\") device mount path \"/mnt/openstack/pv13\"" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:04:30 crc kubenswrapper[5052]: I1203 12:04:30.648285 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/937cceb6-0e2b-4ee2-8489-97ac82f339be-httpd-run\") pod \"glance-default-internal-api-0\" (UID: \"937cceb6-0e2b-4ee2-8489-97ac82f339be\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:04:30 crc kubenswrapper[5052]: I1203 12:04:30.649427 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/937cceb6-0e2b-4ee2-8489-97ac82f339be-logs\") pod \"glance-default-internal-api-0\" (UID: \"937cceb6-0e2b-4ee2-8489-97ac82f339be\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:04:30 crc kubenswrapper[5052]: I1203 12:04:30.650218 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/937cceb6-0e2b-4ee2-8489-97ac82f339be-combined-ca-bundle\") pod \"glance-default-internal-api-0\" (UID: \"937cceb6-0e2b-4ee2-8489-97ac82f339be\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:04:30 crc kubenswrapper[5052]: I1203 12:04:30.662002 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/937cceb6-0e2b-4ee2-8489-97ac82f339be-config-data\") pod \"glance-default-internal-api-0\" (UID: \"937cceb6-0e2b-4ee2-8489-97ac82f339be\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:04:30 crc kubenswrapper[5052]: I1203 12:04:30.664365 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/937cceb6-0e2b-4ee2-8489-97ac82f339be-scripts\") pod \"glance-default-internal-api-0\" (UID: \"937cceb6-0e2b-4ee2-8489-97ac82f339be\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:04:30 crc kubenswrapper[5052]: I1203 12:04:30.691567 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage13-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage13-crc\") pod \"glance-default-internal-api-0\" (UID: \"937cceb6-0e2b-4ee2-8489-97ac82f339be\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:04:30 crc kubenswrapper[5052]: I1203 12:04:30.706058 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-mspt8\" (UniqueName: \"kubernetes.io/projected/937cceb6-0e2b-4ee2-8489-97ac82f339be-kube-api-access-mspt8\") pod \"glance-default-internal-api-0\" (UID: \"937cceb6-0e2b-4ee2-8489-97ac82f339be\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:04:30 crc kubenswrapper[5052]: I1203 12:04:30.725081 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/placement-db-sync-852kv"] Dec 03 12:04:30 crc kubenswrapper[5052]: I1203 12:04:30.752380 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-db-sync-lrrtz"] Dec 03 12:04:30 crc kubenswrapper[5052]: I1203 12:04:30.759730 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:04:30 crc kubenswrapper[5052]: I1203 12:04:30.981124 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-db-sync-jhtdm"] Dec 03 12:04:31 crc kubenswrapper[5052]: W1203 12:04:31.000618 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf3c81a5a_a6cc_406f_95b9_186ae52a3ccb.slice/crio-5440de1ca16f5ac86ac3bfc5a38495ee91f425111048898c80b50d82f094ee04 WatchSource:0}: Error finding container 5440de1ca16f5ac86ac3bfc5a38495ee91f425111048898c80b50d82f094ee04: Status 404 returned error can't find the container with id 5440de1ca16f5ac86ac3bfc5a38495ee91f425111048898c80b50d82f094ee04 Dec 03 12:04:31 crc kubenswrapper[5052]: I1203 12:04:31.008123 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-bootstrap-wbwms" event={"ID":"3d2ba247-e370-4d62-a34e-5b05174363fb","Type":"ContainerStarted","Data":"d69d8109f5a447a6cd0dfcfdb4585aeecf2765db9edf4a41545793d4d4d83f6e"} Dec 03 12:04:31 crc kubenswrapper[5052]: I1203 12:04:31.012609 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-db-sync-4z2hl"] Dec 03 12:04:31 crc kubenswrapper[5052]: I1203 12:04:31.014785 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-db-sync-lrrtz" event={"ID":"d9b7a317-ab7f-4c6a-8f91-b66b409c8980","Type":"ContainerStarted","Data":"e21b878ee5af8a8af6164951d0105fce83c583505e0d5f994bcab1d22eecf09c"} Dec 03 12:04:31 crc kubenswrapper[5052]: I1203 12:04:31.027588 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"356db4d6-f512-4a47-a6ed-a288a212227d","Type":"ContainerStarted","Data":"f65552c32a602e4dea0c6aaca246e64e2f01e001bc13c386797292bdda16fba2"} Dec 03 12:04:31 crc kubenswrapper[5052]: I1203 12:04:31.034954 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-db-sync-852kv" event={"ID":"ab039824-fe2a-44ba-ae0f-302e5c4ea614","Type":"ContainerStarted","Data":"28550e67ac5e91431e3b9408d3292fba15c6a210bc517900b62f9cc7d7862a36"} Dec 03 12:04:31 crc kubenswrapper[5052]: I1203 12:04:31.038502 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-db-sync-jhtdm" event={"ID":"0c8be033-f83e-4d64-b271-f4c97bfdc108","Type":"ContainerStarted","Data":"5cfed3b0e163b3f1807c6243d2a9a296e29c8f421a7f6aa7bcc9ba12e7700ef8"} Dec 03 12:04:31 crc kubenswrapper[5052]: I1203 12:04:31.225766 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:04:31 crc kubenswrapper[5052]: W1203 12:04:31.374402 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod937cceb6_0e2b_4ee2_8489_97ac82f339be.slice/crio-8ea0216c8303a3528a1f7dabcb1d6903e23c1d9e6c8c3669f05ea14a2c2ac5d5 WatchSource:0}: Error finding container 8ea0216c8303a3528a1f7dabcb1d6903e23c1d9e6c8c3669f05ea14a2c2ac5d5: Status 404 returned error can't find the container with id 8ea0216c8303a3528a1f7dabcb1d6903e23c1d9e6c8c3669f05ea14a2c2ac5d5 Dec 03 12:04:31 crc kubenswrapper[5052]: I1203 12:04:31.385107 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:04:32 crc kubenswrapper[5052]: I1203 12:04:32.053498 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-db-sync-4z2hl" event={"ID":"f3c81a5a-a6cc-406f-95b9-186ae52a3ccb","Type":"ContainerStarted","Data":"5440de1ca16f5ac86ac3bfc5a38495ee91f425111048898c80b50d82f094ee04"} Dec 03 12:04:32 crc kubenswrapper[5052]: I1203 12:04:32.058449 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"937cceb6-0e2b-4ee2-8489-97ac82f339be","Type":"ContainerStarted","Data":"8ea0216c8303a3528a1f7dabcb1d6903e23c1d9e6c8c3669f05ea14a2c2ac5d5"} Dec 03 12:04:32 crc kubenswrapper[5052]: I1203 12:04:32.059855 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"ef916b08-df1b-45fe-a024-4d7ece37ca35","Type":"ContainerStarted","Data":"b09f44cf3dc4428f602a669ca73f7926e592091776bca76b1c66e07ddfc405fa"} Dec 03 12:04:32 crc kubenswrapper[5052]: I1203 12:04:32.469109 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:04:32 crc kubenswrapper[5052]: I1203 12:04:32.506159 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:04:32 crc kubenswrapper[5052]: I1203 12:04:32.560318 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:04:37 crc kubenswrapper[5052]: I1203 12:04:37.110420 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-bootstrap-wbwms" event={"ID":"3d2ba247-e370-4d62-a34e-5b05174363fb","Type":"ContainerStarted","Data":"a23efe47ae844777245a54dff972169031df4ea13092e5aabb04cc320ac712df"} Dec 03 12:04:37 crc kubenswrapper[5052]: I1203 12:04:37.430219 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-marketplace/redhat-operators-jjj57" Dec 03 12:04:37 crc kubenswrapper[5052]: I1203 12:04:37.499725 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-marketplace/redhat-operators-jjj57" Dec 03 12:04:37 crc kubenswrapper[5052]: I1203 12:04:37.668358 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openshift-marketplace/redhat-operators-jjj57"] Dec 03 12:04:38 crc kubenswrapper[5052]: I1203 12:04:38.121903 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-db-sync-852kv" event={"ID":"ab039824-fe2a-44ba-ae0f-302e5c4ea614","Type":"ContainerStarted","Data":"32468dac9534ebe102ff57ab3a6ff7547398603ee630025eaac80c0d8f9678c8"} Dec 03 12:04:38 crc kubenswrapper[5052]: I1203 12:04:38.124001 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-db-sync-jhtdm" event={"ID":"0c8be033-f83e-4d64-b271-f4c97bfdc108","Type":"ContainerStarted","Data":"216f652b89ba64b000d5cd54f33f538c801f811bfa201b681b49eb486c4725d9"} Dec 03 12:04:38 crc kubenswrapper[5052]: I1203 12:04:38.125441 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"937cceb6-0e2b-4ee2-8489-97ac82f339be","Type":"ContainerStarted","Data":"758ebf28e00662b6ef188c25f34abaadb9427993d6fbb5f5dc644be7f7fe7652"} Dec 03 12:04:38 crc kubenswrapper[5052]: I1203 12:04:38.126691 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"ef916b08-df1b-45fe-a024-4d7ece37ca35","Type":"ContainerStarted","Data":"4bae4ed58a5f95a7350a39735d066875a32ab0eec38427974d5e6223ed2cda10"} Dec 03 12:04:38 crc kubenswrapper[5052]: I1203 12:04:38.127829 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-db-sync-lrrtz" event={"ID":"d9b7a317-ab7f-4c6a-8f91-b66b409c8980","Type":"ContainerStarted","Data":"f25bc6d11555c8eec8ce818399df69255f9b136b8146d57ac9a33a070d4dc79b"} Dec 03 12:04:38 crc kubenswrapper[5052]: I1203 12:04:38.130799 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-db-sync-4z2hl" event={"ID":"f3c81a5a-a6cc-406f-95b9-186ae52a3ccb","Type":"ContainerStarted","Data":"c28a9606ae5010324a932dd9d90a75063a7e3597bf5e73c2ae2c2451a0afbbab"} Dec 03 12:04:38 crc kubenswrapper[5052]: I1203 12:04:38.159485 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/keystone-bootstrap-wbwms" podStartSLOduration=9.159451172 podStartE2EDuration="9.159451172s" podCreationTimestamp="2025-12-03 12:04:29 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:04:38.148551251 +0000 UTC m=+2668.736793015" watchObservedRunningTime="2025-12-03 12:04:38.159451172 +0000 UTC m=+2668.747692936" Dec 03 12:04:38 crc kubenswrapper[5052]: I1203 12:04:38.170363 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/barbican-db-sync-4z2hl" podStartSLOduration=9.170338463 podStartE2EDuration="9.170338463s" podCreationTimestamp="2025-12-03 12:04:29 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:04:38.165130034 +0000 UTC m=+2668.753371798" watchObservedRunningTime="2025-12-03 12:04:38.170338463 +0000 UTC m=+2668.758580227" Dec 03 12:04:39 crc kubenswrapper[5052]: I1203 12:04:39.140285 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openshift-marketplace/redhat-operators-jjj57" podUID="ede81f70-0e50-47b2-85b7-d30ec858e2e4" containerName="registry-server" containerID="cri-o://23730e89625cbefdb76c39b2c82e8573ab145d63d3aee524c618c90e90b9af8f" gracePeriod=2 Dec 03 12:04:39 crc kubenswrapper[5052]: I1203 12:04:39.167991 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/neutron-db-sync-lrrtz" podStartSLOduration=10.167970235 podStartE2EDuration="10.167970235s" podCreationTimestamp="2025-12-03 12:04:29 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:04:39.163978801 +0000 UTC m=+2669.752220575" watchObservedRunningTime="2025-12-03 12:04:39.167970235 +0000 UTC m=+2669.756211999" Dec 03 12:04:39 crc kubenswrapper[5052]: I1203 12:04:39.182739 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/placement-db-sync-852kv" podStartSLOduration=10.182722107 podStartE2EDuration="10.182722107s" podCreationTimestamp="2025-12-03 12:04:29 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:04:39.175772288 +0000 UTC m=+2669.764014052" watchObservedRunningTime="2025-12-03 12:04:39.182722107 +0000 UTC m=+2669.770963871" Dec 03 12:04:39 crc kubenswrapper[5052]: I1203 12:04:39.211264 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/cinder-db-sync-jhtdm" podStartSLOduration=10.211240462 podStartE2EDuration="10.211240462s" podCreationTimestamp="2025-12-03 12:04:29 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:04:39.206417034 +0000 UTC m=+2669.794658788" watchObservedRunningTime="2025-12-03 12:04:39.211240462 +0000 UTC m=+2669.799482236" Dec 03 12:04:40 crc kubenswrapper[5052]: I1203 12:04:40.152941 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"ef916b08-df1b-45fe-a024-4d7ece37ca35","Type":"ContainerStarted","Data":"cd6f206671d36b56e19fffadbde7b469d524db4f3799bd51d0ee923dbfb51162"} Dec 03 12:04:40 crc kubenswrapper[5052]: I1203 12:04:40.155564 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"937cceb6-0e2b-4ee2-8489-97ac82f339be","Type":"ContainerStarted","Data":"fc1c4042d0bcee43afcb9e370be77b96062717361a92c6021f304dabbd0e4c70"} Dec 03 12:04:41 crc kubenswrapper[5052]: I1203 12:04:41.219000 5052 generic.go:334] "Generic (PLEG): container finished" podID="ede81f70-0e50-47b2-85b7-d30ec858e2e4" containerID="23730e89625cbefdb76c39b2c82e8573ab145d63d3aee524c618c90e90b9af8f" exitCode=0 Dec 03 12:04:41 crc kubenswrapper[5052]: I1203 12:04:41.219903 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-external-api-0" podUID="ef916b08-df1b-45fe-a024-4d7ece37ca35" containerName="glance-log" containerID="cri-o://4bae4ed58a5f95a7350a39735d066875a32ab0eec38427974d5e6223ed2cda10" gracePeriod=30 Dec 03 12:04:41 crc kubenswrapper[5052]: I1203 12:04:41.220063 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-operators-jjj57" event={"ID":"ede81f70-0e50-47b2-85b7-d30ec858e2e4","Type":"ContainerDied","Data":"23730e89625cbefdb76c39b2c82e8573ab145d63d3aee524c618c90e90b9af8f"} Dec 03 12:04:41 crc kubenswrapper[5052]: I1203 12:04:41.220243 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-internal-api-0" podUID="937cceb6-0e2b-4ee2-8489-97ac82f339be" containerName="glance-log" containerID="cri-o://758ebf28e00662b6ef188c25f34abaadb9427993d6fbb5f5dc644be7f7fe7652" gracePeriod=30 Dec 03 12:04:41 crc kubenswrapper[5052]: I1203 12:04:41.220971 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-external-api-0" podUID="ef916b08-df1b-45fe-a024-4d7ece37ca35" containerName="glance-httpd" containerID="cri-o://cd6f206671d36b56e19fffadbde7b469d524db4f3799bd51d0ee923dbfb51162" gracePeriod=30 Dec 03 12:04:41 crc kubenswrapper[5052]: I1203 12:04:41.221338 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-internal-api-0" podUID="937cceb6-0e2b-4ee2-8489-97ac82f339be" containerName="glance-httpd" containerID="cri-o://fc1c4042d0bcee43afcb9e370be77b96062717361a92c6021f304dabbd0e4c70" gracePeriod=30 Dec 03 12:04:41 crc kubenswrapper[5052]: I1203 12:04:41.260234 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/glance-default-external-api-0" podStartSLOduration=12.260212321000001 podStartE2EDuration="12.260212321s" podCreationTimestamp="2025-12-03 12:04:29 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:04:41.256469474 +0000 UTC m=+2671.844711248" watchObservedRunningTime="2025-12-03 12:04:41.260212321 +0000 UTC m=+2671.848454085" Dec 03 12:04:41 crc kubenswrapper[5052]: I1203 12:04:41.288069 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/glance-default-internal-api-0" podStartSLOduration=12.288050207 podStartE2EDuration="12.288050207s" podCreationTimestamp="2025-12-03 12:04:29 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:04:41.286990226 +0000 UTC m=+2671.875232000" watchObservedRunningTime="2025-12-03 12:04:41.288050207 +0000 UTC m=+2671.876291971" Dec 03 12:04:50 crc kubenswrapper[5052]: I1203 12:04:41.996141 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/redhat-operators-jjj57" Dec 03 12:04:50 crc kubenswrapper[5052]: I1203 12:04:42.072568 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/ede81f70-0e50-47b2-85b7-d30ec858e2e4-utilities\") pod \"ede81f70-0e50-47b2-85b7-d30ec858e2e4\" (UID: \"ede81f70-0e50-47b2-85b7-d30ec858e2e4\") " Dec 03 12:04:50 crc kubenswrapper[5052]: I1203 12:04:42.072674 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/ede81f70-0e50-47b2-85b7-d30ec858e2e4-catalog-content\") pod \"ede81f70-0e50-47b2-85b7-d30ec858e2e4\" (UID: \"ede81f70-0e50-47b2-85b7-d30ec858e2e4\") " Dec 03 12:04:50 crc kubenswrapper[5052]: I1203 12:04:42.072751 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lbpxf\" (UniqueName: \"kubernetes.io/projected/ede81f70-0e50-47b2-85b7-d30ec858e2e4-kube-api-access-lbpxf\") pod \"ede81f70-0e50-47b2-85b7-d30ec858e2e4\" (UID: \"ede81f70-0e50-47b2-85b7-d30ec858e2e4\") " Dec 03 12:04:50 crc kubenswrapper[5052]: I1203 12:04:42.074053 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ede81f70-0e50-47b2-85b7-d30ec858e2e4-utilities" (OuterVolumeSpecName: "utilities") pod "ede81f70-0e50-47b2-85b7-d30ec858e2e4" (UID: "ede81f70-0e50-47b2-85b7-d30ec858e2e4"). InnerVolumeSpecName "utilities". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:04:50 crc kubenswrapper[5052]: I1203 12:04:42.079413 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ede81f70-0e50-47b2-85b7-d30ec858e2e4-kube-api-access-lbpxf" (OuterVolumeSpecName: "kube-api-access-lbpxf") pod "ede81f70-0e50-47b2-85b7-d30ec858e2e4" (UID: "ede81f70-0e50-47b2-85b7-d30ec858e2e4"). InnerVolumeSpecName "kube-api-access-lbpxf". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:04:50 crc kubenswrapper[5052]: I1203 12:04:42.176138 5052 reconciler_common.go:293] "Volume detached for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/ede81f70-0e50-47b2-85b7-d30ec858e2e4-utilities\") on node \"crc\" DevicePath \"\"" Dec 03 12:04:50 crc kubenswrapper[5052]: I1203 12:04:42.176191 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-lbpxf\" (UniqueName: \"kubernetes.io/projected/ede81f70-0e50-47b2-85b7-d30ec858e2e4-kube-api-access-lbpxf\") on node \"crc\" DevicePath \"\"" Dec 03 12:04:50 crc kubenswrapper[5052]: I1203 12:04:42.179809 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ede81f70-0e50-47b2-85b7-d30ec858e2e4-catalog-content" (OuterVolumeSpecName: "catalog-content") pod "ede81f70-0e50-47b2-85b7-d30ec858e2e4" (UID: "ede81f70-0e50-47b2-85b7-d30ec858e2e4"). InnerVolumeSpecName "catalog-content". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:04:50 crc kubenswrapper[5052]: I1203 12:04:42.232802 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-operators-jjj57" event={"ID":"ede81f70-0e50-47b2-85b7-d30ec858e2e4","Type":"ContainerDied","Data":"b050828e580762e184f99bf748b8e3628e6af1887a0356967f23e28b640904d0"} Dec 03 12:04:50 crc kubenswrapper[5052]: I1203 12:04:42.232907 5052 scope.go:117] "RemoveContainer" containerID="23730e89625cbefdb76c39b2c82e8573ab145d63d3aee524c618c90e90b9af8f" Dec 03 12:04:50 crc kubenswrapper[5052]: I1203 12:04:42.233311 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/redhat-operators-jjj57" Dec 03 12:04:50 crc kubenswrapper[5052]: I1203 12:04:42.237662 5052 generic.go:334] "Generic (PLEG): container finished" podID="937cceb6-0e2b-4ee2-8489-97ac82f339be" containerID="fc1c4042d0bcee43afcb9e370be77b96062717361a92c6021f304dabbd0e4c70" exitCode=143 Dec 03 12:04:50 crc kubenswrapper[5052]: I1203 12:04:42.237705 5052 generic.go:334] "Generic (PLEG): container finished" podID="937cceb6-0e2b-4ee2-8489-97ac82f339be" containerID="758ebf28e00662b6ef188c25f34abaadb9427993d6fbb5f5dc644be7f7fe7652" exitCode=143 Dec 03 12:04:50 crc kubenswrapper[5052]: I1203 12:04:42.237767 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"937cceb6-0e2b-4ee2-8489-97ac82f339be","Type":"ContainerDied","Data":"fc1c4042d0bcee43afcb9e370be77b96062717361a92c6021f304dabbd0e4c70"} Dec 03 12:04:50 crc kubenswrapper[5052]: I1203 12:04:42.237821 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"937cceb6-0e2b-4ee2-8489-97ac82f339be","Type":"ContainerDied","Data":"758ebf28e00662b6ef188c25f34abaadb9427993d6fbb5f5dc644be7f7fe7652"} Dec 03 12:04:50 crc kubenswrapper[5052]: I1203 12:04:42.240442 5052 generic.go:334] "Generic (PLEG): container finished" podID="ef916b08-df1b-45fe-a024-4d7ece37ca35" containerID="cd6f206671d36b56e19fffadbde7b469d524db4f3799bd51d0ee923dbfb51162" exitCode=143 Dec 03 12:04:50 crc kubenswrapper[5052]: I1203 12:04:42.240534 5052 generic.go:334] "Generic (PLEG): container finished" podID="ef916b08-df1b-45fe-a024-4d7ece37ca35" containerID="4bae4ed58a5f95a7350a39735d066875a32ab0eec38427974d5e6223ed2cda10" exitCode=143 Dec 03 12:04:50 crc kubenswrapper[5052]: I1203 12:04:42.241273 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"ef916b08-df1b-45fe-a024-4d7ece37ca35","Type":"ContainerDied","Data":"cd6f206671d36b56e19fffadbde7b469d524db4f3799bd51d0ee923dbfb51162"} Dec 03 12:04:50 crc kubenswrapper[5052]: I1203 12:04:42.241321 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"ef916b08-df1b-45fe-a024-4d7ece37ca35","Type":"ContainerDied","Data":"4bae4ed58a5f95a7350a39735d066875a32ab0eec38427974d5e6223ed2cda10"} Dec 03 12:04:50 crc kubenswrapper[5052]: I1203 12:04:42.278087 5052 reconciler_common.go:293] "Volume detached for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/ede81f70-0e50-47b2-85b7-d30ec858e2e4-catalog-content\") on node \"crc\" DevicePath \"\"" Dec 03 12:04:50 crc kubenswrapper[5052]: I1203 12:04:42.280282 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openshift-marketplace/redhat-operators-jjj57"] Dec 03 12:04:50 crc kubenswrapper[5052]: I1203 12:04:42.288929 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openshift-marketplace/redhat-operators-jjj57"] Dec 03 12:04:50 crc kubenswrapper[5052]: I1203 12:04:42.904737 5052 scope.go:117] "RemoveContainer" containerID="286a3f1bf5f4923c58cec63c2cdb346e707904aba0bb87a984d018cbba3b63b2" Dec 03 12:04:50 crc kubenswrapper[5052]: I1203 12:04:43.688285 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ede81f70-0e50-47b2-85b7-d30ec858e2e4" path="/var/lib/kubelet/pods/ede81f70-0e50-47b2-85b7-d30ec858e2e4/volumes" Dec 03 12:04:50 crc kubenswrapper[5052]: I1203 12:04:44.306077 5052 scope.go:117] "RemoveContainer" containerID="5fe6a1c2c920a2c8fd804ec100332c776ee457aab74f8d3a733fd1848b5a20d7" Dec 03 12:04:50 crc kubenswrapper[5052]: I1203 12:04:45.276010 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"356db4d6-f512-4a47-a6ed-a288a212227d","Type":"ContainerStarted","Data":"9a53387b8a8a36212c1684331e46cebdf007ce9e1dcfa9efacbd247058d14e91"} Dec 03 12:04:50 crc kubenswrapper[5052]: I1203 12:04:50.730918 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:04:50 crc kubenswrapper[5052]: I1203 12:04:50.741150 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:04:50 crc kubenswrapper[5052]: I1203 12:04:50.839637 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/937cceb6-0e2b-4ee2-8489-97ac82f339be-scripts\") pod \"937cceb6-0e2b-4ee2-8489-97ac82f339be\" (UID: \"937cceb6-0e2b-4ee2-8489-97ac82f339be\") " Dec 03 12:04:50 crc kubenswrapper[5052]: I1203 12:04:50.839719 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/937cceb6-0e2b-4ee2-8489-97ac82f339be-logs\") pod \"937cceb6-0e2b-4ee2-8489-97ac82f339be\" (UID: \"937cceb6-0e2b-4ee2-8489-97ac82f339be\") " Dec 03 12:04:50 crc kubenswrapper[5052]: I1203 12:04:50.839775 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fb4ng\" (UniqueName: \"kubernetes.io/projected/ef916b08-df1b-45fe-a024-4d7ece37ca35-kube-api-access-fb4ng\") pod \"ef916b08-df1b-45fe-a024-4d7ece37ca35\" (UID: \"ef916b08-df1b-45fe-a024-4d7ece37ca35\") " Dec 03 12:04:50 crc kubenswrapper[5052]: I1203 12:04:50.839799 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/937cceb6-0e2b-4ee2-8489-97ac82f339be-combined-ca-bundle\") pod \"937cceb6-0e2b-4ee2-8489-97ac82f339be\" (UID: \"937cceb6-0e2b-4ee2-8489-97ac82f339be\") " Dec 03 12:04:50 crc kubenswrapper[5052]: I1203 12:04:50.839833 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/937cceb6-0e2b-4ee2-8489-97ac82f339be-httpd-run\") pod \"937cceb6-0e2b-4ee2-8489-97ac82f339be\" (UID: \"937cceb6-0e2b-4ee2-8489-97ac82f339be\") " Dec 03 12:04:50 crc kubenswrapper[5052]: I1203 12:04:50.840291 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"glance\" (UniqueName: \"kubernetes.io/local-volume/local-storage13-crc\") pod \"937cceb6-0e2b-4ee2-8489-97ac82f339be\" (UID: \"937cceb6-0e2b-4ee2-8489-97ac82f339be\") " Dec 03 12:04:50 crc kubenswrapper[5052]: I1203 12:04:50.840334 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/ef916b08-df1b-45fe-a024-4d7ece37ca35-scripts\") pod \"ef916b08-df1b-45fe-a024-4d7ece37ca35\" (UID: \"ef916b08-df1b-45fe-a024-4d7ece37ca35\") " Dec 03 12:04:50 crc kubenswrapper[5052]: I1203 12:04:50.840368 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mspt8\" (UniqueName: \"kubernetes.io/projected/937cceb6-0e2b-4ee2-8489-97ac82f339be-kube-api-access-mspt8\") pod \"937cceb6-0e2b-4ee2-8489-97ac82f339be\" (UID: \"937cceb6-0e2b-4ee2-8489-97ac82f339be\") " Dec 03 12:04:50 crc kubenswrapper[5052]: I1203 12:04:50.840440 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/ef916b08-df1b-45fe-a024-4d7ece37ca35-httpd-run\") pod \"ef916b08-df1b-45fe-a024-4d7ece37ca35\" (UID: \"ef916b08-df1b-45fe-a024-4d7ece37ca35\") " Dec 03 12:04:50 crc kubenswrapper[5052]: I1203 12:04:50.840486 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ef916b08-df1b-45fe-a024-4d7ece37ca35-combined-ca-bundle\") pod \"ef916b08-df1b-45fe-a024-4d7ece37ca35\" (UID: \"ef916b08-df1b-45fe-a024-4d7ece37ca35\") " Dec 03 12:04:50 crc kubenswrapper[5052]: I1203 12:04:50.840502 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"glance\" (UniqueName: \"kubernetes.io/local-volume/local-storage11-crc\") pod \"ef916b08-df1b-45fe-a024-4d7ece37ca35\" (UID: \"ef916b08-df1b-45fe-a024-4d7ece37ca35\") " Dec 03 12:04:50 crc kubenswrapper[5052]: I1203 12:04:50.840561 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/937cceb6-0e2b-4ee2-8489-97ac82f339be-config-data\") pod \"937cceb6-0e2b-4ee2-8489-97ac82f339be\" (UID: \"937cceb6-0e2b-4ee2-8489-97ac82f339be\") " Dec 03 12:04:50 crc kubenswrapper[5052]: I1203 12:04:50.840593 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/ef916b08-df1b-45fe-a024-4d7ece37ca35-logs\") pod \"ef916b08-df1b-45fe-a024-4d7ece37ca35\" (UID: \"ef916b08-df1b-45fe-a024-4d7ece37ca35\") " Dec 03 12:04:50 crc kubenswrapper[5052]: I1203 12:04:50.840650 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/ef916b08-df1b-45fe-a024-4d7ece37ca35-config-data\") pod \"ef916b08-df1b-45fe-a024-4d7ece37ca35\" (UID: \"ef916b08-df1b-45fe-a024-4d7ece37ca35\") " Dec 03 12:04:50 crc kubenswrapper[5052]: I1203 12:04:50.841132 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/937cceb6-0e2b-4ee2-8489-97ac82f339be-httpd-run" (OuterVolumeSpecName: "httpd-run") pod "937cceb6-0e2b-4ee2-8489-97ac82f339be" (UID: "937cceb6-0e2b-4ee2-8489-97ac82f339be"). InnerVolumeSpecName "httpd-run". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:04:50 crc kubenswrapper[5052]: I1203 12:04:50.841207 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/937cceb6-0e2b-4ee2-8489-97ac82f339be-logs" (OuterVolumeSpecName: "logs") pod "937cceb6-0e2b-4ee2-8489-97ac82f339be" (UID: "937cceb6-0e2b-4ee2-8489-97ac82f339be"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:04:50 crc kubenswrapper[5052]: I1203 12:04:50.841222 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ef916b08-df1b-45fe-a024-4d7ece37ca35-httpd-run" (OuterVolumeSpecName: "httpd-run") pod "ef916b08-df1b-45fe-a024-4d7ece37ca35" (UID: "ef916b08-df1b-45fe-a024-4d7ece37ca35"). InnerVolumeSpecName "httpd-run". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:04:50 crc kubenswrapper[5052]: I1203 12:04:50.841340 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ef916b08-df1b-45fe-a024-4d7ece37ca35-logs" (OuterVolumeSpecName: "logs") pod "ef916b08-df1b-45fe-a024-4d7ece37ca35" (UID: "ef916b08-df1b-45fe-a024-4d7ece37ca35"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:04:50 crc kubenswrapper[5052]: I1203 12:04:50.842878 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/ef916b08-df1b-45fe-a024-4d7ece37ca35-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:04:50 crc kubenswrapper[5052]: I1203 12:04:50.842906 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/937cceb6-0e2b-4ee2-8489-97ac82f339be-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:04:50 crc kubenswrapper[5052]: I1203 12:04:50.842919 5052 reconciler_common.go:293] "Volume detached for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/937cceb6-0e2b-4ee2-8489-97ac82f339be-httpd-run\") on node \"crc\" DevicePath \"\"" Dec 03 12:04:50 crc kubenswrapper[5052]: I1203 12:04:50.842931 5052 reconciler_common.go:293] "Volume detached for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/ef916b08-df1b-45fe-a024-4d7ece37ca35-httpd-run\") on node \"crc\" DevicePath \"\"" Dec 03 12:04:50 crc kubenswrapper[5052]: I1203 12:04:50.845960 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ef916b08-df1b-45fe-a024-4d7ece37ca35-scripts" (OuterVolumeSpecName: "scripts") pod "ef916b08-df1b-45fe-a024-4d7ece37ca35" (UID: "ef916b08-df1b-45fe-a024-4d7ece37ca35"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:04:50 crc kubenswrapper[5052]: I1203 12:04:50.846102 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage11-crc" (OuterVolumeSpecName: "glance") pod "ef916b08-df1b-45fe-a024-4d7ece37ca35" (UID: "ef916b08-df1b-45fe-a024-4d7ece37ca35"). InnerVolumeSpecName "local-storage11-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 12:04:50 crc kubenswrapper[5052]: I1203 12:04:50.846242 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ef916b08-df1b-45fe-a024-4d7ece37ca35-kube-api-access-fb4ng" (OuterVolumeSpecName: "kube-api-access-fb4ng") pod "ef916b08-df1b-45fe-a024-4d7ece37ca35" (UID: "ef916b08-df1b-45fe-a024-4d7ece37ca35"). InnerVolumeSpecName "kube-api-access-fb4ng". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:04:50 crc kubenswrapper[5052]: I1203 12:04:50.846878 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/937cceb6-0e2b-4ee2-8489-97ac82f339be-kube-api-access-mspt8" (OuterVolumeSpecName: "kube-api-access-mspt8") pod "937cceb6-0e2b-4ee2-8489-97ac82f339be" (UID: "937cceb6-0e2b-4ee2-8489-97ac82f339be"). InnerVolumeSpecName "kube-api-access-mspt8". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:04:50 crc kubenswrapper[5052]: I1203 12:04:50.847205 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/937cceb6-0e2b-4ee2-8489-97ac82f339be-scripts" (OuterVolumeSpecName: "scripts") pod "937cceb6-0e2b-4ee2-8489-97ac82f339be" (UID: "937cceb6-0e2b-4ee2-8489-97ac82f339be"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:04:50 crc kubenswrapper[5052]: I1203 12:04:50.868736 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage13-crc" (OuterVolumeSpecName: "glance") pod "937cceb6-0e2b-4ee2-8489-97ac82f339be" (UID: "937cceb6-0e2b-4ee2-8489-97ac82f339be"). InnerVolumeSpecName "local-storage13-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 12:04:50 crc kubenswrapper[5052]: I1203 12:04:50.874116 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/937cceb6-0e2b-4ee2-8489-97ac82f339be-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "937cceb6-0e2b-4ee2-8489-97ac82f339be" (UID: "937cceb6-0e2b-4ee2-8489-97ac82f339be"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:04:50 crc kubenswrapper[5052]: I1203 12:04:50.876185 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ef916b08-df1b-45fe-a024-4d7ece37ca35-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "ef916b08-df1b-45fe-a024-4d7ece37ca35" (UID: "ef916b08-df1b-45fe-a024-4d7ece37ca35"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:04:50 crc kubenswrapper[5052]: I1203 12:04:50.892707 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/937cceb6-0e2b-4ee2-8489-97ac82f339be-config-data" (OuterVolumeSpecName: "config-data") pod "937cceb6-0e2b-4ee2-8489-97ac82f339be" (UID: "937cceb6-0e2b-4ee2-8489-97ac82f339be"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:04:50 crc kubenswrapper[5052]: I1203 12:04:50.902845 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ef916b08-df1b-45fe-a024-4d7ece37ca35-config-data" (OuterVolumeSpecName: "config-data") pod "ef916b08-df1b-45fe-a024-4d7ece37ca35" (UID: "ef916b08-df1b-45fe-a024-4d7ece37ca35"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:04:50 crc kubenswrapper[5052]: I1203 12:04:50.944949 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ef916b08-df1b-45fe-a024-4d7ece37ca35-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:04:50 crc kubenswrapper[5052]: I1203 12:04:50.945017 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage11-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage11-crc\") on node \"crc\" " Dec 03 12:04:50 crc kubenswrapper[5052]: I1203 12:04:50.945028 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/937cceb6-0e2b-4ee2-8489-97ac82f339be-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:04:50 crc kubenswrapper[5052]: I1203 12:04:50.945037 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/ef916b08-df1b-45fe-a024-4d7ece37ca35-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:04:50 crc kubenswrapper[5052]: I1203 12:04:50.945045 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/937cceb6-0e2b-4ee2-8489-97ac82f339be-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:04:50 crc kubenswrapper[5052]: I1203 12:04:50.945057 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-fb4ng\" (UniqueName: \"kubernetes.io/projected/ef916b08-df1b-45fe-a024-4d7ece37ca35-kube-api-access-fb4ng\") on node \"crc\" DevicePath \"\"" Dec 03 12:04:50 crc kubenswrapper[5052]: I1203 12:04:50.945068 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/937cceb6-0e2b-4ee2-8489-97ac82f339be-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:04:50 crc kubenswrapper[5052]: I1203 12:04:50.945082 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage13-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage13-crc\") on node \"crc\" " Dec 03 12:04:50 crc kubenswrapper[5052]: I1203 12:04:50.945091 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/ef916b08-df1b-45fe-a024-4d7ece37ca35-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:04:50 crc kubenswrapper[5052]: I1203 12:04:50.945099 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-mspt8\" (UniqueName: \"kubernetes.io/projected/937cceb6-0e2b-4ee2-8489-97ac82f339be-kube-api-access-mspt8\") on node \"crc\" DevicePath \"\"" Dec 03 12:04:50 crc kubenswrapper[5052]: I1203 12:04:50.965412 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage13-crc" (UniqueName: "kubernetes.io/local-volume/local-storage13-crc") on node "crc" Dec 03 12:04:50 crc kubenswrapper[5052]: I1203 12:04:50.967092 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage11-crc" (UniqueName: "kubernetes.io/local-volume/local-storage11-crc") on node "crc" Dec 03 12:04:51 crc kubenswrapper[5052]: I1203 12:04:51.046731 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage13-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage13-crc\") on node \"crc\" DevicePath \"\"" Dec 03 12:04:51 crc kubenswrapper[5052]: I1203 12:04:51.046767 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage11-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage11-crc\") on node \"crc\" DevicePath \"\"" Dec 03 12:04:51 crc kubenswrapper[5052]: I1203 12:04:51.348646 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:04:51 crc kubenswrapper[5052]: I1203 12:04:51.348655 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"937cceb6-0e2b-4ee2-8489-97ac82f339be","Type":"ContainerDied","Data":"8ea0216c8303a3528a1f7dabcb1d6903e23c1d9e6c8c3669f05ea14a2c2ac5d5"} Dec 03 12:04:51 crc kubenswrapper[5052]: I1203 12:04:51.348782 5052 scope.go:117] "RemoveContainer" containerID="fc1c4042d0bcee43afcb9e370be77b96062717361a92c6021f304dabbd0e4c70" Dec 03 12:04:51 crc kubenswrapper[5052]: I1203 12:04:51.354127 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:04:51 crc kubenswrapper[5052]: I1203 12:04:51.354162 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"ef916b08-df1b-45fe-a024-4d7ece37ca35","Type":"ContainerDied","Data":"b09f44cf3dc4428f602a669ca73f7926e592091776bca76b1c66e07ddfc405fa"} Dec 03 12:04:51 crc kubenswrapper[5052]: I1203 12:04:51.357416 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"356db4d6-f512-4a47-a6ed-a288a212227d","Type":"ContainerStarted","Data":"78ef929a46d2f2f15380f2fa305de26ae4c7ce59bc2b5ad64c4d9ff26db7f97f"} Dec 03 12:04:51 crc kubenswrapper[5052]: I1203 12:04:51.380129 5052 scope.go:117] "RemoveContainer" containerID="758ebf28e00662b6ef188c25f34abaadb9427993d6fbb5f5dc644be7f7fe7652" Dec 03 12:04:51 crc kubenswrapper[5052]: I1203 12:04:51.386395 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:04:51 crc kubenswrapper[5052]: I1203 12:04:51.397635 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:04:51 crc kubenswrapper[5052]: I1203 12:04:51.410629 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:04:51 crc kubenswrapper[5052]: I1203 12:04:51.417138 5052 scope.go:117] "RemoveContainer" containerID="cd6f206671d36b56e19fffadbde7b469d524db4f3799bd51d0ee923dbfb51162" Dec 03 12:04:51 crc kubenswrapper[5052]: I1203 12:04:51.437887 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:04:51 crc kubenswrapper[5052]: I1203 12:04:51.437962 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:04:51 crc kubenswrapper[5052]: E1203 12:04:51.438377 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="ede81f70-0e50-47b2-85b7-d30ec858e2e4" containerName="extract-utilities" Dec 03 12:04:51 crc kubenswrapper[5052]: I1203 12:04:51.438390 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="ede81f70-0e50-47b2-85b7-d30ec858e2e4" containerName="extract-utilities" Dec 03 12:04:51 crc kubenswrapper[5052]: E1203 12:04:51.438401 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="ef916b08-df1b-45fe-a024-4d7ece37ca35" containerName="glance-log" Dec 03 12:04:51 crc kubenswrapper[5052]: I1203 12:04:51.438407 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="ef916b08-df1b-45fe-a024-4d7ece37ca35" containerName="glance-log" Dec 03 12:04:51 crc kubenswrapper[5052]: E1203 12:04:51.438425 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="937cceb6-0e2b-4ee2-8489-97ac82f339be" containerName="glance-httpd" Dec 03 12:04:51 crc kubenswrapper[5052]: I1203 12:04:51.438431 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="937cceb6-0e2b-4ee2-8489-97ac82f339be" containerName="glance-httpd" Dec 03 12:04:51 crc kubenswrapper[5052]: E1203 12:04:51.438445 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="937cceb6-0e2b-4ee2-8489-97ac82f339be" containerName="glance-log" Dec 03 12:04:51 crc kubenswrapper[5052]: I1203 12:04:51.438451 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="937cceb6-0e2b-4ee2-8489-97ac82f339be" containerName="glance-log" Dec 03 12:04:51 crc kubenswrapper[5052]: E1203 12:04:51.438462 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="ef916b08-df1b-45fe-a024-4d7ece37ca35" containerName="glance-httpd" Dec 03 12:04:51 crc kubenswrapper[5052]: I1203 12:04:51.438468 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="ef916b08-df1b-45fe-a024-4d7ece37ca35" containerName="glance-httpd" Dec 03 12:04:51 crc kubenswrapper[5052]: E1203 12:04:51.438483 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="ede81f70-0e50-47b2-85b7-d30ec858e2e4" containerName="extract-content" Dec 03 12:04:51 crc kubenswrapper[5052]: I1203 12:04:51.438489 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="ede81f70-0e50-47b2-85b7-d30ec858e2e4" containerName="extract-content" Dec 03 12:04:51 crc kubenswrapper[5052]: E1203 12:04:51.438498 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="ede81f70-0e50-47b2-85b7-d30ec858e2e4" containerName="registry-server" Dec 03 12:04:51 crc kubenswrapper[5052]: I1203 12:04:51.438505 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="ede81f70-0e50-47b2-85b7-d30ec858e2e4" containerName="registry-server" Dec 03 12:04:51 crc kubenswrapper[5052]: I1203 12:04:51.438705 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="ef916b08-df1b-45fe-a024-4d7ece37ca35" containerName="glance-httpd" Dec 03 12:04:51 crc kubenswrapper[5052]: I1203 12:04:51.438733 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="937cceb6-0e2b-4ee2-8489-97ac82f339be" containerName="glance-httpd" Dec 03 12:04:51 crc kubenswrapper[5052]: I1203 12:04:51.438744 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="937cceb6-0e2b-4ee2-8489-97ac82f339be" containerName="glance-log" Dec 03 12:04:51 crc kubenswrapper[5052]: I1203 12:04:51.438752 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="ef916b08-df1b-45fe-a024-4d7ece37ca35" containerName="glance-log" Dec 03 12:04:51 crc kubenswrapper[5052]: I1203 12:04:51.438768 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="ede81f70-0e50-47b2-85b7-d30ec858e2e4" containerName="registry-server" Dec 03 12:04:51 crc kubenswrapper[5052]: I1203 12:04:51.439755 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:04:51 crc kubenswrapper[5052]: I1203 12:04:51.439842 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:04:51 crc kubenswrapper[5052]: I1203 12:04:51.467244 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-scripts" Dec 03 12:04:51 crc kubenswrapper[5052]: I1203 12:04:51.467499 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-default-internal-config-data" Dec 03 12:04:51 crc kubenswrapper[5052]: I1203 12:04:51.472819 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-glance-default-internal-svc" Dec 03 12:04:51 crc kubenswrapper[5052]: I1203 12:04:51.475826 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-glance-dockercfg-x6d26" Dec 03 12:04:51 crc kubenswrapper[5052]: I1203 12:04:51.503741 5052 scope.go:117] "RemoveContainer" containerID="4bae4ed58a5f95a7350a39735d066875a32ab0eec38427974d5e6223ed2cda10" Dec 03 12:04:51 crc kubenswrapper[5052]: I1203 12:04:51.512939 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:04:51 crc kubenswrapper[5052]: I1203 12:04:51.514672 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:04:51 crc kubenswrapper[5052]: I1203 12:04:51.517232 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-glance-default-public-svc" Dec 03 12:04:51 crc kubenswrapper[5052]: I1203 12:04:51.518989 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-default-external-config-data" Dec 03 12:04:51 crc kubenswrapper[5052]: I1203 12:04:51.537084 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:04:51 crc kubenswrapper[5052]: I1203 12:04:51.566928 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a3f4c9a7-c441-4487-9814-486ac5e66848-config-data\") pod \"glance-default-internal-api-0\" (UID: \"a3f4c9a7-c441-4487-9814-486ac5e66848\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:04:51 crc kubenswrapper[5052]: I1203 12:04:51.567012 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/a3f4c9a7-c441-4487-9814-486ac5e66848-logs\") pod \"glance-default-internal-api-0\" (UID: \"a3f4c9a7-c441-4487-9814-486ac5e66848\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:04:51 crc kubenswrapper[5052]: I1203 12:04:51.567048 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/a3f4c9a7-c441-4487-9814-486ac5e66848-httpd-run\") pod \"glance-default-internal-api-0\" (UID: \"a3f4c9a7-c441-4487-9814-486ac5e66848\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:04:51 crc kubenswrapper[5052]: I1203 12:04:51.567071 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage13-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage13-crc\") pod \"glance-default-internal-api-0\" (UID: \"a3f4c9a7-c441-4487-9814-486ac5e66848\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:04:51 crc kubenswrapper[5052]: I1203 12:04:51.567091 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q9g8p\" (UniqueName: \"kubernetes.io/projected/a3f4c9a7-c441-4487-9814-486ac5e66848-kube-api-access-q9g8p\") pod \"glance-default-internal-api-0\" (UID: \"a3f4c9a7-c441-4487-9814-486ac5e66848\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:04:51 crc kubenswrapper[5052]: I1203 12:04:51.567119 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a3f4c9a7-c441-4487-9814-486ac5e66848-combined-ca-bundle\") pod \"glance-default-internal-api-0\" (UID: \"a3f4c9a7-c441-4487-9814-486ac5e66848\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:04:51 crc kubenswrapper[5052]: I1203 12:04:51.567166 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/a3f4c9a7-c441-4487-9814-486ac5e66848-scripts\") pod \"glance-default-internal-api-0\" (UID: \"a3f4c9a7-c441-4487-9814-486ac5e66848\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:04:51 crc kubenswrapper[5052]: I1203 12:04:51.567260 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/a3f4c9a7-c441-4487-9814-486ac5e66848-internal-tls-certs\") pod \"glance-default-internal-api-0\" (UID: \"a3f4c9a7-c441-4487-9814-486ac5e66848\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:04:51 crc kubenswrapper[5052]: I1203 12:04:51.668888 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d2588f8c-9004-4e6e-8dd5-c6234b94d5cf-combined-ca-bundle\") pod \"glance-default-external-api-0\" (UID: \"d2588f8c-9004-4e6e-8dd5-c6234b94d5cf\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:04:51 crc kubenswrapper[5052]: I1203 12:04:51.669005 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/a3f4c9a7-c441-4487-9814-486ac5e66848-internal-tls-certs\") pod \"glance-default-internal-api-0\" (UID: \"a3f4c9a7-c441-4487-9814-486ac5e66848\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:04:51 crc kubenswrapper[5052]: I1203 12:04:51.669087 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/d2588f8c-9004-4e6e-8dd5-c6234b94d5cf-public-tls-certs\") pod \"glance-default-external-api-0\" (UID: \"d2588f8c-9004-4e6e-8dd5-c6234b94d5cf\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:04:51 crc kubenswrapper[5052]: I1203 12:04:51.669130 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage11-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage11-crc\") pod \"glance-default-external-api-0\" (UID: \"d2588f8c-9004-4e6e-8dd5-c6234b94d5cf\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:04:51 crc kubenswrapper[5052]: I1203 12:04:51.669163 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a3f4c9a7-c441-4487-9814-486ac5e66848-config-data\") pod \"glance-default-internal-api-0\" (UID: \"a3f4c9a7-c441-4487-9814-486ac5e66848\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:04:51 crc kubenswrapper[5052]: I1203 12:04:51.669220 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/d2588f8c-9004-4e6e-8dd5-c6234b94d5cf-httpd-run\") pod \"glance-default-external-api-0\" (UID: \"d2588f8c-9004-4e6e-8dd5-c6234b94d5cf\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:04:51 crc kubenswrapper[5052]: I1203 12:04:51.669248 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/a3f4c9a7-c441-4487-9814-486ac5e66848-logs\") pod \"glance-default-internal-api-0\" (UID: \"a3f4c9a7-c441-4487-9814-486ac5e66848\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:04:51 crc kubenswrapper[5052]: I1203 12:04:51.669273 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/d2588f8c-9004-4e6e-8dd5-c6234b94d5cf-config-data\") pod \"glance-default-external-api-0\" (UID: \"d2588f8c-9004-4e6e-8dd5-c6234b94d5cf\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:04:51 crc kubenswrapper[5052]: I1203 12:04:51.669319 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vrc6m\" (UniqueName: \"kubernetes.io/projected/d2588f8c-9004-4e6e-8dd5-c6234b94d5cf-kube-api-access-vrc6m\") pod \"glance-default-external-api-0\" (UID: \"d2588f8c-9004-4e6e-8dd5-c6234b94d5cf\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:04:51 crc kubenswrapper[5052]: I1203 12:04:51.669346 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/a3f4c9a7-c441-4487-9814-486ac5e66848-httpd-run\") pod \"glance-default-internal-api-0\" (UID: \"a3f4c9a7-c441-4487-9814-486ac5e66848\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:04:51 crc kubenswrapper[5052]: I1203 12:04:51.669375 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage13-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage13-crc\") pod \"glance-default-internal-api-0\" (UID: \"a3f4c9a7-c441-4487-9814-486ac5e66848\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:04:51 crc kubenswrapper[5052]: I1203 12:04:51.669402 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-q9g8p\" (UniqueName: \"kubernetes.io/projected/a3f4c9a7-c441-4487-9814-486ac5e66848-kube-api-access-q9g8p\") pod \"glance-default-internal-api-0\" (UID: \"a3f4c9a7-c441-4487-9814-486ac5e66848\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:04:51 crc kubenswrapper[5052]: I1203 12:04:51.669436 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a3f4c9a7-c441-4487-9814-486ac5e66848-combined-ca-bundle\") pod \"glance-default-internal-api-0\" (UID: \"a3f4c9a7-c441-4487-9814-486ac5e66848\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:04:51 crc kubenswrapper[5052]: I1203 12:04:51.669471 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/d2588f8c-9004-4e6e-8dd5-c6234b94d5cf-scripts\") pod \"glance-default-external-api-0\" (UID: \"d2588f8c-9004-4e6e-8dd5-c6234b94d5cf\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:04:51 crc kubenswrapper[5052]: I1203 12:04:51.669693 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/a3f4c9a7-c441-4487-9814-486ac5e66848-scripts\") pod \"glance-default-internal-api-0\" (UID: \"a3f4c9a7-c441-4487-9814-486ac5e66848\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:04:51 crc kubenswrapper[5052]: I1203 12:04:51.669727 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/d2588f8c-9004-4e6e-8dd5-c6234b94d5cf-logs\") pod \"glance-default-external-api-0\" (UID: \"d2588f8c-9004-4e6e-8dd5-c6234b94d5cf\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:04:51 crc kubenswrapper[5052]: I1203 12:04:51.670365 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage13-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage13-crc\") pod \"glance-default-internal-api-0\" (UID: \"a3f4c9a7-c441-4487-9814-486ac5e66848\") device mount path \"/mnt/openstack/pv13\"" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:04:51 crc kubenswrapper[5052]: I1203 12:04:51.670815 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/a3f4c9a7-c441-4487-9814-486ac5e66848-logs\") pod \"glance-default-internal-api-0\" (UID: \"a3f4c9a7-c441-4487-9814-486ac5e66848\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:04:51 crc kubenswrapper[5052]: I1203 12:04:51.670823 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/a3f4c9a7-c441-4487-9814-486ac5e66848-httpd-run\") pod \"glance-default-internal-api-0\" (UID: \"a3f4c9a7-c441-4487-9814-486ac5e66848\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:04:51 crc kubenswrapper[5052]: I1203 12:04:51.677269 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a3f4c9a7-c441-4487-9814-486ac5e66848-combined-ca-bundle\") pod \"glance-default-internal-api-0\" (UID: \"a3f4c9a7-c441-4487-9814-486ac5e66848\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:04:51 crc kubenswrapper[5052]: I1203 12:04:51.685303 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/a3f4c9a7-c441-4487-9814-486ac5e66848-internal-tls-certs\") pod \"glance-default-internal-api-0\" (UID: \"a3f4c9a7-c441-4487-9814-486ac5e66848\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:04:51 crc kubenswrapper[5052]: I1203 12:04:51.686291 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a3f4c9a7-c441-4487-9814-486ac5e66848-config-data\") pod \"glance-default-internal-api-0\" (UID: \"a3f4c9a7-c441-4487-9814-486ac5e66848\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:04:51 crc kubenswrapper[5052]: I1203 12:04:51.686826 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/a3f4c9a7-c441-4487-9814-486ac5e66848-scripts\") pod \"glance-default-internal-api-0\" (UID: \"a3f4c9a7-c441-4487-9814-486ac5e66848\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:04:51 crc kubenswrapper[5052]: I1203 12:04:51.691985 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-q9g8p\" (UniqueName: \"kubernetes.io/projected/a3f4c9a7-c441-4487-9814-486ac5e66848-kube-api-access-q9g8p\") pod \"glance-default-internal-api-0\" (UID: \"a3f4c9a7-c441-4487-9814-486ac5e66848\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:04:51 crc kubenswrapper[5052]: I1203 12:04:51.699944 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="937cceb6-0e2b-4ee2-8489-97ac82f339be" path="/var/lib/kubelet/pods/937cceb6-0e2b-4ee2-8489-97ac82f339be/volumes" Dec 03 12:04:51 crc kubenswrapper[5052]: I1203 12:04:51.700631 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ef916b08-df1b-45fe-a024-4d7ece37ca35" path="/var/lib/kubelet/pods/ef916b08-df1b-45fe-a024-4d7ece37ca35/volumes" Dec 03 12:04:51 crc kubenswrapper[5052]: I1203 12:04:51.722060 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage13-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage13-crc\") pod \"glance-default-internal-api-0\" (UID: \"a3f4c9a7-c441-4487-9814-486ac5e66848\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:04:51 crc kubenswrapper[5052]: I1203 12:04:51.771098 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/d2588f8c-9004-4e6e-8dd5-c6234b94d5cf-scripts\") pod \"glance-default-external-api-0\" (UID: \"d2588f8c-9004-4e6e-8dd5-c6234b94d5cf\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:04:51 crc kubenswrapper[5052]: I1203 12:04:51.771181 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/d2588f8c-9004-4e6e-8dd5-c6234b94d5cf-logs\") pod \"glance-default-external-api-0\" (UID: \"d2588f8c-9004-4e6e-8dd5-c6234b94d5cf\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:04:51 crc kubenswrapper[5052]: I1203 12:04:51.771211 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d2588f8c-9004-4e6e-8dd5-c6234b94d5cf-combined-ca-bundle\") pod \"glance-default-external-api-0\" (UID: \"d2588f8c-9004-4e6e-8dd5-c6234b94d5cf\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:04:51 crc kubenswrapper[5052]: I1203 12:04:51.771272 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/d2588f8c-9004-4e6e-8dd5-c6234b94d5cf-public-tls-certs\") pod \"glance-default-external-api-0\" (UID: \"d2588f8c-9004-4e6e-8dd5-c6234b94d5cf\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:04:51 crc kubenswrapper[5052]: I1203 12:04:51.771302 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage11-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage11-crc\") pod \"glance-default-external-api-0\" (UID: \"d2588f8c-9004-4e6e-8dd5-c6234b94d5cf\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:04:51 crc kubenswrapper[5052]: I1203 12:04:51.771341 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/d2588f8c-9004-4e6e-8dd5-c6234b94d5cf-httpd-run\") pod \"glance-default-external-api-0\" (UID: \"d2588f8c-9004-4e6e-8dd5-c6234b94d5cf\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:04:51 crc kubenswrapper[5052]: I1203 12:04:51.771363 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/d2588f8c-9004-4e6e-8dd5-c6234b94d5cf-config-data\") pod \"glance-default-external-api-0\" (UID: \"d2588f8c-9004-4e6e-8dd5-c6234b94d5cf\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:04:51 crc kubenswrapper[5052]: I1203 12:04:51.771393 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-vrc6m\" (UniqueName: \"kubernetes.io/projected/d2588f8c-9004-4e6e-8dd5-c6234b94d5cf-kube-api-access-vrc6m\") pod \"glance-default-external-api-0\" (UID: \"d2588f8c-9004-4e6e-8dd5-c6234b94d5cf\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:04:51 crc kubenswrapper[5052]: I1203 12:04:51.772031 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage11-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage11-crc\") pod \"glance-default-external-api-0\" (UID: \"d2588f8c-9004-4e6e-8dd5-c6234b94d5cf\") device mount path \"/mnt/openstack/pv11\"" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:04:51 crc kubenswrapper[5052]: I1203 12:04:51.772118 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/d2588f8c-9004-4e6e-8dd5-c6234b94d5cf-logs\") pod \"glance-default-external-api-0\" (UID: \"d2588f8c-9004-4e6e-8dd5-c6234b94d5cf\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:04:51 crc kubenswrapper[5052]: I1203 12:04:51.772153 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/d2588f8c-9004-4e6e-8dd5-c6234b94d5cf-httpd-run\") pod \"glance-default-external-api-0\" (UID: \"d2588f8c-9004-4e6e-8dd5-c6234b94d5cf\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:04:51 crc kubenswrapper[5052]: I1203 12:04:51.775722 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/d2588f8c-9004-4e6e-8dd5-c6234b94d5cf-scripts\") pod \"glance-default-external-api-0\" (UID: \"d2588f8c-9004-4e6e-8dd5-c6234b94d5cf\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:04:51 crc kubenswrapper[5052]: I1203 12:04:51.776480 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/d2588f8c-9004-4e6e-8dd5-c6234b94d5cf-public-tls-certs\") pod \"glance-default-external-api-0\" (UID: \"d2588f8c-9004-4e6e-8dd5-c6234b94d5cf\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:04:51 crc kubenswrapper[5052]: I1203 12:04:51.777142 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/d2588f8c-9004-4e6e-8dd5-c6234b94d5cf-config-data\") pod \"glance-default-external-api-0\" (UID: \"d2588f8c-9004-4e6e-8dd5-c6234b94d5cf\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:04:51 crc kubenswrapper[5052]: I1203 12:04:51.791457 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d2588f8c-9004-4e6e-8dd5-c6234b94d5cf-combined-ca-bundle\") pod \"glance-default-external-api-0\" (UID: \"d2588f8c-9004-4e6e-8dd5-c6234b94d5cf\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:04:51 crc kubenswrapper[5052]: I1203 12:04:51.794393 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:04:51 crc kubenswrapper[5052]: I1203 12:04:51.795854 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-vrc6m\" (UniqueName: \"kubernetes.io/projected/d2588f8c-9004-4e6e-8dd5-c6234b94d5cf-kube-api-access-vrc6m\") pod \"glance-default-external-api-0\" (UID: \"d2588f8c-9004-4e6e-8dd5-c6234b94d5cf\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:04:51 crc kubenswrapper[5052]: I1203 12:04:51.801523 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage11-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage11-crc\") pod \"glance-default-external-api-0\" (UID: \"d2588f8c-9004-4e6e-8dd5-c6234b94d5cf\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:04:51 crc kubenswrapper[5052]: I1203 12:04:51.833202 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:04:52 crc kubenswrapper[5052]: I1203 12:04:52.261759 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:04:52 crc kubenswrapper[5052]: I1203 12:04:52.358967 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:04:52 crc kubenswrapper[5052]: W1203 12:04:52.367700 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd2588f8c_9004_4e6e_8dd5_c6234b94d5cf.slice/crio-61285e831813c4ec6f3caeed2d0fd767b4b6eff68d9c716bf1c7a2d9fc0b0532 WatchSource:0}: Error finding container 61285e831813c4ec6f3caeed2d0fd767b4b6eff68d9c716bf1c7a2d9fc0b0532: Status 404 returned error can't find the container with id 61285e831813c4ec6f3caeed2d0fd767b4b6eff68d9c716bf1c7a2d9fc0b0532 Dec 03 12:04:52 crc kubenswrapper[5052]: I1203 12:04:52.376921 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"a3f4c9a7-c441-4487-9814-486ac5e66848","Type":"ContainerStarted","Data":"6bf3c46a2247dda992a234acc98ac30c9bd08ad3e213acae31d386e45944ac38"} Dec 03 12:04:53 crc kubenswrapper[5052]: I1203 12:04:53.394775 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"a3f4c9a7-c441-4487-9814-486ac5e66848","Type":"ContainerStarted","Data":"15bd35b56459bfabff2e71300b23da8680d50a0e069d64db131c072ec6fed100"} Dec 03 12:04:53 crc kubenswrapper[5052]: I1203 12:04:53.396083 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"d2588f8c-9004-4e6e-8dd5-c6234b94d5cf","Type":"ContainerStarted","Data":"71f1cba19f2d1076cbc88a6c4acbeb2179613b427473cbf3c46cdde47f7e75b5"} Dec 03 12:04:53 crc kubenswrapper[5052]: I1203 12:04:53.396110 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"d2588f8c-9004-4e6e-8dd5-c6234b94d5cf","Type":"ContainerStarted","Data":"61285e831813c4ec6f3caeed2d0fd767b4b6eff68d9c716bf1c7a2d9fc0b0532"} Dec 03 12:04:54 crc kubenswrapper[5052]: I1203 12:04:54.251539 5052 scope.go:117] "RemoveContainer" containerID="f877a20aa72fb38f5d985123667f657d88840787bd73214d01bf7253e2f1bf64" Dec 03 12:04:54 crc kubenswrapper[5052]: I1203 12:04:54.285020 5052 scope.go:117] "RemoveContainer" containerID="f9982d5e0a6d516106ed549ea5250927607d0748ff6fe35b4feae30164c85aa3" Dec 03 12:04:54 crc kubenswrapper[5052]: I1203 12:04:54.324239 5052 scope.go:117] "RemoveContainer" containerID="eac316d032f9e574c9c5551e5d62e39ad5141fe5bd8033f5ce21da3b31105028" Dec 03 12:04:54 crc kubenswrapper[5052]: I1203 12:04:54.354390 5052 scope.go:117] "RemoveContainer" containerID="8ddd0225baa5c9e8b247c70bd8287b4bcb735d619e3bc97d77620dc899e06c7e" Dec 03 12:04:54 crc kubenswrapper[5052]: I1203 12:04:54.476112 5052 scope.go:117] "RemoveContainer" containerID="70d329d677837ae32ed6ea2cf9d76bcf62eb95f82373e7dfdeacdfcbca06156a" Dec 03 12:04:54 crc kubenswrapper[5052]: I1203 12:04:54.513369 5052 scope.go:117] "RemoveContainer" containerID="a77396915b8338d90eddac5eb06104115079d2d8327f9130ff83c33c77a22246" Dec 03 12:04:55 crc kubenswrapper[5052]: I1203 12:04:55.437471 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"356db4d6-f512-4a47-a6ed-a288a212227d","Type":"ContainerStarted","Data":"c14e2561f2a0151538e89b4ddf55c4a97c54d6a62c87067b788b0164b7157f92"} Dec 03 12:04:55 crc kubenswrapper[5052]: I1203 12:04:55.441403 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"a3f4c9a7-c441-4487-9814-486ac5e66848","Type":"ContainerStarted","Data":"e6ef915c623f6539b5a4b98c7c0a747b3dd34c5a9b33f8653603dec931503dbe"} Dec 03 12:04:55 crc kubenswrapper[5052]: I1203 12:04:55.465295 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/glance-default-internal-api-0" podStartSLOduration=4.465274644 podStartE2EDuration="4.465274644s" podCreationTimestamp="2025-12-03 12:04:51 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:04:55.462411362 +0000 UTC m=+2686.050653126" watchObservedRunningTime="2025-12-03 12:04:55.465274644 +0000 UTC m=+2686.053516418" Dec 03 12:04:56 crc kubenswrapper[5052]: I1203 12:04:56.457617 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"d2588f8c-9004-4e6e-8dd5-c6234b94d5cf","Type":"ContainerStarted","Data":"65e485c8f05b257462566a96517913bed283ca6386a0edf4f3a7a2023ee080ce"} Dec 03 12:04:56 crc kubenswrapper[5052]: I1203 12:04:56.488927 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/glance-default-external-api-0" podStartSLOduration=5.488905499 podStartE2EDuration="5.488905499s" podCreationTimestamp="2025-12-03 12:04:51 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:04:56.47845377 +0000 UTC m=+2687.066695544" watchObservedRunningTime="2025-12-03 12:04:56.488905499 +0000 UTC m=+2687.077147283" Dec 03 12:04:59 crc kubenswrapper[5052]: I1203 12:04:59.379890 5052 patch_prober.go:28] interesting pod/machine-config-daemon-74bj6 container/machine-config-daemon namespace/openshift-machine-config-operator: Liveness probe status=failure output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" start-of-body= Dec 03 12:04:59 crc kubenswrapper[5052]: I1203 12:04:59.380375 5052 prober.go:107] "Probe failed" probeType="Liveness" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" containerName="machine-config-daemon" probeResult="failure" output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" Dec 03 12:05:00 crc kubenswrapper[5052]: I1203 12:05:00.510108 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"356db4d6-f512-4a47-a6ed-a288a212227d","Type":"ContainerStarted","Data":"9b957ab19135dbc24d351d0e5e7f814c7cae8606d77e42f40f94f74348eb9d88"} Dec 03 12:05:00 crc kubenswrapper[5052]: I1203 12:05:00.510278 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="356db4d6-f512-4a47-a6ed-a288a212227d" containerName="ceilometer-central-agent" containerID="cri-o://9a53387b8a8a36212c1684331e46cebdf007ce9e1dcfa9efacbd247058d14e91" gracePeriod=30 Dec 03 12:05:00 crc kubenswrapper[5052]: I1203 12:05:00.510613 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:05:00 crc kubenswrapper[5052]: I1203 12:05:00.510916 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="356db4d6-f512-4a47-a6ed-a288a212227d" containerName="proxy-httpd" containerID="cri-o://9b957ab19135dbc24d351d0e5e7f814c7cae8606d77e42f40f94f74348eb9d88" gracePeriod=30 Dec 03 12:05:00 crc kubenswrapper[5052]: I1203 12:05:00.510964 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="356db4d6-f512-4a47-a6ed-a288a212227d" containerName="sg-core" containerID="cri-o://c14e2561f2a0151538e89b4ddf55c4a97c54d6a62c87067b788b0164b7157f92" gracePeriod=30 Dec 03 12:05:00 crc kubenswrapper[5052]: I1203 12:05:00.511006 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="356db4d6-f512-4a47-a6ed-a288a212227d" containerName="ceilometer-notification-agent" containerID="cri-o://78ef929a46d2f2f15380f2fa305de26ae4c7ce59bc2b5ad64c4d9ff26db7f97f" gracePeriod=30 Dec 03 12:05:00 crc kubenswrapper[5052]: I1203 12:05:00.538129 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/ceilometer-0" podStartSLOduration=3.112656449 podStartE2EDuration="31.538113454s" podCreationTimestamp="2025-12-03 12:04:29 +0000 UTC" firstStartedPulling="2025-12-03 12:04:30.60867376 +0000 UTC m=+2661.196915524" lastFinishedPulling="2025-12-03 12:04:59.034130765 +0000 UTC m=+2689.622372529" observedRunningTime="2025-12-03 12:05:00.531448153 +0000 UTC m=+2691.119689917" watchObservedRunningTime="2025-12-03 12:05:00.538113454 +0000 UTC m=+2691.126355218" Dec 03 12:05:01 crc kubenswrapper[5052]: I1203 12:05:01.524926 5052 generic.go:334] "Generic (PLEG): container finished" podID="356db4d6-f512-4a47-a6ed-a288a212227d" containerID="9b957ab19135dbc24d351d0e5e7f814c7cae8606d77e42f40f94f74348eb9d88" exitCode=0 Dec 03 12:05:01 crc kubenswrapper[5052]: I1203 12:05:01.524968 5052 generic.go:334] "Generic (PLEG): container finished" podID="356db4d6-f512-4a47-a6ed-a288a212227d" containerID="c14e2561f2a0151538e89b4ddf55c4a97c54d6a62c87067b788b0164b7157f92" exitCode=2 Dec 03 12:05:01 crc kubenswrapper[5052]: I1203 12:05:01.524982 5052 generic.go:334] "Generic (PLEG): container finished" podID="356db4d6-f512-4a47-a6ed-a288a212227d" containerID="78ef929a46d2f2f15380f2fa305de26ae4c7ce59bc2b5ad64c4d9ff26db7f97f" exitCode=0 Dec 03 12:05:01 crc kubenswrapper[5052]: I1203 12:05:01.524994 5052 generic.go:334] "Generic (PLEG): container finished" podID="356db4d6-f512-4a47-a6ed-a288a212227d" containerID="9a53387b8a8a36212c1684331e46cebdf007ce9e1dcfa9efacbd247058d14e91" exitCode=0 Dec 03 12:05:01 crc kubenswrapper[5052]: I1203 12:05:01.525001 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"356db4d6-f512-4a47-a6ed-a288a212227d","Type":"ContainerDied","Data":"9b957ab19135dbc24d351d0e5e7f814c7cae8606d77e42f40f94f74348eb9d88"} Dec 03 12:05:01 crc kubenswrapper[5052]: I1203 12:05:01.525056 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"356db4d6-f512-4a47-a6ed-a288a212227d","Type":"ContainerDied","Data":"c14e2561f2a0151538e89b4ddf55c4a97c54d6a62c87067b788b0164b7157f92"} Dec 03 12:05:01 crc kubenswrapper[5052]: I1203 12:05:01.525073 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"356db4d6-f512-4a47-a6ed-a288a212227d","Type":"ContainerDied","Data":"78ef929a46d2f2f15380f2fa305de26ae4c7ce59bc2b5ad64c4d9ff26db7f97f"} Dec 03 12:05:01 crc kubenswrapper[5052]: I1203 12:05:01.525089 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"356db4d6-f512-4a47-a6ed-a288a212227d","Type":"ContainerDied","Data":"9a53387b8a8a36212c1684331e46cebdf007ce9e1dcfa9efacbd247058d14e91"} Dec 03 12:05:01 crc kubenswrapper[5052]: I1203 12:05:01.795451 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:05:01 crc kubenswrapper[5052]: I1203 12:05:01.795905 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:05:01 crc kubenswrapper[5052]: I1203 12:05:01.834314 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:05:01 crc kubenswrapper[5052]: I1203 12:05:01.835006 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:05:01 crc kubenswrapper[5052]: I1203 12:05:01.835059 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:05:01 crc kubenswrapper[5052]: I1203 12:05:01.843345 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:05:01 crc kubenswrapper[5052]: I1203 12:05:01.871864 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:05:01 crc kubenswrapper[5052]: I1203 12:05:01.886234 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:05:01 crc kubenswrapper[5052]: I1203 12:05:01.897368 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:05:02 crc kubenswrapper[5052]: I1203 12:05:02.000296 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/356db4d6-f512-4a47-a6ed-a288a212227d-combined-ca-bundle\") pod \"356db4d6-f512-4a47-a6ed-a288a212227d\" (UID: \"356db4d6-f512-4a47-a6ed-a288a212227d\") " Dec 03 12:05:02 crc kubenswrapper[5052]: I1203 12:05:02.000404 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/356db4d6-f512-4a47-a6ed-a288a212227d-run-httpd\") pod \"356db4d6-f512-4a47-a6ed-a288a212227d\" (UID: \"356db4d6-f512-4a47-a6ed-a288a212227d\") " Dec 03 12:05:02 crc kubenswrapper[5052]: I1203 12:05:02.000428 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/356db4d6-f512-4a47-a6ed-a288a212227d-scripts\") pod \"356db4d6-f512-4a47-a6ed-a288a212227d\" (UID: \"356db4d6-f512-4a47-a6ed-a288a212227d\") " Dec 03 12:05:02 crc kubenswrapper[5052]: I1203 12:05:02.000587 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pd5ht\" (UniqueName: \"kubernetes.io/projected/356db4d6-f512-4a47-a6ed-a288a212227d-kube-api-access-pd5ht\") pod \"356db4d6-f512-4a47-a6ed-a288a212227d\" (UID: \"356db4d6-f512-4a47-a6ed-a288a212227d\") " Dec 03 12:05:02 crc kubenswrapper[5052]: I1203 12:05:02.000607 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/356db4d6-f512-4a47-a6ed-a288a212227d-config-data\") pod \"356db4d6-f512-4a47-a6ed-a288a212227d\" (UID: \"356db4d6-f512-4a47-a6ed-a288a212227d\") " Dec 03 12:05:02 crc kubenswrapper[5052]: I1203 12:05:02.000656 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/356db4d6-f512-4a47-a6ed-a288a212227d-sg-core-conf-yaml\") pod \"356db4d6-f512-4a47-a6ed-a288a212227d\" (UID: \"356db4d6-f512-4a47-a6ed-a288a212227d\") " Dec 03 12:05:02 crc kubenswrapper[5052]: I1203 12:05:02.000700 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/356db4d6-f512-4a47-a6ed-a288a212227d-log-httpd\") pod \"356db4d6-f512-4a47-a6ed-a288a212227d\" (UID: \"356db4d6-f512-4a47-a6ed-a288a212227d\") " Dec 03 12:05:02 crc kubenswrapper[5052]: I1203 12:05:02.001735 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/356db4d6-f512-4a47-a6ed-a288a212227d-run-httpd" (OuterVolumeSpecName: "run-httpd") pod "356db4d6-f512-4a47-a6ed-a288a212227d" (UID: "356db4d6-f512-4a47-a6ed-a288a212227d"). InnerVolumeSpecName "run-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:05:02 crc kubenswrapper[5052]: I1203 12:05:02.001747 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/356db4d6-f512-4a47-a6ed-a288a212227d-log-httpd" (OuterVolumeSpecName: "log-httpd") pod "356db4d6-f512-4a47-a6ed-a288a212227d" (UID: "356db4d6-f512-4a47-a6ed-a288a212227d"). InnerVolumeSpecName "log-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:05:02 crc kubenswrapper[5052]: I1203 12:05:02.008665 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/356db4d6-f512-4a47-a6ed-a288a212227d-scripts" (OuterVolumeSpecName: "scripts") pod "356db4d6-f512-4a47-a6ed-a288a212227d" (UID: "356db4d6-f512-4a47-a6ed-a288a212227d"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:05:02 crc kubenswrapper[5052]: I1203 12:05:02.008852 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/356db4d6-f512-4a47-a6ed-a288a212227d-kube-api-access-pd5ht" (OuterVolumeSpecName: "kube-api-access-pd5ht") pod "356db4d6-f512-4a47-a6ed-a288a212227d" (UID: "356db4d6-f512-4a47-a6ed-a288a212227d"). InnerVolumeSpecName "kube-api-access-pd5ht". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:05:02 crc kubenswrapper[5052]: I1203 12:05:02.031663 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/356db4d6-f512-4a47-a6ed-a288a212227d-sg-core-conf-yaml" (OuterVolumeSpecName: "sg-core-conf-yaml") pod "356db4d6-f512-4a47-a6ed-a288a212227d" (UID: "356db4d6-f512-4a47-a6ed-a288a212227d"). InnerVolumeSpecName "sg-core-conf-yaml". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:05:02 crc kubenswrapper[5052]: I1203 12:05:02.101029 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/356db4d6-f512-4a47-a6ed-a288a212227d-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "356db4d6-f512-4a47-a6ed-a288a212227d" (UID: "356db4d6-f512-4a47-a6ed-a288a212227d"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:05:02 crc kubenswrapper[5052]: I1203 12:05:02.103451 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-pd5ht\" (UniqueName: \"kubernetes.io/projected/356db4d6-f512-4a47-a6ed-a288a212227d-kube-api-access-pd5ht\") on node \"crc\" DevicePath \"\"" Dec 03 12:05:02 crc kubenswrapper[5052]: I1203 12:05:02.103553 5052 reconciler_common.go:293] "Volume detached for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/356db4d6-f512-4a47-a6ed-a288a212227d-sg-core-conf-yaml\") on node \"crc\" DevicePath \"\"" Dec 03 12:05:02 crc kubenswrapper[5052]: I1203 12:05:02.103572 5052 reconciler_common.go:293] "Volume detached for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/356db4d6-f512-4a47-a6ed-a288a212227d-log-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:05:02 crc kubenswrapper[5052]: I1203 12:05:02.103586 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/356db4d6-f512-4a47-a6ed-a288a212227d-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:05:02 crc kubenswrapper[5052]: I1203 12:05:02.103599 5052 reconciler_common.go:293] "Volume detached for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/356db4d6-f512-4a47-a6ed-a288a212227d-run-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:05:02 crc kubenswrapper[5052]: I1203 12:05:02.103613 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/356db4d6-f512-4a47-a6ed-a288a212227d-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:05:02 crc kubenswrapper[5052]: I1203 12:05:02.129530 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/356db4d6-f512-4a47-a6ed-a288a212227d-config-data" (OuterVolumeSpecName: "config-data") pod "356db4d6-f512-4a47-a6ed-a288a212227d" (UID: "356db4d6-f512-4a47-a6ed-a288a212227d"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:05:02 crc kubenswrapper[5052]: I1203 12:05:02.205847 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/356db4d6-f512-4a47-a6ed-a288a212227d-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:05:02 crc kubenswrapper[5052]: I1203 12:05:02.540112 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"356db4d6-f512-4a47-a6ed-a288a212227d","Type":"ContainerDied","Data":"f65552c32a602e4dea0c6aaca246e64e2f01e001bc13c386797292bdda16fba2"} Dec 03 12:05:02 crc kubenswrapper[5052]: I1203 12:05:02.540192 5052 scope.go:117] "RemoveContainer" containerID="9b957ab19135dbc24d351d0e5e7f814c7cae8606d77e42f40f94f74348eb9d88" Dec 03 12:05:02 crc kubenswrapper[5052]: I1203 12:05:02.540210 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:05:02 crc kubenswrapper[5052]: I1203 12:05:02.542266 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:05:02 crc kubenswrapper[5052]: I1203 12:05:02.542367 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:05:02 crc kubenswrapper[5052]: I1203 12:05:02.542381 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:05:02 crc kubenswrapper[5052]: I1203 12:05:02.542409 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:05:02 crc kubenswrapper[5052]: I1203 12:05:02.571243 5052 scope.go:117] "RemoveContainer" containerID="c14e2561f2a0151538e89b4ddf55c4a97c54d6a62c87067b788b0164b7157f92" Dec 03 12:05:02 crc kubenswrapper[5052]: I1203 12:05:02.582678 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:05:02 crc kubenswrapper[5052]: I1203 12:05:02.590116 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:05:02 crc kubenswrapper[5052]: I1203 12:05:02.606405 5052 scope.go:117] "RemoveContainer" containerID="78ef929a46d2f2f15380f2fa305de26ae4c7ce59bc2b5ad64c4d9ff26db7f97f" Dec 03 12:05:02 crc kubenswrapper[5052]: I1203 12:05:02.609464 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:05:02 crc kubenswrapper[5052]: E1203 12:05:02.609990 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="356db4d6-f512-4a47-a6ed-a288a212227d" containerName="sg-core" Dec 03 12:05:02 crc kubenswrapper[5052]: I1203 12:05:02.610016 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="356db4d6-f512-4a47-a6ed-a288a212227d" containerName="sg-core" Dec 03 12:05:02 crc kubenswrapper[5052]: E1203 12:05:02.610046 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="356db4d6-f512-4a47-a6ed-a288a212227d" containerName="ceilometer-notification-agent" Dec 03 12:05:02 crc kubenswrapper[5052]: I1203 12:05:02.610058 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="356db4d6-f512-4a47-a6ed-a288a212227d" containerName="ceilometer-notification-agent" Dec 03 12:05:02 crc kubenswrapper[5052]: E1203 12:05:02.610080 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="356db4d6-f512-4a47-a6ed-a288a212227d" containerName="ceilometer-central-agent" Dec 03 12:05:02 crc kubenswrapper[5052]: I1203 12:05:02.610108 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="356db4d6-f512-4a47-a6ed-a288a212227d" containerName="ceilometer-central-agent" Dec 03 12:05:02 crc kubenswrapper[5052]: E1203 12:05:02.610126 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="356db4d6-f512-4a47-a6ed-a288a212227d" containerName="proxy-httpd" Dec 03 12:05:02 crc kubenswrapper[5052]: I1203 12:05:02.610134 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="356db4d6-f512-4a47-a6ed-a288a212227d" containerName="proxy-httpd" Dec 03 12:05:02 crc kubenswrapper[5052]: I1203 12:05:02.610392 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="356db4d6-f512-4a47-a6ed-a288a212227d" containerName="ceilometer-central-agent" Dec 03 12:05:02 crc kubenswrapper[5052]: I1203 12:05:02.610429 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="356db4d6-f512-4a47-a6ed-a288a212227d" containerName="proxy-httpd" Dec 03 12:05:02 crc kubenswrapper[5052]: I1203 12:05:02.610450 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="356db4d6-f512-4a47-a6ed-a288a212227d" containerName="sg-core" Dec 03 12:05:02 crc kubenswrapper[5052]: I1203 12:05:02.610464 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="356db4d6-f512-4a47-a6ed-a288a212227d" containerName="ceilometer-notification-agent" Dec 03 12:05:02 crc kubenswrapper[5052]: I1203 12:05:02.612906 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:05:02 crc kubenswrapper[5052]: I1203 12:05:02.618325 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-scripts" Dec 03 12:05:02 crc kubenswrapper[5052]: I1203 12:05:02.618348 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-config-data" Dec 03 12:05:02 crc kubenswrapper[5052]: I1203 12:05:02.627132 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:05:02 crc kubenswrapper[5052]: I1203 12:05:02.641037 5052 scope.go:117] "RemoveContainer" containerID="9a53387b8a8a36212c1684331e46cebdf007ce9e1dcfa9efacbd247058d14e91" Dec 03 12:05:02 crc kubenswrapper[5052]: I1203 12:05:02.717060 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/ae2d7638-3ed1-4ab4-8d55-8eb8608bc97e-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"ae2d7638-3ed1-4ab4-8d55-8eb8608bc97e\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:05:02 crc kubenswrapper[5052]: I1203 12:05:02.717122 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/ae2d7638-3ed1-4ab4-8d55-8eb8608bc97e-run-httpd\") pod \"ceilometer-0\" (UID: \"ae2d7638-3ed1-4ab4-8d55-8eb8608bc97e\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:05:02 crc kubenswrapper[5052]: I1203 12:05:02.717161 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ae2d7638-3ed1-4ab4-8d55-8eb8608bc97e-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"ae2d7638-3ed1-4ab4-8d55-8eb8608bc97e\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:05:02 crc kubenswrapper[5052]: I1203 12:05:02.717189 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6d8kl\" (UniqueName: \"kubernetes.io/projected/ae2d7638-3ed1-4ab4-8d55-8eb8608bc97e-kube-api-access-6d8kl\") pod \"ceilometer-0\" (UID: \"ae2d7638-3ed1-4ab4-8d55-8eb8608bc97e\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:05:02 crc kubenswrapper[5052]: I1203 12:05:02.717342 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/ae2d7638-3ed1-4ab4-8d55-8eb8608bc97e-log-httpd\") pod \"ceilometer-0\" (UID: \"ae2d7638-3ed1-4ab4-8d55-8eb8608bc97e\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:05:02 crc kubenswrapper[5052]: I1203 12:05:02.717398 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/ae2d7638-3ed1-4ab4-8d55-8eb8608bc97e-config-data\") pod \"ceilometer-0\" (UID: \"ae2d7638-3ed1-4ab4-8d55-8eb8608bc97e\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:05:02 crc kubenswrapper[5052]: I1203 12:05:02.717427 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/ae2d7638-3ed1-4ab4-8d55-8eb8608bc97e-scripts\") pod \"ceilometer-0\" (UID: \"ae2d7638-3ed1-4ab4-8d55-8eb8608bc97e\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:05:02 crc kubenswrapper[5052]: I1203 12:05:02.819485 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/ae2d7638-3ed1-4ab4-8d55-8eb8608bc97e-log-httpd\") pod \"ceilometer-0\" (UID: \"ae2d7638-3ed1-4ab4-8d55-8eb8608bc97e\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:05:02 crc kubenswrapper[5052]: I1203 12:05:02.819901 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/ae2d7638-3ed1-4ab4-8d55-8eb8608bc97e-log-httpd\") pod \"ceilometer-0\" (UID: \"ae2d7638-3ed1-4ab4-8d55-8eb8608bc97e\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:05:02 crc kubenswrapper[5052]: I1203 12:05:02.820036 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/ae2d7638-3ed1-4ab4-8d55-8eb8608bc97e-config-data\") pod \"ceilometer-0\" (UID: \"ae2d7638-3ed1-4ab4-8d55-8eb8608bc97e\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:05:02 crc kubenswrapper[5052]: I1203 12:05:02.820064 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/ae2d7638-3ed1-4ab4-8d55-8eb8608bc97e-scripts\") pod \"ceilometer-0\" (UID: \"ae2d7638-3ed1-4ab4-8d55-8eb8608bc97e\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:05:02 crc kubenswrapper[5052]: I1203 12:05:02.820879 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/ae2d7638-3ed1-4ab4-8d55-8eb8608bc97e-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"ae2d7638-3ed1-4ab4-8d55-8eb8608bc97e\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:05:02 crc kubenswrapper[5052]: I1203 12:05:02.820920 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/ae2d7638-3ed1-4ab4-8d55-8eb8608bc97e-run-httpd\") pod \"ceilometer-0\" (UID: \"ae2d7638-3ed1-4ab4-8d55-8eb8608bc97e\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:05:02 crc kubenswrapper[5052]: I1203 12:05:02.820952 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ae2d7638-3ed1-4ab4-8d55-8eb8608bc97e-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"ae2d7638-3ed1-4ab4-8d55-8eb8608bc97e\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:05:02 crc kubenswrapper[5052]: I1203 12:05:02.820976 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-6d8kl\" (UniqueName: \"kubernetes.io/projected/ae2d7638-3ed1-4ab4-8d55-8eb8608bc97e-kube-api-access-6d8kl\") pod \"ceilometer-0\" (UID: \"ae2d7638-3ed1-4ab4-8d55-8eb8608bc97e\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:05:02 crc kubenswrapper[5052]: I1203 12:05:02.822375 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/ae2d7638-3ed1-4ab4-8d55-8eb8608bc97e-run-httpd\") pod \"ceilometer-0\" (UID: \"ae2d7638-3ed1-4ab4-8d55-8eb8608bc97e\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:05:02 crc kubenswrapper[5052]: I1203 12:05:02.824052 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/ae2d7638-3ed1-4ab4-8d55-8eb8608bc97e-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"ae2d7638-3ed1-4ab4-8d55-8eb8608bc97e\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:05:02 crc kubenswrapper[5052]: I1203 12:05:02.824500 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/ae2d7638-3ed1-4ab4-8d55-8eb8608bc97e-config-data\") pod \"ceilometer-0\" (UID: \"ae2d7638-3ed1-4ab4-8d55-8eb8608bc97e\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:05:02 crc kubenswrapper[5052]: I1203 12:05:02.825133 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/ae2d7638-3ed1-4ab4-8d55-8eb8608bc97e-scripts\") pod \"ceilometer-0\" (UID: \"ae2d7638-3ed1-4ab4-8d55-8eb8608bc97e\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:05:02 crc kubenswrapper[5052]: I1203 12:05:02.825707 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ae2d7638-3ed1-4ab4-8d55-8eb8608bc97e-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"ae2d7638-3ed1-4ab4-8d55-8eb8608bc97e\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:05:02 crc kubenswrapper[5052]: I1203 12:05:02.844043 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-6d8kl\" (UniqueName: \"kubernetes.io/projected/ae2d7638-3ed1-4ab4-8d55-8eb8608bc97e-kube-api-access-6d8kl\") pod \"ceilometer-0\" (UID: \"ae2d7638-3ed1-4ab4-8d55-8eb8608bc97e\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:05:02 crc kubenswrapper[5052]: I1203 12:05:02.941904 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:05:03 crc kubenswrapper[5052]: I1203 12:05:03.376924 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:05:03 crc kubenswrapper[5052]: W1203 12:05:03.379353 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podae2d7638_3ed1_4ab4_8d55_8eb8608bc97e.slice/crio-5d95d1e48905db091a08b53cc676eb9b778282ad0e6bd81957f6f330de69f1c8 WatchSource:0}: Error finding container 5d95d1e48905db091a08b53cc676eb9b778282ad0e6bd81957f6f330de69f1c8: Status 404 returned error can't find the container with id 5d95d1e48905db091a08b53cc676eb9b778282ad0e6bd81957f6f330de69f1c8 Dec 03 12:05:03 crc kubenswrapper[5052]: I1203 12:05:03.562320 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"ae2d7638-3ed1-4ab4-8d55-8eb8608bc97e","Type":"ContainerStarted","Data":"5d95d1e48905db091a08b53cc676eb9b778282ad0e6bd81957f6f330de69f1c8"} Dec 03 12:05:03 crc kubenswrapper[5052]: I1203 12:05:03.685694 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="356db4d6-f512-4a47-a6ed-a288a212227d" path="/var/lib/kubelet/pods/356db4d6-f512-4a47-a6ed-a288a212227d/volumes" Dec 03 12:05:04 crc kubenswrapper[5052]: I1203 12:05:04.580015 5052 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Dec 03 12:05:04 crc kubenswrapper[5052]: I1203 12:05:04.580431 5052 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Dec 03 12:05:04 crc kubenswrapper[5052]: I1203 12:05:04.693829 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:05:04 crc kubenswrapper[5052]: I1203 12:05:04.851400 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:05:04 crc kubenswrapper[5052]: I1203 12:05:04.851906 5052 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Dec 03 12:05:04 crc kubenswrapper[5052]: I1203 12:05:04.852989 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:05:04 crc kubenswrapper[5052]: I1203 12:05:04.871022 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:05:05 crc kubenswrapper[5052]: I1203 12:05:05.598258 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"ae2d7638-3ed1-4ab4-8d55-8eb8608bc97e","Type":"ContainerStarted","Data":"aa090b5d18b872f74a8114d8dca0ba1dcc3cf87025d6bf6e190621c1bc9960c4"} Dec 03 12:05:06 crc kubenswrapper[5052]: I1203 12:05:06.611068 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"ae2d7638-3ed1-4ab4-8d55-8eb8608bc97e","Type":"ContainerStarted","Data":"c1b2b526b75e2f6e7dba586cd66a185e3baf50dddd08f9f766d7c67ae4aab2dc"} Dec 03 12:05:07 crc kubenswrapper[5052]: I1203 12:05:07.625171 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"ae2d7638-3ed1-4ab4-8d55-8eb8608bc97e","Type":"ContainerStarted","Data":"e90e2d7c78b50d4ab2e506730826ae6cbc28fbf2dc044144c3b2d062e11c08c5"} Dec 03 12:05:08 crc kubenswrapper[5052]: I1203 12:05:08.635532 5052 generic.go:334] "Generic (PLEG): container finished" podID="3d2ba247-e370-4d62-a34e-5b05174363fb" containerID="a23efe47ae844777245a54dff972169031df4ea13092e5aabb04cc320ac712df" exitCode=0 Dec 03 12:05:08 crc kubenswrapper[5052]: I1203 12:05:08.635630 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-bootstrap-wbwms" event={"ID":"3d2ba247-e370-4d62-a34e-5b05174363fb","Type":"ContainerDied","Data":"a23efe47ae844777245a54dff972169031df4ea13092e5aabb04cc320ac712df"} Dec 03 12:05:09 crc kubenswrapper[5052]: I1203 12:05:09.648596 5052 generic.go:334] "Generic (PLEG): container finished" podID="ab039824-fe2a-44ba-ae0f-302e5c4ea614" containerID="32468dac9534ebe102ff57ab3a6ff7547398603ee630025eaac80c0d8f9678c8" exitCode=0 Dec 03 12:05:09 crc kubenswrapper[5052]: I1203 12:05:09.648685 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-db-sync-852kv" event={"ID":"ab039824-fe2a-44ba-ae0f-302e5c4ea614","Type":"ContainerDied","Data":"32468dac9534ebe102ff57ab3a6ff7547398603ee630025eaac80c0d8f9678c8"} Dec 03 12:05:09 crc kubenswrapper[5052]: I1203 12:05:09.652222 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"ae2d7638-3ed1-4ab4-8d55-8eb8608bc97e","Type":"ContainerStarted","Data":"cb3e22986020c181b5ae6726aae34a38e1bcb7a291bf69547b51d5917d2e744c"} Dec 03 12:05:09 crc kubenswrapper[5052]: I1203 12:05:09.701959 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/ceilometer-0" podStartSLOduration=2.3380371110000002 podStartE2EDuration="7.701940621s" podCreationTimestamp="2025-12-03 12:05:02 +0000 UTC" firstStartedPulling="2025-12-03 12:05:03.38379973 +0000 UTC m=+2693.972041484" lastFinishedPulling="2025-12-03 12:05:08.74770323 +0000 UTC m=+2699.335944994" observedRunningTime="2025-12-03 12:05:09.696432134 +0000 UTC m=+2700.284673898" watchObservedRunningTime="2025-12-03 12:05:09.701940621 +0000 UTC m=+2700.290182395" Dec 03 12:05:10 crc kubenswrapper[5052]: I1203 12:05:10.020100 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-bootstrap-wbwms" Dec 03 12:05:10 crc kubenswrapper[5052]: I1203 12:05:10.165831 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bvbl6\" (UniqueName: \"kubernetes.io/projected/3d2ba247-e370-4d62-a34e-5b05174363fb-kube-api-access-bvbl6\") pod \"3d2ba247-e370-4d62-a34e-5b05174363fb\" (UID: \"3d2ba247-e370-4d62-a34e-5b05174363fb\") " Dec 03 12:05:10 crc kubenswrapper[5052]: I1203 12:05:10.166023 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/3d2ba247-e370-4d62-a34e-5b05174363fb-fernet-keys\") pod \"3d2ba247-e370-4d62-a34e-5b05174363fb\" (UID: \"3d2ba247-e370-4d62-a34e-5b05174363fb\") " Dec 03 12:05:10 crc kubenswrapper[5052]: I1203 12:05:10.166072 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/3d2ba247-e370-4d62-a34e-5b05174363fb-credential-keys\") pod \"3d2ba247-e370-4d62-a34e-5b05174363fb\" (UID: \"3d2ba247-e370-4d62-a34e-5b05174363fb\") " Dec 03 12:05:10 crc kubenswrapper[5052]: I1203 12:05:10.166110 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/3d2ba247-e370-4d62-a34e-5b05174363fb-scripts\") pod \"3d2ba247-e370-4d62-a34e-5b05174363fb\" (UID: \"3d2ba247-e370-4d62-a34e-5b05174363fb\") " Dec 03 12:05:10 crc kubenswrapper[5052]: I1203 12:05:10.166133 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3d2ba247-e370-4d62-a34e-5b05174363fb-combined-ca-bundle\") pod \"3d2ba247-e370-4d62-a34e-5b05174363fb\" (UID: \"3d2ba247-e370-4d62-a34e-5b05174363fb\") " Dec 03 12:05:10 crc kubenswrapper[5052]: I1203 12:05:10.166170 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/3d2ba247-e370-4d62-a34e-5b05174363fb-config-data\") pod \"3d2ba247-e370-4d62-a34e-5b05174363fb\" (UID: \"3d2ba247-e370-4d62-a34e-5b05174363fb\") " Dec 03 12:05:10 crc kubenswrapper[5052]: I1203 12:05:10.173239 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3d2ba247-e370-4d62-a34e-5b05174363fb-fernet-keys" (OuterVolumeSpecName: "fernet-keys") pod "3d2ba247-e370-4d62-a34e-5b05174363fb" (UID: "3d2ba247-e370-4d62-a34e-5b05174363fb"). InnerVolumeSpecName "fernet-keys". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:05:10 crc kubenswrapper[5052]: I1203 12:05:10.173239 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3d2ba247-e370-4d62-a34e-5b05174363fb-kube-api-access-bvbl6" (OuterVolumeSpecName: "kube-api-access-bvbl6") pod "3d2ba247-e370-4d62-a34e-5b05174363fb" (UID: "3d2ba247-e370-4d62-a34e-5b05174363fb"). InnerVolumeSpecName "kube-api-access-bvbl6". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:05:10 crc kubenswrapper[5052]: I1203 12:05:10.173572 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3d2ba247-e370-4d62-a34e-5b05174363fb-credential-keys" (OuterVolumeSpecName: "credential-keys") pod "3d2ba247-e370-4d62-a34e-5b05174363fb" (UID: "3d2ba247-e370-4d62-a34e-5b05174363fb"). InnerVolumeSpecName "credential-keys". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:05:10 crc kubenswrapper[5052]: I1203 12:05:10.179239 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3d2ba247-e370-4d62-a34e-5b05174363fb-scripts" (OuterVolumeSpecName: "scripts") pod "3d2ba247-e370-4d62-a34e-5b05174363fb" (UID: "3d2ba247-e370-4d62-a34e-5b05174363fb"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:05:10 crc kubenswrapper[5052]: I1203 12:05:10.201653 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3d2ba247-e370-4d62-a34e-5b05174363fb-config-data" (OuterVolumeSpecName: "config-data") pod "3d2ba247-e370-4d62-a34e-5b05174363fb" (UID: "3d2ba247-e370-4d62-a34e-5b05174363fb"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:05:10 crc kubenswrapper[5052]: I1203 12:05:10.203889 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3d2ba247-e370-4d62-a34e-5b05174363fb-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "3d2ba247-e370-4d62-a34e-5b05174363fb" (UID: "3d2ba247-e370-4d62-a34e-5b05174363fb"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:05:10 crc kubenswrapper[5052]: I1203 12:05:10.268778 5052 reconciler_common.go:293] "Volume detached for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/3d2ba247-e370-4d62-a34e-5b05174363fb-fernet-keys\") on node \"crc\" DevicePath \"\"" Dec 03 12:05:10 crc kubenswrapper[5052]: I1203 12:05:10.268822 5052 reconciler_common.go:293] "Volume detached for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/3d2ba247-e370-4d62-a34e-5b05174363fb-credential-keys\") on node \"crc\" DevicePath \"\"" Dec 03 12:05:10 crc kubenswrapper[5052]: I1203 12:05:10.268835 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/3d2ba247-e370-4d62-a34e-5b05174363fb-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:05:10 crc kubenswrapper[5052]: I1203 12:05:10.268846 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3d2ba247-e370-4d62-a34e-5b05174363fb-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:05:10 crc kubenswrapper[5052]: I1203 12:05:10.268858 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/3d2ba247-e370-4d62-a34e-5b05174363fb-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:05:10 crc kubenswrapper[5052]: I1203 12:05:10.268870 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-bvbl6\" (UniqueName: \"kubernetes.io/projected/3d2ba247-e370-4d62-a34e-5b05174363fb-kube-api-access-bvbl6\") on node \"crc\" DevicePath \"\"" Dec 03 12:05:10 crc kubenswrapper[5052]: I1203 12:05:10.668178 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-bootstrap-wbwms" Dec 03 12:05:10 crc kubenswrapper[5052]: I1203 12:05:10.668232 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-bootstrap-wbwms" event={"ID":"3d2ba247-e370-4d62-a34e-5b05174363fb","Type":"ContainerDied","Data":"d69d8109f5a447a6cd0dfcfdb4585aeecf2765db9edf4a41545793d4d4d83f6e"} Dec 03 12:05:10 crc kubenswrapper[5052]: I1203 12:05:10.668283 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d69d8109f5a447a6cd0dfcfdb4585aeecf2765db9edf4a41545793d4d4d83f6e" Dec 03 12:05:10 crc kubenswrapper[5052]: I1203 12:05:10.668537 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:05:10 crc kubenswrapper[5052]: I1203 12:05:10.737238 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/keystone-bootstrap-wbwms"] Dec 03 12:05:10 crc kubenswrapper[5052]: I1203 12:05:10.746646 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/keystone-bootstrap-wbwms"] Dec 03 12:05:10 crc kubenswrapper[5052]: I1203 12:05:10.856193 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/keystone-bootstrap-8s4bm"] Dec 03 12:05:10 crc kubenswrapper[5052]: E1203 12:05:10.856623 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="3d2ba247-e370-4d62-a34e-5b05174363fb" containerName="keystone-bootstrap" Dec 03 12:05:10 crc kubenswrapper[5052]: I1203 12:05:10.856641 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="3d2ba247-e370-4d62-a34e-5b05174363fb" containerName="keystone-bootstrap" Dec 03 12:05:10 crc kubenswrapper[5052]: I1203 12:05:10.856830 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="3d2ba247-e370-4d62-a34e-5b05174363fb" containerName="keystone-bootstrap" Dec 03 12:05:10 crc kubenswrapper[5052]: I1203 12:05:10.857527 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-bootstrap-8s4bm" Dec 03 12:05:10 crc kubenswrapper[5052]: I1203 12:05:10.861764 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"osp-secret" Dec 03 12:05:10 crc kubenswrapper[5052]: I1203 12:05:10.861798 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone-scripts" Dec 03 12:05:10 crc kubenswrapper[5052]: I1203 12:05:10.862092 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone-config-data" Dec 03 12:05:10 crc kubenswrapper[5052]: I1203 12:05:10.862208 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone-keystone-dockercfg-m92zx" Dec 03 12:05:10 crc kubenswrapper[5052]: I1203 12:05:10.864913 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone" Dec 03 12:05:10 crc kubenswrapper[5052]: I1203 12:05:10.877962 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-bootstrap-8s4bm"] Dec 03 12:05:10 crc kubenswrapper[5052]: I1203 12:05:10.982953 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/6588734e-3543-47aa-9df6-f6e21a877732-scripts\") pod \"keystone-bootstrap-8s4bm\" (UID: \"6588734e-3543-47aa-9df6-f6e21a877732\") " pod="openstack-kuttl-tests/keystone-bootstrap-8s4bm" Dec 03 12:05:10 crc kubenswrapper[5052]: I1203 12:05:10.983037 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gfjsk\" (UniqueName: \"kubernetes.io/projected/6588734e-3543-47aa-9df6-f6e21a877732-kube-api-access-gfjsk\") pod \"keystone-bootstrap-8s4bm\" (UID: \"6588734e-3543-47aa-9df6-f6e21a877732\") " pod="openstack-kuttl-tests/keystone-bootstrap-8s4bm" Dec 03 12:05:10 crc kubenswrapper[5052]: I1203 12:05:10.983527 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6588734e-3543-47aa-9df6-f6e21a877732-combined-ca-bundle\") pod \"keystone-bootstrap-8s4bm\" (UID: \"6588734e-3543-47aa-9df6-f6e21a877732\") " pod="openstack-kuttl-tests/keystone-bootstrap-8s4bm" Dec 03 12:05:10 crc kubenswrapper[5052]: I1203 12:05:10.983804 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/6588734e-3543-47aa-9df6-f6e21a877732-credential-keys\") pod \"keystone-bootstrap-8s4bm\" (UID: \"6588734e-3543-47aa-9df6-f6e21a877732\") " pod="openstack-kuttl-tests/keystone-bootstrap-8s4bm" Dec 03 12:05:10 crc kubenswrapper[5052]: I1203 12:05:10.983872 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/6588734e-3543-47aa-9df6-f6e21a877732-fernet-keys\") pod \"keystone-bootstrap-8s4bm\" (UID: \"6588734e-3543-47aa-9df6-f6e21a877732\") " pod="openstack-kuttl-tests/keystone-bootstrap-8s4bm" Dec 03 12:05:10 crc kubenswrapper[5052]: I1203 12:05:10.983978 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6588734e-3543-47aa-9df6-f6e21a877732-config-data\") pod \"keystone-bootstrap-8s4bm\" (UID: \"6588734e-3543-47aa-9df6-f6e21a877732\") " pod="openstack-kuttl-tests/keystone-bootstrap-8s4bm" Dec 03 12:05:11 crc kubenswrapper[5052]: I1203 12:05:11.067083 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-db-sync-852kv" Dec 03 12:05:11 crc kubenswrapper[5052]: I1203 12:05:11.087968 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/6588734e-3543-47aa-9df6-f6e21a877732-credential-keys\") pod \"keystone-bootstrap-8s4bm\" (UID: \"6588734e-3543-47aa-9df6-f6e21a877732\") " pod="openstack-kuttl-tests/keystone-bootstrap-8s4bm" Dec 03 12:05:11 crc kubenswrapper[5052]: I1203 12:05:11.088046 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/6588734e-3543-47aa-9df6-f6e21a877732-fernet-keys\") pod \"keystone-bootstrap-8s4bm\" (UID: \"6588734e-3543-47aa-9df6-f6e21a877732\") " pod="openstack-kuttl-tests/keystone-bootstrap-8s4bm" Dec 03 12:05:11 crc kubenswrapper[5052]: I1203 12:05:11.088112 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6588734e-3543-47aa-9df6-f6e21a877732-config-data\") pod \"keystone-bootstrap-8s4bm\" (UID: \"6588734e-3543-47aa-9df6-f6e21a877732\") " pod="openstack-kuttl-tests/keystone-bootstrap-8s4bm" Dec 03 12:05:11 crc kubenswrapper[5052]: I1203 12:05:11.088144 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/6588734e-3543-47aa-9df6-f6e21a877732-scripts\") pod \"keystone-bootstrap-8s4bm\" (UID: \"6588734e-3543-47aa-9df6-f6e21a877732\") " pod="openstack-kuttl-tests/keystone-bootstrap-8s4bm" Dec 03 12:05:11 crc kubenswrapper[5052]: I1203 12:05:11.088200 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-gfjsk\" (UniqueName: \"kubernetes.io/projected/6588734e-3543-47aa-9df6-f6e21a877732-kube-api-access-gfjsk\") pod \"keystone-bootstrap-8s4bm\" (UID: \"6588734e-3543-47aa-9df6-f6e21a877732\") " pod="openstack-kuttl-tests/keystone-bootstrap-8s4bm" Dec 03 12:05:11 crc kubenswrapper[5052]: I1203 12:05:11.088350 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6588734e-3543-47aa-9df6-f6e21a877732-combined-ca-bundle\") pod \"keystone-bootstrap-8s4bm\" (UID: \"6588734e-3543-47aa-9df6-f6e21a877732\") " pod="openstack-kuttl-tests/keystone-bootstrap-8s4bm" Dec 03 12:05:11 crc kubenswrapper[5052]: I1203 12:05:11.094458 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/6588734e-3543-47aa-9df6-f6e21a877732-credential-keys\") pod \"keystone-bootstrap-8s4bm\" (UID: \"6588734e-3543-47aa-9df6-f6e21a877732\") " pod="openstack-kuttl-tests/keystone-bootstrap-8s4bm" Dec 03 12:05:11 crc kubenswrapper[5052]: I1203 12:05:11.095071 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/6588734e-3543-47aa-9df6-f6e21a877732-scripts\") pod \"keystone-bootstrap-8s4bm\" (UID: \"6588734e-3543-47aa-9df6-f6e21a877732\") " pod="openstack-kuttl-tests/keystone-bootstrap-8s4bm" Dec 03 12:05:11 crc kubenswrapper[5052]: I1203 12:05:11.096433 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6588734e-3543-47aa-9df6-f6e21a877732-config-data\") pod \"keystone-bootstrap-8s4bm\" (UID: \"6588734e-3543-47aa-9df6-f6e21a877732\") " pod="openstack-kuttl-tests/keystone-bootstrap-8s4bm" Dec 03 12:05:11 crc kubenswrapper[5052]: I1203 12:05:11.096485 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6588734e-3543-47aa-9df6-f6e21a877732-combined-ca-bundle\") pod \"keystone-bootstrap-8s4bm\" (UID: \"6588734e-3543-47aa-9df6-f6e21a877732\") " pod="openstack-kuttl-tests/keystone-bootstrap-8s4bm" Dec 03 12:05:11 crc kubenswrapper[5052]: I1203 12:05:11.098519 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/6588734e-3543-47aa-9df6-f6e21a877732-fernet-keys\") pod \"keystone-bootstrap-8s4bm\" (UID: \"6588734e-3543-47aa-9df6-f6e21a877732\") " pod="openstack-kuttl-tests/keystone-bootstrap-8s4bm" Dec 03 12:05:11 crc kubenswrapper[5052]: I1203 12:05:11.113761 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-gfjsk\" (UniqueName: \"kubernetes.io/projected/6588734e-3543-47aa-9df6-f6e21a877732-kube-api-access-gfjsk\") pod \"keystone-bootstrap-8s4bm\" (UID: \"6588734e-3543-47aa-9df6-f6e21a877732\") " pod="openstack-kuttl-tests/keystone-bootstrap-8s4bm" Dec 03 12:05:11 crc kubenswrapper[5052]: I1203 12:05:11.181745 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-bootstrap-8s4bm" Dec 03 12:05:11 crc kubenswrapper[5052]: I1203 12:05:11.192526 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ab039824-fe2a-44ba-ae0f-302e5c4ea614-combined-ca-bundle\") pod \"ab039824-fe2a-44ba-ae0f-302e5c4ea614\" (UID: \"ab039824-fe2a-44ba-ae0f-302e5c4ea614\") " Dec 03 12:05:11 crc kubenswrapper[5052]: I1203 12:05:11.192687 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/ab039824-fe2a-44ba-ae0f-302e5c4ea614-config-data\") pod \"ab039824-fe2a-44ba-ae0f-302e5c4ea614\" (UID: \"ab039824-fe2a-44ba-ae0f-302e5c4ea614\") " Dec 03 12:05:11 crc kubenswrapper[5052]: I1203 12:05:11.192742 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kf6cd\" (UniqueName: \"kubernetes.io/projected/ab039824-fe2a-44ba-ae0f-302e5c4ea614-kube-api-access-kf6cd\") pod \"ab039824-fe2a-44ba-ae0f-302e5c4ea614\" (UID: \"ab039824-fe2a-44ba-ae0f-302e5c4ea614\") " Dec 03 12:05:11 crc kubenswrapper[5052]: I1203 12:05:11.192798 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/ab039824-fe2a-44ba-ae0f-302e5c4ea614-scripts\") pod \"ab039824-fe2a-44ba-ae0f-302e5c4ea614\" (UID: \"ab039824-fe2a-44ba-ae0f-302e5c4ea614\") " Dec 03 12:05:11 crc kubenswrapper[5052]: I1203 12:05:11.192817 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/ab039824-fe2a-44ba-ae0f-302e5c4ea614-logs\") pod \"ab039824-fe2a-44ba-ae0f-302e5c4ea614\" (UID: \"ab039824-fe2a-44ba-ae0f-302e5c4ea614\") " Dec 03 12:05:11 crc kubenswrapper[5052]: I1203 12:05:11.194183 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ab039824-fe2a-44ba-ae0f-302e5c4ea614-logs" (OuterVolumeSpecName: "logs") pod "ab039824-fe2a-44ba-ae0f-302e5c4ea614" (UID: "ab039824-fe2a-44ba-ae0f-302e5c4ea614"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:05:11 crc kubenswrapper[5052]: I1203 12:05:11.197223 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ab039824-fe2a-44ba-ae0f-302e5c4ea614-scripts" (OuterVolumeSpecName: "scripts") pod "ab039824-fe2a-44ba-ae0f-302e5c4ea614" (UID: "ab039824-fe2a-44ba-ae0f-302e5c4ea614"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:05:11 crc kubenswrapper[5052]: I1203 12:05:11.197412 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ab039824-fe2a-44ba-ae0f-302e5c4ea614-kube-api-access-kf6cd" (OuterVolumeSpecName: "kube-api-access-kf6cd") pod "ab039824-fe2a-44ba-ae0f-302e5c4ea614" (UID: "ab039824-fe2a-44ba-ae0f-302e5c4ea614"). InnerVolumeSpecName "kube-api-access-kf6cd". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:05:11 crc kubenswrapper[5052]: I1203 12:05:11.223619 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ab039824-fe2a-44ba-ae0f-302e5c4ea614-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "ab039824-fe2a-44ba-ae0f-302e5c4ea614" (UID: "ab039824-fe2a-44ba-ae0f-302e5c4ea614"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:05:11 crc kubenswrapper[5052]: I1203 12:05:11.224763 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ab039824-fe2a-44ba-ae0f-302e5c4ea614-config-data" (OuterVolumeSpecName: "config-data") pod "ab039824-fe2a-44ba-ae0f-302e5c4ea614" (UID: "ab039824-fe2a-44ba-ae0f-302e5c4ea614"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:05:11 crc kubenswrapper[5052]: I1203 12:05:11.299026 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ab039824-fe2a-44ba-ae0f-302e5c4ea614-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:05:11 crc kubenswrapper[5052]: I1203 12:05:11.299073 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/ab039824-fe2a-44ba-ae0f-302e5c4ea614-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:05:11 crc kubenswrapper[5052]: I1203 12:05:11.299088 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-kf6cd\" (UniqueName: \"kubernetes.io/projected/ab039824-fe2a-44ba-ae0f-302e5c4ea614-kube-api-access-kf6cd\") on node \"crc\" DevicePath \"\"" Dec 03 12:05:11 crc kubenswrapper[5052]: I1203 12:05:11.299102 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/ab039824-fe2a-44ba-ae0f-302e5c4ea614-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:05:11 crc kubenswrapper[5052]: I1203 12:05:11.299120 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/ab039824-fe2a-44ba-ae0f-302e5c4ea614-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:05:11 crc kubenswrapper[5052]: I1203 12:05:11.670131 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-bootstrap-8s4bm"] Dec 03 12:05:11 crc kubenswrapper[5052]: I1203 12:05:11.686609 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3d2ba247-e370-4d62-a34e-5b05174363fb" path="/var/lib/kubelet/pods/3d2ba247-e370-4d62-a34e-5b05174363fb/volumes" Dec 03 12:05:11 crc kubenswrapper[5052]: I1203 12:05:11.687474 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-db-sync-852kv" event={"ID":"ab039824-fe2a-44ba-ae0f-302e5c4ea614","Type":"ContainerDied","Data":"28550e67ac5e91431e3b9408d3292fba15c6a210bc517900b62f9cc7d7862a36"} Dec 03 12:05:11 crc kubenswrapper[5052]: I1203 12:05:11.687504 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="28550e67ac5e91431e3b9408d3292fba15c6a210bc517900b62f9cc7d7862a36" Dec 03 12:05:11 crc kubenswrapper[5052]: I1203 12:05:11.687964 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-db-sync-852kv" Dec 03 12:05:11 crc kubenswrapper[5052]: I1203 12:05:11.760262 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/placement-f4dfbd7d-mxsm8"] Dec 03 12:05:11 crc kubenswrapper[5052]: E1203 12:05:11.761139 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="ab039824-fe2a-44ba-ae0f-302e5c4ea614" containerName="placement-db-sync" Dec 03 12:05:11 crc kubenswrapper[5052]: I1203 12:05:11.761161 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="ab039824-fe2a-44ba-ae0f-302e5c4ea614" containerName="placement-db-sync" Dec 03 12:05:11 crc kubenswrapper[5052]: I1203 12:05:11.765587 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="ab039824-fe2a-44ba-ae0f-302e5c4ea614" containerName="placement-db-sync" Dec 03 12:05:11 crc kubenswrapper[5052]: I1203 12:05:11.767557 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-f4dfbd7d-mxsm8" Dec 03 12:05:11 crc kubenswrapper[5052]: I1203 12:05:11.769816 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"placement-scripts" Dec 03 12:05:11 crc kubenswrapper[5052]: I1203 12:05:11.769895 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"placement-config-data" Dec 03 12:05:11 crc kubenswrapper[5052]: I1203 12:05:11.771849 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-placement-internal-svc" Dec 03 12:05:11 crc kubenswrapper[5052]: I1203 12:05:11.772031 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"placement-placement-dockercfg-rmdxz" Dec 03 12:05:11 crc kubenswrapper[5052]: I1203 12:05:11.772268 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-placement-public-svc" Dec 03 12:05:11 crc kubenswrapper[5052]: I1203 12:05:11.779864 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/placement-f4dfbd7d-mxsm8"] Dec 03 12:05:11 crc kubenswrapper[5052]: I1203 12:05:11.932101 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/6999e86b-8f95-4002-9959-6288a7d783bd-scripts\") pod \"placement-f4dfbd7d-mxsm8\" (UID: \"6999e86b-8f95-4002-9959-6288a7d783bd\") " pod="openstack-kuttl-tests/placement-f4dfbd7d-mxsm8" Dec 03 12:05:11 crc kubenswrapper[5052]: I1203 12:05:11.932183 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4zd27\" (UniqueName: \"kubernetes.io/projected/6999e86b-8f95-4002-9959-6288a7d783bd-kube-api-access-4zd27\") pod \"placement-f4dfbd7d-mxsm8\" (UID: \"6999e86b-8f95-4002-9959-6288a7d783bd\") " pod="openstack-kuttl-tests/placement-f4dfbd7d-mxsm8" Dec 03 12:05:11 crc kubenswrapper[5052]: I1203 12:05:11.932210 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6999e86b-8f95-4002-9959-6288a7d783bd-combined-ca-bundle\") pod \"placement-f4dfbd7d-mxsm8\" (UID: \"6999e86b-8f95-4002-9959-6288a7d783bd\") " pod="openstack-kuttl-tests/placement-f4dfbd7d-mxsm8" Dec 03 12:05:11 crc kubenswrapper[5052]: I1203 12:05:11.932235 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6999e86b-8f95-4002-9959-6288a7d783bd-config-data\") pod \"placement-f4dfbd7d-mxsm8\" (UID: \"6999e86b-8f95-4002-9959-6288a7d783bd\") " pod="openstack-kuttl-tests/placement-f4dfbd7d-mxsm8" Dec 03 12:05:11 crc kubenswrapper[5052]: I1203 12:05:11.932275 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/6999e86b-8f95-4002-9959-6288a7d783bd-logs\") pod \"placement-f4dfbd7d-mxsm8\" (UID: \"6999e86b-8f95-4002-9959-6288a7d783bd\") " pod="openstack-kuttl-tests/placement-f4dfbd7d-mxsm8" Dec 03 12:05:11 crc kubenswrapper[5052]: I1203 12:05:11.932298 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/6999e86b-8f95-4002-9959-6288a7d783bd-internal-tls-certs\") pod \"placement-f4dfbd7d-mxsm8\" (UID: \"6999e86b-8f95-4002-9959-6288a7d783bd\") " pod="openstack-kuttl-tests/placement-f4dfbd7d-mxsm8" Dec 03 12:05:11 crc kubenswrapper[5052]: I1203 12:05:11.932346 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/6999e86b-8f95-4002-9959-6288a7d783bd-public-tls-certs\") pod \"placement-f4dfbd7d-mxsm8\" (UID: \"6999e86b-8f95-4002-9959-6288a7d783bd\") " pod="openstack-kuttl-tests/placement-f4dfbd7d-mxsm8" Dec 03 12:05:12 crc kubenswrapper[5052]: I1203 12:05:12.034410 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/6999e86b-8f95-4002-9959-6288a7d783bd-public-tls-certs\") pod \"placement-f4dfbd7d-mxsm8\" (UID: \"6999e86b-8f95-4002-9959-6288a7d783bd\") " pod="openstack-kuttl-tests/placement-f4dfbd7d-mxsm8" Dec 03 12:05:12 crc kubenswrapper[5052]: I1203 12:05:12.034528 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/6999e86b-8f95-4002-9959-6288a7d783bd-scripts\") pod \"placement-f4dfbd7d-mxsm8\" (UID: \"6999e86b-8f95-4002-9959-6288a7d783bd\") " pod="openstack-kuttl-tests/placement-f4dfbd7d-mxsm8" Dec 03 12:05:12 crc kubenswrapper[5052]: I1203 12:05:12.034612 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-4zd27\" (UniqueName: \"kubernetes.io/projected/6999e86b-8f95-4002-9959-6288a7d783bd-kube-api-access-4zd27\") pod \"placement-f4dfbd7d-mxsm8\" (UID: \"6999e86b-8f95-4002-9959-6288a7d783bd\") " pod="openstack-kuttl-tests/placement-f4dfbd7d-mxsm8" Dec 03 12:05:12 crc kubenswrapper[5052]: I1203 12:05:12.034646 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6999e86b-8f95-4002-9959-6288a7d783bd-combined-ca-bundle\") pod \"placement-f4dfbd7d-mxsm8\" (UID: \"6999e86b-8f95-4002-9959-6288a7d783bd\") " pod="openstack-kuttl-tests/placement-f4dfbd7d-mxsm8" Dec 03 12:05:12 crc kubenswrapper[5052]: I1203 12:05:12.034678 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6999e86b-8f95-4002-9959-6288a7d783bd-config-data\") pod \"placement-f4dfbd7d-mxsm8\" (UID: \"6999e86b-8f95-4002-9959-6288a7d783bd\") " pod="openstack-kuttl-tests/placement-f4dfbd7d-mxsm8" Dec 03 12:05:12 crc kubenswrapper[5052]: I1203 12:05:12.034741 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/6999e86b-8f95-4002-9959-6288a7d783bd-logs\") pod \"placement-f4dfbd7d-mxsm8\" (UID: \"6999e86b-8f95-4002-9959-6288a7d783bd\") " pod="openstack-kuttl-tests/placement-f4dfbd7d-mxsm8" Dec 03 12:05:12 crc kubenswrapper[5052]: I1203 12:05:12.034773 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/6999e86b-8f95-4002-9959-6288a7d783bd-internal-tls-certs\") pod \"placement-f4dfbd7d-mxsm8\" (UID: \"6999e86b-8f95-4002-9959-6288a7d783bd\") " pod="openstack-kuttl-tests/placement-f4dfbd7d-mxsm8" Dec 03 12:05:12 crc kubenswrapper[5052]: I1203 12:05:12.036006 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/6999e86b-8f95-4002-9959-6288a7d783bd-logs\") pod \"placement-f4dfbd7d-mxsm8\" (UID: \"6999e86b-8f95-4002-9959-6288a7d783bd\") " pod="openstack-kuttl-tests/placement-f4dfbd7d-mxsm8" Dec 03 12:05:12 crc kubenswrapper[5052]: I1203 12:05:12.039227 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/6999e86b-8f95-4002-9959-6288a7d783bd-scripts\") pod \"placement-f4dfbd7d-mxsm8\" (UID: \"6999e86b-8f95-4002-9959-6288a7d783bd\") " pod="openstack-kuttl-tests/placement-f4dfbd7d-mxsm8" Dec 03 12:05:12 crc kubenswrapper[5052]: I1203 12:05:12.040898 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/6999e86b-8f95-4002-9959-6288a7d783bd-public-tls-certs\") pod \"placement-f4dfbd7d-mxsm8\" (UID: \"6999e86b-8f95-4002-9959-6288a7d783bd\") " pod="openstack-kuttl-tests/placement-f4dfbd7d-mxsm8" Dec 03 12:05:12 crc kubenswrapper[5052]: I1203 12:05:12.041271 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6999e86b-8f95-4002-9959-6288a7d783bd-config-data\") pod \"placement-f4dfbd7d-mxsm8\" (UID: \"6999e86b-8f95-4002-9959-6288a7d783bd\") " pod="openstack-kuttl-tests/placement-f4dfbd7d-mxsm8" Dec 03 12:05:12 crc kubenswrapper[5052]: I1203 12:05:12.044664 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/6999e86b-8f95-4002-9959-6288a7d783bd-internal-tls-certs\") pod \"placement-f4dfbd7d-mxsm8\" (UID: \"6999e86b-8f95-4002-9959-6288a7d783bd\") " pod="openstack-kuttl-tests/placement-f4dfbd7d-mxsm8" Dec 03 12:05:12 crc kubenswrapper[5052]: I1203 12:05:12.053042 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6999e86b-8f95-4002-9959-6288a7d783bd-combined-ca-bundle\") pod \"placement-f4dfbd7d-mxsm8\" (UID: \"6999e86b-8f95-4002-9959-6288a7d783bd\") " pod="openstack-kuttl-tests/placement-f4dfbd7d-mxsm8" Dec 03 12:05:12 crc kubenswrapper[5052]: I1203 12:05:12.055227 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-4zd27\" (UniqueName: \"kubernetes.io/projected/6999e86b-8f95-4002-9959-6288a7d783bd-kube-api-access-4zd27\") pod \"placement-f4dfbd7d-mxsm8\" (UID: \"6999e86b-8f95-4002-9959-6288a7d783bd\") " pod="openstack-kuttl-tests/placement-f4dfbd7d-mxsm8" Dec 03 12:05:12 crc kubenswrapper[5052]: I1203 12:05:12.098695 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-f4dfbd7d-mxsm8" Dec 03 12:05:12 crc kubenswrapper[5052]: I1203 12:05:12.572899 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/placement-f4dfbd7d-mxsm8"] Dec 03 12:05:12 crc kubenswrapper[5052]: I1203 12:05:12.737344 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-f4dfbd7d-mxsm8" event={"ID":"6999e86b-8f95-4002-9959-6288a7d783bd","Type":"ContainerStarted","Data":"2d364625552a7cb7d231a8334b918f370dbb7586042dad1e5579d3dc8073a66f"} Dec 03 12:05:12 crc kubenswrapper[5052]: I1203 12:05:12.743119 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-bootstrap-8s4bm" event={"ID":"6588734e-3543-47aa-9df6-f6e21a877732","Type":"ContainerStarted","Data":"95271173fa771abc9b96986bbbf1cb4713ab271b289e360271247bb505b8a425"} Dec 03 12:05:12 crc kubenswrapper[5052]: I1203 12:05:12.743171 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-bootstrap-8s4bm" event={"ID":"6588734e-3543-47aa-9df6-f6e21a877732","Type":"ContainerStarted","Data":"8ecf1b450bb5b0378ebfc497bc4b3b83396c880a31d27b70702298733fb6fe94"} Dec 03 12:05:12 crc kubenswrapper[5052]: I1203 12:05:12.773329 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/keystone-bootstrap-8s4bm" podStartSLOduration=2.773305379 podStartE2EDuration="2.773305379s" podCreationTimestamp="2025-12-03 12:05:10 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:05:12.769176391 +0000 UTC m=+2703.357418155" watchObservedRunningTime="2025-12-03 12:05:12.773305379 +0000 UTC m=+2703.361547143" Dec 03 12:05:13 crc kubenswrapper[5052]: I1203 12:05:13.753581 5052 generic.go:334] "Generic (PLEG): container finished" podID="f3c81a5a-a6cc-406f-95b9-186ae52a3ccb" containerID="c28a9606ae5010324a932dd9d90a75063a7e3597bf5e73c2ae2c2451a0afbbab" exitCode=0 Dec 03 12:05:13 crc kubenswrapper[5052]: I1203 12:05:13.753955 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-db-sync-4z2hl" event={"ID":"f3c81a5a-a6cc-406f-95b9-186ae52a3ccb","Type":"ContainerDied","Data":"c28a9606ae5010324a932dd9d90a75063a7e3597bf5e73c2ae2c2451a0afbbab"} Dec 03 12:05:13 crc kubenswrapper[5052]: I1203 12:05:13.757969 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-f4dfbd7d-mxsm8" event={"ID":"6999e86b-8f95-4002-9959-6288a7d783bd","Type":"ContainerStarted","Data":"3e8fef46e8b29b153dd3541055f4f411078ff175402ded3c5e2a2532584522cb"} Dec 03 12:05:13 crc kubenswrapper[5052]: I1203 12:05:13.758009 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/placement-f4dfbd7d-mxsm8" Dec 03 12:05:13 crc kubenswrapper[5052]: I1203 12:05:13.758020 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-f4dfbd7d-mxsm8" event={"ID":"6999e86b-8f95-4002-9959-6288a7d783bd","Type":"ContainerStarted","Data":"18f9fa6fd7bb783b320dadc515c1e38c7a2e0212224d2e211b526e64320731a1"} Dec 03 12:05:13 crc kubenswrapper[5052]: I1203 12:05:13.758040 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/placement-f4dfbd7d-mxsm8" Dec 03 12:05:13 crc kubenswrapper[5052]: I1203 12:05:13.803111 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/placement-f4dfbd7d-mxsm8" podStartSLOduration=2.80308826 podStartE2EDuration="2.80308826s" podCreationTimestamp="2025-12-03 12:05:11 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:05:13.802456362 +0000 UTC m=+2704.390698156" watchObservedRunningTime="2025-12-03 12:05:13.80308826 +0000 UTC m=+2704.391330024" Dec 03 12:05:15 crc kubenswrapper[5052]: I1203 12:05:15.106351 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-db-sync-4z2hl" Dec 03 12:05:15 crc kubenswrapper[5052]: I1203 12:05:15.296116 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f3c81a5a-a6cc-406f-95b9-186ae52a3ccb-combined-ca-bundle\") pod \"f3c81a5a-a6cc-406f-95b9-186ae52a3ccb\" (UID: \"f3c81a5a-a6cc-406f-95b9-186ae52a3ccb\") " Dec 03 12:05:15 crc kubenswrapper[5052]: I1203 12:05:15.296977 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/f3c81a5a-a6cc-406f-95b9-186ae52a3ccb-db-sync-config-data\") pod \"f3c81a5a-a6cc-406f-95b9-186ae52a3ccb\" (UID: \"f3c81a5a-a6cc-406f-95b9-186ae52a3ccb\") " Dec 03 12:05:15 crc kubenswrapper[5052]: I1203 12:05:15.297064 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hxq27\" (UniqueName: \"kubernetes.io/projected/f3c81a5a-a6cc-406f-95b9-186ae52a3ccb-kube-api-access-hxq27\") pod \"f3c81a5a-a6cc-406f-95b9-186ae52a3ccb\" (UID: \"f3c81a5a-a6cc-406f-95b9-186ae52a3ccb\") " Dec 03 12:05:15 crc kubenswrapper[5052]: I1203 12:05:15.302600 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f3c81a5a-a6cc-406f-95b9-186ae52a3ccb-db-sync-config-data" (OuterVolumeSpecName: "db-sync-config-data") pod "f3c81a5a-a6cc-406f-95b9-186ae52a3ccb" (UID: "f3c81a5a-a6cc-406f-95b9-186ae52a3ccb"). InnerVolumeSpecName "db-sync-config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:05:15 crc kubenswrapper[5052]: I1203 12:05:15.306365 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f3c81a5a-a6cc-406f-95b9-186ae52a3ccb-kube-api-access-hxq27" (OuterVolumeSpecName: "kube-api-access-hxq27") pod "f3c81a5a-a6cc-406f-95b9-186ae52a3ccb" (UID: "f3c81a5a-a6cc-406f-95b9-186ae52a3ccb"). InnerVolumeSpecName "kube-api-access-hxq27". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:05:15 crc kubenswrapper[5052]: I1203 12:05:15.326576 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f3c81a5a-a6cc-406f-95b9-186ae52a3ccb-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "f3c81a5a-a6cc-406f-95b9-186ae52a3ccb" (UID: "f3c81a5a-a6cc-406f-95b9-186ae52a3ccb"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:05:15 crc kubenswrapper[5052]: I1203 12:05:15.399220 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f3c81a5a-a6cc-406f-95b9-186ae52a3ccb-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:05:15 crc kubenswrapper[5052]: I1203 12:05:15.399256 5052 reconciler_common.go:293] "Volume detached for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/f3c81a5a-a6cc-406f-95b9-186ae52a3ccb-db-sync-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:05:15 crc kubenswrapper[5052]: I1203 12:05:15.399267 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-hxq27\" (UniqueName: \"kubernetes.io/projected/f3c81a5a-a6cc-406f-95b9-186ae52a3ccb-kube-api-access-hxq27\") on node \"crc\" DevicePath \"\"" Dec 03 12:05:15 crc kubenswrapper[5052]: I1203 12:05:15.781474 5052 generic.go:334] "Generic (PLEG): container finished" podID="0c8be033-f83e-4d64-b271-f4c97bfdc108" containerID="216f652b89ba64b000d5cd54f33f538c801f811bfa201b681b49eb486c4725d9" exitCode=0 Dec 03 12:05:15 crc kubenswrapper[5052]: I1203 12:05:15.781624 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-db-sync-jhtdm" event={"ID":"0c8be033-f83e-4d64-b271-f4c97bfdc108","Type":"ContainerDied","Data":"216f652b89ba64b000d5cd54f33f538c801f811bfa201b681b49eb486c4725d9"} Dec 03 12:05:15 crc kubenswrapper[5052]: I1203 12:05:15.784110 5052 generic.go:334] "Generic (PLEG): container finished" podID="6588734e-3543-47aa-9df6-f6e21a877732" containerID="95271173fa771abc9b96986bbbf1cb4713ab271b289e360271247bb505b8a425" exitCode=0 Dec 03 12:05:15 crc kubenswrapper[5052]: I1203 12:05:15.784181 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-bootstrap-8s4bm" event={"ID":"6588734e-3543-47aa-9df6-f6e21a877732","Type":"ContainerDied","Data":"95271173fa771abc9b96986bbbf1cb4713ab271b289e360271247bb505b8a425"} Dec 03 12:05:15 crc kubenswrapper[5052]: I1203 12:05:15.787743 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-db-sync-4z2hl" event={"ID":"f3c81a5a-a6cc-406f-95b9-186ae52a3ccb","Type":"ContainerDied","Data":"5440de1ca16f5ac86ac3bfc5a38495ee91f425111048898c80b50d82f094ee04"} Dec 03 12:05:15 crc kubenswrapper[5052]: I1203 12:05:15.787803 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-db-sync-4z2hl" Dec 03 12:05:15 crc kubenswrapper[5052]: I1203 12:05:15.787814 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5440de1ca16f5ac86ac3bfc5a38495ee91f425111048898c80b50d82f094ee04" Dec 03 12:05:16 crc kubenswrapper[5052]: I1203 12:05:16.037609 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/barbican-worker-c656b9c77-2gf7g"] Dec 03 12:05:16 crc kubenswrapper[5052]: E1203 12:05:16.038145 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="f3c81a5a-a6cc-406f-95b9-186ae52a3ccb" containerName="barbican-db-sync" Dec 03 12:05:16 crc kubenswrapper[5052]: I1203 12:05:16.038166 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="f3c81a5a-a6cc-406f-95b9-186ae52a3ccb" containerName="barbican-db-sync" Dec 03 12:05:16 crc kubenswrapper[5052]: I1203 12:05:16.038365 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="f3c81a5a-a6cc-406f-95b9-186ae52a3ccb" containerName="barbican-db-sync" Dec 03 12:05:16 crc kubenswrapper[5052]: I1203 12:05:16.039415 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-worker-c656b9c77-2gf7g" Dec 03 12:05:16 crc kubenswrapper[5052]: I1203 12:05:16.042037 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"barbican-barbican-dockercfg-7l5nk" Dec 03 12:05:16 crc kubenswrapper[5052]: I1203 12:05:16.042286 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"barbican-worker-config-data" Dec 03 12:05:16 crc kubenswrapper[5052]: I1203 12:05:16.049887 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"barbican-config-data" Dec 03 12:05:16 crc kubenswrapper[5052]: I1203 12:05:16.069684 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/barbican-keystone-listener-64674fc4f4-wfh52"] Dec 03 12:05:16 crc kubenswrapper[5052]: I1203 12:05:16.071442 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-keystone-listener-64674fc4f4-wfh52" Dec 03 12:05:16 crc kubenswrapper[5052]: I1203 12:05:16.074669 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"barbican-keystone-listener-config-data" Dec 03 12:05:16 crc kubenswrapper[5052]: I1203 12:05:16.083216 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-worker-c656b9c77-2gf7g"] Dec 03 12:05:16 crc kubenswrapper[5052]: I1203 12:05:16.118016 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-keystone-listener-64674fc4f4-wfh52"] Dec 03 12:05:16 crc kubenswrapper[5052]: I1203 12:05:16.165494 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/d877b426-724d-4ccf-a113-05a96ab94935-logs\") pod \"barbican-keystone-listener-64674fc4f4-wfh52\" (UID: \"d877b426-724d-4ccf-a113-05a96ab94935\") " pod="openstack-kuttl-tests/barbican-keystone-listener-64674fc4f4-wfh52" Dec 03 12:05:16 crc kubenswrapper[5052]: I1203 12:05:16.165585 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a3da2acb-8a0e-4ab3-8d15-2978eca73850-config-data\") pod \"barbican-worker-c656b9c77-2gf7g\" (UID: \"a3da2acb-8a0e-4ab3-8d15-2978eca73850\") " pod="openstack-kuttl-tests/barbican-worker-c656b9c77-2gf7g" Dec 03 12:05:16 crc kubenswrapper[5052]: I1203 12:05:16.165609 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/d877b426-724d-4ccf-a113-05a96ab94935-config-data-custom\") pod \"barbican-keystone-listener-64674fc4f4-wfh52\" (UID: \"d877b426-724d-4ccf-a113-05a96ab94935\") " pod="openstack-kuttl-tests/barbican-keystone-listener-64674fc4f4-wfh52" Dec 03 12:05:16 crc kubenswrapper[5052]: I1203 12:05:16.165633 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/a3da2acb-8a0e-4ab3-8d15-2978eca73850-logs\") pod \"barbican-worker-c656b9c77-2gf7g\" (UID: \"a3da2acb-8a0e-4ab3-8d15-2978eca73850\") " pod="openstack-kuttl-tests/barbican-worker-c656b9c77-2gf7g" Dec 03 12:05:16 crc kubenswrapper[5052]: I1203 12:05:16.165701 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/d877b426-724d-4ccf-a113-05a96ab94935-config-data\") pod \"barbican-keystone-listener-64674fc4f4-wfh52\" (UID: \"d877b426-724d-4ccf-a113-05a96ab94935\") " pod="openstack-kuttl-tests/barbican-keystone-listener-64674fc4f4-wfh52" Dec 03 12:05:16 crc kubenswrapper[5052]: I1203 12:05:16.166149 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d877b426-724d-4ccf-a113-05a96ab94935-combined-ca-bundle\") pod \"barbican-keystone-listener-64674fc4f4-wfh52\" (UID: \"d877b426-724d-4ccf-a113-05a96ab94935\") " pod="openstack-kuttl-tests/barbican-keystone-listener-64674fc4f4-wfh52" Dec 03 12:05:16 crc kubenswrapper[5052]: I1203 12:05:16.166195 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p9lmx\" (UniqueName: \"kubernetes.io/projected/d877b426-724d-4ccf-a113-05a96ab94935-kube-api-access-p9lmx\") pod \"barbican-keystone-listener-64674fc4f4-wfh52\" (UID: \"d877b426-724d-4ccf-a113-05a96ab94935\") " pod="openstack-kuttl-tests/barbican-keystone-listener-64674fc4f4-wfh52" Dec 03 12:05:16 crc kubenswrapper[5052]: I1203 12:05:16.166365 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jqslb\" (UniqueName: \"kubernetes.io/projected/a3da2acb-8a0e-4ab3-8d15-2978eca73850-kube-api-access-jqslb\") pod \"barbican-worker-c656b9c77-2gf7g\" (UID: \"a3da2acb-8a0e-4ab3-8d15-2978eca73850\") " pod="openstack-kuttl-tests/barbican-worker-c656b9c77-2gf7g" Dec 03 12:05:16 crc kubenswrapper[5052]: I1203 12:05:16.166412 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a3da2acb-8a0e-4ab3-8d15-2978eca73850-combined-ca-bundle\") pod \"barbican-worker-c656b9c77-2gf7g\" (UID: \"a3da2acb-8a0e-4ab3-8d15-2978eca73850\") " pod="openstack-kuttl-tests/barbican-worker-c656b9c77-2gf7g" Dec 03 12:05:16 crc kubenswrapper[5052]: I1203 12:05:16.166644 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/a3da2acb-8a0e-4ab3-8d15-2978eca73850-config-data-custom\") pod \"barbican-worker-c656b9c77-2gf7g\" (UID: \"a3da2acb-8a0e-4ab3-8d15-2978eca73850\") " pod="openstack-kuttl-tests/barbican-worker-c656b9c77-2gf7g" Dec 03 12:05:16 crc kubenswrapper[5052]: I1203 12:05:16.171654 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/barbican-api-5757755d9b-9m6dz"] Dec 03 12:05:16 crc kubenswrapper[5052]: I1203 12:05:16.173921 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-api-5757755d9b-9m6dz" Dec 03 12:05:16 crc kubenswrapper[5052]: I1203 12:05:16.176852 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"barbican-api-config-data" Dec 03 12:05:16 crc kubenswrapper[5052]: I1203 12:05:16.189686 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-api-5757755d9b-9m6dz"] Dec 03 12:05:16 crc kubenswrapper[5052]: I1203 12:05:16.267887 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/d877b426-724d-4ccf-a113-05a96ab94935-config-data-custom\") pod \"barbican-keystone-listener-64674fc4f4-wfh52\" (UID: \"d877b426-724d-4ccf-a113-05a96ab94935\") " pod="openstack-kuttl-tests/barbican-keystone-listener-64674fc4f4-wfh52" Dec 03 12:05:16 crc kubenswrapper[5052]: I1203 12:05:16.267943 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/a3da2acb-8a0e-4ab3-8d15-2978eca73850-logs\") pod \"barbican-worker-c656b9c77-2gf7g\" (UID: \"a3da2acb-8a0e-4ab3-8d15-2978eca73850\") " pod="openstack-kuttl-tests/barbican-worker-c656b9c77-2gf7g" Dec 03 12:05:16 crc kubenswrapper[5052]: I1203 12:05:16.267980 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/d877b426-724d-4ccf-a113-05a96ab94935-config-data\") pod \"barbican-keystone-listener-64674fc4f4-wfh52\" (UID: \"d877b426-724d-4ccf-a113-05a96ab94935\") " pod="openstack-kuttl-tests/barbican-keystone-listener-64674fc4f4-wfh52" Dec 03 12:05:16 crc kubenswrapper[5052]: I1203 12:05:16.268014 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/44965796-ddc0-4cb1-bdf6-f107b9c3373b-config-data\") pod \"barbican-api-5757755d9b-9m6dz\" (UID: \"44965796-ddc0-4cb1-bdf6-f107b9c3373b\") " pod="openstack-kuttl-tests/barbican-api-5757755d9b-9m6dz" Dec 03 12:05:16 crc kubenswrapper[5052]: I1203 12:05:16.268064 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d877b426-724d-4ccf-a113-05a96ab94935-combined-ca-bundle\") pod \"barbican-keystone-listener-64674fc4f4-wfh52\" (UID: \"d877b426-724d-4ccf-a113-05a96ab94935\") " pod="openstack-kuttl-tests/barbican-keystone-listener-64674fc4f4-wfh52" Dec 03 12:05:16 crc kubenswrapper[5052]: I1203 12:05:16.268091 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-p9lmx\" (UniqueName: \"kubernetes.io/projected/d877b426-724d-4ccf-a113-05a96ab94935-kube-api-access-p9lmx\") pod \"barbican-keystone-listener-64674fc4f4-wfh52\" (UID: \"d877b426-724d-4ccf-a113-05a96ab94935\") " pod="openstack-kuttl-tests/barbican-keystone-listener-64674fc4f4-wfh52" Dec 03 12:05:16 crc kubenswrapper[5052]: I1203 12:05:16.268120 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-jqslb\" (UniqueName: \"kubernetes.io/projected/a3da2acb-8a0e-4ab3-8d15-2978eca73850-kube-api-access-jqslb\") pod \"barbican-worker-c656b9c77-2gf7g\" (UID: \"a3da2acb-8a0e-4ab3-8d15-2978eca73850\") " pod="openstack-kuttl-tests/barbican-worker-c656b9c77-2gf7g" Dec 03 12:05:16 crc kubenswrapper[5052]: I1203 12:05:16.268136 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a3da2acb-8a0e-4ab3-8d15-2978eca73850-combined-ca-bundle\") pod \"barbican-worker-c656b9c77-2gf7g\" (UID: \"a3da2acb-8a0e-4ab3-8d15-2978eca73850\") " pod="openstack-kuttl-tests/barbican-worker-c656b9c77-2gf7g" Dec 03 12:05:16 crc kubenswrapper[5052]: I1203 12:05:16.268160 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/44965796-ddc0-4cb1-bdf6-f107b9c3373b-combined-ca-bundle\") pod \"barbican-api-5757755d9b-9m6dz\" (UID: \"44965796-ddc0-4cb1-bdf6-f107b9c3373b\") " pod="openstack-kuttl-tests/barbican-api-5757755d9b-9m6dz" Dec 03 12:05:16 crc kubenswrapper[5052]: I1203 12:05:16.268183 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/44965796-ddc0-4cb1-bdf6-f107b9c3373b-config-data-custom\") pod \"barbican-api-5757755d9b-9m6dz\" (UID: \"44965796-ddc0-4cb1-bdf6-f107b9c3373b\") " pod="openstack-kuttl-tests/barbican-api-5757755d9b-9m6dz" Dec 03 12:05:16 crc kubenswrapper[5052]: I1203 12:05:16.268202 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kx6k2\" (UniqueName: \"kubernetes.io/projected/44965796-ddc0-4cb1-bdf6-f107b9c3373b-kube-api-access-kx6k2\") pod \"barbican-api-5757755d9b-9m6dz\" (UID: \"44965796-ddc0-4cb1-bdf6-f107b9c3373b\") " pod="openstack-kuttl-tests/barbican-api-5757755d9b-9m6dz" Dec 03 12:05:16 crc kubenswrapper[5052]: I1203 12:05:16.268223 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/a3da2acb-8a0e-4ab3-8d15-2978eca73850-config-data-custom\") pod \"barbican-worker-c656b9c77-2gf7g\" (UID: \"a3da2acb-8a0e-4ab3-8d15-2978eca73850\") " pod="openstack-kuttl-tests/barbican-worker-c656b9c77-2gf7g" Dec 03 12:05:16 crc kubenswrapper[5052]: I1203 12:05:16.268260 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/44965796-ddc0-4cb1-bdf6-f107b9c3373b-logs\") pod \"barbican-api-5757755d9b-9m6dz\" (UID: \"44965796-ddc0-4cb1-bdf6-f107b9c3373b\") " pod="openstack-kuttl-tests/barbican-api-5757755d9b-9m6dz" Dec 03 12:05:16 crc kubenswrapper[5052]: I1203 12:05:16.268282 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/d877b426-724d-4ccf-a113-05a96ab94935-logs\") pod \"barbican-keystone-listener-64674fc4f4-wfh52\" (UID: \"d877b426-724d-4ccf-a113-05a96ab94935\") " pod="openstack-kuttl-tests/barbican-keystone-listener-64674fc4f4-wfh52" Dec 03 12:05:16 crc kubenswrapper[5052]: I1203 12:05:16.268305 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a3da2acb-8a0e-4ab3-8d15-2978eca73850-config-data\") pod \"barbican-worker-c656b9c77-2gf7g\" (UID: \"a3da2acb-8a0e-4ab3-8d15-2978eca73850\") " pod="openstack-kuttl-tests/barbican-worker-c656b9c77-2gf7g" Dec 03 12:05:16 crc kubenswrapper[5052]: I1203 12:05:16.269028 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/a3da2acb-8a0e-4ab3-8d15-2978eca73850-logs\") pod \"barbican-worker-c656b9c77-2gf7g\" (UID: \"a3da2acb-8a0e-4ab3-8d15-2978eca73850\") " pod="openstack-kuttl-tests/barbican-worker-c656b9c77-2gf7g" Dec 03 12:05:16 crc kubenswrapper[5052]: I1203 12:05:16.269654 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/d877b426-724d-4ccf-a113-05a96ab94935-logs\") pod \"barbican-keystone-listener-64674fc4f4-wfh52\" (UID: \"d877b426-724d-4ccf-a113-05a96ab94935\") " pod="openstack-kuttl-tests/barbican-keystone-listener-64674fc4f4-wfh52" Dec 03 12:05:16 crc kubenswrapper[5052]: I1203 12:05:16.274465 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/a3da2acb-8a0e-4ab3-8d15-2978eca73850-config-data-custom\") pod \"barbican-worker-c656b9c77-2gf7g\" (UID: \"a3da2acb-8a0e-4ab3-8d15-2978eca73850\") " pod="openstack-kuttl-tests/barbican-worker-c656b9c77-2gf7g" Dec 03 12:05:16 crc kubenswrapper[5052]: I1203 12:05:16.275012 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/d877b426-724d-4ccf-a113-05a96ab94935-config-data\") pod \"barbican-keystone-listener-64674fc4f4-wfh52\" (UID: \"d877b426-724d-4ccf-a113-05a96ab94935\") " pod="openstack-kuttl-tests/barbican-keystone-listener-64674fc4f4-wfh52" Dec 03 12:05:16 crc kubenswrapper[5052]: I1203 12:05:16.275786 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/d877b426-724d-4ccf-a113-05a96ab94935-config-data-custom\") pod \"barbican-keystone-listener-64674fc4f4-wfh52\" (UID: \"d877b426-724d-4ccf-a113-05a96ab94935\") " pod="openstack-kuttl-tests/barbican-keystone-listener-64674fc4f4-wfh52" Dec 03 12:05:16 crc kubenswrapper[5052]: I1203 12:05:16.275952 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a3da2acb-8a0e-4ab3-8d15-2978eca73850-config-data\") pod \"barbican-worker-c656b9c77-2gf7g\" (UID: \"a3da2acb-8a0e-4ab3-8d15-2978eca73850\") " pod="openstack-kuttl-tests/barbican-worker-c656b9c77-2gf7g" Dec 03 12:05:16 crc kubenswrapper[5052]: I1203 12:05:16.276021 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a3da2acb-8a0e-4ab3-8d15-2978eca73850-combined-ca-bundle\") pod \"barbican-worker-c656b9c77-2gf7g\" (UID: \"a3da2acb-8a0e-4ab3-8d15-2978eca73850\") " pod="openstack-kuttl-tests/barbican-worker-c656b9c77-2gf7g" Dec 03 12:05:16 crc kubenswrapper[5052]: I1203 12:05:16.276785 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d877b426-724d-4ccf-a113-05a96ab94935-combined-ca-bundle\") pod \"barbican-keystone-listener-64674fc4f4-wfh52\" (UID: \"d877b426-724d-4ccf-a113-05a96ab94935\") " pod="openstack-kuttl-tests/barbican-keystone-listener-64674fc4f4-wfh52" Dec 03 12:05:16 crc kubenswrapper[5052]: I1203 12:05:16.288707 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-jqslb\" (UniqueName: \"kubernetes.io/projected/a3da2acb-8a0e-4ab3-8d15-2978eca73850-kube-api-access-jqslb\") pod \"barbican-worker-c656b9c77-2gf7g\" (UID: \"a3da2acb-8a0e-4ab3-8d15-2978eca73850\") " pod="openstack-kuttl-tests/barbican-worker-c656b9c77-2gf7g" Dec 03 12:05:16 crc kubenswrapper[5052]: I1203 12:05:16.290483 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-p9lmx\" (UniqueName: \"kubernetes.io/projected/d877b426-724d-4ccf-a113-05a96ab94935-kube-api-access-p9lmx\") pod \"barbican-keystone-listener-64674fc4f4-wfh52\" (UID: \"d877b426-724d-4ccf-a113-05a96ab94935\") " pod="openstack-kuttl-tests/barbican-keystone-listener-64674fc4f4-wfh52" Dec 03 12:05:16 crc kubenswrapper[5052]: I1203 12:05:16.369713 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/44965796-ddc0-4cb1-bdf6-f107b9c3373b-config-data\") pod \"barbican-api-5757755d9b-9m6dz\" (UID: \"44965796-ddc0-4cb1-bdf6-f107b9c3373b\") " pod="openstack-kuttl-tests/barbican-api-5757755d9b-9m6dz" Dec 03 12:05:16 crc kubenswrapper[5052]: I1203 12:05:16.369833 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/44965796-ddc0-4cb1-bdf6-f107b9c3373b-combined-ca-bundle\") pod \"barbican-api-5757755d9b-9m6dz\" (UID: \"44965796-ddc0-4cb1-bdf6-f107b9c3373b\") " pod="openstack-kuttl-tests/barbican-api-5757755d9b-9m6dz" Dec 03 12:05:16 crc kubenswrapper[5052]: I1203 12:05:16.369859 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/44965796-ddc0-4cb1-bdf6-f107b9c3373b-config-data-custom\") pod \"barbican-api-5757755d9b-9m6dz\" (UID: \"44965796-ddc0-4cb1-bdf6-f107b9c3373b\") " pod="openstack-kuttl-tests/barbican-api-5757755d9b-9m6dz" Dec 03 12:05:16 crc kubenswrapper[5052]: I1203 12:05:16.369879 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-kx6k2\" (UniqueName: \"kubernetes.io/projected/44965796-ddc0-4cb1-bdf6-f107b9c3373b-kube-api-access-kx6k2\") pod \"barbican-api-5757755d9b-9m6dz\" (UID: \"44965796-ddc0-4cb1-bdf6-f107b9c3373b\") " pod="openstack-kuttl-tests/barbican-api-5757755d9b-9m6dz" Dec 03 12:05:16 crc kubenswrapper[5052]: I1203 12:05:16.369935 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/44965796-ddc0-4cb1-bdf6-f107b9c3373b-logs\") pod \"barbican-api-5757755d9b-9m6dz\" (UID: \"44965796-ddc0-4cb1-bdf6-f107b9c3373b\") " pod="openstack-kuttl-tests/barbican-api-5757755d9b-9m6dz" Dec 03 12:05:16 crc kubenswrapper[5052]: I1203 12:05:16.370387 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/44965796-ddc0-4cb1-bdf6-f107b9c3373b-logs\") pod \"barbican-api-5757755d9b-9m6dz\" (UID: \"44965796-ddc0-4cb1-bdf6-f107b9c3373b\") " pod="openstack-kuttl-tests/barbican-api-5757755d9b-9m6dz" Dec 03 12:05:16 crc kubenswrapper[5052]: I1203 12:05:16.374982 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/44965796-ddc0-4cb1-bdf6-f107b9c3373b-config-data-custom\") pod \"barbican-api-5757755d9b-9m6dz\" (UID: \"44965796-ddc0-4cb1-bdf6-f107b9c3373b\") " pod="openstack-kuttl-tests/barbican-api-5757755d9b-9m6dz" Dec 03 12:05:16 crc kubenswrapper[5052]: I1203 12:05:16.375440 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-worker-c656b9c77-2gf7g" Dec 03 12:05:16 crc kubenswrapper[5052]: I1203 12:05:16.376120 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/44965796-ddc0-4cb1-bdf6-f107b9c3373b-combined-ca-bundle\") pod \"barbican-api-5757755d9b-9m6dz\" (UID: \"44965796-ddc0-4cb1-bdf6-f107b9c3373b\") " pod="openstack-kuttl-tests/barbican-api-5757755d9b-9m6dz" Dec 03 12:05:16 crc kubenswrapper[5052]: I1203 12:05:16.379337 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/44965796-ddc0-4cb1-bdf6-f107b9c3373b-config-data\") pod \"barbican-api-5757755d9b-9m6dz\" (UID: \"44965796-ddc0-4cb1-bdf6-f107b9c3373b\") " pod="openstack-kuttl-tests/barbican-api-5757755d9b-9m6dz" Dec 03 12:05:16 crc kubenswrapper[5052]: I1203 12:05:16.391827 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-kx6k2\" (UniqueName: \"kubernetes.io/projected/44965796-ddc0-4cb1-bdf6-f107b9c3373b-kube-api-access-kx6k2\") pod \"barbican-api-5757755d9b-9m6dz\" (UID: \"44965796-ddc0-4cb1-bdf6-f107b9c3373b\") " pod="openstack-kuttl-tests/barbican-api-5757755d9b-9m6dz" Dec 03 12:05:16 crc kubenswrapper[5052]: I1203 12:05:16.413812 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-keystone-listener-64674fc4f4-wfh52" Dec 03 12:05:16 crc kubenswrapper[5052]: I1203 12:05:16.503331 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-api-5757755d9b-9m6dz" Dec 03 12:05:16 crc kubenswrapper[5052]: I1203 12:05:16.921042 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-api-5757755d9b-9m6dz"] Dec 03 12:05:16 crc kubenswrapper[5052]: I1203 12:05:16.984520 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-keystone-listener-64674fc4f4-wfh52"] Dec 03 12:05:17 crc kubenswrapper[5052]: I1203 12:05:17.020111 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-worker-c656b9c77-2gf7g"] Dec 03 12:05:17 crc kubenswrapper[5052]: I1203 12:05:17.252199 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-bootstrap-8s4bm" Dec 03 12:05:17 crc kubenswrapper[5052]: I1203 12:05:17.391879 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6588734e-3543-47aa-9df6-f6e21a877732-config-data\") pod \"6588734e-3543-47aa-9df6-f6e21a877732\" (UID: \"6588734e-3543-47aa-9df6-f6e21a877732\") " Dec 03 12:05:17 crc kubenswrapper[5052]: I1203 12:05:17.391953 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gfjsk\" (UniqueName: \"kubernetes.io/projected/6588734e-3543-47aa-9df6-f6e21a877732-kube-api-access-gfjsk\") pod \"6588734e-3543-47aa-9df6-f6e21a877732\" (UID: \"6588734e-3543-47aa-9df6-f6e21a877732\") " Dec 03 12:05:17 crc kubenswrapper[5052]: I1203 12:05:17.391994 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6588734e-3543-47aa-9df6-f6e21a877732-combined-ca-bundle\") pod \"6588734e-3543-47aa-9df6-f6e21a877732\" (UID: \"6588734e-3543-47aa-9df6-f6e21a877732\") " Dec 03 12:05:17 crc kubenswrapper[5052]: I1203 12:05:17.392018 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/6588734e-3543-47aa-9df6-f6e21a877732-fernet-keys\") pod \"6588734e-3543-47aa-9df6-f6e21a877732\" (UID: \"6588734e-3543-47aa-9df6-f6e21a877732\") " Dec 03 12:05:17 crc kubenswrapper[5052]: I1203 12:05:17.392100 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/6588734e-3543-47aa-9df6-f6e21a877732-scripts\") pod \"6588734e-3543-47aa-9df6-f6e21a877732\" (UID: \"6588734e-3543-47aa-9df6-f6e21a877732\") " Dec 03 12:05:17 crc kubenswrapper[5052]: I1203 12:05:17.392169 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/6588734e-3543-47aa-9df6-f6e21a877732-credential-keys\") pod \"6588734e-3543-47aa-9df6-f6e21a877732\" (UID: \"6588734e-3543-47aa-9df6-f6e21a877732\") " Dec 03 12:05:17 crc kubenswrapper[5052]: I1203 12:05:17.399258 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6588734e-3543-47aa-9df6-f6e21a877732-scripts" (OuterVolumeSpecName: "scripts") pod "6588734e-3543-47aa-9df6-f6e21a877732" (UID: "6588734e-3543-47aa-9df6-f6e21a877732"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:05:17 crc kubenswrapper[5052]: I1203 12:05:17.401291 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6588734e-3543-47aa-9df6-f6e21a877732-fernet-keys" (OuterVolumeSpecName: "fernet-keys") pod "6588734e-3543-47aa-9df6-f6e21a877732" (UID: "6588734e-3543-47aa-9df6-f6e21a877732"). InnerVolumeSpecName "fernet-keys". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:05:17 crc kubenswrapper[5052]: I1203 12:05:17.403073 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6588734e-3543-47aa-9df6-f6e21a877732-kube-api-access-gfjsk" (OuterVolumeSpecName: "kube-api-access-gfjsk") pod "6588734e-3543-47aa-9df6-f6e21a877732" (UID: "6588734e-3543-47aa-9df6-f6e21a877732"). InnerVolumeSpecName "kube-api-access-gfjsk". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:05:17 crc kubenswrapper[5052]: I1203 12:05:17.405370 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6588734e-3543-47aa-9df6-f6e21a877732-credential-keys" (OuterVolumeSpecName: "credential-keys") pod "6588734e-3543-47aa-9df6-f6e21a877732" (UID: "6588734e-3543-47aa-9df6-f6e21a877732"). InnerVolumeSpecName "credential-keys". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:05:17 crc kubenswrapper[5052]: I1203 12:05:17.435251 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6588734e-3543-47aa-9df6-f6e21a877732-config-data" (OuterVolumeSpecName: "config-data") pod "6588734e-3543-47aa-9df6-f6e21a877732" (UID: "6588734e-3543-47aa-9df6-f6e21a877732"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:05:17 crc kubenswrapper[5052]: I1203 12:05:17.438643 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6588734e-3543-47aa-9df6-f6e21a877732-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "6588734e-3543-47aa-9df6-f6e21a877732" (UID: "6588734e-3543-47aa-9df6-f6e21a877732"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:05:17 crc kubenswrapper[5052]: I1203 12:05:17.494863 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6588734e-3543-47aa-9df6-f6e21a877732-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:05:17 crc kubenswrapper[5052]: I1203 12:05:17.495065 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-gfjsk\" (UniqueName: \"kubernetes.io/projected/6588734e-3543-47aa-9df6-f6e21a877732-kube-api-access-gfjsk\") on node \"crc\" DevicePath \"\"" Dec 03 12:05:17 crc kubenswrapper[5052]: I1203 12:05:17.495155 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6588734e-3543-47aa-9df6-f6e21a877732-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:05:17 crc kubenswrapper[5052]: I1203 12:05:17.495243 5052 reconciler_common.go:293] "Volume detached for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/6588734e-3543-47aa-9df6-f6e21a877732-fernet-keys\") on node \"crc\" DevicePath \"\"" Dec 03 12:05:17 crc kubenswrapper[5052]: I1203 12:05:17.495326 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/6588734e-3543-47aa-9df6-f6e21a877732-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:05:17 crc kubenswrapper[5052]: I1203 12:05:17.495408 5052 reconciler_common.go:293] "Volume detached for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/6588734e-3543-47aa-9df6-f6e21a877732-credential-keys\") on node \"crc\" DevicePath \"\"" Dec 03 12:05:17 crc kubenswrapper[5052]: I1203 12:05:17.580342 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-db-sync-jhtdm" Dec 03 12:05:17 crc kubenswrapper[5052]: I1203 12:05:17.703288 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/0c8be033-f83e-4d64-b271-f4c97bfdc108-scripts\") pod \"0c8be033-f83e-4d64-b271-f4c97bfdc108\" (UID: \"0c8be033-f83e-4d64-b271-f4c97bfdc108\") " Dec 03 12:05:17 crc kubenswrapper[5052]: I1203 12:05:17.704757 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-57tzg\" (UniqueName: \"kubernetes.io/projected/0c8be033-f83e-4d64-b271-f4c97bfdc108-kube-api-access-57tzg\") pod \"0c8be033-f83e-4d64-b271-f4c97bfdc108\" (UID: \"0c8be033-f83e-4d64-b271-f4c97bfdc108\") " Dec 03 12:05:17 crc kubenswrapper[5052]: I1203 12:05:17.704851 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/0c8be033-f83e-4d64-b271-f4c97bfdc108-db-sync-config-data\") pod \"0c8be033-f83e-4d64-b271-f4c97bfdc108\" (UID: \"0c8be033-f83e-4d64-b271-f4c97bfdc108\") " Dec 03 12:05:17 crc kubenswrapper[5052]: I1203 12:05:17.704890 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/0c8be033-f83e-4d64-b271-f4c97bfdc108-config-data\") pod \"0c8be033-f83e-4d64-b271-f4c97bfdc108\" (UID: \"0c8be033-f83e-4d64-b271-f4c97bfdc108\") " Dec 03 12:05:17 crc kubenswrapper[5052]: I1203 12:05:17.704962 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/0c8be033-f83e-4d64-b271-f4c97bfdc108-etc-machine-id\") pod \"0c8be033-f83e-4d64-b271-f4c97bfdc108\" (UID: \"0c8be033-f83e-4d64-b271-f4c97bfdc108\") " Dec 03 12:05:17 crc kubenswrapper[5052]: I1203 12:05:17.705023 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0c8be033-f83e-4d64-b271-f4c97bfdc108-combined-ca-bundle\") pod \"0c8be033-f83e-4d64-b271-f4c97bfdc108\" (UID: \"0c8be033-f83e-4d64-b271-f4c97bfdc108\") " Dec 03 12:05:17 crc kubenswrapper[5052]: I1203 12:05:17.705855 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/0c8be033-f83e-4d64-b271-f4c97bfdc108-etc-machine-id" (OuterVolumeSpecName: "etc-machine-id") pod "0c8be033-f83e-4d64-b271-f4c97bfdc108" (UID: "0c8be033-f83e-4d64-b271-f4c97bfdc108"). InnerVolumeSpecName "etc-machine-id". PluginName "kubernetes.io/host-path", VolumeGidValue "" Dec 03 12:05:17 crc kubenswrapper[5052]: I1203 12:05:17.706302 5052 reconciler_common.go:293] "Volume detached for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/0c8be033-f83e-4d64-b271-f4c97bfdc108-etc-machine-id\") on node \"crc\" DevicePath \"\"" Dec 03 12:05:17 crc kubenswrapper[5052]: I1203 12:05:17.712571 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0c8be033-f83e-4d64-b271-f4c97bfdc108-kube-api-access-57tzg" (OuterVolumeSpecName: "kube-api-access-57tzg") pod "0c8be033-f83e-4d64-b271-f4c97bfdc108" (UID: "0c8be033-f83e-4d64-b271-f4c97bfdc108"). InnerVolumeSpecName "kube-api-access-57tzg". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:05:17 crc kubenswrapper[5052]: I1203 12:05:17.728665 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0c8be033-f83e-4d64-b271-f4c97bfdc108-scripts" (OuterVolumeSpecName: "scripts") pod "0c8be033-f83e-4d64-b271-f4c97bfdc108" (UID: "0c8be033-f83e-4d64-b271-f4c97bfdc108"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:05:17 crc kubenswrapper[5052]: I1203 12:05:17.739671 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0c8be033-f83e-4d64-b271-f4c97bfdc108-db-sync-config-data" (OuterVolumeSpecName: "db-sync-config-data") pod "0c8be033-f83e-4d64-b271-f4c97bfdc108" (UID: "0c8be033-f83e-4d64-b271-f4c97bfdc108"). InnerVolumeSpecName "db-sync-config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:05:17 crc kubenswrapper[5052]: I1203 12:05:17.758025 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0c8be033-f83e-4d64-b271-f4c97bfdc108-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "0c8be033-f83e-4d64-b271-f4c97bfdc108" (UID: "0c8be033-f83e-4d64-b271-f4c97bfdc108"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:05:17 crc kubenswrapper[5052]: I1203 12:05:17.779542 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0c8be033-f83e-4d64-b271-f4c97bfdc108-config-data" (OuterVolumeSpecName: "config-data") pod "0c8be033-f83e-4d64-b271-f4c97bfdc108" (UID: "0c8be033-f83e-4d64-b271-f4c97bfdc108"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:05:17 crc kubenswrapper[5052]: I1203 12:05:17.809566 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/0c8be033-f83e-4d64-b271-f4c97bfdc108-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:05:17 crc kubenswrapper[5052]: I1203 12:05:17.809603 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-57tzg\" (UniqueName: \"kubernetes.io/projected/0c8be033-f83e-4d64-b271-f4c97bfdc108-kube-api-access-57tzg\") on node \"crc\" DevicePath \"\"" Dec 03 12:05:17 crc kubenswrapper[5052]: I1203 12:05:17.809618 5052 reconciler_common.go:293] "Volume detached for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/0c8be033-f83e-4d64-b271-f4c97bfdc108-db-sync-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:05:17 crc kubenswrapper[5052]: I1203 12:05:17.809629 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/0c8be033-f83e-4d64-b271-f4c97bfdc108-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:05:17 crc kubenswrapper[5052]: I1203 12:05:17.809641 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0c8be033-f83e-4d64-b271-f4c97bfdc108-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:05:17 crc kubenswrapper[5052]: I1203 12:05:17.846779 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-worker-c656b9c77-2gf7g" event={"ID":"a3da2acb-8a0e-4ab3-8d15-2978eca73850","Type":"ContainerStarted","Data":"606f51e20d39e18fce0fe0fe7c840d6c46bd6e03a7e775bc7246b576bec5c132"} Dec 03 12:05:17 crc kubenswrapper[5052]: I1203 12:05:17.846831 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-worker-c656b9c77-2gf7g" event={"ID":"a3da2acb-8a0e-4ab3-8d15-2978eca73850","Type":"ContainerStarted","Data":"80fb2290d9e5fb396a3de096d08940f5a3a89a79f45d052e0d882756596d8b34"} Dec 03 12:05:17 crc kubenswrapper[5052]: I1203 12:05:17.846841 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-worker-c656b9c77-2gf7g" event={"ID":"a3da2acb-8a0e-4ab3-8d15-2978eca73850","Type":"ContainerStarted","Data":"99f10e049c628513e1e1525ed3dbcd424c53f799c5fed159b769c8e0375f0bfc"} Dec 03 12:05:17 crc kubenswrapper[5052]: I1203 12:05:17.860488 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-keystone-listener-64674fc4f4-wfh52" event={"ID":"d877b426-724d-4ccf-a113-05a96ab94935","Type":"ContainerStarted","Data":"8ed05d2153d95fbbc5b927a2573653ac6d68bb4bb432a7dd29017b0640952a6f"} Dec 03 12:05:17 crc kubenswrapper[5052]: I1203 12:05:17.860564 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-keystone-listener-64674fc4f4-wfh52" event={"ID":"d877b426-724d-4ccf-a113-05a96ab94935","Type":"ContainerStarted","Data":"c46f89f982fb61d9c55d36ab8f8e63d0ab12f714f3dcec36d8ab7c7292d44f67"} Dec 03 12:05:17 crc kubenswrapper[5052]: I1203 12:05:17.862150 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-bootstrap-8s4bm" event={"ID":"6588734e-3543-47aa-9df6-f6e21a877732","Type":"ContainerDied","Data":"8ecf1b450bb5b0378ebfc497bc4b3b83396c880a31d27b70702298733fb6fe94"} Dec 03 12:05:17 crc kubenswrapper[5052]: I1203 12:05:17.862186 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8ecf1b450bb5b0378ebfc497bc4b3b83396c880a31d27b70702298733fb6fe94" Dec 03 12:05:17 crc kubenswrapper[5052]: I1203 12:05:17.862261 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-bootstrap-8s4bm" Dec 03 12:05:17 crc kubenswrapper[5052]: I1203 12:05:17.872960 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-db-sync-jhtdm" event={"ID":"0c8be033-f83e-4d64-b271-f4c97bfdc108","Type":"ContainerDied","Data":"5cfed3b0e163b3f1807c6243d2a9a296e29c8f421a7f6aa7bcc9ba12e7700ef8"} Dec 03 12:05:17 crc kubenswrapper[5052]: I1203 12:05:17.873008 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5cfed3b0e163b3f1807c6243d2a9a296e29c8f421a7f6aa7bcc9ba12e7700ef8" Dec 03 12:05:17 crc kubenswrapper[5052]: I1203 12:05:17.873087 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-db-sync-jhtdm" Dec 03 12:05:17 crc kubenswrapper[5052]: I1203 12:05:17.905303 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-5757755d9b-9m6dz" event={"ID":"44965796-ddc0-4cb1-bdf6-f107b9c3373b","Type":"ContainerStarted","Data":"8bc3b52ef1073998ba1546f957d22fef73c229a57932d8bd6026308d8625f311"} Dec 03 12:05:17 crc kubenswrapper[5052]: I1203 12:05:17.905362 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-5757755d9b-9m6dz" event={"ID":"44965796-ddc0-4cb1-bdf6-f107b9c3373b","Type":"ContainerStarted","Data":"2fea933a02532e081e8c1b23f22d0f21410c4756a8d56614e399f1fc7daadcfd"} Dec 03 12:05:17 crc kubenswrapper[5052]: I1203 12:05:17.905375 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-5757755d9b-9m6dz" event={"ID":"44965796-ddc0-4cb1-bdf6-f107b9c3373b","Type":"ContainerStarted","Data":"ef35e9f5b963c8b94868cf33871c4c56c76bcaf97c98b3bb9ba04b62c23d6d34"} Dec 03 12:05:17 crc kubenswrapper[5052]: I1203 12:05:17.953383 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/keystone-6c6748d95d-66jbh"] Dec 03 12:05:17 crc kubenswrapper[5052]: E1203 12:05:17.953876 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="0c8be033-f83e-4d64-b271-f4c97bfdc108" containerName="cinder-db-sync" Dec 03 12:05:17 crc kubenswrapper[5052]: I1203 12:05:17.953899 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="0c8be033-f83e-4d64-b271-f4c97bfdc108" containerName="cinder-db-sync" Dec 03 12:05:17 crc kubenswrapper[5052]: E1203 12:05:17.953910 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="6588734e-3543-47aa-9df6-f6e21a877732" containerName="keystone-bootstrap" Dec 03 12:05:17 crc kubenswrapper[5052]: I1203 12:05:17.953917 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="6588734e-3543-47aa-9df6-f6e21a877732" containerName="keystone-bootstrap" Dec 03 12:05:17 crc kubenswrapper[5052]: I1203 12:05:17.954098 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="0c8be033-f83e-4d64-b271-f4c97bfdc108" containerName="cinder-db-sync" Dec 03 12:05:17 crc kubenswrapper[5052]: I1203 12:05:17.954122 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="6588734e-3543-47aa-9df6-f6e21a877732" containerName="keystone-bootstrap" Dec 03 12:05:17 crc kubenswrapper[5052]: I1203 12:05:17.956031 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-6c6748d95d-66jbh" Dec 03 12:05:17 crc kubenswrapper[5052]: I1203 12:05:17.974447 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone" Dec 03 12:05:17 crc kubenswrapper[5052]: I1203 12:05:17.974670 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone-scripts" Dec 03 12:05:17 crc kubenswrapper[5052]: I1203 12:05:17.976711 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-keystone-internal-svc" Dec 03 12:05:17 crc kubenswrapper[5052]: I1203 12:05:17.977009 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone-config-data" Dec 03 12:05:17 crc kubenswrapper[5052]: I1203 12:05:17.978081 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-keystone-public-svc" Dec 03 12:05:17 crc kubenswrapper[5052]: I1203 12:05:17.982377 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone-keystone-dockercfg-m92zx" Dec 03 12:05:18 crc kubenswrapper[5052]: I1203 12:05:18.012914 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-6c6748d95d-66jbh"] Dec 03 12:05:18 crc kubenswrapper[5052]: I1203 12:05:18.083490 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 12:05:18 crc kubenswrapper[5052]: I1203 12:05:18.085222 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:05:18 crc kubenswrapper[5052]: I1203 12:05:18.097353 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cinder-scheduler-config-data" Dec 03 12:05:18 crc kubenswrapper[5052]: I1203 12:05:18.097553 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cinder-config-data" Dec 03 12:05:18 crc kubenswrapper[5052]: I1203 12:05:18.097753 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cinder-scripts" Dec 03 12:05:18 crc kubenswrapper[5052]: I1203 12:05:18.098030 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cinder-cinder-dockercfg-qpb27" Dec 03 12:05:18 crc kubenswrapper[5052]: I1203 12:05:18.108712 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 12:05:18 crc kubenswrapper[5052]: I1203 12:05:18.132919 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 12:05:18 crc kubenswrapper[5052]: I1203 12:05:18.134948 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:05:18 crc kubenswrapper[5052]: I1203 12:05:18.137229 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7k54s\" (UniqueName: \"kubernetes.io/projected/c2470da9-4c96-44ac-834d-2b560785135f-kube-api-access-7k54s\") pod \"keystone-6c6748d95d-66jbh\" (UID: \"c2470da9-4c96-44ac-834d-2b560785135f\") " pod="openstack-kuttl-tests/keystone-6c6748d95d-66jbh" Dec 03 12:05:18 crc kubenswrapper[5052]: I1203 12:05:18.137267 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/c2470da9-4c96-44ac-834d-2b560785135f-internal-tls-certs\") pod \"keystone-6c6748d95d-66jbh\" (UID: \"c2470da9-4c96-44ac-834d-2b560785135f\") " pod="openstack-kuttl-tests/keystone-6c6748d95d-66jbh" Dec 03 12:05:18 crc kubenswrapper[5052]: I1203 12:05:18.137314 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/c2470da9-4c96-44ac-834d-2b560785135f-credential-keys\") pod \"keystone-6c6748d95d-66jbh\" (UID: \"c2470da9-4c96-44ac-834d-2b560785135f\") " pod="openstack-kuttl-tests/keystone-6c6748d95d-66jbh" Dec 03 12:05:18 crc kubenswrapper[5052]: I1203 12:05:18.137439 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/c2470da9-4c96-44ac-834d-2b560785135f-fernet-keys\") pod \"keystone-6c6748d95d-66jbh\" (UID: \"c2470da9-4c96-44ac-834d-2b560785135f\") " pod="openstack-kuttl-tests/keystone-6c6748d95d-66jbh" Dec 03 12:05:18 crc kubenswrapper[5052]: I1203 12:05:18.137692 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/c2470da9-4c96-44ac-834d-2b560785135f-public-tls-certs\") pod \"keystone-6c6748d95d-66jbh\" (UID: \"c2470da9-4c96-44ac-834d-2b560785135f\") " pod="openstack-kuttl-tests/keystone-6c6748d95d-66jbh" Dec 03 12:05:18 crc kubenswrapper[5052]: I1203 12:05:18.137757 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/c2470da9-4c96-44ac-834d-2b560785135f-scripts\") pod \"keystone-6c6748d95d-66jbh\" (UID: \"c2470da9-4c96-44ac-834d-2b560785135f\") " pod="openstack-kuttl-tests/keystone-6c6748d95d-66jbh" Dec 03 12:05:18 crc kubenswrapper[5052]: I1203 12:05:18.137860 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/c2470da9-4c96-44ac-834d-2b560785135f-config-data\") pod \"keystone-6c6748d95d-66jbh\" (UID: \"c2470da9-4c96-44ac-834d-2b560785135f\") " pod="openstack-kuttl-tests/keystone-6c6748d95d-66jbh" Dec 03 12:05:18 crc kubenswrapper[5052]: I1203 12:05:18.137913 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c2470da9-4c96-44ac-834d-2b560785135f-combined-ca-bundle\") pod \"keystone-6c6748d95d-66jbh\" (UID: \"c2470da9-4c96-44ac-834d-2b560785135f\") " pod="openstack-kuttl-tests/keystone-6c6748d95d-66jbh" Dec 03 12:05:18 crc kubenswrapper[5052]: I1203 12:05:18.142877 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cinder-api-config-data" Dec 03 12:05:18 crc kubenswrapper[5052]: I1203 12:05:18.164602 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 12:05:18 crc kubenswrapper[5052]: I1203 12:05:18.240355 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/ae94377d-8ac6-46e6-95db-703b9b487b4e-config-data\") pod \"cinder-api-0\" (UID: \"ae94377d-8ac6-46e6-95db-703b9b487b4e\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:05:18 crc kubenswrapper[5052]: I1203 12:05:18.240428 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/c2470da9-4c96-44ac-834d-2b560785135f-config-data\") pod \"keystone-6c6748d95d-66jbh\" (UID: \"c2470da9-4c96-44ac-834d-2b560785135f\") " pod="openstack-kuttl-tests/keystone-6c6748d95d-66jbh" Dec 03 12:05:18 crc kubenswrapper[5052]: I1203 12:05:18.240466 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c2470da9-4c96-44ac-834d-2b560785135f-combined-ca-bundle\") pod \"keystone-6c6748d95d-66jbh\" (UID: \"c2470da9-4c96-44ac-834d-2b560785135f\") " pod="openstack-kuttl-tests/keystone-6c6748d95d-66jbh" Dec 03 12:05:18 crc kubenswrapper[5052]: I1203 12:05:18.240528 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/664266a3-132d-4a06-b90b-7b58cde2a6be-scripts\") pod \"cinder-scheduler-0\" (UID: \"664266a3-132d-4a06-b90b-7b58cde2a6be\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:05:18 crc kubenswrapper[5052]: I1203 12:05:18.240594 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-7k54s\" (UniqueName: \"kubernetes.io/projected/c2470da9-4c96-44ac-834d-2b560785135f-kube-api-access-7k54s\") pod \"keystone-6c6748d95d-66jbh\" (UID: \"c2470da9-4c96-44ac-834d-2b560785135f\") " pod="openstack-kuttl-tests/keystone-6c6748d95d-66jbh" Dec 03 12:05:18 crc kubenswrapper[5052]: I1203 12:05:18.240665 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/c2470da9-4c96-44ac-834d-2b560785135f-internal-tls-certs\") pod \"keystone-6c6748d95d-66jbh\" (UID: \"c2470da9-4c96-44ac-834d-2b560785135f\") " pod="openstack-kuttl-tests/keystone-6c6748d95d-66jbh" Dec 03 12:05:18 crc kubenswrapper[5052]: I1203 12:05:18.240711 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/ae94377d-8ac6-46e6-95db-703b9b487b4e-etc-machine-id\") pod \"cinder-api-0\" (UID: \"ae94377d-8ac6-46e6-95db-703b9b487b4e\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:05:18 crc kubenswrapper[5052]: I1203 12:05:18.240733 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/ae94377d-8ac6-46e6-95db-703b9b487b4e-scripts\") pod \"cinder-api-0\" (UID: \"ae94377d-8ac6-46e6-95db-703b9b487b4e\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:05:18 crc kubenswrapper[5052]: I1203 12:05:18.240757 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/ae94377d-8ac6-46e6-95db-703b9b487b4e-logs\") pod \"cinder-api-0\" (UID: \"ae94377d-8ac6-46e6-95db-703b9b487b4e\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:05:18 crc kubenswrapper[5052]: I1203 12:05:18.240781 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ae94377d-8ac6-46e6-95db-703b9b487b4e-combined-ca-bundle\") pod \"cinder-api-0\" (UID: \"ae94377d-8ac6-46e6-95db-703b9b487b4e\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:05:18 crc kubenswrapper[5052]: I1203 12:05:18.240808 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/c2470da9-4c96-44ac-834d-2b560785135f-credential-keys\") pod \"keystone-6c6748d95d-66jbh\" (UID: \"c2470da9-4c96-44ac-834d-2b560785135f\") " pod="openstack-kuttl-tests/keystone-6c6748d95d-66jbh" Dec 03 12:05:18 crc kubenswrapper[5052]: I1203 12:05:18.240831 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/ae94377d-8ac6-46e6-95db-703b9b487b4e-config-data-custom\") pod \"cinder-api-0\" (UID: \"ae94377d-8ac6-46e6-95db-703b9b487b4e\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:05:18 crc kubenswrapper[5052]: I1203 12:05:18.240860 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/664266a3-132d-4a06-b90b-7b58cde2a6be-etc-machine-id\") pod \"cinder-scheduler-0\" (UID: \"664266a3-132d-4a06-b90b-7b58cde2a6be\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:05:18 crc kubenswrapper[5052]: I1203 12:05:18.240880 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/664266a3-132d-4a06-b90b-7b58cde2a6be-config-data-custom\") pod \"cinder-scheduler-0\" (UID: \"664266a3-132d-4a06-b90b-7b58cde2a6be\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:05:18 crc kubenswrapper[5052]: I1203 12:05:18.240923 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4wxqh\" (UniqueName: \"kubernetes.io/projected/ae94377d-8ac6-46e6-95db-703b9b487b4e-kube-api-access-4wxqh\") pod \"cinder-api-0\" (UID: \"ae94377d-8ac6-46e6-95db-703b9b487b4e\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:05:18 crc kubenswrapper[5052]: I1203 12:05:18.240967 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/664266a3-132d-4a06-b90b-7b58cde2a6be-combined-ca-bundle\") pod \"cinder-scheduler-0\" (UID: \"664266a3-132d-4a06-b90b-7b58cde2a6be\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:05:18 crc kubenswrapper[5052]: I1203 12:05:18.241007 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/c2470da9-4c96-44ac-834d-2b560785135f-fernet-keys\") pod \"keystone-6c6748d95d-66jbh\" (UID: \"c2470da9-4c96-44ac-834d-2b560785135f\") " pod="openstack-kuttl-tests/keystone-6c6748d95d-66jbh" Dec 03 12:05:18 crc kubenswrapper[5052]: I1203 12:05:18.241077 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/c2470da9-4c96-44ac-834d-2b560785135f-public-tls-certs\") pod \"keystone-6c6748d95d-66jbh\" (UID: \"c2470da9-4c96-44ac-834d-2b560785135f\") " pod="openstack-kuttl-tests/keystone-6c6748d95d-66jbh" Dec 03 12:05:18 crc kubenswrapper[5052]: I1203 12:05:18.241117 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-shdfc\" (UniqueName: \"kubernetes.io/projected/664266a3-132d-4a06-b90b-7b58cde2a6be-kube-api-access-shdfc\") pod \"cinder-scheduler-0\" (UID: \"664266a3-132d-4a06-b90b-7b58cde2a6be\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:05:18 crc kubenswrapper[5052]: I1203 12:05:18.241139 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/c2470da9-4c96-44ac-834d-2b560785135f-scripts\") pod \"keystone-6c6748d95d-66jbh\" (UID: \"c2470da9-4c96-44ac-834d-2b560785135f\") " pod="openstack-kuttl-tests/keystone-6c6748d95d-66jbh" Dec 03 12:05:18 crc kubenswrapper[5052]: I1203 12:05:18.241172 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/664266a3-132d-4a06-b90b-7b58cde2a6be-config-data\") pod \"cinder-scheduler-0\" (UID: \"664266a3-132d-4a06-b90b-7b58cde2a6be\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:05:18 crc kubenswrapper[5052]: I1203 12:05:18.247115 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/c2470da9-4c96-44ac-834d-2b560785135f-fernet-keys\") pod \"keystone-6c6748d95d-66jbh\" (UID: \"c2470da9-4c96-44ac-834d-2b560785135f\") " pod="openstack-kuttl-tests/keystone-6c6748d95d-66jbh" Dec 03 12:05:18 crc kubenswrapper[5052]: I1203 12:05:18.247648 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/c2470da9-4c96-44ac-834d-2b560785135f-public-tls-certs\") pod \"keystone-6c6748d95d-66jbh\" (UID: \"c2470da9-4c96-44ac-834d-2b560785135f\") " pod="openstack-kuttl-tests/keystone-6c6748d95d-66jbh" Dec 03 12:05:18 crc kubenswrapper[5052]: I1203 12:05:18.249532 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/c2470da9-4c96-44ac-834d-2b560785135f-internal-tls-certs\") pod \"keystone-6c6748d95d-66jbh\" (UID: \"c2470da9-4c96-44ac-834d-2b560785135f\") " pod="openstack-kuttl-tests/keystone-6c6748d95d-66jbh" Dec 03 12:05:18 crc kubenswrapper[5052]: I1203 12:05:18.249664 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/c2470da9-4c96-44ac-834d-2b560785135f-scripts\") pod \"keystone-6c6748d95d-66jbh\" (UID: \"c2470da9-4c96-44ac-834d-2b560785135f\") " pod="openstack-kuttl-tests/keystone-6c6748d95d-66jbh" Dec 03 12:05:18 crc kubenswrapper[5052]: I1203 12:05:18.250227 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c2470da9-4c96-44ac-834d-2b560785135f-combined-ca-bundle\") pod \"keystone-6c6748d95d-66jbh\" (UID: \"c2470da9-4c96-44ac-834d-2b560785135f\") " pod="openstack-kuttl-tests/keystone-6c6748d95d-66jbh" Dec 03 12:05:18 crc kubenswrapper[5052]: I1203 12:05:18.252175 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/c2470da9-4c96-44ac-834d-2b560785135f-config-data\") pod \"keystone-6c6748d95d-66jbh\" (UID: \"c2470da9-4c96-44ac-834d-2b560785135f\") " pod="openstack-kuttl-tests/keystone-6c6748d95d-66jbh" Dec 03 12:05:18 crc kubenswrapper[5052]: I1203 12:05:18.258414 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/c2470da9-4c96-44ac-834d-2b560785135f-credential-keys\") pod \"keystone-6c6748d95d-66jbh\" (UID: \"c2470da9-4c96-44ac-834d-2b560785135f\") " pod="openstack-kuttl-tests/keystone-6c6748d95d-66jbh" Dec 03 12:05:18 crc kubenswrapper[5052]: I1203 12:05:18.301723 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-7k54s\" (UniqueName: \"kubernetes.io/projected/c2470da9-4c96-44ac-834d-2b560785135f-kube-api-access-7k54s\") pod \"keystone-6c6748d95d-66jbh\" (UID: \"c2470da9-4c96-44ac-834d-2b560785135f\") " pod="openstack-kuttl-tests/keystone-6c6748d95d-66jbh" Dec 03 12:05:18 crc kubenswrapper[5052]: I1203 12:05:18.360054 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-shdfc\" (UniqueName: \"kubernetes.io/projected/664266a3-132d-4a06-b90b-7b58cde2a6be-kube-api-access-shdfc\") pod \"cinder-scheduler-0\" (UID: \"664266a3-132d-4a06-b90b-7b58cde2a6be\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:05:18 crc kubenswrapper[5052]: I1203 12:05:18.360102 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/664266a3-132d-4a06-b90b-7b58cde2a6be-config-data\") pod \"cinder-scheduler-0\" (UID: \"664266a3-132d-4a06-b90b-7b58cde2a6be\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:05:18 crc kubenswrapper[5052]: I1203 12:05:18.360135 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/ae94377d-8ac6-46e6-95db-703b9b487b4e-config-data\") pod \"cinder-api-0\" (UID: \"ae94377d-8ac6-46e6-95db-703b9b487b4e\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:05:18 crc kubenswrapper[5052]: I1203 12:05:18.360179 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/664266a3-132d-4a06-b90b-7b58cde2a6be-scripts\") pod \"cinder-scheduler-0\" (UID: \"664266a3-132d-4a06-b90b-7b58cde2a6be\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:05:18 crc kubenswrapper[5052]: I1203 12:05:18.360217 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/ae94377d-8ac6-46e6-95db-703b9b487b4e-etc-machine-id\") pod \"cinder-api-0\" (UID: \"ae94377d-8ac6-46e6-95db-703b9b487b4e\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:05:18 crc kubenswrapper[5052]: I1203 12:05:18.360232 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/ae94377d-8ac6-46e6-95db-703b9b487b4e-scripts\") pod \"cinder-api-0\" (UID: \"ae94377d-8ac6-46e6-95db-703b9b487b4e\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:05:18 crc kubenswrapper[5052]: I1203 12:05:18.360248 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/ae94377d-8ac6-46e6-95db-703b9b487b4e-logs\") pod \"cinder-api-0\" (UID: \"ae94377d-8ac6-46e6-95db-703b9b487b4e\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:05:18 crc kubenswrapper[5052]: I1203 12:05:18.360265 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ae94377d-8ac6-46e6-95db-703b9b487b4e-combined-ca-bundle\") pod \"cinder-api-0\" (UID: \"ae94377d-8ac6-46e6-95db-703b9b487b4e\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:05:18 crc kubenswrapper[5052]: I1203 12:05:18.360280 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/ae94377d-8ac6-46e6-95db-703b9b487b4e-config-data-custom\") pod \"cinder-api-0\" (UID: \"ae94377d-8ac6-46e6-95db-703b9b487b4e\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:05:18 crc kubenswrapper[5052]: I1203 12:05:18.360300 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/664266a3-132d-4a06-b90b-7b58cde2a6be-etc-machine-id\") pod \"cinder-scheduler-0\" (UID: \"664266a3-132d-4a06-b90b-7b58cde2a6be\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:05:18 crc kubenswrapper[5052]: I1203 12:05:18.360314 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/664266a3-132d-4a06-b90b-7b58cde2a6be-config-data-custom\") pod \"cinder-scheduler-0\" (UID: \"664266a3-132d-4a06-b90b-7b58cde2a6be\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:05:18 crc kubenswrapper[5052]: I1203 12:05:18.360335 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-4wxqh\" (UniqueName: \"kubernetes.io/projected/ae94377d-8ac6-46e6-95db-703b9b487b4e-kube-api-access-4wxqh\") pod \"cinder-api-0\" (UID: \"ae94377d-8ac6-46e6-95db-703b9b487b4e\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:05:18 crc kubenswrapper[5052]: I1203 12:05:18.360359 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/664266a3-132d-4a06-b90b-7b58cde2a6be-combined-ca-bundle\") pod \"cinder-scheduler-0\" (UID: \"664266a3-132d-4a06-b90b-7b58cde2a6be\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:05:18 crc kubenswrapper[5052]: I1203 12:05:18.366758 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/ae94377d-8ac6-46e6-95db-703b9b487b4e-etc-machine-id\") pod \"cinder-api-0\" (UID: \"ae94377d-8ac6-46e6-95db-703b9b487b4e\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:05:18 crc kubenswrapper[5052]: I1203 12:05:18.367480 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/ae94377d-8ac6-46e6-95db-703b9b487b4e-scripts\") pod \"cinder-api-0\" (UID: \"ae94377d-8ac6-46e6-95db-703b9b487b4e\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:05:18 crc kubenswrapper[5052]: I1203 12:05:18.367953 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/ae94377d-8ac6-46e6-95db-703b9b487b4e-config-data\") pod \"cinder-api-0\" (UID: \"ae94377d-8ac6-46e6-95db-703b9b487b4e\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:05:18 crc kubenswrapper[5052]: I1203 12:05:18.369288 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/ae94377d-8ac6-46e6-95db-703b9b487b4e-logs\") pod \"cinder-api-0\" (UID: \"ae94377d-8ac6-46e6-95db-703b9b487b4e\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:05:18 crc kubenswrapper[5052]: I1203 12:05:18.369342 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/664266a3-132d-4a06-b90b-7b58cde2a6be-etc-machine-id\") pod \"cinder-scheduler-0\" (UID: \"664266a3-132d-4a06-b90b-7b58cde2a6be\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:05:18 crc kubenswrapper[5052]: I1203 12:05:18.378042 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-6c6748d95d-66jbh" Dec 03 12:05:18 crc kubenswrapper[5052]: I1203 12:05:18.378242 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/664266a3-132d-4a06-b90b-7b58cde2a6be-config-data-custom\") pod \"cinder-scheduler-0\" (UID: \"664266a3-132d-4a06-b90b-7b58cde2a6be\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:05:18 crc kubenswrapper[5052]: I1203 12:05:18.383040 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ae94377d-8ac6-46e6-95db-703b9b487b4e-combined-ca-bundle\") pod \"cinder-api-0\" (UID: \"ae94377d-8ac6-46e6-95db-703b9b487b4e\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:05:18 crc kubenswrapper[5052]: I1203 12:05:18.383922 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/664266a3-132d-4a06-b90b-7b58cde2a6be-scripts\") pod \"cinder-scheduler-0\" (UID: \"664266a3-132d-4a06-b90b-7b58cde2a6be\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:05:18 crc kubenswrapper[5052]: I1203 12:05:18.385467 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/664266a3-132d-4a06-b90b-7b58cde2a6be-config-data\") pod \"cinder-scheduler-0\" (UID: \"664266a3-132d-4a06-b90b-7b58cde2a6be\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:05:18 crc kubenswrapper[5052]: I1203 12:05:18.385900 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/664266a3-132d-4a06-b90b-7b58cde2a6be-combined-ca-bundle\") pod \"cinder-scheduler-0\" (UID: \"664266a3-132d-4a06-b90b-7b58cde2a6be\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:05:18 crc kubenswrapper[5052]: I1203 12:05:18.391664 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-4wxqh\" (UniqueName: \"kubernetes.io/projected/ae94377d-8ac6-46e6-95db-703b9b487b4e-kube-api-access-4wxqh\") pod \"cinder-api-0\" (UID: \"ae94377d-8ac6-46e6-95db-703b9b487b4e\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:05:18 crc kubenswrapper[5052]: I1203 12:05:18.393133 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/ae94377d-8ac6-46e6-95db-703b9b487b4e-config-data-custom\") pod \"cinder-api-0\" (UID: \"ae94377d-8ac6-46e6-95db-703b9b487b4e\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:05:18 crc kubenswrapper[5052]: I1203 12:05:18.418971 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-shdfc\" (UniqueName: \"kubernetes.io/projected/664266a3-132d-4a06-b90b-7b58cde2a6be-kube-api-access-shdfc\") pod \"cinder-scheduler-0\" (UID: \"664266a3-132d-4a06-b90b-7b58cde2a6be\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:05:18 crc kubenswrapper[5052]: I1203 12:05:18.462952 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:05:18 crc kubenswrapper[5052]: I1203 12:05:18.709563 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:05:18 crc kubenswrapper[5052]: W1203 12:05:18.901262 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc2470da9_4c96_44ac_834d_2b560785135f.slice/crio-2a62c47a82e45e290e6522355bc4c339d425819f3278f37e230418c1c7397913 WatchSource:0}: Error finding container 2a62c47a82e45e290e6522355bc4c339d425819f3278f37e230418c1c7397913: Status 404 returned error can't find the container with id 2a62c47a82e45e290e6522355bc4c339d425819f3278f37e230418c1c7397913 Dec 03 12:05:18 crc kubenswrapper[5052]: I1203 12:05:18.902175 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-6c6748d95d-66jbh"] Dec 03 12:05:18 crc kubenswrapper[5052]: I1203 12:05:18.925236 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-keystone-listener-64674fc4f4-wfh52" event={"ID":"d877b426-724d-4ccf-a113-05a96ab94935","Type":"ContainerStarted","Data":"51b740be59469e140e437bc25b7208518a7bb1d9c11f719a8bcca63ec56644a8"} Dec 03 12:05:18 crc kubenswrapper[5052]: I1203 12:05:18.949067 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-6c6748d95d-66jbh" event={"ID":"c2470da9-4c96-44ac-834d-2b560785135f","Type":"ContainerStarted","Data":"2a62c47a82e45e290e6522355bc4c339d425819f3278f37e230418c1c7397913"} Dec 03 12:05:18 crc kubenswrapper[5052]: I1203 12:05:18.949621 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/barbican-api-5757755d9b-9m6dz" Dec 03 12:05:18 crc kubenswrapper[5052]: I1203 12:05:18.949672 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/barbican-api-5757755d9b-9m6dz" Dec 03 12:05:18 crc kubenswrapper[5052]: I1203 12:05:18.967920 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/barbican-keystone-listener-64674fc4f4-wfh52" podStartSLOduration=3.9678962479999997 podStartE2EDuration="3.967896248s" podCreationTimestamp="2025-12-03 12:05:15 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:05:18.94870051 +0000 UTC m=+2709.536942274" watchObservedRunningTime="2025-12-03 12:05:18.967896248 +0000 UTC m=+2709.556138012" Dec 03 12:05:18 crc kubenswrapper[5052]: I1203 12:05:18.989152 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/barbican-worker-c656b9c77-2gf7g" podStartSLOduration=3.988810386 podStartE2EDuration="3.988810386s" podCreationTimestamp="2025-12-03 12:05:15 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:05:18.970498973 +0000 UTC m=+2709.558740737" watchObservedRunningTime="2025-12-03 12:05:18.988810386 +0000 UTC m=+2709.577052150" Dec 03 12:05:19 crc kubenswrapper[5052]: I1203 12:05:19.009326 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/barbican-api-5757755d9b-9m6dz" podStartSLOduration=3.009311392 podStartE2EDuration="3.009311392s" podCreationTimestamp="2025-12-03 12:05:16 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:05:19.006632186 +0000 UTC m=+2709.594873950" watchObservedRunningTime="2025-12-03 12:05:19.009311392 +0000 UTC m=+2709.597553156" Dec 03 12:05:19 crc kubenswrapper[5052]: I1203 12:05:19.049903 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 12:05:19 crc kubenswrapper[5052]: I1203 12:05:19.253432 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 12:05:19 crc kubenswrapper[5052]: I1203 12:05:19.959343 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"664266a3-132d-4a06-b90b-7b58cde2a6be","Type":"ContainerStarted","Data":"a258633e83b65e858760863d0b411747005e97d7da0d42164260ca58f5956e8f"} Dec 03 12:05:19 crc kubenswrapper[5052]: I1203 12:05:19.964049 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-6c6748d95d-66jbh" event={"ID":"c2470da9-4c96-44ac-834d-2b560785135f","Type":"ContainerStarted","Data":"aa4334f0d2af3013c5ace40839543a0d0b6116ced72af25a3daba9191d3d0382"} Dec 03 12:05:19 crc kubenswrapper[5052]: I1203 12:05:19.964121 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/keystone-6c6748d95d-66jbh" Dec 03 12:05:19 crc kubenswrapper[5052]: I1203 12:05:19.966129 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"ae94377d-8ac6-46e6-95db-703b9b487b4e","Type":"ContainerStarted","Data":"606123ccf57439cca4c970e6d1226a1bb4e9da1045f58cb3cb2daf22fda9a70b"} Dec 03 12:05:19 crc kubenswrapper[5052]: I1203 12:05:19.987903 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/keystone-6c6748d95d-66jbh" podStartSLOduration=2.987880089 podStartE2EDuration="2.987880089s" podCreationTimestamp="2025-12-03 12:05:17 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:05:19.979324635 +0000 UTC m=+2710.567566409" watchObservedRunningTime="2025-12-03 12:05:19.987880089 +0000 UTC m=+2710.576121853" Dec 03 12:05:20 crc kubenswrapper[5052]: I1203 12:05:20.984706 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"ae94377d-8ac6-46e6-95db-703b9b487b4e","Type":"ContainerStarted","Data":"1da922994410e8bc5f80bb92596c1689d4cb9fba4571a6d341e2b2f855908b61"} Dec 03 12:05:20 crc kubenswrapper[5052]: I1203 12:05:20.987250 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"664266a3-132d-4a06-b90b-7b58cde2a6be","Type":"ContainerStarted","Data":"93e6d44896e2e21b42567eecae483c6df093a022855acc48c6dd13c597bcc182"} Dec 03 12:05:21 crc kubenswrapper[5052]: I1203 12:05:21.997952 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"664266a3-132d-4a06-b90b-7b58cde2a6be","Type":"ContainerStarted","Data":"a0e5ccb4f4d4f80e8ce1791623b231707bc05e15618e19e8167b50767d14ccfa"} Dec 03 12:05:22 crc kubenswrapper[5052]: I1203 12:05:22.000566 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"ae94377d-8ac6-46e6-95db-703b9b487b4e","Type":"ContainerStarted","Data":"cb3e7bf1263ab88bc50c078fd8de7fa7d30cbadc8af9782efaaba4ec852a4020"} Dec 03 12:05:22 crc kubenswrapper[5052]: I1203 12:05:22.000746 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:05:22 crc kubenswrapper[5052]: I1203 12:05:22.025857 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/cinder-scheduler-0" podStartSLOduration=4.025838434 podStartE2EDuration="4.025838434s" podCreationTimestamp="2025-12-03 12:05:18 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:05:22.016802125 +0000 UTC m=+2712.605043899" watchObservedRunningTime="2025-12-03 12:05:22.025838434 +0000 UTC m=+2712.614080198" Dec 03 12:05:22 crc kubenswrapper[5052]: I1203 12:05:22.090441 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/cinder-api-0" podStartSLOduration=4.09041877 podStartE2EDuration="4.09041877s" podCreationTimestamp="2025-12-03 12:05:18 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:05:22.079277912 +0000 UTC m=+2712.667519676" watchObservedRunningTime="2025-12-03 12:05:22.09041877 +0000 UTC m=+2712.678660544" Dec 03 12:05:22 crc kubenswrapper[5052]: I1203 12:05:22.314381 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 12:05:22 crc kubenswrapper[5052]: I1203 12:05:22.930457 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/barbican-api-d978c699b-kzzst"] Dec 03 12:05:22 crc kubenswrapper[5052]: I1203 12:05:22.932405 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-api-d978c699b-kzzst" Dec 03 12:05:22 crc kubenswrapper[5052]: I1203 12:05:22.935174 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-barbican-public-svc" Dec 03 12:05:22 crc kubenswrapper[5052]: I1203 12:05:22.935312 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-barbican-internal-svc" Dec 03 12:05:22 crc kubenswrapper[5052]: I1203 12:05:22.952410 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-api-d978c699b-kzzst"] Dec 03 12:05:23 crc kubenswrapper[5052]: I1203 12:05:23.099612 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/de04776b-ea02-40d9-8459-0a519c8c64ad-config-data-custom\") pod \"barbican-api-d978c699b-kzzst\" (UID: \"de04776b-ea02-40d9-8459-0a519c8c64ad\") " pod="openstack-kuttl-tests/barbican-api-d978c699b-kzzst" Dec 03 12:05:23 crc kubenswrapper[5052]: I1203 12:05:23.099697 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/de04776b-ea02-40d9-8459-0a519c8c64ad-combined-ca-bundle\") pod \"barbican-api-d978c699b-kzzst\" (UID: \"de04776b-ea02-40d9-8459-0a519c8c64ad\") " pod="openstack-kuttl-tests/barbican-api-d978c699b-kzzst" Dec 03 12:05:23 crc kubenswrapper[5052]: I1203 12:05:23.099911 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/de04776b-ea02-40d9-8459-0a519c8c64ad-public-tls-certs\") pod \"barbican-api-d978c699b-kzzst\" (UID: \"de04776b-ea02-40d9-8459-0a519c8c64ad\") " pod="openstack-kuttl-tests/barbican-api-d978c699b-kzzst" Dec 03 12:05:23 crc kubenswrapper[5052]: I1203 12:05:23.100102 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/de04776b-ea02-40d9-8459-0a519c8c64ad-config-data\") pod \"barbican-api-d978c699b-kzzst\" (UID: \"de04776b-ea02-40d9-8459-0a519c8c64ad\") " pod="openstack-kuttl-tests/barbican-api-d978c699b-kzzst" Dec 03 12:05:23 crc kubenswrapper[5052]: I1203 12:05:23.100153 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/de04776b-ea02-40d9-8459-0a519c8c64ad-logs\") pod \"barbican-api-d978c699b-kzzst\" (UID: \"de04776b-ea02-40d9-8459-0a519c8c64ad\") " pod="openstack-kuttl-tests/barbican-api-d978c699b-kzzst" Dec 03 12:05:23 crc kubenswrapper[5052]: I1203 12:05:23.100378 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-82xxb\" (UniqueName: \"kubernetes.io/projected/de04776b-ea02-40d9-8459-0a519c8c64ad-kube-api-access-82xxb\") pod \"barbican-api-d978c699b-kzzst\" (UID: \"de04776b-ea02-40d9-8459-0a519c8c64ad\") " pod="openstack-kuttl-tests/barbican-api-d978c699b-kzzst" Dec 03 12:05:23 crc kubenswrapper[5052]: I1203 12:05:23.100753 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/de04776b-ea02-40d9-8459-0a519c8c64ad-internal-tls-certs\") pod \"barbican-api-d978c699b-kzzst\" (UID: \"de04776b-ea02-40d9-8459-0a519c8c64ad\") " pod="openstack-kuttl-tests/barbican-api-d978c699b-kzzst" Dec 03 12:05:23 crc kubenswrapper[5052]: I1203 12:05:23.202351 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/de04776b-ea02-40d9-8459-0a519c8c64ad-config-data\") pod \"barbican-api-d978c699b-kzzst\" (UID: \"de04776b-ea02-40d9-8459-0a519c8c64ad\") " pod="openstack-kuttl-tests/barbican-api-d978c699b-kzzst" Dec 03 12:05:23 crc kubenswrapper[5052]: I1203 12:05:23.202402 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/de04776b-ea02-40d9-8459-0a519c8c64ad-logs\") pod \"barbican-api-d978c699b-kzzst\" (UID: \"de04776b-ea02-40d9-8459-0a519c8c64ad\") " pod="openstack-kuttl-tests/barbican-api-d978c699b-kzzst" Dec 03 12:05:23 crc kubenswrapper[5052]: I1203 12:05:23.202440 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-82xxb\" (UniqueName: \"kubernetes.io/projected/de04776b-ea02-40d9-8459-0a519c8c64ad-kube-api-access-82xxb\") pod \"barbican-api-d978c699b-kzzst\" (UID: \"de04776b-ea02-40d9-8459-0a519c8c64ad\") " pod="openstack-kuttl-tests/barbican-api-d978c699b-kzzst" Dec 03 12:05:23 crc kubenswrapper[5052]: I1203 12:05:23.202556 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/de04776b-ea02-40d9-8459-0a519c8c64ad-internal-tls-certs\") pod \"barbican-api-d978c699b-kzzst\" (UID: \"de04776b-ea02-40d9-8459-0a519c8c64ad\") " pod="openstack-kuttl-tests/barbican-api-d978c699b-kzzst" Dec 03 12:05:23 crc kubenswrapper[5052]: I1203 12:05:23.202625 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/de04776b-ea02-40d9-8459-0a519c8c64ad-config-data-custom\") pod \"barbican-api-d978c699b-kzzst\" (UID: \"de04776b-ea02-40d9-8459-0a519c8c64ad\") " pod="openstack-kuttl-tests/barbican-api-d978c699b-kzzst" Dec 03 12:05:23 crc kubenswrapper[5052]: I1203 12:05:23.202658 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/de04776b-ea02-40d9-8459-0a519c8c64ad-combined-ca-bundle\") pod \"barbican-api-d978c699b-kzzst\" (UID: \"de04776b-ea02-40d9-8459-0a519c8c64ad\") " pod="openstack-kuttl-tests/barbican-api-d978c699b-kzzst" Dec 03 12:05:23 crc kubenswrapper[5052]: I1203 12:05:23.202712 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/de04776b-ea02-40d9-8459-0a519c8c64ad-public-tls-certs\") pod \"barbican-api-d978c699b-kzzst\" (UID: \"de04776b-ea02-40d9-8459-0a519c8c64ad\") " pod="openstack-kuttl-tests/barbican-api-d978c699b-kzzst" Dec 03 12:05:23 crc kubenswrapper[5052]: I1203 12:05:23.202931 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/de04776b-ea02-40d9-8459-0a519c8c64ad-logs\") pod \"barbican-api-d978c699b-kzzst\" (UID: \"de04776b-ea02-40d9-8459-0a519c8c64ad\") " pod="openstack-kuttl-tests/barbican-api-d978c699b-kzzst" Dec 03 12:05:23 crc kubenswrapper[5052]: I1203 12:05:23.209109 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/de04776b-ea02-40d9-8459-0a519c8c64ad-config-data-custom\") pod \"barbican-api-d978c699b-kzzst\" (UID: \"de04776b-ea02-40d9-8459-0a519c8c64ad\") " pod="openstack-kuttl-tests/barbican-api-d978c699b-kzzst" Dec 03 12:05:23 crc kubenswrapper[5052]: I1203 12:05:23.209204 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/de04776b-ea02-40d9-8459-0a519c8c64ad-combined-ca-bundle\") pod \"barbican-api-d978c699b-kzzst\" (UID: \"de04776b-ea02-40d9-8459-0a519c8c64ad\") " pod="openstack-kuttl-tests/barbican-api-d978c699b-kzzst" Dec 03 12:05:23 crc kubenswrapper[5052]: I1203 12:05:23.210255 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/de04776b-ea02-40d9-8459-0a519c8c64ad-config-data\") pod \"barbican-api-d978c699b-kzzst\" (UID: \"de04776b-ea02-40d9-8459-0a519c8c64ad\") " pod="openstack-kuttl-tests/barbican-api-d978c699b-kzzst" Dec 03 12:05:23 crc kubenswrapper[5052]: I1203 12:05:23.218417 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/de04776b-ea02-40d9-8459-0a519c8c64ad-public-tls-certs\") pod \"barbican-api-d978c699b-kzzst\" (UID: \"de04776b-ea02-40d9-8459-0a519c8c64ad\") " pod="openstack-kuttl-tests/barbican-api-d978c699b-kzzst" Dec 03 12:05:23 crc kubenswrapper[5052]: I1203 12:05:23.218421 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/de04776b-ea02-40d9-8459-0a519c8c64ad-internal-tls-certs\") pod \"barbican-api-d978c699b-kzzst\" (UID: \"de04776b-ea02-40d9-8459-0a519c8c64ad\") " pod="openstack-kuttl-tests/barbican-api-d978c699b-kzzst" Dec 03 12:05:23 crc kubenswrapper[5052]: I1203 12:05:23.238679 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-82xxb\" (UniqueName: \"kubernetes.io/projected/de04776b-ea02-40d9-8459-0a519c8c64ad-kube-api-access-82xxb\") pod \"barbican-api-d978c699b-kzzst\" (UID: \"de04776b-ea02-40d9-8459-0a519c8c64ad\") " pod="openstack-kuttl-tests/barbican-api-d978c699b-kzzst" Dec 03 12:05:23 crc kubenswrapper[5052]: I1203 12:05:23.254923 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-api-d978c699b-kzzst" Dec 03 12:05:23 crc kubenswrapper[5052]: I1203 12:05:23.709932 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:05:23 crc kubenswrapper[5052]: I1203 12:05:23.825441 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-api-d978c699b-kzzst"] Dec 03 12:05:23 crc kubenswrapper[5052]: W1203 12:05:23.827980 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podde04776b_ea02_40d9_8459_0a519c8c64ad.slice/crio-f15409ad9070b89cee38b736c2856f93ede4172edddf0ae15f61386e4de6869c WatchSource:0}: Error finding container f15409ad9070b89cee38b736c2856f93ede4172edddf0ae15f61386e4de6869c: Status 404 returned error can't find the container with id f15409ad9070b89cee38b736c2856f93ede4172edddf0ae15f61386e4de6869c Dec 03 12:05:24 crc kubenswrapper[5052]: I1203 12:05:24.030903 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-d978c699b-kzzst" event={"ID":"de04776b-ea02-40d9-8459-0a519c8c64ad","Type":"ContainerStarted","Data":"f15409ad9070b89cee38b736c2856f93ede4172edddf0ae15f61386e4de6869c"} Dec 03 12:05:24 crc kubenswrapper[5052]: I1203 12:05:24.031348 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/cinder-api-0" podUID="ae94377d-8ac6-46e6-95db-703b9b487b4e" containerName="cinder-api-log" containerID="cri-o://1da922994410e8bc5f80bb92596c1689d4cb9fba4571a6d341e2b2f855908b61" gracePeriod=30 Dec 03 12:05:24 crc kubenswrapper[5052]: I1203 12:05:24.031673 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/cinder-api-0" podUID="ae94377d-8ac6-46e6-95db-703b9b487b4e" containerName="cinder-api" containerID="cri-o://cb3e7bf1263ab88bc50c078fd8de7fa7d30cbadc8af9782efaaba4ec852a4020" gracePeriod=30 Dec 03 12:05:24 crc kubenswrapper[5052]: I1203 12:05:24.803605 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:05:24 crc kubenswrapper[5052]: I1203 12:05:24.937142 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4wxqh\" (UniqueName: \"kubernetes.io/projected/ae94377d-8ac6-46e6-95db-703b9b487b4e-kube-api-access-4wxqh\") pod \"ae94377d-8ac6-46e6-95db-703b9b487b4e\" (UID: \"ae94377d-8ac6-46e6-95db-703b9b487b4e\") " Dec 03 12:05:24 crc kubenswrapper[5052]: I1203 12:05:24.937372 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/ae94377d-8ac6-46e6-95db-703b9b487b4e-config-data\") pod \"ae94377d-8ac6-46e6-95db-703b9b487b4e\" (UID: \"ae94377d-8ac6-46e6-95db-703b9b487b4e\") " Dec 03 12:05:24 crc kubenswrapper[5052]: I1203 12:05:24.937404 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/ae94377d-8ac6-46e6-95db-703b9b487b4e-config-data-custom\") pod \"ae94377d-8ac6-46e6-95db-703b9b487b4e\" (UID: \"ae94377d-8ac6-46e6-95db-703b9b487b4e\") " Dec 03 12:05:24 crc kubenswrapper[5052]: I1203 12:05:24.937593 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/ae94377d-8ac6-46e6-95db-703b9b487b4e-scripts\") pod \"ae94377d-8ac6-46e6-95db-703b9b487b4e\" (UID: \"ae94377d-8ac6-46e6-95db-703b9b487b4e\") " Dec 03 12:05:24 crc kubenswrapper[5052]: I1203 12:05:24.937641 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ae94377d-8ac6-46e6-95db-703b9b487b4e-combined-ca-bundle\") pod \"ae94377d-8ac6-46e6-95db-703b9b487b4e\" (UID: \"ae94377d-8ac6-46e6-95db-703b9b487b4e\") " Dec 03 12:05:24 crc kubenswrapper[5052]: I1203 12:05:24.937682 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/ae94377d-8ac6-46e6-95db-703b9b487b4e-logs\") pod \"ae94377d-8ac6-46e6-95db-703b9b487b4e\" (UID: \"ae94377d-8ac6-46e6-95db-703b9b487b4e\") " Dec 03 12:05:24 crc kubenswrapper[5052]: I1203 12:05:24.937763 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/ae94377d-8ac6-46e6-95db-703b9b487b4e-etc-machine-id\") pod \"ae94377d-8ac6-46e6-95db-703b9b487b4e\" (UID: \"ae94377d-8ac6-46e6-95db-703b9b487b4e\") " Dec 03 12:05:24 crc kubenswrapper[5052]: I1203 12:05:24.937940 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/ae94377d-8ac6-46e6-95db-703b9b487b4e-etc-machine-id" (OuterVolumeSpecName: "etc-machine-id") pod "ae94377d-8ac6-46e6-95db-703b9b487b4e" (UID: "ae94377d-8ac6-46e6-95db-703b9b487b4e"). InnerVolumeSpecName "etc-machine-id". PluginName "kubernetes.io/host-path", VolumeGidValue "" Dec 03 12:05:24 crc kubenswrapper[5052]: I1203 12:05:24.938032 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae94377d-8ac6-46e6-95db-703b9b487b4e-logs" (OuterVolumeSpecName: "logs") pod "ae94377d-8ac6-46e6-95db-703b9b487b4e" (UID: "ae94377d-8ac6-46e6-95db-703b9b487b4e"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:05:24 crc kubenswrapper[5052]: I1203 12:05:24.938580 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/ae94377d-8ac6-46e6-95db-703b9b487b4e-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:05:24 crc kubenswrapper[5052]: I1203 12:05:24.938601 5052 reconciler_common.go:293] "Volume detached for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/ae94377d-8ac6-46e6-95db-703b9b487b4e-etc-machine-id\") on node \"crc\" DevicePath \"\"" Dec 03 12:05:24 crc kubenswrapper[5052]: I1203 12:05:24.943652 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ae94377d-8ac6-46e6-95db-703b9b487b4e-config-data-custom" (OuterVolumeSpecName: "config-data-custom") pod "ae94377d-8ac6-46e6-95db-703b9b487b4e" (UID: "ae94377d-8ac6-46e6-95db-703b9b487b4e"). InnerVolumeSpecName "config-data-custom". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:05:24 crc kubenswrapper[5052]: I1203 12:05:24.943708 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ae94377d-8ac6-46e6-95db-703b9b487b4e-scripts" (OuterVolumeSpecName: "scripts") pod "ae94377d-8ac6-46e6-95db-703b9b487b4e" (UID: "ae94377d-8ac6-46e6-95db-703b9b487b4e"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:05:24 crc kubenswrapper[5052]: I1203 12:05:24.943780 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ae94377d-8ac6-46e6-95db-703b9b487b4e-kube-api-access-4wxqh" (OuterVolumeSpecName: "kube-api-access-4wxqh") pod "ae94377d-8ac6-46e6-95db-703b9b487b4e" (UID: "ae94377d-8ac6-46e6-95db-703b9b487b4e"). InnerVolumeSpecName "kube-api-access-4wxqh". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:05:24 crc kubenswrapper[5052]: I1203 12:05:24.965486 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ae94377d-8ac6-46e6-95db-703b9b487b4e-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "ae94377d-8ac6-46e6-95db-703b9b487b4e" (UID: "ae94377d-8ac6-46e6-95db-703b9b487b4e"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:05:24 crc kubenswrapper[5052]: I1203 12:05:24.986819 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ae94377d-8ac6-46e6-95db-703b9b487b4e-config-data" (OuterVolumeSpecName: "config-data") pod "ae94377d-8ac6-46e6-95db-703b9b487b4e" (UID: "ae94377d-8ac6-46e6-95db-703b9b487b4e"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:05:25 crc kubenswrapper[5052]: I1203 12:05:25.042847 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/ae94377d-8ac6-46e6-95db-703b9b487b4e-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:05:25 crc kubenswrapper[5052]: I1203 12:05:25.042873 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ae94377d-8ac6-46e6-95db-703b9b487b4e-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:05:25 crc kubenswrapper[5052]: I1203 12:05:25.042883 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-4wxqh\" (UniqueName: \"kubernetes.io/projected/ae94377d-8ac6-46e6-95db-703b9b487b4e-kube-api-access-4wxqh\") on node \"crc\" DevicePath \"\"" Dec 03 12:05:25 crc kubenswrapper[5052]: I1203 12:05:25.042892 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/ae94377d-8ac6-46e6-95db-703b9b487b4e-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:05:25 crc kubenswrapper[5052]: I1203 12:05:25.042901 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/ae94377d-8ac6-46e6-95db-703b9b487b4e-config-data-custom\") on node \"crc\" DevicePath \"\"" Dec 03 12:05:25 crc kubenswrapper[5052]: I1203 12:05:25.044303 5052 generic.go:334] "Generic (PLEG): container finished" podID="ae94377d-8ac6-46e6-95db-703b9b487b4e" containerID="cb3e7bf1263ab88bc50c078fd8de7fa7d30cbadc8af9782efaaba4ec852a4020" exitCode=0 Dec 03 12:05:25 crc kubenswrapper[5052]: I1203 12:05:25.044345 5052 generic.go:334] "Generic (PLEG): container finished" podID="ae94377d-8ac6-46e6-95db-703b9b487b4e" containerID="1da922994410e8bc5f80bb92596c1689d4cb9fba4571a6d341e2b2f855908b61" exitCode=143 Dec 03 12:05:25 crc kubenswrapper[5052]: I1203 12:05:25.044396 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"ae94377d-8ac6-46e6-95db-703b9b487b4e","Type":"ContainerDied","Data":"cb3e7bf1263ab88bc50c078fd8de7fa7d30cbadc8af9782efaaba4ec852a4020"} Dec 03 12:05:25 crc kubenswrapper[5052]: I1203 12:05:25.044438 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"ae94377d-8ac6-46e6-95db-703b9b487b4e","Type":"ContainerDied","Data":"1da922994410e8bc5f80bb92596c1689d4cb9fba4571a6d341e2b2f855908b61"} Dec 03 12:05:25 crc kubenswrapper[5052]: I1203 12:05:25.044452 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"ae94377d-8ac6-46e6-95db-703b9b487b4e","Type":"ContainerDied","Data":"606123ccf57439cca4c970e6d1226a1bb4e9da1045f58cb3cb2daf22fda9a70b"} Dec 03 12:05:25 crc kubenswrapper[5052]: I1203 12:05:25.044467 5052 scope.go:117] "RemoveContainer" containerID="cb3e7bf1263ab88bc50c078fd8de7fa7d30cbadc8af9782efaaba4ec852a4020" Dec 03 12:05:25 crc kubenswrapper[5052]: I1203 12:05:25.044637 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:05:25 crc kubenswrapper[5052]: I1203 12:05:25.050270 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-d978c699b-kzzst" event={"ID":"de04776b-ea02-40d9-8459-0a519c8c64ad","Type":"ContainerStarted","Data":"ebd10ebaaccf168282fe14fc0fe11aa08dcb1e7da4e422f6b5dfdc5b8ddb9b23"} Dec 03 12:05:25 crc kubenswrapper[5052]: I1203 12:05:25.050324 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-d978c699b-kzzst" event={"ID":"de04776b-ea02-40d9-8459-0a519c8c64ad","Type":"ContainerStarted","Data":"649a53bf9bcdd8f3f358ecd746d8c1146c6de92e3ce56e6a224366e0dffc9ed0"} Dec 03 12:05:25 crc kubenswrapper[5052]: I1203 12:05:25.050919 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/barbican-api-d978c699b-kzzst" Dec 03 12:05:25 crc kubenswrapper[5052]: I1203 12:05:25.050947 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/barbican-api-d978c699b-kzzst" Dec 03 12:05:25 crc kubenswrapper[5052]: I1203 12:05:25.069303 5052 scope.go:117] "RemoveContainer" containerID="1da922994410e8bc5f80bb92596c1689d4cb9fba4571a6d341e2b2f855908b61" Dec 03 12:05:25 crc kubenswrapper[5052]: I1203 12:05:25.089657 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/barbican-api-d978c699b-kzzst" podStartSLOduration=3.089631605 podStartE2EDuration="3.089631605s" podCreationTimestamp="2025-12-03 12:05:22 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:05:25.074270726 +0000 UTC m=+2715.662512500" watchObservedRunningTime="2025-12-03 12:05:25.089631605 +0000 UTC m=+2715.677873369" Dec 03 12:05:25 crc kubenswrapper[5052]: I1203 12:05:25.105950 5052 scope.go:117] "RemoveContainer" containerID="cb3e7bf1263ab88bc50c078fd8de7fa7d30cbadc8af9782efaaba4ec852a4020" Dec 03 12:05:25 crc kubenswrapper[5052]: E1203 12:05:25.106413 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cb3e7bf1263ab88bc50c078fd8de7fa7d30cbadc8af9782efaaba4ec852a4020\": container with ID starting with cb3e7bf1263ab88bc50c078fd8de7fa7d30cbadc8af9782efaaba4ec852a4020 not found: ID does not exist" containerID="cb3e7bf1263ab88bc50c078fd8de7fa7d30cbadc8af9782efaaba4ec852a4020" Dec 03 12:05:25 crc kubenswrapper[5052]: I1203 12:05:25.106504 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cb3e7bf1263ab88bc50c078fd8de7fa7d30cbadc8af9782efaaba4ec852a4020"} err="failed to get container status \"cb3e7bf1263ab88bc50c078fd8de7fa7d30cbadc8af9782efaaba4ec852a4020\": rpc error: code = NotFound desc = could not find container \"cb3e7bf1263ab88bc50c078fd8de7fa7d30cbadc8af9782efaaba4ec852a4020\": container with ID starting with cb3e7bf1263ab88bc50c078fd8de7fa7d30cbadc8af9782efaaba4ec852a4020 not found: ID does not exist" Dec 03 12:05:25 crc kubenswrapper[5052]: I1203 12:05:25.106605 5052 scope.go:117] "RemoveContainer" containerID="1da922994410e8bc5f80bb92596c1689d4cb9fba4571a6d341e2b2f855908b61" Dec 03 12:05:25 crc kubenswrapper[5052]: E1203 12:05:25.107127 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1da922994410e8bc5f80bb92596c1689d4cb9fba4571a6d341e2b2f855908b61\": container with ID starting with 1da922994410e8bc5f80bb92596c1689d4cb9fba4571a6d341e2b2f855908b61 not found: ID does not exist" containerID="1da922994410e8bc5f80bb92596c1689d4cb9fba4571a6d341e2b2f855908b61" Dec 03 12:05:25 crc kubenswrapper[5052]: I1203 12:05:25.107198 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1da922994410e8bc5f80bb92596c1689d4cb9fba4571a6d341e2b2f855908b61"} err="failed to get container status \"1da922994410e8bc5f80bb92596c1689d4cb9fba4571a6d341e2b2f855908b61\": rpc error: code = NotFound desc = could not find container \"1da922994410e8bc5f80bb92596c1689d4cb9fba4571a6d341e2b2f855908b61\": container with ID starting with 1da922994410e8bc5f80bb92596c1689d4cb9fba4571a6d341e2b2f855908b61 not found: ID does not exist" Dec 03 12:05:25 crc kubenswrapper[5052]: I1203 12:05:25.107267 5052 scope.go:117] "RemoveContainer" containerID="cb3e7bf1263ab88bc50c078fd8de7fa7d30cbadc8af9782efaaba4ec852a4020" Dec 03 12:05:25 crc kubenswrapper[5052]: I1203 12:05:25.107961 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cb3e7bf1263ab88bc50c078fd8de7fa7d30cbadc8af9782efaaba4ec852a4020"} err="failed to get container status \"cb3e7bf1263ab88bc50c078fd8de7fa7d30cbadc8af9782efaaba4ec852a4020\": rpc error: code = NotFound desc = could not find container \"cb3e7bf1263ab88bc50c078fd8de7fa7d30cbadc8af9782efaaba4ec852a4020\": container with ID starting with cb3e7bf1263ab88bc50c078fd8de7fa7d30cbadc8af9782efaaba4ec852a4020 not found: ID does not exist" Dec 03 12:05:25 crc kubenswrapper[5052]: I1203 12:05:25.108055 5052 scope.go:117] "RemoveContainer" containerID="1da922994410e8bc5f80bb92596c1689d4cb9fba4571a6d341e2b2f855908b61" Dec 03 12:05:25 crc kubenswrapper[5052]: I1203 12:05:25.110152 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1da922994410e8bc5f80bb92596c1689d4cb9fba4571a6d341e2b2f855908b61"} err="failed to get container status \"1da922994410e8bc5f80bb92596c1689d4cb9fba4571a6d341e2b2f855908b61\": rpc error: code = NotFound desc = could not find container \"1da922994410e8bc5f80bb92596c1689d4cb9fba4571a6d341e2b2f855908b61\": container with ID starting with 1da922994410e8bc5f80bb92596c1689d4cb9fba4571a6d341e2b2f855908b61 not found: ID does not exist" Dec 03 12:05:25 crc kubenswrapper[5052]: I1203 12:05:25.114377 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 12:05:25 crc kubenswrapper[5052]: I1203 12:05:25.123684 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 12:05:25 crc kubenswrapper[5052]: I1203 12:05:25.134364 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 12:05:25 crc kubenswrapper[5052]: E1203 12:05:25.134982 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="ae94377d-8ac6-46e6-95db-703b9b487b4e" containerName="cinder-api" Dec 03 12:05:25 crc kubenswrapper[5052]: I1203 12:05:25.135069 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae94377d-8ac6-46e6-95db-703b9b487b4e" containerName="cinder-api" Dec 03 12:05:25 crc kubenswrapper[5052]: E1203 12:05:25.135245 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="ae94377d-8ac6-46e6-95db-703b9b487b4e" containerName="cinder-api-log" Dec 03 12:05:25 crc kubenswrapper[5052]: I1203 12:05:25.135319 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae94377d-8ac6-46e6-95db-703b9b487b4e" containerName="cinder-api-log" Dec 03 12:05:25 crc kubenswrapper[5052]: I1203 12:05:25.135616 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="ae94377d-8ac6-46e6-95db-703b9b487b4e" containerName="cinder-api" Dec 03 12:05:25 crc kubenswrapper[5052]: I1203 12:05:25.135704 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="ae94377d-8ac6-46e6-95db-703b9b487b4e" containerName="cinder-api-log" Dec 03 12:05:25 crc kubenswrapper[5052]: I1203 12:05:25.136910 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:05:25 crc kubenswrapper[5052]: I1203 12:05:25.141653 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-cinder-internal-svc" Dec 03 12:05:25 crc kubenswrapper[5052]: I1203 12:05:25.141923 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cinder-api-config-data" Dec 03 12:05:25 crc kubenswrapper[5052]: I1203 12:05:25.142087 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-cinder-public-svc" Dec 03 12:05:25 crc kubenswrapper[5052]: I1203 12:05:25.154228 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 12:05:25 crc kubenswrapper[5052]: I1203 12:05:25.246778 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/31e6a111-bd4a-45a5-95a0-a6b537added6-public-tls-certs\") pod \"cinder-api-0\" (UID: \"31e6a111-bd4a-45a5-95a0-a6b537added6\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:05:25 crc kubenswrapper[5052]: I1203 12:05:25.246829 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/31e6a111-bd4a-45a5-95a0-a6b537added6-combined-ca-bundle\") pod \"cinder-api-0\" (UID: \"31e6a111-bd4a-45a5-95a0-a6b537added6\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:05:25 crc kubenswrapper[5052]: I1203 12:05:25.246862 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7jrwp\" (UniqueName: \"kubernetes.io/projected/31e6a111-bd4a-45a5-95a0-a6b537added6-kube-api-access-7jrwp\") pod \"cinder-api-0\" (UID: \"31e6a111-bd4a-45a5-95a0-a6b537added6\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:05:25 crc kubenswrapper[5052]: I1203 12:05:25.246896 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/31e6a111-bd4a-45a5-95a0-a6b537added6-internal-tls-certs\") pod \"cinder-api-0\" (UID: \"31e6a111-bd4a-45a5-95a0-a6b537added6\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:05:25 crc kubenswrapper[5052]: I1203 12:05:25.246926 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/31e6a111-bd4a-45a5-95a0-a6b537added6-etc-machine-id\") pod \"cinder-api-0\" (UID: \"31e6a111-bd4a-45a5-95a0-a6b537added6\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:05:25 crc kubenswrapper[5052]: I1203 12:05:25.246957 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/31e6a111-bd4a-45a5-95a0-a6b537added6-config-data\") pod \"cinder-api-0\" (UID: \"31e6a111-bd4a-45a5-95a0-a6b537added6\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:05:25 crc kubenswrapper[5052]: I1203 12:05:25.247036 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/31e6a111-bd4a-45a5-95a0-a6b537added6-logs\") pod \"cinder-api-0\" (UID: \"31e6a111-bd4a-45a5-95a0-a6b537added6\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:05:25 crc kubenswrapper[5052]: I1203 12:05:25.247073 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/31e6a111-bd4a-45a5-95a0-a6b537added6-scripts\") pod \"cinder-api-0\" (UID: \"31e6a111-bd4a-45a5-95a0-a6b537added6\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:05:25 crc kubenswrapper[5052]: I1203 12:05:25.247130 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/31e6a111-bd4a-45a5-95a0-a6b537added6-config-data-custom\") pod \"cinder-api-0\" (UID: \"31e6a111-bd4a-45a5-95a0-a6b537added6\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:05:25 crc kubenswrapper[5052]: I1203 12:05:25.349296 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/31e6a111-bd4a-45a5-95a0-a6b537added6-public-tls-certs\") pod \"cinder-api-0\" (UID: \"31e6a111-bd4a-45a5-95a0-a6b537added6\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:05:25 crc kubenswrapper[5052]: I1203 12:05:25.349348 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/31e6a111-bd4a-45a5-95a0-a6b537added6-combined-ca-bundle\") pod \"cinder-api-0\" (UID: \"31e6a111-bd4a-45a5-95a0-a6b537added6\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:05:25 crc kubenswrapper[5052]: I1203 12:05:25.349379 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-7jrwp\" (UniqueName: \"kubernetes.io/projected/31e6a111-bd4a-45a5-95a0-a6b537added6-kube-api-access-7jrwp\") pod \"cinder-api-0\" (UID: \"31e6a111-bd4a-45a5-95a0-a6b537added6\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:05:25 crc kubenswrapper[5052]: I1203 12:05:25.349416 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/31e6a111-bd4a-45a5-95a0-a6b537added6-internal-tls-certs\") pod \"cinder-api-0\" (UID: \"31e6a111-bd4a-45a5-95a0-a6b537added6\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:05:25 crc kubenswrapper[5052]: I1203 12:05:25.349457 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/31e6a111-bd4a-45a5-95a0-a6b537added6-etc-machine-id\") pod \"cinder-api-0\" (UID: \"31e6a111-bd4a-45a5-95a0-a6b537added6\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:05:25 crc kubenswrapper[5052]: I1203 12:05:25.349491 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/31e6a111-bd4a-45a5-95a0-a6b537added6-config-data\") pod \"cinder-api-0\" (UID: \"31e6a111-bd4a-45a5-95a0-a6b537added6\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:05:25 crc kubenswrapper[5052]: I1203 12:05:25.349591 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/31e6a111-bd4a-45a5-95a0-a6b537added6-logs\") pod \"cinder-api-0\" (UID: \"31e6a111-bd4a-45a5-95a0-a6b537added6\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:05:25 crc kubenswrapper[5052]: I1203 12:05:25.349629 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/31e6a111-bd4a-45a5-95a0-a6b537added6-scripts\") pod \"cinder-api-0\" (UID: \"31e6a111-bd4a-45a5-95a0-a6b537added6\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:05:25 crc kubenswrapper[5052]: I1203 12:05:25.349683 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/31e6a111-bd4a-45a5-95a0-a6b537added6-config-data-custom\") pod \"cinder-api-0\" (UID: \"31e6a111-bd4a-45a5-95a0-a6b537added6\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:05:25 crc kubenswrapper[5052]: I1203 12:05:25.349902 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/31e6a111-bd4a-45a5-95a0-a6b537added6-etc-machine-id\") pod \"cinder-api-0\" (UID: \"31e6a111-bd4a-45a5-95a0-a6b537added6\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:05:25 crc kubenswrapper[5052]: I1203 12:05:25.350615 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/31e6a111-bd4a-45a5-95a0-a6b537added6-logs\") pod \"cinder-api-0\" (UID: \"31e6a111-bd4a-45a5-95a0-a6b537added6\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:05:25 crc kubenswrapper[5052]: I1203 12:05:25.353322 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/31e6a111-bd4a-45a5-95a0-a6b537added6-public-tls-certs\") pod \"cinder-api-0\" (UID: \"31e6a111-bd4a-45a5-95a0-a6b537added6\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:05:25 crc kubenswrapper[5052]: I1203 12:05:25.353341 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/31e6a111-bd4a-45a5-95a0-a6b537added6-combined-ca-bundle\") pod \"cinder-api-0\" (UID: \"31e6a111-bd4a-45a5-95a0-a6b537added6\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:05:25 crc kubenswrapper[5052]: I1203 12:05:25.354708 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/31e6a111-bd4a-45a5-95a0-a6b537added6-internal-tls-certs\") pod \"cinder-api-0\" (UID: \"31e6a111-bd4a-45a5-95a0-a6b537added6\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:05:25 crc kubenswrapper[5052]: I1203 12:05:25.355120 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/31e6a111-bd4a-45a5-95a0-a6b537added6-config-data-custom\") pod \"cinder-api-0\" (UID: \"31e6a111-bd4a-45a5-95a0-a6b537added6\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:05:25 crc kubenswrapper[5052]: I1203 12:05:25.355579 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/31e6a111-bd4a-45a5-95a0-a6b537added6-config-data\") pod \"cinder-api-0\" (UID: \"31e6a111-bd4a-45a5-95a0-a6b537added6\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:05:25 crc kubenswrapper[5052]: I1203 12:05:25.355907 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/31e6a111-bd4a-45a5-95a0-a6b537added6-scripts\") pod \"cinder-api-0\" (UID: \"31e6a111-bd4a-45a5-95a0-a6b537added6\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:05:25 crc kubenswrapper[5052]: I1203 12:05:25.366996 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-7jrwp\" (UniqueName: \"kubernetes.io/projected/31e6a111-bd4a-45a5-95a0-a6b537added6-kube-api-access-7jrwp\") pod \"cinder-api-0\" (UID: \"31e6a111-bd4a-45a5-95a0-a6b537added6\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:05:25 crc kubenswrapper[5052]: I1203 12:05:25.457184 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:05:25 crc kubenswrapper[5052]: I1203 12:05:25.694477 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ae94377d-8ac6-46e6-95db-703b9b487b4e" path="/var/lib/kubelet/pods/ae94377d-8ac6-46e6-95db-703b9b487b4e/volumes" Dec 03 12:05:25 crc kubenswrapper[5052]: I1203 12:05:25.909679 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 12:05:26 crc kubenswrapper[5052]: I1203 12:05:26.070293 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"31e6a111-bd4a-45a5-95a0-a6b537added6","Type":"ContainerStarted","Data":"f440aa9d51a5b63ef7463ff51ba3bb2689f7785c7cee41c23dfa059d063e983e"} Dec 03 12:05:27 crc kubenswrapper[5052]: I1203 12:05:27.085726 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"31e6a111-bd4a-45a5-95a0-a6b537added6","Type":"ContainerStarted","Data":"84283f0da62e2076f919bc27137ad503eb09cc0b4a9e5ff7119c2f488722a65c"} Dec 03 12:05:28 crc kubenswrapper[5052]: I1203 12:05:28.022822 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/barbican-api-5757755d9b-9m6dz" Dec 03 12:05:28 crc kubenswrapper[5052]: I1203 12:05:28.092022 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/barbican-api-5757755d9b-9m6dz" Dec 03 12:05:28 crc kubenswrapper[5052]: I1203 12:05:28.100782 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"31e6a111-bd4a-45a5-95a0-a6b537added6","Type":"ContainerStarted","Data":"545a199ec97461f4a64454b6edecdfc68e50742c7e11692685b952917d9c8b1e"} Dec 03 12:05:28 crc kubenswrapper[5052]: I1203 12:05:28.148690 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/cinder-api-0" podStartSLOduration=3.148667071 podStartE2EDuration="3.148667071s" podCreationTimestamp="2025-12-03 12:05:25 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:05:28.128657589 +0000 UTC m=+2718.716899363" watchObservedRunningTime="2025-12-03 12:05:28.148667071 +0000 UTC m=+2718.736908835" Dec 03 12:05:28 crc kubenswrapper[5052]: I1203 12:05:28.912361 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:05:28 crc kubenswrapper[5052]: I1203 12:05:28.992197 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 12:05:29 crc kubenswrapper[5052]: I1203 12:05:29.116948 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:05:29 crc kubenswrapper[5052]: I1203 12:05:29.117120 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/cinder-scheduler-0" podUID="664266a3-132d-4a06-b90b-7b58cde2a6be" containerName="cinder-scheduler" containerID="cri-o://93e6d44896e2e21b42567eecae483c6df093a022855acc48c6dd13c597bcc182" gracePeriod=30 Dec 03 12:05:29 crc kubenswrapper[5052]: I1203 12:05:29.117701 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/cinder-scheduler-0" podUID="664266a3-132d-4a06-b90b-7b58cde2a6be" containerName="probe" containerID="cri-o://a0e5ccb4f4d4f80e8ce1791623b231707bc05e15618e19e8167b50767d14ccfa" gracePeriod=30 Dec 03 12:05:29 crc kubenswrapper[5052]: I1203 12:05:29.379054 5052 patch_prober.go:28] interesting pod/machine-config-daemon-74bj6 container/machine-config-daemon namespace/openshift-machine-config-operator: Liveness probe status=failure output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" start-of-body= Dec 03 12:05:29 crc kubenswrapper[5052]: I1203 12:05:29.379120 5052 prober.go:107] "Probe failed" probeType="Liveness" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" containerName="machine-config-daemon" probeResult="failure" output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" Dec 03 12:05:30 crc kubenswrapper[5052]: I1203 12:05:30.129750 5052 generic.go:334] "Generic (PLEG): container finished" podID="664266a3-132d-4a06-b90b-7b58cde2a6be" containerID="a0e5ccb4f4d4f80e8ce1791623b231707bc05e15618e19e8167b50767d14ccfa" exitCode=0 Dec 03 12:05:30 crc kubenswrapper[5052]: I1203 12:05:30.129868 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"664266a3-132d-4a06-b90b-7b58cde2a6be","Type":"ContainerDied","Data":"a0e5ccb4f4d4f80e8ce1791623b231707bc05e15618e19e8167b50767d14ccfa"} Dec 03 12:05:32 crc kubenswrapper[5052]: I1203 12:05:32.956071 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:05:33 crc kubenswrapper[5052]: I1203 12:05:33.069907 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:05:33 crc kubenswrapper[5052]: I1203 12:05:33.163289 5052 generic.go:334] "Generic (PLEG): container finished" podID="664266a3-132d-4a06-b90b-7b58cde2a6be" containerID="93e6d44896e2e21b42567eecae483c6df093a022855acc48c6dd13c597bcc182" exitCode=0 Dec 03 12:05:33 crc kubenswrapper[5052]: I1203 12:05:33.163336 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"664266a3-132d-4a06-b90b-7b58cde2a6be","Type":"ContainerDied","Data":"93e6d44896e2e21b42567eecae483c6df093a022855acc48c6dd13c597bcc182"} Dec 03 12:05:33 crc kubenswrapper[5052]: I1203 12:05:33.163367 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"664266a3-132d-4a06-b90b-7b58cde2a6be","Type":"ContainerDied","Data":"a258633e83b65e858760863d0b411747005e97d7da0d42164260ca58f5956e8f"} Dec 03 12:05:33 crc kubenswrapper[5052]: I1203 12:05:33.163387 5052 scope.go:117] "RemoveContainer" containerID="a0e5ccb4f4d4f80e8ce1791623b231707bc05e15618e19e8167b50767d14ccfa" Dec 03 12:05:33 crc kubenswrapper[5052]: I1203 12:05:33.163593 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:05:33 crc kubenswrapper[5052]: I1203 12:05:33.183755 5052 scope.go:117] "RemoveContainer" containerID="93e6d44896e2e21b42567eecae483c6df093a022855acc48c6dd13c597bcc182" Dec 03 12:05:33 crc kubenswrapper[5052]: I1203 12:05:33.207901 5052 scope.go:117] "RemoveContainer" containerID="a0e5ccb4f4d4f80e8ce1791623b231707bc05e15618e19e8167b50767d14ccfa" Dec 03 12:05:33 crc kubenswrapper[5052]: E1203 12:05:33.208460 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a0e5ccb4f4d4f80e8ce1791623b231707bc05e15618e19e8167b50767d14ccfa\": container with ID starting with a0e5ccb4f4d4f80e8ce1791623b231707bc05e15618e19e8167b50767d14ccfa not found: ID does not exist" containerID="a0e5ccb4f4d4f80e8ce1791623b231707bc05e15618e19e8167b50767d14ccfa" Dec 03 12:05:33 crc kubenswrapper[5052]: I1203 12:05:33.208501 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a0e5ccb4f4d4f80e8ce1791623b231707bc05e15618e19e8167b50767d14ccfa"} err="failed to get container status \"a0e5ccb4f4d4f80e8ce1791623b231707bc05e15618e19e8167b50767d14ccfa\": rpc error: code = NotFound desc = could not find container \"a0e5ccb4f4d4f80e8ce1791623b231707bc05e15618e19e8167b50767d14ccfa\": container with ID starting with a0e5ccb4f4d4f80e8ce1791623b231707bc05e15618e19e8167b50767d14ccfa not found: ID does not exist" Dec 03 12:05:33 crc kubenswrapper[5052]: I1203 12:05:33.208542 5052 scope.go:117] "RemoveContainer" containerID="93e6d44896e2e21b42567eecae483c6df093a022855acc48c6dd13c597bcc182" Dec 03 12:05:33 crc kubenswrapper[5052]: E1203 12:05:33.209037 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"93e6d44896e2e21b42567eecae483c6df093a022855acc48c6dd13c597bcc182\": container with ID starting with 93e6d44896e2e21b42567eecae483c6df093a022855acc48c6dd13c597bcc182 not found: ID does not exist" containerID="93e6d44896e2e21b42567eecae483c6df093a022855acc48c6dd13c597bcc182" Dec 03 12:05:33 crc kubenswrapper[5052]: I1203 12:05:33.209067 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"93e6d44896e2e21b42567eecae483c6df093a022855acc48c6dd13c597bcc182"} err="failed to get container status \"93e6d44896e2e21b42567eecae483c6df093a022855acc48c6dd13c597bcc182\": rpc error: code = NotFound desc = could not find container \"93e6d44896e2e21b42567eecae483c6df093a022855acc48c6dd13c597bcc182\": container with ID starting with 93e6d44896e2e21b42567eecae483c6df093a022855acc48c6dd13c597bcc182 not found: ID does not exist" Dec 03 12:05:33 crc kubenswrapper[5052]: I1203 12:05:33.234107 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/664266a3-132d-4a06-b90b-7b58cde2a6be-combined-ca-bundle\") pod \"664266a3-132d-4a06-b90b-7b58cde2a6be\" (UID: \"664266a3-132d-4a06-b90b-7b58cde2a6be\") " Dec 03 12:05:33 crc kubenswrapper[5052]: I1203 12:05:33.234167 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/664266a3-132d-4a06-b90b-7b58cde2a6be-scripts\") pod \"664266a3-132d-4a06-b90b-7b58cde2a6be\" (UID: \"664266a3-132d-4a06-b90b-7b58cde2a6be\") " Dec 03 12:05:33 crc kubenswrapper[5052]: I1203 12:05:33.234238 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/664266a3-132d-4a06-b90b-7b58cde2a6be-config-data-custom\") pod \"664266a3-132d-4a06-b90b-7b58cde2a6be\" (UID: \"664266a3-132d-4a06-b90b-7b58cde2a6be\") " Dec 03 12:05:33 crc kubenswrapper[5052]: I1203 12:05:33.234276 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/664266a3-132d-4a06-b90b-7b58cde2a6be-config-data\") pod \"664266a3-132d-4a06-b90b-7b58cde2a6be\" (UID: \"664266a3-132d-4a06-b90b-7b58cde2a6be\") " Dec 03 12:05:33 crc kubenswrapper[5052]: I1203 12:05:33.234363 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/664266a3-132d-4a06-b90b-7b58cde2a6be-etc-machine-id\") pod \"664266a3-132d-4a06-b90b-7b58cde2a6be\" (UID: \"664266a3-132d-4a06-b90b-7b58cde2a6be\") " Dec 03 12:05:33 crc kubenswrapper[5052]: I1203 12:05:33.234387 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-shdfc\" (UniqueName: \"kubernetes.io/projected/664266a3-132d-4a06-b90b-7b58cde2a6be-kube-api-access-shdfc\") pod \"664266a3-132d-4a06-b90b-7b58cde2a6be\" (UID: \"664266a3-132d-4a06-b90b-7b58cde2a6be\") " Dec 03 12:05:33 crc kubenswrapper[5052]: I1203 12:05:33.234523 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/664266a3-132d-4a06-b90b-7b58cde2a6be-etc-machine-id" (OuterVolumeSpecName: "etc-machine-id") pod "664266a3-132d-4a06-b90b-7b58cde2a6be" (UID: "664266a3-132d-4a06-b90b-7b58cde2a6be"). InnerVolumeSpecName "etc-machine-id". PluginName "kubernetes.io/host-path", VolumeGidValue "" Dec 03 12:05:33 crc kubenswrapper[5052]: I1203 12:05:33.235000 5052 reconciler_common.go:293] "Volume detached for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/664266a3-132d-4a06-b90b-7b58cde2a6be-etc-machine-id\") on node \"crc\" DevicePath \"\"" Dec 03 12:05:33 crc kubenswrapper[5052]: I1203 12:05:33.243738 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/664266a3-132d-4a06-b90b-7b58cde2a6be-kube-api-access-shdfc" (OuterVolumeSpecName: "kube-api-access-shdfc") pod "664266a3-132d-4a06-b90b-7b58cde2a6be" (UID: "664266a3-132d-4a06-b90b-7b58cde2a6be"). InnerVolumeSpecName "kube-api-access-shdfc". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:05:33 crc kubenswrapper[5052]: I1203 12:05:33.243792 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/664266a3-132d-4a06-b90b-7b58cde2a6be-scripts" (OuterVolumeSpecName: "scripts") pod "664266a3-132d-4a06-b90b-7b58cde2a6be" (UID: "664266a3-132d-4a06-b90b-7b58cde2a6be"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:05:33 crc kubenswrapper[5052]: I1203 12:05:33.246052 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/664266a3-132d-4a06-b90b-7b58cde2a6be-config-data-custom" (OuterVolumeSpecName: "config-data-custom") pod "664266a3-132d-4a06-b90b-7b58cde2a6be" (UID: "664266a3-132d-4a06-b90b-7b58cde2a6be"). InnerVolumeSpecName "config-data-custom". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:05:33 crc kubenswrapper[5052]: I1203 12:05:33.299158 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/664266a3-132d-4a06-b90b-7b58cde2a6be-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "664266a3-132d-4a06-b90b-7b58cde2a6be" (UID: "664266a3-132d-4a06-b90b-7b58cde2a6be"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:05:33 crc kubenswrapper[5052]: I1203 12:05:33.337148 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/664266a3-132d-4a06-b90b-7b58cde2a6be-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:05:33 crc kubenswrapper[5052]: I1203 12:05:33.337185 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/664266a3-132d-4a06-b90b-7b58cde2a6be-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:05:33 crc kubenswrapper[5052]: I1203 12:05:33.337194 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/664266a3-132d-4a06-b90b-7b58cde2a6be-config-data-custom\") on node \"crc\" DevicePath \"\"" Dec 03 12:05:33 crc kubenswrapper[5052]: I1203 12:05:33.337203 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-shdfc\" (UniqueName: \"kubernetes.io/projected/664266a3-132d-4a06-b90b-7b58cde2a6be-kube-api-access-shdfc\") on node \"crc\" DevicePath \"\"" Dec 03 12:05:33 crc kubenswrapper[5052]: I1203 12:05:33.353502 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/664266a3-132d-4a06-b90b-7b58cde2a6be-config-data" (OuterVolumeSpecName: "config-data") pod "664266a3-132d-4a06-b90b-7b58cde2a6be" (UID: "664266a3-132d-4a06-b90b-7b58cde2a6be"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:05:33 crc kubenswrapper[5052]: I1203 12:05:33.439067 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/664266a3-132d-4a06-b90b-7b58cde2a6be-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:05:33 crc kubenswrapper[5052]: I1203 12:05:33.501712 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 12:05:33 crc kubenswrapper[5052]: I1203 12:05:33.514189 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 12:05:33 crc kubenswrapper[5052]: I1203 12:05:33.524388 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 12:05:33 crc kubenswrapper[5052]: E1203 12:05:33.524927 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="664266a3-132d-4a06-b90b-7b58cde2a6be" containerName="cinder-scheduler" Dec 03 12:05:33 crc kubenswrapper[5052]: I1203 12:05:33.524953 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="664266a3-132d-4a06-b90b-7b58cde2a6be" containerName="cinder-scheduler" Dec 03 12:05:33 crc kubenswrapper[5052]: E1203 12:05:33.524978 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="664266a3-132d-4a06-b90b-7b58cde2a6be" containerName="probe" Dec 03 12:05:33 crc kubenswrapper[5052]: I1203 12:05:33.524985 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="664266a3-132d-4a06-b90b-7b58cde2a6be" containerName="probe" Dec 03 12:05:33 crc kubenswrapper[5052]: I1203 12:05:33.525171 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="664266a3-132d-4a06-b90b-7b58cde2a6be" containerName="probe" Dec 03 12:05:33 crc kubenswrapper[5052]: I1203 12:05:33.525205 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="664266a3-132d-4a06-b90b-7b58cde2a6be" containerName="cinder-scheduler" Dec 03 12:05:33 crc kubenswrapper[5052]: I1203 12:05:33.526344 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:05:33 crc kubenswrapper[5052]: I1203 12:05:33.528888 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cinder-scheduler-config-data" Dec 03 12:05:33 crc kubenswrapper[5052]: I1203 12:05:33.543669 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 12:05:33 crc kubenswrapper[5052]: I1203 12:05:33.643002 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/60f72892-a5aa-482f-aac3-5713ba028bc8-combined-ca-bundle\") pod \"cinder-scheduler-0\" (UID: \"60f72892-a5aa-482f-aac3-5713ba028bc8\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:05:33 crc kubenswrapper[5052]: I1203 12:05:33.643370 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/60f72892-a5aa-482f-aac3-5713ba028bc8-scripts\") pod \"cinder-scheduler-0\" (UID: \"60f72892-a5aa-482f-aac3-5713ba028bc8\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:05:33 crc kubenswrapper[5052]: I1203 12:05:33.643460 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/60f72892-a5aa-482f-aac3-5713ba028bc8-config-data-custom\") pod \"cinder-scheduler-0\" (UID: \"60f72892-a5aa-482f-aac3-5713ba028bc8\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:05:33 crc kubenswrapper[5052]: I1203 12:05:33.643625 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/60f72892-a5aa-482f-aac3-5713ba028bc8-config-data\") pod \"cinder-scheduler-0\" (UID: \"60f72892-a5aa-482f-aac3-5713ba028bc8\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:05:33 crc kubenswrapper[5052]: I1203 12:05:33.643798 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/60f72892-a5aa-482f-aac3-5713ba028bc8-etc-machine-id\") pod \"cinder-scheduler-0\" (UID: \"60f72892-a5aa-482f-aac3-5713ba028bc8\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:05:33 crc kubenswrapper[5052]: I1203 12:05:33.643845 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hdc52\" (UniqueName: \"kubernetes.io/projected/60f72892-a5aa-482f-aac3-5713ba028bc8-kube-api-access-hdc52\") pod \"cinder-scheduler-0\" (UID: \"60f72892-a5aa-482f-aac3-5713ba028bc8\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:05:33 crc kubenswrapper[5052]: I1203 12:05:33.683810 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="664266a3-132d-4a06-b90b-7b58cde2a6be" path="/var/lib/kubelet/pods/664266a3-132d-4a06-b90b-7b58cde2a6be/volumes" Dec 03 12:05:33 crc kubenswrapper[5052]: I1203 12:05:33.745605 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/60f72892-a5aa-482f-aac3-5713ba028bc8-combined-ca-bundle\") pod \"cinder-scheduler-0\" (UID: \"60f72892-a5aa-482f-aac3-5713ba028bc8\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:05:33 crc kubenswrapper[5052]: I1203 12:05:33.745728 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/60f72892-a5aa-482f-aac3-5713ba028bc8-scripts\") pod \"cinder-scheduler-0\" (UID: \"60f72892-a5aa-482f-aac3-5713ba028bc8\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:05:33 crc kubenswrapper[5052]: I1203 12:05:33.745770 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/60f72892-a5aa-482f-aac3-5713ba028bc8-config-data-custom\") pod \"cinder-scheduler-0\" (UID: \"60f72892-a5aa-482f-aac3-5713ba028bc8\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:05:33 crc kubenswrapper[5052]: I1203 12:05:33.745848 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/60f72892-a5aa-482f-aac3-5713ba028bc8-config-data\") pod \"cinder-scheduler-0\" (UID: \"60f72892-a5aa-482f-aac3-5713ba028bc8\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:05:33 crc kubenswrapper[5052]: I1203 12:05:33.745893 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/60f72892-a5aa-482f-aac3-5713ba028bc8-etc-machine-id\") pod \"cinder-scheduler-0\" (UID: \"60f72892-a5aa-482f-aac3-5713ba028bc8\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:05:33 crc kubenswrapper[5052]: I1203 12:05:33.745916 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-hdc52\" (UniqueName: \"kubernetes.io/projected/60f72892-a5aa-482f-aac3-5713ba028bc8-kube-api-access-hdc52\") pod \"cinder-scheduler-0\" (UID: \"60f72892-a5aa-482f-aac3-5713ba028bc8\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:05:33 crc kubenswrapper[5052]: I1203 12:05:33.746045 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/60f72892-a5aa-482f-aac3-5713ba028bc8-etc-machine-id\") pod \"cinder-scheduler-0\" (UID: \"60f72892-a5aa-482f-aac3-5713ba028bc8\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:05:33 crc kubenswrapper[5052]: I1203 12:05:33.749761 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/60f72892-a5aa-482f-aac3-5713ba028bc8-combined-ca-bundle\") pod \"cinder-scheduler-0\" (UID: \"60f72892-a5aa-482f-aac3-5713ba028bc8\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:05:33 crc kubenswrapper[5052]: I1203 12:05:33.749771 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/60f72892-a5aa-482f-aac3-5713ba028bc8-config-data-custom\") pod \"cinder-scheduler-0\" (UID: \"60f72892-a5aa-482f-aac3-5713ba028bc8\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:05:33 crc kubenswrapper[5052]: I1203 12:05:33.750406 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/60f72892-a5aa-482f-aac3-5713ba028bc8-config-data\") pod \"cinder-scheduler-0\" (UID: \"60f72892-a5aa-482f-aac3-5713ba028bc8\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:05:33 crc kubenswrapper[5052]: I1203 12:05:33.756465 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/60f72892-a5aa-482f-aac3-5713ba028bc8-scripts\") pod \"cinder-scheduler-0\" (UID: \"60f72892-a5aa-482f-aac3-5713ba028bc8\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:05:33 crc kubenswrapper[5052]: I1203 12:05:33.782041 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-hdc52\" (UniqueName: \"kubernetes.io/projected/60f72892-a5aa-482f-aac3-5713ba028bc8-kube-api-access-hdc52\") pod \"cinder-scheduler-0\" (UID: \"60f72892-a5aa-482f-aac3-5713ba028bc8\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:05:33 crc kubenswrapper[5052]: I1203 12:05:33.865380 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:05:34 crc kubenswrapper[5052]: I1203 12:05:34.331038 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 12:05:34 crc kubenswrapper[5052]: I1203 12:05:34.861627 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/barbican-api-d978c699b-kzzst" Dec 03 12:05:34 crc kubenswrapper[5052]: I1203 12:05:34.871855 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/barbican-api-d978c699b-kzzst" Dec 03 12:05:34 crc kubenswrapper[5052]: I1203 12:05:34.989726 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-api-5757755d9b-9m6dz"] Dec 03 12:05:34 crc kubenswrapper[5052]: I1203 12:05:34.990184 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/barbican-api-5757755d9b-9m6dz" podUID="44965796-ddc0-4cb1-bdf6-f107b9c3373b" containerName="barbican-api-log" containerID="cri-o://2fea933a02532e081e8c1b23f22d0f21410c4756a8d56614e399f1fc7daadcfd" gracePeriod=30 Dec 03 12:05:34 crc kubenswrapper[5052]: I1203 12:05:34.990645 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/barbican-api-5757755d9b-9m6dz" podUID="44965796-ddc0-4cb1-bdf6-f107b9c3373b" containerName="barbican-api" containerID="cri-o://8bc3b52ef1073998ba1546f957d22fef73c229a57932d8bd6026308d8625f311" gracePeriod=30 Dec 03 12:05:35 crc kubenswrapper[5052]: I1203 12:05:35.214094 5052 generic.go:334] "Generic (PLEG): container finished" podID="44965796-ddc0-4cb1-bdf6-f107b9c3373b" containerID="2fea933a02532e081e8c1b23f22d0f21410c4756a8d56614e399f1fc7daadcfd" exitCode=143 Dec 03 12:05:35 crc kubenswrapper[5052]: I1203 12:05:35.214168 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-5757755d9b-9m6dz" event={"ID":"44965796-ddc0-4cb1-bdf6-f107b9c3373b","Type":"ContainerDied","Data":"2fea933a02532e081e8c1b23f22d0f21410c4756a8d56614e399f1fc7daadcfd"} Dec 03 12:05:35 crc kubenswrapper[5052]: I1203 12:05:35.215742 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"60f72892-a5aa-482f-aac3-5713ba028bc8","Type":"ContainerStarted","Data":"082794493f1a866da7efb8038f854ca6442bb335e30fc8e05b37438e570e2b64"} Dec 03 12:05:35 crc kubenswrapper[5052]: I1203 12:05:35.215793 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"60f72892-a5aa-482f-aac3-5713ba028bc8","Type":"ContainerStarted","Data":"c31a05c23052597025fe3e6544d6ed87146c1a6be0e31012fc69ff2a6f8fa306"} Dec 03 12:05:36 crc kubenswrapper[5052]: I1203 12:05:36.229135 5052 generic.go:334] "Generic (PLEG): container finished" podID="d9b7a317-ab7f-4c6a-8f91-b66b409c8980" containerID="f25bc6d11555c8eec8ce818399df69255f9b136b8146d57ac9a33a070d4dc79b" exitCode=0 Dec 03 12:05:36 crc kubenswrapper[5052]: I1203 12:05:36.229575 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-db-sync-lrrtz" event={"ID":"d9b7a317-ab7f-4c6a-8f91-b66b409c8980","Type":"ContainerDied","Data":"f25bc6d11555c8eec8ce818399df69255f9b136b8146d57ac9a33a070d4dc79b"} Dec 03 12:05:36 crc kubenswrapper[5052]: I1203 12:05:36.240036 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"60f72892-a5aa-482f-aac3-5713ba028bc8","Type":"ContainerStarted","Data":"6e5a8a55dfb233734ce359d756ebf08a407cc39b08dc02754ac067f403d0a466"} Dec 03 12:05:36 crc kubenswrapper[5052]: I1203 12:05:36.269776 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/cinder-scheduler-0" podStartSLOduration=3.269755918 podStartE2EDuration="3.269755918s" podCreationTimestamp="2025-12-03 12:05:33 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:05:36.263368215 +0000 UTC m=+2726.851609979" watchObservedRunningTime="2025-12-03 12:05:36.269755918 +0000 UTC m=+2726.857997682" Dec 03 12:05:37 crc kubenswrapper[5052]: I1203 12:05:37.503284 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:05:37 crc kubenswrapper[5052]: I1203 12:05:37.621052 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-db-sync-lrrtz" Dec 03 12:05:37 crc kubenswrapper[5052]: I1203 12:05:37.652432 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d9b7a317-ab7f-4c6a-8f91-b66b409c8980-combined-ca-bundle\") pod \"d9b7a317-ab7f-4c6a-8f91-b66b409c8980\" (UID: \"d9b7a317-ab7f-4c6a-8f91-b66b409c8980\") " Dec 03 12:05:37 crc kubenswrapper[5052]: I1203 12:05:37.652647 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/d9b7a317-ab7f-4c6a-8f91-b66b409c8980-config\") pod \"d9b7a317-ab7f-4c6a-8f91-b66b409c8980\" (UID: \"d9b7a317-ab7f-4c6a-8f91-b66b409c8980\") " Dec 03 12:05:37 crc kubenswrapper[5052]: I1203 12:05:37.652710 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r5cw2\" (UniqueName: \"kubernetes.io/projected/d9b7a317-ab7f-4c6a-8f91-b66b409c8980-kube-api-access-r5cw2\") pod \"d9b7a317-ab7f-4c6a-8f91-b66b409c8980\" (UID: \"d9b7a317-ab7f-4c6a-8f91-b66b409c8980\") " Dec 03 12:05:37 crc kubenswrapper[5052]: I1203 12:05:37.664752 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d9b7a317-ab7f-4c6a-8f91-b66b409c8980-kube-api-access-r5cw2" (OuterVolumeSpecName: "kube-api-access-r5cw2") pod "d9b7a317-ab7f-4c6a-8f91-b66b409c8980" (UID: "d9b7a317-ab7f-4c6a-8f91-b66b409c8980"). InnerVolumeSpecName "kube-api-access-r5cw2". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:05:37 crc kubenswrapper[5052]: I1203 12:05:37.685752 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d9b7a317-ab7f-4c6a-8f91-b66b409c8980-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "d9b7a317-ab7f-4c6a-8f91-b66b409c8980" (UID: "d9b7a317-ab7f-4c6a-8f91-b66b409c8980"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:05:37 crc kubenswrapper[5052]: I1203 12:05:37.698058 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d9b7a317-ab7f-4c6a-8f91-b66b409c8980-config" (OuterVolumeSpecName: "config") pod "d9b7a317-ab7f-4c6a-8f91-b66b409c8980" (UID: "d9b7a317-ab7f-4c6a-8f91-b66b409c8980"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:05:37 crc kubenswrapper[5052]: I1203 12:05:37.754625 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d9b7a317-ab7f-4c6a-8f91-b66b409c8980-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:05:37 crc kubenswrapper[5052]: I1203 12:05:37.754661 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/d9b7a317-ab7f-4c6a-8f91-b66b409c8980-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:05:37 crc kubenswrapper[5052]: I1203 12:05:37.754672 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-r5cw2\" (UniqueName: \"kubernetes.io/projected/d9b7a317-ab7f-4c6a-8f91-b66b409c8980-kube-api-access-r5cw2\") on node \"crc\" DevicePath \"\"" Dec 03 12:05:38 crc kubenswrapper[5052]: I1203 12:05:38.176271 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/barbican-api-5757755d9b-9m6dz" podUID="44965796-ddc0-4cb1-bdf6-f107b9c3373b" containerName="barbican-api-log" probeResult="failure" output="Get \"http://10.217.0.152:9311/healthcheck\": read tcp 10.217.0.2:39816->10.217.0.152:9311: read: connection reset by peer" Dec 03 12:05:38 crc kubenswrapper[5052]: I1203 12:05:38.176319 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/barbican-api-5757755d9b-9m6dz" podUID="44965796-ddc0-4cb1-bdf6-f107b9c3373b" containerName="barbican-api" probeResult="failure" output="Get \"http://10.217.0.152:9311/healthcheck\": read tcp 10.217.0.2:39802->10.217.0.152:9311: read: connection reset by peer" Dec 03 12:05:38 crc kubenswrapper[5052]: I1203 12:05:38.263836 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-db-sync-lrrtz" event={"ID":"d9b7a317-ab7f-4c6a-8f91-b66b409c8980","Type":"ContainerDied","Data":"e21b878ee5af8a8af6164951d0105fce83c583505e0d5f994bcab1d22eecf09c"} Dec 03 12:05:38 crc kubenswrapper[5052]: I1203 12:05:38.263877 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e21b878ee5af8a8af6164951d0105fce83c583505e0d5f994bcab1d22eecf09c" Dec 03 12:05:38 crc kubenswrapper[5052]: I1203 12:05:38.263931 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-db-sync-lrrtz" Dec 03 12:05:38 crc kubenswrapper[5052]: I1203 12:05:38.476828 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/neutron-78f8cc6c7-cc8t6"] Dec 03 12:05:38 crc kubenswrapper[5052]: E1203 12:05:38.477727 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="d9b7a317-ab7f-4c6a-8f91-b66b409c8980" containerName="neutron-db-sync" Dec 03 12:05:38 crc kubenswrapper[5052]: I1203 12:05:38.477744 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="d9b7a317-ab7f-4c6a-8f91-b66b409c8980" containerName="neutron-db-sync" Dec 03 12:05:38 crc kubenswrapper[5052]: I1203 12:05:38.477992 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="d9b7a317-ab7f-4c6a-8f91-b66b409c8980" containerName="neutron-db-sync" Dec 03 12:05:38 crc kubenswrapper[5052]: I1203 12:05:38.479263 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-78f8cc6c7-cc8t6" Dec 03 12:05:38 crc kubenswrapper[5052]: I1203 12:05:38.491651 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"neutron-config" Dec 03 12:05:38 crc kubenswrapper[5052]: I1203 12:05:38.493996 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"neutron-neutron-dockercfg-6l9fz" Dec 03 12:05:38 crc kubenswrapper[5052]: I1203 12:05:38.493996 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"neutron-httpd-config" Dec 03 12:05:38 crc kubenswrapper[5052]: I1203 12:05:38.504019 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-78f8cc6c7-cc8t6"] Dec 03 12:05:38 crc kubenswrapper[5052]: I1203 12:05:38.525171 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-neutron-ovndbs" Dec 03 12:05:38 crc kubenswrapper[5052]: I1203 12:05:38.669779 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/c3489d6f-c371-4252-9a13-a055a4a55cbf-config\") pod \"neutron-78f8cc6c7-cc8t6\" (UID: \"c3489d6f-c371-4252-9a13-a055a4a55cbf\") " pod="openstack-kuttl-tests/neutron-78f8cc6c7-cc8t6" Dec 03 12:05:38 crc kubenswrapper[5052]: I1203 12:05:38.669884 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/c3489d6f-c371-4252-9a13-a055a4a55cbf-httpd-config\") pod \"neutron-78f8cc6c7-cc8t6\" (UID: \"c3489d6f-c371-4252-9a13-a055a4a55cbf\") " pod="openstack-kuttl-tests/neutron-78f8cc6c7-cc8t6" Dec 03 12:05:38 crc kubenswrapper[5052]: I1203 12:05:38.670018 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/c3489d6f-c371-4252-9a13-a055a4a55cbf-ovndb-tls-certs\") pod \"neutron-78f8cc6c7-cc8t6\" (UID: \"c3489d6f-c371-4252-9a13-a055a4a55cbf\") " pod="openstack-kuttl-tests/neutron-78f8cc6c7-cc8t6" Dec 03 12:05:38 crc kubenswrapper[5052]: I1203 12:05:38.670151 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v98bp\" (UniqueName: \"kubernetes.io/projected/c3489d6f-c371-4252-9a13-a055a4a55cbf-kube-api-access-v98bp\") pod \"neutron-78f8cc6c7-cc8t6\" (UID: \"c3489d6f-c371-4252-9a13-a055a4a55cbf\") " pod="openstack-kuttl-tests/neutron-78f8cc6c7-cc8t6" Dec 03 12:05:38 crc kubenswrapper[5052]: I1203 12:05:38.670192 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c3489d6f-c371-4252-9a13-a055a4a55cbf-combined-ca-bundle\") pod \"neutron-78f8cc6c7-cc8t6\" (UID: \"c3489d6f-c371-4252-9a13-a055a4a55cbf\") " pod="openstack-kuttl-tests/neutron-78f8cc6c7-cc8t6" Dec 03 12:05:38 crc kubenswrapper[5052]: I1203 12:05:38.772487 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/c3489d6f-c371-4252-9a13-a055a4a55cbf-config\") pod \"neutron-78f8cc6c7-cc8t6\" (UID: \"c3489d6f-c371-4252-9a13-a055a4a55cbf\") " pod="openstack-kuttl-tests/neutron-78f8cc6c7-cc8t6" Dec 03 12:05:38 crc kubenswrapper[5052]: I1203 12:05:38.772639 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/c3489d6f-c371-4252-9a13-a055a4a55cbf-httpd-config\") pod \"neutron-78f8cc6c7-cc8t6\" (UID: \"c3489d6f-c371-4252-9a13-a055a4a55cbf\") " pod="openstack-kuttl-tests/neutron-78f8cc6c7-cc8t6" Dec 03 12:05:38 crc kubenswrapper[5052]: I1203 12:05:38.772710 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/c3489d6f-c371-4252-9a13-a055a4a55cbf-ovndb-tls-certs\") pod \"neutron-78f8cc6c7-cc8t6\" (UID: \"c3489d6f-c371-4252-9a13-a055a4a55cbf\") " pod="openstack-kuttl-tests/neutron-78f8cc6c7-cc8t6" Dec 03 12:05:38 crc kubenswrapper[5052]: I1203 12:05:38.772762 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-v98bp\" (UniqueName: \"kubernetes.io/projected/c3489d6f-c371-4252-9a13-a055a4a55cbf-kube-api-access-v98bp\") pod \"neutron-78f8cc6c7-cc8t6\" (UID: \"c3489d6f-c371-4252-9a13-a055a4a55cbf\") " pod="openstack-kuttl-tests/neutron-78f8cc6c7-cc8t6" Dec 03 12:05:38 crc kubenswrapper[5052]: I1203 12:05:38.772816 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c3489d6f-c371-4252-9a13-a055a4a55cbf-combined-ca-bundle\") pod \"neutron-78f8cc6c7-cc8t6\" (UID: \"c3489d6f-c371-4252-9a13-a055a4a55cbf\") " pod="openstack-kuttl-tests/neutron-78f8cc6c7-cc8t6" Dec 03 12:05:38 crc kubenswrapper[5052]: I1203 12:05:38.778726 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/c3489d6f-c371-4252-9a13-a055a4a55cbf-ovndb-tls-certs\") pod \"neutron-78f8cc6c7-cc8t6\" (UID: \"c3489d6f-c371-4252-9a13-a055a4a55cbf\") " pod="openstack-kuttl-tests/neutron-78f8cc6c7-cc8t6" Dec 03 12:05:38 crc kubenswrapper[5052]: I1203 12:05:38.779222 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/c3489d6f-c371-4252-9a13-a055a4a55cbf-httpd-config\") pod \"neutron-78f8cc6c7-cc8t6\" (UID: \"c3489d6f-c371-4252-9a13-a055a4a55cbf\") " pod="openstack-kuttl-tests/neutron-78f8cc6c7-cc8t6" Dec 03 12:05:38 crc kubenswrapper[5052]: I1203 12:05:38.782093 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/c3489d6f-c371-4252-9a13-a055a4a55cbf-config\") pod \"neutron-78f8cc6c7-cc8t6\" (UID: \"c3489d6f-c371-4252-9a13-a055a4a55cbf\") " pod="openstack-kuttl-tests/neutron-78f8cc6c7-cc8t6" Dec 03 12:05:38 crc kubenswrapper[5052]: I1203 12:05:38.785275 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c3489d6f-c371-4252-9a13-a055a4a55cbf-combined-ca-bundle\") pod \"neutron-78f8cc6c7-cc8t6\" (UID: \"c3489d6f-c371-4252-9a13-a055a4a55cbf\") " pod="openstack-kuttl-tests/neutron-78f8cc6c7-cc8t6" Dec 03 12:05:38 crc kubenswrapper[5052]: I1203 12:05:38.794099 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-v98bp\" (UniqueName: \"kubernetes.io/projected/c3489d6f-c371-4252-9a13-a055a4a55cbf-kube-api-access-v98bp\") pod \"neutron-78f8cc6c7-cc8t6\" (UID: \"c3489d6f-c371-4252-9a13-a055a4a55cbf\") " pod="openstack-kuttl-tests/neutron-78f8cc6c7-cc8t6" Dec 03 12:05:38 crc kubenswrapper[5052]: I1203 12:05:38.815101 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-78f8cc6c7-cc8t6" Dec 03 12:05:38 crc kubenswrapper[5052]: I1203 12:05:38.867735 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:05:39 crc kubenswrapper[5052]: I1203 12:05:39.117276 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-api-5757755d9b-9m6dz" Dec 03 12:05:39 crc kubenswrapper[5052]: I1203 12:05:39.278712 5052 generic.go:334] "Generic (PLEG): container finished" podID="44965796-ddc0-4cb1-bdf6-f107b9c3373b" containerID="8bc3b52ef1073998ba1546f957d22fef73c229a57932d8bd6026308d8625f311" exitCode=0 Dec 03 12:05:39 crc kubenswrapper[5052]: I1203 12:05:39.278763 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-5757755d9b-9m6dz" event={"ID":"44965796-ddc0-4cb1-bdf6-f107b9c3373b","Type":"ContainerDied","Data":"8bc3b52ef1073998ba1546f957d22fef73c229a57932d8bd6026308d8625f311"} Dec 03 12:05:39 crc kubenswrapper[5052]: I1203 12:05:39.278803 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-api-5757755d9b-9m6dz" Dec 03 12:05:39 crc kubenswrapper[5052]: I1203 12:05:39.278827 5052 scope.go:117] "RemoveContainer" containerID="8bc3b52ef1073998ba1546f957d22fef73c229a57932d8bd6026308d8625f311" Dec 03 12:05:39 crc kubenswrapper[5052]: I1203 12:05:39.278812 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-5757755d9b-9m6dz" event={"ID":"44965796-ddc0-4cb1-bdf6-f107b9c3373b","Type":"ContainerDied","Data":"ef35e9f5b963c8b94868cf33871c4c56c76bcaf97c98b3bb9ba04b62c23d6d34"} Dec 03 12:05:39 crc kubenswrapper[5052]: I1203 12:05:39.283201 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/44965796-ddc0-4cb1-bdf6-f107b9c3373b-config-data\") pod \"44965796-ddc0-4cb1-bdf6-f107b9c3373b\" (UID: \"44965796-ddc0-4cb1-bdf6-f107b9c3373b\") " Dec 03 12:05:39 crc kubenswrapper[5052]: I1203 12:05:39.283274 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kx6k2\" (UniqueName: \"kubernetes.io/projected/44965796-ddc0-4cb1-bdf6-f107b9c3373b-kube-api-access-kx6k2\") pod \"44965796-ddc0-4cb1-bdf6-f107b9c3373b\" (UID: \"44965796-ddc0-4cb1-bdf6-f107b9c3373b\") " Dec 03 12:05:39 crc kubenswrapper[5052]: I1203 12:05:39.283310 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/44965796-ddc0-4cb1-bdf6-f107b9c3373b-config-data-custom\") pod \"44965796-ddc0-4cb1-bdf6-f107b9c3373b\" (UID: \"44965796-ddc0-4cb1-bdf6-f107b9c3373b\") " Dec 03 12:05:39 crc kubenswrapper[5052]: I1203 12:05:39.283444 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/44965796-ddc0-4cb1-bdf6-f107b9c3373b-combined-ca-bundle\") pod \"44965796-ddc0-4cb1-bdf6-f107b9c3373b\" (UID: \"44965796-ddc0-4cb1-bdf6-f107b9c3373b\") " Dec 03 12:05:39 crc kubenswrapper[5052]: I1203 12:05:39.283472 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/44965796-ddc0-4cb1-bdf6-f107b9c3373b-logs\") pod \"44965796-ddc0-4cb1-bdf6-f107b9c3373b\" (UID: \"44965796-ddc0-4cb1-bdf6-f107b9c3373b\") " Dec 03 12:05:39 crc kubenswrapper[5052]: I1203 12:05:39.284461 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/44965796-ddc0-4cb1-bdf6-f107b9c3373b-logs" (OuterVolumeSpecName: "logs") pod "44965796-ddc0-4cb1-bdf6-f107b9c3373b" (UID: "44965796-ddc0-4cb1-bdf6-f107b9c3373b"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:05:39 crc kubenswrapper[5052]: I1203 12:05:39.289374 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/44965796-ddc0-4cb1-bdf6-f107b9c3373b-kube-api-access-kx6k2" (OuterVolumeSpecName: "kube-api-access-kx6k2") pod "44965796-ddc0-4cb1-bdf6-f107b9c3373b" (UID: "44965796-ddc0-4cb1-bdf6-f107b9c3373b"). InnerVolumeSpecName "kube-api-access-kx6k2". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:05:39 crc kubenswrapper[5052]: I1203 12:05:39.293585 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/44965796-ddc0-4cb1-bdf6-f107b9c3373b-config-data-custom" (OuterVolumeSpecName: "config-data-custom") pod "44965796-ddc0-4cb1-bdf6-f107b9c3373b" (UID: "44965796-ddc0-4cb1-bdf6-f107b9c3373b"). InnerVolumeSpecName "config-data-custom". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:05:39 crc kubenswrapper[5052]: I1203 12:05:39.312004 5052 scope.go:117] "RemoveContainer" containerID="2fea933a02532e081e8c1b23f22d0f21410c4756a8d56614e399f1fc7daadcfd" Dec 03 12:05:39 crc kubenswrapper[5052]: I1203 12:05:39.316273 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/44965796-ddc0-4cb1-bdf6-f107b9c3373b-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "44965796-ddc0-4cb1-bdf6-f107b9c3373b" (UID: "44965796-ddc0-4cb1-bdf6-f107b9c3373b"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:05:39 crc kubenswrapper[5052]: I1203 12:05:39.333727 5052 scope.go:117] "RemoveContainer" containerID="8bc3b52ef1073998ba1546f957d22fef73c229a57932d8bd6026308d8625f311" Dec 03 12:05:39 crc kubenswrapper[5052]: E1203 12:05:39.334362 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8bc3b52ef1073998ba1546f957d22fef73c229a57932d8bd6026308d8625f311\": container with ID starting with 8bc3b52ef1073998ba1546f957d22fef73c229a57932d8bd6026308d8625f311 not found: ID does not exist" containerID="8bc3b52ef1073998ba1546f957d22fef73c229a57932d8bd6026308d8625f311" Dec 03 12:05:39 crc kubenswrapper[5052]: I1203 12:05:39.334414 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8bc3b52ef1073998ba1546f957d22fef73c229a57932d8bd6026308d8625f311"} err="failed to get container status \"8bc3b52ef1073998ba1546f957d22fef73c229a57932d8bd6026308d8625f311\": rpc error: code = NotFound desc = could not find container \"8bc3b52ef1073998ba1546f957d22fef73c229a57932d8bd6026308d8625f311\": container with ID starting with 8bc3b52ef1073998ba1546f957d22fef73c229a57932d8bd6026308d8625f311 not found: ID does not exist" Dec 03 12:05:39 crc kubenswrapper[5052]: I1203 12:05:39.334449 5052 scope.go:117] "RemoveContainer" containerID="2fea933a02532e081e8c1b23f22d0f21410c4756a8d56614e399f1fc7daadcfd" Dec 03 12:05:39 crc kubenswrapper[5052]: E1203 12:05:39.335327 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2fea933a02532e081e8c1b23f22d0f21410c4756a8d56614e399f1fc7daadcfd\": container with ID starting with 2fea933a02532e081e8c1b23f22d0f21410c4756a8d56614e399f1fc7daadcfd not found: ID does not exist" containerID="2fea933a02532e081e8c1b23f22d0f21410c4756a8d56614e399f1fc7daadcfd" Dec 03 12:05:39 crc kubenswrapper[5052]: I1203 12:05:39.335380 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2fea933a02532e081e8c1b23f22d0f21410c4756a8d56614e399f1fc7daadcfd"} err="failed to get container status \"2fea933a02532e081e8c1b23f22d0f21410c4756a8d56614e399f1fc7daadcfd\": rpc error: code = NotFound desc = could not find container \"2fea933a02532e081e8c1b23f22d0f21410c4756a8d56614e399f1fc7daadcfd\": container with ID starting with 2fea933a02532e081e8c1b23f22d0f21410c4756a8d56614e399f1fc7daadcfd not found: ID does not exist" Dec 03 12:05:39 crc kubenswrapper[5052]: I1203 12:05:39.354427 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/44965796-ddc0-4cb1-bdf6-f107b9c3373b-config-data" (OuterVolumeSpecName: "config-data") pod "44965796-ddc0-4cb1-bdf6-f107b9c3373b" (UID: "44965796-ddc0-4cb1-bdf6-f107b9c3373b"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:05:39 crc kubenswrapper[5052]: I1203 12:05:39.365972 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-78f8cc6c7-cc8t6"] Dec 03 12:05:39 crc kubenswrapper[5052]: W1203 12:05:39.371556 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc3489d6f_c371_4252_9a13_a055a4a55cbf.slice/crio-173db296bf1158238a3f169e35341dfbae99469e380d1f026a323d4ad8cf09d2 WatchSource:0}: Error finding container 173db296bf1158238a3f169e35341dfbae99469e380d1f026a323d4ad8cf09d2: Status 404 returned error can't find the container with id 173db296bf1158238a3f169e35341dfbae99469e380d1f026a323d4ad8cf09d2 Dec 03 12:05:39 crc kubenswrapper[5052]: I1203 12:05:39.385423 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/44965796-ddc0-4cb1-bdf6-f107b9c3373b-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:05:39 crc kubenswrapper[5052]: I1203 12:05:39.385454 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-kx6k2\" (UniqueName: \"kubernetes.io/projected/44965796-ddc0-4cb1-bdf6-f107b9c3373b-kube-api-access-kx6k2\") on node \"crc\" DevicePath \"\"" Dec 03 12:05:39 crc kubenswrapper[5052]: I1203 12:05:39.385465 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/44965796-ddc0-4cb1-bdf6-f107b9c3373b-config-data-custom\") on node \"crc\" DevicePath \"\"" Dec 03 12:05:39 crc kubenswrapper[5052]: I1203 12:05:39.385475 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/44965796-ddc0-4cb1-bdf6-f107b9c3373b-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:05:39 crc kubenswrapper[5052]: I1203 12:05:39.385485 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/44965796-ddc0-4cb1-bdf6-f107b9c3373b-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:05:39 crc kubenswrapper[5052]: I1203 12:05:39.616627 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-api-5757755d9b-9m6dz"] Dec 03 12:05:39 crc kubenswrapper[5052]: I1203 12:05:39.627691 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/barbican-api-5757755d9b-9m6dz"] Dec 03 12:05:39 crc kubenswrapper[5052]: I1203 12:05:39.686383 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="44965796-ddc0-4cb1-bdf6-f107b9c3373b" path="/var/lib/kubelet/pods/44965796-ddc0-4cb1-bdf6-f107b9c3373b/volumes" Dec 03 12:05:40 crc kubenswrapper[5052]: I1203 12:05:40.302861 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-78f8cc6c7-cc8t6" event={"ID":"c3489d6f-c371-4252-9a13-a055a4a55cbf","Type":"ContainerStarted","Data":"ea4edfbe541d4bc73a70c177586471d5c25b2291432f54d5ac0c1ec500004cd1"} Dec 03 12:05:40 crc kubenswrapper[5052]: I1203 12:05:40.302906 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-78f8cc6c7-cc8t6" event={"ID":"c3489d6f-c371-4252-9a13-a055a4a55cbf","Type":"ContainerStarted","Data":"ca0d48f58970de4e1adf1b2ca5b2f51208b9ac83d5b1bf8d4e9788aa6ad57153"} Dec 03 12:05:40 crc kubenswrapper[5052]: I1203 12:05:40.302918 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-78f8cc6c7-cc8t6" event={"ID":"c3489d6f-c371-4252-9a13-a055a4a55cbf","Type":"ContainerStarted","Data":"173db296bf1158238a3f169e35341dfbae99469e380d1f026a323d4ad8cf09d2"} Dec 03 12:05:40 crc kubenswrapper[5052]: I1203 12:05:40.303021 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/neutron-78f8cc6c7-cc8t6" Dec 03 12:05:40 crc kubenswrapper[5052]: I1203 12:05:40.326029 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/neutron-78f8cc6c7-cc8t6" podStartSLOduration=2.326012563 podStartE2EDuration="2.326012563s" podCreationTimestamp="2025-12-03 12:05:38 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:05:40.320098684 +0000 UTC m=+2730.908340448" watchObservedRunningTime="2025-12-03 12:05:40.326012563 +0000 UTC m=+2730.914254327" Dec 03 12:05:40 crc kubenswrapper[5052]: I1203 12:05:40.789907 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/neutron-5d6b56fbdb-ht2b8"] Dec 03 12:05:40 crc kubenswrapper[5052]: E1203 12:05:40.790809 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="44965796-ddc0-4cb1-bdf6-f107b9c3373b" containerName="barbican-api-log" Dec 03 12:05:40 crc kubenswrapper[5052]: I1203 12:05:40.790830 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="44965796-ddc0-4cb1-bdf6-f107b9c3373b" containerName="barbican-api-log" Dec 03 12:05:40 crc kubenswrapper[5052]: E1203 12:05:40.790877 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="44965796-ddc0-4cb1-bdf6-f107b9c3373b" containerName="barbican-api" Dec 03 12:05:40 crc kubenswrapper[5052]: I1203 12:05:40.790886 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="44965796-ddc0-4cb1-bdf6-f107b9c3373b" containerName="barbican-api" Dec 03 12:05:40 crc kubenswrapper[5052]: I1203 12:05:40.791152 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="44965796-ddc0-4cb1-bdf6-f107b9c3373b" containerName="barbican-api-log" Dec 03 12:05:40 crc kubenswrapper[5052]: I1203 12:05:40.791194 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="44965796-ddc0-4cb1-bdf6-f107b9c3373b" containerName="barbican-api" Dec 03 12:05:40 crc kubenswrapper[5052]: I1203 12:05:40.792553 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-5d6b56fbdb-ht2b8" Dec 03 12:05:40 crc kubenswrapper[5052]: I1203 12:05:40.796252 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-neutron-public-svc" Dec 03 12:05:40 crc kubenswrapper[5052]: I1203 12:05:40.796267 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-neutron-internal-svc" Dec 03 12:05:40 crc kubenswrapper[5052]: I1203 12:05:40.813041 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-5d6b56fbdb-ht2b8"] Dec 03 12:05:40 crc kubenswrapper[5052]: I1203 12:05:40.918203 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/141a49da-93b6-483b-a476-1d5f3efc29d1-httpd-config\") pod \"neutron-5d6b56fbdb-ht2b8\" (UID: \"141a49da-93b6-483b-a476-1d5f3efc29d1\") " pod="openstack-kuttl-tests/neutron-5d6b56fbdb-ht2b8" Dec 03 12:05:40 crc kubenswrapper[5052]: I1203 12:05:40.918372 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/141a49da-93b6-483b-a476-1d5f3efc29d1-public-tls-certs\") pod \"neutron-5d6b56fbdb-ht2b8\" (UID: \"141a49da-93b6-483b-a476-1d5f3efc29d1\") " pod="openstack-kuttl-tests/neutron-5d6b56fbdb-ht2b8" Dec 03 12:05:40 crc kubenswrapper[5052]: I1203 12:05:40.918417 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/141a49da-93b6-483b-a476-1d5f3efc29d1-internal-tls-certs\") pod \"neutron-5d6b56fbdb-ht2b8\" (UID: \"141a49da-93b6-483b-a476-1d5f3efc29d1\") " pod="openstack-kuttl-tests/neutron-5d6b56fbdb-ht2b8" Dec 03 12:05:40 crc kubenswrapper[5052]: I1203 12:05:40.918477 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/141a49da-93b6-483b-a476-1d5f3efc29d1-combined-ca-bundle\") pod \"neutron-5d6b56fbdb-ht2b8\" (UID: \"141a49da-93b6-483b-a476-1d5f3efc29d1\") " pod="openstack-kuttl-tests/neutron-5d6b56fbdb-ht2b8" Dec 03 12:05:40 crc kubenswrapper[5052]: I1203 12:05:40.918574 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/141a49da-93b6-483b-a476-1d5f3efc29d1-ovndb-tls-certs\") pod \"neutron-5d6b56fbdb-ht2b8\" (UID: \"141a49da-93b6-483b-a476-1d5f3efc29d1\") " pod="openstack-kuttl-tests/neutron-5d6b56fbdb-ht2b8" Dec 03 12:05:40 crc kubenswrapper[5052]: I1203 12:05:40.918788 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/141a49da-93b6-483b-a476-1d5f3efc29d1-config\") pod \"neutron-5d6b56fbdb-ht2b8\" (UID: \"141a49da-93b6-483b-a476-1d5f3efc29d1\") " pod="openstack-kuttl-tests/neutron-5d6b56fbdb-ht2b8" Dec 03 12:05:40 crc kubenswrapper[5052]: I1203 12:05:40.918892 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zmhzk\" (UniqueName: \"kubernetes.io/projected/141a49da-93b6-483b-a476-1d5f3efc29d1-kube-api-access-zmhzk\") pod \"neutron-5d6b56fbdb-ht2b8\" (UID: \"141a49da-93b6-483b-a476-1d5f3efc29d1\") " pod="openstack-kuttl-tests/neutron-5d6b56fbdb-ht2b8" Dec 03 12:05:41 crc kubenswrapper[5052]: I1203 12:05:41.020915 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/141a49da-93b6-483b-a476-1d5f3efc29d1-public-tls-certs\") pod \"neutron-5d6b56fbdb-ht2b8\" (UID: \"141a49da-93b6-483b-a476-1d5f3efc29d1\") " pod="openstack-kuttl-tests/neutron-5d6b56fbdb-ht2b8" Dec 03 12:05:41 crc kubenswrapper[5052]: I1203 12:05:41.020996 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/141a49da-93b6-483b-a476-1d5f3efc29d1-internal-tls-certs\") pod \"neutron-5d6b56fbdb-ht2b8\" (UID: \"141a49da-93b6-483b-a476-1d5f3efc29d1\") " pod="openstack-kuttl-tests/neutron-5d6b56fbdb-ht2b8" Dec 03 12:05:41 crc kubenswrapper[5052]: I1203 12:05:41.021028 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/141a49da-93b6-483b-a476-1d5f3efc29d1-combined-ca-bundle\") pod \"neutron-5d6b56fbdb-ht2b8\" (UID: \"141a49da-93b6-483b-a476-1d5f3efc29d1\") " pod="openstack-kuttl-tests/neutron-5d6b56fbdb-ht2b8" Dec 03 12:05:41 crc kubenswrapper[5052]: I1203 12:05:41.021070 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/141a49da-93b6-483b-a476-1d5f3efc29d1-ovndb-tls-certs\") pod \"neutron-5d6b56fbdb-ht2b8\" (UID: \"141a49da-93b6-483b-a476-1d5f3efc29d1\") " pod="openstack-kuttl-tests/neutron-5d6b56fbdb-ht2b8" Dec 03 12:05:41 crc kubenswrapper[5052]: I1203 12:05:41.021132 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/141a49da-93b6-483b-a476-1d5f3efc29d1-config\") pod \"neutron-5d6b56fbdb-ht2b8\" (UID: \"141a49da-93b6-483b-a476-1d5f3efc29d1\") " pod="openstack-kuttl-tests/neutron-5d6b56fbdb-ht2b8" Dec 03 12:05:41 crc kubenswrapper[5052]: I1203 12:05:41.021180 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-zmhzk\" (UniqueName: \"kubernetes.io/projected/141a49da-93b6-483b-a476-1d5f3efc29d1-kube-api-access-zmhzk\") pod \"neutron-5d6b56fbdb-ht2b8\" (UID: \"141a49da-93b6-483b-a476-1d5f3efc29d1\") " pod="openstack-kuttl-tests/neutron-5d6b56fbdb-ht2b8" Dec 03 12:05:41 crc kubenswrapper[5052]: I1203 12:05:41.021246 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/141a49da-93b6-483b-a476-1d5f3efc29d1-httpd-config\") pod \"neutron-5d6b56fbdb-ht2b8\" (UID: \"141a49da-93b6-483b-a476-1d5f3efc29d1\") " pod="openstack-kuttl-tests/neutron-5d6b56fbdb-ht2b8" Dec 03 12:05:41 crc kubenswrapper[5052]: I1203 12:05:41.027936 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/141a49da-93b6-483b-a476-1d5f3efc29d1-public-tls-certs\") pod \"neutron-5d6b56fbdb-ht2b8\" (UID: \"141a49da-93b6-483b-a476-1d5f3efc29d1\") " pod="openstack-kuttl-tests/neutron-5d6b56fbdb-ht2b8" Dec 03 12:05:41 crc kubenswrapper[5052]: I1203 12:05:41.028418 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/141a49da-93b6-483b-a476-1d5f3efc29d1-httpd-config\") pod \"neutron-5d6b56fbdb-ht2b8\" (UID: \"141a49da-93b6-483b-a476-1d5f3efc29d1\") " pod="openstack-kuttl-tests/neutron-5d6b56fbdb-ht2b8" Dec 03 12:05:41 crc kubenswrapper[5052]: I1203 12:05:41.029152 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/141a49da-93b6-483b-a476-1d5f3efc29d1-internal-tls-certs\") pod \"neutron-5d6b56fbdb-ht2b8\" (UID: \"141a49da-93b6-483b-a476-1d5f3efc29d1\") " pod="openstack-kuttl-tests/neutron-5d6b56fbdb-ht2b8" Dec 03 12:05:41 crc kubenswrapper[5052]: I1203 12:05:41.041736 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/141a49da-93b6-483b-a476-1d5f3efc29d1-ovndb-tls-certs\") pod \"neutron-5d6b56fbdb-ht2b8\" (UID: \"141a49da-93b6-483b-a476-1d5f3efc29d1\") " pod="openstack-kuttl-tests/neutron-5d6b56fbdb-ht2b8" Dec 03 12:05:41 crc kubenswrapper[5052]: I1203 12:05:41.042671 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/141a49da-93b6-483b-a476-1d5f3efc29d1-combined-ca-bundle\") pod \"neutron-5d6b56fbdb-ht2b8\" (UID: \"141a49da-93b6-483b-a476-1d5f3efc29d1\") " pod="openstack-kuttl-tests/neutron-5d6b56fbdb-ht2b8" Dec 03 12:05:41 crc kubenswrapper[5052]: I1203 12:05:41.043572 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-zmhzk\" (UniqueName: \"kubernetes.io/projected/141a49da-93b6-483b-a476-1d5f3efc29d1-kube-api-access-zmhzk\") pod \"neutron-5d6b56fbdb-ht2b8\" (UID: \"141a49da-93b6-483b-a476-1d5f3efc29d1\") " pod="openstack-kuttl-tests/neutron-5d6b56fbdb-ht2b8" Dec 03 12:05:41 crc kubenswrapper[5052]: I1203 12:05:41.043975 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/141a49da-93b6-483b-a476-1d5f3efc29d1-config\") pod \"neutron-5d6b56fbdb-ht2b8\" (UID: \"141a49da-93b6-483b-a476-1d5f3efc29d1\") " pod="openstack-kuttl-tests/neutron-5d6b56fbdb-ht2b8" Dec 03 12:05:41 crc kubenswrapper[5052]: I1203 12:05:41.112152 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-5d6b56fbdb-ht2b8" Dec 03 12:05:41 crc kubenswrapper[5052]: I1203 12:05:41.584395 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-5d6b56fbdb-ht2b8"] Dec 03 12:05:42 crc kubenswrapper[5052]: I1203 12:05:42.335981 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-5d6b56fbdb-ht2b8" event={"ID":"141a49da-93b6-483b-a476-1d5f3efc29d1","Type":"ContainerStarted","Data":"405640dc668d7b8021968432854058e5508074169be4f1e98d36626dd8db3fbf"} Dec 03 12:05:42 crc kubenswrapper[5052]: I1203 12:05:42.336610 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-5d6b56fbdb-ht2b8" event={"ID":"141a49da-93b6-483b-a476-1d5f3efc29d1","Type":"ContainerStarted","Data":"3d0ba918e75cd950bc4bab7a527b72d23c454cb5e03d5804f02c37caed27d0fa"} Dec 03 12:05:42 crc kubenswrapper[5052]: I1203 12:05:42.338012 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/neutron-5d6b56fbdb-ht2b8" Dec 03 12:05:42 crc kubenswrapper[5052]: I1203 12:05:42.338161 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-5d6b56fbdb-ht2b8" event={"ID":"141a49da-93b6-483b-a476-1d5f3efc29d1","Type":"ContainerStarted","Data":"848ae10ea79a99f5a6ff08cc94ca0107790699ed53098696b686a451c01bd64f"} Dec 03 12:05:42 crc kubenswrapper[5052]: I1203 12:05:42.358938 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/neutron-5d6b56fbdb-ht2b8" podStartSLOduration=2.358920673 podStartE2EDuration="2.358920673s" podCreationTimestamp="2025-12-03 12:05:40 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:05:42.358048108 +0000 UTC m=+2732.946289872" watchObservedRunningTime="2025-12-03 12:05:42.358920673 +0000 UTC m=+2732.947162437" Dec 03 12:05:42 crc kubenswrapper[5052]: I1203 12:05:42.562032 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openshift-marketplace/certified-operators-7lpk6"] Dec 03 12:05:42 crc kubenswrapper[5052]: I1203 12:05:42.564893 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/certified-operators-7lpk6" Dec 03 12:05:42 crc kubenswrapper[5052]: I1203 12:05:42.574127 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/certified-operators-7lpk6"] Dec 03 12:05:42 crc kubenswrapper[5052]: I1203 12:05:42.600233 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r9wq5\" (UniqueName: \"kubernetes.io/projected/cc8c9225-7106-4401-8031-76b004ac1dae-kube-api-access-r9wq5\") pod \"certified-operators-7lpk6\" (UID: \"cc8c9225-7106-4401-8031-76b004ac1dae\") " pod="openshift-marketplace/certified-operators-7lpk6" Dec 03 12:05:42 crc kubenswrapper[5052]: I1203 12:05:42.600583 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/cc8c9225-7106-4401-8031-76b004ac1dae-catalog-content\") pod \"certified-operators-7lpk6\" (UID: \"cc8c9225-7106-4401-8031-76b004ac1dae\") " pod="openshift-marketplace/certified-operators-7lpk6" Dec 03 12:05:42 crc kubenswrapper[5052]: I1203 12:05:42.600999 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/cc8c9225-7106-4401-8031-76b004ac1dae-utilities\") pod \"certified-operators-7lpk6\" (UID: \"cc8c9225-7106-4401-8031-76b004ac1dae\") " pod="openshift-marketplace/certified-operators-7lpk6" Dec 03 12:05:42 crc kubenswrapper[5052]: I1203 12:05:42.703077 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-r9wq5\" (UniqueName: \"kubernetes.io/projected/cc8c9225-7106-4401-8031-76b004ac1dae-kube-api-access-r9wq5\") pod \"certified-operators-7lpk6\" (UID: \"cc8c9225-7106-4401-8031-76b004ac1dae\") " pod="openshift-marketplace/certified-operators-7lpk6" Dec 03 12:05:42 crc kubenswrapper[5052]: I1203 12:05:42.703154 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/cc8c9225-7106-4401-8031-76b004ac1dae-catalog-content\") pod \"certified-operators-7lpk6\" (UID: \"cc8c9225-7106-4401-8031-76b004ac1dae\") " pod="openshift-marketplace/certified-operators-7lpk6" Dec 03 12:05:42 crc kubenswrapper[5052]: I1203 12:05:42.703295 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/cc8c9225-7106-4401-8031-76b004ac1dae-utilities\") pod \"certified-operators-7lpk6\" (UID: \"cc8c9225-7106-4401-8031-76b004ac1dae\") " pod="openshift-marketplace/certified-operators-7lpk6" Dec 03 12:05:42 crc kubenswrapper[5052]: I1203 12:05:42.704042 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/cc8c9225-7106-4401-8031-76b004ac1dae-utilities\") pod \"certified-operators-7lpk6\" (UID: \"cc8c9225-7106-4401-8031-76b004ac1dae\") " pod="openshift-marketplace/certified-operators-7lpk6" Dec 03 12:05:42 crc kubenswrapper[5052]: I1203 12:05:42.704042 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/cc8c9225-7106-4401-8031-76b004ac1dae-catalog-content\") pod \"certified-operators-7lpk6\" (UID: \"cc8c9225-7106-4401-8031-76b004ac1dae\") " pod="openshift-marketplace/certified-operators-7lpk6" Dec 03 12:05:42 crc kubenswrapper[5052]: I1203 12:05:42.730130 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-r9wq5\" (UniqueName: \"kubernetes.io/projected/cc8c9225-7106-4401-8031-76b004ac1dae-kube-api-access-r9wq5\") pod \"certified-operators-7lpk6\" (UID: \"cc8c9225-7106-4401-8031-76b004ac1dae\") " pod="openshift-marketplace/certified-operators-7lpk6" Dec 03 12:05:42 crc kubenswrapper[5052]: I1203 12:05:42.894172 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/certified-operators-7lpk6" Dec 03 12:05:43 crc kubenswrapper[5052]: I1203 12:05:43.503459 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/certified-operators-7lpk6"] Dec 03 12:05:43 crc kubenswrapper[5052]: W1203 12:05:43.507492 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc8c9225_7106_4401_8031_76b004ac1dae.slice/crio-f34bec76ad3f04d96b400ba37715f2da9dae5cfa0ea33fac1deea876cd3db121 WatchSource:0}: Error finding container f34bec76ad3f04d96b400ba37715f2da9dae5cfa0ea33fac1deea876cd3db121: Status 404 returned error can't find the container with id f34bec76ad3f04d96b400ba37715f2da9dae5cfa0ea33fac1deea876cd3db121 Dec 03 12:05:43 crc kubenswrapper[5052]: I1203 12:05:43.546042 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/placement-f4dfbd7d-mxsm8" Dec 03 12:05:43 crc kubenswrapper[5052]: I1203 12:05:43.548345 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/placement-f4dfbd7d-mxsm8" Dec 03 12:05:44 crc kubenswrapper[5052]: I1203 12:05:44.118673 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:05:44 crc kubenswrapper[5052]: I1203 12:05:44.360724 5052 generic.go:334] "Generic (PLEG): container finished" podID="cc8c9225-7106-4401-8031-76b004ac1dae" containerID="35b1c861222993567cb5e2bba2555b9b44575337f6dbef485155d67df49f6fbe" exitCode=0 Dec 03 12:05:44 crc kubenswrapper[5052]: I1203 12:05:44.362198 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/certified-operators-7lpk6" event={"ID":"cc8c9225-7106-4401-8031-76b004ac1dae","Type":"ContainerDied","Data":"35b1c861222993567cb5e2bba2555b9b44575337f6dbef485155d67df49f6fbe"} Dec 03 12:05:44 crc kubenswrapper[5052]: I1203 12:05:44.362228 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/certified-operators-7lpk6" event={"ID":"cc8c9225-7106-4401-8031-76b004ac1dae","Type":"ContainerStarted","Data":"f34bec76ad3f04d96b400ba37715f2da9dae5cfa0ea33fac1deea876cd3db121"} Dec 03 12:05:45 crc kubenswrapper[5052]: I1203 12:05:45.376736 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/certified-operators-7lpk6" event={"ID":"cc8c9225-7106-4401-8031-76b004ac1dae","Type":"ContainerStarted","Data":"ecbde994c55049472c4c720e6d42dbf2f5d5a7baf9b1f38823d137506bd92f2b"} Dec 03 12:05:46 crc kubenswrapper[5052]: I1203 12:05:46.390266 5052 generic.go:334] "Generic (PLEG): container finished" podID="cc8c9225-7106-4401-8031-76b004ac1dae" containerID="ecbde994c55049472c4c720e6d42dbf2f5d5a7baf9b1f38823d137506bd92f2b" exitCode=0 Dec 03 12:05:46 crc kubenswrapper[5052]: I1203 12:05:46.390331 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/certified-operators-7lpk6" event={"ID":"cc8c9225-7106-4401-8031-76b004ac1dae","Type":"ContainerDied","Data":"ecbde994c55049472c4c720e6d42dbf2f5d5a7baf9b1f38823d137506bd92f2b"} Dec 03 12:05:47 crc kubenswrapper[5052]: I1203 12:05:47.403049 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/certified-operators-7lpk6" event={"ID":"cc8c9225-7106-4401-8031-76b004ac1dae","Type":"ContainerStarted","Data":"445a92f219d953a54b102d093b66b02711553d60fcc192a3a47c668172d2f2a5"} Dec 03 12:05:47 crc kubenswrapper[5052]: I1203 12:05:47.429291 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-marketplace/certified-operators-7lpk6" podStartSLOduration=3.043922114 podStartE2EDuration="5.42926479s" podCreationTimestamp="2025-12-03 12:05:42 +0000 UTC" firstStartedPulling="2025-12-03 12:05:44.366074066 +0000 UTC m=+2734.954315830" lastFinishedPulling="2025-12-03 12:05:46.751416742 +0000 UTC m=+2737.339658506" observedRunningTime="2025-12-03 12:05:47.427977233 +0000 UTC m=+2738.016219017" watchObservedRunningTime="2025-12-03 12:05:47.42926479 +0000 UTC m=+2738.017506574" Dec 03 12:05:50 crc kubenswrapper[5052]: I1203 12:05:50.165937 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/keystone-6c6748d95d-66jbh" Dec 03 12:05:52 crc kubenswrapper[5052]: I1203 12:05:52.497159 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/openstackclient"] Dec 03 12:05:52 crc kubenswrapper[5052]: I1203 12:05:52.499297 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstackclient" Dec 03 12:05:52 crc kubenswrapper[5052]: I1203 12:05:52.504963 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"openstack-config" Dec 03 12:05:52 crc kubenswrapper[5052]: I1203 12:05:52.505169 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"openstack-config-secret" Dec 03 12:05:52 crc kubenswrapper[5052]: I1203 12:05:52.505239 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"openstackclient-openstackclient-dockercfg-2hr76" Dec 03 12:05:52 crc kubenswrapper[5052]: I1203 12:05:52.505867 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/openstackclient"] Dec 03 12:05:52 crc kubenswrapper[5052]: I1203 12:05:52.551713 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c7d0ffdb-bb0c-4212-9682-289d630186fd-combined-ca-bundle\") pod \"openstackclient\" (UID: \"c7d0ffdb-bb0c-4212-9682-289d630186fd\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 12:05:52 crc kubenswrapper[5052]: I1203 12:05:52.551812 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5wxll\" (UniqueName: \"kubernetes.io/projected/c7d0ffdb-bb0c-4212-9682-289d630186fd-kube-api-access-5wxll\") pod \"openstackclient\" (UID: \"c7d0ffdb-bb0c-4212-9682-289d630186fd\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 12:05:52 crc kubenswrapper[5052]: I1203 12:05:52.551957 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openstack-config\" (UniqueName: \"kubernetes.io/configmap/c7d0ffdb-bb0c-4212-9682-289d630186fd-openstack-config\") pod \"openstackclient\" (UID: \"c7d0ffdb-bb0c-4212-9682-289d630186fd\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 12:05:52 crc kubenswrapper[5052]: I1203 12:05:52.552025 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openstack-config-secret\" (UniqueName: \"kubernetes.io/secret/c7d0ffdb-bb0c-4212-9682-289d630186fd-openstack-config-secret\") pod \"openstackclient\" (UID: \"c7d0ffdb-bb0c-4212-9682-289d630186fd\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 12:05:52 crc kubenswrapper[5052]: I1203 12:05:52.653744 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"openstack-config-secret\" (UniqueName: \"kubernetes.io/secret/c7d0ffdb-bb0c-4212-9682-289d630186fd-openstack-config-secret\") pod \"openstackclient\" (UID: \"c7d0ffdb-bb0c-4212-9682-289d630186fd\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 12:05:52 crc kubenswrapper[5052]: I1203 12:05:52.654165 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c7d0ffdb-bb0c-4212-9682-289d630186fd-combined-ca-bundle\") pod \"openstackclient\" (UID: \"c7d0ffdb-bb0c-4212-9682-289d630186fd\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 12:05:52 crc kubenswrapper[5052]: I1203 12:05:52.654239 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-5wxll\" (UniqueName: \"kubernetes.io/projected/c7d0ffdb-bb0c-4212-9682-289d630186fd-kube-api-access-5wxll\") pod \"openstackclient\" (UID: \"c7d0ffdb-bb0c-4212-9682-289d630186fd\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 12:05:52 crc kubenswrapper[5052]: I1203 12:05:52.654312 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"openstack-config\" (UniqueName: \"kubernetes.io/configmap/c7d0ffdb-bb0c-4212-9682-289d630186fd-openstack-config\") pod \"openstackclient\" (UID: \"c7d0ffdb-bb0c-4212-9682-289d630186fd\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 12:05:52 crc kubenswrapper[5052]: I1203 12:05:52.655141 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"openstack-config\" (UniqueName: \"kubernetes.io/configmap/c7d0ffdb-bb0c-4212-9682-289d630186fd-openstack-config\") pod \"openstackclient\" (UID: \"c7d0ffdb-bb0c-4212-9682-289d630186fd\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 12:05:52 crc kubenswrapper[5052]: I1203 12:05:52.661638 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c7d0ffdb-bb0c-4212-9682-289d630186fd-combined-ca-bundle\") pod \"openstackclient\" (UID: \"c7d0ffdb-bb0c-4212-9682-289d630186fd\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 12:05:52 crc kubenswrapper[5052]: I1203 12:05:52.669774 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"openstack-config-secret\" (UniqueName: \"kubernetes.io/secret/c7d0ffdb-bb0c-4212-9682-289d630186fd-openstack-config-secret\") pod \"openstackclient\" (UID: \"c7d0ffdb-bb0c-4212-9682-289d630186fd\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 12:05:52 crc kubenswrapper[5052]: I1203 12:05:52.673420 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-5wxll\" (UniqueName: \"kubernetes.io/projected/c7d0ffdb-bb0c-4212-9682-289d630186fd-kube-api-access-5wxll\") pod \"openstackclient\" (UID: \"c7d0ffdb-bb0c-4212-9682-289d630186fd\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 12:05:52 crc kubenswrapper[5052]: I1203 12:05:52.827692 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstackclient" Dec 03 12:05:52 crc kubenswrapper[5052]: I1203 12:05:52.894682 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openshift-marketplace/certified-operators-7lpk6" Dec 03 12:05:52 crc kubenswrapper[5052]: I1203 12:05:52.894746 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-marketplace/certified-operators-7lpk6" Dec 03 12:05:52 crc kubenswrapper[5052]: I1203 12:05:52.962034 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-marketplace/certified-operators-7lpk6" Dec 03 12:05:53 crc kubenswrapper[5052]: I1203 12:05:53.312040 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/openstackclient"] Dec 03 12:05:53 crc kubenswrapper[5052]: I1203 12:05:53.461099 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstackclient" event={"ID":"c7d0ffdb-bb0c-4212-9682-289d630186fd","Type":"ContainerStarted","Data":"0d8443da6e9c2935ae0665b0b6630e3117f1fb8b38d97e435ad13378d6ca5010"} Dec 03 12:05:53 crc kubenswrapper[5052]: I1203 12:05:53.521165 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-marketplace/certified-operators-7lpk6" Dec 03 12:05:53 crc kubenswrapper[5052]: I1203 12:05:53.575330 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openshift-marketplace/certified-operators-7lpk6"] Dec 03 12:05:54 crc kubenswrapper[5052]: I1203 12:05:54.473774 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstackclient" event={"ID":"c7d0ffdb-bb0c-4212-9682-289d630186fd","Type":"ContainerStarted","Data":"dbd211edde83ca19480c688ca164c8a5521ad9be2a864cfa3177c9b6dafa1a07"} Dec 03 12:05:54 crc kubenswrapper[5052]: I1203 12:05:54.494184 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/openstackclient" podStartSLOduration=2.494162352 podStartE2EDuration="2.494162352s" podCreationTimestamp="2025-12-03 12:05:52 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:05:54.488184961 +0000 UTC m=+2745.076426725" watchObservedRunningTime="2025-12-03 12:05:54.494162352 +0000 UTC m=+2745.082404116" Dec 03 12:05:54 crc kubenswrapper[5052]: I1203 12:05:54.716330 5052 scope.go:117] "RemoveContainer" containerID="f562bb4258333809d9cdbdea6ecb675c5df538644d028c44e7b8954cc6a364c8" Dec 03 12:05:54 crc kubenswrapper[5052]: I1203 12:05:54.753919 5052 scope.go:117] "RemoveContainer" containerID="0bb871ee79387ffeef2e7d444c482d183bd44015eb4fc256b2f431a4e4cdb6e7" Dec 03 12:05:54 crc kubenswrapper[5052]: I1203 12:05:54.780481 5052 scope.go:117] "RemoveContainer" containerID="a7260079f4bde2479c2fcef41cd28875413e615dbb2c2031c2b3435fe94b4524" Dec 03 12:05:54 crc kubenswrapper[5052]: I1203 12:05:54.849283 5052 scope.go:117] "RemoveContainer" containerID="fcfc59f028f6c29ffcc2a22439423717d9f8cacc46e9ad92735c695c4aae8643" Dec 03 12:05:54 crc kubenswrapper[5052]: I1203 12:05:54.878283 5052 scope.go:117] "RemoveContainer" containerID="87bc47eac41558bc881491bf1b0295423263bce8a03bd87436868096e25a318d" Dec 03 12:05:54 crc kubenswrapper[5052]: I1203 12:05:54.923209 5052 scope.go:117] "RemoveContainer" containerID="47bcb775089e195a3a2981bacc0958e1cef8109d2909c2918e326e5e17c0b9ba" Dec 03 12:05:54 crc kubenswrapper[5052]: I1203 12:05:54.969089 5052 scope.go:117] "RemoveContainer" containerID="7f62ed7588549b9a67859ad18997d03e123b56ebc6c553d299ea1f897cb73ce2" Dec 03 12:05:55 crc kubenswrapper[5052]: I1203 12:05:55.005337 5052 scope.go:117] "RemoveContainer" containerID="2f0b1bc5e2c1b087f5be9d90327ea9a97c57020ec2b1922d1d41714021fc9db7" Dec 03 12:05:55 crc kubenswrapper[5052]: I1203 12:05:55.027644 5052 scope.go:117] "RemoveContainer" containerID="c88549135746d4c71800918fd34260a35faff14ed50e30eb9b79cd16e17209d2" Dec 03 12:05:55 crc kubenswrapper[5052]: I1203 12:05:55.057775 5052 scope.go:117] "RemoveContainer" containerID="02340be78947f8f3ca41eedbebdaa47f237470f1f23212c0d023f165bb78b32c" Dec 03 12:05:55 crc kubenswrapper[5052]: I1203 12:05:55.105152 5052 scope.go:117] "RemoveContainer" containerID="be4c08c873da291db8d549e72617dd42955fe1f6b45313231d60e035c2c44cc1" Dec 03 12:05:55 crc kubenswrapper[5052]: I1203 12:05:55.130339 5052 scope.go:117] "RemoveContainer" containerID="159a5df23aba4e3c962924f0a774119d3e466b040473b579f0a0904498430999" Dec 03 12:05:55 crc kubenswrapper[5052]: I1203 12:05:55.153697 5052 scope.go:117] "RemoveContainer" containerID="220ad427ac37b8c14a8f2bf882e72550b22b51dfe6dbd65908d5362fb9eed124" Dec 03 12:05:55 crc kubenswrapper[5052]: I1203 12:05:55.177334 5052 scope.go:117] "RemoveContainer" containerID="ae2dd9c7d6d2cf6767ca2ef1b07d9dc82f31acfcfdbfe9432084c6de9851a9f0" Dec 03 12:05:55 crc kubenswrapper[5052]: I1203 12:05:55.487047 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openshift-marketplace/certified-operators-7lpk6" podUID="cc8c9225-7106-4401-8031-76b004ac1dae" containerName="registry-server" containerID="cri-o://445a92f219d953a54b102d093b66b02711553d60fcc192a3a47c668172d2f2a5" gracePeriod=2 Dec 03 12:05:55 crc kubenswrapper[5052]: I1203 12:05:55.963830 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/certified-operators-7lpk6" Dec 03 12:05:56 crc kubenswrapper[5052]: I1203 12:05:56.021210 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/cc8c9225-7106-4401-8031-76b004ac1dae-catalog-content\") pod \"cc8c9225-7106-4401-8031-76b004ac1dae\" (UID: \"cc8c9225-7106-4401-8031-76b004ac1dae\") " Dec 03 12:05:56 crc kubenswrapper[5052]: I1203 12:05:56.021551 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r9wq5\" (UniqueName: \"kubernetes.io/projected/cc8c9225-7106-4401-8031-76b004ac1dae-kube-api-access-r9wq5\") pod \"cc8c9225-7106-4401-8031-76b004ac1dae\" (UID: \"cc8c9225-7106-4401-8031-76b004ac1dae\") " Dec 03 12:05:56 crc kubenswrapper[5052]: I1203 12:05:56.021718 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/cc8c9225-7106-4401-8031-76b004ac1dae-utilities\") pod \"cc8c9225-7106-4401-8031-76b004ac1dae\" (UID: \"cc8c9225-7106-4401-8031-76b004ac1dae\") " Dec 03 12:05:56 crc kubenswrapper[5052]: I1203 12:05:56.022218 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cc8c9225-7106-4401-8031-76b004ac1dae-utilities" (OuterVolumeSpecName: "utilities") pod "cc8c9225-7106-4401-8031-76b004ac1dae" (UID: "cc8c9225-7106-4401-8031-76b004ac1dae"). InnerVolumeSpecName "utilities". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:05:56 crc kubenswrapper[5052]: I1203 12:05:56.022793 5052 reconciler_common.go:293] "Volume detached for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/cc8c9225-7106-4401-8031-76b004ac1dae-utilities\") on node \"crc\" DevicePath \"\"" Dec 03 12:05:56 crc kubenswrapper[5052]: I1203 12:05:56.029301 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cc8c9225-7106-4401-8031-76b004ac1dae-kube-api-access-r9wq5" (OuterVolumeSpecName: "kube-api-access-r9wq5") pod "cc8c9225-7106-4401-8031-76b004ac1dae" (UID: "cc8c9225-7106-4401-8031-76b004ac1dae"). InnerVolumeSpecName "kube-api-access-r9wq5". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:05:56 crc kubenswrapper[5052]: I1203 12:05:56.078467 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cc8c9225-7106-4401-8031-76b004ac1dae-catalog-content" (OuterVolumeSpecName: "catalog-content") pod "cc8c9225-7106-4401-8031-76b004ac1dae" (UID: "cc8c9225-7106-4401-8031-76b004ac1dae"). InnerVolumeSpecName "catalog-content". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:05:56 crc kubenswrapper[5052]: I1203 12:05:56.125139 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-r9wq5\" (UniqueName: \"kubernetes.io/projected/cc8c9225-7106-4401-8031-76b004ac1dae-kube-api-access-r9wq5\") on node \"crc\" DevicePath \"\"" Dec 03 12:05:56 crc kubenswrapper[5052]: I1203 12:05:56.125201 5052 reconciler_common.go:293] "Volume detached for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/cc8c9225-7106-4401-8031-76b004ac1dae-catalog-content\") on node \"crc\" DevicePath \"\"" Dec 03 12:05:56 crc kubenswrapper[5052]: I1203 12:05:56.300925 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/swift-proxy-9bb7db5d-5q7wc"] Dec 03 12:05:56 crc kubenswrapper[5052]: E1203 12:05:56.301406 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="cc8c9225-7106-4401-8031-76b004ac1dae" containerName="registry-server" Dec 03 12:05:56 crc kubenswrapper[5052]: I1203 12:05:56.301432 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="cc8c9225-7106-4401-8031-76b004ac1dae" containerName="registry-server" Dec 03 12:05:56 crc kubenswrapper[5052]: E1203 12:05:56.301459 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="cc8c9225-7106-4401-8031-76b004ac1dae" containerName="extract-content" Dec 03 12:05:56 crc kubenswrapper[5052]: I1203 12:05:56.301467 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="cc8c9225-7106-4401-8031-76b004ac1dae" containerName="extract-content" Dec 03 12:05:56 crc kubenswrapper[5052]: E1203 12:05:56.301486 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="cc8c9225-7106-4401-8031-76b004ac1dae" containerName="extract-utilities" Dec 03 12:05:56 crc kubenswrapper[5052]: I1203 12:05:56.301494 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="cc8c9225-7106-4401-8031-76b004ac1dae" containerName="extract-utilities" Dec 03 12:05:56 crc kubenswrapper[5052]: I1203 12:05:56.301741 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="cc8c9225-7106-4401-8031-76b004ac1dae" containerName="registry-server" Dec 03 12:05:56 crc kubenswrapper[5052]: I1203 12:05:56.302966 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-proxy-9bb7db5d-5q7wc" Dec 03 12:05:56 crc kubenswrapper[5052]: I1203 12:05:56.304689 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"swift-proxy-config-data" Dec 03 12:05:56 crc kubenswrapper[5052]: I1203 12:05:56.305264 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-swift-public-svc" Dec 03 12:05:56 crc kubenswrapper[5052]: I1203 12:05:56.305369 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-swift-internal-svc" Dec 03 12:05:56 crc kubenswrapper[5052]: I1203 12:05:56.324105 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/swift-proxy-9bb7db5d-5q7wc"] Dec 03 12:05:56 crc kubenswrapper[5052]: I1203 12:05:56.328749 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/028738cd-3b98-4853-9506-d2eba91afd87-etc-swift\") pod \"swift-proxy-9bb7db5d-5q7wc\" (UID: \"028738cd-3b98-4853-9506-d2eba91afd87\") " pod="openstack-kuttl-tests/swift-proxy-9bb7db5d-5q7wc" Dec 03 12:05:56 crc kubenswrapper[5052]: I1203 12:05:56.328804 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/028738cd-3b98-4853-9506-d2eba91afd87-combined-ca-bundle\") pod \"swift-proxy-9bb7db5d-5q7wc\" (UID: \"028738cd-3b98-4853-9506-d2eba91afd87\") " pod="openstack-kuttl-tests/swift-proxy-9bb7db5d-5q7wc" Dec 03 12:05:56 crc kubenswrapper[5052]: I1203 12:05:56.328935 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/028738cd-3b98-4853-9506-d2eba91afd87-config-data\") pod \"swift-proxy-9bb7db5d-5q7wc\" (UID: \"028738cd-3b98-4853-9506-d2eba91afd87\") " pod="openstack-kuttl-tests/swift-proxy-9bb7db5d-5q7wc" Dec 03 12:05:56 crc kubenswrapper[5052]: I1203 12:05:56.329088 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/028738cd-3b98-4853-9506-d2eba91afd87-public-tls-certs\") pod \"swift-proxy-9bb7db5d-5q7wc\" (UID: \"028738cd-3b98-4853-9506-d2eba91afd87\") " pod="openstack-kuttl-tests/swift-proxy-9bb7db5d-5q7wc" Dec 03 12:05:56 crc kubenswrapper[5052]: I1203 12:05:56.329154 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/028738cd-3b98-4853-9506-d2eba91afd87-internal-tls-certs\") pod \"swift-proxy-9bb7db5d-5q7wc\" (UID: \"028738cd-3b98-4853-9506-d2eba91afd87\") " pod="openstack-kuttl-tests/swift-proxy-9bb7db5d-5q7wc" Dec 03 12:05:56 crc kubenswrapper[5052]: I1203 12:05:56.329316 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nsksg\" (UniqueName: \"kubernetes.io/projected/028738cd-3b98-4853-9506-d2eba91afd87-kube-api-access-nsksg\") pod \"swift-proxy-9bb7db5d-5q7wc\" (UID: \"028738cd-3b98-4853-9506-d2eba91afd87\") " pod="openstack-kuttl-tests/swift-proxy-9bb7db5d-5q7wc" Dec 03 12:05:56 crc kubenswrapper[5052]: I1203 12:05:56.329453 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/028738cd-3b98-4853-9506-d2eba91afd87-run-httpd\") pod \"swift-proxy-9bb7db5d-5q7wc\" (UID: \"028738cd-3b98-4853-9506-d2eba91afd87\") " pod="openstack-kuttl-tests/swift-proxy-9bb7db5d-5q7wc" Dec 03 12:05:56 crc kubenswrapper[5052]: I1203 12:05:56.329503 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/028738cd-3b98-4853-9506-d2eba91afd87-log-httpd\") pod \"swift-proxy-9bb7db5d-5q7wc\" (UID: \"028738cd-3b98-4853-9506-d2eba91afd87\") " pod="openstack-kuttl-tests/swift-proxy-9bb7db5d-5q7wc" Dec 03 12:05:56 crc kubenswrapper[5052]: I1203 12:05:56.431611 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/028738cd-3b98-4853-9506-d2eba91afd87-etc-swift\") pod \"swift-proxy-9bb7db5d-5q7wc\" (UID: \"028738cd-3b98-4853-9506-d2eba91afd87\") " pod="openstack-kuttl-tests/swift-proxy-9bb7db5d-5q7wc" Dec 03 12:05:56 crc kubenswrapper[5052]: I1203 12:05:56.431691 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/028738cd-3b98-4853-9506-d2eba91afd87-combined-ca-bundle\") pod \"swift-proxy-9bb7db5d-5q7wc\" (UID: \"028738cd-3b98-4853-9506-d2eba91afd87\") " pod="openstack-kuttl-tests/swift-proxy-9bb7db5d-5q7wc" Dec 03 12:05:56 crc kubenswrapper[5052]: I1203 12:05:56.431754 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/028738cd-3b98-4853-9506-d2eba91afd87-config-data\") pod \"swift-proxy-9bb7db5d-5q7wc\" (UID: \"028738cd-3b98-4853-9506-d2eba91afd87\") " pod="openstack-kuttl-tests/swift-proxy-9bb7db5d-5q7wc" Dec 03 12:05:56 crc kubenswrapper[5052]: I1203 12:05:56.431795 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/028738cd-3b98-4853-9506-d2eba91afd87-public-tls-certs\") pod \"swift-proxy-9bb7db5d-5q7wc\" (UID: \"028738cd-3b98-4853-9506-d2eba91afd87\") " pod="openstack-kuttl-tests/swift-proxy-9bb7db5d-5q7wc" Dec 03 12:05:56 crc kubenswrapper[5052]: I1203 12:05:56.431829 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/028738cd-3b98-4853-9506-d2eba91afd87-internal-tls-certs\") pod \"swift-proxy-9bb7db5d-5q7wc\" (UID: \"028738cd-3b98-4853-9506-d2eba91afd87\") " pod="openstack-kuttl-tests/swift-proxy-9bb7db5d-5q7wc" Dec 03 12:05:56 crc kubenswrapper[5052]: I1203 12:05:56.431889 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-nsksg\" (UniqueName: \"kubernetes.io/projected/028738cd-3b98-4853-9506-d2eba91afd87-kube-api-access-nsksg\") pod \"swift-proxy-9bb7db5d-5q7wc\" (UID: \"028738cd-3b98-4853-9506-d2eba91afd87\") " pod="openstack-kuttl-tests/swift-proxy-9bb7db5d-5q7wc" Dec 03 12:05:56 crc kubenswrapper[5052]: I1203 12:05:56.431950 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/028738cd-3b98-4853-9506-d2eba91afd87-run-httpd\") pod \"swift-proxy-9bb7db5d-5q7wc\" (UID: \"028738cd-3b98-4853-9506-d2eba91afd87\") " pod="openstack-kuttl-tests/swift-proxy-9bb7db5d-5q7wc" Dec 03 12:05:56 crc kubenswrapper[5052]: I1203 12:05:56.431980 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/028738cd-3b98-4853-9506-d2eba91afd87-log-httpd\") pod \"swift-proxy-9bb7db5d-5q7wc\" (UID: \"028738cd-3b98-4853-9506-d2eba91afd87\") " pod="openstack-kuttl-tests/swift-proxy-9bb7db5d-5q7wc" Dec 03 12:05:56 crc kubenswrapper[5052]: I1203 12:05:56.432576 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/028738cd-3b98-4853-9506-d2eba91afd87-log-httpd\") pod \"swift-proxy-9bb7db5d-5q7wc\" (UID: \"028738cd-3b98-4853-9506-d2eba91afd87\") " pod="openstack-kuttl-tests/swift-proxy-9bb7db5d-5q7wc" Dec 03 12:05:56 crc kubenswrapper[5052]: I1203 12:05:56.433178 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/028738cd-3b98-4853-9506-d2eba91afd87-run-httpd\") pod \"swift-proxy-9bb7db5d-5q7wc\" (UID: \"028738cd-3b98-4853-9506-d2eba91afd87\") " pod="openstack-kuttl-tests/swift-proxy-9bb7db5d-5q7wc" Dec 03 12:05:56 crc kubenswrapper[5052]: I1203 12:05:56.438049 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/028738cd-3b98-4853-9506-d2eba91afd87-combined-ca-bundle\") pod \"swift-proxy-9bb7db5d-5q7wc\" (UID: \"028738cd-3b98-4853-9506-d2eba91afd87\") " pod="openstack-kuttl-tests/swift-proxy-9bb7db5d-5q7wc" Dec 03 12:05:56 crc kubenswrapper[5052]: I1203 12:05:56.438272 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/028738cd-3b98-4853-9506-d2eba91afd87-public-tls-certs\") pod \"swift-proxy-9bb7db5d-5q7wc\" (UID: \"028738cd-3b98-4853-9506-d2eba91afd87\") " pod="openstack-kuttl-tests/swift-proxy-9bb7db5d-5q7wc" Dec 03 12:05:56 crc kubenswrapper[5052]: I1203 12:05:56.438814 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/028738cd-3b98-4853-9506-d2eba91afd87-etc-swift\") pod \"swift-proxy-9bb7db5d-5q7wc\" (UID: \"028738cd-3b98-4853-9506-d2eba91afd87\") " pod="openstack-kuttl-tests/swift-proxy-9bb7db5d-5q7wc" Dec 03 12:05:56 crc kubenswrapper[5052]: I1203 12:05:56.440178 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/028738cd-3b98-4853-9506-d2eba91afd87-internal-tls-certs\") pod \"swift-proxy-9bb7db5d-5q7wc\" (UID: \"028738cd-3b98-4853-9506-d2eba91afd87\") " pod="openstack-kuttl-tests/swift-proxy-9bb7db5d-5q7wc" Dec 03 12:05:56 crc kubenswrapper[5052]: I1203 12:05:56.441851 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/028738cd-3b98-4853-9506-d2eba91afd87-config-data\") pod \"swift-proxy-9bb7db5d-5q7wc\" (UID: \"028738cd-3b98-4853-9506-d2eba91afd87\") " pod="openstack-kuttl-tests/swift-proxy-9bb7db5d-5q7wc" Dec 03 12:05:56 crc kubenswrapper[5052]: I1203 12:05:56.460505 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-nsksg\" (UniqueName: \"kubernetes.io/projected/028738cd-3b98-4853-9506-d2eba91afd87-kube-api-access-nsksg\") pod \"swift-proxy-9bb7db5d-5q7wc\" (UID: \"028738cd-3b98-4853-9506-d2eba91afd87\") " pod="openstack-kuttl-tests/swift-proxy-9bb7db5d-5q7wc" Dec 03 12:05:56 crc kubenswrapper[5052]: I1203 12:05:56.499340 5052 generic.go:334] "Generic (PLEG): container finished" podID="cc8c9225-7106-4401-8031-76b004ac1dae" containerID="445a92f219d953a54b102d093b66b02711553d60fcc192a3a47c668172d2f2a5" exitCode=0 Dec 03 12:05:56 crc kubenswrapper[5052]: I1203 12:05:56.499390 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/certified-operators-7lpk6" event={"ID":"cc8c9225-7106-4401-8031-76b004ac1dae","Type":"ContainerDied","Data":"445a92f219d953a54b102d093b66b02711553d60fcc192a3a47c668172d2f2a5"} Dec 03 12:05:56 crc kubenswrapper[5052]: I1203 12:05:56.499400 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/certified-operators-7lpk6" Dec 03 12:05:56 crc kubenswrapper[5052]: I1203 12:05:56.499423 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/certified-operators-7lpk6" event={"ID":"cc8c9225-7106-4401-8031-76b004ac1dae","Type":"ContainerDied","Data":"f34bec76ad3f04d96b400ba37715f2da9dae5cfa0ea33fac1deea876cd3db121"} Dec 03 12:05:56 crc kubenswrapper[5052]: I1203 12:05:56.499445 5052 scope.go:117] "RemoveContainer" containerID="445a92f219d953a54b102d093b66b02711553d60fcc192a3a47c668172d2f2a5" Dec 03 12:05:56 crc kubenswrapper[5052]: I1203 12:05:56.521845 5052 scope.go:117] "RemoveContainer" containerID="ecbde994c55049472c4c720e6d42dbf2f5d5a7baf9b1f38823d137506bd92f2b" Dec 03 12:05:56 crc kubenswrapper[5052]: I1203 12:05:56.542441 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openshift-marketplace/certified-operators-7lpk6"] Dec 03 12:05:56 crc kubenswrapper[5052]: I1203 12:05:56.548396 5052 scope.go:117] "RemoveContainer" containerID="35b1c861222993567cb5e2bba2555b9b44575337f6dbef485155d67df49f6fbe" Dec 03 12:05:56 crc kubenswrapper[5052]: I1203 12:05:56.552555 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openshift-marketplace/certified-operators-7lpk6"] Dec 03 12:05:56 crc kubenswrapper[5052]: I1203 12:05:56.570708 5052 scope.go:117] "RemoveContainer" containerID="445a92f219d953a54b102d093b66b02711553d60fcc192a3a47c668172d2f2a5" Dec 03 12:05:56 crc kubenswrapper[5052]: E1203 12:05:56.571261 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"445a92f219d953a54b102d093b66b02711553d60fcc192a3a47c668172d2f2a5\": container with ID starting with 445a92f219d953a54b102d093b66b02711553d60fcc192a3a47c668172d2f2a5 not found: ID does not exist" containerID="445a92f219d953a54b102d093b66b02711553d60fcc192a3a47c668172d2f2a5" Dec 03 12:05:56 crc kubenswrapper[5052]: I1203 12:05:56.571335 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"445a92f219d953a54b102d093b66b02711553d60fcc192a3a47c668172d2f2a5"} err="failed to get container status \"445a92f219d953a54b102d093b66b02711553d60fcc192a3a47c668172d2f2a5\": rpc error: code = NotFound desc = could not find container \"445a92f219d953a54b102d093b66b02711553d60fcc192a3a47c668172d2f2a5\": container with ID starting with 445a92f219d953a54b102d093b66b02711553d60fcc192a3a47c668172d2f2a5 not found: ID does not exist" Dec 03 12:05:56 crc kubenswrapper[5052]: I1203 12:05:56.571375 5052 scope.go:117] "RemoveContainer" containerID="ecbde994c55049472c4c720e6d42dbf2f5d5a7baf9b1f38823d137506bd92f2b" Dec 03 12:05:56 crc kubenswrapper[5052]: E1203 12:05:56.571788 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ecbde994c55049472c4c720e6d42dbf2f5d5a7baf9b1f38823d137506bd92f2b\": container with ID starting with ecbde994c55049472c4c720e6d42dbf2f5d5a7baf9b1f38823d137506bd92f2b not found: ID does not exist" containerID="ecbde994c55049472c4c720e6d42dbf2f5d5a7baf9b1f38823d137506bd92f2b" Dec 03 12:05:56 crc kubenswrapper[5052]: I1203 12:05:56.571829 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ecbde994c55049472c4c720e6d42dbf2f5d5a7baf9b1f38823d137506bd92f2b"} err="failed to get container status \"ecbde994c55049472c4c720e6d42dbf2f5d5a7baf9b1f38823d137506bd92f2b\": rpc error: code = NotFound desc = could not find container \"ecbde994c55049472c4c720e6d42dbf2f5d5a7baf9b1f38823d137506bd92f2b\": container with ID starting with ecbde994c55049472c4c720e6d42dbf2f5d5a7baf9b1f38823d137506bd92f2b not found: ID does not exist" Dec 03 12:05:56 crc kubenswrapper[5052]: I1203 12:05:56.571856 5052 scope.go:117] "RemoveContainer" containerID="35b1c861222993567cb5e2bba2555b9b44575337f6dbef485155d67df49f6fbe" Dec 03 12:05:56 crc kubenswrapper[5052]: E1203 12:05:56.572123 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"35b1c861222993567cb5e2bba2555b9b44575337f6dbef485155d67df49f6fbe\": container with ID starting with 35b1c861222993567cb5e2bba2555b9b44575337f6dbef485155d67df49f6fbe not found: ID does not exist" containerID="35b1c861222993567cb5e2bba2555b9b44575337f6dbef485155d67df49f6fbe" Dec 03 12:05:56 crc kubenswrapper[5052]: I1203 12:05:56.572158 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"35b1c861222993567cb5e2bba2555b9b44575337f6dbef485155d67df49f6fbe"} err="failed to get container status \"35b1c861222993567cb5e2bba2555b9b44575337f6dbef485155d67df49f6fbe\": rpc error: code = NotFound desc = could not find container \"35b1c861222993567cb5e2bba2555b9b44575337f6dbef485155d67df49f6fbe\": container with ID starting with 35b1c861222993567cb5e2bba2555b9b44575337f6dbef485155d67df49f6fbe not found: ID does not exist" Dec 03 12:05:56 crc kubenswrapper[5052]: I1203 12:05:56.621717 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-proxy-9bb7db5d-5q7wc" Dec 03 12:05:56 crc kubenswrapper[5052]: I1203 12:05:56.974489 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:05:56 crc kubenswrapper[5052]: I1203 12:05:56.975223 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="ae2d7638-3ed1-4ab4-8d55-8eb8608bc97e" containerName="ceilometer-central-agent" containerID="cri-o://aa090b5d18b872f74a8114d8dca0ba1dcc3cf87025d6bf6e190621c1bc9960c4" gracePeriod=30 Dec 03 12:05:56 crc kubenswrapper[5052]: I1203 12:05:56.975283 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="ae2d7638-3ed1-4ab4-8d55-8eb8608bc97e" containerName="proxy-httpd" containerID="cri-o://cb3e22986020c181b5ae6726aae34a38e1bcb7a291bf69547b51d5917d2e744c" gracePeriod=30 Dec 03 12:05:56 crc kubenswrapper[5052]: I1203 12:05:56.975344 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="ae2d7638-3ed1-4ab4-8d55-8eb8608bc97e" containerName="sg-core" containerID="cri-o://e90e2d7c78b50d4ab2e506730826ae6cbc28fbf2dc044144c3b2d062e11c08c5" gracePeriod=30 Dec 03 12:05:56 crc kubenswrapper[5052]: I1203 12:05:56.975400 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="ae2d7638-3ed1-4ab4-8d55-8eb8608bc97e" containerName="ceilometer-notification-agent" containerID="cri-o://c1b2b526b75e2f6e7dba586cd66a185e3baf50dddd08f9f766d7c67ae4aab2dc" gracePeriod=30 Dec 03 12:05:57 crc kubenswrapper[5052]: I1203 12:05:57.157391 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/swift-proxy-9bb7db5d-5q7wc"] Dec 03 12:05:57 crc kubenswrapper[5052]: I1203 12:05:57.523210 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-proxy-9bb7db5d-5q7wc" event={"ID":"028738cd-3b98-4853-9506-d2eba91afd87","Type":"ContainerStarted","Data":"531801a1d9fdbb4eab4dc17c841d530240327b26c1a21395bf0c594278f80f5c"} Dec 03 12:05:57 crc kubenswrapper[5052]: I1203 12:05:57.523265 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-proxy-9bb7db5d-5q7wc" event={"ID":"028738cd-3b98-4853-9506-d2eba91afd87","Type":"ContainerStarted","Data":"b9fac34047899cc094f874390d8356eb9d1ec157bffca85662e27dc6448b187a"} Dec 03 12:05:57 crc kubenswrapper[5052]: I1203 12:05:57.529635 5052 generic.go:334] "Generic (PLEG): container finished" podID="ae2d7638-3ed1-4ab4-8d55-8eb8608bc97e" containerID="cb3e22986020c181b5ae6726aae34a38e1bcb7a291bf69547b51d5917d2e744c" exitCode=0 Dec 03 12:05:57 crc kubenswrapper[5052]: I1203 12:05:57.529670 5052 generic.go:334] "Generic (PLEG): container finished" podID="ae2d7638-3ed1-4ab4-8d55-8eb8608bc97e" containerID="e90e2d7c78b50d4ab2e506730826ae6cbc28fbf2dc044144c3b2d062e11c08c5" exitCode=2 Dec 03 12:05:57 crc kubenswrapper[5052]: I1203 12:05:57.529736 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"ae2d7638-3ed1-4ab4-8d55-8eb8608bc97e","Type":"ContainerDied","Data":"cb3e22986020c181b5ae6726aae34a38e1bcb7a291bf69547b51d5917d2e744c"} Dec 03 12:05:57 crc kubenswrapper[5052]: I1203 12:05:57.529770 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"ae2d7638-3ed1-4ab4-8d55-8eb8608bc97e","Type":"ContainerDied","Data":"e90e2d7c78b50d4ab2e506730826ae6cbc28fbf2dc044144c3b2d062e11c08c5"} Dec 03 12:05:57 crc kubenswrapper[5052]: I1203 12:05:57.687161 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cc8c9225-7106-4401-8031-76b004ac1dae" path="/var/lib/kubelet/pods/cc8c9225-7106-4401-8031-76b004ac1dae/volumes" Dec 03 12:05:58 crc kubenswrapper[5052]: I1203 12:05:58.544852 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-proxy-9bb7db5d-5q7wc" event={"ID":"028738cd-3b98-4853-9506-d2eba91afd87","Type":"ContainerStarted","Data":"2d0b57b97cc7d912f25d72df376cda014d9ee39ad7a02f4b38c46e47d7ba19ac"} Dec 03 12:05:58 crc kubenswrapper[5052]: I1203 12:05:58.544944 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/swift-proxy-9bb7db5d-5q7wc" Dec 03 12:05:58 crc kubenswrapper[5052]: I1203 12:05:58.544968 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/swift-proxy-9bb7db5d-5q7wc" Dec 03 12:05:58 crc kubenswrapper[5052]: I1203 12:05:58.548711 5052 generic.go:334] "Generic (PLEG): container finished" podID="ae2d7638-3ed1-4ab4-8d55-8eb8608bc97e" containerID="aa090b5d18b872f74a8114d8dca0ba1dcc3cf87025d6bf6e190621c1bc9960c4" exitCode=0 Dec 03 12:05:58 crc kubenswrapper[5052]: I1203 12:05:58.548763 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"ae2d7638-3ed1-4ab4-8d55-8eb8608bc97e","Type":"ContainerDied","Data":"aa090b5d18b872f74a8114d8dca0ba1dcc3cf87025d6bf6e190621c1bc9960c4"} Dec 03 12:05:58 crc kubenswrapper[5052]: I1203 12:05:58.568221 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/swift-proxy-9bb7db5d-5q7wc" podStartSLOduration=2.568203296 podStartE2EDuration="2.568203296s" podCreationTimestamp="2025-12-03 12:05:56 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:05:58.565797697 +0000 UTC m=+2749.154039461" watchObservedRunningTime="2025-12-03 12:05:58.568203296 +0000 UTC m=+2749.156445060" Dec 03 12:05:58 crc kubenswrapper[5052]: I1203 12:05:58.761089 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:05:58 crc kubenswrapper[5052]: I1203 12:05:58.761472 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-external-api-0" podUID="d2588f8c-9004-4e6e-8dd5-c6234b94d5cf" containerName="glance-log" containerID="cri-o://71f1cba19f2d1076cbc88a6c4acbeb2179613b427473cbf3c46cdde47f7e75b5" gracePeriod=30 Dec 03 12:05:58 crc kubenswrapper[5052]: I1203 12:05:58.761866 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-external-api-0" podUID="d2588f8c-9004-4e6e-8dd5-c6234b94d5cf" containerName="glance-httpd" containerID="cri-o://65e485c8f05b257462566a96517913bed283ca6386a0edf4f3a7a2023ee080ce" gracePeriod=30 Dec 03 12:05:59 crc kubenswrapper[5052]: I1203 12:05:59.379543 5052 patch_prober.go:28] interesting pod/machine-config-daemon-74bj6 container/machine-config-daemon namespace/openshift-machine-config-operator: Liveness probe status=failure output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" start-of-body= Dec 03 12:05:59 crc kubenswrapper[5052]: I1203 12:05:59.379596 5052 prober.go:107] "Probe failed" probeType="Liveness" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" containerName="machine-config-daemon" probeResult="failure" output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" Dec 03 12:05:59 crc kubenswrapper[5052]: I1203 12:05:59.379640 5052 kubelet.go:2542] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" Dec 03 12:05:59 crc kubenswrapper[5052]: I1203 12:05:59.380450 5052 kuberuntime_manager.go:1027] "Message for Container of pod" containerName="machine-config-daemon" containerStatusID={"Type":"cri-o","ID":"fcd7a4f50449fc80828465d45d26c57ddd80c8ab4f967dd642dae3c503a27345"} pod="openshift-machine-config-operator/machine-config-daemon-74bj6" containerMessage="Container machine-config-daemon failed liveness probe, will be restarted" Dec 03 12:05:59 crc kubenswrapper[5052]: I1203 12:05:59.380505 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" containerName="machine-config-daemon" containerID="cri-o://fcd7a4f50449fc80828465d45d26c57ddd80c8ab4f967dd642dae3c503a27345" gracePeriod=600 Dec 03 12:05:59 crc kubenswrapper[5052]: I1203 12:05:59.560506 5052 generic.go:334] "Generic (PLEG): container finished" podID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" containerID="fcd7a4f50449fc80828465d45d26c57ddd80c8ab4f967dd642dae3c503a27345" exitCode=0 Dec 03 12:05:59 crc kubenswrapper[5052]: I1203 12:05:59.560615 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" event={"ID":"10c237bc-72e7-4da3-9f2c-b1688395bf6e","Type":"ContainerDied","Data":"fcd7a4f50449fc80828465d45d26c57ddd80c8ab4f967dd642dae3c503a27345"} Dec 03 12:05:59 crc kubenswrapper[5052]: I1203 12:05:59.560922 5052 scope.go:117] "RemoveContainer" containerID="d224717c6d9ca8135fd510e03a6c40c9f2b1a56dd8b4a3b685af3e2dfa855982" Dec 03 12:05:59 crc kubenswrapper[5052]: I1203 12:05:59.563909 5052 generic.go:334] "Generic (PLEG): container finished" podID="d2588f8c-9004-4e6e-8dd5-c6234b94d5cf" containerID="71f1cba19f2d1076cbc88a6c4acbeb2179613b427473cbf3c46cdde47f7e75b5" exitCode=143 Dec 03 12:05:59 crc kubenswrapper[5052]: I1203 12:05:59.563979 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"d2588f8c-9004-4e6e-8dd5-c6234b94d5cf","Type":"ContainerDied","Data":"71f1cba19f2d1076cbc88a6c4acbeb2179613b427473cbf3c46cdde47f7e75b5"} Dec 03 12:06:00 crc kubenswrapper[5052]: I1203 12:06:00.055576 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:06:00 crc kubenswrapper[5052]: I1203 12:06:00.057022 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-internal-api-0" podUID="a3f4c9a7-c441-4487-9814-486ac5e66848" containerName="glance-log" containerID="cri-o://15bd35b56459bfabff2e71300b23da8680d50a0e069d64db131c072ec6fed100" gracePeriod=30 Dec 03 12:06:00 crc kubenswrapper[5052]: I1203 12:06:00.057082 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-internal-api-0" podUID="a3f4c9a7-c441-4487-9814-486ac5e66848" containerName="glance-httpd" containerID="cri-o://e6ef915c623f6539b5a4b98c7c0a747b3dd34c5a9b33f8653603dec931503dbe" gracePeriod=30 Dec 03 12:06:00 crc kubenswrapper[5052]: I1203 12:06:00.620957 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" event={"ID":"10c237bc-72e7-4da3-9f2c-b1688395bf6e","Type":"ContainerStarted","Data":"63298107b3d090deff9ed92f3d613e5b1c86d9137684d117500257db8ee1c805"} Dec 03 12:06:00 crc kubenswrapper[5052]: I1203 12:06:00.633735 5052 generic.go:334] "Generic (PLEG): container finished" podID="a3f4c9a7-c441-4487-9814-486ac5e66848" containerID="15bd35b56459bfabff2e71300b23da8680d50a0e069d64db131c072ec6fed100" exitCode=143 Dec 03 12:06:00 crc kubenswrapper[5052]: I1203 12:06:00.633827 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"a3f4c9a7-c441-4487-9814-486ac5e66848","Type":"ContainerDied","Data":"15bd35b56459bfabff2e71300b23da8680d50a0e069d64db131c072ec6fed100"} Dec 03 12:06:00 crc kubenswrapper[5052]: I1203 12:06:00.642736 5052 generic.go:334] "Generic (PLEG): container finished" podID="ae2d7638-3ed1-4ab4-8d55-8eb8608bc97e" containerID="c1b2b526b75e2f6e7dba586cd66a185e3baf50dddd08f9f766d7c67ae4aab2dc" exitCode=0 Dec 03 12:06:00 crc kubenswrapper[5052]: I1203 12:06:00.642775 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"ae2d7638-3ed1-4ab4-8d55-8eb8608bc97e","Type":"ContainerDied","Data":"c1b2b526b75e2f6e7dba586cd66a185e3baf50dddd08f9f766d7c67ae4aab2dc"} Dec 03 12:06:00 crc kubenswrapper[5052]: I1203 12:06:00.754594 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-api-db-create-t6q5w"] Dec 03 12:06:00 crc kubenswrapper[5052]: I1203 12:06:00.756570 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-db-create-t6q5w" Dec 03 12:06:00 crc kubenswrapper[5052]: I1203 12:06:00.761958 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/4741b42c-35a5-407d-b5cc-c7d47b07ce0b-operator-scripts\") pod \"nova-api-db-create-t6q5w\" (UID: \"4741b42c-35a5-407d-b5cc-c7d47b07ce0b\") " pod="openstack-kuttl-tests/nova-api-db-create-t6q5w" Dec 03 12:06:00 crc kubenswrapper[5052]: I1203 12:06:00.762306 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sdf5b\" (UniqueName: \"kubernetes.io/projected/4741b42c-35a5-407d-b5cc-c7d47b07ce0b-kube-api-access-sdf5b\") pod \"nova-api-db-create-t6q5w\" (UID: \"4741b42c-35a5-407d-b5cc-c7d47b07ce0b\") " pod="openstack-kuttl-tests/nova-api-db-create-t6q5w" Dec 03 12:06:00 crc kubenswrapper[5052]: I1203 12:06:00.778240 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-db-create-t6q5w"] Dec 03 12:06:00 crc kubenswrapper[5052]: I1203 12:06:00.823317 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell0-db-create-8kvhf"] Dec 03 12:06:00 crc kubenswrapper[5052]: I1203 12:06:00.824891 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-db-create-8kvhf" Dec 03 12:06:00 crc kubenswrapper[5052]: I1203 12:06:00.836908 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-api-48e2-account-create-update-dcqjr"] Dec 03 12:06:00 crc kubenswrapper[5052]: I1203 12:06:00.840868 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-48e2-account-create-update-dcqjr" Dec 03 12:06:00 crc kubenswrapper[5052]: I1203 12:06:00.843223 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-api-db-secret" Dec 03 12:06:00 crc kubenswrapper[5052]: I1203 12:06:00.848708 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell0-db-create-8kvhf"] Dec 03 12:06:00 crc kubenswrapper[5052]: I1203 12:06:00.858164 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-48e2-account-create-update-dcqjr"] Dec 03 12:06:00 crc kubenswrapper[5052]: I1203 12:06:00.866204 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-sdf5b\" (UniqueName: \"kubernetes.io/projected/4741b42c-35a5-407d-b5cc-c7d47b07ce0b-kube-api-access-sdf5b\") pod \"nova-api-db-create-t6q5w\" (UID: \"4741b42c-35a5-407d-b5cc-c7d47b07ce0b\") " pod="openstack-kuttl-tests/nova-api-db-create-t6q5w" Dec 03 12:06:00 crc kubenswrapper[5052]: I1203 12:06:00.866258 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/4741b42c-35a5-407d-b5cc-c7d47b07ce0b-operator-scripts\") pod \"nova-api-db-create-t6q5w\" (UID: \"4741b42c-35a5-407d-b5cc-c7d47b07ce0b\") " pod="openstack-kuttl-tests/nova-api-db-create-t6q5w" Dec 03 12:06:00 crc kubenswrapper[5052]: I1203 12:06:00.868351 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/4741b42c-35a5-407d-b5cc-c7d47b07ce0b-operator-scripts\") pod \"nova-api-db-create-t6q5w\" (UID: \"4741b42c-35a5-407d-b5cc-c7d47b07ce0b\") " pod="openstack-kuttl-tests/nova-api-db-create-t6q5w" Dec 03 12:06:00 crc kubenswrapper[5052]: I1203 12:06:00.891484 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-sdf5b\" (UniqueName: \"kubernetes.io/projected/4741b42c-35a5-407d-b5cc-c7d47b07ce0b-kube-api-access-sdf5b\") pod \"nova-api-db-create-t6q5w\" (UID: \"4741b42c-35a5-407d-b5cc-c7d47b07ce0b\") " pod="openstack-kuttl-tests/nova-api-db-create-t6q5w" Dec 03 12:06:00 crc kubenswrapper[5052]: I1203 12:06:00.913595 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell1-db-create-64bnw"] Dec 03 12:06:00 crc kubenswrapper[5052]: I1203 12:06:00.915490 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-db-create-64bnw" Dec 03 12:06:00 crc kubenswrapper[5052]: I1203 12:06:00.941819 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-db-create-64bnw"] Dec 03 12:06:00 crc kubenswrapper[5052]: I1203 12:06:00.970367 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/c2d35861-9e4b-4f53-b841-e18b26d575cc-operator-scripts\") pod \"nova-cell0-db-create-8kvhf\" (UID: \"c2d35861-9e4b-4f53-b841-e18b26d575cc\") " pod="openstack-kuttl-tests/nova-cell0-db-create-8kvhf" Dec 03 12:06:00 crc kubenswrapper[5052]: I1203 12:06:00.970503 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-blj2x\" (UniqueName: \"kubernetes.io/projected/3af45ac5-7939-4525-891d-8ca1f5e697bf-kube-api-access-blj2x\") pod \"nova-api-48e2-account-create-update-dcqjr\" (UID: \"3af45ac5-7939-4525-891d-8ca1f5e697bf\") " pod="openstack-kuttl-tests/nova-api-48e2-account-create-update-dcqjr" Dec 03 12:06:00 crc kubenswrapper[5052]: I1203 12:06:00.970603 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-smj68\" (UniqueName: \"kubernetes.io/projected/89503fe3-9e65-4405-81f6-45cc9b5db612-kube-api-access-smj68\") pod \"nova-cell1-db-create-64bnw\" (UID: \"89503fe3-9e65-4405-81f6-45cc9b5db612\") " pod="openstack-kuttl-tests/nova-cell1-db-create-64bnw" Dec 03 12:06:00 crc kubenswrapper[5052]: I1203 12:06:00.970670 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-79b27\" (UniqueName: \"kubernetes.io/projected/c2d35861-9e4b-4f53-b841-e18b26d575cc-kube-api-access-79b27\") pod \"nova-cell0-db-create-8kvhf\" (UID: \"c2d35861-9e4b-4f53-b841-e18b26d575cc\") " pod="openstack-kuttl-tests/nova-cell0-db-create-8kvhf" Dec 03 12:06:00 crc kubenswrapper[5052]: I1203 12:06:00.970779 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/89503fe3-9e65-4405-81f6-45cc9b5db612-operator-scripts\") pod \"nova-cell1-db-create-64bnw\" (UID: \"89503fe3-9e65-4405-81f6-45cc9b5db612\") " pod="openstack-kuttl-tests/nova-cell1-db-create-64bnw" Dec 03 12:06:00 crc kubenswrapper[5052]: I1203 12:06:00.970809 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/3af45ac5-7939-4525-891d-8ca1f5e697bf-operator-scripts\") pod \"nova-api-48e2-account-create-update-dcqjr\" (UID: \"3af45ac5-7939-4525-891d-8ca1f5e697bf\") " pod="openstack-kuttl-tests/nova-api-48e2-account-create-update-dcqjr" Dec 03 12:06:00 crc kubenswrapper[5052]: I1203 12:06:00.982955 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:06:01 crc kubenswrapper[5052]: I1203 12:06:01.021608 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell0-d294-account-create-update-tm6pc"] Dec 03 12:06:01 crc kubenswrapper[5052]: E1203 12:06:01.022463 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="ae2d7638-3ed1-4ab4-8d55-8eb8608bc97e" containerName="ceilometer-central-agent" Dec 03 12:06:01 crc kubenswrapper[5052]: I1203 12:06:01.022487 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae2d7638-3ed1-4ab4-8d55-8eb8608bc97e" containerName="ceilometer-central-agent" Dec 03 12:06:01 crc kubenswrapper[5052]: E1203 12:06:01.022535 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="ae2d7638-3ed1-4ab4-8d55-8eb8608bc97e" containerName="sg-core" Dec 03 12:06:01 crc kubenswrapper[5052]: I1203 12:06:01.022542 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae2d7638-3ed1-4ab4-8d55-8eb8608bc97e" containerName="sg-core" Dec 03 12:06:01 crc kubenswrapper[5052]: E1203 12:06:01.022562 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="ae2d7638-3ed1-4ab4-8d55-8eb8608bc97e" containerName="proxy-httpd" Dec 03 12:06:01 crc kubenswrapper[5052]: I1203 12:06:01.022567 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae2d7638-3ed1-4ab4-8d55-8eb8608bc97e" containerName="proxy-httpd" Dec 03 12:06:01 crc kubenswrapper[5052]: E1203 12:06:01.022599 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="ae2d7638-3ed1-4ab4-8d55-8eb8608bc97e" containerName="ceilometer-notification-agent" Dec 03 12:06:01 crc kubenswrapper[5052]: I1203 12:06:01.022605 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae2d7638-3ed1-4ab4-8d55-8eb8608bc97e" containerName="ceilometer-notification-agent" Dec 03 12:06:01 crc kubenswrapper[5052]: I1203 12:06:01.023277 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="ae2d7638-3ed1-4ab4-8d55-8eb8608bc97e" containerName="ceilometer-central-agent" Dec 03 12:06:01 crc kubenswrapper[5052]: I1203 12:06:01.023315 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="ae2d7638-3ed1-4ab4-8d55-8eb8608bc97e" containerName="ceilometer-notification-agent" Dec 03 12:06:01 crc kubenswrapper[5052]: I1203 12:06:01.023326 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="ae2d7638-3ed1-4ab4-8d55-8eb8608bc97e" containerName="sg-core" Dec 03 12:06:01 crc kubenswrapper[5052]: I1203 12:06:01.023342 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="ae2d7638-3ed1-4ab4-8d55-8eb8608bc97e" containerName="proxy-httpd" Dec 03 12:06:01 crc kubenswrapper[5052]: I1203 12:06:01.025382 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-d294-account-create-update-tm6pc" Dec 03 12:06:01 crc kubenswrapper[5052]: I1203 12:06:01.029501 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell0-db-secret" Dec 03 12:06:01 crc kubenswrapper[5052]: I1203 12:06:01.074373 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell0-d294-account-create-update-tm6pc"] Dec 03 12:06:01 crc kubenswrapper[5052]: I1203 12:06:01.079491 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/89503fe3-9e65-4405-81f6-45cc9b5db612-operator-scripts\") pod \"nova-cell1-db-create-64bnw\" (UID: \"89503fe3-9e65-4405-81f6-45cc9b5db612\") " pod="openstack-kuttl-tests/nova-cell1-db-create-64bnw" Dec 03 12:06:01 crc kubenswrapper[5052]: I1203 12:06:01.079595 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/3af45ac5-7939-4525-891d-8ca1f5e697bf-operator-scripts\") pod \"nova-api-48e2-account-create-update-dcqjr\" (UID: \"3af45ac5-7939-4525-891d-8ca1f5e697bf\") " pod="openstack-kuttl-tests/nova-api-48e2-account-create-update-dcqjr" Dec 03 12:06:01 crc kubenswrapper[5052]: I1203 12:06:01.079835 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/c2d35861-9e4b-4f53-b841-e18b26d575cc-operator-scripts\") pod \"nova-cell0-db-create-8kvhf\" (UID: \"c2d35861-9e4b-4f53-b841-e18b26d575cc\") " pod="openstack-kuttl-tests/nova-cell0-db-create-8kvhf" Dec 03 12:06:01 crc kubenswrapper[5052]: I1203 12:06:01.080842 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/3af45ac5-7939-4525-891d-8ca1f5e697bf-operator-scripts\") pod \"nova-api-48e2-account-create-update-dcqjr\" (UID: \"3af45ac5-7939-4525-891d-8ca1f5e697bf\") " pod="openstack-kuttl-tests/nova-api-48e2-account-create-update-dcqjr" Dec 03 12:06:01 crc kubenswrapper[5052]: I1203 12:06:01.081208 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/c2d35861-9e4b-4f53-b841-e18b26d575cc-operator-scripts\") pod \"nova-cell0-db-create-8kvhf\" (UID: \"c2d35861-9e4b-4f53-b841-e18b26d575cc\") " pod="openstack-kuttl-tests/nova-cell0-db-create-8kvhf" Dec 03 12:06:01 crc kubenswrapper[5052]: I1203 12:06:01.081559 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/89503fe3-9e65-4405-81f6-45cc9b5db612-operator-scripts\") pod \"nova-cell1-db-create-64bnw\" (UID: \"89503fe3-9e65-4405-81f6-45cc9b5db612\") " pod="openstack-kuttl-tests/nova-cell1-db-create-64bnw" Dec 03 12:06:01 crc kubenswrapper[5052]: I1203 12:06:01.081648 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-blj2x\" (UniqueName: \"kubernetes.io/projected/3af45ac5-7939-4525-891d-8ca1f5e697bf-kube-api-access-blj2x\") pod \"nova-api-48e2-account-create-update-dcqjr\" (UID: \"3af45ac5-7939-4525-891d-8ca1f5e697bf\") " pod="openstack-kuttl-tests/nova-api-48e2-account-create-update-dcqjr" Dec 03 12:06:01 crc kubenswrapper[5052]: I1203 12:06:01.081744 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-smj68\" (UniqueName: \"kubernetes.io/projected/89503fe3-9e65-4405-81f6-45cc9b5db612-kube-api-access-smj68\") pod \"nova-cell1-db-create-64bnw\" (UID: \"89503fe3-9e65-4405-81f6-45cc9b5db612\") " pod="openstack-kuttl-tests/nova-cell1-db-create-64bnw" Dec 03 12:06:01 crc kubenswrapper[5052]: I1203 12:06:01.081812 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-79b27\" (UniqueName: \"kubernetes.io/projected/c2d35861-9e4b-4f53-b841-e18b26d575cc-kube-api-access-79b27\") pod \"nova-cell0-db-create-8kvhf\" (UID: \"c2d35861-9e4b-4f53-b841-e18b26d575cc\") " pod="openstack-kuttl-tests/nova-cell0-db-create-8kvhf" Dec 03 12:06:01 crc kubenswrapper[5052]: I1203 12:06:01.090951 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-db-create-t6q5w" Dec 03 12:06:01 crc kubenswrapper[5052]: I1203 12:06:01.109186 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-blj2x\" (UniqueName: \"kubernetes.io/projected/3af45ac5-7939-4525-891d-8ca1f5e697bf-kube-api-access-blj2x\") pod \"nova-api-48e2-account-create-update-dcqjr\" (UID: \"3af45ac5-7939-4525-891d-8ca1f5e697bf\") " pod="openstack-kuttl-tests/nova-api-48e2-account-create-update-dcqjr" Dec 03 12:06:01 crc kubenswrapper[5052]: I1203 12:06:01.115810 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-smj68\" (UniqueName: \"kubernetes.io/projected/89503fe3-9e65-4405-81f6-45cc9b5db612-kube-api-access-smj68\") pod \"nova-cell1-db-create-64bnw\" (UID: \"89503fe3-9e65-4405-81f6-45cc9b5db612\") " pod="openstack-kuttl-tests/nova-cell1-db-create-64bnw" Dec 03 12:06:01 crc kubenswrapper[5052]: I1203 12:06:01.117427 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-79b27\" (UniqueName: \"kubernetes.io/projected/c2d35861-9e4b-4f53-b841-e18b26d575cc-kube-api-access-79b27\") pod \"nova-cell0-db-create-8kvhf\" (UID: \"c2d35861-9e4b-4f53-b841-e18b26d575cc\") " pod="openstack-kuttl-tests/nova-cell0-db-create-8kvhf" Dec 03 12:06:01 crc kubenswrapper[5052]: I1203 12:06:01.142866 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-db-create-8kvhf" Dec 03 12:06:01 crc kubenswrapper[5052]: I1203 12:06:01.185336 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6d8kl\" (UniqueName: \"kubernetes.io/projected/ae2d7638-3ed1-4ab4-8d55-8eb8608bc97e-kube-api-access-6d8kl\") pod \"ae2d7638-3ed1-4ab4-8d55-8eb8608bc97e\" (UID: \"ae2d7638-3ed1-4ab4-8d55-8eb8608bc97e\") " Dec 03 12:06:01 crc kubenswrapper[5052]: I1203 12:06:01.185607 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/ae2d7638-3ed1-4ab4-8d55-8eb8608bc97e-sg-core-conf-yaml\") pod \"ae2d7638-3ed1-4ab4-8d55-8eb8608bc97e\" (UID: \"ae2d7638-3ed1-4ab4-8d55-8eb8608bc97e\") " Dec 03 12:06:01 crc kubenswrapper[5052]: I1203 12:06:01.185691 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ae2d7638-3ed1-4ab4-8d55-8eb8608bc97e-combined-ca-bundle\") pod \"ae2d7638-3ed1-4ab4-8d55-8eb8608bc97e\" (UID: \"ae2d7638-3ed1-4ab4-8d55-8eb8608bc97e\") " Dec 03 12:06:01 crc kubenswrapper[5052]: I1203 12:06:01.185735 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/ae2d7638-3ed1-4ab4-8d55-8eb8608bc97e-scripts\") pod \"ae2d7638-3ed1-4ab4-8d55-8eb8608bc97e\" (UID: \"ae2d7638-3ed1-4ab4-8d55-8eb8608bc97e\") " Dec 03 12:06:01 crc kubenswrapper[5052]: I1203 12:06:01.185762 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/ae2d7638-3ed1-4ab4-8d55-8eb8608bc97e-run-httpd\") pod \"ae2d7638-3ed1-4ab4-8d55-8eb8608bc97e\" (UID: \"ae2d7638-3ed1-4ab4-8d55-8eb8608bc97e\") " Dec 03 12:06:01 crc kubenswrapper[5052]: I1203 12:06:01.185798 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/ae2d7638-3ed1-4ab4-8d55-8eb8608bc97e-log-httpd\") pod \"ae2d7638-3ed1-4ab4-8d55-8eb8608bc97e\" (UID: \"ae2d7638-3ed1-4ab4-8d55-8eb8608bc97e\") " Dec 03 12:06:01 crc kubenswrapper[5052]: I1203 12:06:01.185877 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/ae2d7638-3ed1-4ab4-8d55-8eb8608bc97e-config-data\") pod \"ae2d7638-3ed1-4ab4-8d55-8eb8608bc97e\" (UID: \"ae2d7638-3ed1-4ab4-8d55-8eb8608bc97e\") " Dec 03 12:06:01 crc kubenswrapper[5052]: I1203 12:06:01.186165 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-68hkn\" (UniqueName: \"kubernetes.io/projected/53e80028-95b5-48c8-96ce-f6e40ed916cf-kube-api-access-68hkn\") pod \"nova-cell0-d294-account-create-update-tm6pc\" (UID: \"53e80028-95b5-48c8-96ce-f6e40ed916cf\") " pod="openstack-kuttl-tests/nova-cell0-d294-account-create-update-tm6pc" Dec 03 12:06:01 crc kubenswrapper[5052]: I1203 12:06:01.186282 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/53e80028-95b5-48c8-96ce-f6e40ed916cf-operator-scripts\") pod \"nova-cell0-d294-account-create-update-tm6pc\" (UID: \"53e80028-95b5-48c8-96ce-f6e40ed916cf\") " pod="openstack-kuttl-tests/nova-cell0-d294-account-create-update-tm6pc" Dec 03 12:06:01 crc kubenswrapper[5052]: I1203 12:06:01.187111 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae2d7638-3ed1-4ab4-8d55-8eb8608bc97e-log-httpd" (OuterVolumeSpecName: "log-httpd") pod "ae2d7638-3ed1-4ab4-8d55-8eb8608bc97e" (UID: "ae2d7638-3ed1-4ab4-8d55-8eb8608bc97e"). InnerVolumeSpecName "log-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:06:01 crc kubenswrapper[5052]: I1203 12:06:01.189428 5052 reconciler_common.go:293] "Volume detached for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/ae2d7638-3ed1-4ab4-8d55-8eb8608bc97e-log-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:06:01 crc kubenswrapper[5052]: I1203 12:06:01.190001 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae2d7638-3ed1-4ab4-8d55-8eb8608bc97e-run-httpd" (OuterVolumeSpecName: "run-httpd") pod "ae2d7638-3ed1-4ab4-8d55-8eb8608bc97e" (UID: "ae2d7638-3ed1-4ab4-8d55-8eb8608bc97e"). InnerVolumeSpecName "run-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:06:01 crc kubenswrapper[5052]: I1203 12:06:01.192628 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ae2d7638-3ed1-4ab4-8d55-8eb8608bc97e-scripts" (OuterVolumeSpecName: "scripts") pod "ae2d7638-3ed1-4ab4-8d55-8eb8608bc97e" (UID: "ae2d7638-3ed1-4ab4-8d55-8eb8608bc97e"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:06:01 crc kubenswrapper[5052]: I1203 12:06:01.225802 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ae2d7638-3ed1-4ab4-8d55-8eb8608bc97e-kube-api-access-6d8kl" (OuterVolumeSpecName: "kube-api-access-6d8kl") pod "ae2d7638-3ed1-4ab4-8d55-8eb8608bc97e" (UID: "ae2d7638-3ed1-4ab4-8d55-8eb8608bc97e"). InnerVolumeSpecName "kube-api-access-6d8kl". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:06:01 crc kubenswrapper[5052]: I1203 12:06:01.252295 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ae2d7638-3ed1-4ab4-8d55-8eb8608bc97e-sg-core-conf-yaml" (OuterVolumeSpecName: "sg-core-conf-yaml") pod "ae2d7638-3ed1-4ab4-8d55-8eb8608bc97e" (UID: "ae2d7638-3ed1-4ab4-8d55-8eb8608bc97e"). InnerVolumeSpecName "sg-core-conf-yaml". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:06:01 crc kubenswrapper[5052]: I1203 12:06:01.261253 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell1-c2b9-account-create-update-hlm9w"] Dec 03 12:06:01 crc kubenswrapper[5052]: I1203 12:06:01.262819 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-c2b9-account-create-update-hlm9w" Dec 03 12:06:01 crc kubenswrapper[5052]: I1203 12:06:01.277280 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell1-db-secret" Dec 03 12:06:01 crc kubenswrapper[5052]: I1203 12:06:01.280363 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-c2b9-account-create-update-hlm9w"] Dec 03 12:06:01 crc kubenswrapper[5052]: I1203 12:06:01.282891 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-48e2-account-create-update-dcqjr" Dec 03 12:06:01 crc kubenswrapper[5052]: I1203 12:06:01.304427 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-db-create-64bnw" Dec 03 12:06:01 crc kubenswrapper[5052]: I1203 12:06:01.306313 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/53e80028-95b5-48c8-96ce-f6e40ed916cf-operator-scripts\") pod \"nova-cell0-d294-account-create-update-tm6pc\" (UID: \"53e80028-95b5-48c8-96ce-f6e40ed916cf\") " pod="openstack-kuttl-tests/nova-cell0-d294-account-create-update-tm6pc" Dec 03 12:06:01 crc kubenswrapper[5052]: I1203 12:06:01.306399 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/fdc5eec2-010f-4de6-a578-e5bda587d218-operator-scripts\") pod \"nova-cell1-c2b9-account-create-update-hlm9w\" (UID: \"fdc5eec2-010f-4de6-a578-e5bda587d218\") " pod="openstack-kuttl-tests/nova-cell1-c2b9-account-create-update-hlm9w" Dec 03 12:06:01 crc kubenswrapper[5052]: I1203 12:06:01.308021 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-68hkn\" (UniqueName: \"kubernetes.io/projected/53e80028-95b5-48c8-96ce-f6e40ed916cf-kube-api-access-68hkn\") pod \"nova-cell0-d294-account-create-update-tm6pc\" (UID: \"53e80028-95b5-48c8-96ce-f6e40ed916cf\") " pod="openstack-kuttl-tests/nova-cell0-d294-account-create-update-tm6pc" Dec 03 12:06:01 crc kubenswrapper[5052]: I1203 12:06:01.308208 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/53e80028-95b5-48c8-96ce-f6e40ed916cf-operator-scripts\") pod \"nova-cell0-d294-account-create-update-tm6pc\" (UID: \"53e80028-95b5-48c8-96ce-f6e40ed916cf\") " pod="openstack-kuttl-tests/nova-cell0-d294-account-create-update-tm6pc" Dec 03 12:06:01 crc kubenswrapper[5052]: I1203 12:06:01.308255 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b2v2r\" (UniqueName: \"kubernetes.io/projected/fdc5eec2-010f-4de6-a578-e5bda587d218-kube-api-access-b2v2r\") pod \"nova-cell1-c2b9-account-create-update-hlm9w\" (UID: \"fdc5eec2-010f-4de6-a578-e5bda587d218\") " pod="openstack-kuttl-tests/nova-cell1-c2b9-account-create-update-hlm9w" Dec 03 12:06:01 crc kubenswrapper[5052]: I1203 12:06:01.308716 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/ae2d7638-3ed1-4ab4-8d55-8eb8608bc97e-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:06:01 crc kubenswrapper[5052]: I1203 12:06:01.308803 5052 reconciler_common.go:293] "Volume detached for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/ae2d7638-3ed1-4ab4-8d55-8eb8608bc97e-run-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:06:01 crc kubenswrapper[5052]: I1203 12:06:01.308872 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-6d8kl\" (UniqueName: \"kubernetes.io/projected/ae2d7638-3ed1-4ab4-8d55-8eb8608bc97e-kube-api-access-6d8kl\") on node \"crc\" DevicePath \"\"" Dec 03 12:06:01 crc kubenswrapper[5052]: I1203 12:06:01.308935 5052 reconciler_common.go:293] "Volume detached for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/ae2d7638-3ed1-4ab4-8d55-8eb8608bc97e-sg-core-conf-yaml\") on node \"crc\" DevicePath \"\"" Dec 03 12:06:01 crc kubenswrapper[5052]: I1203 12:06:01.327148 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-68hkn\" (UniqueName: \"kubernetes.io/projected/53e80028-95b5-48c8-96ce-f6e40ed916cf-kube-api-access-68hkn\") pod \"nova-cell0-d294-account-create-update-tm6pc\" (UID: \"53e80028-95b5-48c8-96ce-f6e40ed916cf\") " pod="openstack-kuttl-tests/nova-cell0-d294-account-create-update-tm6pc" Dec 03 12:06:01 crc kubenswrapper[5052]: I1203 12:06:01.362658 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ae2d7638-3ed1-4ab4-8d55-8eb8608bc97e-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "ae2d7638-3ed1-4ab4-8d55-8eb8608bc97e" (UID: "ae2d7638-3ed1-4ab4-8d55-8eb8608bc97e"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:06:01 crc kubenswrapper[5052]: I1203 12:06:01.368858 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-d294-account-create-update-tm6pc" Dec 03 12:06:01 crc kubenswrapper[5052]: I1203 12:06:01.415129 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-b2v2r\" (UniqueName: \"kubernetes.io/projected/fdc5eec2-010f-4de6-a578-e5bda587d218-kube-api-access-b2v2r\") pod \"nova-cell1-c2b9-account-create-update-hlm9w\" (UID: \"fdc5eec2-010f-4de6-a578-e5bda587d218\") " pod="openstack-kuttl-tests/nova-cell1-c2b9-account-create-update-hlm9w" Dec 03 12:06:01 crc kubenswrapper[5052]: I1203 12:06:01.416706 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/fdc5eec2-010f-4de6-a578-e5bda587d218-operator-scripts\") pod \"nova-cell1-c2b9-account-create-update-hlm9w\" (UID: \"fdc5eec2-010f-4de6-a578-e5bda587d218\") " pod="openstack-kuttl-tests/nova-cell1-c2b9-account-create-update-hlm9w" Dec 03 12:06:01 crc kubenswrapper[5052]: I1203 12:06:01.416837 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ae2d7638-3ed1-4ab4-8d55-8eb8608bc97e-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:06:01 crc kubenswrapper[5052]: I1203 12:06:01.418074 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/fdc5eec2-010f-4de6-a578-e5bda587d218-operator-scripts\") pod \"nova-cell1-c2b9-account-create-update-hlm9w\" (UID: \"fdc5eec2-010f-4de6-a578-e5bda587d218\") " pod="openstack-kuttl-tests/nova-cell1-c2b9-account-create-update-hlm9w" Dec 03 12:06:01 crc kubenswrapper[5052]: I1203 12:06:01.436918 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-b2v2r\" (UniqueName: \"kubernetes.io/projected/fdc5eec2-010f-4de6-a578-e5bda587d218-kube-api-access-b2v2r\") pod \"nova-cell1-c2b9-account-create-update-hlm9w\" (UID: \"fdc5eec2-010f-4de6-a578-e5bda587d218\") " pod="openstack-kuttl-tests/nova-cell1-c2b9-account-create-update-hlm9w" Dec 03 12:06:01 crc kubenswrapper[5052]: I1203 12:06:01.471890 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ae2d7638-3ed1-4ab4-8d55-8eb8608bc97e-config-data" (OuterVolumeSpecName: "config-data") pod "ae2d7638-3ed1-4ab4-8d55-8eb8608bc97e" (UID: "ae2d7638-3ed1-4ab4-8d55-8eb8608bc97e"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:06:01 crc kubenswrapper[5052]: I1203 12:06:01.518666 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/ae2d7638-3ed1-4ab4-8d55-8eb8608bc97e-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:06:01 crc kubenswrapper[5052]: I1203 12:06:01.555249 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-c2b9-account-create-update-hlm9w" Dec 03 12:06:01 crc kubenswrapper[5052]: I1203 12:06:01.657870 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-db-create-t6q5w"] Dec 03 12:06:01 crc kubenswrapper[5052]: W1203 12:06:01.664659 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4741b42c_35a5_407d_b5cc_c7d47b07ce0b.slice/crio-a3ecfc69fe7556045bc11cfa634434b088b5005e80dc3e662b4531ed46b6b4a7 WatchSource:0}: Error finding container a3ecfc69fe7556045bc11cfa634434b088b5005e80dc3e662b4531ed46b6b4a7: Status 404 returned error can't find the container with id a3ecfc69fe7556045bc11cfa634434b088b5005e80dc3e662b4531ed46b6b4a7 Dec 03 12:06:01 crc kubenswrapper[5052]: I1203 12:06:01.670213 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"ae2d7638-3ed1-4ab4-8d55-8eb8608bc97e","Type":"ContainerDied","Data":"5d95d1e48905db091a08b53cc676eb9b778282ad0e6bd81957f6f330de69f1c8"} Dec 03 12:06:01 crc kubenswrapper[5052]: I1203 12:06:01.670466 5052 scope.go:117] "RemoveContainer" containerID="cb3e22986020c181b5ae6726aae34a38e1bcb7a291bf69547b51d5917d2e744c" Dec 03 12:06:01 crc kubenswrapper[5052]: I1203 12:06:01.670342 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:06:01 crc kubenswrapper[5052]: I1203 12:06:01.733450 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:06:01 crc kubenswrapper[5052]: I1203 12:06:01.760023 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:06:01 crc kubenswrapper[5052]: I1203 12:06:01.775337 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:06:01 crc kubenswrapper[5052]: I1203 12:06:01.778810 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:06:01 crc kubenswrapper[5052]: I1203 12:06:01.779225 5052 scope.go:117] "RemoveContainer" containerID="e90e2d7c78b50d4ab2e506730826ae6cbc28fbf2dc044144c3b2d062e11c08c5" Dec 03 12:06:01 crc kubenswrapper[5052]: I1203 12:06:01.783611 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-scripts" Dec 03 12:06:01 crc kubenswrapper[5052]: I1203 12:06:01.785919 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-config-data" Dec 03 12:06:01 crc kubenswrapper[5052]: I1203 12:06:01.794090 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:06:01 crc kubenswrapper[5052]: I1203 12:06:01.823297 5052 scope.go:117] "RemoveContainer" containerID="c1b2b526b75e2f6e7dba586cd66a185e3baf50dddd08f9f766d7c67ae4aab2dc" Dec 03 12:06:01 crc kubenswrapper[5052]: I1203 12:06:01.832895 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/cf64fec7-c528-4a2c-89b4-a43851b7a903-log-httpd\") pod \"ceilometer-0\" (UID: \"cf64fec7-c528-4a2c-89b4-a43851b7a903\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:06:01 crc kubenswrapper[5052]: I1203 12:06:01.832958 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/cf64fec7-c528-4a2c-89b4-a43851b7a903-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"cf64fec7-c528-4a2c-89b4-a43851b7a903\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:06:01 crc kubenswrapper[5052]: I1203 12:06:01.833006 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/cf64fec7-c528-4a2c-89b4-a43851b7a903-run-httpd\") pod \"ceilometer-0\" (UID: \"cf64fec7-c528-4a2c-89b4-a43851b7a903\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:06:01 crc kubenswrapper[5052]: I1203 12:06:01.833191 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/cf64fec7-c528-4a2c-89b4-a43851b7a903-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"cf64fec7-c528-4a2c-89b4-a43851b7a903\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:06:01 crc kubenswrapper[5052]: I1203 12:06:01.833266 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tlr44\" (UniqueName: \"kubernetes.io/projected/cf64fec7-c528-4a2c-89b4-a43851b7a903-kube-api-access-tlr44\") pod \"ceilometer-0\" (UID: \"cf64fec7-c528-4a2c-89b4-a43851b7a903\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:06:01 crc kubenswrapper[5052]: I1203 12:06:01.833657 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/cf64fec7-c528-4a2c-89b4-a43851b7a903-config-data\") pod \"ceilometer-0\" (UID: \"cf64fec7-c528-4a2c-89b4-a43851b7a903\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:06:01 crc kubenswrapper[5052]: I1203 12:06:01.833691 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/cf64fec7-c528-4a2c-89b4-a43851b7a903-scripts\") pod \"ceilometer-0\" (UID: \"cf64fec7-c528-4a2c-89b4-a43851b7a903\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:06:01 crc kubenswrapper[5052]: I1203 12:06:01.833869 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell0-db-create-8kvhf"] Dec 03 12:06:01 crc kubenswrapper[5052]: I1203 12:06:01.860211 5052 scope.go:117] "RemoveContainer" containerID="aa090b5d18b872f74a8114d8dca0ba1dcc3cf87025d6bf6e190621c1bc9960c4" Dec 03 12:06:01 crc kubenswrapper[5052]: I1203 12:06:01.928440 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-48e2-account-create-update-dcqjr"] Dec 03 12:06:01 crc kubenswrapper[5052]: I1203 12:06:01.938696 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/cf64fec7-c528-4a2c-89b4-a43851b7a903-log-httpd\") pod \"ceilometer-0\" (UID: \"cf64fec7-c528-4a2c-89b4-a43851b7a903\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:06:01 crc kubenswrapper[5052]: I1203 12:06:01.938753 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/cf64fec7-c528-4a2c-89b4-a43851b7a903-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"cf64fec7-c528-4a2c-89b4-a43851b7a903\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:06:01 crc kubenswrapper[5052]: I1203 12:06:01.938785 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/cf64fec7-c528-4a2c-89b4-a43851b7a903-run-httpd\") pod \"ceilometer-0\" (UID: \"cf64fec7-c528-4a2c-89b4-a43851b7a903\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:06:01 crc kubenswrapper[5052]: I1203 12:06:01.938842 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/cf64fec7-c528-4a2c-89b4-a43851b7a903-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"cf64fec7-c528-4a2c-89b4-a43851b7a903\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:06:01 crc kubenswrapper[5052]: I1203 12:06:01.938866 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-tlr44\" (UniqueName: \"kubernetes.io/projected/cf64fec7-c528-4a2c-89b4-a43851b7a903-kube-api-access-tlr44\") pod \"ceilometer-0\" (UID: \"cf64fec7-c528-4a2c-89b4-a43851b7a903\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:06:01 crc kubenswrapper[5052]: I1203 12:06:01.939003 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/cf64fec7-c528-4a2c-89b4-a43851b7a903-config-data\") pod \"ceilometer-0\" (UID: \"cf64fec7-c528-4a2c-89b4-a43851b7a903\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:06:01 crc kubenswrapper[5052]: I1203 12:06:01.939022 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/cf64fec7-c528-4a2c-89b4-a43851b7a903-scripts\") pod \"ceilometer-0\" (UID: \"cf64fec7-c528-4a2c-89b4-a43851b7a903\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:06:01 crc kubenswrapper[5052]: I1203 12:06:01.939199 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/cf64fec7-c528-4a2c-89b4-a43851b7a903-log-httpd\") pod \"ceilometer-0\" (UID: \"cf64fec7-c528-4a2c-89b4-a43851b7a903\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:06:01 crc kubenswrapper[5052]: I1203 12:06:01.940077 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/cf64fec7-c528-4a2c-89b4-a43851b7a903-run-httpd\") pod \"ceilometer-0\" (UID: \"cf64fec7-c528-4a2c-89b4-a43851b7a903\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:06:01 crc kubenswrapper[5052]: I1203 12:06:01.943379 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-db-create-64bnw"] Dec 03 12:06:01 crc kubenswrapper[5052]: I1203 12:06:01.959008 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/cf64fec7-c528-4a2c-89b4-a43851b7a903-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"cf64fec7-c528-4a2c-89b4-a43851b7a903\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:06:01 crc kubenswrapper[5052]: I1203 12:06:01.959232 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/cf64fec7-c528-4a2c-89b4-a43851b7a903-config-data\") pod \"ceilometer-0\" (UID: \"cf64fec7-c528-4a2c-89b4-a43851b7a903\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:06:01 crc kubenswrapper[5052]: I1203 12:06:01.960411 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/cf64fec7-c528-4a2c-89b4-a43851b7a903-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"cf64fec7-c528-4a2c-89b4-a43851b7a903\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:06:01 crc kubenswrapper[5052]: I1203 12:06:01.966736 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/cf64fec7-c528-4a2c-89b4-a43851b7a903-scripts\") pod \"ceilometer-0\" (UID: \"cf64fec7-c528-4a2c-89b4-a43851b7a903\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:06:01 crc kubenswrapper[5052]: I1203 12:06:01.969853 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-tlr44\" (UniqueName: \"kubernetes.io/projected/cf64fec7-c528-4a2c-89b4-a43851b7a903-kube-api-access-tlr44\") pod \"ceilometer-0\" (UID: \"cf64fec7-c528-4a2c-89b4-a43851b7a903\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:06:02 crc kubenswrapper[5052]: I1203 12:06:02.087593 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell0-d294-account-create-update-tm6pc"] Dec 03 12:06:02 crc kubenswrapper[5052]: I1203 12:06:02.114791 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:06:02 crc kubenswrapper[5052]: W1203 12:06:02.143075 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod53e80028_95b5_48c8_96ce_f6e40ed916cf.slice/crio-36f0039208eefa40a1e2cff26434aa005d86ce83fbff4e16b3d7771b8044306b WatchSource:0}: Error finding container 36f0039208eefa40a1e2cff26434aa005d86ce83fbff4e16b3d7771b8044306b: Status 404 returned error can't find the container with id 36f0039208eefa40a1e2cff26434aa005d86ce83fbff4e16b3d7771b8044306b Dec 03 12:06:02 crc kubenswrapper[5052]: I1203 12:06:02.195773 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-c2b9-account-create-update-hlm9w"] Dec 03 12:06:02 crc kubenswrapper[5052]: W1203 12:06:02.208416 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfdc5eec2_010f_4de6_a578_e5bda587d218.slice/crio-1b1bc7e9b4a8a31272dab226d5ac1f8c188f27c696324074fead8869af564364 WatchSource:0}: Error finding container 1b1bc7e9b4a8a31272dab226d5ac1f8c188f27c696324074fead8869af564364: Status 404 returned error can't find the container with id 1b1bc7e9b4a8a31272dab226d5ac1f8c188f27c696324074fead8869af564364 Dec 03 12:06:02 crc kubenswrapper[5052]: I1203 12:06:02.820840 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:06:02 crc kubenswrapper[5052]: I1203 12:06:02.826507 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-c2b9-account-create-update-hlm9w" event={"ID":"fdc5eec2-010f-4de6-a578-e5bda587d218","Type":"ContainerStarted","Data":"ac697ab7fae2d375f75e93f50ee6034e1e0bf8653a46e8447628f29f9a3cceb3"} Dec 03 12:06:02 crc kubenswrapper[5052]: I1203 12:06:02.826695 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-c2b9-account-create-update-hlm9w" event={"ID":"fdc5eec2-010f-4de6-a578-e5bda587d218","Type":"ContainerStarted","Data":"1b1bc7e9b4a8a31272dab226d5ac1f8c188f27c696324074fead8869af564364"} Dec 03 12:06:02 crc kubenswrapper[5052]: I1203 12:06:02.839376 5052 generic.go:334] "Generic (PLEG): container finished" podID="d2588f8c-9004-4e6e-8dd5-c6234b94d5cf" containerID="65e485c8f05b257462566a96517913bed283ca6386a0edf4f3a7a2023ee080ce" exitCode=0 Dec 03 12:06:02 crc kubenswrapper[5052]: I1203 12:06:02.839441 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"d2588f8c-9004-4e6e-8dd5-c6234b94d5cf","Type":"ContainerDied","Data":"65e485c8f05b257462566a96517913bed283ca6386a0edf4f3a7a2023ee080ce"} Dec 03 12:06:02 crc kubenswrapper[5052]: I1203 12:06:02.839474 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"d2588f8c-9004-4e6e-8dd5-c6234b94d5cf","Type":"ContainerDied","Data":"61285e831813c4ec6f3caeed2d0fd767b4b6eff68d9c716bf1c7a2d9fc0b0532"} Dec 03 12:06:02 crc kubenswrapper[5052]: I1203 12:06:02.839494 5052 scope.go:117] "RemoveContainer" containerID="65e485c8f05b257462566a96517913bed283ca6386a0edf4f3a7a2023ee080ce" Dec 03 12:06:02 crc kubenswrapper[5052]: I1203 12:06:02.839674 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:06:02 crc kubenswrapper[5052]: I1203 12:06:02.844267 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:06:02 crc kubenswrapper[5052]: I1203 12:06:02.853828 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-db-create-8kvhf" event={"ID":"c2d35861-9e4b-4f53-b841-e18b26d575cc","Type":"ContainerStarted","Data":"4d472d46d5c7f6ca91aa5161d730340a3e8abead48886bdc090952991731d00a"} Dec 03 12:06:02 crc kubenswrapper[5052]: I1203 12:06:02.853883 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-db-create-8kvhf" event={"ID":"c2d35861-9e4b-4f53-b841-e18b26d575cc","Type":"ContainerStarted","Data":"f21bab82d016786012ffb7e503fe357492f649d072f86fdfa425c16e420f37a6"} Dec 03 12:06:02 crc kubenswrapper[5052]: I1203 12:06:02.873184 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-db-create-64bnw" event={"ID":"89503fe3-9e65-4405-81f6-45cc9b5db612","Type":"ContainerStarted","Data":"9078309d7e958fc36e931932ff0699b628d5fd0921b34acd1cd89a0efd5e8691"} Dec 03 12:06:02 crc kubenswrapper[5052]: I1203 12:06:02.873231 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-db-create-64bnw" event={"ID":"89503fe3-9e65-4405-81f6-45cc9b5db612","Type":"ContainerStarted","Data":"f1fdd2de1c350b5dc60f5fda317c4eb8cc216911f9e0228bbb129f4ade297bdc"} Dec 03 12:06:02 crc kubenswrapper[5052]: I1203 12:06:02.909647 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-d294-account-create-update-tm6pc" event={"ID":"53e80028-95b5-48c8-96ce-f6e40ed916cf","Type":"ContainerStarted","Data":"8b708b7d5d49769665548fc88668b1958dadb43740e66dff74107f13b9ce7ae6"} Dec 03 12:06:02 crc kubenswrapper[5052]: I1203 12:06:02.909730 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-d294-account-create-update-tm6pc" event={"ID":"53e80028-95b5-48c8-96ce-f6e40ed916cf","Type":"ContainerStarted","Data":"36f0039208eefa40a1e2cff26434aa005d86ce83fbff4e16b3d7771b8044306b"} Dec 03 12:06:02 crc kubenswrapper[5052]: I1203 12:06:02.914189 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-db-create-t6q5w" event={"ID":"4741b42c-35a5-407d-b5cc-c7d47b07ce0b","Type":"ContainerStarted","Data":"f68f634ddf0cc7c513c585883315d968fce2a8f67935dd2746313bd6d94c035d"} Dec 03 12:06:02 crc kubenswrapper[5052]: I1203 12:06:02.914242 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-db-create-t6q5w" event={"ID":"4741b42c-35a5-407d-b5cc-c7d47b07ce0b","Type":"ContainerStarted","Data":"a3ecfc69fe7556045bc11cfa634434b088b5005e80dc3e662b4531ed46b6b4a7"} Dec 03 12:06:02 crc kubenswrapper[5052]: I1203 12:06:02.922640 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-48e2-account-create-update-dcqjr" event={"ID":"3af45ac5-7939-4525-891d-8ca1f5e697bf","Type":"ContainerStarted","Data":"e1903fe93354f2eb6210febadfc9a242d90b8681d1287d692e162cc3efd0b892"} Dec 03 12:06:02 crc kubenswrapper[5052]: I1203 12:06:02.922700 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-48e2-account-create-update-dcqjr" event={"ID":"3af45ac5-7939-4525-891d-8ca1f5e697bf","Type":"ContainerStarted","Data":"82043230ef7ae8dd1dd50f6f5b878d8b8863c2d06a0888d357f8a283dff185da"} Dec 03 12:06:02 crc kubenswrapper[5052]: I1203 12:06:02.933151 5052 scope.go:117] "RemoveContainer" containerID="71f1cba19f2d1076cbc88a6c4acbeb2179613b427473cbf3c46cdde47f7e75b5" Dec 03 12:06:02 crc kubenswrapper[5052]: I1203 12:06:02.936650 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-cell1-c2b9-account-create-update-hlm9w" podStartSLOduration=1.936624546 podStartE2EDuration="1.936624546s" podCreationTimestamp="2025-12-03 12:06:01 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:06:02.880234314 +0000 UTC m=+2753.468476078" watchObservedRunningTime="2025-12-03 12:06:02.936624546 +0000 UTC m=+2753.524866310" Dec 03 12:06:02 crc kubenswrapper[5052]: I1203 12:06:02.955503 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-cell0-db-create-8kvhf" podStartSLOduration=2.955475995 podStartE2EDuration="2.955475995s" podCreationTimestamp="2025-12-03 12:06:00 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:06:02.903666484 +0000 UTC m=+2753.491908258" watchObservedRunningTime="2025-12-03 12:06:02.955475995 +0000 UTC m=+2753.543717759" Dec 03 12:06:03 crc kubenswrapper[5052]: I1203 12:06:03.007662 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/d2588f8c-9004-4e6e-8dd5-c6234b94d5cf-logs\") pod \"d2588f8c-9004-4e6e-8dd5-c6234b94d5cf\" (UID: \"d2588f8c-9004-4e6e-8dd5-c6234b94d5cf\") " Dec 03 12:06:03 crc kubenswrapper[5052]: I1203 12:06:03.007732 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d2588f8c-9004-4e6e-8dd5-c6234b94d5cf-combined-ca-bundle\") pod \"d2588f8c-9004-4e6e-8dd5-c6234b94d5cf\" (UID: \"d2588f8c-9004-4e6e-8dd5-c6234b94d5cf\") " Dec 03 12:06:03 crc kubenswrapper[5052]: I1203 12:06:03.007765 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vrc6m\" (UniqueName: \"kubernetes.io/projected/d2588f8c-9004-4e6e-8dd5-c6234b94d5cf-kube-api-access-vrc6m\") pod \"d2588f8c-9004-4e6e-8dd5-c6234b94d5cf\" (UID: \"d2588f8c-9004-4e6e-8dd5-c6234b94d5cf\") " Dec 03 12:06:03 crc kubenswrapper[5052]: I1203 12:06:03.007802 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"glance\" (UniqueName: \"kubernetes.io/local-volume/local-storage11-crc\") pod \"d2588f8c-9004-4e6e-8dd5-c6234b94d5cf\" (UID: \"d2588f8c-9004-4e6e-8dd5-c6234b94d5cf\") " Dec 03 12:06:03 crc kubenswrapper[5052]: I1203 12:06:03.007889 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/d2588f8c-9004-4e6e-8dd5-c6234b94d5cf-public-tls-certs\") pod \"d2588f8c-9004-4e6e-8dd5-c6234b94d5cf\" (UID: \"d2588f8c-9004-4e6e-8dd5-c6234b94d5cf\") " Dec 03 12:06:03 crc kubenswrapper[5052]: I1203 12:06:03.007910 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/d2588f8c-9004-4e6e-8dd5-c6234b94d5cf-scripts\") pod \"d2588f8c-9004-4e6e-8dd5-c6234b94d5cf\" (UID: \"d2588f8c-9004-4e6e-8dd5-c6234b94d5cf\") " Dec 03 12:06:03 crc kubenswrapper[5052]: I1203 12:06:03.008072 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/d2588f8c-9004-4e6e-8dd5-c6234b94d5cf-httpd-run\") pod \"d2588f8c-9004-4e6e-8dd5-c6234b94d5cf\" (UID: \"d2588f8c-9004-4e6e-8dd5-c6234b94d5cf\") " Dec 03 12:06:03 crc kubenswrapper[5052]: I1203 12:06:03.008092 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/d2588f8c-9004-4e6e-8dd5-c6234b94d5cf-config-data\") pod \"d2588f8c-9004-4e6e-8dd5-c6234b94d5cf\" (UID: \"d2588f8c-9004-4e6e-8dd5-c6234b94d5cf\") " Dec 03 12:06:03 crc kubenswrapper[5052]: I1203 12:06:03.008308 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-cell0-d294-account-create-update-tm6pc" podStartSLOduration=3.008287735 podStartE2EDuration="3.008287735s" podCreationTimestamp="2025-12-03 12:06:00 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:06:02.960400816 +0000 UTC m=+2753.548642590" watchObservedRunningTime="2025-12-03 12:06:03.008287735 +0000 UTC m=+2753.596529489" Dec 03 12:06:03 crc kubenswrapper[5052]: I1203 12:06:03.009950 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d2588f8c-9004-4e6e-8dd5-c6234b94d5cf-logs" (OuterVolumeSpecName: "logs") pod "d2588f8c-9004-4e6e-8dd5-c6234b94d5cf" (UID: "d2588f8c-9004-4e6e-8dd5-c6234b94d5cf"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:06:03 crc kubenswrapper[5052]: I1203 12:06:03.012644 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d2588f8c-9004-4e6e-8dd5-c6234b94d5cf-httpd-run" (OuterVolumeSpecName: "httpd-run") pod "d2588f8c-9004-4e6e-8dd5-c6234b94d5cf" (UID: "d2588f8c-9004-4e6e-8dd5-c6234b94d5cf"). InnerVolumeSpecName "httpd-run". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:06:03 crc kubenswrapper[5052]: I1203 12:06:03.015535 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-api-db-create-t6q5w" podStartSLOduration=3.015498931 podStartE2EDuration="3.015498931s" podCreationTimestamp="2025-12-03 12:06:00 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:06:02.980686846 +0000 UTC m=+2753.568928610" watchObservedRunningTime="2025-12-03 12:06:03.015498931 +0000 UTC m=+2753.603740695" Dec 03 12:06:03 crc kubenswrapper[5052]: I1203 12:06:03.017073 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d2588f8c-9004-4e6e-8dd5-c6234b94d5cf-scripts" (OuterVolumeSpecName: "scripts") pod "d2588f8c-9004-4e6e-8dd5-c6234b94d5cf" (UID: "d2588f8c-9004-4e6e-8dd5-c6234b94d5cf"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:06:03 crc kubenswrapper[5052]: I1203 12:06:03.017981 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d2588f8c-9004-4e6e-8dd5-c6234b94d5cf-kube-api-access-vrc6m" (OuterVolumeSpecName: "kube-api-access-vrc6m") pod "d2588f8c-9004-4e6e-8dd5-c6234b94d5cf" (UID: "d2588f8c-9004-4e6e-8dd5-c6234b94d5cf"). InnerVolumeSpecName "kube-api-access-vrc6m". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:06:03 crc kubenswrapper[5052]: I1203 12:06:03.018760 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage11-crc" (OuterVolumeSpecName: "glance") pod "d2588f8c-9004-4e6e-8dd5-c6234b94d5cf" (UID: "d2588f8c-9004-4e6e-8dd5-c6234b94d5cf"). InnerVolumeSpecName "local-storage11-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 12:06:03 crc kubenswrapper[5052]: I1203 12:06:03.034944 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-api-48e2-account-create-update-dcqjr" podStartSLOduration=3.034902776 podStartE2EDuration="3.034902776s" podCreationTimestamp="2025-12-03 12:06:00 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:06:02.997807125 +0000 UTC m=+2753.586048889" watchObservedRunningTime="2025-12-03 12:06:03.034902776 +0000 UTC m=+2753.623144540" Dec 03 12:06:03 crc kubenswrapper[5052]: I1203 12:06:03.048875 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d2588f8c-9004-4e6e-8dd5-c6234b94d5cf-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "d2588f8c-9004-4e6e-8dd5-c6234b94d5cf" (UID: "d2588f8c-9004-4e6e-8dd5-c6234b94d5cf"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:06:03 crc kubenswrapper[5052]: I1203 12:06:03.062442 5052 scope.go:117] "RemoveContainer" containerID="65e485c8f05b257462566a96517913bed283ca6386a0edf4f3a7a2023ee080ce" Dec 03 12:06:03 crc kubenswrapper[5052]: E1203 12:06:03.062950 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"65e485c8f05b257462566a96517913bed283ca6386a0edf4f3a7a2023ee080ce\": container with ID starting with 65e485c8f05b257462566a96517913bed283ca6386a0edf4f3a7a2023ee080ce not found: ID does not exist" containerID="65e485c8f05b257462566a96517913bed283ca6386a0edf4f3a7a2023ee080ce" Dec 03 12:06:03 crc kubenswrapper[5052]: I1203 12:06:03.062995 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"65e485c8f05b257462566a96517913bed283ca6386a0edf4f3a7a2023ee080ce"} err="failed to get container status \"65e485c8f05b257462566a96517913bed283ca6386a0edf4f3a7a2023ee080ce\": rpc error: code = NotFound desc = could not find container \"65e485c8f05b257462566a96517913bed283ca6386a0edf4f3a7a2023ee080ce\": container with ID starting with 65e485c8f05b257462566a96517913bed283ca6386a0edf4f3a7a2023ee080ce not found: ID does not exist" Dec 03 12:06:03 crc kubenswrapper[5052]: I1203 12:06:03.063025 5052 scope.go:117] "RemoveContainer" containerID="71f1cba19f2d1076cbc88a6c4acbeb2179613b427473cbf3c46cdde47f7e75b5" Dec 03 12:06:03 crc kubenswrapper[5052]: E1203 12:06:03.063350 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"71f1cba19f2d1076cbc88a6c4acbeb2179613b427473cbf3c46cdde47f7e75b5\": container with ID starting with 71f1cba19f2d1076cbc88a6c4acbeb2179613b427473cbf3c46cdde47f7e75b5 not found: ID does not exist" containerID="71f1cba19f2d1076cbc88a6c4acbeb2179613b427473cbf3c46cdde47f7e75b5" Dec 03 12:06:03 crc kubenswrapper[5052]: I1203 12:06:03.063385 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"71f1cba19f2d1076cbc88a6c4acbeb2179613b427473cbf3c46cdde47f7e75b5"} err="failed to get container status \"71f1cba19f2d1076cbc88a6c4acbeb2179613b427473cbf3c46cdde47f7e75b5\": rpc error: code = NotFound desc = could not find container \"71f1cba19f2d1076cbc88a6c4acbeb2179613b427473cbf3c46cdde47f7e75b5\": container with ID starting with 71f1cba19f2d1076cbc88a6c4acbeb2179613b427473cbf3c46cdde47f7e75b5 not found: ID does not exist" Dec 03 12:06:03 crc kubenswrapper[5052]: I1203 12:06:03.078220 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d2588f8c-9004-4e6e-8dd5-c6234b94d5cf-config-data" (OuterVolumeSpecName: "config-data") pod "d2588f8c-9004-4e6e-8dd5-c6234b94d5cf" (UID: "d2588f8c-9004-4e6e-8dd5-c6234b94d5cf"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:06:03 crc kubenswrapper[5052]: I1203 12:06:03.095493 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d2588f8c-9004-4e6e-8dd5-c6234b94d5cf-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "d2588f8c-9004-4e6e-8dd5-c6234b94d5cf" (UID: "d2588f8c-9004-4e6e-8dd5-c6234b94d5cf"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:06:03 crc kubenswrapper[5052]: I1203 12:06:03.113281 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/d2588f8c-9004-4e6e-8dd5-c6234b94d5cf-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:06:03 crc kubenswrapper[5052]: I1203 12:06:03.113327 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/d2588f8c-9004-4e6e-8dd5-c6234b94d5cf-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:06:03 crc kubenswrapper[5052]: I1203 12:06:03.113339 5052 reconciler_common.go:293] "Volume detached for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/d2588f8c-9004-4e6e-8dd5-c6234b94d5cf-httpd-run\") on node \"crc\" DevicePath \"\"" Dec 03 12:06:03 crc kubenswrapper[5052]: I1203 12:06:03.113352 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/d2588f8c-9004-4e6e-8dd5-c6234b94d5cf-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:06:03 crc kubenswrapper[5052]: I1203 12:06:03.113364 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/d2588f8c-9004-4e6e-8dd5-c6234b94d5cf-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:06:03 crc kubenswrapper[5052]: I1203 12:06:03.113375 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d2588f8c-9004-4e6e-8dd5-c6234b94d5cf-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:06:03 crc kubenswrapper[5052]: I1203 12:06:03.113386 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-vrc6m\" (UniqueName: \"kubernetes.io/projected/d2588f8c-9004-4e6e-8dd5-c6234b94d5cf-kube-api-access-vrc6m\") on node \"crc\" DevicePath \"\"" Dec 03 12:06:03 crc kubenswrapper[5052]: I1203 12:06:03.113411 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage11-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage11-crc\") on node \"crc\" " Dec 03 12:06:03 crc kubenswrapper[5052]: I1203 12:06:03.140107 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage11-crc" (UniqueName: "kubernetes.io/local-volume/local-storage11-crc") on node "crc" Dec 03 12:06:03 crc kubenswrapper[5052]: I1203 12:06:03.189176 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:06:03 crc kubenswrapper[5052]: I1203 12:06:03.215999 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage11-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage11-crc\") on node \"crc\" DevicePath \"\"" Dec 03 12:06:03 crc kubenswrapper[5052]: I1203 12:06:03.216671 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:06:03 crc kubenswrapper[5052]: I1203 12:06:03.227914 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:06:03 crc kubenswrapper[5052]: E1203 12:06:03.228718 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="d2588f8c-9004-4e6e-8dd5-c6234b94d5cf" containerName="glance-httpd" Dec 03 12:06:03 crc kubenswrapper[5052]: I1203 12:06:03.228742 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="d2588f8c-9004-4e6e-8dd5-c6234b94d5cf" containerName="glance-httpd" Dec 03 12:06:03 crc kubenswrapper[5052]: E1203 12:06:03.228791 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="d2588f8c-9004-4e6e-8dd5-c6234b94d5cf" containerName="glance-log" Dec 03 12:06:03 crc kubenswrapper[5052]: I1203 12:06:03.228801 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="d2588f8c-9004-4e6e-8dd5-c6234b94d5cf" containerName="glance-log" Dec 03 12:06:03 crc kubenswrapper[5052]: I1203 12:06:03.229798 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="d2588f8c-9004-4e6e-8dd5-c6234b94d5cf" containerName="glance-log" Dec 03 12:06:03 crc kubenswrapper[5052]: I1203 12:06:03.229843 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="d2588f8c-9004-4e6e-8dd5-c6234b94d5cf" containerName="glance-httpd" Dec 03 12:06:03 crc kubenswrapper[5052]: I1203 12:06:03.231845 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:06:03 crc kubenswrapper[5052]: I1203 12:06:03.238048 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-default-external-config-data" Dec 03 12:06:03 crc kubenswrapper[5052]: I1203 12:06:03.238320 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-glance-default-public-svc" Dec 03 12:06:03 crc kubenswrapper[5052]: I1203 12:06:03.242213 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:06:03 crc kubenswrapper[5052]: I1203 12:06:03.317567 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/bef08f40-1b01-406f-af05-7b63e0b6e5e7-config-data\") pod \"glance-default-external-api-0\" (UID: \"bef08f40-1b01-406f-af05-7b63e0b6e5e7\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:06:03 crc kubenswrapper[5052]: I1203 12:06:03.317825 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/bef08f40-1b01-406f-af05-7b63e0b6e5e7-combined-ca-bundle\") pod \"glance-default-external-api-0\" (UID: \"bef08f40-1b01-406f-af05-7b63e0b6e5e7\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:06:03 crc kubenswrapper[5052]: I1203 12:06:03.317974 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tc5vs\" (UniqueName: \"kubernetes.io/projected/bef08f40-1b01-406f-af05-7b63e0b6e5e7-kube-api-access-tc5vs\") pod \"glance-default-external-api-0\" (UID: \"bef08f40-1b01-406f-af05-7b63e0b6e5e7\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:06:03 crc kubenswrapper[5052]: I1203 12:06:03.318113 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/bef08f40-1b01-406f-af05-7b63e0b6e5e7-httpd-run\") pod \"glance-default-external-api-0\" (UID: \"bef08f40-1b01-406f-af05-7b63e0b6e5e7\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:06:03 crc kubenswrapper[5052]: I1203 12:06:03.318189 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage11-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage11-crc\") pod \"glance-default-external-api-0\" (UID: \"bef08f40-1b01-406f-af05-7b63e0b6e5e7\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:06:03 crc kubenswrapper[5052]: I1203 12:06:03.318221 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/bef08f40-1b01-406f-af05-7b63e0b6e5e7-scripts\") pod \"glance-default-external-api-0\" (UID: \"bef08f40-1b01-406f-af05-7b63e0b6e5e7\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:06:03 crc kubenswrapper[5052]: I1203 12:06:03.318247 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/bef08f40-1b01-406f-af05-7b63e0b6e5e7-public-tls-certs\") pod \"glance-default-external-api-0\" (UID: \"bef08f40-1b01-406f-af05-7b63e0b6e5e7\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:06:03 crc kubenswrapper[5052]: I1203 12:06:03.318349 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/bef08f40-1b01-406f-af05-7b63e0b6e5e7-logs\") pod \"glance-default-external-api-0\" (UID: \"bef08f40-1b01-406f-af05-7b63e0b6e5e7\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:06:03 crc kubenswrapper[5052]: I1203 12:06:03.420181 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-tc5vs\" (UniqueName: \"kubernetes.io/projected/bef08f40-1b01-406f-af05-7b63e0b6e5e7-kube-api-access-tc5vs\") pod \"glance-default-external-api-0\" (UID: \"bef08f40-1b01-406f-af05-7b63e0b6e5e7\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:06:03 crc kubenswrapper[5052]: I1203 12:06:03.420776 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/bef08f40-1b01-406f-af05-7b63e0b6e5e7-httpd-run\") pod \"glance-default-external-api-0\" (UID: \"bef08f40-1b01-406f-af05-7b63e0b6e5e7\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:06:03 crc kubenswrapper[5052]: I1203 12:06:03.421409 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/bef08f40-1b01-406f-af05-7b63e0b6e5e7-httpd-run\") pod \"glance-default-external-api-0\" (UID: \"bef08f40-1b01-406f-af05-7b63e0b6e5e7\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:06:03 crc kubenswrapper[5052]: I1203 12:06:03.421568 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage11-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage11-crc\") pod \"glance-default-external-api-0\" (UID: \"bef08f40-1b01-406f-af05-7b63e0b6e5e7\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:06:03 crc kubenswrapper[5052]: I1203 12:06:03.421720 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage11-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage11-crc\") pod \"glance-default-external-api-0\" (UID: \"bef08f40-1b01-406f-af05-7b63e0b6e5e7\") device mount path \"/mnt/openstack/pv11\"" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:06:03 crc kubenswrapper[5052]: I1203 12:06:03.425631 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/bef08f40-1b01-406f-af05-7b63e0b6e5e7-scripts\") pod \"glance-default-external-api-0\" (UID: \"bef08f40-1b01-406f-af05-7b63e0b6e5e7\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:06:03 crc kubenswrapper[5052]: I1203 12:06:03.425727 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/bef08f40-1b01-406f-af05-7b63e0b6e5e7-public-tls-certs\") pod \"glance-default-external-api-0\" (UID: \"bef08f40-1b01-406f-af05-7b63e0b6e5e7\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:06:03 crc kubenswrapper[5052]: I1203 12:06:03.425829 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/bef08f40-1b01-406f-af05-7b63e0b6e5e7-logs\") pod \"glance-default-external-api-0\" (UID: \"bef08f40-1b01-406f-af05-7b63e0b6e5e7\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:06:03 crc kubenswrapper[5052]: I1203 12:06:03.425891 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/bef08f40-1b01-406f-af05-7b63e0b6e5e7-config-data\") pod \"glance-default-external-api-0\" (UID: \"bef08f40-1b01-406f-af05-7b63e0b6e5e7\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:06:03 crc kubenswrapper[5052]: I1203 12:06:03.426211 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/bef08f40-1b01-406f-af05-7b63e0b6e5e7-combined-ca-bundle\") pod \"glance-default-external-api-0\" (UID: \"bef08f40-1b01-406f-af05-7b63e0b6e5e7\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:06:03 crc kubenswrapper[5052]: I1203 12:06:03.426454 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/bef08f40-1b01-406f-af05-7b63e0b6e5e7-logs\") pod \"glance-default-external-api-0\" (UID: \"bef08f40-1b01-406f-af05-7b63e0b6e5e7\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:06:03 crc kubenswrapper[5052]: I1203 12:06:03.429434 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/bef08f40-1b01-406f-af05-7b63e0b6e5e7-scripts\") pod \"glance-default-external-api-0\" (UID: \"bef08f40-1b01-406f-af05-7b63e0b6e5e7\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:06:03 crc kubenswrapper[5052]: I1203 12:06:03.430350 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/bef08f40-1b01-406f-af05-7b63e0b6e5e7-public-tls-certs\") pod \"glance-default-external-api-0\" (UID: \"bef08f40-1b01-406f-af05-7b63e0b6e5e7\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:06:03 crc kubenswrapper[5052]: I1203 12:06:03.430617 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/bef08f40-1b01-406f-af05-7b63e0b6e5e7-combined-ca-bundle\") pod \"glance-default-external-api-0\" (UID: \"bef08f40-1b01-406f-af05-7b63e0b6e5e7\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:06:03 crc kubenswrapper[5052]: I1203 12:06:03.433179 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/bef08f40-1b01-406f-af05-7b63e0b6e5e7-config-data\") pod \"glance-default-external-api-0\" (UID: \"bef08f40-1b01-406f-af05-7b63e0b6e5e7\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:06:03 crc kubenswrapper[5052]: I1203 12:06:03.444864 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-tc5vs\" (UniqueName: \"kubernetes.io/projected/bef08f40-1b01-406f-af05-7b63e0b6e5e7-kube-api-access-tc5vs\") pod \"glance-default-external-api-0\" (UID: \"bef08f40-1b01-406f-af05-7b63e0b6e5e7\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:06:03 crc kubenswrapper[5052]: I1203 12:06:03.465877 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage11-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage11-crc\") pod \"glance-default-external-api-0\" (UID: \"bef08f40-1b01-406f-af05-7b63e0b6e5e7\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:06:03 crc kubenswrapper[5052]: I1203 12:06:03.632091 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:06:03 crc kubenswrapper[5052]: I1203 12:06:03.707883 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ae2d7638-3ed1-4ab4-8d55-8eb8608bc97e" path="/var/lib/kubelet/pods/ae2d7638-3ed1-4ab4-8d55-8eb8608bc97e/volumes" Dec 03 12:06:03 crc kubenswrapper[5052]: I1203 12:06:03.708889 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d2588f8c-9004-4e6e-8dd5-c6234b94d5cf" path="/var/lib/kubelet/pods/d2588f8c-9004-4e6e-8dd5-c6234b94d5cf/volumes" Dec 03 12:06:03 crc kubenswrapper[5052]: I1203 12:06:03.843005 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:06:03 crc kubenswrapper[5052]: I1203 12:06:03.945683 5052 generic.go:334] "Generic (PLEG): container finished" podID="3af45ac5-7939-4525-891d-8ca1f5e697bf" containerID="e1903fe93354f2eb6210febadfc9a242d90b8681d1287d692e162cc3efd0b892" exitCode=0 Dec 03 12:06:03 crc kubenswrapper[5052]: I1203 12:06:03.945760 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-48e2-account-create-update-dcqjr" event={"ID":"3af45ac5-7939-4525-891d-8ca1f5e697bf","Type":"ContainerDied","Data":"e1903fe93354f2eb6210febadfc9a242d90b8681d1287d692e162cc3efd0b892"} Dec 03 12:06:03 crc kubenswrapper[5052]: I1203 12:06:03.965780 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/a3f4c9a7-c441-4487-9814-486ac5e66848-httpd-run\") pod \"a3f4c9a7-c441-4487-9814-486ac5e66848\" (UID: \"a3f4c9a7-c441-4487-9814-486ac5e66848\") " Dec 03 12:06:03 crc kubenswrapper[5052]: I1203 12:06:03.965866 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a3f4c9a7-c441-4487-9814-486ac5e66848-config-data\") pod \"a3f4c9a7-c441-4487-9814-486ac5e66848\" (UID: \"a3f4c9a7-c441-4487-9814-486ac5e66848\") " Dec 03 12:06:03 crc kubenswrapper[5052]: I1203 12:06:03.965906 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q9g8p\" (UniqueName: \"kubernetes.io/projected/a3f4c9a7-c441-4487-9814-486ac5e66848-kube-api-access-q9g8p\") pod \"a3f4c9a7-c441-4487-9814-486ac5e66848\" (UID: \"a3f4c9a7-c441-4487-9814-486ac5e66848\") " Dec 03 12:06:03 crc kubenswrapper[5052]: I1203 12:06:03.965974 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a3f4c9a7-c441-4487-9814-486ac5e66848-combined-ca-bundle\") pod \"a3f4c9a7-c441-4487-9814-486ac5e66848\" (UID: \"a3f4c9a7-c441-4487-9814-486ac5e66848\") " Dec 03 12:06:03 crc kubenswrapper[5052]: I1203 12:06:03.966034 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/a3f4c9a7-c441-4487-9814-486ac5e66848-scripts\") pod \"a3f4c9a7-c441-4487-9814-486ac5e66848\" (UID: \"a3f4c9a7-c441-4487-9814-486ac5e66848\") " Dec 03 12:06:03 crc kubenswrapper[5052]: I1203 12:06:03.966058 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/a3f4c9a7-c441-4487-9814-486ac5e66848-internal-tls-certs\") pod \"a3f4c9a7-c441-4487-9814-486ac5e66848\" (UID: \"a3f4c9a7-c441-4487-9814-486ac5e66848\") " Dec 03 12:06:03 crc kubenswrapper[5052]: I1203 12:06:03.966154 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/a3f4c9a7-c441-4487-9814-486ac5e66848-logs\") pod \"a3f4c9a7-c441-4487-9814-486ac5e66848\" (UID: \"a3f4c9a7-c441-4487-9814-486ac5e66848\") " Dec 03 12:06:03 crc kubenswrapper[5052]: I1203 12:06:03.966248 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"glance\" (UniqueName: \"kubernetes.io/local-volume/local-storage13-crc\") pod \"a3f4c9a7-c441-4487-9814-486ac5e66848\" (UID: \"a3f4c9a7-c441-4487-9814-486ac5e66848\") " Dec 03 12:06:03 crc kubenswrapper[5052]: I1203 12:06:03.972303 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage13-crc" (OuterVolumeSpecName: "glance") pod "a3f4c9a7-c441-4487-9814-486ac5e66848" (UID: "a3f4c9a7-c441-4487-9814-486ac5e66848"). InnerVolumeSpecName "local-storage13-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 12:06:03 crc kubenswrapper[5052]: I1203 12:06:03.972340 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3f4c9a7-c441-4487-9814-486ac5e66848-httpd-run" (OuterVolumeSpecName: "httpd-run") pod "a3f4c9a7-c441-4487-9814-486ac5e66848" (UID: "a3f4c9a7-c441-4487-9814-486ac5e66848"). InnerVolumeSpecName "httpd-run". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:06:03 crc kubenswrapper[5052]: I1203 12:06:03.972623 5052 generic.go:334] "Generic (PLEG): container finished" podID="4741b42c-35a5-407d-b5cc-c7d47b07ce0b" containerID="f68f634ddf0cc7c513c585883315d968fce2a8f67935dd2746313bd6d94c035d" exitCode=0 Dec 03 12:06:03 crc kubenswrapper[5052]: I1203 12:06:03.972658 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3f4c9a7-c441-4487-9814-486ac5e66848-logs" (OuterVolumeSpecName: "logs") pod "a3f4c9a7-c441-4487-9814-486ac5e66848" (UID: "a3f4c9a7-c441-4487-9814-486ac5e66848"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:06:03 crc kubenswrapper[5052]: I1203 12:06:03.972719 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-db-create-t6q5w" event={"ID":"4741b42c-35a5-407d-b5cc-c7d47b07ce0b","Type":"ContainerDied","Data":"f68f634ddf0cc7c513c585883315d968fce2a8f67935dd2746313bd6d94c035d"} Dec 03 12:06:03 crc kubenswrapper[5052]: I1203 12:06:03.976743 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a3f4c9a7-c441-4487-9814-486ac5e66848-scripts" (OuterVolumeSpecName: "scripts") pod "a3f4c9a7-c441-4487-9814-486ac5e66848" (UID: "a3f4c9a7-c441-4487-9814-486ac5e66848"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:06:03 crc kubenswrapper[5052]: I1203 12:06:03.987860 5052 generic.go:334] "Generic (PLEG): container finished" podID="fdc5eec2-010f-4de6-a578-e5bda587d218" containerID="ac697ab7fae2d375f75e93f50ee6034e1e0bf8653a46e8447628f29f9a3cceb3" exitCode=0 Dec 03 12:06:03 crc kubenswrapper[5052]: I1203 12:06:03.988615 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-c2b9-account-create-update-hlm9w" event={"ID":"fdc5eec2-010f-4de6-a578-e5bda587d218","Type":"ContainerDied","Data":"ac697ab7fae2d375f75e93f50ee6034e1e0bf8653a46e8447628f29f9a3cceb3"} Dec 03 12:06:04 crc kubenswrapper[5052]: I1203 12:06:04.005947 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a3f4c9a7-c441-4487-9814-486ac5e66848-kube-api-access-q9g8p" (OuterVolumeSpecName: "kube-api-access-q9g8p") pod "a3f4c9a7-c441-4487-9814-486ac5e66848" (UID: "a3f4c9a7-c441-4487-9814-486ac5e66848"). InnerVolumeSpecName "kube-api-access-q9g8p". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:06:04 crc kubenswrapper[5052]: I1203 12:06:04.006801 5052 generic.go:334] "Generic (PLEG): container finished" podID="c2d35861-9e4b-4f53-b841-e18b26d575cc" containerID="4d472d46d5c7f6ca91aa5161d730340a3e8abead48886bdc090952991731d00a" exitCode=0 Dec 03 12:06:04 crc kubenswrapper[5052]: I1203 12:06:04.006908 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-db-create-8kvhf" event={"ID":"c2d35861-9e4b-4f53-b841-e18b26d575cc","Type":"ContainerDied","Data":"4d472d46d5c7f6ca91aa5161d730340a3e8abead48886bdc090952991731d00a"} Dec 03 12:06:04 crc kubenswrapper[5052]: I1203 12:06:04.046584 5052 generic.go:334] "Generic (PLEG): container finished" podID="89503fe3-9e65-4405-81f6-45cc9b5db612" containerID="9078309d7e958fc36e931932ff0699b628d5fd0921b34acd1cd89a0efd5e8691" exitCode=0 Dec 03 12:06:04 crc kubenswrapper[5052]: I1203 12:06:04.046713 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-db-create-64bnw" event={"ID":"89503fe3-9e65-4405-81f6-45cc9b5db612","Type":"ContainerDied","Data":"9078309d7e958fc36e931932ff0699b628d5fd0921b34acd1cd89a0efd5e8691"} Dec 03 12:06:04 crc kubenswrapper[5052]: I1203 12:06:04.048970 5052 generic.go:334] "Generic (PLEG): container finished" podID="a3f4c9a7-c441-4487-9814-486ac5e66848" containerID="e6ef915c623f6539b5a4b98c7c0a747b3dd34c5a9b33f8653603dec931503dbe" exitCode=0 Dec 03 12:06:04 crc kubenswrapper[5052]: I1203 12:06:04.049052 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"a3f4c9a7-c441-4487-9814-486ac5e66848","Type":"ContainerDied","Data":"e6ef915c623f6539b5a4b98c7c0a747b3dd34c5a9b33f8653603dec931503dbe"} Dec 03 12:06:04 crc kubenswrapper[5052]: I1203 12:06:04.049085 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"a3f4c9a7-c441-4487-9814-486ac5e66848","Type":"ContainerDied","Data":"6bf3c46a2247dda992a234acc98ac30c9bd08ad3e213acae31d386e45944ac38"} Dec 03 12:06:04 crc kubenswrapper[5052]: I1203 12:06:04.049107 5052 scope.go:117] "RemoveContainer" containerID="e6ef915c623f6539b5a4b98c7c0a747b3dd34c5a9b33f8653603dec931503dbe" Dec 03 12:06:04 crc kubenswrapper[5052]: I1203 12:06:04.049245 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:06:04 crc kubenswrapper[5052]: I1203 12:06:04.057792 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"cf64fec7-c528-4a2c-89b4-a43851b7a903","Type":"ContainerStarted","Data":"09a940af112f01defba900147a49855d50e201749b807d54ab4bc7d83924e88a"} Dec 03 12:06:04 crc kubenswrapper[5052]: I1203 12:06:04.068333 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage13-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage13-crc\") on node \"crc\" " Dec 03 12:06:04 crc kubenswrapper[5052]: I1203 12:06:04.068361 5052 reconciler_common.go:293] "Volume detached for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/a3f4c9a7-c441-4487-9814-486ac5e66848-httpd-run\") on node \"crc\" DevicePath \"\"" Dec 03 12:06:04 crc kubenswrapper[5052]: I1203 12:06:04.068372 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-q9g8p\" (UniqueName: \"kubernetes.io/projected/a3f4c9a7-c441-4487-9814-486ac5e66848-kube-api-access-q9g8p\") on node \"crc\" DevicePath \"\"" Dec 03 12:06:04 crc kubenswrapper[5052]: I1203 12:06:04.068383 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/a3f4c9a7-c441-4487-9814-486ac5e66848-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:06:04 crc kubenswrapper[5052]: I1203 12:06:04.068396 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/a3f4c9a7-c441-4487-9814-486ac5e66848-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:06:04 crc kubenswrapper[5052]: I1203 12:06:04.121919 5052 scope.go:117] "RemoveContainer" containerID="15bd35b56459bfabff2e71300b23da8680d50a0e069d64db131c072ec6fed100" Dec 03 12:06:04 crc kubenswrapper[5052]: I1203 12:06:04.159905 5052 scope.go:117] "RemoveContainer" containerID="e6ef915c623f6539b5a4b98c7c0a747b3dd34c5a9b33f8653603dec931503dbe" Dec 03 12:06:04 crc kubenswrapper[5052]: E1203 12:06:04.160391 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e6ef915c623f6539b5a4b98c7c0a747b3dd34c5a9b33f8653603dec931503dbe\": container with ID starting with e6ef915c623f6539b5a4b98c7c0a747b3dd34c5a9b33f8653603dec931503dbe not found: ID does not exist" containerID="e6ef915c623f6539b5a4b98c7c0a747b3dd34c5a9b33f8653603dec931503dbe" Dec 03 12:06:04 crc kubenswrapper[5052]: I1203 12:06:04.160429 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e6ef915c623f6539b5a4b98c7c0a747b3dd34c5a9b33f8653603dec931503dbe"} err="failed to get container status \"e6ef915c623f6539b5a4b98c7c0a747b3dd34c5a9b33f8653603dec931503dbe\": rpc error: code = NotFound desc = could not find container \"e6ef915c623f6539b5a4b98c7c0a747b3dd34c5a9b33f8653603dec931503dbe\": container with ID starting with e6ef915c623f6539b5a4b98c7c0a747b3dd34c5a9b33f8653603dec931503dbe not found: ID does not exist" Dec 03 12:06:04 crc kubenswrapper[5052]: I1203 12:06:04.160452 5052 scope.go:117] "RemoveContainer" containerID="15bd35b56459bfabff2e71300b23da8680d50a0e069d64db131c072ec6fed100" Dec 03 12:06:04 crc kubenswrapper[5052]: E1203 12:06:04.161896 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"15bd35b56459bfabff2e71300b23da8680d50a0e069d64db131c072ec6fed100\": container with ID starting with 15bd35b56459bfabff2e71300b23da8680d50a0e069d64db131c072ec6fed100 not found: ID does not exist" containerID="15bd35b56459bfabff2e71300b23da8680d50a0e069d64db131c072ec6fed100" Dec 03 12:06:04 crc kubenswrapper[5052]: I1203 12:06:04.161924 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"15bd35b56459bfabff2e71300b23da8680d50a0e069d64db131c072ec6fed100"} err="failed to get container status \"15bd35b56459bfabff2e71300b23da8680d50a0e069d64db131c072ec6fed100\": rpc error: code = NotFound desc = could not find container \"15bd35b56459bfabff2e71300b23da8680d50a0e069d64db131c072ec6fed100\": container with ID starting with 15bd35b56459bfabff2e71300b23da8680d50a0e069d64db131c072ec6fed100 not found: ID does not exist" Dec 03 12:06:04 crc kubenswrapper[5052]: I1203 12:06:04.202351 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a3f4c9a7-c441-4487-9814-486ac5e66848-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "a3f4c9a7-c441-4487-9814-486ac5e66848" (UID: "a3f4c9a7-c441-4487-9814-486ac5e66848"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:06:04 crc kubenswrapper[5052]: I1203 12:06:04.217900 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a3f4c9a7-c441-4487-9814-486ac5e66848-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "a3f4c9a7-c441-4487-9814-486ac5e66848" (UID: "a3f4c9a7-c441-4487-9814-486ac5e66848"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:06:04 crc kubenswrapper[5052]: I1203 12:06:04.228301 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage13-crc" (UniqueName: "kubernetes.io/local-volume/local-storage13-crc") on node "crc" Dec 03 12:06:04 crc kubenswrapper[5052]: I1203 12:06:04.273376 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage13-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage13-crc\") on node \"crc\" DevicePath \"\"" Dec 03 12:06:04 crc kubenswrapper[5052]: I1203 12:06:04.273397 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a3f4c9a7-c441-4487-9814-486ac5e66848-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:06:04 crc kubenswrapper[5052]: I1203 12:06:04.273408 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/a3f4c9a7-c441-4487-9814-486ac5e66848-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:06:04 crc kubenswrapper[5052]: I1203 12:06:04.278762 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a3f4c9a7-c441-4487-9814-486ac5e66848-config-data" (OuterVolumeSpecName: "config-data") pod "a3f4c9a7-c441-4487-9814-486ac5e66848" (UID: "a3f4c9a7-c441-4487-9814-486ac5e66848"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:06:04 crc kubenswrapper[5052]: I1203 12:06:04.375745 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:06:04 crc kubenswrapper[5052]: I1203 12:06:04.376910 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a3f4c9a7-c441-4487-9814-486ac5e66848-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:06:04 crc kubenswrapper[5052]: I1203 12:06:04.556571 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-db-create-64bnw" Dec 03 12:06:04 crc kubenswrapper[5052]: I1203 12:06:04.571720 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:06:04 crc kubenswrapper[5052]: I1203 12:06:04.594057 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:06:04 crc kubenswrapper[5052]: I1203 12:06:04.672812 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:06:04 crc kubenswrapper[5052]: E1203 12:06:04.673864 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="a3f4c9a7-c441-4487-9814-486ac5e66848" containerName="glance-log" Dec 03 12:06:04 crc kubenswrapper[5052]: I1203 12:06:04.673878 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="a3f4c9a7-c441-4487-9814-486ac5e66848" containerName="glance-log" Dec 03 12:06:04 crc kubenswrapper[5052]: E1203 12:06:04.673922 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="a3f4c9a7-c441-4487-9814-486ac5e66848" containerName="glance-httpd" Dec 03 12:06:04 crc kubenswrapper[5052]: I1203 12:06:04.673930 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="a3f4c9a7-c441-4487-9814-486ac5e66848" containerName="glance-httpd" Dec 03 12:06:04 crc kubenswrapper[5052]: E1203 12:06:04.673958 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="89503fe3-9e65-4405-81f6-45cc9b5db612" containerName="mariadb-database-create" Dec 03 12:06:04 crc kubenswrapper[5052]: I1203 12:06:04.673965 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="89503fe3-9e65-4405-81f6-45cc9b5db612" containerName="mariadb-database-create" Dec 03 12:06:04 crc kubenswrapper[5052]: I1203 12:06:04.674360 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="a3f4c9a7-c441-4487-9814-486ac5e66848" containerName="glance-httpd" Dec 03 12:06:04 crc kubenswrapper[5052]: I1203 12:06:04.674394 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="a3f4c9a7-c441-4487-9814-486ac5e66848" containerName="glance-log" Dec 03 12:06:04 crc kubenswrapper[5052]: I1203 12:06:04.674411 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="89503fe3-9e65-4405-81f6-45cc9b5db612" containerName="mariadb-database-create" Dec 03 12:06:04 crc kubenswrapper[5052]: I1203 12:06:04.676839 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:06:04 crc kubenswrapper[5052]: I1203 12:06:04.683223 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-glance-default-internal-svc" Dec 03 12:06:04 crc kubenswrapper[5052]: I1203 12:06:04.683443 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-default-internal-config-data" Dec 03 12:06:04 crc kubenswrapper[5052]: I1203 12:06:04.685499 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-smj68\" (UniqueName: \"kubernetes.io/projected/89503fe3-9e65-4405-81f6-45cc9b5db612-kube-api-access-smj68\") pod \"89503fe3-9e65-4405-81f6-45cc9b5db612\" (UID: \"89503fe3-9e65-4405-81f6-45cc9b5db612\") " Dec 03 12:06:04 crc kubenswrapper[5052]: I1203 12:06:04.685671 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/89503fe3-9e65-4405-81f6-45cc9b5db612-operator-scripts\") pod \"89503fe3-9e65-4405-81f6-45cc9b5db612\" (UID: \"89503fe3-9e65-4405-81f6-45cc9b5db612\") " Dec 03 12:06:04 crc kubenswrapper[5052]: I1203 12:06:04.688159 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/89503fe3-9e65-4405-81f6-45cc9b5db612-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "89503fe3-9e65-4405-81f6-45cc9b5db612" (UID: "89503fe3-9e65-4405-81f6-45cc9b5db612"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:06:04 crc kubenswrapper[5052]: I1203 12:06:04.691273 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/89503fe3-9e65-4405-81f6-45cc9b5db612-kube-api-access-smj68" (OuterVolumeSpecName: "kube-api-access-smj68") pod "89503fe3-9e65-4405-81f6-45cc9b5db612" (UID: "89503fe3-9e65-4405-81f6-45cc9b5db612"). InnerVolumeSpecName "kube-api-access-smj68". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:06:04 crc kubenswrapper[5052]: I1203 12:06:04.705261 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:06:04 crc kubenswrapper[5052]: I1203 12:06:04.790242 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4a2a1881-e61a-4a99-a2f3-b614aeed81d1-combined-ca-bundle\") pod \"glance-default-internal-api-0\" (UID: \"4a2a1881-e61a-4a99-a2f3-b614aeed81d1\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:06:04 crc kubenswrapper[5052]: I1203 12:06:04.790347 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage13-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage13-crc\") pod \"glance-default-internal-api-0\" (UID: \"4a2a1881-e61a-4a99-a2f3-b614aeed81d1\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:06:04 crc kubenswrapper[5052]: I1203 12:06:04.790369 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/4a2a1881-e61a-4a99-a2f3-b614aeed81d1-scripts\") pod \"glance-default-internal-api-0\" (UID: \"4a2a1881-e61a-4a99-a2f3-b614aeed81d1\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:06:04 crc kubenswrapper[5052]: I1203 12:06:04.790408 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/4a2a1881-e61a-4a99-a2f3-b614aeed81d1-httpd-run\") pod \"glance-default-internal-api-0\" (UID: \"4a2a1881-e61a-4a99-a2f3-b614aeed81d1\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:06:04 crc kubenswrapper[5052]: I1203 12:06:04.790427 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/4a2a1881-e61a-4a99-a2f3-b614aeed81d1-internal-tls-certs\") pod \"glance-default-internal-api-0\" (UID: \"4a2a1881-e61a-4a99-a2f3-b614aeed81d1\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:06:04 crc kubenswrapper[5052]: I1203 12:06:04.790456 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/4a2a1881-e61a-4a99-a2f3-b614aeed81d1-logs\") pod \"glance-default-internal-api-0\" (UID: \"4a2a1881-e61a-4a99-a2f3-b614aeed81d1\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:06:04 crc kubenswrapper[5052]: I1203 12:06:04.790474 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/4a2a1881-e61a-4a99-a2f3-b614aeed81d1-config-data\") pod \"glance-default-internal-api-0\" (UID: \"4a2a1881-e61a-4a99-a2f3-b614aeed81d1\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:06:04 crc kubenswrapper[5052]: I1203 12:06:04.790613 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ds85z\" (UniqueName: \"kubernetes.io/projected/4a2a1881-e61a-4a99-a2f3-b614aeed81d1-kube-api-access-ds85z\") pod \"glance-default-internal-api-0\" (UID: \"4a2a1881-e61a-4a99-a2f3-b614aeed81d1\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:06:04 crc kubenswrapper[5052]: I1203 12:06:04.790659 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/89503fe3-9e65-4405-81f6-45cc9b5db612-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:06:04 crc kubenswrapper[5052]: I1203 12:06:04.790671 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-smj68\" (UniqueName: \"kubernetes.io/projected/89503fe3-9e65-4405-81f6-45cc9b5db612-kube-api-access-smj68\") on node \"crc\" DevicePath \"\"" Dec 03 12:06:04 crc kubenswrapper[5052]: I1203 12:06:04.893213 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-ds85z\" (UniqueName: \"kubernetes.io/projected/4a2a1881-e61a-4a99-a2f3-b614aeed81d1-kube-api-access-ds85z\") pod \"glance-default-internal-api-0\" (UID: \"4a2a1881-e61a-4a99-a2f3-b614aeed81d1\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:06:04 crc kubenswrapper[5052]: I1203 12:06:04.893317 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4a2a1881-e61a-4a99-a2f3-b614aeed81d1-combined-ca-bundle\") pod \"glance-default-internal-api-0\" (UID: \"4a2a1881-e61a-4a99-a2f3-b614aeed81d1\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:06:04 crc kubenswrapper[5052]: I1203 12:06:04.893439 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage13-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage13-crc\") pod \"glance-default-internal-api-0\" (UID: \"4a2a1881-e61a-4a99-a2f3-b614aeed81d1\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:06:04 crc kubenswrapper[5052]: I1203 12:06:04.893472 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/4a2a1881-e61a-4a99-a2f3-b614aeed81d1-scripts\") pod \"glance-default-internal-api-0\" (UID: \"4a2a1881-e61a-4a99-a2f3-b614aeed81d1\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:06:04 crc kubenswrapper[5052]: I1203 12:06:04.893645 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/4a2a1881-e61a-4a99-a2f3-b614aeed81d1-httpd-run\") pod \"glance-default-internal-api-0\" (UID: \"4a2a1881-e61a-4a99-a2f3-b614aeed81d1\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:06:04 crc kubenswrapper[5052]: I1203 12:06:04.893677 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/4a2a1881-e61a-4a99-a2f3-b614aeed81d1-internal-tls-certs\") pod \"glance-default-internal-api-0\" (UID: \"4a2a1881-e61a-4a99-a2f3-b614aeed81d1\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:06:04 crc kubenswrapper[5052]: I1203 12:06:04.893704 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/4a2a1881-e61a-4a99-a2f3-b614aeed81d1-logs\") pod \"glance-default-internal-api-0\" (UID: \"4a2a1881-e61a-4a99-a2f3-b614aeed81d1\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:06:04 crc kubenswrapper[5052]: I1203 12:06:04.893728 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/4a2a1881-e61a-4a99-a2f3-b614aeed81d1-config-data\") pod \"glance-default-internal-api-0\" (UID: \"4a2a1881-e61a-4a99-a2f3-b614aeed81d1\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:06:04 crc kubenswrapper[5052]: I1203 12:06:04.894731 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage13-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage13-crc\") pod \"glance-default-internal-api-0\" (UID: \"4a2a1881-e61a-4a99-a2f3-b614aeed81d1\") device mount path \"/mnt/openstack/pv13\"" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:06:04 crc kubenswrapper[5052]: I1203 12:06:04.894859 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/4a2a1881-e61a-4a99-a2f3-b614aeed81d1-logs\") pod \"glance-default-internal-api-0\" (UID: \"4a2a1881-e61a-4a99-a2f3-b614aeed81d1\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:06:04 crc kubenswrapper[5052]: I1203 12:06:04.895456 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/4a2a1881-e61a-4a99-a2f3-b614aeed81d1-httpd-run\") pod \"glance-default-internal-api-0\" (UID: \"4a2a1881-e61a-4a99-a2f3-b614aeed81d1\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:06:04 crc kubenswrapper[5052]: I1203 12:06:04.903017 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/4a2a1881-e61a-4a99-a2f3-b614aeed81d1-internal-tls-certs\") pod \"glance-default-internal-api-0\" (UID: \"4a2a1881-e61a-4a99-a2f3-b614aeed81d1\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:06:04 crc kubenswrapper[5052]: I1203 12:06:04.903328 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/4a2a1881-e61a-4a99-a2f3-b614aeed81d1-scripts\") pod \"glance-default-internal-api-0\" (UID: \"4a2a1881-e61a-4a99-a2f3-b614aeed81d1\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:06:04 crc kubenswrapper[5052]: I1203 12:06:04.904756 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/4a2a1881-e61a-4a99-a2f3-b614aeed81d1-config-data\") pod \"glance-default-internal-api-0\" (UID: \"4a2a1881-e61a-4a99-a2f3-b614aeed81d1\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:06:04 crc kubenswrapper[5052]: I1203 12:06:04.907698 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4a2a1881-e61a-4a99-a2f3-b614aeed81d1-combined-ca-bundle\") pod \"glance-default-internal-api-0\" (UID: \"4a2a1881-e61a-4a99-a2f3-b614aeed81d1\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:06:04 crc kubenswrapper[5052]: I1203 12:06:04.920371 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-ds85z\" (UniqueName: \"kubernetes.io/projected/4a2a1881-e61a-4a99-a2f3-b614aeed81d1-kube-api-access-ds85z\") pod \"glance-default-internal-api-0\" (UID: \"4a2a1881-e61a-4a99-a2f3-b614aeed81d1\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:06:04 crc kubenswrapper[5052]: I1203 12:06:04.936193 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage13-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage13-crc\") pod \"glance-default-internal-api-0\" (UID: \"4a2a1881-e61a-4a99-a2f3-b614aeed81d1\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:06:05 crc kubenswrapper[5052]: I1203 12:06:05.066535 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:06:05 crc kubenswrapper[5052]: I1203 12:06:05.068367 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-db-create-64bnw" Dec 03 12:06:05 crc kubenswrapper[5052]: I1203 12:06:05.069118 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-db-create-64bnw" event={"ID":"89503fe3-9e65-4405-81f6-45cc9b5db612","Type":"ContainerDied","Data":"f1fdd2de1c350b5dc60f5fda317c4eb8cc216911f9e0228bbb129f4ade297bdc"} Dec 03 12:06:05 crc kubenswrapper[5052]: I1203 12:06:05.069164 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f1fdd2de1c350b5dc60f5fda317c4eb8cc216911f9e0228bbb129f4ade297bdc" Dec 03 12:06:05 crc kubenswrapper[5052]: I1203 12:06:05.085368 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"cf64fec7-c528-4a2c-89b4-a43851b7a903","Type":"ContainerStarted","Data":"1e7c6cfd093da01c0b0e07dd0484985317870469414dddeb15a45b6655ad7f58"} Dec 03 12:06:05 crc kubenswrapper[5052]: I1203 12:06:05.085448 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"cf64fec7-c528-4a2c-89b4-a43851b7a903","Type":"ContainerStarted","Data":"8e6902d44b83c5eec3a438c9b36c5e9341283dffc8678b21795a0f7d6a5bafc4"} Dec 03 12:06:05 crc kubenswrapper[5052]: I1203 12:06:05.089985 5052 generic.go:334] "Generic (PLEG): container finished" podID="53e80028-95b5-48c8-96ce-f6e40ed916cf" containerID="8b708b7d5d49769665548fc88668b1958dadb43740e66dff74107f13b9ce7ae6" exitCode=0 Dec 03 12:06:05 crc kubenswrapper[5052]: I1203 12:06:05.090063 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-d294-account-create-update-tm6pc" event={"ID":"53e80028-95b5-48c8-96ce-f6e40ed916cf","Type":"ContainerDied","Data":"8b708b7d5d49769665548fc88668b1958dadb43740e66dff74107f13b9ce7ae6"} Dec 03 12:06:05 crc kubenswrapper[5052]: I1203 12:06:05.094681 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"bef08f40-1b01-406f-af05-7b63e0b6e5e7","Type":"ContainerStarted","Data":"46f9b1f29a0bfcc855368ead467b689caf617edf4d3c13cbbf5dac62d701a661"} Dec 03 12:06:05 crc kubenswrapper[5052]: I1203 12:06:05.644593 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-48e2-account-create-update-dcqjr" Dec 03 12:06:05 crc kubenswrapper[5052]: I1203 12:06:05.693785 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a3f4c9a7-c441-4487-9814-486ac5e66848" path="/var/lib/kubelet/pods/a3f4c9a7-c441-4487-9814-486ac5e66848/volumes" Dec 03 12:06:05 crc kubenswrapper[5052]: I1203 12:06:05.712208 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-blj2x\" (UniqueName: \"kubernetes.io/projected/3af45ac5-7939-4525-891d-8ca1f5e697bf-kube-api-access-blj2x\") pod \"3af45ac5-7939-4525-891d-8ca1f5e697bf\" (UID: \"3af45ac5-7939-4525-891d-8ca1f5e697bf\") " Dec 03 12:06:05 crc kubenswrapper[5052]: I1203 12:06:05.712440 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/3af45ac5-7939-4525-891d-8ca1f5e697bf-operator-scripts\") pod \"3af45ac5-7939-4525-891d-8ca1f5e697bf\" (UID: \"3af45ac5-7939-4525-891d-8ca1f5e697bf\") " Dec 03 12:06:05 crc kubenswrapper[5052]: I1203 12:06:05.715107 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3af45ac5-7939-4525-891d-8ca1f5e697bf-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "3af45ac5-7939-4525-891d-8ca1f5e697bf" (UID: "3af45ac5-7939-4525-891d-8ca1f5e697bf"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:06:05 crc kubenswrapper[5052]: I1203 12:06:05.740631 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3af45ac5-7939-4525-891d-8ca1f5e697bf-kube-api-access-blj2x" (OuterVolumeSpecName: "kube-api-access-blj2x") pod "3af45ac5-7939-4525-891d-8ca1f5e697bf" (UID: "3af45ac5-7939-4525-891d-8ca1f5e697bf"). InnerVolumeSpecName "kube-api-access-blj2x". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:06:05 crc kubenswrapper[5052]: I1203 12:06:05.818020 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-blj2x\" (UniqueName: \"kubernetes.io/projected/3af45ac5-7939-4525-891d-8ca1f5e697bf-kube-api-access-blj2x\") on node \"crc\" DevicePath \"\"" Dec 03 12:06:05 crc kubenswrapper[5052]: I1203 12:06:05.818052 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/3af45ac5-7939-4525-891d-8ca1f5e697bf-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:06:05 crc kubenswrapper[5052]: I1203 12:06:05.833891 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-db-create-t6q5w" Dec 03 12:06:05 crc kubenswrapper[5052]: I1203 12:06:05.857163 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-db-create-8kvhf" Dec 03 12:06:05 crc kubenswrapper[5052]: I1203 12:06:05.892426 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-c2b9-account-create-update-hlm9w" Dec 03 12:06:05 crc kubenswrapper[5052]: I1203 12:06:05.892617 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:06:05 crc kubenswrapper[5052]: I1203 12:06:05.919378 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/c2d35861-9e4b-4f53-b841-e18b26d575cc-operator-scripts\") pod \"c2d35861-9e4b-4f53-b841-e18b26d575cc\" (UID: \"c2d35861-9e4b-4f53-b841-e18b26d575cc\") " Dec 03 12:06:05 crc kubenswrapper[5052]: I1203 12:06:05.919525 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sdf5b\" (UniqueName: \"kubernetes.io/projected/4741b42c-35a5-407d-b5cc-c7d47b07ce0b-kube-api-access-sdf5b\") pod \"4741b42c-35a5-407d-b5cc-c7d47b07ce0b\" (UID: \"4741b42c-35a5-407d-b5cc-c7d47b07ce0b\") " Dec 03 12:06:05 crc kubenswrapper[5052]: I1203 12:06:05.919600 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-79b27\" (UniqueName: \"kubernetes.io/projected/c2d35861-9e4b-4f53-b841-e18b26d575cc-kube-api-access-79b27\") pod \"c2d35861-9e4b-4f53-b841-e18b26d575cc\" (UID: \"c2d35861-9e4b-4f53-b841-e18b26d575cc\") " Dec 03 12:06:05 crc kubenswrapper[5052]: I1203 12:06:05.919878 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/4741b42c-35a5-407d-b5cc-c7d47b07ce0b-operator-scripts\") pod \"4741b42c-35a5-407d-b5cc-c7d47b07ce0b\" (UID: \"4741b42c-35a5-407d-b5cc-c7d47b07ce0b\") " Dec 03 12:06:05 crc kubenswrapper[5052]: I1203 12:06:05.921160 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4741b42c-35a5-407d-b5cc-c7d47b07ce0b-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "4741b42c-35a5-407d-b5cc-c7d47b07ce0b" (UID: "4741b42c-35a5-407d-b5cc-c7d47b07ce0b"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:06:05 crc kubenswrapper[5052]: I1203 12:06:05.921666 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c2d35861-9e4b-4f53-b841-e18b26d575cc-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "c2d35861-9e4b-4f53-b841-e18b26d575cc" (UID: "c2d35861-9e4b-4f53-b841-e18b26d575cc"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:06:05 crc kubenswrapper[5052]: I1203 12:06:05.924807 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4741b42c-35a5-407d-b5cc-c7d47b07ce0b-kube-api-access-sdf5b" (OuterVolumeSpecName: "kube-api-access-sdf5b") pod "4741b42c-35a5-407d-b5cc-c7d47b07ce0b" (UID: "4741b42c-35a5-407d-b5cc-c7d47b07ce0b"). InnerVolumeSpecName "kube-api-access-sdf5b". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:06:05 crc kubenswrapper[5052]: I1203 12:06:05.927797 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c2d35861-9e4b-4f53-b841-e18b26d575cc-kube-api-access-79b27" (OuterVolumeSpecName: "kube-api-access-79b27") pod "c2d35861-9e4b-4f53-b841-e18b26d575cc" (UID: "c2d35861-9e4b-4f53-b841-e18b26d575cc"). InnerVolumeSpecName "kube-api-access-79b27". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:06:06 crc kubenswrapper[5052]: I1203 12:06:06.022248 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b2v2r\" (UniqueName: \"kubernetes.io/projected/fdc5eec2-010f-4de6-a578-e5bda587d218-kube-api-access-b2v2r\") pod \"fdc5eec2-010f-4de6-a578-e5bda587d218\" (UID: \"fdc5eec2-010f-4de6-a578-e5bda587d218\") " Dec 03 12:06:06 crc kubenswrapper[5052]: I1203 12:06:06.022461 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/fdc5eec2-010f-4de6-a578-e5bda587d218-operator-scripts\") pod \"fdc5eec2-010f-4de6-a578-e5bda587d218\" (UID: \"fdc5eec2-010f-4de6-a578-e5bda587d218\") " Dec 03 12:06:06 crc kubenswrapper[5052]: I1203 12:06:06.023098 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fdc5eec2-010f-4de6-a578-e5bda587d218-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "fdc5eec2-010f-4de6-a578-e5bda587d218" (UID: "fdc5eec2-010f-4de6-a578-e5bda587d218"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:06:06 crc kubenswrapper[5052]: I1203 12:06:06.024030 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/fdc5eec2-010f-4de6-a578-e5bda587d218-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:06:06 crc kubenswrapper[5052]: I1203 12:06:06.024080 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-79b27\" (UniqueName: \"kubernetes.io/projected/c2d35861-9e4b-4f53-b841-e18b26d575cc-kube-api-access-79b27\") on node \"crc\" DevicePath \"\"" Dec 03 12:06:06 crc kubenswrapper[5052]: I1203 12:06:06.024096 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/4741b42c-35a5-407d-b5cc-c7d47b07ce0b-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:06:06 crc kubenswrapper[5052]: I1203 12:06:06.024107 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/c2d35861-9e4b-4f53-b841-e18b26d575cc-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:06:06 crc kubenswrapper[5052]: I1203 12:06:06.024135 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-sdf5b\" (UniqueName: \"kubernetes.io/projected/4741b42c-35a5-407d-b5cc-c7d47b07ce0b-kube-api-access-sdf5b\") on node \"crc\" DevicePath \"\"" Dec 03 12:06:06 crc kubenswrapper[5052]: I1203 12:06:06.030727 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fdc5eec2-010f-4de6-a578-e5bda587d218-kube-api-access-b2v2r" (OuterVolumeSpecName: "kube-api-access-b2v2r") pod "fdc5eec2-010f-4de6-a578-e5bda587d218" (UID: "fdc5eec2-010f-4de6-a578-e5bda587d218"). InnerVolumeSpecName "kube-api-access-b2v2r". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:06:06 crc kubenswrapper[5052]: I1203 12:06:06.113772 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-db-create-8kvhf" Dec 03 12:06:06 crc kubenswrapper[5052]: I1203 12:06:06.113789 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-db-create-8kvhf" event={"ID":"c2d35861-9e4b-4f53-b841-e18b26d575cc","Type":"ContainerDied","Data":"f21bab82d016786012ffb7e503fe357492f649d072f86fdfa425c16e420f37a6"} Dec 03 12:06:06 crc kubenswrapper[5052]: I1203 12:06:06.114731 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f21bab82d016786012ffb7e503fe357492f649d072f86fdfa425c16e420f37a6" Dec 03 12:06:06 crc kubenswrapper[5052]: I1203 12:06:06.120066 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"cf64fec7-c528-4a2c-89b4-a43851b7a903","Type":"ContainerStarted","Data":"459242164e59023a0febb3c309fe5460a0f01a81f08d4860c035db225bbea37e"} Dec 03 12:06:06 crc kubenswrapper[5052]: I1203 12:06:06.125756 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-48e2-account-create-update-dcqjr" event={"ID":"3af45ac5-7939-4525-891d-8ca1f5e697bf","Type":"ContainerDied","Data":"82043230ef7ae8dd1dd50f6f5b878d8b8863c2d06a0888d357f8a283dff185da"} Dec 03 12:06:06 crc kubenswrapper[5052]: I1203 12:06:06.125834 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="82043230ef7ae8dd1dd50f6f5b878d8b8863c2d06a0888d357f8a283dff185da" Dec 03 12:06:06 crc kubenswrapper[5052]: I1203 12:06:06.125839 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-b2v2r\" (UniqueName: \"kubernetes.io/projected/fdc5eec2-010f-4de6-a578-e5bda587d218-kube-api-access-b2v2r\") on node \"crc\" DevicePath \"\"" Dec 03 12:06:06 crc kubenswrapper[5052]: I1203 12:06:06.125923 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-48e2-account-create-update-dcqjr" Dec 03 12:06:06 crc kubenswrapper[5052]: I1203 12:06:06.130589 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-db-create-t6q5w" event={"ID":"4741b42c-35a5-407d-b5cc-c7d47b07ce0b","Type":"ContainerDied","Data":"a3ecfc69fe7556045bc11cfa634434b088b5005e80dc3e662b4531ed46b6b4a7"} Dec 03 12:06:06 crc kubenswrapper[5052]: I1203 12:06:06.130609 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-db-create-t6q5w" Dec 03 12:06:06 crc kubenswrapper[5052]: I1203 12:06:06.130628 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a3ecfc69fe7556045bc11cfa634434b088b5005e80dc3e662b4531ed46b6b4a7" Dec 03 12:06:06 crc kubenswrapper[5052]: I1203 12:06:06.133484 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-c2b9-account-create-update-hlm9w" Dec 03 12:06:06 crc kubenswrapper[5052]: I1203 12:06:06.133485 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-c2b9-account-create-update-hlm9w" event={"ID":"fdc5eec2-010f-4de6-a578-e5bda587d218","Type":"ContainerDied","Data":"1b1bc7e9b4a8a31272dab226d5ac1f8c188f27c696324074fead8869af564364"} Dec 03 12:06:06 crc kubenswrapper[5052]: I1203 12:06:06.133550 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1b1bc7e9b4a8a31272dab226d5ac1f8c188f27c696324074fead8869af564364" Dec 03 12:06:06 crc kubenswrapper[5052]: I1203 12:06:06.135671 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"4a2a1881-e61a-4a99-a2f3-b614aeed81d1","Type":"ContainerStarted","Data":"495f5d9f1bab82087b2a568fa419ea815529bd9df5fa5b0398f027035e5b6b4a"} Dec 03 12:06:06 crc kubenswrapper[5052]: I1203 12:06:06.137857 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"bef08f40-1b01-406f-af05-7b63e0b6e5e7","Type":"ContainerStarted","Data":"df8444be8224ec47d3e75542bba9c0aaa28cc5e0ba5b753982c10dfdcf96b872"} Dec 03 12:06:06 crc kubenswrapper[5052]: I1203 12:06:06.137902 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"bef08f40-1b01-406f-af05-7b63e0b6e5e7","Type":"ContainerStarted","Data":"62853e68ee96c13bce5c6927e9083bfe1dca5e094c03354b4a342f37a5ac3ce4"} Dec 03 12:06:06 crc kubenswrapper[5052]: I1203 12:06:06.168803 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/glance-default-external-api-0" podStartSLOduration=3.16877814 podStartE2EDuration="3.16877814s" podCreationTimestamp="2025-12-03 12:06:03 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:06:06.161226365 +0000 UTC m=+2756.749468139" watchObservedRunningTime="2025-12-03 12:06:06.16877814 +0000 UTC m=+2756.757019904" Dec 03 12:06:06 crc kubenswrapper[5052]: I1203 12:06:06.631637 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/swift-proxy-9bb7db5d-5q7wc" Dec 03 12:06:06 crc kubenswrapper[5052]: I1203 12:06:06.637013 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/swift-proxy-9bb7db5d-5q7wc" Dec 03 12:06:06 crc kubenswrapper[5052]: I1203 12:06:06.637121 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-d294-account-create-update-tm6pc" Dec 03 12:06:06 crc kubenswrapper[5052]: I1203 12:06:06.739263 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/53e80028-95b5-48c8-96ce-f6e40ed916cf-operator-scripts\") pod \"53e80028-95b5-48c8-96ce-f6e40ed916cf\" (UID: \"53e80028-95b5-48c8-96ce-f6e40ed916cf\") " Dec 03 12:06:06 crc kubenswrapper[5052]: I1203 12:06:06.739642 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-68hkn\" (UniqueName: \"kubernetes.io/projected/53e80028-95b5-48c8-96ce-f6e40ed916cf-kube-api-access-68hkn\") pod \"53e80028-95b5-48c8-96ce-f6e40ed916cf\" (UID: \"53e80028-95b5-48c8-96ce-f6e40ed916cf\") " Dec 03 12:06:06 crc kubenswrapper[5052]: I1203 12:06:06.741465 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/53e80028-95b5-48c8-96ce-f6e40ed916cf-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "53e80028-95b5-48c8-96ce-f6e40ed916cf" (UID: "53e80028-95b5-48c8-96ce-f6e40ed916cf"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:06:06 crc kubenswrapper[5052]: I1203 12:06:06.761972 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/53e80028-95b5-48c8-96ce-f6e40ed916cf-kube-api-access-68hkn" (OuterVolumeSpecName: "kube-api-access-68hkn") pod "53e80028-95b5-48c8-96ce-f6e40ed916cf" (UID: "53e80028-95b5-48c8-96ce-f6e40ed916cf"). InnerVolumeSpecName "kube-api-access-68hkn". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:06:06 crc kubenswrapper[5052]: I1203 12:06:06.842129 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-68hkn\" (UniqueName: \"kubernetes.io/projected/53e80028-95b5-48c8-96ce-f6e40ed916cf-kube-api-access-68hkn\") on node \"crc\" DevicePath \"\"" Dec 03 12:06:06 crc kubenswrapper[5052]: I1203 12:06:06.842165 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/53e80028-95b5-48c8-96ce-f6e40ed916cf-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:06:07 crc kubenswrapper[5052]: I1203 12:06:07.150165 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"4a2a1881-e61a-4a99-a2f3-b614aeed81d1","Type":"ContainerStarted","Data":"e3e2ee8f299225cb3b6df3bf35cd7781dcb1aa91cfcdbcdc6d9ceee107ca48ce"} Dec 03 12:06:07 crc kubenswrapper[5052]: I1203 12:06:07.153299 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-d294-account-create-update-tm6pc" event={"ID":"53e80028-95b5-48c8-96ce-f6e40ed916cf","Type":"ContainerDied","Data":"36f0039208eefa40a1e2cff26434aa005d86ce83fbff4e16b3d7771b8044306b"} Dec 03 12:06:07 crc kubenswrapper[5052]: I1203 12:06:07.153345 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="36f0039208eefa40a1e2cff26434aa005d86ce83fbff4e16b3d7771b8044306b" Dec 03 12:06:07 crc kubenswrapper[5052]: I1203 12:06:07.153437 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-d294-account-create-update-tm6pc" Dec 03 12:06:08 crc kubenswrapper[5052]: I1203 12:06:08.166849 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"4a2a1881-e61a-4a99-a2f3-b614aeed81d1","Type":"ContainerStarted","Data":"95dbf47cff378e5b90c76418048911cfa106d1ea36f9592e66a0f3a368e19e25"} Dec 03 12:06:08 crc kubenswrapper[5052]: I1203 12:06:08.170744 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"cf64fec7-c528-4a2c-89b4-a43851b7a903","Type":"ContainerStarted","Data":"942901864f5e6d6f46a75d5cf0fdd500807f0f99fb7255dc85c95281b59017f2"} Dec 03 12:06:08 crc kubenswrapper[5052]: I1203 12:06:08.170971 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:06:08 crc kubenswrapper[5052]: I1203 12:06:08.195169 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/glance-default-internal-api-0" podStartSLOduration=4.195142483 podStartE2EDuration="4.195142483s" podCreationTimestamp="2025-12-03 12:06:04 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:06:08.193043763 +0000 UTC m=+2758.781285527" watchObservedRunningTime="2025-12-03 12:06:08.195142483 +0000 UTC m=+2758.783384247" Dec 03 12:06:08 crc kubenswrapper[5052]: I1203 12:06:08.216690 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/ceilometer-0" podStartSLOduration=2.332638959 podStartE2EDuration="7.216673779s" podCreationTimestamp="2025-12-03 12:06:01 +0000 UTC" firstStartedPulling="2025-12-03 12:06:02.891591219 +0000 UTC m=+2753.479832993" lastFinishedPulling="2025-12-03 12:06:07.775626059 +0000 UTC m=+2758.363867813" observedRunningTime="2025-12-03 12:06:08.213270402 +0000 UTC m=+2758.801512166" watchObservedRunningTime="2025-12-03 12:06:08.216673779 +0000 UTC m=+2758.804915563" Dec 03 12:06:08 crc kubenswrapper[5052]: I1203 12:06:08.822771 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/neutron-78f8cc6c7-cc8t6" Dec 03 12:06:11 crc kubenswrapper[5052]: I1203 12:06:11.143445 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/neutron-5d6b56fbdb-ht2b8" Dec 03 12:06:11 crc kubenswrapper[5052]: I1203 12:06:11.196471 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/neutron-78f8cc6c7-cc8t6"] Dec 03 12:06:11 crc kubenswrapper[5052]: I1203 12:06:11.200647 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/neutron-78f8cc6c7-cc8t6" podUID="c3489d6f-c371-4252-9a13-a055a4a55cbf" containerName="neutron-api" containerID="cri-o://ca0d48f58970de4e1adf1b2ca5b2f51208b9ac83d5b1bf8d4e9788aa6ad57153" gracePeriod=30 Dec 03 12:06:11 crc kubenswrapper[5052]: I1203 12:06:11.200703 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/neutron-78f8cc6c7-cc8t6" podUID="c3489d6f-c371-4252-9a13-a055a4a55cbf" containerName="neutron-httpd" containerID="cri-o://ea4edfbe541d4bc73a70c177586471d5c25b2291432f54d5ac0c1ec500004cd1" gracePeriod=30 Dec 03 12:06:11 crc kubenswrapper[5052]: I1203 12:06:11.292595 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-db-sync-smfjz"] Dec 03 12:06:11 crc kubenswrapper[5052]: E1203 12:06:11.293087 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="53e80028-95b5-48c8-96ce-f6e40ed916cf" containerName="mariadb-account-create-update" Dec 03 12:06:11 crc kubenswrapper[5052]: I1203 12:06:11.293105 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="53e80028-95b5-48c8-96ce-f6e40ed916cf" containerName="mariadb-account-create-update" Dec 03 12:06:11 crc kubenswrapper[5052]: E1203 12:06:11.293115 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="3af45ac5-7939-4525-891d-8ca1f5e697bf" containerName="mariadb-account-create-update" Dec 03 12:06:11 crc kubenswrapper[5052]: I1203 12:06:11.293122 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="3af45ac5-7939-4525-891d-8ca1f5e697bf" containerName="mariadb-account-create-update" Dec 03 12:06:11 crc kubenswrapper[5052]: E1203 12:06:11.293131 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="c2d35861-9e4b-4f53-b841-e18b26d575cc" containerName="mariadb-database-create" Dec 03 12:06:11 crc kubenswrapper[5052]: I1203 12:06:11.293138 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="c2d35861-9e4b-4f53-b841-e18b26d575cc" containerName="mariadb-database-create" Dec 03 12:06:11 crc kubenswrapper[5052]: E1203 12:06:11.293148 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="4741b42c-35a5-407d-b5cc-c7d47b07ce0b" containerName="mariadb-database-create" Dec 03 12:06:11 crc kubenswrapper[5052]: I1203 12:06:11.293155 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="4741b42c-35a5-407d-b5cc-c7d47b07ce0b" containerName="mariadb-database-create" Dec 03 12:06:11 crc kubenswrapper[5052]: E1203 12:06:11.293191 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="fdc5eec2-010f-4de6-a578-e5bda587d218" containerName="mariadb-account-create-update" Dec 03 12:06:11 crc kubenswrapper[5052]: I1203 12:06:11.293197 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="fdc5eec2-010f-4de6-a578-e5bda587d218" containerName="mariadb-account-create-update" Dec 03 12:06:11 crc kubenswrapper[5052]: I1203 12:06:11.293374 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="c2d35861-9e4b-4f53-b841-e18b26d575cc" containerName="mariadb-database-create" Dec 03 12:06:11 crc kubenswrapper[5052]: I1203 12:06:11.293395 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="fdc5eec2-010f-4de6-a578-e5bda587d218" containerName="mariadb-account-create-update" Dec 03 12:06:11 crc kubenswrapper[5052]: I1203 12:06:11.293403 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="4741b42c-35a5-407d-b5cc-c7d47b07ce0b" containerName="mariadb-database-create" Dec 03 12:06:11 crc kubenswrapper[5052]: I1203 12:06:11.293418 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="3af45ac5-7939-4525-891d-8ca1f5e697bf" containerName="mariadb-account-create-update" Dec 03 12:06:11 crc kubenswrapper[5052]: I1203 12:06:11.293427 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="53e80028-95b5-48c8-96ce-f6e40ed916cf" containerName="mariadb-account-create-update" Dec 03 12:06:11 crc kubenswrapper[5052]: I1203 12:06:11.294038 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-smfjz" Dec 03 12:06:11 crc kubenswrapper[5052]: I1203 12:06:11.295673 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell0-conductor-config-data" Dec 03 12:06:11 crc kubenswrapper[5052]: I1203 12:06:11.296535 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell0-conductor-scripts" Dec 03 12:06:11 crc kubenswrapper[5052]: I1203 12:06:11.296753 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-nova-dockercfg-55b29" Dec 03 12:06:11 crc kubenswrapper[5052]: I1203 12:06:11.317572 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-db-sync-smfjz"] Dec 03 12:06:11 crc kubenswrapper[5052]: I1203 12:06:11.332996 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zfdrn\" (UniqueName: \"kubernetes.io/projected/895b843a-707e-4cc5-8991-501116bd2fe3-kube-api-access-zfdrn\") pod \"nova-cell0-conductor-db-sync-smfjz\" (UID: \"895b843a-707e-4cc5-8991-501116bd2fe3\") " pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-smfjz" Dec 03 12:06:11 crc kubenswrapper[5052]: I1203 12:06:11.333096 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/895b843a-707e-4cc5-8991-501116bd2fe3-scripts\") pod \"nova-cell0-conductor-db-sync-smfjz\" (UID: \"895b843a-707e-4cc5-8991-501116bd2fe3\") " pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-smfjz" Dec 03 12:06:11 crc kubenswrapper[5052]: I1203 12:06:11.333274 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/895b843a-707e-4cc5-8991-501116bd2fe3-config-data\") pod \"nova-cell0-conductor-db-sync-smfjz\" (UID: \"895b843a-707e-4cc5-8991-501116bd2fe3\") " pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-smfjz" Dec 03 12:06:11 crc kubenswrapper[5052]: I1203 12:06:11.333299 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/895b843a-707e-4cc5-8991-501116bd2fe3-combined-ca-bundle\") pod \"nova-cell0-conductor-db-sync-smfjz\" (UID: \"895b843a-707e-4cc5-8991-501116bd2fe3\") " pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-smfjz" Dec 03 12:06:11 crc kubenswrapper[5052]: I1203 12:06:11.435935 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/895b843a-707e-4cc5-8991-501116bd2fe3-config-data\") pod \"nova-cell0-conductor-db-sync-smfjz\" (UID: \"895b843a-707e-4cc5-8991-501116bd2fe3\") " pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-smfjz" Dec 03 12:06:11 crc kubenswrapper[5052]: I1203 12:06:11.436021 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/895b843a-707e-4cc5-8991-501116bd2fe3-combined-ca-bundle\") pod \"nova-cell0-conductor-db-sync-smfjz\" (UID: \"895b843a-707e-4cc5-8991-501116bd2fe3\") " pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-smfjz" Dec 03 12:06:11 crc kubenswrapper[5052]: I1203 12:06:11.437033 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-zfdrn\" (UniqueName: \"kubernetes.io/projected/895b843a-707e-4cc5-8991-501116bd2fe3-kube-api-access-zfdrn\") pod \"nova-cell0-conductor-db-sync-smfjz\" (UID: \"895b843a-707e-4cc5-8991-501116bd2fe3\") " pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-smfjz" Dec 03 12:06:11 crc kubenswrapper[5052]: I1203 12:06:11.437455 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/895b843a-707e-4cc5-8991-501116bd2fe3-scripts\") pod \"nova-cell0-conductor-db-sync-smfjz\" (UID: \"895b843a-707e-4cc5-8991-501116bd2fe3\") " pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-smfjz" Dec 03 12:06:11 crc kubenswrapper[5052]: I1203 12:06:11.448291 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/895b843a-707e-4cc5-8991-501116bd2fe3-scripts\") pod \"nova-cell0-conductor-db-sync-smfjz\" (UID: \"895b843a-707e-4cc5-8991-501116bd2fe3\") " pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-smfjz" Dec 03 12:06:11 crc kubenswrapper[5052]: I1203 12:06:11.448399 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/895b843a-707e-4cc5-8991-501116bd2fe3-config-data\") pod \"nova-cell0-conductor-db-sync-smfjz\" (UID: \"895b843a-707e-4cc5-8991-501116bd2fe3\") " pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-smfjz" Dec 03 12:06:11 crc kubenswrapper[5052]: I1203 12:06:11.448820 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/895b843a-707e-4cc5-8991-501116bd2fe3-combined-ca-bundle\") pod \"nova-cell0-conductor-db-sync-smfjz\" (UID: \"895b843a-707e-4cc5-8991-501116bd2fe3\") " pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-smfjz" Dec 03 12:06:11 crc kubenswrapper[5052]: I1203 12:06:11.460047 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-zfdrn\" (UniqueName: \"kubernetes.io/projected/895b843a-707e-4cc5-8991-501116bd2fe3-kube-api-access-zfdrn\") pod \"nova-cell0-conductor-db-sync-smfjz\" (UID: \"895b843a-707e-4cc5-8991-501116bd2fe3\") " pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-smfjz" Dec 03 12:06:11 crc kubenswrapper[5052]: I1203 12:06:11.614199 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-smfjz" Dec 03 12:06:12 crc kubenswrapper[5052]: I1203 12:06:12.105475 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-db-sync-smfjz"] Dec 03 12:06:12 crc kubenswrapper[5052]: I1203 12:06:12.215885 5052 generic.go:334] "Generic (PLEG): container finished" podID="c3489d6f-c371-4252-9a13-a055a4a55cbf" containerID="ea4edfbe541d4bc73a70c177586471d5c25b2291432f54d5ac0c1ec500004cd1" exitCode=0 Dec 03 12:06:12 crc kubenswrapper[5052]: I1203 12:06:12.215964 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-78f8cc6c7-cc8t6" event={"ID":"c3489d6f-c371-4252-9a13-a055a4a55cbf","Type":"ContainerDied","Data":"ea4edfbe541d4bc73a70c177586471d5c25b2291432f54d5ac0c1ec500004cd1"} Dec 03 12:06:12 crc kubenswrapper[5052]: I1203 12:06:12.217375 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-smfjz" event={"ID":"895b843a-707e-4cc5-8991-501116bd2fe3","Type":"ContainerStarted","Data":"95867ac62c0a7908764d6ddd89c5a27b15543c21ad0130c3156862b495ccd48a"} Dec 03 12:06:13 crc kubenswrapper[5052]: I1203 12:06:13.228194 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-smfjz" event={"ID":"895b843a-707e-4cc5-8991-501116bd2fe3","Type":"ContainerStarted","Data":"737462a16fc5e657fb057be690e9512c155f3a35e26b18e1f261c004033bd4d8"} Dec 03 12:06:13 crc kubenswrapper[5052]: I1203 12:06:13.249618 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-smfjz" podStartSLOduration=2.249600076 podStartE2EDuration="2.249600076s" podCreationTimestamp="2025-12-03 12:06:11 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:06:13.245604372 +0000 UTC m=+2763.833846146" watchObservedRunningTime="2025-12-03 12:06:13.249600076 +0000 UTC m=+2763.837841840" Dec 03 12:06:13 crc kubenswrapper[5052]: I1203 12:06:13.633375 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:06:13 crc kubenswrapper[5052]: I1203 12:06:13.634644 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:06:13 crc kubenswrapper[5052]: I1203 12:06:13.670964 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:06:13 crc kubenswrapper[5052]: I1203 12:06:13.685206 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:06:14 crc kubenswrapper[5052]: I1203 12:06:14.239663 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:06:14 crc kubenswrapper[5052]: I1203 12:06:14.239714 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:06:15 crc kubenswrapper[5052]: I1203 12:06:15.067000 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:06:15 crc kubenswrapper[5052]: I1203 12:06:15.067356 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:06:15 crc kubenswrapper[5052]: I1203 12:06:15.101303 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:06:15 crc kubenswrapper[5052]: I1203 12:06:15.110810 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:06:15 crc kubenswrapper[5052]: I1203 12:06:15.250728 5052 generic.go:334] "Generic (PLEG): container finished" podID="c3489d6f-c371-4252-9a13-a055a4a55cbf" containerID="ca0d48f58970de4e1adf1b2ca5b2f51208b9ac83d5b1bf8d4e9788aa6ad57153" exitCode=0 Dec 03 12:06:15 crc kubenswrapper[5052]: I1203 12:06:15.250812 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-78f8cc6c7-cc8t6" event={"ID":"c3489d6f-c371-4252-9a13-a055a4a55cbf","Type":"ContainerDied","Data":"ca0d48f58970de4e1adf1b2ca5b2f51208b9ac83d5b1bf8d4e9788aa6ad57153"} Dec 03 12:06:15 crc kubenswrapper[5052]: I1203 12:06:15.251215 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:06:15 crc kubenswrapper[5052]: I1203 12:06:15.251235 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:06:15 crc kubenswrapper[5052]: I1203 12:06:15.876441 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-78f8cc6c7-cc8t6" Dec 03 12:06:16 crc kubenswrapper[5052]: I1203 12:06:16.049887 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c3489d6f-c371-4252-9a13-a055a4a55cbf-combined-ca-bundle\") pod \"c3489d6f-c371-4252-9a13-a055a4a55cbf\" (UID: \"c3489d6f-c371-4252-9a13-a055a4a55cbf\") " Dec 03 12:06:16 crc kubenswrapper[5052]: I1203 12:06:16.050147 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/c3489d6f-c371-4252-9a13-a055a4a55cbf-ovndb-tls-certs\") pod \"c3489d6f-c371-4252-9a13-a055a4a55cbf\" (UID: \"c3489d6f-c371-4252-9a13-a055a4a55cbf\") " Dec 03 12:06:16 crc kubenswrapper[5052]: I1203 12:06:16.050251 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v98bp\" (UniqueName: \"kubernetes.io/projected/c3489d6f-c371-4252-9a13-a055a4a55cbf-kube-api-access-v98bp\") pod \"c3489d6f-c371-4252-9a13-a055a4a55cbf\" (UID: \"c3489d6f-c371-4252-9a13-a055a4a55cbf\") " Dec 03 12:06:16 crc kubenswrapper[5052]: I1203 12:06:16.050274 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/c3489d6f-c371-4252-9a13-a055a4a55cbf-httpd-config\") pod \"c3489d6f-c371-4252-9a13-a055a4a55cbf\" (UID: \"c3489d6f-c371-4252-9a13-a055a4a55cbf\") " Dec 03 12:06:16 crc kubenswrapper[5052]: I1203 12:06:16.050306 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/c3489d6f-c371-4252-9a13-a055a4a55cbf-config\") pod \"c3489d6f-c371-4252-9a13-a055a4a55cbf\" (UID: \"c3489d6f-c371-4252-9a13-a055a4a55cbf\") " Dec 03 12:06:16 crc kubenswrapper[5052]: I1203 12:06:16.058044 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c3489d6f-c371-4252-9a13-a055a4a55cbf-httpd-config" (OuterVolumeSpecName: "httpd-config") pod "c3489d6f-c371-4252-9a13-a055a4a55cbf" (UID: "c3489d6f-c371-4252-9a13-a055a4a55cbf"). InnerVolumeSpecName "httpd-config". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:06:16 crc kubenswrapper[5052]: I1203 12:06:16.064191 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c3489d6f-c371-4252-9a13-a055a4a55cbf-kube-api-access-v98bp" (OuterVolumeSpecName: "kube-api-access-v98bp") pod "c3489d6f-c371-4252-9a13-a055a4a55cbf" (UID: "c3489d6f-c371-4252-9a13-a055a4a55cbf"). InnerVolumeSpecName "kube-api-access-v98bp". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:06:16 crc kubenswrapper[5052]: I1203 12:06:16.130623 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c3489d6f-c371-4252-9a13-a055a4a55cbf-config" (OuterVolumeSpecName: "config") pod "c3489d6f-c371-4252-9a13-a055a4a55cbf" (UID: "c3489d6f-c371-4252-9a13-a055a4a55cbf"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:06:16 crc kubenswrapper[5052]: I1203 12:06:16.133637 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c3489d6f-c371-4252-9a13-a055a4a55cbf-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "c3489d6f-c371-4252-9a13-a055a4a55cbf" (UID: "c3489d6f-c371-4252-9a13-a055a4a55cbf"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:06:16 crc kubenswrapper[5052]: I1203 12:06:16.152775 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-v98bp\" (UniqueName: \"kubernetes.io/projected/c3489d6f-c371-4252-9a13-a055a4a55cbf-kube-api-access-v98bp\") on node \"crc\" DevicePath \"\"" Dec 03 12:06:16 crc kubenswrapper[5052]: I1203 12:06:16.152813 5052 reconciler_common.go:293] "Volume detached for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/c3489d6f-c371-4252-9a13-a055a4a55cbf-httpd-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:06:16 crc kubenswrapper[5052]: I1203 12:06:16.152828 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/c3489d6f-c371-4252-9a13-a055a4a55cbf-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:06:16 crc kubenswrapper[5052]: I1203 12:06:16.152840 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c3489d6f-c371-4252-9a13-a055a4a55cbf-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:06:16 crc kubenswrapper[5052]: I1203 12:06:16.174631 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c3489d6f-c371-4252-9a13-a055a4a55cbf-ovndb-tls-certs" (OuterVolumeSpecName: "ovndb-tls-certs") pod "c3489d6f-c371-4252-9a13-a055a4a55cbf" (UID: "c3489d6f-c371-4252-9a13-a055a4a55cbf"). InnerVolumeSpecName "ovndb-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:06:16 crc kubenswrapper[5052]: I1203 12:06:16.254373 5052 reconciler_common.go:293] "Volume detached for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/c3489d6f-c371-4252-9a13-a055a4a55cbf-ovndb-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:06:16 crc kubenswrapper[5052]: I1203 12:06:16.268996 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-78f8cc6c7-cc8t6" Dec 03 12:06:16 crc kubenswrapper[5052]: I1203 12:06:16.269112 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-78f8cc6c7-cc8t6" event={"ID":"c3489d6f-c371-4252-9a13-a055a4a55cbf","Type":"ContainerDied","Data":"173db296bf1158238a3f169e35341dfbae99469e380d1f026a323d4ad8cf09d2"} Dec 03 12:06:16 crc kubenswrapper[5052]: I1203 12:06:16.269269 5052 scope.go:117] "RemoveContainer" containerID="ea4edfbe541d4bc73a70c177586471d5c25b2291432f54d5ac0c1ec500004cd1" Dec 03 12:06:16 crc kubenswrapper[5052]: I1203 12:06:16.302504 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/neutron-78f8cc6c7-cc8t6"] Dec 03 12:06:16 crc kubenswrapper[5052]: I1203 12:06:16.311677 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/neutron-78f8cc6c7-cc8t6"] Dec 03 12:06:16 crc kubenswrapper[5052]: I1203 12:06:16.319800 5052 scope.go:117] "RemoveContainer" containerID="ca0d48f58970de4e1adf1b2ca5b2f51208b9ac83d5b1bf8d4e9788aa6ad57153" Dec 03 12:06:16 crc kubenswrapper[5052]: I1203 12:06:16.443359 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:06:16 crc kubenswrapper[5052]: I1203 12:06:16.443495 5052 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Dec 03 12:06:16 crc kubenswrapper[5052]: I1203 12:06:16.447343 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:06:17 crc kubenswrapper[5052]: I1203 12:06:17.363523 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:06:17 crc kubenswrapper[5052]: I1203 12:06:17.363873 5052 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Dec 03 12:06:17 crc kubenswrapper[5052]: I1203 12:06:17.571390 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:06:17 crc kubenswrapper[5052]: I1203 12:06:17.690691 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c3489d6f-c371-4252-9a13-a055a4a55cbf" path="/var/lib/kubelet/pods/c3489d6f-c371-4252-9a13-a055a4a55cbf/volumes" Dec 03 12:06:22 crc kubenswrapper[5052]: I1203 12:06:22.418199 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:06:22 crc kubenswrapper[5052]: I1203 12:06:22.419145 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="cf64fec7-c528-4a2c-89b4-a43851b7a903" containerName="ceilometer-central-agent" containerID="cri-o://8e6902d44b83c5eec3a438c9b36c5e9341283dffc8678b21795a0f7d6a5bafc4" gracePeriod=30 Dec 03 12:06:22 crc kubenswrapper[5052]: I1203 12:06:22.419301 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="cf64fec7-c528-4a2c-89b4-a43851b7a903" containerName="proxy-httpd" containerID="cri-o://942901864f5e6d6f46a75d5cf0fdd500807f0f99fb7255dc85c95281b59017f2" gracePeriod=30 Dec 03 12:06:22 crc kubenswrapper[5052]: I1203 12:06:22.419662 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="cf64fec7-c528-4a2c-89b4-a43851b7a903" containerName="sg-core" containerID="cri-o://459242164e59023a0febb3c309fe5460a0f01a81f08d4860c035db225bbea37e" gracePeriod=30 Dec 03 12:06:22 crc kubenswrapper[5052]: I1203 12:06:22.419753 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="cf64fec7-c528-4a2c-89b4-a43851b7a903" containerName="ceilometer-notification-agent" containerID="cri-o://1e7c6cfd093da01c0b0e07dd0484985317870469414dddeb15a45b6655ad7f58" gracePeriod=30 Dec 03 12:06:22 crc kubenswrapper[5052]: I1203 12:06:22.427581 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/ceilometer-0" podUID="cf64fec7-c528-4a2c-89b4-a43851b7a903" containerName="proxy-httpd" probeResult="failure" output="Get \"http://10.217.0.170:3000/\": EOF" Dec 03 12:06:23 crc kubenswrapper[5052]: I1203 12:06:23.342565 5052 generic.go:334] "Generic (PLEG): container finished" podID="cf64fec7-c528-4a2c-89b4-a43851b7a903" containerID="942901864f5e6d6f46a75d5cf0fdd500807f0f99fb7255dc85c95281b59017f2" exitCode=0 Dec 03 12:06:23 crc kubenswrapper[5052]: I1203 12:06:23.342842 5052 generic.go:334] "Generic (PLEG): container finished" podID="cf64fec7-c528-4a2c-89b4-a43851b7a903" containerID="459242164e59023a0febb3c309fe5460a0f01a81f08d4860c035db225bbea37e" exitCode=2 Dec 03 12:06:23 crc kubenswrapper[5052]: I1203 12:06:23.342852 5052 generic.go:334] "Generic (PLEG): container finished" podID="cf64fec7-c528-4a2c-89b4-a43851b7a903" containerID="8e6902d44b83c5eec3a438c9b36c5e9341283dffc8678b21795a0f7d6a5bafc4" exitCode=0 Dec 03 12:06:23 crc kubenswrapper[5052]: I1203 12:06:23.342642 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"cf64fec7-c528-4a2c-89b4-a43851b7a903","Type":"ContainerDied","Data":"942901864f5e6d6f46a75d5cf0fdd500807f0f99fb7255dc85c95281b59017f2"} Dec 03 12:06:23 crc kubenswrapper[5052]: I1203 12:06:23.342926 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"cf64fec7-c528-4a2c-89b4-a43851b7a903","Type":"ContainerDied","Data":"459242164e59023a0febb3c309fe5460a0f01a81f08d4860c035db225bbea37e"} Dec 03 12:06:23 crc kubenswrapper[5052]: I1203 12:06:23.342940 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"cf64fec7-c528-4a2c-89b4-a43851b7a903","Type":"ContainerDied","Data":"8e6902d44b83c5eec3a438c9b36c5e9341283dffc8678b21795a0f7d6a5bafc4"} Dec 03 12:06:28 crc kubenswrapper[5052]: I1203 12:06:28.229445 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:06:28 crc kubenswrapper[5052]: I1203 12:06:28.309164 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/cf64fec7-c528-4a2c-89b4-a43851b7a903-run-httpd\") pod \"cf64fec7-c528-4a2c-89b4-a43851b7a903\" (UID: \"cf64fec7-c528-4a2c-89b4-a43851b7a903\") " Dec 03 12:06:28 crc kubenswrapper[5052]: I1203 12:06:28.309274 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/cf64fec7-c528-4a2c-89b4-a43851b7a903-scripts\") pod \"cf64fec7-c528-4a2c-89b4-a43851b7a903\" (UID: \"cf64fec7-c528-4a2c-89b4-a43851b7a903\") " Dec 03 12:06:28 crc kubenswrapper[5052]: I1203 12:06:28.309305 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/cf64fec7-c528-4a2c-89b4-a43851b7a903-log-httpd\") pod \"cf64fec7-c528-4a2c-89b4-a43851b7a903\" (UID: \"cf64fec7-c528-4a2c-89b4-a43851b7a903\") " Dec 03 12:06:28 crc kubenswrapper[5052]: I1203 12:06:28.309357 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tlr44\" (UniqueName: \"kubernetes.io/projected/cf64fec7-c528-4a2c-89b4-a43851b7a903-kube-api-access-tlr44\") pod \"cf64fec7-c528-4a2c-89b4-a43851b7a903\" (UID: \"cf64fec7-c528-4a2c-89b4-a43851b7a903\") " Dec 03 12:06:28 crc kubenswrapper[5052]: I1203 12:06:28.309389 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/cf64fec7-c528-4a2c-89b4-a43851b7a903-combined-ca-bundle\") pod \"cf64fec7-c528-4a2c-89b4-a43851b7a903\" (UID: \"cf64fec7-c528-4a2c-89b4-a43851b7a903\") " Dec 03 12:06:28 crc kubenswrapper[5052]: I1203 12:06:28.309408 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/cf64fec7-c528-4a2c-89b4-a43851b7a903-config-data\") pod \"cf64fec7-c528-4a2c-89b4-a43851b7a903\" (UID: \"cf64fec7-c528-4a2c-89b4-a43851b7a903\") " Dec 03 12:06:28 crc kubenswrapper[5052]: I1203 12:06:28.309446 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/cf64fec7-c528-4a2c-89b4-a43851b7a903-sg-core-conf-yaml\") pod \"cf64fec7-c528-4a2c-89b4-a43851b7a903\" (UID: \"cf64fec7-c528-4a2c-89b4-a43851b7a903\") " Dec 03 12:06:28 crc kubenswrapper[5052]: I1203 12:06:28.309746 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf64fec7-c528-4a2c-89b4-a43851b7a903-run-httpd" (OuterVolumeSpecName: "run-httpd") pod "cf64fec7-c528-4a2c-89b4-a43851b7a903" (UID: "cf64fec7-c528-4a2c-89b4-a43851b7a903"). InnerVolumeSpecName "run-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:06:28 crc kubenswrapper[5052]: I1203 12:06:28.310028 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf64fec7-c528-4a2c-89b4-a43851b7a903-log-httpd" (OuterVolumeSpecName: "log-httpd") pod "cf64fec7-c528-4a2c-89b4-a43851b7a903" (UID: "cf64fec7-c528-4a2c-89b4-a43851b7a903"). InnerVolumeSpecName "log-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:06:28 crc kubenswrapper[5052]: I1203 12:06:28.310378 5052 reconciler_common.go:293] "Volume detached for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/cf64fec7-c528-4a2c-89b4-a43851b7a903-run-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:06:28 crc kubenswrapper[5052]: I1203 12:06:28.310393 5052 reconciler_common.go:293] "Volume detached for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/cf64fec7-c528-4a2c-89b4-a43851b7a903-log-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:06:28 crc kubenswrapper[5052]: I1203 12:06:28.314484 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cf64fec7-c528-4a2c-89b4-a43851b7a903-kube-api-access-tlr44" (OuterVolumeSpecName: "kube-api-access-tlr44") pod "cf64fec7-c528-4a2c-89b4-a43851b7a903" (UID: "cf64fec7-c528-4a2c-89b4-a43851b7a903"). InnerVolumeSpecName "kube-api-access-tlr44". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:06:28 crc kubenswrapper[5052]: I1203 12:06:28.324953 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cf64fec7-c528-4a2c-89b4-a43851b7a903-scripts" (OuterVolumeSpecName: "scripts") pod "cf64fec7-c528-4a2c-89b4-a43851b7a903" (UID: "cf64fec7-c528-4a2c-89b4-a43851b7a903"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:06:28 crc kubenswrapper[5052]: I1203 12:06:28.338892 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cf64fec7-c528-4a2c-89b4-a43851b7a903-sg-core-conf-yaml" (OuterVolumeSpecName: "sg-core-conf-yaml") pod "cf64fec7-c528-4a2c-89b4-a43851b7a903" (UID: "cf64fec7-c528-4a2c-89b4-a43851b7a903"). InnerVolumeSpecName "sg-core-conf-yaml". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:06:28 crc kubenswrapper[5052]: I1203 12:06:28.406963 5052 generic.go:334] "Generic (PLEG): container finished" podID="cf64fec7-c528-4a2c-89b4-a43851b7a903" containerID="1e7c6cfd093da01c0b0e07dd0484985317870469414dddeb15a45b6655ad7f58" exitCode=0 Dec 03 12:06:28 crc kubenswrapper[5052]: I1203 12:06:28.407024 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"cf64fec7-c528-4a2c-89b4-a43851b7a903","Type":"ContainerDied","Data":"1e7c6cfd093da01c0b0e07dd0484985317870469414dddeb15a45b6655ad7f58"} Dec 03 12:06:28 crc kubenswrapper[5052]: I1203 12:06:28.407055 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"cf64fec7-c528-4a2c-89b4-a43851b7a903","Type":"ContainerDied","Data":"09a940af112f01defba900147a49855d50e201749b807d54ab4bc7d83924e88a"} Dec 03 12:06:28 crc kubenswrapper[5052]: I1203 12:06:28.407086 5052 scope.go:117] "RemoveContainer" containerID="942901864f5e6d6f46a75d5cf0fdd500807f0f99fb7255dc85c95281b59017f2" Dec 03 12:06:28 crc kubenswrapper[5052]: I1203 12:06:28.407312 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:06:28 crc kubenswrapper[5052]: I1203 12:06:28.414451 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/cf64fec7-c528-4a2c-89b4-a43851b7a903-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:06:28 crc kubenswrapper[5052]: I1203 12:06:28.414525 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-tlr44\" (UniqueName: \"kubernetes.io/projected/cf64fec7-c528-4a2c-89b4-a43851b7a903-kube-api-access-tlr44\") on node \"crc\" DevicePath \"\"" Dec 03 12:06:28 crc kubenswrapper[5052]: I1203 12:06:28.414544 5052 reconciler_common.go:293] "Volume detached for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/cf64fec7-c528-4a2c-89b4-a43851b7a903-sg-core-conf-yaml\") on node \"crc\" DevicePath \"\"" Dec 03 12:06:28 crc kubenswrapper[5052]: I1203 12:06:28.423408 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cf64fec7-c528-4a2c-89b4-a43851b7a903-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "cf64fec7-c528-4a2c-89b4-a43851b7a903" (UID: "cf64fec7-c528-4a2c-89b4-a43851b7a903"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:06:28 crc kubenswrapper[5052]: I1203 12:06:28.446109 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cf64fec7-c528-4a2c-89b4-a43851b7a903-config-data" (OuterVolumeSpecName: "config-data") pod "cf64fec7-c528-4a2c-89b4-a43851b7a903" (UID: "cf64fec7-c528-4a2c-89b4-a43851b7a903"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:06:28 crc kubenswrapper[5052]: I1203 12:06:28.516087 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/cf64fec7-c528-4a2c-89b4-a43851b7a903-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:06:28 crc kubenswrapper[5052]: I1203 12:06:28.516158 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/cf64fec7-c528-4a2c-89b4-a43851b7a903-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:06:28 crc kubenswrapper[5052]: I1203 12:06:28.552947 5052 scope.go:117] "RemoveContainer" containerID="459242164e59023a0febb3c309fe5460a0f01a81f08d4860c035db225bbea37e" Dec 03 12:06:28 crc kubenswrapper[5052]: I1203 12:06:28.575157 5052 scope.go:117] "RemoveContainer" containerID="1e7c6cfd093da01c0b0e07dd0484985317870469414dddeb15a45b6655ad7f58" Dec 03 12:06:28 crc kubenswrapper[5052]: I1203 12:06:28.597795 5052 scope.go:117] "RemoveContainer" containerID="8e6902d44b83c5eec3a438c9b36c5e9341283dffc8678b21795a0f7d6a5bafc4" Dec 03 12:06:28 crc kubenswrapper[5052]: I1203 12:06:28.621709 5052 scope.go:117] "RemoveContainer" containerID="942901864f5e6d6f46a75d5cf0fdd500807f0f99fb7255dc85c95281b59017f2" Dec 03 12:06:28 crc kubenswrapper[5052]: E1203 12:06:28.622180 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"942901864f5e6d6f46a75d5cf0fdd500807f0f99fb7255dc85c95281b59017f2\": container with ID starting with 942901864f5e6d6f46a75d5cf0fdd500807f0f99fb7255dc85c95281b59017f2 not found: ID does not exist" containerID="942901864f5e6d6f46a75d5cf0fdd500807f0f99fb7255dc85c95281b59017f2" Dec 03 12:06:28 crc kubenswrapper[5052]: I1203 12:06:28.622224 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"942901864f5e6d6f46a75d5cf0fdd500807f0f99fb7255dc85c95281b59017f2"} err="failed to get container status \"942901864f5e6d6f46a75d5cf0fdd500807f0f99fb7255dc85c95281b59017f2\": rpc error: code = NotFound desc = could not find container \"942901864f5e6d6f46a75d5cf0fdd500807f0f99fb7255dc85c95281b59017f2\": container with ID starting with 942901864f5e6d6f46a75d5cf0fdd500807f0f99fb7255dc85c95281b59017f2 not found: ID does not exist" Dec 03 12:06:28 crc kubenswrapper[5052]: I1203 12:06:28.622253 5052 scope.go:117] "RemoveContainer" containerID="459242164e59023a0febb3c309fe5460a0f01a81f08d4860c035db225bbea37e" Dec 03 12:06:28 crc kubenswrapper[5052]: E1203 12:06:28.622787 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"459242164e59023a0febb3c309fe5460a0f01a81f08d4860c035db225bbea37e\": container with ID starting with 459242164e59023a0febb3c309fe5460a0f01a81f08d4860c035db225bbea37e not found: ID does not exist" containerID="459242164e59023a0febb3c309fe5460a0f01a81f08d4860c035db225bbea37e" Dec 03 12:06:28 crc kubenswrapper[5052]: I1203 12:06:28.622824 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"459242164e59023a0febb3c309fe5460a0f01a81f08d4860c035db225bbea37e"} err="failed to get container status \"459242164e59023a0febb3c309fe5460a0f01a81f08d4860c035db225bbea37e\": rpc error: code = NotFound desc = could not find container \"459242164e59023a0febb3c309fe5460a0f01a81f08d4860c035db225bbea37e\": container with ID starting with 459242164e59023a0febb3c309fe5460a0f01a81f08d4860c035db225bbea37e not found: ID does not exist" Dec 03 12:06:28 crc kubenswrapper[5052]: I1203 12:06:28.622856 5052 scope.go:117] "RemoveContainer" containerID="1e7c6cfd093da01c0b0e07dd0484985317870469414dddeb15a45b6655ad7f58" Dec 03 12:06:28 crc kubenswrapper[5052]: E1203 12:06:28.623102 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1e7c6cfd093da01c0b0e07dd0484985317870469414dddeb15a45b6655ad7f58\": container with ID starting with 1e7c6cfd093da01c0b0e07dd0484985317870469414dddeb15a45b6655ad7f58 not found: ID does not exist" containerID="1e7c6cfd093da01c0b0e07dd0484985317870469414dddeb15a45b6655ad7f58" Dec 03 12:06:28 crc kubenswrapper[5052]: I1203 12:06:28.623128 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1e7c6cfd093da01c0b0e07dd0484985317870469414dddeb15a45b6655ad7f58"} err="failed to get container status \"1e7c6cfd093da01c0b0e07dd0484985317870469414dddeb15a45b6655ad7f58\": rpc error: code = NotFound desc = could not find container \"1e7c6cfd093da01c0b0e07dd0484985317870469414dddeb15a45b6655ad7f58\": container with ID starting with 1e7c6cfd093da01c0b0e07dd0484985317870469414dddeb15a45b6655ad7f58 not found: ID does not exist" Dec 03 12:06:28 crc kubenswrapper[5052]: I1203 12:06:28.623142 5052 scope.go:117] "RemoveContainer" containerID="8e6902d44b83c5eec3a438c9b36c5e9341283dffc8678b21795a0f7d6a5bafc4" Dec 03 12:06:28 crc kubenswrapper[5052]: E1203 12:06:28.623398 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8e6902d44b83c5eec3a438c9b36c5e9341283dffc8678b21795a0f7d6a5bafc4\": container with ID starting with 8e6902d44b83c5eec3a438c9b36c5e9341283dffc8678b21795a0f7d6a5bafc4 not found: ID does not exist" containerID="8e6902d44b83c5eec3a438c9b36c5e9341283dffc8678b21795a0f7d6a5bafc4" Dec 03 12:06:28 crc kubenswrapper[5052]: I1203 12:06:28.623436 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8e6902d44b83c5eec3a438c9b36c5e9341283dffc8678b21795a0f7d6a5bafc4"} err="failed to get container status \"8e6902d44b83c5eec3a438c9b36c5e9341283dffc8678b21795a0f7d6a5bafc4\": rpc error: code = NotFound desc = could not find container \"8e6902d44b83c5eec3a438c9b36c5e9341283dffc8678b21795a0f7d6a5bafc4\": container with ID starting with 8e6902d44b83c5eec3a438c9b36c5e9341283dffc8678b21795a0f7d6a5bafc4 not found: ID does not exist" Dec 03 12:06:28 crc kubenswrapper[5052]: I1203 12:06:28.749634 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:06:28 crc kubenswrapper[5052]: I1203 12:06:28.759255 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:06:28 crc kubenswrapper[5052]: I1203 12:06:28.775913 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:06:28 crc kubenswrapper[5052]: E1203 12:06:28.776807 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="cf64fec7-c528-4a2c-89b4-a43851b7a903" containerName="proxy-httpd" Dec 03 12:06:28 crc kubenswrapper[5052]: I1203 12:06:28.776918 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="cf64fec7-c528-4a2c-89b4-a43851b7a903" containerName="proxy-httpd" Dec 03 12:06:28 crc kubenswrapper[5052]: E1203 12:06:28.776997 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="cf64fec7-c528-4a2c-89b4-a43851b7a903" containerName="ceilometer-notification-agent" Dec 03 12:06:28 crc kubenswrapper[5052]: I1203 12:06:28.777065 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="cf64fec7-c528-4a2c-89b4-a43851b7a903" containerName="ceilometer-notification-agent" Dec 03 12:06:28 crc kubenswrapper[5052]: E1203 12:06:28.777131 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="c3489d6f-c371-4252-9a13-a055a4a55cbf" containerName="neutron-api" Dec 03 12:06:28 crc kubenswrapper[5052]: I1203 12:06:28.777196 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="c3489d6f-c371-4252-9a13-a055a4a55cbf" containerName="neutron-api" Dec 03 12:06:28 crc kubenswrapper[5052]: E1203 12:06:28.777254 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="cf64fec7-c528-4a2c-89b4-a43851b7a903" containerName="ceilometer-central-agent" Dec 03 12:06:28 crc kubenswrapper[5052]: I1203 12:06:28.777309 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="cf64fec7-c528-4a2c-89b4-a43851b7a903" containerName="ceilometer-central-agent" Dec 03 12:06:28 crc kubenswrapper[5052]: E1203 12:06:28.777363 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="c3489d6f-c371-4252-9a13-a055a4a55cbf" containerName="neutron-httpd" Dec 03 12:06:28 crc kubenswrapper[5052]: I1203 12:06:28.777413 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="c3489d6f-c371-4252-9a13-a055a4a55cbf" containerName="neutron-httpd" Dec 03 12:06:28 crc kubenswrapper[5052]: E1203 12:06:28.777476 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="cf64fec7-c528-4a2c-89b4-a43851b7a903" containerName="sg-core" Dec 03 12:06:28 crc kubenswrapper[5052]: I1203 12:06:28.777545 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="cf64fec7-c528-4a2c-89b4-a43851b7a903" containerName="sg-core" Dec 03 12:06:28 crc kubenswrapper[5052]: I1203 12:06:28.777807 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="cf64fec7-c528-4a2c-89b4-a43851b7a903" containerName="sg-core" Dec 03 12:06:28 crc kubenswrapper[5052]: I1203 12:06:28.777882 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="c3489d6f-c371-4252-9a13-a055a4a55cbf" containerName="neutron-api" Dec 03 12:06:28 crc kubenswrapper[5052]: I1203 12:06:28.777945 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="cf64fec7-c528-4a2c-89b4-a43851b7a903" containerName="proxy-httpd" Dec 03 12:06:28 crc kubenswrapper[5052]: I1203 12:06:28.778023 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="cf64fec7-c528-4a2c-89b4-a43851b7a903" containerName="ceilometer-central-agent" Dec 03 12:06:28 crc kubenswrapper[5052]: I1203 12:06:28.778095 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="cf64fec7-c528-4a2c-89b4-a43851b7a903" containerName="ceilometer-notification-agent" Dec 03 12:06:28 crc kubenswrapper[5052]: I1203 12:06:28.778148 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="c3489d6f-c371-4252-9a13-a055a4a55cbf" containerName="neutron-httpd" Dec 03 12:06:28 crc kubenswrapper[5052]: I1203 12:06:28.779953 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:06:28 crc kubenswrapper[5052]: I1203 12:06:28.782258 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-config-data" Dec 03 12:06:28 crc kubenswrapper[5052]: I1203 12:06:28.782838 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-scripts" Dec 03 12:06:28 crc kubenswrapper[5052]: I1203 12:06:28.797992 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:06:28 crc kubenswrapper[5052]: I1203 12:06:28.821255 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/44c82024-af27-4702-bcf0-84d269840749-config-data\") pod \"ceilometer-0\" (UID: \"44c82024-af27-4702-bcf0-84d269840749\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:06:28 crc kubenswrapper[5052]: I1203 12:06:28.821322 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/44c82024-af27-4702-bcf0-84d269840749-run-httpd\") pod \"ceilometer-0\" (UID: \"44c82024-af27-4702-bcf0-84d269840749\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:06:28 crc kubenswrapper[5052]: I1203 12:06:28.821366 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/44c82024-af27-4702-bcf0-84d269840749-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"44c82024-af27-4702-bcf0-84d269840749\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:06:28 crc kubenswrapper[5052]: I1203 12:06:28.821399 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/44c82024-af27-4702-bcf0-84d269840749-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"44c82024-af27-4702-bcf0-84d269840749\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:06:28 crc kubenswrapper[5052]: I1203 12:06:28.821489 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/44c82024-af27-4702-bcf0-84d269840749-scripts\") pod \"ceilometer-0\" (UID: \"44c82024-af27-4702-bcf0-84d269840749\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:06:28 crc kubenswrapper[5052]: I1203 12:06:28.821531 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gxbgm\" (UniqueName: \"kubernetes.io/projected/44c82024-af27-4702-bcf0-84d269840749-kube-api-access-gxbgm\") pod \"ceilometer-0\" (UID: \"44c82024-af27-4702-bcf0-84d269840749\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:06:28 crc kubenswrapper[5052]: I1203 12:06:28.821835 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/44c82024-af27-4702-bcf0-84d269840749-log-httpd\") pod \"ceilometer-0\" (UID: \"44c82024-af27-4702-bcf0-84d269840749\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:06:28 crc kubenswrapper[5052]: I1203 12:06:28.923824 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/44c82024-af27-4702-bcf0-84d269840749-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"44c82024-af27-4702-bcf0-84d269840749\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:06:28 crc kubenswrapper[5052]: I1203 12:06:28.923906 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/44c82024-af27-4702-bcf0-84d269840749-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"44c82024-af27-4702-bcf0-84d269840749\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:06:28 crc kubenswrapper[5052]: I1203 12:06:28.923987 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/44c82024-af27-4702-bcf0-84d269840749-scripts\") pod \"ceilometer-0\" (UID: \"44c82024-af27-4702-bcf0-84d269840749\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:06:28 crc kubenswrapper[5052]: I1203 12:06:28.924018 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-gxbgm\" (UniqueName: \"kubernetes.io/projected/44c82024-af27-4702-bcf0-84d269840749-kube-api-access-gxbgm\") pod \"ceilometer-0\" (UID: \"44c82024-af27-4702-bcf0-84d269840749\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:06:28 crc kubenswrapper[5052]: I1203 12:06:28.924106 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/44c82024-af27-4702-bcf0-84d269840749-log-httpd\") pod \"ceilometer-0\" (UID: \"44c82024-af27-4702-bcf0-84d269840749\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:06:28 crc kubenswrapper[5052]: I1203 12:06:28.924285 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/44c82024-af27-4702-bcf0-84d269840749-config-data\") pod \"ceilometer-0\" (UID: \"44c82024-af27-4702-bcf0-84d269840749\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:06:28 crc kubenswrapper[5052]: I1203 12:06:28.924321 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/44c82024-af27-4702-bcf0-84d269840749-run-httpd\") pod \"ceilometer-0\" (UID: \"44c82024-af27-4702-bcf0-84d269840749\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:06:28 crc kubenswrapper[5052]: I1203 12:06:28.925104 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/44c82024-af27-4702-bcf0-84d269840749-log-httpd\") pod \"ceilometer-0\" (UID: \"44c82024-af27-4702-bcf0-84d269840749\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:06:28 crc kubenswrapper[5052]: I1203 12:06:28.925211 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/44c82024-af27-4702-bcf0-84d269840749-run-httpd\") pod \"ceilometer-0\" (UID: \"44c82024-af27-4702-bcf0-84d269840749\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:06:28 crc kubenswrapper[5052]: I1203 12:06:28.928146 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/44c82024-af27-4702-bcf0-84d269840749-scripts\") pod \"ceilometer-0\" (UID: \"44c82024-af27-4702-bcf0-84d269840749\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:06:28 crc kubenswrapper[5052]: I1203 12:06:28.928316 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/44c82024-af27-4702-bcf0-84d269840749-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"44c82024-af27-4702-bcf0-84d269840749\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:06:28 crc kubenswrapper[5052]: I1203 12:06:28.928397 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/44c82024-af27-4702-bcf0-84d269840749-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"44c82024-af27-4702-bcf0-84d269840749\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:06:28 crc kubenswrapper[5052]: I1203 12:06:28.929635 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/44c82024-af27-4702-bcf0-84d269840749-config-data\") pod \"ceilometer-0\" (UID: \"44c82024-af27-4702-bcf0-84d269840749\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:06:28 crc kubenswrapper[5052]: I1203 12:06:28.952623 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-gxbgm\" (UniqueName: \"kubernetes.io/projected/44c82024-af27-4702-bcf0-84d269840749-kube-api-access-gxbgm\") pod \"ceilometer-0\" (UID: \"44c82024-af27-4702-bcf0-84d269840749\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:06:29 crc kubenswrapper[5052]: I1203 12:06:29.107682 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:06:29 crc kubenswrapper[5052]: I1203 12:06:29.577321 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:06:29 crc kubenswrapper[5052]: W1203 12:06:29.586812 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod44c82024_af27_4702_bcf0_84d269840749.slice/crio-efbfe1a619c031b602fd0770262b916611bcf429a28a77d07899338833ea03c1 WatchSource:0}: Error finding container efbfe1a619c031b602fd0770262b916611bcf429a28a77d07899338833ea03c1: Status 404 returned error can't find the container with id efbfe1a619c031b602fd0770262b916611bcf429a28a77d07899338833ea03c1 Dec 03 12:06:29 crc kubenswrapper[5052]: I1203 12:06:29.686010 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cf64fec7-c528-4a2c-89b4-a43851b7a903" path="/var/lib/kubelet/pods/cf64fec7-c528-4a2c-89b4-a43851b7a903/volumes" Dec 03 12:06:30 crc kubenswrapper[5052]: I1203 12:06:30.426632 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"44c82024-af27-4702-bcf0-84d269840749","Type":"ContainerStarted","Data":"efbfe1a619c031b602fd0770262b916611bcf429a28a77d07899338833ea03c1"} Dec 03 12:06:31 crc kubenswrapper[5052]: I1203 12:06:31.437490 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"44c82024-af27-4702-bcf0-84d269840749","Type":"ContainerStarted","Data":"a6ba1c579090bbd27d0787f86bce32466d51caef53ac643ede54d5bd5906a7d1"} Dec 03 12:06:31 crc kubenswrapper[5052]: I1203 12:06:31.437788 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"44c82024-af27-4702-bcf0-84d269840749","Type":"ContainerStarted","Data":"b823c74e4bf02ada19452a0921d828e5c3c34365901bf95c5f92c016771c2c7a"} Dec 03 12:06:34 crc kubenswrapper[5052]: I1203 12:06:34.471909 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"44c82024-af27-4702-bcf0-84d269840749","Type":"ContainerStarted","Data":"b63af88913166ec529787d87ae4f944fddf84ea4021499b8b8c9fbdf71317343"} Dec 03 12:06:36 crc kubenswrapper[5052]: I1203 12:06:36.494579 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"44c82024-af27-4702-bcf0-84d269840749","Type":"ContainerStarted","Data":"3ae15f9196f3f57942131b77a6c833870ef71b802eb7d3cf07d241c9e3bc1ff6"} Dec 03 12:06:36 crc kubenswrapper[5052]: I1203 12:06:36.495258 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:06:36 crc kubenswrapper[5052]: I1203 12:06:36.523701 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/ceilometer-0" podStartSLOduration=2.597849443 podStartE2EDuration="8.523682298s" podCreationTimestamp="2025-12-03 12:06:28 +0000 UTC" firstStartedPulling="2025-12-03 12:06:29.590122392 +0000 UTC m=+2780.178364146" lastFinishedPulling="2025-12-03 12:06:35.515955237 +0000 UTC m=+2786.104197001" observedRunningTime="2025-12-03 12:06:36.512736775 +0000 UTC m=+2787.100978559" watchObservedRunningTime="2025-12-03 12:06:36.523682298 +0000 UTC m=+2787.111924062" Dec 03 12:06:49 crc kubenswrapper[5052]: I1203 12:06:49.633393 5052 generic.go:334] "Generic (PLEG): container finished" podID="895b843a-707e-4cc5-8991-501116bd2fe3" containerID="737462a16fc5e657fb057be690e9512c155f3a35e26b18e1f261c004033bd4d8" exitCode=0 Dec 03 12:06:49 crc kubenswrapper[5052]: I1203 12:06:49.633480 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-smfjz" event={"ID":"895b843a-707e-4cc5-8991-501116bd2fe3","Type":"ContainerDied","Data":"737462a16fc5e657fb057be690e9512c155f3a35e26b18e1f261c004033bd4d8"} Dec 03 12:06:51 crc kubenswrapper[5052]: I1203 12:06:51.004192 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-smfjz" Dec 03 12:06:51 crc kubenswrapper[5052]: I1203 12:06:51.105723 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/895b843a-707e-4cc5-8991-501116bd2fe3-config-data\") pod \"895b843a-707e-4cc5-8991-501116bd2fe3\" (UID: \"895b843a-707e-4cc5-8991-501116bd2fe3\") " Dec 03 12:06:51 crc kubenswrapper[5052]: I1203 12:06:51.105836 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zfdrn\" (UniqueName: \"kubernetes.io/projected/895b843a-707e-4cc5-8991-501116bd2fe3-kube-api-access-zfdrn\") pod \"895b843a-707e-4cc5-8991-501116bd2fe3\" (UID: \"895b843a-707e-4cc5-8991-501116bd2fe3\") " Dec 03 12:06:51 crc kubenswrapper[5052]: I1203 12:06:51.105900 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/895b843a-707e-4cc5-8991-501116bd2fe3-scripts\") pod \"895b843a-707e-4cc5-8991-501116bd2fe3\" (UID: \"895b843a-707e-4cc5-8991-501116bd2fe3\") " Dec 03 12:06:51 crc kubenswrapper[5052]: I1203 12:06:51.105936 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/895b843a-707e-4cc5-8991-501116bd2fe3-combined-ca-bundle\") pod \"895b843a-707e-4cc5-8991-501116bd2fe3\" (UID: \"895b843a-707e-4cc5-8991-501116bd2fe3\") " Dec 03 12:06:51 crc kubenswrapper[5052]: I1203 12:06:51.112108 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/895b843a-707e-4cc5-8991-501116bd2fe3-kube-api-access-zfdrn" (OuterVolumeSpecName: "kube-api-access-zfdrn") pod "895b843a-707e-4cc5-8991-501116bd2fe3" (UID: "895b843a-707e-4cc5-8991-501116bd2fe3"). InnerVolumeSpecName "kube-api-access-zfdrn". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:06:51 crc kubenswrapper[5052]: I1203 12:06:51.112441 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/895b843a-707e-4cc5-8991-501116bd2fe3-scripts" (OuterVolumeSpecName: "scripts") pod "895b843a-707e-4cc5-8991-501116bd2fe3" (UID: "895b843a-707e-4cc5-8991-501116bd2fe3"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:06:51 crc kubenswrapper[5052]: I1203 12:06:51.135443 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/895b843a-707e-4cc5-8991-501116bd2fe3-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "895b843a-707e-4cc5-8991-501116bd2fe3" (UID: "895b843a-707e-4cc5-8991-501116bd2fe3"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:06:51 crc kubenswrapper[5052]: I1203 12:06:51.137616 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/895b843a-707e-4cc5-8991-501116bd2fe3-config-data" (OuterVolumeSpecName: "config-data") pod "895b843a-707e-4cc5-8991-501116bd2fe3" (UID: "895b843a-707e-4cc5-8991-501116bd2fe3"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:06:51 crc kubenswrapper[5052]: I1203 12:06:51.208787 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-zfdrn\" (UniqueName: \"kubernetes.io/projected/895b843a-707e-4cc5-8991-501116bd2fe3-kube-api-access-zfdrn\") on node \"crc\" DevicePath \"\"" Dec 03 12:06:51 crc kubenswrapper[5052]: I1203 12:06:51.208819 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/895b843a-707e-4cc5-8991-501116bd2fe3-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:06:51 crc kubenswrapper[5052]: I1203 12:06:51.208829 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/895b843a-707e-4cc5-8991-501116bd2fe3-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:06:51 crc kubenswrapper[5052]: I1203 12:06:51.208838 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/895b843a-707e-4cc5-8991-501116bd2fe3-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:06:51 crc kubenswrapper[5052]: I1203 12:06:51.652235 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-smfjz" event={"ID":"895b843a-707e-4cc5-8991-501116bd2fe3","Type":"ContainerDied","Data":"95867ac62c0a7908764d6ddd89c5a27b15543c21ad0130c3156862b495ccd48a"} Dec 03 12:06:51 crc kubenswrapper[5052]: I1203 12:06:51.652540 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="95867ac62c0a7908764d6ddd89c5a27b15543c21ad0130c3156862b495ccd48a" Dec 03 12:06:51 crc kubenswrapper[5052]: I1203 12:06:51.652314 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-smfjz" Dec 03 12:06:51 crc kubenswrapper[5052]: I1203 12:06:51.734289 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-0"] Dec 03 12:06:51 crc kubenswrapper[5052]: E1203 12:06:51.734868 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="895b843a-707e-4cc5-8991-501116bd2fe3" containerName="nova-cell0-conductor-db-sync" Dec 03 12:06:51 crc kubenswrapper[5052]: I1203 12:06:51.734889 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="895b843a-707e-4cc5-8991-501116bd2fe3" containerName="nova-cell0-conductor-db-sync" Dec 03 12:06:51 crc kubenswrapper[5052]: I1203 12:06:51.735081 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="895b843a-707e-4cc5-8991-501116bd2fe3" containerName="nova-cell0-conductor-db-sync" Dec 03 12:06:51 crc kubenswrapper[5052]: I1203 12:06:51.735957 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:06:51 crc kubenswrapper[5052]: I1203 12:06:51.738489 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell0-conductor-config-data" Dec 03 12:06:51 crc kubenswrapper[5052]: I1203 12:06:51.739086 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-nova-dockercfg-55b29" Dec 03 12:06:51 crc kubenswrapper[5052]: I1203 12:06:51.745466 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-0"] Dec 03 12:06:51 crc kubenswrapper[5052]: I1203 12:06:51.820422 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a98d6a28-9d82-4e03-ac7d-76500c4963bb-config-data\") pod \"nova-cell0-conductor-0\" (UID: \"a98d6a28-9d82-4e03-ac7d-76500c4963bb\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:06:51 crc kubenswrapper[5052]: I1203 12:06:51.820839 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7sqfw\" (UniqueName: \"kubernetes.io/projected/a98d6a28-9d82-4e03-ac7d-76500c4963bb-kube-api-access-7sqfw\") pod \"nova-cell0-conductor-0\" (UID: \"a98d6a28-9d82-4e03-ac7d-76500c4963bb\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:06:51 crc kubenswrapper[5052]: I1203 12:06:51.820965 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a98d6a28-9d82-4e03-ac7d-76500c4963bb-combined-ca-bundle\") pod \"nova-cell0-conductor-0\" (UID: \"a98d6a28-9d82-4e03-ac7d-76500c4963bb\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:06:51 crc kubenswrapper[5052]: I1203 12:06:51.922709 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a98d6a28-9d82-4e03-ac7d-76500c4963bb-config-data\") pod \"nova-cell0-conductor-0\" (UID: \"a98d6a28-9d82-4e03-ac7d-76500c4963bb\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:06:51 crc kubenswrapper[5052]: I1203 12:06:51.922918 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-7sqfw\" (UniqueName: \"kubernetes.io/projected/a98d6a28-9d82-4e03-ac7d-76500c4963bb-kube-api-access-7sqfw\") pod \"nova-cell0-conductor-0\" (UID: \"a98d6a28-9d82-4e03-ac7d-76500c4963bb\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:06:51 crc kubenswrapper[5052]: I1203 12:06:51.922977 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a98d6a28-9d82-4e03-ac7d-76500c4963bb-combined-ca-bundle\") pod \"nova-cell0-conductor-0\" (UID: \"a98d6a28-9d82-4e03-ac7d-76500c4963bb\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:06:51 crc kubenswrapper[5052]: I1203 12:06:51.924348 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-0"] Dec 03 12:06:51 crc kubenswrapper[5052]: I1203 12:06:51.931161 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a98d6a28-9d82-4e03-ac7d-76500c4963bb-combined-ca-bundle\") pod \"nova-cell0-conductor-0\" (UID: \"a98d6a28-9d82-4e03-ac7d-76500c4963bb\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:06:51 crc kubenswrapper[5052]: E1203 12:06:51.931177 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[combined-ca-bundle config-data kube-api-access-7sqfw], unattached volumes=[], failed to process volumes=[]: context canceled" pod="openstack-kuttl-tests/nova-cell0-conductor-0" podUID="a98d6a28-9d82-4e03-ac7d-76500c4963bb" Dec 03 12:06:51 crc kubenswrapper[5052]: I1203 12:06:51.932365 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a98d6a28-9d82-4e03-ac7d-76500c4963bb-config-data\") pod \"nova-cell0-conductor-0\" (UID: \"a98d6a28-9d82-4e03-ac7d-76500c4963bb\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:06:51 crc kubenswrapper[5052]: I1203 12:06:51.944886 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-7sqfw\" (UniqueName: \"kubernetes.io/projected/a98d6a28-9d82-4e03-ac7d-76500c4963bb-kube-api-access-7sqfw\") pod \"nova-cell0-conductor-0\" (UID: \"a98d6a28-9d82-4e03-ac7d-76500c4963bb\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:06:52 crc kubenswrapper[5052]: I1203 12:06:52.665523 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:06:52 crc kubenswrapper[5052]: I1203 12:06:52.679042 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:06:52 crc kubenswrapper[5052]: I1203 12:06:52.765961 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a98d6a28-9d82-4e03-ac7d-76500c4963bb-combined-ca-bundle\") pod \"a98d6a28-9d82-4e03-ac7d-76500c4963bb\" (UID: \"a98d6a28-9d82-4e03-ac7d-76500c4963bb\") " Dec 03 12:06:52 crc kubenswrapper[5052]: I1203 12:06:52.766946 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a98d6a28-9d82-4e03-ac7d-76500c4963bb-config-data\") pod \"a98d6a28-9d82-4e03-ac7d-76500c4963bb\" (UID: \"a98d6a28-9d82-4e03-ac7d-76500c4963bb\") " Dec 03 12:06:52 crc kubenswrapper[5052]: I1203 12:06:52.767123 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7sqfw\" (UniqueName: \"kubernetes.io/projected/a98d6a28-9d82-4e03-ac7d-76500c4963bb-kube-api-access-7sqfw\") pod \"a98d6a28-9d82-4e03-ac7d-76500c4963bb\" (UID: \"a98d6a28-9d82-4e03-ac7d-76500c4963bb\") " Dec 03 12:06:52 crc kubenswrapper[5052]: I1203 12:06:52.770531 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a98d6a28-9d82-4e03-ac7d-76500c4963bb-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "a98d6a28-9d82-4e03-ac7d-76500c4963bb" (UID: "a98d6a28-9d82-4e03-ac7d-76500c4963bb"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:06:52 crc kubenswrapper[5052]: I1203 12:06:52.771490 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a98d6a28-9d82-4e03-ac7d-76500c4963bb-config-data" (OuterVolumeSpecName: "config-data") pod "a98d6a28-9d82-4e03-ac7d-76500c4963bb" (UID: "a98d6a28-9d82-4e03-ac7d-76500c4963bb"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:06:52 crc kubenswrapper[5052]: I1203 12:06:52.773702 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a98d6a28-9d82-4e03-ac7d-76500c4963bb-kube-api-access-7sqfw" (OuterVolumeSpecName: "kube-api-access-7sqfw") pod "a98d6a28-9d82-4e03-ac7d-76500c4963bb" (UID: "a98d6a28-9d82-4e03-ac7d-76500c4963bb"). InnerVolumeSpecName "kube-api-access-7sqfw". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:06:52 crc kubenswrapper[5052]: I1203 12:06:52.870217 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-7sqfw\" (UniqueName: \"kubernetes.io/projected/a98d6a28-9d82-4e03-ac7d-76500c4963bb-kube-api-access-7sqfw\") on node \"crc\" DevicePath \"\"" Dec 03 12:06:52 crc kubenswrapper[5052]: I1203 12:06:52.870275 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a98d6a28-9d82-4e03-ac7d-76500c4963bb-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:06:52 crc kubenswrapper[5052]: I1203 12:06:52.870292 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a98d6a28-9d82-4e03-ac7d-76500c4963bb-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:06:53 crc kubenswrapper[5052]: I1203 12:06:53.488336 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:06:53 crc kubenswrapper[5052]: I1203 12:06:53.488630 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="44c82024-af27-4702-bcf0-84d269840749" containerName="ceilometer-central-agent" containerID="cri-o://b823c74e4bf02ada19452a0921d828e5c3c34365901bf95c5f92c016771c2c7a" gracePeriod=30 Dec 03 12:06:53 crc kubenswrapper[5052]: I1203 12:06:53.488738 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="44c82024-af27-4702-bcf0-84d269840749" containerName="ceilometer-notification-agent" containerID="cri-o://a6ba1c579090bbd27d0787f86bce32466d51caef53ac643ede54d5bd5906a7d1" gracePeriod=30 Dec 03 12:06:53 crc kubenswrapper[5052]: I1203 12:06:53.488786 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="44c82024-af27-4702-bcf0-84d269840749" containerName="sg-core" containerID="cri-o://b63af88913166ec529787d87ae4f944fddf84ea4021499b8b8c9fbdf71317343" gracePeriod=30 Dec 03 12:06:53 crc kubenswrapper[5052]: I1203 12:06:53.488894 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="44c82024-af27-4702-bcf0-84d269840749" containerName="proxy-httpd" containerID="cri-o://3ae15f9196f3f57942131b77a6c833870ef71b802eb7d3cf07d241c9e3bc1ff6" gracePeriod=30 Dec 03 12:06:53 crc kubenswrapper[5052]: I1203 12:06:53.499965 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/ceilometer-0" podUID="44c82024-af27-4702-bcf0-84d269840749" containerName="proxy-httpd" probeResult="failure" output="Get \"http://10.217.0.174:3000/\": EOF" Dec 03 12:06:53 crc kubenswrapper[5052]: I1203 12:06:53.690674 5052 generic.go:334] "Generic (PLEG): container finished" podID="44c82024-af27-4702-bcf0-84d269840749" containerID="3ae15f9196f3f57942131b77a6c833870ef71b802eb7d3cf07d241c9e3bc1ff6" exitCode=0 Dec 03 12:06:53 crc kubenswrapper[5052]: I1203 12:06:53.690716 5052 generic.go:334] "Generic (PLEG): container finished" podID="44c82024-af27-4702-bcf0-84d269840749" containerID="b63af88913166ec529787d87ae4f944fddf84ea4021499b8b8c9fbdf71317343" exitCode=2 Dec 03 12:06:53 crc kubenswrapper[5052]: I1203 12:06:53.690756 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:06:53 crc kubenswrapper[5052]: I1203 12:06:53.693742 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"44c82024-af27-4702-bcf0-84d269840749","Type":"ContainerDied","Data":"3ae15f9196f3f57942131b77a6c833870ef71b802eb7d3cf07d241c9e3bc1ff6"} Dec 03 12:06:53 crc kubenswrapper[5052]: I1203 12:06:53.693800 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"44c82024-af27-4702-bcf0-84d269840749","Type":"ContainerDied","Data":"b63af88913166ec529787d87ae4f944fddf84ea4021499b8b8c9fbdf71317343"} Dec 03 12:06:53 crc kubenswrapper[5052]: I1203 12:06:53.764940 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-0"] Dec 03 12:06:53 crc kubenswrapper[5052]: I1203 12:06:53.782603 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-0"] Dec 03 12:06:53 crc kubenswrapper[5052]: I1203 12:06:53.794013 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-0"] Dec 03 12:06:53 crc kubenswrapper[5052]: I1203 12:06:53.795388 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:06:53 crc kubenswrapper[5052]: I1203 12:06:53.798562 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell0-conductor-config-data" Dec 03 12:06:53 crc kubenswrapper[5052]: I1203 12:06:53.798775 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-nova-dockercfg-55b29" Dec 03 12:06:53 crc kubenswrapper[5052]: I1203 12:06:53.802406 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-0"] Dec 03 12:06:53 crc kubenswrapper[5052]: I1203 12:06:53.888481 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lbx98\" (UniqueName: \"kubernetes.io/projected/4cae4fe5-bc0e-4088-b403-c78e940373c2-kube-api-access-lbx98\") pod \"nova-cell0-conductor-0\" (UID: \"4cae4fe5-bc0e-4088-b403-c78e940373c2\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:06:53 crc kubenswrapper[5052]: I1203 12:06:53.888909 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4cae4fe5-bc0e-4088-b403-c78e940373c2-combined-ca-bundle\") pod \"nova-cell0-conductor-0\" (UID: \"4cae4fe5-bc0e-4088-b403-c78e940373c2\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:06:53 crc kubenswrapper[5052]: I1203 12:06:53.889065 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/4cae4fe5-bc0e-4088-b403-c78e940373c2-config-data\") pod \"nova-cell0-conductor-0\" (UID: \"4cae4fe5-bc0e-4088-b403-c78e940373c2\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:06:53 crc kubenswrapper[5052]: I1203 12:06:53.990592 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4cae4fe5-bc0e-4088-b403-c78e940373c2-combined-ca-bundle\") pod \"nova-cell0-conductor-0\" (UID: \"4cae4fe5-bc0e-4088-b403-c78e940373c2\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:06:53 crc kubenswrapper[5052]: I1203 12:06:53.990722 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/4cae4fe5-bc0e-4088-b403-c78e940373c2-config-data\") pod \"nova-cell0-conductor-0\" (UID: \"4cae4fe5-bc0e-4088-b403-c78e940373c2\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:06:53 crc kubenswrapper[5052]: I1203 12:06:53.990807 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-lbx98\" (UniqueName: \"kubernetes.io/projected/4cae4fe5-bc0e-4088-b403-c78e940373c2-kube-api-access-lbx98\") pod \"nova-cell0-conductor-0\" (UID: \"4cae4fe5-bc0e-4088-b403-c78e940373c2\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:06:54 crc kubenswrapper[5052]: I1203 12:06:53.995742 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4cae4fe5-bc0e-4088-b403-c78e940373c2-combined-ca-bundle\") pod \"nova-cell0-conductor-0\" (UID: \"4cae4fe5-bc0e-4088-b403-c78e940373c2\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:06:54 crc kubenswrapper[5052]: I1203 12:06:53.996277 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/4cae4fe5-bc0e-4088-b403-c78e940373c2-config-data\") pod \"nova-cell0-conductor-0\" (UID: \"4cae4fe5-bc0e-4088-b403-c78e940373c2\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:06:54 crc kubenswrapper[5052]: I1203 12:06:54.011059 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-lbx98\" (UniqueName: \"kubernetes.io/projected/4cae4fe5-bc0e-4088-b403-c78e940373c2-kube-api-access-lbx98\") pod \"nova-cell0-conductor-0\" (UID: \"4cae4fe5-bc0e-4088-b403-c78e940373c2\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:06:54 crc kubenswrapper[5052]: I1203 12:06:54.115587 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:06:54 crc kubenswrapper[5052]: I1203 12:06:54.604048 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-0"] Dec 03 12:06:54 crc kubenswrapper[5052]: W1203 12:06:54.619794 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4cae4fe5_bc0e_4088_b403_c78e940373c2.slice/crio-f273f577e74fa9872c305a62585f94695ac0cbff46915d6a576d5f7689cecadd WatchSource:0}: Error finding container f273f577e74fa9872c305a62585f94695ac0cbff46915d6a576d5f7689cecadd: Status 404 returned error can't find the container with id f273f577e74fa9872c305a62585f94695ac0cbff46915d6a576d5f7689cecadd Dec 03 12:06:54 crc kubenswrapper[5052]: I1203 12:06:54.709091 5052 generic.go:334] "Generic (PLEG): container finished" podID="44c82024-af27-4702-bcf0-84d269840749" containerID="b823c74e4bf02ada19452a0921d828e5c3c34365901bf95c5f92c016771c2c7a" exitCode=0 Dec 03 12:06:54 crc kubenswrapper[5052]: I1203 12:06:54.709137 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"44c82024-af27-4702-bcf0-84d269840749","Type":"ContainerDied","Data":"b823c74e4bf02ada19452a0921d828e5c3c34365901bf95c5f92c016771c2c7a"} Dec 03 12:06:54 crc kubenswrapper[5052]: I1203 12:06:54.710811 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-conductor-0" event={"ID":"4cae4fe5-bc0e-4088-b403-c78e940373c2","Type":"ContainerStarted","Data":"f273f577e74fa9872c305a62585f94695ac0cbff46915d6a576d5f7689cecadd"} Dec 03 12:06:55 crc kubenswrapper[5052]: I1203 12:06:55.148246 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:06:55 crc kubenswrapper[5052]: I1203 12:06:55.221697 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/44c82024-af27-4702-bcf0-84d269840749-log-httpd\") pod \"44c82024-af27-4702-bcf0-84d269840749\" (UID: \"44c82024-af27-4702-bcf0-84d269840749\") " Dec 03 12:06:55 crc kubenswrapper[5052]: I1203 12:06:55.222194 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gxbgm\" (UniqueName: \"kubernetes.io/projected/44c82024-af27-4702-bcf0-84d269840749-kube-api-access-gxbgm\") pod \"44c82024-af27-4702-bcf0-84d269840749\" (UID: \"44c82024-af27-4702-bcf0-84d269840749\") " Dec 03 12:06:55 crc kubenswrapper[5052]: I1203 12:06:55.222466 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/44c82024-af27-4702-bcf0-84d269840749-log-httpd" (OuterVolumeSpecName: "log-httpd") pod "44c82024-af27-4702-bcf0-84d269840749" (UID: "44c82024-af27-4702-bcf0-84d269840749"). InnerVolumeSpecName "log-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:06:55 crc kubenswrapper[5052]: I1203 12:06:55.222975 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/44c82024-af27-4702-bcf0-84d269840749-run-httpd\") pod \"44c82024-af27-4702-bcf0-84d269840749\" (UID: \"44c82024-af27-4702-bcf0-84d269840749\") " Dec 03 12:06:55 crc kubenswrapper[5052]: I1203 12:06:55.223041 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/44c82024-af27-4702-bcf0-84d269840749-scripts\") pod \"44c82024-af27-4702-bcf0-84d269840749\" (UID: \"44c82024-af27-4702-bcf0-84d269840749\") " Dec 03 12:06:55 crc kubenswrapper[5052]: I1203 12:06:55.223074 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/44c82024-af27-4702-bcf0-84d269840749-combined-ca-bundle\") pod \"44c82024-af27-4702-bcf0-84d269840749\" (UID: \"44c82024-af27-4702-bcf0-84d269840749\") " Dec 03 12:06:55 crc kubenswrapper[5052]: I1203 12:06:55.223166 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/44c82024-af27-4702-bcf0-84d269840749-sg-core-conf-yaml\") pod \"44c82024-af27-4702-bcf0-84d269840749\" (UID: \"44c82024-af27-4702-bcf0-84d269840749\") " Dec 03 12:06:55 crc kubenswrapper[5052]: I1203 12:06:55.223203 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/44c82024-af27-4702-bcf0-84d269840749-config-data\") pod \"44c82024-af27-4702-bcf0-84d269840749\" (UID: \"44c82024-af27-4702-bcf0-84d269840749\") " Dec 03 12:06:55 crc kubenswrapper[5052]: I1203 12:06:55.223715 5052 reconciler_common.go:293] "Volume detached for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/44c82024-af27-4702-bcf0-84d269840749-log-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:06:55 crc kubenswrapper[5052]: I1203 12:06:55.223894 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/44c82024-af27-4702-bcf0-84d269840749-run-httpd" (OuterVolumeSpecName: "run-httpd") pod "44c82024-af27-4702-bcf0-84d269840749" (UID: "44c82024-af27-4702-bcf0-84d269840749"). InnerVolumeSpecName "run-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:06:55 crc kubenswrapper[5052]: I1203 12:06:55.227486 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/44c82024-af27-4702-bcf0-84d269840749-kube-api-access-gxbgm" (OuterVolumeSpecName: "kube-api-access-gxbgm") pod "44c82024-af27-4702-bcf0-84d269840749" (UID: "44c82024-af27-4702-bcf0-84d269840749"). InnerVolumeSpecName "kube-api-access-gxbgm". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:06:55 crc kubenswrapper[5052]: I1203 12:06:55.231700 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/44c82024-af27-4702-bcf0-84d269840749-scripts" (OuterVolumeSpecName: "scripts") pod "44c82024-af27-4702-bcf0-84d269840749" (UID: "44c82024-af27-4702-bcf0-84d269840749"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:06:55 crc kubenswrapper[5052]: I1203 12:06:55.255420 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/44c82024-af27-4702-bcf0-84d269840749-sg-core-conf-yaml" (OuterVolumeSpecName: "sg-core-conf-yaml") pod "44c82024-af27-4702-bcf0-84d269840749" (UID: "44c82024-af27-4702-bcf0-84d269840749"). InnerVolumeSpecName "sg-core-conf-yaml". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:06:55 crc kubenswrapper[5052]: I1203 12:06:55.325355 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-gxbgm\" (UniqueName: \"kubernetes.io/projected/44c82024-af27-4702-bcf0-84d269840749-kube-api-access-gxbgm\") on node \"crc\" DevicePath \"\"" Dec 03 12:06:55 crc kubenswrapper[5052]: I1203 12:06:55.325400 5052 reconciler_common.go:293] "Volume detached for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/44c82024-af27-4702-bcf0-84d269840749-run-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:06:55 crc kubenswrapper[5052]: I1203 12:06:55.325412 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/44c82024-af27-4702-bcf0-84d269840749-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:06:55 crc kubenswrapper[5052]: I1203 12:06:55.325428 5052 reconciler_common.go:293] "Volume detached for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/44c82024-af27-4702-bcf0-84d269840749-sg-core-conf-yaml\") on node \"crc\" DevicePath \"\"" Dec 03 12:06:55 crc kubenswrapper[5052]: I1203 12:06:55.326907 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/44c82024-af27-4702-bcf0-84d269840749-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "44c82024-af27-4702-bcf0-84d269840749" (UID: "44c82024-af27-4702-bcf0-84d269840749"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:06:55 crc kubenswrapper[5052]: I1203 12:06:55.346260 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/44c82024-af27-4702-bcf0-84d269840749-config-data" (OuterVolumeSpecName: "config-data") pod "44c82024-af27-4702-bcf0-84d269840749" (UID: "44c82024-af27-4702-bcf0-84d269840749"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:06:55 crc kubenswrapper[5052]: I1203 12:06:55.427575 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/44c82024-af27-4702-bcf0-84d269840749-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:06:55 crc kubenswrapper[5052]: I1203 12:06:55.427617 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/44c82024-af27-4702-bcf0-84d269840749-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:06:55 crc kubenswrapper[5052]: I1203 12:06:55.538162 5052 scope.go:117] "RemoveContainer" containerID="39232820b6e0e754d5a5084f941da2ffd8fab9858a3574cd08f2a2ef9dce5a18" Dec 03 12:06:55 crc kubenswrapper[5052]: I1203 12:06:55.604296 5052 scope.go:117] "RemoveContainer" containerID="84a4a97816b15f9f8b42611faefec10e993e636d5d1d408805fe13fa7df1d13e" Dec 03 12:06:55 crc kubenswrapper[5052]: I1203 12:06:55.687744 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a98d6a28-9d82-4e03-ac7d-76500c4963bb" path="/var/lib/kubelet/pods/a98d6a28-9d82-4e03-ac7d-76500c4963bb/volumes" Dec 03 12:06:55 crc kubenswrapper[5052]: I1203 12:06:55.713052 5052 scope.go:117] "RemoveContainer" containerID="3a4d32fe7507dccb3b5a703d41f75ff7c20cb477aef43d8d0062e58cf58c09bf" Dec 03 12:06:55 crc kubenswrapper[5052]: I1203 12:06:55.731049 5052 generic.go:334] "Generic (PLEG): container finished" podID="44c82024-af27-4702-bcf0-84d269840749" containerID="a6ba1c579090bbd27d0787f86bce32466d51caef53ac643ede54d5bd5906a7d1" exitCode=0 Dec 03 12:06:55 crc kubenswrapper[5052]: I1203 12:06:55.731115 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"44c82024-af27-4702-bcf0-84d269840749","Type":"ContainerDied","Data":"a6ba1c579090bbd27d0787f86bce32466d51caef53ac643ede54d5bd5906a7d1"} Dec 03 12:06:55 crc kubenswrapper[5052]: I1203 12:06:55.731144 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"44c82024-af27-4702-bcf0-84d269840749","Type":"ContainerDied","Data":"efbfe1a619c031b602fd0770262b916611bcf429a28a77d07899338833ea03c1"} Dec 03 12:06:55 crc kubenswrapper[5052]: I1203 12:06:55.731161 5052 scope.go:117] "RemoveContainer" containerID="3ae15f9196f3f57942131b77a6c833870ef71b802eb7d3cf07d241c9e3bc1ff6" Dec 03 12:06:55 crc kubenswrapper[5052]: I1203 12:06:55.731181 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:06:55 crc kubenswrapper[5052]: I1203 12:06:55.739744 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-conductor-0" event={"ID":"4cae4fe5-bc0e-4088-b403-c78e940373c2","Type":"ContainerStarted","Data":"b19b072c0fb517fe3875dc65637b798d06ddeac368aceb24adb64b956c3e8597"} Dec 03 12:06:55 crc kubenswrapper[5052]: I1203 12:06:55.739968 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:06:55 crc kubenswrapper[5052]: I1203 12:06:55.773561 5052 scope.go:117] "RemoveContainer" containerID="b63af88913166ec529787d87ae4f944fddf84ea4021499b8b8c9fbdf71317343" Dec 03 12:06:55 crc kubenswrapper[5052]: I1203 12:06:55.798869 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:06:55 crc kubenswrapper[5052]: I1203 12:06:55.814014 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:06:55 crc kubenswrapper[5052]: I1203 12:06:55.814608 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-cell0-conductor-0" podStartSLOduration=2.81458967 podStartE2EDuration="2.81458967s" podCreationTimestamp="2025-12-03 12:06:53 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:06:55.76734922 +0000 UTC m=+2806.355590974" watchObservedRunningTime="2025-12-03 12:06:55.81458967 +0000 UTC m=+2806.402831434" Dec 03 12:06:55 crc kubenswrapper[5052]: I1203 12:06:55.826253 5052 scope.go:117] "RemoveContainer" containerID="a6ba1c579090bbd27d0787f86bce32466d51caef53ac643ede54d5bd5906a7d1" Dec 03 12:06:55 crc kubenswrapper[5052]: I1203 12:06:55.832317 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:06:55 crc kubenswrapper[5052]: E1203 12:06:55.833065 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="44c82024-af27-4702-bcf0-84d269840749" containerName="proxy-httpd" Dec 03 12:06:55 crc kubenswrapper[5052]: I1203 12:06:55.833101 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="44c82024-af27-4702-bcf0-84d269840749" containerName="proxy-httpd" Dec 03 12:06:55 crc kubenswrapper[5052]: E1203 12:06:55.833122 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="44c82024-af27-4702-bcf0-84d269840749" containerName="ceilometer-central-agent" Dec 03 12:06:55 crc kubenswrapper[5052]: I1203 12:06:55.833132 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="44c82024-af27-4702-bcf0-84d269840749" containerName="ceilometer-central-agent" Dec 03 12:06:55 crc kubenswrapper[5052]: E1203 12:06:55.833145 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="44c82024-af27-4702-bcf0-84d269840749" containerName="sg-core" Dec 03 12:06:55 crc kubenswrapper[5052]: I1203 12:06:55.833154 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="44c82024-af27-4702-bcf0-84d269840749" containerName="sg-core" Dec 03 12:06:55 crc kubenswrapper[5052]: E1203 12:06:55.833170 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="44c82024-af27-4702-bcf0-84d269840749" containerName="ceilometer-notification-agent" Dec 03 12:06:55 crc kubenswrapper[5052]: I1203 12:06:55.833179 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="44c82024-af27-4702-bcf0-84d269840749" containerName="ceilometer-notification-agent" Dec 03 12:06:55 crc kubenswrapper[5052]: I1203 12:06:55.833364 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="44c82024-af27-4702-bcf0-84d269840749" containerName="ceilometer-central-agent" Dec 03 12:06:55 crc kubenswrapper[5052]: I1203 12:06:55.833379 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="44c82024-af27-4702-bcf0-84d269840749" containerName="ceilometer-notification-agent" Dec 03 12:06:55 crc kubenswrapper[5052]: I1203 12:06:55.833396 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="44c82024-af27-4702-bcf0-84d269840749" containerName="proxy-httpd" Dec 03 12:06:55 crc kubenswrapper[5052]: I1203 12:06:55.833408 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="44c82024-af27-4702-bcf0-84d269840749" containerName="sg-core" Dec 03 12:06:55 crc kubenswrapper[5052]: I1203 12:06:55.835908 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:06:55 crc kubenswrapper[5052]: I1203 12:06:55.837949 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-config-data" Dec 03 12:06:55 crc kubenswrapper[5052]: I1203 12:06:55.838180 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-scripts" Dec 03 12:06:55 crc kubenswrapper[5052]: I1203 12:06:55.842686 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:06:55 crc kubenswrapper[5052]: I1203 12:06:55.870816 5052 scope.go:117] "RemoveContainer" containerID="b823c74e4bf02ada19452a0921d828e5c3c34365901bf95c5f92c016771c2c7a" Dec 03 12:06:55 crc kubenswrapper[5052]: I1203 12:06:55.896077 5052 scope.go:117] "RemoveContainer" containerID="3ae15f9196f3f57942131b77a6c833870ef71b802eb7d3cf07d241c9e3bc1ff6" Dec 03 12:06:55 crc kubenswrapper[5052]: E1203 12:06:55.896698 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3ae15f9196f3f57942131b77a6c833870ef71b802eb7d3cf07d241c9e3bc1ff6\": container with ID starting with 3ae15f9196f3f57942131b77a6c833870ef71b802eb7d3cf07d241c9e3bc1ff6 not found: ID does not exist" containerID="3ae15f9196f3f57942131b77a6c833870ef71b802eb7d3cf07d241c9e3bc1ff6" Dec 03 12:06:55 crc kubenswrapper[5052]: I1203 12:06:55.896756 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3ae15f9196f3f57942131b77a6c833870ef71b802eb7d3cf07d241c9e3bc1ff6"} err="failed to get container status \"3ae15f9196f3f57942131b77a6c833870ef71b802eb7d3cf07d241c9e3bc1ff6\": rpc error: code = NotFound desc = could not find container \"3ae15f9196f3f57942131b77a6c833870ef71b802eb7d3cf07d241c9e3bc1ff6\": container with ID starting with 3ae15f9196f3f57942131b77a6c833870ef71b802eb7d3cf07d241c9e3bc1ff6 not found: ID does not exist" Dec 03 12:06:55 crc kubenswrapper[5052]: I1203 12:06:55.896795 5052 scope.go:117] "RemoveContainer" containerID="b63af88913166ec529787d87ae4f944fddf84ea4021499b8b8c9fbdf71317343" Dec 03 12:06:55 crc kubenswrapper[5052]: E1203 12:06:55.897974 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b63af88913166ec529787d87ae4f944fddf84ea4021499b8b8c9fbdf71317343\": container with ID starting with b63af88913166ec529787d87ae4f944fddf84ea4021499b8b8c9fbdf71317343 not found: ID does not exist" containerID="b63af88913166ec529787d87ae4f944fddf84ea4021499b8b8c9fbdf71317343" Dec 03 12:06:55 crc kubenswrapper[5052]: I1203 12:06:55.898019 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b63af88913166ec529787d87ae4f944fddf84ea4021499b8b8c9fbdf71317343"} err="failed to get container status \"b63af88913166ec529787d87ae4f944fddf84ea4021499b8b8c9fbdf71317343\": rpc error: code = NotFound desc = could not find container \"b63af88913166ec529787d87ae4f944fddf84ea4021499b8b8c9fbdf71317343\": container with ID starting with b63af88913166ec529787d87ae4f944fddf84ea4021499b8b8c9fbdf71317343 not found: ID does not exist" Dec 03 12:06:55 crc kubenswrapper[5052]: I1203 12:06:55.898050 5052 scope.go:117] "RemoveContainer" containerID="a6ba1c579090bbd27d0787f86bce32466d51caef53ac643ede54d5bd5906a7d1" Dec 03 12:06:55 crc kubenswrapper[5052]: E1203 12:06:55.898493 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a6ba1c579090bbd27d0787f86bce32466d51caef53ac643ede54d5bd5906a7d1\": container with ID starting with a6ba1c579090bbd27d0787f86bce32466d51caef53ac643ede54d5bd5906a7d1 not found: ID does not exist" containerID="a6ba1c579090bbd27d0787f86bce32466d51caef53ac643ede54d5bd5906a7d1" Dec 03 12:06:55 crc kubenswrapper[5052]: I1203 12:06:55.898542 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a6ba1c579090bbd27d0787f86bce32466d51caef53ac643ede54d5bd5906a7d1"} err="failed to get container status \"a6ba1c579090bbd27d0787f86bce32466d51caef53ac643ede54d5bd5906a7d1\": rpc error: code = NotFound desc = could not find container \"a6ba1c579090bbd27d0787f86bce32466d51caef53ac643ede54d5bd5906a7d1\": container with ID starting with a6ba1c579090bbd27d0787f86bce32466d51caef53ac643ede54d5bd5906a7d1 not found: ID does not exist" Dec 03 12:06:55 crc kubenswrapper[5052]: I1203 12:06:55.898564 5052 scope.go:117] "RemoveContainer" containerID="b823c74e4bf02ada19452a0921d828e5c3c34365901bf95c5f92c016771c2c7a" Dec 03 12:06:55 crc kubenswrapper[5052]: E1203 12:06:55.899112 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b823c74e4bf02ada19452a0921d828e5c3c34365901bf95c5f92c016771c2c7a\": container with ID starting with b823c74e4bf02ada19452a0921d828e5c3c34365901bf95c5f92c016771c2c7a not found: ID does not exist" containerID="b823c74e4bf02ada19452a0921d828e5c3c34365901bf95c5f92c016771c2c7a" Dec 03 12:06:55 crc kubenswrapper[5052]: I1203 12:06:55.899153 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b823c74e4bf02ada19452a0921d828e5c3c34365901bf95c5f92c016771c2c7a"} err="failed to get container status \"b823c74e4bf02ada19452a0921d828e5c3c34365901bf95c5f92c016771c2c7a\": rpc error: code = NotFound desc = could not find container \"b823c74e4bf02ada19452a0921d828e5c3c34365901bf95c5f92c016771c2c7a\": container with ID starting with b823c74e4bf02ada19452a0921d828e5c3c34365901bf95c5f92c016771c2c7a not found: ID does not exist" Dec 03 12:06:55 crc kubenswrapper[5052]: I1203 12:06:55.936258 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/ec281373-bf8b-4f4d-8050-206d92074fc2-log-httpd\") pod \"ceilometer-0\" (UID: \"ec281373-bf8b-4f4d-8050-206d92074fc2\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:06:55 crc kubenswrapper[5052]: I1203 12:06:55.936722 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ec281373-bf8b-4f4d-8050-206d92074fc2-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"ec281373-bf8b-4f4d-8050-206d92074fc2\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:06:55 crc kubenswrapper[5052]: I1203 12:06:55.936789 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/ec281373-bf8b-4f4d-8050-206d92074fc2-scripts\") pod \"ceilometer-0\" (UID: \"ec281373-bf8b-4f4d-8050-206d92074fc2\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:06:55 crc kubenswrapper[5052]: I1203 12:06:55.936864 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/ec281373-bf8b-4f4d-8050-206d92074fc2-run-httpd\") pod \"ceilometer-0\" (UID: \"ec281373-bf8b-4f4d-8050-206d92074fc2\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:06:55 crc kubenswrapper[5052]: I1203 12:06:55.936981 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6v9q9\" (UniqueName: \"kubernetes.io/projected/ec281373-bf8b-4f4d-8050-206d92074fc2-kube-api-access-6v9q9\") pod \"ceilometer-0\" (UID: \"ec281373-bf8b-4f4d-8050-206d92074fc2\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:06:55 crc kubenswrapper[5052]: I1203 12:06:55.937104 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/ec281373-bf8b-4f4d-8050-206d92074fc2-config-data\") pod \"ceilometer-0\" (UID: \"ec281373-bf8b-4f4d-8050-206d92074fc2\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:06:55 crc kubenswrapper[5052]: I1203 12:06:55.937276 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/ec281373-bf8b-4f4d-8050-206d92074fc2-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"ec281373-bf8b-4f4d-8050-206d92074fc2\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:06:56 crc kubenswrapper[5052]: I1203 12:06:56.039350 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/ec281373-bf8b-4f4d-8050-206d92074fc2-log-httpd\") pod \"ceilometer-0\" (UID: \"ec281373-bf8b-4f4d-8050-206d92074fc2\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:06:56 crc kubenswrapper[5052]: I1203 12:06:56.039439 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ec281373-bf8b-4f4d-8050-206d92074fc2-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"ec281373-bf8b-4f4d-8050-206d92074fc2\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:06:56 crc kubenswrapper[5052]: I1203 12:06:56.039462 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/ec281373-bf8b-4f4d-8050-206d92074fc2-scripts\") pod \"ceilometer-0\" (UID: \"ec281373-bf8b-4f4d-8050-206d92074fc2\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:06:56 crc kubenswrapper[5052]: I1203 12:06:56.039493 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/ec281373-bf8b-4f4d-8050-206d92074fc2-run-httpd\") pod \"ceilometer-0\" (UID: \"ec281373-bf8b-4f4d-8050-206d92074fc2\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:06:56 crc kubenswrapper[5052]: I1203 12:06:56.039540 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-6v9q9\" (UniqueName: \"kubernetes.io/projected/ec281373-bf8b-4f4d-8050-206d92074fc2-kube-api-access-6v9q9\") pod \"ceilometer-0\" (UID: \"ec281373-bf8b-4f4d-8050-206d92074fc2\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:06:56 crc kubenswrapper[5052]: I1203 12:06:56.039589 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/ec281373-bf8b-4f4d-8050-206d92074fc2-config-data\") pod \"ceilometer-0\" (UID: \"ec281373-bf8b-4f4d-8050-206d92074fc2\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:06:56 crc kubenswrapper[5052]: I1203 12:06:56.039666 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/ec281373-bf8b-4f4d-8050-206d92074fc2-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"ec281373-bf8b-4f4d-8050-206d92074fc2\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:06:56 crc kubenswrapper[5052]: I1203 12:06:56.039891 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/ec281373-bf8b-4f4d-8050-206d92074fc2-log-httpd\") pod \"ceilometer-0\" (UID: \"ec281373-bf8b-4f4d-8050-206d92074fc2\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:06:56 crc kubenswrapper[5052]: I1203 12:06:56.040096 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/ec281373-bf8b-4f4d-8050-206d92074fc2-run-httpd\") pod \"ceilometer-0\" (UID: \"ec281373-bf8b-4f4d-8050-206d92074fc2\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:06:56 crc kubenswrapper[5052]: I1203 12:06:56.045307 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ec281373-bf8b-4f4d-8050-206d92074fc2-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"ec281373-bf8b-4f4d-8050-206d92074fc2\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:06:56 crc kubenswrapper[5052]: I1203 12:06:56.045838 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/ec281373-bf8b-4f4d-8050-206d92074fc2-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"ec281373-bf8b-4f4d-8050-206d92074fc2\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:06:56 crc kubenswrapper[5052]: I1203 12:06:56.061375 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/ec281373-bf8b-4f4d-8050-206d92074fc2-config-data\") pod \"ceilometer-0\" (UID: \"ec281373-bf8b-4f4d-8050-206d92074fc2\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:06:56 crc kubenswrapper[5052]: I1203 12:06:56.063230 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/ec281373-bf8b-4f4d-8050-206d92074fc2-scripts\") pod \"ceilometer-0\" (UID: \"ec281373-bf8b-4f4d-8050-206d92074fc2\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:06:56 crc kubenswrapper[5052]: I1203 12:06:56.081734 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-6v9q9\" (UniqueName: \"kubernetes.io/projected/ec281373-bf8b-4f4d-8050-206d92074fc2-kube-api-access-6v9q9\") pod \"ceilometer-0\" (UID: \"ec281373-bf8b-4f4d-8050-206d92074fc2\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:06:56 crc kubenswrapper[5052]: I1203 12:06:56.164041 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:06:56 crc kubenswrapper[5052]: I1203 12:06:56.712123 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:06:56 crc kubenswrapper[5052]: I1203 12:06:56.772564 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"ec281373-bf8b-4f4d-8050-206d92074fc2","Type":"ContainerStarted","Data":"40617227a3c892bfd6e9c50dad8dc06942d41abb865300b6d39b74ad2eefa584"} Dec 03 12:06:57 crc kubenswrapper[5052]: I1203 12:06:57.691128 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="44c82024-af27-4702-bcf0-84d269840749" path="/var/lib/kubelet/pods/44c82024-af27-4702-bcf0-84d269840749/volumes" Dec 03 12:06:58 crc kubenswrapper[5052]: I1203 12:06:58.791869 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"ec281373-bf8b-4f4d-8050-206d92074fc2","Type":"ContainerStarted","Data":"c8cec5b21931f0c586ae94d07bb14801f9109afc2ce06eaff70de5126b8d93d0"} Dec 03 12:06:59 crc kubenswrapper[5052]: I1203 12:06:59.142283 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:06:59 crc kubenswrapper[5052]: I1203 12:06:59.581232 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell0-cell-mapping-5r7kw"] Dec 03 12:06:59 crc kubenswrapper[5052]: I1203 12:06:59.582416 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-cell-mapping-5r7kw" Dec 03 12:06:59 crc kubenswrapper[5052]: I1203 12:06:59.591865 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell0-manage-scripts" Dec 03 12:06:59 crc kubenswrapper[5052]: I1203 12:06:59.592121 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell0-manage-config-data" Dec 03 12:06:59 crc kubenswrapper[5052]: I1203 12:06:59.601298 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell0-cell-mapping-5r7kw"] Dec 03 12:06:59 crc kubenswrapper[5052]: I1203 12:06:59.618376 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/8f59423a-caa1-428c-920c-59bb1a3b618d-scripts\") pod \"nova-cell0-cell-mapping-5r7kw\" (UID: \"8f59423a-caa1-428c-920c-59bb1a3b618d\") " pod="openstack-kuttl-tests/nova-cell0-cell-mapping-5r7kw" Dec 03 12:06:59 crc kubenswrapper[5052]: I1203 12:06:59.618465 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8f59423a-caa1-428c-920c-59bb1a3b618d-combined-ca-bundle\") pod \"nova-cell0-cell-mapping-5r7kw\" (UID: \"8f59423a-caa1-428c-920c-59bb1a3b618d\") " pod="openstack-kuttl-tests/nova-cell0-cell-mapping-5r7kw" Dec 03 12:06:59 crc kubenswrapper[5052]: I1203 12:06:59.618540 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2vqqt\" (UniqueName: \"kubernetes.io/projected/8f59423a-caa1-428c-920c-59bb1a3b618d-kube-api-access-2vqqt\") pod \"nova-cell0-cell-mapping-5r7kw\" (UID: \"8f59423a-caa1-428c-920c-59bb1a3b618d\") " pod="openstack-kuttl-tests/nova-cell0-cell-mapping-5r7kw" Dec 03 12:06:59 crc kubenswrapper[5052]: I1203 12:06:59.618785 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/8f59423a-caa1-428c-920c-59bb1a3b618d-config-data\") pod \"nova-cell0-cell-mapping-5r7kw\" (UID: \"8f59423a-caa1-428c-920c-59bb1a3b618d\") " pod="openstack-kuttl-tests/nova-cell0-cell-mapping-5r7kw" Dec 03 12:06:59 crc kubenswrapper[5052]: I1203 12:06:59.744039 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/8f59423a-caa1-428c-920c-59bb1a3b618d-scripts\") pod \"nova-cell0-cell-mapping-5r7kw\" (UID: \"8f59423a-caa1-428c-920c-59bb1a3b618d\") " pod="openstack-kuttl-tests/nova-cell0-cell-mapping-5r7kw" Dec 03 12:06:59 crc kubenswrapper[5052]: I1203 12:06:59.745722 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8f59423a-caa1-428c-920c-59bb1a3b618d-combined-ca-bundle\") pod \"nova-cell0-cell-mapping-5r7kw\" (UID: \"8f59423a-caa1-428c-920c-59bb1a3b618d\") " pod="openstack-kuttl-tests/nova-cell0-cell-mapping-5r7kw" Dec 03 12:06:59 crc kubenswrapper[5052]: I1203 12:06:59.748366 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-2vqqt\" (UniqueName: \"kubernetes.io/projected/8f59423a-caa1-428c-920c-59bb1a3b618d-kube-api-access-2vqqt\") pod \"nova-cell0-cell-mapping-5r7kw\" (UID: \"8f59423a-caa1-428c-920c-59bb1a3b618d\") " pod="openstack-kuttl-tests/nova-cell0-cell-mapping-5r7kw" Dec 03 12:06:59 crc kubenswrapper[5052]: I1203 12:06:59.748725 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/8f59423a-caa1-428c-920c-59bb1a3b618d-config-data\") pod \"nova-cell0-cell-mapping-5r7kw\" (UID: \"8f59423a-caa1-428c-920c-59bb1a3b618d\") " pod="openstack-kuttl-tests/nova-cell0-cell-mapping-5r7kw" Dec 03 12:06:59 crc kubenswrapper[5052]: I1203 12:06:59.760585 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 12:06:59 crc kubenswrapper[5052]: I1203 12:06:59.762979 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:06:59 crc kubenswrapper[5052]: I1203 12:06:59.763009 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/8f59423a-caa1-428c-920c-59bb1a3b618d-scripts\") pod \"nova-cell0-cell-mapping-5r7kw\" (UID: \"8f59423a-caa1-428c-920c-59bb1a3b618d\") " pod="openstack-kuttl-tests/nova-cell0-cell-mapping-5r7kw" Dec 03 12:06:59 crc kubenswrapper[5052]: I1203 12:06:59.767236 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8f59423a-caa1-428c-920c-59bb1a3b618d-combined-ca-bundle\") pod \"nova-cell0-cell-mapping-5r7kw\" (UID: \"8f59423a-caa1-428c-920c-59bb1a3b618d\") " pod="openstack-kuttl-tests/nova-cell0-cell-mapping-5r7kw" Dec 03 12:06:59 crc kubenswrapper[5052]: I1203 12:06:59.770915 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell1-novncproxy-config-data" Dec 03 12:06:59 crc kubenswrapper[5052]: I1203 12:06:59.792186 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/8f59423a-caa1-428c-920c-59bb1a3b618d-config-data\") pod \"nova-cell0-cell-mapping-5r7kw\" (UID: \"8f59423a-caa1-428c-920c-59bb1a3b618d\") " pod="openstack-kuttl-tests/nova-cell0-cell-mapping-5r7kw" Dec 03 12:06:59 crc kubenswrapper[5052]: I1203 12:06:59.810162 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-2vqqt\" (UniqueName: \"kubernetes.io/projected/8f59423a-caa1-428c-920c-59bb1a3b618d-kube-api-access-2vqqt\") pod \"nova-cell0-cell-mapping-5r7kw\" (UID: \"8f59423a-caa1-428c-920c-59bb1a3b618d\") " pod="openstack-kuttl-tests/nova-cell0-cell-mapping-5r7kw" Dec 03 12:06:59 crc kubenswrapper[5052]: I1203 12:06:59.817060 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 12:06:59 crc kubenswrapper[5052]: I1203 12:06:59.835603 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"ec281373-bf8b-4f4d-8050-206d92074fc2","Type":"ContainerStarted","Data":"f00b78456ae51af13add1990be39cd3856c4bfb8861d683f6ed3f06327226113"} Dec 03 12:06:59 crc kubenswrapper[5052]: I1203 12:06:59.851611 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8zhvj\" (UniqueName: \"kubernetes.io/projected/c55dee8f-7615-47cc-a08f-65d79bd2f62d-kube-api-access-8zhvj\") pod \"nova-cell1-novncproxy-0\" (UID: \"c55dee8f-7615-47cc-a08f-65d79bd2f62d\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:06:59 crc kubenswrapper[5052]: I1203 12:06:59.851695 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/c55dee8f-7615-47cc-a08f-65d79bd2f62d-config-data\") pod \"nova-cell1-novncproxy-0\" (UID: \"c55dee8f-7615-47cc-a08f-65d79bd2f62d\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:06:59 crc kubenswrapper[5052]: I1203 12:06:59.851728 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c55dee8f-7615-47cc-a08f-65d79bd2f62d-combined-ca-bundle\") pod \"nova-cell1-novncproxy-0\" (UID: \"c55dee8f-7615-47cc-a08f-65d79bd2f62d\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:06:59 crc kubenswrapper[5052]: I1203 12:06:59.868218 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:06:59 crc kubenswrapper[5052]: I1203 12:06:59.872095 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:06:59 crc kubenswrapper[5052]: I1203 12:06:59.876628 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-metadata-config-data" Dec 03 12:06:59 crc kubenswrapper[5052]: I1203 12:06:59.912852 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-cell-mapping-5r7kw" Dec 03 12:06:59 crc kubenswrapper[5052]: I1203 12:06:59.946723 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:06:59 crc kubenswrapper[5052]: I1203 12:06:59.953297 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/3ef81bc8-170a-490d-8e81-4010c16452c1-config-data\") pod \"nova-metadata-0\" (UID: \"3ef81bc8-170a-490d-8e81-4010c16452c1\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:06:59 crc kubenswrapper[5052]: I1203 12:06:59.953368 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3ef81bc8-170a-490d-8e81-4010c16452c1-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"3ef81bc8-170a-490d-8e81-4010c16452c1\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:06:59 crc kubenswrapper[5052]: I1203 12:06:59.953395 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kc2v2\" (UniqueName: \"kubernetes.io/projected/3ef81bc8-170a-490d-8e81-4010c16452c1-kube-api-access-kc2v2\") pod \"nova-metadata-0\" (UID: \"3ef81bc8-170a-490d-8e81-4010c16452c1\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:06:59 crc kubenswrapper[5052]: I1203 12:06:59.953422 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-8zhvj\" (UniqueName: \"kubernetes.io/projected/c55dee8f-7615-47cc-a08f-65d79bd2f62d-kube-api-access-8zhvj\") pod \"nova-cell1-novncproxy-0\" (UID: \"c55dee8f-7615-47cc-a08f-65d79bd2f62d\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:06:59 crc kubenswrapper[5052]: I1203 12:06:59.953446 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/3ef81bc8-170a-490d-8e81-4010c16452c1-logs\") pod \"nova-metadata-0\" (UID: \"3ef81bc8-170a-490d-8e81-4010c16452c1\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:06:59 crc kubenswrapper[5052]: I1203 12:06:59.953500 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/c55dee8f-7615-47cc-a08f-65d79bd2f62d-config-data\") pod \"nova-cell1-novncproxy-0\" (UID: \"c55dee8f-7615-47cc-a08f-65d79bd2f62d\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:06:59 crc kubenswrapper[5052]: I1203 12:06:59.953546 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c55dee8f-7615-47cc-a08f-65d79bd2f62d-combined-ca-bundle\") pod \"nova-cell1-novncproxy-0\" (UID: \"c55dee8f-7615-47cc-a08f-65d79bd2f62d\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:06:59 crc kubenswrapper[5052]: I1203 12:06:59.970062 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/c55dee8f-7615-47cc-a08f-65d79bd2f62d-config-data\") pod \"nova-cell1-novncproxy-0\" (UID: \"c55dee8f-7615-47cc-a08f-65d79bd2f62d\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:06:59 crc kubenswrapper[5052]: I1203 12:06:59.970718 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c55dee8f-7615-47cc-a08f-65d79bd2f62d-combined-ca-bundle\") pod \"nova-cell1-novncproxy-0\" (UID: \"c55dee8f-7615-47cc-a08f-65d79bd2f62d\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:06:59 crc kubenswrapper[5052]: I1203 12:06:59.981914 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-8zhvj\" (UniqueName: \"kubernetes.io/projected/c55dee8f-7615-47cc-a08f-65d79bd2f62d-kube-api-access-8zhvj\") pod \"nova-cell1-novncproxy-0\" (UID: \"c55dee8f-7615-47cc-a08f-65d79bd2f62d\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:06:59 crc kubenswrapper[5052]: I1203 12:06:59.990301 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:06:59 crc kubenswrapper[5052]: I1203 12:06:59.992551 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:07:00 crc kubenswrapper[5052]: I1203 12:07:00.000950 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-scheduler-config-data" Dec 03 12:07:00 crc kubenswrapper[5052]: I1203 12:07:00.011292 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:07:00 crc kubenswrapper[5052]: I1203 12:07:00.033215 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:07:00 crc kubenswrapper[5052]: I1203 12:07:00.034937 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:07:00 crc kubenswrapper[5052]: I1203 12:07:00.040060 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-api-config-data" Dec 03 12:07:00 crc kubenswrapper[5052]: I1203 12:07:00.043479 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:07:00 crc kubenswrapper[5052]: I1203 12:07:00.056381 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a7579c63-c143-459e-9d94-eddb02f810cf-combined-ca-bundle\") pod \"nova-scheduler-0\" (UID: \"a7579c63-c143-459e-9d94-eddb02f810cf\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:07:00 crc kubenswrapper[5052]: I1203 12:07:00.056486 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9d4cx\" (UniqueName: \"kubernetes.io/projected/a7579c63-c143-459e-9d94-eddb02f810cf-kube-api-access-9d4cx\") pod \"nova-scheduler-0\" (UID: \"a7579c63-c143-459e-9d94-eddb02f810cf\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:07:00 crc kubenswrapper[5052]: I1203 12:07:00.056572 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a7579c63-c143-459e-9d94-eddb02f810cf-config-data\") pod \"nova-scheduler-0\" (UID: \"a7579c63-c143-459e-9d94-eddb02f810cf\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:07:00 crc kubenswrapper[5052]: I1203 12:07:00.056735 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/3ef81bc8-170a-490d-8e81-4010c16452c1-config-data\") pod \"nova-metadata-0\" (UID: \"3ef81bc8-170a-490d-8e81-4010c16452c1\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:07:00 crc kubenswrapper[5052]: I1203 12:07:00.056890 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3ef81bc8-170a-490d-8e81-4010c16452c1-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"3ef81bc8-170a-490d-8e81-4010c16452c1\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:07:00 crc kubenswrapper[5052]: I1203 12:07:00.056920 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-kc2v2\" (UniqueName: \"kubernetes.io/projected/3ef81bc8-170a-490d-8e81-4010c16452c1-kube-api-access-kc2v2\") pod \"nova-metadata-0\" (UID: \"3ef81bc8-170a-490d-8e81-4010c16452c1\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:07:00 crc kubenswrapper[5052]: I1203 12:07:00.056971 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/3ef81bc8-170a-490d-8e81-4010c16452c1-logs\") pod \"nova-metadata-0\" (UID: \"3ef81bc8-170a-490d-8e81-4010c16452c1\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:07:00 crc kubenswrapper[5052]: I1203 12:07:00.057921 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/3ef81bc8-170a-490d-8e81-4010c16452c1-logs\") pod \"nova-metadata-0\" (UID: \"3ef81bc8-170a-490d-8e81-4010c16452c1\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:07:00 crc kubenswrapper[5052]: I1203 12:07:00.069961 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/3ef81bc8-170a-490d-8e81-4010c16452c1-config-data\") pod \"nova-metadata-0\" (UID: \"3ef81bc8-170a-490d-8e81-4010c16452c1\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:07:00 crc kubenswrapper[5052]: I1203 12:07:00.071179 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3ef81bc8-170a-490d-8e81-4010c16452c1-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"3ef81bc8-170a-490d-8e81-4010c16452c1\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:07:00 crc kubenswrapper[5052]: I1203 12:07:00.074771 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-kc2v2\" (UniqueName: \"kubernetes.io/projected/3ef81bc8-170a-490d-8e81-4010c16452c1-kube-api-access-kc2v2\") pod \"nova-metadata-0\" (UID: \"3ef81bc8-170a-490d-8e81-4010c16452c1\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:07:00 crc kubenswrapper[5052]: I1203 12:07:00.160066 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/bdbb00d9-d8ce-4a03-843f-ab8441279186-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"bdbb00d9-d8ce-4a03-843f-ab8441279186\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:07:00 crc kubenswrapper[5052]: I1203 12:07:00.160568 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x5b47\" (UniqueName: \"kubernetes.io/projected/bdbb00d9-d8ce-4a03-843f-ab8441279186-kube-api-access-x5b47\") pod \"nova-api-0\" (UID: \"bdbb00d9-d8ce-4a03-843f-ab8441279186\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:07:00 crc kubenswrapper[5052]: I1203 12:07:00.160646 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/bdbb00d9-d8ce-4a03-843f-ab8441279186-config-data\") pod \"nova-api-0\" (UID: \"bdbb00d9-d8ce-4a03-843f-ab8441279186\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:07:00 crc kubenswrapper[5052]: I1203 12:07:00.160691 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a7579c63-c143-459e-9d94-eddb02f810cf-combined-ca-bundle\") pod \"nova-scheduler-0\" (UID: \"a7579c63-c143-459e-9d94-eddb02f810cf\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:07:00 crc kubenswrapper[5052]: I1203 12:07:00.160724 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/bdbb00d9-d8ce-4a03-843f-ab8441279186-logs\") pod \"nova-api-0\" (UID: \"bdbb00d9-d8ce-4a03-843f-ab8441279186\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:07:00 crc kubenswrapper[5052]: I1203 12:07:00.160754 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-9d4cx\" (UniqueName: \"kubernetes.io/projected/a7579c63-c143-459e-9d94-eddb02f810cf-kube-api-access-9d4cx\") pod \"nova-scheduler-0\" (UID: \"a7579c63-c143-459e-9d94-eddb02f810cf\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:07:00 crc kubenswrapper[5052]: I1203 12:07:00.160804 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a7579c63-c143-459e-9d94-eddb02f810cf-config-data\") pod \"nova-scheduler-0\" (UID: \"a7579c63-c143-459e-9d94-eddb02f810cf\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:07:00 crc kubenswrapper[5052]: I1203 12:07:00.176207 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a7579c63-c143-459e-9d94-eddb02f810cf-config-data\") pod \"nova-scheduler-0\" (UID: \"a7579c63-c143-459e-9d94-eddb02f810cf\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:07:00 crc kubenswrapper[5052]: I1203 12:07:00.176299 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a7579c63-c143-459e-9d94-eddb02f810cf-combined-ca-bundle\") pod \"nova-scheduler-0\" (UID: \"a7579c63-c143-459e-9d94-eddb02f810cf\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:07:00 crc kubenswrapper[5052]: I1203 12:07:00.181147 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:07:00 crc kubenswrapper[5052]: I1203 12:07:00.201435 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:07:00 crc kubenswrapper[5052]: I1203 12:07:00.204129 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-9d4cx\" (UniqueName: \"kubernetes.io/projected/a7579c63-c143-459e-9d94-eddb02f810cf-kube-api-access-9d4cx\") pod \"nova-scheduler-0\" (UID: \"a7579c63-c143-459e-9d94-eddb02f810cf\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:07:00 crc kubenswrapper[5052]: I1203 12:07:00.229144 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:07:00 crc kubenswrapper[5052]: I1203 12:07:00.264599 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/bdbb00d9-d8ce-4a03-843f-ab8441279186-config-data\") pod \"nova-api-0\" (UID: \"bdbb00d9-d8ce-4a03-843f-ab8441279186\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:07:00 crc kubenswrapper[5052]: I1203 12:07:00.264672 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/bdbb00d9-d8ce-4a03-843f-ab8441279186-logs\") pod \"nova-api-0\" (UID: \"bdbb00d9-d8ce-4a03-843f-ab8441279186\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:07:00 crc kubenswrapper[5052]: I1203 12:07:00.264747 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/bdbb00d9-d8ce-4a03-843f-ab8441279186-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"bdbb00d9-d8ce-4a03-843f-ab8441279186\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:07:00 crc kubenswrapper[5052]: I1203 12:07:00.266227 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-x5b47\" (UniqueName: \"kubernetes.io/projected/bdbb00d9-d8ce-4a03-843f-ab8441279186-kube-api-access-x5b47\") pod \"nova-api-0\" (UID: \"bdbb00d9-d8ce-4a03-843f-ab8441279186\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:07:00 crc kubenswrapper[5052]: I1203 12:07:00.269741 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/bdbb00d9-d8ce-4a03-843f-ab8441279186-logs\") pod \"nova-api-0\" (UID: \"bdbb00d9-d8ce-4a03-843f-ab8441279186\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:07:00 crc kubenswrapper[5052]: I1203 12:07:00.270811 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/bdbb00d9-d8ce-4a03-843f-ab8441279186-config-data\") pod \"nova-api-0\" (UID: \"bdbb00d9-d8ce-4a03-843f-ab8441279186\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:07:00 crc kubenswrapper[5052]: I1203 12:07:00.280210 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/bdbb00d9-d8ce-4a03-843f-ab8441279186-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"bdbb00d9-d8ce-4a03-843f-ab8441279186\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:07:00 crc kubenswrapper[5052]: I1203 12:07:00.289355 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-x5b47\" (UniqueName: \"kubernetes.io/projected/bdbb00d9-d8ce-4a03-843f-ab8441279186-kube-api-access-x5b47\") pod \"nova-api-0\" (UID: \"bdbb00d9-d8ce-4a03-843f-ab8441279186\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:07:00 crc kubenswrapper[5052]: I1203 12:07:00.531025 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell0-cell-mapping-5r7kw"] Dec 03 12:07:00 crc kubenswrapper[5052]: I1203 12:07:00.544711 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:07:00 crc kubenswrapper[5052]: W1203 12:07:00.550247 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8f59423a_caa1_428c_920c_59bb1a3b618d.slice/crio-6bd2d42e7b4ee6fe040d7a6f6fc643074bb3e23f8766bd6310e6b1059219041b WatchSource:0}: Error finding container 6bd2d42e7b4ee6fe040d7a6f6fc643074bb3e23f8766bd6310e6b1059219041b: Status 404 returned error can't find the container with id 6bd2d42e7b4ee6fe040d7a6f6fc643074bb3e23f8766bd6310e6b1059219041b Dec 03 12:07:00 crc kubenswrapper[5052]: I1203 12:07:00.600846 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 12:07:00 crc kubenswrapper[5052]: I1203 12:07:00.746980 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-db-sync-sxlxt"] Dec 03 12:07:00 crc kubenswrapper[5052]: I1203 12:07:00.749610 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-sxlxt" Dec 03 12:07:00 crc kubenswrapper[5052]: I1203 12:07:00.753452 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell1-conductor-config-data" Dec 03 12:07:00 crc kubenswrapper[5052]: I1203 12:07:00.753689 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell1-conductor-scripts" Dec 03 12:07:00 crc kubenswrapper[5052]: I1203 12:07:00.776663 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-db-sync-sxlxt"] Dec 03 12:07:00 crc kubenswrapper[5052]: I1203 12:07:00.867043 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-cell-mapping-5r7kw" event={"ID":"8f59423a-caa1-428c-920c-59bb1a3b618d","Type":"ContainerStarted","Data":"6bd2d42e7b4ee6fe040d7a6f6fc643074bb3e23f8766bd6310e6b1059219041b"} Dec 03 12:07:00 crc kubenswrapper[5052]: I1203 12:07:00.867317 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:07:00 crc kubenswrapper[5052]: I1203 12:07:00.881191 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"ec281373-bf8b-4f4d-8050-206d92074fc2","Type":"ContainerStarted","Data":"c689b558d9359e9ed0e4c3c4a7fbba0ce2251869d67d2d0fc23df0e151735dde"} Dec 03 12:07:00 crc kubenswrapper[5052]: I1203 12:07:00.881500 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:07:00 crc kubenswrapper[5052]: I1203 12:07:00.883358 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2066fe4a-e7cc-44a5-b7d9-499d19a40c12-combined-ca-bundle\") pod \"nova-cell1-conductor-db-sync-sxlxt\" (UID: \"2066fe4a-e7cc-44a5-b7d9-499d19a40c12\") " pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-sxlxt" Dec 03 12:07:00 crc kubenswrapper[5052]: I1203 12:07:00.883446 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/2066fe4a-e7cc-44a5-b7d9-499d19a40c12-config-data\") pod \"nova-cell1-conductor-db-sync-sxlxt\" (UID: \"2066fe4a-e7cc-44a5-b7d9-499d19a40c12\") " pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-sxlxt" Dec 03 12:07:00 crc kubenswrapper[5052]: I1203 12:07:00.883584 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6b97m\" (UniqueName: \"kubernetes.io/projected/2066fe4a-e7cc-44a5-b7d9-499d19a40c12-kube-api-access-6b97m\") pod \"nova-cell1-conductor-db-sync-sxlxt\" (UID: \"2066fe4a-e7cc-44a5-b7d9-499d19a40c12\") " pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-sxlxt" Dec 03 12:07:00 crc kubenswrapper[5052]: I1203 12:07:00.883655 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/2066fe4a-e7cc-44a5-b7d9-499d19a40c12-scripts\") pod \"nova-cell1-conductor-db-sync-sxlxt\" (UID: \"2066fe4a-e7cc-44a5-b7d9-499d19a40c12\") " pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-sxlxt" Dec 03 12:07:00 crc kubenswrapper[5052]: I1203 12:07:00.884095 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" event={"ID":"c55dee8f-7615-47cc-a08f-65d79bd2f62d","Type":"ContainerStarted","Data":"660ea93a146e0ec6be1285d711c93c62ce3670050fb25b1fe6dd2da617b4bfe4"} Dec 03 12:07:00 crc kubenswrapper[5052]: I1203 12:07:00.985904 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2066fe4a-e7cc-44a5-b7d9-499d19a40c12-combined-ca-bundle\") pod \"nova-cell1-conductor-db-sync-sxlxt\" (UID: \"2066fe4a-e7cc-44a5-b7d9-499d19a40c12\") " pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-sxlxt" Dec 03 12:07:00 crc kubenswrapper[5052]: I1203 12:07:00.986168 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/2066fe4a-e7cc-44a5-b7d9-499d19a40c12-config-data\") pod \"nova-cell1-conductor-db-sync-sxlxt\" (UID: \"2066fe4a-e7cc-44a5-b7d9-499d19a40c12\") " pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-sxlxt" Dec 03 12:07:00 crc kubenswrapper[5052]: I1203 12:07:00.986378 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-6b97m\" (UniqueName: \"kubernetes.io/projected/2066fe4a-e7cc-44a5-b7d9-499d19a40c12-kube-api-access-6b97m\") pod \"nova-cell1-conductor-db-sync-sxlxt\" (UID: \"2066fe4a-e7cc-44a5-b7d9-499d19a40c12\") " pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-sxlxt" Dec 03 12:07:00 crc kubenswrapper[5052]: I1203 12:07:00.986457 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/2066fe4a-e7cc-44a5-b7d9-499d19a40c12-scripts\") pod \"nova-cell1-conductor-db-sync-sxlxt\" (UID: \"2066fe4a-e7cc-44a5-b7d9-499d19a40c12\") " pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-sxlxt" Dec 03 12:07:00 crc kubenswrapper[5052]: I1203 12:07:00.993259 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/2066fe4a-e7cc-44a5-b7d9-499d19a40c12-scripts\") pod \"nova-cell1-conductor-db-sync-sxlxt\" (UID: \"2066fe4a-e7cc-44a5-b7d9-499d19a40c12\") " pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-sxlxt" Dec 03 12:07:00 crc kubenswrapper[5052]: I1203 12:07:00.993709 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2066fe4a-e7cc-44a5-b7d9-499d19a40c12-combined-ca-bundle\") pod \"nova-cell1-conductor-db-sync-sxlxt\" (UID: \"2066fe4a-e7cc-44a5-b7d9-499d19a40c12\") " pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-sxlxt" Dec 03 12:07:00 crc kubenswrapper[5052]: I1203 12:07:00.999172 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/2066fe4a-e7cc-44a5-b7d9-499d19a40c12-config-data\") pod \"nova-cell1-conductor-db-sync-sxlxt\" (UID: \"2066fe4a-e7cc-44a5-b7d9-499d19a40c12\") " pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-sxlxt" Dec 03 12:07:01 crc kubenswrapper[5052]: I1203 12:07:01.007726 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-6b97m\" (UniqueName: \"kubernetes.io/projected/2066fe4a-e7cc-44a5-b7d9-499d19a40c12-kube-api-access-6b97m\") pod \"nova-cell1-conductor-db-sync-sxlxt\" (UID: \"2066fe4a-e7cc-44a5-b7d9-499d19a40c12\") " pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-sxlxt" Dec 03 12:07:01 crc kubenswrapper[5052]: I1203 12:07:01.111878 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:07:01 crc kubenswrapper[5052]: I1203 12:07:01.112815 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-sxlxt" Dec 03 12:07:01 crc kubenswrapper[5052]: I1203 12:07:01.527089 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-db-sync-sxlxt"] Dec 03 12:07:01 crc kubenswrapper[5052]: I1203 12:07:01.895597 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"bdbb00d9-d8ce-4a03-843f-ab8441279186","Type":"ContainerStarted","Data":"11c4c6496ed9aa094f5dc8ac4a34886ebc95b89aadefbfc8fa7de82792231539"} Dec 03 12:07:01 crc kubenswrapper[5052]: I1203 12:07:01.896759 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"bdbb00d9-d8ce-4a03-843f-ab8441279186","Type":"ContainerStarted","Data":"b203ee66bc093decab58c0f9b826923dd1b1540423969dbbc76c76b27c3e6cb1"} Dec 03 12:07:01 crc kubenswrapper[5052]: I1203 12:07:01.897296 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" event={"ID":"c55dee8f-7615-47cc-a08f-65d79bd2f62d","Type":"ContainerStarted","Data":"fad19e1dd1b0bafc303e31cd36da9923ab67f02480e6f7d2d88248cf19a2ca04"} Dec 03 12:07:01 crc kubenswrapper[5052]: I1203 12:07:01.905059 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-cell-mapping-5r7kw" event={"ID":"8f59423a-caa1-428c-920c-59bb1a3b618d","Type":"ContainerStarted","Data":"15a5f7c7cd882608a85275b1c6458b1bf51b486cc6488418a2e35990d2b843c0"} Dec 03 12:07:01 crc kubenswrapper[5052]: I1203 12:07:01.913762 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"a7579c63-c143-459e-9d94-eddb02f810cf","Type":"ContainerStarted","Data":"14f0cd9f4b15ca56a0c624489f9c66aba039f94373eebbb3da75d53cb4de839a"} Dec 03 12:07:01 crc kubenswrapper[5052]: I1203 12:07:01.913812 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"a7579c63-c143-459e-9d94-eddb02f810cf","Type":"ContainerStarted","Data":"36ff3bba7f293b1b02a6cc5f70a44fea80f9fee7a7af0ec53a39960e555d458b"} Dec 03 12:07:01 crc kubenswrapper[5052]: I1203 12:07:01.927688 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" podStartSLOduration=2.92767206 podStartE2EDuration="2.92767206s" podCreationTimestamp="2025-12-03 12:06:59 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:07:01.925037355 +0000 UTC m=+2812.513279119" watchObservedRunningTime="2025-12-03 12:07:01.92767206 +0000 UTC m=+2812.515913824" Dec 03 12:07:01 crc kubenswrapper[5052]: I1203 12:07:01.930364 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"3ef81bc8-170a-490d-8e81-4010c16452c1","Type":"ContainerStarted","Data":"84bea333f871cb73280c841a737727cb24be09ebf4b63a73ea6cd1f5776a0128"} Dec 03 12:07:01 crc kubenswrapper[5052]: I1203 12:07:01.930425 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"3ef81bc8-170a-490d-8e81-4010c16452c1","Type":"ContainerStarted","Data":"e78181cf88f9aa59bd0d70304297a560b9777ea787ea31d5527de0a3eeec6025"} Dec 03 12:07:01 crc kubenswrapper[5052]: I1203 12:07:01.932130 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-sxlxt" event={"ID":"2066fe4a-e7cc-44a5-b7d9-499d19a40c12","Type":"ContainerStarted","Data":"cd81b7a4a00c975a4914d128b35e8aa28ba1452fc6b882208d8fc5c05e72e3f8"} Dec 03 12:07:01 crc kubenswrapper[5052]: I1203 12:07:01.942163 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-scheduler-0" podStartSLOduration=2.9421466240000003 podStartE2EDuration="2.942146624s" podCreationTimestamp="2025-12-03 12:06:59 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:07:01.938015606 +0000 UTC m=+2812.526257390" watchObservedRunningTime="2025-12-03 12:07:01.942146624 +0000 UTC m=+2812.530388388" Dec 03 12:07:01 crc kubenswrapper[5052]: I1203 12:07:01.962497 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-cell0-cell-mapping-5r7kw" podStartSLOduration=2.962476985 podStartE2EDuration="2.962476985s" podCreationTimestamp="2025-12-03 12:06:59 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:07:01.955721152 +0000 UTC m=+2812.543962916" watchObservedRunningTime="2025-12-03 12:07:01.962476985 +0000 UTC m=+2812.550718749" Dec 03 12:07:02 crc kubenswrapper[5052]: I1203 12:07:02.945856 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"3ef81bc8-170a-490d-8e81-4010c16452c1","Type":"ContainerStarted","Data":"07aa5b46542b59873f354ef15148617b80b850ffea3776c0a68ec27d1b4c327e"} Dec 03 12:07:02 crc kubenswrapper[5052]: I1203 12:07:02.950853 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-sxlxt" event={"ID":"2066fe4a-e7cc-44a5-b7d9-499d19a40c12","Type":"ContainerStarted","Data":"2c81d86ba7b2260bf28aacf6f22b1bf41b3bce50a7118749afee64eacc6c95a5"} Dec 03 12:07:02 crc kubenswrapper[5052]: I1203 12:07:02.953420 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"bdbb00d9-d8ce-4a03-843f-ab8441279186","Type":"ContainerStarted","Data":"70048e89dae40428b8f05096fcb64249be714c08577a13e9cceda0356a73f6ae"} Dec 03 12:07:02 crc kubenswrapper[5052]: I1203 12:07:02.969712 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-metadata-0" podStartSLOduration=3.96969328 podStartE2EDuration="3.96969328s" podCreationTimestamp="2025-12-03 12:06:59 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:07:02.968358302 +0000 UTC m=+2813.556600086" watchObservedRunningTime="2025-12-03 12:07:02.96969328 +0000 UTC m=+2813.557935044" Dec 03 12:07:03 crc kubenswrapper[5052]: I1203 12:07:03.002575 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-sxlxt" podStartSLOduration=3.00255337 podStartE2EDuration="3.00255337s" podCreationTimestamp="2025-12-03 12:07:00 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:07:02.98997641 +0000 UTC m=+2813.578218174" watchObservedRunningTime="2025-12-03 12:07:03.00255337 +0000 UTC m=+2813.590795134" Dec 03 12:07:03 crc kubenswrapper[5052]: I1203 12:07:03.018400 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-api-0" podStartSLOduration=4.018376122 podStartE2EDuration="4.018376122s" podCreationTimestamp="2025-12-03 12:06:59 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:07:03.01691827 +0000 UTC m=+2813.605160034" watchObservedRunningTime="2025-12-03 12:07:03.018376122 +0000 UTC m=+2813.606617896" Dec 03 12:07:03 crc kubenswrapper[5052]: I1203 12:07:03.049844 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:07:03 crc kubenswrapper[5052]: I1203 12:07:03.061924 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 12:07:03 crc kubenswrapper[5052]: I1203 12:07:03.963349 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" podUID="c55dee8f-7615-47cc-a08f-65d79bd2f62d" containerName="nova-cell1-novncproxy-novncproxy" containerID="cri-o://fad19e1dd1b0bafc303e31cd36da9923ab67f02480e6f7d2d88248cf19a2ca04" gracePeriod=30 Dec 03 12:07:04 crc kubenswrapper[5052]: I1203 12:07:04.799830 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:07:04 crc kubenswrapper[5052]: I1203 12:07:04.896833 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/c55dee8f-7615-47cc-a08f-65d79bd2f62d-config-data\") pod \"c55dee8f-7615-47cc-a08f-65d79bd2f62d\" (UID: \"c55dee8f-7615-47cc-a08f-65d79bd2f62d\") " Dec 03 12:07:04 crc kubenswrapper[5052]: I1203 12:07:04.897044 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c55dee8f-7615-47cc-a08f-65d79bd2f62d-combined-ca-bundle\") pod \"c55dee8f-7615-47cc-a08f-65d79bd2f62d\" (UID: \"c55dee8f-7615-47cc-a08f-65d79bd2f62d\") " Dec 03 12:07:04 crc kubenswrapper[5052]: I1203 12:07:04.897099 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8zhvj\" (UniqueName: \"kubernetes.io/projected/c55dee8f-7615-47cc-a08f-65d79bd2f62d-kube-api-access-8zhvj\") pod \"c55dee8f-7615-47cc-a08f-65d79bd2f62d\" (UID: \"c55dee8f-7615-47cc-a08f-65d79bd2f62d\") " Dec 03 12:07:04 crc kubenswrapper[5052]: I1203 12:07:04.901366 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c55dee8f-7615-47cc-a08f-65d79bd2f62d-kube-api-access-8zhvj" (OuterVolumeSpecName: "kube-api-access-8zhvj") pod "c55dee8f-7615-47cc-a08f-65d79bd2f62d" (UID: "c55dee8f-7615-47cc-a08f-65d79bd2f62d"). InnerVolumeSpecName "kube-api-access-8zhvj". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:07:04 crc kubenswrapper[5052]: I1203 12:07:04.924690 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c55dee8f-7615-47cc-a08f-65d79bd2f62d-config-data" (OuterVolumeSpecName: "config-data") pod "c55dee8f-7615-47cc-a08f-65d79bd2f62d" (UID: "c55dee8f-7615-47cc-a08f-65d79bd2f62d"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:07:04 crc kubenswrapper[5052]: I1203 12:07:04.925466 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c55dee8f-7615-47cc-a08f-65d79bd2f62d-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "c55dee8f-7615-47cc-a08f-65d79bd2f62d" (UID: "c55dee8f-7615-47cc-a08f-65d79bd2f62d"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:07:04 crc kubenswrapper[5052]: I1203 12:07:04.975112 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"ec281373-bf8b-4f4d-8050-206d92074fc2","Type":"ContainerStarted","Data":"d7dc9b3662b3541642913d5bde8644885a28a2bee3adc98f6755b9d0edaec722"} Dec 03 12:07:04 crc kubenswrapper[5052]: I1203 12:07:04.975249 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:07:04 crc kubenswrapper[5052]: I1203 12:07:04.977760 5052 generic.go:334] "Generic (PLEG): container finished" podID="c55dee8f-7615-47cc-a08f-65d79bd2f62d" containerID="fad19e1dd1b0bafc303e31cd36da9923ab67f02480e6f7d2d88248cf19a2ca04" exitCode=0 Dec 03 12:07:04 crc kubenswrapper[5052]: I1203 12:07:04.977836 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" event={"ID":"c55dee8f-7615-47cc-a08f-65d79bd2f62d","Type":"ContainerDied","Data":"fad19e1dd1b0bafc303e31cd36da9923ab67f02480e6f7d2d88248cf19a2ca04"} Dec 03 12:07:04 crc kubenswrapper[5052]: I1203 12:07:04.977870 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" event={"ID":"c55dee8f-7615-47cc-a08f-65d79bd2f62d","Type":"ContainerDied","Data":"660ea93a146e0ec6be1285d711c93c62ce3670050fb25b1fe6dd2da617b4bfe4"} Dec 03 12:07:04 crc kubenswrapper[5052]: I1203 12:07:04.977889 5052 scope.go:117] "RemoveContainer" containerID="fad19e1dd1b0bafc303e31cd36da9923ab67f02480e6f7d2d88248cf19a2ca04" Dec 03 12:07:04 crc kubenswrapper[5052]: I1203 12:07:04.977937 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-metadata-0" podUID="3ef81bc8-170a-490d-8e81-4010c16452c1" containerName="nova-metadata-log" containerID="cri-o://84bea333f871cb73280c841a737727cb24be09ebf4b63a73ea6cd1f5776a0128" gracePeriod=30 Dec 03 12:07:04 crc kubenswrapper[5052]: I1203 12:07:04.978033 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-metadata-0" podUID="3ef81bc8-170a-490d-8e81-4010c16452c1" containerName="nova-metadata-metadata" containerID="cri-o://07aa5b46542b59873f354ef15148617b80b850ffea3776c0a68ec27d1b4c327e" gracePeriod=30 Dec 03 12:07:04 crc kubenswrapper[5052]: I1203 12:07:04.978239 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:07:05 crc kubenswrapper[5052]: I1203 12:07:04.999500 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/c55dee8f-7615-47cc-a08f-65d79bd2f62d-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:07:05 crc kubenswrapper[5052]: I1203 12:07:04.999555 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c55dee8f-7615-47cc-a08f-65d79bd2f62d-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:07:05 crc kubenswrapper[5052]: I1203 12:07:04.999569 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-8zhvj\" (UniqueName: \"kubernetes.io/projected/c55dee8f-7615-47cc-a08f-65d79bd2f62d-kube-api-access-8zhvj\") on node \"crc\" DevicePath \"\"" Dec 03 12:07:05 crc kubenswrapper[5052]: I1203 12:07:05.001145 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/ceilometer-0" podStartSLOduration=2.80094393 podStartE2EDuration="10.001124638s" podCreationTimestamp="2025-12-03 12:06:55 +0000 UTC" firstStartedPulling="2025-12-03 12:06:56.709817385 +0000 UTC m=+2807.298059159" lastFinishedPulling="2025-12-03 12:07:03.909998103 +0000 UTC m=+2814.498239867" observedRunningTime="2025-12-03 12:07:04.994487268 +0000 UTC m=+2815.582729032" watchObservedRunningTime="2025-12-03 12:07:05.001124638 +0000 UTC m=+2815.589366402" Dec 03 12:07:05 crc kubenswrapper[5052]: I1203 12:07:05.082956 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 12:07:05 crc kubenswrapper[5052]: I1203 12:07:05.098456 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 12:07:05 crc kubenswrapper[5052]: I1203 12:07:05.109378 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 12:07:05 crc kubenswrapper[5052]: E1203 12:07:05.109864 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="c55dee8f-7615-47cc-a08f-65d79bd2f62d" containerName="nova-cell1-novncproxy-novncproxy" Dec 03 12:07:05 crc kubenswrapper[5052]: I1203 12:07:05.109881 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="c55dee8f-7615-47cc-a08f-65d79bd2f62d" containerName="nova-cell1-novncproxy-novncproxy" Dec 03 12:07:05 crc kubenswrapper[5052]: I1203 12:07:05.110101 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="c55dee8f-7615-47cc-a08f-65d79bd2f62d" containerName="nova-cell1-novncproxy-novncproxy" Dec 03 12:07:05 crc kubenswrapper[5052]: I1203 12:07:05.110935 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:07:05 crc kubenswrapper[5052]: I1203 12:07:05.111939 5052 scope.go:117] "RemoveContainer" containerID="fad19e1dd1b0bafc303e31cd36da9923ab67f02480e6f7d2d88248cf19a2ca04" Dec 03 12:07:05 crc kubenswrapper[5052]: E1203 12:07:05.112391 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fad19e1dd1b0bafc303e31cd36da9923ab67f02480e6f7d2d88248cf19a2ca04\": container with ID starting with fad19e1dd1b0bafc303e31cd36da9923ab67f02480e6f7d2d88248cf19a2ca04 not found: ID does not exist" containerID="fad19e1dd1b0bafc303e31cd36da9923ab67f02480e6f7d2d88248cf19a2ca04" Dec 03 12:07:05 crc kubenswrapper[5052]: I1203 12:07:05.112421 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fad19e1dd1b0bafc303e31cd36da9923ab67f02480e6f7d2d88248cf19a2ca04"} err="failed to get container status \"fad19e1dd1b0bafc303e31cd36da9923ab67f02480e6f7d2d88248cf19a2ca04\": rpc error: code = NotFound desc = could not find container \"fad19e1dd1b0bafc303e31cd36da9923ab67f02480e6f7d2d88248cf19a2ca04\": container with ID starting with fad19e1dd1b0bafc303e31cd36da9923ab67f02480e6f7d2d88248cf19a2ca04 not found: ID does not exist" Dec 03 12:07:05 crc kubenswrapper[5052]: I1203 12:07:05.116384 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell1-novncproxy-config-data" Dec 03 12:07:05 crc kubenswrapper[5052]: I1203 12:07:05.116688 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-nova-novncproxy-cell1-public-svc" Dec 03 12:07:05 crc kubenswrapper[5052]: I1203 12:07:05.116402 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-nova-novncproxy-cell1-vencrypt" Dec 03 12:07:05 crc kubenswrapper[5052]: I1203 12:07:05.148099 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 12:07:05 crc kubenswrapper[5052]: I1203 12:07:05.202646 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:07:05 crc kubenswrapper[5052]: I1203 12:07:05.202705 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:07:05 crc kubenswrapper[5052]: I1203 12:07:05.203769 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/e66cca73-f322-4c6e-adb0-5ec43607bb73-config-data\") pod \"nova-cell1-novncproxy-0\" (UID: \"e66cca73-f322-4c6e-adb0-5ec43607bb73\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:07:05 crc kubenswrapper[5052]: I1203 12:07:05.203903 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"vencrypt-tls-certs\" (UniqueName: \"kubernetes.io/secret/e66cca73-f322-4c6e-adb0-5ec43607bb73-vencrypt-tls-certs\") pod \"nova-cell1-novncproxy-0\" (UID: \"e66cca73-f322-4c6e-adb0-5ec43607bb73\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:07:05 crc kubenswrapper[5052]: I1203 12:07:05.203937 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e66cca73-f322-4c6e-adb0-5ec43607bb73-combined-ca-bundle\") pod \"nova-cell1-novncproxy-0\" (UID: \"e66cca73-f322-4c6e-adb0-5ec43607bb73\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:07:05 crc kubenswrapper[5052]: I1203 12:07:05.204018 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nova-novncproxy-tls-certs\" (UniqueName: \"kubernetes.io/secret/e66cca73-f322-4c6e-adb0-5ec43607bb73-nova-novncproxy-tls-certs\") pod \"nova-cell1-novncproxy-0\" (UID: \"e66cca73-f322-4c6e-adb0-5ec43607bb73\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:07:05 crc kubenswrapper[5052]: I1203 12:07:05.204039 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5j7b2\" (UniqueName: \"kubernetes.io/projected/e66cca73-f322-4c6e-adb0-5ec43607bb73-kube-api-access-5j7b2\") pod \"nova-cell1-novncproxy-0\" (UID: \"e66cca73-f322-4c6e-adb0-5ec43607bb73\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:07:05 crc kubenswrapper[5052]: I1203 12:07:05.230106 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:07:05 crc kubenswrapper[5052]: I1203 12:07:05.305984 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/e66cca73-f322-4c6e-adb0-5ec43607bb73-config-data\") pod \"nova-cell1-novncproxy-0\" (UID: \"e66cca73-f322-4c6e-adb0-5ec43607bb73\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:07:05 crc kubenswrapper[5052]: I1203 12:07:05.306086 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"vencrypt-tls-certs\" (UniqueName: \"kubernetes.io/secret/e66cca73-f322-4c6e-adb0-5ec43607bb73-vencrypt-tls-certs\") pod \"nova-cell1-novncproxy-0\" (UID: \"e66cca73-f322-4c6e-adb0-5ec43607bb73\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:07:05 crc kubenswrapper[5052]: I1203 12:07:05.306117 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e66cca73-f322-4c6e-adb0-5ec43607bb73-combined-ca-bundle\") pod \"nova-cell1-novncproxy-0\" (UID: \"e66cca73-f322-4c6e-adb0-5ec43607bb73\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:07:05 crc kubenswrapper[5052]: I1203 12:07:05.306189 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"nova-novncproxy-tls-certs\" (UniqueName: \"kubernetes.io/secret/e66cca73-f322-4c6e-adb0-5ec43607bb73-nova-novncproxy-tls-certs\") pod \"nova-cell1-novncproxy-0\" (UID: \"e66cca73-f322-4c6e-adb0-5ec43607bb73\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:07:05 crc kubenswrapper[5052]: I1203 12:07:05.306209 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-5j7b2\" (UniqueName: \"kubernetes.io/projected/e66cca73-f322-4c6e-adb0-5ec43607bb73-kube-api-access-5j7b2\") pod \"nova-cell1-novncproxy-0\" (UID: \"e66cca73-f322-4c6e-adb0-5ec43607bb73\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:07:05 crc kubenswrapper[5052]: I1203 12:07:05.309832 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/e66cca73-f322-4c6e-adb0-5ec43607bb73-config-data\") pod \"nova-cell1-novncproxy-0\" (UID: \"e66cca73-f322-4c6e-adb0-5ec43607bb73\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:07:05 crc kubenswrapper[5052]: I1203 12:07:05.310290 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"nova-novncproxy-tls-certs\" (UniqueName: \"kubernetes.io/secret/e66cca73-f322-4c6e-adb0-5ec43607bb73-nova-novncproxy-tls-certs\") pod \"nova-cell1-novncproxy-0\" (UID: \"e66cca73-f322-4c6e-adb0-5ec43607bb73\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:07:05 crc kubenswrapper[5052]: I1203 12:07:05.311677 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"vencrypt-tls-certs\" (UniqueName: \"kubernetes.io/secret/e66cca73-f322-4c6e-adb0-5ec43607bb73-vencrypt-tls-certs\") pod \"nova-cell1-novncproxy-0\" (UID: \"e66cca73-f322-4c6e-adb0-5ec43607bb73\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:07:05 crc kubenswrapper[5052]: I1203 12:07:05.317413 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e66cca73-f322-4c6e-adb0-5ec43607bb73-combined-ca-bundle\") pod \"nova-cell1-novncproxy-0\" (UID: \"e66cca73-f322-4c6e-adb0-5ec43607bb73\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:07:05 crc kubenswrapper[5052]: I1203 12:07:05.325299 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-5j7b2\" (UniqueName: \"kubernetes.io/projected/e66cca73-f322-4c6e-adb0-5ec43607bb73-kube-api-access-5j7b2\") pod \"nova-cell1-novncproxy-0\" (UID: \"e66cca73-f322-4c6e-adb0-5ec43607bb73\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:07:05 crc kubenswrapper[5052]: I1203 12:07:05.479988 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:07:05 crc kubenswrapper[5052]: I1203 12:07:05.690662 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c55dee8f-7615-47cc-a08f-65d79bd2f62d" path="/var/lib/kubelet/pods/c55dee8f-7615-47cc-a08f-65d79bd2f62d/volumes" Dec 03 12:07:06 crc kubenswrapper[5052]: I1203 12:07:06.007719 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 12:07:06 crc kubenswrapper[5052]: I1203 12:07:06.026110 5052 generic.go:334] "Generic (PLEG): container finished" podID="3ef81bc8-170a-490d-8e81-4010c16452c1" containerID="07aa5b46542b59873f354ef15148617b80b850ffea3776c0a68ec27d1b4c327e" exitCode=0 Dec 03 12:07:06 crc kubenswrapper[5052]: I1203 12:07:06.026138 5052 generic.go:334] "Generic (PLEG): container finished" podID="3ef81bc8-170a-490d-8e81-4010c16452c1" containerID="84bea333f871cb73280c841a737727cb24be09ebf4b63a73ea6cd1f5776a0128" exitCode=143 Dec 03 12:07:06 crc kubenswrapper[5052]: I1203 12:07:06.026181 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"3ef81bc8-170a-490d-8e81-4010c16452c1","Type":"ContainerDied","Data":"07aa5b46542b59873f354ef15148617b80b850ffea3776c0a68ec27d1b4c327e"} Dec 03 12:07:06 crc kubenswrapper[5052]: I1203 12:07:06.026224 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"3ef81bc8-170a-490d-8e81-4010c16452c1","Type":"ContainerDied","Data":"84bea333f871cb73280c841a737727cb24be09ebf4b63a73ea6cd1f5776a0128"} Dec 03 12:07:06 crc kubenswrapper[5052]: I1203 12:07:06.211689 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:07:06 crc kubenswrapper[5052]: I1203 12:07:06.330695 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3ef81bc8-170a-490d-8e81-4010c16452c1-combined-ca-bundle\") pod \"3ef81bc8-170a-490d-8e81-4010c16452c1\" (UID: \"3ef81bc8-170a-490d-8e81-4010c16452c1\") " Dec 03 12:07:06 crc kubenswrapper[5052]: I1203 12:07:06.331060 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/3ef81bc8-170a-490d-8e81-4010c16452c1-logs\") pod \"3ef81bc8-170a-490d-8e81-4010c16452c1\" (UID: \"3ef81bc8-170a-490d-8e81-4010c16452c1\") " Dec 03 12:07:06 crc kubenswrapper[5052]: I1203 12:07:06.331212 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kc2v2\" (UniqueName: \"kubernetes.io/projected/3ef81bc8-170a-490d-8e81-4010c16452c1-kube-api-access-kc2v2\") pod \"3ef81bc8-170a-490d-8e81-4010c16452c1\" (UID: \"3ef81bc8-170a-490d-8e81-4010c16452c1\") " Dec 03 12:07:06 crc kubenswrapper[5052]: I1203 12:07:06.331242 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/3ef81bc8-170a-490d-8e81-4010c16452c1-config-data\") pod \"3ef81bc8-170a-490d-8e81-4010c16452c1\" (UID: \"3ef81bc8-170a-490d-8e81-4010c16452c1\") " Dec 03 12:07:06 crc kubenswrapper[5052]: I1203 12:07:06.331915 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3ef81bc8-170a-490d-8e81-4010c16452c1-logs" (OuterVolumeSpecName: "logs") pod "3ef81bc8-170a-490d-8e81-4010c16452c1" (UID: "3ef81bc8-170a-490d-8e81-4010c16452c1"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:07:06 crc kubenswrapper[5052]: I1203 12:07:06.339262 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3ef81bc8-170a-490d-8e81-4010c16452c1-kube-api-access-kc2v2" (OuterVolumeSpecName: "kube-api-access-kc2v2") pod "3ef81bc8-170a-490d-8e81-4010c16452c1" (UID: "3ef81bc8-170a-490d-8e81-4010c16452c1"). InnerVolumeSpecName "kube-api-access-kc2v2". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:07:06 crc kubenswrapper[5052]: I1203 12:07:06.361041 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3ef81bc8-170a-490d-8e81-4010c16452c1-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "3ef81bc8-170a-490d-8e81-4010c16452c1" (UID: "3ef81bc8-170a-490d-8e81-4010c16452c1"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:07:06 crc kubenswrapper[5052]: I1203 12:07:06.361259 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3ef81bc8-170a-490d-8e81-4010c16452c1-config-data" (OuterVolumeSpecName: "config-data") pod "3ef81bc8-170a-490d-8e81-4010c16452c1" (UID: "3ef81bc8-170a-490d-8e81-4010c16452c1"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:07:06 crc kubenswrapper[5052]: I1203 12:07:06.433895 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-kc2v2\" (UniqueName: \"kubernetes.io/projected/3ef81bc8-170a-490d-8e81-4010c16452c1-kube-api-access-kc2v2\") on node \"crc\" DevicePath \"\"" Dec 03 12:07:06 crc kubenswrapper[5052]: I1203 12:07:06.433925 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/3ef81bc8-170a-490d-8e81-4010c16452c1-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:07:06 crc kubenswrapper[5052]: I1203 12:07:06.433935 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3ef81bc8-170a-490d-8e81-4010c16452c1-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:07:06 crc kubenswrapper[5052]: I1203 12:07:06.433944 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/3ef81bc8-170a-490d-8e81-4010c16452c1-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:07:07 crc kubenswrapper[5052]: I1203 12:07:07.039149 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"3ef81bc8-170a-490d-8e81-4010c16452c1","Type":"ContainerDied","Data":"e78181cf88f9aa59bd0d70304297a560b9777ea787ea31d5527de0a3eeec6025"} Dec 03 12:07:07 crc kubenswrapper[5052]: I1203 12:07:07.039234 5052 scope.go:117] "RemoveContainer" containerID="07aa5b46542b59873f354ef15148617b80b850ffea3776c0a68ec27d1b4c327e" Dec 03 12:07:07 crc kubenswrapper[5052]: I1203 12:07:07.039275 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:07:07 crc kubenswrapper[5052]: I1203 12:07:07.043184 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" event={"ID":"e66cca73-f322-4c6e-adb0-5ec43607bb73","Type":"ContainerStarted","Data":"81211fcc809459c82abca1a15005c67c33acfa54010915f26c5a7c1bed4bf50d"} Dec 03 12:07:07 crc kubenswrapper[5052]: I1203 12:07:07.043252 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" event={"ID":"e66cca73-f322-4c6e-adb0-5ec43607bb73","Type":"ContainerStarted","Data":"1141e69944dcd7531261321f3d71467a1ef98b47ca821adbbbfbfb47d1ed7c9c"} Dec 03 12:07:07 crc kubenswrapper[5052]: I1203 12:07:07.069879 5052 scope.go:117] "RemoveContainer" containerID="84bea333f871cb73280c841a737727cb24be09ebf4b63a73ea6cd1f5776a0128" Dec 03 12:07:07 crc kubenswrapper[5052]: I1203 12:07:07.073318 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" podStartSLOduration=2.07330085 podStartE2EDuration="2.07330085s" podCreationTimestamp="2025-12-03 12:07:05 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:07:07.072829636 +0000 UTC m=+2817.661071410" watchObservedRunningTime="2025-12-03 12:07:07.07330085 +0000 UTC m=+2817.661542614" Dec 03 12:07:07 crc kubenswrapper[5052]: I1203 12:07:07.107939 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:07:07 crc kubenswrapper[5052]: I1203 12:07:07.134439 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:07:07 crc kubenswrapper[5052]: I1203 12:07:07.148019 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:07:07 crc kubenswrapper[5052]: E1203 12:07:07.148642 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="3ef81bc8-170a-490d-8e81-4010c16452c1" containerName="nova-metadata-log" Dec 03 12:07:07 crc kubenswrapper[5052]: I1203 12:07:07.148666 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="3ef81bc8-170a-490d-8e81-4010c16452c1" containerName="nova-metadata-log" Dec 03 12:07:07 crc kubenswrapper[5052]: E1203 12:07:07.148700 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="3ef81bc8-170a-490d-8e81-4010c16452c1" containerName="nova-metadata-metadata" Dec 03 12:07:07 crc kubenswrapper[5052]: I1203 12:07:07.148712 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="3ef81bc8-170a-490d-8e81-4010c16452c1" containerName="nova-metadata-metadata" Dec 03 12:07:07 crc kubenswrapper[5052]: I1203 12:07:07.148953 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="3ef81bc8-170a-490d-8e81-4010c16452c1" containerName="nova-metadata-log" Dec 03 12:07:07 crc kubenswrapper[5052]: I1203 12:07:07.148992 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="3ef81bc8-170a-490d-8e81-4010c16452c1" containerName="nova-metadata-metadata" Dec 03 12:07:07 crc kubenswrapper[5052]: I1203 12:07:07.150304 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:07:07 crc kubenswrapper[5052]: I1203 12:07:07.158263 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-metadata-config-data" Dec 03 12:07:07 crc kubenswrapper[5052]: I1203 12:07:07.158535 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-nova-metadata-internal-svc" Dec 03 12:07:07 crc kubenswrapper[5052]: I1203 12:07:07.177441 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:07:07 crc kubenswrapper[5052]: I1203 12:07:07.253029 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/af0152a4-cde0-4223-ac1e-44fcfc7e34be-nova-metadata-tls-certs\") pod \"nova-metadata-0\" (UID: \"af0152a4-cde0-4223-ac1e-44fcfc7e34be\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:07:07 crc kubenswrapper[5052]: I1203 12:07:07.253223 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/af0152a4-cde0-4223-ac1e-44fcfc7e34be-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"af0152a4-cde0-4223-ac1e-44fcfc7e34be\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:07:07 crc kubenswrapper[5052]: I1203 12:07:07.253284 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wk5n5\" (UniqueName: \"kubernetes.io/projected/af0152a4-cde0-4223-ac1e-44fcfc7e34be-kube-api-access-wk5n5\") pod \"nova-metadata-0\" (UID: \"af0152a4-cde0-4223-ac1e-44fcfc7e34be\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:07:07 crc kubenswrapper[5052]: I1203 12:07:07.253360 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/af0152a4-cde0-4223-ac1e-44fcfc7e34be-logs\") pod \"nova-metadata-0\" (UID: \"af0152a4-cde0-4223-ac1e-44fcfc7e34be\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:07:07 crc kubenswrapper[5052]: I1203 12:07:07.253446 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/af0152a4-cde0-4223-ac1e-44fcfc7e34be-config-data\") pod \"nova-metadata-0\" (UID: \"af0152a4-cde0-4223-ac1e-44fcfc7e34be\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:07:07 crc kubenswrapper[5052]: I1203 12:07:07.355630 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-wk5n5\" (UniqueName: \"kubernetes.io/projected/af0152a4-cde0-4223-ac1e-44fcfc7e34be-kube-api-access-wk5n5\") pod \"nova-metadata-0\" (UID: \"af0152a4-cde0-4223-ac1e-44fcfc7e34be\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:07:07 crc kubenswrapper[5052]: I1203 12:07:07.355721 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/af0152a4-cde0-4223-ac1e-44fcfc7e34be-logs\") pod \"nova-metadata-0\" (UID: \"af0152a4-cde0-4223-ac1e-44fcfc7e34be\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:07:07 crc kubenswrapper[5052]: I1203 12:07:07.355795 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/af0152a4-cde0-4223-ac1e-44fcfc7e34be-config-data\") pod \"nova-metadata-0\" (UID: \"af0152a4-cde0-4223-ac1e-44fcfc7e34be\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:07:07 crc kubenswrapper[5052]: I1203 12:07:07.355874 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/af0152a4-cde0-4223-ac1e-44fcfc7e34be-nova-metadata-tls-certs\") pod \"nova-metadata-0\" (UID: \"af0152a4-cde0-4223-ac1e-44fcfc7e34be\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:07:07 crc kubenswrapper[5052]: I1203 12:07:07.355900 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/af0152a4-cde0-4223-ac1e-44fcfc7e34be-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"af0152a4-cde0-4223-ac1e-44fcfc7e34be\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:07:07 crc kubenswrapper[5052]: I1203 12:07:07.356964 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/af0152a4-cde0-4223-ac1e-44fcfc7e34be-logs\") pod \"nova-metadata-0\" (UID: \"af0152a4-cde0-4223-ac1e-44fcfc7e34be\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:07:07 crc kubenswrapper[5052]: I1203 12:07:07.368095 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/af0152a4-cde0-4223-ac1e-44fcfc7e34be-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"af0152a4-cde0-4223-ac1e-44fcfc7e34be\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:07:07 crc kubenswrapper[5052]: I1203 12:07:07.368118 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/af0152a4-cde0-4223-ac1e-44fcfc7e34be-config-data\") pod \"nova-metadata-0\" (UID: \"af0152a4-cde0-4223-ac1e-44fcfc7e34be\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:07:07 crc kubenswrapper[5052]: I1203 12:07:07.374045 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/af0152a4-cde0-4223-ac1e-44fcfc7e34be-nova-metadata-tls-certs\") pod \"nova-metadata-0\" (UID: \"af0152a4-cde0-4223-ac1e-44fcfc7e34be\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:07:07 crc kubenswrapper[5052]: I1203 12:07:07.375587 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-wk5n5\" (UniqueName: \"kubernetes.io/projected/af0152a4-cde0-4223-ac1e-44fcfc7e34be-kube-api-access-wk5n5\") pod \"nova-metadata-0\" (UID: \"af0152a4-cde0-4223-ac1e-44fcfc7e34be\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:07:07 crc kubenswrapper[5052]: I1203 12:07:07.498914 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:07:07 crc kubenswrapper[5052]: I1203 12:07:07.687859 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3ef81bc8-170a-490d-8e81-4010c16452c1" path="/var/lib/kubelet/pods/3ef81bc8-170a-490d-8e81-4010c16452c1/volumes" Dec 03 12:07:07 crc kubenswrapper[5052]: I1203 12:07:07.961591 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:07:08 crc kubenswrapper[5052]: I1203 12:07:08.061930 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"af0152a4-cde0-4223-ac1e-44fcfc7e34be","Type":"ContainerStarted","Data":"6224927f647f9bcdc1cb68be0e7009b727a712eb0d517e8700bb34a6b6d527ae"} Dec 03 12:07:09 crc kubenswrapper[5052]: I1203 12:07:09.087137 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"af0152a4-cde0-4223-ac1e-44fcfc7e34be","Type":"ContainerStarted","Data":"ce4b3facccae770c61783823c56c1a4af9b5606bc3b8983a4a7f3e0176e99097"} Dec 03 12:07:10 crc kubenswrapper[5052]: I1203 12:07:10.100030 5052 generic.go:334] "Generic (PLEG): container finished" podID="8f59423a-caa1-428c-920c-59bb1a3b618d" containerID="15a5f7c7cd882608a85275b1c6458b1bf51b486cc6488418a2e35990d2b843c0" exitCode=0 Dec 03 12:07:10 crc kubenswrapper[5052]: I1203 12:07:10.100114 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-cell-mapping-5r7kw" event={"ID":"8f59423a-caa1-428c-920c-59bb1a3b618d","Type":"ContainerDied","Data":"15a5f7c7cd882608a85275b1c6458b1bf51b486cc6488418a2e35990d2b843c0"} Dec 03 12:07:10 crc kubenswrapper[5052]: I1203 12:07:10.103592 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"af0152a4-cde0-4223-ac1e-44fcfc7e34be","Type":"ContainerStarted","Data":"bf3de7ce59da5145f408038cf6bd81988825025f5640d03ce7c58e73805556c1"} Dec 03 12:07:10 crc kubenswrapper[5052]: I1203 12:07:10.157960 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-metadata-0" podStartSLOduration=3.157937117 podStartE2EDuration="3.157937117s" podCreationTimestamp="2025-12-03 12:07:07 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:07:10.150565287 +0000 UTC m=+2820.738807061" watchObservedRunningTime="2025-12-03 12:07:10.157937117 +0000 UTC m=+2820.746178901" Dec 03 12:07:10 crc kubenswrapper[5052]: I1203 12:07:10.230350 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:07:10 crc kubenswrapper[5052]: I1203 12:07:10.268656 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:07:10 crc kubenswrapper[5052]: I1203 12:07:10.481342 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:07:10 crc kubenswrapper[5052]: I1203 12:07:10.545451 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:07:10 crc kubenswrapper[5052]: I1203 12:07:10.545531 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:07:11 crc kubenswrapper[5052]: I1203 12:07:11.153365 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:07:11 crc kubenswrapper[5052]: I1203 12:07:11.548677 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-cell-mapping-5r7kw" Dec 03 12:07:11 crc kubenswrapper[5052]: I1203 12:07:11.628651 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-api-0" podUID="bdbb00d9-d8ce-4a03-843f-ab8441279186" containerName="nova-api-api" probeResult="failure" output="Get \"http://10.217.0.182:8774/\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Dec 03 12:07:11 crc kubenswrapper[5052]: I1203 12:07:11.629014 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-api-0" podUID="bdbb00d9-d8ce-4a03-843f-ab8441279186" containerName="nova-api-log" probeResult="failure" output="Get \"http://10.217.0.182:8774/\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Dec 03 12:07:11 crc kubenswrapper[5052]: I1203 12:07:11.662340 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2vqqt\" (UniqueName: \"kubernetes.io/projected/8f59423a-caa1-428c-920c-59bb1a3b618d-kube-api-access-2vqqt\") pod \"8f59423a-caa1-428c-920c-59bb1a3b618d\" (UID: \"8f59423a-caa1-428c-920c-59bb1a3b618d\") " Dec 03 12:07:11 crc kubenswrapper[5052]: I1203 12:07:11.663943 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/8f59423a-caa1-428c-920c-59bb1a3b618d-scripts\") pod \"8f59423a-caa1-428c-920c-59bb1a3b618d\" (UID: \"8f59423a-caa1-428c-920c-59bb1a3b618d\") " Dec 03 12:07:11 crc kubenswrapper[5052]: I1203 12:07:11.664158 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/8f59423a-caa1-428c-920c-59bb1a3b618d-config-data\") pod \"8f59423a-caa1-428c-920c-59bb1a3b618d\" (UID: \"8f59423a-caa1-428c-920c-59bb1a3b618d\") " Dec 03 12:07:11 crc kubenswrapper[5052]: I1203 12:07:11.664438 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8f59423a-caa1-428c-920c-59bb1a3b618d-combined-ca-bundle\") pod \"8f59423a-caa1-428c-920c-59bb1a3b618d\" (UID: \"8f59423a-caa1-428c-920c-59bb1a3b618d\") " Dec 03 12:07:11 crc kubenswrapper[5052]: I1203 12:07:11.673648 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8f59423a-caa1-428c-920c-59bb1a3b618d-scripts" (OuterVolumeSpecName: "scripts") pod "8f59423a-caa1-428c-920c-59bb1a3b618d" (UID: "8f59423a-caa1-428c-920c-59bb1a3b618d"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:07:11 crc kubenswrapper[5052]: I1203 12:07:11.673804 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8f59423a-caa1-428c-920c-59bb1a3b618d-kube-api-access-2vqqt" (OuterVolumeSpecName: "kube-api-access-2vqqt") pod "8f59423a-caa1-428c-920c-59bb1a3b618d" (UID: "8f59423a-caa1-428c-920c-59bb1a3b618d"). InnerVolumeSpecName "kube-api-access-2vqqt". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:07:11 crc kubenswrapper[5052]: I1203 12:07:11.727776 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8f59423a-caa1-428c-920c-59bb1a3b618d-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "8f59423a-caa1-428c-920c-59bb1a3b618d" (UID: "8f59423a-caa1-428c-920c-59bb1a3b618d"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:07:11 crc kubenswrapper[5052]: I1203 12:07:11.745488 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8f59423a-caa1-428c-920c-59bb1a3b618d-config-data" (OuterVolumeSpecName: "config-data") pod "8f59423a-caa1-428c-920c-59bb1a3b618d" (UID: "8f59423a-caa1-428c-920c-59bb1a3b618d"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:07:11 crc kubenswrapper[5052]: I1203 12:07:11.774150 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8f59423a-caa1-428c-920c-59bb1a3b618d-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:07:11 crc kubenswrapper[5052]: I1203 12:07:11.774178 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-2vqqt\" (UniqueName: \"kubernetes.io/projected/8f59423a-caa1-428c-920c-59bb1a3b618d-kube-api-access-2vqqt\") on node \"crc\" DevicePath \"\"" Dec 03 12:07:11 crc kubenswrapper[5052]: I1203 12:07:11.774188 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/8f59423a-caa1-428c-920c-59bb1a3b618d-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:07:11 crc kubenswrapper[5052]: I1203 12:07:11.774198 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/8f59423a-caa1-428c-920c-59bb1a3b618d-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:07:12 crc kubenswrapper[5052]: I1203 12:07:12.126923 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-cell-mapping-5r7kw" Dec 03 12:07:12 crc kubenswrapper[5052]: I1203 12:07:12.127588 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-cell-mapping-5r7kw" event={"ID":"8f59423a-caa1-428c-920c-59bb1a3b618d","Type":"ContainerDied","Data":"6bd2d42e7b4ee6fe040d7a6f6fc643074bb3e23f8766bd6310e6b1059219041b"} Dec 03 12:07:12 crc kubenswrapper[5052]: I1203 12:07:12.127640 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6bd2d42e7b4ee6fe040d7a6f6fc643074bb3e23f8766bd6310e6b1059219041b" Dec 03 12:07:12 crc kubenswrapper[5052]: I1203 12:07:12.314484 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:07:12 crc kubenswrapper[5052]: I1203 12:07:12.314749 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-api-0" podUID="bdbb00d9-d8ce-4a03-843f-ab8441279186" containerName="nova-api-log" containerID="cri-o://11c4c6496ed9aa094f5dc8ac4a34886ebc95b89aadefbfc8fa7de82792231539" gracePeriod=30 Dec 03 12:07:12 crc kubenswrapper[5052]: I1203 12:07:12.314844 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-api-0" podUID="bdbb00d9-d8ce-4a03-843f-ab8441279186" containerName="nova-api-api" containerID="cri-o://70048e89dae40428b8f05096fcb64249be714c08577a13e9cceda0356a73f6ae" gracePeriod=30 Dec 03 12:07:12 crc kubenswrapper[5052]: I1203 12:07:12.377009 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:07:12 crc kubenswrapper[5052]: I1203 12:07:12.377451 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-metadata-0" podUID="af0152a4-cde0-4223-ac1e-44fcfc7e34be" containerName="nova-metadata-log" containerID="cri-o://ce4b3facccae770c61783823c56c1a4af9b5606bc3b8983a4a7f3e0176e99097" gracePeriod=30 Dec 03 12:07:12 crc kubenswrapper[5052]: I1203 12:07:12.378070 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-metadata-0" podUID="af0152a4-cde0-4223-ac1e-44fcfc7e34be" containerName="nova-metadata-metadata" containerID="cri-o://bf3de7ce59da5145f408038cf6bd81988825025f5640d03ce7c58e73805556c1" gracePeriod=30 Dec 03 12:07:12 crc kubenswrapper[5052]: I1203 12:07:12.397121 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:07:12 crc kubenswrapper[5052]: I1203 12:07:12.499924 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:07:12 crc kubenswrapper[5052]: I1203 12:07:12.499987 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:07:13 crc kubenswrapper[5052]: I1203 12:07:13.138465 5052 generic.go:334] "Generic (PLEG): container finished" podID="bdbb00d9-d8ce-4a03-843f-ab8441279186" containerID="11c4c6496ed9aa094f5dc8ac4a34886ebc95b89aadefbfc8fa7de82792231539" exitCode=143 Dec 03 12:07:13 crc kubenswrapper[5052]: I1203 12:07:13.138584 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"bdbb00d9-d8ce-4a03-843f-ab8441279186","Type":"ContainerDied","Data":"11c4c6496ed9aa094f5dc8ac4a34886ebc95b89aadefbfc8fa7de82792231539"} Dec 03 12:07:13 crc kubenswrapper[5052]: I1203 12:07:13.143407 5052 generic.go:334] "Generic (PLEG): container finished" podID="af0152a4-cde0-4223-ac1e-44fcfc7e34be" containerID="bf3de7ce59da5145f408038cf6bd81988825025f5640d03ce7c58e73805556c1" exitCode=0 Dec 03 12:07:13 crc kubenswrapper[5052]: I1203 12:07:13.143431 5052 generic.go:334] "Generic (PLEG): container finished" podID="af0152a4-cde0-4223-ac1e-44fcfc7e34be" containerID="ce4b3facccae770c61783823c56c1a4af9b5606bc3b8983a4a7f3e0176e99097" exitCode=143 Dec 03 12:07:13 crc kubenswrapper[5052]: I1203 12:07:13.143529 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"af0152a4-cde0-4223-ac1e-44fcfc7e34be","Type":"ContainerDied","Data":"bf3de7ce59da5145f408038cf6bd81988825025f5640d03ce7c58e73805556c1"} Dec 03 12:07:13 crc kubenswrapper[5052]: I1203 12:07:13.143590 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-scheduler-0" podUID="a7579c63-c143-459e-9d94-eddb02f810cf" containerName="nova-scheduler-scheduler" containerID="cri-o://14f0cd9f4b15ca56a0c624489f9c66aba039f94373eebbb3da75d53cb4de839a" gracePeriod=30 Dec 03 12:07:13 crc kubenswrapper[5052]: I1203 12:07:13.143593 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"af0152a4-cde0-4223-ac1e-44fcfc7e34be","Type":"ContainerDied","Data":"ce4b3facccae770c61783823c56c1a4af9b5606bc3b8983a4a7f3e0176e99097"} Dec 03 12:07:13 crc kubenswrapper[5052]: I1203 12:07:13.633619 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:07:13 crc kubenswrapper[5052]: I1203 12:07:13.716199 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/af0152a4-cde0-4223-ac1e-44fcfc7e34be-logs\") pod \"af0152a4-cde0-4223-ac1e-44fcfc7e34be\" (UID: \"af0152a4-cde0-4223-ac1e-44fcfc7e34be\") " Dec 03 12:07:13 crc kubenswrapper[5052]: I1203 12:07:13.716411 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/af0152a4-cde0-4223-ac1e-44fcfc7e34be-nova-metadata-tls-certs\") pod \"af0152a4-cde0-4223-ac1e-44fcfc7e34be\" (UID: \"af0152a4-cde0-4223-ac1e-44fcfc7e34be\") " Dec 03 12:07:13 crc kubenswrapper[5052]: I1203 12:07:13.716479 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wk5n5\" (UniqueName: \"kubernetes.io/projected/af0152a4-cde0-4223-ac1e-44fcfc7e34be-kube-api-access-wk5n5\") pod \"af0152a4-cde0-4223-ac1e-44fcfc7e34be\" (UID: \"af0152a4-cde0-4223-ac1e-44fcfc7e34be\") " Dec 03 12:07:13 crc kubenswrapper[5052]: I1203 12:07:13.716541 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/af0152a4-cde0-4223-ac1e-44fcfc7e34be-logs" (OuterVolumeSpecName: "logs") pod "af0152a4-cde0-4223-ac1e-44fcfc7e34be" (UID: "af0152a4-cde0-4223-ac1e-44fcfc7e34be"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:07:13 crc kubenswrapper[5052]: I1203 12:07:13.716638 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/af0152a4-cde0-4223-ac1e-44fcfc7e34be-config-data\") pod \"af0152a4-cde0-4223-ac1e-44fcfc7e34be\" (UID: \"af0152a4-cde0-4223-ac1e-44fcfc7e34be\") " Dec 03 12:07:13 crc kubenswrapper[5052]: I1203 12:07:13.716736 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/af0152a4-cde0-4223-ac1e-44fcfc7e34be-combined-ca-bundle\") pod \"af0152a4-cde0-4223-ac1e-44fcfc7e34be\" (UID: \"af0152a4-cde0-4223-ac1e-44fcfc7e34be\") " Dec 03 12:07:13 crc kubenswrapper[5052]: I1203 12:07:13.717364 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/af0152a4-cde0-4223-ac1e-44fcfc7e34be-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:07:13 crc kubenswrapper[5052]: I1203 12:07:13.731133 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/af0152a4-cde0-4223-ac1e-44fcfc7e34be-kube-api-access-wk5n5" (OuterVolumeSpecName: "kube-api-access-wk5n5") pod "af0152a4-cde0-4223-ac1e-44fcfc7e34be" (UID: "af0152a4-cde0-4223-ac1e-44fcfc7e34be"). InnerVolumeSpecName "kube-api-access-wk5n5". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:07:13 crc kubenswrapper[5052]: I1203 12:07:13.749640 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/af0152a4-cde0-4223-ac1e-44fcfc7e34be-config-data" (OuterVolumeSpecName: "config-data") pod "af0152a4-cde0-4223-ac1e-44fcfc7e34be" (UID: "af0152a4-cde0-4223-ac1e-44fcfc7e34be"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:07:13 crc kubenswrapper[5052]: I1203 12:07:13.753731 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/af0152a4-cde0-4223-ac1e-44fcfc7e34be-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "af0152a4-cde0-4223-ac1e-44fcfc7e34be" (UID: "af0152a4-cde0-4223-ac1e-44fcfc7e34be"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:07:13 crc kubenswrapper[5052]: I1203 12:07:13.796763 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/af0152a4-cde0-4223-ac1e-44fcfc7e34be-nova-metadata-tls-certs" (OuterVolumeSpecName: "nova-metadata-tls-certs") pod "af0152a4-cde0-4223-ac1e-44fcfc7e34be" (UID: "af0152a4-cde0-4223-ac1e-44fcfc7e34be"). InnerVolumeSpecName "nova-metadata-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:07:13 crc kubenswrapper[5052]: I1203 12:07:13.819220 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/af0152a4-cde0-4223-ac1e-44fcfc7e34be-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:07:13 crc kubenswrapper[5052]: I1203 12:07:13.819248 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/af0152a4-cde0-4223-ac1e-44fcfc7e34be-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:07:13 crc kubenswrapper[5052]: I1203 12:07:13.819257 5052 reconciler_common.go:293] "Volume detached for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/af0152a4-cde0-4223-ac1e-44fcfc7e34be-nova-metadata-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:07:13 crc kubenswrapper[5052]: I1203 12:07:13.819266 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-wk5n5\" (UniqueName: \"kubernetes.io/projected/af0152a4-cde0-4223-ac1e-44fcfc7e34be-kube-api-access-wk5n5\") on node \"crc\" DevicePath \"\"" Dec 03 12:07:14 crc kubenswrapper[5052]: I1203 12:07:14.156592 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:07:14 crc kubenswrapper[5052]: I1203 12:07:14.156584 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"af0152a4-cde0-4223-ac1e-44fcfc7e34be","Type":"ContainerDied","Data":"6224927f647f9bcdc1cb68be0e7009b727a712eb0d517e8700bb34a6b6d527ae"} Dec 03 12:07:14 crc kubenswrapper[5052]: I1203 12:07:14.157662 5052 scope.go:117] "RemoveContainer" containerID="bf3de7ce59da5145f408038cf6bd81988825025f5640d03ce7c58e73805556c1" Dec 03 12:07:14 crc kubenswrapper[5052]: I1203 12:07:14.185815 5052 scope.go:117] "RemoveContainer" containerID="ce4b3facccae770c61783823c56c1a4af9b5606bc3b8983a4a7f3e0176e99097" Dec 03 12:07:14 crc kubenswrapper[5052]: I1203 12:07:14.193686 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:07:14 crc kubenswrapper[5052]: I1203 12:07:14.203765 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:07:14 crc kubenswrapper[5052]: I1203 12:07:14.214582 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:07:14 crc kubenswrapper[5052]: E1203 12:07:14.214972 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="af0152a4-cde0-4223-ac1e-44fcfc7e34be" containerName="nova-metadata-log" Dec 03 12:07:14 crc kubenswrapper[5052]: I1203 12:07:14.214990 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="af0152a4-cde0-4223-ac1e-44fcfc7e34be" containerName="nova-metadata-log" Dec 03 12:07:14 crc kubenswrapper[5052]: E1203 12:07:14.215020 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="af0152a4-cde0-4223-ac1e-44fcfc7e34be" containerName="nova-metadata-metadata" Dec 03 12:07:14 crc kubenswrapper[5052]: I1203 12:07:14.215027 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="af0152a4-cde0-4223-ac1e-44fcfc7e34be" containerName="nova-metadata-metadata" Dec 03 12:07:14 crc kubenswrapper[5052]: E1203 12:07:14.215052 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="8f59423a-caa1-428c-920c-59bb1a3b618d" containerName="nova-manage" Dec 03 12:07:14 crc kubenswrapper[5052]: I1203 12:07:14.215058 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="8f59423a-caa1-428c-920c-59bb1a3b618d" containerName="nova-manage" Dec 03 12:07:14 crc kubenswrapper[5052]: I1203 12:07:14.215226 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="af0152a4-cde0-4223-ac1e-44fcfc7e34be" containerName="nova-metadata-log" Dec 03 12:07:14 crc kubenswrapper[5052]: I1203 12:07:14.215243 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="af0152a4-cde0-4223-ac1e-44fcfc7e34be" containerName="nova-metadata-metadata" Dec 03 12:07:14 crc kubenswrapper[5052]: I1203 12:07:14.215257 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="8f59423a-caa1-428c-920c-59bb1a3b618d" containerName="nova-manage" Dec 03 12:07:14 crc kubenswrapper[5052]: I1203 12:07:14.216306 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:07:14 crc kubenswrapper[5052]: I1203 12:07:14.220738 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-metadata-config-data" Dec 03 12:07:14 crc kubenswrapper[5052]: I1203 12:07:14.220837 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-nova-metadata-internal-svc" Dec 03 12:07:14 crc kubenswrapper[5052]: I1203 12:07:14.231902 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:07:14 crc kubenswrapper[5052]: I1203 12:07:14.328880 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/26024570-e836-484e-be44-8bf3a86277df-config-data\") pod \"nova-metadata-0\" (UID: \"26024570-e836-484e-be44-8bf3a86277df\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:07:14 crc kubenswrapper[5052]: I1203 12:07:14.328949 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wfrzz\" (UniqueName: \"kubernetes.io/projected/26024570-e836-484e-be44-8bf3a86277df-kube-api-access-wfrzz\") pod \"nova-metadata-0\" (UID: \"26024570-e836-484e-be44-8bf3a86277df\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:07:14 crc kubenswrapper[5052]: I1203 12:07:14.329000 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/26024570-e836-484e-be44-8bf3a86277df-nova-metadata-tls-certs\") pod \"nova-metadata-0\" (UID: \"26024570-e836-484e-be44-8bf3a86277df\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:07:14 crc kubenswrapper[5052]: I1203 12:07:14.329029 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/26024570-e836-484e-be44-8bf3a86277df-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"26024570-e836-484e-be44-8bf3a86277df\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:07:14 crc kubenswrapper[5052]: I1203 12:07:14.329300 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/26024570-e836-484e-be44-8bf3a86277df-logs\") pod \"nova-metadata-0\" (UID: \"26024570-e836-484e-be44-8bf3a86277df\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:07:14 crc kubenswrapper[5052]: I1203 12:07:14.430800 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-wfrzz\" (UniqueName: \"kubernetes.io/projected/26024570-e836-484e-be44-8bf3a86277df-kube-api-access-wfrzz\") pod \"nova-metadata-0\" (UID: \"26024570-e836-484e-be44-8bf3a86277df\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:07:14 crc kubenswrapper[5052]: I1203 12:07:14.430885 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/26024570-e836-484e-be44-8bf3a86277df-nova-metadata-tls-certs\") pod \"nova-metadata-0\" (UID: \"26024570-e836-484e-be44-8bf3a86277df\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:07:14 crc kubenswrapper[5052]: I1203 12:07:14.430925 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/26024570-e836-484e-be44-8bf3a86277df-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"26024570-e836-484e-be44-8bf3a86277df\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:07:14 crc kubenswrapper[5052]: I1203 12:07:14.431025 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/26024570-e836-484e-be44-8bf3a86277df-logs\") pod \"nova-metadata-0\" (UID: \"26024570-e836-484e-be44-8bf3a86277df\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:07:14 crc kubenswrapper[5052]: I1203 12:07:14.431080 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/26024570-e836-484e-be44-8bf3a86277df-config-data\") pod \"nova-metadata-0\" (UID: \"26024570-e836-484e-be44-8bf3a86277df\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:07:14 crc kubenswrapper[5052]: I1203 12:07:14.431762 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/26024570-e836-484e-be44-8bf3a86277df-logs\") pod \"nova-metadata-0\" (UID: \"26024570-e836-484e-be44-8bf3a86277df\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:07:14 crc kubenswrapper[5052]: I1203 12:07:14.436130 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/26024570-e836-484e-be44-8bf3a86277df-config-data\") pod \"nova-metadata-0\" (UID: \"26024570-e836-484e-be44-8bf3a86277df\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:07:14 crc kubenswrapper[5052]: I1203 12:07:14.436362 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/26024570-e836-484e-be44-8bf3a86277df-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"26024570-e836-484e-be44-8bf3a86277df\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:07:14 crc kubenswrapper[5052]: I1203 12:07:14.436374 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/26024570-e836-484e-be44-8bf3a86277df-nova-metadata-tls-certs\") pod \"nova-metadata-0\" (UID: \"26024570-e836-484e-be44-8bf3a86277df\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:07:14 crc kubenswrapper[5052]: I1203 12:07:14.457156 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-wfrzz\" (UniqueName: \"kubernetes.io/projected/26024570-e836-484e-be44-8bf3a86277df-kube-api-access-wfrzz\") pod \"nova-metadata-0\" (UID: \"26024570-e836-484e-be44-8bf3a86277df\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:07:14 crc kubenswrapper[5052]: I1203 12:07:14.576131 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:07:15 crc kubenswrapper[5052]: I1203 12:07:15.089010 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:07:15 crc kubenswrapper[5052]: I1203 12:07:15.181552 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"26024570-e836-484e-be44-8bf3a86277df","Type":"ContainerStarted","Data":"b02b9225723b0c66311defd6447d42e596a8cc2d56455e976cb1ab1c8da5d607"} Dec 03 12:07:15 crc kubenswrapper[5052]: E1203 12:07:15.232381 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="14f0cd9f4b15ca56a0c624489f9c66aba039f94373eebbb3da75d53cb4de839a" cmd=["/usr/bin/pgrep","-r","DRST","nova-scheduler"] Dec 03 12:07:15 crc kubenswrapper[5052]: E1203 12:07:15.236723 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="14f0cd9f4b15ca56a0c624489f9c66aba039f94373eebbb3da75d53cb4de839a" cmd=["/usr/bin/pgrep","-r","DRST","nova-scheduler"] Dec 03 12:07:15 crc kubenswrapper[5052]: E1203 12:07:15.238302 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="14f0cd9f4b15ca56a0c624489f9c66aba039f94373eebbb3da75d53cb4de839a" cmd=["/usr/bin/pgrep","-r","DRST","nova-scheduler"] Dec 03 12:07:15 crc kubenswrapper[5052]: E1203 12:07:15.238381 5052 prober.go:104] "Probe errored" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" probeType="Readiness" pod="openstack-kuttl-tests/nova-scheduler-0" podUID="a7579c63-c143-459e-9d94-eddb02f810cf" containerName="nova-scheduler-scheduler" Dec 03 12:07:15 crc kubenswrapper[5052]: I1203 12:07:15.481632 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:07:15 crc kubenswrapper[5052]: I1203 12:07:15.500690 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:07:15 crc kubenswrapper[5052]: I1203 12:07:15.687382 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="af0152a4-cde0-4223-ac1e-44fcfc7e34be" path="/var/lib/kubelet/pods/af0152a4-cde0-4223-ac1e-44fcfc7e34be/volumes" Dec 03 12:07:16 crc kubenswrapper[5052]: I1203 12:07:16.198565 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"26024570-e836-484e-be44-8bf3a86277df","Type":"ContainerStarted","Data":"99398cd54307f11b2e45075f7d91e929454004eb4ea850ce677de64e98526999"} Dec 03 12:07:16 crc kubenswrapper[5052]: I1203 12:07:16.216639 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:07:17 crc kubenswrapper[5052]: I1203 12:07:17.159573 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:07:17 crc kubenswrapper[5052]: I1203 12:07:17.216012 5052 generic.go:334] "Generic (PLEG): container finished" podID="a7579c63-c143-459e-9d94-eddb02f810cf" containerID="14f0cd9f4b15ca56a0c624489f9c66aba039f94373eebbb3da75d53cb4de839a" exitCode=0 Dec 03 12:07:17 crc kubenswrapper[5052]: I1203 12:07:17.216091 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"a7579c63-c143-459e-9d94-eddb02f810cf","Type":"ContainerDied","Data":"14f0cd9f4b15ca56a0c624489f9c66aba039f94373eebbb3da75d53cb4de839a"} Dec 03 12:07:17 crc kubenswrapper[5052]: I1203 12:07:17.216096 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:07:17 crc kubenswrapper[5052]: I1203 12:07:17.216127 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"a7579c63-c143-459e-9d94-eddb02f810cf","Type":"ContainerDied","Data":"36ff3bba7f293b1b02a6cc5f70a44fea80f9fee7a7af0ec53a39960e555d458b"} Dec 03 12:07:17 crc kubenswrapper[5052]: I1203 12:07:17.216148 5052 scope.go:117] "RemoveContainer" containerID="14f0cd9f4b15ca56a0c624489f9c66aba039f94373eebbb3da75d53cb4de839a" Dec 03 12:07:17 crc kubenswrapper[5052]: I1203 12:07:17.219014 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"26024570-e836-484e-be44-8bf3a86277df","Type":"ContainerStarted","Data":"410a9f8f5acdd3ad89a0440f12ff1b02a0dcf588ebdc64fb9d6d0fd48f3265c5"} Dec 03 12:07:17 crc kubenswrapper[5052]: I1203 12:07:17.243237 5052 scope.go:117] "RemoveContainer" containerID="14f0cd9f4b15ca56a0c624489f9c66aba039f94373eebbb3da75d53cb4de839a" Dec 03 12:07:17 crc kubenswrapper[5052]: E1203 12:07:17.243780 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"14f0cd9f4b15ca56a0c624489f9c66aba039f94373eebbb3da75d53cb4de839a\": container with ID starting with 14f0cd9f4b15ca56a0c624489f9c66aba039f94373eebbb3da75d53cb4de839a not found: ID does not exist" containerID="14f0cd9f4b15ca56a0c624489f9c66aba039f94373eebbb3da75d53cb4de839a" Dec 03 12:07:17 crc kubenswrapper[5052]: I1203 12:07:17.243820 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"14f0cd9f4b15ca56a0c624489f9c66aba039f94373eebbb3da75d53cb4de839a"} err="failed to get container status \"14f0cd9f4b15ca56a0c624489f9c66aba039f94373eebbb3da75d53cb4de839a\": rpc error: code = NotFound desc = could not find container \"14f0cd9f4b15ca56a0c624489f9c66aba039f94373eebbb3da75d53cb4de839a\": container with ID starting with 14f0cd9f4b15ca56a0c624489f9c66aba039f94373eebbb3da75d53cb4de839a not found: ID does not exist" Dec 03 12:07:17 crc kubenswrapper[5052]: I1203 12:07:17.245105 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-metadata-0" podStartSLOduration=3.245077824 podStartE2EDuration="3.245077824s" podCreationTimestamp="2025-12-03 12:07:14 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:07:17.238402023 +0000 UTC m=+2827.826643787" watchObservedRunningTime="2025-12-03 12:07:17.245077824 +0000 UTC m=+2827.833319588" Dec 03 12:07:17 crc kubenswrapper[5052]: I1203 12:07:17.306205 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a7579c63-c143-459e-9d94-eddb02f810cf-config-data\") pod \"a7579c63-c143-459e-9d94-eddb02f810cf\" (UID: \"a7579c63-c143-459e-9d94-eddb02f810cf\") " Dec 03 12:07:17 crc kubenswrapper[5052]: I1203 12:07:17.307130 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9d4cx\" (UniqueName: \"kubernetes.io/projected/a7579c63-c143-459e-9d94-eddb02f810cf-kube-api-access-9d4cx\") pod \"a7579c63-c143-459e-9d94-eddb02f810cf\" (UID: \"a7579c63-c143-459e-9d94-eddb02f810cf\") " Dec 03 12:07:17 crc kubenswrapper[5052]: I1203 12:07:17.307341 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a7579c63-c143-459e-9d94-eddb02f810cf-combined-ca-bundle\") pod \"a7579c63-c143-459e-9d94-eddb02f810cf\" (UID: \"a7579c63-c143-459e-9d94-eddb02f810cf\") " Dec 03 12:07:17 crc kubenswrapper[5052]: I1203 12:07:17.321748 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a7579c63-c143-459e-9d94-eddb02f810cf-kube-api-access-9d4cx" (OuterVolumeSpecName: "kube-api-access-9d4cx") pod "a7579c63-c143-459e-9d94-eddb02f810cf" (UID: "a7579c63-c143-459e-9d94-eddb02f810cf"). InnerVolumeSpecName "kube-api-access-9d4cx". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:07:17 crc kubenswrapper[5052]: I1203 12:07:17.335828 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a7579c63-c143-459e-9d94-eddb02f810cf-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "a7579c63-c143-459e-9d94-eddb02f810cf" (UID: "a7579c63-c143-459e-9d94-eddb02f810cf"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:07:17 crc kubenswrapper[5052]: I1203 12:07:17.343880 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a7579c63-c143-459e-9d94-eddb02f810cf-config-data" (OuterVolumeSpecName: "config-data") pod "a7579c63-c143-459e-9d94-eddb02f810cf" (UID: "a7579c63-c143-459e-9d94-eddb02f810cf"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:07:17 crc kubenswrapper[5052]: I1203 12:07:17.409904 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a7579c63-c143-459e-9d94-eddb02f810cf-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:07:17 crc kubenswrapper[5052]: I1203 12:07:17.409951 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a7579c63-c143-459e-9d94-eddb02f810cf-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:07:17 crc kubenswrapper[5052]: I1203 12:07:17.409964 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-9d4cx\" (UniqueName: \"kubernetes.io/projected/a7579c63-c143-459e-9d94-eddb02f810cf-kube-api-access-9d4cx\") on node \"crc\" DevicePath \"\"" Dec 03 12:07:17 crc kubenswrapper[5052]: I1203 12:07:17.548807 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:07:17 crc kubenswrapper[5052]: I1203 12:07:17.558336 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:07:17 crc kubenswrapper[5052]: I1203 12:07:17.572434 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:07:17 crc kubenswrapper[5052]: E1203 12:07:17.572990 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="a7579c63-c143-459e-9d94-eddb02f810cf" containerName="nova-scheduler-scheduler" Dec 03 12:07:17 crc kubenswrapper[5052]: I1203 12:07:17.573018 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="a7579c63-c143-459e-9d94-eddb02f810cf" containerName="nova-scheduler-scheduler" Dec 03 12:07:17 crc kubenswrapper[5052]: I1203 12:07:17.573243 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="a7579c63-c143-459e-9d94-eddb02f810cf" containerName="nova-scheduler-scheduler" Dec 03 12:07:17 crc kubenswrapper[5052]: I1203 12:07:17.575238 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:07:17 crc kubenswrapper[5052]: I1203 12:07:17.577712 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-scheduler-config-data" Dec 03 12:07:17 crc kubenswrapper[5052]: I1203 12:07:17.584808 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:07:17 crc kubenswrapper[5052]: I1203 12:07:17.686111 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a7579c63-c143-459e-9d94-eddb02f810cf" path="/var/lib/kubelet/pods/a7579c63-c143-459e-9d94-eddb02f810cf/volumes" Dec 03 12:07:17 crc kubenswrapper[5052]: I1203 12:07:17.714873 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vgt8h\" (UniqueName: \"kubernetes.io/projected/d99defb0-caae-4f5f-9a24-d3026d396c31-kube-api-access-vgt8h\") pod \"nova-scheduler-0\" (UID: \"d99defb0-caae-4f5f-9a24-d3026d396c31\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:07:17 crc kubenswrapper[5052]: I1203 12:07:17.714914 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/d99defb0-caae-4f5f-9a24-d3026d396c31-config-data\") pod \"nova-scheduler-0\" (UID: \"d99defb0-caae-4f5f-9a24-d3026d396c31\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:07:17 crc kubenswrapper[5052]: I1203 12:07:17.715022 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d99defb0-caae-4f5f-9a24-d3026d396c31-combined-ca-bundle\") pod \"nova-scheduler-0\" (UID: \"d99defb0-caae-4f5f-9a24-d3026d396c31\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:07:17 crc kubenswrapper[5052]: I1203 12:07:17.817303 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d99defb0-caae-4f5f-9a24-d3026d396c31-combined-ca-bundle\") pod \"nova-scheduler-0\" (UID: \"d99defb0-caae-4f5f-9a24-d3026d396c31\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:07:17 crc kubenswrapper[5052]: I1203 12:07:17.817460 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-vgt8h\" (UniqueName: \"kubernetes.io/projected/d99defb0-caae-4f5f-9a24-d3026d396c31-kube-api-access-vgt8h\") pod \"nova-scheduler-0\" (UID: \"d99defb0-caae-4f5f-9a24-d3026d396c31\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:07:17 crc kubenswrapper[5052]: I1203 12:07:17.817481 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/d99defb0-caae-4f5f-9a24-d3026d396c31-config-data\") pod \"nova-scheduler-0\" (UID: \"d99defb0-caae-4f5f-9a24-d3026d396c31\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:07:17 crc kubenswrapper[5052]: I1203 12:07:17.822275 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/d99defb0-caae-4f5f-9a24-d3026d396c31-config-data\") pod \"nova-scheduler-0\" (UID: \"d99defb0-caae-4f5f-9a24-d3026d396c31\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:07:17 crc kubenswrapper[5052]: I1203 12:07:17.826094 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d99defb0-caae-4f5f-9a24-d3026d396c31-combined-ca-bundle\") pod \"nova-scheduler-0\" (UID: \"d99defb0-caae-4f5f-9a24-d3026d396c31\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:07:17 crc kubenswrapper[5052]: I1203 12:07:17.834342 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-vgt8h\" (UniqueName: \"kubernetes.io/projected/d99defb0-caae-4f5f-9a24-d3026d396c31-kube-api-access-vgt8h\") pod \"nova-scheduler-0\" (UID: \"d99defb0-caae-4f5f-9a24-d3026d396c31\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:07:17 crc kubenswrapper[5052]: I1203 12:07:17.891471 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:07:18 crc kubenswrapper[5052]: I1203 12:07:18.242763 5052 generic.go:334] "Generic (PLEG): container finished" podID="bdbb00d9-d8ce-4a03-843f-ab8441279186" containerID="70048e89dae40428b8f05096fcb64249be714c08577a13e9cceda0356a73f6ae" exitCode=0 Dec 03 12:07:18 crc kubenswrapper[5052]: I1203 12:07:18.243748 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"bdbb00d9-d8ce-4a03-843f-ab8441279186","Type":"ContainerDied","Data":"70048e89dae40428b8f05096fcb64249be714c08577a13e9cceda0356a73f6ae"} Dec 03 12:07:18 crc kubenswrapper[5052]: I1203 12:07:18.344033 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:07:18 crc kubenswrapper[5052]: W1203 12:07:18.344743 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd99defb0_caae_4f5f_9a24_d3026d396c31.slice/crio-94573ff07096de1bcfd7a75e01e072eed25046372c1cacd2c16d5282d1c38efc WatchSource:0}: Error finding container 94573ff07096de1bcfd7a75e01e072eed25046372c1cacd2c16d5282d1c38efc: Status 404 returned error can't find the container with id 94573ff07096de1bcfd7a75e01e072eed25046372c1cacd2c16d5282d1c38efc Dec 03 12:07:18 crc kubenswrapper[5052]: I1203 12:07:18.533941 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:07:18 crc kubenswrapper[5052]: I1203 12:07:18.633333 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/bdbb00d9-d8ce-4a03-843f-ab8441279186-config-data\") pod \"bdbb00d9-d8ce-4a03-843f-ab8441279186\" (UID: \"bdbb00d9-d8ce-4a03-843f-ab8441279186\") " Dec 03 12:07:18 crc kubenswrapper[5052]: I1203 12:07:18.633460 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/bdbb00d9-d8ce-4a03-843f-ab8441279186-combined-ca-bundle\") pod \"bdbb00d9-d8ce-4a03-843f-ab8441279186\" (UID: \"bdbb00d9-d8ce-4a03-843f-ab8441279186\") " Dec 03 12:07:18 crc kubenswrapper[5052]: I1203 12:07:18.633633 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/bdbb00d9-d8ce-4a03-843f-ab8441279186-logs\") pod \"bdbb00d9-d8ce-4a03-843f-ab8441279186\" (UID: \"bdbb00d9-d8ce-4a03-843f-ab8441279186\") " Dec 03 12:07:18 crc kubenswrapper[5052]: I1203 12:07:18.633865 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x5b47\" (UniqueName: \"kubernetes.io/projected/bdbb00d9-d8ce-4a03-843f-ab8441279186-kube-api-access-x5b47\") pod \"bdbb00d9-d8ce-4a03-843f-ab8441279186\" (UID: \"bdbb00d9-d8ce-4a03-843f-ab8441279186\") " Dec 03 12:07:18 crc kubenswrapper[5052]: I1203 12:07:18.634268 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bdbb00d9-d8ce-4a03-843f-ab8441279186-logs" (OuterVolumeSpecName: "logs") pod "bdbb00d9-d8ce-4a03-843f-ab8441279186" (UID: "bdbb00d9-d8ce-4a03-843f-ab8441279186"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:07:18 crc kubenswrapper[5052]: I1203 12:07:18.634424 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/bdbb00d9-d8ce-4a03-843f-ab8441279186-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:07:18 crc kubenswrapper[5052]: I1203 12:07:18.646338 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bdbb00d9-d8ce-4a03-843f-ab8441279186-kube-api-access-x5b47" (OuterVolumeSpecName: "kube-api-access-x5b47") pod "bdbb00d9-d8ce-4a03-843f-ab8441279186" (UID: "bdbb00d9-d8ce-4a03-843f-ab8441279186"). InnerVolumeSpecName "kube-api-access-x5b47". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:07:18 crc kubenswrapper[5052]: I1203 12:07:18.664147 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bdbb00d9-d8ce-4a03-843f-ab8441279186-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "bdbb00d9-d8ce-4a03-843f-ab8441279186" (UID: "bdbb00d9-d8ce-4a03-843f-ab8441279186"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:07:18 crc kubenswrapper[5052]: I1203 12:07:18.665954 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bdbb00d9-d8ce-4a03-843f-ab8441279186-config-data" (OuterVolumeSpecName: "config-data") pod "bdbb00d9-d8ce-4a03-843f-ab8441279186" (UID: "bdbb00d9-d8ce-4a03-843f-ab8441279186"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:07:18 crc kubenswrapper[5052]: I1203 12:07:18.736628 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-x5b47\" (UniqueName: \"kubernetes.io/projected/bdbb00d9-d8ce-4a03-843f-ab8441279186-kube-api-access-x5b47\") on node \"crc\" DevicePath \"\"" Dec 03 12:07:18 crc kubenswrapper[5052]: I1203 12:07:18.736663 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/bdbb00d9-d8ce-4a03-843f-ab8441279186-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:07:18 crc kubenswrapper[5052]: I1203 12:07:18.736676 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/bdbb00d9-d8ce-4a03-843f-ab8441279186-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:07:19 crc kubenswrapper[5052]: I1203 12:07:19.254021 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"d99defb0-caae-4f5f-9a24-d3026d396c31","Type":"ContainerStarted","Data":"94573ff07096de1bcfd7a75e01e072eed25046372c1cacd2c16d5282d1c38efc"} Dec 03 12:07:19 crc kubenswrapper[5052]: I1203 12:07:19.255869 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"bdbb00d9-d8ce-4a03-843f-ab8441279186","Type":"ContainerDied","Data":"b203ee66bc093decab58c0f9b826923dd1b1540423969dbbc76c76b27c3e6cb1"} Dec 03 12:07:19 crc kubenswrapper[5052]: I1203 12:07:19.255921 5052 scope.go:117] "RemoveContainer" containerID="70048e89dae40428b8f05096fcb64249be714c08577a13e9cceda0356a73f6ae" Dec 03 12:07:19 crc kubenswrapper[5052]: I1203 12:07:19.255934 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:07:19 crc kubenswrapper[5052]: I1203 12:07:19.311013 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:07:19 crc kubenswrapper[5052]: I1203 12:07:19.320849 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:07:19 crc kubenswrapper[5052]: I1203 12:07:19.338378 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:07:19 crc kubenswrapper[5052]: E1203 12:07:19.338947 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="bdbb00d9-d8ce-4a03-843f-ab8441279186" containerName="nova-api-log" Dec 03 12:07:19 crc kubenswrapper[5052]: I1203 12:07:19.338971 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="bdbb00d9-d8ce-4a03-843f-ab8441279186" containerName="nova-api-log" Dec 03 12:07:19 crc kubenswrapper[5052]: E1203 12:07:19.338991 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="bdbb00d9-d8ce-4a03-843f-ab8441279186" containerName="nova-api-api" Dec 03 12:07:19 crc kubenswrapper[5052]: I1203 12:07:19.338996 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="bdbb00d9-d8ce-4a03-843f-ab8441279186" containerName="nova-api-api" Dec 03 12:07:19 crc kubenswrapper[5052]: I1203 12:07:19.339211 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="bdbb00d9-d8ce-4a03-843f-ab8441279186" containerName="nova-api-log" Dec 03 12:07:19 crc kubenswrapper[5052]: I1203 12:07:19.339252 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="bdbb00d9-d8ce-4a03-843f-ab8441279186" containerName="nova-api-api" Dec 03 12:07:19 crc kubenswrapper[5052]: I1203 12:07:19.340502 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:07:19 crc kubenswrapper[5052]: I1203 12:07:19.342779 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-api-config-data" Dec 03 12:07:19 crc kubenswrapper[5052]: I1203 12:07:19.350213 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:07:19 crc kubenswrapper[5052]: I1203 12:07:19.451213 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/ff9eaa72-6d5d-48a9-85a5-362de3895ae1-logs\") pod \"nova-api-0\" (UID: \"ff9eaa72-6d5d-48a9-85a5-362de3895ae1\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:07:19 crc kubenswrapper[5052]: I1203 12:07:19.451537 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/ff9eaa72-6d5d-48a9-85a5-362de3895ae1-config-data\") pod \"nova-api-0\" (UID: \"ff9eaa72-6d5d-48a9-85a5-362de3895ae1\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:07:19 crc kubenswrapper[5052]: I1203 12:07:19.451699 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ff9eaa72-6d5d-48a9-85a5-362de3895ae1-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"ff9eaa72-6d5d-48a9-85a5-362de3895ae1\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:07:19 crc kubenswrapper[5052]: I1203 12:07:19.451837 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p2wkc\" (UniqueName: \"kubernetes.io/projected/ff9eaa72-6d5d-48a9-85a5-362de3895ae1-kube-api-access-p2wkc\") pod \"nova-api-0\" (UID: \"ff9eaa72-6d5d-48a9-85a5-362de3895ae1\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:07:19 crc kubenswrapper[5052]: I1203 12:07:19.553469 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/ff9eaa72-6d5d-48a9-85a5-362de3895ae1-logs\") pod \"nova-api-0\" (UID: \"ff9eaa72-6d5d-48a9-85a5-362de3895ae1\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:07:19 crc kubenswrapper[5052]: I1203 12:07:19.553559 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/ff9eaa72-6d5d-48a9-85a5-362de3895ae1-config-data\") pod \"nova-api-0\" (UID: \"ff9eaa72-6d5d-48a9-85a5-362de3895ae1\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:07:19 crc kubenswrapper[5052]: I1203 12:07:19.553599 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ff9eaa72-6d5d-48a9-85a5-362de3895ae1-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"ff9eaa72-6d5d-48a9-85a5-362de3895ae1\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:07:19 crc kubenswrapper[5052]: I1203 12:07:19.553644 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-p2wkc\" (UniqueName: \"kubernetes.io/projected/ff9eaa72-6d5d-48a9-85a5-362de3895ae1-kube-api-access-p2wkc\") pod \"nova-api-0\" (UID: \"ff9eaa72-6d5d-48a9-85a5-362de3895ae1\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:07:19 crc kubenswrapper[5052]: I1203 12:07:19.554409 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/ff9eaa72-6d5d-48a9-85a5-362de3895ae1-logs\") pod \"nova-api-0\" (UID: \"ff9eaa72-6d5d-48a9-85a5-362de3895ae1\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:07:19 crc kubenswrapper[5052]: I1203 12:07:19.559391 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/ff9eaa72-6d5d-48a9-85a5-362de3895ae1-config-data\") pod \"nova-api-0\" (UID: \"ff9eaa72-6d5d-48a9-85a5-362de3895ae1\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:07:19 crc kubenswrapper[5052]: I1203 12:07:19.560197 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ff9eaa72-6d5d-48a9-85a5-362de3895ae1-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"ff9eaa72-6d5d-48a9-85a5-362de3895ae1\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:07:19 crc kubenswrapper[5052]: I1203 12:07:19.576433 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:07:19 crc kubenswrapper[5052]: I1203 12:07:19.576486 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:07:19 crc kubenswrapper[5052]: I1203 12:07:19.580779 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-p2wkc\" (UniqueName: \"kubernetes.io/projected/ff9eaa72-6d5d-48a9-85a5-362de3895ae1-kube-api-access-p2wkc\") pod \"nova-api-0\" (UID: \"ff9eaa72-6d5d-48a9-85a5-362de3895ae1\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:07:19 crc kubenswrapper[5052]: I1203 12:07:19.667504 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:07:19 crc kubenswrapper[5052]: I1203 12:07:19.703413 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bdbb00d9-d8ce-4a03-843f-ab8441279186" path="/var/lib/kubelet/pods/bdbb00d9-d8ce-4a03-843f-ab8441279186/volumes" Dec 03 12:07:19 crc kubenswrapper[5052]: I1203 12:07:19.729536 5052 scope.go:117] "RemoveContainer" containerID="11c4c6496ed9aa094f5dc8ac4a34886ebc95b89aadefbfc8fa7de82792231539" Dec 03 12:07:20 crc kubenswrapper[5052]: I1203 12:07:20.191813 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:07:20 crc kubenswrapper[5052]: I1203 12:07:20.274328 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"d99defb0-caae-4f5f-9a24-d3026d396c31","Type":"ContainerStarted","Data":"c9cc962c26f0896a3372d1fc98911dbd818567d0b3bfed73cedd9a3e3b5cee76"} Dec 03 12:07:20 crc kubenswrapper[5052]: I1203 12:07:20.278729 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"ff9eaa72-6d5d-48a9-85a5-362de3895ae1","Type":"ContainerStarted","Data":"64c0cc84b0ba0b961e6145907450619d1e348b85b1c4b76a640e201a2990086c"} Dec 03 12:07:20 crc kubenswrapper[5052]: I1203 12:07:20.293127 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-scheduler-0" podStartSLOduration=3.293110646 podStartE2EDuration="3.293110646s" podCreationTimestamp="2025-12-03 12:07:17 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:07:20.291347656 +0000 UTC m=+2830.879589420" watchObservedRunningTime="2025-12-03 12:07:20.293110646 +0000 UTC m=+2830.881352410" Dec 03 12:07:21 crc kubenswrapper[5052]: I1203 12:07:21.291316 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"ff9eaa72-6d5d-48a9-85a5-362de3895ae1","Type":"ContainerStarted","Data":"ffc10685c0afd2a841a5fca1660a8031d066ef01fd68e5278b19f1ed0ca8af9a"} Dec 03 12:07:21 crc kubenswrapper[5052]: I1203 12:07:21.292731 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"ff9eaa72-6d5d-48a9-85a5-362de3895ae1","Type":"ContainerStarted","Data":"7724519ba83d2fb0f80b9e295f1c21f87d8043850d773806c593cd8e175c80ad"} Dec 03 12:07:22 crc kubenswrapper[5052]: I1203 12:07:22.892079 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:07:24 crc kubenswrapper[5052]: I1203 12:07:24.577435 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:07:24 crc kubenswrapper[5052]: I1203 12:07:24.577493 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:07:25 crc kubenswrapper[5052]: I1203 12:07:25.589802 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-metadata-0" podUID="26024570-e836-484e-be44-8bf3a86277df" containerName="nova-metadata-metadata" probeResult="failure" output="Get \"https://10.217.0.186:8775/\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Dec 03 12:07:25 crc kubenswrapper[5052]: I1203 12:07:25.589850 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-metadata-0" podUID="26024570-e836-484e-be44-8bf3a86277df" containerName="nova-metadata-log" probeResult="failure" output="Get \"https://10.217.0.186:8775/\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Dec 03 12:07:26 crc kubenswrapper[5052]: I1203 12:07:26.169670 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:07:26 crc kubenswrapper[5052]: I1203 12:07:26.203174 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-api-0" podStartSLOduration=7.20315407 podStartE2EDuration="7.20315407s" podCreationTimestamp="2025-12-03 12:07:19 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:07:21.307187357 +0000 UTC m=+2831.895429121" watchObservedRunningTime="2025-12-03 12:07:26.20315407 +0000 UTC m=+2836.791395834" Dec 03 12:07:27 crc kubenswrapper[5052]: I1203 12:07:27.892697 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:07:27 crc kubenswrapper[5052]: I1203 12:07:27.919125 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:07:28 crc kubenswrapper[5052]: I1203 12:07:28.391618 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:07:29 crc kubenswrapper[5052]: I1203 12:07:29.668191 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:07:29 crc kubenswrapper[5052]: I1203 12:07:29.668274 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:07:30 crc kubenswrapper[5052]: I1203 12:07:30.083771 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/kube-state-metrics-0"] Dec 03 12:07:30 crc kubenswrapper[5052]: I1203 12:07:30.084419 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/kube-state-metrics-0" podUID="8de0e655-f56e-40a1-b154-74ff024542ed" containerName="kube-state-metrics" containerID="cri-o://0572f095ac1d9fd1aaaca356eeaaa88ff3c268ecc5d27416e4297c1a0bd46b82" gracePeriod=30 Dec 03 12:07:30 crc kubenswrapper[5052]: I1203 12:07:30.385134 5052 generic.go:334] "Generic (PLEG): container finished" podID="8de0e655-f56e-40a1-b154-74ff024542ed" containerID="0572f095ac1d9fd1aaaca356eeaaa88ff3c268ecc5d27416e4297c1a0bd46b82" exitCode=2 Dec 03 12:07:30 crc kubenswrapper[5052]: I1203 12:07:30.385196 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/kube-state-metrics-0" event={"ID":"8de0e655-f56e-40a1-b154-74ff024542ed","Type":"ContainerDied","Data":"0572f095ac1d9fd1aaaca356eeaaa88ff3c268ecc5d27416e4297c1a0bd46b82"} Dec 03 12:07:30 crc kubenswrapper[5052]: I1203 12:07:30.751772 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-api-0" podUID="ff9eaa72-6d5d-48a9-85a5-362de3895ae1" containerName="nova-api-log" probeResult="failure" output="Get \"http://10.217.0.188:8774/\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Dec 03 12:07:30 crc kubenswrapper[5052]: I1203 12:07:30.751834 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-api-0" podUID="ff9eaa72-6d5d-48a9-85a5-362de3895ae1" containerName="nova-api-api" probeResult="failure" output="Get \"http://10.217.0.188:8774/\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Dec 03 12:07:31 crc kubenswrapper[5052]: I1203 12:07:31.222648 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:07:31 crc kubenswrapper[5052]: I1203 12:07:31.355780 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-97fxw\" (UniqueName: \"kubernetes.io/projected/8de0e655-f56e-40a1-b154-74ff024542ed-kube-api-access-97fxw\") pod \"8de0e655-f56e-40a1-b154-74ff024542ed\" (UID: \"8de0e655-f56e-40a1-b154-74ff024542ed\") " Dec 03 12:07:31 crc kubenswrapper[5052]: I1203 12:07:31.383716 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8de0e655-f56e-40a1-b154-74ff024542ed-kube-api-access-97fxw" (OuterVolumeSpecName: "kube-api-access-97fxw") pod "8de0e655-f56e-40a1-b154-74ff024542ed" (UID: "8de0e655-f56e-40a1-b154-74ff024542ed"). InnerVolumeSpecName "kube-api-access-97fxw". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:07:31 crc kubenswrapper[5052]: I1203 12:07:31.416735 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/kube-state-metrics-0" event={"ID":"8de0e655-f56e-40a1-b154-74ff024542ed","Type":"ContainerDied","Data":"f9d45e2c0b71c2af7b0a9dbe237c4df2a93ed8ebafd343943bf8443e3208c099"} Dec 03 12:07:31 crc kubenswrapper[5052]: I1203 12:07:31.416790 5052 scope.go:117] "RemoveContainer" containerID="0572f095ac1d9fd1aaaca356eeaaa88ff3c268ecc5d27416e4297c1a0bd46b82" Dec 03 12:07:31 crc kubenswrapper[5052]: I1203 12:07:31.416926 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:07:31 crc kubenswrapper[5052]: I1203 12:07:31.459363 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-97fxw\" (UniqueName: \"kubernetes.io/projected/8de0e655-f56e-40a1-b154-74ff024542ed-kube-api-access-97fxw\") on node \"crc\" DevicePath \"\"" Dec 03 12:07:31 crc kubenswrapper[5052]: I1203 12:07:31.491484 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/kube-state-metrics-0"] Dec 03 12:07:31 crc kubenswrapper[5052]: I1203 12:07:31.505576 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/kube-state-metrics-0"] Dec 03 12:07:31 crc kubenswrapper[5052]: I1203 12:07:31.514601 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/kube-state-metrics-0"] Dec 03 12:07:31 crc kubenswrapper[5052]: E1203 12:07:31.515042 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="8de0e655-f56e-40a1-b154-74ff024542ed" containerName="kube-state-metrics" Dec 03 12:07:31 crc kubenswrapper[5052]: I1203 12:07:31.515066 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="8de0e655-f56e-40a1-b154-74ff024542ed" containerName="kube-state-metrics" Dec 03 12:07:31 crc kubenswrapper[5052]: I1203 12:07:31.515245 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="8de0e655-f56e-40a1-b154-74ff024542ed" containerName="kube-state-metrics" Dec 03 12:07:31 crc kubenswrapper[5052]: I1203 12:07:31.515986 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:07:31 crc kubenswrapper[5052]: I1203 12:07:31.517992 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"kube-state-metrics-tls-config" Dec 03 12:07:31 crc kubenswrapper[5052]: I1203 12:07:31.518239 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-kube-state-metrics-svc" Dec 03 12:07:31 crc kubenswrapper[5052]: I1203 12:07:31.536039 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/kube-state-metrics-0"] Dec 03 12:07:31 crc kubenswrapper[5052]: I1203 12:07:31.663664 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-crnn7\" (UniqueName: \"kubernetes.io/projected/305f2518-9632-4358-84a1-632b4a7d8f6f-kube-api-access-crnn7\") pod \"kube-state-metrics-0\" (UID: \"305f2518-9632-4358-84a1-632b4a7d8f6f\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:07:31 crc kubenswrapper[5052]: I1203 12:07:31.664076 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/305f2518-9632-4358-84a1-632b4a7d8f6f-combined-ca-bundle\") pod \"kube-state-metrics-0\" (UID: \"305f2518-9632-4358-84a1-632b4a7d8f6f\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:07:31 crc kubenswrapper[5052]: I1203 12:07:31.664200 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls-certs\" (UniqueName: \"kubernetes.io/secret/305f2518-9632-4358-84a1-632b4a7d8f6f-kube-state-metrics-tls-certs\") pod \"kube-state-metrics-0\" (UID: \"305f2518-9632-4358-84a1-632b4a7d8f6f\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:07:31 crc kubenswrapper[5052]: I1203 12:07:31.664283 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls-config\" (UniqueName: \"kubernetes.io/secret/305f2518-9632-4358-84a1-632b4a7d8f6f-kube-state-metrics-tls-config\") pod \"kube-state-metrics-0\" (UID: \"305f2518-9632-4358-84a1-632b4a7d8f6f\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:07:31 crc kubenswrapper[5052]: I1203 12:07:31.688032 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8de0e655-f56e-40a1-b154-74ff024542ed" path="/var/lib/kubelet/pods/8de0e655-f56e-40a1-b154-74ff024542ed/volumes" Dec 03 12:07:31 crc kubenswrapper[5052]: I1203 12:07:31.765790 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-crnn7\" (UniqueName: \"kubernetes.io/projected/305f2518-9632-4358-84a1-632b4a7d8f6f-kube-api-access-crnn7\") pod \"kube-state-metrics-0\" (UID: \"305f2518-9632-4358-84a1-632b4a7d8f6f\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:07:31 crc kubenswrapper[5052]: I1203 12:07:31.765894 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/305f2518-9632-4358-84a1-632b4a7d8f6f-combined-ca-bundle\") pod \"kube-state-metrics-0\" (UID: \"305f2518-9632-4358-84a1-632b4a7d8f6f\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:07:31 crc kubenswrapper[5052]: I1203 12:07:31.766068 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls-certs\" (UniqueName: \"kubernetes.io/secret/305f2518-9632-4358-84a1-632b4a7d8f6f-kube-state-metrics-tls-certs\") pod \"kube-state-metrics-0\" (UID: \"305f2518-9632-4358-84a1-632b4a7d8f6f\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:07:31 crc kubenswrapper[5052]: I1203 12:07:31.766156 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls-config\" (UniqueName: \"kubernetes.io/secret/305f2518-9632-4358-84a1-632b4a7d8f6f-kube-state-metrics-tls-config\") pod \"kube-state-metrics-0\" (UID: \"305f2518-9632-4358-84a1-632b4a7d8f6f\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:07:31 crc kubenswrapper[5052]: I1203 12:07:31.774132 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls-config\" (UniqueName: \"kubernetes.io/secret/305f2518-9632-4358-84a1-632b4a7d8f6f-kube-state-metrics-tls-config\") pod \"kube-state-metrics-0\" (UID: \"305f2518-9632-4358-84a1-632b4a7d8f6f\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:07:31 crc kubenswrapper[5052]: I1203 12:07:31.774465 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls-certs\" (UniqueName: \"kubernetes.io/secret/305f2518-9632-4358-84a1-632b4a7d8f6f-kube-state-metrics-tls-certs\") pod \"kube-state-metrics-0\" (UID: \"305f2518-9632-4358-84a1-632b4a7d8f6f\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:07:31 crc kubenswrapper[5052]: I1203 12:07:31.778555 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/305f2518-9632-4358-84a1-632b4a7d8f6f-combined-ca-bundle\") pod \"kube-state-metrics-0\" (UID: \"305f2518-9632-4358-84a1-632b4a7d8f6f\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:07:31 crc kubenswrapper[5052]: I1203 12:07:31.785647 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-crnn7\" (UniqueName: \"kubernetes.io/projected/305f2518-9632-4358-84a1-632b4a7d8f6f-kube-api-access-crnn7\") pod \"kube-state-metrics-0\" (UID: \"305f2518-9632-4358-84a1-632b4a7d8f6f\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:07:31 crc kubenswrapper[5052]: I1203 12:07:31.844239 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:07:32 crc kubenswrapper[5052]: I1203 12:07:32.370357 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/kube-state-metrics-0"] Dec 03 12:07:32 crc kubenswrapper[5052]: W1203 12:07:32.379063 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod305f2518_9632_4358_84a1_632b4a7d8f6f.slice/crio-9601255802d345a0ea7eb7506def54601b12e61f8b10070fefb875a78e96c4a3 WatchSource:0}: Error finding container 9601255802d345a0ea7eb7506def54601b12e61f8b10070fefb875a78e96c4a3: Status 404 returned error can't find the container with id 9601255802d345a0ea7eb7506def54601b12e61f8b10070fefb875a78e96c4a3 Dec 03 12:07:32 crc kubenswrapper[5052]: I1203 12:07:32.439194 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/kube-state-metrics-0" event={"ID":"305f2518-9632-4358-84a1-632b4a7d8f6f","Type":"ContainerStarted","Data":"9601255802d345a0ea7eb7506def54601b12e61f8b10070fefb875a78e96c4a3"} Dec 03 12:07:32 crc kubenswrapper[5052]: I1203 12:07:32.619023 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:07:32 crc kubenswrapper[5052]: I1203 12:07:32.619731 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="ec281373-bf8b-4f4d-8050-206d92074fc2" containerName="ceilometer-central-agent" containerID="cri-o://c8cec5b21931f0c586ae94d07bb14801f9109afc2ce06eaff70de5126b8d93d0" gracePeriod=30 Dec 03 12:07:32 crc kubenswrapper[5052]: I1203 12:07:32.621689 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="ec281373-bf8b-4f4d-8050-206d92074fc2" containerName="proxy-httpd" containerID="cri-o://d7dc9b3662b3541642913d5bde8644885a28a2bee3adc98f6755b9d0edaec722" gracePeriod=30 Dec 03 12:07:32 crc kubenswrapper[5052]: I1203 12:07:32.621931 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="ec281373-bf8b-4f4d-8050-206d92074fc2" containerName="sg-core" containerID="cri-o://c689b558d9359e9ed0e4c3c4a7fbba0ce2251869d67d2d0fc23df0e151735dde" gracePeriod=30 Dec 03 12:07:32 crc kubenswrapper[5052]: I1203 12:07:32.622088 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="ec281373-bf8b-4f4d-8050-206d92074fc2" containerName="ceilometer-notification-agent" containerID="cri-o://f00b78456ae51af13add1990be39cd3856c4bfb8861d683f6ed3f06327226113" gracePeriod=30 Dec 03 12:07:33 crc kubenswrapper[5052]: I1203 12:07:33.453113 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/kube-state-metrics-0" event={"ID":"305f2518-9632-4358-84a1-632b4a7d8f6f","Type":"ContainerStarted","Data":"73022661e4a723ea9ac382d6839ad1127d1eaf64b3149c3d92b5bdf83208e996"} Dec 03 12:07:33 crc kubenswrapper[5052]: I1203 12:07:33.455264 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:07:33 crc kubenswrapper[5052]: I1203 12:07:33.459071 5052 generic.go:334] "Generic (PLEG): container finished" podID="ec281373-bf8b-4f4d-8050-206d92074fc2" containerID="d7dc9b3662b3541642913d5bde8644885a28a2bee3adc98f6755b9d0edaec722" exitCode=0 Dec 03 12:07:33 crc kubenswrapper[5052]: I1203 12:07:33.459176 5052 generic.go:334] "Generic (PLEG): container finished" podID="ec281373-bf8b-4f4d-8050-206d92074fc2" containerID="c689b558d9359e9ed0e4c3c4a7fbba0ce2251869d67d2d0fc23df0e151735dde" exitCode=2 Dec 03 12:07:33 crc kubenswrapper[5052]: I1203 12:07:33.459189 5052 generic.go:334] "Generic (PLEG): container finished" podID="ec281373-bf8b-4f4d-8050-206d92074fc2" containerID="c8cec5b21931f0c586ae94d07bb14801f9109afc2ce06eaff70de5126b8d93d0" exitCode=0 Dec 03 12:07:33 crc kubenswrapper[5052]: I1203 12:07:33.459128 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"ec281373-bf8b-4f4d-8050-206d92074fc2","Type":"ContainerDied","Data":"d7dc9b3662b3541642913d5bde8644885a28a2bee3adc98f6755b9d0edaec722"} Dec 03 12:07:33 crc kubenswrapper[5052]: I1203 12:07:33.459249 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"ec281373-bf8b-4f4d-8050-206d92074fc2","Type":"ContainerDied","Data":"c689b558d9359e9ed0e4c3c4a7fbba0ce2251869d67d2d0fc23df0e151735dde"} Dec 03 12:07:33 crc kubenswrapper[5052]: I1203 12:07:33.459265 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"ec281373-bf8b-4f4d-8050-206d92074fc2","Type":"ContainerDied","Data":"c8cec5b21931f0c586ae94d07bb14801f9109afc2ce06eaff70de5126b8d93d0"} Dec 03 12:07:33 crc kubenswrapper[5052]: I1203 12:07:33.483722 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/kube-state-metrics-0" podStartSLOduration=2.095059424 podStartE2EDuration="2.483702056s" podCreationTimestamp="2025-12-03 12:07:31 +0000 UTC" firstStartedPulling="2025-12-03 12:07:32.382193014 +0000 UTC m=+2842.970434778" lastFinishedPulling="2025-12-03 12:07:32.770835646 +0000 UTC m=+2843.359077410" observedRunningTime="2025-12-03 12:07:33.471840027 +0000 UTC m=+2844.060081791" watchObservedRunningTime="2025-12-03 12:07:33.483702056 +0000 UTC m=+2844.071943820" Dec 03 12:07:34 crc kubenswrapper[5052]: I1203 12:07:34.399728 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:07:34 crc kubenswrapper[5052]: I1203 12:07:34.477926 5052 generic.go:334] "Generic (PLEG): container finished" podID="ec281373-bf8b-4f4d-8050-206d92074fc2" containerID="f00b78456ae51af13add1990be39cd3856c4bfb8861d683f6ed3f06327226113" exitCode=0 Dec 03 12:07:34 crc kubenswrapper[5052]: I1203 12:07:34.478272 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"ec281373-bf8b-4f4d-8050-206d92074fc2","Type":"ContainerDied","Data":"f00b78456ae51af13add1990be39cd3856c4bfb8861d683f6ed3f06327226113"} Dec 03 12:07:34 crc kubenswrapper[5052]: I1203 12:07:34.478337 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"ec281373-bf8b-4f4d-8050-206d92074fc2","Type":"ContainerDied","Data":"40617227a3c892bfd6e9c50dad8dc06942d41abb865300b6d39b74ad2eefa584"} Dec 03 12:07:34 crc kubenswrapper[5052]: I1203 12:07:34.478369 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:07:34 crc kubenswrapper[5052]: I1203 12:07:34.478427 5052 scope.go:117] "RemoveContainer" containerID="d7dc9b3662b3541642913d5bde8644885a28a2bee3adc98f6755b9d0edaec722" Dec 03 12:07:34 crc kubenswrapper[5052]: I1203 12:07:34.503465 5052 scope.go:117] "RemoveContainer" containerID="c689b558d9359e9ed0e4c3c4a7fbba0ce2251869d67d2d0fc23df0e151735dde" Dec 03 12:07:34 crc kubenswrapper[5052]: I1203 12:07:34.531336 5052 scope.go:117] "RemoveContainer" containerID="f00b78456ae51af13add1990be39cd3856c4bfb8861d683f6ed3f06327226113" Dec 03 12:07:34 crc kubenswrapper[5052]: I1203 12:07:34.556744 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/ec281373-bf8b-4f4d-8050-206d92074fc2-log-httpd\") pod \"ec281373-bf8b-4f4d-8050-206d92074fc2\" (UID: \"ec281373-bf8b-4f4d-8050-206d92074fc2\") " Dec 03 12:07:34 crc kubenswrapper[5052]: I1203 12:07:34.556854 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/ec281373-bf8b-4f4d-8050-206d92074fc2-scripts\") pod \"ec281373-bf8b-4f4d-8050-206d92074fc2\" (UID: \"ec281373-bf8b-4f4d-8050-206d92074fc2\") " Dec 03 12:07:34 crc kubenswrapper[5052]: I1203 12:07:34.556889 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/ec281373-bf8b-4f4d-8050-206d92074fc2-sg-core-conf-yaml\") pod \"ec281373-bf8b-4f4d-8050-206d92074fc2\" (UID: \"ec281373-bf8b-4f4d-8050-206d92074fc2\") " Dec 03 12:07:34 crc kubenswrapper[5052]: I1203 12:07:34.556934 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/ec281373-bf8b-4f4d-8050-206d92074fc2-config-data\") pod \"ec281373-bf8b-4f4d-8050-206d92074fc2\" (UID: \"ec281373-bf8b-4f4d-8050-206d92074fc2\") " Dec 03 12:07:34 crc kubenswrapper[5052]: I1203 12:07:34.556999 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/ec281373-bf8b-4f4d-8050-206d92074fc2-run-httpd\") pod \"ec281373-bf8b-4f4d-8050-206d92074fc2\" (UID: \"ec281373-bf8b-4f4d-8050-206d92074fc2\") " Dec 03 12:07:34 crc kubenswrapper[5052]: I1203 12:07:34.557080 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6v9q9\" (UniqueName: \"kubernetes.io/projected/ec281373-bf8b-4f4d-8050-206d92074fc2-kube-api-access-6v9q9\") pod \"ec281373-bf8b-4f4d-8050-206d92074fc2\" (UID: \"ec281373-bf8b-4f4d-8050-206d92074fc2\") " Dec 03 12:07:34 crc kubenswrapper[5052]: I1203 12:07:34.557143 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ec281373-bf8b-4f4d-8050-206d92074fc2-combined-ca-bundle\") pod \"ec281373-bf8b-4f4d-8050-206d92074fc2\" (UID: \"ec281373-bf8b-4f4d-8050-206d92074fc2\") " Dec 03 12:07:34 crc kubenswrapper[5052]: I1203 12:07:34.559090 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec281373-bf8b-4f4d-8050-206d92074fc2-run-httpd" (OuterVolumeSpecName: "run-httpd") pod "ec281373-bf8b-4f4d-8050-206d92074fc2" (UID: "ec281373-bf8b-4f4d-8050-206d92074fc2"). InnerVolumeSpecName "run-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:07:34 crc kubenswrapper[5052]: I1203 12:07:34.559307 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec281373-bf8b-4f4d-8050-206d92074fc2-log-httpd" (OuterVolumeSpecName: "log-httpd") pod "ec281373-bf8b-4f4d-8050-206d92074fc2" (UID: "ec281373-bf8b-4f4d-8050-206d92074fc2"). InnerVolumeSpecName "log-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:07:34 crc kubenswrapper[5052]: I1203 12:07:34.560439 5052 scope.go:117] "RemoveContainer" containerID="c8cec5b21931f0c586ae94d07bb14801f9109afc2ce06eaff70de5126b8d93d0" Dec 03 12:07:34 crc kubenswrapper[5052]: I1203 12:07:34.566011 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ec281373-bf8b-4f4d-8050-206d92074fc2-kube-api-access-6v9q9" (OuterVolumeSpecName: "kube-api-access-6v9q9") pod "ec281373-bf8b-4f4d-8050-206d92074fc2" (UID: "ec281373-bf8b-4f4d-8050-206d92074fc2"). InnerVolumeSpecName "kube-api-access-6v9q9". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:07:34 crc kubenswrapper[5052]: I1203 12:07:34.579733 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ec281373-bf8b-4f4d-8050-206d92074fc2-scripts" (OuterVolumeSpecName: "scripts") pod "ec281373-bf8b-4f4d-8050-206d92074fc2" (UID: "ec281373-bf8b-4f4d-8050-206d92074fc2"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:07:34 crc kubenswrapper[5052]: I1203 12:07:34.595834 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:07:34 crc kubenswrapper[5052]: I1203 12:07:34.596205 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ec281373-bf8b-4f4d-8050-206d92074fc2-sg-core-conf-yaml" (OuterVolumeSpecName: "sg-core-conf-yaml") pod "ec281373-bf8b-4f4d-8050-206d92074fc2" (UID: "ec281373-bf8b-4f4d-8050-206d92074fc2"). InnerVolumeSpecName "sg-core-conf-yaml". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:07:34 crc kubenswrapper[5052]: I1203 12:07:34.601147 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:07:34 crc kubenswrapper[5052]: I1203 12:07:34.610804 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:07:34 crc kubenswrapper[5052]: I1203 12:07:34.660056 5052 reconciler_common.go:293] "Volume detached for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/ec281373-bf8b-4f4d-8050-206d92074fc2-log-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:07:34 crc kubenswrapper[5052]: I1203 12:07:34.660082 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/ec281373-bf8b-4f4d-8050-206d92074fc2-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:07:34 crc kubenswrapper[5052]: I1203 12:07:34.660091 5052 reconciler_common.go:293] "Volume detached for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/ec281373-bf8b-4f4d-8050-206d92074fc2-sg-core-conf-yaml\") on node \"crc\" DevicePath \"\"" Dec 03 12:07:34 crc kubenswrapper[5052]: I1203 12:07:34.660099 5052 reconciler_common.go:293] "Volume detached for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/ec281373-bf8b-4f4d-8050-206d92074fc2-run-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:07:34 crc kubenswrapper[5052]: I1203 12:07:34.660110 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-6v9q9\" (UniqueName: \"kubernetes.io/projected/ec281373-bf8b-4f4d-8050-206d92074fc2-kube-api-access-6v9q9\") on node \"crc\" DevicePath \"\"" Dec 03 12:07:34 crc kubenswrapper[5052]: I1203 12:07:34.695538 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ec281373-bf8b-4f4d-8050-206d92074fc2-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "ec281373-bf8b-4f4d-8050-206d92074fc2" (UID: "ec281373-bf8b-4f4d-8050-206d92074fc2"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:07:34 crc kubenswrapper[5052]: I1203 12:07:34.735804 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ec281373-bf8b-4f4d-8050-206d92074fc2-config-data" (OuterVolumeSpecName: "config-data") pod "ec281373-bf8b-4f4d-8050-206d92074fc2" (UID: "ec281373-bf8b-4f4d-8050-206d92074fc2"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:07:34 crc kubenswrapper[5052]: I1203 12:07:34.762422 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/ec281373-bf8b-4f4d-8050-206d92074fc2-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:07:34 crc kubenswrapper[5052]: I1203 12:07:34.762459 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ec281373-bf8b-4f4d-8050-206d92074fc2-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:07:34 crc kubenswrapper[5052]: I1203 12:07:34.818802 5052 scope.go:117] "RemoveContainer" containerID="d7dc9b3662b3541642913d5bde8644885a28a2bee3adc98f6755b9d0edaec722" Dec 03 12:07:34 crc kubenswrapper[5052]: E1203 12:07:34.819399 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d7dc9b3662b3541642913d5bde8644885a28a2bee3adc98f6755b9d0edaec722\": container with ID starting with d7dc9b3662b3541642913d5bde8644885a28a2bee3adc98f6755b9d0edaec722 not found: ID does not exist" containerID="d7dc9b3662b3541642913d5bde8644885a28a2bee3adc98f6755b9d0edaec722" Dec 03 12:07:34 crc kubenswrapper[5052]: I1203 12:07:34.819453 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d7dc9b3662b3541642913d5bde8644885a28a2bee3adc98f6755b9d0edaec722"} err="failed to get container status \"d7dc9b3662b3541642913d5bde8644885a28a2bee3adc98f6755b9d0edaec722\": rpc error: code = NotFound desc = could not find container \"d7dc9b3662b3541642913d5bde8644885a28a2bee3adc98f6755b9d0edaec722\": container with ID starting with d7dc9b3662b3541642913d5bde8644885a28a2bee3adc98f6755b9d0edaec722 not found: ID does not exist" Dec 03 12:07:34 crc kubenswrapper[5052]: I1203 12:07:34.819484 5052 scope.go:117] "RemoveContainer" containerID="c689b558d9359e9ed0e4c3c4a7fbba0ce2251869d67d2d0fc23df0e151735dde" Dec 03 12:07:34 crc kubenswrapper[5052]: E1203 12:07:34.819923 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c689b558d9359e9ed0e4c3c4a7fbba0ce2251869d67d2d0fc23df0e151735dde\": container with ID starting with c689b558d9359e9ed0e4c3c4a7fbba0ce2251869d67d2d0fc23df0e151735dde not found: ID does not exist" containerID="c689b558d9359e9ed0e4c3c4a7fbba0ce2251869d67d2d0fc23df0e151735dde" Dec 03 12:07:34 crc kubenswrapper[5052]: I1203 12:07:34.819950 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c689b558d9359e9ed0e4c3c4a7fbba0ce2251869d67d2d0fc23df0e151735dde"} err="failed to get container status \"c689b558d9359e9ed0e4c3c4a7fbba0ce2251869d67d2d0fc23df0e151735dde\": rpc error: code = NotFound desc = could not find container \"c689b558d9359e9ed0e4c3c4a7fbba0ce2251869d67d2d0fc23df0e151735dde\": container with ID starting with c689b558d9359e9ed0e4c3c4a7fbba0ce2251869d67d2d0fc23df0e151735dde not found: ID does not exist" Dec 03 12:07:34 crc kubenswrapper[5052]: I1203 12:07:34.819964 5052 scope.go:117] "RemoveContainer" containerID="f00b78456ae51af13add1990be39cd3856c4bfb8861d683f6ed3f06327226113" Dec 03 12:07:34 crc kubenswrapper[5052]: E1203 12:07:34.820204 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f00b78456ae51af13add1990be39cd3856c4bfb8861d683f6ed3f06327226113\": container with ID starting with f00b78456ae51af13add1990be39cd3856c4bfb8861d683f6ed3f06327226113 not found: ID does not exist" containerID="f00b78456ae51af13add1990be39cd3856c4bfb8861d683f6ed3f06327226113" Dec 03 12:07:34 crc kubenswrapper[5052]: I1203 12:07:34.820229 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f00b78456ae51af13add1990be39cd3856c4bfb8861d683f6ed3f06327226113"} err="failed to get container status \"f00b78456ae51af13add1990be39cd3856c4bfb8861d683f6ed3f06327226113\": rpc error: code = NotFound desc = could not find container \"f00b78456ae51af13add1990be39cd3856c4bfb8861d683f6ed3f06327226113\": container with ID starting with f00b78456ae51af13add1990be39cd3856c4bfb8861d683f6ed3f06327226113 not found: ID does not exist" Dec 03 12:07:34 crc kubenswrapper[5052]: I1203 12:07:34.820244 5052 scope.go:117] "RemoveContainer" containerID="c8cec5b21931f0c586ae94d07bb14801f9109afc2ce06eaff70de5126b8d93d0" Dec 03 12:07:34 crc kubenswrapper[5052]: E1203 12:07:34.820460 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c8cec5b21931f0c586ae94d07bb14801f9109afc2ce06eaff70de5126b8d93d0\": container with ID starting with c8cec5b21931f0c586ae94d07bb14801f9109afc2ce06eaff70de5126b8d93d0 not found: ID does not exist" containerID="c8cec5b21931f0c586ae94d07bb14801f9109afc2ce06eaff70de5126b8d93d0" Dec 03 12:07:34 crc kubenswrapper[5052]: I1203 12:07:34.820490 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c8cec5b21931f0c586ae94d07bb14801f9109afc2ce06eaff70de5126b8d93d0"} err="failed to get container status \"c8cec5b21931f0c586ae94d07bb14801f9109afc2ce06eaff70de5126b8d93d0\": rpc error: code = NotFound desc = could not find container \"c8cec5b21931f0c586ae94d07bb14801f9109afc2ce06eaff70de5126b8d93d0\": container with ID starting with c8cec5b21931f0c586ae94d07bb14801f9109afc2ce06eaff70de5126b8d93d0 not found: ID does not exist" Dec 03 12:07:34 crc kubenswrapper[5052]: I1203 12:07:34.826563 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:07:34 crc kubenswrapper[5052]: I1203 12:07:34.839719 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:07:34 crc kubenswrapper[5052]: I1203 12:07:34.854215 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:07:34 crc kubenswrapper[5052]: E1203 12:07:34.854786 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="ec281373-bf8b-4f4d-8050-206d92074fc2" containerName="ceilometer-central-agent" Dec 03 12:07:34 crc kubenswrapper[5052]: I1203 12:07:34.854808 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="ec281373-bf8b-4f4d-8050-206d92074fc2" containerName="ceilometer-central-agent" Dec 03 12:07:34 crc kubenswrapper[5052]: E1203 12:07:34.854841 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="ec281373-bf8b-4f4d-8050-206d92074fc2" containerName="ceilometer-notification-agent" Dec 03 12:07:34 crc kubenswrapper[5052]: I1203 12:07:34.854850 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="ec281373-bf8b-4f4d-8050-206d92074fc2" containerName="ceilometer-notification-agent" Dec 03 12:07:34 crc kubenswrapper[5052]: E1203 12:07:34.854874 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="ec281373-bf8b-4f4d-8050-206d92074fc2" containerName="sg-core" Dec 03 12:07:34 crc kubenswrapper[5052]: I1203 12:07:34.854881 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="ec281373-bf8b-4f4d-8050-206d92074fc2" containerName="sg-core" Dec 03 12:07:34 crc kubenswrapper[5052]: E1203 12:07:34.854904 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="ec281373-bf8b-4f4d-8050-206d92074fc2" containerName="proxy-httpd" Dec 03 12:07:34 crc kubenswrapper[5052]: I1203 12:07:34.854911 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="ec281373-bf8b-4f4d-8050-206d92074fc2" containerName="proxy-httpd" Dec 03 12:07:34 crc kubenswrapper[5052]: I1203 12:07:34.855205 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="ec281373-bf8b-4f4d-8050-206d92074fc2" containerName="proxy-httpd" Dec 03 12:07:34 crc kubenswrapper[5052]: I1203 12:07:34.855226 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="ec281373-bf8b-4f4d-8050-206d92074fc2" containerName="ceilometer-notification-agent" Dec 03 12:07:34 crc kubenswrapper[5052]: I1203 12:07:34.855242 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="ec281373-bf8b-4f4d-8050-206d92074fc2" containerName="sg-core" Dec 03 12:07:34 crc kubenswrapper[5052]: I1203 12:07:34.855250 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="ec281373-bf8b-4f4d-8050-206d92074fc2" containerName="ceilometer-central-agent" Dec 03 12:07:34 crc kubenswrapper[5052]: I1203 12:07:34.859475 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:07:34 crc kubenswrapper[5052]: I1203 12:07:34.862274 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-scripts" Dec 03 12:07:34 crc kubenswrapper[5052]: I1203 12:07:34.862436 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-config-data" Dec 03 12:07:34 crc kubenswrapper[5052]: I1203 12:07:34.862469 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-ceilometer-internal-svc" Dec 03 12:07:34 crc kubenswrapper[5052]: I1203 12:07:34.871282 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:07:34 crc kubenswrapper[5052]: I1203 12:07:34.965975 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/6c9d4db1-a93f-4624-a6e1-d3a31285d74f-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"6c9d4db1-a93f-4624-a6e1-d3a31285d74f\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:07:34 crc kubenswrapper[5052]: I1203 12:07:34.966047 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/6c9d4db1-a93f-4624-a6e1-d3a31285d74f-log-httpd\") pod \"ceilometer-0\" (UID: \"6c9d4db1-a93f-4624-a6e1-d3a31285d74f\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:07:34 crc kubenswrapper[5052]: I1203 12:07:34.966074 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6c9d4db1-a93f-4624-a6e1-d3a31285d74f-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"6c9d4db1-a93f-4624-a6e1-d3a31285d74f\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:07:34 crc kubenswrapper[5052]: I1203 12:07:34.966124 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dtsh2\" (UniqueName: \"kubernetes.io/projected/6c9d4db1-a93f-4624-a6e1-d3a31285d74f-kube-api-access-dtsh2\") pod \"ceilometer-0\" (UID: \"6c9d4db1-a93f-4624-a6e1-d3a31285d74f\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:07:34 crc kubenswrapper[5052]: I1203 12:07:34.966430 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6c9d4db1-a93f-4624-a6e1-d3a31285d74f-config-data\") pod \"ceilometer-0\" (UID: \"6c9d4db1-a93f-4624-a6e1-d3a31285d74f\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:07:34 crc kubenswrapper[5052]: I1203 12:07:34.966498 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/6c9d4db1-a93f-4624-a6e1-d3a31285d74f-run-httpd\") pod \"ceilometer-0\" (UID: \"6c9d4db1-a93f-4624-a6e1-d3a31285d74f\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:07:34 crc kubenswrapper[5052]: I1203 12:07:34.966657 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/6c9d4db1-a93f-4624-a6e1-d3a31285d74f-ceilometer-tls-certs\") pod \"ceilometer-0\" (UID: \"6c9d4db1-a93f-4624-a6e1-d3a31285d74f\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:07:34 crc kubenswrapper[5052]: I1203 12:07:34.966773 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/6c9d4db1-a93f-4624-a6e1-d3a31285d74f-scripts\") pod \"ceilometer-0\" (UID: \"6c9d4db1-a93f-4624-a6e1-d3a31285d74f\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:07:35 crc kubenswrapper[5052]: I1203 12:07:35.068525 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/6c9d4db1-a93f-4624-a6e1-d3a31285d74f-scripts\") pod \"ceilometer-0\" (UID: \"6c9d4db1-a93f-4624-a6e1-d3a31285d74f\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:07:35 crc kubenswrapper[5052]: I1203 12:07:35.069196 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/6c9d4db1-a93f-4624-a6e1-d3a31285d74f-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"6c9d4db1-a93f-4624-a6e1-d3a31285d74f\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:07:35 crc kubenswrapper[5052]: I1203 12:07:35.069309 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/6c9d4db1-a93f-4624-a6e1-d3a31285d74f-log-httpd\") pod \"ceilometer-0\" (UID: \"6c9d4db1-a93f-4624-a6e1-d3a31285d74f\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:07:35 crc kubenswrapper[5052]: I1203 12:07:35.069398 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6c9d4db1-a93f-4624-a6e1-d3a31285d74f-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"6c9d4db1-a93f-4624-a6e1-d3a31285d74f\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:07:35 crc kubenswrapper[5052]: I1203 12:07:35.069497 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-dtsh2\" (UniqueName: \"kubernetes.io/projected/6c9d4db1-a93f-4624-a6e1-d3a31285d74f-kube-api-access-dtsh2\") pod \"ceilometer-0\" (UID: \"6c9d4db1-a93f-4624-a6e1-d3a31285d74f\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:07:35 crc kubenswrapper[5052]: I1203 12:07:35.069752 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6c9d4db1-a93f-4624-a6e1-d3a31285d74f-config-data\") pod \"ceilometer-0\" (UID: \"6c9d4db1-a93f-4624-a6e1-d3a31285d74f\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:07:35 crc kubenswrapper[5052]: I1203 12:07:35.069859 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/6c9d4db1-a93f-4624-a6e1-d3a31285d74f-run-httpd\") pod \"ceilometer-0\" (UID: \"6c9d4db1-a93f-4624-a6e1-d3a31285d74f\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:07:35 crc kubenswrapper[5052]: I1203 12:07:35.069992 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/6c9d4db1-a93f-4624-a6e1-d3a31285d74f-ceilometer-tls-certs\") pod \"ceilometer-0\" (UID: \"6c9d4db1-a93f-4624-a6e1-d3a31285d74f\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:07:35 crc kubenswrapper[5052]: I1203 12:07:35.069997 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/6c9d4db1-a93f-4624-a6e1-d3a31285d74f-log-httpd\") pod \"ceilometer-0\" (UID: \"6c9d4db1-a93f-4624-a6e1-d3a31285d74f\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:07:35 crc kubenswrapper[5052]: I1203 12:07:35.070244 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/6c9d4db1-a93f-4624-a6e1-d3a31285d74f-run-httpd\") pod \"ceilometer-0\" (UID: \"6c9d4db1-a93f-4624-a6e1-d3a31285d74f\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:07:35 crc kubenswrapper[5052]: I1203 12:07:35.073627 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/6c9d4db1-a93f-4624-a6e1-d3a31285d74f-scripts\") pod \"ceilometer-0\" (UID: \"6c9d4db1-a93f-4624-a6e1-d3a31285d74f\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:07:35 crc kubenswrapper[5052]: I1203 12:07:35.073846 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6c9d4db1-a93f-4624-a6e1-d3a31285d74f-config-data\") pod \"ceilometer-0\" (UID: \"6c9d4db1-a93f-4624-a6e1-d3a31285d74f\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:07:35 crc kubenswrapper[5052]: I1203 12:07:35.076139 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/6c9d4db1-a93f-4624-a6e1-d3a31285d74f-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"6c9d4db1-a93f-4624-a6e1-d3a31285d74f\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:07:35 crc kubenswrapper[5052]: I1203 12:07:35.078117 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6c9d4db1-a93f-4624-a6e1-d3a31285d74f-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"6c9d4db1-a93f-4624-a6e1-d3a31285d74f\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:07:35 crc kubenswrapper[5052]: I1203 12:07:35.079244 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/6c9d4db1-a93f-4624-a6e1-d3a31285d74f-ceilometer-tls-certs\") pod \"ceilometer-0\" (UID: \"6c9d4db1-a93f-4624-a6e1-d3a31285d74f\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:07:35 crc kubenswrapper[5052]: I1203 12:07:35.089329 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-dtsh2\" (UniqueName: \"kubernetes.io/projected/6c9d4db1-a93f-4624-a6e1-d3a31285d74f-kube-api-access-dtsh2\") pod \"ceilometer-0\" (UID: \"6c9d4db1-a93f-4624-a6e1-d3a31285d74f\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:07:35 crc kubenswrapper[5052]: I1203 12:07:35.192816 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:07:35 crc kubenswrapper[5052]: I1203 12:07:35.509176 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:07:35 crc kubenswrapper[5052]: I1203 12:07:35.700998 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ec281373-bf8b-4f4d-8050-206d92074fc2" path="/var/lib/kubelet/pods/ec281373-bf8b-4f4d-8050-206d92074fc2/volumes" Dec 03 12:07:35 crc kubenswrapper[5052]: I1203 12:07:35.701903 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:07:36 crc kubenswrapper[5052]: I1203 12:07:36.506355 5052 generic.go:334] "Generic (PLEG): container finished" podID="2066fe4a-e7cc-44a5-b7d9-499d19a40c12" containerID="2c81d86ba7b2260bf28aacf6f22b1bf41b3bce50a7118749afee64eacc6c95a5" exitCode=0 Dec 03 12:07:36 crc kubenswrapper[5052]: I1203 12:07:36.506456 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-sxlxt" event={"ID":"2066fe4a-e7cc-44a5-b7d9-499d19a40c12","Type":"ContainerDied","Data":"2c81d86ba7b2260bf28aacf6f22b1bf41b3bce50a7118749afee64eacc6c95a5"} Dec 03 12:07:36 crc kubenswrapper[5052]: I1203 12:07:36.508015 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"6c9d4db1-a93f-4624-a6e1-d3a31285d74f","Type":"ContainerStarted","Data":"bbd05716e18f09f08d9e492a9aa982871f57b0fdae64a3b5e90400e6e6b0d021"} Dec 03 12:07:37 crc kubenswrapper[5052]: I1203 12:07:37.521404 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"6c9d4db1-a93f-4624-a6e1-d3a31285d74f","Type":"ContainerStarted","Data":"10cd1c0b34caba4bc9ff8ab8584ffb6af47033dc0321fe224635ffd1061e6c3c"} Dec 03 12:07:37 crc kubenswrapper[5052]: I1203 12:07:37.521806 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"6c9d4db1-a93f-4624-a6e1-d3a31285d74f","Type":"ContainerStarted","Data":"660744ff88d9dd18080ad97ca00100247da7f8bd7e7a74352c8cea4db0046533"} Dec 03 12:07:37 crc kubenswrapper[5052]: I1203 12:07:37.892555 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-sxlxt" Dec 03 12:07:38 crc kubenswrapper[5052]: I1203 12:07:38.047442 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2066fe4a-e7cc-44a5-b7d9-499d19a40c12-combined-ca-bundle\") pod \"2066fe4a-e7cc-44a5-b7d9-499d19a40c12\" (UID: \"2066fe4a-e7cc-44a5-b7d9-499d19a40c12\") " Dec 03 12:07:38 crc kubenswrapper[5052]: I1203 12:07:38.047602 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/2066fe4a-e7cc-44a5-b7d9-499d19a40c12-scripts\") pod \"2066fe4a-e7cc-44a5-b7d9-499d19a40c12\" (UID: \"2066fe4a-e7cc-44a5-b7d9-499d19a40c12\") " Dec 03 12:07:38 crc kubenswrapper[5052]: I1203 12:07:38.047635 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/2066fe4a-e7cc-44a5-b7d9-499d19a40c12-config-data\") pod \"2066fe4a-e7cc-44a5-b7d9-499d19a40c12\" (UID: \"2066fe4a-e7cc-44a5-b7d9-499d19a40c12\") " Dec 03 12:07:38 crc kubenswrapper[5052]: I1203 12:07:38.047665 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6b97m\" (UniqueName: \"kubernetes.io/projected/2066fe4a-e7cc-44a5-b7d9-499d19a40c12-kube-api-access-6b97m\") pod \"2066fe4a-e7cc-44a5-b7d9-499d19a40c12\" (UID: \"2066fe4a-e7cc-44a5-b7d9-499d19a40c12\") " Dec 03 12:07:38 crc kubenswrapper[5052]: I1203 12:07:38.053927 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2066fe4a-e7cc-44a5-b7d9-499d19a40c12-kube-api-access-6b97m" (OuterVolumeSpecName: "kube-api-access-6b97m") pod "2066fe4a-e7cc-44a5-b7d9-499d19a40c12" (UID: "2066fe4a-e7cc-44a5-b7d9-499d19a40c12"). InnerVolumeSpecName "kube-api-access-6b97m". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:07:38 crc kubenswrapper[5052]: I1203 12:07:38.054629 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2066fe4a-e7cc-44a5-b7d9-499d19a40c12-scripts" (OuterVolumeSpecName: "scripts") pod "2066fe4a-e7cc-44a5-b7d9-499d19a40c12" (UID: "2066fe4a-e7cc-44a5-b7d9-499d19a40c12"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:07:38 crc kubenswrapper[5052]: I1203 12:07:38.080736 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2066fe4a-e7cc-44a5-b7d9-499d19a40c12-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "2066fe4a-e7cc-44a5-b7d9-499d19a40c12" (UID: "2066fe4a-e7cc-44a5-b7d9-499d19a40c12"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:07:38 crc kubenswrapper[5052]: I1203 12:07:38.088250 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2066fe4a-e7cc-44a5-b7d9-499d19a40c12-config-data" (OuterVolumeSpecName: "config-data") pod "2066fe4a-e7cc-44a5-b7d9-499d19a40c12" (UID: "2066fe4a-e7cc-44a5-b7d9-499d19a40c12"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:07:38 crc kubenswrapper[5052]: I1203 12:07:38.151087 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/2066fe4a-e7cc-44a5-b7d9-499d19a40c12-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:07:38 crc kubenswrapper[5052]: I1203 12:07:38.151198 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-6b97m\" (UniqueName: \"kubernetes.io/projected/2066fe4a-e7cc-44a5-b7d9-499d19a40c12-kube-api-access-6b97m\") on node \"crc\" DevicePath \"\"" Dec 03 12:07:38 crc kubenswrapper[5052]: I1203 12:07:38.151214 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2066fe4a-e7cc-44a5-b7d9-499d19a40c12-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:07:38 crc kubenswrapper[5052]: I1203 12:07:38.151224 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/2066fe4a-e7cc-44a5-b7d9-499d19a40c12-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:07:38 crc kubenswrapper[5052]: I1203 12:07:38.534275 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"6c9d4db1-a93f-4624-a6e1-d3a31285d74f","Type":"ContainerStarted","Data":"ead485560d4f65efe1542f65c009ca06bccbffe91f339143855a352039c0bcfc"} Dec 03 12:07:38 crc kubenswrapper[5052]: I1203 12:07:38.536688 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-sxlxt" event={"ID":"2066fe4a-e7cc-44a5-b7d9-499d19a40c12","Type":"ContainerDied","Data":"cd81b7a4a00c975a4914d128b35e8aa28ba1452fc6b882208d8fc5c05e72e3f8"} Dec 03 12:07:38 crc kubenswrapper[5052]: I1203 12:07:38.536744 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cd81b7a4a00c975a4914d128b35e8aa28ba1452fc6b882208d8fc5c05e72e3f8" Dec 03 12:07:38 crc kubenswrapper[5052]: I1203 12:07:38.536748 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-sxlxt" Dec 03 12:07:38 crc kubenswrapper[5052]: I1203 12:07:38.608373 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-0"] Dec 03 12:07:38 crc kubenswrapper[5052]: E1203 12:07:38.608993 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="2066fe4a-e7cc-44a5-b7d9-499d19a40c12" containerName="nova-cell1-conductor-db-sync" Dec 03 12:07:38 crc kubenswrapper[5052]: I1203 12:07:38.609014 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="2066fe4a-e7cc-44a5-b7d9-499d19a40c12" containerName="nova-cell1-conductor-db-sync" Dec 03 12:07:38 crc kubenswrapper[5052]: I1203 12:07:38.609306 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="2066fe4a-e7cc-44a5-b7d9-499d19a40c12" containerName="nova-cell1-conductor-db-sync" Dec 03 12:07:38 crc kubenswrapper[5052]: I1203 12:07:38.610461 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:07:38 crc kubenswrapper[5052]: I1203 12:07:38.613944 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell1-conductor-config-data" Dec 03 12:07:38 crc kubenswrapper[5052]: I1203 12:07:38.618554 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-0"] Dec 03 12:07:38 crc kubenswrapper[5052]: I1203 12:07:38.765747 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/e92fa4d7-9bd6-4586-b41a-a557fb672439-config-data\") pod \"nova-cell1-conductor-0\" (UID: \"e92fa4d7-9bd6-4586-b41a-a557fb672439\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:07:38 crc kubenswrapper[5052]: I1203 12:07:38.765868 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dd67s\" (UniqueName: \"kubernetes.io/projected/e92fa4d7-9bd6-4586-b41a-a557fb672439-kube-api-access-dd67s\") pod \"nova-cell1-conductor-0\" (UID: \"e92fa4d7-9bd6-4586-b41a-a557fb672439\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:07:38 crc kubenswrapper[5052]: I1203 12:07:38.765920 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e92fa4d7-9bd6-4586-b41a-a557fb672439-combined-ca-bundle\") pod \"nova-cell1-conductor-0\" (UID: \"e92fa4d7-9bd6-4586-b41a-a557fb672439\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:07:38 crc kubenswrapper[5052]: I1203 12:07:38.867805 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/e92fa4d7-9bd6-4586-b41a-a557fb672439-config-data\") pod \"nova-cell1-conductor-0\" (UID: \"e92fa4d7-9bd6-4586-b41a-a557fb672439\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:07:38 crc kubenswrapper[5052]: I1203 12:07:38.867898 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-dd67s\" (UniqueName: \"kubernetes.io/projected/e92fa4d7-9bd6-4586-b41a-a557fb672439-kube-api-access-dd67s\") pod \"nova-cell1-conductor-0\" (UID: \"e92fa4d7-9bd6-4586-b41a-a557fb672439\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:07:38 crc kubenswrapper[5052]: I1203 12:07:38.867955 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e92fa4d7-9bd6-4586-b41a-a557fb672439-combined-ca-bundle\") pod \"nova-cell1-conductor-0\" (UID: \"e92fa4d7-9bd6-4586-b41a-a557fb672439\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:07:38 crc kubenswrapper[5052]: I1203 12:07:38.876083 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/e92fa4d7-9bd6-4586-b41a-a557fb672439-config-data\") pod \"nova-cell1-conductor-0\" (UID: \"e92fa4d7-9bd6-4586-b41a-a557fb672439\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:07:38 crc kubenswrapper[5052]: I1203 12:07:38.882807 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e92fa4d7-9bd6-4586-b41a-a557fb672439-combined-ca-bundle\") pod \"nova-cell1-conductor-0\" (UID: \"e92fa4d7-9bd6-4586-b41a-a557fb672439\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:07:38 crc kubenswrapper[5052]: I1203 12:07:38.888886 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-dd67s\" (UniqueName: \"kubernetes.io/projected/e92fa4d7-9bd6-4586-b41a-a557fb672439-kube-api-access-dd67s\") pod \"nova-cell1-conductor-0\" (UID: \"e92fa4d7-9bd6-4586-b41a-a557fb672439\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:07:38 crc kubenswrapper[5052]: I1203 12:07:38.952638 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:07:39 crc kubenswrapper[5052]: I1203 12:07:39.492598 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-0"] Dec 03 12:07:39 crc kubenswrapper[5052]: I1203 12:07:39.550668 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-conductor-0" event={"ID":"e92fa4d7-9bd6-4586-b41a-a557fb672439","Type":"ContainerStarted","Data":"606c5f9fc4d72d003134f9303e7eb72bdb86f38ae1c336683de7950b82e20c36"} Dec 03 12:07:39 crc kubenswrapper[5052]: I1203 12:07:39.688942 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:07:39 crc kubenswrapper[5052]: I1203 12:07:39.689581 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:07:39 crc kubenswrapper[5052]: I1203 12:07:39.699450 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:07:39 crc kubenswrapper[5052]: I1203 12:07:39.718270 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:07:40 crc kubenswrapper[5052]: I1203 12:07:40.569437 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"6c9d4db1-a93f-4624-a6e1-d3a31285d74f","Type":"ContainerStarted","Data":"af09b05dfff50c7e8a096546e5b0af12b3a12a1c2a6ddc7910ef8be50cc05e03"} Dec 03 12:07:40 crc kubenswrapper[5052]: I1203 12:07:40.570082 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:07:40 crc kubenswrapper[5052]: I1203 12:07:40.575252 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-conductor-0" event={"ID":"e92fa4d7-9bd6-4586-b41a-a557fb672439","Type":"ContainerStarted","Data":"8709692766b7010f69ca646129fa46d2b1115bc2a7f05706db51e69fa086eafa"} Dec 03 12:07:40 crc kubenswrapper[5052]: I1203 12:07:40.575305 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:07:40 crc kubenswrapper[5052]: I1203 12:07:40.575392 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:07:40 crc kubenswrapper[5052]: I1203 12:07:40.583869 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:07:40 crc kubenswrapper[5052]: I1203 12:07:40.618611 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/ceilometer-0" podStartSLOduration=2.755800484 podStartE2EDuration="6.618579888s" podCreationTimestamp="2025-12-03 12:07:34 +0000 UTC" firstStartedPulling="2025-12-03 12:07:35.687004167 +0000 UTC m=+2846.275245931" lastFinishedPulling="2025-12-03 12:07:39.549783571 +0000 UTC m=+2850.138025335" observedRunningTime="2025-12-03 12:07:40.600684446 +0000 UTC m=+2851.188926230" watchObservedRunningTime="2025-12-03 12:07:40.618579888 +0000 UTC m=+2851.206821652" Dec 03 12:07:40 crc kubenswrapper[5052]: I1203 12:07:40.624794 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-cell1-conductor-0" podStartSLOduration=2.624763115 podStartE2EDuration="2.624763115s" podCreationTimestamp="2025-12-03 12:07:38 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:07:40.62075963 +0000 UTC m=+2851.209001414" watchObservedRunningTime="2025-12-03 12:07:40.624763115 +0000 UTC m=+2851.213004889" Dec 03 12:07:41 crc kubenswrapper[5052]: I1203 12:07:41.858283 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:07:43 crc kubenswrapper[5052]: I1203 12:07:43.387057 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:07:43 crc kubenswrapper[5052]: I1203 12:07:43.387685 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="6c9d4db1-a93f-4624-a6e1-d3a31285d74f" containerName="ceilometer-central-agent" containerID="cri-o://660744ff88d9dd18080ad97ca00100247da7f8bd7e7a74352c8cea4db0046533" gracePeriod=30 Dec 03 12:07:43 crc kubenswrapper[5052]: I1203 12:07:43.387751 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="6c9d4db1-a93f-4624-a6e1-d3a31285d74f" containerName="proxy-httpd" containerID="cri-o://af09b05dfff50c7e8a096546e5b0af12b3a12a1c2a6ddc7910ef8be50cc05e03" gracePeriod=30 Dec 03 12:07:43 crc kubenswrapper[5052]: I1203 12:07:43.387767 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="6c9d4db1-a93f-4624-a6e1-d3a31285d74f" containerName="ceilometer-notification-agent" containerID="cri-o://10cd1c0b34caba4bc9ff8ab8584ffb6af47033dc0321fe224635ffd1061e6c3c" gracePeriod=30 Dec 03 12:07:43 crc kubenswrapper[5052]: I1203 12:07:43.387926 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="6c9d4db1-a93f-4624-a6e1-d3a31285d74f" containerName="sg-core" containerID="cri-o://ead485560d4f65efe1542f65c009ca06bccbffe91f339143855a352039c0bcfc" gracePeriod=30 Dec 03 12:07:43 crc kubenswrapper[5052]: I1203 12:07:43.630708 5052 generic.go:334] "Generic (PLEG): container finished" podID="6c9d4db1-a93f-4624-a6e1-d3a31285d74f" containerID="af09b05dfff50c7e8a096546e5b0af12b3a12a1c2a6ddc7910ef8be50cc05e03" exitCode=0 Dec 03 12:07:43 crc kubenswrapper[5052]: I1203 12:07:43.631210 5052 generic.go:334] "Generic (PLEG): container finished" podID="6c9d4db1-a93f-4624-a6e1-d3a31285d74f" containerID="ead485560d4f65efe1542f65c009ca06bccbffe91f339143855a352039c0bcfc" exitCode=2 Dec 03 12:07:43 crc kubenswrapper[5052]: I1203 12:07:43.630786 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"6c9d4db1-a93f-4624-a6e1-d3a31285d74f","Type":"ContainerDied","Data":"af09b05dfff50c7e8a096546e5b0af12b3a12a1c2a6ddc7910ef8be50cc05e03"} Dec 03 12:07:43 crc kubenswrapper[5052]: I1203 12:07:43.631365 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"6c9d4db1-a93f-4624-a6e1-d3a31285d74f","Type":"ContainerDied","Data":"ead485560d4f65efe1542f65c009ca06bccbffe91f339143855a352039c0bcfc"} Dec 03 12:07:43 crc kubenswrapper[5052]: I1203 12:07:43.754653 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:07:43 crc kubenswrapper[5052]: I1203 12:07:43.755120 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-api-0" podUID="ff9eaa72-6d5d-48a9-85a5-362de3895ae1" containerName="nova-api-log" containerID="cri-o://7724519ba83d2fb0f80b9e295f1c21f87d8043850d773806c593cd8e175c80ad" gracePeriod=30 Dec 03 12:07:43 crc kubenswrapper[5052]: I1203 12:07:43.755375 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-api-0" podUID="ff9eaa72-6d5d-48a9-85a5-362de3895ae1" containerName="nova-api-api" containerID="cri-o://ffc10685c0afd2a841a5fca1660a8031d066ef01fd68e5278b19f1ed0ca8af9a" gracePeriod=30 Dec 03 12:07:44 crc kubenswrapper[5052]: I1203 12:07:44.644431 5052 generic.go:334] "Generic (PLEG): container finished" podID="ff9eaa72-6d5d-48a9-85a5-362de3895ae1" containerID="7724519ba83d2fb0f80b9e295f1c21f87d8043850d773806c593cd8e175c80ad" exitCode=143 Dec 03 12:07:44 crc kubenswrapper[5052]: I1203 12:07:44.644550 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"ff9eaa72-6d5d-48a9-85a5-362de3895ae1","Type":"ContainerDied","Data":"7724519ba83d2fb0f80b9e295f1c21f87d8043850d773806c593cd8e175c80ad"} Dec 03 12:07:44 crc kubenswrapper[5052]: I1203 12:07:44.646710 5052 generic.go:334] "Generic (PLEG): container finished" podID="6c9d4db1-a93f-4624-a6e1-d3a31285d74f" containerID="10cd1c0b34caba4bc9ff8ab8584ffb6af47033dc0321fe224635ffd1061e6c3c" exitCode=0 Dec 03 12:07:44 crc kubenswrapper[5052]: I1203 12:07:44.646741 5052 generic.go:334] "Generic (PLEG): container finished" podID="6c9d4db1-a93f-4624-a6e1-d3a31285d74f" containerID="660744ff88d9dd18080ad97ca00100247da7f8bd7e7a74352c8cea4db0046533" exitCode=0 Dec 03 12:07:44 crc kubenswrapper[5052]: I1203 12:07:44.646759 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"6c9d4db1-a93f-4624-a6e1-d3a31285d74f","Type":"ContainerDied","Data":"10cd1c0b34caba4bc9ff8ab8584ffb6af47033dc0321fe224635ffd1061e6c3c"} Dec 03 12:07:44 crc kubenswrapper[5052]: I1203 12:07:44.646786 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"6c9d4db1-a93f-4624-a6e1-d3a31285d74f","Type":"ContainerDied","Data":"660744ff88d9dd18080ad97ca00100247da7f8bd7e7a74352c8cea4db0046533"} Dec 03 12:07:45 crc kubenswrapper[5052]: I1203 12:07:45.049976 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:07:45 crc kubenswrapper[5052]: I1203 12:07:45.211158 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/6c9d4db1-a93f-4624-a6e1-d3a31285d74f-scripts\") pod \"6c9d4db1-a93f-4624-a6e1-d3a31285d74f\" (UID: \"6c9d4db1-a93f-4624-a6e1-d3a31285d74f\") " Dec 03 12:07:45 crc kubenswrapper[5052]: I1203 12:07:45.211414 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6c9d4db1-a93f-4624-a6e1-d3a31285d74f-combined-ca-bundle\") pod \"6c9d4db1-a93f-4624-a6e1-d3a31285d74f\" (UID: \"6c9d4db1-a93f-4624-a6e1-d3a31285d74f\") " Dec 03 12:07:45 crc kubenswrapper[5052]: I1203 12:07:45.211442 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dtsh2\" (UniqueName: \"kubernetes.io/projected/6c9d4db1-a93f-4624-a6e1-d3a31285d74f-kube-api-access-dtsh2\") pod \"6c9d4db1-a93f-4624-a6e1-d3a31285d74f\" (UID: \"6c9d4db1-a93f-4624-a6e1-d3a31285d74f\") " Dec 03 12:07:45 crc kubenswrapper[5052]: I1203 12:07:45.211488 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/6c9d4db1-a93f-4624-a6e1-d3a31285d74f-ceilometer-tls-certs\") pod \"6c9d4db1-a93f-4624-a6e1-d3a31285d74f\" (UID: \"6c9d4db1-a93f-4624-a6e1-d3a31285d74f\") " Dec 03 12:07:45 crc kubenswrapper[5052]: I1203 12:07:45.211533 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/6c9d4db1-a93f-4624-a6e1-d3a31285d74f-log-httpd\") pod \"6c9d4db1-a93f-4624-a6e1-d3a31285d74f\" (UID: \"6c9d4db1-a93f-4624-a6e1-d3a31285d74f\") " Dec 03 12:07:45 crc kubenswrapper[5052]: I1203 12:07:45.211552 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6c9d4db1-a93f-4624-a6e1-d3a31285d74f-config-data\") pod \"6c9d4db1-a93f-4624-a6e1-d3a31285d74f\" (UID: \"6c9d4db1-a93f-4624-a6e1-d3a31285d74f\") " Dec 03 12:07:45 crc kubenswrapper[5052]: I1203 12:07:45.211600 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/6c9d4db1-a93f-4624-a6e1-d3a31285d74f-run-httpd\") pod \"6c9d4db1-a93f-4624-a6e1-d3a31285d74f\" (UID: \"6c9d4db1-a93f-4624-a6e1-d3a31285d74f\") " Dec 03 12:07:45 crc kubenswrapper[5052]: I1203 12:07:45.211626 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/6c9d4db1-a93f-4624-a6e1-d3a31285d74f-sg-core-conf-yaml\") pod \"6c9d4db1-a93f-4624-a6e1-d3a31285d74f\" (UID: \"6c9d4db1-a93f-4624-a6e1-d3a31285d74f\") " Dec 03 12:07:45 crc kubenswrapper[5052]: I1203 12:07:45.212001 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6c9d4db1-a93f-4624-a6e1-d3a31285d74f-run-httpd" (OuterVolumeSpecName: "run-httpd") pod "6c9d4db1-a93f-4624-a6e1-d3a31285d74f" (UID: "6c9d4db1-a93f-4624-a6e1-d3a31285d74f"). InnerVolumeSpecName "run-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:07:45 crc kubenswrapper[5052]: I1203 12:07:45.212049 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6c9d4db1-a93f-4624-a6e1-d3a31285d74f-log-httpd" (OuterVolumeSpecName: "log-httpd") pod "6c9d4db1-a93f-4624-a6e1-d3a31285d74f" (UID: "6c9d4db1-a93f-4624-a6e1-d3a31285d74f"). InnerVolumeSpecName "log-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:07:45 crc kubenswrapper[5052]: I1203 12:07:45.213012 5052 reconciler_common.go:293] "Volume detached for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/6c9d4db1-a93f-4624-a6e1-d3a31285d74f-log-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:07:45 crc kubenswrapper[5052]: I1203 12:07:45.213031 5052 reconciler_common.go:293] "Volume detached for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/6c9d4db1-a93f-4624-a6e1-d3a31285d74f-run-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:07:45 crc kubenswrapper[5052]: I1203 12:07:45.238766 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6c9d4db1-a93f-4624-a6e1-d3a31285d74f-scripts" (OuterVolumeSpecName: "scripts") pod "6c9d4db1-a93f-4624-a6e1-d3a31285d74f" (UID: "6c9d4db1-a93f-4624-a6e1-d3a31285d74f"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:07:45 crc kubenswrapper[5052]: I1203 12:07:45.242117 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6c9d4db1-a93f-4624-a6e1-d3a31285d74f-kube-api-access-dtsh2" (OuterVolumeSpecName: "kube-api-access-dtsh2") pod "6c9d4db1-a93f-4624-a6e1-d3a31285d74f" (UID: "6c9d4db1-a93f-4624-a6e1-d3a31285d74f"). InnerVolumeSpecName "kube-api-access-dtsh2". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:07:45 crc kubenswrapper[5052]: I1203 12:07:45.269045 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6c9d4db1-a93f-4624-a6e1-d3a31285d74f-sg-core-conf-yaml" (OuterVolumeSpecName: "sg-core-conf-yaml") pod "6c9d4db1-a93f-4624-a6e1-d3a31285d74f" (UID: "6c9d4db1-a93f-4624-a6e1-d3a31285d74f"). InnerVolumeSpecName "sg-core-conf-yaml". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:07:45 crc kubenswrapper[5052]: I1203 12:07:45.314719 5052 reconciler_common.go:293] "Volume detached for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/6c9d4db1-a93f-4624-a6e1-d3a31285d74f-sg-core-conf-yaml\") on node \"crc\" DevicePath \"\"" Dec 03 12:07:45 crc kubenswrapper[5052]: I1203 12:07:45.314760 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/6c9d4db1-a93f-4624-a6e1-d3a31285d74f-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:07:45 crc kubenswrapper[5052]: I1203 12:07:45.314774 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-dtsh2\" (UniqueName: \"kubernetes.io/projected/6c9d4db1-a93f-4624-a6e1-d3a31285d74f-kube-api-access-dtsh2\") on node \"crc\" DevicePath \"\"" Dec 03 12:07:45 crc kubenswrapper[5052]: I1203 12:07:45.354730 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6c9d4db1-a93f-4624-a6e1-d3a31285d74f-ceilometer-tls-certs" (OuterVolumeSpecName: "ceilometer-tls-certs") pod "6c9d4db1-a93f-4624-a6e1-d3a31285d74f" (UID: "6c9d4db1-a93f-4624-a6e1-d3a31285d74f"). InnerVolumeSpecName "ceilometer-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:07:45 crc kubenswrapper[5052]: I1203 12:07:45.388674 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6c9d4db1-a93f-4624-a6e1-d3a31285d74f-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "6c9d4db1-a93f-4624-a6e1-d3a31285d74f" (UID: "6c9d4db1-a93f-4624-a6e1-d3a31285d74f"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:07:45 crc kubenswrapper[5052]: I1203 12:07:45.417854 5052 reconciler_common.go:293] "Volume detached for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/6c9d4db1-a93f-4624-a6e1-d3a31285d74f-ceilometer-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:07:45 crc kubenswrapper[5052]: I1203 12:07:45.418151 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6c9d4db1-a93f-4624-a6e1-d3a31285d74f-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:07:45 crc kubenswrapper[5052]: I1203 12:07:45.436480 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6c9d4db1-a93f-4624-a6e1-d3a31285d74f-config-data" (OuterVolumeSpecName: "config-data") pod "6c9d4db1-a93f-4624-a6e1-d3a31285d74f" (UID: "6c9d4db1-a93f-4624-a6e1-d3a31285d74f"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:07:45 crc kubenswrapper[5052]: I1203 12:07:45.520300 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6c9d4db1-a93f-4624-a6e1-d3a31285d74f-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:07:45 crc kubenswrapper[5052]: I1203 12:07:45.675905 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:07:45 crc kubenswrapper[5052]: I1203 12:07:45.690308 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"6c9d4db1-a93f-4624-a6e1-d3a31285d74f","Type":"ContainerDied","Data":"bbd05716e18f09f08d9e492a9aa982871f57b0fdae64a3b5e90400e6e6b0d021"} Dec 03 12:07:45 crc kubenswrapper[5052]: I1203 12:07:45.694163 5052 scope.go:117] "RemoveContainer" containerID="af09b05dfff50c7e8a096546e5b0af12b3a12a1c2a6ddc7910ef8be50cc05e03" Dec 03 12:07:45 crc kubenswrapper[5052]: I1203 12:07:45.773533 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:07:45 crc kubenswrapper[5052]: I1203 12:07:45.774964 5052 scope.go:117] "RemoveContainer" containerID="ead485560d4f65efe1542f65c009ca06bccbffe91f339143855a352039c0bcfc" Dec 03 12:07:45 crc kubenswrapper[5052]: I1203 12:07:45.803132 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:07:45 crc kubenswrapper[5052]: I1203 12:07:45.823754 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:07:45 crc kubenswrapper[5052]: E1203 12:07:45.824995 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="6c9d4db1-a93f-4624-a6e1-d3a31285d74f" containerName="ceilometer-notification-agent" Dec 03 12:07:45 crc kubenswrapper[5052]: I1203 12:07:45.825084 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="6c9d4db1-a93f-4624-a6e1-d3a31285d74f" containerName="ceilometer-notification-agent" Dec 03 12:07:45 crc kubenswrapper[5052]: E1203 12:07:45.825158 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="6c9d4db1-a93f-4624-a6e1-d3a31285d74f" containerName="ceilometer-central-agent" Dec 03 12:07:45 crc kubenswrapper[5052]: I1203 12:07:45.825958 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="6c9d4db1-a93f-4624-a6e1-d3a31285d74f" containerName="ceilometer-central-agent" Dec 03 12:07:45 crc kubenswrapper[5052]: E1203 12:07:45.826064 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="6c9d4db1-a93f-4624-a6e1-d3a31285d74f" containerName="sg-core" Dec 03 12:07:45 crc kubenswrapper[5052]: I1203 12:07:45.826134 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="6c9d4db1-a93f-4624-a6e1-d3a31285d74f" containerName="sg-core" Dec 03 12:07:45 crc kubenswrapper[5052]: E1203 12:07:45.826247 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="6c9d4db1-a93f-4624-a6e1-d3a31285d74f" containerName="proxy-httpd" Dec 03 12:07:45 crc kubenswrapper[5052]: I1203 12:07:45.826332 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="6c9d4db1-a93f-4624-a6e1-d3a31285d74f" containerName="proxy-httpd" Dec 03 12:07:45 crc kubenswrapper[5052]: I1203 12:07:45.826779 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="6c9d4db1-a93f-4624-a6e1-d3a31285d74f" containerName="ceilometer-central-agent" Dec 03 12:07:45 crc kubenswrapper[5052]: I1203 12:07:45.826882 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="6c9d4db1-a93f-4624-a6e1-d3a31285d74f" containerName="sg-core" Dec 03 12:07:45 crc kubenswrapper[5052]: I1203 12:07:45.826971 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="6c9d4db1-a93f-4624-a6e1-d3a31285d74f" containerName="ceilometer-notification-agent" Dec 03 12:07:45 crc kubenswrapper[5052]: I1203 12:07:45.827067 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="6c9d4db1-a93f-4624-a6e1-d3a31285d74f" containerName="proxy-httpd" Dec 03 12:07:45 crc kubenswrapper[5052]: I1203 12:07:45.832945 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:07:45 crc kubenswrapper[5052]: I1203 12:07:45.842136 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-ceilometer-internal-svc" Dec 03 12:07:45 crc kubenswrapper[5052]: I1203 12:07:45.842603 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-scripts" Dec 03 12:07:45 crc kubenswrapper[5052]: I1203 12:07:45.842789 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-config-data" Dec 03 12:07:45 crc kubenswrapper[5052]: I1203 12:07:45.845247 5052 scope.go:117] "RemoveContainer" containerID="10cd1c0b34caba4bc9ff8ab8584ffb6af47033dc0321fe224635ffd1061e6c3c" Dec 03 12:07:45 crc kubenswrapper[5052]: I1203 12:07:45.852754 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:07:45 crc kubenswrapper[5052]: I1203 12:07:45.890767 5052 scope.go:117] "RemoveContainer" containerID="660744ff88d9dd18080ad97ca00100247da7f8bd7e7a74352c8cea4db0046533" Dec 03 12:07:45 crc kubenswrapper[5052]: I1203 12:07:45.957784 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/6edd826b-b9ae-4e97-90be-8bc4954a1a77-log-httpd\") pod \"ceilometer-0\" (UID: \"6edd826b-b9ae-4e97-90be-8bc4954a1a77\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:07:45 crc kubenswrapper[5052]: I1203 12:07:45.957852 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/6edd826b-b9ae-4e97-90be-8bc4954a1a77-run-httpd\") pod \"ceilometer-0\" (UID: \"6edd826b-b9ae-4e97-90be-8bc4954a1a77\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:07:45 crc kubenswrapper[5052]: I1203 12:07:45.957886 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-njm9x\" (UniqueName: \"kubernetes.io/projected/6edd826b-b9ae-4e97-90be-8bc4954a1a77-kube-api-access-njm9x\") pod \"ceilometer-0\" (UID: \"6edd826b-b9ae-4e97-90be-8bc4954a1a77\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:07:45 crc kubenswrapper[5052]: I1203 12:07:45.957905 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6edd826b-b9ae-4e97-90be-8bc4954a1a77-config-data\") pod \"ceilometer-0\" (UID: \"6edd826b-b9ae-4e97-90be-8bc4954a1a77\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:07:45 crc kubenswrapper[5052]: I1203 12:07:45.958045 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/6edd826b-b9ae-4e97-90be-8bc4954a1a77-scripts\") pod \"ceilometer-0\" (UID: \"6edd826b-b9ae-4e97-90be-8bc4954a1a77\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:07:45 crc kubenswrapper[5052]: I1203 12:07:45.958162 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/6edd826b-b9ae-4e97-90be-8bc4954a1a77-ceilometer-tls-certs\") pod \"ceilometer-0\" (UID: \"6edd826b-b9ae-4e97-90be-8bc4954a1a77\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:07:45 crc kubenswrapper[5052]: I1203 12:07:45.958269 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6edd826b-b9ae-4e97-90be-8bc4954a1a77-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"6edd826b-b9ae-4e97-90be-8bc4954a1a77\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:07:45 crc kubenswrapper[5052]: I1203 12:07:45.958366 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/6edd826b-b9ae-4e97-90be-8bc4954a1a77-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"6edd826b-b9ae-4e97-90be-8bc4954a1a77\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:07:46 crc kubenswrapper[5052]: I1203 12:07:46.010332 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:07:46 crc kubenswrapper[5052]: E1203 12:07:46.011086 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[ceilometer-tls-certs combined-ca-bundle config-data kube-api-access-njm9x log-httpd run-httpd scripts sg-core-conf-yaml], unattached volumes=[], failed to process volumes=[]: context canceled" pod="openstack-kuttl-tests/ceilometer-0" podUID="6edd826b-b9ae-4e97-90be-8bc4954a1a77" Dec 03 12:07:46 crc kubenswrapper[5052]: I1203 12:07:46.060841 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/6edd826b-b9ae-4e97-90be-8bc4954a1a77-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"6edd826b-b9ae-4e97-90be-8bc4954a1a77\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:07:46 crc kubenswrapper[5052]: I1203 12:07:46.060949 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/6edd826b-b9ae-4e97-90be-8bc4954a1a77-log-httpd\") pod \"ceilometer-0\" (UID: \"6edd826b-b9ae-4e97-90be-8bc4954a1a77\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:07:46 crc kubenswrapper[5052]: I1203 12:07:46.061005 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/6edd826b-b9ae-4e97-90be-8bc4954a1a77-run-httpd\") pod \"ceilometer-0\" (UID: \"6edd826b-b9ae-4e97-90be-8bc4954a1a77\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:07:46 crc kubenswrapper[5052]: I1203 12:07:46.061672 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/6edd826b-b9ae-4e97-90be-8bc4954a1a77-run-httpd\") pod \"ceilometer-0\" (UID: \"6edd826b-b9ae-4e97-90be-8bc4954a1a77\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:07:46 crc kubenswrapper[5052]: I1203 12:07:46.061048 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-njm9x\" (UniqueName: \"kubernetes.io/projected/6edd826b-b9ae-4e97-90be-8bc4954a1a77-kube-api-access-njm9x\") pod \"ceilometer-0\" (UID: \"6edd826b-b9ae-4e97-90be-8bc4954a1a77\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:07:46 crc kubenswrapper[5052]: I1203 12:07:46.061767 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6edd826b-b9ae-4e97-90be-8bc4954a1a77-config-data\") pod \"ceilometer-0\" (UID: \"6edd826b-b9ae-4e97-90be-8bc4954a1a77\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:07:46 crc kubenswrapper[5052]: I1203 12:07:46.061768 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/6edd826b-b9ae-4e97-90be-8bc4954a1a77-log-httpd\") pod \"ceilometer-0\" (UID: \"6edd826b-b9ae-4e97-90be-8bc4954a1a77\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:07:46 crc kubenswrapper[5052]: I1203 12:07:46.061881 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/6edd826b-b9ae-4e97-90be-8bc4954a1a77-scripts\") pod \"ceilometer-0\" (UID: \"6edd826b-b9ae-4e97-90be-8bc4954a1a77\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:07:46 crc kubenswrapper[5052]: I1203 12:07:46.061992 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/6edd826b-b9ae-4e97-90be-8bc4954a1a77-ceilometer-tls-certs\") pod \"ceilometer-0\" (UID: \"6edd826b-b9ae-4e97-90be-8bc4954a1a77\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:07:46 crc kubenswrapper[5052]: I1203 12:07:46.062111 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6edd826b-b9ae-4e97-90be-8bc4954a1a77-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"6edd826b-b9ae-4e97-90be-8bc4954a1a77\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:07:46 crc kubenswrapper[5052]: I1203 12:07:46.068559 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/6edd826b-b9ae-4e97-90be-8bc4954a1a77-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"6edd826b-b9ae-4e97-90be-8bc4954a1a77\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:07:46 crc kubenswrapper[5052]: I1203 12:07:46.069207 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6edd826b-b9ae-4e97-90be-8bc4954a1a77-config-data\") pod \"ceilometer-0\" (UID: \"6edd826b-b9ae-4e97-90be-8bc4954a1a77\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:07:46 crc kubenswrapper[5052]: I1203 12:07:46.069356 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6edd826b-b9ae-4e97-90be-8bc4954a1a77-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"6edd826b-b9ae-4e97-90be-8bc4954a1a77\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:07:46 crc kubenswrapper[5052]: I1203 12:07:46.069356 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/6edd826b-b9ae-4e97-90be-8bc4954a1a77-ceilometer-tls-certs\") pod \"ceilometer-0\" (UID: \"6edd826b-b9ae-4e97-90be-8bc4954a1a77\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:07:46 crc kubenswrapper[5052]: I1203 12:07:46.084286 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/6edd826b-b9ae-4e97-90be-8bc4954a1a77-scripts\") pod \"ceilometer-0\" (UID: \"6edd826b-b9ae-4e97-90be-8bc4954a1a77\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:07:46 crc kubenswrapper[5052]: I1203 12:07:46.087271 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-njm9x\" (UniqueName: \"kubernetes.io/projected/6edd826b-b9ae-4e97-90be-8bc4954a1a77-kube-api-access-njm9x\") pod \"ceilometer-0\" (UID: \"6edd826b-b9ae-4e97-90be-8bc4954a1a77\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:07:46 crc kubenswrapper[5052]: I1203 12:07:46.685474 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:07:46 crc kubenswrapper[5052]: I1203 12:07:46.698329 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:07:46 crc kubenswrapper[5052]: I1203 12:07:46.775129 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/6edd826b-b9ae-4e97-90be-8bc4954a1a77-sg-core-conf-yaml\") pod \"6edd826b-b9ae-4e97-90be-8bc4954a1a77\" (UID: \"6edd826b-b9ae-4e97-90be-8bc4954a1a77\") " Dec 03 12:07:46 crc kubenswrapper[5052]: I1203 12:07:46.775254 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6edd826b-b9ae-4e97-90be-8bc4954a1a77-config-data\") pod \"6edd826b-b9ae-4e97-90be-8bc4954a1a77\" (UID: \"6edd826b-b9ae-4e97-90be-8bc4954a1a77\") " Dec 03 12:07:46 crc kubenswrapper[5052]: I1203 12:07:46.775330 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/6edd826b-b9ae-4e97-90be-8bc4954a1a77-ceilometer-tls-certs\") pod \"6edd826b-b9ae-4e97-90be-8bc4954a1a77\" (UID: \"6edd826b-b9ae-4e97-90be-8bc4954a1a77\") " Dec 03 12:07:46 crc kubenswrapper[5052]: I1203 12:07:46.775362 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/6edd826b-b9ae-4e97-90be-8bc4954a1a77-scripts\") pod \"6edd826b-b9ae-4e97-90be-8bc4954a1a77\" (UID: \"6edd826b-b9ae-4e97-90be-8bc4954a1a77\") " Dec 03 12:07:46 crc kubenswrapper[5052]: I1203 12:07:46.775385 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6edd826b-b9ae-4e97-90be-8bc4954a1a77-combined-ca-bundle\") pod \"6edd826b-b9ae-4e97-90be-8bc4954a1a77\" (UID: \"6edd826b-b9ae-4e97-90be-8bc4954a1a77\") " Dec 03 12:07:46 crc kubenswrapper[5052]: I1203 12:07:46.775443 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/6edd826b-b9ae-4e97-90be-8bc4954a1a77-log-httpd\") pod \"6edd826b-b9ae-4e97-90be-8bc4954a1a77\" (UID: \"6edd826b-b9ae-4e97-90be-8bc4954a1a77\") " Dec 03 12:07:46 crc kubenswrapper[5052]: I1203 12:07:46.775728 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/6edd826b-b9ae-4e97-90be-8bc4954a1a77-run-httpd\") pod \"6edd826b-b9ae-4e97-90be-8bc4954a1a77\" (UID: \"6edd826b-b9ae-4e97-90be-8bc4954a1a77\") " Dec 03 12:07:46 crc kubenswrapper[5052]: I1203 12:07:46.775763 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-njm9x\" (UniqueName: \"kubernetes.io/projected/6edd826b-b9ae-4e97-90be-8bc4954a1a77-kube-api-access-njm9x\") pod \"6edd826b-b9ae-4e97-90be-8bc4954a1a77\" (UID: \"6edd826b-b9ae-4e97-90be-8bc4954a1a77\") " Dec 03 12:07:46 crc kubenswrapper[5052]: I1203 12:07:46.776043 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6edd826b-b9ae-4e97-90be-8bc4954a1a77-log-httpd" (OuterVolumeSpecName: "log-httpd") pod "6edd826b-b9ae-4e97-90be-8bc4954a1a77" (UID: "6edd826b-b9ae-4e97-90be-8bc4954a1a77"). InnerVolumeSpecName "log-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:07:46 crc kubenswrapper[5052]: I1203 12:07:46.776118 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6edd826b-b9ae-4e97-90be-8bc4954a1a77-run-httpd" (OuterVolumeSpecName: "run-httpd") pod "6edd826b-b9ae-4e97-90be-8bc4954a1a77" (UID: "6edd826b-b9ae-4e97-90be-8bc4954a1a77"). InnerVolumeSpecName "run-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:07:46 crc kubenswrapper[5052]: I1203 12:07:46.778961 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6edd826b-b9ae-4e97-90be-8bc4954a1a77-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "6edd826b-b9ae-4e97-90be-8bc4954a1a77" (UID: "6edd826b-b9ae-4e97-90be-8bc4954a1a77"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:07:46 crc kubenswrapper[5052]: I1203 12:07:46.780332 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6edd826b-b9ae-4e97-90be-8bc4954a1a77-kube-api-access-njm9x" (OuterVolumeSpecName: "kube-api-access-njm9x") pod "6edd826b-b9ae-4e97-90be-8bc4954a1a77" (UID: "6edd826b-b9ae-4e97-90be-8bc4954a1a77"). InnerVolumeSpecName "kube-api-access-njm9x". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:07:46 crc kubenswrapper[5052]: I1203 12:07:46.780778 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6edd826b-b9ae-4e97-90be-8bc4954a1a77-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:07:46 crc kubenswrapper[5052]: I1203 12:07:46.780805 5052 reconciler_common.go:293] "Volume detached for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/6edd826b-b9ae-4e97-90be-8bc4954a1a77-log-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:07:46 crc kubenswrapper[5052]: I1203 12:07:46.780816 5052 reconciler_common.go:293] "Volume detached for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/6edd826b-b9ae-4e97-90be-8bc4954a1a77-run-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:07:46 crc kubenswrapper[5052]: I1203 12:07:46.780826 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-njm9x\" (UniqueName: \"kubernetes.io/projected/6edd826b-b9ae-4e97-90be-8bc4954a1a77-kube-api-access-njm9x\") on node \"crc\" DevicePath \"\"" Dec 03 12:07:46 crc kubenswrapper[5052]: I1203 12:07:46.780962 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6edd826b-b9ae-4e97-90be-8bc4954a1a77-sg-core-conf-yaml" (OuterVolumeSpecName: "sg-core-conf-yaml") pod "6edd826b-b9ae-4e97-90be-8bc4954a1a77" (UID: "6edd826b-b9ae-4e97-90be-8bc4954a1a77"). InnerVolumeSpecName "sg-core-conf-yaml". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:07:46 crc kubenswrapper[5052]: I1203 12:07:46.782323 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6edd826b-b9ae-4e97-90be-8bc4954a1a77-ceilometer-tls-certs" (OuterVolumeSpecName: "ceilometer-tls-certs") pod "6edd826b-b9ae-4e97-90be-8bc4954a1a77" (UID: "6edd826b-b9ae-4e97-90be-8bc4954a1a77"). InnerVolumeSpecName "ceilometer-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:07:46 crc kubenswrapper[5052]: I1203 12:07:46.784616 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6edd826b-b9ae-4e97-90be-8bc4954a1a77-scripts" (OuterVolumeSpecName: "scripts") pod "6edd826b-b9ae-4e97-90be-8bc4954a1a77" (UID: "6edd826b-b9ae-4e97-90be-8bc4954a1a77"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:07:46 crc kubenswrapper[5052]: I1203 12:07:46.797779 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6edd826b-b9ae-4e97-90be-8bc4954a1a77-config-data" (OuterVolumeSpecName: "config-data") pod "6edd826b-b9ae-4e97-90be-8bc4954a1a77" (UID: "6edd826b-b9ae-4e97-90be-8bc4954a1a77"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:07:46 crc kubenswrapper[5052]: I1203 12:07:46.882902 5052 reconciler_common.go:293] "Volume detached for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/6edd826b-b9ae-4e97-90be-8bc4954a1a77-sg-core-conf-yaml\") on node \"crc\" DevicePath \"\"" Dec 03 12:07:46 crc kubenswrapper[5052]: I1203 12:07:46.882955 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6edd826b-b9ae-4e97-90be-8bc4954a1a77-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:07:46 crc kubenswrapper[5052]: I1203 12:07:46.882970 5052 reconciler_common.go:293] "Volume detached for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/6edd826b-b9ae-4e97-90be-8bc4954a1a77-ceilometer-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:07:46 crc kubenswrapper[5052]: I1203 12:07:46.882984 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/6edd826b-b9ae-4e97-90be-8bc4954a1a77-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:07:47 crc kubenswrapper[5052]: I1203 12:07:47.403736 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:07:47 crc kubenswrapper[5052]: I1203 12:07:47.494316 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ff9eaa72-6d5d-48a9-85a5-362de3895ae1-combined-ca-bundle\") pod \"ff9eaa72-6d5d-48a9-85a5-362de3895ae1\" (UID: \"ff9eaa72-6d5d-48a9-85a5-362de3895ae1\") " Dec 03 12:07:47 crc kubenswrapper[5052]: I1203 12:07:47.494471 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/ff9eaa72-6d5d-48a9-85a5-362de3895ae1-config-data\") pod \"ff9eaa72-6d5d-48a9-85a5-362de3895ae1\" (UID: \"ff9eaa72-6d5d-48a9-85a5-362de3895ae1\") " Dec 03 12:07:47 crc kubenswrapper[5052]: I1203 12:07:47.494608 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/ff9eaa72-6d5d-48a9-85a5-362de3895ae1-logs\") pod \"ff9eaa72-6d5d-48a9-85a5-362de3895ae1\" (UID: \"ff9eaa72-6d5d-48a9-85a5-362de3895ae1\") " Dec 03 12:07:47 crc kubenswrapper[5052]: I1203 12:07:47.494638 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p2wkc\" (UniqueName: \"kubernetes.io/projected/ff9eaa72-6d5d-48a9-85a5-362de3895ae1-kube-api-access-p2wkc\") pod \"ff9eaa72-6d5d-48a9-85a5-362de3895ae1\" (UID: \"ff9eaa72-6d5d-48a9-85a5-362de3895ae1\") " Dec 03 12:07:47 crc kubenswrapper[5052]: I1203 12:07:47.495248 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff9eaa72-6d5d-48a9-85a5-362de3895ae1-logs" (OuterVolumeSpecName: "logs") pod "ff9eaa72-6d5d-48a9-85a5-362de3895ae1" (UID: "ff9eaa72-6d5d-48a9-85a5-362de3895ae1"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:07:47 crc kubenswrapper[5052]: I1203 12:07:47.513822 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ff9eaa72-6d5d-48a9-85a5-362de3895ae1-kube-api-access-p2wkc" (OuterVolumeSpecName: "kube-api-access-p2wkc") pod "ff9eaa72-6d5d-48a9-85a5-362de3895ae1" (UID: "ff9eaa72-6d5d-48a9-85a5-362de3895ae1"). InnerVolumeSpecName "kube-api-access-p2wkc". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:07:47 crc kubenswrapper[5052]: I1203 12:07:47.566832 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ff9eaa72-6d5d-48a9-85a5-362de3895ae1-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "ff9eaa72-6d5d-48a9-85a5-362de3895ae1" (UID: "ff9eaa72-6d5d-48a9-85a5-362de3895ae1"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:07:47 crc kubenswrapper[5052]: I1203 12:07:47.592663 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ff9eaa72-6d5d-48a9-85a5-362de3895ae1-config-data" (OuterVolumeSpecName: "config-data") pod "ff9eaa72-6d5d-48a9-85a5-362de3895ae1" (UID: "ff9eaa72-6d5d-48a9-85a5-362de3895ae1"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:07:47 crc kubenswrapper[5052]: I1203 12:07:47.596802 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ff9eaa72-6d5d-48a9-85a5-362de3895ae1-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:07:47 crc kubenswrapper[5052]: I1203 12:07:47.596829 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/ff9eaa72-6d5d-48a9-85a5-362de3895ae1-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:07:47 crc kubenswrapper[5052]: I1203 12:07:47.596843 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/ff9eaa72-6d5d-48a9-85a5-362de3895ae1-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:07:47 crc kubenswrapper[5052]: I1203 12:07:47.596853 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-p2wkc\" (UniqueName: \"kubernetes.io/projected/ff9eaa72-6d5d-48a9-85a5-362de3895ae1-kube-api-access-p2wkc\") on node \"crc\" DevicePath \"\"" Dec 03 12:07:47 crc kubenswrapper[5052]: I1203 12:07:47.690149 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6c9d4db1-a93f-4624-a6e1-d3a31285d74f" path="/var/lib/kubelet/pods/6c9d4db1-a93f-4624-a6e1-d3a31285d74f/volumes" Dec 03 12:07:47 crc kubenswrapper[5052]: I1203 12:07:47.704028 5052 generic.go:334] "Generic (PLEG): container finished" podID="ff9eaa72-6d5d-48a9-85a5-362de3895ae1" containerID="ffc10685c0afd2a841a5fca1660a8031d066ef01fd68e5278b19f1ed0ca8af9a" exitCode=0 Dec 03 12:07:47 crc kubenswrapper[5052]: I1203 12:07:47.704109 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:07:47 crc kubenswrapper[5052]: I1203 12:07:47.704328 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:07:47 crc kubenswrapper[5052]: I1203 12:07:47.704725 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"ff9eaa72-6d5d-48a9-85a5-362de3895ae1","Type":"ContainerDied","Data":"ffc10685c0afd2a841a5fca1660a8031d066ef01fd68e5278b19f1ed0ca8af9a"} Dec 03 12:07:47 crc kubenswrapper[5052]: I1203 12:07:47.704777 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"ff9eaa72-6d5d-48a9-85a5-362de3895ae1","Type":"ContainerDied","Data":"64c0cc84b0ba0b961e6145907450619d1e348b85b1c4b76a640e201a2990086c"} Dec 03 12:07:47 crc kubenswrapper[5052]: I1203 12:07:47.704797 5052 scope.go:117] "RemoveContainer" containerID="ffc10685c0afd2a841a5fca1660a8031d066ef01fd68e5278b19f1ed0ca8af9a" Dec 03 12:07:47 crc kubenswrapper[5052]: I1203 12:07:47.792346 5052 scope.go:117] "RemoveContainer" containerID="7724519ba83d2fb0f80b9e295f1c21f87d8043850d773806c593cd8e175c80ad" Dec 03 12:07:47 crc kubenswrapper[5052]: I1203 12:07:47.804059 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:07:47 crc kubenswrapper[5052]: I1203 12:07:47.813750 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:07:47 crc kubenswrapper[5052]: I1203 12:07:47.861086 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:07:47 crc kubenswrapper[5052]: E1203 12:07:47.861675 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="ff9eaa72-6d5d-48a9-85a5-362de3895ae1" containerName="nova-api-api" Dec 03 12:07:47 crc kubenswrapper[5052]: I1203 12:07:47.861698 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="ff9eaa72-6d5d-48a9-85a5-362de3895ae1" containerName="nova-api-api" Dec 03 12:07:47 crc kubenswrapper[5052]: E1203 12:07:47.861746 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="ff9eaa72-6d5d-48a9-85a5-362de3895ae1" containerName="nova-api-log" Dec 03 12:07:47 crc kubenswrapper[5052]: I1203 12:07:47.861754 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="ff9eaa72-6d5d-48a9-85a5-362de3895ae1" containerName="nova-api-log" Dec 03 12:07:47 crc kubenswrapper[5052]: I1203 12:07:47.862005 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="ff9eaa72-6d5d-48a9-85a5-362de3895ae1" containerName="nova-api-api" Dec 03 12:07:47 crc kubenswrapper[5052]: I1203 12:07:47.862035 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="ff9eaa72-6d5d-48a9-85a5-362de3895ae1" containerName="nova-api-log" Dec 03 12:07:47 crc kubenswrapper[5052]: I1203 12:07:47.863465 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:07:47 crc kubenswrapper[5052]: I1203 12:07:47.874399 5052 scope.go:117] "RemoveContainer" containerID="ffc10685c0afd2a841a5fca1660a8031d066ef01fd68e5278b19f1ed0ca8af9a" Dec 03 12:07:47 crc kubenswrapper[5052]: E1203 12:07:47.875649 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ffc10685c0afd2a841a5fca1660a8031d066ef01fd68e5278b19f1ed0ca8af9a\": container with ID starting with ffc10685c0afd2a841a5fca1660a8031d066ef01fd68e5278b19f1ed0ca8af9a not found: ID does not exist" containerID="ffc10685c0afd2a841a5fca1660a8031d066ef01fd68e5278b19f1ed0ca8af9a" Dec 03 12:07:47 crc kubenswrapper[5052]: I1203 12:07:47.875689 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ffc10685c0afd2a841a5fca1660a8031d066ef01fd68e5278b19f1ed0ca8af9a"} err="failed to get container status \"ffc10685c0afd2a841a5fca1660a8031d066ef01fd68e5278b19f1ed0ca8af9a\": rpc error: code = NotFound desc = could not find container \"ffc10685c0afd2a841a5fca1660a8031d066ef01fd68e5278b19f1ed0ca8af9a\": container with ID starting with ffc10685c0afd2a841a5fca1660a8031d066ef01fd68e5278b19f1ed0ca8af9a not found: ID does not exist" Dec 03 12:07:47 crc kubenswrapper[5052]: I1203 12:07:47.875715 5052 scope.go:117] "RemoveContainer" containerID="7724519ba83d2fb0f80b9e295f1c21f87d8043850d773806c593cd8e175c80ad" Dec 03 12:07:47 crc kubenswrapper[5052]: E1203 12:07:47.876146 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7724519ba83d2fb0f80b9e295f1c21f87d8043850d773806c593cd8e175c80ad\": container with ID starting with 7724519ba83d2fb0f80b9e295f1c21f87d8043850d773806c593cd8e175c80ad not found: ID does not exist" containerID="7724519ba83d2fb0f80b9e295f1c21f87d8043850d773806c593cd8e175c80ad" Dec 03 12:07:47 crc kubenswrapper[5052]: I1203 12:07:47.876170 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7724519ba83d2fb0f80b9e295f1c21f87d8043850d773806c593cd8e175c80ad"} err="failed to get container status \"7724519ba83d2fb0f80b9e295f1c21f87d8043850d773806c593cd8e175c80ad\": rpc error: code = NotFound desc = could not find container \"7724519ba83d2fb0f80b9e295f1c21f87d8043850d773806c593cd8e175c80ad\": container with ID starting with 7724519ba83d2fb0f80b9e295f1c21f87d8043850d773806c593cd8e175c80ad not found: ID does not exist" Dec 03 12:07:47 crc kubenswrapper[5052]: I1203 12:07:47.880390 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-nova-public-svc" Dec 03 12:07:47 crc kubenswrapper[5052]: I1203 12:07:47.880622 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-api-config-data" Dec 03 12:07:47 crc kubenswrapper[5052]: I1203 12:07:47.880770 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-nova-internal-svc" Dec 03 12:07:47 crc kubenswrapper[5052]: I1203 12:07:47.904352 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/e0362d1f-5452-4d2a-bc12-02cc0c991157-internal-tls-certs\") pod \"nova-api-0\" (UID: \"e0362d1f-5452-4d2a-bc12-02cc0c991157\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:07:47 crc kubenswrapper[5052]: I1203 12:07:47.904429 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/e0362d1f-5452-4d2a-bc12-02cc0c991157-logs\") pod \"nova-api-0\" (UID: \"e0362d1f-5452-4d2a-bc12-02cc0c991157\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:07:47 crc kubenswrapper[5052]: I1203 12:07:47.904474 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hj4tw\" (UniqueName: \"kubernetes.io/projected/e0362d1f-5452-4d2a-bc12-02cc0c991157-kube-api-access-hj4tw\") pod \"nova-api-0\" (UID: \"e0362d1f-5452-4d2a-bc12-02cc0c991157\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:07:47 crc kubenswrapper[5052]: I1203 12:07:47.904644 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/e0362d1f-5452-4d2a-bc12-02cc0c991157-config-data\") pod \"nova-api-0\" (UID: \"e0362d1f-5452-4d2a-bc12-02cc0c991157\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:07:47 crc kubenswrapper[5052]: I1203 12:07:47.904709 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/e0362d1f-5452-4d2a-bc12-02cc0c991157-public-tls-certs\") pod \"nova-api-0\" (UID: \"e0362d1f-5452-4d2a-bc12-02cc0c991157\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:07:47 crc kubenswrapper[5052]: I1203 12:07:47.904742 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e0362d1f-5452-4d2a-bc12-02cc0c991157-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"e0362d1f-5452-4d2a-bc12-02cc0c991157\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:07:47 crc kubenswrapper[5052]: I1203 12:07:47.912427 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:07:47 crc kubenswrapper[5052]: I1203 12:07:47.933037 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:07:47 crc kubenswrapper[5052]: I1203 12:07:47.946133 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:07:47 crc kubenswrapper[5052]: I1203 12:07:47.958755 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:07:47 crc kubenswrapper[5052]: I1203 12:07:47.962056 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:07:47 crc kubenswrapper[5052]: I1203 12:07:47.966736 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-scripts" Dec 03 12:07:47 crc kubenswrapper[5052]: I1203 12:07:47.966940 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-ceilometer-internal-svc" Dec 03 12:07:47 crc kubenswrapper[5052]: I1203 12:07:47.967082 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-config-data" Dec 03 12:07:47 crc kubenswrapper[5052]: I1203 12:07:47.973064 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:07:48 crc kubenswrapper[5052]: I1203 12:07:48.006977 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/e0362d1f-5452-4d2a-bc12-02cc0c991157-internal-tls-certs\") pod \"nova-api-0\" (UID: \"e0362d1f-5452-4d2a-bc12-02cc0c991157\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:07:48 crc kubenswrapper[5052]: I1203 12:07:48.007754 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/19229e07-3c78-4369-998e-fc9e78eb647d-run-httpd\") pod \"ceilometer-0\" (UID: \"19229e07-3c78-4369-998e-fc9e78eb647d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:07:48 crc kubenswrapper[5052]: I1203 12:07:48.007794 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/e0362d1f-5452-4d2a-bc12-02cc0c991157-logs\") pod \"nova-api-0\" (UID: \"e0362d1f-5452-4d2a-bc12-02cc0c991157\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:07:48 crc kubenswrapper[5052]: I1203 12:07:48.007833 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/19229e07-3c78-4369-998e-fc9e78eb647d-scripts\") pod \"ceilometer-0\" (UID: \"19229e07-3c78-4369-998e-fc9e78eb647d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:07:48 crc kubenswrapper[5052]: I1203 12:07:48.007864 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-hj4tw\" (UniqueName: \"kubernetes.io/projected/e0362d1f-5452-4d2a-bc12-02cc0c991157-kube-api-access-hj4tw\") pod \"nova-api-0\" (UID: \"e0362d1f-5452-4d2a-bc12-02cc0c991157\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:07:48 crc kubenswrapper[5052]: I1203 12:07:48.008113 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/e0362d1f-5452-4d2a-bc12-02cc0c991157-logs\") pod \"nova-api-0\" (UID: \"e0362d1f-5452-4d2a-bc12-02cc0c991157\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:07:48 crc kubenswrapper[5052]: I1203 12:07:48.007935 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/19229e07-3c78-4369-998e-fc9e78eb647d-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"19229e07-3c78-4369-998e-fc9e78eb647d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:07:48 crc kubenswrapper[5052]: I1203 12:07:48.008402 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/19229e07-3c78-4369-998e-fc9e78eb647d-ceilometer-tls-certs\") pod \"ceilometer-0\" (UID: \"19229e07-3c78-4369-998e-fc9e78eb647d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:07:48 crc kubenswrapper[5052]: I1203 12:07:48.008463 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/e0362d1f-5452-4d2a-bc12-02cc0c991157-config-data\") pod \"nova-api-0\" (UID: \"e0362d1f-5452-4d2a-bc12-02cc0c991157\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:07:48 crc kubenswrapper[5052]: I1203 12:07:48.008532 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/19229e07-3c78-4369-998e-fc9e78eb647d-log-httpd\") pod \"ceilometer-0\" (UID: \"19229e07-3c78-4369-998e-fc9e78eb647d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:07:48 crc kubenswrapper[5052]: I1203 12:07:48.008572 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/19229e07-3c78-4369-998e-fc9e78eb647d-config-data\") pod \"ceilometer-0\" (UID: \"19229e07-3c78-4369-998e-fc9e78eb647d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:07:48 crc kubenswrapper[5052]: I1203 12:07:48.008600 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7dmbd\" (UniqueName: \"kubernetes.io/projected/19229e07-3c78-4369-998e-fc9e78eb647d-kube-api-access-7dmbd\") pod \"ceilometer-0\" (UID: \"19229e07-3c78-4369-998e-fc9e78eb647d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:07:48 crc kubenswrapper[5052]: I1203 12:07:48.008634 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/e0362d1f-5452-4d2a-bc12-02cc0c991157-public-tls-certs\") pod \"nova-api-0\" (UID: \"e0362d1f-5452-4d2a-bc12-02cc0c991157\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:07:48 crc kubenswrapper[5052]: I1203 12:07:48.008674 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e0362d1f-5452-4d2a-bc12-02cc0c991157-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"e0362d1f-5452-4d2a-bc12-02cc0c991157\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:07:48 crc kubenswrapper[5052]: I1203 12:07:48.008698 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/19229e07-3c78-4369-998e-fc9e78eb647d-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"19229e07-3c78-4369-998e-fc9e78eb647d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:07:48 crc kubenswrapper[5052]: I1203 12:07:48.012185 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/e0362d1f-5452-4d2a-bc12-02cc0c991157-config-data\") pod \"nova-api-0\" (UID: \"e0362d1f-5452-4d2a-bc12-02cc0c991157\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:07:48 crc kubenswrapper[5052]: I1203 12:07:48.012717 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/e0362d1f-5452-4d2a-bc12-02cc0c991157-internal-tls-certs\") pod \"nova-api-0\" (UID: \"e0362d1f-5452-4d2a-bc12-02cc0c991157\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:07:48 crc kubenswrapper[5052]: I1203 12:07:48.014880 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/e0362d1f-5452-4d2a-bc12-02cc0c991157-public-tls-certs\") pod \"nova-api-0\" (UID: \"e0362d1f-5452-4d2a-bc12-02cc0c991157\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:07:48 crc kubenswrapper[5052]: I1203 12:07:48.018052 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e0362d1f-5452-4d2a-bc12-02cc0c991157-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"e0362d1f-5452-4d2a-bc12-02cc0c991157\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:07:48 crc kubenswrapper[5052]: I1203 12:07:48.029328 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-hj4tw\" (UniqueName: \"kubernetes.io/projected/e0362d1f-5452-4d2a-bc12-02cc0c991157-kube-api-access-hj4tw\") pod \"nova-api-0\" (UID: \"e0362d1f-5452-4d2a-bc12-02cc0c991157\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:07:48 crc kubenswrapper[5052]: I1203 12:07:48.110676 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/19229e07-3c78-4369-998e-fc9e78eb647d-ceilometer-tls-certs\") pod \"ceilometer-0\" (UID: \"19229e07-3c78-4369-998e-fc9e78eb647d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:07:48 crc kubenswrapper[5052]: I1203 12:07:48.110771 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/19229e07-3c78-4369-998e-fc9e78eb647d-log-httpd\") pod \"ceilometer-0\" (UID: \"19229e07-3c78-4369-998e-fc9e78eb647d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:07:48 crc kubenswrapper[5052]: I1203 12:07:48.111431 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/19229e07-3c78-4369-998e-fc9e78eb647d-log-httpd\") pod \"ceilometer-0\" (UID: \"19229e07-3c78-4369-998e-fc9e78eb647d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:07:48 crc kubenswrapper[5052]: I1203 12:07:48.111497 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/19229e07-3c78-4369-998e-fc9e78eb647d-config-data\") pod \"ceilometer-0\" (UID: \"19229e07-3c78-4369-998e-fc9e78eb647d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:07:48 crc kubenswrapper[5052]: I1203 12:07:48.111968 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-7dmbd\" (UniqueName: \"kubernetes.io/projected/19229e07-3c78-4369-998e-fc9e78eb647d-kube-api-access-7dmbd\") pod \"ceilometer-0\" (UID: \"19229e07-3c78-4369-998e-fc9e78eb647d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:07:48 crc kubenswrapper[5052]: I1203 12:07:48.113656 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/19229e07-3c78-4369-998e-fc9e78eb647d-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"19229e07-3c78-4369-998e-fc9e78eb647d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:07:48 crc kubenswrapper[5052]: I1203 12:07:48.113849 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/19229e07-3c78-4369-998e-fc9e78eb647d-run-httpd\") pod \"ceilometer-0\" (UID: \"19229e07-3c78-4369-998e-fc9e78eb647d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:07:48 crc kubenswrapper[5052]: I1203 12:07:48.114810 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/19229e07-3c78-4369-998e-fc9e78eb647d-run-httpd\") pod \"ceilometer-0\" (UID: \"19229e07-3c78-4369-998e-fc9e78eb647d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:07:48 crc kubenswrapper[5052]: I1203 12:07:48.114958 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/19229e07-3c78-4369-998e-fc9e78eb647d-scripts\") pod \"ceilometer-0\" (UID: \"19229e07-3c78-4369-998e-fc9e78eb647d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:07:48 crc kubenswrapper[5052]: I1203 12:07:48.116178 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/19229e07-3c78-4369-998e-fc9e78eb647d-ceilometer-tls-certs\") pod \"ceilometer-0\" (UID: \"19229e07-3c78-4369-998e-fc9e78eb647d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:07:48 crc kubenswrapper[5052]: I1203 12:07:48.118026 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/19229e07-3c78-4369-998e-fc9e78eb647d-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"19229e07-3c78-4369-998e-fc9e78eb647d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:07:48 crc kubenswrapper[5052]: I1203 12:07:48.121185 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/19229e07-3c78-4369-998e-fc9e78eb647d-scripts\") pod \"ceilometer-0\" (UID: \"19229e07-3c78-4369-998e-fc9e78eb647d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:07:48 crc kubenswrapper[5052]: I1203 12:07:48.121250 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/19229e07-3c78-4369-998e-fc9e78eb647d-config-data\") pod \"ceilometer-0\" (UID: \"19229e07-3c78-4369-998e-fc9e78eb647d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:07:48 crc kubenswrapper[5052]: I1203 12:07:48.121789 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/19229e07-3c78-4369-998e-fc9e78eb647d-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"19229e07-3c78-4369-998e-fc9e78eb647d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:07:48 crc kubenswrapper[5052]: I1203 12:07:48.127319 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/19229e07-3c78-4369-998e-fc9e78eb647d-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"19229e07-3c78-4369-998e-fc9e78eb647d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:07:48 crc kubenswrapper[5052]: I1203 12:07:48.140648 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-7dmbd\" (UniqueName: \"kubernetes.io/projected/19229e07-3c78-4369-998e-fc9e78eb647d-kube-api-access-7dmbd\") pod \"ceilometer-0\" (UID: \"19229e07-3c78-4369-998e-fc9e78eb647d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:07:48 crc kubenswrapper[5052]: I1203 12:07:48.185792 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:07:48 crc kubenswrapper[5052]: I1203 12:07:48.238468 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:07:48 crc kubenswrapper[5052]: W1203 12:07:48.476352 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode0362d1f_5452_4d2a_bc12_02cc0c991157.slice/crio-31fad397c39674c4aed11d6dbfd0a27c869c099713f8ac2a30fc6f63fc8ef837 WatchSource:0}: Error finding container 31fad397c39674c4aed11d6dbfd0a27c869c099713f8ac2a30fc6f63fc8ef837: Status 404 returned error can't find the container with id 31fad397c39674c4aed11d6dbfd0a27c869c099713f8ac2a30fc6f63fc8ef837 Dec 03 12:07:48 crc kubenswrapper[5052]: I1203 12:07:48.476190 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:07:48 crc kubenswrapper[5052]: I1203 12:07:48.719189 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"e0362d1f-5452-4d2a-bc12-02cc0c991157","Type":"ContainerStarted","Data":"9e3d819774442f654f376e23ec4c44bc429c94d81d40ef13a24adb583e96f4a6"} Dec 03 12:07:48 crc kubenswrapper[5052]: I1203 12:07:48.720391 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"e0362d1f-5452-4d2a-bc12-02cc0c991157","Type":"ContainerStarted","Data":"31fad397c39674c4aed11d6dbfd0a27c869c099713f8ac2a30fc6f63fc8ef837"} Dec 03 12:07:48 crc kubenswrapper[5052]: I1203 12:07:48.774118 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:07:48 crc kubenswrapper[5052]: W1203 12:07:48.776591 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod19229e07_3c78_4369_998e_fc9e78eb647d.slice/crio-f75e647877896e217d788f87299009cfac0175499a1272dc8466684d66e3648d WatchSource:0}: Error finding container f75e647877896e217d788f87299009cfac0175499a1272dc8466684d66e3648d: Status 404 returned error can't find the container with id f75e647877896e217d788f87299009cfac0175499a1272dc8466684d66e3648d Dec 03 12:07:48 crc kubenswrapper[5052]: I1203 12:07:48.981344 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:07:49 crc kubenswrapper[5052]: I1203 12:07:49.430444 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell1-cell-mapping-95t79"] Dec 03 12:07:49 crc kubenswrapper[5052]: I1203 12:07:49.432752 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-cell-mapping-95t79" Dec 03 12:07:49 crc kubenswrapper[5052]: I1203 12:07:49.435343 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell1-manage-scripts" Dec 03 12:07:49 crc kubenswrapper[5052]: I1203 12:07:49.435624 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell1-manage-config-data" Dec 03 12:07:49 crc kubenswrapper[5052]: I1203 12:07:49.442049 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-cell-mapping-95t79"] Dec 03 12:07:49 crc kubenswrapper[5052]: I1203 12:07:49.559268 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f2d639ef-d667-416d-8899-1a080bca471c-combined-ca-bundle\") pod \"nova-cell1-cell-mapping-95t79\" (UID: \"f2d639ef-d667-416d-8899-1a080bca471c\") " pod="openstack-kuttl-tests/nova-cell1-cell-mapping-95t79" Dec 03 12:07:49 crc kubenswrapper[5052]: I1203 12:07:49.559378 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/f2d639ef-d667-416d-8899-1a080bca471c-config-data\") pod \"nova-cell1-cell-mapping-95t79\" (UID: \"f2d639ef-d667-416d-8899-1a080bca471c\") " pod="openstack-kuttl-tests/nova-cell1-cell-mapping-95t79" Dec 03 12:07:49 crc kubenswrapper[5052]: I1203 12:07:49.559425 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-58g4b\" (UniqueName: \"kubernetes.io/projected/f2d639ef-d667-416d-8899-1a080bca471c-kube-api-access-58g4b\") pod \"nova-cell1-cell-mapping-95t79\" (UID: \"f2d639ef-d667-416d-8899-1a080bca471c\") " pod="openstack-kuttl-tests/nova-cell1-cell-mapping-95t79" Dec 03 12:07:49 crc kubenswrapper[5052]: I1203 12:07:49.559497 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/f2d639ef-d667-416d-8899-1a080bca471c-scripts\") pod \"nova-cell1-cell-mapping-95t79\" (UID: \"f2d639ef-d667-416d-8899-1a080bca471c\") " pod="openstack-kuttl-tests/nova-cell1-cell-mapping-95t79" Dec 03 12:07:49 crc kubenswrapper[5052]: I1203 12:07:49.661323 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-58g4b\" (UniqueName: \"kubernetes.io/projected/f2d639ef-d667-416d-8899-1a080bca471c-kube-api-access-58g4b\") pod \"nova-cell1-cell-mapping-95t79\" (UID: \"f2d639ef-d667-416d-8899-1a080bca471c\") " pod="openstack-kuttl-tests/nova-cell1-cell-mapping-95t79" Dec 03 12:07:49 crc kubenswrapper[5052]: I1203 12:07:49.661423 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/f2d639ef-d667-416d-8899-1a080bca471c-scripts\") pod \"nova-cell1-cell-mapping-95t79\" (UID: \"f2d639ef-d667-416d-8899-1a080bca471c\") " pod="openstack-kuttl-tests/nova-cell1-cell-mapping-95t79" Dec 03 12:07:49 crc kubenswrapper[5052]: I1203 12:07:49.661535 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f2d639ef-d667-416d-8899-1a080bca471c-combined-ca-bundle\") pod \"nova-cell1-cell-mapping-95t79\" (UID: \"f2d639ef-d667-416d-8899-1a080bca471c\") " pod="openstack-kuttl-tests/nova-cell1-cell-mapping-95t79" Dec 03 12:07:49 crc kubenswrapper[5052]: I1203 12:07:49.661603 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/f2d639ef-d667-416d-8899-1a080bca471c-config-data\") pod \"nova-cell1-cell-mapping-95t79\" (UID: \"f2d639ef-d667-416d-8899-1a080bca471c\") " pod="openstack-kuttl-tests/nova-cell1-cell-mapping-95t79" Dec 03 12:07:49 crc kubenswrapper[5052]: I1203 12:07:49.667589 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/f2d639ef-d667-416d-8899-1a080bca471c-scripts\") pod \"nova-cell1-cell-mapping-95t79\" (UID: \"f2d639ef-d667-416d-8899-1a080bca471c\") " pod="openstack-kuttl-tests/nova-cell1-cell-mapping-95t79" Dec 03 12:07:49 crc kubenswrapper[5052]: I1203 12:07:49.668073 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f2d639ef-d667-416d-8899-1a080bca471c-combined-ca-bundle\") pod \"nova-cell1-cell-mapping-95t79\" (UID: \"f2d639ef-d667-416d-8899-1a080bca471c\") " pod="openstack-kuttl-tests/nova-cell1-cell-mapping-95t79" Dec 03 12:07:49 crc kubenswrapper[5052]: I1203 12:07:49.668543 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/f2d639ef-d667-416d-8899-1a080bca471c-config-data\") pod \"nova-cell1-cell-mapping-95t79\" (UID: \"f2d639ef-d667-416d-8899-1a080bca471c\") " pod="openstack-kuttl-tests/nova-cell1-cell-mapping-95t79" Dec 03 12:07:49 crc kubenswrapper[5052]: I1203 12:07:49.684642 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-58g4b\" (UniqueName: \"kubernetes.io/projected/f2d639ef-d667-416d-8899-1a080bca471c-kube-api-access-58g4b\") pod \"nova-cell1-cell-mapping-95t79\" (UID: \"f2d639ef-d667-416d-8899-1a080bca471c\") " pod="openstack-kuttl-tests/nova-cell1-cell-mapping-95t79" Dec 03 12:07:49 crc kubenswrapper[5052]: I1203 12:07:49.688693 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6edd826b-b9ae-4e97-90be-8bc4954a1a77" path="/var/lib/kubelet/pods/6edd826b-b9ae-4e97-90be-8bc4954a1a77/volumes" Dec 03 12:07:49 crc kubenswrapper[5052]: I1203 12:07:49.689216 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ff9eaa72-6d5d-48a9-85a5-362de3895ae1" path="/var/lib/kubelet/pods/ff9eaa72-6d5d-48a9-85a5-362de3895ae1/volumes" Dec 03 12:07:49 crc kubenswrapper[5052]: I1203 12:07:49.731781 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"19229e07-3c78-4369-998e-fc9e78eb647d","Type":"ContainerStarted","Data":"f75e647877896e217d788f87299009cfac0175499a1272dc8466684d66e3648d"} Dec 03 12:07:49 crc kubenswrapper[5052]: I1203 12:07:49.736088 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"e0362d1f-5452-4d2a-bc12-02cc0c991157","Type":"ContainerStarted","Data":"248eea37d95b7bce148505f9ebb84d5705f6ae9ad51af39667cc9f8383e4659b"} Dec 03 12:07:49 crc kubenswrapper[5052]: I1203 12:07:49.768299 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-cell-mapping-95t79" Dec 03 12:07:49 crc kubenswrapper[5052]: I1203 12:07:49.769160 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-api-0" podStartSLOduration=2.769143545 podStartE2EDuration="2.769143545s" podCreationTimestamp="2025-12-03 12:07:47 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:07:49.756930486 +0000 UTC m=+2860.345172270" watchObservedRunningTime="2025-12-03 12:07:49.769143545 +0000 UTC m=+2860.357385309" Dec 03 12:07:50 crc kubenswrapper[5052]: I1203 12:07:50.239841 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-cell-mapping-95t79"] Dec 03 12:07:50 crc kubenswrapper[5052]: I1203 12:07:50.751416 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"19229e07-3c78-4369-998e-fc9e78eb647d","Type":"ContainerStarted","Data":"85d1ba6ba0934dbfc1b081d24d64fbe85278667fa8c3b4ccf7dd7749c336f706"} Dec 03 12:07:50 crc kubenswrapper[5052]: I1203 12:07:50.751819 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"19229e07-3c78-4369-998e-fc9e78eb647d","Type":"ContainerStarted","Data":"d1c0d412f0ba24368b4e09fb49e650d7f9751d7ab3deaf51a87873def7cef029"} Dec 03 12:07:50 crc kubenswrapper[5052]: I1203 12:07:50.754057 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-cell-mapping-95t79" event={"ID":"f2d639ef-d667-416d-8899-1a080bca471c","Type":"ContainerStarted","Data":"1b8800a111aa6f20fcc4e337231da09240be718253d0c4917217959ec3ed78f2"} Dec 03 12:07:50 crc kubenswrapper[5052]: I1203 12:07:50.754136 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-cell-mapping-95t79" event={"ID":"f2d639ef-d667-416d-8899-1a080bca471c","Type":"ContainerStarted","Data":"55e9c2e660ffe2cfad2cd5f0e80df3d24fec4b519ecc6d0ed996afaac1c98e3c"} Dec 03 12:07:50 crc kubenswrapper[5052]: I1203 12:07:50.783880 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-cell1-cell-mapping-95t79" podStartSLOduration=1.783862045 podStartE2EDuration="1.783862045s" podCreationTimestamp="2025-12-03 12:07:49 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:07:50.767643812 +0000 UTC m=+2861.355885576" watchObservedRunningTime="2025-12-03 12:07:50.783862045 +0000 UTC m=+2861.372103799" Dec 03 12:07:51 crc kubenswrapper[5052]: I1203 12:07:51.768593 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"19229e07-3c78-4369-998e-fc9e78eb647d","Type":"ContainerStarted","Data":"0e1e468e3a4838dddbe49481471a1e007e23016bc27634f15327a9a15546fcb0"} Dec 03 12:07:53 crc kubenswrapper[5052]: I1203 12:07:53.789444 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"19229e07-3c78-4369-998e-fc9e78eb647d","Type":"ContainerStarted","Data":"bc70dbc3ccd05a052e7fc3b5729f64a5f4c046c9a200d68d09f9e35887ba8604"} Dec 03 12:07:53 crc kubenswrapper[5052]: I1203 12:07:53.791099 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:07:53 crc kubenswrapper[5052]: I1203 12:07:53.828030 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/ceilometer-0" podStartSLOduration=2.816819671 podStartE2EDuration="6.828003756s" podCreationTimestamp="2025-12-03 12:07:47 +0000 UTC" firstStartedPulling="2025-12-03 12:07:48.781501311 +0000 UTC m=+2859.369743075" lastFinishedPulling="2025-12-03 12:07:52.792685386 +0000 UTC m=+2863.380927160" observedRunningTime="2025-12-03 12:07:53.819616686 +0000 UTC m=+2864.407858470" watchObservedRunningTime="2025-12-03 12:07:53.828003756 +0000 UTC m=+2864.416245520" Dec 03 12:07:55 crc kubenswrapper[5052]: I1203 12:07:55.812989 5052 generic.go:334] "Generic (PLEG): container finished" podID="f2d639ef-d667-416d-8899-1a080bca471c" containerID="1b8800a111aa6f20fcc4e337231da09240be718253d0c4917217959ec3ed78f2" exitCode=0 Dec 03 12:07:55 crc kubenswrapper[5052]: I1203 12:07:55.813343 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-cell-mapping-95t79" event={"ID":"f2d639ef-d667-416d-8899-1a080bca471c","Type":"ContainerDied","Data":"1b8800a111aa6f20fcc4e337231da09240be718253d0c4917217959ec3ed78f2"} Dec 03 12:07:55 crc kubenswrapper[5052]: I1203 12:07:55.991887 5052 scope.go:117] "RemoveContainer" containerID="1b59ddf16b90baa2fbbd6151a5b6d8907aeddafbb8a0245c485597f1baf52aef" Dec 03 12:07:56 crc kubenswrapper[5052]: I1203 12:07:56.018735 5052 scope.go:117] "RemoveContainer" containerID="428b572eb3f0325bc4882ff8b1b7b1e0f00b59900d1b774323cd1ac0a5bb910c" Dec 03 12:07:56 crc kubenswrapper[5052]: I1203 12:07:56.071837 5052 scope.go:117] "RemoveContainer" containerID="cb6c2c93752f2f8c0a07df86d1815e82e87a24854f9d5ff8d5cb21edecd1b88a" Dec 03 12:07:56 crc kubenswrapper[5052]: I1203 12:07:56.112218 5052 scope.go:117] "RemoveContainer" containerID="015cccbd4c1596494b8217997ac4a87be227da3e9b2b7729dd71f4bc53db28b6" Dec 03 12:07:56 crc kubenswrapper[5052]: I1203 12:07:56.144811 5052 scope.go:117] "RemoveContainer" containerID="f7eae34142d8abb346f6280e3bbdc08126ce436adb89c7086b7aa2dead8ef908" Dec 03 12:07:56 crc kubenswrapper[5052]: I1203 12:07:56.193731 5052 scope.go:117] "RemoveContainer" containerID="63357c27c7708913d6e53569543c75fd7424796b936ce997d25a360f5d1d6f99" Dec 03 12:07:56 crc kubenswrapper[5052]: I1203 12:07:56.220462 5052 scope.go:117] "RemoveContainer" containerID="65e23cdd1d480da8bdf0b2780f64b3f291ae9fc3c647fcf6b00e0dd8e2199e7e" Dec 03 12:07:57 crc kubenswrapper[5052]: I1203 12:07:57.265750 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-cell-mapping-95t79" Dec 03 12:07:57 crc kubenswrapper[5052]: I1203 12:07:57.335613 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/f2d639ef-d667-416d-8899-1a080bca471c-scripts\") pod \"f2d639ef-d667-416d-8899-1a080bca471c\" (UID: \"f2d639ef-d667-416d-8899-1a080bca471c\") " Dec 03 12:07:57 crc kubenswrapper[5052]: I1203 12:07:57.335740 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/f2d639ef-d667-416d-8899-1a080bca471c-config-data\") pod \"f2d639ef-d667-416d-8899-1a080bca471c\" (UID: \"f2d639ef-d667-416d-8899-1a080bca471c\") " Dec 03 12:07:57 crc kubenswrapper[5052]: I1203 12:07:57.335779 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f2d639ef-d667-416d-8899-1a080bca471c-combined-ca-bundle\") pod \"f2d639ef-d667-416d-8899-1a080bca471c\" (UID: \"f2d639ef-d667-416d-8899-1a080bca471c\") " Dec 03 12:07:57 crc kubenswrapper[5052]: I1203 12:07:57.335825 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-58g4b\" (UniqueName: \"kubernetes.io/projected/f2d639ef-d667-416d-8899-1a080bca471c-kube-api-access-58g4b\") pod \"f2d639ef-d667-416d-8899-1a080bca471c\" (UID: \"f2d639ef-d667-416d-8899-1a080bca471c\") " Dec 03 12:07:57 crc kubenswrapper[5052]: I1203 12:07:57.340943 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f2d639ef-d667-416d-8899-1a080bca471c-scripts" (OuterVolumeSpecName: "scripts") pod "f2d639ef-d667-416d-8899-1a080bca471c" (UID: "f2d639ef-d667-416d-8899-1a080bca471c"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:07:57 crc kubenswrapper[5052]: I1203 12:07:57.345090 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f2d639ef-d667-416d-8899-1a080bca471c-kube-api-access-58g4b" (OuterVolumeSpecName: "kube-api-access-58g4b") pod "f2d639ef-d667-416d-8899-1a080bca471c" (UID: "f2d639ef-d667-416d-8899-1a080bca471c"). InnerVolumeSpecName "kube-api-access-58g4b". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:07:57 crc kubenswrapper[5052]: I1203 12:07:57.369868 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f2d639ef-d667-416d-8899-1a080bca471c-config-data" (OuterVolumeSpecName: "config-data") pod "f2d639ef-d667-416d-8899-1a080bca471c" (UID: "f2d639ef-d667-416d-8899-1a080bca471c"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:07:57 crc kubenswrapper[5052]: I1203 12:07:57.369977 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f2d639ef-d667-416d-8899-1a080bca471c-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "f2d639ef-d667-416d-8899-1a080bca471c" (UID: "f2d639ef-d667-416d-8899-1a080bca471c"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:07:57 crc kubenswrapper[5052]: I1203 12:07:57.437874 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/f2d639ef-d667-416d-8899-1a080bca471c-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:07:57 crc kubenswrapper[5052]: I1203 12:07:57.437917 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f2d639ef-d667-416d-8899-1a080bca471c-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:07:57 crc kubenswrapper[5052]: I1203 12:07:57.437933 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-58g4b\" (UniqueName: \"kubernetes.io/projected/f2d639ef-d667-416d-8899-1a080bca471c-kube-api-access-58g4b\") on node \"crc\" DevicePath \"\"" Dec 03 12:07:57 crc kubenswrapper[5052]: I1203 12:07:57.437945 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/f2d639ef-d667-416d-8899-1a080bca471c-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:07:57 crc kubenswrapper[5052]: I1203 12:07:57.851093 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-cell-mapping-95t79" event={"ID":"f2d639ef-d667-416d-8899-1a080bca471c","Type":"ContainerDied","Data":"55e9c2e660ffe2cfad2cd5f0e80df3d24fec4b519ecc6d0ed996afaac1c98e3c"} Dec 03 12:07:57 crc kubenswrapper[5052]: I1203 12:07:57.851147 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="55e9c2e660ffe2cfad2cd5f0e80df3d24fec4b519ecc6d0ed996afaac1c98e3c" Dec 03 12:07:57 crc kubenswrapper[5052]: I1203 12:07:57.851227 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-cell-mapping-95t79" Dec 03 12:07:58 crc kubenswrapper[5052]: I1203 12:07:58.021839 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:07:58 crc kubenswrapper[5052]: I1203 12:07:58.022212 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-api-0" podUID="e0362d1f-5452-4d2a-bc12-02cc0c991157" containerName="nova-api-log" containerID="cri-o://9e3d819774442f654f376e23ec4c44bc429c94d81d40ef13a24adb583e96f4a6" gracePeriod=30 Dec 03 12:07:58 crc kubenswrapper[5052]: I1203 12:07:58.022341 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-api-0" podUID="e0362d1f-5452-4d2a-bc12-02cc0c991157" containerName="nova-api-api" containerID="cri-o://248eea37d95b7bce148505f9ebb84d5705f6ae9ad51af39667cc9f8383e4659b" gracePeriod=30 Dec 03 12:07:58 crc kubenswrapper[5052]: I1203 12:07:58.066346 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:07:58 crc kubenswrapper[5052]: I1203 12:07:58.067650 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-scheduler-0" podUID="d99defb0-caae-4f5f-9a24-d3026d396c31" containerName="nova-scheduler-scheduler" containerID="cri-o://c9cc962c26f0896a3372d1fc98911dbd818567d0b3bfed73cedd9a3e3b5cee76" gracePeriod=30 Dec 03 12:07:58 crc kubenswrapper[5052]: I1203 12:07:58.095953 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:07:58 crc kubenswrapper[5052]: I1203 12:07:58.096250 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-metadata-0" podUID="26024570-e836-484e-be44-8bf3a86277df" containerName="nova-metadata-log" containerID="cri-o://99398cd54307f11b2e45075f7d91e929454004eb4ea850ce677de64e98526999" gracePeriod=30 Dec 03 12:07:58 crc kubenswrapper[5052]: I1203 12:07:58.096343 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-metadata-0" podUID="26024570-e836-484e-be44-8bf3a86277df" containerName="nova-metadata-metadata" containerID="cri-o://410a9f8f5acdd3ad89a0440f12ff1b02a0dcf588ebdc64fb9d6d0fd48f3265c5" gracePeriod=30 Dec 03 12:07:58 crc kubenswrapper[5052]: I1203 12:07:58.646711 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:07:58 crc kubenswrapper[5052]: I1203 12:07:58.764578 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e0362d1f-5452-4d2a-bc12-02cc0c991157-combined-ca-bundle\") pod \"e0362d1f-5452-4d2a-bc12-02cc0c991157\" (UID: \"e0362d1f-5452-4d2a-bc12-02cc0c991157\") " Dec 03 12:07:58 crc kubenswrapper[5052]: I1203 12:07:58.764651 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/e0362d1f-5452-4d2a-bc12-02cc0c991157-public-tls-certs\") pod \"e0362d1f-5452-4d2a-bc12-02cc0c991157\" (UID: \"e0362d1f-5452-4d2a-bc12-02cc0c991157\") " Dec 03 12:07:58 crc kubenswrapper[5052]: I1203 12:07:58.764691 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/e0362d1f-5452-4d2a-bc12-02cc0c991157-internal-tls-certs\") pod \"e0362d1f-5452-4d2a-bc12-02cc0c991157\" (UID: \"e0362d1f-5452-4d2a-bc12-02cc0c991157\") " Dec 03 12:07:58 crc kubenswrapper[5052]: I1203 12:07:58.764875 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/e0362d1f-5452-4d2a-bc12-02cc0c991157-logs\") pod \"e0362d1f-5452-4d2a-bc12-02cc0c991157\" (UID: \"e0362d1f-5452-4d2a-bc12-02cc0c991157\") " Dec 03 12:07:58 crc kubenswrapper[5052]: I1203 12:07:58.764985 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hj4tw\" (UniqueName: \"kubernetes.io/projected/e0362d1f-5452-4d2a-bc12-02cc0c991157-kube-api-access-hj4tw\") pod \"e0362d1f-5452-4d2a-bc12-02cc0c991157\" (UID: \"e0362d1f-5452-4d2a-bc12-02cc0c991157\") " Dec 03 12:07:58 crc kubenswrapper[5052]: I1203 12:07:58.765017 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/e0362d1f-5452-4d2a-bc12-02cc0c991157-config-data\") pod \"e0362d1f-5452-4d2a-bc12-02cc0c991157\" (UID: \"e0362d1f-5452-4d2a-bc12-02cc0c991157\") " Dec 03 12:07:58 crc kubenswrapper[5052]: I1203 12:07:58.765280 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e0362d1f-5452-4d2a-bc12-02cc0c991157-logs" (OuterVolumeSpecName: "logs") pod "e0362d1f-5452-4d2a-bc12-02cc0c991157" (UID: "e0362d1f-5452-4d2a-bc12-02cc0c991157"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:07:58 crc kubenswrapper[5052]: I1203 12:07:58.765922 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/e0362d1f-5452-4d2a-bc12-02cc0c991157-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:07:58 crc kubenswrapper[5052]: I1203 12:07:58.772044 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e0362d1f-5452-4d2a-bc12-02cc0c991157-kube-api-access-hj4tw" (OuterVolumeSpecName: "kube-api-access-hj4tw") pod "e0362d1f-5452-4d2a-bc12-02cc0c991157" (UID: "e0362d1f-5452-4d2a-bc12-02cc0c991157"). InnerVolumeSpecName "kube-api-access-hj4tw". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:07:58 crc kubenswrapper[5052]: I1203 12:07:58.804593 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e0362d1f-5452-4d2a-bc12-02cc0c991157-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "e0362d1f-5452-4d2a-bc12-02cc0c991157" (UID: "e0362d1f-5452-4d2a-bc12-02cc0c991157"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:07:58 crc kubenswrapper[5052]: I1203 12:07:58.811083 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e0362d1f-5452-4d2a-bc12-02cc0c991157-config-data" (OuterVolumeSpecName: "config-data") pod "e0362d1f-5452-4d2a-bc12-02cc0c991157" (UID: "e0362d1f-5452-4d2a-bc12-02cc0c991157"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:07:58 crc kubenswrapper[5052]: I1203 12:07:58.828436 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e0362d1f-5452-4d2a-bc12-02cc0c991157-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "e0362d1f-5452-4d2a-bc12-02cc0c991157" (UID: "e0362d1f-5452-4d2a-bc12-02cc0c991157"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:07:58 crc kubenswrapper[5052]: I1203 12:07:58.861258 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e0362d1f-5452-4d2a-bc12-02cc0c991157-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "e0362d1f-5452-4d2a-bc12-02cc0c991157" (UID: "e0362d1f-5452-4d2a-bc12-02cc0c991157"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:07:58 crc kubenswrapper[5052]: I1203 12:07:58.863358 5052 generic.go:334] "Generic (PLEG): container finished" podID="e0362d1f-5452-4d2a-bc12-02cc0c991157" containerID="248eea37d95b7bce148505f9ebb84d5705f6ae9ad51af39667cc9f8383e4659b" exitCode=0 Dec 03 12:07:58 crc kubenswrapper[5052]: I1203 12:07:58.863390 5052 generic.go:334] "Generic (PLEG): container finished" podID="e0362d1f-5452-4d2a-bc12-02cc0c991157" containerID="9e3d819774442f654f376e23ec4c44bc429c94d81d40ef13a24adb583e96f4a6" exitCode=143 Dec 03 12:07:58 crc kubenswrapper[5052]: I1203 12:07:58.863436 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:07:58 crc kubenswrapper[5052]: I1203 12:07:58.863449 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"e0362d1f-5452-4d2a-bc12-02cc0c991157","Type":"ContainerDied","Data":"248eea37d95b7bce148505f9ebb84d5705f6ae9ad51af39667cc9f8383e4659b"} Dec 03 12:07:58 crc kubenswrapper[5052]: I1203 12:07:58.863483 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"e0362d1f-5452-4d2a-bc12-02cc0c991157","Type":"ContainerDied","Data":"9e3d819774442f654f376e23ec4c44bc429c94d81d40ef13a24adb583e96f4a6"} Dec 03 12:07:58 crc kubenswrapper[5052]: I1203 12:07:58.863493 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"e0362d1f-5452-4d2a-bc12-02cc0c991157","Type":"ContainerDied","Data":"31fad397c39674c4aed11d6dbfd0a27c869c099713f8ac2a30fc6f63fc8ef837"} Dec 03 12:07:58 crc kubenswrapper[5052]: I1203 12:07:58.863527 5052 scope.go:117] "RemoveContainer" containerID="248eea37d95b7bce148505f9ebb84d5705f6ae9ad51af39667cc9f8383e4659b" Dec 03 12:07:58 crc kubenswrapper[5052]: I1203 12:07:58.866861 5052 generic.go:334] "Generic (PLEG): container finished" podID="26024570-e836-484e-be44-8bf3a86277df" containerID="99398cd54307f11b2e45075f7d91e929454004eb4ea850ce677de64e98526999" exitCode=143 Dec 03 12:07:58 crc kubenswrapper[5052]: I1203 12:07:58.866899 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"26024570-e836-484e-be44-8bf3a86277df","Type":"ContainerDied","Data":"99398cd54307f11b2e45075f7d91e929454004eb4ea850ce677de64e98526999"} Dec 03 12:07:58 crc kubenswrapper[5052]: I1203 12:07:58.868210 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/e0362d1f-5452-4d2a-bc12-02cc0c991157-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:07:58 crc kubenswrapper[5052]: I1203 12:07:58.868255 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e0362d1f-5452-4d2a-bc12-02cc0c991157-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:07:58 crc kubenswrapper[5052]: I1203 12:07:58.868272 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/e0362d1f-5452-4d2a-bc12-02cc0c991157-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:07:58 crc kubenswrapper[5052]: I1203 12:07:58.868285 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/e0362d1f-5452-4d2a-bc12-02cc0c991157-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:07:58 crc kubenswrapper[5052]: I1203 12:07:58.868297 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-hj4tw\" (UniqueName: \"kubernetes.io/projected/e0362d1f-5452-4d2a-bc12-02cc0c991157-kube-api-access-hj4tw\") on node \"crc\" DevicePath \"\"" Dec 03 12:07:58 crc kubenswrapper[5052]: I1203 12:07:58.886929 5052 scope.go:117] "RemoveContainer" containerID="9e3d819774442f654f376e23ec4c44bc429c94d81d40ef13a24adb583e96f4a6" Dec 03 12:07:58 crc kubenswrapper[5052]: I1203 12:07:58.915587 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:07:58 crc kubenswrapper[5052]: I1203 12:07:58.919352 5052 scope.go:117] "RemoveContainer" containerID="248eea37d95b7bce148505f9ebb84d5705f6ae9ad51af39667cc9f8383e4659b" Dec 03 12:07:58 crc kubenswrapper[5052]: E1203 12:07:58.921233 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"248eea37d95b7bce148505f9ebb84d5705f6ae9ad51af39667cc9f8383e4659b\": container with ID starting with 248eea37d95b7bce148505f9ebb84d5705f6ae9ad51af39667cc9f8383e4659b not found: ID does not exist" containerID="248eea37d95b7bce148505f9ebb84d5705f6ae9ad51af39667cc9f8383e4659b" Dec 03 12:07:58 crc kubenswrapper[5052]: I1203 12:07:58.921295 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"248eea37d95b7bce148505f9ebb84d5705f6ae9ad51af39667cc9f8383e4659b"} err="failed to get container status \"248eea37d95b7bce148505f9ebb84d5705f6ae9ad51af39667cc9f8383e4659b\": rpc error: code = NotFound desc = could not find container \"248eea37d95b7bce148505f9ebb84d5705f6ae9ad51af39667cc9f8383e4659b\": container with ID starting with 248eea37d95b7bce148505f9ebb84d5705f6ae9ad51af39667cc9f8383e4659b not found: ID does not exist" Dec 03 12:07:58 crc kubenswrapper[5052]: I1203 12:07:58.921328 5052 scope.go:117] "RemoveContainer" containerID="9e3d819774442f654f376e23ec4c44bc429c94d81d40ef13a24adb583e96f4a6" Dec 03 12:07:58 crc kubenswrapper[5052]: E1203 12:07:58.921719 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9e3d819774442f654f376e23ec4c44bc429c94d81d40ef13a24adb583e96f4a6\": container with ID starting with 9e3d819774442f654f376e23ec4c44bc429c94d81d40ef13a24adb583e96f4a6 not found: ID does not exist" containerID="9e3d819774442f654f376e23ec4c44bc429c94d81d40ef13a24adb583e96f4a6" Dec 03 12:07:58 crc kubenswrapper[5052]: I1203 12:07:58.921780 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9e3d819774442f654f376e23ec4c44bc429c94d81d40ef13a24adb583e96f4a6"} err="failed to get container status \"9e3d819774442f654f376e23ec4c44bc429c94d81d40ef13a24adb583e96f4a6\": rpc error: code = NotFound desc = could not find container \"9e3d819774442f654f376e23ec4c44bc429c94d81d40ef13a24adb583e96f4a6\": container with ID starting with 9e3d819774442f654f376e23ec4c44bc429c94d81d40ef13a24adb583e96f4a6 not found: ID does not exist" Dec 03 12:07:58 crc kubenswrapper[5052]: I1203 12:07:58.921805 5052 scope.go:117] "RemoveContainer" containerID="248eea37d95b7bce148505f9ebb84d5705f6ae9ad51af39667cc9f8383e4659b" Dec 03 12:07:58 crc kubenswrapper[5052]: I1203 12:07:58.922161 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"248eea37d95b7bce148505f9ebb84d5705f6ae9ad51af39667cc9f8383e4659b"} err="failed to get container status \"248eea37d95b7bce148505f9ebb84d5705f6ae9ad51af39667cc9f8383e4659b\": rpc error: code = NotFound desc = could not find container \"248eea37d95b7bce148505f9ebb84d5705f6ae9ad51af39667cc9f8383e4659b\": container with ID starting with 248eea37d95b7bce148505f9ebb84d5705f6ae9ad51af39667cc9f8383e4659b not found: ID does not exist" Dec 03 12:07:58 crc kubenswrapper[5052]: I1203 12:07:58.922215 5052 scope.go:117] "RemoveContainer" containerID="9e3d819774442f654f376e23ec4c44bc429c94d81d40ef13a24adb583e96f4a6" Dec 03 12:07:58 crc kubenswrapper[5052]: I1203 12:07:58.922495 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9e3d819774442f654f376e23ec4c44bc429c94d81d40ef13a24adb583e96f4a6"} err="failed to get container status \"9e3d819774442f654f376e23ec4c44bc429c94d81d40ef13a24adb583e96f4a6\": rpc error: code = NotFound desc = could not find container \"9e3d819774442f654f376e23ec4c44bc429c94d81d40ef13a24adb583e96f4a6\": container with ID starting with 9e3d819774442f654f376e23ec4c44bc429c94d81d40ef13a24adb583e96f4a6 not found: ID does not exist" Dec 03 12:07:58 crc kubenswrapper[5052]: I1203 12:07:58.945631 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:07:58 crc kubenswrapper[5052]: I1203 12:07:58.963131 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:07:58 crc kubenswrapper[5052]: E1203 12:07:58.963624 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="e0362d1f-5452-4d2a-bc12-02cc0c991157" containerName="nova-api-api" Dec 03 12:07:58 crc kubenswrapper[5052]: I1203 12:07:58.963643 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="e0362d1f-5452-4d2a-bc12-02cc0c991157" containerName="nova-api-api" Dec 03 12:07:58 crc kubenswrapper[5052]: E1203 12:07:58.963662 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="f2d639ef-d667-416d-8899-1a080bca471c" containerName="nova-manage" Dec 03 12:07:58 crc kubenswrapper[5052]: I1203 12:07:58.963669 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="f2d639ef-d667-416d-8899-1a080bca471c" containerName="nova-manage" Dec 03 12:07:58 crc kubenswrapper[5052]: E1203 12:07:58.963683 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="e0362d1f-5452-4d2a-bc12-02cc0c991157" containerName="nova-api-log" Dec 03 12:07:58 crc kubenswrapper[5052]: I1203 12:07:58.963689 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="e0362d1f-5452-4d2a-bc12-02cc0c991157" containerName="nova-api-log" Dec 03 12:07:58 crc kubenswrapper[5052]: I1203 12:07:58.963852 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="e0362d1f-5452-4d2a-bc12-02cc0c991157" containerName="nova-api-log" Dec 03 12:07:58 crc kubenswrapper[5052]: I1203 12:07:58.963869 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="e0362d1f-5452-4d2a-bc12-02cc0c991157" containerName="nova-api-api" Dec 03 12:07:58 crc kubenswrapper[5052]: I1203 12:07:58.963884 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="f2d639ef-d667-416d-8899-1a080bca471c" containerName="nova-manage" Dec 03 12:07:58 crc kubenswrapper[5052]: I1203 12:07:58.964964 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:07:58 crc kubenswrapper[5052]: I1203 12:07:58.968522 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-nova-public-svc" Dec 03 12:07:58 crc kubenswrapper[5052]: I1203 12:07:58.969352 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-api-config-data" Dec 03 12:07:58 crc kubenswrapper[5052]: I1203 12:07:58.969751 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-nova-internal-svc" Dec 03 12:07:58 crc kubenswrapper[5052]: I1203 12:07:58.975747 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:07:59 crc kubenswrapper[5052]: I1203 12:07:59.071121 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/45e26e8c-71b5-4b90-bba7-5bad2650d9b2-internal-tls-certs\") pod \"nova-api-0\" (UID: \"45e26e8c-71b5-4b90-bba7-5bad2650d9b2\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:07:59 crc kubenswrapper[5052]: I1203 12:07:59.071224 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/45e26e8c-71b5-4b90-bba7-5bad2650d9b2-public-tls-certs\") pod \"nova-api-0\" (UID: \"45e26e8c-71b5-4b90-bba7-5bad2650d9b2\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:07:59 crc kubenswrapper[5052]: I1203 12:07:59.071321 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/45e26e8c-71b5-4b90-bba7-5bad2650d9b2-config-data\") pod \"nova-api-0\" (UID: \"45e26e8c-71b5-4b90-bba7-5bad2650d9b2\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:07:59 crc kubenswrapper[5052]: I1203 12:07:59.071424 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/45e26e8c-71b5-4b90-bba7-5bad2650d9b2-logs\") pod \"nova-api-0\" (UID: \"45e26e8c-71b5-4b90-bba7-5bad2650d9b2\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:07:59 crc kubenswrapper[5052]: I1203 12:07:59.071472 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qhlxn\" (UniqueName: \"kubernetes.io/projected/45e26e8c-71b5-4b90-bba7-5bad2650d9b2-kube-api-access-qhlxn\") pod \"nova-api-0\" (UID: \"45e26e8c-71b5-4b90-bba7-5bad2650d9b2\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:07:59 crc kubenswrapper[5052]: I1203 12:07:59.071568 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/45e26e8c-71b5-4b90-bba7-5bad2650d9b2-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"45e26e8c-71b5-4b90-bba7-5bad2650d9b2\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:07:59 crc kubenswrapper[5052]: I1203 12:07:59.173562 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/45e26e8c-71b5-4b90-bba7-5bad2650d9b2-internal-tls-certs\") pod \"nova-api-0\" (UID: \"45e26e8c-71b5-4b90-bba7-5bad2650d9b2\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:07:59 crc kubenswrapper[5052]: I1203 12:07:59.173862 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/45e26e8c-71b5-4b90-bba7-5bad2650d9b2-public-tls-certs\") pod \"nova-api-0\" (UID: \"45e26e8c-71b5-4b90-bba7-5bad2650d9b2\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:07:59 crc kubenswrapper[5052]: I1203 12:07:59.174014 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/45e26e8c-71b5-4b90-bba7-5bad2650d9b2-config-data\") pod \"nova-api-0\" (UID: \"45e26e8c-71b5-4b90-bba7-5bad2650d9b2\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:07:59 crc kubenswrapper[5052]: I1203 12:07:59.174219 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/45e26e8c-71b5-4b90-bba7-5bad2650d9b2-logs\") pod \"nova-api-0\" (UID: \"45e26e8c-71b5-4b90-bba7-5bad2650d9b2\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:07:59 crc kubenswrapper[5052]: I1203 12:07:59.174350 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-qhlxn\" (UniqueName: \"kubernetes.io/projected/45e26e8c-71b5-4b90-bba7-5bad2650d9b2-kube-api-access-qhlxn\") pod \"nova-api-0\" (UID: \"45e26e8c-71b5-4b90-bba7-5bad2650d9b2\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:07:59 crc kubenswrapper[5052]: I1203 12:07:59.174498 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/45e26e8c-71b5-4b90-bba7-5bad2650d9b2-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"45e26e8c-71b5-4b90-bba7-5bad2650d9b2\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:07:59 crc kubenswrapper[5052]: I1203 12:07:59.174673 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/45e26e8c-71b5-4b90-bba7-5bad2650d9b2-logs\") pod \"nova-api-0\" (UID: \"45e26e8c-71b5-4b90-bba7-5bad2650d9b2\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:07:59 crc kubenswrapper[5052]: I1203 12:07:59.177552 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/45e26e8c-71b5-4b90-bba7-5bad2650d9b2-internal-tls-certs\") pod \"nova-api-0\" (UID: \"45e26e8c-71b5-4b90-bba7-5bad2650d9b2\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:07:59 crc kubenswrapper[5052]: I1203 12:07:59.177947 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/45e26e8c-71b5-4b90-bba7-5bad2650d9b2-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"45e26e8c-71b5-4b90-bba7-5bad2650d9b2\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:07:59 crc kubenswrapper[5052]: I1203 12:07:59.180572 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/45e26e8c-71b5-4b90-bba7-5bad2650d9b2-config-data\") pod \"nova-api-0\" (UID: \"45e26e8c-71b5-4b90-bba7-5bad2650d9b2\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:07:59 crc kubenswrapper[5052]: I1203 12:07:59.181434 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/45e26e8c-71b5-4b90-bba7-5bad2650d9b2-public-tls-certs\") pod \"nova-api-0\" (UID: \"45e26e8c-71b5-4b90-bba7-5bad2650d9b2\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:07:59 crc kubenswrapper[5052]: I1203 12:07:59.197851 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-qhlxn\" (UniqueName: \"kubernetes.io/projected/45e26e8c-71b5-4b90-bba7-5bad2650d9b2-kube-api-access-qhlxn\") pod \"nova-api-0\" (UID: \"45e26e8c-71b5-4b90-bba7-5bad2650d9b2\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:07:59 crc kubenswrapper[5052]: I1203 12:07:59.303277 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:07:59 crc kubenswrapper[5052]: I1203 12:07:59.380696 5052 patch_prober.go:28] interesting pod/machine-config-daemon-74bj6 container/machine-config-daemon namespace/openshift-machine-config-operator: Liveness probe status=failure output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" start-of-body= Dec 03 12:07:59 crc kubenswrapper[5052]: I1203 12:07:59.380936 5052 prober.go:107] "Probe failed" probeType="Liveness" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" containerName="machine-config-daemon" probeResult="failure" output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" Dec 03 12:07:59 crc kubenswrapper[5052]: I1203 12:07:59.683679 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e0362d1f-5452-4d2a-bc12-02cc0c991157" path="/var/lib/kubelet/pods/e0362d1f-5452-4d2a-bc12-02cc0c991157/volumes" Dec 03 12:07:59 crc kubenswrapper[5052]: I1203 12:07:59.730693 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:07:59 crc kubenswrapper[5052]: W1203 12:07:59.733146 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod45e26e8c_71b5_4b90_bba7_5bad2650d9b2.slice/crio-c3a994755bb38f1b0e921a8cf82017f3c0390ce0425049a85226095ee88f384d WatchSource:0}: Error finding container c3a994755bb38f1b0e921a8cf82017f3c0390ce0425049a85226095ee88f384d: Status 404 returned error can't find the container with id c3a994755bb38f1b0e921a8cf82017f3c0390ce0425049a85226095ee88f384d Dec 03 12:07:59 crc kubenswrapper[5052]: I1203 12:07:59.878736 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"45e26e8c-71b5-4b90-bba7-5bad2650d9b2","Type":"ContainerStarted","Data":"c3a994755bb38f1b0e921a8cf82017f3c0390ce0425049a85226095ee88f384d"} Dec 03 12:08:00 crc kubenswrapper[5052]: I1203 12:08:00.894269 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"45e26e8c-71b5-4b90-bba7-5bad2650d9b2","Type":"ContainerStarted","Data":"a96c58179a61355fd41c8699c34a2a6efcdd96a6901a2acedb4bb445fe0b84fe"} Dec 03 12:08:00 crc kubenswrapper[5052]: I1203 12:08:00.894627 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"45e26e8c-71b5-4b90-bba7-5bad2650d9b2","Type":"ContainerStarted","Data":"37bdeb3b3f3f5f77fea4db0897de61d07f8d244f4f0ab18853e3fa400f45ac54"} Dec 03 12:08:00 crc kubenswrapper[5052]: I1203 12:08:00.918566 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-api-0" podStartSLOduration=2.91854388 podStartE2EDuration="2.91854388s" podCreationTimestamp="2025-12-03 12:07:58 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:08:00.912241249 +0000 UTC m=+2871.500483033" watchObservedRunningTime="2025-12-03 12:08:00.91854388 +0000 UTC m=+2871.506785644" Dec 03 12:08:01 crc kubenswrapper[5052]: I1203 12:08:01.224015 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/nova-metadata-0" podUID="26024570-e836-484e-be44-8bf3a86277df" containerName="nova-metadata-metadata" probeResult="failure" output="Get \"https://10.217.0.186:8775/\": read tcp 10.217.0.2:33992->10.217.0.186:8775: read: connection reset by peer" Dec 03 12:08:01 crc kubenswrapper[5052]: I1203 12:08:01.224046 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/nova-metadata-0" podUID="26024570-e836-484e-be44-8bf3a86277df" containerName="nova-metadata-log" probeResult="failure" output="Get \"https://10.217.0.186:8775/\": read tcp 10.217.0.2:34008->10.217.0.186:8775: read: connection reset by peer" Dec 03 12:08:01 crc kubenswrapper[5052]: I1203 12:08:01.865617 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:08:01 crc kubenswrapper[5052]: I1203 12:08:01.914364 5052 generic.go:334] "Generic (PLEG): container finished" podID="26024570-e836-484e-be44-8bf3a86277df" containerID="410a9f8f5acdd3ad89a0440f12ff1b02a0dcf588ebdc64fb9d6d0fd48f3265c5" exitCode=0 Dec 03 12:08:01 crc kubenswrapper[5052]: I1203 12:08:01.914455 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:08:01 crc kubenswrapper[5052]: I1203 12:08:01.914499 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"26024570-e836-484e-be44-8bf3a86277df","Type":"ContainerDied","Data":"410a9f8f5acdd3ad89a0440f12ff1b02a0dcf588ebdc64fb9d6d0fd48f3265c5"} Dec 03 12:08:01 crc kubenswrapper[5052]: I1203 12:08:01.914550 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"26024570-e836-484e-be44-8bf3a86277df","Type":"ContainerDied","Data":"b02b9225723b0c66311defd6447d42e596a8cc2d56455e976cb1ab1c8da5d607"} Dec 03 12:08:01 crc kubenswrapper[5052]: I1203 12:08:01.914573 5052 scope.go:117] "RemoveContainer" containerID="410a9f8f5acdd3ad89a0440f12ff1b02a0dcf588ebdc64fb9d6d0fd48f3265c5" Dec 03 12:08:01 crc kubenswrapper[5052]: I1203 12:08:01.936682 5052 scope.go:117] "RemoveContainer" containerID="99398cd54307f11b2e45075f7d91e929454004eb4ea850ce677de64e98526999" Dec 03 12:08:01 crc kubenswrapper[5052]: I1203 12:08:01.963287 5052 scope.go:117] "RemoveContainer" containerID="410a9f8f5acdd3ad89a0440f12ff1b02a0dcf588ebdc64fb9d6d0fd48f3265c5" Dec 03 12:08:01 crc kubenswrapper[5052]: E1203 12:08:01.963912 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"410a9f8f5acdd3ad89a0440f12ff1b02a0dcf588ebdc64fb9d6d0fd48f3265c5\": container with ID starting with 410a9f8f5acdd3ad89a0440f12ff1b02a0dcf588ebdc64fb9d6d0fd48f3265c5 not found: ID does not exist" containerID="410a9f8f5acdd3ad89a0440f12ff1b02a0dcf588ebdc64fb9d6d0fd48f3265c5" Dec 03 12:08:01 crc kubenswrapper[5052]: I1203 12:08:01.963971 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"410a9f8f5acdd3ad89a0440f12ff1b02a0dcf588ebdc64fb9d6d0fd48f3265c5"} err="failed to get container status \"410a9f8f5acdd3ad89a0440f12ff1b02a0dcf588ebdc64fb9d6d0fd48f3265c5\": rpc error: code = NotFound desc = could not find container \"410a9f8f5acdd3ad89a0440f12ff1b02a0dcf588ebdc64fb9d6d0fd48f3265c5\": container with ID starting with 410a9f8f5acdd3ad89a0440f12ff1b02a0dcf588ebdc64fb9d6d0fd48f3265c5 not found: ID does not exist" Dec 03 12:08:01 crc kubenswrapper[5052]: I1203 12:08:01.964014 5052 scope.go:117] "RemoveContainer" containerID="99398cd54307f11b2e45075f7d91e929454004eb4ea850ce677de64e98526999" Dec 03 12:08:01 crc kubenswrapper[5052]: E1203 12:08:01.964242 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"99398cd54307f11b2e45075f7d91e929454004eb4ea850ce677de64e98526999\": container with ID starting with 99398cd54307f11b2e45075f7d91e929454004eb4ea850ce677de64e98526999 not found: ID does not exist" containerID="99398cd54307f11b2e45075f7d91e929454004eb4ea850ce677de64e98526999" Dec 03 12:08:01 crc kubenswrapper[5052]: I1203 12:08:01.964288 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"99398cd54307f11b2e45075f7d91e929454004eb4ea850ce677de64e98526999"} err="failed to get container status \"99398cd54307f11b2e45075f7d91e929454004eb4ea850ce677de64e98526999\": rpc error: code = NotFound desc = could not find container \"99398cd54307f11b2e45075f7d91e929454004eb4ea850ce677de64e98526999\": container with ID starting with 99398cd54307f11b2e45075f7d91e929454004eb4ea850ce677de64e98526999 not found: ID does not exist" Dec 03 12:08:02 crc kubenswrapper[5052]: I1203 12:08:02.032801 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/26024570-e836-484e-be44-8bf3a86277df-logs\") pod \"26024570-e836-484e-be44-8bf3a86277df\" (UID: \"26024570-e836-484e-be44-8bf3a86277df\") " Dec 03 12:08:02 crc kubenswrapper[5052]: I1203 12:08:02.032866 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/26024570-e836-484e-be44-8bf3a86277df-combined-ca-bundle\") pod \"26024570-e836-484e-be44-8bf3a86277df\" (UID: \"26024570-e836-484e-be44-8bf3a86277df\") " Dec 03 12:08:02 crc kubenswrapper[5052]: I1203 12:08:02.032990 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/26024570-e836-484e-be44-8bf3a86277df-nova-metadata-tls-certs\") pod \"26024570-e836-484e-be44-8bf3a86277df\" (UID: \"26024570-e836-484e-be44-8bf3a86277df\") " Dec 03 12:08:02 crc kubenswrapper[5052]: I1203 12:08:02.033072 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wfrzz\" (UniqueName: \"kubernetes.io/projected/26024570-e836-484e-be44-8bf3a86277df-kube-api-access-wfrzz\") pod \"26024570-e836-484e-be44-8bf3a86277df\" (UID: \"26024570-e836-484e-be44-8bf3a86277df\") " Dec 03 12:08:02 crc kubenswrapper[5052]: I1203 12:08:02.033245 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/26024570-e836-484e-be44-8bf3a86277df-config-data\") pod \"26024570-e836-484e-be44-8bf3a86277df\" (UID: \"26024570-e836-484e-be44-8bf3a86277df\") " Dec 03 12:08:02 crc kubenswrapper[5052]: I1203 12:08:02.034101 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/26024570-e836-484e-be44-8bf3a86277df-logs" (OuterVolumeSpecName: "logs") pod "26024570-e836-484e-be44-8bf3a86277df" (UID: "26024570-e836-484e-be44-8bf3a86277df"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:08:02 crc kubenswrapper[5052]: I1203 12:08:02.041889 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/26024570-e836-484e-be44-8bf3a86277df-kube-api-access-wfrzz" (OuterVolumeSpecName: "kube-api-access-wfrzz") pod "26024570-e836-484e-be44-8bf3a86277df" (UID: "26024570-e836-484e-be44-8bf3a86277df"). InnerVolumeSpecName "kube-api-access-wfrzz". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:08:02 crc kubenswrapper[5052]: I1203 12:08:02.069317 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/26024570-e836-484e-be44-8bf3a86277df-config-data" (OuterVolumeSpecName: "config-data") pod "26024570-e836-484e-be44-8bf3a86277df" (UID: "26024570-e836-484e-be44-8bf3a86277df"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:08:02 crc kubenswrapper[5052]: I1203 12:08:02.081349 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/26024570-e836-484e-be44-8bf3a86277df-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "26024570-e836-484e-be44-8bf3a86277df" (UID: "26024570-e836-484e-be44-8bf3a86277df"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:08:02 crc kubenswrapper[5052]: I1203 12:08:02.094005 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/26024570-e836-484e-be44-8bf3a86277df-nova-metadata-tls-certs" (OuterVolumeSpecName: "nova-metadata-tls-certs") pod "26024570-e836-484e-be44-8bf3a86277df" (UID: "26024570-e836-484e-be44-8bf3a86277df"). InnerVolumeSpecName "nova-metadata-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:08:02 crc kubenswrapper[5052]: I1203 12:08:02.135502 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/26024570-e836-484e-be44-8bf3a86277df-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:02 crc kubenswrapper[5052]: I1203 12:08:02.135559 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/26024570-e836-484e-be44-8bf3a86277df-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:02 crc kubenswrapper[5052]: I1203 12:08:02.135574 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/26024570-e836-484e-be44-8bf3a86277df-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:02 crc kubenswrapper[5052]: I1203 12:08:02.135586 5052 reconciler_common.go:293] "Volume detached for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/26024570-e836-484e-be44-8bf3a86277df-nova-metadata-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:02 crc kubenswrapper[5052]: I1203 12:08:02.135598 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-wfrzz\" (UniqueName: \"kubernetes.io/projected/26024570-e836-484e-be44-8bf3a86277df-kube-api-access-wfrzz\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:02 crc kubenswrapper[5052]: I1203 12:08:02.253556 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:08:02 crc kubenswrapper[5052]: I1203 12:08:02.266834 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:08:02 crc kubenswrapper[5052]: I1203 12:08:02.310611 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:08:02 crc kubenswrapper[5052]: E1203 12:08:02.311587 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="26024570-e836-484e-be44-8bf3a86277df" containerName="nova-metadata-metadata" Dec 03 12:08:02 crc kubenswrapper[5052]: I1203 12:08:02.311605 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="26024570-e836-484e-be44-8bf3a86277df" containerName="nova-metadata-metadata" Dec 03 12:08:02 crc kubenswrapper[5052]: E1203 12:08:02.311644 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="26024570-e836-484e-be44-8bf3a86277df" containerName="nova-metadata-log" Dec 03 12:08:02 crc kubenswrapper[5052]: I1203 12:08:02.311652 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="26024570-e836-484e-be44-8bf3a86277df" containerName="nova-metadata-log" Dec 03 12:08:02 crc kubenswrapper[5052]: I1203 12:08:02.312037 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="26024570-e836-484e-be44-8bf3a86277df" containerName="nova-metadata-log" Dec 03 12:08:02 crc kubenswrapper[5052]: I1203 12:08:02.312094 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="26024570-e836-484e-be44-8bf3a86277df" containerName="nova-metadata-metadata" Dec 03 12:08:02 crc kubenswrapper[5052]: I1203 12:08:02.314727 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:08:02 crc kubenswrapper[5052]: I1203 12:08:02.318046 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-metadata-config-data" Dec 03 12:08:02 crc kubenswrapper[5052]: I1203 12:08:02.318766 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-nova-metadata-internal-svc" Dec 03 12:08:02 crc kubenswrapper[5052]: I1203 12:08:02.336281 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:08:02 crc kubenswrapper[5052]: I1203 12:08:02.442248 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/b9efd251-95c2-4224-b318-a34279ee1017-logs\") pod \"nova-metadata-0\" (UID: \"b9efd251-95c2-4224-b318-a34279ee1017\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:08:02 crc kubenswrapper[5052]: I1203 12:08:02.442380 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b5qq5\" (UniqueName: \"kubernetes.io/projected/b9efd251-95c2-4224-b318-a34279ee1017-kube-api-access-b5qq5\") pod \"nova-metadata-0\" (UID: \"b9efd251-95c2-4224-b318-a34279ee1017\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:08:02 crc kubenswrapper[5052]: I1203 12:08:02.442407 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b9efd251-95c2-4224-b318-a34279ee1017-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"b9efd251-95c2-4224-b318-a34279ee1017\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:08:02 crc kubenswrapper[5052]: I1203 12:08:02.442441 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/b9efd251-95c2-4224-b318-a34279ee1017-config-data\") pod \"nova-metadata-0\" (UID: \"b9efd251-95c2-4224-b318-a34279ee1017\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:08:02 crc kubenswrapper[5052]: I1203 12:08:02.442481 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/b9efd251-95c2-4224-b318-a34279ee1017-nova-metadata-tls-certs\") pod \"nova-metadata-0\" (UID: \"b9efd251-95c2-4224-b318-a34279ee1017\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:08:02 crc kubenswrapper[5052]: I1203 12:08:02.544532 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/b9efd251-95c2-4224-b318-a34279ee1017-nova-metadata-tls-certs\") pod \"nova-metadata-0\" (UID: \"b9efd251-95c2-4224-b318-a34279ee1017\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:08:02 crc kubenswrapper[5052]: I1203 12:08:02.544648 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/b9efd251-95c2-4224-b318-a34279ee1017-logs\") pod \"nova-metadata-0\" (UID: \"b9efd251-95c2-4224-b318-a34279ee1017\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:08:02 crc kubenswrapper[5052]: I1203 12:08:02.544757 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-b5qq5\" (UniqueName: \"kubernetes.io/projected/b9efd251-95c2-4224-b318-a34279ee1017-kube-api-access-b5qq5\") pod \"nova-metadata-0\" (UID: \"b9efd251-95c2-4224-b318-a34279ee1017\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:08:02 crc kubenswrapper[5052]: I1203 12:08:02.544785 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b9efd251-95c2-4224-b318-a34279ee1017-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"b9efd251-95c2-4224-b318-a34279ee1017\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:08:02 crc kubenswrapper[5052]: I1203 12:08:02.544823 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/b9efd251-95c2-4224-b318-a34279ee1017-config-data\") pod \"nova-metadata-0\" (UID: \"b9efd251-95c2-4224-b318-a34279ee1017\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:08:02 crc kubenswrapper[5052]: I1203 12:08:02.545290 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/b9efd251-95c2-4224-b318-a34279ee1017-logs\") pod \"nova-metadata-0\" (UID: \"b9efd251-95c2-4224-b318-a34279ee1017\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:08:02 crc kubenswrapper[5052]: I1203 12:08:02.548768 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b9efd251-95c2-4224-b318-a34279ee1017-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"b9efd251-95c2-4224-b318-a34279ee1017\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:08:02 crc kubenswrapper[5052]: I1203 12:08:02.549429 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/b9efd251-95c2-4224-b318-a34279ee1017-nova-metadata-tls-certs\") pod \"nova-metadata-0\" (UID: \"b9efd251-95c2-4224-b318-a34279ee1017\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:08:02 crc kubenswrapper[5052]: I1203 12:08:02.549678 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/b9efd251-95c2-4224-b318-a34279ee1017-config-data\") pod \"nova-metadata-0\" (UID: \"b9efd251-95c2-4224-b318-a34279ee1017\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:08:02 crc kubenswrapper[5052]: I1203 12:08:02.593201 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-b5qq5\" (UniqueName: \"kubernetes.io/projected/b9efd251-95c2-4224-b318-a34279ee1017-kube-api-access-b5qq5\") pod \"nova-metadata-0\" (UID: \"b9efd251-95c2-4224-b318-a34279ee1017\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:08:02 crc kubenswrapper[5052]: I1203 12:08:02.641926 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:08:02 crc kubenswrapper[5052]: E1203 12:08:02.894649 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="c9cc962c26f0896a3372d1fc98911dbd818567d0b3bfed73cedd9a3e3b5cee76" cmd=["/usr/bin/pgrep","-r","DRST","nova-scheduler"] Dec 03 12:08:02 crc kubenswrapper[5052]: E1203 12:08:02.896718 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="c9cc962c26f0896a3372d1fc98911dbd818567d0b3bfed73cedd9a3e3b5cee76" cmd=["/usr/bin/pgrep","-r","DRST","nova-scheduler"] Dec 03 12:08:02 crc kubenswrapper[5052]: E1203 12:08:02.898241 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="c9cc962c26f0896a3372d1fc98911dbd818567d0b3bfed73cedd9a3e3b5cee76" cmd=["/usr/bin/pgrep","-r","DRST","nova-scheduler"] Dec 03 12:08:02 crc kubenswrapper[5052]: E1203 12:08:02.898301 5052 prober.go:104] "Probe errored" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" probeType="Readiness" pod="openstack-kuttl-tests/nova-scheduler-0" podUID="d99defb0-caae-4f5f-9a24-d3026d396c31" containerName="nova-scheduler-scheduler" Dec 03 12:08:03 crc kubenswrapper[5052]: I1203 12:08:03.135552 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:08:03 crc kubenswrapper[5052]: I1203 12:08:03.686143 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="26024570-e836-484e-be44-8bf3a86277df" path="/var/lib/kubelet/pods/26024570-e836-484e-be44-8bf3a86277df/volumes" Dec 03 12:08:03 crc kubenswrapper[5052]: I1203 12:08:03.936259 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"b9efd251-95c2-4224-b318-a34279ee1017","Type":"ContainerStarted","Data":"b42c2e5aaf3a181255a02334958b7382f6e8aaa9faf937e59f371e0e658d9523"} Dec 03 12:08:04 crc kubenswrapper[5052]: I1203 12:08:04.919311 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:08:04 crc kubenswrapper[5052]: I1203 12:08:04.951102 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"b9efd251-95c2-4224-b318-a34279ee1017","Type":"ContainerStarted","Data":"0d0239c7224731fd55404636504e2c22a3b640502408dba4f002511377ec7f33"} Dec 03 12:08:04 crc kubenswrapper[5052]: I1203 12:08:04.951159 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"b9efd251-95c2-4224-b318-a34279ee1017","Type":"ContainerStarted","Data":"179d0d417d11399d428ed2dce00676182f65bdc218984235dfbdc87fb4e0c5c0"} Dec 03 12:08:04 crc kubenswrapper[5052]: I1203 12:08:04.956639 5052 generic.go:334] "Generic (PLEG): container finished" podID="d99defb0-caae-4f5f-9a24-d3026d396c31" containerID="c9cc962c26f0896a3372d1fc98911dbd818567d0b3bfed73cedd9a3e3b5cee76" exitCode=0 Dec 03 12:08:04 crc kubenswrapper[5052]: I1203 12:08:04.957000 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"d99defb0-caae-4f5f-9a24-d3026d396c31","Type":"ContainerDied","Data":"c9cc962c26f0896a3372d1fc98911dbd818567d0b3bfed73cedd9a3e3b5cee76"} Dec 03 12:08:04 crc kubenswrapper[5052]: I1203 12:08:04.957032 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"d99defb0-caae-4f5f-9a24-d3026d396c31","Type":"ContainerDied","Data":"94573ff07096de1bcfd7a75e01e072eed25046372c1cacd2c16d5282d1c38efc"} Dec 03 12:08:04 crc kubenswrapper[5052]: I1203 12:08:04.957055 5052 scope.go:117] "RemoveContainer" containerID="c9cc962c26f0896a3372d1fc98911dbd818567d0b3bfed73cedd9a3e3b5cee76" Dec 03 12:08:04 crc kubenswrapper[5052]: I1203 12:08:04.957207 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:08:04 crc kubenswrapper[5052]: I1203 12:08:04.977666 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-metadata-0" podStartSLOduration=2.977646867 podStartE2EDuration="2.977646867s" podCreationTimestamp="2025-12-03 12:08:02 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:08:04.96900503 +0000 UTC m=+2875.557246804" watchObservedRunningTime="2025-12-03 12:08:04.977646867 +0000 UTC m=+2875.565888631" Dec 03 12:08:04 crc kubenswrapper[5052]: I1203 12:08:04.987841 5052 scope.go:117] "RemoveContainer" containerID="c9cc962c26f0896a3372d1fc98911dbd818567d0b3bfed73cedd9a3e3b5cee76" Dec 03 12:08:04 crc kubenswrapper[5052]: E1203 12:08:04.988270 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c9cc962c26f0896a3372d1fc98911dbd818567d0b3bfed73cedd9a3e3b5cee76\": container with ID starting with c9cc962c26f0896a3372d1fc98911dbd818567d0b3bfed73cedd9a3e3b5cee76 not found: ID does not exist" containerID="c9cc962c26f0896a3372d1fc98911dbd818567d0b3bfed73cedd9a3e3b5cee76" Dec 03 12:08:04 crc kubenswrapper[5052]: I1203 12:08:04.988300 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c9cc962c26f0896a3372d1fc98911dbd818567d0b3bfed73cedd9a3e3b5cee76"} err="failed to get container status \"c9cc962c26f0896a3372d1fc98911dbd818567d0b3bfed73cedd9a3e3b5cee76\": rpc error: code = NotFound desc = could not find container \"c9cc962c26f0896a3372d1fc98911dbd818567d0b3bfed73cedd9a3e3b5cee76\": container with ID starting with c9cc962c26f0896a3372d1fc98911dbd818567d0b3bfed73cedd9a3e3b5cee76 not found: ID does not exist" Dec 03 12:08:05 crc kubenswrapper[5052]: I1203 12:08:05.016127 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vgt8h\" (UniqueName: \"kubernetes.io/projected/d99defb0-caae-4f5f-9a24-d3026d396c31-kube-api-access-vgt8h\") pod \"d99defb0-caae-4f5f-9a24-d3026d396c31\" (UID: \"d99defb0-caae-4f5f-9a24-d3026d396c31\") " Dec 03 12:08:05 crc kubenswrapper[5052]: I1203 12:08:05.016237 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d99defb0-caae-4f5f-9a24-d3026d396c31-combined-ca-bundle\") pod \"d99defb0-caae-4f5f-9a24-d3026d396c31\" (UID: \"d99defb0-caae-4f5f-9a24-d3026d396c31\") " Dec 03 12:08:05 crc kubenswrapper[5052]: I1203 12:08:05.016330 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/d99defb0-caae-4f5f-9a24-d3026d396c31-config-data\") pod \"d99defb0-caae-4f5f-9a24-d3026d396c31\" (UID: \"d99defb0-caae-4f5f-9a24-d3026d396c31\") " Dec 03 12:08:05 crc kubenswrapper[5052]: I1203 12:08:05.023545 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d99defb0-caae-4f5f-9a24-d3026d396c31-kube-api-access-vgt8h" (OuterVolumeSpecName: "kube-api-access-vgt8h") pod "d99defb0-caae-4f5f-9a24-d3026d396c31" (UID: "d99defb0-caae-4f5f-9a24-d3026d396c31"). InnerVolumeSpecName "kube-api-access-vgt8h". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:08:05 crc kubenswrapper[5052]: I1203 12:08:05.052954 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d99defb0-caae-4f5f-9a24-d3026d396c31-config-data" (OuterVolumeSpecName: "config-data") pod "d99defb0-caae-4f5f-9a24-d3026d396c31" (UID: "d99defb0-caae-4f5f-9a24-d3026d396c31"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:08:05 crc kubenswrapper[5052]: I1203 12:08:05.056829 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d99defb0-caae-4f5f-9a24-d3026d396c31-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "d99defb0-caae-4f5f-9a24-d3026d396c31" (UID: "d99defb0-caae-4f5f-9a24-d3026d396c31"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:08:05 crc kubenswrapper[5052]: I1203 12:08:05.123388 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/d99defb0-caae-4f5f-9a24-d3026d396c31-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:05 crc kubenswrapper[5052]: I1203 12:08:05.123420 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-vgt8h\" (UniqueName: \"kubernetes.io/projected/d99defb0-caae-4f5f-9a24-d3026d396c31-kube-api-access-vgt8h\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:05 crc kubenswrapper[5052]: I1203 12:08:05.123431 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d99defb0-caae-4f5f-9a24-d3026d396c31-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:05 crc kubenswrapper[5052]: I1203 12:08:05.292793 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:08:05 crc kubenswrapper[5052]: I1203 12:08:05.304805 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:08:05 crc kubenswrapper[5052]: I1203 12:08:05.326867 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:08:05 crc kubenswrapper[5052]: E1203 12:08:05.327952 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="d99defb0-caae-4f5f-9a24-d3026d396c31" containerName="nova-scheduler-scheduler" Dec 03 12:08:05 crc kubenswrapper[5052]: I1203 12:08:05.327979 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="d99defb0-caae-4f5f-9a24-d3026d396c31" containerName="nova-scheduler-scheduler" Dec 03 12:08:05 crc kubenswrapper[5052]: I1203 12:08:05.328420 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="d99defb0-caae-4f5f-9a24-d3026d396c31" containerName="nova-scheduler-scheduler" Dec 03 12:08:05 crc kubenswrapper[5052]: I1203 12:08:05.329608 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:08:05 crc kubenswrapper[5052]: I1203 12:08:05.332869 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-scheduler-config-data" Dec 03 12:08:05 crc kubenswrapper[5052]: I1203 12:08:05.355331 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:08:05 crc kubenswrapper[5052]: I1203 12:08:05.437093 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0b100677-2da0-4717-a15a-d2463ee6713e-combined-ca-bundle\") pod \"nova-scheduler-0\" (UID: \"0b100677-2da0-4717-a15a-d2463ee6713e\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:08:05 crc kubenswrapper[5052]: I1203 12:08:05.437230 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ct7fq\" (UniqueName: \"kubernetes.io/projected/0b100677-2da0-4717-a15a-d2463ee6713e-kube-api-access-ct7fq\") pod \"nova-scheduler-0\" (UID: \"0b100677-2da0-4717-a15a-d2463ee6713e\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:08:05 crc kubenswrapper[5052]: I1203 12:08:05.437263 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/0b100677-2da0-4717-a15a-d2463ee6713e-config-data\") pod \"nova-scheduler-0\" (UID: \"0b100677-2da0-4717-a15a-d2463ee6713e\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:08:05 crc kubenswrapper[5052]: I1203 12:08:05.540616 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0b100677-2da0-4717-a15a-d2463ee6713e-combined-ca-bundle\") pod \"nova-scheduler-0\" (UID: \"0b100677-2da0-4717-a15a-d2463ee6713e\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:08:05 crc kubenswrapper[5052]: I1203 12:08:05.540717 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-ct7fq\" (UniqueName: \"kubernetes.io/projected/0b100677-2da0-4717-a15a-d2463ee6713e-kube-api-access-ct7fq\") pod \"nova-scheduler-0\" (UID: \"0b100677-2da0-4717-a15a-d2463ee6713e\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:08:05 crc kubenswrapper[5052]: I1203 12:08:05.540746 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/0b100677-2da0-4717-a15a-d2463ee6713e-config-data\") pod \"nova-scheduler-0\" (UID: \"0b100677-2da0-4717-a15a-d2463ee6713e\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:08:05 crc kubenswrapper[5052]: I1203 12:08:05.545912 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/0b100677-2da0-4717-a15a-d2463ee6713e-config-data\") pod \"nova-scheduler-0\" (UID: \"0b100677-2da0-4717-a15a-d2463ee6713e\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:08:05 crc kubenswrapper[5052]: I1203 12:08:05.559492 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0b100677-2da0-4717-a15a-d2463ee6713e-combined-ca-bundle\") pod \"nova-scheduler-0\" (UID: \"0b100677-2da0-4717-a15a-d2463ee6713e\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:08:05 crc kubenswrapper[5052]: I1203 12:08:05.560277 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-ct7fq\" (UniqueName: \"kubernetes.io/projected/0b100677-2da0-4717-a15a-d2463ee6713e-kube-api-access-ct7fq\") pod \"nova-scheduler-0\" (UID: \"0b100677-2da0-4717-a15a-d2463ee6713e\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:08:05 crc kubenswrapper[5052]: I1203 12:08:05.664233 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:08:05 crc kubenswrapper[5052]: I1203 12:08:05.687864 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d99defb0-caae-4f5f-9a24-d3026d396c31" path="/var/lib/kubelet/pods/d99defb0-caae-4f5f-9a24-d3026d396c31/volumes" Dec 03 12:08:06 crc kubenswrapper[5052]: I1203 12:08:06.095259 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:08:06 crc kubenswrapper[5052]: I1203 12:08:06.991571 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"0b100677-2da0-4717-a15a-d2463ee6713e","Type":"ContainerStarted","Data":"df9dfde29f7996c2208b2860c440e3ab6120cd657e154f046ad4cdb1b318a678"} Dec 03 12:08:06 crc kubenswrapper[5052]: I1203 12:08:06.991619 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"0b100677-2da0-4717-a15a-d2463ee6713e","Type":"ContainerStarted","Data":"1da1fd81580901fcd43d9bc625b28fcabd26928a5c783a3c40a4628aa15caa92"} Dec 03 12:08:07 crc kubenswrapper[5052]: I1203 12:08:07.642330 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:08:07 crc kubenswrapper[5052]: I1203 12:08:07.643692 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:08:09 crc kubenswrapper[5052]: I1203 12:08:09.303658 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:08:09 crc kubenswrapper[5052]: I1203 12:08:09.304840 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:08:10 crc kubenswrapper[5052]: I1203 12:08:10.315669 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-api-0" podUID="45e26e8c-71b5-4b90-bba7-5bad2650d9b2" containerName="nova-api-log" probeResult="failure" output="Get \"https://10.217.0.196:8774/\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Dec 03 12:08:10 crc kubenswrapper[5052]: I1203 12:08:10.315710 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-api-0" podUID="45e26e8c-71b5-4b90-bba7-5bad2650d9b2" containerName="nova-api-api" probeResult="failure" output="Get \"https://10.217.0.196:8774/\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Dec 03 12:08:10 crc kubenswrapper[5052]: I1203 12:08:10.665084 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:08:12 crc kubenswrapper[5052]: I1203 12:08:12.642717 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:08:12 crc kubenswrapper[5052]: I1203 12:08:12.642802 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:08:13 crc kubenswrapper[5052]: I1203 12:08:13.654643 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-metadata-0" podUID="b9efd251-95c2-4224-b318-a34279ee1017" containerName="nova-metadata-log" probeResult="failure" output="Get \"https://10.217.0.197:8775/\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Dec 03 12:08:13 crc kubenswrapper[5052]: I1203 12:08:13.654648 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-metadata-0" podUID="b9efd251-95c2-4224-b318-a34279ee1017" containerName="nova-metadata-metadata" probeResult="failure" output="Get \"https://10.217.0.197:8775/\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Dec 03 12:08:15 crc kubenswrapper[5052]: I1203 12:08:15.664885 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:08:15 crc kubenswrapper[5052]: I1203 12:08:15.694891 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:08:15 crc kubenswrapper[5052]: I1203 12:08:15.724878 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-scheduler-0" podStartSLOduration=10.724854272 podStartE2EDuration="10.724854272s" podCreationTimestamp="2025-12-03 12:08:05 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:08:07.013992114 +0000 UTC m=+2877.602233878" watchObservedRunningTime="2025-12-03 12:08:15.724854272 +0000 UTC m=+2886.313096036" Dec 03 12:08:16 crc kubenswrapper[5052]: I1203 12:08:16.128315 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:08:18 crc kubenswrapper[5052]: I1203 12:08:18.249921 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:08:19 crc kubenswrapper[5052]: I1203 12:08:19.313546 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:08:19 crc kubenswrapper[5052]: I1203 12:08:19.314801 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:08:19 crc kubenswrapper[5052]: I1203 12:08:19.314930 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:08:19 crc kubenswrapper[5052]: I1203 12:08:19.320974 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:08:20 crc kubenswrapper[5052]: I1203 12:08:20.136710 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:08:20 crc kubenswrapper[5052]: I1203 12:08:20.148124 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:08:23 crc kubenswrapper[5052]: I1203 12:08:23.225124 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:08:23 crc kubenswrapper[5052]: I1203 12:08:23.225557 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:08:23 crc kubenswrapper[5052]: I1203 12:08:23.236775 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:08:23 crc kubenswrapper[5052]: I1203 12:08:23.247332 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:08:25 crc kubenswrapper[5052]: I1203 12:08:25.318133 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/openstackclient"] Dec 03 12:08:25 crc kubenswrapper[5052]: I1203 12:08:25.318614 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/openstackclient" podUID="c7d0ffdb-bb0c-4212-9682-289d630186fd" containerName="openstackclient" containerID="cri-o://dbd211edde83ca19480c688ca164c8a5521ad9be2a864cfa3177c9b6dafa1a07" gracePeriod=2 Dec 03 12:08:25 crc kubenswrapper[5052]: I1203 12:08:25.335850 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/openstackclient"] Dec 03 12:08:25 crc kubenswrapper[5052]: I1203 12:08:25.414128 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/rabbitmq-server-1"] Dec 03 12:08:25 crc kubenswrapper[5052]: I1203 12:08:25.461982 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/rabbitmq-server-0"] Dec 03 12:08:25 crc kubenswrapper[5052]: I1203 12:08:25.496733 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/rabbitmq-server-2"] Dec 03 12:08:25 crc kubenswrapper[5052]: E1203 12:08:25.578701 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/rabbitmq-config-data: configmap "rabbitmq-config-data" not found Dec 03 12:08:25 crc kubenswrapper[5052]: E1203 12:08:25.578783 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2397d896-0d11-4656-a336-eac8ec9fe7f9-config-data podName:2397d896-0d11-4656-a336-eac8ec9fe7f9 nodeName:}" failed. No retries permitted until 2025-12-03 12:08:26.078763097 +0000 UTC m=+2896.667004861 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "config-data" (UniqueName: "kubernetes.io/configmap/2397d896-0d11-4656-a336-eac8ec9fe7f9-config-data") pod "rabbitmq-server-0" (UID: "2397d896-0d11-4656-a336-eac8ec9fe7f9") : configmap "rabbitmq-config-data" not found Dec 03 12:08:25 crc kubenswrapper[5052]: E1203 12:08:25.579309 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/rabbitmq-config-data: configmap "rabbitmq-config-data" not found Dec 03 12:08:25 crc kubenswrapper[5052]: E1203 12:08:25.579585 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7901ce85-4e6f-4e8b-86a6-f50a85803637-config-data podName:7901ce85-4e6f-4e8b-86a6-f50a85803637 nodeName:}" failed. No retries permitted until 2025-12-03 12:08:26.079495998 +0000 UTC m=+2896.667737762 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "config-data" (UniqueName: "kubernetes.io/configmap/7901ce85-4e6f-4e8b-86a6-f50a85803637-config-data") pod "rabbitmq-server-1" (UID: "7901ce85-4e6f-4e8b-86a6-f50a85803637") : configmap "rabbitmq-config-data" not found Dec 03 12:08:25 crc kubenswrapper[5052]: E1203 12:08:25.695839 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/rabbitmq-config-data: configmap "rabbitmq-config-data" not found Dec 03 12:08:25 crc kubenswrapper[5052]: E1203 12:08:25.695895 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7b4f6078-41a3-433d-acaf-f2d67391ab6e-config-data podName:7b4f6078-41a3-433d-acaf-f2d67391ab6e nodeName:}" failed. No retries permitted until 2025-12-03 12:08:26.195881645 +0000 UTC m=+2896.784123409 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "config-data" (UniqueName: "kubernetes.io/configmap/7b4f6078-41a3-433d-acaf-f2d67391ab6e-config-data") pod "rabbitmq-server-2" (UID: "7b4f6078-41a3-433d-acaf-f2d67391ab6e") : configmap "rabbitmq-config-data" not found Dec 03 12:08:25 crc kubenswrapper[5052]: I1203 12:08:25.740689 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/glance8fcf-account-delete-4xhcc"] Dec 03 12:08:25 crc kubenswrapper[5052]: E1203 12:08:25.741136 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="c7d0ffdb-bb0c-4212-9682-289d630186fd" containerName="openstackclient" Dec 03 12:08:25 crc kubenswrapper[5052]: I1203 12:08:25.741163 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="c7d0ffdb-bb0c-4212-9682-289d630186fd" containerName="openstackclient" Dec 03 12:08:25 crc kubenswrapper[5052]: I1203 12:08:25.741444 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="c7d0ffdb-bb0c-4212-9682-289d630186fd" containerName="openstackclient" Dec 03 12:08:25 crc kubenswrapper[5052]: I1203 12:08:25.742237 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ovn-northd-0"] Dec 03 12:08:25 crc kubenswrapper[5052]: I1203 12:08:25.742466 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ovn-northd-0" podUID="fad6f453-acf5-4f13-94d6-7d46e300f28c" containerName="ovn-northd" containerID="cri-o://f82e335d0814000e829c2020ade84de1617dea07a919392e9736c05912d0b4bf" gracePeriod=30 Dec 03 12:08:25 crc kubenswrapper[5052]: I1203 12:08:25.745027 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance8fcf-account-delete-4xhcc" Dec 03 12:08:25 crc kubenswrapper[5052]: I1203 12:08:25.745328 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ovn-northd-0" podUID="fad6f453-acf5-4f13-94d6-7d46e300f28c" containerName="openstack-network-exporter" containerID="cri-o://c15f114c02ee16b234e664feb08dc9f03835f614029770d390cfbeb8fa520a9a" gracePeriod=30 Dec 03 12:08:25 crc kubenswrapper[5052]: I1203 12:08:25.746127 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance8fcf-account-delete-4xhcc"] Dec 03 12:08:25 crc kubenswrapper[5052]: I1203 12:08:25.778132 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-nb-0"] Dec 03 12:08:25 crc kubenswrapper[5052]: I1203 12:08:25.778466 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ovsdbserver-nb-0" podUID="327a4d19-96ba-4b5e-a19f-56ba15a0cc82" containerName="openstack-network-exporter" containerID="cri-o://023f6e55b1e44cead794ab88e34aa43141c551b1a2bce50e249cb4eeec4d2009" gracePeriod=300 Dec 03 12:08:25 crc kubenswrapper[5052]: I1203 12:08:25.828300 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/placement3c40-account-delete-8qsdr"] Dec 03 12:08:25 crc kubenswrapper[5052]: I1203 12:08:25.833284 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement3c40-account-delete-8qsdr" Dec 03 12:08:25 crc kubenswrapper[5052]: I1203 12:08:25.881062 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/placement3c40-account-delete-8qsdr"] Dec 03 12:08:25 crc kubenswrapper[5052]: I1203 12:08:25.914531 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-db-sync-2lq57"] Dec 03 12:08:25 crc kubenswrapper[5052]: I1203 12:08:25.922754 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/rabbitmq-cell1-server-0"] Dec 03 12:08:25 crc kubenswrapper[5052]: I1203 12:08:25.925619 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vjnlh\" (UniqueName: \"kubernetes.io/projected/2f3b4f6a-5f11-4039-a3bd-7b553ab5684d-kube-api-access-vjnlh\") pod \"glance8fcf-account-delete-4xhcc\" (UID: \"2f3b4f6a-5f11-4039-a3bd-7b553ab5684d\") " pod="openstack-kuttl-tests/glance8fcf-account-delete-4xhcc" Dec 03 12:08:25 crc kubenswrapper[5052]: I1203 12:08:25.925990 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/2f3b4f6a-5f11-4039-a3bd-7b553ab5684d-operator-scripts\") pod \"glance8fcf-account-delete-4xhcc\" (UID: \"2f3b4f6a-5f11-4039-a3bd-7b553ab5684d\") " pod="openstack-kuttl-tests/glance8fcf-account-delete-4xhcc" Dec 03 12:08:25 crc kubenswrapper[5052]: I1203 12:08:25.951777 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/ovn-northd-0" podUID="fad6f453-acf5-4f13-94d6-7d46e300f28c" containerName="ovn-northd" probeResult="failure" output=< Dec 03 12:08:25 crc kubenswrapper[5052]: 2025-12-03T12:08:25Z|00001|unixctl|WARN|failed to connect to /tmp/ovn-northd.1.ctl Dec 03 12:08:25 crc kubenswrapper[5052]: ovn-appctl: cannot connect to "/tmp/ovn-northd.1.ctl" (No such file or directory) Dec 03 12:08:25 crc kubenswrapper[5052]: 2025-12-03T12:08:25Z|00001|unixctl|WARN|failed to connect to /tmp/ovn-northd.1.ctl Dec 03 12:08:25 crc kubenswrapper[5052]: ovn-appctl: cannot connect to "/tmp/ovn-northd.1.ctl" (No such file or directory) Dec 03 12:08:25 crc kubenswrapper[5052]: > Dec 03 12:08:25 crc kubenswrapper[5052]: I1203 12:08:25.978222 5052 prober.go:107] "Probe failed" probeType="Liveness" pod="openstack-kuttl-tests/ovn-northd-0" podUID="fad6f453-acf5-4f13-94d6-7d46e300f28c" containerName="ovn-northd" probeResult="failure" output=< Dec 03 12:08:25 crc kubenswrapper[5052]: 2025-12-03T12:08:25Z|00001|unixctl|WARN|failed to connect to /tmp/ovn-northd.1.ctl Dec 03 12:08:25 crc kubenswrapper[5052]: ovn-appctl: cannot connect to "/tmp/ovn-northd.1.ctl" (No such file or directory) Dec 03 12:08:25 crc kubenswrapper[5052]: 2025-12-03T12:08:25Z|00001|unixctl|WARN|failed to connect to /tmp/ovn-northd.1.ctl Dec 03 12:08:25 crc kubenswrapper[5052]: ovn-appctl: cannot connect to "/tmp/ovn-northd.1.ctl" (No such file or directory) Dec 03 12:08:25 crc kubenswrapper[5052]: > Dec 03 12:08:25 crc kubenswrapper[5052]: I1203 12:08:25.993108 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/glance-db-sync-2lq57"] Dec 03 12:08:26 crc kubenswrapper[5052]: I1203 12:08:26.028386 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6bm6b\" (UniqueName: \"kubernetes.io/projected/aaa3db64-a10b-45ea-9e25-de4916f30617-kube-api-access-6bm6b\") pod \"placement3c40-account-delete-8qsdr\" (UID: \"aaa3db64-a10b-45ea-9e25-de4916f30617\") " pod="openstack-kuttl-tests/placement3c40-account-delete-8qsdr" Dec 03 12:08:26 crc kubenswrapper[5052]: I1203 12:08:26.028574 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-vjnlh\" (UniqueName: \"kubernetes.io/projected/2f3b4f6a-5f11-4039-a3bd-7b553ab5684d-kube-api-access-vjnlh\") pod \"glance8fcf-account-delete-4xhcc\" (UID: \"2f3b4f6a-5f11-4039-a3bd-7b553ab5684d\") " pod="openstack-kuttl-tests/glance8fcf-account-delete-4xhcc" Dec 03 12:08:26 crc kubenswrapper[5052]: I1203 12:08:26.028712 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/2f3b4f6a-5f11-4039-a3bd-7b553ab5684d-operator-scripts\") pod \"glance8fcf-account-delete-4xhcc\" (UID: \"2f3b4f6a-5f11-4039-a3bd-7b553ab5684d\") " pod="openstack-kuttl-tests/glance8fcf-account-delete-4xhcc" Dec 03 12:08:26 crc kubenswrapper[5052]: I1203 12:08:26.028769 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/aaa3db64-a10b-45ea-9e25-de4916f30617-operator-scripts\") pod \"placement3c40-account-delete-8qsdr\" (UID: \"aaa3db64-a10b-45ea-9e25-de4916f30617\") " pod="openstack-kuttl-tests/placement3c40-account-delete-8qsdr" Dec 03 12:08:26 crc kubenswrapper[5052]: E1203 12:08:26.029743 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/rabbitmq-cell1-config-data: configmap "rabbitmq-cell1-config-data" not found Dec 03 12:08:26 crc kubenswrapper[5052]: E1203 12:08:26.029833 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/185fdd2a-cd57-45e5-9986-bebd15e5761f-config-data podName:185fdd2a-cd57-45e5-9986-bebd15e5761f nodeName:}" failed. No retries permitted until 2025-12-03 12:08:26.529812322 +0000 UTC m=+2897.118054086 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "config-data" (UniqueName: "kubernetes.io/configmap/185fdd2a-cd57-45e5-9986-bebd15e5761f-config-data") pod "rabbitmq-cell1-server-0" (UID: "185fdd2a-cd57-45e5-9986-bebd15e5761f") : configmap "rabbitmq-cell1-config-data" not found Dec 03 12:08:26 crc kubenswrapper[5052]: I1203 12:08:26.030444 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/2f3b4f6a-5f11-4039-a3bd-7b553ab5684d-operator-scripts\") pod \"glance8fcf-account-delete-4xhcc\" (UID: \"2f3b4f6a-5f11-4039-a3bd-7b553ab5684d\") " pod="openstack-kuttl-tests/glance8fcf-account-delete-4xhcc" Dec 03 12:08:26 crc kubenswrapper[5052]: I1203 12:08:26.030683 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/rabbitmq-cell1-server-2"] Dec 03 12:08:26 crc kubenswrapper[5052]: I1203 12:08:26.076224 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-vjnlh\" (UniqueName: \"kubernetes.io/projected/2f3b4f6a-5f11-4039-a3bd-7b553ab5684d-kube-api-access-vjnlh\") pod \"glance8fcf-account-delete-4xhcc\" (UID: \"2f3b4f6a-5f11-4039-a3bd-7b553ab5684d\") " pod="openstack-kuttl-tests/glance8fcf-account-delete-4xhcc" Dec 03 12:08:26 crc kubenswrapper[5052]: I1203 12:08:26.108272 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance8fcf-account-delete-4xhcc" Dec 03 12:08:26 crc kubenswrapper[5052]: I1203 12:08:26.130454 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/aaa3db64-a10b-45ea-9e25-de4916f30617-operator-scripts\") pod \"placement3c40-account-delete-8qsdr\" (UID: \"aaa3db64-a10b-45ea-9e25-de4916f30617\") " pod="openstack-kuttl-tests/placement3c40-account-delete-8qsdr" Dec 03 12:08:26 crc kubenswrapper[5052]: I1203 12:08:26.130546 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-6bm6b\" (UniqueName: \"kubernetes.io/projected/aaa3db64-a10b-45ea-9e25-de4916f30617-kube-api-access-6bm6b\") pod \"placement3c40-account-delete-8qsdr\" (UID: \"aaa3db64-a10b-45ea-9e25-de4916f30617\") " pod="openstack-kuttl-tests/placement3c40-account-delete-8qsdr" Dec 03 12:08:26 crc kubenswrapper[5052]: E1203 12:08:26.131416 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/rabbitmq-config-data: configmap "rabbitmq-config-data" not found Dec 03 12:08:26 crc kubenswrapper[5052]: I1203 12:08:26.131437 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/aaa3db64-a10b-45ea-9e25-de4916f30617-operator-scripts\") pod \"placement3c40-account-delete-8qsdr\" (UID: \"aaa3db64-a10b-45ea-9e25-de4916f30617\") " pod="openstack-kuttl-tests/placement3c40-account-delete-8qsdr" Dec 03 12:08:26 crc kubenswrapper[5052]: E1203 12:08:26.131457 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7901ce85-4e6f-4e8b-86a6-f50a85803637-config-data podName:7901ce85-4e6f-4e8b-86a6-f50a85803637 nodeName:}" failed. No retries permitted until 2025-12-03 12:08:27.131443828 +0000 UTC m=+2897.719685592 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "config-data" (UniqueName: "kubernetes.io/configmap/7901ce85-4e6f-4e8b-86a6-f50a85803637-config-data") pod "rabbitmq-server-1" (UID: "7901ce85-4e6f-4e8b-86a6-f50a85803637") : configmap "rabbitmq-config-data" not found Dec 03 12:08:26 crc kubenswrapper[5052]: E1203 12:08:26.131573 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/rabbitmq-cell1-config-data: configmap "rabbitmq-cell1-config-data" not found Dec 03 12:08:26 crc kubenswrapper[5052]: E1203 12:08:26.132237 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/rabbitmq-config-data: configmap "rabbitmq-config-data" not found Dec 03 12:08:26 crc kubenswrapper[5052]: E1203 12:08:26.132263 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2397d896-0d11-4656-a336-eac8ec9fe7f9-config-data podName:2397d896-0d11-4656-a336-eac8ec9fe7f9 nodeName:}" failed. No retries permitted until 2025-12-03 12:08:27.132255191 +0000 UTC m=+2897.720496955 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "config-data" (UniqueName: "kubernetes.io/configmap/2397d896-0d11-4656-a336-eac8ec9fe7f9-config-data") pod "rabbitmq-server-0" (UID: "2397d896-0d11-4656-a336-eac8ec9fe7f9") : configmap "rabbitmq-config-data" not found Dec 03 12:08:26 crc kubenswrapper[5052]: E1203 12:08:26.136639 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/06bfd2db-abe0-4b4d-a110-94a6a1e02211-config-data podName:06bfd2db-abe0-4b4d-a110-94a6a1e02211 nodeName:}" failed. No retries permitted until 2025-12-03 12:08:26.636590945 +0000 UTC m=+2897.224832709 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "config-data" (UniqueName: "kubernetes.io/configmap/06bfd2db-abe0-4b4d-a110-94a6a1e02211-config-data") pod "rabbitmq-cell1-server-2" (UID: "06bfd2db-abe0-4b4d-a110-94a6a1e02211") : configmap "rabbitmq-cell1-config-data" not found Dec 03 12:08:26 crc kubenswrapper[5052]: I1203 12:08:26.202655 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-6bm6b\" (UniqueName: \"kubernetes.io/projected/aaa3db64-a10b-45ea-9e25-de4916f30617-kube-api-access-6bm6b\") pod \"placement3c40-account-delete-8qsdr\" (UID: \"aaa3db64-a10b-45ea-9e25-de4916f30617\") " pod="openstack-kuttl-tests/placement3c40-account-delete-8qsdr" Dec 03 12:08:26 crc kubenswrapper[5052]: I1203 12:08:26.220126 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement3c40-account-delete-8qsdr" Dec 03 12:08:26 crc kubenswrapper[5052]: E1203 12:08:26.232743 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/rabbitmq-config-data: configmap "rabbitmq-config-data" not found Dec 03 12:08:26 crc kubenswrapper[5052]: E1203 12:08:26.232806 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7b4f6078-41a3-433d-acaf-f2d67391ab6e-config-data podName:7b4f6078-41a3-433d-acaf-f2d67391ab6e nodeName:}" failed. No retries permitted until 2025-12-03 12:08:27.232790495 +0000 UTC m=+2897.821032259 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "config-data" (UniqueName: "kubernetes.io/configmap/7b4f6078-41a3-433d-acaf-f2d67391ab6e-config-data") pod "rabbitmq-server-2" (UID: "7b4f6078-41a3-433d-acaf-f2d67391ab6e") : configmap "rabbitmq-config-data" not found Dec 03 12:08:26 crc kubenswrapper[5052]: I1203 12:08:26.255983 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/cindere6fc-account-delete-z7w6l"] Dec 03 12:08:26 crc kubenswrapper[5052]: I1203 12:08:26.257401 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cindere6fc-account-delete-z7w6l" Dec 03 12:08:26 crc kubenswrapper[5052]: I1203 12:08:26.289039 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/rabbitmq-cell1-server-1"] Dec 03 12:08:26 crc kubenswrapper[5052]: I1203 12:08:26.319585 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cindere6fc-account-delete-z7w6l"] Dec 03 12:08:26 crc kubenswrapper[5052]: I1203 12:08:26.333890 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d5spc\" (UniqueName: \"kubernetes.io/projected/89ee1bab-2c68-4fc4-9171-a7330922faf6-kube-api-access-d5spc\") pod \"cindere6fc-account-delete-z7w6l\" (UID: \"89ee1bab-2c68-4fc4-9171-a7330922faf6\") " pod="openstack-kuttl-tests/cindere6fc-account-delete-z7w6l" Dec 03 12:08:26 crc kubenswrapper[5052]: I1203 12:08:26.333965 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/89ee1bab-2c68-4fc4-9171-a7330922faf6-operator-scripts\") pod \"cindere6fc-account-delete-z7w6l\" (UID: \"89ee1bab-2c68-4fc4-9171-a7330922faf6\") " pod="openstack-kuttl-tests/cindere6fc-account-delete-z7w6l" Dec 03 12:08:26 crc kubenswrapper[5052]: I1203 12:08:26.355017 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/placement-db-sync-852kv"] Dec 03 12:08:26 crc kubenswrapper[5052]: I1203 12:08:26.370997 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/placement-db-sync-852kv"] Dec 03 12:08:26 crc kubenswrapper[5052]: I1203 12:08:26.413934 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder-db-sync-jhtdm"] Dec 03 12:08:26 crc kubenswrapper[5052]: I1203 12:08:26.428791 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/barbican5c55-account-delete-zttb9"] Dec 03 12:08:26 crc kubenswrapper[5052]: I1203 12:08:26.437778 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican5c55-account-delete-zttb9" Dec 03 12:08:26 crc kubenswrapper[5052]: I1203 12:08:26.440129 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-d5spc\" (UniqueName: \"kubernetes.io/projected/89ee1bab-2c68-4fc4-9171-a7330922faf6-kube-api-access-d5spc\") pod \"cindere6fc-account-delete-z7w6l\" (UID: \"89ee1bab-2c68-4fc4-9171-a7330922faf6\") " pod="openstack-kuttl-tests/cindere6fc-account-delete-z7w6l" Dec 03 12:08:26 crc kubenswrapper[5052]: I1203 12:08:26.440227 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/89ee1bab-2c68-4fc4-9171-a7330922faf6-operator-scripts\") pod \"cindere6fc-account-delete-z7w6l\" (UID: \"89ee1bab-2c68-4fc4-9171-a7330922faf6\") " pod="openstack-kuttl-tests/cindere6fc-account-delete-z7w6l" Dec 03 12:08:26 crc kubenswrapper[5052]: E1203 12:08:26.440802 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/rabbitmq-cell1-config-data: configmap "rabbitmq-cell1-config-data" not found Dec 03 12:08:26 crc kubenswrapper[5052]: E1203 12:08:26.440856 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3-config-data podName:f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3 nodeName:}" failed. No retries permitted until 2025-12-03 12:08:26.940839803 +0000 UTC m=+2897.529081567 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "config-data" (UniqueName: "kubernetes.io/configmap/f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3-config-data") pod "rabbitmq-cell1-server-1" (UID: "f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3") : configmap "rabbitmq-cell1-config-data" not found Dec 03 12:08:26 crc kubenswrapper[5052]: I1203 12:08:26.441310 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/89ee1bab-2c68-4fc4-9171-a7330922faf6-operator-scripts\") pod \"cindere6fc-account-delete-z7w6l\" (UID: \"89ee1bab-2c68-4fc4-9171-a7330922faf6\") " pod="openstack-kuttl-tests/cindere6fc-account-delete-z7w6l" Dec 03 12:08:26 crc kubenswrapper[5052]: I1203 12:08:26.443691 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/cinder-db-sync-jhtdm"] Dec 03 12:08:26 crc kubenswrapper[5052]: I1203 12:08:26.461591 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican5c55-account-delete-zttb9"] Dec 03 12:08:26 crc kubenswrapper[5052]: I1203 12:08:26.462798 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-d5spc\" (UniqueName: \"kubernetes.io/projected/89ee1bab-2c68-4fc4-9171-a7330922faf6-kube-api-access-d5spc\") pod \"cindere6fc-account-delete-z7w6l\" (UID: \"89ee1bab-2c68-4fc4-9171-a7330922faf6\") " pod="openstack-kuttl-tests/cindere6fc-account-delete-z7w6l" Dec 03 12:08:26 crc kubenswrapper[5052]: I1203 12:08:26.474223 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/neutrond5c4-account-delete-zj77r"] Dec 03 12:08:26 crc kubenswrapper[5052]: I1203 12:08:26.475577 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutrond5c4-account-delete-zj77r" Dec 03 12:08:26 crc kubenswrapper[5052]: I1203 12:08:26.481653 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutrond5c4-account-delete-zj77r"] Dec 03 12:08:26 crc kubenswrapper[5052]: I1203 12:08:26.489354 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/novacell0d294-account-delete-b64wn"] Dec 03 12:08:26 crc kubenswrapper[5052]: I1203 12:08:26.495812 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/novacell0d294-account-delete-b64wn" Dec 03 12:08:26 crc kubenswrapper[5052]: I1203 12:08:26.498565 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/novacell0d294-account-delete-b64wn"] Dec 03 12:08:26 crc kubenswrapper[5052]: I1203 12:08:26.520631 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-sb-0"] Dec 03 12:08:26 crc kubenswrapper[5052]: I1203 12:08:26.521353 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ovsdbserver-sb-0" podUID="78dad0d3-18e5-4900-932c-56ede3279dc0" containerName="openstack-network-exporter" containerID="cri-o://b746c813f9a2abaab7eee298828559f29ccd2f1796875dbbe947a37faa740dfb" gracePeriod=300 Dec 03 12:08:26 crc kubenswrapper[5052]: I1203 12:08:26.522268 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ovsdbserver-nb-0" podUID="327a4d19-96ba-4b5e-a19f-56ba15a0cc82" containerName="ovsdbserver-nb" containerID="cri-o://04c71fa1d03c17ad50a53dfb4585c12716ea5f19b448fb01a9fbd6d86935e208" gracePeriod=300 Dec 03 12:08:26 crc kubenswrapper[5052]: I1203 12:08:26.541660 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hkwq9\" (UniqueName: \"kubernetes.io/projected/9e6af047-b7ee-459a-b362-da5f777d0fa0-kube-api-access-hkwq9\") pod \"neutrond5c4-account-delete-zj77r\" (UID: \"9e6af047-b7ee-459a-b362-da5f777d0fa0\") " pod="openstack-kuttl-tests/neutrond5c4-account-delete-zj77r" Dec 03 12:08:26 crc kubenswrapper[5052]: I1203 12:08:26.541719 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/9e6af047-b7ee-459a-b362-da5f777d0fa0-operator-scripts\") pod \"neutrond5c4-account-delete-zj77r\" (UID: \"9e6af047-b7ee-459a-b362-da5f777d0fa0\") " pod="openstack-kuttl-tests/neutrond5c4-account-delete-zj77r" Dec 03 12:08:26 crc kubenswrapper[5052]: I1203 12:08:26.541758 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-drvct\" (UniqueName: \"kubernetes.io/projected/1bc7c8b1-cc0d-4611-b335-01c2b91713e2-kube-api-access-drvct\") pod \"barbican5c55-account-delete-zttb9\" (UID: \"1bc7c8b1-cc0d-4611-b335-01c2b91713e2\") " pod="openstack-kuttl-tests/barbican5c55-account-delete-zttb9" Dec 03 12:08:26 crc kubenswrapper[5052]: I1203 12:08:26.541776 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/1bc7c8b1-cc0d-4611-b335-01c2b91713e2-operator-scripts\") pod \"barbican5c55-account-delete-zttb9\" (UID: \"1bc7c8b1-cc0d-4611-b335-01c2b91713e2\") " pod="openstack-kuttl-tests/barbican5c55-account-delete-zttb9" Dec 03 12:08:26 crc kubenswrapper[5052]: E1203 12:08:26.543377 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/rabbitmq-cell1-config-data: configmap "rabbitmq-cell1-config-data" not found Dec 03 12:08:26 crc kubenswrapper[5052]: E1203 12:08:26.543431 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/185fdd2a-cd57-45e5-9986-bebd15e5761f-config-data podName:185fdd2a-cd57-45e5-9986-bebd15e5761f nodeName:}" failed. No retries permitted until 2025-12-03 12:08:27.543414676 +0000 UTC m=+2898.131656430 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "config-data" (UniqueName: "kubernetes.io/configmap/185fdd2a-cd57-45e5-9986-bebd15e5761f-config-data") pod "rabbitmq-cell1-server-0" (UID: "185fdd2a-cd57-45e5-9986-bebd15e5761f") : configmap "rabbitmq-cell1-config-data" not found Dec 03 12:08:26 crc kubenswrapper[5052]: I1203 12:08:26.548321 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/novaapi48e2-account-delete-kqhz7"] Dec 03 12:08:26 crc kubenswrapper[5052]: I1203 12:08:26.572696 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/novaapi48e2-account-delete-kqhz7"] Dec 03 12:08:26 crc kubenswrapper[5052]: I1203 12:08:26.572802 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/novaapi48e2-account-delete-kqhz7" Dec 03 12:08:26 crc kubenswrapper[5052]: I1203 12:08:26.580802 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-db-sync-4z2hl"] Dec 03 12:08:26 crc kubenswrapper[5052]: I1203 12:08:26.595311 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ovsdbserver-sb-0" podUID="78dad0d3-18e5-4900-932c-56ede3279dc0" containerName="ovsdbserver-sb" containerID="cri-o://3fa9a29a156d24f947438d4dd3ac36264903dbd4e777dddc9352f1aab01266e5" gracePeriod=300 Dec 03 12:08:26 crc kubenswrapper[5052]: I1203 12:08:26.601639 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cindere6fc-account-delete-z7w6l" Dec 03 12:08:26 crc kubenswrapper[5052]: I1203 12:08:26.626669 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/barbican-db-sync-4z2hl"] Dec 03 12:08:26 crc kubenswrapper[5052]: I1203 12:08:26.655676 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2j7sw\" (UniqueName: \"kubernetes.io/projected/70122e67-6175-4a14-a120-6a2731c5dcb1-kube-api-access-2j7sw\") pod \"novaapi48e2-account-delete-kqhz7\" (UID: \"70122e67-6175-4a14-a120-6a2731c5dcb1\") " pod="openstack-kuttl-tests/novaapi48e2-account-delete-kqhz7" Dec 03 12:08:26 crc kubenswrapper[5052]: I1203 12:08:26.655910 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-hkwq9\" (UniqueName: \"kubernetes.io/projected/9e6af047-b7ee-459a-b362-da5f777d0fa0-kube-api-access-hkwq9\") pod \"neutrond5c4-account-delete-zj77r\" (UID: \"9e6af047-b7ee-459a-b362-da5f777d0fa0\") " pod="openstack-kuttl-tests/neutrond5c4-account-delete-zj77r" Dec 03 12:08:26 crc kubenswrapper[5052]: I1203 12:08:26.656062 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/9e6af047-b7ee-459a-b362-da5f777d0fa0-operator-scripts\") pod \"neutrond5c4-account-delete-zj77r\" (UID: \"9e6af047-b7ee-459a-b362-da5f777d0fa0\") " pod="openstack-kuttl-tests/neutrond5c4-account-delete-zj77r" Dec 03 12:08:26 crc kubenswrapper[5052]: I1203 12:08:26.656151 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-drvct\" (UniqueName: \"kubernetes.io/projected/1bc7c8b1-cc0d-4611-b335-01c2b91713e2-kube-api-access-drvct\") pod \"barbican5c55-account-delete-zttb9\" (UID: \"1bc7c8b1-cc0d-4611-b335-01c2b91713e2\") " pod="openstack-kuttl-tests/barbican5c55-account-delete-zttb9" Dec 03 12:08:26 crc kubenswrapper[5052]: I1203 12:08:26.656182 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/1bc7c8b1-cc0d-4611-b335-01c2b91713e2-operator-scripts\") pod \"barbican5c55-account-delete-zttb9\" (UID: \"1bc7c8b1-cc0d-4611-b335-01c2b91713e2\") " pod="openstack-kuttl-tests/barbican5c55-account-delete-zttb9" Dec 03 12:08:26 crc kubenswrapper[5052]: I1203 12:08:26.656341 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/030d4f76-eb30-4c44-b782-fa7147febf41-operator-scripts\") pod \"novacell0d294-account-delete-b64wn\" (UID: \"030d4f76-eb30-4c44-b782-fa7147febf41\") " pod="openstack-kuttl-tests/novacell0d294-account-delete-b64wn" Dec 03 12:08:26 crc kubenswrapper[5052]: I1203 12:08:26.656366 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6t26c\" (UniqueName: \"kubernetes.io/projected/030d4f76-eb30-4c44-b782-fa7147febf41-kube-api-access-6t26c\") pod \"novacell0d294-account-delete-b64wn\" (UID: \"030d4f76-eb30-4c44-b782-fa7147febf41\") " pod="openstack-kuttl-tests/novacell0d294-account-delete-b64wn" Dec 03 12:08:26 crc kubenswrapper[5052]: I1203 12:08:26.656399 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/70122e67-6175-4a14-a120-6a2731c5dcb1-operator-scripts\") pod \"novaapi48e2-account-delete-kqhz7\" (UID: \"70122e67-6175-4a14-a120-6a2731c5dcb1\") " pod="openstack-kuttl-tests/novaapi48e2-account-delete-kqhz7" Dec 03 12:08:26 crc kubenswrapper[5052]: E1203 12:08:26.656768 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/rabbitmq-cell1-config-data: configmap "rabbitmq-cell1-config-data" not found Dec 03 12:08:26 crc kubenswrapper[5052]: E1203 12:08:26.656889 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/06bfd2db-abe0-4b4d-a110-94a6a1e02211-config-data podName:06bfd2db-abe0-4b4d-a110-94a6a1e02211 nodeName:}" failed. No retries permitted until 2025-12-03 12:08:27.65686779 +0000 UTC m=+2898.245109554 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "config-data" (UniqueName: "kubernetes.io/configmap/06bfd2db-abe0-4b4d-a110-94a6a1e02211-config-data") pod "rabbitmq-cell1-server-2" (UID: "06bfd2db-abe0-4b4d-a110-94a6a1e02211") : configmap "rabbitmq-cell1-config-data" not found Dec 03 12:08:26 crc kubenswrapper[5052]: I1203 12:08:26.658429 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/1bc7c8b1-cc0d-4611-b335-01c2b91713e2-operator-scripts\") pod \"barbican5c55-account-delete-zttb9\" (UID: \"1bc7c8b1-cc0d-4611-b335-01c2b91713e2\") " pod="openstack-kuttl-tests/barbican5c55-account-delete-zttb9" Dec 03 12:08:26 crc kubenswrapper[5052]: I1203 12:08:26.669691 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/9e6af047-b7ee-459a-b362-da5f777d0fa0-operator-scripts\") pod \"neutrond5c4-account-delete-zj77r\" (UID: \"9e6af047-b7ee-459a-b362-da5f777d0fa0\") " pod="openstack-kuttl-tests/neutrond5c4-account-delete-zj77r" Dec 03 12:08:26 crc kubenswrapper[5052]: I1203 12:08:26.689872 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-hkwq9\" (UniqueName: \"kubernetes.io/projected/9e6af047-b7ee-459a-b362-da5f777d0fa0-kube-api-access-hkwq9\") pod \"neutrond5c4-account-delete-zj77r\" (UID: \"9e6af047-b7ee-459a-b362-da5f777d0fa0\") " pod="openstack-kuttl-tests/neutrond5c4-account-delete-zj77r" Dec 03 12:08:26 crc kubenswrapper[5052]: I1203 12:08:26.694709 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-drvct\" (UniqueName: \"kubernetes.io/projected/1bc7c8b1-cc0d-4611-b335-01c2b91713e2-kube-api-access-drvct\") pod \"barbican5c55-account-delete-zttb9\" (UID: \"1bc7c8b1-cc0d-4611-b335-01c2b91713e2\") " pod="openstack-kuttl-tests/barbican5c55-account-delete-zttb9" Dec 03 12:08:26 crc kubenswrapper[5052]: I1203 12:08:26.703130 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/placement-f4dfbd7d-mxsm8"] Dec 03 12:08:26 crc kubenswrapper[5052]: I1203 12:08:26.705394 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/placement-f4dfbd7d-mxsm8" podUID="6999e86b-8f95-4002-9959-6288a7d783bd" containerName="placement-api" containerID="cri-o://3e8fef46e8b29b153dd3541055f4f411078ff175402ded3c5e2a2532584522cb" gracePeriod=30 Dec 03 12:08:26 crc kubenswrapper[5052]: I1203 12:08:26.705230 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/placement-f4dfbd7d-mxsm8" podUID="6999e86b-8f95-4002-9959-6288a7d783bd" containerName="placement-log" containerID="cri-o://18f9fa6fd7bb783b320dadc515c1e38c7a2e0212224d2e211b526e64320731a1" gracePeriod=30 Dec 03 12:08:26 crc kubenswrapper[5052]: I1203 12:08:26.722094 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:08:26 crc kubenswrapper[5052]: I1203 12:08:26.722611 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-internal-api-0" podUID="4a2a1881-e61a-4a99-a2f3-b614aeed81d1" containerName="glance-log" containerID="cri-o://e3e2ee8f299225cb3b6df3bf35cd7781dcb1aa91cfcdbcdc6d9ceee107ca48ce" gracePeriod=30 Dec 03 12:08:26 crc kubenswrapper[5052]: I1203 12:08:26.723254 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-internal-api-0" podUID="4a2a1881-e61a-4a99-a2f3-b614aeed81d1" containerName="glance-httpd" containerID="cri-o://95dbf47cff378e5b90c76418048911cfa106d1ea36f9592e66a0f3a368e19e25" gracePeriod=30 Dec 03 12:08:26 crc kubenswrapper[5052]: I1203 12:08:26.773890 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/neutron-db-sync-lrrtz"] Dec 03 12:08:26 crc kubenswrapper[5052]: I1203 12:08:26.781073 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican5c55-account-delete-zttb9" Dec 03 12:08:26 crc kubenswrapper[5052]: I1203 12:08:26.782202 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/030d4f76-eb30-4c44-b782-fa7147febf41-operator-scripts\") pod \"novacell0d294-account-delete-b64wn\" (UID: \"030d4f76-eb30-4c44-b782-fa7147febf41\") " pod="openstack-kuttl-tests/novacell0d294-account-delete-b64wn" Dec 03 12:08:26 crc kubenswrapper[5052]: I1203 12:08:26.782236 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-6t26c\" (UniqueName: \"kubernetes.io/projected/030d4f76-eb30-4c44-b782-fa7147febf41-kube-api-access-6t26c\") pod \"novacell0d294-account-delete-b64wn\" (UID: \"030d4f76-eb30-4c44-b782-fa7147febf41\") " pod="openstack-kuttl-tests/novacell0d294-account-delete-b64wn" Dec 03 12:08:26 crc kubenswrapper[5052]: I1203 12:08:26.782262 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/70122e67-6175-4a14-a120-6a2731c5dcb1-operator-scripts\") pod \"novaapi48e2-account-delete-kqhz7\" (UID: \"70122e67-6175-4a14-a120-6a2731c5dcb1\") " pod="openstack-kuttl-tests/novaapi48e2-account-delete-kqhz7" Dec 03 12:08:26 crc kubenswrapper[5052]: I1203 12:08:26.782373 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-2j7sw\" (UniqueName: \"kubernetes.io/projected/70122e67-6175-4a14-a120-6a2731c5dcb1-kube-api-access-2j7sw\") pod \"novaapi48e2-account-delete-kqhz7\" (UID: \"70122e67-6175-4a14-a120-6a2731c5dcb1\") " pod="openstack-kuttl-tests/novaapi48e2-account-delete-kqhz7" Dec 03 12:08:26 crc kubenswrapper[5052]: I1203 12:08:26.784653 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/70122e67-6175-4a14-a120-6a2731c5dcb1-operator-scripts\") pod \"novaapi48e2-account-delete-kqhz7\" (UID: \"70122e67-6175-4a14-a120-6a2731c5dcb1\") " pod="openstack-kuttl-tests/novaapi48e2-account-delete-kqhz7" Dec 03 12:08:26 crc kubenswrapper[5052]: I1203 12:08:26.785347 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/030d4f76-eb30-4c44-b782-fa7147febf41-operator-scripts\") pod \"novacell0d294-account-delete-b64wn\" (UID: \"030d4f76-eb30-4c44-b782-fa7147febf41\") " pod="openstack-kuttl-tests/novacell0d294-account-delete-b64wn" Dec 03 12:08:26 crc kubenswrapper[5052]: I1203 12:08:26.804724 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/neutron-db-sync-lrrtz"] Dec 03 12:08:26 crc kubenswrapper[5052]: I1203 12:08:26.812018 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutrond5c4-account-delete-zj77r" Dec 03 12:08:26 crc kubenswrapper[5052]: I1203 12:08:26.831001 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-6t26c\" (UniqueName: \"kubernetes.io/projected/030d4f76-eb30-4c44-b782-fa7147febf41-kube-api-access-6t26c\") pod \"novacell0d294-account-delete-b64wn\" (UID: \"030d4f76-eb30-4c44-b782-fa7147febf41\") " pod="openstack-kuttl-tests/novacell0d294-account-delete-b64wn" Dec 03 12:08:26 crc kubenswrapper[5052]: I1203 12:08:26.859557 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-2j7sw\" (UniqueName: \"kubernetes.io/projected/70122e67-6175-4a14-a120-6a2731c5dcb1-kube-api-access-2j7sw\") pod \"novaapi48e2-account-delete-kqhz7\" (UID: \"70122e67-6175-4a14-a120-6a2731c5dcb1\") " pod="openstack-kuttl-tests/novaapi48e2-account-delete-kqhz7" Dec 03 12:08:26 crc kubenswrapper[5052]: I1203 12:08:26.862937 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/novacell0d294-account-delete-b64wn" Dec 03 12:08:26 crc kubenswrapper[5052]: I1203 12:08:26.871419 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell0-cell-mapping-5r7kw"] Dec 03 12:08:26 crc kubenswrapper[5052]: I1203 12:08:26.927057 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/novaapi48e2-account-delete-kqhz7" Dec 03 12:08:26 crc kubenswrapper[5052]: I1203 12:08:26.928257 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell0-cell-mapping-5r7kw"] Dec 03 12:08:26 crc kubenswrapper[5052]: I1203 12:08:26.942925 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell1-cell-mapping-95t79"] Dec 03 12:08:26 crc kubenswrapper[5052]: I1203 12:08:26.955755 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell1-cell-mapping-95t79"] Dec 03 12:08:26 crc kubenswrapper[5052]: E1203 12:08:26.992784 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/rabbitmq-cell1-config-data: configmap "rabbitmq-cell1-config-data" not found Dec 03 12:08:26 crc kubenswrapper[5052]: E1203 12:08:26.992869 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3-config-data podName:f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3 nodeName:}" failed. No retries permitted until 2025-12-03 12:08:27.992848595 +0000 UTC m=+2898.581090359 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "config-data" (UniqueName: "kubernetes.io/configmap/f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3-config-data") pod "rabbitmq-cell1-server-1" (UID: "f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3") : configmap "rabbitmq-cell1-config-data" not found Dec 03 12:08:27 crc kubenswrapper[5052]: I1203 12:08:27.007693 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/swift-ring-rebalance-g9qd6"] Dec 03 12:08:27 crc kubenswrapper[5052]: I1203 12:08:27.023616 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/swift-ring-rebalance-g9qd6"] Dec 03 12:08:27 crc kubenswrapper[5052]: I1203 12:08:27.034636 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance8fcf-account-delete-4xhcc"] Dec 03 12:08:27 crc kubenswrapper[5052]: I1203 12:08:27.051967 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:08:27 crc kubenswrapper[5052]: I1203 12:08:27.052873 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-external-api-0" podUID="bef08f40-1b01-406f-af05-7b63e0b6e5e7" containerName="glance-log" containerID="cri-o://62853e68ee96c13bce5c6927e9083bfe1dca5e094c03354b4a342f37a5ac3ce4" gracePeriod=30 Dec 03 12:08:27 crc kubenswrapper[5052]: I1203 12:08:27.052941 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-external-api-0" podUID="bef08f40-1b01-406f-af05-7b63e0b6e5e7" containerName="glance-httpd" containerID="cri-o://df8444be8224ec47d3e75542bba9c0aaa28cc5e0ba5b753982c10dfdcf96b872" gracePeriod=30 Dec 03 12:08:27 crc kubenswrapper[5052]: I1203 12:08:27.122890 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 12:08:27 crc kubenswrapper[5052]: I1203 12:08:27.123804 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/cinder-scheduler-0" podUID="60f72892-a5aa-482f-aac3-5713ba028bc8" containerName="cinder-scheduler" containerID="cri-o://082794493f1a866da7efb8038f854ca6442bb335e30fc8e05b37438e570e2b64" gracePeriod=30 Dec 03 12:08:27 crc kubenswrapper[5052]: I1203 12:08:27.124318 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/cinder-scheduler-0" podUID="60f72892-a5aa-482f-aac3-5713ba028bc8" containerName="probe" containerID="cri-o://6e5a8a55dfb233734ce359d756ebf08a407cc39b08dc02754ac067f403d0a466" gracePeriod=30 Dec 03 12:08:27 crc kubenswrapper[5052]: E1203 12:08:27.206903 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/rabbitmq-config-data: configmap "rabbitmq-config-data" not found Dec 03 12:08:27 crc kubenswrapper[5052]: E1203 12:08:27.207198 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7901ce85-4e6f-4e8b-86a6-f50a85803637-config-data podName:7901ce85-4e6f-4e8b-86a6-f50a85803637 nodeName:}" failed. No retries permitted until 2025-12-03 12:08:29.207182503 +0000 UTC m=+2899.795424267 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "config-data" (UniqueName: "kubernetes.io/configmap/7901ce85-4e6f-4e8b-86a6-f50a85803637-config-data") pod "rabbitmq-server-1" (UID: "7901ce85-4e6f-4e8b-86a6-f50a85803637") : configmap "rabbitmq-config-data" not found Dec 03 12:08:27 crc kubenswrapper[5052]: E1203 12:08:27.207601 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/rabbitmq-config-data: configmap "rabbitmq-config-data" not found Dec 03 12:08:27 crc kubenswrapper[5052]: E1203 12:08:27.207700 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2397d896-0d11-4656-a336-eac8ec9fe7f9-config-data podName:2397d896-0d11-4656-a336-eac8ec9fe7f9 nodeName:}" failed. No retries permitted until 2025-12-03 12:08:29.207691468 +0000 UTC m=+2899.795933222 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "config-data" (UniqueName: "kubernetes.io/configmap/2397d896-0d11-4656-a336-eac8ec9fe7f9-config-data") pod "rabbitmq-server-0" (UID: "2397d896-0d11-4656-a336-eac8ec9fe7f9") : configmap "rabbitmq-config-data" not found Dec 03 12:08:27 crc kubenswrapper[5052]: I1203 12:08:27.231161 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 12:08:27 crc kubenswrapper[5052]: I1203 12:08:27.231467 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/cinder-api-0" podUID="31e6a111-bd4a-45a5-95a0-a6b537added6" containerName="cinder-api-log" containerID="cri-o://84283f0da62e2076f919bc27137ad503eb09cc0b4a9e5ff7119c2f488722a65c" gracePeriod=30 Dec 03 12:08:27 crc kubenswrapper[5052]: I1203 12:08:27.232086 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/cinder-api-0" podUID="31e6a111-bd4a-45a5-95a0-a6b537added6" containerName="cinder-api" containerID="cri-o://545a199ec97461f4a64454b6edecdfc68e50742c7e11692685b952917d9c8b1e" gracePeriod=30 Dec 03 12:08:27 crc kubenswrapper[5052]: E1203 12:08:27.315184 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/rabbitmq-config-data: configmap "rabbitmq-config-data" not found Dec 03 12:08:27 crc kubenswrapper[5052]: E1203 12:08:27.323747 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7b4f6078-41a3-433d-acaf-f2d67391ab6e-config-data podName:7b4f6078-41a3-433d-acaf-f2d67391ab6e nodeName:}" failed. No retries permitted until 2025-12-03 12:08:29.323717985 +0000 UTC m=+2899.911959749 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "config-data" (UniqueName: "kubernetes.io/configmap/7b4f6078-41a3-433d-acaf-f2d67391ab6e-config-data") pod "rabbitmq-server-2" (UID: "7b4f6078-41a3-433d-acaf-f2d67391ab6e") : configmap "rabbitmq-config-data" not found Dec 03 12:08:27 crc kubenswrapper[5052]: I1203 12:08:27.367909 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/rabbitmq-server-1"] Dec 03 12:08:27 crc kubenswrapper[5052]: I1203 12:08:27.382492 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/rabbitmq-server-0"] Dec 03 12:08:27 crc kubenswrapper[5052]: I1203 12:08:27.387233 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance8fcf-account-delete-4xhcc" event={"ID":"2f3b4f6a-5f11-4039-a3bd-7b553ab5684d","Type":"ContainerStarted","Data":"9276b3afe43ed2cac6cf8ba87bf8a179c142f8c59beb0358ba1e9657f5941a71"} Dec 03 12:08:27 crc kubenswrapper[5052]: I1203 12:08:27.399575 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/rabbitmq-server-2"] Dec 03 12:08:27 crc kubenswrapper[5052]: E1203 12:08:27.428704 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of 04c71fa1d03c17ad50a53dfb4585c12716ea5f19b448fb01a9fbd6d86935e208 is running failed: container process not found" containerID="04c71fa1d03c17ad50a53dfb4585c12716ea5f19b448fb01a9fbd6d86935e208" cmd=["/usr/bin/pidof","ovsdb-server"] Dec 03 12:08:27 crc kubenswrapper[5052]: E1203 12:08:27.443899 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of 04c71fa1d03c17ad50a53dfb4585c12716ea5f19b448fb01a9fbd6d86935e208 is running failed: container process not found" containerID="04c71fa1d03c17ad50a53dfb4585c12716ea5f19b448fb01a9fbd6d86935e208" cmd=["/usr/bin/pidof","ovsdb-server"] Dec 03 12:08:27 crc kubenswrapper[5052]: E1203 12:08:27.447100 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of 04c71fa1d03c17ad50a53dfb4585c12716ea5f19b448fb01a9fbd6d86935e208 is running failed: container process not found" containerID="04c71fa1d03c17ad50a53dfb4585c12716ea5f19b448fb01a9fbd6d86935e208" cmd=["/usr/bin/pidof","ovsdb-server"] Dec 03 12:08:27 crc kubenswrapper[5052]: E1203 12:08:27.453179 5052 prober.go:104] "Probe errored" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of 04c71fa1d03c17ad50a53dfb4585c12716ea5f19b448fb01a9fbd6d86935e208 is running failed: container process not found" probeType="Readiness" pod="openstack-kuttl-tests/ovsdbserver-nb-0" podUID="327a4d19-96ba-4b5e-a19f-56ba15a0cc82" containerName="ovsdbserver-nb" Dec 03 12:08:27 crc kubenswrapper[5052]: I1203 12:08:27.448983 5052 generic.go:334] "Generic (PLEG): container finished" podID="fad6f453-acf5-4f13-94d6-7d46e300f28c" containerID="c15f114c02ee16b234e664feb08dc9f03835f614029770d390cfbeb8fa520a9a" exitCode=2 Dec 03 12:08:27 crc kubenswrapper[5052]: I1203 12:08:27.450271 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/rabbitmq-server-1" podUID="7901ce85-4e6f-4e8b-86a6-f50a85803637" containerName="rabbitmq" containerID="cri-o://88ec697999833a01b97e7879439a3382717780668a2e4ef174d64567bf8d897a" gracePeriod=604800 Dec 03 12:08:27 crc kubenswrapper[5052]: I1203 12:08:27.449011 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovn-northd-0" event={"ID":"fad6f453-acf5-4f13-94d6-7d46e300f28c","Type":"ContainerDied","Data":"c15f114c02ee16b234e664feb08dc9f03835f614029770d390cfbeb8fa520a9a"} Dec 03 12:08:27 crc kubenswrapper[5052]: I1203 12:08:27.458405 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/swift-storage-0"] Dec 03 12:08:27 crc kubenswrapper[5052]: I1203 12:08:27.459009 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="95bf5f72-82f3-4e88-a576-01dae857541f" containerName="account-server" containerID="cri-o://003e4620ad1989f1fa65349bb9193c7be42568b4061355bee57a6b9ae2b3d5a2" gracePeriod=30 Dec 03 12:08:27 crc kubenswrapper[5052]: I1203 12:08:27.459152 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="95bf5f72-82f3-4e88-a576-01dae857541f" containerName="object-server" containerID="cri-o://da24ecbec4ad70b7a0db3646cffcca7dc56b37efffe6e261de63720e146b2b8e" gracePeriod=30 Dec 03 12:08:27 crc kubenswrapper[5052]: I1203 12:08:27.459330 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="95bf5f72-82f3-4e88-a576-01dae857541f" containerName="container-updater" containerID="cri-o://2577c8659d7bc84c157b82cc64e9d927df3b8f1b4fe022b955ef7fe05c5dadbb" gracePeriod=30 Dec 03 12:08:27 crc kubenswrapper[5052]: I1203 12:08:27.459485 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="95bf5f72-82f3-4e88-a576-01dae857541f" containerName="container-auditor" containerID="cri-o://a4050b9976060a4f925ffaa2cb889b98f3b107441d8ca49ad092c8be65814639" gracePeriod=30 Dec 03 12:08:27 crc kubenswrapper[5052]: I1203 12:08:27.459689 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="95bf5f72-82f3-4e88-a576-01dae857541f" containerName="swift-recon-cron" containerID="cri-o://c0e56624ca96c43f72b2c3cc860c510ab86560649ca040b145fbe1cdc7988153" gracePeriod=30 Dec 03 12:08:27 crc kubenswrapper[5052]: I1203 12:08:27.459702 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="95bf5f72-82f3-4e88-a576-01dae857541f" containerName="rsync" containerID="cri-o://4a4d3e25edd055d3f3503821db30046cfd815c08421228eb4d5e76a3f4e38a23" gracePeriod=30 Dec 03 12:08:27 crc kubenswrapper[5052]: I1203 12:08:27.459715 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="95bf5f72-82f3-4e88-a576-01dae857541f" containerName="object-expirer" containerID="cri-o://2d047aaa650efd207aa517746b6cde98bb8bae986bd03e7b545eda7fd36f1023" gracePeriod=30 Dec 03 12:08:27 crc kubenswrapper[5052]: I1203 12:08:27.459727 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="95bf5f72-82f3-4e88-a576-01dae857541f" containerName="object-updater" containerID="cri-o://c6d65f1e5859c5423b2bed9ad8130035fdbcdb06ce5b34713168ebc78916b853" gracePeriod=30 Dec 03 12:08:27 crc kubenswrapper[5052]: I1203 12:08:27.459735 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="95bf5f72-82f3-4e88-a576-01dae857541f" containerName="object-auditor" containerID="cri-o://b1f5592550eedafe273eda6ddd62bcf2957cdd483d534a5369391fc101829850" gracePeriod=30 Dec 03 12:08:27 crc kubenswrapper[5052]: I1203 12:08:27.459744 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="95bf5f72-82f3-4e88-a576-01dae857541f" containerName="object-replicator" containerID="cri-o://ff53c665fe25113ca5552ca3edb72dc98d7ffacf8d96d01a610c8292d0ca0504" gracePeriod=30 Dec 03 12:08:27 crc kubenswrapper[5052]: I1203 12:08:27.459770 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="95bf5f72-82f3-4e88-a576-01dae857541f" containerName="account-reaper" containerID="cri-o://2732548b8572203d895c3e589eef6680d882e9fcbe498714de0778d4c35813ec" gracePeriod=30 Dec 03 12:08:27 crc kubenswrapper[5052]: I1203 12:08:27.459779 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="95bf5f72-82f3-4e88-a576-01dae857541f" containerName="container-replicator" containerID="cri-o://ae9aaced3b606148dcf3e097c33141f00f123ef8589615b163bff1470454ae80" gracePeriod=30 Dec 03 12:08:27 crc kubenswrapper[5052]: I1203 12:08:27.459788 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="95bf5f72-82f3-4e88-a576-01dae857541f" containerName="container-server" containerID="cri-o://9fa20306dd177e25e71d2c3ec088ca38dc2879ff9f7569b37d3b1ab695d20bed" gracePeriod=30 Dec 03 12:08:27 crc kubenswrapper[5052]: I1203 12:08:27.459797 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="95bf5f72-82f3-4e88-a576-01dae857541f" containerName="account-auditor" containerID="cri-o://6a7d071e07e4e8b8e192642aeae3ccfa4e50a1bfae563ab609f77db309009d6d" gracePeriod=30 Dec 03 12:08:27 crc kubenswrapper[5052]: I1203 12:08:27.459806 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="95bf5f72-82f3-4e88-a576-01dae857541f" containerName="account-replicator" containerID="cri-o://3604bfccbdf55c7f8cc4aa8921d57387c338bcccd177ab45652bc037ec8fdad6" gracePeriod=30 Dec 03 12:08:27 crc kubenswrapper[5052]: I1203 12:08:27.489278 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/neutron-5d6b56fbdb-ht2b8"] Dec 03 12:08:27 crc kubenswrapper[5052]: I1203 12:08:27.489602 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/neutron-5d6b56fbdb-ht2b8" podUID="141a49da-93b6-483b-a476-1d5f3efc29d1" containerName="neutron-api" containerID="cri-o://3d0ba918e75cd950bc4bab7a527b72d23c454cb5e03d5804f02c37caed27d0fa" gracePeriod=30 Dec 03 12:08:27 crc kubenswrapper[5052]: I1203 12:08:27.490116 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/neutron-5d6b56fbdb-ht2b8" podUID="141a49da-93b6-483b-a476-1d5f3efc29d1" containerName="neutron-httpd" containerID="cri-o://405640dc668d7b8021968432854058e5508074169be4f1e98d36626dd8db3fbf" gracePeriod=30 Dec 03 12:08:27 crc kubenswrapper[5052]: I1203 12:08:27.515853 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/openstack-cell1-galera-0"] Dec 03 12:08:27 crc kubenswrapper[5052]: I1203 12:08:27.540675 5052 generic.go:334] "Generic (PLEG): container finished" podID="78dad0d3-18e5-4900-932c-56ede3279dc0" containerID="b746c813f9a2abaab7eee298828559f29ccd2f1796875dbbe947a37faa740dfb" exitCode=2 Dec 03 12:08:27 crc kubenswrapper[5052]: I1203 12:08:27.540770 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/openstack-cell1-galera-2"] Dec 03 12:08:27 crc kubenswrapper[5052]: I1203 12:08:27.540802 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-sb-0" event={"ID":"78dad0d3-18e5-4900-932c-56ede3279dc0","Type":"ContainerDied","Data":"b746c813f9a2abaab7eee298828559f29ccd2f1796875dbbe947a37faa740dfb"} Dec 03 12:08:27 crc kubenswrapper[5052]: I1203 12:08:27.548969 5052 generic.go:334] "Generic (PLEG): container finished" podID="327a4d19-96ba-4b5e-a19f-56ba15a0cc82" containerID="023f6e55b1e44cead794ab88e34aa43141c551b1a2bce50e249cb4eeec4d2009" exitCode=2 Dec 03 12:08:27 crc kubenswrapper[5052]: I1203 12:08:27.549080 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-nb-0" event={"ID":"327a4d19-96ba-4b5e-a19f-56ba15a0cc82","Type":"ContainerDied","Data":"023f6e55b1e44cead794ab88e34aa43141c551b1a2bce50e249cb4eeec4d2009"} Dec 03 12:08:27 crc kubenswrapper[5052]: I1203 12:08:27.563092 5052 generic.go:334] "Generic (PLEG): container finished" podID="6999e86b-8f95-4002-9959-6288a7d783bd" containerID="18f9fa6fd7bb783b320dadc515c1e38c7a2e0212224d2e211b526e64320731a1" exitCode=143 Dec 03 12:08:27 crc kubenswrapper[5052]: I1203 12:08:27.563156 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-f4dfbd7d-mxsm8" event={"ID":"6999e86b-8f95-4002-9959-6288a7d783bd","Type":"ContainerDied","Data":"18f9fa6fd7bb783b320dadc515c1e38c7a2e0212224d2e211b526e64320731a1"} Dec 03 12:08:27 crc kubenswrapper[5052]: I1203 12:08:27.580442 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/openstack-cell1-galera-1"] Dec 03 12:08:27 crc kubenswrapper[5052]: W1203 12:08:27.611634 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod89ee1bab_2c68_4fc4_9171_a7330922faf6.slice/crio-c304cae678688e4fe4a4d9700dc6cb4c64dcdaf312fab8bfbf39b2457db150ce WatchSource:0}: Error finding container c304cae678688e4fe4a4d9700dc6cb4c64dcdaf312fab8bfbf39b2457db150ce: Status 404 returned error can't find the container with id c304cae678688e4fe4a4d9700dc6cb4c64dcdaf312fab8bfbf39b2457db150ce Dec 03 12:08:27 crc kubenswrapper[5052]: E1203 12:08:27.620586 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/rabbitmq-cell1-config-data: configmap "rabbitmq-cell1-config-data" not found Dec 03 12:08:27 crc kubenswrapper[5052]: E1203 12:08:27.620653 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/185fdd2a-cd57-45e5-9986-bebd15e5761f-config-data podName:185fdd2a-cd57-45e5-9986-bebd15e5761f nodeName:}" failed. No retries permitted until 2025-12-03 12:08:29.620635183 +0000 UTC m=+2900.208876947 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "config-data" (UniqueName: "kubernetes.io/configmap/185fdd2a-cd57-45e5-9986-bebd15e5761f-config-data") pod "rabbitmq-cell1-server-0" (UID: "185fdd2a-cd57-45e5-9986-bebd15e5761f") : configmap "rabbitmq-cell1-config-data" not found Dec 03 12:08:27 crc kubenswrapper[5052]: I1203 12:08:27.646619 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-kcb42"] Dec 03 12:08:27 crc kubenswrapper[5052]: I1203 12:08:27.648401 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-kcb42" Dec 03 12:08:27 crc kubenswrapper[5052]: I1203 12:08:27.650792 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/rabbitmq-server-2" podUID="7b4f6078-41a3-433d-acaf-f2d67391ab6e" containerName="rabbitmq" containerID="cri-o://aad64a682ded16a9f4098098d689931d1934e194e45eecdc993e62379504781b" gracePeriod=604800 Dec 03 12:08:27 crc kubenswrapper[5052]: I1203 12:08:27.673767 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/rabbitmq-server-0" podUID="2397d896-0d11-4656-a336-eac8ec9fe7f9" containerName="rabbitmq" containerID="cri-o://16dd1c84a3d482ce5adc97f8fdc99d27080aa0bd2d9fba51e986dc52caa0daf6" gracePeriod=604800 Dec 03 12:08:27 crc kubenswrapper[5052]: I1203 12:08:27.690536 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0c8be033-f83e-4d64-b271-f4c97bfdc108" path="/var/lib/kubelet/pods/0c8be033-f83e-4d64-b271-f4c97bfdc108/volumes" Dec 03 12:08:27 crc kubenswrapper[5052]: I1203 12:08:27.691338 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8f59423a-caa1-428c-920c-59bb1a3b618d" path="/var/lib/kubelet/pods/8f59423a-caa1-428c-920c-59bb1a3b618d/volumes" Dec 03 12:08:27 crc kubenswrapper[5052]: I1203 12:08:27.692677 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="907d9f47-ab0b-44c8-9aa6-3d135e1c312d" path="/var/lib/kubelet/pods/907d9f47-ab0b-44c8-9aa6-3d135e1c312d/volumes" Dec 03 12:08:27 crc kubenswrapper[5052]: I1203 12:08:27.693278 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ab039824-fe2a-44ba-ae0f-302e5c4ea614" path="/var/lib/kubelet/pods/ab039824-fe2a-44ba-ae0f-302e5c4ea614/volumes" Dec 03 12:08:27 crc kubenswrapper[5052]: I1203 12:08:27.693797 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d9b7a317-ab7f-4c6a-8f91-b66b409c8980" path="/var/lib/kubelet/pods/d9b7a317-ab7f-4c6a-8f91-b66b409c8980/volumes" Dec 03 12:08:27 crc kubenswrapper[5052]: I1203 12:08:27.694852 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="df00a025-ee23-44aa-8970-6f1232146b98" path="/var/lib/kubelet/pods/df00a025-ee23-44aa-8970-6f1232146b98/volumes" Dec 03 12:08:27 crc kubenswrapper[5052]: I1203 12:08:27.695502 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f2d639ef-d667-416d-8899-1a080bca471c" path="/var/lib/kubelet/pods/f2d639ef-d667-416d-8899-1a080bca471c/volumes" Dec 03 12:08:27 crc kubenswrapper[5052]: I1203 12:08:27.696047 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f3c81a5a-a6cc-406f-95b9-186ae52a3ccb" path="/var/lib/kubelet/pods/f3c81a5a-a6cc-406f-95b9-186ae52a3ccb/volumes" Dec 03 12:08:27 crc kubenswrapper[5052]: I1203 12:08:27.697037 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:08:27 crc kubenswrapper[5052]: I1203 12:08:27.697265 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-metadata-0" podUID="b9efd251-95c2-4224-b318-a34279ee1017" containerName="nova-metadata-log" containerID="cri-o://179d0d417d11399d428ed2dce00676182f65bdc218984235dfbdc87fb4e0c5c0" gracePeriod=30 Dec 03 12:08:27 crc kubenswrapper[5052]: I1203 12:08:27.698176 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-metadata-0" podUID="b9efd251-95c2-4224-b318-a34279ee1017" containerName="nova-metadata-metadata" containerID="cri-o://0d0239c7224731fd55404636504e2c22a3b640502408dba4f002511377ec7f33" gracePeriod=30 Dec 03 12:08:27 crc kubenswrapper[5052]: I1203 12:08:27.709219 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-kcb42"] Dec 03 12:08:27 crc kubenswrapper[5052]: I1203 12:08:27.717376 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/nova-metadata-0" podUID="b9efd251-95c2-4224-b318-a34279ee1017" containerName="nova-metadata-metadata" probeResult="failure" output="Get \"https://10.217.0.197:8775/\": read tcp 10.217.0.2:42058->10.217.0.197:8775: read: connection reset by peer" Dec 03 12:08:27 crc kubenswrapper[5052]: E1203 12:08:27.722447 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/rabbitmq-cell1-config-data: configmap "rabbitmq-cell1-config-data" not found Dec 03 12:08:27 crc kubenswrapper[5052]: E1203 12:08:27.722545 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/06bfd2db-abe0-4b4d-a110-94a6a1e02211-config-data podName:06bfd2db-abe0-4b4d-a110-94a6a1e02211 nodeName:}" failed. No retries permitted until 2025-12-03 12:08:29.722525347 +0000 UTC m=+2900.310767131 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "config-data" (UniqueName: "kubernetes.io/configmap/06bfd2db-abe0-4b4d-a110-94a6a1e02211-config-data") pod "rabbitmq-cell1-server-2" (UID: "06bfd2db-abe0-4b4d-a110-94a6a1e02211") : configmap "rabbitmq-cell1-config-data" not found Dec 03 12:08:27 crc kubenswrapper[5052]: I1203 12:08:27.735123 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:08:27 crc kubenswrapper[5052]: I1203 12:08:27.735411 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-api-0" podUID="45e26e8c-71b5-4b90-bba7-5bad2650d9b2" containerName="nova-api-log" containerID="cri-o://37bdeb3b3f3f5f77fea4db0897de61d07f8d244f4f0ab18853e3fa400f45ac54" gracePeriod=30 Dec 03 12:08:27 crc kubenswrapper[5052]: I1203 12:08:27.735806 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-api-0" podUID="45e26e8c-71b5-4b90-bba7-5bad2650d9b2" containerName="nova-api-api" containerID="cri-o://a96c58179a61355fd41c8699c34a2a6efcdd96a6901a2acedb4bb445fe0b84fe" gracePeriod=30 Dec 03 12:08:27 crc kubenswrapper[5052]: I1203 12:08:27.818656 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell1-db-create-64bnw"] Dec 03 12:08:27 crc kubenswrapper[5052]: I1203 12:08:27.824689 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f657b0c3-883e-4e3d-b161-1b3ef0988ab0-config\") pod \"dnsmasq-dnsmasq-84b9f45d47-kcb42\" (UID: \"f657b0c3-883e-4e3d-b161-1b3ef0988ab0\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-kcb42" Dec 03 12:08:27 crc kubenswrapper[5052]: I1203 12:08:27.824737 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-47gfp\" (UniqueName: \"kubernetes.io/projected/f657b0c3-883e-4e3d-b161-1b3ef0988ab0-kube-api-access-47gfp\") pod \"dnsmasq-dnsmasq-84b9f45d47-kcb42\" (UID: \"f657b0c3-883e-4e3d-b161-1b3ef0988ab0\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-kcb42" Dec 03 12:08:27 crc kubenswrapper[5052]: I1203 12:08:27.825029 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/f657b0c3-883e-4e3d-b161-1b3ef0988ab0-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-84b9f45d47-kcb42\" (UID: \"f657b0c3-883e-4e3d-b161-1b3ef0988ab0\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-kcb42" Dec 03 12:08:27 crc kubenswrapper[5052]: I1203 12:08:27.880551 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell1-db-create-64bnw"] Dec 03 12:08:27 crc kubenswrapper[5052]: I1203 12:08:27.899052 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/swift-proxy-9bb7db5d-5q7wc"] Dec 03 12:08:27 crc kubenswrapper[5052]: I1203 12:08:27.899298 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-proxy-9bb7db5d-5q7wc" podUID="028738cd-3b98-4853-9506-d2eba91afd87" containerName="proxy-httpd" containerID="cri-o://531801a1d9fdbb4eab4dc17c841d530240327b26c1a21395bf0c594278f80f5c" gracePeriod=30 Dec 03 12:08:27 crc kubenswrapper[5052]: I1203 12:08:27.899712 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-proxy-9bb7db5d-5q7wc" podUID="028738cd-3b98-4853-9506-d2eba91afd87" containerName="proxy-server" containerID="cri-o://2d0b57b97cc7d912f25d72df376cda014d9ee39ad7a02f4b38c46e47d7ba19ac" gracePeriod=30 Dec 03 12:08:27 crc kubenswrapper[5052]: I1203 12:08:27.918361 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-keystone-listener-64674fc4f4-wfh52"] Dec 03 12:08:27 crc kubenswrapper[5052]: I1203 12:08:27.918702 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/barbican-keystone-listener-64674fc4f4-wfh52" podUID="d877b426-724d-4ccf-a113-05a96ab94935" containerName="barbican-keystone-listener-log" containerID="cri-o://8ed05d2153d95fbbc5b927a2573653ac6d68bb4bb432a7dd29017b0640952a6f" gracePeriod=30 Dec 03 12:08:27 crc kubenswrapper[5052]: I1203 12:08:27.918944 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/barbican-keystone-listener-64674fc4f4-wfh52" podUID="d877b426-724d-4ccf-a113-05a96ab94935" containerName="barbican-keystone-listener" containerID="cri-o://51b740be59469e140e437bc25b7208518a7bb1d9c11f719a8bcca63ec56644a8" gracePeriod=30 Dec 03 12:08:27 crc kubenswrapper[5052]: I1203 12:08:27.922125 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-worker-c656b9c77-2gf7g"] Dec 03 12:08:27 crc kubenswrapper[5052]: I1203 12:08:27.922367 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/barbican-worker-c656b9c77-2gf7g" podUID="a3da2acb-8a0e-4ab3-8d15-2978eca73850" containerName="barbican-worker-log" containerID="cri-o://80fb2290d9e5fb396a3de096d08940f5a3a89a79f45d052e0d882756596d8b34" gracePeriod=30 Dec 03 12:08:27 crc kubenswrapper[5052]: I1203 12:08:27.922478 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/barbican-worker-c656b9c77-2gf7g" podUID="a3da2acb-8a0e-4ab3-8d15-2978eca73850" containerName="barbican-worker" containerID="cri-o://606f51e20d39e18fce0fe0fe7c840d6c46bd6e03a7e775bc7246b576bec5c132" gracePeriod=30 Dec 03 12:08:27 crc kubenswrapper[5052]: I1203 12:08:27.927397 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/f657b0c3-883e-4e3d-b161-1b3ef0988ab0-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-84b9f45d47-kcb42\" (UID: \"f657b0c3-883e-4e3d-b161-1b3ef0988ab0\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-kcb42" Dec 03 12:08:27 crc kubenswrapper[5052]: I1203 12:08:27.927492 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f657b0c3-883e-4e3d-b161-1b3ef0988ab0-config\") pod \"dnsmasq-dnsmasq-84b9f45d47-kcb42\" (UID: \"f657b0c3-883e-4e3d-b161-1b3ef0988ab0\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-kcb42" Dec 03 12:08:27 crc kubenswrapper[5052]: I1203 12:08:27.927607 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-47gfp\" (UniqueName: \"kubernetes.io/projected/f657b0c3-883e-4e3d-b161-1b3ef0988ab0-kube-api-access-47gfp\") pod \"dnsmasq-dnsmasq-84b9f45d47-kcb42\" (UID: \"f657b0c3-883e-4e3d-b161-1b3ef0988ab0\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-kcb42" Dec 03 12:08:27 crc kubenswrapper[5052]: I1203 12:08:27.928759 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/f657b0c3-883e-4e3d-b161-1b3ef0988ab0-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-84b9f45d47-kcb42\" (UID: \"f657b0c3-883e-4e3d-b161-1b3ef0988ab0\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-kcb42" Dec 03 12:08:27 crc kubenswrapper[5052]: I1203 12:08:27.929251 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f657b0c3-883e-4e3d-b161-1b3ef0988ab0-config\") pod \"dnsmasq-dnsmasq-84b9f45d47-kcb42\" (UID: \"f657b0c3-883e-4e3d-b161-1b3ef0988ab0\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-kcb42" Dec 03 12:08:27 crc kubenswrapper[5052]: I1203 12:08:27.933629 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell1-c2b9-account-create-update-hlm9w"] Dec 03 12:08:27 crc kubenswrapper[5052]: I1203 12:08:27.957945 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell1-c2b9-account-create-update-hlm9w"] Dec 03 12:08:27 crc kubenswrapper[5052]: I1203 12:08:27.991828 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 12:08:27 crc kubenswrapper[5052]: I1203 12:08:27.992047 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" podUID="e66cca73-f322-4c6e-adb0-5ec43607bb73" containerName="nova-cell1-novncproxy-novncproxy" containerID="cri-o://81211fcc809459c82abca1a15005c67c33acfa54010915f26c5a7c1bed4bf50d" gracePeriod=30 Dec 03 12:08:28 crc kubenswrapper[5052]: I1203 12:08:28.000221 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-api-d978c699b-kzzst"] Dec 03 12:08:28 crc kubenswrapper[5052]: I1203 12:08:28.000455 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/barbican-api-d978c699b-kzzst" podUID="de04776b-ea02-40d9-8459-0a519c8c64ad" containerName="barbican-api-log" containerID="cri-o://649a53bf9bcdd8f3f358ecd746d8c1146c6de92e3ce56e6a224366e0dffc9ed0" gracePeriod=30 Dec 03 12:08:28 crc kubenswrapper[5052]: I1203 12:08:28.000877 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/barbican-api-d978c699b-kzzst" podUID="de04776b-ea02-40d9-8459-0a519c8c64ad" containerName="barbican-api" containerID="cri-o://ebd10ebaaccf168282fe14fc0fe11aa08dcb1e7da4e422f6b5dfdc5b8ddb9b23" gracePeriod=30 Dec 03 12:08:28 crc kubenswrapper[5052]: I1203 12:08:28.007741 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-47gfp\" (UniqueName: \"kubernetes.io/projected/f657b0c3-883e-4e3d-b161-1b3ef0988ab0-kube-api-access-47gfp\") pod \"dnsmasq-dnsmasq-84b9f45d47-kcb42\" (UID: \"f657b0c3-883e-4e3d-b161-1b3ef0988ab0\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-kcb42" Dec 03 12:08:28 crc kubenswrapper[5052]: I1203 12:08:28.014963 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/placement3c40-account-delete-8qsdr"] Dec 03 12:08:28 crc kubenswrapper[5052]: E1203 12:08:28.042372 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/rabbitmq-cell1-config-data: configmap "rabbitmq-cell1-config-data" not found Dec 03 12:08:28 crc kubenswrapper[5052]: E1203 12:08:28.042465 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3-config-data podName:f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3 nodeName:}" failed. No retries permitted until 2025-12-03 12:08:30.042445103 +0000 UTC m=+2900.630686867 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "config-data" (UniqueName: "kubernetes.io/configmap/f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3-config-data") pod "rabbitmq-cell1-server-1" (UID: "f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3") : configmap "rabbitmq-cell1-config-data" not found Dec 03 12:08:28 crc kubenswrapper[5052]: I1203 12:08:28.046171 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cindere6fc-account-delete-z7w6l"] Dec 03 12:08:28 crc kubenswrapper[5052]: I1203 12:08:28.046716 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-kcb42" Dec 03 12:08:28 crc kubenswrapper[5052]: I1203 12:08:28.120566 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-db-sync-sxlxt"] Dec 03 12:08:28 crc kubenswrapper[5052]: I1203 12:08:28.164006 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-0"] Dec 03 12:08:28 crc kubenswrapper[5052]: I1203 12:08:28.164329 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-cell1-conductor-0" podUID="e92fa4d7-9bd6-4586-b41a-a557fb672439" containerName="nova-cell1-conductor-conductor" containerID="cri-o://8709692766b7010f69ca646129fa46d2b1115bc2a7f05706db51e69fa086eafa" gracePeriod=30 Dec 03 12:08:28 crc kubenswrapper[5052]: I1203 12:08:28.176580 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-db-sync-sxlxt"] Dec 03 12:08:28 crc kubenswrapper[5052]: I1203 12:08:28.185455 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-db-sync-smfjz"] Dec 03 12:08:28 crc kubenswrapper[5052]: I1203 12:08:28.189152 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/openstack-cell1-galera-2" podUID="6ac95bd3-125c-4966-b9a7-4eb4a7842c67" containerName="galera" containerID="cri-o://edf48c4037d4be3adfc21b5d97a50d68b5261315fe92282773dd8556852ce51a" gracePeriod=30 Dec 03 12:08:28 crc kubenswrapper[5052]: I1203 12:08:28.203726 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-db-sync-smfjz"] Dec 03 12:08:28 crc kubenswrapper[5052]: I1203 12:08:28.240225 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-0"] Dec 03 12:08:28 crc kubenswrapper[5052]: I1203 12:08:28.240485 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-cell0-conductor-0" podUID="4cae4fe5-bc0e-4088-b403-c78e940373c2" containerName="nova-cell0-conductor-conductor" containerID="cri-o://b19b072c0fb517fe3875dc65637b798d06ddeac368aceb24adb64b956c3e8597" gracePeriod=30 Dec 03 12:08:28 crc kubenswrapper[5052]: I1203 12:08:28.325711 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican5c55-account-delete-zttb9"] Dec 03 12:08:28 crc kubenswrapper[5052]: I1203 12:08:28.479981 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/novaapi48e2-account-delete-kqhz7"] Dec 03 12:08:28 crc kubenswrapper[5052]: W1203 12:08:28.501632 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod70122e67_6175_4a14_a120_6a2731c5dcb1.slice/crio-fb0cac54ec9d6c243455911815c99ff48403152f29b6f9186659e82c363412ff WatchSource:0}: Error finding container fb0cac54ec9d6c243455911815c99ff48403152f29b6f9186659e82c363412ff: Status 404 returned error can't find the container with id fb0cac54ec9d6c243455911815c99ff48403152f29b6f9186659e82c363412ff Dec 03 12:08:28 crc kubenswrapper[5052]: I1203 12:08:28.508611 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutrond5c4-account-delete-zj77r"] Dec 03 12:08:28 crc kubenswrapper[5052]: I1203 12:08:28.536322 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/novacell0d294-account-delete-b64wn"] Dec 03 12:08:28 crc kubenswrapper[5052]: I1203 12:08:28.588713 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutrond5c4-account-delete-zj77r" event={"ID":"9e6af047-b7ee-459a-b362-da5f777d0fa0","Type":"ContainerStarted","Data":"cfc37f2b6a6dfb5b579ba00eaffe633f2160ebe7c26d6474081a194e015a503c"} Dec 03 12:08:28 crc kubenswrapper[5052]: I1203 12:08:28.592875 5052 generic.go:334] "Generic (PLEG): container finished" podID="c7d0ffdb-bb0c-4212-9682-289d630186fd" containerID="dbd211edde83ca19480c688ca164c8a5521ad9be2a864cfa3177c9b6dafa1a07" exitCode=137 Dec 03 12:08:28 crc kubenswrapper[5052]: I1203 12:08:28.603870 5052 generic.go:334] "Generic (PLEG): container finished" podID="de04776b-ea02-40d9-8459-0a519c8c64ad" containerID="649a53bf9bcdd8f3f358ecd746d8c1146c6de92e3ce56e6a224366e0dffc9ed0" exitCode=143 Dec 03 12:08:28 crc kubenswrapper[5052]: I1203 12:08:28.604278 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-d978c699b-kzzst" event={"ID":"de04776b-ea02-40d9-8459-0a519c8c64ad","Type":"ContainerDied","Data":"649a53bf9bcdd8f3f358ecd746d8c1146c6de92e3ce56e6a224366e0dffc9ed0"} Dec 03 12:08:28 crc kubenswrapper[5052]: I1203 12:08:28.607872 5052 generic.go:334] "Generic (PLEG): container finished" podID="bef08f40-1b01-406f-af05-7b63e0b6e5e7" containerID="62853e68ee96c13bce5c6927e9083bfe1dca5e094c03354b4a342f37a5ac3ce4" exitCode=143 Dec 03 12:08:28 crc kubenswrapper[5052]: I1203 12:08:28.607940 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"bef08f40-1b01-406f-af05-7b63e0b6e5e7","Type":"ContainerDied","Data":"62853e68ee96c13bce5c6927e9083bfe1dca5e094c03354b4a342f37a5ac3ce4"} Dec 03 12:08:28 crc kubenswrapper[5052]: I1203 12:08:28.609487 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cindere6fc-account-delete-z7w6l" event={"ID":"89ee1bab-2c68-4fc4-9171-a7330922faf6","Type":"ContainerStarted","Data":"c304cae678688e4fe4a4d9700dc6cb4c64dcdaf312fab8bfbf39b2457db150ce"} Dec 03 12:08:28 crc kubenswrapper[5052]: I1203 12:08:28.610688 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican5c55-account-delete-zttb9" event={"ID":"1bc7c8b1-cc0d-4611-b335-01c2b91713e2","Type":"ContainerStarted","Data":"35532bae8900df426551c12f66e807c943adb4ee0815e79acd82f5019c0c99bb"} Dec 03 12:08:28 crc kubenswrapper[5052]: I1203 12:08:28.616577 5052 generic.go:334] "Generic (PLEG): container finished" podID="31e6a111-bd4a-45a5-95a0-a6b537added6" containerID="84283f0da62e2076f919bc27137ad503eb09cc0b4a9e5ff7119c2f488722a65c" exitCode=143 Dec 03 12:08:28 crc kubenswrapper[5052]: I1203 12:08:28.616640 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"31e6a111-bd4a-45a5-95a0-a6b537added6","Type":"ContainerDied","Data":"84283f0da62e2076f919bc27137ad503eb09cc0b4a9e5ff7119c2f488722a65c"} Dec 03 12:08:28 crc kubenswrapper[5052]: I1203 12:08:28.635440 5052 generic.go:334] "Generic (PLEG): container finished" podID="95bf5f72-82f3-4e88-a576-01dae857541f" containerID="2d047aaa650efd207aa517746b6cde98bb8bae986bd03e7b545eda7fd36f1023" exitCode=0 Dec 03 12:08:28 crc kubenswrapper[5052]: I1203 12:08:28.635473 5052 generic.go:334] "Generic (PLEG): container finished" podID="95bf5f72-82f3-4e88-a576-01dae857541f" containerID="c6d65f1e5859c5423b2bed9ad8130035fdbcdb06ce5b34713168ebc78916b853" exitCode=0 Dec 03 12:08:28 crc kubenswrapper[5052]: I1203 12:08:28.635482 5052 generic.go:334] "Generic (PLEG): container finished" podID="95bf5f72-82f3-4e88-a576-01dae857541f" containerID="b1f5592550eedafe273eda6ddd62bcf2957cdd483d534a5369391fc101829850" exitCode=0 Dec 03 12:08:28 crc kubenswrapper[5052]: I1203 12:08:28.635489 5052 generic.go:334] "Generic (PLEG): container finished" podID="95bf5f72-82f3-4e88-a576-01dae857541f" containerID="ff53c665fe25113ca5552ca3edb72dc98d7ffacf8d96d01a610c8292d0ca0504" exitCode=0 Dec 03 12:08:28 crc kubenswrapper[5052]: I1203 12:08:28.635496 5052 generic.go:334] "Generic (PLEG): container finished" podID="95bf5f72-82f3-4e88-a576-01dae857541f" containerID="2577c8659d7bc84c157b82cc64e9d927df3b8f1b4fe022b955ef7fe05c5dadbb" exitCode=0 Dec 03 12:08:28 crc kubenswrapper[5052]: I1203 12:08:28.635502 5052 generic.go:334] "Generic (PLEG): container finished" podID="95bf5f72-82f3-4e88-a576-01dae857541f" containerID="ae9aaced3b606148dcf3e097c33141f00f123ef8589615b163bff1470454ae80" exitCode=0 Dec 03 12:08:28 crc kubenswrapper[5052]: I1203 12:08:28.635523 5052 generic.go:334] "Generic (PLEG): container finished" podID="95bf5f72-82f3-4e88-a576-01dae857541f" containerID="2732548b8572203d895c3e589eef6680d882e9fcbe498714de0778d4c35813ec" exitCode=0 Dec 03 12:08:28 crc kubenswrapper[5052]: I1203 12:08:28.635531 5052 generic.go:334] "Generic (PLEG): container finished" podID="95bf5f72-82f3-4e88-a576-01dae857541f" containerID="6a7d071e07e4e8b8e192642aeae3ccfa4e50a1bfae563ab609f77db309009d6d" exitCode=0 Dec 03 12:08:28 crc kubenswrapper[5052]: I1203 12:08:28.635537 5052 generic.go:334] "Generic (PLEG): container finished" podID="95bf5f72-82f3-4e88-a576-01dae857541f" containerID="3604bfccbdf55c7f8cc4aa8921d57387c338bcccd177ab45652bc037ec8fdad6" exitCode=0 Dec 03 12:08:28 crc kubenswrapper[5052]: I1203 12:08:28.635602 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"95bf5f72-82f3-4e88-a576-01dae857541f","Type":"ContainerDied","Data":"2d047aaa650efd207aa517746b6cde98bb8bae986bd03e7b545eda7fd36f1023"} Dec 03 12:08:28 crc kubenswrapper[5052]: I1203 12:08:28.635631 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"95bf5f72-82f3-4e88-a576-01dae857541f","Type":"ContainerDied","Data":"c6d65f1e5859c5423b2bed9ad8130035fdbcdb06ce5b34713168ebc78916b853"} Dec 03 12:08:28 crc kubenswrapper[5052]: I1203 12:08:28.635642 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"95bf5f72-82f3-4e88-a576-01dae857541f","Type":"ContainerDied","Data":"b1f5592550eedafe273eda6ddd62bcf2957cdd483d534a5369391fc101829850"} Dec 03 12:08:28 crc kubenswrapper[5052]: I1203 12:08:28.635650 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"95bf5f72-82f3-4e88-a576-01dae857541f","Type":"ContainerDied","Data":"ff53c665fe25113ca5552ca3edb72dc98d7ffacf8d96d01a610c8292d0ca0504"} Dec 03 12:08:28 crc kubenswrapper[5052]: I1203 12:08:28.635659 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"95bf5f72-82f3-4e88-a576-01dae857541f","Type":"ContainerDied","Data":"2577c8659d7bc84c157b82cc64e9d927df3b8f1b4fe022b955ef7fe05c5dadbb"} Dec 03 12:08:28 crc kubenswrapper[5052]: I1203 12:08:28.635667 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"95bf5f72-82f3-4e88-a576-01dae857541f","Type":"ContainerDied","Data":"ae9aaced3b606148dcf3e097c33141f00f123ef8589615b163bff1470454ae80"} Dec 03 12:08:28 crc kubenswrapper[5052]: I1203 12:08:28.635676 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"95bf5f72-82f3-4e88-a576-01dae857541f","Type":"ContainerDied","Data":"2732548b8572203d895c3e589eef6680d882e9fcbe498714de0778d4c35813ec"} Dec 03 12:08:28 crc kubenswrapper[5052]: I1203 12:08:28.635684 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"95bf5f72-82f3-4e88-a576-01dae857541f","Type":"ContainerDied","Data":"6a7d071e07e4e8b8e192642aeae3ccfa4e50a1bfae563ab609f77db309009d6d"} Dec 03 12:08:28 crc kubenswrapper[5052]: I1203 12:08:28.635694 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"95bf5f72-82f3-4e88-a576-01dae857541f","Type":"ContainerDied","Data":"3604bfccbdf55c7f8cc4aa8921d57387c338bcccd177ab45652bc037ec8fdad6"} Dec 03 12:08:28 crc kubenswrapper[5052]: I1203 12:08:28.659168 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_ovsdbserver-nb-0_327a4d19-96ba-4b5e-a19f-56ba15a0cc82/ovsdbserver-nb/0.log" Dec 03 12:08:28 crc kubenswrapper[5052]: I1203 12:08:28.659215 5052 generic.go:334] "Generic (PLEG): container finished" podID="327a4d19-96ba-4b5e-a19f-56ba15a0cc82" containerID="04c71fa1d03c17ad50a53dfb4585c12716ea5f19b448fb01a9fbd6d86935e208" exitCode=143 Dec 03 12:08:28 crc kubenswrapper[5052]: I1203 12:08:28.659286 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-nb-0" event={"ID":"327a4d19-96ba-4b5e-a19f-56ba15a0cc82","Type":"ContainerDied","Data":"04c71fa1d03c17ad50a53dfb4585c12716ea5f19b448fb01a9fbd6d86935e208"} Dec 03 12:08:28 crc kubenswrapper[5052]: I1203 12:08:28.666000 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/novaapi48e2-account-delete-kqhz7" event={"ID":"70122e67-6175-4a14-a120-6a2731c5dcb1","Type":"ContainerStarted","Data":"fb0cac54ec9d6c243455911815c99ff48403152f29b6f9186659e82c363412ff"} Dec 03 12:08:28 crc kubenswrapper[5052]: I1203 12:08:28.667873 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement3c40-account-delete-8qsdr" event={"ID":"aaa3db64-a10b-45ea-9e25-de4916f30617","Type":"ContainerStarted","Data":"0a99a26443e4ad86179f2230393e6f19e8d1ac332ad1a9478891666bf36d948e"} Dec 03 12:08:28 crc kubenswrapper[5052]: I1203 12:08:28.670276 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_ovsdbserver-sb-0_78dad0d3-18e5-4900-932c-56ede3279dc0/ovsdbserver-sb/0.log" Dec 03 12:08:28 crc kubenswrapper[5052]: I1203 12:08:28.670306 5052 generic.go:334] "Generic (PLEG): container finished" podID="78dad0d3-18e5-4900-932c-56ede3279dc0" containerID="3fa9a29a156d24f947438d4dd3ac36264903dbd4e777dddc9352f1aab01266e5" exitCode=143 Dec 03 12:08:28 crc kubenswrapper[5052]: I1203 12:08:28.670348 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-sb-0" event={"ID":"78dad0d3-18e5-4900-932c-56ede3279dc0","Type":"ContainerDied","Data":"3fa9a29a156d24f947438d4dd3ac36264903dbd4e777dddc9352f1aab01266e5"} Dec 03 12:08:28 crc kubenswrapper[5052]: I1203 12:08:28.672211 5052 generic.go:334] "Generic (PLEG): container finished" podID="b9efd251-95c2-4224-b318-a34279ee1017" containerID="179d0d417d11399d428ed2dce00676182f65bdc218984235dfbdc87fb4e0c5c0" exitCode=143 Dec 03 12:08:28 crc kubenswrapper[5052]: I1203 12:08:28.672272 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"b9efd251-95c2-4224-b318-a34279ee1017","Type":"ContainerDied","Data":"179d0d417d11399d428ed2dce00676182f65bdc218984235dfbdc87fb4e0c5c0"} Dec 03 12:08:28 crc kubenswrapper[5052]: I1203 12:08:28.676762 5052 generic.go:334] "Generic (PLEG): container finished" podID="4a2a1881-e61a-4a99-a2f3-b614aeed81d1" containerID="e3e2ee8f299225cb3b6df3bf35cd7781dcb1aa91cfcdbcdc6d9ceee107ca48ce" exitCode=143 Dec 03 12:08:28 crc kubenswrapper[5052]: I1203 12:08:28.676842 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"4a2a1881-e61a-4a99-a2f3-b614aeed81d1","Type":"ContainerDied","Data":"e3e2ee8f299225cb3b6df3bf35cd7781dcb1aa91cfcdbcdc6d9ceee107ca48ce"} Dec 03 12:08:28 crc kubenswrapper[5052]: I1203 12:08:28.679097 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/novacell0d294-account-delete-b64wn" event={"ID":"030d4f76-eb30-4c44-b782-fa7147febf41","Type":"ContainerStarted","Data":"d84a227d521e482f2af3610d890a33ba5ee3263757d3a6f6e31b17331a144841"} Dec 03 12:08:28 crc kubenswrapper[5052]: I1203 12:08:28.682462 5052 generic.go:334] "Generic (PLEG): container finished" podID="45e26e8c-71b5-4b90-bba7-5bad2650d9b2" containerID="37bdeb3b3f3f5f77fea4db0897de61d07f8d244f4f0ab18853e3fa400f45ac54" exitCode=143 Dec 03 12:08:28 crc kubenswrapper[5052]: I1203 12:08:28.682506 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"45e26e8c-71b5-4b90-bba7-5bad2650d9b2","Type":"ContainerDied","Data":"37bdeb3b3f3f5f77fea4db0897de61d07f8d244f4f0ab18853e3fa400f45ac54"} Dec 03 12:08:28 crc kubenswrapper[5052]: I1203 12:08:28.842427 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-kcb42"] Dec 03 12:08:28 crc kubenswrapper[5052]: W1203 12:08:28.855744 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf657b0c3_883e_4e3d_b161_1b3ef0988ab0.slice/crio-4c8cb428cd412adcc11141dc9c500eb7f270887e14784e9522cf1ff216e1d58e WatchSource:0}: Error finding container 4c8cb428cd412adcc11141dc9c500eb7f270887e14784e9522cf1ff216e1d58e: Status 404 returned error can't find the container with id 4c8cb428cd412adcc11141dc9c500eb7f270887e14784e9522cf1ff216e1d58e Dec 03 12:08:28 crc kubenswrapper[5052]: E1203 12:08:28.963041 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="8709692766b7010f69ca646129fa46d2b1115bc2a7f05706db51e69fa086eafa" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:08:28 crc kubenswrapper[5052]: E1203 12:08:28.969270 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="8709692766b7010f69ca646129fa46d2b1115bc2a7f05706db51e69fa086eafa" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:08:28 crc kubenswrapper[5052]: E1203 12:08:28.970425 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="8709692766b7010f69ca646129fa46d2b1115bc2a7f05706db51e69fa086eafa" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:08:28 crc kubenswrapper[5052]: E1203 12:08:28.970462 5052 prober.go:104] "Probe errored" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" probeType="Readiness" pod="openstack-kuttl-tests/nova-cell1-conductor-0" podUID="e92fa4d7-9bd6-4586-b41a-a557fb672439" containerName="nova-cell1-conductor-conductor" Dec 03 12:08:29 crc kubenswrapper[5052]: E1203 12:08:29.016675 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of 3fa9a29a156d24f947438d4dd3ac36264903dbd4e777dddc9352f1aab01266e5 is running failed: container process not found" containerID="3fa9a29a156d24f947438d4dd3ac36264903dbd4e777dddc9352f1aab01266e5" cmd=["/usr/bin/pidof","ovsdb-server"] Dec 03 12:08:29 crc kubenswrapper[5052]: E1203 12:08:29.027447 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of 3fa9a29a156d24f947438d4dd3ac36264903dbd4e777dddc9352f1aab01266e5 is running failed: container process not found" containerID="3fa9a29a156d24f947438d4dd3ac36264903dbd4e777dddc9352f1aab01266e5" cmd=["/usr/bin/pidof","ovsdb-server"] Dec 03 12:08:29 crc kubenswrapper[5052]: E1203 12:08:29.028265 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of 3fa9a29a156d24f947438d4dd3ac36264903dbd4e777dddc9352f1aab01266e5 is running failed: container process not found" containerID="3fa9a29a156d24f947438d4dd3ac36264903dbd4e777dddc9352f1aab01266e5" cmd=["/usr/bin/pidof","ovsdb-server"] Dec 03 12:08:29 crc kubenswrapper[5052]: E1203 12:08:29.028504 5052 prober.go:104] "Probe errored" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of 3fa9a29a156d24f947438d4dd3ac36264903dbd4e777dddc9352f1aab01266e5 is running failed: container process not found" probeType="Readiness" pod="openstack-kuttl-tests/ovsdbserver-sb-0" podUID="78dad0d3-18e5-4900-932c-56ede3279dc0" containerName="ovsdbserver-sb" Dec 03 12:08:29 crc kubenswrapper[5052]: E1203 12:08:29.121243 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="b19b072c0fb517fe3875dc65637b798d06ddeac368aceb24adb64b956c3e8597" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:08:29 crc kubenswrapper[5052]: E1203 12:08:29.127283 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="b19b072c0fb517fe3875dc65637b798d06ddeac368aceb24adb64b956c3e8597" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:08:29 crc kubenswrapper[5052]: I1203 12:08:29.134468 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_ovsdbserver-nb-0_327a4d19-96ba-4b5e-a19f-56ba15a0cc82/ovsdbserver-nb/0.log" Dec 03 12:08:29 crc kubenswrapper[5052]: I1203 12:08:29.134567 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:08:29 crc kubenswrapper[5052]: E1203 12:08:29.157440 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="b19b072c0fb517fe3875dc65637b798d06ddeac368aceb24adb64b956c3e8597" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:08:29 crc kubenswrapper[5052]: E1203 12:08:29.157540 5052 prober.go:104] "Probe errored" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" probeType="Readiness" pod="openstack-kuttl-tests/nova-cell0-conductor-0" podUID="4cae4fe5-bc0e-4088-b403-c78e940373c2" containerName="nova-cell0-conductor-conductor" Dec 03 12:08:29 crc kubenswrapper[5052]: I1203 12:08:29.202094 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/rabbitmq-cell1-server-1"] Dec 03 12:08:29 crc kubenswrapper[5052]: I1203 12:08:29.238862 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/rabbitmq-cell1-server-2"] Dec 03 12:08:29 crc kubenswrapper[5052]: I1203 12:08:29.246661 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/rabbitmq-cell1-server-0"] Dec 03 12:08:29 crc kubenswrapper[5052]: I1203 12:08:29.313257 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/rabbitmq-cell1-server-1" podUID="f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3" containerName="rabbitmq" containerID="cri-o://ae74ed54ba186a0e9a780dc48252d0e6409d74b51697be203e1eefe16b5793fe" gracePeriod=604800 Dec 03 12:08:29 crc kubenswrapper[5052]: I1203 12:08:29.314763 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j7hd4\" (UniqueName: \"kubernetes.io/projected/327a4d19-96ba-4b5e-a19f-56ba15a0cc82-kube-api-access-j7hd4\") pod \"327a4d19-96ba-4b5e-a19f-56ba15a0cc82\" (UID: \"327a4d19-96ba-4b5e-a19f-56ba15a0cc82\") " Dec 03 12:08:29 crc kubenswrapper[5052]: I1203 12:08:29.314803 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/327a4d19-96ba-4b5e-a19f-56ba15a0cc82-config\") pod \"327a4d19-96ba-4b5e-a19f-56ba15a0cc82\" (UID: \"327a4d19-96ba-4b5e-a19f-56ba15a0cc82\") " Dec 03 12:08:29 crc kubenswrapper[5052]: I1203 12:08:29.314882 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovndbcluster-nb-etc-ovn\" (UniqueName: \"kubernetes.io/local-volume/local-storage19-crc\") pod \"327a4d19-96ba-4b5e-a19f-56ba15a0cc82\" (UID: \"327a4d19-96ba-4b5e-a19f-56ba15a0cc82\") " Dec 03 12:08:29 crc kubenswrapper[5052]: I1203 12:08:29.314907 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/327a4d19-96ba-4b5e-a19f-56ba15a0cc82-combined-ca-bundle\") pod \"327a4d19-96ba-4b5e-a19f-56ba15a0cc82\" (UID: \"327a4d19-96ba-4b5e-a19f-56ba15a0cc82\") " Dec 03 12:08:29 crc kubenswrapper[5052]: I1203 12:08:29.314956 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovsdbserver-nb-tls-certs\" (UniqueName: \"kubernetes.io/secret/327a4d19-96ba-4b5e-a19f-56ba15a0cc82-ovsdbserver-nb-tls-certs\") pod \"327a4d19-96ba-4b5e-a19f-56ba15a0cc82\" (UID: \"327a4d19-96ba-4b5e-a19f-56ba15a0cc82\") " Dec 03 12:08:29 crc kubenswrapper[5052]: I1203 12:08:29.315002 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/327a4d19-96ba-4b5e-a19f-56ba15a0cc82-scripts\") pod \"327a4d19-96ba-4b5e-a19f-56ba15a0cc82\" (UID: \"327a4d19-96ba-4b5e-a19f-56ba15a0cc82\") " Dec 03 12:08:29 crc kubenswrapper[5052]: I1203 12:08:29.315047 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/327a4d19-96ba-4b5e-a19f-56ba15a0cc82-ovsdb-rundir\") pod \"327a4d19-96ba-4b5e-a19f-56ba15a0cc82\" (UID: \"327a4d19-96ba-4b5e-a19f-56ba15a0cc82\") " Dec 03 12:08:29 crc kubenswrapper[5052]: I1203 12:08:29.315065 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/327a4d19-96ba-4b5e-a19f-56ba15a0cc82-metrics-certs-tls-certs\") pod \"327a4d19-96ba-4b5e-a19f-56ba15a0cc82\" (UID: \"327a4d19-96ba-4b5e-a19f-56ba15a0cc82\") " Dec 03 12:08:29 crc kubenswrapper[5052]: E1203 12:08:29.315441 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/rabbitmq-config-data: configmap "rabbitmq-config-data" not found Dec 03 12:08:29 crc kubenswrapper[5052]: E1203 12:08:29.315490 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2397d896-0d11-4656-a336-eac8ec9fe7f9-config-data podName:2397d896-0d11-4656-a336-eac8ec9fe7f9 nodeName:}" failed. No retries permitted until 2025-12-03 12:08:33.315477647 +0000 UTC m=+2903.903719411 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "config-data" (UniqueName: "kubernetes.io/configmap/2397d896-0d11-4656-a336-eac8ec9fe7f9-config-data") pod "rabbitmq-server-0" (UID: "2397d896-0d11-4656-a336-eac8ec9fe7f9") : configmap "rabbitmq-config-data" not found Dec 03 12:08:29 crc kubenswrapper[5052]: I1203 12:08:29.323891 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/327a4d19-96ba-4b5e-a19f-56ba15a0cc82-config" (OuterVolumeSpecName: "config") pod "327a4d19-96ba-4b5e-a19f-56ba15a0cc82" (UID: "327a4d19-96ba-4b5e-a19f-56ba15a0cc82"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:08:29 crc kubenswrapper[5052]: I1203 12:08:29.324000 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/327a4d19-96ba-4b5e-a19f-56ba15a0cc82-ovsdb-rundir" (OuterVolumeSpecName: "ovsdb-rundir") pod "327a4d19-96ba-4b5e-a19f-56ba15a0cc82" (UID: "327a4d19-96ba-4b5e-a19f-56ba15a0cc82"). InnerVolumeSpecName "ovsdb-rundir". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:08:29 crc kubenswrapper[5052]: E1203 12:08:29.324149 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/rabbitmq-config-data: configmap "rabbitmq-config-data" not found Dec 03 12:08:29 crc kubenswrapper[5052]: E1203 12:08:29.324222 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7901ce85-4e6f-4e8b-86a6-f50a85803637-config-data podName:7901ce85-4e6f-4e8b-86a6-f50a85803637 nodeName:}" failed. No retries permitted until 2025-12-03 12:08:33.324184996 +0000 UTC m=+2903.912426760 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "config-data" (UniqueName: "kubernetes.io/configmap/7901ce85-4e6f-4e8b-86a6-f50a85803637-config-data") pod "rabbitmq-server-1" (UID: "7901ce85-4e6f-4e8b-86a6-f50a85803637") : configmap "rabbitmq-config-data" not found Dec 03 12:08:29 crc kubenswrapper[5052]: I1203 12:08:29.338825 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/327a4d19-96ba-4b5e-a19f-56ba15a0cc82-scripts" (OuterVolumeSpecName: "scripts") pod "327a4d19-96ba-4b5e-a19f-56ba15a0cc82" (UID: "327a4d19-96ba-4b5e-a19f-56ba15a0cc82"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:08:29 crc kubenswrapper[5052]: I1203 12:08:29.339667 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage19-crc" (OuterVolumeSpecName: "ovndbcluster-nb-etc-ovn") pod "327a4d19-96ba-4b5e-a19f-56ba15a0cc82" (UID: "327a4d19-96ba-4b5e-a19f-56ba15a0cc82"). InnerVolumeSpecName "local-storage19-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 12:08:29 crc kubenswrapper[5052]: I1203 12:08:29.351502 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/327a4d19-96ba-4b5e-a19f-56ba15a0cc82-kube-api-access-j7hd4" (OuterVolumeSpecName: "kube-api-access-j7hd4") pod "327a4d19-96ba-4b5e-a19f-56ba15a0cc82" (UID: "327a4d19-96ba-4b5e-a19f-56ba15a0cc82"). InnerVolumeSpecName "kube-api-access-j7hd4". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:08:29 crc kubenswrapper[5052]: I1203 12:08:29.363057 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/rabbitmq-cell1-server-2" podUID="06bfd2db-abe0-4b4d-a110-94a6a1e02211" containerName="rabbitmq" containerID="cri-o://61952ac0175a2f78cbd3919ba15907c05a42be11756520e4a28f961c6f3c039e" gracePeriod=604800 Dec 03 12:08:29 crc kubenswrapper[5052]: I1203 12:08:29.379117 5052 patch_prober.go:28] interesting pod/machine-config-daemon-74bj6 container/machine-config-daemon namespace/openshift-machine-config-operator: Liveness probe status=failure output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" start-of-body= Dec 03 12:08:29 crc kubenswrapper[5052]: I1203 12:08:29.379353 5052 prober.go:107] "Probe failed" probeType="Liveness" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" containerName="machine-config-daemon" probeResult="failure" output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" Dec 03 12:08:29 crc kubenswrapper[5052]: I1203 12:08:29.380322 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/327a4d19-96ba-4b5e-a19f-56ba15a0cc82-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "327a4d19-96ba-4b5e-a19f-56ba15a0cc82" (UID: "327a4d19-96ba-4b5e-a19f-56ba15a0cc82"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:08:29 crc kubenswrapper[5052]: I1203 12:08:29.426941 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/327a4d19-96ba-4b5e-a19f-56ba15a0cc82-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:29 crc kubenswrapper[5052]: I1203 12:08:29.426974 5052 reconciler_common.go:293] "Volume detached for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/327a4d19-96ba-4b5e-a19f-56ba15a0cc82-ovsdb-rundir\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:29 crc kubenswrapper[5052]: I1203 12:08:29.426984 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-j7hd4\" (UniqueName: \"kubernetes.io/projected/327a4d19-96ba-4b5e-a19f-56ba15a0cc82-kube-api-access-j7hd4\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:29 crc kubenswrapper[5052]: I1203 12:08:29.426993 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/327a4d19-96ba-4b5e-a19f-56ba15a0cc82-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:29 crc kubenswrapper[5052]: I1203 12:08:29.427012 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage19-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage19-crc\") on node \"crc\" " Dec 03 12:08:29 crc kubenswrapper[5052]: I1203 12:08:29.427022 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/327a4d19-96ba-4b5e-a19f-56ba15a0cc82-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:29 crc kubenswrapper[5052]: E1203 12:08:29.442823 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/rabbitmq-config-data: configmap "rabbitmq-config-data" not found Dec 03 12:08:29 crc kubenswrapper[5052]: E1203 12:08:29.443129 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7b4f6078-41a3-433d-acaf-f2d67391ab6e-config-data podName:7b4f6078-41a3-433d-acaf-f2d67391ab6e nodeName:}" failed. No retries permitted until 2025-12-03 12:08:33.443111696 +0000 UTC m=+2904.031353460 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "config-data" (UniqueName: "kubernetes.io/configmap/7b4f6078-41a3-433d-acaf-f2d67391ab6e-config-data") pod "rabbitmq-server-2" (UID: "7b4f6078-41a3-433d-acaf-f2d67391ab6e") : configmap "rabbitmq-config-data" not found Dec 03 12:08:29 crc kubenswrapper[5052]: I1203 12:08:29.456361 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage19-crc" (UniqueName: "kubernetes.io/local-volume/local-storage19-crc") on node "crc" Dec 03 12:08:29 crc kubenswrapper[5052]: I1203 12:08:29.481288 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/327a4d19-96ba-4b5e-a19f-56ba15a0cc82-metrics-certs-tls-certs" (OuterVolumeSpecName: "metrics-certs-tls-certs") pod "327a4d19-96ba-4b5e-a19f-56ba15a0cc82" (UID: "327a4d19-96ba-4b5e-a19f-56ba15a0cc82"). InnerVolumeSpecName "metrics-certs-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:08:29 crc kubenswrapper[5052]: I1203 12:08:29.532139 5052 reconciler_common.go:293] "Volume detached for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/327a4d19-96ba-4b5e-a19f-56ba15a0cc82-metrics-certs-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:29 crc kubenswrapper[5052]: I1203 12:08:29.532167 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage19-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage19-crc\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:29 crc kubenswrapper[5052]: I1203 12:08:29.582617 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/327a4d19-96ba-4b5e-a19f-56ba15a0cc82-ovsdbserver-nb-tls-certs" (OuterVolumeSpecName: "ovsdbserver-nb-tls-certs") pod "327a4d19-96ba-4b5e-a19f-56ba15a0cc82" (UID: "327a4d19-96ba-4b5e-a19f-56ba15a0cc82"). InnerVolumeSpecName "ovsdbserver-nb-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:08:29 crc kubenswrapper[5052]: I1203 12:08:29.635969 5052 reconciler_common.go:293] "Volume detached for volume \"ovsdbserver-nb-tls-certs\" (UniqueName: \"kubernetes.io/secret/327a4d19-96ba-4b5e-a19f-56ba15a0cc82-ovsdbserver-nb-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:29 crc kubenswrapper[5052]: E1203 12:08:29.636053 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/rabbitmq-cell1-config-data: configmap "rabbitmq-cell1-config-data" not found Dec 03 12:08:29 crc kubenswrapper[5052]: E1203 12:08:29.636102 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/185fdd2a-cd57-45e5-9986-bebd15e5761f-config-data podName:185fdd2a-cd57-45e5-9986-bebd15e5761f nodeName:}" failed. No retries permitted until 2025-12-03 12:08:33.636087023 +0000 UTC m=+2904.224328787 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "config-data" (UniqueName: "kubernetes.io/configmap/185fdd2a-cd57-45e5-9986-bebd15e5761f-config-data") pod "rabbitmq-cell1-server-0" (UID: "185fdd2a-cd57-45e5-9986-bebd15e5761f") : configmap "rabbitmq-cell1-config-data" not found Dec 03 12:08:29 crc kubenswrapper[5052]: E1203 12:08:29.737704 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/rabbitmq-cell1-config-data: configmap "rabbitmq-cell1-config-data" not found Dec 03 12:08:29 crc kubenswrapper[5052]: E1203 12:08:29.740672 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/06bfd2db-abe0-4b4d-a110-94a6a1e02211-config-data podName:06bfd2db-abe0-4b4d-a110-94a6a1e02211 nodeName:}" failed. No retries permitted until 2025-12-03 12:08:33.740648113 +0000 UTC m=+2904.328889877 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "config-data" (UniqueName: "kubernetes.io/configmap/06bfd2db-abe0-4b4d-a110-94a6a1e02211-config-data") pod "rabbitmq-cell1-server-2" (UID: "06bfd2db-abe0-4b4d-a110-94a6a1e02211") : configmap "rabbitmq-cell1-config-data" not found Dec 03 12:08:29 crc kubenswrapper[5052]: I1203 12:08:29.754353 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2066fe4a-e7cc-44a5-b7d9-499d19a40c12" path="/var/lib/kubelet/pods/2066fe4a-e7cc-44a5-b7d9-499d19a40c12/volumes" Dec 03 12:08:29 crc kubenswrapper[5052]: I1203 12:08:29.766275 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="89503fe3-9e65-4405-81f6-45cc9b5db612" path="/var/lib/kubelet/pods/89503fe3-9e65-4405-81f6-45cc9b5db612/volumes" Dec 03 12:08:29 crc kubenswrapper[5052]: I1203 12:08:29.766838 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="895b843a-707e-4cc5-8991-501116bd2fe3" path="/var/lib/kubelet/pods/895b843a-707e-4cc5-8991-501116bd2fe3/volumes" Dec 03 12:08:29 crc kubenswrapper[5052]: I1203 12:08:29.767384 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fdc5eec2-010f-4de6-a578-e5bda587d218" path="/var/lib/kubelet/pods/fdc5eec2-010f-4de6-a578-e5bda587d218/volumes" Dec 03 12:08:29 crc kubenswrapper[5052]: I1203 12:08:29.801408 5052 generic.go:334] "Generic (PLEG): container finished" podID="a3da2acb-8a0e-4ab3-8d15-2978eca73850" containerID="80fb2290d9e5fb396a3de096d08940f5a3a89a79f45d052e0d882756596d8b34" exitCode=143 Dec 03 12:08:29 crc kubenswrapper[5052]: I1203 12:08:29.801494 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-worker-c656b9c77-2gf7g" event={"ID":"a3da2acb-8a0e-4ab3-8d15-2978eca73850","Type":"ContainerDied","Data":"80fb2290d9e5fb396a3de096d08940f5a3a89a79f45d052e0d882756596d8b34"} Dec 03 12:08:29 crc kubenswrapper[5052]: I1203 12:08:29.809949 5052 generic.go:334] "Generic (PLEG): container finished" podID="d877b426-724d-4ccf-a113-05a96ab94935" containerID="8ed05d2153d95fbbc5b927a2573653ac6d68bb4bb432a7dd29017b0640952a6f" exitCode=143 Dec 03 12:08:29 crc kubenswrapper[5052]: I1203 12:08:29.810034 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-keystone-listener-64674fc4f4-wfh52" event={"ID":"d877b426-724d-4ccf-a113-05a96ab94935","Type":"ContainerDied","Data":"8ed05d2153d95fbbc5b927a2573653ac6d68bb4bb432a7dd29017b0640952a6f"} Dec 03 12:08:29 crc kubenswrapper[5052]: E1203 12:08:29.844764 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/dns-swift-storage-0: configmap "dns-swift-storage-0" not found Dec 03 12:08:29 crc kubenswrapper[5052]: I1203 12:08:29.845019 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement3c40-account-delete-8qsdr" event={"ID":"aaa3db64-a10b-45ea-9e25-de4916f30617","Type":"ContainerStarted","Data":"19566104651a9853da1345044a96ed40b73615a3c7ab75d772d57f5e5fdc9b84"} Dec 03 12:08:29 crc kubenswrapper[5052]: E1203 12:08:29.846263 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/77077fe5-d1ff-49c4-b049-3a0f2cee23b0-dns-swift-storage-0 podName:77077fe5-d1ff-49c4-b049-3a0f2cee23b0 nodeName:}" failed. No retries permitted until 2025-12-03 12:08:30.346239302 +0000 UTC m=+2900.934481066 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "dns-swift-storage-0" (UniqueName: "kubernetes.io/configmap/77077fe5-d1ff-49c4-b049-3a0f2cee23b0-dns-swift-storage-0") pod "dnsmasq-dnsmasq-85f8bc6fb5-b86k2" (UID: "77077fe5-d1ff-49c4-b049-3a0f2cee23b0") : configmap "dns-swift-storage-0" not found Dec 03 12:08:29 crc kubenswrapper[5052]: I1203 12:08:29.896061 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_ovsdbserver-nb-0_327a4d19-96ba-4b5e-a19f-56ba15a0cc82/ovsdbserver-nb/0.log" Dec 03 12:08:29 crc kubenswrapper[5052]: I1203 12:08:29.896381 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-nb-0" event={"ID":"327a4d19-96ba-4b5e-a19f-56ba15a0cc82","Type":"ContainerDied","Data":"487b73a468df70bd41429eee51087d37654668b064aa17d30ffaa4ebbede1a52"} Dec 03 12:08:29 crc kubenswrapper[5052]: I1203 12:08:29.896457 5052 scope.go:117] "RemoveContainer" containerID="023f6e55b1e44cead794ab88e34aa43141c551b1a2bce50e249cb4eeec4d2009" Dec 03 12:08:29 crc kubenswrapper[5052]: I1203 12:08:29.896645 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:08:29 crc kubenswrapper[5052]: I1203 12:08:29.932716 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/novaapi48e2-account-delete-kqhz7" event={"ID":"70122e67-6175-4a14-a120-6a2731c5dcb1","Type":"ContainerStarted","Data":"265855fad894b2f79c87685abb31a04136664b6466e2b8a541217f1335a297fc"} Dec 03 12:08:29 crc kubenswrapper[5052]: I1203 12:08:29.972164 5052 generic.go:334] "Generic (PLEG): container finished" podID="028738cd-3b98-4853-9506-d2eba91afd87" containerID="2d0b57b97cc7d912f25d72df376cda014d9ee39ad7a02f4b38c46e47d7ba19ac" exitCode=0 Dec 03 12:08:29 crc kubenswrapper[5052]: I1203 12:08:29.972191 5052 generic.go:334] "Generic (PLEG): container finished" podID="028738cd-3b98-4853-9506-d2eba91afd87" containerID="531801a1d9fdbb4eab4dc17c841d530240327b26c1a21395bf0c594278f80f5c" exitCode=0 Dec 03 12:08:29 crc kubenswrapper[5052]: I1203 12:08:29.972238 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-proxy-9bb7db5d-5q7wc" event={"ID":"028738cd-3b98-4853-9506-d2eba91afd87","Type":"ContainerDied","Data":"2d0b57b97cc7d912f25d72df376cda014d9ee39ad7a02f4b38c46e47d7ba19ac"} Dec 03 12:08:29 crc kubenswrapper[5052]: I1203 12:08:29.972271 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-proxy-9bb7db5d-5q7wc" event={"ID":"028738cd-3b98-4853-9506-d2eba91afd87","Type":"ContainerDied","Data":"531801a1d9fdbb4eab4dc17c841d530240327b26c1a21395bf0c594278f80f5c"} Dec 03 12:08:29 crc kubenswrapper[5052]: I1203 12:08:29.981867 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cindere6fc-account-delete-z7w6l" event={"ID":"89ee1bab-2c68-4fc4-9171-a7330922faf6","Type":"ContainerStarted","Data":"e1e5f1e1f180364add520eba6060fbec83c2ab074ee509c5c280da93bca650db"} Dec 03 12:08:30 crc kubenswrapper[5052]: I1203 12:08:30.017442 5052 generic.go:334] "Generic (PLEG): container finished" podID="e66cca73-f322-4c6e-adb0-5ec43607bb73" containerID="81211fcc809459c82abca1a15005c67c33acfa54010915f26c5a7c1bed4bf50d" exitCode=0 Dec 03 12:08:30 crc kubenswrapper[5052]: I1203 12:08:30.017586 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" event={"ID":"e66cca73-f322-4c6e-adb0-5ec43607bb73","Type":"ContainerDied","Data":"81211fcc809459c82abca1a15005c67c33acfa54010915f26c5a7c1bed4bf50d"} Dec 03 12:08:30 crc kubenswrapper[5052]: I1203 12:08:30.023346 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutrond5c4-account-delete-zj77r" event={"ID":"9e6af047-b7ee-459a-b362-da5f777d0fa0","Type":"ContainerStarted","Data":"cd9af7528dabedd1b57634f875a45a028137dc589398e5472f322ce8266304eb"} Dec 03 12:08:30 crc kubenswrapper[5052]: I1203 12:08:30.033974 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-kcb42" event={"ID":"f657b0c3-883e-4e3d-b161-1b3ef0988ab0","Type":"ContainerStarted","Data":"4c8cb428cd412adcc11141dc9c500eb7f270887e14784e9522cf1ff216e1d58e"} Dec 03 12:08:30 crc kubenswrapper[5052]: I1203 12:08:30.046576 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance8fcf-account-delete-4xhcc" event={"ID":"2f3b4f6a-5f11-4039-a3bd-7b553ab5684d","Type":"ContainerStarted","Data":"003f41bab553dfad821839177281778304bb5889d3df5b1416ecb71017b910c3"} Dec 03 12:08:30 crc kubenswrapper[5052]: I1203 12:08:30.068234 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican5c55-account-delete-zttb9" event={"ID":"1bc7c8b1-cc0d-4611-b335-01c2b91713e2","Type":"ContainerStarted","Data":"9798f70f1a0313c2fd23b4af0551bf82e2750ff87ce05f947fc3131408eba2e5"} Dec 03 12:08:30 crc kubenswrapper[5052]: E1203 12:08:30.072726 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/rabbitmq-cell1-config-data: configmap "rabbitmq-cell1-config-data" not found Dec 03 12:08:30 crc kubenswrapper[5052]: E1203 12:08:30.072795 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3-config-data podName:f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3 nodeName:}" failed. No retries permitted until 2025-12-03 12:08:34.072777368 +0000 UTC m=+2904.661019132 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "config-data" (UniqueName: "kubernetes.io/configmap/f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3-config-data") pod "rabbitmq-cell1-server-1" (UID: "f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3") : configmap "rabbitmq-cell1-config-data" not found Dec 03 12:08:30 crc kubenswrapper[5052]: I1203 12:08:30.085739 5052 generic.go:334] "Generic (PLEG): container finished" podID="141a49da-93b6-483b-a476-1d5f3efc29d1" containerID="405640dc668d7b8021968432854058e5508074169be4f1e98d36626dd8db3fbf" exitCode=0 Dec 03 12:08:30 crc kubenswrapper[5052]: I1203 12:08:30.085812 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-5d6b56fbdb-ht2b8" event={"ID":"141a49da-93b6-483b-a476-1d5f3efc29d1","Type":"ContainerDied","Data":"405640dc668d7b8021968432854058e5508074169be4f1e98d36626dd8db3fbf"} Dec 03 12:08:30 crc kubenswrapper[5052]: I1203 12:08:30.124078 5052 generic.go:334] "Generic (PLEG): container finished" podID="95bf5f72-82f3-4e88-a576-01dae857541f" containerID="4a4d3e25edd055d3f3503821db30046cfd815c08421228eb4d5e76a3f4e38a23" exitCode=0 Dec 03 12:08:30 crc kubenswrapper[5052]: I1203 12:08:30.124120 5052 generic.go:334] "Generic (PLEG): container finished" podID="95bf5f72-82f3-4e88-a576-01dae857541f" containerID="da24ecbec4ad70b7a0db3646cffcca7dc56b37efffe6e261de63720e146b2b8e" exitCode=0 Dec 03 12:08:30 crc kubenswrapper[5052]: I1203 12:08:30.124129 5052 generic.go:334] "Generic (PLEG): container finished" podID="95bf5f72-82f3-4e88-a576-01dae857541f" containerID="a4050b9976060a4f925ffaa2cb889b98f3b107441d8ca49ad092c8be65814639" exitCode=0 Dec 03 12:08:30 crc kubenswrapper[5052]: I1203 12:08:30.124138 5052 generic.go:334] "Generic (PLEG): container finished" podID="95bf5f72-82f3-4e88-a576-01dae857541f" containerID="9fa20306dd177e25e71d2c3ec088ca38dc2879ff9f7569b37d3b1ab695d20bed" exitCode=0 Dec 03 12:08:30 crc kubenswrapper[5052]: I1203 12:08:30.124146 5052 generic.go:334] "Generic (PLEG): container finished" podID="95bf5f72-82f3-4e88-a576-01dae857541f" containerID="003e4620ad1989f1fa65349bb9193c7be42568b4061355bee57a6b9ae2b3d5a2" exitCode=0 Dec 03 12:08:30 crc kubenswrapper[5052]: I1203 12:08:30.124242 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"95bf5f72-82f3-4e88-a576-01dae857541f","Type":"ContainerDied","Data":"4a4d3e25edd055d3f3503821db30046cfd815c08421228eb4d5e76a3f4e38a23"} Dec 03 12:08:30 crc kubenswrapper[5052]: I1203 12:08:30.124272 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"95bf5f72-82f3-4e88-a576-01dae857541f","Type":"ContainerDied","Data":"da24ecbec4ad70b7a0db3646cffcca7dc56b37efffe6e261de63720e146b2b8e"} Dec 03 12:08:30 crc kubenswrapper[5052]: I1203 12:08:30.124283 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"95bf5f72-82f3-4e88-a576-01dae857541f","Type":"ContainerDied","Data":"a4050b9976060a4f925ffaa2cb889b98f3b107441d8ca49ad092c8be65814639"} Dec 03 12:08:30 crc kubenswrapper[5052]: I1203 12:08:30.124292 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"95bf5f72-82f3-4e88-a576-01dae857541f","Type":"ContainerDied","Data":"9fa20306dd177e25e71d2c3ec088ca38dc2879ff9f7569b37d3b1ab695d20bed"} Dec 03 12:08:30 crc kubenswrapper[5052]: I1203 12:08:30.124303 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"95bf5f72-82f3-4e88-a576-01dae857541f","Type":"ContainerDied","Data":"003e4620ad1989f1fa65349bb9193c7be42568b4061355bee57a6b9ae2b3d5a2"} Dec 03 12:08:30 crc kubenswrapper[5052]: I1203 12:08:30.141299 5052 generic.go:334] "Generic (PLEG): container finished" podID="60f72892-a5aa-482f-aac3-5713ba028bc8" containerID="6e5a8a55dfb233734ce359d756ebf08a407cc39b08dc02754ac067f403d0a466" exitCode=0 Dec 03 12:08:30 crc kubenswrapper[5052]: I1203 12:08:30.141362 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"60f72892-a5aa-482f-aac3-5713ba028bc8","Type":"ContainerDied","Data":"6e5a8a55dfb233734ce359d756ebf08a407cc39b08dc02754ac067f403d0a466"} Dec 03 12:08:30 crc kubenswrapper[5052]: I1203 12:08:30.214857 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstackclient" Dec 03 12:08:30 crc kubenswrapper[5052]: I1203 12:08:30.218733 5052 scope.go:117] "RemoveContainer" containerID="04c71fa1d03c17ad50a53dfb4585c12716ea5f19b448fb01a9fbd6d86935e208" Dec 03 12:08:30 crc kubenswrapper[5052]: I1203 12:08:30.227575 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-nb-0"] Dec 03 12:08:30 crc kubenswrapper[5052]: I1203 12:08:30.240786 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-nb-0"] Dec 03 12:08:30 crc kubenswrapper[5052]: I1203 12:08:30.279679 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:08:30 crc kubenswrapper[5052]: I1203 12:08:30.281711 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-scheduler-0" podUID="0b100677-2da0-4717-a15a-d2463ee6713e" containerName="nova-scheduler-scheduler" containerID="cri-o://df9dfde29f7996c2208b2860c440e3ab6120cd657e154f046ad4cdb1b318a678" gracePeriod=30 Dec 03 12:08:30 crc kubenswrapper[5052]: I1203 12:08:30.382607 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c7d0ffdb-bb0c-4212-9682-289d630186fd-combined-ca-bundle\") pod \"c7d0ffdb-bb0c-4212-9682-289d630186fd\" (UID: \"c7d0ffdb-bb0c-4212-9682-289d630186fd\") " Dec 03 12:08:30 crc kubenswrapper[5052]: I1203 12:08:30.382756 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"openstack-config-secret\" (UniqueName: \"kubernetes.io/secret/c7d0ffdb-bb0c-4212-9682-289d630186fd-openstack-config-secret\") pod \"c7d0ffdb-bb0c-4212-9682-289d630186fd\" (UID: \"c7d0ffdb-bb0c-4212-9682-289d630186fd\") " Dec 03 12:08:30 crc kubenswrapper[5052]: I1203 12:08:30.382824 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"openstack-config\" (UniqueName: \"kubernetes.io/configmap/c7d0ffdb-bb0c-4212-9682-289d630186fd-openstack-config\") pod \"c7d0ffdb-bb0c-4212-9682-289d630186fd\" (UID: \"c7d0ffdb-bb0c-4212-9682-289d630186fd\") " Dec 03 12:08:30 crc kubenswrapper[5052]: I1203 12:08:30.382890 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5wxll\" (UniqueName: \"kubernetes.io/projected/c7d0ffdb-bb0c-4212-9682-289d630186fd-kube-api-access-5wxll\") pod \"c7d0ffdb-bb0c-4212-9682-289d630186fd\" (UID: \"c7d0ffdb-bb0c-4212-9682-289d630186fd\") " Dec 03 12:08:30 crc kubenswrapper[5052]: E1203 12:08:30.383483 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/dns-swift-storage-0: configmap "dns-swift-storage-0" not found Dec 03 12:08:30 crc kubenswrapper[5052]: E1203 12:08:30.383699 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/77077fe5-d1ff-49c4-b049-3a0f2cee23b0-dns-swift-storage-0 podName:77077fe5-d1ff-49c4-b049-3a0f2cee23b0 nodeName:}" failed. No retries permitted until 2025-12-03 12:08:31.383541323 +0000 UTC m=+2901.971783087 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "dns-swift-storage-0" (UniqueName: "kubernetes.io/configmap/77077fe5-d1ff-49c4-b049-3a0f2cee23b0-dns-swift-storage-0") pod "dnsmasq-dnsmasq-85f8bc6fb5-b86k2" (UID: "77077fe5-d1ff-49c4-b049-3a0f2cee23b0") : configmap "dns-swift-storage-0" not found Dec 03 12:08:30 crc kubenswrapper[5052]: I1203 12:08:30.400788 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c7d0ffdb-bb0c-4212-9682-289d630186fd-kube-api-access-5wxll" (OuterVolumeSpecName: "kube-api-access-5wxll") pod "c7d0ffdb-bb0c-4212-9682-289d630186fd" (UID: "c7d0ffdb-bb0c-4212-9682-289d630186fd"). InnerVolumeSpecName "kube-api-access-5wxll". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:08:30 crc kubenswrapper[5052]: I1203 12:08:30.481175 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" podUID="e66cca73-f322-4c6e-adb0-5ec43607bb73" containerName="nova-cell1-novncproxy-novncproxy" probeResult="failure" output="Get \"https://10.217.0.184:6080/vnc_lite.html\": dial tcp 10.217.0.184:6080: connect: connection refused" Dec 03 12:08:30 crc kubenswrapper[5052]: I1203 12:08:30.486924 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c7d0ffdb-bb0c-4212-9682-289d630186fd-openstack-config" (OuterVolumeSpecName: "openstack-config") pod "c7d0ffdb-bb0c-4212-9682-289d630186fd" (UID: "c7d0ffdb-bb0c-4212-9682-289d630186fd"). InnerVolumeSpecName "openstack-config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:08:30 crc kubenswrapper[5052]: I1203 12:08:30.487316 5052 reconciler_common.go:293] "Volume detached for volume \"openstack-config\" (UniqueName: \"kubernetes.io/configmap/c7d0ffdb-bb0c-4212-9682-289d630186fd-openstack-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:30 crc kubenswrapper[5052]: I1203 12:08:30.487338 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-5wxll\" (UniqueName: \"kubernetes.io/projected/c7d0ffdb-bb0c-4212-9682-289d630186fd-kube-api-access-5wxll\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:30 crc kubenswrapper[5052]: I1203 12:08:30.496677 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c7d0ffdb-bb0c-4212-9682-289d630186fd-openstack-config-secret" (OuterVolumeSpecName: "openstack-config-secret") pod "c7d0ffdb-bb0c-4212-9682-289d630186fd" (UID: "c7d0ffdb-bb0c-4212-9682-289d630186fd"). InnerVolumeSpecName "openstack-config-secret". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:08:30 crc kubenswrapper[5052]: I1203 12:08:30.528608 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c7d0ffdb-bb0c-4212-9682-289d630186fd-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "c7d0ffdb-bb0c-4212-9682-289d630186fd" (UID: "c7d0ffdb-bb0c-4212-9682-289d630186fd"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:08:30 crc kubenswrapper[5052]: I1203 12:08:30.568684 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_ovsdbserver-sb-0_78dad0d3-18e5-4900-932c-56ede3279dc0/ovsdbserver-sb/0.log" Dec 03 12:08:30 crc kubenswrapper[5052]: I1203 12:08:30.568813 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:08:30 crc kubenswrapper[5052]: I1203 12:08:30.589276 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c7d0ffdb-bb0c-4212-9682-289d630186fd-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:30 crc kubenswrapper[5052]: I1203 12:08:30.589311 5052 reconciler_common.go:293] "Volume detached for volume \"openstack-config-secret\" (UniqueName: \"kubernetes.io/secret/c7d0ffdb-bb0c-4212-9682-289d630186fd-openstack-config-secret\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:30 crc kubenswrapper[5052]: E1203 12:08:30.656159 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="f82e335d0814000e829c2020ade84de1617dea07a919392e9736c05912d0b4bf" cmd=["/usr/local/bin/container-scripts/status_check.sh"] Dec 03 12:08:30 crc kubenswrapper[5052]: E1203 12:08:30.658972 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="f82e335d0814000e829c2020ade84de1617dea07a919392e9736c05912d0b4bf" cmd=["/usr/local/bin/container-scripts/status_check.sh"] Dec 03 12:08:30 crc kubenswrapper[5052]: E1203 12:08:30.666787 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="df9dfde29f7996c2208b2860c440e3ab6120cd657e154f046ad4cdb1b318a678" cmd=["/usr/bin/pgrep","-r","DRST","nova-scheduler"] Dec 03 12:08:30 crc kubenswrapper[5052]: E1203 12:08:30.667242 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="f82e335d0814000e829c2020ade84de1617dea07a919392e9736c05912d0b4bf" cmd=["/usr/local/bin/container-scripts/status_check.sh"] Dec 03 12:08:30 crc kubenswrapper[5052]: E1203 12:08:30.667273 5052 prober.go:104] "Probe errored" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" probeType="Readiness" pod="openstack-kuttl-tests/ovn-northd-0" podUID="fad6f453-acf5-4f13-94d6-7d46e300f28c" containerName="ovn-northd" Dec 03 12:08:30 crc kubenswrapper[5052]: E1203 12:08:30.668119 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="df9dfde29f7996c2208b2860c440e3ab6120cd657e154f046ad4cdb1b318a678" cmd=["/usr/bin/pgrep","-r","DRST","nova-scheduler"] Dec 03 12:08:30 crc kubenswrapper[5052]: E1203 12:08:30.669478 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="df9dfde29f7996c2208b2860c440e3ab6120cd657e154f046ad4cdb1b318a678" cmd=["/usr/bin/pgrep","-r","DRST","nova-scheduler"] Dec 03 12:08:30 crc kubenswrapper[5052]: E1203 12:08:30.669503 5052 prober.go:104] "Probe errored" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" probeType="Readiness" pod="openstack-kuttl-tests/nova-scheduler-0" podUID="0b100677-2da0-4717-a15a-d2463ee6713e" containerName="nova-scheduler-scheduler" Dec 03 12:08:30 crc kubenswrapper[5052]: I1203 12:08:30.692619 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jp6lk\" (UniqueName: \"kubernetes.io/projected/78dad0d3-18e5-4900-932c-56ede3279dc0-kube-api-access-jp6lk\") pod \"78dad0d3-18e5-4900-932c-56ede3279dc0\" (UID: \"78dad0d3-18e5-4900-932c-56ede3279dc0\") " Dec 03 12:08:30 crc kubenswrapper[5052]: I1203 12:08:30.692760 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/78dad0d3-18e5-4900-932c-56ede3279dc0-ovsdb-rundir\") pod \"78dad0d3-18e5-4900-932c-56ede3279dc0\" (UID: \"78dad0d3-18e5-4900-932c-56ede3279dc0\") " Dec 03 12:08:30 crc kubenswrapper[5052]: I1203 12:08:30.693254 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/78dad0d3-18e5-4900-932c-56ede3279dc0-ovsdb-rundir" (OuterVolumeSpecName: "ovsdb-rundir") pod "78dad0d3-18e5-4900-932c-56ede3279dc0" (UID: "78dad0d3-18e5-4900-932c-56ede3279dc0"). InnerVolumeSpecName "ovsdb-rundir". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:08:30 crc kubenswrapper[5052]: I1203 12:08:30.693372 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovsdbserver-sb-tls-certs\" (UniqueName: \"kubernetes.io/secret/78dad0d3-18e5-4900-932c-56ede3279dc0-ovsdbserver-sb-tls-certs\") pod \"78dad0d3-18e5-4900-932c-56ede3279dc0\" (UID: \"78dad0d3-18e5-4900-932c-56ede3279dc0\") " Dec 03 12:08:30 crc kubenswrapper[5052]: I1203 12:08:30.693443 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovndbcluster-sb-etc-ovn\" (UniqueName: \"kubernetes.io/local-volume/local-storage17-crc\") pod \"78dad0d3-18e5-4900-932c-56ede3279dc0\" (UID: \"78dad0d3-18e5-4900-932c-56ede3279dc0\") " Dec 03 12:08:30 crc kubenswrapper[5052]: I1203 12:08:30.693768 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/78dad0d3-18e5-4900-932c-56ede3279dc0-metrics-certs-tls-certs\") pod \"78dad0d3-18e5-4900-932c-56ede3279dc0\" (UID: \"78dad0d3-18e5-4900-932c-56ede3279dc0\") " Dec 03 12:08:30 crc kubenswrapper[5052]: I1203 12:08:30.693818 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/78dad0d3-18e5-4900-932c-56ede3279dc0-config\") pod \"78dad0d3-18e5-4900-932c-56ede3279dc0\" (UID: \"78dad0d3-18e5-4900-932c-56ede3279dc0\") " Dec 03 12:08:30 crc kubenswrapper[5052]: I1203 12:08:30.694790 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/78dad0d3-18e5-4900-932c-56ede3279dc0-config" (OuterVolumeSpecName: "config") pod "78dad0d3-18e5-4900-932c-56ede3279dc0" (UID: "78dad0d3-18e5-4900-932c-56ede3279dc0"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:08:30 crc kubenswrapper[5052]: I1203 12:08:30.695086 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/78dad0d3-18e5-4900-932c-56ede3279dc0-combined-ca-bundle\") pod \"78dad0d3-18e5-4900-932c-56ede3279dc0\" (UID: \"78dad0d3-18e5-4900-932c-56ede3279dc0\") " Dec 03 12:08:30 crc kubenswrapper[5052]: I1203 12:08:30.695197 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/78dad0d3-18e5-4900-932c-56ede3279dc0-scripts\") pod \"78dad0d3-18e5-4900-932c-56ede3279dc0\" (UID: \"78dad0d3-18e5-4900-932c-56ede3279dc0\") " Dec 03 12:08:30 crc kubenswrapper[5052]: I1203 12:08:30.696802 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/78dad0d3-18e5-4900-932c-56ede3279dc0-scripts" (OuterVolumeSpecName: "scripts") pod "78dad0d3-18e5-4900-932c-56ede3279dc0" (UID: "78dad0d3-18e5-4900-932c-56ede3279dc0"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:08:30 crc kubenswrapper[5052]: I1203 12:08:30.696952 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/78dad0d3-18e5-4900-932c-56ede3279dc0-kube-api-access-jp6lk" (OuterVolumeSpecName: "kube-api-access-jp6lk") pod "78dad0d3-18e5-4900-932c-56ede3279dc0" (UID: "78dad0d3-18e5-4900-932c-56ede3279dc0"). InnerVolumeSpecName "kube-api-access-jp6lk". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:08:30 crc kubenswrapper[5052]: I1203 12:08:30.697189 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage17-crc" (OuterVolumeSpecName: "ovndbcluster-sb-etc-ovn") pod "78dad0d3-18e5-4900-932c-56ede3279dc0" (UID: "78dad0d3-18e5-4900-932c-56ede3279dc0"). InnerVolumeSpecName "local-storage17-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 12:08:30 crc kubenswrapper[5052]: I1203 12:08:30.697893 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage17-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage17-crc\") on node \"crc\" " Dec 03 12:08:30 crc kubenswrapper[5052]: I1203 12:08:30.697913 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/78dad0d3-18e5-4900-932c-56ede3279dc0-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:30 crc kubenswrapper[5052]: I1203 12:08:30.697923 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/78dad0d3-18e5-4900-932c-56ede3279dc0-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:30 crc kubenswrapper[5052]: I1203 12:08:30.704699 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-jp6lk\" (UniqueName: \"kubernetes.io/projected/78dad0d3-18e5-4900-932c-56ede3279dc0-kube-api-access-jp6lk\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:30 crc kubenswrapper[5052]: I1203 12:08:30.704728 5052 reconciler_common.go:293] "Volume detached for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/78dad0d3-18e5-4900-932c-56ede3279dc0-ovsdb-rundir\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:30 crc kubenswrapper[5052]: I1203 12:08:30.723083 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/78dad0d3-18e5-4900-932c-56ede3279dc0-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "78dad0d3-18e5-4900-932c-56ede3279dc0" (UID: "78dad0d3-18e5-4900-932c-56ede3279dc0"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:08:30 crc kubenswrapper[5052]: I1203 12:08:30.724286 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage17-crc" (UniqueName: "kubernetes.io/local-volume/local-storage17-crc") on node "crc" Dec 03 12:08:30 crc kubenswrapper[5052]: I1203 12:08:30.779668 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/78dad0d3-18e5-4900-932c-56ede3279dc0-metrics-certs-tls-certs" (OuterVolumeSpecName: "metrics-certs-tls-certs") pod "78dad0d3-18e5-4900-932c-56ede3279dc0" (UID: "78dad0d3-18e5-4900-932c-56ede3279dc0"). InnerVolumeSpecName "metrics-certs-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:08:30 crc kubenswrapper[5052]: I1203 12:08:30.785442 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/78dad0d3-18e5-4900-932c-56ede3279dc0-ovsdbserver-sb-tls-certs" (OuterVolumeSpecName: "ovsdbserver-sb-tls-certs") pod "78dad0d3-18e5-4900-932c-56ede3279dc0" (UID: "78dad0d3-18e5-4900-932c-56ede3279dc0"). InnerVolumeSpecName "ovsdbserver-sb-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:08:30 crc kubenswrapper[5052]: I1203 12:08:30.806403 5052 reconciler_common.go:293] "Volume detached for volume \"ovsdbserver-sb-tls-certs\" (UniqueName: \"kubernetes.io/secret/78dad0d3-18e5-4900-932c-56ede3279dc0-ovsdbserver-sb-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:30 crc kubenswrapper[5052]: I1203 12:08:30.806441 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage17-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage17-crc\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:30 crc kubenswrapper[5052]: I1203 12:08:30.806450 5052 reconciler_common.go:293] "Volume detached for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/78dad0d3-18e5-4900-932c-56ede3279dc0-metrics-certs-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:30 crc kubenswrapper[5052]: I1203 12:08:30.806458 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/78dad0d3-18e5-4900-932c-56ede3279dc0-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:31 crc kubenswrapper[5052]: I1203 12:08:31.190735 5052 scope.go:117] "RemoveContainer" containerID="dbd211edde83ca19480c688ca164c8a5521ad9be2a864cfa3177c9b6dafa1a07" Dec 03 12:08:31 crc kubenswrapper[5052]: I1203 12:08:31.191181 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstackclient" Dec 03 12:08:31 crc kubenswrapper[5052]: I1203 12:08:31.223742 5052 generic.go:334] "Generic (PLEG): container finished" podID="e92fa4d7-9bd6-4586-b41a-a557fb672439" containerID="8709692766b7010f69ca646129fa46d2b1115bc2a7f05706db51e69fa086eafa" exitCode=0 Dec 03 12:08:31 crc kubenswrapper[5052]: I1203 12:08:31.223833 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-conductor-0" event={"ID":"e92fa4d7-9bd6-4586-b41a-a557fb672439","Type":"ContainerDied","Data":"8709692766b7010f69ca646129fa46d2b1115bc2a7f05706db51e69fa086eafa"} Dec 03 12:08:31 crc kubenswrapper[5052]: I1203 12:08:31.266803 5052 generic.go:334] "Generic (PLEG): container finished" podID="4cae4fe5-bc0e-4088-b403-c78e940373c2" containerID="b19b072c0fb517fe3875dc65637b798d06ddeac368aceb24adb64b956c3e8597" exitCode=0 Dec 03 12:08:31 crc kubenswrapper[5052]: I1203 12:08:31.266893 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-conductor-0" event={"ID":"4cae4fe5-bc0e-4088-b403-c78e940373c2","Type":"ContainerDied","Data":"b19b072c0fb517fe3875dc65637b798d06ddeac368aceb24adb64b956c3e8597"} Dec 03 12:08:31 crc kubenswrapper[5052]: I1203 12:08:31.284476 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_ovsdbserver-sb-0_78dad0d3-18e5-4900-932c-56ede3279dc0/ovsdbserver-sb/0.log" Dec 03 12:08:31 crc kubenswrapper[5052]: I1203 12:08:31.284621 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-sb-0" event={"ID":"78dad0d3-18e5-4900-932c-56ede3279dc0","Type":"ContainerDied","Data":"0cc27cb717dc9537f83778189f76fdddfa28ca3f3a6e4b31159ccfb03fb8e97a"} Dec 03 12:08:31 crc kubenswrapper[5052]: I1203 12:08:31.284707 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:08:31 crc kubenswrapper[5052]: I1203 12:08:31.297603 5052 scope.go:117] "RemoveContainer" containerID="b746c813f9a2abaab7eee298828559f29ccd2f1796875dbbe947a37faa740dfb" Dec 03 12:08:31 crc kubenswrapper[5052]: I1203 12:08:31.303521 5052 generic.go:334] "Generic (PLEG): container finished" podID="6999e86b-8f95-4002-9959-6288a7d783bd" containerID="3e8fef46e8b29b153dd3541055f4f411078ff175402ded3c5e2a2532584522cb" exitCode=0 Dec 03 12:08:31 crc kubenswrapper[5052]: I1203 12:08:31.303567 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-f4dfbd7d-mxsm8" event={"ID":"6999e86b-8f95-4002-9959-6288a7d783bd","Type":"ContainerDied","Data":"3e8fef46e8b29b153dd3541055f4f411078ff175402ded3c5e2a2532584522cb"} Dec 03 12:08:31 crc kubenswrapper[5052]: I1203 12:08:31.356054 5052 generic.go:334] "Generic (PLEG): container finished" podID="b9efd251-95c2-4224-b318-a34279ee1017" containerID="0d0239c7224731fd55404636504e2c22a3b640502408dba4f002511377ec7f33" exitCode=0 Dec 03 12:08:31 crc kubenswrapper[5052]: I1203 12:08:31.356130 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"b9efd251-95c2-4224-b318-a34279ee1017","Type":"ContainerDied","Data":"0d0239c7224731fd55404636504e2c22a3b640502408dba4f002511377ec7f33"} Dec 03 12:08:31 crc kubenswrapper[5052]: I1203 12:08:31.408388 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-sb-0"] Dec 03 12:08:31 crc kubenswrapper[5052]: I1203 12:08:31.416371 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/novacell0d294-account-delete-b64wn" event={"ID":"030d4f76-eb30-4c44-b782-fa7147febf41","Type":"ContainerStarted","Data":"f6003a5dd700c21fc31c9193f8092e42fdfba4431f6f7eb85497fdf04c513ed1"} Dec 03 12:08:31 crc kubenswrapper[5052]: E1203 12:08:31.436865 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/dns-swift-storage-0: configmap "dns-swift-storage-0" not found Dec 03 12:08:31 crc kubenswrapper[5052]: E1203 12:08:31.436932 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/77077fe5-d1ff-49c4-b049-3a0f2cee23b0-dns-swift-storage-0 podName:77077fe5-d1ff-49c4-b049-3a0f2cee23b0 nodeName:}" failed. No retries permitted until 2025-12-03 12:08:33.436917378 +0000 UTC m=+2904.025159142 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "dns-swift-storage-0" (UniqueName: "kubernetes.io/configmap/77077fe5-d1ff-49c4-b049-3a0f2cee23b0-dns-swift-storage-0") pod "dnsmasq-dnsmasq-85f8bc6fb5-b86k2" (UID: "77077fe5-d1ff-49c4-b049-3a0f2cee23b0") : configmap "dns-swift-storage-0" not found Dec 03 12:08:31 crc kubenswrapper[5052]: I1203 12:08:31.450976 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-sb-0"] Dec 03 12:08:31 crc kubenswrapper[5052]: I1203 12:08:31.454721 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-kcb42" event={"ID":"f657b0c3-883e-4e3d-b161-1b3ef0988ab0","Type":"ContainerStarted","Data":"686b357f3c7f38691d18d6fa7206dbd5e60dc6a7b6acf91c922e01438257cfeb"} Dec 03 12:08:31 crc kubenswrapper[5052]: I1203 12:08:31.459918 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/novacell0d294-account-delete-b64wn" podStartSLOduration=5.459897635 podStartE2EDuration="5.459897635s" podCreationTimestamp="2025-12-03 12:08:26 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:08:31.452671009 +0000 UTC m=+2902.040912803" watchObservedRunningTime="2025-12-03 12:08:31.459897635 +0000 UTC m=+2902.048139399" Dec 03 12:08:31 crc kubenswrapper[5052]: I1203 12:08:31.474717 5052 scope.go:117] "RemoveContainer" containerID="3fa9a29a156d24f947438d4dd3ac36264903dbd4e777dddc9352f1aab01266e5" Dec 03 12:08:31 crc kubenswrapper[5052]: I1203 12:08:31.503070 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/cindere6fc-account-delete-z7w6l" podStartSLOduration=6.503050419 podStartE2EDuration="6.503050419s" podCreationTimestamp="2025-12-03 12:08:25 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:08:31.501816644 +0000 UTC m=+2902.090058408" watchObservedRunningTime="2025-12-03 12:08:31.503050419 +0000 UTC m=+2902.091292183" Dec 03 12:08:31 crc kubenswrapper[5052]: I1203 12:08:31.514386 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/nova-api-0" podUID="45e26e8c-71b5-4b90-bba7-5bad2650d9b2" containerName="nova-api-log" probeResult="failure" output="Get \"https://10.217.0.196:8774/\": read tcp 10.217.0.2:53040->10.217.0.196:8774: read: connection reset by peer" Dec 03 12:08:31 crc kubenswrapper[5052]: I1203 12:08:31.514668 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/nova-api-0" podUID="45e26e8c-71b5-4b90-bba7-5bad2650d9b2" containerName="nova-api-api" probeResult="failure" output="Get \"https://10.217.0.196:8774/\": read tcp 10.217.0.2:53024->10.217.0.196:8774: read: connection reset by peer" Dec 03 12:08:31 crc kubenswrapper[5052]: I1203 12:08:31.543078 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/barbican-api-d978c699b-kzzst" podUID="de04776b-ea02-40d9-8459-0a519c8c64ad" containerName="barbican-api-log" probeResult="failure" output="Get \"https://10.217.0.156:9311/healthcheck\": read tcp 10.217.0.2:48736->10.217.0.156:9311: read: connection reset by peer" Dec 03 12:08:31 crc kubenswrapper[5052]: I1203 12:08:31.543796 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/barbican-api-d978c699b-kzzst" podUID="de04776b-ea02-40d9-8459-0a519c8c64ad" containerName="barbican-api" probeResult="failure" output="Get \"https://10.217.0.156:9311/healthcheck\": read tcp 10.217.0.2:48748->10.217.0.156:9311: read: connection reset by peer" Dec 03 12:08:31 crc kubenswrapper[5052]: I1203 12:08:31.657676 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/neutrond5c4-account-delete-zj77r" podStartSLOduration=5.657654519 podStartE2EDuration="5.657654519s" podCreationTimestamp="2025-12-03 12:08:26 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:08:31.604222882 +0000 UTC m=+2902.192464646" watchObservedRunningTime="2025-12-03 12:08:31.657654519 +0000 UTC m=+2902.245896283" Dec 03 12:08:31 crc kubenswrapper[5052]: I1203 12:08:31.658437 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/barbican5c55-account-delete-zttb9" podStartSLOduration=6.658431782 podStartE2EDuration="6.658431782s" podCreationTimestamp="2025-12-03 12:08:25 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:08:31.584480747 +0000 UTC m=+2902.172722511" watchObservedRunningTime="2025-12-03 12:08:31.658431782 +0000 UTC m=+2902.246673546" Dec 03 12:08:31 crc kubenswrapper[5052]: I1203 12:08:31.676116 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/novaapi48e2-account-delete-kqhz7" podStartSLOduration=5.676097147 podStartE2EDuration="5.676097147s" podCreationTimestamp="2025-12-03 12:08:26 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:08:31.633045716 +0000 UTC m=+2902.221287480" watchObservedRunningTime="2025-12-03 12:08:31.676097147 +0000 UTC m=+2902.264338911" Dec 03 12:08:31 crc kubenswrapper[5052]: I1203 12:08:31.727530 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/glance8fcf-account-delete-4xhcc" podStartSLOduration=6.727495086 podStartE2EDuration="6.727495086s" podCreationTimestamp="2025-12-03 12:08:25 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:08:31.678454264 +0000 UTC m=+2902.266696028" watchObservedRunningTime="2025-12-03 12:08:31.727495086 +0000 UTC m=+2902.315736850" Dec 03 12:08:31 crc kubenswrapper[5052]: I1203 12:08:31.735337 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/placement3c40-account-delete-8qsdr" podStartSLOduration=6.73531826 podStartE2EDuration="6.73531826s" podCreationTimestamp="2025-12-03 12:08:25 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:08:31.699073834 +0000 UTC m=+2902.287315598" watchObservedRunningTime="2025-12-03 12:08:31.73531826 +0000 UTC m=+2902.323560024" Dec 03 12:08:31 crc kubenswrapper[5052]: I1203 12:08:31.741776 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/cinder-api-0" podUID="31e6a111-bd4a-45a5-95a0-a6b537added6" containerName="cinder-api" probeResult="failure" output="Get \"https://10.217.0.157:8776/healthcheck\": read tcp 10.217.0.2:40462->10.217.0.157:8776: read: connection reset by peer" Dec 03 12:08:31 crc kubenswrapper[5052]: I1203 12:08:31.753766 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="327a4d19-96ba-4b5e-a19f-56ba15a0cc82" path="/var/lib/kubelet/pods/327a4d19-96ba-4b5e-a19f-56ba15a0cc82/volumes" Dec 03 12:08:31 crc kubenswrapper[5052]: I1203 12:08:31.754677 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="78dad0d3-18e5-4900-932c-56ede3279dc0" path="/var/lib/kubelet/pods/78dad0d3-18e5-4900-932c-56ede3279dc0/volumes" Dec 03 12:08:31 crc kubenswrapper[5052]: I1203 12:08:31.755896 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c7d0ffdb-bb0c-4212-9682-289d630186fd" path="/var/lib/kubelet/pods/c7d0ffdb-bb0c-4212-9682-289d630186fd/volumes" Dec 03 12:08:31 crc kubenswrapper[5052]: I1203 12:08:31.923461 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-proxy-9bb7db5d-5q7wc" Dec 03 12:08:31 crc kubenswrapper[5052]: I1203 12:08:31.927216 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:08:31 crc kubenswrapper[5052]: I1203 12:08:31.975356 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:08:31 crc kubenswrapper[5052]: I1203 12:08:31.987808 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/028738cd-3b98-4853-9506-d2eba91afd87-log-httpd\") pod \"028738cd-3b98-4853-9506-d2eba91afd87\" (UID: \"028738cd-3b98-4853-9506-d2eba91afd87\") " Dec 03 12:08:31 crc kubenswrapper[5052]: I1203 12:08:31.987887 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/028738cd-3b98-4853-9506-d2eba91afd87-combined-ca-bundle\") pod \"028738cd-3b98-4853-9506-d2eba91afd87\" (UID: \"028738cd-3b98-4853-9506-d2eba91afd87\") " Dec 03 12:08:31 crc kubenswrapper[5052]: I1203 12:08:31.987930 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nsksg\" (UniqueName: \"kubernetes.io/projected/028738cd-3b98-4853-9506-d2eba91afd87-kube-api-access-nsksg\") pod \"028738cd-3b98-4853-9506-d2eba91afd87\" (UID: \"028738cd-3b98-4853-9506-d2eba91afd87\") " Dec 03 12:08:31 crc kubenswrapper[5052]: I1203 12:08:31.987975 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/4cae4fe5-bc0e-4088-b403-c78e940373c2-config-data\") pod \"4cae4fe5-bc0e-4088-b403-c78e940373c2\" (UID: \"4cae4fe5-bc0e-4088-b403-c78e940373c2\") " Dec 03 12:08:31 crc kubenswrapper[5052]: I1203 12:08:31.988088 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4cae4fe5-bc0e-4088-b403-c78e940373c2-combined-ca-bundle\") pod \"4cae4fe5-bc0e-4088-b403-c78e940373c2\" (UID: \"4cae4fe5-bc0e-4088-b403-c78e940373c2\") " Dec 03 12:08:31 crc kubenswrapper[5052]: I1203 12:08:31.988123 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/028738cd-3b98-4853-9506-d2eba91afd87-etc-swift\") pod \"028738cd-3b98-4853-9506-d2eba91afd87\" (UID: \"028738cd-3b98-4853-9506-d2eba91afd87\") " Dec 03 12:08:31 crc kubenswrapper[5052]: I1203 12:08:31.988153 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/028738cd-3b98-4853-9506-d2eba91afd87-internal-tls-certs\") pod \"028738cd-3b98-4853-9506-d2eba91afd87\" (UID: \"028738cd-3b98-4853-9506-d2eba91afd87\") " Dec 03 12:08:31 crc kubenswrapper[5052]: I1203 12:08:31.988193 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lbx98\" (UniqueName: \"kubernetes.io/projected/4cae4fe5-bc0e-4088-b403-c78e940373c2-kube-api-access-lbx98\") pod \"4cae4fe5-bc0e-4088-b403-c78e940373c2\" (UID: \"4cae4fe5-bc0e-4088-b403-c78e940373c2\") " Dec 03 12:08:31 crc kubenswrapper[5052]: I1203 12:08:31.988217 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/028738cd-3b98-4853-9506-d2eba91afd87-public-tls-certs\") pod \"028738cd-3b98-4853-9506-d2eba91afd87\" (UID: \"028738cd-3b98-4853-9506-d2eba91afd87\") " Dec 03 12:08:31 crc kubenswrapper[5052]: I1203 12:08:31.988244 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/028738cd-3b98-4853-9506-d2eba91afd87-config-data\") pod \"028738cd-3b98-4853-9506-d2eba91afd87\" (UID: \"028738cd-3b98-4853-9506-d2eba91afd87\") " Dec 03 12:08:31 crc kubenswrapper[5052]: I1203 12:08:31.988267 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/028738cd-3b98-4853-9506-d2eba91afd87-run-httpd\") pod \"028738cd-3b98-4853-9506-d2eba91afd87\" (UID: \"028738cd-3b98-4853-9506-d2eba91afd87\") " Dec 03 12:08:31 crc kubenswrapper[5052]: I1203 12:08:31.989946 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/028738cd-3b98-4853-9506-d2eba91afd87-run-httpd" (OuterVolumeSpecName: "run-httpd") pod "028738cd-3b98-4853-9506-d2eba91afd87" (UID: "028738cd-3b98-4853-9506-d2eba91afd87"). InnerVolumeSpecName "run-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:08:31 crc kubenswrapper[5052]: I1203 12:08:31.990220 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/028738cd-3b98-4853-9506-d2eba91afd87-log-httpd" (OuterVolumeSpecName: "log-httpd") pod "028738cd-3b98-4853-9506-d2eba91afd87" (UID: "028738cd-3b98-4853-9506-d2eba91afd87"). InnerVolumeSpecName "log-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:08:32 crc kubenswrapper[5052]: I1203 12:08:32.027621 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/028738cd-3b98-4853-9506-d2eba91afd87-kube-api-access-nsksg" (OuterVolumeSpecName: "kube-api-access-nsksg") pod "028738cd-3b98-4853-9506-d2eba91afd87" (UID: "028738cd-3b98-4853-9506-d2eba91afd87"). InnerVolumeSpecName "kube-api-access-nsksg". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:08:32 crc kubenswrapper[5052]: I1203 12:08:32.030743 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4cae4fe5-bc0e-4088-b403-c78e940373c2-kube-api-access-lbx98" (OuterVolumeSpecName: "kube-api-access-lbx98") pod "4cae4fe5-bc0e-4088-b403-c78e940373c2" (UID: "4cae4fe5-bc0e-4088-b403-c78e940373c2"). InnerVolumeSpecName "kube-api-access-lbx98". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:08:32 crc kubenswrapper[5052]: I1203 12:08:32.030843 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/028738cd-3b98-4853-9506-d2eba91afd87-etc-swift" (OuterVolumeSpecName: "etc-swift") pod "028738cd-3b98-4853-9506-d2eba91afd87" (UID: "028738cd-3b98-4853-9506-d2eba91afd87"). InnerVolumeSpecName "etc-swift". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:08:32 crc kubenswrapper[5052]: I1203 12:08:32.088969 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/rabbitmq-server-0" podUID="2397d896-0d11-4656-a336-eac8ec9fe7f9" containerName="rabbitmq" probeResult="failure" output="dial tcp 10.217.0.100:5671: connect: connection refused" Dec 03 12:08:32 crc kubenswrapper[5052]: I1203 12:08:32.091738 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e66cca73-f322-4c6e-adb0-5ec43607bb73-combined-ca-bundle\") pod \"e66cca73-f322-4c6e-adb0-5ec43607bb73\" (UID: \"e66cca73-f322-4c6e-adb0-5ec43607bb73\") " Dec 03 12:08:32 crc kubenswrapper[5052]: I1203 12:08:32.091821 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"vencrypt-tls-certs\" (UniqueName: \"kubernetes.io/secret/e66cca73-f322-4c6e-adb0-5ec43607bb73-vencrypt-tls-certs\") pod \"e66cca73-f322-4c6e-adb0-5ec43607bb73\" (UID: \"e66cca73-f322-4c6e-adb0-5ec43607bb73\") " Dec 03 12:08:32 crc kubenswrapper[5052]: I1203 12:08:32.091894 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5j7b2\" (UniqueName: \"kubernetes.io/projected/e66cca73-f322-4c6e-adb0-5ec43607bb73-kube-api-access-5j7b2\") pod \"e66cca73-f322-4c6e-adb0-5ec43607bb73\" (UID: \"e66cca73-f322-4c6e-adb0-5ec43607bb73\") " Dec 03 12:08:32 crc kubenswrapper[5052]: I1203 12:08:32.091925 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"nova-novncproxy-tls-certs\" (UniqueName: \"kubernetes.io/secret/e66cca73-f322-4c6e-adb0-5ec43607bb73-nova-novncproxy-tls-certs\") pod \"e66cca73-f322-4c6e-adb0-5ec43607bb73\" (UID: \"e66cca73-f322-4c6e-adb0-5ec43607bb73\") " Dec 03 12:08:32 crc kubenswrapper[5052]: I1203 12:08:32.092187 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/e66cca73-f322-4c6e-adb0-5ec43607bb73-config-data\") pod \"e66cca73-f322-4c6e-adb0-5ec43607bb73\" (UID: \"e66cca73-f322-4c6e-adb0-5ec43607bb73\") " Dec 03 12:08:32 crc kubenswrapper[5052]: I1203 12:08:32.098240 5052 reconciler_common.go:293] "Volume detached for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/028738cd-3b98-4853-9506-d2eba91afd87-log-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:32 crc kubenswrapper[5052]: I1203 12:08:32.098276 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-nsksg\" (UniqueName: \"kubernetes.io/projected/028738cd-3b98-4853-9506-d2eba91afd87-kube-api-access-nsksg\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:32 crc kubenswrapper[5052]: I1203 12:08:32.098290 5052 reconciler_common.go:293] "Volume detached for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/028738cd-3b98-4853-9506-d2eba91afd87-etc-swift\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:32 crc kubenswrapper[5052]: I1203 12:08:32.098303 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-lbx98\" (UniqueName: \"kubernetes.io/projected/4cae4fe5-bc0e-4088-b403-c78e940373c2-kube-api-access-lbx98\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:32 crc kubenswrapper[5052]: I1203 12:08:32.098313 5052 reconciler_common.go:293] "Volume detached for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/028738cd-3b98-4853-9506-d2eba91afd87-run-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:32 crc kubenswrapper[5052]: I1203 12:08:32.125330 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e66cca73-f322-4c6e-adb0-5ec43607bb73-kube-api-access-5j7b2" (OuterVolumeSpecName: "kube-api-access-5j7b2") pod "e66cca73-f322-4c6e-adb0-5ec43607bb73" (UID: "e66cca73-f322-4c6e-adb0-5ec43607bb73"). InnerVolumeSpecName "kube-api-access-5j7b2". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:08:32 crc kubenswrapper[5052]: I1203 12:08:32.137539 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/rabbitmq-server-2" podUID="7b4f6078-41a3-433d-acaf-f2d67391ab6e" containerName="rabbitmq" probeResult="failure" output="dial tcp 10.217.0.101:5671: connect: connection refused" Dec 03 12:08:32 crc kubenswrapper[5052]: I1203 12:08:32.158075 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/rabbitmq-server-1" podUID="7901ce85-4e6f-4e8b-86a6-f50a85803637" containerName="rabbitmq" probeResult="failure" output="dial tcp 10.217.0.102:5671: connect: connection refused" Dec 03 12:08:32 crc kubenswrapper[5052]: I1203 12:08:32.164928 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4cae4fe5-bc0e-4088-b403-c78e940373c2-config-data" (OuterVolumeSpecName: "config-data") pod "4cae4fe5-bc0e-4088-b403-c78e940373c2" (UID: "4cae4fe5-bc0e-4088-b403-c78e940373c2"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:08:32 crc kubenswrapper[5052]: I1203 12:08:32.188694 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/028738cd-3b98-4853-9506-d2eba91afd87-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "028738cd-3b98-4853-9506-d2eba91afd87" (UID: "028738cd-3b98-4853-9506-d2eba91afd87"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:08:32 crc kubenswrapper[5052]: I1203 12:08:32.192195 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4cae4fe5-bc0e-4088-b403-c78e940373c2-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "4cae4fe5-bc0e-4088-b403-c78e940373c2" (UID: "4cae4fe5-bc0e-4088-b403-c78e940373c2"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:08:32 crc kubenswrapper[5052]: I1203 12:08:32.207768 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/4cae4fe5-bc0e-4088-b403-c78e940373c2-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:32 crc kubenswrapper[5052]: I1203 12:08:32.207798 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4cae4fe5-bc0e-4088-b403-c78e940373c2-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:32 crc kubenswrapper[5052]: I1203 12:08:32.207808 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/028738cd-3b98-4853-9506-d2eba91afd87-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:32 crc kubenswrapper[5052]: I1203 12:08:32.207842 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-5j7b2\" (UniqueName: \"kubernetes.io/projected/e66cca73-f322-4c6e-adb0-5ec43607bb73-kube-api-access-5j7b2\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:32 crc kubenswrapper[5052]: I1203 12:08:32.211317 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e66cca73-f322-4c6e-adb0-5ec43607bb73-config-data" (OuterVolumeSpecName: "config-data") pod "e66cca73-f322-4c6e-adb0-5ec43607bb73" (UID: "e66cca73-f322-4c6e-adb0-5ec43607bb73"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:08:32 crc kubenswrapper[5052]: I1203 12:08:32.242666 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e66cca73-f322-4c6e-adb0-5ec43607bb73-nova-novncproxy-tls-certs" (OuterVolumeSpecName: "nova-novncproxy-tls-certs") pod "e66cca73-f322-4c6e-adb0-5ec43607bb73" (UID: "e66cca73-f322-4c6e-adb0-5ec43607bb73"). InnerVolumeSpecName "nova-novncproxy-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:08:32 crc kubenswrapper[5052]: I1203 12:08:32.249150 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/028738cd-3b98-4853-9506-d2eba91afd87-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "028738cd-3b98-4853-9506-d2eba91afd87" (UID: "028738cd-3b98-4853-9506-d2eba91afd87"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:08:32 crc kubenswrapper[5052]: I1203 12:08:32.277459 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/028738cd-3b98-4853-9506-d2eba91afd87-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "028738cd-3b98-4853-9506-d2eba91afd87" (UID: "028738cd-3b98-4853-9506-d2eba91afd87"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:08:32 crc kubenswrapper[5052]: I1203 12:08:32.285290 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e66cca73-f322-4c6e-adb0-5ec43607bb73-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "e66cca73-f322-4c6e-adb0-5ec43607bb73" (UID: "e66cca73-f322-4c6e-adb0-5ec43607bb73"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:08:32 crc kubenswrapper[5052]: I1203 12:08:32.314101 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/028738cd-3b98-4853-9506-d2eba91afd87-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:32 crc kubenswrapper[5052]: I1203 12:08:32.314192 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/e66cca73-f322-4c6e-adb0-5ec43607bb73-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:32 crc kubenswrapper[5052]: I1203 12:08:32.314207 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/028738cd-3b98-4853-9506-d2eba91afd87-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:32 crc kubenswrapper[5052]: I1203 12:08:32.314217 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e66cca73-f322-4c6e-adb0-5ec43607bb73-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:32 crc kubenswrapper[5052]: I1203 12:08:32.314231 5052 reconciler_common.go:293] "Volume detached for volume \"nova-novncproxy-tls-certs\" (UniqueName: \"kubernetes.io/secret/e66cca73-f322-4c6e-adb0-5ec43607bb73-nova-novncproxy-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:32 crc kubenswrapper[5052]: I1203 12:08:32.333763 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e66cca73-f322-4c6e-adb0-5ec43607bb73-vencrypt-tls-certs" (OuterVolumeSpecName: "vencrypt-tls-certs") pod "e66cca73-f322-4c6e-adb0-5ec43607bb73" (UID: "e66cca73-f322-4c6e-adb0-5ec43607bb73"). InnerVolumeSpecName "vencrypt-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:08:32 crc kubenswrapper[5052]: I1203 12:08:32.355235 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/028738cd-3b98-4853-9506-d2eba91afd87-config-data" (OuterVolumeSpecName: "config-data") pod "028738cd-3b98-4853-9506-d2eba91afd87" (UID: "028738cd-3b98-4853-9506-d2eba91afd87"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:08:32 crc kubenswrapper[5052]: I1203 12:08:32.420139 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/028738cd-3b98-4853-9506-d2eba91afd87-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:32 crc kubenswrapper[5052]: I1203 12:08:32.420192 5052 reconciler_common.go:293] "Volume detached for volume \"vencrypt-tls-certs\" (UniqueName: \"kubernetes.io/secret/e66cca73-f322-4c6e-adb0-5ec43607bb73-vencrypt-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:32 crc kubenswrapper[5052]: I1203 12:08:32.474558 5052 generic.go:334] "Generic (PLEG): container finished" podID="a3da2acb-8a0e-4ab3-8d15-2978eca73850" containerID="606f51e20d39e18fce0fe0fe7c840d6c46bd6e03a7e775bc7246b576bec5c132" exitCode=0 Dec 03 12:08:32 crc kubenswrapper[5052]: I1203 12:08:32.474625 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-worker-c656b9c77-2gf7g" event={"ID":"a3da2acb-8a0e-4ab3-8d15-2978eca73850","Type":"ContainerDied","Data":"606f51e20d39e18fce0fe0fe7c840d6c46bd6e03a7e775bc7246b576bec5c132"} Dec 03 12:08:32 crc kubenswrapper[5052]: I1203 12:08:32.476456 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" event={"ID":"e66cca73-f322-4c6e-adb0-5ec43607bb73","Type":"ContainerDied","Data":"1141e69944dcd7531261321f3d71467a1ef98b47ca821adbbbfbfb47d1ed7c9c"} Dec 03 12:08:32 crc kubenswrapper[5052]: I1203 12:08:32.476482 5052 scope.go:117] "RemoveContainer" containerID="81211fcc809459c82abca1a15005c67c33acfa54010915f26c5a7c1bed4bf50d" Dec 03 12:08:32 crc kubenswrapper[5052]: I1203 12:08:32.476804 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:08:32 crc kubenswrapper[5052]: I1203 12:08:32.484655 5052 generic.go:334] "Generic (PLEG): container finished" podID="45e26e8c-71b5-4b90-bba7-5bad2650d9b2" containerID="a96c58179a61355fd41c8699c34a2a6efcdd96a6901a2acedb4bb445fe0b84fe" exitCode=0 Dec 03 12:08:32 crc kubenswrapper[5052]: I1203 12:08:32.484769 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"45e26e8c-71b5-4b90-bba7-5bad2650d9b2","Type":"ContainerDied","Data":"a96c58179a61355fd41c8699c34a2a6efcdd96a6901a2acedb4bb445fe0b84fe"} Dec 03 12:08:32 crc kubenswrapper[5052]: I1203 12:08:32.499654 5052 generic.go:334] "Generic (PLEG): container finished" podID="0b100677-2da0-4717-a15a-d2463ee6713e" containerID="df9dfde29f7996c2208b2860c440e3ab6120cd657e154f046ad4cdb1b318a678" exitCode=0 Dec 03 12:08:32 crc kubenswrapper[5052]: I1203 12:08:32.499890 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"0b100677-2da0-4717-a15a-d2463ee6713e","Type":"ContainerDied","Data":"df9dfde29f7996c2208b2860c440e3ab6120cd657e154f046ad4cdb1b318a678"} Dec 03 12:08:32 crc kubenswrapper[5052]: I1203 12:08:32.518023 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-conductor-0" event={"ID":"4cae4fe5-bc0e-4088-b403-c78e940373c2","Type":"ContainerDied","Data":"f273f577e74fa9872c305a62585f94695ac0cbff46915d6a576d5f7689cecadd"} Dec 03 12:08:32 crc kubenswrapper[5052]: I1203 12:08:32.518102 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:08:32 crc kubenswrapper[5052]: I1203 12:08:32.525745 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-proxy-9bb7db5d-5q7wc" event={"ID":"028738cd-3b98-4853-9506-d2eba91afd87","Type":"ContainerDied","Data":"b9fac34047899cc094f874390d8356eb9d1ec157bffca85662e27dc6448b187a"} Dec 03 12:08:32 crc kubenswrapper[5052]: I1203 12:08:32.525841 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-proxy-9bb7db5d-5q7wc" Dec 03 12:08:32 crc kubenswrapper[5052]: I1203 12:08:32.531391 5052 generic.go:334] "Generic (PLEG): container finished" podID="31e6a111-bd4a-45a5-95a0-a6b537added6" containerID="545a199ec97461f4a64454b6edecdfc68e50742c7e11692685b952917d9c8b1e" exitCode=0 Dec 03 12:08:32 crc kubenswrapper[5052]: I1203 12:08:32.531568 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"31e6a111-bd4a-45a5-95a0-a6b537added6","Type":"ContainerDied","Data":"545a199ec97461f4a64454b6edecdfc68e50742c7e11692685b952917d9c8b1e"} Dec 03 12:08:32 crc kubenswrapper[5052]: I1203 12:08:32.535382 5052 generic.go:334] "Generic (PLEG): container finished" podID="de04776b-ea02-40d9-8459-0a519c8c64ad" containerID="ebd10ebaaccf168282fe14fc0fe11aa08dcb1e7da4e422f6b5dfdc5b8ddb9b23" exitCode=0 Dec 03 12:08:32 crc kubenswrapper[5052]: I1203 12:08:32.535462 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-d978c699b-kzzst" event={"ID":"de04776b-ea02-40d9-8459-0a519c8c64ad","Type":"ContainerDied","Data":"ebd10ebaaccf168282fe14fc0fe11aa08dcb1e7da4e422f6b5dfdc5b8ddb9b23"} Dec 03 12:08:32 crc kubenswrapper[5052]: I1203 12:08:32.537907 5052 generic.go:334] "Generic (PLEG): container finished" podID="f657b0c3-883e-4e3d-b161-1b3ef0988ab0" containerID="686b357f3c7f38691d18d6fa7206dbd5e60dc6a7b6acf91c922e01438257cfeb" exitCode=0 Dec 03 12:08:32 crc kubenswrapper[5052]: I1203 12:08:32.537951 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-kcb42" event={"ID":"f657b0c3-883e-4e3d-b161-1b3ef0988ab0","Type":"ContainerDied","Data":"686b357f3c7f38691d18d6fa7206dbd5e60dc6a7b6acf91c922e01438257cfeb"} Dec 03 12:08:32 crc kubenswrapper[5052]: I1203 12:08:32.537969 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-kcb42" event={"ID":"f657b0c3-883e-4e3d-b161-1b3ef0988ab0","Type":"ContainerStarted","Data":"5cb3fa6d03cfa9a06f36028d84a28f6c0d21f3556efec764d29b063d223df34a"} Dec 03 12:08:32 crc kubenswrapper[5052]: I1203 12:08:32.541358 5052 generic.go:334] "Generic (PLEG): container finished" podID="4a2a1881-e61a-4a99-a2f3-b614aeed81d1" containerID="95dbf47cff378e5b90c76418048911cfa106d1ea36f9592e66a0f3a368e19e25" exitCode=0 Dec 03 12:08:32 crc kubenswrapper[5052]: I1203 12:08:32.541421 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"4a2a1881-e61a-4a99-a2f3-b614aeed81d1","Type":"ContainerDied","Data":"95dbf47cff378e5b90c76418048911cfa106d1ea36f9592e66a0f3a368e19e25"} Dec 03 12:08:32 crc kubenswrapper[5052]: I1203 12:08:32.547078 5052 generic.go:334] "Generic (PLEG): container finished" podID="bef08f40-1b01-406f-af05-7b63e0b6e5e7" containerID="df8444be8224ec47d3e75542bba9c0aaa28cc5e0ba5b753982c10dfdcf96b872" exitCode=0 Dec 03 12:08:32 crc kubenswrapper[5052]: I1203 12:08:32.547193 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"bef08f40-1b01-406f-af05-7b63e0b6e5e7","Type":"ContainerDied","Data":"df8444be8224ec47d3e75542bba9c0aaa28cc5e0ba5b753982c10dfdcf96b872"} Dec 03 12:08:32 crc kubenswrapper[5052]: I1203 12:08:32.551492 5052 generic.go:334] "Generic (PLEG): container finished" podID="d877b426-724d-4ccf-a113-05a96ab94935" containerID="51b740be59469e140e437bc25b7208518a7bb1d9c11f719a8bcca63ec56644a8" exitCode=0 Dec 03 12:08:32 crc kubenswrapper[5052]: I1203 12:08:32.551790 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-keystone-listener-64674fc4f4-wfh52" event={"ID":"d877b426-724d-4ccf-a113-05a96ab94935","Type":"ContainerDied","Data":"51b740be59469e140e437bc25b7208518a7bb1d9c11f719a8bcca63ec56644a8"} Dec 03 12:08:32 crc kubenswrapper[5052]: I1203 12:08:32.602481 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:08:32 crc kubenswrapper[5052]: I1203 12:08:32.602776 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="19229e07-3c78-4369-998e-fc9e78eb647d" containerName="ceilometer-central-agent" containerID="cri-o://d1c0d412f0ba24368b4e09fb49e650d7f9751d7ab3deaf51a87873def7cef029" gracePeriod=30 Dec 03 12:08:32 crc kubenswrapper[5052]: I1203 12:08:32.602905 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="19229e07-3c78-4369-998e-fc9e78eb647d" containerName="proxy-httpd" containerID="cri-o://bc70dbc3ccd05a052e7fc3b5729f64a5f4c046c9a200d68d09f9e35887ba8604" gracePeriod=30 Dec 03 12:08:32 crc kubenswrapper[5052]: I1203 12:08:32.602944 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="19229e07-3c78-4369-998e-fc9e78eb647d" containerName="sg-core" containerID="cri-o://0e1e468e3a4838dddbe49481471a1e007e23016bc27634f15327a9a15546fcb0" gracePeriod=30 Dec 03 12:08:32 crc kubenswrapper[5052]: I1203 12:08:32.602974 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="19229e07-3c78-4369-998e-fc9e78eb647d" containerName="ceilometer-notification-agent" containerID="cri-o://85d1ba6ba0934dbfc1b081d24d64fbe85278667fa8c3b4ccf7dd7749c336f706" gracePeriod=30 Dec 03 12:08:32 crc kubenswrapper[5052]: I1203 12:08:32.619837 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/kube-state-metrics-0"] Dec 03 12:08:32 crc kubenswrapper[5052]: I1203 12:08:32.620108 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/kube-state-metrics-0" podUID="305f2518-9632-4358-84a1-632b4a7d8f6f" containerName="kube-state-metrics" containerID="cri-o://73022661e4a723ea9ac382d6839ad1127d1eaf64b3149c3d92b5bdf83208e996" gracePeriod=30 Dec 03 12:08:32 crc kubenswrapper[5052]: I1203 12:08:32.622219 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/openstack-cell1-galera-1" podUID="b34bb4d0-7906-42e8-a9aa-bd1a4203a312" containerName="galera" containerID="cri-o://c55564d5848c1a333ba2db20a9c5ecdbf152ba4ebaa847c6e28f6ed69c3a99ed" gracePeriod=25 Dec 03 12:08:32 crc kubenswrapper[5052]: I1203 12:08:32.685121 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:08:32 crc kubenswrapper[5052]: I1203 12:08:32.703770 5052 scope.go:117] "RemoveContainer" containerID="b19b072c0fb517fe3875dc65637b798d06ddeac368aceb24adb64b956c3e8597" Dec 03 12:08:32 crc kubenswrapper[5052]: I1203 12:08:32.720958 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-0"] Dec 03 12:08:32 crc kubenswrapper[5052]: I1203 12:08:32.725792 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:08:32 crc kubenswrapper[5052]: I1203 12:08:32.736547 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e92fa4d7-9bd6-4586-b41a-a557fb672439-combined-ca-bundle\") pod \"e92fa4d7-9bd6-4586-b41a-a557fb672439\" (UID: \"e92fa4d7-9bd6-4586-b41a-a557fb672439\") " Dec 03 12:08:32 crc kubenswrapper[5052]: I1203 12:08:32.737467 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dd67s\" (UniqueName: \"kubernetes.io/projected/e92fa4d7-9bd6-4586-b41a-a557fb672439-kube-api-access-dd67s\") pod \"e92fa4d7-9bd6-4586-b41a-a557fb672439\" (UID: \"e92fa4d7-9bd6-4586-b41a-a557fb672439\") " Dec 03 12:08:32 crc kubenswrapper[5052]: I1203 12:08:32.737659 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/e92fa4d7-9bd6-4586-b41a-a557fb672439-config-data\") pod \"e92fa4d7-9bd6-4586-b41a-a557fb672439\" (UID: \"e92fa4d7-9bd6-4586-b41a-a557fb672439\") " Dec 03 12:08:32 crc kubenswrapper[5052]: I1203 12:08:32.743492 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-0"] Dec 03 12:08:32 crc kubenswrapper[5052]: I1203 12:08:32.753317 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e92fa4d7-9bd6-4586-b41a-a557fb672439-kube-api-access-dd67s" (OuterVolumeSpecName: "kube-api-access-dd67s") pod "e92fa4d7-9bd6-4586-b41a-a557fb672439" (UID: "e92fa4d7-9bd6-4586-b41a-a557fb672439"). InnerVolumeSpecName "kube-api-access-dd67s". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:08:32 crc kubenswrapper[5052]: I1203 12:08:32.772711 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/swift-proxy-9bb7db5d-5q7wc"] Dec 03 12:08:32 crc kubenswrapper[5052]: I1203 12:08:32.796602 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/swift-proxy-9bb7db5d-5q7wc"] Dec 03 12:08:32 crc kubenswrapper[5052]: I1203 12:08:32.797052 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e92fa4d7-9bd6-4586-b41a-a557fb672439-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "e92fa4d7-9bd6-4586-b41a-a557fb672439" (UID: "e92fa4d7-9bd6-4586-b41a-a557fb672439"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:08:32 crc kubenswrapper[5052]: I1203 12:08:32.832864 5052 scope.go:117] "RemoveContainer" containerID="2d0b57b97cc7d912f25d72df376cda014d9ee39ad7a02f4b38c46e47d7ba19ac" Dec 03 12:08:32 crc kubenswrapper[5052]: I1203 12:08:32.833279 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/memcached-0"] Dec 03 12:08:32 crc kubenswrapper[5052]: I1203 12:08:32.833609 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/memcached-0" podUID="bbdcaf66-8813-4bed-9311-552fef62ac2d" containerName="memcached" containerID="cri-o://e6787f6e4b03711dd124764348c5ed1eaf82f566af649919f63f24cc48189367" gracePeriod=30 Dec 03 12:08:32 crc kubenswrapper[5052]: I1203 12:08:32.841683 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e92fa4d7-9bd6-4586-b41a-a557fb672439-config-data" (OuterVolumeSpecName: "config-data") pod "e92fa4d7-9bd6-4586-b41a-a557fb672439" (UID: "e92fa4d7-9bd6-4586-b41a-a557fb672439"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:08:32 crc kubenswrapper[5052]: I1203 12:08:32.842426 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/b9efd251-95c2-4224-b318-a34279ee1017-config-data\") pod \"b9efd251-95c2-4224-b318-a34279ee1017\" (UID: \"b9efd251-95c2-4224-b318-a34279ee1017\") " Dec 03 12:08:32 crc kubenswrapper[5052]: I1203 12:08:32.842476 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/b9efd251-95c2-4224-b318-a34279ee1017-logs\") pod \"b9efd251-95c2-4224-b318-a34279ee1017\" (UID: \"b9efd251-95c2-4224-b318-a34279ee1017\") " Dec 03 12:08:32 crc kubenswrapper[5052]: I1203 12:08:32.842543 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/b9efd251-95c2-4224-b318-a34279ee1017-nova-metadata-tls-certs\") pod \"b9efd251-95c2-4224-b318-a34279ee1017\" (UID: \"b9efd251-95c2-4224-b318-a34279ee1017\") " Dec 03 12:08:32 crc kubenswrapper[5052]: I1203 12:08:32.842576 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b5qq5\" (UniqueName: \"kubernetes.io/projected/b9efd251-95c2-4224-b318-a34279ee1017-kube-api-access-b5qq5\") pod \"b9efd251-95c2-4224-b318-a34279ee1017\" (UID: \"b9efd251-95c2-4224-b318-a34279ee1017\") " Dec 03 12:08:32 crc kubenswrapper[5052]: I1203 12:08:32.842869 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b9efd251-95c2-4224-b318-a34279ee1017-combined-ca-bundle\") pod \"b9efd251-95c2-4224-b318-a34279ee1017\" (UID: \"b9efd251-95c2-4224-b318-a34279ee1017\") " Dec 03 12:08:32 crc kubenswrapper[5052]: I1203 12:08:32.846961 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b9efd251-95c2-4224-b318-a34279ee1017-logs" (OuterVolumeSpecName: "logs") pod "b9efd251-95c2-4224-b318-a34279ee1017" (UID: "b9efd251-95c2-4224-b318-a34279ee1017"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:08:32 crc kubenswrapper[5052]: I1203 12:08:32.862864 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/b9efd251-95c2-4224-b318-a34279ee1017-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:32 crc kubenswrapper[5052]: I1203 12:08:32.862917 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e92fa4d7-9bd6-4586-b41a-a557fb672439-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:32 crc kubenswrapper[5052]: I1203 12:08:32.862934 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-dd67s\" (UniqueName: \"kubernetes.io/projected/e92fa4d7-9bd6-4586-b41a-a557fb672439-kube-api-access-dd67s\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:32 crc kubenswrapper[5052]: I1203 12:08:32.862946 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/e92fa4d7-9bd6-4586-b41a-a557fb672439-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:32 crc kubenswrapper[5052]: I1203 12:08:32.872200 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 12:08:32 crc kubenswrapper[5052]: I1203 12:08:32.885691 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b9efd251-95c2-4224-b318-a34279ee1017-kube-api-access-b5qq5" (OuterVolumeSpecName: "kube-api-access-b5qq5") pod "b9efd251-95c2-4224-b318-a34279ee1017" (UID: "b9efd251-95c2-4224-b318-a34279ee1017"). InnerVolumeSpecName "kube-api-access-b5qq5". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:08:32 crc kubenswrapper[5052]: I1203 12:08:32.935672 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 12:08:32 crc kubenswrapper[5052]: I1203 12:08:32.935963 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b9efd251-95c2-4224-b318-a34279ee1017-config-data" (OuterVolumeSpecName: "config-data") pod "b9efd251-95c2-4224-b318-a34279ee1017" (UID: "b9efd251-95c2-4224-b318-a34279ee1017"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:08:32 crc kubenswrapper[5052]: I1203 12:08:32.968769 5052 scope.go:117] "RemoveContainer" containerID="531801a1d9fdbb4eab4dc17c841d530240327b26c1a21395bf0c594278f80f5c" Dec 03 12:08:32 crc kubenswrapper[5052]: I1203 12:08:32.972419 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/b9efd251-95c2-4224-b318-a34279ee1017-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:32 crc kubenswrapper[5052]: I1203 12:08:32.972450 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-b5qq5\" (UniqueName: \"kubernetes.io/projected/b9efd251-95c2-4224-b318-a34279ee1017-kube-api-access-b5qq5\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:32 crc kubenswrapper[5052]: I1203 12:08:32.986540 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/keystone-bootstrap-8s4bm"] Dec 03 12:08:32 crc kubenswrapper[5052]: I1203 12:08:32.997078 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b9efd251-95c2-4224-b318-a34279ee1017-nova-metadata-tls-certs" (OuterVolumeSpecName: "nova-metadata-tls-certs") pod "b9efd251-95c2-4224-b318-a34279ee1017" (UID: "b9efd251-95c2-4224-b318-a34279ee1017"). InnerVolumeSpecName "nova-metadata-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.024290 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/keystone-bootstrap-8s4bm"] Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.054780 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-f4dfbd7d-mxsm8" Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.063188 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b9efd251-95c2-4224-b318-a34279ee1017-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "b9efd251-95c2-4224-b318-a34279ee1017" (UID: "b9efd251-95c2-4224-b318-a34279ee1017"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.079946 5052 reconciler_common.go:293] "Volume detached for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/b9efd251-95c2-4224-b318-a34279ee1017-nova-metadata-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.079974 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b9efd251-95c2-4224-b318-a34279ee1017-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.080040 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/keystone-db-sync-8xzwc"] Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.089578 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/keystone-6c6748d95d-66jbh"] Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.089827 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/keystone-6c6748d95d-66jbh" podUID="c2470da9-4c96-44ac-834d-2b560785135f" containerName="keystone-api" containerID="cri-o://aa4334f0d2af3013c5ace40839543a0d0b6116ced72af25a3daba9191d3d0382" gracePeriod=30 Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.103807 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/keystone-db-sync-8xzwc"] Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.110497 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/keystone25ff-account-delete-2rdct"] Dec 03 12:08:33 crc kubenswrapper[5052]: E1203 12:08:33.110960 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="6999e86b-8f95-4002-9959-6288a7d783bd" containerName="placement-api" Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.110973 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="6999e86b-8f95-4002-9959-6288a7d783bd" containerName="placement-api" Dec 03 12:08:33 crc kubenswrapper[5052]: E1203 12:08:33.110992 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="327a4d19-96ba-4b5e-a19f-56ba15a0cc82" containerName="ovsdbserver-nb" Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.110997 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="327a4d19-96ba-4b5e-a19f-56ba15a0cc82" containerName="ovsdbserver-nb" Dec 03 12:08:33 crc kubenswrapper[5052]: E1203 12:08:33.111008 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="b9efd251-95c2-4224-b318-a34279ee1017" containerName="nova-metadata-log" Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.111016 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="b9efd251-95c2-4224-b318-a34279ee1017" containerName="nova-metadata-log" Dec 03 12:08:33 crc kubenswrapper[5052]: E1203 12:08:33.111027 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="6999e86b-8f95-4002-9959-6288a7d783bd" containerName="placement-log" Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.111033 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="6999e86b-8f95-4002-9959-6288a7d783bd" containerName="placement-log" Dec 03 12:08:33 crc kubenswrapper[5052]: E1203 12:08:33.111042 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="b9efd251-95c2-4224-b318-a34279ee1017" containerName="nova-metadata-metadata" Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.111049 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="b9efd251-95c2-4224-b318-a34279ee1017" containerName="nova-metadata-metadata" Dec 03 12:08:33 crc kubenswrapper[5052]: E1203 12:08:33.111062 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="028738cd-3b98-4853-9506-d2eba91afd87" containerName="proxy-server" Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.111069 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="028738cd-3b98-4853-9506-d2eba91afd87" containerName="proxy-server" Dec 03 12:08:33 crc kubenswrapper[5052]: E1203 12:08:33.111090 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="78dad0d3-18e5-4900-932c-56ede3279dc0" containerName="openstack-network-exporter" Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.111096 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="78dad0d3-18e5-4900-932c-56ede3279dc0" containerName="openstack-network-exporter" Dec 03 12:08:33 crc kubenswrapper[5052]: E1203 12:08:33.111294 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="e92fa4d7-9bd6-4586-b41a-a557fb672439" containerName="nova-cell1-conductor-conductor" Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.111305 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="e92fa4d7-9bd6-4586-b41a-a557fb672439" containerName="nova-cell1-conductor-conductor" Dec 03 12:08:33 crc kubenswrapper[5052]: E1203 12:08:33.111317 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="028738cd-3b98-4853-9506-d2eba91afd87" containerName="proxy-httpd" Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.111324 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="028738cd-3b98-4853-9506-d2eba91afd87" containerName="proxy-httpd" Dec 03 12:08:33 crc kubenswrapper[5052]: E1203 12:08:33.111334 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="4cae4fe5-bc0e-4088-b403-c78e940373c2" containerName="nova-cell0-conductor-conductor" Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.111340 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="4cae4fe5-bc0e-4088-b403-c78e940373c2" containerName="nova-cell0-conductor-conductor" Dec 03 12:08:33 crc kubenswrapper[5052]: E1203 12:08:33.111365 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="327a4d19-96ba-4b5e-a19f-56ba15a0cc82" containerName="openstack-network-exporter" Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.111371 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="327a4d19-96ba-4b5e-a19f-56ba15a0cc82" containerName="openstack-network-exporter" Dec 03 12:08:33 crc kubenswrapper[5052]: E1203 12:08:33.111383 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="e66cca73-f322-4c6e-adb0-5ec43607bb73" containerName="nova-cell1-novncproxy-novncproxy" Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.111389 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="e66cca73-f322-4c6e-adb0-5ec43607bb73" containerName="nova-cell1-novncproxy-novncproxy" Dec 03 12:08:33 crc kubenswrapper[5052]: E1203 12:08:33.111398 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="78dad0d3-18e5-4900-932c-56ede3279dc0" containerName="ovsdbserver-sb" Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.111404 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="78dad0d3-18e5-4900-932c-56ede3279dc0" containerName="ovsdbserver-sb" Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.111590 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="6999e86b-8f95-4002-9959-6288a7d783bd" containerName="placement-api" Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.111615 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="6999e86b-8f95-4002-9959-6288a7d783bd" containerName="placement-log" Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.111624 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="e92fa4d7-9bd6-4586-b41a-a557fb672439" containerName="nova-cell1-conductor-conductor" Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.111634 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="78dad0d3-18e5-4900-932c-56ede3279dc0" containerName="openstack-network-exporter" Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.111646 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="327a4d19-96ba-4b5e-a19f-56ba15a0cc82" containerName="openstack-network-exporter" Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.111656 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="b9efd251-95c2-4224-b318-a34279ee1017" containerName="nova-metadata-log" Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.111664 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="4cae4fe5-bc0e-4088-b403-c78e940373c2" containerName="nova-cell0-conductor-conductor" Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.111675 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="e66cca73-f322-4c6e-adb0-5ec43607bb73" containerName="nova-cell1-novncproxy-novncproxy" Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.111686 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="78dad0d3-18e5-4900-932c-56ede3279dc0" containerName="ovsdbserver-sb" Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.111702 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="028738cd-3b98-4853-9506-d2eba91afd87" containerName="proxy-server" Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.111711 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="028738cd-3b98-4853-9506-d2eba91afd87" containerName="proxy-httpd" Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.111721 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="b9efd251-95c2-4224-b318-a34279ee1017" containerName="nova-metadata-metadata" Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.111727 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="327a4d19-96ba-4b5e-a19f-56ba15a0cc82" containerName="ovsdbserver-nb" Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.112813 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone25ff-account-delete-2rdct" Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.113565 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/rabbitmq-cell1-server-1" podUID="f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3" containerName="rabbitmq" probeResult="failure" output="dial tcp 10.217.0.103:5671: connect: connection refused" Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.173926 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone25ff-account-delete-2rdct"] Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.181341 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/6999e86b-8f95-4002-9959-6288a7d783bd-scripts\") pod \"6999e86b-8f95-4002-9959-6288a7d783bd\" (UID: \"6999e86b-8f95-4002-9959-6288a7d783bd\") " Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.181526 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6999e86b-8f95-4002-9959-6288a7d783bd-config-data\") pod \"6999e86b-8f95-4002-9959-6288a7d783bd\" (UID: \"6999e86b-8f95-4002-9959-6288a7d783bd\") " Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.181567 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/6999e86b-8f95-4002-9959-6288a7d783bd-logs\") pod \"6999e86b-8f95-4002-9959-6288a7d783bd\" (UID: \"6999e86b-8f95-4002-9959-6288a7d783bd\") " Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.181680 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4zd27\" (UniqueName: \"kubernetes.io/projected/6999e86b-8f95-4002-9959-6288a7d783bd-kube-api-access-4zd27\") pod \"6999e86b-8f95-4002-9959-6288a7d783bd\" (UID: \"6999e86b-8f95-4002-9959-6288a7d783bd\") " Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.181718 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/6999e86b-8f95-4002-9959-6288a7d783bd-public-tls-certs\") pod \"6999e86b-8f95-4002-9959-6288a7d783bd\" (UID: \"6999e86b-8f95-4002-9959-6288a7d783bd\") " Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.181805 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/6999e86b-8f95-4002-9959-6288a7d783bd-internal-tls-certs\") pod \"6999e86b-8f95-4002-9959-6288a7d783bd\" (UID: \"6999e86b-8f95-4002-9959-6288a7d783bd\") " Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.181847 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6999e86b-8f95-4002-9959-6288a7d783bd-combined-ca-bundle\") pod \"6999e86b-8f95-4002-9959-6288a7d783bd\" (UID: \"6999e86b-8f95-4002-9959-6288a7d783bd\") " Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.182306 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d2d77\" (UniqueName: \"kubernetes.io/projected/19d83b78-c03a-482a-909e-74bb991d974a-kube-api-access-d2d77\") pod \"keystone25ff-account-delete-2rdct\" (UID: \"19d83b78-c03a-482a-909e-74bb991d974a\") " pod="openstack-kuttl-tests/keystone25ff-account-delete-2rdct" Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.182422 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/19d83b78-c03a-482a-909e-74bb991d974a-operator-scripts\") pod \"keystone25ff-account-delete-2rdct\" (UID: \"19d83b78-c03a-482a-909e-74bb991d974a\") " pod="openstack-kuttl-tests/keystone25ff-account-delete-2rdct" Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.185955 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6999e86b-8f95-4002-9959-6288a7d783bd-logs" (OuterVolumeSpecName: "logs") pod "6999e86b-8f95-4002-9959-6288a7d783bd" (UID: "6999e86b-8f95-4002-9959-6288a7d783bd"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.192631 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/openstack-galera-0"] Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.194362 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6999e86b-8f95-4002-9959-6288a7d783bd-scripts" (OuterVolumeSpecName: "scripts") pod "6999e86b-8f95-4002-9959-6288a7d783bd" (UID: "6999e86b-8f95-4002-9959-6288a7d783bd"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.212038 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6999e86b-8f95-4002-9959-6288a7d783bd-kube-api-access-4zd27" (OuterVolumeSpecName: "kube-api-access-4zd27") pod "6999e86b-8f95-4002-9959-6288a7d783bd" (UID: "6999e86b-8f95-4002-9959-6288a7d783bd"). InnerVolumeSpecName "kube-api-access-4zd27". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.231123 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/openstack-galera-1"] Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.252905 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/openstack-galera-2"] Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.274607 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/keystone-db-create-zgx94"] Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.284616 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-d2d77\" (UniqueName: \"kubernetes.io/projected/19d83b78-c03a-482a-909e-74bb991d974a-kube-api-access-d2d77\") pod \"keystone25ff-account-delete-2rdct\" (UID: \"19d83b78-c03a-482a-909e-74bb991d974a\") " pod="openstack-kuttl-tests/keystone25ff-account-delete-2rdct" Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.284758 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/19d83b78-c03a-482a-909e-74bb991d974a-operator-scripts\") pod \"keystone25ff-account-delete-2rdct\" (UID: \"19d83b78-c03a-482a-909e-74bb991d974a\") " pod="openstack-kuttl-tests/keystone25ff-account-delete-2rdct" Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.284962 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/6999e86b-8f95-4002-9959-6288a7d783bd-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.284985 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-4zd27\" (UniqueName: \"kubernetes.io/projected/6999e86b-8f95-4002-9959-6288a7d783bd-kube-api-access-4zd27\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.284997 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/6999e86b-8f95-4002-9959-6288a7d783bd-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:33 crc kubenswrapper[5052]: E1203 12:08:33.285069 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:08:33 crc kubenswrapper[5052]: E1203 12:08:33.285131 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/19d83b78-c03a-482a-909e-74bb991d974a-operator-scripts podName:19d83b78-c03a-482a-909e-74bb991d974a nodeName:}" failed. No retries permitted until 2025-12-03 12:08:33.785109817 +0000 UTC m=+2904.373351581 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/19d83b78-c03a-482a-909e-74bb991d974a-operator-scripts") pod "keystone25ff-account-delete-2rdct" (UID: "19d83b78-c03a-482a-909e-74bb991d974a") : configmap "openstack-scripts" not found Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.290663 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/keystone-db-create-zgx94"] Dec 03 12:08:33 crc kubenswrapper[5052]: E1203 12:08:33.292365 5052 projected.go:194] Error preparing data for projected volume kube-api-access-d2d77 for pod openstack-kuttl-tests/keystone25ff-account-delete-2rdct: failed to fetch token: serviceaccounts "galera-openstack" not found Dec 03 12:08:33 crc kubenswrapper[5052]: E1203 12:08:33.292438 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/19d83b78-c03a-482a-909e-74bb991d974a-kube-api-access-d2d77 podName:19d83b78-c03a-482a-909e-74bb991d974a nodeName:}" failed. No retries permitted until 2025-12-03 12:08:33.792418696 +0000 UTC m=+2904.380660460 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-d2d77" (UniqueName: "kubernetes.io/projected/19d83b78-c03a-482a-909e-74bb991d974a-kube-api-access-d2d77") pod "keystone25ff-account-delete-2rdct" (UID: "19d83b78-c03a-482a-909e-74bb991d974a") : failed to fetch token: serviceaccounts "galera-openstack" not found Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.343603 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/keystone25ff-account-delete-2rdct"] Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.382573 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/keystone-25ff-account-create-update-hcnv4"] Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.383264 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6999e86b-8f95-4002-9959-6288a7d783bd-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "6999e86b-8f95-4002-9959-6288a7d783bd" (UID: "6999e86b-8f95-4002-9959-6288a7d783bd"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.387407 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6999e86b-8f95-4002-9959-6288a7d783bd-config-data" (OuterVolumeSpecName: "config-data") pod "6999e86b-8f95-4002-9959-6288a7d783bd" (UID: "6999e86b-8f95-4002-9959-6288a7d783bd"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:08:33 crc kubenswrapper[5052]: E1203 12:08:33.388991 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/rabbitmq-config-data: configmap "rabbitmq-config-data" not found Dec 03 12:08:33 crc kubenswrapper[5052]: E1203 12:08:33.389062 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2397d896-0d11-4656-a336-eac8ec9fe7f9-config-data podName:2397d896-0d11-4656-a336-eac8ec9fe7f9 nodeName:}" failed. No retries permitted until 2025-12-03 12:08:41.389043788 +0000 UTC m=+2911.977285552 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "config-data" (UniqueName: "kubernetes.io/configmap/2397d896-0d11-4656-a336-eac8ec9fe7f9-config-data") pod "rabbitmq-server-0" (UID: "2397d896-0d11-4656-a336-eac8ec9fe7f9") : configmap "rabbitmq-config-data" not found Dec 03 12:08:33 crc kubenswrapper[5052]: E1203 12:08:33.389439 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/rabbitmq-config-data: configmap "rabbitmq-config-data" not found Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.389523 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6999e86b-8f95-4002-9959-6288a7d783bd-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.389543 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6999e86b-8f95-4002-9959-6288a7d783bd-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:33 crc kubenswrapper[5052]: E1203 12:08:33.389692 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7901ce85-4e6f-4e8b-86a6-f50a85803637-config-data podName:7901ce85-4e6f-4e8b-86a6-f50a85803637 nodeName:}" failed. No retries permitted until 2025-12-03 12:08:41.389673486 +0000 UTC m=+2911.977915250 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "config-data" (UniqueName: "kubernetes.io/configmap/7901ce85-4e6f-4e8b-86a6-f50a85803637-config-data") pod "rabbitmq-server-1" (UID: "7901ce85-4e6f-4e8b-86a6-f50a85803637") : configmap "rabbitmq-config-data" not found Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.395822 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/keystone-25ff-account-create-update-hcnv4"] Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.424157 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/rabbitmq-cell1-server-2" podUID="06bfd2db-abe0-4b4d-a110-94a6a1e02211" containerName="rabbitmq" probeResult="failure" output="dial tcp 10.217.0.105:5671: connect: connection refused" Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.462030 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6999e86b-8f95-4002-9959-6288a7d783bd-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "6999e86b-8f95-4002-9959-6288a7d783bd" (UID: "6999e86b-8f95-4002-9959-6288a7d783bd"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.493938 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/6999e86b-8f95-4002-9959-6288a7d783bd-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:33 crc kubenswrapper[5052]: E1203 12:08:33.494023 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/dns-swift-storage-0: configmap "dns-swift-storage-0" not found Dec 03 12:08:33 crc kubenswrapper[5052]: E1203 12:08:33.494071 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/77077fe5-d1ff-49c4-b049-3a0f2cee23b0-dns-swift-storage-0 podName:77077fe5-d1ff-49c4-b049-3a0f2cee23b0 nodeName:}" failed. No retries permitted until 2025-12-03 12:08:37.49405814 +0000 UTC m=+2908.082299904 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "dns-swift-storage-0" (UniqueName: "kubernetes.io/configmap/77077fe5-d1ff-49c4-b049-3a0f2cee23b0-dns-swift-storage-0") pod "dnsmasq-dnsmasq-85f8bc6fb5-b86k2" (UID: "77077fe5-d1ff-49c4-b049-3a0f2cee23b0") : configmap "dns-swift-storage-0" not found Dec 03 12:08:33 crc kubenswrapper[5052]: E1203 12:08:33.494236 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/rabbitmq-config-data: configmap "rabbitmq-config-data" not found Dec 03 12:08:33 crc kubenswrapper[5052]: E1203 12:08:33.494320 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7b4f6078-41a3-433d-acaf-f2d67391ab6e-config-data podName:7b4f6078-41a3-433d-acaf-f2d67391ab6e nodeName:}" failed. No retries permitted until 2025-12-03 12:08:41.494300787 +0000 UTC m=+2912.082542551 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "config-data" (UniqueName: "kubernetes.io/configmap/7b4f6078-41a3-433d-acaf-f2d67391ab6e-config-data") pod "rabbitmq-server-2" (UID: "7b4f6078-41a3-433d-acaf-f2d67391ab6e") : configmap "rabbitmq-config-data" not found Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.503285 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6999e86b-8f95-4002-9959-6288a7d783bd-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "6999e86b-8f95-4002-9959-6288a7d783bd" (UID: "6999e86b-8f95-4002-9959-6288a7d783bd"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:08:33 crc kubenswrapper[5052]: E1203 12:08:33.547117 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-d2d77 operator-scripts], unattached volumes=[], failed to process volumes=[]: context canceled" pod="openstack-kuttl-tests/keystone25ff-account-delete-2rdct" podUID="19d83b78-c03a-482a-909e-74bb991d974a" Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.563087 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.592915 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-api-d978c699b-kzzst" Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.595567 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/6999e86b-8f95-4002-9959-6288a7d783bd-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.602976 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-f4dfbd7d-mxsm8" event={"ID":"6999e86b-8f95-4002-9959-6288a7d783bd","Type":"ContainerDied","Data":"2d364625552a7cb7d231a8334b918f370dbb7586042dad1e5579d3dc8073a66f"} Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.603022 5052 scope.go:117] "RemoveContainer" containerID="3e8fef46e8b29b153dd3541055f4f411078ff175402ded3c5e2a2532584522cb" Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.603127 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-f4dfbd7d-mxsm8" Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.623769 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-worker-c656b9c77-2gf7g" Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.641732 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.641863 5052 generic.go:334] "Generic (PLEG): container finished" podID="19229e07-3c78-4369-998e-fc9e78eb647d" containerID="bc70dbc3ccd05a052e7fc3b5729f64a5f4c046c9a200d68d09f9e35887ba8604" exitCode=0 Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.641878 5052 generic.go:334] "Generic (PLEG): container finished" podID="19229e07-3c78-4369-998e-fc9e78eb647d" containerID="0e1e468e3a4838dddbe49481471a1e007e23016bc27634f15327a9a15546fcb0" exitCode=2 Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.641909 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"19229e07-3c78-4369-998e-fc9e78eb647d","Type":"ContainerDied","Data":"bc70dbc3ccd05a052e7fc3b5729f64a5f4c046c9a200d68d09f9e35887ba8604"} Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.641932 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"19229e07-3c78-4369-998e-fc9e78eb647d","Type":"ContainerDied","Data":"0e1e468e3a4838dddbe49481471a1e007e23016bc27634f15327a9a15546fcb0"} Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.656168 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.656641 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"4a2a1881-e61a-4a99-a2f3-b614aeed81d1","Type":"ContainerDied","Data":"495f5d9f1bab82087b2a568fa419ea815529bd9df5fa5b0398f027035e5b6b4a"} Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.659920 5052 scope.go:117] "RemoveContainer" containerID="18f9fa6fd7bb783b320dadc515c1e38c7a2e0212224d2e211b526e64320731a1" Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.673789 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-api-d978c699b-kzzst" Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.678850 5052 generic.go:334] "Generic (PLEG): container finished" podID="305f2518-9632-4358-84a1-632b4a7d8f6f" containerID="73022661e4a723ea9ac382d6839ad1127d1eaf64b3149c3d92b5bdf83208e996" exitCode=2 Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.681198 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.697910 5052 scope.go:117] "RemoveContainer" containerID="95dbf47cff378e5b90c76418048911cfa106d1ea36f9592e66a0f3a368e19e25" Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.701424 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/de04776b-ea02-40d9-8459-0a519c8c64ad-config-data-custom\") pod \"de04776b-ea02-40d9-8459-0a519c8c64ad\" (UID: \"de04776b-ea02-40d9-8459-0a519c8c64ad\") " Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.701460 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"glance\" (UniqueName: \"kubernetes.io/local-volume/local-storage13-crc\") pod \"4a2a1881-e61a-4a99-a2f3-b614aeed81d1\" (UID: \"4a2a1881-e61a-4a99-a2f3-b614aeed81d1\") " Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.701492 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/bef08f40-1b01-406f-af05-7b63e0b6e5e7-scripts\") pod \"bef08f40-1b01-406f-af05-7b63e0b6e5e7\" (UID: \"bef08f40-1b01-406f-af05-7b63e0b6e5e7\") " Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.701533 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/de04776b-ea02-40d9-8459-0a519c8c64ad-combined-ca-bundle\") pod \"de04776b-ea02-40d9-8459-0a519c8c64ad\" (UID: \"de04776b-ea02-40d9-8459-0a519c8c64ad\") " Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.701569 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4a2a1881-e61a-4a99-a2f3-b614aeed81d1-combined-ca-bundle\") pod \"4a2a1881-e61a-4a99-a2f3-b614aeed81d1\" (UID: \"4a2a1881-e61a-4a99-a2f3-b614aeed81d1\") " Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.702489 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/bef08f40-1b01-406f-af05-7b63e0b6e5e7-logs\") pod \"bef08f40-1b01-406f-af05-7b63e0b6e5e7\" (UID: \"bef08f40-1b01-406f-af05-7b63e0b6e5e7\") " Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.702533 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/bef08f40-1b01-406f-af05-7b63e0b6e5e7-combined-ca-bundle\") pod \"bef08f40-1b01-406f-af05-7b63e0b6e5e7\" (UID: \"bef08f40-1b01-406f-af05-7b63e0b6e5e7\") " Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.702562 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/de04776b-ea02-40d9-8459-0a519c8c64ad-logs\") pod \"de04776b-ea02-40d9-8459-0a519c8c64ad\" (UID: \"de04776b-ea02-40d9-8459-0a519c8c64ad\") " Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.702587 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/4a2a1881-e61a-4a99-a2f3-b614aeed81d1-logs\") pod \"4a2a1881-e61a-4a99-a2f3-b614aeed81d1\" (UID: \"4a2a1881-e61a-4a99-a2f3-b614aeed81d1\") " Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.702610 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/a3da2acb-8a0e-4ab3-8d15-2978eca73850-logs\") pod \"a3da2acb-8a0e-4ab3-8d15-2978eca73850\" (UID: \"a3da2acb-8a0e-4ab3-8d15-2978eca73850\") " Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.702626 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/4a2a1881-e61a-4a99-a2f3-b614aeed81d1-internal-tls-certs\") pod \"4a2a1881-e61a-4a99-a2f3-b614aeed81d1\" (UID: \"4a2a1881-e61a-4a99-a2f3-b614aeed81d1\") " Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.702648 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jqslb\" (UniqueName: \"kubernetes.io/projected/a3da2acb-8a0e-4ab3-8d15-2978eca73850-kube-api-access-jqslb\") pod \"a3da2acb-8a0e-4ab3-8d15-2978eca73850\" (UID: \"a3da2acb-8a0e-4ab3-8d15-2978eca73850\") " Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.702680 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/4a2a1881-e61a-4a99-a2f3-b614aeed81d1-scripts\") pod \"4a2a1881-e61a-4a99-a2f3-b614aeed81d1\" (UID: \"4a2a1881-e61a-4a99-a2f3-b614aeed81d1\") " Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.702704 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"glance\" (UniqueName: \"kubernetes.io/local-volume/local-storage11-crc\") pod \"bef08f40-1b01-406f-af05-7b63e0b6e5e7\" (UID: \"bef08f40-1b01-406f-af05-7b63e0b6e5e7\") " Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.702730 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/a3da2acb-8a0e-4ab3-8d15-2978eca73850-config-data-custom\") pod \"a3da2acb-8a0e-4ab3-8d15-2978eca73850\" (UID: \"a3da2acb-8a0e-4ab3-8d15-2978eca73850\") " Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.702764 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-82xxb\" (UniqueName: \"kubernetes.io/projected/de04776b-ea02-40d9-8459-0a519c8c64ad-kube-api-access-82xxb\") pod \"de04776b-ea02-40d9-8459-0a519c8c64ad\" (UID: \"de04776b-ea02-40d9-8459-0a519c8c64ad\") " Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.702785 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/de04776b-ea02-40d9-8459-0a519c8c64ad-config-data\") pod \"de04776b-ea02-40d9-8459-0a519c8c64ad\" (UID: \"de04776b-ea02-40d9-8459-0a519c8c64ad\") " Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.702803 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/4a2a1881-e61a-4a99-a2f3-b614aeed81d1-config-data\") pod \"4a2a1881-e61a-4a99-a2f3-b614aeed81d1\" (UID: \"4a2a1881-e61a-4a99-a2f3-b614aeed81d1\") " Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.702823 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/bef08f40-1b01-406f-af05-7b63e0b6e5e7-config-data\") pod \"bef08f40-1b01-406f-af05-7b63e0b6e5e7\" (UID: \"bef08f40-1b01-406f-af05-7b63e0b6e5e7\") " Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.702841 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/bef08f40-1b01-406f-af05-7b63e0b6e5e7-public-tls-certs\") pod \"bef08f40-1b01-406f-af05-7b63e0b6e5e7\" (UID: \"bef08f40-1b01-406f-af05-7b63e0b6e5e7\") " Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.702859 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/de04776b-ea02-40d9-8459-0a519c8c64ad-public-tls-certs\") pod \"de04776b-ea02-40d9-8459-0a519c8c64ad\" (UID: \"de04776b-ea02-40d9-8459-0a519c8c64ad\") " Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.702898 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a3da2acb-8a0e-4ab3-8d15-2978eca73850-config-data\") pod \"a3da2acb-8a0e-4ab3-8d15-2978eca73850\" (UID: \"a3da2acb-8a0e-4ab3-8d15-2978eca73850\") " Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.702915 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a3da2acb-8a0e-4ab3-8d15-2978eca73850-combined-ca-bundle\") pod \"a3da2acb-8a0e-4ab3-8d15-2978eca73850\" (UID: \"a3da2acb-8a0e-4ab3-8d15-2978eca73850\") " Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.702930 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/4a2a1881-e61a-4a99-a2f3-b614aeed81d1-httpd-run\") pod \"4a2a1881-e61a-4a99-a2f3-b614aeed81d1\" (UID: \"4a2a1881-e61a-4a99-a2f3-b614aeed81d1\") " Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.702988 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/bef08f40-1b01-406f-af05-7b63e0b6e5e7-httpd-run\") pod \"bef08f40-1b01-406f-af05-7b63e0b6e5e7\" (UID: \"bef08f40-1b01-406f-af05-7b63e0b6e5e7\") " Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.703002 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ds85z\" (UniqueName: \"kubernetes.io/projected/4a2a1881-e61a-4a99-a2f3-b614aeed81d1-kube-api-access-ds85z\") pod \"4a2a1881-e61a-4a99-a2f3-b614aeed81d1\" (UID: \"4a2a1881-e61a-4a99-a2f3-b614aeed81d1\") " Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.703028 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/de04776b-ea02-40d9-8459-0a519c8c64ad-internal-tls-certs\") pod \"de04776b-ea02-40d9-8459-0a519c8c64ad\" (UID: \"de04776b-ea02-40d9-8459-0a519c8c64ad\") " Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.703066 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tc5vs\" (UniqueName: \"kubernetes.io/projected/bef08f40-1b01-406f-af05-7b63e0b6e5e7-kube-api-access-tc5vs\") pod \"bef08f40-1b01-406f-af05-7b63e0b6e5e7\" (UID: \"bef08f40-1b01-406f-af05-7b63e0b6e5e7\") " Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.703285 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de04776b-ea02-40d9-8459-0a519c8c64ad-logs" (OuterVolumeSpecName: "logs") pod "de04776b-ea02-40d9-8459-0a519c8c64ad" (UID: "de04776b-ea02-40d9-8459-0a519c8c64ad"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.706086 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bef08f40-1b01-406f-af05-7b63e0b6e5e7-logs" (OuterVolumeSpecName: "logs") pod "bef08f40-1b01-406f-af05-7b63e0b6e5e7" (UID: "bef08f40-1b01-406f-af05-7b63e0b6e5e7"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:08:33 crc kubenswrapper[5052]: E1203 12:08:33.710699 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/rabbitmq-cell1-config-data: configmap "rabbitmq-cell1-config-data" not found Dec 03 12:08:33 crc kubenswrapper[5052]: E1203 12:08:33.710800 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/185fdd2a-cd57-45e5-9986-bebd15e5761f-config-data podName:185fdd2a-cd57-45e5-9986-bebd15e5761f nodeName:}" failed. No retries permitted until 2025-12-03 12:08:41.710776326 +0000 UTC m=+2912.299018160 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "config-data" (UniqueName: "kubernetes.io/configmap/185fdd2a-cd57-45e5-9986-bebd15e5761f-config-data") pod "rabbitmq-cell1-server-0" (UID: "185fdd2a-cd57-45e5-9986-bebd15e5761f") : configmap "rabbitmq-cell1-config-data" not found Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.712060 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a2a1881-e61a-4a99-a2f3-b614aeed81d1-logs" (OuterVolumeSpecName: "logs") pod "4a2a1881-e61a-4a99-a2f3-b614aeed81d1" (UID: "4a2a1881-e61a-4a99-a2f3-b614aeed81d1"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.712667 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3da2acb-8a0e-4ab3-8d15-2978eca73850-logs" (OuterVolumeSpecName: "logs") pod "a3da2acb-8a0e-4ab3-8d15-2978eca73850" (UID: "a3da2acb-8a0e-4ab3-8d15-2978eca73850"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.715287 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage13-crc" (OuterVolumeSpecName: "glance") pod "4a2a1881-e61a-4a99-a2f3-b614aeed81d1" (UID: "4a2a1881-e61a-4a99-a2f3-b614aeed81d1"). InnerVolumeSpecName "local-storage13-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.724615 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="028738cd-3b98-4853-9506-d2eba91afd87" path="/var/lib/kubelet/pods/028738cd-3b98-4853-9506-d2eba91afd87/volumes" Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.724672 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bef08f40-1b01-406f-af05-7b63e0b6e5e7-httpd-run" (OuterVolumeSpecName: "httpd-run") pod "bef08f40-1b01-406f-af05-7b63e0b6e5e7" (UID: "bef08f40-1b01-406f-af05-7b63e0b6e5e7"). InnerVolumeSpecName "httpd-run". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.725019 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/de04776b-ea02-40d9-8459-0a519c8c64ad-config-data-custom" (OuterVolumeSpecName: "config-data-custom") pod "de04776b-ea02-40d9-8459-0a519c8c64ad" (UID: "de04776b-ea02-40d9-8459-0a519c8c64ad"). InnerVolumeSpecName "config-data-custom". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.725498 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1e7cf8cd-09e8-4bfe-b8d3-9e509d4eae43" path="/var/lib/kubelet/pods/1e7cf8cd-09e8-4bfe-b8d3-9e509d4eae43/volumes" Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.726587 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="46eb04b8-4d02-4ffe-a0f6-bb4b236a4956" path="/var/lib/kubelet/pods/46eb04b8-4d02-4ffe-a0f6-bb4b236a4956/volumes" Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.728924 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4cae4fe5-bc0e-4088-b403-c78e940373c2" path="/var/lib/kubelet/pods/4cae4fe5-bc0e-4088-b403-c78e940373c2/volumes" Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.729439 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6588734e-3543-47aa-9df6-f6e21a877732" path="/var/lib/kubelet/pods/6588734e-3543-47aa-9df6-f6e21a877732/volumes" Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.732207 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="66b581e0-3a48-46c9-b1a7-384b87d2b7f4" path="/var/lib/kubelet/pods/66b581e0-3a48-46c9-b1a7-384b87d2b7f4/volumes" Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.732907 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a3da2acb-8a0e-4ab3-8d15-2978eca73850-config-data-custom" (OuterVolumeSpecName: "config-data-custom") pod "a3da2acb-8a0e-4ab3-8d15-2978eca73850" (UID: "a3da2acb-8a0e-4ab3-8d15-2978eca73850"). InnerVolumeSpecName "config-data-custom". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.733028 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.734463 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone25ff-account-delete-2rdct" Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.735174 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bef08f40-1b01-406f-af05-7b63e0b6e5e7-scripts" (OuterVolumeSpecName: "scripts") pod "bef08f40-1b01-406f-af05-7b63e0b6e5e7" (UID: "bef08f40-1b01-406f-af05-7b63e0b6e5e7"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.735803 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a2a1881-e61a-4a99-a2f3-b614aeed81d1-httpd-run" (OuterVolumeSpecName: "httpd-run") pod "4a2a1881-e61a-4a99-a2f3-b614aeed81d1" (UID: "4a2a1881-e61a-4a99-a2f3-b614aeed81d1"). InnerVolumeSpecName "httpd-run". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.740287 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a3da2acb-8a0e-4ab3-8d15-2978eca73850-kube-api-access-jqslb" (OuterVolumeSpecName: "kube-api-access-jqslb") pod "a3da2acb-8a0e-4ab3-8d15-2978eca73850" (UID: "a3da2acb-8a0e-4ab3-8d15-2978eca73850"). InnerVolumeSpecName "kube-api-access-jqslb". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.740358 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4a2a1881-e61a-4a99-a2f3-b614aeed81d1-scripts" (OuterVolumeSpecName: "scripts") pod "4a2a1881-e61a-4a99-a2f3-b614aeed81d1" (UID: "4a2a1881-e61a-4a99-a2f3-b614aeed81d1"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.744752 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4a2a1881-e61a-4a99-a2f3-b614aeed81d1-kube-api-access-ds85z" (OuterVolumeSpecName: "kube-api-access-ds85z") pod "4a2a1881-e61a-4a99-a2f3-b614aeed81d1" (UID: "4a2a1881-e61a-4a99-a2f3-b614aeed81d1"). InnerVolumeSpecName "kube-api-access-ds85z". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.747022 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bef08f40-1b01-406f-af05-7b63e0b6e5e7-kube-api-access-tc5vs" (OuterVolumeSpecName: "kube-api-access-tc5vs") pod "bef08f40-1b01-406f-af05-7b63e0b6e5e7" (UID: "bef08f40-1b01-406f-af05-7b63e0b6e5e7"). InnerVolumeSpecName "kube-api-access-tc5vs". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.748664 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/openstack-galera-2" podUID="143f4f1d-4f91-4ffa-a277-d6e551d19838" containerName="galera" containerID="cri-o://a154e1968b2288216f60d8cd5f0368f47e36bcdef4f248bc7568b9b7174e4331" gracePeriod=30 Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.749040 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e66cca73-f322-4c6e-adb0-5ec43607bb73" path="/var/lib/kubelet/pods/e66cca73-f322-4c6e-adb0-5ec43607bb73/volumes" Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.774853 5052 scope.go:117] "RemoveContainer" containerID="e3e2ee8f299225cb3b6df3bf35cd7781dcb1aa91cfcdbcdc6d9ceee107ca48ce" Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.779657 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone25ff-account-delete-2rdct" Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.786837 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-d978c699b-kzzst" event={"ID":"de04776b-ea02-40d9-8459-0a519c8c64ad","Type":"ContainerDied","Data":"f15409ad9070b89cee38b736c2856f93ede4172edddf0ae15f61386e4de6869c"} Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.786894 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-kcb42" Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.786916 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/placement-f4dfbd7d-mxsm8"] Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.786944 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/placement-f4dfbd7d-mxsm8"] Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.786959 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/kube-state-metrics-0" event={"ID":"305f2518-9632-4358-84a1-632b4a7d8f6f","Type":"ContainerDied","Data":"73022661e4a723ea9ac382d6839ad1127d1eaf64b3149c3d92b5bdf83208e996"} Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.786972 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-0"] Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.786985 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-conductor-0" event={"ID":"e92fa4d7-9bd6-4586-b41a-a557fb672439","Type":"ContainerDied","Data":"606c5f9fc4d72d003134f9303e7eb72bdb86f38ae1c336683de7950b82e20c36"} Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.786998 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"b9efd251-95c2-4224-b318-a34279ee1017","Type":"ContainerDied","Data":"b42c2e5aaf3a181255a02334958b7382f6e8aaa9faf937e59f371e0e658d9523"} Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.798737 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/de04776b-ea02-40d9-8459-0a519c8c64ad-kube-api-access-82xxb" (OuterVolumeSpecName: "kube-api-access-82xxb") pod "de04776b-ea02-40d9-8459-0a519c8c64ad" (UID: "de04776b-ea02-40d9-8459-0a519c8c64ad"). InnerVolumeSpecName "kube-api-access-82xxb". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.800121 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage11-crc" (OuterVolumeSpecName: "glance") pod "bef08f40-1b01-406f-af05-7b63e0b6e5e7" (UID: "bef08f40-1b01-406f-af05-7b63e0b6e5e7"). InnerVolumeSpecName "local-storage11-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.818722 5052 scope.go:117] "RemoveContainer" containerID="ebd10ebaaccf168282fe14fc0fe11aa08dcb1e7da4e422f6b5dfdc5b8ddb9b23" Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.822841 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-0"] Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.823462 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-d2d77\" (UniqueName: \"kubernetes.io/projected/19d83b78-c03a-482a-909e-74bb991d974a-kube-api-access-d2d77\") pod \"keystone25ff-account-delete-2rdct\" (UID: \"19d83b78-c03a-482a-909e-74bb991d974a\") " pod="openstack-kuttl-tests/keystone25ff-account-delete-2rdct" Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.823747 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/19d83b78-c03a-482a-909e-74bb991d974a-operator-scripts\") pod \"keystone25ff-account-delete-2rdct\" (UID: \"19d83b78-c03a-482a-909e-74bb991d974a\") " pod="openstack-kuttl-tests/keystone25ff-account-delete-2rdct" Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.824020 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/a3da2acb-8a0e-4ab3-8d15-2978eca73850-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.824042 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-jqslb\" (UniqueName: \"kubernetes.io/projected/a3da2acb-8a0e-4ab3-8d15-2978eca73850-kube-api-access-jqslb\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.824056 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/4a2a1881-e61a-4a99-a2f3-b614aeed81d1-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.824079 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage11-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage11-crc\") on node \"crc\" " Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.824090 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/a3da2acb-8a0e-4ab3-8d15-2978eca73850-config-data-custom\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.824102 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-82xxb\" (UniqueName: \"kubernetes.io/projected/de04776b-ea02-40d9-8459-0a519c8c64ad-kube-api-access-82xxb\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.824111 5052 reconciler_common.go:293] "Volume detached for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/4a2a1881-e61a-4a99-a2f3-b614aeed81d1-httpd-run\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.824124 5052 reconciler_common.go:293] "Volume detached for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/bef08f40-1b01-406f-af05-7b63e0b6e5e7-httpd-run\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.824133 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-ds85z\" (UniqueName: \"kubernetes.io/projected/4a2a1881-e61a-4a99-a2f3-b614aeed81d1-kube-api-access-ds85z\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.824144 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-tc5vs\" (UniqueName: \"kubernetes.io/projected/bef08f40-1b01-406f-af05-7b63e0b6e5e7-kube-api-access-tc5vs\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.824153 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/de04776b-ea02-40d9-8459-0a519c8c64ad-config-data-custom\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.824172 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage13-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage13-crc\") on node \"crc\" " Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.824182 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/bef08f40-1b01-406f-af05-7b63e0b6e5e7-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.824195 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/bef08f40-1b01-406f-af05-7b63e0b6e5e7-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.824204 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/de04776b-ea02-40d9-8459-0a519c8c64ad-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.824213 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/4a2a1881-e61a-4a99-a2f3-b614aeed81d1-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:33 crc kubenswrapper[5052]: E1203 12:08:33.825092 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/rabbitmq-cell1-config-data: configmap "rabbitmq-cell1-config-data" not found Dec 03 12:08:33 crc kubenswrapper[5052]: E1203 12:08:33.825146 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/06bfd2db-abe0-4b4d-a110-94a6a1e02211-config-data podName:06bfd2db-abe0-4b4d-a110-94a6a1e02211 nodeName:}" failed. No retries permitted until 2025-12-03 12:08:41.825127466 +0000 UTC m=+2912.413369220 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "config-data" (UniqueName: "kubernetes.io/configmap/06bfd2db-abe0-4b4d-a110-94a6a1e02211-config-data") pod "rabbitmq-cell1-server-2" (UID: "06bfd2db-abe0-4b4d-a110-94a6a1e02211") : configmap "rabbitmq-cell1-config-data" not found Dec 03 12:08:33 crc kubenswrapper[5052]: E1203 12:08:33.825784 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:08:33 crc kubenswrapper[5052]: E1203 12:08:33.825897 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/19d83b78-c03a-482a-909e-74bb991d974a-operator-scripts podName:19d83b78-c03a-482a-909e-74bb991d974a nodeName:}" failed. No retries permitted until 2025-12-03 12:08:34.825872257 +0000 UTC m=+2905.414114021 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/19d83b78-c03a-482a-909e-74bb991d974a-operator-scripts") pod "keystone25ff-account-delete-2rdct" (UID: "19d83b78-c03a-482a-909e-74bb991d974a") : configmap "openstack-scripts" not found Dec 03 12:08:33 crc kubenswrapper[5052]: E1203 12:08:33.830041 5052 projected.go:194] Error preparing data for projected volume kube-api-access-d2d77 for pod openstack-kuttl-tests/keystone25ff-account-delete-2rdct: failed to fetch token: serviceaccounts "galera-openstack" not found Dec 03 12:08:33 crc kubenswrapper[5052]: E1203 12:08:33.830090 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/19d83b78-c03a-482a-909e-74bb991d974a-kube-api-access-d2d77 podName:19d83b78-c03a-482a-909e-74bb991d974a nodeName:}" failed. No retries permitted until 2025-12-03 12:08:34.830072337 +0000 UTC m=+2905.418314091 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-d2d77" (UniqueName: "kubernetes.io/projected/19d83b78-c03a-482a-909e-74bb991d974a-kube-api-access-d2d77") pod "keystone25ff-account-delete-2rdct" (UID: "19d83b78-c03a-482a-909e-74bb991d974a") : failed to fetch token: serviceaccounts "galera-openstack" not found Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.840258 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4a2a1881-e61a-4a99-a2f3-b614aeed81d1-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "4a2a1881-e61a-4a99-a2f3-b614aeed81d1" (UID: "4a2a1881-e61a-4a99-a2f3-b614aeed81d1"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.840744 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-kcb42" podStartSLOduration=6.840727952 podStartE2EDuration="6.840727952s" podCreationTimestamp="2025-12-03 12:08:27 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:08:33.793307036 +0000 UTC m=+2904.381548800" watchObservedRunningTime="2025-12-03 12:08:33.840727952 +0000 UTC m=+2904.428969716" Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.873484 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.874783 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bef08f40-1b01-406f-af05-7b63e0b6e5e7-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "bef08f40-1b01-406f-af05-7b63e0b6e5e7" (UID: "bef08f40-1b01-406f-af05-7b63e0b6e5e7"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.879533 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage11-crc" (UniqueName: "kubernetes.io/local-volume/local-storage11-crc") on node "crc" Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.887091 5052 scope.go:117] "RemoveContainer" containerID="649a53bf9bcdd8f3f358ecd746d8c1146c6de92e3ce56e6a224366e0dffc9ed0" Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.897217 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.908921 5052 scope.go:117] "RemoveContainer" containerID="8709692766b7010f69ca646129fa46d2b1115bc2a7f05706db51e69fa086eafa" Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.924670 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/de04776b-ea02-40d9-8459-0a519c8c64ad-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "de04776b-ea02-40d9-8459-0a519c8c64ad" (UID: "de04776b-ea02-40d9-8459-0a519c8c64ad"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.925986 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage11-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage11-crc\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.926009 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/de04776b-ea02-40d9-8459-0a519c8c64ad-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.926020 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4a2a1881-e61a-4a99-a2f3-b614aeed81d1-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.926029 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/bef08f40-1b01-406f-af05-7b63e0b6e5e7-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.956715 5052 scope.go:117] "RemoveContainer" containerID="0d0239c7224731fd55404636504e2c22a3b640502408dba4f002511377ec7f33" Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.966801 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a3da2acb-8a0e-4ab3-8d15-2978eca73850-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "a3da2acb-8a0e-4ab3-8d15-2978eca73850" (UID: "a3da2acb-8a0e-4ab3-8d15-2978eca73850"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:08:33 crc kubenswrapper[5052]: I1203 12:08:33.992016 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage13-crc" (UniqueName: "kubernetes.io/local-volume/local-storage13-crc") on node "crc" Dec 03 12:08:34 crc kubenswrapper[5052]: I1203 12:08:34.016419 5052 scope.go:117] "RemoveContainer" containerID="179d0d417d11399d428ed2dce00676182f65bdc218984235dfbdc87fb4e0c5c0" Dec 03 12:08:34 crc kubenswrapper[5052]: I1203 12:08:34.018908 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/de04776b-ea02-40d9-8459-0a519c8c64ad-config-data" (OuterVolumeSpecName: "config-data") pod "de04776b-ea02-40d9-8459-0a519c8c64ad" (UID: "de04776b-ea02-40d9-8459-0a519c8c64ad"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:08:34 crc kubenswrapper[5052]: I1203 12:08:34.031378 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/de04776b-ea02-40d9-8459-0a519c8c64ad-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "de04776b-ea02-40d9-8459-0a519c8c64ad" (UID: "de04776b-ea02-40d9-8459-0a519c8c64ad"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:08:34 crc kubenswrapper[5052]: I1203 12:08:34.031646 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a3da2acb-8a0e-4ab3-8d15-2978eca73850-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:34 crc kubenswrapper[5052]: I1203 12:08:34.031679 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage13-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage13-crc\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:34 crc kubenswrapper[5052]: I1203 12:08:34.031691 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/de04776b-ea02-40d9-8459-0a519c8c64ad-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:34 crc kubenswrapper[5052]: I1203 12:08:34.031699 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/de04776b-ea02-40d9-8459-0a519c8c64ad-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:34 crc kubenswrapper[5052]: I1203 12:08:34.032629 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bef08f40-1b01-406f-af05-7b63e0b6e5e7-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "bef08f40-1b01-406f-af05-7b63e0b6e5e7" (UID: "bef08f40-1b01-406f-af05-7b63e0b6e5e7"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:08:34 crc kubenswrapper[5052]: I1203 12:08:34.060238 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:08:34 crc kubenswrapper[5052]: I1203 12:08:34.060892 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a3da2acb-8a0e-4ab3-8d15-2978eca73850-config-data" (OuterVolumeSpecName: "config-data") pod "a3da2acb-8a0e-4ab3-8d15-2978eca73850" (UID: "a3da2acb-8a0e-4ab3-8d15-2978eca73850"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:34.066570 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bef08f40-1b01-406f-af05-7b63e0b6e5e7-config-data" (OuterVolumeSpecName: "config-data") pod "bef08f40-1b01-406f-af05-7b63e0b6e5e7" (UID: "bef08f40-1b01-406f-af05-7b63e0b6e5e7"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:34.072400 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:34.125643 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-keystone-listener-64674fc4f4-wfh52" Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:34.135783 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ct7fq\" (UniqueName: \"kubernetes.io/projected/0b100677-2da0-4717-a15a-d2463ee6713e-kube-api-access-ct7fq\") pod \"0b100677-2da0-4717-a15a-d2463ee6713e\" (UID: \"0b100677-2da0-4717-a15a-d2463ee6713e\") " Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:34.136028 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0b100677-2da0-4717-a15a-d2463ee6713e-combined-ca-bundle\") pod \"0b100677-2da0-4717-a15a-d2463ee6713e\" (UID: \"0b100677-2da0-4717-a15a-d2463ee6713e\") " Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:34.136102 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/0b100677-2da0-4717-a15a-d2463ee6713e-config-data\") pod \"0b100677-2da0-4717-a15a-d2463ee6713e\" (UID: \"0b100677-2da0-4717-a15a-d2463ee6713e\") " Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:34.137046 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/bef08f40-1b01-406f-af05-7b63e0b6e5e7-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:34.137070 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/bef08f40-1b01-406f-af05-7b63e0b6e5e7-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:34.137083 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a3da2acb-8a0e-4ab3-8d15-2978eca73850-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:36 crc kubenswrapper[5052]: E1203 12:08:34.137157 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/rabbitmq-cell1-config-data: configmap "rabbitmq-cell1-config-data" not found Dec 03 12:08:36 crc kubenswrapper[5052]: E1203 12:08:34.137217 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3-config-data podName:f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3 nodeName:}" failed. No retries permitted until 2025-12-03 12:08:42.137197208 +0000 UTC m=+2912.725438972 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "config-data" (UniqueName: "kubernetes.io/configmap/f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3-config-data") pod "rabbitmq-cell1-server-1" (UID: "f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3") : configmap "rabbitmq-cell1-config-data" not found Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:34.165829 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0b100677-2da0-4717-a15a-d2463ee6713e-kube-api-access-ct7fq" (OuterVolumeSpecName: "kube-api-access-ct7fq") pod "0b100677-2da0-4717-a15a-d2463ee6713e" (UID: "0b100677-2da0-4717-a15a-d2463ee6713e"). InnerVolumeSpecName "kube-api-access-ct7fq". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:34.167244 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4a2a1881-e61a-4a99-a2f3-b614aeed81d1-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "4a2a1881-e61a-4a99-a2f3-b614aeed81d1" (UID: "4a2a1881-e61a-4a99-a2f3-b614aeed81d1"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:34.210876 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/de04776b-ea02-40d9-8459-0a519c8c64ad-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "de04776b-ea02-40d9-8459-0a519c8c64ad" (UID: "de04776b-ea02-40d9-8459-0a519c8c64ad"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:34.214950 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0b100677-2da0-4717-a15a-d2463ee6713e-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "0b100677-2da0-4717-a15a-d2463ee6713e" (UID: "0b100677-2da0-4717-a15a-d2463ee6713e"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:34.224432 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4a2a1881-e61a-4a99-a2f3-b614aeed81d1-config-data" (OuterVolumeSpecName: "config-data") pod "4a2a1881-e61a-4a99-a2f3-b614aeed81d1" (UID: "4a2a1881-e61a-4a99-a2f3-b614aeed81d1"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:34.237762 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/45e26e8c-71b5-4b90-bba7-5bad2650d9b2-logs\") pod \"45e26e8c-71b5-4b90-bba7-5bad2650d9b2\" (UID: \"45e26e8c-71b5-4b90-bba7-5bad2650d9b2\") " Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:34.237837 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/45e26e8c-71b5-4b90-bba7-5bad2650d9b2-config-data\") pod \"45e26e8c-71b5-4b90-bba7-5bad2650d9b2\" (UID: \"45e26e8c-71b5-4b90-bba7-5bad2650d9b2\") " Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:34.237886 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p9lmx\" (UniqueName: \"kubernetes.io/projected/d877b426-724d-4ccf-a113-05a96ab94935-kube-api-access-p9lmx\") pod \"d877b426-724d-4ccf-a113-05a96ab94935\" (UID: \"d877b426-724d-4ccf-a113-05a96ab94935\") " Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:34.237931 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/d877b426-724d-4ccf-a113-05a96ab94935-logs\") pod \"d877b426-724d-4ccf-a113-05a96ab94935\" (UID: \"d877b426-724d-4ccf-a113-05a96ab94935\") " Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:34.237956 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/45e26e8c-71b5-4b90-bba7-5bad2650d9b2-internal-tls-certs\") pod \"45e26e8c-71b5-4b90-bba7-5bad2650d9b2\" (UID: \"45e26e8c-71b5-4b90-bba7-5bad2650d9b2\") " Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:34.238007 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qhlxn\" (UniqueName: \"kubernetes.io/projected/45e26e8c-71b5-4b90-bba7-5bad2650d9b2-kube-api-access-qhlxn\") pod \"45e26e8c-71b5-4b90-bba7-5bad2650d9b2\" (UID: \"45e26e8c-71b5-4b90-bba7-5bad2650d9b2\") " Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:34.238029 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/d877b426-724d-4ccf-a113-05a96ab94935-config-data-custom\") pod \"d877b426-724d-4ccf-a113-05a96ab94935\" (UID: \"d877b426-724d-4ccf-a113-05a96ab94935\") " Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:34.238100 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d877b426-724d-4ccf-a113-05a96ab94935-combined-ca-bundle\") pod \"d877b426-724d-4ccf-a113-05a96ab94935\" (UID: \"d877b426-724d-4ccf-a113-05a96ab94935\") " Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:34.238126 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/45e26e8c-71b5-4b90-bba7-5bad2650d9b2-combined-ca-bundle\") pod \"45e26e8c-71b5-4b90-bba7-5bad2650d9b2\" (UID: \"45e26e8c-71b5-4b90-bba7-5bad2650d9b2\") " Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:34.238146 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/d877b426-724d-4ccf-a113-05a96ab94935-config-data\") pod \"d877b426-724d-4ccf-a113-05a96ab94935\" (UID: \"d877b426-724d-4ccf-a113-05a96ab94935\") " Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:34.238549 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/45e26e8c-71b5-4b90-bba7-5bad2650d9b2-public-tls-certs\") pod \"45e26e8c-71b5-4b90-bba7-5bad2650d9b2\" (UID: \"45e26e8c-71b5-4b90-bba7-5bad2650d9b2\") " Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:34.239020 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/4a2a1881-e61a-4a99-a2f3-b614aeed81d1-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:34.239033 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-ct7fq\" (UniqueName: \"kubernetes.io/projected/0b100677-2da0-4717-a15a-d2463ee6713e-kube-api-access-ct7fq\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:34.239044 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0b100677-2da0-4717-a15a-d2463ee6713e-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:34.239053 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/de04776b-ea02-40d9-8459-0a519c8c64ad-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:34.239062 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/4a2a1881-e61a-4a99-a2f3-b614aeed81d1-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:34.238314 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/45e26e8c-71b5-4b90-bba7-5bad2650d9b2-logs" (OuterVolumeSpecName: "logs") pod "45e26e8c-71b5-4b90-bba7-5bad2650d9b2" (UID: "45e26e8c-71b5-4b90-bba7-5bad2650d9b2"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:34.245985 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d877b426-724d-4ccf-a113-05a96ab94935-logs" (OuterVolumeSpecName: "logs") pod "d877b426-724d-4ccf-a113-05a96ab94935" (UID: "d877b426-724d-4ccf-a113-05a96ab94935"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:34.251933 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d877b426-724d-4ccf-a113-05a96ab94935-config-data-custom" (OuterVolumeSpecName: "config-data-custom") pod "d877b426-724d-4ccf-a113-05a96ab94935" (UID: "d877b426-724d-4ccf-a113-05a96ab94935"). InnerVolumeSpecName "config-data-custom". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:34.253117 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/45e26e8c-71b5-4b90-bba7-5bad2650d9b2-kube-api-access-qhlxn" (OuterVolumeSpecName: "kube-api-access-qhlxn") pod "45e26e8c-71b5-4b90-bba7-5bad2650d9b2" (UID: "45e26e8c-71b5-4b90-bba7-5bad2650d9b2"). InnerVolumeSpecName "kube-api-access-qhlxn". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:08:36 crc kubenswrapper[5052]: E1203 12:08:34.264988 5052 cadvisor_stats_provider.go:516] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod19229e07_3c78_4369_998e_fc9e78eb647d.slice/crio-conmon-d1c0d412f0ba24368b4e09fb49e650d7f9751d7ab3deaf51a87873def7cef029.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbbdcaf66_8813_4bed_9311_552fef62ac2d.slice/crio-conmon-e6787f6e4b03711dd124764348c5ed1eaf82f566af649919f63f24cc48189367.scope\": RecentStats: unable to find data in memory cache], [\"/system.slice/system-systemd\\\\x2dcoredump.slice/systemd-coredump@2-203472-0.service\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfad6f453_acf5_4f13_94d6_7d46e300f28c.slice/crio-f82e335d0814000e829c2020ade84de1617dea07a919392e9736c05912d0b4bf.scope\": RecentStats: unable to find data in memory cache]" Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:34.276623 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d877b426-724d-4ccf-a113-05a96ab94935-kube-api-access-p9lmx" (OuterVolumeSpecName: "kube-api-access-p9lmx") pod "d877b426-724d-4ccf-a113-05a96ab94935" (UID: "d877b426-724d-4ccf-a113-05a96ab94935"). InnerVolumeSpecName "kube-api-access-p9lmx". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:34.285069 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0b100677-2da0-4717-a15a-d2463ee6713e-config-data" (OuterVolumeSpecName: "config-data") pod "0b100677-2da0-4717-a15a-d2463ee6713e" (UID: "0b100677-2da0-4717-a15a-d2463ee6713e"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:34.309185 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/45e26e8c-71b5-4b90-bba7-5bad2650d9b2-config-data" (OuterVolumeSpecName: "config-data") pod "45e26e8c-71b5-4b90-bba7-5bad2650d9b2" (UID: "45e26e8c-71b5-4b90-bba7-5bad2650d9b2"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:34.321994 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/45e26e8c-71b5-4b90-bba7-5bad2650d9b2-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "45e26e8c-71b5-4b90-bba7-5bad2650d9b2" (UID: "45e26e8c-71b5-4b90-bba7-5bad2650d9b2"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:34.329738 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/45e26e8c-71b5-4b90-bba7-5bad2650d9b2-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "45e26e8c-71b5-4b90-bba7-5bad2650d9b2" (UID: "45e26e8c-71b5-4b90-bba7-5bad2650d9b2"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:34.349422 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/45e26e8c-71b5-4b90-bba7-5bad2650d9b2-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:34.349706 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/0b100677-2da0-4717-a15a-d2463ee6713e-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:34.349726 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-p9lmx\" (UniqueName: \"kubernetes.io/projected/d877b426-724d-4ccf-a113-05a96ab94935-kube-api-access-p9lmx\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:34.349742 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/d877b426-724d-4ccf-a113-05a96ab94935-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:34.349944 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-qhlxn\" (UniqueName: \"kubernetes.io/projected/45e26e8c-71b5-4b90-bba7-5bad2650d9b2-kube-api-access-qhlxn\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:34.349964 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/d877b426-724d-4ccf-a113-05a96ab94935-config-data-custom\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:34.349977 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/45e26e8c-71b5-4b90-bba7-5bad2650d9b2-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:34.349990 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/45e26e8c-71b5-4b90-bba7-5bad2650d9b2-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:34.350196 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/45e26e8c-71b5-4b90-bba7-5bad2650d9b2-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:34.350630 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d877b426-724d-4ccf-a113-05a96ab94935-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "d877b426-724d-4ccf-a113-05a96ab94935" (UID: "d877b426-724d-4ccf-a113-05a96ab94935"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:34.370108 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/45e26e8c-71b5-4b90-bba7-5bad2650d9b2-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "45e26e8c-71b5-4b90-bba7-5bad2650d9b2" (UID: "45e26e8c-71b5-4b90-bba7-5bad2650d9b2"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:34.380996 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d877b426-724d-4ccf-a113-05a96ab94935-config-data" (OuterVolumeSpecName: "config-data") pod "d877b426-724d-4ccf-a113-05a96ab94935" (UID: "d877b426-724d-4ccf-a113-05a96ab94935"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:34.451936 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d877b426-724d-4ccf-a113-05a96ab94935-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:34.451964 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/d877b426-724d-4ccf-a113-05a96ab94935-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:34.451975 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/45e26e8c-71b5-4b90-bba7-5bad2650d9b2-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:36 crc kubenswrapper[5052]: E1203 12:08:34.538723 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="a154e1968b2288216f60d8cd5f0368f47e36bcdef4f248bc7568b9b7174e4331" cmd=["/bin/bash","/var/lib/operator-scripts/mysql_probe.sh","readiness"] Dec 03 12:08:36 crc kubenswrapper[5052]: E1203 12:08:34.541017 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="a154e1968b2288216f60d8cd5f0368f47e36bcdef4f248bc7568b9b7174e4331" cmd=["/bin/bash","/var/lib/operator-scripts/mysql_probe.sh","readiness"] Dec 03 12:08:36 crc kubenswrapper[5052]: E1203 12:08:34.542657 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="a154e1968b2288216f60d8cd5f0368f47e36bcdef4f248bc7568b9b7174e4331" cmd=["/bin/bash","/var/lib/operator-scripts/mysql_probe.sh","readiness"] Dec 03 12:08:36 crc kubenswrapper[5052]: E1203 12:08:34.542696 5052 prober.go:104] "Probe errored" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" probeType="Readiness" pod="openstack-kuttl-tests/openstack-galera-2" podUID="143f4f1d-4f91-4ffa-a277-d6e551d19838" containerName="galera" Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:34.740759 5052 generic.go:334] "Generic (PLEG): container finished" podID="aaa3db64-a10b-45ea-9e25-de4916f30617" containerID="19566104651a9853da1345044a96ed40b73615a3c7ab75d772d57f5e5fdc9b84" exitCode=1 Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:34.740820 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement3c40-account-delete-8qsdr" event={"ID":"aaa3db64-a10b-45ea-9e25-de4916f30617","Type":"ContainerDied","Data":"19566104651a9853da1345044a96ed40b73615a3c7ab75d772d57f5e5fdc9b84"} Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:34.741405 5052 kubelet_pods.go:1007] "Unable to retrieve pull secret, the image pull may not succeed." pod="openstack-kuttl-tests/placement3c40-account-delete-8qsdr" secret="" err="secret \"galera-openstack-dockercfg-8287r\" not found" Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:34.741436 5052 scope.go:117] "RemoveContainer" containerID="19566104651a9853da1345044a96ed40b73615a3c7ab75d772d57f5e5fdc9b84" Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:34.747799 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-worker-c656b9c77-2gf7g" event={"ID":"a3da2acb-8a0e-4ab3-8d15-2978eca73850","Type":"ContainerDied","Data":"99f10e049c628513e1e1525ed3dbcd424c53f799c5fed159b769c8e0375f0bfc"} Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:34.747836 5052 scope.go:117] "RemoveContainer" containerID="606f51e20d39e18fce0fe0fe7c840d6c46bd6e03a7e775bc7246b576bec5c132" Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:34.747887 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-worker-c656b9c77-2gf7g" Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:34.752636 5052 generic.go:334] "Generic (PLEG): container finished" podID="bbdcaf66-8813-4bed-9311-552fef62ac2d" containerID="e6787f6e4b03711dd124764348c5ed1eaf82f566af649919f63f24cc48189367" exitCode=0 Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:34.753047 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/memcached-0" event={"ID":"bbdcaf66-8813-4bed-9311-552fef62ac2d","Type":"ContainerDied","Data":"e6787f6e4b03711dd124764348c5ed1eaf82f566af649919f63f24cc48189367"} Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:34.762957 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:34.767406 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"0b100677-2da0-4717-a15a-d2463ee6713e","Type":"ContainerDied","Data":"1da1fd81580901fcd43d9bc625b28fcabd26928a5c783a3c40a4628aa15caa92"} Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:34.783343 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"bef08f40-1b01-406f-af05-7b63e0b6e5e7","Type":"ContainerDied","Data":"46f9b1f29a0bfcc855368ead467b689caf617edf4d3c13cbbf5dac62d701a661"} Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:34.783437 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:34.785781 5052 generic.go:334] "Generic (PLEG): container finished" podID="70122e67-6175-4a14-a120-6a2731c5dcb1" containerID="265855fad894b2f79c87685abb31a04136664b6466e2b8a541217f1335a297fc" exitCode=1 Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:34.785843 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/novaapi48e2-account-delete-kqhz7" event={"ID":"70122e67-6175-4a14-a120-6a2731c5dcb1","Type":"ContainerDied","Data":"265855fad894b2f79c87685abb31a04136664b6466e2b8a541217f1335a297fc"} Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:34.786401 5052 kubelet_pods.go:1007] "Unable to retrieve pull secret, the image pull may not succeed." pod="openstack-kuttl-tests/novaapi48e2-account-delete-kqhz7" secret="" err="secret \"galera-openstack-dockercfg-8287r\" not found" Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:34.786434 5052 scope.go:117] "RemoveContainer" containerID="265855fad894b2f79c87685abb31a04136664b6466e2b8a541217f1335a297fc" Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:34.788623 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_ovn-northd-0_fad6f453-acf5-4f13-94d6-7d46e300f28c/ovn-northd/0.log" Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:34.788648 5052 generic.go:334] "Generic (PLEG): container finished" podID="fad6f453-acf5-4f13-94d6-7d46e300f28c" containerID="f82e335d0814000e829c2020ade84de1617dea07a919392e9736c05912d0b4bf" exitCode=139 Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:34.788689 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovn-northd-0" event={"ID":"fad6f453-acf5-4f13-94d6-7d46e300f28c","Type":"ContainerDied","Data":"f82e335d0814000e829c2020ade84de1617dea07a919392e9736c05912d0b4bf"} Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:34.793810 5052 generic.go:334] "Generic (PLEG): container finished" podID="89ee1bab-2c68-4fc4-9171-a7330922faf6" containerID="e1e5f1e1f180364add520eba6060fbec83c2ab074ee509c5c280da93bca650db" exitCode=1 Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:34.793933 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cindere6fc-account-delete-z7w6l" event={"ID":"89ee1bab-2c68-4fc4-9171-a7330922faf6","Type":"ContainerDied","Data":"e1e5f1e1f180364add520eba6060fbec83c2ab074ee509c5c280da93bca650db"} Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:34.795519 5052 kubelet_pods.go:1007] "Unable to retrieve pull secret, the image pull may not succeed." pod="openstack-kuttl-tests/cindere6fc-account-delete-z7w6l" secret="" err="secret \"galera-openstack-dockercfg-8287r\" not found" Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:34.795605 5052 scope.go:117] "RemoveContainer" containerID="e1e5f1e1f180364add520eba6060fbec83c2ab074ee509c5c280da93bca650db" Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:34.820879 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"31e6a111-bd4a-45a5-95a0-a6b537added6","Type":"ContainerDied","Data":"f440aa9d51a5b63ef7463ff51ba3bb2689f7785c7cee41c23dfa059d063e983e"} Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:34.820923 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f440aa9d51a5b63ef7463ff51ba3bb2689f7785c7cee41c23dfa059d063e983e" Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:34.823442 5052 generic.go:334] "Generic (PLEG): container finished" podID="9e6af047-b7ee-459a-b362-da5f777d0fa0" containerID="cd9af7528dabedd1b57634f875a45a028137dc589398e5472f322ce8266304eb" exitCode=1 Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:34.823491 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutrond5c4-account-delete-zj77r" event={"ID":"9e6af047-b7ee-459a-b362-da5f777d0fa0","Type":"ContainerDied","Data":"cd9af7528dabedd1b57634f875a45a028137dc589398e5472f322ce8266304eb"} Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:34.824191 5052 kubelet_pods.go:1007] "Unable to retrieve pull secret, the image pull may not succeed." pod="openstack-kuttl-tests/neutrond5c4-account-delete-zj77r" secret="" err="secret \"galera-openstack-dockercfg-8287r\" not found" Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:34.824236 5052 scope.go:117] "RemoveContainer" containerID="cd9af7528dabedd1b57634f875a45a028137dc589398e5472f322ce8266304eb" Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:34.826579 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/kube-state-metrics-0" event={"ID":"305f2518-9632-4358-84a1-632b4a7d8f6f","Type":"ContainerDied","Data":"9601255802d345a0ea7eb7506def54601b12e61f8b10070fefb875a78e96c4a3"} Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:34.826607 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9601255802d345a0ea7eb7506def54601b12e61f8b10070fefb875a78e96c4a3" Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:34.831873 5052 generic.go:334] "Generic (PLEG): container finished" podID="2f3b4f6a-5f11-4039-a3bd-7b553ab5684d" containerID="003f41bab553dfad821839177281778304bb5889d3df5b1416ecb71017b910c3" exitCode=1 Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:34.831957 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance8fcf-account-delete-4xhcc" event={"ID":"2f3b4f6a-5f11-4039-a3bd-7b553ab5684d","Type":"ContainerDied","Data":"003f41bab553dfad821839177281778304bb5889d3df5b1416ecb71017b910c3"} Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:34.832815 5052 kubelet_pods.go:1007] "Unable to retrieve pull secret, the image pull may not succeed." pod="openstack-kuttl-tests/glance8fcf-account-delete-4xhcc" secret="" err="secret \"galera-openstack-dockercfg-8287r\" not found" Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:34.832867 5052 scope.go:117] "RemoveContainer" containerID="003f41bab553dfad821839177281778304bb5889d3df5b1416ecb71017b910c3" Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:34.837623 5052 generic.go:334] "Generic (PLEG): container finished" podID="1bc7c8b1-cc0d-4611-b335-01c2b91713e2" containerID="9798f70f1a0313c2fd23b4af0551bf82e2750ff87ce05f947fc3131408eba2e5" exitCode=1 Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:34.837676 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican5c55-account-delete-zttb9" event={"ID":"1bc7c8b1-cc0d-4611-b335-01c2b91713e2","Type":"ContainerDied","Data":"9798f70f1a0313c2fd23b4af0551bf82e2750ff87ce05f947fc3131408eba2e5"} Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:34.838740 5052 kubelet_pods.go:1007] "Unable to retrieve pull secret, the image pull may not succeed." pod="openstack-kuttl-tests/barbican5c55-account-delete-zttb9" secret="" err="secret \"galera-openstack-dockercfg-8287r\" not found" Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:34.838773 5052 scope.go:117] "RemoveContainer" containerID="9798f70f1a0313c2fd23b4af0551bf82e2750ff87ce05f947fc3131408eba2e5" Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:34.851332 5052 generic.go:334] "Generic (PLEG): container finished" podID="030d4f76-eb30-4c44-b782-fa7147febf41" containerID="f6003a5dd700c21fc31c9193f8092e42fdfba4431f6f7eb85497fdf04c513ed1" exitCode=1 Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:34.851456 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/novacell0d294-account-delete-b64wn" event={"ID":"030d4f76-eb30-4c44-b782-fa7147febf41","Type":"ContainerDied","Data":"f6003a5dd700c21fc31c9193f8092e42fdfba4431f6f7eb85497fdf04c513ed1"} Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:34.852283 5052 kubelet_pods.go:1007] "Unable to retrieve pull secret, the image pull may not succeed." pod="openstack-kuttl-tests/novacell0d294-account-delete-b64wn" secret="" err="secret \"galera-openstack-dockercfg-8287r\" not found" Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:34.852329 5052 scope.go:117] "RemoveContainer" containerID="f6003a5dd700c21fc31c9193f8092e42fdfba4431f6f7eb85497fdf04c513ed1" Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:34.857316 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"45e26e8c-71b5-4b90-bba7-5bad2650d9b2","Type":"ContainerDied","Data":"c3a994755bb38f1b0e921a8cf82017f3c0390ce0425049a85226095ee88f384d"} Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:34.857469 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:34.861026 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/19d83b78-c03a-482a-909e-74bb991d974a-operator-scripts\") pod \"keystone25ff-account-delete-2rdct\" (UID: \"19d83b78-c03a-482a-909e-74bb991d974a\") " pod="openstack-kuttl-tests/keystone25ff-account-delete-2rdct" Dec 03 12:08:36 crc kubenswrapper[5052]: E1203 12:08:34.861136 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:08:36 crc kubenswrapper[5052]: E1203 12:08:34.861396 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/19d83b78-c03a-482a-909e-74bb991d974a-operator-scripts podName:19d83b78-c03a-482a-909e-74bb991d974a nodeName:}" failed. No retries permitted until 2025-12-03 12:08:36.861183246 +0000 UTC m=+2907.449425080 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/19d83b78-c03a-482a-909e-74bb991d974a-operator-scripts") pod "keystone25ff-account-delete-2rdct" (UID: "19d83b78-c03a-482a-909e-74bb991d974a") : configmap "openstack-scripts" not found Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:34.861447 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-d2d77\" (UniqueName: \"kubernetes.io/projected/19d83b78-c03a-482a-909e-74bb991d974a-kube-api-access-d2d77\") pod \"keystone25ff-account-delete-2rdct\" (UID: \"19d83b78-c03a-482a-909e-74bb991d974a\") " pod="openstack-kuttl-tests/keystone25ff-account-delete-2rdct" Dec 03 12:08:36 crc kubenswrapper[5052]: E1203 12:08:34.861490 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:08:36 crc kubenswrapper[5052]: E1203 12:08:34.861568 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/aaa3db64-a10b-45ea-9e25-de4916f30617-operator-scripts podName:aaa3db64-a10b-45ea-9e25-de4916f30617 nodeName:}" failed. No retries permitted until 2025-12-03 12:08:35.361551157 +0000 UTC m=+2905.949792921 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/aaa3db64-a10b-45ea-9e25-de4916f30617-operator-scripts") pod "placement3c40-account-delete-8qsdr" (UID: "aaa3db64-a10b-45ea-9e25-de4916f30617") : configmap "openstack-scripts" not found Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:34.862412 5052 generic.go:334] "Generic (PLEG): container finished" podID="19229e07-3c78-4369-998e-fc9e78eb647d" containerID="d1c0d412f0ba24368b4e09fb49e650d7f9751d7ab3deaf51a87873def7cef029" exitCode=0 Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:34.862485 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"19229e07-3c78-4369-998e-fc9e78eb647d","Type":"ContainerDied","Data":"d1c0d412f0ba24368b4e09fb49e650d7f9751d7ab3deaf51a87873def7cef029"} Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:34.871951 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone25ff-account-delete-2rdct" Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:34.871956 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-keystone-listener-64674fc4f4-wfh52" event={"ID":"d877b426-724d-4ccf-a113-05a96ab94935","Type":"ContainerDied","Data":"c46f89f982fb61d9c55d36ab8f8e63d0ab12f714f3dcec36d8ab7c7292d44f67"} Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:34.872123 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-keystone-listener-64674fc4f4-wfh52" Dec 03 12:08:36 crc kubenswrapper[5052]: E1203 12:08:34.875380 5052 projected.go:194] Error preparing data for projected volume kube-api-access-d2d77 for pod openstack-kuttl-tests/keystone25ff-account-delete-2rdct: failed to fetch token: serviceaccounts "galera-openstack" not found Dec 03 12:08:36 crc kubenswrapper[5052]: E1203 12:08:34.875437 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/19d83b78-c03a-482a-909e-74bb991d974a-kube-api-access-d2d77 podName:19d83b78-c03a-482a-909e-74bb991d974a nodeName:}" failed. No retries permitted until 2025-12-03 12:08:36.875420253 +0000 UTC m=+2907.463662007 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-d2d77" (UniqueName: "kubernetes.io/projected/19d83b78-c03a-482a-909e-74bb991d974a-kube-api-access-d2d77") pod "keystone25ff-account-delete-2rdct" (UID: "19d83b78-c03a-482a-909e-74bb991d974a") : failed to fetch token: serviceaccounts "galera-openstack" not found Dec 03 12:08:36 crc kubenswrapper[5052]: E1203 12:08:34.964974 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:08:36 crc kubenswrapper[5052]: E1203 12:08:34.965060 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:08:36 crc kubenswrapper[5052]: E1203 12:08:34.965079 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9e6af047-b7ee-459a-b362-da5f777d0fa0-operator-scripts podName:9e6af047-b7ee-459a-b362-da5f777d0fa0 nodeName:}" failed. No retries permitted until 2025-12-03 12:08:35.465056436 +0000 UTC m=+2906.053298200 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/9e6af047-b7ee-459a-b362-da5f777d0fa0-operator-scripts") pod "neutrond5c4-account-delete-zj77r" (UID: "9e6af047-b7ee-459a-b362-da5f777d0fa0") : configmap "openstack-scripts" not found Dec 03 12:08:36 crc kubenswrapper[5052]: E1203 12:08:34.965595 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/030d4f76-eb30-4c44-b782-fa7147febf41-operator-scripts podName:030d4f76-eb30-4c44-b782-fa7147febf41 nodeName:}" failed. No retries permitted until 2025-12-03 12:08:35.465575801 +0000 UTC m=+2906.053817575 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/030d4f76-eb30-4c44-b782-fa7147febf41-operator-scripts") pod "novacell0d294-account-delete-b64wn" (UID: "030d4f76-eb30-4c44-b782-fa7147febf41") : configmap "openstack-scripts" not found Dec 03 12:08:36 crc kubenswrapper[5052]: E1203 12:08:34.964989 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:08:36 crc kubenswrapper[5052]: E1203 12:08:34.965644 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1bc7c8b1-cc0d-4611-b335-01c2b91713e2-operator-scripts podName:1bc7c8b1-cc0d-4611-b335-01c2b91713e2 nodeName:}" failed. No retries permitted until 2025-12-03 12:08:35.465636582 +0000 UTC m=+2906.053878446 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/1bc7c8b1-cc0d-4611-b335-01c2b91713e2-operator-scripts") pod "barbican5c55-account-delete-zttb9" (UID: "1bc7c8b1-cc0d-4611-b335-01c2b91713e2") : configmap "openstack-scripts" not found Dec 03 12:08:36 crc kubenswrapper[5052]: E1203 12:08:34.965777 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:08:36 crc kubenswrapper[5052]: E1203 12:08:34.965813 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/89ee1bab-2c68-4fc4-9171-a7330922faf6-operator-scripts podName:89ee1bab-2c68-4fc4-9171-a7330922faf6 nodeName:}" failed. No retries permitted until 2025-12-03 12:08:35.465797657 +0000 UTC m=+2906.054039421 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/89ee1bab-2c68-4fc4-9171-a7330922faf6-operator-scripts") pod "cindere6fc-account-delete-z7w6l" (UID: "89ee1bab-2c68-4fc4-9171-a7330922faf6") : configmap "openstack-scripts" not found Dec 03 12:08:36 crc kubenswrapper[5052]: E1203 12:08:34.965851 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:08:36 crc kubenswrapper[5052]: E1203 12:08:34.965875 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/70122e67-6175-4a14-a120-6a2731c5dcb1-operator-scripts podName:70122e67-6175-4a14-a120-6a2731c5dcb1 nodeName:}" failed. No retries permitted until 2025-12-03 12:08:35.465868539 +0000 UTC m=+2906.054110543 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/70122e67-6175-4a14-a120-6a2731c5dcb1-operator-scripts") pod "novaapi48e2-account-delete-kqhz7" (UID: "70122e67-6175-4a14-a120-6a2731c5dcb1") : configmap "openstack-scripts" not found Dec 03 12:08:36 crc kubenswrapper[5052]: E1203 12:08:34.967769 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:08:36 crc kubenswrapper[5052]: E1203 12:08:34.967990 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2f3b4f6a-5f11-4039-a3bd-7b553ab5684d-operator-scripts podName:2f3b4f6a-5f11-4039-a3bd-7b553ab5684d nodeName:}" failed. No retries permitted until 2025-12-03 12:08:35.467909047 +0000 UTC m=+2906.056150811 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/2f3b4f6a-5f11-4039-a3bd-7b553ab5684d-operator-scripts") pod "glance8fcf-account-delete-4xhcc" (UID: "2f3b4f6a-5f11-4039-a3bd-7b553ab5684d") : configmap "openstack-scripts" not found Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:35.129795 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:35.139833 5052 scope.go:117] "RemoveContainer" containerID="80fb2290d9e5fb396a3de096d08940f5a3a89a79f45d052e0d882756596d8b34" Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:35.142606 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:35.276228 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/31e6a111-bd4a-45a5-95a0-a6b537added6-combined-ca-bundle\") pod \"31e6a111-bd4a-45a5-95a0-a6b537added6\" (UID: \"31e6a111-bd4a-45a5-95a0-a6b537added6\") " Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:35.276326 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/31e6a111-bd4a-45a5-95a0-a6b537added6-config-data\") pod \"31e6a111-bd4a-45a5-95a0-a6b537added6\" (UID: \"31e6a111-bd4a-45a5-95a0-a6b537added6\") " Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:35.276355 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/31e6a111-bd4a-45a5-95a0-a6b537added6-scripts\") pod \"31e6a111-bd4a-45a5-95a0-a6b537added6\" (UID: \"31e6a111-bd4a-45a5-95a0-a6b537added6\") " Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:35.276467 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/31e6a111-bd4a-45a5-95a0-a6b537added6-config-data-custom\") pod \"31e6a111-bd4a-45a5-95a0-a6b537added6\" (UID: \"31e6a111-bd4a-45a5-95a0-a6b537added6\") " Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:35.276530 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-state-metrics-tls-certs\" (UniqueName: \"kubernetes.io/secret/305f2518-9632-4358-84a1-632b4a7d8f6f-kube-state-metrics-tls-certs\") pod \"305f2518-9632-4358-84a1-632b4a7d8f6f\" (UID: \"305f2518-9632-4358-84a1-632b4a7d8f6f\") " Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:35.276567 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-state-metrics-tls-config\" (UniqueName: \"kubernetes.io/secret/305f2518-9632-4358-84a1-632b4a7d8f6f-kube-state-metrics-tls-config\") pod \"305f2518-9632-4358-84a1-632b4a7d8f6f\" (UID: \"305f2518-9632-4358-84a1-632b4a7d8f6f\") " Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:35.276601 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/31e6a111-bd4a-45a5-95a0-a6b537added6-internal-tls-certs\") pod \"31e6a111-bd4a-45a5-95a0-a6b537added6\" (UID: \"31e6a111-bd4a-45a5-95a0-a6b537added6\") " Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:35.276629 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/31e6a111-bd4a-45a5-95a0-a6b537added6-logs\") pod \"31e6a111-bd4a-45a5-95a0-a6b537added6\" (UID: \"31e6a111-bd4a-45a5-95a0-a6b537added6\") " Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:35.276654 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-crnn7\" (UniqueName: \"kubernetes.io/projected/305f2518-9632-4358-84a1-632b4a7d8f6f-kube-api-access-crnn7\") pod \"305f2518-9632-4358-84a1-632b4a7d8f6f\" (UID: \"305f2518-9632-4358-84a1-632b4a7d8f6f\") " Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:35.276693 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/305f2518-9632-4358-84a1-632b4a7d8f6f-combined-ca-bundle\") pod \"305f2518-9632-4358-84a1-632b4a7d8f6f\" (UID: \"305f2518-9632-4358-84a1-632b4a7d8f6f\") " Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:35.276748 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7jrwp\" (UniqueName: \"kubernetes.io/projected/31e6a111-bd4a-45a5-95a0-a6b537added6-kube-api-access-7jrwp\") pod \"31e6a111-bd4a-45a5-95a0-a6b537added6\" (UID: \"31e6a111-bd4a-45a5-95a0-a6b537added6\") " Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:35.276773 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/31e6a111-bd4a-45a5-95a0-a6b537added6-etc-machine-id\") pod \"31e6a111-bd4a-45a5-95a0-a6b537added6\" (UID: \"31e6a111-bd4a-45a5-95a0-a6b537added6\") " Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:35.276793 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/31e6a111-bd4a-45a5-95a0-a6b537added6-public-tls-certs\") pod \"31e6a111-bd4a-45a5-95a0-a6b537added6\" (UID: \"31e6a111-bd4a-45a5-95a0-a6b537added6\") " Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:35.277369 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/31e6a111-bd4a-45a5-95a0-a6b537added6-etc-machine-id" (OuterVolumeSpecName: "etc-machine-id") pod "31e6a111-bd4a-45a5-95a0-a6b537added6" (UID: "31e6a111-bd4a-45a5-95a0-a6b537added6"). InnerVolumeSpecName "etc-machine-id". PluginName "kubernetes.io/host-path", VolumeGidValue "" Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:35.278024 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/31e6a111-bd4a-45a5-95a0-a6b537added6-logs" (OuterVolumeSpecName: "logs") pod "31e6a111-bd4a-45a5-95a0-a6b537added6" (UID: "31e6a111-bd4a-45a5-95a0-a6b537added6"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:35.282208 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/31e6a111-bd4a-45a5-95a0-a6b537added6-config-data-custom" (OuterVolumeSpecName: "config-data-custom") pod "31e6a111-bd4a-45a5-95a0-a6b537added6" (UID: "31e6a111-bd4a-45a5-95a0-a6b537added6"). InnerVolumeSpecName "config-data-custom". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:35.282361 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/305f2518-9632-4358-84a1-632b4a7d8f6f-kube-api-access-crnn7" (OuterVolumeSpecName: "kube-api-access-crnn7") pod "305f2518-9632-4358-84a1-632b4a7d8f6f" (UID: "305f2518-9632-4358-84a1-632b4a7d8f6f"). InnerVolumeSpecName "kube-api-access-crnn7". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:35.283807 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/31e6a111-bd4a-45a5-95a0-a6b537added6-kube-api-access-7jrwp" (OuterVolumeSpecName: "kube-api-access-7jrwp") pod "31e6a111-bd4a-45a5-95a0-a6b537added6" (UID: "31e6a111-bd4a-45a5-95a0-a6b537added6"). InnerVolumeSpecName "kube-api-access-7jrwp". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:35.296332 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/31e6a111-bd4a-45a5-95a0-a6b537added6-scripts" (OuterVolumeSpecName: "scripts") pod "31e6a111-bd4a-45a5-95a0-a6b537added6" (UID: "31e6a111-bd4a-45a5-95a0-a6b537added6"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:35.316183 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/305f2518-9632-4358-84a1-632b4a7d8f6f-kube-state-metrics-tls-config" (OuterVolumeSpecName: "kube-state-metrics-tls-config") pod "305f2518-9632-4358-84a1-632b4a7d8f6f" (UID: "305f2518-9632-4358-84a1-632b4a7d8f6f"). InnerVolumeSpecName "kube-state-metrics-tls-config". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:35.321282 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/305f2518-9632-4358-84a1-632b4a7d8f6f-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "305f2518-9632-4358-84a1-632b4a7d8f6f" (UID: "305f2518-9632-4358-84a1-632b4a7d8f6f"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:35.328801 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/31e6a111-bd4a-45a5-95a0-a6b537added6-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "31e6a111-bd4a-45a5-95a0-a6b537added6" (UID: "31e6a111-bd4a-45a5-95a0-a6b537added6"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:35.338749 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/31e6a111-bd4a-45a5-95a0-a6b537added6-config-data" (OuterVolumeSpecName: "config-data") pod "31e6a111-bd4a-45a5-95a0-a6b537added6" (UID: "31e6a111-bd4a-45a5-95a0-a6b537added6"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:35.342542 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/31e6a111-bd4a-45a5-95a0-a6b537added6-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "31e6a111-bd4a-45a5-95a0-a6b537added6" (UID: "31e6a111-bd4a-45a5-95a0-a6b537added6"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:35.353772 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/31e6a111-bd4a-45a5-95a0-a6b537added6-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "31e6a111-bd4a-45a5-95a0-a6b537added6" (UID: "31e6a111-bd4a-45a5-95a0-a6b537added6"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:35.362711 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/305f2518-9632-4358-84a1-632b4a7d8f6f-kube-state-metrics-tls-certs" (OuterVolumeSpecName: "kube-state-metrics-tls-certs") pod "305f2518-9632-4358-84a1-632b4a7d8f6f" (UID: "305f2518-9632-4358-84a1-632b4a7d8f6f"). InnerVolumeSpecName "kube-state-metrics-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:35.394904 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/31e6a111-bd4a-45a5-95a0-a6b537added6-config-data-custom\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:35.394979 5052 reconciler_common.go:293] "Volume detached for volume \"kube-state-metrics-tls-certs\" (UniqueName: \"kubernetes.io/secret/305f2518-9632-4358-84a1-632b4a7d8f6f-kube-state-metrics-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:35.394998 5052 reconciler_common.go:293] "Volume detached for volume \"kube-state-metrics-tls-config\" (UniqueName: \"kubernetes.io/secret/305f2518-9632-4358-84a1-632b4a7d8f6f-kube-state-metrics-tls-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:35.395012 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/31e6a111-bd4a-45a5-95a0-a6b537added6-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:35.395025 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/31e6a111-bd4a-45a5-95a0-a6b537added6-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:35.395038 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-crnn7\" (UniqueName: \"kubernetes.io/projected/305f2518-9632-4358-84a1-632b4a7d8f6f-kube-api-access-crnn7\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:35.395051 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/305f2518-9632-4358-84a1-632b4a7d8f6f-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:35.395063 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-7jrwp\" (UniqueName: \"kubernetes.io/projected/31e6a111-bd4a-45a5-95a0-a6b537added6-kube-api-access-7jrwp\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:35.395077 5052 reconciler_common.go:293] "Volume detached for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/31e6a111-bd4a-45a5-95a0-a6b537added6-etc-machine-id\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:35.395089 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/31e6a111-bd4a-45a5-95a0-a6b537added6-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:35.395102 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/31e6a111-bd4a-45a5-95a0-a6b537added6-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:35.395118 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/31e6a111-bd4a-45a5-95a0-a6b537added6-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:35.395131 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/31e6a111-bd4a-45a5-95a0-a6b537added6-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:36 crc kubenswrapper[5052]: E1203 12:08:35.395242 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:08:36 crc kubenswrapper[5052]: E1203 12:08:35.395330 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/aaa3db64-a10b-45ea-9e25-de4916f30617-operator-scripts podName:aaa3db64-a10b-45ea-9e25-de4916f30617 nodeName:}" failed. No retries permitted until 2025-12-03 12:08:36.395300586 +0000 UTC m=+2906.983542350 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/aaa3db64-a10b-45ea-9e25-de4916f30617-operator-scripts") pod "placement3c40-account-delete-8qsdr" (UID: "aaa3db64-a10b-45ea-9e25-de4916f30617") : configmap "openstack-scripts" not found Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:35.462282 5052 scope.go:117] "RemoveContainer" containerID="df9dfde29f7996c2208b2860c440e3ab6120cd657e154f046ad4cdb1b318a678" Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:35.503438 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-api-d978c699b-kzzst"] Dec 03 12:08:36 crc kubenswrapper[5052]: E1203 12:08:35.503607 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:08:36 crc kubenswrapper[5052]: E1203 12:08:35.503665 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1bc7c8b1-cc0d-4611-b335-01c2b91713e2-operator-scripts podName:1bc7c8b1-cc0d-4611-b335-01c2b91713e2 nodeName:}" failed. No retries permitted until 2025-12-03 12:08:36.503648044 +0000 UTC m=+2907.091889808 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/1bc7c8b1-cc0d-4611-b335-01c2b91713e2-operator-scripts") pod "barbican5c55-account-delete-zttb9" (UID: "1bc7c8b1-cc0d-4611-b335-01c2b91713e2") : configmap "openstack-scripts" not found Dec 03 12:08:36 crc kubenswrapper[5052]: E1203 12:08:35.504140 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:08:36 crc kubenswrapper[5052]: E1203 12:08:35.504175 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/89ee1bab-2c68-4fc4-9171-a7330922faf6-operator-scripts podName:89ee1bab-2c68-4fc4-9171-a7330922faf6 nodeName:}" failed. No retries permitted until 2025-12-03 12:08:36.504159939 +0000 UTC m=+2907.092401703 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/89ee1bab-2c68-4fc4-9171-a7330922faf6-operator-scripts") pod "cindere6fc-account-delete-z7w6l" (UID: "89ee1bab-2c68-4fc4-9171-a7330922faf6") : configmap "openstack-scripts" not found Dec 03 12:08:36 crc kubenswrapper[5052]: E1203 12:08:35.504212 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:08:36 crc kubenswrapper[5052]: E1203 12:08:35.504237 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9e6af047-b7ee-459a-b362-da5f777d0fa0-operator-scripts podName:9e6af047-b7ee-459a-b362-da5f777d0fa0 nodeName:}" failed. No retries permitted until 2025-12-03 12:08:36.504230131 +0000 UTC m=+2907.092471895 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/9e6af047-b7ee-459a-b362-da5f777d0fa0-operator-scripts") pod "neutrond5c4-account-delete-zj77r" (UID: "9e6af047-b7ee-459a-b362-da5f777d0fa0") : configmap "openstack-scripts" not found Dec 03 12:08:36 crc kubenswrapper[5052]: E1203 12:08:35.504285 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:08:36 crc kubenswrapper[5052]: E1203 12:08:35.504371 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2f3b4f6a-5f11-4039-a3bd-7b553ab5684d-operator-scripts podName:2f3b4f6a-5f11-4039-a3bd-7b553ab5684d nodeName:}" failed. No retries permitted until 2025-12-03 12:08:36.504352824 +0000 UTC m=+2907.092594588 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/2f3b4f6a-5f11-4039-a3bd-7b553ab5684d-operator-scripts") pod "glance8fcf-account-delete-4xhcc" (UID: "2f3b4f6a-5f11-4039-a3bd-7b553ab5684d") : configmap "openstack-scripts" not found Dec 03 12:08:36 crc kubenswrapper[5052]: E1203 12:08:35.504428 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:08:36 crc kubenswrapper[5052]: E1203 12:08:35.504451 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/030d4f76-eb30-4c44-b782-fa7147febf41-operator-scripts podName:030d4f76-eb30-4c44-b782-fa7147febf41 nodeName:}" failed. No retries permitted until 2025-12-03 12:08:36.504440837 +0000 UTC m=+2907.092682601 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/030d4f76-eb30-4c44-b782-fa7147febf41-operator-scripts") pod "novacell0d294-account-delete-b64wn" (UID: "030d4f76-eb30-4c44-b782-fa7147febf41") : configmap "openstack-scripts" not found Dec 03 12:08:36 crc kubenswrapper[5052]: E1203 12:08:35.504486 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:08:36 crc kubenswrapper[5052]: E1203 12:08:35.504507 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/70122e67-6175-4a14-a120-6a2731c5dcb1-operator-scripts podName:70122e67-6175-4a14-a120-6a2731c5dcb1 nodeName:}" failed. No retries permitted until 2025-12-03 12:08:36.504501148 +0000 UTC m=+2907.092742912 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/70122e67-6175-4a14-a120-6a2731c5dcb1-operator-scripts") pod "novaapi48e2-account-delete-kqhz7" (UID: "70122e67-6175-4a14-a120-6a2731c5dcb1") : configmap "openstack-scripts" not found Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:35.515783 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/barbican-api-d978c699b-kzzst"] Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:35.526192 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:35.535328 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:35.555816 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:35.565872 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:35.579193 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-worker-c656b9c77-2gf7g"] Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:35.588898 5052 scope.go:117] "RemoveContainer" containerID="df8444be8224ec47d3e75542bba9c0aaa28cc5e0ba5b753982c10dfdcf96b872" Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:35.589774 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/barbican-worker-c656b9c77-2gf7g"] Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:35.602303 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:35.612246 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:35.624458 5052 scope.go:117] "RemoveContainer" containerID="62853e68ee96c13bce5c6927e9083bfe1dca5e094c03354b4a342f37a5ac3ce4" Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:35.624599 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-keystone-listener-64674fc4f4-wfh52"] Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:35.641109 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/barbican-keystone-listener-64674fc4f4-wfh52"] Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:35.652002 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:08:36 crc kubenswrapper[5052]: E1203 12:08:35.655996 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of f82e335d0814000e829c2020ade84de1617dea07a919392e9736c05912d0b4bf is running failed: container process not found" containerID="f82e335d0814000e829c2020ade84de1617dea07a919392e9736c05912d0b4bf" cmd=["/usr/local/bin/container-scripts/status_check.sh"] Dec 03 12:08:36 crc kubenswrapper[5052]: E1203 12:08:35.661269 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of f82e335d0814000e829c2020ade84de1617dea07a919392e9736c05912d0b4bf is running failed: container process not found" containerID="f82e335d0814000e829c2020ade84de1617dea07a919392e9736c05912d0b4bf" cmd=["/usr/local/bin/container-scripts/status_check.sh"] Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:35.661413 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:08:36 crc kubenswrapper[5052]: E1203 12:08:35.664128 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of f82e335d0814000e829c2020ade84de1617dea07a919392e9736c05912d0b4bf is running failed: container process not found" containerID="f82e335d0814000e829c2020ade84de1617dea07a919392e9736c05912d0b4bf" cmd=["/usr/local/bin/container-scripts/status_check.sh"] Dec 03 12:08:36 crc kubenswrapper[5052]: E1203 12:08:35.664171 5052 prober.go:104] "Probe errored" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of f82e335d0814000e829c2020ade84de1617dea07a919392e9736c05912d0b4bf is running failed: container process not found" probeType="Readiness" pod="openstack-kuttl-tests/ovn-northd-0" podUID="fad6f453-acf5-4f13-94d6-7d46e300f28c" containerName="ovn-northd" Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:35.664706 5052 scope.go:117] "RemoveContainer" containerID="a96c58179a61355fd41c8699c34a2a6efcdd96a6901a2acedb4bb445fe0b84fe" Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:35.695994 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0b100677-2da0-4717-a15a-d2463ee6713e" path="/var/lib/kubelet/pods/0b100677-2da0-4717-a15a-d2463ee6713e/volumes" Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:35.696756 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="45e26e8c-71b5-4b90-bba7-5bad2650d9b2" path="/var/lib/kubelet/pods/45e26e8c-71b5-4b90-bba7-5bad2650d9b2/volumes" Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:35.697456 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4a2a1881-e61a-4a99-a2f3-b614aeed81d1" path="/var/lib/kubelet/pods/4a2a1881-e61a-4a99-a2f3-b614aeed81d1/volumes" Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:35.712153 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6999e86b-8f95-4002-9959-6288a7d783bd" path="/var/lib/kubelet/pods/6999e86b-8f95-4002-9959-6288a7d783bd/volumes" Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:35.712932 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a3da2acb-8a0e-4ab3-8d15-2978eca73850" path="/var/lib/kubelet/pods/a3da2acb-8a0e-4ab3-8d15-2978eca73850/volumes" Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:35.714197 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b9efd251-95c2-4224-b318-a34279ee1017" path="/var/lib/kubelet/pods/b9efd251-95c2-4224-b318-a34279ee1017/volumes" Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:35.714890 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bef08f40-1b01-406f-af05-7b63e0b6e5e7" path="/var/lib/kubelet/pods/bef08f40-1b01-406f-af05-7b63e0b6e5e7/volumes" Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:35.725858 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d877b426-724d-4ccf-a113-05a96ab94935" path="/var/lib/kubelet/pods/d877b426-724d-4ccf-a113-05a96ab94935/volumes" Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:35.727104 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="de04776b-ea02-40d9-8459-0a519c8c64ad" path="/var/lib/kubelet/pods/de04776b-ea02-40d9-8459-0a519c8c64ad/volumes" Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:35.727711 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e92fa4d7-9bd6-4586-b41a-a557fb672439" path="/var/lib/kubelet/pods/e92fa4d7-9bd6-4586-b41a-a557fb672439/volumes" Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:35.739906 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/openstack-galera-0" podUID="d66ed4ec-fd97-4a62-82cb-cee3fe877524" containerName="galera" probeResult="failure" output="command timed out" Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:35.744533 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/keystone25ff-account-delete-2rdct"] Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:35.744572 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/keystone25ff-account-delete-2rdct"] Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:35.744595 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-db-create-jpmhr"] Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:35.744610 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/glance-db-create-jpmhr"] Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:35.748424 5052 scope.go:117] "RemoveContainer" containerID="37bdeb3b3f3f5f77fea4db0897de61d07f8d244f4f0ab18853e3fa400f45ac54" Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:35.769589 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-8fcf-account-create-update-r5h2k"] Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:35.798708 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/glance-8fcf-account-create-update-r5h2k"] Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:35.833142 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance8fcf-account-delete-4xhcc"] Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:35.847899 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/placement-db-create-cmps4"] Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:35.881318 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/placement-db-create-cmps4"] Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:35.909620 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/placement3c40-account-delete-8qsdr"] Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:35.912737 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/placement-3c40-account-create-update-dtmv4"] Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:35.920871 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/19d83b78-c03a-482a-909e-74bb991d974a-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:35.920900 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-d2d77\" (UniqueName: \"kubernetes.io/projected/19d83b78-c03a-482a-909e-74bb991d974a-kube-api-access-d2d77\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:35.921075 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/placement-3c40-account-create-update-dtmv4"] Dec 03 12:08:36 crc kubenswrapper[5052]: E1203 12:08:35.930130 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="c55564d5848c1a333ba2db20a9c5ecdbf152ba4ebaa847c6e28f6ed69c3a99ed" cmd=["/bin/bash","/var/lib/operator-scripts/mysql_probe.sh","readiness"] Dec 03 12:08:36 crc kubenswrapper[5052]: E1203 12:08:35.948703 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="c55564d5848c1a333ba2db20a9c5ecdbf152ba4ebaa847c6e28f6ed69c3a99ed" cmd=["/bin/bash","/var/lib/operator-scripts/mysql_probe.sh","readiness"] Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:35.950802 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder-db-create-gsdcc"] Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:35.952723 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:35.955876 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:08:36 crc kubenswrapper[5052]: E1203 12:08:35.968331 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="c55564d5848c1a333ba2db20a9c5ecdbf152ba4ebaa847c6e28f6ed69c3a99ed" cmd=["/bin/bash","/var/lib/operator-scripts/mysql_probe.sh","readiness"] Dec 03 12:08:36 crc kubenswrapper[5052]: E1203 12:08:35.968382 5052 prober.go:104] "Probe errored" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" probeType="Readiness" pod="openstack-kuttl-tests/openstack-cell1-galera-1" podUID="b34bb4d0-7906-42e8-a9aa-bd1a4203a312" containerName="galera" Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:35.975777 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/cinder-db-create-gsdcc"] Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:36.014567 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder-e6fc-account-create-update-j5n92"] Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:36.020167 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/cinder-e6fc-account-create-update-j5n92"] Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:36.028584 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cindere6fc-account-delete-z7w6l"] Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:36.034273 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-db-create-z645k"] Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:36.040819 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/barbican-db-create-z645k"] Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:36.057656 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican5c55-account-delete-zttb9"] Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:36.071577 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-5c55-account-create-update-vz6m5"] Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:36.071629 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/barbican-5c55-account-create-update-vz6m5"] Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:36.085746 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/neutron-db-create-z9ftc"] Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:36.105381 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/neutron-db-create-z9ftc"] Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:36.119423 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/neutrond5c4-account-delete-zj77r"] Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:36.130825 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/neutron-d5c4-account-create-update-jngcd"] Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:36.147804 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/neutron-d5c4-account-create-update-jngcd"] Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:36.221543 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell0-db-create-8kvhf"] Dec 03 12:08:36 crc kubenswrapper[5052]: E1203 12:08:36.227855 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="edf48c4037d4be3adfc21b5d97a50d68b5261315fe92282773dd8556852ce51a" cmd=["/bin/bash","/var/lib/operator-scripts/mysql_probe.sh","readiness"] Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:36.231953 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell0-db-create-8kvhf"] Dec 03 12:08:36 crc kubenswrapper[5052]: E1203 12:08:36.232324 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="edf48c4037d4be3adfc21b5d97a50d68b5261315fe92282773dd8556852ce51a" cmd=["/bin/bash","/var/lib/operator-scripts/mysql_probe.sh","readiness"] Dec 03 12:08:36 crc kubenswrapper[5052]: E1203 12:08:36.234302 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="edf48c4037d4be3adfc21b5d97a50d68b5261315fe92282773dd8556852ce51a" cmd=["/bin/bash","/var/lib/operator-scripts/mysql_probe.sh","readiness"] Dec 03 12:08:36 crc kubenswrapper[5052]: E1203 12:08:36.234393 5052 prober.go:104] "Probe errored" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" probeType="Readiness" pod="openstack-kuttl-tests/openstack-cell1-galera-2" podUID="6ac95bd3-125c-4966-b9a7-4eb4a7842c67" containerName="galera" Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:36.256223 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell0-d294-account-create-update-tm6pc"] Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:36.262653 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/novacell0d294-account-delete-b64wn"] Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:36.270309 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell0-d294-account-create-update-tm6pc"] Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:36.294686 5052 scope.go:117] "RemoveContainer" containerID="51b740be59469e140e437bc25b7208518a7bb1d9c11f719a8bcca63ec56644a8" Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:36.307249 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:36.319427 5052 scope.go:117] "RemoveContainer" containerID="8ed05d2153d95fbbc5b927a2573653ac6d68bb4bb432a7dd29017b0640952a6f" Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:36.320390 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:36.334717 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-api-db-create-t6q5w"] Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:36.345572 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-api-db-create-t6q5w"] Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:36.354904 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/novaapi48e2-account-delete-kqhz7"] Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:36.362823 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-api-48e2-account-create-update-dcqjr"] Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:36.371204 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-api-48e2-account-create-update-dcqjr"] Dec 03 12:08:36 crc kubenswrapper[5052]: E1203 12:08:36.431310 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:08:36 crc kubenswrapper[5052]: E1203 12:08:36.431386 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/aaa3db64-a10b-45ea-9e25-de4916f30617-operator-scripts podName:aaa3db64-a10b-45ea-9e25-de4916f30617 nodeName:}" failed. No retries permitted until 2025-12-03 12:08:38.431366516 +0000 UTC m=+2909.019608280 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/aaa3db64-a10b-45ea-9e25-de4916f30617-operator-scripts") pod "placement3c40-account-delete-8qsdr" (UID: "aaa3db64-a10b-45ea-9e25-de4916f30617") : configmap "openstack-scripts" not found Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:36.514598 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/memcached-0" podUID="bbdcaf66-8813-4bed-9311-552fef62ac2d" containerName="memcached" probeResult="failure" output="dial tcp 10.217.0.112:11211: connect: connection refused" Dec 03 12:08:36 crc kubenswrapper[5052]: E1203 12:08:36.533631 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:08:36 crc kubenswrapper[5052]: E1203 12:08:36.533728 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/70122e67-6175-4a14-a120-6a2731c5dcb1-operator-scripts podName:70122e67-6175-4a14-a120-6a2731c5dcb1 nodeName:}" failed. No retries permitted until 2025-12-03 12:08:38.533706842 +0000 UTC m=+2909.121948646 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/70122e67-6175-4a14-a120-6a2731c5dcb1-operator-scripts") pod "novaapi48e2-account-delete-kqhz7" (UID: "70122e67-6175-4a14-a120-6a2731c5dcb1") : configmap "openstack-scripts" not found Dec 03 12:08:36 crc kubenswrapper[5052]: E1203 12:08:36.533964 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:08:36 crc kubenswrapper[5052]: E1203 12:08:36.534040 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1bc7c8b1-cc0d-4611-b335-01c2b91713e2-operator-scripts podName:1bc7c8b1-cc0d-4611-b335-01c2b91713e2 nodeName:}" failed. No retries permitted until 2025-12-03 12:08:38.534017161 +0000 UTC m=+2909.122258925 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/1bc7c8b1-cc0d-4611-b335-01c2b91713e2-operator-scripts") pod "barbican5c55-account-delete-zttb9" (UID: "1bc7c8b1-cc0d-4611-b335-01c2b91713e2") : configmap "openstack-scripts" not found Dec 03 12:08:36 crc kubenswrapper[5052]: E1203 12:08:36.534079 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:08:36 crc kubenswrapper[5052]: E1203 12:08:36.534102 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:08:36 crc kubenswrapper[5052]: E1203 12:08:36.534113 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:08:36 crc kubenswrapper[5052]: E1203 12:08:36.534109 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9e6af047-b7ee-459a-b362-da5f777d0fa0-operator-scripts podName:9e6af047-b7ee-459a-b362-da5f777d0fa0 nodeName:}" failed. No retries permitted until 2025-12-03 12:08:38.534100643 +0000 UTC m=+2909.122342497 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/9e6af047-b7ee-459a-b362-da5f777d0fa0-operator-scripts") pod "neutrond5c4-account-delete-zj77r" (UID: "9e6af047-b7ee-459a-b362-da5f777d0fa0") : configmap "openstack-scripts" not found Dec 03 12:08:36 crc kubenswrapper[5052]: E1203 12:08:36.534162 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2f3b4f6a-5f11-4039-a3bd-7b553ab5684d-operator-scripts podName:2f3b4f6a-5f11-4039-a3bd-7b553ab5684d nodeName:}" failed. No retries permitted until 2025-12-03 12:08:38.534144165 +0000 UTC m=+2909.122385929 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/2f3b4f6a-5f11-4039-a3bd-7b553ab5684d-operator-scripts") pod "glance8fcf-account-delete-4xhcc" (UID: "2f3b4f6a-5f11-4039-a3bd-7b553ab5684d") : configmap "openstack-scripts" not found Dec 03 12:08:36 crc kubenswrapper[5052]: E1203 12:08:36.534173 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/89ee1bab-2c68-4fc4-9171-a7330922faf6-operator-scripts podName:89ee1bab-2c68-4fc4-9171-a7330922faf6 nodeName:}" failed. No retries permitted until 2025-12-03 12:08:38.534167975 +0000 UTC m=+2909.122409729 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/89ee1bab-2c68-4fc4-9171-a7330922faf6-operator-scripts") pod "cindere6fc-account-delete-z7w6l" (UID: "89ee1bab-2c68-4fc4-9171-a7330922faf6") : configmap "openstack-scripts" not found Dec 03 12:08:36 crc kubenswrapper[5052]: E1203 12:08:36.534335 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:08:36 crc kubenswrapper[5052]: E1203 12:08:36.534427 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/030d4f76-eb30-4c44-b782-fa7147febf41-operator-scripts podName:030d4f76-eb30-4c44-b782-fa7147febf41 nodeName:}" failed. No retries permitted until 2025-12-03 12:08:38.534408472 +0000 UTC m=+2909.122650236 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/030d4f76-eb30-4c44-b782-fa7147febf41-operator-scripts") pod "novacell0d294-account-delete-b64wn" (UID: "030d4f76-eb30-4c44-b782-fa7147febf41") : configmap "openstack-scripts" not found Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:36.622610 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/swift-proxy-9bb7db5d-5q7wc" podUID="028738cd-3b98-4853-9506-d2eba91afd87" containerName="proxy-httpd" probeResult="failure" output="Get \"https://10.217.0.163:8080/healthcheck\": dial tcp 10.217.0.163:8080: i/o timeout" Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:36.622724 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/swift-proxy-9bb7db5d-5q7wc" podUID="028738cd-3b98-4853-9506-d2eba91afd87" containerName="proxy-server" probeResult="failure" output="Get \"https://10.217.0.163:8080/healthcheck\": dial tcp 10.217.0.163:8080: i/o timeout" Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:36.988855 5052 generic.go:334] "Generic (PLEG): container finished" podID="19229e07-3c78-4369-998e-fc9e78eb647d" containerID="85d1ba6ba0934dbfc1b081d24d64fbe85278667fa8c3b4ccf7dd7749c336f706" exitCode=0 Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:36.988949 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"19229e07-3c78-4369-998e-fc9e78eb647d","Type":"ContainerDied","Data":"85d1ba6ba0934dbfc1b081d24d64fbe85278667fa8c3b4ccf7dd7749c336f706"} Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:36.998341 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/memcached-0" event={"ID":"bbdcaf66-8813-4bed-9311-552fef62ac2d","Type":"ContainerDied","Data":"9461f788efd8441e720cf3cc20734008a6a4bac027929586fefd7e176e64de95"} Dec 03 12:08:36 crc kubenswrapper[5052]: I1203 12:08:36.998380 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9461f788efd8441e720cf3cc20734008a6a4bac027929586fefd7e176e64de95" Dec 03 12:08:37 crc kubenswrapper[5052]: I1203 12:08:37.001348 5052 generic.go:334] "Generic (PLEG): container finished" podID="60f72892-a5aa-482f-aac3-5713ba028bc8" containerID="082794493f1a866da7efb8038f854ca6442bb335e30fc8e05b37438e570e2b64" exitCode=0 Dec 03 12:08:37 crc kubenswrapper[5052]: I1203 12:08:37.001399 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"60f72892-a5aa-482f-aac3-5713ba028bc8","Type":"ContainerDied","Data":"082794493f1a866da7efb8038f854ca6442bb335e30fc8e05b37438e570e2b64"} Dec 03 12:08:37 crc kubenswrapper[5052]: I1203 12:08:37.003976 5052 generic.go:334] "Generic (PLEG): container finished" podID="9e6af047-b7ee-459a-b362-da5f777d0fa0" containerID="10aa046238c04ac35e5af15753b94eae0f3932f159a2bc2af4aa74369e06cf8e" exitCode=1 Dec 03 12:08:37 crc kubenswrapper[5052]: I1203 12:08:37.004016 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutrond5c4-account-delete-zj77r" event={"ID":"9e6af047-b7ee-459a-b362-da5f777d0fa0","Type":"ContainerDied","Data":"10aa046238c04ac35e5af15753b94eae0f3932f159a2bc2af4aa74369e06cf8e"} Dec 03 12:08:37 crc kubenswrapper[5052]: I1203 12:08:37.004038 5052 scope.go:117] "RemoveContainer" containerID="cd9af7528dabedd1b57634f875a45a028137dc589398e5472f322ce8266304eb" Dec 03 12:08:37 crc kubenswrapper[5052]: I1203 12:08:37.013489 5052 generic.go:334] "Generic (PLEG): container finished" podID="2f3b4f6a-5f11-4039-a3bd-7b553ab5684d" containerID="41d7b7e369d34dc3d1117c0153f51d17b6ab0a7309a0b40c692c80168a43bba0" exitCode=1 Dec 03 12:08:37 crc kubenswrapper[5052]: I1203 12:08:37.013574 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance8fcf-account-delete-4xhcc" event={"ID":"2f3b4f6a-5f11-4039-a3bd-7b553ab5684d","Type":"ContainerDied","Data":"41d7b7e369d34dc3d1117c0153f51d17b6ab0a7309a0b40c692c80168a43bba0"} Dec 03 12:08:37 crc kubenswrapper[5052]: I1203 12:08:37.017591 5052 generic.go:334] "Generic (PLEG): container finished" podID="1bc7c8b1-cc0d-4611-b335-01c2b91713e2" containerID="90fdc12931b30ef7c91db47e3cdd46baa57f5468f761a99d507e1b30d5373dd1" exitCode=1 Dec 03 12:08:37 crc kubenswrapper[5052]: I1203 12:08:37.017675 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican5c55-account-delete-zttb9" event={"ID":"1bc7c8b1-cc0d-4611-b335-01c2b91713e2","Type":"ContainerDied","Data":"90fdc12931b30ef7c91db47e3cdd46baa57f5468f761a99d507e1b30d5373dd1"} Dec 03 12:08:37 crc kubenswrapper[5052]: I1203 12:08:37.075554 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_ovn-northd-0_fad6f453-acf5-4f13-94d6-7d46e300f28c/ovn-northd/0.log" Dec 03 12:08:37 crc kubenswrapper[5052]: I1203 12:08:37.075705 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovn-northd-0" event={"ID":"fad6f453-acf5-4f13-94d6-7d46e300f28c","Type":"ContainerDied","Data":"1ebd09bbdb8baf71a090c3b4929e61b584b38934f5432877c609354b4da540cb"} Dec 03 12:08:37 crc kubenswrapper[5052]: I1203 12:08:37.075742 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1ebd09bbdb8baf71a090c3b4929e61b584b38934f5432877c609354b4da540cb" Dec 03 12:08:37 crc kubenswrapper[5052]: I1203 12:08:37.080478 5052 generic.go:334] "Generic (PLEG): container finished" podID="89ee1bab-2c68-4fc4-9171-a7330922faf6" containerID="1440892d9f916d23a5ce08a7caddfbd149760ed9506b5d350d942e0b4765bf28" exitCode=1 Dec 03 12:08:37 crc kubenswrapper[5052]: I1203 12:08:37.080555 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cindere6fc-account-delete-z7w6l" event={"ID":"89ee1bab-2c68-4fc4-9171-a7330922faf6","Type":"ContainerDied","Data":"1440892d9f916d23a5ce08a7caddfbd149760ed9506b5d350d942e0b4765bf28"} Dec 03 12:08:37 crc kubenswrapper[5052]: I1203 12:08:37.088537 5052 generic.go:334] "Generic (PLEG): container finished" podID="c2470da9-4c96-44ac-834d-2b560785135f" containerID="aa4334f0d2af3013c5ace40839543a0d0b6116ced72af25a3daba9191d3d0382" exitCode=0 Dec 03 12:08:37 crc kubenswrapper[5052]: I1203 12:08:37.088611 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-6c6748d95d-66jbh" event={"ID":"c2470da9-4c96-44ac-834d-2b560785135f","Type":"ContainerDied","Data":"aa4334f0d2af3013c5ace40839543a0d0b6116ced72af25a3daba9191d3d0382"} Dec 03 12:08:37 crc kubenswrapper[5052]: I1203 12:08:37.101091 5052 generic.go:334] "Generic (PLEG): container finished" podID="141a49da-93b6-483b-a476-1d5f3efc29d1" containerID="3d0ba918e75cd950bc4bab7a527b72d23c454cb5e03d5804f02c37caed27d0fa" exitCode=0 Dec 03 12:08:37 crc kubenswrapper[5052]: I1203 12:08:37.101153 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-5d6b56fbdb-ht2b8" event={"ID":"141a49da-93b6-483b-a476-1d5f3efc29d1","Type":"ContainerDied","Data":"3d0ba918e75cd950bc4bab7a527b72d23c454cb5e03d5804f02c37caed27d0fa"} Dec 03 12:08:37 crc kubenswrapper[5052]: I1203 12:08:37.147018 5052 generic.go:334] "Generic (PLEG): container finished" podID="030d4f76-eb30-4c44-b782-fa7147febf41" containerID="ee4e5d9018afd7520475044120c78ad156782c169bd6979998e8db4bb75386a2" exitCode=1 Dec 03 12:08:37 crc kubenswrapper[5052]: I1203 12:08:37.147084 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/novacell0d294-account-delete-b64wn" event={"ID":"030d4f76-eb30-4c44-b782-fa7147febf41","Type":"ContainerDied","Data":"ee4e5d9018afd7520475044120c78ad156782c169bd6979998e8db4bb75386a2"} Dec 03 12:08:37 crc kubenswrapper[5052]: I1203 12:08:37.164945 5052 generic.go:334] "Generic (PLEG): container finished" podID="70122e67-6175-4a14-a120-6a2731c5dcb1" containerID="5ae82a1363577b2eb43c12684bdc2300b53209b11ba26536b7047f5b4704d361" exitCode=1 Dec 03 12:08:37 crc kubenswrapper[5052]: I1203 12:08:37.165009 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/novaapi48e2-account-delete-kqhz7" event={"ID":"70122e67-6175-4a14-a120-6a2731c5dcb1","Type":"ContainerDied","Data":"5ae82a1363577b2eb43c12684bdc2300b53209b11ba26536b7047f5b4704d361"} Dec 03 12:08:37 crc kubenswrapper[5052]: I1203 12:08:37.178785 5052 generic.go:334] "Generic (PLEG): container finished" podID="aaa3db64-a10b-45ea-9e25-de4916f30617" containerID="b5a72c6d1c868fd1b07944228c1d42fca5faaf1bc649f58101b2d16f25813311" exitCode=1 Dec 03 12:08:37 crc kubenswrapper[5052]: I1203 12:08:37.178822 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement3c40-account-delete-8qsdr" event={"ID":"aaa3db64-a10b-45ea-9e25-de4916f30617","Type":"ContainerDied","Data":"b5a72c6d1c868fd1b07944228c1d42fca5faaf1bc649f58101b2d16f25813311"} Dec 03 12:08:37 crc kubenswrapper[5052]: I1203 12:08:37.279232 5052 scope.go:117] "RemoveContainer" containerID="003f41bab553dfad821839177281778304bb5889d3df5b1416ecb71017b910c3" Dec 03 12:08:37 crc kubenswrapper[5052]: I1203 12:08:37.294314 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_ovn-northd-0_fad6f453-acf5-4f13-94d6-7d46e300f28c/ovn-northd/0.log" Dec 03 12:08:37 crc kubenswrapper[5052]: I1203 12:08:37.294410 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:08:37 crc kubenswrapper[5052]: I1203 12:08:37.299626 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/memcached-0" Dec 03 12:08:37 crc kubenswrapper[5052]: I1203 12:08:37.312775 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:08:37 crc kubenswrapper[5052]: I1203 12:08:37.333835 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:08:37 crc kubenswrapper[5052]: I1203 12:08:37.344613 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-5d6b56fbdb-ht2b8" Dec 03 12:08:37 crc kubenswrapper[5052]: I1203 12:08:37.415988 5052 scope.go:117] "RemoveContainer" containerID="9798f70f1a0313c2fd23b4af0551bf82e2750ff87ce05f947fc3131408eba2e5" Dec 03 12:08:37 crc kubenswrapper[5052]: I1203 12:08:37.456880 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/19229e07-3c78-4369-998e-fc9e78eb647d-ceilometer-tls-certs\") pod \"19229e07-3c78-4369-998e-fc9e78eb647d\" (UID: \"19229e07-3c78-4369-998e-fc9e78eb647d\") " Dec 03 12:08:37 crc kubenswrapper[5052]: I1203 12:08:37.456951 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/bbdcaf66-8813-4bed-9311-552fef62ac2d-combined-ca-bundle\") pod \"bbdcaf66-8813-4bed-9311-552fef62ac2d\" (UID: \"bbdcaf66-8813-4bed-9311-552fef62ac2d\") " Dec 03 12:08:37 crc kubenswrapper[5052]: I1203 12:08:37.456986 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zmhzk\" (UniqueName: \"kubernetes.io/projected/141a49da-93b6-483b-a476-1d5f3efc29d1-kube-api-access-zmhzk\") pod \"141a49da-93b6-483b-a476-1d5f3efc29d1\" (UID: \"141a49da-93b6-483b-a476-1d5f3efc29d1\") " Dec 03 12:08:37 crc kubenswrapper[5052]: I1203 12:08:37.457019 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/19229e07-3c78-4369-998e-fc9e78eb647d-config-data\") pod \"19229e07-3c78-4369-998e-fc9e78eb647d\" (UID: \"19229e07-3c78-4369-998e-fc9e78eb647d\") " Dec 03 12:08:37 crc kubenswrapper[5052]: I1203 12:08:37.457041 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/19229e07-3c78-4369-998e-fc9e78eb647d-scripts\") pod \"19229e07-3c78-4369-998e-fc9e78eb647d\" (UID: \"19229e07-3c78-4369-998e-fc9e78eb647d\") " Dec 03 12:08:37 crc kubenswrapper[5052]: I1203 12:08:37.457066 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/60f72892-a5aa-482f-aac3-5713ba028bc8-etc-machine-id\") pod \"60f72892-a5aa-482f-aac3-5713ba028bc8\" (UID: \"60f72892-a5aa-482f-aac3-5713ba028bc8\") " Dec 03 12:08:37 crc kubenswrapper[5052]: I1203 12:08:37.457093 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/60f72892-a5aa-482f-aac3-5713ba028bc8-config-data\") pod \"60f72892-a5aa-482f-aac3-5713ba028bc8\" (UID: \"60f72892-a5aa-482f-aac3-5713ba028bc8\") " Dec 03 12:08:37 crc kubenswrapper[5052]: I1203 12:08:37.457115 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/141a49da-93b6-483b-a476-1d5f3efc29d1-config\") pod \"141a49da-93b6-483b-a476-1d5f3efc29d1\" (UID: \"141a49da-93b6-483b-a476-1d5f3efc29d1\") " Dec 03 12:08:37 crc kubenswrapper[5052]: I1203 12:08:37.457139 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/141a49da-93b6-483b-a476-1d5f3efc29d1-ovndb-tls-certs\") pod \"141a49da-93b6-483b-a476-1d5f3efc29d1\" (UID: \"141a49da-93b6-483b-a476-1d5f3efc29d1\") " Dec 03 12:08:37 crc kubenswrapper[5052]: I1203 12:08:37.457177 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/19229e07-3c78-4369-998e-fc9e78eb647d-log-httpd\") pod \"19229e07-3c78-4369-998e-fc9e78eb647d\" (UID: \"19229e07-3c78-4369-998e-fc9e78eb647d\") " Dec 03 12:08:37 crc kubenswrapper[5052]: I1203 12:08:37.457209 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/141a49da-93b6-483b-a476-1d5f3efc29d1-internal-tls-certs\") pod \"141a49da-93b6-483b-a476-1d5f3efc29d1\" (UID: \"141a49da-93b6-483b-a476-1d5f3efc29d1\") " Dec 03 12:08:37 crc kubenswrapper[5052]: I1203 12:08:37.457238 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/141a49da-93b6-483b-a476-1d5f3efc29d1-public-tls-certs\") pod \"141a49da-93b6-483b-a476-1d5f3efc29d1\" (UID: \"141a49da-93b6-483b-a476-1d5f3efc29d1\") " Dec 03 12:08:37 crc kubenswrapper[5052]: I1203 12:08:37.457295 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/141a49da-93b6-483b-a476-1d5f3efc29d1-httpd-config\") pod \"141a49da-93b6-483b-a476-1d5f3efc29d1\" (UID: \"141a49da-93b6-483b-a476-1d5f3efc29d1\") " Dec 03 12:08:37 crc kubenswrapper[5052]: I1203 12:08:37.457344 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hdc52\" (UniqueName: \"kubernetes.io/projected/60f72892-a5aa-482f-aac3-5713ba028bc8-kube-api-access-hdc52\") pod \"60f72892-a5aa-482f-aac3-5713ba028bc8\" (UID: \"60f72892-a5aa-482f-aac3-5713ba028bc8\") " Dec 03 12:08:37 crc kubenswrapper[5052]: I1203 12:08:37.457375 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/60f72892-a5aa-482f-aac3-5713ba028bc8-etc-machine-id" (OuterVolumeSpecName: "etc-machine-id") pod "60f72892-a5aa-482f-aac3-5713ba028bc8" (UID: "60f72892-a5aa-482f-aac3-5713ba028bc8"). InnerVolumeSpecName "etc-machine-id". PluginName "kubernetes.io/host-path", VolumeGidValue "" Dec 03 12:08:37 crc kubenswrapper[5052]: I1203 12:08:37.457384 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/fad6f453-acf5-4f13-94d6-7d46e300f28c-combined-ca-bundle\") pod \"fad6f453-acf5-4f13-94d6-7d46e300f28c\" (UID: \"fad6f453-acf5-4f13-94d6-7d46e300f28c\") " Dec 03 12:08:37 crc kubenswrapper[5052]: I1203 12:08:37.457503 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/19229e07-3c78-4369-998e-fc9e78eb647d-run-httpd\") pod \"19229e07-3c78-4369-998e-fc9e78eb647d\" (UID: \"19229e07-3c78-4369-998e-fc9e78eb647d\") " Dec 03 12:08:37 crc kubenswrapper[5052]: I1203 12:08:37.457563 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/bbdcaf66-8813-4bed-9311-552fef62ac2d-config-data\") pod \"bbdcaf66-8813-4bed-9311-552fef62ac2d\" (UID: \"bbdcaf66-8813-4bed-9311-552fef62ac2d\") " Dec 03 12:08:37 crc kubenswrapper[5052]: I1203 12:08:37.457618 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/60f72892-a5aa-482f-aac3-5713ba028bc8-combined-ca-bundle\") pod \"60f72892-a5aa-482f-aac3-5713ba028bc8\" (UID: \"60f72892-a5aa-482f-aac3-5713ba028bc8\") " Dec 03 12:08:37 crc kubenswrapper[5052]: I1203 12:08:37.457641 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/141a49da-93b6-483b-a476-1d5f3efc29d1-combined-ca-bundle\") pod \"141a49da-93b6-483b-a476-1d5f3efc29d1\" (UID: \"141a49da-93b6-483b-a476-1d5f3efc29d1\") " Dec 03 12:08:37 crc kubenswrapper[5052]: I1203 12:08:37.457668 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/bbdcaf66-8813-4bed-9311-552fef62ac2d-kolla-config\") pod \"bbdcaf66-8813-4bed-9311-552fef62ac2d\" (UID: \"bbdcaf66-8813-4bed-9311-552fef62ac2d\") " Dec 03 12:08:37 crc kubenswrapper[5052]: I1203 12:08:37.457687 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"memcached-tls-certs\" (UniqueName: \"kubernetes.io/secret/bbdcaf66-8813-4bed-9311-552fef62ac2d-memcached-tls-certs\") pod \"bbdcaf66-8813-4bed-9311-552fef62ac2d\" (UID: \"bbdcaf66-8813-4bed-9311-552fef62ac2d\") " Dec 03 12:08:37 crc kubenswrapper[5052]: I1203 12:08:37.457705 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/fad6f453-acf5-4f13-94d6-7d46e300f28c-config\") pod \"fad6f453-acf5-4f13-94d6-7d46e300f28c\" (UID: \"fad6f453-acf5-4f13-94d6-7d46e300f28c\") " Dec 03 12:08:37 crc kubenswrapper[5052]: I1203 12:08:37.457725 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/60f72892-a5aa-482f-aac3-5713ba028bc8-scripts\") pod \"60f72892-a5aa-482f-aac3-5713ba028bc8\" (UID: \"60f72892-a5aa-482f-aac3-5713ba028bc8\") " Dec 03 12:08:37 crc kubenswrapper[5052]: I1203 12:08:37.457742 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/fad6f453-acf5-4f13-94d6-7d46e300f28c-scripts\") pod \"fad6f453-acf5-4f13-94d6-7d46e300f28c\" (UID: \"fad6f453-acf5-4f13-94d6-7d46e300f28c\") " Dec 03 12:08:37 crc kubenswrapper[5052]: I1203 12:08:37.457766 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/fad6f453-acf5-4f13-94d6-7d46e300f28c-metrics-certs-tls-certs\") pod \"fad6f453-acf5-4f13-94d6-7d46e300f28c\" (UID: \"fad6f453-acf5-4f13-94d6-7d46e300f28c\") " Dec 03 12:08:37 crc kubenswrapper[5052]: I1203 12:08:37.457798 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p47k8\" (UniqueName: \"kubernetes.io/projected/bbdcaf66-8813-4bed-9311-552fef62ac2d-kube-api-access-p47k8\") pod \"bbdcaf66-8813-4bed-9311-552fef62ac2d\" (UID: \"bbdcaf66-8813-4bed-9311-552fef62ac2d\") " Dec 03 12:08:37 crc kubenswrapper[5052]: I1203 12:08:37.457817 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/19229e07-3c78-4369-998e-fc9e78eb647d-combined-ca-bundle\") pod \"19229e07-3c78-4369-998e-fc9e78eb647d\" (UID: \"19229e07-3c78-4369-998e-fc9e78eb647d\") " Dec 03 12:08:37 crc kubenswrapper[5052]: I1203 12:08:37.457835 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/19229e07-3c78-4369-998e-fc9e78eb647d-sg-core-conf-yaml\") pod \"19229e07-3c78-4369-998e-fc9e78eb647d\" (UID: \"19229e07-3c78-4369-998e-fc9e78eb647d\") " Dec 03 12:08:37 crc kubenswrapper[5052]: I1203 12:08:37.457858 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovn-northd-tls-certs\" (UniqueName: \"kubernetes.io/secret/fad6f453-acf5-4f13-94d6-7d46e300f28c-ovn-northd-tls-certs\") pod \"fad6f453-acf5-4f13-94d6-7d46e300f28c\" (UID: \"fad6f453-acf5-4f13-94d6-7d46e300f28c\") " Dec 03 12:08:37 crc kubenswrapper[5052]: I1203 12:08:37.457873 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tmd66\" (UniqueName: \"kubernetes.io/projected/fad6f453-acf5-4f13-94d6-7d46e300f28c-kube-api-access-tmd66\") pod \"fad6f453-acf5-4f13-94d6-7d46e300f28c\" (UID: \"fad6f453-acf5-4f13-94d6-7d46e300f28c\") " Dec 03 12:08:37 crc kubenswrapper[5052]: I1203 12:08:37.457895 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/60f72892-a5aa-482f-aac3-5713ba028bc8-config-data-custom\") pod \"60f72892-a5aa-482f-aac3-5713ba028bc8\" (UID: \"60f72892-a5aa-482f-aac3-5713ba028bc8\") " Dec 03 12:08:37 crc kubenswrapper[5052]: I1203 12:08:37.457914 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7dmbd\" (UniqueName: \"kubernetes.io/projected/19229e07-3c78-4369-998e-fc9e78eb647d-kube-api-access-7dmbd\") pod \"19229e07-3c78-4369-998e-fc9e78eb647d\" (UID: \"19229e07-3c78-4369-998e-fc9e78eb647d\") " Dec 03 12:08:37 crc kubenswrapper[5052]: I1203 12:08:37.457938 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovn-rundir\" (UniqueName: \"kubernetes.io/empty-dir/fad6f453-acf5-4f13-94d6-7d46e300f28c-ovn-rundir\") pod \"fad6f453-acf5-4f13-94d6-7d46e300f28c\" (UID: \"fad6f453-acf5-4f13-94d6-7d46e300f28c\") " Dec 03 12:08:37 crc kubenswrapper[5052]: I1203 12:08:37.458727 5052 reconciler_common.go:293] "Volume detached for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/60f72892-a5aa-482f-aac3-5713ba028bc8-etc-machine-id\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:37 crc kubenswrapper[5052]: I1203 12:08:37.459061 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fad6f453-acf5-4f13-94d6-7d46e300f28c-ovn-rundir" (OuterVolumeSpecName: "ovn-rundir") pod "fad6f453-acf5-4f13-94d6-7d46e300f28c" (UID: "fad6f453-acf5-4f13-94d6-7d46e300f28c"). InnerVolumeSpecName "ovn-rundir". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:08:37 crc kubenswrapper[5052]: I1203 12:08:37.459714 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19229e07-3c78-4369-998e-fc9e78eb647d-log-httpd" (OuterVolumeSpecName: "log-httpd") pod "19229e07-3c78-4369-998e-fc9e78eb647d" (UID: "19229e07-3c78-4369-998e-fc9e78eb647d"). InnerVolumeSpecName "log-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:08:37 crc kubenswrapper[5052]: I1203 12:08:37.465673 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/141a49da-93b6-483b-a476-1d5f3efc29d1-kube-api-access-zmhzk" (OuterVolumeSpecName: "kube-api-access-zmhzk") pod "141a49da-93b6-483b-a476-1d5f3efc29d1" (UID: "141a49da-93b6-483b-a476-1d5f3efc29d1"). InnerVolumeSpecName "kube-api-access-zmhzk". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:08:37 crc kubenswrapper[5052]: I1203 12:08:37.468386 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fad6f453-acf5-4f13-94d6-7d46e300f28c-scripts" (OuterVolumeSpecName: "scripts") pod "fad6f453-acf5-4f13-94d6-7d46e300f28c" (UID: "fad6f453-acf5-4f13-94d6-7d46e300f28c"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:08:37 crc kubenswrapper[5052]: I1203 12:08:37.470925 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bbdcaf66-8813-4bed-9311-552fef62ac2d-kolla-config" (OuterVolumeSpecName: "kolla-config") pod "bbdcaf66-8813-4bed-9311-552fef62ac2d" (UID: "bbdcaf66-8813-4bed-9311-552fef62ac2d"). InnerVolumeSpecName "kolla-config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:08:37 crc kubenswrapper[5052]: I1203 12:08:37.471933 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/141a49da-93b6-483b-a476-1d5f3efc29d1-httpd-config" (OuterVolumeSpecName: "httpd-config") pod "141a49da-93b6-483b-a476-1d5f3efc29d1" (UID: "141a49da-93b6-483b-a476-1d5f3efc29d1"). InnerVolumeSpecName "httpd-config". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:08:37 crc kubenswrapper[5052]: I1203 12:08:37.472061 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fad6f453-acf5-4f13-94d6-7d46e300f28c-config" (OuterVolumeSpecName: "config") pod "fad6f453-acf5-4f13-94d6-7d46e300f28c" (UID: "fad6f453-acf5-4f13-94d6-7d46e300f28c"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:08:37 crc kubenswrapper[5052]: I1203 12:08:37.472533 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19229e07-3c78-4369-998e-fc9e78eb647d-run-httpd" (OuterVolumeSpecName: "run-httpd") pod "19229e07-3c78-4369-998e-fc9e78eb647d" (UID: "19229e07-3c78-4369-998e-fc9e78eb647d"). InnerVolumeSpecName "run-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:08:37 crc kubenswrapper[5052]: I1203 12:08:37.473020 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bbdcaf66-8813-4bed-9311-552fef62ac2d-config-data" (OuterVolumeSpecName: "config-data") pod "bbdcaf66-8813-4bed-9311-552fef62ac2d" (UID: "bbdcaf66-8813-4bed-9311-552fef62ac2d"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:08:37 crc kubenswrapper[5052]: I1203 12:08:37.484535 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/60f72892-a5aa-482f-aac3-5713ba028bc8-scripts" (OuterVolumeSpecName: "scripts") pod "60f72892-a5aa-482f-aac3-5713ba028bc8" (UID: "60f72892-a5aa-482f-aac3-5713ba028bc8"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:08:37 crc kubenswrapper[5052]: I1203 12:08:37.486748 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/60f72892-a5aa-482f-aac3-5713ba028bc8-config-data-custom" (OuterVolumeSpecName: "config-data-custom") pod "60f72892-a5aa-482f-aac3-5713ba028bc8" (UID: "60f72892-a5aa-482f-aac3-5713ba028bc8"). InnerVolumeSpecName "config-data-custom". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:08:37 crc kubenswrapper[5052]: I1203 12:08:37.488952 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/60f72892-a5aa-482f-aac3-5713ba028bc8-kube-api-access-hdc52" (OuterVolumeSpecName: "kube-api-access-hdc52") pod "60f72892-a5aa-482f-aac3-5713ba028bc8" (UID: "60f72892-a5aa-482f-aac3-5713ba028bc8"). InnerVolumeSpecName "kube-api-access-hdc52". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:08:37 crc kubenswrapper[5052]: I1203 12:08:37.491172 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bbdcaf66-8813-4bed-9311-552fef62ac2d-kube-api-access-p47k8" (OuterVolumeSpecName: "kube-api-access-p47k8") pod "bbdcaf66-8813-4bed-9311-552fef62ac2d" (UID: "bbdcaf66-8813-4bed-9311-552fef62ac2d"). InnerVolumeSpecName "kube-api-access-p47k8". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:08:37 crc kubenswrapper[5052]: I1203 12:08:37.494885 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fad6f453-acf5-4f13-94d6-7d46e300f28c-kube-api-access-tmd66" (OuterVolumeSpecName: "kube-api-access-tmd66") pod "fad6f453-acf5-4f13-94d6-7d46e300f28c" (UID: "fad6f453-acf5-4f13-94d6-7d46e300f28c"). InnerVolumeSpecName "kube-api-access-tmd66". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:08:37 crc kubenswrapper[5052]: I1203 12:08:37.502364 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/19229e07-3c78-4369-998e-fc9e78eb647d-scripts" (OuterVolumeSpecName: "scripts") pod "19229e07-3c78-4369-998e-fc9e78eb647d" (UID: "19229e07-3c78-4369-998e-fc9e78eb647d"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:08:37 crc kubenswrapper[5052]: I1203 12:08:37.507138 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/19229e07-3c78-4369-998e-fc9e78eb647d-kube-api-access-7dmbd" (OuterVolumeSpecName: "kube-api-access-7dmbd") pod "19229e07-3c78-4369-998e-fc9e78eb647d" (UID: "19229e07-3c78-4369-998e-fc9e78eb647d"). InnerVolumeSpecName "kube-api-access-7dmbd". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:08:37 crc kubenswrapper[5052]: I1203 12:08:37.559977 5052 scope.go:117] "RemoveContainer" containerID="e1e5f1e1f180364add520eba6060fbec83c2ab074ee509c5c280da93bca650db" Dec 03 12:08:37 crc kubenswrapper[5052]: I1203 12:08:37.559981 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fad6f453-acf5-4f13-94d6-7d46e300f28c-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "fad6f453-acf5-4f13-94d6-7d46e300f28c" (UID: "fad6f453-acf5-4f13-94d6-7d46e300f28c"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:08:37 crc kubenswrapper[5052]: E1203 12:08:37.563491 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/dns-swift-storage-0: configmap "dns-swift-storage-0" not found Dec 03 12:08:37 crc kubenswrapper[5052]: E1203 12:08:37.563571 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/77077fe5-d1ff-49c4-b049-3a0f2cee23b0-dns-swift-storage-0 podName:77077fe5-d1ff-49c4-b049-3a0f2cee23b0 nodeName:}" failed. No retries permitted until 2025-12-03 12:08:45.563550265 +0000 UTC m=+2916.151792029 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "dns-swift-storage-0" (UniqueName: "kubernetes.io/configmap/77077fe5-d1ff-49c4-b049-3a0f2cee23b0-dns-swift-storage-0") pod "dnsmasq-dnsmasq-85f8bc6fb5-b86k2" (UID: "77077fe5-d1ff-49c4-b049-3a0f2cee23b0") : configmap "dns-swift-storage-0" not found Dec 03 12:08:37 crc kubenswrapper[5052]: I1203 12:08:37.563651 5052 reconciler_common.go:293] "Volume detached for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/141a49da-93b6-483b-a476-1d5f3efc29d1-httpd-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:37 crc kubenswrapper[5052]: I1203 12:08:37.563706 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-hdc52\" (UniqueName: \"kubernetes.io/projected/60f72892-a5aa-482f-aac3-5713ba028bc8-kube-api-access-hdc52\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:37 crc kubenswrapper[5052]: I1203 12:08:37.563793 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/fad6f453-acf5-4f13-94d6-7d46e300f28c-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:37 crc kubenswrapper[5052]: I1203 12:08:37.563838 5052 reconciler_common.go:293] "Volume detached for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/19229e07-3c78-4369-998e-fc9e78eb647d-run-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:37 crc kubenswrapper[5052]: I1203 12:08:37.563909 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/bbdcaf66-8813-4bed-9311-552fef62ac2d-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:37 crc kubenswrapper[5052]: I1203 12:08:37.563925 5052 reconciler_common.go:293] "Volume detached for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/bbdcaf66-8813-4bed-9311-552fef62ac2d-kolla-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:37 crc kubenswrapper[5052]: I1203 12:08:37.563936 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/fad6f453-acf5-4f13-94d6-7d46e300f28c-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:37 crc kubenswrapper[5052]: I1203 12:08:37.563949 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/60f72892-a5aa-482f-aac3-5713ba028bc8-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:37 crc kubenswrapper[5052]: I1203 12:08:37.563960 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/fad6f453-acf5-4f13-94d6-7d46e300f28c-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:37 crc kubenswrapper[5052]: I1203 12:08:37.563972 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-p47k8\" (UniqueName: \"kubernetes.io/projected/bbdcaf66-8813-4bed-9311-552fef62ac2d-kube-api-access-p47k8\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:37 crc kubenswrapper[5052]: I1203 12:08:37.563985 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-tmd66\" (UniqueName: \"kubernetes.io/projected/fad6f453-acf5-4f13-94d6-7d46e300f28c-kube-api-access-tmd66\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:37 crc kubenswrapper[5052]: I1203 12:08:37.563996 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/60f72892-a5aa-482f-aac3-5713ba028bc8-config-data-custom\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:37 crc kubenswrapper[5052]: I1203 12:08:37.564008 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-7dmbd\" (UniqueName: \"kubernetes.io/projected/19229e07-3c78-4369-998e-fc9e78eb647d-kube-api-access-7dmbd\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:37 crc kubenswrapper[5052]: I1203 12:08:37.564019 5052 reconciler_common.go:293] "Volume detached for volume \"ovn-rundir\" (UniqueName: \"kubernetes.io/empty-dir/fad6f453-acf5-4f13-94d6-7d46e300f28c-ovn-rundir\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:37 crc kubenswrapper[5052]: I1203 12:08:37.564033 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-zmhzk\" (UniqueName: \"kubernetes.io/projected/141a49da-93b6-483b-a476-1d5f3efc29d1-kube-api-access-zmhzk\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:37 crc kubenswrapper[5052]: I1203 12:08:37.564045 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/19229e07-3c78-4369-998e-fc9e78eb647d-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:37 crc kubenswrapper[5052]: I1203 12:08:37.564057 5052 reconciler_common.go:293] "Volume detached for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/19229e07-3c78-4369-998e-fc9e78eb647d-log-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:37 crc kubenswrapper[5052]: I1203 12:08:37.645087 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/nova-metadata-0" podUID="b9efd251-95c2-4224-b318-a34279ee1017" containerName="nova-metadata-metadata" probeResult="failure" output="Get \"https://10.217.0.197:8775/\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Dec 03 12:08:37 crc kubenswrapper[5052]: I1203 12:08:37.645659 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/nova-metadata-0" podUID="b9efd251-95c2-4224-b318-a34279ee1017" containerName="nova-metadata-log" probeResult="failure" output="Get \"https://10.217.0.197:8775/\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Dec 03 12:08:37 crc kubenswrapper[5052]: I1203 12:08:37.661336 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bbdcaf66-8813-4bed-9311-552fef62ac2d-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "bbdcaf66-8813-4bed-9311-552fef62ac2d" (UID: "bbdcaf66-8813-4bed-9311-552fef62ac2d"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:08:37 crc kubenswrapper[5052]: I1203 12:08:37.665600 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/bbdcaf66-8813-4bed-9311-552fef62ac2d-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:37 crc kubenswrapper[5052]: I1203 12:08:37.671890 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/19229e07-3c78-4369-998e-fc9e78eb647d-ceilometer-tls-certs" (OuterVolumeSpecName: "ceilometer-tls-certs") pod "19229e07-3c78-4369-998e-fc9e78eb647d" (UID: "19229e07-3c78-4369-998e-fc9e78eb647d"). InnerVolumeSpecName "ceilometer-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:08:37 crc kubenswrapper[5052]: I1203 12:08:37.672883 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bbdcaf66-8813-4bed-9311-552fef62ac2d-memcached-tls-certs" (OuterVolumeSpecName: "memcached-tls-certs") pod "bbdcaf66-8813-4bed-9311-552fef62ac2d" (UID: "bbdcaf66-8813-4bed-9311-552fef62ac2d"). InnerVolumeSpecName "memcached-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:08:37 crc kubenswrapper[5052]: I1203 12:08:37.682842 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/19229e07-3c78-4369-998e-fc9e78eb647d-sg-core-conf-yaml" (OuterVolumeSpecName: "sg-core-conf-yaml") pod "19229e07-3c78-4369-998e-fc9e78eb647d" (UID: "19229e07-3c78-4369-998e-fc9e78eb647d"). InnerVolumeSpecName "sg-core-conf-yaml". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:08:37 crc kubenswrapper[5052]: I1203 12:08:37.685882 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/141a49da-93b6-483b-a476-1d5f3efc29d1-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "141a49da-93b6-483b-a476-1d5f3efc29d1" (UID: "141a49da-93b6-483b-a476-1d5f3efc29d1"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:08:37 crc kubenswrapper[5052]: I1203 12:08:37.686449 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="19d83b78-c03a-482a-909e-74bb991d974a" path="/var/lib/kubelet/pods/19d83b78-c03a-482a-909e-74bb991d974a/volumes" Dec 03 12:08:37 crc kubenswrapper[5052]: I1203 12:08:37.686912 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="246ced2f-39a7-496b-93cd-9f8ab898e2bb" path="/var/lib/kubelet/pods/246ced2f-39a7-496b-93cd-9f8ab898e2bb/volumes" Dec 03 12:08:37 crc kubenswrapper[5052]: I1203 12:08:37.687475 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2a25df1a-4f92-4ed6-9c34-2d6b0212593b" path="/var/lib/kubelet/pods/2a25df1a-4f92-4ed6-9c34-2d6b0212593b/volumes" Dec 03 12:08:37 crc kubenswrapper[5052]: I1203 12:08:37.688383 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2b7332a9-078d-4457-b136-899c4cdfaaae" path="/var/lib/kubelet/pods/2b7332a9-078d-4457-b136-899c4cdfaaae/volumes" Dec 03 12:08:37 crc kubenswrapper[5052]: I1203 12:08:37.689466 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="31e6a111-bd4a-45a5-95a0-a6b537added6" path="/var/lib/kubelet/pods/31e6a111-bd4a-45a5-95a0-a6b537added6/volumes" Dec 03 12:08:37 crc kubenswrapper[5052]: I1203 12:08:37.690263 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3af45ac5-7939-4525-891d-8ca1f5e697bf" path="/var/lib/kubelet/pods/3af45ac5-7939-4525-891d-8ca1f5e697bf/volumes" Dec 03 12:08:37 crc kubenswrapper[5052]: I1203 12:08:37.691303 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4741b42c-35a5-407d-b5cc-c7d47b07ce0b" path="/var/lib/kubelet/pods/4741b42c-35a5-407d-b5cc-c7d47b07ce0b/volumes" Dec 03 12:08:37 crc kubenswrapper[5052]: I1203 12:08:37.692824 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="53e80028-95b5-48c8-96ce-f6e40ed916cf" path="/var/lib/kubelet/pods/53e80028-95b5-48c8-96ce-f6e40ed916cf/volumes" Dec 03 12:08:37 crc kubenswrapper[5052]: I1203 12:08:37.693452 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="59c3e3b8-e2a6-4384-87f0-d1d7cbb4d738" path="/var/lib/kubelet/pods/59c3e3b8-e2a6-4384-87f0-d1d7cbb4d738/volumes" Dec 03 12:08:37 crc kubenswrapper[5052]: I1203 12:08:37.694032 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="67b28447-93fa-45c3-8c94-9dba5eba8f94" path="/var/lib/kubelet/pods/67b28447-93fa-45c3-8c94-9dba5eba8f94/volumes" Dec 03 12:08:37 crc kubenswrapper[5052]: I1203 12:08:37.694711 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="77f55a4f-32ec-4c57-82bc-ff0dd0aac965" path="/var/lib/kubelet/pods/77f55a4f-32ec-4c57-82bc-ff0dd0aac965/volumes" Dec 03 12:08:37 crc kubenswrapper[5052]: I1203 12:08:37.695960 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="79e77d8b-2481-4cbf-875f-cb626cf89463" path="/var/lib/kubelet/pods/79e77d8b-2481-4cbf-875f-cb626cf89463/volumes" Dec 03 12:08:37 crc kubenswrapper[5052]: I1203 12:08:37.696541 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8ba21481-e9f8-41f3-80e6-c96f860b7771" path="/var/lib/kubelet/pods/8ba21481-e9f8-41f3-80e6-c96f860b7771/volumes" Dec 03 12:08:37 crc kubenswrapper[5052]: I1203 12:08:37.699443 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="90d21e8c-a52c-4109-811b-7d113748cea7" path="/var/lib/kubelet/pods/90d21e8c-a52c-4109-811b-7d113748cea7/volumes" Dec 03 12:08:37 crc kubenswrapper[5052]: I1203 12:08:37.700744 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c2d35861-9e4b-4f53-b841-e18b26d575cc" path="/var/lib/kubelet/pods/c2d35861-9e4b-4f53-b841-e18b26d575cc/volumes" Dec 03 12:08:37 crc kubenswrapper[5052]: I1203 12:08:37.702051 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c665a3eb-31e2-4ac4-99c8-02c8d262d9c9" path="/var/lib/kubelet/pods/c665a3eb-31e2-4ac4-99c8-02c8d262d9c9/volumes" Dec 03 12:08:37 crc kubenswrapper[5052]: I1203 12:08:37.722902 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fad6f453-acf5-4f13-94d6-7d46e300f28c-metrics-certs-tls-certs" (OuterVolumeSpecName: "metrics-certs-tls-certs") pod "fad6f453-acf5-4f13-94d6-7d46e300f28c" (UID: "fad6f453-acf5-4f13-94d6-7d46e300f28c"). InnerVolumeSpecName "metrics-certs-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:08:37 crc kubenswrapper[5052]: I1203 12:08:37.729758 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/60f72892-a5aa-482f-aac3-5713ba028bc8-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "60f72892-a5aa-482f-aac3-5713ba028bc8" (UID: "60f72892-a5aa-482f-aac3-5713ba028bc8"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:08:37 crc kubenswrapper[5052]: I1203 12:08:37.734678 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/141a49da-93b6-483b-a476-1d5f3efc29d1-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "141a49da-93b6-483b-a476-1d5f3efc29d1" (UID: "141a49da-93b6-483b-a476-1d5f3efc29d1"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:08:37 crc kubenswrapper[5052]: I1203 12:08:37.735537 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/19229e07-3c78-4369-998e-fc9e78eb647d-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "19229e07-3c78-4369-998e-fc9e78eb647d" (UID: "19229e07-3c78-4369-998e-fc9e78eb647d"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:08:37 crc kubenswrapper[5052]: I1203 12:08:37.737325 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/141a49da-93b6-483b-a476-1d5f3efc29d1-config" (OuterVolumeSpecName: "config") pod "141a49da-93b6-483b-a476-1d5f3efc29d1" (UID: "141a49da-93b6-483b-a476-1d5f3efc29d1"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:08:37 crc kubenswrapper[5052]: I1203 12:08:37.748781 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/19229e07-3c78-4369-998e-fc9e78eb647d-config-data" (OuterVolumeSpecName: "config-data") pod "19229e07-3c78-4369-998e-fc9e78eb647d" (UID: "19229e07-3c78-4369-998e-fc9e78eb647d"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:08:37 crc kubenswrapper[5052]: I1203 12:08:37.748920 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/141a49da-93b6-483b-a476-1d5f3efc29d1-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "141a49da-93b6-483b-a476-1d5f3efc29d1" (UID: "141a49da-93b6-483b-a476-1d5f3efc29d1"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:08:37 crc kubenswrapper[5052]: I1203 12:08:37.748797 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/141a49da-93b6-483b-a476-1d5f3efc29d1-ovndb-tls-certs" (OuterVolumeSpecName: "ovndb-tls-certs") pod "141a49da-93b6-483b-a476-1d5f3efc29d1" (UID: "141a49da-93b6-483b-a476-1d5f3efc29d1"). InnerVolumeSpecName "ovndb-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:08:37 crc kubenswrapper[5052]: I1203 12:08:37.768141 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/60f72892-a5aa-482f-aac3-5713ba028bc8-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:37 crc kubenswrapper[5052]: I1203 12:08:37.768386 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/141a49da-93b6-483b-a476-1d5f3efc29d1-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:37 crc kubenswrapper[5052]: I1203 12:08:37.769140 5052 reconciler_common.go:293] "Volume detached for volume \"memcached-tls-certs\" (UniqueName: \"kubernetes.io/secret/bbdcaf66-8813-4bed-9311-552fef62ac2d-memcached-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:37 crc kubenswrapper[5052]: I1203 12:08:37.769313 5052 reconciler_common.go:293] "Volume detached for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/fad6f453-acf5-4f13-94d6-7d46e300f28c-metrics-certs-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:37 crc kubenswrapper[5052]: I1203 12:08:37.769420 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/19229e07-3c78-4369-998e-fc9e78eb647d-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:37 crc kubenswrapper[5052]: I1203 12:08:37.769536 5052 reconciler_common.go:293] "Volume detached for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/19229e07-3c78-4369-998e-fc9e78eb647d-sg-core-conf-yaml\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:37 crc kubenswrapper[5052]: I1203 12:08:37.769621 5052 reconciler_common.go:293] "Volume detached for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/19229e07-3c78-4369-998e-fc9e78eb647d-ceilometer-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:37 crc kubenswrapper[5052]: I1203 12:08:37.769710 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/19229e07-3c78-4369-998e-fc9e78eb647d-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:37 crc kubenswrapper[5052]: I1203 12:08:37.769792 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/141a49da-93b6-483b-a476-1d5f3efc29d1-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:37 crc kubenswrapper[5052]: I1203 12:08:37.769869 5052 reconciler_common.go:293] "Volume detached for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/141a49da-93b6-483b-a476-1d5f3efc29d1-ovndb-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:37 crc kubenswrapper[5052]: I1203 12:08:37.769958 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/141a49da-93b6-483b-a476-1d5f3efc29d1-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:37 crc kubenswrapper[5052]: I1203 12:08:37.770035 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/141a49da-93b6-483b-a476-1d5f3efc29d1-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:37 crc kubenswrapper[5052]: I1203 12:08:37.770149 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fad6f453-acf5-4f13-94d6-7d46e300f28c-ovn-northd-tls-certs" (OuterVolumeSpecName: "ovn-northd-tls-certs") pod "fad6f453-acf5-4f13-94d6-7d46e300f28c" (UID: "fad6f453-acf5-4f13-94d6-7d46e300f28c"). InnerVolumeSpecName "ovn-northd-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:08:37 crc kubenswrapper[5052]: I1203 12:08:37.820063 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/60f72892-a5aa-482f-aac3-5713ba028bc8-config-data" (OuterVolumeSpecName: "config-data") pod "60f72892-a5aa-482f-aac3-5713ba028bc8" (UID: "60f72892-a5aa-482f-aac3-5713ba028bc8"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:08:37 crc kubenswrapper[5052]: I1203 12:08:37.875208 5052 reconciler_common.go:293] "Volume detached for volume \"ovn-northd-tls-certs\" (UniqueName: \"kubernetes.io/secret/fad6f453-acf5-4f13-94d6-7d46e300f28c-ovn-northd-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:37 crc kubenswrapper[5052]: I1203 12:08:37.875251 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/60f72892-a5aa-482f-aac3-5713ba028bc8-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.124605 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/openstack-cell1-galera-0" podUID="3b8ce5b7-67a1-48ef-892f-7408010a84f0" containerName="galera" containerID="cri-o://5a5421259d7095f5c7b35c6d5fcacfbf43056034387141eb0830f2281a204092" gracePeriod=20 Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.159665 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-kcb42" Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.208742 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/openstack-galera-1" podUID="6b92f88e-1325-4c3d-8d4e-4089a9033775" containerName="galera" containerID="cri-o://527038c3f4433583ba7d8bdd06bfde009abf48c5c09978d5bdd3df6912110859" gracePeriod=26 Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.227463 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican5c55-account-delete-zttb9" Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.247399 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance8fcf-account-delete-4xhcc" event={"ID":"2f3b4f6a-5f11-4039-a3bd-7b553ab5684d","Type":"ContainerDied","Data":"9276b3afe43ed2cac6cf8ba87bf8a179c142f8c59beb0358ba1e9657f5941a71"} Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.247489 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9276b3afe43ed2cac6cf8ba87bf8a179c142f8c59beb0358ba1e9657f5941a71" Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.255888 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/novacell0d294-account-delete-b64wn" Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.257819 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/barbican-api-d978c699b-kzzst" podUID="de04776b-ea02-40d9-8459-0a519c8c64ad" containerName="barbican-api-log" probeResult="failure" output="Get \"https://10.217.0.156:9311/healthcheck\": dial tcp 10.217.0.156:9311: i/o timeout" Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.258578 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/barbican-api-d978c699b-kzzst" podUID="de04776b-ea02-40d9-8459-0a519c8c64ad" containerName="barbican-api" probeResult="failure" output="Get \"https://10.217.0.156:9311/healthcheck\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.267002 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutrond5c4-account-delete-zj77r" Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.267181 5052 generic.go:334] "Generic (PLEG): container finished" podID="06bfd2db-abe0-4b4d-a110-94a6a1e02211" containerID="61952ac0175a2f78cbd3919ba15907c05a42be11756520e4a28f961c6f3c039e" exitCode=0 Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.267247 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-cell1-server-2" event={"ID":"06bfd2db-abe0-4b4d-a110-94a6a1e02211","Type":"ContainerDied","Data":"61952ac0175a2f78cbd3919ba15907c05a42be11756520e4a28f961c6f3c039e"} Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.270108 5052 scope.go:117] "RemoveContainer" containerID="f6003a5dd700c21fc31c9193f8092e42fdfba4431f6f7eb85497fdf04c513ed1" Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.270218 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-85f8bc6fb5-b86k2"] Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.270561 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-85f8bc6fb5-b86k2" podUID="77077fe5-d1ff-49c4-b049-3a0f2cee23b0" containerName="dnsmasq-dns" containerID="cri-o://bc3186646511337244c91e6de61ef79f4b96f53697165b0fe223dd0c36cfeadf" gracePeriod=10 Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.292418 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/1bc7c8b1-cc0d-4611-b335-01c2b91713e2-operator-scripts\") pod \"1bc7c8b1-cc0d-4611-b335-01c2b91713e2\" (UID: \"1bc7c8b1-cc0d-4611-b335-01c2b91713e2\") " Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.292649 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-drvct\" (UniqueName: \"kubernetes.io/projected/1bc7c8b1-cc0d-4611-b335-01c2b91713e2-kube-api-access-drvct\") pod \"1bc7c8b1-cc0d-4611-b335-01c2b91713e2\" (UID: \"1bc7c8b1-cc0d-4611-b335-01c2b91713e2\") " Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.292968 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1bc7c8b1-cc0d-4611-b335-01c2b91713e2-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "1bc7c8b1-cc0d-4611-b335-01c2b91713e2" (UID: "1bc7c8b1-cc0d-4611-b335-01c2b91713e2"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.293455 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-6c6748d95d-66jbh" Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.293881 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/1bc7c8b1-cc0d-4611-b335-01c2b91713e2-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.307991 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1bc7c8b1-cc0d-4611-b335-01c2b91713e2-kube-api-access-drvct" (OuterVolumeSpecName: "kube-api-access-drvct") pod "1bc7c8b1-cc0d-4611-b335-01c2b91713e2" (UID: "1bc7c8b1-cc0d-4611-b335-01c2b91713e2"). InnerVolumeSpecName "kube-api-access-drvct". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.310222 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"19229e07-3c78-4369-998e-fc9e78eb647d","Type":"ContainerDied","Data":"f75e647877896e217d788f87299009cfac0175499a1272dc8466684d66e3648d"} Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.310321 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/novaapi48e2-account-delete-kqhz7" Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.310336 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.312956 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican5c55-account-delete-zttb9" Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.313036 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican5c55-account-delete-zttb9" event={"ID":"1bc7c8b1-cc0d-4611-b335-01c2b91713e2","Type":"ContainerDied","Data":"35532bae8900df426551c12f66e807c943adb4ee0815e79acd82f5019c0c99bb"} Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.337299 5052 scope.go:117] "RemoveContainer" containerID="265855fad894b2f79c87685abb31a04136664b6466e2b8a541217f1335a297fc" Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.346623 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"60f72892-a5aa-482f-aac3-5713ba028bc8","Type":"ContainerDied","Data":"c31a05c23052597025fe3e6544d6ed87146c1a6be0e31012fc69ff2a6f8fa306"} Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.346731 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.351831 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement3c40-account-delete-8qsdr" Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.353266 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/novacell0d294-account-delete-b64wn" event={"ID":"030d4f76-eb30-4c44-b782-fa7147febf41","Type":"ContainerDied","Data":"d84a227d521e482f2af3610d890a33ba5ee3263757d3a6f6e31b17331a144841"} Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.354803 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/novacell0d294-account-delete-b64wn" Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.357806 5052 generic.go:334] "Generic (PLEG): container finished" podID="7901ce85-4e6f-4e8b-86a6-f50a85803637" containerID="88ec697999833a01b97e7879439a3382717780668a2e4ef174d64567bf8d897a" exitCode=0 Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.357872 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-server-1" event={"ID":"7901ce85-4e6f-4e8b-86a6-f50a85803637","Type":"ContainerDied","Data":"88ec697999833a01b97e7879439a3382717780668a2e4ef174d64567bf8d897a"} Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.363632 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutrond5c4-account-delete-zj77r" Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.363706 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutrond5c4-account-delete-zj77r" event={"ID":"9e6af047-b7ee-459a-b362-da5f777d0fa0","Type":"ContainerDied","Data":"cfc37f2b6a6dfb5b579ba00eaffe633f2160ebe7c26d6474081a194e015a503c"} Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.376489 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-5d6b56fbdb-ht2b8" event={"ID":"141a49da-93b6-483b-a476-1d5f3efc29d1","Type":"ContainerDied","Data":"848ae10ea79a99f5a6ff08cc94ca0107790699ed53098696b686a451c01bd64f"} Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.376731 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-5d6b56fbdb-ht2b8" Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.395887 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c2470da9-4c96-44ac-834d-2b560785135f-combined-ca-bundle\") pod \"c2470da9-4c96-44ac-834d-2b560785135f\" (UID: \"c2470da9-4c96-44ac-834d-2b560785135f\") " Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.395938 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/9e6af047-b7ee-459a-b362-da5f777d0fa0-operator-scripts\") pod \"9e6af047-b7ee-459a-b362-da5f777d0fa0\" (UID: \"9e6af047-b7ee-459a-b362-da5f777d0fa0\") " Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.395969 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/c2470da9-4c96-44ac-834d-2b560785135f-config-data\") pod \"c2470da9-4c96-44ac-834d-2b560785135f\" (UID: \"c2470da9-4c96-44ac-834d-2b560785135f\") " Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.395993 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6t26c\" (UniqueName: \"kubernetes.io/projected/030d4f76-eb30-4c44-b782-fa7147febf41-kube-api-access-6t26c\") pod \"030d4f76-eb30-4c44-b782-fa7147febf41\" (UID: \"030d4f76-eb30-4c44-b782-fa7147febf41\") " Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.396034 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/c2470da9-4c96-44ac-834d-2b560785135f-fernet-keys\") pod \"c2470da9-4c96-44ac-834d-2b560785135f\" (UID: \"c2470da9-4c96-44ac-834d-2b560785135f\") " Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.396082 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/c2470da9-4c96-44ac-834d-2b560785135f-scripts\") pod \"c2470da9-4c96-44ac-834d-2b560785135f\" (UID: \"c2470da9-4c96-44ac-834d-2b560785135f\") " Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.396127 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2j7sw\" (UniqueName: \"kubernetes.io/projected/70122e67-6175-4a14-a120-6a2731c5dcb1-kube-api-access-2j7sw\") pod \"70122e67-6175-4a14-a120-6a2731c5dcb1\" (UID: \"70122e67-6175-4a14-a120-6a2731c5dcb1\") " Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.396187 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/c2470da9-4c96-44ac-834d-2b560785135f-credential-keys\") pod \"c2470da9-4c96-44ac-834d-2b560785135f\" (UID: \"c2470da9-4c96-44ac-834d-2b560785135f\") " Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.396203 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/030d4f76-eb30-4c44-b782-fa7147febf41-operator-scripts\") pod \"030d4f76-eb30-4c44-b782-fa7147febf41\" (UID: \"030d4f76-eb30-4c44-b782-fa7147febf41\") " Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.396224 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/70122e67-6175-4a14-a120-6a2731c5dcb1-operator-scripts\") pod \"70122e67-6175-4a14-a120-6a2731c5dcb1\" (UID: \"70122e67-6175-4a14-a120-6a2731c5dcb1\") " Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.396286 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hkwq9\" (UniqueName: \"kubernetes.io/projected/9e6af047-b7ee-459a-b362-da5f777d0fa0-kube-api-access-hkwq9\") pod \"9e6af047-b7ee-459a-b362-da5f777d0fa0\" (UID: \"9e6af047-b7ee-459a-b362-da5f777d0fa0\") " Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.396306 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7k54s\" (UniqueName: \"kubernetes.io/projected/c2470da9-4c96-44ac-834d-2b560785135f-kube-api-access-7k54s\") pod \"c2470da9-4c96-44ac-834d-2b560785135f\" (UID: \"c2470da9-4c96-44ac-834d-2b560785135f\") " Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.396335 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/c2470da9-4c96-44ac-834d-2b560785135f-internal-tls-certs\") pod \"c2470da9-4c96-44ac-834d-2b560785135f\" (UID: \"c2470da9-4c96-44ac-834d-2b560785135f\") " Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.396360 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/c2470da9-4c96-44ac-834d-2b560785135f-public-tls-certs\") pod \"c2470da9-4c96-44ac-834d-2b560785135f\" (UID: \"c2470da9-4c96-44ac-834d-2b560785135f\") " Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.396845 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-drvct\" (UniqueName: \"kubernetes.io/projected/1bc7c8b1-cc0d-4611-b335-01c2b91713e2-kube-api-access-drvct\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.400768 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/70122e67-6175-4a14-a120-6a2731c5dcb1-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "70122e67-6175-4a14-a120-6a2731c5dcb1" (UID: "70122e67-6175-4a14-a120-6a2731c5dcb1"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.404952 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c2470da9-4c96-44ac-834d-2b560785135f-credential-keys" (OuterVolumeSpecName: "credential-keys") pod "c2470da9-4c96-44ac-834d-2b560785135f" (UID: "c2470da9-4c96-44ac-834d-2b560785135f"). InnerVolumeSpecName "credential-keys". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.405338 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/030d4f76-eb30-4c44-b782-fa7147febf41-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "030d4f76-eb30-4c44-b782-fa7147febf41" (UID: "030d4f76-eb30-4c44-b782-fa7147febf41"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.410045 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c2470da9-4c96-44ac-834d-2b560785135f-fernet-keys" (OuterVolumeSpecName: "fernet-keys") pod "c2470da9-4c96-44ac-834d-2b560785135f" (UID: "c2470da9-4c96-44ac-834d-2b560785135f"). InnerVolumeSpecName "fernet-keys". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.411372 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/030d4f76-eb30-4c44-b782-fa7147febf41-kube-api-access-6t26c" (OuterVolumeSpecName: "kube-api-access-6t26c") pod "030d4f76-eb30-4c44-b782-fa7147febf41" (UID: "030d4f76-eb30-4c44-b782-fa7147febf41"). InnerVolumeSpecName "kube-api-access-6t26c". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.412677 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c2470da9-4c96-44ac-834d-2b560785135f-kube-api-access-7k54s" (OuterVolumeSpecName: "kube-api-access-7k54s") pod "c2470da9-4c96-44ac-834d-2b560785135f" (UID: "c2470da9-4c96-44ac-834d-2b560785135f"). InnerVolumeSpecName "kube-api-access-7k54s". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.415635 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9e6af047-b7ee-459a-b362-da5f777d0fa0-kube-api-access-hkwq9" (OuterVolumeSpecName: "kube-api-access-hkwq9") pod "9e6af047-b7ee-459a-b362-da5f777d0fa0" (UID: "9e6af047-b7ee-459a-b362-da5f777d0fa0"). InnerVolumeSpecName "kube-api-access-hkwq9". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.419674 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/70122e67-6175-4a14-a120-6a2731c5dcb1-kube-api-access-2j7sw" (OuterVolumeSpecName: "kube-api-access-2j7sw") pod "70122e67-6175-4a14-a120-6a2731c5dcb1" (UID: "70122e67-6175-4a14-a120-6a2731c5dcb1"). InnerVolumeSpecName "kube-api-access-2j7sw". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.425816 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9e6af047-b7ee-459a-b362-da5f777d0fa0-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "9e6af047-b7ee-459a-b362-da5f777d0fa0" (UID: "9e6af047-b7ee-459a-b362-da5f777d0fa0"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.429622 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c2470da9-4c96-44ac-834d-2b560785135f-scripts" (OuterVolumeSpecName: "scripts") pod "c2470da9-4c96-44ac-834d-2b560785135f" (UID: "c2470da9-4c96-44ac-834d-2b560785135f"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.453296 5052 generic.go:334] "Generic (PLEG): container finished" podID="f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3" containerID="ae74ed54ba186a0e9a780dc48252d0e6409d74b51697be203e1eefe16b5793fe" exitCode=0 Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.454949 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-cell1-server-1" event={"ID":"f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3","Type":"ContainerDied","Data":"ae74ed54ba186a0e9a780dc48252d0e6409d74b51697be203e1eefe16b5793fe"} Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.468188 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c2470da9-4c96-44ac-834d-2b560785135f-config-data" (OuterVolumeSpecName: "config-data") pod "c2470da9-4c96-44ac-834d-2b560785135f" (UID: "c2470da9-4c96-44ac-834d-2b560785135f"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.471015 5052 scope.go:117] "RemoveContainer" containerID="19566104651a9853da1345044a96ed40b73615a3c7ab75d772d57f5e5fdc9b84" Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.485704 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c2470da9-4c96-44ac-834d-2b560785135f-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "c2470da9-4c96-44ac-834d-2b560785135f" (UID: "c2470da9-4c96-44ac-834d-2b560785135f"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.491485 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cindere6fc-account-delete-z7w6l" event={"ID":"89ee1bab-2c68-4fc4-9171-a7330922faf6","Type":"ContainerDied","Data":"c304cae678688e4fe4a4d9700dc6cb4c64dcdaf312fab8bfbf39b2457db150ce"} Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.491544 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c304cae678688e4fe4a4d9700dc6cb4c64dcdaf312fab8bfbf39b2457db150ce" Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.491663 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance8fcf-account-delete-4xhcc" Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.501878 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cindere6fc-account-delete-z7w6l" Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.502803 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6bm6b\" (UniqueName: \"kubernetes.io/projected/aaa3db64-a10b-45ea-9e25-de4916f30617-kube-api-access-6bm6b\") pod \"aaa3db64-a10b-45ea-9e25-de4916f30617\" (UID: \"aaa3db64-a10b-45ea-9e25-de4916f30617\") " Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.503053 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/aaa3db64-a10b-45ea-9e25-de4916f30617-operator-scripts\") pod \"aaa3db64-a10b-45ea-9e25-de4916f30617\" (UID: \"aaa3db64-a10b-45ea-9e25-de4916f30617\") " Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.512134 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aaa3db64-a10b-45ea-9e25-de4916f30617-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "aaa3db64-a10b-45ea-9e25-de4916f30617" (UID: "aaa3db64-a10b-45ea-9e25-de4916f30617"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.512344 5052 generic.go:334] "Generic (PLEG): container finished" podID="7b4f6078-41a3-433d-acaf-f2d67391ab6e" containerID="aad64a682ded16a9f4098098d689931d1934e194e45eecdc993e62379504781b" exitCode=0 Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.512352 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/c2470da9-4c96-44ac-834d-2b560785135f-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.512397 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-6t26c\" (UniqueName: \"kubernetes.io/projected/030d4f76-eb30-4c44-b782-fa7147febf41-kube-api-access-6t26c\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.512412 5052 reconciler_common.go:293] "Volume detached for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/c2470da9-4c96-44ac-834d-2b560785135f-fernet-keys\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.512424 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/c2470da9-4c96-44ac-834d-2b560785135f-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.512435 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-2j7sw\" (UniqueName: \"kubernetes.io/projected/70122e67-6175-4a14-a120-6a2731c5dcb1-kube-api-access-2j7sw\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.512447 5052 reconciler_common.go:293] "Volume detached for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/c2470da9-4c96-44ac-834d-2b560785135f-credential-keys\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.512459 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/030d4f76-eb30-4c44-b782-fa7147febf41-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.512471 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/70122e67-6175-4a14-a120-6a2731c5dcb1-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.512482 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/aaa3db64-a10b-45ea-9e25-de4916f30617-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.512493 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-hkwq9\" (UniqueName: \"kubernetes.io/projected/9e6af047-b7ee-459a-b362-da5f777d0fa0-kube-api-access-hkwq9\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.512504 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-7k54s\" (UniqueName: \"kubernetes.io/projected/c2470da9-4c96-44ac-834d-2b560785135f-kube-api-access-7k54s\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.512532 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c2470da9-4c96-44ac-834d-2b560785135f-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.512542 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/9e6af047-b7ee-459a-b362-da5f777d0fa0-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.512410 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-server-2" event={"ID":"7b4f6078-41a3-433d-acaf-f2d67391ab6e","Type":"ContainerDied","Data":"aad64a682ded16a9f4098098d689931d1934e194e45eecdc993e62379504781b"} Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.518350 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aaa3db64-a10b-45ea-9e25-de4916f30617-kube-api-access-6bm6b" (OuterVolumeSpecName: "kube-api-access-6bm6b") pod "aaa3db64-a10b-45ea-9e25-de4916f30617" (UID: "aaa3db64-a10b-45ea-9e25-de4916f30617"). InnerVolumeSpecName "kube-api-access-6bm6b". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.535297 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/novaapi48e2-account-delete-kqhz7" event={"ID":"70122e67-6175-4a14-a120-6a2731c5dcb1","Type":"ContainerDied","Data":"fb0cac54ec9d6c243455911815c99ff48403152f29b6f9186659e82c363412ff"} Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.535401 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/novaapi48e2-account-delete-kqhz7" Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.540948 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c2470da9-4c96-44ac-834d-2b560785135f-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "c2470da9-4c96-44ac-834d-2b560785135f" (UID: "c2470da9-4c96-44ac-834d-2b560785135f"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.542009 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c2470da9-4c96-44ac-834d-2b560785135f-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "c2470da9-4c96-44ac-834d-2b560785135f" (UID: "c2470da9-4c96-44ac-834d-2b560785135f"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.550990 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement3c40-account-delete-8qsdr" event={"ID":"aaa3db64-a10b-45ea-9e25-de4916f30617","Type":"ContainerDied","Data":"0a99a26443e4ad86179f2230393e6f19e8d1ac332ad1a9478891666bf36d948e"} Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.551086 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement3c40-account-delete-8qsdr" Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.562716 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-6c6748d95d-66jbh" event={"ID":"c2470da9-4c96-44ac-834d-2b560785135f","Type":"ContainerDied","Data":"2a62c47a82e45e290e6522355bc4c339d425819f3278f37e230418c1c7397913"} Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.562957 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-6c6748d95d-66jbh" Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.584931 5052 generic.go:334] "Generic (PLEG): container finished" podID="2397d896-0d11-4656-a336-eac8ec9fe7f9" containerID="16dd1c84a3d482ce5adc97f8fdc99d27080aa0bd2d9fba51e986dc52caa0daf6" exitCode=0 Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.585116 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/memcached-0" Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.585868 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-server-0" event={"ID":"2397d896-0d11-4656-a336-eac8ec9fe7f9","Type":"ContainerDied","Data":"16dd1c84a3d482ce5adc97f8fdc99d27080aa0bd2d9fba51e986dc52caa0daf6"} Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.586046 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.588852 5052 scope.go:117] "RemoveContainer" containerID="bc70dbc3ccd05a052e7fc3b5729f64a5f4c046c9a200d68d09f9e35887ba8604" Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.589031 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/rabbitmq-cell1-server-2" Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.614726 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vjnlh\" (UniqueName: \"kubernetes.io/projected/2f3b4f6a-5f11-4039-a3bd-7b553ab5684d-kube-api-access-vjnlh\") pod \"2f3b4f6a-5f11-4039-a3bd-7b553ab5684d\" (UID: \"2f3b4f6a-5f11-4039-a3bd-7b553ab5684d\") " Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.615035 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/89ee1bab-2c68-4fc4-9171-a7330922faf6-operator-scripts\") pod \"89ee1bab-2c68-4fc4-9171-a7330922faf6\" (UID: \"89ee1bab-2c68-4fc4-9171-a7330922faf6\") " Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.615149 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/2f3b4f6a-5f11-4039-a3bd-7b553ab5684d-operator-scripts\") pod \"2f3b4f6a-5f11-4039-a3bd-7b553ab5684d\" (UID: \"2f3b4f6a-5f11-4039-a3bd-7b553ab5684d\") " Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.615216 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d5spc\" (UniqueName: \"kubernetes.io/projected/89ee1bab-2c68-4fc4-9171-a7330922faf6-kube-api-access-d5spc\") pod \"89ee1bab-2c68-4fc4-9171-a7330922faf6\" (UID: \"89ee1bab-2c68-4fc4-9171-a7330922faf6\") " Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.615885 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/c2470da9-4c96-44ac-834d-2b560785135f-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.615961 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-6bm6b\" (UniqueName: \"kubernetes.io/projected/aaa3db64-a10b-45ea-9e25-de4916f30617-kube-api-access-6bm6b\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.616036 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/c2470da9-4c96-44ac-834d-2b560785135f-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.621414 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.622667 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2f3b4f6a-5f11-4039-a3bd-7b553ab5684d-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "2f3b4f6a-5f11-4039-a3bd-7b553ab5684d" (UID: "2f3b4f6a-5f11-4039-a3bd-7b553ab5684d"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.624832 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/89ee1bab-2c68-4fc4-9171-a7330922faf6-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "89ee1bab-2c68-4fc4-9171-a7330922faf6" (UID: "89ee1bab-2c68-4fc4-9171-a7330922faf6"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.631223 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2f3b4f6a-5f11-4039-a3bd-7b553ab5684d-kube-api-access-vjnlh" (OuterVolumeSpecName: "kube-api-access-vjnlh") pod "2f3b4f6a-5f11-4039-a3bd-7b553ab5684d" (UID: "2f3b4f6a-5f11-4039-a3bd-7b553ab5684d"). InnerVolumeSpecName "kube-api-access-vjnlh". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.633722 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/89ee1bab-2c68-4fc4-9171-a7330922faf6-kube-api-access-d5spc" (OuterVolumeSpecName: "kube-api-access-d5spc") pod "89ee1bab-2c68-4fc4-9171-a7330922faf6" (UID: "89ee1bab-2c68-4fc4-9171-a7330922faf6"). InnerVolumeSpecName "kube-api-access-d5spc". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.641916 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.668734 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.676937 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.685435 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/neutron-5d6b56fbdb-ht2b8"] Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.701296 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/neutron-5d6b56fbdb-ht2b8"] Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.716603 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"rabbitmq-erlang-cookie\" (UniqueName: \"kubernetes.io/empty-dir/06bfd2db-abe0-4b4d-a110-94a6a1e02211-rabbitmq-erlang-cookie\") pod \"06bfd2db-abe0-4b4d-a110-94a6a1e02211\" (UID: \"06bfd2db-abe0-4b4d-a110-94a6a1e02211\") " Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.716670 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/06bfd2db-abe0-4b4d-a110-94a6a1e02211-config-data\") pod \"06bfd2db-abe0-4b4d-a110-94a6a1e02211\" (UID: \"06bfd2db-abe0-4b4d-a110-94a6a1e02211\") " Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.716710 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"rabbitmq-plugins\" (UniqueName: \"kubernetes.io/empty-dir/06bfd2db-abe0-4b4d-a110-94a6a1e02211-rabbitmq-plugins\") pod \"06bfd2db-abe0-4b4d-a110-94a6a1e02211\" (UID: \"06bfd2db-abe0-4b4d-a110-94a6a1e02211\") " Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.716750 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"rabbitmq-confd\" (UniqueName: \"kubernetes.io/projected/06bfd2db-abe0-4b4d-a110-94a6a1e02211-rabbitmq-confd\") pod \"06bfd2db-abe0-4b4d-a110-94a6a1e02211\" (UID: \"06bfd2db-abe0-4b4d-a110-94a6a1e02211\") " Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.716843 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"plugins-conf\" (UniqueName: \"kubernetes.io/configmap/06bfd2db-abe0-4b4d-a110-94a6a1e02211-plugins-conf\") pod \"06bfd2db-abe0-4b4d-a110-94a6a1e02211\" (UID: \"06bfd2db-abe0-4b4d-a110-94a6a1e02211\") " Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.716867 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"rabbitmq-tls\" (UniqueName: \"kubernetes.io/projected/06bfd2db-abe0-4b4d-a110-94a6a1e02211-rabbitmq-tls\") pod \"06bfd2db-abe0-4b4d-a110-94a6a1e02211\" (UID: \"06bfd2db-abe0-4b4d-a110-94a6a1e02211\") " Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.716916 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"server-conf\" (UniqueName: \"kubernetes.io/configmap/06bfd2db-abe0-4b4d-a110-94a6a1e02211-server-conf\") pod \"06bfd2db-abe0-4b4d-a110-94a6a1e02211\" (UID: \"06bfd2db-abe0-4b4d-a110-94a6a1e02211\") " Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.716953 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"pod-info\" (UniqueName: \"kubernetes.io/downward-api/06bfd2db-abe0-4b4d-a110-94a6a1e02211-pod-info\") pod \"06bfd2db-abe0-4b4d-a110-94a6a1e02211\" (UID: \"06bfd2db-abe0-4b4d-a110-94a6a1e02211\") " Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.716993 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2qnm9\" (UniqueName: \"kubernetes.io/projected/06bfd2db-abe0-4b4d-a110-94a6a1e02211-kube-api-access-2qnm9\") pod \"06bfd2db-abe0-4b4d-a110-94a6a1e02211\" (UID: \"06bfd2db-abe0-4b4d-a110-94a6a1e02211\") " Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.717037 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"persistence\" (UniqueName: \"kubernetes.io/local-volume/local-storage20-crc\") pod \"06bfd2db-abe0-4b4d-a110-94a6a1e02211\" (UID: \"06bfd2db-abe0-4b4d-a110-94a6a1e02211\") " Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.717061 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"erlang-cookie-secret\" (UniqueName: \"kubernetes.io/secret/06bfd2db-abe0-4b4d-a110-94a6a1e02211-erlang-cookie-secret\") pod \"06bfd2db-abe0-4b4d-a110-94a6a1e02211\" (UID: \"06bfd2db-abe0-4b4d-a110-94a6a1e02211\") " Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.717336 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/06bfd2db-abe0-4b4d-a110-94a6a1e02211-plugins-conf" (OuterVolumeSpecName: "plugins-conf") pod "06bfd2db-abe0-4b4d-a110-94a6a1e02211" (UID: "06bfd2db-abe0-4b4d-a110-94a6a1e02211"). InnerVolumeSpecName "plugins-conf". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.717540 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/89ee1bab-2c68-4fc4-9171-a7330922faf6-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.717565 5052 reconciler_common.go:293] "Volume detached for volume \"plugins-conf\" (UniqueName: \"kubernetes.io/configmap/06bfd2db-abe0-4b4d-a110-94a6a1e02211-plugins-conf\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.717577 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/2f3b4f6a-5f11-4039-a3bd-7b553ab5684d-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.717589 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-d5spc\" (UniqueName: \"kubernetes.io/projected/89ee1bab-2c68-4fc4-9171-a7330922faf6-kube-api-access-d5spc\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.717654 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-vjnlh\" (UniqueName: \"kubernetes.io/projected/2f3b4f6a-5f11-4039-a3bd-7b553ab5684d-kube-api-access-vjnlh\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.717785 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/06bfd2db-abe0-4b4d-a110-94a6a1e02211-rabbitmq-plugins" (OuterVolumeSpecName: "rabbitmq-plugins") pod "06bfd2db-abe0-4b4d-a110-94a6a1e02211" (UID: "06bfd2db-abe0-4b4d-a110-94a6a1e02211"). InnerVolumeSpecName "rabbitmq-plugins". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.717818 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican5c55-account-delete-zttb9"] Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.718559 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/06bfd2db-abe0-4b4d-a110-94a6a1e02211-rabbitmq-erlang-cookie" (OuterVolumeSpecName: "rabbitmq-erlang-cookie") pod "06bfd2db-abe0-4b4d-a110-94a6a1e02211" (UID: "06bfd2db-abe0-4b4d-a110-94a6a1e02211"). InnerVolumeSpecName "rabbitmq-erlang-cookie". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.722433 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/06bfd2db-abe0-4b4d-a110-94a6a1e02211-kube-api-access-2qnm9" (OuterVolumeSpecName: "kube-api-access-2qnm9") pod "06bfd2db-abe0-4b4d-a110-94a6a1e02211" (UID: "06bfd2db-abe0-4b4d-a110-94a6a1e02211"). InnerVolumeSpecName "kube-api-access-2qnm9". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.722982 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/06bfd2db-abe0-4b4d-a110-94a6a1e02211-pod-info" (OuterVolumeSpecName: "pod-info") pod "06bfd2db-abe0-4b4d-a110-94a6a1e02211" (UID: "06bfd2db-abe0-4b4d-a110-94a6a1e02211"). InnerVolumeSpecName "pod-info". PluginName "kubernetes.io/downward-api", VolumeGidValue "" Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.723255 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/06bfd2db-abe0-4b4d-a110-94a6a1e02211-erlang-cookie-secret" (OuterVolumeSpecName: "erlang-cookie-secret") pod "06bfd2db-abe0-4b4d-a110-94a6a1e02211" (UID: "06bfd2db-abe0-4b4d-a110-94a6a1e02211"). InnerVolumeSpecName "erlang-cookie-secret". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.724771 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage20-crc" (OuterVolumeSpecName: "persistence") pod "06bfd2db-abe0-4b4d-a110-94a6a1e02211" (UID: "06bfd2db-abe0-4b4d-a110-94a6a1e02211"). InnerVolumeSpecName "local-storage20-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.724914 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/06bfd2db-abe0-4b4d-a110-94a6a1e02211-rabbitmq-tls" (OuterVolumeSpecName: "rabbitmq-tls") pod "06bfd2db-abe0-4b4d-a110-94a6a1e02211" (UID: "06bfd2db-abe0-4b4d-a110-94a6a1e02211"). InnerVolumeSpecName "rabbitmq-tls". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.731551 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/barbican5c55-account-delete-zttb9"] Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.738576 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/06bfd2db-abe0-4b4d-a110-94a6a1e02211-config-data" (OuterVolumeSpecName: "config-data") pod "06bfd2db-abe0-4b4d-a110-94a6a1e02211" (UID: "06bfd2db-abe0-4b4d-a110-94a6a1e02211"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.765668 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/06bfd2db-abe0-4b4d-a110-94a6a1e02211-server-conf" (OuterVolumeSpecName: "server-conf") pod "06bfd2db-abe0-4b4d-a110-94a6a1e02211" (UID: "06bfd2db-abe0-4b4d-a110-94a6a1e02211"). InnerVolumeSpecName "server-conf". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.820209 5052 reconciler_common.go:293] "Volume detached for volume \"server-conf\" (UniqueName: \"kubernetes.io/configmap/06bfd2db-abe0-4b4d-a110-94a6a1e02211-server-conf\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.820328 5052 reconciler_common.go:293] "Volume detached for volume \"pod-info\" (UniqueName: \"kubernetes.io/downward-api/06bfd2db-abe0-4b4d-a110-94a6a1e02211-pod-info\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.820341 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-2qnm9\" (UniqueName: \"kubernetes.io/projected/06bfd2db-abe0-4b4d-a110-94a6a1e02211-kube-api-access-2qnm9\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.820378 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage20-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage20-crc\") on node \"crc\" " Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.820393 5052 reconciler_common.go:293] "Volume detached for volume \"erlang-cookie-secret\" (UniqueName: \"kubernetes.io/secret/06bfd2db-abe0-4b4d-a110-94a6a1e02211-erlang-cookie-secret\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.820404 5052 reconciler_common.go:293] "Volume detached for volume \"rabbitmq-erlang-cookie\" (UniqueName: \"kubernetes.io/empty-dir/06bfd2db-abe0-4b4d-a110-94a6a1e02211-rabbitmq-erlang-cookie\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.820415 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/06bfd2db-abe0-4b4d-a110-94a6a1e02211-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.820426 5052 reconciler_common.go:293] "Volume detached for volume \"rabbitmq-plugins\" (UniqueName: \"kubernetes.io/empty-dir/06bfd2db-abe0-4b4d-a110-94a6a1e02211-rabbitmq-plugins\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.820437 5052 reconciler_common.go:293] "Volume detached for volume \"rabbitmq-tls\" (UniqueName: \"kubernetes.io/projected/06bfd2db-abe0-4b4d-a110-94a6a1e02211-rabbitmq-tls\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.848813 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/06bfd2db-abe0-4b4d-a110-94a6a1e02211-rabbitmq-confd" (OuterVolumeSpecName: "rabbitmq-confd") pod "06bfd2db-abe0-4b4d-a110-94a6a1e02211" (UID: "06bfd2db-abe0-4b4d-a110-94a6a1e02211"). InnerVolumeSpecName "rabbitmq-confd". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.870284 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage20-crc" (UniqueName: "kubernetes.io/local-volume/local-storage20-crc") on node "crc" Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.911684 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/rabbitmq-server-2" Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.922092 5052 reconciler_common.go:293] "Volume detached for volume \"rabbitmq-confd\" (UniqueName: \"kubernetes.io/projected/06bfd2db-abe0-4b4d-a110-94a6a1e02211-rabbitmq-confd\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.922123 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage20-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage20-crc\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.927656 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/rabbitmq-server-1" Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.970642 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/novaapi48e2-account-delete-kqhz7"] Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.981042 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/novaapi48e2-account-delete-kqhz7"] Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.993822 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/memcached-0"] Dec 03 12:08:38 crc kubenswrapper[5052]: I1203 12:08:38.999921 5052 scope.go:117] "RemoveContainer" containerID="0e1e468e3a4838dddbe49481471a1e007e23016bc27634f15327a9a15546fcb0" Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.003791 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/memcached-0"] Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.006463 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/rabbitmq-cell1-server-1" Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.016596 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/placement3c40-account-delete-8qsdr"] Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.017400 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.025938 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8zh7k\" (UniqueName: \"kubernetes.io/projected/7b4f6078-41a3-433d-acaf-f2d67391ab6e-kube-api-access-8zh7k\") pod \"7b4f6078-41a3-433d-acaf-f2d67391ab6e\" (UID: \"7b4f6078-41a3-433d-acaf-f2d67391ab6e\") " Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.026036 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4v29h\" (UniqueName: \"kubernetes.io/projected/7901ce85-4e6f-4e8b-86a6-f50a85803637-kube-api-access-4v29h\") pod \"7901ce85-4e6f-4e8b-86a6-f50a85803637\" (UID: \"7901ce85-4e6f-4e8b-86a6-f50a85803637\") " Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.026431 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"rabbitmq-tls\" (UniqueName: \"kubernetes.io/projected/7901ce85-4e6f-4e8b-86a6-f50a85803637-rabbitmq-tls\") pod \"7901ce85-4e6f-4e8b-86a6-f50a85803637\" (UID: \"7901ce85-4e6f-4e8b-86a6-f50a85803637\") " Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.026479 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"rabbitmq-erlang-cookie\" (UniqueName: \"kubernetes.io/empty-dir/7b4f6078-41a3-433d-acaf-f2d67391ab6e-rabbitmq-erlang-cookie\") pod \"7b4f6078-41a3-433d-acaf-f2d67391ab6e\" (UID: \"7b4f6078-41a3-433d-acaf-f2d67391ab6e\") " Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.026532 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"rabbitmq-plugins\" (UniqueName: \"kubernetes.io/empty-dir/7b4f6078-41a3-433d-acaf-f2d67391ab6e-rabbitmq-plugins\") pod \"7b4f6078-41a3-433d-acaf-f2d67391ab6e\" (UID: \"7b4f6078-41a3-433d-acaf-f2d67391ab6e\") " Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.026591 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"rabbitmq-erlang-cookie\" (UniqueName: \"kubernetes.io/empty-dir/7901ce85-4e6f-4e8b-86a6-f50a85803637-rabbitmq-erlang-cookie\") pod \"7901ce85-4e6f-4e8b-86a6-f50a85803637\" (UID: \"7901ce85-4e6f-4e8b-86a6-f50a85803637\") " Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.028112 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7b4f6078-41a3-433d-acaf-f2d67391ab6e-rabbitmq-erlang-cookie" (OuterVolumeSpecName: "rabbitmq-erlang-cookie") pod "7b4f6078-41a3-433d-acaf-f2d67391ab6e" (UID: "7b4f6078-41a3-433d-acaf-f2d67391ab6e"). InnerVolumeSpecName "rabbitmq-erlang-cookie". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.028133 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7b4f6078-41a3-433d-acaf-f2d67391ab6e-rabbitmq-plugins" (OuterVolumeSpecName: "rabbitmq-plugins") pod "7b4f6078-41a3-433d-acaf-f2d67391ab6e" (UID: "7b4f6078-41a3-433d-acaf-f2d67391ab6e"). InnerVolumeSpecName "rabbitmq-plugins". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.029814 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7901ce85-4e6f-4e8b-86a6-f50a85803637-rabbitmq-erlang-cookie" (OuterVolumeSpecName: "rabbitmq-erlang-cookie") pod "7901ce85-4e6f-4e8b-86a6-f50a85803637" (UID: "7901ce85-4e6f-4e8b-86a6-f50a85803637"). InnerVolumeSpecName "rabbitmq-erlang-cookie". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.030801 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/placement3c40-account-delete-8qsdr"] Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.031627 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7901ce85-4e6f-4e8b-86a6-f50a85803637-kube-api-access-4v29h" (OuterVolumeSpecName: "kube-api-access-4v29h") pod "7901ce85-4e6f-4e8b-86a6-f50a85803637" (UID: "7901ce85-4e6f-4e8b-86a6-f50a85803637"). InnerVolumeSpecName "kube-api-access-4v29h". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.031669 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7b4f6078-41a3-433d-acaf-f2d67391ab6e-kube-api-access-8zh7k" (OuterVolumeSpecName: "kube-api-access-8zh7k") pod "7b4f6078-41a3-433d-acaf-f2d67391ab6e" (UID: "7b4f6078-41a3-433d-acaf-f2d67391ab6e"). InnerVolumeSpecName "kube-api-access-8zh7k". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.031764 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"pod-info\" (UniqueName: \"kubernetes.io/downward-api/7b4f6078-41a3-433d-acaf-f2d67391ab6e-pod-info\") pod \"7b4f6078-41a3-433d-acaf-f2d67391ab6e\" (UID: \"7b4f6078-41a3-433d-acaf-f2d67391ab6e\") " Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.031790 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"persistence\" (UniqueName: \"kubernetes.io/local-volume/local-storage05-crc\") pod \"7b4f6078-41a3-433d-acaf-f2d67391ab6e\" (UID: \"7b4f6078-41a3-433d-acaf-f2d67391ab6e\") " Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.031828 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"server-conf\" (UniqueName: \"kubernetes.io/configmap/7901ce85-4e6f-4e8b-86a6-f50a85803637-server-conf\") pod \"7901ce85-4e6f-4e8b-86a6-f50a85803637\" (UID: \"7901ce85-4e6f-4e8b-86a6-f50a85803637\") " Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.031859 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"plugins-conf\" (UniqueName: \"kubernetes.io/configmap/7901ce85-4e6f-4e8b-86a6-f50a85803637-plugins-conf\") pod \"7901ce85-4e6f-4e8b-86a6-f50a85803637\" (UID: \"7901ce85-4e6f-4e8b-86a6-f50a85803637\") " Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.031881 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"rabbitmq-confd\" (UniqueName: \"kubernetes.io/projected/7b4f6078-41a3-433d-acaf-f2d67391ab6e-rabbitmq-confd\") pod \"7b4f6078-41a3-433d-acaf-f2d67391ab6e\" (UID: \"7b4f6078-41a3-433d-acaf-f2d67391ab6e\") " Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.031901 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"persistence\" (UniqueName: \"kubernetes.io/local-volume/local-storage02-crc\") pod \"7901ce85-4e6f-4e8b-86a6-f50a85803637\" (UID: \"7901ce85-4e6f-4e8b-86a6-f50a85803637\") " Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.031942 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"erlang-cookie-secret\" (UniqueName: \"kubernetes.io/secret/7901ce85-4e6f-4e8b-86a6-f50a85803637-erlang-cookie-secret\") pod \"7901ce85-4e6f-4e8b-86a6-f50a85803637\" (UID: \"7901ce85-4e6f-4e8b-86a6-f50a85803637\") " Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.031981 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/7b4f6078-41a3-433d-acaf-f2d67391ab6e-config-data\") pod \"7b4f6078-41a3-433d-acaf-f2d67391ab6e\" (UID: \"7b4f6078-41a3-433d-acaf-f2d67391ab6e\") " Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.032016 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"plugins-conf\" (UniqueName: \"kubernetes.io/configmap/7b4f6078-41a3-433d-acaf-f2d67391ab6e-plugins-conf\") pod \"7b4f6078-41a3-433d-acaf-f2d67391ab6e\" (UID: \"7b4f6078-41a3-433d-acaf-f2d67391ab6e\") " Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.032107 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"rabbitmq-plugins\" (UniqueName: \"kubernetes.io/empty-dir/7901ce85-4e6f-4e8b-86a6-f50a85803637-rabbitmq-plugins\") pod \"7901ce85-4e6f-4e8b-86a6-f50a85803637\" (UID: \"7901ce85-4e6f-4e8b-86a6-f50a85803637\") " Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.032150 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/7901ce85-4e6f-4e8b-86a6-f50a85803637-config-data\") pod \"7901ce85-4e6f-4e8b-86a6-f50a85803637\" (UID: \"7901ce85-4e6f-4e8b-86a6-f50a85803637\") " Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.032175 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"pod-info\" (UniqueName: \"kubernetes.io/downward-api/7901ce85-4e6f-4e8b-86a6-f50a85803637-pod-info\") pod \"7901ce85-4e6f-4e8b-86a6-f50a85803637\" (UID: \"7901ce85-4e6f-4e8b-86a6-f50a85803637\") " Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.032216 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"rabbitmq-confd\" (UniqueName: \"kubernetes.io/projected/7901ce85-4e6f-4e8b-86a6-f50a85803637-rabbitmq-confd\") pod \"7901ce85-4e6f-4e8b-86a6-f50a85803637\" (UID: \"7901ce85-4e6f-4e8b-86a6-f50a85803637\") " Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.032259 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"server-conf\" (UniqueName: \"kubernetes.io/configmap/7b4f6078-41a3-433d-acaf-f2d67391ab6e-server-conf\") pod \"7b4f6078-41a3-433d-acaf-f2d67391ab6e\" (UID: \"7b4f6078-41a3-433d-acaf-f2d67391ab6e\") " Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.032285 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"rabbitmq-tls\" (UniqueName: \"kubernetes.io/projected/7b4f6078-41a3-433d-acaf-f2d67391ab6e-rabbitmq-tls\") pod \"7b4f6078-41a3-433d-acaf-f2d67391ab6e\" (UID: \"7b4f6078-41a3-433d-acaf-f2d67391ab6e\") " Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.032307 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"erlang-cookie-secret\" (UniqueName: \"kubernetes.io/secret/7b4f6078-41a3-433d-acaf-f2d67391ab6e-erlang-cookie-secret\") pod \"7b4f6078-41a3-433d-acaf-f2d67391ab6e\" (UID: \"7b4f6078-41a3-433d-acaf-f2d67391ab6e\") " Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.033007 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-8zh7k\" (UniqueName: \"kubernetes.io/projected/7b4f6078-41a3-433d-acaf-f2d67391ab6e-kube-api-access-8zh7k\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.033040 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-4v29h\" (UniqueName: \"kubernetes.io/projected/7901ce85-4e6f-4e8b-86a6-f50a85803637-kube-api-access-4v29h\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.033055 5052 reconciler_common.go:293] "Volume detached for volume \"rabbitmq-erlang-cookie\" (UniqueName: \"kubernetes.io/empty-dir/7b4f6078-41a3-433d-acaf-f2d67391ab6e-rabbitmq-erlang-cookie\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.033069 5052 reconciler_common.go:293] "Volume detached for volume \"rabbitmq-plugins\" (UniqueName: \"kubernetes.io/empty-dir/7b4f6078-41a3-433d-acaf-f2d67391ab6e-rabbitmq-plugins\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.033087 5052 reconciler_common.go:293] "Volume detached for volume \"rabbitmq-erlang-cookie\" (UniqueName: \"kubernetes.io/empty-dir/7901ce85-4e6f-4e8b-86a6-f50a85803637-rabbitmq-erlang-cookie\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.038231 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7901ce85-4e6f-4e8b-86a6-f50a85803637-rabbitmq-tls" (OuterVolumeSpecName: "rabbitmq-tls") pod "7901ce85-4e6f-4e8b-86a6-f50a85803637" (UID: "7901ce85-4e6f-4e8b-86a6-f50a85803637"). InnerVolumeSpecName "rabbitmq-tls". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.038708 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7901ce85-4e6f-4e8b-86a6-f50a85803637-rabbitmq-plugins" (OuterVolumeSpecName: "rabbitmq-plugins") pod "7901ce85-4e6f-4e8b-86a6-f50a85803637" (UID: "7901ce85-4e6f-4e8b-86a6-f50a85803637"). InnerVolumeSpecName "rabbitmq-plugins". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.039109 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7b4f6078-41a3-433d-acaf-f2d67391ab6e-plugins-conf" (OuterVolumeSpecName: "plugins-conf") pod "7b4f6078-41a3-433d-acaf-f2d67391ab6e" (UID: "7b4f6078-41a3-433d-acaf-f2d67391ab6e"). InnerVolumeSpecName "plugins-conf". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.040783 5052 scope.go:117] "RemoveContainer" containerID="85d1ba6ba0934dbfc1b081d24d64fbe85278667fa8c3b4ccf7dd7749c336f706" Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.041028 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7901ce85-4e6f-4e8b-86a6-f50a85803637-pod-info" (OuterVolumeSpecName: "pod-info") pod "7901ce85-4e6f-4e8b-86a6-f50a85803637" (UID: "7901ce85-4e6f-4e8b-86a6-f50a85803637"). InnerVolumeSpecName "pod-info". PluginName "kubernetes.io/downward-api", VolumeGidValue "" Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.043064 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage05-crc" (OuterVolumeSpecName: "persistence") pod "7b4f6078-41a3-433d-acaf-f2d67391ab6e" (UID: "7b4f6078-41a3-433d-acaf-f2d67391ab6e"). InnerVolumeSpecName "local-storage05-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.043214 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7b4f6078-41a3-433d-acaf-f2d67391ab6e-erlang-cookie-secret" (OuterVolumeSpecName: "erlang-cookie-secret") pod "7b4f6078-41a3-433d-acaf-f2d67391ab6e" (UID: "7b4f6078-41a3-433d-acaf-f2d67391ab6e"). InnerVolumeSpecName "erlang-cookie-secret". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.043720 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7901ce85-4e6f-4e8b-86a6-f50a85803637-plugins-conf" (OuterVolumeSpecName: "plugins-conf") pod "7901ce85-4e6f-4e8b-86a6-f50a85803637" (UID: "7901ce85-4e6f-4e8b-86a6-f50a85803637"). InnerVolumeSpecName "plugins-conf". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.047850 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7b4f6078-41a3-433d-acaf-f2d67391ab6e-pod-info" (OuterVolumeSpecName: "pod-info") pod "7b4f6078-41a3-433d-acaf-f2d67391ab6e" (UID: "7b4f6078-41a3-433d-acaf-f2d67391ab6e"). InnerVolumeSpecName "pod-info". PluginName "kubernetes.io/downward-api", VolumeGidValue "" Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.055349 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-85f8bc6fb5-b86k2" Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.069727 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ovn-northd-0"] Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.075543 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ovn-northd-0"] Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.079826 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7b4f6078-41a3-433d-acaf-f2d67391ab6e-rabbitmq-tls" (OuterVolumeSpecName: "rabbitmq-tls") pod "7b4f6078-41a3-433d-acaf-f2d67391ab6e" (UID: "7b4f6078-41a3-433d-acaf-f2d67391ab6e"). InnerVolumeSpecName "rabbitmq-tls". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.083134 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7901ce85-4e6f-4e8b-86a6-f50a85803637-erlang-cookie-secret" (OuterVolumeSpecName: "erlang-cookie-secret") pod "7901ce85-4e6f-4e8b-86a6-f50a85803637" (UID: "7901ce85-4e6f-4e8b-86a6-f50a85803637"). InnerVolumeSpecName "erlang-cookie-secret". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.085604 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage02-crc" (OuterVolumeSpecName: "persistence") pod "7901ce85-4e6f-4e8b-86a6-f50a85803637" (UID: "7901ce85-4e6f-4e8b-86a6-f50a85803637"). InnerVolumeSpecName "local-storage02-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.094442 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/novacell0d294-account-delete-b64wn"] Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.095816 5052 scope.go:117] "RemoveContainer" containerID="d1c0d412f0ba24368b4e09fb49e650d7f9751d7ab3deaf51a87873def7cef029" Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.123152 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7b4f6078-41a3-433d-acaf-f2d67391ab6e-config-data" (OuterVolumeSpecName: "config-data") pod "7b4f6078-41a3-433d-acaf-f2d67391ab6e" (UID: "7b4f6078-41a3-433d-acaf-f2d67391ab6e"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.123194 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7901ce85-4e6f-4e8b-86a6-f50a85803637-config-data" (OuterVolumeSpecName: "config-data") pod "7901ce85-4e6f-4e8b-86a6-f50a85803637" (UID: "7901ce85-4e6f-4e8b-86a6-f50a85803637"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.125558 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/novacell0d294-account-delete-b64wn"] Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.128556 5052 scope.go:117] "RemoveContainer" containerID="90fdc12931b30ef7c91db47e3cdd46baa57f5468f761a99d507e1b30d5373dd1" Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.133902 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"rabbitmq-plugins\" (UniqueName: \"kubernetes.io/empty-dir/2397d896-0d11-4656-a336-eac8ec9fe7f9-rabbitmq-plugins\") pod \"2397d896-0d11-4656-a336-eac8ec9fe7f9\" (UID: \"2397d896-0d11-4656-a336-eac8ec9fe7f9\") " Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.134046 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"persistence\" (UniqueName: \"kubernetes.io/local-volume/local-storage15-crc\") pod \"2397d896-0d11-4656-a336-eac8ec9fe7f9\" (UID: \"2397d896-0d11-4656-a336-eac8ec9fe7f9\") " Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.134086 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"erlang-cookie-secret\" (UniqueName: \"kubernetes.io/secret/2397d896-0d11-4656-a336-eac8ec9fe7f9-erlang-cookie-secret\") pod \"2397d896-0d11-4656-a336-eac8ec9fe7f9\" (UID: \"2397d896-0d11-4656-a336-eac8ec9fe7f9\") " Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.134118 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/77077fe5-d1ff-49c4-b049-3a0f2cee23b0-config\") pod \"77077fe5-d1ff-49c4-b049-3a0f2cee23b0\" (UID: \"77077fe5-d1ff-49c4-b049-3a0f2cee23b0\") " Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.134146 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dfcpx\" (UniqueName: \"kubernetes.io/projected/2397d896-0d11-4656-a336-eac8ec9fe7f9-kube-api-access-dfcpx\") pod \"2397d896-0d11-4656-a336-eac8ec9fe7f9\" (UID: \"2397d896-0d11-4656-a336-eac8ec9fe7f9\") " Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.134162 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jckkg\" (UniqueName: \"kubernetes.io/projected/77077fe5-d1ff-49c4-b049-3a0f2cee23b0-kube-api-access-jckkg\") pod \"77077fe5-d1ff-49c4-b049-3a0f2cee23b0\" (UID: \"77077fe5-d1ff-49c4-b049-3a0f2cee23b0\") " Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.134191 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/2397d896-0d11-4656-a336-eac8ec9fe7f9-config-data\") pod \"2397d896-0d11-4656-a336-eac8ec9fe7f9\" (UID: \"2397d896-0d11-4656-a336-eac8ec9fe7f9\") " Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.134214 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"server-conf\" (UniqueName: \"kubernetes.io/configmap/f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3-server-conf\") pod \"f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3\" (UID: \"f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3\") " Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.134237 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"rabbitmq-erlang-cookie\" (UniqueName: \"kubernetes.io/empty-dir/2397d896-0d11-4656-a336-eac8ec9fe7f9-rabbitmq-erlang-cookie\") pod \"2397d896-0d11-4656-a336-eac8ec9fe7f9\" (UID: \"2397d896-0d11-4656-a336-eac8ec9fe7f9\") " Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.134254 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/77077fe5-d1ff-49c4-b049-3a0f2cee23b0-dnsmasq-svc\") pod \"77077fe5-d1ff-49c4-b049-3a0f2cee23b0\" (UID: \"77077fe5-d1ff-49c4-b049-3a0f2cee23b0\") " Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.134276 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"rabbitmq-confd\" (UniqueName: \"kubernetes.io/projected/f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3-rabbitmq-confd\") pod \"f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3\" (UID: \"f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3\") " Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.134292 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"rabbitmq-tls\" (UniqueName: \"kubernetes.io/projected/f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3-rabbitmq-tls\") pod \"f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3\" (UID: \"f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3\") " Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.134316 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"rabbitmq-erlang-cookie\" (UniqueName: \"kubernetes.io/empty-dir/f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3-rabbitmq-erlang-cookie\") pod \"f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3\" (UID: \"f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3\") " Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.134334 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"rabbitmq-confd\" (UniqueName: \"kubernetes.io/projected/2397d896-0d11-4656-a336-eac8ec9fe7f9-rabbitmq-confd\") pod \"2397d896-0d11-4656-a336-eac8ec9fe7f9\" (UID: \"2397d896-0d11-4656-a336-eac8ec9fe7f9\") " Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.134376 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"dns-swift-storage-0\" (UniqueName: \"kubernetes.io/configmap/77077fe5-d1ff-49c4-b049-3a0f2cee23b0-dns-swift-storage-0\") pod \"77077fe5-d1ff-49c4-b049-3a0f2cee23b0\" (UID: \"77077fe5-d1ff-49c4-b049-3a0f2cee23b0\") " Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.134411 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"plugins-conf\" (UniqueName: \"kubernetes.io/configmap/2397d896-0d11-4656-a336-eac8ec9fe7f9-plugins-conf\") pod \"2397d896-0d11-4656-a336-eac8ec9fe7f9\" (UID: \"2397d896-0d11-4656-a336-eac8ec9fe7f9\") " Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.134450 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"pod-info\" (UniqueName: \"kubernetes.io/downward-api/2397d896-0d11-4656-a336-eac8ec9fe7f9-pod-info\") pod \"2397d896-0d11-4656-a336-eac8ec9fe7f9\" (UID: \"2397d896-0d11-4656-a336-eac8ec9fe7f9\") " Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.134470 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"erlang-cookie-secret\" (UniqueName: \"kubernetes.io/secret/f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3-erlang-cookie-secret\") pod \"f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3\" (UID: \"f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3\") " Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.134489 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"plugins-conf\" (UniqueName: \"kubernetes.io/configmap/f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3-plugins-conf\") pod \"f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3\" (UID: \"f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3\") " Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.134528 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"rabbitmq-tls\" (UniqueName: \"kubernetes.io/projected/2397d896-0d11-4656-a336-eac8ec9fe7f9-rabbitmq-tls\") pod \"2397d896-0d11-4656-a336-eac8ec9fe7f9\" (UID: \"2397d896-0d11-4656-a336-eac8ec9fe7f9\") " Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.134563 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"pod-info\" (UniqueName: \"kubernetes.io/downward-api/f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3-pod-info\") pod \"f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3\" (UID: \"f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3\") " Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.134587 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"rabbitmq-plugins\" (UniqueName: \"kubernetes.io/empty-dir/f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3-rabbitmq-plugins\") pod \"f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3\" (UID: \"f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3\") " Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.134617 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"server-conf\" (UniqueName: \"kubernetes.io/configmap/2397d896-0d11-4656-a336-eac8ec9fe7f9-server-conf\") pod \"2397d896-0d11-4656-a336-eac8ec9fe7f9\" (UID: \"2397d896-0d11-4656-a336-eac8ec9fe7f9\") " Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.134633 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"persistence\" (UniqueName: \"kubernetes.io/local-volume/local-storage16-crc\") pod \"f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3\" (UID: \"f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3\") " Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.134651 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3-config-data\") pod \"f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3\" (UID: \"f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3\") " Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.134670 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4sns5\" (UniqueName: \"kubernetes.io/projected/f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3-kube-api-access-4sns5\") pod \"f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3\" (UID: \"f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3\") " Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.135007 5052 reconciler_common.go:293] "Volume detached for volume \"rabbitmq-plugins\" (UniqueName: \"kubernetes.io/empty-dir/7901ce85-4e6f-4e8b-86a6-f50a85803637-rabbitmq-plugins\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.135024 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/7901ce85-4e6f-4e8b-86a6-f50a85803637-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.135034 5052 reconciler_common.go:293] "Volume detached for volume \"pod-info\" (UniqueName: \"kubernetes.io/downward-api/7901ce85-4e6f-4e8b-86a6-f50a85803637-pod-info\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.135043 5052 reconciler_common.go:293] "Volume detached for volume \"rabbitmq-tls\" (UniqueName: \"kubernetes.io/projected/7b4f6078-41a3-433d-acaf-f2d67391ab6e-rabbitmq-tls\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.135051 5052 reconciler_common.go:293] "Volume detached for volume \"erlang-cookie-secret\" (UniqueName: \"kubernetes.io/secret/7b4f6078-41a3-433d-acaf-f2d67391ab6e-erlang-cookie-secret\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.135059 5052 reconciler_common.go:293] "Volume detached for volume \"rabbitmq-tls\" (UniqueName: \"kubernetes.io/projected/7901ce85-4e6f-4e8b-86a6-f50a85803637-rabbitmq-tls\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.135068 5052 reconciler_common.go:293] "Volume detached for volume \"pod-info\" (UniqueName: \"kubernetes.io/downward-api/7b4f6078-41a3-433d-acaf-f2d67391ab6e-pod-info\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.135085 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage05-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage05-crc\") on node \"crc\" " Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.135095 5052 reconciler_common.go:293] "Volume detached for volume \"plugins-conf\" (UniqueName: \"kubernetes.io/configmap/7901ce85-4e6f-4e8b-86a6-f50a85803637-plugins-conf\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.135118 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage02-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage02-crc\") on node \"crc\" " Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.135129 5052 reconciler_common.go:293] "Volume detached for volume \"erlang-cookie-secret\" (UniqueName: \"kubernetes.io/secret/7901ce85-4e6f-4e8b-86a6-f50a85803637-erlang-cookie-secret\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.135139 5052 reconciler_common.go:293] "Volume detached for volume \"plugins-conf\" (UniqueName: \"kubernetes.io/configmap/7b4f6078-41a3-433d-acaf-f2d67391ab6e-plugins-conf\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.135148 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/7b4f6078-41a3-433d-acaf-f2d67391ab6e-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.137282 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2397d896-0d11-4656-a336-eac8ec9fe7f9-rabbitmq-plugins" (OuterVolumeSpecName: "rabbitmq-plugins") pod "2397d896-0d11-4656-a336-eac8ec9fe7f9" (UID: "2397d896-0d11-4656-a336-eac8ec9fe7f9"). InnerVolumeSpecName "rabbitmq-plugins". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.140084 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage15-crc" (OuterVolumeSpecName: "persistence") pod "2397d896-0d11-4656-a336-eac8ec9fe7f9" (UID: "2397d896-0d11-4656-a336-eac8ec9fe7f9"). InnerVolumeSpecName "local-storage15-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.140297 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2397d896-0d11-4656-a336-eac8ec9fe7f9-plugins-conf" (OuterVolumeSpecName: "plugins-conf") pod "2397d896-0d11-4656-a336-eac8ec9fe7f9" (UID: "2397d896-0d11-4656-a336-eac8ec9fe7f9"). InnerVolumeSpecName "plugins-conf". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.140622 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2397d896-0d11-4656-a336-eac8ec9fe7f9-rabbitmq-erlang-cookie" (OuterVolumeSpecName: "rabbitmq-erlang-cookie") pod "2397d896-0d11-4656-a336-eac8ec9fe7f9" (UID: "2397d896-0d11-4656-a336-eac8ec9fe7f9"). InnerVolumeSpecName "rabbitmq-erlang-cookie". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.142845 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3-rabbitmq-plugins" (OuterVolumeSpecName: "rabbitmq-plugins") pod "f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3" (UID: "f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3"). InnerVolumeSpecName "rabbitmq-plugins". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.143106 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3-plugins-conf" (OuterVolumeSpecName: "plugins-conf") pod "f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3" (UID: "f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3"). InnerVolumeSpecName "plugins-conf". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.143910 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/neutrond5c4-account-delete-zj77r"] Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.156773 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3-rabbitmq-erlang-cookie" (OuterVolumeSpecName: "rabbitmq-erlang-cookie") pod "f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3" (UID: "f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3"). InnerVolumeSpecName "rabbitmq-erlang-cookie". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.186005 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/neutrond5c4-account-delete-zj77r"] Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.186210 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2397d896-0d11-4656-a336-eac8ec9fe7f9-erlang-cookie-secret" (OuterVolumeSpecName: "erlang-cookie-secret") pod "2397d896-0d11-4656-a336-eac8ec9fe7f9" (UID: "2397d896-0d11-4656-a336-eac8ec9fe7f9"). InnerVolumeSpecName "erlang-cookie-secret". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.186349 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2397d896-0d11-4656-a336-eac8ec9fe7f9-rabbitmq-tls" (OuterVolumeSpecName: "rabbitmq-tls") pod "2397d896-0d11-4656-a336-eac8ec9fe7f9" (UID: "2397d896-0d11-4656-a336-eac8ec9fe7f9"). InnerVolumeSpecName "rabbitmq-tls". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.190493 5052 scope.go:117] "RemoveContainer" containerID="6e5a8a55dfb233734ce359d756ebf08a407cc39b08dc02754ac067f403d0a466" Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.192756 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/77077fe5-d1ff-49c4-b049-3a0f2cee23b0-kube-api-access-jckkg" (OuterVolumeSpecName: "kube-api-access-jckkg") pod "77077fe5-d1ff-49c4-b049-3a0f2cee23b0" (UID: "77077fe5-d1ff-49c4-b049-3a0f2cee23b0"). InnerVolumeSpecName "kube-api-access-jckkg". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.192863 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3-rabbitmq-tls" (OuterVolumeSpecName: "rabbitmq-tls") pod "f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3" (UID: "f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3"). InnerVolumeSpecName "rabbitmq-tls". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.192883 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage16-crc" (OuterVolumeSpecName: "persistence") pod "f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3" (UID: "f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3"). InnerVolumeSpecName "local-storage16-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.193235 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2397d896-0d11-4656-a336-eac8ec9fe7f9-kube-api-access-dfcpx" (OuterVolumeSpecName: "kube-api-access-dfcpx") pod "2397d896-0d11-4656-a336-eac8ec9fe7f9" (UID: "2397d896-0d11-4656-a336-eac8ec9fe7f9"). InnerVolumeSpecName "kube-api-access-dfcpx". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.193348 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3-kube-api-access-4sns5" (OuterVolumeSpecName: "kube-api-access-4sns5") pod "f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3" (UID: "f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3"). InnerVolumeSpecName "kube-api-access-4sns5". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.199704 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage02-crc" (UniqueName: "kubernetes.io/local-volume/local-storage02-crc") on node "crc" Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.206537 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7901ce85-4e6f-4e8b-86a6-f50a85803637-server-conf" (OuterVolumeSpecName: "server-conf") pod "7901ce85-4e6f-4e8b-86a6-f50a85803637" (UID: "7901ce85-4e6f-4e8b-86a6-f50a85803637"). InnerVolumeSpecName "server-conf". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.210484 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2397d896-0d11-4656-a336-eac8ec9fe7f9-pod-info" (OuterVolumeSpecName: "pod-info") pod "2397d896-0d11-4656-a336-eac8ec9fe7f9" (UID: "2397d896-0d11-4656-a336-eac8ec9fe7f9"). InnerVolumeSpecName "pod-info". PluginName "kubernetes.io/downward-api", VolumeGidValue "" Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.234582 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/keystone-6c6748d95d-66jbh"] Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.237124 5052 reconciler_common.go:293] "Volume detached for volume \"rabbitmq-plugins\" (UniqueName: \"kubernetes.io/empty-dir/2397d896-0d11-4656-a336-eac8ec9fe7f9-rabbitmq-plugins\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.237174 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage15-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage15-crc\") on node \"crc\" " Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.237186 5052 reconciler_common.go:293] "Volume detached for volume \"erlang-cookie-secret\" (UniqueName: \"kubernetes.io/secret/2397d896-0d11-4656-a336-eac8ec9fe7f9-erlang-cookie-secret\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.237201 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-dfcpx\" (UniqueName: \"kubernetes.io/projected/2397d896-0d11-4656-a336-eac8ec9fe7f9-kube-api-access-dfcpx\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.237220 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-jckkg\" (UniqueName: \"kubernetes.io/projected/77077fe5-d1ff-49c4-b049-3a0f2cee23b0-kube-api-access-jckkg\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.237233 5052 reconciler_common.go:293] "Volume detached for volume \"rabbitmq-erlang-cookie\" (UniqueName: \"kubernetes.io/empty-dir/2397d896-0d11-4656-a336-eac8ec9fe7f9-rabbitmq-erlang-cookie\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.237247 5052 reconciler_common.go:293] "Volume detached for volume \"rabbitmq-tls\" (UniqueName: \"kubernetes.io/projected/f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3-rabbitmq-tls\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.237260 5052 reconciler_common.go:293] "Volume detached for volume \"rabbitmq-erlang-cookie\" (UniqueName: \"kubernetes.io/empty-dir/f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3-rabbitmq-erlang-cookie\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.237277 5052 reconciler_common.go:293] "Volume detached for volume \"server-conf\" (UniqueName: \"kubernetes.io/configmap/7901ce85-4e6f-4e8b-86a6-f50a85803637-server-conf\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.237289 5052 reconciler_common.go:293] "Volume detached for volume \"plugins-conf\" (UniqueName: \"kubernetes.io/configmap/2397d896-0d11-4656-a336-eac8ec9fe7f9-plugins-conf\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.237302 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage02-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage02-crc\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.237311 5052 reconciler_common.go:293] "Volume detached for volume \"pod-info\" (UniqueName: \"kubernetes.io/downward-api/2397d896-0d11-4656-a336-eac8ec9fe7f9-pod-info\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.237323 5052 reconciler_common.go:293] "Volume detached for volume \"plugins-conf\" (UniqueName: \"kubernetes.io/configmap/f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3-plugins-conf\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.237332 5052 reconciler_common.go:293] "Volume detached for volume \"rabbitmq-tls\" (UniqueName: \"kubernetes.io/projected/2397d896-0d11-4656-a336-eac8ec9fe7f9-rabbitmq-tls\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.237342 5052 reconciler_common.go:293] "Volume detached for volume \"rabbitmq-plugins\" (UniqueName: \"kubernetes.io/empty-dir/f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3-rabbitmq-plugins\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.237362 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage16-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage16-crc\") on node \"crc\" " Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.237371 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-4sns5\" (UniqueName: \"kubernetes.io/projected/f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3-kube-api-access-4sns5\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.239624 5052 scope.go:117] "RemoveContainer" containerID="082794493f1a866da7efb8038f854ca6442bb335e30fc8e05b37438e570e2b64" Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.244461 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3-pod-info" (OuterVolumeSpecName: "pod-info") pod "f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3" (UID: "f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3"). InnerVolumeSpecName "pod-info". PluginName "kubernetes.io/downward-api", VolumeGidValue "" Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.250324 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/keystone-6c6748d95d-66jbh"] Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.268657 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3-erlang-cookie-secret" (OuterVolumeSpecName: "erlang-cookie-secret") pod "f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3" (UID: "f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3"). InnerVolumeSpecName "erlang-cookie-secret". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.285227 5052 scope.go:117] "RemoveContainer" containerID="ee4e5d9018afd7520475044120c78ad156782c169bd6979998e8db4bb75386a2" Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.331434 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2397d896-0d11-4656-a336-eac8ec9fe7f9-config-data" (OuterVolumeSpecName: "config-data") pod "2397d896-0d11-4656-a336-eac8ec9fe7f9" (UID: "2397d896-0d11-4656-a336-eac8ec9fe7f9"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.331731 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2397d896-0d11-4656-a336-eac8ec9fe7f9-server-conf" (OuterVolumeSpecName: "server-conf") pod "2397d896-0d11-4656-a336-eac8ec9fe7f9" (UID: "2397d896-0d11-4656-a336-eac8ec9fe7f9"). InnerVolumeSpecName "server-conf". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.337321 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7b4f6078-41a3-433d-acaf-f2d67391ab6e-server-conf" (OuterVolumeSpecName: "server-conf") pod "7b4f6078-41a3-433d-acaf-f2d67391ab6e" (UID: "7b4f6078-41a3-433d-acaf-f2d67391ab6e"). InnerVolumeSpecName "server-conf". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.337448 5052 scope.go:117] "RemoveContainer" containerID="10aa046238c04ac35e5af15753b94eae0f3932f159a2bc2af4aa74369e06cf8e" Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.339974 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage15-crc" (UniqueName: "kubernetes.io/local-volume/local-storage15-crc") on node "crc" Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.340163 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/2397d896-0d11-4656-a336-eac8ec9fe7f9-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.340187 5052 reconciler_common.go:293] "Volume detached for volume \"erlang-cookie-secret\" (UniqueName: \"kubernetes.io/secret/f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3-erlang-cookie-secret\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.340199 5052 reconciler_common.go:293] "Volume detached for volume \"pod-info\" (UniqueName: \"kubernetes.io/downward-api/f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3-pod-info\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.340211 5052 reconciler_common.go:293] "Volume detached for volume \"server-conf\" (UniqueName: \"kubernetes.io/configmap/2397d896-0d11-4656-a336-eac8ec9fe7f9-server-conf\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.340222 5052 reconciler_common.go:293] "Volume detached for volume \"server-conf\" (UniqueName: \"kubernetes.io/configmap/7b4f6078-41a3-433d-acaf-f2d67391ab6e-server-conf\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.340232 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage15-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage15-crc\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.345979 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage05-crc" (UniqueName: "kubernetes.io/local-volume/local-storage05-crc") on node "crc" Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.349992 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage16-crc" (UniqueName: "kubernetes.io/local-volume/local-storage16-crc") on node "crc" Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.369338 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3-server-conf" (OuterVolumeSpecName: "server-conf") pod "f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3" (UID: "f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3"). InnerVolumeSpecName "server-conf". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.375009 5052 scope.go:117] "RemoveContainer" containerID="405640dc668d7b8021968432854058e5508074169be4f1e98d36626dd8db3fbf" Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.376470 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/77077fe5-d1ff-49c4-b049-3a0f2cee23b0-config" (OuterVolumeSpecName: "config") pod "77077fe5-d1ff-49c4-b049-3a0f2cee23b0" (UID: "77077fe5-d1ff-49c4-b049-3a0f2cee23b0"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.388441 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3-config-data" (OuterVolumeSpecName: "config-data") pod "f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3" (UID: "f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.394447 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/77077fe5-d1ff-49c4-b049-3a0f2cee23b0-dnsmasq-svc" (OuterVolumeSpecName: "dnsmasq-svc") pod "77077fe5-d1ff-49c4-b049-3a0f2cee23b0" (UID: "77077fe5-d1ff-49c4-b049-3a0f2cee23b0"). InnerVolumeSpecName "dnsmasq-svc". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.397761 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/77077fe5-d1ff-49c4-b049-3a0f2cee23b0-dns-swift-storage-0" (OuterVolumeSpecName: "dns-swift-storage-0") pod "77077fe5-d1ff-49c4-b049-3a0f2cee23b0" (UID: "77077fe5-d1ff-49c4-b049-3a0f2cee23b0"). InnerVolumeSpecName "dns-swift-storage-0". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.403595 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7901ce85-4e6f-4e8b-86a6-f50a85803637-rabbitmq-confd" (OuterVolumeSpecName: "rabbitmq-confd") pod "7901ce85-4e6f-4e8b-86a6-f50a85803637" (UID: "7901ce85-4e6f-4e8b-86a6-f50a85803637"). InnerVolumeSpecName "rabbitmq-confd". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.429043 5052 scope.go:117] "RemoveContainer" containerID="3d0ba918e75cd950bc4bab7a527b72d23c454cb5e03d5804f02c37caed27d0fa" Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.429181 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7b4f6078-41a3-433d-acaf-f2d67391ab6e-rabbitmq-confd" (OuterVolumeSpecName: "rabbitmq-confd") pod "7b4f6078-41a3-433d-acaf-f2d67391ab6e" (UID: "7b4f6078-41a3-433d-acaf-f2d67391ab6e"). InnerVolumeSpecName "rabbitmq-confd". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.441338 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage16-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage16-crc\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.441376 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.441389 5052 reconciler_common.go:293] "Volume detached for volume \"rabbitmq-confd\" (UniqueName: \"kubernetes.io/projected/7901ce85-4e6f-4e8b-86a6-f50a85803637-rabbitmq-confd\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.441398 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/77077fe5-d1ff-49c4-b049-3a0f2cee23b0-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.441409 5052 reconciler_common.go:293] "Volume detached for volume \"server-conf\" (UniqueName: \"kubernetes.io/configmap/f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3-server-conf\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.441418 5052 reconciler_common.go:293] "Volume detached for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/77077fe5-d1ff-49c4-b049-3a0f2cee23b0-dnsmasq-svc\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.441427 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage05-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage05-crc\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.441436 5052 reconciler_common.go:293] "Volume detached for volume \"dns-swift-storage-0\" (UniqueName: \"kubernetes.io/configmap/77077fe5-d1ff-49c4-b049-3a0f2cee23b0-dns-swift-storage-0\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.441446 5052 reconciler_common.go:293] "Volume detached for volume \"rabbitmq-confd\" (UniqueName: \"kubernetes.io/projected/7b4f6078-41a3-433d-acaf-f2d67391ab6e-rabbitmq-confd\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.451561 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2397d896-0d11-4656-a336-eac8ec9fe7f9-rabbitmq-confd" (OuterVolumeSpecName: "rabbitmq-confd") pod "2397d896-0d11-4656-a336-eac8ec9fe7f9" (UID: "2397d896-0d11-4656-a336-eac8ec9fe7f9"). InnerVolumeSpecName "rabbitmq-confd". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.461600 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3-rabbitmq-confd" (OuterVolumeSpecName: "rabbitmq-confd") pod "f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3" (UID: "f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3"). InnerVolumeSpecName "rabbitmq-confd". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.496099 5052 scope.go:117] "RemoveContainer" containerID="5ae82a1363577b2eb43c12684bdc2300b53209b11ba26536b7047f5b4704d361" Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.543208 5052 reconciler_common.go:293] "Volume detached for volume \"rabbitmq-confd\" (UniqueName: \"kubernetes.io/projected/f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3-rabbitmq-confd\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.543237 5052 reconciler_common.go:293] "Volume detached for volume \"rabbitmq-confd\" (UniqueName: \"kubernetes.io/projected/2397d896-0d11-4656-a336-eac8ec9fe7f9-rabbitmq-confd\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.569312 5052 scope.go:117] "RemoveContainer" containerID="b5a72c6d1c868fd1b07944228c1d42fca5faaf1bc649f58101b2d16f25813311" Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.606119 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-server-1" event={"ID":"7901ce85-4e6f-4e8b-86a6-f50a85803637","Type":"ContainerDied","Data":"9217e8f13f1d2136b88ccaf97e116bdf86a8e5417a0e7f7f09c3e798776eee44"} Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.606235 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/rabbitmq-server-1" Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.616803 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cindere6fc-account-delete-z7w6l" Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.626453 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-cell1-server-2" event={"ID":"06bfd2db-abe0-4b4d-a110-94a6a1e02211","Type":"ContainerDied","Data":"0191429cd34cf2c22173ad9222165ff9d2df01c0d470d59a5a082e087cd7c9b1"} Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.626580 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/rabbitmq-cell1-server-2" Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.632103 5052 generic.go:334] "Generic (PLEG): container finished" podID="77077fe5-d1ff-49c4-b049-3a0f2cee23b0" containerID="bc3186646511337244c91e6de61ef79f4b96f53697165b0fe223dd0c36cfeadf" exitCode=0 Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.632162 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-85f8bc6fb5-b86k2" event={"ID":"77077fe5-d1ff-49c4-b049-3a0f2cee23b0","Type":"ContainerDied","Data":"bc3186646511337244c91e6de61ef79f4b96f53697165b0fe223dd0c36cfeadf"} Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.632186 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-85f8bc6fb5-b86k2" event={"ID":"77077fe5-d1ff-49c4-b049-3a0f2cee23b0","Type":"ContainerDied","Data":"eec0587de577f0935369e48dc238153fc5b64bb2a1e2911d9a07e70aa47e4cf1"} Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.632191 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-85f8bc6fb5-b86k2" Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.644451 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.644282 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-server-0" event={"ID":"2397d896-0d11-4656-a336-eac8ec9fe7f9","Type":"ContainerDied","Data":"33f0c07e2aab29bf325981bed639c4e20ae10db667a2bc8a838a6c8012d98f79"} Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.646943 5052 generic.go:334] "Generic (PLEG): container finished" podID="b34bb4d0-7906-42e8-a9aa-bd1a4203a312" containerID="c55564d5848c1a333ba2db20a9c5ecdbf152ba4ebaa847c6e28f6ed69c3a99ed" exitCode=0 Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.648910 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-cell1-galera-1" event={"ID":"b34bb4d0-7906-42e8-a9aa-bd1a4203a312","Type":"ContainerDied","Data":"c55564d5848c1a333ba2db20a9c5ecdbf152ba4ebaa847c6e28f6ed69c3a99ed"} Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.651450 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-cell1-server-1" event={"ID":"f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3","Type":"ContainerDied","Data":"ae046ba315453cf7f27015c994cf66e9fd7e0b45ca2564c433fa8b4244e15245"} Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.651611 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/rabbitmq-cell1-server-1" Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.660068 5052 generic.go:334] "Generic (PLEG): container finished" podID="6ac95bd3-125c-4966-b9a7-4eb4a7842c67" containerID="edf48c4037d4be3adfc21b5d97a50d68b5261315fe92282773dd8556852ce51a" exitCode=0 Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.660122 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-cell1-galera-2" event={"ID":"6ac95bd3-125c-4966-b9a7-4eb4a7842c67","Type":"ContainerDied","Data":"edf48c4037d4be3adfc21b5d97a50d68b5261315fe92282773dd8556852ce51a"} Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.682646 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/rabbitmq-server-2" Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.682705 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance8fcf-account-delete-4xhcc" Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.725019 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="030d4f76-eb30-4c44-b782-fa7147febf41" path="/var/lib/kubelet/pods/030d4f76-eb30-4c44-b782-fa7147febf41/volumes" Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.725822 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="141a49da-93b6-483b-a476-1d5f3efc29d1" path="/var/lib/kubelet/pods/141a49da-93b6-483b-a476-1d5f3efc29d1/volumes" Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.726387 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="19229e07-3c78-4369-998e-fc9e78eb647d" path="/var/lib/kubelet/pods/19229e07-3c78-4369-998e-fc9e78eb647d/volumes" Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.727576 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1bc7c8b1-cc0d-4611-b335-01c2b91713e2" path="/var/lib/kubelet/pods/1bc7c8b1-cc0d-4611-b335-01c2b91713e2/volumes" Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.728080 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="60f72892-a5aa-482f-aac3-5713ba028bc8" path="/var/lib/kubelet/pods/60f72892-a5aa-482f-aac3-5713ba028bc8/volumes" Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.728872 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="70122e67-6175-4a14-a120-6a2731c5dcb1" path="/var/lib/kubelet/pods/70122e67-6175-4a14-a120-6a2731c5dcb1/volumes" Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.729922 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9e6af047-b7ee-459a-b362-da5f777d0fa0" path="/var/lib/kubelet/pods/9e6af047-b7ee-459a-b362-da5f777d0fa0/volumes" Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.730368 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aaa3db64-a10b-45ea-9e25-de4916f30617" path="/var/lib/kubelet/pods/aaa3db64-a10b-45ea-9e25-de4916f30617/volumes" Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.731027 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bbdcaf66-8813-4bed-9311-552fef62ac2d" path="/var/lib/kubelet/pods/bbdcaf66-8813-4bed-9311-552fef62ac2d/volumes" Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.732139 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c2470da9-4c96-44ac-834d-2b560785135f" path="/var/lib/kubelet/pods/c2470da9-4c96-44ac-834d-2b560785135f/volumes" Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.732919 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fad6f453-acf5-4f13-94d6-7d46e300f28c" path="/var/lib/kubelet/pods/fad6f453-acf5-4f13-94d6-7d46e300f28c/volumes" Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.735288 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-server-2" event={"ID":"7b4f6078-41a3-433d-acaf-f2d67391ab6e","Type":"ContainerDied","Data":"0ecd86b4dce73c50418f75d596549890a4e7015146f1f050b2673ef00e329423"} Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.765496 5052 scope.go:117] "RemoveContainer" containerID="19566104651a9853da1345044a96ed40b73615a3c7ab75d772d57f5e5fdc9b84" Dec 03 12:08:39 crc kubenswrapper[5052]: E1203 12:08:39.766828 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"19566104651a9853da1345044a96ed40b73615a3c7ab75d772d57f5e5fdc9b84\": container with ID starting with 19566104651a9853da1345044a96ed40b73615a3c7ab75d772d57f5e5fdc9b84 not found: ID does not exist" containerID="19566104651a9853da1345044a96ed40b73615a3c7ab75d772d57f5e5fdc9b84" Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.766869 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"19566104651a9853da1345044a96ed40b73615a3c7ab75d772d57f5e5fdc9b84"} err="failed to get container status \"19566104651a9853da1345044a96ed40b73615a3c7ab75d772d57f5e5fdc9b84\": rpc error: code = NotFound desc = could not find container \"19566104651a9853da1345044a96ed40b73615a3c7ab75d772d57f5e5fdc9b84\": container with ID starting with 19566104651a9853da1345044a96ed40b73615a3c7ab75d772d57f5e5fdc9b84 not found: ID does not exist" Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.766896 5052 scope.go:117] "RemoveContainer" containerID="aa4334f0d2af3013c5ace40839543a0d0b6116ced72af25a3daba9191d3d0382" Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.864496 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/rabbitmq-server-0"] Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.875678 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/rabbitmq-server-0"] Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.881010 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cindere6fc-account-delete-z7w6l"] Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.889350 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/cindere6fc-account-delete-z7w6l"] Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.905589 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstack-cell1-galera-2" Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.907483 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/rabbitmq-server-1"] Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.909294 5052 scope.go:117] "RemoveContainer" containerID="88ec697999833a01b97e7879439a3382717780668a2e4ef174d64567bf8d897a" Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.914650 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstack-cell1-galera-1" Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.921564 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/rabbitmq-server-1"] Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.935281 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/rabbitmq-cell1-server-2"] Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.979977 5052 scope.go:117] "RemoveContainer" containerID="ccd68041ae4063c32d99edce20b9aa0bf23f148287d9fcd6406b6a5e57a96ff9" Dec 03 12:08:39 crc kubenswrapper[5052]: I1203 12:08:39.988246 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/rabbitmq-cell1-server-2"] Dec 03 12:08:40 crc kubenswrapper[5052]: I1203 12:08:40.010195 5052 scope.go:117] "RemoveContainer" containerID="61952ac0175a2f78cbd3919ba15907c05a42be11756520e4a28f961c6f3c039e" Dec 03 12:08:40 crc kubenswrapper[5052]: I1203 12:08:40.014463 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-85f8bc6fb5-b86k2"] Dec 03 12:08:40 crc kubenswrapper[5052]: I1203 12:08:40.023110 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-85f8bc6fb5-b86k2"] Dec 03 12:08:40 crc kubenswrapper[5052]: I1203 12:08:40.038376 5052 scope.go:117] "RemoveContainer" containerID="9889df1b1220aa2bd33fc61e1b06bb56a5427700764018be0e9dc52ae30425fd" Dec 03 12:08:40 crc kubenswrapper[5052]: I1203 12:08:40.039307 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/rabbitmq-cell1-server-1"] Dec 03 12:08:40 crc kubenswrapper[5052]: I1203 12:08:40.046101 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/rabbitmq-cell1-server-1"] Dec 03 12:08:40 crc kubenswrapper[5052]: I1203 12:08:40.054126 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance8fcf-account-delete-4xhcc"] Dec 03 12:08:40 crc kubenswrapper[5052]: I1203 12:08:40.057460 5052 scope.go:117] "RemoveContainer" containerID="bc3186646511337244c91e6de61ef79f4b96f53697165b0fe223dd0c36cfeadf" Dec 03 12:08:40 crc kubenswrapper[5052]: I1203 12:08:40.058631 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hqgfz\" (UniqueName: \"kubernetes.io/projected/6ac95bd3-125c-4966-b9a7-4eb4a7842c67-kube-api-access-hqgfz\") pod \"6ac95bd3-125c-4966-b9a7-4eb4a7842c67\" (UID: \"6ac95bd3-125c-4966-b9a7-4eb4a7842c67\") " Dec 03 12:08:40 crc kubenswrapper[5052]: I1203 12:08:40.058732 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/6ac95bd3-125c-4966-b9a7-4eb4a7842c67-config-data-generated\") pod \"6ac95bd3-125c-4966-b9a7-4eb4a7842c67\" (UID: \"6ac95bd3-125c-4966-b9a7-4eb4a7842c67\") " Dec 03 12:08:40 crc kubenswrapper[5052]: I1203 12:08:40.058784 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"mysql-db\" (UniqueName: \"kubernetes.io/local-volume/local-storage14-crc\") pod \"6ac95bd3-125c-4966-b9a7-4eb4a7842c67\" (UID: \"6ac95bd3-125c-4966-b9a7-4eb4a7842c67\") " Dec 03 12:08:40 crc kubenswrapper[5052]: I1203 12:08:40.058826 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b34bb4d0-7906-42e8-a9aa-bd1a4203a312-combined-ca-bundle\") pod \"b34bb4d0-7906-42e8-a9aa-bd1a4203a312\" (UID: \"b34bb4d0-7906-42e8-a9aa-bd1a4203a312\") " Dec 03 12:08:40 crc kubenswrapper[5052]: I1203 12:08:40.058884 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/6ac95bd3-125c-4966-b9a7-4eb4a7842c67-kolla-config\") pod \"6ac95bd3-125c-4966-b9a7-4eb4a7842c67\" (UID: \"6ac95bd3-125c-4966-b9a7-4eb4a7842c67\") " Dec 03 12:08:40 crc kubenswrapper[5052]: I1203 12:08:40.058925 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/b34bb4d0-7906-42e8-a9aa-bd1a4203a312-kolla-config\") pod \"b34bb4d0-7906-42e8-a9aa-bd1a4203a312\" (UID: \"b34bb4d0-7906-42e8-a9aa-bd1a4203a312\") " Dec 03 12:08:40 crc kubenswrapper[5052]: I1203 12:08:40.058974 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kmr95\" (UniqueName: \"kubernetes.io/projected/b34bb4d0-7906-42e8-a9aa-bd1a4203a312-kube-api-access-kmr95\") pod \"b34bb4d0-7906-42e8-a9aa-bd1a4203a312\" (UID: \"b34bb4d0-7906-42e8-a9aa-bd1a4203a312\") " Dec 03 12:08:40 crc kubenswrapper[5052]: I1203 12:08:40.059014 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/6ac95bd3-125c-4966-b9a7-4eb4a7842c67-galera-tls-certs\") pod \"6ac95bd3-125c-4966-b9a7-4eb4a7842c67\" (UID: \"6ac95bd3-125c-4966-b9a7-4eb4a7842c67\") " Dec 03 12:08:40 crc kubenswrapper[5052]: I1203 12:08:40.059043 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6ac95bd3-125c-4966-b9a7-4eb4a7842c67-combined-ca-bundle\") pod \"6ac95bd3-125c-4966-b9a7-4eb4a7842c67\" (UID: \"6ac95bd3-125c-4966-b9a7-4eb4a7842c67\") " Dec 03 12:08:40 crc kubenswrapper[5052]: I1203 12:08:40.059093 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/b34bb4d0-7906-42e8-a9aa-bd1a4203a312-config-data-generated\") pod \"b34bb4d0-7906-42e8-a9aa-bd1a4203a312\" (UID: \"b34bb4d0-7906-42e8-a9aa-bd1a4203a312\") " Dec 03 12:08:40 crc kubenswrapper[5052]: I1203 12:08:40.059126 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/b34bb4d0-7906-42e8-a9aa-bd1a4203a312-galera-tls-certs\") pod \"b34bb4d0-7906-42e8-a9aa-bd1a4203a312\" (UID: \"b34bb4d0-7906-42e8-a9aa-bd1a4203a312\") " Dec 03 12:08:40 crc kubenswrapper[5052]: I1203 12:08:40.059231 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/b34bb4d0-7906-42e8-a9aa-bd1a4203a312-operator-scripts\") pod \"b34bb4d0-7906-42e8-a9aa-bd1a4203a312\" (UID: \"b34bb4d0-7906-42e8-a9aa-bd1a4203a312\") " Dec 03 12:08:40 crc kubenswrapper[5052]: I1203 12:08:40.059258 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/b34bb4d0-7906-42e8-a9aa-bd1a4203a312-config-data-default\") pod \"b34bb4d0-7906-42e8-a9aa-bd1a4203a312\" (UID: \"b34bb4d0-7906-42e8-a9aa-bd1a4203a312\") " Dec 03 12:08:40 crc kubenswrapper[5052]: I1203 12:08:40.059347 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/6ac95bd3-125c-4966-b9a7-4eb4a7842c67-operator-scripts\") pod \"6ac95bd3-125c-4966-b9a7-4eb4a7842c67\" (UID: \"6ac95bd3-125c-4966-b9a7-4eb4a7842c67\") " Dec 03 12:08:40 crc kubenswrapper[5052]: I1203 12:08:40.060215 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6ac95bd3-125c-4966-b9a7-4eb4a7842c67-config-data-generated" (OuterVolumeSpecName: "config-data-generated") pod "6ac95bd3-125c-4966-b9a7-4eb4a7842c67" (UID: "6ac95bd3-125c-4966-b9a7-4eb4a7842c67"). InnerVolumeSpecName "config-data-generated". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:08:40 crc kubenswrapper[5052]: I1203 12:08:40.060248 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b34bb4d0-7906-42e8-a9aa-bd1a4203a312-config-data-generated" (OuterVolumeSpecName: "config-data-generated") pod "b34bb4d0-7906-42e8-a9aa-bd1a4203a312" (UID: "b34bb4d0-7906-42e8-a9aa-bd1a4203a312"). InnerVolumeSpecName "config-data-generated". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:08:40 crc kubenswrapper[5052]: I1203 12:08:40.060757 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b34bb4d0-7906-42e8-a9aa-bd1a4203a312-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "b34bb4d0-7906-42e8-a9aa-bd1a4203a312" (UID: "b34bb4d0-7906-42e8-a9aa-bd1a4203a312"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:08:40 crc kubenswrapper[5052]: I1203 12:08:40.060879 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b34bb4d0-7906-42e8-a9aa-bd1a4203a312-config-data-default" (OuterVolumeSpecName: "config-data-default") pod "b34bb4d0-7906-42e8-a9aa-bd1a4203a312" (UID: "b34bb4d0-7906-42e8-a9aa-bd1a4203a312"). InnerVolumeSpecName "config-data-default". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:08:40 crc kubenswrapper[5052]: I1203 12:08:40.061194 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6ac95bd3-125c-4966-b9a7-4eb4a7842c67-kolla-config" (OuterVolumeSpecName: "kolla-config") pod "6ac95bd3-125c-4966-b9a7-4eb4a7842c67" (UID: "6ac95bd3-125c-4966-b9a7-4eb4a7842c67"). InnerVolumeSpecName "kolla-config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:08:40 crc kubenswrapper[5052]: I1203 12:08:40.061303 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b34bb4d0-7906-42e8-a9aa-bd1a4203a312-kolla-config" (OuterVolumeSpecName: "kolla-config") pod "b34bb4d0-7906-42e8-a9aa-bd1a4203a312" (UID: "b34bb4d0-7906-42e8-a9aa-bd1a4203a312"). InnerVolumeSpecName "kolla-config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:08:40 crc kubenswrapper[5052]: I1203 12:08:40.061959 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6ac95bd3-125c-4966-b9a7-4eb4a7842c67-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "6ac95bd3-125c-4966-b9a7-4eb4a7842c67" (UID: "6ac95bd3-125c-4966-b9a7-4eb4a7842c67"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:08:40 crc kubenswrapper[5052]: I1203 12:08:40.062195 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"mysql-db\" (UniqueName: \"kubernetes.io/local-volume/local-storage07-crc\") pod \"b34bb4d0-7906-42e8-a9aa-bd1a4203a312\" (UID: \"b34bb4d0-7906-42e8-a9aa-bd1a4203a312\") " Dec 03 12:08:40 crc kubenswrapper[5052]: I1203 12:08:40.062339 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/6ac95bd3-125c-4966-b9a7-4eb4a7842c67-config-data-default\") pod \"6ac95bd3-125c-4966-b9a7-4eb4a7842c67\" (UID: \"6ac95bd3-125c-4966-b9a7-4eb4a7842c67\") " Dec 03 12:08:40 crc kubenswrapper[5052]: I1203 12:08:40.063445 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/6ac95bd3-125c-4966-b9a7-4eb4a7842c67-config-data-generated\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:40 crc kubenswrapper[5052]: I1203 12:08:40.063476 5052 reconciler_common.go:293] "Volume detached for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/6ac95bd3-125c-4966-b9a7-4eb4a7842c67-kolla-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:40 crc kubenswrapper[5052]: I1203 12:08:40.063490 5052 reconciler_common.go:293] "Volume detached for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/b34bb4d0-7906-42e8-a9aa-bd1a4203a312-kolla-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:40 crc kubenswrapper[5052]: I1203 12:08:40.063542 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/b34bb4d0-7906-42e8-a9aa-bd1a4203a312-config-data-generated\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:40 crc kubenswrapper[5052]: I1203 12:08:40.063563 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/b34bb4d0-7906-42e8-a9aa-bd1a4203a312-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:40 crc kubenswrapper[5052]: I1203 12:08:40.063575 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/b34bb4d0-7906-42e8-a9aa-bd1a4203a312-config-data-default\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:40 crc kubenswrapper[5052]: I1203 12:08:40.063588 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/6ac95bd3-125c-4966-b9a7-4eb4a7842c67-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:40 crc kubenswrapper[5052]: I1203 12:08:40.063874 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/glance8fcf-account-delete-4xhcc"] Dec 03 12:08:40 crc kubenswrapper[5052]: I1203 12:08:40.066332 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6ac95bd3-125c-4966-b9a7-4eb4a7842c67-config-data-default" (OuterVolumeSpecName: "config-data-default") pod "6ac95bd3-125c-4966-b9a7-4eb4a7842c67" (UID: "6ac95bd3-125c-4966-b9a7-4eb4a7842c67"). InnerVolumeSpecName "config-data-default". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:08:40 crc kubenswrapper[5052]: I1203 12:08:40.073860 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage07-crc" (OuterVolumeSpecName: "mysql-db") pod "b34bb4d0-7906-42e8-a9aa-bd1a4203a312" (UID: "b34bb4d0-7906-42e8-a9aa-bd1a4203a312"). InnerVolumeSpecName "local-storage07-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 12:08:40 crc kubenswrapper[5052]: I1203 12:08:40.074375 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/rabbitmq-server-2"] Dec 03 12:08:40 crc kubenswrapper[5052]: I1203 12:08:40.074563 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b34bb4d0-7906-42e8-a9aa-bd1a4203a312-kube-api-access-kmr95" (OuterVolumeSpecName: "kube-api-access-kmr95") pod "b34bb4d0-7906-42e8-a9aa-bd1a4203a312" (UID: "b34bb4d0-7906-42e8-a9aa-bd1a4203a312"). InnerVolumeSpecName "kube-api-access-kmr95". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:08:40 crc kubenswrapper[5052]: I1203 12:08:40.074788 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6ac95bd3-125c-4966-b9a7-4eb4a7842c67-kube-api-access-hqgfz" (OuterVolumeSpecName: "kube-api-access-hqgfz") pod "6ac95bd3-125c-4966-b9a7-4eb4a7842c67" (UID: "6ac95bd3-125c-4966-b9a7-4eb4a7842c67"). InnerVolumeSpecName "kube-api-access-hqgfz". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:08:40 crc kubenswrapper[5052]: I1203 12:08:40.082346 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage14-crc" (OuterVolumeSpecName: "mysql-db") pod "6ac95bd3-125c-4966-b9a7-4eb4a7842c67" (UID: "6ac95bd3-125c-4966-b9a7-4eb4a7842c67"). InnerVolumeSpecName "local-storage14-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 12:08:40 crc kubenswrapper[5052]: I1203 12:08:40.082872 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/rabbitmq-server-2"] Dec 03 12:08:40 crc kubenswrapper[5052]: I1203 12:08:40.086943 5052 scope.go:117] "RemoveContainer" containerID="b3fb4c5fa05d10cd90154cc20449cb5477be079f2a3ceaf70a240ebedfabea19" Dec 03 12:08:40 crc kubenswrapper[5052]: I1203 12:08:40.098068 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b34bb4d0-7906-42e8-a9aa-bd1a4203a312-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "b34bb4d0-7906-42e8-a9aa-bd1a4203a312" (UID: "b34bb4d0-7906-42e8-a9aa-bd1a4203a312"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:08:40 crc kubenswrapper[5052]: I1203 12:08:40.102157 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6ac95bd3-125c-4966-b9a7-4eb4a7842c67-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "6ac95bd3-125c-4966-b9a7-4eb4a7842c67" (UID: "6ac95bd3-125c-4966-b9a7-4eb4a7842c67"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:08:40 crc kubenswrapper[5052]: I1203 12:08:40.110967 5052 scope.go:117] "RemoveContainer" containerID="bc3186646511337244c91e6de61ef79f4b96f53697165b0fe223dd0c36cfeadf" Dec 03 12:08:40 crc kubenswrapper[5052]: E1203 12:08:40.111413 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bc3186646511337244c91e6de61ef79f4b96f53697165b0fe223dd0c36cfeadf\": container with ID starting with bc3186646511337244c91e6de61ef79f4b96f53697165b0fe223dd0c36cfeadf not found: ID does not exist" containerID="bc3186646511337244c91e6de61ef79f4b96f53697165b0fe223dd0c36cfeadf" Dec 03 12:08:40 crc kubenswrapper[5052]: I1203 12:08:40.111446 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bc3186646511337244c91e6de61ef79f4b96f53697165b0fe223dd0c36cfeadf"} err="failed to get container status \"bc3186646511337244c91e6de61ef79f4b96f53697165b0fe223dd0c36cfeadf\": rpc error: code = NotFound desc = could not find container \"bc3186646511337244c91e6de61ef79f4b96f53697165b0fe223dd0c36cfeadf\": container with ID starting with bc3186646511337244c91e6de61ef79f4b96f53697165b0fe223dd0c36cfeadf not found: ID does not exist" Dec 03 12:08:40 crc kubenswrapper[5052]: I1203 12:08:40.111468 5052 scope.go:117] "RemoveContainer" containerID="b3fb4c5fa05d10cd90154cc20449cb5477be079f2a3ceaf70a240ebedfabea19" Dec 03 12:08:40 crc kubenswrapper[5052]: E1203 12:08:40.111947 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b3fb4c5fa05d10cd90154cc20449cb5477be079f2a3ceaf70a240ebedfabea19\": container with ID starting with b3fb4c5fa05d10cd90154cc20449cb5477be079f2a3ceaf70a240ebedfabea19 not found: ID does not exist" containerID="b3fb4c5fa05d10cd90154cc20449cb5477be079f2a3ceaf70a240ebedfabea19" Dec 03 12:08:40 crc kubenswrapper[5052]: I1203 12:08:40.112031 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b3fb4c5fa05d10cd90154cc20449cb5477be079f2a3ceaf70a240ebedfabea19"} err="failed to get container status \"b3fb4c5fa05d10cd90154cc20449cb5477be079f2a3ceaf70a240ebedfabea19\": rpc error: code = NotFound desc = could not find container \"b3fb4c5fa05d10cd90154cc20449cb5477be079f2a3ceaf70a240ebedfabea19\": container with ID starting with b3fb4c5fa05d10cd90154cc20449cb5477be079f2a3ceaf70a240ebedfabea19 not found: ID does not exist" Dec 03 12:08:40 crc kubenswrapper[5052]: I1203 12:08:40.112086 5052 scope.go:117] "RemoveContainer" containerID="16dd1c84a3d482ce5adc97f8fdc99d27080aa0bd2d9fba51e986dc52caa0daf6" Dec 03 12:08:40 crc kubenswrapper[5052]: I1203 12:08:40.119194 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b34bb4d0-7906-42e8-a9aa-bd1a4203a312-galera-tls-certs" (OuterVolumeSpecName: "galera-tls-certs") pod "b34bb4d0-7906-42e8-a9aa-bd1a4203a312" (UID: "b34bb4d0-7906-42e8-a9aa-bd1a4203a312"). InnerVolumeSpecName "galera-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:08:40 crc kubenswrapper[5052]: I1203 12:08:40.129371 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6ac95bd3-125c-4966-b9a7-4eb4a7842c67-galera-tls-certs" (OuterVolumeSpecName: "galera-tls-certs") pod "6ac95bd3-125c-4966-b9a7-4eb4a7842c67" (UID: "6ac95bd3-125c-4966-b9a7-4eb4a7842c67"). InnerVolumeSpecName "galera-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:08:40 crc kubenswrapper[5052]: I1203 12:08:40.133978 5052 scope.go:117] "RemoveContainer" containerID="c716ca61940f2dfaed65a15c3a09f0b9191853485d4f681077e718bcb0869b41" Dec 03 12:08:40 crc kubenswrapper[5052]: I1203 12:08:40.151795 5052 scope.go:117] "RemoveContainer" containerID="ae74ed54ba186a0e9a780dc48252d0e6409d74b51697be203e1eefe16b5793fe" Dec 03 12:08:40 crc kubenswrapper[5052]: I1203 12:08:40.164910 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-kmr95\" (UniqueName: \"kubernetes.io/projected/b34bb4d0-7906-42e8-a9aa-bd1a4203a312-kube-api-access-kmr95\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:40 crc kubenswrapper[5052]: I1203 12:08:40.164941 5052 reconciler_common.go:293] "Volume detached for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/6ac95bd3-125c-4966-b9a7-4eb4a7842c67-galera-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:40 crc kubenswrapper[5052]: I1203 12:08:40.164951 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6ac95bd3-125c-4966-b9a7-4eb4a7842c67-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:40 crc kubenswrapper[5052]: I1203 12:08:40.164961 5052 reconciler_common.go:293] "Volume detached for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/b34bb4d0-7906-42e8-a9aa-bd1a4203a312-galera-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:40 crc kubenswrapper[5052]: I1203 12:08:40.164970 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/6ac95bd3-125c-4966-b9a7-4eb4a7842c67-config-data-default\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:40 crc kubenswrapper[5052]: I1203 12:08:40.165000 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage07-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage07-crc\") on node \"crc\" " Dec 03 12:08:40 crc kubenswrapper[5052]: I1203 12:08:40.165013 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-hqgfz\" (UniqueName: \"kubernetes.io/projected/6ac95bd3-125c-4966-b9a7-4eb4a7842c67-kube-api-access-hqgfz\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:40 crc kubenswrapper[5052]: I1203 12:08:40.165027 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage14-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage14-crc\") on node \"crc\" " Dec 03 12:08:40 crc kubenswrapper[5052]: I1203 12:08:40.165036 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b34bb4d0-7906-42e8-a9aa-bd1a4203a312-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:40 crc kubenswrapper[5052]: I1203 12:08:40.176606 5052 scope.go:117] "RemoveContainer" containerID="0790cbdcd312b0f701867eb108daefe7d9c2065afd80108a7c514db8349d75a3" Dec 03 12:08:40 crc kubenswrapper[5052]: I1203 12:08:40.186593 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage07-crc" (UniqueName: "kubernetes.io/local-volume/local-storage07-crc") on node "crc" Dec 03 12:08:40 crc kubenswrapper[5052]: I1203 12:08:40.209302 5052 scope.go:117] "RemoveContainer" containerID="aad64a682ded16a9f4098098d689931d1934e194e45eecdc993e62379504781b" Dec 03 12:08:40 crc kubenswrapper[5052]: I1203 12:08:40.214672 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage14-crc" (UniqueName: "kubernetes.io/local-volume/local-storage14-crc") on node "crc" Dec 03 12:08:40 crc kubenswrapper[5052]: I1203 12:08:40.267771 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage07-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage07-crc\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:40 crc kubenswrapper[5052]: I1203 12:08:40.268119 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage14-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage14-crc\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:40 crc kubenswrapper[5052]: I1203 12:08:40.288295 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/openstack-galera-0" podUID="d66ed4ec-fd97-4a62-82cb-cee3fe877524" containerName="galera" containerID="cri-o://7e2247eb34fdd68938be37e473102ca45e588792a2cc0096e976b082bdaee0f9" gracePeriod=23 Dec 03 12:08:40 crc kubenswrapper[5052]: I1203 12:08:40.296296 5052 scope.go:117] "RemoveContainer" containerID="142af50f0ab6083a28092a109ed34ea8ab8faf77ec5102e5c37130c564ddc949" Dec 03 12:08:40 crc kubenswrapper[5052]: I1203 12:08:40.524204 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:08:40 crc kubenswrapper[5052]: I1203 12:08:40.678991 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8n58j\" (UniqueName: \"kubernetes.io/projected/3b8ce5b7-67a1-48ef-892f-7408010a84f0-kube-api-access-8n58j\") pod \"3b8ce5b7-67a1-48ef-892f-7408010a84f0\" (UID: \"3b8ce5b7-67a1-48ef-892f-7408010a84f0\") " Dec 03 12:08:40 crc kubenswrapper[5052]: I1203 12:08:40.679434 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/3b8ce5b7-67a1-48ef-892f-7408010a84f0-config-data-generated\") pod \"3b8ce5b7-67a1-48ef-892f-7408010a84f0\" (UID: \"3b8ce5b7-67a1-48ef-892f-7408010a84f0\") " Dec 03 12:08:40 crc kubenswrapper[5052]: I1203 12:08:40.679480 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/3b8ce5b7-67a1-48ef-892f-7408010a84f0-kolla-config\") pod \"3b8ce5b7-67a1-48ef-892f-7408010a84f0\" (UID: \"3b8ce5b7-67a1-48ef-892f-7408010a84f0\") " Dec 03 12:08:40 crc kubenswrapper[5052]: I1203 12:08:40.679588 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/3b8ce5b7-67a1-48ef-892f-7408010a84f0-config-data-default\") pod \"3b8ce5b7-67a1-48ef-892f-7408010a84f0\" (UID: \"3b8ce5b7-67a1-48ef-892f-7408010a84f0\") " Dec 03 12:08:40 crc kubenswrapper[5052]: I1203 12:08:40.679627 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3b8ce5b7-67a1-48ef-892f-7408010a84f0-combined-ca-bundle\") pod \"3b8ce5b7-67a1-48ef-892f-7408010a84f0\" (UID: \"3b8ce5b7-67a1-48ef-892f-7408010a84f0\") " Dec 03 12:08:40 crc kubenswrapper[5052]: I1203 12:08:40.679736 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/3b8ce5b7-67a1-48ef-892f-7408010a84f0-galera-tls-certs\") pod \"3b8ce5b7-67a1-48ef-892f-7408010a84f0\" (UID: \"3b8ce5b7-67a1-48ef-892f-7408010a84f0\") " Dec 03 12:08:40 crc kubenswrapper[5052]: I1203 12:08:40.679777 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"mysql-db\" (UniqueName: \"kubernetes.io/local-volume/local-storage09-crc\") pod \"3b8ce5b7-67a1-48ef-892f-7408010a84f0\" (UID: \"3b8ce5b7-67a1-48ef-892f-7408010a84f0\") " Dec 03 12:08:40 crc kubenswrapper[5052]: I1203 12:08:40.679838 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/3b8ce5b7-67a1-48ef-892f-7408010a84f0-operator-scripts\") pod \"3b8ce5b7-67a1-48ef-892f-7408010a84f0\" (UID: \"3b8ce5b7-67a1-48ef-892f-7408010a84f0\") " Dec 03 12:08:40 crc kubenswrapper[5052]: I1203 12:08:40.681156 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3b8ce5b7-67a1-48ef-892f-7408010a84f0-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "3b8ce5b7-67a1-48ef-892f-7408010a84f0" (UID: "3b8ce5b7-67a1-48ef-892f-7408010a84f0"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:08:40 crc kubenswrapper[5052]: I1203 12:08:40.681938 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b8ce5b7-67a1-48ef-892f-7408010a84f0-config-data-generated" (OuterVolumeSpecName: "config-data-generated") pod "3b8ce5b7-67a1-48ef-892f-7408010a84f0" (UID: "3b8ce5b7-67a1-48ef-892f-7408010a84f0"). InnerVolumeSpecName "config-data-generated". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:08:40 crc kubenswrapper[5052]: I1203 12:08:40.682188 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3b8ce5b7-67a1-48ef-892f-7408010a84f0-kolla-config" (OuterVolumeSpecName: "kolla-config") pod "3b8ce5b7-67a1-48ef-892f-7408010a84f0" (UID: "3b8ce5b7-67a1-48ef-892f-7408010a84f0"). InnerVolumeSpecName "kolla-config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:08:40 crc kubenswrapper[5052]: I1203 12:08:40.682477 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3b8ce5b7-67a1-48ef-892f-7408010a84f0-config-data-default" (OuterVolumeSpecName: "config-data-default") pod "3b8ce5b7-67a1-48ef-892f-7408010a84f0" (UID: "3b8ce5b7-67a1-48ef-892f-7408010a84f0"). InnerVolumeSpecName "config-data-default". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:08:40 crc kubenswrapper[5052]: I1203 12:08:40.685124 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3b8ce5b7-67a1-48ef-892f-7408010a84f0-kube-api-access-8n58j" (OuterVolumeSpecName: "kube-api-access-8n58j") pod "3b8ce5b7-67a1-48ef-892f-7408010a84f0" (UID: "3b8ce5b7-67a1-48ef-892f-7408010a84f0"). InnerVolumeSpecName "kube-api-access-8n58j". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:08:40 crc kubenswrapper[5052]: I1203 12:08:40.691085 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage09-crc" (OuterVolumeSpecName: "mysql-db") pod "3b8ce5b7-67a1-48ef-892f-7408010a84f0" (UID: "3b8ce5b7-67a1-48ef-892f-7408010a84f0"). InnerVolumeSpecName "local-storage09-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 12:08:40 crc kubenswrapper[5052]: I1203 12:08:40.706366 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-cell1-galera-2" event={"ID":"6ac95bd3-125c-4966-b9a7-4eb4a7842c67","Type":"ContainerDied","Data":"5d0618ff0b0c564d02f8c132d2bdf496a4eb2cd52871397cdc06d0a57306a948"} Dec 03 12:08:40 crc kubenswrapper[5052]: I1203 12:08:40.706421 5052 scope.go:117] "RemoveContainer" containerID="edf48c4037d4be3adfc21b5d97a50d68b5261315fe92282773dd8556852ce51a" Dec 03 12:08:40 crc kubenswrapper[5052]: I1203 12:08:40.706575 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstack-cell1-galera-2" Dec 03 12:08:40 crc kubenswrapper[5052]: I1203 12:08:40.715422 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3b8ce5b7-67a1-48ef-892f-7408010a84f0-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "3b8ce5b7-67a1-48ef-892f-7408010a84f0" (UID: "3b8ce5b7-67a1-48ef-892f-7408010a84f0"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:08:40 crc kubenswrapper[5052]: I1203 12:08:40.724413 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-cell1-galera-1" event={"ID":"b34bb4d0-7906-42e8-a9aa-bd1a4203a312","Type":"ContainerDied","Data":"69e7ac97fd0fcb77437d78b9a5a2c861a52f2027179a08c9a2487f6d93b9a0da"} Dec 03 12:08:40 crc kubenswrapper[5052]: I1203 12:08:40.724571 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstack-cell1-galera-1" Dec 03 12:08:40 crc kubenswrapper[5052]: I1203 12:08:40.725549 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3b8ce5b7-67a1-48ef-892f-7408010a84f0-galera-tls-certs" (OuterVolumeSpecName: "galera-tls-certs") pod "3b8ce5b7-67a1-48ef-892f-7408010a84f0" (UID: "3b8ce5b7-67a1-48ef-892f-7408010a84f0"). InnerVolumeSpecName "galera-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:08:40 crc kubenswrapper[5052]: I1203 12:08:40.730541 5052 generic.go:334] "Generic (PLEG): container finished" podID="3b8ce5b7-67a1-48ef-892f-7408010a84f0" containerID="5a5421259d7095f5c7b35c6d5fcacfbf43056034387141eb0830f2281a204092" exitCode=0 Dec 03 12:08:40 crc kubenswrapper[5052]: I1203 12:08:40.730577 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-cell1-galera-0" event={"ID":"3b8ce5b7-67a1-48ef-892f-7408010a84f0","Type":"ContainerDied","Data":"5a5421259d7095f5c7b35c6d5fcacfbf43056034387141eb0830f2281a204092"} Dec 03 12:08:40 crc kubenswrapper[5052]: I1203 12:08:40.730598 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-cell1-galera-0" event={"ID":"3b8ce5b7-67a1-48ef-892f-7408010a84f0","Type":"ContainerDied","Data":"96fc8571648dfbfa0bf3c3f62c3ad7b38970d612280158cac29836794efa1ab8"} Dec 03 12:08:40 crc kubenswrapper[5052]: I1203 12:08:40.730601 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:08:40 crc kubenswrapper[5052]: I1203 12:08:40.781901 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-8n58j\" (UniqueName: \"kubernetes.io/projected/3b8ce5b7-67a1-48ef-892f-7408010a84f0-kube-api-access-8n58j\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:40 crc kubenswrapper[5052]: I1203 12:08:40.781956 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/3b8ce5b7-67a1-48ef-892f-7408010a84f0-config-data-generated\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:40 crc kubenswrapper[5052]: I1203 12:08:40.781969 5052 reconciler_common.go:293] "Volume detached for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/3b8ce5b7-67a1-48ef-892f-7408010a84f0-kolla-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:40 crc kubenswrapper[5052]: I1203 12:08:40.781982 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/3b8ce5b7-67a1-48ef-892f-7408010a84f0-config-data-default\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:40 crc kubenswrapper[5052]: I1203 12:08:40.781995 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3b8ce5b7-67a1-48ef-892f-7408010a84f0-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:40 crc kubenswrapper[5052]: I1203 12:08:40.782004 5052 reconciler_common.go:293] "Volume detached for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/3b8ce5b7-67a1-48ef-892f-7408010a84f0-galera-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:40 crc kubenswrapper[5052]: I1203 12:08:40.782025 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage09-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage09-crc\") on node \"crc\" " Dec 03 12:08:40 crc kubenswrapper[5052]: I1203 12:08:40.782034 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/3b8ce5b7-67a1-48ef-892f-7408010a84f0-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:40 crc kubenswrapper[5052]: I1203 12:08:40.793049 5052 scope.go:117] "RemoveContainer" containerID="6f25f4851b71de7e5136ec08e6aa9f8b2530fdd3a88edfd5b32d7294b08cdcfa" Dec 03 12:08:40 crc kubenswrapper[5052]: I1203 12:08:40.800151 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/openstack-cell1-galera-1"] Dec 03 12:08:40 crc kubenswrapper[5052]: I1203 12:08:40.814887 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/openstack-cell1-galera-1"] Dec 03 12:08:40 crc kubenswrapper[5052]: I1203 12:08:40.824732 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/openstack-cell1-galera-2"] Dec 03 12:08:40 crc kubenswrapper[5052]: I1203 12:08:40.831773 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage09-crc" (UniqueName: "kubernetes.io/local-volume/local-storage09-crc") on node "crc" Dec 03 12:08:40 crc kubenswrapper[5052]: I1203 12:08:40.833123 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/openstack-cell1-galera-2"] Dec 03 12:08:40 crc kubenswrapper[5052]: I1203 12:08:40.837577 5052 scope.go:117] "RemoveContainer" containerID="c55564d5848c1a333ba2db20a9c5ecdbf152ba4ebaa847c6e28f6ed69c3a99ed" Dec 03 12:08:40 crc kubenswrapper[5052]: I1203 12:08:40.840360 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/openstack-cell1-galera-0"] Dec 03 12:08:40 crc kubenswrapper[5052]: I1203 12:08:40.847625 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/openstack-cell1-galera-0"] Dec 03 12:08:40 crc kubenswrapper[5052]: I1203 12:08:40.882630 5052 scope.go:117] "RemoveContainer" containerID="cfa78e68ee04705ba5099d5662bbfe0cc1eb410ee1f31655537d62c888b13793" Dec 03 12:08:40 crc kubenswrapper[5052]: I1203 12:08:40.883878 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage09-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage09-crc\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:40 crc kubenswrapper[5052]: I1203 12:08:40.907721 5052 scope.go:117] "RemoveContainer" containerID="5a5421259d7095f5c7b35c6d5fcacfbf43056034387141eb0830f2281a204092" Dec 03 12:08:40 crc kubenswrapper[5052]: I1203 12:08:40.949091 5052 scope.go:117] "RemoveContainer" containerID="fc6a07b3ee2f8d37a641173f6de8b77d45fd5bc8c9701900d684dde2e6532818" Dec 03 12:08:40 crc kubenswrapper[5052]: I1203 12:08:40.973042 5052 scope.go:117] "RemoveContainer" containerID="5a5421259d7095f5c7b35c6d5fcacfbf43056034387141eb0830f2281a204092" Dec 03 12:08:40 crc kubenswrapper[5052]: E1203 12:08:40.973558 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5a5421259d7095f5c7b35c6d5fcacfbf43056034387141eb0830f2281a204092\": container with ID starting with 5a5421259d7095f5c7b35c6d5fcacfbf43056034387141eb0830f2281a204092 not found: ID does not exist" containerID="5a5421259d7095f5c7b35c6d5fcacfbf43056034387141eb0830f2281a204092" Dec 03 12:08:40 crc kubenswrapper[5052]: I1203 12:08:40.973598 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5a5421259d7095f5c7b35c6d5fcacfbf43056034387141eb0830f2281a204092"} err="failed to get container status \"5a5421259d7095f5c7b35c6d5fcacfbf43056034387141eb0830f2281a204092\": rpc error: code = NotFound desc = could not find container \"5a5421259d7095f5c7b35c6d5fcacfbf43056034387141eb0830f2281a204092\": container with ID starting with 5a5421259d7095f5c7b35c6d5fcacfbf43056034387141eb0830f2281a204092 not found: ID does not exist" Dec 03 12:08:40 crc kubenswrapper[5052]: I1203 12:08:40.973621 5052 scope.go:117] "RemoveContainer" containerID="fc6a07b3ee2f8d37a641173f6de8b77d45fd5bc8c9701900d684dde2e6532818" Dec 03 12:08:40 crc kubenswrapper[5052]: E1203 12:08:40.973879 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fc6a07b3ee2f8d37a641173f6de8b77d45fd5bc8c9701900d684dde2e6532818\": container with ID starting with fc6a07b3ee2f8d37a641173f6de8b77d45fd5bc8c9701900d684dde2e6532818 not found: ID does not exist" containerID="fc6a07b3ee2f8d37a641173f6de8b77d45fd5bc8c9701900d684dde2e6532818" Dec 03 12:08:40 crc kubenswrapper[5052]: I1203 12:08:40.973904 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fc6a07b3ee2f8d37a641173f6de8b77d45fd5bc8c9701900d684dde2e6532818"} err="failed to get container status \"fc6a07b3ee2f8d37a641173f6de8b77d45fd5bc8c9701900d684dde2e6532818\": rpc error: code = NotFound desc = could not find container \"fc6a07b3ee2f8d37a641173f6de8b77d45fd5bc8c9701900d684dde2e6532818\": container with ID starting with fc6a07b3ee2f8d37a641173f6de8b77d45fd5bc8c9701900d684dde2e6532818 not found: ID does not exist" Dec 03 12:08:41 crc kubenswrapper[5052]: I1203 12:08:41.701019 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="06bfd2db-abe0-4b4d-a110-94a6a1e02211" path="/var/lib/kubelet/pods/06bfd2db-abe0-4b4d-a110-94a6a1e02211/volumes" Dec 03 12:08:41 crc kubenswrapper[5052]: I1203 12:08:41.701975 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2397d896-0d11-4656-a336-eac8ec9fe7f9" path="/var/lib/kubelet/pods/2397d896-0d11-4656-a336-eac8ec9fe7f9/volumes" Dec 03 12:08:41 crc kubenswrapper[5052]: I1203 12:08:41.703764 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2f3b4f6a-5f11-4039-a3bd-7b553ab5684d" path="/var/lib/kubelet/pods/2f3b4f6a-5f11-4039-a3bd-7b553ab5684d/volumes" Dec 03 12:08:41 crc kubenswrapper[5052]: I1203 12:08:41.705700 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3b8ce5b7-67a1-48ef-892f-7408010a84f0" path="/var/lib/kubelet/pods/3b8ce5b7-67a1-48ef-892f-7408010a84f0/volumes" Dec 03 12:08:41 crc kubenswrapper[5052]: I1203 12:08:41.706526 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6ac95bd3-125c-4966-b9a7-4eb4a7842c67" path="/var/lib/kubelet/pods/6ac95bd3-125c-4966-b9a7-4eb4a7842c67/volumes" Dec 03 12:08:41 crc kubenswrapper[5052]: I1203 12:08:41.707711 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="77077fe5-d1ff-49c4-b049-3a0f2cee23b0" path="/var/lib/kubelet/pods/77077fe5-d1ff-49c4-b049-3a0f2cee23b0/volumes" Dec 03 12:08:41 crc kubenswrapper[5052]: I1203 12:08:41.708460 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7901ce85-4e6f-4e8b-86a6-f50a85803637" path="/var/lib/kubelet/pods/7901ce85-4e6f-4e8b-86a6-f50a85803637/volumes" Dec 03 12:08:41 crc kubenswrapper[5052]: I1203 12:08:41.709901 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7b4f6078-41a3-433d-acaf-f2d67391ab6e" path="/var/lib/kubelet/pods/7b4f6078-41a3-433d-acaf-f2d67391ab6e/volumes" Dec 03 12:08:41 crc kubenswrapper[5052]: I1203 12:08:41.710726 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="89ee1bab-2c68-4fc4-9171-a7330922faf6" path="/var/lib/kubelet/pods/89ee1bab-2c68-4fc4-9171-a7330922faf6/volumes" Dec 03 12:08:41 crc kubenswrapper[5052]: I1203 12:08:41.711366 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b34bb4d0-7906-42e8-a9aa-bd1a4203a312" path="/var/lib/kubelet/pods/b34bb4d0-7906-42e8-a9aa-bd1a4203a312/volumes" Dec 03 12:08:41 crc kubenswrapper[5052]: I1203 12:08:41.712558 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3" path="/var/lib/kubelet/pods/f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3/volumes" Dec 03 12:08:41 crc kubenswrapper[5052]: I1203 12:08:41.745023 5052 generic.go:334] "Generic (PLEG): container finished" podID="143f4f1d-4f91-4ffa-a277-d6e551d19838" containerID="a154e1968b2288216f60d8cd5f0368f47e36bcdef4f248bc7568b9b7174e4331" exitCode=0 Dec 03 12:08:41 crc kubenswrapper[5052]: I1203 12:08:41.745057 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-galera-2" event={"ID":"143f4f1d-4f91-4ffa-a277-d6e551d19838","Type":"ContainerDied","Data":"a154e1968b2288216f60d8cd5f0368f47e36bcdef4f248bc7568b9b7174e4331"} Dec 03 12:08:41 crc kubenswrapper[5052]: E1203 12:08:41.797980 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/rabbitmq-cell1-config-data: configmap "rabbitmq-cell1-config-data" not found Dec 03 12:08:41 crc kubenswrapper[5052]: E1203 12:08:41.798039 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/185fdd2a-cd57-45e5-9986-bebd15e5761f-config-data podName:185fdd2a-cd57-45e5-9986-bebd15e5761f nodeName:}" failed. No retries permitted until 2025-12-03 12:08:57.798024886 +0000 UTC m=+2928.386266650 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "config-data" (UniqueName: "kubernetes.io/configmap/185fdd2a-cd57-45e5-9986-bebd15e5761f-config-data") pod "rabbitmq-cell1-server-0" (UID: "185fdd2a-cd57-45e5-9986-bebd15e5761f") : configmap "rabbitmq-cell1-config-data" not found Dec 03 12:08:41 crc kubenswrapper[5052]: I1203 12:08:41.921940 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstack-galera-2" Dec 03 12:08:42 crc kubenswrapper[5052]: I1203 12:08:41.999799 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/143f4f1d-4f91-4ffa-a277-d6e551d19838-galera-tls-certs\") pod \"143f4f1d-4f91-4ffa-a277-d6e551d19838\" (UID: \"143f4f1d-4f91-4ffa-a277-d6e551d19838\") " Dec 03 12:08:42 crc kubenswrapper[5052]: I1203 12:08:42.000228 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/143f4f1d-4f91-4ffa-a277-d6e551d19838-combined-ca-bundle\") pod \"143f4f1d-4f91-4ffa-a277-d6e551d19838\" (UID: \"143f4f1d-4f91-4ffa-a277-d6e551d19838\") " Dec 03 12:08:42 crc kubenswrapper[5052]: I1203 12:08:42.000690 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-897ts\" (UniqueName: \"kubernetes.io/projected/143f4f1d-4f91-4ffa-a277-d6e551d19838-kube-api-access-897ts\") pod \"143f4f1d-4f91-4ffa-a277-d6e551d19838\" (UID: \"143f4f1d-4f91-4ffa-a277-d6e551d19838\") " Dec 03 12:08:42 crc kubenswrapper[5052]: I1203 12:08:42.000843 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/143f4f1d-4f91-4ffa-a277-d6e551d19838-config-data-generated\") pod \"143f4f1d-4f91-4ffa-a277-d6e551d19838\" (UID: \"143f4f1d-4f91-4ffa-a277-d6e551d19838\") " Dec 03 12:08:42 crc kubenswrapper[5052]: I1203 12:08:42.001554 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/143f4f1d-4f91-4ffa-a277-d6e551d19838-config-data-generated" (OuterVolumeSpecName: "config-data-generated") pod "143f4f1d-4f91-4ffa-a277-d6e551d19838" (UID: "143f4f1d-4f91-4ffa-a277-d6e551d19838"). InnerVolumeSpecName "config-data-generated". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:08:42 crc kubenswrapper[5052]: I1203 12:08:42.001664 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"mysql-db\" (UniqueName: \"kubernetes.io/local-volume/local-storage08-crc\") pod \"143f4f1d-4f91-4ffa-a277-d6e551d19838\" (UID: \"143f4f1d-4f91-4ffa-a277-d6e551d19838\") " Dec 03 12:08:42 crc kubenswrapper[5052]: I1203 12:08:42.002058 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/143f4f1d-4f91-4ffa-a277-d6e551d19838-operator-scripts\") pod \"143f4f1d-4f91-4ffa-a277-d6e551d19838\" (UID: \"143f4f1d-4f91-4ffa-a277-d6e551d19838\") " Dec 03 12:08:42 crc kubenswrapper[5052]: I1203 12:08:42.002121 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/143f4f1d-4f91-4ffa-a277-d6e551d19838-kolla-config\") pod \"143f4f1d-4f91-4ffa-a277-d6e551d19838\" (UID: \"143f4f1d-4f91-4ffa-a277-d6e551d19838\") " Dec 03 12:08:42 crc kubenswrapper[5052]: I1203 12:08:42.002163 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/143f4f1d-4f91-4ffa-a277-d6e551d19838-config-data-default\") pod \"143f4f1d-4f91-4ffa-a277-d6e551d19838\" (UID: \"143f4f1d-4f91-4ffa-a277-d6e551d19838\") " Dec 03 12:08:42 crc kubenswrapper[5052]: I1203 12:08:42.002911 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/143f4f1d-4f91-4ffa-a277-d6e551d19838-config-data-generated\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:42 crc kubenswrapper[5052]: I1203 12:08:42.003026 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/143f4f1d-4f91-4ffa-a277-d6e551d19838-kolla-config" (OuterVolumeSpecName: "kolla-config") pod "143f4f1d-4f91-4ffa-a277-d6e551d19838" (UID: "143f4f1d-4f91-4ffa-a277-d6e551d19838"). InnerVolumeSpecName "kolla-config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:08:42 crc kubenswrapper[5052]: I1203 12:08:42.003077 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/143f4f1d-4f91-4ffa-a277-d6e551d19838-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "143f4f1d-4f91-4ffa-a277-d6e551d19838" (UID: "143f4f1d-4f91-4ffa-a277-d6e551d19838"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:08:42 crc kubenswrapper[5052]: I1203 12:08:42.003569 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/143f4f1d-4f91-4ffa-a277-d6e551d19838-config-data-default" (OuterVolumeSpecName: "config-data-default") pod "143f4f1d-4f91-4ffa-a277-d6e551d19838" (UID: "143f4f1d-4f91-4ffa-a277-d6e551d19838"). InnerVolumeSpecName "config-data-default". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:08:42 crc kubenswrapper[5052]: I1203 12:08:42.012019 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage08-crc" (OuterVolumeSpecName: "mysql-db") pod "143f4f1d-4f91-4ffa-a277-d6e551d19838" (UID: "143f4f1d-4f91-4ffa-a277-d6e551d19838"). InnerVolumeSpecName "local-storage08-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 12:08:42 crc kubenswrapper[5052]: I1203 12:08:42.022449 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/143f4f1d-4f91-4ffa-a277-d6e551d19838-kube-api-access-897ts" (OuterVolumeSpecName: "kube-api-access-897ts") pod "143f4f1d-4f91-4ffa-a277-d6e551d19838" (UID: "143f4f1d-4f91-4ffa-a277-d6e551d19838"). InnerVolumeSpecName "kube-api-access-897ts". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:08:42 crc kubenswrapper[5052]: I1203 12:08:42.028151 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/143f4f1d-4f91-4ffa-a277-d6e551d19838-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "143f4f1d-4f91-4ffa-a277-d6e551d19838" (UID: "143f4f1d-4f91-4ffa-a277-d6e551d19838"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:08:42 crc kubenswrapper[5052]: I1203 12:08:42.045723 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/143f4f1d-4f91-4ffa-a277-d6e551d19838-galera-tls-certs" (OuterVolumeSpecName: "galera-tls-certs") pod "143f4f1d-4f91-4ffa-a277-d6e551d19838" (UID: "143f4f1d-4f91-4ffa-a277-d6e551d19838"). InnerVolumeSpecName "galera-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:08:42 crc kubenswrapper[5052]: I1203 12:08:42.104347 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage08-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage08-crc\") on node \"crc\" " Dec 03 12:08:42 crc kubenswrapper[5052]: I1203 12:08:42.104384 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/143f4f1d-4f91-4ffa-a277-d6e551d19838-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:42 crc kubenswrapper[5052]: I1203 12:08:42.104395 5052 reconciler_common.go:293] "Volume detached for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/143f4f1d-4f91-4ffa-a277-d6e551d19838-kolla-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:42 crc kubenswrapper[5052]: I1203 12:08:42.104403 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/143f4f1d-4f91-4ffa-a277-d6e551d19838-config-data-default\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:42 crc kubenswrapper[5052]: I1203 12:08:42.104417 5052 reconciler_common.go:293] "Volume detached for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/143f4f1d-4f91-4ffa-a277-d6e551d19838-galera-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:42 crc kubenswrapper[5052]: I1203 12:08:42.104426 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/143f4f1d-4f91-4ffa-a277-d6e551d19838-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:42 crc kubenswrapper[5052]: I1203 12:08:42.104435 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-897ts\" (UniqueName: \"kubernetes.io/projected/143f4f1d-4f91-4ffa-a277-d6e551d19838-kube-api-access-897ts\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:42 crc kubenswrapper[5052]: I1203 12:08:42.122962 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage08-crc" (UniqueName: "kubernetes.io/local-volume/local-storage08-crc") on node "crc" Dec 03 12:08:42 crc kubenswrapper[5052]: I1203 12:08:42.205648 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage08-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage08-crc\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:43 crc kubenswrapper[5052]: I1203 12:08:42.788184 5052 generic.go:334] "Generic (PLEG): container finished" podID="d66ed4ec-fd97-4a62-82cb-cee3fe877524" containerID="7e2247eb34fdd68938be37e473102ca45e588792a2cc0096e976b082bdaee0f9" exitCode=0 Dec 03 12:08:43 crc kubenswrapper[5052]: I1203 12:08:42.788233 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-galera-0" event={"ID":"d66ed4ec-fd97-4a62-82cb-cee3fe877524","Type":"ContainerDied","Data":"7e2247eb34fdd68938be37e473102ca45e588792a2cc0096e976b082bdaee0f9"} Dec 03 12:08:43 crc kubenswrapper[5052]: I1203 12:08:42.799419 5052 generic.go:334] "Generic (PLEG): container finished" podID="6b92f88e-1325-4c3d-8d4e-4089a9033775" containerID="527038c3f4433583ba7d8bdd06bfde009abf48c5c09978d5bdd3df6912110859" exitCode=0 Dec 03 12:08:43 crc kubenswrapper[5052]: I1203 12:08:42.799482 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-galera-1" event={"ID":"6b92f88e-1325-4c3d-8d4e-4089a9033775","Type":"ContainerDied","Data":"527038c3f4433583ba7d8bdd06bfde009abf48c5c09978d5bdd3df6912110859"} Dec 03 12:08:43 crc kubenswrapper[5052]: I1203 12:08:42.802190 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-galera-2" event={"ID":"143f4f1d-4f91-4ffa-a277-d6e551d19838","Type":"ContainerDied","Data":"734f7beb409866abb41ccc1b21dd823ca31dcff8c3df4adc426e457918b2ba55"} Dec 03 12:08:43 crc kubenswrapper[5052]: I1203 12:08:42.802222 5052 scope.go:117] "RemoveContainer" containerID="a154e1968b2288216f60d8cd5f0368f47e36bcdef4f248bc7568b9b7174e4331" Dec 03 12:08:43 crc kubenswrapper[5052]: I1203 12:08:42.802363 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstack-galera-2" Dec 03 12:08:43 crc kubenswrapper[5052]: I1203 12:08:42.849279 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/openstack-galera-2"] Dec 03 12:08:43 crc kubenswrapper[5052]: I1203 12:08:42.859215 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/openstack-galera-2"] Dec 03 12:08:43 crc kubenswrapper[5052]: I1203 12:08:42.863445 5052 scope.go:117] "RemoveContainer" containerID="218bda99a56c19c275c25f001fccbc4fcd8937679bb8bced56c2b578dae2ff3a" Dec 03 12:08:43 crc kubenswrapper[5052]: I1203 12:08:42.959267 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstack-galera-1" Dec 03 12:08:43 crc kubenswrapper[5052]: I1203 12:08:43.065362 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"mysql-db\" (UniqueName: \"kubernetes.io/local-volume/local-storage06-crc\") pod \"6b92f88e-1325-4c3d-8d4e-4089a9033775\" (UID: \"6b92f88e-1325-4c3d-8d4e-4089a9033775\") " Dec 03 12:08:43 crc kubenswrapper[5052]: I1203 12:08:43.065470 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/6b92f88e-1325-4c3d-8d4e-4089a9033775-config-data-default\") pod \"6b92f88e-1325-4c3d-8d4e-4089a9033775\" (UID: \"6b92f88e-1325-4c3d-8d4e-4089a9033775\") " Dec 03 12:08:43 crc kubenswrapper[5052]: I1203 12:08:43.065532 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/6b92f88e-1325-4c3d-8d4e-4089a9033775-config-data-generated\") pod \"6b92f88e-1325-4c3d-8d4e-4089a9033775\" (UID: \"6b92f88e-1325-4c3d-8d4e-4089a9033775\") " Dec 03 12:08:43 crc kubenswrapper[5052]: I1203 12:08:43.065556 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/6b92f88e-1325-4c3d-8d4e-4089a9033775-kolla-config\") pod \"6b92f88e-1325-4c3d-8d4e-4089a9033775\" (UID: \"6b92f88e-1325-4c3d-8d4e-4089a9033775\") " Dec 03 12:08:43 crc kubenswrapper[5052]: I1203 12:08:43.065591 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/6b92f88e-1325-4c3d-8d4e-4089a9033775-galera-tls-certs\") pod \"6b92f88e-1325-4c3d-8d4e-4089a9033775\" (UID: \"6b92f88e-1325-4c3d-8d4e-4089a9033775\") " Dec 03 12:08:43 crc kubenswrapper[5052]: I1203 12:08:43.065647 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/6b92f88e-1325-4c3d-8d4e-4089a9033775-operator-scripts\") pod \"6b92f88e-1325-4c3d-8d4e-4089a9033775\" (UID: \"6b92f88e-1325-4c3d-8d4e-4089a9033775\") " Dec 03 12:08:43 crc kubenswrapper[5052]: I1203 12:08:43.065728 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d29gn\" (UniqueName: \"kubernetes.io/projected/6b92f88e-1325-4c3d-8d4e-4089a9033775-kube-api-access-d29gn\") pod \"6b92f88e-1325-4c3d-8d4e-4089a9033775\" (UID: \"6b92f88e-1325-4c3d-8d4e-4089a9033775\") " Dec 03 12:08:43 crc kubenswrapper[5052]: I1203 12:08:43.065760 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6b92f88e-1325-4c3d-8d4e-4089a9033775-combined-ca-bundle\") pod \"6b92f88e-1325-4c3d-8d4e-4089a9033775\" (UID: \"6b92f88e-1325-4c3d-8d4e-4089a9033775\") " Dec 03 12:08:43 crc kubenswrapper[5052]: I1203 12:08:43.066012 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6b92f88e-1325-4c3d-8d4e-4089a9033775-config-data-default" (OuterVolumeSpecName: "config-data-default") pod "6b92f88e-1325-4c3d-8d4e-4089a9033775" (UID: "6b92f88e-1325-4c3d-8d4e-4089a9033775"). InnerVolumeSpecName "config-data-default". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:08:43 crc kubenswrapper[5052]: I1203 12:08:43.066870 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b92f88e-1325-4c3d-8d4e-4089a9033775-config-data-generated" (OuterVolumeSpecName: "config-data-generated") pod "6b92f88e-1325-4c3d-8d4e-4089a9033775" (UID: "6b92f88e-1325-4c3d-8d4e-4089a9033775"). InnerVolumeSpecName "config-data-generated". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:08:43 crc kubenswrapper[5052]: I1203 12:08:43.067271 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6b92f88e-1325-4c3d-8d4e-4089a9033775-kolla-config" (OuterVolumeSpecName: "kolla-config") pod "6b92f88e-1325-4c3d-8d4e-4089a9033775" (UID: "6b92f88e-1325-4c3d-8d4e-4089a9033775"). InnerVolumeSpecName "kolla-config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:08:43 crc kubenswrapper[5052]: I1203 12:08:43.067493 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6b92f88e-1325-4c3d-8d4e-4089a9033775-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "6b92f88e-1325-4c3d-8d4e-4089a9033775" (UID: "6b92f88e-1325-4c3d-8d4e-4089a9033775"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:08:43 crc kubenswrapper[5052]: I1203 12:08:43.067675 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/6b92f88e-1325-4c3d-8d4e-4089a9033775-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:43 crc kubenswrapper[5052]: I1203 12:08:43.067690 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/6b92f88e-1325-4c3d-8d4e-4089a9033775-config-data-default\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:43 crc kubenswrapper[5052]: I1203 12:08:43.067702 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/6b92f88e-1325-4c3d-8d4e-4089a9033775-config-data-generated\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:43 crc kubenswrapper[5052]: I1203 12:08:43.067713 5052 reconciler_common.go:293] "Volume detached for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/6b92f88e-1325-4c3d-8d4e-4089a9033775-kolla-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:43 crc kubenswrapper[5052]: I1203 12:08:43.071323 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6b92f88e-1325-4c3d-8d4e-4089a9033775-kube-api-access-d29gn" (OuterVolumeSpecName: "kube-api-access-d29gn") pod "6b92f88e-1325-4c3d-8d4e-4089a9033775" (UID: "6b92f88e-1325-4c3d-8d4e-4089a9033775"). InnerVolumeSpecName "kube-api-access-d29gn". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:08:43 crc kubenswrapper[5052]: I1203 12:08:43.076533 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage06-crc" (OuterVolumeSpecName: "mysql-db") pod "6b92f88e-1325-4c3d-8d4e-4089a9033775" (UID: "6b92f88e-1325-4c3d-8d4e-4089a9033775"). InnerVolumeSpecName "local-storage06-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 12:08:43 crc kubenswrapper[5052]: I1203 12:08:43.089703 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6b92f88e-1325-4c3d-8d4e-4089a9033775-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "6b92f88e-1325-4c3d-8d4e-4089a9033775" (UID: "6b92f88e-1325-4c3d-8d4e-4089a9033775"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:08:43 crc kubenswrapper[5052]: I1203 12:08:43.110278 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6b92f88e-1325-4c3d-8d4e-4089a9033775-galera-tls-certs" (OuterVolumeSpecName: "galera-tls-certs") pod "6b92f88e-1325-4c3d-8d4e-4089a9033775" (UID: "6b92f88e-1325-4c3d-8d4e-4089a9033775"). InnerVolumeSpecName "galera-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:08:43 crc kubenswrapper[5052]: I1203 12:08:43.169222 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage06-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage06-crc\") on node \"crc\" " Dec 03 12:08:43 crc kubenswrapper[5052]: I1203 12:08:43.169257 5052 reconciler_common.go:293] "Volume detached for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/6b92f88e-1325-4c3d-8d4e-4089a9033775-galera-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:43 crc kubenswrapper[5052]: I1203 12:08:43.169270 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-d29gn\" (UniqueName: \"kubernetes.io/projected/6b92f88e-1325-4c3d-8d4e-4089a9033775-kube-api-access-d29gn\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:43 crc kubenswrapper[5052]: I1203 12:08:43.169279 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6b92f88e-1325-4c3d-8d4e-4089a9033775-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:43 crc kubenswrapper[5052]: I1203 12:08:43.191249 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage06-crc" (UniqueName: "kubernetes.io/local-volume/local-storage06-crc") on node "crc" Dec 03 12:08:43 crc kubenswrapper[5052]: I1203 12:08:43.270367 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage06-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage06-crc\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:43 crc kubenswrapper[5052]: I1203 12:08:43.436357 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" podUID="185fdd2a-cd57-45e5-9986-bebd15e5761f" containerName="rabbitmq" probeResult="failure" output="dial tcp 10.217.0.104:5671: connect: connection refused" Dec 03 12:08:43 crc kubenswrapper[5052]: I1203 12:08:43.684974 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="143f4f1d-4f91-4ffa-a277-d6e551d19838" path="/var/lib/kubelet/pods/143f4f1d-4f91-4ffa-a277-d6e551d19838/volumes" Dec 03 12:08:43 crc kubenswrapper[5052]: I1203 12:08:43.814576 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-galera-1" event={"ID":"6b92f88e-1325-4c3d-8d4e-4089a9033775","Type":"ContainerDied","Data":"01c655f248edf79273060fb5f909b338b166fc31d6c5a9a9785381a3b0ba8dac"} Dec 03 12:08:43 crc kubenswrapper[5052]: I1203 12:08:43.814679 5052 scope.go:117] "RemoveContainer" containerID="527038c3f4433583ba7d8bdd06bfde009abf48c5c09978d5bdd3df6912110859" Dec 03 12:08:43 crc kubenswrapper[5052]: I1203 12:08:43.817713 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstack-galera-1" Dec 03 12:08:43 crc kubenswrapper[5052]: I1203 12:08:43.892709 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:08:43 crc kubenswrapper[5052]: I1203 12:08:43.902769 5052 scope.go:117] "RemoveContainer" containerID="797aefb21d4fd0eef8fa64544950d707d51b6474ea9063aae71828835bd51da9" Dec 03 12:08:43 crc kubenswrapper[5052]: I1203 12:08:43.915796 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/openstack-galera-1"] Dec 03 12:08:43 crc kubenswrapper[5052]: I1203 12:08:43.921406 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/openstack-galera-1"] Dec 03 12:08:43 crc kubenswrapper[5052]: I1203 12:08:43.981410 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d66ed4ec-fd97-4a62-82cb-cee3fe877524-combined-ca-bundle\") pod \"d66ed4ec-fd97-4a62-82cb-cee3fe877524\" (UID: \"d66ed4ec-fd97-4a62-82cb-cee3fe877524\") " Dec 03 12:08:43 crc kubenswrapper[5052]: I1203 12:08:43.981549 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bwb74\" (UniqueName: \"kubernetes.io/projected/d66ed4ec-fd97-4a62-82cb-cee3fe877524-kube-api-access-bwb74\") pod \"d66ed4ec-fd97-4a62-82cb-cee3fe877524\" (UID: \"d66ed4ec-fd97-4a62-82cb-cee3fe877524\") " Dec 03 12:08:43 crc kubenswrapper[5052]: I1203 12:08:43.981657 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/d66ed4ec-fd97-4a62-82cb-cee3fe877524-config-data-default\") pod \"d66ed4ec-fd97-4a62-82cb-cee3fe877524\" (UID: \"d66ed4ec-fd97-4a62-82cb-cee3fe877524\") " Dec 03 12:08:43 crc kubenswrapper[5052]: I1203 12:08:43.981772 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"mysql-db\" (UniqueName: \"kubernetes.io/local-volume/local-storage01-crc\") pod \"d66ed4ec-fd97-4a62-82cb-cee3fe877524\" (UID: \"d66ed4ec-fd97-4a62-82cb-cee3fe877524\") " Dec 03 12:08:43 crc kubenswrapper[5052]: I1203 12:08:43.982206 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/d66ed4ec-fd97-4a62-82cb-cee3fe877524-config-data-generated\") pod \"d66ed4ec-fd97-4a62-82cb-cee3fe877524\" (UID: \"d66ed4ec-fd97-4a62-82cb-cee3fe877524\") " Dec 03 12:08:43 crc kubenswrapper[5052]: I1203 12:08:43.982328 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/d66ed4ec-fd97-4a62-82cb-cee3fe877524-kolla-config\") pod \"d66ed4ec-fd97-4a62-82cb-cee3fe877524\" (UID: \"d66ed4ec-fd97-4a62-82cb-cee3fe877524\") " Dec 03 12:08:43 crc kubenswrapper[5052]: I1203 12:08:43.982439 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/d66ed4ec-fd97-4a62-82cb-cee3fe877524-operator-scripts\") pod \"d66ed4ec-fd97-4a62-82cb-cee3fe877524\" (UID: \"d66ed4ec-fd97-4a62-82cb-cee3fe877524\") " Dec 03 12:08:43 crc kubenswrapper[5052]: I1203 12:08:43.982485 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d66ed4ec-fd97-4a62-82cb-cee3fe877524-config-data-generated" (OuterVolumeSpecName: "config-data-generated") pod "d66ed4ec-fd97-4a62-82cb-cee3fe877524" (UID: "d66ed4ec-fd97-4a62-82cb-cee3fe877524"). InnerVolumeSpecName "config-data-generated". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:08:43 crc kubenswrapper[5052]: I1203 12:08:43.982598 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/d66ed4ec-fd97-4a62-82cb-cee3fe877524-galera-tls-certs\") pod \"d66ed4ec-fd97-4a62-82cb-cee3fe877524\" (UID: \"d66ed4ec-fd97-4a62-82cb-cee3fe877524\") " Dec 03 12:08:43 crc kubenswrapper[5052]: I1203 12:08:43.982856 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d66ed4ec-fd97-4a62-82cb-cee3fe877524-kolla-config" (OuterVolumeSpecName: "kolla-config") pod "d66ed4ec-fd97-4a62-82cb-cee3fe877524" (UID: "d66ed4ec-fd97-4a62-82cb-cee3fe877524"). InnerVolumeSpecName "kolla-config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:08:43 crc kubenswrapper[5052]: I1203 12:08:43.982912 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d66ed4ec-fd97-4a62-82cb-cee3fe877524-config-data-default" (OuterVolumeSpecName: "config-data-default") pod "d66ed4ec-fd97-4a62-82cb-cee3fe877524" (UID: "d66ed4ec-fd97-4a62-82cb-cee3fe877524"). InnerVolumeSpecName "config-data-default". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:08:43 crc kubenswrapper[5052]: I1203 12:08:43.983147 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/d66ed4ec-fd97-4a62-82cb-cee3fe877524-config-data-default\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:43 crc kubenswrapper[5052]: I1203 12:08:43.983215 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/d66ed4ec-fd97-4a62-82cb-cee3fe877524-config-data-generated\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:43 crc kubenswrapper[5052]: I1203 12:08:43.983279 5052 reconciler_common.go:293] "Volume detached for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/d66ed4ec-fd97-4a62-82cb-cee3fe877524-kolla-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:43 crc kubenswrapper[5052]: I1203 12:08:43.983257 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d66ed4ec-fd97-4a62-82cb-cee3fe877524-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "d66ed4ec-fd97-4a62-82cb-cee3fe877524" (UID: "d66ed4ec-fd97-4a62-82cb-cee3fe877524"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:08:43 crc kubenswrapper[5052]: I1203 12:08:43.985808 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d66ed4ec-fd97-4a62-82cb-cee3fe877524-kube-api-access-bwb74" (OuterVolumeSpecName: "kube-api-access-bwb74") pod "d66ed4ec-fd97-4a62-82cb-cee3fe877524" (UID: "d66ed4ec-fd97-4a62-82cb-cee3fe877524"). InnerVolumeSpecName "kube-api-access-bwb74". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:08:43 crc kubenswrapper[5052]: I1203 12:08:43.990181 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage01-crc" (OuterVolumeSpecName: "mysql-db") pod "d66ed4ec-fd97-4a62-82cb-cee3fe877524" (UID: "d66ed4ec-fd97-4a62-82cb-cee3fe877524"). InnerVolumeSpecName "local-storage01-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 12:08:44 crc kubenswrapper[5052]: I1203 12:08:44.004181 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d66ed4ec-fd97-4a62-82cb-cee3fe877524-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "d66ed4ec-fd97-4a62-82cb-cee3fe877524" (UID: "d66ed4ec-fd97-4a62-82cb-cee3fe877524"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:08:44 crc kubenswrapper[5052]: I1203 12:08:44.037257 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d66ed4ec-fd97-4a62-82cb-cee3fe877524-galera-tls-certs" (OuterVolumeSpecName: "galera-tls-certs") pod "d66ed4ec-fd97-4a62-82cb-cee3fe877524" (UID: "d66ed4ec-fd97-4a62-82cb-cee3fe877524"). InnerVolumeSpecName "galera-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:08:44 crc kubenswrapper[5052]: I1203 12:08:44.084750 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/d66ed4ec-fd97-4a62-82cb-cee3fe877524-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:44 crc kubenswrapper[5052]: I1203 12:08:44.084801 5052 reconciler_common.go:293] "Volume detached for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/d66ed4ec-fd97-4a62-82cb-cee3fe877524-galera-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:44 crc kubenswrapper[5052]: I1203 12:08:44.084811 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d66ed4ec-fd97-4a62-82cb-cee3fe877524-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:44 crc kubenswrapper[5052]: I1203 12:08:44.084821 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-bwb74\" (UniqueName: \"kubernetes.io/projected/d66ed4ec-fd97-4a62-82cb-cee3fe877524-kube-api-access-bwb74\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:44 crc kubenswrapper[5052]: I1203 12:08:44.084855 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage01-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage01-crc\") on node \"crc\" " Dec 03 12:08:44 crc kubenswrapper[5052]: I1203 12:08:44.102142 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage01-crc" (UniqueName: "kubernetes.io/local-volume/local-storage01-crc") on node "crc" Dec 03 12:08:44 crc kubenswrapper[5052]: I1203 12:08:44.186650 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage01-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage01-crc\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:44 crc kubenswrapper[5052]: I1203 12:08:44.837589 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-galera-0" event={"ID":"d66ed4ec-fd97-4a62-82cb-cee3fe877524","Type":"ContainerDied","Data":"893e1b658eab674cf7bf9866c1d9bbba9e853171a4415f27f1f45dfbb657af35"} Dec 03 12:08:44 crc kubenswrapper[5052]: I1203 12:08:44.838019 5052 scope.go:117] "RemoveContainer" containerID="7e2247eb34fdd68938be37e473102ca45e588792a2cc0096e976b082bdaee0f9" Dec 03 12:08:44 crc kubenswrapper[5052]: I1203 12:08:44.837912 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:08:44 crc kubenswrapper[5052]: I1203 12:08:44.868226 5052 scope.go:117] "RemoveContainer" containerID="c379302f8861ae5abdb296b0309b5345e5b9fa7962d9bc64cf7cb65a109171d6" Dec 03 12:08:44 crc kubenswrapper[5052]: I1203 12:08:44.880118 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/openstack-galera-0"] Dec 03 12:08:44 crc kubenswrapper[5052]: I1203 12:08:44.888048 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/openstack-galera-0"] Dec 03 12:08:45 crc kubenswrapper[5052]: I1203 12:08:45.685037 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6b92f88e-1325-4c3d-8d4e-4089a9033775" path="/var/lib/kubelet/pods/6b92f88e-1325-4c3d-8d4e-4089a9033775/volumes" Dec 03 12:08:45 crc kubenswrapper[5052]: I1203 12:08:45.686042 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d66ed4ec-fd97-4a62-82cb-cee3fe877524" path="/var/lib/kubelet/pods/d66ed4ec-fd97-4a62-82cb-cee3fe877524/volumes" Dec 03 12:08:53 crc kubenswrapper[5052]: I1203 12:08:53.436390 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" podUID="185fdd2a-cd57-45e5-9986-bebd15e5761f" containerName="rabbitmq" probeResult="failure" output="dial tcp 10.217.0.104:5671: connect: connection refused" Dec 03 12:08:56 crc kubenswrapper[5052]: I1203 12:08:56.528830 5052 scope.go:117] "RemoveContainer" containerID="71a1c1292c165a451da8f37d64d511b73b4e6bbaa158240dcc9e02122099a881" Dec 03 12:08:56 crc kubenswrapper[5052]: I1203 12:08:56.558108 5052 scope.go:117] "RemoveContainer" containerID="f82e335d0814000e829c2020ade84de1617dea07a919392e9736c05912d0b4bf" Dec 03 12:08:56 crc kubenswrapper[5052]: I1203 12:08:56.605564 5052 scope.go:117] "RemoveContainer" containerID="340f29d9a49e735b2a09dd64a1f01405a241208a475c2a1a778de21c81848ea0" Dec 03 12:08:56 crc kubenswrapper[5052]: I1203 12:08:56.629680 5052 scope.go:117] "RemoveContainer" containerID="c15f114c02ee16b234e664feb08dc9f03835f614029770d390cfbeb8fa520a9a" Dec 03 12:08:56 crc kubenswrapper[5052]: I1203 12:08:56.667742 5052 scope.go:117] "RemoveContainer" containerID="cf02feda51ed3db039ca069c4ae4a593ecba062d0a9b67f4c7d1a608b509c46f" Dec 03 12:08:56 crc kubenswrapper[5052]: I1203 12:08:56.697709 5052 scope.go:117] "RemoveContainer" containerID="1f0a6242e855eb2de1f01d290d9a30281da2eebae9925041a603e7661cfaf07c" Dec 03 12:08:56 crc kubenswrapper[5052]: I1203 12:08:56.741260 5052 scope.go:117] "RemoveContainer" containerID="e6787f6e4b03711dd124764348c5ed1eaf82f566af649919f63f24cc48189367" Dec 03 12:08:56 crc kubenswrapper[5052]: I1203 12:08:56.782164 5052 scope.go:117] "RemoveContainer" containerID="feac02bd195cd4010d3557ca2d9bccbb1a2e3e5d4cdedd3139212667bfa80b4b" Dec 03 12:08:56 crc kubenswrapper[5052]: I1203 12:08:56.827935 5052 scope.go:117] "RemoveContainer" containerID="368138b9d13c9ac7ee6c0551f792b75b2fa0c005de037787049a4e19da105713" Dec 03 12:08:56 crc kubenswrapper[5052]: I1203 12:08:56.853966 5052 scope.go:117] "RemoveContainer" containerID="22c5907c0a73443041c9628057637dc9e498c4fbefa6b6e588b40e69b81c8b9c" Dec 03 12:08:57 crc kubenswrapper[5052]: E1203 12:08:57.803252 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/rabbitmq-cell1-config-data: configmap "rabbitmq-cell1-config-data" not found Dec 03 12:08:57 crc kubenswrapper[5052]: E1203 12:08:57.805357 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/185fdd2a-cd57-45e5-9986-bebd15e5761f-config-data podName:185fdd2a-cd57-45e5-9986-bebd15e5761f nodeName:}" failed. No retries permitted until 2025-12-03 12:09:29.805332528 +0000 UTC m=+2960.393574372 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "config-data" (UniqueName: "kubernetes.io/configmap/185fdd2a-cd57-45e5-9986-bebd15e5761f-config-data") pod "rabbitmq-cell1-server-0" (UID: "185fdd2a-cd57-45e5-9986-bebd15e5761f") : configmap "rabbitmq-cell1-config-data" not found Dec 03 12:08:58 crc kubenswrapper[5052]: I1203 12:08:58.006431 5052 generic.go:334] "Generic (PLEG): container finished" podID="95bf5f72-82f3-4e88-a576-01dae857541f" containerID="c0e56624ca96c43f72b2c3cc860c510ab86560649ca040b145fbe1cdc7988153" exitCode=137 Dec 03 12:08:58 crc kubenswrapper[5052]: I1203 12:08:58.006490 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"95bf5f72-82f3-4e88-a576-01dae857541f","Type":"ContainerDied","Data":"c0e56624ca96c43f72b2c3cc860c510ab86560649ca040b145fbe1cdc7988153"} Dec 03 12:08:58 crc kubenswrapper[5052]: I1203 12:08:58.873023 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:08:58 crc kubenswrapper[5052]: I1203 12:08:58.927014 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-98m8t\" (UniqueName: \"kubernetes.io/projected/95bf5f72-82f3-4e88-a576-01dae857541f-kube-api-access-98m8t\") pod \"95bf5f72-82f3-4e88-a576-01dae857541f\" (UID: \"95bf5f72-82f3-4e88-a576-01dae857541f\") " Dec 03 12:08:58 crc kubenswrapper[5052]: I1203 12:08:58.927103 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/95bf5f72-82f3-4e88-a576-01dae857541f-etc-swift\") pod \"95bf5f72-82f3-4e88-a576-01dae857541f\" (UID: \"95bf5f72-82f3-4e88-a576-01dae857541f\") " Dec 03 12:08:58 crc kubenswrapper[5052]: I1203 12:08:58.927143 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"lock\" (UniqueName: \"kubernetes.io/empty-dir/95bf5f72-82f3-4e88-a576-01dae857541f-lock\") pod \"95bf5f72-82f3-4e88-a576-01dae857541f\" (UID: \"95bf5f72-82f3-4e88-a576-01dae857541f\") " Dec 03 12:08:58 crc kubenswrapper[5052]: I1203 12:08:58.927190 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"cache\" (UniqueName: \"kubernetes.io/empty-dir/95bf5f72-82f3-4e88-a576-01dae857541f-cache\") pod \"95bf5f72-82f3-4e88-a576-01dae857541f\" (UID: \"95bf5f72-82f3-4e88-a576-01dae857541f\") " Dec 03 12:08:58 crc kubenswrapper[5052]: I1203 12:08:58.927249 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"swift\" (UniqueName: \"kubernetes.io/local-volume/local-storage04-crc\") pod \"95bf5f72-82f3-4e88-a576-01dae857541f\" (UID: \"95bf5f72-82f3-4e88-a576-01dae857541f\") " Dec 03 12:08:58 crc kubenswrapper[5052]: I1203 12:08:58.927617 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/95bf5f72-82f3-4e88-a576-01dae857541f-lock" (OuterVolumeSpecName: "lock") pod "95bf5f72-82f3-4e88-a576-01dae857541f" (UID: "95bf5f72-82f3-4e88-a576-01dae857541f"). InnerVolumeSpecName "lock". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:08:58 crc kubenswrapper[5052]: I1203 12:08:58.927701 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/95bf5f72-82f3-4e88-a576-01dae857541f-cache" (OuterVolumeSpecName: "cache") pod "95bf5f72-82f3-4e88-a576-01dae857541f" (UID: "95bf5f72-82f3-4e88-a576-01dae857541f"). InnerVolumeSpecName "cache". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:08:58 crc kubenswrapper[5052]: I1203 12:08:58.932803 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/95bf5f72-82f3-4e88-a576-01dae857541f-etc-swift" (OuterVolumeSpecName: "etc-swift") pod "95bf5f72-82f3-4e88-a576-01dae857541f" (UID: "95bf5f72-82f3-4e88-a576-01dae857541f"). InnerVolumeSpecName "etc-swift". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:08:58 crc kubenswrapper[5052]: I1203 12:08:58.932936 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage04-crc" (OuterVolumeSpecName: "swift") pod "95bf5f72-82f3-4e88-a576-01dae857541f" (UID: "95bf5f72-82f3-4e88-a576-01dae857541f"). InnerVolumeSpecName "local-storage04-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 12:08:58 crc kubenswrapper[5052]: I1203 12:08:58.933465 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/95bf5f72-82f3-4e88-a576-01dae857541f-kube-api-access-98m8t" (OuterVolumeSpecName: "kube-api-access-98m8t") pod "95bf5f72-82f3-4e88-a576-01dae857541f" (UID: "95bf5f72-82f3-4e88-a576-01dae857541f"). InnerVolumeSpecName "kube-api-access-98m8t". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:08:59 crc kubenswrapper[5052]: I1203 12:08:59.024678 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"95bf5f72-82f3-4e88-a576-01dae857541f","Type":"ContainerDied","Data":"5230c658f0b920f5e8fe62e2ad61cb5ee6000feba05d7cc0ea6bcde57579cfe1"} Dec 03 12:08:59 crc kubenswrapper[5052]: I1203 12:08:59.024770 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:08:59 crc kubenswrapper[5052]: I1203 12:08:59.024790 5052 scope.go:117] "RemoveContainer" containerID="c0e56624ca96c43f72b2c3cc860c510ab86560649ca040b145fbe1cdc7988153" Dec 03 12:08:59 crc kubenswrapper[5052]: I1203 12:08:59.028914 5052 reconciler_common.go:293] "Volume detached for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/95bf5f72-82f3-4e88-a576-01dae857541f-etc-swift\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:59 crc kubenswrapper[5052]: I1203 12:08:59.028941 5052 reconciler_common.go:293] "Volume detached for volume \"lock\" (UniqueName: \"kubernetes.io/empty-dir/95bf5f72-82f3-4e88-a576-01dae857541f-lock\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:59 crc kubenswrapper[5052]: I1203 12:08:59.028952 5052 reconciler_common.go:293] "Volume detached for volume \"cache\" (UniqueName: \"kubernetes.io/empty-dir/95bf5f72-82f3-4e88-a576-01dae857541f-cache\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:59 crc kubenswrapper[5052]: I1203 12:08:59.028986 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage04-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage04-crc\") on node \"crc\" " Dec 03 12:08:59 crc kubenswrapper[5052]: I1203 12:08:59.029000 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-98m8t\" (UniqueName: \"kubernetes.io/projected/95bf5f72-82f3-4e88-a576-01dae857541f-kube-api-access-98m8t\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:59 crc kubenswrapper[5052]: I1203 12:08:59.045873 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage04-crc" (UniqueName: "kubernetes.io/local-volume/local-storage04-crc") on node "crc" Dec 03 12:08:59 crc kubenswrapper[5052]: I1203 12:08:59.059092 5052 scope.go:117] "RemoveContainer" containerID="4a4d3e25edd055d3f3503821db30046cfd815c08421228eb4d5e76a3f4e38a23" Dec 03 12:08:59 crc kubenswrapper[5052]: I1203 12:08:59.100892 5052 scope.go:117] "RemoveContainer" containerID="2d047aaa650efd207aa517746b6cde98bb8bae986bd03e7b545eda7fd36f1023" Dec 03 12:08:59 crc kubenswrapper[5052]: I1203 12:08:59.112148 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/swift-storage-0"] Dec 03 12:08:59 crc kubenswrapper[5052]: I1203 12:08:59.120546 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/swift-storage-0"] Dec 03 12:08:59 crc kubenswrapper[5052]: I1203 12:08:59.125641 5052 scope.go:117] "RemoveContainer" containerID="c6d65f1e5859c5423b2bed9ad8130035fdbcdb06ce5b34713168ebc78916b853" Dec 03 12:08:59 crc kubenswrapper[5052]: I1203 12:08:59.129864 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage04-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage04-crc\") on node \"crc\" DevicePath \"\"" Dec 03 12:08:59 crc kubenswrapper[5052]: I1203 12:08:59.147456 5052 scope.go:117] "RemoveContainer" containerID="b1f5592550eedafe273eda6ddd62bcf2957cdd483d534a5369391fc101829850" Dec 03 12:08:59 crc kubenswrapper[5052]: I1203 12:08:59.177937 5052 scope.go:117] "RemoveContainer" containerID="ff53c665fe25113ca5552ca3edb72dc98d7ffacf8d96d01a610c8292d0ca0504" Dec 03 12:08:59 crc kubenswrapper[5052]: I1203 12:08:59.193621 5052 scope.go:117] "RemoveContainer" containerID="da24ecbec4ad70b7a0db3646cffcca7dc56b37efffe6e261de63720e146b2b8e" Dec 03 12:08:59 crc kubenswrapper[5052]: I1203 12:08:59.210331 5052 scope.go:117] "RemoveContainer" containerID="2577c8659d7bc84c157b82cc64e9d927df3b8f1b4fe022b955ef7fe05c5dadbb" Dec 03 12:08:59 crc kubenswrapper[5052]: I1203 12:08:59.228183 5052 scope.go:117] "RemoveContainer" containerID="a4050b9976060a4f925ffaa2cb889b98f3b107441d8ca49ad092c8be65814639" Dec 03 12:08:59 crc kubenswrapper[5052]: I1203 12:08:59.247786 5052 scope.go:117] "RemoveContainer" containerID="ae9aaced3b606148dcf3e097c33141f00f123ef8589615b163bff1470454ae80" Dec 03 12:08:59 crc kubenswrapper[5052]: I1203 12:08:59.272362 5052 scope.go:117] "RemoveContainer" containerID="9fa20306dd177e25e71d2c3ec088ca38dc2879ff9f7569b37d3b1ab695d20bed" Dec 03 12:08:59 crc kubenswrapper[5052]: I1203 12:08:59.293559 5052 scope.go:117] "RemoveContainer" containerID="2732548b8572203d895c3e589eef6680d882e9fcbe498714de0778d4c35813ec" Dec 03 12:08:59 crc kubenswrapper[5052]: I1203 12:08:59.315050 5052 scope.go:117] "RemoveContainer" containerID="6a7d071e07e4e8b8e192642aeae3ccfa4e50a1bfae563ab609f77db309009d6d" Dec 03 12:08:59 crc kubenswrapper[5052]: I1203 12:08:59.332550 5052 scope.go:117] "RemoveContainer" containerID="3604bfccbdf55c7f8cc4aa8921d57387c338bcccd177ab45652bc037ec8fdad6" Dec 03 12:08:59 crc kubenswrapper[5052]: I1203 12:08:59.351699 5052 scope.go:117] "RemoveContainer" containerID="003e4620ad1989f1fa65349bb9193c7be42568b4061355bee57a6b9ae2b3d5a2" Dec 03 12:08:59 crc kubenswrapper[5052]: I1203 12:08:59.378994 5052 patch_prober.go:28] interesting pod/machine-config-daemon-74bj6 container/machine-config-daemon namespace/openshift-machine-config-operator: Liveness probe status=failure output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" start-of-body= Dec 03 12:08:59 crc kubenswrapper[5052]: I1203 12:08:59.379295 5052 prober.go:107] "Probe failed" probeType="Liveness" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" containerName="machine-config-daemon" probeResult="failure" output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" Dec 03 12:08:59 crc kubenswrapper[5052]: I1203 12:08:59.379354 5052 kubelet.go:2542] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" Dec 03 12:08:59 crc kubenswrapper[5052]: I1203 12:08:59.381030 5052 kuberuntime_manager.go:1027] "Message for Container of pod" containerName="machine-config-daemon" containerStatusID={"Type":"cri-o","ID":"63298107b3d090deff9ed92f3d613e5b1c86d9137684d117500257db8ee1c805"} pod="openshift-machine-config-operator/machine-config-daemon-74bj6" containerMessage="Container machine-config-daemon failed liveness probe, will be restarted" Dec 03 12:08:59 crc kubenswrapper[5052]: I1203 12:08:59.381201 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" containerName="machine-config-daemon" containerID="cri-o://63298107b3d090deff9ed92f3d613e5b1c86d9137684d117500257db8ee1c805" gracePeriod=600 Dec 03 12:08:59 crc kubenswrapper[5052]: I1203 12:08:59.703433 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="95bf5f72-82f3-4e88-a576-01dae857541f" path="/var/lib/kubelet/pods/95bf5f72-82f3-4e88-a576-01dae857541f/volumes" Dec 03 12:09:00 crc kubenswrapper[5052]: I1203 12:09:00.038193 5052 generic.go:334] "Generic (PLEG): container finished" podID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" containerID="63298107b3d090deff9ed92f3d613e5b1c86d9137684d117500257db8ee1c805" exitCode=0 Dec 03 12:09:00 crc kubenswrapper[5052]: I1203 12:09:00.038276 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" event={"ID":"10c237bc-72e7-4da3-9f2c-b1688395bf6e","Type":"ContainerDied","Data":"63298107b3d090deff9ed92f3d613e5b1c86d9137684d117500257db8ee1c805"} Dec 03 12:09:00 crc kubenswrapper[5052]: I1203 12:09:00.038559 5052 scope.go:117] "RemoveContainer" containerID="fcd7a4f50449fc80828465d45d26c57ddd80c8ab4f967dd642dae3c503a27345" Dec 03 12:09:00 crc kubenswrapper[5052]: E1203 12:09:00.227032 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 12:09:01 crc kubenswrapper[5052]: I1203 12:09:01.048357 5052 scope.go:117] "RemoveContainer" containerID="63298107b3d090deff9ed92f3d613e5b1c86d9137684d117500257db8ee1c805" Dec 03 12:09:01 crc kubenswrapper[5052]: E1203 12:09:01.048613 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 12:09:03 crc kubenswrapper[5052]: I1203 12:09:03.440903 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" podUID="185fdd2a-cd57-45e5-9986-bebd15e5761f" containerName="rabbitmq" probeResult="failure" output="dial tcp 10.217.0.104:5671: connect: connection refused" Dec 03 12:09:03 crc kubenswrapper[5052]: I1203 12:09:03.441620 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:09:06 crc kubenswrapper[5052]: I1203 12:09:06.278587 5052 pod_container_manager_linux.go:210] "Failed to delete cgroup paths" cgroupName=["kubepods","besteffort","pod305f2518-9632-4358-84a1-632b4a7d8f6f"] err="unable to destroy cgroup paths for cgroup [kubepods besteffort pod305f2518-9632-4358-84a1-632b4a7d8f6f] : Timed out while waiting for systemd to remove kubepods-besteffort-pod305f2518_9632_4358_84a1_632b4a7d8f6f.slice" Dec 03 12:09:06 crc kubenswrapper[5052]: E1203 12:09:06.278650 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to delete cgroup paths for [kubepods besteffort pod305f2518-9632-4358-84a1-632b4a7d8f6f] : unable to destroy cgroup paths for cgroup [kubepods besteffort pod305f2518-9632-4358-84a1-632b4a7d8f6f] : Timed out while waiting for systemd to remove kubepods-besteffort-pod305f2518_9632_4358_84a1_632b4a7d8f6f.slice" pod="openstack-kuttl-tests/kube-state-metrics-0" podUID="305f2518-9632-4358-84a1-632b4a7d8f6f" Dec 03 12:09:07 crc kubenswrapper[5052]: I1203 12:09:07.098146 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:09:07 crc kubenswrapper[5052]: I1203 12:09:07.118362 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/kube-state-metrics-0"] Dec 03 12:09:07 crc kubenswrapper[5052]: I1203 12:09:07.124866 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/kube-state-metrics-0"] Dec 03 12:09:07 crc kubenswrapper[5052]: I1203 12:09:07.685835 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="305f2518-9632-4358-84a1-632b4a7d8f6f" path="/var/lib/kubelet/pods/305f2518-9632-4358-84a1-632b4a7d8f6f/volumes" Dec 03 12:09:13 crc kubenswrapper[5052]: I1203 12:09:13.435647 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" podUID="185fdd2a-cd57-45e5-9986-bebd15e5761f" containerName="rabbitmq" probeResult="failure" output="dial tcp 10.217.0.104:5671: connect: connection refused" Dec 03 12:09:16 crc kubenswrapper[5052]: I1203 12:09:16.673739 5052 scope.go:117] "RemoveContainer" containerID="63298107b3d090deff9ed92f3d613e5b1c86d9137684d117500257db8ee1c805" Dec 03 12:09:16 crc kubenswrapper[5052]: E1203 12:09:16.674788 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 12:09:23 crc kubenswrapper[5052]: I1203 12:09:23.435406 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" podUID="185fdd2a-cd57-45e5-9986-bebd15e5761f" containerName="rabbitmq" probeResult="failure" output="dial tcp 10.217.0.104:5671: connect: connection refused" Dec 03 12:09:29 crc kubenswrapper[5052]: E1203 12:09:29.829499 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/rabbitmq-cell1-config-data: configmap "rabbitmq-cell1-config-data" not found Dec 03 12:09:29 crc kubenswrapper[5052]: E1203 12:09:29.829998 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/185fdd2a-cd57-45e5-9986-bebd15e5761f-config-data podName:185fdd2a-cd57-45e5-9986-bebd15e5761f nodeName:}" failed. No retries permitted until 2025-12-03 12:10:33.829983839 +0000 UTC m=+3024.418225603 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "config-data" (UniqueName: "kubernetes.io/configmap/185fdd2a-cd57-45e5-9986-bebd15e5761f-config-data") pod "rabbitmq-cell1-server-0" (UID: "185fdd2a-cd57-45e5-9986-bebd15e5761f") : configmap "rabbitmq-cell1-config-data" not found Dec 03 12:09:30 crc kubenswrapper[5052]: I1203 12:09:30.673175 5052 scope.go:117] "RemoveContainer" containerID="63298107b3d090deff9ed92f3d613e5b1c86d9137684d117500257db8ee1c805" Dec 03 12:09:30 crc kubenswrapper[5052]: E1203 12:09:30.673506 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 12:09:31 crc kubenswrapper[5052]: E1203 12:09:31.261862 5052 handlers.go:78] "Exec lifecycle hook for Container in Pod failed" err=< Dec 03 12:09:31 crc kubenswrapper[5052]: command '/bin/bash -c if [ ! -z "$(cat /etc/pod-info/skipPreStopChecks)" ]; then exit 0; fi; rabbitmq-upgrade await_online_quorum_plus_one -t 604800 && rabbitmq-upgrade await_online_synchronized_mirror -t 604800 || true && rabbitmq-upgrade drain -t 604800' exited with 69: Error: unable to perform an operation on node 'rabbit@rabbitmq-cell1-server-0.rabbitmq-cell1-nodes.openstack-kuttl-tests'. Please see diagnostics information and suggestions below. Dec 03 12:09:31 crc kubenswrapper[5052]: Dec 03 12:09:31 crc kubenswrapper[5052]: Most common reasons for this are: Dec 03 12:09:31 crc kubenswrapper[5052]: Dec 03 12:09:31 crc kubenswrapper[5052]: * Target node is unreachable (e.g. due to hostname resolution, TCP connection or firewall issues) Dec 03 12:09:31 crc kubenswrapper[5052]: * CLI tool fails to authenticate with the server (e.g. due to CLI tool's Erlang cookie not matching that of the server) Dec 03 12:09:31 crc kubenswrapper[5052]: * Target node is not running Dec 03 12:09:31 crc kubenswrapper[5052]: Dec 03 12:09:31 crc kubenswrapper[5052]: In addition to the diagnostics info below: Dec 03 12:09:31 crc kubenswrapper[5052]: Dec 03 12:09:31 crc kubenswrapper[5052]: * See the CLI, clustering and networking guides on https://rabbitmq.com/documentation.html to learn more Dec 03 12:09:31 crc kubenswrapper[5052]: * Consult server logs on node rabbit@rabbitmq-cell1-server-0.rabbitmq-cell1-nodes.openstack-kuttl-tests Dec 03 12:09:31 crc kubenswrapper[5052]: * If target node is configured to use long node names, don't forget to use --longnames with CLI tools Dec 03 12:09:31 crc kubenswrapper[5052]: Dec 03 12:09:31 crc kubenswrapper[5052]: DIAGNOSTICS Dec 03 12:09:31 crc kubenswrapper[5052]: =========== Dec 03 12:09:31 crc kubenswrapper[5052]: Dec 03 12:09:31 crc kubenswrapper[5052]: attempted to contact: ['rabbit@rabbitmq-cell1-server-0.rabbitmq-cell1-nodes.openstack-kuttl-tests'] Dec 03 12:09:31 crc kubenswrapper[5052]: Dec 03 12:09:31 crc kubenswrapper[5052]: rabbit@rabbitmq-cell1-server-0.rabbitmq-cell1-nodes.openstack-kuttl-tests: Dec 03 12:09:31 crc kubenswrapper[5052]: * unable to connect to epmd (port 4369) on rabbitmq-cell1-server-0.rabbitmq-cell1-nodes.openstack-kuttl-tests: nxdomain (non-existing domain) Dec 03 12:09:31 crc kubenswrapper[5052]: Dec 03 12:09:31 crc kubenswrapper[5052]: Dec 03 12:09:31 crc kubenswrapper[5052]: Current node details: Dec 03 12:09:31 crc kubenswrapper[5052]: * node name: 'rabbitmqcli-239-rabbit@rabbitmq-cell1-server-0.rabbitmq-cell1-nodes.openstack-kuttl-tests' Dec 03 12:09:31 crc kubenswrapper[5052]: * effective user's home directory: /var/lib/rabbitmq Dec 03 12:09:31 crc kubenswrapper[5052]: * Erlang cookie hash: UY6LesrloYsLvq3rzl/CqA== Dec 03 12:09:31 crc kubenswrapper[5052]: Dec 03 12:09:31 crc kubenswrapper[5052]: Error: unable to perform an operation on node 'rabbit@rabbitmq-cell1-server-0.rabbitmq-cell1-nodes.openstack-kuttl-tests'. Please see diagnostics information and suggestions below. Dec 03 12:09:31 crc kubenswrapper[5052]: Dec 03 12:09:31 crc kubenswrapper[5052]: Most common reasons for this are: Dec 03 12:09:31 crc kubenswrapper[5052]: Dec 03 12:09:31 crc kubenswrapper[5052]: * Target node is unreachable (e.g. due to hostname resolution, TCP connection or firewall issues) Dec 03 12:09:31 crc kubenswrapper[5052]: * CLI tool fails to authenticate with the server (e.g. due to CLI tool's Erlang cookie not matching that of the server) Dec 03 12:09:31 crc kubenswrapper[5052]: * Target node is not running Dec 03 12:09:31 crc kubenswrapper[5052]: Dec 03 12:09:31 crc kubenswrapper[5052]: In addition to the diagnostics info below: Dec 03 12:09:31 crc kubenswrapper[5052]: Dec 03 12:09:31 crc kubenswrapper[5052]: * See the CLI, clustering and networking guides on https://rabbitmq.com/documentation.html to learn more Dec 03 12:09:31 crc kubenswrapper[5052]: * Consult server logs on node rabbit@rabbitmq-cell1-server-0.rabbitmq-cell1-nodes.openstack-kuttl-tests Dec 03 12:09:31 crc kubenswrapper[5052]: * If target node is configured to use long node names, don't forget to use --longnames with CLI tools Dec 03 12:09:31 crc kubenswrapper[5052]: Dec 03 12:09:31 crc kubenswrapper[5052]: DIAGNOSTICS Dec 03 12:09:31 crc kubenswrapper[5052]: =========== Dec 03 12:09:31 crc kubenswrapper[5052]: Dec 03 12:09:31 crc kubenswrapper[5052]: attempted to contact: ['rabbit@rabbitmq-cell1-server-0.rabbitmq-cell1-nodes.openstack-kuttl-tests'] Dec 03 12:09:31 crc kubenswrapper[5052]: Dec 03 12:09:31 crc kubenswrapper[5052]: rabbit@rabbitmq-cell1-server-0.rabbitmq-cell1-nodes.openstack-kuttl-tests: Dec 03 12:09:31 crc kubenswrapper[5052]: * unable to connect to epmd (port 4369) on rabbitmq-cell1-server-0.rabbitmq-cell1-nodes.openstack-kuttl-tests: nxdomain (non-existing domain) Dec 03 12:09:31 crc kubenswrapper[5052]: Dec 03 12:09:31 crc kubenswrapper[5052]: Dec 03 12:09:31 crc kubenswrapper[5052]: Current node details: Dec 03 12:09:31 crc kubenswrapper[5052]: * node name: 'rabbitmqcli-78-rabbit@rabbitmq-cell1-server-0.rabbitmq-cell1-nodes.openstack-kuttl-tests' Dec 03 12:09:31 crc kubenswrapper[5052]: * effective user's home directory: /var/lib/rabbitmq Dec 03 12:09:31 crc kubenswrapper[5052]: * Erlang cookie hash: UY6LesrloYsLvq3rzl/CqA== Dec 03 12:09:31 crc kubenswrapper[5052]: Dec 03 12:09:31 crc kubenswrapper[5052]: > execCommand=["/bin/bash","-c","if [ ! -z \"$(cat /etc/pod-info/skipPreStopChecks)\" ]; then exit 0; fi; rabbitmq-upgrade await_online_quorum_plus_one -t 604800 \u0026\u0026 rabbitmq-upgrade await_online_synchronized_mirror -t 604800 || true \u0026\u0026 rabbitmq-upgrade drain -t 604800"] containerName="rabbitmq" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" message=< Dec 03 12:09:31 crc kubenswrapper[5052]: Will put node rabbit@rabbitmq-cell1-server-0.rabbitmq-cell1-nodes.openstack-kuttl-tests into maintenance mode. The node will no longer serve any client traffic! Dec 03 12:09:31 crc kubenswrapper[5052]: Error: unable to perform an operation on node 'rabbit@rabbitmq-cell1-server-0.rabbitmq-cell1-nodes.openstack-kuttl-tests'. Please see diagnostics information and suggestions below. Dec 03 12:09:31 crc kubenswrapper[5052]: Dec 03 12:09:31 crc kubenswrapper[5052]: Most common reasons for this are: Dec 03 12:09:31 crc kubenswrapper[5052]: Dec 03 12:09:31 crc kubenswrapper[5052]: * Target node is unreachable (e.g. due to hostname resolution, TCP connection or firewall issues) Dec 03 12:09:31 crc kubenswrapper[5052]: * CLI tool fails to authenticate with the server (e.g. due to CLI tool's Erlang cookie not matching that of the server) Dec 03 12:09:31 crc kubenswrapper[5052]: * Target node is not running Dec 03 12:09:31 crc kubenswrapper[5052]: Dec 03 12:09:31 crc kubenswrapper[5052]: In addition to the diagnostics info below: Dec 03 12:09:31 crc kubenswrapper[5052]: Dec 03 12:09:31 crc kubenswrapper[5052]: * See the CLI, clustering and networking guides on https://rabbitmq.com/documentation.html to learn more Dec 03 12:09:31 crc kubenswrapper[5052]: * Consult server logs on node rabbit@rabbitmq-cell1-server-0.rabbitmq-cell1-nodes.openstack-kuttl-tests Dec 03 12:09:31 crc kubenswrapper[5052]: * If target node is configured to use long node names, don't forget to use --longnames with CLI tools Dec 03 12:09:31 crc kubenswrapper[5052]: Dec 03 12:09:31 crc kubenswrapper[5052]: DIAGNOSTICS Dec 03 12:09:31 crc kubenswrapper[5052]: =========== Dec 03 12:09:31 crc kubenswrapper[5052]: Dec 03 12:09:31 crc kubenswrapper[5052]: attempted to contact: ['rabbit@rabbitmq-cell1-server-0.rabbitmq-cell1-nodes.openstack-kuttl-tests'] Dec 03 12:09:31 crc kubenswrapper[5052]: Dec 03 12:09:31 crc kubenswrapper[5052]: rabbit@rabbitmq-cell1-server-0.rabbitmq-cell1-nodes.openstack-kuttl-tests: Dec 03 12:09:31 crc kubenswrapper[5052]: * unable to connect to epmd (port 4369) on rabbitmq-cell1-server-0.rabbitmq-cell1-nodes.openstack-kuttl-tests: nxdomain (non-existing domain) Dec 03 12:09:31 crc kubenswrapper[5052]: Dec 03 12:09:31 crc kubenswrapper[5052]: Dec 03 12:09:31 crc kubenswrapper[5052]: Current node details: Dec 03 12:09:31 crc kubenswrapper[5052]: * node name: 'rabbitmqcli-239-rabbit@rabbitmq-cell1-server-0.rabbitmq-cell1-nodes.openstack-kuttl-tests' Dec 03 12:09:31 crc kubenswrapper[5052]: * effective user's home directory: /var/lib/rabbitmq Dec 03 12:09:31 crc kubenswrapper[5052]: * Erlang cookie hash: UY6LesrloYsLvq3rzl/CqA== Dec 03 12:09:31 crc kubenswrapper[5052]: Dec 03 12:09:31 crc kubenswrapper[5052]: Error: unable to perform an operation on node 'rabbit@rabbitmq-cell1-server-0.rabbitmq-cell1-nodes.openstack-kuttl-tests'. Please see diagnostics information and suggestions below. Dec 03 12:09:31 crc kubenswrapper[5052]: Dec 03 12:09:31 crc kubenswrapper[5052]: Most common reasons for this are: Dec 03 12:09:31 crc kubenswrapper[5052]: Dec 03 12:09:31 crc kubenswrapper[5052]: * Target node is unreachable (e.g. due to hostname resolution, TCP connection or firewall issues) Dec 03 12:09:31 crc kubenswrapper[5052]: * CLI tool fails to authenticate with the server (e.g. due to CLI tool's Erlang cookie not matching that of the server) Dec 03 12:09:31 crc kubenswrapper[5052]: * Target node is not running Dec 03 12:09:31 crc kubenswrapper[5052]: Dec 03 12:09:31 crc kubenswrapper[5052]: In addition to the diagnostics info below: Dec 03 12:09:31 crc kubenswrapper[5052]: Dec 03 12:09:31 crc kubenswrapper[5052]: * See the CLI, clustering and networking guides on https://rabbitmq.com/documentation.html to learn more Dec 03 12:09:31 crc kubenswrapper[5052]: * Consult server logs on node rabbit@rabbitmq-cell1-server-0.rabbitmq-cell1-nodes.openstack-kuttl-tests Dec 03 12:09:31 crc kubenswrapper[5052]: * If target node is configured to use long node names, don't forget to use --longnames with CLI tools Dec 03 12:09:31 crc kubenswrapper[5052]: Dec 03 12:09:31 crc kubenswrapper[5052]: DIAGNOSTICS Dec 03 12:09:31 crc kubenswrapper[5052]: =========== Dec 03 12:09:31 crc kubenswrapper[5052]: Dec 03 12:09:31 crc kubenswrapper[5052]: attempted to contact: ['rabbit@rabbitmq-cell1-server-0.rabbitmq-cell1-nodes.openstack-kuttl-tests'] Dec 03 12:09:31 crc kubenswrapper[5052]: Dec 03 12:09:31 crc kubenswrapper[5052]: rabbit@rabbitmq-cell1-server-0.rabbitmq-cell1-nodes.openstack-kuttl-tests: Dec 03 12:09:31 crc kubenswrapper[5052]: * unable to connect to epmd (port 4369) on rabbitmq-cell1-server-0.rabbitmq-cell1-nodes.openstack-kuttl-tests: nxdomain (non-existing domain) Dec 03 12:09:31 crc kubenswrapper[5052]: Dec 03 12:09:31 crc kubenswrapper[5052]: Dec 03 12:09:31 crc kubenswrapper[5052]: Current node details: Dec 03 12:09:31 crc kubenswrapper[5052]: * node name: 'rabbitmqcli-78-rabbit@rabbitmq-cell1-server-0.rabbitmq-cell1-nodes.openstack-kuttl-tests' Dec 03 12:09:31 crc kubenswrapper[5052]: * effective user's home directory: /var/lib/rabbitmq Dec 03 12:09:31 crc kubenswrapper[5052]: * Erlang cookie hash: UY6LesrloYsLvq3rzl/CqA== Dec 03 12:09:31 crc kubenswrapper[5052]: Dec 03 12:09:31 crc kubenswrapper[5052]: > Dec 03 12:09:31 crc kubenswrapper[5052]: E1203 12:09:31.262305 5052 kuberuntime_container.go:691] "PreStop hook failed" err=< Dec 03 12:09:31 crc kubenswrapper[5052]: command '/bin/bash -c if [ ! -z "$(cat /etc/pod-info/skipPreStopChecks)" ]; then exit 0; fi; rabbitmq-upgrade await_online_quorum_plus_one -t 604800 && rabbitmq-upgrade await_online_synchronized_mirror -t 604800 || true && rabbitmq-upgrade drain -t 604800' exited with 69: Error: unable to perform an operation on node 'rabbit@rabbitmq-cell1-server-0.rabbitmq-cell1-nodes.openstack-kuttl-tests'. Please see diagnostics information and suggestions below. Dec 03 12:09:31 crc kubenswrapper[5052]: Dec 03 12:09:31 crc kubenswrapper[5052]: Most common reasons for this are: Dec 03 12:09:31 crc kubenswrapper[5052]: Dec 03 12:09:31 crc kubenswrapper[5052]: * Target node is unreachable (e.g. due to hostname resolution, TCP connection or firewall issues) Dec 03 12:09:31 crc kubenswrapper[5052]: * CLI tool fails to authenticate with the server (e.g. due to CLI tool's Erlang cookie not matching that of the server) Dec 03 12:09:31 crc kubenswrapper[5052]: * Target node is not running Dec 03 12:09:31 crc kubenswrapper[5052]: Dec 03 12:09:31 crc kubenswrapper[5052]: In addition to the diagnostics info below: Dec 03 12:09:31 crc kubenswrapper[5052]: Dec 03 12:09:31 crc kubenswrapper[5052]: * See the CLI, clustering and networking guides on https://rabbitmq.com/documentation.html to learn more Dec 03 12:09:31 crc kubenswrapper[5052]: * Consult server logs on node rabbit@rabbitmq-cell1-server-0.rabbitmq-cell1-nodes.openstack-kuttl-tests Dec 03 12:09:31 crc kubenswrapper[5052]: * If target node is configured to use long node names, don't forget to use --longnames with CLI tools Dec 03 12:09:31 crc kubenswrapper[5052]: Dec 03 12:09:31 crc kubenswrapper[5052]: DIAGNOSTICS Dec 03 12:09:31 crc kubenswrapper[5052]: =========== Dec 03 12:09:31 crc kubenswrapper[5052]: Dec 03 12:09:31 crc kubenswrapper[5052]: attempted to contact: ['rabbit@rabbitmq-cell1-server-0.rabbitmq-cell1-nodes.openstack-kuttl-tests'] Dec 03 12:09:31 crc kubenswrapper[5052]: Dec 03 12:09:31 crc kubenswrapper[5052]: rabbit@rabbitmq-cell1-server-0.rabbitmq-cell1-nodes.openstack-kuttl-tests: Dec 03 12:09:31 crc kubenswrapper[5052]: * unable to connect to epmd (port 4369) on rabbitmq-cell1-server-0.rabbitmq-cell1-nodes.openstack-kuttl-tests: nxdomain (non-existing domain) Dec 03 12:09:31 crc kubenswrapper[5052]: Dec 03 12:09:31 crc kubenswrapper[5052]: Dec 03 12:09:31 crc kubenswrapper[5052]: Current node details: Dec 03 12:09:31 crc kubenswrapper[5052]: * node name: 'rabbitmqcli-239-rabbit@rabbitmq-cell1-server-0.rabbitmq-cell1-nodes.openstack-kuttl-tests' Dec 03 12:09:31 crc kubenswrapper[5052]: * effective user's home directory: /var/lib/rabbitmq Dec 03 12:09:31 crc kubenswrapper[5052]: * Erlang cookie hash: UY6LesrloYsLvq3rzl/CqA== Dec 03 12:09:31 crc kubenswrapper[5052]: Dec 03 12:09:31 crc kubenswrapper[5052]: Error: unable to perform an operation on node 'rabbit@rabbitmq-cell1-server-0.rabbitmq-cell1-nodes.openstack-kuttl-tests'. Please see diagnostics information and suggestions below. Dec 03 12:09:31 crc kubenswrapper[5052]: Dec 03 12:09:31 crc kubenswrapper[5052]: Most common reasons for this are: Dec 03 12:09:31 crc kubenswrapper[5052]: Dec 03 12:09:31 crc kubenswrapper[5052]: * Target node is unreachable (e.g. due to hostname resolution, TCP connection or firewall issues) Dec 03 12:09:31 crc kubenswrapper[5052]: * CLI tool fails to authenticate with the server (e.g. due to CLI tool's Erlang cookie not matching that of the server) Dec 03 12:09:31 crc kubenswrapper[5052]: * Target node is not running Dec 03 12:09:31 crc kubenswrapper[5052]: Dec 03 12:09:31 crc kubenswrapper[5052]: In addition to the diagnostics info below: Dec 03 12:09:31 crc kubenswrapper[5052]: Dec 03 12:09:31 crc kubenswrapper[5052]: * See the CLI, clustering and networking guides on https://rabbitmq.com/documentation.html to learn more Dec 03 12:09:31 crc kubenswrapper[5052]: * Consult server logs on node rabbit@rabbitmq-cell1-server-0.rabbitmq-cell1-nodes.openstack-kuttl-tests Dec 03 12:09:31 crc kubenswrapper[5052]: * If target node is configured to use long node names, don't forget to use --longnames with CLI tools Dec 03 12:09:31 crc kubenswrapper[5052]: Dec 03 12:09:31 crc kubenswrapper[5052]: DIAGNOSTICS Dec 03 12:09:31 crc kubenswrapper[5052]: =========== Dec 03 12:09:31 crc kubenswrapper[5052]: Dec 03 12:09:31 crc kubenswrapper[5052]: attempted to contact: ['rabbit@rabbitmq-cell1-server-0.rabbitmq-cell1-nodes.openstack-kuttl-tests'] Dec 03 12:09:31 crc kubenswrapper[5052]: Dec 03 12:09:31 crc kubenswrapper[5052]: rabbit@rabbitmq-cell1-server-0.rabbitmq-cell1-nodes.openstack-kuttl-tests: Dec 03 12:09:31 crc kubenswrapper[5052]: * unable to connect to epmd (port 4369) on rabbitmq-cell1-server-0.rabbitmq-cell1-nodes.openstack-kuttl-tests: nxdomain (non-existing domain) Dec 03 12:09:31 crc kubenswrapper[5052]: Dec 03 12:09:31 crc kubenswrapper[5052]: Dec 03 12:09:31 crc kubenswrapper[5052]: Current node details: Dec 03 12:09:31 crc kubenswrapper[5052]: * node name: 'rabbitmqcli-78-rabbit@rabbitmq-cell1-server-0.rabbitmq-cell1-nodes.openstack-kuttl-tests' Dec 03 12:09:31 crc kubenswrapper[5052]: * effective user's home directory: /var/lib/rabbitmq Dec 03 12:09:31 crc kubenswrapper[5052]: * Erlang cookie hash: UY6LesrloYsLvq3rzl/CqA== Dec 03 12:09:31 crc kubenswrapper[5052]: Dec 03 12:09:31 crc kubenswrapper[5052]: > pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" podUID="185fdd2a-cd57-45e5-9986-bebd15e5761f" containerName="rabbitmq" containerID="cri-o://c2f60fc531710d84b2e54ecd132f7312264860d279c9cbb5f54ba76be0eb79aa" Dec 03 12:09:31 crc kubenswrapper[5052]: I1203 12:09:31.262363 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" podUID="185fdd2a-cd57-45e5-9986-bebd15e5761f" containerName="rabbitmq" containerID="cri-o://c2f60fc531710d84b2e54ecd132f7312264860d279c9cbb5f54ba76be0eb79aa" gracePeriod=604738 Dec 03 12:09:33 crc kubenswrapper[5052]: I1203 12:09:33.072986 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:09:33 crc kubenswrapper[5052]: I1203 12:09:33.087843 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/185fdd2a-cd57-45e5-9986-bebd15e5761f-config-data\") pod \"185fdd2a-cd57-45e5-9986-bebd15e5761f\" (UID: \"185fdd2a-cd57-45e5-9986-bebd15e5761f\") " Dec 03 12:09:33 crc kubenswrapper[5052]: I1203 12:09:33.087890 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"rabbitmq-plugins\" (UniqueName: \"kubernetes.io/empty-dir/185fdd2a-cd57-45e5-9986-bebd15e5761f-rabbitmq-plugins\") pod \"185fdd2a-cd57-45e5-9986-bebd15e5761f\" (UID: \"185fdd2a-cd57-45e5-9986-bebd15e5761f\") " Dec 03 12:09:33 crc kubenswrapper[5052]: I1203 12:09:33.087940 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"rabbitmq-tls\" (UniqueName: \"kubernetes.io/projected/185fdd2a-cd57-45e5-9986-bebd15e5761f-rabbitmq-tls\") pod \"185fdd2a-cd57-45e5-9986-bebd15e5761f\" (UID: \"185fdd2a-cd57-45e5-9986-bebd15e5761f\") " Dec 03 12:09:33 crc kubenswrapper[5052]: I1203 12:09:33.087973 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"rabbitmq-erlang-cookie\" (UniqueName: \"kubernetes.io/empty-dir/185fdd2a-cd57-45e5-9986-bebd15e5761f-rabbitmq-erlang-cookie\") pod \"185fdd2a-cd57-45e5-9986-bebd15e5761f\" (UID: \"185fdd2a-cd57-45e5-9986-bebd15e5761f\") " Dec 03 12:09:33 crc kubenswrapper[5052]: I1203 12:09:33.088012 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"erlang-cookie-secret\" (UniqueName: \"kubernetes.io/secret/185fdd2a-cd57-45e5-9986-bebd15e5761f-erlang-cookie-secret\") pod \"185fdd2a-cd57-45e5-9986-bebd15e5761f\" (UID: \"185fdd2a-cd57-45e5-9986-bebd15e5761f\") " Dec 03 12:09:33 crc kubenswrapper[5052]: I1203 12:09:33.088041 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"pod-info\" (UniqueName: \"kubernetes.io/downward-api/185fdd2a-cd57-45e5-9986-bebd15e5761f-pod-info\") pod \"185fdd2a-cd57-45e5-9986-bebd15e5761f\" (UID: \"185fdd2a-cd57-45e5-9986-bebd15e5761f\") " Dec 03 12:09:33 crc kubenswrapper[5052]: I1203 12:09:33.088101 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"plugins-conf\" (UniqueName: \"kubernetes.io/configmap/185fdd2a-cd57-45e5-9986-bebd15e5761f-plugins-conf\") pod \"185fdd2a-cd57-45e5-9986-bebd15e5761f\" (UID: \"185fdd2a-cd57-45e5-9986-bebd15e5761f\") " Dec 03 12:09:33 crc kubenswrapper[5052]: I1203 12:09:33.088125 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p77nm\" (UniqueName: \"kubernetes.io/projected/185fdd2a-cd57-45e5-9986-bebd15e5761f-kube-api-access-p77nm\") pod \"185fdd2a-cd57-45e5-9986-bebd15e5761f\" (UID: \"185fdd2a-cd57-45e5-9986-bebd15e5761f\") " Dec 03 12:09:33 crc kubenswrapper[5052]: I1203 12:09:33.088147 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"server-conf\" (UniqueName: \"kubernetes.io/configmap/185fdd2a-cd57-45e5-9986-bebd15e5761f-server-conf\") pod \"185fdd2a-cd57-45e5-9986-bebd15e5761f\" (UID: \"185fdd2a-cd57-45e5-9986-bebd15e5761f\") " Dec 03 12:09:33 crc kubenswrapper[5052]: I1203 12:09:33.088190 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"persistence\" (UniqueName: \"kubernetes.io/local-volume/local-storage18-crc\") pod \"185fdd2a-cd57-45e5-9986-bebd15e5761f\" (UID: \"185fdd2a-cd57-45e5-9986-bebd15e5761f\") " Dec 03 12:09:33 crc kubenswrapper[5052]: I1203 12:09:33.088213 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"rabbitmq-confd\" (UniqueName: \"kubernetes.io/projected/185fdd2a-cd57-45e5-9986-bebd15e5761f-rabbitmq-confd\") pod \"185fdd2a-cd57-45e5-9986-bebd15e5761f\" (UID: \"185fdd2a-cd57-45e5-9986-bebd15e5761f\") " Dec 03 12:09:33 crc kubenswrapper[5052]: I1203 12:09:33.091251 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/185fdd2a-cd57-45e5-9986-bebd15e5761f-plugins-conf" (OuterVolumeSpecName: "plugins-conf") pod "185fdd2a-cd57-45e5-9986-bebd15e5761f" (UID: "185fdd2a-cd57-45e5-9986-bebd15e5761f"). InnerVolumeSpecName "plugins-conf". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:09:33 crc kubenswrapper[5052]: I1203 12:09:33.091569 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/185fdd2a-cd57-45e5-9986-bebd15e5761f-rabbitmq-plugins" (OuterVolumeSpecName: "rabbitmq-plugins") pod "185fdd2a-cd57-45e5-9986-bebd15e5761f" (UID: "185fdd2a-cd57-45e5-9986-bebd15e5761f"). InnerVolumeSpecName "rabbitmq-plugins". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:09:33 crc kubenswrapper[5052]: I1203 12:09:33.092211 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/185fdd2a-cd57-45e5-9986-bebd15e5761f-rabbitmq-erlang-cookie" (OuterVolumeSpecName: "rabbitmq-erlang-cookie") pod "185fdd2a-cd57-45e5-9986-bebd15e5761f" (UID: "185fdd2a-cd57-45e5-9986-bebd15e5761f"). InnerVolumeSpecName "rabbitmq-erlang-cookie". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:09:33 crc kubenswrapper[5052]: I1203 12:09:33.095866 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/185fdd2a-cd57-45e5-9986-bebd15e5761f-pod-info" (OuterVolumeSpecName: "pod-info") pod "185fdd2a-cd57-45e5-9986-bebd15e5761f" (UID: "185fdd2a-cd57-45e5-9986-bebd15e5761f"). InnerVolumeSpecName "pod-info". PluginName "kubernetes.io/downward-api", VolumeGidValue "" Dec 03 12:09:33 crc kubenswrapper[5052]: I1203 12:09:33.095887 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/185fdd2a-cd57-45e5-9986-bebd15e5761f-erlang-cookie-secret" (OuterVolumeSpecName: "erlang-cookie-secret") pod "185fdd2a-cd57-45e5-9986-bebd15e5761f" (UID: "185fdd2a-cd57-45e5-9986-bebd15e5761f"). InnerVolumeSpecName "erlang-cookie-secret". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:09:33 crc kubenswrapper[5052]: I1203 12:09:33.097323 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/185fdd2a-cd57-45e5-9986-bebd15e5761f-kube-api-access-p77nm" (OuterVolumeSpecName: "kube-api-access-p77nm") pod "185fdd2a-cd57-45e5-9986-bebd15e5761f" (UID: "185fdd2a-cd57-45e5-9986-bebd15e5761f"). InnerVolumeSpecName "kube-api-access-p77nm". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:09:33 crc kubenswrapper[5052]: I1203 12:09:33.097594 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/185fdd2a-cd57-45e5-9986-bebd15e5761f-rabbitmq-tls" (OuterVolumeSpecName: "rabbitmq-tls") pod "185fdd2a-cd57-45e5-9986-bebd15e5761f" (UID: "185fdd2a-cd57-45e5-9986-bebd15e5761f"). InnerVolumeSpecName "rabbitmq-tls". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:09:33 crc kubenswrapper[5052]: I1203 12:09:33.111905 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage18-crc" (OuterVolumeSpecName: "persistence") pod "185fdd2a-cd57-45e5-9986-bebd15e5761f" (UID: "185fdd2a-cd57-45e5-9986-bebd15e5761f"). InnerVolumeSpecName "local-storage18-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 12:09:33 crc kubenswrapper[5052]: I1203 12:09:33.121893 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/185fdd2a-cd57-45e5-9986-bebd15e5761f-config-data" (OuterVolumeSpecName: "config-data") pod "185fdd2a-cd57-45e5-9986-bebd15e5761f" (UID: "185fdd2a-cd57-45e5-9986-bebd15e5761f"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:09:33 crc kubenswrapper[5052]: I1203 12:09:33.150283 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/185fdd2a-cd57-45e5-9986-bebd15e5761f-server-conf" (OuterVolumeSpecName: "server-conf") pod "185fdd2a-cd57-45e5-9986-bebd15e5761f" (UID: "185fdd2a-cd57-45e5-9986-bebd15e5761f"). InnerVolumeSpecName "server-conf". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:09:33 crc kubenswrapper[5052]: I1203 12:09:33.188873 5052 reconciler_common.go:293] "Volume detached for volume \"rabbitmq-tls\" (UniqueName: \"kubernetes.io/projected/185fdd2a-cd57-45e5-9986-bebd15e5761f-rabbitmq-tls\") on node \"crc\" DevicePath \"\"" Dec 03 12:09:33 crc kubenswrapper[5052]: I1203 12:09:33.189098 5052 reconciler_common.go:293] "Volume detached for volume \"rabbitmq-erlang-cookie\" (UniqueName: \"kubernetes.io/empty-dir/185fdd2a-cd57-45e5-9986-bebd15e5761f-rabbitmq-erlang-cookie\") on node \"crc\" DevicePath \"\"" Dec 03 12:09:33 crc kubenswrapper[5052]: I1203 12:09:33.189157 5052 reconciler_common.go:293] "Volume detached for volume \"erlang-cookie-secret\" (UniqueName: \"kubernetes.io/secret/185fdd2a-cd57-45e5-9986-bebd15e5761f-erlang-cookie-secret\") on node \"crc\" DevicePath \"\"" Dec 03 12:09:33 crc kubenswrapper[5052]: I1203 12:09:33.189211 5052 reconciler_common.go:293] "Volume detached for volume \"pod-info\" (UniqueName: \"kubernetes.io/downward-api/185fdd2a-cd57-45e5-9986-bebd15e5761f-pod-info\") on node \"crc\" DevicePath \"\"" Dec 03 12:09:33 crc kubenswrapper[5052]: I1203 12:09:33.189269 5052 reconciler_common.go:293] "Volume detached for volume \"plugins-conf\" (UniqueName: \"kubernetes.io/configmap/185fdd2a-cd57-45e5-9986-bebd15e5761f-plugins-conf\") on node \"crc\" DevicePath \"\"" Dec 03 12:09:33 crc kubenswrapper[5052]: I1203 12:09:33.189321 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-p77nm\" (UniqueName: \"kubernetes.io/projected/185fdd2a-cd57-45e5-9986-bebd15e5761f-kube-api-access-p77nm\") on node \"crc\" DevicePath \"\"" Dec 03 12:09:33 crc kubenswrapper[5052]: I1203 12:09:33.189371 5052 reconciler_common.go:293] "Volume detached for volume \"server-conf\" (UniqueName: \"kubernetes.io/configmap/185fdd2a-cd57-45e5-9986-bebd15e5761f-server-conf\") on node \"crc\" DevicePath \"\"" Dec 03 12:09:33 crc kubenswrapper[5052]: I1203 12:09:33.189447 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage18-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage18-crc\") on node \"crc\" " Dec 03 12:09:33 crc kubenswrapper[5052]: I1203 12:09:33.189502 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/185fdd2a-cd57-45e5-9986-bebd15e5761f-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:09:33 crc kubenswrapper[5052]: I1203 12:09:33.189576 5052 reconciler_common.go:293] "Volume detached for volume \"rabbitmq-plugins\" (UniqueName: \"kubernetes.io/empty-dir/185fdd2a-cd57-45e5-9986-bebd15e5761f-rabbitmq-plugins\") on node \"crc\" DevicePath \"\"" Dec 03 12:09:33 crc kubenswrapper[5052]: I1203 12:09:33.203249 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/185fdd2a-cd57-45e5-9986-bebd15e5761f-rabbitmq-confd" (OuterVolumeSpecName: "rabbitmq-confd") pod "185fdd2a-cd57-45e5-9986-bebd15e5761f" (UID: "185fdd2a-cd57-45e5-9986-bebd15e5761f"). InnerVolumeSpecName "rabbitmq-confd". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:09:33 crc kubenswrapper[5052]: I1203 12:09:33.212022 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage18-crc" (UniqueName: "kubernetes.io/local-volume/local-storage18-crc") on node "crc" Dec 03 12:09:33 crc kubenswrapper[5052]: I1203 12:09:33.290108 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage18-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage18-crc\") on node \"crc\" DevicePath \"\"" Dec 03 12:09:33 crc kubenswrapper[5052]: I1203 12:09:33.290132 5052 reconciler_common.go:293] "Volume detached for volume \"rabbitmq-confd\" (UniqueName: \"kubernetes.io/projected/185fdd2a-cd57-45e5-9986-bebd15e5761f-rabbitmq-confd\") on node \"crc\" DevicePath \"\"" Dec 03 12:09:33 crc kubenswrapper[5052]: I1203 12:09:33.323044 5052 generic.go:334] "Generic (PLEG): container finished" podID="185fdd2a-cd57-45e5-9986-bebd15e5761f" containerID="c2f60fc531710d84b2e54ecd132f7312264860d279c9cbb5f54ba76be0eb79aa" exitCode=0 Dec 03 12:09:33 crc kubenswrapper[5052]: I1203 12:09:33.323285 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" event={"ID":"185fdd2a-cd57-45e5-9986-bebd15e5761f","Type":"ContainerDied","Data":"c2f60fc531710d84b2e54ecd132f7312264860d279c9cbb5f54ba76be0eb79aa"} Dec 03 12:09:33 crc kubenswrapper[5052]: I1203 12:09:33.323363 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" event={"ID":"185fdd2a-cd57-45e5-9986-bebd15e5761f","Type":"ContainerDied","Data":"a2993efc1065f6ca99a9c6fd96b8a330dba2947b6fec8ceb1edc381ae2fcfce5"} Dec 03 12:09:33 crc kubenswrapper[5052]: I1203 12:09:33.323428 5052 scope.go:117] "RemoveContainer" containerID="c2f60fc531710d84b2e54ecd132f7312264860d279c9cbb5f54ba76be0eb79aa" Dec 03 12:09:33 crc kubenswrapper[5052]: I1203 12:09:33.323641 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:09:33 crc kubenswrapper[5052]: I1203 12:09:33.359788 5052 scope.go:117] "RemoveContainer" containerID="6b72ea83c326e823c684041451c505add9ed54cb24251b5b99e7e2a2086b0b19" Dec 03 12:09:33 crc kubenswrapper[5052]: I1203 12:09:33.369381 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/rabbitmq-cell1-server-0"] Dec 03 12:09:33 crc kubenswrapper[5052]: I1203 12:09:33.381470 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/rabbitmq-cell1-server-0"] Dec 03 12:09:33 crc kubenswrapper[5052]: I1203 12:09:33.393865 5052 scope.go:117] "RemoveContainer" containerID="c2f60fc531710d84b2e54ecd132f7312264860d279c9cbb5f54ba76be0eb79aa" Dec 03 12:09:33 crc kubenswrapper[5052]: E1203 12:09:33.394578 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c2f60fc531710d84b2e54ecd132f7312264860d279c9cbb5f54ba76be0eb79aa\": container with ID starting with c2f60fc531710d84b2e54ecd132f7312264860d279c9cbb5f54ba76be0eb79aa not found: ID does not exist" containerID="c2f60fc531710d84b2e54ecd132f7312264860d279c9cbb5f54ba76be0eb79aa" Dec 03 12:09:33 crc kubenswrapper[5052]: I1203 12:09:33.394616 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c2f60fc531710d84b2e54ecd132f7312264860d279c9cbb5f54ba76be0eb79aa"} err="failed to get container status \"c2f60fc531710d84b2e54ecd132f7312264860d279c9cbb5f54ba76be0eb79aa\": rpc error: code = NotFound desc = could not find container \"c2f60fc531710d84b2e54ecd132f7312264860d279c9cbb5f54ba76be0eb79aa\": container with ID starting with c2f60fc531710d84b2e54ecd132f7312264860d279c9cbb5f54ba76be0eb79aa not found: ID does not exist" Dec 03 12:09:33 crc kubenswrapper[5052]: I1203 12:09:33.394639 5052 scope.go:117] "RemoveContainer" containerID="6b72ea83c326e823c684041451c505add9ed54cb24251b5b99e7e2a2086b0b19" Dec 03 12:09:33 crc kubenswrapper[5052]: E1203 12:09:33.395038 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6b72ea83c326e823c684041451c505add9ed54cb24251b5b99e7e2a2086b0b19\": container with ID starting with 6b72ea83c326e823c684041451c505add9ed54cb24251b5b99e7e2a2086b0b19 not found: ID does not exist" containerID="6b72ea83c326e823c684041451c505add9ed54cb24251b5b99e7e2a2086b0b19" Dec 03 12:09:33 crc kubenswrapper[5052]: I1203 12:09:33.395079 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6b72ea83c326e823c684041451c505add9ed54cb24251b5b99e7e2a2086b0b19"} err="failed to get container status \"6b72ea83c326e823c684041451c505add9ed54cb24251b5b99e7e2a2086b0b19\": rpc error: code = NotFound desc = could not find container \"6b72ea83c326e823c684041451c505add9ed54cb24251b5b99e7e2a2086b0b19\": container with ID starting with 6b72ea83c326e823c684041451c505add9ed54cb24251b5b99e7e2a2086b0b19 not found: ID does not exist" Dec 03 12:09:33 crc kubenswrapper[5052]: I1203 12:09:33.683535 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="185fdd2a-cd57-45e5-9986-bebd15e5761f" path="/var/lib/kubelet/pods/185fdd2a-cd57-45e5-9986-bebd15e5761f/volumes" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.368495 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["crc-storage/crc-storage-crc-zhb4j"] Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.376246 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["crc-storage/crc-storage-crc-zhb4j"] Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.532577 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["crc-storage/crc-storage-crc-gl6m8"] Dec 03 12:09:40 crc kubenswrapper[5052]: E1203 12:09:40.533044 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="45e26e8c-71b5-4b90-bba7-5bad2650d9b2" containerName="nova-api-log" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.533067 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="45e26e8c-71b5-4b90-bba7-5bad2650d9b2" containerName="nova-api-log" Dec 03 12:09:40 crc kubenswrapper[5052]: E1203 12:09:40.533086 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="95bf5f72-82f3-4e88-a576-01dae857541f" containerName="object-replicator" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.533096 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="95bf5f72-82f3-4e88-a576-01dae857541f" containerName="object-replicator" Dec 03 12:09:40 crc kubenswrapper[5052]: E1203 12:09:40.533110 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="fad6f453-acf5-4f13-94d6-7d46e300f28c" containerName="ovn-northd" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.533118 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="fad6f453-acf5-4f13-94d6-7d46e300f28c" containerName="ovn-northd" Dec 03 12:09:40 crc kubenswrapper[5052]: E1203 12:09:40.533129 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="95bf5f72-82f3-4e88-a576-01dae857541f" containerName="object-server" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.533135 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="95bf5f72-82f3-4e88-a576-01dae857541f" containerName="object-server" Dec 03 12:09:40 crc kubenswrapper[5052]: E1203 12:09:40.533146 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="de04776b-ea02-40d9-8459-0a519c8c64ad" containerName="barbican-api" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.533151 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="de04776b-ea02-40d9-8459-0a519c8c64ad" containerName="barbican-api" Dec 03 12:09:40 crc kubenswrapper[5052]: E1203 12:09:40.533160 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="19229e07-3c78-4369-998e-fc9e78eb647d" containerName="sg-core" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.533165 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="19229e07-3c78-4369-998e-fc9e78eb647d" containerName="sg-core" Dec 03 12:09:40 crc kubenswrapper[5052]: E1203 12:09:40.533171 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="185fdd2a-cd57-45e5-9986-bebd15e5761f" containerName="rabbitmq" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.533177 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="185fdd2a-cd57-45e5-9986-bebd15e5761f" containerName="rabbitmq" Dec 03 12:09:40 crc kubenswrapper[5052]: E1203 12:09:40.533187 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="95bf5f72-82f3-4e88-a576-01dae857541f" containerName="object-auditor" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.533193 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="95bf5f72-82f3-4e88-a576-01dae857541f" containerName="object-auditor" Dec 03 12:09:40 crc kubenswrapper[5052]: E1203 12:09:40.533201 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="2f3b4f6a-5f11-4039-a3bd-7b553ab5684d" containerName="mariadb-account-delete" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.533206 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f3b4f6a-5f11-4039-a3bd-7b553ab5684d" containerName="mariadb-account-delete" Dec 03 12:09:40 crc kubenswrapper[5052]: E1203 12:09:40.533215 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="aaa3db64-a10b-45ea-9e25-de4916f30617" containerName="mariadb-account-delete" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.533221 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="aaa3db64-a10b-45ea-9e25-de4916f30617" containerName="mariadb-account-delete" Dec 03 12:09:40 crc kubenswrapper[5052]: E1203 12:09:40.533233 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="d66ed4ec-fd97-4a62-82cb-cee3fe877524" containerName="galera" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.533239 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="d66ed4ec-fd97-4a62-82cb-cee3fe877524" containerName="galera" Dec 03 12:09:40 crc kubenswrapper[5052]: E1203 12:09:40.533248 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="95bf5f72-82f3-4e88-a576-01dae857541f" containerName="container-server" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.533254 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="95bf5f72-82f3-4e88-a576-01dae857541f" containerName="container-server" Dec 03 12:09:40 crc kubenswrapper[5052]: E1203 12:09:40.533263 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="89ee1bab-2c68-4fc4-9171-a7330922faf6" containerName="mariadb-account-delete" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.533269 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="89ee1bab-2c68-4fc4-9171-a7330922faf6" containerName="mariadb-account-delete" Dec 03 12:09:40 crc kubenswrapper[5052]: E1203 12:09:40.533277 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="9e6af047-b7ee-459a-b362-da5f777d0fa0" containerName="mariadb-account-delete" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.533284 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="9e6af047-b7ee-459a-b362-da5f777d0fa0" containerName="mariadb-account-delete" Dec 03 12:09:40 crc kubenswrapper[5052]: E1203 12:09:40.533294 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="3b8ce5b7-67a1-48ef-892f-7408010a84f0" containerName="mysql-bootstrap" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.533300 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b8ce5b7-67a1-48ef-892f-7408010a84f0" containerName="mysql-bootstrap" Dec 03 12:09:40 crc kubenswrapper[5052]: E1203 12:09:40.533312 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="b34bb4d0-7906-42e8-a9aa-bd1a4203a312" containerName="galera" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.533318 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="b34bb4d0-7906-42e8-a9aa-bd1a4203a312" containerName="galera" Dec 03 12:09:40 crc kubenswrapper[5052]: E1203 12:09:40.533325 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="95bf5f72-82f3-4e88-a576-01dae857541f" containerName="swift-recon-cron" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.533330 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="95bf5f72-82f3-4e88-a576-01dae857541f" containerName="swift-recon-cron" Dec 03 12:09:40 crc kubenswrapper[5052]: E1203 12:09:40.533338 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="de04776b-ea02-40d9-8459-0a519c8c64ad" containerName="barbican-api-log" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.533344 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="de04776b-ea02-40d9-8459-0a519c8c64ad" containerName="barbican-api-log" Dec 03 12:09:40 crc kubenswrapper[5052]: E1203 12:09:40.533352 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="a3da2acb-8a0e-4ab3-8d15-2978eca73850" containerName="barbican-worker" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.533357 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="a3da2acb-8a0e-4ab3-8d15-2978eca73850" containerName="barbican-worker" Dec 03 12:09:40 crc kubenswrapper[5052]: E1203 12:09:40.533365 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="aaa3db64-a10b-45ea-9e25-de4916f30617" containerName="mariadb-account-delete" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.533370 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="aaa3db64-a10b-45ea-9e25-de4916f30617" containerName="mariadb-account-delete" Dec 03 12:09:40 crc kubenswrapper[5052]: E1203 12:09:40.533381 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="bef08f40-1b01-406f-af05-7b63e0b6e5e7" containerName="glance-httpd" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.533386 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="bef08f40-1b01-406f-af05-7b63e0b6e5e7" containerName="glance-httpd" Dec 03 12:09:40 crc kubenswrapper[5052]: E1203 12:09:40.533394 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="4a2a1881-e61a-4a99-a2f3-b614aeed81d1" containerName="glance-log" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.533400 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="4a2a1881-e61a-4a99-a2f3-b614aeed81d1" containerName="glance-log" Dec 03 12:09:40 crc kubenswrapper[5052]: E1203 12:09:40.533409 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="45e26e8c-71b5-4b90-bba7-5bad2650d9b2" containerName="nova-api-api" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.533415 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="45e26e8c-71b5-4b90-bba7-5bad2650d9b2" containerName="nova-api-api" Dec 03 12:09:40 crc kubenswrapper[5052]: E1203 12:09:40.533425 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="2f3b4f6a-5f11-4039-a3bd-7b553ab5684d" containerName="mariadb-account-delete" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.533439 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f3b4f6a-5f11-4039-a3bd-7b553ab5684d" containerName="mariadb-account-delete" Dec 03 12:09:40 crc kubenswrapper[5052]: E1203 12:09:40.533452 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="06bfd2db-abe0-4b4d-a110-94a6a1e02211" containerName="setup-container" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.533458 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="06bfd2db-abe0-4b4d-a110-94a6a1e02211" containerName="setup-container" Dec 03 12:09:40 crc kubenswrapper[5052]: E1203 12:09:40.533465 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="31e6a111-bd4a-45a5-95a0-a6b537added6" containerName="cinder-api" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.533471 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="31e6a111-bd4a-45a5-95a0-a6b537added6" containerName="cinder-api" Dec 03 12:09:40 crc kubenswrapper[5052]: E1203 12:09:40.533479 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="95bf5f72-82f3-4e88-a576-01dae857541f" containerName="account-reaper" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.533484 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="95bf5f72-82f3-4e88-a576-01dae857541f" containerName="account-reaper" Dec 03 12:09:40 crc kubenswrapper[5052]: E1203 12:09:40.533491 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="95bf5f72-82f3-4e88-a576-01dae857541f" containerName="object-expirer" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.533496 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="95bf5f72-82f3-4e88-a576-01dae857541f" containerName="object-expirer" Dec 03 12:09:40 crc kubenswrapper[5052]: E1203 12:09:40.533503 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="a3da2acb-8a0e-4ab3-8d15-2978eca73850" containerName="barbican-worker-log" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.533560 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="a3da2acb-8a0e-4ab3-8d15-2978eca73850" containerName="barbican-worker-log" Dec 03 12:09:40 crc kubenswrapper[5052]: E1203 12:09:40.533571 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="143f4f1d-4f91-4ffa-a277-d6e551d19838" containerName="galera" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.533577 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="143f4f1d-4f91-4ffa-a277-d6e551d19838" containerName="galera" Dec 03 12:09:40 crc kubenswrapper[5052]: E1203 12:09:40.533587 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3" containerName="rabbitmq" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.533593 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3" containerName="rabbitmq" Dec 03 12:09:40 crc kubenswrapper[5052]: E1203 12:09:40.533602 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="95bf5f72-82f3-4e88-a576-01dae857541f" containerName="account-server" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.533609 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="95bf5f72-82f3-4e88-a576-01dae857541f" containerName="account-server" Dec 03 12:09:40 crc kubenswrapper[5052]: E1203 12:09:40.533617 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="9e6af047-b7ee-459a-b362-da5f777d0fa0" containerName="mariadb-account-delete" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.533623 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="9e6af047-b7ee-459a-b362-da5f777d0fa0" containerName="mariadb-account-delete" Dec 03 12:09:40 crc kubenswrapper[5052]: E1203 12:09:40.533632 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="4a2a1881-e61a-4a99-a2f3-b614aeed81d1" containerName="glance-httpd" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.533638 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="4a2a1881-e61a-4a99-a2f3-b614aeed81d1" containerName="glance-httpd" Dec 03 12:09:40 crc kubenswrapper[5052]: E1203 12:09:40.533649 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="95bf5f72-82f3-4e88-a576-01dae857541f" containerName="container-updater" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.533655 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="95bf5f72-82f3-4e88-a576-01dae857541f" containerName="container-updater" Dec 03 12:09:40 crc kubenswrapper[5052]: E1203 12:09:40.533667 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="185fdd2a-cd57-45e5-9986-bebd15e5761f" containerName="setup-container" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.533673 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="185fdd2a-cd57-45e5-9986-bebd15e5761f" containerName="setup-container" Dec 03 12:09:40 crc kubenswrapper[5052]: E1203 12:09:40.533683 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="c2470da9-4c96-44ac-834d-2b560785135f" containerName="keystone-api" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.533689 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="c2470da9-4c96-44ac-834d-2b560785135f" containerName="keystone-api" Dec 03 12:09:40 crc kubenswrapper[5052]: E1203 12:09:40.533696 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="305f2518-9632-4358-84a1-632b4a7d8f6f" containerName="kube-state-metrics" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.533703 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="305f2518-9632-4358-84a1-632b4a7d8f6f" containerName="kube-state-metrics" Dec 03 12:09:40 crc kubenswrapper[5052]: E1203 12:09:40.533715 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="fad6f453-acf5-4f13-94d6-7d46e300f28c" containerName="openstack-network-exporter" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.533722 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="fad6f453-acf5-4f13-94d6-7d46e300f28c" containerName="openstack-network-exporter" Dec 03 12:09:40 crc kubenswrapper[5052]: E1203 12:09:40.533731 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="95bf5f72-82f3-4e88-a576-01dae857541f" containerName="container-replicator" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.533738 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="95bf5f72-82f3-4e88-a576-01dae857541f" containerName="container-replicator" Dec 03 12:09:40 crc kubenswrapper[5052]: E1203 12:09:40.533746 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="70122e67-6175-4a14-a120-6a2731c5dcb1" containerName="mariadb-account-delete" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.533751 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="70122e67-6175-4a14-a120-6a2731c5dcb1" containerName="mariadb-account-delete" Dec 03 12:09:40 crc kubenswrapper[5052]: E1203 12:09:40.533758 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="89ee1bab-2c68-4fc4-9171-a7330922faf6" containerName="mariadb-account-delete" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.533763 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="89ee1bab-2c68-4fc4-9171-a7330922faf6" containerName="mariadb-account-delete" Dec 03 12:09:40 crc kubenswrapper[5052]: E1203 12:09:40.533771 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="2397d896-0d11-4656-a336-eac8ec9fe7f9" containerName="setup-container" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.533776 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="2397d896-0d11-4656-a336-eac8ec9fe7f9" containerName="setup-container" Dec 03 12:09:40 crc kubenswrapper[5052]: E1203 12:09:40.533784 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="d66ed4ec-fd97-4a62-82cb-cee3fe877524" containerName="mysql-bootstrap" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.533790 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="d66ed4ec-fd97-4a62-82cb-cee3fe877524" containerName="mysql-bootstrap" Dec 03 12:09:40 crc kubenswrapper[5052]: E1203 12:09:40.533799 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="95bf5f72-82f3-4e88-a576-01dae857541f" containerName="container-auditor" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.533805 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="95bf5f72-82f3-4e88-a576-01dae857541f" containerName="container-auditor" Dec 03 12:09:40 crc kubenswrapper[5052]: E1203 12:09:40.533814 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="d877b426-724d-4ccf-a113-05a96ab94935" containerName="barbican-keystone-listener" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.533819 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="d877b426-724d-4ccf-a113-05a96ab94935" containerName="barbican-keystone-listener" Dec 03 12:09:40 crc kubenswrapper[5052]: E1203 12:09:40.533827 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="95bf5f72-82f3-4e88-a576-01dae857541f" containerName="account-replicator" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.533834 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="95bf5f72-82f3-4e88-a576-01dae857541f" containerName="account-replicator" Dec 03 12:09:40 crc kubenswrapper[5052]: E1203 12:09:40.533844 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="0b100677-2da0-4717-a15a-d2463ee6713e" containerName="nova-scheduler-scheduler" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.533849 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="0b100677-2da0-4717-a15a-d2463ee6713e" containerName="nova-scheduler-scheduler" Dec 03 12:09:40 crc kubenswrapper[5052]: E1203 12:09:40.533859 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="d877b426-724d-4ccf-a113-05a96ab94935" containerName="barbican-keystone-listener-log" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.533865 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="d877b426-724d-4ccf-a113-05a96ab94935" containerName="barbican-keystone-listener-log" Dec 03 12:09:40 crc kubenswrapper[5052]: E1203 12:09:40.533876 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="1bc7c8b1-cc0d-4611-b335-01c2b91713e2" containerName="mariadb-account-delete" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.533881 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="1bc7c8b1-cc0d-4611-b335-01c2b91713e2" containerName="mariadb-account-delete" Dec 03 12:09:40 crc kubenswrapper[5052]: E1203 12:09:40.533890 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="bbdcaf66-8813-4bed-9311-552fef62ac2d" containerName="memcached" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.533896 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="bbdcaf66-8813-4bed-9311-552fef62ac2d" containerName="memcached" Dec 03 12:09:40 crc kubenswrapper[5052]: E1203 12:09:40.533907 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="6ac95bd3-125c-4966-b9a7-4eb4a7842c67" containerName="mysql-bootstrap" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.533912 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="6ac95bd3-125c-4966-b9a7-4eb4a7842c67" containerName="mysql-bootstrap" Dec 03 12:09:40 crc kubenswrapper[5052]: E1203 12:09:40.533921 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="7901ce85-4e6f-4e8b-86a6-f50a85803637" containerName="setup-container" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.533926 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="7901ce85-4e6f-4e8b-86a6-f50a85803637" containerName="setup-container" Dec 03 12:09:40 crc kubenswrapper[5052]: E1203 12:09:40.533935 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="60f72892-a5aa-482f-aac3-5713ba028bc8" containerName="cinder-scheduler" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.533941 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="60f72892-a5aa-482f-aac3-5713ba028bc8" containerName="cinder-scheduler" Dec 03 12:09:40 crc kubenswrapper[5052]: E1203 12:09:40.533949 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="1bc7c8b1-cc0d-4611-b335-01c2b91713e2" containerName="mariadb-account-delete" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.533955 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="1bc7c8b1-cc0d-4611-b335-01c2b91713e2" containerName="mariadb-account-delete" Dec 03 12:09:40 crc kubenswrapper[5052]: E1203 12:09:40.533962 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="60f72892-a5aa-482f-aac3-5713ba028bc8" containerName="probe" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.533967 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="60f72892-a5aa-482f-aac3-5713ba028bc8" containerName="probe" Dec 03 12:09:40 crc kubenswrapper[5052]: E1203 12:09:40.533973 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="77077fe5-d1ff-49c4-b049-3a0f2cee23b0" containerName="dnsmasq-dns" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.533979 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="77077fe5-d1ff-49c4-b049-3a0f2cee23b0" containerName="dnsmasq-dns" Dec 03 12:09:40 crc kubenswrapper[5052]: E1203 12:09:40.533988 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="70122e67-6175-4a14-a120-6a2731c5dcb1" containerName="mariadb-account-delete" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.533993 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="70122e67-6175-4a14-a120-6a2731c5dcb1" containerName="mariadb-account-delete" Dec 03 12:09:40 crc kubenswrapper[5052]: E1203 12:09:40.534004 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="7b4f6078-41a3-433d-acaf-f2d67391ab6e" containerName="setup-container" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.534009 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="7b4f6078-41a3-433d-acaf-f2d67391ab6e" containerName="setup-container" Dec 03 12:09:40 crc kubenswrapper[5052]: E1203 12:09:40.534019 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="141a49da-93b6-483b-a476-1d5f3efc29d1" containerName="neutron-api" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.534024 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="141a49da-93b6-483b-a476-1d5f3efc29d1" containerName="neutron-api" Dec 03 12:09:40 crc kubenswrapper[5052]: E1203 12:09:40.534032 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="19229e07-3c78-4369-998e-fc9e78eb647d" containerName="proxy-httpd" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.534037 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="19229e07-3c78-4369-998e-fc9e78eb647d" containerName="proxy-httpd" Dec 03 12:09:40 crc kubenswrapper[5052]: E1203 12:09:40.534046 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="7901ce85-4e6f-4e8b-86a6-f50a85803637" containerName="rabbitmq" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.534052 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="7901ce85-4e6f-4e8b-86a6-f50a85803637" containerName="rabbitmq" Dec 03 12:09:40 crc kubenswrapper[5052]: E1203 12:09:40.534058 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="bef08f40-1b01-406f-af05-7b63e0b6e5e7" containerName="glance-log" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.534064 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="bef08f40-1b01-406f-af05-7b63e0b6e5e7" containerName="glance-log" Dec 03 12:09:40 crc kubenswrapper[5052]: E1203 12:09:40.534075 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="b34bb4d0-7906-42e8-a9aa-bd1a4203a312" containerName="mysql-bootstrap" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.534081 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="b34bb4d0-7906-42e8-a9aa-bd1a4203a312" containerName="mysql-bootstrap" Dec 03 12:09:40 crc kubenswrapper[5052]: E1203 12:09:40.534088 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="143f4f1d-4f91-4ffa-a277-d6e551d19838" containerName="mysql-bootstrap" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.534093 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="143f4f1d-4f91-4ffa-a277-d6e551d19838" containerName="mysql-bootstrap" Dec 03 12:09:40 crc kubenswrapper[5052]: E1203 12:09:40.534102 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="3b8ce5b7-67a1-48ef-892f-7408010a84f0" containerName="galera" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.534107 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b8ce5b7-67a1-48ef-892f-7408010a84f0" containerName="galera" Dec 03 12:09:40 crc kubenswrapper[5052]: E1203 12:09:40.534115 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="95bf5f72-82f3-4e88-a576-01dae857541f" containerName="object-updater" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.534121 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="95bf5f72-82f3-4e88-a576-01dae857541f" containerName="object-updater" Dec 03 12:09:40 crc kubenswrapper[5052]: E1203 12:09:40.534129 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="77077fe5-d1ff-49c4-b049-3a0f2cee23b0" containerName="init" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.534134 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="77077fe5-d1ff-49c4-b049-3a0f2cee23b0" containerName="init" Dec 03 12:09:40 crc kubenswrapper[5052]: E1203 12:09:40.534143 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="6b92f88e-1325-4c3d-8d4e-4089a9033775" containerName="galera" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.534148 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="6b92f88e-1325-4c3d-8d4e-4089a9033775" containerName="galera" Dec 03 12:09:40 crc kubenswrapper[5052]: E1203 12:09:40.534156 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="6ac95bd3-125c-4966-b9a7-4eb4a7842c67" containerName="galera" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.534162 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="6ac95bd3-125c-4966-b9a7-4eb4a7842c67" containerName="galera" Dec 03 12:09:40 crc kubenswrapper[5052]: E1203 12:09:40.534171 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="030d4f76-eb30-4c44-b782-fa7147febf41" containerName="mariadb-account-delete" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.534177 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="030d4f76-eb30-4c44-b782-fa7147febf41" containerName="mariadb-account-delete" Dec 03 12:09:40 crc kubenswrapper[5052]: E1203 12:09:40.534184 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="95bf5f72-82f3-4e88-a576-01dae857541f" containerName="rsync" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.534190 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="95bf5f72-82f3-4e88-a576-01dae857541f" containerName="rsync" Dec 03 12:09:40 crc kubenswrapper[5052]: E1203 12:09:40.534198 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3" containerName="setup-container" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.534204 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3" containerName="setup-container" Dec 03 12:09:40 crc kubenswrapper[5052]: E1203 12:09:40.534215 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="6b92f88e-1325-4c3d-8d4e-4089a9033775" containerName="mysql-bootstrap" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.534221 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="6b92f88e-1325-4c3d-8d4e-4089a9033775" containerName="mysql-bootstrap" Dec 03 12:09:40 crc kubenswrapper[5052]: E1203 12:09:40.534228 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="06bfd2db-abe0-4b4d-a110-94a6a1e02211" containerName="rabbitmq" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.534233 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="06bfd2db-abe0-4b4d-a110-94a6a1e02211" containerName="rabbitmq" Dec 03 12:09:40 crc kubenswrapper[5052]: E1203 12:09:40.534240 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="030d4f76-eb30-4c44-b782-fa7147febf41" containerName="mariadb-account-delete" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.534247 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="030d4f76-eb30-4c44-b782-fa7147febf41" containerName="mariadb-account-delete" Dec 03 12:09:40 crc kubenswrapper[5052]: E1203 12:09:40.534254 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="95bf5f72-82f3-4e88-a576-01dae857541f" containerName="account-auditor" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.534261 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="95bf5f72-82f3-4e88-a576-01dae857541f" containerName="account-auditor" Dec 03 12:09:40 crc kubenswrapper[5052]: E1203 12:09:40.534272 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="7b4f6078-41a3-433d-acaf-f2d67391ab6e" containerName="rabbitmq" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.534277 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="7b4f6078-41a3-433d-acaf-f2d67391ab6e" containerName="rabbitmq" Dec 03 12:09:40 crc kubenswrapper[5052]: E1203 12:09:40.534287 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="2397d896-0d11-4656-a336-eac8ec9fe7f9" containerName="rabbitmq" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.534293 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="2397d896-0d11-4656-a336-eac8ec9fe7f9" containerName="rabbitmq" Dec 03 12:09:40 crc kubenswrapper[5052]: E1203 12:09:40.534300 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="19229e07-3c78-4369-998e-fc9e78eb647d" containerName="ceilometer-notification-agent" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.534306 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="19229e07-3c78-4369-998e-fc9e78eb647d" containerName="ceilometer-notification-agent" Dec 03 12:09:40 crc kubenswrapper[5052]: E1203 12:09:40.534316 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="31e6a111-bd4a-45a5-95a0-a6b537added6" containerName="cinder-api-log" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.534322 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="31e6a111-bd4a-45a5-95a0-a6b537added6" containerName="cinder-api-log" Dec 03 12:09:40 crc kubenswrapper[5052]: E1203 12:09:40.534328 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="141a49da-93b6-483b-a476-1d5f3efc29d1" containerName="neutron-httpd" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.534334 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="141a49da-93b6-483b-a476-1d5f3efc29d1" containerName="neutron-httpd" Dec 03 12:09:40 crc kubenswrapper[5052]: E1203 12:09:40.534342 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="19229e07-3c78-4369-998e-fc9e78eb647d" containerName="ceilometer-central-agent" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.534348 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="19229e07-3c78-4369-998e-fc9e78eb647d" containerName="ceilometer-central-agent" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.534487 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="b34bb4d0-7906-42e8-a9aa-bd1a4203a312" containerName="galera" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.534498 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="89ee1bab-2c68-4fc4-9171-a7330922faf6" containerName="mariadb-account-delete" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.534529 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="030d4f76-eb30-4c44-b782-fa7147febf41" containerName="mariadb-account-delete" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.534544 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="9e6af047-b7ee-459a-b362-da5f777d0fa0" containerName="mariadb-account-delete" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.534554 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="60f72892-a5aa-482f-aac3-5713ba028bc8" containerName="cinder-scheduler" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.534566 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="a3da2acb-8a0e-4ab3-8d15-2978eca73850" containerName="barbican-worker-log" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.534573 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="bef08f40-1b01-406f-af05-7b63e0b6e5e7" containerName="glance-httpd" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.534586 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="d877b426-724d-4ccf-a113-05a96ab94935" containerName="barbican-keystone-listener" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.534595 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="95bf5f72-82f3-4e88-a576-01dae857541f" containerName="object-updater" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.534605 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="06bfd2db-abe0-4b4d-a110-94a6a1e02211" containerName="rabbitmq" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.534613 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="030d4f76-eb30-4c44-b782-fa7147febf41" containerName="mariadb-account-delete" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.534620 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="95bf5f72-82f3-4e88-a576-01dae857541f" containerName="account-replicator" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.534629 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="fad6f453-acf5-4f13-94d6-7d46e300f28c" containerName="openstack-network-exporter" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.534639 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="6b92f88e-1325-4c3d-8d4e-4089a9033775" containerName="galera" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.534647 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="c2470da9-4c96-44ac-834d-2b560785135f" containerName="keystone-api" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.534656 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="de04776b-ea02-40d9-8459-0a519c8c64ad" containerName="barbican-api-log" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.534663 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="185fdd2a-cd57-45e5-9986-bebd15e5761f" containerName="rabbitmq" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.534672 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="f7d2a2ad-3910-425b-8bfe-e7fdc5e0f1e3" containerName="rabbitmq" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.534679 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="7901ce85-4e6f-4e8b-86a6-f50a85803637" containerName="rabbitmq" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.534684 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="19229e07-3c78-4369-998e-fc9e78eb647d" containerName="ceilometer-central-agent" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.534691 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="19229e07-3c78-4369-998e-fc9e78eb647d" containerName="proxy-httpd" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.534697 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="77077fe5-d1ff-49c4-b049-3a0f2cee23b0" containerName="dnsmasq-dns" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.534702 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="95bf5f72-82f3-4e88-a576-01dae857541f" containerName="container-auditor" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.534708 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="95bf5f72-82f3-4e88-a576-01dae857541f" containerName="swift-recon-cron" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.534715 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="95bf5f72-82f3-4e88-a576-01dae857541f" containerName="container-replicator" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.534721 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="95bf5f72-82f3-4e88-a576-01dae857541f" containerName="container-updater" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.534727 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="4a2a1881-e61a-4a99-a2f3-b614aeed81d1" containerName="glance-httpd" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.534737 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="143f4f1d-4f91-4ffa-a277-d6e551d19838" containerName="galera" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.534744 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="fad6f453-acf5-4f13-94d6-7d46e300f28c" containerName="ovn-northd" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.534754 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="95bf5f72-82f3-4e88-a576-01dae857541f" containerName="object-replicator" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.534763 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="141a49da-93b6-483b-a476-1d5f3efc29d1" containerName="neutron-api" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.534769 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="1bc7c8b1-cc0d-4611-b335-01c2b91713e2" containerName="mariadb-account-delete" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.534776 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="6ac95bd3-125c-4966-b9a7-4eb4a7842c67" containerName="galera" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.534785 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="1bc7c8b1-cc0d-4611-b335-01c2b91713e2" containerName="mariadb-account-delete" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.534792 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="141a49da-93b6-483b-a476-1d5f3efc29d1" containerName="neutron-httpd" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.534798 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="70122e67-6175-4a14-a120-6a2731c5dcb1" containerName="mariadb-account-delete" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.534806 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="7b4f6078-41a3-433d-acaf-f2d67391ab6e" containerName="rabbitmq" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.534816 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="95bf5f72-82f3-4e88-a576-01dae857541f" containerName="object-auditor" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.534825 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="60f72892-a5aa-482f-aac3-5713ba028bc8" containerName="probe" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.534832 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="d66ed4ec-fd97-4a62-82cb-cee3fe877524" containerName="galera" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.534839 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="2397d896-0d11-4656-a336-eac8ec9fe7f9" containerName="rabbitmq" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.534846 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="2f3b4f6a-5f11-4039-a3bd-7b553ab5684d" containerName="mariadb-account-delete" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.534855 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="31e6a111-bd4a-45a5-95a0-a6b537added6" containerName="cinder-api-log" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.534861 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="aaa3db64-a10b-45ea-9e25-de4916f30617" containerName="mariadb-account-delete" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.534867 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="31e6a111-bd4a-45a5-95a0-a6b537added6" containerName="cinder-api" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.534876 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="95bf5f72-82f3-4e88-a576-01dae857541f" containerName="object-expirer" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.534886 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="45e26e8c-71b5-4b90-bba7-5bad2650d9b2" containerName="nova-api-log" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.534895 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="45e26e8c-71b5-4b90-bba7-5bad2650d9b2" containerName="nova-api-api" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.534902 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="4a2a1881-e61a-4a99-a2f3-b614aeed81d1" containerName="glance-log" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.534909 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="d877b426-724d-4ccf-a113-05a96ab94935" containerName="barbican-keystone-listener-log" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.534917 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="305f2518-9632-4358-84a1-632b4a7d8f6f" containerName="kube-state-metrics" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.534924 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="2f3b4f6a-5f11-4039-a3bd-7b553ab5684d" containerName="mariadb-account-delete" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.534931 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="19229e07-3c78-4369-998e-fc9e78eb647d" containerName="sg-core" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.534939 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="95bf5f72-82f3-4e88-a576-01dae857541f" containerName="account-auditor" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.534946 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="3b8ce5b7-67a1-48ef-892f-7408010a84f0" containerName="galera" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.534954 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="bef08f40-1b01-406f-af05-7b63e0b6e5e7" containerName="glance-log" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.534961 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="de04776b-ea02-40d9-8459-0a519c8c64ad" containerName="barbican-api" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.534968 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="bbdcaf66-8813-4bed-9311-552fef62ac2d" containerName="memcached" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.534976 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="95bf5f72-82f3-4e88-a576-01dae857541f" containerName="rsync" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.534984 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="19229e07-3c78-4369-998e-fc9e78eb647d" containerName="ceilometer-notification-agent" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.534990 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="95bf5f72-82f3-4e88-a576-01dae857541f" containerName="account-reaper" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.534996 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="0b100677-2da0-4717-a15a-d2463ee6713e" containerName="nova-scheduler-scheduler" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.535002 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="a3da2acb-8a0e-4ab3-8d15-2978eca73850" containerName="barbican-worker" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.535010 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="95bf5f72-82f3-4e88-a576-01dae857541f" containerName="container-server" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.535017 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="95bf5f72-82f3-4e88-a576-01dae857541f" containerName="object-server" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.535028 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="95bf5f72-82f3-4e88-a576-01dae857541f" containerName="account-server" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.535610 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="crc-storage/crc-storage-crc-gl6m8" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.537668 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"crc-storage"/"kube-root-ca.crt" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.537925 5052 reflector.go:368] Caches populated for *v1.Secret from object-"crc-storage"/"crc-storage-dockercfg-mf76g" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.538003 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"crc-storage"/"crc-storage" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.538773 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"crc-storage"/"openshift-service-ca.crt" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.541306 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["crc-storage/crc-storage-crc-gl6m8"] Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.541582 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/e776931b-5cd1-4d1f-8e59-70d60f4750ae-crc-storage\") pod \"crc-storage-crc-gl6m8\" (UID: \"e776931b-5cd1-4d1f-8e59-70d60f4750ae\") " pod="crc-storage/crc-storage-crc-gl6m8" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.643658 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cnrw4\" (UniqueName: \"kubernetes.io/projected/e776931b-5cd1-4d1f-8e59-70d60f4750ae-kube-api-access-cnrw4\") pod \"crc-storage-crc-gl6m8\" (UID: \"e776931b-5cd1-4d1f-8e59-70d60f4750ae\") " pod="crc-storage/crc-storage-crc-gl6m8" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.643793 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/e776931b-5cd1-4d1f-8e59-70d60f4750ae-crc-storage\") pod \"crc-storage-crc-gl6m8\" (UID: \"e776931b-5cd1-4d1f-8e59-70d60f4750ae\") " pod="crc-storage/crc-storage-crc-gl6m8" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.643835 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/e776931b-5cd1-4d1f-8e59-70d60f4750ae-node-mnt\") pod \"crc-storage-crc-gl6m8\" (UID: \"e776931b-5cd1-4d1f-8e59-70d60f4750ae\") " pod="crc-storage/crc-storage-crc-gl6m8" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.644409 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/e776931b-5cd1-4d1f-8e59-70d60f4750ae-crc-storage\") pod \"crc-storage-crc-gl6m8\" (UID: \"e776931b-5cd1-4d1f-8e59-70d60f4750ae\") " pod="crc-storage/crc-storage-crc-gl6m8" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.744935 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/e776931b-5cd1-4d1f-8e59-70d60f4750ae-node-mnt\") pod \"crc-storage-crc-gl6m8\" (UID: \"e776931b-5cd1-4d1f-8e59-70d60f4750ae\") " pod="crc-storage/crc-storage-crc-gl6m8" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.745246 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/e776931b-5cd1-4d1f-8e59-70d60f4750ae-node-mnt\") pod \"crc-storage-crc-gl6m8\" (UID: \"e776931b-5cd1-4d1f-8e59-70d60f4750ae\") " pod="crc-storage/crc-storage-crc-gl6m8" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.745486 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-cnrw4\" (UniqueName: \"kubernetes.io/projected/e776931b-5cd1-4d1f-8e59-70d60f4750ae-kube-api-access-cnrw4\") pod \"crc-storage-crc-gl6m8\" (UID: \"e776931b-5cd1-4d1f-8e59-70d60f4750ae\") " pod="crc-storage/crc-storage-crc-gl6m8" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.764889 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-cnrw4\" (UniqueName: \"kubernetes.io/projected/e776931b-5cd1-4d1f-8e59-70d60f4750ae-kube-api-access-cnrw4\") pod \"crc-storage-crc-gl6m8\" (UID: \"e776931b-5cd1-4d1f-8e59-70d60f4750ae\") " pod="crc-storage/crc-storage-crc-gl6m8" Dec 03 12:09:40 crc kubenswrapper[5052]: I1203 12:09:40.858390 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="crc-storage/crc-storage-crc-gl6m8" Dec 03 12:09:41 crc kubenswrapper[5052]: I1203 12:09:41.293563 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["crc-storage/crc-storage-crc-gl6m8"] Dec 03 12:09:41 crc kubenswrapper[5052]: I1203 12:09:41.300454 5052 provider.go:102] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Dec 03 12:09:41 crc kubenswrapper[5052]: I1203 12:09:41.391566 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="crc-storage/crc-storage-crc-gl6m8" event={"ID":"e776931b-5cd1-4d1f-8e59-70d60f4750ae","Type":"ContainerStarted","Data":"50043e455bbcaf73da0a12e62c78b51ad2abafdd4862e311a54ed2baaa40977b"} Dec 03 12:09:41 crc kubenswrapper[5052]: I1203 12:09:41.672731 5052 scope.go:117] "RemoveContainer" containerID="63298107b3d090deff9ed92f3d613e5b1c86d9137684d117500257db8ee1c805" Dec 03 12:09:41 crc kubenswrapper[5052]: E1203 12:09:41.673191 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 12:09:41 crc kubenswrapper[5052]: I1203 12:09:41.681656 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cb802b7f-5630-4a20-b23f-7d3f1501f7a8" path="/var/lib/kubelet/pods/cb802b7f-5630-4a20-b23f-7d3f1501f7a8/volumes" Dec 03 12:09:44 crc kubenswrapper[5052]: I1203 12:09:44.428863 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="crc-storage/crc-storage-crc-gl6m8" event={"ID":"e776931b-5cd1-4d1f-8e59-70d60f4750ae","Type":"ContainerStarted","Data":"cb976b53b838bf480cfbd836377b628ef3ca7feb9864a7f13f8fe243b48ea276"} Dec 03 12:09:45 crc kubenswrapper[5052]: I1203 12:09:45.440944 5052 generic.go:334] "Generic (PLEG): container finished" podID="e776931b-5cd1-4d1f-8e59-70d60f4750ae" containerID="cb976b53b838bf480cfbd836377b628ef3ca7feb9864a7f13f8fe243b48ea276" exitCode=0 Dec 03 12:09:45 crc kubenswrapper[5052]: I1203 12:09:45.440999 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="crc-storage/crc-storage-crc-gl6m8" event={"ID":"e776931b-5cd1-4d1f-8e59-70d60f4750ae","Type":"ContainerDied","Data":"cb976b53b838bf480cfbd836377b628ef3ca7feb9864a7f13f8fe243b48ea276"} Dec 03 12:09:46 crc kubenswrapper[5052]: I1203 12:09:46.798411 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="crc-storage/crc-storage-crc-gl6m8" Dec 03 12:09:46 crc kubenswrapper[5052]: I1203 12:09:46.940274 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cnrw4\" (UniqueName: \"kubernetes.io/projected/e776931b-5cd1-4d1f-8e59-70d60f4750ae-kube-api-access-cnrw4\") pod \"e776931b-5cd1-4d1f-8e59-70d60f4750ae\" (UID: \"e776931b-5cd1-4d1f-8e59-70d60f4750ae\") " Dec 03 12:09:46 crc kubenswrapper[5052]: I1203 12:09:46.940496 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/e776931b-5cd1-4d1f-8e59-70d60f4750ae-crc-storage\") pod \"e776931b-5cd1-4d1f-8e59-70d60f4750ae\" (UID: \"e776931b-5cd1-4d1f-8e59-70d60f4750ae\") " Dec 03 12:09:46 crc kubenswrapper[5052]: I1203 12:09:46.940642 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/e776931b-5cd1-4d1f-8e59-70d60f4750ae-node-mnt\") pod \"e776931b-5cd1-4d1f-8e59-70d60f4750ae\" (UID: \"e776931b-5cd1-4d1f-8e59-70d60f4750ae\") " Dec 03 12:09:46 crc kubenswrapper[5052]: I1203 12:09:46.941121 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/e776931b-5cd1-4d1f-8e59-70d60f4750ae-node-mnt" (OuterVolumeSpecName: "node-mnt") pod "e776931b-5cd1-4d1f-8e59-70d60f4750ae" (UID: "e776931b-5cd1-4d1f-8e59-70d60f4750ae"). InnerVolumeSpecName "node-mnt". PluginName "kubernetes.io/host-path", VolumeGidValue "" Dec 03 12:09:46 crc kubenswrapper[5052]: I1203 12:09:46.945715 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e776931b-5cd1-4d1f-8e59-70d60f4750ae-kube-api-access-cnrw4" (OuterVolumeSpecName: "kube-api-access-cnrw4") pod "e776931b-5cd1-4d1f-8e59-70d60f4750ae" (UID: "e776931b-5cd1-4d1f-8e59-70d60f4750ae"). InnerVolumeSpecName "kube-api-access-cnrw4". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:09:46 crc kubenswrapper[5052]: I1203 12:09:46.959317 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e776931b-5cd1-4d1f-8e59-70d60f4750ae-crc-storage" (OuterVolumeSpecName: "crc-storage") pod "e776931b-5cd1-4d1f-8e59-70d60f4750ae" (UID: "e776931b-5cd1-4d1f-8e59-70d60f4750ae"). InnerVolumeSpecName "crc-storage". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:09:47 crc kubenswrapper[5052]: I1203 12:09:47.042975 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-cnrw4\" (UniqueName: \"kubernetes.io/projected/e776931b-5cd1-4d1f-8e59-70d60f4750ae-kube-api-access-cnrw4\") on node \"crc\" DevicePath \"\"" Dec 03 12:09:47 crc kubenswrapper[5052]: I1203 12:09:47.043011 5052 reconciler_common.go:293] "Volume detached for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/e776931b-5cd1-4d1f-8e59-70d60f4750ae-crc-storage\") on node \"crc\" DevicePath \"\"" Dec 03 12:09:47 crc kubenswrapper[5052]: I1203 12:09:47.043023 5052 reconciler_common.go:293] "Volume detached for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/e776931b-5cd1-4d1f-8e59-70d60f4750ae-node-mnt\") on node \"crc\" DevicePath \"\"" Dec 03 12:09:47 crc kubenswrapper[5052]: I1203 12:09:47.462044 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="crc-storage/crc-storage-crc-gl6m8" event={"ID":"e776931b-5cd1-4d1f-8e59-70d60f4750ae","Type":"ContainerDied","Data":"50043e455bbcaf73da0a12e62c78b51ad2abafdd4862e311a54ed2baaa40977b"} Dec 03 12:09:47 crc kubenswrapper[5052]: I1203 12:09:47.462094 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="50043e455bbcaf73da0a12e62c78b51ad2abafdd4862e311a54ed2baaa40977b" Dec 03 12:09:47 crc kubenswrapper[5052]: I1203 12:09:47.462099 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="crc-storage/crc-storage-crc-gl6m8" Dec 03 12:09:50 crc kubenswrapper[5052]: I1203 12:09:50.677766 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["crc-storage/crc-storage-crc-gl6m8"] Dec 03 12:09:50 crc kubenswrapper[5052]: I1203 12:09:50.686071 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["crc-storage/crc-storage-crc-gl6m8"] Dec 03 12:09:50 crc kubenswrapper[5052]: I1203 12:09:50.841552 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["crc-storage/crc-storage-crc-nrblr"] Dec 03 12:09:50 crc kubenswrapper[5052]: E1203 12:09:50.842077 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="e776931b-5cd1-4d1f-8e59-70d60f4750ae" containerName="storage" Dec 03 12:09:50 crc kubenswrapper[5052]: I1203 12:09:50.842104 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="e776931b-5cd1-4d1f-8e59-70d60f4750ae" containerName="storage" Dec 03 12:09:50 crc kubenswrapper[5052]: I1203 12:09:50.842322 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="e776931b-5cd1-4d1f-8e59-70d60f4750ae" containerName="storage" Dec 03 12:09:50 crc kubenswrapper[5052]: I1203 12:09:50.842388 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="70122e67-6175-4a14-a120-6a2731c5dcb1" containerName="mariadb-account-delete" Dec 03 12:09:50 crc kubenswrapper[5052]: I1203 12:09:50.842410 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="aaa3db64-a10b-45ea-9e25-de4916f30617" containerName="mariadb-account-delete" Dec 03 12:09:50 crc kubenswrapper[5052]: I1203 12:09:50.842432 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="89ee1bab-2c68-4fc4-9171-a7330922faf6" containerName="mariadb-account-delete" Dec 03 12:09:50 crc kubenswrapper[5052]: I1203 12:09:50.842446 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="9e6af047-b7ee-459a-b362-da5f777d0fa0" containerName="mariadb-account-delete" Dec 03 12:09:50 crc kubenswrapper[5052]: I1203 12:09:50.843379 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="crc-storage/crc-storage-crc-nrblr" Dec 03 12:09:50 crc kubenswrapper[5052]: I1203 12:09:50.846303 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"crc-storage"/"openshift-service-ca.crt" Dec 03 12:09:50 crc kubenswrapper[5052]: I1203 12:09:50.847323 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"crc-storage"/"crc-storage" Dec 03 12:09:50 crc kubenswrapper[5052]: I1203 12:09:50.847363 5052 reflector.go:368] Caches populated for *v1.Secret from object-"crc-storage"/"crc-storage-dockercfg-mf76g" Dec 03 12:09:50 crc kubenswrapper[5052]: I1203 12:09:50.849471 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"crc-storage"/"kube-root-ca.crt" Dec 03 12:09:50 crc kubenswrapper[5052]: I1203 12:09:50.874217 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["crc-storage/crc-storage-crc-nrblr"] Dec 03 12:09:50 crc kubenswrapper[5052]: I1203 12:09:50.950681 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/6c7ba7e0-b39e-432d-bda2-da1fb27e197a-node-mnt\") pod \"crc-storage-crc-nrblr\" (UID: \"6c7ba7e0-b39e-432d-bda2-da1fb27e197a\") " pod="crc-storage/crc-storage-crc-nrblr" Dec 03 12:09:50 crc kubenswrapper[5052]: I1203 12:09:50.950767 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4szjq\" (UniqueName: \"kubernetes.io/projected/6c7ba7e0-b39e-432d-bda2-da1fb27e197a-kube-api-access-4szjq\") pod \"crc-storage-crc-nrblr\" (UID: \"6c7ba7e0-b39e-432d-bda2-da1fb27e197a\") " pod="crc-storage/crc-storage-crc-nrblr" Dec 03 12:09:50 crc kubenswrapper[5052]: I1203 12:09:50.950846 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/6c7ba7e0-b39e-432d-bda2-da1fb27e197a-crc-storage\") pod \"crc-storage-crc-nrblr\" (UID: \"6c7ba7e0-b39e-432d-bda2-da1fb27e197a\") " pod="crc-storage/crc-storage-crc-nrblr" Dec 03 12:09:51 crc kubenswrapper[5052]: I1203 12:09:51.051878 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/6c7ba7e0-b39e-432d-bda2-da1fb27e197a-crc-storage\") pod \"crc-storage-crc-nrblr\" (UID: \"6c7ba7e0-b39e-432d-bda2-da1fb27e197a\") " pod="crc-storage/crc-storage-crc-nrblr" Dec 03 12:09:51 crc kubenswrapper[5052]: I1203 12:09:51.052021 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/6c7ba7e0-b39e-432d-bda2-da1fb27e197a-node-mnt\") pod \"crc-storage-crc-nrblr\" (UID: \"6c7ba7e0-b39e-432d-bda2-da1fb27e197a\") " pod="crc-storage/crc-storage-crc-nrblr" Dec 03 12:09:51 crc kubenswrapper[5052]: I1203 12:09:51.052049 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-4szjq\" (UniqueName: \"kubernetes.io/projected/6c7ba7e0-b39e-432d-bda2-da1fb27e197a-kube-api-access-4szjq\") pod \"crc-storage-crc-nrblr\" (UID: \"6c7ba7e0-b39e-432d-bda2-da1fb27e197a\") " pod="crc-storage/crc-storage-crc-nrblr" Dec 03 12:09:51 crc kubenswrapper[5052]: I1203 12:09:51.052691 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/6c7ba7e0-b39e-432d-bda2-da1fb27e197a-node-mnt\") pod \"crc-storage-crc-nrblr\" (UID: \"6c7ba7e0-b39e-432d-bda2-da1fb27e197a\") " pod="crc-storage/crc-storage-crc-nrblr" Dec 03 12:09:51 crc kubenswrapper[5052]: I1203 12:09:51.052878 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/6c7ba7e0-b39e-432d-bda2-da1fb27e197a-crc-storage\") pod \"crc-storage-crc-nrblr\" (UID: \"6c7ba7e0-b39e-432d-bda2-da1fb27e197a\") " pod="crc-storage/crc-storage-crc-nrblr" Dec 03 12:09:51 crc kubenswrapper[5052]: I1203 12:09:51.078054 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-4szjq\" (UniqueName: \"kubernetes.io/projected/6c7ba7e0-b39e-432d-bda2-da1fb27e197a-kube-api-access-4szjq\") pod \"crc-storage-crc-nrblr\" (UID: \"6c7ba7e0-b39e-432d-bda2-da1fb27e197a\") " pod="crc-storage/crc-storage-crc-nrblr" Dec 03 12:09:51 crc kubenswrapper[5052]: I1203 12:09:51.176299 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="crc-storage/crc-storage-crc-nrblr" Dec 03 12:09:51 crc kubenswrapper[5052]: I1203 12:09:51.628791 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["crc-storage/crc-storage-crc-nrblr"] Dec 03 12:09:51 crc kubenswrapper[5052]: W1203 12:09:51.636140 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6c7ba7e0_b39e_432d_bda2_da1fb27e197a.slice/crio-51382e3bf26abc734ccda8ef4d97d585280ea61abd2e9ae96c182a06adc21db1 WatchSource:0}: Error finding container 51382e3bf26abc734ccda8ef4d97d585280ea61abd2e9ae96c182a06adc21db1: Status 404 returned error can't find the container with id 51382e3bf26abc734ccda8ef4d97d585280ea61abd2e9ae96c182a06adc21db1 Dec 03 12:09:51 crc kubenswrapper[5052]: I1203 12:09:51.690646 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e776931b-5cd1-4d1f-8e59-70d60f4750ae" path="/var/lib/kubelet/pods/e776931b-5cd1-4d1f-8e59-70d60f4750ae/volumes" Dec 03 12:09:52 crc kubenswrapper[5052]: I1203 12:09:52.513044 5052 generic.go:334] "Generic (PLEG): container finished" podID="6c7ba7e0-b39e-432d-bda2-da1fb27e197a" containerID="5b4ba199f7649c85533a15076c1e1e29d06aba5da6d519402cfa536c066e590b" exitCode=0 Dec 03 12:09:52 crc kubenswrapper[5052]: I1203 12:09:52.513139 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="crc-storage/crc-storage-crc-nrblr" event={"ID":"6c7ba7e0-b39e-432d-bda2-da1fb27e197a","Type":"ContainerDied","Data":"5b4ba199f7649c85533a15076c1e1e29d06aba5da6d519402cfa536c066e590b"} Dec 03 12:09:52 crc kubenswrapper[5052]: I1203 12:09:52.513382 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="crc-storage/crc-storage-crc-nrblr" event={"ID":"6c7ba7e0-b39e-432d-bda2-da1fb27e197a","Type":"ContainerStarted","Data":"51382e3bf26abc734ccda8ef4d97d585280ea61abd2e9ae96c182a06adc21db1"} Dec 03 12:09:53 crc kubenswrapper[5052]: I1203 12:09:53.828683 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="crc-storage/crc-storage-crc-nrblr" Dec 03 12:09:54 crc kubenswrapper[5052]: I1203 12:09:54.004937 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4szjq\" (UniqueName: \"kubernetes.io/projected/6c7ba7e0-b39e-432d-bda2-da1fb27e197a-kube-api-access-4szjq\") pod \"6c7ba7e0-b39e-432d-bda2-da1fb27e197a\" (UID: \"6c7ba7e0-b39e-432d-bda2-da1fb27e197a\") " Dec 03 12:09:54 crc kubenswrapper[5052]: I1203 12:09:54.005120 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/6c7ba7e0-b39e-432d-bda2-da1fb27e197a-crc-storage\") pod \"6c7ba7e0-b39e-432d-bda2-da1fb27e197a\" (UID: \"6c7ba7e0-b39e-432d-bda2-da1fb27e197a\") " Dec 03 12:09:54 crc kubenswrapper[5052]: I1203 12:09:54.005151 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/6c7ba7e0-b39e-432d-bda2-da1fb27e197a-node-mnt\") pod \"6c7ba7e0-b39e-432d-bda2-da1fb27e197a\" (UID: \"6c7ba7e0-b39e-432d-bda2-da1fb27e197a\") " Dec 03 12:09:54 crc kubenswrapper[5052]: I1203 12:09:54.005561 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/6c7ba7e0-b39e-432d-bda2-da1fb27e197a-node-mnt" (OuterVolumeSpecName: "node-mnt") pod "6c7ba7e0-b39e-432d-bda2-da1fb27e197a" (UID: "6c7ba7e0-b39e-432d-bda2-da1fb27e197a"). InnerVolumeSpecName "node-mnt". PluginName "kubernetes.io/host-path", VolumeGidValue "" Dec 03 12:09:54 crc kubenswrapper[5052]: I1203 12:09:54.018802 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6c7ba7e0-b39e-432d-bda2-da1fb27e197a-kube-api-access-4szjq" (OuterVolumeSpecName: "kube-api-access-4szjq") pod "6c7ba7e0-b39e-432d-bda2-da1fb27e197a" (UID: "6c7ba7e0-b39e-432d-bda2-da1fb27e197a"). InnerVolumeSpecName "kube-api-access-4szjq". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:09:54 crc kubenswrapper[5052]: I1203 12:09:54.023681 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6c7ba7e0-b39e-432d-bda2-da1fb27e197a-crc-storage" (OuterVolumeSpecName: "crc-storage") pod "6c7ba7e0-b39e-432d-bda2-da1fb27e197a" (UID: "6c7ba7e0-b39e-432d-bda2-da1fb27e197a"). InnerVolumeSpecName "crc-storage". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:09:54 crc kubenswrapper[5052]: I1203 12:09:54.107832 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-4szjq\" (UniqueName: \"kubernetes.io/projected/6c7ba7e0-b39e-432d-bda2-da1fb27e197a-kube-api-access-4szjq\") on node \"crc\" DevicePath \"\"" Dec 03 12:09:54 crc kubenswrapper[5052]: I1203 12:09:54.107877 5052 reconciler_common.go:293] "Volume detached for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/6c7ba7e0-b39e-432d-bda2-da1fb27e197a-crc-storage\") on node \"crc\" DevicePath \"\"" Dec 03 12:09:54 crc kubenswrapper[5052]: I1203 12:09:54.107887 5052 reconciler_common.go:293] "Volume detached for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/6c7ba7e0-b39e-432d-bda2-da1fb27e197a-node-mnt\") on node \"crc\" DevicePath \"\"" Dec 03 12:09:54 crc kubenswrapper[5052]: I1203 12:09:54.530426 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="crc-storage/crc-storage-crc-nrblr" event={"ID":"6c7ba7e0-b39e-432d-bda2-da1fb27e197a","Type":"ContainerDied","Data":"51382e3bf26abc734ccda8ef4d97d585280ea61abd2e9ae96c182a06adc21db1"} Dec 03 12:09:54 crc kubenswrapper[5052]: I1203 12:09:54.530474 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="51382e3bf26abc734ccda8ef4d97d585280ea61abd2e9ae96c182a06adc21db1" Dec 03 12:09:54 crc kubenswrapper[5052]: I1203 12:09:54.530542 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="crc-storage/crc-storage-crc-nrblr" Dec 03 12:09:55 crc kubenswrapper[5052]: I1203 12:09:55.673301 5052 scope.go:117] "RemoveContainer" containerID="63298107b3d090deff9ed92f3d613e5b1c86d9137684d117500257db8ee1c805" Dec 03 12:09:55 crc kubenswrapper[5052]: E1203 12:09:55.674148 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 12:09:57 crc kubenswrapper[5052]: I1203 12:09:57.637883 5052 scope.go:117] "RemoveContainer" containerID="987e8c8d1466de648f02bf3d76979423cfcca9f6520f4198385ea09c6ae74d6e" Dec 03 12:09:57 crc kubenswrapper[5052]: I1203 12:09:57.674801 5052 scope.go:117] "RemoveContainer" containerID="68e08571183dbfc32af1f09b3eda2efa53f0247f79e3a38eb83e5b3438697496" Dec 03 12:10:06 crc kubenswrapper[5052]: I1203 12:10:06.673249 5052 scope.go:117] "RemoveContainer" containerID="63298107b3d090deff9ed92f3d613e5b1c86d9137684d117500257db8ee1c805" Dec 03 12:10:06 crc kubenswrapper[5052]: E1203 12:10:06.674020 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 12:10:07 crc kubenswrapper[5052]: I1203 12:10:07.017806 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/rabbitmq-server-0"] Dec 03 12:10:07 crc kubenswrapper[5052]: E1203 12:10:07.019277 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="6c7ba7e0-b39e-432d-bda2-da1fb27e197a" containerName="storage" Dec 03 12:10:07 crc kubenswrapper[5052]: I1203 12:10:07.019378 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="6c7ba7e0-b39e-432d-bda2-da1fb27e197a" containerName="storage" Dec 03 12:10:07 crc kubenswrapper[5052]: I1203 12:10:07.019722 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="6c7ba7e0-b39e-432d-bda2-da1fb27e197a" containerName="storage" Dec 03 12:10:07 crc kubenswrapper[5052]: I1203 12:10:07.020777 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:10:07 crc kubenswrapper[5052]: I1203 12:10:07.022934 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"rabbitmq-erlang-cookie" Dec 03 12:10:07 crc kubenswrapper[5052]: I1203 12:10:07.022945 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-rabbitmq-svc" Dec 03 12:10:07 crc kubenswrapper[5052]: I1203 12:10:07.023541 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"rabbitmq-plugins-conf" Dec 03 12:10:07 crc kubenswrapper[5052]: I1203 12:10:07.023568 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"rabbitmq-server-conf" Dec 03 12:10:07 crc kubenswrapper[5052]: I1203 12:10:07.025323 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"rabbitmq-config-data" Dec 03 12:10:07 crc kubenswrapper[5052]: I1203 12:10:07.026690 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"rabbitmq-default-user" Dec 03 12:10:07 crc kubenswrapper[5052]: I1203 12:10:07.030736 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"rabbitmq-server-dockercfg-822hw" Dec 03 12:10:07 crc kubenswrapper[5052]: I1203 12:10:07.036331 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/rabbitmq-server-0"] Dec 03 12:10:07 crc kubenswrapper[5052]: I1203 12:10:07.199958 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage05-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage05-crc\") pod \"rabbitmq-server-0\" (UID: \"3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:10:07 crc kubenswrapper[5052]: I1203 12:10:07.200006 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"rabbitmq-confd\" (UniqueName: \"kubernetes.io/projected/3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f-rabbitmq-confd\") pod \"rabbitmq-server-0\" (UID: \"3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:10:07 crc kubenswrapper[5052]: I1203 12:10:07.200046 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"rabbitmq-erlang-cookie\" (UniqueName: \"kubernetes.io/empty-dir/3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f-rabbitmq-erlang-cookie\") pod \"rabbitmq-server-0\" (UID: \"3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:10:07 crc kubenswrapper[5052]: I1203 12:10:07.200064 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"rabbitmq-plugins\" (UniqueName: \"kubernetes.io/empty-dir/3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f-rabbitmq-plugins\") pod \"rabbitmq-server-0\" (UID: \"3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:10:07 crc kubenswrapper[5052]: I1203 12:10:07.200080 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"rabbitmq-tls\" (UniqueName: \"kubernetes.io/projected/3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f-rabbitmq-tls\") pod \"rabbitmq-server-0\" (UID: \"3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:10:07 crc kubenswrapper[5052]: I1203 12:10:07.200170 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f-config-data\") pod \"rabbitmq-server-0\" (UID: \"3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:10:07 crc kubenswrapper[5052]: I1203 12:10:07.200226 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-conf\" (UniqueName: \"kubernetes.io/configmap/3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f-plugins-conf\") pod \"rabbitmq-server-0\" (UID: \"3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:10:07 crc kubenswrapper[5052]: I1203 12:10:07.200247 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"pod-info\" (UniqueName: \"kubernetes.io/downward-api/3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f-pod-info\") pod \"rabbitmq-server-0\" (UID: \"3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:10:07 crc kubenswrapper[5052]: I1203 12:10:07.200264 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"server-conf\" (UniqueName: \"kubernetes.io/configmap/3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f-server-conf\") pod \"rabbitmq-server-0\" (UID: \"3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:10:07 crc kubenswrapper[5052]: I1203 12:10:07.200297 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"erlang-cookie-secret\" (UniqueName: \"kubernetes.io/secret/3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f-erlang-cookie-secret\") pod \"rabbitmq-server-0\" (UID: \"3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:10:07 crc kubenswrapper[5052]: I1203 12:10:07.200323 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fm9bz\" (UniqueName: \"kubernetes.io/projected/3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f-kube-api-access-fm9bz\") pod \"rabbitmq-server-0\" (UID: \"3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:10:07 crc kubenswrapper[5052]: I1203 12:10:07.301362 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"plugins-conf\" (UniqueName: \"kubernetes.io/configmap/3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f-plugins-conf\") pod \"rabbitmq-server-0\" (UID: \"3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:10:07 crc kubenswrapper[5052]: I1203 12:10:07.301413 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"pod-info\" (UniqueName: \"kubernetes.io/downward-api/3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f-pod-info\") pod \"rabbitmq-server-0\" (UID: \"3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:10:07 crc kubenswrapper[5052]: I1203 12:10:07.301446 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"server-conf\" (UniqueName: \"kubernetes.io/configmap/3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f-server-conf\") pod \"rabbitmq-server-0\" (UID: \"3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:10:07 crc kubenswrapper[5052]: I1203 12:10:07.301498 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"erlang-cookie-secret\" (UniqueName: \"kubernetes.io/secret/3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f-erlang-cookie-secret\") pod \"rabbitmq-server-0\" (UID: \"3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:10:07 crc kubenswrapper[5052]: I1203 12:10:07.301558 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-fm9bz\" (UniqueName: \"kubernetes.io/projected/3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f-kube-api-access-fm9bz\") pod \"rabbitmq-server-0\" (UID: \"3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:10:07 crc kubenswrapper[5052]: I1203 12:10:07.301599 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage05-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage05-crc\") pod \"rabbitmq-server-0\" (UID: \"3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:10:07 crc kubenswrapper[5052]: I1203 12:10:07.301622 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"rabbitmq-confd\" (UniqueName: \"kubernetes.io/projected/3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f-rabbitmq-confd\") pod \"rabbitmq-server-0\" (UID: \"3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:10:07 crc kubenswrapper[5052]: I1203 12:10:07.301665 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"rabbitmq-erlang-cookie\" (UniqueName: \"kubernetes.io/empty-dir/3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f-rabbitmq-erlang-cookie\") pod \"rabbitmq-server-0\" (UID: \"3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:10:07 crc kubenswrapper[5052]: I1203 12:10:07.301692 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"rabbitmq-plugins\" (UniqueName: \"kubernetes.io/empty-dir/3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f-rabbitmq-plugins\") pod \"rabbitmq-server-0\" (UID: \"3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:10:07 crc kubenswrapper[5052]: I1203 12:10:07.301716 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"rabbitmq-tls\" (UniqueName: \"kubernetes.io/projected/3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f-rabbitmq-tls\") pod \"rabbitmq-server-0\" (UID: \"3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:10:07 crc kubenswrapper[5052]: I1203 12:10:07.301749 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f-config-data\") pod \"rabbitmq-server-0\" (UID: \"3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:10:07 crc kubenswrapper[5052]: I1203 12:10:07.302074 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage05-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage05-crc\") pod \"rabbitmq-server-0\" (UID: \"3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f\") device mount path \"/mnt/openstack/pv05\"" pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:10:07 crc kubenswrapper[5052]: I1203 12:10:07.302607 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"rabbitmq-plugins\" (UniqueName: \"kubernetes.io/empty-dir/3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f-rabbitmq-plugins\") pod \"rabbitmq-server-0\" (UID: \"3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:10:07 crc kubenswrapper[5052]: I1203 12:10:07.302665 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"plugins-conf\" (UniqueName: \"kubernetes.io/configmap/3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f-plugins-conf\") pod \"rabbitmq-server-0\" (UID: \"3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:10:07 crc kubenswrapper[5052]: I1203 12:10:07.302726 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"rabbitmq-erlang-cookie\" (UniqueName: \"kubernetes.io/empty-dir/3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f-rabbitmq-erlang-cookie\") pod \"rabbitmq-server-0\" (UID: \"3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:10:07 crc kubenswrapper[5052]: I1203 12:10:07.302870 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f-config-data\") pod \"rabbitmq-server-0\" (UID: \"3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:10:07 crc kubenswrapper[5052]: I1203 12:10:07.303144 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"server-conf\" (UniqueName: \"kubernetes.io/configmap/3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f-server-conf\") pod \"rabbitmq-server-0\" (UID: \"3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:10:07 crc kubenswrapper[5052]: I1203 12:10:07.309179 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"rabbitmq-confd\" (UniqueName: \"kubernetes.io/projected/3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f-rabbitmq-confd\") pod \"rabbitmq-server-0\" (UID: \"3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:10:07 crc kubenswrapper[5052]: I1203 12:10:07.309540 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"rabbitmq-tls\" (UniqueName: \"kubernetes.io/projected/3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f-rabbitmq-tls\") pod \"rabbitmq-server-0\" (UID: \"3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:10:07 crc kubenswrapper[5052]: I1203 12:10:07.311843 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"erlang-cookie-secret\" (UniqueName: \"kubernetes.io/secret/3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f-erlang-cookie-secret\") pod \"rabbitmq-server-0\" (UID: \"3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:10:07 crc kubenswrapper[5052]: I1203 12:10:07.311885 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"pod-info\" (UniqueName: \"kubernetes.io/downward-api/3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f-pod-info\") pod \"rabbitmq-server-0\" (UID: \"3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:10:07 crc kubenswrapper[5052]: I1203 12:10:07.321072 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-fm9bz\" (UniqueName: \"kubernetes.io/projected/3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f-kube-api-access-fm9bz\") pod \"rabbitmq-server-0\" (UID: \"3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:10:07 crc kubenswrapper[5052]: I1203 12:10:07.327990 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage05-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage05-crc\") pod \"rabbitmq-server-0\" (UID: \"3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:10:07 crc kubenswrapper[5052]: I1203 12:10:07.341702 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/rabbitmq-cell1-server-0"] Dec 03 12:10:07 crc kubenswrapper[5052]: I1203 12:10:07.343736 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:10:07 crc kubenswrapper[5052]: I1203 12:10:07.345821 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"rabbitmq-cell1-default-user" Dec 03 12:10:07 crc kubenswrapper[5052]: I1203 12:10:07.346127 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"rabbitmq-cell1-erlang-cookie" Dec 03 12:10:07 crc kubenswrapper[5052]: I1203 12:10:07.346286 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"rabbitmq-cell1-server-conf" Dec 03 12:10:07 crc kubenswrapper[5052]: I1203 12:10:07.346570 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"rabbitmq-cell1-server-dockercfg-bjl9r" Dec 03 12:10:07 crc kubenswrapper[5052]: I1203 12:10:07.346759 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"rabbitmq-cell1-plugins-conf" Dec 03 12:10:07 crc kubenswrapper[5052]: I1203 12:10:07.347021 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"rabbitmq-cell1-config-data" Dec 03 12:10:07 crc kubenswrapper[5052]: I1203 12:10:07.347435 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:10:07 crc kubenswrapper[5052]: I1203 12:10:07.348225 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-rabbitmq-cell1-svc" Dec 03 12:10:07 crc kubenswrapper[5052]: I1203 12:10:07.357411 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/rabbitmq-cell1-server-0"] Dec 03 12:10:07 crc kubenswrapper[5052]: I1203 12:10:07.504958 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"rabbitmq-confd\" (UniqueName: \"kubernetes.io/projected/e63eb1eb-e0b9-46ea-a19b-8f5c018728ff-rabbitmq-confd\") pod \"rabbitmq-cell1-server-0\" (UID: \"e63eb1eb-e0b9-46ea-a19b-8f5c018728ff\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:10:07 crc kubenswrapper[5052]: I1203 12:10:07.505315 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"rabbitmq-erlang-cookie\" (UniqueName: \"kubernetes.io/empty-dir/e63eb1eb-e0b9-46ea-a19b-8f5c018728ff-rabbitmq-erlang-cookie\") pod \"rabbitmq-cell1-server-0\" (UID: \"e63eb1eb-e0b9-46ea-a19b-8f5c018728ff\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:10:07 crc kubenswrapper[5052]: I1203 12:10:07.505374 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"server-conf\" (UniqueName: \"kubernetes.io/configmap/e63eb1eb-e0b9-46ea-a19b-8f5c018728ff-server-conf\") pod \"rabbitmq-cell1-server-0\" (UID: \"e63eb1eb-e0b9-46ea-a19b-8f5c018728ff\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:10:07 crc kubenswrapper[5052]: I1203 12:10:07.505398 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zlvwl\" (UniqueName: \"kubernetes.io/projected/e63eb1eb-e0b9-46ea-a19b-8f5c018728ff-kube-api-access-zlvwl\") pod \"rabbitmq-cell1-server-0\" (UID: \"e63eb1eb-e0b9-46ea-a19b-8f5c018728ff\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:10:07 crc kubenswrapper[5052]: I1203 12:10:07.505594 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"rabbitmq-plugins\" (UniqueName: \"kubernetes.io/empty-dir/e63eb1eb-e0b9-46ea-a19b-8f5c018728ff-rabbitmq-plugins\") pod \"rabbitmq-cell1-server-0\" (UID: \"e63eb1eb-e0b9-46ea-a19b-8f5c018728ff\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:10:07 crc kubenswrapper[5052]: I1203 12:10:07.505649 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage04-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage04-crc\") pod \"rabbitmq-cell1-server-0\" (UID: \"e63eb1eb-e0b9-46ea-a19b-8f5c018728ff\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:10:07 crc kubenswrapper[5052]: I1203 12:10:07.505694 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/e63eb1eb-e0b9-46ea-a19b-8f5c018728ff-config-data\") pod \"rabbitmq-cell1-server-0\" (UID: \"e63eb1eb-e0b9-46ea-a19b-8f5c018728ff\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:10:07 crc kubenswrapper[5052]: I1203 12:10:07.505717 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"pod-info\" (UniqueName: \"kubernetes.io/downward-api/e63eb1eb-e0b9-46ea-a19b-8f5c018728ff-pod-info\") pod \"rabbitmq-cell1-server-0\" (UID: \"e63eb1eb-e0b9-46ea-a19b-8f5c018728ff\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:10:07 crc kubenswrapper[5052]: I1203 12:10:07.505758 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"erlang-cookie-secret\" (UniqueName: \"kubernetes.io/secret/e63eb1eb-e0b9-46ea-a19b-8f5c018728ff-erlang-cookie-secret\") pod \"rabbitmq-cell1-server-0\" (UID: \"e63eb1eb-e0b9-46ea-a19b-8f5c018728ff\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:10:07 crc kubenswrapper[5052]: I1203 12:10:07.505860 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"rabbitmq-tls\" (UniqueName: \"kubernetes.io/projected/e63eb1eb-e0b9-46ea-a19b-8f5c018728ff-rabbitmq-tls\") pod \"rabbitmq-cell1-server-0\" (UID: \"e63eb1eb-e0b9-46ea-a19b-8f5c018728ff\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:10:07 crc kubenswrapper[5052]: I1203 12:10:07.505907 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-conf\" (UniqueName: \"kubernetes.io/configmap/e63eb1eb-e0b9-46ea-a19b-8f5c018728ff-plugins-conf\") pod \"rabbitmq-cell1-server-0\" (UID: \"e63eb1eb-e0b9-46ea-a19b-8f5c018728ff\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:10:07 crc kubenswrapper[5052]: I1203 12:10:07.606756 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"rabbitmq-tls\" (UniqueName: \"kubernetes.io/projected/e63eb1eb-e0b9-46ea-a19b-8f5c018728ff-rabbitmq-tls\") pod \"rabbitmq-cell1-server-0\" (UID: \"e63eb1eb-e0b9-46ea-a19b-8f5c018728ff\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:10:07 crc kubenswrapper[5052]: I1203 12:10:07.606815 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"plugins-conf\" (UniqueName: \"kubernetes.io/configmap/e63eb1eb-e0b9-46ea-a19b-8f5c018728ff-plugins-conf\") pod \"rabbitmq-cell1-server-0\" (UID: \"e63eb1eb-e0b9-46ea-a19b-8f5c018728ff\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:10:07 crc kubenswrapper[5052]: I1203 12:10:07.606868 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"rabbitmq-confd\" (UniqueName: \"kubernetes.io/projected/e63eb1eb-e0b9-46ea-a19b-8f5c018728ff-rabbitmq-confd\") pod \"rabbitmq-cell1-server-0\" (UID: \"e63eb1eb-e0b9-46ea-a19b-8f5c018728ff\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:10:07 crc kubenswrapper[5052]: I1203 12:10:07.606895 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"rabbitmq-erlang-cookie\" (UniqueName: \"kubernetes.io/empty-dir/e63eb1eb-e0b9-46ea-a19b-8f5c018728ff-rabbitmq-erlang-cookie\") pod \"rabbitmq-cell1-server-0\" (UID: \"e63eb1eb-e0b9-46ea-a19b-8f5c018728ff\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:10:07 crc kubenswrapper[5052]: I1203 12:10:07.606931 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"server-conf\" (UniqueName: \"kubernetes.io/configmap/e63eb1eb-e0b9-46ea-a19b-8f5c018728ff-server-conf\") pod \"rabbitmq-cell1-server-0\" (UID: \"e63eb1eb-e0b9-46ea-a19b-8f5c018728ff\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:10:07 crc kubenswrapper[5052]: I1203 12:10:07.606954 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-zlvwl\" (UniqueName: \"kubernetes.io/projected/e63eb1eb-e0b9-46ea-a19b-8f5c018728ff-kube-api-access-zlvwl\") pod \"rabbitmq-cell1-server-0\" (UID: \"e63eb1eb-e0b9-46ea-a19b-8f5c018728ff\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:10:07 crc kubenswrapper[5052]: I1203 12:10:07.606986 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"rabbitmq-plugins\" (UniqueName: \"kubernetes.io/empty-dir/e63eb1eb-e0b9-46ea-a19b-8f5c018728ff-rabbitmq-plugins\") pod \"rabbitmq-cell1-server-0\" (UID: \"e63eb1eb-e0b9-46ea-a19b-8f5c018728ff\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:10:07 crc kubenswrapper[5052]: I1203 12:10:07.607003 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage04-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage04-crc\") pod \"rabbitmq-cell1-server-0\" (UID: \"e63eb1eb-e0b9-46ea-a19b-8f5c018728ff\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:10:07 crc kubenswrapper[5052]: I1203 12:10:07.607021 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/e63eb1eb-e0b9-46ea-a19b-8f5c018728ff-config-data\") pod \"rabbitmq-cell1-server-0\" (UID: \"e63eb1eb-e0b9-46ea-a19b-8f5c018728ff\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:10:07 crc kubenswrapper[5052]: I1203 12:10:07.607036 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"pod-info\" (UniqueName: \"kubernetes.io/downward-api/e63eb1eb-e0b9-46ea-a19b-8f5c018728ff-pod-info\") pod \"rabbitmq-cell1-server-0\" (UID: \"e63eb1eb-e0b9-46ea-a19b-8f5c018728ff\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:10:07 crc kubenswrapper[5052]: I1203 12:10:07.607053 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"erlang-cookie-secret\" (UniqueName: \"kubernetes.io/secret/e63eb1eb-e0b9-46ea-a19b-8f5c018728ff-erlang-cookie-secret\") pod \"rabbitmq-cell1-server-0\" (UID: \"e63eb1eb-e0b9-46ea-a19b-8f5c018728ff\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:10:07 crc kubenswrapper[5052]: I1203 12:10:07.607711 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage04-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage04-crc\") pod \"rabbitmq-cell1-server-0\" (UID: \"e63eb1eb-e0b9-46ea-a19b-8f5c018728ff\") device mount path \"/mnt/openstack/pv04\"" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:10:07 crc kubenswrapper[5052]: I1203 12:10:07.608038 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"rabbitmq-plugins\" (UniqueName: \"kubernetes.io/empty-dir/e63eb1eb-e0b9-46ea-a19b-8f5c018728ff-rabbitmq-plugins\") pod \"rabbitmq-cell1-server-0\" (UID: \"e63eb1eb-e0b9-46ea-a19b-8f5c018728ff\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:10:07 crc kubenswrapper[5052]: I1203 12:10:07.609308 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"rabbitmq-erlang-cookie\" (UniqueName: \"kubernetes.io/empty-dir/e63eb1eb-e0b9-46ea-a19b-8f5c018728ff-rabbitmq-erlang-cookie\") pod \"rabbitmq-cell1-server-0\" (UID: \"e63eb1eb-e0b9-46ea-a19b-8f5c018728ff\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:10:07 crc kubenswrapper[5052]: I1203 12:10:07.609543 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/e63eb1eb-e0b9-46ea-a19b-8f5c018728ff-config-data\") pod \"rabbitmq-cell1-server-0\" (UID: \"e63eb1eb-e0b9-46ea-a19b-8f5c018728ff\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:10:07 crc kubenswrapper[5052]: I1203 12:10:07.609628 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"plugins-conf\" (UniqueName: \"kubernetes.io/configmap/e63eb1eb-e0b9-46ea-a19b-8f5c018728ff-plugins-conf\") pod \"rabbitmq-cell1-server-0\" (UID: \"e63eb1eb-e0b9-46ea-a19b-8f5c018728ff\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:10:07 crc kubenswrapper[5052]: I1203 12:10:07.609885 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"server-conf\" (UniqueName: \"kubernetes.io/configmap/e63eb1eb-e0b9-46ea-a19b-8f5c018728ff-server-conf\") pod \"rabbitmq-cell1-server-0\" (UID: \"e63eb1eb-e0b9-46ea-a19b-8f5c018728ff\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:10:07 crc kubenswrapper[5052]: I1203 12:10:07.616017 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"pod-info\" (UniqueName: \"kubernetes.io/downward-api/e63eb1eb-e0b9-46ea-a19b-8f5c018728ff-pod-info\") pod \"rabbitmq-cell1-server-0\" (UID: \"e63eb1eb-e0b9-46ea-a19b-8f5c018728ff\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:10:07 crc kubenswrapper[5052]: I1203 12:10:07.616648 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"erlang-cookie-secret\" (UniqueName: \"kubernetes.io/secret/e63eb1eb-e0b9-46ea-a19b-8f5c018728ff-erlang-cookie-secret\") pod \"rabbitmq-cell1-server-0\" (UID: \"e63eb1eb-e0b9-46ea-a19b-8f5c018728ff\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:10:07 crc kubenswrapper[5052]: I1203 12:10:07.624119 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"rabbitmq-tls\" (UniqueName: \"kubernetes.io/projected/e63eb1eb-e0b9-46ea-a19b-8f5c018728ff-rabbitmq-tls\") pod \"rabbitmq-cell1-server-0\" (UID: \"e63eb1eb-e0b9-46ea-a19b-8f5c018728ff\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:10:07 crc kubenswrapper[5052]: I1203 12:10:07.628326 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-zlvwl\" (UniqueName: \"kubernetes.io/projected/e63eb1eb-e0b9-46ea-a19b-8f5c018728ff-kube-api-access-zlvwl\") pod \"rabbitmq-cell1-server-0\" (UID: \"e63eb1eb-e0b9-46ea-a19b-8f5c018728ff\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:10:07 crc kubenswrapper[5052]: I1203 12:10:07.637576 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"rabbitmq-confd\" (UniqueName: \"kubernetes.io/projected/e63eb1eb-e0b9-46ea-a19b-8f5c018728ff-rabbitmq-confd\") pod \"rabbitmq-cell1-server-0\" (UID: \"e63eb1eb-e0b9-46ea-a19b-8f5c018728ff\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:10:07 crc kubenswrapper[5052]: I1203 12:10:07.680482 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage04-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage04-crc\") pod \"rabbitmq-cell1-server-0\" (UID: \"e63eb1eb-e0b9-46ea-a19b-8f5c018728ff\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:10:07 crc kubenswrapper[5052]: I1203 12:10:07.765362 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:10:07 crc kubenswrapper[5052]: I1203 12:10:07.808124 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/rabbitmq-server-0"] Dec 03 12:10:08 crc kubenswrapper[5052]: I1203 12:10:08.017341 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/rabbitmq-cell1-server-0"] Dec 03 12:10:08 crc kubenswrapper[5052]: W1203 12:10:08.024717 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode63eb1eb_e0b9_46ea_a19b_8f5c018728ff.slice/crio-3fe470592acd81e517aa2c5a619c15d20df1c0a04a8a73f91848113c03fe7642 WatchSource:0}: Error finding container 3fe470592acd81e517aa2c5a619c15d20df1c0a04a8a73f91848113c03fe7642: Status 404 returned error can't find the container with id 3fe470592acd81e517aa2c5a619c15d20df1c0a04a8a73f91848113c03fe7642 Dec 03 12:10:08 crc kubenswrapper[5052]: I1203 12:10:08.664817 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" event={"ID":"e63eb1eb-e0b9-46ea-a19b-8f5c018728ff","Type":"ContainerStarted","Data":"3fe470592acd81e517aa2c5a619c15d20df1c0a04a8a73f91848113c03fe7642"} Dec 03 12:10:08 crc kubenswrapper[5052]: I1203 12:10:08.666255 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-server-0" event={"ID":"3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f","Type":"ContainerStarted","Data":"d41f4f8ea80d58bed016506bfda8d5de94ceed5986cd6ec7e2e81583625c5c68"} Dec 03 12:10:08 crc kubenswrapper[5052]: I1203 12:10:08.855752 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/openstack-galera-0"] Dec 03 12:10:08 crc kubenswrapper[5052]: I1203 12:10:08.857356 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:10:08 crc kubenswrapper[5052]: I1203 12:10:08.860448 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-galera-openstack-svc" Dec 03 12:10:08 crc kubenswrapper[5052]: I1203 12:10:08.860682 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"openstack-config-data" Dec 03 12:10:08 crc kubenswrapper[5052]: I1203 12:10:08.861033 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"openstack-scripts" Dec 03 12:10:08 crc kubenswrapper[5052]: I1203 12:10:08.861037 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"galera-openstack-dockercfg-vpq7c" Dec 03 12:10:08 crc kubenswrapper[5052]: I1203 12:10:08.866812 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"combined-ca-bundle" Dec 03 12:10:08 crc kubenswrapper[5052]: I1203 12:10:08.878678 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/openstack-galera-0"] Dec 03 12:10:09 crc kubenswrapper[5052]: I1203 12:10:09.044758 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/85f5676f-32e8-4c3d-8729-e1eb2fdc8dc9-config-data-default\") pod \"openstack-galera-0\" (UID: \"85f5676f-32e8-4c3d-8729-e1eb2fdc8dc9\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:10:09 crc kubenswrapper[5052]: I1203 12:10:09.044801 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/85f5676f-32e8-4c3d-8729-e1eb2fdc8dc9-config-data-generated\") pod \"openstack-galera-0\" (UID: \"85f5676f-32e8-4c3d-8729-e1eb2fdc8dc9\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:10:09 crc kubenswrapper[5052]: I1203 12:10:09.044834 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/85f5676f-32e8-4c3d-8729-e1eb2fdc8dc9-galera-tls-certs\") pod \"openstack-galera-0\" (UID: \"85f5676f-32e8-4c3d-8729-e1eb2fdc8dc9\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:10:09 crc kubenswrapper[5052]: I1203 12:10:09.044927 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/85f5676f-32e8-4c3d-8729-e1eb2fdc8dc9-kolla-config\") pod \"openstack-galera-0\" (UID: \"85f5676f-32e8-4c3d-8729-e1eb2fdc8dc9\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:10:09 crc kubenswrapper[5052]: I1203 12:10:09.045077 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage16-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage16-crc\") pod \"openstack-galera-0\" (UID: \"85f5676f-32e8-4c3d-8729-e1eb2fdc8dc9\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:10:09 crc kubenswrapper[5052]: I1203 12:10:09.045168 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/85f5676f-32e8-4c3d-8729-e1eb2fdc8dc9-operator-scripts\") pod \"openstack-galera-0\" (UID: \"85f5676f-32e8-4c3d-8729-e1eb2fdc8dc9\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:10:09 crc kubenswrapper[5052]: I1203 12:10:09.045369 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rbmfn\" (UniqueName: \"kubernetes.io/projected/85f5676f-32e8-4c3d-8729-e1eb2fdc8dc9-kube-api-access-rbmfn\") pod \"openstack-galera-0\" (UID: \"85f5676f-32e8-4c3d-8729-e1eb2fdc8dc9\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:10:09 crc kubenswrapper[5052]: I1203 12:10:09.045407 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/85f5676f-32e8-4c3d-8729-e1eb2fdc8dc9-combined-ca-bundle\") pod \"openstack-galera-0\" (UID: \"85f5676f-32e8-4c3d-8729-e1eb2fdc8dc9\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:10:09 crc kubenswrapper[5052]: I1203 12:10:09.147125 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-rbmfn\" (UniqueName: \"kubernetes.io/projected/85f5676f-32e8-4c3d-8729-e1eb2fdc8dc9-kube-api-access-rbmfn\") pod \"openstack-galera-0\" (UID: \"85f5676f-32e8-4c3d-8729-e1eb2fdc8dc9\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:10:09 crc kubenswrapper[5052]: I1203 12:10:09.147194 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/85f5676f-32e8-4c3d-8729-e1eb2fdc8dc9-combined-ca-bundle\") pod \"openstack-galera-0\" (UID: \"85f5676f-32e8-4c3d-8729-e1eb2fdc8dc9\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:10:09 crc kubenswrapper[5052]: I1203 12:10:09.147387 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/85f5676f-32e8-4c3d-8729-e1eb2fdc8dc9-config-data-default\") pod \"openstack-galera-0\" (UID: \"85f5676f-32e8-4c3d-8729-e1eb2fdc8dc9\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:10:09 crc kubenswrapper[5052]: I1203 12:10:09.147432 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/85f5676f-32e8-4c3d-8729-e1eb2fdc8dc9-config-data-generated\") pod \"openstack-galera-0\" (UID: \"85f5676f-32e8-4c3d-8729-e1eb2fdc8dc9\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:10:09 crc kubenswrapper[5052]: I1203 12:10:09.147475 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/85f5676f-32e8-4c3d-8729-e1eb2fdc8dc9-galera-tls-certs\") pod \"openstack-galera-0\" (UID: \"85f5676f-32e8-4c3d-8729-e1eb2fdc8dc9\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:10:09 crc kubenswrapper[5052]: I1203 12:10:09.147523 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/85f5676f-32e8-4c3d-8729-e1eb2fdc8dc9-kolla-config\") pod \"openstack-galera-0\" (UID: \"85f5676f-32e8-4c3d-8729-e1eb2fdc8dc9\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:10:09 crc kubenswrapper[5052]: I1203 12:10:09.147557 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage16-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage16-crc\") pod \"openstack-galera-0\" (UID: \"85f5676f-32e8-4c3d-8729-e1eb2fdc8dc9\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:10:09 crc kubenswrapper[5052]: I1203 12:10:09.147589 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/85f5676f-32e8-4c3d-8729-e1eb2fdc8dc9-operator-scripts\") pod \"openstack-galera-0\" (UID: \"85f5676f-32e8-4c3d-8729-e1eb2fdc8dc9\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:10:09 crc kubenswrapper[5052]: I1203 12:10:09.148049 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage16-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage16-crc\") pod \"openstack-galera-0\" (UID: \"85f5676f-32e8-4c3d-8729-e1eb2fdc8dc9\") device mount path \"/mnt/openstack/pv16\"" pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:10:09 crc kubenswrapper[5052]: I1203 12:10:09.148800 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/85f5676f-32e8-4c3d-8729-e1eb2fdc8dc9-config-data-generated\") pod \"openstack-galera-0\" (UID: \"85f5676f-32e8-4c3d-8729-e1eb2fdc8dc9\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:10:09 crc kubenswrapper[5052]: I1203 12:10:09.149092 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/85f5676f-32e8-4c3d-8729-e1eb2fdc8dc9-kolla-config\") pod \"openstack-galera-0\" (UID: \"85f5676f-32e8-4c3d-8729-e1eb2fdc8dc9\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:10:09 crc kubenswrapper[5052]: I1203 12:10:09.149912 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/85f5676f-32e8-4c3d-8729-e1eb2fdc8dc9-operator-scripts\") pod \"openstack-galera-0\" (UID: \"85f5676f-32e8-4c3d-8729-e1eb2fdc8dc9\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:10:09 crc kubenswrapper[5052]: I1203 12:10:09.150014 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/85f5676f-32e8-4c3d-8729-e1eb2fdc8dc9-config-data-default\") pod \"openstack-galera-0\" (UID: \"85f5676f-32e8-4c3d-8729-e1eb2fdc8dc9\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:10:09 crc kubenswrapper[5052]: I1203 12:10:09.159768 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/85f5676f-32e8-4c3d-8729-e1eb2fdc8dc9-galera-tls-certs\") pod \"openstack-galera-0\" (UID: \"85f5676f-32e8-4c3d-8729-e1eb2fdc8dc9\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:10:09 crc kubenswrapper[5052]: I1203 12:10:09.172038 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-rbmfn\" (UniqueName: \"kubernetes.io/projected/85f5676f-32e8-4c3d-8729-e1eb2fdc8dc9-kube-api-access-rbmfn\") pod \"openstack-galera-0\" (UID: \"85f5676f-32e8-4c3d-8729-e1eb2fdc8dc9\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:10:09 crc kubenswrapper[5052]: I1203 12:10:09.173566 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/85f5676f-32e8-4c3d-8729-e1eb2fdc8dc9-combined-ca-bundle\") pod \"openstack-galera-0\" (UID: \"85f5676f-32e8-4c3d-8729-e1eb2fdc8dc9\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:10:09 crc kubenswrapper[5052]: I1203 12:10:09.174585 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage16-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage16-crc\") pod \"openstack-galera-0\" (UID: \"85f5676f-32e8-4c3d-8729-e1eb2fdc8dc9\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:10:09 crc kubenswrapper[5052]: I1203 12:10:09.257210 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:10:09 crc kubenswrapper[5052]: I1203 12:10:09.686478 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" event={"ID":"e63eb1eb-e0b9-46ea-a19b-8f5c018728ff","Type":"ContainerStarted","Data":"e2d7edf07d19fee0525d0d93f60f9850672844f551c1265e9b21fb9528510d56"} Dec 03 12:10:09 crc kubenswrapper[5052]: I1203 12:10:09.687132 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-server-0" event={"ID":"3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f","Type":"ContainerStarted","Data":"a60401c5cce1c04ba232d9ed1392bdc059663fb8d3f36429c2b1a642a6fc8fdf"} Dec 03 12:10:09 crc kubenswrapper[5052]: I1203 12:10:09.744429 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/openstack-galera-0"] Dec 03 12:10:10 crc kubenswrapper[5052]: I1203 12:10:10.169408 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/openstack-cell1-galera-0"] Dec 03 12:10:10 crc kubenswrapper[5052]: I1203 12:10:10.189143 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:10:10 crc kubenswrapper[5052]: I1203 12:10:10.191856 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-galera-openstack-cell1-svc" Dec 03 12:10:10 crc kubenswrapper[5052]: I1203 12:10:10.192376 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"galera-openstack-cell1-dockercfg-slzm6" Dec 03 12:10:10 crc kubenswrapper[5052]: I1203 12:10:10.192916 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"openstack-cell1-config-data" Dec 03 12:10:10 crc kubenswrapper[5052]: I1203 12:10:10.193242 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"openstack-cell1-scripts" Dec 03 12:10:10 crc kubenswrapper[5052]: I1203 12:10:10.197632 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/openstack-cell1-galera-0"] Dec 03 12:10:10 crc kubenswrapper[5052]: I1203 12:10:10.267587 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/00a9935f-ef2d-4976-97a0-78ee6325433d-kolla-config\") pod \"openstack-cell1-galera-0\" (UID: \"00a9935f-ef2d-4976-97a0-78ee6325433d\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:10:10 crc kubenswrapper[5052]: I1203 12:10:10.267960 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/00a9935f-ef2d-4976-97a0-78ee6325433d-galera-tls-certs\") pod \"openstack-cell1-galera-0\" (UID: \"00a9935f-ef2d-4976-97a0-78ee6325433d\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:10:10 crc kubenswrapper[5052]: I1203 12:10:10.268086 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/00a9935f-ef2d-4976-97a0-78ee6325433d-config-data-default\") pod \"openstack-cell1-galera-0\" (UID: \"00a9935f-ef2d-4976-97a0-78ee6325433d\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:10:10 crc kubenswrapper[5052]: I1203 12:10:10.268224 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/00a9935f-ef2d-4976-97a0-78ee6325433d-operator-scripts\") pod \"openstack-cell1-galera-0\" (UID: \"00a9935f-ef2d-4976-97a0-78ee6325433d\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:10:10 crc kubenswrapper[5052]: I1203 12:10:10.268340 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage06-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage06-crc\") pod \"openstack-cell1-galera-0\" (UID: \"00a9935f-ef2d-4976-97a0-78ee6325433d\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:10:10 crc kubenswrapper[5052]: I1203 12:10:10.268437 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gd9zs\" (UniqueName: \"kubernetes.io/projected/00a9935f-ef2d-4976-97a0-78ee6325433d-kube-api-access-gd9zs\") pod \"openstack-cell1-galera-0\" (UID: \"00a9935f-ef2d-4976-97a0-78ee6325433d\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:10:10 crc kubenswrapper[5052]: I1203 12:10:10.268612 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/00a9935f-ef2d-4976-97a0-78ee6325433d-config-data-generated\") pod \"openstack-cell1-galera-0\" (UID: \"00a9935f-ef2d-4976-97a0-78ee6325433d\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:10:10 crc kubenswrapper[5052]: I1203 12:10:10.268730 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/00a9935f-ef2d-4976-97a0-78ee6325433d-combined-ca-bundle\") pod \"openstack-cell1-galera-0\" (UID: \"00a9935f-ef2d-4976-97a0-78ee6325433d\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:10:10 crc kubenswrapper[5052]: I1203 12:10:10.370972 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/00a9935f-ef2d-4976-97a0-78ee6325433d-kolla-config\") pod \"openstack-cell1-galera-0\" (UID: \"00a9935f-ef2d-4976-97a0-78ee6325433d\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:10:10 crc kubenswrapper[5052]: I1203 12:10:10.371257 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/00a9935f-ef2d-4976-97a0-78ee6325433d-galera-tls-certs\") pod \"openstack-cell1-galera-0\" (UID: \"00a9935f-ef2d-4976-97a0-78ee6325433d\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:10:10 crc kubenswrapper[5052]: I1203 12:10:10.371373 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/00a9935f-ef2d-4976-97a0-78ee6325433d-config-data-default\") pod \"openstack-cell1-galera-0\" (UID: \"00a9935f-ef2d-4976-97a0-78ee6325433d\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:10:10 crc kubenswrapper[5052]: I1203 12:10:10.371444 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/00a9935f-ef2d-4976-97a0-78ee6325433d-operator-scripts\") pod \"openstack-cell1-galera-0\" (UID: \"00a9935f-ef2d-4976-97a0-78ee6325433d\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:10:10 crc kubenswrapper[5052]: I1203 12:10:10.371539 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage06-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage06-crc\") pod \"openstack-cell1-galera-0\" (UID: \"00a9935f-ef2d-4976-97a0-78ee6325433d\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:10:10 crc kubenswrapper[5052]: I1203 12:10:10.371620 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-gd9zs\" (UniqueName: \"kubernetes.io/projected/00a9935f-ef2d-4976-97a0-78ee6325433d-kube-api-access-gd9zs\") pod \"openstack-cell1-galera-0\" (UID: \"00a9935f-ef2d-4976-97a0-78ee6325433d\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:10:10 crc kubenswrapper[5052]: I1203 12:10:10.371690 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/00a9935f-ef2d-4976-97a0-78ee6325433d-config-data-generated\") pod \"openstack-cell1-galera-0\" (UID: \"00a9935f-ef2d-4976-97a0-78ee6325433d\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:10:10 crc kubenswrapper[5052]: I1203 12:10:10.371767 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/00a9935f-ef2d-4976-97a0-78ee6325433d-combined-ca-bundle\") pod \"openstack-cell1-galera-0\" (UID: \"00a9935f-ef2d-4976-97a0-78ee6325433d\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:10:10 crc kubenswrapper[5052]: I1203 12:10:10.372068 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/00a9935f-ef2d-4976-97a0-78ee6325433d-kolla-config\") pod \"openstack-cell1-galera-0\" (UID: \"00a9935f-ef2d-4976-97a0-78ee6325433d\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:10:10 crc kubenswrapper[5052]: I1203 12:10:10.372639 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage06-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage06-crc\") pod \"openstack-cell1-galera-0\" (UID: \"00a9935f-ef2d-4976-97a0-78ee6325433d\") device mount path \"/mnt/openstack/pv06\"" pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:10:10 crc kubenswrapper[5052]: I1203 12:10:10.373090 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/00a9935f-ef2d-4976-97a0-78ee6325433d-config-data-default\") pod \"openstack-cell1-galera-0\" (UID: \"00a9935f-ef2d-4976-97a0-78ee6325433d\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:10:10 crc kubenswrapper[5052]: I1203 12:10:10.373603 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/00a9935f-ef2d-4976-97a0-78ee6325433d-config-data-generated\") pod \"openstack-cell1-galera-0\" (UID: \"00a9935f-ef2d-4976-97a0-78ee6325433d\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:10:10 crc kubenswrapper[5052]: I1203 12:10:10.374342 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/00a9935f-ef2d-4976-97a0-78ee6325433d-operator-scripts\") pod \"openstack-cell1-galera-0\" (UID: \"00a9935f-ef2d-4976-97a0-78ee6325433d\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:10:10 crc kubenswrapper[5052]: I1203 12:10:10.377666 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/00a9935f-ef2d-4976-97a0-78ee6325433d-galera-tls-certs\") pod \"openstack-cell1-galera-0\" (UID: \"00a9935f-ef2d-4976-97a0-78ee6325433d\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:10:10 crc kubenswrapper[5052]: I1203 12:10:10.386864 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/00a9935f-ef2d-4976-97a0-78ee6325433d-combined-ca-bundle\") pod \"openstack-cell1-galera-0\" (UID: \"00a9935f-ef2d-4976-97a0-78ee6325433d\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:10:10 crc kubenswrapper[5052]: I1203 12:10:10.398222 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-gd9zs\" (UniqueName: \"kubernetes.io/projected/00a9935f-ef2d-4976-97a0-78ee6325433d-kube-api-access-gd9zs\") pod \"openstack-cell1-galera-0\" (UID: \"00a9935f-ef2d-4976-97a0-78ee6325433d\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:10:10 crc kubenswrapper[5052]: I1203 12:10:10.406392 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage06-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage06-crc\") pod \"openstack-cell1-galera-0\" (UID: \"00a9935f-ef2d-4976-97a0-78ee6325433d\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:10:10 crc kubenswrapper[5052]: I1203 12:10:10.492295 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/memcached-0"] Dec 03 12:10:10 crc kubenswrapper[5052]: I1203 12:10:10.493798 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/memcached-0" Dec 03 12:10:10 crc kubenswrapper[5052]: I1203 12:10:10.498299 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"memcached-memcached-dockercfg-vhwd2" Dec 03 12:10:10 crc kubenswrapper[5052]: I1203 12:10:10.498723 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-memcached-svc" Dec 03 12:10:10 crc kubenswrapper[5052]: I1203 12:10:10.500135 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"memcached-config-data" Dec 03 12:10:10 crc kubenswrapper[5052]: I1203 12:10:10.503755 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/memcached-0"] Dec 03 12:10:10 crc kubenswrapper[5052]: I1203 12:10:10.513926 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:10:10 crc kubenswrapper[5052]: I1203 12:10:10.575022 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"memcached-tls-certs\" (UniqueName: \"kubernetes.io/secret/b28a125a-7966-4f97-91c1-fa52d6062405-memcached-tls-certs\") pod \"memcached-0\" (UID: \"b28a125a-7966-4f97-91c1-fa52d6062405\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:10:10 crc kubenswrapper[5052]: I1203 12:10:10.575088 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/b28a125a-7966-4f97-91c1-fa52d6062405-config-data\") pod \"memcached-0\" (UID: \"b28a125a-7966-4f97-91c1-fa52d6062405\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:10:10 crc kubenswrapper[5052]: I1203 12:10:10.575418 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8rz4s\" (UniqueName: \"kubernetes.io/projected/b28a125a-7966-4f97-91c1-fa52d6062405-kube-api-access-8rz4s\") pod \"memcached-0\" (UID: \"b28a125a-7966-4f97-91c1-fa52d6062405\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:10:10 crc kubenswrapper[5052]: I1203 12:10:10.575721 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/b28a125a-7966-4f97-91c1-fa52d6062405-kolla-config\") pod \"memcached-0\" (UID: \"b28a125a-7966-4f97-91c1-fa52d6062405\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:10:10 crc kubenswrapper[5052]: I1203 12:10:10.575809 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b28a125a-7966-4f97-91c1-fa52d6062405-combined-ca-bundle\") pod \"memcached-0\" (UID: \"b28a125a-7966-4f97-91c1-fa52d6062405\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:10:10 crc kubenswrapper[5052]: I1203 12:10:10.680267 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"memcached-tls-certs\" (UniqueName: \"kubernetes.io/secret/b28a125a-7966-4f97-91c1-fa52d6062405-memcached-tls-certs\") pod \"memcached-0\" (UID: \"b28a125a-7966-4f97-91c1-fa52d6062405\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:10:10 crc kubenswrapper[5052]: I1203 12:10:10.680949 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/b28a125a-7966-4f97-91c1-fa52d6062405-config-data\") pod \"memcached-0\" (UID: \"b28a125a-7966-4f97-91c1-fa52d6062405\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:10:10 crc kubenswrapper[5052]: I1203 12:10:10.681087 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-8rz4s\" (UniqueName: \"kubernetes.io/projected/b28a125a-7966-4f97-91c1-fa52d6062405-kube-api-access-8rz4s\") pod \"memcached-0\" (UID: \"b28a125a-7966-4f97-91c1-fa52d6062405\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:10:10 crc kubenswrapper[5052]: I1203 12:10:10.681205 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/b28a125a-7966-4f97-91c1-fa52d6062405-kolla-config\") pod \"memcached-0\" (UID: \"b28a125a-7966-4f97-91c1-fa52d6062405\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:10:10 crc kubenswrapper[5052]: I1203 12:10:10.681240 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b28a125a-7966-4f97-91c1-fa52d6062405-combined-ca-bundle\") pod \"memcached-0\" (UID: \"b28a125a-7966-4f97-91c1-fa52d6062405\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:10:10 crc kubenswrapper[5052]: I1203 12:10:10.684213 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/b28a125a-7966-4f97-91c1-fa52d6062405-kolla-config\") pod \"memcached-0\" (UID: \"b28a125a-7966-4f97-91c1-fa52d6062405\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:10:10 crc kubenswrapper[5052]: I1203 12:10:10.684288 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/b28a125a-7966-4f97-91c1-fa52d6062405-config-data\") pod \"memcached-0\" (UID: \"b28a125a-7966-4f97-91c1-fa52d6062405\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:10:10 crc kubenswrapper[5052]: I1203 12:10:10.699472 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b28a125a-7966-4f97-91c1-fa52d6062405-combined-ca-bundle\") pod \"memcached-0\" (UID: \"b28a125a-7966-4f97-91c1-fa52d6062405\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:10:10 crc kubenswrapper[5052]: I1203 12:10:10.700000 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"memcached-tls-certs\" (UniqueName: \"kubernetes.io/secret/b28a125a-7966-4f97-91c1-fa52d6062405-memcached-tls-certs\") pod \"memcached-0\" (UID: \"b28a125a-7966-4f97-91c1-fa52d6062405\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:10:10 crc kubenswrapper[5052]: I1203 12:10:10.704853 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-8rz4s\" (UniqueName: \"kubernetes.io/projected/b28a125a-7966-4f97-91c1-fa52d6062405-kube-api-access-8rz4s\") pod \"memcached-0\" (UID: \"b28a125a-7966-4f97-91c1-fa52d6062405\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:10:10 crc kubenswrapper[5052]: I1203 12:10:10.710236 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-galera-0" event={"ID":"85f5676f-32e8-4c3d-8729-e1eb2fdc8dc9","Type":"ContainerStarted","Data":"83ffd9d45c67b785bed3d072b308795c1b569e98f4500db5effcc832d24bf349"} Dec 03 12:10:10 crc kubenswrapper[5052]: I1203 12:10:10.710284 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-galera-0" event={"ID":"85f5676f-32e8-4c3d-8729-e1eb2fdc8dc9","Type":"ContainerStarted","Data":"8079da34f90ae24fe3455ebeaf544cb53b92f6b936c319f989b8c4d8f8ba1026"} Dec 03 12:10:10 crc kubenswrapper[5052]: I1203 12:10:10.811175 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/memcached-0" Dec 03 12:10:11 crc kubenswrapper[5052]: I1203 12:10:11.030143 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/openstack-cell1-galera-0"] Dec 03 12:10:11 crc kubenswrapper[5052]: I1203 12:10:11.273965 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/memcached-0"] Dec 03 12:10:11 crc kubenswrapper[5052]: W1203 12:10:11.276533 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb28a125a_7966_4f97_91c1_fa52d6062405.slice/crio-1947bc52826f5255a366a00d509039b09060cb4706c21b38e5917db37d5e2fc2 WatchSource:0}: Error finding container 1947bc52826f5255a366a00d509039b09060cb4706c21b38e5917db37d5e2fc2: Status 404 returned error can't find the container with id 1947bc52826f5255a366a00d509039b09060cb4706c21b38e5917db37d5e2fc2 Dec 03 12:10:11 crc kubenswrapper[5052]: I1203 12:10:11.716556 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-cell1-galera-0" event={"ID":"00a9935f-ef2d-4976-97a0-78ee6325433d","Type":"ContainerStarted","Data":"aabef9c4a1cc4f1273de63111a3705eddf616b50266ee22fb415a42c62f1f823"} Dec 03 12:10:11 crc kubenswrapper[5052]: I1203 12:10:11.718004 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/memcached-0" event={"ID":"b28a125a-7966-4f97-91c1-fa52d6062405","Type":"ContainerStarted","Data":"1947bc52826f5255a366a00d509039b09060cb4706c21b38e5917db37d5e2fc2"} Dec 03 12:10:12 crc kubenswrapper[5052]: I1203 12:10:12.332777 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/kube-state-metrics-0"] Dec 03 12:10:12 crc kubenswrapper[5052]: I1203 12:10:12.334248 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:10:12 crc kubenswrapper[5052]: I1203 12:10:12.337318 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"telemetry-ceilometer-dockercfg-zm4x9" Dec 03 12:10:12 crc kubenswrapper[5052]: I1203 12:10:12.364029 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/kube-state-metrics-0"] Dec 03 12:10:12 crc kubenswrapper[5052]: I1203 12:10:12.422757 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kmf8k\" (UniqueName: \"kubernetes.io/projected/cdef7499-3235-479a-8065-de26b912d8ea-kube-api-access-kmf8k\") pod \"kube-state-metrics-0\" (UID: \"cdef7499-3235-479a-8065-de26b912d8ea\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:10:12 crc kubenswrapper[5052]: I1203 12:10:12.524529 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-kmf8k\" (UniqueName: \"kubernetes.io/projected/cdef7499-3235-479a-8065-de26b912d8ea-kube-api-access-kmf8k\") pod \"kube-state-metrics-0\" (UID: \"cdef7499-3235-479a-8065-de26b912d8ea\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:10:12 crc kubenswrapper[5052]: I1203 12:10:12.545456 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-kmf8k\" (UniqueName: \"kubernetes.io/projected/cdef7499-3235-479a-8065-de26b912d8ea-kube-api-access-kmf8k\") pod \"kube-state-metrics-0\" (UID: \"cdef7499-3235-479a-8065-de26b912d8ea\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:10:12 crc kubenswrapper[5052]: I1203 12:10:12.659595 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:10:12 crc kubenswrapper[5052]: I1203 12:10:12.730856 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-cell1-galera-0" event={"ID":"00a9935f-ef2d-4976-97a0-78ee6325433d","Type":"ContainerStarted","Data":"a3147e7344c54bf91a2b2c3ace19353f08a7f75ada18932e0399780c89d82766"} Dec 03 12:10:12 crc kubenswrapper[5052]: I1203 12:10:12.735045 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/memcached-0" event={"ID":"b28a125a-7966-4f97-91c1-fa52d6062405","Type":"ContainerStarted","Data":"dc5c838958d7fbbc47d97aabd5b021dc0188e172b8b2ed5ab168cef70e171e07"} Dec 03 12:10:12 crc kubenswrapper[5052]: I1203 12:10:12.735208 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/memcached-0" Dec 03 12:10:12 crc kubenswrapper[5052]: I1203 12:10:12.786794 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/memcached-0" podStartSLOduration=2.786777814 podStartE2EDuration="2.786777814s" podCreationTimestamp="2025-12-03 12:10:10 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:10:12.784992012 +0000 UTC m=+3003.373233776" watchObservedRunningTime="2025-12-03 12:10:12.786777814 +0000 UTC m=+3003.375019578" Dec 03 12:10:13 crc kubenswrapper[5052]: I1203 12:10:13.105040 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/kube-state-metrics-0"] Dec 03 12:10:13 crc kubenswrapper[5052]: I1203 12:10:13.746406 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/kube-state-metrics-0" event={"ID":"cdef7499-3235-479a-8065-de26b912d8ea","Type":"ContainerStarted","Data":"478edfe57f1c5c41ae50aa682267c2e73e1011cc8a294da2c80f1a3ec3757ef1"} Dec 03 12:10:14 crc kubenswrapper[5052]: I1203 12:10:14.756499 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/kube-state-metrics-0" event={"ID":"cdef7499-3235-479a-8065-de26b912d8ea","Type":"ContainerStarted","Data":"7f0cbafa963452fc9d1387d850c95247fd3653a544ede26e15ff50d89f0a33f4"} Dec 03 12:10:14 crc kubenswrapper[5052]: I1203 12:10:14.757089 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:10:14 crc kubenswrapper[5052]: I1203 12:10:14.776566 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/kube-state-metrics-0" podStartSLOduration=2.310724637 podStartE2EDuration="2.776538808s" podCreationTimestamp="2025-12-03 12:10:12 +0000 UTC" firstStartedPulling="2025-12-03 12:10:13.110307841 +0000 UTC m=+3003.698549605" lastFinishedPulling="2025-12-03 12:10:13.576122012 +0000 UTC m=+3004.164363776" observedRunningTime="2025-12-03 12:10:14.771397861 +0000 UTC m=+3005.359639635" watchObservedRunningTime="2025-12-03 12:10:14.776538808 +0000 UTC m=+3005.364780572" Dec 03 12:10:15 crc kubenswrapper[5052]: I1203 12:10:15.479746 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ovsdbserver-nb-0"] Dec 03 12:10:15 crc kubenswrapper[5052]: I1203 12:10:15.481095 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:10:15 crc kubenswrapper[5052]: I1203 12:10:15.484605 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-ovn-metrics" Dec 03 12:10:15 crc kubenswrapper[5052]: I1203 12:10:15.484893 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ovncluster-ovndbcluster-nb-dockercfg-h9t7j" Dec 03 12:10:15 crc kubenswrapper[5052]: I1203 12:10:15.485066 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"ovndbcluster-nb-config" Dec 03 12:10:15 crc kubenswrapper[5052]: I1203 12:10:15.485184 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-ovndbcluster-nb-ovndbs" Dec 03 12:10:15 crc kubenswrapper[5052]: I1203 12:10:15.485325 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"ovndbcluster-nb-scripts" Dec 03 12:10:15 crc kubenswrapper[5052]: I1203 12:10:15.493553 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-nb-0"] Dec 03 12:10:15 crc kubenswrapper[5052]: I1203 12:10:15.673452 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/0a2de558-2a4b-46ef-9b53-cada8cd466c4-ovsdb-rundir\") pod \"ovsdbserver-nb-0\" (UID: \"0a2de558-2a4b-46ef-9b53-cada8cd466c4\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:10:15 crc kubenswrapper[5052]: I1203 12:10:15.673499 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0a2de558-2a4b-46ef-9b53-cada8cd466c4-config\") pod \"ovsdbserver-nb-0\" (UID: \"0a2de558-2a4b-46ef-9b53-cada8cd466c4\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:10:15 crc kubenswrapper[5052]: I1203 12:10:15.673537 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovsdbserver-nb-tls-certs\" (UniqueName: \"kubernetes.io/secret/0a2de558-2a4b-46ef-9b53-cada8cd466c4-ovsdbserver-nb-tls-certs\") pod \"ovsdbserver-nb-0\" (UID: \"0a2de558-2a4b-46ef-9b53-cada8cd466c4\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:10:15 crc kubenswrapper[5052]: I1203 12:10:15.673564 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2x6wd\" (UniqueName: \"kubernetes.io/projected/0a2de558-2a4b-46ef-9b53-cada8cd466c4-kube-api-access-2x6wd\") pod \"ovsdbserver-nb-0\" (UID: \"0a2de558-2a4b-46ef-9b53-cada8cd466c4\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:10:15 crc kubenswrapper[5052]: I1203 12:10:15.673762 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage07-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage07-crc\") pod \"ovsdbserver-nb-0\" (UID: \"0a2de558-2a4b-46ef-9b53-cada8cd466c4\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:10:15 crc kubenswrapper[5052]: I1203 12:10:15.673801 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/0a2de558-2a4b-46ef-9b53-cada8cd466c4-scripts\") pod \"ovsdbserver-nb-0\" (UID: \"0a2de558-2a4b-46ef-9b53-cada8cd466c4\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:10:15 crc kubenswrapper[5052]: I1203 12:10:15.673939 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0a2de558-2a4b-46ef-9b53-cada8cd466c4-combined-ca-bundle\") pod \"ovsdbserver-nb-0\" (UID: \"0a2de558-2a4b-46ef-9b53-cada8cd466c4\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:10:15 crc kubenswrapper[5052]: I1203 12:10:15.674077 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/0a2de558-2a4b-46ef-9b53-cada8cd466c4-metrics-certs-tls-certs\") pod \"ovsdbserver-nb-0\" (UID: \"0a2de558-2a4b-46ef-9b53-cada8cd466c4\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:10:15 crc kubenswrapper[5052]: I1203 12:10:15.765282 5052 generic.go:334] "Generic (PLEG): container finished" podID="00a9935f-ef2d-4976-97a0-78ee6325433d" containerID="a3147e7344c54bf91a2b2c3ace19353f08a7f75ada18932e0399780c89d82766" exitCode=0 Dec 03 12:10:15 crc kubenswrapper[5052]: I1203 12:10:15.765365 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-cell1-galera-0" event={"ID":"00a9935f-ef2d-4976-97a0-78ee6325433d","Type":"ContainerDied","Data":"a3147e7344c54bf91a2b2c3ace19353f08a7f75ada18932e0399780c89d82766"} Dec 03 12:10:15 crc kubenswrapper[5052]: I1203 12:10:15.767389 5052 generic.go:334] "Generic (PLEG): container finished" podID="85f5676f-32e8-4c3d-8729-e1eb2fdc8dc9" containerID="83ffd9d45c67b785bed3d072b308795c1b569e98f4500db5effcc832d24bf349" exitCode=0 Dec 03 12:10:15 crc kubenswrapper[5052]: I1203 12:10:15.767476 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-galera-0" event={"ID":"85f5676f-32e8-4c3d-8729-e1eb2fdc8dc9","Type":"ContainerDied","Data":"83ffd9d45c67b785bed3d072b308795c1b569e98f4500db5effcc832d24bf349"} Dec 03 12:10:15 crc kubenswrapper[5052]: I1203 12:10:15.775304 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0a2de558-2a4b-46ef-9b53-cada8cd466c4-config\") pod \"ovsdbserver-nb-0\" (UID: \"0a2de558-2a4b-46ef-9b53-cada8cd466c4\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:10:15 crc kubenswrapper[5052]: I1203 12:10:15.775603 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovsdbserver-nb-tls-certs\" (UniqueName: \"kubernetes.io/secret/0a2de558-2a4b-46ef-9b53-cada8cd466c4-ovsdbserver-nb-tls-certs\") pod \"ovsdbserver-nb-0\" (UID: \"0a2de558-2a4b-46ef-9b53-cada8cd466c4\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:10:15 crc kubenswrapper[5052]: I1203 12:10:15.775637 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-2x6wd\" (UniqueName: \"kubernetes.io/projected/0a2de558-2a4b-46ef-9b53-cada8cd466c4-kube-api-access-2x6wd\") pod \"ovsdbserver-nb-0\" (UID: \"0a2de558-2a4b-46ef-9b53-cada8cd466c4\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:10:15 crc kubenswrapper[5052]: I1203 12:10:15.775712 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage07-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage07-crc\") pod \"ovsdbserver-nb-0\" (UID: \"0a2de558-2a4b-46ef-9b53-cada8cd466c4\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:10:15 crc kubenswrapper[5052]: I1203 12:10:15.775733 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/0a2de558-2a4b-46ef-9b53-cada8cd466c4-scripts\") pod \"ovsdbserver-nb-0\" (UID: \"0a2de558-2a4b-46ef-9b53-cada8cd466c4\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:10:15 crc kubenswrapper[5052]: I1203 12:10:15.775842 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0a2de558-2a4b-46ef-9b53-cada8cd466c4-combined-ca-bundle\") pod \"ovsdbserver-nb-0\" (UID: \"0a2de558-2a4b-46ef-9b53-cada8cd466c4\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:10:15 crc kubenswrapper[5052]: I1203 12:10:15.778352 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/0a2de558-2a4b-46ef-9b53-cada8cd466c4-metrics-certs-tls-certs\") pod \"ovsdbserver-nb-0\" (UID: \"0a2de558-2a4b-46ef-9b53-cada8cd466c4\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:10:15 crc kubenswrapper[5052]: I1203 12:10:15.778400 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/0a2de558-2a4b-46ef-9b53-cada8cd466c4-ovsdb-rundir\") pod \"ovsdbserver-nb-0\" (UID: \"0a2de558-2a4b-46ef-9b53-cada8cd466c4\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:10:15 crc kubenswrapper[5052]: I1203 12:10:15.778851 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage07-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage07-crc\") pod \"ovsdbserver-nb-0\" (UID: \"0a2de558-2a4b-46ef-9b53-cada8cd466c4\") device mount path \"/mnt/openstack/pv07\"" pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:10:15 crc kubenswrapper[5052]: I1203 12:10:15.779326 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/0a2de558-2a4b-46ef-9b53-cada8cd466c4-ovsdb-rundir\") pod \"ovsdbserver-nb-0\" (UID: \"0a2de558-2a4b-46ef-9b53-cada8cd466c4\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:10:15 crc kubenswrapper[5052]: I1203 12:10:15.779952 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0a2de558-2a4b-46ef-9b53-cada8cd466c4-config\") pod \"ovsdbserver-nb-0\" (UID: \"0a2de558-2a4b-46ef-9b53-cada8cd466c4\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:10:15 crc kubenswrapper[5052]: I1203 12:10:15.780391 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/0a2de558-2a4b-46ef-9b53-cada8cd466c4-scripts\") pod \"ovsdbserver-nb-0\" (UID: \"0a2de558-2a4b-46ef-9b53-cada8cd466c4\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:10:15 crc kubenswrapper[5052]: I1203 12:10:15.782031 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovsdbserver-nb-tls-certs\" (UniqueName: \"kubernetes.io/secret/0a2de558-2a4b-46ef-9b53-cada8cd466c4-ovsdbserver-nb-tls-certs\") pod \"ovsdbserver-nb-0\" (UID: \"0a2de558-2a4b-46ef-9b53-cada8cd466c4\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:10:15 crc kubenswrapper[5052]: I1203 12:10:15.783742 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0a2de558-2a4b-46ef-9b53-cada8cd466c4-combined-ca-bundle\") pod \"ovsdbserver-nb-0\" (UID: \"0a2de558-2a4b-46ef-9b53-cada8cd466c4\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:10:15 crc kubenswrapper[5052]: I1203 12:10:15.811477 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/0a2de558-2a4b-46ef-9b53-cada8cd466c4-metrics-certs-tls-certs\") pod \"ovsdbserver-nb-0\" (UID: \"0a2de558-2a4b-46ef-9b53-cada8cd466c4\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:10:15 crc kubenswrapper[5052]: I1203 12:10:15.816480 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-2x6wd\" (UniqueName: \"kubernetes.io/projected/0a2de558-2a4b-46ef-9b53-cada8cd466c4-kube-api-access-2x6wd\") pod \"ovsdbserver-nb-0\" (UID: \"0a2de558-2a4b-46ef-9b53-cada8cd466c4\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:10:15 crc kubenswrapper[5052]: I1203 12:10:15.846060 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage07-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage07-crc\") pod \"ovsdbserver-nb-0\" (UID: \"0a2de558-2a4b-46ef-9b53-cada8cd466c4\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:10:16 crc kubenswrapper[5052]: I1203 12:10:16.103134 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:10:16 crc kubenswrapper[5052]: I1203 12:10:16.558478 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-nb-0"] Dec 03 12:10:16 crc kubenswrapper[5052]: W1203 12:10:16.564843 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0a2de558_2a4b_46ef_9b53_cada8cd466c4.slice/crio-ad711c538dba8e33d365eb420757fc8e4b34b6a8568c18585364f148b4aa302e WatchSource:0}: Error finding container ad711c538dba8e33d365eb420757fc8e4b34b6a8568c18585364f148b4aa302e: Status 404 returned error can't find the container with id ad711c538dba8e33d365eb420757fc8e4b34b6a8568c18585364f148b4aa302e Dec 03 12:10:16 crc kubenswrapper[5052]: I1203 12:10:16.781615 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-galera-0" event={"ID":"85f5676f-32e8-4c3d-8729-e1eb2fdc8dc9","Type":"ContainerStarted","Data":"3ba0d39f21ff90002edd3d3fb8c1a2fd07737053623cac5f18054747025f4d6e"} Dec 03 12:10:16 crc kubenswrapper[5052]: I1203 12:10:16.783315 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-cell1-galera-0" event={"ID":"00a9935f-ef2d-4976-97a0-78ee6325433d","Type":"ContainerStarted","Data":"de161b1412d41d5ec322763c71b1221e12135067ced1a67a7f68542d387c4abb"} Dec 03 12:10:16 crc kubenswrapper[5052]: I1203 12:10:16.785148 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-nb-0" event={"ID":"0a2de558-2a4b-46ef-9b53-cada8cd466c4","Type":"ContainerStarted","Data":"b53a930fff1916d446b8a6d1e63e019c2f77d3ae6348e99f24ce01cb0d9242b1"} Dec 03 12:10:16 crc kubenswrapper[5052]: I1203 12:10:16.785172 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-nb-0" event={"ID":"0a2de558-2a4b-46ef-9b53-cada8cd466c4","Type":"ContainerStarted","Data":"ad711c538dba8e33d365eb420757fc8e4b34b6a8568c18585364f148b4aa302e"} Dec 03 12:10:16 crc kubenswrapper[5052]: I1203 12:10:16.802288 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/openstack-galera-0" podStartSLOduration=9.802267288 podStartE2EDuration="9.802267288s" podCreationTimestamp="2025-12-03 12:10:07 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:10:16.799115978 +0000 UTC m=+3007.387357742" watchObservedRunningTime="2025-12-03 12:10:16.802267288 +0000 UTC m=+3007.390509042" Dec 03 12:10:16 crc kubenswrapper[5052]: I1203 12:10:16.835061 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/openstack-cell1-galera-0" podStartSLOduration=7.835038914 podStartE2EDuration="7.835038914s" podCreationTimestamp="2025-12-03 12:10:09 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:10:16.834127628 +0000 UTC m=+3007.422369572" watchObservedRunningTime="2025-12-03 12:10:16.835038914 +0000 UTC m=+3007.423280678" Dec 03 12:10:17 crc kubenswrapper[5052]: I1203 12:10:17.802264 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-nb-0" event={"ID":"0a2de558-2a4b-46ef-9b53-cada8cd466c4","Type":"ContainerStarted","Data":"210f225008eb18ae291abefbe19ad7b03eaf69b1879a6f6bbe585e72babe1848"} Dec 03 12:10:17 crc kubenswrapper[5052]: I1203 12:10:17.823290 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/ovsdbserver-nb-0" podStartSLOduration=3.823269191 podStartE2EDuration="3.823269191s" podCreationTimestamp="2025-12-03 12:10:14 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:10:17.822237702 +0000 UTC m=+3008.410479466" watchObservedRunningTime="2025-12-03 12:10:17.823269191 +0000 UTC m=+3008.411510955" Dec 03 12:10:18 crc kubenswrapper[5052]: I1203 12:10:18.472639 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ovsdbserver-sb-0"] Dec 03 12:10:18 crc kubenswrapper[5052]: I1203 12:10:18.475111 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:10:18 crc kubenswrapper[5052]: I1203 12:10:18.479079 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"ovndbcluster-sb-config" Dec 03 12:10:18 crc kubenswrapper[5052]: I1203 12:10:18.479144 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-ovndbcluster-sb-ovndbs" Dec 03 12:10:18 crc kubenswrapper[5052]: I1203 12:10:18.481933 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ovncluster-ovndbcluster-sb-dockercfg-8vhk6" Dec 03 12:10:18 crc kubenswrapper[5052]: I1203 12:10:18.482094 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"ovndbcluster-sb-scripts" Dec 03 12:10:18 crc kubenswrapper[5052]: I1203 12:10:18.484890 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-sb-0"] Dec 03 12:10:18 crc kubenswrapper[5052]: I1203 12:10:18.521932 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mp7fw\" (UniqueName: \"kubernetes.io/projected/baa1c9d4-5887-47e3-be99-96f718178f58-kube-api-access-mp7fw\") pod \"ovsdbserver-sb-0\" (UID: \"baa1c9d4-5887-47e3-be99-96f718178f58\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:10:18 crc kubenswrapper[5052]: I1203 12:10:18.521994 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/baa1c9d4-5887-47e3-be99-96f718178f58-ovsdb-rundir\") pod \"ovsdbserver-sb-0\" (UID: \"baa1c9d4-5887-47e3-be99-96f718178f58\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:10:18 crc kubenswrapper[5052]: I1203 12:10:18.522093 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage11-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage11-crc\") pod \"ovsdbserver-sb-0\" (UID: \"baa1c9d4-5887-47e3-be99-96f718178f58\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:10:18 crc kubenswrapper[5052]: I1203 12:10:18.522191 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/baa1c9d4-5887-47e3-be99-96f718178f58-metrics-certs-tls-certs\") pod \"ovsdbserver-sb-0\" (UID: \"baa1c9d4-5887-47e3-be99-96f718178f58\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:10:18 crc kubenswrapper[5052]: I1203 12:10:18.522237 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/baa1c9d4-5887-47e3-be99-96f718178f58-config\") pod \"ovsdbserver-sb-0\" (UID: \"baa1c9d4-5887-47e3-be99-96f718178f58\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:10:18 crc kubenswrapper[5052]: I1203 12:10:18.522283 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/baa1c9d4-5887-47e3-be99-96f718178f58-combined-ca-bundle\") pod \"ovsdbserver-sb-0\" (UID: \"baa1c9d4-5887-47e3-be99-96f718178f58\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:10:18 crc kubenswrapper[5052]: I1203 12:10:18.522347 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovsdbserver-sb-tls-certs\" (UniqueName: \"kubernetes.io/secret/baa1c9d4-5887-47e3-be99-96f718178f58-ovsdbserver-sb-tls-certs\") pod \"ovsdbserver-sb-0\" (UID: \"baa1c9d4-5887-47e3-be99-96f718178f58\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:10:18 crc kubenswrapper[5052]: I1203 12:10:18.522365 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/baa1c9d4-5887-47e3-be99-96f718178f58-scripts\") pod \"ovsdbserver-sb-0\" (UID: \"baa1c9d4-5887-47e3-be99-96f718178f58\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:10:18 crc kubenswrapper[5052]: I1203 12:10:18.623212 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-mp7fw\" (UniqueName: \"kubernetes.io/projected/baa1c9d4-5887-47e3-be99-96f718178f58-kube-api-access-mp7fw\") pod \"ovsdbserver-sb-0\" (UID: \"baa1c9d4-5887-47e3-be99-96f718178f58\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:10:18 crc kubenswrapper[5052]: I1203 12:10:18.623296 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/baa1c9d4-5887-47e3-be99-96f718178f58-ovsdb-rundir\") pod \"ovsdbserver-sb-0\" (UID: \"baa1c9d4-5887-47e3-be99-96f718178f58\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:10:18 crc kubenswrapper[5052]: I1203 12:10:18.623348 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage11-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage11-crc\") pod \"ovsdbserver-sb-0\" (UID: \"baa1c9d4-5887-47e3-be99-96f718178f58\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:10:18 crc kubenswrapper[5052]: I1203 12:10:18.623402 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/baa1c9d4-5887-47e3-be99-96f718178f58-metrics-certs-tls-certs\") pod \"ovsdbserver-sb-0\" (UID: \"baa1c9d4-5887-47e3-be99-96f718178f58\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:10:18 crc kubenswrapper[5052]: I1203 12:10:18.623424 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/baa1c9d4-5887-47e3-be99-96f718178f58-config\") pod \"ovsdbserver-sb-0\" (UID: \"baa1c9d4-5887-47e3-be99-96f718178f58\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:10:18 crc kubenswrapper[5052]: I1203 12:10:18.623448 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/baa1c9d4-5887-47e3-be99-96f718178f58-combined-ca-bundle\") pod \"ovsdbserver-sb-0\" (UID: \"baa1c9d4-5887-47e3-be99-96f718178f58\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:10:18 crc kubenswrapper[5052]: I1203 12:10:18.623481 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovsdbserver-sb-tls-certs\" (UniqueName: \"kubernetes.io/secret/baa1c9d4-5887-47e3-be99-96f718178f58-ovsdbserver-sb-tls-certs\") pod \"ovsdbserver-sb-0\" (UID: \"baa1c9d4-5887-47e3-be99-96f718178f58\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:10:18 crc kubenswrapper[5052]: I1203 12:10:18.623502 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/baa1c9d4-5887-47e3-be99-96f718178f58-scripts\") pod \"ovsdbserver-sb-0\" (UID: \"baa1c9d4-5887-47e3-be99-96f718178f58\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:10:18 crc kubenswrapper[5052]: I1203 12:10:18.623764 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/baa1c9d4-5887-47e3-be99-96f718178f58-ovsdb-rundir\") pod \"ovsdbserver-sb-0\" (UID: \"baa1c9d4-5887-47e3-be99-96f718178f58\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:10:18 crc kubenswrapper[5052]: I1203 12:10:18.624046 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage11-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage11-crc\") pod \"ovsdbserver-sb-0\" (UID: \"baa1c9d4-5887-47e3-be99-96f718178f58\") device mount path \"/mnt/openstack/pv11\"" pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:10:18 crc kubenswrapper[5052]: I1203 12:10:18.625017 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/baa1c9d4-5887-47e3-be99-96f718178f58-config\") pod \"ovsdbserver-sb-0\" (UID: \"baa1c9d4-5887-47e3-be99-96f718178f58\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:10:18 crc kubenswrapper[5052]: I1203 12:10:18.625308 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/baa1c9d4-5887-47e3-be99-96f718178f58-scripts\") pod \"ovsdbserver-sb-0\" (UID: \"baa1c9d4-5887-47e3-be99-96f718178f58\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:10:18 crc kubenswrapper[5052]: I1203 12:10:18.631320 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/baa1c9d4-5887-47e3-be99-96f718178f58-metrics-certs-tls-certs\") pod \"ovsdbserver-sb-0\" (UID: \"baa1c9d4-5887-47e3-be99-96f718178f58\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:10:18 crc kubenswrapper[5052]: I1203 12:10:18.632821 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/baa1c9d4-5887-47e3-be99-96f718178f58-combined-ca-bundle\") pod \"ovsdbserver-sb-0\" (UID: \"baa1c9d4-5887-47e3-be99-96f718178f58\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:10:18 crc kubenswrapper[5052]: I1203 12:10:18.640396 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovsdbserver-sb-tls-certs\" (UniqueName: \"kubernetes.io/secret/baa1c9d4-5887-47e3-be99-96f718178f58-ovsdbserver-sb-tls-certs\") pod \"ovsdbserver-sb-0\" (UID: \"baa1c9d4-5887-47e3-be99-96f718178f58\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:10:18 crc kubenswrapper[5052]: I1203 12:10:18.650409 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage11-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage11-crc\") pod \"ovsdbserver-sb-0\" (UID: \"baa1c9d4-5887-47e3-be99-96f718178f58\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:10:18 crc kubenswrapper[5052]: I1203 12:10:18.652072 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-mp7fw\" (UniqueName: \"kubernetes.io/projected/baa1c9d4-5887-47e3-be99-96f718178f58-kube-api-access-mp7fw\") pod \"ovsdbserver-sb-0\" (UID: \"baa1c9d4-5887-47e3-be99-96f718178f58\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:10:18 crc kubenswrapper[5052]: I1203 12:10:18.794656 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:10:19 crc kubenswrapper[5052]: I1203 12:10:19.104037 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:10:19 crc kubenswrapper[5052]: I1203 12:10:19.147015 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:10:19 crc kubenswrapper[5052]: I1203 12:10:19.257696 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:10:19 crc kubenswrapper[5052]: I1203 12:10:19.257767 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:10:19 crc kubenswrapper[5052]: I1203 12:10:19.306015 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-sb-0"] Dec 03 12:10:19 crc kubenswrapper[5052]: W1203 12:10:19.310733 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbaa1c9d4_5887_47e3_be99_96f718178f58.slice/crio-b548a4c8f3c848f6aed22eb43289bcd3f6999edee2eb2631b52567a1e2f69b61 WatchSource:0}: Error finding container b548a4c8f3c848f6aed22eb43289bcd3f6999edee2eb2631b52567a1e2f69b61: Status 404 returned error can't find the container with id b548a4c8f3c848f6aed22eb43289bcd3f6999edee2eb2631b52567a1e2f69b61 Dec 03 12:10:19 crc kubenswrapper[5052]: I1203 12:10:19.820484 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-sb-0" event={"ID":"baa1c9d4-5887-47e3-be99-96f718178f58","Type":"ContainerStarted","Data":"7d5cb223b2febf0f7150eb3ecf28694a92fc2e544ffbcbf9721f02dc30547b38"} Dec 03 12:10:19 crc kubenswrapper[5052]: I1203 12:10:19.820879 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-sb-0" event={"ID":"baa1c9d4-5887-47e3-be99-96f718178f58","Type":"ContainerStarted","Data":"b548a4c8f3c848f6aed22eb43289bcd3f6999edee2eb2631b52567a1e2f69b61"} Dec 03 12:10:19 crc kubenswrapper[5052]: I1203 12:10:19.820902 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:10:20 crc kubenswrapper[5052]: I1203 12:10:20.514281 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:10:20 crc kubenswrapper[5052]: I1203 12:10:20.514343 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:10:20 crc kubenswrapper[5052]: I1203 12:10:20.812403 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/memcached-0" Dec 03 12:10:21 crc kubenswrapper[5052]: I1203 12:10:21.186820 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:10:21 crc kubenswrapper[5052]: I1203 12:10:21.673320 5052 scope.go:117] "RemoveContainer" containerID="63298107b3d090deff9ed92f3d613e5b1c86d9137684d117500257db8ee1c805" Dec 03 12:10:21 crc kubenswrapper[5052]: E1203 12:10:21.673826 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 12:10:21 crc kubenswrapper[5052]: I1203 12:10:21.840755 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-sb-0" event={"ID":"baa1c9d4-5887-47e3-be99-96f718178f58","Type":"ContainerStarted","Data":"b8259e68d536f622f925db84f9b7dd7940949212302ab0a104352098a97b2889"} Dec 03 12:10:22 crc kubenswrapper[5052]: I1203 12:10:22.666373 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:10:22 crc kubenswrapper[5052]: I1203 12:10:22.869564 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/ovsdbserver-sb-0" podStartSLOduration=5.869549018 podStartE2EDuration="5.869549018s" podCreationTimestamp="2025-12-03 12:10:17 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:10:22.867078177 +0000 UTC m=+3013.455319941" watchObservedRunningTime="2025-12-03 12:10:22.869549018 +0000 UTC m=+3013.457790782" Dec 03 12:10:23 crc kubenswrapper[5052]: I1203 12:10:23.781023 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/swift-storage-0"] Dec 03 12:10:23 crc kubenswrapper[5052]: I1203 12:10:23.787965 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:10:23 crc kubenswrapper[5052]: I1203 12:10:23.791644 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"swift-conf" Dec 03 12:10:23 crc kubenswrapper[5052]: I1203 12:10:23.792897 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"swift-ring-files" Dec 03 12:10:23 crc kubenswrapper[5052]: I1203 12:10:23.793442 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"swift-storage-config-data" Dec 03 12:10:23 crc kubenswrapper[5052]: I1203 12:10:23.793747 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"swift-swift-dockercfg-2jv2v" Dec 03 12:10:23 crc kubenswrapper[5052]: I1203 12:10:23.794988 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:10:23 crc kubenswrapper[5052]: I1203 12:10:23.806808 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/swift-storage-0"] Dec 03 12:10:23 crc kubenswrapper[5052]: I1203 12:10:23.931783 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dbgxt\" (UniqueName: \"kubernetes.io/projected/694f3a22-38d9-487d-b887-a915fa8f99b0-kube-api-access-dbgxt\") pod \"swift-storage-0\" (UID: \"694f3a22-38d9-487d-b887-a915fa8f99b0\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:10:23 crc kubenswrapper[5052]: I1203 12:10:23.931935 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage09-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage09-crc\") pod \"swift-storage-0\" (UID: \"694f3a22-38d9-487d-b887-a915fa8f99b0\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:10:23 crc kubenswrapper[5052]: I1203 12:10:23.932001 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cache\" (UniqueName: \"kubernetes.io/empty-dir/694f3a22-38d9-487d-b887-a915fa8f99b0-cache\") pod \"swift-storage-0\" (UID: \"694f3a22-38d9-487d-b887-a915fa8f99b0\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:10:23 crc kubenswrapper[5052]: I1203 12:10:23.932065 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lock\" (UniqueName: \"kubernetes.io/empty-dir/694f3a22-38d9-487d-b887-a915fa8f99b0-lock\") pod \"swift-storage-0\" (UID: \"694f3a22-38d9-487d-b887-a915fa8f99b0\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:10:23 crc kubenswrapper[5052]: I1203 12:10:23.932387 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/694f3a22-38d9-487d-b887-a915fa8f99b0-etc-swift\") pod \"swift-storage-0\" (UID: \"694f3a22-38d9-487d-b887-a915fa8f99b0\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:10:24 crc kubenswrapper[5052]: I1203 12:10:24.034033 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/694f3a22-38d9-487d-b887-a915fa8f99b0-etc-swift\") pod \"swift-storage-0\" (UID: \"694f3a22-38d9-487d-b887-a915fa8f99b0\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:10:24 crc kubenswrapper[5052]: I1203 12:10:24.034101 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-dbgxt\" (UniqueName: \"kubernetes.io/projected/694f3a22-38d9-487d-b887-a915fa8f99b0-kube-api-access-dbgxt\") pod \"swift-storage-0\" (UID: \"694f3a22-38d9-487d-b887-a915fa8f99b0\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:10:24 crc kubenswrapper[5052]: I1203 12:10:24.034133 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage09-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage09-crc\") pod \"swift-storage-0\" (UID: \"694f3a22-38d9-487d-b887-a915fa8f99b0\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:10:24 crc kubenswrapper[5052]: I1203 12:10:24.034152 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"cache\" (UniqueName: \"kubernetes.io/empty-dir/694f3a22-38d9-487d-b887-a915fa8f99b0-cache\") pod \"swift-storage-0\" (UID: \"694f3a22-38d9-487d-b887-a915fa8f99b0\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:10:24 crc kubenswrapper[5052]: I1203 12:10:24.034190 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"lock\" (UniqueName: \"kubernetes.io/empty-dir/694f3a22-38d9-487d-b887-a915fa8f99b0-lock\") pod \"swift-storage-0\" (UID: \"694f3a22-38d9-487d-b887-a915fa8f99b0\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:10:24 crc kubenswrapper[5052]: E1203 12:10:24.035676 5052 projected.go:288] Couldn't get configMap openstack-kuttl-tests/swift-ring-files: configmap "swift-ring-files" not found Dec 03 12:10:24 crc kubenswrapper[5052]: E1203 12:10:24.035699 5052 projected.go:194] Error preparing data for projected volume etc-swift for pod openstack-kuttl-tests/swift-storage-0: configmap "swift-ring-files" not found Dec 03 12:10:24 crc kubenswrapper[5052]: E1203 12:10:24.035748 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/694f3a22-38d9-487d-b887-a915fa8f99b0-etc-swift podName:694f3a22-38d9-487d-b887-a915fa8f99b0 nodeName:}" failed. No retries permitted until 2025-12-03 12:10:24.535727485 +0000 UTC m=+3015.123969249 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "etc-swift" (UniqueName: "kubernetes.io/projected/694f3a22-38d9-487d-b887-a915fa8f99b0-etc-swift") pod "swift-storage-0" (UID: "694f3a22-38d9-487d-b887-a915fa8f99b0") : configmap "swift-ring-files" not found Dec 03 12:10:24 crc kubenswrapper[5052]: I1203 12:10:24.036833 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage09-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage09-crc\") pod \"swift-storage-0\" (UID: \"694f3a22-38d9-487d-b887-a915fa8f99b0\") device mount path \"/mnt/openstack/pv09\"" pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:10:24 crc kubenswrapper[5052]: I1203 12:10:24.041548 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"lock\" (UniqueName: \"kubernetes.io/empty-dir/694f3a22-38d9-487d-b887-a915fa8f99b0-lock\") pod \"swift-storage-0\" (UID: \"694f3a22-38d9-487d-b887-a915fa8f99b0\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:10:24 crc kubenswrapper[5052]: I1203 12:10:24.043369 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"cache\" (UniqueName: \"kubernetes.io/empty-dir/694f3a22-38d9-487d-b887-a915fa8f99b0-cache\") pod \"swift-storage-0\" (UID: \"694f3a22-38d9-487d-b887-a915fa8f99b0\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:10:24 crc kubenswrapper[5052]: I1203 12:10:24.066925 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage09-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage09-crc\") pod \"swift-storage-0\" (UID: \"694f3a22-38d9-487d-b887-a915fa8f99b0\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:10:24 crc kubenswrapper[5052]: I1203 12:10:24.069546 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-dbgxt\" (UniqueName: \"kubernetes.io/projected/694f3a22-38d9-487d-b887-a915fa8f99b0-kube-api-access-dbgxt\") pod \"swift-storage-0\" (UID: \"694f3a22-38d9-487d-b887-a915fa8f99b0\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:10:24 crc kubenswrapper[5052]: I1203 12:10:24.545872 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/694f3a22-38d9-487d-b887-a915fa8f99b0-etc-swift\") pod \"swift-storage-0\" (UID: \"694f3a22-38d9-487d-b887-a915fa8f99b0\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:10:24 crc kubenswrapper[5052]: E1203 12:10:24.546157 5052 projected.go:288] Couldn't get configMap openstack-kuttl-tests/swift-ring-files: configmap "swift-ring-files" not found Dec 03 12:10:24 crc kubenswrapper[5052]: E1203 12:10:24.546196 5052 projected.go:194] Error preparing data for projected volume etc-swift for pod openstack-kuttl-tests/swift-storage-0: configmap "swift-ring-files" not found Dec 03 12:10:24 crc kubenswrapper[5052]: E1203 12:10:24.546295 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/694f3a22-38d9-487d-b887-a915fa8f99b0-etc-swift podName:694f3a22-38d9-487d-b887-a915fa8f99b0 nodeName:}" failed. No retries permitted until 2025-12-03 12:10:25.546261922 +0000 UTC m=+3016.134503846 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "etc-swift" (UniqueName: "kubernetes.io/projected/694f3a22-38d9-487d-b887-a915fa8f99b0-etc-swift") pod "swift-storage-0" (UID: "694f3a22-38d9-487d-b887-a915fa8f99b0") : configmap "swift-ring-files" not found Dec 03 12:10:24 crc kubenswrapper[5052]: I1203 12:10:24.795743 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:10:24 crc kubenswrapper[5052]: I1203 12:10:24.834211 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:10:24 crc kubenswrapper[5052]: I1203 12:10:24.936445 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:10:25 crc kubenswrapper[5052]: I1203 12:10:25.092104 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ovn-northd-0"] Dec 03 12:10:25 crc kubenswrapper[5052]: I1203 12:10:25.093930 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:10:25 crc kubenswrapper[5052]: I1203 12:10:25.095631 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ovnnorthd-ovnnorthd-dockercfg-wd6kp" Dec 03 12:10:25 crc kubenswrapper[5052]: I1203 12:10:25.096700 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-ovnnorthd-ovndbs" Dec 03 12:10:25 crc kubenswrapper[5052]: I1203 12:10:25.098181 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"ovnnorthd-scripts" Dec 03 12:10:25 crc kubenswrapper[5052]: I1203 12:10:25.101595 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"ovnnorthd-config" Dec 03 12:10:25 crc kubenswrapper[5052]: I1203 12:10:25.108457 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ovn-northd-0"] Dec 03 12:10:25 crc kubenswrapper[5052]: I1203 12:10:25.267102 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/00ab3478-6308-427a-8e34-4133611a90b2-combined-ca-bundle\") pod \"ovn-northd-0\" (UID: \"00ab3478-6308-427a-8e34-4133611a90b2\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:10:25 crc kubenswrapper[5052]: I1203 12:10:25.267147 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-rundir\" (UniqueName: \"kubernetes.io/empty-dir/00ab3478-6308-427a-8e34-4133611a90b2-ovn-rundir\") pod \"ovn-northd-0\" (UID: \"00ab3478-6308-427a-8e34-4133611a90b2\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:10:25 crc kubenswrapper[5052]: I1203 12:10:25.267166 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/00ab3478-6308-427a-8e34-4133611a90b2-config\") pod \"ovn-northd-0\" (UID: \"00ab3478-6308-427a-8e34-4133611a90b2\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:10:25 crc kubenswrapper[5052]: I1203 12:10:25.267216 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-24whk\" (UniqueName: \"kubernetes.io/projected/00ab3478-6308-427a-8e34-4133611a90b2-kube-api-access-24whk\") pod \"ovn-northd-0\" (UID: \"00ab3478-6308-427a-8e34-4133611a90b2\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:10:25 crc kubenswrapper[5052]: I1203 12:10:25.267238 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/00ab3478-6308-427a-8e34-4133611a90b2-scripts\") pod \"ovn-northd-0\" (UID: \"00ab3478-6308-427a-8e34-4133611a90b2\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:10:25 crc kubenswrapper[5052]: I1203 12:10:25.267258 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-northd-tls-certs\" (UniqueName: \"kubernetes.io/secret/00ab3478-6308-427a-8e34-4133611a90b2-ovn-northd-tls-certs\") pod \"ovn-northd-0\" (UID: \"00ab3478-6308-427a-8e34-4133611a90b2\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:10:25 crc kubenswrapper[5052]: I1203 12:10:25.267285 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/00ab3478-6308-427a-8e34-4133611a90b2-metrics-certs-tls-certs\") pod \"ovn-northd-0\" (UID: \"00ab3478-6308-427a-8e34-4133611a90b2\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:10:25 crc kubenswrapper[5052]: I1203 12:10:25.368689 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/00ab3478-6308-427a-8e34-4133611a90b2-combined-ca-bundle\") pod \"ovn-northd-0\" (UID: \"00ab3478-6308-427a-8e34-4133611a90b2\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:10:25 crc kubenswrapper[5052]: I1203 12:10:25.368742 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovn-rundir\" (UniqueName: \"kubernetes.io/empty-dir/00ab3478-6308-427a-8e34-4133611a90b2-ovn-rundir\") pod \"ovn-northd-0\" (UID: \"00ab3478-6308-427a-8e34-4133611a90b2\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:10:25 crc kubenswrapper[5052]: I1203 12:10:25.368764 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/00ab3478-6308-427a-8e34-4133611a90b2-config\") pod \"ovn-northd-0\" (UID: \"00ab3478-6308-427a-8e34-4133611a90b2\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:10:25 crc kubenswrapper[5052]: I1203 12:10:25.368815 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-24whk\" (UniqueName: \"kubernetes.io/projected/00ab3478-6308-427a-8e34-4133611a90b2-kube-api-access-24whk\") pod \"ovn-northd-0\" (UID: \"00ab3478-6308-427a-8e34-4133611a90b2\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:10:25 crc kubenswrapper[5052]: I1203 12:10:25.368839 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/00ab3478-6308-427a-8e34-4133611a90b2-scripts\") pod \"ovn-northd-0\" (UID: \"00ab3478-6308-427a-8e34-4133611a90b2\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:10:25 crc kubenswrapper[5052]: I1203 12:10:25.368860 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovn-northd-tls-certs\" (UniqueName: \"kubernetes.io/secret/00ab3478-6308-427a-8e34-4133611a90b2-ovn-northd-tls-certs\") pod \"ovn-northd-0\" (UID: \"00ab3478-6308-427a-8e34-4133611a90b2\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:10:25 crc kubenswrapper[5052]: I1203 12:10:25.368886 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/00ab3478-6308-427a-8e34-4133611a90b2-metrics-certs-tls-certs\") pod \"ovn-northd-0\" (UID: \"00ab3478-6308-427a-8e34-4133611a90b2\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:10:25 crc kubenswrapper[5052]: I1203 12:10:25.369367 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovn-rundir\" (UniqueName: \"kubernetes.io/empty-dir/00ab3478-6308-427a-8e34-4133611a90b2-ovn-rundir\") pod \"ovn-northd-0\" (UID: \"00ab3478-6308-427a-8e34-4133611a90b2\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:10:25 crc kubenswrapper[5052]: I1203 12:10:25.369933 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/00ab3478-6308-427a-8e34-4133611a90b2-scripts\") pod \"ovn-northd-0\" (UID: \"00ab3478-6308-427a-8e34-4133611a90b2\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:10:25 crc kubenswrapper[5052]: I1203 12:10:25.369975 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/00ab3478-6308-427a-8e34-4133611a90b2-config\") pod \"ovn-northd-0\" (UID: \"00ab3478-6308-427a-8e34-4133611a90b2\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:10:25 crc kubenswrapper[5052]: I1203 12:10:25.377538 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/00ab3478-6308-427a-8e34-4133611a90b2-combined-ca-bundle\") pod \"ovn-northd-0\" (UID: \"00ab3478-6308-427a-8e34-4133611a90b2\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:10:25 crc kubenswrapper[5052]: I1203 12:10:25.377911 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovn-northd-tls-certs\" (UniqueName: \"kubernetes.io/secret/00ab3478-6308-427a-8e34-4133611a90b2-ovn-northd-tls-certs\") pod \"ovn-northd-0\" (UID: \"00ab3478-6308-427a-8e34-4133611a90b2\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:10:25 crc kubenswrapper[5052]: I1203 12:10:25.381202 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/00ab3478-6308-427a-8e34-4133611a90b2-metrics-certs-tls-certs\") pod \"ovn-northd-0\" (UID: \"00ab3478-6308-427a-8e34-4133611a90b2\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:10:25 crc kubenswrapper[5052]: I1203 12:10:25.426127 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-24whk\" (UniqueName: \"kubernetes.io/projected/00ab3478-6308-427a-8e34-4133611a90b2-kube-api-access-24whk\") pod \"ovn-northd-0\" (UID: \"00ab3478-6308-427a-8e34-4133611a90b2\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:10:25 crc kubenswrapper[5052]: I1203 12:10:25.572261 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/694f3a22-38d9-487d-b887-a915fa8f99b0-etc-swift\") pod \"swift-storage-0\" (UID: \"694f3a22-38d9-487d-b887-a915fa8f99b0\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:10:25 crc kubenswrapper[5052]: E1203 12:10:25.572456 5052 projected.go:288] Couldn't get configMap openstack-kuttl-tests/swift-ring-files: configmap "swift-ring-files" not found Dec 03 12:10:25 crc kubenswrapper[5052]: E1203 12:10:25.572485 5052 projected.go:194] Error preparing data for projected volume etc-swift for pod openstack-kuttl-tests/swift-storage-0: configmap "swift-ring-files" not found Dec 03 12:10:25 crc kubenswrapper[5052]: E1203 12:10:25.572578 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/694f3a22-38d9-487d-b887-a915fa8f99b0-etc-swift podName:694f3a22-38d9-487d-b887-a915fa8f99b0 nodeName:}" failed. No retries permitted until 2025-12-03 12:10:27.572556726 +0000 UTC m=+3018.160798490 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "etc-swift" (UniqueName: "kubernetes.io/projected/694f3a22-38d9-487d-b887-a915fa8f99b0-etc-swift") pod "swift-storage-0" (UID: "694f3a22-38d9-487d-b887-a915fa8f99b0") : configmap "swift-ring-files" not found Dec 03 12:10:25 crc kubenswrapper[5052]: I1203 12:10:25.717107 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:10:26 crc kubenswrapper[5052]: I1203 12:10:26.163939 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ovn-northd-0"] Dec 03 12:10:26 crc kubenswrapper[5052]: W1203 12:10:26.165617 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod00ab3478_6308_427a_8e34_4133611a90b2.slice/crio-78e693d6cab6fe2b4a54716cf3cf7a78b81ff4948a0beca6b155cf2bcbabfa2d WatchSource:0}: Error finding container 78e693d6cab6fe2b4a54716cf3cf7a78b81ff4948a0beca6b155cf2bcbabfa2d: Status 404 returned error can't find the container with id 78e693d6cab6fe2b4a54716cf3cf7a78b81ff4948a0beca6b155cf2bcbabfa2d Dec 03 12:10:26 crc kubenswrapper[5052]: I1203 12:10:26.523241 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:10:26 crc kubenswrapper[5052]: I1203 12:10:26.627038 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:10:26 crc kubenswrapper[5052]: I1203 12:10:26.915483 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovn-northd-0" event={"ID":"00ab3478-6308-427a-8e34-4133611a90b2","Type":"ContainerStarted","Data":"b77011f57a038af73fce8dd2b2cefedfa4b2e4023e9b43da72366d20be9344f4"} Dec 03 12:10:26 crc kubenswrapper[5052]: I1203 12:10:26.915798 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovn-northd-0" event={"ID":"00ab3478-6308-427a-8e34-4133611a90b2","Type":"ContainerStarted","Data":"98d14ce0cb9919bb83964fb391e45a00edf1d049296adef59b4e2e0f042c63d2"} Dec 03 12:10:26 crc kubenswrapper[5052]: I1203 12:10:26.915813 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovn-northd-0" event={"ID":"00ab3478-6308-427a-8e34-4133611a90b2","Type":"ContainerStarted","Data":"78e693d6cab6fe2b4a54716cf3cf7a78b81ff4948a0beca6b155cf2bcbabfa2d"} Dec 03 12:10:26 crc kubenswrapper[5052]: I1203 12:10:26.938390 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/ovn-northd-0" podStartSLOduration=1.938367794 podStartE2EDuration="1.938367794s" podCreationTimestamp="2025-12-03 12:10:25 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:10:26.934811842 +0000 UTC m=+3017.523053606" watchObservedRunningTime="2025-12-03 12:10:26.938367794 +0000 UTC m=+3017.526609558" Dec 03 12:10:27 crc kubenswrapper[5052]: I1203 12:10:27.611294 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/694f3a22-38d9-487d-b887-a915fa8f99b0-etc-swift\") pod \"swift-storage-0\" (UID: \"694f3a22-38d9-487d-b887-a915fa8f99b0\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:10:27 crc kubenswrapper[5052]: E1203 12:10:27.611491 5052 projected.go:288] Couldn't get configMap openstack-kuttl-tests/swift-ring-files: configmap "swift-ring-files" not found Dec 03 12:10:27 crc kubenswrapper[5052]: E1203 12:10:27.611527 5052 projected.go:194] Error preparing data for projected volume etc-swift for pod openstack-kuttl-tests/swift-storage-0: configmap "swift-ring-files" not found Dec 03 12:10:27 crc kubenswrapper[5052]: E1203 12:10:27.611592 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/694f3a22-38d9-487d-b887-a915fa8f99b0-etc-swift podName:694f3a22-38d9-487d-b887-a915fa8f99b0 nodeName:}" failed. No retries permitted until 2025-12-03 12:10:31.611569435 +0000 UTC m=+3022.199811199 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "etc-swift" (UniqueName: "kubernetes.io/projected/694f3a22-38d9-487d-b887-a915fa8f99b0-etc-swift") pod "swift-storage-0" (UID: "694f3a22-38d9-487d-b887-a915fa8f99b0") : configmap "swift-ring-files" not found Dec 03 12:10:27 crc kubenswrapper[5052]: I1203 12:10:27.685980 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/swift-ring-rebalance-9pfsn"] Dec 03 12:10:27 crc kubenswrapper[5052]: I1203 12:10:27.707945 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-ring-rebalance-9pfsn" Dec 03 12:10:27 crc kubenswrapper[5052]: I1203 12:10:27.716534 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"swift-proxy-config-data" Dec 03 12:10:27 crc kubenswrapper[5052]: I1203 12:10:27.716721 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"swift-ring-config-data" Dec 03 12:10:27 crc kubenswrapper[5052]: I1203 12:10:27.716774 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"swift-ring-scripts" Dec 03 12:10:27 crc kubenswrapper[5052]: I1203 12:10:27.717225 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/swift-ring-rebalance-9pfsn"] Dec 03 12:10:27 crc kubenswrapper[5052]: I1203 12:10:27.747023 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/swift-ring-rebalance-9pfsn"] Dec 03 12:10:27 crc kubenswrapper[5052]: E1203 12:10:27.749452 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[combined-ca-bundle dispersionconf etc-swift kube-api-access-k9ffr ring-data-devices scripts swiftconf], unattached volumes=[], failed to process volumes=[]: context canceled" pod="openstack-kuttl-tests/swift-ring-rebalance-9pfsn" podUID="af7a21ca-6e92-4a09-9899-e520d42921b4" Dec 03 12:10:27 crc kubenswrapper[5052]: I1203 12:10:27.765147 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/swift-ring-rebalance-zqqdm"] Dec 03 12:10:27 crc kubenswrapper[5052]: I1203 12:10:27.766479 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-ring-rebalance-zqqdm" Dec 03 12:10:27 crc kubenswrapper[5052]: I1203 12:10:27.783869 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/swift-ring-rebalance-zqqdm"] Dec 03 12:10:27 crc kubenswrapper[5052]: I1203 12:10:27.818689 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k9ffr\" (UniqueName: \"kubernetes.io/projected/af7a21ca-6e92-4a09-9899-e520d42921b4-kube-api-access-k9ffr\") pod \"swift-ring-rebalance-9pfsn\" (UID: \"af7a21ca-6e92-4a09-9899-e520d42921b4\") " pod="openstack-kuttl-tests/swift-ring-rebalance-9pfsn" Dec 03 12:10:27 crc kubenswrapper[5052]: I1203 12:10:27.818769 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ring-data-devices\" (UniqueName: \"kubernetes.io/configmap/af7a21ca-6e92-4a09-9899-e520d42921b4-ring-data-devices\") pod \"swift-ring-rebalance-9pfsn\" (UID: \"af7a21ca-6e92-4a09-9899-e520d42921b4\") " pod="openstack-kuttl-tests/swift-ring-rebalance-9pfsn" Dec 03 12:10:27 crc kubenswrapper[5052]: I1203 12:10:27.818792 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dispersionconf\" (UniqueName: \"kubernetes.io/secret/af7a21ca-6e92-4a09-9899-e520d42921b4-dispersionconf\") pod \"swift-ring-rebalance-9pfsn\" (UID: \"af7a21ca-6e92-4a09-9899-e520d42921b4\") " pod="openstack-kuttl-tests/swift-ring-rebalance-9pfsn" Dec 03 12:10:27 crc kubenswrapper[5052]: I1203 12:10:27.818924 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/af7a21ca-6e92-4a09-9899-e520d42921b4-combined-ca-bundle\") pod \"swift-ring-rebalance-9pfsn\" (UID: \"af7a21ca-6e92-4a09-9899-e520d42921b4\") " pod="openstack-kuttl-tests/swift-ring-rebalance-9pfsn" Dec 03 12:10:27 crc kubenswrapper[5052]: I1203 12:10:27.818984 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/af7a21ca-6e92-4a09-9899-e520d42921b4-scripts\") pod \"swift-ring-rebalance-9pfsn\" (UID: \"af7a21ca-6e92-4a09-9899-e520d42921b4\") " pod="openstack-kuttl-tests/swift-ring-rebalance-9pfsn" Dec 03 12:10:27 crc kubenswrapper[5052]: I1203 12:10:27.819148 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"swiftconf\" (UniqueName: \"kubernetes.io/secret/af7a21ca-6e92-4a09-9899-e520d42921b4-swiftconf\") pod \"swift-ring-rebalance-9pfsn\" (UID: \"af7a21ca-6e92-4a09-9899-e520d42921b4\") " pod="openstack-kuttl-tests/swift-ring-rebalance-9pfsn" Dec 03 12:10:27 crc kubenswrapper[5052]: I1203 12:10:27.819208 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/empty-dir/af7a21ca-6e92-4a09-9899-e520d42921b4-etc-swift\") pod \"swift-ring-rebalance-9pfsn\" (UID: \"af7a21ca-6e92-4a09-9899-e520d42921b4\") " pod="openstack-kuttl-tests/swift-ring-rebalance-9pfsn" Dec 03 12:10:27 crc kubenswrapper[5052]: I1203 12:10:27.920520 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/855f1c2d-3e1c-44b1-a691-41673832e6bc-scripts\") pod \"swift-ring-rebalance-zqqdm\" (UID: \"855f1c2d-3e1c-44b1-a691-41673832e6bc\") " pod="openstack-kuttl-tests/swift-ring-rebalance-zqqdm" Dec 03 12:10:27 crc kubenswrapper[5052]: I1203 12:10:27.920617 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"swiftconf\" (UniqueName: \"kubernetes.io/secret/855f1c2d-3e1c-44b1-a691-41673832e6bc-swiftconf\") pod \"swift-ring-rebalance-zqqdm\" (UID: \"855f1c2d-3e1c-44b1-a691-41673832e6bc\") " pod="openstack-kuttl-tests/swift-ring-rebalance-zqqdm" Dec 03 12:10:27 crc kubenswrapper[5052]: I1203 12:10:27.920654 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ring-data-devices\" (UniqueName: \"kubernetes.io/configmap/855f1c2d-3e1c-44b1-a691-41673832e6bc-ring-data-devices\") pod \"swift-ring-rebalance-zqqdm\" (UID: \"855f1c2d-3e1c-44b1-a691-41673832e6bc\") " pod="openstack-kuttl-tests/swift-ring-rebalance-zqqdm" Dec 03 12:10:27 crc kubenswrapper[5052]: I1203 12:10:27.920703 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ring-data-devices\" (UniqueName: \"kubernetes.io/configmap/af7a21ca-6e92-4a09-9899-e520d42921b4-ring-data-devices\") pod \"swift-ring-rebalance-9pfsn\" (UID: \"af7a21ca-6e92-4a09-9899-e520d42921b4\") " pod="openstack-kuttl-tests/swift-ring-rebalance-9pfsn" Dec 03 12:10:27 crc kubenswrapper[5052]: I1203 12:10:27.920736 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"dispersionconf\" (UniqueName: \"kubernetes.io/secret/af7a21ca-6e92-4a09-9899-e520d42921b4-dispersionconf\") pod \"swift-ring-rebalance-9pfsn\" (UID: \"af7a21ca-6e92-4a09-9899-e520d42921b4\") " pod="openstack-kuttl-tests/swift-ring-rebalance-9pfsn" Dec 03 12:10:27 crc kubenswrapper[5052]: I1203 12:10:27.920772 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/af7a21ca-6e92-4a09-9899-e520d42921b4-combined-ca-bundle\") pod \"swift-ring-rebalance-9pfsn\" (UID: \"af7a21ca-6e92-4a09-9899-e520d42921b4\") " pod="openstack-kuttl-tests/swift-ring-rebalance-9pfsn" Dec 03 12:10:27 crc kubenswrapper[5052]: I1203 12:10:27.920804 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/af7a21ca-6e92-4a09-9899-e520d42921b4-scripts\") pod \"swift-ring-rebalance-9pfsn\" (UID: \"af7a21ca-6e92-4a09-9899-e520d42921b4\") " pod="openstack-kuttl-tests/swift-ring-rebalance-9pfsn" Dec 03 12:10:27 crc kubenswrapper[5052]: I1203 12:10:27.920832 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dispersionconf\" (UniqueName: \"kubernetes.io/secret/855f1c2d-3e1c-44b1-a691-41673832e6bc-dispersionconf\") pod \"swift-ring-rebalance-zqqdm\" (UID: \"855f1c2d-3e1c-44b1-a691-41673832e6bc\") " pod="openstack-kuttl-tests/swift-ring-rebalance-zqqdm" Dec 03 12:10:27 crc kubenswrapper[5052]: I1203 12:10:27.920862 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z4gb7\" (UniqueName: \"kubernetes.io/projected/855f1c2d-3e1c-44b1-a691-41673832e6bc-kube-api-access-z4gb7\") pod \"swift-ring-rebalance-zqqdm\" (UID: \"855f1c2d-3e1c-44b1-a691-41673832e6bc\") " pod="openstack-kuttl-tests/swift-ring-rebalance-zqqdm" Dec 03 12:10:27 crc kubenswrapper[5052]: I1203 12:10:27.921007 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/855f1c2d-3e1c-44b1-a691-41673832e6bc-combined-ca-bundle\") pod \"swift-ring-rebalance-zqqdm\" (UID: \"855f1c2d-3e1c-44b1-a691-41673832e6bc\") " pod="openstack-kuttl-tests/swift-ring-rebalance-zqqdm" Dec 03 12:10:27 crc kubenswrapper[5052]: I1203 12:10:27.921065 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"swiftconf\" (UniqueName: \"kubernetes.io/secret/af7a21ca-6e92-4a09-9899-e520d42921b4-swiftconf\") pod \"swift-ring-rebalance-9pfsn\" (UID: \"af7a21ca-6e92-4a09-9899-e520d42921b4\") " pod="openstack-kuttl-tests/swift-ring-rebalance-9pfsn" Dec 03 12:10:27 crc kubenswrapper[5052]: I1203 12:10:27.921141 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/empty-dir/af7a21ca-6e92-4a09-9899-e520d42921b4-etc-swift\") pod \"swift-ring-rebalance-9pfsn\" (UID: \"af7a21ca-6e92-4a09-9899-e520d42921b4\") " pod="openstack-kuttl-tests/swift-ring-rebalance-9pfsn" Dec 03 12:10:27 crc kubenswrapper[5052]: I1203 12:10:27.921218 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/empty-dir/855f1c2d-3e1c-44b1-a691-41673832e6bc-etc-swift\") pod \"swift-ring-rebalance-zqqdm\" (UID: \"855f1c2d-3e1c-44b1-a691-41673832e6bc\") " pod="openstack-kuttl-tests/swift-ring-rebalance-zqqdm" Dec 03 12:10:27 crc kubenswrapper[5052]: I1203 12:10:27.921267 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-k9ffr\" (UniqueName: \"kubernetes.io/projected/af7a21ca-6e92-4a09-9899-e520d42921b4-kube-api-access-k9ffr\") pod \"swift-ring-rebalance-9pfsn\" (UID: \"af7a21ca-6e92-4a09-9899-e520d42921b4\") " pod="openstack-kuttl-tests/swift-ring-rebalance-9pfsn" Dec 03 12:10:27 crc kubenswrapper[5052]: I1203 12:10:27.922303 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ring-data-devices\" (UniqueName: \"kubernetes.io/configmap/af7a21ca-6e92-4a09-9899-e520d42921b4-ring-data-devices\") pod \"swift-ring-rebalance-9pfsn\" (UID: \"af7a21ca-6e92-4a09-9899-e520d42921b4\") " pod="openstack-kuttl-tests/swift-ring-rebalance-9pfsn" Dec 03 12:10:27 crc kubenswrapper[5052]: I1203 12:10:27.922725 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"etc-swift\" (UniqueName: \"kubernetes.io/empty-dir/af7a21ca-6e92-4a09-9899-e520d42921b4-etc-swift\") pod \"swift-ring-rebalance-9pfsn\" (UID: \"af7a21ca-6e92-4a09-9899-e520d42921b4\") " pod="openstack-kuttl-tests/swift-ring-rebalance-9pfsn" Dec 03 12:10:27 crc kubenswrapper[5052]: I1203 12:10:27.922859 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-ring-rebalance-9pfsn" Dec 03 12:10:27 crc kubenswrapper[5052]: I1203 12:10:27.921773 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/af7a21ca-6e92-4a09-9899-e520d42921b4-scripts\") pod \"swift-ring-rebalance-9pfsn\" (UID: \"af7a21ca-6e92-4a09-9899-e520d42921b4\") " pod="openstack-kuttl-tests/swift-ring-rebalance-9pfsn" Dec 03 12:10:27 crc kubenswrapper[5052]: I1203 12:10:27.923409 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:10:27 crc kubenswrapper[5052]: I1203 12:10:27.927088 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"swiftconf\" (UniqueName: \"kubernetes.io/secret/af7a21ca-6e92-4a09-9899-e520d42921b4-swiftconf\") pod \"swift-ring-rebalance-9pfsn\" (UID: \"af7a21ca-6e92-4a09-9899-e520d42921b4\") " pod="openstack-kuttl-tests/swift-ring-rebalance-9pfsn" Dec 03 12:10:27 crc kubenswrapper[5052]: I1203 12:10:27.929212 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"dispersionconf\" (UniqueName: \"kubernetes.io/secret/af7a21ca-6e92-4a09-9899-e520d42921b4-dispersionconf\") pod \"swift-ring-rebalance-9pfsn\" (UID: \"af7a21ca-6e92-4a09-9899-e520d42921b4\") " pod="openstack-kuttl-tests/swift-ring-rebalance-9pfsn" Dec 03 12:10:27 crc kubenswrapper[5052]: I1203 12:10:27.930667 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/af7a21ca-6e92-4a09-9899-e520d42921b4-combined-ca-bundle\") pod \"swift-ring-rebalance-9pfsn\" (UID: \"af7a21ca-6e92-4a09-9899-e520d42921b4\") " pod="openstack-kuttl-tests/swift-ring-rebalance-9pfsn" Dec 03 12:10:27 crc kubenswrapper[5052]: I1203 12:10:27.938267 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-k9ffr\" (UniqueName: \"kubernetes.io/projected/af7a21ca-6e92-4a09-9899-e520d42921b4-kube-api-access-k9ffr\") pod \"swift-ring-rebalance-9pfsn\" (UID: \"af7a21ca-6e92-4a09-9899-e520d42921b4\") " pod="openstack-kuttl-tests/swift-ring-rebalance-9pfsn" Dec 03 12:10:27 crc kubenswrapper[5052]: I1203 12:10:27.994605 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-ring-rebalance-9pfsn" Dec 03 12:10:28 crc kubenswrapper[5052]: I1203 12:10:28.022759 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/empty-dir/855f1c2d-3e1c-44b1-a691-41673832e6bc-etc-swift\") pod \"swift-ring-rebalance-zqqdm\" (UID: \"855f1c2d-3e1c-44b1-a691-41673832e6bc\") " pod="openstack-kuttl-tests/swift-ring-rebalance-zqqdm" Dec 03 12:10:28 crc kubenswrapper[5052]: I1203 12:10:28.022846 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/855f1c2d-3e1c-44b1-a691-41673832e6bc-scripts\") pod \"swift-ring-rebalance-zqqdm\" (UID: \"855f1c2d-3e1c-44b1-a691-41673832e6bc\") " pod="openstack-kuttl-tests/swift-ring-rebalance-zqqdm" Dec 03 12:10:28 crc kubenswrapper[5052]: I1203 12:10:28.022869 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"swiftconf\" (UniqueName: \"kubernetes.io/secret/855f1c2d-3e1c-44b1-a691-41673832e6bc-swiftconf\") pod \"swift-ring-rebalance-zqqdm\" (UID: \"855f1c2d-3e1c-44b1-a691-41673832e6bc\") " pod="openstack-kuttl-tests/swift-ring-rebalance-zqqdm" Dec 03 12:10:28 crc kubenswrapper[5052]: I1203 12:10:28.022892 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ring-data-devices\" (UniqueName: \"kubernetes.io/configmap/855f1c2d-3e1c-44b1-a691-41673832e6bc-ring-data-devices\") pod \"swift-ring-rebalance-zqqdm\" (UID: \"855f1c2d-3e1c-44b1-a691-41673832e6bc\") " pod="openstack-kuttl-tests/swift-ring-rebalance-zqqdm" Dec 03 12:10:28 crc kubenswrapper[5052]: I1203 12:10:28.022952 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"dispersionconf\" (UniqueName: \"kubernetes.io/secret/855f1c2d-3e1c-44b1-a691-41673832e6bc-dispersionconf\") pod \"swift-ring-rebalance-zqqdm\" (UID: \"855f1c2d-3e1c-44b1-a691-41673832e6bc\") " pod="openstack-kuttl-tests/swift-ring-rebalance-zqqdm" Dec 03 12:10:28 crc kubenswrapper[5052]: I1203 12:10:28.022981 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-z4gb7\" (UniqueName: \"kubernetes.io/projected/855f1c2d-3e1c-44b1-a691-41673832e6bc-kube-api-access-z4gb7\") pod \"swift-ring-rebalance-zqqdm\" (UID: \"855f1c2d-3e1c-44b1-a691-41673832e6bc\") " pod="openstack-kuttl-tests/swift-ring-rebalance-zqqdm" Dec 03 12:10:28 crc kubenswrapper[5052]: I1203 12:10:28.023066 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/855f1c2d-3e1c-44b1-a691-41673832e6bc-combined-ca-bundle\") pod \"swift-ring-rebalance-zqqdm\" (UID: \"855f1c2d-3e1c-44b1-a691-41673832e6bc\") " pod="openstack-kuttl-tests/swift-ring-rebalance-zqqdm" Dec 03 12:10:28 crc kubenswrapper[5052]: I1203 12:10:28.023890 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"etc-swift\" (UniqueName: \"kubernetes.io/empty-dir/855f1c2d-3e1c-44b1-a691-41673832e6bc-etc-swift\") pod \"swift-ring-rebalance-zqqdm\" (UID: \"855f1c2d-3e1c-44b1-a691-41673832e6bc\") " pod="openstack-kuttl-tests/swift-ring-rebalance-zqqdm" Dec 03 12:10:28 crc kubenswrapper[5052]: I1203 12:10:28.023901 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/855f1c2d-3e1c-44b1-a691-41673832e6bc-scripts\") pod \"swift-ring-rebalance-zqqdm\" (UID: \"855f1c2d-3e1c-44b1-a691-41673832e6bc\") " pod="openstack-kuttl-tests/swift-ring-rebalance-zqqdm" Dec 03 12:10:28 crc kubenswrapper[5052]: I1203 12:10:28.024113 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ring-data-devices\" (UniqueName: \"kubernetes.io/configmap/855f1c2d-3e1c-44b1-a691-41673832e6bc-ring-data-devices\") pod \"swift-ring-rebalance-zqqdm\" (UID: \"855f1c2d-3e1c-44b1-a691-41673832e6bc\") " pod="openstack-kuttl-tests/swift-ring-rebalance-zqqdm" Dec 03 12:10:28 crc kubenswrapper[5052]: I1203 12:10:28.026396 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"swiftconf\" (UniqueName: \"kubernetes.io/secret/855f1c2d-3e1c-44b1-a691-41673832e6bc-swiftconf\") pod \"swift-ring-rebalance-zqqdm\" (UID: \"855f1c2d-3e1c-44b1-a691-41673832e6bc\") " pod="openstack-kuttl-tests/swift-ring-rebalance-zqqdm" Dec 03 12:10:28 crc kubenswrapper[5052]: I1203 12:10:28.032122 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/855f1c2d-3e1c-44b1-a691-41673832e6bc-combined-ca-bundle\") pod \"swift-ring-rebalance-zqqdm\" (UID: \"855f1c2d-3e1c-44b1-a691-41673832e6bc\") " pod="openstack-kuttl-tests/swift-ring-rebalance-zqqdm" Dec 03 12:10:28 crc kubenswrapper[5052]: I1203 12:10:28.032923 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"dispersionconf\" (UniqueName: \"kubernetes.io/secret/855f1c2d-3e1c-44b1-a691-41673832e6bc-dispersionconf\") pod \"swift-ring-rebalance-zqqdm\" (UID: \"855f1c2d-3e1c-44b1-a691-41673832e6bc\") " pod="openstack-kuttl-tests/swift-ring-rebalance-zqqdm" Dec 03 12:10:28 crc kubenswrapper[5052]: I1203 12:10:28.040385 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-z4gb7\" (UniqueName: \"kubernetes.io/projected/855f1c2d-3e1c-44b1-a691-41673832e6bc-kube-api-access-z4gb7\") pod \"swift-ring-rebalance-zqqdm\" (UID: \"855f1c2d-3e1c-44b1-a691-41673832e6bc\") " pod="openstack-kuttl-tests/swift-ring-rebalance-zqqdm" Dec 03 12:10:28 crc kubenswrapper[5052]: I1203 12:10:28.086677 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-ring-rebalance-zqqdm" Dec 03 12:10:28 crc kubenswrapper[5052]: I1203 12:10:28.124292 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"swiftconf\" (UniqueName: \"kubernetes.io/secret/af7a21ca-6e92-4a09-9899-e520d42921b4-swiftconf\") pod \"af7a21ca-6e92-4a09-9899-e520d42921b4\" (UID: \"af7a21ca-6e92-4a09-9899-e520d42921b4\") " Dec 03 12:10:28 crc kubenswrapper[5052]: I1203 12:10:28.124643 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"dispersionconf\" (UniqueName: \"kubernetes.io/secret/af7a21ca-6e92-4a09-9899-e520d42921b4-dispersionconf\") pod \"af7a21ca-6e92-4a09-9899-e520d42921b4\" (UID: \"af7a21ca-6e92-4a09-9899-e520d42921b4\") " Dec 03 12:10:28 crc kubenswrapper[5052]: I1203 12:10:28.124687 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/af7a21ca-6e92-4a09-9899-e520d42921b4-combined-ca-bundle\") pod \"af7a21ca-6e92-4a09-9899-e520d42921b4\" (UID: \"af7a21ca-6e92-4a09-9899-e520d42921b4\") " Dec 03 12:10:28 crc kubenswrapper[5052]: I1203 12:10:28.124709 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/af7a21ca-6e92-4a09-9899-e520d42921b4-scripts\") pod \"af7a21ca-6e92-4a09-9899-e520d42921b4\" (UID: \"af7a21ca-6e92-4a09-9899-e520d42921b4\") " Dec 03 12:10:28 crc kubenswrapper[5052]: I1203 12:10:28.124734 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/empty-dir/af7a21ca-6e92-4a09-9899-e520d42921b4-etc-swift\") pod \"af7a21ca-6e92-4a09-9899-e520d42921b4\" (UID: \"af7a21ca-6e92-4a09-9899-e520d42921b4\") " Dec 03 12:10:28 crc kubenswrapper[5052]: I1203 12:10:28.124759 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k9ffr\" (UniqueName: \"kubernetes.io/projected/af7a21ca-6e92-4a09-9899-e520d42921b4-kube-api-access-k9ffr\") pod \"af7a21ca-6e92-4a09-9899-e520d42921b4\" (UID: \"af7a21ca-6e92-4a09-9899-e520d42921b4\") " Dec 03 12:10:28 crc kubenswrapper[5052]: I1203 12:10:28.124886 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ring-data-devices\" (UniqueName: \"kubernetes.io/configmap/af7a21ca-6e92-4a09-9899-e520d42921b4-ring-data-devices\") pod \"af7a21ca-6e92-4a09-9899-e520d42921b4\" (UID: \"af7a21ca-6e92-4a09-9899-e520d42921b4\") " Dec 03 12:10:28 crc kubenswrapper[5052]: I1203 12:10:28.125697 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/af7a21ca-6e92-4a09-9899-e520d42921b4-scripts" (OuterVolumeSpecName: "scripts") pod "af7a21ca-6e92-4a09-9899-e520d42921b4" (UID: "af7a21ca-6e92-4a09-9899-e520d42921b4"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:10:28 crc kubenswrapper[5052]: I1203 12:10:28.125989 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/af7a21ca-6e92-4a09-9899-e520d42921b4-etc-swift" (OuterVolumeSpecName: "etc-swift") pod "af7a21ca-6e92-4a09-9899-e520d42921b4" (UID: "af7a21ca-6e92-4a09-9899-e520d42921b4"). InnerVolumeSpecName "etc-swift". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:10:28 crc kubenswrapper[5052]: I1203 12:10:28.126888 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/af7a21ca-6e92-4a09-9899-e520d42921b4-ring-data-devices" (OuterVolumeSpecName: "ring-data-devices") pod "af7a21ca-6e92-4a09-9899-e520d42921b4" (UID: "af7a21ca-6e92-4a09-9899-e520d42921b4"). InnerVolumeSpecName "ring-data-devices". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:10:28 crc kubenswrapper[5052]: I1203 12:10:28.127260 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/af7a21ca-6e92-4a09-9899-e520d42921b4-swiftconf" (OuterVolumeSpecName: "swiftconf") pod "af7a21ca-6e92-4a09-9899-e520d42921b4" (UID: "af7a21ca-6e92-4a09-9899-e520d42921b4"). InnerVolumeSpecName "swiftconf". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:10:28 crc kubenswrapper[5052]: I1203 12:10:28.127992 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/af7a21ca-6e92-4a09-9899-e520d42921b4-dispersionconf" (OuterVolumeSpecName: "dispersionconf") pod "af7a21ca-6e92-4a09-9899-e520d42921b4" (UID: "af7a21ca-6e92-4a09-9899-e520d42921b4"). InnerVolumeSpecName "dispersionconf". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:10:28 crc kubenswrapper[5052]: I1203 12:10:28.128981 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/af7a21ca-6e92-4a09-9899-e520d42921b4-kube-api-access-k9ffr" (OuterVolumeSpecName: "kube-api-access-k9ffr") pod "af7a21ca-6e92-4a09-9899-e520d42921b4" (UID: "af7a21ca-6e92-4a09-9899-e520d42921b4"). InnerVolumeSpecName "kube-api-access-k9ffr". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:10:28 crc kubenswrapper[5052]: I1203 12:10:28.129620 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/af7a21ca-6e92-4a09-9899-e520d42921b4-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "af7a21ca-6e92-4a09-9899-e520d42921b4" (UID: "af7a21ca-6e92-4a09-9899-e520d42921b4"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:10:28 crc kubenswrapper[5052]: I1203 12:10:28.227780 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/af7a21ca-6e92-4a09-9899-e520d42921b4-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:10:28 crc kubenswrapper[5052]: I1203 12:10:28.227814 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/af7a21ca-6e92-4a09-9899-e520d42921b4-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:10:28 crc kubenswrapper[5052]: I1203 12:10:28.227826 5052 reconciler_common.go:293] "Volume detached for volume \"etc-swift\" (UniqueName: \"kubernetes.io/empty-dir/af7a21ca-6e92-4a09-9899-e520d42921b4-etc-swift\") on node \"crc\" DevicePath \"\"" Dec 03 12:10:28 crc kubenswrapper[5052]: I1203 12:10:28.227839 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-k9ffr\" (UniqueName: \"kubernetes.io/projected/af7a21ca-6e92-4a09-9899-e520d42921b4-kube-api-access-k9ffr\") on node \"crc\" DevicePath \"\"" Dec 03 12:10:28 crc kubenswrapper[5052]: I1203 12:10:28.227853 5052 reconciler_common.go:293] "Volume detached for volume \"ring-data-devices\" (UniqueName: \"kubernetes.io/configmap/af7a21ca-6e92-4a09-9899-e520d42921b4-ring-data-devices\") on node \"crc\" DevicePath \"\"" Dec 03 12:10:28 crc kubenswrapper[5052]: I1203 12:10:28.227865 5052 reconciler_common.go:293] "Volume detached for volume \"swiftconf\" (UniqueName: \"kubernetes.io/secret/af7a21ca-6e92-4a09-9899-e520d42921b4-swiftconf\") on node \"crc\" DevicePath \"\"" Dec 03 12:10:28 crc kubenswrapper[5052]: I1203 12:10:28.227876 5052 reconciler_common.go:293] "Volume detached for volume \"dispersionconf\" (UniqueName: \"kubernetes.io/secret/af7a21ca-6e92-4a09-9899-e520d42921b4-dispersionconf\") on node \"crc\" DevicePath \"\"" Dec 03 12:10:28 crc kubenswrapper[5052]: I1203 12:10:28.524901 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/swift-ring-rebalance-zqqdm"] Dec 03 12:10:28 crc kubenswrapper[5052]: I1203 12:10:28.635998 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:10:28 crc kubenswrapper[5052]: I1203 12:10:28.755669 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:10:28 crc kubenswrapper[5052]: I1203 12:10:28.950195 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-ring-rebalance-zqqdm" event={"ID":"855f1c2d-3e1c-44b1-a691-41673832e6bc","Type":"ContainerStarted","Data":"c9905267b6558bc3332728a3891c7daa7a01ebb18411fc152bcfc514ff6b7b6e"} Dec 03 12:10:28 crc kubenswrapper[5052]: I1203 12:10:28.950235 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-ring-rebalance-zqqdm" event={"ID":"855f1c2d-3e1c-44b1-a691-41673832e6bc","Type":"ContainerStarted","Data":"4902d4ad52d256c58ec77050171ae59fcff1a8ef50fc1fdb5c64acf7adcb43d4"} Dec 03 12:10:28 crc kubenswrapper[5052]: I1203 12:10:28.950276 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-ring-rebalance-9pfsn" Dec 03 12:10:28 crc kubenswrapper[5052]: I1203 12:10:28.983073 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/swift-ring-rebalance-zqqdm" podStartSLOduration=1.983053966 podStartE2EDuration="1.983053966s" podCreationTimestamp="2025-12-03 12:10:27 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:10:28.970804976 +0000 UTC m=+3019.559046740" watchObservedRunningTime="2025-12-03 12:10:28.983053966 +0000 UTC m=+3019.571295730" Dec 03 12:10:29 crc kubenswrapper[5052]: I1203 12:10:29.041754 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/swift-ring-rebalance-9pfsn"] Dec 03 12:10:29 crc kubenswrapper[5052]: I1203 12:10:29.048916 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/swift-ring-rebalance-9pfsn"] Dec 03 12:10:29 crc kubenswrapper[5052]: I1203 12:10:29.684340 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="af7a21ca-6e92-4a09-9899-e520d42921b4" path="/var/lib/kubelet/pods/af7a21ca-6e92-4a09-9899-e520d42921b4/volumes" Dec 03 12:10:30 crc kubenswrapper[5052]: I1203 12:10:30.402806 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/keystone-db-create-6kjm6"] Dec 03 12:10:30 crc kubenswrapper[5052]: I1203 12:10:30.406788 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-db-create-6kjm6" Dec 03 12:10:30 crc kubenswrapper[5052]: I1203 12:10:30.417640 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-db-create-6kjm6"] Dec 03 12:10:30 crc kubenswrapper[5052]: I1203 12:10:30.467143 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/9e1f00b4-e4b8-4696-aacc-92f6b67f99f4-operator-scripts\") pod \"keystone-db-create-6kjm6\" (UID: \"9e1f00b4-e4b8-4696-aacc-92f6b67f99f4\") " pod="openstack-kuttl-tests/keystone-db-create-6kjm6" Dec 03 12:10:30 crc kubenswrapper[5052]: I1203 12:10:30.467246 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rm8l2\" (UniqueName: \"kubernetes.io/projected/9e1f00b4-e4b8-4696-aacc-92f6b67f99f4-kube-api-access-rm8l2\") pod \"keystone-db-create-6kjm6\" (UID: \"9e1f00b4-e4b8-4696-aacc-92f6b67f99f4\") " pod="openstack-kuttl-tests/keystone-db-create-6kjm6" Dec 03 12:10:30 crc kubenswrapper[5052]: I1203 12:10:30.538989 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/keystone-1874-account-create-update-7t9ms"] Dec 03 12:10:30 crc kubenswrapper[5052]: I1203 12:10:30.541390 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-1874-account-create-update-7t9ms" Dec 03 12:10:30 crc kubenswrapper[5052]: I1203 12:10:30.544659 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone-db-secret" Dec 03 12:10:30 crc kubenswrapper[5052]: I1203 12:10:30.547876 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-1874-account-create-update-7t9ms"] Dec 03 12:10:30 crc kubenswrapper[5052]: I1203 12:10:30.569156 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/9e1f00b4-e4b8-4696-aacc-92f6b67f99f4-operator-scripts\") pod \"keystone-db-create-6kjm6\" (UID: \"9e1f00b4-e4b8-4696-aacc-92f6b67f99f4\") " pod="openstack-kuttl-tests/keystone-db-create-6kjm6" Dec 03 12:10:30 crc kubenswrapper[5052]: I1203 12:10:30.569287 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-rm8l2\" (UniqueName: \"kubernetes.io/projected/9e1f00b4-e4b8-4696-aacc-92f6b67f99f4-kube-api-access-rm8l2\") pod \"keystone-db-create-6kjm6\" (UID: \"9e1f00b4-e4b8-4696-aacc-92f6b67f99f4\") " pod="openstack-kuttl-tests/keystone-db-create-6kjm6" Dec 03 12:10:30 crc kubenswrapper[5052]: I1203 12:10:30.570213 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/9e1f00b4-e4b8-4696-aacc-92f6b67f99f4-operator-scripts\") pod \"keystone-db-create-6kjm6\" (UID: \"9e1f00b4-e4b8-4696-aacc-92f6b67f99f4\") " pod="openstack-kuttl-tests/keystone-db-create-6kjm6" Dec 03 12:10:30 crc kubenswrapper[5052]: I1203 12:10:30.597044 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-rm8l2\" (UniqueName: \"kubernetes.io/projected/9e1f00b4-e4b8-4696-aacc-92f6b67f99f4-kube-api-access-rm8l2\") pod \"keystone-db-create-6kjm6\" (UID: \"9e1f00b4-e4b8-4696-aacc-92f6b67f99f4\") " pod="openstack-kuttl-tests/keystone-db-create-6kjm6" Dec 03 12:10:30 crc kubenswrapper[5052]: I1203 12:10:30.670879 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/3fdeea22-1f1b-4237-affe-9368dd14fe6b-operator-scripts\") pod \"keystone-1874-account-create-update-7t9ms\" (UID: \"3fdeea22-1f1b-4237-affe-9368dd14fe6b\") " pod="openstack-kuttl-tests/keystone-1874-account-create-update-7t9ms" Dec 03 12:10:30 crc kubenswrapper[5052]: I1203 12:10:30.671129 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x2rx4\" (UniqueName: \"kubernetes.io/projected/3fdeea22-1f1b-4237-affe-9368dd14fe6b-kube-api-access-x2rx4\") pod \"keystone-1874-account-create-update-7t9ms\" (UID: \"3fdeea22-1f1b-4237-affe-9368dd14fe6b\") " pod="openstack-kuttl-tests/keystone-1874-account-create-update-7t9ms" Dec 03 12:10:30 crc kubenswrapper[5052]: I1203 12:10:30.725333 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-db-create-6kjm6" Dec 03 12:10:30 crc kubenswrapper[5052]: I1203 12:10:30.738485 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/placement-db-create-fwcd6"] Dec 03 12:10:30 crc kubenswrapper[5052]: I1203 12:10:30.739642 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-db-create-fwcd6" Dec 03 12:10:30 crc kubenswrapper[5052]: I1203 12:10:30.747870 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/placement-db-create-fwcd6"] Dec 03 12:10:30 crc kubenswrapper[5052]: I1203 12:10:30.773869 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/3fdeea22-1f1b-4237-affe-9368dd14fe6b-operator-scripts\") pod \"keystone-1874-account-create-update-7t9ms\" (UID: \"3fdeea22-1f1b-4237-affe-9368dd14fe6b\") " pod="openstack-kuttl-tests/keystone-1874-account-create-update-7t9ms" Dec 03 12:10:30 crc kubenswrapper[5052]: I1203 12:10:30.774307 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-x2rx4\" (UniqueName: \"kubernetes.io/projected/3fdeea22-1f1b-4237-affe-9368dd14fe6b-kube-api-access-x2rx4\") pod \"keystone-1874-account-create-update-7t9ms\" (UID: \"3fdeea22-1f1b-4237-affe-9368dd14fe6b\") " pod="openstack-kuttl-tests/keystone-1874-account-create-update-7t9ms" Dec 03 12:10:30 crc kubenswrapper[5052]: I1203 12:10:30.774979 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/3fdeea22-1f1b-4237-affe-9368dd14fe6b-operator-scripts\") pod \"keystone-1874-account-create-update-7t9ms\" (UID: \"3fdeea22-1f1b-4237-affe-9368dd14fe6b\") " pod="openstack-kuttl-tests/keystone-1874-account-create-update-7t9ms" Dec 03 12:10:30 crc kubenswrapper[5052]: I1203 12:10:30.792945 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-x2rx4\" (UniqueName: \"kubernetes.io/projected/3fdeea22-1f1b-4237-affe-9368dd14fe6b-kube-api-access-x2rx4\") pod \"keystone-1874-account-create-update-7t9ms\" (UID: \"3fdeea22-1f1b-4237-affe-9368dd14fe6b\") " pod="openstack-kuttl-tests/keystone-1874-account-create-update-7t9ms" Dec 03 12:10:30 crc kubenswrapper[5052]: I1203 12:10:30.845682 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/placement-7cee-account-create-update-rcvs6"] Dec 03 12:10:30 crc kubenswrapper[5052]: I1203 12:10:30.847444 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-7cee-account-create-update-rcvs6" Dec 03 12:10:30 crc kubenswrapper[5052]: I1203 12:10:30.851939 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"placement-db-secret" Dec 03 12:10:30 crc kubenswrapper[5052]: I1203 12:10:30.866946 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-1874-account-create-update-7t9ms" Dec 03 12:10:30 crc kubenswrapper[5052]: I1203 12:10:30.870298 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/placement-7cee-account-create-update-rcvs6"] Dec 03 12:10:30 crc kubenswrapper[5052]: I1203 12:10:30.875854 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-snz4p\" (UniqueName: \"kubernetes.io/projected/5093f3f0-cee5-4e84-b99c-6652546075ce-kube-api-access-snz4p\") pod \"placement-db-create-fwcd6\" (UID: \"5093f3f0-cee5-4e84-b99c-6652546075ce\") " pod="openstack-kuttl-tests/placement-db-create-fwcd6" Dec 03 12:10:30 crc kubenswrapper[5052]: I1203 12:10:30.875910 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/5093f3f0-cee5-4e84-b99c-6652546075ce-operator-scripts\") pod \"placement-db-create-fwcd6\" (UID: \"5093f3f0-cee5-4e84-b99c-6652546075ce\") " pod="openstack-kuttl-tests/placement-db-create-fwcd6" Dec 03 12:10:30 crc kubenswrapper[5052]: I1203 12:10:30.978288 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/955cd7c4-bd55-4a7e-80e7-75588d3e325a-operator-scripts\") pod \"placement-7cee-account-create-update-rcvs6\" (UID: \"955cd7c4-bd55-4a7e-80e7-75588d3e325a\") " pod="openstack-kuttl-tests/placement-7cee-account-create-update-rcvs6" Dec 03 12:10:30 crc kubenswrapper[5052]: I1203 12:10:30.978359 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4l79c\" (UniqueName: \"kubernetes.io/projected/955cd7c4-bd55-4a7e-80e7-75588d3e325a-kube-api-access-4l79c\") pod \"placement-7cee-account-create-update-rcvs6\" (UID: \"955cd7c4-bd55-4a7e-80e7-75588d3e325a\") " pod="openstack-kuttl-tests/placement-7cee-account-create-update-rcvs6" Dec 03 12:10:30 crc kubenswrapper[5052]: I1203 12:10:30.978458 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-snz4p\" (UniqueName: \"kubernetes.io/projected/5093f3f0-cee5-4e84-b99c-6652546075ce-kube-api-access-snz4p\") pod \"placement-db-create-fwcd6\" (UID: \"5093f3f0-cee5-4e84-b99c-6652546075ce\") " pod="openstack-kuttl-tests/placement-db-create-fwcd6" Dec 03 12:10:30 crc kubenswrapper[5052]: I1203 12:10:30.978496 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/5093f3f0-cee5-4e84-b99c-6652546075ce-operator-scripts\") pod \"placement-db-create-fwcd6\" (UID: \"5093f3f0-cee5-4e84-b99c-6652546075ce\") " pod="openstack-kuttl-tests/placement-db-create-fwcd6" Dec 03 12:10:30 crc kubenswrapper[5052]: I1203 12:10:30.979425 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/5093f3f0-cee5-4e84-b99c-6652546075ce-operator-scripts\") pod \"placement-db-create-fwcd6\" (UID: \"5093f3f0-cee5-4e84-b99c-6652546075ce\") " pod="openstack-kuttl-tests/placement-db-create-fwcd6" Dec 03 12:10:30 crc kubenswrapper[5052]: I1203 12:10:30.999836 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-snz4p\" (UniqueName: \"kubernetes.io/projected/5093f3f0-cee5-4e84-b99c-6652546075ce-kube-api-access-snz4p\") pod \"placement-db-create-fwcd6\" (UID: \"5093f3f0-cee5-4e84-b99c-6652546075ce\") " pod="openstack-kuttl-tests/placement-db-create-fwcd6" Dec 03 12:10:31 crc kubenswrapper[5052]: I1203 12:10:31.030372 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/glance-db-create-jr6mb"] Dec 03 12:10:31 crc kubenswrapper[5052]: I1203 12:10:31.031737 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-db-create-jr6mb" Dec 03 12:10:31 crc kubenswrapper[5052]: I1203 12:10:31.036930 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-db-create-jr6mb"] Dec 03 12:10:31 crc kubenswrapper[5052]: I1203 12:10:31.079808 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-4l79c\" (UniqueName: \"kubernetes.io/projected/955cd7c4-bd55-4a7e-80e7-75588d3e325a-kube-api-access-4l79c\") pod \"placement-7cee-account-create-update-rcvs6\" (UID: \"955cd7c4-bd55-4a7e-80e7-75588d3e325a\") " pod="openstack-kuttl-tests/placement-7cee-account-create-update-rcvs6" Dec 03 12:10:31 crc kubenswrapper[5052]: I1203 12:10:31.079978 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/955cd7c4-bd55-4a7e-80e7-75588d3e325a-operator-scripts\") pod \"placement-7cee-account-create-update-rcvs6\" (UID: \"955cd7c4-bd55-4a7e-80e7-75588d3e325a\") " pod="openstack-kuttl-tests/placement-7cee-account-create-update-rcvs6" Dec 03 12:10:31 crc kubenswrapper[5052]: I1203 12:10:31.080755 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/955cd7c4-bd55-4a7e-80e7-75588d3e325a-operator-scripts\") pod \"placement-7cee-account-create-update-rcvs6\" (UID: \"955cd7c4-bd55-4a7e-80e7-75588d3e325a\") " pod="openstack-kuttl-tests/placement-7cee-account-create-update-rcvs6" Dec 03 12:10:31 crc kubenswrapper[5052]: I1203 12:10:31.097910 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-4l79c\" (UniqueName: \"kubernetes.io/projected/955cd7c4-bd55-4a7e-80e7-75588d3e325a-kube-api-access-4l79c\") pod \"placement-7cee-account-create-update-rcvs6\" (UID: \"955cd7c4-bd55-4a7e-80e7-75588d3e325a\") " pod="openstack-kuttl-tests/placement-7cee-account-create-update-rcvs6" Dec 03 12:10:31 crc kubenswrapper[5052]: I1203 12:10:31.141313 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/glance-f253-account-create-update-zl642"] Dec 03 12:10:31 crc kubenswrapper[5052]: I1203 12:10:31.142846 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-f253-account-create-update-zl642" Dec 03 12:10:31 crc kubenswrapper[5052]: I1203 12:10:31.147425 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-db-secret" Dec 03 12:10:31 crc kubenswrapper[5052]: I1203 12:10:31.167886 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-f253-account-create-update-zl642"] Dec 03 12:10:31 crc kubenswrapper[5052]: I1203 12:10:31.174854 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-db-create-fwcd6" Dec 03 12:10:31 crc kubenswrapper[5052]: I1203 12:10:31.183016 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/81fcccc3-61d8-48d2-9422-92cfbb850d40-operator-scripts\") pod \"glance-db-create-jr6mb\" (UID: \"81fcccc3-61d8-48d2-9422-92cfbb850d40\") " pod="openstack-kuttl-tests/glance-db-create-jr6mb" Dec 03 12:10:31 crc kubenswrapper[5052]: I1203 12:10:31.183308 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nfd8j\" (UniqueName: \"kubernetes.io/projected/81fcccc3-61d8-48d2-9422-92cfbb850d40-kube-api-access-nfd8j\") pod \"glance-db-create-jr6mb\" (UID: \"81fcccc3-61d8-48d2-9422-92cfbb850d40\") " pod="openstack-kuttl-tests/glance-db-create-jr6mb" Dec 03 12:10:31 crc kubenswrapper[5052]: I1203 12:10:31.191537 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-7cee-account-create-update-rcvs6" Dec 03 12:10:31 crc kubenswrapper[5052]: I1203 12:10:31.252077 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-db-create-6kjm6"] Dec 03 12:10:31 crc kubenswrapper[5052]: W1203 12:10:31.256204 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9e1f00b4_e4b8_4696_aacc_92f6b67f99f4.slice/crio-e69a3817038d43e1dd4121041ff0aa610d3ee25a1f490a16784864109a33d9db WatchSource:0}: Error finding container e69a3817038d43e1dd4121041ff0aa610d3ee25a1f490a16784864109a33d9db: Status 404 returned error can't find the container with id e69a3817038d43e1dd4121041ff0aa610d3ee25a1f490a16784864109a33d9db Dec 03 12:10:31 crc kubenswrapper[5052]: I1203 12:10:31.285658 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/81fcccc3-61d8-48d2-9422-92cfbb850d40-operator-scripts\") pod \"glance-db-create-jr6mb\" (UID: \"81fcccc3-61d8-48d2-9422-92cfbb850d40\") " pod="openstack-kuttl-tests/glance-db-create-jr6mb" Dec 03 12:10:31 crc kubenswrapper[5052]: I1203 12:10:31.285709 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/a81519a6-c5c5-4f47-86de-4d63ebef81fe-operator-scripts\") pod \"glance-f253-account-create-update-zl642\" (UID: \"a81519a6-c5c5-4f47-86de-4d63ebef81fe\") " pod="openstack-kuttl-tests/glance-f253-account-create-update-zl642" Dec 03 12:10:31 crc kubenswrapper[5052]: I1203 12:10:31.285816 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-nfd8j\" (UniqueName: \"kubernetes.io/projected/81fcccc3-61d8-48d2-9422-92cfbb850d40-kube-api-access-nfd8j\") pod \"glance-db-create-jr6mb\" (UID: \"81fcccc3-61d8-48d2-9422-92cfbb850d40\") " pod="openstack-kuttl-tests/glance-db-create-jr6mb" Dec 03 12:10:31 crc kubenswrapper[5052]: I1203 12:10:31.285846 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fdhwh\" (UniqueName: \"kubernetes.io/projected/a81519a6-c5c5-4f47-86de-4d63ebef81fe-kube-api-access-fdhwh\") pod \"glance-f253-account-create-update-zl642\" (UID: \"a81519a6-c5c5-4f47-86de-4d63ebef81fe\") " pod="openstack-kuttl-tests/glance-f253-account-create-update-zl642" Dec 03 12:10:31 crc kubenswrapper[5052]: I1203 12:10:31.290357 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/81fcccc3-61d8-48d2-9422-92cfbb850d40-operator-scripts\") pod \"glance-db-create-jr6mb\" (UID: \"81fcccc3-61d8-48d2-9422-92cfbb850d40\") " pod="openstack-kuttl-tests/glance-db-create-jr6mb" Dec 03 12:10:31 crc kubenswrapper[5052]: I1203 12:10:31.316867 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-nfd8j\" (UniqueName: \"kubernetes.io/projected/81fcccc3-61d8-48d2-9422-92cfbb850d40-kube-api-access-nfd8j\") pod \"glance-db-create-jr6mb\" (UID: \"81fcccc3-61d8-48d2-9422-92cfbb850d40\") " pod="openstack-kuttl-tests/glance-db-create-jr6mb" Dec 03 12:10:31 crc kubenswrapper[5052]: I1203 12:10:31.353274 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-db-create-jr6mb" Dec 03 12:10:31 crc kubenswrapper[5052]: I1203 12:10:31.376291 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-1874-account-create-update-7t9ms"] Dec 03 12:10:31 crc kubenswrapper[5052]: I1203 12:10:31.387497 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/a81519a6-c5c5-4f47-86de-4d63ebef81fe-operator-scripts\") pod \"glance-f253-account-create-update-zl642\" (UID: \"a81519a6-c5c5-4f47-86de-4d63ebef81fe\") " pod="openstack-kuttl-tests/glance-f253-account-create-update-zl642" Dec 03 12:10:31 crc kubenswrapper[5052]: I1203 12:10:31.387637 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-fdhwh\" (UniqueName: \"kubernetes.io/projected/a81519a6-c5c5-4f47-86de-4d63ebef81fe-kube-api-access-fdhwh\") pod \"glance-f253-account-create-update-zl642\" (UID: \"a81519a6-c5c5-4f47-86de-4d63ebef81fe\") " pod="openstack-kuttl-tests/glance-f253-account-create-update-zl642" Dec 03 12:10:31 crc kubenswrapper[5052]: I1203 12:10:31.388531 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/a81519a6-c5c5-4f47-86de-4d63ebef81fe-operator-scripts\") pod \"glance-f253-account-create-update-zl642\" (UID: \"a81519a6-c5c5-4f47-86de-4d63ebef81fe\") " pod="openstack-kuttl-tests/glance-f253-account-create-update-zl642" Dec 03 12:10:31 crc kubenswrapper[5052]: W1203 12:10:31.390937 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3fdeea22_1f1b_4237_affe_9368dd14fe6b.slice/crio-3bfdf49a2ce9f7f97fd31414b3cc5186be4c4cff1106f8d7941e62caecddfc03 WatchSource:0}: Error finding container 3bfdf49a2ce9f7f97fd31414b3cc5186be4c4cff1106f8d7941e62caecddfc03: Status 404 returned error can't find the container with id 3bfdf49a2ce9f7f97fd31414b3cc5186be4c4cff1106f8d7941e62caecddfc03 Dec 03 12:10:31 crc kubenswrapper[5052]: I1203 12:10:31.405108 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-fdhwh\" (UniqueName: \"kubernetes.io/projected/a81519a6-c5c5-4f47-86de-4d63ebef81fe-kube-api-access-fdhwh\") pod \"glance-f253-account-create-update-zl642\" (UID: \"a81519a6-c5c5-4f47-86de-4d63ebef81fe\") " pod="openstack-kuttl-tests/glance-f253-account-create-update-zl642" Dec 03 12:10:31 crc kubenswrapper[5052]: I1203 12:10:31.470274 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-f253-account-create-update-zl642" Dec 03 12:10:31 crc kubenswrapper[5052]: I1203 12:10:31.656603 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/placement-db-create-fwcd6"] Dec 03 12:10:31 crc kubenswrapper[5052]: I1203 12:10:31.692726 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/694f3a22-38d9-487d-b887-a915fa8f99b0-etc-swift\") pod \"swift-storage-0\" (UID: \"694f3a22-38d9-487d-b887-a915fa8f99b0\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:10:31 crc kubenswrapper[5052]: E1203 12:10:31.692948 5052 projected.go:288] Couldn't get configMap openstack-kuttl-tests/swift-ring-files: configmap "swift-ring-files" not found Dec 03 12:10:31 crc kubenswrapper[5052]: E1203 12:10:31.692970 5052 projected.go:194] Error preparing data for projected volume etc-swift for pod openstack-kuttl-tests/swift-storage-0: configmap "swift-ring-files" not found Dec 03 12:10:31 crc kubenswrapper[5052]: E1203 12:10:31.693025 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/694f3a22-38d9-487d-b887-a915fa8f99b0-etc-swift podName:694f3a22-38d9-487d-b887-a915fa8f99b0 nodeName:}" failed. No retries permitted until 2025-12-03 12:10:39.693007793 +0000 UTC m=+3030.281249567 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "etc-swift" (UniqueName: "kubernetes.io/projected/694f3a22-38d9-487d-b887-a915fa8f99b0-etc-swift") pod "swift-storage-0" (UID: "694f3a22-38d9-487d-b887-a915fa8f99b0") : configmap "swift-ring-files" not found Dec 03 12:10:31 crc kubenswrapper[5052]: W1203 12:10:31.702369 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5093f3f0_cee5_4e84_b99c_6652546075ce.slice/crio-28836ddaabec3d37f9e2a6b6363243ec5e8747d0b1c33d7aaa7e4dea8fa24476 WatchSource:0}: Error finding container 28836ddaabec3d37f9e2a6b6363243ec5e8747d0b1c33d7aaa7e4dea8fa24476: Status 404 returned error can't find the container with id 28836ddaabec3d37f9e2a6b6363243ec5e8747d0b1c33d7aaa7e4dea8fa24476 Dec 03 12:10:31 crc kubenswrapper[5052]: I1203 12:10:31.731874 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/placement-7cee-account-create-update-rcvs6"] Dec 03 12:10:31 crc kubenswrapper[5052]: W1203 12:10:31.738660 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod955cd7c4_bd55_4a7e_80e7_75588d3e325a.slice/crio-094cd2661efb2cabe8775c8103ac5f902aad0c7ddb22a76485d61bf860338cf5 WatchSource:0}: Error finding container 094cd2661efb2cabe8775c8103ac5f902aad0c7ddb22a76485d61bf860338cf5: Status 404 returned error can't find the container with id 094cd2661efb2cabe8775c8103ac5f902aad0c7ddb22a76485d61bf860338cf5 Dec 03 12:10:31 crc kubenswrapper[5052]: I1203 12:10:31.888873 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-db-create-jr6mb"] Dec 03 12:10:31 crc kubenswrapper[5052]: W1203 12:10:31.924349 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod81fcccc3_61d8_48d2_9422_92cfbb850d40.slice/crio-1b1c50b0c4426004ce75f8024d4a2965181f3ae2ca4338e85204ca7b2b051895 WatchSource:0}: Error finding container 1b1c50b0c4426004ce75f8024d4a2965181f3ae2ca4338e85204ca7b2b051895: Status 404 returned error can't find the container with id 1b1c50b0c4426004ce75f8024d4a2965181f3ae2ca4338e85204ca7b2b051895 Dec 03 12:10:31 crc kubenswrapper[5052]: I1203 12:10:31.989045 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-db-create-fwcd6" event={"ID":"5093f3f0-cee5-4e84-b99c-6652546075ce","Type":"ContainerStarted","Data":"28836ddaabec3d37f9e2a6b6363243ec5e8747d0b1c33d7aaa7e4dea8fa24476"} Dec 03 12:10:31 crc kubenswrapper[5052]: I1203 12:10:31.991976 5052 generic.go:334] "Generic (PLEG): container finished" podID="9e1f00b4-e4b8-4696-aacc-92f6b67f99f4" containerID="44f6c3de6269728aee0300a683f398adb92ca35b09bbc01daff9bd5bfff2bcd3" exitCode=0 Dec 03 12:10:31 crc kubenswrapper[5052]: I1203 12:10:31.992067 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-db-create-6kjm6" event={"ID":"9e1f00b4-e4b8-4696-aacc-92f6b67f99f4","Type":"ContainerDied","Data":"44f6c3de6269728aee0300a683f398adb92ca35b09bbc01daff9bd5bfff2bcd3"} Dec 03 12:10:31 crc kubenswrapper[5052]: I1203 12:10:31.992102 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-db-create-6kjm6" event={"ID":"9e1f00b4-e4b8-4696-aacc-92f6b67f99f4","Type":"ContainerStarted","Data":"e69a3817038d43e1dd4121041ff0aa610d3ee25a1f490a16784864109a33d9db"} Dec 03 12:10:31 crc kubenswrapper[5052]: I1203 12:10:31.994983 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-db-create-jr6mb" event={"ID":"81fcccc3-61d8-48d2-9422-92cfbb850d40","Type":"ContainerStarted","Data":"1b1c50b0c4426004ce75f8024d4a2965181f3ae2ca4338e85204ca7b2b051895"} Dec 03 12:10:32 crc kubenswrapper[5052]: I1203 12:10:32.000738 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-f253-account-create-update-zl642"] Dec 03 12:10:32 crc kubenswrapper[5052]: I1203 12:10:32.011049 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-7cee-account-create-update-rcvs6" event={"ID":"955cd7c4-bd55-4a7e-80e7-75588d3e325a","Type":"ContainerStarted","Data":"094cd2661efb2cabe8775c8103ac5f902aad0c7ddb22a76485d61bf860338cf5"} Dec 03 12:10:32 crc kubenswrapper[5052]: I1203 12:10:32.015216 5052 generic.go:334] "Generic (PLEG): container finished" podID="3fdeea22-1f1b-4237-affe-9368dd14fe6b" containerID="7bf695872e862a8fb97b7b705d1ea5a2d8ed02b1b073a00b69dc98aeab010dbd" exitCode=0 Dec 03 12:10:32 crc kubenswrapper[5052]: I1203 12:10:32.015271 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-1874-account-create-update-7t9ms" event={"ID":"3fdeea22-1f1b-4237-affe-9368dd14fe6b","Type":"ContainerDied","Data":"7bf695872e862a8fb97b7b705d1ea5a2d8ed02b1b073a00b69dc98aeab010dbd"} Dec 03 12:10:32 crc kubenswrapper[5052]: I1203 12:10:32.015301 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-1874-account-create-update-7t9ms" event={"ID":"3fdeea22-1f1b-4237-affe-9368dd14fe6b","Type":"ContainerStarted","Data":"3bfdf49a2ce9f7f97fd31414b3cc5186be4c4cff1106f8d7941e62caecddfc03"} Dec 03 12:10:33 crc kubenswrapper[5052]: I1203 12:10:33.025808 5052 generic.go:334] "Generic (PLEG): container finished" podID="5093f3f0-cee5-4e84-b99c-6652546075ce" containerID="50e44a430f57ac6bd96da39c820b96952a8f8446aba9b14209511df5a891cac2" exitCode=0 Dec 03 12:10:33 crc kubenswrapper[5052]: I1203 12:10:33.025880 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-db-create-fwcd6" event={"ID":"5093f3f0-cee5-4e84-b99c-6652546075ce","Type":"ContainerDied","Data":"50e44a430f57ac6bd96da39c820b96952a8f8446aba9b14209511df5a891cac2"} Dec 03 12:10:33 crc kubenswrapper[5052]: I1203 12:10:33.028779 5052 generic.go:334] "Generic (PLEG): container finished" podID="a81519a6-c5c5-4f47-86de-4d63ebef81fe" containerID="90d318c8258cb1c966ff66e10dc4b544bb9db909d657651ffa26b1e4b73924f6" exitCode=0 Dec 03 12:10:33 crc kubenswrapper[5052]: I1203 12:10:33.028964 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-f253-account-create-update-zl642" event={"ID":"a81519a6-c5c5-4f47-86de-4d63ebef81fe","Type":"ContainerDied","Data":"90d318c8258cb1c966ff66e10dc4b544bb9db909d657651ffa26b1e4b73924f6"} Dec 03 12:10:33 crc kubenswrapper[5052]: I1203 12:10:33.029014 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-f253-account-create-update-zl642" event={"ID":"a81519a6-c5c5-4f47-86de-4d63ebef81fe","Type":"ContainerStarted","Data":"df753f7bf48679e7df46c492774bc65f493e21d19dc0b630f0bd9e8cee878963"} Dec 03 12:10:33 crc kubenswrapper[5052]: I1203 12:10:33.031505 5052 generic.go:334] "Generic (PLEG): container finished" podID="81fcccc3-61d8-48d2-9422-92cfbb850d40" containerID="087bf897c860ad2b3bd2cd4f2aa80759a612a4adda5ec15314f2e7579f48e9e2" exitCode=0 Dec 03 12:10:33 crc kubenswrapper[5052]: I1203 12:10:33.031618 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-db-create-jr6mb" event={"ID":"81fcccc3-61d8-48d2-9422-92cfbb850d40","Type":"ContainerDied","Data":"087bf897c860ad2b3bd2cd4f2aa80759a612a4adda5ec15314f2e7579f48e9e2"} Dec 03 12:10:33 crc kubenswrapper[5052]: I1203 12:10:33.033352 5052 generic.go:334] "Generic (PLEG): container finished" podID="955cd7c4-bd55-4a7e-80e7-75588d3e325a" containerID="2daa465a549ba89fd54fb30d8b03b5b9fae793709253f37943466cd74fa59032" exitCode=0 Dec 03 12:10:33 crc kubenswrapper[5052]: I1203 12:10:33.033408 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-7cee-account-create-update-rcvs6" event={"ID":"955cd7c4-bd55-4a7e-80e7-75588d3e325a","Type":"ContainerDied","Data":"2daa465a549ba89fd54fb30d8b03b5b9fae793709253f37943466cd74fa59032"} Dec 03 12:10:33 crc kubenswrapper[5052]: I1203 12:10:33.492160 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-1874-account-create-update-7t9ms" Dec 03 12:10:33 crc kubenswrapper[5052]: I1203 12:10:33.603739 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-db-create-6kjm6" Dec 03 12:10:33 crc kubenswrapper[5052]: I1203 12:10:33.625329 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x2rx4\" (UniqueName: \"kubernetes.io/projected/3fdeea22-1f1b-4237-affe-9368dd14fe6b-kube-api-access-x2rx4\") pod \"3fdeea22-1f1b-4237-affe-9368dd14fe6b\" (UID: \"3fdeea22-1f1b-4237-affe-9368dd14fe6b\") " Dec 03 12:10:33 crc kubenswrapper[5052]: I1203 12:10:33.625598 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/3fdeea22-1f1b-4237-affe-9368dd14fe6b-operator-scripts\") pod \"3fdeea22-1f1b-4237-affe-9368dd14fe6b\" (UID: \"3fdeea22-1f1b-4237-affe-9368dd14fe6b\") " Dec 03 12:10:33 crc kubenswrapper[5052]: I1203 12:10:33.626733 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3fdeea22-1f1b-4237-affe-9368dd14fe6b-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "3fdeea22-1f1b-4237-affe-9368dd14fe6b" (UID: "3fdeea22-1f1b-4237-affe-9368dd14fe6b"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:10:33 crc kubenswrapper[5052]: I1203 12:10:33.631382 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3fdeea22-1f1b-4237-affe-9368dd14fe6b-kube-api-access-x2rx4" (OuterVolumeSpecName: "kube-api-access-x2rx4") pod "3fdeea22-1f1b-4237-affe-9368dd14fe6b" (UID: "3fdeea22-1f1b-4237-affe-9368dd14fe6b"). InnerVolumeSpecName "kube-api-access-x2rx4". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:10:33 crc kubenswrapper[5052]: I1203 12:10:33.673831 5052 scope.go:117] "RemoveContainer" containerID="63298107b3d090deff9ed92f3d613e5b1c86d9137684d117500257db8ee1c805" Dec 03 12:10:33 crc kubenswrapper[5052]: E1203 12:10:33.674200 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 12:10:33 crc kubenswrapper[5052]: I1203 12:10:33.727735 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rm8l2\" (UniqueName: \"kubernetes.io/projected/9e1f00b4-e4b8-4696-aacc-92f6b67f99f4-kube-api-access-rm8l2\") pod \"9e1f00b4-e4b8-4696-aacc-92f6b67f99f4\" (UID: \"9e1f00b4-e4b8-4696-aacc-92f6b67f99f4\") " Dec 03 12:10:33 crc kubenswrapper[5052]: I1203 12:10:33.727870 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/9e1f00b4-e4b8-4696-aacc-92f6b67f99f4-operator-scripts\") pod \"9e1f00b4-e4b8-4696-aacc-92f6b67f99f4\" (UID: \"9e1f00b4-e4b8-4696-aacc-92f6b67f99f4\") " Dec 03 12:10:33 crc kubenswrapper[5052]: I1203 12:10:33.728435 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/3fdeea22-1f1b-4237-affe-9368dd14fe6b-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:10:33 crc kubenswrapper[5052]: I1203 12:10:33.728457 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-x2rx4\" (UniqueName: \"kubernetes.io/projected/3fdeea22-1f1b-4237-affe-9368dd14fe6b-kube-api-access-x2rx4\") on node \"crc\" DevicePath \"\"" Dec 03 12:10:33 crc kubenswrapper[5052]: I1203 12:10:33.729031 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9e1f00b4-e4b8-4696-aacc-92f6b67f99f4-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "9e1f00b4-e4b8-4696-aacc-92f6b67f99f4" (UID: "9e1f00b4-e4b8-4696-aacc-92f6b67f99f4"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:10:33 crc kubenswrapper[5052]: I1203 12:10:33.732572 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9e1f00b4-e4b8-4696-aacc-92f6b67f99f4-kube-api-access-rm8l2" (OuterVolumeSpecName: "kube-api-access-rm8l2") pod "9e1f00b4-e4b8-4696-aacc-92f6b67f99f4" (UID: "9e1f00b4-e4b8-4696-aacc-92f6b67f99f4"). InnerVolumeSpecName "kube-api-access-rm8l2". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:10:33 crc kubenswrapper[5052]: I1203 12:10:33.829829 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-rm8l2\" (UniqueName: \"kubernetes.io/projected/9e1f00b4-e4b8-4696-aacc-92f6b67f99f4-kube-api-access-rm8l2\") on node \"crc\" DevicePath \"\"" Dec 03 12:10:33 crc kubenswrapper[5052]: I1203 12:10:33.830104 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/9e1f00b4-e4b8-4696-aacc-92f6b67f99f4-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:10:34 crc kubenswrapper[5052]: I1203 12:10:34.046108 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-1874-account-create-update-7t9ms" event={"ID":"3fdeea22-1f1b-4237-affe-9368dd14fe6b","Type":"ContainerDied","Data":"3bfdf49a2ce9f7f97fd31414b3cc5186be4c4cff1106f8d7941e62caecddfc03"} Dec 03 12:10:34 crc kubenswrapper[5052]: I1203 12:10:34.046156 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3bfdf49a2ce9f7f97fd31414b3cc5186be4c4cff1106f8d7941e62caecddfc03" Dec 03 12:10:34 crc kubenswrapper[5052]: I1203 12:10:34.046220 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-1874-account-create-update-7t9ms" Dec 03 12:10:34 crc kubenswrapper[5052]: I1203 12:10:34.051369 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-db-create-6kjm6" Dec 03 12:10:34 crc kubenswrapper[5052]: I1203 12:10:34.051403 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-db-create-6kjm6" event={"ID":"9e1f00b4-e4b8-4696-aacc-92f6b67f99f4","Type":"ContainerDied","Data":"e69a3817038d43e1dd4121041ff0aa610d3ee25a1f490a16784864109a33d9db"} Dec 03 12:10:34 crc kubenswrapper[5052]: I1203 12:10:34.051431 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e69a3817038d43e1dd4121041ff0aa610d3ee25a1f490a16784864109a33d9db" Dec 03 12:10:34 crc kubenswrapper[5052]: I1203 12:10:34.575666 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-f253-account-create-update-zl642" Dec 03 12:10:34 crc kubenswrapper[5052]: I1203 12:10:34.644047 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fdhwh\" (UniqueName: \"kubernetes.io/projected/a81519a6-c5c5-4f47-86de-4d63ebef81fe-kube-api-access-fdhwh\") pod \"a81519a6-c5c5-4f47-86de-4d63ebef81fe\" (UID: \"a81519a6-c5c5-4f47-86de-4d63ebef81fe\") " Dec 03 12:10:34 crc kubenswrapper[5052]: I1203 12:10:34.644184 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/a81519a6-c5c5-4f47-86de-4d63ebef81fe-operator-scripts\") pod \"a81519a6-c5c5-4f47-86de-4d63ebef81fe\" (UID: \"a81519a6-c5c5-4f47-86de-4d63ebef81fe\") " Dec 03 12:10:34 crc kubenswrapper[5052]: I1203 12:10:34.645619 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a81519a6-c5c5-4f47-86de-4d63ebef81fe-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "a81519a6-c5c5-4f47-86de-4d63ebef81fe" (UID: "a81519a6-c5c5-4f47-86de-4d63ebef81fe"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:10:34 crc kubenswrapper[5052]: I1203 12:10:34.656407 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a81519a6-c5c5-4f47-86de-4d63ebef81fe-kube-api-access-fdhwh" (OuterVolumeSpecName: "kube-api-access-fdhwh") pod "a81519a6-c5c5-4f47-86de-4d63ebef81fe" (UID: "a81519a6-c5c5-4f47-86de-4d63ebef81fe"). InnerVolumeSpecName "kube-api-access-fdhwh". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:10:34 crc kubenswrapper[5052]: I1203 12:10:34.747177 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-fdhwh\" (UniqueName: \"kubernetes.io/projected/a81519a6-c5c5-4f47-86de-4d63ebef81fe-kube-api-access-fdhwh\") on node \"crc\" DevicePath \"\"" Dec 03 12:10:34 crc kubenswrapper[5052]: I1203 12:10:34.747255 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/a81519a6-c5c5-4f47-86de-4d63ebef81fe-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:10:34 crc kubenswrapper[5052]: I1203 12:10:34.821857 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-db-create-jr6mb" Dec 03 12:10:34 crc kubenswrapper[5052]: I1203 12:10:34.834131 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-db-create-fwcd6" Dec 03 12:10:34 crc kubenswrapper[5052]: I1203 12:10:34.849446 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-7cee-account-create-update-rcvs6" Dec 03 12:10:34 crc kubenswrapper[5052]: I1203 12:10:34.952080 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/81fcccc3-61d8-48d2-9422-92cfbb850d40-operator-scripts\") pod \"81fcccc3-61d8-48d2-9422-92cfbb850d40\" (UID: \"81fcccc3-61d8-48d2-9422-92cfbb850d40\") " Dec 03 12:10:34 crc kubenswrapper[5052]: I1203 12:10:34.952662 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-snz4p\" (UniqueName: \"kubernetes.io/projected/5093f3f0-cee5-4e84-b99c-6652546075ce-kube-api-access-snz4p\") pod \"5093f3f0-cee5-4e84-b99c-6652546075ce\" (UID: \"5093f3f0-cee5-4e84-b99c-6652546075ce\") " Dec 03 12:10:34 crc kubenswrapper[5052]: I1203 12:10:34.953437 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4l79c\" (UniqueName: \"kubernetes.io/projected/955cd7c4-bd55-4a7e-80e7-75588d3e325a-kube-api-access-4l79c\") pod \"955cd7c4-bd55-4a7e-80e7-75588d3e325a\" (UID: \"955cd7c4-bd55-4a7e-80e7-75588d3e325a\") " Dec 03 12:10:34 crc kubenswrapper[5052]: I1203 12:10:34.953733 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nfd8j\" (UniqueName: \"kubernetes.io/projected/81fcccc3-61d8-48d2-9422-92cfbb850d40-kube-api-access-nfd8j\") pod \"81fcccc3-61d8-48d2-9422-92cfbb850d40\" (UID: \"81fcccc3-61d8-48d2-9422-92cfbb850d40\") " Dec 03 12:10:34 crc kubenswrapper[5052]: I1203 12:10:34.953844 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/955cd7c4-bd55-4a7e-80e7-75588d3e325a-operator-scripts\") pod \"955cd7c4-bd55-4a7e-80e7-75588d3e325a\" (UID: \"955cd7c4-bd55-4a7e-80e7-75588d3e325a\") " Dec 03 12:10:34 crc kubenswrapper[5052]: I1203 12:10:34.952785 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/81fcccc3-61d8-48d2-9422-92cfbb850d40-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "81fcccc3-61d8-48d2-9422-92cfbb850d40" (UID: "81fcccc3-61d8-48d2-9422-92cfbb850d40"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:10:34 crc kubenswrapper[5052]: I1203 12:10:34.954381 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/955cd7c4-bd55-4a7e-80e7-75588d3e325a-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "955cd7c4-bd55-4a7e-80e7-75588d3e325a" (UID: "955cd7c4-bd55-4a7e-80e7-75588d3e325a"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:10:34 crc kubenswrapper[5052]: I1203 12:10:34.954750 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/5093f3f0-cee5-4e84-b99c-6652546075ce-operator-scripts\") pod \"5093f3f0-cee5-4e84-b99c-6652546075ce\" (UID: \"5093f3f0-cee5-4e84-b99c-6652546075ce\") " Dec 03 12:10:34 crc kubenswrapper[5052]: I1203 12:10:34.955314 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5093f3f0-cee5-4e84-b99c-6652546075ce-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "5093f3f0-cee5-4e84-b99c-6652546075ce" (UID: "5093f3f0-cee5-4e84-b99c-6652546075ce"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:10:34 crc kubenswrapper[5052]: I1203 12:10:34.955659 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5093f3f0-cee5-4e84-b99c-6652546075ce-kube-api-access-snz4p" (OuterVolumeSpecName: "kube-api-access-snz4p") pod "5093f3f0-cee5-4e84-b99c-6652546075ce" (UID: "5093f3f0-cee5-4e84-b99c-6652546075ce"). InnerVolumeSpecName "kube-api-access-snz4p". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:10:34 crc kubenswrapper[5052]: I1203 12:10:34.956200 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/81fcccc3-61d8-48d2-9422-92cfbb850d40-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:10:34 crc kubenswrapper[5052]: I1203 12:10:34.956270 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-snz4p\" (UniqueName: \"kubernetes.io/projected/5093f3f0-cee5-4e84-b99c-6652546075ce-kube-api-access-snz4p\") on node \"crc\" DevicePath \"\"" Dec 03 12:10:34 crc kubenswrapper[5052]: I1203 12:10:34.956493 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/955cd7c4-bd55-4a7e-80e7-75588d3e325a-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:10:34 crc kubenswrapper[5052]: I1203 12:10:34.956603 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/5093f3f0-cee5-4e84-b99c-6652546075ce-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:10:34 crc kubenswrapper[5052]: I1203 12:10:34.956787 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/955cd7c4-bd55-4a7e-80e7-75588d3e325a-kube-api-access-4l79c" (OuterVolumeSpecName: "kube-api-access-4l79c") pod "955cd7c4-bd55-4a7e-80e7-75588d3e325a" (UID: "955cd7c4-bd55-4a7e-80e7-75588d3e325a"). InnerVolumeSpecName "kube-api-access-4l79c". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:10:34 crc kubenswrapper[5052]: I1203 12:10:34.966466 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/81fcccc3-61d8-48d2-9422-92cfbb850d40-kube-api-access-nfd8j" (OuterVolumeSpecName: "kube-api-access-nfd8j") pod "81fcccc3-61d8-48d2-9422-92cfbb850d40" (UID: "81fcccc3-61d8-48d2-9422-92cfbb850d40"). InnerVolumeSpecName "kube-api-access-nfd8j". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:10:35 crc kubenswrapper[5052]: I1203 12:10:35.057946 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-4l79c\" (UniqueName: \"kubernetes.io/projected/955cd7c4-bd55-4a7e-80e7-75588d3e325a-kube-api-access-4l79c\") on node \"crc\" DevicePath \"\"" Dec 03 12:10:35 crc kubenswrapper[5052]: I1203 12:10:35.059349 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-nfd8j\" (UniqueName: \"kubernetes.io/projected/81fcccc3-61d8-48d2-9422-92cfbb850d40-kube-api-access-nfd8j\") on node \"crc\" DevicePath \"\"" Dec 03 12:10:35 crc kubenswrapper[5052]: I1203 12:10:35.066876 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-db-create-fwcd6" event={"ID":"5093f3f0-cee5-4e84-b99c-6652546075ce","Type":"ContainerDied","Data":"28836ddaabec3d37f9e2a6b6363243ec5e8747d0b1c33d7aaa7e4dea8fa24476"} Dec 03 12:10:35 crc kubenswrapper[5052]: I1203 12:10:35.067161 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="28836ddaabec3d37f9e2a6b6363243ec5e8747d0b1c33d7aaa7e4dea8fa24476" Dec 03 12:10:35 crc kubenswrapper[5052]: I1203 12:10:35.066911 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-db-create-fwcd6" Dec 03 12:10:35 crc kubenswrapper[5052]: I1203 12:10:35.070126 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-f253-account-create-update-zl642" event={"ID":"a81519a6-c5c5-4f47-86de-4d63ebef81fe","Type":"ContainerDied","Data":"df753f7bf48679e7df46c492774bc65f493e21d19dc0b630f0bd9e8cee878963"} Dec 03 12:10:35 crc kubenswrapper[5052]: I1203 12:10:35.070195 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="df753f7bf48679e7df46c492774bc65f493e21d19dc0b630f0bd9e8cee878963" Dec 03 12:10:35 crc kubenswrapper[5052]: I1203 12:10:35.070294 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-f253-account-create-update-zl642" Dec 03 12:10:35 crc kubenswrapper[5052]: I1203 12:10:35.074073 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-db-create-jr6mb" event={"ID":"81fcccc3-61d8-48d2-9422-92cfbb850d40","Type":"ContainerDied","Data":"1b1c50b0c4426004ce75f8024d4a2965181f3ae2ca4338e85204ca7b2b051895"} Dec 03 12:10:35 crc kubenswrapper[5052]: I1203 12:10:35.074264 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1b1c50b0c4426004ce75f8024d4a2965181f3ae2ca4338e85204ca7b2b051895" Dec 03 12:10:35 crc kubenswrapper[5052]: I1203 12:10:35.074089 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-db-create-jr6mb" Dec 03 12:10:35 crc kubenswrapper[5052]: I1203 12:10:35.077568 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-7cee-account-create-update-rcvs6" Dec 03 12:10:35 crc kubenswrapper[5052]: I1203 12:10:35.077585 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-7cee-account-create-update-rcvs6" event={"ID":"955cd7c4-bd55-4a7e-80e7-75588d3e325a","Type":"ContainerDied","Data":"094cd2661efb2cabe8775c8103ac5f902aad0c7ddb22a76485d61bf860338cf5"} Dec 03 12:10:35 crc kubenswrapper[5052]: I1203 12:10:35.077864 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="094cd2661efb2cabe8775c8103ac5f902aad0c7ddb22a76485d61bf860338cf5" Dec 03 12:10:36 crc kubenswrapper[5052]: I1203 12:10:36.289024 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/glance-db-sync-nt4p4"] Dec 03 12:10:36 crc kubenswrapper[5052]: E1203 12:10:36.297614 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="a81519a6-c5c5-4f47-86de-4d63ebef81fe" containerName="mariadb-account-create-update" Dec 03 12:10:36 crc kubenswrapper[5052]: I1203 12:10:36.297653 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="a81519a6-c5c5-4f47-86de-4d63ebef81fe" containerName="mariadb-account-create-update" Dec 03 12:10:36 crc kubenswrapper[5052]: E1203 12:10:36.297701 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="955cd7c4-bd55-4a7e-80e7-75588d3e325a" containerName="mariadb-account-create-update" Dec 03 12:10:36 crc kubenswrapper[5052]: I1203 12:10:36.297711 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="955cd7c4-bd55-4a7e-80e7-75588d3e325a" containerName="mariadb-account-create-update" Dec 03 12:10:36 crc kubenswrapper[5052]: E1203 12:10:36.297742 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="3fdeea22-1f1b-4237-affe-9368dd14fe6b" containerName="mariadb-account-create-update" Dec 03 12:10:36 crc kubenswrapper[5052]: I1203 12:10:36.297750 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="3fdeea22-1f1b-4237-affe-9368dd14fe6b" containerName="mariadb-account-create-update" Dec 03 12:10:36 crc kubenswrapper[5052]: E1203 12:10:36.297770 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="9e1f00b4-e4b8-4696-aacc-92f6b67f99f4" containerName="mariadb-database-create" Dec 03 12:10:36 crc kubenswrapper[5052]: I1203 12:10:36.297777 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="9e1f00b4-e4b8-4696-aacc-92f6b67f99f4" containerName="mariadb-database-create" Dec 03 12:10:36 crc kubenswrapper[5052]: E1203 12:10:36.297796 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="81fcccc3-61d8-48d2-9422-92cfbb850d40" containerName="mariadb-database-create" Dec 03 12:10:36 crc kubenswrapper[5052]: I1203 12:10:36.297803 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="81fcccc3-61d8-48d2-9422-92cfbb850d40" containerName="mariadb-database-create" Dec 03 12:10:36 crc kubenswrapper[5052]: E1203 12:10:36.297829 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="5093f3f0-cee5-4e84-b99c-6652546075ce" containerName="mariadb-database-create" Dec 03 12:10:36 crc kubenswrapper[5052]: I1203 12:10:36.297836 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="5093f3f0-cee5-4e84-b99c-6652546075ce" containerName="mariadb-database-create" Dec 03 12:10:36 crc kubenswrapper[5052]: I1203 12:10:36.298152 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="81fcccc3-61d8-48d2-9422-92cfbb850d40" containerName="mariadb-database-create" Dec 03 12:10:36 crc kubenswrapper[5052]: I1203 12:10:36.298186 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="5093f3f0-cee5-4e84-b99c-6652546075ce" containerName="mariadb-database-create" Dec 03 12:10:36 crc kubenswrapper[5052]: I1203 12:10:36.298200 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="a81519a6-c5c5-4f47-86de-4d63ebef81fe" containerName="mariadb-account-create-update" Dec 03 12:10:36 crc kubenswrapper[5052]: I1203 12:10:36.298227 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="9e1f00b4-e4b8-4696-aacc-92f6b67f99f4" containerName="mariadb-database-create" Dec 03 12:10:36 crc kubenswrapper[5052]: I1203 12:10:36.298244 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="3fdeea22-1f1b-4237-affe-9368dd14fe6b" containerName="mariadb-account-create-update" Dec 03 12:10:36 crc kubenswrapper[5052]: I1203 12:10:36.298262 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="955cd7c4-bd55-4a7e-80e7-75588d3e325a" containerName="mariadb-account-create-update" Dec 03 12:10:36 crc kubenswrapper[5052]: I1203 12:10:36.299264 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-db-sync-nt4p4"] Dec 03 12:10:36 crc kubenswrapper[5052]: I1203 12:10:36.299410 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-db-sync-nt4p4" Dec 03 12:10:36 crc kubenswrapper[5052]: I1203 12:10:36.302795 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-config-data" Dec 03 12:10:36 crc kubenswrapper[5052]: I1203 12:10:36.303320 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-glance-dockercfg-hhggs" Dec 03 12:10:36 crc kubenswrapper[5052]: I1203 12:10:36.386317 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6d8c591d-96a8-4bef-908c-89ae9d1b46fb-config-data\") pod \"glance-db-sync-nt4p4\" (UID: \"6d8c591d-96a8-4bef-908c-89ae9d1b46fb\") " pod="openstack-kuttl-tests/glance-db-sync-nt4p4" Dec 03 12:10:36 crc kubenswrapper[5052]: I1203 12:10:36.386622 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6d8c591d-96a8-4bef-908c-89ae9d1b46fb-combined-ca-bundle\") pod \"glance-db-sync-nt4p4\" (UID: \"6d8c591d-96a8-4bef-908c-89ae9d1b46fb\") " pod="openstack-kuttl-tests/glance-db-sync-nt4p4" Dec 03 12:10:36 crc kubenswrapper[5052]: I1203 12:10:36.386746 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wd26w\" (UniqueName: \"kubernetes.io/projected/6d8c591d-96a8-4bef-908c-89ae9d1b46fb-kube-api-access-wd26w\") pod \"glance-db-sync-nt4p4\" (UID: \"6d8c591d-96a8-4bef-908c-89ae9d1b46fb\") " pod="openstack-kuttl-tests/glance-db-sync-nt4p4" Dec 03 12:10:36 crc kubenswrapper[5052]: I1203 12:10:36.386844 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/6d8c591d-96a8-4bef-908c-89ae9d1b46fb-db-sync-config-data\") pod \"glance-db-sync-nt4p4\" (UID: \"6d8c591d-96a8-4bef-908c-89ae9d1b46fb\") " pod="openstack-kuttl-tests/glance-db-sync-nt4p4" Dec 03 12:10:36 crc kubenswrapper[5052]: I1203 12:10:36.488849 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6d8c591d-96a8-4bef-908c-89ae9d1b46fb-config-data\") pod \"glance-db-sync-nt4p4\" (UID: \"6d8c591d-96a8-4bef-908c-89ae9d1b46fb\") " pod="openstack-kuttl-tests/glance-db-sync-nt4p4" Dec 03 12:10:36 crc kubenswrapper[5052]: I1203 12:10:36.488937 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6d8c591d-96a8-4bef-908c-89ae9d1b46fb-combined-ca-bundle\") pod \"glance-db-sync-nt4p4\" (UID: \"6d8c591d-96a8-4bef-908c-89ae9d1b46fb\") " pod="openstack-kuttl-tests/glance-db-sync-nt4p4" Dec 03 12:10:36 crc kubenswrapper[5052]: I1203 12:10:36.489000 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/6d8c591d-96a8-4bef-908c-89ae9d1b46fb-db-sync-config-data\") pod \"glance-db-sync-nt4p4\" (UID: \"6d8c591d-96a8-4bef-908c-89ae9d1b46fb\") " pod="openstack-kuttl-tests/glance-db-sync-nt4p4" Dec 03 12:10:36 crc kubenswrapper[5052]: I1203 12:10:36.489044 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-wd26w\" (UniqueName: \"kubernetes.io/projected/6d8c591d-96a8-4bef-908c-89ae9d1b46fb-kube-api-access-wd26w\") pod \"glance-db-sync-nt4p4\" (UID: \"6d8c591d-96a8-4bef-908c-89ae9d1b46fb\") " pod="openstack-kuttl-tests/glance-db-sync-nt4p4" Dec 03 12:10:36 crc kubenswrapper[5052]: I1203 12:10:36.493003 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6d8c591d-96a8-4bef-908c-89ae9d1b46fb-config-data\") pod \"glance-db-sync-nt4p4\" (UID: \"6d8c591d-96a8-4bef-908c-89ae9d1b46fb\") " pod="openstack-kuttl-tests/glance-db-sync-nt4p4" Dec 03 12:10:36 crc kubenswrapper[5052]: I1203 12:10:36.493048 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6d8c591d-96a8-4bef-908c-89ae9d1b46fb-combined-ca-bundle\") pod \"glance-db-sync-nt4p4\" (UID: \"6d8c591d-96a8-4bef-908c-89ae9d1b46fb\") " pod="openstack-kuttl-tests/glance-db-sync-nt4p4" Dec 03 12:10:36 crc kubenswrapper[5052]: I1203 12:10:36.493369 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/6d8c591d-96a8-4bef-908c-89ae9d1b46fb-db-sync-config-data\") pod \"glance-db-sync-nt4p4\" (UID: \"6d8c591d-96a8-4bef-908c-89ae9d1b46fb\") " pod="openstack-kuttl-tests/glance-db-sync-nt4p4" Dec 03 12:10:36 crc kubenswrapper[5052]: I1203 12:10:36.505986 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-wd26w\" (UniqueName: \"kubernetes.io/projected/6d8c591d-96a8-4bef-908c-89ae9d1b46fb-kube-api-access-wd26w\") pod \"glance-db-sync-nt4p4\" (UID: \"6d8c591d-96a8-4bef-908c-89ae9d1b46fb\") " pod="openstack-kuttl-tests/glance-db-sync-nt4p4" Dec 03 12:10:36 crc kubenswrapper[5052]: I1203 12:10:36.640472 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-db-sync-nt4p4" Dec 03 12:10:37 crc kubenswrapper[5052]: I1203 12:10:37.129992 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-db-sync-nt4p4"] Dec 03 12:10:38 crc kubenswrapper[5052]: I1203 12:10:38.105122 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-db-sync-nt4p4" event={"ID":"6d8c591d-96a8-4bef-908c-89ae9d1b46fb","Type":"ContainerStarted","Data":"5546110e826607fda4ef25f17d2fb70bb3bbf733803298d3030e388c70a537d9"} Dec 03 12:10:38 crc kubenswrapper[5052]: I1203 12:10:38.105190 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-db-sync-nt4p4" event={"ID":"6d8c591d-96a8-4bef-908c-89ae9d1b46fb","Type":"ContainerStarted","Data":"4a9f1e5270a097c6309d7ac7f85258df79952aa39f9bc4466f0712f2e8d56364"} Dec 03 12:10:38 crc kubenswrapper[5052]: I1203 12:10:38.107076 5052 generic.go:334] "Generic (PLEG): container finished" podID="855f1c2d-3e1c-44b1-a691-41673832e6bc" containerID="c9905267b6558bc3332728a3891c7daa7a01ebb18411fc152bcfc514ff6b7b6e" exitCode=0 Dec 03 12:10:38 crc kubenswrapper[5052]: I1203 12:10:38.107136 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-ring-rebalance-zqqdm" event={"ID":"855f1c2d-3e1c-44b1-a691-41673832e6bc","Type":"ContainerDied","Data":"c9905267b6558bc3332728a3891c7daa7a01ebb18411fc152bcfc514ff6b7b6e"} Dec 03 12:10:38 crc kubenswrapper[5052]: I1203 12:10:38.135438 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/glance-db-sync-nt4p4" podStartSLOduration=2.135412503 podStartE2EDuration="2.135412503s" podCreationTimestamp="2025-12-03 12:10:36 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:10:38.122960097 +0000 UTC m=+3028.711201861" watchObservedRunningTime="2025-12-03 12:10:38.135412503 +0000 UTC m=+3028.723654287" Dec 03 12:10:39 crc kubenswrapper[5052]: I1203 12:10:39.466009 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-ring-rebalance-zqqdm" Dec 03 12:10:39 crc kubenswrapper[5052]: I1203 12:10:39.544035 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/empty-dir/855f1c2d-3e1c-44b1-a691-41673832e6bc-etc-swift\") pod \"855f1c2d-3e1c-44b1-a691-41673832e6bc\" (UID: \"855f1c2d-3e1c-44b1-a691-41673832e6bc\") " Dec 03 12:10:39 crc kubenswrapper[5052]: I1203 12:10:39.544097 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ring-data-devices\" (UniqueName: \"kubernetes.io/configmap/855f1c2d-3e1c-44b1-a691-41673832e6bc-ring-data-devices\") pod \"855f1c2d-3e1c-44b1-a691-41673832e6bc\" (UID: \"855f1c2d-3e1c-44b1-a691-41673832e6bc\") " Dec 03 12:10:39 crc kubenswrapper[5052]: I1203 12:10:39.544136 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"swiftconf\" (UniqueName: \"kubernetes.io/secret/855f1c2d-3e1c-44b1-a691-41673832e6bc-swiftconf\") pod \"855f1c2d-3e1c-44b1-a691-41673832e6bc\" (UID: \"855f1c2d-3e1c-44b1-a691-41673832e6bc\") " Dec 03 12:10:39 crc kubenswrapper[5052]: I1203 12:10:39.544208 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"dispersionconf\" (UniqueName: \"kubernetes.io/secret/855f1c2d-3e1c-44b1-a691-41673832e6bc-dispersionconf\") pod \"855f1c2d-3e1c-44b1-a691-41673832e6bc\" (UID: \"855f1c2d-3e1c-44b1-a691-41673832e6bc\") " Dec 03 12:10:39 crc kubenswrapper[5052]: I1203 12:10:39.544242 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/855f1c2d-3e1c-44b1-a691-41673832e6bc-scripts\") pod \"855f1c2d-3e1c-44b1-a691-41673832e6bc\" (UID: \"855f1c2d-3e1c-44b1-a691-41673832e6bc\") " Dec 03 12:10:39 crc kubenswrapper[5052]: I1203 12:10:39.544293 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/855f1c2d-3e1c-44b1-a691-41673832e6bc-combined-ca-bundle\") pod \"855f1c2d-3e1c-44b1-a691-41673832e6bc\" (UID: \"855f1c2d-3e1c-44b1-a691-41673832e6bc\") " Dec 03 12:10:39 crc kubenswrapper[5052]: I1203 12:10:39.544321 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z4gb7\" (UniqueName: \"kubernetes.io/projected/855f1c2d-3e1c-44b1-a691-41673832e6bc-kube-api-access-z4gb7\") pod \"855f1c2d-3e1c-44b1-a691-41673832e6bc\" (UID: \"855f1c2d-3e1c-44b1-a691-41673832e6bc\") " Dec 03 12:10:39 crc kubenswrapper[5052]: I1203 12:10:39.544985 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/855f1c2d-3e1c-44b1-a691-41673832e6bc-ring-data-devices" (OuterVolumeSpecName: "ring-data-devices") pod "855f1c2d-3e1c-44b1-a691-41673832e6bc" (UID: "855f1c2d-3e1c-44b1-a691-41673832e6bc"). InnerVolumeSpecName "ring-data-devices". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:10:39 crc kubenswrapper[5052]: I1203 12:10:39.545234 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/855f1c2d-3e1c-44b1-a691-41673832e6bc-etc-swift" (OuterVolumeSpecName: "etc-swift") pod "855f1c2d-3e1c-44b1-a691-41673832e6bc" (UID: "855f1c2d-3e1c-44b1-a691-41673832e6bc"). InnerVolumeSpecName "etc-swift". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:10:39 crc kubenswrapper[5052]: I1203 12:10:39.549848 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/855f1c2d-3e1c-44b1-a691-41673832e6bc-kube-api-access-z4gb7" (OuterVolumeSpecName: "kube-api-access-z4gb7") pod "855f1c2d-3e1c-44b1-a691-41673832e6bc" (UID: "855f1c2d-3e1c-44b1-a691-41673832e6bc"). InnerVolumeSpecName "kube-api-access-z4gb7". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:10:39 crc kubenswrapper[5052]: I1203 12:10:39.555361 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/855f1c2d-3e1c-44b1-a691-41673832e6bc-dispersionconf" (OuterVolumeSpecName: "dispersionconf") pod "855f1c2d-3e1c-44b1-a691-41673832e6bc" (UID: "855f1c2d-3e1c-44b1-a691-41673832e6bc"). InnerVolumeSpecName "dispersionconf". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:10:39 crc kubenswrapper[5052]: I1203 12:10:39.569842 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/855f1c2d-3e1c-44b1-a691-41673832e6bc-swiftconf" (OuterVolumeSpecName: "swiftconf") pod "855f1c2d-3e1c-44b1-a691-41673832e6bc" (UID: "855f1c2d-3e1c-44b1-a691-41673832e6bc"). InnerVolumeSpecName "swiftconf". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:10:39 crc kubenswrapper[5052]: I1203 12:10:39.570693 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/855f1c2d-3e1c-44b1-a691-41673832e6bc-scripts" (OuterVolumeSpecName: "scripts") pod "855f1c2d-3e1c-44b1-a691-41673832e6bc" (UID: "855f1c2d-3e1c-44b1-a691-41673832e6bc"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:10:39 crc kubenswrapper[5052]: I1203 12:10:39.572778 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/855f1c2d-3e1c-44b1-a691-41673832e6bc-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "855f1c2d-3e1c-44b1-a691-41673832e6bc" (UID: "855f1c2d-3e1c-44b1-a691-41673832e6bc"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:10:39 crc kubenswrapper[5052]: I1203 12:10:39.646602 5052 reconciler_common.go:293] "Volume detached for volume \"etc-swift\" (UniqueName: \"kubernetes.io/empty-dir/855f1c2d-3e1c-44b1-a691-41673832e6bc-etc-swift\") on node \"crc\" DevicePath \"\"" Dec 03 12:10:39 crc kubenswrapper[5052]: I1203 12:10:39.646645 5052 reconciler_common.go:293] "Volume detached for volume \"ring-data-devices\" (UniqueName: \"kubernetes.io/configmap/855f1c2d-3e1c-44b1-a691-41673832e6bc-ring-data-devices\") on node \"crc\" DevicePath \"\"" Dec 03 12:10:39 crc kubenswrapper[5052]: I1203 12:10:39.646660 5052 reconciler_common.go:293] "Volume detached for volume \"swiftconf\" (UniqueName: \"kubernetes.io/secret/855f1c2d-3e1c-44b1-a691-41673832e6bc-swiftconf\") on node \"crc\" DevicePath \"\"" Dec 03 12:10:39 crc kubenswrapper[5052]: I1203 12:10:39.646671 5052 reconciler_common.go:293] "Volume detached for volume \"dispersionconf\" (UniqueName: \"kubernetes.io/secret/855f1c2d-3e1c-44b1-a691-41673832e6bc-dispersionconf\") on node \"crc\" DevicePath \"\"" Dec 03 12:10:39 crc kubenswrapper[5052]: I1203 12:10:39.646681 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/855f1c2d-3e1c-44b1-a691-41673832e6bc-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:10:39 crc kubenswrapper[5052]: I1203 12:10:39.646691 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/855f1c2d-3e1c-44b1-a691-41673832e6bc-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:10:39 crc kubenswrapper[5052]: I1203 12:10:39.646703 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-z4gb7\" (UniqueName: \"kubernetes.io/projected/855f1c2d-3e1c-44b1-a691-41673832e6bc-kube-api-access-z4gb7\") on node \"crc\" DevicePath \"\"" Dec 03 12:10:39 crc kubenswrapper[5052]: I1203 12:10:39.748193 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/694f3a22-38d9-487d-b887-a915fa8f99b0-etc-swift\") pod \"swift-storage-0\" (UID: \"694f3a22-38d9-487d-b887-a915fa8f99b0\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:10:39 crc kubenswrapper[5052]: I1203 12:10:39.753457 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/694f3a22-38d9-487d-b887-a915fa8f99b0-etc-swift\") pod \"swift-storage-0\" (UID: \"694f3a22-38d9-487d-b887-a915fa8f99b0\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:10:40 crc kubenswrapper[5052]: I1203 12:10:40.007390 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:10:40 crc kubenswrapper[5052]: I1203 12:10:40.141058 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-ring-rebalance-zqqdm" event={"ID":"855f1c2d-3e1c-44b1-a691-41673832e6bc","Type":"ContainerDied","Data":"4902d4ad52d256c58ec77050171ae59fcff1a8ef50fc1fdb5c64acf7adcb43d4"} Dec 03 12:10:40 crc kubenswrapper[5052]: I1203 12:10:40.142410 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4902d4ad52d256c58ec77050171ae59fcff1a8ef50fc1fdb5c64acf7adcb43d4" Dec 03 12:10:40 crc kubenswrapper[5052]: I1203 12:10:40.141406 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-ring-rebalance-zqqdm" Dec 03 12:10:40 crc kubenswrapper[5052]: I1203 12:10:40.516064 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/swift-storage-0"] Dec 03 12:10:40 crc kubenswrapper[5052]: W1203 12:10:40.521699 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod694f3a22_38d9_487d_b887_a915fa8f99b0.slice/crio-cade3940c863bac7871d97d10ee1b99b6179e8e161a970d2eca55f430665b17b WatchSource:0}: Error finding container cade3940c863bac7871d97d10ee1b99b6179e8e161a970d2eca55f430665b17b: Status 404 returned error can't find the container with id cade3940c863bac7871d97d10ee1b99b6179e8e161a970d2eca55f430665b17b Dec 03 12:10:40 crc kubenswrapper[5052]: I1203 12:10:40.783303 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:10:41 crc kubenswrapper[5052]: I1203 12:10:41.153114 5052 generic.go:334] "Generic (PLEG): container finished" podID="e63eb1eb-e0b9-46ea-a19b-8f5c018728ff" containerID="e2d7edf07d19fee0525d0d93f60f9850672844f551c1265e9b21fb9528510d56" exitCode=0 Dec 03 12:10:41 crc kubenswrapper[5052]: I1203 12:10:41.153275 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" event={"ID":"e63eb1eb-e0b9-46ea-a19b-8f5c018728ff","Type":"ContainerDied","Data":"e2d7edf07d19fee0525d0d93f60f9850672844f551c1265e9b21fb9528510d56"} Dec 03 12:10:41 crc kubenswrapper[5052]: I1203 12:10:41.158616 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"694f3a22-38d9-487d-b887-a915fa8f99b0","Type":"ContainerStarted","Data":"daff85236f8b1825cad56f8cf5a450330943257c15e8a198c3cbd529f86943a9"} Dec 03 12:10:41 crc kubenswrapper[5052]: I1203 12:10:41.158665 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"694f3a22-38d9-487d-b887-a915fa8f99b0","Type":"ContainerStarted","Data":"52c9d3753f1d83dc61693ea154f30426c527a127d3d6f00ae6379a1cbd9e5899"} Dec 03 12:10:41 crc kubenswrapper[5052]: I1203 12:10:41.158675 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"694f3a22-38d9-487d-b887-a915fa8f99b0","Type":"ContainerStarted","Data":"cade3940c863bac7871d97d10ee1b99b6179e8e161a970d2eca55f430665b17b"} Dec 03 12:10:42 crc kubenswrapper[5052]: I1203 12:10:42.168744 5052 generic.go:334] "Generic (PLEG): container finished" podID="3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f" containerID="a60401c5cce1c04ba232d9ed1392bdc059663fb8d3f36429c2b1a642a6fc8fdf" exitCode=0 Dec 03 12:10:42 crc kubenswrapper[5052]: I1203 12:10:42.168820 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-server-0" event={"ID":"3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f","Type":"ContainerDied","Data":"a60401c5cce1c04ba232d9ed1392bdc059663fb8d3f36429c2b1a642a6fc8fdf"} Dec 03 12:10:42 crc kubenswrapper[5052]: I1203 12:10:42.173018 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" event={"ID":"e63eb1eb-e0b9-46ea-a19b-8f5c018728ff","Type":"ContainerStarted","Data":"114f5fc894c9a2d64f9ed86ad645c9cb54b0a4c854a1d0287b2c2a82e3627521"} Dec 03 12:10:42 crc kubenswrapper[5052]: I1203 12:10:42.173283 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:10:42 crc kubenswrapper[5052]: I1203 12:10:42.177039 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"694f3a22-38d9-487d-b887-a915fa8f99b0","Type":"ContainerStarted","Data":"4be1f7ec0e5ec2598abc5fccb1e3a336e6d2ce904a40e4470bfebd9a2432aa0d"} Dec 03 12:10:42 crc kubenswrapper[5052]: I1203 12:10:42.177073 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"694f3a22-38d9-487d-b887-a915fa8f99b0","Type":"ContainerStarted","Data":"ffacc5bf5e6a05d80223ac48d049c379298b3d4840c2186fb767cc10e1492580"} Dec 03 12:10:42 crc kubenswrapper[5052]: I1203 12:10:42.177085 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"694f3a22-38d9-487d-b887-a915fa8f99b0","Type":"ContainerStarted","Data":"9abaf8aac6af7b9078e0f7fa87225ebcfdfa4003e45df36fe94117f331cca24b"} Dec 03 12:10:42 crc kubenswrapper[5052]: I1203 12:10:42.177094 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"694f3a22-38d9-487d-b887-a915fa8f99b0","Type":"ContainerStarted","Data":"8de12c6937f84a484a57da6623c0b720805ae449cf86cbf7781810fff5f7dbd6"} Dec 03 12:10:42 crc kubenswrapper[5052]: I1203 12:10:42.235977 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" podStartSLOduration=36.235029619 podStartE2EDuration="36.235029619s" podCreationTimestamp="2025-12-03 12:10:06 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:10:42.229936984 +0000 UTC m=+3032.818178748" watchObservedRunningTime="2025-12-03 12:10:42.235029619 +0000 UTC m=+3032.823271393" Dec 03 12:10:43 crc kubenswrapper[5052]: I1203 12:10:43.189006 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-server-0" event={"ID":"3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f","Type":"ContainerStarted","Data":"353f13dfb90cbf0598b28536ad392c1afc49332edb49e65bfaddb985602e41d1"} Dec 03 12:10:43 crc kubenswrapper[5052]: I1203 12:10:43.189290 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:10:43 crc kubenswrapper[5052]: I1203 12:10:43.195206 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"694f3a22-38d9-487d-b887-a915fa8f99b0","Type":"ContainerStarted","Data":"edccdb7e982ee726341d6181308a200c1c7f7e489199aae3ff7897c6686d1e53"} Dec 03 12:10:43 crc kubenswrapper[5052]: I1203 12:10:43.195267 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"694f3a22-38d9-487d-b887-a915fa8f99b0","Type":"ContainerStarted","Data":"bfc910b016024c52ad21685e4f00e1ebf323cb412cbdac44e180ce0af3b5c974"} Dec 03 12:10:43 crc kubenswrapper[5052]: I1203 12:10:43.195282 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"694f3a22-38d9-487d-b887-a915fa8f99b0","Type":"ContainerStarted","Data":"3577cb2e84ebe92b200c0bf67af07582c963a380e9d15f90356b2162da1491e3"} Dec 03 12:10:43 crc kubenswrapper[5052]: I1203 12:10:43.195292 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"694f3a22-38d9-487d-b887-a915fa8f99b0","Type":"ContainerStarted","Data":"a9d69281c8f09f62933866859d2a7f994441cb27231474ecb5c4087f61ef9e48"} Dec 03 12:10:43 crc kubenswrapper[5052]: I1203 12:10:43.231702 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/rabbitmq-server-0" podStartSLOduration=38.231686997 podStartE2EDuration="38.231686997s" podCreationTimestamp="2025-12-03 12:10:05 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:10:43.220696783 +0000 UTC m=+3033.808938547" watchObservedRunningTime="2025-12-03 12:10:43.231686997 +0000 UTC m=+3033.819928761" Dec 03 12:10:44 crc kubenswrapper[5052]: I1203 12:10:44.211830 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"694f3a22-38d9-487d-b887-a915fa8f99b0","Type":"ContainerStarted","Data":"2b4fde6dfdf88033dd51e40f592f3092bbadafedc12b51e3dac6d06c7be816e2"} Dec 03 12:10:44 crc kubenswrapper[5052]: I1203 12:10:44.212227 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"694f3a22-38d9-487d-b887-a915fa8f99b0","Type":"ContainerStarted","Data":"d34ac8e54a2a8c78fedd6aae86ea4f7c01fa3fb75b7877784aed4c550716ec17"} Dec 03 12:10:45 crc kubenswrapper[5052]: I1203 12:10:45.227082 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"694f3a22-38d9-487d-b887-a915fa8f99b0","Type":"ContainerStarted","Data":"d7754f88aed158a69d3f255fe77907743b8120c66e854103907d2301b4d085dd"} Dec 03 12:10:45 crc kubenswrapper[5052]: I1203 12:10:45.227479 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"694f3a22-38d9-487d-b887-a915fa8f99b0","Type":"ContainerStarted","Data":"4ca9b25da15ecf1176a45c4f622d417575fd72f1258e4941eaab46adc37d9954"} Dec 03 12:10:45 crc kubenswrapper[5052]: I1203 12:10:45.227496 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"694f3a22-38d9-487d-b887-a915fa8f99b0","Type":"ContainerStarted","Data":"c8a4d6f709c57c2c8c86b3ead5f40b5eed433a688c9bda706475d1a83dc38289"} Dec 03 12:10:45 crc kubenswrapper[5052]: I1203 12:10:45.272486 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/swift-storage-0" podStartSLOduration=23.272460536 podStartE2EDuration="23.272460536s" podCreationTimestamp="2025-12-03 12:10:22 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:10:45.259247899 +0000 UTC m=+3035.847489663" watchObservedRunningTime="2025-12-03 12:10:45.272460536 +0000 UTC m=+3035.860702300" Dec 03 12:10:45 crc kubenswrapper[5052]: I1203 12:10:45.428322 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-6bcdfcd6b5-mhvjp"] Dec 03 12:10:45 crc kubenswrapper[5052]: E1203 12:10:45.428686 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="855f1c2d-3e1c-44b1-a691-41673832e6bc" containerName="swift-ring-rebalance" Dec 03 12:10:45 crc kubenswrapper[5052]: I1203 12:10:45.428704 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="855f1c2d-3e1c-44b1-a691-41673832e6bc" containerName="swift-ring-rebalance" Dec 03 12:10:45 crc kubenswrapper[5052]: I1203 12:10:45.428891 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="855f1c2d-3e1c-44b1-a691-41673832e6bc" containerName="swift-ring-rebalance" Dec 03 12:10:45 crc kubenswrapper[5052]: I1203 12:10:45.429867 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-6bcdfcd6b5-mhvjp" Dec 03 12:10:45 crc kubenswrapper[5052]: I1203 12:10:45.435726 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"dns-swift-storage-0" Dec 03 12:10:45 crc kubenswrapper[5052]: I1203 12:10:45.439319 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-6bcdfcd6b5-mhvjp"] Dec 03 12:10:45 crc kubenswrapper[5052]: I1203 12:10:45.546073 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/9a25d0bb-42c4-4214-b81d-23b29f335155-config\") pod \"dnsmasq-dnsmasq-6bcdfcd6b5-mhvjp\" (UID: \"9a25d0bb-42c4-4214-b81d-23b29f335155\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-6bcdfcd6b5-mhvjp" Dec 03 12:10:45 crc kubenswrapper[5052]: I1203 12:10:45.546140 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9wblq\" (UniqueName: \"kubernetes.io/projected/9a25d0bb-42c4-4214-b81d-23b29f335155-kube-api-access-9wblq\") pod \"dnsmasq-dnsmasq-6bcdfcd6b5-mhvjp\" (UID: \"9a25d0bb-42c4-4214-b81d-23b29f335155\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-6bcdfcd6b5-mhvjp" Dec 03 12:10:45 crc kubenswrapper[5052]: I1203 12:10:45.546194 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/9a25d0bb-42c4-4214-b81d-23b29f335155-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-6bcdfcd6b5-mhvjp\" (UID: \"9a25d0bb-42c4-4214-b81d-23b29f335155\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-6bcdfcd6b5-mhvjp" Dec 03 12:10:45 crc kubenswrapper[5052]: I1203 12:10:45.546234 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dns-swift-storage-0\" (UniqueName: \"kubernetes.io/configmap/9a25d0bb-42c4-4214-b81d-23b29f335155-dns-swift-storage-0\") pod \"dnsmasq-dnsmasq-6bcdfcd6b5-mhvjp\" (UID: \"9a25d0bb-42c4-4214-b81d-23b29f335155\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-6bcdfcd6b5-mhvjp" Dec 03 12:10:45 crc kubenswrapper[5052]: I1203 12:10:45.647901 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/9a25d0bb-42c4-4214-b81d-23b29f335155-config\") pod \"dnsmasq-dnsmasq-6bcdfcd6b5-mhvjp\" (UID: \"9a25d0bb-42c4-4214-b81d-23b29f335155\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-6bcdfcd6b5-mhvjp" Dec 03 12:10:45 crc kubenswrapper[5052]: I1203 12:10:45.647985 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-9wblq\" (UniqueName: \"kubernetes.io/projected/9a25d0bb-42c4-4214-b81d-23b29f335155-kube-api-access-9wblq\") pod \"dnsmasq-dnsmasq-6bcdfcd6b5-mhvjp\" (UID: \"9a25d0bb-42c4-4214-b81d-23b29f335155\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-6bcdfcd6b5-mhvjp" Dec 03 12:10:45 crc kubenswrapper[5052]: I1203 12:10:45.648034 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/9a25d0bb-42c4-4214-b81d-23b29f335155-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-6bcdfcd6b5-mhvjp\" (UID: \"9a25d0bb-42c4-4214-b81d-23b29f335155\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-6bcdfcd6b5-mhvjp" Dec 03 12:10:45 crc kubenswrapper[5052]: I1203 12:10:45.648089 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"dns-swift-storage-0\" (UniqueName: \"kubernetes.io/configmap/9a25d0bb-42c4-4214-b81d-23b29f335155-dns-swift-storage-0\") pod \"dnsmasq-dnsmasq-6bcdfcd6b5-mhvjp\" (UID: \"9a25d0bb-42c4-4214-b81d-23b29f335155\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-6bcdfcd6b5-mhvjp" Dec 03 12:10:45 crc kubenswrapper[5052]: I1203 12:10:45.649348 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"dns-swift-storage-0\" (UniqueName: \"kubernetes.io/configmap/9a25d0bb-42c4-4214-b81d-23b29f335155-dns-swift-storage-0\") pod \"dnsmasq-dnsmasq-6bcdfcd6b5-mhvjp\" (UID: \"9a25d0bb-42c4-4214-b81d-23b29f335155\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-6bcdfcd6b5-mhvjp" Dec 03 12:10:45 crc kubenswrapper[5052]: I1203 12:10:45.649925 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/9a25d0bb-42c4-4214-b81d-23b29f335155-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-6bcdfcd6b5-mhvjp\" (UID: \"9a25d0bb-42c4-4214-b81d-23b29f335155\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-6bcdfcd6b5-mhvjp" Dec 03 12:10:45 crc kubenswrapper[5052]: I1203 12:10:45.649925 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/9a25d0bb-42c4-4214-b81d-23b29f335155-config\") pod \"dnsmasq-dnsmasq-6bcdfcd6b5-mhvjp\" (UID: \"9a25d0bb-42c4-4214-b81d-23b29f335155\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-6bcdfcd6b5-mhvjp" Dec 03 12:10:45 crc kubenswrapper[5052]: I1203 12:10:45.678700 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-9wblq\" (UniqueName: \"kubernetes.io/projected/9a25d0bb-42c4-4214-b81d-23b29f335155-kube-api-access-9wblq\") pod \"dnsmasq-dnsmasq-6bcdfcd6b5-mhvjp\" (UID: \"9a25d0bb-42c4-4214-b81d-23b29f335155\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-6bcdfcd6b5-mhvjp" Dec 03 12:10:45 crc kubenswrapper[5052]: I1203 12:10:45.758583 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-6bcdfcd6b5-mhvjp" Dec 03 12:10:46 crc kubenswrapper[5052]: I1203 12:10:46.223907 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-6bcdfcd6b5-mhvjp"] Dec 03 12:10:46 crc kubenswrapper[5052]: I1203 12:10:46.242841 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-6bcdfcd6b5-mhvjp" event={"ID":"9a25d0bb-42c4-4214-b81d-23b29f335155","Type":"ContainerStarted","Data":"b5f99f908aae3482cc53681f026be2b723bc889ecebc87531a0e82daae1e7264"} Dec 03 12:10:47 crc kubenswrapper[5052]: I1203 12:10:47.280526 5052 generic.go:334] "Generic (PLEG): container finished" podID="9a25d0bb-42c4-4214-b81d-23b29f335155" containerID="4912275ac49c2d4e0e4d1b6267954dc550f95013e82ac3cf4070fdd4ba7e5da6" exitCode=0 Dec 03 12:10:47 crc kubenswrapper[5052]: I1203 12:10:47.280653 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-6bcdfcd6b5-mhvjp" event={"ID":"9a25d0bb-42c4-4214-b81d-23b29f335155","Type":"ContainerDied","Data":"4912275ac49c2d4e0e4d1b6267954dc550f95013e82ac3cf4070fdd4ba7e5da6"} Dec 03 12:10:47 crc kubenswrapper[5052]: I1203 12:10:47.283131 5052 generic.go:334] "Generic (PLEG): container finished" podID="6d8c591d-96a8-4bef-908c-89ae9d1b46fb" containerID="5546110e826607fda4ef25f17d2fb70bb3bbf733803298d3030e388c70a537d9" exitCode=0 Dec 03 12:10:47 crc kubenswrapper[5052]: I1203 12:10:47.283201 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-db-sync-nt4p4" event={"ID":"6d8c591d-96a8-4bef-908c-89ae9d1b46fb","Type":"ContainerDied","Data":"5546110e826607fda4ef25f17d2fb70bb3bbf733803298d3030e388c70a537d9"} Dec 03 12:10:47 crc kubenswrapper[5052]: I1203 12:10:47.673124 5052 scope.go:117] "RemoveContainer" containerID="63298107b3d090deff9ed92f3d613e5b1c86d9137684d117500257db8ee1c805" Dec 03 12:10:47 crc kubenswrapper[5052]: E1203 12:10:47.673623 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 12:10:48 crc kubenswrapper[5052]: I1203 12:10:48.295747 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-6bcdfcd6b5-mhvjp" event={"ID":"9a25d0bb-42c4-4214-b81d-23b29f335155","Type":"ContainerStarted","Data":"f2236846a20b2c05e69f531a234547ed762dc4e7260c85c9dc2d7a23360fd2cf"} Dec 03 12:10:48 crc kubenswrapper[5052]: I1203 12:10:48.327378 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-6bcdfcd6b5-mhvjp" podStartSLOduration=3.327354732 podStartE2EDuration="3.327354732s" podCreationTimestamp="2025-12-03 12:10:45 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:10:48.322117793 +0000 UTC m=+3038.910359557" watchObservedRunningTime="2025-12-03 12:10:48.327354732 +0000 UTC m=+3038.915596496" Dec 03 12:10:48 crc kubenswrapper[5052]: I1203 12:10:48.708405 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-db-sync-nt4p4" Dec 03 12:10:48 crc kubenswrapper[5052]: I1203 12:10:48.810611 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6d8c591d-96a8-4bef-908c-89ae9d1b46fb-combined-ca-bundle\") pod \"6d8c591d-96a8-4bef-908c-89ae9d1b46fb\" (UID: \"6d8c591d-96a8-4bef-908c-89ae9d1b46fb\") " Dec 03 12:10:48 crc kubenswrapper[5052]: I1203 12:10:48.810682 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/6d8c591d-96a8-4bef-908c-89ae9d1b46fb-db-sync-config-data\") pod \"6d8c591d-96a8-4bef-908c-89ae9d1b46fb\" (UID: \"6d8c591d-96a8-4bef-908c-89ae9d1b46fb\") " Dec 03 12:10:48 crc kubenswrapper[5052]: I1203 12:10:48.810718 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wd26w\" (UniqueName: \"kubernetes.io/projected/6d8c591d-96a8-4bef-908c-89ae9d1b46fb-kube-api-access-wd26w\") pod \"6d8c591d-96a8-4bef-908c-89ae9d1b46fb\" (UID: \"6d8c591d-96a8-4bef-908c-89ae9d1b46fb\") " Dec 03 12:10:48 crc kubenswrapper[5052]: I1203 12:10:48.810787 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6d8c591d-96a8-4bef-908c-89ae9d1b46fb-config-data\") pod \"6d8c591d-96a8-4bef-908c-89ae9d1b46fb\" (UID: \"6d8c591d-96a8-4bef-908c-89ae9d1b46fb\") " Dec 03 12:10:48 crc kubenswrapper[5052]: I1203 12:10:48.823755 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6d8c591d-96a8-4bef-908c-89ae9d1b46fb-kube-api-access-wd26w" (OuterVolumeSpecName: "kube-api-access-wd26w") pod "6d8c591d-96a8-4bef-908c-89ae9d1b46fb" (UID: "6d8c591d-96a8-4bef-908c-89ae9d1b46fb"). InnerVolumeSpecName "kube-api-access-wd26w". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:10:48 crc kubenswrapper[5052]: I1203 12:10:48.838588 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6d8c591d-96a8-4bef-908c-89ae9d1b46fb-db-sync-config-data" (OuterVolumeSpecName: "db-sync-config-data") pod "6d8c591d-96a8-4bef-908c-89ae9d1b46fb" (UID: "6d8c591d-96a8-4bef-908c-89ae9d1b46fb"). InnerVolumeSpecName "db-sync-config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:10:48 crc kubenswrapper[5052]: I1203 12:10:48.839655 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6d8c591d-96a8-4bef-908c-89ae9d1b46fb-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "6d8c591d-96a8-4bef-908c-89ae9d1b46fb" (UID: "6d8c591d-96a8-4bef-908c-89ae9d1b46fb"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:10:48 crc kubenswrapper[5052]: I1203 12:10:48.868389 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6d8c591d-96a8-4bef-908c-89ae9d1b46fb-config-data" (OuterVolumeSpecName: "config-data") pod "6d8c591d-96a8-4bef-908c-89ae9d1b46fb" (UID: "6d8c591d-96a8-4bef-908c-89ae9d1b46fb"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:10:48 crc kubenswrapper[5052]: I1203 12:10:48.914632 5052 reconciler_common.go:293] "Volume detached for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/6d8c591d-96a8-4bef-908c-89ae9d1b46fb-db-sync-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:10:48 crc kubenswrapper[5052]: I1203 12:10:48.914662 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-wd26w\" (UniqueName: \"kubernetes.io/projected/6d8c591d-96a8-4bef-908c-89ae9d1b46fb-kube-api-access-wd26w\") on node \"crc\" DevicePath \"\"" Dec 03 12:10:48 crc kubenswrapper[5052]: I1203 12:10:48.914672 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6d8c591d-96a8-4bef-908c-89ae9d1b46fb-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:10:48 crc kubenswrapper[5052]: I1203 12:10:48.914680 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6d8c591d-96a8-4bef-908c-89ae9d1b46fb-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:10:49 crc kubenswrapper[5052]: I1203 12:10:49.306127 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-db-sync-nt4p4" event={"ID":"6d8c591d-96a8-4bef-908c-89ae9d1b46fb","Type":"ContainerDied","Data":"4a9f1e5270a097c6309d7ac7f85258df79952aa39f9bc4466f0712f2e8d56364"} Dec 03 12:10:49 crc kubenswrapper[5052]: I1203 12:10:49.306160 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-db-sync-nt4p4" Dec 03 12:10:49 crc kubenswrapper[5052]: I1203 12:10:49.306180 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4a9f1e5270a097c6309d7ac7f85258df79952aa39f9bc4466f0712f2e8d56364" Dec 03 12:10:49 crc kubenswrapper[5052]: I1203 12:10:49.306295 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-6bcdfcd6b5-mhvjp" Dec 03 12:10:55 crc kubenswrapper[5052]: I1203 12:10:55.760909 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-6bcdfcd6b5-mhvjp" Dec 03 12:10:55 crc kubenswrapper[5052]: I1203 12:10:55.819823 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-kcb42"] Dec 03 12:10:55 crc kubenswrapper[5052]: I1203 12:10:55.820058 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-kcb42" podUID="f657b0c3-883e-4e3d-b161-1b3ef0988ab0" containerName="dnsmasq-dns" containerID="cri-o://5cb3fa6d03cfa9a06f36028d84a28f6c0d21f3556efec764d29b063d223df34a" gracePeriod=10 Dec 03 12:10:56 crc kubenswrapper[5052]: I1203 12:10:56.367366 5052 generic.go:334] "Generic (PLEG): container finished" podID="f657b0c3-883e-4e3d-b161-1b3ef0988ab0" containerID="5cb3fa6d03cfa9a06f36028d84a28f6c0d21f3556efec764d29b063d223df34a" exitCode=0 Dec 03 12:10:56 crc kubenswrapper[5052]: I1203 12:10:56.367473 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-kcb42" event={"ID":"f657b0c3-883e-4e3d-b161-1b3ef0988ab0","Type":"ContainerDied","Data":"5cb3fa6d03cfa9a06f36028d84a28f6c0d21f3556efec764d29b063d223df34a"} Dec 03 12:10:56 crc kubenswrapper[5052]: I1203 12:10:56.886813 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-kcb42" Dec 03 12:10:56 crc kubenswrapper[5052]: I1203 12:10:56.956181 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-47gfp\" (UniqueName: \"kubernetes.io/projected/f657b0c3-883e-4e3d-b161-1b3ef0988ab0-kube-api-access-47gfp\") pod \"f657b0c3-883e-4e3d-b161-1b3ef0988ab0\" (UID: \"f657b0c3-883e-4e3d-b161-1b3ef0988ab0\") " Dec 03 12:10:56 crc kubenswrapper[5052]: I1203 12:10:56.963897 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/f657b0c3-883e-4e3d-b161-1b3ef0988ab0-dnsmasq-svc\") pod \"f657b0c3-883e-4e3d-b161-1b3ef0988ab0\" (UID: \"f657b0c3-883e-4e3d-b161-1b3ef0988ab0\") " Dec 03 12:10:56 crc kubenswrapper[5052]: I1203 12:10:56.964058 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f657b0c3-883e-4e3d-b161-1b3ef0988ab0-config\") pod \"f657b0c3-883e-4e3d-b161-1b3ef0988ab0\" (UID: \"f657b0c3-883e-4e3d-b161-1b3ef0988ab0\") " Dec 03 12:10:56 crc kubenswrapper[5052]: I1203 12:10:56.969905 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f657b0c3-883e-4e3d-b161-1b3ef0988ab0-kube-api-access-47gfp" (OuterVolumeSpecName: "kube-api-access-47gfp") pod "f657b0c3-883e-4e3d-b161-1b3ef0988ab0" (UID: "f657b0c3-883e-4e3d-b161-1b3ef0988ab0"). InnerVolumeSpecName "kube-api-access-47gfp". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:10:57 crc kubenswrapper[5052]: I1203 12:10:57.009822 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f657b0c3-883e-4e3d-b161-1b3ef0988ab0-config" (OuterVolumeSpecName: "config") pod "f657b0c3-883e-4e3d-b161-1b3ef0988ab0" (UID: "f657b0c3-883e-4e3d-b161-1b3ef0988ab0"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:10:57 crc kubenswrapper[5052]: I1203 12:10:57.010217 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f657b0c3-883e-4e3d-b161-1b3ef0988ab0-dnsmasq-svc" (OuterVolumeSpecName: "dnsmasq-svc") pod "f657b0c3-883e-4e3d-b161-1b3ef0988ab0" (UID: "f657b0c3-883e-4e3d-b161-1b3ef0988ab0"). InnerVolumeSpecName "dnsmasq-svc". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:10:57 crc kubenswrapper[5052]: I1203 12:10:57.066615 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-47gfp\" (UniqueName: \"kubernetes.io/projected/f657b0c3-883e-4e3d-b161-1b3ef0988ab0-kube-api-access-47gfp\") on node \"crc\" DevicePath \"\"" Dec 03 12:10:57 crc kubenswrapper[5052]: I1203 12:10:57.066646 5052 reconciler_common.go:293] "Volume detached for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/f657b0c3-883e-4e3d-b161-1b3ef0988ab0-dnsmasq-svc\") on node \"crc\" DevicePath \"\"" Dec 03 12:10:57 crc kubenswrapper[5052]: I1203 12:10:57.066655 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f657b0c3-883e-4e3d-b161-1b3ef0988ab0-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:10:57 crc kubenswrapper[5052]: I1203 12:10:57.352735 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:10:57 crc kubenswrapper[5052]: I1203 12:10:57.379173 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-kcb42" event={"ID":"f657b0c3-883e-4e3d-b161-1b3ef0988ab0","Type":"ContainerDied","Data":"4c8cb428cd412adcc11141dc9c500eb7f270887e14784e9522cf1ff216e1d58e"} Dec 03 12:10:57 crc kubenswrapper[5052]: I1203 12:10:57.379238 5052 scope.go:117] "RemoveContainer" containerID="5cb3fa6d03cfa9a06f36028d84a28f6c0d21f3556efec764d29b063d223df34a" Dec 03 12:10:57 crc kubenswrapper[5052]: I1203 12:10:57.379240 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-kcb42" Dec 03 12:10:57 crc kubenswrapper[5052]: I1203 12:10:57.419924 5052 scope.go:117] "RemoveContainer" containerID="686b357f3c7f38691d18d6fa7206dbd5e60dc6a7b6acf91c922e01438257cfeb" Dec 03 12:10:57 crc kubenswrapper[5052]: I1203 12:10:57.421947 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-kcb42"] Dec 03 12:10:57 crc kubenswrapper[5052]: I1203 12:10:57.430777 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-kcb42"] Dec 03 12:10:57 crc kubenswrapper[5052]: I1203 12:10:57.640374 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/barbican-db-create-bn2nj"] Dec 03 12:10:57 crc kubenswrapper[5052]: E1203 12:10:57.641021 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="f657b0c3-883e-4e3d-b161-1b3ef0988ab0" containerName="init" Dec 03 12:10:57 crc kubenswrapper[5052]: I1203 12:10:57.641045 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="f657b0c3-883e-4e3d-b161-1b3ef0988ab0" containerName="init" Dec 03 12:10:57 crc kubenswrapper[5052]: E1203 12:10:57.641067 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="f657b0c3-883e-4e3d-b161-1b3ef0988ab0" containerName="dnsmasq-dns" Dec 03 12:10:57 crc kubenswrapper[5052]: I1203 12:10:57.641076 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="f657b0c3-883e-4e3d-b161-1b3ef0988ab0" containerName="dnsmasq-dns" Dec 03 12:10:57 crc kubenswrapper[5052]: E1203 12:10:57.641122 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="6d8c591d-96a8-4bef-908c-89ae9d1b46fb" containerName="glance-db-sync" Dec 03 12:10:57 crc kubenswrapper[5052]: I1203 12:10:57.641131 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="6d8c591d-96a8-4bef-908c-89ae9d1b46fb" containerName="glance-db-sync" Dec 03 12:10:57 crc kubenswrapper[5052]: I1203 12:10:57.641368 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="6d8c591d-96a8-4bef-908c-89ae9d1b46fb" containerName="glance-db-sync" Dec 03 12:10:57 crc kubenswrapper[5052]: I1203 12:10:57.641394 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="f657b0c3-883e-4e3d-b161-1b3ef0988ab0" containerName="dnsmasq-dns" Dec 03 12:10:57 crc kubenswrapper[5052]: I1203 12:10:57.642102 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-db-create-bn2nj" Dec 03 12:10:57 crc kubenswrapper[5052]: I1203 12:10:57.657018 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-db-create-bn2nj"] Dec 03 12:10:57 crc kubenswrapper[5052]: I1203 12:10:57.687626 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f657b0c3-883e-4e3d-b161-1b3ef0988ab0" path="/var/lib/kubelet/pods/f657b0c3-883e-4e3d-b161-1b3ef0988ab0/volumes" Dec 03 12:10:57 crc kubenswrapper[5052]: I1203 12:10:57.739697 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/cinder-db-create-lqkc5"] Dec 03 12:10:57 crc kubenswrapper[5052]: I1203 12:10:57.741499 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-db-create-lqkc5" Dec 03 12:10:57 crc kubenswrapper[5052]: I1203 12:10:57.769713 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:10:57 crc kubenswrapper[5052]: I1203 12:10:57.770038 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/barbican-8992-account-create-update-gbt4t"] Dec 03 12:10:57 crc kubenswrapper[5052]: I1203 12:10:57.771557 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-8992-account-create-update-gbt4t" Dec 03 12:10:57 crc kubenswrapper[5052]: I1203 12:10:57.780447 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-khhp9\" (UniqueName: \"kubernetes.io/projected/d9613cb1-0439-44fb-a577-c621439a6328-kube-api-access-khhp9\") pod \"barbican-db-create-bn2nj\" (UID: \"d9613cb1-0439-44fb-a577-c621439a6328\") " pod="openstack-kuttl-tests/barbican-db-create-bn2nj" Dec 03 12:10:57 crc kubenswrapper[5052]: I1203 12:10:57.780722 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/d9613cb1-0439-44fb-a577-c621439a6328-operator-scripts\") pod \"barbican-db-create-bn2nj\" (UID: \"d9613cb1-0439-44fb-a577-c621439a6328\") " pod="openstack-kuttl-tests/barbican-db-create-bn2nj" Dec 03 12:10:57 crc kubenswrapper[5052]: I1203 12:10:57.781978 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"barbican-db-secret" Dec 03 12:10:57 crc kubenswrapper[5052]: I1203 12:10:57.786937 5052 scope.go:117] "RemoveContainer" containerID="f25bc6d11555c8eec8ce818399df69255f9b136b8146d57ac9a33a070d4dc79b" Dec 03 12:10:57 crc kubenswrapper[5052]: I1203 12:10:57.793695 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-db-create-lqkc5"] Dec 03 12:10:57 crc kubenswrapper[5052]: I1203 12:10:57.804675 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-8992-account-create-update-gbt4t"] Dec 03 12:10:57 crc kubenswrapper[5052]: I1203 12:10:57.874584 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/cinder-7878-account-create-update-cxttb"] Dec 03 12:10:57 crc kubenswrapper[5052]: I1203 12:10:57.875732 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-7878-account-create-update-cxttb" Dec 03 12:10:57 crc kubenswrapper[5052]: I1203 12:10:57.878267 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cinder-db-secret" Dec 03 12:10:57 crc kubenswrapper[5052]: I1203 12:10:57.898748 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-khhp9\" (UniqueName: \"kubernetes.io/projected/d9613cb1-0439-44fb-a577-c621439a6328-kube-api-access-khhp9\") pod \"barbican-db-create-bn2nj\" (UID: \"d9613cb1-0439-44fb-a577-c621439a6328\") " pod="openstack-kuttl-tests/barbican-db-create-bn2nj" Dec 03 12:10:57 crc kubenswrapper[5052]: I1203 12:10:57.898844 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-77pc7\" (UniqueName: \"kubernetes.io/projected/2047705d-ae83-4972-913b-868f2a623022-kube-api-access-77pc7\") pod \"cinder-db-create-lqkc5\" (UID: \"2047705d-ae83-4972-913b-868f2a623022\") " pod="openstack-kuttl-tests/cinder-db-create-lqkc5" Dec 03 12:10:57 crc kubenswrapper[5052]: I1203 12:10:57.898910 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/d9613cb1-0439-44fb-a577-c621439a6328-operator-scripts\") pod \"barbican-db-create-bn2nj\" (UID: \"d9613cb1-0439-44fb-a577-c621439a6328\") " pod="openstack-kuttl-tests/barbican-db-create-bn2nj" Dec 03 12:10:57 crc kubenswrapper[5052]: I1203 12:10:57.898937 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/7e802e69-975f-4748-a902-d3715174eefa-operator-scripts\") pod \"barbican-8992-account-create-update-gbt4t\" (UID: \"7e802e69-975f-4748-a902-d3715174eefa\") " pod="openstack-kuttl-tests/barbican-8992-account-create-update-gbt4t" Dec 03 12:10:57 crc kubenswrapper[5052]: I1203 12:10:57.898984 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dtvd6\" (UniqueName: \"kubernetes.io/projected/7e802e69-975f-4748-a902-d3715174eefa-kube-api-access-dtvd6\") pod \"barbican-8992-account-create-update-gbt4t\" (UID: \"7e802e69-975f-4748-a902-d3715174eefa\") " pod="openstack-kuttl-tests/barbican-8992-account-create-update-gbt4t" Dec 03 12:10:57 crc kubenswrapper[5052]: I1203 12:10:57.899024 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/2047705d-ae83-4972-913b-868f2a623022-operator-scripts\") pod \"cinder-db-create-lqkc5\" (UID: \"2047705d-ae83-4972-913b-868f2a623022\") " pod="openstack-kuttl-tests/cinder-db-create-lqkc5" Dec 03 12:10:57 crc kubenswrapper[5052]: I1203 12:10:57.901556 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/d9613cb1-0439-44fb-a577-c621439a6328-operator-scripts\") pod \"barbican-db-create-bn2nj\" (UID: \"d9613cb1-0439-44fb-a577-c621439a6328\") " pod="openstack-kuttl-tests/barbican-db-create-bn2nj" Dec 03 12:10:57 crc kubenswrapper[5052]: I1203 12:10:57.911411 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-7878-account-create-update-cxttb"] Dec 03 12:10:57 crc kubenswrapper[5052]: I1203 12:10:57.922206 5052 scope.go:117] "RemoveContainer" containerID="216f652b89ba64b000d5cd54f33f538c801f811bfa201b681b49eb486c4725d9" Dec 03 12:10:57 crc kubenswrapper[5052]: I1203 12:10:57.940571 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-khhp9\" (UniqueName: \"kubernetes.io/projected/d9613cb1-0439-44fb-a577-c621439a6328-kube-api-access-khhp9\") pod \"barbican-db-create-bn2nj\" (UID: \"d9613cb1-0439-44fb-a577-c621439a6328\") " pod="openstack-kuttl-tests/barbican-db-create-bn2nj" Dec 03 12:10:57 crc kubenswrapper[5052]: I1203 12:10:57.958583 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/neutron-db-create-2s5ld"] Dec 03 12:10:57 crc kubenswrapper[5052]: I1203 12:10:57.958871 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-db-create-bn2nj" Dec 03 12:10:57 crc kubenswrapper[5052]: I1203 12:10:57.959967 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-db-create-2s5ld" Dec 03 12:10:57 crc kubenswrapper[5052]: I1203 12:10:57.972100 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-db-create-2s5ld"] Dec 03 12:10:58 crc kubenswrapper[5052]: I1203 12:10:58.001113 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/2047705d-ae83-4972-913b-868f2a623022-operator-scripts\") pod \"cinder-db-create-lqkc5\" (UID: \"2047705d-ae83-4972-913b-868f2a623022\") " pod="openstack-kuttl-tests/cinder-db-create-lqkc5" Dec 03 12:10:58 crc kubenswrapper[5052]: I1203 12:10:58.001192 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/eae2f902-0630-43d2-9a05-cc0d68350068-operator-scripts\") pod \"cinder-7878-account-create-update-cxttb\" (UID: \"eae2f902-0630-43d2-9a05-cc0d68350068\") " pod="openstack-kuttl-tests/cinder-7878-account-create-update-cxttb" Dec 03 12:10:58 crc kubenswrapper[5052]: I1203 12:10:58.001221 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8c4mn\" (UniqueName: \"kubernetes.io/projected/eae2f902-0630-43d2-9a05-cc0d68350068-kube-api-access-8c4mn\") pod \"cinder-7878-account-create-update-cxttb\" (UID: \"eae2f902-0630-43d2-9a05-cc0d68350068\") " pod="openstack-kuttl-tests/cinder-7878-account-create-update-cxttb" Dec 03 12:10:58 crc kubenswrapper[5052]: I1203 12:10:58.001286 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-77pc7\" (UniqueName: \"kubernetes.io/projected/2047705d-ae83-4972-913b-868f2a623022-kube-api-access-77pc7\") pod \"cinder-db-create-lqkc5\" (UID: \"2047705d-ae83-4972-913b-868f2a623022\") " pod="openstack-kuttl-tests/cinder-db-create-lqkc5" Dec 03 12:10:58 crc kubenswrapper[5052]: I1203 12:10:58.001322 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/7e802e69-975f-4748-a902-d3715174eefa-operator-scripts\") pod \"barbican-8992-account-create-update-gbt4t\" (UID: \"7e802e69-975f-4748-a902-d3715174eefa\") " pod="openstack-kuttl-tests/barbican-8992-account-create-update-gbt4t" Dec 03 12:10:58 crc kubenswrapper[5052]: I1203 12:10:58.001367 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-dtvd6\" (UniqueName: \"kubernetes.io/projected/7e802e69-975f-4748-a902-d3715174eefa-kube-api-access-dtvd6\") pod \"barbican-8992-account-create-update-gbt4t\" (UID: \"7e802e69-975f-4748-a902-d3715174eefa\") " pod="openstack-kuttl-tests/barbican-8992-account-create-update-gbt4t" Dec 03 12:10:58 crc kubenswrapper[5052]: I1203 12:10:58.008876 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/7e802e69-975f-4748-a902-d3715174eefa-operator-scripts\") pod \"barbican-8992-account-create-update-gbt4t\" (UID: \"7e802e69-975f-4748-a902-d3715174eefa\") " pod="openstack-kuttl-tests/barbican-8992-account-create-update-gbt4t" Dec 03 12:10:58 crc kubenswrapper[5052]: I1203 12:10:58.011706 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/2047705d-ae83-4972-913b-868f2a623022-operator-scripts\") pod \"cinder-db-create-lqkc5\" (UID: \"2047705d-ae83-4972-913b-868f2a623022\") " pod="openstack-kuttl-tests/cinder-db-create-lqkc5" Dec 03 12:10:58 crc kubenswrapper[5052]: I1203 12:10:58.028283 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-dtvd6\" (UniqueName: \"kubernetes.io/projected/7e802e69-975f-4748-a902-d3715174eefa-kube-api-access-dtvd6\") pod \"barbican-8992-account-create-update-gbt4t\" (UID: \"7e802e69-975f-4748-a902-d3715174eefa\") " pod="openstack-kuttl-tests/barbican-8992-account-create-update-gbt4t" Dec 03 12:10:58 crc kubenswrapper[5052]: I1203 12:10:58.029326 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/keystone-db-sync-95hjc"] Dec 03 12:10:58 crc kubenswrapper[5052]: I1203 12:10:58.030749 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-db-sync-95hjc" Dec 03 12:10:58 crc kubenswrapper[5052]: I1203 12:10:58.039068 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone-keystone-dockercfg-4wch9" Dec 03 12:10:58 crc kubenswrapper[5052]: I1203 12:10:58.042942 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone-scripts" Dec 03 12:10:58 crc kubenswrapper[5052]: I1203 12:10:58.043201 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone-config-data" Dec 03 12:10:58 crc kubenswrapper[5052]: I1203 12:10:58.043411 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone" Dec 03 12:10:58 crc kubenswrapper[5052]: I1203 12:10:58.046205 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-77pc7\" (UniqueName: \"kubernetes.io/projected/2047705d-ae83-4972-913b-868f2a623022-kube-api-access-77pc7\") pod \"cinder-db-create-lqkc5\" (UID: \"2047705d-ae83-4972-913b-868f2a623022\") " pod="openstack-kuttl-tests/cinder-db-create-lqkc5" Dec 03 12:10:58 crc kubenswrapper[5052]: I1203 12:10:58.063425 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-db-sync-95hjc"] Dec 03 12:10:58 crc kubenswrapper[5052]: I1203 12:10:58.078277 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-db-create-lqkc5" Dec 03 12:10:58 crc kubenswrapper[5052]: I1203 12:10:58.087269 5052 scope.go:117] "RemoveContainer" containerID="1faba2664a45d58a12ee28b49eef362ded8c4da1f52ad06d789532a9d315d90a" Dec 03 12:10:58 crc kubenswrapper[5052]: I1203 12:10:58.092646 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-8992-account-create-update-gbt4t" Dec 03 12:10:58 crc kubenswrapper[5052]: I1203 12:10:58.108882 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/58c97692-2e69-4232-ba13-23806ab9fb67-combined-ca-bundle\") pod \"keystone-db-sync-95hjc\" (UID: \"58c97692-2e69-4232-ba13-23806ab9fb67\") " pod="openstack-kuttl-tests/keystone-db-sync-95hjc" Dec 03 12:10:58 crc kubenswrapper[5052]: I1203 12:10:58.109155 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/fb88d1bf-1334-41ec-8605-e93a852623b3-operator-scripts\") pod \"neutron-db-create-2s5ld\" (UID: \"fb88d1bf-1334-41ec-8605-e93a852623b3\") " pod="openstack-kuttl-tests/neutron-db-create-2s5ld" Dec 03 12:10:58 crc kubenswrapper[5052]: I1203 12:10:58.109423 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/eae2f902-0630-43d2-9a05-cc0d68350068-operator-scripts\") pod \"cinder-7878-account-create-update-cxttb\" (UID: \"eae2f902-0630-43d2-9a05-cc0d68350068\") " pod="openstack-kuttl-tests/cinder-7878-account-create-update-cxttb" Dec 03 12:10:58 crc kubenswrapper[5052]: I1203 12:10:58.109543 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-8c4mn\" (UniqueName: \"kubernetes.io/projected/eae2f902-0630-43d2-9a05-cc0d68350068-kube-api-access-8c4mn\") pod \"cinder-7878-account-create-update-cxttb\" (UID: \"eae2f902-0630-43d2-9a05-cc0d68350068\") " pod="openstack-kuttl-tests/cinder-7878-account-create-update-cxttb" Dec 03 12:10:58 crc kubenswrapper[5052]: I1203 12:10:58.109654 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/58c97692-2e69-4232-ba13-23806ab9fb67-config-data\") pod \"keystone-db-sync-95hjc\" (UID: \"58c97692-2e69-4232-ba13-23806ab9fb67\") " pod="openstack-kuttl-tests/keystone-db-sync-95hjc" Dec 03 12:10:58 crc kubenswrapper[5052]: I1203 12:10:58.109760 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8clcj\" (UniqueName: \"kubernetes.io/projected/fb88d1bf-1334-41ec-8605-e93a852623b3-kube-api-access-8clcj\") pod \"neutron-db-create-2s5ld\" (UID: \"fb88d1bf-1334-41ec-8605-e93a852623b3\") " pod="openstack-kuttl-tests/neutron-db-create-2s5ld" Dec 03 12:10:58 crc kubenswrapper[5052]: I1203 12:10:58.109896 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nclx4\" (UniqueName: \"kubernetes.io/projected/58c97692-2e69-4232-ba13-23806ab9fb67-kube-api-access-nclx4\") pod \"keystone-db-sync-95hjc\" (UID: \"58c97692-2e69-4232-ba13-23806ab9fb67\") " pod="openstack-kuttl-tests/keystone-db-sync-95hjc" Dec 03 12:10:58 crc kubenswrapper[5052]: I1203 12:10:58.110709 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/eae2f902-0630-43d2-9a05-cc0d68350068-operator-scripts\") pod \"cinder-7878-account-create-update-cxttb\" (UID: \"eae2f902-0630-43d2-9a05-cc0d68350068\") " pod="openstack-kuttl-tests/cinder-7878-account-create-update-cxttb" Dec 03 12:10:58 crc kubenswrapper[5052]: I1203 12:10:58.151599 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/neutron-1287-account-create-update-56qp8"] Dec 03 12:10:58 crc kubenswrapper[5052]: I1203 12:10:58.157919 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-1287-account-create-update-56qp8" Dec 03 12:10:58 crc kubenswrapper[5052]: I1203 12:10:58.160877 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"neutron-db-secret" Dec 03 12:10:58 crc kubenswrapper[5052]: I1203 12:10:58.167134 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-8c4mn\" (UniqueName: \"kubernetes.io/projected/eae2f902-0630-43d2-9a05-cc0d68350068-kube-api-access-8c4mn\") pod \"cinder-7878-account-create-update-cxttb\" (UID: \"eae2f902-0630-43d2-9a05-cc0d68350068\") " pod="openstack-kuttl-tests/cinder-7878-account-create-update-cxttb" Dec 03 12:10:58 crc kubenswrapper[5052]: I1203 12:10:58.185737 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-1287-account-create-update-56qp8"] Dec 03 12:10:58 crc kubenswrapper[5052]: I1203 12:10:58.212977 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/58c97692-2e69-4232-ba13-23806ab9fb67-combined-ca-bundle\") pod \"keystone-db-sync-95hjc\" (UID: \"58c97692-2e69-4232-ba13-23806ab9fb67\") " pod="openstack-kuttl-tests/keystone-db-sync-95hjc" Dec 03 12:10:58 crc kubenswrapper[5052]: I1203 12:10:58.213032 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/fb88d1bf-1334-41ec-8605-e93a852623b3-operator-scripts\") pod \"neutron-db-create-2s5ld\" (UID: \"fb88d1bf-1334-41ec-8605-e93a852623b3\") " pod="openstack-kuttl-tests/neutron-db-create-2s5ld" Dec 03 12:10:58 crc kubenswrapper[5052]: I1203 12:10:58.213100 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/884268a1-ff3a-4aaf-8004-89bb72e79d90-operator-scripts\") pod \"neutron-1287-account-create-update-56qp8\" (UID: \"884268a1-ff3a-4aaf-8004-89bb72e79d90\") " pod="openstack-kuttl-tests/neutron-1287-account-create-update-56qp8" Dec 03 12:10:58 crc kubenswrapper[5052]: I1203 12:10:58.213136 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/58c97692-2e69-4232-ba13-23806ab9fb67-config-data\") pod \"keystone-db-sync-95hjc\" (UID: \"58c97692-2e69-4232-ba13-23806ab9fb67\") " pod="openstack-kuttl-tests/keystone-db-sync-95hjc" Dec 03 12:10:58 crc kubenswrapper[5052]: I1203 12:10:58.213163 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-8clcj\" (UniqueName: \"kubernetes.io/projected/fb88d1bf-1334-41ec-8605-e93a852623b3-kube-api-access-8clcj\") pod \"neutron-db-create-2s5ld\" (UID: \"fb88d1bf-1334-41ec-8605-e93a852623b3\") " pod="openstack-kuttl-tests/neutron-db-create-2s5ld" Dec 03 12:10:58 crc kubenswrapper[5052]: I1203 12:10:58.213183 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4rgwc\" (UniqueName: \"kubernetes.io/projected/884268a1-ff3a-4aaf-8004-89bb72e79d90-kube-api-access-4rgwc\") pod \"neutron-1287-account-create-update-56qp8\" (UID: \"884268a1-ff3a-4aaf-8004-89bb72e79d90\") " pod="openstack-kuttl-tests/neutron-1287-account-create-update-56qp8" Dec 03 12:10:58 crc kubenswrapper[5052]: I1203 12:10:58.213217 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-nclx4\" (UniqueName: \"kubernetes.io/projected/58c97692-2e69-4232-ba13-23806ab9fb67-kube-api-access-nclx4\") pod \"keystone-db-sync-95hjc\" (UID: \"58c97692-2e69-4232-ba13-23806ab9fb67\") " pod="openstack-kuttl-tests/keystone-db-sync-95hjc" Dec 03 12:10:58 crc kubenswrapper[5052]: I1203 12:10:58.215254 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/fb88d1bf-1334-41ec-8605-e93a852623b3-operator-scripts\") pod \"neutron-db-create-2s5ld\" (UID: \"fb88d1bf-1334-41ec-8605-e93a852623b3\") " pod="openstack-kuttl-tests/neutron-db-create-2s5ld" Dec 03 12:10:58 crc kubenswrapper[5052]: I1203 12:10:58.217597 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/58c97692-2e69-4232-ba13-23806ab9fb67-config-data\") pod \"keystone-db-sync-95hjc\" (UID: \"58c97692-2e69-4232-ba13-23806ab9fb67\") " pod="openstack-kuttl-tests/keystone-db-sync-95hjc" Dec 03 12:10:58 crc kubenswrapper[5052]: I1203 12:10:58.218121 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/58c97692-2e69-4232-ba13-23806ab9fb67-combined-ca-bundle\") pod \"keystone-db-sync-95hjc\" (UID: \"58c97692-2e69-4232-ba13-23806ab9fb67\") " pod="openstack-kuttl-tests/keystone-db-sync-95hjc" Dec 03 12:10:58 crc kubenswrapper[5052]: I1203 12:10:58.224554 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-7878-account-create-update-cxttb" Dec 03 12:10:58 crc kubenswrapper[5052]: I1203 12:10:58.240302 5052 scope.go:117] "RemoveContainer" containerID="32468dac9534ebe102ff57ab3a6ff7547398603ee630025eaac80c0d8f9678c8" Dec 03 12:10:58 crc kubenswrapper[5052]: I1203 12:10:58.249942 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-8clcj\" (UniqueName: \"kubernetes.io/projected/fb88d1bf-1334-41ec-8605-e93a852623b3-kube-api-access-8clcj\") pod \"neutron-db-create-2s5ld\" (UID: \"fb88d1bf-1334-41ec-8605-e93a852623b3\") " pod="openstack-kuttl-tests/neutron-db-create-2s5ld" Dec 03 12:10:58 crc kubenswrapper[5052]: I1203 12:10:58.267378 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-nclx4\" (UniqueName: \"kubernetes.io/projected/58c97692-2e69-4232-ba13-23806ab9fb67-kube-api-access-nclx4\") pod \"keystone-db-sync-95hjc\" (UID: \"58c97692-2e69-4232-ba13-23806ab9fb67\") " pod="openstack-kuttl-tests/keystone-db-sync-95hjc" Dec 03 12:10:58 crc kubenswrapper[5052]: I1203 12:10:58.315099 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-4rgwc\" (UniqueName: \"kubernetes.io/projected/884268a1-ff3a-4aaf-8004-89bb72e79d90-kube-api-access-4rgwc\") pod \"neutron-1287-account-create-update-56qp8\" (UID: \"884268a1-ff3a-4aaf-8004-89bb72e79d90\") " pod="openstack-kuttl-tests/neutron-1287-account-create-update-56qp8" Dec 03 12:10:58 crc kubenswrapper[5052]: I1203 12:10:58.315283 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/884268a1-ff3a-4aaf-8004-89bb72e79d90-operator-scripts\") pod \"neutron-1287-account-create-update-56qp8\" (UID: \"884268a1-ff3a-4aaf-8004-89bb72e79d90\") " pod="openstack-kuttl-tests/neutron-1287-account-create-update-56qp8" Dec 03 12:10:58 crc kubenswrapper[5052]: I1203 12:10:58.316264 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/884268a1-ff3a-4aaf-8004-89bb72e79d90-operator-scripts\") pod \"neutron-1287-account-create-update-56qp8\" (UID: \"884268a1-ff3a-4aaf-8004-89bb72e79d90\") " pod="openstack-kuttl-tests/neutron-1287-account-create-update-56qp8" Dec 03 12:10:58 crc kubenswrapper[5052]: I1203 12:10:58.319714 5052 scope.go:117] "RemoveContainer" containerID="445784d8ad0c8332dc5eca6c089a13cfe1fa7b4f744c5922a315d621eed9a21d" Dec 03 12:10:58 crc kubenswrapper[5052]: I1203 12:10:58.333872 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-4rgwc\" (UniqueName: \"kubernetes.io/projected/884268a1-ff3a-4aaf-8004-89bb72e79d90-kube-api-access-4rgwc\") pod \"neutron-1287-account-create-update-56qp8\" (UID: \"884268a1-ff3a-4aaf-8004-89bb72e79d90\") " pod="openstack-kuttl-tests/neutron-1287-account-create-update-56qp8" Dec 03 12:10:58 crc kubenswrapper[5052]: I1203 12:10:58.381545 5052 scope.go:117] "RemoveContainer" containerID="19256df03b425ac2ffa4ab3adf5ae27b797e8391dbfdc95ffe913a963e62c47d" Dec 03 12:10:58 crc kubenswrapper[5052]: I1203 12:10:58.416879 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-db-create-2s5ld" Dec 03 12:10:58 crc kubenswrapper[5052]: I1203 12:10:58.442761 5052 scope.go:117] "RemoveContainer" containerID="a23efe47ae844777245a54dff972169031df4ea13092e5aabb04cc320ac712df" Dec 03 12:10:58 crc kubenswrapper[5052]: I1203 12:10:58.514427 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-db-sync-95hjc" Dec 03 12:10:58 crc kubenswrapper[5052]: I1203 12:10:58.518198 5052 scope.go:117] "RemoveContainer" containerID="c5b31ce4de482ce67274e759f4b4c8f79eb6d33839525a4f9726eab7cd834275" Dec 03 12:10:58 crc kubenswrapper[5052]: I1203 12:10:58.542488 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-1287-account-create-update-56qp8" Dec 03 12:10:58 crc kubenswrapper[5052]: I1203 12:10:58.595023 5052 scope.go:117] "RemoveContainer" containerID="13208c18005d83a4860b544d6bc809d62861e10156063a4cca40ac62076a90b2" Dec 03 12:10:58 crc kubenswrapper[5052]: I1203 12:10:58.643889 5052 scope.go:117] "RemoveContainer" containerID="d56d69a932a4844dfd6d420c8d3e39f3e14070df95fe2a5c2fbf44ba6ea2a2ba" Dec 03 12:10:58 crc kubenswrapper[5052]: I1203 12:10:58.668956 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-db-create-lqkc5"] Dec 03 12:10:58 crc kubenswrapper[5052]: I1203 12:10:58.693974 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-db-create-bn2nj"] Dec 03 12:10:58 crc kubenswrapper[5052]: W1203 12:10:58.697673 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2047705d_ae83_4972_913b_868f2a623022.slice/crio-5a5e9f9c30042aef9ce259243dd51510a83887f1b2ba74ac41a12d45a8501c7f WatchSource:0}: Error finding container 5a5e9f9c30042aef9ce259243dd51510a83887f1b2ba74ac41a12d45a8501c7f: Status 404 returned error can't find the container with id 5a5e9f9c30042aef9ce259243dd51510a83887f1b2ba74ac41a12d45a8501c7f Dec 03 12:10:58 crc kubenswrapper[5052]: I1203 12:10:58.758584 5052 scope.go:117] "RemoveContainer" containerID="a370d0d01033a8b135a4295ba72e03bb44219e4dfe9ea35c01614ac7a79ce4b1" Dec 03 12:10:58 crc kubenswrapper[5052]: I1203 12:10:58.820477 5052 scope.go:117] "RemoveContainer" containerID="c28a9606ae5010324a932dd9d90a75063a7e3597bf5e73c2ae2c2451a0afbbab" Dec 03 12:10:58 crc kubenswrapper[5052]: I1203 12:10:58.855950 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-8992-account-create-update-gbt4t"] Dec 03 12:10:58 crc kubenswrapper[5052]: I1203 12:10:58.870070 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-7878-account-create-update-cxttb"] Dec 03 12:10:59 crc kubenswrapper[5052]: I1203 12:10:59.035667 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-db-create-2s5ld"] Dec 03 12:10:59 crc kubenswrapper[5052]: W1203 12:10:59.051274 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfb88d1bf_1334_41ec_8605_e93a852623b3.slice/crio-85afa17f51a1c46a028ff0c6d040bc2e03fce35dc12694fce44803f910ad6662 WatchSource:0}: Error finding container 85afa17f51a1c46a028ff0c6d040bc2e03fce35dc12694fce44803f910ad6662: Status 404 returned error can't find the container with id 85afa17f51a1c46a028ff0c6d040bc2e03fce35dc12694fce44803f910ad6662 Dec 03 12:10:59 crc kubenswrapper[5052]: I1203 12:10:59.173129 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-db-sync-95hjc"] Dec 03 12:10:59 crc kubenswrapper[5052]: I1203 12:10:59.189598 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-1287-account-create-update-56qp8"] Dec 03 12:10:59 crc kubenswrapper[5052]: W1203 12:10:59.198603 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod58c97692_2e69_4232_ba13_23806ab9fb67.slice/crio-5f6108ddf0d2cd49a2122bf14af72d20b58e9bf489317ad406647dd9da96ccbc WatchSource:0}: Error finding container 5f6108ddf0d2cd49a2122bf14af72d20b58e9bf489317ad406647dd9da96ccbc: Status 404 returned error can't find the container with id 5f6108ddf0d2cd49a2122bf14af72d20b58e9bf489317ad406647dd9da96ccbc Dec 03 12:10:59 crc kubenswrapper[5052]: W1203 12:10:59.202069 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod884268a1_ff3a_4aaf_8004_89bb72e79d90.slice/crio-cbe312f9044f2dd1d487b97eeb67e24ddb5d230fa6bbe0c935f14e4f54afce8f WatchSource:0}: Error finding container cbe312f9044f2dd1d487b97eeb67e24ddb5d230fa6bbe0c935f14e4f54afce8f: Status 404 returned error can't find the container with id cbe312f9044f2dd1d487b97eeb67e24ddb5d230fa6bbe0c935f14e4f54afce8f Dec 03 12:10:59 crc kubenswrapper[5052]: I1203 12:10:59.425873 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-7878-account-create-update-cxttb" event={"ID":"eae2f902-0630-43d2-9a05-cc0d68350068","Type":"ContainerStarted","Data":"2148ec305b2c6455e0a7a297728c85e7be875c005f08c3e77fff33cf4d3bde2d"} Dec 03 12:10:59 crc kubenswrapper[5052]: I1203 12:10:59.426290 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-7878-account-create-update-cxttb" event={"ID":"eae2f902-0630-43d2-9a05-cc0d68350068","Type":"ContainerStarted","Data":"4013f005a3e0b743da81fa2eb0064f1bcca85a9757a421f078e9e09d3aff4ed0"} Dec 03 12:10:59 crc kubenswrapper[5052]: I1203 12:10:59.427955 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-db-sync-95hjc" event={"ID":"58c97692-2e69-4232-ba13-23806ab9fb67","Type":"ContainerStarted","Data":"5f6108ddf0d2cd49a2122bf14af72d20b58e9bf489317ad406647dd9da96ccbc"} Dec 03 12:10:59 crc kubenswrapper[5052]: I1203 12:10:59.430699 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-db-create-bn2nj" event={"ID":"d9613cb1-0439-44fb-a577-c621439a6328","Type":"ContainerStarted","Data":"e9482eb27c41eb5dac4cd251777a2b7ad32fe9cc1d2f868c13cb8db63c2720b0"} Dec 03 12:10:59 crc kubenswrapper[5052]: I1203 12:10:59.430758 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-db-create-bn2nj" event={"ID":"d9613cb1-0439-44fb-a577-c621439a6328","Type":"ContainerStarted","Data":"7cba63dd6e61f177daca3dcf42eaaba74b382a346d7d87396438db8d5ba0dd42"} Dec 03 12:10:59 crc kubenswrapper[5052]: I1203 12:10:59.432381 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-db-create-2s5ld" event={"ID":"fb88d1bf-1334-41ec-8605-e93a852623b3","Type":"ContainerStarted","Data":"85afa17f51a1c46a028ff0c6d040bc2e03fce35dc12694fce44803f910ad6662"} Dec 03 12:10:59 crc kubenswrapper[5052]: I1203 12:10:59.443693 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-8992-account-create-update-gbt4t" event={"ID":"7e802e69-975f-4748-a902-d3715174eefa","Type":"ContainerStarted","Data":"0ce8d3aa98265aafb7a39d170fa6017ca01e2e308bce70bae94d80594915a43f"} Dec 03 12:10:59 crc kubenswrapper[5052]: I1203 12:10:59.443750 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-8992-account-create-update-gbt4t" event={"ID":"7e802e69-975f-4748-a902-d3715174eefa","Type":"ContainerStarted","Data":"1fec7aa4504a50a3545e71ce63126c94e6676b9e17a0023c4c6772883ea4a93d"} Dec 03 12:10:59 crc kubenswrapper[5052]: I1203 12:10:59.450147 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-1287-account-create-update-56qp8" event={"ID":"884268a1-ff3a-4aaf-8004-89bb72e79d90","Type":"ContainerStarted","Data":"cbe312f9044f2dd1d487b97eeb67e24ddb5d230fa6bbe0c935f14e4f54afce8f"} Dec 03 12:10:59 crc kubenswrapper[5052]: I1203 12:10:59.454688 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-db-create-lqkc5" event={"ID":"2047705d-ae83-4972-913b-868f2a623022","Type":"ContainerStarted","Data":"fd4788c2774ac207384c1a4ed34e2dd899d31e1787be5216e43104b4382b3dca"} Dec 03 12:10:59 crc kubenswrapper[5052]: I1203 12:10:59.454743 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-db-create-lqkc5" event={"ID":"2047705d-ae83-4972-913b-868f2a623022","Type":"ContainerStarted","Data":"5a5e9f9c30042aef9ce259243dd51510a83887f1b2ba74ac41a12d45a8501c7f"} Dec 03 12:11:00 crc kubenswrapper[5052]: I1203 12:11:00.464767 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-db-sync-95hjc" event={"ID":"58c97692-2e69-4232-ba13-23806ab9fb67","Type":"ContainerStarted","Data":"328b5a10a350b22ad4abac8cb75074633d0d2442e8c1278b06e290b169197ede"} Dec 03 12:11:00 crc kubenswrapper[5052]: I1203 12:11:00.466984 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-1287-account-create-update-56qp8" event={"ID":"884268a1-ff3a-4aaf-8004-89bb72e79d90","Type":"ContainerStarted","Data":"3a6988a519408f10e2c7e85d6879bc627c5cf388c6b01b100a31df75d3b69f87"} Dec 03 12:11:00 crc kubenswrapper[5052]: I1203 12:11:00.468312 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-db-create-2s5ld" event={"ID":"fb88d1bf-1334-41ec-8605-e93a852623b3","Type":"ContainerStarted","Data":"5a2cb3d5fc8ac42d6634d0be3922a93e66ea5b5349f071f28e904e46567ec72d"} Dec 03 12:11:00 crc kubenswrapper[5052]: I1203 12:11:00.470032 5052 generic.go:334] "Generic (PLEG): container finished" podID="2047705d-ae83-4972-913b-868f2a623022" containerID="fd4788c2774ac207384c1a4ed34e2dd899d31e1787be5216e43104b4382b3dca" exitCode=0 Dec 03 12:11:00 crc kubenswrapper[5052]: I1203 12:11:00.470179 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-db-create-lqkc5" event={"ID":"2047705d-ae83-4972-913b-868f2a623022","Type":"ContainerDied","Data":"fd4788c2774ac207384c1a4ed34e2dd899d31e1787be5216e43104b4382b3dca"} Dec 03 12:11:00 crc kubenswrapper[5052]: I1203 12:11:00.486026 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/keystone-db-sync-95hjc" podStartSLOduration=3.486008849 podStartE2EDuration="3.486008849s" podCreationTimestamp="2025-12-03 12:10:57 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:11:00.480337177 +0000 UTC m=+3051.068578941" watchObservedRunningTime="2025-12-03 12:11:00.486008849 +0000 UTC m=+3051.074250613" Dec 03 12:11:00 crc kubenswrapper[5052]: I1203 12:11:00.503366 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/barbican-db-create-bn2nj" podStartSLOduration=3.503343624 podStartE2EDuration="3.503343624s" podCreationTimestamp="2025-12-03 12:10:57 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:11:00.500863373 +0000 UTC m=+3051.089105147" watchObservedRunningTime="2025-12-03 12:11:00.503343624 +0000 UTC m=+3051.091585388" Dec 03 12:11:00 crc kubenswrapper[5052]: I1203 12:11:00.522234 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/neutron-db-create-2s5ld" podStartSLOduration=3.522219823 podStartE2EDuration="3.522219823s" podCreationTimestamp="2025-12-03 12:10:57 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:11:00.519612918 +0000 UTC m=+3051.107854692" watchObservedRunningTime="2025-12-03 12:11:00.522219823 +0000 UTC m=+3051.110461587" Dec 03 12:11:00 crc kubenswrapper[5052]: I1203 12:11:00.547652 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/neutron-1287-account-create-update-56qp8" podStartSLOduration=2.5476280879999997 podStartE2EDuration="2.547628088s" podCreationTimestamp="2025-12-03 12:10:58 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:11:00.53822443 +0000 UTC m=+3051.126466194" watchObservedRunningTime="2025-12-03 12:11:00.547628088 +0000 UTC m=+3051.135869852" Dec 03 12:11:00 crc kubenswrapper[5052]: I1203 12:11:00.600465 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/barbican-8992-account-create-update-gbt4t" podStartSLOduration=3.600430186 podStartE2EDuration="3.600430186s" podCreationTimestamp="2025-12-03 12:10:57 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:11:00.565014274 +0000 UTC m=+3051.153256028" watchObservedRunningTime="2025-12-03 12:11:00.600430186 +0000 UTC m=+3051.188671950" Dec 03 12:11:00 crc kubenswrapper[5052]: I1203 12:11:00.638830 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/cinder-7878-account-create-update-cxttb" podStartSLOduration=3.638792191 podStartE2EDuration="3.638792191s" podCreationTimestamp="2025-12-03 12:10:57 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:11:00.627139388 +0000 UTC m=+3051.215381152" watchObservedRunningTime="2025-12-03 12:11:00.638792191 +0000 UTC m=+3051.227033945" Dec 03 12:11:01 crc kubenswrapper[5052]: I1203 12:11:01.483691 5052 generic.go:334] "Generic (PLEG): container finished" podID="fb88d1bf-1334-41ec-8605-e93a852623b3" containerID="5a2cb3d5fc8ac42d6634d0be3922a93e66ea5b5349f071f28e904e46567ec72d" exitCode=0 Dec 03 12:11:01 crc kubenswrapper[5052]: I1203 12:11:01.483744 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-db-create-2s5ld" event={"ID":"fb88d1bf-1334-41ec-8605-e93a852623b3","Type":"ContainerDied","Data":"5a2cb3d5fc8ac42d6634d0be3922a93e66ea5b5349f071f28e904e46567ec72d"} Dec 03 12:11:01 crc kubenswrapper[5052]: I1203 12:11:01.487123 5052 generic.go:334] "Generic (PLEG): container finished" podID="d9613cb1-0439-44fb-a577-c621439a6328" containerID="e9482eb27c41eb5dac4cd251777a2b7ad32fe9cc1d2f868c13cb8db63c2720b0" exitCode=0 Dec 03 12:11:01 crc kubenswrapper[5052]: I1203 12:11:01.487256 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-db-create-bn2nj" event={"ID":"d9613cb1-0439-44fb-a577-c621439a6328","Type":"ContainerDied","Data":"e9482eb27c41eb5dac4cd251777a2b7ad32fe9cc1d2f868c13cb8db63c2720b0"} Dec 03 12:11:01 crc kubenswrapper[5052]: I1203 12:11:01.490468 5052 generic.go:334] "Generic (PLEG): container finished" podID="eae2f902-0630-43d2-9a05-cc0d68350068" containerID="2148ec305b2c6455e0a7a297728c85e7be875c005f08c3e77fff33cf4d3bde2d" exitCode=0 Dec 03 12:11:01 crc kubenswrapper[5052]: I1203 12:11:01.491169 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-7878-account-create-update-cxttb" event={"ID":"eae2f902-0630-43d2-9a05-cc0d68350068","Type":"ContainerDied","Data":"2148ec305b2c6455e0a7a297728c85e7be875c005f08c3e77fff33cf4d3bde2d"} Dec 03 12:11:01 crc kubenswrapper[5052]: I1203 12:11:01.672645 5052 scope.go:117] "RemoveContainer" containerID="63298107b3d090deff9ed92f3d613e5b1c86d9137684d117500257db8ee1c805" Dec 03 12:11:01 crc kubenswrapper[5052]: E1203 12:11:01.673251 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 12:11:01 crc kubenswrapper[5052]: I1203 12:11:01.837057 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-db-create-lqkc5" Dec 03 12:11:01 crc kubenswrapper[5052]: I1203 12:11:01.986461 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-77pc7\" (UniqueName: \"kubernetes.io/projected/2047705d-ae83-4972-913b-868f2a623022-kube-api-access-77pc7\") pod \"2047705d-ae83-4972-913b-868f2a623022\" (UID: \"2047705d-ae83-4972-913b-868f2a623022\") " Dec 03 12:11:01 crc kubenswrapper[5052]: I1203 12:11:01.986563 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/2047705d-ae83-4972-913b-868f2a623022-operator-scripts\") pod \"2047705d-ae83-4972-913b-868f2a623022\" (UID: \"2047705d-ae83-4972-913b-868f2a623022\") " Dec 03 12:11:01 crc kubenswrapper[5052]: I1203 12:11:01.987009 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2047705d-ae83-4972-913b-868f2a623022-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "2047705d-ae83-4972-913b-868f2a623022" (UID: "2047705d-ae83-4972-913b-868f2a623022"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:11:01 crc kubenswrapper[5052]: I1203 12:11:01.992801 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2047705d-ae83-4972-913b-868f2a623022-kube-api-access-77pc7" (OuterVolumeSpecName: "kube-api-access-77pc7") pod "2047705d-ae83-4972-913b-868f2a623022" (UID: "2047705d-ae83-4972-913b-868f2a623022"). InnerVolumeSpecName "kube-api-access-77pc7". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:11:02 crc kubenswrapper[5052]: I1203 12:11:02.088925 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-77pc7\" (UniqueName: \"kubernetes.io/projected/2047705d-ae83-4972-913b-868f2a623022-kube-api-access-77pc7\") on node \"crc\" DevicePath \"\"" Dec 03 12:11:02 crc kubenswrapper[5052]: I1203 12:11:02.088968 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/2047705d-ae83-4972-913b-868f2a623022-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:11:02 crc kubenswrapper[5052]: I1203 12:11:02.501865 5052 generic.go:334] "Generic (PLEG): container finished" podID="884268a1-ff3a-4aaf-8004-89bb72e79d90" containerID="3a6988a519408f10e2c7e85d6879bc627c5cf388c6b01b100a31df75d3b69f87" exitCode=0 Dec 03 12:11:02 crc kubenswrapper[5052]: I1203 12:11:02.501952 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-1287-account-create-update-56qp8" event={"ID":"884268a1-ff3a-4aaf-8004-89bb72e79d90","Type":"ContainerDied","Data":"3a6988a519408f10e2c7e85d6879bc627c5cf388c6b01b100a31df75d3b69f87"} Dec 03 12:11:02 crc kubenswrapper[5052]: I1203 12:11:02.504382 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-db-create-lqkc5" event={"ID":"2047705d-ae83-4972-913b-868f2a623022","Type":"ContainerDied","Data":"5a5e9f9c30042aef9ce259243dd51510a83887f1b2ba74ac41a12d45a8501c7f"} Dec 03 12:11:02 crc kubenswrapper[5052]: I1203 12:11:02.504417 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5a5e9f9c30042aef9ce259243dd51510a83887f1b2ba74ac41a12d45a8501c7f" Dec 03 12:11:02 crc kubenswrapper[5052]: I1203 12:11:02.504470 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-db-create-lqkc5" Dec 03 12:11:02 crc kubenswrapper[5052]: I1203 12:11:02.507291 5052 generic.go:334] "Generic (PLEG): container finished" podID="7e802e69-975f-4748-a902-d3715174eefa" containerID="0ce8d3aa98265aafb7a39d170fa6017ca01e2e308bce70bae94d80594915a43f" exitCode=0 Dec 03 12:11:02 crc kubenswrapper[5052]: I1203 12:11:02.507359 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-8992-account-create-update-gbt4t" event={"ID":"7e802e69-975f-4748-a902-d3715174eefa","Type":"ContainerDied","Data":"0ce8d3aa98265aafb7a39d170fa6017ca01e2e308bce70bae94d80594915a43f"} Dec 03 12:11:02 crc kubenswrapper[5052]: I1203 12:11:02.896714 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-7878-account-create-update-cxttb" Dec 03 12:11:03 crc kubenswrapper[5052]: I1203 12:11:03.004406 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/eae2f902-0630-43d2-9a05-cc0d68350068-operator-scripts\") pod \"eae2f902-0630-43d2-9a05-cc0d68350068\" (UID: \"eae2f902-0630-43d2-9a05-cc0d68350068\") " Dec 03 12:11:03 crc kubenswrapper[5052]: I1203 12:11:03.004565 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8c4mn\" (UniqueName: \"kubernetes.io/projected/eae2f902-0630-43d2-9a05-cc0d68350068-kube-api-access-8c4mn\") pod \"eae2f902-0630-43d2-9a05-cc0d68350068\" (UID: \"eae2f902-0630-43d2-9a05-cc0d68350068\") " Dec 03 12:11:03 crc kubenswrapper[5052]: I1203 12:11:03.006032 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eae2f902-0630-43d2-9a05-cc0d68350068-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "eae2f902-0630-43d2-9a05-cc0d68350068" (UID: "eae2f902-0630-43d2-9a05-cc0d68350068"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:11:03 crc kubenswrapper[5052]: I1203 12:11:03.009865 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eae2f902-0630-43d2-9a05-cc0d68350068-kube-api-access-8c4mn" (OuterVolumeSpecName: "kube-api-access-8c4mn") pod "eae2f902-0630-43d2-9a05-cc0d68350068" (UID: "eae2f902-0630-43d2-9a05-cc0d68350068"). InnerVolumeSpecName "kube-api-access-8c4mn". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:11:03 crc kubenswrapper[5052]: I1203 12:11:03.065868 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-db-create-2s5ld" Dec 03 12:11:03 crc kubenswrapper[5052]: I1203 12:11:03.075585 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-db-create-bn2nj" Dec 03 12:11:03 crc kubenswrapper[5052]: I1203 12:11:03.106226 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/eae2f902-0630-43d2-9a05-cc0d68350068-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:11:03 crc kubenswrapper[5052]: I1203 12:11:03.106255 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-8c4mn\" (UniqueName: \"kubernetes.io/projected/eae2f902-0630-43d2-9a05-cc0d68350068-kube-api-access-8c4mn\") on node \"crc\" DevicePath \"\"" Dec 03 12:11:03 crc kubenswrapper[5052]: I1203 12:11:03.207502 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/fb88d1bf-1334-41ec-8605-e93a852623b3-operator-scripts\") pod \"fb88d1bf-1334-41ec-8605-e93a852623b3\" (UID: \"fb88d1bf-1334-41ec-8605-e93a852623b3\") " Dec 03 12:11:03 crc kubenswrapper[5052]: I1203 12:11:03.207882 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8clcj\" (UniqueName: \"kubernetes.io/projected/fb88d1bf-1334-41ec-8605-e93a852623b3-kube-api-access-8clcj\") pod \"fb88d1bf-1334-41ec-8605-e93a852623b3\" (UID: \"fb88d1bf-1334-41ec-8605-e93a852623b3\") " Dec 03 12:11:03 crc kubenswrapper[5052]: I1203 12:11:03.207937 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-khhp9\" (UniqueName: \"kubernetes.io/projected/d9613cb1-0439-44fb-a577-c621439a6328-kube-api-access-khhp9\") pod \"d9613cb1-0439-44fb-a577-c621439a6328\" (UID: \"d9613cb1-0439-44fb-a577-c621439a6328\") " Dec 03 12:11:03 crc kubenswrapper[5052]: I1203 12:11:03.208021 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/d9613cb1-0439-44fb-a577-c621439a6328-operator-scripts\") pod \"d9613cb1-0439-44fb-a577-c621439a6328\" (UID: \"d9613cb1-0439-44fb-a577-c621439a6328\") " Dec 03 12:11:03 crc kubenswrapper[5052]: I1203 12:11:03.208054 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fb88d1bf-1334-41ec-8605-e93a852623b3-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "fb88d1bf-1334-41ec-8605-e93a852623b3" (UID: "fb88d1bf-1334-41ec-8605-e93a852623b3"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:11:03 crc kubenswrapper[5052]: I1203 12:11:03.208831 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d9613cb1-0439-44fb-a577-c621439a6328-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "d9613cb1-0439-44fb-a577-c621439a6328" (UID: "d9613cb1-0439-44fb-a577-c621439a6328"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:11:03 crc kubenswrapper[5052]: I1203 12:11:03.209498 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/d9613cb1-0439-44fb-a577-c621439a6328-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:11:03 crc kubenswrapper[5052]: I1203 12:11:03.209552 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/fb88d1bf-1334-41ec-8605-e93a852623b3-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:11:03 crc kubenswrapper[5052]: I1203 12:11:03.226553 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d9613cb1-0439-44fb-a577-c621439a6328-kube-api-access-khhp9" (OuterVolumeSpecName: "kube-api-access-khhp9") pod "d9613cb1-0439-44fb-a577-c621439a6328" (UID: "d9613cb1-0439-44fb-a577-c621439a6328"). InnerVolumeSpecName "kube-api-access-khhp9". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:11:03 crc kubenswrapper[5052]: I1203 12:11:03.226757 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fb88d1bf-1334-41ec-8605-e93a852623b3-kube-api-access-8clcj" (OuterVolumeSpecName: "kube-api-access-8clcj") pod "fb88d1bf-1334-41ec-8605-e93a852623b3" (UID: "fb88d1bf-1334-41ec-8605-e93a852623b3"). InnerVolumeSpecName "kube-api-access-8clcj". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:11:03 crc kubenswrapper[5052]: I1203 12:11:03.311467 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-8clcj\" (UniqueName: \"kubernetes.io/projected/fb88d1bf-1334-41ec-8605-e93a852623b3-kube-api-access-8clcj\") on node \"crc\" DevicePath \"\"" Dec 03 12:11:03 crc kubenswrapper[5052]: I1203 12:11:03.311507 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-khhp9\" (UniqueName: \"kubernetes.io/projected/d9613cb1-0439-44fb-a577-c621439a6328-kube-api-access-khhp9\") on node \"crc\" DevicePath \"\"" Dec 03 12:11:03 crc kubenswrapper[5052]: I1203 12:11:03.519354 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-db-create-bn2nj" event={"ID":"d9613cb1-0439-44fb-a577-c621439a6328","Type":"ContainerDied","Data":"7cba63dd6e61f177daca3dcf42eaaba74b382a346d7d87396438db8d5ba0dd42"} Dec 03 12:11:03 crc kubenswrapper[5052]: I1203 12:11:03.519399 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7cba63dd6e61f177daca3dcf42eaaba74b382a346d7d87396438db8d5ba0dd42" Dec 03 12:11:03 crc kubenswrapper[5052]: I1203 12:11:03.519408 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-db-create-bn2nj" Dec 03 12:11:03 crc kubenswrapper[5052]: I1203 12:11:03.521798 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-7878-account-create-update-cxttb" Dec 03 12:11:03 crc kubenswrapper[5052]: I1203 12:11:03.521799 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-7878-account-create-update-cxttb" event={"ID":"eae2f902-0630-43d2-9a05-cc0d68350068","Type":"ContainerDied","Data":"4013f005a3e0b743da81fa2eb0064f1bcca85a9757a421f078e9e09d3aff4ed0"} Dec 03 12:11:03 crc kubenswrapper[5052]: I1203 12:11:03.521984 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4013f005a3e0b743da81fa2eb0064f1bcca85a9757a421f078e9e09d3aff4ed0" Dec 03 12:11:03 crc kubenswrapper[5052]: I1203 12:11:03.525855 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-db-create-2s5ld" event={"ID":"fb88d1bf-1334-41ec-8605-e93a852623b3","Type":"ContainerDied","Data":"85afa17f51a1c46a028ff0c6d040bc2e03fce35dc12694fce44803f910ad6662"} Dec 03 12:11:03 crc kubenswrapper[5052]: I1203 12:11:03.525923 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="85afa17f51a1c46a028ff0c6d040bc2e03fce35dc12694fce44803f910ad6662" Dec 03 12:11:03 crc kubenswrapper[5052]: I1203 12:11:03.525932 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-db-create-2s5ld" Dec 03 12:11:03 crc kubenswrapper[5052]: I1203 12:11:03.848183 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-1287-account-create-update-56qp8" Dec 03 12:11:03 crc kubenswrapper[5052]: I1203 12:11:03.925968 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4rgwc\" (UniqueName: \"kubernetes.io/projected/884268a1-ff3a-4aaf-8004-89bb72e79d90-kube-api-access-4rgwc\") pod \"884268a1-ff3a-4aaf-8004-89bb72e79d90\" (UID: \"884268a1-ff3a-4aaf-8004-89bb72e79d90\") " Dec 03 12:11:03 crc kubenswrapper[5052]: I1203 12:11:03.926306 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/884268a1-ff3a-4aaf-8004-89bb72e79d90-operator-scripts\") pod \"884268a1-ff3a-4aaf-8004-89bb72e79d90\" (UID: \"884268a1-ff3a-4aaf-8004-89bb72e79d90\") " Dec 03 12:11:03 crc kubenswrapper[5052]: I1203 12:11:03.927487 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/884268a1-ff3a-4aaf-8004-89bb72e79d90-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "884268a1-ff3a-4aaf-8004-89bb72e79d90" (UID: "884268a1-ff3a-4aaf-8004-89bb72e79d90"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:11:03 crc kubenswrapper[5052]: I1203 12:11:03.937501 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/884268a1-ff3a-4aaf-8004-89bb72e79d90-kube-api-access-4rgwc" (OuterVolumeSpecName: "kube-api-access-4rgwc") pod "884268a1-ff3a-4aaf-8004-89bb72e79d90" (UID: "884268a1-ff3a-4aaf-8004-89bb72e79d90"). InnerVolumeSpecName "kube-api-access-4rgwc". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:11:03 crc kubenswrapper[5052]: I1203 12:11:03.981451 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-8992-account-create-update-gbt4t" Dec 03 12:11:04 crc kubenswrapper[5052]: I1203 12:11:04.028378 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-4rgwc\" (UniqueName: \"kubernetes.io/projected/884268a1-ff3a-4aaf-8004-89bb72e79d90-kube-api-access-4rgwc\") on node \"crc\" DevicePath \"\"" Dec 03 12:11:04 crc kubenswrapper[5052]: I1203 12:11:04.028424 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/884268a1-ff3a-4aaf-8004-89bb72e79d90-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:11:04 crc kubenswrapper[5052]: I1203 12:11:04.129998 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/7e802e69-975f-4748-a902-d3715174eefa-operator-scripts\") pod \"7e802e69-975f-4748-a902-d3715174eefa\" (UID: \"7e802e69-975f-4748-a902-d3715174eefa\") " Dec 03 12:11:04 crc kubenswrapper[5052]: I1203 12:11:04.130179 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dtvd6\" (UniqueName: \"kubernetes.io/projected/7e802e69-975f-4748-a902-d3715174eefa-kube-api-access-dtvd6\") pod \"7e802e69-975f-4748-a902-d3715174eefa\" (UID: \"7e802e69-975f-4748-a902-d3715174eefa\") " Dec 03 12:11:04 crc kubenswrapper[5052]: I1203 12:11:04.130517 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7e802e69-975f-4748-a902-d3715174eefa-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "7e802e69-975f-4748-a902-d3715174eefa" (UID: "7e802e69-975f-4748-a902-d3715174eefa"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:11:04 crc kubenswrapper[5052]: I1203 12:11:04.130673 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/7e802e69-975f-4748-a902-d3715174eefa-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:11:04 crc kubenswrapper[5052]: I1203 12:11:04.143985 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7e802e69-975f-4748-a902-d3715174eefa-kube-api-access-dtvd6" (OuterVolumeSpecName: "kube-api-access-dtvd6") pod "7e802e69-975f-4748-a902-d3715174eefa" (UID: "7e802e69-975f-4748-a902-d3715174eefa"). InnerVolumeSpecName "kube-api-access-dtvd6". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:11:04 crc kubenswrapper[5052]: I1203 12:11:04.232600 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-dtvd6\" (UniqueName: \"kubernetes.io/projected/7e802e69-975f-4748-a902-d3715174eefa-kube-api-access-dtvd6\") on node \"crc\" DevicePath \"\"" Dec 03 12:11:04 crc kubenswrapper[5052]: I1203 12:11:04.537321 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-8992-account-create-update-gbt4t" event={"ID":"7e802e69-975f-4748-a902-d3715174eefa","Type":"ContainerDied","Data":"1fec7aa4504a50a3545e71ce63126c94e6676b9e17a0023c4c6772883ea4a93d"} Dec 03 12:11:04 crc kubenswrapper[5052]: I1203 12:11:04.537371 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1fec7aa4504a50a3545e71ce63126c94e6676b9e17a0023c4c6772883ea4a93d" Dec 03 12:11:04 crc kubenswrapper[5052]: I1203 12:11:04.537374 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-8992-account-create-update-gbt4t" Dec 03 12:11:04 crc kubenswrapper[5052]: I1203 12:11:04.539267 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-1287-account-create-update-56qp8" event={"ID":"884268a1-ff3a-4aaf-8004-89bb72e79d90","Type":"ContainerDied","Data":"cbe312f9044f2dd1d487b97eeb67e24ddb5d230fa6bbe0c935f14e4f54afce8f"} Dec 03 12:11:04 crc kubenswrapper[5052]: I1203 12:11:04.539313 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cbe312f9044f2dd1d487b97eeb67e24ddb5d230fa6bbe0c935f14e4f54afce8f" Dec 03 12:11:04 crc kubenswrapper[5052]: I1203 12:11:04.539355 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-1287-account-create-update-56qp8" Dec 03 12:11:11 crc kubenswrapper[5052]: I1203 12:11:11.607794 5052 generic.go:334] "Generic (PLEG): container finished" podID="58c97692-2e69-4232-ba13-23806ab9fb67" containerID="328b5a10a350b22ad4abac8cb75074633d0d2442e8c1278b06e290b169197ede" exitCode=0 Dec 03 12:11:11 crc kubenswrapper[5052]: I1203 12:11:11.607890 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-db-sync-95hjc" event={"ID":"58c97692-2e69-4232-ba13-23806ab9fb67","Type":"ContainerDied","Data":"328b5a10a350b22ad4abac8cb75074633d0d2442e8c1278b06e290b169197ede"} Dec 03 12:11:12 crc kubenswrapper[5052]: I1203 12:11:12.960279 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-db-sync-95hjc" Dec 03 12:11:13 crc kubenswrapper[5052]: I1203 12:11:13.002242 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/58c97692-2e69-4232-ba13-23806ab9fb67-config-data\") pod \"58c97692-2e69-4232-ba13-23806ab9fb67\" (UID: \"58c97692-2e69-4232-ba13-23806ab9fb67\") " Dec 03 12:11:13 crc kubenswrapper[5052]: I1203 12:11:13.002317 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nclx4\" (UniqueName: \"kubernetes.io/projected/58c97692-2e69-4232-ba13-23806ab9fb67-kube-api-access-nclx4\") pod \"58c97692-2e69-4232-ba13-23806ab9fb67\" (UID: \"58c97692-2e69-4232-ba13-23806ab9fb67\") " Dec 03 12:11:13 crc kubenswrapper[5052]: I1203 12:11:13.002393 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/58c97692-2e69-4232-ba13-23806ab9fb67-combined-ca-bundle\") pod \"58c97692-2e69-4232-ba13-23806ab9fb67\" (UID: \"58c97692-2e69-4232-ba13-23806ab9fb67\") " Dec 03 12:11:13 crc kubenswrapper[5052]: I1203 12:11:13.009887 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/58c97692-2e69-4232-ba13-23806ab9fb67-kube-api-access-nclx4" (OuterVolumeSpecName: "kube-api-access-nclx4") pod "58c97692-2e69-4232-ba13-23806ab9fb67" (UID: "58c97692-2e69-4232-ba13-23806ab9fb67"). InnerVolumeSpecName "kube-api-access-nclx4". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:11:13 crc kubenswrapper[5052]: I1203 12:11:13.032315 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/58c97692-2e69-4232-ba13-23806ab9fb67-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "58c97692-2e69-4232-ba13-23806ab9fb67" (UID: "58c97692-2e69-4232-ba13-23806ab9fb67"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:11:13 crc kubenswrapper[5052]: I1203 12:11:13.057107 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/58c97692-2e69-4232-ba13-23806ab9fb67-config-data" (OuterVolumeSpecName: "config-data") pod "58c97692-2e69-4232-ba13-23806ab9fb67" (UID: "58c97692-2e69-4232-ba13-23806ab9fb67"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:11:13 crc kubenswrapper[5052]: I1203 12:11:13.104445 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/58c97692-2e69-4232-ba13-23806ab9fb67-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:11:13 crc kubenswrapper[5052]: I1203 12:11:13.104489 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-nclx4\" (UniqueName: \"kubernetes.io/projected/58c97692-2e69-4232-ba13-23806ab9fb67-kube-api-access-nclx4\") on node \"crc\" DevicePath \"\"" Dec 03 12:11:13 crc kubenswrapper[5052]: I1203 12:11:13.104500 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/58c97692-2e69-4232-ba13-23806ab9fb67-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:11:13 crc kubenswrapper[5052]: I1203 12:11:13.626931 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-db-sync-95hjc" event={"ID":"58c97692-2e69-4232-ba13-23806ab9fb67","Type":"ContainerDied","Data":"5f6108ddf0d2cd49a2122bf14af72d20b58e9bf489317ad406647dd9da96ccbc"} Dec 03 12:11:13 crc kubenswrapper[5052]: I1203 12:11:13.627317 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5f6108ddf0d2cd49a2122bf14af72d20b58e9bf489317ad406647dd9da96ccbc" Dec 03 12:11:13 crc kubenswrapper[5052]: I1203 12:11:13.626984 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-db-sync-95hjc" Dec 03 12:11:13 crc kubenswrapper[5052]: I1203 12:11:13.744334 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/keystone-bootstrap-zz8h5"] Dec 03 12:11:13 crc kubenswrapper[5052]: E1203 12:11:13.744780 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="2047705d-ae83-4972-913b-868f2a623022" containerName="mariadb-database-create" Dec 03 12:11:13 crc kubenswrapper[5052]: I1203 12:11:13.744802 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="2047705d-ae83-4972-913b-868f2a623022" containerName="mariadb-database-create" Dec 03 12:11:13 crc kubenswrapper[5052]: E1203 12:11:13.744831 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="fb88d1bf-1334-41ec-8605-e93a852623b3" containerName="mariadb-database-create" Dec 03 12:11:13 crc kubenswrapper[5052]: I1203 12:11:13.744840 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb88d1bf-1334-41ec-8605-e93a852623b3" containerName="mariadb-database-create" Dec 03 12:11:13 crc kubenswrapper[5052]: E1203 12:11:13.744864 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="7e802e69-975f-4748-a902-d3715174eefa" containerName="mariadb-account-create-update" Dec 03 12:11:13 crc kubenswrapper[5052]: I1203 12:11:13.744872 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="7e802e69-975f-4748-a902-d3715174eefa" containerName="mariadb-account-create-update" Dec 03 12:11:13 crc kubenswrapper[5052]: E1203 12:11:13.744888 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="d9613cb1-0439-44fb-a577-c621439a6328" containerName="mariadb-database-create" Dec 03 12:11:13 crc kubenswrapper[5052]: I1203 12:11:13.744895 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="d9613cb1-0439-44fb-a577-c621439a6328" containerName="mariadb-database-create" Dec 03 12:11:13 crc kubenswrapper[5052]: E1203 12:11:13.744907 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="eae2f902-0630-43d2-9a05-cc0d68350068" containerName="mariadb-account-create-update" Dec 03 12:11:13 crc kubenswrapper[5052]: I1203 12:11:13.744917 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="eae2f902-0630-43d2-9a05-cc0d68350068" containerName="mariadb-account-create-update" Dec 03 12:11:13 crc kubenswrapper[5052]: E1203 12:11:13.744929 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="58c97692-2e69-4232-ba13-23806ab9fb67" containerName="keystone-db-sync" Dec 03 12:11:13 crc kubenswrapper[5052]: I1203 12:11:13.744938 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="58c97692-2e69-4232-ba13-23806ab9fb67" containerName="keystone-db-sync" Dec 03 12:11:13 crc kubenswrapper[5052]: E1203 12:11:13.744963 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="884268a1-ff3a-4aaf-8004-89bb72e79d90" containerName="mariadb-account-create-update" Dec 03 12:11:13 crc kubenswrapper[5052]: I1203 12:11:13.744970 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="884268a1-ff3a-4aaf-8004-89bb72e79d90" containerName="mariadb-account-create-update" Dec 03 12:11:13 crc kubenswrapper[5052]: I1203 12:11:13.745188 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="d9613cb1-0439-44fb-a577-c621439a6328" containerName="mariadb-database-create" Dec 03 12:11:13 crc kubenswrapper[5052]: I1203 12:11:13.745204 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="7e802e69-975f-4748-a902-d3715174eefa" containerName="mariadb-account-create-update" Dec 03 12:11:13 crc kubenswrapper[5052]: I1203 12:11:13.745219 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="884268a1-ff3a-4aaf-8004-89bb72e79d90" containerName="mariadb-account-create-update" Dec 03 12:11:13 crc kubenswrapper[5052]: I1203 12:11:13.745241 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="58c97692-2e69-4232-ba13-23806ab9fb67" containerName="keystone-db-sync" Dec 03 12:11:13 crc kubenswrapper[5052]: I1203 12:11:13.745255 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="2047705d-ae83-4972-913b-868f2a623022" containerName="mariadb-database-create" Dec 03 12:11:13 crc kubenswrapper[5052]: I1203 12:11:13.745275 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="eae2f902-0630-43d2-9a05-cc0d68350068" containerName="mariadb-account-create-update" Dec 03 12:11:13 crc kubenswrapper[5052]: I1203 12:11:13.745286 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="fb88d1bf-1334-41ec-8605-e93a852623b3" containerName="mariadb-database-create" Dec 03 12:11:13 crc kubenswrapper[5052]: I1203 12:11:13.746623 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-bootstrap-zz8h5" Dec 03 12:11:13 crc kubenswrapper[5052]: I1203 12:11:13.758201 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone-keystone-dockercfg-4wch9" Dec 03 12:11:13 crc kubenswrapper[5052]: I1203 12:11:13.758402 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone" Dec 03 12:11:13 crc kubenswrapper[5052]: I1203 12:11:13.758408 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone-config-data" Dec 03 12:11:13 crc kubenswrapper[5052]: I1203 12:11:13.758763 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"osp-secret" Dec 03 12:11:13 crc kubenswrapper[5052]: I1203 12:11:13.758878 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone-scripts" Dec 03 12:11:13 crc kubenswrapper[5052]: I1203 12:11:13.770930 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-bootstrap-zz8h5"] Dec 03 12:11:13 crc kubenswrapper[5052]: I1203 12:11:13.816157 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x6sfm\" (UniqueName: \"kubernetes.io/projected/581cd00d-4ede-4150-a4ea-aeac7033f0ab-kube-api-access-x6sfm\") pod \"keystone-bootstrap-zz8h5\" (UID: \"581cd00d-4ede-4150-a4ea-aeac7033f0ab\") " pod="openstack-kuttl-tests/keystone-bootstrap-zz8h5" Dec 03 12:11:13 crc kubenswrapper[5052]: I1203 12:11:13.816256 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/581cd00d-4ede-4150-a4ea-aeac7033f0ab-config-data\") pod \"keystone-bootstrap-zz8h5\" (UID: \"581cd00d-4ede-4150-a4ea-aeac7033f0ab\") " pod="openstack-kuttl-tests/keystone-bootstrap-zz8h5" Dec 03 12:11:13 crc kubenswrapper[5052]: I1203 12:11:13.816293 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/581cd00d-4ede-4150-a4ea-aeac7033f0ab-combined-ca-bundle\") pod \"keystone-bootstrap-zz8h5\" (UID: \"581cd00d-4ede-4150-a4ea-aeac7033f0ab\") " pod="openstack-kuttl-tests/keystone-bootstrap-zz8h5" Dec 03 12:11:13 crc kubenswrapper[5052]: I1203 12:11:13.816330 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/581cd00d-4ede-4150-a4ea-aeac7033f0ab-fernet-keys\") pod \"keystone-bootstrap-zz8h5\" (UID: \"581cd00d-4ede-4150-a4ea-aeac7033f0ab\") " pod="openstack-kuttl-tests/keystone-bootstrap-zz8h5" Dec 03 12:11:13 crc kubenswrapper[5052]: I1203 12:11:13.816363 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/581cd00d-4ede-4150-a4ea-aeac7033f0ab-credential-keys\") pod \"keystone-bootstrap-zz8h5\" (UID: \"581cd00d-4ede-4150-a4ea-aeac7033f0ab\") " pod="openstack-kuttl-tests/keystone-bootstrap-zz8h5" Dec 03 12:11:13 crc kubenswrapper[5052]: I1203 12:11:13.816419 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/581cd00d-4ede-4150-a4ea-aeac7033f0ab-scripts\") pod \"keystone-bootstrap-zz8h5\" (UID: \"581cd00d-4ede-4150-a4ea-aeac7033f0ab\") " pod="openstack-kuttl-tests/keystone-bootstrap-zz8h5" Dec 03 12:11:13 crc kubenswrapper[5052]: I1203 12:11:13.918149 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-x6sfm\" (UniqueName: \"kubernetes.io/projected/581cd00d-4ede-4150-a4ea-aeac7033f0ab-kube-api-access-x6sfm\") pod \"keystone-bootstrap-zz8h5\" (UID: \"581cd00d-4ede-4150-a4ea-aeac7033f0ab\") " pod="openstack-kuttl-tests/keystone-bootstrap-zz8h5" Dec 03 12:11:13 crc kubenswrapper[5052]: I1203 12:11:13.918290 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/581cd00d-4ede-4150-a4ea-aeac7033f0ab-config-data\") pod \"keystone-bootstrap-zz8h5\" (UID: \"581cd00d-4ede-4150-a4ea-aeac7033f0ab\") " pod="openstack-kuttl-tests/keystone-bootstrap-zz8h5" Dec 03 12:11:13 crc kubenswrapper[5052]: I1203 12:11:13.918325 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/581cd00d-4ede-4150-a4ea-aeac7033f0ab-combined-ca-bundle\") pod \"keystone-bootstrap-zz8h5\" (UID: \"581cd00d-4ede-4150-a4ea-aeac7033f0ab\") " pod="openstack-kuttl-tests/keystone-bootstrap-zz8h5" Dec 03 12:11:13 crc kubenswrapper[5052]: I1203 12:11:13.918359 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/581cd00d-4ede-4150-a4ea-aeac7033f0ab-fernet-keys\") pod \"keystone-bootstrap-zz8h5\" (UID: \"581cd00d-4ede-4150-a4ea-aeac7033f0ab\") " pod="openstack-kuttl-tests/keystone-bootstrap-zz8h5" Dec 03 12:11:13 crc kubenswrapper[5052]: I1203 12:11:13.918383 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/581cd00d-4ede-4150-a4ea-aeac7033f0ab-credential-keys\") pod \"keystone-bootstrap-zz8h5\" (UID: \"581cd00d-4ede-4150-a4ea-aeac7033f0ab\") " pod="openstack-kuttl-tests/keystone-bootstrap-zz8h5" Dec 03 12:11:13 crc kubenswrapper[5052]: I1203 12:11:13.918417 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/581cd00d-4ede-4150-a4ea-aeac7033f0ab-scripts\") pod \"keystone-bootstrap-zz8h5\" (UID: \"581cd00d-4ede-4150-a4ea-aeac7033f0ab\") " pod="openstack-kuttl-tests/keystone-bootstrap-zz8h5" Dec 03 12:11:13 crc kubenswrapper[5052]: I1203 12:11:13.923178 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/581cd00d-4ede-4150-a4ea-aeac7033f0ab-combined-ca-bundle\") pod \"keystone-bootstrap-zz8h5\" (UID: \"581cd00d-4ede-4150-a4ea-aeac7033f0ab\") " pod="openstack-kuttl-tests/keystone-bootstrap-zz8h5" Dec 03 12:11:13 crc kubenswrapper[5052]: I1203 12:11:13.923919 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/581cd00d-4ede-4150-a4ea-aeac7033f0ab-config-data\") pod \"keystone-bootstrap-zz8h5\" (UID: \"581cd00d-4ede-4150-a4ea-aeac7033f0ab\") " pod="openstack-kuttl-tests/keystone-bootstrap-zz8h5" Dec 03 12:11:13 crc kubenswrapper[5052]: I1203 12:11:13.924322 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/581cd00d-4ede-4150-a4ea-aeac7033f0ab-fernet-keys\") pod \"keystone-bootstrap-zz8h5\" (UID: \"581cd00d-4ede-4150-a4ea-aeac7033f0ab\") " pod="openstack-kuttl-tests/keystone-bootstrap-zz8h5" Dec 03 12:11:13 crc kubenswrapper[5052]: I1203 12:11:13.925114 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/581cd00d-4ede-4150-a4ea-aeac7033f0ab-scripts\") pod \"keystone-bootstrap-zz8h5\" (UID: \"581cd00d-4ede-4150-a4ea-aeac7033f0ab\") " pod="openstack-kuttl-tests/keystone-bootstrap-zz8h5" Dec 03 12:11:13 crc kubenswrapper[5052]: I1203 12:11:13.941612 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-x6sfm\" (UniqueName: \"kubernetes.io/projected/581cd00d-4ede-4150-a4ea-aeac7033f0ab-kube-api-access-x6sfm\") pod \"keystone-bootstrap-zz8h5\" (UID: \"581cd00d-4ede-4150-a4ea-aeac7033f0ab\") " pod="openstack-kuttl-tests/keystone-bootstrap-zz8h5" Dec 03 12:11:13 crc kubenswrapper[5052]: I1203 12:11:13.944047 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/581cd00d-4ede-4150-a4ea-aeac7033f0ab-credential-keys\") pod \"keystone-bootstrap-zz8h5\" (UID: \"581cd00d-4ede-4150-a4ea-aeac7033f0ab\") " pod="openstack-kuttl-tests/keystone-bootstrap-zz8h5" Dec 03 12:11:13 crc kubenswrapper[5052]: I1203 12:11:13.990560 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:11:13 crc kubenswrapper[5052]: I1203 12:11:13.992524 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:11:14 crc kubenswrapper[5052]: I1203 12:11:14.014029 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-scripts" Dec 03 12:11:14 crc kubenswrapper[5052]: I1203 12:11:14.015044 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-config-data" Dec 03 12:11:14 crc kubenswrapper[5052]: I1203 12:11:14.059088 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/cinder-db-sync-tgpn8"] Dec 03 12:11:14 crc kubenswrapper[5052]: I1203 12:11:14.060270 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-db-sync-tgpn8" Dec 03 12:11:14 crc kubenswrapper[5052]: I1203 12:11:14.067615 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cinder-cinder-dockercfg-qc8wr" Dec 03 12:11:14 crc kubenswrapper[5052]: I1203 12:11:14.067864 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cinder-config-data" Dec 03 12:11:14 crc kubenswrapper[5052]: I1203 12:11:14.067995 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cinder-scripts" Dec 03 12:11:14 crc kubenswrapper[5052]: I1203 12:11:14.068746 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-bootstrap-zz8h5" Dec 03 12:11:14 crc kubenswrapper[5052]: I1203 12:11:14.084648 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:11:14 crc kubenswrapper[5052]: I1203 12:11:14.104575 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/neutron-db-sync-cbdcz"] Dec 03 12:11:14 crc kubenswrapper[5052]: I1203 12:11:14.105725 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-db-sync-cbdcz" Dec 03 12:11:14 crc kubenswrapper[5052]: I1203 12:11:14.110186 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"neutron-config" Dec 03 12:11:14 crc kubenswrapper[5052]: I1203 12:11:14.110450 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"neutron-httpd-config" Dec 03 12:11:14 crc kubenswrapper[5052]: I1203 12:11:14.110637 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"neutron-neutron-dockercfg-s5qsh" Dec 03 12:11:14 crc kubenswrapper[5052]: I1203 12:11:14.121046 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b2qlw\" (UniqueName: \"kubernetes.io/projected/01e1d9d4-9390-460c-86dc-f2cfb5148209-kube-api-access-b2qlw\") pod \"ceilometer-0\" (UID: \"01e1d9d4-9390-460c-86dc-f2cfb5148209\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:11:14 crc kubenswrapper[5052]: I1203 12:11:14.121108 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/971c703f-d93b-4ae2-b8c2-4d9458793408-etc-machine-id\") pod \"cinder-db-sync-tgpn8\" (UID: \"971c703f-d93b-4ae2-b8c2-4d9458793408\") " pod="openstack-kuttl-tests/cinder-db-sync-tgpn8" Dec 03 12:11:14 crc kubenswrapper[5052]: I1203 12:11:14.121136 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/971c703f-d93b-4ae2-b8c2-4d9458793408-config-data\") pod \"cinder-db-sync-tgpn8\" (UID: \"971c703f-d93b-4ae2-b8c2-4d9458793408\") " pod="openstack-kuttl-tests/cinder-db-sync-tgpn8" Dec 03 12:11:14 crc kubenswrapper[5052]: I1203 12:11:14.121169 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/01e1d9d4-9390-460c-86dc-f2cfb5148209-log-httpd\") pod \"ceilometer-0\" (UID: \"01e1d9d4-9390-460c-86dc-f2cfb5148209\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:11:14 crc kubenswrapper[5052]: I1203 12:11:14.121208 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/971c703f-d93b-4ae2-b8c2-4d9458793408-db-sync-config-data\") pod \"cinder-db-sync-tgpn8\" (UID: \"971c703f-d93b-4ae2-b8c2-4d9458793408\") " pod="openstack-kuttl-tests/cinder-db-sync-tgpn8" Dec 03 12:11:14 crc kubenswrapper[5052]: I1203 12:11:14.121231 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/971c703f-d93b-4ae2-b8c2-4d9458793408-scripts\") pod \"cinder-db-sync-tgpn8\" (UID: \"971c703f-d93b-4ae2-b8c2-4d9458793408\") " pod="openstack-kuttl-tests/cinder-db-sync-tgpn8" Dec 03 12:11:14 crc kubenswrapper[5052]: I1203 12:11:14.121277 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/01e1d9d4-9390-460c-86dc-f2cfb5148209-scripts\") pod \"ceilometer-0\" (UID: \"01e1d9d4-9390-460c-86dc-f2cfb5148209\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:11:14 crc kubenswrapper[5052]: I1203 12:11:14.121307 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/971c703f-d93b-4ae2-b8c2-4d9458793408-combined-ca-bundle\") pod \"cinder-db-sync-tgpn8\" (UID: \"971c703f-d93b-4ae2-b8c2-4d9458793408\") " pod="openstack-kuttl-tests/cinder-db-sync-tgpn8" Dec 03 12:11:14 crc kubenswrapper[5052]: I1203 12:11:14.121334 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/01e1d9d4-9390-460c-86dc-f2cfb5148209-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"01e1d9d4-9390-460c-86dc-f2cfb5148209\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:11:14 crc kubenswrapper[5052]: I1203 12:11:14.121367 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/01e1d9d4-9390-460c-86dc-f2cfb5148209-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"01e1d9d4-9390-460c-86dc-f2cfb5148209\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:11:14 crc kubenswrapper[5052]: I1203 12:11:14.121394 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/01e1d9d4-9390-460c-86dc-f2cfb5148209-run-httpd\") pod \"ceilometer-0\" (UID: \"01e1d9d4-9390-460c-86dc-f2cfb5148209\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:11:14 crc kubenswrapper[5052]: I1203 12:11:14.121416 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/01e1d9d4-9390-460c-86dc-f2cfb5148209-config-data\") pod \"ceilometer-0\" (UID: \"01e1d9d4-9390-460c-86dc-f2cfb5148209\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:11:14 crc kubenswrapper[5052]: I1203 12:11:14.121468 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zsh28\" (UniqueName: \"kubernetes.io/projected/971c703f-d93b-4ae2-b8c2-4d9458793408-kube-api-access-zsh28\") pod \"cinder-db-sync-tgpn8\" (UID: \"971c703f-d93b-4ae2-b8c2-4d9458793408\") " pod="openstack-kuttl-tests/cinder-db-sync-tgpn8" Dec 03 12:11:14 crc kubenswrapper[5052]: I1203 12:11:14.125415 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-db-sync-cbdcz"] Dec 03 12:11:14 crc kubenswrapper[5052]: I1203 12:11:14.164972 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-db-sync-tgpn8"] Dec 03 12:11:14 crc kubenswrapper[5052]: I1203 12:11:14.181881 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/barbican-db-sync-9k6nv"] Dec 03 12:11:14 crc kubenswrapper[5052]: I1203 12:11:14.183077 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-db-sync-9k6nv" Dec 03 12:11:14 crc kubenswrapper[5052]: I1203 12:11:14.186646 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"barbican-config-data" Dec 03 12:11:14 crc kubenswrapper[5052]: I1203 12:11:14.186862 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"barbican-barbican-dockercfg-k9mf5" Dec 03 12:11:14 crc kubenswrapper[5052]: I1203 12:11:14.201209 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/placement-db-sync-xr2pm"] Dec 03 12:11:14 crc kubenswrapper[5052]: I1203 12:11:14.202444 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-db-sync-xr2pm" Dec 03 12:11:14 crc kubenswrapper[5052]: I1203 12:11:14.204273 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"placement-placement-dockercfg-xvh6b" Dec 03 12:11:14 crc kubenswrapper[5052]: I1203 12:11:14.206978 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"placement-scripts" Dec 03 12:11:14 crc kubenswrapper[5052]: I1203 12:11:14.214442 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"placement-config-data" Dec 03 12:11:14 crc kubenswrapper[5052]: I1203 12:11:14.222543 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-db-sync-9k6nv"] Dec 03 12:11:14 crc kubenswrapper[5052]: I1203 12:11:14.222588 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/01e1d9d4-9390-460c-86dc-f2cfb5148209-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"01e1d9d4-9390-460c-86dc-f2cfb5148209\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:11:14 crc kubenswrapper[5052]: I1203 12:11:14.222683 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/049d1d7a-9b34-4d27-a922-ebb89410b3fa-db-sync-config-data\") pod \"barbican-db-sync-9k6nv\" (UID: \"049d1d7a-9b34-4d27-a922-ebb89410b3fa\") " pod="openstack-kuttl-tests/barbican-db-sync-9k6nv" Dec 03 12:11:14 crc kubenswrapper[5052]: I1203 12:11:14.222740 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/01e1d9d4-9390-460c-86dc-f2cfb5148209-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"01e1d9d4-9390-460c-86dc-f2cfb5148209\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:11:14 crc kubenswrapper[5052]: I1203 12:11:14.222768 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/01e1d9d4-9390-460c-86dc-f2cfb5148209-run-httpd\") pod \"ceilometer-0\" (UID: \"01e1d9d4-9390-460c-86dc-f2cfb5148209\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:11:14 crc kubenswrapper[5052]: I1203 12:11:14.222818 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/01e1d9d4-9390-460c-86dc-f2cfb5148209-config-data\") pod \"ceilometer-0\" (UID: \"01e1d9d4-9390-460c-86dc-f2cfb5148209\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:11:14 crc kubenswrapper[5052]: I1203 12:11:14.222844 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b6l42\" (UniqueName: \"kubernetes.io/projected/8474b807-15d5-405a-ade8-1e4fa8954ce6-kube-api-access-b6l42\") pod \"neutron-db-sync-cbdcz\" (UID: \"8474b807-15d5-405a-ade8-1e4fa8954ce6\") " pod="openstack-kuttl-tests/neutron-db-sync-cbdcz" Dec 03 12:11:14 crc kubenswrapper[5052]: I1203 12:11:14.222907 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-zsh28\" (UniqueName: \"kubernetes.io/projected/971c703f-d93b-4ae2-b8c2-4d9458793408-kube-api-access-zsh28\") pod \"cinder-db-sync-tgpn8\" (UID: \"971c703f-d93b-4ae2-b8c2-4d9458793408\") " pod="openstack-kuttl-tests/cinder-db-sync-tgpn8" Dec 03 12:11:14 crc kubenswrapper[5052]: I1203 12:11:14.222983 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cnnn2\" (UniqueName: \"kubernetes.io/projected/049d1d7a-9b34-4d27-a922-ebb89410b3fa-kube-api-access-cnnn2\") pod \"barbican-db-sync-9k6nv\" (UID: \"049d1d7a-9b34-4d27-a922-ebb89410b3fa\") " pod="openstack-kuttl-tests/barbican-db-sync-9k6nv" Dec 03 12:11:14 crc kubenswrapper[5052]: I1203 12:11:14.223063 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8474b807-15d5-405a-ade8-1e4fa8954ce6-combined-ca-bundle\") pod \"neutron-db-sync-cbdcz\" (UID: \"8474b807-15d5-405a-ade8-1e4fa8954ce6\") " pod="openstack-kuttl-tests/neutron-db-sync-cbdcz" Dec 03 12:11:14 crc kubenswrapper[5052]: I1203 12:11:14.223085 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/8474b807-15d5-405a-ade8-1e4fa8954ce6-config\") pod \"neutron-db-sync-cbdcz\" (UID: \"8474b807-15d5-405a-ade8-1e4fa8954ce6\") " pod="openstack-kuttl-tests/neutron-db-sync-cbdcz" Dec 03 12:11:14 crc kubenswrapper[5052]: I1203 12:11:14.223141 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-b2qlw\" (UniqueName: \"kubernetes.io/projected/01e1d9d4-9390-460c-86dc-f2cfb5148209-kube-api-access-b2qlw\") pod \"ceilometer-0\" (UID: \"01e1d9d4-9390-460c-86dc-f2cfb5148209\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:11:14 crc kubenswrapper[5052]: I1203 12:11:14.223173 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/971c703f-d93b-4ae2-b8c2-4d9458793408-etc-machine-id\") pod \"cinder-db-sync-tgpn8\" (UID: \"971c703f-d93b-4ae2-b8c2-4d9458793408\") " pod="openstack-kuttl-tests/cinder-db-sync-tgpn8" Dec 03 12:11:14 crc kubenswrapper[5052]: I1203 12:11:14.223222 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/971c703f-d93b-4ae2-b8c2-4d9458793408-config-data\") pod \"cinder-db-sync-tgpn8\" (UID: \"971c703f-d93b-4ae2-b8c2-4d9458793408\") " pod="openstack-kuttl-tests/cinder-db-sync-tgpn8" Dec 03 12:11:14 crc kubenswrapper[5052]: I1203 12:11:14.223254 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/01e1d9d4-9390-460c-86dc-f2cfb5148209-log-httpd\") pod \"ceilometer-0\" (UID: \"01e1d9d4-9390-460c-86dc-f2cfb5148209\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:11:14 crc kubenswrapper[5052]: I1203 12:11:14.223310 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/971c703f-d93b-4ae2-b8c2-4d9458793408-db-sync-config-data\") pod \"cinder-db-sync-tgpn8\" (UID: \"971c703f-d93b-4ae2-b8c2-4d9458793408\") " pod="openstack-kuttl-tests/cinder-db-sync-tgpn8" Dec 03 12:11:14 crc kubenswrapper[5052]: I1203 12:11:14.223333 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/971c703f-d93b-4ae2-b8c2-4d9458793408-scripts\") pod \"cinder-db-sync-tgpn8\" (UID: \"971c703f-d93b-4ae2-b8c2-4d9458793408\") " pod="openstack-kuttl-tests/cinder-db-sync-tgpn8" Dec 03 12:11:14 crc kubenswrapper[5052]: I1203 12:11:14.223775 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/01e1d9d4-9390-460c-86dc-f2cfb5148209-run-httpd\") pod \"ceilometer-0\" (UID: \"01e1d9d4-9390-460c-86dc-f2cfb5148209\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:11:14 crc kubenswrapper[5052]: I1203 12:11:14.223879 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/971c703f-d93b-4ae2-b8c2-4d9458793408-etc-machine-id\") pod \"cinder-db-sync-tgpn8\" (UID: \"971c703f-d93b-4ae2-b8c2-4d9458793408\") " pod="openstack-kuttl-tests/cinder-db-sync-tgpn8" Dec 03 12:11:14 crc kubenswrapper[5052]: I1203 12:11:14.226818 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/01e1d9d4-9390-460c-86dc-f2cfb5148209-log-httpd\") pod \"ceilometer-0\" (UID: \"01e1d9d4-9390-460c-86dc-f2cfb5148209\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:11:14 crc kubenswrapper[5052]: I1203 12:11:14.226952 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/01e1d9d4-9390-460c-86dc-f2cfb5148209-scripts\") pod \"ceilometer-0\" (UID: \"01e1d9d4-9390-460c-86dc-f2cfb5148209\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:11:14 crc kubenswrapper[5052]: I1203 12:11:14.226998 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/049d1d7a-9b34-4d27-a922-ebb89410b3fa-combined-ca-bundle\") pod \"barbican-db-sync-9k6nv\" (UID: \"049d1d7a-9b34-4d27-a922-ebb89410b3fa\") " pod="openstack-kuttl-tests/barbican-db-sync-9k6nv" Dec 03 12:11:14 crc kubenswrapper[5052]: I1203 12:11:14.227031 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/971c703f-d93b-4ae2-b8c2-4d9458793408-combined-ca-bundle\") pod \"cinder-db-sync-tgpn8\" (UID: \"971c703f-d93b-4ae2-b8c2-4d9458793408\") " pod="openstack-kuttl-tests/cinder-db-sync-tgpn8" Dec 03 12:11:14 crc kubenswrapper[5052]: I1203 12:11:14.230239 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/01e1d9d4-9390-460c-86dc-f2cfb5148209-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"01e1d9d4-9390-460c-86dc-f2cfb5148209\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:11:14 crc kubenswrapper[5052]: I1203 12:11:14.231282 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/01e1d9d4-9390-460c-86dc-f2cfb5148209-config-data\") pod \"ceilometer-0\" (UID: \"01e1d9d4-9390-460c-86dc-f2cfb5148209\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:11:14 crc kubenswrapper[5052]: I1203 12:11:14.231616 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/971c703f-d93b-4ae2-b8c2-4d9458793408-db-sync-config-data\") pod \"cinder-db-sync-tgpn8\" (UID: \"971c703f-d93b-4ae2-b8c2-4d9458793408\") " pod="openstack-kuttl-tests/cinder-db-sync-tgpn8" Dec 03 12:11:14 crc kubenswrapper[5052]: I1203 12:11:14.231720 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/971c703f-d93b-4ae2-b8c2-4d9458793408-combined-ca-bundle\") pod \"cinder-db-sync-tgpn8\" (UID: \"971c703f-d93b-4ae2-b8c2-4d9458793408\") " pod="openstack-kuttl-tests/cinder-db-sync-tgpn8" Dec 03 12:11:14 crc kubenswrapper[5052]: I1203 12:11:14.234817 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/01e1d9d4-9390-460c-86dc-f2cfb5148209-scripts\") pod \"ceilometer-0\" (UID: \"01e1d9d4-9390-460c-86dc-f2cfb5148209\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:11:14 crc kubenswrapper[5052]: I1203 12:11:14.238792 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/971c703f-d93b-4ae2-b8c2-4d9458793408-scripts\") pod \"cinder-db-sync-tgpn8\" (UID: \"971c703f-d93b-4ae2-b8c2-4d9458793408\") " pod="openstack-kuttl-tests/cinder-db-sync-tgpn8" Dec 03 12:11:14 crc kubenswrapper[5052]: I1203 12:11:14.251485 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/placement-db-sync-xr2pm"] Dec 03 12:11:14 crc kubenswrapper[5052]: I1203 12:11:14.267211 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/971c703f-d93b-4ae2-b8c2-4d9458793408-config-data\") pod \"cinder-db-sync-tgpn8\" (UID: \"971c703f-d93b-4ae2-b8c2-4d9458793408\") " pod="openstack-kuttl-tests/cinder-db-sync-tgpn8" Dec 03 12:11:14 crc kubenswrapper[5052]: I1203 12:11:14.267832 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/01e1d9d4-9390-460c-86dc-f2cfb5148209-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"01e1d9d4-9390-460c-86dc-f2cfb5148209\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:11:14 crc kubenswrapper[5052]: I1203 12:11:14.268374 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-zsh28\" (UniqueName: \"kubernetes.io/projected/971c703f-d93b-4ae2-b8c2-4d9458793408-kube-api-access-zsh28\") pod \"cinder-db-sync-tgpn8\" (UID: \"971c703f-d93b-4ae2-b8c2-4d9458793408\") " pod="openstack-kuttl-tests/cinder-db-sync-tgpn8" Dec 03 12:11:14 crc kubenswrapper[5052]: I1203 12:11:14.276962 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-b2qlw\" (UniqueName: \"kubernetes.io/projected/01e1d9d4-9390-460c-86dc-f2cfb5148209-kube-api-access-b2qlw\") pod \"ceilometer-0\" (UID: \"01e1d9d4-9390-460c-86dc-f2cfb5148209\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:11:14 crc kubenswrapper[5052]: I1203 12:11:14.334305 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/049d1d7a-9b34-4d27-a922-ebb89410b3fa-combined-ca-bundle\") pod \"barbican-db-sync-9k6nv\" (UID: \"049d1d7a-9b34-4d27-a922-ebb89410b3fa\") " pod="openstack-kuttl-tests/barbican-db-sync-9k6nv" Dec 03 12:11:14 crc kubenswrapper[5052]: I1203 12:11:14.334362 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/db210aa9-d72d-4114-945c-0aa978c27a18-combined-ca-bundle\") pod \"placement-db-sync-xr2pm\" (UID: \"db210aa9-d72d-4114-945c-0aa978c27a18\") " pod="openstack-kuttl-tests/placement-db-sync-xr2pm" Dec 03 12:11:14 crc kubenswrapper[5052]: I1203 12:11:14.334390 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/049d1d7a-9b34-4d27-a922-ebb89410b3fa-db-sync-config-data\") pod \"barbican-db-sync-9k6nv\" (UID: \"049d1d7a-9b34-4d27-a922-ebb89410b3fa\") " pod="openstack-kuttl-tests/barbican-db-sync-9k6nv" Dec 03 12:11:14 crc kubenswrapper[5052]: I1203 12:11:14.334425 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-b6l42\" (UniqueName: \"kubernetes.io/projected/8474b807-15d5-405a-ade8-1e4fa8954ce6-kube-api-access-b6l42\") pod \"neutron-db-sync-cbdcz\" (UID: \"8474b807-15d5-405a-ade8-1e4fa8954ce6\") " pod="openstack-kuttl-tests/neutron-db-sync-cbdcz" Dec 03 12:11:14 crc kubenswrapper[5052]: I1203 12:11:14.334445 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/db210aa9-d72d-4114-945c-0aa978c27a18-config-data\") pod \"placement-db-sync-xr2pm\" (UID: \"db210aa9-d72d-4114-945c-0aa978c27a18\") " pod="openstack-kuttl-tests/placement-db-sync-xr2pm" Dec 03 12:11:14 crc kubenswrapper[5052]: I1203 12:11:14.334481 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/db210aa9-d72d-4114-945c-0aa978c27a18-logs\") pod \"placement-db-sync-xr2pm\" (UID: \"db210aa9-d72d-4114-945c-0aa978c27a18\") " pod="openstack-kuttl-tests/placement-db-sync-xr2pm" Dec 03 12:11:14 crc kubenswrapper[5052]: I1203 12:11:14.334522 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-cnnn2\" (UniqueName: \"kubernetes.io/projected/049d1d7a-9b34-4d27-a922-ebb89410b3fa-kube-api-access-cnnn2\") pod \"barbican-db-sync-9k6nv\" (UID: \"049d1d7a-9b34-4d27-a922-ebb89410b3fa\") " pod="openstack-kuttl-tests/barbican-db-sync-9k6nv" Dec 03 12:11:14 crc kubenswrapper[5052]: I1203 12:11:14.334548 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8474b807-15d5-405a-ade8-1e4fa8954ce6-combined-ca-bundle\") pod \"neutron-db-sync-cbdcz\" (UID: \"8474b807-15d5-405a-ade8-1e4fa8954ce6\") " pod="openstack-kuttl-tests/neutron-db-sync-cbdcz" Dec 03 12:11:14 crc kubenswrapper[5052]: I1203 12:11:14.334568 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xq4kj\" (UniqueName: \"kubernetes.io/projected/db210aa9-d72d-4114-945c-0aa978c27a18-kube-api-access-xq4kj\") pod \"placement-db-sync-xr2pm\" (UID: \"db210aa9-d72d-4114-945c-0aa978c27a18\") " pod="openstack-kuttl-tests/placement-db-sync-xr2pm" Dec 03 12:11:14 crc kubenswrapper[5052]: I1203 12:11:14.334590 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/8474b807-15d5-405a-ade8-1e4fa8954ce6-config\") pod \"neutron-db-sync-cbdcz\" (UID: \"8474b807-15d5-405a-ade8-1e4fa8954ce6\") " pod="openstack-kuttl-tests/neutron-db-sync-cbdcz" Dec 03 12:11:14 crc kubenswrapper[5052]: I1203 12:11:14.334623 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/db210aa9-d72d-4114-945c-0aa978c27a18-scripts\") pod \"placement-db-sync-xr2pm\" (UID: \"db210aa9-d72d-4114-945c-0aa978c27a18\") " pod="openstack-kuttl-tests/placement-db-sync-xr2pm" Dec 03 12:11:14 crc kubenswrapper[5052]: I1203 12:11:14.342068 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/049d1d7a-9b34-4d27-a922-ebb89410b3fa-combined-ca-bundle\") pod \"barbican-db-sync-9k6nv\" (UID: \"049d1d7a-9b34-4d27-a922-ebb89410b3fa\") " pod="openstack-kuttl-tests/barbican-db-sync-9k6nv" Dec 03 12:11:14 crc kubenswrapper[5052]: I1203 12:11:14.346754 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/049d1d7a-9b34-4d27-a922-ebb89410b3fa-db-sync-config-data\") pod \"barbican-db-sync-9k6nv\" (UID: \"049d1d7a-9b34-4d27-a922-ebb89410b3fa\") " pod="openstack-kuttl-tests/barbican-db-sync-9k6nv" Dec 03 12:11:14 crc kubenswrapper[5052]: I1203 12:11:14.347674 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8474b807-15d5-405a-ade8-1e4fa8954ce6-combined-ca-bundle\") pod \"neutron-db-sync-cbdcz\" (UID: \"8474b807-15d5-405a-ade8-1e4fa8954ce6\") " pod="openstack-kuttl-tests/neutron-db-sync-cbdcz" Dec 03 12:11:14 crc kubenswrapper[5052]: I1203 12:11:14.352360 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/8474b807-15d5-405a-ade8-1e4fa8954ce6-config\") pod \"neutron-db-sync-cbdcz\" (UID: \"8474b807-15d5-405a-ade8-1e4fa8954ce6\") " pod="openstack-kuttl-tests/neutron-db-sync-cbdcz" Dec 03 12:11:14 crc kubenswrapper[5052]: I1203 12:11:14.357503 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:11:14 crc kubenswrapper[5052]: I1203 12:11:14.363520 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-cnnn2\" (UniqueName: \"kubernetes.io/projected/049d1d7a-9b34-4d27-a922-ebb89410b3fa-kube-api-access-cnnn2\") pod \"barbican-db-sync-9k6nv\" (UID: \"049d1d7a-9b34-4d27-a922-ebb89410b3fa\") " pod="openstack-kuttl-tests/barbican-db-sync-9k6nv" Dec 03 12:11:14 crc kubenswrapper[5052]: I1203 12:11:14.369588 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-b6l42\" (UniqueName: \"kubernetes.io/projected/8474b807-15d5-405a-ade8-1e4fa8954ce6-kube-api-access-b6l42\") pod \"neutron-db-sync-cbdcz\" (UID: \"8474b807-15d5-405a-ade8-1e4fa8954ce6\") " pod="openstack-kuttl-tests/neutron-db-sync-cbdcz" Dec 03 12:11:14 crc kubenswrapper[5052]: I1203 12:11:14.405463 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-db-sync-tgpn8" Dec 03 12:11:14 crc kubenswrapper[5052]: I1203 12:11:14.436318 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/db210aa9-d72d-4114-945c-0aa978c27a18-combined-ca-bundle\") pod \"placement-db-sync-xr2pm\" (UID: \"db210aa9-d72d-4114-945c-0aa978c27a18\") " pod="openstack-kuttl-tests/placement-db-sync-xr2pm" Dec 03 12:11:14 crc kubenswrapper[5052]: I1203 12:11:14.436376 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/db210aa9-d72d-4114-945c-0aa978c27a18-config-data\") pod \"placement-db-sync-xr2pm\" (UID: \"db210aa9-d72d-4114-945c-0aa978c27a18\") " pod="openstack-kuttl-tests/placement-db-sync-xr2pm" Dec 03 12:11:14 crc kubenswrapper[5052]: I1203 12:11:14.436413 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/db210aa9-d72d-4114-945c-0aa978c27a18-logs\") pod \"placement-db-sync-xr2pm\" (UID: \"db210aa9-d72d-4114-945c-0aa978c27a18\") " pod="openstack-kuttl-tests/placement-db-sync-xr2pm" Dec 03 12:11:14 crc kubenswrapper[5052]: I1203 12:11:14.436440 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-xq4kj\" (UniqueName: \"kubernetes.io/projected/db210aa9-d72d-4114-945c-0aa978c27a18-kube-api-access-xq4kj\") pod \"placement-db-sync-xr2pm\" (UID: \"db210aa9-d72d-4114-945c-0aa978c27a18\") " pod="openstack-kuttl-tests/placement-db-sync-xr2pm" Dec 03 12:11:14 crc kubenswrapper[5052]: I1203 12:11:14.436467 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/db210aa9-d72d-4114-945c-0aa978c27a18-scripts\") pod \"placement-db-sync-xr2pm\" (UID: \"db210aa9-d72d-4114-945c-0aa978c27a18\") " pod="openstack-kuttl-tests/placement-db-sync-xr2pm" Dec 03 12:11:14 crc kubenswrapper[5052]: I1203 12:11:14.447340 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/db210aa9-d72d-4114-945c-0aa978c27a18-scripts\") pod \"placement-db-sync-xr2pm\" (UID: \"db210aa9-d72d-4114-945c-0aa978c27a18\") " pod="openstack-kuttl-tests/placement-db-sync-xr2pm" Dec 03 12:11:14 crc kubenswrapper[5052]: I1203 12:11:14.447580 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/db210aa9-d72d-4114-945c-0aa978c27a18-config-data\") pod \"placement-db-sync-xr2pm\" (UID: \"db210aa9-d72d-4114-945c-0aa978c27a18\") " pod="openstack-kuttl-tests/placement-db-sync-xr2pm" Dec 03 12:11:14 crc kubenswrapper[5052]: I1203 12:11:14.447725 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/db210aa9-d72d-4114-945c-0aa978c27a18-logs\") pod \"placement-db-sync-xr2pm\" (UID: \"db210aa9-d72d-4114-945c-0aa978c27a18\") " pod="openstack-kuttl-tests/placement-db-sync-xr2pm" Dec 03 12:11:14 crc kubenswrapper[5052]: I1203 12:11:14.447986 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/db210aa9-d72d-4114-945c-0aa978c27a18-combined-ca-bundle\") pod \"placement-db-sync-xr2pm\" (UID: \"db210aa9-d72d-4114-945c-0aa978c27a18\") " pod="openstack-kuttl-tests/placement-db-sync-xr2pm" Dec 03 12:11:14 crc kubenswrapper[5052]: I1203 12:11:14.459696 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-bootstrap-zz8h5"] Dec 03 12:11:14 crc kubenswrapper[5052]: I1203 12:11:14.467104 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-xq4kj\" (UniqueName: \"kubernetes.io/projected/db210aa9-d72d-4114-945c-0aa978c27a18-kube-api-access-xq4kj\") pod \"placement-db-sync-xr2pm\" (UID: \"db210aa9-d72d-4114-945c-0aa978c27a18\") " pod="openstack-kuttl-tests/placement-db-sync-xr2pm" Dec 03 12:11:14 crc kubenswrapper[5052]: I1203 12:11:14.516596 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-db-sync-cbdcz" Dec 03 12:11:14 crc kubenswrapper[5052]: I1203 12:11:14.593902 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-db-sync-9k6nv" Dec 03 12:11:14 crc kubenswrapper[5052]: I1203 12:11:14.605317 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-db-sync-xr2pm" Dec 03 12:11:14 crc kubenswrapper[5052]: I1203 12:11:14.689323 5052 scope.go:117] "RemoveContainer" containerID="63298107b3d090deff9ed92f3d613e5b1c86d9137684d117500257db8ee1c805" Dec 03 12:11:14 crc kubenswrapper[5052]: E1203 12:11:14.690210 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 12:11:14 crc kubenswrapper[5052]: I1203 12:11:14.716565 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-bootstrap-zz8h5" event={"ID":"581cd00d-4ede-4150-a4ea-aeac7033f0ab","Type":"ContainerStarted","Data":"a5dbb466d13627ccde65d5493b120aca766cf69207417cb7bd07b949a9b6abc7"} Dec 03 12:11:14 crc kubenswrapper[5052]: I1203 12:11:14.902722 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:11:14 crc kubenswrapper[5052]: I1203 12:11:14.904499 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:11:14 crc kubenswrapper[5052]: I1203 12:11:14.909314 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-scripts" Dec 03 12:11:14 crc kubenswrapper[5052]: I1203 12:11:14.909418 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-glance-default-public-svc" Dec 03 12:11:14 crc kubenswrapper[5052]: I1203 12:11:14.909846 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-default-external-config-data" Dec 03 12:11:14 crc kubenswrapper[5052]: I1203 12:11:14.909897 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-glance-dockercfg-hhggs" Dec 03 12:11:14 crc kubenswrapper[5052]: I1203 12:11:14.920420 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:11:14 crc kubenswrapper[5052]: I1203 12:11:14.946498 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage20-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage20-crc\") pod \"glance-default-external-api-0\" (UID: \"f20d775c-c6a5-4810-bf71-9ce90bd5e8c3\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:11:14 crc kubenswrapper[5052]: I1203 12:11:14.946576 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/f20d775c-c6a5-4810-bf71-9ce90bd5e8c3-public-tls-certs\") pod \"glance-default-external-api-0\" (UID: \"f20d775c-c6a5-4810-bf71-9ce90bd5e8c3\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:11:14 crc kubenswrapper[5052]: I1203 12:11:14.946612 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/f20d775c-c6a5-4810-bf71-9ce90bd5e8c3-scripts\") pod \"glance-default-external-api-0\" (UID: \"f20d775c-c6a5-4810-bf71-9ce90bd5e8c3\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:11:14 crc kubenswrapper[5052]: I1203 12:11:14.946655 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f20d775c-c6a5-4810-bf71-9ce90bd5e8c3-combined-ca-bundle\") pod \"glance-default-external-api-0\" (UID: \"f20d775c-c6a5-4810-bf71-9ce90bd5e8c3\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:11:14 crc kubenswrapper[5052]: I1203 12:11:14.947335 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/f20d775c-c6a5-4810-bf71-9ce90bd5e8c3-config-data\") pod \"glance-default-external-api-0\" (UID: \"f20d775c-c6a5-4810-bf71-9ce90bd5e8c3\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:11:14 crc kubenswrapper[5052]: I1203 12:11:14.947402 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zr4fh\" (UniqueName: \"kubernetes.io/projected/f20d775c-c6a5-4810-bf71-9ce90bd5e8c3-kube-api-access-zr4fh\") pod \"glance-default-external-api-0\" (UID: \"f20d775c-c6a5-4810-bf71-9ce90bd5e8c3\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:11:14 crc kubenswrapper[5052]: I1203 12:11:14.947439 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/f20d775c-c6a5-4810-bf71-9ce90bd5e8c3-httpd-run\") pod \"glance-default-external-api-0\" (UID: \"f20d775c-c6a5-4810-bf71-9ce90bd5e8c3\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:11:14 crc kubenswrapper[5052]: I1203 12:11:14.947592 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/f20d775c-c6a5-4810-bf71-9ce90bd5e8c3-logs\") pod \"glance-default-external-api-0\" (UID: \"f20d775c-c6a5-4810-bf71-9ce90bd5e8c3\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:11:14 crc kubenswrapper[5052]: I1203 12:11:14.954385 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:11:14 crc kubenswrapper[5052]: I1203 12:11:14.962505 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:11:14 crc kubenswrapper[5052]: I1203 12:11:14.966418 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-glance-default-internal-svc" Dec 03 12:11:14 crc kubenswrapper[5052]: I1203 12:11:14.966620 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:11:14 crc kubenswrapper[5052]: I1203 12:11:14.966679 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-default-internal-config-data" Dec 03 12:11:14 crc kubenswrapper[5052]: I1203 12:11:14.973238 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:11:15 crc kubenswrapper[5052]: I1203 12:11:15.048970 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/f20d775c-c6a5-4810-bf71-9ce90bd5e8c3-config-data\") pod \"glance-default-external-api-0\" (UID: \"f20d775c-c6a5-4810-bf71-9ce90bd5e8c3\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:11:15 crc kubenswrapper[5052]: I1203 12:11:15.049044 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6ef9faa8-7b50-4812-83ff-89ad2df492ee-combined-ca-bundle\") pod \"glance-default-internal-api-0\" (UID: \"6ef9faa8-7b50-4812-83ff-89ad2df492ee\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:11:15 crc kubenswrapper[5052]: I1203 12:11:15.049094 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-zr4fh\" (UniqueName: \"kubernetes.io/projected/f20d775c-c6a5-4810-bf71-9ce90bd5e8c3-kube-api-access-zr4fh\") pod \"glance-default-external-api-0\" (UID: \"f20d775c-c6a5-4810-bf71-9ce90bd5e8c3\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:11:15 crc kubenswrapper[5052]: I1203 12:11:15.049131 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/6ef9faa8-7b50-4812-83ff-89ad2df492ee-logs\") pod \"glance-default-internal-api-0\" (UID: \"6ef9faa8-7b50-4812-83ff-89ad2df492ee\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:11:15 crc kubenswrapper[5052]: I1203 12:11:15.049200 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/f20d775c-c6a5-4810-bf71-9ce90bd5e8c3-httpd-run\") pod \"glance-default-external-api-0\" (UID: \"f20d775c-c6a5-4810-bf71-9ce90bd5e8c3\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:11:15 crc kubenswrapper[5052]: I1203 12:11:15.050331 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/f20d775c-c6a5-4810-bf71-9ce90bd5e8c3-httpd-run\") pod \"glance-default-external-api-0\" (UID: \"f20d775c-c6a5-4810-bf71-9ce90bd5e8c3\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:11:15 crc kubenswrapper[5052]: I1203 12:11:15.050494 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/f20d775c-c6a5-4810-bf71-9ce90bd5e8c3-logs\") pod \"glance-default-external-api-0\" (UID: \"f20d775c-c6a5-4810-bf71-9ce90bd5e8c3\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:11:15 crc kubenswrapper[5052]: I1203 12:11:15.050542 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6ef9faa8-7b50-4812-83ff-89ad2df492ee-config-data\") pod \"glance-default-internal-api-0\" (UID: \"6ef9faa8-7b50-4812-83ff-89ad2df492ee\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:11:15 crc kubenswrapper[5052]: I1203 12:11:15.050643 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage20-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage20-crc\") pod \"glance-default-external-api-0\" (UID: \"f20d775c-c6a5-4810-bf71-9ce90bd5e8c3\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:11:15 crc kubenswrapper[5052]: I1203 12:11:15.050677 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/f20d775c-c6a5-4810-bf71-9ce90bd5e8c3-public-tls-certs\") pod \"glance-default-external-api-0\" (UID: \"f20d775c-c6a5-4810-bf71-9ce90bd5e8c3\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:11:15 crc kubenswrapper[5052]: I1203 12:11:15.050730 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/f20d775c-c6a5-4810-bf71-9ce90bd5e8c3-scripts\") pod \"glance-default-external-api-0\" (UID: \"f20d775c-c6a5-4810-bf71-9ce90bd5e8c3\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:11:15 crc kubenswrapper[5052]: I1203 12:11:15.050750 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cq8bc\" (UniqueName: \"kubernetes.io/projected/6ef9faa8-7b50-4812-83ff-89ad2df492ee-kube-api-access-cq8bc\") pod \"glance-default-internal-api-0\" (UID: \"6ef9faa8-7b50-4812-83ff-89ad2df492ee\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:11:15 crc kubenswrapper[5052]: I1203 12:11:15.050785 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/6ef9faa8-7b50-4812-83ff-89ad2df492ee-scripts\") pod \"glance-default-internal-api-0\" (UID: \"6ef9faa8-7b50-4812-83ff-89ad2df492ee\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:11:15 crc kubenswrapper[5052]: I1203 12:11:15.050846 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f20d775c-c6a5-4810-bf71-9ce90bd5e8c3-combined-ca-bundle\") pod \"glance-default-external-api-0\" (UID: \"f20d775c-c6a5-4810-bf71-9ce90bd5e8c3\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:11:15 crc kubenswrapper[5052]: I1203 12:11:15.050884 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/6ef9faa8-7b50-4812-83ff-89ad2df492ee-httpd-run\") pod \"glance-default-internal-api-0\" (UID: \"6ef9faa8-7b50-4812-83ff-89ad2df492ee\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:11:15 crc kubenswrapper[5052]: I1203 12:11:15.050940 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/6ef9faa8-7b50-4812-83ff-89ad2df492ee-internal-tls-certs\") pod \"glance-default-internal-api-0\" (UID: \"6ef9faa8-7b50-4812-83ff-89ad2df492ee\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:11:15 crc kubenswrapper[5052]: I1203 12:11:15.050959 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage03-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage03-crc\") pod \"glance-default-internal-api-0\" (UID: \"6ef9faa8-7b50-4812-83ff-89ad2df492ee\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:11:15 crc kubenswrapper[5052]: I1203 12:11:15.052174 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/f20d775c-c6a5-4810-bf71-9ce90bd5e8c3-logs\") pod \"glance-default-external-api-0\" (UID: \"f20d775c-c6a5-4810-bf71-9ce90bd5e8c3\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:11:15 crc kubenswrapper[5052]: I1203 12:11:15.052597 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage20-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage20-crc\") pod \"glance-default-external-api-0\" (UID: \"f20d775c-c6a5-4810-bf71-9ce90bd5e8c3\") device mount path \"/mnt/openstack/pv20\"" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:11:15 crc kubenswrapper[5052]: I1203 12:11:15.055718 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/f20d775c-c6a5-4810-bf71-9ce90bd5e8c3-scripts\") pod \"glance-default-external-api-0\" (UID: \"f20d775c-c6a5-4810-bf71-9ce90bd5e8c3\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:11:15 crc kubenswrapper[5052]: I1203 12:11:15.055767 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-db-sync-cbdcz"] Dec 03 12:11:15 crc kubenswrapper[5052]: I1203 12:11:15.058622 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/f20d775c-c6a5-4810-bf71-9ce90bd5e8c3-public-tls-certs\") pod \"glance-default-external-api-0\" (UID: \"f20d775c-c6a5-4810-bf71-9ce90bd5e8c3\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:11:15 crc kubenswrapper[5052]: I1203 12:11:15.060129 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f20d775c-c6a5-4810-bf71-9ce90bd5e8c3-combined-ca-bundle\") pod \"glance-default-external-api-0\" (UID: \"f20d775c-c6a5-4810-bf71-9ce90bd5e8c3\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:11:15 crc kubenswrapper[5052]: W1203 12:11:15.078621 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod971c703f_d93b_4ae2_b8c2_4d9458793408.slice/crio-67ff46954f9ac5a8310a56741301b02c4c6fadf37c49dd47104cf64b1d951f9f WatchSource:0}: Error finding container 67ff46954f9ac5a8310a56741301b02c4c6fadf37c49dd47104cf64b1d951f9f: Status 404 returned error can't find the container with id 67ff46954f9ac5a8310a56741301b02c4c6fadf37c49dd47104cf64b1d951f9f Dec 03 12:11:15 crc kubenswrapper[5052]: I1203 12:11:15.080761 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-zr4fh\" (UniqueName: \"kubernetes.io/projected/f20d775c-c6a5-4810-bf71-9ce90bd5e8c3-kube-api-access-zr4fh\") pod \"glance-default-external-api-0\" (UID: \"f20d775c-c6a5-4810-bf71-9ce90bd5e8c3\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:11:15 crc kubenswrapper[5052]: I1203 12:11:15.082268 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/f20d775c-c6a5-4810-bf71-9ce90bd5e8c3-config-data\") pod \"glance-default-external-api-0\" (UID: \"f20d775c-c6a5-4810-bf71-9ce90bd5e8c3\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:11:15 crc kubenswrapper[5052]: I1203 12:11:15.095078 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage20-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage20-crc\") pod \"glance-default-external-api-0\" (UID: \"f20d775c-c6a5-4810-bf71-9ce90bd5e8c3\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:11:15 crc kubenswrapper[5052]: I1203 12:11:15.102143 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-db-sync-tgpn8"] Dec 03 12:11:15 crc kubenswrapper[5052]: I1203 12:11:15.152186 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/6ef9faa8-7b50-4812-83ff-89ad2df492ee-httpd-run\") pod \"glance-default-internal-api-0\" (UID: \"6ef9faa8-7b50-4812-83ff-89ad2df492ee\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:11:15 crc kubenswrapper[5052]: I1203 12:11:15.152335 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/6ef9faa8-7b50-4812-83ff-89ad2df492ee-internal-tls-certs\") pod \"glance-default-internal-api-0\" (UID: \"6ef9faa8-7b50-4812-83ff-89ad2df492ee\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:11:15 crc kubenswrapper[5052]: I1203 12:11:15.152368 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage03-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage03-crc\") pod \"glance-default-internal-api-0\" (UID: \"6ef9faa8-7b50-4812-83ff-89ad2df492ee\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:11:15 crc kubenswrapper[5052]: I1203 12:11:15.152420 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6ef9faa8-7b50-4812-83ff-89ad2df492ee-combined-ca-bundle\") pod \"glance-default-internal-api-0\" (UID: \"6ef9faa8-7b50-4812-83ff-89ad2df492ee\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:11:15 crc kubenswrapper[5052]: I1203 12:11:15.152446 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/6ef9faa8-7b50-4812-83ff-89ad2df492ee-logs\") pod \"glance-default-internal-api-0\" (UID: \"6ef9faa8-7b50-4812-83ff-89ad2df492ee\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:11:15 crc kubenswrapper[5052]: I1203 12:11:15.152534 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6ef9faa8-7b50-4812-83ff-89ad2df492ee-config-data\") pod \"glance-default-internal-api-0\" (UID: \"6ef9faa8-7b50-4812-83ff-89ad2df492ee\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:11:15 crc kubenswrapper[5052]: I1203 12:11:15.152586 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage03-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage03-crc\") pod \"glance-default-internal-api-0\" (UID: \"6ef9faa8-7b50-4812-83ff-89ad2df492ee\") device mount path \"/mnt/openstack/pv03\"" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:11:15 crc kubenswrapper[5052]: I1203 12:11:15.152602 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-cq8bc\" (UniqueName: \"kubernetes.io/projected/6ef9faa8-7b50-4812-83ff-89ad2df492ee-kube-api-access-cq8bc\") pod \"glance-default-internal-api-0\" (UID: \"6ef9faa8-7b50-4812-83ff-89ad2df492ee\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:11:15 crc kubenswrapper[5052]: I1203 12:11:15.152628 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/6ef9faa8-7b50-4812-83ff-89ad2df492ee-scripts\") pod \"glance-default-internal-api-0\" (UID: \"6ef9faa8-7b50-4812-83ff-89ad2df492ee\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:11:15 crc kubenswrapper[5052]: I1203 12:11:15.153003 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/6ef9faa8-7b50-4812-83ff-89ad2df492ee-httpd-run\") pod \"glance-default-internal-api-0\" (UID: \"6ef9faa8-7b50-4812-83ff-89ad2df492ee\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:11:15 crc kubenswrapper[5052]: I1203 12:11:15.153042 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/6ef9faa8-7b50-4812-83ff-89ad2df492ee-logs\") pod \"glance-default-internal-api-0\" (UID: \"6ef9faa8-7b50-4812-83ff-89ad2df492ee\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:11:15 crc kubenswrapper[5052]: I1203 12:11:15.159191 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/6ef9faa8-7b50-4812-83ff-89ad2df492ee-scripts\") pod \"glance-default-internal-api-0\" (UID: \"6ef9faa8-7b50-4812-83ff-89ad2df492ee\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:11:15 crc kubenswrapper[5052]: I1203 12:11:15.165213 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6ef9faa8-7b50-4812-83ff-89ad2df492ee-config-data\") pod \"glance-default-internal-api-0\" (UID: \"6ef9faa8-7b50-4812-83ff-89ad2df492ee\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:11:15 crc kubenswrapper[5052]: I1203 12:11:15.166180 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6ef9faa8-7b50-4812-83ff-89ad2df492ee-combined-ca-bundle\") pod \"glance-default-internal-api-0\" (UID: \"6ef9faa8-7b50-4812-83ff-89ad2df492ee\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:11:15 crc kubenswrapper[5052]: I1203 12:11:15.166854 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/6ef9faa8-7b50-4812-83ff-89ad2df492ee-internal-tls-certs\") pod \"glance-default-internal-api-0\" (UID: \"6ef9faa8-7b50-4812-83ff-89ad2df492ee\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:11:15 crc kubenswrapper[5052]: I1203 12:11:15.171183 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-cq8bc\" (UniqueName: \"kubernetes.io/projected/6ef9faa8-7b50-4812-83ff-89ad2df492ee-kube-api-access-cq8bc\") pod \"glance-default-internal-api-0\" (UID: \"6ef9faa8-7b50-4812-83ff-89ad2df492ee\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:11:15 crc kubenswrapper[5052]: I1203 12:11:15.174161 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:11:15 crc kubenswrapper[5052]: I1203 12:11:15.203492 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage03-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage03-crc\") pod \"glance-default-internal-api-0\" (UID: \"6ef9faa8-7b50-4812-83ff-89ad2df492ee\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:11:15 crc kubenswrapper[5052]: I1203 12:11:15.251967 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/placement-db-sync-xr2pm"] Dec 03 12:11:15 crc kubenswrapper[5052]: I1203 12:11:15.260211 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-db-sync-9k6nv"] Dec 03 12:11:15 crc kubenswrapper[5052]: W1203 12:11:15.268743 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddb210aa9_d72d_4114_945c_0aa978c27a18.slice/crio-1b26b5bb220817a212890103338522656533e160a2e17b3e6d5fb0b59f0dba6d WatchSource:0}: Error finding container 1b26b5bb220817a212890103338522656533e160a2e17b3e6d5fb0b59f0dba6d: Status 404 returned error can't find the container with id 1b26b5bb220817a212890103338522656533e160a2e17b3e6d5fb0b59f0dba6d Dec 03 12:11:15 crc kubenswrapper[5052]: W1203 12:11:15.276111 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod049d1d7a_9b34_4d27_a922_ebb89410b3fa.slice/crio-1caba2386268b9a1ff276f86660f391eb7a9d879dca589e1ae6554dab47e0b90 WatchSource:0}: Error finding container 1caba2386268b9a1ff276f86660f391eb7a9d879dca589e1ae6554dab47e0b90: Status 404 returned error can't find the container with id 1caba2386268b9a1ff276f86660f391eb7a9d879dca589e1ae6554dab47e0b90 Dec 03 12:11:15 crc kubenswrapper[5052]: I1203 12:11:15.506895 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:11:15 crc kubenswrapper[5052]: I1203 12:11:15.691745 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:11:15 crc kubenswrapper[5052]: I1203 12:11:15.894878 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-db-sync-cbdcz" event={"ID":"8474b807-15d5-405a-ade8-1e4fa8954ce6","Type":"ContainerStarted","Data":"0519af1e24d904a84f986fb58221ec301343653caf8111b439f0966c5ec6f27b"} Dec 03 12:11:15 crc kubenswrapper[5052]: I1203 12:11:15.894933 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-db-sync-cbdcz" event={"ID":"8474b807-15d5-405a-ade8-1e4fa8954ce6","Type":"ContainerStarted","Data":"6bf8e70864db28e932a6d4b2efdfe6e9d604e6d6b4bf97d23af8ac4a2ee431c5"} Dec 03 12:11:15 crc kubenswrapper[5052]: I1203 12:11:15.898540 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-db-sync-xr2pm" event={"ID":"db210aa9-d72d-4114-945c-0aa978c27a18","Type":"ContainerStarted","Data":"425e1d453b59af4c319776f6782cfa58108a322f2cd520919a3ce1749cbf5b9a"} Dec 03 12:11:15 crc kubenswrapper[5052]: I1203 12:11:15.898593 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-db-sync-xr2pm" event={"ID":"db210aa9-d72d-4114-945c-0aa978c27a18","Type":"ContainerStarted","Data":"1b26b5bb220817a212890103338522656533e160a2e17b3e6d5fb0b59f0dba6d"} Dec 03 12:11:15 crc kubenswrapper[5052]: I1203 12:11:15.902107 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"f20d775c-c6a5-4810-bf71-9ce90bd5e8c3","Type":"ContainerStarted","Data":"e7c178e689723d53fc2d9acb96feb5bbb379adefb86c7b62db07dbbfb6477375"} Dec 03 12:11:15 crc kubenswrapper[5052]: I1203 12:11:15.902895 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:11:15 crc kubenswrapper[5052]: I1203 12:11:15.905331 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-db-sync-9k6nv" event={"ID":"049d1d7a-9b34-4d27-a922-ebb89410b3fa","Type":"ContainerStarted","Data":"d25bb92aee772ca0fd8734997ea7b7df542c4bbee1daf0a9eae2d669f91bf473"} Dec 03 12:11:15 crc kubenswrapper[5052]: I1203 12:11:15.905369 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-db-sync-9k6nv" event={"ID":"049d1d7a-9b34-4d27-a922-ebb89410b3fa","Type":"ContainerStarted","Data":"1caba2386268b9a1ff276f86660f391eb7a9d879dca589e1ae6554dab47e0b90"} Dec 03 12:11:15 crc kubenswrapper[5052]: I1203 12:11:15.924230 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/neutron-db-sync-cbdcz" podStartSLOduration=2.924210176 podStartE2EDuration="2.924210176s" podCreationTimestamp="2025-12-03 12:11:13 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:11:15.9191003 +0000 UTC m=+3066.507342054" watchObservedRunningTime="2025-12-03 12:11:15.924210176 +0000 UTC m=+3066.512451940" Dec 03 12:11:15 crc kubenswrapper[5052]: I1203 12:11:15.959699 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-bootstrap-zz8h5" event={"ID":"581cd00d-4ede-4150-a4ea-aeac7033f0ab","Type":"ContainerStarted","Data":"b2b84bb90c0815c25cd39fbb85a799acfc5bc3d5a578b8bf8f3a88a0db65844c"} Dec 03 12:11:15 crc kubenswrapper[5052]: I1203 12:11:15.964906 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"01e1d9d4-9390-460c-86dc-f2cfb5148209","Type":"ContainerStarted","Data":"f017781293cbe1892ae07ddac86f479884a15497b33aeb870e92a2e294dcf7e9"} Dec 03 12:11:15 crc kubenswrapper[5052]: I1203 12:11:15.979022 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-db-sync-tgpn8" event={"ID":"971c703f-d93b-4ae2-b8c2-4d9458793408","Type":"ContainerStarted","Data":"67ff46954f9ac5a8310a56741301b02c4c6fadf37c49dd47104cf64b1d951f9f"} Dec 03 12:11:16 crc kubenswrapper[5052]: I1203 12:11:16.006934 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/keystone-bootstrap-zz8h5" podStartSLOduration=3.006908067 podStartE2EDuration="3.006908067s" podCreationTimestamp="2025-12-03 12:11:13 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:11:15.994260876 +0000 UTC m=+3066.582502640" watchObservedRunningTime="2025-12-03 12:11:16.006908067 +0000 UTC m=+3066.595149831" Dec 03 12:11:16 crc kubenswrapper[5052]: I1203 12:11:16.990106 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"6ef9faa8-7b50-4812-83ff-89ad2df492ee","Type":"ContainerStarted","Data":"76dfc2f731d5d598af329d083e0f978b30a8c5fea665e6802ce210636567ced9"} Dec 03 12:11:16 crc kubenswrapper[5052]: I1203 12:11:16.992859 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-db-sync-tgpn8" event={"ID":"971c703f-d93b-4ae2-b8c2-4d9458793408","Type":"ContainerStarted","Data":"f360d4e646d5e381237bbb8eca54754e311f0f72353048f82b3866d760e09157"} Dec 03 12:11:17 crc kubenswrapper[5052]: I1203 12:11:17.077393 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:11:17 crc kubenswrapper[5052]: I1203 12:11:17.095406 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/placement-db-sync-xr2pm" podStartSLOduration=3.095377956 podStartE2EDuration="3.095377956s" podCreationTimestamp="2025-12-03 12:11:14 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:11:17.047345824 +0000 UTC m=+3067.635587588" watchObservedRunningTime="2025-12-03 12:11:17.095377956 +0000 UTC m=+3067.683619720" Dec 03 12:11:17 crc kubenswrapper[5052]: I1203 12:11:17.103287 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/cinder-db-sync-tgpn8" podStartSLOduration=4.103261401 podStartE2EDuration="4.103261401s" podCreationTimestamp="2025-12-03 12:11:13 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:11:17.092436252 +0000 UTC m=+3067.680678016" watchObservedRunningTime="2025-12-03 12:11:17.103261401 +0000 UTC m=+3067.691503165" Dec 03 12:11:17 crc kubenswrapper[5052]: I1203 12:11:17.135667 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:11:17 crc kubenswrapper[5052]: I1203 12:11:17.148732 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:11:17 crc kubenswrapper[5052]: I1203 12:11:17.150504 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/barbican-db-sync-9k6nv" podStartSLOduration=3.1504824989999998 podStartE2EDuration="3.150482499s" podCreationTimestamp="2025-12-03 12:11:14 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:11:17.110704303 +0000 UTC m=+3067.698946067" watchObservedRunningTime="2025-12-03 12:11:17.150482499 +0000 UTC m=+3067.738724263" Dec 03 12:11:18 crc kubenswrapper[5052]: I1203 12:11:18.004618 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"6ef9faa8-7b50-4812-83ff-89ad2df492ee","Type":"ContainerStarted","Data":"039cc990c9289af9cb8a79f23644db4677248760d7e9b0bed50697164e6eab50"} Dec 03 12:11:19 crc kubenswrapper[5052]: I1203 12:11:19.014731 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"f20d775c-c6a5-4810-bf71-9ce90bd5e8c3","Type":"ContainerStarted","Data":"cc8ca7e8ad14a878b059fb5d52eccae062e18662bebc256d1c0dea4fdd1a22e6"} Dec 03 12:11:22 crc kubenswrapper[5052]: I1203 12:11:22.050135 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"f20d775c-c6a5-4810-bf71-9ce90bd5e8c3","Type":"ContainerStarted","Data":"e67d5a4c4a166f47a2f3f34a35220f6c74353c6160afc30682079713c0fa19d5"} Dec 03 12:11:22 crc kubenswrapper[5052]: I1203 12:11:22.050902 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-external-api-0" podUID="f20d775c-c6a5-4810-bf71-9ce90bd5e8c3" containerName="glance-httpd" containerID="cri-o://e67d5a4c4a166f47a2f3f34a35220f6c74353c6160afc30682079713c0fa19d5" gracePeriod=30 Dec 03 12:11:22 crc kubenswrapper[5052]: I1203 12:11:22.050250 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-external-api-0" podUID="f20d775c-c6a5-4810-bf71-9ce90bd5e8c3" containerName="glance-log" containerID="cri-o://cc8ca7e8ad14a878b059fb5d52eccae062e18662bebc256d1c0dea4fdd1a22e6" gracePeriod=30 Dec 03 12:11:22 crc kubenswrapper[5052]: I1203 12:11:22.060743 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"01e1d9d4-9390-460c-86dc-f2cfb5148209","Type":"ContainerStarted","Data":"8b6afaa16b9f60cf12229f4f2a0fc34d5ea0673d3b569cbbe1a3dc7b36f0b8a6"} Dec 03 12:11:22 crc kubenswrapper[5052]: I1203 12:11:22.063236 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"6ef9faa8-7b50-4812-83ff-89ad2df492ee","Type":"ContainerStarted","Data":"7fbe26a33bf7ef3481dec97c313ee9c728b24599684814786581d02580b8116b"} Dec 03 12:11:22 crc kubenswrapper[5052]: I1203 12:11:22.063431 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-internal-api-0" podUID="6ef9faa8-7b50-4812-83ff-89ad2df492ee" containerName="glance-log" containerID="cri-o://039cc990c9289af9cb8a79f23644db4677248760d7e9b0bed50697164e6eab50" gracePeriod=30 Dec 03 12:11:22 crc kubenswrapper[5052]: I1203 12:11:22.063588 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-internal-api-0" podUID="6ef9faa8-7b50-4812-83ff-89ad2df492ee" containerName="glance-httpd" containerID="cri-o://7fbe26a33bf7ef3481dec97c313ee9c728b24599684814786581d02580b8116b" gracePeriod=30 Dec 03 12:11:22 crc kubenswrapper[5052]: I1203 12:11:22.085049 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/glance-default-external-api-0" podStartSLOduration=9.085026725 podStartE2EDuration="9.085026725s" podCreationTimestamp="2025-12-03 12:11:13 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:11:22.080880817 +0000 UTC m=+3072.669122581" watchObservedRunningTime="2025-12-03 12:11:22.085026725 +0000 UTC m=+3072.673268499" Dec 03 12:11:22 crc kubenswrapper[5052]: I1203 12:11:22.120342 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/glance-default-internal-api-0" podStartSLOduration=9.120319923 podStartE2EDuration="9.120319923s" podCreationTimestamp="2025-12-03 12:11:13 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:11:22.110176543 +0000 UTC m=+3072.698418327" watchObservedRunningTime="2025-12-03 12:11:22.120319923 +0000 UTC m=+3072.708561687" Dec 03 12:11:23 crc kubenswrapper[5052]: I1203 12:11:23.078359 5052 generic.go:334] "Generic (PLEG): container finished" podID="f20d775c-c6a5-4810-bf71-9ce90bd5e8c3" containerID="e67d5a4c4a166f47a2f3f34a35220f6c74353c6160afc30682079713c0fa19d5" exitCode=0 Dec 03 12:11:23 crc kubenswrapper[5052]: I1203 12:11:23.078910 5052 generic.go:334] "Generic (PLEG): container finished" podID="f20d775c-c6a5-4810-bf71-9ce90bd5e8c3" containerID="cc8ca7e8ad14a878b059fb5d52eccae062e18662bebc256d1c0dea4fdd1a22e6" exitCode=143 Dec 03 12:11:23 crc kubenswrapper[5052]: I1203 12:11:23.078972 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"f20d775c-c6a5-4810-bf71-9ce90bd5e8c3","Type":"ContainerDied","Data":"e67d5a4c4a166f47a2f3f34a35220f6c74353c6160afc30682079713c0fa19d5"} Dec 03 12:11:23 crc kubenswrapper[5052]: I1203 12:11:23.079020 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"f20d775c-c6a5-4810-bf71-9ce90bd5e8c3","Type":"ContainerDied","Data":"cc8ca7e8ad14a878b059fb5d52eccae062e18662bebc256d1c0dea4fdd1a22e6"} Dec 03 12:11:23 crc kubenswrapper[5052]: I1203 12:11:23.084945 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"01e1d9d4-9390-460c-86dc-f2cfb5148209","Type":"ContainerStarted","Data":"a94f5461b868f74204d9ad2fb2ddbda37b0a047c5e160973481c33c23045e8cf"} Dec 03 12:11:23 crc kubenswrapper[5052]: I1203 12:11:23.103095 5052 generic.go:334] "Generic (PLEG): container finished" podID="6ef9faa8-7b50-4812-83ff-89ad2df492ee" containerID="7fbe26a33bf7ef3481dec97c313ee9c728b24599684814786581d02580b8116b" exitCode=0 Dec 03 12:11:23 crc kubenswrapper[5052]: I1203 12:11:23.103133 5052 generic.go:334] "Generic (PLEG): container finished" podID="6ef9faa8-7b50-4812-83ff-89ad2df492ee" containerID="039cc990c9289af9cb8a79f23644db4677248760d7e9b0bed50697164e6eab50" exitCode=143 Dec 03 12:11:23 crc kubenswrapper[5052]: I1203 12:11:23.103164 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"6ef9faa8-7b50-4812-83ff-89ad2df492ee","Type":"ContainerDied","Data":"7fbe26a33bf7ef3481dec97c313ee9c728b24599684814786581d02580b8116b"} Dec 03 12:11:23 crc kubenswrapper[5052]: I1203 12:11:23.103203 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"6ef9faa8-7b50-4812-83ff-89ad2df492ee","Type":"ContainerDied","Data":"039cc990c9289af9cb8a79f23644db4677248760d7e9b0bed50697164e6eab50"} Dec 03 12:11:23 crc kubenswrapper[5052]: I1203 12:11:23.317292 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:11:23 crc kubenswrapper[5052]: I1203 12:11:23.327210 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:11:23 crc kubenswrapper[5052]: I1203 12:11:23.409329 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"glance\" (UniqueName: \"kubernetes.io/local-volume/local-storage20-crc\") pod \"f20d775c-c6a5-4810-bf71-9ce90bd5e8c3\" (UID: \"f20d775c-c6a5-4810-bf71-9ce90bd5e8c3\") " Dec 03 12:11:23 crc kubenswrapper[5052]: I1203 12:11:23.409413 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6ef9faa8-7b50-4812-83ff-89ad2df492ee-combined-ca-bundle\") pod \"6ef9faa8-7b50-4812-83ff-89ad2df492ee\" (UID: \"6ef9faa8-7b50-4812-83ff-89ad2df492ee\") " Dec 03 12:11:23 crc kubenswrapper[5052]: I1203 12:11:23.409500 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zr4fh\" (UniqueName: \"kubernetes.io/projected/f20d775c-c6a5-4810-bf71-9ce90bd5e8c3-kube-api-access-zr4fh\") pod \"f20d775c-c6a5-4810-bf71-9ce90bd5e8c3\" (UID: \"f20d775c-c6a5-4810-bf71-9ce90bd5e8c3\") " Dec 03 12:11:23 crc kubenswrapper[5052]: I1203 12:11:23.409559 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6ef9faa8-7b50-4812-83ff-89ad2df492ee-config-data\") pod \"6ef9faa8-7b50-4812-83ff-89ad2df492ee\" (UID: \"6ef9faa8-7b50-4812-83ff-89ad2df492ee\") " Dec 03 12:11:23 crc kubenswrapper[5052]: I1203 12:11:23.409579 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/6ef9faa8-7b50-4812-83ff-89ad2df492ee-scripts\") pod \"6ef9faa8-7b50-4812-83ff-89ad2df492ee\" (UID: \"6ef9faa8-7b50-4812-83ff-89ad2df492ee\") " Dec 03 12:11:23 crc kubenswrapper[5052]: I1203 12:11:23.409635 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/f20d775c-c6a5-4810-bf71-9ce90bd5e8c3-config-data\") pod \"f20d775c-c6a5-4810-bf71-9ce90bd5e8c3\" (UID: \"f20d775c-c6a5-4810-bf71-9ce90bd5e8c3\") " Dec 03 12:11:23 crc kubenswrapper[5052]: I1203 12:11:23.409657 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/6ef9faa8-7b50-4812-83ff-89ad2df492ee-logs\") pod \"6ef9faa8-7b50-4812-83ff-89ad2df492ee\" (UID: \"6ef9faa8-7b50-4812-83ff-89ad2df492ee\") " Dec 03 12:11:23 crc kubenswrapper[5052]: I1203 12:11:23.409722 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/6ef9faa8-7b50-4812-83ff-89ad2df492ee-internal-tls-certs\") pod \"6ef9faa8-7b50-4812-83ff-89ad2df492ee\" (UID: \"6ef9faa8-7b50-4812-83ff-89ad2df492ee\") " Dec 03 12:11:23 crc kubenswrapper[5052]: I1203 12:11:23.409794 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cq8bc\" (UniqueName: \"kubernetes.io/projected/6ef9faa8-7b50-4812-83ff-89ad2df492ee-kube-api-access-cq8bc\") pod \"6ef9faa8-7b50-4812-83ff-89ad2df492ee\" (UID: \"6ef9faa8-7b50-4812-83ff-89ad2df492ee\") " Dec 03 12:11:23 crc kubenswrapper[5052]: I1203 12:11:23.409899 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/f20d775c-c6a5-4810-bf71-9ce90bd5e8c3-logs\") pod \"f20d775c-c6a5-4810-bf71-9ce90bd5e8c3\" (UID: \"f20d775c-c6a5-4810-bf71-9ce90bd5e8c3\") " Dec 03 12:11:23 crc kubenswrapper[5052]: I1203 12:11:23.410158 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/f20d775c-c6a5-4810-bf71-9ce90bd5e8c3-scripts\") pod \"f20d775c-c6a5-4810-bf71-9ce90bd5e8c3\" (UID: \"f20d775c-c6a5-4810-bf71-9ce90bd5e8c3\") " Dec 03 12:11:23 crc kubenswrapper[5052]: I1203 12:11:23.410179 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"glance\" (UniqueName: \"kubernetes.io/local-volume/local-storage03-crc\") pod \"6ef9faa8-7b50-4812-83ff-89ad2df492ee\" (UID: \"6ef9faa8-7b50-4812-83ff-89ad2df492ee\") " Dec 03 12:11:23 crc kubenswrapper[5052]: I1203 12:11:23.410244 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/f20d775c-c6a5-4810-bf71-9ce90bd5e8c3-public-tls-certs\") pod \"f20d775c-c6a5-4810-bf71-9ce90bd5e8c3\" (UID: \"f20d775c-c6a5-4810-bf71-9ce90bd5e8c3\") " Dec 03 12:11:23 crc kubenswrapper[5052]: I1203 12:11:23.410266 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f20d775c-c6a5-4810-bf71-9ce90bd5e8c3-combined-ca-bundle\") pod \"f20d775c-c6a5-4810-bf71-9ce90bd5e8c3\" (UID: \"f20d775c-c6a5-4810-bf71-9ce90bd5e8c3\") " Dec 03 12:11:23 crc kubenswrapper[5052]: I1203 12:11:23.410325 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/f20d775c-c6a5-4810-bf71-9ce90bd5e8c3-httpd-run\") pod \"f20d775c-c6a5-4810-bf71-9ce90bd5e8c3\" (UID: \"f20d775c-c6a5-4810-bf71-9ce90bd5e8c3\") " Dec 03 12:11:23 crc kubenswrapper[5052]: I1203 12:11:23.410348 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/6ef9faa8-7b50-4812-83ff-89ad2df492ee-httpd-run\") pod \"6ef9faa8-7b50-4812-83ff-89ad2df492ee\" (UID: \"6ef9faa8-7b50-4812-83ff-89ad2df492ee\") " Dec 03 12:11:23 crc kubenswrapper[5052]: I1203 12:11:23.410823 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f20d775c-c6a5-4810-bf71-9ce90bd5e8c3-logs" (OuterVolumeSpecName: "logs") pod "f20d775c-c6a5-4810-bf71-9ce90bd5e8c3" (UID: "f20d775c-c6a5-4810-bf71-9ce90bd5e8c3"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:11:23 crc kubenswrapper[5052]: I1203 12:11:23.411030 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6ef9faa8-7b50-4812-83ff-89ad2df492ee-logs" (OuterVolumeSpecName: "logs") pod "6ef9faa8-7b50-4812-83ff-89ad2df492ee" (UID: "6ef9faa8-7b50-4812-83ff-89ad2df492ee"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:11:23 crc kubenswrapper[5052]: I1203 12:11:23.411918 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f20d775c-c6a5-4810-bf71-9ce90bd5e8c3-httpd-run" (OuterVolumeSpecName: "httpd-run") pod "f20d775c-c6a5-4810-bf71-9ce90bd5e8c3" (UID: "f20d775c-c6a5-4810-bf71-9ce90bd5e8c3"). InnerVolumeSpecName "httpd-run". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:11:23 crc kubenswrapper[5052]: I1203 12:11:23.414483 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f20d775c-c6a5-4810-bf71-9ce90bd5e8c3-scripts" (OuterVolumeSpecName: "scripts") pod "f20d775c-c6a5-4810-bf71-9ce90bd5e8c3" (UID: "f20d775c-c6a5-4810-bf71-9ce90bd5e8c3"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:11:23 crc kubenswrapper[5052]: I1203 12:11:23.415021 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6ef9faa8-7b50-4812-83ff-89ad2df492ee-httpd-run" (OuterVolumeSpecName: "httpd-run") pod "6ef9faa8-7b50-4812-83ff-89ad2df492ee" (UID: "6ef9faa8-7b50-4812-83ff-89ad2df492ee"). InnerVolumeSpecName "httpd-run". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:11:23 crc kubenswrapper[5052]: I1203 12:11:23.424659 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6ef9faa8-7b50-4812-83ff-89ad2df492ee-kube-api-access-cq8bc" (OuterVolumeSpecName: "kube-api-access-cq8bc") pod "6ef9faa8-7b50-4812-83ff-89ad2df492ee" (UID: "6ef9faa8-7b50-4812-83ff-89ad2df492ee"). InnerVolumeSpecName "kube-api-access-cq8bc". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:11:23 crc kubenswrapper[5052]: I1203 12:11:23.424872 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6ef9faa8-7b50-4812-83ff-89ad2df492ee-scripts" (OuterVolumeSpecName: "scripts") pod "6ef9faa8-7b50-4812-83ff-89ad2df492ee" (UID: "6ef9faa8-7b50-4812-83ff-89ad2df492ee"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:11:23 crc kubenswrapper[5052]: I1203 12:11:23.426439 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f20d775c-c6a5-4810-bf71-9ce90bd5e8c3-kube-api-access-zr4fh" (OuterVolumeSpecName: "kube-api-access-zr4fh") pod "f20d775c-c6a5-4810-bf71-9ce90bd5e8c3" (UID: "f20d775c-c6a5-4810-bf71-9ce90bd5e8c3"). InnerVolumeSpecName "kube-api-access-zr4fh". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:11:23 crc kubenswrapper[5052]: I1203 12:11:23.435772 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage03-crc" (OuterVolumeSpecName: "glance") pod "6ef9faa8-7b50-4812-83ff-89ad2df492ee" (UID: "6ef9faa8-7b50-4812-83ff-89ad2df492ee"). InnerVolumeSpecName "local-storage03-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 12:11:23 crc kubenswrapper[5052]: I1203 12:11:23.438239 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage20-crc" (OuterVolumeSpecName: "glance") pod "f20d775c-c6a5-4810-bf71-9ce90bd5e8c3" (UID: "f20d775c-c6a5-4810-bf71-9ce90bd5e8c3"). InnerVolumeSpecName "local-storage20-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 12:11:23 crc kubenswrapper[5052]: I1203 12:11:23.461860 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f20d775c-c6a5-4810-bf71-9ce90bd5e8c3-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "f20d775c-c6a5-4810-bf71-9ce90bd5e8c3" (UID: "f20d775c-c6a5-4810-bf71-9ce90bd5e8c3"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:11:23 crc kubenswrapper[5052]: I1203 12:11:23.484884 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6ef9faa8-7b50-4812-83ff-89ad2df492ee-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "6ef9faa8-7b50-4812-83ff-89ad2df492ee" (UID: "6ef9faa8-7b50-4812-83ff-89ad2df492ee"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:11:23 crc kubenswrapper[5052]: I1203 12:11:23.487180 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f20d775c-c6a5-4810-bf71-9ce90bd5e8c3-config-data" (OuterVolumeSpecName: "config-data") pod "f20d775c-c6a5-4810-bf71-9ce90bd5e8c3" (UID: "f20d775c-c6a5-4810-bf71-9ce90bd5e8c3"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:11:23 crc kubenswrapper[5052]: I1203 12:11:23.499469 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6ef9faa8-7b50-4812-83ff-89ad2df492ee-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "6ef9faa8-7b50-4812-83ff-89ad2df492ee" (UID: "6ef9faa8-7b50-4812-83ff-89ad2df492ee"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:11:23 crc kubenswrapper[5052]: I1203 12:11:23.500820 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6ef9faa8-7b50-4812-83ff-89ad2df492ee-config-data" (OuterVolumeSpecName: "config-data") pod "6ef9faa8-7b50-4812-83ff-89ad2df492ee" (UID: "6ef9faa8-7b50-4812-83ff-89ad2df492ee"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:11:23 crc kubenswrapper[5052]: I1203 12:11:23.501619 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f20d775c-c6a5-4810-bf71-9ce90bd5e8c3-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "f20d775c-c6a5-4810-bf71-9ce90bd5e8c3" (UID: "f20d775c-c6a5-4810-bf71-9ce90bd5e8c3"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:11:23 crc kubenswrapper[5052]: I1203 12:11:23.512804 5052 reconciler_common.go:293] "Volume detached for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/f20d775c-c6a5-4810-bf71-9ce90bd5e8c3-httpd-run\") on node \"crc\" DevicePath \"\"" Dec 03 12:11:23 crc kubenswrapper[5052]: I1203 12:11:23.512830 5052 reconciler_common.go:293] "Volume detached for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/6ef9faa8-7b50-4812-83ff-89ad2df492ee-httpd-run\") on node \"crc\" DevicePath \"\"" Dec 03 12:11:23 crc kubenswrapper[5052]: I1203 12:11:23.512866 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage20-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage20-crc\") on node \"crc\" " Dec 03 12:11:23 crc kubenswrapper[5052]: I1203 12:11:23.512876 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6ef9faa8-7b50-4812-83ff-89ad2df492ee-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:11:23 crc kubenswrapper[5052]: I1203 12:11:23.512890 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-zr4fh\" (UniqueName: \"kubernetes.io/projected/f20d775c-c6a5-4810-bf71-9ce90bd5e8c3-kube-api-access-zr4fh\") on node \"crc\" DevicePath \"\"" Dec 03 12:11:23 crc kubenswrapper[5052]: I1203 12:11:23.512899 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6ef9faa8-7b50-4812-83ff-89ad2df492ee-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:11:23 crc kubenswrapper[5052]: I1203 12:11:23.512907 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/6ef9faa8-7b50-4812-83ff-89ad2df492ee-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:11:23 crc kubenswrapper[5052]: I1203 12:11:23.512915 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/f20d775c-c6a5-4810-bf71-9ce90bd5e8c3-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:11:23 crc kubenswrapper[5052]: I1203 12:11:23.512923 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/6ef9faa8-7b50-4812-83ff-89ad2df492ee-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:11:23 crc kubenswrapper[5052]: I1203 12:11:23.512932 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/6ef9faa8-7b50-4812-83ff-89ad2df492ee-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:11:23 crc kubenswrapper[5052]: I1203 12:11:23.512940 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-cq8bc\" (UniqueName: \"kubernetes.io/projected/6ef9faa8-7b50-4812-83ff-89ad2df492ee-kube-api-access-cq8bc\") on node \"crc\" DevicePath \"\"" Dec 03 12:11:23 crc kubenswrapper[5052]: I1203 12:11:23.512948 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/f20d775c-c6a5-4810-bf71-9ce90bd5e8c3-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:11:23 crc kubenswrapper[5052]: I1203 12:11:23.512955 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/f20d775c-c6a5-4810-bf71-9ce90bd5e8c3-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:11:23 crc kubenswrapper[5052]: I1203 12:11:23.512975 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage03-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage03-crc\") on node \"crc\" " Dec 03 12:11:23 crc kubenswrapper[5052]: I1203 12:11:23.512983 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/f20d775c-c6a5-4810-bf71-9ce90bd5e8c3-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:11:23 crc kubenswrapper[5052]: I1203 12:11:23.512992 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f20d775c-c6a5-4810-bf71-9ce90bd5e8c3-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:11:23 crc kubenswrapper[5052]: I1203 12:11:23.529965 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage20-crc" (UniqueName: "kubernetes.io/local-volume/local-storage20-crc") on node "crc" Dec 03 12:11:23 crc kubenswrapper[5052]: I1203 12:11:23.531426 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage03-crc" (UniqueName: "kubernetes.io/local-volume/local-storage03-crc") on node "crc" Dec 03 12:11:23 crc kubenswrapper[5052]: I1203 12:11:23.615175 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage20-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage20-crc\") on node \"crc\" DevicePath \"\"" Dec 03 12:11:23 crc kubenswrapper[5052]: I1203 12:11:23.615229 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage03-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage03-crc\") on node \"crc\" DevicePath \"\"" Dec 03 12:11:24 crc kubenswrapper[5052]: I1203 12:11:24.122596 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"f20d775c-c6a5-4810-bf71-9ce90bd5e8c3","Type":"ContainerDied","Data":"e7c178e689723d53fc2d9acb96feb5bbb379adefb86c7b62db07dbbfb6477375"} Dec 03 12:11:24 crc kubenswrapper[5052]: I1203 12:11:24.123034 5052 scope.go:117] "RemoveContainer" containerID="e67d5a4c4a166f47a2f3f34a35220f6c74353c6160afc30682079713c0fa19d5" Dec 03 12:11:24 crc kubenswrapper[5052]: I1203 12:11:24.123283 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:11:24 crc kubenswrapper[5052]: I1203 12:11:24.130233 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"6ef9faa8-7b50-4812-83ff-89ad2df492ee","Type":"ContainerDied","Data":"76dfc2f731d5d598af329d083e0f978b30a8c5fea665e6802ce210636567ced9"} Dec 03 12:11:24 crc kubenswrapper[5052]: I1203 12:11:24.130367 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:11:24 crc kubenswrapper[5052]: I1203 12:11:24.187162 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:11:24 crc kubenswrapper[5052]: I1203 12:11:24.215942 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:11:24 crc kubenswrapper[5052]: I1203 12:11:24.225469 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:11:24 crc kubenswrapper[5052]: I1203 12:11:24.235292 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:11:24 crc kubenswrapper[5052]: I1203 12:11:24.243458 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:11:24 crc kubenswrapper[5052]: E1203 12:11:24.243981 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="f20d775c-c6a5-4810-bf71-9ce90bd5e8c3" containerName="glance-log" Dec 03 12:11:24 crc kubenswrapper[5052]: I1203 12:11:24.244003 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="f20d775c-c6a5-4810-bf71-9ce90bd5e8c3" containerName="glance-log" Dec 03 12:11:24 crc kubenswrapper[5052]: E1203 12:11:24.244033 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="6ef9faa8-7b50-4812-83ff-89ad2df492ee" containerName="glance-log" Dec 03 12:11:24 crc kubenswrapper[5052]: I1203 12:11:24.244040 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="6ef9faa8-7b50-4812-83ff-89ad2df492ee" containerName="glance-log" Dec 03 12:11:24 crc kubenswrapper[5052]: E1203 12:11:24.244051 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="6ef9faa8-7b50-4812-83ff-89ad2df492ee" containerName="glance-httpd" Dec 03 12:11:24 crc kubenswrapper[5052]: I1203 12:11:24.244058 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="6ef9faa8-7b50-4812-83ff-89ad2df492ee" containerName="glance-httpd" Dec 03 12:11:24 crc kubenswrapper[5052]: E1203 12:11:24.244073 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="f20d775c-c6a5-4810-bf71-9ce90bd5e8c3" containerName="glance-httpd" Dec 03 12:11:24 crc kubenswrapper[5052]: I1203 12:11:24.244079 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="f20d775c-c6a5-4810-bf71-9ce90bd5e8c3" containerName="glance-httpd" Dec 03 12:11:24 crc kubenswrapper[5052]: I1203 12:11:24.244244 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="6ef9faa8-7b50-4812-83ff-89ad2df492ee" containerName="glance-httpd" Dec 03 12:11:24 crc kubenswrapper[5052]: I1203 12:11:24.244269 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="6ef9faa8-7b50-4812-83ff-89ad2df492ee" containerName="glance-log" Dec 03 12:11:24 crc kubenswrapper[5052]: I1203 12:11:24.244281 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="f20d775c-c6a5-4810-bf71-9ce90bd5e8c3" containerName="glance-log" Dec 03 12:11:24 crc kubenswrapper[5052]: I1203 12:11:24.244295 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="f20d775c-c6a5-4810-bf71-9ce90bd5e8c3" containerName="glance-httpd" Dec 03 12:11:24 crc kubenswrapper[5052]: I1203 12:11:24.246282 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:11:24 crc kubenswrapper[5052]: I1203 12:11:24.250223 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-glance-dockercfg-hhggs" Dec 03 12:11:24 crc kubenswrapper[5052]: I1203 12:11:24.250731 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-glance-default-public-svc" Dec 03 12:11:24 crc kubenswrapper[5052]: I1203 12:11:24.251026 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-default-external-config-data" Dec 03 12:11:24 crc kubenswrapper[5052]: I1203 12:11:24.251268 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-scripts" Dec 03 12:11:24 crc kubenswrapper[5052]: I1203 12:11:24.252149 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:11:24 crc kubenswrapper[5052]: I1203 12:11:24.254141 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:11:24 crc kubenswrapper[5052]: I1203 12:11:24.257202 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-default-internal-config-data" Dec 03 12:11:24 crc kubenswrapper[5052]: I1203 12:11:24.257646 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-glance-default-internal-svc" Dec 03 12:11:24 crc kubenswrapper[5052]: I1203 12:11:24.260824 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:11:24 crc kubenswrapper[5052]: I1203 12:11:24.284643 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:11:24 crc kubenswrapper[5052]: I1203 12:11:24.337166 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/03919b22-bf40-4bf2-88cb-fdefc426072c-logs\") pod \"glance-default-internal-api-0\" (UID: \"03919b22-bf40-4bf2-88cb-fdefc426072c\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:11:24 crc kubenswrapper[5052]: I1203 12:11:24.337252 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2jqgq\" (UniqueName: \"kubernetes.io/projected/03919b22-bf40-4bf2-88cb-fdefc426072c-kube-api-access-2jqgq\") pod \"glance-default-internal-api-0\" (UID: \"03919b22-bf40-4bf2-88cb-fdefc426072c\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:11:24 crc kubenswrapper[5052]: I1203 12:11:24.337283 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage20-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage20-crc\") pod \"glance-default-external-api-0\" (UID: \"0fd9a636-d4a2-45f7-8a00-0b3a2fe364cb\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:11:24 crc kubenswrapper[5052]: I1203 12:11:24.337493 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/0fd9a636-d4a2-45f7-8a00-0b3a2fe364cb-config-data\") pod \"glance-default-external-api-0\" (UID: \"0fd9a636-d4a2-45f7-8a00-0b3a2fe364cb\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:11:24 crc kubenswrapper[5052]: I1203 12:11:24.337646 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/0fd9a636-d4a2-45f7-8a00-0b3a2fe364cb-scripts\") pod \"glance-default-external-api-0\" (UID: \"0fd9a636-d4a2-45f7-8a00-0b3a2fe364cb\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:11:24 crc kubenswrapper[5052]: I1203 12:11:24.337740 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0fd9a636-d4a2-45f7-8a00-0b3a2fe364cb-combined-ca-bundle\") pod \"glance-default-external-api-0\" (UID: \"0fd9a636-d4a2-45f7-8a00-0b3a2fe364cb\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:11:24 crc kubenswrapper[5052]: I1203 12:11:24.337805 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/0fd9a636-d4a2-45f7-8a00-0b3a2fe364cb-logs\") pod \"glance-default-external-api-0\" (UID: \"0fd9a636-d4a2-45f7-8a00-0b3a2fe364cb\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:11:24 crc kubenswrapper[5052]: I1203 12:11:24.337835 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/03919b22-bf40-4bf2-88cb-fdefc426072c-scripts\") pod \"glance-default-internal-api-0\" (UID: \"03919b22-bf40-4bf2-88cb-fdefc426072c\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:11:24 crc kubenswrapper[5052]: I1203 12:11:24.337898 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/0fd9a636-d4a2-45f7-8a00-0b3a2fe364cb-public-tls-certs\") pod \"glance-default-external-api-0\" (UID: \"0fd9a636-d4a2-45f7-8a00-0b3a2fe364cb\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:11:24 crc kubenswrapper[5052]: I1203 12:11:24.338015 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage03-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage03-crc\") pod \"glance-default-internal-api-0\" (UID: \"03919b22-bf40-4bf2-88cb-fdefc426072c\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:11:24 crc kubenswrapper[5052]: I1203 12:11:24.338218 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jc5mm\" (UniqueName: \"kubernetes.io/projected/0fd9a636-d4a2-45f7-8a00-0b3a2fe364cb-kube-api-access-jc5mm\") pod \"glance-default-external-api-0\" (UID: \"0fd9a636-d4a2-45f7-8a00-0b3a2fe364cb\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:11:24 crc kubenswrapper[5052]: I1203 12:11:24.338256 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/03919b22-bf40-4bf2-88cb-fdefc426072c-config-data\") pod \"glance-default-internal-api-0\" (UID: \"03919b22-bf40-4bf2-88cb-fdefc426072c\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:11:24 crc kubenswrapper[5052]: I1203 12:11:24.338315 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/03919b22-bf40-4bf2-88cb-fdefc426072c-combined-ca-bundle\") pod \"glance-default-internal-api-0\" (UID: \"03919b22-bf40-4bf2-88cb-fdefc426072c\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:11:24 crc kubenswrapper[5052]: I1203 12:11:24.338461 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/03919b22-bf40-4bf2-88cb-fdefc426072c-internal-tls-certs\") pod \"glance-default-internal-api-0\" (UID: \"03919b22-bf40-4bf2-88cb-fdefc426072c\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:11:24 crc kubenswrapper[5052]: I1203 12:11:24.338499 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/0fd9a636-d4a2-45f7-8a00-0b3a2fe364cb-httpd-run\") pod \"glance-default-external-api-0\" (UID: \"0fd9a636-d4a2-45f7-8a00-0b3a2fe364cb\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:11:24 crc kubenswrapper[5052]: I1203 12:11:24.338563 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/03919b22-bf40-4bf2-88cb-fdefc426072c-httpd-run\") pod \"glance-default-internal-api-0\" (UID: \"03919b22-bf40-4bf2-88cb-fdefc426072c\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:11:24 crc kubenswrapper[5052]: I1203 12:11:24.439673 5052 scope.go:117] "RemoveContainer" containerID="cc8ca7e8ad14a878b059fb5d52eccae062e18662bebc256d1c0dea4fdd1a22e6" Dec 03 12:11:24 crc kubenswrapper[5052]: I1203 12:11:24.440988 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/03919b22-bf40-4bf2-88cb-fdefc426072c-logs\") pod \"glance-default-internal-api-0\" (UID: \"03919b22-bf40-4bf2-88cb-fdefc426072c\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:11:24 crc kubenswrapper[5052]: I1203 12:11:24.441048 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-2jqgq\" (UniqueName: \"kubernetes.io/projected/03919b22-bf40-4bf2-88cb-fdefc426072c-kube-api-access-2jqgq\") pod \"glance-default-internal-api-0\" (UID: \"03919b22-bf40-4bf2-88cb-fdefc426072c\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:11:24 crc kubenswrapper[5052]: I1203 12:11:24.441082 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage20-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage20-crc\") pod \"glance-default-external-api-0\" (UID: \"0fd9a636-d4a2-45f7-8a00-0b3a2fe364cb\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:11:24 crc kubenswrapper[5052]: I1203 12:11:24.441121 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/0fd9a636-d4a2-45f7-8a00-0b3a2fe364cb-config-data\") pod \"glance-default-external-api-0\" (UID: \"0fd9a636-d4a2-45f7-8a00-0b3a2fe364cb\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:11:24 crc kubenswrapper[5052]: I1203 12:11:24.441167 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/0fd9a636-d4a2-45f7-8a00-0b3a2fe364cb-scripts\") pod \"glance-default-external-api-0\" (UID: \"0fd9a636-d4a2-45f7-8a00-0b3a2fe364cb\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:11:24 crc kubenswrapper[5052]: I1203 12:11:24.441201 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0fd9a636-d4a2-45f7-8a00-0b3a2fe364cb-combined-ca-bundle\") pod \"glance-default-external-api-0\" (UID: \"0fd9a636-d4a2-45f7-8a00-0b3a2fe364cb\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:11:24 crc kubenswrapper[5052]: I1203 12:11:24.441225 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/0fd9a636-d4a2-45f7-8a00-0b3a2fe364cb-logs\") pod \"glance-default-external-api-0\" (UID: \"0fd9a636-d4a2-45f7-8a00-0b3a2fe364cb\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:11:24 crc kubenswrapper[5052]: I1203 12:11:24.441241 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/03919b22-bf40-4bf2-88cb-fdefc426072c-scripts\") pod \"glance-default-internal-api-0\" (UID: \"03919b22-bf40-4bf2-88cb-fdefc426072c\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:11:24 crc kubenswrapper[5052]: I1203 12:11:24.441263 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/0fd9a636-d4a2-45f7-8a00-0b3a2fe364cb-public-tls-certs\") pod \"glance-default-external-api-0\" (UID: \"0fd9a636-d4a2-45f7-8a00-0b3a2fe364cb\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:11:24 crc kubenswrapper[5052]: I1203 12:11:24.441296 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage03-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage03-crc\") pod \"glance-default-internal-api-0\" (UID: \"03919b22-bf40-4bf2-88cb-fdefc426072c\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:11:24 crc kubenswrapper[5052]: I1203 12:11:24.441351 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-jc5mm\" (UniqueName: \"kubernetes.io/projected/0fd9a636-d4a2-45f7-8a00-0b3a2fe364cb-kube-api-access-jc5mm\") pod \"glance-default-external-api-0\" (UID: \"0fd9a636-d4a2-45f7-8a00-0b3a2fe364cb\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:11:24 crc kubenswrapper[5052]: I1203 12:11:24.441379 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/03919b22-bf40-4bf2-88cb-fdefc426072c-config-data\") pod \"glance-default-internal-api-0\" (UID: \"03919b22-bf40-4bf2-88cb-fdefc426072c\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:11:24 crc kubenswrapper[5052]: I1203 12:11:24.441405 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/03919b22-bf40-4bf2-88cb-fdefc426072c-combined-ca-bundle\") pod \"glance-default-internal-api-0\" (UID: \"03919b22-bf40-4bf2-88cb-fdefc426072c\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:11:24 crc kubenswrapper[5052]: I1203 12:11:24.441476 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/03919b22-bf40-4bf2-88cb-fdefc426072c-internal-tls-certs\") pod \"glance-default-internal-api-0\" (UID: \"03919b22-bf40-4bf2-88cb-fdefc426072c\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:11:24 crc kubenswrapper[5052]: I1203 12:11:24.441530 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/0fd9a636-d4a2-45f7-8a00-0b3a2fe364cb-httpd-run\") pod \"glance-default-external-api-0\" (UID: \"0fd9a636-d4a2-45f7-8a00-0b3a2fe364cb\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:11:24 crc kubenswrapper[5052]: I1203 12:11:24.441559 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/03919b22-bf40-4bf2-88cb-fdefc426072c-httpd-run\") pod \"glance-default-internal-api-0\" (UID: \"03919b22-bf40-4bf2-88cb-fdefc426072c\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:11:24 crc kubenswrapper[5052]: I1203 12:11:24.441616 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/03919b22-bf40-4bf2-88cb-fdefc426072c-logs\") pod \"glance-default-internal-api-0\" (UID: \"03919b22-bf40-4bf2-88cb-fdefc426072c\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:11:24 crc kubenswrapper[5052]: I1203 12:11:24.441986 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/03919b22-bf40-4bf2-88cb-fdefc426072c-httpd-run\") pod \"glance-default-internal-api-0\" (UID: \"03919b22-bf40-4bf2-88cb-fdefc426072c\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:11:24 crc kubenswrapper[5052]: I1203 12:11:24.442337 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/0fd9a636-d4a2-45f7-8a00-0b3a2fe364cb-logs\") pod \"glance-default-external-api-0\" (UID: \"0fd9a636-d4a2-45f7-8a00-0b3a2fe364cb\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:11:24 crc kubenswrapper[5052]: I1203 12:11:24.443330 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage20-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage20-crc\") pod \"glance-default-external-api-0\" (UID: \"0fd9a636-d4a2-45f7-8a00-0b3a2fe364cb\") device mount path \"/mnt/openstack/pv20\"" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:11:24 crc kubenswrapper[5052]: I1203 12:11:24.443420 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage03-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage03-crc\") pod \"glance-default-internal-api-0\" (UID: \"03919b22-bf40-4bf2-88cb-fdefc426072c\") device mount path \"/mnt/openstack/pv03\"" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:11:24 crc kubenswrapper[5052]: I1203 12:11:24.448778 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/03919b22-bf40-4bf2-88cb-fdefc426072c-internal-tls-certs\") pod \"glance-default-internal-api-0\" (UID: \"03919b22-bf40-4bf2-88cb-fdefc426072c\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:11:24 crc kubenswrapper[5052]: I1203 12:11:24.451070 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/0fd9a636-d4a2-45f7-8a00-0b3a2fe364cb-httpd-run\") pod \"glance-default-external-api-0\" (UID: \"0fd9a636-d4a2-45f7-8a00-0b3a2fe364cb\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:11:24 crc kubenswrapper[5052]: I1203 12:11:24.452132 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/03919b22-bf40-4bf2-88cb-fdefc426072c-scripts\") pod \"glance-default-internal-api-0\" (UID: \"03919b22-bf40-4bf2-88cb-fdefc426072c\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:11:24 crc kubenswrapper[5052]: I1203 12:11:24.452973 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/03919b22-bf40-4bf2-88cb-fdefc426072c-combined-ca-bundle\") pod \"glance-default-internal-api-0\" (UID: \"03919b22-bf40-4bf2-88cb-fdefc426072c\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:11:24 crc kubenswrapper[5052]: I1203 12:11:24.456027 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/03919b22-bf40-4bf2-88cb-fdefc426072c-config-data\") pod \"glance-default-internal-api-0\" (UID: \"03919b22-bf40-4bf2-88cb-fdefc426072c\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:11:24 crc kubenswrapper[5052]: I1203 12:11:24.457867 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0fd9a636-d4a2-45f7-8a00-0b3a2fe364cb-combined-ca-bundle\") pod \"glance-default-external-api-0\" (UID: \"0fd9a636-d4a2-45f7-8a00-0b3a2fe364cb\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:11:24 crc kubenswrapper[5052]: I1203 12:11:24.463419 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/0fd9a636-d4a2-45f7-8a00-0b3a2fe364cb-scripts\") pod \"glance-default-external-api-0\" (UID: \"0fd9a636-d4a2-45f7-8a00-0b3a2fe364cb\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:11:24 crc kubenswrapper[5052]: I1203 12:11:24.464940 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/0fd9a636-d4a2-45f7-8a00-0b3a2fe364cb-public-tls-certs\") pod \"glance-default-external-api-0\" (UID: \"0fd9a636-d4a2-45f7-8a00-0b3a2fe364cb\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:11:24 crc kubenswrapper[5052]: I1203 12:11:24.465661 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/0fd9a636-d4a2-45f7-8a00-0b3a2fe364cb-config-data\") pod \"glance-default-external-api-0\" (UID: \"0fd9a636-d4a2-45f7-8a00-0b3a2fe364cb\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:11:24 crc kubenswrapper[5052]: I1203 12:11:24.466642 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-jc5mm\" (UniqueName: \"kubernetes.io/projected/0fd9a636-d4a2-45f7-8a00-0b3a2fe364cb-kube-api-access-jc5mm\") pod \"glance-default-external-api-0\" (UID: \"0fd9a636-d4a2-45f7-8a00-0b3a2fe364cb\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:11:24 crc kubenswrapper[5052]: I1203 12:11:24.469356 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-2jqgq\" (UniqueName: \"kubernetes.io/projected/03919b22-bf40-4bf2-88cb-fdefc426072c-kube-api-access-2jqgq\") pod \"glance-default-internal-api-0\" (UID: \"03919b22-bf40-4bf2-88cb-fdefc426072c\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:11:24 crc kubenswrapper[5052]: I1203 12:11:24.488910 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage03-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage03-crc\") pod \"glance-default-internal-api-0\" (UID: \"03919b22-bf40-4bf2-88cb-fdefc426072c\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:11:24 crc kubenswrapper[5052]: I1203 12:11:24.490650 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage20-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage20-crc\") pod \"glance-default-external-api-0\" (UID: \"0fd9a636-d4a2-45f7-8a00-0b3a2fe364cb\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:11:24 crc kubenswrapper[5052]: I1203 12:11:24.590074 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:11:24 crc kubenswrapper[5052]: I1203 12:11:24.597696 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:11:25 crc kubenswrapper[5052]: I1203 12:11:25.134444 5052 scope.go:117] "RemoveContainer" containerID="7fbe26a33bf7ef3481dec97c313ee9c728b24599684814786581d02580b8116b" Dec 03 12:11:25 crc kubenswrapper[5052]: I1203 12:11:25.312450 5052 scope.go:117] "RemoveContainer" containerID="039cc990c9289af9cb8a79f23644db4677248760d7e9b0bed50697164e6eab50" Dec 03 12:11:25 crc kubenswrapper[5052]: I1203 12:11:25.649629 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:11:25 crc kubenswrapper[5052]: I1203 12:11:25.659257 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:11:25 crc kubenswrapper[5052]: W1203 12:11:25.661050 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0fd9a636_d4a2_45f7_8a00_0b3a2fe364cb.slice/crio-7fe5008dd6a2fa7e7e752c99afa8f2187cc6b0accf3b245645b7274fe1ca2d1e WatchSource:0}: Error finding container 7fe5008dd6a2fa7e7e752c99afa8f2187cc6b0accf3b245645b7274fe1ca2d1e: Status 404 returned error can't find the container with id 7fe5008dd6a2fa7e7e752c99afa8f2187cc6b0accf3b245645b7274fe1ca2d1e Dec 03 12:11:25 crc kubenswrapper[5052]: I1203 12:11:25.673197 5052 scope.go:117] "RemoveContainer" containerID="63298107b3d090deff9ed92f3d613e5b1c86d9137684d117500257db8ee1c805" Dec 03 12:11:25 crc kubenswrapper[5052]: E1203 12:11:25.673805 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 12:11:25 crc kubenswrapper[5052]: I1203 12:11:25.685403 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6ef9faa8-7b50-4812-83ff-89ad2df492ee" path="/var/lib/kubelet/pods/6ef9faa8-7b50-4812-83ff-89ad2df492ee/volumes" Dec 03 12:11:25 crc kubenswrapper[5052]: I1203 12:11:25.686711 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f20d775c-c6a5-4810-bf71-9ce90bd5e8c3" path="/var/lib/kubelet/pods/f20d775c-c6a5-4810-bf71-9ce90bd5e8c3/volumes" Dec 03 12:11:25 crc kubenswrapper[5052]: W1203 12:11:25.693729 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod03919b22_bf40_4bf2_88cb_fdefc426072c.slice/crio-656d7c09c157877e0d721311854348a66ad7554bb1d2e175c1191e79657ee3ba WatchSource:0}: Error finding container 656d7c09c157877e0d721311854348a66ad7554bb1d2e175c1191e79657ee3ba: Status 404 returned error can't find the container with id 656d7c09c157877e0d721311854348a66ad7554bb1d2e175c1191e79657ee3ba Dec 03 12:11:26 crc kubenswrapper[5052]: I1203 12:11:26.163431 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"0fd9a636-d4a2-45f7-8a00-0b3a2fe364cb","Type":"ContainerStarted","Data":"7fe5008dd6a2fa7e7e752c99afa8f2187cc6b0accf3b245645b7274fe1ca2d1e"} Dec 03 12:11:26 crc kubenswrapper[5052]: I1203 12:11:26.166018 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"01e1d9d4-9390-460c-86dc-f2cfb5148209","Type":"ContainerStarted","Data":"2be1b9e05da92f72260c1e5b030336d162110a18704cc792ed3143ab5e749128"} Dec 03 12:11:26 crc kubenswrapper[5052]: I1203 12:11:26.168666 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"03919b22-bf40-4bf2-88cb-fdefc426072c","Type":"ContainerStarted","Data":"656d7c09c157877e0d721311854348a66ad7554bb1d2e175c1191e79657ee3ba"} Dec 03 12:11:27 crc kubenswrapper[5052]: I1203 12:11:27.178905 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"03919b22-bf40-4bf2-88cb-fdefc426072c","Type":"ContainerStarted","Data":"622d66b2983e1d4bdc07c013e8cf0e59732996d900fcb033de628153df313bb1"} Dec 03 12:11:27 crc kubenswrapper[5052]: I1203 12:11:27.181020 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"0fd9a636-d4a2-45f7-8a00-0b3a2fe364cb","Type":"ContainerStarted","Data":"0c0725eafb777c0b6d0ad246f980467eb5fda2912f89f7a5b4867df0ec771fce"} Dec 03 12:11:28 crc kubenswrapper[5052]: I1203 12:11:28.190292 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"03919b22-bf40-4bf2-88cb-fdefc426072c","Type":"ContainerStarted","Data":"73d8869b154d3281892503903ffd7422325318d2bf903924c2df74fb075f4f4e"} Dec 03 12:11:28 crc kubenswrapper[5052]: I1203 12:11:28.193570 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"0fd9a636-d4a2-45f7-8a00-0b3a2fe364cb","Type":"ContainerStarted","Data":"3980c92ec54354398e968f8f5cd2dd8d079b22bf5a7562059de4e49cc45e8c2b"} Dec 03 12:11:28 crc kubenswrapper[5052]: I1203 12:11:28.214538 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/glance-default-internal-api-0" podStartSLOduration=4.21449734 podStartE2EDuration="4.21449734s" podCreationTimestamp="2025-12-03 12:11:24 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:11:28.206564803 +0000 UTC m=+3078.794806577" watchObservedRunningTime="2025-12-03 12:11:28.21449734 +0000 UTC m=+3078.802739104" Dec 03 12:11:28 crc kubenswrapper[5052]: I1203 12:11:28.230831 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/glance-default-external-api-0" podStartSLOduration=4.230812276 podStartE2EDuration="4.230812276s" podCreationTimestamp="2025-12-03 12:11:24 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:11:28.226828222 +0000 UTC m=+3078.815069986" watchObservedRunningTime="2025-12-03 12:11:28.230812276 +0000 UTC m=+3078.819054040" Dec 03 12:11:31 crc kubenswrapper[5052]: I1203 12:11:31.225151 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"01e1d9d4-9390-460c-86dc-f2cfb5148209","Type":"ContainerStarted","Data":"1f8175c2e5033eaafc0c27ef7c289b6ddb8db3ce9413544584cd5ec561f92fcf"} Dec 03 12:11:31 crc kubenswrapper[5052]: I1203 12:11:31.225498 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="01e1d9d4-9390-460c-86dc-f2cfb5148209" containerName="ceilometer-notification-agent" containerID="cri-o://a94f5461b868f74204d9ad2fb2ddbda37b0a047c5e160973481c33c23045e8cf" gracePeriod=30 Dec 03 12:11:31 crc kubenswrapper[5052]: I1203 12:11:31.225372 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="01e1d9d4-9390-460c-86dc-f2cfb5148209" containerName="ceilometer-central-agent" containerID="cri-o://8b6afaa16b9f60cf12229f4f2a0fc34d5ea0673d3b569cbbe1a3dc7b36f0b8a6" gracePeriod=30 Dec 03 12:11:31 crc kubenswrapper[5052]: I1203 12:11:31.225437 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="01e1d9d4-9390-460c-86dc-f2cfb5148209" containerName="sg-core" containerID="cri-o://2be1b9e05da92f72260c1e5b030336d162110a18704cc792ed3143ab5e749128" gracePeriod=30 Dec 03 12:11:31 crc kubenswrapper[5052]: I1203 12:11:31.225456 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="01e1d9d4-9390-460c-86dc-f2cfb5148209" containerName="proxy-httpd" containerID="cri-o://1f8175c2e5033eaafc0c27ef7c289b6ddb8db3ce9413544584cd5ec561f92fcf" gracePeriod=30 Dec 03 12:11:31 crc kubenswrapper[5052]: I1203 12:11:31.225721 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:11:31 crc kubenswrapper[5052]: I1203 12:11:31.276088 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/ceilometer-0" podStartSLOduration=3.386305599 podStartE2EDuration="18.276059236s" podCreationTimestamp="2025-12-03 12:11:13 +0000 UTC" firstStartedPulling="2025-12-03 12:11:14.961121907 +0000 UTC m=+3065.549363671" lastFinishedPulling="2025-12-03 12:11:29.850875544 +0000 UTC m=+3080.439117308" observedRunningTime="2025-12-03 12:11:31.2558624 +0000 UTC m=+3081.844104164" watchObservedRunningTime="2025-12-03 12:11:31.276059236 +0000 UTC m=+3081.864301000" Dec 03 12:11:32 crc kubenswrapper[5052]: I1203 12:11:32.242608 5052 generic.go:334] "Generic (PLEG): container finished" podID="01e1d9d4-9390-460c-86dc-f2cfb5148209" containerID="1f8175c2e5033eaafc0c27ef7c289b6ddb8db3ce9413544584cd5ec561f92fcf" exitCode=0 Dec 03 12:11:32 crc kubenswrapper[5052]: I1203 12:11:32.242651 5052 generic.go:334] "Generic (PLEG): container finished" podID="01e1d9d4-9390-460c-86dc-f2cfb5148209" containerID="2be1b9e05da92f72260c1e5b030336d162110a18704cc792ed3143ab5e749128" exitCode=2 Dec 03 12:11:32 crc kubenswrapper[5052]: I1203 12:11:32.242666 5052 generic.go:334] "Generic (PLEG): container finished" podID="01e1d9d4-9390-460c-86dc-f2cfb5148209" containerID="a94f5461b868f74204d9ad2fb2ddbda37b0a047c5e160973481c33c23045e8cf" exitCode=0 Dec 03 12:11:32 crc kubenswrapper[5052]: I1203 12:11:32.242677 5052 generic.go:334] "Generic (PLEG): container finished" podID="01e1d9d4-9390-460c-86dc-f2cfb5148209" containerID="8b6afaa16b9f60cf12229f4f2a0fc34d5ea0673d3b569cbbe1a3dc7b36f0b8a6" exitCode=0 Dec 03 12:11:32 crc kubenswrapper[5052]: I1203 12:11:32.242698 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"01e1d9d4-9390-460c-86dc-f2cfb5148209","Type":"ContainerDied","Data":"1f8175c2e5033eaafc0c27ef7c289b6ddb8db3ce9413544584cd5ec561f92fcf"} Dec 03 12:11:32 crc kubenswrapper[5052]: I1203 12:11:32.242763 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"01e1d9d4-9390-460c-86dc-f2cfb5148209","Type":"ContainerDied","Data":"2be1b9e05da92f72260c1e5b030336d162110a18704cc792ed3143ab5e749128"} Dec 03 12:11:32 crc kubenswrapper[5052]: I1203 12:11:32.242778 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"01e1d9d4-9390-460c-86dc-f2cfb5148209","Type":"ContainerDied","Data":"a94f5461b868f74204d9ad2fb2ddbda37b0a047c5e160973481c33c23045e8cf"} Dec 03 12:11:32 crc kubenswrapper[5052]: I1203 12:11:32.242791 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"01e1d9d4-9390-460c-86dc-f2cfb5148209","Type":"ContainerDied","Data":"8b6afaa16b9f60cf12229f4f2a0fc34d5ea0673d3b569cbbe1a3dc7b36f0b8a6"} Dec 03 12:11:32 crc kubenswrapper[5052]: I1203 12:11:32.612632 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:11:32 crc kubenswrapper[5052]: I1203 12:11:32.706239 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/01e1d9d4-9390-460c-86dc-f2cfb5148209-log-httpd\") pod \"01e1d9d4-9390-460c-86dc-f2cfb5148209\" (UID: \"01e1d9d4-9390-460c-86dc-f2cfb5148209\") " Dec 03 12:11:32 crc kubenswrapper[5052]: I1203 12:11:32.706281 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/01e1d9d4-9390-460c-86dc-f2cfb5148209-config-data\") pod \"01e1d9d4-9390-460c-86dc-f2cfb5148209\" (UID: \"01e1d9d4-9390-460c-86dc-f2cfb5148209\") " Dec 03 12:11:32 crc kubenswrapper[5052]: I1203 12:11:32.706391 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/01e1d9d4-9390-460c-86dc-f2cfb5148209-combined-ca-bundle\") pod \"01e1d9d4-9390-460c-86dc-f2cfb5148209\" (UID: \"01e1d9d4-9390-460c-86dc-f2cfb5148209\") " Dec 03 12:11:32 crc kubenswrapper[5052]: I1203 12:11:32.706444 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/01e1d9d4-9390-460c-86dc-f2cfb5148209-scripts\") pod \"01e1d9d4-9390-460c-86dc-f2cfb5148209\" (UID: \"01e1d9d4-9390-460c-86dc-f2cfb5148209\") " Dec 03 12:11:32 crc kubenswrapper[5052]: I1203 12:11:32.706468 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/01e1d9d4-9390-460c-86dc-f2cfb5148209-sg-core-conf-yaml\") pod \"01e1d9d4-9390-460c-86dc-f2cfb5148209\" (UID: \"01e1d9d4-9390-460c-86dc-f2cfb5148209\") " Dec 03 12:11:32 crc kubenswrapper[5052]: I1203 12:11:32.706543 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/01e1d9d4-9390-460c-86dc-f2cfb5148209-run-httpd\") pod \"01e1d9d4-9390-460c-86dc-f2cfb5148209\" (UID: \"01e1d9d4-9390-460c-86dc-f2cfb5148209\") " Dec 03 12:11:32 crc kubenswrapper[5052]: I1203 12:11:32.706655 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b2qlw\" (UniqueName: \"kubernetes.io/projected/01e1d9d4-9390-460c-86dc-f2cfb5148209-kube-api-access-b2qlw\") pod \"01e1d9d4-9390-460c-86dc-f2cfb5148209\" (UID: \"01e1d9d4-9390-460c-86dc-f2cfb5148209\") " Dec 03 12:11:32 crc kubenswrapper[5052]: I1203 12:11:32.707058 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/01e1d9d4-9390-460c-86dc-f2cfb5148209-log-httpd" (OuterVolumeSpecName: "log-httpd") pod "01e1d9d4-9390-460c-86dc-f2cfb5148209" (UID: "01e1d9d4-9390-460c-86dc-f2cfb5148209"). InnerVolumeSpecName "log-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:11:32 crc kubenswrapper[5052]: I1203 12:11:32.707617 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/01e1d9d4-9390-460c-86dc-f2cfb5148209-run-httpd" (OuterVolumeSpecName: "run-httpd") pod "01e1d9d4-9390-460c-86dc-f2cfb5148209" (UID: "01e1d9d4-9390-460c-86dc-f2cfb5148209"). InnerVolumeSpecName "run-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:11:32 crc kubenswrapper[5052]: I1203 12:11:32.713187 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/01e1d9d4-9390-460c-86dc-f2cfb5148209-scripts" (OuterVolumeSpecName: "scripts") pod "01e1d9d4-9390-460c-86dc-f2cfb5148209" (UID: "01e1d9d4-9390-460c-86dc-f2cfb5148209"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:11:32 crc kubenswrapper[5052]: I1203 12:11:32.713266 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/01e1d9d4-9390-460c-86dc-f2cfb5148209-kube-api-access-b2qlw" (OuterVolumeSpecName: "kube-api-access-b2qlw") pod "01e1d9d4-9390-460c-86dc-f2cfb5148209" (UID: "01e1d9d4-9390-460c-86dc-f2cfb5148209"). InnerVolumeSpecName "kube-api-access-b2qlw". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:11:32 crc kubenswrapper[5052]: I1203 12:11:32.736961 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/01e1d9d4-9390-460c-86dc-f2cfb5148209-sg-core-conf-yaml" (OuterVolumeSpecName: "sg-core-conf-yaml") pod "01e1d9d4-9390-460c-86dc-f2cfb5148209" (UID: "01e1d9d4-9390-460c-86dc-f2cfb5148209"). InnerVolumeSpecName "sg-core-conf-yaml". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:11:32 crc kubenswrapper[5052]: I1203 12:11:32.776089 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/01e1d9d4-9390-460c-86dc-f2cfb5148209-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "01e1d9d4-9390-460c-86dc-f2cfb5148209" (UID: "01e1d9d4-9390-460c-86dc-f2cfb5148209"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:11:32 crc kubenswrapper[5052]: I1203 12:11:32.807265 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/01e1d9d4-9390-460c-86dc-f2cfb5148209-config-data" (OuterVolumeSpecName: "config-data") pod "01e1d9d4-9390-460c-86dc-f2cfb5148209" (UID: "01e1d9d4-9390-460c-86dc-f2cfb5148209"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:11:32 crc kubenswrapper[5052]: I1203 12:11:32.808572 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/01e1d9d4-9390-460c-86dc-f2cfb5148209-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:11:32 crc kubenswrapper[5052]: I1203 12:11:32.808605 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/01e1d9d4-9390-460c-86dc-f2cfb5148209-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:11:32 crc kubenswrapper[5052]: I1203 12:11:32.808615 5052 reconciler_common.go:293] "Volume detached for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/01e1d9d4-9390-460c-86dc-f2cfb5148209-sg-core-conf-yaml\") on node \"crc\" DevicePath \"\"" Dec 03 12:11:32 crc kubenswrapper[5052]: I1203 12:11:32.808623 5052 reconciler_common.go:293] "Volume detached for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/01e1d9d4-9390-460c-86dc-f2cfb5148209-run-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:11:32 crc kubenswrapper[5052]: I1203 12:11:32.808637 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-b2qlw\" (UniqueName: \"kubernetes.io/projected/01e1d9d4-9390-460c-86dc-f2cfb5148209-kube-api-access-b2qlw\") on node \"crc\" DevicePath \"\"" Dec 03 12:11:32 crc kubenswrapper[5052]: I1203 12:11:32.808657 5052 reconciler_common.go:293] "Volume detached for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/01e1d9d4-9390-460c-86dc-f2cfb5148209-log-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:11:32 crc kubenswrapper[5052]: I1203 12:11:32.808668 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/01e1d9d4-9390-460c-86dc-f2cfb5148209-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:11:33 crc kubenswrapper[5052]: I1203 12:11:33.254563 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"01e1d9d4-9390-460c-86dc-f2cfb5148209","Type":"ContainerDied","Data":"f017781293cbe1892ae07ddac86f479884a15497b33aeb870e92a2e294dcf7e9"} Dec 03 12:11:33 crc kubenswrapper[5052]: I1203 12:11:33.254976 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:11:33 crc kubenswrapper[5052]: I1203 12:11:33.254986 5052 scope.go:117] "RemoveContainer" containerID="1f8175c2e5033eaafc0c27ef7c289b6ddb8db3ce9413544584cd5ec561f92fcf" Dec 03 12:11:33 crc kubenswrapper[5052]: I1203 12:11:33.284667 5052 scope.go:117] "RemoveContainer" containerID="2be1b9e05da92f72260c1e5b030336d162110a18704cc792ed3143ab5e749128" Dec 03 12:11:33 crc kubenswrapper[5052]: I1203 12:11:33.291405 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:11:33 crc kubenswrapper[5052]: I1203 12:11:33.301749 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:11:33 crc kubenswrapper[5052]: I1203 12:11:33.316833 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:11:33 crc kubenswrapper[5052]: E1203 12:11:33.317278 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="01e1d9d4-9390-460c-86dc-f2cfb5148209" containerName="sg-core" Dec 03 12:11:33 crc kubenswrapper[5052]: I1203 12:11:33.317300 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="01e1d9d4-9390-460c-86dc-f2cfb5148209" containerName="sg-core" Dec 03 12:11:33 crc kubenswrapper[5052]: E1203 12:11:33.317330 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="01e1d9d4-9390-460c-86dc-f2cfb5148209" containerName="ceilometer-central-agent" Dec 03 12:11:33 crc kubenswrapper[5052]: I1203 12:11:33.317339 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="01e1d9d4-9390-460c-86dc-f2cfb5148209" containerName="ceilometer-central-agent" Dec 03 12:11:33 crc kubenswrapper[5052]: E1203 12:11:33.317353 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="01e1d9d4-9390-460c-86dc-f2cfb5148209" containerName="proxy-httpd" Dec 03 12:11:33 crc kubenswrapper[5052]: I1203 12:11:33.317360 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="01e1d9d4-9390-460c-86dc-f2cfb5148209" containerName="proxy-httpd" Dec 03 12:11:33 crc kubenswrapper[5052]: E1203 12:11:33.317381 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="01e1d9d4-9390-460c-86dc-f2cfb5148209" containerName="ceilometer-notification-agent" Dec 03 12:11:33 crc kubenswrapper[5052]: I1203 12:11:33.317388 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="01e1d9d4-9390-460c-86dc-f2cfb5148209" containerName="ceilometer-notification-agent" Dec 03 12:11:33 crc kubenswrapper[5052]: I1203 12:11:33.317622 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="01e1d9d4-9390-460c-86dc-f2cfb5148209" containerName="sg-core" Dec 03 12:11:33 crc kubenswrapper[5052]: I1203 12:11:33.317642 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="01e1d9d4-9390-460c-86dc-f2cfb5148209" containerName="ceilometer-notification-agent" Dec 03 12:11:33 crc kubenswrapper[5052]: I1203 12:11:33.317656 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="01e1d9d4-9390-460c-86dc-f2cfb5148209" containerName="proxy-httpd" Dec 03 12:11:33 crc kubenswrapper[5052]: I1203 12:11:33.317669 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="01e1d9d4-9390-460c-86dc-f2cfb5148209" containerName="ceilometer-central-agent" Dec 03 12:11:33 crc kubenswrapper[5052]: I1203 12:11:33.319609 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:11:33 crc kubenswrapper[5052]: I1203 12:11:33.322966 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-scripts" Dec 03 12:11:33 crc kubenswrapper[5052]: I1203 12:11:33.323143 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-config-data" Dec 03 12:11:33 crc kubenswrapper[5052]: I1203 12:11:33.333667 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:11:33 crc kubenswrapper[5052]: I1203 12:11:33.354619 5052 scope.go:117] "RemoveContainer" containerID="a94f5461b868f74204d9ad2fb2ddbda37b0a047c5e160973481c33c23045e8cf" Dec 03 12:11:33 crc kubenswrapper[5052]: I1203 12:11:33.387709 5052 scope.go:117] "RemoveContainer" containerID="8b6afaa16b9f60cf12229f4f2a0fc34d5ea0673d3b569cbbe1a3dc7b36f0b8a6" Dec 03 12:11:33 crc kubenswrapper[5052]: I1203 12:11:33.424166 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p8vz6\" (UniqueName: \"kubernetes.io/projected/18240d78-5173-4d34-a1b7-8d51cac46b74-kube-api-access-p8vz6\") pod \"ceilometer-0\" (UID: \"18240d78-5173-4d34-a1b7-8d51cac46b74\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:11:33 crc kubenswrapper[5052]: I1203 12:11:33.424284 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/18240d78-5173-4d34-a1b7-8d51cac46b74-log-httpd\") pod \"ceilometer-0\" (UID: \"18240d78-5173-4d34-a1b7-8d51cac46b74\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:11:33 crc kubenswrapper[5052]: I1203 12:11:33.424448 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/18240d78-5173-4d34-a1b7-8d51cac46b74-run-httpd\") pod \"ceilometer-0\" (UID: \"18240d78-5173-4d34-a1b7-8d51cac46b74\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:11:33 crc kubenswrapper[5052]: I1203 12:11:33.424550 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/18240d78-5173-4d34-a1b7-8d51cac46b74-scripts\") pod \"ceilometer-0\" (UID: \"18240d78-5173-4d34-a1b7-8d51cac46b74\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:11:33 crc kubenswrapper[5052]: I1203 12:11:33.424679 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/18240d78-5173-4d34-a1b7-8d51cac46b74-config-data\") pod \"ceilometer-0\" (UID: \"18240d78-5173-4d34-a1b7-8d51cac46b74\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:11:33 crc kubenswrapper[5052]: I1203 12:11:33.424710 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/18240d78-5173-4d34-a1b7-8d51cac46b74-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"18240d78-5173-4d34-a1b7-8d51cac46b74\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:11:33 crc kubenswrapper[5052]: I1203 12:11:33.424747 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/18240d78-5173-4d34-a1b7-8d51cac46b74-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"18240d78-5173-4d34-a1b7-8d51cac46b74\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:11:33 crc kubenswrapper[5052]: I1203 12:11:33.526541 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/18240d78-5173-4d34-a1b7-8d51cac46b74-run-httpd\") pod \"ceilometer-0\" (UID: \"18240d78-5173-4d34-a1b7-8d51cac46b74\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:11:33 crc kubenswrapper[5052]: I1203 12:11:33.526630 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/18240d78-5173-4d34-a1b7-8d51cac46b74-scripts\") pod \"ceilometer-0\" (UID: \"18240d78-5173-4d34-a1b7-8d51cac46b74\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:11:33 crc kubenswrapper[5052]: I1203 12:11:33.526700 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/18240d78-5173-4d34-a1b7-8d51cac46b74-config-data\") pod \"ceilometer-0\" (UID: \"18240d78-5173-4d34-a1b7-8d51cac46b74\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:11:33 crc kubenswrapper[5052]: I1203 12:11:33.526736 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/18240d78-5173-4d34-a1b7-8d51cac46b74-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"18240d78-5173-4d34-a1b7-8d51cac46b74\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:11:33 crc kubenswrapper[5052]: I1203 12:11:33.526766 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/18240d78-5173-4d34-a1b7-8d51cac46b74-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"18240d78-5173-4d34-a1b7-8d51cac46b74\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:11:33 crc kubenswrapper[5052]: I1203 12:11:33.526813 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-p8vz6\" (UniqueName: \"kubernetes.io/projected/18240d78-5173-4d34-a1b7-8d51cac46b74-kube-api-access-p8vz6\") pod \"ceilometer-0\" (UID: \"18240d78-5173-4d34-a1b7-8d51cac46b74\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:11:33 crc kubenswrapper[5052]: I1203 12:11:33.526837 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/18240d78-5173-4d34-a1b7-8d51cac46b74-log-httpd\") pod \"ceilometer-0\" (UID: \"18240d78-5173-4d34-a1b7-8d51cac46b74\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:11:33 crc kubenswrapper[5052]: I1203 12:11:33.527473 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/18240d78-5173-4d34-a1b7-8d51cac46b74-log-httpd\") pod \"ceilometer-0\" (UID: \"18240d78-5173-4d34-a1b7-8d51cac46b74\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:11:33 crc kubenswrapper[5052]: I1203 12:11:33.527485 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/18240d78-5173-4d34-a1b7-8d51cac46b74-run-httpd\") pod \"ceilometer-0\" (UID: \"18240d78-5173-4d34-a1b7-8d51cac46b74\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:11:33 crc kubenswrapper[5052]: I1203 12:11:33.535649 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/18240d78-5173-4d34-a1b7-8d51cac46b74-scripts\") pod \"ceilometer-0\" (UID: \"18240d78-5173-4d34-a1b7-8d51cac46b74\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:11:33 crc kubenswrapper[5052]: I1203 12:11:33.539883 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/18240d78-5173-4d34-a1b7-8d51cac46b74-config-data\") pod \"ceilometer-0\" (UID: \"18240d78-5173-4d34-a1b7-8d51cac46b74\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:11:33 crc kubenswrapper[5052]: I1203 12:11:33.545551 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/18240d78-5173-4d34-a1b7-8d51cac46b74-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"18240d78-5173-4d34-a1b7-8d51cac46b74\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:11:33 crc kubenswrapper[5052]: I1203 12:11:33.546045 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/18240d78-5173-4d34-a1b7-8d51cac46b74-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"18240d78-5173-4d34-a1b7-8d51cac46b74\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:11:33 crc kubenswrapper[5052]: I1203 12:11:33.555313 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-p8vz6\" (UniqueName: \"kubernetes.io/projected/18240d78-5173-4d34-a1b7-8d51cac46b74-kube-api-access-p8vz6\") pod \"ceilometer-0\" (UID: \"18240d78-5173-4d34-a1b7-8d51cac46b74\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:11:33 crc kubenswrapper[5052]: I1203 12:11:33.668183 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:11:33 crc kubenswrapper[5052]: I1203 12:11:33.683983 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="01e1d9d4-9390-460c-86dc-f2cfb5148209" path="/var/lib/kubelet/pods/01e1d9d4-9390-460c-86dc-f2cfb5148209/volumes" Dec 03 12:11:34 crc kubenswrapper[5052]: I1203 12:11:34.216618 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:11:34 crc kubenswrapper[5052]: W1203 12:11:34.219385 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod18240d78_5173_4d34_a1b7_8d51cac46b74.slice/crio-98bd94408261570ebaaa7927e396c3a599bf8561c103a382034a1e90fca91ee4 WatchSource:0}: Error finding container 98bd94408261570ebaaa7927e396c3a599bf8561c103a382034a1e90fca91ee4: Status 404 returned error can't find the container with id 98bd94408261570ebaaa7927e396c3a599bf8561c103a382034a1e90fca91ee4 Dec 03 12:11:34 crc kubenswrapper[5052]: I1203 12:11:34.275366 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"18240d78-5173-4d34-a1b7-8d51cac46b74","Type":"ContainerStarted","Data":"98bd94408261570ebaaa7927e396c3a599bf8561c103a382034a1e90fca91ee4"} Dec 03 12:11:34 crc kubenswrapper[5052]: I1203 12:11:34.591941 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:11:34 crc kubenswrapper[5052]: I1203 12:11:34.592647 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:11:34 crc kubenswrapper[5052]: I1203 12:11:34.600815 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:11:34 crc kubenswrapper[5052]: I1203 12:11:34.601296 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:11:34 crc kubenswrapper[5052]: I1203 12:11:34.627612 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:11:34 crc kubenswrapper[5052]: I1203 12:11:34.633557 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:11:34 crc kubenswrapper[5052]: I1203 12:11:34.642818 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:11:34 crc kubenswrapper[5052]: I1203 12:11:34.644717 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:11:35 crc kubenswrapper[5052]: I1203 12:11:35.285721 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"18240d78-5173-4d34-a1b7-8d51cac46b74","Type":"ContainerStarted","Data":"1a2115a0a85bfc97ed65c66337befc7d3164462079d6fd960da7e80b6dd2ac30"} Dec 03 12:11:35 crc kubenswrapper[5052]: I1203 12:11:35.286771 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:11:35 crc kubenswrapper[5052]: I1203 12:11:35.286809 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:11:35 crc kubenswrapper[5052]: I1203 12:11:35.286822 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:11:35 crc kubenswrapper[5052]: I1203 12:11:35.286831 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:11:36 crc kubenswrapper[5052]: I1203 12:11:36.297045 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"18240d78-5173-4d34-a1b7-8d51cac46b74","Type":"ContainerStarted","Data":"600afe54035e6ba3cfd43d7a781ebae8aa663487d01caeaadb229c09d9865654"} Dec 03 12:11:37 crc kubenswrapper[5052]: I1203 12:11:37.312979 5052 generic.go:334] "Generic (PLEG): container finished" podID="581cd00d-4ede-4150-a4ea-aeac7033f0ab" containerID="b2b84bb90c0815c25cd39fbb85a799acfc5bc3d5a578b8bf8f3a88a0db65844c" exitCode=0 Dec 03 12:11:37 crc kubenswrapper[5052]: I1203 12:11:37.313349 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-bootstrap-zz8h5" event={"ID":"581cd00d-4ede-4150-a4ea-aeac7033f0ab","Type":"ContainerDied","Data":"b2b84bb90c0815c25cd39fbb85a799acfc5bc3d5a578b8bf8f3a88a0db65844c"} Dec 03 12:11:37 crc kubenswrapper[5052]: I1203 12:11:37.317242 5052 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Dec 03 12:11:37 crc kubenswrapper[5052]: I1203 12:11:37.317271 5052 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Dec 03 12:11:37 crc kubenswrapper[5052]: I1203 12:11:37.318280 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"18240d78-5173-4d34-a1b7-8d51cac46b74","Type":"ContainerStarted","Data":"6818f70d48c9182195a247515369fd4e561d427183eeca88cc603aba0d5819c7"} Dec 03 12:11:37 crc kubenswrapper[5052]: I1203 12:11:37.555742 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:11:37 crc kubenswrapper[5052]: I1203 12:11:37.556934 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:11:37 crc kubenswrapper[5052]: I1203 12:11:37.606359 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:11:37 crc kubenswrapper[5052]: I1203 12:11:37.606492 5052 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Dec 03 12:11:37 crc kubenswrapper[5052]: I1203 12:11:37.659589 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:11:37 crc kubenswrapper[5052]: I1203 12:11:37.673350 5052 scope.go:117] "RemoveContainer" containerID="63298107b3d090deff9ed92f3d613e5b1c86d9137684d117500257db8ee1c805" Dec 03 12:11:37 crc kubenswrapper[5052]: E1203 12:11:37.673611 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 12:11:38 crc kubenswrapper[5052]: I1203 12:11:38.763535 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-bootstrap-zz8h5" Dec 03 12:11:38 crc kubenswrapper[5052]: I1203 12:11:38.860719 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x6sfm\" (UniqueName: \"kubernetes.io/projected/581cd00d-4ede-4150-a4ea-aeac7033f0ab-kube-api-access-x6sfm\") pod \"581cd00d-4ede-4150-a4ea-aeac7033f0ab\" (UID: \"581cd00d-4ede-4150-a4ea-aeac7033f0ab\") " Dec 03 12:11:38 crc kubenswrapper[5052]: I1203 12:11:38.860802 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/581cd00d-4ede-4150-a4ea-aeac7033f0ab-combined-ca-bundle\") pod \"581cd00d-4ede-4150-a4ea-aeac7033f0ab\" (UID: \"581cd00d-4ede-4150-a4ea-aeac7033f0ab\") " Dec 03 12:11:38 crc kubenswrapper[5052]: I1203 12:11:38.860989 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/581cd00d-4ede-4150-a4ea-aeac7033f0ab-credential-keys\") pod \"581cd00d-4ede-4150-a4ea-aeac7033f0ab\" (UID: \"581cd00d-4ede-4150-a4ea-aeac7033f0ab\") " Dec 03 12:11:38 crc kubenswrapper[5052]: I1203 12:11:38.861058 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/581cd00d-4ede-4150-a4ea-aeac7033f0ab-fernet-keys\") pod \"581cd00d-4ede-4150-a4ea-aeac7033f0ab\" (UID: \"581cd00d-4ede-4150-a4ea-aeac7033f0ab\") " Dec 03 12:11:38 crc kubenswrapper[5052]: I1203 12:11:38.861200 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/581cd00d-4ede-4150-a4ea-aeac7033f0ab-config-data\") pod \"581cd00d-4ede-4150-a4ea-aeac7033f0ab\" (UID: \"581cd00d-4ede-4150-a4ea-aeac7033f0ab\") " Dec 03 12:11:38 crc kubenswrapper[5052]: I1203 12:11:38.861259 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/581cd00d-4ede-4150-a4ea-aeac7033f0ab-scripts\") pod \"581cd00d-4ede-4150-a4ea-aeac7033f0ab\" (UID: \"581cd00d-4ede-4150-a4ea-aeac7033f0ab\") " Dec 03 12:11:38 crc kubenswrapper[5052]: I1203 12:11:38.884098 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/581cd00d-4ede-4150-a4ea-aeac7033f0ab-fernet-keys" (OuterVolumeSpecName: "fernet-keys") pod "581cd00d-4ede-4150-a4ea-aeac7033f0ab" (UID: "581cd00d-4ede-4150-a4ea-aeac7033f0ab"). InnerVolumeSpecName "fernet-keys". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:11:38 crc kubenswrapper[5052]: I1203 12:11:38.884109 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/581cd00d-4ede-4150-a4ea-aeac7033f0ab-kube-api-access-x6sfm" (OuterVolumeSpecName: "kube-api-access-x6sfm") pod "581cd00d-4ede-4150-a4ea-aeac7033f0ab" (UID: "581cd00d-4ede-4150-a4ea-aeac7033f0ab"). InnerVolumeSpecName "kube-api-access-x6sfm". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:11:38 crc kubenswrapper[5052]: I1203 12:11:38.884330 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/581cd00d-4ede-4150-a4ea-aeac7033f0ab-scripts" (OuterVolumeSpecName: "scripts") pod "581cd00d-4ede-4150-a4ea-aeac7033f0ab" (UID: "581cd00d-4ede-4150-a4ea-aeac7033f0ab"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:11:38 crc kubenswrapper[5052]: I1203 12:11:38.902762 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/581cd00d-4ede-4150-a4ea-aeac7033f0ab-config-data" (OuterVolumeSpecName: "config-data") pod "581cd00d-4ede-4150-a4ea-aeac7033f0ab" (UID: "581cd00d-4ede-4150-a4ea-aeac7033f0ab"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:11:38 crc kubenswrapper[5052]: I1203 12:11:38.902906 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/581cd00d-4ede-4150-a4ea-aeac7033f0ab-credential-keys" (OuterVolumeSpecName: "credential-keys") pod "581cd00d-4ede-4150-a4ea-aeac7033f0ab" (UID: "581cd00d-4ede-4150-a4ea-aeac7033f0ab"). InnerVolumeSpecName "credential-keys". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:11:38 crc kubenswrapper[5052]: I1203 12:11:38.912576 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/581cd00d-4ede-4150-a4ea-aeac7033f0ab-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "581cd00d-4ede-4150-a4ea-aeac7033f0ab" (UID: "581cd00d-4ede-4150-a4ea-aeac7033f0ab"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:11:38 crc kubenswrapper[5052]: I1203 12:11:38.964924 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-x6sfm\" (UniqueName: \"kubernetes.io/projected/581cd00d-4ede-4150-a4ea-aeac7033f0ab-kube-api-access-x6sfm\") on node \"crc\" DevicePath \"\"" Dec 03 12:11:38 crc kubenswrapper[5052]: I1203 12:11:38.964984 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/581cd00d-4ede-4150-a4ea-aeac7033f0ab-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:11:38 crc kubenswrapper[5052]: I1203 12:11:38.964994 5052 reconciler_common.go:293] "Volume detached for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/581cd00d-4ede-4150-a4ea-aeac7033f0ab-credential-keys\") on node \"crc\" DevicePath \"\"" Dec 03 12:11:38 crc kubenswrapper[5052]: I1203 12:11:38.965004 5052 reconciler_common.go:293] "Volume detached for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/581cd00d-4ede-4150-a4ea-aeac7033f0ab-fernet-keys\") on node \"crc\" DevicePath \"\"" Dec 03 12:11:38 crc kubenswrapper[5052]: I1203 12:11:38.965015 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/581cd00d-4ede-4150-a4ea-aeac7033f0ab-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:11:38 crc kubenswrapper[5052]: I1203 12:11:38.965025 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/581cd00d-4ede-4150-a4ea-aeac7033f0ab-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:11:39 crc kubenswrapper[5052]: I1203 12:11:39.375306 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"18240d78-5173-4d34-a1b7-8d51cac46b74","Type":"ContainerStarted","Data":"63454317bd2f90c442d2189fe71d8ca20e75f95212e175181074a919d866aec2"} Dec 03 12:11:39 crc kubenswrapper[5052]: I1203 12:11:39.376059 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:11:39 crc kubenswrapper[5052]: I1203 12:11:39.397088 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-bootstrap-zz8h5" Dec 03 12:11:39 crc kubenswrapper[5052]: I1203 12:11:39.397792 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-bootstrap-zz8h5" event={"ID":"581cd00d-4ede-4150-a4ea-aeac7033f0ab","Type":"ContainerDied","Data":"a5dbb466d13627ccde65d5493b120aca766cf69207417cb7bd07b949a9b6abc7"} Dec 03 12:11:39 crc kubenswrapper[5052]: I1203 12:11:39.397845 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a5dbb466d13627ccde65d5493b120aca766cf69207417cb7bd07b949a9b6abc7" Dec 03 12:11:39 crc kubenswrapper[5052]: I1203 12:11:39.441814 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/ceilometer-0" podStartSLOduration=2.107823086 podStartE2EDuration="6.441793023s" podCreationTimestamp="2025-12-03 12:11:33 +0000 UTC" firstStartedPulling="2025-12-03 12:11:34.222422724 +0000 UTC m=+3084.810664488" lastFinishedPulling="2025-12-03 12:11:38.556392661 +0000 UTC m=+3089.144634425" observedRunningTime="2025-12-03 12:11:39.43889508 +0000 UTC m=+3090.027136844" watchObservedRunningTime="2025-12-03 12:11:39.441793023 +0000 UTC m=+3090.030034807" Dec 03 12:11:39 crc kubenswrapper[5052]: I1203 12:11:39.547828 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/keystone-bootstrap-zz8h5"] Dec 03 12:11:39 crc kubenswrapper[5052]: I1203 12:11:39.556506 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/keystone-bootstrap-zz8h5"] Dec 03 12:11:39 crc kubenswrapper[5052]: I1203 12:11:39.586111 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/keystone-bootstrap-v44q4"] Dec 03 12:11:39 crc kubenswrapper[5052]: E1203 12:11:39.586533 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="581cd00d-4ede-4150-a4ea-aeac7033f0ab" containerName="keystone-bootstrap" Dec 03 12:11:39 crc kubenswrapper[5052]: I1203 12:11:39.586548 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="581cd00d-4ede-4150-a4ea-aeac7033f0ab" containerName="keystone-bootstrap" Dec 03 12:11:39 crc kubenswrapper[5052]: I1203 12:11:39.586739 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="581cd00d-4ede-4150-a4ea-aeac7033f0ab" containerName="keystone-bootstrap" Dec 03 12:11:39 crc kubenswrapper[5052]: I1203 12:11:39.587414 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-bootstrap-v44q4" Dec 03 12:11:39 crc kubenswrapper[5052]: I1203 12:11:39.590677 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone" Dec 03 12:11:39 crc kubenswrapper[5052]: I1203 12:11:39.591159 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone-scripts" Dec 03 12:11:39 crc kubenswrapper[5052]: I1203 12:11:39.591460 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"osp-secret" Dec 03 12:11:39 crc kubenswrapper[5052]: I1203 12:11:39.591654 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone-keystone-dockercfg-4wch9" Dec 03 12:11:39 crc kubenswrapper[5052]: I1203 12:11:39.601811 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-bootstrap-v44q4"] Dec 03 12:11:39 crc kubenswrapper[5052]: I1203 12:11:39.607134 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone-config-data" Dec 03 12:11:39 crc kubenswrapper[5052]: I1203 12:11:39.683083 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4ef6dc74-a5f4-489c-8527-df175eb265ec-combined-ca-bundle\") pod \"keystone-bootstrap-v44q4\" (UID: \"4ef6dc74-a5f4-489c-8527-df175eb265ec\") " pod="openstack-kuttl-tests/keystone-bootstrap-v44q4" Dec 03 12:11:39 crc kubenswrapper[5052]: I1203 12:11:39.683575 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/4ef6dc74-a5f4-489c-8527-df175eb265ec-fernet-keys\") pod \"keystone-bootstrap-v44q4\" (UID: \"4ef6dc74-a5f4-489c-8527-df175eb265ec\") " pod="openstack-kuttl-tests/keystone-bootstrap-v44q4" Dec 03 12:11:39 crc kubenswrapper[5052]: I1203 12:11:39.683760 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/4ef6dc74-a5f4-489c-8527-df175eb265ec-scripts\") pod \"keystone-bootstrap-v44q4\" (UID: \"4ef6dc74-a5f4-489c-8527-df175eb265ec\") " pod="openstack-kuttl-tests/keystone-bootstrap-v44q4" Dec 03 12:11:39 crc kubenswrapper[5052]: I1203 12:11:39.683886 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/4ef6dc74-a5f4-489c-8527-df175eb265ec-credential-keys\") pod \"keystone-bootstrap-v44q4\" (UID: \"4ef6dc74-a5f4-489c-8527-df175eb265ec\") " pod="openstack-kuttl-tests/keystone-bootstrap-v44q4" Dec 03 12:11:39 crc kubenswrapper[5052]: I1203 12:11:39.683997 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/4ef6dc74-a5f4-489c-8527-df175eb265ec-config-data\") pod \"keystone-bootstrap-v44q4\" (UID: \"4ef6dc74-a5f4-489c-8527-df175eb265ec\") " pod="openstack-kuttl-tests/keystone-bootstrap-v44q4" Dec 03 12:11:39 crc kubenswrapper[5052]: I1203 12:11:39.684121 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kpbpj\" (UniqueName: \"kubernetes.io/projected/4ef6dc74-a5f4-489c-8527-df175eb265ec-kube-api-access-kpbpj\") pod \"keystone-bootstrap-v44q4\" (UID: \"4ef6dc74-a5f4-489c-8527-df175eb265ec\") " pod="openstack-kuttl-tests/keystone-bootstrap-v44q4" Dec 03 12:11:39 crc kubenswrapper[5052]: I1203 12:11:39.690321 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="581cd00d-4ede-4150-a4ea-aeac7033f0ab" path="/var/lib/kubelet/pods/581cd00d-4ede-4150-a4ea-aeac7033f0ab/volumes" Dec 03 12:11:39 crc kubenswrapper[5052]: I1203 12:11:39.785304 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/4ef6dc74-a5f4-489c-8527-df175eb265ec-fernet-keys\") pod \"keystone-bootstrap-v44q4\" (UID: \"4ef6dc74-a5f4-489c-8527-df175eb265ec\") " pod="openstack-kuttl-tests/keystone-bootstrap-v44q4" Dec 03 12:11:39 crc kubenswrapper[5052]: I1203 12:11:39.785393 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/4ef6dc74-a5f4-489c-8527-df175eb265ec-scripts\") pod \"keystone-bootstrap-v44q4\" (UID: \"4ef6dc74-a5f4-489c-8527-df175eb265ec\") " pod="openstack-kuttl-tests/keystone-bootstrap-v44q4" Dec 03 12:11:39 crc kubenswrapper[5052]: I1203 12:11:39.785436 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/4ef6dc74-a5f4-489c-8527-df175eb265ec-credential-keys\") pod \"keystone-bootstrap-v44q4\" (UID: \"4ef6dc74-a5f4-489c-8527-df175eb265ec\") " pod="openstack-kuttl-tests/keystone-bootstrap-v44q4" Dec 03 12:11:39 crc kubenswrapper[5052]: I1203 12:11:39.785463 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/4ef6dc74-a5f4-489c-8527-df175eb265ec-config-data\") pod \"keystone-bootstrap-v44q4\" (UID: \"4ef6dc74-a5f4-489c-8527-df175eb265ec\") " pod="openstack-kuttl-tests/keystone-bootstrap-v44q4" Dec 03 12:11:39 crc kubenswrapper[5052]: I1203 12:11:39.785498 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-kpbpj\" (UniqueName: \"kubernetes.io/projected/4ef6dc74-a5f4-489c-8527-df175eb265ec-kube-api-access-kpbpj\") pod \"keystone-bootstrap-v44q4\" (UID: \"4ef6dc74-a5f4-489c-8527-df175eb265ec\") " pod="openstack-kuttl-tests/keystone-bootstrap-v44q4" Dec 03 12:11:39 crc kubenswrapper[5052]: I1203 12:11:39.785550 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4ef6dc74-a5f4-489c-8527-df175eb265ec-combined-ca-bundle\") pod \"keystone-bootstrap-v44q4\" (UID: \"4ef6dc74-a5f4-489c-8527-df175eb265ec\") " pod="openstack-kuttl-tests/keystone-bootstrap-v44q4" Dec 03 12:11:39 crc kubenswrapper[5052]: I1203 12:11:39.791032 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/4ef6dc74-a5f4-489c-8527-df175eb265ec-scripts\") pod \"keystone-bootstrap-v44q4\" (UID: \"4ef6dc74-a5f4-489c-8527-df175eb265ec\") " pod="openstack-kuttl-tests/keystone-bootstrap-v44q4" Dec 03 12:11:39 crc kubenswrapper[5052]: I1203 12:11:39.791522 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/4ef6dc74-a5f4-489c-8527-df175eb265ec-config-data\") pod \"keystone-bootstrap-v44q4\" (UID: \"4ef6dc74-a5f4-489c-8527-df175eb265ec\") " pod="openstack-kuttl-tests/keystone-bootstrap-v44q4" Dec 03 12:11:39 crc kubenswrapper[5052]: I1203 12:11:39.791936 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/4ef6dc74-a5f4-489c-8527-df175eb265ec-credential-keys\") pod \"keystone-bootstrap-v44q4\" (UID: \"4ef6dc74-a5f4-489c-8527-df175eb265ec\") " pod="openstack-kuttl-tests/keystone-bootstrap-v44q4" Dec 03 12:11:39 crc kubenswrapper[5052]: I1203 12:11:39.792319 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/4ef6dc74-a5f4-489c-8527-df175eb265ec-fernet-keys\") pod \"keystone-bootstrap-v44q4\" (UID: \"4ef6dc74-a5f4-489c-8527-df175eb265ec\") " pod="openstack-kuttl-tests/keystone-bootstrap-v44q4" Dec 03 12:11:39 crc kubenswrapper[5052]: I1203 12:11:39.808662 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4ef6dc74-a5f4-489c-8527-df175eb265ec-combined-ca-bundle\") pod \"keystone-bootstrap-v44q4\" (UID: \"4ef6dc74-a5f4-489c-8527-df175eb265ec\") " pod="openstack-kuttl-tests/keystone-bootstrap-v44q4" Dec 03 12:11:39 crc kubenswrapper[5052]: I1203 12:11:39.811286 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-kpbpj\" (UniqueName: \"kubernetes.io/projected/4ef6dc74-a5f4-489c-8527-df175eb265ec-kube-api-access-kpbpj\") pod \"keystone-bootstrap-v44q4\" (UID: \"4ef6dc74-a5f4-489c-8527-df175eb265ec\") " pod="openstack-kuttl-tests/keystone-bootstrap-v44q4" Dec 03 12:11:39 crc kubenswrapper[5052]: I1203 12:11:39.905952 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-bootstrap-v44q4" Dec 03 12:11:40 crc kubenswrapper[5052]: I1203 12:11:40.408599 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-bootstrap-v44q4"] Dec 03 12:11:40 crc kubenswrapper[5052]: W1203 12:11:40.410467 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4ef6dc74_a5f4_489c_8527_df175eb265ec.slice/crio-86b82324d1ec982b43348a426986a60c4a121028f5d4b359f45f94d107757ddf WatchSource:0}: Error finding container 86b82324d1ec982b43348a426986a60c4a121028f5d4b359f45f94d107757ddf: Status 404 returned error can't find the container with id 86b82324d1ec982b43348a426986a60c4a121028f5d4b359f45f94d107757ddf Dec 03 12:11:41 crc kubenswrapper[5052]: I1203 12:11:41.418054 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-bootstrap-v44q4" event={"ID":"4ef6dc74-a5f4-489c-8527-df175eb265ec","Type":"ContainerStarted","Data":"f020c0314d37f1ebed5754d085ae61111bf03a1246b93ef4ff86bfaf736db918"} Dec 03 12:11:41 crc kubenswrapper[5052]: I1203 12:11:41.418407 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-bootstrap-v44q4" event={"ID":"4ef6dc74-a5f4-489c-8527-df175eb265ec","Type":"ContainerStarted","Data":"86b82324d1ec982b43348a426986a60c4a121028f5d4b359f45f94d107757ddf"} Dec 03 12:11:41 crc kubenswrapper[5052]: I1203 12:11:41.420290 5052 generic.go:334] "Generic (PLEG): container finished" podID="db210aa9-d72d-4114-945c-0aa978c27a18" containerID="425e1d453b59af4c319776f6782cfa58108a322f2cd520919a3ce1749cbf5b9a" exitCode=0 Dec 03 12:11:41 crc kubenswrapper[5052]: I1203 12:11:41.420321 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-db-sync-xr2pm" event={"ID":"db210aa9-d72d-4114-945c-0aa978c27a18","Type":"ContainerDied","Data":"425e1d453b59af4c319776f6782cfa58108a322f2cd520919a3ce1749cbf5b9a"} Dec 03 12:11:41 crc kubenswrapper[5052]: I1203 12:11:41.445943 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/keystone-bootstrap-v44q4" podStartSLOduration=2.445922517 podStartE2EDuration="2.445922517s" podCreationTimestamp="2025-12-03 12:11:39 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:11:41.437067184 +0000 UTC m=+3092.025308948" watchObservedRunningTime="2025-12-03 12:11:41.445922517 +0000 UTC m=+3092.034164281" Dec 03 12:11:42 crc kubenswrapper[5052]: I1203 12:11:42.832981 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-db-sync-xr2pm" Dec 03 12:11:42 crc kubenswrapper[5052]: I1203 12:11:42.988347 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/db210aa9-d72d-4114-945c-0aa978c27a18-logs\") pod \"db210aa9-d72d-4114-945c-0aa978c27a18\" (UID: \"db210aa9-d72d-4114-945c-0aa978c27a18\") " Dec 03 12:11:42 crc kubenswrapper[5052]: I1203 12:11:42.988432 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/db210aa9-d72d-4114-945c-0aa978c27a18-combined-ca-bundle\") pod \"db210aa9-d72d-4114-945c-0aa978c27a18\" (UID: \"db210aa9-d72d-4114-945c-0aa978c27a18\") " Dec 03 12:11:42 crc kubenswrapper[5052]: I1203 12:11:42.988576 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/db210aa9-d72d-4114-945c-0aa978c27a18-scripts\") pod \"db210aa9-d72d-4114-945c-0aa978c27a18\" (UID: \"db210aa9-d72d-4114-945c-0aa978c27a18\") " Dec 03 12:11:42 crc kubenswrapper[5052]: I1203 12:11:42.988673 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xq4kj\" (UniqueName: \"kubernetes.io/projected/db210aa9-d72d-4114-945c-0aa978c27a18-kube-api-access-xq4kj\") pod \"db210aa9-d72d-4114-945c-0aa978c27a18\" (UID: \"db210aa9-d72d-4114-945c-0aa978c27a18\") " Dec 03 12:11:42 crc kubenswrapper[5052]: I1203 12:11:42.988711 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/db210aa9-d72d-4114-945c-0aa978c27a18-config-data\") pod \"db210aa9-d72d-4114-945c-0aa978c27a18\" (UID: \"db210aa9-d72d-4114-945c-0aa978c27a18\") " Dec 03 12:11:42 crc kubenswrapper[5052]: I1203 12:11:42.989917 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db210aa9-d72d-4114-945c-0aa978c27a18-logs" (OuterVolumeSpecName: "logs") pod "db210aa9-d72d-4114-945c-0aa978c27a18" (UID: "db210aa9-d72d-4114-945c-0aa978c27a18"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:11:42 crc kubenswrapper[5052]: I1203 12:11:42.994286 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/db210aa9-d72d-4114-945c-0aa978c27a18-kube-api-access-xq4kj" (OuterVolumeSpecName: "kube-api-access-xq4kj") pod "db210aa9-d72d-4114-945c-0aa978c27a18" (UID: "db210aa9-d72d-4114-945c-0aa978c27a18"). InnerVolumeSpecName "kube-api-access-xq4kj". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:11:42 crc kubenswrapper[5052]: I1203 12:11:42.995215 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/db210aa9-d72d-4114-945c-0aa978c27a18-scripts" (OuterVolumeSpecName: "scripts") pod "db210aa9-d72d-4114-945c-0aa978c27a18" (UID: "db210aa9-d72d-4114-945c-0aa978c27a18"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:11:43 crc kubenswrapper[5052]: I1203 12:11:43.016565 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/db210aa9-d72d-4114-945c-0aa978c27a18-config-data" (OuterVolumeSpecName: "config-data") pod "db210aa9-d72d-4114-945c-0aa978c27a18" (UID: "db210aa9-d72d-4114-945c-0aa978c27a18"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:11:43 crc kubenswrapper[5052]: I1203 12:11:43.017143 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/db210aa9-d72d-4114-945c-0aa978c27a18-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "db210aa9-d72d-4114-945c-0aa978c27a18" (UID: "db210aa9-d72d-4114-945c-0aa978c27a18"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:11:43 crc kubenswrapper[5052]: I1203 12:11:43.090956 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/db210aa9-d72d-4114-945c-0aa978c27a18-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:11:43 crc kubenswrapper[5052]: I1203 12:11:43.091009 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/db210aa9-d72d-4114-945c-0aa978c27a18-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:11:43 crc kubenswrapper[5052]: I1203 12:11:43.091027 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/db210aa9-d72d-4114-945c-0aa978c27a18-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:11:43 crc kubenswrapper[5052]: I1203 12:11:43.091040 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/db210aa9-d72d-4114-945c-0aa978c27a18-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:11:43 crc kubenswrapper[5052]: I1203 12:11:43.091053 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-xq4kj\" (UniqueName: \"kubernetes.io/projected/db210aa9-d72d-4114-945c-0aa978c27a18-kube-api-access-xq4kj\") on node \"crc\" DevicePath \"\"" Dec 03 12:11:43 crc kubenswrapper[5052]: I1203 12:11:43.439349 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-db-sync-xr2pm" event={"ID":"db210aa9-d72d-4114-945c-0aa978c27a18","Type":"ContainerDied","Data":"1b26b5bb220817a212890103338522656533e160a2e17b3e6d5fb0b59f0dba6d"} Dec 03 12:11:43 crc kubenswrapper[5052]: I1203 12:11:43.439409 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-db-sync-xr2pm" Dec 03 12:11:43 crc kubenswrapper[5052]: I1203 12:11:43.439420 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1b26b5bb220817a212890103338522656533e160a2e17b3e6d5fb0b59f0dba6d" Dec 03 12:11:43 crc kubenswrapper[5052]: I1203 12:11:43.577705 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/placement-79d5cb4fdb-7h2fn"] Dec 03 12:11:43 crc kubenswrapper[5052]: E1203 12:11:43.578195 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="db210aa9-d72d-4114-945c-0aa978c27a18" containerName="placement-db-sync" Dec 03 12:11:43 crc kubenswrapper[5052]: I1203 12:11:43.578219 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="db210aa9-d72d-4114-945c-0aa978c27a18" containerName="placement-db-sync" Dec 03 12:11:43 crc kubenswrapper[5052]: I1203 12:11:43.578424 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="db210aa9-d72d-4114-945c-0aa978c27a18" containerName="placement-db-sync" Dec 03 12:11:43 crc kubenswrapper[5052]: I1203 12:11:43.579609 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-79d5cb4fdb-7h2fn" Dec 03 12:11:43 crc kubenswrapper[5052]: I1203 12:11:43.582055 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-placement-public-svc" Dec 03 12:11:43 crc kubenswrapper[5052]: I1203 12:11:43.582267 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-placement-internal-svc" Dec 03 12:11:43 crc kubenswrapper[5052]: I1203 12:11:43.582429 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"placement-config-data" Dec 03 12:11:43 crc kubenswrapper[5052]: I1203 12:11:43.582625 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"placement-placement-dockercfg-xvh6b" Dec 03 12:11:43 crc kubenswrapper[5052]: I1203 12:11:43.599701 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/8283de30-8976-47bd-bd09-aab53140f68d-internal-tls-certs\") pod \"placement-79d5cb4fdb-7h2fn\" (UID: \"8283de30-8976-47bd-bd09-aab53140f68d\") " pod="openstack-kuttl-tests/placement-79d5cb4fdb-7h2fn" Dec 03 12:11:43 crc kubenswrapper[5052]: I1203 12:11:43.599761 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/8283de30-8976-47bd-bd09-aab53140f68d-config-data\") pod \"placement-79d5cb4fdb-7h2fn\" (UID: \"8283de30-8976-47bd-bd09-aab53140f68d\") " pod="openstack-kuttl-tests/placement-79d5cb4fdb-7h2fn" Dec 03 12:11:43 crc kubenswrapper[5052]: I1203 12:11:43.599875 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/8283de30-8976-47bd-bd09-aab53140f68d-public-tls-certs\") pod \"placement-79d5cb4fdb-7h2fn\" (UID: \"8283de30-8976-47bd-bd09-aab53140f68d\") " pod="openstack-kuttl-tests/placement-79d5cb4fdb-7h2fn" Dec 03 12:11:43 crc kubenswrapper[5052]: I1203 12:11:43.599918 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/8283de30-8976-47bd-bd09-aab53140f68d-logs\") pod \"placement-79d5cb4fdb-7h2fn\" (UID: \"8283de30-8976-47bd-bd09-aab53140f68d\") " pod="openstack-kuttl-tests/placement-79d5cb4fdb-7h2fn" Dec 03 12:11:43 crc kubenswrapper[5052]: I1203 12:11:43.599954 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8283de30-8976-47bd-bd09-aab53140f68d-combined-ca-bundle\") pod \"placement-79d5cb4fdb-7h2fn\" (UID: \"8283de30-8976-47bd-bd09-aab53140f68d\") " pod="openstack-kuttl-tests/placement-79d5cb4fdb-7h2fn" Dec 03 12:11:43 crc kubenswrapper[5052]: I1203 12:11:43.599979 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-69g8d\" (UniqueName: \"kubernetes.io/projected/8283de30-8976-47bd-bd09-aab53140f68d-kube-api-access-69g8d\") pod \"placement-79d5cb4fdb-7h2fn\" (UID: \"8283de30-8976-47bd-bd09-aab53140f68d\") " pod="openstack-kuttl-tests/placement-79d5cb4fdb-7h2fn" Dec 03 12:11:43 crc kubenswrapper[5052]: I1203 12:11:43.600091 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/8283de30-8976-47bd-bd09-aab53140f68d-scripts\") pod \"placement-79d5cb4fdb-7h2fn\" (UID: \"8283de30-8976-47bd-bd09-aab53140f68d\") " pod="openstack-kuttl-tests/placement-79d5cb4fdb-7h2fn" Dec 03 12:11:43 crc kubenswrapper[5052]: I1203 12:11:43.613458 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"placement-scripts" Dec 03 12:11:43 crc kubenswrapper[5052]: I1203 12:11:43.624776 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/placement-79d5cb4fdb-7h2fn"] Dec 03 12:11:43 crc kubenswrapper[5052]: I1203 12:11:43.700958 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/8283de30-8976-47bd-bd09-aab53140f68d-internal-tls-certs\") pod \"placement-79d5cb4fdb-7h2fn\" (UID: \"8283de30-8976-47bd-bd09-aab53140f68d\") " pod="openstack-kuttl-tests/placement-79d5cb4fdb-7h2fn" Dec 03 12:11:43 crc kubenswrapper[5052]: I1203 12:11:43.701020 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/8283de30-8976-47bd-bd09-aab53140f68d-config-data\") pod \"placement-79d5cb4fdb-7h2fn\" (UID: \"8283de30-8976-47bd-bd09-aab53140f68d\") " pod="openstack-kuttl-tests/placement-79d5cb4fdb-7h2fn" Dec 03 12:11:43 crc kubenswrapper[5052]: I1203 12:11:43.701171 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/8283de30-8976-47bd-bd09-aab53140f68d-public-tls-certs\") pod \"placement-79d5cb4fdb-7h2fn\" (UID: \"8283de30-8976-47bd-bd09-aab53140f68d\") " pod="openstack-kuttl-tests/placement-79d5cb4fdb-7h2fn" Dec 03 12:11:43 crc kubenswrapper[5052]: I1203 12:11:43.701243 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/8283de30-8976-47bd-bd09-aab53140f68d-logs\") pod \"placement-79d5cb4fdb-7h2fn\" (UID: \"8283de30-8976-47bd-bd09-aab53140f68d\") " pod="openstack-kuttl-tests/placement-79d5cb4fdb-7h2fn" Dec 03 12:11:43 crc kubenswrapper[5052]: I1203 12:11:43.701287 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8283de30-8976-47bd-bd09-aab53140f68d-combined-ca-bundle\") pod \"placement-79d5cb4fdb-7h2fn\" (UID: \"8283de30-8976-47bd-bd09-aab53140f68d\") " pod="openstack-kuttl-tests/placement-79d5cb4fdb-7h2fn" Dec 03 12:11:43 crc kubenswrapper[5052]: I1203 12:11:43.701316 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-69g8d\" (UniqueName: \"kubernetes.io/projected/8283de30-8976-47bd-bd09-aab53140f68d-kube-api-access-69g8d\") pod \"placement-79d5cb4fdb-7h2fn\" (UID: \"8283de30-8976-47bd-bd09-aab53140f68d\") " pod="openstack-kuttl-tests/placement-79d5cb4fdb-7h2fn" Dec 03 12:11:43 crc kubenswrapper[5052]: I1203 12:11:43.701359 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/8283de30-8976-47bd-bd09-aab53140f68d-scripts\") pod \"placement-79d5cb4fdb-7h2fn\" (UID: \"8283de30-8976-47bd-bd09-aab53140f68d\") " pod="openstack-kuttl-tests/placement-79d5cb4fdb-7h2fn" Dec 03 12:11:43 crc kubenswrapper[5052]: I1203 12:11:43.702538 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/8283de30-8976-47bd-bd09-aab53140f68d-logs\") pod \"placement-79d5cb4fdb-7h2fn\" (UID: \"8283de30-8976-47bd-bd09-aab53140f68d\") " pod="openstack-kuttl-tests/placement-79d5cb4fdb-7h2fn" Dec 03 12:11:43 crc kubenswrapper[5052]: I1203 12:11:43.707299 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/8283de30-8976-47bd-bd09-aab53140f68d-internal-tls-certs\") pod \"placement-79d5cb4fdb-7h2fn\" (UID: \"8283de30-8976-47bd-bd09-aab53140f68d\") " pod="openstack-kuttl-tests/placement-79d5cb4fdb-7h2fn" Dec 03 12:11:43 crc kubenswrapper[5052]: I1203 12:11:43.707871 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8283de30-8976-47bd-bd09-aab53140f68d-combined-ca-bundle\") pod \"placement-79d5cb4fdb-7h2fn\" (UID: \"8283de30-8976-47bd-bd09-aab53140f68d\") " pod="openstack-kuttl-tests/placement-79d5cb4fdb-7h2fn" Dec 03 12:11:43 crc kubenswrapper[5052]: I1203 12:11:43.709380 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/8283de30-8976-47bd-bd09-aab53140f68d-scripts\") pod \"placement-79d5cb4fdb-7h2fn\" (UID: \"8283de30-8976-47bd-bd09-aab53140f68d\") " pod="openstack-kuttl-tests/placement-79d5cb4fdb-7h2fn" Dec 03 12:11:43 crc kubenswrapper[5052]: I1203 12:11:43.709980 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/8283de30-8976-47bd-bd09-aab53140f68d-public-tls-certs\") pod \"placement-79d5cb4fdb-7h2fn\" (UID: \"8283de30-8976-47bd-bd09-aab53140f68d\") " pod="openstack-kuttl-tests/placement-79d5cb4fdb-7h2fn" Dec 03 12:11:43 crc kubenswrapper[5052]: I1203 12:11:43.724701 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/8283de30-8976-47bd-bd09-aab53140f68d-config-data\") pod \"placement-79d5cb4fdb-7h2fn\" (UID: \"8283de30-8976-47bd-bd09-aab53140f68d\") " pod="openstack-kuttl-tests/placement-79d5cb4fdb-7h2fn" Dec 03 12:11:43 crc kubenswrapper[5052]: I1203 12:11:43.731812 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-69g8d\" (UniqueName: \"kubernetes.io/projected/8283de30-8976-47bd-bd09-aab53140f68d-kube-api-access-69g8d\") pod \"placement-79d5cb4fdb-7h2fn\" (UID: \"8283de30-8976-47bd-bd09-aab53140f68d\") " pod="openstack-kuttl-tests/placement-79d5cb4fdb-7h2fn" Dec 03 12:11:43 crc kubenswrapper[5052]: I1203 12:11:43.945616 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-79d5cb4fdb-7h2fn" Dec 03 12:11:44 crc kubenswrapper[5052]: I1203 12:11:44.444870 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/placement-79d5cb4fdb-7h2fn"] Dec 03 12:11:44 crc kubenswrapper[5052]: W1203 12:11:44.462030 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8283de30_8976_47bd_bd09_aab53140f68d.slice/crio-a8fc9e7655d23ddd94c7d18ce5577d0d76d7c6c69c0d07ad157edcd42a1d42fd WatchSource:0}: Error finding container a8fc9e7655d23ddd94c7d18ce5577d0d76d7c6c69c0d07ad157edcd42a1d42fd: Status 404 returned error can't find the container with id a8fc9e7655d23ddd94c7d18ce5577d0d76d7c6c69c0d07ad157edcd42a1d42fd Dec 03 12:11:45 crc kubenswrapper[5052]: I1203 12:11:45.464187 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-79d5cb4fdb-7h2fn" event={"ID":"8283de30-8976-47bd-bd09-aab53140f68d","Type":"ContainerStarted","Data":"2814b190b744e9cf8edd55553cf57bc90eebd7268c2360b6efc06f7b369e975c"} Dec 03 12:11:45 crc kubenswrapper[5052]: I1203 12:11:45.464619 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-79d5cb4fdb-7h2fn" event={"ID":"8283de30-8976-47bd-bd09-aab53140f68d","Type":"ContainerStarted","Data":"2b2c845f4fc350fd56a0f9ca2b46def3a0f193a23ad1b9dd526750dac6bdc0e7"} Dec 03 12:11:45 crc kubenswrapper[5052]: I1203 12:11:45.464637 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-79d5cb4fdb-7h2fn" event={"ID":"8283de30-8976-47bd-bd09-aab53140f68d","Type":"ContainerStarted","Data":"a8fc9e7655d23ddd94c7d18ce5577d0d76d7c6c69c0d07ad157edcd42a1d42fd"} Dec 03 12:11:45 crc kubenswrapper[5052]: I1203 12:11:45.464678 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/placement-79d5cb4fdb-7h2fn" Dec 03 12:11:45 crc kubenswrapper[5052]: I1203 12:11:45.464695 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/placement-79d5cb4fdb-7h2fn" Dec 03 12:11:45 crc kubenswrapper[5052]: I1203 12:11:45.494025 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/placement-79d5cb4fdb-7h2fn" podStartSLOduration=2.494006611 podStartE2EDuration="2.494006611s" podCreationTimestamp="2025-12-03 12:11:43 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:11:45.485980082 +0000 UTC m=+3096.074221846" watchObservedRunningTime="2025-12-03 12:11:45.494006611 +0000 UTC m=+3096.082248375" Dec 03 12:11:46 crc kubenswrapper[5052]: I1203 12:11:46.479091 5052 generic.go:334] "Generic (PLEG): container finished" podID="4ef6dc74-a5f4-489c-8527-df175eb265ec" containerID="f020c0314d37f1ebed5754d085ae61111bf03a1246b93ef4ff86bfaf736db918" exitCode=0 Dec 03 12:11:46 crc kubenswrapper[5052]: I1203 12:11:46.479208 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-bootstrap-v44q4" event={"ID":"4ef6dc74-a5f4-489c-8527-df175eb265ec","Type":"ContainerDied","Data":"f020c0314d37f1ebed5754d085ae61111bf03a1246b93ef4ff86bfaf736db918"} Dec 03 12:11:47 crc kubenswrapper[5052]: I1203 12:11:47.832196 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-bootstrap-v44q4" Dec 03 12:11:47 crc kubenswrapper[5052]: I1203 12:11:47.986095 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kpbpj\" (UniqueName: \"kubernetes.io/projected/4ef6dc74-a5f4-489c-8527-df175eb265ec-kube-api-access-kpbpj\") pod \"4ef6dc74-a5f4-489c-8527-df175eb265ec\" (UID: \"4ef6dc74-a5f4-489c-8527-df175eb265ec\") " Dec 03 12:11:47 crc kubenswrapper[5052]: I1203 12:11:47.986209 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/4ef6dc74-a5f4-489c-8527-df175eb265ec-credential-keys\") pod \"4ef6dc74-a5f4-489c-8527-df175eb265ec\" (UID: \"4ef6dc74-a5f4-489c-8527-df175eb265ec\") " Dec 03 12:11:47 crc kubenswrapper[5052]: I1203 12:11:47.986265 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/4ef6dc74-a5f4-489c-8527-df175eb265ec-scripts\") pod \"4ef6dc74-a5f4-489c-8527-df175eb265ec\" (UID: \"4ef6dc74-a5f4-489c-8527-df175eb265ec\") " Dec 03 12:11:47 crc kubenswrapper[5052]: I1203 12:11:47.986285 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4ef6dc74-a5f4-489c-8527-df175eb265ec-combined-ca-bundle\") pod \"4ef6dc74-a5f4-489c-8527-df175eb265ec\" (UID: \"4ef6dc74-a5f4-489c-8527-df175eb265ec\") " Dec 03 12:11:47 crc kubenswrapper[5052]: I1203 12:11:47.986343 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/4ef6dc74-a5f4-489c-8527-df175eb265ec-fernet-keys\") pod \"4ef6dc74-a5f4-489c-8527-df175eb265ec\" (UID: \"4ef6dc74-a5f4-489c-8527-df175eb265ec\") " Dec 03 12:11:47 crc kubenswrapper[5052]: I1203 12:11:47.986393 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/4ef6dc74-a5f4-489c-8527-df175eb265ec-config-data\") pod \"4ef6dc74-a5f4-489c-8527-df175eb265ec\" (UID: \"4ef6dc74-a5f4-489c-8527-df175eb265ec\") " Dec 03 12:11:47 crc kubenswrapper[5052]: I1203 12:11:47.993607 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4ef6dc74-a5f4-489c-8527-df175eb265ec-fernet-keys" (OuterVolumeSpecName: "fernet-keys") pod "4ef6dc74-a5f4-489c-8527-df175eb265ec" (UID: "4ef6dc74-a5f4-489c-8527-df175eb265ec"). InnerVolumeSpecName "fernet-keys". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:11:47 crc kubenswrapper[5052]: I1203 12:11:47.993894 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4ef6dc74-a5f4-489c-8527-df175eb265ec-kube-api-access-kpbpj" (OuterVolumeSpecName: "kube-api-access-kpbpj") pod "4ef6dc74-a5f4-489c-8527-df175eb265ec" (UID: "4ef6dc74-a5f4-489c-8527-df175eb265ec"). InnerVolumeSpecName "kube-api-access-kpbpj". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:11:47 crc kubenswrapper[5052]: I1203 12:11:47.994674 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4ef6dc74-a5f4-489c-8527-df175eb265ec-credential-keys" (OuterVolumeSpecName: "credential-keys") pod "4ef6dc74-a5f4-489c-8527-df175eb265ec" (UID: "4ef6dc74-a5f4-489c-8527-df175eb265ec"). InnerVolumeSpecName "credential-keys". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:11:47 crc kubenswrapper[5052]: I1203 12:11:47.995298 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4ef6dc74-a5f4-489c-8527-df175eb265ec-scripts" (OuterVolumeSpecName: "scripts") pod "4ef6dc74-a5f4-489c-8527-df175eb265ec" (UID: "4ef6dc74-a5f4-489c-8527-df175eb265ec"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:11:48 crc kubenswrapper[5052]: I1203 12:11:48.019142 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4ef6dc74-a5f4-489c-8527-df175eb265ec-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "4ef6dc74-a5f4-489c-8527-df175eb265ec" (UID: "4ef6dc74-a5f4-489c-8527-df175eb265ec"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:11:48 crc kubenswrapper[5052]: I1203 12:11:48.028738 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4ef6dc74-a5f4-489c-8527-df175eb265ec-config-data" (OuterVolumeSpecName: "config-data") pod "4ef6dc74-a5f4-489c-8527-df175eb265ec" (UID: "4ef6dc74-a5f4-489c-8527-df175eb265ec"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:11:48 crc kubenswrapper[5052]: I1203 12:11:48.087899 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/4ef6dc74-a5f4-489c-8527-df175eb265ec-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:11:48 crc kubenswrapper[5052]: I1203 12:11:48.087935 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-kpbpj\" (UniqueName: \"kubernetes.io/projected/4ef6dc74-a5f4-489c-8527-df175eb265ec-kube-api-access-kpbpj\") on node \"crc\" DevicePath \"\"" Dec 03 12:11:48 crc kubenswrapper[5052]: I1203 12:11:48.087945 5052 reconciler_common.go:293] "Volume detached for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/4ef6dc74-a5f4-489c-8527-df175eb265ec-credential-keys\") on node \"crc\" DevicePath \"\"" Dec 03 12:11:48 crc kubenswrapper[5052]: I1203 12:11:48.087955 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/4ef6dc74-a5f4-489c-8527-df175eb265ec-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:11:48 crc kubenswrapper[5052]: I1203 12:11:48.087964 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4ef6dc74-a5f4-489c-8527-df175eb265ec-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:11:48 crc kubenswrapper[5052]: I1203 12:11:48.087973 5052 reconciler_common.go:293] "Volume detached for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/4ef6dc74-a5f4-489c-8527-df175eb265ec-fernet-keys\") on node \"crc\" DevicePath \"\"" Dec 03 12:11:48 crc kubenswrapper[5052]: I1203 12:11:48.508010 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-bootstrap-v44q4" event={"ID":"4ef6dc74-a5f4-489c-8527-df175eb265ec","Type":"ContainerDied","Data":"86b82324d1ec982b43348a426986a60c4a121028f5d4b359f45f94d107757ddf"} Dec 03 12:11:48 crc kubenswrapper[5052]: I1203 12:11:48.508451 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="86b82324d1ec982b43348a426986a60c4a121028f5d4b359f45f94d107757ddf" Dec 03 12:11:48 crc kubenswrapper[5052]: I1203 12:11:48.508065 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-bootstrap-v44q4" Dec 03 12:11:48 crc kubenswrapper[5052]: I1203 12:11:48.607228 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/keystone-6fd4994758-fsw2c"] Dec 03 12:11:48 crc kubenswrapper[5052]: E1203 12:11:48.607596 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="4ef6dc74-a5f4-489c-8527-df175eb265ec" containerName="keystone-bootstrap" Dec 03 12:11:48 crc kubenswrapper[5052]: I1203 12:11:48.607612 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="4ef6dc74-a5f4-489c-8527-df175eb265ec" containerName="keystone-bootstrap" Dec 03 12:11:48 crc kubenswrapper[5052]: I1203 12:11:48.607839 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="4ef6dc74-a5f4-489c-8527-df175eb265ec" containerName="keystone-bootstrap" Dec 03 12:11:48 crc kubenswrapper[5052]: I1203 12:11:48.608446 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-6fd4994758-fsw2c" Dec 03 12:11:48 crc kubenswrapper[5052]: I1203 12:11:48.611198 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone-keystone-dockercfg-4wch9" Dec 03 12:11:48 crc kubenswrapper[5052]: I1203 12:11:48.611361 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone-config-data" Dec 03 12:11:48 crc kubenswrapper[5052]: I1203 12:11:48.611799 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone-scripts" Dec 03 12:11:48 crc kubenswrapper[5052]: I1203 12:11:48.612308 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-keystone-public-svc" Dec 03 12:11:48 crc kubenswrapper[5052]: I1203 12:11:48.612517 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone" Dec 03 12:11:48 crc kubenswrapper[5052]: I1203 12:11:48.612732 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-keystone-internal-svc" Dec 03 12:11:48 crc kubenswrapper[5052]: I1203 12:11:48.623536 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-6fd4994758-fsw2c"] Dec 03 12:11:48 crc kubenswrapper[5052]: I1203 12:11:48.802823 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/e14dd2e3-f6e1-450c-a688-1dc4eb2ee622-scripts\") pod \"keystone-6fd4994758-fsw2c\" (UID: \"e14dd2e3-f6e1-450c-a688-1dc4eb2ee622\") " pod="openstack-kuttl-tests/keystone-6fd4994758-fsw2c" Dec 03 12:11:48 crc kubenswrapper[5052]: I1203 12:11:48.802974 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/e14dd2e3-f6e1-450c-a688-1dc4eb2ee622-config-data\") pod \"keystone-6fd4994758-fsw2c\" (UID: \"e14dd2e3-f6e1-450c-a688-1dc4eb2ee622\") " pod="openstack-kuttl-tests/keystone-6fd4994758-fsw2c" Dec 03 12:11:48 crc kubenswrapper[5052]: I1203 12:11:48.803019 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/e14dd2e3-f6e1-450c-a688-1dc4eb2ee622-public-tls-certs\") pod \"keystone-6fd4994758-fsw2c\" (UID: \"e14dd2e3-f6e1-450c-a688-1dc4eb2ee622\") " pod="openstack-kuttl-tests/keystone-6fd4994758-fsw2c" Dec 03 12:11:48 crc kubenswrapper[5052]: I1203 12:11:48.803049 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/e14dd2e3-f6e1-450c-a688-1dc4eb2ee622-credential-keys\") pod \"keystone-6fd4994758-fsw2c\" (UID: \"e14dd2e3-f6e1-450c-a688-1dc4eb2ee622\") " pod="openstack-kuttl-tests/keystone-6fd4994758-fsw2c" Dec 03 12:11:48 crc kubenswrapper[5052]: I1203 12:11:48.803074 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e14dd2e3-f6e1-450c-a688-1dc4eb2ee622-combined-ca-bundle\") pod \"keystone-6fd4994758-fsw2c\" (UID: \"e14dd2e3-f6e1-450c-a688-1dc4eb2ee622\") " pod="openstack-kuttl-tests/keystone-6fd4994758-fsw2c" Dec 03 12:11:48 crc kubenswrapper[5052]: I1203 12:11:48.803095 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gnjbz\" (UniqueName: \"kubernetes.io/projected/e14dd2e3-f6e1-450c-a688-1dc4eb2ee622-kube-api-access-gnjbz\") pod \"keystone-6fd4994758-fsw2c\" (UID: \"e14dd2e3-f6e1-450c-a688-1dc4eb2ee622\") " pod="openstack-kuttl-tests/keystone-6fd4994758-fsw2c" Dec 03 12:11:48 crc kubenswrapper[5052]: I1203 12:11:48.803184 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/e14dd2e3-f6e1-450c-a688-1dc4eb2ee622-internal-tls-certs\") pod \"keystone-6fd4994758-fsw2c\" (UID: \"e14dd2e3-f6e1-450c-a688-1dc4eb2ee622\") " pod="openstack-kuttl-tests/keystone-6fd4994758-fsw2c" Dec 03 12:11:48 crc kubenswrapper[5052]: I1203 12:11:48.803244 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/e14dd2e3-f6e1-450c-a688-1dc4eb2ee622-fernet-keys\") pod \"keystone-6fd4994758-fsw2c\" (UID: \"e14dd2e3-f6e1-450c-a688-1dc4eb2ee622\") " pod="openstack-kuttl-tests/keystone-6fd4994758-fsw2c" Dec 03 12:11:48 crc kubenswrapper[5052]: I1203 12:11:48.905081 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/e14dd2e3-f6e1-450c-a688-1dc4eb2ee622-config-data\") pod \"keystone-6fd4994758-fsw2c\" (UID: \"e14dd2e3-f6e1-450c-a688-1dc4eb2ee622\") " pod="openstack-kuttl-tests/keystone-6fd4994758-fsw2c" Dec 03 12:11:48 crc kubenswrapper[5052]: I1203 12:11:48.905131 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/e14dd2e3-f6e1-450c-a688-1dc4eb2ee622-public-tls-certs\") pod \"keystone-6fd4994758-fsw2c\" (UID: \"e14dd2e3-f6e1-450c-a688-1dc4eb2ee622\") " pod="openstack-kuttl-tests/keystone-6fd4994758-fsw2c" Dec 03 12:11:48 crc kubenswrapper[5052]: I1203 12:11:48.905155 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/e14dd2e3-f6e1-450c-a688-1dc4eb2ee622-credential-keys\") pod \"keystone-6fd4994758-fsw2c\" (UID: \"e14dd2e3-f6e1-450c-a688-1dc4eb2ee622\") " pod="openstack-kuttl-tests/keystone-6fd4994758-fsw2c" Dec 03 12:11:48 crc kubenswrapper[5052]: I1203 12:11:48.905173 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e14dd2e3-f6e1-450c-a688-1dc4eb2ee622-combined-ca-bundle\") pod \"keystone-6fd4994758-fsw2c\" (UID: \"e14dd2e3-f6e1-450c-a688-1dc4eb2ee622\") " pod="openstack-kuttl-tests/keystone-6fd4994758-fsw2c" Dec 03 12:11:48 crc kubenswrapper[5052]: I1203 12:11:48.905192 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-gnjbz\" (UniqueName: \"kubernetes.io/projected/e14dd2e3-f6e1-450c-a688-1dc4eb2ee622-kube-api-access-gnjbz\") pod \"keystone-6fd4994758-fsw2c\" (UID: \"e14dd2e3-f6e1-450c-a688-1dc4eb2ee622\") " pod="openstack-kuttl-tests/keystone-6fd4994758-fsw2c" Dec 03 12:11:48 crc kubenswrapper[5052]: I1203 12:11:48.905212 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/e14dd2e3-f6e1-450c-a688-1dc4eb2ee622-internal-tls-certs\") pod \"keystone-6fd4994758-fsw2c\" (UID: \"e14dd2e3-f6e1-450c-a688-1dc4eb2ee622\") " pod="openstack-kuttl-tests/keystone-6fd4994758-fsw2c" Dec 03 12:11:48 crc kubenswrapper[5052]: I1203 12:11:48.905231 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/e14dd2e3-f6e1-450c-a688-1dc4eb2ee622-fernet-keys\") pod \"keystone-6fd4994758-fsw2c\" (UID: \"e14dd2e3-f6e1-450c-a688-1dc4eb2ee622\") " pod="openstack-kuttl-tests/keystone-6fd4994758-fsw2c" Dec 03 12:11:48 crc kubenswrapper[5052]: I1203 12:11:48.905277 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/e14dd2e3-f6e1-450c-a688-1dc4eb2ee622-scripts\") pod \"keystone-6fd4994758-fsw2c\" (UID: \"e14dd2e3-f6e1-450c-a688-1dc4eb2ee622\") " pod="openstack-kuttl-tests/keystone-6fd4994758-fsw2c" Dec 03 12:11:48 crc kubenswrapper[5052]: I1203 12:11:48.910311 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/e14dd2e3-f6e1-450c-a688-1dc4eb2ee622-scripts\") pod \"keystone-6fd4994758-fsw2c\" (UID: \"e14dd2e3-f6e1-450c-a688-1dc4eb2ee622\") " pod="openstack-kuttl-tests/keystone-6fd4994758-fsw2c" Dec 03 12:11:48 crc kubenswrapper[5052]: I1203 12:11:48.910541 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/e14dd2e3-f6e1-450c-a688-1dc4eb2ee622-config-data\") pod \"keystone-6fd4994758-fsw2c\" (UID: \"e14dd2e3-f6e1-450c-a688-1dc4eb2ee622\") " pod="openstack-kuttl-tests/keystone-6fd4994758-fsw2c" Dec 03 12:11:48 crc kubenswrapper[5052]: I1203 12:11:48.910925 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/e14dd2e3-f6e1-450c-a688-1dc4eb2ee622-internal-tls-certs\") pod \"keystone-6fd4994758-fsw2c\" (UID: \"e14dd2e3-f6e1-450c-a688-1dc4eb2ee622\") " pod="openstack-kuttl-tests/keystone-6fd4994758-fsw2c" Dec 03 12:11:48 crc kubenswrapper[5052]: I1203 12:11:48.911079 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/e14dd2e3-f6e1-450c-a688-1dc4eb2ee622-credential-keys\") pod \"keystone-6fd4994758-fsw2c\" (UID: \"e14dd2e3-f6e1-450c-a688-1dc4eb2ee622\") " pod="openstack-kuttl-tests/keystone-6fd4994758-fsw2c" Dec 03 12:11:48 crc kubenswrapper[5052]: I1203 12:11:48.911106 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/e14dd2e3-f6e1-450c-a688-1dc4eb2ee622-public-tls-certs\") pod \"keystone-6fd4994758-fsw2c\" (UID: \"e14dd2e3-f6e1-450c-a688-1dc4eb2ee622\") " pod="openstack-kuttl-tests/keystone-6fd4994758-fsw2c" Dec 03 12:11:48 crc kubenswrapper[5052]: I1203 12:11:48.911433 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e14dd2e3-f6e1-450c-a688-1dc4eb2ee622-combined-ca-bundle\") pod \"keystone-6fd4994758-fsw2c\" (UID: \"e14dd2e3-f6e1-450c-a688-1dc4eb2ee622\") " pod="openstack-kuttl-tests/keystone-6fd4994758-fsw2c" Dec 03 12:11:48 crc kubenswrapper[5052]: I1203 12:11:48.913753 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/e14dd2e3-f6e1-450c-a688-1dc4eb2ee622-fernet-keys\") pod \"keystone-6fd4994758-fsw2c\" (UID: \"e14dd2e3-f6e1-450c-a688-1dc4eb2ee622\") " pod="openstack-kuttl-tests/keystone-6fd4994758-fsw2c" Dec 03 12:11:48 crc kubenswrapper[5052]: I1203 12:11:48.933817 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-gnjbz\" (UniqueName: \"kubernetes.io/projected/e14dd2e3-f6e1-450c-a688-1dc4eb2ee622-kube-api-access-gnjbz\") pod \"keystone-6fd4994758-fsw2c\" (UID: \"e14dd2e3-f6e1-450c-a688-1dc4eb2ee622\") " pod="openstack-kuttl-tests/keystone-6fd4994758-fsw2c" Dec 03 12:11:48 crc kubenswrapper[5052]: I1203 12:11:48.941386 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-6fd4994758-fsw2c" Dec 03 12:11:49 crc kubenswrapper[5052]: I1203 12:11:49.403464 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-6fd4994758-fsw2c"] Dec 03 12:11:49 crc kubenswrapper[5052]: I1203 12:11:49.528103 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-6fd4994758-fsw2c" event={"ID":"e14dd2e3-f6e1-450c-a688-1dc4eb2ee622","Type":"ContainerStarted","Data":"fc4bfab3cfdea7d654ca03ffbcf76a128e0cb31b85de7afed7f10f20633fa5f2"} Dec 03 12:11:50 crc kubenswrapper[5052]: I1203 12:11:50.542308 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-6fd4994758-fsw2c" event={"ID":"e14dd2e3-f6e1-450c-a688-1dc4eb2ee622","Type":"ContainerStarted","Data":"6544c0c4bb0b5942eaa383c638931be4bd4915098254d54ab63534949d4fc2c6"} Dec 03 12:11:50 crc kubenswrapper[5052]: I1203 12:11:50.542925 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/keystone-6fd4994758-fsw2c" Dec 03 12:11:50 crc kubenswrapper[5052]: I1203 12:11:50.574247 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/keystone-6fd4994758-fsw2c" podStartSLOduration=2.574217567 podStartE2EDuration="2.574217567s" podCreationTimestamp="2025-12-03 12:11:48 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:11:50.563973895 +0000 UTC m=+3101.152215659" watchObservedRunningTime="2025-12-03 12:11:50.574217567 +0000 UTC m=+3101.162459341" Dec 03 12:11:52 crc kubenswrapper[5052]: I1203 12:11:52.673405 5052 scope.go:117] "RemoveContainer" containerID="63298107b3d090deff9ed92f3d613e5b1c86d9137684d117500257db8ee1c805" Dec 03 12:11:52 crc kubenswrapper[5052]: E1203 12:11:52.674098 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 12:11:53 crc kubenswrapper[5052]: I1203 12:11:53.574736 5052 generic.go:334] "Generic (PLEG): container finished" podID="049d1d7a-9b34-4d27-a922-ebb89410b3fa" containerID="d25bb92aee772ca0fd8734997ea7b7df542c4bbee1daf0a9eae2d669f91bf473" exitCode=0 Dec 03 12:11:53 crc kubenswrapper[5052]: I1203 12:11:53.574835 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-db-sync-9k6nv" event={"ID":"049d1d7a-9b34-4d27-a922-ebb89410b3fa","Type":"ContainerDied","Data":"d25bb92aee772ca0fd8734997ea7b7df542c4bbee1daf0a9eae2d669f91bf473"} Dec 03 12:11:55 crc kubenswrapper[5052]: I1203 12:11:55.037283 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-db-sync-9k6nv" Dec 03 12:11:55 crc kubenswrapper[5052]: I1203 12:11:55.125478 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cnnn2\" (UniqueName: \"kubernetes.io/projected/049d1d7a-9b34-4d27-a922-ebb89410b3fa-kube-api-access-cnnn2\") pod \"049d1d7a-9b34-4d27-a922-ebb89410b3fa\" (UID: \"049d1d7a-9b34-4d27-a922-ebb89410b3fa\") " Dec 03 12:11:55 crc kubenswrapper[5052]: I1203 12:11:55.125653 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/049d1d7a-9b34-4d27-a922-ebb89410b3fa-db-sync-config-data\") pod \"049d1d7a-9b34-4d27-a922-ebb89410b3fa\" (UID: \"049d1d7a-9b34-4d27-a922-ebb89410b3fa\") " Dec 03 12:11:55 crc kubenswrapper[5052]: I1203 12:11:55.125719 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/049d1d7a-9b34-4d27-a922-ebb89410b3fa-combined-ca-bundle\") pod \"049d1d7a-9b34-4d27-a922-ebb89410b3fa\" (UID: \"049d1d7a-9b34-4d27-a922-ebb89410b3fa\") " Dec 03 12:11:55 crc kubenswrapper[5052]: I1203 12:11:55.143667 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/049d1d7a-9b34-4d27-a922-ebb89410b3fa-kube-api-access-cnnn2" (OuterVolumeSpecName: "kube-api-access-cnnn2") pod "049d1d7a-9b34-4d27-a922-ebb89410b3fa" (UID: "049d1d7a-9b34-4d27-a922-ebb89410b3fa"). InnerVolumeSpecName "kube-api-access-cnnn2". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:11:55 crc kubenswrapper[5052]: I1203 12:11:55.162755 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/049d1d7a-9b34-4d27-a922-ebb89410b3fa-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "049d1d7a-9b34-4d27-a922-ebb89410b3fa" (UID: "049d1d7a-9b34-4d27-a922-ebb89410b3fa"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:11:55 crc kubenswrapper[5052]: I1203 12:11:55.163948 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/049d1d7a-9b34-4d27-a922-ebb89410b3fa-db-sync-config-data" (OuterVolumeSpecName: "db-sync-config-data") pod "049d1d7a-9b34-4d27-a922-ebb89410b3fa" (UID: "049d1d7a-9b34-4d27-a922-ebb89410b3fa"). InnerVolumeSpecName "db-sync-config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:11:55 crc kubenswrapper[5052]: I1203 12:11:55.228075 5052 reconciler_common.go:293] "Volume detached for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/049d1d7a-9b34-4d27-a922-ebb89410b3fa-db-sync-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:11:55 crc kubenswrapper[5052]: I1203 12:11:55.228113 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/049d1d7a-9b34-4d27-a922-ebb89410b3fa-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:11:55 crc kubenswrapper[5052]: I1203 12:11:55.228127 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-cnnn2\" (UniqueName: \"kubernetes.io/projected/049d1d7a-9b34-4d27-a922-ebb89410b3fa-kube-api-access-cnnn2\") on node \"crc\" DevicePath \"\"" Dec 03 12:11:55 crc kubenswrapper[5052]: I1203 12:11:55.595175 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-db-sync-9k6nv" event={"ID":"049d1d7a-9b34-4d27-a922-ebb89410b3fa","Type":"ContainerDied","Data":"1caba2386268b9a1ff276f86660f391eb7a9d879dca589e1ae6554dab47e0b90"} Dec 03 12:11:55 crc kubenswrapper[5052]: I1203 12:11:55.595226 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1caba2386268b9a1ff276f86660f391eb7a9d879dca589e1ae6554dab47e0b90" Dec 03 12:11:55 crc kubenswrapper[5052]: I1203 12:11:55.595292 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-db-sync-9k6nv" Dec 03 12:11:55 crc kubenswrapper[5052]: I1203 12:11:55.805340 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/barbican-worker-6ff59c8c65-fsb2k"] Dec 03 12:11:55 crc kubenswrapper[5052]: E1203 12:11:55.805754 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="049d1d7a-9b34-4d27-a922-ebb89410b3fa" containerName="barbican-db-sync" Dec 03 12:11:55 crc kubenswrapper[5052]: I1203 12:11:55.805770 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="049d1d7a-9b34-4d27-a922-ebb89410b3fa" containerName="barbican-db-sync" Dec 03 12:11:55 crc kubenswrapper[5052]: I1203 12:11:55.805941 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="049d1d7a-9b34-4d27-a922-ebb89410b3fa" containerName="barbican-db-sync" Dec 03 12:11:55 crc kubenswrapper[5052]: I1203 12:11:55.806851 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-worker-6ff59c8c65-fsb2k" Dec 03 12:11:55 crc kubenswrapper[5052]: I1203 12:11:55.821145 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"barbican-config-data" Dec 03 12:11:55 crc kubenswrapper[5052]: I1203 12:11:55.821335 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"barbican-worker-config-data" Dec 03 12:11:55 crc kubenswrapper[5052]: I1203 12:11:55.821454 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"barbican-barbican-dockercfg-k9mf5" Dec 03 12:11:55 crc kubenswrapper[5052]: I1203 12:11:55.851567 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-worker-6ff59c8c65-fsb2k"] Dec 03 12:11:55 crc kubenswrapper[5052]: I1203 12:11:55.874267 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/barbican-keystone-listener-85f96c8f8-b6m5f"] Dec 03 12:11:55 crc kubenswrapper[5052]: I1203 12:11:55.875610 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-keystone-listener-85f96c8f8-b6m5f" Dec 03 12:11:55 crc kubenswrapper[5052]: I1203 12:11:55.901098 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"barbican-keystone-listener-config-data" Dec 03 12:11:55 crc kubenswrapper[5052]: I1203 12:11:55.941581 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/1c98e0a9-267b-46b6-be02-fbd163462940-config-data-custom\") pod \"barbican-worker-6ff59c8c65-fsb2k\" (UID: \"1c98e0a9-267b-46b6-be02-fbd163462940\") " pod="openstack-kuttl-tests/barbican-worker-6ff59c8c65-fsb2k" Dec 03 12:11:55 crc kubenswrapper[5052]: I1203 12:11:55.941679 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fddgs\" (UniqueName: \"kubernetes.io/projected/1c98e0a9-267b-46b6-be02-fbd163462940-kube-api-access-fddgs\") pod \"barbican-worker-6ff59c8c65-fsb2k\" (UID: \"1c98e0a9-267b-46b6-be02-fbd163462940\") " pod="openstack-kuttl-tests/barbican-worker-6ff59c8c65-fsb2k" Dec 03 12:11:55 crc kubenswrapper[5052]: I1203 12:11:55.941750 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/1c98e0a9-267b-46b6-be02-fbd163462940-config-data\") pod \"barbican-worker-6ff59c8c65-fsb2k\" (UID: \"1c98e0a9-267b-46b6-be02-fbd163462940\") " pod="openstack-kuttl-tests/barbican-worker-6ff59c8c65-fsb2k" Dec 03 12:11:55 crc kubenswrapper[5052]: I1203 12:11:55.941781 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/1c98e0a9-267b-46b6-be02-fbd163462940-logs\") pod \"barbican-worker-6ff59c8c65-fsb2k\" (UID: \"1c98e0a9-267b-46b6-be02-fbd163462940\") " pod="openstack-kuttl-tests/barbican-worker-6ff59c8c65-fsb2k" Dec 03 12:11:55 crc kubenswrapper[5052]: I1203 12:11:55.941806 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1c98e0a9-267b-46b6-be02-fbd163462940-combined-ca-bundle\") pod \"barbican-worker-6ff59c8c65-fsb2k\" (UID: \"1c98e0a9-267b-46b6-be02-fbd163462940\") " pod="openstack-kuttl-tests/barbican-worker-6ff59c8c65-fsb2k" Dec 03 12:11:55 crc kubenswrapper[5052]: I1203 12:11:55.963579 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-keystone-listener-85f96c8f8-b6m5f"] Dec 03 12:11:56 crc kubenswrapper[5052]: I1203 12:11:56.041413 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/barbican-api-649f8bf5b8-wjzcq"] Dec 03 12:11:56 crc kubenswrapper[5052]: I1203 12:11:56.042817 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-api-649f8bf5b8-wjzcq" Dec 03 12:11:56 crc kubenswrapper[5052]: I1203 12:11:56.043555 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/296a2858-22b8-41a4-91ac-a9bf055b2848-config-data-custom\") pod \"barbican-keystone-listener-85f96c8f8-b6m5f\" (UID: \"296a2858-22b8-41a4-91ac-a9bf055b2848\") " pod="openstack-kuttl-tests/barbican-keystone-listener-85f96c8f8-b6m5f" Dec 03 12:11:56 crc kubenswrapper[5052]: I1203 12:11:56.043611 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dhznr\" (UniqueName: \"kubernetes.io/projected/296a2858-22b8-41a4-91ac-a9bf055b2848-kube-api-access-dhznr\") pod \"barbican-keystone-listener-85f96c8f8-b6m5f\" (UID: \"296a2858-22b8-41a4-91ac-a9bf055b2848\") " pod="openstack-kuttl-tests/barbican-keystone-listener-85f96c8f8-b6m5f" Dec 03 12:11:56 crc kubenswrapper[5052]: I1203 12:11:56.043642 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/1c98e0a9-267b-46b6-be02-fbd163462940-config-data-custom\") pod \"barbican-worker-6ff59c8c65-fsb2k\" (UID: \"1c98e0a9-267b-46b6-be02-fbd163462940\") " pod="openstack-kuttl-tests/barbican-worker-6ff59c8c65-fsb2k" Dec 03 12:11:56 crc kubenswrapper[5052]: I1203 12:11:56.043692 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-fddgs\" (UniqueName: \"kubernetes.io/projected/1c98e0a9-267b-46b6-be02-fbd163462940-kube-api-access-fddgs\") pod \"barbican-worker-6ff59c8c65-fsb2k\" (UID: \"1c98e0a9-267b-46b6-be02-fbd163462940\") " pod="openstack-kuttl-tests/barbican-worker-6ff59c8c65-fsb2k" Dec 03 12:11:56 crc kubenswrapper[5052]: I1203 12:11:56.043710 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/296a2858-22b8-41a4-91ac-a9bf055b2848-config-data\") pod \"barbican-keystone-listener-85f96c8f8-b6m5f\" (UID: \"296a2858-22b8-41a4-91ac-a9bf055b2848\") " pod="openstack-kuttl-tests/barbican-keystone-listener-85f96c8f8-b6m5f" Dec 03 12:11:56 crc kubenswrapper[5052]: I1203 12:11:56.043733 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/296a2858-22b8-41a4-91ac-a9bf055b2848-combined-ca-bundle\") pod \"barbican-keystone-listener-85f96c8f8-b6m5f\" (UID: \"296a2858-22b8-41a4-91ac-a9bf055b2848\") " pod="openstack-kuttl-tests/barbican-keystone-listener-85f96c8f8-b6m5f" Dec 03 12:11:56 crc kubenswrapper[5052]: I1203 12:11:56.043769 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/296a2858-22b8-41a4-91ac-a9bf055b2848-logs\") pod \"barbican-keystone-listener-85f96c8f8-b6m5f\" (UID: \"296a2858-22b8-41a4-91ac-a9bf055b2848\") " pod="openstack-kuttl-tests/barbican-keystone-listener-85f96c8f8-b6m5f" Dec 03 12:11:56 crc kubenswrapper[5052]: I1203 12:11:56.043790 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/1c98e0a9-267b-46b6-be02-fbd163462940-config-data\") pod \"barbican-worker-6ff59c8c65-fsb2k\" (UID: \"1c98e0a9-267b-46b6-be02-fbd163462940\") " pod="openstack-kuttl-tests/barbican-worker-6ff59c8c65-fsb2k" Dec 03 12:11:56 crc kubenswrapper[5052]: I1203 12:11:56.043809 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/1c98e0a9-267b-46b6-be02-fbd163462940-logs\") pod \"barbican-worker-6ff59c8c65-fsb2k\" (UID: \"1c98e0a9-267b-46b6-be02-fbd163462940\") " pod="openstack-kuttl-tests/barbican-worker-6ff59c8c65-fsb2k" Dec 03 12:11:56 crc kubenswrapper[5052]: I1203 12:11:56.043825 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1c98e0a9-267b-46b6-be02-fbd163462940-combined-ca-bundle\") pod \"barbican-worker-6ff59c8c65-fsb2k\" (UID: \"1c98e0a9-267b-46b6-be02-fbd163462940\") " pod="openstack-kuttl-tests/barbican-worker-6ff59c8c65-fsb2k" Dec 03 12:11:56 crc kubenswrapper[5052]: I1203 12:11:56.047648 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"barbican-api-config-data" Dec 03 12:11:56 crc kubenswrapper[5052]: I1203 12:11:56.049981 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/1c98e0a9-267b-46b6-be02-fbd163462940-logs\") pod \"barbican-worker-6ff59c8c65-fsb2k\" (UID: \"1c98e0a9-267b-46b6-be02-fbd163462940\") " pod="openstack-kuttl-tests/barbican-worker-6ff59c8c65-fsb2k" Dec 03 12:11:56 crc kubenswrapper[5052]: I1203 12:11:56.055597 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1c98e0a9-267b-46b6-be02-fbd163462940-combined-ca-bundle\") pod \"barbican-worker-6ff59c8c65-fsb2k\" (UID: \"1c98e0a9-267b-46b6-be02-fbd163462940\") " pod="openstack-kuttl-tests/barbican-worker-6ff59c8c65-fsb2k" Dec 03 12:11:56 crc kubenswrapper[5052]: I1203 12:11:56.056097 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/1c98e0a9-267b-46b6-be02-fbd163462940-config-data\") pod \"barbican-worker-6ff59c8c65-fsb2k\" (UID: \"1c98e0a9-267b-46b6-be02-fbd163462940\") " pod="openstack-kuttl-tests/barbican-worker-6ff59c8c65-fsb2k" Dec 03 12:11:56 crc kubenswrapper[5052]: I1203 12:11:56.071628 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/1c98e0a9-267b-46b6-be02-fbd163462940-config-data-custom\") pod \"barbican-worker-6ff59c8c65-fsb2k\" (UID: \"1c98e0a9-267b-46b6-be02-fbd163462940\") " pod="openstack-kuttl-tests/barbican-worker-6ff59c8c65-fsb2k" Dec 03 12:11:56 crc kubenswrapper[5052]: I1203 12:11:56.083105 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-fddgs\" (UniqueName: \"kubernetes.io/projected/1c98e0a9-267b-46b6-be02-fbd163462940-kube-api-access-fddgs\") pod \"barbican-worker-6ff59c8c65-fsb2k\" (UID: \"1c98e0a9-267b-46b6-be02-fbd163462940\") " pod="openstack-kuttl-tests/barbican-worker-6ff59c8c65-fsb2k" Dec 03 12:11:56 crc kubenswrapper[5052]: I1203 12:11:56.099933 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-api-649f8bf5b8-wjzcq"] Dec 03 12:11:56 crc kubenswrapper[5052]: I1203 12:11:56.139635 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-worker-6ff59c8c65-fsb2k" Dec 03 12:11:56 crc kubenswrapper[5052]: I1203 12:11:56.147447 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/296a2858-22b8-41a4-91ac-a9bf055b2848-config-data-custom\") pod \"barbican-keystone-listener-85f96c8f8-b6m5f\" (UID: \"296a2858-22b8-41a4-91ac-a9bf055b2848\") " pod="openstack-kuttl-tests/barbican-keystone-listener-85f96c8f8-b6m5f" Dec 03 12:11:56 crc kubenswrapper[5052]: I1203 12:11:56.147500 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-dhznr\" (UniqueName: \"kubernetes.io/projected/296a2858-22b8-41a4-91ac-a9bf055b2848-kube-api-access-dhznr\") pod \"barbican-keystone-listener-85f96c8f8-b6m5f\" (UID: \"296a2858-22b8-41a4-91ac-a9bf055b2848\") " pod="openstack-kuttl-tests/barbican-keystone-listener-85f96c8f8-b6m5f" Dec 03 12:11:56 crc kubenswrapper[5052]: I1203 12:11:56.147549 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/ba056586-628e-45da-852f-9fc1736ff76f-logs\") pod \"barbican-api-649f8bf5b8-wjzcq\" (UID: \"ba056586-628e-45da-852f-9fc1736ff76f\") " pod="openstack-kuttl-tests/barbican-api-649f8bf5b8-wjzcq" Dec 03 12:11:56 crc kubenswrapper[5052]: I1203 12:11:56.147589 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/ba056586-628e-45da-852f-9fc1736ff76f-config-data\") pod \"barbican-api-649f8bf5b8-wjzcq\" (UID: \"ba056586-628e-45da-852f-9fc1736ff76f\") " pod="openstack-kuttl-tests/barbican-api-649f8bf5b8-wjzcq" Dec 03 12:11:56 crc kubenswrapper[5052]: I1203 12:11:56.147617 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/296a2858-22b8-41a4-91ac-a9bf055b2848-config-data\") pod \"barbican-keystone-listener-85f96c8f8-b6m5f\" (UID: \"296a2858-22b8-41a4-91ac-a9bf055b2848\") " pod="openstack-kuttl-tests/barbican-keystone-listener-85f96c8f8-b6m5f" Dec 03 12:11:56 crc kubenswrapper[5052]: I1203 12:11:56.147639 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/296a2858-22b8-41a4-91ac-a9bf055b2848-combined-ca-bundle\") pod \"barbican-keystone-listener-85f96c8f8-b6m5f\" (UID: \"296a2858-22b8-41a4-91ac-a9bf055b2848\") " pod="openstack-kuttl-tests/barbican-keystone-listener-85f96c8f8-b6m5f" Dec 03 12:11:56 crc kubenswrapper[5052]: I1203 12:11:56.147672 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/296a2858-22b8-41a4-91ac-a9bf055b2848-logs\") pod \"barbican-keystone-listener-85f96c8f8-b6m5f\" (UID: \"296a2858-22b8-41a4-91ac-a9bf055b2848\") " pod="openstack-kuttl-tests/barbican-keystone-listener-85f96c8f8-b6m5f" Dec 03 12:11:56 crc kubenswrapper[5052]: I1203 12:11:56.147697 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xbp6p\" (UniqueName: \"kubernetes.io/projected/ba056586-628e-45da-852f-9fc1736ff76f-kube-api-access-xbp6p\") pod \"barbican-api-649f8bf5b8-wjzcq\" (UID: \"ba056586-628e-45da-852f-9fc1736ff76f\") " pod="openstack-kuttl-tests/barbican-api-649f8bf5b8-wjzcq" Dec 03 12:11:56 crc kubenswrapper[5052]: I1203 12:11:56.147724 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/ba056586-628e-45da-852f-9fc1736ff76f-config-data-custom\") pod \"barbican-api-649f8bf5b8-wjzcq\" (UID: \"ba056586-628e-45da-852f-9fc1736ff76f\") " pod="openstack-kuttl-tests/barbican-api-649f8bf5b8-wjzcq" Dec 03 12:11:56 crc kubenswrapper[5052]: I1203 12:11:56.147757 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ba056586-628e-45da-852f-9fc1736ff76f-combined-ca-bundle\") pod \"barbican-api-649f8bf5b8-wjzcq\" (UID: \"ba056586-628e-45da-852f-9fc1736ff76f\") " pod="openstack-kuttl-tests/barbican-api-649f8bf5b8-wjzcq" Dec 03 12:11:56 crc kubenswrapper[5052]: I1203 12:11:56.153230 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/296a2858-22b8-41a4-91ac-a9bf055b2848-logs\") pod \"barbican-keystone-listener-85f96c8f8-b6m5f\" (UID: \"296a2858-22b8-41a4-91ac-a9bf055b2848\") " pod="openstack-kuttl-tests/barbican-keystone-listener-85f96c8f8-b6m5f" Dec 03 12:11:56 crc kubenswrapper[5052]: I1203 12:11:56.168699 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/296a2858-22b8-41a4-91ac-a9bf055b2848-config-data\") pod \"barbican-keystone-listener-85f96c8f8-b6m5f\" (UID: \"296a2858-22b8-41a4-91ac-a9bf055b2848\") " pod="openstack-kuttl-tests/barbican-keystone-listener-85f96c8f8-b6m5f" Dec 03 12:11:56 crc kubenswrapper[5052]: I1203 12:11:56.173114 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/296a2858-22b8-41a4-91ac-a9bf055b2848-config-data-custom\") pod \"barbican-keystone-listener-85f96c8f8-b6m5f\" (UID: \"296a2858-22b8-41a4-91ac-a9bf055b2848\") " pod="openstack-kuttl-tests/barbican-keystone-listener-85f96c8f8-b6m5f" Dec 03 12:11:56 crc kubenswrapper[5052]: I1203 12:11:56.175189 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/296a2858-22b8-41a4-91ac-a9bf055b2848-combined-ca-bundle\") pod \"barbican-keystone-listener-85f96c8f8-b6m5f\" (UID: \"296a2858-22b8-41a4-91ac-a9bf055b2848\") " pod="openstack-kuttl-tests/barbican-keystone-listener-85f96c8f8-b6m5f" Dec 03 12:11:56 crc kubenswrapper[5052]: I1203 12:11:56.176402 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-dhznr\" (UniqueName: \"kubernetes.io/projected/296a2858-22b8-41a4-91ac-a9bf055b2848-kube-api-access-dhznr\") pod \"barbican-keystone-listener-85f96c8f8-b6m5f\" (UID: \"296a2858-22b8-41a4-91ac-a9bf055b2848\") " pod="openstack-kuttl-tests/barbican-keystone-listener-85f96c8f8-b6m5f" Dec 03 12:11:56 crc kubenswrapper[5052]: I1203 12:11:56.201999 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-keystone-listener-85f96c8f8-b6m5f" Dec 03 12:11:56 crc kubenswrapper[5052]: I1203 12:11:56.250540 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-xbp6p\" (UniqueName: \"kubernetes.io/projected/ba056586-628e-45da-852f-9fc1736ff76f-kube-api-access-xbp6p\") pod \"barbican-api-649f8bf5b8-wjzcq\" (UID: \"ba056586-628e-45da-852f-9fc1736ff76f\") " pod="openstack-kuttl-tests/barbican-api-649f8bf5b8-wjzcq" Dec 03 12:11:56 crc kubenswrapper[5052]: I1203 12:11:56.250612 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/ba056586-628e-45da-852f-9fc1736ff76f-config-data-custom\") pod \"barbican-api-649f8bf5b8-wjzcq\" (UID: \"ba056586-628e-45da-852f-9fc1736ff76f\") " pod="openstack-kuttl-tests/barbican-api-649f8bf5b8-wjzcq" Dec 03 12:11:56 crc kubenswrapper[5052]: I1203 12:11:56.250648 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ba056586-628e-45da-852f-9fc1736ff76f-combined-ca-bundle\") pod \"barbican-api-649f8bf5b8-wjzcq\" (UID: \"ba056586-628e-45da-852f-9fc1736ff76f\") " pod="openstack-kuttl-tests/barbican-api-649f8bf5b8-wjzcq" Dec 03 12:11:56 crc kubenswrapper[5052]: I1203 12:11:56.250720 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/ba056586-628e-45da-852f-9fc1736ff76f-logs\") pod \"barbican-api-649f8bf5b8-wjzcq\" (UID: \"ba056586-628e-45da-852f-9fc1736ff76f\") " pod="openstack-kuttl-tests/barbican-api-649f8bf5b8-wjzcq" Dec 03 12:11:56 crc kubenswrapper[5052]: I1203 12:11:56.250745 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/ba056586-628e-45da-852f-9fc1736ff76f-config-data\") pod \"barbican-api-649f8bf5b8-wjzcq\" (UID: \"ba056586-628e-45da-852f-9fc1736ff76f\") " pod="openstack-kuttl-tests/barbican-api-649f8bf5b8-wjzcq" Dec 03 12:11:56 crc kubenswrapper[5052]: I1203 12:11:56.261816 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/ba056586-628e-45da-852f-9fc1736ff76f-config-data\") pod \"barbican-api-649f8bf5b8-wjzcq\" (UID: \"ba056586-628e-45da-852f-9fc1736ff76f\") " pod="openstack-kuttl-tests/barbican-api-649f8bf5b8-wjzcq" Dec 03 12:11:56 crc kubenswrapper[5052]: I1203 12:11:56.267474 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/ba056586-628e-45da-852f-9fc1736ff76f-logs\") pod \"barbican-api-649f8bf5b8-wjzcq\" (UID: \"ba056586-628e-45da-852f-9fc1736ff76f\") " pod="openstack-kuttl-tests/barbican-api-649f8bf5b8-wjzcq" Dec 03 12:11:56 crc kubenswrapper[5052]: I1203 12:11:56.271327 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/ba056586-628e-45da-852f-9fc1736ff76f-config-data-custom\") pod \"barbican-api-649f8bf5b8-wjzcq\" (UID: \"ba056586-628e-45da-852f-9fc1736ff76f\") " pod="openstack-kuttl-tests/barbican-api-649f8bf5b8-wjzcq" Dec 03 12:11:56 crc kubenswrapper[5052]: I1203 12:11:56.288193 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ba056586-628e-45da-852f-9fc1736ff76f-combined-ca-bundle\") pod \"barbican-api-649f8bf5b8-wjzcq\" (UID: \"ba056586-628e-45da-852f-9fc1736ff76f\") " pod="openstack-kuttl-tests/barbican-api-649f8bf5b8-wjzcq" Dec 03 12:11:56 crc kubenswrapper[5052]: I1203 12:11:56.295173 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-xbp6p\" (UniqueName: \"kubernetes.io/projected/ba056586-628e-45da-852f-9fc1736ff76f-kube-api-access-xbp6p\") pod \"barbican-api-649f8bf5b8-wjzcq\" (UID: \"ba056586-628e-45da-852f-9fc1736ff76f\") " pod="openstack-kuttl-tests/barbican-api-649f8bf5b8-wjzcq" Dec 03 12:11:56 crc kubenswrapper[5052]: I1203 12:11:56.572021 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-api-649f8bf5b8-wjzcq" Dec 03 12:11:56 crc kubenswrapper[5052]: I1203 12:11:56.832851 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-worker-6ff59c8c65-fsb2k"] Dec 03 12:11:56 crc kubenswrapper[5052]: I1203 12:11:56.913462 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-keystone-listener-85f96c8f8-b6m5f"] Dec 03 12:11:57 crc kubenswrapper[5052]: I1203 12:11:57.123412 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-api-649f8bf5b8-wjzcq"] Dec 03 12:11:57 crc kubenswrapper[5052]: W1203 12:11:57.138959 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podba056586_628e_45da_852f_9fc1736ff76f.slice/crio-2a7e9168d6dbc722bd52ede7ce54deb4c1036cbd1faa23d6ca0f17287634c7de WatchSource:0}: Error finding container 2a7e9168d6dbc722bd52ede7ce54deb4c1036cbd1faa23d6ca0f17287634c7de: Status 404 returned error can't find the container with id 2a7e9168d6dbc722bd52ede7ce54deb4c1036cbd1faa23d6ca0f17287634c7de Dec 03 12:11:57 crc kubenswrapper[5052]: I1203 12:11:57.657317 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-649f8bf5b8-wjzcq" event={"ID":"ba056586-628e-45da-852f-9fc1736ff76f","Type":"ContainerStarted","Data":"2a7e9168d6dbc722bd52ede7ce54deb4c1036cbd1faa23d6ca0f17287634c7de"} Dec 03 12:11:57 crc kubenswrapper[5052]: I1203 12:11:57.659437 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-worker-6ff59c8c65-fsb2k" event={"ID":"1c98e0a9-267b-46b6-be02-fbd163462940","Type":"ContainerStarted","Data":"ba1ddda28f438ca657b63e17167786d2db03d16bf5ae3221146d8c6f9efc02d5"} Dec 03 12:11:57 crc kubenswrapper[5052]: I1203 12:11:57.659526 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-worker-6ff59c8c65-fsb2k" event={"ID":"1c98e0a9-267b-46b6-be02-fbd163462940","Type":"ContainerStarted","Data":"7fedb3be91342f96626a4b3d67a9a2fc8957182a74f1741a767f99ef8a035320"} Dec 03 12:11:57 crc kubenswrapper[5052]: I1203 12:11:57.660866 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-keystone-listener-85f96c8f8-b6m5f" event={"ID":"296a2858-22b8-41a4-91ac-a9bf055b2848","Type":"ContainerStarted","Data":"88ce6f1697038254e636269a53ace16da9bd7ec94e5543e34a4bab6f9c43e974"} Dec 03 12:11:57 crc kubenswrapper[5052]: I1203 12:11:57.660900 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-keystone-listener-85f96c8f8-b6m5f" event={"ID":"296a2858-22b8-41a4-91ac-a9bf055b2848","Type":"ContainerStarted","Data":"77e4cfb577e8baa1ba5da75e7dd402d3eca4274be0b1d163f90d407d7c6c40f9"} Dec 03 12:11:58 crc kubenswrapper[5052]: I1203 12:11:58.674452 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-worker-6ff59c8c65-fsb2k" event={"ID":"1c98e0a9-267b-46b6-be02-fbd163462940","Type":"ContainerStarted","Data":"7c74e4821c6d3bec61337629bd8bf43a58b069eaa5ca6ceb459795bf5d1ec7ae"} Dec 03 12:11:58 crc kubenswrapper[5052]: I1203 12:11:58.678531 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-keystone-listener-85f96c8f8-b6m5f" event={"ID":"296a2858-22b8-41a4-91ac-a9bf055b2848","Type":"ContainerStarted","Data":"dd65ec28df374ef84fa1a5f87d159edfa4265d92e582b66969d9f54e2878b585"} Dec 03 12:11:58 crc kubenswrapper[5052]: I1203 12:11:58.684337 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-649f8bf5b8-wjzcq" event={"ID":"ba056586-628e-45da-852f-9fc1736ff76f","Type":"ContainerStarted","Data":"7f3f8bf2361bf8c8c6620a120657d63671e3dde0dad82dcad1fe05618f40acbb"} Dec 03 12:11:58 crc kubenswrapper[5052]: I1203 12:11:58.715666 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/barbican-worker-6ff59c8c65-fsb2k" podStartSLOduration=3.71564295 podStartE2EDuration="3.71564295s" podCreationTimestamp="2025-12-03 12:11:55 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:11:58.694692141 +0000 UTC m=+3109.282933905" watchObservedRunningTime="2025-12-03 12:11:58.71564295 +0000 UTC m=+3109.303884714" Dec 03 12:11:58 crc kubenswrapper[5052]: I1203 12:11:58.723502 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/barbican-keystone-listener-85f96c8f8-b6m5f" podStartSLOduration=3.723478963 podStartE2EDuration="3.723478963s" podCreationTimestamp="2025-12-03 12:11:55 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:11:58.713966192 +0000 UTC m=+3109.302207966" watchObservedRunningTime="2025-12-03 12:11:58.723478963 +0000 UTC m=+3109.311720727" Dec 03 12:11:59 crc kubenswrapper[5052]: I1203 12:11:59.150532 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/barbican-api-58464649dd-sx8mw"] Dec 03 12:11:59 crc kubenswrapper[5052]: I1203 12:11:59.152386 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-api-58464649dd-sx8mw" Dec 03 12:11:59 crc kubenswrapper[5052]: I1203 12:11:59.155610 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-barbican-public-svc" Dec 03 12:11:59 crc kubenswrapper[5052]: I1203 12:11:59.156141 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-barbican-internal-svc" Dec 03 12:11:59 crc kubenswrapper[5052]: I1203 12:11:59.183727 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-api-58464649dd-sx8mw"] Dec 03 12:11:59 crc kubenswrapper[5052]: I1203 12:11:59.245306 5052 scope.go:117] "RemoveContainer" containerID="545a199ec97461f4a64454b6edecdfc68e50742c7e11692685b952917d9c8b1e" Dec 03 12:11:59 crc kubenswrapper[5052]: I1203 12:11:59.267931 5052 scope.go:117] "RemoveContainer" containerID="84283f0da62e2076f919bc27137ad503eb09cc0b4a9e5ff7119c2f488722a65c" Dec 03 12:11:59 crc kubenswrapper[5052]: I1203 12:11:59.287631 5052 scope.go:117] "RemoveContainer" containerID="95271173fa771abc9b96986bbbf1cb4713ab271b289e360271247bb505b8a425" Dec 03 12:11:59 crc kubenswrapper[5052]: I1203 12:11:59.328597 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/b25d2a0c-1e93-4b3d-965a-a7c021f9649d-config-data-custom\") pod \"barbican-api-58464649dd-sx8mw\" (UID: \"b25d2a0c-1e93-4b3d-965a-a7c021f9649d\") " pod="openstack-kuttl-tests/barbican-api-58464649dd-sx8mw" Dec 03 12:11:59 crc kubenswrapper[5052]: I1203 12:11:59.328689 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/b25d2a0c-1e93-4b3d-965a-a7c021f9649d-config-data\") pod \"barbican-api-58464649dd-sx8mw\" (UID: \"b25d2a0c-1e93-4b3d-965a-a7c021f9649d\") " pod="openstack-kuttl-tests/barbican-api-58464649dd-sx8mw" Dec 03 12:11:59 crc kubenswrapper[5052]: I1203 12:11:59.328723 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/b25d2a0c-1e93-4b3d-965a-a7c021f9649d-internal-tls-certs\") pod \"barbican-api-58464649dd-sx8mw\" (UID: \"b25d2a0c-1e93-4b3d-965a-a7c021f9649d\") " pod="openstack-kuttl-tests/barbican-api-58464649dd-sx8mw" Dec 03 12:11:59 crc kubenswrapper[5052]: I1203 12:11:59.328776 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/b25d2a0c-1e93-4b3d-965a-a7c021f9649d-logs\") pod \"barbican-api-58464649dd-sx8mw\" (UID: \"b25d2a0c-1e93-4b3d-965a-a7c021f9649d\") " pod="openstack-kuttl-tests/barbican-api-58464649dd-sx8mw" Dec 03 12:11:59 crc kubenswrapper[5052]: I1203 12:11:59.328830 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b25d2a0c-1e93-4b3d-965a-a7c021f9649d-combined-ca-bundle\") pod \"barbican-api-58464649dd-sx8mw\" (UID: \"b25d2a0c-1e93-4b3d-965a-a7c021f9649d\") " pod="openstack-kuttl-tests/barbican-api-58464649dd-sx8mw" Dec 03 12:11:59 crc kubenswrapper[5052]: I1203 12:11:59.328853 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/b25d2a0c-1e93-4b3d-965a-a7c021f9649d-public-tls-certs\") pod \"barbican-api-58464649dd-sx8mw\" (UID: \"b25d2a0c-1e93-4b3d-965a-a7c021f9649d\") " pod="openstack-kuttl-tests/barbican-api-58464649dd-sx8mw" Dec 03 12:11:59 crc kubenswrapper[5052]: I1203 12:11:59.328937 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r765c\" (UniqueName: \"kubernetes.io/projected/b25d2a0c-1e93-4b3d-965a-a7c021f9649d-kube-api-access-r765c\") pod \"barbican-api-58464649dd-sx8mw\" (UID: \"b25d2a0c-1e93-4b3d-965a-a7c021f9649d\") " pod="openstack-kuttl-tests/barbican-api-58464649dd-sx8mw" Dec 03 12:11:59 crc kubenswrapper[5052]: I1203 12:11:59.430249 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b25d2a0c-1e93-4b3d-965a-a7c021f9649d-combined-ca-bundle\") pod \"barbican-api-58464649dd-sx8mw\" (UID: \"b25d2a0c-1e93-4b3d-965a-a7c021f9649d\") " pod="openstack-kuttl-tests/barbican-api-58464649dd-sx8mw" Dec 03 12:11:59 crc kubenswrapper[5052]: I1203 12:11:59.430285 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/b25d2a0c-1e93-4b3d-965a-a7c021f9649d-public-tls-certs\") pod \"barbican-api-58464649dd-sx8mw\" (UID: \"b25d2a0c-1e93-4b3d-965a-a7c021f9649d\") " pod="openstack-kuttl-tests/barbican-api-58464649dd-sx8mw" Dec 03 12:11:59 crc kubenswrapper[5052]: I1203 12:11:59.430353 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-r765c\" (UniqueName: \"kubernetes.io/projected/b25d2a0c-1e93-4b3d-965a-a7c021f9649d-kube-api-access-r765c\") pod \"barbican-api-58464649dd-sx8mw\" (UID: \"b25d2a0c-1e93-4b3d-965a-a7c021f9649d\") " pod="openstack-kuttl-tests/barbican-api-58464649dd-sx8mw" Dec 03 12:11:59 crc kubenswrapper[5052]: I1203 12:11:59.430399 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/b25d2a0c-1e93-4b3d-965a-a7c021f9649d-config-data-custom\") pod \"barbican-api-58464649dd-sx8mw\" (UID: \"b25d2a0c-1e93-4b3d-965a-a7c021f9649d\") " pod="openstack-kuttl-tests/barbican-api-58464649dd-sx8mw" Dec 03 12:11:59 crc kubenswrapper[5052]: I1203 12:11:59.430433 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/b25d2a0c-1e93-4b3d-965a-a7c021f9649d-config-data\") pod \"barbican-api-58464649dd-sx8mw\" (UID: \"b25d2a0c-1e93-4b3d-965a-a7c021f9649d\") " pod="openstack-kuttl-tests/barbican-api-58464649dd-sx8mw" Dec 03 12:11:59 crc kubenswrapper[5052]: I1203 12:11:59.430455 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/b25d2a0c-1e93-4b3d-965a-a7c021f9649d-internal-tls-certs\") pod \"barbican-api-58464649dd-sx8mw\" (UID: \"b25d2a0c-1e93-4b3d-965a-a7c021f9649d\") " pod="openstack-kuttl-tests/barbican-api-58464649dd-sx8mw" Dec 03 12:11:59 crc kubenswrapper[5052]: I1203 12:11:59.430493 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/b25d2a0c-1e93-4b3d-965a-a7c021f9649d-logs\") pod \"barbican-api-58464649dd-sx8mw\" (UID: \"b25d2a0c-1e93-4b3d-965a-a7c021f9649d\") " pod="openstack-kuttl-tests/barbican-api-58464649dd-sx8mw" Dec 03 12:11:59 crc kubenswrapper[5052]: I1203 12:11:59.431172 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/b25d2a0c-1e93-4b3d-965a-a7c021f9649d-logs\") pod \"barbican-api-58464649dd-sx8mw\" (UID: \"b25d2a0c-1e93-4b3d-965a-a7c021f9649d\") " pod="openstack-kuttl-tests/barbican-api-58464649dd-sx8mw" Dec 03 12:11:59 crc kubenswrapper[5052]: I1203 12:11:59.435872 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/b25d2a0c-1e93-4b3d-965a-a7c021f9649d-config-data-custom\") pod \"barbican-api-58464649dd-sx8mw\" (UID: \"b25d2a0c-1e93-4b3d-965a-a7c021f9649d\") " pod="openstack-kuttl-tests/barbican-api-58464649dd-sx8mw" Dec 03 12:11:59 crc kubenswrapper[5052]: I1203 12:11:59.437140 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/b25d2a0c-1e93-4b3d-965a-a7c021f9649d-config-data\") pod \"barbican-api-58464649dd-sx8mw\" (UID: \"b25d2a0c-1e93-4b3d-965a-a7c021f9649d\") " pod="openstack-kuttl-tests/barbican-api-58464649dd-sx8mw" Dec 03 12:11:59 crc kubenswrapper[5052]: I1203 12:11:59.438384 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/b25d2a0c-1e93-4b3d-965a-a7c021f9649d-internal-tls-certs\") pod \"barbican-api-58464649dd-sx8mw\" (UID: \"b25d2a0c-1e93-4b3d-965a-a7c021f9649d\") " pod="openstack-kuttl-tests/barbican-api-58464649dd-sx8mw" Dec 03 12:11:59 crc kubenswrapper[5052]: I1203 12:11:59.439104 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/b25d2a0c-1e93-4b3d-965a-a7c021f9649d-public-tls-certs\") pod \"barbican-api-58464649dd-sx8mw\" (UID: \"b25d2a0c-1e93-4b3d-965a-a7c021f9649d\") " pod="openstack-kuttl-tests/barbican-api-58464649dd-sx8mw" Dec 03 12:11:59 crc kubenswrapper[5052]: I1203 12:11:59.456284 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b25d2a0c-1e93-4b3d-965a-a7c021f9649d-combined-ca-bundle\") pod \"barbican-api-58464649dd-sx8mw\" (UID: \"b25d2a0c-1e93-4b3d-965a-a7c021f9649d\") " pod="openstack-kuttl-tests/barbican-api-58464649dd-sx8mw" Dec 03 12:11:59 crc kubenswrapper[5052]: I1203 12:11:59.463968 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-r765c\" (UniqueName: \"kubernetes.io/projected/b25d2a0c-1e93-4b3d-965a-a7c021f9649d-kube-api-access-r765c\") pod \"barbican-api-58464649dd-sx8mw\" (UID: \"b25d2a0c-1e93-4b3d-965a-a7c021f9649d\") " pod="openstack-kuttl-tests/barbican-api-58464649dd-sx8mw" Dec 03 12:11:59 crc kubenswrapper[5052]: I1203 12:11:59.476355 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-api-58464649dd-sx8mw" Dec 03 12:11:59 crc kubenswrapper[5052]: I1203 12:11:59.714587 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-649f8bf5b8-wjzcq" event={"ID":"ba056586-628e-45da-852f-9fc1736ff76f","Type":"ContainerStarted","Data":"019542a1d362fb4a4e52a6614c1be3532669828ab5dd32337c547c9e0c866e5c"} Dec 03 12:11:59 crc kubenswrapper[5052]: I1203 12:11:59.715392 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/barbican-api-649f8bf5b8-wjzcq" Dec 03 12:11:59 crc kubenswrapper[5052]: I1203 12:11:59.715420 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/barbican-api-649f8bf5b8-wjzcq" Dec 03 12:11:59 crc kubenswrapper[5052]: I1203 12:11:59.759379 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/barbican-api-649f8bf5b8-wjzcq" podStartSLOduration=3.75936044 podStartE2EDuration="3.75936044s" podCreationTimestamp="2025-12-03 12:11:56 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:11:59.75094986 +0000 UTC m=+3110.339191634" watchObservedRunningTime="2025-12-03 12:11:59.75936044 +0000 UTC m=+3110.347602204" Dec 03 12:11:59 crc kubenswrapper[5052]: I1203 12:11:59.774701 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-api-58464649dd-sx8mw"] Dec 03 12:12:00 crc kubenswrapper[5052]: I1203 12:12:00.734368 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-58464649dd-sx8mw" event={"ID":"b25d2a0c-1e93-4b3d-965a-a7c021f9649d","Type":"ContainerStarted","Data":"d4bb4f5c0829cb278a445845849850d561fa98757159ecf40155346ec8896bde"} Dec 03 12:12:00 crc kubenswrapper[5052]: I1203 12:12:00.734743 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-58464649dd-sx8mw" event={"ID":"b25d2a0c-1e93-4b3d-965a-a7c021f9649d","Type":"ContainerStarted","Data":"6908da1783700654af60d79f711ce4360ed719f4f83031effe704c70de2ab47f"} Dec 03 12:12:00 crc kubenswrapper[5052]: I1203 12:12:00.734759 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-58464649dd-sx8mw" event={"ID":"b25d2a0c-1e93-4b3d-965a-a7c021f9649d","Type":"ContainerStarted","Data":"e63247ed7bd80b962fd6a6dd8426c057c8a6295d6246b0f68c805e0f3cb05e2a"} Dec 03 12:12:00 crc kubenswrapper[5052]: I1203 12:12:00.757311 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/barbican-api-58464649dd-sx8mw" podStartSLOduration=1.7572941050000002 podStartE2EDuration="1.757294105s" podCreationTimestamp="2025-12-03 12:11:59 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:12:00.755278807 +0000 UTC m=+3111.343520571" watchObservedRunningTime="2025-12-03 12:12:00.757294105 +0000 UTC m=+3111.345535869" Dec 03 12:12:01 crc kubenswrapper[5052]: I1203 12:12:01.742891 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/barbican-api-58464649dd-sx8mw" Dec 03 12:12:01 crc kubenswrapper[5052]: I1203 12:12:01.743752 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/barbican-api-58464649dd-sx8mw" Dec 03 12:12:03 crc kubenswrapper[5052]: I1203 12:12:03.685760 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:12:03 crc kubenswrapper[5052]: I1203 12:12:03.760589 5052 generic.go:334] "Generic (PLEG): container finished" podID="971c703f-d93b-4ae2-b8c2-4d9458793408" containerID="f360d4e646d5e381237bbb8eca54754e311f0f72353048f82b3866d760e09157" exitCode=0 Dec 03 12:12:03 crc kubenswrapper[5052]: I1203 12:12:03.760670 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-db-sync-tgpn8" event={"ID":"971c703f-d93b-4ae2-b8c2-4d9458793408","Type":"ContainerDied","Data":"f360d4e646d5e381237bbb8eca54754e311f0f72353048f82b3866d760e09157"} Dec 03 12:12:05 crc kubenswrapper[5052]: I1203 12:12:05.324651 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-db-sync-tgpn8" Dec 03 12:12:05 crc kubenswrapper[5052]: I1203 12:12:05.363989 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zsh28\" (UniqueName: \"kubernetes.io/projected/971c703f-d93b-4ae2-b8c2-4d9458793408-kube-api-access-zsh28\") pod \"971c703f-d93b-4ae2-b8c2-4d9458793408\" (UID: \"971c703f-d93b-4ae2-b8c2-4d9458793408\") " Dec 03 12:12:05 crc kubenswrapper[5052]: I1203 12:12:05.364321 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/971c703f-d93b-4ae2-b8c2-4d9458793408-config-data\") pod \"971c703f-d93b-4ae2-b8c2-4d9458793408\" (UID: \"971c703f-d93b-4ae2-b8c2-4d9458793408\") " Dec 03 12:12:05 crc kubenswrapper[5052]: I1203 12:12:05.364402 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/971c703f-d93b-4ae2-b8c2-4d9458793408-db-sync-config-data\") pod \"971c703f-d93b-4ae2-b8c2-4d9458793408\" (UID: \"971c703f-d93b-4ae2-b8c2-4d9458793408\") " Dec 03 12:12:05 crc kubenswrapper[5052]: I1203 12:12:05.364444 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/971c703f-d93b-4ae2-b8c2-4d9458793408-etc-machine-id\") pod \"971c703f-d93b-4ae2-b8c2-4d9458793408\" (UID: \"971c703f-d93b-4ae2-b8c2-4d9458793408\") " Dec 03 12:12:05 crc kubenswrapper[5052]: I1203 12:12:05.364497 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/971c703f-d93b-4ae2-b8c2-4d9458793408-scripts\") pod \"971c703f-d93b-4ae2-b8c2-4d9458793408\" (UID: \"971c703f-d93b-4ae2-b8c2-4d9458793408\") " Dec 03 12:12:05 crc kubenswrapper[5052]: I1203 12:12:05.364625 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/971c703f-d93b-4ae2-b8c2-4d9458793408-combined-ca-bundle\") pod \"971c703f-d93b-4ae2-b8c2-4d9458793408\" (UID: \"971c703f-d93b-4ae2-b8c2-4d9458793408\") " Dec 03 12:12:05 crc kubenswrapper[5052]: I1203 12:12:05.364652 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/971c703f-d93b-4ae2-b8c2-4d9458793408-etc-machine-id" (OuterVolumeSpecName: "etc-machine-id") pod "971c703f-d93b-4ae2-b8c2-4d9458793408" (UID: "971c703f-d93b-4ae2-b8c2-4d9458793408"). InnerVolumeSpecName "etc-machine-id". PluginName "kubernetes.io/host-path", VolumeGidValue "" Dec 03 12:12:05 crc kubenswrapper[5052]: I1203 12:12:05.365046 5052 reconciler_common.go:293] "Volume detached for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/971c703f-d93b-4ae2-b8c2-4d9458793408-etc-machine-id\") on node \"crc\" DevicePath \"\"" Dec 03 12:12:05 crc kubenswrapper[5052]: I1203 12:12:05.376278 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/971c703f-d93b-4ae2-b8c2-4d9458793408-db-sync-config-data" (OuterVolumeSpecName: "db-sync-config-data") pod "971c703f-d93b-4ae2-b8c2-4d9458793408" (UID: "971c703f-d93b-4ae2-b8c2-4d9458793408"). InnerVolumeSpecName "db-sync-config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:12:05 crc kubenswrapper[5052]: I1203 12:12:05.376791 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/971c703f-d93b-4ae2-b8c2-4d9458793408-scripts" (OuterVolumeSpecName: "scripts") pod "971c703f-d93b-4ae2-b8c2-4d9458793408" (UID: "971c703f-d93b-4ae2-b8c2-4d9458793408"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:12:05 crc kubenswrapper[5052]: I1203 12:12:05.377233 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/971c703f-d93b-4ae2-b8c2-4d9458793408-kube-api-access-zsh28" (OuterVolumeSpecName: "kube-api-access-zsh28") pod "971c703f-d93b-4ae2-b8c2-4d9458793408" (UID: "971c703f-d93b-4ae2-b8c2-4d9458793408"). InnerVolumeSpecName "kube-api-access-zsh28". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:12:05 crc kubenswrapper[5052]: I1203 12:12:05.392676 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/971c703f-d93b-4ae2-b8c2-4d9458793408-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "971c703f-d93b-4ae2-b8c2-4d9458793408" (UID: "971c703f-d93b-4ae2-b8c2-4d9458793408"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:12:05 crc kubenswrapper[5052]: I1203 12:12:05.427378 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/971c703f-d93b-4ae2-b8c2-4d9458793408-config-data" (OuterVolumeSpecName: "config-data") pod "971c703f-d93b-4ae2-b8c2-4d9458793408" (UID: "971c703f-d93b-4ae2-b8c2-4d9458793408"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:12:05 crc kubenswrapper[5052]: I1203 12:12:05.467584 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-zsh28\" (UniqueName: \"kubernetes.io/projected/971c703f-d93b-4ae2-b8c2-4d9458793408-kube-api-access-zsh28\") on node \"crc\" DevicePath \"\"" Dec 03 12:12:05 crc kubenswrapper[5052]: I1203 12:12:05.467664 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/971c703f-d93b-4ae2-b8c2-4d9458793408-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:12:05 crc kubenswrapper[5052]: I1203 12:12:05.467691 5052 reconciler_common.go:293] "Volume detached for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/971c703f-d93b-4ae2-b8c2-4d9458793408-db-sync-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:12:05 crc kubenswrapper[5052]: I1203 12:12:05.467714 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/971c703f-d93b-4ae2-b8c2-4d9458793408-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:12:05 crc kubenswrapper[5052]: I1203 12:12:05.467839 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/971c703f-d93b-4ae2-b8c2-4d9458793408-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:12:05 crc kubenswrapper[5052]: I1203 12:12:05.788366 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-db-sync-tgpn8" event={"ID":"971c703f-d93b-4ae2-b8c2-4d9458793408","Type":"ContainerDied","Data":"67ff46954f9ac5a8310a56741301b02c4c6fadf37c49dd47104cf64b1d951f9f"} Dec 03 12:12:05 crc kubenswrapper[5052]: I1203 12:12:05.788449 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="67ff46954f9ac5a8310a56741301b02c4c6fadf37c49dd47104cf64b1d951f9f" Dec 03 12:12:05 crc kubenswrapper[5052]: I1203 12:12:05.788607 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-db-sync-tgpn8" Dec 03 12:12:06 crc kubenswrapper[5052]: I1203 12:12:06.079273 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 12:12:06 crc kubenswrapper[5052]: E1203 12:12:06.079760 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="971c703f-d93b-4ae2-b8c2-4d9458793408" containerName="cinder-db-sync" Dec 03 12:12:06 crc kubenswrapper[5052]: I1203 12:12:06.079778 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="971c703f-d93b-4ae2-b8c2-4d9458793408" containerName="cinder-db-sync" Dec 03 12:12:06 crc kubenswrapper[5052]: I1203 12:12:06.079974 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="971c703f-d93b-4ae2-b8c2-4d9458793408" containerName="cinder-db-sync" Dec 03 12:12:06 crc kubenswrapper[5052]: I1203 12:12:06.080993 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:12:06 crc kubenswrapper[5052]: I1203 12:12:06.083688 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cinder-cinder-dockercfg-qc8wr" Dec 03 12:12:06 crc kubenswrapper[5052]: I1203 12:12:06.083928 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cinder-config-data" Dec 03 12:12:06 crc kubenswrapper[5052]: I1203 12:12:06.084658 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cinder-scripts" Dec 03 12:12:06 crc kubenswrapper[5052]: I1203 12:12:06.084892 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cinder-scheduler-config-data" Dec 03 12:12:06 crc kubenswrapper[5052]: I1203 12:12:06.095962 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 12:12:06 crc kubenswrapper[5052]: I1203 12:12:06.183375 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/63ba4f58-6d06-452e-86b7-54a61a998c03-combined-ca-bundle\") pod \"cinder-scheduler-0\" (UID: \"63ba4f58-6d06-452e-86b7-54a61a998c03\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:12:06 crc kubenswrapper[5052]: I1203 12:12:06.183453 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/63ba4f58-6d06-452e-86b7-54a61a998c03-config-data-custom\") pod \"cinder-scheduler-0\" (UID: \"63ba4f58-6d06-452e-86b7-54a61a998c03\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:12:06 crc kubenswrapper[5052]: I1203 12:12:06.183498 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/63ba4f58-6d06-452e-86b7-54a61a998c03-etc-machine-id\") pod \"cinder-scheduler-0\" (UID: \"63ba4f58-6d06-452e-86b7-54a61a998c03\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:12:06 crc kubenswrapper[5052]: I1203 12:12:06.183560 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/63ba4f58-6d06-452e-86b7-54a61a998c03-config-data\") pod \"cinder-scheduler-0\" (UID: \"63ba4f58-6d06-452e-86b7-54a61a998c03\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:12:06 crc kubenswrapper[5052]: I1203 12:12:06.183584 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/63ba4f58-6d06-452e-86b7-54a61a998c03-scripts\") pod \"cinder-scheduler-0\" (UID: \"63ba4f58-6d06-452e-86b7-54a61a998c03\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:12:06 crc kubenswrapper[5052]: I1203 12:12:06.183633 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j7zp2\" (UniqueName: \"kubernetes.io/projected/63ba4f58-6d06-452e-86b7-54a61a998c03-kube-api-access-j7zp2\") pod \"cinder-scheduler-0\" (UID: \"63ba4f58-6d06-452e-86b7-54a61a998c03\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:12:06 crc kubenswrapper[5052]: I1203 12:12:06.203581 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 12:12:06 crc kubenswrapper[5052]: I1203 12:12:06.205391 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:12:06 crc kubenswrapper[5052]: I1203 12:12:06.210838 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cinder-api-config-data" Dec 03 12:12:06 crc kubenswrapper[5052]: I1203 12:12:06.213380 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 12:12:06 crc kubenswrapper[5052]: I1203 12:12:06.284935 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/14c699e0-a2a5-4e95-a2b2-450d291e2a2a-etc-machine-id\") pod \"cinder-api-0\" (UID: \"14c699e0-a2a5-4e95-a2b2-450d291e2a2a\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:12:06 crc kubenswrapper[5052]: I1203 12:12:06.285008 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/14c699e0-a2a5-4e95-a2b2-450d291e2a2a-combined-ca-bundle\") pod \"cinder-api-0\" (UID: \"14c699e0-a2a5-4e95-a2b2-450d291e2a2a\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:12:06 crc kubenswrapper[5052]: I1203 12:12:06.285071 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/63ba4f58-6d06-452e-86b7-54a61a998c03-combined-ca-bundle\") pod \"cinder-scheduler-0\" (UID: \"63ba4f58-6d06-452e-86b7-54a61a998c03\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:12:06 crc kubenswrapper[5052]: I1203 12:12:06.285096 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/14c699e0-a2a5-4e95-a2b2-450d291e2a2a-scripts\") pod \"cinder-api-0\" (UID: \"14c699e0-a2a5-4e95-a2b2-450d291e2a2a\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:12:06 crc kubenswrapper[5052]: I1203 12:12:06.285146 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/63ba4f58-6d06-452e-86b7-54a61a998c03-config-data-custom\") pod \"cinder-scheduler-0\" (UID: \"63ba4f58-6d06-452e-86b7-54a61a998c03\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:12:06 crc kubenswrapper[5052]: I1203 12:12:06.285172 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/14c699e0-a2a5-4e95-a2b2-450d291e2a2a-config-data\") pod \"cinder-api-0\" (UID: \"14c699e0-a2a5-4e95-a2b2-450d291e2a2a\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:12:06 crc kubenswrapper[5052]: I1203 12:12:06.285212 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bvwlv\" (UniqueName: \"kubernetes.io/projected/14c699e0-a2a5-4e95-a2b2-450d291e2a2a-kube-api-access-bvwlv\") pod \"cinder-api-0\" (UID: \"14c699e0-a2a5-4e95-a2b2-450d291e2a2a\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:12:06 crc kubenswrapper[5052]: I1203 12:12:06.285239 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/63ba4f58-6d06-452e-86b7-54a61a998c03-etc-machine-id\") pod \"cinder-scheduler-0\" (UID: \"63ba4f58-6d06-452e-86b7-54a61a998c03\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:12:06 crc kubenswrapper[5052]: I1203 12:12:06.285282 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/63ba4f58-6d06-452e-86b7-54a61a998c03-config-data\") pod \"cinder-scheduler-0\" (UID: \"63ba4f58-6d06-452e-86b7-54a61a998c03\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:12:06 crc kubenswrapper[5052]: I1203 12:12:06.285321 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/63ba4f58-6d06-452e-86b7-54a61a998c03-scripts\") pod \"cinder-scheduler-0\" (UID: \"63ba4f58-6d06-452e-86b7-54a61a998c03\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:12:06 crc kubenswrapper[5052]: I1203 12:12:06.285355 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/63ba4f58-6d06-452e-86b7-54a61a998c03-etc-machine-id\") pod \"cinder-scheduler-0\" (UID: \"63ba4f58-6d06-452e-86b7-54a61a998c03\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:12:06 crc kubenswrapper[5052]: I1203 12:12:06.285365 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/14c699e0-a2a5-4e95-a2b2-450d291e2a2a-logs\") pod \"cinder-api-0\" (UID: \"14c699e0-a2a5-4e95-a2b2-450d291e2a2a\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:12:06 crc kubenswrapper[5052]: I1203 12:12:06.285429 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-j7zp2\" (UniqueName: \"kubernetes.io/projected/63ba4f58-6d06-452e-86b7-54a61a998c03-kube-api-access-j7zp2\") pod \"cinder-scheduler-0\" (UID: \"63ba4f58-6d06-452e-86b7-54a61a998c03\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:12:06 crc kubenswrapper[5052]: I1203 12:12:06.285453 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/14c699e0-a2a5-4e95-a2b2-450d291e2a2a-config-data-custom\") pod \"cinder-api-0\" (UID: \"14c699e0-a2a5-4e95-a2b2-450d291e2a2a\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:12:06 crc kubenswrapper[5052]: I1203 12:12:06.289223 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/63ba4f58-6d06-452e-86b7-54a61a998c03-config-data-custom\") pod \"cinder-scheduler-0\" (UID: \"63ba4f58-6d06-452e-86b7-54a61a998c03\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:12:06 crc kubenswrapper[5052]: I1203 12:12:06.289556 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/63ba4f58-6d06-452e-86b7-54a61a998c03-combined-ca-bundle\") pod \"cinder-scheduler-0\" (UID: \"63ba4f58-6d06-452e-86b7-54a61a998c03\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:12:06 crc kubenswrapper[5052]: I1203 12:12:06.294014 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/63ba4f58-6d06-452e-86b7-54a61a998c03-scripts\") pod \"cinder-scheduler-0\" (UID: \"63ba4f58-6d06-452e-86b7-54a61a998c03\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:12:06 crc kubenswrapper[5052]: I1203 12:12:06.305216 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/63ba4f58-6d06-452e-86b7-54a61a998c03-config-data\") pod \"cinder-scheduler-0\" (UID: \"63ba4f58-6d06-452e-86b7-54a61a998c03\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:12:06 crc kubenswrapper[5052]: I1203 12:12:06.305960 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-j7zp2\" (UniqueName: \"kubernetes.io/projected/63ba4f58-6d06-452e-86b7-54a61a998c03-kube-api-access-j7zp2\") pod \"cinder-scheduler-0\" (UID: \"63ba4f58-6d06-452e-86b7-54a61a998c03\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:12:06 crc kubenswrapper[5052]: I1203 12:12:06.387549 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/14c699e0-a2a5-4e95-a2b2-450d291e2a2a-config-data\") pod \"cinder-api-0\" (UID: \"14c699e0-a2a5-4e95-a2b2-450d291e2a2a\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:12:06 crc kubenswrapper[5052]: I1203 12:12:06.387630 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-bvwlv\" (UniqueName: \"kubernetes.io/projected/14c699e0-a2a5-4e95-a2b2-450d291e2a2a-kube-api-access-bvwlv\") pod \"cinder-api-0\" (UID: \"14c699e0-a2a5-4e95-a2b2-450d291e2a2a\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:12:06 crc kubenswrapper[5052]: I1203 12:12:06.387712 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/14c699e0-a2a5-4e95-a2b2-450d291e2a2a-logs\") pod \"cinder-api-0\" (UID: \"14c699e0-a2a5-4e95-a2b2-450d291e2a2a\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:12:06 crc kubenswrapper[5052]: I1203 12:12:06.387743 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/14c699e0-a2a5-4e95-a2b2-450d291e2a2a-config-data-custom\") pod \"cinder-api-0\" (UID: \"14c699e0-a2a5-4e95-a2b2-450d291e2a2a\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:12:06 crc kubenswrapper[5052]: I1203 12:12:06.387801 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/14c699e0-a2a5-4e95-a2b2-450d291e2a2a-etc-machine-id\") pod \"cinder-api-0\" (UID: \"14c699e0-a2a5-4e95-a2b2-450d291e2a2a\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:12:06 crc kubenswrapper[5052]: I1203 12:12:06.387850 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/14c699e0-a2a5-4e95-a2b2-450d291e2a2a-combined-ca-bundle\") pod \"cinder-api-0\" (UID: \"14c699e0-a2a5-4e95-a2b2-450d291e2a2a\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:12:06 crc kubenswrapper[5052]: I1203 12:12:06.387906 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/14c699e0-a2a5-4e95-a2b2-450d291e2a2a-scripts\") pod \"cinder-api-0\" (UID: \"14c699e0-a2a5-4e95-a2b2-450d291e2a2a\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:12:06 crc kubenswrapper[5052]: I1203 12:12:06.388683 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/14c699e0-a2a5-4e95-a2b2-450d291e2a2a-etc-machine-id\") pod \"cinder-api-0\" (UID: \"14c699e0-a2a5-4e95-a2b2-450d291e2a2a\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:12:06 crc kubenswrapper[5052]: I1203 12:12:06.389079 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/14c699e0-a2a5-4e95-a2b2-450d291e2a2a-logs\") pod \"cinder-api-0\" (UID: \"14c699e0-a2a5-4e95-a2b2-450d291e2a2a\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:12:06 crc kubenswrapper[5052]: I1203 12:12:06.391893 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/14c699e0-a2a5-4e95-a2b2-450d291e2a2a-scripts\") pod \"cinder-api-0\" (UID: \"14c699e0-a2a5-4e95-a2b2-450d291e2a2a\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:12:06 crc kubenswrapper[5052]: I1203 12:12:06.391957 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/14c699e0-a2a5-4e95-a2b2-450d291e2a2a-config-data-custom\") pod \"cinder-api-0\" (UID: \"14c699e0-a2a5-4e95-a2b2-450d291e2a2a\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:12:06 crc kubenswrapper[5052]: I1203 12:12:06.398019 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:12:06 crc kubenswrapper[5052]: I1203 12:12:06.398270 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/14c699e0-a2a5-4e95-a2b2-450d291e2a2a-combined-ca-bundle\") pod \"cinder-api-0\" (UID: \"14c699e0-a2a5-4e95-a2b2-450d291e2a2a\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:12:06 crc kubenswrapper[5052]: I1203 12:12:06.398658 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/14c699e0-a2a5-4e95-a2b2-450d291e2a2a-config-data\") pod \"cinder-api-0\" (UID: \"14c699e0-a2a5-4e95-a2b2-450d291e2a2a\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:12:06 crc kubenswrapper[5052]: I1203 12:12:06.406503 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-bvwlv\" (UniqueName: \"kubernetes.io/projected/14c699e0-a2a5-4e95-a2b2-450d291e2a2a-kube-api-access-bvwlv\") pod \"cinder-api-0\" (UID: \"14c699e0-a2a5-4e95-a2b2-450d291e2a2a\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:12:06 crc kubenswrapper[5052]: I1203 12:12:06.522693 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:12:06 crc kubenswrapper[5052]: W1203 12:12:06.921460 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod63ba4f58_6d06_452e_86b7_54a61a998c03.slice/crio-ee37db74a901acb1bf8dbf41bbb26a7caa8487a8005ee0d96c5dcdfc997cd020 WatchSource:0}: Error finding container ee37db74a901acb1bf8dbf41bbb26a7caa8487a8005ee0d96c5dcdfc997cd020: Status 404 returned error can't find the container with id ee37db74a901acb1bf8dbf41bbb26a7caa8487a8005ee0d96c5dcdfc997cd020 Dec 03 12:12:06 crc kubenswrapper[5052]: I1203 12:12:06.923679 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 12:12:07 crc kubenswrapper[5052]: I1203 12:12:07.074097 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 12:12:07 crc kubenswrapper[5052]: W1203 12:12:07.094907 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod14c699e0_a2a5_4e95_a2b2_450d291e2a2a.slice/crio-c7254ea92b6fcf8df2a2035fc71afff5478666bae7831c6b3f1bd20d2d3fb692 WatchSource:0}: Error finding container c7254ea92b6fcf8df2a2035fc71afff5478666bae7831c6b3f1bd20d2d3fb692: Status 404 returned error can't find the container with id c7254ea92b6fcf8df2a2035fc71afff5478666bae7831c6b3f1bd20d2d3fb692 Dec 03 12:12:07 crc kubenswrapper[5052]: I1203 12:12:07.674964 5052 scope.go:117] "RemoveContainer" containerID="63298107b3d090deff9ed92f3d613e5b1c86d9137684d117500257db8ee1c805" Dec 03 12:12:07 crc kubenswrapper[5052]: E1203 12:12:07.675960 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 12:12:07 crc kubenswrapper[5052]: I1203 12:12:07.774564 5052 prober.go:107] "Probe failed" probeType="Liveness" pod="openstack-kuttl-tests/barbican-api-58464649dd-sx8mw" podUID="b25d2a0c-1e93-4b3d-965a-a7c021f9649d" containerName="barbican-api" probeResult="failure" output="HTTP probe failed with statuscode: 500" Dec 03 12:12:07 crc kubenswrapper[5052]: I1203 12:12:07.813870 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"63ba4f58-6d06-452e-86b7-54a61a998c03","Type":"ContainerStarted","Data":"ee37db74a901acb1bf8dbf41bbb26a7caa8487a8005ee0d96c5dcdfc997cd020"} Dec 03 12:12:07 crc kubenswrapper[5052]: I1203 12:12:07.816342 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"14c699e0-a2a5-4e95-a2b2-450d291e2a2a","Type":"ContainerStarted","Data":"c7254ea92b6fcf8df2a2035fc71afff5478666bae7831c6b3f1bd20d2d3fb692"} Dec 03 12:12:08 crc kubenswrapper[5052]: I1203 12:12:08.355153 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/barbican-api-649f8bf5b8-wjzcq" Dec 03 12:12:08 crc kubenswrapper[5052]: I1203 12:12:08.574479 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/barbican-api-649f8bf5b8-wjzcq" Dec 03 12:12:08 crc kubenswrapper[5052]: I1203 12:12:08.892593 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"14c699e0-a2a5-4e95-a2b2-450d291e2a2a","Type":"ContainerStarted","Data":"26a40626d678fa90af391f42ad231bd39cebad7cd8f28a73b6c879a0cf104442"} Dec 03 12:12:08 crc kubenswrapper[5052]: I1203 12:12:08.919621 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"63ba4f58-6d06-452e-86b7-54a61a998c03","Type":"ContainerStarted","Data":"4cb6e9620ad9bca18eb34b849b06c51b05cc9bee37b75d985fde32193b027d50"} Dec 03 12:12:09 crc kubenswrapper[5052]: I1203 12:12:09.073757 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 12:12:09 crc kubenswrapper[5052]: I1203 12:12:09.929342 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"63ba4f58-6d06-452e-86b7-54a61a998c03","Type":"ContainerStarted","Data":"2a7d4aa75fed7462d0c0db261e38e723cb83192e669ec7409e8c399c0ea325ee"} Dec 03 12:12:09 crc kubenswrapper[5052]: I1203 12:12:09.936647 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"14c699e0-a2a5-4e95-a2b2-450d291e2a2a","Type":"ContainerStarted","Data":"f3074002aaeec5863c2e5ff033e83c4d0d9cf44fd549bbfe3139618b09748e91"} Dec 03 12:12:09 crc kubenswrapper[5052]: I1203 12:12:09.936800 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/cinder-api-0" podUID="14c699e0-a2a5-4e95-a2b2-450d291e2a2a" containerName="cinder-api-log" containerID="cri-o://26a40626d678fa90af391f42ad231bd39cebad7cd8f28a73b6c879a0cf104442" gracePeriod=30 Dec 03 12:12:09 crc kubenswrapper[5052]: I1203 12:12:09.937090 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:12:09 crc kubenswrapper[5052]: I1203 12:12:09.937146 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/cinder-api-0" podUID="14c699e0-a2a5-4e95-a2b2-450d291e2a2a" containerName="cinder-api" containerID="cri-o://f3074002aaeec5863c2e5ff033e83c4d0d9cf44fd549bbfe3139618b09748e91" gracePeriod=30 Dec 03 12:12:10 crc kubenswrapper[5052]: I1203 12:12:10.002385 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/cinder-scheduler-0" podStartSLOduration=4.002361759 podStartE2EDuration="4.002361759s" podCreationTimestamp="2025-12-03 12:12:06 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:12:09.994890406 +0000 UTC m=+3120.583132160" watchObservedRunningTime="2025-12-03 12:12:10.002361759 +0000 UTC m=+3120.590603533" Dec 03 12:12:10 crc kubenswrapper[5052]: I1203 12:12:10.017752 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/cinder-api-0" podStartSLOduration=4.017728518 podStartE2EDuration="4.017728518s" podCreationTimestamp="2025-12-03 12:12:06 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:12:10.017379298 +0000 UTC m=+3120.605621062" watchObservedRunningTime="2025-12-03 12:12:10.017728518 +0000 UTC m=+3120.605970272" Dec 03 12:12:10 crc kubenswrapper[5052]: I1203 12:12:10.797092 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:12:10 crc kubenswrapper[5052]: I1203 12:12:10.938034 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/14c699e0-a2a5-4e95-a2b2-450d291e2a2a-logs\") pod \"14c699e0-a2a5-4e95-a2b2-450d291e2a2a\" (UID: \"14c699e0-a2a5-4e95-a2b2-450d291e2a2a\") " Dec 03 12:12:10 crc kubenswrapper[5052]: I1203 12:12:10.938213 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/14c699e0-a2a5-4e95-a2b2-450d291e2a2a-scripts\") pod \"14c699e0-a2a5-4e95-a2b2-450d291e2a2a\" (UID: \"14c699e0-a2a5-4e95-a2b2-450d291e2a2a\") " Dec 03 12:12:10 crc kubenswrapper[5052]: I1203 12:12:10.938243 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/14c699e0-a2a5-4e95-a2b2-450d291e2a2a-config-data\") pod \"14c699e0-a2a5-4e95-a2b2-450d291e2a2a\" (UID: \"14c699e0-a2a5-4e95-a2b2-450d291e2a2a\") " Dec 03 12:12:10 crc kubenswrapper[5052]: I1203 12:12:10.938293 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/14c699e0-a2a5-4e95-a2b2-450d291e2a2a-combined-ca-bundle\") pod \"14c699e0-a2a5-4e95-a2b2-450d291e2a2a\" (UID: \"14c699e0-a2a5-4e95-a2b2-450d291e2a2a\") " Dec 03 12:12:10 crc kubenswrapper[5052]: I1203 12:12:10.938317 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/14c699e0-a2a5-4e95-a2b2-450d291e2a2a-etc-machine-id\") pod \"14c699e0-a2a5-4e95-a2b2-450d291e2a2a\" (UID: \"14c699e0-a2a5-4e95-a2b2-450d291e2a2a\") " Dec 03 12:12:10 crc kubenswrapper[5052]: I1203 12:12:10.938341 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bvwlv\" (UniqueName: \"kubernetes.io/projected/14c699e0-a2a5-4e95-a2b2-450d291e2a2a-kube-api-access-bvwlv\") pod \"14c699e0-a2a5-4e95-a2b2-450d291e2a2a\" (UID: \"14c699e0-a2a5-4e95-a2b2-450d291e2a2a\") " Dec 03 12:12:10 crc kubenswrapper[5052]: I1203 12:12:10.938369 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/14c699e0-a2a5-4e95-a2b2-450d291e2a2a-config-data-custom\") pod \"14c699e0-a2a5-4e95-a2b2-450d291e2a2a\" (UID: \"14c699e0-a2a5-4e95-a2b2-450d291e2a2a\") " Dec 03 12:12:10 crc kubenswrapper[5052]: I1203 12:12:10.938455 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/14c699e0-a2a5-4e95-a2b2-450d291e2a2a-logs" (OuterVolumeSpecName: "logs") pod "14c699e0-a2a5-4e95-a2b2-450d291e2a2a" (UID: "14c699e0-a2a5-4e95-a2b2-450d291e2a2a"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:12:10 crc kubenswrapper[5052]: I1203 12:12:10.938821 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/14c699e0-a2a5-4e95-a2b2-450d291e2a2a-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:12:10 crc kubenswrapper[5052]: I1203 12:12:10.938899 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/14c699e0-a2a5-4e95-a2b2-450d291e2a2a-etc-machine-id" (OuterVolumeSpecName: "etc-machine-id") pod "14c699e0-a2a5-4e95-a2b2-450d291e2a2a" (UID: "14c699e0-a2a5-4e95-a2b2-450d291e2a2a"). InnerVolumeSpecName "etc-machine-id". PluginName "kubernetes.io/host-path", VolumeGidValue "" Dec 03 12:12:10 crc kubenswrapper[5052]: I1203 12:12:10.944784 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/14c699e0-a2a5-4e95-a2b2-450d291e2a2a-kube-api-access-bvwlv" (OuterVolumeSpecName: "kube-api-access-bvwlv") pod "14c699e0-a2a5-4e95-a2b2-450d291e2a2a" (UID: "14c699e0-a2a5-4e95-a2b2-450d291e2a2a"). InnerVolumeSpecName "kube-api-access-bvwlv". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:12:10 crc kubenswrapper[5052]: I1203 12:12:10.946590 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/14c699e0-a2a5-4e95-a2b2-450d291e2a2a-config-data-custom" (OuterVolumeSpecName: "config-data-custom") pod "14c699e0-a2a5-4e95-a2b2-450d291e2a2a" (UID: "14c699e0-a2a5-4e95-a2b2-450d291e2a2a"). InnerVolumeSpecName "config-data-custom". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:12:10 crc kubenswrapper[5052]: I1203 12:12:10.959970 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/14c699e0-a2a5-4e95-a2b2-450d291e2a2a-scripts" (OuterVolumeSpecName: "scripts") pod "14c699e0-a2a5-4e95-a2b2-450d291e2a2a" (UID: "14c699e0-a2a5-4e95-a2b2-450d291e2a2a"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:12:10 crc kubenswrapper[5052]: I1203 12:12:10.960266 5052 generic.go:334] "Generic (PLEG): container finished" podID="14c699e0-a2a5-4e95-a2b2-450d291e2a2a" containerID="f3074002aaeec5863c2e5ff033e83c4d0d9cf44fd549bbfe3139618b09748e91" exitCode=0 Dec 03 12:12:10 crc kubenswrapper[5052]: I1203 12:12:10.960307 5052 generic.go:334] "Generic (PLEG): container finished" podID="14c699e0-a2a5-4e95-a2b2-450d291e2a2a" containerID="26a40626d678fa90af391f42ad231bd39cebad7cd8f28a73b6c879a0cf104442" exitCode=143 Dec 03 12:12:10 crc kubenswrapper[5052]: I1203 12:12:10.960619 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"14c699e0-a2a5-4e95-a2b2-450d291e2a2a","Type":"ContainerDied","Data":"f3074002aaeec5863c2e5ff033e83c4d0d9cf44fd549bbfe3139618b09748e91"} Dec 03 12:12:10 crc kubenswrapper[5052]: I1203 12:12:10.960670 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"14c699e0-a2a5-4e95-a2b2-450d291e2a2a","Type":"ContainerDied","Data":"26a40626d678fa90af391f42ad231bd39cebad7cd8f28a73b6c879a0cf104442"} Dec 03 12:12:10 crc kubenswrapper[5052]: I1203 12:12:10.960669 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:12:10 crc kubenswrapper[5052]: I1203 12:12:10.960693 5052 scope.go:117] "RemoveContainer" containerID="f3074002aaeec5863c2e5ff033e83c4d0d9cf44fd549bbfe3139618b09748e91" Dec 03 12:12:10 crc kubenswrapper[5052]: I1203 12:12:10.960681 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"14c699e0-a2a5-4e95-a2b2-450d291e2a2a","Type":"ContainerDied","Data":"c7254ea92b6fcf8df2a2035fc71afff5478666bae7831c6b3f1bd20d2d3fb692"} Dec 03 12:12:10 crc kubenswrapper[5052]: I1203 12:12:10.988033 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/14c699e0-a2a5-4e95-a2b2-450d291e2a2a-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "14c699e0-a2a5-4e95-a2b2-450d291e2a2a" (UID: "14c699e0-a2a5-4e95-a2b2-450d291e2a2a"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:12:11 crc kubenswrapper[5052]: I1203 12:12:11.020090 5052 scope.go:117] "RemoveContainer" containerID="26a40626d678fa90af391f42ad231bd39cebad7cd8f28a73b6c879a0cf104442" Dec 03 12:12:11 crc kubenswrapper[5052]: I1203 12:12:11.041219 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/14c699e0-a2a5-4e95-a2b2-450d291e2a2a-config-data" (OuterVolumeSpecName: "config-data") pod "14c699e0-a2a5-4e95-a2b2-450d291e2a2a" (UID: "14c699e0-a2a5-4e95-a2b2-450d291e2a2a"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:12:11 crc kubenswrapper[5052]: I1203 12:12:11.042241 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/14c699e0-a2a5-4e95-a2b2-450d291e2a2a-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:12:11 crc kubenswrapper[5052]: I1203 12:12:11.042271 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/14c699e0-a2a5-4e95-a2b2-450d291e2a2a-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:12:11 crc kubenswrapper[5052]: I1203 12:12:11.042280 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/14c699e0-a2a5-4e95-a2b2-450d291e2a2a-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:12:11 crc kubenswrapper[5052]: I1203 12:12:11.042291 5052 reconciler_common.go:293] "Volume detached for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/14c699e0-a2a5-4e95-a2b2-450d291e2a2a-etc-machine-id\") on node \"crc\" DevicePath \"\"" Dec 03 12:12:11 crc kubenswrapper[5052]: I1203 12:12:11.042301 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-bvwlv\" (UniqueName: \"kubernetes.io/projected/14c699e0-a2a5-4e95-a2b2-450d291e2a2a-kube-api-access-bvwlv\") on node \"crc\" DevicePath \"\"" Dec 03 12:12:11 crc kubenswrapper[5052]: I1203 12:12:11.042308 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/14c699e0-a2a5-4e95-a2b2-450d291e2a2a-config-data-custom\") on node \"crc\" DevicePath \"\"" Dec 03 12:12:11 crc kubenswrapper[5052]: I1203 12:12:11.081708 5052 scope.go:117] "RemoveContainer" containerID="f3074002aaeec5863c2e5ff033e83c4d0d9cf44fd549bbfe3139618b09748e91" Dec 03 12:12:11 crc kubenswrapper[5052]: E1203 12:12:11.085608 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f3074002aaeec5863c2e5ff033e83c4d0d9cf44fd549bbfe3139618b09748e91\": container with ID starting with f3074002aaeec5863c2e5ff033e83c4d0d9cf44fd549bbfe3139618b09748e91 not found: ID does not exist" containerID="f3074002aaeec5863c2e5ff033e83c4d0d9cf44fd549bbfe3139618b09748e91" Dec 03 12:12:11 crc kubenswrapper[5052]: I1203 12:12:11.085642 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f3074002aaeec5863c2e5ff033e83c4d0d9cf44fd549bbfe3139618b09748e91"} err="failed to get container status \"f3074002aaeec5863c2e5ff033e83c4d0d9cf44fd549bbfe3139618b09748e91\": rpc error: code = NotFound desc = could not find container \"f3074002aaeec5863c2e5ff033e83c4d0d9cf44fd549bbfe3139618b09748e91\": container with ID starting with f3074002aaeec5863c2e5ff033e83c4d0d9cf44fd549bbfe3139618b09748e91 not found: ID does not exist" Dec 03 12:12:11 crc kubenswrapper[5052]: I1203 12:12:11.085663 5052 scope.go:117] "RemoveContainer" containerID="26a40626d678fa90af391f42ad231bd39cebad7cd8f28a73b6c879a0cf104442" Dec 03 12:12:11 crc kubenswrapper[5052]: E1203 12:12:11.088678 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"26a40626d678fa90af391f42ad231bd39cebad7cd8f28a73b6c879a0cf104442\": container with ID starting with 26a40626d678fa90af391f42ad231bd39cebad7cd8f28a73b6c879a0cf104442 not found: ID does not exist" containerID="26a40626d678fa90af391f42ad231bd39cebad7cd8f28a73b6c879a0cf104442" Dec 03 12:12:11 crc kubenswrapper[5052]: I1203 12:12:11.088715 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"26a40626d678fa90af391f42ad231bd39cebad7cd8f28a73b6c879a0cf104442"} err="failed to get container status \"26a40626d678fa90af391f42ad231bd39cebad7cd8f28a73b6c879a0cf104442\": rpc error: code = NotFound desc = could not find container \"26a40626d678fa90af391f42ad231bd39cebad7cd8f28a73b6c879a0cf104442\": container with ID starting with 26a40626d678fa90af391f42ad231bd39cebad7cd8f28a73b6c879a0cf104442 not found: ID does not exist" Dec 03 12:12:11 crc kubenswrapper[5052]: I1203 12:12:11.088740 5052 scope.go:117] "RemoveContainer" containerID="f3074002aaeec5863c2e5ff033e83c4d0d9cf44fd549bbfe3139618b09748e91" Dec 03 12:12:11 crc kubenswrapper[5052]: I1203 12:12:11.089071 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f3074002aaeec5863c2e5ff033e83c4d0d9cf44fd549bbfe3139618b09748e91"} err="failed to get container status \"f3074002aaeec5863c2e5ff033e83c4d0d9cf44fd549bbfe3139618b09748e91\": rpc error: code = NotFound desc = could not find container \"f3074002aaeec5863c2e5ff033e83c4d0d9cf44fd549bbfe3139618b09748e91\": container with ID starting with f3074002aaeec5863c2e5ff033e83c4d0d9cf44fd549bbfe3139618b09748e91 not found: ID does not exist" Dec 03 12:12:11 crc kubenswrapper[5052]: I1203 12:12:11.089092 5052 scope.go:117] "RemoveContainer" containerID="26a40626d678fa90af391f42ad231bd39cebad7cd8f28a73b6c879a0cf104442" Dec 03 12:12:11 crc kubenswrapper[5052]: I1203 12:12:11.092616 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"26a40626d678fa90af391f42ad231bd39cebad7cd8f28a73b6c879a0cf104442"} err="failed to get container status \"26a40626d678fa90af391f42ad231bd39cebad7cd8f28a73b6c879a0cf104442\": rpc error: code = NotFound desc = could not find container \"26a40626d678fa90af391f42ad231bd39cebad7cd8f28a73b6c879a0cf104442\": container with ID starting with 26a40626d678fa90af391f42ad231bd39cebad7cd8f28a73b6c879a0cf104442 not found: ID does not exist" Dec 03 12:12:11 crc kubenswrapper[5052]: I1203 12:12:11.291391 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 12:12:11 crc kubenswrapper[5052]: I1203 12:12:11.300645 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 12:12:11 crc kubenswrapper[5052]: I1203 12:12:11.322952 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 12:12:11 crc kubenswrapper[5052]: E1203 12:12:11.323438 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="14c699e0-a2a5-4e95-a2b2-450d291e2a2a" containerName="cinder-api" Dec 03 12:12:11 crc kubenswrapper[5052]: I1203 12:12:11.323464 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="14c699e0-a2a5-4e95-a2b2-450d291e2a2a" containerName="cinder-api" Dec 03 12:12:11 crc kubenswrapper[5052]: E1203 12:12:11.323526 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="14c699e0-a2a5-4e95-a2b2-450d291e2a2a" containerName="cinder-api-log" Dec 03 12:12:11 crc kubenswrapper[5052]: I1203 12:12:11.323537 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="14c699e0-a2a5-4e95-a2b2-450d291e2a2a" containerName="cinder-api-log" Dec 03 12:12:11 crc kubenswrapper[5052]: I1203 12:12:11.323777 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="14c699e0-a2a5-4e95-a2b2-450d291e2a2a" containerName="cinder-api" Dec 03 12:12:11 crc kubenswrapper[5052]: I1203 12:12:11.323820 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="14c699e0-a2a5-4e95-a2b2-450d291e2a2a" containerName="cinder-api-log" Dec 03 12:12:11 crc kubenswrapper[5052]: I1203 12:12:11.324997 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:12:11 crc kubenswrapper[5052]: I1203 12:12:11.328265 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-cinder-public-svc" Dec 03 12:12:11 crc kubenswrapper[5052]: I1203 12:12:11.328264 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cinder-api-config-data" Dec 03 12:12:11 crc kubenswrapper[5052]: I1203 12:12:11.329285 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-cinder-internal-svc" Dec 03 12:12:11 crc kubenswrapper[5052]: I1203 12:12:11.352838 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 12:12:11 crc kubenswrapper[5052]: I1203 12:12:11.399013 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:12:11 crc kubenswrapper[5052]: I1203 12:12:11.450237 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2bxjn\" (UniqueName: \"kubernetes.io/projected/388bc903-097f-4f97-8f19-2f7bcdb49cc5-kube-api-access-2bxjn\") pod \"cinder-api-0\" (UID: \"388bc903-097f-4f97-8f19-2f7bcdb49cc5\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:12:11 crc kubenswrapper[5052]: I1203 12:12:11.450304 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/388bc903-097f-4f97-8f19-2f7bcdb49cc5-internal-tls-certs\") pod \"cinder-api-0\" (UID: \"388bc903-097f-4f97-8f19-2f7bcdb49cc5\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:12:11 crc kubenswrapper[5052]: I1203 12:12:11.450598 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/388bc903-097f-4f97-8f19-2f7bcdb49cc5-config-data\") pod \"cinder-api-0\" (UID: \"388bc903-097f-4f97-8f19-2f7bcdb49cc5\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:12:11 crc kubenswrapper[5052]: I1203 12:12:11.450711 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/388bc903-097f-4f97-8f19-2f7bcdb49cc5-logs\") pod \"cinder-api-0\" (UID: \"388bc903-097f-4f97-8f19-2f7bcdb49cc5\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:12:11 crc kubenswrapper[5052]: I1203 12:12:11.450817 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/388bc903-097f-4f97-8f19-2f7bcdb49cc5-config-data-custom\") pod \"cinder-api-0\" (UID: \"388bc903-097f-4f97-8f19-2f7bcdb49cc5\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:12:11 crc kubenswrapper[5052]: I1203 12:12:11.450887 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/388bc903-097f-4f97-8f19-2f7bcdb49cc5-etc-machine-id\") pod \"cinder-api-0\" (UID: \"388bc903-097f-4f97-8f19-2f7bcdb49cc5\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:12:11 crc kubenswrapper[5052]: I1203 12:12:11.450938 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/388bc903-097f-4f97-8f19-2f7bcdb49cc5-scripts\") pod \"cinder-api-0\" (UID: \"388bc903-097f-4f97-8f19-2f7bcdb49cc5\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:12:11 crc kubenswrapper[5052]: I1203 12:12:11.450956 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/388bc903-097f-4f97-8f19-2f7bcdb49cc5-public-tls-certs\") pod \"cinder-api-0\" (UID: \"388bc903-097f-4f97-8f19-2f7bcdb49cc5\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:12:11 crc kubenswrapper[5052]: I1203 12:12:11.451031 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/388bc903-097f-4f97-8f19-2f7bcdb49cc5-combined-ca-bundle\") pod \"cinder-api-0\" (UID: \"388bc903-097f-4f97-8f19-2f7bcdb49cc5\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:12:11 crc kubenswrapper[5052]: I1203 12:12:11.468249 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/barbican-api-58464649dd-sx8mw" Dec 03 12:12:11 crc kubenswrapper[5052]: I1203 12:12:11.552825 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/388bc903-097f-4f97-8f19-2f7bcdb49cc5-internal-tls-certs\") pod \"cinder-api-0\" (UID: \"388bc903-097f-4f97-8f19-2f7bcdb49cc5\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:12:11 crc kubenswrapper[5052]: I1203 12:12:11.552896 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/388bc903-097f-4f97-8f19-2f7bcdb49cc5-config-data\") pod \"cinder-api-0\" (UID: \"388bc903-097f-4f97-8f19-2f7bcdb49cc5\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:12:11 crc kubenswrapper[5052]: I1203 12:12:11.552923 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/388bc903-097f-4f97-8f19-2f7bcdb49cc5-logs\") pod \"cinder-api-0\" (UID: \"388bc903-097f-4f97-8f19-2f7bcdb49cc5\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:12:11 crc kubenswrapper[5052]: I1203 12:12:11.552971 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/388bc903-097f-4f97-8f19-2f7bcdb49cc5-config-data-custom\") pod \"cinder-api-0\" (UID: \"388bc903-097f-4f97-8f19-2f7bcdb49cc5\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:12:11 crc kubenswrapper[5052]: I1203 12:12:11.553273 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/388bc903-097f-4f97-8f19-2f7bcdb49cc5-logs\") pod \"cinder-api-0\" (UID: \"388bc903-097f-4f97-8f19-2f7bcdb49cc5\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:12:11 crc kubenswrapper[5052]: I1203 12:12:11.553548 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/388bc903-097f-4f97-8f19-2f7bcdb49cc5-etc-machine-id\") pod \"cinder-api-0\" (UID: \"388bc903-097f-4f97-8f19-2f7bcdb49cc5\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:12:11 crc kubenswrapper[5052]: I1203 12:12:11.553585 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/388bc903-097f-4f97-8f19-2f7bcdb49cc5-public-tls-certs\") pod \"cinder-api-0\" (UID: \"388bc903-097f-4f97-8f19-2f7bcdb49cc5\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:12:11 crc kubenswrapper[5052]: I1203 12:12:11.553607 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/388bc903-097f-4f97-8f19-2f7bcdb49cc5-scripts\") pod \"cinder-api-0\" (UID: \"388bc903-097f-4f97-8f19-2f7bcdb49cc5\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:12:11 crc kubenswrapper[5052]: I1203 12:12:11.553682 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/388bc903-097f-4f97-8f19-2f7bcdb49cc5-combined-ca-bundle\") pod \"cinder-api-0\" (UID: \"388bc903-097f-4f97-8f19-2f7bcdb49cc5\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:12:11 crc kubenswrapper[5052]: I1203 12:12:11.553724 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-2bxjn\" (UniqueName: \"kubernetes.io/projected/388bc903-097f-4f97-8f19-2f7bcdb49cc5-kube-api-access-2bxjn\") pod \"cinder-api-0\" (UID: \"388bc903-097f-4f97-8f19-2f7bcdb49cc5\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:12:11 crc kubenswrapper[5052]: I1203 12:12:11.554312 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/388bc903-097f-4f97-8f19-2f7bcdb49cc5-etc-machine-id\") pod \"cinder-api-0\" (UID: \"388bc903-097f-4f97-8f19-2f7bcdb49cc5\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:12:11 crc kubenswrapper[5052]: I1203 12:12:11.556963 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/388bc903-097f-4f97-8f19-2f7bcdb49cc5-scripts\") pod \"cinder-api-0\" (UID: \"388bc903-097f-4f97-8f19-2f7bcdb49cc5\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:12:11 crc kubenswrapper[5052]: I1203 12:12:11.557948 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/388bc903-097f-4f97-8f19-2f7bcdb49cc5-combined-ca-bundle\") pod \"cinder-api-0\" (UID: \"388bc903-097f-4f97-8f19-2f7bcdb49cc5\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:12:11 crc kubenswrapper[5052]: I1203 12:12:11.558146 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/388bc903-097f-4f97-8f19-2f7bcdb49cc5-internal-tls-certs\") pod \"cinder-api-0\" (UID: \"388bc903-097f-4f97-8f19-2f7bcdb49cc5\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:12:11 crc kubenswrapper[5052]: I1203 12:12:11.558200 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/388bc903-097f-4f97-8f19-2f7bcdb49cc5-config-data\") pod \"cinder-api-0\" (UID: \"388bc903-097f-4f97-8f19-2f7bcdb49cc5\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:12:11 crc kubenswrapper[5052]: I1203 12:12:11.558379 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/388bc903-097f-4f97-8f19-2f7bcdb49cc5-public-tls-certs\") pod \"cinder-api-0\" (UID: \"388bc903-097f-4f97-8f19-2f7bcdb49cc5\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:12:11 crc kubenswrapper[5052]: I1203 12:12:11.562249 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/388bc903-097f-4f97-8f19-2f7bcdb49cc5-config-data-custom\") pod \"cinder-api-0\" (UID: \"388bc903-097f-4f97-8f19-2f7bcdb49cc5\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:12:11 crc kubenswrapper[5052]: I1203 12:12:11.570917 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-2bxjn\" (UniqueName: \"kubernetes.io/projected/388bc903-097f-4f97-8f19-2f7bcdb49cc5-kube-api-access-2bxjn\") pod \"cinder-api-0\" (UID: \"388bc903-097f-4f97-8f19-2f7bcdb49cc5\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:12:11 crc kubenswrapper[5052]: I1203 12:12:11.618869 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/barbican-api-58464649dd-sx8mw" Dec 03 12:12:11 crc kubenswrapper[5052]: I1203 12:12:11.641087 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:12:11 crc kubenswrapper[5052]: I1203 12:12:11.716006 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="14c699e0-a2a5-4e95-a2b2-450d291e2a2a" path="/var/lib/kubelet/pods/14c699e0-a2a5-4e95-a2b2-450d291e2a2a/volumes" Dec 03 12:12:11 crc kubenswrapper[5052]: I1203 12:12:11.716834 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-api-649f8bf5b8-wjzcq"] Dec 03 12:12:11 crc kubenswrapper[5052]: I1203 12:12:11.717110 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/barbican-api-649f8bf5b8-wjzcq" podUID="ba056586-628e-45da-852f-9fc1736ff76f" containerName="barbican-api-log" containerID="cri-o://7f3f8bf2361bf8c8c6620a120657d63671e3dde0dad82dcad1fe05618f40acbb" gracePeriod=30 Dec 03 12:12:11 crc kubenswrapper[5052]: I1203 12:12:11.719813 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/barbican-api-649f8bf5b8-wjzcq" podUID="ba056586-628e-45da-852f-9fc1736ff76f" containerName="barbican-api" containerID="cri-o://019542a1d362fb4a4e52a6614c1be3532669828ab5dd32337c547c9e0c866e5c" gracePeriod=30 Dec 03 12:12:11 crc kubenswrapper[5052]: I1203 12:12:11.727779 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/barbican-api-649f8bf5b8-wjzcq" podUID="ba056586-628e-45da-852f-9fc1736ff76f" containerName="barbican-api-log" probeResult="failure" output="Get \"http://10.217.0.253:9311/healthcheck\": EOF" Dec 03 12:12:12 crc kubenswrapper[5052]: I1203 12:12:12.195875 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 12:12:13 crc kubenswrapper[5052]: I1203 12:12:12.997933 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"388bc903-097f-4f97-8f19-2f7bcdb49cc5","Type":"ContainerStarted","Data":"c15684b97ff4a9d54a9eeca81697f4cced793ffe157adb3548bedd21bf3249d0"} Dec 03 12:12:13 crc kubenswrapper[5052]: I1203 12:12:12.998437 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"388bc903-097f-4f97-8f19-2f7bcdb49cc5","Type":"ContainerStarted","Data":"9b446c712f47a339ef4a4e66b7ebd4d2243479663b879feaf0dd616123564040"} Dec 03 12:12:13 crc kubenswrapper[5052]: I1203 12:12:13.000341 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openshift-marketplace/community-operators-q4qt9"] Dec 03 12:12:13 crc kubenswrapper[5052]: I1203 12:12:13.006187 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/community-operators-q4qt9" Dec 03 12:12:13 crc kubenswrapper[5052]: I1203 12:12:13.007675 5052 generic.go:334] "Generic (PLEG): container finished" podID="ba056586-628e-45da-852f-9fc1736ff76f" containerID="7f3f8bf2361bf8c8c6620a120657d63671e3dde0dad82dcad1fe05618f40acbb" exitCode=143 Dec 03 12:12:13 crc kubenswrapper[5052]: I1203 12:12:13.007723 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-649f8bf5b8-wjzcq" event={"ID":"ba056586-628e-45da-852f-9fc1736ff76f","Type":"ContainerDied","Data":"7f3f8bf2361bf8c8c6620a120657d63671e3dde0dad82dcad1fe05618f40acbb"} Dec 03 12:12:13 crc kubenswrapper[5052]: I1203 12:12:13.016065 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/community-operators-q4qt9"] Dec 03 12:12:13 crc kubenswrapper[5052]: I1203 12:12:13.112007 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/686067c2-dedc-4a04-a433-3df6e20f405d-catalog-content\") pod \"community-operators-q4qt9\" (UID: \"686067c2-dedc-4a04-a433-3df6e20f405d\") " pod="openshift-marketplace/community-operators-q4qt9" Dec 03 12:12:13 crc kubenswrapper[5052]: I1203 12:12:13.112413 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vxmx8\" (UniqueName: \"kubernetes.io/projected/686067c2-dedc-4a04-a433-3df6e20f405d-kube-api-access-vxmx8\") pod \"community-operators-q4qt9\" (UID: \"686067c2-dedc-4a04-a433-3df6e20f405d\") " pod="openshift-marketplace/community-operators-q4qt9" Dec 03 12:12:13 crc kubenswrapper[5052]: I1203 12:12:13.112474 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/686067c2-dedc-4a04-a433-3df6e20f405d-utilities\") pod \"community-operators-q4qt9\" (UID: \"686067c2-dedc-4a04-a433-3df6e20f405d\") " pod="openshift-marketplace/community-operators-q4qt9" Dec 03 12:12:13 crc kubenswrapper[5052]: I1203 12:12:13.214063 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-vxmx8\" (UniqueName: \"kubernetes.io/projected/686067c2-dedc-4a04-a433-3df6e20f405d-kube-api-access-vxmx8\") pod \"community-operators-q4qt9\" (UID: \"686067c2-dedc-4a04-a433-3df6e20f405d\") " pod="openshift-marketplace/community-operators-q4qt9" Dec 03 12:12:13 crc kubenswrapper[5052]: I1203 12:12:13.214165 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/686067c2-dedc-4a04-a433-3df6e20f405d-utilities\") pod \"community-operators-q4qt9\" (UID: \"686067c2-dedc-4a04-a433-3df6e20f405d\") " pod="openshift-marketplace/community-operators-q4qt9" Dec 03 12:12:13 crc kubenswrapper[5052]: I1203 12:12:13.214846 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/686067c2-dedc-4a04-a433-3df6e20f405d-catalog-content\") pod \"community-operators-q4qt9\" (UID: \"686067c2-dedc-4a04-a433-3df6e20f405d\") " pod="openshift-marketplace/community-operators-q4qt9" Dec 03 12:12:13 crc kubenswrapper[5052]: I1203 12:12:13.216858 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/686067c2-dedc-4a04-a433-3df6e20f405d-utilities\") pod \"community-operators-q4qt9\" (UID: \"686067c2-dedc-4a04-a433-3df6e20f405d\") " pod="openshift-marketplace/community-operators-q4qt9" Dec 03 12:12:13 crc kubenswrapper[5052]: I1203 12:12:13.214199 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/686067c2-dedc-4a04-a433-3df6e20f405d-catalog-content\") pod \"community-operators-q4qt9\" (UID: \"686067c2-dedc-4a04-a433-3df6e20f405d\") " pod="openshift-marketplace/community-operators-q4qt9" Dec 03 12:12:13 crc kubenswrapper[5052]: I1203 12:12:13.241335 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-vxmx8\" (UniqueName: \"kubernetes.io/projected/686067c2-dedc-4a04-a433-3df6e20f405d-kube-api-access-vxmx8\") pod \"community-operators-q4qt9\" (UID: \"686067c2-dedc-4a04-a433-3df6e20f405d\") " pod="openshift-marketplace/community-operators-q4qt9" Dec 03 12:12:13 crc kubenswrapper[5052]: I1203 12:12:13.335856 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/community-operators-q4qt9" Dec 03 12:12:13 crc kubenswrapper[5052]: I1203 12:12:13.861702 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/community-operators-q4qt9"] Dec 03 12:12:14 crc kubenswrapper[5052]: I1203 12:12:14.017998 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/community-operators-q4qt9" event={"ID":"686067c2-dedc-4a04-a433-3df6e20f405d","Type":"ContainerStarted","Data":"93c999e82f9a6f29ef3649f41f565a7580da592cd2697a0ff12e1e348fb8942e"} Dec 03 12:12:14 crc kubenswrapper[5052]: I1203 12:12:14.018373 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/community-operators-q4qt9" event={"ID":"686067c2-dedc-4a04-a433-3df6e20f405d","Type":"ContainerStarted","Data":"d6c22f1e9a303ddbb26b8f582bae1cd5928211640df9e4ba1d1a75146ca95077"} Dec 03 12:12:14 crc kubenswrapper[5052]: I1203 12:12:14.021797 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"388bc903-097f-4f97-8f19-2f7bcdb49cc5","Type":"ContainerStarted","Data":"7024c66624b9b95f71fa0589d2f50417eaa07d7b921b5356f06724331235a546"} Dec 03 12:12:14 crc kubenswrapper[5052]: I1203 12:12:14.022016 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:12:14 crc kubenswrapper[5052]: I1203 12:12:14.071688 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/cinder-api-0" podStartSLOduration=3.071656849 podStartE2EDuration="3.071656849s" podCreationTimestamp="2025-12-03 12:12:11 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:12:14.068095757 +0000 UTC m=+3124.656337521" watchObservedRunningTime="2025-12-03 12:12:14.071656849 +0000 UTC m=+3124.659898613" Dec 03 12:12:15 crc kubenswrapper[5052]: I1203 12:12:14.999362 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/placement-79d5cb4fdb-7h2fn" Dec 03 12:12:15 crc kubenswrapper[5052]: I1203 12:12:15.012770 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/placement-79d5cb4fdb-7h2fn" Dec 03 12:12:15 crc kubenswrapper[5052]: I1203 12:12:15.045610 5052 generic.go:334] "Generic (PLEG): container finished" podID="686067c2-dedc-4a04-a433-3df6e20f405d" containerID="93c999e82f9a6f29ef3649f41f565a7580da592cd2697a0ff12e1e348fb8942e" exitCode=0 Dec 03 12:12:15 crc kubenswrapper[5052]: I1203 12:12:15.046021 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/community-operators-q4qt9" event={"ID":"686067c2-dedc-4a04-a433-3df6e20f405d","Type":"ContainerDied","Data":"93c999e82f9a6f29ef3649f41f565a7580da592cd2697a0ff12e1e348fb8942e"} Dec 03 12:12:15 crc kubenswrapper[5052]: I1203 12:12:15.605773 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openshift-marketplace/redhat-marketplace-p2mgk"] Dec 03 12:12:15 crc kubenswrapper[5052]: I1203 12:12:15.607939 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/redhat-marketplace-p2mgk" Dec 03 12:12:15 crc kubenswrapper[5052]: I1203 12:12:15.624452 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/redhat-marketplace-p2mgk"] Dec 03 12:12:15 crc kubenswrapper[5052]: I1203 12:12:15.774129 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/658ad8d2-0efc-4752-a4c5-54e313bdb305-utilities\") pod \"redhat-marketplace-p2mgk\" (UID: \"658ad8d2-0efc-4752-a4c5-54e313bdb305\") " pod="openshift-marketplace/redhat-marketplace-p2mgk" Dec 03 12:12:15 crc kubenswrapper[5052]: I1203 12:12:15.774186 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pqfdv\" (UniqueName: \"kubernetes.io/projected/658ad8d2-0efc-4752-a4c5-54e313bdb305-kube-api-access-pqfdv\") pod \"redhat-marketplace-p2mgk\" (UID: \"658ad8d2-0efc-4752-a4c5-54e313bdb305\") " pod="openshift-marketplace/redhat-marketplace-p2mgk" Dec 03 12:12:15 crc kubenswrapper[5052]: I1203 12:12:15.774326 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/658ad8d2-0efc-4752-a4c5-54e313bdb305-catalog-content\") pod \"redhat-marketplace-p2mgk\" (UID: \"658ad8d2-0efc-4752-a4c5-54e313bdb305\") " pod="openshift-marketplace/redhat-marketplace-p2mgk" Dec 03 12:12:15 crc kubenswrapper[5052]: I1203 12:12:15.875904 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/658ad8d2-0efc-4752-a4c5-54e313bdb305-utilities\") pod \"redhat-marketplace-p2mgk\" (UID: \"658ad8d2-0efc-4752-a4c5-54e313bdb305\") " pod="openshift-marketplace/redhat-marketplace-p2mgk" Dec 03 12:12:15 crc kubenswrapper[5052]: I1203 12:12:15.876456 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-pqfdv\" (UniqueName: \"kubernetes.io/projected/658ad8d2-0efc-4752-a4c5-54e313bdb305-kube-api-access-pqfdv\") pod \"redhat-marketplace-p2mgk\" (UID: \"658ad8d2-0efc-4752-a4c5-54e313bdb305\") " pod="openshift-marketplace/redhat-marketplace-p2mgk" Dec 03 12:12:15 crc kubenswrapper[5052]: I1203 12:12:15.876596 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/658ad8d2-0efc-4752-a4c5-54e313bdb305-utilities\") pod \"redhat-marketplace-p2mgk\" (UID: \"658ad8d2-0efc-4752-a4c5-54e313bdb305\") " pod="openshift-marketplace/redhat-marketplace-p2mgk" Dec 03 12:12:15 crc kubenswrapper[5052]: I1203 12:12:15.876627 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/658ad8d2-0efc-4752-a4c5-54e313bdb305-catalog-content\") pod \"redhat-marketplace-p2mgk\" (UID: \"658ad8d2-0efc-4752-a4c5-54e313bdb305\") " pod="openshift-marketplace/redhat-marketplace-p2mgk" Dec 03 12:12:15 crc kubenswrapper[5052]: I1203 12:12:15.877108 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/658ad8d2-0efc-4752-a4c5-54e313bdb305-catalog-content\") pod \"redhat-marketplace-p2mgk\" (UID: \"658ad8d2-0efc-4752-a4c5-54e313bdb305\") " pod="openshift-marketplace/redhat-marketplace-p2mgk" Dec 03 12:12:15 crc kubenswrapper[5052]: I1203 12:12:15.893803 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-pqfdv\" (UniqueName: \"kubernetes.io/projected/658ad8d2-0efc-4752-a4c5-54e313bdb305-kube-api-access-pqfdv\") pod \"redhat-marketplace-p2mgk\" (UID: \"658ad8d2-0efc-4752-a4c5-54e313bdb305\") " pod="openshift-marketplace/redhat-marketplace-p2mgk" Dec 03 12:12:15 crc kubenswrapper[5052]: I1203 12:12:15.941250 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/redhat-marketplace-p2mgk" Dec 03 12:12:16 crc kubenswrapper[5052]: I1203 12:12:16.070888 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/community-operators-q4qt9" event={"ID":"686067c2-dedc-4a04-a433-3df6e20f405d","Type":"ContainerStarted","Data":"0a054fff8d1370300becab0cf5eed626d5e4176710b1b3744d8b748467865bc4"} Dec 03 12:12:16 crc kubenswrapper[5052]: I1203 12:12:16.443339 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/redhat-marketplace-p2mgk"] Dec 03 12:12:16 crc kubenswrapper[5052]: W1203 12:12:16.451685 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod658ad8d2_0efc_4752_a4c5_54e313bdb305.slice/crio-eb054f91f0d01114e422dbf45f721346c3d6a2e7d470dfcd459f00d7ba67e39b WatchSource:0}: Error finding container eb054f91f0d01114e422dbf45f721346c3d6a2e7d470dfcd459f00d7ba67e39b: Status 404 returned error can't find the container with id eb054f91f0d01114e422dbf45f721346c3d6a2e7d470dfcd459f00d7ba67e39b Dec 03 12:12:16 crc kubenswrapper[5052]: I1203 12:12:16.611201 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:12:16 crc kubenswrapper[5052]: I1203 12:12:16.650174 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 12:12:17 crc kubenswrapper[5052]: I1203 12:12:17.081131 5052 generic.go:334] "Generic (PLEG): container finished" podID="686067c2-dedc-4a04-a433-3df6e20f405d" containerID="0a054fff8d1370300becab0cf5eed626d5e4176710b1b3744d8b748467865bc4" exitCode=0 Dec 03 12:12:17 crc kubenswrapper[5052]: I1203 12:12:17.081197 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/community-operators-q4qt9" event={"ID":"686067c2-dedc-4a04-a433-3df6e20f405d","Type":"ContainerDied","Data":"0a054fff8d1370300becab0cf5eed626d5e4176710b1b3744d8b748467865bc4"} Dec 03 12:12:17 crc kubenswrapper[5052]: I1203 12:12:17.083280 5052 generic.go:334] "Generic (PLEG): container finished" podID="658ad8d2-0efc-4752-a4c5-54e313bdb305" containerID="6495ca125925b04c59734d6881a3281d4c16ec80168c3fb71673bdc1b85f4ce2" exitCode=0 Dec 03 12:12:17 crc kubenswrapper[5052]: I1203 12:12:17.083407 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-marketplace-p2mgk" event={"ID":"658ad8d2-0efc-4752-a4c5-54e313bdb305","Type":"ContainerDied","Data":"6495ca125925b04c59734d6881a3281d4c16ec80168c3fb71673bdc1b85f4ce2"} Dec 03 12:12:17 crc kubenswrapper[5052]: I1203 12:12:17.083460 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/cinder-scheduler-0" podUID="63ba4f58-6d06-452e-86b7-54a61a998c03" containerName="cinder-scheduler" containerID="cri-o://4cb6e9620ad9bca18eb34b849b06c51b05cc9bee37b75d985fde32193b027d50" gracePeriod=30 Dec 03 12:12:17 crc kubenswrapper[5052]: I1203 12:12:17.083495 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-marketplace-p2mgk" event={"ID":"658ad8d2-0efc-4752-a4c5-54e313bdb305","Type":"ContainerStarted","Data":"eb054f91f0d01114e422dbf45f721346c3d6a2e7d470dfcd459f00d7ba67e39b"} Dec 03 12:12:17 crc kubenswrapper[5052]: I1203 12:12:17.083561 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/cinder-scheduler-0" podUID="63ba4f58-6d06-452e-86b7-54a61a998c03" containerName="probe" containerID="cri-o://2a7d4aa75fed7462d0c0db261e38e723cb83192e669ec7409e8c399c0ea325ee" gracePeriod=30 Dec 03 12:12:17 crc kubenswrapper[5052]: I1203 12:12:17.145887 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/barbican-api-649f8bf5b8-wjzcq" podUID="ba056586-628e-45da-852f-9fc1736ff76f" containerName="barbican-api" probeResult="failure" output="Get \"http://10.217.0.253:9311/healthcheck\": read tcp 10.217.0.2:40876->10.217.0.253:9311: read: connection reset by peer" Dec 03 12:12:17 crc kubenswrapper[5052]: I1203 12:12:17.147068 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/barbican-api-649f8bf5b8-wjzcq" podUID="ba056586-628e-45da-852f-9fc1736ff76f" containerName="barbican-api-log" probeResult="failure" output="Get \"http://10.217.0.253:9311/healthcheck\": read tcp 10.217.0.2:40878->10.217.0.253:9311: read: connection reset by peer" Dec 03 12:12:17 crc kubenswrapper[5052]: I1203 12:12:17.687253 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-api-649f8bf5b8-wjzcq" Dec 03 12:12:17 crc kubenswrapper[5052]: I1203 12:12:17.809248 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/ba056586-628e-45da-852f-9fc1736ff76f-config-data\") pod \"ba056586-628e-45da-852f-9fc1736ff76f\" (UID: \"ba056586-628e-45da-852f-9fc1736ff76f\") " Dec 03 12:12:17 crc kubenswrapper[5052]: I1203 12:12:17.809377 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xbp6p\" (UniqueName: \"kubernetes.io/projected/ba056586-628e-45da-852f-9fc1736ff76f-kube-api-access-xbp6p\") pod \"ba056586-628e-45da-852f-9fc1736ff76f\" (UID: \"ba056586-628e-45da-852f-9fc1736ff76f\") " Dec 03 12:12:17 crc kubenswrapper[5052]: I1203 12:12:17.809499 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/ba056586-628e-45da-852f-9fc1736ff76f-logs\") pod \"ba056586-628e-45da-852f-9fc1736ff76f\" (UID: \"ba056586-628e-45da-852f-9fc1736ff76f\") " Dec 03 12:12:17 crc kubenswrapper[5052]: I1203 12:12:17.809605 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ba056586-628e-45da-852f-9fc1736ff76f-combined-ca-bundle\") pod \"ba056586-628e-45da-852f-9fc1736ff76f\" (UID: \"ba056586-628e-45da-852f-9fc1736ff76f\") " Dec 03 12:12:17 crc kubenswrapper[5052]: I1203 12:12:17.809678 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/ba056586-628e-45da-852f-9fc1736ff76f-config-data-custom\") pod \"ba056586-628e-45da-852f-9fc1736ff76f\" (UID: \"ba056586-628e-45da-852f-9fc1736ff76f\") " Dec 03 12:12:17 crc kubenswrapper[5052]: I1203 12:12:17.810130 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba056586-628e-45da-852f-9fc1736ff76f-logs" (OuterVolumeSpecName: "logs") pod "ba056586-628e-45da-852f-9fc1736ff76f" (UID: "ba056586-628e-45da-852f-9fc1736ff76f"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:12:17 crc kubenswrapper[5052]: I1203 12:12:17.814492 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ba056586-628e-45da-852f-9fc1736ff76f-kube-api-access-xbp6p" (OuterVolumeSpecName: "kube-api-access-xbp6p") pod "ba056586-628e-45da-852f-9fc1736ff76f" (UID: "ba056586-628e-45da-852f-9fc1736ff76f"). InnerVolumeSpecName "kube-api-access-xbp6p". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:12:17 crc kubenswrapper[5052]: I1203 12:12:17.817542 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ba056586-628e-45da-852f-9fc1736ff76f-config-data-custom" (OuterVolumeSpecName: "config-data-custom") pod "ba056586-628e-45da-852f-9fc1736ff76f" (UID: "ba056586-628e-45da-852f-9fc1736ff76f"). InnerVolumeSpecName "config-data-custom". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:12:17 crc kubenswrapper[5052]: I1203 12:12:17.844493 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ba056586-628e-45da-852f-9fc1736ff76f-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "ba056586-628e-45da-852f-9fc1736ff76f" (UID: "ba056586-628e-45da-852f-9fc1736ff76f"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:12:17 crc kubenswrapper[5052]: I1203 12:12:17.867738 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ba056586-628e-45da-852f-9fc1736ff76f-config-data" (OuterVolumeSpecName: "config-data") pod "ba056586-628e-45da-852f-9fc1736ff76f" (UID: "ba056586-628e-45da-852f-9fc1736ff76f"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:12:17 crc kubenswrapper[5052]: I1203 12:12:17.918229 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ba056586-628e-45da-852f-9fc1736ff76f-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:12:17 crc kubenswrapper[5052]: I1203 12:12:17.918534 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/ba056586-628e-45da-852f-9fc1736ff76f-config-data-custom\") on node \"crc\" DevicePath \"\"" Dec 03 12:12:17 crc kubenswrapper[5052]: I1203 12:12:17.918544 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/ba056586-628e-45da-852f-9fc1736ff76f-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:12:17 crc kubenswrapper[5052]: I1203 12:12:17.918554 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-xbp6p\" (UniqueName: \"kubernetes.io/projected/ba056586-628e-45da-852f-9fc1736ff76f-kube-api-access-xbp6p\") on node \"crc\" DevicePath \"\"" Dec 03 12:12:17 crc kubenswrapper[5052]: I1203 12:12:17.918564 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/ba056586-628e-45da-852f-9fc1736ff76f-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:12:18 crc kubenswrapper[5052]: I1203 12:12:18.094631 5052 generic.go:334] "Generic (PLEG): container finished" podID="63ba4f58-6d06-452e-86b7-54a61a998c03" containerID="2a7d4aa75fed7462d0c0db261e38e723cb83192e669ec7409e8c399c0ea325ee" exitCode=0 Dec 03 12:12:18 crc kubenswrapper[5052]: I1203 12:12:18.094703 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"63ba4f58-6d06-452e-86b7-54a61a998c03","Type":"ContainerDied","Data":"2a7d4aa75fed7462d0c0db261e38e723cb83192e669ec7409e8c399c0ea325ee"} Dec 03 12:12:18 crc kubenswrapper[5052]: I1203 12:12:18.096645 5052 generic.go:334] "Generic (PLEG): container finished" podID="ba056586-628e-45da-852f-9fc1736ff76f" containerID="019542a1d362fb4a4e52a6614c1be3532669828ab5dd32337c547c9e0c866e5c" exitCode=0 Dec 03 12:12:18 crc kubenswrapper[5052]: I1203 12:12:18.096700 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-649f8bf5b8-wjzcq" event={"ID":"ba056586-628e-45da-852f-9fc1736ff76f","Type":"ContainerDied","Data":"019542a1d362fb4a4e52a6614c1be3532669828ab5dd32337c547c9e0c866e5c"} Dec 03 12:12:18 crc kubenswrapper[5052]: I1203 12:12:18.096725 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-649f8bf5b8-wjzcq" event={"ID":"ba056586-628e-45da-852f-9fc1736ff76f","Type":"ContainerDied","Data":"2a7e9168d6dbc722bd52ede7ce54deb4c1036cbd1faa23d6ca0f17287634c7de"} Dec 03 12:12:18 crc kubenswrapper[5052]: I1203 12:12:18.096744 5052 scope.go:117] "RemoveContainer" containerID="019542a1d362fb4a4e52a6614c1be3532669828ab5dd32337c547c9e0c866e5c" Dec 03 12:12:18 crc kubenswrapper[5052]: I1203 12:12:18.096918 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-api-649f8bf5b8-wjzcq" Dec 03 12:12:18 crc kubenswrapper[5052]: I1203 12:12:18.103339 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/community-operators-q4qt9" event={"ID":"686067c2-dedc-4a04-a433-3df6e20f405d","Type":"ContainerStarted","Data":"63eeecd303bb8e2a2e060597115b296d7a1812ba2bd0989d4647e8fa974fe2c7"} Dec 03 12:12:18 crc kubenswrapper[5052]: I1203 12:12:18.138492 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-marketplace/community-operators-q4qt9" podStartSLOduration=3.68557156 podStartE2EDuration="6.138475309s" podCreationTimestamp="2025-12-03 12:12:12 +0000 UTC" firstStartedPulling="2025-12-03 12:12:15.060985157 +0000 UTC m=+3125.649226921" lastFinishedPulling="2025-12-03 12:12:17.513888896 +0000 UTC m=+3128.102130670" observedRunningTime="2025-12-03 12:12:18.137018717 +0000 UTC m=+3128.725260481" watchObservedRunningTime="2025-12-03 12:12:18.138475309 +0000 UTC m=+3128.726717073" Dec 03 12:12:18 crc kubenswrapper[5052]: I1203 12:12:18.145708 5052 scope.go:117] "RemoveContainer" containerID="7f3f8bf2361bf8c8c6620a120657d63671e3dde0dad82dcad1fe05618f40acbb" Dec 03 12:12:18 crc kubenswrapper[5052]: I1203 12:12:18.188574 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-api-649f8bf5b8-wjzcq"] Dec 03 12:12:18 crc kubenswrapper[5052]: I1203 12:12:18.192682 5052 scope.go:117] "RemoveContainer" containerID="019542a1d362fb4a4e52a6614c1be3532669828ab5dd32337c547c9e0c866e5c" Dec 03 12:12:18 crc kubenswrapper[5052]: E1203 12:12:18.197689 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"019542a1d362fb4a4e52a6614c1be3532669828ab5dd32337c547c9e0c866e5c\": container with ID starting with 019542a1d362fb4a4e52a6614c1be3532669828ab5dd32337c547c9e0c866e5c not found: ID does not exist" containerID="019542a1d362fb4a4e52a6614c1be3532669828ab5dd32337c547c9e0c866e5c" Dec 03 12:12:18 crc kubenswrapper[5052]: I1203 12:12:18.197748 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"019542a1d362fb4a4e52a6614c1be3532669828ab5dd32337c547c9e0c866e5c"} err="failed to get container status \"019542a1d362fb4a4e52a6614c1be3532669828ab5dd32337c547c9e0c866e5c\": rpc error: code = NotFound desc = could not find container \"019542a1d362fb4a4e52a6614c1be3532669828ab5dd32337c547c9e0c866e5c\": container with ID starting with 019542a1d362fb4a4e52a6614c1be3532669828ab5dd32337c547c9e0c866e5c not found: ID does not exist" Dec 03 12:12:18 crc kubenswrapper[5052]: I1203 12:12:18.197781 5052 scope.go:117] "RemoveContainer" containerID="7f3f8bf2361bf8c8c6620a120657d63671e3dde0dad82dcad1fe05618f40acbb" Dec 03 12:12:18 crc kubenswrapper[5052]: E1203 12:12:18.202700 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7f3f8bf2361bf8c8c6620a120657d63671e3dde0dad82dcad1fe05618f40acbb\": container with ID starting with 7f3f8bf2361bf8c8c6620a120657d63671e3dde0dad82dcad1fe05618f40acbb not found: ID does not exist" containerID="7f3f8bf2361bf8c8c6620a120657d63671e3dde0dad82dcad1fe05618f40acbb" Dec 03 12:12:18 crc kubenswrapper[5052]: I1203 12:12:18.202748 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7f3f8bf2361bf8c8c6620a120657d63671e3dde0dad82dcad1fe05618f40acbb"} err="failed to get container status \"7f3f8bf2361bf8c8c6620a120657d63671e3dde0dad82dcad1fe05618f40acbb\": rpc error: code = NotFound desc = could not find container \"7f3f8bf2361bf8c8c6620a120657d63671e3dde0dad82dcad1fe05618f40acbb\": container with ID starting with 7f3f8bf2361bf8c8c6620a120657d63671e3dde0dad82dcad1fe05618f40acbb not found: ID does not exist" Dec 03 12:12:18 crc kubenswrapper[5052]: I1203 12:12:18.204837 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/barbican-api-649f8bf5b8-wjzcq"] Dec 03 12:12:18 crc kubenswrapper[5052]: I1203 12:12:18.672835 5052 scope.go:117] "RemoveContainer" containerID="63298107b3d090deff9ed92f3d613e5b1c86d9137684d117500257db8ee1c805" Dec 03 12:12:18 crc kubenswrapper[5052]: E1203 12:12:18.673196 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 12:12:19 crc kubenswrapper[5052]: I1203 12:12:19.115036 5052 generic.go:334] "Generic (PLEG): container finished" podID="658ad8d2-0efc-4752-a4c5-54e313bdb305" containerID="399b2e1c1557872473b7c499b4421f32ec0a1fc5c1e91f0333f9d0d389d40b22" exitCode=0 Dec 03 12:12:19 crc kubenswrapper[5052]: I1203 12:12:19.116345 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-marketplace-p2mgk" event={"ID":"658ad8d2-0efc-4752-a4c5-54e313bdb305","Type":"ContainerDied","Data":"399b2e1c1557872473b7c499b4421f32ec0a1fc5c1e91f0333f9d0d389d40b22"} Dec 03 12:12:19 crc kubenswrapper[5052]: I1203 12:12:19.689434 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ba056586-628e-45da-852f-9fc1736ff76f" path="/var/lib/kubelet/pods/ba056586-628e-45da-852f-9fc1736ff76f/volumes" Dec 03 12:12:20 crc kubenswrapper[5052]: I1203 12:12:20.125885 5052 generic.go:334] "Generic (PLEG): container finished" podID="8474b807-15d5-405a-ade8-1e4fa8954ce6" containerID="0519af1e24d904a84f986fb58221ec301343653caf8111b439f0966c5ec6f27b" exitCode=0 Dec 03 12:12:20 crc kubenswrapper[5052]: I1203 12:12:20.125940 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-db-sync-cbdcz" event={"ID":"8474b807-15d5-405a-ade8-1e4fa8954ce6","Type":"ContainerDied","Data":"0519af1e24d904a84f986fb58221ec301343653caf8111b439f0966c5ec6f27b"} Dec 03 12:12:20 crc kubenswrapper[5052]: I1203 12:12:20.129163 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-marketplace-p2mgk" event={"ID":"658ad8d2-0efc-4752-a4c5-54e313bdb305","Type":"ContainerStarted","Data":"b2cb242e355b8c375f8b20b6132c7e0d7475ada1b01193b4b00caa311f6f8fa9"} Dec 03 12:12:20 crc kubenswrapper[5052]: I1203 12:12:20.170967 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-marketplace/redhat-marketplace-p2mgk" podStartSLOduration=2.728119012 podStartE2EDuration="5.170947402s" podCreationTimestamp="2025-12-03 12:12:15 +0000 UTC" firstStartedPulling="2025-12-03 12:12:17.084706741 +0000 UTC m=+3127.672948505" lastFinishedPulling="2025-12-03 12:12:19.527535131 +0000 UTC m=+3130.115776895" observedRunningTime="2025-12-03 12:12:20.167836834 +0000 UTC m=+3130.756078598" watchObservedRunningTime="2025-12-03 12:12:20.170947402 +0000 UTC m=+3130.759189166" Dec 03 12:12:20 crc kubenswrapper[5052]: I1203 12:12:20.605990 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/keystone-6fd4994758-fsw2c" Dec 03 12:12:21 crc kubenswrapper[5052]: I1203 12:12:21.535144 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-db-sync-cbdcz" Dec 03 12:12:21 crc kubenswrapper[5052]: I1203 12:12:21.690640 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8474b807-15d5-405a-ade8-1e4fa8954ce6-combined-ca-bundle\") pod \"8474b807-15d5-405a-ade8-1e4fa8954ce6\" (UID: \"8474b807-15d5-405a-ade8-1e4fa8954ce6\") " Dec 03 12:12:21 crc kubenswrapper[5052]: I1203 12:12:21.690869 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b6l42\" (UniqueName: \"kubernetes.io/projected/8474b807-15d5-405a-ade8-1e4fa8954ce6-kube-api-access-b6l42\") pod \"8474b807-15d5-405a-ade8-1e4fa8954ce6\" (UID: \"8474b807-15d5-405a-ade8-1e4fa8954ce6\") " Dec 03 12:12:21 crc kubenswrapper[5052]: I1203 12:12:21.690901 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/8474b807-15d5-405a-ade8-1e4fa8954ce6-config\") pod \"8474b807-15d5-405a-ade8-1e4fa8954ce6\" (UID: \"8474b807-15d5-405a-ade8-1e4fa8954ce6\") " Dec 03 12:12:21 crc kubenswrapper[5052]: I1203 12:12:21.696676 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8474b807-15d5-405a-ade8-1e4fa8954ce6-kube-api-access-b6l42" (OuterVolumeSpecName: "kube-api-access-b6l42") pod "8474b807-15d5-405a-ade8-1e4fa8954ce6" (UID: "8474b807-15d5-405a-ade8-1e4fa8954ce6"). InnerVolumeSpecName "kube-api-access-b6l42". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:12:21 crc kubenswrapper[5052]: I1203 12:12:21.720965 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8474b807-15d5-405a-ade8-1e4fa8954ce6-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "8474b807-15d5-405a-ade8-1e4fa8954ce6" (UID: "8474b807-15d5-405a-ade8-1e4fa8954ce6"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:12:21 crc kubenswrapper[5052]: I1203 12:12:21.725864 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8474b807-15d5-405a-ade8-1e4fa8954ce6-config" (OuterVolumeSpecName: "config") pod "8474b807-15d5-405a-ade8-1e4fa8954ce6" (UID: "8474b807-15d5-405a-ade8-1e4fa8954ce6"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:12:21 crc kubenswrapper[5052]: I1203 12:12:21.793833 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8474b807-15d5-405a-ade8-1e4fa8954ce6-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:12:21 crc kubenswrapper[5052]: I1203 12:12:21.793881 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-b6l42\" (UniqueName: \"kubernetes.io/projected/8474b807-15d5-405a-ade8-1e4fa8954ce6-kube-api-access-b6l42\") on node \"crc\" DevicePath \"\"" Dec 03 12:12:21 crc kubenswrapper[5052]: I1203 12:12:21.793894 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/8474b807-15d5-405a-ade8-1e4fa8954ce6-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:12:21 crc kubenswrapper[5052]: I1203 12:12:21.960079 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/openstackclient"] Dec 03 12:12:21 crc kubenswrapper[5052]: E1203 12:12:21.961115 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="8474b807-15d5-405a-ade8-1e4fa8954ce6" containerName="neutron-db-sync" Dec 03 12:12:21 crc kubenswrapper[5052]: I1203 12:12:21.961133 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="8474b807-15d5-405a-ade8-1e4fa8954ce6" containerName="neutron-db-sync" Dec 03 12:12:21 crc kubenswrapper[5052]: E1203 12:12:21.961171 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="ba056586-628e-45da-852f-9fc1736ff76f" containerName="barbican-api-log" Dec 03 12:12:21 crc kubenswrapper[5052]: I1203 12:12:21.961178 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="ba056586-628e-45da-852f-9fc1736ff76f" containerName="barbican-api-log" Dec 03 12:12:21 crc kubenswrapper[5052]: E1203 12:12:21.961211 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="ba056586-628e-45da-852f-9fc1736ff76f" containerName="barbican-api" Dec 03 12:12:21 crc kubenswrapper[5052]: I1203 12:12:21.961217 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="ba056586-628e-45da-852f-9fc1736ff76f" containerName="barbican-api" Dec 03 12:12:21 crc kubenswrapper[5052]: I1203 12:12:21.961387 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="8474b807-15d5-405a-ade8-1e4fa8954ce6" containerName="neutron-db-sync" Dec 03 12:12:21 crc kubenswrapper[5052]: I1203 12:12:21.961400 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="ba056586-628e-45da-852f-9fc1736ff76f" containerName="barbican-api" Dec 03 12:12:21 crc kubenswrapper[5052]: I1203 12:12:21.961423 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="ba056586-628e-45da-852f-9fc1736ff76f" containerName="barbican-api-log" Dec 03 12:12:21 crc kubenswrapper[5052]: I1203 12:12:21.962554 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstackclient" Dec 03 12:12:21 crc kubenswrapper[5052]: I1203 12:12:21.965552 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"openstack-config" Dec 03 12:12:21 crc kubenswrapper[5052]: I1203 12:12:21.965948 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"openstack-config-secret" Dec 03 12:12:21 crc kubenswrapper[5052]: I1203 12:12:21.965961 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"openstackclient-openstackclient-dockercfg-tzkfl" Dec 03 12:12:21 crc kubenswrapper[5052]: I1203 12:12:21.989404 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/openstackclient"] Dec 03 12:12:22 crc kubenswrapper[5052]: I1203 12:12:22.098305 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openstack-config\" (UniqueName: \"kubernetes.io/configmap/4e574f0c-bbe2-44a9-935c-923622c44443-openstack-config\") pod \"openstackclient\" (UID: \"4e574f0c-bbe2-44a9-935c-923622c44443\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 12:12:22 crc kubenswrapper[5052]: I1203 12:12:22.098554 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4e574f0c-bbe2-44a9-935c-923622c44443-combined-ca-bundle\") pod \"openstackclient\" (UID: \"4e574f0c-bbe2-44a9-935c-923622c44443\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 12:12:22 crc kubenswrapper[5052]: I1203 12:12:22.098638 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openstack-config-secret\" (UniqueName: \"kubernetes.io/secret/4e574f0c-bbe2-44a9-935c-923622c44443-openstack-config-secret\") pod \"openstackclient\" (UID: \"4e574f0c-bbe2-44a9-935c-923622c44443\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 12:12:22 crc kubenswrapper[5052]: I1203 12:12:22.098681 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m2tvf\" (UniqueName: \"kubernetes.io/projected/4e574f0c-bbe2-44a9-935c-923622c44443-kube-api-access-m2tvf\") pod \"openstackclient\" (UID: \"4e574f0c-bbe2-44a9-935c-923622c44443\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 12:12:22 crc kubenswrapper[5052]: I1203 12:12:22.158453 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-db-sync-cbdcz" event={"ID":"8474b807-15d5-405a-ade8-1e4fa8954ce6","Type":"ContainerDied","Data":"6bf8e70864db28e932a6d4b2efdfe6e9d604e6d6b4bf97d23af8ac4a2ee431c5"} Dec 03 12:12:22 crc kubenswrapper[5052]: I1203 12:12:22.158486 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-db-sync-cbdcz" Dec 03 12:12:22 crc kubenswrapper[5052]: I1203 12:12:22.158496 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6bf8e70864db28e932a6d4b2efdfe6e9d604e6d6b4bf97d23af8ac4a2ee431c5" Dec 03 12:12:22 crc kubenswrapper[5052]: I1203 12:12:22.162745 5052 generic.go:334] "Generic (PLEG): container finished" podID="63ba4f58-6d06-452e-86b7-54a61a998c03" containerID="4cb6e9620ad9bca18eb34b849b06c51b05cc9bee37b75d985fde32193b027d50" exitCode=0 Dec 03 12:12:22 crc kubenswrapper[5052]: I1203 12:12:22.162789 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"63ba4f58-6d06-452e-86b7-54a61a998c03","Type":"ContainerDied","Data":"4cb6e9620ad9bca18eb34b849b06c51b05cc9bee37b75d985fde32193b027d50"} Dec 03 12:12:22 crc kubenswrapper[5052]: I1203 12:12:22.200887 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"openstack-config\" (UniqueName: \"kubernetes.io/configmap/4e574f0c-bbe2-44a9-935c-923622c44443-openstack-config\") pod \"openstackclient\" (UID: \"4e574f0c-bbe2-44a9-935c-923622c44443\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 12:12:22 crc kubenswrapper[5052]: I1203 12:12:22.201071 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4e574f0c-bbe2-44a9-935c-923622c44443-combined-ca-bundle\") pod \"openstackclient\" (UID: \"4e574f0c-bbe2-44a9-935c-923622c44443\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 12:12:22 crc kubenswrapper[5052]: I1203 12:12:22.201233 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"openstack-config-secret\" (UniqueName: \"kubernetes.io/secret/4e574f0c-bbe2-44a9-935c-923622c44443-openstack-config-secret\") pod \"openstackclient\" (UID: \"4e574f0c-bbe2-44a9-935c-923622c44443\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 12:12:22 crc kubenswrapper[5052]: I1203 12:12:22.201263 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-m2tvf\" (UniqueName: \"kubernetes.io/projected/4e574f0c-bbe2-44a9-935c-923622c44443-kube-api-access-m2tvf\") pod \"openstackclient\" (UID: \"4e574f0c-bbe2-44a9-935c-923622c44443\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 12:12:22 crc kubenswrapper[5052]: I1203 12:12:22.204986 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"openstack-config\" (UniqueName: \"kubernetes.io/configmap/4e574f0c-bbe2-44a9-935c-923622c44443-openstack-config\") pod \"openstackclient\" (UID: \"4e574f0c-bbe2-44a9-935c-923622c44443\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 12:12:22 crc kubenswrapper[5052]: I1203 12:12:22.207574 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"openstack-config-secret\" (UniqueName: \"kubernetes.io/secret/4e574f0c-bbe2-44a9-935c-923622c44443-openstack-config-secret\") pod \"openstackclient\" (UID: \"4e574f0c-bbe2-44a9-935c-923622c44443\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 12:12:22 crc kubenswrapper[5052]: I1203 12:12:22.207959 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4e574f0c-bbe2-44a9-935c-923622c44443-combined-ca-bundle\") pod \"openstackclient\" (UID: \"4e574f0c-bbe2-44a9-935c-923622c44443\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 12:12:22 crc kubenswrapper[5052]: I1203 12:12:22.225484 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-m2tvf\" (UniqueName: \"kubernetes.io/projected/4e574f0c-bbe2-44a9-935c-923622c44443-kube-api-access-m2tvf\") pod \"openstackclient\" (UID: \"4e574f0c-bbe2-44a9-935c-923622c44443\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 12:12:22 crc kubenswrapper[5052]: I1203 12:12:22.283169 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:12:22 crc kubenswrapper[5052]: I1203 12:12:22.295071 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstackclient" Dec 03 12:12:22 crc kubenswrapper[5052]: I1203 12:12:22.359342 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/neutron-74495c68d7-59m75"] Dec 03 12:12:22 crc kubenswrapper[5052]: E1203 12:12:22.359750 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="63ba4f58-6d06-452e-86b7-54a61a998c03" containerName="cinder-scheduler" Dec 03 12:12:22 crc kubenswrapper[5052]: I1203 12:12:22.359762 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="63ba4f58-6d06-452e-86b7-54a61a998c03" containerName="cinder-scheduler" Dec 03 12:12:22 crc kubenswrapper[5052]: E1203 12:12:22.359797 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="63ba4f58-6d06-452e-86b7-54a61a998c03" containerName="probe" Dec 03 12:12:22 crc kubenswrapper[5052]: I1203 12:12:22.359803 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="63ba4f58-6d06-452e-86b7-54a61a998c03" containerName="probe" Dec 03 12:12:22 crc kubenswrapper[5052]: I1203 12:12:22.360158 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="63ba4f58-6d06-452e-86b7-54a61a998c03" containerName="cinder-scheduler" Dec 03 12:12:22 crc kubenswrapper[5052]: I1203 12:12:22.360173 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="63ba4f58-6d06-452e-86b7-54a61a998c03" containerName="probe" Dec 03 12:12:22 crc kubenswrapper[5052]: I1203 12:12:22.369769 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-74495c68d7-59m75" Dec 03 12:12:22 crc kubenswrapper[5052]: I1203 12:12:22.372722 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-neutron-ovndbs" Dec 03 12:12:22 crc kubenswrapper[5052]: I1203 12:12:22.372939 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"neutron-neutron-dockercfg-s5qsh" Dec 03 12:12:22 crc kubenswrapper[5052]: I1203 12:12:22.373755 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"neutron-config" Dec 03 12:12:22 crc kubenswrapper[5052]: I1203 12:12:22.373949 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"neutron-httpd-config" Dec 03 12:12:22 crc kubenswrapper[5052]: I1203 12:12:22.391111 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-74495c68d7-59m75"] Dec 03 12:12:22 crc kubenswrapper[5052]: I1203 12:12:22.404957 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/63ba4f58-6d06-452e-86b7-54a61a998c03-scripts\") pod \"63ba4f58-6d06-452e-86b7-54a61a998c03\" (UID: \"63ba4f58-6d06-452e-86b7-54a61a998c03\") " Dec 03 12:12:22 crc kubenswrapper[5052]: I1203 12:12:22.405446 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/63ba4f58-6d06-452e-86b7-54a61a998c03-etc-machine-id\") pod \"63ba4f58-6d06-452e-86b7-54a61a998c03\" (UID: \"63ba4f58-6d06-452e-86b7-54a61a998c03\") " Dec 03 12:12:22 crc kubenswrapper[5052]: I1203 12:12:22.405954 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j7zp2\" (UniqueName: \"kubernetes.io/projected/63ba4f58-6d06-452e-86b7-54a61a998c03-kube-api-access-j7zp2\") pod \"63ba4f58-6d06-452e-86b7-54a61a998c03\" (UID: \"63ba4f58-6d06-452e-86b7-54a61a998c03\") " Dec 03 12:12:22 crc kubenswrapper[5052]: I1203 12:12:22.406125 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/63ba4f58-6d06-452e-86b7-54a61a998c03-combined-ca-bundle\") pod \"63ba4f58-6d06-452e-86b7-54a61a998c03\" (UID: \"63ba4f58-6d06-452e-86b7-54a61a998c03\") " Dec 03 12:12:22 crc kubenswrapper[5052]: I1203 12:12:22.406360 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/63ba4f58-6d06-452e-86b7-54a61a998c03-config-data-custom\") pod \"63ba4f58-6d06-452e-86b7-54a61a998c03\" (UID: \"63ba4f58-6d06-452e-86b7-54a61a998c03\") " Dec 03 12:12:22 crc kubenswrapper[5052]: I1203 12:12:22.406621 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/63ba4f58-6d06-452e-86b7-54a61a998c03-config-data\") pod \"63ba4f58-6d06-452e-86b7-54a61a998c03\" (UID: \"63ba4f58-6d06-452e-86b7-54a61a998c03\") " Dec 03 12:12:22 crc kubenswrapper[5052]: I1203 12:12:22.408730 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/63ba4f58-6d06-452e-86b7-54a61a998c03-etc-machine-id" (OuterVolumeSpecName: "etc-machine-id") pod "63ba4f58-6d06-452e-86b7-54a61a998c03" (UID: "63ba4f58-6d06-452e-86b7-54a61a998c03"). InnerVolumeSpecName "etc-machine-id". PluginName "kubernetes.io/host-path", VolumeGidValue "" Dec 03 12:12:22 crc kubenswrapper[5052]: I1203 12:12:22.411001 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/63ba4f58-6d06-452e-86b7-54a61a998c03-scripts" (OuterVolumeSpecName: "scripts") pod "63ba4f58-6d06-452e-86b7-54a61a998c03" (UID: "63ba4f58-6d06-452e-86b7-54a61a998c03"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:12:22 crc kubenswrapper[5052]: I1203 12:12:22.421728 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/63ba4f58-6d06-452e-86b7-54a61a998c03-kube-api-access-j7zp2" (OuterVolumeSpecName: "kube-api-access-j7zp2") pod "63ba4f58-6d06-452e-86b7-54a61a998c03" (UID: "63ba4f58-6d06-452e-86b7-54a61a998c03"). InnerVolumeSpecName "kube-api-access-j7zp2". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:12:22 crc kubenswrapper[5052]: I1203 12:12:22.433743 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/63ba4f58-6d06-452e-86b7-54a61a998c03-config-data-custom" (OuterVolumeSpecName: "config-data-custom") pod "63ba4f58-6d06-452e-86b7-54a61a998c03" (UID: "63ba4f58-6d06-452e-86b7-54a61a998c03"). InnerVolumeSpecName "config-data-custom". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:12:22 crc kubenswrapper[5052]: I1203 12:12:22.498439 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/63ba4f58-6d06-452e-86b7-54a61a998c03-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "63ba4f58-6d06-452e-86b7-54a61a998c03" (UID: "63ba4f58-6d06-452e-86b7-54a61a998c03"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:12:22 crc kubenswrapper[5052]: I1203 12:12:22.510305 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/49095880-d707-4afb-a3cc-a10ece6970de-httpd-config\") pod \"neutron-74495c68d7-59m75\" (UID: \"49095880-d707-4afb-a3cc-a10ece6970de\") " pod="openstack-kuttl-tests/neutron-74495c68d7-59m75" Dec 03 12:12:22 crc kubenswrapper[5052]: I1203 12:12:22.510363 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/49095880-d707-4afb-a3cc-a10ece6970de-config\") pod \"neutron-74495c68d7-59m75\" (UID: \"49095880-d707-4afb-a3cc-a10ece6970de\") " pod="openstack-kuttl-tests/neutron-74495c68d7-59m75" Dec 03 12:12:22 crc kubenswrapper[5052]: I1203 12:12:22.510425 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mgm9k\" (UniqueName: \"kubernetes.io/projected/49095880-d707-4afb-a3cc-a10ece6970de-kube-api-access-mgm9k\") pod \"neutron-74495c68d7-59m75\" (UID: \"49095880-d707-4afb-a3cc-a10ece6970de\") " pod="openstack-kuttl-tests/neutron-74495c68d7-59m75" Dec 03 12:12:22 crc kubenswrapper[5052]: I1203 12:12:22.510448 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/49095880-d707-4afb-a3cc-a10ece6970de-combined-ca-bundle\") pod \"neutron-74495c68d7-59m75\" (UID: \"49095880-d707-4afb-a3cc-a10ece6970de\") " pod="openstack-kuttl-tests/neutron-74495c68d7-59m75" Dec 03 12:12:22 crc kubenswrapper[5052]: I1203 12:12:22.510521 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/49095880-d707-4afb-a3cc-a10ece6970de-ovndb-tls-certs\") pod \"neutron-74495c68d7-59m75\" (UID: \"49095880-d707-4afb-a3cc-a10ece6970de\") " pod="openstack-kuttl-tests/neutron-74495c68d7-59m75" Dec 03 12:12:22 crc kubenswrapper[5052]: I1203 12:12:22.510581 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-j7zp2\" (UniqueName: \"kubernetes.io/projected/63ba4f58-6d06-452e-86b7-54a61a998c03-kube-api-access-j7zp2\") on node \"crc\" DevicePath \"\"" Dec 03 12:12:22 crc kubenswrapper[5052]: I1203 12:12:22.510594 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/63ba4f58-6d06-452e-86b7-54a61a998c03-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:12:22 crc kubenswrapper[5052]: I1203 12:12:22.510605 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/63ba4f58-6d06-452e-86b7-54a61a998c03-config-data-custom\") on node \"crc\" DevicePath \"\"" Dec 03 12:12:22 crc kubenswrapper[5052]: I1203 12:12:22.510615 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/63ba4f58-6d06-452e-86b7-54a61a998c03-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:12:22 crc kubenswrapper[5052]: I1203 12:12:22.510624 5052 reconciler_common.go:293] "Volume detached for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/63ba4f58-6d06-452e-86b7-54a61a998c03-etc-machine-id\") on node \"crc\" DevicePath \"\"" Dec 03 12:12:22 crc kubenswrapper[5052]: I1203 12:12:22.614298 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-mgm9k\" (UniqueName: \"kubernetes.io/projected/49095880-d707-4afb-a3cc-a10ece6970de-kube-api-access-mgm9k\") pod \"neutron-74495c68d7-59m75\" (UID: \"49095880-d707-4afb-a3cc-a10ece6970de\") " pod="openstack-kuttl-tests/neutron-74495c68d7-59m75" Dec 03 12:12:22 crc kubenswrapper[5052]: I1203 12:12:22.614383 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/49095880-d707-4afb-a3cc-a10ece6970de-combined-ca-bundle\") pod \"neutron-74495c68d7-59m75\" (UID: \"49095880-d707-4afb-a3cc-a10ece6970de\") " pod="openstack-kuttl-tests/neutron-74495c68d7-59m75" Dec 03 12:12:22 crc kubenswrapper[5052]: I1203 12:12:22.614550 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/49095880-d707-4afb-a3cc-a10ece6970de-ovndb-tls-certs\") pod \"neutron-74495c68d7-59m75\" (UID: \"49095880-d707-4afb-a3cc-a10ece6970de\") " pod="openstack-kuttl-tests/neutron-74495c68d7-59m75" Dec 03 12:12:22 crc kubenswrapper[5052]: I1203 12:12:22.614796 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/49095880-d707-4afb-a3cc-a10ece6970de-httpd-config\") pod \"neutron-74495c68d7-59m75\" (UID: \"49095880-d707-4afb-a3cc-a10ece6970de\") " pod="openstack-kuttl-tests/neutron-74495c68d7-59m75" Dec 03 12:12:22 crc kubenswrapper[5052]: I1203 12:12:22.614871 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/49095880-d707-4afb-a3cc-a10ece6970de-config\") pod \"neutron-74495c68d7-59m75\" (UID: \"49095880-d707-4afb-a3cc-a10ece6970de\") " pod="openstack-kuttl-tests/neutron-74495c68d7-59m75" Dec 03 12:12:22 crc kubenswrapper[5052]: I1203 12:12:22.621741 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/49095880-d707-4afb-a3cc-a10ece6970de-httpd-config\") pod \"neutron-74495c68d7-59m75\" (UID: \"49095880-d707-4afb-a3cc-a10ece6970de\") " pod="openstack-kuttl-tests/neutron-74495c68d7-59m75" Dec 03 12:12:22 crc kubenswrapper[5052]: I1203 12:12:22.621997 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/49095880-d707-4afb-a3cc-a10ece6970de-combined-ca-bundle\") pod \"neutron-74495c68d7-59m75\" (UID: \"49095880-d707-4afb-a3cc-a10ece6970de\") " pod="openstack-kuttl-tests/neutron-74495c68d7-59m75" Dec 03 12:12:22 crc kubenswrapper[5052]: I1203 12:12:22.622596 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/63ba4f58-6d06-452e-86b7-54a61a998c03-config-data" (OuterVolumeSpecName: "config-data") pod "63ba4f58-6d06-452e-86b7-54a61a998c03" (UID: "63ba4f58-6d06-452e-86b7-54a61a998c03"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:12:22 crc kubenswrapper[5052]: I1203 12:12:22.622777 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/49095880-d707-4afb-a3cc-a10ece6970de-ovndb-tls-certs\") pod \"neutron-74495c68d7-59m75\" (UID: \"49095880-d707-4afb-a3cc-a10ece6970de\") " pod="openstack-kuttl-tests/neutron-74495c68d7-59m75" Dec 03 12:12:22 crc kubenswrapper[5052]: I1203 12:12:22.633668 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/49095880-d707-4afb-a3cc-a10ece6970de-config\") pod \"neutron-74495c68d7-59m75\" (UID: \"49095880-d707-4afb-a3cc-a10ece6970de\") " pod="openstack-kuttl-tests/neutron-74495c68d7-59m75" Dec 03 12:12:22 crc kubenswrapper[5052]: I1203 12:12:22.634298 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-mgm9k\" (UniqueName: \"kubernetes.io/projected/49095880-d707-4afb-a3cc-a10ece6970de-kube-api-access-mgm9k\") pod \"neutron-74495c68d7-59m75\" (UID: \"49095880-d707-4afb-a3cc-a10ece6970de\") " pod="openstack-kuttl-tests/neutron-74495c68d7-59m75" Dec 03 12:12:22 crc kubenswrapper[5052]: I1203 12:12:22.658848 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-74495c68d7-59m75" Dec 03 12:12:22 crc kubenswrapper[5052]: I1203 12:12:22.717074 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/63ba4f58-6d06-452e-86b7-54a61a998c03-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:12:23 crc kubenswrapper[5052]: I1203 12:12:23.490957 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-marketplace/community-operators-q4qt9" Dec 03 12:12:23 crc kubenswrapper[5052]: I1203 12:12:23.492377 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openshift-marketplace/community-operators-q4qt9" Dec 03 12:12:23 crc kubenswrapper[5052]: I1203 12:12:23.522782 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"63ba4f58-6d06-452e-86b7-54a61a998c03","Type":"ContainerDied","Data":"ee37db74a901acb1bf8dbf41bbb26a7caa8487a8005ee0d96c5dcdfc997cd020"} Dec 03 12:12:23 crc kubenswrapper[5052]: I1203 12:12:23.522855 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/openstackclient"] Dec 03 12:12:23 crc kubenswrapper[5052]: I1203 12:12:23.522885 5052 scope.go:117] "RemoveContainer" containerID="2a7d4aa75fed7462d0c0db261e38e723cb83192e669ec7409e8c399c0ea325ee" Dec 03 12:12:23 crc kubenswrapper[5052]: I1203 12:12:23.522903 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:12:23 crc kubenswrapper[5052]: I1203 12:12:23.581019 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-marketplace/community-operators-q4qt9" Dec 03 12:12:23 crc kubenswrapper[5052]: I1203 12:12:23.608183 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-74495c68d7-59m75"] Dec 03 12:12:23 crc kubenswrapper[5052]: I1203 12:12:23.758718 5052 scope.go:117] "RemoveContainer" containerID="4cb6e9620ad9bca18eb34b849b06c51b05cc9bee37b75d985fde32193b027d50" Dec 03 12:12:23 crc kubenswrapper[5052]: I1203 12:12:23.802733 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 12:12:23 crc kubenswrapper[5052]: I1203 12:12:23.815345 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 12:12:23 crc kubenswrapper[5052]: I1203 12:12:23.833158 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 12:12:23 crc kubenswrapper[5052]: I1203 12:12:23.844575 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:12:23 crc kubenswrapper[5052]: I1203 12:12:23.853614 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cinder-scheduler-config-data" Dec 03 12:12:23 crc kubenswrapper[5052]: I1203 12:12:23.873103 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 12:12:23 crc kubenswrapper[5052]: I1203 12:12:23.918219 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/89b056a7-e448-447d-be7d-b69f3d4862a1-config-data-custom\") pod \"cinder-scheduler-0\" (UID: \"89b056a7-e448-447d-be7d-b69f3d4862a1\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:12:23 crc kubenswrapper[5052]: I1203 12:12:23.918328 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/89b056a7-e448-447d-be7d-b69f3d4862a1-combined-ca-bundle\") pod \"cinder-scheduler-0\" (UID: \"89b056a7-e448-447d-be7d-b69f3d4862a1\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:12:23 crc kubenswrapper[5052]: I1203 12:12:23.918393 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/89b056a7-e448-447d-be7d-b69f3d4862a1-scripts\") pod \"cinder-scheduler-0\" (UID: \"89b056a7-e448-447d-be7d-b69f3d4862a1\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:12:23 crc kubenswrapper[5052]: I1203 12:12:23.918498 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/89b056a7-e448-447d-be7d-b69f3d4862a1-config-data\") pod \"cinder-scheduler-0\" (UID: \"89b056a7-e448-447d-be7d-b69f3d4862a1\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:12:23 crc kubenswrapper[5052]: I1203 12:12:23.918847 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pfjjz\" (UniqueName: \"kubernetes.io/projected/89b056a7-e448-447d-be7d-b69f3d4862a1-kube-api-access-pfjjz\") pod \"cinder-scheduler-0\" (UID: \"89b056a7-e448-447d-be7d-b69f3d4862a1\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:12:23 crc kubenswrapper[5052]: I1203 12:12:23.918911 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/89b056a7-e448-447d-be7d-b69f3d4862a1-etc-machine-id\") pod \"cinder-scheduler-0\" (UID: \"89b056a7-e448-447d-be7d-b69f3d4862a1\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:12:24 crc kubenswrapper[5052]: I1203 12:12:24.021227 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/89b056a7-e448-447d-be7d-b69f3d4862a1-etc-machine-id\") pod \"cinder-scheduler-0\" (UID: \"89b056a7-e448-447d-be7d-b69f3d4862a1\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:12:24 crc kubenswrapper[5052]: I1203 12:12:24.021338 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/89b056a7-e448-447d-be7d-b69f3d4862a1-config-data-custom\") pod \"cinder-scheduler-0\" (UID: \"89b056a7-e448-447d-be7d-b69f3d4862a1\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:12:24 crc kubenswrapper[5052]: I1203 12:12:24.021402 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/89b056a7-e448-447d-be7d-b69f3d4862a1-combined-ca-bundle\") pod \"cinder-scheduler-0\" (UID: \"89b056a7-e448-447d-be7d-b69f3d4862a1\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:12:24 crc kubenswrapper[5052]: I1203 12:12:24.021437 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/89b056a7-e448-447d-be7d-b69f3d4862a1-scripts\") pod \"cinder-scheduler-0\" (UID: \"89b056a7-e448-447d-be7d-b69f3d4862a1\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:12:24 crc kubenswrapper[5052]: I1203 12:12:24.021459 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/89b056a7-e448-447d-be7d-b69f3d4862a1-config-data\") pod \"cinder-scheduler-0\" (UID: \"89b056a7-e448-447d-be7d-b69f3d4862a1\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:12:24 crc kubenswrapper[5052]: I1203 12:12:24.021569 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-pfjjz\" (UniqueName: \"kubernetes.io/projected/89b056a7-e448-447d-be7d-b69f3d4862a1-kube-api-access-pfjjz\") pod \"cinder-scheduler-0\" (UID: \"89b056a7-e448-447d-be7d-b69f3d4862a1\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:12:24 crc kubenswrapper[5052]: I1203 12:12:24.021794 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/89b056a7-e448-447d-be7d-b69f3d4862a1-etc-machine-id\") pod \"cinder-scheduler-0\" (UID: \"89b056a7-e448-447d-be7d-b69f3d4862a1\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:12:24 crc kubenswrapper[5052]: I1203 12:12:24.026307 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/89b056a7-e448-447d-be7d-b69f3d4862a1-scripts\") pod \"cinder-scheduler-0\" (UID: \"89b056a7-e448-447d-be7d-b69f3d4862a1\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:12:24 crc kubenswrapper[5052]: I1203 12:12:24.027454 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/89b056a7-e448-447d-be7d-b69f3d4862a1-config-data\") pod \"cinder-scheduler-0\" (UID: \"89b056a7-e448-447d-be7d-b69f3d4862a1\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:12:24 crc kubenswrapper[5052]: I1203 12:12:24.027551 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/89b056a7-e448-447d-be7d-b69f3d4862a1-combined-ca-bundle\") pod \"cinder-scheduler-0\" (UID: \"89b056a7-e448-447d-be7d-b69f3d4862a1\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:12:24 crc kubenswrapper[5052]: I1203 12:12:24.028617 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/89b056a7-e448-447d-be7d-b69f3d4862a1-config-data-custom\") pod \"cinder-scheduler-0\" (UID: \"89b056a7-e448-447d-be7d-b69f3d4862a1\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:12:24 crc kubenswrapper[5052]: I1203 12:12:24.051682 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-pfjjz\" (UniqueName: \"kubernetes.io/projected/89b056a7-e448-447d-be7d-b69f3d4862a1-kube-api-access-pfjjz\") pod \"cinder-scheduler-0\" (UID: \"89b056a7-e448-447d-be7d-b69f3d4862a1\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:12:24 crc kubenswrapper[5052]: I1203 12:12:24.190024 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:12:24 crc kubenswrapper[5052]: I1203 12:12:24.535633 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstackclient" event={"ID":"4e574f0c-bbe2-44a9-935c-923622c44443","Type":"ContainerStarted","Data":"e87fc441a48ac46737cf3f11739a69de3c29e3f8904ec764a0ffe9819beb9685"} Dec 03 12:12:24 crc kubenswrapper[5052]: I1203 12:12:24.542961 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-74495c68d7-59m75" event={"ID":"49095880-d707-4afb-a3cc-a10ece6970de","Type":"ContainerStarted","Data":"500a6bc23532192b4a1634e44580baa95afa6bef3031a94faf97ea7415c74a2c"} Dec 03 12:12:24 crc kubenswrapper[5052]: I1203 12:12:24.590956 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-marketplace/community-operators-q4qt9" Dec 03 12:12:24 crc kubenswrapper[5052]: I1203 12:12:24.740470 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 12:12:24 crc kubenswrapper[5052]: I1203 12:12:24.909510 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/neutron-5b979cdcb6-6hqqv"] Dec 03 12:12:24 crc kubenswrapper[5052]: I1203 12:12:24.911667 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-5b979cdcb6-6hqqv" Dec 03 12:12:24 crc kubenswrapper[5052]: I1203 12:12:24.916017 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-neutron-internal-svc" Dec 03 12:12:24 crc kubenswrapper[5052]: I1203 12:12:24.916360 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-neutron-public-svc" Dec 03 12:12:24 crc kubenswrapper[5052]: I1203 12:12:24.924148 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-5b979cdcb6-6hqqv"] Dec 03 12:12:24 crc kubenswrapper[5052]: I1203 12:12:24.939145 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/718e2899-d5c3-41a4-a2f9-4830e18355d3-ovndb-tls-certs\") pod \"neutron-5b979cdcb6-6hqqv\" (UID: \"718e2899-d5c3-41a4-a2f9-4830e18355d3\") " pod="openstack-kuttl-tests/neutron-5b979cdcb6-6hqqv" Dec 03 12:12:24 crc kubenswrapper[5052]: I1203 12:12:24.939280 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/718e2899-d5c3-41a4-a2f9-4830e18355d3-internal-tls-certs\") pod \"neutron-5b979cdcb6-6hqqv\" (UID: \"718e2899-d5c3-41a4-a2f9-4830e18355d3\") " pod="openstack-kuttl-tests/neutron-5b979cdcb6-6hqqv" Dec 03 12:12:24 crc kubenswrapper[5052]: I1203 12:12:24.939361 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/718e2899-d5c3-41a4-a2f9-4830e18355d3-public-tls-certs\") pod \"neutron-5b979cdcb6-6hqqv\" (UID: \"718e2899-d5c3-41a4-a2f9-4830e18355d3\") " pod="openstack-kuttl-tests/neutron-5b979cdcb6-6hqqv" Dec 03 12:12:24 crc kubenswrapper[5052]: I1203 12:12:24.939392 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pvvtk\" (UniqueName: \"kubernetes.io/projected/718e2899-d5c3-41a4-a2f9-4830e18355d3-kube-api-access-pvvtk\") pod \"neutron-5b979cdcb6-6hqqv\" (UID: \"718e2899-d5c3-41a4-a2f9-4830e18355d3\") " pod="openstack-kuttl-tests/neutron-5b979cdcb6-6hqqv" Dec 03 12:12:24 crc kubenswrapper[5052]: I1203 12:12:24.939524 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/718e2899-d5c3-41a4-a2f9-4830e18355d3-config\") pod \"neutron-5b979cdcb6-6hqqv\" (UID: \"718e2899-d5c3-41a4-a2f9-4830e18355d3\") " pod="openstack-kuttl-tests/neutron-5b979cdcb6-6hqqv" Dec 03 12:12:24 crc kubenswrapper[5052]: I1203 12:12:24.939574 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/718e2899-d5c3-41a4-a2f9-4830e18355d3-combined-ca-bundle\") pod \"neutron-5b979cdcb6-6hqqv\" (UID: \"718e2899-d5c3-41a4-a2f9-4830e18355d3\") " pod="openstack-kuttl-tests/neutron-5b979cdcb6-6hqqv" Dec 03 12:12:24 crc kubenswrapper[5052]: I1203 12:12:24.939608 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/718e2899-d5c3-41a4-a2f9-4830e18355d3-httpd-config\") pod \"neutron-5b979cdcb6-6hqqv\" (UID: \"718e2899-d5c3-41a4-a2f9-4830e18355d3\") " pod="openstack-kuttl-tests/neutron-5b979cdcb6-6hqqv" Dec 03 12:12:25 crc kubenswrapper[5052]: I1203 12:12:25.047769 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/718e2899-d5c3-41a4-a2f9-4830e18355d3-ovndb-tls-certs\") pod \"neutron-5b979cdcb6-6hqqv\" (UID: \"718e2899-d5c3-41a4-a2f9-4830e18355d3\") " pod="openstack-kuttl-tests/neutron-5b979cdcb6-6hqqv" Dec 03 12:12:25 crc kubenswrapper[5052]: I1203 12:12:25.047844 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/718e2899-d5c3-41a4-a2f9-4830e18355d3-internal-tls-certs\") pod \"neutron-5b979cdcb6-6hqqv\" (UID: \"718e2899-d5c3-41a4-a2f9-4830e18355d3\") " pod="openstack-kuttl-tests/neutron-5b979cdcb6-6hqqv" Dec 03 12:12:25 crc kubenswrapper[5052]: I1203 12:12:25.047885 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/718e2899-d5c3-41a4-a2f9-4830e18355d3-public-tls-certs\") pod \"neutron-5b979cdcb6-6hqqv\" (UID: \"718e2899-d5c3-41a4-a2f9-4830e18355d3\") " pod="openstack-kuttl-tests/neutron-5b979cdcb6-6hqqv" Dec 03 12:12:25 crc kubenswrapper[5052]: I1203 12:12:25.047910 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-pvvtk\" (UniqueName: \"kubernetes.io/projected/718e2899-d5c3-41a4-a2f9-4830e18355d3-kube-api-access-pvvtk\") pod \"neutron-5b979cdcb6-6hqqv\" (UID: \"718e2899-d5c3-41a4-a2f9-4830e18355d3\") " pod="openstack-kuttl-tests/neutron-5b979cdcb6-6hqqv" Dec 03 12:12:25 crc kubenswrapper[5052]: I1203 12:12:25.047974 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/718e2899-d5c3-41a4-a2f9-4830e18355d3-config\") pod \"neutron-5b979cdcb6-6hqqv\" (UID: \"718e2899-d5c3-41a4-a2f9-4830e18355d3\") " pod="openstack-kuttl-tests/neutron-5b979cdcb6-6hqqv" Dec 03 12:12:25 crc kubenswrapper[5052]: I1203 12:12:25.048008 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/718e2899-d5c3-41a4-a2f9-4830e18355d3-combined-ca-bundle\") pod \"neutron-5b979cdcb6-6hqqv\" (UID: \"718e2899-d5c3-41a4-a2f9-4830e18355d3\") " pod="openstack-kuttl-tests/neutron-5b979cdcb6-6hqqv" Dec 03 12:12:25 crc kubenswrapper[5052]: I1203 12:12:25.048068 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/718e2899-d5c3-41a4-a2f9-4830e18355d3-httpd-config\") pod \"neutron-5b979cdcb6-6hqqv\" (UID: \"718e2899-d5c3-41a4-a2f9-4830e18355d3\") " pod="openstack-kuttl-tests/neutron-5b979cdcb6-6hqqv" Dec 03 12:12:25 crc kubenswrapper[5052]: I1203 12:12:25.055203 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/718e2899-d5c3-41a4-a2f9-4830e18355d3-public-tls-certs\") pod \"neutron-5b979cdcb6-6hqqv\" (UID: \"718e2899-d5c3-41a4-a2f9-4830e18355d3\") " pod="openstack-kuttl-tests/neutron-5b979cdcb6-6hqqv" Dec 03 12:12:25 crc kubenswrapper[5052]: I1203 12:12:25.057592 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/718e2899-d5c3-41a4-a2f9-4830e18355d3-ovndb-tls-certs\") pod \"neutron-5b979cdcb6-6hqqv\" (UID: \"718e2899-d5c3-41a4-a2f9-4830e18355d3\") " pod="openstack-kuttl-tests/neutron-5b979cdcb6-6hqqv" Dec 03 12:12:25 crc kubenswrapper[5052]: I1203 12:12:25.058172 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/718e2899-d5c3-41a4-a2f9-4830e18355d3-combined-ca-bundle\") pod \"neutron-5b979cdcb6-6hqqv\" (UID: \"718e2899-d5c3-41a4-a2f9-4830e18355d3\") " pod="openstack-kuttl-tests/neutron-5b979cdcb6-6hqqv" Dec 03 12:12:25 crc kubenswrapper[5052]: I1203 12:12:25.063764 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/718e2899-d5c3-41a4-a2f9-4830e18355d3-httpd-config\") pod \"neutron-5b979cdcb6-6hqqv\" (UID: \"718e2899-d5c3-41a4-a2f9-4830e18355d3\") " pod="openstack-kuttl-tests/neutron-5b979cdcb6-6hqqv" Dec 03 12:12:25 crc kubenswrapper[5052]: I1203 12:12:25.066536 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/718e2899-d5c3-41a4-a2f9-4830e18355d3-config\") pod \"neutron-5b979cdcb6-6hqqv\" (UID: \"718e2899-d5c3-41a4-a2f9-4830e18355d3\") " pod="openstack-kuttl-tests/neutron-5b979cdcb6-6hqqv" Dec 03 12:12:25 crc kubenswrapper[5052]: I1203 12:12:25.066875 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-pvvtk\" (UniqueName: \"kubernetes.io/projected/718e2899-d5c3-41a4-a2f9-4830e18355d3-kube-api-access-pvvtk\") pod \"neutron-5b979cdcb6-6hqqv\" (UID: \"718e2899-d5c3-41a4-a2f9-4830e18355d3\") " pod="openstack-kuttl-tests/neutron-5b979cdcb6-6hqqv" Dec 03 12:12:25 crc kubenswrapper[5052]: I1203 12:12:25.073890 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/718e2899-d5c3-41a4-a2f9-4830e18355d3-internal-tls-certs\") pod \"neutron-5b979cdcb6-6hqqv\" (UID: \"718e2899-d5c3-41a4-a2f9-4830e18355d3\") " pod="openstack-kuttl-tests/neutron-5b979cdcb6-6hqqv" Dec 03 12:12:25 crc kubenswrapper[5052]: I1203 12:12:25.279681 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-5b979cdcb6-6hqqv" Dec 03 12:12:25 crc kubenswrapper[5052]: I1203 12:12:25.556844 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"89b056a7-e448-447d-be7d-b69f3d4862a1","Type":"ContainerStarted","Data":"0404e3426375893df66b349094edc2e791298eba44431629b420076bbfc5fe1d"} Dec 03 12:12:25 crc kubenswrapper[5052]: I1203 12:12:25.688355 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="63ba4f58-6d06-452e-86b7-54a61a998c03" path="/var/lib/kubelet/pods/63ba4f58-6d06-452e-86b7-54a61a998c03/volumes" Dec 03 12:12:25 crc kubenswrapper[5052]: I1203 12:12:25.824676 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-5b979cdcb6-6hqqv"] Dec 03 12:12:25 crc kubenswrapper[5052]: I1203 12:12:25.942024 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openshift-marketplace/redhat-marketplace-p2mgk" Dec 03 12:12:25 crc kubenswrapper[5052]: I1203 12:12:25.943324 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-marketplace/redhat-marketplace-p2mgk" Dec 03 12:12:26 crc kubenswrapper[5052]: I1203 12:12:26.014485 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-marketplace/redhat-marketplace-p2mgk" Dec 03 12:12:26 crc kubenswrapper[5052]: I1203 12:12:26.023077 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:12:26 crc kubenswrapper[5052]: I1203 12:12:26.570171 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"89b056a7-e448-447d-be7d-b69f3d4862a1","Type":"ContainerStarted","Data":"6b5e16bdf4a9b75d7e426aac04d9e0f302e3e926580d5ea1db33ccce8cef8eb9"} Dec 03 12:12:26 crc kubenswrapper[5052]: I1203 12:12:26.572889 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstackclient" event={"ID":"4e574f0c-bbe2-44a9-935c-923622c44443","Type":"ContainerStarted","Data":"27087053cd65b0655720b99b7846639054cddedb51bef7c1cc746fabc39c1401"} Dec 03 12:12:26 crc kubenswrapper[5052]: I1203 12:12:26.577355 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-74495c68d7-59m75" event={"ID":"49095880-d707-4afb-a3cc-a10ece6970de","Type":"ContainerStarted","Data":"f94cbc489fbc2c2f8115d1902907b2ba57a8926f2d1730a50bf76e99d2e355cf"} Dec 03 12:12:26 crc kubenswrapper[5052]: I1203 12:12:26.577411 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-74495c68d7-59m75" event={"ID":"49095880-d707-4afb-a3cc-a10ece6970de","Type":"ContainerStarted","Data":"e5291229716bb69e07dd59d25b94079d3c162d2e091119be3de021794e666c7c"} Dec 03 12:12:26 crc kubenswrapper[5052]: I1203 12:12:26.577608 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/neutron-74495c68d7-59m75" Dec 03 12:12:26 crc kubenswrapper[5052]: I1203 12:12:26.579129 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-5b979cdcb6-6hqqv" event={"ID":"718e2899-d5c3-41a4-a2f9-4830e18355d3","Type":"ContainerStarted","Data":"8b2970b007db09e4b4c0137ca8209b26be8338c0d78012060389d6232ffbb06a"} Dec 03 12:12:26 crc kubenswrapper[5052]: I1203 12:12:26.579164 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-5b979cdcb6-6hqqv" event={"ID":"718e2899-d5c3-41a4-a2f9-4830e18355d3","Type":"ContainerStarted","Data":"005e86831435d920e9fdea739d977c3bb3d174fae83069f9fbb6d758792f23e1"} Dec 03 12:12:26 crc kubenswrapper[5052]: I1203 12:12:26.601973 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/openstackclient" podStartSLOduration=5.6019495070000005 podStartE2EDuration="5.601949507s" podCreationTimestamp="2025-12-03 12:12:21 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:12:26.587982078 +0000 UTC m=+3137.176223842" watchObservedRunningTime="2025-12-03 12:12:26.601949507 +0000 UTC m=+3137.190191271" Dec 03 12:12:26 crc kubenswrapper[5052]: I1203 12:12:26.611099 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/neutron-74495c68d7-59m75" podStartSLOduration=4.611081918 podStartE2EDuration="4.611081918s" podCreationTimestamp="2025-12-03 12:12:22 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:12:26.605850098 +0000 UTC m=+3137.194091892" watchObservedRunningTime="2025-12-03 12:12:26.611081918 +0000 UTC m=+3137.199323682" Dec 03 12:12:26 crc kubenswrapper[5052]: I1203 12:12:26.634755 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-marketplace/redhat-marketplace-p2mgk" Dec 03 12:12:27 crc kubenswrapper[5052]: I1203 12:12:27.387584 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openshift-marketplace/community-operators-q4qt9"] Dec 03 12:12:27 crc kubenswrapper[5052]: I1203 12:12:27.388123 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openshift-marketplace/community-operators-q4qt9" podUID="686067c2-dedc-4a04-a433-3df6e20f405d" containerName="registry-server" containerID="cri-o://63eeecd303bb8e2a2e060597115b296d7a1812ba2bd0989d4647e8fa974fe2c7" gracePeriod=2 Dec 03 12:12:27 crc kubenswrapper[5052]: I1203 12:12:27.604032 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-5b979cdcb6-6hqqv" event={"ID":"718e2899-d5c3-41a4-a2f9-4830e18355d3","Type":"ContainerStarted","Data":"84e05639993606274e28568021cb8fdaf723254416f7db2a86eb1f0830df3b64"} Dec 03 12:12:27 crc kubenswrapper[5052]: I1203 12:12:27.604482 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/neutron-5b979cdcb6-6hqqv" Dec 03 12:12:27 crc kubenswrapper[5052]: I1203 12:12:27.620939 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"89b056a7-e448-447d-be7d-b69f3d4862a1","Type":"ContainerStarted","Data":"ac8820d4342df6998e1122358c0ed26bd8debe54ca84f5a0d380bb533f8c00eb"} Dec 03 12:12:27 crc kubenswrapper[5052]: I1203 12:12:27.635270 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/neutron-5b979cdcb6-6hqqv" podStartSLOduration=3.635250661 podStartE2EDuration="3.635250661s" podCreationTimestamp="2025-12-03 12:12:24 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:12:27.626676096 +0000 UTC m=+3138.214917880" watchObservedRunningTime="2025-12-03 12:12:27.635250661 +0000 UTC m=+3138.223492425" Dec 03 12:12:27 crc kubenswrapper[5052]: I1203 12:12:27.644142 5052 generic.go:334] "Generic (PLEG): container finished" podID="686067c2-dedc-4a04-a433-3df6e20f405d" containerID="63eeecd303bb8e2a2e060597115b296d7a1812ba2bd0989d4647e8fa974fe2c7" exitCode=0 Dec 03 12:12:27 crc kubenswrapper[5052]: I1203 12:12:27.644701 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/community-operators-q4qt9" event={"ID":"686067c2-dedc-4a04-a433-3df6e20f405d","Type":"ContainerDied","Data":"63eeecd303bb8e2a2e060597115b296d7a1812ba2bd0989d4647e8fa974fe2c7"} Dec 03 12:12:27 crc kubenswrapper[5052]: I1203 12:12:27.668910 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/cinder-scheduler-0" podStartSLOduration=4.668885431 podStartE2EDuration="4.668885431s" podCreationTimestamp="2025-12-03 12:12:23 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:12:27.653157132 +0000 UTC m=+3138.241398896" watchObservedRunningTime="2025-12-03 12:12:27.668885431 +0000 UTC m=+3138.257127215" Dec 03 12:12:27 crc kubenswrapper[5052]: I1203 12:12:27.955727 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/community-operators-q4qt9" Dec 03 12:12:28 crc kubenswrapper[5052]: I1203 12:12:28.054836 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/686067c2-dedc-4a04-a433-3df6e20f405d-utilities\") pod \"686067c2-dedc-4a04-a433-3df6e20f405d\" (UID: \"686067c2-dedc-4a04-a433-3df6e20f405d\") " Dec 03 12:12:28 crc kubenswrapper[5052]: I1203 12:12:28.054954 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vxmx8\" (UniqueName: \"kubernetes.io/projected/686067c2-dedc-4a04-a433-3df6e20f405d-kube-api-access-vxmx8\") pod \"686067c2-dedc-4a04-a433-3df6e20f405d\" (UID: \"686067c2-dedc-4a04-a433-3df6e20f405d\") " Dec 03 12:12:28 crc kubenswrapper[5052]: I1203 12:12:28.055045 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/686067c2-dedc-4a04-a433-3df6e20f405d-catalog-content\") pod \"686067c2-dedc-4a04-a433-3df6e20f405d\" (UID: \"686067c2-dedc-4a04-a433-3df6e20f405d\") " Dec 03 12:12:28 crc kubenswrapper[5052]: I1203 12:12:28.055950 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/686067c2-dedc-4a04-a433-3df6e20f405d-utilities" (OuterVolumeSpecName: "utilities") pod "686067c2-dedc-4a04-a433-3df6e20f405d" (UID: "686067c2-dedc-4a04-a433-3df6e20f405d"). InnerVolumeSpecName "utilities". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:12:28 crc kubenswrapper[5052]: I1203 12:12:28.063815 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/686067c2-dedc-4a04-a433-3df6e20f405d-kube-api-access-vxmx8" (OuterVolumeSpecName: "kube-api-access-vxmx8") pod "686067c2-dedc-4a04-a433-3df6e20f405d" (UID: "686067c2-dedc-4a04-a433-3df6e20f405d"). InnerVolumeSpecName "kube-api-access-vxmx8". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:12:28 crc kubenswrapper[5052]: I1203 12:12:28.102492 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/686067c2-dedc-4a04-a433-3df6e20f405d-catalog-content" (OuterVolumeSpecName: "catalog-content") pod "686067c2-dedc-4a04-a433-3df6e20f405d" (UID: "686067c2-dedc-4a04-a433-3df6e20f405d"). InnerVolumeSpecName "catalog-content". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:12:28 crc kubenswrapper[5052]: I1203 12:12:28.158102 5052 reconciler_common.go:293] "Volume detached for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/686067c2-dedc-4a04-a433-3df6e20f405d-utilities\") on node \"crc\" DevicePath \"\"" Dec 03 12:12:28 crc kubenswrapper[5052]: I1203 12:12:28.158143 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-vxmx8\" (UniqueName: \"kubernetes.io/projected/686067c2-dedc-4a04-a433-3df6e20f405d-kube-api-access-vxmx8\") on node \"crc\" DevicePath \"\"" Dec 03 12:12:28 crc kubenswrapper[5052]: I1203 12:12:28.158156 5052 reconciler_common.go:293] "Volume detached for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/686067c2-dedc-4a04-a433-3df6e20f405d-catalog-content\") on node \"crc\" DevicePath \"\"" Dec 03 12:12:28 crc kubenswrapper[5052]: I1203 12:12:28.656384 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/swift-proxy-6b68c4db5d-g8nvc"] Dec 03 12:12:28 crc kubenswrapper[5052]: E1203 12:12:28.657018 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="686067c2-dedc-4a04-a433-3df6e20f405d" containerName="extract-utilities" Dec 03 12:12:28 crc kubenswrapper[5052]: I1203 12:12:28.657038 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="686067c2-dedc-4a04-a433-3df6e20f405d" containerName="extract-utilities" Dec 03 12:12:28 crc kubenswrapper[5052]: E1203 12:12:28.657069 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="686067c2-dedc-4a04-a433-3df6e20f405d" containerName="registry-server" Dec 03 12:12:28 crc kubenswrapper[5052]: I1203 12:12:28.657078 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="686067c2-dedc-4a04-a433-3df6e20f405d" containerName="registry-server" Dec 03 12:12:28 crc kubenswrapper[5052]: E1203 12:12:28.657105 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="686067c2-dedc-4a04-a433-3df6e20f405d" containerName="extract-content" Dec 03 12:12:28 crc kubenswrapper[5052]: I1203 12:12:28.657114 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="686067c2-dedc-4a04-a433-3df6e20f405d" containerName="extract-content" Dec 03 12:12:28 crc kubenswrapper[5052]: I1203 12:12:28.657329 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="686067c2-dedc-4a04-a433-3df6e20f405d" containerName="registry-server" Dec 03 12:12:28 crc kubenswrapper[5052]: I1203 12:12:28.658752 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-proxy-6b68c4db5d-g8nvc" Dec 03 12:12:28 crc kubenswrapper[5052]: I1203 12:12:28.661587 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/community-operators-q4qt9" event={"ID":"686067c2-dedc-4a04-a433-3df6e20f405d","Type":"ContainerDied","Data":"d6c22f1e9a303ddbb26b8f582bae1cd5928211640df9e4ba1d1a75146ca95077"} Dec 03 12:12:28 crc kubenswrapper[5052]: I1203 12:12:28.661796 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/community-operators-q4qt9" Dec 03 12:12:28 crc kubenswrapper[5052]: I1203 12:12:28.663361 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-swift-public-svc" Dec 03 12:12:28 crc kubenswrapper[5052]: I1203 12:12:28.667745 5052 scope.go:117] "RemoveContainer" containerID="63eeecd303bb8e2a2e060597115b296d7a1812ba2bd0989d4647e8fa974fe2c7" Dec 03 12:12:28 crc kubenswrapper[5052]: I1203 12:12:28.669097 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-swift-internal-svc" Dec 03 12:12:28 crc kubenswrapper[5052]: I1203 12:12:28.669963 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"swift-proxy-config-data" Dec 03 12:12:28 crc kubenswrapper[5052]: I1203 12:12:28.674170 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/swift-proxy-6b68c4db5d-g8nvc"] Dec 03 12:12:28 crc kubenswrapper[5052]: I1203 12:12:28.714684 5052 scope.go:117] "RemoveContainer" containerID="0a054fff8d1370300becab0cf5eed626d5e4176710b1b3744d8b748467865bc4" Dec 03 12:12:28 crc kubenswrapper[5052]: I1203 12:12:28.727624 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openshift-marketplace/community-operators-q4qt9"] Dec 03 12:12:28 crc kubenswrapper[5052]: I1203 12:12:28.752729 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openshift-marketplace/community-operators-q4qt9"] Dec 03 12:12:28 crc kubenswrapper[5052]: I1203 12:12:28.781640 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/ae6940a4-0f4c-46ab-8218-56bf5c5d3084-internal-tls-certs\") pod \"swift-proxy-6b68c4db5d-g8nvc\" (UID: \"ae6940a4-0f4c-46ab-8218-56bf5c5d3084\") " pod="openstack-kuttl-tests/swift-proxy-6b68c4db5d-g8nvc" Dec 03 12:12:28 crc kubenswrapper[5052]: I1203 12:12:28.781792 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/ae6940a4-0f4c-46ab-8218-56bf5c5d3084-public-tls-certs\") pod \"swift-proxy-6b68c4db5d-g8nvc\" (UID: \"ae6940a4-0f4c-46ab-8218-56bf5c5d3084\") " pod="openstack-kuttl-tests/swift-proxy-6b68c4db5d-g8nvc" Dec 03 12:12:28 crc kubenswrapper[5052]: I1203 12:12:28.781838 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/ae6940a4-0f4c-46ab-8218-56bf5c5d3084-log-httpd\") pod \"swift-proxy-6b68c4db5d-g8nvc\" (UID: \"ae6940a4-0f4c-46ab-8218-56bf5c5d3084\") " pod="openstack-kuttl-tests/swift-proxy-6b68c4db5d-g8nvc" Dec 03 12:12:28 crc kubenswrapper[5052]: I1203 12:12:28.781878 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/ae6940a4-0f4c-46ab-8218-56bf5c5d3084-config-data\") pod \"swift-proxy-6b68c4db5d-g8nvc\" (UID: \"ae6940a4-0f4c-46ab-8218-56bf5c5d3084\") " pod="openstack-kuttl-tests/swift-proxy-6b68c4db5d-g8nvc" Dec 03 12:12:28 crc kubenswrapper[5052]: I1203 12:12:28.781917 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ae6940a4-0f4c-46ab-8218-56bf5c5d3084-combined-ca-bundle\") pod \"swift-proxy-6b68c4db5d-g8nvc\" (UID: \"ae6940a4-0f4c-46ab-8218-56bf5c5d3084\") " pod="openstack-kuttl-tests/swift-proxy-6b68c4db5d-g8nvc" Dec 03 12:12:28 crc kubenswrapper[5052]: I1203 12:12:28.781954 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/ae6940a4-0f4c-46ab-8218-56bf5c5d3084-etc-swift\") pod \"swift-proxy-6b68c4db5d-g8nvc\" (UID: \"ae6940a4-0f4c-46ab-8218-56bf5c5d3084\") " pod="openstack-kuttl-tests/swift-proxy-6b68c4db5d-g8nvc" Dec 03 12:12:28 crc kubenswrapper[5052]: I1203 12:12:28.781970 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/ae6940a4-0f4c-46ab-8218-56bf5c5d3084-run-httpd\") pod \"swift-proxy-6b68c4db5d-g8nvc\" (UID: \"ae6940a4-0f4c-46ab-8218-56bf5c5d3084\") " pod="openstack-kuttl-tests/swift-proxy-6b68c4db5d-g8nvc" Dec 03 12:12:28 crc kubenswrapper[5052]: I1203 12:12:28.782043 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n4nmc\" (UniqueName: \"kubernetes.io/projected/ae6940a4-0f4c-46ab-8218-56bf5c5d3084-kube-api-access-n4nmc\") pod \"swift-proxy-6b68c4db5d-g8nvc\" (UID: \"ae6940a4-0f4c-46ab-8218-56bf5c5d3084\") " pod="openstack-kuttl-tests/swift-proxy-6b68c4db5d-g8nvc" Dec 03 12:12:28 crc kubenswrapper[5052]: I1203 12:12:28.792649 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openshift-marketplace/redhat-marketplace-p2mgk"] Dec 03 12:12:28 crc kubenswrapper[5052]: I1203 12:12:28.794864 5052 scope.go:117] "RemoveContainer" containerID="93c999e82f9a6f29ef3649f41f565a7580da592cd2697a0ff12e1e348fb8942e" Dec 03 12:12:28 crc kubenswrapper[5052]: I1203 12:12:28.884466 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/ae6940a4-0f4c-46ab-8218-56bf5c5d3084-internal-tls-certs\") pod \"swift-proxy-6b68c4db5d-g8nvc\" (UID: \"ae6940a4-0f4c-46ab-8218-56bf5c5d3084\") " pod="openstack-kuttl-tests/swift-proxy-6b68c4db5d-g8nvc" Dec 03 12:12:28 crc kubenswrapper[5052]: I1203 12:12:28.884607 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/ae6940a4-0f4c-46ab-8218-56bf5c5d3084-public-tls-certs\") pod \"swift-proxy-6b68c4db5d-g8nvc\" (UID: \"ae6940a4-0f4c-46ab-8218-56bf5c5d3084\") " pod="openstack-kuttl-tests/swift-proxy-6b68c4db5d-g8nvc" Dec 03 12:12:28 crc kubenswrapper[5052]: I1203 12:12:28.884645 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/ae6940a4-0f4c-46ab-8218-56bf5c5d3084-log-httpd\") pod \"swift-proxy-6b68c4db5d-g8nvc\" (UID: \"ae6940a4-0f4c-46ab-8218-56bf5c5d3084\") " pod="openstack-kuttl-tests/swift-proxy-6b68c4db5d-g8nvc" Dec 03 12:12:28 crc kubenswrapper[5052]: I1203 12:12:28.884695 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/ae6940a4-0f4c-46ab-8218-56bf5c5d3084-config-data\") pod \"swift-proxy-6b68c4db5d-g8nvc\" (UID: \"ae6940a4-0f4c-46ab-8218-56bf5c5d3084\") " pod="openstack-kuttl-tests/swift-proxy-6b68c4db5d-g8nvc" Dec 03 12:12:28 crc kubenswrapper[5052]: I1203 12:12:28.884724 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ae6940a4-0f4c-46ab-8218-56bf5c5d3084-combined-ca-bundle\") pod \"swift-proxy-6b68c4db5d-g8nvc\" (UID: \"ae6940a4-0f4c-46ab-8218-56bf5c5d3084\") " pod="openstack-kuttl-tests/swift-proxy-6b68c4db5d-g8nvc" Dec 03 12:12:28 crc kubenswrapper[5052]: I1203 12:12:28.884763 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/ae6940a4-0f4c-46ab-8218-56bf5c5d3084-etc-swift\") pod \"swift-proxy-6b68c4db5d-g8nvc\" (UID: \"ae6940a4-0f4c-46ab-8218-56bf5c5d3084\") " pod="openstack-kuttl-tests/swift-proxy-6b68c4db5d-g8nvc" Dec 03 12:12:28 crc kubenswrapper[5052]: I1203 12:12:28.884782 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/ae6940a4-0f4c-46ab-8218-56bf5c5d3084-run-httpd\") pod \"swift-proxy-6b68c4db5d-g8nvc\" (UID: \"ae6940a4-0f4c-46ab-8218-56bf5c5d3084\") " pod="openstack-kuttl-tests/swift-proxy-6b68c4db5d-g8nvc" Dec 03 12:12:28 crc kubenswrapper[5052]: I1203 12:12:28.884827 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-n4nmc\" (UniqueName: \"kubernetes.io/projected/ae6940a4-0f4c-46ab-8218-56bf5c5d3084-kube-api-access-n4nmc\") pod \"swift-proxy-6b68c4db5d-g8nvc\" (UID: \"ae6940a4-0f4c-46ab-8218-56bf5c5d3084\") " pod="openstack-kuttl-tests/swift-proxy-6b68c4db5d-g8nvc" Dec 03 12:12:28 crc kubenswrapper[5052]: I1203 12:12:28.885271 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/ae6940a4-0f4c-46ab-8218-56bf5c5d3084-log-httpd\") pod \"swift-proxy-6b68c4db5d-g8nvc\" (UID: \"ae6940a4-0f4c-46ab-8218-56bf5c5d3084\") " pod="openstack-kuttl-tests/swift-proxy-6b68c4db5d-g8nvc" Dec 03 12:12:28 crc kubenswrapper[5052]: I1203 12:12:28.886938 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/ae6940a4-0f4c-46ab-8218-56bf5c5d3084-run-httpd\") pod \"swift-proxy-6b68c4db5d-g8nvc\" (UID: \"ae6940a4-0f4c-46ab-8218-56bf5c5d3084\") " pod="openstack-kuttl-tests/swift-proxy-6b68c4db5d-g8nvc" Dec 03 12:12:28 crc kubenswrapper[5052]: I1203 12:12:28.897242 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/ae6940a4-0f4c-46ab-8218-56bf5c5d3084-internal-tls-certs\") pod \"swift-proxy-6b68c4db5d-g8nvc\" (UID: \"ae6940a4-0f4c-46ab-8218-56bf5c5d3084\") " pod="openstack-kuttl-tests/swift-proxy-6b68c4db5d-g8nvc" Dec 03 12:12:28 crc kubenswrapper[5052]: I1203 12:12:28.897499 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/ae6940a4-0f4c-46ab-8218-56bf5c5d3084-config-data\") pod \"swift-proxy-6b68c4db5d-g8nvc\" (UID: \"ae6940a4-0f4c-46ab-8218-56bf5c5d3084\") " pod="openstack-kuttl-tests/swift-proxy-6b68c4db5d-g8nvc" Dec 03 12:12:28 crc kubenswrapper[5052]: I1203 12:12:28.897860 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ae6940a4-0f4c-46ab-8218-56bf5c5d3084-combined-ca-bundle\") pod \"swift-proxy-6b68c4db5d-g8nvc\" (UID: \"ae6940a4-0f4c-46ab-8218-56bf5c5d3084\") " pod="openstack-kuttl-tests/swift-proxy-6b68c4db5d-g8nvc" Dec 03 12:12:28 crc kubenswrapper[5052]: I1203 12:12:28.899195 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/ae6940a4-0f4c-46ab-8218-56bf5c5d3084-public-tls-certs\") pod \"swift-proxy-6b68c4db5d-g8nvc\" (UID: \"ae6940a4-0f4c-46ab-8218-56bf5c5d3084\") " pod="openstack-kuttl-tests/swift-proxy-6b68c4db5d-g8nvc" Dec 03 12:12:28 crc kubenswrapper[5052]: I1203 12:12:28.902398 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/ae6940a4-0f4c-46ab-8218-56bf5c5d3084-etc-swift\") pod \"swift-proxy-6b68c4db5d-g8nvc\" (UID: \"ae6940a4-0f4c-46ab-8218-56bf5c5d3084\") " pod="openstack-kuttl-tests/swift-proxy-6b68c4db5d-g8nvc" Dec 03 12:12:28 crc kubenswrapper[5052]: I1203 12:12:28.905687 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-n4nmc\" (UniqueName: \"kubernetes.io/projected/ae6940a4-0f4c-46ab-8218-56bf5c5d3084-kube-api-access-n4nmc\") pod \"swift-proxy-6b68c4db5d-g8nvc\" (UID: \"ae6940a4-0f4c-46ab-8218-56bf5c5d3084\") " pod="openstack-kuttl-tests/swift-proxy-6b68c4db5d-g8nvc" Dec 03 12:12:29 crc kubenswrapper[5052]: I1203 12:12:29.005201 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-proxy-6b68c4db5d-g8nvc" Dec 03 12:12:29 crc kubenswrapper[5052]: I1203 12:12:29.083071 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:12:29 crc kubenswrapper[5052]: I1203 12:12:29.083639 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="18240d78-5173-4d34-a1b7-8d51cac46b74" containerName="ceilometer-central-agent" containerID="cri-o://1a2115a0a85bfc97ed65c66337befc7d3164462079d6fd960da7e80b6dd2ac30" gracePeriod=30 Dec 03 12:12:29 crc kubenswrapper[5052]: I1203 12:12:29.083675 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="18240d78-5173-4d34-a1b7-8d51cac46b74" containerName="proxy-httpd" containerID="cri-o://63454317bd2f90c442d2189fe71d8ca20e75f95212e175181074a919d866aec2" gracePeriod=30 Dec 03 12:12:29 crc kubenswrapper[5052]: I1203 12:12:29.083850 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="18240d78-5173-4d34-a1b7-8d51cac46b74" containerName="sg-core" containerID="cri-o://6818f70d48c9182195a247515369fd4e561d427183eeca88cc603aba0d5819c7" gracePeriod=30 Dec 03 12:12:29 crc kubenswrapper[5052]: I1203 12:12:29.083897 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="18240d78-5173-4d34-a1b7-8d51cac46b74" containerName="ceilometer-notification-agent" containerID="cri-o://600afe54035e6ba3cfd43d7a781ebae8aa663487d01caeaadb229c09d9865654" gracePeriod=30 Dec 03 12:12:29 crc kubenswrapper[5052]: I1203 12:12:29.190670 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:12:29 crc kubenswrapper[5052]: I1203 12:12:29.507608 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/swift-proxy-6b68c4db5d-g8nvc"] Dec 03 12:12:29 crc kubenswrapper[5052]: W1203 12:12:29.510166 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podae6940a4_0f4c_46ab_8218_56bf5c5d3084.slice/crio-d9473142a7df3ab74c3cc40fff2c3b5ebfc68071ca057ae58c77c13b1a8cf630 WatchSource:0}: Error finding container d9473142a7df3ab74c3cc40fff2c3b5ebfc68071ca057ae58c77c13b1a8cf630: Status 404 returned error can't find the container with id d9473142a7df3ab74c3cc40fff2c3b5ebfc68071ca057ae58c77c13b1a8cf630 Dec 03 12:12:29 crc kubenswrapper[5052]: I1203 12:12:29.678476 5052 generic.go:334] "Generic (PLEG): container finished" podID="18240d78-5173-4d34-a1b7-8d51cac46b74" containerID="63454317bd2f90c442d2189fe71d8ca20e75f95212e175181074a919d866aec2" exitCode=0 Dec 03 12:12:29 crc kubenswrapper[5052]: I1203 12:12:29.678738 5052 generic.go:334] "Generic (PLEG): container finished" podID="18240d78-5173-4d34-a1b7-8d51cac46b74" containerID="6818f70d48c9182195a247515369fd4e561d427183eeca88cc603aba0d5819c7" exitCode=2 Dec 03 12:12:29 crc kubenswrapper[5052]: I1203 12:12:29.678746 5052 generic.go:334] "Generic (PLEG): container finished" podID="18240d78-5173-4d34-a1b7-8d51cac46b74" containerID="1a2115a0a85bfc97ed65c66337befc7d3164462079d6fd960da7e80b6dd2ac30" exitCode=0 Dec 03 12:12:29 crc kubenswrapper[5052]: I1203 12:12:29.687319 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="686067c2-dedc-4a04-a433-3df6e20f405d" path="/var/lib/kubelet/pods/686067c2-dedc-4a04-a433-3df6e20f405d/volumes" Dec 03 12:12:29 crc kubenswrapper[5052]: I1203 12:12:29.687383 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openshift-marketplace/redhat-marketplace-p2mgk" podUID="658ad8d2-0efc-4752-a4c5-54e313bdb305" containerName="registry-server" containerID="cri-o://b2cb242e355b8c375f8b20b6132c7e0d7475ada1b01193b4b00caa311f6f8fa9" gracePeriod=2 Dec 03 12:12:29 crc kubenswrapper[5052]: I1203 12:12:29.688155 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-proxy-6b68c4db5d-g8nvc" event={"ID":"ae6940a4-0f4c-46ab-8218-56bf5c5d3084","Type":"ContainerStarted","Data":"d9473142a7df3ab74c3cc40fff2c3b5ebfc68071ca057ae58c77c13b1a8cf630"} Dec 03 12:12:29 crc kubenswrapper[5052]: I1203 12:12:29.688190 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"18240d78-5173-4d34-a1b7-8d51cac46b74","Type":"ContainerDied","Data":"63454317bd2f90c442d2189fe71d8ca20e75f95212e175181074a919d866aec2"} Dec 03 12:12:29 crc kubenswrapper[5052]: I1203 12:12:29.688205 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"18240d78-5173-4d34-a1b7-8d51cac46b74","Type":"ContainerDied","Data":"6818f70d48c9182195a247515369fd4e561d427183eeca88cc603aba0d5819c7"} Dec 03 12:12:29 crc kubenswrapper[5052]: I1203 12:12:29.688217 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"18240d78-5173-4d34-a1b7-8d51cac46b74","Type":"ContainerDied","Data":"1a2115a0a85bfc97ed65c66337befc7d3164462079d6fd960da7e80b6dd2ac30"} Dec 03 12:12:29 crc kubenswrapper[5052]: E1203 12:12:29.711778 5052 cadvisor_stats_provider.go:516] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod18240d78_5173_4d34_a1b7_8d51cac46b74.slice/crio-conmon-1a2115a0a85bfc97ed65c66337befc7d3164462079d6fd960da7e80b6dd2ac30.scope\": RecentStats: unable to find data in memory cache]" Dec 03 12:12:30 crc kubenswrapper[5052]: I1203 12:12:30.699491 5052 generic.go:334] "Generic (PLEG): container finished" podID="658ad8d2-0efc-4752-a4c5-54e313bdb305" containerID="b2cb242e355b8c375f8b20b6132c7e0d7475ada1b01193b4b00caa311f6f8fa9" exitCode=0 Dec 03 12:12:30 crc kubenswrapper[5052]: I1203 12:12:30.699615 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-marketplace-p2mgk" event={"ID":"658ad8d2-0efc-4752-a4c5-54e313bdb305","Type":"ContainerDied","Data":"b2cb242e355b8c375f8b20b6132c7e0d7475ada1b01193b4b00caa311f6f8fa9"} Dec 03 12:12:30 crc kubenswrapper[5052]: I1203 12:12:30.710624 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-proxy-6b68c4db5d-g8nvc" event={"ID":"ae6940a4-0f4c-46ab-8218-56bf5c5d3084","Type":"ContainerStarted","Data":"6d0e263d1fbe61d44b6dd94d1a87bc31a61607f209df9eb30c44c6f6386c7381"} Dec 03 12:12:30 crc kubenswrapper[5052]: I1203 12:12:30.710675 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-proxy-6b68c4db5d-g8nvc" event={"ID":"ae6940a4-0f4c-46ab-8218-56bf5c5d3084","Type":"ContainerStarted","Data":"22c61e9465e05e82a9b75ec36f8899daba6ae848986e069b783febfc60abd647"} Dec 03 12:12:30 crc kubenswrapper[5052]: I1203 12:12:30.712601 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/swift-proxy-6b68c4db5d-g8nvc" Dec 03 12:12:30 crc kubenswrapper[5052]: I1203 12:12:30.712638 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/swift-proxy-6b68c4db5d-g8nvc" Dec 03 12:12:30 crc kubenswrapper[5052]: I1203 12:12:30.735111 5052 generic.go:334] "Generic (PLEG): container finished" podID="18240d78-5173-4d34-a1b7-8d51cac46b74" containerID="600afe54035e6ba3cfd43d7a781ebae8aa663487d01caeaadb229c09d9865654" exitCode=0 Dec 03 12:12:30 crc kubenswrapper[5052]: I1203 12:12:30.735161 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"18240d78-5173-4d34-a1b7-8d51cac46b74","Type":"ContainerDied","Data":"600afe54035e6ba3cfd43d7a781ebae8aa663487d01caeaadb229c09d9865654"} Dec 03 12:12:30 crc kubenswrapper[5052]: I1203 12:12:30.761758 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/swift-proxy-6b68c4db5d-g8nvc" podStartSLOduration=2.761735691 podStartE2EDuration="2.761735691s" podCreationTimestamp="2025-12-03 12:12:28 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:12:30.750194791 +0000 UTC m=+3141.338436555" watchObservedRunningTime="2025-12-03 12:12:30.761735691 +0000 UTC m=+3141.349977455" Dec 03 12:12:31 crc kubenswrapper[5052]: I1203 12:12:31.045685 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/redhat-marketplace-p2mgk" Dec 03 12:12:31 crc kubenswrapper[5052]: I1203 12:12:31.151572 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pqfdv\" (UniqueName: \"kubernetes.io/projected/658ad8d2-0efc-4752-a4c5-54e313bdb305-kube-api-access-pqfdv\") pod \"658ad8d2-0efc-4752-a4c5-54e313bdb305\" (UID: \"658ad8d2-0efc-4752-a4c5-54e313bdb305\") " Dec 03 12:12:31 crc kubenswrapper[5052]: I1203 12:12:31.151645 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/658ad8d2-0efc-4752-a4c5-54e313bdb305-catalog-content\") pod \"658ad8d2-0efc-4752-a4c5-54e313bdb305\" (UID: \"658ad8d2-0efc-4752-a4c5-54e313bdb305\") " Dec 03 12:12:31 crc kubenswrapper[5052]: I1203 12:12:31.151738 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/658ad8d2-0efc-4752-a4c5-54e313bdb305-utilities\") pod \"658ad8d2-0efc-4752-a4c5-54e313bdb305\" (UID: \"658ad8d2-0efc-4752-a4c5-54e313bdb305\") " Dec 03 12:12:31 crc kubenswrapper[5052]: I1203 12:12:31.152931 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/658ad8d2-0efc-4752-a4c5-54e313bdb305-utilities" (OuterVolumeSpecName: "utilities") pod "658ad8d2-0efc-4752-a4c5-54e313bdb305" (UID: "658ad8d2-0efc-4752-a4c5-54e313bdb305"). InnerVolumeSpecName "utilities". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:12:31 crc kubenswrapper[5052]: I1203 12:12:31.159648 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/658ad8d2-0efc-4752-a4c5-54e313bdb305-kube-api-access-pqfdv" (OuterVolumeSpecName: "kube-api-access-pqfdv") pod "658ad8d2-0efc-4752-a4c5-54e313bdb305" (UID: "658ad8d2-0efc-4752-a4c5-54e313bdb305"). InnerVolumeSpecName "kube-api-access-pqfdv". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:12:31 crc kubenswrapper[5052]: I1203 12:12:31.172355 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/658ad8d2-0efc-4752-a4c5-54e313bdb305-catalog-content" (OuterVolumeSpecName: "catalog-content") pod "658ad8d2-0efc-4752-a4c5-54e313bdb305" (UID: "658ad8d2-0efc-4752-a4c5-54e313bdb305"). InnerVolumeSpecName "catalog-content". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:12:31 crc kubenswrapper[5052]: I1203 12:12:31.253886 5052 reconciler_common.go:293] "Volume detached for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/658ad8d2-0efc-4752-a4c5-54e313bdb305-utilities\") on node \"crc\" DevicePath \"\"" Dec 03 12:12:31 crc kubenswrapper[5052]: I1203 12:12:31.253917 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-pqfdv\" (UniqueName: \"kubernetes.io/projected/658ad8d2-0efc-4752-a4c5-54e313bdb305-kube-api-access-pqfdv\") on node \"crc\" DevicePath \"\"" Dec 03 12:12:31 crc kubenswrapper[5052]: I1203 12:12:31.253928 5052 reconciler_common.go:293] "Volume detached for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/658ad8d2-0efc-4752-a4c5-54e313bdb305-catalog-content\") on node \"crc\" DevicePath \"\"" Dec 03 12:12:31 crc kubenswrapper[5052]: I1203 12:12:31.260735 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:12:31 crc kubenswrapper[5052]: I1203 12:12:31.354677 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/18240d78-5173-4d34-a1b7-8d51cac46b74-run-httpd\") pod \"18240d78-5173-4d34-a1b7-8d51cac46b74\" (UID: \"18240d78-5173-4d34-a1b7-8d51cac46b74\") " Dec 03 12:12:31 crc kubenswrapper[5052]: I1203 12:12:31.354734 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/18240d78-5173-4d34-a1b7-8d51cac46b74-log-httpd\") pod \"18240d78-5173-4d34-a1b7-8d51cac46b74\" (UID: \"18240d78-5173-4d34-a1b7-8d51cac46b74\") " Dec 03 12:12:31 crc kubenswrapper[5052]: I1203 12:12:31.354801 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p8vz6\" (UniqueName: \"kubernetes.io/projected/18240d78-5173-4d34-a1b7-8d51cac46b74-kube-api-access-p8vz6\") pod \"18240d78-5173-4d34-a1b7-8d51cac46b74\" (UID: \"18240d78-5173-4d34-a1b7-8d51cac46b74\") " Dec 03 12:12:31 crc kubenswrapper[5052]: I1203 12:12:31.354840 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/18240d78-5173-4d34-a1b7-8d51cac46b74-config-data\") pod \"18240d78-5173-4d34-a1b7-8d51cac46b74\" (UID: \"18240d78-5173-4d34-a1b7-8d51cac46b74\") " Dec 03 12:12:31 crc kubenswrapper[5052]: I1203 12:12:31.354925 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/18240d78-5173-4d34-a1b7-8d51cac46b74-sg-core-conf-yaml\") pod \"18240d78-5173-4d34-a1b7-8d51cac46b74\" (UID: \"18240d78-5173-4d34-a1b7-8d51cac46b74\") " Dec 03 12:12:31 crc kubenswrapper[5052]: I1203 12:12:31.354999 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/18240d78-5173-4d34-a1b7-8d51cac46b74-scripts\") pod \"18240d78-5173-4d34-a1b7-8d51cac46b74\" (UID: \"18240d78-5173-4d34-a1b7-8d51cac46b74\") " Dec 03 12:12:31 crc kubenswrapper[5052]: I1203 12:12:31.355063 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/18240d78-5173-4d34-a1b7-8d51cac46b74-run-httpd" (OuterVolumeSpecName: "run-httpd") pod "18240d78-5173-4d34-a1b7-8d51cac46b74" (UID: "18240d78-5173-4d34-a1b7-8d51cac46b74"). InnerVolumeSpecName "run-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:12:31 crc kubenswrapper[5052]: I1203 12:12:31.355211 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/18240d78-5173-4d34-a1b7-8d51cac46b74-log-httpd" (OuterVolumeSpecName: "log-httpd") pod "18240d78-5173-4d34-a1b7-8d51cac46b74" (UID: "18240d78-5173-4d34-a1b7-8d51cac46b74"). InnerVolumeSpecName "log-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:12:31 crc kubenswrapper[5052]: I1203 12:12:31.355554 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/18240d78-5173-4d34-a1b7-8d51cac46b74-combined-ca-bundle\") pod \"18240d78-5173-4d34-a1b7-8d51cac46b74\" (UID: \"18240d78-5173-4d34-a1b7-8d51cac46b74\") " Dec 03 12:12:31 crc kubenswrapper[5052]: I1203 12:12:31.356053 5052 reconciler_common.go:293] "Volume detached for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/18240d78-5173-4d34-a1b7-8d51cac46b74-run-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:12:31 crc kubenswrapper[5052]: I1203 12:12:31.356076 5052 reconciler_common.go:293] "Volume detached for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/18240d78-5173-4d34-a1b7-8d51cac46b74-log-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:12:31 crc kubenswrapper[5052]: I1203 12:12:31.358057 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/18240d78-5173-4d34-a1b7-8d51cac46b74-scripts" (OuterVolumeSpecName: "scripts") pod "18240d78-5173-4d34-a1b7-8d51cac46b74" (UID: "18240d78-5173-4d34-a1b7-8d51cac46b74"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:12:31 crc kubenswrapper[5052]: I1203 12:12:31.360678 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/18240d78-5173-4d34-a1b7-8d51cac46b74-kube-api-access-p8vz6" (OuterVolumeSpecName: "kube-api-access-p8vz6") pod "18240d78-5173-4d34-a1b7-8d51cac46b74" (UID: "18240d78-5173-4d34-a1b7-8d51cac46b74"). InnerVolumeSpecName "kube-api-access-p8vz6". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:12:31 crc kubenswrapper[5052]: I1203 12:12:31.393080 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/18240d78-5173-4d34-a1b7-8d51cac46b74-sg-core-conf-yaml" (OuterVolumeSpecName: "sg-core-conf-yaml") pod "18240d78-5173-4d34-a1b7-8d51cac46b74" (UID: "18240d78-5173-4d34-a1b7-8d51cac46b74"). InnerVolumeSpecName "sg-core-conf-yaml". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:12:31 crc kubenswrapper[5052]: I1203 12:12:31.429287 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/18240d78-5173-4d34-a1b7-8d51cac46b74-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "18240d78-5173-4d34-a1b7-8d51cac46b74" (UID: "18240d78-5173-4d34-a1b7-8d51cac46b74"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:12:31 crc kubenswrapper[5052]: I1203 12:12:31.457804 5052 reconciler_common.go:293] "Volume detached for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/18240d78-5173-4d34-a1b7-8d51cac46b74-sg-core-conf-yaml\") on node \"crc\" DevicePath \"\"" Dec 03 12:12:31 crc kubenswrapper[5052]: I1203 12:12:31.457838 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/18240d78-5173-4d34-a1b7-8d51cac46b74-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:12:31 crc kubenswrapper[5052]: I1203 12:12:31.457848 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/18240d78-5173-4d34-a1b7-8d51cac46b74-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:12:31 crc kubenswrapper[5052]: I1203 12:12:31.457858 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-p8vz6\" (UniqueName: \"kubernetes.io/projected/18240d78-5173-4d34-a1b7-8d51cac46b74-kube-api-access-p8vz6\") on node \"crc\" DevicePath \"\"" Dec 03 12:12:31 crc kubenswrapper[5052]: I1203 12:12:31.460309 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/18240d78-5173-4d34-a1b7-8d51cac46b74-config-data" (OuterVolumeSpecName: "config-data") pod "18240d78-5173-4d34-a1b7-8d51cac46b74" (UID: "18240d78-5173-4d34-a1b7-8d51cac46b74"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:12:31 crc kubenswrapper[5052]: I1203 12:12:31.559429 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/18240d78-5173-4d34-a1b7-8d51cac46b74-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:12:31 crc kubenswrapper[5052]: I1203 12:12:31.769712 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-marketplace-p2mgk" event={"ID":"658ad8d2-0efc-4752-a4c5-54e313bdb305","Type":"ContainerDied","Data":"eb054f91f0d01114e422dbf45f721346c3d6a2e7d470dfcd459f00d7ba67e39b"} Dec 03 12:12:31 crc kubenswrapper[5052]: I1203 12:12:31.769773 5052 scope.go:117] "RemoveContainer" containerID="b2cb242e355b8c375f8b20b6132c7e0d7475ada1b01193b4b00caa311f6f8fa9" Dec 03 12:12:31 crc kubenswrapper[5052]: I1203 12:12:31.769936 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/redhat-marketplace-p2mgk" Dec 03 12:12:31 crc kubenswrapper[5052]: I1203 12:12:31.822234 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:12:31 crc kubenswrapper[5052]: I1203 12:12:31.822999 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"18240d78-5173-4d34-a1b7-8d51cac46b74","Type":"ContainerDied","Data":"98bd94408261570ebaaa7927e396c3a599bf8561c103a382034a1e90fca91ee4"} Dec 03 12:12:31 crc kubenswrapper[5052]: I1203 12:12:31.866230 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openshift-marketplace/redhat-marketplace-p2mgk"] Dec 03 12:12:31 crc kubenswrapper[5052]: I1203 12:12:31.877409 5052 scope.go:117] "RemoveContainer" containerID="399b2e1c1557872473b7c499b4421f32ec0a1fc5c1e91f0333f9d0d389d40b22" Dec 03 12:12:31 crc kubenswrapper[5052]: I1203 12:12:31.932656 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openshift-marketplace/redhat-marketplace-p2mgk"] Dec 03 12:12:31 crc kubenswrapper[5052]: I1203 12:12:31.934790 5052 scope.go:117] "RemoveContainer" containerID="6495ca125925b04c59734d6881a3281d4c16ec80168c3fb71673bdc1b85f4ce2" Dec 03 12:12:31 crc kubenswrapper[5052]: I1203 12:12:31.962158 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:12:31 crc kubenswrapper[5052]: I1203 12:12:31.979615 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:12:31 crc kubenswrapper[5052]: I1203 12:12:31.990583 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:12:31 crc kubenswrapper[5052]: E1203 12:12:31.991130 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="18240d78-5173-4d34-a1b7-8d51cac46b74" containerName="sg-core" Dec 03 12:12:31 crc kubenswrapper[5052]: I1203 12:12:31.991153 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="18240d78-5173-4d34-a1b7-8d51cac46b74" containerName="sg-core" Dec 03 12:12:31 crc kubenswrapper[5052]: E1203 12:12:31.991169 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="658ad8d2-0efc-4752-a4c5-54e313bdb305" containerName="registry-server" Dec 03 12:12:31 crc kubenswrapper[5052]: I1203 12:12:31.991178 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="658ad8d2-0efc-4752-a4c5-54e313bdb305" containerName="registry-server" Dec 03 12:12:31 crc kubenswrapper[5052]: E1203 12:12:31.991195 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="658ad8d2-0efc-4752-a4c5-54e313bdb305" containerName="extract-utilities" Dec 03 12:12:31 crc kubenswrapper[5052]: I1203 12:12:31.991203 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="658ad8d2-0efc-4752-a4c5-54e313bdb305" containerName="extract-utilities" Dec 03 12:12:31 crc kubenswrapper[5052]: E1203 12:12:31.991222 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="18240d78-5173-4d34-a1b7-8d51cac46b74" containerName="ceilometer-notification-agent" Dec 03 12:12:31 crc kubenswrapper[5052]: I1203 12:12:31.991229 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="18240d78-5173-4d34-a1b7-8d51cac46b74" containerName="ceilometer-notification-agent" Dec 03 12:12:31 crc kubenswrapper[5052]: E1203 12:12:31.991240 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="18240d78-5173-4d34-a1b7-8d51cac46b74" containerName="proxy-httpd" Dec 03 12:12:31 crc kubenswrapper[5052]: I1203 12:12:31.991247 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="18240d78-5173-4d34-a1b7-8d51cac46b74" containerName="proxy-httpd" Dec 03 12:12:31 crc kubenswrapper[5052]: E1203 12:12:31.991260 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="18240d78-5173-4d34-a1b7-8d51cac46b74" containerName="ceilometer-central-agent" Dec 03 12:12:31 crc kubenswrapper[5052]: I1203 12:12:31.991267 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="18240d78-5173-4d34-a1b7-8d51cac46b74" containerName="ceilometer-central-agent" Dec 03 12:12:31 crc kubenswrapper[5052]: E1203 12:12:31.991283 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="658ad8d2-0efc-4752-a4c5-54e313bdb305" containerName="extract-content" Dec 03 12:12:31 crc kubenswrapper[5052]: I1203 12:12:31.991291 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="658ad8d2-0efc-4752-a4c5-54e313bdb305" containerName="extract-content" Dec 03 12:12:31 crc kubenswrapper[5052]: I1203 12:12:31.991498 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="18240d78-5173-4d34-a1b7-8d51cac46b74" containerName="sg-core" Dec 03 12:12:31 crc kubenswrapper[5052]: I1203 12:12:31.991538 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="18240d78-5173-4d34-a1b7-8d51cac46b74" containerName="proxy-httpd" Dec 03 12:12:31 crc kubenswrapper[5052]: I1203 12:12:31.991566 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="18240d78-5173-4d34-a1b7-8d51cac46b74" containerName="ceilometer-central-agent" Dec 03 12:12:31 crc kubenswrapper[5052]: I1203 12:12:31.991577 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="18240d78-5173-4d34-a1b7-8d51cac46b74" containerName="ceilometer-notification-agent" Dec 03 12:12:31 crc kubenswrapper[5052]: I1203 12:12:31.991587 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="658ad8d2-0efc-4752-a4c5-54e313bdb305" containerName="registry-server" Dec 03 12:12:31 crc kubenswrapper[5052]: I1203 12:12:31.993770 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:12:31 crc kubenswrapper[5052]: I1203 12:12:31.996061 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-scripts" Dec 03 12:12:32 crc kubenswrapper[5052]: I1203 12:12:31.999917 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-config-data" Dec 03 12:12:32 crc kubenswrapper[5052]: I1203 12:12:32.005496 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:12:32 crc kubenswrapper[5052]: I1203 12:12:32.025545 5052 scope.go:117] "RemoveContainer" containerID="63454317bd2f90c442d2189fe71d8ca20e75f95212e175181074a919d866aec2" Dec 03 12:12:32 crc kubenswrapper[5052]: I1203 12:12:32.072186 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/62324d6b-4dc8-475e-845d-ade03d0aeb66-config-data\") pod \"ceilometer-0\" (UID: \"62324d6b-4dc8-475e-845d-ade03d0aeb66\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:12:32 crc kubenswrapper[5052]: I1203 12:12:32.072229 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/62324d6b-4dc8-475e-845d-ade03d0aeb66-scripts\") pod \"ceilometer-0\" (UID: \"62324d6b-4dc8-475e-845d-ade03d0aeb66\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:12:32 crc kubenswrapper[5052]: I1203 12:12:32.072255 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/62324d6b-4dc8-475e-845d-ade03d0aeb66-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"62324d6b-4dc8-475e-845d-ade03d0aeb66\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:12:32 crc kubenswrapper[5052]: I1203 12:12:32.072302 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jxq8j\" (UniqueName: \"kubernetes.io/projected/62324d6b-4dc8-475e-845d-ade03d0aeb66-kube-api-access-jxq8j\") pod \"ceilometer-0\" (UID: \"62324d6b-4dc8-475e-845d-ade03d0aeb66\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:12:32 crc kubenswrapper[5052]: I1203 12:12:32.072366 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/62324d6b-4dc8-475e-845d-ade03d0aeb66-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"62324d6b-4dc8-475e-845d-ade03d0aeb66\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:12:32 crc kubenswrapper[5052]: I1203 12:12:32.072407 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/62324d6b-4dc8-475e-845d-ade03d0aeb66-log-httpd\") pod \"ceilometer-0\" (UID: \"62324d6b-4dc8-475e-845d-ade03d0aeb66\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:12:32 crc kubenswrapper[5052]: I1203 12:12:32.072442 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/62324d6b-4dc8-475e-845d-ade03d0aeb66-run-httpd\") pod \"ceilometer-0\" (UID: \"62324d6b-4dc8-475e-845d-ade03d0aeb66\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:12:32 crc kubenswrapper[5052]: I1203 12:12:32.084258 5052 scope.go:117] "RemoveContainer" containerID="6818f70d48c9182195a247515369fd4e561d427183eeca88cc603aba0d5819c7" Dec 03 12:12:32 crc kubenswrapper[5052]: I1203 12:12:32.102663 5052 scope.go:117] "RemoveContainer" containerID="600afe54035e6ba3cfd43d7a781ebae8aa663487d01caeaadb229c09d9865654" Dec 03 12:12:32 crc kubenswrapper[5052]: I1203 12:12:32.123065 5052 scope.go:117] "RemoveContainer" containerID="1a2115a0a85bfc97ed65c66337befc7d3164462079d6fd960da7e80b6dd2ac30" Dec 03 12:12:32 crc kubenswrapper[5052]: I1203 12:12:32.174614 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/62324d6b-4dc8-475e-845d-ade03d0aeb66-config-data\") pod \"ceilometer-0\" (UID: \"62324d6b-4dc8-475e-845d-ade03d0aeb66\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:12:32 crc kubenswrapper[5052]: I1203 12:12:32.174675 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/62324d6b-4dc8-475e-845d-ade03d0aeb66-scripts\") pod \"ceilometer-0\" (UID: \"62324d6b-4dc8-475e-845d-ade03d0aeb66\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:12:32 crc kubenswrapper[5052]: I1203 12:12:32.174715 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/62324d6b-4dc8-475e-845d-ade03d0aeb66-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"62324d6b-4dc8-475e-845d-ade03d0aeb66\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:12:32 crc kubenswrapper[5052]: I1203 12:12:32.174773 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-jxq8j\" (UniqueName: \"kubernetes.io/projected/62324d6b-4dc8-475e-845d-ade03d0aeb66-kube-api-access-jxq8j\") pod \"ceilometer-0\" (UID: \"62324d6b-4dc8-475e-845d-ade03d0aeb66\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:12:32 crc kubenswrapper[5052]: I1203 12:12:32.174821 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/62324d6b-4dc8-475e-845d-ade03d0aeb66-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"62324d6b-4dc8-475e-845d-ade03d0aeb66\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:12:32 crc kubenswrapper[5052]: I1203 12:12:32.174870 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/62324d6b-4dc8-475e-845d-ade03d0aeb66-log-httpd\") pod \"ceilometer-0\" (UID: \"62324d6b-4dc8-475e-845d-ade03d0aeb66\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:12:32 crc kubenswrapper[5052]: I1203 12:12:32.174926 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/62324d6b-4dc8-475e-845d-ade03d0aeb66-run-httpd\") pod \"ceilometer-0\" (UID: \"62324d6b-4dc8-475e-845d-ade03d0aeb66\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:12:32 crc kubenswrapper[5052]: I1203 12:12:32.175447 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/62324d6b-4dc8-475e-845d-ade03d0aeb66-run-httpd\") pod \"ceilometer-0\" (UID: \"62324d6b-4dc8-475e-845d-ade03d0aeb66\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:12:32 crc kubenswrapper[5052]: I1203 12:12:32.176667 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/62324d6b-4dc8-475e-845d-ade03d0aeb66-log-httpd\") pod \"ceilometer-0\" (UID: \"62324d6b-4dc8-475e-845d-ade03d0aeb66\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:12:32 crc kubenswrapper[5052]: I1203 12:12:32.181024 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/62324d6b-4dc8-475e-845d-ade03d0aeb66-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"62324d6b-4dc8-475e-845d-ade03d0aeb66\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:12:32 crc kubenswrapper[5052]: I1203 12:12:32.181407 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/62324d6b-4dc8-475e-845d-ade03d0aeb66-scripts\") pod \"ceilometer-0\" (UID: \"62324d6b-4dc8-475e-845d-ade03d0aeb66\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:12:32 crc kubenswrapper[5052]: I1203 12:12:32.181562 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/62324d6b-4dc8-475e-845d-ade03d0aeb66-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"62324d6b-4dc8-475e-845d-ade03d0aeb66\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:12:32 crc kubenswrapper[5052]: I1203 12:12:32.185317 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/62324d6b-4dc8-475e-845d-ade03d0aeb66-config-data\") pod \"ceilometer-0\" (UID: \"62324d6b-4dc8-475e-845d-ade03d0aeb66\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:12:32 crc kubenswrapper[5052]: I1203 12:12:32.195345 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-jxq8j\" (UniqueName: \"kubernetes.io/projected/62324d6b-4dc8-475e-845d-ade03d0aeb66-kube-api-access-jxq8j\") pod \"ceilometer-0\" (UID: \"62324d6b-4dc8-475e-845d-ade03d0aeb66\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:12:32 crc kubenswrapper[5052]: I1203 12:12:32.377062 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:12:32 crc kubenswrapper[5052]: I1203 12:12:32.876842 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:12:33 crc kubenswrapper[5052]: I1203 12:12:33.673221 5052 scope.go:117] "RemoveContainer" containerID="63298107b3d090deff9ed92f3d613e5b1c86d9137684d117500257db8ee1c805" Dec 03 12:12:33 crc kubenswrapper[5052]: E1203 12:12:33.673925 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 12:12:33 crc kubenswrapper[5052]: I1203 12:12:33.684169 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="18240d78-5173-4d34-a1b7-8d51cac46b74" path="/var/lib/kubelet/pods/18240d78-5173-4d34-a1b7-8d51cac46b74/volumes" Dec 03 12:12:33 crc kubenswrapper[5052]: I1203 12:12:33.685149 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="658ad8d2-0efc-4752-a4c5-54e313bdb305" path="/var/lib/kubelet/pods/658ad8d2-0efc-4752-a4c5-54e313bdb305/volumes" Dec 03 12:12:33 crc kubenswrapper[5052]: I1203 12:12:33.844329 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"62324d6b-4dc8-475e-845d-ade03d0aeb66","Type":"ContainerStarted","Data":"18124cd859d2a7dbd10315d6f162299bc1509060136071432394850b9aa109a8"} Dec 03 12:12:34 crc kubenswrapper[5052]: I1203 12:12:34.017750 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/swift-proxy-6b68c4db5d-g8nvc" Dec 03 12:12:34 crc kubenswrapper[5052]: I1203 12:12:34.414967 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:12:34 crc kubenswrapper[5052]: I1203 12:12:34.855590 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"62324d6b-4dc8-475e-845d-ade03d0aeb66","Type":"ContainerStarted","Data":"1b5e9176625dce69f16041ac8842c83d6f98cc78dd1d675d303aa5991fbeb50e"} Dec 03 12:12:35 crc kubenswrapper[5052]: I1203 12:12:35.865068 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"62324d6b-4dc8-475e-845d-ade03d0aeb66","Type":"ContainerStarted","Data":"561134ce71b168b26e0e8ed43abe0e914973e8a5d6b3341d25487208e3088e1f"} Dec 03 12:12:36 crc kubenswrapper[5052]: I1203 12:12:36.243152 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:12:36 crc kubenswrapper[5052]: I1203 12:12:36.877732 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"62324d6b-4dc8-475e-845d-ade03d0aeb66","Type":"ContainerStarted","Data":"cc1d906a68871533f939b6588bccbe9a3527cc639e711e946364382e647bea70"} Dec 03 12:12:37 crc kubenswrapper[5052]: I1203 12:12:37.889461 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"62324d6b-4dc8-475e-845d-ade03d0aeb66","Type":"ContainerStarted","Data":"aa1eb04f39b67e01fee6a0d26805faf02b005c3d50eeded450d11c030ad3e088"} Dec 03 12:12:37 crc kubenswrapper[5052]: I1203 12:12:37.889929 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:12:37 crc kubenswrapper[5052]: I1203 12:12:37.889927 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="62324d6b-4dc8-475e-845d-ade03d0aeb66" containerName="proxy-httpd" containerID="cri-o://aa1eb04f39b67e01fee6a0d26805faf02b005c3d50eeded450d11c030ad3e088" gracePeriod=30 Dec 03 12:12:37 crc kubenswrapper[5052]: I1203 12:12:37.889972 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="62324d6b-4dc8-475e-845d-ade03d0aeb66" containerName="sg-core" containerID="cri-o://cc1d906a68871533f939b6588bccbe9a3527cc639e711e946364382e647bea70" gracePeriod=30 Dec 03 12:12:37 crc kubenswrapper[5052]: I1203 12:12:37.889983 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="62324d6b-4dc8-475e-845d-ade03d0aeb66" containerName="ceilometer-notification-agent" containerID="cri-o://561134ce71b168b26e0e8ed43abe0e914973e8a5d6b3341d25487208e3088e1f" gracePeriod=30 Dec 03 12:12:37 crc kubenswrapper[5052]: I1203 12:12:37.890293 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="62324d6b-4dc8-475e-845d-ade03d0aeb66" containerName="ceilometer-central-agent" containerID="cri-o://1b5e9176625dce69f16041ac8842c83d6f98cc78dd1d675d303aa5991fbeb50e" gracePeriod=30 Dec 03 12:12:37 crc kubenswrapper[5052]: I1203 12:12:37.931451 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/ceilometer-0" podStartSLOduration=2.5564786489999998 podStartE2EDuration="6.931426907s" podCreationTimestamp="2025-12-03 12:12:31 +0000 UTC" firstStartedPulling="2025-12-03 12:12:32.882482844 +0000 UTC m=+3143.470724608" lastFinishedPulling="2025-12-03 12:12:37.257431102 +0000 UTC m=+3147.845672866" observedRunningTime="2025-12-03 12:12:37.918628952 +0000 UTC m=+3148.506870726" watchObservedRunningTime="2025-12-03 12:12:37.931426907 +0000 UTC m=+3148.519668671" Dec 03 12:12:38 crc kubenswrapper[5052]: I1203 12:12:38.833741 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:12:38 crc kubenswrapper[5052]: I1203 12:12:38.901973 5052 generic.go:334] "Generic (PLEG): container finished" podID="62324d6b-4dc8-475e-845d-ade03d0aeb66" containerID="aa1eb04f39b67e01fee6a0d26805faf02b005c3d50eeded450d11c030ad3e088" exitCode=0 Dec 03 12:12:38 crc kubenswrapper[5052]: I1203 12:12:38.902002 5052 generic.go:334] "Generic (PLEG): container finished" podID="62324d6b-4dc8-475e-845d-ade03d0aeb66" containerID="cc1d906a68871533f939b6588bccbe9a3527cc639e711e946364382e647bea70" exitCode=2 Dec 03 12:12:38 crc kubenswrapper[5052]: I1203 12:12:38.902010 5052 generic.go:334] "Generic (PLEG): container finished" podID="62324d6b-4dc8-475e-845d-ade03d0aeb66" containerID="561134ce71b168b26e0e8ed43abe0e914973e8a5d6b3341d25487208e3088e1f" exitCode=0 Dec 03 12:12:38 crc kubenswrapper[5052]: I1203 12:12:38.902018 5052 generic.go:334] "Generic (PLEG): container finished" podID="62324d6b-4dc8-475e-845d-ade03d0aeb66" containerID="1b5e9176625dce69f16041ac8842c83d6f98cc78dd1d675d303aa5991fbeb50e" exitCode=0 Dec 03 12:12:38 crc kubenswrapper[5052]: I1203 12:12:38.902038 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"62324d6b-4dc8-475e-845d-ade03d0aeb66","Type":"ContainerDied","Data":"aa1eb04f39b67e01fee6a0d26805faf02b005c3d50eeded450d11c030ad3e088"} Dec 03 12:12:38 crc kubenswrapper[5052]: I1203 12:12:38.902049 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:12:38 crc kubenswrapper[5052]: I1203 12:12:38.902066 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"62324d6b-4dc8-475e-845d-ade03d0aeb66","Type":"ContainerDied","Data":"cc1d906a68871533f939b6588bccbe9a3527cc639e711e946364382e647bea70"} Dec 03 12:12:38 crc kubenswrapper[5052]: I1203 12:12:38.902078 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"62324d6b-4dc8-475e-845d-ade03d0aeb66","Type":"ContainerDied","Data":"561134ce71b168b26e0e8ed43abe0e914973e8a5d6b3341d25487208e3088e1f"} Dec 03 12:12:38 crc kubenswrapper[5052]: I1203 12:12:38.902088 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"62324d6b-4dc8-475e-845d-ade03d0aeb66","Type":"ContainerDied","Data":"1b5e9176625dce69f16041ac8842c83d6f98cc78dd1d675d303aa5991fbeb50e"} Dec 03 12:12:38 crc kubenswrapper[5052]: I1203 12:12:38.902097 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"62324d6b-4dc8-475e-845d-ade03d0aeb66","Type":"ContainerDied","Data":"18124cd859d2a7dbd10315d6f162299bc1509060136071432394850b9aa109a8"} Dec 03 12:12:38 crc kubenswrapper[5052]: I1203 12:12:38.902113 5052 scope.go:117] "RemoveContainer" containerID="aa1eb04f39b67e01fee6a0d26805faf02b005c3d50eeded450d11c030ad3e088" Dec 03 12:12:38 crc kubenswrapper[5052]: I1203 12:12:38.923272 5052 scope.go:117] "RemoveContainer" containerID="cc1d906a68871533f939b6588bccbe9a3527cc639e711e946364382e647bea70" Dec 03 12:12:38 crc kubenswrapper[5052]: I1203 12:12:38.942498 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jxq8j\" (UniqueName: \"kubernetes.io/projected/62324d6b-4dc8-475e-845d-ade03d0aeb66-kube-api-access-jxq8j\") pod \"62324d6b-4dc8-475e-845d-ade03d0aeb66\" (UID: \"62324d6b-4dc8-475e-845d-ade03d0aeb66\") " Dec 03 12:12:38 crc kubenswrapper[5052]: I1203 12:12:38.942579 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/62324d6b-4dc8-475e-845d-ade03d0aeb66-scripts\") pod \"62324d6b-4dc8-475e-845d-ade03d0aeb66\" (UID: \"62324d6b-4dc8-475e-845d-ade03d0aeb66\") " Dec 03 12:12:38 crc kubenswrapper[5052]: I1203 12:12:38.942606 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/62324d6b-4dc8-475e-845d-ade03d0aeb66-sg-core-conf-yaml\") pod \"62324d6b-4dc8-475e-845d-ade03d0aeb66\" (UID: \"62324d6b-4dc8-475e-845d-ade03d0aeb66\") " Dec 03 12:12:38 crc kubenswrapper[5052]: I1203 12:12:38.942668 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/62324d6b-4dc8-475e-845d-ade03d0aeb66-log-httpd\") pod \"62324d6b-4dc8-475e-845d-ade03d0aeb66\" (UID: \"62324d6b-4dc8-475e-845d-ade03d0aeb66\") " Dec 03 12:12:38 crc kubenswrapper[5052]: I1203 12:12:38.942714 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/62324d6b-4dc8-475e-845d-ade03d0aeb66-combined-ca-bundle\") pod \"62324d6b-4dc8-475e-845d-ade03d0aeb66\" (UID: \"62324d6b-4dc8-475e-845d-ade03d0aeb66\") " Dec 03 12:12:38 crc kubenswrapper[5052]: I1203 12:12:38.943314 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/62324d6b-4dc8-475e-845d-ade03d0aeb66-config-data\") pod \"62324d6b-4dc8-475e-845d-ade03d0aeb66\" (UID: \"62324d6b-4dc8-475e-845d-ade03d0aeb66\") " Dec 03 12:12:38 crc kubenswrapper[5052]: I1203 12:12:38.943248 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/62324d6b-4dc8-475e-845d-ade03d0aeb66-log-httpd" (OuterVolumeSpecName: "log-httpd") pod "62324d6b-4dc8-475e-845d-ade03d0aeb66" (UID: "62324d6b-4dc8-475e-845d-ade03d0aeb66"). InnerVolumeSpecName "log-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:12:38 crc kubenswrapper[5052]: I1203 12:12:38.943800 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/62324d6b-4dc8-475e-845d-ade03d0aeb66-run-httpd\") pod \"62324d6b-4dc8-475e-845d-ade03d0aeb66\" (UID: \"62324d6b-4dc8-475e-845d-ade03d0aeb66\") " Dec 03 12:12:38 crc kubenswrapper[5052]: I1203 12:12:38.944225 5052 reconciler_common.go:293] "Volume detached for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/62324d6b-4dc8-475e-845d-ade03d0aeb66-log-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:12:38 crc kubenswrapper[5052]: I1203 12:12:38.944451 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/62324d6b-4dc8-475e-845d-ade03d0aeb66-run-httpd" (OuterVolumeSpecName: "run-httpd") pod "62324d6b-4dc8-475e-845d-ade03d0aeb66" (UID: "62324d6b-4dc8-475e-845d-ade03d0aeb66"). InnerVolumeSpecName "run-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:12:38 crc kubenswrapper[5052]: I1203 12:12:38.944464 5052 scope.go:117] "RemoveContainer" containerID="561134ce71b168b26e0e8ed43abe0e914973e8a5d6b3341d25487208e3088e1f" Dec 03 12:12:38 crc kubenswrapper[5052]: I1203 12:12:38.953764 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/62324d6b-4dc8-475e-845d-ade03d0aeb66-scripts" (OuterVolumeSpecName: "scripts") pod "62324d6b-4dc8-475e-845d-ade03d0aeb66" (UID: "62324d6b-4dc8-475e-845d-ade03d0aeb66"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:12:38 crc kubenswrapper[5052]: I1203 12:12:38.953838 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/62324d6b-4dc8-475e-845d-ade03d0aeb66-kube-api-access-jxq8j" (OuterVolumeSpecName: "kube-api-access-jxq8j") pod "62324d6b-4dc8-475e-845d-ade03d0aeb66" (UID: "62324d6b-4dc8-475e-845d-ade03d0aeb66"). InnerVolumeSpecName "kube-api-access-jxq8j". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:12:38 crc kubenswrapper[5052]: I1203 12:12:38.977436 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/62324d6b-4dc8-475e-845d-ade03d0aeb66-sg-core-conf-yaml" (OuterVolumeSpecName: "sg-core-conf-yaml") pod "62324d6b-4dc8-475e-845d-ade03d0aeb66" (UID: "62324d6b-4dc8-475e-845d-ade03d0aeb66"). InnerVolumeSpecName "sg-core-conf-yaml". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:12:39 crc kubenswrapper[5052]: I1203 12:12:39.018575 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/swift-proxy-6b68c4db5d-g8nvc" Dec 03 12:12:39 crc kubenswrapper[5052]: I1203 12:12:39.022680 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/62324d6b-4dc8-475e-845d-ade03d0aeb66-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "62324d6b-4dc8-475e-845d-ade03d0aeb66" (UID: "62324d6b-4dc8-475e-845d-ade03d0aeb66"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:12:39 crc kubenswrapper[5052]: I1203 12:12:39.058046 5052 reconciler_common.go:293] "Volume detached for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/62324d6b-4dc8-475e-845d-ade03d0aeb66-run-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:12:39 crc kubenswrapper[5052]: I1203 12:12:39.058080 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-jxq8j\" (UniqueName: \"kubernetes.io/projected/62324d6b-4dc8-475e-845d-ade03d0aeb66-kube-api-access-jxq8j\") on node \"crc\" DevicePath \"\"" Dec 03 12:12:39 crc kubenswrapper[5052]: I1203 12:12:39.058092 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/62324d6b-4dc8-475e-845d-ade03d0aeb66-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:12:39 crc kubenswrapper[5052]: I1203 12:12:39.058103 5052 reconciler_common.go:293] "Volume detached for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/62324d6b-4dc8-475e-845d-ade03d0aeb66-sg-core-conf-yaml\") on node \"crc\" DevicePath \"\"" Dec 03 12:12:39 crc kubenswrapper[5052]: I1203 12:12:39.058114 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/62324d6b-4dc8-475e-845d-ade03d0aeb66-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:12:39 crc kubenswrapper[5052]: I1203 12:12:39.082194 5052 scope.go:117] "RemoveContainer" containerID="1b5e9176625dce69f16041ac8842c83d6f98cc78dd1d675d303aa5991fbeb50e" Dec 03 12:12:39 crc kubenswrapper[5052]: I1203 12:12:39.093044 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/62324d6b-4dc8-475e-845d-ade03d0aeb66-config-data" (OuterVolumeSpecName: "config-data") pod "62324d6b-4dc8-475e-845d-ade03d0aeb66" (UID: "62324d6b-4dc8-475e-845d-ade03d0aeb66"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:12:39 crc kubenswrapper[5052]: I1203 12:12:39.107815 5052 scope.go:117] "RemoveContainer" containerID="aa1eb04f39b67e01fee6a0d26805faf02b005c3d50eeded450d11c030ad3e088" Dec 03 12:12:39 crc kubenswrapper[5052]: E1203 12:12:39.108882 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aa1eb04f39b67e01fee6a0d26805faf02b005c3d50eeded450d11c030ad3e088\": container with ID starting with aa1eb04f39b67e01fee6a0d26805faf02b005c3d50eeded450d11c030ad3e088 not found: ID does not exist" containerID="aa1eb04f39b67e01fee6a0d26805faf02b005c3d50eeded450d11c030ad3e088" Dec 03 12:12:39 crc kubenswrapper[5052]: I1203 12:12:39.109000 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aa1eb04f39b67e01fee6a0d26805faf02b005c3d50eeded450d11c030ad3e088"} err="failed to get container status \"aa1eb04f39b67e01fee6a0d26805faf02b005c3d50eeded450d11c030ad3e088\": rpc error: code = NotFound desc = could not find container \"aa1eb04f39b67e01fee6a0d26805faf02b005c3d50eeded450d11c030ad3e088\": container with ID starting with aa1eb04f39b67e01fee6a0d26805faf02b005c3d50eeded450d11c030ad3e088 not found: ID does not exist" Dec 03 12:12:39 crc kubenswrapper[5052]: I1203 12:12:39.109089 5052 scope.go:117] "RemoveContainer" containerID="cc1d906a68871533f939b6588bccbe9a3527cc639e711e946364382e647bea70" Dec 03 12:12:39 crc kubenswrapper[5052]: E1203 12:12:39.109384 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cc1d906a68871533f939b6588bccbe9a3527cc639e711e946364382e647bea70\": container with ID starting with cc1d906a68871533f939b6588bccbe9a3527cc639e711e946364382e647bea70 not found: ID does not exist" containerID="cc1d906a68871533f939b6588bccbe9a3527cc639e711e946364382e647bea70" Dec 03 12:12:39 crc kubenswrapper[5052]: I1203 12:12:39.109414 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cc1d906a68871533f939b6588bccbe9a3527cc639e711e946364382e647bea70"} err="failed to get container status \"cc1d906a68871533f939b6588bccbe9a3527cc639e711e946364382e647bea70\": rpc error: code = NotFound desc = could not find container \"cc1d906a68871533f939b6588bccbe9a3527cc639e711e946364382e647bea70\": container with ID starting with cc1d906a68871533f939b6588bccbe9a3527cc639e711e946364382e647bea70 not found: ID does not exist" Dec 03 12:12:39 crc kubenswrapper[5052]: I1203 12:12:39.109427 5052 scope.go:117] "RemoveContainer" containerID="561134ce71b168b26e0e8ed43abe0e914973e8a5d6b3341d25487208e3088e1f" Dec 03 12:12:39 crc kubenswrapper[5052]: E1203 12:12:39.109675 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"561134ce71b168b26e0e8ed43abe0e914973e8a5d6b3341d25487208e3088e1f\": container with ID starting with 561134ce71b168b26e0e8ed43abe0e914973e8a5d6b3341d25487208e3088e1f not found: ID does not exist" containerID="561134ce71b168b26e0e8ed43abe0e914973e8a5d6b3341d25487208e3088e1f" Dec 03 12:12:39 crc kubenswrapper[5052]: I1203 12:12:39.109750 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"561134ce71b168b26e0e8ed43abe0e914973e8a5d6b3341d25487208e3088e1f"} err="failed to get container status \"561134ce71b168b26e0e8ed43abe0e914973e8a5d6b3341d25487208e3088e1f\": rpc error: code = NotFound desc = could not find container \"561134ce71b168b26e0e8ed43abe0e914973e8a5d6b3341d25487208e3088e1f\": container with ID starting with 561134ce71b168b26e0e8ed43abe0e914973e8a5d6b3341d25487208e3088e1f not found: ID does not exist" Dec 03 12:12:39 crc kubenswrapper[5052]: I1203 12:12:39.109827 5052 scope.go:117] "RemoveContainer" containerID="1b5e9176625dce69f16041ac8842c83d6f98cc78dd1d675d303aa5991fbeb50e" Dec 03 12:12:39 crc kubenswrapper[5052]: E1203 12:12:39.113176 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1b5e9176625dce69f16041ac8842c83d6f98cc78dd1d675d303aa5991fbeb50e\": container with ID starting with 1b5e9176625dce69f16041ac8842c83d6f98cc78dd1d675d303aa5991fbeb50e not found: ID does not exist" containerID="1b5e9176625dce69f16041ac8842c83d6f98cc78dd1d675d303aa5991fbeb50e" Dec 03 12:12:39 crc kubenswrapper[5052]: I1203 12:12:39.113407 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1b5e9176625dce69f16041ac8842c83d6f98cc78dd1d675d303aa5991fbeb50e"} err="failed to get container status \"1b5e9176625dce69f16041ac8842c83d6f98cc78dd1d675d303aa5991fbeb50e\": rpc error: code = NotFound desc = could not find container \"1b5e9176625dce69f16041ac8842c83d6f98cc78dd1d675d303aa5991fbeb50e\": container with ID starting with 1b5e9176625dce69f16041ac8842c83d6f98cc78dd1d675d303aa5991fbeb50e not found: ID does not exist" Dec 03 12:12:39 crc kubenswrapper[5052]: I1203 12:12:39.113554 5052 scope.go:117] "RemoveContainer" containerID="aa1eb04f39b67e01fee6a0d26805faf02b005c3d50eeded450d11c030ad3e088" Dec 03 12:12:39 crc kubenswrapper[5052]: I1203 12:12:39.113935 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aa1eb04f39b67e01fee6a0d26805faf02b005c3d50eeded450d11c030ad3e088"} err="failed to get container status \"aa1eb04f39b67e01fee6a0d26805faf02b005c3d50eeded450d11c030ad3e088\": rpc error: code = NotFound desc = could not find container \"aa1eb04f39b67e01fee6a0d26805faf02b005c3d50eeded450d11c030ad3e088\": container with ID starting with aa1eb04f39b67e01fee6a0d26805faf02b005c3d50eeded450d11c030ad3e088 not found: ID does not exist" Dec 03 12:12:39 crc kubenswrapper[5052]: I1203 12:12:39.114211 5052 scope.go:117] "RemoveContainer" containerID="cc1d906a68871533f939b6588bccbe9a3527cc639e711e946364382e647bea70" Dec 03 12:12:39 crc kubenswrapper[5052]: I1203 12:12:39.115180 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cc1d906a68871533f939b6588bccbe9a3527cc639e711e946364382e647bea70"} err="failed to get container status \"cc1d906a68871533f939b6588bccbe9a3527cc639e711e946364382e647bea70\": rpc error: code = NotFound desc = could not find container \"cc1d906a68871533f939b6588bccbe9a3527cc639e711e946364382e647bea70\": container with ID starting with cc1d906a68871533f939b6588bccbe9a3527cc639e711e946364382e647bea70 not found: ID does not exist" Dec 03 12:12:39 crc kubenswrapper[5052]: I1203 12:12:39.115324 5052 scope.go:117] "RemoveContainer" containerID="561134ce71b168b26e0e8ed43abe0e914973e8a5d6b3341d25487208e3088e1f" Dec 03 12:12:39 crc kubenswrapper[5052]: I1203 12:12:39.115688 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"561134ce71b168b26e0e8ed43abe0e914973e8a5d6b3341d25487208e3088e1f"} err="failed to get container status \"561134ce71b168b26e0e8ed43abe0e914973e8a5d6b3341d25487208e3088e1f\": rpc error: code = NotFound desc = could not find container \"561134ce71b168b26e0e8ed43abe0e914973e8a5d6b3341d25487208e3088e1f\": container with ID starting with 561134ce71b168b26e0e8ed43abe0e914973e8a5d6b3341d25487208e3088e1f not found: ID does not exist" Dec 03 12:12:39 crc kubenswrapper[5052]: I1203 12:12:39.115787 5052 scope.go:117] "RemoveContainer" containerID="1b5e9176625dce69f16041ac8842c83d6f98cc78dd1d675d303aa5991fbeb50e" Dec 03 12:12:39 crc kubenswrapper[5052]: I1203 12:12:39.116243 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1b5e9176625dce69f16041ac8842c83d6f98cc78dd1d675d303aa5991fbeb50e"} err="failed to get container status \"1b5e9176625dce69f16041ac8842c83d6f98cc78dd1d675d303aa5991fbeb50e\": rpc error: code = NotFound desc = could not find container \"1b5e9176625dce69f16041ac8842c83d6f98cc78dd1d675d303aa5991fbeb50e\": container with ID starting with 1b5e9176625dce69f16041ac8842c83d6f98cc78dd1d675d303aa5991fbeb50e not found: ID does not exist" Dec 03 12:12:39 crc kubenswrapper[5052]: I1203 12:12:39.116368 5052 scope.go:117] "RemoveContainer" containerID="aa1eb04f39b67e01fee6a0d26805faf02b005c3d50eeded450d11c030ad3e088" Dec 03 12:12:39 crc kubenswrapper[5052]: I1203 12:12:39.116729 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aa1eb04f39b67e01fee6a0d26805faf02b005c3d50eeded450d11c030ad3e088"} err="failed to get container status \"aa1eb04f39b67e01fee6a0d26805faf02b005c3d50eeded450d11c030ad3e088\": rpc error: code = NotFound desc = could not find container \"aa1eb04f39b67e01fee6a0d26805faf02b005c3d50eeded450d11c030ad3e088\": container with ID starting with aa1eb04f39b67e01fee6a0d26805faf02b005c3d50eeded450d11c030ad3e088 not found: ID does not exist" Dec 03 12:12:39 crc kubenswrapper[5052]: I1203 12:12:39.116829 5052 scope.go:117] "RemoveContainer" containerID="cc1d906a68871533f939b6588bccbe9a3527cc639e711e946364382e647bea70" Dec 03 12:12:39 crc kubenswrapper[5052]: I1203 12:12:39.117310 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cc1d906a68871533f939b6588bccbe9a3527cc639e711e946364382e647bea70"} err="failed to get container status \"cc1d906a68871533f939b6588bccbe9a3527cc639e711e946364382e647bea70\": rpc error: code = NotFound desc = could not find container \"cc1d906a68871533f939b6588bccbe9a3527cc639e711e946364382e647bea70\": container with ID starting with cc1d906a68871533f939b6588bccbe9a3527cc639e711e946364382e647bea70 not found: ID does not exist" Dec 03 12:12:39 crc kubenswrapper[5052]: I1203 12:12:39.117340 5052 scope.go:117] "RemoveContainer" containerID="561134ce71b168b26e0e8ed43abe0e914973e8a5d6b3341d25487208e3088e1f" Dec 03 12:12:39 crc kubenswrapper[5052]: I1203 12:12:39.117670 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"561134ce71b168b26e0e8ed43abe0e914973e8a5d6b3341d25487208e3088e1f"} err="failed to get container status \"561134ce71b168b26e0e8ed43abe0e914973e8a5d6b3341d25487208e3088e1f\": rpc error: code = NotFound desc = could not find container \"561134ce71b168b26e0e8ed43abe0e914973e8a5d6b3341d25487208e3088e1f\": container with ID starting with 561134ce71b168b26e0e8ed43abe0e914973e8a5d6b3341d25487208e3088e1f not found: ID does not exist" Dec 03 12:12:39 crc kubenswrapper[5052]: I1203 12:12:39.117699 5052 scope.go:117] "RemoveContainer" containerID="1b5e9176625dce69f16041ac8842c83d6f98cc78dd1d675d303aa5991fbeb50e" Dec 03 12:12:39 crc kubenswrapper[5052]: I1203 12:12:39.118014 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1b5e9176625dce69f16041ac8842c83d6f98cc78dd1d675d303aa5991fbeb50e"} err="failed to get container status \"1b5e9176625dce69f16041ac8842c83d6f98cc78dd1d675d303aa5991fbeb50e\": rpc error: code = NotFound desc = could not find container \"1b5e9176625dce69f16041ac8842c83d6f98cc78dd1d675d303aa5991fbeb50e\": container with ID starting with 1b5e9176625dce69f16041ac8842c83d6f98cc78dd1d675d303aa5991fbeb50e not found: ID does not exist" Dec 03 12:12:39 crc kubenswrapper[5052]: I1203 12:12:39.118030 5052 scope.go:117] "RemoveContainer" containerID="aa1eb04f39b67e01fee6a0d26805faf02b005c3d50eeded450d11c030ad3e088" Dec 03 12:12:39 crc kubenswrapper[5052]: I1203 12:12:39.118212 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aa1eb04f39b67e01fee6a0d26805faf02b005c3d50eeded450d11c030ad3e088"} err="failed to get container status \"aa1eb04f39b67e01fee6a0d26805faf02b005c3d50eeded450d11c030ad3e088\": rpc error: code = NotFound desc = could not find container \"aa1eb04f39b67e01fee6a0d26805faf02b005c3d50eeded450d11c030ad3e088\": container with ID starting with aa1eb04f39b67e01fee6a0d26805faf02b005c3d50eeded450d11c030ad3e088 not found: ID does not exist" Dec 03 12:12:39 crc kubenswrapper[5052]: I1203 12:12:39.118236 5052 scope.go:117] "RemoveContainer" containerID="cc1d906a68871533f939b6588bccbe9a3527cc639e711e946364382e647bea70" Dec 03 12:12:39 crc kubenswrapper[5052]: I1203 12:12:39.118453 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cc1d906a68871533f939b6588bccbe9a3527cc639e711e946364382e647bea70"} err="failed to get container status \"cc1d906a68871533f939b6588bccbe9a3527cc639e711e946364382e647bea70\": rpc error: code = NotFound desc = could not find container \"cc1d906a68871533f939b6588bccbe9a3527cc639e711e946364382e647bea70\": container with ID starting with cc1d906a68871533f939b6588bccbe9a3527cc639e711e946364382e647bea70 not found: ID does not exist" Dec 03 12:12:39 crc kubenswrapper[5052]: I1203 12:12:39.118472 5052 scope.go:117] "RemoveContainer" containerID="561134ce71b168b26e0e8ed43abe0e914973e8a5d6b3341d25487208e3088e1f" Dec 03 12:12:39 crc kubenswrapper[5052]: I1203 12:12:39.118750 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"561134ce71b168b26e0e8ed43abe0e914973e8a5d6b3341d25487208e3088e1f"} err="failed to get container status \"561134ce71b168b26e0e8ed43abe0e914973e8a5d6b3341d25487208e3088e1f\": rpc error: code = NotFound desc = could not find container \"561134ce71b168b26e0e8ed43abe0e914973e8a5d6b3341d25487208e3088e1f\": container with ID starting with 561134ce71b168b26e0e8ed43abe0e914973e8a5d6b3341d25487208e3088e1f not found: ID does not exist" Dec 03 12:12:39 crc kubenswrapper[5052]: I1203 12:12:39.118772 5052 scope.go:117] "RemoveContainer" containerID="1b5e9176625dce69f16041ac8842c83d6f98cc78dd1d675d303aa5991fbeb50e" Dec 03 12:12:39 crc kubenswrapper[5052]: I1203 12:12:39.118999 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1b5e9176625dce69f16041ac8842c83d6f98cc78dd1d675d303aa5991fbeb50e"} err="failed to get container status \"1b5e9176625dce69f16041ac8842c83d6f98cc78dd1d675d303aa5991fbeb50e\": rpc error: code = NotFound desc = could not find container \"1b5e9176625dce69f16041ac8842c83d6f98cc78dd1d675d303aa5991fbeb50e\": container with ID starting with 1b5e9176625dce69f16041ac8842c83d6f98cc78dd1d675d303aa5991fbeb50e not found: ID does not exist" Dec 03 12:12:39 crc kubenswrapper[5052]: I1203 12:12:39.160269 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/62324d6b-4dc8-475e-845d-ade03d0aeb66-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:12:39 crc kubenswrapper[5052]: I1203 12:12:39.257493 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:12:39 crc kubenswrapper[5052]: I1203 12:12:39.282400 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:12:39 crc kubenswrapper[5052]: I1203 12:12:39.301167 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:12:39 crc kubenswrapper[5052]: E1203 12:12:39.301688 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="62324d6b-4dc8-475e-845d-ade03d0aeb66" containerName="ceilometer-notification-agent" Dec 03 12:12:39 crc kubenswrapper[5052]: I1203 12:12:39.301704 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="62324d6b-4dc8-475e-845d-ade03d0aeb66" containerName="ceilometer-notification-agent" Dec 03 12:12:39 crc kubenswrapper[5052]: E1203 12:12:39.301730 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="62324d6b-4dc8-475e-845d-ade03d0aeb66" containerName="proxy-httpd" Dec 03 12:12:39 crc kubenswrapper[5052]: I1203 12:12:39.301738 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="62324d6b-4dc8-475e-845d-ade03d0aeb66" containerName="proxy-httpd" Dec 03 12:12:39 crc kubenswrapper[5052]: E1203 12:12:39.301759 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="62324d6b-4dc8-475e-845d-ade03d0aeb66" containerName="sg-core" Dec 03 12:12:39 crc kubenswrapper[5052]: I1203 12:12:39.301767 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="62324d6b-4dc8-475e-845d-ade03d0aeb66" containerName="sg-core" Dec 03 12:12:39 crc kubenswrapper[5052]: E1203 12:12:39.301803 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="62324d6b-4dc8-475e-845d-ade03d0aeb66" containerName="ceilometer-central-agent" Dec 03 12:12:39 crc kubenswrapper[5052]: I1203 12:12:39.301811 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="62324d6b-4dc8-475e-845d-ade03d0aeb66" containerName="ceilometer-central-agent" Dec 03 12:12:39 crc kubenswrapper[5052]: I1203 12:12:39.303212 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="62324d6b-4dc8-475e-845d-ade03d0aeb66" containerName="proxy-httpd" Dec 03 12:12:39 crc kubenswrapper[5052]: I1203 12:12:39.303234 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="62324d6b-4dc8-475e-845d-ade03d0aeb66" containerName="ceilometer-central-agent" Dec 03 12:12:39 crc kubenswrapper[5052]: I1203 12:12:39.303263 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="62324d6b-4dc8-475e-845d-ade03d0aeb66" containerName="sg-core" Dec 03 12:12:39 crc kubenswrapper[5052]: I1203 12:12:39.303305 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="62324d6b-4dc8-475e-845d-ade03d0aeb66" containerName="ceilometer-notification-agent" Dec 03 12:12:39 crc kubenswrapper[5052]: I1203 12:12:39.309255 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:12:39 crc kubenswrapper[5052]: I1203 12:12:39.314311 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-scripts" Dec 03 12:12:39 crc kubenswrapper[5052]: I1203 12:12:39.314534 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-config-data" Dec 03 12:12:39 crc kubenswrapper[5052]: I1203 12:12:39.318305 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:12:39 crc kubenswrapper[5052]: I1203 12:12:39.469485 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5b95r\" (UniqueName: \"kubernetes.io/projected/6525e97e-19f7-40e8-b4fa-03efcc539ba1-kube-api-access-5b95r\") pod \"ceilometer-0\" (UID: \"6525e97e-19f7-40e8-b4fa-03efcc539ba1\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:12:39 crc kubenswrapper[5052]: I1203 12:12:39.469593 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/6525e97e-19f7-40e8-b4fa-03efcc539ba1-run-httpd\") pod \"ceilometer-0\" (UID: \"6525e97e-19f7-40e8-b4fa-03efcc539ba1\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:12:39 crc kubenswrapper[5052]: I1203 12:12:39.469707 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6525e97e-19f7-40e8-b4fa-03efcc539ba1-config-data\") pod \"ceilometer-0\" (UID: \"6525e97e-19f7-40e8-b4fa-03efcc539ba1\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:12:39 crc kubenswrapper[5052]: I1203 12:12:39.469772 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/6525e97e-19f7-40e8-b4fa-03efcc539ba1-log-httpd\") pod \"ceilometer-0\" (UID: \"6525e97e-19f7-40e8-b4fa-03efcc539ba1\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:12:39 crc kubenswrapper[5052]: I1203 12:12:39.469807 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/6525e97e-19f7-40e8-b4fa-03efcc539ba1-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"6525e97e-19f7-40e8-b4fa-03efcc539ba1\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:12:39 crc kubenswrapper[5052]: I1203 12:12:39.469890 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6525e97e-19f7-40e8-b4fa-03efcc539ba1-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"6525e97e-19f7-40e8-b4fa-03efcc539ba1\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:12:39 crc kubenswrapper[5052]: I1203 12:12:39.469929 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/6525e97e-19f7-40e8-b4fa-03efcc539ba1-scripts\") pod \"ceilometer-0\" (UID: \"6525e97e-19f7-40e8-b4fa-03efcc539ba1\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:12:39 crc kubenswrapper[5052]: I1203 12:12:39.546997 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:12:39 crc kubenswrapper[5052]: E1203 12:12:39.547800 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[combined-ca-bundle config-data kube-api-access-5b95r log-httpd run-httpd scripts sg-core-conf-yaml], unattached volumes=[], failed to process volumes=[]: context canceled" pod="openstack-kuttl-tests/ceilometer-0" podUID="6525e97e-19f7-40e8-b4fa-03efcc539ba1" Dec 03 12:12:39 crc kubenswrapper[5052]: I1203 12:12:39.572315 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-5b95r\" (UniqueName: \"kubernetes.io/projected/6525e97e-19f7-40e8-b4fa-03efcc539ba1-kube-api-access-5b95r\") pod \"ceilometer-0\" (UID: \"6525e97e-19f7-40e8-b4fa-03efcc539ba1\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:12:39 crc kubenswrapper[5052]: I1203 12:12:39.572396 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/6525e97e-19f7-40e8-b4fa-03efcc539ba1-run-httpd\") pod \"ceilometer-0\" (UID: \"6525e97e-19f7-40e8-b4fa-03efcc539ba1\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:12:39 crc kubenswrapper[5052]: I1203 12:12:39.572472 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6525e97e-19f7-40e8-b4fa-03efcc539ba1-config-data\") pod \"ceilometer-0\" (UID: \"6525e97e-19f7-40e8-b4fa-03efcc539ba1\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:12:39 crc kubenswrapper[5052]: I1203 12:12:39.572541 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/6525e97e-19f7-40e8-b4fa-03efcc539ba1-log-httpd\") pod \"ceilometer-0\" (UID: \"6525e97e-19f7-40e8-b4fa-03efcc539ba1\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:12:39 crc kubenswrapper[5052]: I1203 12:12:39.572565 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/6525e97e-19f7-40e8-b4fa-03efcc539ba1-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"6525e97e-19f7-40e8-b4fa-03efcc539ba1\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:12:39 crc kubenswrapper[5052]: I1203 12:12:39.572631 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6525e97e-19f7-40e8-b4fa-03efcc539ba1-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"6525e97e-19f7-40e8-b4fa-03efcc539ba1\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:12:39 crc kubenswrapper[5052]: I1203 12:12:39.572677 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/6525e97e-19f7-40e8-b4fa-03efcc539ba1-scripts\") pod \"ceilometer-0\" (UID: \"6525e97e-19f7-40e8-b4fa-03efcc539ba1\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:12:39 crc kubenswrapper[5052]: I1203 12:12:39.573393 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/6525e97e-19f7-40e8-b4fa-03efcc539ba1-log-httpd\") pod \"ceilometer-0\" (UID: \"6525e97e-19f7-40e8-b4fa-03efcc539ba1\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:12:39 crc kubenswrapper[5052]: I1203 12:12:39.573855 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/6525e97e-19f7-40e8-b4fa-03efcc539ba1-run-httpd\") pod \"ceilometer-0\" (UID: \"6525e97e-19f7-40e8-b4fa-03efcc539ba1\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:12:39 crc kubenswrapper[5052]: I1203 12:12:39.577869 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/6525e97e-19f7-40e8-b4fa-03efcc539ba1-scripts\") pod \"ceilometer-0\" (UID: \"6525e97e-19f7-40e8-b4fa-03efcc539ba1\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:12:39 crc kubenswrapper[5052]: I1203 12:12:39.578361 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6525e97e-19f7-40e8-b4fa-03efcc539ba1-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"6525e97e-19f7-40e8-b4fa-03efcc539ba1\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:12:39 crc kubenswrapper[5052]: I1203 12:12:39.581593 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6525e97e-19f7-40e8-b4fa-03efcc539ba1-config-data\") pod \"ceilometer-0\" (UID: \"6525e97e-19f7-40e8-b4fa-03efcc539ba1\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:12:39 crc kubenswrapper[5052]: I1203 12:12:39.587259 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/6525e97e-19f7-40e8-b4fa-03efcc539ba1-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"6525e97e-19f7-40e8-b4fa-03efcc539ba1\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:12:39 crc kubenswrapper[5052]: I1203 12:12:39.594414 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-5b95r\" (UniqueName: \"kubernetes.io/projected/6525e97e-19f7-40e8-b4fa-03efcc539ba1-kube-api-access-5b95r\") pod \"ceilometer-0\" (UID: \"6525e97e-19f7-40e8-b4fa-03efcc539ba1\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:12:39 crc kubenswrapper[5052]: I1203 12:12:39.690945 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="62324d6b-4dc8-475e-845d-ade03d0aeb66" path="/var/lib/kubelet/pods/62324d6b-4dc8-475e-845d-ade03d0aeb66/volumes" Dec 03 12:12:39 crc kubenswrapper[5052]: I1203 12:12:39.919244 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:12:39 crc kubenswrapper[5052]: I1203 12:12:39.934998 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:12:39 crc kubenswrapper[5052]: I1203 12:12:39.957211 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:12:39 crc kubenswrapper[5052]: I1203 12:12:39.957481 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-external-api-0" podUID="0fd9a636-d4a2-45f7-8a00-0b3a2fe364cb" containerName="glance-log" containerID="cri-o://0c0725eafb777c0b6d0ad246f980467eb5fda2912f89f7a5b4867df0ec771fce" gracePeriod=30 Dec 03 12:12:39 crc kubenswrapper[5052]: I1203 12:12:39.957556 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-external-api-0" podUID="0fd9a636-d4a2-45f7-8a00-0b3a2fe364cb" containerName="glance-httpd" containerID="cri-o://3980c92ec54354398e968f8f5cd2dd8d079b22bf5a7562059de4e49cc45e8c2b" gracePeriod=30 Dec 03 12:12:40 crc kubenswrapper[5052]: I1203 12:12:40.082971 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/6525e97e-19f7-40e8-b4fa-03efcc539ba1-sg-core-conf-yaml\") pod \"6525e97e-19f7-40e8-b4fa-03efcc539ba1\" (UID: \"6525e97e-19f7-40e8-b4fa-03efcc539ba1\") " Dec 03 12:12:40 crc kubenswrapper[5052]: I1203 12:12:40.083021 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/6525e97e-19f7-40e8-b4fa-03efcc539ba1-run-httpd\") pod \"6525e97e-19f7-40e8-b4fa-03efcc539ba1\" (UID: \"6525e97e-19f7-40e8-b4fa-03efcc539ba1\") " Dec 03 12:12:40 crc kubenswrapper[5052]: I1203 12:12:40.083052 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/6525e97e-19f7-40e8-b4fa-03efcc539ba1-scripts\") pod \"6525e97e-19f7-40e8-b4fa-03efcc539ba1\" (UID: \"6525e97e-19f7-40e8-b4fa-03efcc539ba1\") " Dec 03 12:12:40 crc kubenswrapper[5052]: I1203 12:12:40.083114 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/6525e97e-19f7-40e8-b4fa-03efcc539ba1-log-httpd\") pod \"6525e97e-19f7-40e8-b4fa-03efcc539ba1\" (UID: \"6525e97e-19f7-40e8-b4fa-03efcc539ba1\") " Dec 03 12:12:40 crc kubenswrapper[5052]: I1203 12:12:40.083149 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6525e97e-19f7-40e8-b4fa-03efcc539ba1-combined-ca-bundle\") pod \"6525e97e-19f7-40e8-b4fa-03efcc539ba1\" (UID: \"6525e97e-19f7-40e8-b4fa-03efcc539ba1\") " Dec 03 12:12:40 crc kubenswrapper[5052]: I1203 12:12:40.083212 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6525e97e-19f7-40e8-b4fa-03efcc539ba1-config-data\") pod \"6525e97e-19f7-40e8-b4fa-03efcc539ba1\" (UID: \"6525e97e-19f7-40e8-b4fa-03efcc539ba1\") " Dec 03 12:12:40 crc kubenswrapper[5052]: I1203 12:12:40.083378 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5b95r\" (UniqueName: \"kubernetes.io/projected/6525e97e-19f7-40e8-b4fa-03efcc539ba1-kube-api-access-5b95r\") pod \"6525e97e-19f7-40e8-b4fa-03efcc539ba1\" (UID: \"6525e97e-19f7-40e8-b4fa-03efcc539ba1\") " Dec 03 12:12:40 crc kubenswrapper[5052]: I1203 12:12:40.086838 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6525e97e-19f7-40e8-b4fa-03efcc539ba1-run-httpd" (OuterVolumeSpecName: "run-httpd") pod "6525e97e-19f7-40e8-b4fa-03efcc539ba1" (UID: "6525e97e-19f7-40e8-b4fa-03efcc539ba1"). InnerVolumeSpecName "run-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:12:40 crc kubenswrapper[5052]: I1203 12:12:40.087097 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6525e97e-19f7-40e8-b4fa-03efcc539ba1-log-httpd" (OuterVolumeSpecName: "log-httpd") pod "6525e97e-19f7-40e8-b4fa-03efcc539ba1" (UID: "6525e97e-19f7-40e8-b4fa-03efcc539ba1"). InnerVolumeSpecName "log-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:12:40 crc kubenswrapper[5052]: I1203 12:12:40.091702 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6525e97e-19f7-40e8-b4fa-03efcc539ba1-config-data" (OuterVolumeSpecName: "config-data") pod "6525e97e-19f7-40e8-b4fa-03efcc539ba1" (UID: "6525e97e-19f7-40e8-b4fa-03efcc539ba1"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:12:40 crc kubenswrapper[5052]: I1203 12:12:40.091723 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6525e97e-19f7-40e8-b4fa-03efcc539ba1-sg-core-conf-yaml" (OuterVolumeSpecName: "sg-core-conf-yaml") pod "6525e97e-19f7-40e8-b4fa-03efcc539ba1" (UID: "6525e97e-19f7-40e8-b4fa-03efcc539ba1"). InnerVolumeSpecName "sg-core-conf-yaml". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:12:40 crc kubenswrapper[5052]: I1203 12:12:40.093525 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6525e97e-19f7-40e8-b4fa-03efcc539ba1-kube-api-access-5b95r" (OuterVolumeSpecName: "kube-api-access-5b95r") pod "6525e97e-19f7-40e8-b4fa-03efcc539ba1" (UID: "6525e97e-19f7-40e8-b4fa-03efcc539ba1"). InnerVolumeSpecName "kube-api-access-5b95r". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:12:40 crc kubenswrapper[5052]: I1203 12:12:40.093686 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6525e97e-19f7-40e8-b4fa-03efcc539ba1-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "6525e97e-19f7-40e8-b4fa-03efcc539ba1" (UID: "6525e97e-19f7-40e8-b4fa-03efcc539ba1"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:12:40 crc kubenswrapper[5052]: I1203 12:12:40.095660 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6525e97e-19f7-40e8-b4fa-03efcc539ba1-scripts" (OuterVolumeSpecName: "scripts") pod "6525e97e-19f7-40e8-b4fa-03efcc539ba1" (UID: "6525e97e-19f7-40e8-b4fa-03efcc539ba1"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:12:40 crc kubenswrapper[5052]: I1203 12:12:40.185857 5052 reconciler_common.go:293] "Volume detached for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/6525e97e-19f7-40e8-b4fa-03efcc539ba1-run-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:12:40 crc kubenswrapper[5052]: I1203 12:12:40.185908 5052 reconciler_common.go:293] "Volume detached for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/6525e97e-19f7-40e8-b4fa-03efcc539ba1-sg-core-conf-yaml\") on node \"crc\" DevicePath \"\"" Dec 03 12:12:40 crc kubenswrapper[5052]: I1203 12:12:40.185925 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/6525e97e-19f7-40e8-b4fa-03efcc539ba1-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:12:40 crc kubenswrapper[5052]: I1203 12:12:40.185937 5052 reconciler_common.go:293] "Volume detached for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/6525e97e-19f7-40e8-b4fa-03efcc539ba1-log-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:12:40 crc kubenswrapper[5052]: I1203 12:12:40.185949 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6525e97e-19f7-40e8-b4fa-03efcc539ba1-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:12:40 crc kubenswrapper[5052]: I1203 12:12:40.185960 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6525e97e-19f7-40e8-b4fa-03efcc539ba1-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:12:40 crc kubenswrapper[5052]: I1203 12:12:40.185973 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-5b95r\" (UniqueName: \"kubernetes.io/projected/6525e97e-19f7-40e8-b4fa-03efcc539ba1-kube-api-access-5b95r\") on node \"crc\" DevicePath \"\"" Dec 03 12:12:40 crc kubenswrapper[5052]: I1203 12:12:40.827273 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:12:40 crc kubenswrapper[5052]: I1203 12:12:40.827916 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-internal-api-0" podUID="03919b22-bf40-4bf2-88cb-fdefc426072c" containerName="glance-log" containerID="cri-o://622d66b2983e1d4bdc07c013e8cf0e59732996d900fcb033de628153df313bb1" gracePeriod=30 Dec 03 12:12:40 crc kubenswrapper[5052]: I1203 12:12:40.827962 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-internal-api-0" podUID="03919b22-bf40-4bf2-88cb-fdefc426072c" containerName="glance-httpd" containerID="cri-o://73d8869b154d3281892503903ffd7422325318d2bf903924c2df74fb075f4f4e" gracePeriod=30 Dec 03 12:12:40 crc kubenswrapper[5052]: I1203 12:12:40.929732 5052 generic.go:334] "Generic (PLEG): container finished" podID="0fd9a636-d4a2-45f7-8a00-0b3a2fe364cb" containerID="0c0725eafb777c0b6d0ad246f980467eb5fda2912f89f7a5b4867df0ec771fce" exitCode=143 Dec 03 12:12:40 crc kubenswrapper[5052]: I1203 12:12:40.929814 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:12:40 crc kubenswrapper[5052]: I1203 12:12:40.936730 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"0fd9a636-d4a2-45f7-8a00-0b3a2fe364cb","Type":"ContainerDied","Data":"0c0725eafb777c0b6d0ad246f980467eb5fda2912f89f7a5b4867df0ec771fce"} Dec 03 12:12:41 crc kubenswrapper[5052]: I1203 12:12:41.016374 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:12:41 crc kubenswrapper[5052]: I1203 12:12:41.032163 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:12:41 crc kubenswrapper[5052]: I1203 12:12:41.046439 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:12:41 crc kubenswrapper[5052]: I1203 12:12:41.049242 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:12:41 crc kubenswrapper[5052]: I1203 12:12:41.053287 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-config-data" Dec 03 12:12:41 crc kubenswrapper[5052]: I1203 12:12:41.053727 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-scripts" Dec 03 12:12:41 crc kubenswrapper[5052]: I1203 12:12:41.056213 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:12:41 crc kubenswrapper[5052]: I1203 12:12:41.102538 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ktrld\" (UniqueName: \"kubernetes.io/projected/f3339fc5-9d8a-4a19-ba83-b5f02f9d86d3-kube-api-access-ktrld\") pod \"ceilometer-0\" (UID: \"f3339fc5-9d8a-4a19-ba83-b5f02f9d86d3\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:12:41 crc kubenswrapper[5052]: I1203 12:12:41.102743 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/f3339fc5-9d8a-4a19-ba83-b5f02f9d86d3-run-httpd\") pod \"ceilometer-0\" (UID: \"f3339fc5-9d8a-4a19-ba83-b5f02f9d86d3\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:12:41 crc kubenswrapper[5052]: I1203 12:12:41.102812 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/f3339fc5-9d8a-4a19-ba83-b5f02f9d86d3-log-httpd\") pod \"ceilometer-0\" (UID: \"f3339fc5-9d8a-4a19-ba83-b5f02f9d86d3\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:12:41 crc kubenswrapper[5052]: I1203 12:12:41.102864 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f3339fc5-9d8a-4a19-ba83-b5f02f9d86d3-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"f3339fc5-9d8a-4a19-ba83-b5f02f9d86d3\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:12:41 crc kubenswrapper[5052]: I1203 12:12:41.102997 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/f3339fc5-9d8a-4a19-ba83-b5f02f9d86d3-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"f3339fc5-9d8a-4a19-ba83-b5f02f9d86d3\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:12:41 crc kubenswrapper[5052]: I1203 12:12:41.103084 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/f3339fc5-9d8a-4a19-ba83-b5f02f9d86d3-config-data\") pod \"ceilometer-0\" (UID: \"f3339fc5-9d8a-4a19-ba83-b5f02f9d86d3\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:12:41 crc kubenswrapper[5052]: I1203 12:12:41.103144 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/f3339fc5-9d8a-4a19-ba83-b5f02f9d86d3-scripts\") pod \"ceilometer-0\" (UID: \"f3339fc5-9d8a-4a19-ba83-b5f02f9d86d3\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:12:41 crc kubenswrapper[5052]: I1203 12:12:41.205044 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/f3339fc5-9d8a-4a19-ba83-b5f02f9d86d3-scripts\") pod \"ceilometer-0\" (UID: \"f3339fc5-9d8a-4a19-ba83-b5f02f9d86d3\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:12:41 crc kubenswrapper[5052]: I1203 12:12:41.205112 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-ktrld\" (UniqueName: \"kubernetes.io/projected/f3339fc5-9d8a-4a19-ba83-b5f02f9d86d3-kube-api-access-ktrld\") pod \"ceilometer-0\" (UID: \"f3339fc5-9d8a-4a19-ba83-b5f02f9d86d3\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:12:41 crc kubenswrapper[5052]: I1203 12:12:41.205177 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/f3339fc5-9d8a-4a19-ba83-b5f02f9d86d3-run-httpd\") pod \"ceilometer-0\" (UID: \"f3339fc5-9d8a-4a19-ba83-b5f02f9d86d3\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:12:41 crc kubenswrapper[5052]: I1203 12:12:41.205201 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/f3339fc5-9d8a-4a19-ba83-b5f02f9d86d3-log-httpd\") pod \"ceilometer-0\" (UID: \"f3339fc5-9d8a-4a19-ba83-b5f02f9d86d3\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:12:41 crc kubenswrapper[5052]: I1203 12:12:41.205223 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f3339fc5-9d8a-4a19-ba83-b5f02f9d86d3-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"f3339fc5-9d8a-4a19-ba83-b5f02f9d86d3\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:12:41 crc kubenswrapper[5052]: I1203 12:12:41.205266 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/f3339fc5-9d8a-4a19-ba83-b5f02f9d86d3-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"f3339fc5-9d8a-4a19-ba83-b5f02f9d86d3\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:12:41 crc kubenswrapper[5052]: I1203 12:12:41.205297 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/f3339fc5-9d8a-4a19-ba83-b5f02f9d86d3-config-data\") pod \"ceilometer-0\" (UID: \"f3339fc5-9d8a-4a19-ba83-b5f02f9d86d3\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:12:41 crc kubenswrapper[5052]: I1203 12:12:41.206603 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/f3339fc5-9d8a-4a19-ba83-b5f02f9d86d3-run-httpd\") pod \"ceilometer-0\" (UID: \"f3339fc5-9d8a-4a19-ba83-b5f02f9d86d3\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:12:41 crc kubenswrapper[5052]: I1203 12:12:41.206628 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/f3339fc5-9d8a-4a19-ba83-b5f02f9d86d3-log-httpd\") pod \"ceilometer-0\" (UID: \"f3339fc5-9d8a-4a19-ba83-b5f02f9d86d3\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:12:41 crc kubenswrapper[5052]: I1203 12:12:41.211325 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/f3339fc5-9d8a-4a19-ba83-b5f02f9d86d3-config-data\") pod \"ceilometer-0\" (UID: \"f3339fc5-9d8a-4a19-ba83-b5f02f9d86d3\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:12:41 crc kubenswrapper[5052]: I1203 12:12:41.212039 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/f3339fc5-9d8a-4a19-ba83-b5f02f9d86d3-scripts\") pod \"ceilometer-0\" (UID: \"f3339fc5-9d8a-4a19-ba83-b5f02f9d86d3\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:12:41 crc kubenswrapper[5052]: I1203 12:12:41.212653 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/f3339fc5-9d8a-4a19-ba83-b5f02f9d86d3-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"f3339fc5-9d8a-4a19-ba83-b5f02f9d86d3\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:12:41 crc kubenswrapper[5052]: I1203 12:12:41.220635 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f3339fc5-9d8a-4a19-ba83-b5f02f9d86d3-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"f3339fc5-9d8a-4a19-ba83-b5f02f9d86d3\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:12:41 crc kubenswrapper[5052]: I1203 12:12:41.223495 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-ktrld\" (UniqueName: \"kubernetes.io/projected/f3339fc5-9d8a-4a19-ba83-b5f02f9d86d3-kube-api-access-ktrld\") pod \"ceilometer-0\" (UID: \"f3339fc5-9d8a-4a19-ba83-b5f02f9d86d3\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:12:41 crc kubenswrapper[5052]: I1203 12:12:41.354688 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:12:41 crc kubenswrapper[5052]: I1203 12:12:41.355981 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:12:41 crc kubenswrapper[5052]: I1203 12:12:41.684163 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6525e97e-19f7-40e8-b4fa-03efcc539ba1" path="/var/lib/kubelet/pods/6525e97e-19f7-40e8-b4fa-03efcc539ba1/volumes" Dec 03 12:12:41 crc kubenswrapper[5052]: W1203 12:12:41.811592 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf3339fc5_9d8a_4a19_ba83_b5f02f9d86d3.slice/crio-3e04272a19e601d7860e121937041377a46908c6403553d4bb7fa51c29ddd7b0 WatchSource:0}: Error finding container 3e04272a19e601d7860e121937041377a46908c6403553d4bb7fa51c29ddd7b0: Status 404 returned error can't find the container with id 3e04272a19e601d7860e121937041377a46908c6403553d4bb7fa51c29ddd7b0 Dec 03 12:12:41 crc kubenswrapper[5052]: I1203 12:12:41.813133 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:12:41 crc kubenswrapper[5052]: I1203 12:12:41.955047 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"f3339fc5-9d8a-4a19-ba83-b5f02f9d86d3","Type":"ContainerStarted","Data":"3e04272a19e601d7860e121937041377a46908c6403553d4bb7fa51c29ddd7b0"} Dec 03 12:12:41 crc kubenswrapper[5052]: I1203 12:12:41.958337 5052 generic.go:334] "Generic (PLEG): container finished" podID="03919b22-bf40-4bf2-88cb-fdefc426072c" containerID="622d66b2983e1d4bdc07c013e8cf0e59732996d900fcb033de628153df313bb1" exitCode=143 Dec 03 12:12:41 crc kubenswrapper[5052]: I1203 12:12:41.958391 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"03919b22-bf40-4bf2-88cb-fdefc426072c","Type":"ContainerDied","Data":"622d66b2983e1d4bdc07c013e8cf0e59732996d900fcb033de628153df313bb1"} Dec 03 12:12:42 crc kubenswrapper[5052]: I1203 12:12:42.975767 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"f3339fc5-9d8a-4a19-ba83-b5f02f9d86d3","Type":"ContainerStarted","Data":"86aeba625efdf4888a66dbb87161495cc7e53e5502649308eea48c73eebda680"} Dec 03 12:12:43 crc kubenswrapper[5052]: I1203 12:12:43.557483 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:12:43 crc kubenswrapper[5052]: I1203 12:12:43.657884 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jc5mm\" (UniqueName: \"kubernetes.io/projected/0fd9a636-d4a2-45f7-8a00-0b3a2fe364cb-kube-api-access-jc5mm\") pod \"0fd9a636-d4a2-45f7-8a00-0b3a2fe364cb\" (UID: \"0fd9a636-d4a2-45f7-8a00-0b3a2fe364cb\") " Dec 03 12:12:43 crc kubenswrapper[5052]: I1203 12:12:43.657954 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/0fd9a636-d4a2-45f7-8a00-0b3a2fe364cb-httpd-run\") pod \"0fd9a636-d4a2-45f7-8a00-0b3a2fe364cb\" (UID: \"0fd9a636-d4a2-45f7-8a00-0b3a2fe364cb\") " Dec 03 12:12:43 crc kubenswrapper[5052]: I1203 12:12:43.658119 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/0fd9a636-d4a2-45f7-8a00-0b3a2fe364cb-scripts\") pod \"0fd9a636-d4a2-45f7-8a00-0b3a2fe364cb\" (UID: \"0fd9a636-d4a2-45f7-8a00-0b3a2fe364cb\") " Dec 03 12:12:43 crc kubenswrapper[5052]: I1203 12:12:43.660711 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0fd9a636-d4a2-45f7-8a00-0b3a2fe364cb-combined-ca-bundle\") pod \"0fd9a636-d4a2-45f7-8a00-0b3a2fe364cb\" (UID: \"0fd9a636-d4a2-45f7-8a00-0b3a2fe364cb\") " Dec 03 12:12:43 crc kubenswrapper[5052]: I1203 12:12:43.660761 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/0fd9a636-d4a2-45f7-8a00-0b3a2fe364cb-logs\") pod \"0fd9a636-d4a2-45f7-8a00-0b3a2fe364cb\" (UID: \"0fd9a636-d4a2-45f7-8a00-0b3a2fe364cb\") " Dec 03 12:12:43 crc kubenswrapper[5052]: I1203 12:12:43.660815 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/0fd9a636-d4a2-45f7-8a00-0b3a2fe364cb-config-data\") pod \"0fd9a636-d4a2-45f7-8a00-0b3a2fe364cb\" (UID: \"0fd9a636-d4a2-45f7-8a00-0b3a2fe364cb\") " Dec 03 12:12:43 crc kubenswrapper[5052]: I1203 12:12:43.660849 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"glance\" (UniqueName: \"kubernetes.io/local-volume/local-storage20-crc\") pod \"0fd9a636-d4a2-45f7-8a00-0b3a2fe364cb\" (UID: \"0fd9a636-d4a2-45f7-8a00-0b3a2fe364cb\") " Dec 03 12:12:43 crc kubenswrapper[5052]: I1203 12:12:43.660901 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0fd9a636-d4a2-45f7-8a00-0b3a2fe364cb-httpd-run" (OuterVolumeSpecName: "httpd-run") pod "0fd9a636-d4a2-45f7-8a00-0b3a2fe364cb" (UID: "0fd9a636-d4a2-45f7-8a00-0b3a2fe364cb"). InnerVolumeSpecName "httpd-run". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:12:43 crc kubenswrapper[5052]: I1203 12:12:43.660932 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/0fd9a636-d4a2-45f7-8a00-0b3a2fe364cb-public-tls-certs\") pod \"0fd9a636-d4a2-45f7-8a00-0b3a2fe364cb\" (UID: \"0fd9a636-d4a2-45f7-8a00-0b3a2fe364cb\") " Dec 03 12:12:43 crc kubenswrapper[5052]: I1203 12:12:43.661764 5052 reconciler_common.go:293] "Volume detached for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/0fd9a636-d4a2-45f7-8a00-0b3a2fe364cb-httpd-run\") on node \"crc\" DevicePath \"\"" Dec 03 12:12:43 crc kubenswrapper[5052]: I1203 12:12:43.662565 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0fd9a636-d4a2-45f7-8a00-0b3a2fe364cb-logs" (OuterVolumeSpecName: "logs") pod "0fd9a636-d4a2-45f7-8a00-0b3a2fe364cb" (UID: "0fd9a636-d4a2-45f7-8a00-0b3a2fe364cb"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:12:43 crc kubenswrapper[5052]: I1203 12:12:43.664019 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0fd9a636-d4a2-45f7-8a00-0b3a2fe364cb-scripts" (OuterVolumeSpecName: "scripts") pod "0fd9a636-d4a2-45f7-8a00-0b3a2fe364cb" (UID: "0fd9a636-d4a2-45f7-8a00-0b3a2fe364cb"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:12:43 crc kubenswrapper[5052]: I1203 12:12:43.664078 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0fd9a636-d4a2-45f7-8a00-0b3a2fe364cb-kube-api-access-jc5mm" (OuterVolumeSpecName: "kube-api-access-jc5mm") pod "0fd9a636-d4a2-45f7-8a00-0b3a2fe364cb" (UID: "0fd9a636-d4a2-45f7-8a00-0b3a2fe364cb"). InnerVolumeSpecName "kube-api-access-jc5mm". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:12:43 crc kubenswrapper[5052]: I1203 12:12:43.667595 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage20-crc" (OuterVolumeSpecName: "glance") pod "0fd9a636-d4a2-45f7-8a00-0b3a2fe364cb" (UID: "0fd9a636-d4a2-45f7-8a00-0b3a2fe364cb"). InnerVolumeSpecName "local-storage20-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 12:12:43 crc kubenswrapper[5052]: I1203 12:12:43.698648 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0fd9a636-d4a2-45f7-8a00-0b3a2fe364cb-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "0fd9a636-d4a2-45f7-8a00-0b3a2fe364cb" (UID: "0fd9a636-d4a2-45f7-8a00-0b3a2fe364cb"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:12:43 crc kubenswrapper[5052]: I1203 12:12:43.719500 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0fd9a636-d4a2-45f7-8a00-0b3a2fe364cb-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "0fd9a636-d4a2-45f7-8a00-0b3a2fe364cb" (UID: "0fd9a636-d4a2-45f7-8a00-0b3a2fe364cb"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:12:43 crc kubenswrapper[5052]: I1203 12:12:43.729298 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0fd9a636-d4a2-45f7-8a00-0b3a2fe364cb-config-data" (OuterVolumeSpecName: "config-data") pod "0fd9a636-d4a2-45f7-8a00-0b3a2fe364cb" (UID: "0fd9a636-d4a2-45f7-8a00-0b3a2fe364cb"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:12:43 crc kubenswrapper[5052]: I1203 12:12:43.763627 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-jc5mm\" (UniqueName: \"kubernetes.io/projected/0fd9a636-d4a2-45f7-8a00-0b3a2fe364cb-kube-api-access-jc5mm\") on node \"crc\" DevicePath \"\"" Dec 03 12:12:43 crc kubenswrapper[5052]: I1203 12:12:43.763665 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/0fd9a636-d4a2-45f7-8a00-0b3a2fe364cb-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:12:43 crc kubenswrapper[5052]: I1203 12:12:43.763679 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0fd9a636-d4a2-45f7-8a00-0b3a2fe364cb-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:12:43 crc kubenswrapper[5052]: I1203 12:12:43.763693 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/0fd9a636-d4a2-45f7-8a00-0b3a2fe364cb-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:12:43 crc kubenswrapper[5052]: I1203 12:12:43.763705 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/0fd9a636-d4a2-45f7-8a00-0b3a2fe364cb-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:12:43 crc kubenswrapper[5052]: I1203 12:12:43.763731 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage20-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage20-crc\") on node \"crc\" " Dec 03 12:12:43 crc kubenswrapper[5052]: I1203 12:12:43.763742 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/0fd9a636-d4a2-45f7-8a00-0b3a2fe364cb-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:12:43 crc kubenswrapper[5052]: I1203 12:12:43.785817 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage20-crc" (UniqueName: "kubernetes.io/local-volume/local-storage20-crc") on node "crc" Dec 03 12:12:43 crc kubenswrapper[5052]: I1203 12:12:43.865341 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage20-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage20-crc\") on node \"crc\" DevicePath \"\"" Dec 03 12:12:43 crc kubenswrapper[5052]: I1203 12:12:43.987533 5052 generic.go:334] "Generic (PLEG): container finished" podID="0fd9a636-d4a2-45f7-8a00-0b3a2fe364cb" containerID="3980c92ec54354398e968f8f5cd2dd8d079b22bf5a7562059de4e49cc45e8c2b" exitCode=0 Dec 03 12:12:43 crc kubenswrapper[5052]: I1203 12:12:43.987599 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:12:43 crc kubenswrapper[5052]: I1203 12:12:43.987605 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"0fd9a636-d4a2-45f7-8a00-0b3a2fe364cb","Type":"ContainerDied","Data":"3980c92ec54354398e968f8f5cd2dd8d079b22bf5a7562059de4e49cc45e8c2b"} Dec 03 12:12:43 crc kubenswrapper[5052]: I1203 12:12:43.987640 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"0fd9a636-d4a2-45f7-8a00-0b3a2fe364cb","Type":"ContainerDied","Data":"7fe5008dd6a2fa7e7e752c99afa8f2187cc6b0accf3b245645b7274fe1ca2d1e"} Dec 03 12:12:43 crc kubenswrapper[5052]: I1203 12:12:43.988278 5052 scope.go:117] "RemoveContainer" containerID="3980c92ec54354398e968f8f5cd2dd8d079b22bf5a7562059de4e49cc45e8c2b" Dec 03 12:12:43 crc kubenswrapper[5052]: I1203 12:12:43.990885 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"f3339fc5-9d8a-4a19-ba83-b5f02f9d86d3","Type":"ContainerStarted","Data":"a128294473b2ae65eb81c532b3040ade4790b89f3cfa21e3d64a4929843efc65"} Dec 03 12:12:43 crc kubenswrapper[5052]: I1203 12:12:43.990949 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"f3339fc5-9d8a-4a19-ba83-b5f02f9d86d3","Type":"ContainerStarted","Data":"aa0bc6f7ead0eddf2779c1b4d8c7ab91d106b7722a51650dfdf4c53883d3f5aa"} Dec 03 12:12:44 crc kubenswrapper[5052]: I1203 12:12:44.119622 5052 scope.go:117] "RemoveContainer" containerID="0c0725eafb777c0b6d0ad246f980467eb5fda2912f89f7a5b4867df0ec771fce" Dec 03 12:12:44 crc kubenswrapper[5052]: I1203 12:12:44.120991 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:12:44 crc kubenswrapper[5052]: I1203 12:12:44.157240 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:12:44 crc kubenswrapper[5052]: I1203 12:12:44.173734 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:12:44 crc kubenswrapper[5052]: E1203 12:12:44.174277 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="0fd9a636-d4a2-45f7-8a00-0b3a2fe364cb" containerName="glance-log" Dec 03 12:12:44 crc kubenswrapper[5052]: I1203 12:12:44.174296 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="0fd9a636-d4a2-45f7-8a00-0b3a2fe364cb" containerName="glance-log" Dec 03 12:12:44 crc kubenswrapper[5052]: E1203 12:12:44.174312 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="0fd9a636-d4a2-45f7-8a00-0b3a2fe364cb" containerName="glance-httpd" Dec 03 12:12:44 crc kubenswrapper[5052]: I1203 12:12:44.174319 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="0fd9a636-d4a2-45f7-8a00-0b3a2fe364cb" containerName="glance-httpd" Dec 03 12:12:44 crc kubenswrapper[5052]: I1203 12:12:44.174633 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="0fd9a636-d4a2-45f7-8a00-0b3a2fe364cb" containerName="glance-log" Dec 03 12:12:44 crc kubenswrapper[5052]: I1203 12:12:44.174780 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="0fd9a636-d4a2-45f7-8a00-0b3a2fe364cb" containerName="glance-httpd" Dec 03 12:12:44 crc kubenswrapper[5052]: I1203 12:12:44.177976 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:12:44 crc kubenswrapper[5052]: I1203 12:12:44.179019 5052 scope.go:117] "RemoveContainer" containerID="3980c92ec54354398e968f8f5cd2dd8d079b22bf5a7562059de4e49cc45e8c2b" Dec 03 12:12:44 crc kubenswrapper[5052]: E1203 12:12:44.179482 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3980c92ec54354398e968f8f5cd2dd8d079b22bf5a7562059de4e49cc45e8c2b\": container with ID starting with 3980c92ec54354398e968f8f5cd2dd8d079b22bf5a7562059de4e49cc45e8c2b not found: ID does not exist" containerID="3980c92ec54354398e968f8f5cd2dd8d079b22bf5a7562059de4e49cc45e8c2b" Dec 03 12:12:44 crc kubenswrapper[5052]: I1203 12:12:44.179555 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3980c92ec54354398e968f8f5cd2dd8d079b22bf5a7562059de4e49cc45e8c2b"} err="failed to get container status \"3980c92ec54354398e968f8f5cd2dd8d079b22bf5a7562059de4e49cc45e8c2b\": rpc error: code = NotFound desc = could not find container \"3980c92ec54354398e968f8f5cd2dd8d079b22bf5a7562059de4e49cc45e8c2b\": container with ID starting with 3980c92ec54354398e968f8f5cd2dd8d079b22bf5a7562059de4e49cc45e8c2b not found: ID does not exist" Dec 03 12:12:44 crc kubenswrapper[5052]: I1203 12:12:44.179588 5052 scope.go:117] "RemoveContainer" containerID="0c0725eafb777c0b6d0ad246f980467eb5fda2912f89f7a5b4867df0ec771fce" Dec 03 12:12:44 crc kubenswrapper[5052]: E1203 12:12:44.180929 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0c0725eafb777c0b6d0ad246f980467eb5fda2912f89f7a5b4867df0ec771fce\": container with ID starting with 0c0725eafb777c0b6d0ad246f980467eb5fda2912f89f7a5b4867df0ec771fce not found: ID does not exist" containerID="0c0725eafb777c0b6d0ad246f980467eb5fda2912f89f7a5b4867df0ec771fce" Dec 03 12:12:44 crc kubenswrapper[5052]: I1203 12:12:44.180965 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0c0725eafb777c0b6d0ad246f980467eb5fda2912f89f7a5b4867df0ec771fce"} err="failed to get container status \"0c0725eafb777c0b6d0ad246f980467eb5fda2912f89f7a5b4867df0ec771fce\": rpc error: code = NotFound desc = could not find container \"0c0725eafb777c0b6d0ad246f980467eb5fda2912f89f7a5b4867df0ec771fce\": container with ID starting with 0c0725eafb777c0b6d0ad246f980467eb5fda2912f89f7a5b4867df0ec771fce not found: ID does not exist" Dec 03 12:12:44 crc kubenswrapper[5052]: I1203 12:12:44.186187 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-default-external-config-data" Dec 03 12:12:44 crc kubenswrapper[5052]: I1203 12:12:44.186370 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-glance-default-public-svc" Dec 03 12:12:44 crc kubenswrapper[5052]: I1203 12:12:44.190399 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:12:44 crc kubenswrapper[5052]: I1203 12:12:44.275597 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/2a349843-a7b7-4dff-a5ae-f513337c5809-httpd-run\") pod \"glance-default-external-api-0\" (UID: \"2a349843-a7b7-4dff-a5ae-f513337c5809\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:12:44 crc kubenswrapper[5052]: I1203 12:12:44.275648 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rzrj5\" (UniqueName: \"kubernetes.io/projected/2a349843-a7b7-4dff-a5ae-f513337c5809-kube-api-access-rzrj5\") pod \"glance-default-external-api-0\" (UID: \"2a349843-a7b7-4dff-a5ae-f513337c5809\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:12:44 crc kubenswrapper[5052]: I1203 12:12:44.275713 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2a349843-a7b7-4dff-a5ae-f513337c5809-combined-ca-bundle\") pod \"glance-default-external-api-0\" (UID: \"2a349843-a7b7-4dff-a5ae-f513337c5809\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:12:44 crc kubenswrapper[5052]: I1203 12:12:44.275746 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage20-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage20-crc\") pod \"glance-default-external-api-0\" (UID: \"2a349843-a7b7-4dff-a5ae-f513337c5809\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:12:44 crc kubenswrapper[5052]: I1203 12:12:44.275824 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/2a349843-a7b7-4dff-a5ae-f513337c5809-public-tls-certs\") pod \"glance-default-external-api-0\" (UID: \"2a349843-a7b7-4dff-a5ae-f513337c5809\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:12:44 crc kubenswrapper[5052]: I1203 12:12:44.275876 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/2a349843-a7b7-4dff-a5ae-f513337c5809-config-data\") pod \"glance-default-external-api-0\" (UID: \"2a349843-a7b7-4dff-a5ae-f513337c5809\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:12:44 crc kubenswrapper[5052]: I1203 12:12:44.275926 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/2a349843-a7b7-4dff-a5ae-f513337c5809-logs\") pod \"glance-default-external-api-0\" (UID: \"2a349843-a7b7-4dff-a5ae-f513337c5809\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:12:44 crc kubenswrapper[5052]: I1203 12:12:44.275960 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/2a349843-a7b7-4dff-a5ae-f513337c5809-scripts\") pod \"glance-default-external-api-0\" (UID: \"2a349843-a7b7-4dff-a5ae-f513337c5809\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:12:44 crc kubenswrapper[5052]: I1203 12:12:44.378076 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/2a349843-a7b7-4dff-a5ae-f513337c5809-public-tls-certs\") pod \"glance-default-external-api-0\" (UID: \"2a349843-a7b7-4dff-a5ae-f513337c5809\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:12:44 crc kubenswrapper[5052]: I1203 12:12:44.378141 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/2a349843-a7b7-4dff-a5ae-f513337c5809-config-data\") pod \"glance-default-external-api-0\" (UID: \"2a349843-a7b7-4dff-a5ae-f513337c5809\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:12:44 crc kubenswrapper[5052]: I1203 12:12:44.378205 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/2a349843-a7b7-4dff-a5ae-f513337c5809-logs\") pod \"glance-default-external-api-0\" (UID: \"2a349843-a7b7-4dff-a5ae-f513337c5809\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:12:44 crc kubenswrapper[5052]: I1203 12:12:44.378247 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/2a349843-a7b7-4dff-a5ae-f513337c5809-scripts\") pod \"glance-default-external-api-0\" (UID: \"2a349843-a7b7-4dff-a5ae-f513337c5809\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:12:44 crc kubenswrapper[5052]: I1203 12:12:44.378305 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/2a349843-a7b7-4dff-a5ae-f513337c5809-httpd-run\") pod \"glance-default-external-api-0\" (UID: \"2a349843-a7b7-4dff-a5ae-f513337c5809\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:12:44 crc kubenswrapper[5052]: I1203 12:12:44.378332 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-rzrj5\" (UniqueName: \"kubernetes.io/projected/2a349843-a7b7-4dff-a5ae-f513337c5809-kube-api-access-rzrj5\") pod \"glance-default-external-api-0\" (UID: \"2a349843-a7b7-4dff-a5ae-f513337c5809\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:12:44 crc kubenswrapper[5052]: I1203 12:12:44.378397 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2a349843-a7b7-4dff-a5ae-f513337c5809-combined-ca-bundle\") pod \"glance-default-external-api-0\" (UID: \"2a349843-a7b7-4dff-a5ae-f513337c5809\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:12:44 crc kubenswrapper[5052]: I1203 12:12:44.378434 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage20-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage20-crc\") pod \"glance-default-external-api-0\" (UID: \"2a349843-a7b7-4dff-a5ae-f513337c5809\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:12:44 crc kubenswrapper[5052]: I1203 12:12:44.379467 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage20-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage20-crc\") pod \"glance-default-external-api-0\" (UID: \"2a349843-a7b7-4dff-a5ae-f513337c5809\") device mount path \"/mnt/openstack/pv20\"" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:12:44 crc kubenswrapper[5052]: I1203 12:12:44.381418 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/2a349843-a7b7-4dff-a5ae-f513337c5809-logs\") pod \"glance-default-external-api-0\" (UID: \"2a349843-a7b7-4dff-a5ae-f513337c5809\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:12:44 crc kubenswrapper[5052]: I1203 12:12:44.381446 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/2a349843-a7b7-4dff-a5ae-f513337c5809-httpd-run\") pod \"glance-default-external-api-0\" (UID: \"2a349843-a7b7-4dff-a5ae-f513337c5809\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:12:44 crc kubenswrapper[5052]: I1203 12:12:44.387088 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/2a349843-a7b7-4dff-a5ae-f513337c5809-scripts\") pod \"glance-default-external-api-0\" (UID: \"2a349843-a7b7-4dff-a5ae-f513337c5809\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:12:44 crc kubenswrapper[5052]: I1203 12:12:44.387898 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2a349843-a7b7-4dff-a5ae-f513337c5809-combined-ca-bundle\") pod \"glance-default-external-api-0\" (UID: \"2a349843-a7b7-4dff-a5ae-f513337c5809\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:12:44 crc kubenswrapper[5052]: I1203 12:12:44.388152 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/2a349843-a7b7-4dff-a5ae-f513337c5809-public-tls-certs\") pod \"glance-default-external-api-0\" (UID: \"2a349843-a7b7-4dff-a5ae-f513337c5809\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:12:44 crc kubenswrapper[5052]: I1203 12:12:44.390167 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/2a349843-a7b7-4dff-a5ae-f513337c5809-config-data\") pod \"glance-default-external-api-0\" (UID: \"2a349843-a7b7-4dff-a5ae-f513337c5809\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:12:44 crc kubenswrapper[5052]: I1203 12:12:44.400630 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-rzrj5\" (UniqueName: \"kubernetes.io/projected/2a349843-a7b7-4dff-a5ae-f513337c5809-kube-api-access-rzrj5\") pod \"glance-default-external-api-0\" (UID: \"2a349843-a7b7-4dff-a5ae-f513337c5809\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:12:44 crc kubenswrapper[5052]: I1203 12:12:44.411694 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage20-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage20-crc\") pod \"glance-default-external-api-0\" (UID: \"2a349843-a7b7-4dff-a5ae-f513337c5809\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:12:44 crc kubenswrapper[5052]: I1203 12:12:44.507543 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:12:44 crc kubenswrapper[5052]: I1203 12:12:44.507818 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:12:44 crc kubenswrapper[5052]: I1203 12:12:44.581433 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/03919b22-bf40-4bf2-88cb-fdefc426072c-config-data\") pod \"03919b22-bf40-4bf2-88cb-fdefc426072c\" (UID: \"03919b22-bf40-4bf2-88cb-fdefc426072c\") " Dec 03 12:12:44 crc kubenswrapper[5052]: I1203 12:12:44.581796 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/03919b22-bf40-4bf2-88cb-fdefc426072c-internal-tls-certs\") pod \"03919b22-bf40-4bf2-88cb-fdefc426072c\" (UID: \"03919b22-bf40-4bf2-88cb-fdefc426072c\") " Dec 03 12:12:44 crc kubenswrapper[5052]: I1203 12:12:44.581908 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/03919b22-bf40-4bf2-88cb-fdefc426072c-httpd-run\") pod \"03919b22-bf40-4bf2-88cb-fdefc426072c\" (UID: \"03919b22-bf40-4bf2-88cb-fdefc426072c\") " Dec 03 12:12:44 crc kubenswrapper[5052]: I1203 12:12:44.581953 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2jqgq\" (UniqueName: \"kubernetes.io/projected/03919b22-bf40-4bf2-88cb-fdefc426072c-kube-api-access-2jqgq\") pod \"03919b22-bf40-4bf2-88cb-fdefc426072c\" (UID: \"03919b22-bf40-4bf2-88cb-fdefc426072c\") " Dec 03 12:12:44 crc kubenswrapper[5052]: I1203 12:12:44.582005 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"glance\" (UniqueName: \"kubernetes.io/local-volume/local-storage03-crc\") pod \"03919b22-bf40-4bf2-88cb-fdefc426072c\" (UID: \"03919b22-bf40-4bf2-88cb-fdefc426072c\") " Dec 03 12:12:44 crc kubenswrapper[5052]: I1203 12:12:44.582033 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/03919b22-bf40-4bf2-88cb-fdefc426072c-combined-ca-bundle\") pod \"03919b22-bf40-4bf2-88cb-fdefc426072c\" (UID: \"03919b22-bf40-4bf2-88cb-fdefc426072c\") " Dec 03 12:12:44 crc kubenswrapper[5052]: I1203 12:12:44.582074 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/03919b22-bf40-4bf2-88cb-fdefc426072c-logs\") pod \"03919b22-bf40-4bf2-88cb-fdefc426072c\" (UID: \"03919b22-bf40-4bf2-88cb-fdefc426072c\") " Dec 03 12:12:44 crc kubenswrapper[5052]: I1203 12:12:44.582104 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/03919b22-bf40-4bf2-88cb-fdefc426072c-scripts\") pod \"03919b22-bf40-4bf2-88cb-fdefc426072c\" (UID: \"03919b22-bf40-4bf2-88cb-fdefc426072c\") " Dec 03 12:12:44 crc kubenswrapper[5052]: I1203 12:12:44.584852 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/03919b22-bf40-4bf2-88cb-fdefc426072c-logs" (OuterVolumeSpecName: "logs") pod "03919b22-bf40-4bf2-88cb-fdefc426072c" (UID: "03919b22-bf40-4bf2-88cb-fdefc426072c"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:12:44 crc kubenswrapper[5052]: I1203 12:12:44.586908 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/03919b22-bf40-4bf2-88cb-fdefc426072c-httpd-run" (OuterVolumeSpecName: "httpd-run") pod "03919b22-bf40-4bf2-88cb-fdefc426072c" (UID: "03919b22-bf40-4bf2-88cb-fdefc426072c"). InnerVolumeSpecName "httpd-run". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:12:44 crc kubenswrapper[5052]: I1203 12:12:44.594617 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/03919b22-bf40-4bf2-88cb-fdefc426072c-kube-api-access-2jqgq" (OuterVolumeSpecName: "kube-api-access-2jqgq") pod "03919b22-bf40-4bf2-88cb-fdefc426072c" (UID: "03919b22-bf40-4bf2-88cb-fdefc426072c"). InnerVolumeSpecName "kube-api-access-2jqgq". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:12:44 crc kubenswrapper[5052]: I1203 12:12:44.597168 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/03919b22-bf40-4bf2-88cb-fdefc426072c-scripts" (OuterVolumeSpecName: "scripts") pod "03919b22-bf40-4bf2-88cb-fdefc426072c" (UID: "03919b22-bf40-4bf2-88cb-fdefc426072c"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:12:44 crc kubenswrapper[5052]: I1203 12:12:44.606644 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage03-crc" (OuterVolumeSpecName: "glance") pod "03919b22-bf40-4bf2-88cb-fdefc426072c" (UID: "03919b22-bf40-4bf2-88cb-fdefc426072c"). InnerVolumeSpecName "local-storage03-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 12:12:44 crc kubenswrapper[5052]: I1203 12:12:44.645609 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/03919b22-bf40-4bf2-88cb-fdefc426072c-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "03919b22-bf40-4bf2-88cb-fdefc426072c" (UID: "03919b22-bf40-4bf2-88cb-fdefc426072c"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:12:44 crc kubenswrapper[5052]: I1203 12:12:44.649395 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/03919b22-bf40-4bf2-88cb-fdefc426072c-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "03919b22-bf40-4bf2-88cb-fdefc426072c" (UID: "03919b22-bf40-4bf2-88cb-fdefc426072c"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:12:44 crc kubenswrapper[5052]: I1203 12:12:44.691566 5052 reconciler_common.go:293] "Volume detached for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/03919b22-bf40-4bf2-88cb-fdefc426072c-httpd-run\") on node \"crc\" DevicePath \"\"" Dec 03 12:12:44 crc kubenswrapper[5052]: I1203 12:12:44.699798 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-2jqgq\" (UniqueName: \"kubernetes.io/projected/03919b22-bf40-4bf2-88cb-fdefc426072c-kube-api-access-2jqgq\") on node \"crc\" DevicePath \"\"" Dec 03 12:12:44 crc kubenswrapper[5052]: I1203 12:12:44.699880 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage03-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage03-crc\") on node \"crc\" " Dec 03 12:12:44 crc kubenswrapper[5052]: I1203 12:12:44.699896 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/03919b22-bf40-4bf2-88cb-fdefc426072c-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:12:44 crc kubenswrapper[5052]: I1203 12:12:44.699910 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/03919b22-bf40-4bf2-88cb-fdefc426072c-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:12:44 crc kubenswrapper[5052]: I1203 12:12:44.699925 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/03919b22-bf40-4bf2-88cb-fdefc426072c-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:12:44 crc kubenswrapper[5052]: I1203 12:12:44.699937 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/03919b22-bf40-4bf2-88cb-fdefc426072c-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:12:44 crc kubenswrapper[5052]: I1203 12:12:44.709265 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/03919b22-bf40-4bf2-88cb-fdefc426072c-config-data" (OuterVolumeSpecName: "config-data") pod "03919b22-bf40-4bf2-88cb-fdefc426072c" (UID: "03919b22-bf40-4bf2-88cb-fdefc426072c"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:12:44 crc kubenswrapper[5052]: I1203 12:12:44.813899 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/03919b22-bf40-4bf2-88cb-fdefc426072c-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:12:44 crc kubenswrapper[5052]: I1203 12:12:44.863247 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage03-crc" (UniqueName: "kubernetes.io/local-volume/local-storage03-crc") on node "crc" Dec 03 12:12:44 crc kubenswrapper[5052]: I1203 12:12:44.917185 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage03-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage03-crc\") on node \"crc\" DevicePath \"\"" Dec 03 12:12:45 crc kubenswrapper[5052]: I1203 12:12:45.012268 5052 generic.go:334] "Generic (PLEG): container finished" podID="03919b22-bf40-4bf2-88cb-fdefc426072c" containerID="73d8869b154d3281892503903ffd7422325318d2bf903924c2df74fb075f4f4e" exitCode=0 Dec 03 12:12:45 crc kubenswrapper[5052]: I1203 12:12:45.012348 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"03919b22-bf40-4bf2-88cb-fdefc426072c","Type":"ContainerDied","Data":"73d8869b154d3281892503903ffd7422325318d2bf903924c2df74fb075f4f4e"} Dec 03 12:12:45 crc kubenswrapper[5052]: I1203 12:12:45.012381 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"03919b22-bf40-4bf2-88cb-fdefc426072c","Type":"ContainerDied","Data":"656d7c09c157877e0d721311854348a66ad7554bb1d2e175c1191e79657ee3ba"} Dec 03 12:12:45 crc kubenswrapper[5052]: I1203 12:12:45.012402 5052 scope.go:117] "RemoveContainer" containerID="73d8869b154d3281892503903ffd7422325318d2bf903924c2df74fb075f4f4e" Dec 03 12:12:45 crc kubenswrapper[5052]: I1203 12:12:45.012626 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:12:45 crc kubenswrapper[5052]: I1203 12:12:45.056680 5052 scope.go:117] "RemoveContainer" containerID="622d66b2983e1d4bdc07c013e8cf0e59732996d900fcb033de628153df313bb1" Dec 03 12:12:45 crc kubenswrapper[5052]: I1203 12:12:45.074562 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:12:45 crc kubenswrapper[5052]: I1203 12:12:45.095147 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:12:45 crc kubenswrapper[5052]: I1203 12:12:45.115650 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:12:45 crc kubenswrapper[5052]: I1203 12:12:45.128124 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:12:45 crc kubenswrapper[5052]: E1203 12:12:45.134921 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="03919b22-bf40-4bf2-88cb-fdefc426072c" containerName="glance-httpd" Dec 03 12:12:45 crc kubenswrapper[5052]: I1203 12:12:45.134962 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="03919b22-bf40-4bf2-88cb-fdefc426072c" containerName="glance-httpd" Dec 03 12:12:45 crc kubenswrapper[5052]: E1203 12:12:45.134981 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="03919b22-bf40-4bf2-88cb-fdefc426072c" containerName="glance-log" Dec 03 12:12:45 crc kubenswrapper[5052]: I1203 12:12:45.134988 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="03919b22-bf40-4bf2-88cb-fdefc426072c" containerName="glance-log" Dec 03 12:12:45 crc kubenswrapper[5052]: I1203 12:12:45.135208 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="03919b22-bf40-4bf2-88cb-fdefc426072c" containerName="glance-httpd" Dec 03 12:12:45 crc kubenswrapper[5052]: I1203 12:12:45.135221 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="03919b22-bf40-4bf2-88cb-fdefc426072c" containerName="glance-log" Dec 03 12:12:45 crc kubenswrapper[5052]: I1203 12:12:45.136360 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:12:45 crc kubenswrapper[5052]: I1203 12:12:45.143715 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-default-internal-config-data" Dec 03 12:12:45 crc kubenswrapper[5052]: I1203 12:12:45.143917 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-glance-default-internal-svc" Dec 03 12:12:45 crc kubenswrapper[5052]: I1203 12:12:45.154574 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:12:45 crc kubenswrapper[5052]: I1203 12:12:45.168496 5052 scope.go:117] "RemoveContainer" containerID="73d8869b154d3281892503903ffd7422325318d2bf903924c2df74fb075f4f4e" Dec 03 12:12:45 crc kubenswrapper[5052]: E1203 12:12:45.174856 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"73d8869b154d3281892503903ffd7422325318d2bf903924c2df74fb075f4f4e\": container with ID starting with 73d8869b154d3281892503903ffd7422325318d2bf903924c2df74fb075f4f4e not found: ID does not exist" containerID="73d8869b154d3281892503903ffd7422325318d2bf903924c2df74fb075f4f4e" Dec 03 12:12:45 crc kubenswrapper[5052]: I1203 12:12:45.175084 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"73d8869b154d3281892503903ffd7422325318d2bf903924c2df74fb075f4f4e"} err="failed to get container status \"73d8869b154d3281892503903ffd7422325318d2bf903924c2df74fb075f4f4e\": rpc error: code = NotFound desc = could not find container \"73d8869b154d3281892503903ffd7422325318d2bf903924c2df74fb075f4f4e\": container with ID starting with 73d8869b154d3281892503903ffd7422325318d2bf903924c2df74fb075f4f4e not found: ID does not exist" Dec 03 12:12:45 crc kubenswrapper[5052]: I1203 12:12:45.175190 5052 scope.go:117] "RemoveContainer" containerID="622d66b2983e1d4bdc07c013e8cf0e59732996d900fcb033de628153df313bb1" Dec 03 12:12:45 crc kubenswrapper[5052]: E1203 12:12:45.176205 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"622d66b2983e1d4bdc07c013e8cf0e59732996d900fcb033de628153df313bb1\": container with ID starting with 622d66b2983e1d4bdc07c013e8cf0e59732996d900fcb033de628153df313bb1 not found: ID does not exist" containerID="622d66b2983e1d4bdc07c013e8cf0e59732996d900fcb033de628153df313bb1" Dec 03 12:12:45 crc kubenswrapper[5052]: I1203 12:12:45.176252 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"622d66b2983e1d4bdc07c013e8cf0e59732996d900fcb033de628153df313bb1"} err="failed to get container status \"622d66b2983e1d4bdc07c013e8cf0e59732996d900fcb033de628153df313bb1\": rpc error: code = NotFound desc = could not find container \"622d66b2983e1d4bdc07c013e8cf0e59732996d900fcb033de628153df313bb1\": container with ID starting with 622d66b2983e1d4bdc07c013e8cf0e59732996d900fcb033de628153df313bb1 not found: ID does not exist" Dec 03 12:12:45 crc kubenswrapper[5052]: I1203 12:12:45.223183 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d81b09dc-75a1-45f8-ace4-786531ad3bfd-combined-ca-bundle\") pod \"glance-default-internal-api-0\" (UID: \"d81b09dc-75a1-45f8-ace4-786531ad3bfd\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:12:45 crc kubenswrapper[5052]: I1203 12:12:45.223229 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/d81b09dc-75a1-45f8-ace4-786531ad3bfd-logs\") pod \"glance-default-internal-api-0\" (UID: \"d81b09dc-75a1-45f8-ace4-786531ad3bfd\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:12:45 crc kubenswrapper[5052]: I1203 12:12:45.223279 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/d81b09dc-75a1-45f8-ace4-786531ad3bfd-internal-tls-certs\") pod \"glance-default-internal-api-0\" (UID: \"d81b09dc-75a1-45f8-ace4-786531ad3bfd\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:12:45 crc kubenswrapper[5052]: I1203 12:12:45.223304 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/d81b09dc-75a1-45f8-ace4-786531ad3bfd-httpd-run\") pod \"glance-default-internal-api-0\" (UID: \"d81b09dc-75a1-45f8-ace4-786531ad3bfd\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:12:45 crc kubenswrapper[5052]: I1203 12:12:45.223340 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kng9h\" (UniqueName: \"kubernetes.io/projected/d81b09dc-75a1-45f8-ace4-786531ad3bfd-kube-api-access-kng9h\") pod \"glance-default-internal-api-0\" (UID: \"d81b09dc-75a1-45f8-ace4-786531ad3bfd\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:12:45 crc kubenswrapper[5052]: I1203 12:12:45.223371 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/d81b09dc-75a1-45f8-ace4-786531ad3bfd-config-data\") pod \"glance-default-internal-api-0\" (UID: \"d81b09dc-75a1-45f8-ace4-786531ad3bfd\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:12:45 crc kubenswrapper[5052]: I1203 12:12:45.223402 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage03-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage03-crc\") pod \"glance-default-internal-api-0\" (UID: \"d81b09dc-75a1-45f8-ace4-786531ad3bfd\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:12:45 crc kubenswrapper[5052]: I1203 12:12:45.223418 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/d81b09dc-75a1-45f8-ace4-786531ad3bfd-scripts\") pod \"glance-default-internal-api-0\" (UID: \"d81b09dc-75a1-45f8-ace4-786531ad3bfd\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:12:45 crc kubenswrapper[5052]: I1203 12:12:45.324572 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-kng9h\" (UniqueName: \"kubernetes.io/projected/d81b09dc-75a1-45f8-ace4-786531ad3bfd-kube-api-access-kng9h\") pod \"glance-default-internal-api-0\" (UID: \"d81b09dc-75a1-45f8-ace4-786531ad3bfd\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:12:45 crc kubenswrapper[5052]: I1203 12:12:45.324651 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/d81b09dc-75a1-45f8-ace4-786531ad3bfd-config-data\") pod \"glance-default-internal-api-0\" (UID: \"d81b09dc-75a1-45f8-ace4-786531ad3bfd\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:12:45 crc kubenswrapper[5052]: I1203 12:12:45.324702 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage03-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage03-crc\") pod \"glance-default-internal-api-0\" (UID: \"d81b09dc-75a1-45f8-ace4-786531ad3bfd\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:12:45 crc kubenswrapper[5052]: I1203 12:12:45.324726 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/d81b09dc-75a1-45f8-ace4-786531ad3bfd-scripts\") pod \"glance-default-internal-api-0\" (UID: \"d81b09dc-75a1-45f8-ace4-786531ad3bfd\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:12:45 crc kubenswrapper[5052]: I1203 12:12:45.324819 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d81b09dc-75a1-45f8-ace4-786531ad3bfd-combined-ca-bundle\") pod \"glance-default-internal-api-0\" (UID: \"d81b09dc-75a1-45f8-ace4-786531ad3bfd\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:12:45 crc kubenswrapper[5052]: I1203 12:12:45.324848 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/d81b09dc-75a1-45f8-ace4-786531ad3bfd-logs\") pod \"glance-default-internal-api-0\" (UID: \"d81b09dc-75a1-45f8-ace4-786531ad3bfd\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:12:45 crc kubenswrapper[5052]: I1203 12:12:45.324904 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/d81b09dc-75a1-45f8-ace4-786531ad3bfd-internal-tls-certs\") pod \"glance-default-internal-api-0\" (UID: \"d81b09dc-75a1-45f8-ace4-786531ad3bfd\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:12:45 crc kubenswrapper[5052]: I1203 12:12:45.324936 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/d81b09dc-75a1-45f8-ace4-786531ad3bfd-httpd-run\") pod \"glance-default-internal-api-0\" (UID: \"d81b09dc-75a1-45f8-ace4-786531ad3bfd\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:12:45 crc kubenswrapper[5052]: I1203 12:12:45.325140 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage03-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage03-crc\") pod \"glance-default-internal-api-0\" (UID: \"d81b09dc-75a1-45f8-ace4-786531ad3bfd\") device mount path \"/mnt/openstack/pv03\"" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:12:45 crc kubenswrapper[5052]: I1203 12:12:45.325878 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/d81b09dc-75a1-45f8-ace4-786531ad3bfd-httpd-run\") pod \"glance-default-internal-api-0\" (UID: \"d81b09dc-75a1-45f8-ace4-786531ad3bfd\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:12:45 crc kubenswrapper[5052]: I1203 12:12:45.325958 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/d81b09dc-75a1-45f8-ace4-786531ad3bfd-logs\") pod \"glance-default-internal-api-0\" (UID: \"d81b09dc-75a1-45f8-ace4-786531ad3bfd\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:12:45 crc kubenswrapper[5052]: I1203 12:12:45.332912 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/d81b09dc-75a1-45f8-ace4-786531ad3bfd-scripts\") pod \"glance-default-internal-api-0\" (UID: \"d81b09dc-75a1-45f8-ace4-786531ad3bfd\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:12:45 crc kubenswrapper[5052]: I1203 12:12:45.335896 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/d81b09dc-75a1-45f8-ace4-786531ad3bfd-config-data\") pod \"glance-default-internal-api-0\" (UID: \"d81b09dc-75a1-45f8-ace4-786531ad3bfd\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:12:45 crc kubenswrapper[5052]: I1203 12:12:45.336119 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d81b09dc-75a1-45f8-ace4-786531ad3bfd-combined-ca-bundle\") pod \"glance-default-internal-api-0\" (UID: \"d81b09dc-75a1-45f8-ace4-786531ad3bfd\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:12:45 crc kubenswrapper[5052]: I1203 12:12:45.339024 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/d81b09dc-75a1-45f8-ace4-786531ad3bfd-internal-tls-certs\") pod \"glance-default-internal-api-0\" (UID: \"d81b09dc-75a1-45f8-ace4-786531ad3bfd\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:12:45 crc kubenswrapper[5052]: I1203 12:12:45.346502 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-kng9h\" (UniqueName: \"kubernetes.io/projected/d81b09dc-75a1-45f8-ace4-786531ad3bfd-kube-api-access-kng9h\") pod \"glance-default-internal-api-0\" (UID: \"d81b09dc-75a1-45f8-ace4-786531ad3bfd\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:12:45 crc kubenswrapper[5052]: I1203 12:12:45.368220 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage03-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage03-crc\") pod \"glance-default-internal-api-0\" (UID: \"d81b09dc-75a1-45f8-ace4-786531ad3bfd\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:12:45 crc kubenswrapper[5052]: I1203 12:12:45.480295 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:12:45 crc kubenswrapper[5052]: I1203 12:12:45.696865 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="03919b22-bf40-4bf2-88cb-fdefc426072c" path="/var/lib/kubelet/pods/03919b22-bf40-4bf2-88cb-fdefc426072c/volumes" Dec 03 12:12:45 crc kubenswrapper[5052]: I1203 12:12:45.697904 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0fd9a636-d4a2-45f7-8a00-0b3a2fe364cb" path="/var/lib/kubelet/pods/0fd9a636-d4a2-45f7-8a00-0b3a2fe364cb/volumes" Dec 03 12:12:45 crc kubenswrapper[5052]: W1203 12:12:45.980263 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd81b09dc_75a1_45f8_ace4_786531ad3bfd.slice/crio-3563429b0947d41b3ec949fe5606dcbd5f966d6dc0206580a74f817c192e1e64 WatchSource:0}: Error finding container 3563429b0947d41b3ec949fe5606dcbd5f966d6dc0206580a74f817c192e1e64: Status 404 returned error can't find the container with id 3563429b0947d41b3ec949fe5606dcbd5f966d6dc0206580a74f817c192e1e64 Dec 03 12:12:45 crc kubenswrapper[5052]: I1203 12:12:45.988199 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:12:46 crc kubenswrapper[5052]: I1203 12:12:46.038134 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"d81b09dc-75a1-45f8-ace4-786531ad3bfd","Type":"ContainerStarted","Data":"3563429b0947d41b3ec949fe5606dcbd5f966d6dc0206580a74f817c192e1e64"} Dec 03 12:12:46 crc kubenswrapper[5052]: I1203 12:12:46.083225 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"2a349843-a7b7-4dff-a5ae-f513337c5809","Type":"ContainerStarted","Data":"237be9059aceba08b75de8d8ecd8ee5c1eb28f5bd5c20bb5c20a33088beca081"} Dec 03 12:12:46 crc kubenswrapper[5052]: I1203 12:12:46.083290 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"2a349843-a7b7-4dff-a5ae-f513337c5809","Type":"ContainerStarted","Data":"5f18143bbfa921f999a45907b4264b014c3cbd86a24fabeba94b8ae7143fcb1d"} Dec 03 12:12:46 crc kubenswrapper[5052]: I1203 12:12:46.089586 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"f3339fc5-9d8a-4a19-ba83-b5f02f9d86d3","Type":"ContainerStarted","Data":"c51412b43d032effc161eda7a5d9e1847c358086f4766840050e85e65b009a7b"} Dec 03 12:12:46 crc kubenswrapper[5052]: I1203 12:12:46.090197 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="f3339fc5-9d8a-4a19-ba83-b5f02f9d86d3" containerName="ceilometer-central-agent" containerID="cri-o://86aeba625efdf4888a66dbb87161495cc7e53e5502649308eea48c73eebda680" gracePeriod=30 Dec 03 12:12:46 crc kubenswrapper[5052]: I1203 12:12:46.090415 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:12:46 crc kubenswrapper[5052]: I1203 12:12:46.090444 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="f3339fc5-9d8a-4a19-ba83-b5f02f9d86d3" containerName="sg-core" containerID="cri-o://a128294473b2ae65eb81c532b3040ade4790b89f3cfa21e3d64a4929843efc65" gracePeriod=30 Dec 03 12:12:46 crc kubenswrapper[5052]: I1203 12:12:46.090440 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="f3339fc5-9d8a-4a19-ba83-b5f02f9d86d3" containerName="proxy-httpd" containerID="cri-o://c51412b43d032effc161eda7a5d9e1847c358086f4766840050e85e65b009a7b" gracePeriod=30 Dec 03 12:12:46 crc kubenswrapper[5052]: I1203 12:12:46.090634 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="f3339fc5-9d8a-4a19-ba83-b5f02f9d86d3" containerName="ceilometer-notification-agent" containerID="cri-o://aa0bc6f7ead0eddf2779c1b4d8c7ab91d106b7722a51650dfdf4c53883d3f5aa" gracePeriod=30 Dec 03 12:12:46 crc kubenswrapper[5052]: I1203 12:12:46.138730 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/ceilometer-0" podStartSLOduration=2.300718667 podStartE2EDuration="5.13870252s" podCreationTimestamp="2025-12-03 12:12:41 +0000 UTC" firstStartedPulling="2025-12-03 12:12:41.814303605 +0000 UTC m=+3152.402545369" lastFinishedPulling="2025-12-03 12:12:44.652287458 +0000 UTC m=+3155.240529222" observedRunningTime="2025-12-03 12:12:46.12014107 +0000 UTC m=+3156.708382844" watchObservedRunningTime="2025-12-03 12:12:46.13870252 +0000 UTC m=+3156.726944294" Dec 03 12:12:47 crc kubenswrapper[5052]: I1203 12:12:47.106553 5052 generic.go:334] "Generic (PLEG): container finished" podID="f3339fc5-9d8a-4a19-ba83-b5f02f9d86d3" containerID="c51412b43d032effc161eda7a5d9e1847c358086f4766840050e85e65b009a7b" exitCode=0 Dec 03 12:12:47 crc kubenswrapper[5052]: I1203 12:12:47.106908 5052 generic.go:334] "Generic (PLEG): container finished" podID="f3339fc5-9d8a-4a19-ba83-b5f02f9d86d3" containerID="a128294473b2ae65eb81c532b3040ade4790b89f3cfa21e3d64a4929843efc65" exitCode=2 Dec 03 12:12:47 crc kubenswrapper[5052]: I1203 12:12:47.106919 5052 generic.go:334] "Generic (PLEG): container finished" podID="f3339fc5-9d8a-4a19-ba83-b5f02f9d86d3" containerID="86aeba625efdf4888a66dbb87161495cc7e53e5502649308eea48c73eebda680" exitCode=0 Dec 03 12:12:47 crc kubenswrapper[5052]: I1203 12:12:47.106639 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"f3339fc5-9d8a-4a19-ba83-b5f02f9d86d3","Type":"ContainerDied","Data":"c51412b43d032effc161eda7a5d9e1847c358086f4766840050e85e65b009a7b"} Dec 03 12:12:47 crc kubenswrapper[5052]: I1203 12:12:47.107008 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"f3339fc5-9d8a-4a19-ba83-b5f02f9d86d3","Type":"ContainerDied","Data":"a128294473b2ae65eb81c532b3040ade4790b89f3cfa21e3d64a4929843efc65"} Dec 03 12:12:47 crc kubenswrapper[5052]: I1203 12:12:47.107023 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"f3339fc5-9d8a-4a19-ba83-b5f02f9d86d3","Type":"ContainerDied","Data":"86aeba625efdf4888a66dbb87161495cc7e53e5502649308eea48c73eebda680"} Dec 03 12:12:47 crc kubenswrapper[5052]: I1203 12:12:47.110929 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"d81b09dc-75a1-45f8-ace4-786531ad3bfd","Type":"ContainerStarted","Data":"be83ec436e5bb13951ed23e2fd4e1e84fd5f3b6ce1f0a6f46b5c44dcd3634ee4"} Dec 03 12:12:47 crc kubenswrapper[5052]: I1203 12:12:47.673568 5052 scope.go:117] "RemoveContainer" containerID="63298107b3d090deff9ed92f3d613e5b1c86d9137684d117500257db8ee1c805" Dec 03 12:12:47 crc kubenswrapper[5052]: E1203 12:12:47.674266 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 12:12:48 crc kubenswrapper[5052]: I1203 12:12:48.128016 5052 generic.go:334] "Generic (PLEG): container finished" podID="f3339fc5-9d8a-4a19-ba83-b5f02f9d86d3" containerID="aa0bc6f7ead0eddf2779c1b4d8c7ab91d106b7722a51650dfdf4c53883d3f5aa" exitCode=0 Dec 03 12:12:48 crc kubenswrapper[5052]: I1203 12:12:48.128120 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"f3339fc5-9d8a-4a19-ba83-b5f02f9d86d3","Type":"ContainerDied","Data":"aa0bc6f7ead0eddf2779c1b4d8c7ab91d106b7722a51650dfdf4c53883d3f5aa"} Dec 03 12:12:48 crc kubenswrapper[5052]: I1203 12:12:48.128477 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"f3339fc5-9d8a-4a19-ba83-b5f02f9d86d3","Type":"ContainerDied","Data":"3e04272a19e601d7860e121937041377a46908c6403553d4bb7fa51c29ddd7b0"} Dec 03 12:12:48 crc kubenswrapper[5052]: I1203 12:12:48.128493 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3e04272a19e601d7860e121937041377a46908c6403553d4bb7fa51c29ddd7b0" Dec 03 12:12:48 crc kubenswrapper[5052]: I1203 12:12:48.131385 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"d81b09dc-75a1-45f8-ace4-786531ad3bfd","Type":"ContainerStarted","Data":"e29fe95d4add082f6c46305d5e3fe12f454dfccd4b1d5c2355d53782ac82fa29"} Dec 03 12:12:48 crc kubenswrapper[5052]: I1203 12:12:48.133843 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"2a349843-a7b7-4dff-a5ae-f513337c5809","Type":"ContainerStarted","Data":"f30fe921af5f7b637639949f8977c3720f1ebb48e3b4f39277fcbc953207de23"} Dec 03 12:12:48 crc kubenswrapper[5052]: I1203 12:12:48.160820 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/glance-default-internal-api-0" podStartSLOduration=3.160799616 podStartE2EDuration="3.160799616s" podCreationTimestamp="2025-12-03 12:12:45 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:12:48.159052446 +0000 UTC m=+3158.747294220" watchObservedRunningTime="2025-12-03 12:12:48.160799616 +0000 UTC m=+3158.749041380" Dec 03 12:12:48 crc kubenswrapper[5052]: I1203 12:12:48.190898 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/glance-default-external-api-0" podStartSLOduration=4.190872205 podStartE2EDuration="4.190872205s" podCreationTimestamp="2025-12-03 12:12:44 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:12:48.186892691 +0000 UTC m=+3158.775134465" watchObservedRunningTime="2025-12-03 12:12:48.190872205 +0000 UTC m=+3158.779113969" Dec 03 12:12:48 crc kubenswrapper[5052]: I1203 12:12:48.205989 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:12:48 crc kubenswrapper[5052]: I1203 12:12:48.331317 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/f3339fc5-9d8a-4a19-ba83-b5f02f9d86d3-scripts\") pod \"f3339fc5-9d8a-4a19-ba83-b5f02f9d86d3\" (UID: \"f3339fc5-9d8a-4a19-ba83-b5f02f9d86d3\") " Dec 03 12:12:48 crc kubenswrapper[5052]: I1203 12:12:48.331884 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/f3339fc5-9d8a-4a19-ba83-b5f02f9d86d3-run-httpd\") pod \"f3339fc5-9d8a-4a19-ba83-b5f02f9d86d3\" (UID: \"f3339fc5-9d8a-4a19-ba83-b5f02f9d86d3\") " Dec 03 12:12:48 crc kubenswrapper[5052]: I1203 12:12:48.331941 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ktrld\" (UniqueName: \"kubernetes.io/projected/f3339fc5-9d8a-4a19-ba83-b5f02f9d86d3-kube-api-access-ktrld\") pod \"f3339fc5-9d8a-4a19-ba83-b5f02f9d86d3\" (UID: \"f3339fc5-9d8a-4a19-ba83-b5f02f9d86d3\") " Dec 03 12:12:48 crc kubenswrapper[5052]: I1203 12:12:48.332017 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/f3339fc5-9d8a-4a19-ba83-b5f02f9d86d3-sg-core-conf-yaml\") pod \"f3339fc5-9d8a-4a19-ba83-b5f02f9d86d3\" (UID: \"f3339fc5-9d8a-4a19-ba83-b5f02f9d86d3\") " Dec 03 12:12:48 crc kubenswrapper[5052]: I1203 12:12:48.332073 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f3339fc5-9d8a-4a19-ba83-b5f02f9d86d3-combined-ca-bundle\") pod \"f3339fc5-9d8a-4a19-ba83-b5f02f9d86d3\" (UID: \"f3339fc5-9d8a-4a19-ba83-b5f02f9d86d3\") " Dec 03 12:12:48 crc kubenswrapper[5052]: I1203 12:12:48.332152 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/f3339fc5-9d8a-4a19-ba83-b5f02f9d86d3-log-httpd\") pod \"f3339fc5-9d8a-4a19-ba83-b5f02f9d86d3\" (UID: \"f3339fc5-9d8a-4a19-ba83-b5f02f9d86d3\") " Dec 03 12:12:48 crc kubenswrapper[5052]: I1203 12:12:48.332243 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/f3339fc5-9d8a-4a19-ba83-b5f02f9d86d3-config-data\") pod \"f3339fc5-9d8a-4a19-ba83-b5f02f9d86d3\" (UID: \"f3339fc5-9d8a-4a19-ba83-b5f02f9d86d3\") " Dec 03 12:12:48 crc kubenswrapper[5052]: I1203 12:12:48.332492 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f3339fc5-9d8a-4a19-ba83-b5f02f9d86d3-run-httpd" (OuterVolumeSpecName: "run-httpd") pod "f3339fc5-9d8a-4a19-ba83-b5f02f9d86d3" (UID: "f3339fc5-9d8a-4a19-ba83-b5f02f9d86d3"). InnerVolumeSpecName "run-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:12:48 crc kubenswrapper[5052]: I1203 12:12:48.333122 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f3339fc5-9d8a-4a19-ba83-b5f02f9d86d3-log-httpd" (OuterVolumeSpecName: "log-httpd") pod "f3339fc5-9d8a-4a19-ba83-b5f02f9d86d3" (UID: "f3339fc5-9d8a-4a19-ba83-b5f02f9d86d3"). InnerVolumeSpecName "log-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:12:48 crc kubenswrapper[5052]: I1203 12:12:48.334010 5052 reconciler_common.go:293] "Volume detached for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/f3339fc5-9d8a-4a19-ba83-b5f02f9d86d3-run-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:12:48 crc kubenswrapper[5052]: I1203 12:12:48.334035 5052 reconciler_common.go:293] "Volume detached for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/f3339fc5-9d8a-4a19-ba83-b5f02f9d86d3-log-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:12:48 crc kubenswrapper[5052]: I1203 12:12:48.337766 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f3339fc5-9d8a-4a19-ba83-b5f02f9d86d3-kube-api-access-ktrld" (OuterVolumeSpecName: "kube-api-access-ktrld") pod "f3339fc5-9d8a-4a19-ba83-b5f02f9d86d3" (UID: "f3339fc5-9d8a-4a19-ba83-b5f02f9d86d3"). InnerVolumeSpecName "kube-api-access-ktrld". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:12:48 crc kubenswrapper[5052]: I1203 12:12:48.355312 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f3339fc5-9d8a-4a19-ba83-b5f02f9d86d3-scripts" (OuterVolumeSpecName: "scripts") pod "f3339fc5-9d8a-4a19-ba83-b5f02f9d86d3" (UID: "f3339fc5-9d8a-4a19-ba83-b5f02f9d86d3"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:12:48 crc kubenswrapper[5052]: I1203 12:12:48.366546 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f3339fc5-9d8a-4a19-ba83-b5f02f9d86d3-sg-core-conf-yaml" (OuterVolumeSpecName: "sg-core-conf-yaml") pod "f3339fc5-9d8a-4a19-ba83-b5f02f9d86d3" (UID: "f3339fc5-9d8a-4a19-ba83-b5f02f9d86d3"). InnerVolumeSpecName "sg-core-conf-yaml". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:12:48 crc kubenswrapper[5052]: I1203 12:12:48.419142 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f3339fc5-9d8a-4a19-ba83-b5f02f9d86d3-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "f3339fc5-9d8a-4a19-ba83-b5f02f9d86d3" (UID: "f3339fc5-9d8a-4a19-ba83-b5f02f9d86d3"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:12:48 crc kubenswrapper[5052]: I1203 12:12:48.435305 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-ktrld\" (UniqueName: \"kubernetes.io/projected/f3339fc5-9d8a-4a19-ba83-b5f02f9d86d3-kube-api-access-ktrld\") on node \"crc\" DevicePath \"\"" Dec 03 12:12:48 crc kubenswrapper[5052]: I1203 12:12:48.435334 5052 reconciler_common.go:293] "Volume detached for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/f3339fc5-9d8a-4a19-ba83-b5f02f9d86d3-sg-core-conf-yaml\") on node \"crc\" DevicePath \"\"" Dec 03 12:12:48 crc kubenswrapper[5052]: I1203 12:12:48.435345 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f3339fc5-9d8a-4a19-ba83-b5f02f9d86d3-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:12:48 crc kubenswrapper[5052]: I1203 12:12:48.435356 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/f3339fc5-9d8a-4a19-ba83-b5f02f9d86d3-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:12:48 crc kubenswrapper[5052]: I1203 12:12:48.448676 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f3339fc5-9d8a-4a19-ba83-b5f02f9d86d3-config-data" (OuterVolumeSpecName: "config-data") pod "f3339fc5-9d8a-4a19-ba83-b5f02f9d86d3" (UID: "f3339fc5-9d8a-4a19-ba83-b5f02f9d86d3"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:12:48 crc kubenswrapper[5052]: I1203 12:12:48.537297 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/f3339fc5-9d8a-4a19-ba83-b5f02f9d86d3-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:12:49 crc kubenswrapper[5052]: I1203 12:12:49.142409 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:12:49 crc kubenswrapper[5052]: I1203 12:12:49.184933 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:12:49 crc kubenswrapper[5052]: I1203 12:12:49.194134 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:12:49 crc kubenswrapper[5052]: I1203 12:12:49.221662 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:12:49 crc kubenswrapper[5052]: E1203 12:12:49.222165 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="f3339fc5-9d8a-4a19-ba83-b5f02f9d86d3" containerName="ceilometer-central-agent" Dec 03 12:12:49 crc kubenswrapper[5052]: I1203 12:12:49.222189 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="f3339fc5-9d8a-4a19-ba83-b5f02f9d86d3" containerName="ceilometer-central-agent" Dec 03 12:12:49 crc kubenswrapper[5052]: E1203 12:12:49.222239 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="f3339fc5-9d8a-4a19-ba83-b5f02f9d86d3" containerName="ceilometer-notification-agent" Dec 03 12:12:49 crc kubenswrapper[5052]: I1203 12:12:49.222245 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="f3339fc5-9d8a-4a19-ba83-b5f02f9d86d3" containerName="ceilometer-notification-agent" Dec 03 12:12:49 crc kubenswrapper[5052]: E1203 12:12:49.222261 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="f3339fc5-9d8a-4a19-ba83-b5f02f9d86d3" containerName="proxy-httpd" Dec 03 12:12:49 crc kubenswrapper[5052]: I1203 12:12:49.222268 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="f3339fc5-9d8a-4a19-ba83-b5f02f9d86d3" containerName="proxy-httpd" Dec 03 12:12:49 crc kubenswrapper[5052]: E1203 12:12:49.222279 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="f3339fc5-9d8a-4a19-ba83-b5f02f9d86d3" containerName="sg-core" Dec 03 12:12:49 crc kubenswrapper[5052]: I1203 12:12:49.222285 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="f3339fc5-9d8a-4a19-ba83-b5f02f9d86d3" containerName="sg-core" Dec 03 12:12:49 crc kubenswrapper[5052]: I1203 12:12:49.222450 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="f3339fc5-9d8a-4a19-ba83-b5f02f9d86d3" containerName="ceilometer-notification-agent" Dec 03 12:12:49 crc kubenswrapper[5052]: I1203 12:12:49.222470 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="f3339fc5-9d8a-4a19-ba83-b5f02f9d86d3" containerName="proxy-httpd" Dec 03 12:12:49 crc kubenswrapper[5052]: I1203 12:12:49.222481 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="f3339fc5-9d8a-4a19-ba83-b5f02f9d86d3" containerName="ceilometer-central-agent" Dec 03 12:12:49 crc kubenswrapper[5052]: I1203 12:12:49.222490 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="f3339fc5-9d8a-4a19-ba83-b5f02f9d86d3" containerName="sg-core" Dec 03 12:12:49 crc kubenswrapper[5052]: I1203 12:12:49.224601 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:12:49 crc kubenswrapper[5052]: I1203 12:12:49.226776 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-scripts" Dec 03 12:12:49 crc kubenswrapper[5052]: I1203 12:12:49.232006 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:12:49 crc kubenswrapper[5052]: I1203 12:12:49.237099 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-config-data" Dec 03 12:12:49 crc kubenswrapper[5052]: I1203 12:12:49.356386 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/0d97800f-6d5a-46ed-bcb5-4452b4ce86e8-run-httpd\") pod \"ceilometer-0\" (UID: \"0d97800f-6d5a-46ed-bcb5-4452b4ce86e8\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:12:49 crc kubenswrapper[5052]: I1203 12:12:49.356440 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c777f\" (UniqueName: \"kubernetes.io/projected/0d97800f-6d5a-46ed-bcb5-4452b4ce86e8-kube-api-access-c777f\") pod \"ceilometer-0\" (UID: \"0d97800f-6d5a-46ed-bcb5-4452b4ce86e8\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:12:49 crc kubenswrapper[5052]: I1203 12:12:49.356470 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0d97800f-6d5a-46ed-bcb5-4452b4ce86e8-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"0d97800f-6d5a-46ed-bcb5-4452b4ce86e8\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:12:49 crc kubenswrapper[5052]: I1203 12:12:49.356548 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/0d97800f-6d5a-46ed-bcb5-4452b4ce86e8-log-httpd\") pod \"ceilometer-0\" (UID: \"0d97800f-6d5a-46ed-bcb5-4452b4ce86e8\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:12:49 crc kubenswrapper[5052]: I1203 12:12:49.356602 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/0d97800f-6d5a-46ed-bcb5-4452b4ce86e8-scripts\") pod \"ceilometer-0\" (UID: \"0d97800f-6d5a-46ed-bcb5-4452b4ce86e8\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:12:49 crc kubenswrapper[5052]: I1203 12:12:49.356660 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/0d97800f-6d5a-46ed-bcb5-4452b4ce86e8-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"0d97800f-6d5a-46ed-bcb5-4452b4ce86e8\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:12:49 crc kubenswrapper[5052]: I1203 12:12:49.356733 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/0d97800f-6d5a-46ed-bcb5-4452b4ce86e8-config-data\") pod \"ceilometer-0\" (UID: \"0d97800f-6d5a-46ed-bcb5-4452b4ce86e8\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:12:49 crc kubenswrapper[5052]: I1203 12:12:49.458675 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/0d97800f-6d5a-46ed-bcb5-4452b4ce86e8-scripts\") pod \"ceilometer-0\" (UID: \"0d97800f-6d5a-46ed-bcb5-4452b4ce86e8\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:12:49 crc kubenswrapper[5052]: I1203 12:12:49.458777 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/0d97800f-6d5a-46ed-bcb5-4452b4ce86e8-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"0d97800f-6d5a-46ed-bcb5-4452b4ce86e8\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:12:49 crc kubenswrapper[5052]: I1203 12:12:49.458871 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/0d97800f-6d5a-46ed-bcb5-4452b4ce86e8-config-data\") pod \"ceilometer-0\" (UID: \"0d97800f-6d5a-46ed-bcb5-4452b4ce86e8\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:12:49 crc kubenswrapper[5052]: I1203 12:12:49.458951 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/0d97800f-6d5a-46ed-bcb5-4452b4ce86e8-run-httpd\") pod \"ceilometer-0\" (UID: \"0d97800f-6d5a-46ed-bcb5-4452b4ce86e8\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:12:49 crc kubenswrapper[5052]: I1203 12:12:49.458982 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-c777f\" (UniqueName: \"kubernetes.io/projected/0d97800f-6d5a-46ed-bcb5-4452b4ce86e8-kube-api-access-c777f\") pod \"ceilometer-0\" (UID: \"0d97800f-6d5a-46ed-bcb5-4452b4ce86e8\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:12:49 crc kubenswrapper[5052]: I1203 12:12:49.459016 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0d97800f-6d5a-46ed-bcb5-4452b4ce86e8-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"0d97800f-6d5a-46ed-bcb5-4452b4ce86e8\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:12:49 crc kubenswrapper[5052]: I1203 12:12:49.459058 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/0d97800f-6d5a-46ed-bcb5-4452b4ce86e8-log-httpd\") pod \"ceilometer-0\" (UID: \"0d97800f-6d5a-46ed-bcb5-4452b4ce86e8\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:12:49 crc kubenswrapper[5052]: I1203 12:12:49.459504 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/0d97800f-6d5a-46ed-bcb5-4452b4ce86e8-run-httpd\") pod \"ceilometer-0\" (UID: \"0d97800f-6d5a-46ed-bcb5-4452b4ce86e8\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:12:49 crc kubenswrapper[5052]: I1203 12:12:49.459540 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/0d97800f-6d5a-46ed-bcb5-4452b4ce86e8-log-httpd\") pod \"ceilometer-0\" (UID: \"0d97800f-6d5a-46ed-bcb5-4452b4ce86e8\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:12:49 crc kubenswrapper[5052]: I1203 12:12:49.464597 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/0d97800f-6d5a-46ed-bcb5-4452b4ce86e8-config-data\") pod \"ceilometer-0\" (UID: \"0d97800f-6d5a-46ed-bcb5-4452b4ce86e8\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:12:49 crc kubenswrapper[5052]: I1203 12:12:49.465588 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0d97800f-6d5a-46ed-bcb5-4452b4ce86e8-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"0d97800f-6d5a-46ed-bcb5-4452b4ce86e8\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:12:49 crc kubenswrapper[5052]: I1203 12:12:49.467003 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/0d97800f-6d5a-46ed-bcb5-4452b4ce86e8-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"0d97800f-6d5a-46ed-bcb5-4452b4ce86e8\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:12:49 crc kubenswrapper[5052]: I1203 12:12:49.468693 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/0d97800f-6d5a-46ed-bcb5-4452b4ce86e8-scripts\") pod \"ceilometer-0\" (UID: \"0d97800f-6d5a-46ed-bcb5-4452b4ce86e8\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:12:49 crc kubenswrapper[5052]: I1203 12:12:49.481668 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-c777f\" (UniqueName: \"kubernetes.io/projected/0d97800f-6d5a-46ed-bcb5-4452b4ce86e8-kube-api-access-c777f\") pod \"ceilometer-0\" (UID: \"0d97800f-6d5a-46ed-bcb5-4452b4ce86e8\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:12:49 crc kubenswrapper[5052]: I1203 12:12:49.540880 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:12:49 crc kubenswrapper[5052]: I1203 12:12:49.687952 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f3339fc5-9d8a-4a19-ba83-b5f02f9d86d3" path="/var/lib/kubelet/pods/f3339fc5-9d8a-4a19-ba83-b5f02f9d86d3/volumes" Dec 03 12:12:50 crc kubenswrapper[5052]: I1203 12:12:50.028597 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:12:50 crc kubenswrapper[5052]: I1203 12:12:50.155486 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"0d97800f-6d5a-46ed-bcb5-4452b4ce86e8","Type":"ContainerStarted","Data":"998284bedfe3195b513f8dc5ec980921f3c637c456e6968e528812b414e9cd81"} Dec 03 12:12:50 crc kubenswrapper[5052]: I1203 12:12:50.388104 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-api-db-create-mnlg4"] Dec 03 12:12:50 crc kubenswrapper[5052]: I1203 12:12:50.389176 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-db-create-mnlg4" Dec 03 12:12:50 crc kubenswrapper[5052]: I1203 12:12:50.403403 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-db-create-mnlg4"] Dec 03 12:12:50 crc kubenswrapper[5052]: I1203 12:12:50.477573 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pvjmg\" (UniqueName: \"kubernetes.io/projected/a2493bc9-910f-4f3d-9e74-cbd5721c838d-kube-api-access-pvjmg\") pod \"nova-api-db-create-mnlg4\" (UID: \"a2493bc9-910f-4f3d-9e74-cbd5721c838d\") " pod="openstack-kuttl-tests/nova-api-db-create-mnlg4" Dec 03 12:12:50 crc kubenswrapper[5052]: I1203 12:12:50.477790 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/a2493bc9-910f-4f3d-9e74-cbd5721c838d-operator-scripts\") pod \"nova-api-db-create-mnlg4\" (UID: \"a2493bc9-910f-4f3d-9e74-cbd5721c838d\") " pod="openstack-kuttl-tests/nova-api-db-create-mnlg4" Dec 03 12:12:50 crc kubenswrapper[5052]: I1203 12:12:50.501404 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-api-06eb-account-create-update-cbfzl"] Dec 03 12:12:50 crc kubenswrapper[5052]: I1203 12:12:50.503081 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-06eb-account-create-update-cbfzl" Dec 03 12:12:50 crc kubenswrapper[5052]: I1203 12:12:50.506732 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-api-db-secret" Dec 03 12:12:50 crc kubenswrapper[5052]: I1203 12:12:50.509002 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell0-db-create-bd44j"] Dec 03 12:12:50 crc kubenswrapper[5052]: I1203 12:12:50.510577 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-db-create-bd44j" Dec 03 12:12:50 crc kubenswrapper[5052]: I1203 12:12:50.528705 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell0-db-create-bd44j"] Dec 03 12:12:50 crc kubenswrapper[5052]: I1203 12:12:50.549659 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-06eb-account-create-update-cbfzl"] Dec 03 12:12:50 crc kubenswrapper[5052]: I1203 12:12:50.580251 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8dsv6\" (UniqueName: \"kubernetes.io/projected/57bde9b6-d244-4159-840f-25e35a2f8dc6-kube-api-access-8dsv6\") pod \"nova-cell0-db-create-bd44j\" (UID: \"57bde9b6-d244-4159-840f-25e35a2f8dc6\") " pod="openstack-kuttl-tests/nova-cell0-db-create-bd44j" Dec 03 12:12:50 crc kubenswrapper[5052]: I1203 12:12:50.580316 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-pvjmg\" (UniqueName: \"kubernetes.io/projected/a2493bc9-910f-4f3d-9e74-cbd5721c838d-kube-api-access-pvjmg\") pod \"nova-api-db-create-mnlg4\" (UID: \"a2493bc9-910f-4f3d-9e74-cbd5721c838d\") " pod="openstack-kuttl-tests/nova-api-db-create-mnlg4" Dec 03 12:12:50 crc kubenswrapper[5052]: I1203 12:12:50.580429 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/57bde9b6-d244-4159-840f-25e35a2f8dc6-operator-scripts\") pod \"nova-cell0-db-create-bd44j\" (UID: \"57bde9b6-d244-4159-840f-25e35a2f8dc6\") " pod="openstack-kuttl-tests/nova-cell0-db-create-bd44j" Dec 03 12:12:50 crc kubenswrapper[5052]: I1203 12:12:50.580562 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/16ff7253-a0ad-48a5-92e3-748f8053fc1a-operator-scripts\") pod \"nova-api-06eb-account-create-update-cbfzl\" (UID: \"16ff7253-a0ad-48a5-92e3-748f8053fc1a\") " pod="openstack-kuttl-tests/nova-api-06eb-account-create-update-cbfzl" Dec 03 12:12:50 crc kubenswrapper[5052]: I1203 12:12:50.580729 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fhjhz\" (UniqueName: \"kubernetes.io/projected/16ff7253-a0ad-48a5-92e3-748f8053fc1a-kube-api-access-fhjhz\") pod \"nova-api-06eb-account-create-update-cbfzl\" (UID: \"16ff7253-a0ad-48a5-92e3-748f8053fc1a\") " pod="openstack-kuttl-tests/nova-api-06eb-account-create-update-cbfzl" Dec 03 12:12:50 crc kubenswrapper[5052]: I1203 12:12:50.580944 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/a2493bc9-910f-4f3d-9e74-cbd5721c838d-operator-scripts\") pod \"nova-api-db-create-mnlg4\" (UID: \"a2493bc9-910f-4f3d-9e74-cbd5721c838d\") " pod="openstack-kuttl-tests/nova-api-db-create-mnlg4" Dec 03 12:12:50 crc kubenswrapper[5052]: I1203 12:12:50.583355 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/a2493bc9-910f-4f3d-9e74-cbd5721c838d-operator-scripts\") pod \"nova-api-db-create-mnlg4\" (UID: \"a2493bc9-910f-4f3d-9e74-cbd5721c838d\") " pod="openstack-kuttl-tests/nova-api-db-create-mnlg4" Dec 03 12:12:50 crc kubenswrapper[5052]: I1203 12:12:50.607431 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-pvjmg\" (UniqueName: \"kubernetes.io/projected/a2493bc9-910f-4f3d-9e74-cbd5721c838d-kube-api-access-pvjmg\") pod \"nova-api-db-create-mnlg4\" (UID: \"a2493bc9-910f-4f3d-9e74-cbd5721c838d\") " pod="openstack-kuttl-tests/nova-api-db-create-mnlg4" Dec 03 12:12:50 crc kubenswrapper[5052]: I1203 12:12:50.682089 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-8dsv6\" (UniqueName: \"kubernetes.io/projected/57bde9b6-d244-4159-840f-25e35a2f8dc6-kube-api-access-8dsv6\") pod \"nova-cell0-db-create-bd44j\" (UID: \"57bde9b6-d244-4159-840f-25e35a2f8dc6\") " pod="openstack-kuttl-tests/nova-cell0-db-create-bd44j" Dec 03 12:12:50 crc kubenswrapper[5052]: I1203 12:12:50.682144 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/57bde9b6-d244-4159-840f-25e35a2f8dc6-operator-scripts\") pod \"nova-cell0-db-create-bd44j\" (UID: \"57bde9b6-d244-4159-840f-25e35a2f8dc6\") " pod="openstack-kuttl-tests/nova-cell0-db-create-bd44j" Dec 03 12:12:50 crc kubenswrapper[5052]: I1203 12:12:50.682167 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/16ff7253-a0ad-48a5-92e3-748f8053fc1a-operator-scripts\") pod \"nova-api-06eb-account-create-update-cbfzl\" (UID: \"16ff7253-a0ad-48a5-92e3-748f8053fc1a\") " pod="openstack-kuttl-tests/nova-api-06eb-account-create-update-cbfzl" Dec 03 12:12:50 crc kubenswrapper[5052]: I1203 12:12:50.682212 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-fhjhz\" (UniqueName: \"kubernetes.io/projected/16ff7253-a0ad-48a5-92e3-748f8053fc1a-kube-api-access-fhjhz\") pod \"nova-api-06eb-account-create-update-cbfzl\" (UID: \"16ff7253-a0ad-48a5-92e3-748f8053fc1a\") " pod="openstack-kuttl-tests/nova-api-06eb-account-create-update-cbfzl" Dec 03 12:12:50 crc kubenswrapper[5052]: I1203 12:12:50.683283 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/16ff7253-a0ad-48a5-92e3-748f8053fc1a-operator-scripts\") pod \"nova-api-06eb-account-create-update-cbfzl\" (UID: \"16ff7253-a0ad-48a5-92e3-748f8053fc1a\") " pod="openstack-kuttl-tests/nova-api-06eb-account-create-update-cbfzl" Dec 03 12:12:50 crc kubenswrapper[5052]: I1203 12:12:50.683608 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/57bde9b6-d244-4159-840f-25e35a2f8dc6-operator-scripts\") pod \"nova-cell0-db-create-bd44j\" (UID: \"57bde9b6-d244-4159-840f-25e35a2f8dc6\") " pod="openstack-kuttl-tests/nova-cell0-db-create-bd44j" Dec 03 12:12:50 crc kubenswrapper[5052]: I1203 12:12:50.701828 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell1-db-create-2mq6x"] Dec 03 12:12:50 crc kubenswrapper[5052]: I1203 12:12:50.703121 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-db-create-2mq6x" Dec 03 12:12:50 crc kubenswrapper[5052]: I1203 12:12:50.707457 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-db-create-mnlg4" Dec 03 12:12:50 crc kubenswrapper[5052]: I1203 12:12:50.708824 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-fhjhz\" (UniqueName: \"kubernetes.io/projected/16ff7253-a0ad-48a5-92e3-748f8053fc1a-kube-api-access-fhjhz\") pod \"nova-api-06eb-account-create-update-cbfzl\" (UID: \"16ff7253-a0ad-48a5-92e3-748f8053fc1a\") " pod="openstack-kuttl-tests/nova-api-06eb-account-create-update-cbfzl" Dec 03 12:12:50 crc kubenswrapper[5052]: I1203 12:12:50.709657 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-8dsv6\" (UniqueName: \"kubernetes.io/projected/57bde9b6-d244-4159-840f-25e35a2f8dc6-kube-api-access-8dsv6\") pod \"nova-cell0-db-create-bd44j\" (UID: \"57bde9b6-d244-4159-840f-25e35a2f8dc6\") " pod="openstack-kuttl-tests/nova-cell0-db-create-bd44j" Dec 03 12:12:50 crc kubenswrapper[5052]: I1203 12:12:50.713783 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell0-0bb5-account-create-update-6flhc"] Dec 03 12:12:50 crc kubenswrapper[5052]: I1203 12:12:50.728913 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-0bb5-account-create-update-6flhc" Dec 03 12:12:50 crc kubenswrapper[5052]: I1203 12:12:50.737664 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell0-db-secret" Dec 03 12:12:50 crc kubenswrapper[5052]: I1203 12:12:50.740655 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-db-create-2mq6x"] Dec 03 12:12:50 crc kubenswrapper[5052]: I1203 12:12:50.752981 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell0-0bb5-account-create-update-6flhc"] Dec 03 12:12:50 crc kubenswrapper[5052]: I1203 12:12:50.783822 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zckk8\" (UniqueName: \"kubernetes.io/projected/2e0f8c25-7b0e-439d-8518-1a95a325e502-kube-api-access-zckk8\") pod \"nova-cell1-db-create-2mq6x\" (UID: \"2e0f8c25-7b0e-439d-8518-1a95a325e502\") " pod="openstack-kuttl-tests/nova-cell1-db-create-2mq6x" Dec 03 12:12:50 crc kubenswrapper[5052]: I1203 12:12:50.783897 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/59c0dc2a-1f1d-46a1-93c1-6553bf351704-operator-scripts\") pod \"nova-cell0-0bb5-account-create-update-6flhc\" (UID: \"59c0dc2a-1f1d-46a1-93c1-6553bf351704\") " pod="openstack-kuttl-tests/nova-cell0-0bb5-account-create-update-6flhc" Dec 03 12:12:50 crc kubenswrapper[5052]: I1203 12:12:50.783940 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/2e0f8c25-7b0e-439d-8518-1a95a325e502-operator-scripts\") pod \"nova-cell1-db-create-2mq6x\" (UID: \"2e0f8c25-7b0e-439d-8518-1a95a325e502\") " pod="openstack-kuttl-tests/nova-cell1-db-create-2mq6x" Dec 03 12:12:50 crc kubenswrapper[5052]: I1203 12:12:50.783963 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6s8v9\" (UniqueName: \"kubernetes.io/projected/59c0dc2a-1f1d-46a1-93c1-6553bf351704-kube-api-access-6s8v9\") pod \"nova-cell0-0bb5-account-create-update-6flhc\" (UID: \"59c0dc2a-1f1d-46a1-93c1-6553bf351704\") " pod="openstack-kuttl-tests/nova-cell0-0bb5-account-create-update-6flhc" Dec 03 12:12:50 crc kubenswrapper[5052]: I1203 12:12:50.827608 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-06eb-account-create-update-cbfzl" Dec 03 12:12:50 crc kubenswrapper[5052]: I1203 12:12:50.839944 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-db-create-bd44j" Dec 03 12:12:50 crc kubenswrapper[5052]: I1203 12:12:50.886252 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-zckk8\" (UniqueName: \"kubernetes.io/projected/2e0f8c25-7b0e-439d-8518-1a95a325e502-kube-api-access-zckk8\") pod \"nova-cell1-db-create-2mq6x\" (UID: \"2e0f8c25-7b0e-439d-8518-1a95a325e502\") " pod="openstack-kuttl-tests/nova-cell1-db-create-2mq6x" Dec 03 12:12:50 crc kubenswrapper[5052]: I1203 12:12:50.886317 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/59c0dc2a-1f1d-46a1-93c1-6553bf351704-operator-scripts\") pod \"nova-cell0-0bb5-account-create-update-6flhc\" (UID: \"59c0dc2a-1f1d-46a1-93c1-6553bf351704\") " pod="openstack-kuttl-tests/nova-cell0-0bb5-account-create-update-6flhc" Dec 03 12:12:50 crc kubenswrapper[5052]: I1203 12:12:50.886357 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/2e0f8c25-7b0e-439d-8518-1a95a325e502-operator-scripts\") pod \"nova-cell1-db-create-2mq6x\" (UID: \"2e0f8c25-7b0e-439d-8518-1a95a325e502\") " pod="openstack-kuttl-tests/nova-cell1-db-create-2mq6x" Dec 03 12:12:50 crc kubenswrapper[5052]: I1203 12:12:50.886378 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-6s8v9\" (UniqueName: \"kubernetes.io/projected/59c0dc2a-1f1d-46a1-93c1-6553bf351704-kube-api-access-6s8v9\") pod \"nova-cell0-0bb5-account-create-update-6flhc\" (UID: \"59c0dc2a-1f1d-46a1-93c1-6553bf351704\") " pod="openstack-kuttl-tests/nova-cell0-0bb5-account-create-update-6flhc" Dec 03 12:12:50 crc kubenswrapper[5052]: I1203 12:12:50.887341 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/2e0f8c25-7b0e-439d-8518-1a95a325e502-operator-scripts\") pod \"nova-cell1-db-create-2mq6x\" (UID: \"2e0f8c25-7b0e-439d-8518-1a95a325e502\") " pod="openstack-kuttl-tests/nova-cell1-db-create-2mq6x" Dec 03 12:12:50 crc kubenswrapper[5052]: I1203 12:12:50.887930 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/59c0dc2a-1f1d-46a1-93c1-6553bf351704-operator-scripts\") pod \"nova-cell0-0bb5-account-create-update-6flhc\" (UID: \"59c0dc2a-1f1d-46a1-93c1-6553bf351704\") " pod="openstack-kuttl-tests/nova-cell0-0bb5-account-create-update-6flhc" Dec 03 12:12:50 crc kubenswrapper[5052]: I1203 12:12:50.899298 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell1-f1c3-account-create-update-5mzt2"] Dec 03 12:12:50 crc kubenswrapper[5052]: I1203 12:12:50.900601 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-f1c3-account-create-update-5mzt2" Dec 03 12:12:50 crc kubenswrapper[5052]: I1203 12:12:50.904617 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell1-db-secret" Dec 03 12:12:50 crc kubenswrapper[5052]: I1203 12:12:50.907390 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-6s8v9\" (UniqueName: \"kubernetes.io/projected/59c0dc2a-1f1d-46a1-93c1-6553bf351704-kube-api-access-6s8v9\") pod \"nova-cell0-0bb5-account-create-update-6flhc\" (UID: \"59c0dc2a-1f1d-46a1-93c1-6553bf351704\") " pod="openstack-kuttl-tests/nova-cell0-0bb5-account-create-update-6flhc" Dec 03 12:12:50 crc kubenswrapper[5052]: I1203 12:12:50.907832 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-zckk8\" (UniqueName: \"kubernetes.io/projected/2e0f8c25-7b0e-439d-8518-1a95a325e502-kube-api-access-zckk8\") pod \"nova-cell1-db-create-2mq6x\" (UID: \"2e0f8c25-7b0e-439d-8518-1a95a325e502\") " pod="openstack-kuttl-tests/nova-cell1-db-create-2mq6x" Dec 03 12:12:50 crc kubenswrapper[5052]: I1203 12:12:50.921679 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-f1c3-account-create-update-5mzt2"] Dec 03 12:12:50 crc kubenswrapper[5052]: I1203 12:12:50.990608 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g8h2r\" (UniqueName: \"kubernetes.io/projected/bb817663-1fe7-4c73-803b-5e64dc4ba616-kube-api-access-g8h2r\") pod \"nova-cell1-f1c3-account-create-update-5mzt2\" (UID: \"bb817663-1fe7-4c73-803b-5e64dc4ba616\") " pod="openstack-kuttl-tests/nova-cell1-f1c3-account-create-update-5mzt2" Dec 03 12:12:50 crc kubenswrapper[5052]: I1203 12:12:50.990653 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/bb817663-1fe7-4c73-803b-5e64dc4ba616-operator-scripts\") pod \"nova-cell1-f1c3-account-create-update-5mzt2\" (UID: \"bb817663-1fe7-4c73-803b-5e64dc4ba616\") " pod="openstack-kuttl-tests/nova-cell1-f1c3-account-create-update-5mzt2" Dec 03 12:12:51 crc kubenswrapper[5052]: I1203 12:12:51.092563 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-g8h2r\" (UniqueName: \"kubernetes.io/projected/bb817663-1fe7-4c73-803b-5e64dc4ba616-kube-api-access-g8h2r\") pod \"nova-cell1-f1c3-account-create-update-5mzt2\" (UID: \"bb817663-1fe7-4c73-803b-5e64dc4ba616\") " pod="openstack-kuttl-tests/nova-cell1-f1c3-account-create-update-5mzt2" Dec 03 12:12:51 crc kubenswrapper[5052]: I1203 12:12:51.092929 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/bb817663-1fe7-4c73-803b-5e64dc4ba616-operator-scripts\") pod \"nova-cell1-f1c3-account-create-update-5mzt2\" (UID: \"bb817663-1fe7-4c73-803b-5e64dc4ba616\") " pod="openstack-kuttl-tests/nova-cell1-f1c3-account-create-update-5mzt2" Dec 03 12:12:51 crc kubenswrapper[5052]: I1203 12:12:51.094689 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/bb817663-1fe7-4c73-803b-5e64dc4ba616-operator-scripts\") pod \"nova-cell1-f1c3-account-create-update-5mzt2\" (UID: \"bb817663-1fe7-4c73-803b-5e64dc4ba616\") " pod="openstack-kuttl-tests/nova-cell1-f1c3-account-create-update-5mzt2" Dec 03 12:12:51 crc kubenswrapper[5052]: I1203 12:12:51.110114 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-db-create-2mq6x" Dec 03 12:12:51 crc kubenswrapper[5052]: I1203 12:12:51.115197 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-g8h2r\" (UniqueName: \"kubernetes.io/projected/bb817663-1fe7-4c73-803b-5e64dc4ba616-kube-api-access-g8h2r\") pod \"nova-cell1-f1c3-account-create-update-5mzt2\" (UID: \"bb817663-1fe7-4c73-803b-5e64dc4ba616\") " pod="openstack-kuttl-tests/nova-cell1-f1c3-account-create-update-5mzt2" Dec 03 12:12:51 crc kubenswrapper[5052]: I1203 12:12:51.149748 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-0bb5-account-create-update-6flhc" Dec 03 12:12:51 crc kubenswrapper[5052]: I1203 12:12:51.177027 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"0d97800f-6d5a-46ed-bcb5-4452b4ce86e8","Type":"ContainerStarted","Data":"3d331db8159cbcbfe79807aa8e190cb7755d381dfb7c94ca948f3ebf3f8ed7b5"} Dec 03 12:12:51 crc kubenswrapper[5052]: I1203 12:12:51.221906 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-f1c3-account-create-update-5mzt2" Dec 03 12:12:51 crc kubenswrapper[5052]: I1203 12:12:51.240362 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-db-create-mnlg4"] Dec 03 12:12:51 crc kubenswrapper[5052]: W1203 12:12:51.259906 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda2493bc9_910f_4f3d_9e74_cbd5721c838d.slice/crio-7f2aea7050002e0f96125b2e4965d9db95f020f44b2607d26bc1e7a83dd65628 WatchSource:0}: Error finding container 7f2aea7050002e0f96125b2e4965d9db95f020f44b2607d26bc1e7a83dd65628: Status 404 returned error can't find the container with id 7f2aea7050002e0f96125b2e4965d9db95f020f44b2607d26bc1e7a83dd65628 Dec 03 12:12:51 crc kubenswrapper[5052]: I1203 12:12:51.406465 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell0-db-create-bd44j"] Dec 03 12:12:51 crc kubenswrapper[5052]: I1203 12:12:51.479546 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-06eb-account-create-update-cbfzl"] Dec 03 12:12:51 crc kubenswrapper[5052]: I1203 12:12:51.701808 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-db-create-2mq6x"] Dec 03 12:12:51 crc kubenswrapper[5052]: W1203 12:12:51.713938 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2e0f8c25_7b0e_439d_8518_1a95a325e502.slice/crio-60497bb6527a7487263b5ad94f81ae007d6d5c47d32436f470315f0cc21d3a9a WatchSource:0}: Error finding container 60497bb6527a7487263b5ad94f81ae007d6d5c47d32436f470315f0cc21d3a9a: Status 404 returned error can't find the container with id 60497bb6527a7487263b5ad94f81ae007d6d5c47d32436f470315f0cc21d3a9a Dec 03 12:12:51 crc kubenswrapper[5052]: I1203 12:12:51.878576 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-f1c3-account-create-update-5mzt2"] Dec 03 12:12:51 crc kubenswrapper[5052]: W1203 12:12:51.936829 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod59c0dc2a_1f1d_46a1_93c1_6553bf351704.slice/crio-92cb4b09bc57d99c5d8e8c441348e496214060eef961d8b50b7c0096197315d1 WatchSource:0}: Error finding container 92cb4b09bc57d99c5d8e8c441348e496214060eef961d8b50b7c0096197315d1: Status 404 returned error can't find the container with id 92cb4b09bc57d99c5d8e8c441348e496214060eef961d8b50b7c0096197315d1 Dec 03 12:12:51 crc kubenswrapper[5052]: I1203 12:12:51.938723 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell0-0bb5-account-create-update-6flhc"] Dec 03 12:12:52 crc kubenswrapper[5052]: I1203 12:12:52.187348 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-0bb5-account-create-update-6flhc" event={"ID":"59c0dc2a-1f1d-46a1-93c1-6553bf351704","Type":"ContainerStarted","Data":"6f3d99a733e885f4083824dc0ca93f91a9d364725590a1230f9348d03355db05"} Dec 03 12:12:52 crc kubenswrapper[5052]: I1203 12:12:52.187736 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-0bb5-account-create-update-6flhc" event={"ID":"59c0dc2a-1f1d-46a1-93c1-6553bf351704","Type":"ContainerStarted","Data":"92cb4b09bc57d99c5d8e8c441348e496214060eef961d8b50b7c0096197315d1"} Dec 03 12:12:52 crc kubenswrapper[5052]: I1203 12:12:52.189386 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-db-create-2mq6x" event={"ID":"2e0f8c25-7b0e-439d-8518-1a95a325e502","Type":"ContainerStarted","Data":"734003ce7f213657dd0c54361c6ddb117b5dc376cf9d9bfc93668c52a0c7900e"} Dec 03 12:12:52 crc kubenswrapper[5052]: I1203 12:12:52.189457 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-db-create-2mq6x" event={"ID":"2e0f8c25-7b0e-439d-8518-1a95a325e502","Type":"ContainerStarted","Data":"60497bb6527a7487263b5ad94f81ae007d6d5c47d32436f470315f0cc21d3a9a"} Dec 03 12:12:52 crc kubenswrapper[5052]: I1203 12:12:52.193918 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"0d97800f-6d5a-46ed-bcb5-4452b4ce86e8","Type":"ContainerStarted","Data":"5901c5ab26c06b30d3c533d9bf510ac1a9c302730f990ead101e695d9a6054b3"} Dec 03 12:12:52 crc kubenswrapper[5052]: I1203 12:12:52.195603 5052 generic.go:334] "Generic (PLEG): container finished" podID="57bde9b6-d244-4159-840f-25e35a2f8dc6" containerID="256120295e912e4fd3dc859c0d55655b280c1f38dc2bf15611be732d0cf7213d" exitCode=0 Dec 03 12:12:52 crc kubenswrapper[5052]: I1203 12:12:52.195705 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-db-create-bd44j" event={"ID":"57bde9b6-d244-4159-840f-25e35a2f8dc6","Type":"ContainerDied","Data":"256120295e912e4fd3dc859c0d55655b280c1f38dc2bf15611be732d0cf7213d"} Dec 03 12:12:52 crc kubenswrapper[5052]: I1203 12:12:52.195753 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-db-create-bd44j" event={"ID":"57bde9b6-d244-4159-840f-25e35a2f8dc6","Type":"ContainerStarted","Data":"f37f700d60e8d3d460c5d5e87a1c34ce2571ad19a25dd5a82a4764c94967898a"} Dec 03 12:12:52 crc kubenswrapper[5052]: I1203 12:12:52.197933 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-f1c3-account-create-update-5mzt2" event={"ID":"bb817663-1fe7-4c73-803b-5e64dc4ba616","Type":"ContainerStarted","Data":"d7514d8a22f7396ecebd2be340bf89d19d446e00e42b69441a979d5db4b2df63"} Dec 03 12:12:52 crc kubenswrapper[5052]: I1203 12:12:52.197963 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-f1c3-account-create-update-5mzt2" event={"ID":"bb817663-1fe7-4c73-803b-5e64dc4ba616","Type":"ContainerStarted","Data":"1d723fc7fd5030a493770b7f397efeffd21d0e0d73d5cdb6766baa304bd298c4"} Dec 03 12:12:52 crc kubenswrapper[5052]: I1203 12:12:52.199704 5052 generic.go:334] "Generic (PLEG): container finished" podID="a2493bc9-910f-4f3d-9e74-cbd5721c838d" containerID="ec2fa1dad0e9bf5a1f5fbb88480a091dc02ea8944490375c6dd24553d51aadee" exitCode=0 Dec 03 12:12:52 crc kubenswrapper[5052]: I1203 12:12:52.199784 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-db-create-mnlg4" event={"ID":"a2493bc9-910f-4f3d-9e74-cbd5721c838d","Type":"ContainerDied","Data":"ec2fa1dad0e9bf5a1f5fbb88480a091dc02ea8944490375c6dd24553d51aadee"} Dec 03 12:12:52 crc kubenswrapper[5052]: I1203 12:12:52.199806 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-db-create-mnlg4" event={"ID":"a2493bc9-910f-4f3d-9e74-cbd5721c838d","Type":"ContainerStarted","Data":"7f2aea7050002e0f96125b2e4965d9db95f020f44b2607d26bc1e7a83dd65628"} Dec 03 12:12:52 crc kubenswrapper[5052]: I1203 12:12:52.201234 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-06eb-account-create-update-cbfzl" event={"ID":"16ff7253-a0ad-48a5-92e3-748f8053fc1a","Type":"ContainerStarted","Data":"f570e33c32f271c1641da025fe74e75e6be9f91e37adc2f545d4947ca1189d6b"} Dec 03 12:12:52 crc kubenswrapper[5052]: I1203 12:12:52.201276 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-06eb-account-create-update-cbfzl" event={"ID":"16ff7253-a0ad-48a5-92e3-748f8053fc1a","Type":"ContainerStarted","Data":"4a06079c12b093121c9ecfba357b7b8914482eb185bacaebf19e59eb12c8cee8"} Dec 03 12:12:52 crc kubenswrapper[5052]: I1203 12:12:52.208011 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-cell0-0bb5-account-create-update-6flhc" podStartSLOduration=2.207995436 podStartE2EDuration="2.207995436s" podCreationTimestamp="2025-12-03 12:12:50 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:12:52.205019711 +0000 UTC m=+3162.793261475" watchObservedRunningTime="2025-12-03 12:12:52.207995436 +0000 UTC m=+3162.796237200" Dec 03 12:12:52 crc kubenswrapper[5052]: I1203 12:12:52.262214 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-api-06eb-account-create-update-cbfzl" podStartSLOduration=2.262194913 podStartE2EDuration="2.262194913s" podCreationTimestamp="2025-12-03 12:12:50 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:12:52.242209943 +0000 UTC m=+3162.830451707" watchObservedRunningTime="2025-12-03 12:12:52.262194913 +0000 UTC m=+3162.850436677" Dec 03 12:12:52 crc kubenswrapper[5052]: I1203 12:12:52.283638 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-cell1-f1c3-account-create-update-5mzt2" podStartSLOduration=2.283612455 podStartE2EDuration="2.283612455s" podCreationTimestamp="2025-12-03 12:12:50 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:12:52.270192302 +0000 UTC m=+3162.858434066" watchObservedRunningTime="2025-12-03 12:12:52.283612455 +0000 UTC m=+3162.871854219" Dec 03 12:12:52 crc kubenswrapper[5052]: I1203 12:12:52.299452 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-cell1-db-create-2mq6x" podStartSLOduration=2.299427896 podStartE2EDuration="2.299427896s" podCreationTimestamp="2025-12-03 12:12:50 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:12:52.282237606 +0000 UTC m=+3162.870479380" watchObservedRunningTime="2025-12-03 12:12:52.299427896 +0000 UTC m=+3162.887669660" Dec 03 12:12:52 crc kubenswrapper[5052]: I1203 12:12:52.670286 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/neutron-74495c68d7-59m75" Dec 03 12:12:53 crc kubenswrapper[5052]: I1203 12:12:53.215238 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"0d97800f-6d5a-46ed-bcb5-4452b4ce86e8","Type":"ContainerStarted","Data":"458487a215cc64b6ccec6a6716dd0bbd74c0a7b8b7a88e1ee4934d149afb6e88"} Dec 03 12:12:53 crc kubenswrapper[5052]: I1203 12:12:53.217347 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-f1c3-account-create-update-5mzt2" event={"ID":"bb817663-1fe7-4c73-803b-5e64dc4ba616","Type":"ContainerDied","Data":"d7514d8a22f7396ecebd2be340bf89d19d446e00e42b69441a979d5db4b2df63"} Dec 03 12:12:53 crc kubenswrapper[5052]: I1203 12:12:53.217945 5052 generic.go:334] "Generic (PLEG): container finished" podID="bb817663-1fe7-4c73-803b-5e64dc4ba616" containerID="d7514d8a22f7396ecebd2be340bf89d19d446e00e42b69441a979d5db4b2df63" exitCode=0 Dec 03 12:12:53 crc kubenswrapper[5052]: I1203 12:12:53.221148 5052 generic.go:334] "Generic (PLEG): container finished" podID="16ff7253-a0ad-48a5-92e3-748f8053fc1a" containerID="f570e33c32f271c1641da025fe74e75e6be9f91e37adc2f545d4947ca1189d6b" exitCode=0 Dec 03 12:12:53 crc kubenswrapper[5052]: I1203 12:12:53.221459 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-06eb-account-create-update-cbfzl" event={"ID":"16ff7253-a0ad-48a5-92e3-748f8053fc1a","Type":"ContainerDied","Data":"f570e33c32f271c1641da025fe74e75e6be9f91e37adc2f545d4947ca1189d6b"} Dec 03 12:12:53 crc kubenswrapper[5052]: I1203 12:12:53.223929 5052 generic.go:334] "Generic (PLEG): container finished" podID="59c0dc2a-1f1d-46a1-93c1-6553bf351704" containerID="6f3d99a733e885f4083824dc0ca93f91a9d364725590a1230f9348d03355db05" exitCode=0 Dec 03 12:12:53 crc kubenswrapper[5052]: I1203 12:12:53.224080 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-0bb5-account-create-update-6flhc" event={"ID":"59c0dc2a-1f1d-46a1-93c1-6553bf351704","Type":"ContainerDied","Data":"6f3d99a733e885f4083824dc0ca93f91a9d364725590a1230f9348d03355db05"} Dec 03 12:12:53 crc kubenswrapper[5052]: I1203 12:12:53.229169 5052 generic.go:334] "Generic (PLEG): container finished" podID="2e0f8c25-7b0e-439d-8518-1a95a325e502" containerID="734003ce7f213657dd0c54361c6ddb117b5dc376cf9d9bfc93668c52a0c7900e" exitCode=0 Dec 03 12:12:53 crc kubenswrapper[5052]: I1203 12:12:53.229618 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-db-create-2mq6x" event={"ID":"2e0f8c25-7b0e-439d-8518-1a95a325e502","Type":"ContainerDied","Data":"734003ce7f213657dd0c54361c6ddb117b5dc376cf9d9bfc93668c52a0c7900e"} Dec 03 12:12:53 crc kubenswrapper[5052]: I1203 12:12:53.721110 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-db-create-bd44j" Dec 03 12:12:53 crc kubenswrapper[5052]: I1203 12:12:53.728497 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-db-create-mnlg4" Dec 03 12:12:53 crc kubenswrapper[5052]: I1203 12:12:53.761427 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8dsv6\" (UniqueName: \"kubernetes.io/projected/57bde9b6-d244-4159-840f-25e35a2f8dc6-kube-api-access-8dsv6\") pod \"57bde9b6-d244-4159-840f-25e35a2f8dc6\" (UID: \"57bde9b6-d244-4159-840f-25e35a2f8dc6\") " Dec 03 12:12:53 crc kubenswrapper[5052]: I1203 12:12:53.761646 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/57bde9b6-d244-4159-840f-25e35a2f8dc6-operator-scripts\") pod \"57bde9b6-d244-4159-840f-25e35a2f8dc6\" (UID: \"57bde9b6-d244-4159-840f-25e35a2f8dc6\") " Dec 03 12:12:53 crc kubenswrapper[5052]: I1203 12:12:53.766309 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/57bde9b6-d244-4159-840f-25e35a2f8dc6-kube-api-access-8dsv6" (OuterVolumeSpecName: "kube-api-access-8dsv6") pod "57bde9b6-d244-4159-840f-25e35a2f8dc6" (UID: "57bde9b6-d244-4159-840f-25e35a2f8dc6"). InnerVolumeSpecName "kube-api-access-8dsv6". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:12:53 crc kubenswrapper[5052]: I1203 12:12:53.766956 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/57bde9b6-d244-4159-840f-25e35a2f8dc6-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "57bde9b6-d244-4159-840f-25e35a2f8dc6" (UID: "57bde9b6-d244-4159-840f-25e35a2f8dc6"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:12:53 crc kubenswrapper[5052]: I1203 12:12:53.863652 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pvjmg\" (UniqueName: \"kubernetes.io/projected/a2493bc9-910f-4f3d-9e74-cbd5721c838d-kube-api-access-pvjmg\") pod \"a2493bc9-910f-4f3d-9e74-cbd5721c838d\" (UID: \"a2493bc9-910f-4f3d-9e74-cbd5721c838d\") " Dec 03 12:12:53 crc kubenswrapper[5052]: I1203 12:12:53.863809 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/a2493bc9-910f-4f3d-9e74-cbd5721c838d-operator-scripts\") pod \"a2493bc9-910f-4f3d-9e74-cbd5721c838d\" (UID: \"a2493bc9-910f-4f3d-9e74-cbd5721c838d\") " Dec 03 12:12:53 crc kubenswrapper[5052]: I1203 12:12:53.864252 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a2493bc9-910f-4f3d-9e74-cbd5721c838d-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "a2493bc9-910f-4f3d-9e74-cbd5721c838d" (UID: "a2493bc9-910f-4f3d-9e74-cbd5721c838d"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:12:53 crc kubenswrapper[5052]: I1203 12:12:53.864740 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/57bde9b6-d244-4159-840f-25e35a2f8dc6-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:12:53 crc kubenswrapper[5052]: I1203 12:12:53.864791 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/a2493bc9-910f-4f3d-9e74-cbd5721c838d-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:12:53 crc kubenswrapper[5052]: I1203 12:12:53.864835 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-8dsv6\" (UniqueName: \"kubernetes.io/projected/57bde9b6-d244-4159-840f-25e35a2f8dc6-kube-api-access-8dsv6\") on node \"crc\" DevicePath \"\"" Dec 03 12:12:53 crc kubenswrapper[5052]: I1203 12:12:53.867688 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a2493bc9-910f-4f3d-9e74-cbd5721c838d-kube-api-access-pvjmg" (OuterVolumeSpecName: "kube-api-access-pvjmg") pod "a2493bc9-910f-4f3d-9e74-cbd5721c838d" (UID: "a2493bc9-910f-4f3d-9e74-cbd5721c838d"). InnerVolumeSpecName "kube-api-access-pvjmg". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:12:53 crc kubenswrapper[5052]: I1203 12:12:53.966919 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-pvjmg\" (UniqueName: \"kubernetes.io/projected/a2493bc9-910f-4f3d-9e74-cbd5721c838d-kube-api-access-pvjmg\") on node \"crc\" DevicePath \"\"" Dec 03 12:12:54 crc kubenswrapper[5052]: I1203 12:12:54.078934 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:12:54 crc kubenswrapper[5052]: I1203 12:12:54.238914 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-db-create-bd44j" Dec 03 12:12:54 crc kubenswrapper[5052]: I1203 12:12:54.240285 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-db-create-bd44j" event={"ID":"57bde9b6-d244-4159-840f-25e35a2f8dc6","Type":"ContainerDied","Data":"f37f700d60e8d3d460c5d5e87a1c34ce2571ad19a25dd5a82a4764c94967898a"} Dec 03 12:12:54 crc kubenswrapper[5052]: I1203 12:12:54.240709 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-db-create-mnlg4" Dec 03 12:12:54 crc kubenswrapper[5052]: I1203 12:12:54.240991 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f37f700d60e8d3d460c5d5e87a1c34ce2571ad19a25dd5a82a4764c94967898a" Dec 03 12:12:54 crc kubenswrapper[5052]: I1203 12:12:54.241023 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-db-create-mnlg4" event={"ID":"a2493bc9-910f-4f3d-9e74-cbd5721c838d","Type":"ContainerDied","Data":"7f2aea7050002e0f96125b2e4965d9db95f020f44b2607d26bc1e7a83dd65628"} Dec 03 12:12:54 crc kubenswrapper[5052]: I1203 12:12:54.241067 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7f2aea7050002e0f96125b2e4965d9db95f020f44b2607d26bc1e7a83dd65628" Dec 03 12:12:54 crc kubenswrapper[5052]: I1203 12:12:54.246556 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"0d97800f-6d5a-46ed-bcb5-4452b4ce86e8","Type":"ContainerStarted","Data":"d88a6d8aff5c3b9c680401e2f8105107db4a2ec6862732281efba40f34160fc7"} Dec 03 12:12:54 crc kubenswrapper[5052]: I1203 12:12:54.247137 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="0d97800f-6d5a-46ed-bcb5-4452b4ce86e8" containerName="ceilometer-central-agent" containerID="cri-o://3d331db8159cbcbfe79807aa8e190cb7755d381dfb7c94ca948f3ebf3f8ed7b5" gracePeriod=30 Dec 03 12:12:54 crc kubenswrapper[5052]: I1203 12:12:54.248197 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="0d97800f-6d5a-46ed-bcb5-4452b4ce86e8" containerName="sg-core" containerID="cri-o://458487a215cc64b6ccec6a6716dd0bbd74c0a7b8b7a88e1ee4934d149afb6e88" gracePeriod=30 Dec 03 12:12:54 crc kubenswrapper[5052]: I1203 12:12:54.248280 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="0d97800f-6d5a-46ed-bcb5-4452b4ce86e8" containerName="proxy-httpd" containerID="cri-o://d88a6d8aff5c3b9c680401e2f8105107db4a2ec6862732281efba40f34160fc7" gracePeriod=30 Dec 03 12:12:54 crc kubenswrapper[5052]: I1203 12:12:54.248322 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="0d97800f-6d5a-46ed-bcb5-4452b4ce86e8" containerName="ceilometer-notification-agent" containerID="cri-o://5901c5ab26c06b30d3c533d9bf510ac1a9c302730f990ead101e695d9a6054b3" gracePeriod=30 Dec 03 12:12:54 crc kubenswrapper[5052]: I1203 12:12:54.286034 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/ceilometer-0" podStartSLOduration=1.3585708890000001 podStartE2EDuration="5.286018639s" podCreationTimestamp="2025-12-03 12:12:49 +0000 UTC" firstStartedPulling="2025-12-03 12:12:50.034098884 +0000 UTC m=+3160.622340688" lastFinishedPulling="2025-12-03 12:12:53.961546674 +0000 UTC m=+3164.549788438" observedRunningTime="2025-12-03 12:12:54.275525869 +0000 UTC m=+3164.863767633" watchObservedRunningTime="2025-12-03 12:12:54.286018639 +0000 UTC m=+3164.874260403" Dec 03 12:12:54 crc kubenswrapper[5052]: I1203 12:12:54.508141 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:12:54 crc kubenswrapper[5052]: I1203 12:12:54.508172 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:12:54 crc kubenswrapper[5052]: I1203 12:12:54.526938 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-0bb5-account-create-update-6flhc" Dec 03 12:12:54 crc kubenswrapper[5052]: I1203 12:12:54.573639 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:12:54 crc kubenswrapper[5052]: I1203 12:12:54.574808 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:12:54 crc kubenswrapper[5052]: I1203 12:12:54.578807 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/59c0dc2a-1f1d-46a1-93c1-6553bf351704-operator-scripts\") pod \"59c0dc2a-1f1d-46a1-93c1-6553bf351704\" (UID: \"59c0dc2a-1f1d-46a1-93c1-6553bf351704\") " Dec 03 12:12:54 crc kubenswrapper[5052]: I1203 12:12:54.578992 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6s8v9\" (UniqueName: \"kubernetes.io/projected/59c0dc2a-1f1d-46a1-93c1-6553bf351704-kube-api-access-6s8v9\") pod \"59c0dc2a-1f1d-46a1-93c1-6553bf351704\" (UID: \"59c0dc2a-1f1d-46a1-93c1-6553bf351704\") " Dec 03 12:12:54 crc kubenswrapper[5052]: I1203 12:12:54.581623 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/59c0dc2a-1f1d-46a1-93c1-6553bf351704-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "59c0dc2a-1f1d-46a1-93c1-6553bf351704" (UID: "59c0dc2a-1f1d-46a1-93c1-6553bf351704"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:12:54 crc kubenswrapper[5052]: I1203 12:12:54.595735 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/59c0dc2a-1f1d-46a1-93c1-6553bf351704-kube-api-access-6s8v9" (OuterVolumeSpecName: "kube-api-access-6s8v9") pod "59c0dc2a-1f1d-46a1-93c1-6553bf351704" (UID: "59c0dc2a-1f1d-46a1-93c1-6553bf351704"). InnerVolumeSpecName "kube-api-access-6s8v9". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:12:54 crc kubenswrapper[5052]: I1203 12:12:54.686484 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-6s8v9\" (UniqueName: \"kubernetes.io/projected/59c0dc2a-1f1d-46a1-93c1-6553bf351704-kube-api-access-6s8v9\") on node \"crc\" DevicePath \"\"" Dec 03 12:12:54 crc kubenswrapper[5052]: I1203 12:12:54.686840 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/59c0dc2a-1f1d-46a1-93c1-6553bf351704-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:12:54 crc kubenswrapper[5052]: I1203 12:12:54.900787 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-f1c3-account-create-update-5mzt2" Dec 03 12:12:54 crc kubenswrapper[5052]: I1203 12:12:54.938652 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-db-create-2mq6x" Dec 03 12:12:54 crc kubenswrapper[5052]: I1203 12:12:54.972450 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-06eb-account-create-update-cbfzl" Dec 03 12:12:54 crc kubenswrapper[5052]: I1203 12:12:54.994727 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/2e0f8c25-7b0e-439d-8518-1a95a325e502-operator-scripts\") pod \"2e0f8c25-7b0e-439d-8518-1a95a325e502\" (UID: \"2e0f8c25-7b0e-439d-8518-1a95a325e502\") " Dec 03 12:12:54 crc kubenswrapper[5052]: I1203 12:12:54.994857 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g8h2r\" (UniqueName: \"kubernetes.io/projected/bb817663-1fe7-4c73-803b-5e64dc4ba616-kube-api-access-g8h2r\") pod \"bb817663-1fe7-4c73-803b-5e64dc4ba616\" (UID: \"bb817663-1fe7-4c73-803b-5e64dc4ba616\") " Dec 03 12:12:54 crc kubenswrapper[5052]: I1203 12:12:54.994894 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/bb817663-1fe7-4c73-803b-5e64dc4ba616-operator-scripts\") pod \"bb817663-1fe7-4c73-803b-5e64dc4ba616\" (UID: \"bb817663-1fe7-4c73-803b-5e64dc4ba616\") " Dec 03 12:12:54 crc kubenswrapper[5052]: I1203 12:12:54.995011 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zckk8\" (UniqueName: \"kubernetes.io/projected/2e0f8c25-7b0e-439d-8518-1a95a325e502-kube-api-access-zckk8\") pod \"2e0f8c25-7b0e-439d-8518-1a95a325e502\" (UID: \"2e0f8c25-7b0e-439d-8518-1a95a325e502\") " Dec 03 12:12:54 crc kubenswrapper[5052]: I1203 12:12:54.998972 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2e0f8c25-7b0e-439d-8518-1a95a325e502-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "2e0f8c25-7b0e-439d-8518-1a95a325e502" (UID: "2e0f8c25-7b0e-439d-8518-1a95a325e502"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:12:54 crc kubenswrapper[5052]: I1203 12:12:54.999050 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bb817663-1fe7-4c73-803b-5e64dc4ba616-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "bb817663-1fe7-4c73-803b-5e64dc4ba616" (UID: "bb817663-1fe7-4c73-803b-5e64dc4ba616"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:12:54 crc kubenswrapper[5052]: I1203 12:12:54.999965 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2e0f8c25-7b0e-439d-8518-1a95a325e502-kube-api-access-zckk8" (OuterVolumeSpecName: "kube-api-access-zckk8") pod "2e0f8c25-7b0e-439d-8518-1a95a325e502" (UID: "2e0f8c25-7b0e-439d-8518-1a95a325e502"). InnerVolumeSpecName "kube-api-access-zckk8". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:12:55 crc kubenswrapper[5052]: I1203 12:12:55.000356 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bb817663-1fe7-4c73-803b-5e64dc4ba616-kube-api-access-g8h2r" (OuterVolumeSpecName: "kube-api-access-g8h2r") pod "bb817663-1fe7-4c73-803b-5e64dc4ba616" (UID: "bb817663-1fe7-4c73-803b-5e64dc4ba616"). InnerVolumeSpecName "kube-api-access-g8h2r". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:12:55 crc kubenswrapper[5052]: I1203 12:12:55.056000 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:12:55 crc kubenswrapper[5052]: I1203 12:12:55.096493 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/0d97800f-6d5a-46ed-bcb5-4452b4ce86e8-config-data\") pod \"0d97800f-6d5a-46ed-bcb5-4452b4ce86e8\" (UID: \"0d97800f-6d5a-46ed-bcb5-4452b4ce86e8\") " Dec 03 12:12:55 crc kubenswrapper[5052]: I1203 12:12:55.096631 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c777f\" (UniqueName: \"kubernetes.io/projected/0d97800f-6d5a-46ed-bcb5-4452b4ce86e8-kube-api-access-c777f\") pod \"0d97800f-6d5a-46ed-bcb5-4452b4ce86e8\" (UID: \"0d97800f-6d5a-46ed-bcb5-4452b4ce86e8\") " Dec 03 12:12:55 crc kubenswrapper[5052]: I1203 12:12:55.096725 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/0d97800f-6d5a-46ed-bcb5-4452b4ce86e8-scripts\") pod \"0d97800f-6d5a-46ed-bcb5-4452b4ce86e8\" (UID: \"0d97800f-6d5a-46ed-bcb5-4452b4ce86e8\") " Dec 03 12:12:55 crc kubenswrapper[5052]: I1203 12:12:55.096761 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0d97800f-6d5a-46ed-bcb5-4452b4ce86e8-combined-ca-bundle\") pod \"0d97800f-6d5a-46ed-bcb5-4452b4ce86e8\" (UID: \"0d97800f-6d5a-46ed-bcb5-4452b4ce86e8\") " Dec 03 12:12:55 crc kubenswrapper[5052]: I1203 12:12:55.096864 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/16ff7253-a0ad-48a5-92e3-748f8053fc1a-operator-scripts\") pod \"16ff7253-a0ad-48a5-92e3-748f8053fc1a\" (UID: \"16ff7253-a0ad-48a5-92e3-748f8053fc1a\") " Dec 03 12:12:55 crc kubenswrapper[5052]: I1203 12:12:55.096887 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fhjhz\" (UniqueName: \"kubernetes.io/projected/16ff7253-a0ad-48a5-92e3-748f8053fc1a-kube-api-access-fhjhz\") pod \"16ff7253-a0ad-48a5-92e3-748f8053fc1a\" (UID: \"16ff7253-a0ad-48a5-92e3-748f8053fc1a\") " Dec 03 12:12:55 crc kubenswrapper[5052]: I1203 12:12:55.096910 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/0d97800f-6d5a-46ed-bcb5-4452b4ce86e8-run-httpd\") pod \"0d97800f-6d5a-46ed-bcb5-4452b4ce86e8\" (UID: \"0d97800f-6d5a-46ed-bcb5-4452b4ce86e8\") " Dec 03 12:12:55 crc kubenswrapper[5052]: I1203 12:12:55.096961 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/0d97800f-6d5a-46ed-bcb5-4452b4ce86e8-log-httpd\") pod \"0d97800f-6d5a-46ed-bcb5-4452b4ce86e8\" (UID: \"0d97800f-6d5a-46ed-bcb5-4452b4ce86e8\") " Dec 03 12:12:55 crc kubenswrapper[5052]: I1203 12:12:55.096993 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/0d97800f-6d5a-46ed-bcb5-4452b4ce86e8-sg-core-conf-yaml\") pod \"0d97800f-6d5a-46ed-bcb5-4452b4ce86e8\" (UID: \"0d97800f-6d5a-46ed-bcb5-4452b4ce86e8\") " Dec 03 12:12:55 crc kubenswrapper[5052]: I1203 12:12:55.097280 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d97800f-6d5a-46ed-bcb5-4452b4ce86e8-run-httpd" (OuterVolumeSpecName: "run-httpd") pod "0d97800f-6d5a-46ed-bcb5-4452b4ce86e8" (UID: "0d97800f-6d5a-46ed-bcb5-4452b4ce86e8"). InnerVolumeSpecName "run-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:12:55 crc kubenswrapper[5052]: I1203 12:12:55.097347 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/16ff7253-a0ad-48a5-92e3-748f8053fc1a-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "16ff7253-a0ad-48a5-92e3-748f8053fc1a" (UID: "16ff7253-a0ad-48a5-92e3-748f8053fc1a"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:12:55 crc kubenswrapper[5052]: I1203 12:12:55.097712 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/bb817663-1fe7-4c73-803b-5e64dc4ba616-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:12:55 crc kubenswrapper[5052]: I1203 12:12:55.097732 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/16ff7253-a0ad-48a5-92e3-748f8053fc1a-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:12:55 crc kubenswrapper[5052]: I1203 12:12:55.097744 5052 reconciler_common.go:293] "Volume detached for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/0d97800f-6d5a-46ed-bcb5-4452b4ce86e8-run-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:12:55 crc kubenswrapper[5052]: I1203 12:12:55.097755 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-zckk8\" (UniqueName: \"kubernetes.io/projected/2e0f8c25-7b0e-439d-8518-1a95a325e502-kube-api-access-zckk8\") on node \"crc\" DevicePath \"\"" Dec 03 12:12:55 crc kubenswrapper[5052]: I1203 12:12:55.097769 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/2e0f8c25-7b0e-439d-8518-1a95a325e502-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:12:55 crc kubenswrapper[5052]: I1203 12:12:55.097780 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-g8h2r\" (UniqueName: \"kubernetes.io/projected/bb817663-1fe7-4c73-803b-5e64dc4ba616-kube-api-access-g8h2r\") on node \"crc\" DevicePath \"\"" Dec 03 12:12:55 crc kubenswrapper[5052]: I1203 12:12:55.098415 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d97800f-6d5a-46ed-bcb5-4452b4ce86e8-log-httpd" (OuterVolumeSpecName: "log-httpd") pod "0d97800f-6d5a-46ed-bcb5-4452b4ce86e8" (UID: "0d97800f-6d5a-46ed-bcb5-4452b4ce86e8"). InnerVolumeSpecName "log-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:12:55 crc kubenswrapper[5052]: I1203 12:12:55.100393 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0d97800f-6d5a-46ed-bcb5-4452b4ce86e8-kube-api-access-c777f" (OuterVolumeSpecName: "kube-api-access-c777f") pod "0d97800f-6d5a-46ed-bcb5-4452b4ce86e8" (UID: "0d97800f-6d5a-46ed-bcb5-4452b4ce86e8"). InnerVolumeSpecName "kube-api-access-c777f". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:12:55 crc kubenswrapper[5052]: I1203 12:12:55.100700 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0d97800f-6d5a-46ed-bcb5-4452b4ce86e8-scripts" (OuterVolumeSpecName: "scripts") pod "0d97800f-6d5a-46ed-bcb5-4452b4ce86e8" (UID: "0d97800f-6d5a-46ed-bcb5-4452b4ce86e8"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:12:55 crc kubenswrapper[5052]: I1203 12:12:55.102302 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/16ff7253-a0ad-48a5-92e3-748f8053fc1a-kube-api-access-fhjhz" (OuterVolumeSpecName: "kube-api-access-fhjhz") pod "16ff7253-a0ad-48a5-92e3-748f8053fc1a" (UID: "16ff7253-a0ad-48a5-92e3-748f8053fc1a"). InnerVolumeSpecName "kube-api-access-fhjhz". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:12:55 crc kubenswrapper[5052]: I1203 12:12:55.123313 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0d97800f-6d5a-46ed-bcb5-4452b4ce86e8-sg-core-conf-yaml" (OuterVolumeSpecName: "sg-core-conf-yaml") pod "0d97800f-6d5a-46ed-bcb5-4452b4ce86e8" (UID: "0d97800f-6d5a-46ed-bcb5-4452b4ce86e8"). InnerVolumeSpecName "sg-core-conf-yaml". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:12:55 crc kubenswrapper[5052]: I1203 12:12:55.177297 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0d97800f-6d5a-46ed-bcb5-4452b4ce86e8-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "0d97800f-6d5a-46ed-bcb5-4452b4ce86e8" (UID: "0d97800f-6d5a-46ed-bcb5-4452b4ce86e8"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:12:55 crc kubenswrapper[5052]: I1203 12:12:55.202475 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-c777f\" (UniqueName: \"kubernetes.io/projected/0d97800f-6d5a-46ed-bcb5-4452b4ce86e8-kube-api-access-c777f\") on node \"crc\" DevicePath \"\"" Dec 03 12:12:55 crc kubenswrapper[5052]: I1203 12:12:55.202539 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/0d97800f-6d5a-46ed-bcb5-4452b4ce86e8-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:12:55 crc kubenswrapper[5052]: I1203 12:12:55.202554 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0d97800f-6d5a-46ed-bcb5-4452b4ce86e8-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:12:55 crc kubenswrapper[5052]: I1203 12:12:55.202568 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-fhjhz\" (UniqueName: \"kubernetes.io/projected/16ff7253-a0ad-48a5-92e3-748f8053fc1a-kube-api-access-fhjhz\") on node \"crc\" DevicePath \"\"" Dec 03 12:12:55 crc kubenswrapper[5052]: I1203 12:12:55.202580 5052 reconciler_common.go:293] "Volume detached for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/0d97800f-6d5a-46ed-bcb5-4452b4ce86e8-log-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:12:55 crc kubenswrapper[5052]: I1203 12:12:55.202598 5052 reconciler_common.go:293] "Volume detached for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/0d97800f-6d5a-46ed-bcb5-4452b4ce86e8-sg-core-conf-yaml\") on node \"crc\" DevicePath \"\"" Dec 03 12:12:55 crc kubenswrapper[5052]: I1203 12:12:55.213761 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0d97800f-6d5a-46ed-bcb5-4452b4ce86e8-config-data" (OuterVolumeSpecName: "config-data") pod "0d97800f-6d5a-46ed-bcb5-4452b4ce86e8" (UID: "0d97800f-6d5a-46ed-bcb5-4452b4ce86e8"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:12:55 crc kubenswrapper[5052]: I1203 12:12:55.268175 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-06eb-account-create-update-cbfzl" event={"ID":"16ff7253-a0ad-48a5-92e3-748f8053fc1a","Type":"ContainerDied","Data":"4a06079c12b093121c9ecfba357b7b8914482eb185bacaebf19e59eb12c8cee8"} Dec 03 12:12:55 crc kubenswrapper[5052]: I1203 12:12:55.269369 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4a06079c12b093121c9ecfba357b7b8914482eb185bacaebf19e59eb12c8cee8" Dec 03 12:12:55 crc kubenswrapper[5052]: I1203 12:12:55.269481 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-06eb-account-create-update-cbfzl" Dec 03 12:12:55 crc kubenswrapper[5052]: I1203 12:12:55.283553 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-0bb5-account-create-update-6flhc" event={"ID":"59c0dc2a-1f1d-46a1-93c1-6553bf351704","Type":"ContainerDied","Data":"92cb4b09bc57d99c5d8e8c441348e496214060eef961d8b50b7c0096197315d1"} Dec 03 12:12:55 crc kubenswrapper[5052]: I1203 12:12:55.283998 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="92cb4b09bc57d99c5d8e8c441348e496214060eef961d8b50b7c0096197315d1" Dec 03 12:12:55 crc kubenswrapper[5052]: I1203 12:12:55.283610 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-0bb5-account-create-update-6flhc" Dec 03 12:12:55 crc kubenswrapper[5052]: I1203 12:12:55.286087 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-db-create-2mq6x" event={"ID":"2e0f8c25-7b0e-439d-8518-1a95a325e502","Type":"ContainerDied","Data":"60497bb6527a7487263b5ad94f81ae007d6d5c47d32436f470315f0cc21d3a9a"} Dec 03 12:12:55 crc kubenswrapper[5052]: I1203 12:12:55.286132 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="60497bb6527a7487263b5ad94f81ae007d6d5c47d32436f470315f0cc21d3a9a" Dec 03 12:12:55 crc kubenswrapper[5052]: I1203 12:12:55.286203 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-db-create-2mq6x" Dec 03 12:12:55 crc kubenswrapper[5052]: I1203 12:12:55.289935 5052 generic.go:334] "Generic (PLEG): container finished" podID="0d97800f-6d5a-46ed-bcb5-4452b4ce86e8" containerID="d88a6d8aff5c3b9c680401e2f8105107db4a2ec6862732281efba40f34160fc7" exitCode=0 Dec 03 12:12:55 crc kubenswrapper[5052]: I1203 12:12:55.289966 5052 generic.go:334] "Generic (PLEG): container finished" podID="0d97800f-6d5a-46ed-bcb5-4452b4ce86e8" containerID="458487a215cc64b6ccec6a6716dd0bbd74c0a7b8b7a88e1ee4934d149afb6e88" exitCode=2 Dec 03 12:12:55 crc kubenswrapper[5052]: I1203 12:12:55.289974 5052 generic.go:334] "Generic (PLEG): container finished" podID="0d97800f-6d5a-46ed-bcb5-4452b4ce86e8" containerID="5901c5ab26c06b30d3c533d9bf510ac1a9c302730f990ead101e695d9a6054b3" exitCode=0 Dec 03 12:12:55 crc kubenswrapper[5052]: I1203 12:12:55.289981 5052 generic.go:334] "Generic (PLEG): container finished" podID="0d97800f-6d5a-46ed-bcb5-4452b4ce86e8" containerID="3d331db8159cbcbfe79807aa8e190cb7755d381dfb7c94ca948f3ebf3f8ed7b5" exitCode=0 Dec 03 12:12:55 crc kubenswrapper[5052]: I1203 12:12:55.290032 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:12:55 crc kubenswrapper[5052]: I1203 12:12:55.290024 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"0d97800f-6d5a-46ed-bcb5-4452b4ce86e8","Type":"ContainerDied","Data":"d88a6d8aff5c3b9c680401e2f8105107db4a2ec6862732281efba40f34160fc7"} Dec 03 12:12:55 crc kubenswrapper[5052]: I1203 12:12:55.290195 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"0d97800f-6d5a-46ed-bcb5-4452b4ce86e8","Type":"ContainerDied","Data":"458487a215cc64b6ccec6a6716dd0bbd74c0a7b8b7a88e1ee4934d149afb6e88"} Dec 03 12:12:55 crc kubenswrapper[5052]: I1203 12:12:55.290224 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"0d97800f-6d5a-46ed-bcb5-4452b4ce86e8","Type":"ContainerDied","Data":"5901c5ab26c06b30d3c533d9bf510ac1a9c302730f990ead101e695d9a6054b3"} Dec 03 12:12:55 crc kubenswrapper[5052]: I1203 12:12:55.290238 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"0d97800f-6d5a-46ed-bcb5-4452b4ce86e8","Type":"ContainerDied","Data":"3d331db8159cbcbfe79807aa8e190cb7755d381dfb7c94ca948f3ebf3f8ed7b5"} Dec 03 12:12:55 crc kubenswrapper[5052]: I1203 12:12:55.290251 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"0d97800f-6d5a-46ed-bcb5-4452b4ce86e8","Type":"ContainerDied","Data":"998284bedfe3195b513f8dc5ec980921f3c637c456e6968e528812b414e9cd81"} Dec 03 12:12:55 crc kubenswrapper[5052]: I1203 12:12:55.290273 5052 scope.go:117] "RemoveContainer" containerID="d88a6d8aff5c3b9c680401e2f8105107db4a2ec6862732281efba40f34160fc7" Dec 03 12:12:55 crc kubenswrapper[5052]: I1203 12:12:55.292833 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-f1c3-account-create-update-5mzt2" Dec 03 12:12:55 crc kubenswrapper[5052]: I1203 12:12:55.292861 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-f1c3-account-create-update-5mzt2" event={"ID":"bb817663-1fe7-4c73-803b-5e64dc4ba616","Type":"ContainerDied","Data":"1d723fc7fd5030a493770b7f397efeffd21d0e0d73d5cdb6766baa304bd298c4"} Dec 03 12:12:55 crc kubenswrapper[5052]: I1203 12:12:55.292897 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1d723fc7fd5030a493770b7f397efeffd21d0e0d73d5cdb6766baa304bd298c4" Dec 03 12:12:55 crc kubenswrapper[5052]: I1203 12:12:55.293040 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:12:55 crc kubenswrapper[5052]: I1203 12:12:55.293075 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:12:55 crc kubenswrapper[5052]: I1203 12:12:55.303876 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/0d97800f-6d5a-46ed-bcb5-4452b4ce86e8-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:12:55 crc kubenswrapper[5052]: I1203 12:12:55.310436 5052 scope.go:117] "RemoveContainer" containerID="458487a215cc64b6ccec6a6716dd0bbd74c0a7b8b7a88e1ee4934d149afb6e88" Dec 03 12:12:55 crc kubenswrapper[5052]: I1203 12:12:55.313266 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/neutron-5b979cdcb6-6hqqv" Dec 03 12:12:55 crc kubenswrapper[5052]: I1203 12:12:55.337123 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:12:55 crc kubenswrapper[5052]: I1203 12:12:55.349875 5052 scope.go:117] "RemoveContainer" containerID="5901c5ab26c06b30d3c533d9bf510ac1a9c302730f990ead101e695d9a6054b3" Dec 03 12:12:55 crc kubenswrapper[5052]: I1203 12:12:55.359884 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:12:55 crc kubenswrapper[5052]: I1203 12:12:55.371440 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:12:55 crc kubenswrapper[5052]: E1203 12:12:55.372073 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="bb817663-1fe7-4c73-803b-5e64dc4ba616" containerName="mariadb-account-create-update" Dec 03 12:12:55 crc kubenswrapper[5052]: I1203 12:12:55.372153 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb817663-1fe7-4c73-803b-5e64dc4ba616" containerName="mariadb-account-create-update" Dec 03 12:12:55 crc kubenswrapper[5052]: E1203 12:12:55.372214 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="57bde9b6-d244-4159-840f-25e35a2f8dc6" containerName="mariadb-database-create" Dec 03 12:12:55 crc kubenswrapper[5052]: I1203 12:12:55.372265 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="57bde9b6-d244-4159-840f-25e35a2f8dc6" containerName="mariadb-database-create" Dec 03 12:12:55 crc kubenswrapper[5052]: E1203 12:12:55.372352 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="0d97800f-6d5a-46ed-bcb5-4452b4ce86e8" containerName="ceilometer-notification-agent" Dec 03 12:12:55 crc kubenswrapper[5052]: I1203 12:12:55.372406 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="0d97800f-6d5a-46ed-bcb5-4452b4ce86e8" containerName="ceilometer-notification-agent" Dec 03 12:12:55 crc kubenswrapper[5052]: E1203 12:12:55.372481 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="0d97800f-6d5a-46ed-bcb5-4452b4ce86e8" containerName="sg-core" Dec 03 12:12:55 crc kubenswrapper[5052]: I1203 12:12:55.372589 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="0d97800f-6d5a-46ed-bcb5-4452b4ce86e8" containerName="sg-core" Dec 03 12:12:55 crc kubenswrapper[5052]: E1203 12:12:55.372677 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="59c0dc2a-1f1d-46a1-93c1-6553bf351704" containerName="mariadb-account-create-update" Dec 03 12:12:55 crc kubenswrapper[5052]: I1203 12:12:55.372733 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="59c0dc2a-1f1d-46a1-93c1-6553bf351704" containerName="mariadb-account-create-update" Dec 03 12:12:55 crc kubenswrapper[5052]: E1203 12:12:55.372801 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="16ff7253-a0ad-48a5-92e3-748f8053fc1a" containerName="mariadb-account-create-update" Dec 03 12:12:55 crc kubenswrapper[5052]: I1203 12:12:55.372857 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="16ff7253-a0ad-48a5-92e3-748f8053fc1a" containerName="mariadb-account-create-update" Dec 03 12:12:55 crc kubenswrapper[5052]: E1203 12:12:55.372932 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="0d97800f-6d5a-46ed-bcb5-4452b4ce86e8" containerName="ceilometer-central-agent" Dec 03 12:12:55 crc kubenswrapper[5052]: I1203 12:12:55.372993 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="0d97800f-6d5a-46ed-bcb5-4452b4ce86e8" containerName="ceilometer-central-agent" Dec 03 12:12:55 crc kubenswrapper[5052]: E1203 12:12:55.373065 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="a2493bc9-910f-4f3d-9e74-cbd5721c838d" containerName="mariadb-database-create" Dec 03 12:12:55 crc kubenswrapper[5052]: I1203 12:12:55.373117 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="a2493bc9-910f-4f3d-9e74-cbd5721c838d" containerName="mariadb-database-create" Dec 03 12:12:55 crc kubenswrapper[5052]: E1203 12:12:55.391704 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="2e0f8c25-7b0e-439d-8518-1a95a325e502" containerName="mariadb-database-create" Dec 03 12:12:55 crc kubenswrapper[5052]: I1203 12:12:55.404637 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="2e0f8c25-7b0e-439d-8518-1a95a325e502" containerName="mariadb-database-create" Dec 03 12:12:55 crc kubenswrapper[5052]: E1203 12:12:55.405054 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="0d97800f-6d5a-46ed-bcb5-4452b4ce86e8" containerName="proxy-httpd" Dec 03 12:12:55 crc kubenswrapper[5052]: I1203 12:12:55.405151 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="0d97800f-6d5a-46ed-bcb5-4452b4ce86e8" containerName="proxy-httpd" Dec 03 12:12:55 crc kubenswrapper[5052]: I1203 12:12:55.405681 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="2e0f8c25-7b0e-439d-8518-1a95a325e502" containerName="mariadb-database-create" Dec 03 12:12:55 crc kubenswrapper[5052]: I1203 12:12:55.405766 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="bb817663-1fe7-4c73-803b-5e64dc4ba616" containerName="mariadb-account-create-update" Dec 03 12:12:55 crc kubenswrapper[5052]: I1203 12:12:55.405855 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="59c0dc2a-1f1d-46a1-93c1-6553bf351704" containerName="mariadb-account-create-update" Dec 03 12:12:55 crc kubenswrapper[5052]: I1203 12:12:55.406034 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="57bde9b6-d244-4159-840f-25e35a2f8dc6" containerName="mariadb-database-create" Dec 03 12:12:55 crc kubenswrapper[5052]: I1203 12:12:55.406096 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="16ff7253-a0ad-48a5-92e3-748f8053fc1a" containerName="mariadb-account-create-update" Dec 03 12:12:55 crc kubenswrapper[5052]: I1203 12:12:55.406160 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="0d97800f-6d5a-46ed-bcb5-4452b4ce86e8" containerName="sg-core" Dec 03 12:12:55 crc kubenswrapper[5052]: I1203 12:12:55.406223 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="a2493bc9-910f-4f3d-9e74-cbd5721c838d" containerName="mariadb-database-create" Dec 03 12:12:55 crc kubenswrapper[5052]: I1203 12:12:55.406286 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="0d97800f-6d5a-46ed-bcb5-4452b4ce86e8" containerName="proxy-httpd" Dec 03 12:12:55 crc kubenswrapper[5052]: I1203 12:12:55.406380 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="0d97800f-6d5a-46ed-bcb5-4452b4ce86e8" containerName="ceilometer-notification-agent" Dec 03 12:12:55 crc kubenswrapper[5052]: I1203 12:12:55.406454 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="0d97800f-6d5a-46ed-bcb5-4452b4ce86e8" containerName="ceilometer-central-agent" Dec 03 12:12:55 crc kubenswrapper[5052]: I1203 12:12:55.409447 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:12:55 crc kubenswrapper[5052]: I1203 12:12:55.409745 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:12:55 crc kubenswrapper[5052]: I1203 12:12:55.415144 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-config-data" Dec 03 12:12:55 crc kubenswrapper[5052]: I1203 12:12:55.416103 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-scripts" Dec 03 12:12:55 crc kubenswrapper[5052]: I1203 12:12:55.428923 5052 scope.go:117] "RemoveContainer" containerID="3d331db8159cbcbfe79807aa8e190cb7755d381dfb7c94ca948f3ebf3f8ed7b5" Dec 03 12:12:55 crc kubenswrapper[5052]: I1203 12:12:55.441164 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/neutron-74495c68d7-59m75"] Dec 03 12:12:55 crc kubenswrapper[5052]: I1203 12:12:55.441502 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/neutron-74495c68d7-59m75" podUID="49095880-d707-4afb-a3cc-a10ece6970de" containerName="neutron-api" containerID="cri-o://e5291229716bb69e07dd59d25b94079d3c162d2e091119be3de021794e666c7c" gracePeriod=30 Dec 03 12:12:55 crc kubenswrapper[5052]: I1203 12:12:55.442253 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/neutron-74495c68d7-59m75" podUID="49095880-d707-4afb-a3cc-a10ece6970de" containerName="neutron-httpd" containerID="cri-o://f94cbc489fbc2c2f8115d1902907b2ba57a8926f2d1730a50bf76e99d2e355cf" gracePeriod=30 Dec 03 12:12:55 crc kubenswrapper[5052]: I1203 12:12:55.476921 5052 scope.go:117] "RemoveContainer" containerID="d88a6d8aff5c3b9c680401e2f8105107db4a2ec6862732281efba40f34160fc7" Dec 03 12:12:55 crc kubenswrapper[5052]: E1203 12:12:55.477435 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d88a6d8aff5c3b9c680401e2f8105107db4a2ec6862732281efba40f34160fc7\": container with ID starting with d88a6d8aff5c3b9c680401e2f8105107db4a2ec6862732281efba40f34160fc7 not found: ID does not exist" containerID="d88a6d8aff5c3b9c680401e2f8105107db4a2ec6862732281efba40f34160fc7" Dec 03 12:12:55 crc kubenswrapper[5052]: I1203 12:12:55.477463 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d88a6d8aff5c3b9c680401e2f8105107db4a2ec6862732281efba40f34160fc7"} err="failed to get container status \"d88a6d8aff5c3b9c680401e2f8105107db4a2ec6862732281efba40f34160fc7\": rpc error: code = NotFound desc = could not find container \"d88a6d8aff5c3b9c680401e2f8105107db4a2ec6862732281efba40f34160fc7\": container with ID starting with d88a6d8aff5c3b9c680401e2f8105107db4a2ec6862732281efba40f34160fc7 not found: ID does not exist" Dec 03 12:12:55 crc kubenswrapper[5052]: I1203 12:12:55.477484 5052 scope.go:117] "RemoveContainer" containerID="458487a215cc64b6ccec6a6716dd0bbd74c0a7b8b7a88e1ee4934d149afb6e88" Dec 03 12:12:55 crc kubenswrapper[5052]: E1203 12:12:55.478132 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"458487a215cc64b6ccec6a6716dd0bbd74c0a7b8b7a88e1ee4934d149afb6e88\": container with ID starting with 458487a215cc64b6ccec6a6716dd0bbd74c0a7b8b7a88e1ee4934d149afb6e88 not found: ID does not exist" containerID="458487a215cc64b6ccec6a6716dd0bbd74c0a7b8b7a88e1ee4934d149afb6e88" Dec 03 12:12:55 crc kubenswrapper[5052]: I1203 12:12:55.478196 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"458487a215cc64b6ccec6a6716dd0bbd74c0a7b8b7a88e1ee4934d149afb6e88"} err="failed to get container status \"458487a215cc64b6ccec6a6716dd0bbd74c0a7b8b7a88e1ee4934d149afb6e88\": rpc error: code = NotFound desc = could not find container \"458487a215cc64b6ccec6a6716dd0bbd74c0a7b8b7a88e1ee4934d149afb6e88\": container with ID starting with 458487a215cc64b6ccec6a6716dd0bbd74c0a7b8b7a88e1ee4934d149afb6e88 not found: ID does not exist" Dec 03 12:12:55 crc kubenswrapper[5052]: I1203 12:12:55.478230 5052 scope.go:117] "RemoveContainer" containerID="5901c5ab26c06b30d3c533d9bf510ac1a9c302730f990ead101e695d9a6054b3" Dec 03 12:12:55 crc kubenswrapper[5052]: E1203 12:12:55.478602 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5901c5ab26c06b30d3c533d9bf510ac1a9c302730f990ead101e695d9a6054b3\": container with ID starting with 5901c5ab26c06b30d3c533d9bf510ac1a9c302730f990ead101e695d9a6054b3 not found: ID does not exist" containerID="5901c5ab26c06b30d3c533d9bf510ac1a9c302730f990ead101e695d9a6054b3" Dec 03 12:12:55 crc kubenswrapper[5052]: I1203 12:12:55.478640 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5901c5ab26c06b30d3c533d9bf510ac1a9c302730f990ead101e695d9a6054b3"} err="failed to get container status \"5901c5ab26c06b30d3c533d9bf510ac1a9c302730f990ead101e695d9a6054b3\": rpc error: code = NotFound desc = could not find container \"5901c5ab26c06b30d3c533d9bf510ac1a9c302730f990ead101e695d9a6054b3\": container with ID starting with 5901c5ab26c06b30d3c533d9bf510ac1a9c302730f990ead101e695d9a6054b3 not found: ID does not exist" Dec 03 12:12:55 crc kubenswrapper[5052]: I1203 12:12:55.478670 5052 scope.go:117] "RemoveContainer" containerID="3d331db8159cbcbfe79807aa8e190cb7755d381dfb7c94ca948f3ebf3f8ed7b5" Dec 03 12:12:55 crc kubenswrapper[5052]: E1203 12:12:55.479155 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3d331db8159cbcbfe79807aa8e190cb7755d381dfb7c94ca948f3ebf3f8ed7b5\": container with ID starting with 3d331db8159cbcbfe79807aa8e190cb7755d381dfb7c94ca948f3ebf3f8ed7b5 not found: ID does not exist" containerID="3d331db8159cbcbfe79807aa8e190cb7755d381dfb7c94ca948f3ebf3f8ed7b5" Dec 03 12:12:55 crc kubenswrapper[5052]: I1203 12:12:55.479181 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3d331db8159cbcbfe79807aa8e190cb7755d381dfb7c94ca948f3ebf3f8ed7b5"} err="failed to get container status \"3d331db8159cbcbfe79807aa8e190cb7755d381dfb7c94ca948f3ebf3f8ed7b5\": rpc error: code = NotFound desc = could not find container \"3d331db8159cbcbfe79807aa8e190cb7755d381dfb7c94ca948f3ebf3f8ed7b5\": container with ID starting with 3d331db8159cbcbfe79807aa8e190cb7755d381dfb7c94ca948f3ebf3f8ed7b5 not found: ID does not exist" Dec 03 12:12:55 crc kubenswrapper[5052]: I1203 12:12:55.479197 5052 scope.go:117] "RemoveContainer" containerID="d88a6d8aff5c3b9c680401e2f8105107db4a2ec6862732281efba40f34160fc7" Dec 03 12:12:55 crc kubenswrapper[5052]: I1203 12:12:55.479479 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d88a6d8aff5c3b9c680401e2f8105107db4a2ec6862732281efba40f34160fc7"} err="failed to get container status \"d88a6d8aff5c3b9c680401e2f8105107db4a2ec6862732281efba40f34160fc7\": rpc error: code = NotFound desc = could not find container \"d88a6d8aff5c3b9c680401e2f8105107db4a2ec6862732281efba40f34160fc7\": container with ID starting with d88a6d8aff5c3b9c680401e2f8105107db4a2ec6862732281efba40f34160fc7 not found: ID does not exist" Dec 03 12:12:55 crc kubenswrapper[5052]: I1203 12:12:55.479500 5052 scope.go:117] "RemoveContainer" containerID="458487a215cc64b6ccec6a6716dd0bbd74c0a7b8b7a88e1ee4934d149afb6e88" Dec 03 12:12:55 crc kubenswrapper[5052]: I1203 12:12:55.480416 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"458487a215cc64b6ccec6a6716dd0bbd74c0a7b8b7a88e1ee4934d149afb6e88"} err="failed to get container status \"458487a215cc64b6ccec6a6716dd0bbd74c0a7b8b7a88e1ee4934d149afb6e88\": rpc error: code = NotFound desc = could not find container \"458487a215cc64b6ccec6a6716dd0bbd74c0a7b8b7a88e1ee4934d149afb6e88\": container with ID starting with 458487a215cc64b6ccec6a6716dd0bbd74c0a7b8b7a88e1ee4934d149afb6e88 not found: ID does not exist" Dec 03 12:12:55 crc kubenswrapper[5052]: I1203 12:12:55.480437 5052 scope.go:117] "RemoveContainer" containerID="5901c5ab26c06b30d3c533d9bf510ac1a9c302730f990ead101e695d9a6054b3" Dec 03 12:12:55 crc kubenswrapper[5052]: I1203 12:12:55.481212 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:12:55 crc kubenswrapper[5052]: I1203 12:12:55.481873 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:12:55 crc kubenswrapper[5052]: I1203 12:12:55.483716 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5901c5ab26c06b30d3c533d9bf510ac1a9c302730f990ead101e695d9a6054b3"} err="failed to get container status \"5901c5ab26c06b30d3c533d9bf510ac1a9c302730f990ead101e695d9a6054b3\": rpc error: code = NotFound desc = could not find container \"5901c5ab26c06b30d3c533d9bf510ac1a9c302730f990ead101e695d9a6054b3\": container with ID starting with 5901c5ab26c06b30d3c533d9bf510ac1a9c302730f990ead101e695d9a6054b3 not found: ID does not exist" Dec 03 12:12:55 crc kubenswrapper[5052]: I1203 12:12:55.483745 5052 scope.go:117] "RemoveContainer" containerID="3d331db8159cbcbfe79807aa8e190cb7755d381dfb7c94ca948f3ebf3f8ed7b5" Dec 03 12:12:55 crc kubenswrapper[5052]: I1203 12:12:55.484134 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3d331db8159cbcbfe79807aa8e190cb7755d381dfb7c94ca948f3ebf3f8ed7b5"} err="failed to get container status \"3d331db8159cbcbfe79807aa8e190cb7755d381dfb7c94ca948f3ebf3f8ed7b5\": rpc error: code = NotFound desc = could not find container \"3d331db8159cbcbfe79807aa8e190cb7755d381dfb7c94ca948f3ebf3f8ed7b5\": container with ID starting with 3d331db8159cbcbfe79807aa8e190cb7755d381dfb7c94ca948f3ebf3f8ed7b5 not found: ID does not exist" Dec 03 12:12:55 crc kubenswrapper[5052]: I1203 12:12:55.484171 5052 scope.go:117] "RemoveContainer" containerID="d88a6d8aff5c3b9c680401e2f8105107db4a2ec6862732281efba40f34160fc7" Dec 03 12:12:55 crc kubenswrapper[5052]: I1203 12:12:55.484457 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d88a6d8aff5c3b9c680401e2f8105107db4a2ec6862732281efba40f34160fc7"} err="failed to get container status \"d88a6d8aff5c3b9c680401e2f8105107db4a2ec6862732281efba40f34160fc7\": rpc error: code = NotFound desc = could not find container \"d88a6d8aff5c3b9c680401e2f8105107db4a2ec6862732281efba40f34160fc7\": container with ID starting with d88a6d8aff5c3b9c680401e2f8105107db4a2ec6862732281efba40f34160fc7 not found: ID does not exist" Dec 03 12:12:55 crc kubenswrapper[5052]: I1203 12:12:55.484475 5052 scope.go:117] "RemoveContainer" containerID="458487a215cc64b6ccec6a6716dd0bbd74c0a7b8b7a88e1ee4934d149afb6e88" Dec 03 12:12:55 crc kubenswrapper[5052]: I1203 12:12:55.484893 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"458487a215cc64b6ccec6a6716dd0bbd74c0a7b8b7a88e1ee4934d149afb6e88"} err="failed to get container status \"458487a215cc64b6ccec6a6716dd0bbd74c0a7b8b7a88e1ee4934d149afb6e88\": rpc error: code = NotFound desc = could not find container \"458487a215cc64b6ccec6a6716dd0bbd74c0a7b8b7a88e1ee4934d149afb6e88\": container with ID starting with 458487a215cc64b6ccec6a6716dd0bbd74c0a7b8b7a88e1ee4934d149afb6e88 not found: ID does not exist" Dec 03 12:12:55 crc kubenswrapper[5052]: I1203 12:12:55.484915 5052 scope.go:117] "RemoveContainer" containerID="5901c5ab26c06b30d3c533d9bf510ac1a9c302730f990ead101e695d9a6054b3" Dec 03 12:12:55 crc kubenswrapper[5052]: I1203 12:12:55.485104 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5901c5ab26c06b30d3c533d9bf510ac1a9c302730f990ead101e695d9a6054b3"} err="failed to get container status \"5901c5ab26c06b30d3c533d9bf510ac1a9c302730f990ead101e695d9a6054b3\": rpc error: code = NotFound desc = could not find container \"5901c5ab26c06b30d3c533d9bf510ac1a9c302730f990ead101e695d9a6054b3\": container with ID starting with 5901c5ab26c06b30d3c533d9bf510ac1a9c302730f990ead101e695d9a6054b3 not found: ID does not exist" Dec 03 12:12:55 crc kubenswrapper[5052]: I1203 12:12:55.485122 5052 scope.go:117] "RemoveContainer" containerID="3d331db8159cbcbfe79807aa8e190cb7755d381dfb7c94ca948f3ebf3f8ed7b5" Dec 03 12:12:55 crc kubenswrapper[5052]: I1203 12:12:55.485356 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3d331db8159cbcbfe79807aa8e190cb7755d381dfb7c94ca948f3ebf3f8ed7b5"} err="failed to get container status \"3d331db8159cbcbfe79807aa8e190cb7755d381dfb7c94ca948f3ebf3f8ed7b5\": rpc error: code = NotFound desc = could not find container \"3d331db8159cbcbfe79807aa8e190cb7755d381dfb7c94ca948f3ebf3f8ed7b5\": container with ID starting with 3d331db8159cbcbfe79807aa8e190cb7755d381dfb7c94ca948f3ebf3f8ed7b5 not found: ID does not exist" Dec 03 12:12:55 crc kubenswrapper[5052]: I1203 12:12:55.485407 5052 scope.go:117] "RemoveContainer" containerID="d88a6d8aff5c3b9c680401e2f8105107db4a2ec6862732281efba40f34160fc7" Dec 03 12:12:55 crc kubenswrapper[5052]: I1203 12:12:55.488076 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d88a6d8aff5c3b9c680401e2f8105107db4a2ec6862732281efba40f34160fc7"} err="failed to get container status \"d88a6d8aff5c3b9c680401e2f8105107db4a2ec6862732281efba40f34160fc7\": rpc error: code = NotFound desc = could not find container \"d88a6d8aff5c3b9c680401e2f8105107db4a2ec6862732281efba40f34160fc7\": container with ID starting with d88a6d8aff5c3b9c680401e2f8105107db4a2ec6862732281efba40f34160fc7 not found: ID does not exist" Dec 03 12:12:55 crc kubenswrapper[5052]: I1203 12:12:55.488200 5052 scope.go:117] "RemoveContainer" containerID="458487a215cc64b6ccec6a6716dd0bbd74c0a7b8b7a88e1ee4934d149afb6e88" Dec 03 12:12:55 crc kubenswrapper[5052]: I1203 12:12:55.490247 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"458487a215cc64b6ccec6a6716dd0bbd74c0a7b8b7a88e1ee4934d149afb6e88"} err="failed to get container status \"458487a215cc64b6ccec6a6716dd0bbd74c0a7b8b7a88e1ee4934d149afb6e88\": rpc error: code = NotFound desc = could not find container \"458487a215cc64b6ccec6a6716dd0bbd74c0a7b8b7a88e1ee4934d149afb6e88\": container with ID starting with 458487a215cc64b6ccec6a6716dd0bbd74c0a7b8b7a88e1ee4934d149afb6e88 not found: ID does not exist" Dec 03 12:12:55 crc kubenswrapper[5052]: I1203 12:12:55.490284 5052 scope.go:117] "RemoveContainer" containerID="5901c5ab26c06b30d3c533d9bf510ac1a9c302730f990ead101e695d9a6054b3" Dec 03 12:12:55 crc kubenswrapper[5052]: I1203 12:12:55.490935 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5901c5ab26c06b30d3c533d9bf510ac1a9c302730f990ead101e695d9a6054b3"} err="failed to get container status \"5901c5ab26c06b30d3c533d9bf510ac1a9c302730f990ead101e695d9a6054b3\": rpc error: code = NotFound desc = could not find container \"5901c5ab26c06b30d3c533d9bf510ac1a9c302730f990ead101e695d9a6054b3\": container with ID starting with 5901c5ab26c06b30d3c533d9bf510ac1a9c302730f990ead101e695d9a6054b3 not found: ID does not exist" Dec 03 12:12:55 crc kubenswrapper[5052]: I1203 12:12:55.490965 5052 scope.go:117] "RemoveContainer" containerID="3d331db8159cbcbfe79807aa8e190cb7755d381dfb7c94ca948f3ebf3f8ed7b5" Dec 03 12:12:55 crc kubenswrapper[5052]: I1203 12:12:55.491434 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3d331db8159cbcbfe79807aa8e190cb7755d381dfb7c94ca948f3ebf3f8ed7b5"} err="failed to get container status \"3d331db8159cbcbfe79807aa8e190cb7755d381dfb7c94ca948f3ebf3f8ed7b5\": rpc error: code = NotFound desc = could not find container \"3d331db8159cbcbfe79807aa8e190cb7755d381dfb7c94ca948f3ebf3f8ed7b5\": container with ID starting with 3d331db8159cbcbfe79807aa8e190cb7755d381dfb7c94ca948f3ebf3f8ed7b5 not found: ID does not exist" Dec 03 12:12:55 crc kubenswrapper[5052]: I1203 12:12:55.513351 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:12:55 crc kubenswrapper[5052]: I1203 12:12:55.542358 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:12:55 crc kubenswrapper[5052]: I1203 12:12:55.611185 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/b670a8a7-3e17-424f-9efe-18f028449ca6-log-httpd\") pod \"ceilometer-0\" (UID: \"b670a8a7-3e17-424f-9efe-18f028449ca6\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:12:55 crc kubenswrapper[5052]: I1203 12:12:55.611255 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/b670a8a7-3e17-424f-9efe-18f028449ca6-config-data\") pod \"ceilometer-0\" (UID: \"b670a8a7-3e17-424f-9efe-18f028449ca6\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:12:55 crc kubenswrapper[5052]: I1203 12:12:55.611360 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b670a8a7-3e17-424f-9efe-18f028449ca6-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"b670a8a7-3e17-424f-9efe-18f028449ca6\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:12:55 crc kubenswrapper[5052]: I1203 12:12:55.611401 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/b670a8a7-3e17-424f-9efe-18f028449ca6-run-httpd\") pod \"ceilometer-0\" (UID: \"b670a8a7-3e17-424f-9efe-18f028449ca6\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:12:55 crc kubenswrapper[5052]: I1203 12:12:55.611493 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9xhv2\" (UniqueName: \"kubernetes.io/projected/b670a8a7-3e17-424f-9efe-18f028449ca6-kube-api-access-9xhv2\") pod \"ceilometer-0\" (UID: \"b670a8a7-3e17-424f-9efe-18f028449ca6\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:12:55 crc kubenswrapper[5052]: I1203 12:12:55.611699 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/b670a8a7-3e17-424f-9efe-18f028449ca6-scripts\") pod \"ceilometer-0\" (UID: \"b670a8a7-3e17-424f-9efe-18f028449ca6\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:12:55 crc kubenswrapper[5052]: I1203 12:12:55.611760 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/b670a8a7-3e17-424f-9efe-18f028449ca6-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"b670a8a7-3e17-424f-9efe-18f028449ca6\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:12:55 crc kubenswrapper[5052]: I1203 12:12:55.687188 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0d97800f-6d5a-46ed-bcb5-4452b4ce86e8" path="/var/lib/kubelet/pods/0d97800f-6d5a-46ed-bcb5-4452b4ce86e8/volumes" Dec 03 12:12:55 crc kubenswrapper[5052]: I1203 12:12:55.713698 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/b670a8a7-3e17-424f-9efe-18f028449ca6-scripts\") pod \"ceilometer-0\" (UID: \"b670a8a7-3e17-424f-9efe-18f028449ca6\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:12:55 crc kubenswrapper[5052]: I1203 12:12:55.713775 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/b670a8a7-3e17-424f-9efe-18f028449ca6-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"b670a8a7-3e17-424f-9efe-18f028449ca6\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:12:55 crc kubenswrapper[5052]: I1203 12:12:55.713825 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/b670a8a7-3e17-424f-9efe-18f028449ca6-config-data\") pod \"ceilometer-0\" (UID: \"b670a8a7-3e17-424f-9efe-18f028449ca6\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:12:55 crc kubenswrapper[5052]: I1203 12:12:55.713844 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/b670a8a7-3e17-424f-9efe-18f028449ca6-log-httpd\") pod \"ceilometer-0\" (UID: \"b670a8a7-3e17-424f-9efe-18f028449ca6\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:12:55 crc kubenswrapper[5052]: I1203 12:12:55.713884 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b670a8a7-3e17-424f-9efe-18f028449ca6-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"b670a8a7-3e17-424f-9efe-18f028449ca6\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:12:55 crc kubenswrapper[5052]: I1203 12:12:55.713914 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/b670a8a7-3e17-424f-9efe-18f028449ca6-run-httpd\") pod \"ceilometer-0\" (UID: \"b670a8a7-3e17-424f-9efe-18f028449ca6\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:12:55 crc kubenswrapper[5052]: I1203 12:12:55.714096 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-9xhv2\" (UniqueName: \"kubernetes.io/projected/b670a8a7-3e17-424f-9efe-18f028449ca6-kube-api-access-9xhv2\") pod \"ceilometer-0\" (UID: \"b670a8a7-3e17-424f-9efe-18f028449ca6\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:12:55 crc kubenswrapper[5052]: I1203 12:12:55.715344 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/b670a8a7-3e17-424f-9efe-18f028449ca6-log-httpd\") pod \"ceilometer-0\" (UID: \"b670a8a7-3e17-424f-9efe-18f028449ca6\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:12:55 crc kubenswrapper[5052]: I1203 12:12:55.717908 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/b670a8a7-3e17-424f-9efe-18f028449ca6-run-httpd\") pod \"ceilometer-0\" (UID: \"b670a8a7-3e17-424f-9efe-18f028449ca6\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:12:55 crc kubenswrapper[5052]: I1203 12:12:55.719000 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/b670a8a7-3e17-424f-9efe-18f028449ca6-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"b670a8a7-3e17-424f-9efe-18f028449ca6\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:12:55 crc kubenswrapper[5052]: I1203 12:12:55.719110 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b670a8a7-3e17-424f-9efe-18f028449ca6-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"b670a8a7-3e17-424f-9efe-18f028449ca6\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:12:55 crc kubenswrapper[5052]: I1203 12:12:55.719543 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/b670a8a7-3e17-424f-9efe-18f028449ca6-config-data\") pod \"ceilometer-0\" (UID: \"b670a8a7-3e17-424f-9efe-18f028449ca6\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:12:55 crc kubenswrapper[5052]: I1203 12:12:55.729355 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/b670a8a7-3e17-424f-9efe-18f028449ca6-scripts\") pod \"ceilometer-0\" (UID: \"b670a8a7-3e17-424f-9efe-18f028449ca6\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:12:55 crc kubenswrapper[5052]: I1203 12:12:55.744318 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-9xhv2\" (UniqueName: \"kubernetes.io/projected/b670a8a7-3e17-424f-9efe-18f028449ca6-kube-api-access-9xhv2\") pod \"ceilometer-0\" (UID: \"b670a8a7-3e17-424f-9efe-18f028449ca6\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:12:55 crc kubenswrapper[5052]: I1203 12:12:55.750071 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:12:55 crc kubenswrapper[5052]: I1203 12:12:55.899530 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-db-sync-pmm7g"] Dec 03 12:12:55 crc kubenswrapper[5052]: I1203 12:12:55.901279 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-pmm7g" Dec 03 12:12:55 crc kubenswrapper[5052]: I1203 12:12:55.904688 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell0-conductor-scripts" Dec 03 12:12:55 crc kubenswrapper[5052]: I1203 12:12:55.904714 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-nova-dockercfg-8cctz" Dec 03 12:12:55 crc kubenswrapper[5052]: I1203 12:12:55.905032 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell0-conductor-config-data" Dec 03 12:12:55 crc kubenswrapper[5052]: I1203 12:12:55.908942 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-db-sync-pmm7g"] Dec 03 12:12:56 crc kubenswrapper[5052]: I1203 12:12:56.021850 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/7600241d-6e5b-41ee-b17c-f5e15f1cab11-scripts\") pod \"nova-cell0-conductor-db-sync-pmm7g\" (UID: \"7600241d-6e5b-41ee-b17c-f5e15f1cab11\") " pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-pmm7g" Dec 03 12:12:56 crc kubenswrapper[5052]: I1203 12:12:56.021933 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/7600241d-6e5b-41ee-b17c-f5e15f1cab11-config-data\") pod \"nova-cell0-conductor-db-sync-pmm7g\" (UID: \"7600241d-6e5b-41ee-b17c-f5e15f1cab11\") " pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-pmm7g" Dec 03 12:12:56 crc kubenswrapper[5052]: I1203 12:12:56.021999 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5vmcl\" (UniqueName: \"kubernetes.io/projected/7600241d-6e5b-41ee-b17c-f5e15f1cab11-kube-api-access-5vmcl\") pod \"nova-cell0-conductor-db-sync-pmm7g\" (UID: \"7600241d-6e5b-41ee-b17c-f5e15f1cab11\") " pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-pmm7g" Dec 03 12:12:56 crc kubenswrapper[5052]: I1203 12:12:56.022023 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/7600241d-6e5b-41ee-b17c-f5e15f1cab11-combined-ca-bundle\") pod \"nova-cell0-conductor-db-sync-pmm7g\" (UID: \"7600241d-6e5b-41ee-b17c-f5e15f1cab11\") " pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-pmm7g" Dec 03 12:12:56 crc kubenswrapper[5052]: I1203 12:12:56.123490 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/7600241d-6e5b-41ee-b17c-f5e15f1cab11-scripts\") pod \"nova-cell0-conductor-db-sync-pmm7g\" (UID: \"7600241d-6e5b-41ee-b17c-f5e15f1cab11\") " pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-pmm7g" Dec 03 12:12:56 crc kubenswrapper[5052]: I1203 12:12:56.123898 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/7600241d-6e5b-41ee-b17c-f5e15f1cab11-config-data\") pod \"nova-cell0-conductor-db-sync-pmm7g\" (UID: \"7600241d-6e5b-41ee-b17c-f5e15f1cab11\") " pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-pmm7g" Dec 03 12:12:56 crc kubenswrapper[5052]: I1203 12:12:56.123969 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-5vmcl\" (UniqueName: \"kubernetes.io/projected/7600241d-6e5b-41ee-b17c-f5e15f1cab11-kube-api-access-5vmcl\") pod \"nova-cell0-conductor-db-sync-pmm7g\" (UID: \"7600241d-6e5b-41ee-b17c-f5e15f1cab11\") " pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-pmm7g" Dec 03 12:12:56 crc kubenswrapper[5052]: I1203 12:12:56.123994 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/7600241d-6e5b-41ee-b17c-f5e15f1cab11-combined-ca-bundle\") pod \"nova-cell0-conductor-db-sync-pmm7g\" (UID: \"7600241d-6e5b-41ee-b17c-f5e15f1cab11\") " pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-pmm7g" Dec 03 12:12:56 crc kubenswrapper[5052]: I1203 12:12:56.130054 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/7600241d-6e5b-41ee-b17c-f5e15f1cab11-config-data\") pod \"nova-cell0-conductor-db-sync-pmm7g\" (UID: \"7600241d-6e5b-41ee-b17c-f5e15f1cab11\") " pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-pmm7g" Dec 03 12:12:56 crc kubenswrapper[5052]: I1203 12:12:56.133427 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/7600241d-6e5b-41ee-b17c-f5e15f1cab11-scripts\") pod \"nova-cell0-conductor-db-sync-pmm7g\" (UID: \"7600241d-6e5b-41ee-b17c-f5e15f1cab11\") " pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-pmm7g" Dec 03 12:12:56 crc kubenswrapper[5052]: I1203 12:12:56.136068 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/7600241d-6e5b-41ee-b17c-f5e15f1cab11-combined-ca-bundle\") pod \"nova-cell0-conductor-db-sync-pmm7g\" (UID: \"7600241d-6e5b-41ee-b17c-f5e15f1cab11\") " pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-pmm7g" Dec 03 12:12:56 crc kubenswrapper[5052]: I1203 12:12:56.150921 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-5vmcl\" (UniqueName: \"kubernetes.io/projected/7600241d-6e5b-41ee-b17c-f5e15f1cab11-kube-api-access-5vmcl\") pod \"nova-cell0-conductor-db-sync-pmm7g\" (UID: \"7600241d-6e5b-41ee-b17c-f5e15f1cab11\") " pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-pmm7g" Dec 03 12:12:56 crc kubenswrapper[5052]: I1203 12:12:56.223605 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-pmm7g" Dec 03 12:12:56 crc kubenswrapper[5052]: I1203 12:12:56.309158 5052 generic.go:334] "Generic (PLEG): container finished" podID="49095880-d707-4afb-a3cc-a10ece6970de" containerID="f94cbc489fbc2c2f8115d1902907b2ba57a8926f2d1730a50bf76e99d2e355cf" exitCode=0 Dec 03 12:12:56 crc kubenswrapper[5052]: I1203 12:12:56.311212 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-74495c68d7-59m75" event={"ID":"49095880-d707-4afb-a3cc-a10ece6970de","Type":"ContainerDied","Data":"f94cbc489fbc2c2f8115d1902907b2ba57a8926f2d1730a50bf76e99d2e355cf"} Dec 03 12:12:56 crc kubenswrapper[5052]: I1203 12:12:56.311259 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:12:56 crc kubenswrapper[5052]: I1203 12:12:56.311277 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:12:56 crc kubenswrapper[5052]: I1203 12:12:56.404489 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:12:56 crc kubenswrapper[5052]: W1203 12:12:56.408410 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb670a8a7_3e17_424f_9efe_18f028449ca6.slice/crio-e532be54ec177fd3d08a1368946f6bc7d005cbd76ac62449ee73ef96f6798ef1 WatchSource:0}: Error finding container e532be54ec177fd3d08a1368946f6bc7d005cbd76ac62449ee73ef96f6798ef1: Status 404 returned error can't find the container with id e532be54ec177fd3d08a1368946f6bc7d005cbd76ac62449ee73ef96f6798ef1 Dec 03 12:12:56 crc kubenswrapper[5052]: I1203 12:12:56.713692 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-db-sync-pmm7g"] Dec 03 12:12:56 crc kubenswrapper[5052]: W1203 12:12:56.718103 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7600241d_6e5b_41ee_b17c_f5e15f1cab11.slice/crio-24fcc54fde3cbf4b82a1634572796d01e7a380e5792c771ed962018a18c534e6 WatchSource:0}: Error finding container 24fcc54fde3cbf4b82a1634572796d01e7a380e5792c771ed962018a18c534e6: Status 404 returned error can't find the container with id 24fcc54fde3cbf4b82a1634572796d01e7a380e5792c771ed962018a18c534e6 Dec 03 12:12:57 crc kubenswrapper[5052]: I1203 12:12:57.322329 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-pmm7g" event={"ID":"7600241d-6e5b-41ee-b17c-f5e15f1cab11","Type":"ContainerStarted","Data":"24fcc54fde3cbf4b82a1634572796d01e7a380e5792c771ed962018a18c534e6"} Dec 03 12:12:57 crc kubenswrapper[5052]: I1203 12:12:57.326095 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"b670a8a7-3e17-424f-9efe-18f028449ca6","Type":"ContainerStarted","Data":"e532be54ec177fd3d08a1368946f6bc7d005cbd76ac62449ee73ef96f6798ef1"} Dec 03 12:12:57 crc kubenswrapper[5052]: I1203 12:12:57.607547 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:12:57 crc kubenswrapper[5052]: I1203 12:12:57.607700 5052 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Dec 03 12:12:57 crc kubenswrapper[5052]: I1203 12:12:57.750912 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:12:58 crc kubenswrapper[5052]: I1203 12:12:58.339615 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-pmm7g" event={"ID":"7600241d-6e5b-41ee-b17c-f5e15f1cab11","Type":"ContainerStarted","Data":"598592c093c893935a6c3dd121e70f0606c05f1732db9c2404c00c490b70c675"} Dec 03 12:12:58 crc kubenswrapper[5052]: I1203 12:12:58.339663 5052 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Dec 03 12:12:58 crc kubenswrapper[5052]: I1203 12:12:58.340000 5052 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Dec 03 12:12:58 crc kubenswrapper[5052]: I1203 12:12:58.475674 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:12:58 crc kubenswrapper[5052]: I1203 12:12:58.546070 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:12:59 crc kubenswrapper[5052]: I1203 12:12:59.351543 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"b670a8a7-3e17-424f-9efe-18f028449ca6","Type":"ContainerStarted","Data":"856e85b20893d0c08665315c89bcb7e4a614ebbfb1bffee82d066914c2afc3e3"} Dec 03 12:12:59 crc kubenswrapper[5052]: I1203 12:12:59.366689 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-pmm7g" podStartSLOduration=4.366670677 podStartE2EDuration="4.366670677s" podCreationTimestamp="2025-12-03 12:12:55 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:12:59.36569827 +0000 UTC m=+3169.953940034" watchObservedRunningTime="2025-12-03 12:12:59.366670677 +0000 UTC m=+3169.954912441" Dec 03 12:12:59 crc kubenswrapper[5052]: I1203 12:12:59.435041 5052 scope.go:117] "RemoveContainer" containerID="9078309d7e958fc36e931932ff0699b628d5fd0921b34acd1cd89a0efd5e8691" Dec 03 12:12:59 crc kubenswrapper[5052]: I1203 12:12:59.459777 5052 scope.go:117] "RemoveContainer" containerID="ac697ab7fae2d375f75e93f50ee6034e1e0bf8653a46e8447628f29f9a3cceb3" Dec 03 12:12:59 crc kubenswrapper[5052]: I1203 12:12:59.520388 5052 scope.go:117] "RemoveContainer" containerID="8b708b7d5d49769665548fc88668b1958dadb43740e66dff74107f13b9ce7ae6" Dec 03 12:12:59 crc kubenswrapper[5052]: I1203 12:12:59.559858 5052 scope.go:117] "RemoveContainer" containerID="e1903fe93354f2eb6210febadfc9a242d90b8681d1287d692e162cc3efd0b892" Dec 03 12:12:59 crc kubenswrapper[5052]: I1203 12:12:59.584046 5052 scope.go:117] "RemoveContainer" containerID="4d472d46d5c7f6ca91aa5161d730340a3e8abead48886bdc090952991731d00a" Dec 03 12:12:59 crc kubenswrapper[5052]: I1203 12:12:59.621198 5052 scope.go:117] "RemoveContainer" containerID="737462a16fc5e657fb057be690e9512c155f3a35e26b18e1f261c004033bd4d8" Dec 03 12:12:59 crc kubenswrapper[5052]: I1203 12:12:59.668138 5052 scope.go:117] "RemoveContainer" containerID="f68f634ddf0cc7c513c585883315d968fce2a8f67935dd2746313bd6d94c035d" Dec 03 12:13:01 crc kubenswrapper[5052]: I1203 12:13:01.375843 5052 generic.go:334] "Generic (PLEG): container finished" podID="49095880-d707-4afb-a3cc-a10ece6970de" containerID="e5291229716bb69e07dd59d25b94079d3c162d2e091119be3de021794e666c7c" exitCode=0 Dec 03 12:13:01 crc kubenswrapper[5052]: I1203 12:13:01.375924 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-74495c68d7-59m75" event={"ID":"49095880-d707-4afb-a3cc-a10ece6970de","Type":"ContainerDied","Data":"e5291229716bb69e07dd59d25b94079d3c162d2e091119be3de021794e666c7c"} Dec 03 12:13:01 crc kubenswrapper[5052]: I1203 12:13:01.378055 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"b670a8a7-3e17-424f-9efe-18f028449ca6","Type":"ContainerStarted","Data":"7c69e8c9302ce4a5eaf0c099dd3835200c1d10a17e04ac21130275e5c3ffe1e6"} Dec 03 12:13:01 crc kubenswrapper[5052]: I1203 12:13:01.673716 5052 scope.go:117] "RemoveContainer" containerID="63298107b3d090deff9ed92f3d613e5b1c86d9137684d117500257db8ee1c805" Dec 03 12:13:01 crc kubenswrapper[5052]: E1203 12:13:01.674359 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 12:13:01 crc kubenswrapper[5052]: I1203 12:13:01.871479 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-74495c68d7-59m75" Dec 03 12:13:01 crc kubenswrapper[5052]: I1203 12:13:01.962386 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/49095880-d707-4afb-a3cc-a10ece6970de-httpd-config\") pod \"49095880-d707-4afb-a3cc-a10ece6970de\" (UID: \"49095880-d707-4afb-a3cc-a10ece6970de\") " Dec 03 12:13:01 crc kubenswrapper[5052]: I1203 12:13:01.962769 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/49095880-d707-4afb-a3cc-a10ece6970de-ovndb-tls-certs\") pod \"49095880-d707-4afb-a3cc-a10ece6970de\" (UID: \"49095880-d707-4afb-a3cc-a10ece6970de\") " Dec 03 12:13:01 crc kubenswrapper[5052]: I1203 12:13:01.962830 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/49095880-d707-4afb-a3cc-a10ece6970de-combined-ca-bundle\") pod \"49095880-d707-4afb-a3cc-a10ece6970de\" (UID: \"49095880-d707-4afb-a3cc-a10ece6970de\") " Dec 03 12:13:01 crc kubenswrapper[5052]: I1203 12:13:01.962918 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/49095880-d707-4afb-a3cc-a10ece6970de-config\") pod \"49095880-d707-4afb-a3cc-a10ece6970de\" (UID: \"49095880-d707-4afb-a3cc-a10ece6970de\") " Dec 03 12:13:01 crc kubenswrapper[5052]: I1203 12:13:01.962976 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mgm9k\" (UniqueName: \"kubernetes.io/projected/49095880-d707-4afb-a3cc-a10ece6970de-kube-api-access-mgm9k\") pod \"49095880-d707-4afb-a3cc-a10ece6970de\" (UID: \"49095880-d707-4afb-a3cc-a10ece6970de\") " Dec 03 12:13:01 crc kubenswrapper[5052]: I1203 12:13:01.968668 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/49095880-d707-4afb-a3cc-a10ece6970de-httpd-config" (OuterVolumeSpecName: "httpd-config") pod "49095880-d707-4afb-a3cc-a10ece6970de" (UID: "49095880-d707-4afb-a3cc-a10ece6970de"). InnerVolumeSpecName "httpd-config". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:13:01 crc kubenswrapper[5052]: I1203 12:13:01.969413 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/49095880-d707-4afb-a3cc-a10ece6970de-kube-api-access-mgm9k" (OuterVolumeSpecName: "kube-api-access-mgm9k") pod "49095880-d707-4afb-a3cc-a10ece6970de" (UID: "49095880-d707-4afb-a3cc-a10ece6970de"). InnerVolumeSpecName "kube-api-access-mgm9k". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:13:02 crc kubenswrapper[5052]: I1203 12:13:02.022857 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/49095880-d707-4afb-a3cc-a10ece6970de-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "49095880-d707-4afb-a3cc-a10ece6970de" (UID: "49095880-d707-4afb-a3cc-a10ece6970de"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:13:02 crc kubenswrapper[5052]: I1203 12:13:02.023398 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/49095880-d707-4afb-a3cc-a10ece6970de-config" (OuterVolumeSpecName: "config") pod "49095880-d707-4afb-a3cc-a10ece6970de" (UID: "49095880-d707-4afb-a3cc-a10ece6970de"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:13:02 crc kubenswrapper[5052]: I1203 12:13:02.059567 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/49095880-d707-4afb-a3cc-a10ece6970de-ovndb-tls-certs" (OuterVolumeSpecName: "ovndb-tls-certs") pod "49095880-d707-4afb-a3cc-a10ece6970de" (UID: "49095880-d707-4afb-a3cc-a10ece6970de"). InnerVolumeSpecName "ovndb-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:13:02 crc kubenswrapper[5052]: I1203 12:13:02.065705 5052 reconciler_common.go:293] "Volume detached for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/49095880-d707-4afb-a3cc-a10ece6970de-httpd-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:13:02 crc kubenswrapper[5052]: I1203 12:13:02.065737 5052 reconciler_common.go:293] "Volume detached for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/49095880-d707-4afb-a3cc-a10ece6970de-ovndb-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:13:02 crc kubenswrapper[5052]: I1203 12:13:02.065747 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/49095880-d707-4afb-a3cc-a10ece6970de-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:13:02 crc kubenswrapper[5052]: I1203 12:13:02.065756 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/49095880-d707-4afb-a3cc-a10ece6970de-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:13:02 crc kubenswrapper[5052]: I1203 12:13:02.065766 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-mgm9k\" (UniqueName: \"kubernetes.io/projected/49095880-d707-4afb-a3cc-a10ece6970de-kube-api-access-mgm9k\") on node \"crc\" DevicePath \"\"" Dec 03 12:13:02 crc kubenswrapper[5052]: I1203 12:13:02.389577 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"b670a8a7-3e17-424f-9efe-18f028449ca6","Type":"ContainerStarted","Data":"1b70b579f4f38be27aaefeb5561c0b9527b93fc22c75c9584022b51bc6ed8608"} Dec 03 12:13:02 crc kubenswrapper[5052]: I1203 12:13:02.391446 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-74495c68d7-59m75" event={"ID":"49095880-d707-4afb-a3cc-a10ece6970de","Type":"ContainerDied","Data":"500a6bc23532192b4a1634e44580baa95afa6bef3031a94faf97ea7415c74a2c"} Dec 03 12:13:02 crc kubenswrapper[5052]: I1203 12:13:02.391486 5052 scope.go:117] "RemoveContainer" containerID="f94cbc489fbc2c2f8115d1902907b2ba57a8926f2d1730a50bf76e99d2e355cf" Dec 03 12:13:02 crc kubenswrapper[5052]: I1203 12:13:02.391803 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-74495c68d7-59m75" Dec 03 12:13:02 crc kubenswrapper[5052]: I1203 12:13:02.411496 5052 scope.go:117] "RemoveContainer" containerID="e5291229716bb69e07dd59d25b94079d3c162d2e091119be3de021794e666c7c" Dec 03 12:13:02 crc kubenswrapper[5052]: I1203 12:13:02.442498 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/neutron-74495c68d7-59m75"] Dec 03 12:13:02 crc kubenswrapper[5052]: I1203 12:13:02.453055 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/neutron-74495c68d7-59m75"] Dec 03 12:13:03 crc kubenswrapper[5052]: I1203 12:13:03.684232 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="49095880-d707-4afb-a3cc-a10ece6970de" path="/var/lib/kubelet/pods/49095880-d707-4afb-a3cc-a10ece6970de/volumes" Dec 03 12:13:04 crc kubenswrapper[5052]: I1203 12:13:04.416876 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"b670a8a7-3e17-424f-9efe-18f028449ca6","Type":"ContainerStarted","Data":"94de023d1837fe9f4efd468362e177ec6c30b3f05cbb4d9043728bb05ccf37a2"} Dec 03 12:13:04 crc kubenswrapper[5052]: I1203 12:13:04.417427 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:13:04 crc kubenswrapper[5052]: I1203 12:13:04.472419 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/ceilometer-0" podStartSLOduration=2.408179996 podStartE2EDuration="9.47239261s" podCreationTimestamp="2025-12-03 12:12:55 +0000 UTC" firstStartedPulling="2025-12-03 12:12:56.415789251 +0000 UTC m=+3167.004031015" lastFinishedPulling="2025-12-03 12:13:03.480001845 +0000 UTC m=+3174.068243629" observedRunningTime="2025-12-03 12:13:04.452028618 +0000 UTC m=+3175.040270392" watchObservedRunningTime="2025-12-03 12:13:04.47239261 +0000 UTC m=+3175.060634384" Dec 03 12:13:14 crc kubenswrapper[5052]: I1203 12:13:14.526043 5052 generic.go:334] "Generic (PLEG): container finished" podID="7600241d-6e5b-41ee-b17c-f5e15f1cab11" containerID="598592c093c893935a6c3dd121e70f0606c05f1732db9c2404c00c490b70c675" exitCode=0 Dec 03 12:13:14 crc kubenswrapper[5052]: I1203 12:13:14.526172 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-pmm7g" event={"ID":"7600241d-6e5b-41ee-b17c-f5e15f1cab11","Type":"ContainerDied","Data":"598592c093c893935a6c3dd121e70f0606c05f1732db9c2404c00c490b70c675"} Dec 03 12:13:15 crc kubenswrapper[5052]: I1203 12:13:15.912527 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-pmm7g" Dec 03 12:13:16 crc kubenswrapper[5052]: I1203 12:13:16.041652 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/7600241d-6e5b-41ee-b17c-f5e15f1cab11-combined-ca-bundle\") pod \"7600241d-6e5b-41ee-b17c-f5e15f1cab11\" (UID: \"7600241d-6e5b-41ee-b17c-f5e15f1cab11\") " Dec 03 12:13:16 crc kubenswrapper[5052]: I1203 12:13:16.041748 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5vmcl\" (UniqueName: \"kubernetes.io/projected/7600241d-6e5b-41ee-b17c-f5e15f1cab11-kube-api-access-5vmcl\") pod \"7600241d-6e5b-41ee-b17c-f5e15f1cab11\" (UID: \"7600241d-6e5b-41ee-b17c-f5e15f1cab11\") " Dec 03 12:13:16 crc kubenswrapper[5052]: I1203 12:13:16.041806 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/7600241d-6e5b-41ee-b17c-f5e15f1cab11-scripts\") pod \"7600241d-6e5b-41ee-b17c-f5e15f1cab11\" (UID: \"7600241d-6e5b-41ee-b17c-f5e15f1cab11\") " Dec 03 12:13:16 crc kubenswrapper[5052]: I1203 12:13:16.042548 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/7600241d-6e5b-41ee-b17c-f5e15f1cab11-config-data\") pod \"7600241d-6e5b-41ee-b17c-f5e15f1cab11\" (UID: \"7600241d-6e5b-41ee-b17c-f5e15f1cab11\") " Dec 03 12:13:16 crc kubenswrapper[5052]: I1203 12:13:16.047125 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7600241d-6e5b-41ee-b17c-f5e15f1cab11-scripts" (OuterVolumeSpecName: "scripts") pod "7600241d-6e5b-41ee-b17c-f5e15f1cab11" (UID: "7600241d-6e5b-41ee-b17c-f5e15f1cab11"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:13:16 crc kubenswrapper[5052]: I1203 12:13:16.054769 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7600241d-6e5b-41ee-b17c-f5e15f1cab11-kube-api-access-5vmcl" (OuterVolumeSpecName: "kube-api-access-5vmcl") pod "7600241d-6e5b-41ee-b17c-f5e15f1cab11" (UID: "7600241d-6e5b-41ee-b17c-f5e15f1cab11"). InnerVolumeSpecName "kube-api-access-5vmcl". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:13:16 crc kubenswrapper[5052]: I1203 12:13:16.067384 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7600241d-6e5b-41ee-b17c-f5e15f1cab11-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "7600241d-6e5b-41ee-b17c-f5e15f1cab11" (UID: "7600241d-6e5b-41ee-b17c-f5e15f1cab11"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:13:16 crc kubenswrapper[5052]: I1203 12:13:16.068334 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7600241d-6e5b-41ee-b17c-f5e15f1cab11-config-data" (OuterVolumeSpecName: "config-data") pod "7600241d-6e5b-41ee-b17c-f5e15f1cab11" (UID: "7600241d-6e5b-41ee-b17c-f5e15f1cab11"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:13:16 crc kubenswrapper[5052]: I1203 12:13:16.144726 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/7600241d-6e5b-41ee-b17c-f5e15f1cab11-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:13:16 crc kubenswrapper[5052]: I1203 12:13:16.144785 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-5vmcl\" (UniqueName: \"kubernetes.io/projected/7600241d-6e5b-41ee-b17c-f5e15f1cab11-kube-api-access-5vmcl\") on node \"crc\" DevicePath \"\"" Dec 03 12:13:16 crc kubenswrapper[5052]: I1203 12:13:16.144798 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/7600241d-6e5b-41ee-b17c-f5e15f1cab11-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:13:16 crc kubenswrapper[5052]: I1203 12:13:16.144809 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/7600241d-6e5b-41ee-b17c-f5e15f1cab11-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:13:16 crc kubenswrapper[5052]: I1203 12:13:16.544998 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-pmm7g" event={"ID":"7600241d-6e5b-41ee-b17c-f5e15f1cab11","Type":"ContainerDied","Data":"24fcc54fde3cbf4b82a1634572796d01e7a380e5792c771ed962018a18c534e6"} Dec 03 12:13:16 crc kubenswrapper[5052]: I1203 12:13:16.545038 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="24fcc54fde3cbf4b82a1634572796d01e7a380e5792c771ed962018a18c534e6" Dec 03 12:13:16 crc kubenswrapper[5052]: I1203 12:13:16.545058 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-pmm7g" Dec 03 12:13:16 crc kubenswrapper[5052]: I1203 12:13:16.627871 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-0"] Dec 03 12:13:16 crc kubenswrapper[5052]: E1203 12:13:16.628288 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="7600241d-6e5b-41ee-b17c-f5e15f1cab11" containerName="nova-cell0-conductor-db-sync" Dec 03 12:13:16 crc kubenswrapper[5052]: I1203 12:13:16.628305 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="7600241d-6e5b-41ee-b17c-f5e15f1cab11" containerName="nova-cell0-conductor-db-sync" Dec 03 12:13:16 crc kubenswrapper[5052]: E1203 12:13:16.628318 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="49095880-d707-4afb-a3cc-a10ece6970de" containerName="neutron-api" Dec 03 12:13:16 crc kubenswrapper[5052]: I1203 12:13:16.628326 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="49095880-d707-4afb-a3cc-a10ece6970de" containerName="neutron-api" Dec 03 12:13:16 crc kubenswrapper[5052]: E1203 12:13:16.628339 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="49095880-d707-4afb-a3cc-a10ece6970de" containerName="neutron-httpd" Dec 03 12:13:16 crc kubenswrapper[5052]: I1203 12:13:16.628344 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="49095880-d707-4afb-a3cc-a10ece6970de" containerName="neutron-httpd" Dec 03 12:13:16 crc kubenswrapper[5052]: I1203 12:13:16.628528 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="7600241d-6e5b-41ee-b17c-f5e15f1cab11" containerName="nova-cell0-conductor-db-sync" Dec 03 12:13:16 crc kubenswrapper[5052]: I1203 12:13:16.628557 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="49095880-d707-4afb-a3cc-a10ece6970de" containerName="neutron-api" Dec 03 12:13:16 crc kubenswrapper[5052]: I1203 12:13:16.628572 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="49095880-d707-4afb-a3cc-a10ece6970de" containerName="neutron-httpd" Dec 03 12:13:16 crc kubenswrapper[5052]: I1203 12:13:16.629256 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:13:16 crc kubenswrapper[5052]: I1203 12:13:16.632610 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-nova-dockercfg-8cctz" Dec 03 12:13:16 crc kubenswrapper[5052]: I1203 12:13:16.635375 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell0-conductor-config-data" Dec 03 12:13:16 crc kubenswrapper[5052]: I1203 12:13:16.653605 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-0"] Dec 03 12:13:16 crc kubenswrapper[5052]: I1203 12:13:16.653732 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/14774d58-37e8-4e65-b0b0-d355c0070bca-config-data\") pod \"nova-cell0-conductor-0\" (UID: \"14774d58-37e8-4e65-b0b0-d355c0070bca\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:13:16 crc kubenswrapper[5052]: I1203 12:13:16.653790 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/14774d58-37e8-4e65-b0b0-d355c0070bca-combined-ca-bundle\") pod \"nova-cell0-conductor-0\" (UID: \"14774d58-37e8-4e65-b0b0-d355c0070bca\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:13:16 crc kubenswrapper[5052]: I1203 12:13:16.653834 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bv6k4\" (UniqueName: \"kubernetes.io/projected/14774d58-37e8-4e65-b0b0-d355c0070bca-kube-api-access-bv6k4\") pod \"nova-cell0-conductor-0\" (UID: \"14774d58-37e8-4e65-b0b0-d355c0070bca\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:13:16 crc kubenswrapper[5052]: I1203 12:13:16.673396 5052 scope.go:117] "RemoveContainer" containerID="63298107b3d090deff9ed92f3d613e5b1c86d9137684d117500257db8ee1c805" Dec 03 12:13:16 crc kubenswrapper[5052]: E1203 12:13:16.673658 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 12:13:16 crc kubenswrapper[5052]: I1203 12:13:16.755534 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/14774d58-37e8-4e65-b0b0-d355c0070bca-config-data\") pod \"nova-cell0-conductor-0\" (UID: \"14774d58-37e8-4e65-b0b0-d355c0070bca\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:13:16 crc kubenswrapper[5052]: I1203 12:13:16.755628 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/14774d58-37e8-4e65-b0b0-d355c0070bca-combined-ca-bundle\") pod \"nova-cell0-conductor-0\" (UID: \"14774d58-37e8-4e65-b0b0-d355c0070bca\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:13:16 crc kubenswrapper[5052]: I1203 12:13:16.755728 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-bv6k4\" (UniqueName: \"kubernetes.io/projected/14774d58-37e8-4e65-b0b0-d355c0070bca-kube-api-access-bv6k4\") pod \"nova-cell0-conductor-0\" (UID: \"14774d58-37e8-4e65-b0b0-d355c0070bca\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:13:16 crc kubenswrapper[5052]: I1203 12:13:16.767970 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/14774d58-37e8-4e65-b0b0-d355c0070bca-config-data\") pod \"nova-cell0-conductor-0\" (UID: \"14774d58-37e8-4e65-b0b0-d355c0070bca\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:13:16 crc kubenswrapper[5052]: I1203 12:13:16.771702 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/14774d58-37e8-4e65-b0b0-d355c0070bca-combined-ca-bundle\") pod \"nova-cell0-conductor-0\" (UID: \"14774d58-37e8-4e65-b0b0-d355c0070bca\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:13:16 crc kubenswrapper[5052]: I1203 12:13:16.772235 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-bv6k4\" (UniqueName: \"kubernetes.io/projected/14774d58-37e8-4e65-b0b0-d355c0070bca-kube-api-access-bv6k4\") pod \"nova-cell0-conductor-0\" (UID: \"14774d58-37e8-4e65-b0b0-d355c0070bca\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:13:16 crc kubenswrapper[5052]: I1203 12:13:16.945833 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:13:17 crc kubenswrapper[5052]: I1203 12:13:17.176298 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:13:17 crc kubenswrapper[5052]: I1203 12:13:17.176885 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="b670a8a7-3e17-424f-9efe-18f028449ca6" containerName="ceilometer-central-agent" containerID="cri-o://856e85b20893d0c08665315c89bcb7e4a614ebbfb1bffee82d066914c2afc3e3" gracePeriod=30 Dec 03 12:13:17 crc kubenswrapper[5052]: I1203 12:13:17.177148 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="b670a8a7-3e17-424f-9efe-18f028449ca6" containerName="proxy-httpd" containerID="cri-o://94de023d1837fe9f4efd468362e177ec6c30b3f05cbb4d9043728bb05ccf37a2" gracePeriod=30 Dec 03 12:13:17 crc kubenswrapper[5052]: I1203 12:13:17.177253 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="b670a8a7-3e17-424f-9efe-18f028449ca6" containerName="ceilometer-notification-agent" containerID="cri-o://7c69e8c9302ce4a5eaf0c099dd3835200c1d10a17e04ac21130275e5c3ffe1e6" gracePeriod=30 Dec 03 12:13:17 crc kubenswrapper[5052]: I1203 12:13:17.177531 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="b670a8a7-3e17-424f-9efe-18f028449ca6" containerName="sg-core" containerID="cri-o://1b70b579f4f38be27aaefeb5561c0b9527b93fc22c75c9584022b51bc6ed8608" gracePeriod=30 Dec 03 12:13:17 crc kubenswrapper[5052]: I1203 12:13:17.200701 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:13:17 crc kubenswrapper[5052]: I1203 12:13:17.398725 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-0"] Dec 03 12:13:17 crc kubenswrapper[5052]: W1203 12:13:17.399305 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod14774d58_37e8_4e65_b0b0_d355c0070bca.slice/crio-e8a19d510f65a07fc3950e37b6113aaec9bb3cdae16c006c819b91c1bb678de4 WatchSource:0}: Error finding container e8a19d510f65a07fc3950e37b6113aaec9bb3cdae16c006c819b91c1bb678de4: Status 404 returned error can't find the container with id e8a19d510f65a07fc3950e37b6113aaec9bb3cdae16c006c819b91c1bb678de4 Dec 03 12:13:17 crc kubenswrapper[5052]: I1203 12:13:17.557345 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-conductor-0" event={"ID":"14774d58-37e8-4e65-b0b0-d355c0070bca","Type":"ContainerStarted","Data":"e8a19d510f65a07fc3950e37b6113aaec9bb3cdae16c006c819b91c1bb678de4"} Dec 03 12:13:17 crc kubenswrapper[5052]: I1203 12:13:17.562017 5052 generic.go:334] "Generic (PLEG): container finished" podID="b670a8a7-3e17-424f-9efe-18f028449ca6" containerID="94de023d1837fe9f4efd468362e177ec6c30b3f05cbb4d9043728bb05ccf37a2" exitCode=0 Dec 03 12:13:17 crc kubenswrapper[5052]: I1203 12:13:17.562047 5052 generic.go:334] "Generic (PLEG): container finished" podID="b670a8a7-3e17-424f-9efe-18f028449ca6" containerID="1b70b579f4f38be27aaefeb5561c0b9527b93fc22c75c9584022b51bc6ed8608" exitCode=2 Dec 03 12:13:17 crc kubenswrapper[5052]: I1203 12:13:17.562066 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"b670a8a7-3e17-424f-9efe-18f028449ca6","Type":"ContainerDied","Data":"94de023d1837fe9f4efd468362e177ec6c30b3f05cbb4d9043728bb05ccf37a2"} Dec 03 12:13:17 crc kubenswrapper[5052]: I1203 12:13:17.562087 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"b670a8a7-3e17-424f-9efe-18f028449ca6","Type":"ContainerDied","Data":"1b70b579f4f38be27aaefeb5561c0b9527b93fc22c75c9584022b51bc6ed8608"} Dec 03 12:13:18 crc kubenswrapper[5052]: I1203 12:13:18.343962 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:13:18 crc kubenswrapper[5052]: I1203 12:13:18.482567 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9xhv2\" (UniqueName: \"kubernetes.io/projected/b670a8a7-3e17-424f-9efe-18f028449ca6-kube-api-access-9xhv2\") pod \"b670a8a7-3e17-424f-9efe-18f028449ca6\" (UID: \"b670a8a7-3e17-424f-9efe-18f028449ca6\") " Dec 03 12:13:18 crc kubenswrapper[5052]: I1203 12:13:18.482659 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b670a8a7-3e17-424f-9efe-18f028449ca6-combined-ca-bundle\") pod \"b670a8a7-3e17-424f-9efe-18f028449ca6\" (UID: \"b670a8a7-3e17-424f-9efe-18f028449ca6\") " Dec 03 12:13:18 crc kubenswrapper[5052]: I1203 12:13:18.482738 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/b670a8a7-3e17-424f-9efe-18f028449ca6-log-httpd\") pod \"b670a8a7-3e17-424f-9efe-18f028449ca6\" (UID: \"b670a8a7-3e17-424f-9efe-18f028449ca6\") " Dec 03 12:13:18 crc kubenswrapper[5052]: I1203 12:13:18.482801 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/b670a8a7-3e17-424f-9efe-18f028449ca6-scripts\") pod \"b670a8a7-3e17-424f-9efe-18f028449ca6\" (UID: \"b670a8a7-3e17-424f-9efe-18f028449ca6\") " Dec 03 12:13:18 crc kubenswrapper[5052]: I1203 12:13:18.482859 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/b670a8a7-3e17-424f-9efe-18f028449ca6-run-httpd\") pod \"b670a8a7-3e17-424f-9efe-18f028449ca6\" (UID: \"b670a8a7-3e17-424f-9efe-18f028449ca6\") " Dec 03 12:13:18 crc kubenswrapper[5052]: I1203 12:13:18.483000 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/b670a8a7-3e17-424f-9efe-18f028449ca6-config-data\") pod \"b670a8a7-3e17-424f-9efe-18f028449ca6\" (UID: \"b670a8a7-3e17-424f-9efe-18f028449ca6\") " Dec 03 12:13:18 crc kubenswrapper[5052]: I1203 12:13:18.483039 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/b670a8a7-3e17-424f-9efe-18f028449ca6-sg-core-conf-yaml\") pod \"b670a8a7-3e17-424f-9efe-18f028449ca6\" (UID: \"b670a8a7-3e17-424f-9efe-18f028449ca6\") " Dec 03 12:13:18 crc kubenswrapper[5052]: I1203 12:13:18.484684 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b670a8a7-3e17-424f-9efe-18f028449ca6-log-httpd" (OuterVolumeSpecName: "log-httpd") pod "b670a8a7-3e17-424f-9efe-18f028449ca6" (UID: "b670a8a7-3e17-424f-9efe-18f028449ca6"). InnerVolumeSpecName "log-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:13:18 crc kubenswrapper[5052]: I1203 12:13:18.484893 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b670a8a7-3e17-424f-9efe-18f028449ca6-run-httpd" (OuterVolumeSpecName: "run-httpd") pod "b670a8a7-3e17-424f-9efe-18f028449ca6" (UID: "b670a8a7-3e17-424f-9efe-18f028449ca6"). InnerVolumeSpecName "run-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:13:18 crc kubenswrapper[5052]: I1203 12:13:18.488757 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b670a8a7-3e17-424f-9efe-18f028449ca6-scripts" (OuterVolumeSpecName: "scripts") pod "b670a8a7-3e17-424f-9efe-18f028449ca6" (UID: "b670a8a7-3e17-424f-9efe-18f028449ca6"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:13:18 crc kubenswrapper[5052]: I1203 12:13:18.499744 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b670a8a7-3e17-424f-9efe-18f028449ca6-kube-api-access-9xhv2" (OuterVolumeSpecName: "kube-api-access-9xhv2") pod "b670a8a7-3e17-424f-9efe-18f028449ca6" (UID: "b670a8a7-3e17-424f-9efe-18f028449ca6"). InnerVolumeSpecName "kube-api-access-9xhv2". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:13:18 crc kubenswrapper[5052]: I1203 12:13:18.517091 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b670a8a7-3e17-424f-9efe-18f028449ca6-sg-core-conf-yaml" (OuterVolumeSpecName: "sg-core-conf-yaml") pod "b670a8a7-3e17-424f-9efe-18f028449ca6" (UID: "b670a8a7-3e17-424f-9efe-18f028449ca6"). InnerVolumeSpecName "sg-core-conf-yaml". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:13:18 crc kubenswrapper[5052]: I1203 12:13:18.551641 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b670a8a7-3e17-424f-9efe-18f028449ca6-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "b670a8a7-3e17-424f-9efe-18f028449ca6" (UID: "b670a8a7-3e17-424f-9efe-18f028449ca6"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:13:18 crc kubenswrapper[5052]: I1203 12:13:18.572481 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b670a8a7-3e17-424f-9efe-18f028449ca6-config-data" (OuterVolumeSpecName: "config-data") pod "b670a8a7-3e17-424f-9efe-18f028449ca6" (UID: "b670a8a7-3e17-424f-9efe-18f028449ca6"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:13:18 crc kubenswrapper[5052]: I1203 12:13:18.577555 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-conductor-0" event={"ID":"14774d58-37e8-4e65-b0b0-d355c0070bca","Type":"ContainerStarted","Data":"33c38af311895b3ea2da41b9880bb47d91d38eadcf28dcf815946f0d92d3f569"} Dec 03 12:13:18 crc kubenswrapper[5052]: I1203 12:13:18.577748 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:13:18 crc kubenswrapper[5052]: I1203 12:13:18.581677 5052 generic.go:334] "Generic (PLEG): container finished" podID="b670a8a7-3e17-424f-9efe-18f028449ca6" containerID="7c69e8c9302ce4a5eaf0c099dd3835200c1d10a17e04ac21130275e5c3ffe1e6" exitCode=0 Dec 03 12:13:18 crc kubenswrapper[5052]: I1203 12:13:18.581709 5052 generic.go:334] "Generic (PLEG): container finished" podID="b670a8a7-3e17-424f-9efe-18f028449ca6" containerID="856e85b20893d0c08665315c89bcb7e4a614ebbfb1bffee82d066914c2afc3e3" exitCode=0 Dec 03 12:13:18 crc kubenswrapper[5052]: I1203 12:13:18.581735 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"b670a8a7-3e17-424f-9efe-18f028449ca6","Type":"ContainerDied","Data":"7c69e8c9302ce4a5eaf0c099dd3835200c1d10a17e04ac21130275e5c3ffe1e6"} Dec 03 12:13:18 crc kubenswrapper[5052]: I1203 12:13:18.581762 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"b670a8a7-3e17-424f-9efe-18f028449ca6","Type":"ContainerDied","Data":"856e85b20893d0c08665315c89bcb7e4a614ebbfb1bffee82d066914c2afc3e3"} Dec 03 12:13:18 crc kubenswrapper[5052]: I1203 12:13:18.581776 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"b670a8a7-3e17-424f-9efe-18f028449ca6","Type":"ContainerDied","Data":"e532be54ec177fd3d08a1368946f6bc7d005cbd76ac62449ee73ef96f6798ef1"} Dec 03 12:13:18 crc kubenswrapper[5052]: I1203 12:13:18.581797 5052 scope.go:117] "RemoveContainer" containerID="94de023d1837fe9f4efd468362e177ec6c30b3f05cbb4d9043728bb05ccf37a2" Dec 03 12:13:18 crc kubenswrapper[5052]: I1203 12:13:18.581828 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:13:18 crc kubenswrapper[5052]: I1203 12:13:18.586312 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/b670a8a7-3e17-424f-9efe-18f028449ca6-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:13:18 crc kubenswrapper[5052]: I1203 12:13:18.586331 5052 reconciler_common.go:293] "Volume detached for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/b670a8a7-3e17-424f-9efe-18f028449ca6-run-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:13:18 crc kubenswrapper[5052]: I1203 12:13:18.586343 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/b670a8a7-3e17-424f-9efe-18f028449ca6-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:13:18 crc kubenswrapper[5052]: I1203 12:13:18.586352 5052 reconciler_common.go:293] "Volume detached for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/b670a8a7-3e17-424f-9efe-18f028449ca6-sg-core-conf-yaml\") on node \"crc\" DevicePath \"\"" Dec 03 12:13:18 crc kubenswrapper[5052]: I1203 12:13:18.586363 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-9xhv2\" (UniqueName: \"kubernetes.io/projected/b670a8a7-3e17-424f-9efe-18f028449ca6-kube-api-access-9xhv2\") on node \"crc\" DevicePath \"\"" Dec 03 12:13:18 crc kubenswrapper[5052]: I1203 12:13:18.586373 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b670a8a7-3e17-424f-9efe-18f028449ca6-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:13:18 crc kubenswrapper[5052]: I1203 12:13:18.586386 5052 reconciler_common.go:293] "Volume detached for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/b670a8a7-3e17-424f-9efe-18f028449ca6-log-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:13:18 crc kubenswrapper[5052]: I1203 12:13:18.594688 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-cell0-conductor-0" podStartSLOduration=2.5946652329999997 podStartE2EDuration="2.594665233s" podCreationTimestamp="2025-12-03 12:13:16 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:13:18.591218134 +0000 UTC m=+3189.179459898" watchObservedRunningTime="2025-12-03 12:13:18.594665233 +0000 UTC m=+3189.182906997" Dec 03 12:13:18 crc kubenswrapper[5052]: I1203 12:13:18.609300 5052 scope.go:117] "RemoveContainer" containerID="1b70b579f4f38be27aaefeb5561c0b9527b93fc22c75c9584022b51bc6ed8608" Dec 03 12:13:18 crc kubenswrapper[5052]: I1203 12:13:18.621795 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:13:18 crc kubenswrapper[5052]: I1203 12:13:18.630224 5052 scope.go:117] "RemoveContainer" containerID="7c69e8c9302ce4a5eaf0c099dd3835200c1d10a17e04ac21130275e5c3ffe1e6" Dec 03 12:13:18 crc kubenswrapper[5052]: I1203 12:13:18.638756 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:13:18 crc kubenswrapper[5052]: I1203 12:13:18.658476 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:13:18 crc kubenswrapper[5052]: E1203 12:13:18.659153 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="b670a8a7-3e17-424f-9efe-18f028449ca6" containerName="sg-core" Dec 03 12:13:18 crc kubenswrapper[5052]: I1203 12:13:18.659172 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="b670a8a7-3e17-424f-9efe-18f028449ca6" containerName="sg-core" Dec 03 12:13:18 crc kubenswrapper[5052]: E1203 12:13:18.659192 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="b670a8a7-3e17-424f-9efe-18f028449ca6" containerName="ceilometer-central-agent" Dec 03 12:13:18 crc kubenswrapper[5052]: I1203 12:13:18.659202 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="b670a8a7-3e17-424f-9efe-18f028449ca6" containerName="ceilometer-central-agent" Dec 03 12:13:18 crc kubenswrapper[5052]: E1203 12:13:18.659216 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="b670a8a7-3e17-424f-9efe-18f028449ca6" containerName="ceilometer-notification-agent" Dec 03 12:13:18 crc kubenswrapper[5052]: I1203 12:13:18.659225 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="b670a8a7-3e17-424f-9efe-18f028449ca6" containerName="ceilometer-notification-agent" Dec 03 12:13:18 crc kubenswrapper[5052]: E1203 12:13:18.659242 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="b670a8a7-3e17-424f-9efe-18f028449ca6" containerName="proxy-httpd" Dec 03 12:13:18 crc kubenswrapper[5052]: I1203 12:13:18.659249 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="b670a8a7-3e17-424f-9efe-18f028449ca6" containerName="proxy-httpd" Dec 03 12:13:18 crc kubenswrapper[5052]: I1203 12:13:18.659733 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="b670a8a7-3e17-424f-9efe-18f028449ca6" containerName="proxy-httpd" Dec 03 12:13:18 crc kubenswrapper[5052]: I1203 12:13:18.659764 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="b670a8a7-3e17-424f-9efe-18f028449ca6" containerName="ceilometer-central-agent" Dec 03 12:13:18 crc kubenswrapper[5052]: I1203 12:13:18.659782 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="b670a8a7-3e17-424f-9efe-18f028449ca6" containerName="sg-core" Dec 03 12:13:18 crc kubenswrapper[5052]: I1203 12:13:18.659794 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="b670a8a7-3e17-424f-9efe-18f028449ca6" containerName="ceilometer-notification-agent" Dec 03 12:13:18 crc kubenswrapper[5052]: I1203 12:13:18.662226 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:13:18 crc kubenswrapper[5052]: I1203 12:13:18.665670 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-config-data" Dec 03 12:13:18 crc kubenswrapper[5052]: I1203 12:13:18.666017 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-scripts" Dec 03 12:13:18 crc kubenswrapper[5052]: I1203 12:13:18.667859 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:13:18 crc kubenswrapper[5052]: I1203 12:13:18.669752 5052 scope.go:117] "RemoveContainer" containerID="856e85b20893d0c08665315c89bcb7e4a614ebbfb1bffee82d066914c2afc3e3" Dec 03 12:13:18 crc kubenswrapper[5052]: I1203 12:13:18.690153 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/6a508669-24ef-4c02-82b4-bfb8b3b9420b-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"6a508669-24ef-4c02-82b4-bfb8b3b9420b\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:13:18 crc kubenswrapper[5052]: I1203 12:13:18.690188 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/6a508669-24ef-4c02-82b4-bfb8b3b9420b-scripts\") pod \"ceilometer-0\" (UID: \"6a508669-24ef-4c02-82b4-bfb8b3b9420b\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:13:18 crc kubenswrapper[5052]: I1203 12:13:18.690239 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6a508669-24ef-4c02-82b4-bfb8b3b9420b-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"6a508669-24ef-4c02-82b4-bfb8b3b9420b\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:13:18 crc kubenswrapper[5052]: I1203 12:13:18.690361 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6a508669-24ef-4c02-82b4-bfb8b3b9420b-config-data\") pod \"ceilometer-0\" (UID: \"6a508669-24ef-4c02-82b4-bfb8b3b9420b\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:13:18 crc kubenswrapper[5052]: I1203 12:13:18.690476 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l2ksm\" (UniqueName: \"kubernetes.io/projected/6a508669-24ef-4c02-82b4-bfb8b3b9420b-kube-api-access-l2ksm\") pod \"ceilometer-0\" (UID: \"6a508669-24ef-4c02-82b4-bfb8b3b9420b\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:13:19 crc kubenswrapper[5052]: I1203 12:13:18.690569 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/6a508669-24ef-4c02-82b4-bfb8b3b9420b-run-httpd\") pod \"ceilometer-0\" (UID: \"6a508669-24ef-4c02-82b4-bfb8b3b9420b\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:13:19 crc kubenswrapper[5052]: I1203 12:13:18.690626 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/6a508669-24ef-4c02-82b4-bfb8b3b9420b-log-httpd\") pod \"ceilometer-0\" (UID: \"6a508669-24ef-4c02-82b4-bfb8b3b9420b\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:13:19 crc kubenswrapper[5052]: I1203 12:13:18.698354 5052 scope.go:117] "RemoveContainer" containerID="94de023d1837fe9f4efd468362e177ec6c30b3f05cbb4d9043728bb05ccf37a2" Dec 03 12:13:19 crc kubenswrapper[5052]: E1203 12:13:18.698896 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"94de023d1837fe9f4efd468362e177ec6c30b3f05cbb4d9043728bb05ccf37a2\": container with ID starting with 94de023d1837fe9f4efd468362e177ec6c30b3f05cbb4d9043728bb05ccf37a2 not found: ID does not exist" containerID="94de023d1837fe9f4efd468362e177ec6c30b3f05cbb4d9043728bb05ccf37a2" Dec 03 12:13:19 crc kubenswrapper[5052]: I1203 12:13:18.698930 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"94de023d1837fe9f4efd468362e177ec6c30b3f05cbb4d9043728bb05ccf37a2"} err="failed to get container status \"94de023d1837fe9f4efd468362e177ec6c30b3f05cbb4d9043728bb05ccf37a2\": rpc error: code = NotFound desc = could not find container \"94de023d1837fe9f4efd468362e177ec6c30b3f05cbb4d9043728bb05ccf37a2\": container with ID starting with 94de023d1837fe9f4efd468362e177ec6c30b3f05cbb4d9043728bb05ccf37a2 not found: ID does not exist" Dec 03 12:13:19 crc kubenswrapper[5052]: I1203 12:13:18.698955 5052 scope.go:117] "RemoveContainer" containerID="1b70b579f4f38be27aaefeb5561c0b9527b93fc22c75c9584022b51bc6ed8608" Dec 03 12:13:19 crc kubenswrapper[5052]: E1203 12:13:18.699207 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1b70b579f4f38be27aaefeb5561c0b9527b93fc22c75c9584022b51bc6ed8608\": container with ID starting with 1b70b579f4f38be27aaefeb5561c0b9527b93fc22c75c9584022b51bc6ed8608 not found: ID does not exist" containerID="1b70b579f4f38be27aaefeb5561c0b9527b93fc22c75c9584022b51bc6ed8608" Dec 03 12:13:19 crc kubenswrapper[5052]: I1203 12:13:18.699235 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1b70b579f4f38be27aaefeb5561c0b9527b93fc22c75c9584022b51bc6ed8608"} err="failed to get container status \"1b70b579f4f38be27aaefeb5561c0b9527b93fc22c75c9584022b51bc6ed8608\": rpc error: code = NotFound desc = could not find container \"1b70b579f4f38be27aaefeb5561c0b9527b93fc22c75c9584022b51bc6ed8608\": container with ID starting with 1b70b579f4f38be27aaefeb5561c0b9527b93fc22c75c9584022b51bc6ed8608 not found: ID does not exist" Dec 03 12:13:19 crc kubenswrapper[5052]: I1203 12:13:18.699261 5052 scope.go:117] "RemoveContainer" containerID="7c69e8c9302ce4a5eaf0c099dd3835200c1d10a17e04ac21130275e5c3ffe1e6" Dec 03 12:13:19 crc kubenswrapper[5052]: E1203 12:13:18.699533 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7c69e8c9302ce4a5eaf0c099dd3835200c1d10a17e04ac21130275e5c3ffe1e6\": container with ID starting with 7c69e8c9302ce4a5eaf0c099dd3835200c1d10a17e04ac21130275e5c3ffe1e6 not found: ID does not exist" containerID="7c69e8c9302ce4a5eaf0c099dd3835200c1d10a17e04ac21130275e5c3ffe1e6" Dec 03 12:13:19 crc kubenswrapper[5052]: I1203 12:13:18.699553 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7c69e8c9302ce4a5eaf0c099dd3835200c1d10a17e04ac21130275e5c3ffe1e6"} err="failed to get container status \"7c69e8c9302ce4a5eaf0c099dd3835200c1d10a17e04ac21130275e5c3ffe1e6\": rpc error: code = NotFound desc = could not find container \"7c69e8c9302ce4a5eaf0c099dd3835200c1d10a17e04ac21130275e5c3ffe1e6\": container with ID starting with 7c69e8c9302ce4a5eaf0c099dd3835200c1d10a17e04ac21130275e5c3ffe1e6 not found: ID does not exist" Dec 03 12:13:19 crc kubenswrapper[5052]: I1203 12:13:18.699568 5052 scope.go:117] "RemoveContainer" containerID="856e85b20893d0c08665315c89bcb7e4a614ebbfb1bffee82d066914c2afc3e3" Dec 03 12:13:19 crc kubenswrapper[5052]: E1203 12:13:18.699823 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"856e85b20893d0c08665315c89bcb7e4a614ebbfb1bffee82d066914c2afc3e3\": container with ID starting with 856e85b20893d0c08665315c89bcb7e4a614ebbfb1bffee82d066914c2afc3e3 not found: ID does not exist" containerID="856e85b20893d0c08665315c89bcb7e4a614ebbfb1bffee82d066914c2afc3e3" Dec 03 12:13:19 crc kubenswrapper[5052]: I1203 12:13:18.699841 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"856e85b20893d0c08665315c89bcb7e4a614ebbfb1bffee82d066914c2afc3e3"} err="failed to get container status \"856e85b20893d0c08665315c89bcb7e4a614ebbfb1bffee82d066914c2afc3e3\": rpc error: code = NotFound desc = could not find container \"856e85b20893d0c08665315c89bcb7e4a614ebbfb1bffee82d066914c2afc3e3\": container with ID starting with 856e85b20893d0c08665315c89bcb7e4a614ebbfb1bffee82d066914c2afc3e3 not found: ID does not exist" Dec 03 12:13:19 crc kubenswrapper[5052]: I1203 12:13:18.699855 5052 scope.go:117] "RemoveContainer" containerID="94de023d1837fe9f4efd468362e177ec6c30b3f05cbb4d9043728bb05ccf37a2" Dec 03 12:13:19 crc kubenswrapper[5052]: I1203 12:13:18.700065 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"94de023d1837fe9f4efd468362e177ec6c30b3f05cbb4d9043728bb05ccf37a2"} err="failed to get container status \"94de023d1837fe9f4efd468362e177ec6c30b3f05cbb4d9043728bb05ccf37a2\": rpc error: code = NotFound desc = could not find container \"94de023d1837fe9f4efd468362e177ec6c30b3f05cbb4d9043728bb05ccf37a2\": container with ID starting with 94de023d1837fe9f4efd468362e177ec6c30b3f05cbb4d9043728bb05ccf37a2 not found: ID does not exist" Dec 03 12:13:19 crc kubenswrapper[5052]: I1203 12:13:18.700080 5052 scope.go:117] "RemoveContainer" containerID="1b70b579f4f38be27aaefeb5561c0b9527b93fc22c75c9584022b51bc6ed8608" Dec 03 12:13:19 crc kubenswrapper[5052]: I1203 12:13:18.700313 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1b70b579f4f38be27aaefeb5561c0b9527b93fc22c75c9584022b51bc6ed8608"} err="failed to get container status \"1b70b579f4f38be27aaefeb5561c0b9527b93fc22c75c9584022b51bc6ed8608\": rpc error: code = NotFound desc = could not find container \"1b70b579f4f38be27aaefeb5561c0b9527b93fc22c75c9584022b51bc6ed8608\": container with ID starting with 1b70b579f4f38be27aaefeb5561c0b9527b93fc22c75c9584022b51bc6ed8608 not found: ID does not exist" Dec 03 12:13:19 crc kubenswrapper[5052]: I1203 12:13:18.700335 5052 scope.go:117] "RemoveContainer" containerID="7c69e8c9302ce4a5eaf0c099dd3835200c1d10a17e04ac21130275e5c3ffe1e6" Dec 03 12:13:19 crc kubenswrapper[5052]: I1203 12:13:18.700583 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7c69e8c9302ce4a5eaf0c099dd3835200c1d10a17e04ac21130275e5c3ffe1e6"} err="failed to get container status \"7c69e8c9302ce4a5eaf0c099dd3835200c1d10a17e04ac21130275e5c3ffe1e6\": rpc error: code = NotFound desc = could not find container \"7c69e8c9302ce4a5eaf0c099dd3835200c1d10a17e04ac21130275e5c3ffe1e6\": container with ID starting with 7c69e8c9302ce4a5eaf0c099dd3835200c1d10a17e04ac21130275e5c3ffe1e6 not found: ID does not exist" Dec 03 12:13:19 crc kubenswrapper[5052]: I1203 12:13:18.700600 5052 scope.go:117] "RemoveContainer" containerID="856e85b20893d0c08665315c89bcb7e4a614ebbfb1bffee82d066914c2afc3e3" Dec 03 12:13:19 crc kubenswrapper[5052]: I1203 12:13:18.700812 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"856e85b20893d0c08665315c89bcb7e4a614ebbfb1bffee82d066914c2afc3e3"} err="failed to get container status \"856e85b20893d0c08665315c89bcb7e4a614ebbfb1bffee82d066914c2afc3e3\": rpc error: code = NotFound desc = could not find container \"856e85b20893d0c08665315c89bcb7e4a614ebbfb1bffee82d066914c2afc3e3\": container with ID starting with 856e85b20893d0c08665315c89bcb7e4a614ebbfb1bffee82d066914c2afc3e3 not found: ID does not exist" Dec 03 12:13:19 crc kubenswrapper[5052]: I1203 12:13:18.791863 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/6a508669-24ef-4c02-82b4-bfb8b3b9420b-run-httpd\") pod \"ceilometer-0\" (UID: \"6a508669-24ef-4c02-82b4-bfb8b3b9420b\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:13:19 crc kubenswrapper[5052]: I1203 12:13:18.791950 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/6a508669-24ef-4c02-82b4-bfb8b3b9420b-log-httpd\") pod \"ceilometer-0\" (UID: \"6a508669-24ef-4c02-82b4-bfb8b3b9420b\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:13:19 crc kubenswrapper[5052]: I1203 12:13:18.792002 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/6a508669-24ef-4c02-82b4-bfb8b3b9420b-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"6a508669-24ef-4c02-82b4-bfb8b3b9420b\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:13:19 crc kubenswrapper[5052]: I1203 12:13:18.792029 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/6a508669-24ef-4c02-82b4-bfb8b3b9420b-scripts\") pod \"ceilometer-0\" (UID: \"6a508669-24ef-4c02-82b4-bfb8b3b9420b\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:13:19 crc kubenswrapper[5052]: I1203 12:13:18.792070 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6a508669-24ef-4c02-82b4-bfb8b3b9420b-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"6a508669-24ef-4c02-82b4-bfb8b3b9420b\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:13:19 crc kubenswrapper[5052]: I1203 12:13:18.792132 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6a508669-24ef-4c02-82b4-bfb8b3b9420b-config-data\") pod \"ceilometer-0\" (UID: \"6a508669-24ef-4c02-82b4-bfb8b3b9420b\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:13:19 crc kubenswrapper[5052]: I1203 12:13:18.792162 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-l2ksm\" (UniqueName: \"kubernetes.io/projected/6a508669-24ef-4c02-82b4-bfb8b3b9420b-kube-api-access-l2ksm\") pod \"ceilometer-0\" (UID: \"6a508669-24ef-4c02-82b4-bfb8b3b9420b\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:13:19 crc kubenswrapper[5052]: I1203 12:13:18.793030 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/6a508669-24ef-4c02-82b4-bfb8b3b9420b-run-httpd\") pod \"ceilometer-0\" (UID: \"6a508669-24ef-4c02-82b4-bfb8b3b9420b\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:13:19 crc kubenswrapper[5052]: I1203 12:13:18.793282 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/6a508669-24ef-4c02-82b4-bfb8b3b9420b-log-httpd\") pod \"ceilometer-0\" (UID: \"6a508669-24ef-4c02-82b4-bfb8b3b9420b\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:13:19 crc kubenswrapper[5052]: I1203 12:13:18.797986 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/6a508669-24ef-4c02-82b4-bfb8b3b9420b-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"6a508669-24ef-4c02-82b4-bfb8b3b9420b\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:13:19 crc kubenswrapper[5052]: I1203 12:13:18.798747 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6a508669-24ef-4c02-82b4-bfb8b3b9420b-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"6a508669-24ef-4c02-82b4-bfb8b3b9420b\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:13:19 crc kubenswrapper[5052]: I1203 12:13:18.808217 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/6a508669-24ef-4c02-82b4-bfb8b3b9420b-scripts\") pod \"ceilometer-0\" (UID: \"6a508669-24ef-4c02-82b4-bfb8b3b9420b\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:13:19 crc kubenswrapper[5052]: I1203 12:13:18.810391 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6a508669-24ef-4c02-82b4-bfb8b3b9420b-config-data\") pod \"ceilometer-0\" (UID: \"6a508669-24ef-4c02-82b4-bfb8b3b9420b\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:13:19 crc kubenswrapper[5052]: I1203 12:13:18.822856 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-l2ksm\" (UniqueName: \"kubernetes.io/projected/6a508669-24ef-4c02-82b4-bfb8b3b9420b-kube-api-access-l2ksm\") pod \"ceilometer-0\" (UID: \"6a508669-24ef-4c02-82b4-bfb8b3b9420b\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:13:19 crc kubenswrapper[5052]: I1203 12:13:18.994007 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:13:19 crc kubenswrapper[5052]: I1203 12:13:19.410167 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:13:19 crc kubenswrapper[5052]: I1203 12:13:19.592492 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"6a508669-24ef-4c02-82b4-bfb8b3b9420b","Type":"ContainerStarted","Data":"87a017bf40232dce1ff49a60df7ae13dab9f9e9bdfd562c7345697313262f85a"} Dec 03 12:13:19 crc kubenswrapper[5052]: I1203 12:13:19.696540 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b670a8a7-3e17-424f-9efe-18f028449ca6" path="/var/lib/kubelet/pods/b670a8a7-3e17-424f-9efe-18f028449ca6/volumes" Dec 03 12:13:20 crc kubenswrapper[5052]: I1203 12:13:20.604793 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"6a508669-24ef-4c02-82b4-bfb8b3b9420b","Type":"ContainerStarted","Data":"f0ec59ff17f5ac98f8efe3ba2d6110db4488e144489ba6f93945be948047d889"} Dec 03 12:13:21 crc kubenswrapper[5052]: I1203 12:13:21.620303 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"6a508669-24ef-4c02-82b4-bfb8b3b9420b","Type":"ContainerStarted","Data":"12cf806b890c3285f6292098c13247c6168955b831065259ec141d41eff9f0b8"} Dec 03 12:13:21 crc kubenswrapper[5052]: I1203 12:13:21.620617 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"6a508669-24ef-4c02-82b4-bfb8b3b9420b","Type":"ContainerStarted","Data":"17d55a4176395a4183e27fbdad33c760894ac49731d57a48e1e495ee12293d29"} Dec 03 12:13:22 crc kubenswrapper[5052]: I1203 12:13:22.181921 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:13:22 crc kubenswrapper[5052]: I1203 12:13:22.656632 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell0-cell-mapping-ztgcf"] Dec 03 12:13:22 crc kubenswrapper[5052]: I1203 12:13:22.658615 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-cell-mapping-ztgcf" Dec 03 12:13:22 crc kubenswrapper[5052]: I1203 12:13:22.663597 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell0-manage-config-data" Dec 03 12:13:22 crc kubenswrapper[5052]: I1203 12:13:22.663597 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell0-manage-scripts" Dec 03 12:13:22 crc kubenswrapper[5052]: I1203 12:13:22.676619 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell0-cell-mapping-ztgcf"] Dec 03 12:13:22 crc kubenswrapper[5052]: I1203 12:13:22.756668 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/4e0e5cde-a60c-4c0e-afd7-818c9ad44c83-config-data\") pod \"nova-cell0-cell-mapping-ztgcf\" (UID: \"4e0e5cde-a60c-4c0e-afd7-818c9ad44c83\") " pod="openstack-kuttl-tests/nova-cell0-cell-mapping-ztgcf" Dec 03 12:13:22 crc kubenswrapper[5052]: I1203 12:13:22.756873 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6b42q\" (UniqueName: \"kubernetes.io/projected/4e0e5cde-a60c-4c0e-afd7-818c9ad44c83-kube-api-access-6b42q\") pod \"nova-cell0-cell-mapping-ztgcf\" (UID: \"4e0e5cde-a60c-4c0e-afd7-818c9ad44c83\") " pod="openstack-kuttl-tests/nova-cell0-cell-mapping-ztgcf" Dec 03 12:13:22 crc kubenswrapper[5052]: I1203 12:13:22.756908 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/4e0e5cde-a60c-4c0e-afd7-818c9ad44c83-scripts\") pod \"nova-cell0-cell-mapping-ztgcf\" (UID: \"4e0e5cde-a60c-4c0e-afd7-818c9ad44c83\") " pod="openstack-kuttl-tests/nova-cell0-cell-mapping-ztgcf" Dec 03 12:13:22 crc kubenswrapper[5052]: I1203 12:13:22.756938 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4e0e5cde-a60c-4c0e-afd7-818c9ad44c83-combined-ca-bundle\") pod \"nova-cell0-cell-mapping-ztgcf\" (UID: \"4e0e5cde-a60c-4c0e-afd7-818c9ad44c83\") " pod="openstack-kuttl-tests/nova-cell0-cell-mapping-ztgcf" Dec 03 12:13:22 crc kubenswrapper[5052]: I1203 12:13:22.834631 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:13:22 crc kubenswrapper[5052]: I1203 12:13:22.836852 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:13:22 crc kubenswrapper[5052]: I1203 12:13:22.839985 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-api-config-data" Dec 03 12:13:22 crc kubenswrapper[5052]: I1203 12:13:22.848948 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:13:22 crc kubenswrapper[5052]: I1203 12:13:22.860708 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4e5dd94c-ddfa-4d9e-b5af-fc890a712ea2-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"4e5dd94c-ddfa-4d9e-b5af-fc890a712ea2\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:13:22 crc kubenswrapper[5052]: I1203 12:13:22.860774 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-6b42q\" (UniqueName: \"kubernetes.io/projected/4e0e5cde-a60c-4c0e-afd7-818c9ad44c83-kube-api-access-6b42q\") pod \"nova-cell0-cell-mapping-ztgcf\" (UID: \"4e0e5cde-a60c-4c0e-afd7-818c9ad44c83\") " pod="openstack-kuttl-tests/nova-cell0-cell-mapping-ztgcf" Dec 03 12:13:22 crc kubenswrapper[5052]: I1203 12:13:22.860805 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/4e0e5cde-a60c-4c0e-afd7-818c9ad44c83-scripts\") pod \"nova-cell0-cell-mapping-ztgcf\" (UID: \"4e0e5cde-a60c-4c0e-afd7-818c9ad44c83\") " pod="openstack-kuttl-tests/nova-cell0-cell-mapping-ztgcf" Dec 03 12:13:22 crc kubenswrapper[5052]: I1203 12:13:22.860828 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4e0e5cde-a60c-4c0e-afd7-818c9ad44c83-combined-ca-bundle\") pod \"nova-cell0-cell-mapping-ztgcf\" (UID: \"4e0e5cde-a60c-4c0e-afd7-818c9ad44c83\") " pod="openstack-kuttl-tests/nova-cell0-cell-mapping-ztgcf" Dec 03 12:13:22 crc kubenswrapper[5052]: I1203 12:13:22.860856 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/4e5dd94c-ddfa-4d9e-b5af-fc890a712ea2-config-data\") pod \"nova-api-0\" (UID: \"4e5dd94c-ddfa-4d9e-b5af-fc890a712ea2\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:13:22 crc kubenswrapper[5052]: I1203 12:13:22.860932 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4vk9t\" (UniqueName: \"kubernetes.io/projected/4e5dd94c-ddfa-4d9e-b5af-fc890a712ea2-kube-api-access-4vk9t\") pod \"nova-api-0\" (UID: \"4e5dd94c-ddfa-4d9e-b5af-fc890a712ea2\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:13:22 crc kubenswrapper[5052]: I1203 12:13:22.860965 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/4e0e5cde-a60c-4c0e-afd7-818c9ad44c83-config-data\") pod \"nova-cell0-cell-mapping-ztgcf\" (UID: \"4e0e5cde-a60c-4c0e-afd7-818c9ad44c83\") " pod="openstack-kuttl-tests/nova-cell0-cell-mapping-ztgcf" Dec 03 12:13:22 crc kubenswrapper[5052]: I1203 12:13:22.861009 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/4e5dd94c-ddfa-4d9e-b5af-fc890a712ea2-logs\") pod \"nova-api-0\" (UID: \"4e5dd94c-ddfa-4d9e-b5af-fc890a712ea2\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:13:22 crc kubenswrapper[5052]: I1203 12:13:22.874176 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4e0e5cde-a60c-4c0e-afd7-818c9ad44c83-combined-ca-bundle\") pod \"nova-cell0-cell-mapping-ztgcf\" (UID: \"4e0e5cde-a60c-4c0e-afd7-818c9ad44c83\") " pod="openstack-kuttl-tests/nova-cell0-cell-mapping-ztgcf" Dec 03 12:13:22 crc kubenswrapper[5052]: I1203 12:13:22.875415 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/4e0e5cde-a60c-4c0e-afd7-818c9ad44c83-scripts\") pod \"nova-cell0-cell-mapping-ztgcf\" (UID: \"4e0e5cde-a60c-4c0e-afd7-818c9ad44c83\") " pod="openstack-kuttl-tests/nova-cell0-cell-mapping-ztgcf" Dec 03 12:13:22 crc kubenswrapper[5052]: I1203 12:13:22.875474 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/4e0e5cde-a60c-4c0e-afd7-818c9ad44c83-config-data\") pod \"nova-cell0-cell-mapping-ztgcf\" (UID: \"4e0e5cde-a60c-4c0e-afd7-818c9ad44c83\") " pod="openstack-kuttl-tests/nova-cell0-cell-mapping-ztgcf" Dec 03 12:13:22 crc kubenswrapper[5052]: I1203 12:13:22.939350 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-6b42q\" (UniqueName: \"kubernetes.io/projected/4e0e5cde-a60c-4c0e-afd7-818c9ad44c83-kube-api-access-6b42q\") pod \"nova-cell0-cell-mapping-ztgcf\" (UID: \"4e0e5cde-a60c-4c0e-afd7-818c9ad44c83\") " pod="openstack-kuttl-tests/nova-cell0-cell-mapping-ztgcf" Dec 03 12:13:22 crc kubenswrapper[5052]: I1203 12:13:22.967374 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4e5dd94c-ddfa-4d9e-b5af-fc890a712ea2-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"4e5dd94c-ddfa-4d9e-b5af-fc890a712ea2\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:13:22 crc kubenswrapper[5052]: I1203 12:13:22.967440 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/4e5dd94c-ddfa-4d9e-b5af-fc890a712ea2-config-data\") pod \"nova-api-0\" (UID: \"4e5dd94c-ddfa-4d9e-b5af-fc890a712ea2\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:13:22 crc kubenswrapper[5052]: I1203 12:13:22.967534 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-4vk9t\" (UniqueName: \"kubernetes.io/projected/4e5dd94c-ddfa-4d9e-b5af-fc890a712ea2-kube-api-access-4vk9t\") pod \"nova-api-0\" (UID: \"4e5dd94c-ddfa-4d9e-b5af-fc890a712ea2\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:13:22 crc kubenswrapper[5052]: I1203 12:13:22.967594 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/4e5dd94c-ddfa-4d9e-b5af-fc890a712ea2-logs\") pod \"nova-api-0\" (UID: \"4e5dd94c-ddfa-4d9e-b5af-fc890a712ea2\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:13:22 crc kubenswrapper[5052]: I1203 12:13:22.968057 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/4e5dd94c-ddfa-4d9e-b5af-fc890a712ea2-logs\") pod \"nova-api-0\" (UID: \"4e5dd94c-ddfa-4d9e-b5af-fc890a712ea2\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:13:22 crc kubenswrapper[5052]: I1203 12:13:22.986131 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-cell-mapping-ztgcf" Dec 03 12:13:22 crc kubenswrapper[5052]: I1203 12:13:22.987229 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/4e5dd94c-ddfa-4d9e-b5af-fc890a712ea2-config-data\") pod \"nova-api-0\" (UID: \"4e5dd94c-ddfa-4d9e-b5af-fc890a712ea2\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:13:22 crc kubenswrapper[5052]: I1203 12:13:22.987251 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4e5dd94c-ddfa-4d9e-b5af-fc890a712ea2-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"4e5dd94c-ddfa-4d9e-b5af-fc890a712ea2\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:13:23 crc kubenswrapper[5052]: I1203 12:13:23.038941 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:13:23 crc kubenswrapper[5052]: I1203 12:13:23.039207 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-4vk9t\" (UniqueName: \"kubernetes.io/projected/4e5dd94c-ddfa-4d9e-b5af-fc890a712ea2-kube-api-access-4vk9t\") pod \"nova-api-0\" (UID: \"4e5dd94c-ddfa-4d9e-b5af-fc890a712ea2\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:13:23 crc kubenswrapper[5052]: I1203 12:13:23.048873 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:13:23 crc kubenswrapper[5052]: I1203 12:13:23.076843 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-metadata-config-data" Dec 03 12:13:23 crc kubenswrapper[5052]: I1203 12:13:23.093321 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:13:23 crc kubenswrapper[5052]: I1203 12:13:23.094557 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:13:23 crc kubenswrapper[5052]: I1203 12:13:23.100020 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-scheduler-config-data" Dec 03 12:13:23 crc kubenswrapper[5052]: I1203 12:13:23.113587 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:13:23 crc kubenswrapper[5052]: I1203 12:13:23.156580 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:13:23 crc kubenswrapper[5052]: I1203 12:13:23.168402 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:13:23 crc kubenswrapper[5052]: I1203 12:13:23.169625 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 12:13:23 crc kubenswrapper[5052]: I1203 12:13:23.174985 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:13:23 crc kubenswrapper[5052]: I1203 12:13:23.182398 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6cdf956e-96ef-4642-8b28-d0d038a5b5a0-config-data\") pod \"nova-metadata-0\" (UID: \"6cdf956e-96ef-4642-8b28-d0d038a5b5a0\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:13:23 crc kubenswrapper[5052]: I1203 12:13:23.182625 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6cbzj\" (UniqueName: \"kubernetes.io/projected/6cdf956e-96ef-4642-8b28-d0d038a5b5a0-kube-api-access-6cbzj\") pod \"nova-metadata-0\" (UID: \"6cdf956e-96ef-4642-8b28-d0d038a5b5a0\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:13:23 crc kubenswrapper[5052]: I1203 12:13:23.182690 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/6cdf956e-96ef-4642-8b28-d0d038a5b5a0-logs\") pod \"nova-metadata-0\" (UID: \"6cdf956e-96ef-4642-8b28-d0d038a5b5a0\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:13:23 crc kubenswrapper[5052]: I1203 12:13:23.182730 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6cdf956e-96ef-4642-8b28-d0d038a5b5a0-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"6cdf956e-96ef-4642-8b28-d0d038a5b5a0\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:13:23 crc kubenswrapper[5052]: I1203 12:13:23.183894 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell1-novncproxy-config-data" Dec 03 12:13:23 crc kubenswrapper[5052]: I1203 12:13:23.193617 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 12:13:23 crc kubenswrapper[5052]: I1203 12:13:23.311523 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-slmxw\" (UniqueName: \"kubernetes.io/projected/6db171a5-bb58-4636-88d6-6b853614b4db-kube-api-access-slmxw\") pod \"nova-scheduler-0\" (UID: \"6db171a5-bb58-4636-88d6-6b853614b4db\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:13:23 crc kubenswrapper[5052]: I1203 12:13:23.312063 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8478e1e7-59a0-4701-aa30-ec5ef9e0578f-combined-ca-bundle\") pod \"nova-cell1-novncproxy-0\" (UID: \"8478e1e7-59a0-4701-aa30-ec5ef9e0578f\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:13:23 crc kubenswrapper[5052]: I1203 12:13:23.312225 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-6cbzj\" (UniqueName: \"kubernetes.io/projected/6cdf956e-96ef-4642-8b28-d0d038a5b5a0-kube-api-access-6cbzj\") pod \"nova-metadata-0\" (UID: \"6cdf956e-96ef-4642-8b28-d0d038a5b5a0\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:13:23 crc kubenswrapper[5052]: I1203 12:13:23.312269 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/6cdf956e-96ef-4642-8b28-d0d038a5b5a0-logs\") pod \"nova-metadata-0\" (UID: \"6cdf956e-96ef-4642-8b28-d0d038a5b5a0\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:13:23 crc kubenswrapper[5052]: I1203 12:13:23.312727 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/8478e1e7-59a0-4701-aa30-ec5ef9e0578f-config-data\") pod \"nova-cell1-novncproxy-0\" (UID: \"8478e1e7-59a0-4701-aa30-ec5ef9e0578f\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:13:23 crc kubenswrapper[5052]: I1203 12:13:23.348144 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/6cdf956e-96ef-4642-8b28-d0d038a5b5a0-logs\") pod \"nova-metadata-0\" (UID: \"6cdf956e-96ef-4642-8b28-d0d038a5b5a0\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:13:23 crc kubenswrapper[5052]: I1203 12:13:23.348730 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6cdf956e-96ef-4642-8b28-d0d038a5b5a0-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"6cdf956e-96ef-4642-8b28-d0d038a5b5a0\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:13:23 crc kubenswrapper[5052]: I1203 12:13:23.348833 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6db171a5-bb58-4636-88d6-6b853614b4db-combined-ca-bundle\") pod \"nova-scheduler-0\" (UID: \"6db171a5-bb58-4636-88d6-6b853614b4db\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:13:23 crc kubenswrapper[5052]: I1203 12:13:23.348997 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6cdf956e-96ef-4642-8b28-d0d038a5b5a0-config-data\") pod \"nova-metadata-0\" (UID: \"6cdf956e-96ef-4642-8b28-d0d038a5b5a0\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:13:23 crc kubenswrapper[5052]: I1203 12:13:23.349049 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tlds2\" (UniqueName: \"kubernetes.io/projected/8478e1e7-59a0-4701-aa30-ec5ef9e0578f-kube-api-access-tlds2\") pod \"nova-cell1-novncproxy-0\" (UID: \"8478e1e7-59a0-4701-aa30-ec5ef9e0578f\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:13:23 crc kubenswrapper[5052]: I1203 12:13:23.349122 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6db171a5-bb58-4636-88d6-6b853614b4db-config-data\") pod \"nova-scheduler-0\" (UID: \"6db171a5-bb58-4636-88d6-6b853614b4db\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:13:23 crc kubenswrapper[5052]: I1203 12:13:23.364459 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6cdf956e-96ef-4642-8b28-d0d038a5b5a0-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"6cdf956e-96ef-4642-8b28-d0d038a5b5a0\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:13:23 crc kubenswrapper[5052]: I1203 12:13:23.366360 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6cdf956e-96ef-4642-8b28-d0d038a5b5a0-config-data\") pod \"nova-metadata-0\" (UID: \"6cdf956e-96ef-4642-8b28-d0d038a5b5a0\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:13:23 crc kubenswrapper[5052]: I1203 12:13:23.379581 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-6cbzj\" (UniqueName: \"kubernetes.io/projected/6cdf956e-96ef-4642-8b28-d0d038a5b5a0-kube-api-access-6cbzj\") pod \"nova-metadata-0\" (UID: \"6cdf956e-96ef-4642-8b28-d0d038a5b5a0\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:13:23 crc kubenswrapper[5052]: I1203 12:13:23.452248 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-tlds2\" (UniqueName: \"kubernetes.io/projected/8478e1e7-59a0-4701-aa30-ec5ef9e0578f-kube-api-access-tlds2\") pod \"nova-cell1-novncproxy-0\" (UID: \"8478e1e7-59a0-4701-aa30-ec5ef9e0578f\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:13:23 crc kubenswrapper[5052]: I1203 12:13:23.452547 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6db171a5-bb58-4636-88d6-6b853614b4db-config-data\") pod \"nova-scheduler-0\" (UID: \"6db171a5-bb58-4636-88d6-6b853614b4db\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:13:23 crc kubenswrapper[5052]: I1203 12:13:23.452648 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-slmxw\" (UniqueName: \"kubernetes.io/projected/6db171a5-bb58-4636-88d6-6b853614b4db-kube-api-access-slmxw\") pod \"nova-scheduler-0\" (UID: \"6db171a5-bb58-4636-88d6-6b853614b4db\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:13:23 crc kubenswrapper[5052]: I1203 12:13:23.452679 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8478e1e7-59a0-4701-aa30-ec5ef9e0578f-combined-ca-bundle\") pod \"nova-cell1-novncproxy-0\" (UID: \"8478e1e7-59a0-4701-aa30-ec5ef9e0578f\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:13:23 crc kubenswrapper[5052]: I1203 12:13:23.452790 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/8478e1e7-59a0-4701-aa30-ec5ef9e0578f-config-data\") pod \"nova-cell1-novncproxy-0\" (UID: \"8478e1e7-59a0-4701-aa30-ec5ef9e0578f\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:13:23 crc kubenswrapper[5052]: I1203 12:13:23.452859 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6db171a5-bb58-4636-88d6-6b853614b4db-combined-ca-bundle\") pod \"nova-scheduler-0\" (UID: \"6db171a5-bb58-4636-88d6-6b853614b4db\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:13:23 crc kubenswrapper[5052]: I1203 12:13:23.460571 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8478e1e7-59a0-4701-aa30-ec5ef9e0578f-combined-ca-bundle\") pod \"nova-cell1-novncproxy-0\" (UID: \"8478e1e7-59a0-4701-aa30-ec5ef9e0578f\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:13:23 crc kubenswrapper[5052]: I1203 12:13:23.467088 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6db171a5-bb58-4636-88d6-6b853614b4db-config-data\") pod \"nova-scheduler-0\" (UID: \"6db171a5-bb58-4636-88d6-6b853614b4db\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:13:23 crc kubenswrapper[5052]: I1203 12:13:23.469233 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/8478e1e7-59a0-4701-aa30-ec5ef9e0578f-config-data\") pod \"nova-cell1-novncproxy-0\" (UID: \"8478e1e7-59a0-4701-aa30-ec5ef9e0578f\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:13:23 crc kubenswrapper[5052]: I1203 12:13:23.469957 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6db171a5-bb58-4636-88d6-6b853614b4db-combined-ca-bundle\") pod \"nova-scheduler-0\" (UID: \"6db171a5-bb58-4636-88d6-6b853614b4db\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:13:23 crc kubenswrapper[5052]: I1203 12:13:23.492178 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:13:23 crc kubenswrapper[5052]: I1203 12:13:23.508781 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-slmxw\" (UniqueName: \"kubernetes.io/projected/6db171a5-bb58-4636-88d6-6b853614b4db-kube-api-access-slmxw\") pod \"nova-scheduler-0\" (UID: \"6db171a5-bb58-4636-88d6-6b853614b4db\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:13:23 crc kubenswrapper[5052]: I1203 12:13:23.517039 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-tlds2\" (UniqueName: \"kubernetes.io/projected/8478e1e7-59a0-4701-aa30-ec5ef9e0578f-kube-api-access-tlds2\") pod \"nova-cell1-novncproxy-0\" (UID: \"8478e1e7-59a0-4701-aa30-ec5ef9e0578f\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:13:23 crc kubenswrapper[5052]: I1203 12:13:23.529407 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:13:23 crc kubenswrapper[5052]: I1203 12:13:23.592972 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:13:23 crc kubenswrapper[5052]: I1203 12:13:23.670168 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"6a508669-24ef-4c02-82b4-bfb8b3b9420b","Type":"ContainerStarted","Data":"3b54ccfe9d1b5d1b336e3adcc5013135c78f03401138c69e1aac9777fd76c5ed"} Dec 03 12:13:23 crc kubenswrapper[5052]: I1203 12:13:23.672350 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:13:23 crc kubenswrapper[5052]: I1203 12:13:23.723580 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/ceilometer-0" podStartSLOduration=2.740122853 podStartE2EDuration="5.723549688s" podCreationTimestamp="2025-12-03 12:13:18 +0000 UTC" firstStartedPulling="2025-12-03 12:13:19.420346279 +0000 UTC m=+3190.008588043" lastFinishedPulling="2025-12-03 12:13:22.403773114 +0000 UTC m=+3192.992014878" observedRunningTime="2025-12-03 12:13:23.713315856 +0000 UTC m=+3194.301557630" watchObservedRunningTime="2025-12-03 12:13:23.723549688 +0000 UTC m=+3194.311791452" Dec 03 12:13:23 crc kubenswrapper[5052]: I1203 12:13:23.817359 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell0-cell-mapping-ztgcf"] Dec 03 12:13:23 crc kubenswrapper[5052]: W1203 12:13:23.827166 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4e0e5cde_a60c_4c0e_afd7_818c9ad44c83.slice/crio-f82e124ea6fffb6b16fedc233ee0332ed1b24008ca22e1024194d86f066c10d9 WatchSource:0}: Error finding container f82e124ea6fffb6b16fedc233ee0332ed1b24008ca22e1024194d86f066c10d9: Status 404 returned error can't find the container with id f82e124ea6fffb6b16fedc233ee0332ed1b24008ca22e1024194d86f066c10d9 Dec 03 12:13:23 crc kubenswrapper[5052]: I1203 12:13:23.947002 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:13:24 crc kubenswrapper[5052]: I1203 12:13:24.129227 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-db-sync-wnzql"] Dec 03 12:13:24 crc kubenswrapper[5052]: I1203 12:13:24.130906 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-wnzql" Dec 03 12:13:24 crc kubenswrapper[5052]: I1203 12:13:24.137856 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell1-conductor-config-data" Dec 03 12:13:24 crc kubenswrapper[5052]: I1203 12:13:24.138537 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell1-conductor-scripts" Dec 03 12:13:24 crc kubenswrapper[5052]: I1203 12:13:24.158397 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-db-sync-wnzql"] Dec 03 12:13:24 crc kubenswrapper[5052]: I1203 12:13:24.188343 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:13:24 crc kubenswrapper[5052]: I1203 12:13:24.279271 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/f2a04bd0-579e-467c-b0f6-72af16ffb143-config-data\") pod \"nova-cell1-conductor-db-sync-wnzql\" (UID: \"f2a04bd0-579e-467c-b0f6-72af16ffb143\") " pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-wnzql" Dec 03 12:13:24 crc kubenswrapper[5052]: I1203 12:13:24.279363 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/f2a04bd0-579e-467c-b0f6-72af16ffb143-scripts\") pod \"nova-cell1-conductor-db-sync-wnzql\" (UID: \"f2a04bd0-579e-467c-b0f6-72af16ffb143\") " pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-wnzql" Dec 03 12:13:24 crc kubenswrapper[5052]: I1203 12:13:24.279385 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f2a04bd0-579e-467c-b0f6-72af16ffb143-combined-ca-bundle\") pod \"nova-cell1-conductor-db-sync-wnzql\" (UID: \"f2a04bd0-579e-467c-b0f6-72af16ffb143\") " pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-wnzql" Dec 03 12:13:24 crc kubenswrapper[5052]: I1203 12:13:24.279440 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r2w7p\" (UniqueName: \"kubernetes.io/projected/f2a04bd0-579e-467c-b0f6-72af16ffb143-kube-api-access-r2w7p\") pod \"nova-cell1-conductor-db-sync-wnzql\" (UID: \"f2a04bd0-579e-467c-b0f6-72af16ffb143\") " pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-wnzql" Dec 03 12:13:24 crc kubenswrapper[5052]: I1203 12:13:24.281281 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 12:13:24 crc kubenswrapper[5052]: W1203 12:13:24.283414 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8478e1e7_59a0_4701_aa30_ec5ef9e0578f.slice/crio-a69d1899a7a32af7625c549ed9c55850cf167d2920f2e9a7164067430971bcea WatchSource:0}: Error finding container a69d1899a7a32af7625c549ed9c55850cf167d2920f2e9a7164067430971bcea: Status 404 returned error can't find the container with id a69d1899a7a32af7625c549ed9c55850cf167d2920f2e9a7164067430971bcea Dec 03 12:13:24 crc kubenswrapper[5052]: I1203 12:13:24.293095 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:13:24 crc kubenswrapper[5052]: I1203 12:13:24.381610 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-r2w7p\" (UniqueName: \"kubernetes.io/projected/f2a04bd0-579e-467c-b0f6-72af16ffb143-kube-api-access-r2w7p\") pod \"nova-cell1-conductor-db-sync-wnzql\" (UID: \"f2a04bd0-579e-467c-b0f6-72af16ffb143\") " pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-wnzql" Dec 03 12:13:24 crc kubenswrapper[5052]: I1203 12:13:24.381755 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/f2a04bd0-579e-467c-b0f6-72af16ffb143-config-data\") pod \"nova-cell1-conductor-db-sync-wnzql\" (UID: \"f2a04bd0-579e-467c-b0f6-72af16ffb143\") " pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-wnzql" Dec 03 12:13:24 crc kubenswrapper[5052]: I1203 12:13:24.381833 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/f2a04bd0-579e-467c-b0f6-72af16ffb143-scripts\") pod \"nova-cell1-conductor-db-sync-wnzql\" (UID: \"f2a04bd0-579e-467c-b0f6-72af16ffb143\") " pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-wnzql" Dec 03 12:13:24 crc kubenswrapper[5052]: I1203 12:13:24.381857 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f2a04bd0-579e-467c-b0f6-72af16ffb143-combined-ca-bundle\") pod \"nova-cell1-conductor-db-sync-wnzql\" (UID: \"f2a04bd0-579e-467c-b0f6-72af16ffb143\") " pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-wnzql" Dec 03 12:13:24 crc kubenswrapper[5052]: I1203 12:13:24.388011 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/f2a04bd0-579e-467c-b0f6-72af16ffb143-scripts\") pod \"nova-cell1-conductor-db-sync-wnzql\" (UID: \"f2a04bd0-579e-467c-b0f6-72af16ffb143\") " pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-wnzql" Dec 03 12:13:24 crc kubenswrapper[5052]: I1203 12:13:24.388673 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/f2a04bd0-579e-467c-b0f6-72af16ffb143-config-data\") pod \"nova-cell1-conductor-db-sync-wnzql\" (UID: \"f2a04bd0-579e-467c-b0f6-72af16ffb143\") " pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-wnzql" Dec 03 12:13:24 crc kubenswrapper[5052]: I1203 12:13:24.392602 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f2a04bd0-579e-467c-b0f6-72af16ffb143-combined-ca-bundle\") pod \"nova-cell1-conductor-db-sync-wnzql\" (UID: \"f2a04bd0-579e-467c-b0f6-72af16ffb143\") " pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-wnzql" Dec 03 12:13:24 crc kubenswrapper[5052]: I1203 12:13:24.403466 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-r2w7p\" (UniqueName: \"kubernetes.io/projected/f2a04bd0-579e-467c-b0f6-72af16ffb143-kube-api-access-r2w7p\") pod \"nova-cell1-conductor-db-sync-wnzql\" (UID: \"f2a04bd0-579e-467c-b0f6-72af16ffb143\") " pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-wnzql" Dec 03 12:13:24 crc kubenswrapper[5052]: I1203 12:13:24.462019 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-wnzql" Dec 03 12:13:24 crc kubenswrapper[5052]: I1203 12:13:24.699566 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"6db171a5-bb58-4636-88d6-6b853614b4db","Type":"ContainerStarted","Data":"0b34a2dacc14517a7a2bb7d529e7d47c9a37b24dc2bbb3c28ba5b1768efc7a95"} Dec 03 12:13:24 crc kubenswrapper[5052]: I1203 12:13:24.699874 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"6db171a5-bb58-4636-88d6-6b853614b4db","Type":"ContainerStarted","Data":"51c910c631cf0a28f759453d76d9c12c998cfaad6d19ee63246ef4894643b265"} Dec 03 12:13:24 crc kubenswrapper[5052]: I1203 12:13:24.706665 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-cell-mapping-ztgcf" event={"ID":"4e0e5cde-a60c-4c0e-afd7-818c9ad44c83","Type":"ContainerStarted","Data":"e2af5ecd95d6d829c6c19029b5a86f5c757439ed343b6c5d710eff37b7fddd69"} Dec 03 12:13:24 crc kubenswrapper[5052]: I1203 12:13:24.706701 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-cell-mapping-ztgcf" event={"ID":"4e0e5cde-a60c-4c0e-afd7-818c9ad44c83","Type":"ContainerStarted","Data":"f82e124ea6fffb6b16fedc233ee0332ed1b24008ca22e1024194d86f066c10d9"} Dec 03 12:13:24 crc kubenswrapper[5052]: I1203 12:13:24.710006 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"4e5dd94c-ddfa-4d9e-b5af-fc890a712ea2","Type":"ContainerStarted","Data":"5b640e0fce110cba8efb298c6be8bbce1834d0dda04bf593b0a7dfc90b902797"} Dec 03 12:13:24 crc kubenswrapper[5052]: I1203 12:13:24.710040 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"4e5dd94c-ddfa-4d9e-b5af-fc890a712ea2","Type":"ContainerStarted","Data":"384172a9d89e7ce7470e93a792ed1e7f0c5dd86b20fdbac86dcf823b305d5cb6"} Dec 03 12:13:24 crc kubenswrapper[5052]: I1203 12:13:24.710054 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"4e5dd94c-ddfa-4d9e-b5af-fc890a712ea2","Type":"ContainerStarted","Data":"5b4bfad21d120433762281e0e2a1822eec048f9e35f8c8c3e513957ccd3b308e"} Dec 03 12:13:24 crc kubenswrapper[5052]: I1203 12:13:24.712557 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"6cdf956e-96ef-4642-8b28-d0d038a5b5a0","Type":"ContainerStarted","Data":"3a554dc1467cfd4a7552d8505d48a96ed9997cbdaedf25de59fca29916911491"} Dec 03 12:13:24 crc kubenswrapper[5052]: I1203 12:13:24.715667 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" event={"ID":"8478e1e7-59a0-4701-aa30-ec5ef9e0578f","Type":"ContainerStarted","Data":"a69d1899a7a32af7625c549ed9c55850cf167d2920f2e9a7164067430971bcea"} Dec 03 12:13:24 crc kubenswrapper[5052]: I1203 12:13:24.742988 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-scheduler-0" podStartSLOduration=1.742970976 podStartE2EDuration="1.742970976s" podCreationTimestamp="2025-12-03 12:13:23 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:13:24.738414056 +0000 UTC m=+3195.326655820" watchObservedRunningTime="2025-12-03 12:13:24.742970976 +0000 UTC m=+3195.331212740" Dec 03 12:13:24 crc kubenswrapper[5052]: I1203 12:13:24.765926 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-api-0" podStartSLOduration=2.765906561 podStartE2EDuration="2.765906561s" podCreationTimestamp="2025-12-03 12:13:22 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:13:24.764205692 +0000 UTC m=+3195.352447466" watchObservedRunningTime="2025-12-03 12:13:24.765906561 +0000 UTC m=+3195.354148325" Dec 03 12:13:24 crc kubenswrapper[5052]: I1203 12:13:24.806968 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-cell0-cell-mapping-ztgcf" podStartSLOduration=2.8069435130000002 podStartE2EDuration="2.806943513s" podCreationTimestamp="2025-12-03 12:13:22 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:13:24.782160445 +0000 UTC m=+3195.370402209" watchObservedRunningTime="2025-12-03 12:13:24.806943513 +0000 UTC m=+3195.395185287" Dec 03 12:13:25 crc kubenswrapper[5052]: I1203 12:13:25.073664 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-db-sync-wnzql"] Dec 03 12:13:25 crc kubenswrapper[5052]: I1203 12:13:25.743739 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-wnzql" event={"ID":"f2a04bd0-579e-467c-b0f6-72af16ffb143","Type":"ContainerStarted","Data":"624a18da80d875d17af9c7c2a3b638755eddc32b1144b57ac4834a2414178324"} Dec 03 12:13:25 crc kubenswrapper[5052]: I1203 12:13:25.746053 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-wnzql" event={"ID":"f2a04bd0-579e-467c-b0f6-72af16ffb143","Type":"ContainerStarted","Data":"db4efc30d6404712f1a7ed0a00d444495fb6a372845a9022993f2afc0d9195ee"} Dec 03 12:13:25 crc kubenswrapper[5052]: I1203 12:13:25.749269 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"6cdf956e-96ef-4642-8b28-d0d038a5b5a0","Type":"ContainerStarted","Data":"ffd81de11b848a78ca787b4ebca75d051943604b73c4d666c7d7d5f23e4c0374"} Dec 03 12:13:25 crc kubenswrapper[5052]: I1203 12:13:25.749334 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"6cdf956e-96ef-4642-8b28-d0d038a5b5a0","Type":"ContainerStarted","Data":"5b90529341092b6a7a729daba15d9847ded530bc72209e11a3d7985b6f1a8029"} Dec 03 12:13:25 crc kubenswrapper[5052]: I1203 12:13:25.777792 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" event={"ID":"8478e1e7-59a0-4701-aa30-ec5ef9e0578f","Type":"ContainerStarted","Data":"7719bf7efc2ec847a05855736e778e85606a65b98e843dbddd60067e15f34b31"} Dec 03 12:13:25 crc kubenswrapper[5052]: I1203 12:13:25.804072 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" podStartSLOduration=2.804055482 podStartE2EDuration="2.804055482s" podCreationTimestamp="2025-12-03 12:13:23 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:13:25.798595926 +0000 UTC m=+3196.386837700" watchObservedRunningTime="2025-12-03 12:13:25.804055482 +0000 UTC m=+3196.392297246" Dec 03 12:13:26 crc kubenswrapper[5052]: I1203 12:13:26.078450 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:13:26 crc kubenswrapper[5052]: I1203 12:13:26.098443 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 12:13:26 crc kubenswrapper[5052]: I1203 12:13:26.842979 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-metadata-0" podStartSLOduration=4.842960257 podStartE2EDuration="4.842960257s" podCreationTimestamp="2025-12-03 12:13:22 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:13:26.841090423 +0000 UTC m=+3197.429332187" watchObservedRunningTime="2025-12-03 12:13:26.842960257 +0000 UTC m=+3197.431202021" Dec 03 12:13:26 crc kubenswrapper[5052]: I1203 12:13:26.876599 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-wnzql" podStartSLOduration=2.876496934 podStartE2EDuration="2.876496934s" podCreationTimestamp="2025-12-03 12:13:24 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:13:26.872126309 +0000 UTC m=+3197.460368073" watchObservedRunningTime="2025-12-03 12:13:26.876496934 +0000 UTC m=+3197.464738698" Dec 03 12:13:27 crc kubenswrapper[5052]: I1203 12:13:27.673190 5052 scope.go:117] "RemoveContainer" containerID="63298107b3d090deff9ed92f3d613e5b1c86d9137684d117500257db8ee1c805" Dec 03 12:13:27 crc kubenswrapper[5052]: E1203 12:13:27.673785 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 12:13:27 crc kubenswrapper[5052]: I1203 12:13:27.796936 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-metadata-0" podUID="6cdf956e-96ef-4642-8b28-d0d038a5b5a0" containerName="nova-metadata-log" containerID="cri-o://5b90529341092b6a7a729daba15d9847ded530bc72209e11a3d7985b6f1a8029" gracePeriod=30 Dec 03 12:13:27 crc kubenswrapper[5052]: I1203 12:13:27.796969 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-metadata-0" podUID="6cdf956e-96ef-4642-8b28-d0d038a5b5a0" containerName="nova-metadata-metadata" containerID="cri-o://ffd81de11b848a78ca787b4ebca75d051943604b73c4d666c7d7d5f23e4c0374" gracePeriod=30 Dec 03 12:13:27 crc kubenswrapper[5052]: I1203 12:13:27.797009 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" podUID="8478e1e7-59a0-4701-aa30-ec5ef9e0578f" containerName="nova-cell1-novncproxy-novncproxy" containerID="cri-o://7719bf7efc2ec847a05855736e778e85606a65b98e843dbddd60067e15f34b31" gracePeriod=30 Dec 03 12:13:28 crc kubenswrapper[5052]: I1203 12:13:28.492481 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:13:28 crc kubenswrapper[5052]: I1203 12:13:28.492802 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:13:28 crc kubenswrapper[5052]: I1203 12:13:28.529793 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:13:28 crc kubenswrapper[5052]: I1203 12:13:28.595831 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:13:28 crc kubenswrapper[5052]: I1203 12:13:28.788799 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:13:28 crc kubenswrapper[5052]: I1203 12:13:28.813304 5052 generic.go:334] "Generic (PLEG): container finished" podID="8478e1e7-59a0-4701-aa30-ec5ef9e0578f" containerID="7719bf7efc2ec847a05855736e778e85606a65b98e843dbddd60067e15f34b31" exitCode=0 Dec 03 12:13:28 crc kubenswrapper[5052]: I1203 12:13:28.813382 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" event={"ID":"8478e1e7-59a0-4701-aa30-ec5ef9e0578f","Type":"ContainerDied","Data":"7719bf7efc2ec847a05855736e778e85606a65b98e843dbddd60067e15f34b31"} Dec 03 12:13:28 crc kubenswrapper[5052]: I1203 12:13:28.815851 5052 generic.go:334] "Generic (PLEG): container finished" podID="6cdf956e-96ef-4642-8b28-d0d038a5b5a0" containerID="ffd81de11b848a78ca787b4ebca75d051943604b73c4d666c7d7d5f23e4c0374" exitCode=0 Dec 03 12:13:28 crc kubenswrapper[5052]: I1203 12:13:28.815997 5052 generic.go:334] "Generic (PLEG): container finished" podID="6cdf956e-96ef-4642-8b28-d0d038a5b5a0" containerID="5b90529341092b6a7a729daba15d9847ded530bc72209e11a3d7985b6f1a8029" exitCode=143 Dec 03 12:13:28 crc kubenswrapper[5052]: I1203 12:13:28.816030 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"6cdf956e-96ef-4642-8b28-d0d038a5b5a0","Type":"ContainerDied","Data":"ffd81de11b848a78ca787b4ebca75d051943604b73c4d666c7d7d5f23e4c0374"} Dec 03 12:13:28 crc kubenswrapper[5052]: I1203 12:13:28.816068 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"6cdf956e-96ef-4642-8b28-d0d038a5b5a0","Type":"ContainerDied","Data":"5b90529341092b6a7a729daba15d9847ded530bc72209e11a3d7985b6f1a8029"} Dec 03 12:13:28 crc kubenswrapper[5052]: I1203 12:13:28.816081 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"6cdf956e-96ef-4642-8b28-d0d038a5b5a0","Type":"ContainerDied","Data":"3a554dc1467cfd4a7552d8505d48a96ed9997cbdaedf25de59fca29916911491"} Dec 03 12:13:28 crc kubenswrapper[5052]: I1203 12:13:28.816100 5052 scope.go:117] "RemoveContainer" containerID="ffd81de11b848a78ca787b4ebca75d051943604b73c4d666c7d7d5f23e4c0374" Dec 03 12:13:28 crc kubenswrapper[5052]: I1203 12:13:28.816174 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:13:28 crc kubenswrapper[5052]: I1203 12:13:28.843281 5052 scope.go:117] "RemoveContainer" containerID="5b90529341092b6a7a729daba15d9847ded530bc72209e11a3d7985b6f1a8029" Dec 03 12:13:28 crc kubenswrapper[5052]: I1203 12:13:28.865781 5052 scope.go:117] "RemoveContainer" containerID="ffd81de11b848a78ca787b4ebca75d051943604b73c4d666c7d7d5f23e4c0374" Dec 03 12:13:28 crc kubenswrapper[5052]: E1203 12:13:28.866877 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ffd81de11b848a78ca787b4ebca75d051943604b73c4d666c7d7d5f23e4c0374\": container with ID starting with ffd81de11b848a78ca787b4ebca75d051943604b73c4d666c7d7d5f23e4c0374 not found: ID does not exist" containerID="ffd81de11b848a78ca787b4ebca75d051943604b73c4d666c7d7d5f23e4c0374" Dec 03 12:13:28 crc kubenswrapper[5052]: I1203 12:13:28.866945 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ffd81de11b848a78ca787b4ebca75d051943604b73c4d666c7d7d5f23e4c0374"} err="failed to get container status \"ffd81de11b848a78ca787b4ebca75d051943604b73c4d666c7d7d5f23e4c0374\": rpc error: code = NotFound desc = could not find container \"ffd81de11b848a78ca787b4ebca75d051943604b73c4d666c7d7d5f23e4c0374\": container with ID starting with ffd81de11b848a78ca787b4ebca75d051943604b73c4d666c7d7d5f23e4c0374 not found: ID does not exist" Dec 03 12:13:28 crc kubenswrapper[5052]: I1203 12:13:28.866992 5052 scope.go:117] "RemoveContainer" containerID="5b90529341092b6a7a729daba15d9847ded530bc72209e11a3d7985b6f1a8029" Dec 03 12:13:28 crc kubenswrapper[5052]: E1203 12:13:28.867491 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5b90529341092b6a7a729daba15d9847ded530bc72209e11a3d7985b6f1a8029\": container with ID starting with 5b90529341092b6a7a729daba15d9847ded530bc72209e11a3d7985b6f1a8029 not found: ID does not exist" containerID="5b90529341092b6a7a729daba15d9847ded530bc72209e11a3d7985b6f1a8029" Dec 03 12:13:28 crc kubenswrapper[5052]: I1203 12:13:28.867537 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5b90529341092b6a7a729daba15d9847ded530bc72209e11a3d7985b6f1a8029"} err="failed to get container status \"5b90529341092b6a7a729daba15d9847ded530bc72209e11a3d7985b6f1a8029\": rpc error: code = NotFound desc = could not find container \"5b90529341092b6a7a729daba15d9847ded530bc72209e11a3d7985b6f1a8029\": container with ID starting with 5b90529341092b6a7a729daba15d9847ded530bc72209e11a3d7985b6f1a8029 not found: ID does not exist" Dec 03 12:13:28 crc kubenswrapper[5052]: I1203 12:13:28.867563 5052 scope.go:117] "RemoveContainer" containerID="ffd81de11b848a78ca787b4ebca75d051943604b73c4d666c7d7d5f23e4c0374" Dec 03 12:13:28 crc kubenswrapper[5052]: I1203 12:13:28.867838 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ffd81de11b848a78ca787b4ebca75d051943604b73c4d666c7d7d5f23e4c0374"} err="failed to get container status \"ffd81de11b848a78ca787b4ebca75d051943604b73c4d666c7d7d5f23e4c0374\": rpc error: code = NotFound desc = could not find container \"ffd81de11b848a78ca787b4ebca75d051943604b73c4d666c7d7d5f23e4c0374\": container with ID starting with ffd81de11b848a78ca787b4ebca75d051943604b73c4d666c7d7d5f23e4c0374 not found: ID does not exist" Dec 03 12:13:28 crc kubenswrapper[5052]: I1203 12:13:28.867883 5052 scope.go:117] "RemoveContainer" containerID="5b90529341092b6a7a729daba15d9847ded530bc72209e11a3d7985b6f1a8029" Dec 03 12:13:28 crc kubenswrapper[5052]: I1203 12:13:28.868270 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5b90529341092b6a7a729daba15d9847ded530bc72209e11a3d7985b6f1a8029"} err="failed to get container status \"5b90529341092b6a7a729daba15d9847ded530bc72209e11a3d7985b6f1a8029\": rpc error: code = NotFound desc = could not find container \"5b90529341092b6a7a729daba15d9847ded530bc72209e11a3d7985b6f1a8029\": container with ID starting with 5b90529341092b6a7a729daba15d9847ded530bc72209e11a3d7985b6f1a8029 not found: ID does not exist" Dec 03 12:13:28 crc kubenswrapper[5052]: I1203 12:13:28.979013 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6cdf956e-96ef-4642-8b28-d0d038a5b5a0-combined-ca-bundle\") pod \"6cdf956e-96ef-4642-8b28-d0d038a5b5a0\" (UID: \"6cdf956e-96ef-4642-8b28-d0d038a5b5a0\") " Dec 03 12:13:28 crc kubenswrapper[5052]: I1203 12:13:28.979109 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/6cdf956e-96ef-4642-8b28-d0d038a5b5a0-logs\") pod \"6cdf956e-96ef-4642-8b28-d0d038a5b5a0\" (UID: \"6cdf956e-96ef-4642-8b28-d0d038a5b5a0\") " Dec 03 12:13:28 crc kubenswrapper[5052]: I1203 12:13:28.980358 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6cbzj\" (UniqueName: \"kubernetes.io/projected/6cdf956e-96ef-4642-8b28-d0d038a5b5a0-kube-api-access-6cbzj\") pod \"6cdf956e-96ef-4642-8b28-d0d038a5b5a0\" (UID: \"6cdf956e-96ef-4642-8b28-d0d038a5b5a0\") " Dec 03 12:13:28 crc kubenswrapper[5052]: I1203 12:13:28.980396 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6cdf956e-96ef-4642-8b28-d0d038a5b5a0-config-data\") pod \"6cdf956e-96ef-4642-8b28-d0d038a5b5a0\" (UID: \"6cdf956e-96ef-4642-8b28-d0d038a5b5a0\") " Dec 03 12:13:28 crc kubenswrapper[5052]: I1203 12:13:28.981382 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6cdf956e-96ef-4642-8b28-d0d038a5b5a0-logs" (OuterVolumeSpecName: "logs") pod "6cdf956e-96ef-4642-8b28-d0d038a5b5a0" (UID: "6cdf956e-96ef-4642-8b28-d0d038a5b5a0"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:13:29 crc kubenswrapper[5052]: I1203 12:13:28.996168 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6cdf956e-96ef-4642-8b28-d0d038a5b5a0-kube-api-access-6cbzj" (OuterVolumeSpecName: "kube-api-access-6cbzj") pod "6cdf956e-96ef-4642-8b28-d0d038a5b5a0" (UID: "6cdf956e-96ef-4642-8b28-d0d038a5b5a0"). InnerVolumeSpecName "kube-api-access-6cbzj". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:13:29 crc kubenswrapper[5052]: I1203 12:13:29.021293 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6cdf956e-96ef-4642-8b28-d0d038a5b5a0-config-data" (OuterVolumeSpecName: "config-data") pod "6cdf956e-96ef-4642-8b28-d0d038a5b5a0" (UID: "6cdf956e-96ef-4642-8b28-d0d038a5b5a0"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:13:29 crc kubenswrapper[5052]: I1203 12:13:29.024599 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6cdf956e-96ef-4642-8b28-d0d038a5b5a0-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "6cdf956e-96ef-4642-8b28-d0d038a5b5a0" (UID: "6cdf956e-96ef-4642-8b28-d0d038a5b5a0"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:13:29 crc kubenswrapper[5052]: I1203 12:13:29.081688 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6cdf956e-96ef-4642-8b28-d0d038a5b5a0-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:13:29 crc kubenswrapper[5052]: I1203 12:13:29.081715 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/6cdf956e-96ef-4642-8b28-d0d038a5b5a0-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:13:29 crc kubenswrapper[5052]: I1203 12:13:29.081734 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-6cbzj\" (UniqueName: \"kubernetes.io/projected/6cdf956e-96ef-4642-8b28-d0d038a5b5a0-kube-api-access-6cbzj\") on node \"crc\" DevicePath \"\"" Dec 03 12:13:29 crc kubenswrapper[5052]: I1203 12:13:29.081744 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6cdf956e-96ef-4642-8b28-d0d038a5b5a0-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:13:29 crc kubenswrapper[5052]: I1203 12:13:29.152491 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:13:29 crc kubenswrapper[5052]: I1203 12:13:29.166259 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:13:29 crc kubenswrapper[5052]: I1203 12:13:29.180951 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:13:29 crc kubenswrapper[5052]: E1203 12:13:29.181466 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="6cdf956e-96ef-4642-8b28-d0d038a5b5a0" containerName="nova-metadata-metadata" Dec 03 12:13:29 crc kubenswrapper[5052]: I1203 12:13:29.181490 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="6cdf956e-96ef-4642-8b28-d0d038a5b5a0" containerName="nova-metadata-metadata" Dec 03 12:13:29 crc kubenswrapper[5052]: E1203 12:13:29.181532 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="6cdf956e-96ef-4642-8b28-d0d038a5b5a0" containerName="nova-metadata-log" Dec 03 12:13:29 crc kubenswrapper[5052]: I1203 12:13:29.181541 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="6cdf956e-96ef-4642-8b28-d0d038a5b5a0" containerName="nova-metadata-log" Dec 03 12:13:29 crc kubenswrapper[5052]: I1203 12:13:29.181790 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="6cdf956e-96ef-4642-8b28-d0d038a5b5a0" containerName="nova-metadata-metadata" Dec 03 12:13:29 crc kubenswrapper[5052]: I1203 12:13:29.181809 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="6cdf956e-96ef-4642-8b28-d0d038a5b5a0" containerName="nova-metadata-log" Dec 03 12:13:29 crc kubenswrapper[5052]: I1203 12:13:29.184836 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:13:29 crc kubenswrapper[5052]: I1203 12:13:29.187275 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-nova-metadata-internal-svc" Dec 03 12:13:29 crc kubenswrapper[5052]: I1203 12:13:29.188383 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-metadata-config-data" Dec 03 12:13:29 crc kubenswrapper[5052]: I1203 12:13:29.229555 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:13:29 crc kubenswrapper[5052]: I1203 12:13:29.285430 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9092923f-ae28-4bf1-84d2-5fd6b36a3cd1-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"9092923f-ae28-4bf1-84d2-5fd6b36a3cd1\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:13:29 crc kubenswrapper[5052]: I1203 12:13:29.285488 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/9092923f-ae28-4bf1-84d2-5fd6b36a3cd1-logs\") pod \"nova-metadata-0\" (UID: \"9092923f-ae28-4bf1-84d2-5fd6b36a3cd1\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:13:29 crc kubenswrapper[5052]: I1203 12:13:29.286215 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-56w5z\" (UniqueName: \"kubernetes.io/projected/9092923f-ae28-4bf1-84d2-5fd6b36a3cd1-kube-api-access-56w5z\") pod \"nova-metadata-0\" (UID: \"9092923f-ae28-4bf1-84d2-5fd6b36a3cd1\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:13:29 crc kubenswrapper[5052]: I1203 12:13:29.286281 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/9092923f-ae28-4bf1-84d2-5fd6b36a3cd1-config-data\") pod \"nova-metadata-0\" (UID: \"9092923f-ae28-4bf1-84d2-5fd6b36a3cd1\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:13:29 crc kubenswrapper[5052]: I1203 12:13:29.286336 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/9092923f-ae28-4bf1-84d2-5fd6b36a3cd1-nova-metadata-tls-certs\") pod \"nova-metadata-0\" (UID: \"9092923f-ae28-4bf1-84d2-5fd6b36a3cd1\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:13:29 crc kubenswrapper[5052]: I1203 12:13:29.387698 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/9092923f-ae28-4bf1-84d2-5fd6b36a3cd1-nova-metadata-tls-certs\") pod \"nova-metadata-0\" (UID: \"9092923f-ae28-4bf1-84d2-5fd6b36a3cd1\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:13:29 crc kubenswrapper[5052]: I1203 12:13:29.387784 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9092923f-ae28-4bf1-84d2-5fd6b36a3cd1-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"9092923f-ae28-4bf1-84d2-5fd6b36a3cd1\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:13:29 crc kubenswrapper[5052]: I1203 12:13:29.387818 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/9092923f-ae28-4bf1-84d2-5fd6b36a3cd1-logs\") pod \"nova-metadata-0\" (UID: \"9092923f-ae28-4bf1-84d2-5fd6b36a3cd1\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:13:29 crc kubenswrapper[5052]: I1203 12:13:29.387893 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-56w5z\" (UniqueName: \"kubernetes.io/projected/9092923f-ae28-4bf1-84d2-5fd6b36a3cd1-kube-api-access-56w5z\") pod \"nova-metadata-0\" (UID: \"9092923f-ae28-4bf1-84d2-5fd6b36a3cd1\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:13:29 crc kubenswrapper[5052]: I1203 12:13:29.387929 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/9092923f-ae28-4bf1-84d2-5fd6b36a3cd1-config-data\") pod \"nova-metadata-0\" (UID: \"9092923f-ae28-4bf1-84d2-5fd6b36a3cd1\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:13:29 crc kubenswrapper[5052]: I1203 12:13:29.388635 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/9092923f-ae28-4bf1-84d2-5fd6b36a3cd1-logs\") pod \"nova-metadata-0\" (UID: \"9092923f-ae28-4bf1-84d2-5fd6b36a3cd1\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:13:29 crc kubenswrapper[5052]: I1203 12:13:29.392229 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/9092923f-ae28-4bf1-84d2-5fd6b36a3cd1-nova-metadata-tls-certs\") pod \"nova-metadata-0\" (UID: \"9092923f-ae28-4bf1-84d2-5fd6b36a3cd1\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:13:29 crc kubenswrapper[5052]: I1203 12:13:29.392246 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/9092923f-ae28-4bf1-84d2-5fd6b36a3cd1-config-data\") pod \"nova-metadata-0\" (UID: \"9092923f-ae28-4bf1-84d2-5fd6b36a3cd1\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:13:29 crc kubenswrapper[5052]: I1203 12:13:29.392392 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9092923f-ae28-4bf1-84d2-5fd6b36a3cd1-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"9092923f-ae28-4bf1-84d2-5fd6b36a3cd1\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:13:29 crc kubenswrapper[5052]: I1203 12:13:29.415144 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-56w5z\" (UniqueName: \"kubernetes.io/projected/9092923f-ae28-4bf1-84d2-5fd6b36a3cd1-kube-api-access-56w5z\") pod \"nova-metadata-0\" (UID: \"9092923f-ae28-4bf1-84d2-5fd6b36a3cd1\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:13:29 crc kubenswrapper[5052]: I1203 12:13:29.470226 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:13:29 crc kubenswrapper[5052]: I1203 12:13:29.523082 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:13:29 crc kubenswrapper[5052]: I1203 12:13:29.590576 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tlds2\" (UniqueName: \"kubernetes.io/projected/8478e1e7-59a0-4701-aa30-ec5ef9e0578f-kube-api-access-tlds2\") pod \"8478e1e7-59a0-4701-aa30-ec5ef9e0578f\" (UID: \"8478e1e7-59a0-4701-aa30-ec5ef9e0578f\") " Dec 03 12:13:29 crc kubenswrapper[5052]: I1203 12:13:29.590793 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8478e1e7-59a0-4701-aa30-ec5ef9e0578f-combined-ca-bundle\") pod \"8478e1e7-59a0-4701-aa30-ec5ef9e0578f\" (UID: \"8478e1e7-59a0-4701-aa30-ec5ef9e0578f\") " Dec 03 12:13:29 crc kubenswrapper[5052]: I1203 12:13:29.590825 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/8478e1e7-59a0-4701-aa30-ec5ef9e0578f-config-data\") pod \"8478e1e7-59a0-4701-aa30-ec5ef9e0578f\" (UID: \"8478e1e7-59a0-4701-aa30-ec5ef9e0578f\") " Dec 03 12:13:29 crc kubenswrapper[5052]: I1203 12:13:29.598396 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8478e1e7-59a0-4701-aa30-ec5ef9e0578f-kube-api-access-tlds2" (OuterVolumeSpecName: "kube-api-access-tlds2") pod "8478e1e7-59a0-4701-aa30-ec5ef9e0578f" (UID: "8478e1e7-59a0-4701-aa30-ec5ef9e0578f"). InnerVolumeSpecName "kube-api-access-tlds2". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:13:29 crc kubenswrapper[5052]: I1203 12:13:29.642763 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8478e1e7-59a0-4701-aa30-ec5ef9e0578f-config-data" (OuterVolumeSpecName: "config-data") pod "8478e1e7-59a0-4701-aa30-ec5ef9e0578f" (UID: "8478e1e7-59a0-4701-aa30-ec5ef9e0578f"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:13:29 crc kubenswrapper[5052]: I1203 12:13:29.660323 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8478e1e7-59a0-4701-aa30-ec5ef9e0578f-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "8478e1e7-59a0-4701-aa30-ec5ef9e0578f" (UID: "8478e1e7-59a0-4701-aa30-ec5ef9e0578f"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:13:29 crc kubenswrapper[5052]: I1203 12:13:29.690801 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6cdf956e-96ef-4642-8b28-d0d038a5b5a0" path="/var/lib/kubelet/pods/6cdf956e-96ef-4642-8b28-d0d038a5b5a0/volumes" Dec 03 12:13:29 crc kubenswrapper[5052]: I1203 12:13:29.694157 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8478e1e7-59a0-4701-aa30-ec5ef9e0578f-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:13:29 crc kubenswrapper[5052]: I1203 12:13:29.694408 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/8478e1e7-59a0-4701-aa30-ec5ef9e0578f-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:13:29 crc kubenswrapper[5052]: I1203 12:13:29.694469 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-tlds2\" (UniqueName: \"kubernetes.io/projected/8478e1e7-59a0-4701-aa30-ec5ef9e0578f-kube-api-access-tlds2\") on node \"crc\" DevicePath \"\"" Dec 03 12:13:29 crc kubenswrapper[5052]: I1203 12:13:29.831254 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" event={"ID":"8478e1e7-59a0-4701-aa30-ec5ef9e0578f","Type":"ContainerDied","Data":"a69d1899a7a32af7625c549ed9c55850cf167d2920f2e9a7164067430971bcea"} Dec 03 12:13:29 crc kubenswrapper[5052]: I1203 12:13:29.831310 5052 scope.go:117] "RemoveContainer" containerID="7719bf7efc2ec847a05855736e778e85606a65b98e843dbddd60067e15f34b31" Dec 03 12:13:29 crc kubenswrapper[5052]: I1203 12:13:29.831470 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:13:29 crc kubenswrapper[5052]: I1203 12:13:29.860475 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 12:13:29 crc kubenswrapper[5052]: I1203 12:13:29.879380 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 12:13:29 crc kubenswrapper[5052]: I1203 12:13:29.889634 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 12:13:29 crc kubenswrapper[5052]: E1203 12:13:29.890116 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="8478e1e7-59a0-4701-aa30-ec5ef9e0578f" containerName="nova-cell1-novncproxy-novncproxy" Dec 03 12:13:29 crc kubenswrapper[5052]: I1203 12:13:29.890129 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="8478e1e7-59a0-4701-aa30-ec5ef9e0578f" containerName="nova-cell1-novncproxy-novncproxy" Dec 03 12:13:29 crc kubenswrapper[5052]: I1203 12:13:29.890325 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="8478e1e7-59a0-4701-aa30-ec5ef9e0578f" containerName="nova-cell1-novncproxy-novncproxy" Dec 03 12:13:29 crc kubenswrapper[5052]: I1203 12:13:29.891055 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:13:29 crc kubenswrapper[5052]: I1203 12:13:29.893886 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-nova-novncproxy-cell1-vencrypt" Dec 03 12:13:29 crc kubenswrapper[5052]: I1203 12:13:29.893989 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell1-novncproxy-config-data" Dec 03 12:13:29 crc kubenswrapper[5052]: I1203 12:13:29.894182 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-nova-novncproxy-cell1-public-svc" Dec 03 12:13:29 crc kubenswrapper[5052]: I1203 12:13:29.902461 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 12:13:29 crc kubenswrapper[5052]: I1203 12:13:29.998827 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vvbz9\" (UniqueName: \"kubernetes.io/projected/a15b6144-952c-4d41-a141-5bca5c7fd402-kube-api-access-vvbz9\") pod \"nova-cell1-novncproxy-0\" (UID: \"a15b6144-952c-4d41-a141-5bca5c7fd402\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:13:29 crc kubenswrapper[5052]: I1203 12:13:29.998889 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"vencrypt-tls-certs\" (UniqueName: \"kubernetes.io/secret/a15b6144-952c-4d41-a141-5bca5c7fd402-vencrypt-tls-certs\") pod \"nova-cell1-novncproxy-0\" (UID: \"a15b6144-952c-4d41-a141-5bca5c7fd402\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:13:29 crc kubenswrapper[5052]: I1203 12:13:29.998974 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nova-novncproxy-tls-certs\" (UniqueName: \"kubernetes.io/secret/a15b6144-952c-4d41-a141-5bca5c7fd402-nova-novncproxy-tls-certs\") pod \"nova-cell1-novncproxy-0\" (UID: \"a15b6144-952c-4d41-a141-5bca5c7fd402\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:13:29 crc kubenswrapper[5052]: I1203 12:13:29.999029 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a15b6144-952c-4d41-a141-5bca5c7fd402-config-data\") pod \"nova-cell1-novncproxy-0\" (UID: \"a15b6144-952c-4d41-a141-5bca5c7fd402\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:13:29 crc kubenswrapper[5052]: I1203 12:13:29.999050 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a15b6144-952c-4d41-a141-5bca5c7fd402-combined-ca-bundle\") pod \"nova-cell1-novncproxy-0\" (UID: \"a15b6144-952c-4d41-a141-5bca5c7fd402\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:13:30 crc kubenswrapper[5052]: W1203 12:13:30.031954 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9092923f_ae28_4bf1_84d2_5fd6b36a3cd1.slice/crio-3e5f60cd30222efb2f1421cce88978c7b0522cccfd75b282ff723fc016c51218 WatchSource:0}: Error finding container 3e5f60cd30222efb2f1421cce88978c7b0522cccfd75b282ff723fc016c51218: Status 404 returned error can't find the container with id 3e5f60cd30222efb2f1421cce88978c7b0522cccfd75b282ff723fc016c51218 Dec 03 12:13:30 crc kubenswrapper[5052]: I1203 12:13:30.036097 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:13:30 crc kubenswrapper[5052]: I1203 12:13:30.100654 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-vvbz9\" (UniqueName: \"kubernetes.io/projected/a15b6144-952c-4d41-a141-5bca5c7fd402-kube-api-access-vvbz9\") pod \"nova-cell1-novncproxy-0\" (UID: \"a15b6144-952c-4d41-a141-5bca5c7fd402\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:13:30 crc kubenswrapper[5052]: I1203 12:13:30.100713 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"vencrypt-tls-certs\" (UniqueName: \"kubernetes.io/secret/a15b6144-952c-4d41-a141-5bca5c7fd402-vencrypt-tls-certs\") pod \"nova-cell1-novncproxy-0\" (UID: \"a15b6144-952c-4d41-a141-5bca5c7fd402\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:13:30 crc kubenswrapper[5052]: I1203 12:13:30.100763 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"nova-novncproxy-tls-certs\" (UniqueName: \"kubernetes.io/secret/a15b6144-952c-4d41-a141-5bca5c7fd402-nova-novncproxy-tls-certs\") pod \"nova-cell1-novncproxy-0\" (UID: \"a15b6144-952c-4d41-a141-5bca5c7fd402\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:13:30 crc kubenswrapper[5052]: I1203 12:13:30.100816 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a15b6144-952c-4d41-a141-5bca5c7fd402-config-data\") pod \"nova-cell1-novncproxy-0\" (UID: \"a15b6144-952c-4d41-a141-5bca5c7fd402\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:13:30 crc kubenswrapper[5052]: I1203 12:13:30.100834 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a15b6144-952c-4d41-a141-5bca5c7fd402-combined-ca-bundle\") pod \"nova-cell1-novncproxy-0\" (UID: \"a15b6144-952c-4d41-a141-5bca5c7fd402\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:13:30 crc kubenswrapper[5052]: I1203 12:13:30.109544 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"vencrypt-tls-certs\" (UniqueName: \"kubernetes.io/secret/a15b6144-952c-4d41-a141-5bca5c7fd402-vencrypt-tls-certs\") pod \"nova-cell1-novncproxy-0\" (UID: \"a15b6144-952c-4d41-a141-5bca5c7fd402\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:13:30 crc kubenswrapper[5052]: I1203 12:13:30.110150 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"nova-novncproxy-tls-certs\" (UniqueName: \"kubernetes.io/secret/a15b6144-952c-4d41-a141-5bca5c7fd402-nova-novncproxy-tls-certs\") pod \"nova-cell1-novncproxy-0\" (UID: \"a15b6144-952c-4d41-a141-5bca5c7fd402\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:13:30 crc kubenswrapper[5052]: I1203 12:13:30.111961 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a15b6144-952c-4d41-a141-5bca5c7fd402-config-data\") pod \"nova-cell1-novncproxy-0\" (UID: \"a15b6144-952c-4d41-a141-5bca5c7fd402\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:13:30 crc kubenswrapper[5052]: I1203 12:13:30.112114 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a15b6144-952c-4d41-a141-5bca5c7fd402-combined-ca-bundle\") pod \"nova-cell1-novncproxy-0\" (UID: \"a15b6144-952c-4d41-a141-5bca5c7fd402\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:13:30 crc kubenswrapper[5052]: I1203 12:13:30.116721 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-vvbz9\" (UniqueName: \"kubernetes.io/projected/a15b6144-952c-4d41-a141-5bca5c7fd402-kube-api-access-vvbz9\") pod \"nova-cell1-novncproxy-0\" (UID: \"a15b6144-952c-4d41-a141-5bca5c7fd402\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:13:30 crc kubenswrapper[5052]: I1203 12:13:30.217824 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:13:30 crc kubenswrapper[5052]: I1203 12:13:30.692389 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 12:13:30 crc kubenswrapper[5052]: I1203 12:13:30.843576 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" event={"ID":"a15b6144-952c-4d41-a141-5bca5c7fd402","Type":"ContainerStarted","Data":"961f5bdb73f99eb227d2d7e24cb06ea807c3c40817f760ff50342f68d207b2a7"} Dec 03 12:13:30 crc kubenswrapper[5052]: I1203 12:13:30.849283 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"9092923f-ae28-4bf1-84d2-5fd6b36a3cd1","Type":"ContainerStarted","Data":"0bc886bfe95842a603aa7e8e4961f4aa26056f95c68bd011b5e401fed8724a6a"} Dec 03 12:13:30 crc kubenswrapper[5052]: I1203 12:13:30.849337 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"9092923f-ae28-4bf1-84d2-5fd6b36a3cd1","Type":"ContainerStarted","Data":"3e5f60cd30222efb2f1421cce88978c7b0522cccfd75b282ff723fc016c51218"} Dec 03 12:13:31 crc kubenswrapper[5052]: I1203 12:13:31.684245 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8478e1e7-59a0-4701-aa30-ec5ef9e0578f" path="/var/lib/kubelet/pods/8478e1e7-59a0-4701-aa30-ec5ef9e0578f/volumes" Dec 03 12:13:31 crc kubenswrapper[5052]: I1203 12:13:31.860286 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" event={"ID":"a15b6144-952c-4d41-a141-5bca5c7fd402","Type":"ContainerStarted","Data":"6b64e3ed870b9dc27d7d4df12ca7a0eb2dff4a0cc61910bf73a9e7a4b368dedc"} Dec 03 12:13:31 crc kubenswrapper[5052]: I1203 12:13:31.862033 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"9092923f-ae28-4bf1-84d2-5fd6b36a3cd1","Type":"ContainerStarted","Data":"10233b217e80e908c4d38224e88a865c7ae01dc6059610bf6bab721059646a1f"} Dec 03 12:13:31 crc kubenswrapper[5052]: I1203 12:13:31.887820 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" podStartSLOduration=2.887801702 podStartE2EDuration="2.887801702s" podCreationTimestamp="2025-12-03 12:13:29 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:13:31.877142878 +0000 UTC m=+3202.465384642" watchObservedRunningTime="2025-12-03 12:13:31.887801702 +0000 UTC m=+3202.476043466" Dec 03 12:13:31 crc kubenswrapper[5052]: I1203 12:13:31.900637 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-metadata-0" podStartSLOduration=2.900619888 podStartE2EDuration="2.900619888s" podCreationTimestamp="2025-12-03 12:13:29 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:13:31.896566132 +0000 UTC m=+3202.484807886" watchObservedRunningTime="2025-12-03 12:13:31.900619888 +0000 UTC m=+3202.488861652" Dec 03 12:13:33 crc kubenswrapper[5052]: I1203 12:13:33.169220 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:13:33 crc kubenswrapper[5052]: I1203 12:13:33.169584 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:13:33 crc kubenswrapper[5052]: I1203 12:13:33.529674 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:13:33 crc kubenswrapper[5052]: I1203 12:13:33.560896 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:13:33 crc kubenswrapper[5052]: I1203 12:13:33.882927 5052 generic.go:334] "Generic (PLEG): container finished" podID="4e0e5cde-a60c-4c0e-afd7-818c9ad44c83" containerID="e2af5ecd95d6d829c6c19029b5a86f5c757439ed343b6c5d710eff37b7fddd69" exitCode=0 Dec 03 12:13:33 crc kubenswrapper[5052]: I1203 12:13:33.882979 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-cell-mapping-ztgcf" event={"ID":"4e0e5cde-a60c-4c0e-afd7-818c9ad44c83","Type":"ContainerDied","Data":"e2af5ecd95d6d829c6c19029b5a86f5c757439ed343b6c5d710eff37b7fddd69"} Dec 03 12:13:33 crc kubenswrapper[5052]: I1203 12:13:33.930001 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:13:34 crc kubenswrapper[5052]: I1203 12:13:34.252689 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-api-0" podUID="4e5dd94c-ddfa-4d9e-b5af-fc890a712ea2" containerName="nova-api-log" probeResult="failure" output="Get \"http://10.217.1.27:8774/\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Dec 03 12:13:34 crc kubenswrapper[5052]: I1203 12:13:34.252717 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-api-0" podUID="4e5dd94c-ddfa-4d9e-b5af-fc890a712ea2" containerName="nova-api-api" probeResult="failure" output="Get \"http://10.217.1.27:8774/\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Dec 03 12:13:34 crc kubenswrapper[5052]: I1203 12:13:34.524107 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:13:34 crc kubenswrapper[5052]: I1203 12:13:34.524372 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:13:35 crc kubenswrapper[5052]: I1203 12:13:35.217987 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:13:35 crc kubenswrapper[5052]: I1203 12:13:35.359442 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-cell-mapping-ztgcf" Dec 03 12:13:35 crc kubenswrapper[5052]: I1203 12:13:35.516720 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6b42q\" (UniqueName: \"kubernetes.io/projected/4e0e5cde-a60c-4c0e-afd7-818c9ad44c83-kube-api-access-6b42q\") pod \"4e0e5cde-a60c-4c0e-afd7-818c9ad44c83\" (UID: \"4e0e5cde-a60c-4c0e-afd7-818c9ad44c83\") " Dec 03 12:13:35 crc kubenswrapper[5052]: I1203 12:13:35.516762 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/4e0e5cde-a60c-4c0e-afd7-818c9ad44c83-scripts\") pod \"4e0e5cde-a60c-4c0e-afd7-818c9ad44c83\" (UID: \"4e0e5cde-a60c-4c0e-afd7-818c9ad44c83\") " Dec 03 12:13:35 crc kubenswrapper[5052]: I1203 12:13:35.516839 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/4e0e5cde-a60c-4c0e-afd7-818c9ad44c83-config-data\") pod \"4e0e5cde-a60c-4c0e-afd7-818c9ad44c83\" (UID: \"4e0e5cde-a60c-4c0e-afd7-818c9ad44c83\") " Dec 03 12:13:35 crc kubenswrapper[5052]: I1203 12:13:35.516885 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4e0e5cde-a60c-4c0e-afd7-818c9ad44c83-combined-ca-bundle\") pod \"4e0e5cde-a60c-4c0e-afd7-818c9ad44c83\" (UID: \"4e0e5cde-a60c-4c0e-afd7-818c9ad44c83\") " Dec 03 12:13:35 crc kubenswrapper[5052]: I1203 12:13:35.522164 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4e0e5cde-a60c-4c0e-afd7-818c9ad44c83-scripts" (OuterVolumeSpecName: "scripts") pod "4e0e5cde-a60c-4c0e-afd7-818c9ad44c83" (UID: "4e0e5cde-a60c-4c0e-afd7-818c9ad44c83"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:13:35 crc kubenswrapper[5052]: I1203 12:13:35.523024 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4e0e5cde-a60c-4c0e-afd7-818c9ad44c83-kube-api-access-6b42q" (OuterVolumeSpecName: "kube-api-access-6b42q") pod "4e0e5cde-a60c-4c0e-afd7-818c9ad44c83" (UID: "4e0e5cde-a60c-4c0e-afd7-818c9ad44c83"). InnerVolumeSpecName "kube-api-access-6b42q". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:13:35 crc kubenswrapper[5052]: I1203 12:13:35.544493 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4e0e5cde-a60c-4c0e-afd7-818c9ad44c83-config-data" (OuterVolumeSpecName: "config-data") pod "4e0e5cde-a60c-4c0e-afd7-818c9ad44c83" (UID: "4e0e5cde-a60c-4c0e-afd7-818c9ad44c83"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:13:35 crc kubenswrapper[5052]: I1203 12:13:35.549908 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4e0e5cde-a60c-4c0e-afd7-818c9ad44c83-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "4e0e5cde-a60c-4c0e-afd7-818c9ad44c83" (UID: "4e0e5cde-a60c-4c0e-afd7-818c9ad44c83"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:13:35 crc kubenswrapper[5052]: I1203 12:13:35.620174 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-6b42q\" (UniqueName: \"kubernetes.io/projected/4e0e5cde-a60c-4c0e-afd7-818c9ad44c83-kube-api-access-6b42q\") on node \"crc\" DevicePath \"\"" Dec 03 12:13:35 crc kubenswrapper[5052]: I1203 12:13:35.620255 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/4e0e5cde-a60c-4c0e-afd7-818c9ad44c83-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:13:35 crc kubenswrapper[5052]: I1203 12:13:35.620283 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/4e0e5cde-a60c-4c0e-afd7-818c9ad44c83-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:13:35 crc kubenswrapper[5052]: I1203 12:13:35.620306 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4e0e5cde-a60c-4c0e-afd7-818c9ad44c83-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:13:35 crc kubenswrapper[5052]: I1203 12:13:35.903434 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-cell-mapping-ztgcf" event={"ID":"4e0e5cde-a60c-4c0e-afd7-818c9ad44c83","Type":"ContainerDied","Data":"f82e124ea6fffb6b16fedc233ee0332ed1b24008ca22e1024194d86f066c10d9"} Dec 03 12:13:35 crc kubenswrapper[5052]: I1203 12:13:35.903475 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f82e124ea6fffb6b16fedc233ee0332ed1b24008ca22e1024194d86f066c10d9" Dec 03 12:13:35 crc kubenswrapper[5052]: I1203 12:13:35.903547 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-cell-mapping-ztgcf" Dec 03 12:13:36 crc kubenswrapper[5052]: I1203 12:13:36.086485 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:13:36 crc kubenswrapper[5052]: I1203 12:13:36.086803 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-api-0" podUID="4e5dd94c-ddfa-4d9e-b5af-fc890a712ea2" containerName="nova-api-log" containerID="cri-o://384172a9d89e7ce7470e93a792ed1e7f0c5dd86b20fdbac86dcf823b305d5cb6" gracePeriod=30 Dec 03 12:13:36 crc kubenswrapper[5052]: I1203 12:13:36.087414 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-api-0" podUID="4e5dd94c-ddfa-4d9e-b5af-fc890a712ea2" containerName="nova-api-api" containerID="cri-o://5b640e0fce110cba8efb298c6be8bbce1834d0dda04bf593b0a7dfc90b902797" gracePeriod=30 Dec 03 12:13:36 crc kubenswrapper[5052]: I1203 12:13:36.102112 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:13:36 crc kubenswrapper[5052]: I1203 12:13:36.102521 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-scheduler-0" podUID="6db171a5-bb58-4636-88d6-6b853614b4db" containerName="nova-scheduler-scheduler" containerID="cri-o://0b34a2dacc14517a7a2bb7d529e7d47c9a37b24dc2bbb3c28ba5b1768efc7a95" gracePeriod=30 Dec 03 12:13:36 crc kubenswrapper[5052]: I1203 12:13:36.114761 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:13:36 crc kubenswrapper[5052]: I1203 12:13:36.115043 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-metadata-0" podUID="9092923f-ae28-4bf1-84d2-5fd6b36a3cd1" containerName="nova-metadata-log" containerID="cri-o://0bc886bfe95842a603aa7e8e4961f4aa26056f95c68bd011b5e401fed8724a6a" gracePeriod=30 Dec 03 12:13:36 crc kubenswrapper[5052]: I1203 12:13:36.115218 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-metadata-0" podUID="9092923f-ae28-4bf1-84d2-5fd6b36a3cd1" containerName="nova-metadata-metadata" containerID="cri-o://10233b217e80e908c4d38224e88a865c7ae01dc6059610bf6bab721059646a1f" gracePeriod=30 Dec 03 12:13:36 crc kubenswrapper[5052]: I1203 12:13:36.723811 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:13:36 crc kubenswrapper[5052]: I1203 12:13:36.848215 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/9092923f-ae28-4bf1-84d2-5fd6b36a3cd1-nova-metadata-tls-certs\") pod \"9092923f-ae28-4bf1-84d2-5fd6b36a3cd1\" (UID: \"9092923f-ae28-4bf1-84d2-5fd6b36a3cd1\") " Dec 03 12:13:36 crc kubenswrapper[5052]: I1203 12:13:36.848313 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9092923f-ae28-4bf1-84d2-5fd6b36a3cd1-combined-ca-bundle\") pod \"9092923f-ae28-4bf1-84d2-5fd6b36a3cd1\" (UID: \"9092923f-ae28-4bf1-84d2-5fd6b36a3cd1\") " Dec 03 12:13:36 crc kubenswrapper[5052]: I1203 12:13:36.848354 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/9092923f-ae28-4bf1-84d2-5fd6b36a3cd1-config-data\") pod \"9092923f-ae28-4bf1-84d2-5fd6b36a3cd1\" (UID: \"9092923f-ae28-4bf1-84d2-5fd6b36a3cd1\") " Dec 03 12:13:36 crc kubenswrapper[5052]: I1203 12:13:36.848392 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/9092923f-ae28-4bf1-84d2-5fd6b36a3cd1-logs\") pod \"9092923f-ae28-4bf1-84d2-5fd6b36a3cd1\" (UID: \"9092923f-ae28-4bf1-84d2-5fd6b36a3cd1\") " Dec 03 12:13:36 crc kubenswrapper[5052]: I1203 12:13:36.848570 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-56w5z\" (UniqueName: \"kubernetes.io/projected/9092923f-ae28-4bf1-84d2-5fd6b36a3cd1-kube-api-access-56w5z\") pod \"9092923f-ae28-4bf1-84d2-5fd6b36a3cd1\" (UID: \"9092923f-ae28-4bf1-84d2-5fd6b36a3cd1\") " Dec 03 12:13:36 crc kubenswrapper[5052]: I1203 12:13:36.849030 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9092923f-ae28-4bf1-84d2-5fd6b36a3cd1-logs" (OuterVolumeSpecName: "logs") pod "9092923f-ae28-4bf1-84d2-5fd6b36a3cd1" (UID: "9092923f-ae28-4bf1-84d2-5fd6b36a3cd1"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:13:36 crc kubenswrapper[5052]: I1203 12:13:36.874256 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9092923f-ae28-4bf1-84d2-5fd6b36a3cd1-kube-api-access-56w5z" (OuterVolumeSpecName: "kube-api-access-56w5z") pod "9092923f-ae28-4bf1-84d2-5fd6b36a3cd1" (UID: "9092923f-ae28-4bf1-84d2-5fd6b36a3cd1"). InnerVolumeSpecName "kube-api-access-56w5z". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:13:36 crc kubenswrapper[5052]: I1203 12:13:36.878825 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9092923f-ae28-4bf1-84d2-5fd6b36a3cd1-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "9092923f-ae28-4bf1-84d2-5fd6b36a3cd1" (UID: "9092923f-ae28-4bf1-84d2-5fd6b36a3cd1"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:13:36 crc kubenswrapper[5052]: I1203 12:13:36.884870 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9092923f-ae28-4bf1-84d2-5fd6b36a3cd1-config-data" (OuterVolumeSpecName: "config-data") pod "9092923f-ae28-4bf1-84d2-5fd6b36a3cd1" (UID: "9092923f-ae28-4bf1-84d2-5fd6b36a3cd1"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:13:36 crc kubenswrapper[5052]: I1203 12:13:36.903299 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9092923f-ae28-4bf1-84d2-5fd6b36a3cd1-nova-metadata-tls-certs" (OuterVolumeSpecName: "nova-metadata-tls-certs") pod "9092923f-ae28-4bf1-84d2-5fd6b36a3cd1" (UID: "9092923f-ae28-4bf1-84d2-5fd6b36a3cd1"). InnerVolumeSpecName "nova-metadata-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:13:36 crc kubenswrapper[5052]: I1203 12:13:36.928183 5052 generic.go:334] "Generic (PLEG): container finished" podID="9092923f-ae28-4bf1-84d2-5fd6b36a3cd1" containerID="10233b217e80e908c4d38224e88a865c7ae01dc6059610bf6bab721059646a1f" exitCode=0 Dec 03 12:13:36 crc kubenswrapper[5052]: I1203 12:13:36.928223 5052 generic.go:334] "Generic (PLEG): container finished" podID="9092923f-ae28-4bf1-84d2-5fd6b36a3cd1" containerID="0bc886bfe95842a603aa7e8e4961f4aa26056f95c68bd011b5e401fed8724a6a" exitCode=143 Dec 03 12:13:36 crc kubenswrapper[5052]: I1203 12:13:36.928252 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"9092923f-ae28-4bf1-84d2-5fd6b36a3cd1","Type":"ContainerDied","Data":"10233b217e80e908c4d38224e88a865c7ae01dc6059610bf6bab721059646a1f"} Dec 03 12:13:36 crc kubenswrapper[5052]: I1203 12:13:36.928312 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"9092923f-ae28-4bf1-84d2-5fd6b36a3cd1","Type":"ContainerDied","Data":"0bc886bfe95842a603aa7e8e4961f4aa26056f95c68bd011b5e401fed8724a6a"} Dec 03 12:13:36 crc kubenswrapper[5052]: I1203 12:13:36.928322 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"9092923f-ae28-4bf1-84d2-5fd6b36a3cd1","Type":"ContainerDied","Data":"3e5f60cd30222efb2f1421cce88978c7b0522cccfd75b282ff723fc016c51218"} Dec 03 12:13:36 crc kubenswrapper[5052]: I1203 12:13:36.928268 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:13:36 crc kubenswrapper[5052]: I1203 12:13:36.928343 5052 scope.go:117] "RemoveContainer" containerID="10233b217e80e908c4d38224e88a865c7ae01dc6059610bf6bab721059646a1f" Dec 03 12:13:36 crc kubenswrapper[5052]: I1203 12:13:36.930407 5052 generic.go:334] "Generic (PLEG): container finished" podID="f2a04bd0-579e-467c-b0f6-72af16ffb143" containerID="624a18da80d875d17af9c7c2a3b638755eddc32b1144b57ac4834a2414178324" exitCode=0 Dec 03 12:13:36 crc kubenswrapper[5052]: I1203 12:13:36.930465 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-wnzql" event={"ID":"f2a04bd0-579e-467c-b0f6-72af16ffb143","Type":"ContainerDied","Data":"624a18da80d875d17af9c7c2a3b638755eddc32b1144b57ac4834a2414178324"} Dec 03 12:13:36 crc kubenswrapper[5052]: I1203 12:13:36.933354 5052 generic.go:334] "Generic (PLEG): container finished" podID="4e5dd94c-ddfa-4d9e-b5af-fc890a712ea2" containerID="384172a9d89e7ce7470e93a792ed1e7f0c5dd86b20fdbac86dcf823b305d5cb6" exitCode=143 Dec 03 12:13:36 crc kubenswrapper[5052]: I1203 12:13:36.933407 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"4e5dd94c-ddfa-4d9e-b5af-fc890a712ea2","Type":"ContainerDied","Data":"384172a9d89e7ce7470e93a792ed1e7f0c5dd86b20fdbac86dcf823b305d5cb6"} Dec 03 12:13:36 crc kubenswrapper[5052]: I1203 12:13:36.951105 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-56w5z\" (UniqueName: \"kubernetes.io/projected/9092923f-ae28-4bf1-84d2-5fd6b36a3cd1-kube-api-access-56w5z\") on node \"crc\" DevicePath \"\"" Dec 03 12:13:36 crc kubenswrapper[5052]: I1203 12:13:36.951142 5052 reconciler_common.go:293] "Volume detached for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/9092923f-ae28-4bf1-84d2-5fd6b36a3cd1-nova-metadata-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:13:36 crc kubenswrapper[5052]: I1203 12:13:36.951155 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9092923f-ae28-4bf1-84d2-5fd6b36a3cd1-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:13:36 crc kubenswrapper[5052]: I1203 12:13:36.951169 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/9092923f-ae28-4bf1-84d2-5fd6b36a3cd1-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:13:36 crc kubenswrapper[5052]: I1203 12:13:36.951181 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/9092923f-ae28-4bf1-84d2-5fd6b36a3cd1-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:13:36 crc kubenswrapper[5052]: I1203 12:13:36.952299 5052 scope.go:117] "RemoveContainer" containerID="0bc886bfe95842a603aa7e8e4961f4aa26056f95c68bd011b5e401fed8724a6a" Dec 03 12:13:36 crc kubenswrapper[5052]: I1203 12:13:36.979181 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:13:36 crc kubenswrapper[5052]: I1203 12:13:36.983652 5052 scope.go:117] "RemoveContainer" containerID="10233b217e80e908c4d38224e88a865c7ae01dc6059610bf6bab721059646a1f" Dec 03 12:13:36 crc kubenswrapper[5052]: E1203 12:13:36.985279 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"10233b217e80e908c4d38224e88a865c7ae01dc6059610bf6bab721059646a1f\": container with ID starting with 10233b217e80e908c4d38224e88a865c7ae01dc6059610bf6bab721059646a1f not found: ID does not exist" containerID="10233b217e80e908c4d38224e88a865c7ae01dc6059610bf6bab721059646a1f" Dec 03 12:13:36 crc kubenswrapper[5052]: I1203 12:13:36.985316 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"10233b217e80e908c4d38224e88a865c7ae01dc6059610bf6bab721059646a1f"} err="failed to get container status \"10233b217e80e908c4d38224e88a865c7ae01dc6059610bf6bab721059646a1f\": rpc error: code = NotFound desc = could not find container \"10233b217e80e908c4d38224e88a865c7ae01dc6059610bf6bab721059646a1f\": container with ID starting with 10233b217e80e908c4d38224e88a865c7ae01dc6059610bf6bab721059646a1f not found: ID does not exist" Dec 03 12:13:36 crc kubenswrapper[5052]: I1203 12:13:36.985340 5052 scope.go:117] "RemoveContainer" containerID="0bc886bfe95842a603aa7e8e4961f4aa26056f95c68bd011b5e401fed8724a6a" Dec 03 12:13:36 crc kubenswrapper[5052]: E1203 12:13:36.985783 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0bc886bfe95842a603aa7e8e4961f4aa26056f95c68bd011b5e401fed8724a6a\": container with ID starting with 0bc886bfe95842a603aa7e8e4961f4aa26056f95c68bd011b5e401fed8724a6a not found: ID does not exist" containerID="0bc886bfe95842a603aa7e8e4961f4aa26056f95c68bd011b5e401fed8724a6a" Dec 03 12:13:36 crc kubenswrapper[5052]: I1203 12:13:36.985810 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0bc886bfe95842a603aa7e8e4961f4aa26056f95c68bd011b5e401fed8724a6a"} err="failed to get container status \"0bc886bfe95842a603aa7e8e4961f4aa26056f95c68bd011b5e401fed8724a6a\": rpc error: code = NotFound desc = could not find container \"0bc886bfe95842a603aa7e8e4961f4aa26056f95c68bd011b5e401fed8724a6a\": container with ID starting with 0bc886bfe95842a603aa7e8e4961f4aa26056f95c68bd011b5e401fed8724a6a not found: ID does not exist" Dec 03 12:13:36 crc kubenswrapper[5052]: I1203 12:13:36.985826 5052 scope.go:117] "RemoveContainer" containerID="10233b217e80e908c4d38224e88a865c7ae01dc6059610bf6bab721059646a1f" Dec 03 12:13:36 crc kubenswrapper[5052]: I1203 12:13:36.986115 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"10233b217e80e908c4d38224e88a865c7ae01dc6059610bf6bab721059646a1f"} err="failed to get container status \"10233b217e80e908c4d38224e88a865c7ae01dc6059610bf6bab721059646a1f\": rpc error: code = NotFound desc = could not find container \"10233b217e80e908c4d38224e88a865c7ae01dc6059610bf6bab721059646a1f\": container with ID starting with 10233b217e80e908c4d38224e88a865c7ae01dc6059610bf6bab721059646a1f not found: ID does not exist" Dec 03 12:13:36 crc kubenswrapper[5052]: I1203 12:13:36.986166 5052 scope.go:117] "RemoveContainer" containerID="0bc886bfe95842a603aa7e8e4961f4aa26056f95c68bd011b5e401fed8724a6a" Dec 03 12:13:36 crc kubenswrapper[5052]: I1203 12:13:36.986562 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0bc886bfe95842a603aa7e8e4961f4aa26056f95c68bd011b5e401fed8724a6a"} err="failed to get container status \"0bc886bfe95842a603aa7e8e4961f4aa26056f95c68bd011b5e401fed8724a6a\": rpc error: code = NotFound desc = could not find container \"0bc886bfe95842a603aa7e8e4961f4aa26056f95c68bd011b5e401fed8724a6a\": container with ID starting with 0bc886bfe95842a603aa7e8e4961f4aa26056f95c68bd011b5e401fed8724a6a not found: ID does not exist" Dec 03 12:13:36 crc kubenswrapper[5052]: I1203 12:13:36.999349 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:13:37 crc kubenswrapper[5052]: I1203 12:13:37.012957 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:13:37 crc kubenswrapper[5052]: E1203 12:13:37.013430 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="9092923f-ae28-4bf1-84d2-5fd6b36a3cd1" containerName="nova-metadata-log" Dec 03 12:13:37 crc kubenswrapper[5052]: I1203 12:13:37.013450 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="9092923f-ae28-4bf1-84d2-5fd6b36a3cd1" containerName="nova-metadata-log" Dec 03 12:13:37 crc kubenswrapper[5052]: E1203 12:13:37.013480 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="9092923f-ae28-4bf1-84d2-5fd6b36a3cd1" containerName="nova-metadata-metadata" Dec 03 12:13:37 crc kubenswrapper[5052]: I1203 12:13:37.013487 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="9092923f-ae28-4bf1-84d2-5fd6b36a3cd1" containerName="nova-metadata-metadata" Dec 03 12:13:37 crc kubenswrapper[5052]: E1203 12:13:37.013532 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="4e0e5cde-a60c-4c0e-afd7-818c9ad44c83" containerName="nova-manage" Dec 03 12:13:37 crc kubenswrapper[5052]: I1203 12:13:37.013542 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e0e5cde-a60c-4c0e-afd7-818c9ad44c83" containerName="nova-manage" Dec 03 12:13:37 crc kubenswrapper[5052]: I1203 12:13:37.013793 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="9092923f-ae28-4bf1-84d2-5fd6b36a3cd1" containerName="nova-metadata-log" Dec 03 12:13:37 crc kubenswrapper[5052]: I1203 12:13:37.013823 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="9092923f-ae28-4bf1-84d2-5fd6b36a3cd1" containerName="nova-metadata-metadata" Dec 03 12:13:37 crc kubenswrapper[5052]: I1203 12:13:37.013841 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="4e0e5cde-a60c-4c0e-afd7-818c9ad44c83" containerName="nova-manage" Dec 03 12:13:37 crc kubenswrapper[5052]: I1203 12:13:37.014963 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:13:37 crc kubenswrapper[5052]: I1203 12:13:37.018138 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-nova-metadata-internal-svc" Dec 03 12:13:37 crc kubenswrapper[5052]: I1203 12:13:37.018482 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-metadata-config-data" Dec 03 12:13:37 crc kubenswrapper[5052]: I1203 12:13:37.022596 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:13:37 crc kubenswrapper[5052]: I1203 12:13:37.154182 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/3487eda5-6351-40b2-babe-458b38432b7f-config-data\") pod \"nova-metadata-0\" (UID: \"3487eda5-6351-40b2-babe-458b38432b7f\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:13:37 crc kubenswrapper[5052]: I1203 12:13:37.154544 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/3487eda5-6351-40b2-babe-458b38432b7f-nova-metadata-tls-certs\") pod \"nova-metadata-0\" (UID: \"3487eda5-6351-40b2-babe-458b38432b7f\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:13:37 crc kubenswrapper[5052]: I1203 12:13:37.154616 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-76gsx\" (UniqueName: \"kubernetes.io/projected/3487eda5-6351-40b2-babe-458b38432b7f-kube-api-access-76gsx\") pod \"nova-metadata-0\" (UID: \"3487eda5-6351-40b2-babe-458b38432b7f\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:13:37 crc kubenswrapper[5052]: I1203 12:13:37.154637 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/3487eda5-6351-40b2-babe-458b38432b7f-logs\") pod \"nova-metadata-0\" (UID: \"3487eda5-6351-40b2-babe-458b38432b7f\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:13:37 crc kubenswrapper[5052]: I1203 12:13:37.154699 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3487eda5-6351-40b2-babe-458b38432b7f-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"3487eda5-6351-40b2-babe-458b38432b7f\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:13:37 crc kubenswrapper[5052]: I1203 12:13:37.257438 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/3487eda5-6351-40b2-babe-458b38432b7f-logs\") pod \"nova-metadata-0\" (UID: \"3487eda5-6351-40b2-babe-458b38432b7f\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:13:37 crc kubenswrapper[5052]: I1203 12:13:37.257683 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3487eda5-6351-40b2-babe-458b38432b7f-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"3487eda5-6351-40b2-babe-458b38432b7f\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:13:37 crc kubenswrapper[5052]: I1203 12:13:37.257744 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/3487eda5-6351-40b2-babe-458b38432b7f-config-data\") pod \"nova-metadata-0\" (UID: \"3487eda5-6351-40b2-babe-458b38432b7f\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:13:37 crc kubenswrapper[5052]: I1203 12:13:37.257786 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/3487eda5-6351-40b2-babe-458b38432b7f-nova-metadata-tls-certs\") pod \"nova-metadata-0\" (UID: \"3487eda5-6351-40b2-babe-458b38432b7f\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:13:37 crc kubenswrapper[5052]: I1203 12:13:37.257850 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-76gsx\" (UniqueName: \"kubernetes.io/projected/3487eda5-6351-40b2-babe-458b38432b7f-kube-api-access-76gsx\") pod \"nova-metadata-0\" (UID: \"3487eda5-6351-40b2-babe-458b38432b7f\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:13:37 crc kubenswrapper[5052]: I1203 12:13:37.258536 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/3487eda5-6351-40b2-babe-458b38432b7f-logs\") pod \"nova-metadata-0\" (UID: \"3487eda5-6351-40b2-babe-458b38432b7f\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:13:37 crc kubenswrapper[5052]: I1203 12:13:37.263548 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3487eda5-6351-40b2-babe-458b38432b7f-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"3487eda5-6351-40b2-babe-458b38432b7f\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:13:37 crc kubenswrapper[5052]: I1203 12:13:37.265329 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/3487eda5-6351-40b2-babe-458b38432b7f-nova-metadata-tls-certs\") pod \"nova-metadata-0\" (UID: \"3487eda5-6351-40b2-babe-458b38432b7f\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:13:37 crc kubenswrapper[5052]: I1203 12:13:37.270941 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/3487eda5-6351-40b2-babe-458b38432b7f-config-data\") pod \"nova-metadata-0\" (UID: \"3487eda5-6351-40b2-babe-458b38432b7f\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:13:37 crc kubenswrapper[5052]: I1203 12:13:37.280221 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-76gsx\" (UniqueName: \"kubernetes.io/projected/3487eda5-6351-40b2-babe-458b38432b7f-kube-api-access-76gsx\") pod \"nova-metadata-0\" (UID: \"3487eda5-6351-40b2-babe-458b38432b7f\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:13:37 crc kubenswrapper[5052]: I1203 12:13:37.341659 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:13:37 crc kubenswrapper[5052]: I1203 12:13:37.513855 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:13:37 crc kubenswrapper[5052]: I1203 12:13:37.667842 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6db171a5-bb58-4636-88d6-6b853614b4db-config-data\") pod \"6db171a5-bb58-4636-88d6-6b853614b4db\" (UID: \"6db171a5-bb58-4636-88d6-6b853614b4db\") " Dec 03 12:13:37 crc kubenswrapper[5052]: I1203 12:13:37.667962 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-slmxw\" (UniqueName: \"kubernetes.io/projected/6db171a5-bb58-4636-88d6-6b853614b4db-kube-api-access-slmxw\") pod \"6db171a5-bb58-4636-88d6-6b853614b4db\" (UID: \"6db171a5-bb58-4636-88d6-6b853614b4db\") " Dec 03 12:13:37 crc kubenswrapper[5052]: I1203 12:13:37.668021 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6db171a5-bb58-4636-88d6-6b853614b4db-combined-ca-bundle\") pod \"6db171a5-bb58-4636-88d6-6b853614b4db\" (UID: \"6db171a5-bb58-4636-88d6-6b853614b4db\") " Dec 03 12:13:37 crc kubenswrapper[5052]: I1203 12:13:37.679874 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6db171a5-bb58-4636-88d6-6b853614b4db-kube-api-access-slmxw" (OuterVolumeSpecName: "kube-api-access-slmxw") pod "6db171a5-bb58-4636-88d6-6b853614b4db" (UID: "6db171a5-bb58-4636-88d6-6b853614b4db"). InnerVolumeSpecName "kube-api-access-slmxw". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:13:37 crc kubenswrapper[5052]: I1203 12:13:37.703290 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9092923f-ae28-4bf1-84d2-5fd6b36a3cd1" path="/var/lib/kubelet/pods/9092923f-ae28-4bf1-84d2-5fd6b36a3cd1/volumes" Dec 03 12:13:37 crc kubenswrapper[5052]: I1203 12:13:37.704917 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6db171a5-bb58-4636-88d6-6b853614b4db-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "6db171a5-bb58-4636-88d6-6b853614b4db" (UID: "6db171a5-bb58-4636-88d6-6b853614b4db"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:13:37 crc kubenswrapper[5052]: I1203 12:13:37.706802 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6db171a5-bb58-4636-88d6-6b853614b4db-config-data" (OuterVolumeSpecName: "config-data") pod "6db171a5-bb58-4636-88d6-6b853614b4db" (UID: "6db171a5-bb58-4636-88d6-6b853614b4db"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:13:37 crc kubenswrapper[5052]: I1203 12:13:37.770929 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6db171a5-bb58-4636-88d6-6b853614b4db-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:13:37 crc kubenswrapper[5052]: I1203 12:13:37.770955 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-slmxw\" (UniqueName: \"kubernetes.io/projected/6db171a5-bb58-4636-88d6-6b853614b4db-kube-api-access-slmxw\") on node \"crc\" DevicePath \"\"" Dec 03 12:13:37 crc kubenswrapper[5052]: I1203 12:13:37.770965 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6db171a5-bb58-4636-88d6-6b853614b4db-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:13:37 crc kubenswrapper[5052]: I1203 12:13:37.773830 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:13:37 crc kubenswrapper[5052]: W1203 12:13:37.775450 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3487eda5_6351_40b2_babe_458b38432b7f.slice/crio-0ffcd1dc5a58757a8975d5fe7eff25ee148ebd5bd47c7922f798af820a7a4191 WatchSource:0}: Error finding container 0ffcd1dc5a58757a8975d5fe7eff25ee148ebd5bd47c7922f798af820a7a4191: Status 404 returned error can't find the container with id 0ffcd1dc5a58757a8975d5fe7eff25ee148ebd5bd47c7922f798af820a7a4191 Dec 03 12:13:37 crc kubenswrapper[5052]: I1203 12:13:37.944239 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"3487eda5-6351-40b2-babe-458b38432b7f","Type":"ContainerStarted","Data":"0ffcd1dc5a58757a8975d5fe7eff25ee148ebd5bd47c7922f798af820a7a4191"} Dec 03 12:13:37 crc kubenswrapper[5052]: I1203 12:13:37.945858 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:13:37 crc kubenswrapper[5052]: I1203 12:13:37.945847 5052 generic.go:334] "Generic (PLEG): container finished" podID="6db171a5-bb58-4636-88d6-6b853614b4db" containerID="0b34a2dacc14517a7a2bb7d529e7d47c9a37b24dc2bbb3c28ba5b1768efc7a95" exitCode=0 Dec 03 12:13:37 crc kubenswrapper[5052]: I1203 12:13:37.945894 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"6db171a5-bb58-4636-88d6-6b853614b4db","Type":"ContainerDied","Data":"0b34a2dacc14517a7a2bb7d529e7d47c9a37b24dc2bbb3c28ba5b1768efc7a95"} Dec 03 12:13:37 crc kubenswrapper[5052]: I1203 12:13:37.945942 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"6db171a5-bb58-4636-88d6-6b853614b4db","Type":"ContainerDied","Data":"51c910c631cf0a28f759453d76d9c12c998cfaad6d19ee63246ef4894643b265"} Dec 03 12:13:37 crc kubenswrapper[5052]: I1203 12:13:37.945960 5052 scope.go:117] "RemoveContainer" containerID="0b34a2dacc14517a7a2bb7d529e7d47c9a37b24dc2bbb3c28ba5b1768efc7a95" Dec 03 12:13:37 crc kubenswrapper[5052]: I1203 12:13:37.968838 5052 scope.go:117] "RemoveContainer" containerID="0b34a2dacc14517a7a2bb7d529e7d47c9a37b24dc2bbb3c28ba5b1768efc7a95" Dec 03 12:13:37 crc kubenswrapper[5052]: E1203 12:13:37.969247 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0b34a2dacc14517a7a2bb7d529e7d47c9a37b24dc2bbb3c28ba5b1768efc7a95\": container with ID starting with 0b34a2dacc14517a7a2bb7d529e7d47c9a37b24dc2bbb3c28ba5b1768efc7a95 not found: ID does not exist" containerID="0b34a2dacc14517a7a2bb7d529e7d47c9a37b24dc2bbb3c28ba5b1768efc7a95" Dec 03 12:13:37 crc kubenswrapper[5052]: I1203 12:13:37.969279 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0b34a2dacc14517a7a2bb7d529e7d47c9a37b24dc2bbb3c28ba5b1768efc7a95"} err="failed to get container status \"0b34a2dacc14517a7a2bb7d529e7d47c9a37b24dc2bbb3c28ba5b1768efc7a95\": rpc error: code = NotFound desc = could not find container \"0b34a2dacc14517a7a2bb7d529e7d47c9a37b24dc2bbb3c28ba5b1768efc7a95\": container with ID starting with 0b34a2dacc14517a7a2bb7d529e7d47c9a37b24dc2bbb3c28ba5b1768efc7a95 not found: ID does not exist" Dec 03 12:13:37 crc kubenswrapper[5052]: I1203 12:13:37.985398 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:13:37 crc kubenswrapper[5052]: I1203 12:13:37.998463 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:13:38 crc kubenswrapper[5052]: I1203 12:13:38.008677 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:13:38 crc kubenswrapper[5052]: E1203 12:13:38.009181 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="6db171a5-bb58-4636-88d6-6b853614b4db" containerName="nova-scheduler-scheduler" Dec 03 12:13:38 crc kubenswrapper[5052]: I1203 12:13:38.009201 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="6db171a5-bb58-4636-88d6-6b853614b4db" containerName="nova-scheduler-scheduler" Dec 03 12:13:38 crc kubenswrapper[5052]: I1203 12:13:38.009411 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="6db171a5-bb58-4636-88d6-6b853614b4db" containerName="nova-scheduler-scheduler" Dec 03 12:13:38 crc kubenswrapper[5052]: I1203 12:13:38.010145 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:13:38 crc kubenswrapper[5052]: I1203 12:13:38.012821 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-scheduler-config-data" Dec 03 12:13:38 crc kubenswrapper[5052]: I1203 12:13:38.021244 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:13:38 crc kubenswrapper[5052]: I1203 12:13:38.183274 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/ee6da6e8-a00d-4068-b5be-9ef9fdd61448-config-data\") pod \"nova-scheduler-0\" (UID: \"ee6da6e8-a00d-4068-b5be-9ef9fdd61448\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:13:38 crc kubenswrapper[5052]: I1203 12:13:38.183619 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ee6da6e8-a00d-4068-b5be-9ef9fdd61448-combined-ca-bundle\") pod \"nova-scheduler-0\" (UID: \"ee6da6e8-a00d-4068-b5be-9ef9fdd61448\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:13:38 crc kubenswrapper[5052]: I1203 12:13:38.183664 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zvf48\" (UniqueName: \"kubernetes.io/projected/ee6da6e8-a00d-4068-b5be-9ef9fdd61448-kube-api-access-zvf48\") pod \"nova-scheduler-0\" (UID: \"ee6da6e8-a00d-4068-b5be-9ef9fdd61448\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:13:38 crc kubenswrapper[5052]: I1203 12:13:38.285668 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ee6da6e8-a00d-4068-b5be-9ef9fdd61448-combined-ca-bundle\") pod \"nova-scheduler-0\" (UID: \"ee6da6e8-a00d-4068-b5be-9ef9fdd61448\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:13:38 crc kubenswrapper[5052]: I1203 12:13:38.285728 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-zvf48\" (UniqueName: \"kubernetes.io/projected/ee6da6e8-a00d-4068-b5be-9ef9fdd61448-kube-api-access-zvf48\") pod \"nova-scheduler-0\" (UID: \"ee6da6e8-a00d-4068-b5be-9ef9fdd61448\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:13:38 crc kubenswrapper[5052]: I1203 12:13:38.285879 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/ee6da6e8-a00d-4068-b5be-9ef9fdd61448-config-data\") pod \"nova-scheduler-0\" (UID: \"ee6da6e8-a00d-4068-b5be-9ef9fdd61448\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:13:38 crc kubenswrapper[5052]: I1203 12:13:38.296256 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/ee6da6e8-a00d-4068-b5be-9ef9fdd61448-config-data\") pod \"nova-scheduler-0\" (UID: \"ee6da6e8-a00d-4068-b5be-9ef9fdd61448\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:13:38 crc kubenswrapper[5052]: I1203 12:13:38.296317 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ee6da6e8-a00d-4068-b5be-9ef9fdd61448-combined-ca-bundle\") pod \"nova-scheduler-0\" (UID: \"ee6da6e8-a00d-4068-b5be-9ef9fdd61448\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:13:38 crc kubenswrapper[5052]: I1203 12:13:38.307230 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-zvf48\" (UniqueName: \"kubernetes.io/projected/ee6da6e8-a00d-4068-b5be-9ef9fdd61448-kube-api-access-zvf48\") pod \"nova-scheduler-0\" (UID: \"ee6da6e8-a00d-4068-b5be-9ef9fdd61448\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:13:38 crc kubenswrapper[5052]: I1203 12:13:38.330377 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:13:38 crc kubenswrapper[5052]: I1203 12:13:38.423374 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-wnzql" Dec 03 12:13:38 crc kubenswrapper[5052]: I1203 12:13:38.591282 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r2w7p\" (UniqueName: \"kubernetes.io/projected/f2a04bd0-579e-467c-b0f6-72af16ffb143-kube-api-access-r2w7p\") pod \"f2a04bd0-579e-467c-b0f6-72af16ffb143\" (UID: \"f2a04bd0-579e-467c-b0f6-72af16ffb143\") " Dec 03 12:13:38 crc kubenswrapper[5052]: I1203 12:13:38.591355 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/f2a04bd0-579e-467c-b0f6-72af16ffb143-config-data\") pod \"f2a04bd0-579e-467c-b0f6-72af16ffb143\" (UID: \"f2a04bd0-579e-467c-b0f6-72af16ffb143\") " Dec 03 12:13:38 crc kubenswrapper[5052]: I1203 12:13:38.591393 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/f2a04bd0-579e-467c-b0f6-72af16ffb143-scripts\") pod \"f2a04bd0-579e-467c-b0f6-72af16ffb143\" (UID: \"f2a04bd0-579e-467c-b0f6-72af16ffb143\") " Dec 03 12:13:38 crc kubenswrapper[5052]: I1203 12:13:38.591481 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f2a04bd0-579e-467c-b0f6-72af16ffb143-combined-ca-bundle\") pod \"f2a04bd0-579e-467c-b0f6-72af16ffb143\" (UID: \"f2a04bd0-579e-467c-b0f6-72af16ffb143\") " Dec 03 12:13:38 crc kubenswrapper[5052]: I1203 12:13:38.596232 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f2a04bd0-579e-467c-b0f6-72af16ffb143-kube-api-access-r2w7p" (OuterVolumeSpecName: "kube-api-access-r2w7p") pod "f2a04bd0-579e-467c-b0f6-72af16ffb143" (UID: "f2a04bd0-579e-467c-b0f6-72af16ffb143"). InnerVolumeSpecName "kube-api-access-r2w7p". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:13:38 crc kubenswrapper[5052]: I1203 12:13:38.596758 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f2a04bd0-579e-467c-b0f6-72af16ffb143-scripts" (OuterVolumeSpecName: "scripts") pod "f2a04bd0-579e-467c-b0f6-72af16ffb143" (UID: "f2a04bd0-579e-467c-b0f6-72af16ffb143"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:13:38 crc kubenswrapper[5052]: I1203 12:13:38.621867 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f2a04bd0-579e-467c-b0f6-72af16ffb143-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "f2a04bd0-579e-467c-b0f6-72af16ffb143" (UID: "f2a04bd0-579e-467c-b0f6-72af16ffb143"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:13:38 crc kubenswrapper[5052]: I1203 12:13:38.636302 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f2a04bd0-579e-467c-b0f6-72af16ffb143-config-data" (OuterVolumeSpecName: "config-data") pod "f2a04bd0-579e-467c-b0f6-72af16ffb143" (UID: "f2a04bd0-579e-467c-b0f6-72af16ffb143"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:13:38 crc kubenswrapper[5052]: I1203 12:13:38.693603 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f2a04bd0-579e-467c-b0f6-72af16ffb143-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:13:38 crc kubenswrapper[5052]: I1203 12:13:38.693642 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-r2w7p\" (UniqueName: \"kubernetes.io/projected/f2a04bd0-579e-467c-b0f6-72af16ffb143-kube-api-access-r2w7p\") on node \"crc\" DevicePath \"\"" Dec 03 12:13:38 crc kubenswrapper[5052]: I1203 12:13:38.693657 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/f2a04bd0-579e-467c-b0f6-72af16ffb143-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:13:38 crc kubenswrapper[5052]: I1203 12:13:38.693666 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/f2a04bd0-579e-467c-b0f6-72af16ffb143-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:13:38 crc kubenswrapper[5052]: I1203 12:13:38.844176 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:13:38 crc kubenswrapper[5052]: I1203 12:13:38.962178 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"3487eda5-6351-40b2-babe-458b38432b7f","Type":"ContainerStarted","Data":"7e5ed174c64d6f31e099a5002b59ab1714e3ae3774c2bbed9284ee785cff7b19"} Dec 03 12:13:38 crc kubenswrapper[5052]: I1203 12:13:38.962461 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"3487eda5-6351-40b2-babe-458b38432b7f","Type":"ContainerStarted","Data":"b616896b8d682e7215c2e892d7490ff87002eb4de1dd5abfe0d3693910b66db1"} Dec 03 12:13:38 crc kubenswrapper[5052]: I1203 12:13:38.963831 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"ee6da6e8-a00d-4068-b5be-9ef9fdd61448","Type":"ContainerStarted","Data":"e770ea55ceec207d15c01f4c783750c684593183790ed51dc4fb89c3e6617dd2"} Dec 03 12:13:38 crc kubenswrapper[5052]: I1203 12:13:38.968215 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-wnzql" event={"ID":"f2a04bd0-579e-467c-b0f6-72af16ffb143","Type":"ContainerDied","Data":"db4efc30d6404712f1a7ed0a00d444495fb6a372845a9022993f2afc0d9195ee"} Dec 03 12:13:38 crc kubenswrapper[5052]: I1203 12:13:38.968258 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="db4efc30d6404712f1a7ed0a00d444495fb6a372845a9022993f2afc0d9195ee" Dec 03 12:13:38 crc kubenswrapper[5052]: I1203 12:13:38.968262 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-wnzql" Dec 03 12:13:38 crc kubenswrapper[5052]: I1203 12:13:38.997273 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-metadata-0" podStartSLOduration=2.997253198 podStartE2EDuration="2.997253198s" podCreationTimestamp="2025-12-03 12:13:36 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:13:38.995310003 +0000 UTC m=+3209.583551777" watchObservedRunningTime="2025-12-03 12:13:38.997253198 +0000 UTC m=+3209.585494962" Dec 03 12:13:39 crc kubenswrapper[5052]: I1203 12:13:39.024403 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-0"] Dec 03 12:13:39 crc kubenswrapper[5052]: E1203 12:13:39.024858 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="f2a04bd0-579e-467c-b0f6-72af16ffb143" containerName="nova-cell1-conductor-db-sync" Dec 03 12:13:39 crc kubenswrapper[5052]: I1203 12:13:39.024876 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="f2a04bd0-579e-467c-b0f6-72af16ffb143" containerName="nova-cell1-conductor-db-sync" Dec 03 12:13:39 crc kubenswrapper[5052]: I1203 12:13:39.025073 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="f2a04bd0-579e-467c-b0f6-72af16ffb143" containerName="nova-cell1-conductor-db-sync" Dec 03 12:13:39 crc kubenswrapper[5052]: I1203 12:13:39.025733 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:13:39 crc kubenswrapper[5052]: I1203 12:13:39.028148 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell1-conductor-config-data" Dec 03 12:13:39 crc kubenswrapper[5052]: I1203 12:13:39.043254 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-0"] Dec 03 12:13:39 crc kubenswrapper[5052]: I1203 12:13:39.206903 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9qccn\" (UniqueName: \"kubernetes.io/projected/c2a65caa-2eec-48af-bdad-8fc19d827867-kube-api-access-9qccn\") pod \"nova-cell1-conductor-0\" (UID: \"c2a65caa-2eec-48af-bdad-8fc19d827867\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:13:39 crc kubenswrapper[5052]: I1203 12:13:39.206965 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c2a65caa-2eec-48af-bdad-8fc19d827867-combined-ca-bundle\") pod \"nova-cell1-conductor-0\" (UID: \"c2a65caa-2eec-48af-bdad-8fc19d827867\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:13:39 crc kubenswrapper[5052]: I1203 12:13:39.207053 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/c2a65caa-2eec-48af-bdad-8fc19d827867-config-data\") pod \"nova-cell1-conductor-0\" (UID: \"c2a65caa-2eec-48af-bdad-8fc19d827867\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:13:39 crc kubenswrapper[5052]: I1203 12:13:39.308432 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/c2a65caa-2eec-48af-bdad-8fc19d827867-config-data\") pod \"nova-cell1-conductor-0\" (UID: \"c2a65caa-2eec-48af-bdad-8fc19d827867\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:13:39 crc kubenswrapper[5052]: I1203 12:13:39.308601 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-9qccn\" (UniqueName: \"kubernetes.io/projected/c2a65caa-2eec-48af-bdad-8fc19d827867-kube-api-access-9qccn\") pod \"nova-cell1-conductor-0\" (UID: \"c2a65caa-2eec-48af-bdad-8fc19d827867\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:13:39 crc kubenswrapper[5052]: I1203 12:13:39.308657 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c2a65caa-2eec-48af-bdad-8fc19d827867-combined-ca-bundle\") pod \"nova-cell1-conductor-0\" (UID: \"c2a65caa-2eec-48af-bdad-8fc19d827867\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:13:39 crc kubenswrapper[5052]: I1203 12:13:39.317663 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/c2a65caa-2eec-48af-bdad-8fc19d827867-config-data\") pod \"nova-cell1-conductor-0\" (UID: \"c2a65caa-2eec-48af-bdad-8fc19d827867\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:13:39 crc kubenswrapper[5052]: I1203 12:13:39.320334 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c2a65caa-2eec-48af-bdad-8fc19d827867-combined-ca-bundle\") pod \"nova-cell1-conductor-0\" (UID: \"c2a65caa-2eec-48af-bdad-8fc19d827867\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:13:39 crc kubenswrapper[5052]: I1203 12:13:39.330325 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-9qccn\" (UniqueName: \"kubernetes.io/projected/c2a65caa-2eec-48af-bdad-8fc19d827867-kube-api-access-9qccn\") pod \"nova-cell1-conductor-0\" (UID: \"c2a65caa-2eec-48af-bdad-8fc19d827867\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:13:39 crc kubenswrapper[5052]: I1203 12:13:39.347260 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:13:39 crc kubenswrapper[5052]: I1203 12:13:39.692852 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6db171a5-bb58-4636-88d6-6b853614b4db" path="/var/lib/kubelet/pods/6db171a5-bb58-4636-88d6-6b853614b4db/volumes" Dec 03 12:13:39 crc kubenswrapper[5052]: I1203 12:13:39.931349 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-0"] Dec 03 12:13:39 crc kubenswrapper[5052]: I1203 12:13:39.969822 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:13:39 crc kubenswrapper[5052]: I1203 12:13:39.978664 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"ee6da6e8-a00d-4068-b5be-9ef9fdd61448","Type":"ContainerStarted","Data":"5e7b9ba77c14a6a49af3189f8abda89e1cd53d2c71393839dc6cdf47f4382890"} Dec 03 12:13:39 crc kubenswrapper[5052]: I1203 12:13:39.979809 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-conductor-0" event={"ID":"c2a65caa-2eec-48af-bdad-8fc19d827867","Type":"ContainerStarted","Data":"7d145059d90062ed272404781a290d6f56d30b3f105c3f72c109e13818483a5f"} Dec 03 12:13:39 crc kubenswrapper[5052]: I1203 12:13:39.981893 5052 generic.go:334] "Generic (PLEG): container finished" podID="4e5dd94c-ddfa-4d9e-b5af-fc890a712ea2" containerID="5b640e0fce110cba8efb298c6be8bbce1834d0dda04bf593b0a7dfc90b902797" exitCode=0 Dec 03 12:13:39 crc kubenswrapper[5052]: I1203 12:13:39.981959 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"4e5dd94c-ddfa-4d9e-b5af-fc890a712ea2","Type":"ContainerDied","Data":"5b640e0fce110cba8efb298c6be8bbce1834d0dda04bf593b0a7dfc90b902797"} Dec 03 12:13:39 crc kubenswrapper[5052]: I1203 12:13:39.981999 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"4e5dd94c-ddfa-4d9e-b5af-fc890a712ea2","Type":"ContainerDied","Data":"5b4bfad21d120433762281e0e2a1822eec048f9e35f8c8c3e513957ccd3b308e"} Dec 03 12:13:39 crc kubenswrapper[5052]: I1203 12:13:39.982019 5052 scope.go:117] "RemoveContainer" containerID="5b640e0fce110cba8efb298c6be8bbce1834d0dda04bf593b0a7dfc90b902797" Dec 03 12:13:39 crc kubenswrapper[5052]: I1203 12:13:39.981968 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:13:40 crc kubenswrapper[5052]: I1203 12:13:40.015890 5052 scope.go:117] "RemoveContainer" containerID="384172a9d89e7ce7470e93a792ed1e7f0c5dd86b20fdbac86dcf823b305d5cb6" Dec 03 12:13:40 crc kubenswrapper[5052]: I1203 12:13:40.016666 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-scheduler-0" podStartSLOduration=3.016645394 podStartE2EDuration="3.016645394s" podCreationTimestamp="2025-12-03 12:13:37 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:13:40.00667615 +0000 UTC m=+3210.594917914" watchObservedRunningTime="2025-12-03 12:13:40.016645394 +0000 UTC m=+3210.604887158" Dec 03 12:13:40 crc kubenswrapper[5052]: I1203 12:13:40.038720 5052 scope.go:117] "RemoveContainer" containerID="5b640e0fce110cba8efb298c6be8bbce1834d0dda04bf593b0a7dfc90b902797" Dec 03 12:13:40 crc kubenswrapper[5052]: E1203 12:13:40.039164 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5b640e0fce110cba8efb298c6be8bbce1834d0dda04bf593b0a7dfc90b902797\": container with ID starting with 5b640e0fce110cba8efb298c6be8bbce1834d0dda04bf593b0a7dfc90b902797 not found: ID does not exist" containerID="5b640e0fce110cba8efb298c6be8bbce1834d0dda04bf593b0a7dfc90b902797" Dec 03 12:13:40 crc kubenswrapper[5052]: I1203 12:13:40.039221 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5b640e0fce110cba8efb298c6be8bbce1834d0dda04bf593b0a7dfc90b902797"} err="failed to get container status \"5b640e0fce110cba8efb298c6be8bbce1834d0dda04bf593b0a7dfc90b902797\": rpc error: code = NotFound desc = could not find container \"5b640e0fce110cba8efb298c6be8bbce1834d0dda04bf593b0a7dfc90b902797\": container with ID starting with 5b640e0fce110cba8efb298c6be8bbce1834d0dda04bf593b0a7dfc90b902797 not found: ID does not exist" Dec 03 12:13:40 crc kubenswrapper[5052]: I1203 12:13:40.039254 5052 scope.go:117] "RemoveContainer" containerID="384172a9d89e7ce7470e93a792ed1e7f0c5dd86b20fdbac86dcf823b305d5cb6" Dec 03 12:13:40 crc kubenswrapper[5052]: E1203 12:13:40.039729 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"384172a9d89e7ce7470e93a792ed1e7f0c5dd86b20fdbac86dcf823b305d5cb6\": container with ID starting with 384172a9d89e7ce7470e93a792ed1e7f0c5dd86b20fdbac86dcf823b305d5cb6 not found: ID does not exist" containerID="384172a9d89e7ce7470e93a792ed1e7f0c5dd86b20fdbac86dcf823b305d5cb6" Dec 03 12:13:40 crc kubenswrapper[5052]: I1203 12:13:40.039756 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"384172a9d89e7ce7470e93a792ed1e7f0c5dd86b20fdbac86dcf823b305d5cb6"} err="failed to get container status \"384172a9d89e7ce7470e93a792ed1e7f0c5dd86b20fdbac86dcf823b305d5cb6\": rpc error: code = NotFound desc = could not find container \"384172a9d89e7ce7470e93a792ed1e7f0c5dd86b20fdbac86dcf823b305d5cb6\": container with ID starting with 384172a9d89e7ce7470e93a792ed1e7f0c5dd86b20fdbac86dcf823b305d5cb6 not found: ID does not exist" Dec 03 12:13:40 crc kubenswrapper[5052]: I1203 12:13:40.121144 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4vk9t\" (UniqueName: \"kubernetes.io/projected/4e5dd94c-ddfa-4d9e-b5af-fc890a712ea2-kube-api-access-4vk9t\") pod \"4e5dd94c-ddfa-4d9e-b5af-fc890a712ea2\" (UID: \"4e5dd94c-ddfa-4d9e-b5af-fc890a712ea2\") " Dec 03 12:13:40 crc kubenswrapper[5052]: I1203 12:13:40.121373 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/4e5dd94c-ddfa-4d9e-b5af-fc890a712ea2-logs\") pod \"4e5dd94c-ddfa-4d9e-b5af-fc890a712ea2\" (UID: \"4e5dd94c-ddfa-4d9e-b5af-fc890a712ea2\") " Dec 03 12:13:40 crc kubenswrapper[5052]: I1203 12:13:40.121415 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4e5dd94c-ddfa-4d9e-b5af-fc890a712ea2-combined-ca-bundle\") pod \"4e5dd94c-ddfa-4d9e-b5af-fc890a712ea2\" (UID: \"4e5dd94c-ddfa-4d9e-b5af-fc890a712ea2\") " Dec 03 12:13:40 crc kubenswrapper[5052]: I1203 12:13:40.121502 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/4e5dd94c-ddfa-4d9e-b5af-fc890a712ea2-config-data\") pod \"4e5dd94c-ddfa-4d9e-b5af-fc890a712ea2\" (UID: \"4e5dd94c-ddfa-4d9e-b5af-fc890a712ea2\") " Dec 03 12:13:40 crc kubenswrapper[5052]: I1203 12:13:40.122500 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e5dd94c-ddfa-4d9e-b5af-fc890a712ea2-logs" (OuterVolumeSpecName: "logs") pod "4e5dd94c-ddfa-4d9e-b5af-fc890a712ea2" (UID: "4e5dd94c-ddfa-4d9e-b5af-fc890a712ea2"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:13:40 crc kubenswrapper[5052]: I1203 12:13:40.125615 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4e5dd94c-ddfa-4d9e-b5af-fc890a712ea2-kube-api-access-4vk9t" (OuterVolumeSpecName: "kube-api-access-4vk9t") pod "4e5dd94c-ddfa-4d9e-b5af-fc890a712ea2" (UID: "4e5dd94c-ddfa-4d9e-b5af-fc890a712ea2"). InnerVolumeSpecName "kube-api-access-4vk9t". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:13:40 crc kubenswrapper[5052]: I1203 12:13:40.148005 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4e5dd94c-ddfa-4d9e-b5af-fc890a712ea2-config-data" (OuterVolumeSpecName: "config-data") pod "4e5dd94c-ddfa-4d9e-b5af-fc890a712ea2" (UID: "4e5dd94c-ddfa-4d9e-b5af-fc890a712ea2"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:13:40 crc kubenswrapper[5052]: I1203 12:13:40.170054 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4e5dd94c-ddfa-4d9e-b5af-fc890a712ea2-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "4e5dd94c-ddfa-4d9e-b5af-fc890a712ea2" (UID: "4e5dd94c-ddfa-4d9e-b5af-fc890a712ea2"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:13:40 crc kubenswrapper[5052]: I1203 12:13:40.218502 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:13:40 crc kubenswrapper[5052]: I1203 12:13:40.224096 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/4e5dd94c-ddfa-4d9e-b5af-fc890a712ea2-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:13:40 crc kubenswrapper[5052]: I1203 12:13:40.224249 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4e5dd94c-ddfa-4d9e-b5af-fc890a712ea2-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:13:40 crc kubenswrapper[5052]: I1203 12:13:40.224320 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/4e5dd94c-ddfa-4d9e-b5af-fc890a712ea2-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:13:40 crc kubenswrapper[5052]: I1203 12:13:40.224374 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-4vk9t\" (UniqueName: \"kubernetes.io/projected/4e5dd94c-ddfa-4d9e-b5af-fc890a712ea2-kube-api-access-4vk9t\") on node \"crc\" DevicePath \"\"" Dec 03 12:13:40 crc kubenswrapper[5052]: I1203 12:13:40.243843 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:13:40 crc kubenswrapper[5052]: I1203 12:13:40.354121 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:13:40 crc kubenswrapper[5052]: I1203 12:13:40.366375 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:13:40 crc kubenswrapper[5052]: I1203 12:13:40.380341 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:13:40 crc kubenswrapper[5052]: E1203 12:13:40.380937 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="4e5dd94c-ddfa-4d9e-b5af-fc890a712ea2" containerName="nova-api-log" Dec 03 12:13:40 crc kubenswrapper[5052]: I1203 12:13:40.380962 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e5dd94c-ddfa-4d9e-b5af-fc890a712ea2" containerName="nova-api-log" Dec 03 12:13:40 crc kubenswrapper[5052]: E1203 12:13:40.380990 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="4e5dd94c-ddfa-4d9e-b5af-fc890a712ea2" containerName="nova-api-api" Dec 03 12:13:40 crc kubenswrapper[5052]: I1203 12:13:40.380999 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e5dd94c-ddfa-4d9e-b5af-fc890a712ea2" containerName="nova-api-api" Dec 03 12:13:40 crc kubenswrapper[5052]: I1203 12:13:40.381245 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="4e5dd94c-ddfa-4d9e-b5af-fc890a712ea2" containerName="nova-api-api" Dec 03 12:13:40 crc kubenswrapper[5052]: I1203 12:13:40.381273 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="4e5dd94c-ddfa-4d9e-b5af-fc890a712ea2" containerName="nova-api-log" Dec 03 12:13:40 crc kubenswrapper[5052]: I1203 12:13:40.382697 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:13:40 crc kubenswrapper[5052]: I1203 12:13:40.385233 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-api-config-data" Dec 03 12:13:40 crc kubenswrapper[5052]: I1203 12:13:40.390265 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:13:40 crc kubenswrapper[5052]: I1203 12:13:40.529470 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/a8676e0f-8c55-4277-b100-1e84ba22abc9-logs\") pod \"nova-api-0\" (UID: \"a8676e0f-8c55-4277-b100-1e84ba22abc9\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:13:40 crc kubenswrapper[5052]: I1203 12:13:40.529532 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a8676e0f-8c55-4277-b100-1e84ba22abc9-config-data\") pod \"nova-api-0\" (UID: \"a8676e0f-8c55-4277-b100-1e84ba22abc9\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:13:40 crc kubenswrapper[5052]: I1203 12:13:40.529560 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a8676e0f-8c55-4277-b100-1e84ba22abc9-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"a8676e0f-8c55-4277-b100-1e84ba22abc9\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:13:40 crc kubenswrapper[5052]: I1203 12:13:40.530203 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mf8l7\" (UniqueName: \"kubernetes.io/projected/a8676e0f-8c55-4277-b100-1e84ba22abc9-kube-api-access-mf8l7\") pod \"nova-api-0\" (UID: \"a8676e0f-8c55-4277-b100-1e84ba22abc9\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:13:40 crc kubenswrapper[5052]: I1203 12:13:40.631884 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/a8676e0f-8c55-4277-b100-1e84ba22abc9-logs\") pod \"nova-api-0\" (UID: \"a8676e0f-8c55-4277-b100-1e84ba22abc9\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:13:40 crc kubenswrapper[5052]: I1203 12:13:40.631925 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a8676e0f-8c55-4277-b100-1e84ba22abc9-config-data\") pod \"nova-api-0\" (UID: \"a8676e0f-8c55-4277-b100-1e84ba22abc9\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:13:40 crc kubenswrapper[5052]: I1203 12:13:40.631952 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a8676e0f-8c55-4277-b100-1e84ba22abc9-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"a8676e0f-8c55-4277-b100-1e84ba22abc9\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:13:40 crc kubenswrapper[5052]: I1203 12:13:40.632061 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-mf8l7\" (UniqueName: \"kubernetes.io/projected/a8676e0f-8c55-4277-b100-1e84ba22abc9-kube-api-access-mf8l7\") pod \"nova-api-0\" (UID: \"a8676e0f-8c55-4277-b100-1e84ba22abc9\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:13:40 crc kubenswrapper[5052]: I1203 12:13:40.632815 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/a8676e0f-8c55-4277-b100-1e84ba22abc9-logs\") pod \"nova-api-0\" (UID: \"a8676e0f-8c55-4277-b100-1e84ba22abc9\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:13:40 crc kubenswrapper[5052]: I1203 12:13:40.637530 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a8676e0f-8c55-4277-b100-1e84ba22abc9-config-data\") pod \"nova-api-0\" (UID: \"a8676e0f-8c55-4277-b100-1e84ba22abc9\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:13:40 crc kubenswrapper[5052]: I1203 12:13:40.650034 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-mf8l7\" (UniqueName: \"kubernetes.io/projected/a8676e0f-8c55-4277-b100-1e84ba22abc9-kube-api-access-mf8l7\") pod \"nova-api-0\" (UID: \"a8676e0f-8c55-4277-b100-1e84ba22abc9\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:13:40 crc kubenswrapper[5052]: I1203 12:13:40.650134 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a8676e0f-8c55-4277-b100-1e84ba22abc9-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"a8676e0f-8c55-4277-b100-1e84ba22abc9\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:13:40 crc kubenswrapper[5052]: I1203 12:13:40.698587 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:13:40 crc kubenswrapper[5052]: I1203 12:13:40.996614 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-conductor-0" event={"ID":"c2a65caa-2eec-48af-bdad-8fc19d827867","Type":"ContainerStarted","Data":"28049a9b7828f990e7cdf628c7744bd85e07ead3fc6256992ef7d5bf23e045a3"} Dec 03 12:13:40 crc kubenswrapper[5052]: I1203 12:13:40.997185 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:13:41 crc kubenswrapper[5052]: I1203 12:13:41.011239 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-cell1-conductor-0" podStartSLOduration=2.011214022 podStartE2EDuration="2.011214022s" podCreationTimestamp="2025-12-03 12:13:39 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:13:41.011186112 +0000 UTC m=+3211.599427896" watchObservedRunningTime="2025-12-03 12:13:41.011214022 +0000 UTC m=+3211.599455786" Dec 03 12:13:41 crc kubenswrapper[5052]: I1203 12:13:41.024226 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:13:41 crc kubenswrapper[5052]: I1203 12:13:41.145296 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:13:41 crc kubenswrapper[5052]: I1203 12:13:41.683587 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4e5dd94c-ddfa-4d9e-b5af-fc890a712ea2" path="/var/lib/kubelet/pods/4e5dd94c-ddfa-4d9e-b5af-fc890a712ea2/volumes" Dec 03 12:13:42 crc kubenswrapper[5052]: I1203 12:13:42.011269 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"a8676e0f-8c55-4277-b100-1e84ba22abc9","Type":"ContainerStarted","Data":"c2b50503852254698d9532f6a6a498bfc995d2771d2ebb215bddaec5c35f5522"} Dec 03 12:13:42 crc kubenswrapper[5052]: I1203 12:13:42.011316 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"a8676e0f-8c55-4277-b100-1e84ba22abc9","Type":"ContainerStarted","Data":"627a90164ddbda989135947935a8b58484e90e221d3808645e49f99985e3aa76"} Dec 03 12:13:42 crc kubenswrapper[5052]: I1203 12:13:42.011330 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"a8676e0f-8c55-4277-b100-1e84ba22abc9","Type":"ContainerStarted","Data":"9390647b5d0ffab7aad012c4ed70464f64be8b01539a8265e3e8ce8e6869b571"} Dec 03 12:13:42 crc kubenswrapper[5052]: I1203 12:13:42.033111 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-api-0" podStartSLOduration=2.0330895 podStartE2EDuration="2.0330895s" podCreationTimestamp="2025-12-03 12:13:40 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:13:42.026656607 +0000 UTC m=+3212.614898371" watchObservedRunningTime="2025-12-03 12:13:42.0330895 +0000 UTC m=+3212.621331264" Dec 03 12:13:42 crc kubenswrapper[5052]: I1203 12:13:42.342208 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:13:42 crc kubenswrapper[5052]: I1203 12:13:42.342353 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:13:42 crc kubenswrapper[5052]: I1203 12:13:42.673915 5052 scope.go:117] "RemoveContainer" containerID="63298107b3d090deff9ed92f3d613e5b1c86d9137684d117500257db8ee1c805" Dec 03 12:13:42 crc kubenswrapper[5052]: E1203 12:13:42.674377 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 12:13:43 crc kubenswrapper[5052]: I1203 12:13:43.331569 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:13:47 crc kubenswrapper[5052]: I1203 12:13:47.342586 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:13:47 crc kubenswrapper[5052]: I1203 12:13:47.343565 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:13:48 crc kubenswrapper[5052]: I1203 12:13:48.330951 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:13:48 crc kubenswrapper[5052]: I1203 12:13:48.355675 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-metadata-0" podUID="3487eda5-6351-40b2-babe-458b38432b7f" containerName="nova-metadata-log" probeResult="failure" output="Get \"https://10.217.1.34:8775/\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Dec 03 12:13:48 crc kubenswrapper[5052]: I1203 12:13:48.355732 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-metadata-0" podUID="3487eda5-6351-40b2-babe-458b38432b7f" containerName="nova-metadata-metadata" probeResult="failure" output="Get \"https://10.217.1.34:8775/\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Dec 03 12:13:48 crc kubenswrapper[5052]: I1203 12:13:48.356261 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:13:49 crc kubenswrapper[5052]: I1203 12:13:49.007854 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:13:49 crc kubenswrapper[5052]: I1203 12:13:49.113632 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:13:49 crc kubenswrapper[5052]: I1203 12:13:49.374845 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:13:49 crc kubenswrapper[5052]: I1203 12:13:49.892927 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell1-cell-mapping-tb27n"] Dec 03 12:13:49 crc kubenswrapper[5052]: I1203 12:13:49.894937 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-cell-mapping-tb27n" Dec 03 12:13:49 crc kubenswrapper[5052]: I1203 12:13:49.898210 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell1-manage-scripts" Dec 03 12:13:49 crc kubenswrapper[5052]: I1203 12:13:49.899356 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell1-manage-config-data" Dec 03 12:13:49 crc kubenswrapper[5052]: I1203 12:13:49.932799 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-cell-mapping-tb27n"] Dec 03 12:13:50 crc kubenswrapper[5052]: I1203 12:13:50.063660 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ndps4\" (UniqueName: \"kubernetes.io/projected/03b10bd5-7748-43f4-a4e2-346928c502f1-kube-api-access-ndps4\") pod \"nova-cell1-cell-mapping-tb27n\" (UID: \"03b10bd5-7748-43f4-a4e2-346928c502f1\") " pod="openstack-kuttl-tests/nova-cell1-cell-mapping-tb27n" Dec 03 12:13:50 crc kubenswrapper[5052]: I1203 12:13:50.064037 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/03b10bd5-7748-43f4-a4e2-346928c502f1-config-data\") pod \"nova-cell1-cell-mapping-tb27n\" (UID: \"03b10bd5-7748-43f4-a4e2-346928c502f1\") " pod="openstack-kuttl-tests/nova-cell1-cell-mapping-tb27n" Dec 03 12:13:50 crc kubenswrapper[5052]: I1203 12:13:50.064118 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/03b10bd5-7748-43f4-a4e2-346928c502f1-combined-ca-bundle\") pod \"nova-cell1-cell-mapping-tb27n\" (UID: \"03b10bd5-7748-43f4-a4e2-346928c502f1\") " pod="openstack-kuttl-tests/nova-cell1-cell-mapping-tb27n" Dec 03 12:13:50 crc kubenswrapper[5052]: I1203 12:13:50.064276 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/03b10bd5-7748-43f4-a4e2-346928c502f1-scripts\") pod \"nova-cell1-cell-mapping-tb27n\" (UID: \"03b10bd5-7748-43f4-a4e2-346928c502f1\") " pod="openstack-kuttl-tests/nova-cell1-cell-mapping-tb27n" Dec 03 12:13:50 crc kubenswrapper[5052]: I1203 12:13:50.165776 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/03b10bd5-7748-43f4-a4e2-346928c502f1-scripts\") pod \"nova-cell1-cell-mapping-tb27n\" (UID: \"03b10bd5-7748-43f4-a4e2-346928c502f1\") " pod="openstack-kuttl-tests/nova-cell1-cell-mapping-tb27n" Dec 03 12:13:50 crc kubenswrapper[5052]: I1203 12:13:50.165849 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-ndps4\" (UniqueName: \"kubernetes.io/projected/03b10bd5-7748-43f4-a4e2-346928c502f1-kube-api-access-ndps4\") pod \"nova-cell1-cell-mapping-tb27n\" (UID: \"03b10bd5-7748-43f4-a4e2-346928c502f1\") " pod="openstack-kuttl-tests/nova-cell1-cell-mapping-tb27n" Dec 03 12:13:50 crc kubenswrapper[5052]: I1203 12:13:50.165887 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/03b10bd5-7748-43f4-a4e2-346928c502f1-config-data\") pod \"nova-cell1-cell-mapping-tb27n\" (UID: \"03b10bd5-7748-43f4-a4e2-346928c502f1\") " pod="openstack-kuttl-tests/nova-cell1-cell-mapping-tb27n" Dec 03 12:13:50 crc kubenswrapper[5052]: I1203 12:13:50.165919 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/03b10bd5-7748-43f4-a4e2-346928c502f1-combined-ca-bundle\") pod \"nova-cell1-cell-mapping-tb27n\" (UID: \"03b10bd5-7748-43f4-a4e2-346928c502f1\") " pod="openstack-kuttl-tests/nova-cell1-cell-mapping-tb27n" Dec 03 12:13:50 crc kubenswrapper[5052]: I1203 12:13:50.174884 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/03b10bd5-7748-43f4-a4e2-346928c502f1-config-data\") pod \"nova-cell1-cell-mapping-tb27n\" (UID: \"03b10bd5-7748-43f4-a4e2-346928c502f1\") " pod="openstack-kuttl-tests/nova-cell1-cell-mapping-tb27n" Dec 03 12:13:50 crc kubenswrapper[5052]: I1203 12:13:50.175006 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/03b10bd5-7748-43f4-a4e2-346928c502f1-scripts\") pod \"nova-cell1-cell-mapping-tb27n\" (UID: \"03b10bd5-7748-43f4-a4e2-346928c502f1\") " pod="openstack-kuttl-tests/nova-cell1-cell-mapping-tb27n" Dec 03 12:13:50 crc kubenswrapper[5052]: I1203 12:13:50.175083 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/03b10bd5-7748-43f4-a4e2-346928c502f1-combined-ca-bundle\") pod \"nova-cell1-cell-mapping-tb27n\" (UID: \"03b10bd5-7748-43f4-a4e2-346928c502f1\") " pod="openstack-kuttl-tests/nova-cell1-cell-mapping-tb27n" Dec 03 12:13:50 crc kubenswrapper[5052]: I1203 12:13:50.185188 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-ndps4\" (UniqueName: \"kubernetes.io/projected/03b10bd5-7748-43f4-a4e2-346928c502f1-kube-api-access-ndps4\") pod \"nova-cell1-cell-mapping-tb27n\" (UID: \"03b10bd5-7748-43f4-a4e2-346928c502f1\") " pod="openstack-kuttl-tests/nova-cell1-cell-mapping-tb27n" Dec 03 12:13:50 crc kubenswrapper[5052]: I1203 12:13:50.283133 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-cell-mapping-tb27n" Dec 03 12:13:50 crc kubenswrapper[5052]: I1203 12:13:50.698908 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:13:50 crc kubenswrapper[5052]: I1203 12:13:50.699501 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:13:50 crc kubenswrapper[5052]: I1203 12:13:50.765119 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-cell-mapping-tb27n"] Dec 03 12:13:51 crc kubenswrapper[5052]: I1203 12:13:51.113501 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-cell-mapping-tb27n" event={"ID":"03b10bd5-7748-43f4-a4e2-346928c502f1","Type":"ContainerStarted","Data":"6372dcfe9923fddf61186704a3c44371519577584921fc8a7994171c3519040e"} Dec 03 12:13:51 crc kubenswrapper[5052]: I1203 12:13:51.780775 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-api-0" podUID="a8676e0f-8c55-4277-b100-1e84ba22abc9" containerName="nova-api-api" probeResult="failure" output="Get \"http://10.217.1.37:8774/\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Dec 03 12:13:51 crc kubenswrapper[5052]: I1203 12:13:51.780790 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-api-0" podUID="a8676e0f-8c55-4277-b100-1e84ba22abc9" containerName="nova-api-log" probeResult="failure" output="Get \"http://10.217.1.37:8774/\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Dec 03 12:13:52 crc kubenswrapper[5052]: I1203 12:13:52.127807 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-cell-mapping-tb27n" event={"ID":"03b10bd5-7748-43f4-a4e2-346928c502f1","Type":"ContainerStarted","Data":"63ba8a0e2bdcd79bccfbfa1aced2e45ca9e753ae90253a074759197725efe5f1"} Dec 03 12:13:52 crc kubenswrapper[5052]: I1203 12:13:52.158111 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-cell1-cell-mapping-tb27n" podStartSLOduration=3.158087638 podStartE2EDuration="3.158087638s" podCreationTimestamp="2025-12-03 12:13:49 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:13:52.14940271 +0000 UTC m=+3222.737644474" watchObservedRunningTime="2025-12-03 12:13:52.158087638 +0000 UTC m=+3222.746329402" Dec 03 12:13:52 crc kubenswrapper[5052]: I1203 12:13:52.862066 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/kube-state-metrics-0"] Dec 03 12:13:52 crc kubenswrapper[5052]: I1203 12:13:52.862291 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/kube-state-metrics-0" podUID="cdef7499-3235-479a-8065-de26b912d8ea" containerName="kube-state-metrics" containerID="cri-o://7f0cbafa963452fc9d1387d850c95247fd3653a544ede26e15ff50d89f0a33f4" gracePeriod=30 Dec 03 12:13:53 crc kubenswrapper[5052]: I1203 12:13:53.141064 5052 generic.go:334] "Generic (PLEG): container finished" podID="cdef7499-3235-479a-8065-de26b912d8ea" containerID="7f0cbafa963452fc9d1387d850c95247fd3653a544ede26e15ff50d89f0a33f4" exitCode=2 Dec 03 12:13:53 crc kubenswrapper[5052]: I1203 12:13:53.142051 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/kube-state-metrics-0" event={"ID":"cdef7499-3235-479a-8065-de26b912d8ea","Type":"ContainerDied","Data":"7f0cbafa963452fc9d1387d850c95247fd3653a544ede26e15ff50d89f0a33f4"} Dec 03 12:13:53 crc kubenswrapper[5052]: I1203 12:13:53.518149 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:13:53 crc kubenswrapper[5052]: I1203 12:13:53.641392 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kmf8k\" (UniqueName: \"kubernetes.io/projected/cdef7499-3235-479a-8065-de26b912d8ea-kube-api-access-kmf8k\") pod \"cdef7499-3235-479a-8065-de26b912d8ea\" (UID: \"cdef7499-3235-479a-8065-de26b912d8ea\") " Dec 03 12:13:53 crc kubenswrapper[5052]: I1203 12:13:53.647798 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cdef7499-3235-479a-8065-de26b912d8ea-kube-api-access-kmf8k" (OuterVolumeSpecName: "kube-api-access-kmf8k") pod "cdef7499-3235-479a-8065-de26b912d8ea" (UID: "cdef7499-3235-479a-8065-de26b912d8ea"). InnerVolumeSpecName "kube-api-access-kmf8k". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:13:53 crc kubenswrapper[5052]: I1203 12:13:53.673159 5052 scope.go:117] "RemoveContainer" containerID="63298107b3d090deff9ed92f3d613e5b1c86d9137684d117500257db8ee1c805" Dec 03 12:13:53 crc kubenswrapper[5052]: E1203 12:13:53.673657 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 12:13:53 crc kubenswrapper[5052]: I1203 12:13:53.743983 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-kmf8k\" (UniqueName: \"kubernetes.io/projected/cdef7499-3235-479a-8065-de26b912d8ea-kube-api-access-kmf8k\") on node \"crc\" DevicePath \"\"" Dec 03 12:13:54 crc kubenswrapper[5052]: I1203 12:13:54.155797 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/kube-state-metrics-0" event={"ID":"cdef7499-3235-479a-8065-de26b912d8ea","Type":"ContainerDied","Data":"478edfe57f1c5c41ae50aa682267c2e73e1011cc8a294da2c80f1a3ec3757ef1"} Dec 03 12:13:54 crc kubenswrapper[5052]: I1203 12:13:54.155881 5052 scope.go:117] "RemoveContainer" containerID="7f0cbafa963452fc9d1387d850c95247fd3653a544ede26e15ff50d89f0a33f4" Dec 03 12:13:54 crc kubenswrapper[5052]: I1203 12:13:54.156001 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:13:54 crc kubenswrapper[5052]: I1203 12:13:54.181076 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/kube-state-metrics-0"] Dec 03 12:13:54 crc kubenswrapper[5052]: I1203 12:13:54.201923 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/kube-state-metrics-0"] Dec 03 12:13:54 crc kubenswrapper[5052]: I1203 12:13:54.213013 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/kube-state-metrics-0"] Dec 03 12:13:54 crc kubenswrapper[5052]: E1203 12:13:54.213520 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="cdef7499-3235-479a-8065-de26b912d8ea" containerName="kube-state-metrics" Dec 03 12:13:54 crc kubenswrapper[5052]: I1203 12:13:54.213549 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="cdef7499-3235-479a-8065-de26b912d8ea" containerName="kube-state-metrics" Dec 03 12:13:54 crc kubenswrapper[5052]: I1203 12:13:54.213730 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="cdef7499-3235-479a-8065-de26b912d8ea" containerName="kube-state-metrics" Dec 03 12:13:54 crc kubenswrapper[5052]: I1203 12:13:54.214459 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:13:54 crc kubenswrapper[5052]: I1203 12:13:54.216900 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"kube-state-metrics-tls-config" Dec 03 12:13:54 crc kubenswrapper[5052]: I1203 12:13:54.217096 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-kube-state-metrics-svc" Dec 03 12:13:54 crc kubenswrapper[5052]: I1203 12:13:54.221912 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/kube-state-metrics-0"] Dec 03 12:13:54 crc kubenswrapper[5052]: I1203 12:13:54.353326 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls-config\" (UniqueName: \"kubernetes.io/secret/dfdd8013-ee6e-4ec6-b5d1-fb423f444430-kube-state-metrics-tls-config\") pod \"kube-state-metrics-0\" (UID: \"dfdd8013-ee6e-4ec6-b5d1-fb423f444430\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:13:54 crc kubenswrapper[5052]: I1203 12:13:54.353423 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ljg7n\" (UniqueName: \"kubernetes.io/projected/dfdd8013-ee6e-4ec6-b5d1-fb423f444430-kube-api-access-ljg7n\") pod \"kube-state-metrics-0\" (UID: \"dfdd8013-ee6e-4ec6-b5d1-fb423f444430\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:13:54 crc kubenswrapper[5052]: I1203 12:13:54.353506 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/dfdd8013-ee6e-4ec6-b5d1-fb423f444430-combined-ca-bundle\") pod \"kube-state-metrics-0\" (UID: \"dfdd8013-ee6e-4ec6-b5d1-fb423f444430\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:13:54 crc kubenswrapper[5052]: I1203 12:13:54.353561 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls-certs\" (UniqueName: \"kubernetes.io/secret/dfdd8013-ee6e-4ec6-b5d1-fb423f444430-kube-state-metrics-tls-certs\") pod \"kube-state-metrics-0\" (UID: \"dfdd8013-ee6e-4ec6-b5d1-fb423f444430\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:13:54 crc kubenswrapper[5052]: I1203 12:13:54.455232 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls-config\" (UniqueName: \"kubernetes.io/secret/dfdd8013-ee6e-4ec6-b5d1-fb423f444430-kube-state-metrics-tls-config\") pod \"kube-state-metrics-0\" (UID: \"dfdd8013-ee6e-4ec6-b5d1-fb423f444430\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:13:54 crc kubenswrapper[5052]: I1203 12:13:54.455370 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-ljg7n\" (UniqueName: \"kubernetes.io/projected/dfdd8013-ee6e-4ec6-b5d1-fb423f444430-kube-api-access-ljg7n\") pod \"kube-state-metrics-0\" (UID: \"dfdd8013-ee6e-4ec6-b5d1-fb423f444430\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:13:54 crc kubenswrapper[5052]: I1203 12:13:54.455428 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/dfdd8013-ee6e-4ec6-b5d1-fb423f444430-combined-ca-bundle\") pod \"kube-state-metrics-0\" (UID: \"dfdd8013-ee6e-4ec6-b5d1-fb423f444430\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:13:54 crc kubenswrapper[5052]: I1203 12:13:54.455470 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls-certs\" (UniqueName: \"kubernetes.io/secret/dfdd8013-ee6e-4ec6-b5d1-fb423f444430-kube-state-metrics-tls-certs\") pod \"kube-state-metrics-0\" (UID: \"dfdd8013-ee6e-4ec6-b5d1-fb423f444430\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:13:54 crc kubenswrapper[5052]: I1203 12:13:54.460318 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/dfdd8013-ee6e-4ec6-b5d1-fb423f444430-combined-ca-bundle\") pod \"kube-state-metrics-0\" (UID: \"dfdd8013-ee6e-4ec6-b5d1-fb423f444430\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:13:54 crc kubenswrapper[5052]: I1203 12:13:54.463401 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls-config\" (UniqueName: \"kubernetes.io/secret/dfdd8013-ee6e-4ec6-b5d1-fb423f444430-kube-state-metrics-tls-config\") pod \"kube-state-metrics-0\" (UID: \"dfdd8013-ee6e-4ec6-b5d1-fb423f444430\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:13:54 crc kubenswrapper[5052]: I1203 12:13:54.471020 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls-certs\" (UniqueName: \"kubernetes.io/secret/dfdd8013-ee6e-4ec6-b5d1-fb423f444430-kube-state-metrics-tls-certs\") pod \"kube-state-metrics-0\" (UID: \"dfdd8013-ee6e-4ec6-b5d1-fb423f444430\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:13:54 crc kubenswrapper[5052]: I1203 12:13:54.478698 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-ljg7n\" (UniqueName: \"kubernetes.io/projected/dfdd8013-ee6e-4ec6-b5d1-fb423f444430-kube-api-access-ljg7n\") pod \"kube-state-metrics-0\" (UID: \"dfdd8013-ee6e-4ec6-b5d1-fb423f444430\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:13:54 crc kubenswrapper[5052]: I1203 12:13:54.537415 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:13:54 crc kubenswrapper[5052]: I1203 12:13:54.800880 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:13:54 crc kubenswrapper[5052]: I1203 12:13:54.801441 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="6a508669-24ef-4c02-82b4-bfb8b3b9420b" containerName="ceilometer-central-agent" containerID="cri-o://f0ec59ff17f5ac98f8efe3ba2d6110db4488e144489ba6f93945be948047d889" gracePeriod=30 Dec 03 12:13:54 crc kubenswrapper[5052]: I1203 12:13:54.801681 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="6a508669-24ef-4c02-82b4-bfb8b3b9420b" containerName="proxy-httpd" containerID="cri-o://3b54ccfe9d1b5d1b336e3adcc5013135c78f03401138c69e1aac9777fd76c5ed" gracePeriod=30 Dec 03 12:13:54 crc kubenswrapper[5052]: I1203 12:13:54.801720 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="6a508669-24ef-4c02-82b4-bfb8b3b9420b" containerName="sg-core" containerID="cri-o://12cf806b890c3285f6292098c13247c6168955b831065259ec141d41eff9f0b8" gracePeriod=30 Dec 03 12:13:54 crc kubenswrapper[5052]: I1203 12:13:54.801702 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="6a508669-24ef-4c02-82b4-bfb8b3b9420b" containerName="ceilometer-notification-agent" containerID="cri-o://17d55a4176395a4183e27fbdad33c760894ac49731d57a48e1e495ee12293d29" gracePeriod=30 Dec 03 12:13:55 crc kubenswrapper[5052]: I1203 12:13:55.009323 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/kube-state-metrics-0"] Dec 03 12:13:55 crc kubenswrapper[5052]: W1203 12:13:55.009347 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddfdd8013_ee6e_4ec6_b5d1_fb423f444430.slice/crio-ca527bbb6d8d57411900c0d35849885371f2b836f9c39e33e47863b9f3b38d61 WatchSource:0}: Error finding container ca527bbb6d8d57411900c0d35849885371f2b836f9c39e33e47863b9f3b38d61: Status 404 returned error can't find the container with id ca527bbb6d8d57411900c0d35849885371f2b836f9c39e33e47863b9f3b38d61 Dec 03 12:13:55 crc kubenswrapper[5052]: I1203 12:13:55.172889 5052 generic.go:334] "Generic (PLEG): container finished" podID="6a508669-24ef-4c02-82b4-bfb8b3b9420b" containerID="3b54ccfe9d1b5d1b336e3adcc5013135c78f03401138c69e1aac9777fd76c5ed" exitCode=0 Dec 03 12:13:55 crc kubenswrapper[5052]: I1203 12:13:55.172937 5052 generic.go:334] "Generic (PLEG): container finished" podID="6a508669-24ef-4c02-82b4-bfb8b3b9420b" containerID="12cf806b890c3285f6292098c13247c6168955b831065259ec141d41eff9f0b8" exitCode=2 Dec 03 12:13:55 crc kubenswrapper[5052]: I1203 12:13:55.173008 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"6a508669-24ef-4c02-82b4-bfb8b3b9420b","Type":"ContainerDied","Data":"3b54ccfe9d1b5d1b336e3adcc5013135c78f03401138c69e1aac9777fd76c5ed"} Dec 03 12:13:55 crc kubenswrapper[5052]: I1203 12:13:55.173044 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"6a508669-24ef-4c02-82b4-bfb8b3b9420b","Type":"ContainerDied","Data":"12cf806b890c3285f6292098c13247c6168955b831065259ec141d41eff9f0b8"} Dec 03 12:13:55 crc kubenswrapper[5052]: I1203 12:13:55.175734 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/kube-state-metrics-0" event={"ID":"dfdd8013-ee6e-4ec6-b5d1-fb423f444430","Type":"ContainerStarted","Data":"ca527bbb6d8d57411900c0d35849885371f2b836f9c39e33e47863b9f3b38d61"} Dec 03 12:13:55 crc kubenswrapper[5052]: I1203 12:13:55.686844 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cdef7499-3235-479a-8065-de26b912d8ea" path="/var/lib/kubelet/pods/cdef7499-3235-479a-8065-de26b912d8ea/volumes" Dec 03 12:13:56 crc kubenswrapper[5052]: I1203 12:13:56.186831 5052 generic.go:334] "Generic (PLEG): container finished" podID="6a508669-24ef-4c02-82b4-bfb8b3b9420b" containerID="f0ec59ff17f5ac98f8efe3ba2d6110db4488e144489ba6f93945be948047d889" exitCode=0 Dec 03 12:13:56 crc kubenswrapper[5052]: I1203 12:13:56.186998 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"6a508669-24ef-4c02-82b4-bfb8b3b9420b","Type":"ContainerDied","Data":"f0ec59ff17f5ac98f8efe3ba2d6110db4488e144489ba6f93945be948047d889"} Dec 03 12:13:56 crc kubenswrapper[5052]: I1203 12:13:56.188585 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/kube-state-metrics-0" event={"ID":"dfdd8013-ee6e-4ec6-b5d1-fb423f444430","Type":"ContainerStarted","Data":"8021f4fcdb6f78190623eed4c27195e38d52e80b4781dc680a2591d5bf74777d"} Dec 03 12:13:56 crc kubenswrapper[5052]: I1203 12:13:56.188957 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:13:56 crc kubenswrapper[5052]: I1203 12:13:56.190768 5052 generic.go:334] "Generic (PLEG): container finished" podID="03b10bd5-7748-43f4-a4e2-346928c502f1" containerID="63ba8a0e2bdcd79bccfbfa1aced2e45ca9e753ae90253a074759197725efe5f1" exitCode=0 Dec 03 12:13:56 crc kubenswrapper[5052]: I1203 12:13:56.190811 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-cell-mapping-tb27n" event={"ID":"03b10bd5-7748-43f4-a4e2-346928c502f1","Type":"ContainerDied","Data":"63ba8a0e2bdcd79bccfbfa1aced2e45ca9e753ae90253a074759197725efe5f1"} Dec 03 12:13:56 crc kubenswrapper[5052]: I1203 12:13:56.217231 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/kube-state-metrics-0" podStartSLOduration=1.842802627 podStartE2EDuration="2.217213768s" podCreationTimestamp="2025-12-03 12:13:54 +0000 UTC" firstStartedPulling="2025-12-03 12:13:55.018717687 +0000 UTC m=+3225.606959451" lastFinishedPulling="2025-12-03 12:13:55.393128828 +0000 UTC m=+3225.981370592" observedRunningTime="2025-12-03 12:13:56.20957578 +0000 UTC m=+3226.797817554" watchObservedRunningTime="2025-12-03 12:13:56.217213768 +0000 UTC m=+3226.805455532" Dec 03 12:13:57 crc kubenswrapper[5052]: I1203 12:13:57.354659 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:13:57 crc kubenswrapper[5052]: I1203 12:13:57.355980 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:13:57 crc kubenswrapper[5052]: I1203 12:13:57.375488 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:13:57 crc kubenswrapper[5052]: I1203 12:13:57.607187 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-cell-mapping-tb27n" Dec 03 12:13:57 crc kubenswrapper[5052]: I1203 12:13:57.724320 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/03b10bd5-7748-43f4-a4e2-346928c502f1-combined-ca-bundle\") pod \"03b10bd5-7748-43f4-a4e2-346928c502f1\" (UID: \"03b10bd5-7748-43f4-a4e2-346928c502f1\") " Dec 03 12:13:57 crc kubenswrapper[5052]: I1203 12:13:57.724457 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/03b10bd5-7748-43f4-a4e2-346928c502f1-config-data\") pod \"03b10bd5-7748-43f4-a4e2-346928c502f1\" (UID: \"03b10bd5-7748-43f4-a4e2-346928c502f1\") " Dec 03 12:13:57 crc kubenswrapper[5052]: I1203 12:13:57.724530 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/03b10bd5-7748-43f4-a4e2-346928c502f1-scripts\") pod \"03b10bd5-7748-43f4-a4e2-346928c502f1\" (UID: \"03b10bd5-7748-43f4-a4e2-346928c502f1\") " Dec 03 12:13:57 crc kubenswrapper[5052]: I1203 12:13:57.724575 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ndps4\" (UniqueName: \"kubernetes.io/projected/03b10bd5-7748-43f4-a4e2-346928c502f1-kube-api-access-ndps4\") pod \"03b10bd5-7748-43f4-a4e2-346928c502f1\" (UID: \"03b10bd5-7748-43f4-a4e2-346928c502f1\") " Dec 03 12:13:57 crc kubenswrapper[5052]: I1203 12:13:57.735297 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/03b10bd5-7748-43f4-a4e2-346928c502f1-scripts" (OuterVolumeSpecName: "scripts") pod "03b10bd5-7748-43f4-a4e2-346928c502f1" (UID: "03b10bd5-7748-43f4-a4e2-346928c502f1"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:13:57 crc kubenswrapper[5052]: I1203 12:13:57.735326 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/03b10bd5-7748-43f4-a4e2-346928c502f1-kube-api-access-ndps4" (OuterVolumeSpecName: "kube-api-access-ndps4") pod "03b10bd5-7748-43f4-a4e2-346928c502f1" (UID: "03b10bd5-7748-43f4-a4e2-346928c502f1"). InnerVolumeSpecName "kube-api-access-ndps4". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:13:57 crc kubenswrapper[5052]: I1203 12:13:57.752309 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/03b10bd5-7748-43f4-a4e2-346928c502f1-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "03b10bd5-7748-43f4-a4e2-346928c502f1" (UID: "03b10bd5-7748-43f4-a4e2-346928c502f1"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:13:57 crc kubenswrapper[5052]: I1203 12:13:57.761749 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/03b10bd5-7748-43f4-a4e2-346928c502f1-config-data" (OuterVolumeSpecName: "config-data") pod "03b10bd5-7748-43f4-a4e2-346928c502f1" (UID: "03b10bd5-7748-43f4-a4e2-346928c502f1"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:13:57 crc kubenswrapper[5052]: I1203 12:13:57.827074 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/03b10bd5-7748-43f4-a4e2-346928c502f1-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:13:57 crc kubenswrapper[5052]: I1203 12:13:57.827142 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-ndps4\" (UniqueName: \"kubernetes.io/projected/03b10bd5-7748-43f4-a4e2-346928c502f1-kube-api-access-ndps4\") on node \"crc\" DevicePath \"\"" Dec 03 12:13:57 crc kubenswrapper[5052]: I1203 12:13:57.827154 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/03b10bd5-7748-43f4-a4e2-346928c502f1-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:13:57 crc kubenswrapper[5052]: I1203 12:13:57.827163 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/03b10bd5-7748-43f4-a4e2-346928c502f1-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:13:58 crc kubenswrapper[5052]: I1203 12:13:58.210063 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-cell-mapping-tb27n" event={"ID":"03b10bd5-7748-43f4-a4e2-346928c502f1","Type":"ContainerDied","Data":"6372dcfe9923fddf61186704a3c44371519577584921fc8a7994171c3519040e"} Dec 03 12:13:58 crc kubenswrapper[5052]: I1203 12:13:58.210120 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6372dcfe9923fddf61186704a3c44371519577584921fc8a7994171c3519040e" Dec 03 12:13:58 crc kubenswrapper[5052]: I1203 12:13:58.210140 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-cell-mapping-tb27n" Dec 03 12:13:58 crc kubenswrapper[5052]: I1203 12:13:58.215848 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:13:58 crc kubenswrapper[5052]: I1203 12:13:58.441666 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:13:58 crc kubenswrapper[5052]: I1203 12:13:58.442361 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-api-0" podUID="a8676e0f-8c55-4277-b100-1e84ba22abc9" containerName="nova-api-log" containerID="cri-o://627a90164ddbda989135947935a8b58484e90e221d3808645e49f99985e3aa76" gracePeriod=30 Dec 03 12:13:58 crc kubenswrapper[5052]: I1203 12:13:58.442390 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-api-0" podUID="a8676e0f-8c55-4277-b100-1e84ba22abc9" containerName="nova-api-api" containerID="cri-o://c2b50503852254698d9532f6a6a498bfc995d2771d2ebb215bddaec5c35f5522" gracePeriod=30 Dec 03 12:13:58 crc kubenswrapper[5052]: I1203 12:13:58.451177 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:13:58 crc kubenswrapper[5052]: I1203 12:13:58.451426 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-scheduler-0" podUID="ee6da6e8-a00d-4068-b5be-9ef9fdd61448" containerName="nova-scheduler-scheduler" containerID="cri-o://5e7b9ba77c14a6a49af3189f8abda89e1cd53d2c71393839dc6cdf47f4382890" gracePeriod=30 Dec 03 12:13:58 crc kubenswrapper[5052]: I1203 12:13:58.480977 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:13:59 crc kubenswrapper[5052]: I1203 12:13:59.226146 5052 generic.go:334] "Generic (PLEG): container finished" podID="a8676e0f-8c55-4277-b100-1e84ba22abc9" containerID="627a90164ddbda989135947935a8b58484e90e221d3808645e49f99985e3aa76" exitCode=143 Dec 03 12:13:59 crc kubenswrapper[5052]: I1203 12:13:59.226231 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"a8676e0f-8c55-4277-b100-1e84ba22abc9","Type":"ContainerDied","Data":"627a90164ddbda989135947935a8b58484e90e221d3808645e49f99985e3aa76"} Dec 03 12:13:59 crc kubenswrapper[5052]: I1203 12:13:59.229839 5052 generic.go:334] "Generic (PLEG): container finished" podID="6a508669-24ef-4c02-82b4-bfb8b3b9420b" containerID="17d55a4176395a4183e27fbdad33c760894ac49731d57a48e1e495ee12293d29" exitCode=0 Dec 03 12:13:59 crc kubenswrapper[5052]: I1203 12:13:59.230007 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"6a508669-24ef-4c02-82b4-bfb8b3b9420b","Type":"ContainerDied","Data":"17d55a4176395a4183e27fbdad33c760894ac49731d57a48e1e495ee12293d29"} Dec 03 12:13:59 crc kubenswrapper[5052]: I1203 12:13:59.518900 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:13:59 crc kubenswrapper[5052]: I1203 12:13:59.660257 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/6a508669-24ef-4c02-82b4-bfb8b3b9420b-scripts\") pod \"6a508669-24ef-4c02-82b4-bfb8b3b9420b\" (UID: \"6a508669-24ef-4c02-82b4-bfb8b3b9420b\") " Dec 03 12:13:59 crc kubenswrapper[5052]: I1203 12:13:59.660366 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6a508669-24ef-4c02-82b4-bfb8b3b9420b-config-data\") pod \"6a508669-24ef-4c02-82b4-bfb8b3b9420b\" (UID: \"6a508669-24ef-4c02-82b4-bfb8b3b9420b\") " Dec 03 12:13:59 crc kubenswrapper[5052]: I1203 12:13:59.660447 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6a508669-24ef-4c02-82b4-bfb8b3b9420b-combined-ca-bundle\") pod \"6a508669-24ef-4c02-82b4-bfb8b3b9420b\" (UID: \"6a508669-24ef-4c02-82b4-bfb8b3b9420b\") " Dec 03 12:13:59 crc kubenswrapper[5052]: I1203 12:13:59.660480 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/6a508669-24ef-4c02-82b4-bfb8b3b9420b-log-httpd\") pod \"6a508669-24ef-4c02-82b4-bfb8b3b9420b\" (UID: \"6a508669-24ef-4c02-82b4-bfb8b3b9420b\") " Dec 03 12:13:59 crc kubenswrapper[5052]: I1203 12:13:59.660500 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l2ksm\" (UniqueName: \"kubernetes.io/projected/6a508669-24ef-4c02-82b4-bfb8b3b9420b-kube-api-access-l2ksm\") pod \"6a508669-24ef-4c02-82b4-bfb8b3b9420b\" (UID: \"6a508669-24ef-4c02-82b4-bfb8b3b9420b\") " Dec 03 12:13:59 crc kubenswrapper[5052]: I1203 12:13:59.660685 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/6a508669-24ef-4c02-82b4-bfb8b3b9420b-sg-core-conf-yaml\") pod \"6a508669-24ef-4c02-82b4-bfb8b3b9420b\" (UID: \"6a508669-24ef-4c02-82b4-bfb8b3b9420b\") " Dec 03 12:13:59 crc kubenswrapper[5052]: I1203 12:13:59.660788 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/6a508669-24ef-4c02-82b4-bfb8b3b9420b-run-httpd\") pod \"6a508669-24ef-4c02-82b4-bfb8b3b9420b\" (UID: \"6a508669-24ef-4c02-82b4-bfb8b3b9420b\") " Dec 03 12:13:59 crc kubenswrapper[5052]: I1203 12:13:59.661360 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a508669-24ef-4c02-82b4-bfb8b3b9420b-log-httpd" (OuterVolumeSpecName: "log-httpd") pod "6a508669-24ef-4c02-82b4-bfb8b3b9420b" (UID: "6a508669-24ef-4c02-82b4-bfb8b3b9420b"). InnerVolumeSpecName "log-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:13:59 crc kubenswrapper[5052]: I1203 12:13:59.661492 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a508669-24ef-4c02-82b4-bfb8b3b9420b-run-httpd" (OuterVolumeSpecName: "run-httpd") pod "6a508669-24ef-4c02-82b4-bfb8b3b9420b" (UID: "6a508669-24ef-4c02-82b4-bfb8b3b9420b"). InnerVolumeSpecName "run-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:13:59 crc kubenswrapper[5052]: I1203 12:13:59.666059 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6a508669-24ef-4c02-82b4-bfb8b3b9420b-kube-api-access-l2ksm" (OuterVolumeSpecName: "kube-api-access-l2ksm") pod "6a508669-24ef-4c02-82b4-bfb8b3b9420b" (UID: "6a508669-24ef-4c02-82b4-bfb8b3b9420b"). InnerVolumeSpecName "kube-api-access-l2ksm". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:13:59 crc kubenswrapper[5052]: I1203 12:13:59.684929 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6a508669-24ef-4c02-82b4-bfb8b3b9420b-scripts" (OuterVolumeSpecName: "scripts") pod "6a508669-24ef-4c02-82b4-bfb8b3b9420b" (UID: "6a508669-24ef-4c02-82b4-bfb8b3b9420b"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:13:59 crc kubenswrapper[5052]: I1203 12:13:59.697170 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6a508669-24ef-4c02-82b4-bfb8b3b9420b-sg-core-conf-yaml" (OuterVolumeSpecName: "sg-core-conf-yaml") pod "6a508669-24ef-4c02-82b4-bfb8b3b9420b" (UID: "6a508669-24ef-4c02-82b4-bfb8b3b9420b"). InnerVolumeSpecName "sg-core-conf-yaml". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:13:59 crc kubenswrapper[5052]: I1203 12:13:59.749090 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6a508669-24ef-4c02-82b4-bfb8b3b9420b-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "6a508669-24ef-4c02-82b4-bfb8b3b9420b" (UID: "6a508669-24ef-4c02-82b4-bfb8b3b9420b"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:13:59 crc kubenswrapper[5052]: I1203 12:13:59.763770 5052 reconciler_common.go:293] "Volume detached for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/6a508669-24ef-4c02-82b4-bfb8b3b9420b-sg-core-conf-yaml\") on node \"crc\" DevicePath \"\"" Dec 03 12:13:59 crc kubenswrapper[5052]: I1203 12:13:59.764064 5052 reconciler_common.go:293] "Volume detached for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/6a508669-24ef-4c02-82b4-bfb8b3b9420b-run-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:13:59 crc kubenswrapper[5052]: I1203 12:13:59.764078 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/6a508669-24ef-4c02-82b4-bfb8b3b9420b-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:13:59 crc kubenswrapper[5052]: I1203 12:13:59.764089 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6a508669-24ef-4c02-82b4-bfb8b3b9420b-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:13:59 crc kubenswrapper[5052]: I1203 12:13:59.764100 5052 reconciler_common.go:293] "Volume detached for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/6a508669-24ef-4c02-82b4-bfb8b3b9420b-log-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:13:59 crc kubenswrapper[5052]: I1203 12:13:59.764112 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-l2ksm\" (UniqueName: \"kubernetes.io/projected/6a508669-24ef-4c02-82b4-bfb8b3b9420b-kube-api-access-l2ksm\") on node \"crc\" DevicePath \"\"" Dec 03 12:13:59 crc kubenswrapper[5052]: I1203 12:13:59.781370 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6a508669-24ef-4c02-82b4-bfb8b3b9420b-config-data" (OuterVolumeSpecName: "config-data") pod "6a508669-24ef-4c02-82b4-bfb8b3b9420b" (UID: "6a508669-24ef-4c02-82b4-bfb8b3b9420b"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:13:59 crc kubenswrapper[5052]: I1203 12:13:59.866444 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6a508669-24ef-4c02-82b4-bfb8b3b9420b-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:14:00 crc kubenswrapper[5052]: I1203 12:14:00.015266 5052 scope.go:117] "RemoveContainer" containerID="1b8800a111aa6f20fcc4e337231da09240be718253d0c4917217959ec3ed78f2" Dec 03 12:14:00 crc kubenswrapper[5052]: I1203 12:14:00.061964 5052 scope.go:117] "RemoveContainer" containerID="15a5f7c7cd882608a85275b1c6458b1bf51b486cc6488418a2e35990d2b843c0" Dec 03 12:14:00 crc kubenswrapper[5052]: I1203 12:14:00.101996 5052 scope.go:117] "RemoveContainer" containerID="73022661e4a723ea9ac382d6839ad1127d1eaf64b3149c3d92b5bdf83208e996" Dec 03 12:14:00 crc kubenswrapper[5052]: I1203 12:14:00.129334 5052 scope.go:117] "RemoveContainer" containerID="2c81d86ba7b2260bf28aacf6f22b1bf41b3bce50a7118749afee64eacc6c95a5" Dec 03 12:14:00 crc kubenswrapper[5052]: I1203 12:14:00.244335 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"6a508669-24ef-4c02-82b4-bfb8b3b9420b","Type":"ContainerDied","Data":"87a017bf40232dce1ff49a60df7ae13dab9f9e9bdfd562c7345697313262f85a"} Dec 03 12:14:00 crc kubenswrapper[5052]: I1203 12:14:00.244392 5052 scope.go:117] "RemoveContainer" containerID="3b54ccfe9d1b5d1b336e3adcc5013135c78f03401138c69e1aac9777fd76c5ed" Dec 03 12:14:00 crc kubenswrapper[5052]: I1203 12:14:00.244404 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:14:00 crc kubenswrapper[5052]: I1203 12:14:00.245516 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-metadata-0" podUID="3487eda5-6351-40b2-babe-458b38432b7f" containerName="nova-metadata-metadata" containerID="cri-o://7e5ed174c64d6f31e099a5002b59ab1714e3ae3774c2bbed9284ee785cff7b19" gracePeriod=30 Dec 03 12:14:00 crc kubenswrapper[5052]: I1203 12:14:00.245510 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-metadata-0" podUID="3487eda5-6351-40b2-babe-458b38432b7f" containerName="nova-metadata-log" containerID="cri-o://b616896b8d682e7215c2e892d7490ff87002eb4de1dd5abfe0d3693910b66db1" gracePeriod=30 Dec 03 12:14:00 crc kubenswrapper[5052]: I1203 12:14:00.279838 5052 scope.go:117] "RemoveContainer" containerID="12cf806b890c3285f6292098c13247c6168955b831065259ec141d41eff9f0b8" Dec 03 12:14:00 crc kubenswrapper[5052]: I1203 12:14:00.295017 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:14:00 crc kubenswrapper[5052]: I1203 12:14:00.307837 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:14:00 crc kubenswrapper[5052]: I1203 12:14:00.325821 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:14:00 crc kubenswrapper[5052]: E1203 12:14:00.326258 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="6a508669-24ef-4c02-82b4-bfb8b3b9420b" containerName="ceilometer-central-agent" Dec 03 12:14:00 crc kubenswrapper[5052]: I1203 12:14:00.326467 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="6a508669-24ef-4c02-82b4-bfb8b3b9420b" containerName="ceilometer-central-agent" Dec 03 12:14:00 crc kubenswrapper[5052]: E1203 12:14:00.327441 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="6a508669-24ef-4c02-82b4-bfb8b3b9420b" containerName="sg-core" Dec 03 12:14:00 crc kubenswrapper[5052]: I1203 12:14:00.327459 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="6a508669-24ef-4c02-82b4-bfb8b3b9420b" containerName="sg-core" Dec 03 12:14:00 crc kubenswrapper[5052]: E1203 12:14:00.327474 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="6a508669-24ef-4c02-82b4-bfb8b3b9420b" containerName="proxy-httpd" Dec 03 12:14:00 crc kubenswrapper[5052]: I1203 12:14:00.327483 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="6a508669-24ef-4c02-82b4-bfb8b3b9420b" containerName="proxy-httpd" Dec 03 12:14:00 crc kubenswrapper[5052]: E1203 12:14:00.327530 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="6a508669-24ef-4c02-82b4-bfb8b3b9420b" containerName="ceilometer-notification-agent" Dec 03 12:14:00 crc kubenswrapper[5052]: I1203 12:14:00.327540 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="6a508669-24ef-4c02-82b4-bfb8b3b9420b" containerName="ceilometer-notification-agent" Dec 03 12:14:00 crc kubenswrapper[5052]: E1203 12:14:00.327563 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="03b10bd5-7748-43f4-a4e2-346928c502f1" containerName="nova-manage" Dec 03 12:14:00 crc kubenswrapper[5052]: I1203 12:14:00.327570 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="03b10bd5-7748-43f4-a4e2-346928c502f1" containerName="nova-manage" Dec 03 12:14:00 crc kubenswrapper[5052]: I1203 12:14:00.327845 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="6a508669-24ef-4c02-82b4-bfb8b3b9420b" containerName="sg-core" Dec 03 12:14:00 crc kubenswrapper[5052]: I1203 12:14:00.327868 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="03b10bd5-7748-43f4-a4e2-346928c502f1" containerName="nova-manage" Dec 03 12:14:00 crc kubenswrapper[5052]: I1203 12:14:00.327900 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="6a508669-24ef-4c02-82b4-bfb8b3b9420b" containerName="proxy-httpd" Dec 03 12:14:00 crc kubenswrapper[5052]: I1203 12:14:00.327913 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="6a508669-24ef-4c02-82b4-bfb8b3b9420b" containerName="ceilometer-central-agent" Dec 03 12:14:00 crc kubenswrapper[5052]: I1203 12:14:00.327923 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="6a508669-24ef-4c02-82b4-bfb8b3b9420b" containerName="ceilometer-notification-agent" Dec 03 12:14:00 crc kubenswrapper[5052]: I1203 12:14:00.330253 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:14:00 crc kubenswrapper[5052]: I1203 12:14:00.331849 5052 scope.go:117] "RemoveContainer" containerID="17d55a4176395a4183e27fbdad33c760894ac49731d57a48e1e495ee12293d29" Dec 03 12:14:00 crc kubenswrapper[5052]: I1203 12:14:00.338171 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-scripts" Dec 03 12:14:00 crc kubenswrapper[5052]: I1203 12:14:00.338501 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-config-data" Dec 03 12:14:00 crc kubenswrapper[5052]: I1203 12:14:00.338689 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-ceilometer-internal-svc" Dec 03 12:14:00 crc kubenswrapper[5052]: I1203 12:14:00.339588 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:14:00 crc kubenswrapper[5052]: I1203 12:14:00.376442 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/da9be43d-a787-4511-9b0c-4b3f307cb891-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"da9be43d-a787-4511-9b0c-4b3f307cb891\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:14:00 crc kubenswrapper[5052]: I1203 12:14:00.376513 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/da9be43d-a787-4511-9b0c-4b3f307cb891-log-httpd\") pod \"ceilometer-0\" (UID: \"da9be43d-a787-4511-9b0c-4b3f307cb891\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:14:00 crc kubenswrapper[5052]: I1203 12:14:00.376773 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/da9be43d-a787-4511-9b0c-4b3f307cb891-config-data\") pod \"ceilometer-0\" (UID: \"da9be43d-a787-4511-9b0c-4b3f307cb891\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:14:00 crc kubenswrapper[5052]: I1203 12:14:00.376910 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/da9be43d-a787-4511-9b0c-4b3f307cb891-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"da9be43d-a787-4511-9b0c-4b3f307cb891\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:14:00 crc kubenswrapper[5052]: I1203 12:14:00.376937 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/da9be43d-a787-4511-9b0c-4b3f307cb891-ceilometer-tls-certs\") pod \"ceilometer-0\" (UID: \"da9be43d-a787-4511-9b0c-4b3f307cb891\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:14:00 crc kubenswrapper[5052]: I1203 12:14:00.377034 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9d96n\" (UniqueName: \"kubernetes.io/projected/da9be43d-a787-4511-9b0c-4b3f307cb891-kube-api-access-9d96n\") pod \"ceilometer-0\" (UID: \"da9be43d-a787-4511-9b0c-4b3f307cb891\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:14:00 crc kubenswrapper[5052]: I1203 12:14:00.377119 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/da9be43d-a787-4511-9b0c-4b3f307cb891-run-httpd\") pod \"ceilometer-0\" (UID: \"da9be43d-a787-4511-9b0c-4b3f307cb891\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:14:00 crc kubenswrapper[5052]: I1203 12:14:00.377204 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/da9be43d-a787-4511-9b0c-4b3f307cb891-scripts\") pod \"ceilometer-0\" (UID: \"da9be43d-a787-4511-9b0c-4b3f307cb891\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:14:00 crc kubenswrapper[5052]: I1203 12:14:00.417057 5052 scope.go:117] "RemoveContainer" containerID="f0ec59ff17f5ac98f8efe3ba2d6110db4488e144489ba6f93945be948047d889" Dec 03 12:14:00 crc kubenswrapper[5052]: I1203 12:14:00.479687 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/da9be43d-a787-4511-9b0c-4b3f307cb891-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"da9be43d-a787-4511-9b0c-4b3f307cb891\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:14:00 crc kubenswrapper[5052]: I1203 12:14:00.479747 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/da9be43d-a787-4511-9b0c-4b3f307cb891-log-httpd\") pod \"ceilometer-0\" (UID: \"da9be43d-a787-4511-9b0c-4b3f307cb891\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:14:00 crc kubenswrapper[5052]: I1203 12:14:00.479798 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/da9be43d-a787-4511-9b0c-4b3f307cb891-config-data\") pod \"ceilometer-0\" (UID: \"da9be43d-a787-4511-9b0c-4b3f307cb891\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:14:00 crc kubenswrapper[5052]: I1203 12:14:00.479856 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/da9be43d-a787-4511-9b0c-4b3f307cb891-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"da9be43d-a787-4511-9b0c-4b3f307cb891\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:14:00 crc kubenswrapper[5052]: I1203 12:14:00.479880 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/da9be43d-a787-4511-9b0c-4b3f307cb891-ceilometer-tls-certs\") pod \"ceilometer-0\" (UID: \"da9be43d-a787-4511-9b0c-4b3f307cb891\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:14:00 crc kubenswrapper[5052]: I1203 12:14:00.479935 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-9d96n\" (UniqueName: \"kubernetes.io/projected/da9be43d-a787-4511-9b0c-4b3f307cb891-kube-api-access-9d96n\") pod \"ceilometer-0\" (UID: \"da9be43d-a787-4511-9b0c-4b3f307cb891\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:14:00 crc kubenswrapper[5052]: I1203 12:14:00.479984 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/da9be43d-a787-4511-9b0c-4b3f307cb891-run-httpd\") pod \"ceilometer-0\" (UID: \"da9be43d-a787-4511-9b0c-4b3f307cb891\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:14:00 crc kubenswrapper[5052]: I1203 12:14:00.480027 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/da9be43d-a787-4511-9b0c-4b3f307cb891-scripts\") pod \"ceilometer-0\" (UID: \"da9be43d-a787-4511-9b0c-4b3f307cb891\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:14:00 crc kubenswrapper[5052]: I1203 12:14:00.480505 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/da9be43d-a787-4511-9b0c-4b3f307cb891-log-httpd\") pod \"ceilometer-0\" (UID: \"da9be43d-a787-4511-9b0c-4b3f307cb891\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:14:00 crc kubenswrapper[5052]: I1203 12:14:00.483833 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/da9be43d-a787-4511-9b0c-4b3f307cb891-run-httpd\") pod \"ceilometer-0\" (UID: \"da9be43d-a787-4511-9b0c-4b3f307cb891\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:14:00 crc kubenswrapper[5052]: I1203 12:14:00.484784 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/da9be43d-a787-4511-9b0c-4b3f307cb891-config-data\") pod \"ceilometer-0\" (UID: \"da9be43d-a787-4511-9b0c-4b3f307cb891\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:14:00 crc kubenswrapper[5052]: I1203 12:14:00.485238 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/da9be43d-a787-4511-9b0c-4b3f307cb891-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"da9be43d-a787-4511-9b0c-4b3f307cb891\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:14:00 crc kubenswrapper[5052]: I1203 12:14:00.485310 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/da9be43d-a787-4511-9b0c-4b3f307cb891-ceilometer-tls-certs\") pod \"ceilometer-0\" (UID: \"da9be43d-a787-4511-9b0c-4b3f307cb891\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:14:00 crc kubenswrapper[5052]: I1203 12:14:00.487074 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/da9be43d-a787-4511-9b0c-4b3f307cb891-scripts\") pod \"ceilometer-0\" (UID: \"da9be43d-a787-4511-9b0c-4b3f307cb891\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:14:00 crc kubenswrapper[5052]: I1203 12:14:00.490298 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/da9be43d-a787-4511-9b0c-4b3f307cb891-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"da9be43d-a787-4511-9b0c-4b3f307cb891\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:14:00 crc kubenswrapper[5052]: I1203 12:14:00.507246 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-9d96n\" (UniqueName: \"kubernetes.io/projected/da9be43d-a787-4511-9b0c-4b3f307cb891-kube-api-access-9d96n\") pod \"ceilometer-0\" (UID: \"da9be43d-a787-4511-9b0c-4b3f307cb891\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:14:00 crc kubenswrapper[5052]: I1203 12:14:00.661372 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:14:01 crc kubenswrapper[5052]: I1203 12:14:01.156480 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:14:01 crc kubenswrapper[5052]: W1203 12:14:01.179725 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podda9be43d_a787_4511_9b0c_4b3f307cb891.slice/crio-391bda9a44718db5214e627a0236bad3e9b28894de64534a944c5ffb3e3dcb9a WatchSource:0}: Error finding container 391bda9a44718db5214e627a0236bad3e9b28894de64534a944c5ffb3e3dcb9a: Status 404 returned error can't find the container with id 391bda9a44718db5214e627a0236bad3e9b28894de64534a944c5ffb3e3dcb9a Dec 03 12:14:01 crc kubenswrapper[5052]: I1203 12:14:01.270290 5052 generic.go:334] "Generic (PLEG): container finished" podID="3487eda5-6351-40b2-babe-458b38432b7f" containerID="b616896b8d682e7215c2e892d7490ff87002eb4de1dd5abfe0d3693910b66db1" exitCode=143 Dec 03 12:14:01 crc kubenswrapper[5052]: I1203 12:14:01.270394 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"3487eda5-6351-40b2-babe-458b38432b7f","Type":"ContainerDied","Data":"b616896b8d682e7215c2e892d7490ff87002eb4de1dd5abfe0d3693910b66db1"} Dec 03 12:14:01 crc kubenswrapper[5052]: I1203 12:14:01.271566 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"da9be43d-a787-4511-9b0c-4b3f307cb891","Type":"ContainerStarted","Data":"391bda9a44718db5214e627a0236bad3e9b28894de64534a944c5ffb3e3dcb9a"} Dec 03 12:14:01 crc kubenswrapper[5052]: I1203 12:14:01.272783 5052 generic.go:334] "Generic (PLEG): container finished" podID="ee6da6e8-a00d-4068-b5be-9ef9fdd61448" containerID="5e7b9ba77c14a6a49af3189f8abda89e1cd53d2c71393839dc6cdf47f4382890" exitCode=0 Dec 03 12:14:01 crc kubenswrapper[5052]: I1203 12:14:01.272814 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"ee6da6e8-a00d-4068-b5be-9ef9fdd61448","Type":"ContainerDied","Data":"5e7b9ba77c14a6a49af3189f8abda89e1cd53d2c71393839dc6cdf47f4382890"} Dec 03 12:14:01 crc kubenswrapper[5052]: I1203 12:14:01.664869 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:14:01 crc kubenswrapper[5052]: I1203 12:14:01.701028 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ee6da6e8-a00d-4068-b5be-9ef9fdd61448-combined-ca-bundle\") pod \"ee6da6e8-a00d-4068-b5be-9ef9fdd61448\" (UID: \"ee6da6e8-a00d-4068-b5be-9ef9fdd61448\") " Dec 03 12:14:01 crc kubenswrapper[5052]: I1203 12:14:01.701089 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zvf48\" (UniqueName: \"kubernetes.io/projected/ee6da6e8-a00d-4068-b5be-9ef9fdd61448-kube-api-access-zvf48\") pod \"ee6da6e8-a00d-4068-b5be-9ef9fdd61448\" (UID: \"ee6da6e8-a00d-4068-b5be-9ef9fdd61448\") " Dec 03 12:14:01 crc kubenswrapper[5052]: I1203 12:14:01.701138 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/ee6da6e8-a00d-4068-b5be-9ef9fdd61448-config-data\") pod \"ee6da6e8-a00d-4068-b5be-9ef9fdd61448\" (UID: \"ee6da6e8-a00d-4068-b5be-9ef9fdd61448\") " Dec 03 12:14:01 crc kubenswrapper[5052]: I1203 12:14:01.706210 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ee6da6e8-a00d-4068-b5be-9ef9fdd61448-kube-api-access-zvf48" (OuterVolumeSpecName: "kube-api-access-zvf48") pod "ee6da6e8-a00d-4068-b5be-9ef9fdd61448" (UID: "ee6da6e8-a00d-4068-b5be-9ef9fdd61448"). InnerVolumeSpecName "kube-api-access-zvf48". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:14:01 crc kubenswrapper[5052]: I1203 12:14:01.706432 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6a508669-24ef-4c02-82b4-bfb8b3b9420b" path="/var/lib/kubelet/pods/6a508669-24ef-4c02-82b4-bfb8b3b9420b/volumes" Dec 03 12:14:01 crc kubenswrapper[5052]: I1203 12:14:01.745686 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ee6da6e8-a00d-4068-b5be-9ef9fdd61448-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "ee6da6e8-a00d-4068-b5be-9ef9fdd61448" (UID: "ee6da6e8-a00d-4068-b5be-9ef9fdd61448"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:14:01 crc kubenswrapper[5052]: I1203 12:14:01.755595 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ee6da6e8-a00d-4068-b5be-9ef9fdd61448-config-data" (OuterVolumeSpecName: "config-data") pod "ee6da6e8-a00d-4068-b5be-9ef9fdd61448" (UID: "ee6da6e8-a00d-4068-b5be-9ef9fdd61448"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:14:01 crc kubenswrapper[5052]: I1203 12:14:01.805278 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ee6da6e8-a00d-4068-b5be-9ef9fdd61448-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:14:01 crc kubenswrapper[5052]: I1203 12:14:01.805434 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-zvf48\" (UniqueName: \"kubernetes.io/projected/ee6da6e8-a00d-4068-b5be-9ef9fdd61448-kube-api-access-zvf48\") on node \"crc\" DevicePath \"\"" Dec 03 12:14:01 crc kubenswrapper[5052]: I1203 12:14:01.805461 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/ee6da6e8-a00d-4068-b5be-9ef9fdd61448-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:14:02 crc kubenswrapper[5052]: I1203 12:14:02.098931 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:14:02 crc kubenswrapper[5052]: I1203 12:14:02.220807 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/a8676e0f-8c55-4277-b100-1e84ba22abc9-logs\") pod \"a8676e0f-8c55-4277-b100-1e84ba22abc9\" (UID: \"a8676e0f-8c55-4277-b100-1e84ba22abc9\") " Dec 03 12:14:02 crc kubenswrapper[5052]: I1203 12:14:02.220924 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mf8l7\" (UniqueName: \"kubernetes.io/projected/a8676e0f-8c55-4277-b100-1e84ba22abc9-kube-api-access-mf8l7\") pod \"a8676e0f-8c55-4277-b100-1e84ba22abc9\" (UID: \"a8676e0f-8c55-4277-b100-1e84ba22abc9\") " Dec 03 12:14:02 crc kubenswrapper[5052]: I1203 12:14:02.220984 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a8676e0f-8c55-4277-b100-1e84ba22abc9-combined-ca-bundle\") pod \"a8676e0f-8c55-4277-b100-1e84ba22abc9\" (UID: \"a8676e0f-8c55-4277-b100-1e84ba22abc9\") " Dec 03 12:14:02 crc kubenswrapper[5052]: I1203 12:14:02.221114 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a8676e0f-8c55-4277-b100-1e84ba22abc9-config-data\") pod \"a8676e0f-8c55-4277-b100-1e84ba22abc9\" (UID: \"a8676e0f-8c55-4277-b100-1e84ba22abc9\") " Dec 03 12:14:02 crc kubenswrapper[5052]: I1203 12:14:02.221420 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a8676e0f-8c55-4277-b100-1e84ba22abc9-logs" (OuterVolumeSpecName: "logs") pod "a8676e0f-8c55-4277-b100-1e84ba22abc9" (UID: "a8676e0f-8c55-4277-b100-1e84ba22abc9"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:14:02 crc kubenswrapper[5052]: I1203 12:14:02.221701 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/a8676e0f-8c55-4277-b100-1e84ba22abc9-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:14:02 crc kubenswrapper[5052]: I1203 12:14:02.225991 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a8676e0f-8c55-4277-b100-1e84ba22abc9-kube-api-access-mf8l7" (OuterVolumeSpecName: "kube-api-access-mf8l7") pod "a8676e0f-8c55-4277-b100-1e84ba22abc9" (UID: "a8676e0f-8c55-4277-b100-1e84ba22abc9"). InnerVolumeSpecName "kube-api-access-mf8l7". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:14:02 crc kubenswrapper[5052]: I1203 12:14:02.249817 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a8676e0f-8c55-4277-b100-1e84ba22abc9-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "a8676e0f-8c55-4277-b100-1e84ba22abc9" (UID: "a8676e0f-8c55-4277-b100-1e84ba22abc9"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:14:02 crc kubenswrapper[5052]: I1203 12:14:02.266015 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a8676e0f-8c55-4277-b100-1e84ba22abc9-config-data" (OuterVolumeSpecName: "config-data") pod "a8676e0f-8c55-4277-b100-1e84ba22abc9" (UID: "a8676e0f-8c55-4277-b100-1e84ba22abc9"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:14:02 crc kubenswrapper[5052]: I1203 12:14:02.289440 5052 generic.go:334] "Generic (PLEG): container finished" podID="a8676e0f-8c55-4277-b100-1e84ba22abc9" containerID="c2b50503852254698d9532f6a6a498bfc995d2771d2ebb215bddaec5c35f5522" exitCode=0 Dec 03 12:14:02 crc kubenswrapper[5052]: I1203 12:14:02.289531 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"a8676e0f-8c55-4277-b100-1e84ba22abc9","Type":"ContainerDied","Data":"c2b50503852254698d9532f6a6a498bfc995d2771d2ebb215bddaec5c35f5522"} Dec 03 12:14:02 crc kubenswrapper[5052]: I1203 12:14:02.289567 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"a8676e0f-8c55-4277-b100-1e84ba22abc9","Type":"ContainerDied","Data":"9390647b5d0ffab7aad012c4ed70464f64be8b01539a8265e3e8ce8e6869b571"} Dec 03 12:14:02 crc kubenswrapper[5052]: I1203 12:14:02.289585 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:14:02 crc kubenswrapper[5052]: I1203 12:14:02.289590 5052 scope.go:117] "RemoveContainer" containerID="c2b50503852254698d9532f6a6a498bfc995d2771d2ebb215bddaec5c35f5522" Dec 03 12:14:02 crc kubenswrapper[5052]: I1203 12:14:02.295904 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"ee6da6e8-a00d-4068-b5be-9ef9fdd61448","Type":"ContainerDied","Data":"e770ea55ceec207d15c01f4c783750c684593183790ed51dc4fb89c3e6617dd2"} Dec 03 12:14:02 crc kubenswrapper[5052]: I1203 12:14:02.296029 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:14:02 crc kubenswrapper[5052]: I1203 12:14:02.298842 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"da9be43d-a787-4511-9b0c-4b3f307cb891","Type":"ContainerStarted","Data":"d92cbc5bb31cf05026bada3e2ff528d9bffb62aee81e5876a53cd8465313f7d5"} Dec 03 12:14:02 crc kubenswrapper[5052]: I1203 12:14:02.323848 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-mf8l7\" (UniqueName: \"kubernetes.io/projected/a8676e0f-8c55-4277-b100-1e84ba22abc9-kube-api-access-mf8l7\") on node \"crc\" DevicePath \"\"" Dec 03 12:14:02 crc kubenswrapper[5052]: I1203 12:14:02.323891 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a8676e0f-8c55-4277-b100-1e84ba22abc9-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:14:02 crc kubenswrapper[5052]: I1203 12:14:02.323904 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a8676e0f-8c55-4277-b100-1e84ba22abc9-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:14:02 crc kubenswrapper[5052]: I1203 12:14:02.325835 5052 scope.go:117] "RemoveContainer" containerID="627a90164ddbda989135947935a8b58484e90e221d3808645e49f99985e3aa76" Dec 03 12:14:02 crc kubenswrapper[5052]: I1203 12:14:02.352648 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:14:02 crc kubenswrapper[5052]: I1203 12:14:02.363785 5052 scope.go:117] "RemoveContainer" containerID="c2b50503852254698d9532f6a6a498bfc995d2771d2ebb215bddaec5c35f5522" Dec 03 12:14:02 crc kubenswrapper[5052]: E1203 12:14:02.364427 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c2b50503852254698d9532f6a6a498bfc995d2771d2ebb215bddaec5c35f5522\": container with ID starting with c2b50503852254698d9532f6a6a498bfc995d2771d2ebb215bddaec5c35f5522 not found: ID does not exist" containerID="c2b50503852254698d9532f6a6a498bfc995d2771d2ebb215bddaec5c35f5522" Dec 03 12:14:02 crc kubenswrapper[5052]: I1203 12:14:02.364481 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c2b50503852254698d9532f6a6a498bfc995d2771d2ebb215bddaec5c35f5522"} err="failed to get container status \"c2b50503852254698d9532f6a6a498bfc995d2771d2ebb215bddaec5c35f5522\": rpc error: code = NotFound desc = could not find container \"c2b50503852254698d9532f6a6a498bfc995d2771d2ebb215bddaec5c35f5522\": container with ID starting with c2b50503852254698d9532f6a6a498bfc995d2771d2ebb215bddaec5c35f5522 not found: ID does not exist" Dec 03 12:14:02 crc kubenswrapper[5052]: I1203 12:14:02.364532 5052 scope.go:117] "RemoveContainer" containerID="627a90164ddbda989135947935a8b58484e90e221d3808645e49f99985e3aa76" Dec 03 12:14:02 crc kubenswrapper[5052]: E1203 12:14:02.364979 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"627a90164ddbda989135947935a8b58484e90e221d3808645e49f99985e3aa76\": container with ID starting with 627a90164ddbda989135947935a8b58484e90e221d3808645e49f99985e3aa76 not found: ID does not exist" containerID="627a90164ddbda989135947935a8b58484e90e221d3808645e49f99985e3aa76" Dec 03 12:14:02 crc kubenswrapper[5052]: I1203 12:14:02.365010 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"627a90164ddbda989135947935a8b58484e90e221d3808645e49f99985e3aa76"} err="failed to get container status \"627a90164ddbda989135947935a8b58484e90e221d3808645e49f99985e3aa76\": rpc error: code = NotFound desc = could not find container \"627a90164ddbda989135947935a8b58484e90e221d3808645e49f99985e3aa76\": container with ID starting with 627a90164ddbda989135947935a8b58484e90e221d3808645e49f99985e3aa76 not found: ID does not exist" Dec 03 12:14:02 crc kubenswrapper[5052]: I1203 12:14:02.365030 5052 scope.go:117] "RemoveContainer" containerID="5e7b9ba77c14a6a49af3189f8abda89e1cd53d2c71393839dc6cdf47f4382890" Dec 03 12:14:02 crc kubenswrapper[5052]: I1203 12:14:02.375097 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:14:02 crc kubenswrapper[5052]: I1203 12:14:02.395811 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:14:02 crc kubenswrapper[5052]: I1203 12:14:02.418649 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:14:02 crc kubenswrapper[5052]: I1203 12:14:02.427847 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:14:02 crc kubenswrapper[5052]: E1203 12:14:02.428713 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="ee6da6e8-a00d-4068-b5be-9ef9fdd61448" containerName="nova-scheduler-scheduler" Dec 03 12:14:02 crc kubenswrapper[5052]: I1203 12:14:02.428818 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee6da6e8-a00d-4068-b5be-9ef9fdd61448" containerName="nova-scheduler-scheduler" Dec 03 12:14:02 crc kubenswrapper[5052]: E1203 12:14:02.428928 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="a8676e0f-8c55-4277-b100-1e84ba22abc9" containerName="nova-api-api" Dec 03 12:14:02 crc kubenswrapper[5052]: I1203 12:14:02.429041 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="a8676e0f-8c55-4277-b100-1e84ba22abc9" containerName="nova-api-api" Dec 03 12:14:02 crc kubenswrapper[5052]: E1203 12:14:02.429119 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="a8676e0f-8c55-4277-b100-1e84ba22abc9" containerName="nova-api-log" Dec 03 12:14:02 crc kubenswrapper[5052]: I1203 12:14:02.429186 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="a8676e0f-8c55-4277-b100-1e84ba22abc9" containerName="nova-api-log" Dec 03 12:14:02 crc kubenswrapper[5052]: I1203 12:14:02.429482 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="a8676e0f-8c55-4277-b100-1e84ba22abc9" containerName="nova-api-api" Dec 03 12:14:02 crc kubenswrapper[5052]: I1203 12:14:02.429848 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="a8676e0f-8c55-4277-b100-1e84ba22abc9" containerName="nova-api-log" Dec 03 12:14:02 crc kubenswrapper[5052]: I1203 12:14:02.429934 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="ee6da6e8-a00d-4068-b5be-9ef9fdd61448" containerName="nova-scheduler-scheduler" Dec 03 12:14:02 crc kubenswrapper[5052]: I1203 12:14:02.431552 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:14:02 crc kubenswrapper[5052]: I1203 12:14:02.434075 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-api-config-data" Dec 03 12:14:02 crc kubenswrapper[5052]: I1203 12:14:02.438832 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:14:02 crc kubenswrapper[5052]: I1203 12:14:02.450585 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:14:02 crc kubenswrapper[5052]: I1203 12:14:02.452704 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:14:02 crc kubenswrapper[5052]: I1203 12:14:02.456680 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-scheduler-config-data" Dec 03 12:14:02 crc kubenswrapper[5052]: I1203 12:14:02.484315 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:14:02 crc kubenswrapper[5052]: I1203 12:14:02.527906 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/ec4beccd-125d-45c0-979e-6870c39627ee-config-data\") pod \"nova-api-0\" (UID: \"ec4beccd-125d-45c0-979e-6870c39627ee\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:14:02 crc kubenswrapper[5052]: I1203 12:14:02.527990 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ec4beccd-125d-45c0-979e-6870c39627ee-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"ec4beccd-125d-45c0-979e-6870c39627ee\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:14:02 crc kubenswrapper[5052]: I1203 12:14:02.528089 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9m45m\" (UniqueName: \"kubernetes.io/projected/cf3ed088-9069-40cf-9c9e-ee9bdf7761d9-kube-api-access-9m45m\") pod \"nova-scheduler-0\" (UID: \"cf3ed088-9069-40cf-9c9e-ee9bdf7761d9\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:14:02 crc kubenswrapper[5052]: I1203 12:14:02.528132 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/ec4beccd-125d-45c0-979e-6870c39627ee-logs\") pod \"nova-api-0\" (UID: \"ec4beccd-125d-45c0-979e-6870c39627ee\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:14:02 crc kubenswrapper[5052]: I1203 12:14:02.528162 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r57vx\" (UniqueName: \"kubernetes.io/projected/ec4beccd-125d-45c0-979e-6870c39627ee-kube-api-access-r57vx\") pod \"nova-api-0\" (UID: \"ec4beccd-125d-45c0-979e-6870c39627ee\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:14:02 crc kubenswrapper[5052]: I1203 12:14:02.528189 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/cf3ed088-9069-40cf-9c9e-ee9bdf7761d9-combined-ca-bundle\") pod \"nova-scheduler-0\" (UID: \"cf3ed088-9069-40cf-9c9e-ee9bdf7761d9\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:14:02 crc kubenswrapper[5052]: I1203 12:14:02.528214 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/cf3ed088-9069-40cf-9c9e-ee9bdf7761d9-config-data\") pod \"nova-scheduler-0\" (UID: \"cf3ed088-9069-40cf-9c9e-ee9bdf7761d9\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:14:02 crc kubenswrapper[5052]: I1203 12:14:02.630174 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/ec4beccd-125d-45c0-979e-6870c39627ee-config-data\") pod \"nova-api-0\" (UID: \"ec4beccd-125d-45c0-979e-6870c39627ee\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:14:02 crc kubenswrapper[5052]: I1203 12:14:02.630244 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ec4beccd-125d-45c0-979e-6870c39627ee-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"ec4beccd-125d-45c0-979e-6870c39627ee\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:14:02 crc kubenswrapper[5052]: I1203 12:14:02.630328 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-9m45m\" (UniqueName: \"kubernetes.io/projected/cf3ed088-9069-40cf-9c9e-ee9bdf7761d9-kube-api-access-9m45m\") pod \"nova-scheduler-0\" (UID: \"cf3ed088-9069-40cf-9c9e-ee9bdf7761d9\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:14:02 crc kubenswrapper[5052]: I1203 12:14:02.630357 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/ec4beccd-125d-45c0-979e-6870c39627ee-logs\") pod \"nova-api-0\" (UID: \"ec4beccd-125d-45c0-979e-6870c39627ee\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:14:02 crc kubenswrapper[5052]: I1203 12:14:02.630379 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-r57vx\" (UniqueName: \"kubernetes.io/projected/ec4beccd-125d-45c0-979e-6870c39627ee-kube-api-access-r57vx\") pod \"nova-api-0\" (UID: \"ec4beccd-125d-45c0-979e-6870c39627ee\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:14:02 crc kubenswrapper[5052]: I1203 12:14:02.630398 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/cf3ed088-9069-40cf-9c9e-ee9bdf7761d9-combined-ca-bundle\") pod \"nova-scheduler-0\" (UID: \"cf3ed088-9069-40cf-9c9e-ee9bdf7761d9\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:14:02 crc kubenswrapper[5052]: I1203 12:14:02.630415 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/cf3ed088-9069-40cf-9c9e-ee9bdf7761d9-config-data\") pod \"nova-scheduler-0\" (UID: \"cf3ed088-9069-40cf-9c9e-ee9bdf7761d9\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:14:02 crc kubenswrapper[5052]: I1203 12:14:02.631287 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/ec4beccd-125d-45c0-979e-6870c39627ee-logs\") pod \"nova-api-0\" (UID: \"ec4beccd-125d-45c0-979e-6870c39627ee\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:14:02 crc kubenswrapper[5052]: I1203 12:14:02.634976 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/ec4beccd-125d-45c0-979e-6870c39627ee-config-data\") pod \"nova-api-0\" (UID: \"ec4beccd-125d-45c0-979e-6870c39627ee\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:14:02 crc kubenswrapper[5052]: I1203 12:14:02.635791 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ec4beccd-125d-45c0-979e-6870c39627ee-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"ec4beccd-125d-45c0-979e-6870c39627ee\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:14:02 crc kubenswrapper[5052]: I1203 12:14:02.636987 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/cf3ed088-9069-40cf-9c9e-ee9bdf7761d9-config-data\") pod \"nova-scheduler-0\" (UID: \"cf3ed088-9069-40cf-9c9e-ee9bdf7761d9\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:14:02 crc kubenswrapper[5052]: I1203 12:14:02.643460 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/cf3ed088-9069-40cf-9c9e-ee9bdf7761d9-combined-ca-bundle\") pod \"nova-scheduler-0\" (UID: \"cf3ed088-9069-40cf-9c9e-ee9bdf7761d9\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:14:02 crc kubenswrapper[5052]: I1203 12:14:02.649592 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-r57vx\" (UniqueName: \"kubernetes.io/projected/ec4beccd-125d-45c0-979e-6870c39627ee-kube-api-access-r57vx\") pod \"nova-api-0\" (UID: \"ec4beccd-125d-45c0-979e-6870c39627ee\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:14:02 crc kubenswrapper[5052]: I1203 12:14:02.652108 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-9m45m\" (UniqueName: \"kubernetes.io/projected/cf3ed088-9069-40cf-9c9e-ee9bdf7761d9-kube-api-access-9m45m\") pod \"nova-scheduler-0\" (UID: \"cf3ed088-9069-40cf-9c9e-ee9bdf7761d9\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:14:02 crc kubenswrapper[5052]: I1203 12:14:02.755850 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:14:02 crc kubenswrapper[5052]: I1203 12:14:02.796515 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:14:03 crc kubenswrapper[5052]: I1203 12:14:03.236064 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:14:03 crc kubenswrapper[5052]: I1203 12:14:03.315397 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"da9be43d-a787-4511-9b0c-4b3f307cb891","Type":"ContainerStarted","Data":"8c8b0a51f39bd122aed7b8fee5cba3c506b692f8e1b218daf6800788487afea8"} Dec 03 12:14:03 crc kubenswrapper[5052]: I1203 12:14:03.315451 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"da9be43d-a787-4511-9b0c-4b3f307cb891","Type":"ContainerStarted","Data":"21ba3a277b2c2e8c629ffb7b800e7a6375bea92da2c176f4c34f1d3393d5da4f"} Dec 03 12:14:03 crc kubenswrapper[5052]: I1203 12:14:03.320418 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:14:03 crc kubenswrapper[5052]: I1203 12:14:03.329683 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"ec4beccd-125d-45c0-979e-6870c39627ee","Type":"ContainerStarted","Data":"cd6dc2d6c74dda47c9b04f8f0eb17d144f42ce6599ac53c1e3704640e0cd7e19"} Dec 03 12:14:03 crc kubenswrapper[5052]: I1203 12:14:03.391203 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/nova-metadata-0" podUID="3487eda5-6351-40b2-babe-458b38432b7f" containerName="nova-metadata-metadata" probeResult="failure" output="Get \"https://10.217.1.34:8775/\": read tcp 10.217.0.2:52798->10.217.1.34:8775: read: connection reset by peer" Dec 03 12:14:03 crc kubenswrapper[5052]: I1203 12:14:03.391652 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/nova-metadata-0" podUID="3487eda5-6351-40b2-babe-458b38432b7f" containerName="nova-metadata-log" probeResult="failure" output="Get \"https://10.217.1.34:8775/\": read tcp 10.217.0.2:52794->10.217.1.34:8775: read: connection reset by peer" Dec 03 12:14:03 crc kubenswrapper[5052]: I1203 12:14:03.728257 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a8676e0f-8c55-4277-b100-1e84ba22abc9" path="/var/lib/kubelet/pods/a8676e0f-8c55-4277-b100-1e84ba22abc9/volumes" Dec 03 12:14:03 crc kubenswrapper[5052]: I1203 12:14:03.731292 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ee6da6e8-a00d-4068-b5be-9ef9fdd61448" path="/var/lib/kubelet/pods/ee6da6e8-a00d-4068-b5be-9ef9fdd61448/volumes" Dec 03 12:14:03 crc kubenswrapper[5052]: I1203 12:14:03.851618 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:14:03 crc kubenswrapper[5052]: I1203 12:14:03.965780 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/3487eda5-6351-40b2-babe-458b38432b7f-nova-metadata-tls-certs\") pod \"3487eda5-6351-40b2-babe-458b38432b7f\" (UID: \"3487eda5-6351-40b2-babe-458b38432b7f\") " Dec 03 12:14:03 crc kubenswrapper[5052]: I1203 12:14:03.965840 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-76gsx\" (UniqueName: \"kubernetes.io/projected/3487eda5-6351-40b2-babe-458b38432b7f-kube-api-access-76gsx\") pod \"3487eda5-6351-40b2-babe-458b38432b7f\" (UID: \"3487eda5-6351-40b2-babe-458b38432b7f\") " Dec 03 12:14:03 crc kubenswrapper[5052]: I1203 12:14:03.966000 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/3487eda5-6351-40b2-babe-458b38432b7f-logs\") pod \"3487eda5-6351-40b2-babe-458b38432b7f\" (UID: \"3487eda5-6351-40b2-babe-458b38432b7f\") " Dec 03 12:14:03 crc kubenswrapper[5052]: I1203 12:14:03.966117 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/3487eda5-6351-40b2-babe-458b38432b7f-config-data\") pod \"3487eda5-6351-40b2-babe-458b38432b7f\" (UID: \"3487eda5-6351-40b2-babe-458b38432b7f\") " Dec 03 12:14:03 crc kubenswrapper[5052]: I1203 12:14:03.966163 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3487eda5-6351-40b2-babe-458b38432b7f-combined-ca-bundle\") pod \"3487eda5-6351-40b2-babe-458b38432b7f\" (UID: \"3487eda5-6351-40b2-babe-458b38432b7f\") " Dec 03 12:14:03 crc kubenswrapper[5052]: I1203 12:14:03.967659 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3487eda5-6351-40b2-babe-458b38432b7f-logs" (OuterVolumeSpecName: "logs") pod "3487eda5-6351-40b2-babe-458b38432b7f" (UID: "3487eda5-6351-40b2-babe-458b38432b7f"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:14:03 crc kubenswrapper[5052]: I1203 12:14:03.967812 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/3487eda5-6351-40b2-babe-458b38432b7f-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:14:03 crc kubenswrapper[5052]: I1203 12:14:03.971709 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3487eda5-6351-40b2-babe-458b38432b7f-kube-api-access-76gsx" (OuterVolumeSpecName: "kube-api-access-76gsx") pod "3487eda5-6351-40b2-babe-458b38432b7f" (UID: "3487eda5-6351-40b2-babe-458b38432b7f"). InnerVolumeSpecName "kube-api-access-76gsx". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:14:03 crc kubenswrapper[5052]: I1203 12:14:03.997343 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3487eda5-6351-40b2-babe-458b38432b7f-config-data" (OuterVolumeSpecName: "config-data") pod "3487eda5-6351-40b2-babe-458b38432b7f" (UID: "3487eda5-6351-40b2-babe-458b38432b7f"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:14:04 crc kubenswrapper[5052]: I1203 12:14:04.020109 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3487eda5-6351-40b2-babe-458b38432b7f-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "3487eda5-6351-40b2-babe-458b38432b7f" (UID: "3487eda5-6351-40b2-babe-458b38432b7f"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:14:04 crc kubenswrapper[5052]: I1203 12:14:04.033035 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3487eda5-6351-40b2-babe-458b38432b7f-nova-metadata-tls-certs" (OuterVolumeSpecName: "nova-metadata-tls-certs") pod "3487eda5-6351-40b2-babe-458b38432b7f" (UID: "3487eda5-6351-40b2-babe-458b38432b7f"). InnerVolumeSpecName "nova-metadata-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:14:04 crc kubenswrapper[5052]: I1203 12:14:04.070365 5052 reconciler_common.go:293] "Volume detached for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/3487eda5-6351-40b2-babe-458b38432b7f-nova-metadata-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:14:04 crc kubenswrapper[5052]: I1203 12:14:04.070396 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-76gsx\" (UniqueName: \"kubernetes.io/projected/3487eda5-6351-40b2-babe-458b38432b7f-kube-api-access-76gsx\") on node \"crc\" DevicePath \"\"" Dec 03 12:14:04 crc kubenswrapper[5052]: I1203 12:14:04.070422 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/3487eda5-6351-40b2-babe-458b38432b7f-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:14:04 crc kubenswrapper[5052]: I1203 12:14:04.070431 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3487eda5-6351-40b2-babe-458b38432b7f-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:14:04 crc kubenswrapper[5052]: I1203 12:14:04.340362 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"cf3ed088-9069-40cf-9c9e-ee9bdf7761d9","Type":"ContainerStarted","Data":"81879a070623f490e700ae5e038359faa2d3ce34a28b8f7e4b856f6c4f95fa7f"} Dec 03 12:14:04 crc kubenswrapper[5052]: I1203 12:14:04.340436 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"cf3ed088-9069-40cf-9c9e-ee9bdf7761d9","Type":"ContainerStarted","Data":"e071655d60b98a1f28aeee8beb7d104c3a460126999dbdcac767c9abe5620039"} Dec 03 12:14:04 crc kubenswrapper[5052]: I1203 12:14:04.346814 5052 generic.go:334] "Generic (PLEG): container finished" podID="3487eda5-6351-40b2-babe-458b38432b7f" containerID="7e5ed174c64d6f31e099a5002b59ab1714e3ae3774c2bbed9284ee785cff7b19" exitCode=0 Dec 03 12:14:04 crc kubenswrapper[5052]: I1203 12:14:04.346885 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"3487eda5-6351-40b2-babe-458b38432b7f","Type":"ContainerDied","Data":"7e5ed174c64d6f31e099a5002b59ab1714e3ae3774c2bbed9284ee785cff7b19"} Dec 03 12:14:04 crc kubenswrapper[5052]: I1203 12:14:04.346914 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"3487eda5-6351-40b2-babe-458b38432b7f","Type":"ContainerDied","Data":"0ffcd1dc5a58757a8975d5fe7eff25ee148ebd5bd47c7922f798af820a7a4191"} Dec 03 12:14:04 crc kubenswrapper[5052]: I1203 12:14:04.346932 5052 scope.go:117] "RemoveContainer" containerID="7e5ed174c64d6f31e099a5002b59ab1714e3ae3774c2bbed9284ee785cff7b19" Dec 03 12:14:04 crc kubenswrapper[5052]: I1203 12:14:04.347051 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:14:04 crc kubenswrapper[5052]: I1203 12:14:04.360139 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"ec4beccd-125d-45c0-979e-6870c39627ee","Type":"ContainerStarted","Data":"671603f6c2dbb3d970e6e1bfc0969c023dd161f664b07e39725aa24183b94261"} Dec 03 12:14:04 crc kubenswrapper[5052]: I1203 12:14:04.360184 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"ec4beccd-125d-45c0-979e-6870c39627ee","Type":"ContainerStarted","Data":"553bdbe6fc8821a81836112777f8b731078a09f0f29b262cd63f6b8862cbac39"} Dec 03 12:14:04 crc kubenswrapper[5052]: I1203 12:14:04.364305 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-scheduler-0" podStartSLOduration=2.364283322 podStartE2EDuration="2.364283322s" podCreationTimestamp="2025-12-03 12:14:02 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:14:04.359271259 +0000 UTC m=+3234.947513023" watchObservedRunningTime="2025-12-03 12:14:04.364283322 +0000 UTC m=+3234.952525106" Dec 03 12:14:04 crc kubenswrapper[5052]: I1203 12:14:04.397081 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-api-0" podStartSLOduration=2.397059277 podStartE2EDuration="2.397059277s" podCreationTimestamp="2025-12-03 12:14:02 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:14:04.387458473 +0000 UTC m=+3234.975700227" watchObservedRunningTime="2025-12-03 12:14:04.397059277 +0000 UTC m=+3234.985301041" Dec 03 12:14:04 crc kubenswrapper[5052]: I1203 12:14:04.426306 5052 scope.go:117] "RemoveContainer" containerID="b616896b8d682e7215c2e892d7490ff87002eb4de1dd5abfe0d3693910b66db1" Dec 03 12:14:04 crc kubenswrapper[5052]: I1203 12:14:04.440833 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:14:04 crc kubenswrapper[5052]: I1203 12:14:04.455442 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:14:04 crc kubenswrapper[5052]: I1203 12:14:04.467697 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:14:04 crc kubenswrapper[5052]: E1203 12:14:04.468252 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="3487eda5-6351-40b2-babe-458b38432b7f" containerName="nova-metadata-metadata" Dec 03 12:14:04 crc kubenswrapper[5052]: I1203 12:14:04.468273 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="3487eda5-6351-40b2-babe-458b38432b7f" containerName="nova-metadata-metadata" Dec 03 12:14:04 crc kubenswrapper[5052]: E1203 12:14:04.468303 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="3487eda5-6351-40b2-babe-458b38432b7f" containerName="nova-metadata-log" Dec 03 12:14:04 crc kubenswrapper[5052]: I1203 12:14:04.468309 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="3487eda5-6351-40b2-babe-458b38432b7f" containerName="nova-metadata-log" Dec 03 12:14:04 crc kubenswrapper[5052]: I1203 12:14:04.468498 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="3487eda5-6351-40b2-babe-458b38432b7f" containerName="nova-metadata-log" Dec 03 12:14:04 crc kubenswrapper[5052]: I1203 12:14:04.468552 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="3487eda5-6351-40b2-babe-458b38432b7f" containerName="nova-metadata-metadata" Dec 03 12:14:04 crc kubenswrapper[5052]: I1203 12:14:04.469925 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:14:04 crc kubenswrapper[5052]: I1203 12:14:04.473671 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-nova-metadata-internal-svc" Dec 03 12:14:04 crc kubenswrapper[5052]: I1203 12:14:04.475096 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-metadata-config-data" Dec 03 12:14:04 crc kubenswrapper[5052]: I1203 12:14:04.479836 5052 scope.go:117] "RemoveContainer" containerID="7e5ed174c64d6f31e099a5002b59ab1714e3ae3774c2bbed9284ee785cff7b19" Dec 03 12:14:04 crc kubenswrapper[5052]: I1203 12:14:04.480452 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:14:04 crc kubenswrapper[5052]: E1203 12:14:04.482334 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7e5ed174c64d6f31e099a5002b59ab1714e3ae3774c2bbed9284ee785cff7b19\": container with ID starting with 7e5ed174c64d6f31e099a5002b59ab1714e3ae3774c2bbed9284ee785cff7b19 not found: ID does not exist" containerID="7e5ed174c64d6f31e099a5002b59ab1714e3ae3774c2bbed9284ee785cff7b19" Dec 03 12:14:04 crc kubenswrapper[5052]: I1203 12:14:04.482400 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7e5ed174c64d6f31e099a5002b59ab1714e3ae3774c2bbed9284ee785cff7b19"} err="failed to get container status \"7e5ed174c64d6f31e099a5002b59ab1714e3ae3774c2bbed9284ee785cff7b19\": rpc error: code = NotFound desc = could not find container \"7e5ed174c64d6f31e099a5002b59ab1714e3ae3774c2bbed9284ee785cff7b19\": container with ID starting with 7e5ed174c64d6f31e099a5002b59ab1714e3ae3774c2bbed9284ee785cff7b19 not found: ID does not exist" Dec 03 12:14:04 crc kubenswrapper[5052]: I1203 12:14:04.482437 5052 scope.go:117] "RemoveContainer" containerID="b616896b8d682e7215c2e892d7490ff87002eb4de1dd5abfe0d3693910b66db1" Dec 03 12:14:04 crc kubenswrapper[5052]: E1203 12:14:04.483407 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b616896b8d682e7215c2e892d7490ff87002eb4de1dd5abfe0d3693910b66db1\": container with ID starting with b616896b8d682e7215c2e892d7490ff87002eb4de1dd5abfe0d3693910b66db1 not found: ID does not exist" containerID="b616896b8d682e7215c2e892d7490ff87002eb4de1dd5abfe0d3693910b66db1" Dec 03 12:14:04 crc kubenswrapper[5052]: I1203 12:14:04.483471 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b616896b8d682e7215c2e892d7490ff87002eb4de1dd5abfe0d3693910b66db1"} err="failed to get container status \"b616896b8d682e7215c2e892d7490ff87002eb4de1dd5abfe0d3693910b66db1\": rpc error: code = NotFound desc = could not find container \"b616896b8d682e7215c2e892d7490ff87002eb4de1dd5abfe0d3693910b66db1\": container with ID starting with b616896b8d682e7215c2e892d7490ff87002eb4de1dd5abfe0d3693910b66db1 not found: ID does not exist" Dec 03 12:14:04 crc kubenswrapper[5052]: I1203 12:14:04.554057 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:14:04 crc kubenswrapper[5052]: I1203 12:14:04.599286 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/cd486343-0aad-4039-a23b-34c60cd1a381-nova-metadata-tls-certs\") pod \"nova-metadata-0\" (UID: \"cd486343-0aad-4039-a23b-34c60cd1a381\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:14:04 crc kubenswrapper[5052]: I1203 12:14:04.599352 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cclpd\" (UniqueName: \"kubernetes.io/projected/cd486343-0aad-4039-a23b-34c60cd1a381-kube-api-access-cclpd\") pod \"nova-metadata-0\" (UID: \"cd486343-0aad-4039-a23b-34c60cd1a381\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:14:04 crc kubenswrapper[5052]: I1203 12:14:04.599486 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/cd486343-0aad-4039-a23b-34c60cd1a381-logs\") pod \"nova-metadata-0\" (UID: \"cd486343-0aad-4039-a23b-34c60cd1a381\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:14:04 crc kubenswrapper[5052]: I1203 12:14:04.599520 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/cd486343-0aad-4039-a23b-34c60cd1a381-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"cd486343-0aad-4039-a23b-34c60cd1a381\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:14:04 crc kubenswrapper[5052]: I1203 12:14:04.599613 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/cd486343-0aad-4039-a23b-34c60cd1a381-config-data\") pod \"nova-metadata-0\" (UID: \"cd486343-0aad-4039-a23b-34c60cd1a381\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:14:04 crc kubenswrapper[5052]: I1203 12:14:04.673760 5052 scope.go:117] "RemoveContainer" containerID="63298107b3d090deff9ed92f3d613e5b1c86d9137684d117500257db8ee1c805" Dec 03 12:14:04 crc kubenswrapper[5052]: I1203 12:14:04.701341 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/cd486343-0aad-4039-a23b-34c60cd1a381-logs\") pod \"nova-metadata-0\" (UID: \"cd486343-0aad-4039-a23b-34c60cd1a381\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:14:04 crc kubenswrapper[5052]: I1203 12:14:04.701387 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/cd486343-0aad-4039-a23b-34c60cd1a381-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"cd486343-0aad-4039-a23b-34c60cd1a381\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:14:04 crc kubenswrapper[5052]: I1203 12:14:04.701472 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/cd486343-0aad-4039-a23b-34c60cd1a381-config-data\") pod \"nova-metadata-0\" (UID: \"cd486343-0aad-4039-a23b-34c60cd1a381\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:14:04 crc kubenswrapper[5052]: I1203 12:14:04.701565 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/cd486343-0aad-4039-a23b-34c60cd1a381-nova-metadata-tls-certs\") pod \"nova-metadata-0\" (UID: \"cd486343-0aad-4039-a23b-34c60cd1a381\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:14:04 crc kubenswrapper[5052]: I1203 12:14:04.701599 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-cclpd\" (UniqueName: \"kubernetes.io/projected/cd486343-0aad-4039-a23b-34c60cd1a381-kube-api-access-cclpd\") pod \"nova-metadata-0\" (UID: \"cd486343-0aad-4039-a23b-34c60cd1a381\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:14:04 crc kubenswrapper[5052]: I1203 12:14:04.702874 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/cd486343-0aad-4039-a23b-34c60cd1a381-logs\") pod \"nova-metadata-0\" (UID: \"cd486343-0aad-4039-a23b-34c60cd1a381\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:14:04 crc kubenswrapper[5052]: I1203 12:14:04.708266 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/cd486343-0aad-4039-a23b-34c60cd1a381-config-data\") pod \"nova-metadata-0\" (UID: \"cd486343-0aad-4039-a23b-34c60cd1a381\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:14:04 crc kubenswrapper[5052]: I1203 12:14:04.708600 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/cd486343-0aad-4039-a23b-34c60cd1a381-nova-metadata-tls-certs\") pod \"nova-metadata-0\" (UID: \"cd486343-0aad-4039-a23b-34c60cd1a381\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:14:04 crc kubenswrapper[5052]: I1203 12:14:04.709179 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/cd486343-0aad-4039-a23b-34c60cd1a381-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"cd486343-0aad-4039-a23b-34c60cd1a381\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:14:04 crc kubenswrapper[5052]: I1203 12:14:04.722419 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-cclpd\" (UniqueName: \"kubernetes.io/projected/cd486343-0aad-4039-a23b-34c60cd1a381-kube-api-access-cclpd\") pod \"nova-metadata-0\" (UID: \"cd486343-0aad-4039-a23b-34c60cd1a381\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:14:04 crc kubenswrapper[5052]: I1203 12:14:04.796825 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:14:05 crc kubenswrapper[5052]: I1203 12:14:05.315113 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:14:05 crc kubenswrapper[5052]: I1203 12:14:05.372099 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" event={"ID":"10c237bc-72e7-4da3-9f2c-b1688395bf6e","Type":"ContainerStarted","Data":"79e965387252796514d44c1fd52f83c66501c2dc3200a49ddb7dacbcbcf015ce"} Dec 03 12:14:05 crc kubenswrapper[5052]: I1203 12:14:05.378554 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"cd486343-0aad-4039-a23b-34c60cd1a381","Type":"ContainerStarted","Data":"5b5a66a88c3807b2a543ebb34939109f35b4a3d23a9dc84017aad6be0ac33835"} Dec 03 12:14:05 crc kubenswrapper[5052]: I1203 12:14:05.381267 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"da9be43d-a787-4511-9b0c-4b3f307cb891","Type":"ContainerStarted","Data":"598a1678f0b225317e34ba70d42bcf62414fadb1d4ddb69e0d9ece6029ae74cf"} Dec 03 12:14:05 crc kubenswrapper[5052]: I1203 12:14:05.382007 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:14:05 crc kubenswrapper[5052]: I1203 12:14:05.421188 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/ceilometer-0" podStartSLOduration=1.832228323 podStartE2EDuration="5.421168998s" podCreationTimestamp="2025-12-03 12:14:00 +0000 UTC" firstStartedPulling="2025-12-03 12:14:01.185171329 +0000 UTC m=+3231.773413093" lastFinishedPulling="2025-12-03 12:14:04.774112004 +0000 UTC m=+3235.362353768" observedRunningTime="2025-12-03 12:14:05.407600891 +0000 UTC m=+3235.995842655" watchObservedRunningTime="2025-12-03 12:14:05.421168998 +0000 UTC m=+3236.009410762" Dec 03 12:14:05 crc kubenswrapper[5052]: I1203 12:14:05.696604 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3487eda5-6351-40b2-babe-458b38432b7f" path="/var/lib/kubelet/pods/3487eda5-6351-40b2-babe-458b38432b7f/volumes" Dec 03 12:14:06 crc kubenswrapper[5052]: I1203 12:14:06.399102 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"cd486343-0aad-4039-a23b-34c60cd1a381","Type":"ContainerStarted","Data":"ec66627f7d357cab05559c6d34ef624f9d761ad2e4531d42cccb0b06d7bb2cba"} Dec 03 12:14:06 crc kubenswrapper[5052]: I1203 12:14:06.399425 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"cd486343-0aad-4039-a23b-34c60cd1a381","Type":"ContainerStarted","Data":"99335f1182dfcd9be7ce5b7f281d04f752ed33796c96039024fa49675032786b"} Dec 03 12:14:07 crc kubenswrapper[5052]: I1203 12:14:07.797857 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:14:09 crc kubenswrapper[5052]: I1203 12:14:09.797501 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:14:09 crc kubenswrapper[5052]: I1203 12:14:09.797878 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:14:12 crc kubenswrapper[5052]: I1203 12:14:12.763283 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:14:12 crc kubenswrapper[5052]: I1203 12:14:12.763851 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:14:12 crc kubenswrapper[5052]: I1203 12:14:12.797798 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:14:12 crc kubenswrapper[5052]: I1203 12:14:12.827005 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:14:12 crc kubenswrapper[5052]: I1203 12:14:12.849264 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-metadata-0" podStartSLOduration=8.849245752 podStartE2EDuration="8.849245752s" podCreationTimestamp="2025-12-03 12:14:04 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:14:06.424758514 +0000 UTC m=+3237.013000288" watchObservedRunningTime="2025-12-03 12:14:12.849245752 +0000 UTC m=+3243.437487516" Dec 03 12:14:13 crc kubenswrapper[5052]: I1203 12:14:13.495191 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:14:13 crc kubenswrapper[5052]: I1203 12:14:13.847740 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-api-0" podUID="ec4beccd-125d-45c0-979e-6870c39627ee" containerName="nova-api-log" probeResult="failure" output="Get \"http://10.217.1.41:8774/\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Dec 03 12:14:13 crc kubenswrapper[5052]: I1203 12:14:13.848096 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-api-0" podUID="ec4beccd-125d-45c0-979e-6870c39627ee" containerName="nova-api-api" probeResult="failure" output="Get \"http://10.217.1.41:8774/\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Dec 03 12:14:14 crc kubenswrapper[5052]: I1203 12:14:14.797997 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:14:14 crc kubenswrapper[5052]: I1203 12:14:14.798392 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:14:15 crc kubenswrapper[5052]: I1203 12:14:15.808809 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-metadata-0" podUID="cd486343-0aad-4039-a23b-34c60cd1a381" containerName="nova-metadata-metadata" probeResult="failure" output="Get \"https://10.217.1.43:8775/\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Dec 03 12:14:15 crc kubenswrapper[5052]: I1203 12:14:15.808863 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-metadata-0" podUID="cd486343-0aad-4039-a23b-34c60cd1a381" containerName="nova-metadata-log" probeResult="failure" output="Get \"https://10.217.1.43:8775/\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Dec 03 12:14:22 crc kubenswrapper[5052]: I1203 12:14:22.761565 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:14:22 crc kubenswrapper[5052]: I1203 12:14:22.762553 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:14:22 crc kubenswrapper[5052]: I1203 12:14:22.762680 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:14:22 crc kubenswrapper[5052]: I1203 12:14:22.766630 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:14:23 crc kubenswrapper[5052]: I1203 12:14:23.557993 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:14:23 crc kubenswrapper[5052]: I1203 12:14:23.562008 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:14:24 crc kubenswrapper[5052]: I1203 12:14:24.804067 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:14:24 crc kubenswrapper[5052]: I1203 12:14:24.817719 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:14:24 crc kubenswrapper[5052]: I1203 12:14:24.818282 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:14:25 crc kubenswrapper[5052]: I1203 12:14:25.592839 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:14:26 crc kubenswrapper[5052]: I1203 12:14:26.207497 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:14:26 crc kubenswrapper[5052]: I1203 12:14:26.208124 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="da9be43d-a787-4511-9b0c-4b3f307cb891" containerName="ceilometer-central-agent" containerID="cri-o://d92cbc5bb31cf05026bada3e2ff528d9bffb62aee81e5876a53cd8465313f7d5" gracePeriod=30 Dec 03 12:14:26 crc kubenswrapper[5052]: I1203 12:14:26.208258 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="da9be43d-a787-4511-9b0c-4b3f307cb891" containerName="proxy-httpd" containerID="cri-o://598a1678f0b225317e34ba70d42bcf62414fadb1d4ddb69e0d9ece6029ae74cf" gracePeriod=30 Dec 03 12:14:26 crc kubenswrapper[5052]: I1203 12:14:26.208305 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="da9be43d-a787-4511-9b0c-4b3f307cb891" containerName="sg-core" containerID="cri-o://8c8b0a51f39bd122aed7b8fee5cba3c506b692f8e1b218daf6800788487afea8" gracePeriod=30 Dec 03 12:14:26 crc kubenswrapper[5052]: I1203 12:14:26.208337 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="da9be43d-a787-4511-9b0c-4b3f307cb891" containerName="ceilometer-notification-agent" containerID="cri-o://21ba3a277b2c2e8c629ffb7b800e7a6375bea92da2c176f4c34f1d3393d5da4f" gracePeriod=30 Dec 03 12:14:26 crc kubenswrapper[5052]: I1203 12:14:26.217521 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/ceilometer-0" podUID="da9be43d-a787-4511-9b0c-4b3f307cb891" containerName="proxy-httpd" probeResult="failure" output="Get \"https://10.217.1.40:3000/\": EOF" Dec 03 12:14:26 crc kubenswrapper[5052]: I1203 12:14:26.598468 5052 generic.go:334] "Generic (PLEG): container finished" podID="da9be43d-a787-4511-9b0c-4b3f307cb891" containerID="598a1678f0b225317e34ba70d42bcf62414fadb1d4ddb69e0d9ece6029ae74cf" exitCode=0 Dec 03 12:14:26 crc kubenswrapper[5052]: I1203 12:14:26.598503 5052 generic.go:334] "Generic (PLEG): container finished" podID="da9be43d-a787-4511-9b0c-4b3f307cb891" containerID="8c8b0a51f39bd122aed7b8fee5cba3c506b692f8e1b218daf6800788487afea8" exitCode=2 Dec 03 12:14:26 crc kubenswrapper[5052]: I1203 12:14:26.598531 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"da9be43d-a787-4511-9b0c-4b3f307cb891","Type":"ContainerDied","Data":"598a1678f0b225317e34ba70d42bcf62414fadb1d4ddb69e0d9ece6029ae74cf"} Dec 03 12:14:26 crc kubenswrapper[5052]: I1203 12:14:26.598586 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"da9be43d-a787-4511-9b0c-4b3f307cb891","Type":"ContainerDied","Data":"8c8b0a51f39bd122aed7b8fee5cba3c506b692f8e1b218daf6800788487afea8"} Dec 03 12:14:26 crc kubenswrapper[5052]: I1203 12:14:26.798571 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:14:26 crc kubenswrapper[5052]: I1203 12:14:26.798867 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-api-0" podUID="ec4beccd-125d-45c0-979e-6870c39627ee" containerName="nova-api-log" containerID="cri-o://553bdbe6fc8821a81836112777f8b731078a09f0f29b262cd63f6b8862cbac39" gracePeriod=30 Dec 03 12:14:26 crc kubenswrapper[5052]: I1203 12:14:26.798954 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-api-0" podUID="ec4beccd-125d-45c0-979e-6870c39627ee" containerName="nova-api-api" containerID="cri-o://671603f6c2dbb3d970e6e1bfc0969c023dd161f664b07e39725aa24183b94261" gracePeriod=30 Dec 03 12:14:27 crc kubenswrapper[5052]: I1203 12:14:27.611634 5052 generic.go:334] "Generic (PLEG): container finished" podID="ec4beccd-125d-45c0-979e-6870c39627ee" containerID="553bdbe6fc8821a81836112777f8b731078a09f0f29b262cd63f6b8862cbac39" exitCode=143 Dec 03 12:14:27 crc kubenswrapper[5052]: I1203 12:14:27.611712 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"ec4beccd-125d-45c0-979e-6870c39627ee","Type":"ContainerDied","Data":"553bdbe6fc8821a81836112777f8b731078a09f0f29b262cd63f6b8862cbac39"} Dec 03 12:14:27 crc kubenswrapper[5052]: I1203 12:14:27.616388 5052 generic.go:334] "Generic (PLEG): container finished" podID="da9be43d-a787-4511-9b0c-4b3f307cb891" containerID="d92cbc5bb31cf05026bada3e2ff528d9bffb62aee81e5876a53cd8465313f7d5" exitCode=0 Dec 03 12:14:27 crc kubenswrapper[5052]: I1203 12:14:27.616444 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"da9be43d-a787-4511-9b0c-4b3f307cb891","Type":"ContainerDied","Data":"d92cbc5bb31cf05026bada3e2ff528d9bffb62aee81e5876a53cd8465313f7d5"} Dec 03 12:14:29 crc kubenswrapper[5052]: I1203 12:14:29.650673 5052 generic.go:334] "Generic (PLEG): container finished" podID="da9be43d-a787-4511-9b0c-4b3f307cb891" containerID="21ba3a277b2c2e8c629ffb7b800e7a6375bea92da2c176f4c34f1d3393d5da4f" exitCode=0 Dec 03 12:14:29 crc kubenswrapper[5052]: I1203 12:14:29.651153 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"da9be43d-a787-4511-9b0c-4b3f307cb891","Type":"ContainerDied","Data":"21ba3a277b2c2e8c629ffb7b800e7a6375bea92da2c176f4c34f1d3393d5da4f"} Dec 03 12:14:29 crc kubenswrapper[5052]: I1203 12:14:29.858161 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:14:29 crc kubenswrapper[5052]: I1203 12:14:29.918650 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/da9be43d-a787-4511-9b0c-4b3f307cb891-config-data\") pod \"da9be43d-a787-4511-9b0c-4b3f307cb891\" (UID: \"da9be43d-a787-4511-9b0c-4b3f307cb891\") " Dec 03 12:14:29 crc kubenswrapper[5052]: I1203 12:14:29.918940 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/da9be43d-a787-4511-9b0c-4b3f307cb891-log-httpd\") pod \"da9be43d-a787-4511-9b0c-4b3f307cb891\" (UID: \"da9be43d-a787-4511-9b0c-4b3f307cb891\") " Dec 03 12:14:29 crc kubenswrapper[5052]: I1203 12:14:29.919019 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/da9be43d-a787-4511-9b0c-4b3f307cb891-ceilometer-tls-certs\") pod \"da9be43d-a787-4511-9b0c-4b3f307cb891\" (UID: \"da9be43d-a787-4511-9b0c-4b3f307cb891\") " Dec 03 12:14:29 crc kubenswrapper[5052]: I1203 12:14:29.919109 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/da9be43d-a787-4511-9b0c-4b3f307cb891-sg-core-conf-yaml\") pod \"da9be43d-a787-4511-9b0c-4b3f307cb891\" (UID: \"da9be43d-a787-4511-9b0c-4b3f307cb891\") " Dec 03 12:14:29 crc kubenswrapper[5052]: I1203 12:14:29.919229 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9d96n\" (UniqueName: \"kubernetes.io/projected/da9be43d-a787-4511-9b0c-4b3f307cb891-kube-api-access-9d96n\") pod \"da9be43d-a787-4511-9b0c-4b3f307cb891\" (UID: \"da9be43d-a787-4511-9b0c-4b3f307cb891\") " Dec 03 12:14:29 crc kubenswrapper[5052]: I1203 12:14:29.919296 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/da9be43d-a787-4511-9b0c-4b3f307cb891-combined-ca-bundle\") pod \"da9be43d-a787-4511-9b0c-4b3f307cb891\" (UID: \"da9be43d-a787-4511-9b0c-4b3f307cb891\") " Dec 03 12:14:29 crc kubenswrapper[5052]: I1203 12:14:29.919388 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/da9be43d-a787-4511-9b0c-4b3f307cb891-run-httpd\") pod \"da9be43d-a787-4511-9b0c-4b3f307cb891\" (UID: \"da9be43d-a787-4511-9b0c-4b3f307cb891\") " Dec 03 12:14:29 crc kubenswrapper[5052]: I1203 12:14:29.919477 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/da9be43d-a787-4511-9b0c-4b3f307cb891-scripts\") pod \"da9be43d-a787-4511-9b0c-4b3f307cb891\" (UID: \"da9be43d-a787-4511-9b0c-4b3f307cb891\") " Dec 03 12:14:29 crc kubenswrapper[5052]: I1203 12:14:29.921356 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/da9be43d-a787-4511-9b0c-4b3f307cb891-log-httpd" (OuterVolumeSpecName: "log-httpd") pod "da9be43d-a787-4511-9b0c-4b3f307cb891" (UID: "da9be43d-a787-4511-9b0c-4b3f307cb891"). InnerVolumeSpecName "log-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:14:29 crc kubenswrapper[5052]: I1203 12:14:29.921520 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/da9be43d-a787-4511-9b0c-4b3f307cb891-run-httpd" (OuterVolumeSpecName: "run-httpd") pod "da9be43d-a787-4511-9b0c-4b3f307cb891" (UID: "da9be43d-a787-4511-9b0c-4b3f307cb891"). InnerVolumeSpecName "run-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:14:29 crc kubenswrapper[5052]: I1203 12:14:29.934779 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/da9be43d-a787-4511-9b0c-4b3f307cb891-scripts" (OuterVolumeSpecName: "scripts") pod "da9be43d-a787-4511-9b0c-4b3f307cb891" (UID: "da9be43d-a787-4511-9b0c-4b3f307cb891"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:14:29 crc kubenswrapper[5052]: I1203 12:14:29.940542 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/da9be43d-a787-4511-9b0c-4b3f307cb891-kube-api-access-9d96n" (OuterVolumeSpecName: "kube-api-access-9d96n") pod "da9be43d-a787-4511-9b0c-4b3f307cb891" (UID: "da9be43d-a787-4511-9b0c-4b3f307cb891"). InnerVolumeSpecName "kube-api-access-9d96n". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:14:29 crc kubenswrapper[5052]: I1203 12:14:29.952150 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/da9be43d-a787-4511-9b0c-4b3f307cb891-sg-core-conf-yaml" (OuterVolumeSpecName: "sg-core-conf-yaml") pod "da9be43d-a787-4511-9b0c-4b3f307cb891" (UID: "da9be43d-a787-4511-9b0c-4b3f307cb891"). InnerVolumeSpecName "sg-core-conf-yaml". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:14:29 crc kubenswrapper[5052]: I1203 12:14:29.991019 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/da9be43d-a787-4511-9b0c-4b3f307cb891-ceilometer-tls-certs" (OuterVolumeSpecName: "ceilometer-tls-certs") pod "da9be43d-a787-4511-9b0c-4b3f307cb891" (UID: "da9be43d-a787-4511-9b0c-4b3f307cb891"). InnerVolumeSpecName "ceilometer-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:14:30 crc kubenswrapper[5052]: I1203 12:14:30.023073 5052 reconciler_common.go:293] "Volume detached for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/da9be43d-a787-4511-9b0c-4b3f307cb891-ceilometer-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:14:30 crc kubenswrapper[5052]: I1203 12:14:30.023134 5052 reconciler_common.go:293] "Volume detached for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/da9be43d-a787-4511-9b0c-4b3f307cb891-sg-core-conf-yaml\") on node \"crc\" DevicePath \"\"" Dec 03 12:14:30 crc kubenswrapper[5052]: I1203 12:14:30.023147 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-9d96n\" (UniqueName: \"kubernetes.io/projected/da9be43d-a787-4511-9b0c-4b3f307cb891-kube-api-access-9d96n\") on node \"crc\" DevicePath \"\"" Dec 03 12:14:30 crc kubenswrapper[5052]: I1203 12:14:30.023157 5052 reconciler_common.go:293] "Volume detached for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/da9be43d-a787-4511-9b0c-4b3f307cb891-run-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:14:30 crc kubenswrapper[5052]: I1203 12:14:30.023167 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/da9be43d-a787-4511-9b0c-4b3f307cb891-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:14:30 crc kubenswrapper[5052]: I1203 12:14:30.023176 5052 reconciler_common.go:293] "Volume detached for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/da9be43d-a787-4511-9b0c-4b3f307cb891-log-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:14:30 crc kubenswrapper[5052]: I1203 12:14:30.048151 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/da9be43d-a787-4511-9b0c-4b3f307cb891-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "da9be43d-a787-4511-9b0c-4b3f307cb891" (UID: "da9be43d-a787-4511-9b0c-4b3f307cb891"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:14:30 crc kubenswrapper[5052]: I1203 12:14:30.057887 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/da9be43d-a787-4511-9b0c-4b3f307cb891-config-data" (OuterVolumeSpecName: "config-data") pod "da9be43d-a787-4511-9b0c-4b3f307cb891" (UID: "da9be43d-a787-4511-9b0c-4b3f307cb891"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:14:30 crc kubenswrapper[5052]: I1203 12:14:30.124550 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/da9be43d-a787-4511-9b0c-4b3f307cb891-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:14:30 crc kubenswrapper[5052]: I1203 12:14:30.124588 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/da9be43d-a787-4511-9b0c-4b3f307cb891-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:14:30 crc kubenswrapper[5052]: I1203 12:14:30.344848 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:14:30 crc kubenswrapper[5052]: I1203 12:14:30.429244 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ec4beccd-125d-45c0-979e-6870c39627ee-combined-ca-bundle\") pod \"ec4beccd-125d-45c0-979e-6870c39627ee\" (UID: \"ec4beccd-125d-45c0-979e-6870c39627ee\") " Dec 03 12:14:30 crc kubenswrapper[5052]: I1203 12:14:30.429310 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/ec4beccd-125d-45c0-979e-6870c39627ee-logs\") pod \"ec4beccd-125d-45c0-979e-6870c39627ee\" (UID: \"ec4beccd-125d-45c0-979e-6870c39627ee\") " Dec 03 12:14:30 crc kubenswrapper[5052]: I1203 12:14:30.429389 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r57vx\" (UniqueName: \"kubernetes.io/projected/ec4beccd-125d-45c0-979e-6870c39627ee-kube-api-access-r57vx\") pod \"ec4beccd-125d-45c0-979e-6870c39627ee\" (UID: \"ec4beccd-125d-45c0-979e-6870c39627ee\") " Dec 03 12:14:30 crc kubenswrapper[5052]: I1203 12:14:30.429744 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/ec4beccd-125d-45c0-979e-6870c39627ee-config-data\") pod \"ec4beccd-125d-45c0-979e-6870c39627ee\" (UID: \"ec4beccd-125d-45c0-979e-6870c39627ee\") " Dec 03 12:14:30 crc kubenswrapper[5052]: I1203 12:14:30.432446 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec4beccd-125d-45c0-979e-6870c39627ee-logs" (OuterVolumeSpecName: "logs") pod "ec4beccd-125d-45c0-979e-6870c39627ee" (UID: "ec4beccd-125d-45c0-979e-6870c39627ee"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:14:30 crc kubenswrapper[5052]: I1203 12:14:30.434227 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ec4beccd-125d-45c0-979e-6870c39627ee-kube-api-access-r57vx" (OuterVolumeSpecName: "kube-api-access-r57vx") pod "ec4beccd-125d-45c0-979e-6870c39627ee" (UID: "ec4beccd-125d-45c0-979e-6870c39627ee"). InnerVolumeSpecName "kube-api-access-r57vx". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:14:30 crc kubenswrapper[5052]: I1203 12:14:30.459835 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ec4beccd-125d-45c0-979e-6870c39627ee-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "ec4beccd-125d-45c0-979e-6870c39627ee" (UID: "ec4beccd-125d-45c0-979e-6870c39627ee"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:14:30 crc kubenswrapper[5052]: I1203 12:14:30.469486 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ec4beccd-125d-45c0-979e-6870c39627ee-config-data" (OuterVolumeSpecName: "config-data") pod "ec4beccd-125d-45c0-979e-6870c39627ee" (UID: "ec4beccd-125d-45c0-979e-6870c39627ee"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:14:30 crc kubenswrapper[5052]: I1203 12:14:30.532498 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ec4beccd-125d-45c0-979e-6870c39627ee-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:14:30 crc kubenswrapper[5052]: I1203 12:14:30.532568 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/ec4beccd-125d-45c0-979e-6870c39627ee-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:14:30 crc kubenswrapper[5052]: I1203 12:14:30.532583 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-r57vx\" (UniqueName: \"kubernetes.io/projected/ec4beccd-125d-45c0-979e-6870c39627ee-kube-api-access-r57vx\") on node \"crc\" DevicePath \"\"" Dec 03 12:14:30 crc kubenswrapper[5052]: I1203 12:14:30.532597 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/ec4beccd-125d-45c0-979e-6870c39627ee-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:14:30 crc kubenswrapper[5052]: I1203 12:14:30.662556 5052 generic.go:334] "Generic (PLEG): container finished" podID="ec4beccd-125d-45c0-979e-6870c39627ee" containerID="671603f6c2dbb3d970e6e1bfc0969c023dd161f664b07e39725aa24183b94261" exitCode=0 Dec 03 12:14:30 crc kubenswrapper[5052]: I1203 12:14:30.662630 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:14:30 crc kubenswrapper[5052]: I1203 12:14:30.662636 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"ec4beccd-125d-45c0-979e-6870c39627ee","Type":"ContainerDied","Data":"671603f6c2dbb3d970e6e1bfc0969c023dd161f664b07e39725aa24183b94261"} Dec 03 12:14:30 crc kubenswrapper[5052]: I1203 12:14:30.662814 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"ec4beccd-125d-45c0-979e-6870c39627ee","Type":"ContainerDied","Data":"cd6dc2d6c74dda47c9b04f8f0eb17d144f42ce6599ac53c1e3704640e0cd7e19"} Dec 03 12:14:30 crc kubenswrapper[5052]: I1203 12:14:30.662862 5052 scope.go:117] "RemoveContainer" containerID="671603f6c2dbb3d970e6e1bfc0969c023dd161f664b07e39725aa24183b94261" Dec 03 12:14:30 crc kubenswrapper[5052]: I1203 12:14:30.667844 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"da9be43d-a787-4511-9b0c-4b3f307cb891","Type":"ContainerDied","Data":"391bda9a44718db5214e627a0236bad3e9b28894de64534a944c5ffb3e3dcb9a"} Dec 03 12:14:30 crc kubenswrapper[5052]: I1203 12:14:30.667983 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:14:30 crc kubenswrapper[5052]: I1203 12:14:30.695634 5052 scope.go:117] "RemoveContainer" containerID="553bdbe6fc8821a81836112777f8b731078a09f0f29b262cd63f6b8862cbac39" Dec 03 12:14:30 crc kubenswrapper[5052]: I1203 12:14:30.711096 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:14:30 crc kubenswrapper[5052]: I1203 12:14:30.721638 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:14:30 crc kubenswrapper[5052]: I1203 12:14:30.728673 5052 scope.go:117] "RemoveContainer" containerID="671603f6c2dbb3d970e6e1bfc0969c023dd161f664b07e39725aa24183b94261" Dec 03 12:14:30 crc kubenswrapper[5052]: E1203 12:14:30.731192 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"671603f6c2dbb3d970e6e1bfc0969c023dd161f664b07e39725aa24183b94261\": container with ID starting with 671603f6c2dbb3d970e6e1bfc0969c023dd161f664b07e39725aa24183b94261 not found: ID does not exist" containerID="671603f6c2dbb3d970e6e1bfc0969c023dd161f664b07e39725aa24183b94261" Dec 03 12:14:30 crc kubenswrapper[5052]: I1203 12:14:30.731227 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"671603f6c2dbb3d970e6e1bfc0969c023dd161f664b07e39725aa24183b94261"} err="failed to get container status \"671603f6c2dbb3d970e6e1bfc0969c023dd161f664b07e39725aa24183b94261\": rpc error: code = NotFound desc = could not find container \"671603f6c2dbb3d970e6e1bfc0969c023dd161f664b07e39725aa24183b94261\": container with ID starting with 671603f6c2dbb3d970e6e1bfc0969c023dd161f664b07e39725aa24183b94261 not found: ID does not exist" Dec 03 12:14:30 crc kubenswrapper[5052]: I1203 12:14:30.731249 5052 scope.go:117] "RemoveContainer" containerID="553bdbe6fc8821a81836112777f8b731078a09f0f29b262cd63f6b8862cbac39" Dec 03 12:14:30 crc kubenswrapper[5052]: E1203 12:14:30.731784 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"553bdbe6fc8821a81836112777f8b731078a09f0f29b262cd63f6b8862cbac39\": container with ID starting with 553bdbe6fc8821a81836112777f8b731078a09f0f29b262cd63f6b8862cbac39 not found: ID does not exist" containerID="553bdbe6fc8821a81836112777f8b731078a09f0f29b262cd63f6b8862cbac39" Dec 03 12:14:30 crc kubenswrapper[5052]: I1203 12:14:30.731808 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"553bdbe6fc8821a81836112777f8b731078a09f0f29b262cd63f6b8862cbac39"} err="failed to get container status \"553bdbe6fc8821a81836112777f8b731078a09f0f29b262cd63f6b8862cbac39\": rpc error: code = NotFound desc = could not find container \"553bdbe6fc8821a81836112777f8b731078a09f0f29b262cd63f6b8862cbac39\": container with ID starting with 553bdbe6fc8821a81836112777f8b731078a09f0f29b262cd63f6b8862cbac39 not found: ID does not exist" Dec 03 12:14:30 crc kubenswrapper[5052]: I1203 12:14:30.731820 5052 scope.go:117] "RemoveContainer" containerID="598a1678f0b225317e34ba70d42bcf62414fadb1d4ddb69e0d9ece6029ae74cf" Dec 03 12:14:30 crc kubenswrapper[5052]: I1203 12:14:30.758881 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:14:30 crc kubenswrapper[5052]: I1203 12:14:30.786833 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:14:30 crc kubenswrapper[5052]: I1203 12:14:30.805071 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:14:30 crc kubenswrapper[5052]: E1203 12:14:30.805553 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="da9be43d-a787-4511-9b0c-4b3f307cb891" containerName="sg-core" Dec 03 12:14:30 crc kubenswrapper[5052]: I1203 12:14:30.805577 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="da9be43d-a787-4511-9b0c-4b3f307cb891" containerName="sg-core" Dec 03 12:14:30 crc kubenswrapper[5052]: E1203 12:14:30.805603 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="ec4beccd-125d-45c0-979e-6870c39627ee" containerName="nova-api-api" Dec 03 12:14:30 crc kubenswrapper[5052]: I1203 12:14:30.805612 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="ec4beccd-125d-45c0-979e-6870c39627ee" containerName="nova-api-api" Dec 03 12:14:30 crc kubenswrapper[5052]: E1203 12:14:30.805631 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="ec4beccd-125d-45c0-979e-6870c39627ee" containerName="nova-api-log" Dec 03 12:14:30 crc kubenswrapper[5052]: I1203 12:14:30.805638 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="ec4beccd-125d-45c0-979e-6870c39627ee" containerName="nova-api-log" Dec 03 12:14:30 crc kubenswrapper[5052]: E1203 12:14:30.805651 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="da9be43d-a787-4511-9b0c-4b3f307cb891" containerName="proxy-httpd" Dec 03 12:14:30 crc kubenswrapper[5052]: I1203 12:14:30.805658 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="da9be43d-a787-4511-9b0c-4b3f307cb891" containerName="proxy-httpd" Dec 03 12:14:30 crc kubenswrapper[5052]: E1203 12:14:30.805687 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="da9be43d-a787-4511-9b0c-4b3f307cb891" containerName="ceilometer-notification-agent" Dec 03 12:14:30 crc kubenswrapper[5052]: I1203 12:14:30.805699 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="da9be43d-a787-4511-9b0c-4b3f307cb891" containerName="ceilometer-notification-agent" Dec 03 12:14:30 crc kubenswrapper[5052]: E1203 12:14:30.805720 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="da9be43d-a787-4511-9b0c-4b3f307cb891" containerName="ceilometer-central-agent" Dec 03 12:14:30 crc kubenswrapper[5052]: I1203 12:14:30.805728 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="da9be43d-a787-4511-9b0c-4b3f307cb891" containerName="ceilometer-central-agent" Dec 03 12:14:30 crc kubenswrapper[5052]: I1203 12:14:30.805923 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="ec4beccd-125d-45c0-979e-6870c39627ee" containerName="nova-api-log" Dec 03 12:14:30 crc kubenswrapper[5052]: I1203 12:14:30.805940 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="da9be43d-a787-4511-9b0c-4b3f307cb891" containerName="ceilometer-notification-agent" Dec 03 12:14:30 crc kubenswrapper[5052]: I1203 12:14:30.805957 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="da9be43d-a787-4511-9b0c-4b3f307cb891" containerName="ceilometer-central-agent" Dec 03 12:14:30 crc kubenswrapper[5052]: I1203 12:14:30.805969 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="da9be43d-a787-4511-9b0c-4b3f307cb891" containerName="sg-core" Dec 03 12:14:30 crc kubenswrapper[5052]: I1203 12:14:30.805989 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="ec4beccd-125d-45c0-979e-6870c39627ee" containerName="nova-api-api" Dec 03 12:14:30 crc kubenswrapper[5052]: I1203 12:14:30.806001 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="da9be43d-a787-4511-9b0c-4b3f307cb891" containerName="proxy-httpd" Dec 03 12:14:30 crc kubenswrapper[5052]: I1203 12:14:30.807747 5052 scope.go:117] "RemoveContainer" containerID="8c8b0a51f39bd122aed7b8fee5cba3c506b692f8e1b218daf6800788487afea8" Dec 03 12:14:30 crc kubenswrapper[5052]: I1203 12:14:30.808318 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:14:30 crc kubenswrapper[5052]: I1203 12:14:30.811007 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-scripts" Dec 03 12:14:30 crc kubenswrapper[5052]: I1203 12:14:30.813676 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-config-data" Dec 03 12:14:30 crc kubenswrapper[5052]: I1203 12:14:30.817331 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-ceilometer-internal-svc" Dec 03 12:14:30 crc kubenswrapper[5052]: I1203 12:14:30.833591 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:14:30 crc kubenswrapper[5052]: I1203 12:14:30.855378 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:14:30 crc kubenswrapper[5052]: I1203 12:14:30.857534 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:14:30 crc kubenswrapper[5052]: I1203 12:14:30.858789 5052 scope.go:117] "RemoveContainer" containerID="21ba3a277b2c2e8c629ffb7b800e7a6375bea92da2c176f4c34f1d3393d5da4f" Dec 03 12:14:30 crc kubenswrapper[5052]: I1203 12:14:30.860201 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-api-config-data" Dec 03 12:14:30 crc kubenswrapper[5052]: I1203 12:14:30.860374 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-nova-internal-svc" Dec 03 12:14:30 crc kubenswrapper[5052]: I1203 12:14:30.866089 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-nova-public-svc" Dec 03 12:14:30 crc kubenswrapper[5052]: I1203 12:14:30.867642 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:14:30 crc kubenswrapper[5052]: I1203 12:14:30.886066 5052 scope.go:117] "RemoveContainer" containerID="d92cbc5bb31cf05026bada3e2ff528d9bffb62aee81e5876a53cd8465313f7d5" Dec 03 12:14:30 crc kubenswrapper[5052]: I1203 12:14:30.944681 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/1661efff-d6ce-479d-bfb9-90849fa6ff40-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"1661efff-d6ce-479d-bfb9-90849fa6ff40\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:14:30 crc kubenswrapper[5052]: I1203 12:14:30.944779 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/1661efff-d6ce-479d-bfb9-90849fa6ff40-log-httpd\") pod \"ceilometer-0\" (UID: \"1661efff-d6ce-479d-bfb9-90849fa6ff40\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:14:30 crc kubenswrapper[5052]: I1203 12:14:30.944855 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/1661efff-d6ce-479d-bfb9-90849fa6ff40-scripts\") pod \"ceilometer-0\" (UID: \"1661efff-d6ce-479d-bfb9-90849fa6ff40\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:14:30 crc kubenswrapper[5052]: I1203 12:14:30.944983 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zvhkx\" (UniqueName: \"kubernetes.io/projected/1661efff-d6ce-479d-bfb9-90849fa6ff40-kube-api-access-zvhkx\") pod \"ceilometer-0\" (UID: \"1661efff-d6ce-479d-bfb9-90849fa6ff40\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:14:30 crc kubenswrapper[5052]: I1203 12:14:30.945051 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1661efff-d6ce-479d-bfb9-90849fa6ff40-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"1661efff-d6ce-479d-bfb9-90849fa6ff40\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:14:30 crc kubenswrapper[5052]: I1203 12:14:30.945178 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/1661efff-d6ce-479d-bfb9-90849fa6ff40-config-data\") pod \"ceilometer-0\" (UID: \"1661efff-d6ce-479d-bfb9-90849fa6ff40\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:14:30 crc kubenswrapper[5052]: I1203 12:14:30.945219 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/1661efff-d6ce-479d-bfb9-90849fa6ff40-run-httpd\") pod \"ceilometer-0\" (UID: \"1661efff-d6ce-479d-bfb9-90849fa6ff40\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:14:30 crc kubenswrapper[5052]: I1203 12:14:30.945284 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/1661efff-d6ce-479d-bfb9-90849fa6ff40-ceilometer-tls-certs\") pod \"ceilometer-0\" (UID: \"1661efff-d6ce-479d-bfb9-90849fa6ff40\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:14:31 crc kubenswrapper[5052]: I1203 12:14:31.046692 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/1661efff-d6ce-479d-bfb9-90849fa6ff40-log-httpd\") pod \"ceilometer-0\" (UID: \"1661efff-d6ce-479d-bfb9-90849fa6ff40\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:14:31 crc kubenswrapper[5052]: I1203 12:14:31.046733 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/1661efff-d6ce-479d-bfb9-90849fa6ff40-scripts\") pod \"ceilometer-0\" (UID: \"1661efff-d6ce-479d-bfb9-90849fa6ff40\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:14:31 crc kubenswrapper[5052]: I1203 12:14:31.046787 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-zvhkx\" (UniqueName: \"kubernetes.io/projected/1661efff-d6ce-479d-bfb9-90849fa6ff40-kube-api-access-zvhkx\") pod \"ceilometer-0\" (UID: \"1661efff-d6ce-479d-bfb9-90849fa6ff40\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:14:31 crc kubenswrapper[5052]: I1203 12:14:31.046816 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1661efff-d6ce-479d-bfb9-90849fa6ff40-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"1661efff-d6ce-479d-bfb9-90849fa6ff40\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:14:31 crc kubenswrapper[5052]: I1203 12:14:31.046863 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/1661efff-d6ce-479d-bfb9-90849fa6ff40-config-data\") pod \"ceilometer-0\" (UID: \"1661efff-d6ce-479d-bfb9-90849fa6ff40\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:14:31 crc kubenswrapper[5052]: I1203 12:14:31.046882 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/1661efff-d6ce-479d-bfb9-90849fa6ff40-run-httpd\") pod \"ceilometer-0\" (UID: \"1661efff-d6ce-479d-bfb9-90849fa6ff40\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:14:31 crc kubenswrapper[5052]: I1203 12:14:31.046915 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/cb68adf1-26f2-4b14-9874-dde28826aba3-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"cb68adf1-26f2-4b14-9874-dde28826aba3\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:14:31 crc kubenswrapper[5052]: I1203 12:14:31.046937 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/1661efff-d6ce-479d-bfb9-90849fa6ff40-ceilometer-tls-certs\") pod \"ceilometer-0\" (UID: \"1661efff-d6ce-479d-bfb9-90849fa6ff40\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:14:31 crc kubenswrapper[5052]: I1203 12:14:31.046964 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/cb68adf1-26f2-4b14-9874-dde28826aba3-public-tls-certs\") pod \"nova-api-0\" (UID: \"cb68adf1-26f2-4b14-9874-dde28826aba3\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:14:31 crc kubenswrapper[5052]: I1203 12:14:31.046984 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/cb68adf1-26f2-4b14-9874-dde28826aba3-config-data\") pod \"nova-api-0\" (UID: \"cb68adf1-26f2-4b14-9874-dde28826aba3\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:14:31 crc kubenswrapper[5052]: I1203 12:14:31.047016 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/1661efff-d6ce-479d-bfb9-90849fa6ff40-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"1661efff-d6ce-479d-bfb9-90849fa6ff40\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:14:31 crc kubenswrapper[5052]: I1203 12:14:31.047031 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/cb68adf1-26f2-4b14-9874-dde28826aba3-logs\") pod \"nova-api-0\" (UID: \"cb68adf1-26f2-4b14-9874-dde28826aba3\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:14:31 crc kubenswrapper[5052]: I1203 12:14:31.047048 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/cb68adf1-26f2-4b14-9874-dde28826aba3-internal-tls-certs\") pod \"nova-api-0\" (UID: \"cb68adf1-26f2-4b14-9874-dde28826aba3\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:14:31 crc kubenswrapper[5052]: I1203 12:14:31.047093 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gvwsd\" (UniqueName: \"kubernetes.io/projected/cb68adf1-26f2-4b14-9874-dde28826aba3-kube-api-access-gvwsd\") pod \"nova-api-0\" (UID: \"cb68adf1-26f2-4b14-9874-dde28826aba3\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:14:31 crc kubenswrapper[5052]: I1203 12:14:31.047599 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/1661efff-d6ce-479d-bfb9-90849fa6ff40-log-httpd\") pod \"ceilometer-0\" (UID: \"1661efff-d6ce-479d-bfb9-90849fa6ff40\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:14:31 crc kubenswrapper[5052]: I1203 12:14:31.053451 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/1661efff-d6ce-479d-bfb9-90849fa6ff40-scripts\") pod \"ceilometer-0\" (UID: \"1661efff-d6ce-479d-bfb9-90849fa6ff40\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:14:31 crc kubenswrapper[5052]: I1203 12:14:31.053921 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/1661efff-d6ce-479d-bfb9-90849fa6ff40-ceilometer-tls-certs\") pod \"ceilometer-0\" (UID: \"1661efff-d6ce-479d-bfb9-90849fa6ff40\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:14:31 crc kubenswrapper[5052]: I1203 12:14:31.054161 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/1661efff-d6ce-479d-bfb9-90849fa6ff40-run-httpd\") pod \"ceilometer-0\" (UID: \"1661efff-d6ce-479d-bfb9-90849fa6ff40\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:14:31 crc kubenswrapper[5052]: I1203 12:14:31.055773 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/1661efff-d6ce-479d-bfb9-90849fa6ff40-config-data\") pod \"ceilometer-0\" (UID: \"1661efff-d6ce-479d-bfb9-90849fa6ff40\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:14:31 crc kubenswrapper[5052]: I1203 12:14:31.056175 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/1661efff-d6ce-479d-bfb9-90849fa6ff40-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"1661efff-d6ce-479d-bfb9-90849fa6ff40\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:14:31 crc kubenswrapper[5052]: I1203 12:14:31.060455 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1661efff-d6ce-479d-bfb9-90849fa6ff40-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"1661efff-d6ce-479d-bfb9-90849fa6ff40\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:14:31 crc kubenswrapper[5052]: I1203 12:14:31.072303 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-zvhkx\" (UniqueName: \"kubernetes.io/projected/1661efff-d6ce-479d-bfb9-90849fa6ff40-kube-api-access-zvhkx\") pod \"ceilometer-0\" (UID: \"1661efff-d6ce-479d-bfb9-90849fa6ff40\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:14:31 crc kubenswrapper[5052]: I1203 12:14:31.146423 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:14:31 crc kubenswrapper[5052]: I1203 12:14:31.148168 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-gvwsd\" (UniqueName: \"kubernetes.io/projected/cb68adf1-26f2-4b14-9874-dde28826aba3-kube-api-access-gvwsd\") pod \"nova-api-0\" (UID: \"cb68adf1-26f2-4b14-9874-dde28826aba3\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:14:31 crc kubenswrapper[5052]: I1203 12:14:31.148277 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/cb68adf1-26f2-4b14-9874-dde28826aba3-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"cb68adf1-26f2-4b14-9874-dde28826aba3\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:14:31 crc kubenswrapper[5052]: I1203 12:14:31.148303 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/cb68adf1-26f2-4b14-9874-dde28826aba3-public-tls-certs\") pod \"nova-api-0\" (UID: \"cb68adf1-26f2-4b14-9874-dde28826aba3\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:14:31 crc kubenswrapper[5052]: I1203 12:14:31.148324 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/cb68adf1-26f2-4b14-9874-dde28826aba3-config-data\") pod \"nova-api-0\" (UID: \"cb68adf1-26f2-4b14-9874-dde28826aba3\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:14:31 crc kubenswrapper[5052]: I1203 12:14:31.148352 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/cb68adf1-26f2-4b14-9874-dde28826aba3-logs\") pod \"nova-api-0\" (UID: \"cb68adf1-26f2-4b14-9874-dde28826aba3\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:14:31 crc kubenswrapper[5052]: I1203 12:14:31.148371 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/cb68adf1-26f2-4b14-9874-dde28826aba3-internal-tls-certs\") pod \"nova-api-0\" (UID: \"cb68adf1-26f2-4b14-9874-dde28826aba3\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:14:31 crc kubenswrapper[5052]: I1203 12:14:31.149070 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/cb68adf1-26f2-4b14-9874-dde28826aba3-logs\") pod \"nova-api-0\" (UID: \"cb68adf1-26f2-4b14-9874-dde28826aba3\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:14:31 crc kubenswrapper[5052]: I1203 12:14:31.152369 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/cb68adf1-26f2-4b14-9874-dde28826aba3-config-data\") pod \"nova-api-0\" (UID: \"cb68adf1-26f2-4b14-9874-dde28826aba3\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:14:31 crc kubenswrapper[5052]: I1203 12:14:31.153036 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/cb68adf1-26f2-4b14-9874-dde28826aba3-public-tls-certs\") pod \"nova-api-0\" (UID: \"cb68adf1-26f2-4b14-9874-dde28826aba3\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:14:31 crc kubenswrapper[5052]: I1203 12:14:31.153081 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/cb68adf1-26f2-4b14-9874-dde28826aba3-internal-tls-certs\") pod \"nova-api-0\" (UID: \"cb68adf1-26f2-4b14-9874-dde28826aba3\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:14:31 crc kubenswrapper[5052]: I1203 12:14:31.162219 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/cb68adf1-26f2-4b14-9874-dde28826aba3-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"cb68adf1-26f2-4b14-9874-dde28826aba3\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:14:31 crc kubenswrapper[5052]: I1203 12:14:31.170156 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-gvwsd\" (UniqueName: \"kubernetes.io/projected/cb68adf1-26f2-4b14-9874-dde28826aba3-kube-api-access-gvwsd\") pod \"nova-api-0\" (UID: \"cb68adf1-26f2-4b14-9874-dde28826aba3\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:14:31 crc kubenswrapper[5052]: I1203 12:14:31.180951 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:14:31 crc kubenswrapper[5052]: I1203 12:14:31.702640 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="da9be43d-a787-4511-9b0c-4b3f307cb891" path="/var/lib/kubelet/pods/da9be43d-a787-4511-9b0c-4b3f307cb891/volumes" Dec 03 12:14:31 crc kubenswrapper[5052]: I1203 12:14:31.704066 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ec4beccd-125d-45c0-979e-6870c39627ee" path="/var/lib/kubelet/pods/ec4beccd-125d-45c0-979e-6870c39627ee/volumes" Dec 03 12:14:31 crc kubenswrapper[5052]: I1203 12:14:31.720209 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:14:31 crc kubenswrapper[5052]: I1203 12:14:31.803868 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:14:32 crc kubenswrapper[5052]: I1203 12:14:32.727937 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"cb68adf1-26f2-4b14-9874-dde28826aba3","Type":"ContainerStarted","Data":"96dc968ef689e331a6df033c2d763c10c20b7c7df416d6abc28abb2477d6b9ee"} Dec 03 12:14:32 crc kubenswrapper[5052]: I1203 12:14:32.728198 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"cb68adf1-26f2-4b14-9874-dde28826aba3","Type":"ContainerStarted","Data":"df21ee50a92cecc1b709d001db731836cf35f7648b75dbb8f7cbdbbb13f8e727"} Dec 03 12:14:32 crc kubenswrapper[5052]: I1203 12:14:32.728210 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"cb68adf1-26f2-4b14-9874-dde28826aba3","Type":"ContainerStarted","Data":"e4952ddc111a31dd387626299292e99f5a098bbfd611559dcea41c8a62d58daf"} Dec 03 12:14:32 crc kubenswrapper[5052]: I1203 12:14:32.729846 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"1661efff-d6ce-479d-bfb9-90849fa6ff40","Type":"ContainerStarted","Data":"e964aba7c310e135f899bcb05dcd28df12a6cfb568f657650c49176388d578b0"} Dec 03 12:14:32 crc kubenswrapper[5052]: I1203 12:14:32.729941 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"1661efff-d6ce-479d-bfb9-90849fa6ff40","Type":"ContainerStarted","Data":"3b8b6d2940fd19f6bb32b188acbc0cebb086a70566ef76013495a68808a45efd"} Dec 03 12:14:32 crc kubenswrapper[5052]: I1203 12:14:32.757584 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-api-0" podStartSLOduration=2.757565745 podStartE2EDuration="2.757565745s" podCreationTimestamp="2025-12-03 12:14:30 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:14:32.743424611 +0000 UTC m=+3263.331666375" watchObservedRunningTime="2025-12-03 12:14:32.757565745 +0000 UTC m=+3263.345807509" Dec 03 12:14:33 crc kubenswrapper[5052]: I1203 12:14:33.750134 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"1661efff-d6ce-479d-bfb9-90849fa6ff40","Type":"ContainerStarted","Data":"95a8366b935cbf627521e68e3391e0f202e2b033969a53119fa8a475d79c1be3"} Dec 03 12:14:33 crc kubenswrapper[5052]: I1203 12:14:33.750781 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"1661efff-d6ce-479d-bfb9-90849fa6ff40","Type":"ContainerStarted","Data":"aee5b94189aa0133ba610d83708c280ddef3068baf105fae0fbe385e8bb3d748"} Dec 03 12:14:35 crc kubenswrapper[5052]: I1203 12:14:35.771425 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"1661efff-d6ce-479d-bfb9-90849fa6ff40","Type":"ContainerStarted","Data":"240bba2f32d305ed667a2f5bc674c5d7f4bda3e122633105398b9c8c9930cfb5"} Dec 03 12:14:35 crc kubenswrapper[5052]: I1203 12:14:35.772017 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:14:35 crc kubenswrapper[5052]: I1203 12:14:35.799328 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/ceilometer-0" podStartSLOduration=2.705677884 podStartE2EDuration="5.799313526s" podCreationTimestamp="2025-12-03 12:14:30 +0000 UTC" firstStartedPulling="2025-12-03 12:14:31.732687922 +0000 UTC m=+3262.320929686" lastFinishedPulling="2025-12-03 12:14:34.826323564 +0000 UTC m=+3265.414565328" observedRunningTime="2025-12-03 12:14:35.794374945 +0000 UTC m=+3266.382616709" watchObservedRunningTime="2025-12-03 12:14:35.799313526 +0000 UTC m=+3266.387555290" Dec 03 12:14:41 crc kubenswrapper[5052]: I1203 12:14:41.181378 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:14:41 crc kubenswrapper[5052]: I1203 12:14:41.182481 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:14:42 crc kubenswrapper[5052]: I1203 12:14:42.195696 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-api-0" podUID="cb68adf1-26f2-4b14-9874-dde28826aba3" containerName="nova-api-api" probeResult="failure" output="Get \"https://10.217.1.45:8774/\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Dec 03 12:14:42 crc kubenswrapper[5052]: I1203 12:14:42.195871 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-api-0" podUID="cb68adf1-26f2-4b14-9874-dde28826aba3" containerName="nova-api-log" probeResult="failure" output="Get \"https://10.217.1.45:8774/\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Dec 03 12:14:51 crc kubenswrapper[5052]: I1203 12:14:51.192706 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:14:51 crc kubenswrapper[5052]: I1203 12:14:51.193392 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:14:51 crc kubenswrapper[5052]: I1203 12:14:51.193814 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:14:51 crc kubenswrapper[5052]: I1203 12:14:51.193864 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:14:51 crc kubenswrapper[5052]: I1203 12:14:51.198614 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:14:51 crc kubenswrapper[5052]: I1203 12:14:51.200891 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:15:00 crc kubenswrapper[5052]: I1203 12:15:00.141392 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openshift-operator-lifecycle-manager/collect-profiles-29412735-8kbb9"] Dec 03 12:15:00 crc kubenswrapper[5052]: I1203 12:15:00.143205 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-operator-lifecycle-manager/collect-profiles-29412735-8kbb9" Dec 03 12:15:00 crc kubenswrapper[5052]: I1203 12:15:00.145022 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-operator-lifecycle-manager"/"collect-profiles-dockercfg-kzf4t" Dec 03 12:15:00 crc kubenswrapper[5052]: I1203 12:15:00.145381 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-operator-lifecycle-manager"/"collect-profiles-config" Dec 03 12:15:00 crc kubenswrapper[5052]: I1203 12:15:00.155966 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-operator-lifecycle-manager/collect-profiles-29412735-8kbb9"] Dec 03 12:15:00 crc kubenswrapper[5052]: I1203 12:15:00.198371 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f792d620-f1d1-472d-a013-9f1ae6930b0f-config-volume\") pod \"collect-profiles-29412735-8kbb9\" (UID: \"f792d620-f1d1-472d-a013-9f1ae6930b0f\") " pod="openshift-operator-lifecycle-manager/collect-profiles-29412735-8kbb9" Dec 03 12:15:00 crc kubenswrapper[5052]: I1203 12:15:00.198452 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xn8dk\" (UniqueName: \"kubernetes.io/projected/f792d620-f1d1-472d-a013-9f1ae6930b0f-kube-api-access-xn8dk\") pod \"collect-profiles-29412735-8kbb9\" (UID: \"f792d620-f1d1-472d-a013-9f1ae6930b0f\") " pod="openshift-operator-lifecycle-manager/collect-profiles-29412735-8kbb9" Dec 03 12:15:00 crc kubenswrapper[5052]: I1203 12:15:00.198690 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-volume\" (UniqueName: \"kubernetes.io/secret/f792d620-f1d1-472d-a013-9f1ae6930b0f-secret-volume\") pod \"collect-profiles-29412735-8kbb9\" (UID: \"f792d620-f1d1-472d-a013-9f1ae6930b0f\") " pod="openshift-operator-lifecycle-manager/collect-profiles-29412735-8kbb9" Dec 03 12:15:00 crc kubenswrapper[5052]: I1203 12:15:00.300787 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"secret-volume\" (UniqueName: \"kubernetes.io/secret/f792d620-f1d1-472d-a013-9f1ae6930b0f-secret-volume\") pod \"collect-profiles-29412735-8kbb9\" (UID: \"f792d620-f1d1-472d-a013-9f1ae6930b0f\") " pod="openshift-operator-lifecycle-manager/collect-profiles-29412735-8kbb9" Dec 03 12:15:00 crc kubenswrapper[5052]: I1203 12:15:00.300912 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f792d620-f1d1-472d-a013-9f1ae6930b0f-config-volume\") pod \"collect-profiles-29412735-8kbb9\" (UID: \"f792d620-f1d1-472d-a013-9f1ae6930b0f\") " pod="openshift-operator-lifecycle-manager/collect-profiles-29412735-8kbb9" Dec 03 12:15:00 crc kubenswrapper[5052]: I1203 12:15:00.300949 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-xn8dk\" (UniqueName: \"kubernetes.io/projected/f792d620-f1d1-472d-a013-9f1ae6930b0f-kube-api-access-xn8dk\") pod \"collect-profiles-29412735-8kbb9\" (UID: \"f792d620-f1d1-472d-a013-9f1ae6930b0f\") " pod="openshift-operator-lifecycle-manager/collect-profiles-29412735-8kbb9" Dec 03 12:15:00 crc kubenswrapper[5052]: I1203 12:15:00.301971 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f792d620-f1d1-472d-a013-9f1ae6930b0f-config-volume\") pod \"collect-profiles-29412735-8kbb9\" (UID: \"f792d620-f1d1-472d-a013-9f1ae6930b0f\") " pod="openshift-operator-lifecycle-manager/collect-profiles-29412735-8kbb9" Dec 03 12:15:00 crc kubenswrapper[5052]: I1203 12:15:00.310462 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"secret-volume\" (UniqueName: \"kubernetes.io/secret/f792d620-f1d1-472d-a013-9f1ae6930b0f-secret-volume\") pod \"collect-profiles-29412735-8kbb9\" (UID: \"f792d620-f1d1-472d-a013-9f1ae6930b0f\") " pod="openshift-operator-lifecycle-manager/collect-profiles-29412735-8kbb9" Dec 03 12:15:00 crc kubenswrapper[5052]: I1203 12:15:00.321020 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-xn8dk\" (UniqueName: \"kubernetes.io/projected/f792d620-f1d1-472d-a013-9f1ae6930b0f-kube-api-access-xn8dk\") pod \"collect-profiles-29412735-8kbb9\" (UID: \"f792d620-f1d1-472d-a013-9f1ae6930b0f\") " pod="openshift-operator-lifecycle-manager/collect-profiles-29412735-8kbb9" Dec 03 12:15:00 crc kubenswrapper[5052]: I1203 12:15:00.491109 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-operator-lifecycle-manager/collect-profiles-29412735-8kbb9" Dec 03 12:15:00 crc kubenswrapper[5052]: I1203 12:15:00.623405 5052 scope.go:117] "RemoveContainer" containerID="41d7b7e369d34dc3d1117c0153f51d17b6ab0a7309a0b40c692c80168a43bba0" Dec 03 12:15:00 crc kubenswrapper[5052]: I1203 12:15:00.676883 5052 scope.go:117] "RemoveContainer" containerID="1440892d9f916d23a5ce08a7caddfbd149760ed9506b5d350d942e0b4765bf28" Dec 03 12:15:00 crc kubenswrapper[5052]: I1203 12:15:00.968798 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-operator-lifecycle-manager/collect-profiles-29412735-8kbb9"] Dec 03 12:15:01 crc kubenswrapper[5052]: I1203 12:15:01.014339 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-operator-lifecycle-manager/collect-profiles-29412735-8kbb9" event={"ID":"f792d620-f1d1-472d-a013-9f1ae6930b0f","Type":"ContainerStarted","Data":"8b4ead24cf19521f059be05ae8db2efcf469e24d21b0acc5bfa64f869080c86f"} Dec 03 12:15:01 crc kubenswrapper[5052]: I1203 12:15:01.158951 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:15:02 crc kubenswrapper[5052]: I1203 12:15:02.024297 5052 generic.go:334] "Generic (PLEG): container finished" podID="f792d620-f1d1-472d-a013-9f1ae6930b0f" containerID="3e4782da22f3455b238f322c1423c58483f7f0f10c60f639fa7854c8bfb97b96" exitCode=0 Dec 03 12:15:02 crc kubenswrapper[5052]: I1203 12:15:02.024348 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-operator-lifecycle-manager/collect-profiles-29412735-8kbb9" event={"ID":"f792d620-f1d1-472d-a013-9f1ae6930b0f","Type":"ContainerDied","Data":"3e4782da22f3455b238f322c1423c58483f7f0f10c60f639fa7854c8bfb97b96"} Dec 03 12:15:03 crc kubenswrapper[5052]: I1203 12:15:03.410788 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-operator-lifecycle-manager/collect-profiles-29412735-8kbb9" Dec 03 12:15:03 crc kubenswrapper[5052]: I1203 12:15:03.461431 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f792d620-f1d1-472d-a013-9f1ae6930b0f-config-volume\") pod \"f792d620-f1d1-472d-a013-9f1ae6930b0f\" (UID: \"f792d620-f1d1-472d-a013-9f1ae6930b0f\") " Dec 03 12:15:03 crc kubenswrapper[5052]: I1203 12:15:03.461614 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xn8dk\" (UniqueName: \"kubernetes.io/projected/f792d620-f1d1-472d-a013-9f1ae6930b0f-kube-api-access-xn8dk\") pod \"f792d620-f1d1-472d-a013-9f1ae6930b0f\" (UID: \"f792d620-f1d1-472d-a013-9f1ae6930b0f\") " Dec 03 12:15:03 crc kubenswrapper[5052]: I1203 12:15:03.461653 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"secret-volume\" (UniqueName: \"kubernetes.io/secret/f792d620-f1d1-472d-a013-9f1ae6930b0f-secret-volume\") pod \"f792d620-f1d1-472d-a013-9f1ae6930b0f\" (UID: \"f792d620-f1d1-472d-a013-9f1ae6930b0f\") " Dec 03 12:15:03 crc kubenswrapper[5052]: I1203 12:15:03.463338 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f792d620-f1d1-472d-a013-9f1ae6930b0f-config-volume" (OuterVolumeSpecName: "config-volume") pod "f792d620-f1d1-472d-a013-9f1ae6930b0f" (UID: "f792d620-f1d1-472d-a013-9f1ae6930b0f"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:15:03 crc kubenswrapper[5052]: I1203 12:15:03.468723 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f792d620-f1d1-472d-a013-9f1ae6930b0f-secret-volume" (OuterVolumeSpecName: "secret-volume") pod "f792d620-f1d1-472d-a013-9f1ae6930b0f" (UID: "f792d620-f1d1-472d-a013-9f1ae6930b0f"). InnerVolumeSpecName "secret-volume". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:15:03 crc kubenswrapper[5052]: I1203 12:15:03.470771 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f792d620-f1d1-472d-a013-9f1ae6930b0f-kube-api-access-xn8dk" (OuterVolumeSpecName: "kube-api-access-xn8dk") pod "f792d620-f1d1-472d-a013-9f1ae6930b0f" (UID: "f792d620-f1d1-472d-a013-9f1ae6930b0f"). InnerVolumeSpecName "kube-api-access-xn8dk". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:15:03 crc kubenswrapper[5052]: I1203 12:15:03.563465 5052 reconciler_common.go:293] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f792d620-f1d1-472d-a013-9f1ae6930b0f-config-volume\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:03 crc kubenswrapper[5052]: I1203 12:15:03.563711 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-xn8dk\" (UniqueName: \"kubernetes.io/projected/f792d620-f1d1-472d-a013-9f1ae6930b0f-kube-api-access-xn8dk\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:03 crc kubenswrapper[5052]: I1203 12:15:03.563788 5052 reconciler_common.go:293] "Volume detached for volume \"secret-volume\" (UniqueName: \"kubernetes.io/secret/f792d620-f1d1-472d-a013-9f1ae6930b0f-secret-volume\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:04 crc kubenswrapper[5052]: I1203 12:15:04.044497 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-operator-lifecycle-manager/collect-profiles-29412735-8kbb9" event={"ID":"f792d620-f1d1-472d-a013-9f1ae6930b0f","Type":"ContainerDied","Data":"8b4ead24cf19521f059be05ae8db2efcf469e24d21b0acc5bfa64f869080c86f"} Dec 03 12:15:04 crc kubenswrapper[5052]: I1203 12:15:04.044561 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8b4ead24cf19521f059be05ae8db2efcf469e24d21b0acc5bfa64f869080c86f" Dec 03 12:15:04 crc kubenswrapper[5052]: I1203 12:15:04.044632 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-operator-lifecycle-manager/collect-profiles-29412735-8kbb9" Dec 03 12:15:04 crc kubenswrapper[5052]: I1203 12:15:04.392809 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/openstackclient"] Dec 03 12:15:04 crc kubenswrapper[5052]: I1203 12:15:04.393345 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/openstackclient" podUID="4e574f0c-bbe2-44a9-935c-923622c44443" containerName="openstackclient" containerID="cri-o://27087053cd65b0655720b99b7846639054cddedb51bef7c1cc746fabc39c1401" gracePeriod=2 Dec 03 12:15:04 crc kubenswrapper[5052]: I1203 12:15:04.401689 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/openstackclient"] Dec 03 12:15:04 crc kubenswrapper[5052]: I1203 12:15:04.558347 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/rabbitmq-cell1-server-0"] Dec 03 12:15:04 crc kubenswrapper[5052]: E1203 12:15:04.793195 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/rabbitmq-cell1-config-data: configmap "rabbitmq-cell1-config-data" not found Dec 03 12:15:04 crc kubenswrapper[5052]: E1203 12:15:04.793254 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e63eb1eb-e0b9-46ea-a19b-8f5c018728ff-config-data podName:e63eb1eb-e0b9-46ea-a19b-8f5c018728ff nodeName:}" failed. No retries permitted until 2025-12-03 12:15:05.29323743 +0000 UTC m=+3295.881479194 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "config-data" (UniqueName: "kubernetes.io/configmap/e63eb1eb-e0b9-46ea-a19b-8f5c018728ff-config-data") pod "rabbitmq-cell1-server-0" (UID: "e63eb1eb-e0b9-46ea-a19b-8f5c018728ff") : configmap "rabbitmq-cell1-config-data" not found Dec 03 12:15:05 crc kubenswrapper[5052]: I1203 12:15:05.096616 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/rabbitmq-server-0"] Dec 03 12:15:05 crc kubenswrapper[5052]: I1203 12:15:05.164354 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/placement7cee-account-delete-7wgkx"] Dec 03 12:15:05 crc kubenswrapper[5052]: E1203 12:15:05.165719 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="4e574f0c-bbe2-44a9-935c-923622c44443" containerName="openstackclient" Dec 03 12:15:05 crc kubenswrapper[5052]: I1203 12:15:05.165878 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e574f0c-bbe2-44a9-935c-923622c44443" containerName="openstackclient" Dec 03 12:15:05 crc kubenswrapper[5052]: E1203 12:15:05.166003 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="f792d620-f1d1-472d-a013-9f1ae6930b0f" containerName="collect-profiles" Dec 03 12:15:05 crc kubenswrapper[5052]: I1203 12:15:05.166076 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="f792d620-f1d1-472d-a013-9f1ae6930b0f" containerName="collect-profiles" Dec 03 12:15:05 crc kubenswrapper[5052]: I1203 12:15:05.166377 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="f792d620-f1d1-472d-a013-9f1ae6930b0f" containerName="collect-profiles" Dec 03 12:15:05 crc kubenswrapper[5052]: I1203 12:15:05.166488 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="4e574f0c-bbe2-44a9-935c-923622c44443" containerName="openstackclient" Dec 03 12:15:05 crc kubenswrapper[5052]: I1203 12:15:05.167199 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement7cee-account-delete-7wgkx" Dec 03 12:15:05 crc kubenswrapper[5052]: E1203 12:15:05.199896 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/rabbitmq-config-data: configmap "rabbitmq-config-data" not found Dec 03 12:15:05 crc kubenswrapper[5052]: E1203 12:15:05.199960 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f-config-data podName:3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f nodeName:}" failed. No retries permitted until 2025-12-03 12:15:05.699947122 +0000 UTC m=+3296.288188886 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "config-data" (UniqueName: "kubernetes.io/configmap/3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f-config-data") pod "rabbitmq-server-0" (UID: "3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f") : configmap "rabbitmq-config-data" not found Dec 03 12:15:05 crc kubenswrapper[5052]: I1203 12:15:05.208410 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/placement7cee-account-delete-7wgkx"] Dec 03 12:15:05 crc kubenswrapper[5052]: I1203 12:15:05.306319 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-skrwg\" (UniqueName: \"kubernetes.io/projected/6099fea1-bbb4-4413-a826-539b3dfc3eec-kube-api-access-skrwg\") pod \"placement7cee-account-delete-7wgkx\" (UID: \"6099fea1-bbb4-4413-a826-539b3dfc3eec\") " pod="openstack-kuttl-tests/placement7cee-account-delete-7wgkx" Dec 03 12:15:05 crc kubenswrapper[5052]: I1203 12:15:05.306584 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/6099fea1-bbb4-4413-a826-539b3dfc3eec-operator-scripts\") pod \"placement7cee-account-delete-7wgkx\" (UID: \"6099fea1-bbb4-4413-a826-539b3dfc3eec\") " pod="openstack-kuttl-tests/placement7cee-account-delete-7wgkx" Dec 03 12:15:05 crc kubenswrapper[5052]: E1203 12:15:05.306842 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/rabbitmq-cell1-config-data: configmap "rabbitmq-cell1-config-data" not found Dec 03 12:15:05 crc kubenswrapper[5052]: E1203 12:15:05.306953 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e63eb1eb-e0b9-46ea-a19b-8f5c018728ff-config-data podName:e63eb1eb-e0b9-46ea-a19b-8f5c018728ff nodeName:}" failed. No retries permitted until 2025-12-03 12:15:06.306931207 +0000 UTC m=+3296.895172961 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "config-data" (UniqueName: "kubernetes.io/configmap/e63eb1eb-e0b9-46ea-a19b-8f5c018728ff-config-data") pod "rabbitmq-cell1-server-0" (UID: "e63eb1eb-e0b9-46ea-a19b-8f5c018728ff") : configmap "rabbitmq-cell1-config-data" not found Dec 03 12:15:05 crc kubenswrapper[5052]: I1203 12:15:05.310121 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-sb-0"] Dec 03 12:15:05 crc kubenswrapper[5052]: I1203 12:15:05.310917 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ovsdbserver-sb-0" podUID="baa1c9d4-5887-47e3-be99-96f718178f58" containerName="openstack-network-exporter" containerID="cri-o://b8259e68d536f622f925db84f9b7dd7940949212302ab0a104352098a97b2889" gracePeriod=300 Dec 03 12:15:05 crc kubenswrapper[5052]: I1203 12:15:05.361535 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-nb-0"] Dec 03 12:15:05 crc kubenswrapper[5052]: I1203 12:15:05.362376 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ovsdbserver-nb-0" podUID="0a2de558-2a4b-46ef-9b53-cada8cd466c4" containerName="openstack-network-exporter" containerID="cri-o://210f225008eb18ae291abefbe19ad7b03eaf69b1879a6f6bbe585e72babe1848" gracePeriod=300 Dec 03 12:15:05 crc kubenswrapper[5052]: I1203 12:15:05.404756 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ovn-northd-0"] Dec 03 12:15:05 crc kubenswrapper[5052]: I1203 12:15:05.405120 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ovn-northd-0" podUID="00ab3478-6308-427a-8e34-4133611a90b2" containerName="ovn-northd" containerID="cri-o://98d14ce0cb9919bb83964fb391e45a00edf1d049296adef59b4e2e0f042c63d2" gracePeriod=30 Dec 03 12:15:05 crc kubenswrapper[5052]: I1203 12:15:05.405307 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ovn-northd-0" podUID="00ab3478-6308-427a-8e34-4133611a90b2" containerName="openstack-network-exporter" containerID="cri-o://b77011f57a038af73fce8dd2b2cefedfa4b2e4023e9b43da72366d20be9344f4" gracePeriod=30 Dec 03 12:15:05 crc kubenswrapper[5052]: I1203 12:15:05.409971 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-skrwg\" (UniqueName: \"kubernetes.io/projected/6099fea1-bbb4-4413-a826-539b3dfc3eec-kube-api-access-skrwg\") pod \"placement7cee-account-delete-7wgkx\" (UID: \"6099fea1-bbb4-4413-a826-539b3dfc3eec\") " pod="openstack-kuttl-tests/placement7cee-account-delete-7wgkx" Dec 03 12:15:05 crc kubenswrapper[5052]: I1203 12:15:05.410670 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/6099fea1-bbb4-4413-a826-539b3dfc3eec-operator-scripts\") pod \"placement7cee-account-delete-7wgkx\" (UID: \"6099fea1-bbb4-4413-a826-539b3dfc3eec\") " pod="openstack-kuttl-tests/placement7cee-account-delete-7wgkx" Dec 03 12:15:05 crc kubenswrapper[5052]: I1203 12:15:05.412185 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/6099fea1-bbb4-4413-a826-539b3dfc3eec-operator-scripts\") pod \"placement7cee-account-delete-7wgkx\" (UID: \"6099fea1-bbb4-4413-a826-539b3dfc3eec\") " pod="openstack-kuttl-tests/placement7cee-account-delete-7wgkx" Dec 03 12:15:05 crc kubenswrapper[5052]: I1203 12:15:05.459047 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-skrwg\" (UniqueName: \"kubernetes.io/projected/6099fea1-bbb4-4413-a826-539b3dfc3eec-kube-api-access-skrwg\") pod \"placement7cee-account-delete-7wgkx\" (UID: \"6099fea1-bbb4-4413-a826-539b3dfc3eec\") " pod="openstack-kuttl-tests/placement7cee-account-delete-7wgkx" Dec 03 12:15:05 crc kubenswrapper[5052]: I1203 12:15:05.465358 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/neutron1287-account-delete-t8dcs"] Dec 03 12:15:05 crc kubenswrapper[5052]: I1203 12:15:05.466582 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron1287-account-delete-t8dcs" Dec 03 12:15:05 crc kubenswrapper[5052]: I1203 12:15:05.492336 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openshift-operator-lifecycle-manager/collect-profiles-29412690-ql6kh"] Dec 03 12:15:05 crc kubenswrapper[5052]: I1203 12:15:05.510084 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/placement-db-sync-xr2pm"] Dec 03 12:15:05 crc kubenswrapper[5052]: I1203 12:15:05.513338 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/f1619fbf-df68-484f-99d4-09b81a7a334e-operator-scripts\") pod \"neutron1287-account-delete-t8dcs\" (UID: \"f1619fbf-df68-484f-99d4-09b81a7a334e\") " pod="openstack-kuttl-tests/neutron1287-account-delete-t8dcs" Dec 03 12:15:05 crc kubenswrapper[5052]: I1203 12:15:05.513528 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement7cee-account-delete-7wgkx" Dec 03 12:15:05 crc kubenswrapper[5052]: I1203 12:15:05.516905 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xxqfp\" (UniqueName: \"kubernetes.io/projected/f1619fbf-df68-484f-99d4-09b81a7a334e-kube-api-access-xxqfp\") pod \"neutron1287-account-delete-t8dcs\" (UID: \"f1619fbf-df68-484f-99d4-09b81a7a334e\") " pod="openstack-kuttl-tests/neutron1287-account-delete-t8dcs" Dec 03 12:15:05 crc kubenswrapper[5052]: I1203 12:15:05.527829 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/placement-db-sync-xr2pm"] Dec 03 12:15:05 crc kubenswrapper[5052]: I1203 12:15:05.545572 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openshift-operator-lifecycle-manager/collect-profiles-29412690-ql6kh"] Dec 03 12:15:06 crc kubenswrapper[5052]: I1203 12:15:05.620605 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/f1619fbf-df68-484f-99d4-09b81a7a334e-operator-scripts\") pod \"neutron1287-account-delete-t8dcs\" (UID: \"f1619fbf-df68-484f-99d4-09b81a7a334e\") " pod="openstack-kuttl-tests/neutron1287-account-delete-t8dcs" Dec 03 12:15:06 crc kubenswrapper[5052]: I1203 12:15:05.620807 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-xxqfp\" (UniqueName: \"kubernetes.io/projected/f1619fbf-df68-484f-99d4-09b81a7a334e-kube-api-access-xxqfp\") pod \"neutron1287-account-delete-t8dcs\" (UID: \"f1619fbf-df68-484f-99d4-09b81a7a334e\") " pod="openstack-kuttl-tests/neutron1287-account-delete-t8dcs" Dec 03 12:15:06 crc kubenswrapper[5052]: I1203 12:15:05.622388 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/f1619fbf-df68-484f-99d4-09b81a7a334e-operator-scripts\") pod \"neutron1287-account-delete-t8dcs\" (UID: \"f1619fbf-df68-484f-99d4-09b81a7a334e\") " pod="openstack-kuttl-tests/neutron1287-account-delete-t8dcs" Dec 03 12:15:06 crc kubenswrapper[5052]: I1203 12:15:05.676326 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-xxqfp\" (UniqueName: \"kubernetes.io/projected/f1619fbf-df68-484f-99d4-09b81a7a334e-kube-api-access-xxqfp\") pod \"neutron1287-account-delete-t8dcs\" (UID: \"f1619fbf-df68-484f-99d4-09b81a7a334e\") " pod="openstack-kuttl-tests/neutron1287-account-delete-t8dcs" Dec 03 12:15:06 crc kubenswrapper[5052]: E1203 12:15:05.723925 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/rabbitmq-config-data: configmap "rabbitmq-config-data" not found Dec 03 12:15:06 crc kubenswrapper[5052]: E1203 12:15:05.730228 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f-config-data podName:3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f nodeName:}" failed. No retries permitted until 2025-12-03 12:15:06.730185102 +0000 UTC m=+3297.318426866 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "config-data" (UniqueName: "kubernetes.io/configmap/3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f-config-data") pod "rabbitmq-server-0" (UID: "3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f") : configmap "rabbitmq-config-data" not found Dec 03 12:15:06 crc kubenswrapper[5052]: E1203 12:15:05.760176 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="98d14ce0cb9919bb83964fb391e45a00edf1d049296adef59b4e2e0f042c63d2" cmd=["/usr/local/bin/container-scripts/status_check.sh"] Dec 03 12:15:06 crc kubenswrapper[5052]: E1203 12:15:05.798630 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="98d14ce0cb9919bb83964fb391e45a00edf1d049296adef59b4e2e0f042c63d2" cmd=["/usr/local/bin/container-scripts/status_check.sh"] Dec 03 12:15:06 crc kubenswrapper[5052]: E1203 12:15:05.819533 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="98d14ce0cb9919bb83964fb391e45a00edf1d049296adef59b4e2e0f042c63d2" cmd=["/usr/local/bin/container-scripts/status_check.sh"] Dec 03 12:15:06 crc kubenswrapper[5052]: E1203 12:15:05.819582 5052 prober.go:104] "Probe errored" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" probeType="Readiness" pod="openstack-kuttl-tests/ovn-northd-0" podUID="00ab3478-6308-427a-8e34-4133611a90b2" containerName="ovn-northd" Dec 03 12:15:06 crc kubenswrapper[5052]: I1203 12:15:05.942020 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ovsdbserver-sb-0" podUID="baa1c9d4-5887-47e3-be99-96f718178f58" containerName="ovsdbserver-sb" containerID="cri-o://7d5cb223b2febf0f7150eb3ecf28694a92fc2e544ffbcbf9721f02dc30547b38" gracePeriod=300 Dec 03 12:15:06 crc kubenswrapper[5052]: I1203 12:15:06.120474 5052 generic.go:334] "Generic (PLEG): container finished" podID="0a2de558-2a4b-46ef-9b53-cada8cd466c4" containerID="210f225008eb18ae291abefbe19ad7b03eaf69b1879a6f6bbe585e72babe1848" exitCode=2 Dec 03 12:15:06 crc kubenswrapper[5052]: I1203 12:15:06.141897 5052 generic.go:334] "Generic (PLEG): container finished" podID="00ab3478-6308-427a-8e34-4133611a90b2" containerID="b77011f57a038af73fce8dd2b2cefedfa4b2e4023e9b43da72366d20be9344f4" exitCode=2 Dec 03 12:15:06 crc kubenswrapper[5052]: I1203 12:15:06.157837 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_ovsdbserver-sb-0_baa1c9d4-5887-47e3-be99-96f718178f58/ovsdbserver-sb/0.log" Dec 03 12:15:06 crc kubenswrapper[5052]: I1203 12:15:06.157876 5052 generic.go:334] "Generic (PLEG): container finished" podID="baa1c9d4-5887-47e3-be99-96f718178f58" containerID="b8259e68d536f622f925db84f9b7dd7940949212302ab0a104352098a97b2889" exitCode=2 Dec 03 12:15:06 crc kubenswrapper[5052]: I1203 12:15:06.157890 5052 generic.go:334] "Generic (PLEG): container finished" podID="baa1c9d4-5887-47e3-be99-96f718178f58" containerID="7d5cb223b2febf0f7150eb3ecf28694a92fc2e544ffbcbf9721f02dc30547b38" exitCode=143 Dec 03 12:15:06 crc kubenswrapper[5052]: E1203 12:15:06.188768 5052 log.go:32] "ExecSync cmd from runtime service failed" err=< Dec 03 12:15:06 crc kubenswrapper[5052]: rpc error: code = Unknown desc = command error: setns `mnt`: Bad file descriptor Dec 03 12:15:06 crc kubenswrapper[5052]: fail startup Dec 03 12:15:06 crc kubenswrapper[5052]: , stdout: , stderr: , exit code -1 Dec 03 12:15:06 crc kubenswrapper[5052]: > containerID="b53a930fff1916d446b8a6d1e63e019c2f77d3ae6348e99f24ce01cb0d9242b1" cmd=["/usr/bin/pidof","ovsdb-server"] Dec 03 12:15:06 crc kubenswrapper[5052]: I1203 12:15:06.188900 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ovsdbserver-nb-0" podUID="0a2de558-2a4b-46ef-9b53-cada8cd466c4" containerName="ovsdbserver-nb" containerID="cri-o://b53a930fff1916d446b8a6d1e63e019c2f77d3ae6348e99f24ce01cb0d9242b1" gracePeriod=300 Dec 03 12:15:06 crc kubenswrapper[5052]: E1203 12:15:06.190084 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of b53a930fff1916d446b8a6d1e63e019c2f77d3ae6348e99f24ce01cb0d9242b1 is running failed: container process not found" containerID="b53a930fff1916d446b8a6d1e63e019c2f77d3ae6348e99f24ce01cb0d9242b1" cmd=["/usr/bin/pidof","ovsdb-server"] Dec 03 12:15:06 crc kubenswrapper[5052]: E1203 12:15:06.190677 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of b53a930fff1916d446b8a6d1e63e019c2f77d3ae6348e99f24ce01cb0d9242b1 is running failed: container process not found" containerID="b53a930fff1916d446b8a6d1e63e019c2f77d3ae6348e99f24ce01cb0d9242b1" cmd=["/usr/bin/pidof","ovsdb-server"] Dec 03 12:15:06 crc kubenswrapper[5052]: E1203 12:15:06.190751 5052 prober.go:104] "Probe errored" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of b53a930fff1916d446b8a6d1e63e019c2f77d3ae6348e99f24ce01cb0d9242b1 is running failed: container process not found" probeType="Readiness" pod="openstack-kuttl-tests/ovsdbserver-nb-0" podUID="0a2de558-2a4b-46ef-9b53-cada8cd466c4" containerName="ovsdbserver-nb" Dec 03 12:15:06 crc kubenswrapper[5052]: I1203 12:15:06.223635 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1ae0ed52-3a96-42c1-97ed-ae40c39997e1" path="/var/lib/kubelet/pods/1ae0ed52-3a96-42c1-97ed-ae40c39997e1/volumes" Dec 03 12:15:06 crc kubenswrapper[5052]: I1203 12:15:06.224740 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="db210aa9-d72d-4114-945c-0aa978c27a18" path="/var/lib/kubelet/pods/db210aa9-d72d-4114-945c-0aa978c27a18/volumes" Dec 03 12:15:06 crc kubenswrapper[5052]: I1203 12:15:06.231920 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-nb-0" event={"ID":"0a2de558-2a4b-46ef-9b53-cada8cd466c4","Type":"ContainerDied","Data":"210f225008eb18ae291abefbe19ad7b03eaf69b1879a6f6bbe585e72babe1848"} Dec 03 12:15:06 crc kubenswrapper[5052]: I1203 12:15:06.231997 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron1287-account-delete-t8dcs"] Dec 03 12:15:06 crc kubenswrapper[5052]: I1203 12:15:06.232027 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovn-northd-0" event={"ID":"00ab3478-6308-427a-8e34-4133611a90b2","Type":"ContainerDied","Data":"b77011f57a038af73fce8dd2b2cefedfa4b2e4023e9b43da72366d20be9344f4"} Dec 03 12:15:06 crc kubenswrapper[5052]: I1203 12:15:06.232043 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-sb-0" event={"ID":"baa1c9d4-5887-47e3-be99-96f718178f58","Type":"ContainerDied","Data":"b8259e68d536f622f925db84f9b7dd7940949212302ab0a104352098a97b2889"} Dec 03 12:15:06 crc kubenswrapper[5052]: I1203 12:15:06.232059 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-sb-0" event={"ID":"baa1c9d4-5887-47e3-be99-96f718178f58","Type":"ContainerDied","Data":"7d5cb223b2febf0f7150eb3ecf28694a92fc2e544ffbcbf9721f02dc30547b38"} Dec 03 12:15:06 crc kubenswrapper[5052]: I1203 12:15:06.232074 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/neutron-db-sync-cbdcz"] Dec 03 12:15:06 crc kubenswrapper[5052]: I1203 12:15:06.232094 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/neutron-db-sync-cbdcz"] Dec 03 12:15:06 crc kubenswrapper[5052]: I1203 12:15:06.232115 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/barbican8992-account-delete-26lhs"] Dec 03 12:15:06 crc kubenswrapper[5052]: I1203 12:15:06.246417 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican8992-account-delete-26lhs"] Dec 03 12:15:06 crc kubenswrapper[5052]: I1203 12:15:06.246457 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-db-sync-9k6nv"] Dec 03 12:15:06 crc kubenswrapper[5052]: I1203 12:15:06.246468 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-db-sync-nt4p4"] Dec 03 12:15:06 crc kubenswrapper[5052]: I1203 12:15:06.246482 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/glance-db-sync-nt4p4"] Dec 03 12:15:06 crc kubenswrapper[5052]: I1203 12:15:06.246502 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/glancef253-account-delete-cxxlg"] Dec 03 12:15:06 crc kubenswrapper[5052]: I1203 12:15:06.247769 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/barbican-db-sync-9k6nv"] Dec 03 12:15:06 crc kubenswrapper[5052]: I1203 12:15:06.247807 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder-db-sync-tgpn8"] Dec 03 12:15:06 crc kubenswrapper[5052]: I1203 12:15:06.247818 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/cinder-db-sync-tgpn8"] Dec 03 12:15:06 crc kubenswrapper[5052]: I1203 12:15:06.247829 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glancef253-account-delete-cxxlg"] Dec 03 12:15:06 crc kubenswrapper[5052]: I1203 12:15:06.247840 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/cinder7878-account-delete-jwtvt"] Dec 03 12:15:06 crc kubenswrapper[5052]: I1203 12:15:06.248888 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/swift-ring-rebalance-zqqdm"] Dec 03 12:15:06 crc kubenswrapper[5052]: I1203 12:15:06.248907 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/swift-ring-rebalance-zqqdm"] Dec 03 12:15:06 crc kubenswrapper[5052]: I1203 12:15:06.248920 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder7878-account-delete-jwtvt"] Dec 03 12:15:06 crc kubenswrapper[5052]: I1203 12:15:06.248929 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/rabbitmq-cell1-server-0"] Dec 03 12:15:06 crc kubenswrapper[5052]: I1203 12:15:06.248941 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/placement-79d5cb4fdb-7h2fn"] Dec 03 12:15:06 crc kubenswrapper[5052]: I1203 12:15:06.248952 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/swift-storage-0"] Dec 03 12:15:06 crc kubenswrapper[5052]: I1203 12:15:06.248965 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/neutron-5b979cdcb6-6hqqv"] Dec 03 12:15:06 crc kubenswrapper[5052]: I1203 12:15:06.248975 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell1-cell-mapping-tb27n"] Dec 03 12:15:06 crc kubenswrapper[5052]: I1203 12:15:06.249216 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/neutron-5b979cdcb6-6hqqv" podUID="718e2899-d5c3-41a4-a2f9-4830e18355d3" containerName="neutron-api" containerID="cri-o://8b2970b007db09e4b4c0137ca8209b26be8338c0d78012060389d6232ffbb06a" gracePeriod=30 Dec 03 12:15:06 crc kubenswrapper[5052]: I1203 12:15:06.249412 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican8992-account-delete-26lhs" Dec 03 12:15:06 crc kubenswrapper[5052]: I1203 12:15:06.249616 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glancef253-account-delete-cxxlg" Dec 03 12:15:06 crc kubenswrapper[5052]: I1203 12:15:06.249697 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder7878-account-delete-jwtvt" Dec 03 12:15:06 crc kubenswrapper[5052]: I1203 12:15:06.254423 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/placement-79d5cb4fdb-7h2fn" podUID="8283de30-8976-47bd-bd09-aab53140f68d" containerName="placement-log" containerID="cri-o://2b2c845f4fc350fd56a0f9ca2b46def3a0f193a23ad1b9dd526750dac6bdc0e7" gracePeriod=30 Dec 03 12:15:06 crc kubenswrapper[5052]: I1203 12:15:06.255108 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="694f3a22-38d9-487d-b887-a915fa8f99b0" containerName="account-server" containerID="cri-o://52c9d3753f1d83dc61693ea154f30426c527a127d3d6f00ae6379a1cbd9e5899" gracePeriod=30 Dec 03 12:15:06 crc kubenswrapper[5052]: I1203 12:15:06.255720 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/placement-79d5cb4fdb-7h2fn" podUID="8283de30-8976-47bd-bd09-aab53140f68d" containerName="placement-api" containerID="cri-o://2814b190b744e9cf8edd55553cf57bc90eebd7268c2360b6efc06f7b369e975c" gracePeriod=30 Dec 03 12:15:06 crc kubenswrapper[5052]: I1203 12:15:06.255852 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="694f3a22-38d9-487d-b887-a915fa8f99b0" containerName="swift-recon-cron" containerID="cri-o://d7754f88aed158a69d3f255fe77907743b8120c66e854103907d2301b4d085dd" gracePeriod=30 Dec 03 12:15:06 crc kubenswrapper[5052]: I1203 12:15:06.256028 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="694f3a22-38d9-487d-b887-a915fa8f99b0" containerName="rsync" containerID="cri-o://4ca9b25da15ecf1176a45c4f622d417575fd72f1258e4941eaab46adc37d9954" gracePeriod=30 Dec 03 12:15:06 crc kubenswrapper[5052]: I1203 12:15:06.256190 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="694f3a22-38d9-487d-b887-a915fa8f99b0" containerName="object-expirer" containerID="cri-o://c8a4d6f709c57c2c8c86b3ead5f40b5eed433a688c9bda706475d1a83dc38289" gracePeriod=30 Dec 03 12:15:06 crc kubenswrapper[5052]: I1203 12:15:06.256332 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="694f3a22-38d9-487d-b887-a915fa8f99b0" containerName="object-updater" containerID="cri-o://2b4fde6dfdf88033dd51e40f592f3092bbadafedc12b51e3dac6d06c7be816e2" gracePeriod=30 Dec 03 12:15:06 crc kubenswrapper[5052]: I1203 12:15:06.256513 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="694f3a22-38d9-487d-b887-a915fa8f99b0" containerName="object-auditor" containerID="cri-o://d34ac8e54a2a8c78fedd6aae86ea4f7c01fa3fb75b7877784aed4c550716ec17" gracePeriod=30 Dec 03 12:15:06 crc kubenswrapper[5052]: I1203 12:15:06.257417 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="694f3a22-38d9-487d-b887-a915fa8f99b0" containerName="container-replicator" containerID="cri-o://4be1f7ec0e5ec2598abc5fccb1e3a336e6d2ce904a40e4470bfebd9a2432aa0d" gracePeriod=30 Dec 03 12:15:06 crc kubenswrapper[5052]: I1203 12:15:06.257738 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="694f3a22-38d9-487d-b887-a915fa8f99b0" containerName="object-replicator" containerID="cri-o://edccdb7e982ee726341d6181308a200c1c7f7e489199aae3ff7897c6686d1e53" gracePeriod=30 Dec 03 12:15:06 crc kubenswrapper[5052]: I1203 12:15:06.257870 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="694f3a22-38d9-487d-b887-a915fa8f99b0" containerName="object-server" containerID="cri-o://bfc910b016024c52ad21685e4f00e1ebf323cb412cbdac44e180ce0af3b5c974" gracePeriod=30 Dec 03 12:15:06 crc kubenswrapper[5052]: I1203 12:15:06.258008 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="694f3a22-38d9-487d-b887-a915fa8f99b0" containerName="container-updater" containerID="cri-o://3577cb2e84ebe92b200c0bf67af07582c963a380e9d15f90356b2162da1491e3" gracePeriod=30 Dec 03 12:15:06 crc kubenswrapper[5052]: I1203 12:15:06.258143 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="694f3a22-38d9-487d-b887-a915fa8f99b0" containerName="container-auditor" containerID="cri-o://a9d69281c8f09f62933866859d2a7f994441cb27231474ecb5c4087f61ef9e48" gracePeriod=30 Dec 03 12:15:06 crc kubenswrapper[5052]: I1203 12:15:06.258300 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="694f3a22-38d9-487d-b887-a915fa8f99b0" containerName="account-reaper" containerID="cri-o://9abaf8aac6af7b9078e0f7fa87225ebcfdfa4003e45df36fe94117f331cca24b" gracePeriod=30 Dec 03 12:15:06 crc kubenswrapper[5052]: I1203 12:15:06.258641 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="694f3a22-38d9-487d-b887-a915fa8f99b0" containerName="container-server" containerID="cri-o://ffacc5bf5e6a05d80223ac48d049c379298b3d4840c2186fb767cc10e1492580" gracePeriod=30 Dec 03 12:15:06 crc kubenswrapper[5052]: I1203 12:15:06.258813 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="694f3a22-38d9-487d-b887-a915fa8f99b0" containerName="account-auditor" containerID="cri-o://8de12c6937f84a484a57da6623c0b720805ae449cf86cbf7781810fff5f7dbd6" gracePeriod=30 Dec 03 12:15:06 crc kubenswrapper[5052]: I1203 12:15:06.258953 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="694f3a22-38d9-487d-b887-a915fa8f99b0" containerName="account-replicator" containerID="cri-o://daff85236f8b1825cad56f8cf5a450330943257c15e8a198c3cbd529f86943a9" gracePeriod=30 Dec 03 12:15:06 crc kubenswrapper[5052]: I1203 12:15:06.259094 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/neutron-5b979cdcb6-6hqqv" podUID="718e2899-d5c3-41a4-a2f9-4830e18355d3" containerName="neutron-httpd" containerID="cri-o://84e05639993606274e28568021cb8fdaf723254416f7db2a86eb1f0830df3b64" gracePeriod=30 Dec 03 12:15:06 crc kubenswrapper[5052]: I1203 12:15:06.300826 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell1-cell-mapping-tb27n"] Dec 03 12:15:06 crc kubenswrapper[5052]: I1203 12:15:06.343926 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/novacell1f1c3-account-delete-6zqsd"] Dec 03 12:15:06 crc kubenswrapper[5052]: I1203 12:15:06.344789 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kwtt5\" (UniqueName: \"kubernetes.io/projected/5856107a-0498-4956-beba-4390f5477ca0-kube-api-access-kwtt5\") pod \"barbican8992-account-delete-26lhs\" (UID: \"5856107a-0498-4956-beba-4390f5477ca0\") " pod="openstack-kuttl-tests/barbican8992-account-delete-26lhs" Dec 03 12:15:06 crc kubenswrapper[5052]: I1203 12:15:06.344897 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/fee7d732-4fa3-40d5-8fff-cb88272db0ce-operator-scripts\") pod \"glancef253-account-delete-cxxlg\" (UID: \"fee7d732-4fa3-40d5-8fff-cb88272db0ce\") " pod="openstack-kuttl-tests/glancef253-account-delete-cxxlg" Dec 03 12:15:06 crc kubenswrapper[5052]: I1203 12:15:06.344973 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/5856107a-0498-4956-beba-4390f5477ca0-operator-scripts\") pod \"barbican8992-account-delete-26lhs\" (UID: \"5856107a-0498-4956-beba-4390f5477ca0\") " pod="openstack-kuttl-tests/barbican8992-account-delete-26lhs" Dec 03 12:15:06 crc kubenswrapper[5052]: I1203 12:15:06.344998 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v2r2s\" (UniqueName: \"kubernetes.io/projected/679f17b9-0e40-480b-bfde-4f63899cb972-kube-api-access-v2r2s\") pod \"cinder7878-account-delete-jwtvt\" (UID: \"679f17b9-0e40-480b-bfde-4f63899cb972\") " pod="openstack-kuttl-tests/cinder7878-account-delete-jwtvt" Dec 03 12:15:06 crc kubenswrapper[5052]: E1203 12:15:06.345019 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/rabbitmq-cell1-config-data: configmap "rabbitmq-cell1-config-data" not found Dec 03 12:15:06 crc kubenswrapper[5052]: E1203 12:15:06.345115 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e63eb1eb-e0b9-46ea-a19b-8f5c018728ff-config-data podName:e63eb1eb-e0b9-46ea-a19b-8f5c018728ff nodeName:}" failed. No retries permitted until 2025-12-03 12:15:08.345092919 +0000 UTC m=+3298.933334743 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "config-data" (UniqueName: "kubernetes.io/configmap/e63eb1eb-e0b9-46ea-a19b-8f5c018728ff-config-data") pod "rabbitmq-cell1-server-0" (UID: "e63eb1eb-e0b9-46ea-a19b-8f5c018728ff") : configmap "rabbitmq-cell1-config-data" not found Dec 03 12:15:06 crc kubenswrapper[5052]: I1203 12:15:06.345043 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qwfl2\" (UniqueName: \"kubernetes.io/projected/fee7d732-4fa3-40d5-8fff-cb88272db0ce-kube-api-access-qwfl2\") pod \"glancef253-account-delete-cxxlg\" (UID: \"fee7d732-4fa3-40d5-8fff-cb88272db0ce\") " pod="openstack-kuttl-tests/glancef253-account-delete-cxxlg" Dec 03 12:15:06 crc kubenswrapper[5052]: I1203 12:15:06.345491 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/679f17b9-0e40-480b-bfde-4f63899cb972-operator-scripts\") pod \"cinder7878-account-delete-jwtvt\" (UID: \"679f17b9-0e40-480b-bfde-4f63899cb972\") " pod="openstack-kuttl-tests/cinder7878-account-delete-jwtvt" Dec 03 12:15:06 crc kubenswrapper[5052]: I1203 12:15:06.358769 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/novacell1f1c3-account-delete-6zqsd" Dec 03 12:15:06 crc kubenswrapper[5052]: I1203 12:15:06.372804 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/novacell1f1c3-account-delete-6zqsd"] Dec 03 12:15:06 crc kubenswrapper[5052]: I1203 12:15:06.384870 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell0-cell-mapping-ztgcf"] Dec 03 12:15:06 crc kubenswrapper[5052]: I1203 12:15:06.393437 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell0-cell-mapping-ztgcf"] Dec 03 12:15:06 crc kubenswrapper[5052]: I1203 12:15:06.405065 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-kl2c8"] Dec 03 12:15:06 crc kubenswrapper[5052]: I1203 12:15:06.406778 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-kl2c8" Dec 03 12:15:06 crc kubenswrapper[5052]: I1203 12:15:06.417995 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/novaapi06eb-account-delete-kvlp2"] Dec 03 12:15:06 crc kubenswrapper[5052]: I1203 12:15:06.421341 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/novaapi06eb-account-delete-kvlp2" Dec 03 12:15:06 crc kubenswrapper[5052]: I1203 12:15:06.431611 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-kl2c8"] Dec 03 12:15:06 crc kubenswrapper[5052]: I1203 12:15:06.445596 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/novaapi06eb-account-delete-kvlp2"] Dec 03 12:15:06 crc kubenswrapper[5052]: I1203 12:15:06.447415 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/5856107a-0498-4956-beba-4390f5477ca0-operator-scripts\") pod \"barbican8992-account-delete-26lhs\" (UID: \"5856107a-0498-4956-beba-4390f5477ca0\") " pod="openstack-kuttl-tests/barbican8992-account-delete-26lhs" Dec 03 12:15:06 crc kubenswrapper[5052]: I1203 12:15:06.447493 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-v2r2s\" (UniqueName: \"kubernetes.io/projected/679f17b9-0e40-480b-bfde-4f63899cb972-kube-api-access-v2r2s\") pod \"cinder7878-account-delete-jwtvt\" (UID: \"679f17b9-0e40-480b-bfde-4f63899cb972\") " pod="openstack-kuttl-tests/cinder7878-account-delete-jwtvt" Dec 03 12:15:06 crc kubenswrapper[5052]: I1203 12:15:06.448911 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-qwfl2\" (UniqueName: \"kubernetes.io/projected/fee7d732-4fa3-40d5-8fff-cb88272db0ce-kube-api-access-qwfl2\") pod \"glancef253-account-delete-cxxlg\" (UID: \"fee7d732-4fa3-40d5-8fff-cb88272db0ce\") " pod="openstack-kuttl-tests/glancef253-account-delete-cxxlg" Dec 03 12:15:06 crc kubenswrapper[5052]: I1203 12:15:06.449052 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/679f17b9-0e40-480b-bfde-4f63899cb972-operator-scripts\") pod \"cinder7878-account-delete-jwtvt\" (UID: \"679f17b9-0e40-480b-bfde-4f63899cb972\") " pod="openstack-kuttl-tests/cinder7878-account-delete-jwtvt" Dec 03 12:15:06 crc kubenswrapper[5052]: I1203 12:15:06.449269 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-kwtt5\" (UniqueName: \"kubernetes.io/projected/5856107a-0498-4956-beba-4390f5477ca0-kube-api-access-kwtt5\") pod \"barbican8992-account-delete-26lhs\" (UID: \"5856107a-0498-4956-beba-4390f5477ca0\") " pod="openstack-kuttl-tests/barbican8992-account-delete-26lhs" Dec 03 12:15:06 crc kubenswrapper[5052]: I1203 12:15:06.449430 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/fee7d732-4fa3-40d5-8fff-cb88272db0ce-operator-scripts\") pod \"glancef253-account-delete-cxxlg\" (UID: \"fee7d732-4fa3-40d5-8fff-cb88272db0ce\") " pod="openstack-kuttl-tests/glancef253-account-delete-cxxlg" Dec 03 12:15:06 crc kubenswrapper[5052]: I1203 12:15:06.450872 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/5856107a-0498-4956-beba-4390f5477ca0-operator-scripts\") pod \"barbican8992-account-delete-26lhs\" (UID: \"5856107a-0498-4956-beba-4390f5477ca0\") " pod="openstack-kuttl-tests/barbican8992-account-delete-26lhs" Dec 03 12:15:06 crc kubenswrapper[5052]: I1203 12:15:06.451154 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/fee7d732-4fa3-40d5-8fff-cb88272db0ce-operator-scripts\") pod \"glancef253-account-delete-cxxlg\" (UID: \"fee7d732-4fa3-40d5-8fff-cb88272db0ce\") " pod="openstack-kuttl-tests/glancef253-account-delete-cxxlg" Dec 03 12:15:06 crc kubenswrapper[5052]: I1203 12:15:06.453239 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/679f17b9-0e40-480b-bfde-4f63899cb972-operator-scripts\") pod \"cinder7878-account-delete-jwtvt\" (UID: \"679f17b9-0e40-480b-bfde-4f63899cb972\") " pod="openstack-kuttl-tests/cinder7878-account-delete-jwtvt" Dec 03 12:15:06 crc kubenswrapper[5052]: I1203 12:15:06.462701 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/novacell00bb5-account-delete-r2csc"] Dec 03 12:15:06 crc kubenswrapper[5052]: I1203 12:15:06.463628 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" podUID="e63eb1eb-e0b9-46ea-a19b-8f5c018728ff" containerName="rabbitmq" containerID="cri-o://114f5fc894c9a2d64f9ed86ad645c9cb54b0a4c854a1d0287b2c2a82e3627521" gracePeriod=604800 Dec 03 12:15:06 crc kubenswrapper[5052]: I1203 12:15:06.464290 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/novacell00bb5-account-delete-r2csc" Dec 03 12:15:06 crc kubenswrapper[5052]: I1203 12:15:06.474684 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-qwfl2\" (UniqueName: \"kubernetes.io/projected/fee7d732-4fa3-40d5-8fff-cb88272db0ce-kube-api-access-qwfl2\") pod \"glancef253-account-delete-cxxlg\" (UID: \"fee7d732-4fa3-40d5-8fff-cb88272db0ce\") " pod="openstack-kuttl-tests/glancef253-account-delete-cxxlg" Dec 03 12:15:06 crc kubenswrapper[5052]: I1203 12:15:06.475543 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-v2r2s\" (UniqueName: \"kubernetes.io/projected/679f17b9-0e40-480b-bfde-4f63899cb972-kube-api-access-v2r2s\") pod \"cinder7878-account-delete-jwtvt\" (UID: \"679f17b9-0e40-480b-bfde-4f63899cb972\") " pod="openstack-kuttl-tests/cinder7878-account-delete-jwtvt" Dec 03 12:15:06 crc kubenswrapper[5052]: I1203 12:15:06.485514 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/novacell00bb5-account-delete-r2csc"] Dec 03 12:15:06 crc kubenswrapper[5052]: I1203 12:15:06.498049 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/rabbitmq-server-0"] Dec 03 12:15:06 crc kubenswrapper[5052]: I1203 12:15:06.509393 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:15:06 crc kubenswrapper[5052]: I1203 12:15:06.509714 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-internal-api-0" podUID="d81b09dc-75a1-45f8-ace4-786531ad3bfd" containerName="glance-log" containerID="cri-o://be83ec436e5bb13951ed23e2fd4e1e84fd5f3b6ce1f0a6f46b5c44dcd3634ee4" gracePeriod=30 Dec 03 12:15:06 crc kubenswrapper[5052]: I1203 12:15:06.509877 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-internal-api-0" podUID="d81b09dc-75a1-45f8-ace4-786531ad3bfd" containerName="glance-httpd" containerID="cri-o://e29fe95d4add082f6c46305d5e3fe12f454dfccd4b1d5c2355d53782ac82fa29" gracePeriod=30 Dec 03 12:15:06 crc kubenswrapper[5052]: I1203 12:15:06.518037 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-kwtt5\" (UniqueName: \"kubernetes.io/projected/5856107a-0498-4956-beba-4390f5477ca0-kube-api-access-kwtt5\") pod \"barbican8992-account-delete-26lhs\" (UID: \"5856107a-0498-4956-beba-4390f5477ca0\") " pod="openstack-kuttl-tests/barbican8992-account-delete-26lhs" Dec 03 12:15:06 crc kubenswrapper[5052]: I1203 12:15:06.532834 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 12:15:06 crc kubenswrapper[5052]: I1203 12:15:06.533125 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/cinder-scheduler-0" podUID="89b056a7-e448-447d-be7d-b69f3d4862a1" containerName="cinder-scheduler" containerID="cri-o://6b5e16bdf4a9b75d7e426aac04d9e0f302e3e926580d5ea1db33ccce8cef8eb9" gracePeriod=30 Dec 03 12:15:06 crc kubenswrapper[5052]: I1203 12:15:06.533580 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/cinder-scheduler-0" podUID="89b056a7-e448-447d-be7d-b69f3d4862a1" containerName="probe" containerID="cri-o://ac8820d4342df6998e1122358c0ed26bd8debe54ca84f5a0d380bb533f8c00eb" gracePeriod=30 Dec 03 12:15:06 crc kubenswrapper[5052]: I1203 12:15:06.546527 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:15:06 crc kubenswrapper[5052]: I1203 12:15:06.546859 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-external-api-0" podUID="2a349843-a7b7-4dff-a5ae-f513337c5809" containerName="glance-log" containerID="cri-o://237be9059aceba08b75de8d8ecd8ee5c1eb28f5bd5c20bb5c20a33088beca081" gracePeriod=30 Dec 03 12:15:06 crc kubenswrapper[5052]: I1203 12:15:06.547361 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-external-api-0" podUID="2a349843-a7b7-4dff-a5ae-f513337c5809" containerName="glance-httpd" containerID="cri-o://f30fe921af5f7b637639949f8977c3720f1ebb48e3b4f39277fcbc953207de23" gracePeriod=30 Dec 03 12:15:06 crc kubenswrapper[5052]: I1203 12:15:06.578425 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/rabbitmq-server-0" podUID="3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f" containerName="rabbitmq" containerID="cri-o://353f13dfb90cbf0598b28536ad392c1afc49332edb49e65bfaddb985602e41d1" gracePeriod=604800 Dec 03 12:15:06 crc kubenswrapper[5052]: I1203 12:15:06.608644 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zwj9p\" (UniqueName: \"kubernetes.io/projected/b853a2c2-337f-423e-84bf-ea2814cf59da-kube-api-access-zwj9p\") pod \"novacell1f1c3-account-delete-6zqsd\" (UID: \"b853a2c2-337f-423e-84bf-ea2814cf59da\") " pod="openstack-kuttl-tests/novacell1f1c3-account-delete-6zqsd" Dec 03 12:15:06 crc kubenswrapper[5052]: I1203 12:15:06.608710 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t9b6r\" (UniqueName: \"kubernetes.io/projected/72a72aa7-f35b-4b95-b5fd-caaf1272943a-kube-api-access-t9b6r\") pod \"novaapi06eb-account-delete-kvlp2\" (UID: \"72a72aa7-f35b-4b95-b5fd-caaf1272943a\") " pod="openstack-kuttl-tests/novaapi06eb-account-delete-kvlp2" Dec 03 12:15:06 crc kubenswrapper[5052]: I1203 12:15:06.608769 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b7zkq\" (UniqueName: \"kubernetes.io/projected/c200c246-74d3-48dc-8b24-ac4b37611069-kube-api-access-b7zkq\") pod \"novacell00bb5-account-delete-r2csc\" (UID: \"c200c246-74d3-48dc-8b24-ac4b37611069\") " pod="openstack-kuttl-tests/novacell00bb5-account-delete-r2csc" Dec 03 12:15:06 crc kubenswrapper[5052]: I1203 12:15:06.608900 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/b853a2c2-337f-423e-84bf-ea2814cf59da-operator-scripts\") pod \"novacell1f1c3-account-delete-6zqsd\" (UID: \"b853a2c2-337f-423e-84bf-ea2814cf59da\") " pod="openstack-kuttl-tests/novacell1f1c3-account-delete-6zqsd" Dec 03 12:15:06 crc kubenswrapper[5052]: I1203 12:15:06.609005 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fknkw\" (UniqueName: \"kubernetes.io/projected/ebf52f1d-a87e-4300-a650-4c4cb36e80d2-kube-api-access-fknkw\") pod \"dnsmasq-dnsmasq-84b9f45d47-kl2c8\" (UID: \"ebf52f1d-a87e-4300-a650-4c4cb36e80d2\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-kl2c8" Dec 03 12:15:06 crc kubenswrapper[5052]: I1203 12:15:06.609101 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/ebf52f1d-a87e-4300-a650-4c4cb36e80d2-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-84b9f45d47-kl2c8\" (UID: \"ebf52f1d-a87e-4300-a650-4c4cb36e80d2\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-kl2c8" Dec 03 12:15:06 crc kubenswrapper[5052]: I1203 12:15:06.609171 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ebf52f1d-a87e-4300-a650-4c4cb36e80d2-config\") pod \"dnsmasq-dnsmasq-84b9f45d47-kl2c8\" (UID: \"ebf52f1d-a87e-4300-a650-4c4cb36e80d2\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-kl2c8" Dec 03 12:15:06 crc kubenswrapper[5052]: I1203 12:15:06.609331 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/c200c246-74d3-48dc-8b24-ac4b37611069-operator-scripts\") pod \"novacell00bb5-account-delete-r2csc\" (UID: \"c200c246-74d3-48dc-8b24-ac4b37611069\") " pod="openstack-kuttl-tests/novacell00bb5-account-delete-r2csc" Dec 03 12:15:06 crc kubenswrapper[5052]: I1203 12:15:06.609480 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/72a72aa7-f35b-4b95-b5fd-caaf1272943a-operator-scripts\") pod \"novaapi06eb-account-delete-kvlp2\" (UID: \"72a72aa7-f35b-4b95-b5fd-caaf1272943a\") " pod="openstack-kuttl-tests/novaapi06eb-account-delete-kvlp2" Dec 03 12:15:06 crc kubenswrapper[5052]: I1203 12:15:06.614373 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 12:15:06 crc kubenswrapper[5052]: I1203 12:15:06.614776 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/cinder-api-0" podUID="388bc903-097f-4f97-8f19-2f7bcdb49cc5" containerName="cinder-api-log" containerID="cri-o://c15684b97ff4a9d54a9eeca81697f4cced793ffe157adb3548bedd21bf3249d0" gracePeriod=30 Dec 03 12:15:06 crc kubenswrapper[5052]: I1203 12:15:06.614939 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/cinder-api-0" podUID="388bc903-097f-4f97-8f19-2f7bcdb49cc5" containerName="cinder-api" containerID="cri-o://7024c66624b9b95f71fa0589d2f50417eaa07d7b921b5356f06724331235a546" gracePeriod=30 Dec 03 12:15:06 crc kubenswrapper[5052]: I1203 12:15:06.645022 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-worker-6ff59c8c65-fsb2k"] Dec 03 12:15:06 crc kubenswrapper[5052]: I1203 12:15:06.645272 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/barbican-worker-6ff59c8c65-fsb2k" podUID="1c98e0a9-267b-46b6-be02-fbd163462940" containerName="barbican-worker-log" containerID="cri-o://ba1ddda28f438ca657b63e17167786d2db03d16bf5ae3221146d8c6f9efc02d5" gracePeriod=30 Dec 03 12:15:06 crc kubenswrapper[5052]: I1203 12:15:06.645394 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/barbican-worker-6ff59c8c65-fsb2k" podUID="1c98e0a9-267b-46b6-be02-fbd163462940" containerName="barbican-worker" containerID="cri-o://7c74e4821c6d3bec61337629bd8bf43a58b069eaa5ca6ceb459795bf5d1ec7ae" gracePeriod=30 Dec 03 12:15:06 crc kubenswrapper[5052]: I1203 12:15:06.691613 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-keystone-listener-85f96c8f8-b6m5f"] Dec 03 12:15:06 crc kubenswrapper[5052]: I1203 12:15:06.691929 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/barbican-keystone-listener-85f96c8f8-b6m5f" podUID="296a2858-22b8-41a4-91ac-a9bf055b2848" containerName="barbican-keystone-listener-log" containerID="cri-o://88ce6f1697038254e636269a53ace16da9bd7ec94e5543e34a4bab6f9c43e974" gracePeriod=30 Dec 03 12:15:06 crc kubenswrapper[5052]: I1203 12:15:06.692162 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/barbican-keystone-listener-85f96c8f8-b6m5f" podUID="296a2858-22b8-41a4-91ac-a9bf055b2848" containerName="barbican-keystone-listener" containerID="cri-o://dd65ec28df374ef84fa1a5f87d159edfa4265d92e582b66969d9f54e2878b585" gracePeriod=30 Dec 03 12:15:06 crc kubenswrapper[5052]: I1203 12:15:06.749092 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ebf52f1d-a87e-4300-a650-4c4cb36e80d2-config\") pod \"dnsmasq-dnsmasq-84b9f45d47-kl2c8\" (UID: \"ebf52f1d-a87e-4300-a650-4c4cb36e80d2\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-kl2c8" Dec 03 12:15:06 crc kubenswrapper[5052]: I1203 12:15:06.749255 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/c200c246-74d3-48dc-8b24-ac4b37611069-operator-scripts\") pod \"novacell00bb5-account-delete-r2csc\" (UID: \"c200c246-74d3-48dc-8b24-ac4b37611069\") " pod="openstack-kuttl-tests/novacell00bb5-account-delete-r2csc" Dec 03 12:15:06 crc kubenswrapper[5052]: I1203 12:15:06.749395 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/72a72aa7-f35b-4b95-b5fd-caaf1272943a-operator-scripts\") pod \"novaapi06eb-account-delete-kvlp2\" (UID: \"72a72aa7-f35b-4b95-b5fd-caaf1272943a\") " pod="openstack-kuttl-tests/novaapi06eb-account-delete-kvlp2" Dec 03 12:15:06 crc kubenswrapper[5052]: I1203 12:15:06.749466 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-t9b6r\" (UniqueName: \"kubernetes.io/projected/72a72aa7-f35b-4b95-b5fd-caaf1272943a-kube-api-access-t9b6r\") pod \"novaapi06eb-account-delete-kvlp2\" (UID: \"72a72aa7-f35b-4b95-b5fd-caaf1272943a\") " pod="openstack-kuttl-tests/novaapi06eb-account-delete-kvlp2" Dec 03 12:15:06 crc kubenswrapper[5052]: I1203 12:15:06.749487 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-zwj9p\" (UniqueName: \"kubernetes.io/projected/b853a2c2-337f-423e-84bf-ea2814cf59da-kube-api-access-zwj9p\") pod \"novacell1f1c3-account-delete-6zqsd\" (UID: \"b853a2c2-337f-423e-84bf-ea2814cf59da\") " pod="openstack-kuttl-tests/novacell1f1c3-account-delete-6zqsd" Dec 03 12:15:06 crc kubenswrapper[5052]: I1203 12:15:06.749545 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-b7zkq\" (UniqueName: \"kubernetes.io/projected/c200c246-74d3-48dc-8b24-ac4b37611069-kube-api-access-b7zkq\") pod \"novacell00bb5-account-delete-r2csc\" (UID: \"c200c246-74d3-48dc-8b24-ac4b37611069\") " pod="openstack-kuttl-tests/novacell00bb5-account-delete-r2csc" Dec 03 12:15:06 crc kubenswrapper[5052]: I1203 12:15:06.749685 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/b853a2c2-337f-423e-84bf-ea2814cf59da-operator-scripts\") pod \"novacell1f1c3-account-delete-6zqsd\" (UID: \"b853a2c2-337f-423e-84bf-ea2814cf59da\") " pod="openstack-kuttl-tests/novacell1f1c3-account-delete-6zqsd" Dec 03 12:15:06 crc kubenswrapper[5052]: I1203 12:15:06.749780 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-fknkw\" (UniqueName: \"kubernetes.io/projected/ebf52f1d-a87e-4300-a650-4c4cb36e80d2-kube-api-access-fknkw\") pod \"dnsmasq-dnsmasq-84b9f45d47-kl2c8\" (UID: \"ebf52f1d-a87e-4300-a650-4c4cb36e80d2\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-kl2c8" Dec 03 12:15:06 crc kubenswrapper[5052]: I1203 12:15:06.749862 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/ebf52f1d-a87e-4300-a650-4c4cb36e80d2-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-84b9f45d47-kl2c8\" (UID: \"ebf52f1d-a87e-4300-a650-4c4cb36e80d2\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-kl2c8" Dec 03 12:15:06 crc kubenswrapper[5052]: E1203 12:15:06.753122 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-cell1-scripts: configmap "openstack-cell1-scripts" not found Dec 03 12:15:06 crc kubenswrapper[5052]: E1203 12:15:06.753187 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b853a2c2-337f-423e-84bf-ea2814cf59da-operator-scripts podName:b853a2c2-337f-423e-84bf-ea2814cf59da nodeName:}" failed. No retries permitted until 2025-12-03 12:15:07.253162831 +0000 UTC m=+3297.841404595 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/b853a2c2-337f-423e-84bf-ea2814cf59da-operator-scripts") pod "novacell1f1c3-account-delete-6zqsd" (UID: "b853a2c2-337f-423e-84bf-ea2814cf59da") : configmap "openstack-cell1-scripts" not found Dec 03 12:15:06 crc kubenswrapper[5052]: E1203 12:15:06.753568 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/rabbitmq-config-data: configmap "rabbitmq-config-data" not found Dec 03 12:15:06 crc kubenswrapper[5052]: E1203 12:15:06.753599 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f-config-data podName:3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f nodeName:}" failed. No retries permitted until 2025-12-03 12:15:08.753586583 +0000 UTC m=+3299.341828347 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "config-data" (UniqueName: "kubernetes.io/configmap/3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f-config-data") pod "rabbitmq-server-0" (UID: "3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f") : configmap "rabbitmq-config-data" not found Dec 03 12:15:06 crc kubenswrapper[5052]: I1203 12:15:06.756350 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/c200c246-74d3-48dc-8b24-ac4b37611069-operator-scripts\") pod \"novacell00bb5-account-delete-r2csc\" (UID: \"c200c246-74d3-48dc-8b24-ac4b37611069\") " pod="openstack-kuttl-tests/novacell00bb5-account-delete-r2csc" Dec 03 12:15:06 crc kubenswrapper[5052]: I1203 12:15:06.757503 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/72a72aa7-f35b-4b95-b5fd-caaf1272943a-operator-scripts\") pod \"novaapi06eb-account-delete-kvlp2\" (UID: \"72a72aa7-f35b-4b95-b5fd-caaf1272943a\") " pod="openstack-kuttl-tests/novaapi06eb-account-delete-kvlp2" Dec 03 12:15:06 crc kubenswrapper[5052]: E1203 12:15:06.772280 5052 projected.go:194] Error preparing data for projected volume kube-api-access-zwj9p for pod openstack-kuttl-tests/novacell1f1c3-account-delete-6zqsd: failed to fetch token: serviceaccounts "galera-openstack-cell1" not found Dec 03 12:15:06 crc kubenswrapper[5052]: E1203 12:15:06.772355 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b853a2c2-337f-423e-84bf-ea2814cf59da-kube-api-access-zwj9p podName:b853a2c2-337f-423e-84bf-ea2814cf59da nodeName:}" failed. No retries permitted until 2025-12-03 12:15:07.272329448 +0000 UTC m=+3297.860571212 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-zwj9p" (UniqueName: "kubernetes.io/projected/b853a2c2-337f-423e-84bf-ea2814cf59da-kube-api-access-zwj9p") pod "novacell1f1c3-account-delete-6zqsd" (UID: "b853a2c2-337f-423e-84bf-ea2814cf59da") : failed to fetch token: serviceaccounts "galera-openstack-cell1" not found Dec 03 12:15:06 crc kubenswrapper[5052]: I1203 12:15:06.773696 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ebf52f1d-a87e-4300-a650-4c4cb36e80d2-config\") pod \"dnsmasq-dnsmasq-84b9f45d47-kl2c8\" (UID: \"ebf52f1d-a87e-4300-a650-4c4cb36e80d2\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-kl2c8" Dec 03 12:15:06 crc kubenswrapper[5052]: I1203 12:15:06.774086 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/ebf52f1d-a87e-4300-a650-4c4cb36e80d2-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-84b9f45d47-kl2c8\" (UID: \"ebf52f1d-a87e-4300-a650-4c4cb36e80d2\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-kl2c8" Dec 03 12:15:06 crc kubenswrapper[5052]: I1203 12:15:06.780656 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-b7zkq\" (UniqueName: \"kubernetes.io/projected/c200c246-74d3-48dc-8b24-ac4b37611069-kube-api-access-b7zkq\") pod \"novacell00bb5-account-delete-r2csc\" (UID: \"c200c246-74d3-48dc-8b24-ac4b37611069\") " pod="openstack-kuttl-tests/novacell00bb5-account-delete-r2csc" Dec 03 12:15:06 crc kubenswrapper[5052]: I1203 12:15:06.782923 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-fknkw\" (UniqueName: \"kubernetes.io/projected/ebf52f1d-a87e-4300-a650-4c4cb36e80d2-kube-api-access-fknkw\") pod \"dnsmasq-dnsmasq-84b9f45d47-kl2c8\" (UID: \"ebf52f1d-a87e-4300-a650-4c4cb36e80d2\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-kl2c8" Dec 03 12:15:06 crc kubenswrapper[5052]: I1203 12:15:06.788241 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-t9b6r\" (UniqueName: \"kubernetes.io/projected/72a72aa7-f35b-4b95-b5fd-caaf1272943a-kube-api-access-t9b6r\") pod \"novaapi06eb-account-delete-kvlp2\" (UID: \"72a72aa7-f35b-4b95-b5fd-caaf1272943a\") " pod="openstack-kuttl-tests/novaapi06eb-account-delete-kvlp2" Dec 03 12:15:06 crc kubenswrapper[5052]: I1203 12:15:06.802227 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:15:06 crc kubenswrapper[5052]: I1203 12:15:06.802439 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-metadata-0" podUID="cd486343-0aad-4039-a23b-34c60cd1a381" containerName="nova-metadata-log" containerID="cri-o://99335f1182dfcd9be7ce5b7f281d04f752ed33796c96039024fa49675032786b" gracePeriod=30 Dec 03 12:15:06 crc kubenswrapper[5052]: I1203 12:15:06.804698 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-metadata-0" podUID="cd486343-0aad-4039-a23b-34c60cd1a381" containerName="nova-metadata-metadata" containerID="cri-o://ec66627f7d357cab05559c6d34ef624f9d761ad2e4531d42cccb0b06d7bb2cba" gracePeriod=30 Dec 03 12:15:06 crc kubenswrapper[5052]: I1203 12:15:06.867395 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-api-58464649dd-sx8mw"] Dec 03 12:15:06 crc kubenswrapper[5052]: I1203 12:15:06.868047 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/barbican-api-58464649dd-sx8mw" podUID="b25d2a0c-1e93-4b3d-965a-a7c021f9649d" containerName="barbican-api-log" containerID="cri-o://6908da1783700654af60d79f711ce4360ed719f4f83031effe704c70de2ab47f" gracePeriod=30 Dec 03 12:15:06 crc kubenswrapper[5052]: I1203 12:15:06.870019 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/barbican-api-58464649dd-sx8mw" podUID="b25d2a0c-1e93-4b3d-965a-a7c021f9649d" containerName="barbican-api" containerID="cri-o://d4bb4f5c0829cb278a445845849850d561fa98757159ecf40155346ec8896bde" gracePeriod=30 Dec 03 12:15:06 crc kubenswrapper[5052]: I1203 12:15:06.876413 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:15:06 crc kubenswrapper[5052]: I1203 12:15:06.876703 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-api-0" podUID="cb68adf1-26f2-4b14-9874-dde28826aba3" containerName="nova-api-log" containerID="cri-o://df21ee50a92cecc1b709d001db731836cf35f7648b75dbb8f7cbdbbb13f8e727" gracePeriod=30 Dec 03 12:15:06 crc kubenswrapper[5052]: I1203 12:15:06.877044 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-api-0" podUID="cb68adf1-26f2-4b14-9874-dde28826aba3" containerName="nova-api-api" containerID="cri-o://96dc968ef689e331a6df033c2d763c10c20b7c7df416d6abc28abb2477d6b9ee" gracePeriod=30 Dec 03 12:15:06 crc kubenswrapper[5052]: I1203 12:15:06.951961 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/openstack-cell1-galera-0"] Dec 03 12:15:06 crc kubenswrapper[5052]: I1203 12:15:06.955717 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron1287-account-delete-t8dcs" Dec 03 12:15:06 crc kubenswrapper[5052]: W1203 12:15:06.973823 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6099fea1_bbb4_4413_a826_539b3dfc3eec.slice/crio-3ccafc1a9eceecb8999301b5f75d2abc1de50fd4b00703fc52a48fd5e6e875bf WatchSource:0}: Error finding container 3ccafc1a9eceecb8999301b5f75d2abc1de50fd4b00703fc52a48fd5e6e875bf: Status 404 returned error can't find the container with id 3ccafc1a9eceecb8999301b5f75d2abc1de50fd4b00703fc52a48fd5e6e875bf Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.002792 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican8992-account-delete-26lhs" Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.012743 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.013038 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" podUID="a15b6144-952c-4d41-a141-5bca5c7fd402" containerName="nova-cell1-novncproxy-novncproxy" containerID="cri-o://6b64e3ed870b9dc27d7d4df12ca7a0eb2dff4a0cc61910bf73a9e7a4b368dedc" gracePeriod=30 Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.022764 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell1-db-create-2mq6x"] Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.029796 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell1-db-create-2mq6x"] Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.032041 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glancef253-account-delete-cxxlg" Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.046133 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell1-f1c3-account-create-update-5mzt2"] Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.055280 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder7878-account-delete-jwtvt" Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.055655 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/novacell1f1c3-account-delete-6zqsd"] Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.063558 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell1-f1c3-account-create-update-5mzt2"] Dec 03 12:15:07 crc kubenswrapper[5052]: E1203 12:15:07.071640 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-zwj9p operator-scripts], unattached volumes=[], failed to process volumes=[]: context canceled" pod="openstack-kuttl-tests/novacell1f1c3-account-delete-6zqsd" podUID="b853a2c2-337f-423e-84bf-ea2814cf59da" Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.084144 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-kl2c8" Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.100233 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/novaapi06eb-account-delete-kvlp2" Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.109237 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/placement7cee-account-delete-7wgkx"] Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.117364 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.117746 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-scheduler-0" podUID="cf3ed088-9069-40cf-9c9e-ee9bdf7761d9" containerName="nova-scheduler-scheduler" containerID="cri-o://81879a070623f490e700ae5e038359faa2d3ce34a28b8f7e4b856f6c4f95fa7f" gracePeriod=30 Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.129250 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/novacell00bb5-account-delete-r2csc" Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.153598 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.229418 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_ovsdbserver-sb-0_baa1c9d4-5887-47e3-be99-96f718178f58/ovsdbserver-sb/0.log" Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.231441 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.235729 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstackclient" Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.241585 5052 generic.go:334] "Generic (PLEG): container finished" podID="cd486343-0aad-4039-a23b-34c60cd1a381" containerID="99335f1182dfcd9be7ce5b7f281d04f752ed33796c96039024fa49675032786b" exitCode=143 Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.241673 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"cd486343-0aad-4039-a23b-34c60cd1a381","Type":"ContainerDied","Data":"99335f1182dfcd9be7ce5b7f281d04f752ed33796c96039024fa49675032786b"} Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.280133 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/0a2de558-2a4b-46ef-9b53-cada8cd466c4-metrics-certs-tls-certs\") pod \"0a2de558-2a4b-46ef-9b53-cada8cd466c4\" (UID: \"0a2de558-2a4b-46ef-9b53-cada8cd466c4\") " Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.280214 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0a2de558-2a4b-46ef-9b53-cada8cd466c4-combined-ca-bundle\") pod \"0a2de558-2a4b-46ef-9b53-cada8cd466c4\" (UID: \"0a2de558-2a4b-46ef-9b53-cada8cd466c4\") " Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.280264 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovsdbserver-nb-tls-certs\" (UniqueName: \"kubernetes.io/secret/0a2de558-2a4b-46ef-9b53-cada8cd466c4-ovsdbserver-nb-tls-certs\") pod \"0a2de558-2a4b-46ef-9b53-cada8cd466c4\" (UID: \"0a2de558-2a4b-46ef-9b53-cada8cd466c4\") " Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.280370 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/0a2de558-2a4b-46ef-9b53-cada8cd466c4-ovsdb-rundir\") pod \"0a2de558-2a4b-46ef-9b53-cada8cd466c4\" (UID: \"0a2de558-2a4b-46ef-9b53-cada8cd466c4\") " Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.280504 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovndbcluster-nb-etc-ovn\" (UniqueName: \"kubernetes.io/local-volume/local-storage07-crc\") pod \"0a2de558-2a4b-46ef-9b53-cada8cd466c4\" (UID: \"0a2de558-2a4b-46ef-9b53-cada8cd466c4\") " Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.280554 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2x6wd\" (UniqueName: \"kubernetes.io/projected/0a2de558-2a4b-46ef-9b53-cada8cd466c4-kube-api-access-2x6wd\") pod \"0a2de558-2a4b-46ef-9b53-cada8cd466c4\" (UID: \"0a2de558-2a4b-46ef-9b53-cada8cd466c4\") " Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.280596 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0a2de558-2a4b-46ef-9b53-cada8cd466c4-config\") pod \"0a2de558-2a4b-46ef-9b53-cada8cd466c4\" (UID: \"0a2de558-2a4b-46ef-9b53-cada8cd466c4\") " Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.280674 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/0a2de558-2a4b-46ef-9b53-cada8cd466c4-scripts\") pod \"0a2de558-2a4b-46ef-9b53-cada8cd466c4\" (UID: \"0a2de558-2a4b-46ef-9b53-cada8cd466c4\") " Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.280874 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/openstack-cell1-galera-0" podUID="00a9935f-ef2d-4976-97a0-78ee6325433d" containerName="galera" containerID="cri-o://de161b1412d41d5ec322763c71b1221e12135067ced1a67a7f68542d387c4abb" gracePeriod=30 Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.281305 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a2de558-2a4b-46ef-9b53-cada8cd466c4-ovsdb-rundir" (OuterVolumeSpecName: "ovsdb-rundir") pod "0a2de558-2a4b-46ef-9b53-cada8cd466c4" (UID: "0a2de558-2a4b-46ef-9b53-cada8cd466c4"). InnerVolumeSpecName "ovsdb-rundir". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.281723 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0a2de558-2a4b-46ef-9b53-cada8cd466c4-config" (OuterVolumeSpecName: "config") pod "0a2de558-2a4b-46ef-9b53-cada8cd466c4" (UID: "0a2de558-2a4b-46ef-9b53-cada8cd466c4"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.282265 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-zwj9p\" (UniqueName: \"kubernetes.io/projected/b853a2c2-337f-423e-84bf-ea2814cf59da-kube-api-access-zwj9p\") pod \"novacell1f1c3-account-delete-6zqsd\" (UID: \"b853a2c2-337f-423e-84bf-ea2814cf59da\") " pod="openstack-kuttl-tests/novacell1f1c3-account-delete-6zqsd" Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.282378 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/b853a2c2-337f-423e-84bf-ea2814cf59da-operator-scripts\") pod \"novacell1f1c3-account-delete-6zqsd\" (UID: \"b853a2c2-337f-423e-84bf-ea2814cf59da\") " pod="openstack-kuttl-tests/novacell1f1c3-account-delete-6zqsd" Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.282617 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0a2de558-2a4b-46ef-9b53-cada8cd466c4-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.282639 5052 reconciler_common.go:293] "Volume detached for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/0a2de558-2a4b-46ef-9b53-cada8cd466c4-ovsdb-rundir\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:07 crc kubenswrapper[5052]: E1203 12:15:07.282912 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-cell1-scripts: configmap "openstack-cell1-scripts" not found Dec 03 12:15:07 crc kubenswrapper[5052]: E1203 12:15:07.282962 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b853a2c2-337f-423e-84bf-ea2814cf59da-operator-scripts podName:b853a2c2-337f-423e-84bf-ea2814cf59da nodeName:}" failed. No retries permitted until 2025-12-03 12:15:08.282946018 +0000 UTC m=+3298.871187782 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/b853a2c2-337f-423e-84bf-ea2814cf59da-operator-scripts") pod "novacell1f1c3-account-delete-6zqsd" (UID: "b853a2c2-337f-423e-84bf-ea2814cf59da") : configmap "openstack-cell1-scripts" not found Dec 03 12:15:07 crc kubenswrapper[5052]: E1203 12:15:07.287128 5052 projected.go:194] Error preparing data for projected volume kube-api-access-zwj9p for pod openstack-kuttl-tests/novacell1f1c3-account-delete-6zqsd: failed to fetch token: serviceaccounts "galera-openstack-cell1" not found Dec 03 12:15:07 crc kubenswrapper[5052]: E1203 12:15:07.287193 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b853a2c2-337f-423e-84bf-ea2814cf59da-kube-api-access-zwj9p podName:b853a2c2-337f-423e-84bf-ea2814cf59da nodeName:}" failed. No retries permitted until 2025-12-03 12:15:08.287175049 +0000 UTC m=+3298.875416803 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-zwj9p" (UniqueName: "kubernetes.io/projected/b853a2c2-337f-423e-84bf-ea2814cf59da-kube-api-access-zwj9p") pod "novacell1f1c3-account-delete-6zqsd" (UID: "b853a2c2-337f-423e-84bf-ea2814cf59da") : failed to fetch token: serviceaccounts "galera-openstack-cell1" not found Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.290024 5052 generic.go:334] "Generic (PLEG): container finished" podID="4e574f0c-bbe2-44a9-935c-923622c44443" containerID="27087053cd65b0655720b99b7846639054cddedb51bef7c1cc746fabc39c1401" exitCode=137 Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.290112 5052 scope.go:117] "RemoveContainer" containerID="27087053cd65b0655720b99b7846639054cddedb51bef7c1cc746fabc39c1401" Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.290247 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstackclient" Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.293253 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage07-crc" (OuterVolumeSpecName: "ovndbcluster-nb-etc-ovn") pod "0a2de558-2a4b-46ef-9b53-cada8cd466c4" (UID: "0a2de558-2a4b-46ef-9b53-cada8cd466c4"). InnerVolumeSpecName "local-storage07-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.318773 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0a2de558-2a4b-46ef-9b53-cada8cd466c4-kube-api-access-2x6wd" (OuterVolumeSpecName: "kube-api-access-2x6wd") pod "0a2de558-2a4b-46ef-9b53-cada8cd466c4" (UID: "0a2de558-2a4b-46ef-9b53-cada8cd466c4"). InnerVolumeSpecName "kube-api-access-2x6wd". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.320493 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement7cee-account-delete-7wgkx" event={"ID":"6099fea1-bbb4-4413-a826-539b3dfc3eec","Type":"ContainerStarted","Data":"3ccafc1a9eceecb8999301b5f75d2abc1de50fd4b00703fc52a48fd5e6e875bf"} Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.321081 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0a2de558-2a4b-46ef-9b53-cada8cd466c4-scripts" (OuterVolumeSpecName: "scripts") pod "0a2de558-2a4b-46ef-9b53-cada8cd466c4" (UID: "0a2de558-2a4b-46ef-9b53-cada8cd466c4"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.345767 5052 generic.go:334] "Generic (PLEG): container finished" podID="1c98e0a9-267b-46b6-be02-fbd163462940" containerID="ba1ddda28f438ca657b63e17167786d2db03d16bf5ae3221146d8c6f9efc02d5" exitCode=143 Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.345938 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-worker-6ff59c8c65-fsb2k" event={"ID":"1c98e0a9-267b-46b6-be02-fbd163462940","Type":"ContainerDied","Data":"ba1ddda28f438ca657b63e17167786d2db03d16bf5ae3221146d8c6f9efc02d5"} Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.352434 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/rabbitmq-server-0" podUID="3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f" containerName="rabbitmq" probeResult="failure" output="dial tcp 10.217.0.210:5671: connect: connection refused" Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.355769 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0a2de558-2a4b-46ef-9b53-cada8cd466c4-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "0a2de558-2a4b-46ef-9b53-cada8cd466c4" (UID: "0a2de558-2a4b-46ef-9b53-cada8cd466c4"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.373564 5052 generic.go:334] "Generic (PLEG): container finished" podID="296a2858-22b8-41a4-91ac-a9bf055b2848" containerID="88ce6f1697038254e636269a53ace16da9bd7ec94e5543e34a4bab6f9c43e974" exitCode=143 Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.373979 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-keystone-listener-85f96c8f8-b6m5f" event={"ID":"296a2858-22b8-41a4-91ac-a9bf055b2848","Type":"ContainerDied","Data":"88ce6f1697038254e636269a53ace16da9bd7ec94e5543e34a4bab6f9c43e974"} Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.383352 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/baa1c9d4-5887-47e3-be99-96f718178f58-config\") pod \"baa1c9d4-5887-47e3-be99-96f718178f58\" (UID: \"baa1c9d4-5887-47e3-be99-96f718178f58\") " Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.383396 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/baa1c9d4-5887-47e3-be99-96f718178f58-scripts\") pod \"baa1c9d4-5887-47e3-be99-96f718178f58\" (UID: \"baa1c9d4-5887-47e3-be99-96f718178f58\") " Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.383471 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/baa1c9d4-5887-47e3-be99-96f718178f58-metrics-certs-tls-certs\") pod \"baa1c9d4-5887-47e3-be99-96f718178f58\" (UID: \"baa1c9d4-5887-47e3-be99-96f718178f58\") " Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.383497 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"openstack-config-secret\" (UniqueName: \"kubernetes.io/secret/4e574f0c-bbe2-44a9-935c-923622c44443-openstack-config-secret\") pod \"4e574f0c-bbe2-44a9-935c-923622c44443\" (UID: \"4e574f0c-bbe2-44a9-935c-923622c44443\") " Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.383556 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4e574f0c-bbe2-44a9-935c-923622c44443-combined-ca-bundle\") pod \"4e574f0c-bbe2-44a9-935c-923622c44443\" (UID: \"4e574f0c-bbe2-44a9-935c-923622c44443\") " Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.383644 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"openstack-config\" (UniqueName: \"kubernetes.io/configmap/4e574f0c-bbe2-44a9-935c-923622c44443-openstack-config\") pod \"4e574f0c-bbe2-44a9-935c-923622c44443\" (UID: \"4e574f0c-bbe2-44a9-935c-923622c44443\") " Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.383673 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovsdbserver-sb-tls-certs\" (UniqueName: \"kubernetes.io/secret/baa1c9d4-5887-47e3-be99-96f718178f58-ovsdbserver-sb-tls-certs\") pod \"baa1c9d4-5887-47e3-be99-96f718178f58\" (UID: \"baa1c9d4-5887-47e3-be99-96f718178f58\") " Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.383747 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mp7fw\" (UniqueName: \"kubernetes.io/projected/baa1c9d4-5887-47e3-be99-96f718178f58-kube-api-access-mp7fw\") pod \"baa1c9d4-5887-47e3-be99-96f718178f58\" (UID: \"baa1c9d4-5887-47e3-be99-96f718178f58\") " Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.383787 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m2tvf\" (UniqueName: \"kubernetes.io/projected/4e574f0c-bbe2-44a9-935c-923622c44443-kube-api-access-m2tvf\") pod \"4e574f0c-bbe2-44a9-935c-923622c44443\" (UID: \"4e574f0c-bbe2-44a9-935c-923622c44443\") " Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.383843 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/baa1c9d4-5887-47e3-be99-96f718178f58-combined-ca-bundle\") pod \"baa1c9d4-5887-47e3-be99-96f718178f58\" (UID: \"baa1c9d4-5887-47e3-be99-96f718178f58\") " Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.383875 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/baa1c9d4-5887-47e3-be99-96f718178f58-ovsdb-rundir\") pod \"baa1c9d4-5887-47e3-be99-96f718178f58\" (UID: \"baa1c9d4-5887-47e3-be99-96f718178f58\") " Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.383906 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovndbcluster-sb-etc-ovn\" (UniqueName: \"kubernetes.io/local-volume/local-storage11-crc\") pod \"baa1c9d4-5887-47e3-be99-96f718178f58\" (UID: \"baa1c9d4-5887-47e3-be99-96f718178f58\") " Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.384629 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage07-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage07-crc\") on node \"crc\" " Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.384652 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-2x6wd\" (UniqueName: \"kubernetes.io/projected/0a2de558-2a4b-46ef-9b53-cada8cd466c4-kube-api-access-2x6wd\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.384667 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/0a2de558-2a4b-46ef-9b53-cada8cd466c4-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.384678 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0a2de558-2a4b-46ef-9b53-cada8cd466c4-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.393897 5052 generic.go:334] "Generic (PLEG): container finished" podID="d81b09dc-75a1-45f8-ace4-786531ad3bfd" containerID="be83ec436e5bb13951ed23e2fd4e1e84fd5f3b6ce1f0a6f46b5c44dcd3634ee4" exitCode=143 Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.393980 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"d81b09dc-75a1-45f8-ace4-786531ad3bfd","Type":"ContainerDied","Data":"be83ec436e5bb13951ed23e2fd4e1e84fd5f3b6ce1f0a6f46b5c44dcd3634ee4"} Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.395013 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/baa1c9d4-5887-47e3-be99-96f718178f58-config" (OuterVolumeSpecName: "config") pod "baa1c9d4-5887-47e3-be99-96f718178f58" (UID: "baa1c9d4-5887-47e3-be99-96f718178f58"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.395651 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/baa1c9d4-5887-47e3-be99-96f718178f58-scripts" (OuterVolumeSpecName: "scripts") pod "baa1c9d4-5887-47e3-be99-96f718178f58" (UID: "baa1c9d4-5887-47e3-be99-96f718178f58"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.404929 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4e574f0c-bbe2-44a9-935c-923622c44443-kube-api-access-m2tvf" (OuterVolumeSpecName: "kube-api-access-m2tvf") pod "4e574f0c-bbe2-44a9-935c-923622c44443" (UID: "4e574f0c-bbe2-44a9-935c-923622c44443"). InnerVolumeSpecName "kube-api-access-m2tvf". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.405567 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/baa1c9d4-5887-47e3-be99-96f718178f58-ovsdb-rundir" (OuterVolumeSpecName: "ovsdb-rundir") pod "baa1c9d4-5887-47e3-be99-96f718178f58" (UID: "baa1c9d4-5887-47e3-be99-96f718178f58"). InnerVolumeSpecName "ovsdb-rundir". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.419021 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_ovsdbserver-sb-0_baa1c9d4-5887-47e3-be99-96f718178f58/ovsdbserver-sb/0.log" Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.419151 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-sb-0" event={"ID":"baa1c9d4-5887-47e3-be99-96f718178f58","Type":"ContainerDied","Data":"b548a4c8f3c848f6aed22eb43289bcd3f6999edee2eb2631b52567a1e2f69b61"} Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.419265 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.429738 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/baa1c9d4-5887-47e3-be99-96f718178f58-kube-api-access-mp7fw" (OuterVolumeSpecName: "kube-api-access-mp7fw") pod "baa1c9d4-5887-47e3-be99-96f718178f58" (UID: "baa1c9d4-5887-47e3-be99-96f718178f58"). InnerVolumeSpecName "kube-api-access-mp7fw". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.449087 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage11-crc" (OuterVolumeSpecName: "ovndbcluster-sb-etc-ovn") pod "baa1c9d4-5887-47e3-be99-96f718178f58" (UID: "baa1c9d4-5887-47e3-be99-96f718178f58"). InnerVolumeSpecName "local-storage11-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.477849 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage07-crc" (UniqueName: "kubernetes.io/local-volume/local-storage07-crc") on node "crc" Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.494676 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-m2tvf\" (UniqueName: \"kubernetes.io/projected/4e574f0c-bbe2-44a9-935c-923622c44443-kube-api-access-m2tvf\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.494708 5052 reconciler_common.go:293] "Volume detached for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/baa1c9d4-5887-47e3-be99-96f718178f58-ovsdb-rundir\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.494733 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage11-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage11-crc\") on node \"crc\" " Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.494745 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage07-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage07-crc\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.494755 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/baa1c9d4-5887-47e3-be99-96f718178f58-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.494765 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/baa1c9d4-5887-47e3-be99-96f718178f58-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.494775 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-mp7fw\" (UniqueName: \"kubernetes.io/projected/baa1c9d4-5887-47e3-be99-96f718178f58-kube-api-access-mp7fw\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.497789 5052 generic.go:334] "Generic (PLEG): container finished" podID="2a349843-a7b7-4dff-a5ae-f513337c5809" containerID="237be9059aceba08b75de8d8ecd8ee5c1eb28f5bd5c20bb5c20a33088beca081" exitCode=143 Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.497879 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"2a349843-a7b7-4dff-a5ae-f513337c5809","Type":"ContainerDied","Data":"237be9059aceba08b75de8d8ecd8ee5c1eb28f5bd5c20bb5c20a33088beca081"} Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.511817 5052 generic.go:334] "Generic (PLEG): container finished" podID="cb68adf1-26f2-4b14-9874-dde28826aba3" containerID="df21ee50a92cecc1b709d001db731836cf35f7648b75dbb8f7cbdbbb13f8e727" exitCode=143 Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.511910 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"cb68adf1-26f2-4b14-9874-dde28826aba3","Type":"ContainerDied","Data":"df21ee50a92cecc1b709d001db731836cf35f7648b75dbb8f7cbdbbb13f8e727"} Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.515923 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0a2de558-2a4b-46ef-9b53-cada8cd466c4-metrics-certs-tls-certs" (OuterVolumeSpecName: "metrics-certs-tls-certs") pod "0a2de558-2a4b-46ef-9b53-cada8cd466c4" (UID: "0a2de558-2a4b-46ef-9b53-cada8cd466c4"). InnerVolumeSpecName "metrics-certs-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.519060 5052 generic.go:334] "Generic (PLEG): container finished" podID="718e2899-d5c3-41a4-a2f9-4830e18355d3" containerID="84e05639993606274e28568021cb8fdaf723254416f7db2a86eb1f0830df3b64" exitCode=0 Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.519223 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-5b979cdcb6-6hqqv" event={"ID":"718e2899-d5c3-41a4-a2f9-4830e18355d3","Type":"ContainerDied","Data":"84e05639993606274e28568021cb8fdaf723254416f7db2a86eb1f0830df3b64"} Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.525450 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-db-sync-wnzql"] Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.556762 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-db-sync-wnzql"] Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.557375 5052 generic.go:334] "Generic (PLEG): container finished" podID="694f3a22-38d9-487d-b887-a915fa8f99b0" containerID="4ca9b25da15ecf1176a45c4f622d417575fd72f1258e4941eaab46adc37d9954" exitCode=0 Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.557454 5052 generic.go:334] "Generic (PLEG): container finished" podID="694f3a22-38d9-487d-b887-a915fa8f99b0" containerID="c8a4d6f709c57c2c8c86b3ead5f40b5eed433a688c9bda706475d1a83dc38289" exitCode=0 Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.557577 5052 generic.go:334] "Generic (PLEG): container finished" podID="694f3a22-38d9-487d-b887-a915fa8f99b0" containerID="2b4fde6dfdf88033dd51e40f592f3092bbadafedc12b51e3dac6d06c7be816e2" exitCode=0 Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.557632 5052 generic.go:334] "Generic (PLEG): container finished" podID="694f3a22-38d9-487d-b887-a915fa8f99b0" containerID="d34ac8e54a2a8c78fedd6aae86ea4f7c01fa3fb75b7877784aed4c550716ec17" exitCode=0 Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.557699 5052 generic.go:334] "Generic (PLEG): container finished" podID="694f3a22-38d9-487d-b887-a915fa8f99b0" containerID="edccdb7e982ee726341d6181308a200c1c7f7e489199aae3ff7897c6686d1e53" exitCode=0 Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.557759 5052 generic.go:334] "Generic (PLEG): container finished" podID="694f3a22-38d9-487d-b887-a915fa8f99b0" containerID="3577cb2e84ebe92b200c0bf67af07582c963a380e9d15f90356b2162da1491e3" exitCode=0 Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.557808 5052 generic.go:334] "Generic (PLEG): container finished" podID="694f3a22-38d9-487d-b887-a915fa8f99b0" containerID="a9d69281c8f09f62933866859d2a7f994441cb27231474ecb5c4087f61ef9e48" exitCode=0 Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.557857 5052 generic.go:334] "Generic (PLEG): container finished" podID="694f3a22-38d9-487d-b887-a915fa8f99b0" containerID="4be1f7ec0e5ec2598abc5fccb1e3a336e6d2ce904a40e4470bfebd9a2432aa0d" exitCode=0 Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.557905 5052 generic.go:334] "Generic (PLEG): container finished" podID="694f3a22-38d9-487d-b887-a915fa8f99b0" containerID="ffacc5bf5e6a05d80223ac48d049c379298b3d4840c2186fb767cc10e1492580" exitCode=0 Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.557954 5052 generic.go:334] "Generic (PLEG): container finished" podID="694f3a22-38d9-487d-b887-a915fa8f99b0" containerID="9abaf8aac6af7b9078e0f7fa87225ebcfdfa4003e45df36fe94117f331cca24b" exitCode=0 Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.558007 5052 generic.go:334] "Generic (PLEG): container finished" podID="694f3a22-38d9-487d-b887-a915fa8f99b0" containerID="8de12c6937f84a484a57da6623c0b720805ae449cf86cbf7781810fff5f7dbd6" exitCode=0 Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.558057 5052 generic.go:334] "Generic (PLEG): container finished" podID="694f3a22-38d9-487d-b887-a915fa8f99b0" containerID="daff85236f8b1825cad56f8cf5a450330943257c15e8a198c3cbd529f86943a9" exitCode=0 Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.558108 5052 generic.go:334] "Generic (PLEG): container finished" podID="694f3a22-38d9-487d-b887-a915fa8f99b0" containerID="52c9d3753f1d83dc61693ea154f30426c527a127d3d6f00ae6379a1cbd9e5899" exitCode=0 Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.558195 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"694f3a22-38d9-487d-b887-a915fa8f99b0","Type":"ContainerDied","Data":"4ca9b25da15ecf1176a45c4f622d417575fd72f1258e4941eaab46adc37d9954"} Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.558265 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"694f3a22-38d9-487d-b887-a915fa8f99b0","Type":"ContainerDied","Data":"c8a4d6f709c57c2c8c86b3ead5f40b5eed433a688c9bda706475d1a83dc38289"} Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.560892 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"694f3a22-38d9-487d-b887-a915fa8f99b0","Type":"ContainerDied","Data":"2b4fde6dfdf88033dd51e40f592f3092bbadafedc12b51e3dac6d06c7be816e2"} Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.561007 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"694f3a22-38d9-487d-b887-a915fa8f99b0","Type":"ContainerDied","Data":"d34ac8e54a2a8c78fedd6aae86ea4f7c01fa3fb75b7877784aed4c550716ec17"} Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.561068 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"694f3a22-38d9-487d-b887-a915fa8f99b0","Type":"ContainerDied","Data":"edccdb7e982ee726341d6181308a200c1c7f7e489199aae3ff7897c6686d1e53"} Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.561123 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"694f3a22-38d9-487d-b887-a915fa8f99b0","Type":"ContainerDied","Data":"3577cb2e84ebe92b200c0bf67af07582c963a380e9d15f90356b2162da1491e3"} Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.561179 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"694f3a22-38d9-487d-b887-a915fa8f99b0","Type":"ContainerDied","Data":"a9d69281c8f09f62933866859d2a7f994441cb27231474ecb5c4087f61ef9e48"} Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.561238 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"694f3a22-38d9-487d-b887-a915fa8f99b0","Type":"ContainerDied","Data":"4be1f7ec0e5ec2598abc5fccb1e3a336e6d2ce904a40e4470bfebd9a2432aa0d"} Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.561298 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"694f3a22-38d9-487d-b887-a915fa8f99b0","Type":"ContainerDied","Data":"ffacc5bf5e6a05d80223ac48d049c379298b3d4840c2186fb767cc10e1492580"} Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.561355 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"694f3a22-38d9-487d-b887-a915fa8f99b0","Type":"ContainerDied","Data":"9abaf8aac6af7b9078e0f7fa87225ebcfdfa4003e45df36fe94117f331cca24b"} Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.561409 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"694f3a22-38d9-487d-b887-a915fa8f99b0","Type":"ContainerDied","Data":"8de12c6937f84a484a57da6623c0b720805ae449cf86cbf7781810fff5f7dbd6"} Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.561462 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"694f3a22-38d9-487d-b887-a915fa8f99b0","Type":"ContainerDied","Data":"daff85236f8b1825cad56f8cf5a450330943257c15e8a198c3cbd529f86943a9"} Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.561535 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"694f3a22-38d9-487d-b887-a915fa8f99b0","Type":"ContainerDied","Data":"52c9d3753f1d83dc61693ea154f30426c527a127d3d6f00ae6379a1cbd9e5899"} Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.566988 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-0"] Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.567349 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-cell1-conductor-0" podUID="c2a65caa-2eec-48af-bdad-8fc19d827867" containerName="nova-cell1-conductor-conductor" containerID="cri-o://28049a9b7828f990e7cdf628c7744bd85e07ead3fc6256992ef7d5bf23e045a3" gracePeriod=30 Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.570615 5052 generic.go:334] "Generic (PLEG): container finished" podID="388bc903-097f-4f97-8f19-2f7bcdb49cc5" containerID="c15684b97ff4a9d54a9eeca81697f4cced793ffe157adb3548bedd21bf3249d0" exitCode=143 Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.570977 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"388bc903-097f-4f97-8f19-2f7bcdb49cc5","Type":"ContainerDied","Data":"c15684b97ff4a9d54a9eeca81697f4cced793ffe157adb3548bedd21bf3249d0"} Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.575727 5052 generic.go:334] "Generic (PLEG): container finished" podID="0a2de558-2a4b-46ef-9b53-cada8cd466c4" containerID="b53a930fff1916d446b8a6d1e63e019c2f77d3ae6348e99f24ce01cb0d9242b1" exitCode=0 Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.576013 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-nb-0" event={"ID":"0a2de558-2a4b-46ef-9b53-cada8cd466c4","Type":"ContainerDied","Data":"b53a930fff1916d446b8a6d1e63e019c2f77d3ae6348e99f24ce01cb0d9242b1"} Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.576226 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-nb-0" event={"ID":"0a2de558-2a4b-46ef-9b53-cada8cd466c4","Type":"ContainerDied","Data":"ad711c538dba8e33d365eb420757fc8e4b34b6a8568c18585364f148b4aa302e"} Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.576345 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.588029 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/baa1c9d4-5887-47e3-be99-96f718178f58-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "baa1c9d4-5887-47e3-be99-96f718178f58" (UID: "baa1c9d4-5887-47e3-be99-96f718178f58"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.589780 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0a2de558-2a4b-46ef-9b53-cada8cd466c4-ovsdbserver-nb-tls-certs" (OuterVolumeSpecName: "ovsdbserver-nb-tls-certs") pod "0a2de558-2a4b-46ef-9b53-cada8cd466c4" (UID: "0a2de558-2a4b-46ef-9b53-cada8cd466c4"). InnerVolumeSpecName "ovsdbserver-nb-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.590233 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4e574f0c-bbe2-44a9-935c-923622c44443-openstack-config" (OuterVolumeSpecName: "openstack-config") pod "4e574f0c-bbe2-44a9-935c-923622c44443" (UID: "4e574f0c-bbe2-44a9-935c-923622c44443"). InnerVolumeSpecName "openstack-config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.590288 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage11-crc" (UniqueName: "kubernetes.io/local-volume/local-storage11-crc") on node "crc" Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.600562 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-0"] Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.600772 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-cell0-conductor-0" podUID="14774d58-37e8-4e65-b0b0-d355c0070bca" containerName="nova-cell0-conductor-conductor" containerID="cri-o://33c38af311895b3ea2da41b9880bb47d91d38eadcf28dcf815946f0d92d3f569" gracePeriod=30 Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.602489 5052 reconciler_common.go:293] "Volume detached for volume \"ovsdbserver-nb-tls-certs\" (UniqueName: \"kubernetes.io/secret/0a2de558-2a4b-46ef-9b53-cada8cd466c4-ovsdbserver-nb-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.602677 5052 reconciler_common.go:293] "Volume detached for volume \"openstack-config\" (UniqueName: \"kubernetes.io/configmap/4e574f0c-bbe2-44a9-935c-923622c44443-openstack-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.602789 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/baa1c9d4-5887-47e3-be99-96f718178f58-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.602846 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage11-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage11-crc\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.602911 5052 reconciler_common.go:293] "Volume detached for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/0a2de558-2a4b-46ef-9b53-cada8cd466c4-metrics-certs-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.619914 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-db-sync-pmm7g"] Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.626081 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-db-sync-pmm7g"] Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.629896 5052 generic.go:334] "Generic (PLEG): container finished" podID="8283de30-8976-47bd-bd09-aab53140f68d" containerID="2b2c845f4fc350fd56a0f9ca2b46def3a0f193a23ad1b9dd526750dac6bdc0e7" exitCode=143 Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.629938 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-79d5cb4fdb-7h2fn" event={"ID":"8283de30-8976-47bd-bd09-aab53140f68d","Type":"ContainerDied","Data":"2b2c845f4fc350fd56a0f9ca2b46def3a0f193a23ad1b9dd526750dac6bdc0e7"} Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.637663 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4e574f0c-bbe2-44a9-935c-923622c44443-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "4e574f0c-bbe2-44a9-935c-923622c44443" (UID: "4e574f0c-bbe2-44a9-935c-923622c44443"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.638925 5052 generic.go:334] "Generic (PLEG): container finished" podID="b25d2a0c-1e93-4b3d-965a-a7c021f9649d" containerID="6908da1783700654af60d79f711ce4360ed719f4f83031effe704c70de2ab47f" exitCode=143 Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.638994 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/novacell1f1c3-account-delete-6zqsd" Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.639628 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-58464649dd-sx8mw" event={"ID":"b25d2a0c-1e93-4b3d-965a-a7c021f9649d","Type":"ContainerDied","Data":"6908da1783700654af60d79f711ce4360ed719f4f83031effe704c70de2ab47f"} Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.674715 5052 kubelet_pods.go:1007] "Unable to retrieve pull secret, the image pull may not succeed." pod="openstack-kuttl-tests/swift-proxy-6b68c4db5d-g8nvc" secret="" err="secret \"swift-swift-dockercfg-2jv2v\" not found" Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.696730 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="03b10bd5-7748-43f4-a4e2-346928c502f1" path="/var/lib/kubelet/pods/03b10bd5-7748-43f4-a4e2-346928c502f1/volumes" Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.697498 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/baa1c9d4-5887-47e3-be99-96f718178f58-metrics-certs-tls-certs" (OuterVolumeSpecName: "metrics-certs-tls-certs") pod "baa1c9d4-5887-47e3-be99-96f718178f58" (UID: "baa1c9d4-5887-47e3-be99-96f718178f58"). InnerVolumeSpecName "metrics-certs-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.698690 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="049d1d7a-9b34-4d27-a922-ebb89410b3fa" path="/var/lib/kubelet/pods/049d1d7a-9b34-4d27-a922-ebb89410b3fa/volumes" Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.699324 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2e0f8c25-7b0e-439d-8518-1a95a325e502" path="/var/lib/kubelet/pods/2e0f8c25-7b0e-439d-8518-1a95a325e502/volumes" Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.701330 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4e0e5cde-a60c-4c0e-afd7-818c9ad44c83" path="/var/lib/kubelet/pods/4e0e5cde-a60c-4c0e-afd7-818c9ad44c83/volumes" Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.702774 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6d8c591d-96a8-4bef-908c-89ae9d1b46fb" path="/var/lib/kubelet/pods/6d8c591d-96a8-4bef-908c-89ae9d1b46fb/volumes" Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.704426 5052 reconciler_common.go:293] "Volume detached for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/baa1c9d4-5887-47e3-be99-96f718178f58-metrics-certs-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.704907 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4e574f0c-bbe2-44a9-935c-923622c44443-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.705380 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7600241d-6e5b-41ee-b17c-f5e15f1cab11" path="/var/lib/kubelet/pods/7600241d-6e5b-41ee-b17c-f5e15f1cab11/volumes" Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.706053 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8474b807-15d5-405a-ade8-1e4fa8954ce6" path="/var/lib/kubelet/pods/8474b807-15d5-405a-ade8-1e4fa8954ce6/volumes" Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.707235 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="855f1c2d-3e1c-44b1-a691-41673832e6bc" path="/var/lib/kubelet/pods/855f1c2d-3e1c-44b1-a691-41673832e6bc/volumes" Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.708316 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="971c703f-d93b-4ae2-b8c2-4d9458793408" path="/var/lib/kubelet/pods/971c703f-d93b-4ae2-b8c2-4d9458793408/volumes" Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.709172 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bb817663-1fe7-4c73-803b-5e64dc4ba616" path="/var/lib/kubelet/pods/bb817663-1fe7-4c73-803b-5e64dc4ba616/volumes" Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.709777 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f2a04bd0-579e-467c-b0f6-72af16ffb143" path="/var/lib/kubelet/pods/f2a04bd0-579e-467c-b0f6-72af16ffb143/volumes" Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.732704 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4e574f0c-bbe2-44a9-935c-923622c44443-openstack-config-secret" (OuterVolumeSpecName: "openstack-config-secret") pod "4e574f0c-bbe2-44a9-935c-923622c44443" (UID: "4e574f0c-bbe2-44a9-935c-923622c44443"). InnerVolumeSpecName "openstack-config-secret". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.766670 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" podUID="e63eb1eb-e0b9-46ea-a19b-8f5c018728ff" containerName="rabbitmq" probeResult="failure" output="dial tcp 10.217.0.211:5671: connect: connection refused" Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.771223 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/baa1c9d4-5887-47e3-be99-96f718178f58-ovsdbserver-sb-tls-certs" (OuterVolumeSpecName: "ovsdbserver-sb-tls-certs") pod "baa1c9d4-5887-47e3-be99-96f718178f58" (UID: "baa1c9d4-5887-47e3-be99-96f718178f58"). InnerVolumeSpecName "ovsdbserver-sb-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.798844 5052 scope.go:117] "RemoveContainer" containerID="27087053cd65b0655720b99b7846639054cddedb51bef7c1cc746fabc39c1401" Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.813142 5052 reconciler_common.go:293] "Volume detached for volume \"openstack-config-secret\" (UniqueName: \"kubernetes.io/secret/4e574f0c-bbe2-44a9-935c-923622c44443-openstack-config-secret\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.813192 5052 reconciler_common.go:293] "Volume detached for volume \"ovsdbserver-sb-tls-certs\" (UniqueName: \"kubernetes.io/secret/baa1c9d4-5887-47e3-be99-96f718178f58-ovsdbserver-sb-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:07 crc kubenswrapper[5052]: E1203 12:15:07.821805 5052 projected.go:263] Couldn't get secret openstack-kuttl-tests/swift-conf: secret "swift-conf" not found Dec 03 12:15:07 crc kubenswrapper[5052]: E1203 12:15:07.821866 5052 projected.go:288] Couldn't get configMap openstack-kuttl-tests/swift-ring-files: configmap "swift-ring-files" not found Dec 03 12:15:07 crc kubenswrapper[5052]: E1203 12:15:07.821892 5052 projected.go:194] Error preparing data for projected volume etc-swift for pod openstack-kuttl-tests/swift-proxy-6b68c4db5d-g8nvc: [secret "swift-conf" not found, configmap "swift-ring-files" not found] Dec 03 12:15:07 crc kubenswrapper[5052]: E1203 12:15:07.821968 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ae6940a4-0f4c-46ab-8218-56bf5c5d3084-etc-swift podName:ae6940a4-0f4c-46ab-8218-56bf5c5d3084 nodeName:}" failed. No retries permitted until 2025-12-03 12:15:08.321940488 +0000 UTC m=+3298.910182252 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "etc-swift" (UniqueName: "kubernetes.io/projected/ae6940a4-0f4c-46ab-8218-56bf5c5d3084-etc-swift") pod "swift-proxy-6b68c4db5d-g8nvc" (UID: "ae6940a4-0f4c-46ab-8218-56bf5c5d3084") : [secret "swift-conf" not found, configmap "swift-ring-files" not found] Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.844720 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron1287-account-delete-t8dcs"] Dec 03 12:15:07 crc kubenswrapper[5052]: E1203 12:15:07.851125 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"27087053cd65b0655720b99b7846639054cddedb51bef7c1cc746fabc39c1401\": container with ID starting with 27087053cd65b0655720b99b7846639054cddedb51bef7c1cc746fabc39c1401 not found: ID does not exist" containerID="27087053cd65b0655720b99b7846639054cddedb51bef7c1cc746fabc39c1401" Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.851160 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"27087053cd65b0655720b99b7846639054cddedb51bef7c1cc746fabc39c1401"} err="failed to get container status \"27087053cd65b0655720b99b7846639054cddedb51bef7c1cc746fabc39c1401\": rpc error: code = NotFound desc = could not find container \"27087053cd65b0655720b99b7846639054cddedb51bef7c1cc746fabc39c1401\": container with ID starting with 27087053cd65b0655720b99b7846639054cddedb51bef7c1cc746fabc39c1401 not found: ID does not exist" Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.851191 5052 scope.go:117] "RemoveContainer" containerID="b8259e68d536f622f925db84f9b7dd7940949212302ab0a104352098a97b2889" Dec 03 12:15:07 crc kubenswrapper[5052]: E1203 12:15:07.863088 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="81879a070623f490e700ae5e038359faa2d3ce34a28b8f7e4b856f6c4f95fa7f" cmd=["/usr/bin/pgrep","-r","DRST","nova-scheduler"] Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.864681 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/novacell1f1c3-account-delete-6zqsd" Dec 03 12:15:07 crc kubenswrapper[5052]: E1203 12:15:07.865738 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="81879a070623f490e700ae5e038359faa2d3ce34a28b8f7e4b856f6c4f95fa7f" cmd=["/usr/bin/pgrep","-r","DRST","nova-scheduler"] Dec 03 12:15:07 crc kubenswrapper[5052]: E1203 12:15:07.872247 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="81879a070623f490e700ae5e038359faa2d3ce34a28b8f7e4b856f6c4f95fa7f" cmd=["/usr/bin/pgrep","-r","DRST","nova-scheduler"] Dec 03 12:15:07 crc kubenswrapper[5052]: E1203 12:15:07.872298 5052 prober.go:104] "Probe errored" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" probeType="Readiness" pod="openstack-kuttl-tests/nova-scheduler-0" podUID="cf3ed088-9069-40cf-9c9e-ee9bdf7761d9" containerName="nova-scheduler-scheduler" Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.958798 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-nb-0"] Dec 03 12:15:07 crc kubenswrapper[5052]: I1203 12:15:07.981976 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-nb-0"] Dec 03 12:15:08 crc kubenswrapper[5052]: W1203 12:15:08.000747 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf1619fbf_df68_484f_99d4_09b81a7a334e.slice/crio-f44845aa3e508f5ce4fdd2b4e7dcd364803cb5dbfd95eb5cec06972871fb290d WatchSource:0}: Error finding container f44845aa3e508f5ce4fdd2b4e7dcd364803cb5dbfd95eb5cec06972871fb290d: Status 404 returned error can't find the container with id f44845aa3e508f5ce4fdd2b4e7dcd364803cb5dbfd95eb5cec06972871fb290d Dec 03 12:15:08 crc kubenswrapper[5052]: I1203 12:15:08.106167 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican8992-account-delete-26lhs"] Dec 03 12:15:08 crc kubenswrapper[5052]: I1203 12:15:08.110945 5052 scope.go:117] "RemoveContainer" containerID="7d5cb223b2febf0f7150eb3ecf28694a92fc2e544ffbcbf9721f02dc30547b38" Dec 03 12:15:08 crc kubenswrapper[5052]: I1203 12:15:08.131568 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glancef253-account-delete-cxxlg"] Dec 03 12:15:08 crc kubenswrapper[5052]: I1203 12:15:08.150356 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-sb-0"] Dec 03 12:15:08 crc kubenswrapper[5052]: I1203 12:15:08.154180 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-sb-0"] Dec 03 12:15:08 crc kubenswrapper[5052]: I1203 12:15:08.164305 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:15:08 crc kubenswrapper[5052]: I1203 12:15:08.164659 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="1661efff-d6ce-479d-bfb9-90849fa6ff40" containerName="ceilometer-central-agent" containerID="cri-o://e964aba7c310e135f899bcb05dcd28df12a6cfb568f657650c49176388d578b0" gracePeriod=30 Dec 03 12:15:08 crc kubenswrapper[5052]: I1203 12:15:08.165814 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="1661efff-d6ce-479d-bfb9-90849fa6ff40" containerName="proxy-httpd" containerID="cri-o://240bba2f32d305ed667a2f5bc674c5d7f4bda3e122633105398b9c8c9930cfb5" gracePeriod=30 Dec 03 12:15:08 crc kubenswrapper[5052]: I1203 12:15:08.165858 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="1661efff-d6ce-479d-bfb9-90849fa6ff40" containerName="ceilometer-notification-agent" containerID="cri-o://aee5b94189aa0133ba610d83708c280ddef3068baf105fae0fbe385e8bb3d748" gracePeriod=30 Dec 03 12:15:08 crc kubenswrapper[5052]: I1203 12:15:08.165836 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="1661efff-d6ce-479d-bfb9-90849fa6ff40" containerName="sg-core" containerID="cri-o://95a8366b935cbf627521e68e3391e0f202e2b033969a53119fa8a475d79c1be3" gracePeriod=30 Dec 03 12:15:08 crc kubenswrapper[5052]: W1203 12:15:08.178654 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfee7d732_4fa3_40d5_8fff_cb88272db0ce.slice/crio-77f6420de0ee4d7fc7f116d4e9e916a4b2a7b00e8e935caa9e5e9c678683d77e WatchSource:0}: Error finding container 77f6420de0ee4d7fc7f116d4e9e916a4b2a7b00e8e935caa9e5e9c678683d77e: Status 404 returned error can't find the container with id 77f6420de0ee4d7fc7f116d4e9e916a4b2a7b00e8e935caa9e5e9c678683d77e Dec 03 12:15:08 crc kubenswrapper[5052]: I1203 12:15:08.178941 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/kube-state-metrics-0"] Dec 03 12:15:08 crc kubenswrapper[5052]: I1203 12:15:08.179179 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/kube-state-metrics-0" podUID="dfdd8013-ee6e-4ec6-b5d1-fb423f444430" containerName="kube-state-metrics" containerID="cri-o://8021f4fcdb6f78190623eed4c27195e38d52e80b4781dc680a2591d5bf74777d" gracePeriod=30 Dec 03 12:15:08 crc kubenswrapper[5052]: I1203 12:15:08.230213 5052 scope.go:117] "RemoveContainer" containerID="210f225008eb18ae291abefbe19ad7b03eaf69b1879a6f6bbe585e72babe1848" Dec 03 12:15:08 crc kubenswrapper[5052]: I1203 12:15:08.328110 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-zwj9p\" (UniqueName: \"kubernetes.io/projected/b853a2c2-337f-423e-84bf-ea2814cf59da-kube-api-access-zwj9p\") pod \"novacell1f1c3-account-delete-6zqsd\" (UID: \"b853a2c2-337f-423e-84bf-ea2814cf59da\") " pod="openstack-kuttl-tests/novacell1f1c3-account-delete-6zqsd" Dec 03 12:15:08 crc kubenswrapper[5052]: I1203 12:15:08.328200 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/b853a2c2-337f-423e-84bf-ea2814cf59da-operator-scripts\") pod \"novacell1f1c3-account-delete-6zqsd\" (UID: \"b853a2c2-337f-423e-84bf-ea2814cf59da\") " pod="openstack-kuttl-tests/novacell1f1c3-account-delete-6zqsd" Dec 03 12:15:08 crc kubenswrapper[5052]: E1203 12:15:08.328454 5052 projected.go:263] Couldn't get secret openstack-kuttl-tests/swift-conf: secret "swift-conf" not found Dec 03 12:15:08 crc kubenswrapper[5052]: E1203 12:15:08.328473 5052 projected.go:288] Couldn't get configMap openstack-kuttl-tests/swift-ring-files: configmap "swift-ring-files" not found Dec 03 12:15:08 crc kubenswrapper[5052]: E1203 12:15:08.328485 5052 projected.go:194] Error preparing data for projected volume etc-swift for pod openstack-kuttl-tests/swift-proxy-6b68c4db5d-g8nvc: [secret "swift-conf" not found, configmap "swift-ring-files" not found] Dec 03 12:15:08 crc kubenswrapper[5052]: E1203 12:15:08.328558 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ae6940a4-0f4c-46ab-8218-56bf5c5d3084-etc-swift podName:ae6940a4-0f4c-46ab-8218-56bf5c5d3084 nodeName:}" failed. No retries permitted until 2025-12-03 12:15:09.328541153 +0000 UTC m=+3299.916782917 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "etc-swift" (UniqueName: "kubernetes.io/projected/ae6940a4-0f4c-46ab-8218-56bf5c5d3084-etc-swift") pod "swift-proxy-6b68c4db5d-g8nvc" (UID: "ae6940a4-0f4c-46ab-8218-56bf5c5d3084") : [secret "swift-conf" not found, configmap "swift-ring-files" not found] Dec 03 12:15:08 crc kubenswrapper[5052]: E1203 12:15:08.328640 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-cell1-scripts: configmap "openstack-cell1-scripts" not found Dec 03 12:15:08 crc kubenswrapper[5052]: E1203 12:15:08.328709 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b853a2c2-337f-423e-84bf-ea2814cf59da-operator-scripts podName:b853a2c2-337f-423e-84bf-ea2814cf59da nodeName:}" failed. No retries permitted until 2025-12-03 12:15:10.328687607 +0000 UTC m=+3300.916929421 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/b853a2c2-337f-423e-84bf-ea2814cf59da-operator-scripts") pod "novacell1f1c3-account-delete-6zqsd" (UID: "b853a2c2-337f-423e-84bf-ea2814cf59da") : configmap "openstack-cell1-scripts" not found Dec 03 12:15:08 crc kubenswrapper[5052]: E1203 12:15:08.338833 5052 projected.go:194] Error preparing data for projected volume kube-api-access-zwj9p for pod openstack-kuttl-tests/novacell1f1c3-account-delete-6zqsd: failed to fetch token: serviceaccounts "galera-openstack-cell1" not found Dec 03 12:15:08 crc kubenswrapper[5052]: E1203 12:15:08.339182 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b853a2c2-337f-423e-84bf-ea2814cf59da-kube-api-access-zwj9p podName:b853a2c2-337f-423e-84bf-ea2814cf59da nodeName:}" failed. No retries permitted until 2025-12-03 12:15:10.339158496 +0000 UTC m=+3300.927400260 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-zwj9p" (UniqueName: "kubernetes.io/projected/b853a2c2-337f-423e-84bf-ea2814cf59da-kube-api-access-zwj9p") pod "novacell1f1c3-account-delete-6zqsd" (UID: "b853a2c2-337f-423e-84bf-ea2814cf59da") : failed to fetch token: serviceaccounts "galera-openstack-cell1" not found Dec 03 12:15:08 crc kubenswrapper[5052]: I1203 12:15:08.449442 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/swift-proxy-6b68c4db5d-g8nvc"] Dec 03 12:15:08 crc kubenswrapper[5052]: E1203 12:15:08.451111 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/rabbitmq-cell1-config-data: configmap "rabbitmq-cell1-config-data" not found Dec 03 12:15:08 crc kubenswrapper[5052]: E1203 12:15:08.453749 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e63eb1eb-e0b9-46ea-a19b-8f5c018728ff-config-data podName:e63eb1eb-e0b9-46ea-a19b-8f5c018728ff nodeName:}" failed. No retries permitted until 2025-12-03 12:15:12.453723257 +0000 UTC m=+3303.041965011 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "config-data" (UniqueName: "kubernetes.io/configmap/e63eb1eb-e0b9-46ea-a19b-8f5c018728ff-config-data") pod "rabbitmq-cell1-server-0" (UID: "e63eb1eb-e0b9-46ea-a19b-8f5c018728ff") : configmap "rabbitmq-cell1-config-data" not found Dec 03 12:15:08 crc kubenswrapper[5052]: I1203 12:15:08.647457 5052 scope.go:117] "RemoveContainer" containerID="b53a930fff1916d446b8a6d1e63e019c2f77d3ae6348e99f24ce01cb0d9242b1" Dec 03 12:15:08 crc kubenswrapper[5052]: I1203 12:15:08.700099 5052 generic.go:334] "Generic (PLEG): container finished" podID="dfdd8013-ee6e-4ec6-b5d1-fb423f444430" containerID="8021f4fcdb6f78190623eed4c27195e38d52e80b4781dc680a2591d5bf74777d" exitCode=2 Dec 03 12:15:08 crc kubenswrapper[5052]: I1203 12:15:08.700177 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/kube-state-metrics-0" event={"ID":"dfdd8013-ee6e-4ec6-b5d1-fb423f444430","Type":"ContainerDied","Data":"8021f4fcdb6f78190623eed4c27195e38d52e80b4781dc680a2591d5bf74777d"} Dec 03 12:15:08 crc kubenswrapper[5052]: E1203 12:15:08.703763 5052 desired_state_of_world_populator.go:312] "Error processing volume" err="error processing PVC openstack-kuttl-tests/mysql-db-openstack-galera-0: PVC is being deleted" pod="openstack-kuttl-tests/openstack-galera-0" volumeName="mysql-db" Dec 03 12:15:08 crc kubenswrapper[5052]: I1203 12:15:08.708005 5052 generic.go:334] "Generic (PLEG): container finished" podID="00a9935f-ef2d-4976-97a0-78ee6325433d" containerID="de161b1412d41d5ec322763c71b1221e12135067ced1a67a7f68542d387c4abb" exitCode=0 Dec 03 12:15:08 crc kubenswrapper[5052]: I1203 12:15:08.708129 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-cell1-galera-0" event={"ID":"00a9935f-ef2d-4976-97a0-78ee6325433d","Type":"ContainerDied","Data":"de161b1412d41d5ec322763c71b1221e12135067ced1a67a7f68542d387c4abb"} Dec 03 12:15:08 crc kubenswrapper[5052]: I1203 12:15:08.708194 5052 scope.go:117] "RemoveContainer" containerID="210f225008eb18ae291abefbe19ad7b03eaf69b1879a6f6bbe585e72babe1848" Dec 03 12:15:08 crc kubenswrapper[5052]: E1203 12:15:08.709049 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"210f225008eb18ae291abefbe19ad7b03eaf69b1879a6f6bbe585e72babe1848\": container with ID starting with 210f225008eb18ae291abefbe19ad7b03eaf69b1879a6f6bbe585e72babe1848 not found: ID does not exist" containerID="210f225008eb18ae291abefbe19ad7b03eaf69b1879a6f6bbe585e72babe1848" Dec 03 12:15:08 crc kubenswrapper[5052]: I1203 12:15:08.709087 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"210f225008eb18ae291abefbe19ad7b03eaf69b1879a6f6bbe585e72babe1848"} err="failed to get container status \"210f225008eb18ae291abefbe19ad7b03eaf69b1879a6f6bbe585e72babe1848\": rpc error: code = NotFound desc = could not find container \"210f225008eb18ae291abefbe19ad7b03eaf69b1879a6f6bbe585e72babe1848\": container with ID starting with 210f225008eb18ae291abefbe19ad7b03eaf69b1879a6f6bbe585e72babe1848 not found: ID does not exist" Dec 03 12:15:08 crc kubenswrapper[5052]: I1203 12:15:08.709137 5052 scope.go:117] "RemoveContainer" containerID="b53a930fff1916d446b8a6d1e63e019c2f77d3ae6348e99f24ce01cb0d9242b1" Dec 03 12:15:08 crc kubenswrapper[5052]: E1203 12:15:08.710018 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b53a930fff1916d446b8a6d1e63e019c2f77d3ae6348e99f24ce01cb0d9242b1\": container with ID starting with b53a930fff1916d446b8a6d1e63e019c2f77d3ae6348e99f24ce01cb0d9242b1 not found: ID does not exist" containerID="b53a930fff1916d446b8a6d1e63e019c2f77d3ae6348e99f24ce01cb0d9242b1" Dec 03 12:15:08 crc kubenswrapper[5052]: I1203 12:15:08.710042 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b53a930fff1916d446b8a6d1e63e019c2f77d3ae6348e99f24ce01cb0d9242b1"} err="failed to get container status \"b53a930fff1916d446b8a6d1e63e019c2f77d3ae6348e99f24ce01cb0d9242b1\": rpc error: code = NotFound desc = could not find container \"b53a930fff1916d446b8a6d1e63e019c2f77d3ae6348e99f24ce01cb0d9242b1\": container with ID starting with b53a930fff1916d446b8a6d1e63e019c2f77d3ae6348e99f24ce01cb0d9242b1 not found: ID does not exist" Dec 03 12:15:08 crc kubenswrapper[5052]: I1203 12:15:08.712747 5052 generic.go:334] "Generic (PLEG): container finished" podID="1661efff-d6ce-479d-bfb9-90849fa6ff40" containerID="240bba2f32d305ed667a2f5bc674c5d7f4bda3e122633105398b9c8c9930cfb5" exitCode=0 Dec 03 12:15:08 crc kubenswrapper[5052]: I1203 12:15:08.712776 5052 generic.go:334] "Generic (PLEG): container finished" podID="1661efff-d6ce-479d-bfb9-90849fa6ff40" containerID="95a8366b935cbf627521e68e3391e0f202e2b033969a53119fa8a475d79c1be3" exitCode=2 Dec 03 12:15:08 crc kubenswrapper[5052]: I1203 12:15:08.712825 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"1661efff-d6ce-479d-bfb9-90849fa6ff40","Type":"ContainerDied","Data":"240bba2f32d305ed667a2f5bc674c5d7f4bda3e122633105398b9c8c9930cfb5"} Dec 03 12:15:08 crc kubenswrapper[5052]: I1203 12:15:08.712854 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"1661efff-d6ce-479d-bfb9-90849fa6ff40","Type":"ContainerDied","Data":"95a8366b935cbf627521e68e3391e0f202e2b033969a53119fa8a475d79c1be3"} Dec 03 12:15:08 crc kubenswrapper[5052]: I1203 12:15:08.744093 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-kl2c8"] Dec 03 12:15:08 crc kubenswrapper[5052]: E1203 12:15:08.767654 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/rabbitmq-config-data: configmap "rabbitmq-config-data" not found Dec 03 12:15:08 crc kubenswrapper[5052]: E1203 12:15:08.767922 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f-config-data podName:3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f nodeName:}" failed. No retries permitted until 2025-12-03 12:15:12.767900408 +0000 UTC m=+3303.356142172 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "config-data" (UniqueName: "kubernetes.io/configmap/3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f-config-data") pod "rabbitmq-server-0" (UID: "3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f") : configmap "rabbitmq-config-data" not found Dec 03 12:15:08 crc kubenswrapper[5052]: I1203 12:15:08.769894 5052 generic.go:334] "Generic (PLEG): container finished" podID="694f3a22-38d9-487d-b887-a915fa8f99b0" containerID="bfc910b016024c52ad21685e4f00e1ebf323cb412cbdac44e180ce0af3b5c974" exitCode=0 Dec 03 12:15:08 crc kubenswrapper[5052]: I1203 12:15:08.769978 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"694f3a22-38d9-487d-b887-a915fa8f99b0","Type":"ContainerDied","Data":"bfc910b016024c52ad21685e4f00e1ebf323cb412cbdac44e180ce0af3b5c974"} Dec 03 12:15:08 crc kubenswrapper[5052]: I1203 12:15:08.787732 5052 generic.go:334] "Generic (PLEG): container finished" podID="a15b6144-952c-4d41-a141-5bca5c7fd402" containerID="6b64e3ed870b9dc27d7d4df12ca7a0eb2dff4a0cc61910bf73a9e7a4b368dedc" exitCode=0 Dec 03 12:15:08 crc kubenswrapper[5052]: I1203 12:15:08.787787 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" event={"ID":"a15b6144-952c-4d41-a141-5bca5c7fd402","Type":"ContainerDied","Data":"6b64e3ed870b9dc27d7d4df12ca7a0eb2dff4a0cc61910bf73a9e7a4b368dedc"} Dec 03 12:15:08 crc kubenswrapper[5052]: I1203 12:15:08.812839 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron1287-account-delete-t8dcs" event={"ID":"f1619fbf-df68-484f-99d4-09b81a7a334e","Type":"ContainerStarted","Data":"f44845aa3e508f5ce4fdd2b4e7dcd364803cb5dbfd95eb5cec06972871fb290d"} Dec 03 12:15:08 crc kubenswrapper[5052]: I1203 12:15:08.844458 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/neutron1287-account-delete-t8dcs" podStartSLOduration=3.844444494 podStartE2EDuration="3.844444494s" podCreationTimestamp="2025-12-03 12:15:05 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:15:08.839888814 +0000 UTC m=+3299.428130578" watchObservedRunningTime="2025-12-03 12:15:08.844444494 +0000 UTC m=+3299.432686258" Dec 03 12:15:08 crc kubenswrapper[5052]: I1203 12:15:08.853547 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glancef253-account-delete-cxxlg" event={"ID":"fee7d732-4fa3-40d5-8fff-cb88272db0ce","Type":"ContainerStarted","Data":"77f6420de0ee4d7fc7f116d4e9e916a4b2a7b00e8e935caa9e5e9c678683d77e"} Dec 03 12:15:08 crc kubenswrapper[5052]: I1203 12:15:08.878581 5052 generic.go:334] "Generic (PLEG): container finished" podID="89b056a7-e448-447d-be7d-b69f3d4862a1" containerID="ac8820d4342df6998e1122358c0ed26bd8debe54ca84f5a0d380bb533f8c00eb" exitCode=0 Dec 03 12:15:08 crc kubenswrapper[5052]: I1203 12:15:08.878699 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"89b056a7-e448-447d-be7d-b69f3d4862a1","Type":"ContainerDied","Data":"ac8820d4342df6998e1122358c0ed26bd8debe54ca84f5a0d380bb533f8c00eb"} Dec 03 12:15:08 crc kubenswrapper[5052]: I1203 12:15:08.952940 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement7cee-account-delete-7wgkx" event={"ID":"6099fea1-bbb4-4413-a826-539b3dfc3eec","Type":"ContainerStarted","Data":"998bba76a5f7f109bda9a23387d0dfd8298ff434e5f70a2d60d2a1801c384af4"} Dec 03 12:15:08 crc kubenswrapper[5052]: I1203 12:15:08.958875 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/glancef253-account-delete-cxxlg" podStartSLOduration=3.95885146 podStartE2EDuration="3.95885146s" podCreationTimestamp="2025-12-03 12:15:05 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:15:08.92976677 +0000 UTC m=+3299.518008524" watchObservedRunningTime="2025-12-03 12:15:08.95885146 +0000 UTC m=+3299.547093224" Dec 03 12:15:08 crc kubenswrapper[5052]: I1203 12:15:08.966831 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican8992-account-delete-26lhs" event={"ID":"5856107a-0498-4956-beba-4390f5477ca0","Type":"ContainerStarted","Data":"a31833a368a8d17106bbd18312736160d0ab49635a5fa9613d45237fa2a59bfe"} Dec 03 12:15:08 crc kubenswrapper[5052]: I1203 12:15:08.968543 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-proxy-6b68c4db5d-g8nvc" podUID="ae6940a4-0f4c-46ab-8218-56bf5c5d3084" containerName="proxy-httpd" containerID="cri-o://22c61e9465e05e82a9b75ec36f8899daba6ae848986e069b783febfc60abd647" gracePeriod=30 Dec 03 12:15:08 crc kubenswrapper[5052]: I1203 12:15:08.968875 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/novacell1f1c3-account-delete-6zqsd" Dec 03 12:15:08 crc kubenswrapper[5052]: I1203 12:15:08.973339 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-proxy-6b68c4db5d-g8nvc" podUID="ae6940a4-0f4c-46ab-8218-56bf5c5d3084" containerName="proxy-server" containerID="cri-o://6d0e263d1fbe61d44b6dd94d1a87bc31a61607f209df9eb30c44c6f6386c7381" gracePeriod=30 Dec 03 12:15:09 crc kubenswrapper[5052]: I1203 12:15:09.154588 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/swift-proxy-6b68c4db5d-g8nvc" podUID="ae6940a4-0f4c-46ab-8218-56bf5c5d3084" containerName="proxy-server" probeResult="failure" output="Get \"https://10.217.1.9:8080/healthcheck\": read tcp 10.217.0.2:47824->10.217.1.9:8080: read: connection reset by peer" Dec 03 12:15:09 crc kubenswrapper[5052]: I1203 12:15:09.182037 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:15:09 crc kubenswrapper[5052]: I1203 12:15:09.195101 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/novacell1f1c3-account-delete-6zqsd"] Dec 03 12:15:09 crc kubenswrapper[5052]: I1203 12:15:09.200447 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/novacell1f1c3-account-delete-6zqsd"] Dec 03 12:15:09 crc kubenswrapper[5052]: I1203 12:15:09.246353 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:15:09 crc kubenswrapper[5052]: I1203 12:15:09.302410 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a15b6144-952c-4d41-a141-5bca5c7fd402-combined-ca-bundle\") pod \"a15b6144-952c-4d41-a141-5bca5c7fd402\" (UID: \"a15b6144-952c-4d41-a141-5bca5c7fd402\") " Dec 03 12:15:09 crc kubenswrapper[5052]: I1203 12:15:09.302557 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"vencrypt-tls-certs\" (UniqueName: \"kubernetes.io/secret/a15b6144-952c-4d41-a141-5bca5c7fd402-vencrypt-tls-certs\") pod \"a15b6144-952c-4d41-a141-5bca5c7fd402\" (UID: \"a15b6144-952c-4d41-a141-5bca5c7fd402\") " Dec 03 12:15:09 crc kubenswrapper[5052]: I1203 12:15:09.302615 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/dfdd8013-ee6e-4ec6-b5d1-fb423f444430-combined-ca-bundle\") pod \"dfdd8013-ee6e-4ec6-b5d1-fb423f444430\" (UID: \"dfdd8013-ee6e-4ec6-b5d1-fb423f444430\") " Dec 03 12:15:09 crc kubenswrapper[5052]: I1203 12:15:09.302673 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-state-metrics-tls-certs\" (UniqueName: \"kubernetes.io/secret/dfdd8013-ee6e-4ec6-b5d1-fb423f444430-kube-state-metrics-tls-certs\") pod \"dfdd8013-ee6e-4ec6-b5d1-fb423f444430\" (UID: \"dfdd8013-ee6e-4ec6-b5d1-fb423f444430\") " Dec 03 12:15:09 crc kubenswrapper[5052]: I1203 12:15:09.302711 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a15b6144-952c-4d41-a141-5bca5c7fd402-config-data\") pod \"a15b6144-952c-4d41-a141-5bca5c7fd402\" (UID: \"a15b6144-952c-4d41-a141-5bca5c7fd402\") " Dec 03 12:15:09 crc kubenswrapper[5052]: I1203 12:15:09.302767 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"nova-novncproxy-tls-certs\" (UniqueName: \"kubernetes.io/secret/a15b6144-952c-4d41-a141-5bca5c7fd402-nova-novncproxy-tls-certs\") pod \"a15b6144-952c-4d41-a141-5bca5c7fd402\" (UID: \"a15b6144-952c-4d41-a141-5bca5c7fd402\") " Dec 03 12:15:09 crc kubenswrapper[5052]: I1203 12:15:09.302817 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vvbz9\" (UniqueName: \"kubernetes.io/projected/a15b6144-952c-4d41-a141-5bca5c7fd402-kube-api-access-vvbz9\") pod \"a15b6144-952c-4d41-a141-5bca5c7fd402\" (UID: \"a15b6144-952c-4d41-a141-5bca5c7fd402\") " Dec 03 12:15:09 crc kubenswrapper[5052]: I1203 12:15:09.302852 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-state-metrics-tls-config\" (UniqueName: \"kubernetes.io/secret/dfdd8013-ee6e-4ec6-b5d1-fb423f444430-kube-state-metrics-tls-config\") pod \"dfdd8013-ee6e-4ec6-b5d1-fb423f444430\" (UID: \"dfdd8013-ee6e-4ec6-b5d1-fb423f444430\") " Dec 03 12:15:09 crc kubenswrapper[5052]: I1203 12:15:09.302882 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ljg7n\" (UniqueName: \"kubernetes.io/projected/dfdd8013-ee6e-4ec6-b5d1-fb423f444430-kube-api-access-ljg7n\") pod \"dfdd8013-ee6e-4ec6-b5d1-fb423f444430\" (UID: \"dfdd8013-ee6e-4ec6-b5d1-fb423f444430\") " Dec 03 12:15:09 crc kubenswrapper[5052]: I1203 12:15:09.303354 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-zwj9p\" (UniqueName: \"kubernetes.io/projected/b853a2c2-337f-423e-84bf-ea2814cf59da-kube-api-access-zwj9p\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:09 crc kubenswrapper[5052]: I1203 12:15:09.303375 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/b853a2c2-337f-423e-84bf-ea2814cf59da-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:09 crc kubenswrapper[5052]: I1203 12:15:09.316944 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a15b6144-952c-4d41-a141-5bca5c7fd402-kube-api-access-vvbz9" (OuterVolumeSpecName: "kube-api-access-vvbz9") pod "a15b6144-952c-4d41-a141-5bca5c7fd402" (UID: "a15b6144-952c-4d41-a141-5bca5c7fd402"). InnerVolumeSpecName "kube-api-access-vvbz9". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:15:09 crc kubenswrapper[5052]: E1203 12:15:09.352656 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="28049a9b7828f990e7cdf628c7744bd85e07ead3fc6256992ef7d5bf23e045a3" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:15:09 crc kubenswrapper[5052]: E1203 12:15:09.360644 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="28049a9b7828f990e7cdf628c7744bd85e07ead3fc6256992ef7d5bf23e045a3" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:15:09 crc kubenswrapper[5052]: E1203 12:15:09.362149 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="28049a9b7828f990e7cdf628c7744bd85e07ead3fc6256992ef7d5bf23e045a3" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:15:09 crc kubenswrapper[5052]: E1203 12:15:09.362191 5052 prober.go:104] "Probe errored" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" probeType="Readiness" pod="openstack-kuttl-tests/nova-cell1-conductor-0" podUID="c2a65caa-2eec-48af-bdad-8fc19d827867" containerName="nova-cell1-conductor-conductor" Dec 03 12:15:09 crc kubenswrapper[5052]: I1203 12:15:09.362687 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dfdd8013-ee6e-4ec6-b5d1-fb423f444430-kube-api-access-ljg7n" (OuterVolumeSpecName: "kube-api-access-ljg7n") pod "dfdd8013-ee6e-4ec6-b5d1-fb423f444430" (UID: "dfdd8013-ee6e-4ec6-b5d1-fb423f444430"). InnerVolumeSpecName "kube-api-access-ljg7n". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:15:09 crc kubenswrapper[5052]: I1203 12:15:09.411749 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-vvbz9\" (UniqueName: \"kubernetes.io/projected/a15b6144-952c-4d41-a141-5bca5c7fd402-kube-api-access-vvbz9\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:09 crc kubenswrapper[5052]: I1203 12:15:09.411790 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-ljg7n\" (UniqueName: \"kubernetes.io/projected/dfdd8013-ee6e-4ec6-b5d1-fb423f444430-kube-api-access-ljg7n\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:09 crc kubenswrapper[5052]: E1203 12:15:09.411901 5052 projected.go:263] Couldn't get secret openstack-kuttl-tests/swift-proxy-config-data: secret "swift-proxy-config-data" not found Dec 03 12:15:09 crc kubenswrapper[5052]: E1203 12:15:09.411919 5052 projected.go:263] Couldn't get secret openstack-kuttl-tests/swift-conf: secret "swift-conf" not found Dec 03 12:15:09 crc kubenswrapper[5052]: E1203 12:15:09.411931 5052 projected.go:288] Couldn't get configMap openstack-kuttl-tests/swift-ring-files: configmap "swift-ring-files" not found Dec 03 12:15:09 crc kubenswrapper[5052]: E1203 12:15:09.411944 5052 projected.go:194] Error preparing data for projected volume etc-swift for pod openstack-kuttl-tests/swift-proxy-6b68c4db5d-g8nvc: [secret "swift-proxy-config-data" not found, secret "swift-conf" not found, configmap "swift-ring-files" not found] Dec 03 12:15:09 crc kubenswrapper[5052]: E1203 12:15:09.412002 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ae6940a4-0f4c-46ab-8218-56bf5c5d3084-etc-swift podName:ae6940a4-0f4c-46ab-8218-56bf5c5d3084 nodeName:}" failed. No retries permitted until 2025-12-03 12:15:11.411982548 +0000 UTC m=+3302.000224312 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "etc-swift" (UniqueName: "kubernetes.io/projected/ae6940a4-0f4c-46ab-8218-56bf5c5d3084-etc-swift") pod "swift-proxy-6b68c4db5d-g8nvc" (UID: "ae6940a4-0f4c-46ab-8218-56bf5c5d3084") : [secret "swift-proxy-config-data" not found, secret "swift-conf" not found, configmap "swift-ring-files" not found] Dec 03 12:15:09 crc kubenswrapper[5052]: I1203 12:15:09.423454 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a15b6144-952c-4d41-a141-5bca5c7fd402-config-data" (OuterVolumeSpecName: "config-data") pod "a15b6144-952c-4d41-a141-5bca5c7fd402" (UID: "a15b6144-952c-4d41-a141-5bca5c7fd402"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:15:09 crc kubenswrapper[5052]: I1203 12:15:09.456679 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a15b6144-952c-4d41-a141-5bca5c7fd402-vencrypt-tls-certs" (OuterVolumeSpecName: "vencrypt-tls-certs") pod "a15b6144-952c-4d41-a141-5bca5c7fd402" (UID: "a15b6144-952c-4d41-a141-5bca5c7fd402"). InnerVolumeSpecName "vencrypt-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:15:09 crc kubenswrapper[5052]: I1203 12:15:09.473621 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dfdd8013-ee6e-4ec6-b5d1-fb423f444430-kube-state-metrics-tls-config" (OuterVolumeSpecName: "kube-state-metrics-tls-config") pod "dfdd8013-ee6e-4ec6-b5d1-fb423f444430" (UID: "dfdd8013-ee6e-4ec6-b5d1-fb423f444430"). InnerVolumeSpecName "kube-state-metrics-tls-config". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:15:09 crc kubenswrapper[5052]: I1203 12:15:09.476590 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a15b6144-952c-4d41-a141-5bca5c7fd402-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "a15b6144-952c-4d41-a141-5bca5c7fd402" (UID: "a15b6144-952c-4d41-a141-5bca5c7fd402"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:15:09 crc kubenswrapper[5052]: I1203 12:15:09.493726 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a15b6144-952c-4d41-a141-5bca5c7fd402-nova-novncproxy-tls-certs" (OuterVolumeSpecName: "nova-novncproxy-tls-certs") pod "a15b6144-952c-4d41-a141-5bca5c7fd402" (UID: "a15b6144-952c-4d41-a141-5bca5c7fd402"). InnerVolumeSpecName "nova-novncproxy-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:15:09 crc kubenswrapper[5052]: I1203 12:15:09.493799 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dfdd8013-ee6e-4ec6-b5d1-fb423f444430-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "dfdd8013-ee6e-4ec6-b5d1-fb423f444430" (UID: "dfdd8013-ee6e-4ec6-b5d1-fb423f444430"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:15:09 crc kubenswrapper[5052]: I1203 12:15:09.513648 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a15b6144-952c-4d41-a141-5bca5c7fd402-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:09 crc kubenswrapper[5052]: I1203 12:15:09.513675 5052 reconciler_common.go:293] "Volume detached for volume \"vencrypt-tls-certs\" (UniqueName: \"kubernetes.io/secret/a15b6144-952c-4d41-a141-5bca5c7fd402-vencrypt-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:09 crc kubenswrapper[5052]: I1203 12:15:09.513687 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/dfdd8013-ee6e-4ec6-b5d1-fb423f444430-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:09 crc kubenswrapper[5052]: I1203 12:15:09.513696 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a15b6144-952c-4d41-a141-5bca5c7fd402-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:09 crc kubenswrapper[5052]: I1203 12:15:09.513706 5052 reconciler_common.go:293] "Volume detached for volume \"nova-novncproxy-tls-certs\" (UniqueName: \"kubernetes.io/secret/a15b6144-952c-4d41-a141-5bca5c7fd402-nova-novncproxy-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:09 crc kubenswrapper[5052]: I1203 12:15:09.513718 5052 reconciler_common.go:293] "Volume detached for volume \"kube-state-metrics-tls-config\" (UniqueName: \"kubernetes.io/secret/dfdd8013-ee6e-4ec6-b5d1-fb423f444430-kube-state-metrics-tls-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:09 crc kubenswrapper[5052]: I1203 12:15:09.577780 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/novacell00bb5-account-delete-r2csc"] Dec 03 12:15:09 crc kubenswrapper[5052]: I1203 12:15:09.598350 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/novaapi06eb-account-delete-kvlp2"] Dec 03 12:15:09 crc kubenswrapper[5052]: I1203 12:15:09.665133 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dfdd8013-ee6e-4ec6-b5d1-fb423f444430-kube-state-metrics-tls-certs" (OuterVolumeSpecName: "kube-state-metrics-tls-certs") pod "dfdd8013-ee6e-4ec6-b5d1-fb423f444430" (UID: "dfdd8013-ee6e-4ec6-b5d1-fb423f444430"). InnerVolumeSpecName "kube-state-metrics-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:15:09 crc kubenswrapper[5052]: I1203 12:15:09.719757 5052 reconciler_common.go:293] "Volume detached for volume \"kube-state-metrics-tls-certs\" (UniqueName: \"kubernetes.io/secret/dfdd8013-ee6e-4ec6-b5d1-fb423f444430-kube-state-metrics-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:09 crc kubenswrapper[5052]: I1203 12:15:09.769117 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0a2de558-2a4b-46ef-9b53-cada8cd466c4" path="/var/lib/kubelet/pods/0a2de558-2a4b-46ef-9b53-cada8cd466c4/volumes" Dec 03 12:15:09 crc kubenswrapper[5052]: I1203 12:15:09.772056 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4e574f0c-bbe2-44a9-935c-923622c44443" path="/var/lib/kubelet/pods/4e574f0c-bbe2-44a9-935c-923622c44443/volumes" Dec 03 12:15:09 crc kubenswrapper[5052]: I1203 12:15:09.774124 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b853a2c2-337f-423e-84bf-ea2814cf59da" path="/var/lib/kubelet/pods/b853a2c2-337f-423e-84bf-ea2814cf59da/volumes" Dec 03 12:15:09 crc kubenswrapper[5052]: I1203 12:15:09.826235 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="baa1c9d4-5887-47e3-be99-96f718178f58" path="/var/lib/kubelet/pods/baa1c9d4-5887-47e3-be99-96f718178f58/volumes" Dec 03 12:15:09 crc kubenswrapper[5052]: I1203 12:15:09.827721 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder7878-account-delete-jwtvt"] Dec 03 12:15:09 crc kubenswrapper[5052]: I1203 12:15:09.980647 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/nova-metadata-0" podUID="cd486343-0aad-4039-a23b-34c60cd1a381" containerName="nova-metadata-log" probeResult="failure" output="Get \"https://10.217.1.43:8775/\": read tcp 10.217.0.2:52368->10.217.1.43:8775: read: connection reset by peer" Dec 03 12:15:09 crc kubenswrapper[5052]: I1203 12:15:09.981004 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/nova-metadata-0" podUID="cd486343-0aad-4039-a23b-34c60cd1a381" containerName="nova-metadata-metadata" probeResult="failure" output="Get \"https://10.217.1.43:8775/\": read tcp 10.217.0.2:52366->10.217.1.43:8775: read: connection reset by peer" Dec 03 12:15:09 crc kubenswrapper[5052]: I1203 12:15:09.992072 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder7878-account-delete-jwtvt" event={"ID":"679f17b9-0e40-480b-bfde-4f63899cb972","Type":"ContainerStarted","Data":"bb3bc4ace76df2b60f85d9bcc20c9ae03f8246692b64143eca974b94cb127a47"} Dec 03 12:15:10 crc kubenswrapper[5052]: I1203 12:15:10.009171 5052 generic.go:334] "Generic (PLEG): container finished" podID="6099fea1-bbb4-4413-a826-539b3dfc3eec" containerID="998bba76a5f7f109bda9a23387d0dfd8298ff434e5f70a2d60d2a1801c384af4" exitCode=0 Dec 03 12:15:10 crc kubenswrapper[5052]: I1203 12:15:10.009314 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement7cee-account-delete-7wgkx" event={"ID":"6099fea1-bbb4-4413-a826-539b3dfc3eec","Type":"ContainerDied","Data":"998bba76a5f7f109bda9a23387d0dfd8298ff434e5f70a2d60d2a1801c384af4"} Dec 03 12:15:10 crc kubenswrapper[5052]: I1203 12:15:10.021185 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/novaapi06eb-account-delete-kvlp2" event={"ID":"72a72aa7-f35b-4b95-b5fd-caaf1272943a","Type":"ContainerStarted","Data":"1b718b05fd3c5f8e7a2449b1d22b045b8adb4bf91592bb2a833f1416926ad385"} Dec 03 12:15:10 crc kubenswrapper[5052]: I1203 12:15:10.031835 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:15:10 crc kubenswrapper[5052]: I1203 12:15:10.034869 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" event={"ID":"a15b6144-952c-4d41-a141-5bca5c7fd402","Type":"ContainerDied","Data":"961f5bdb73f99eb227d2d7e24cb06ea807c3c40817f760ff50342f68d207b2a7"} Dec 03 12:15:10 crc kubenswrapper[5052]: I1203 12:15:10.034941 5052 scope.go:117] "RemoveContainer" containerID="6b64e3ed870b9dc27d7d4df12ca7a0eb2dff4a0cc61910bf73a9e7a4b368dedc" Dec 03 12:15:10 crc kubenswrapper[5052]: I1203 12:15:10.043832 5052 generic.go:334] "Generic (PLEG): container finished" podID="fee7d732-4fa3-40d5-8fff-cb88272db0ce" containerID="e012c7962e8153da90fe1b2a14dc53f9e2c75d7dcb586b6279ae4612ffd6977b" exitCode=0 Dec 03 12:15:10 crc kubenswrapper[5052]: I1203 12:15:10.044056 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glancef253-account-delete-cxxlg" event={"ID":"fee7d732-4fa3-40d5-8fff-cb88272db0ce","Type":"ContainerDied","Data":"e012c7962e8153da90fe1b2a14dc53f9e2c75d7dcb586b6279ae4612ffd6977b"} Dec 03 12:15:10 crc kubenswrapper[5052]: I1203 12:15:10.054044 5052 generic.go:334] "Generic (PLEG): container finished" podID="d81b09dc-75a1-45f8-ace4-786531ad3bfd" containerID="e29fe95d4add082f6c46305d5e3fe12f454dfccd4b1d5c2355d53782ac82fa29" exitCode=0 Dec 03 12:15:10 crc kubenswrapper[5052]: I1203 12:15:10.054183 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"d81b09dc-75a1-45f8-ace4-786531ad3bfd","Type":"ContainerDied","Data":"e29fe95d4add082f6c46305d5e3fe12f454dfccd4b1d5c2355d53782ac82fa29"} Dec 03 12:15:10 crc kubenswrapper[5052]: I1203 12:15:10.064568 5052 generic.go:334] "Generic (PLEG): container finished" podID="8283de30-8976-47bd-bd09-aab53140f68d" containerID="2814b190b744e9cf8edd55553cf57bc90eebd7268c2360b6efc06f7b369e975c" exitCode=0 Dec 03 12:15:10 crc kubenswrapper[5052]: I1203 12:15:10.064677 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-79d5cb4fdb-7h2fn" event={"ID":"8283de30-8976-47bd-bd09-aab53140f68d","Type":"ContainerDied","Data":"2814b190b744e9cf8edd55553cf57bc90eebd7268c2360b6efc06f7b369e975c"} Dec 03 12:15:10 crc kubenswrapper[5052]: I1203 12:15:10.068966 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/novacell00bb5-account-delete-r2csc" event={"ID":"c200c246-74d3-48dc-8b24-ac4b37611069","Type":"ContainerStarted","Data":"b30037926737b8288cb289a95633b6703674d732f8671e54cacf83c371b2ed55"} Dec 03 12:15:10 crc kubenswrapper[5052]: I1203 12:15:10.070115 5052 generic.go:334] "Generic (PLEG): container finished" podID="f1619fbf-df68-484f-99d4-09b81a7a334e" containerID="ad600490ee3191e301f7e30a3ba1fcff1867ac26a31d4e56b6e4a17fc40b417b" exitCode=0 Dec 03 12:15:10 crc kubenswrapper[5052]: I1203 12:15:10.070167 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron1287-account-delete-t8dcs" event={"ID":"f1619fbf-df68-484f-99d4-09b81a7a334e","Type":"ContainerDied","Data":"ad600490ee3191e301f7e30a3ba1fcff1867ac26a31d4e56b6e4a17fc40b417b"} Dec 03 12:15:10 crc kubenswrapper[5052]: I1203 12:15:10.077675 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/kube-state-metrics-0" event={"ID":"dfdd8013-ee6e-4ec6-b5d1-fb423f444430","Type":"ContainerDied","Data":"ca527bbb6d8d57411900c0d35849885371f2b836f9c39e33e47863b9f3b38d61"} Dec 03 12:15:10 crc kubenswrapper[5052]: I1203 12:15:10.077779 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:15:10 crc kubenswrapper[5052]: I1203 12:15:10.105658 5052 generic.go:334] "Generic (PLEG): container finished" podID="5856107a-0498-4956-beba-4390f5477ca0" containerID="67c22604bca84b1f1e31bb59eb85a540591562d9df9164cbcf210f964d1d5beb" exitCode=0 Dec 03 12:15:10 crc kubenswrapper[5052]: I1203 12:15:10.105742 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican8992-account-delete-26lhs" event={"ID":"5856107a-0498-4956-beba-4390f5477ca0","Type":"ContainerDied","Data":"67c22604bca84b1f1e31bb59eb85a540591562d9df9164cbcf210f964d1d5beb"} Dec 03 12:15:10 crc kubenswrapper[5052]: I1203 12:15:10.112748 5052 generic.go:334] "Generic (PLEG): container finished" podID="ebf52f1d-a87e-4300-a650-4c4cb36e80d2" containerID="679522668d46d6b55c6186c11c5c0d53cb5cd3280e0e0dec04e125121ba4047a" exitCode=0 Dec 03 12:15:10 crc kubenswrapper[5052]: I1203 12:15:10.112817 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-kl2c8" event={"ID":"ebf52f1d-a87e-4300-a650-4c4cb36e80d2","Type":"ContainerDied","Data":"679522668d46d6b55c6186c11c5c0d53cb5cd3280e0e0dec04e125121ba4047a"} Dec 03 12:15:10 crc kubenswrapper[5052]: I1203 12:15:10.112847 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-kl2c8" event={"ID":"ebf52f1d-a87e-4300-a650-4c4cb36e80d2","Type":"ContainerStarted","Data":"255fcc6be8986f3475fa6b9c263e74d466e135cd4ba0b85c82b314b0abb90ee9"} Dec 03 12:15:10 crc kubenswrapper[5052]: I1203 12:15:10.159766 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/cinder-api-0" podUID="388bc903-097f-4f97-8f19-2f7bcdb49cc5" containerName="cinder-api" probeResult="failure" output="Get \"https://10.217.1.2:8776/healthcheck\": read tcp 10.217.0.2:54548->10.217.1.2:8776: read: connection reset by peer" Dec 03 12:15:10 crc kubenswrapper[5052]: I1203 12:15:10.169279 5052 generic.go:334] "Generic (PLEG): container finished" podID="1661efff-d6ce-479d-bfb9-90849fa6ff40" containerID="e964aba7c310e135f899bcb05dcd28df12a6cfb568f657650c49176388d578b0" exitCode=0 Dec 03 12:15:10 crc kubenswrapper[5052]: I1203 12:15:10.169357 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"1661efff-d6ce-479d-bfb9-90849fa6ff40","Type":"ContainerDied","Data":"e964aba7c310e135f899bcb05dcd28df12a6cfb568f657650c49176388d578b0"} Dec 03 12:15:10 crc kubenswrapper[5052]: I1203 12:15:10.172821 5052 generic.go:334] "Generic (PLEG): container finished" podID="ae6940a4-0f4c-46ab-8218-56bf5c5d3084" containerID="6d0e263d1fbe61d44b6dd94d1a87bc31a61607f209df9eb30c44c6f6386c7381" exitCode=0 Dec 03 12:15:10 crc kubenswrapper[5052]: I1203 12:15:10.172856 5052 generic.go:334] "Generic (PLEG): container finished" podID="ae6940a4-0f4c-46ab-8218-56bf5c5d3084" containerID="22c61e9465e05e82a9b75ec36f8899daba6ae848986e069b783febfc60abd647" exitCode=0 Dec 03 12:15:10 crc kubenswrapper[5052]: I1203 12:15:10.172911 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-proxy-6b68c4db5d-g8nvc" event={"ID":"ae6940a4-0f4c-46ab-8218-56bf5c5d3084","Type":"ContainerDied","Data":"6d0e263d1fbe61d44b6dd94d1a87bc31a61607f209df9eb30c44c6f6386c7381"} Dec 03 12:15:10 crc kubenswrapper[5052]: I1203 12:15:10.172937 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-proxy-6b68c4db5d-g8nvc" event={"ID":"ae6940a4-0f4c-46ab-8218-56bf5c5d3084","Type":"ContainerDied","Data":"22c61e9465e05e82a9b75ec36f8899daba6ae848986e069b783febfc60abd647"} Dec 03 12:15:10 crc kubenswrapper[5052]: I1203 12:15:10.199247 5052 generic.go:334] "Generic (PLEG): container finished" podID="2a349843-a7b7-4dff-a5ae-f513337c5809" containerID="f30fe921af5f7b637639949f8977c3720f1ebb48e3b4f39277fcbc953207de23" exitCode=0 Dec 03 12:15:10 crc kubenswrapper[5052]: I1203 12:15:10.199430 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"2a349843-a7b7-4dff-a5ae-f513337c5809","Type":"ContainerDied","Data":"f30fe921af5f7b637639949f8977c3720f1ebb48e3b4f39277fcbc953207de23"} Dec 03 12:15:10 crc kubenswrapper[5052]: I1203 12:15:10.225930 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-cell1-galera-0" event={"ID":"00a9935f-ef2d-4976-97a0-78ee6325433d","Type":"ContainerDied","Data":"aabef9c4a1cc4f1273de63111a3705eddf616b50266ee22fb415a42c62f1f823"} Dec 03 12:15:10 crc kubenswrapper[5052]: I1203 12:15:10.225983 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="aabef9c4a1cc4f1273de63111a3705eddf616b50266ee22fb415a42c62f1f823" Dec 03 12:15:10 crc kubenswrapper[5052]: I1203 12:15:10.372710 5052 scope.go:117] "RemoveContainer" containerID="8021f4fcdb6f78190623eed4c27195e38d52e80b4781dc680a2591d5bf74777d" Dec 03 12:15:10 crc kubenswrapper[5052]: I1203 12:15:10.407237 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/barbican-api-58464649dd-sx8mw" podUID="b25d2a0c-1e93-4b3d-965a-a7c021f9649d" containerName="barbican-api-log" probeResult="failure" output="Get \"https://10.217.0.254:9311/healthcheck\": read tcp 10.217.0.2:46944->10.217.0.254:9311: read: connection reset by peer" Dec 03 12:15:10 crc kubenswrapper[5052]: I1203 12:15:10.407262 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/barbican-api-58464649dd-sx8mw" podUID="b25d2a0c-1e93-4b3d-965a-a7c021f9649d" containerName="barbican-api" probeResult="failure" output="Get \"https://10.217.0.254:9311/healthcheck\": read tcp 10.217.0.2:46958->10.217.0.254:9311: read: connection reset by peer" Dec 03 12:15:10 crc kubenswrapper[5052]: I1203 12:15:10.460679 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:15:10 crc kubenswrapper[5052]: I1203 12:15:10.671479 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"mysql-db\" (UniqueName: \"kubernetes.io/local-volume/local-storage06-crc\") pod \"00a9935f-ef2d-4976-97a0-78ee6325433d\" (UID: \"00a9935f-ef2d-4976-97a0-78ee6325433d\") " Dec 03 12:15:10 crc kubenswrapper[5052]: I1203 12:15:10.671609 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gd9zs\" (UniqueName: \"kubernetes.io/projected/00a9935f-ef2d-4976-97a0-78ee6325433d-kube-api-access-gd9zs\") pod \"00a9935f-ef2d-4976-97a0-78ee6325433d\" (UID: \"00a9935f-ef2d-4976-97a0-78ee6325433d\") " Dec 03 12:15:10 crc kubenswrapper[5052]: I1203 12:15:10.671651 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/00a9935f-ef2d-4976-97a0-78ee6325433d-galera-tls-certs\") pod \"00a9935f-ef2d-4976-97a0-78ee6325433d\" (UID: \"00a9935f-ef2d-4976-97a0-78ee6325433d\") " Dec 03 12:15:10 crc kubenswrapper[5052]: I1203 12:15:10.671732 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/00a9935f-ef2d-4976-97a0-78ee6325433d-config-data-default\") pod \"00a9935f-ef2d-4976-97a0-78ee6325433d\" (UID: \"00a9935f-ef2d-4976-97a0-78ee6325433d\") " Dec 03 12:15:10 crc kubenswrapper[5052]: I1203 12:15:10.671764 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/00a9935f-ef2d-4976-97a0-78ee6325433d-combined-ca-bundle\") pod \"00a9935f-ef2d-4976-97a0-78ee6325433d\" (UID: \"00a9935f-ef2d-4976-97a0-78ee6325433d\") " Dec 03 12:15:10 crc kubenswrapper[5052]: I1203 12:15:10.671801 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/00a9935f-ef2d-4976-97a0-78ee6325433d-kolla-config\") pod \"00a9935f-ef2d-4976-97a0-78ee6325433d\" (UID: \"00a9935f-ef2d-4976-97a0-78ee6325433d\") " Dec 03 12:15:10 crc kubenswrapper[5052]: I1203 12:15:10.671900 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/00a9935f-ef2d-4976-97a0-78ee6325433d-config-data-generated\") pod \"00a9935f-ef2d-4976-97a0-78ee6325433d\" (UID: \"00a9935f-ef2d-4976-97a0-78ee6325433d\") " Dec 03 12:15:10 crc kubenswrapper[5052]: I1203 12:15:10.671979 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/00a9935f-ef2d-4976-97a0-78ee6325433d-operator-scripts\") pod \"00a9935f-ef2d-4976-97a0-78ee6325433d\" (UID: \"00a9935f-ef2d-4976-97a0-78ee6325433d\") " Dec 03 12:15:10 crc kubenswrapper[5052]: I1203 12:15:10.673186 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/00a9935f-ef2d-4976-97a0-78ee6325433d-config-data-default" (OuterVolumeSpecName: "config-data-default") pod "00a9935f-ef2d-4976-97a0-78ee6325433d" (UID: "00a9935f-ef2d-4976-97a0-78ee6325433d"). InnerVolumeSpecName "config-data-default". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:15:10 crc kubenswrapper[5052]: I1203 12:15:10.673863 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/00a9935f-ef2d-4976-97a0-78ee6325433d-kolla-config" (OuterVolumeSpecName: "kolla-config") pod "00a9935f-ef2d-4976-97a0-78ee6325433d" (UID: "00a9935f-ef2d-4976-97a0-78ee6325433d"). InnerVolumeSpecName "kolla-config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:15:10 crc kubenswrapper[5052]: I1203 12:15:10.674201 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/00a9935f-ef2d-4976-97a0-78ee6325433d-config-data-generated" (OuterVolumeSpecName: "config-data-generated") pod "00a9935f-ef2d-4976-97a0-78ee6325433d" (UID: "00a9935f-ef2d-4976-97a0-78ee6325433d"). InnerVolumeSpecName "config-data-generated". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:15:10 crc kubenswrapper[5052]: I1203 12:15:10.684359 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/00a9935f-ef2d-4976-97a0-78ee6325433d-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "00a9935f-ef2d-4976-97a0-78ee6325433d" (UID: "00a9935f-ef2d-4976-97a0-78ee6325433d"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:15:10 crc kubenswrapper[5052]: I1203 12:15:10.705076 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/00a9935f-ef2d-4976-97a0-78ee6325433d-kube-api-access-gd9zs" (OuterVolumeSpecName: "kube-api-access-gd9zs") pod "00a9935f-ef2d-4976-97a0-78ee6325433d" (UID: "00a9935f-ef2d-4976-97a0-78ee6325433d"). InnerVolumeSpecName "kube-api-access-gd9zs". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:15:10 crc kubenswrapper[5052]: I1203 12:15:10.721188 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage06-crc" (OuterVolumeSpecName: "mysql-db") pod "00a9935f-ef2d-4976-97a0-78ee6325433d" (UID: "00a9935f-ef2d-4976-97a0-78ee6325433d"). InnerVolumeSpecName "local-storage06-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 12:15:10 crc kubenswrapper[5052]: I1203 12:15:10.722670 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/00a9935f-ef2d-4976-97a0-78ee6325433d-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "00a9935f-ef2d-4976-97a0-78ee6325433d" (UID: "00a9935f-ef2d-4976-97a0-78ee6325433d"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:15:10 crc kubenswrapper[5052]: E1203 12:15:10.724705 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="98d14ce0cb9919bb83964fb391e45a00edf1d049296adef59b4e2e0f042c63d2" cmd=["/usr/local/bin/container-scripts/status_check.sh"] Dec 03 12:15:10 crc kubenswrapper[5052]: E1203 12:15:10.734654 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="98d14ce0cb9919bb83964fb391e45a00edf1d049296adef59b4e2e0f042c63d2" cmd=["/usr/local/bin/container-scripts/status_check.sh"] Dec 03 12:15:10 crc kubenswrapper[5052]: E1203 12:15:10.745835 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="98d14ce0cb9919bb83964fb391e45a00edf1d049296adef59b4e2e0f042c63d2" cmd=["/usr/local/bin/container-scripts/status_check.sh"] Dec 03 12:15:10 crc kubenswrapper[5052]: E1203 12:15:10.745876 5052 prober.go:104] "Probe errored" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" probeType="Readiness" pod="openstack-kuttl-tests/ovn-northd-0" podUID="00ab3478-6308-427a-8e34-4133611a90b2" containerName="ovn-northd" Dec 03 12:15:10 crc kubenswrapper[5052]: I1203 12:15:10.772590 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/00a9935f-ef2d-4976-97a0-78ee6325433d-galera-tls-certs" (OuterVolumeSpecName: "galera-tls-certs") pod "00a9935f-ef2d-4976-97a0-78ee6325433d" (UID: "00a9935f-ef2d-4976-97a0-78ee6325433d"). InnerVolumeSpecName "galera-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:15:10 crc kubenswrapper[5052]: I1203 12:15:10.774448 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage06-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage06-crc\") on node \"crc\" " Dec 03 12:15:10 crc kubenswrapper[5052]: I1203 12:15:10.774559 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-gd9zs\" (UniqueName: \"kubernetes.io/projected/00a9935f-ef2d-4976-97a0-78ee6325433d-kube-api-access-gd9zs\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:10 crc kubenswrapper[5052]: I1203 12:15:10.774641 5052 reconciler_common.go:293] "Volume detached for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/00a9935f-ef2d-4976-97a0-78ee6325433d-galera-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:10 crc kubenswrapper[5052]: I1203 12:15:10.774697 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/00a9935f-ef2d-4976-97a0-78ee6325433d-config-data-default\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:10 crc kubenswrapper[5052]: I1203 12:15:10.774759 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/00a9935f-ef2d-4976-97a0-78ee6325433d-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:10 crc kubenswrapper[5052]: I1203 12:15:10.774818 5052 reconciler_common.go:293] "Volume detached for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/00a9935f-ef2d-4976-97a0-78ee6325433d-kolla-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:10 crc kubenswrapper[5052]: I1203 12:15:10.774870 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/00a9935f-ef2d-4976-97a0-78ee6325433d-config-data-generated\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:10 crc kubenswrapper[5052]: I1203 12:15:10.774923 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/00a9935f-ef2d-4976-97a0-78ee6325433d-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:10 crc kubenswrapper[5052]: I1203 12:15:10.797702 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage06-crc" (UniqueName: "kubernetes.io/local-volume/local-storage06-crc") on node "crc" Dec 03 12:15:10 crc kubenswrapper[5052]: I1203 12:15:10.879218 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/memcached-0"] Dec 03 12:15:10 crc kubenswrapper[5052]: I1203 12:15:10.879833 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/memcached-0" podUID="b28a125a-7966-4f97-91c1-fa52d6062405" containerName="memcached" containerID="cri-o://dc5c838958d7fbbc47d97aabd5b021dc0188e172b8b2ed5ab168cef70e171e07" gracePeriod=30 Dec 03 12:15:10 crc kubenswrapper[5052]: I1203 12:15:10.883605 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage06-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage06-crc\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:10 crc kubenswrapper[5052]: I1203 12:15:10.954664 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/keystone-db-sync-95hjc"] Dec 03 12:15:10 crc kubenswrapper[5052]: I1203 12:15:10.955683 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-proxy-6b68c4db5d-g8nvc" Dec 03 12:15:10 crc kubenswrapper[5052]: I1203 12:15:10.970889 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/keystone-db-sync-95hjc"] Dec 03 12:15:10 crc kubenswrapper[5052]: I1203 12:15:10.980241 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/keystone-bootstrap-v44q4"] Dec 03 12:15:10 crc kubenswrapper[5052]: I1203 12:15:10.984388 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n4nmc\" (UniqueName: \"kubernetes.io/projected/ae6940a4-0f4c-46ab-8218-56bf5c5d3084-kube-api-access-n4nmc\") pod \"ae6940a4-0f4c-46ab-8218-56bf5c5d3084\" (UID: \"ae6940a4-0f4c-46ab-8218-56bf5c5d3084\") " Dec 03 12:15:10 crc kubenswrapper[5052]: I1203 12:15:10.984586 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/ae6940a4-0f4c-46ab-8218-56bf5c5d3084-run-httpd\") pod \"ae6940a4-0f4c-46ab-8218-56bf5c5d3084\" (UID: \"ae6940a4-0f4c-46ab-8218-56bf5c5d3084\") " Dec 03 12:15:10 crc kubenswrapper[5052]: I1203 12:15:10.984640 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/ae6940a4-0f4c-46ab-8218-56bf5c5d3084-config-data\") pod \"ae6940a4-0f4c-46ab-8218-56bf5c5d3084\" (UID: \"ae6940a4-0f4c-46ab-8218-56bf5c5d3084\") " Dec 03 12:15:10 crc kubenswrapper[5052]: I1203 12:15:10.984682 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ae6940a4-0f4c-46ab-8218-56bf5c5d3084-combined-ca-bundle\") pod \"ae6940a4-0f4c-46ab-8218-56bf5c5d3084\" (UID: \"ae6940a4-0f4c-46ab-8218-56bf5c5d3084\") " Dec 03 12:15:10 crc kubenswrapper[5052]: I1203 12:15:10.984713 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/ae6940a4-0f4c-46ab-8218-56bf5c5d3084-etc-swift\") pod \"ae6940a4-0f4c-46ab-8218-56bf5c5d3084\" (UID: \"ae6940a4-0f4c-46ab-8218-56bf5c5d3084\") " Dec 03 12:15:10 crc kubenswrapper[5052]: I1203 12:15:10.984756 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/ae6940a4-0f4c-46ab-8218-56bf5c5d3084-internal-tls-certs\") pod \"ae6940a4-0f4c-46ab-8218-56bf5c5d3084\" (UID: \"ae6940a4-0f4c-46ab-8218-56bf5c5d3084\") " Dec 03 12:15:10 crc kubenswrapper[5052]: I1203 12:15:10.984773 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/ae6940a4-0f4c-46ab-8218-56bf5c5d3084-log-httpd\") pod \"ae6940a4-0f4c-46ab-8218-56bf5c5d3084\" (UID: \"ae6940a4-0f4c-46ab-8218-56bf5c5d3084\") " Dec 03 12:15:10 crc kubenswrapper[5052]: I1203 12:15:10.984797 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/ae6940a4-0f4c-46ab-8218-56bf5c5d3084-public-tls-certs\") pod \"ae6940a4-0f4c-46ab-8218-56bf5c5d3084\" (UID: \"ae6940a4-0f4c-46ab-8218-56bf5c5d3084\") " Dec 03 12:15:10 crc kubenswrapper[5052]: I1203 12:15:10.985812 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae6940a4-0f4c-46ab-8218-56bf5c5d3084-run-httpd" (OuterVolumeSpecName: "run-httpd") pod "ae6940a4-0f4c-46ab-8218-56bf5c5d3084" (UID: "ae6940a4-0f4c-46ab-8218-56bf5c5d3084"). InnerVolumeSpecName "run-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:15:10 crc kubenswrapper[5052]: I1203 12:15:10.988717 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae6940a4-0f4c-46ab-8218-56bf5c5d3084-log-httpd" (OuterVolumeSpecName: "log-httpd") pod "ae6940a4-0f4c-46ab-8218-56bf5c5d3084" (UID: "ae6940a4-0f4c-46ab-8218-56bf5c5d3084"). InnerVolumeSpecName "log-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:15:10 crc kubenswrapper[5052]: I1203 12:15:10.988958 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:15:10 crc kubenswrapper[5052]: I1203 12:15:10.996649 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/keystone-bootstrap-v44q4"] Dec 03 12:15:10 crc kubenswrapper[5052]: I1203 12:15:10.998033 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement7cee-account-delete-7wgkx" Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.003409 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ae6940a4-0f4c-46ab-8218-56bf5c5d3084-etc-swift" (OuterVolumeSpecName: "etc-swift") pod "ae6940a4-0f4c-46ab-8218-56bf5c5d3084" (UID: "ae6940a4-0f4c-46ab-8218-56bf5c5d3084"). InnerVolumeSpecName "etc-swift". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.008046 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/keystone-6fd4994758-fsw2c"] Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.008274 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/keystone-6fd4994758-fsw2c" podUID="e14dd2e3-f6e1-450c-a688-1dc4eb2ee622" containerName="keystone-api" containerID="cri-o://6544c0c4bb0b5942eaa383c638931be4bd4915098254d54ab63534949d4fc2c6" gracePeriod=30 Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.013217 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.018059 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/kube-state-metrics-0"] Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.025936 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/kube-state-metrics-0"] Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.031562 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ae6940a4-0f4c-46ab-8218-56bf5c5d3084-kube-api-access-n4nmc" (OuterVolumeSpecName: "kube-api-access-n4nmc") pod "ae6940a4-0f4c-46ab-8218-56bf5c5d3084" (UID: "ae6940a4-0f4c-46ab-8218-56bf5c5d3084"). InnerVolumeSpecName "kube-api-access-n4nmc". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.033983 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/openstack-galera-0"] Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.050421 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.052991 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-79d5cb4fdb-7h2fn" Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.075785 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.085600 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kng9h\" (UniqueName: \"kubernetes.io/projected/d81b09dc-75a1-45f8-ace4-786531ad3bfd-kube-api-access-kng9h\") pod \"d81b09dc-75a1-45f8-ace4-786531ad3bfd\" (UID: \"d81b09dc-75a1-45f8-ace4-786531ad3bfd\") " Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.085643 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/8283de30-8976-47bd-bd09-aab53140f68d-public-tls-certs\") pod \"8283de30-8976-47bd-bd09-aab53140f68d\" (UID: \"8283de30-8976-47bd-bd09-aab53140f68d\") " Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.085661 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/8283de30-8976-47bd-bd09-aab53140f68d-scripts\") pod \"8283de30-8976-47bd-bd09-aab53140f68d\" (UID: \"8283de30-8976-47bd-bd09-aab53140f68d\") " Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.085690 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rzrj5\" (UniqueName: \"kubernetes.io/projected/2a349843-a7b7-4dff-a5ae-f513337c5809-kube-api-access-rzrj5\") pod \"2a349843-a7b7-4dff-a5ae-f513337c5809\" (UID: \"2a349843-a7b7-4dff-a5ae-f513337c5809\") " Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.085707 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/2a349843-a7b7-4dff-a5ae-f513337c5809-config-data\") pod \"2a349843-a7b7-4dff-a5ae-f513337c5809\" (UID: \"2a349843-a7b7-4dff-a5ae-f513337c5809\") " Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.085727 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-skrwg\" (UniqueName: \"kubernetes.io/projected/6099fea1-bbb4-4413-a826-539b3dfc3eec-kube-api-access-skrwg\") pod \"6099fea1-bbb4-4413-a826-539b3dfc3eec\" (UID: \"6099fea1-bbb4-4413-a826-539b3dfc3eec\") " Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.085755 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/8283de30-8976-47bd-bd09-aab53140f68d-logs\") pod \"8283de30-8976-47bd-bd09-aab53140f68d\" (UID: \"8283de30-8976-47bd-bd09-aab53140f68d\") " Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.085784 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/6099fea1-bbb4-4413-a826-539b3dfc3eec-operator-scripts\") pod \"6099fea1-bbb4-4413-a826-539b3dfc3eec\" (UID: \"6099fea1-bbb4-4413-a826-539b3dfc3eec\") " Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.085809 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2a349843-a7b7-4dff-a5ae-f513337c5809-combined-ca-bundle\") pod \"2a349843-a7b7-4dff-a5ae-f513337c5809\" (UID: \"2a349843-a7b7-4dff-a5ae-f513337c5809\") " Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.085834 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8283de30-8976-47bd-bd09-aab53140f68d-combined-ca-bundle\") pod \"8283de30-8976-47bd-bd09-aab53140f68d\" (UID: \"8283de30-8976-47bd-bd09-aab53140f68d\") " Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.085853 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-69g8d\" (UniqueName: \"kubernetes.io/projected/8283de30-8976-47bd-bd09-aab53140f68d-kube-api-access-69g8d\") pod \"8283de30-8976-47bd-bd09-aab53140f68d\" (UID: \"8283de30-8976-47bd-bd09-aab53140f68d\") " Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.085872 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/d81b09dc-75a1-45f8-ace4-786531ad3bfd-logs\") pod \"d81b09dc-75a1-45f8-ace4-786531ad3bfd\" (UID: \"d81b09dc-75a1-45f8-ace4-786531ad3bfd\") " Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.085889 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/d81b09dc-75a1-45f8-ace4-786531ad3bfd-config-data\") pod \"d81b09dc-75a1-45f8-ace4-786531ad3bfd\" (UID: \"d81b09dc-75a1-45f8-ace4-786531ad3bfd\") " Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.085910 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/2a349843-a7b7-4dff-a5ae-f513337c5809-public-tls-certs\") pod \"2a349843-a7b7-4dff-a5ae-f513337c5809\" (UID: \"2a349843-a7b7-4dff-a5ae-f513337c5809\") " Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.085928 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/8283de30-8976-47bd-bd09-aab53140f68d-internal-tls-certs\") pod \"8283de30-8976-47bd-bd09-aab53140f68d\" (UID: \"8283de30-8976-47bd-bd09-aab53140f68d\") " Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.085966 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"glance\" (UniqueName: \"kubernetes.io/local-volume/local-storage03-crc\") pod \"d81b09dc-75a1-45f8-ace4-786531ad3bfd\" (UID: \"d81b09dc-75a1-45f8-ace4-786531ad3bfd\") " Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.086026 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d81b09dc-75a1-45f8-ace4-786531ad3bfd-combined-ca-bundle\") pod \"d81b09dc-75a1-45f8-ace4-786531ad3bfd\" (UID: \"d81b09dc-75a1-45f8-ace4-786531ad3bfd\") " Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.086049 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/d81b09dc-75a1-45f8-ace4-786531ad3bfd-httpd-run\") pod \"d81b09dc-75a1-45f8-ace4-786531ad3bfd\" (UID: \"d81b09dc-75a1-45f8-ace4-786531ad3bfd\") " Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.086079 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/2a349843-a7b7-4dff-a5ae-f513337c5809-logs\") pod \"2a349843-a7b7-4dff-a5ae-f513337c5809\" (UID: \"2a349843-a7b7-4dff-a5ae-f513337c5809\") " Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.086121 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/8283de30-8976-47bd-bd09-aab53140f68d-config-data\") pod \"8283de30-8976-47bd-bd09-aab53140f68d\" (UID: \"8283de30-8976-47bd-bd09-aab53140f68d\") " Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.086150 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/2a349843-a7b7-4dff-a5ae-f513337c5809-scripts\") pod \"2a349843-a7b7-4dff-a5ae-f513337c5809\" (UID: \"2a349843-a7b7-4dff-a5ae-f513337c5809\") " Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.086175 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/d81b09dc-75a1-45f8-ace4-786531ad3bfd-internal-tls-certs\") pod \"d81b09dc-75a1-45f8-ace4-786531ad3bfd\" (UID: \"d81b09dc-75a1-45f8-ace4-786531ad3bfd\") " Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.086205 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"glance\" (UniqueName: \"kubernetes.io/local-volume/local-storage20-crc\") pod \"2a349843-a7b7-4dff-a5ae-f513337c5809\" (UID: \"2a349843-a7b7-4dff-a5ae-f513337c5809\") " Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.086241 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/d81b09dc-75a1-45f8-ace4-786531ad3bfd-scripts\") pod \"d81b09dc-75a1-45f8-ace4-786531ad3bfd\" (UID: \"d81b09dc-75a1-45f8-ace4-786531ad3bfd\") " Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.086258 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/2a349843-a7b7-4dff-a5ae-f513337c5809-httpd-run\") pod \"2a349843-a7b7-4dff-a5ae-f513337c5809\" (UID: \"2a349843-a7b7-4dff-a5ae-f513337c5809\") " Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.092898 5052 reconciler_common.go:293] "Volume detached for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/ae6940a4-0f4c-46ab-8218-56bf5c5d3084-run-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.092951 5052 reconciler_common.go:293] "Volume detached for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/ae6940a4-0f4c-46ab-8218-56bf5c5d3084-etc-swift\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.093015 5052 reconciler_common.go:293] "Volume detached for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/ae6940a4-0f4c-46ab-8218-56bf5c5d3084-log-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.093027 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-n4nmc\" (UniqueName: \"kubernetes.io/projected/ae6940a4-0f4c-46ab-8218-56bf5c5d3084-kube-api-access-n4nmc\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.093988 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d81b09dc-75a1-45f8-ace4-786531ad3bfd-httpd-run" (OuterVolumeSpecName: "httpd-run") pod "d81b09dc-75a1-45f8-ace4-786531ad3bfd" (UID: "d81b09dc-75a1-45f8-ace4-786531ad3bfd"). InnerVolumeSpecName "httpd-run". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.098183 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d81b09dc-75a1-45f8-ace4-786531ad3bfd-logs" (OuterVolumeSpecName: "logs") pod "d81b09dc-75a1-45f8-ace4-786531ad3bfd" (UID: "d81b09dc-75a1-45f8-ace4-786531ad3bfd"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.101778 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2a349843-a7b7-4dff-a5ae-f513337c5809-httpd-run" (OuterVolumeSpecName: "httpd-run") pod "2a349843-a7b7-4dff-a5ae-f513337c5809" (UID: "2a349843-a7b7-4dff-a5ae-f513337c5809"). InnerVolumeSpecName "httpd-run". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.102101 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2a349843-a7b7-4dff-a5ae-f513337c5809-logs" (OuterVolumeSpecName: "logs") pod "2a349843-a7b7-4dff-a5ae-f513337c5809" (UID: "2a349843-a7b7-4dff-a5ae-f513337c5809"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.109141 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6099fea1-bbb4-4413-a826-539b3dfc3eec-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "6099fea1-bbb4-4413-a826-539b3dfc3eec" (UID: "6099fea1-bbb4-4413-a826-539b3dfc3eec"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.131386 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-api-db-create-mnlg4"] Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.134811 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8283de30-8976-47bd-bd09-aab53140f68d-logs" (OuterVolumeSpecName: "logs") pod "8283de30-8976-47bd-bd09-aab53140f68d" (UID: "8283de30-8976-47bd-bd09-aab53140f68d"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.139536 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6099fea1-bbb4-4413-a826-539b3dfc3eec-kube-api-access-skrwg" (OuterVolumeSpecName: "kube-api-access-skrwg") pod "6099fea1-bbb4-4413-a826-539b3dfc3eec" (UID: "6099fea1-bbb4-4413-a826-539b3dfc3eec"). InnerVolumeSpecName "kube-api-access-skrwg". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.160549 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-api-db-create-mnlg4"] Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.188709 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage20-crc" (OuterVolumeSpecName: "glance") pod "2a349843-a7b7-4dff-a5ae-f513337c5809" (UID: "2a349843-a7b7-4dff-a5ae-f513337c5809"). InnerVolumeSpecName "local-storage20-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.190566 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2a349843-a7b7-4dff-a5ae-f513337c5809-scripts" (OuterVolumeSpecName: "scripts") pod "2a349843-a7b7-4dff-a5ae-f513337c5809" (UID: "2a349843-a7b7-4dff-a5ae-f513337c5809"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.202442 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-api-06eb-account-create-update-cbfzl"] Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.224297 5052 reconciler_common.go:293] "Volume detached for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/2a349843-a7b7-4dff-a5ae-f513337c5809-httpd-run\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.224334 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-skrwg\" (UniqueName: \"kubernetes.io/projected/6099fea1-bbb4-4413-a826-539b3dfc3eec-kube-api-access-skrwg\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.224346 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/8283de30-8976-47bd-bd09-aab53140f68d-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.224362 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/6099fea1-bbb4-4413-a826-539b3dfc3eec-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.224372 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/d81b09dc-75a1-45f8-ace4-786531ad3bfd-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.224381 5052 reconciler_common.go:293] "Volume detached for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/d81b09dc-75a1-45f8-ace4-786531ad3bfd-httpd-run\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.224390 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/2a349843-a7b7-4dff-a5ae-f513337c5809-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.224400 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/2a349843-a7b7-4dff-a5ae-f513337c5809-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.224431 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage20-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage20-crc\") on node \"crc\" " Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.257710 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/novaapi06eb-account-delete-kvlp2"] Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.269192 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-api-06eb-account-create-update-cbfzl"] Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.289236 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/keystone-db-create-6kjm6"] Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.289695 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8283de30-8976-47bd-bd09-aab53140f68d-kube-api-access-69g8d" (OuterVolumeSpecName: "kube-api-access-69g8d") pod "8283de30-8976-47bd-bd09-aab53140f68d" (UID: "8283de30-8976-47bd-bd09-aab53140f68d"). InnerVolumeSpecName "kube-api-access-69g8d". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.289883 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8283de30-8976-47bd-bd09-aab53140f68d-scripts" (OuterVolumeSpecName: "scripts") pod "8283de30-8976-47bd-bd09-aab53140f68d" (UID: "8283de30-8976-47bd-bd09-aab53140f68d"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.290303 5052 generic.go:334] "Generic (PLEG): container finished" podID="89b056a7-e448-447d-be7d-b69f3d4862a1" containerID="6b5e16bdf4a9b75d7e426aac04d9e0f302e3e926580d5ea1db33ccce8cef8eb9" exitCode=0 Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.290446 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"89b056a7-e448-447d-be7d-b69f3d4862a1","Type":"ContainerDied","Data":"6b5e16bdf4a9b75d7e426aac04d9e0f302e3e926580d5ea1db33ccce8cef8eb9"} Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.295303 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement7cee-account-delete-7wgkx" event={"ID":"6099fea1-bbb4-4413-a826-539b3dfc3eec","Type":"ContainerDied","Data":"3ccafc1a9eceecb8999301b5f75d2abc1de50fd4b00703fc52a48fd5e6e875bf"} Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.295348 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3ccafc1a9eceecb8999301b5f75d2abc1de50fd4b00703fc52a48fd5e6e875bf" Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.295417 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement7cee-account-delete-7wgkx" Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.304267 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/keystone-db-create-6kjm6"] Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.304867 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d81b09dc-75a1-45f8-ace4-786531ad3bfd-scripts" (OuterVolumeSpecName: "scripts") pod "d81b09dc-75a1-45f8-ace4-786531ad3bfd" (UID: "d81b09dc-75a1-45f8-ace4-786531ad3bfd"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.305933 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.305967 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"d81b09dc-75a1-45f8-ace4-786531ad3bfd","Type":"ContainerDied","Data":"3563429b0947d41b3ec949fe5606dcbd5f966d6dc0206580a74f817c192e1e64"} Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.310790 5052 generic.go:334] "Generic (PLEG): container finished" podID="388bc903-097f-4f97-8f19-2f7bcdb49cc5" containerID="7024c66624b9b95f71fa0589d2f50417eaa07d7b921b5356f06724331235a546" exitCode=0 Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.310860 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"388bc903-097f-4f97-8f19-2f7bcdb49cc5","Type":"ContainerDied","Data":"7024c66624b9b95f71fa0589d2f50417eaa07d7b921b5356f06724331235a546"} Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.306001 5052 scope.go:117] "RemoveContainer" containerID="e29fe95d4add082f6c46305d5e3fe12f454dfccd4b1d5c2355d53782ac82fa29" Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.313651 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d81b09dc-75a1-45f8-ace4-786531ad3bfd-kube-api-access-kng9h" (OuterVolumeSpecName: "kube-api-access-kng9h") pod "d81b09dc-75a1-45f8-ace4-786531ad3bfd" (UID: "d81b09dc-75a1-45f8-ace4-786531ad3bfd"). InnerVolumeSpecName "kube-api-access-kng9h". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.314615 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2a349843-a7b7-4dff-a5ae-f513337c5809-kube-api-access-rzrj5" (OuterVolumeSpecName: "kube-api-access-rzrj5") pod "2a349843-a7b7-4dff-a5ae-f513337c5809" (UID: "2a349843-a7b7-4dff-a5ae-f513337c5809"). InnerVolumeSpecName "kube-api-access-rzrj5". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.315750 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage03-crc" (OuterVolumeSpecName: "glance") pod "d81b09dc-75a1-45f8-ace4-786531ad3bfd" (UID: "d81b09dc-75a1-45f8-ace4-786531ad3bfd"). InnerVolumeSpecName "local-storage03-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.314339 5052 generic.go:334] "Generic (PLEG): container finished" podID="cd486343-0aad-4039-a23b-34c60cd1a381" containerID="ec66627f7d357cab05559c6d34ef624f9d761ad2e4531d42cccb0b06d7bb2cba" exitCode=0 Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.314352 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"cd486343-0aad-4039-a23b-34c60cd1a381","Type":"ContainerDied","Data":"ec66627f7d357cab05559c6d34ef624f9d761ad2e4531d42cccb0b06d7bb2cba"} Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.324022 5052 generic.go:334] "Generic (PLEG): container finished" podID="b25d2a0c-1e93-4b3d-965a-a7c021f9649d" containerID="d4bb4f5c0829cb278a445845849850d561fa98757159ecf40155346ec8896bde" exitCode=0 Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.324132 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-58464649dd-sx8mw" event={"ID":"b25d2a0c-1e93-4b3d-965a-a7c021f9649d","Type":"ContainerDied","Data":"d4bb4f5c0829cb278a445845849850d561fa98757159ecf40155346ec8896bde"} Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.327325 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/d81b09dc-75a1-45f8-ace4-786531ad3bfd-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.327546 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-kng9h\" (UniqueName: \"kubernetes.io/projected/d81b09dc-75a1-45f8-ace4-786531ad3bfd-kube-api-access-kng9h\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.327568 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/8283de30-8976-47bd-bd09-aab53140f68d-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.327582 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-rzrj5\" (UniqueName: \"kubernetes.io/projected/2a349843-a7b7-4dff-a5ae-f513337c5809-kube-api-access-rzrj5\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.327594 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-69g8d\" (UniqueName: \"kubernetes.io/projected/8283de30-8976-47bd-bd09-aab53140f68d-kube-api-access-69g8d\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.327846 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage03-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage03-crc\") on node \"crc\" " Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.355317 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/keystone-1874-account-create-update-7t9ms"] Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.356731 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-proxy-6b68c4db5d-g8nvc" Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.356714 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-proxy-6b68c4db5d-g8nvc" event={"ID":"ae6940a4-0f4c-46ab-8218-56bf5c5d3084","Type":"ContainerDied","Data":"d9473142a7df3ab74c3cc40fff2c3b5ebfc68071ca057ae58c77c13b1a8cf630"} Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.363807 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"2a349843-a7b7-4dff-a5ae-f513337c5809","Type":"ContainerDied","Data":"5f18143bbfa921f999a45907b4264b014c3cbd86a24fabeba94b8ae7143fcb1d"} Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.363937 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.368122 5052 generic.go:334] "Generic (PLEG): container finished" podID="cb68adf1-26f2-4b14-9874-dde28826aba3" containerID="96dc968ef689e331a6df033c2d763c10c20b7c7df416d6abc28abb2477d6b9ee" exitCode=0 Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.368191 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"cb68adf1-26f2-4b14-9874-dde28826aba3","Type":"ContainerDied","Data":"96dc968ef689e331a6df033c2d763c10c20b7c7df416d6abc28abb2477d6b9ee"} Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.369163 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.369797 5052 generic.go:334] "Generic (PLEG): container finished" podID="c2a65caa-2eec-48af-bdad-8fc19d827867" containerID="28049a9b7828f990e7cdf628c7744bd85e07ead3fc6256992ef7d5bf23e045a3" exitCode=0 Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.369909 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-conductor-0" event={"ID":"c2a65caa-2eec-48af-bdad-8fc19d827867","Type":"ContainerDied","Data":"28049a9b7828f990e7cdf628c7744bd85e07ead3fc6256992ef7d5bf23e045a3"} Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.371483 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.373602 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-79d5cb4fdb-7h2fn" Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.374167 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-79d5cb4fdb-7h2fn" event={"ID":"8283de30-8976-47bd-bd09-aab53140f68d","Type":"ContainerDied","Data":"a8fc9e7655d23ddd94c7d18ce5577d0d76d7c6c69c0d07ad157edcd42a1d42fd"} Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.381316 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/keystone-1874-account-create-update-7t9ms"] Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.382094 5052 scope.go:117] "RemoveContainer" containerID="be83ec436e5bb13951ed23e2fd4e1e84fd5f3b6ce1f0a6f46b5c44dcd3634ee4" Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.409099 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell0-db-create-bd44j"] Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.430795 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cclpd\" (UniqueName: \"kubernetes.io/projected/cd486343-0aad-4039-a23b-34c60cd1a381-kube-api-access-cclpd\") pod \"cd486343-0aad-4039-a23b-34c60cd1a381\" (UID: \"cd486343-0aad-4039-a23b-34c60cd1a381\") " Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.431274 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.432849 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell0-db-create-bd44j"] Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.445066 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/cd486343-0aad-4039-a23b-34c60cd1a381-combined-ca-bundle\") pod \"cd486343-0aad-4039-a23b-34c60cd1a381\" (UID: \"cd486343-0aad-4039-a23b-34c60cd1a381\") " Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.445159 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/cd486343-0aad-4039-a23b-34c60cd1a381-config-data\") pod \"cd486343-0aad-4039-a23b-34c60cd1a381\" (UID: \"cd486343-0aad-4039-a23b-34c60cd1a381\") " Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.445287 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/cd486343-0aad-4039-a23b-34c60cd1a381-logs\") pod \"cd486343-0aad-4039-a23b-34c60cd1a381\" (UID: \"cd486343-0aad-4039-a23b-34c60cd1a381\") " Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.445364 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/cd486343-0aad-4039-a23b-34c60cd1a381-nova-metadata-tls-certs\") pod \"cd486343-0aad-4039-a23b-34c60cd1a381\" (UID: \"cd486343-0aad-4039-a23b-34c60cd1a381\") " Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.447980 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd486343-0aad-4039-a23b-34c60cd1a381-logs" (OuterVolumeSpecName: "logs") pod "cd486343-0aad-4039-a23b-34c60cd1a381" (UID: "cd486343-0aad-4039-a23b-34c60cd1a381"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.456385 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.456640 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell0-0bb5-account-create-update-6flhc"] Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.469164 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/novacell00bb5-account-delete-r2csc"] Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.470835 5052 scope.go:117] "RemoveContainer" containerID="6d0e263d1fbe61d44b6dd94d1a87bc31a61607f209df9eb30c44c6f6386c7381" Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.487665 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cd486343-0aad-4039-a23b-34c60cd1a381-kube-api-access-cclpd" (OuterVolumeSpecName: "kube-api-access-cclpd") pod "cd486343-0aad-4039-a23b-34c60cd1a381" (UID: "cd486343-0aad-4039-a23b-34c60cd1a381"). InnerVolumeSpecName "kube-api-access-cclpd". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.487749 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell0-0bb5-account-create-update-6flhc"] Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.547009 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/cb68adf1-26f2-4b14-9874-dde28826aba3-logs\") pod \"cb68adf1-26f2-4b14-9874-dde28826aba3\" (UID: \"cb68adf1-26f2-4b14-9874-dde28826aba3\") " Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.547253 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/cb68adf1-26f2-4b14-9874-dde28826aba3-public-tls-certs\") pod \"cb68adf1-26f2-4b14-9874-dde28826aba3\" (UID: \"cb68adf1-26f2-4b14-9874-dde28826aba3\") " Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.547363 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/388bc903-097f-4f97-8f19-2f7bcdb49cc5-combined-ca-bundle\") pod \"388bc903-097f-4f97-8f19-2f7bcdb49cc5\" (UID: \"388bc903-097f-4f97-8f19-2f7bcdb49cc5\") " Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.547482 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/388bc903-097f-4f97-8f19-2f7bcdb49cc5-scripts\") pod \"388bc903-097f-4f97-8f19-2f7bcdb49cc5\" (UID: \"388bc903-097f-4f97-8f19-2f7bcdb49cc5\") " Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.547571 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gvwsd\" (UniqueName: \"kubernetes.io/projected/cb68adf1-26f2-4b14-9874-dde28826aba3-kube-api-access-gvwsd\") pod \"cb68adf1-26f2-4b14-9874-dde28826aba3\" (UID: \"cb68adf1-26f2-4b14-9874-dde28826aba3\") " Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.547658 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/388bc903-097f-4f97-8f19-2f7bcdb49cc5-config-data-custom\") pod \"388bc903-097f-4f97-8f19-2f7bcdb49cc5\" (UID: \"388bc903-097f-4f97-8f19-2f7bcdb49cc5\") " Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.547747 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/388bc903-097f-4f97-8f19-2f7bcdb49cc5-config-data\") pod \"388bc903-097f-4f97-8f19-2f7bcdb49cc5\" (UID: \"388bc903-097f-4f97-8f19-2f7bcdb49cc5\") " Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.547880 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/cb68adf1-26f2-4b14-9874-dde28826aba3-combined-ca-bundle\") pod \"cb68adf1-26f2-4b14-9874-dde28826aba3\" (UID: \"cb68adf1-26f2-4b14-9874-dde28826aba3\") " Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.547955 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/388bc903-097f-4f97-8f19-2f7bcdb49cc5-internal-tls-certs\") pod \"388bc903-097f-4f97-8f19-2f7bcdb49cc5\" (UID: \"388bc903-097f-4f97-8f19-2f7bcdb49cc5\") " Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.548375 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/cb68adf1-26f2-4b14-9874-dde28826aba3-config-data\") pod \"cb68adf1-26f2-4b14-9874-dde28826aba3\" (UID: \"cb68adf1-26f2-4b14-9874-dde28826aba3\") " Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.548477 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/cb68adf1-26f2-4b14-9874-dde28826aba3-internal-tls-certs\") pod \"cb68adf1-26f2-4b14-9874-dde28826aba3\" (UID: \"cb68adf1-26f2-4b14-9874-dde28826aba3\") " Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.548659 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/388bc903-097f-4f97-8f19-2f7bcdb49cc5-logs\") pod \"388bc903-097f-4f97-8f19-2f7bcdb49cc5\" (UID: \"388bc903-097f-4f97-8f19-2f7bcdb49cc5\") " Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.548756 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/388bc903-097f-4f97-8f19-2f7bcdb49cc5-etc-machine-id\") pod \"388bc903-097f-4f97-8f19-2f7bcdb49cc5\" (UID: \"388bc903-097f-4f97-8f19-2f7bcdb49cc5\") " Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.548847 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/388bc903-097f-4f97-8f19-2f7bcdb49cc5-public-tls-certs\") pod \"388bc903-097f-4f97-8f19-2f7bcdb49cc5\" (UID: \"388bc903-097f-4f97-8f19-2f7bcdb49cc5\") " Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.548926 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2bxjn\" (UniqueName: \"kubernetes.io/projected/388bc903-097f-4f97-8f19-2f7bcdb49cc5-kube-api-access-2bxjn\") pod \"388bc903-097f-4f97-8f19-2f7bcdb49cc5\" (UID: \"388bc903-097f-4f97-8f19-2f7bcdb49cc5\") " Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.549460 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-cclpd\" (UniqueName: \"kubernetes.io/projected/cd486343-0aad-4039-a23b-34c60cd1a381-kube-api-access-cclpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.549543 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/cd486343-0aad-4039-a23b-34c60cd1a381-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.553501 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cb68adf1-26f2-4b14-9874-dde28826aba3-logs" (OuterVolumeSpecName: "logs") pod "cb68adf1-26f2-4b14-9874-dde28826aba3" (UID: "cb68adf1-26f2-4b14-9874-dde28826aba3"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.553955 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/388bc903-097f-4f97-8f19-2f7bcdb49cc5-etc-machine-id" (OuterVolumeSpecName: "etc-machine-id") pod "388bc903-097f-4f97-8f19-2f7bcdb49cc5" (UID: "388bc903-097f-4f97-8f19-2f7bcdb49cc5"). InnerVolumeSpecName "etc-machine-id". PluginName "kubernetes.io/host-path", VolumeGidValue "" Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.554277 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/388bc903-097f-4f97-8f19-2f7bcdb49cc5-logs" (OuterVolumeSpecName: "logs") pod "388bc903-097f-4f97-8f19-2f7bcdb49cc5" (UID: "388bc903-097f-4f97-8f19-2f7bcdb49cc5"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.569473 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/openstack-cell1-galera-0"] Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.576105 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/openstack-cell1-galera-0"] Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.612846 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/388bc903-097f-4f97-8f19-2f7bcdb49cc5-kube-api-access-2bxjn" (OuterVolumeSpecName: "kube-api-access-2bxjn") pod "388bc903-097f-4f97-8f19-2f7bcdb49cc5" (UID: "388bc903-097f-4f97-8f19-2f7bcdb49cc5"). InnerVolumeSpecName "kube-api-access-2bxjn". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.620394 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/388bc903-097f-4f97-8f19-2f7bcdb49cc5-config-data-custom" (OuterVolumeSpecName: "config-data-custom") pod "388bc903-097f-4f97-8f19-2f7bcdb49cc5" (UID: "388bc903-097f-4f97-8f19-2f7bcdb49cc5"). InnerVolumeSpecName "config-data-custom". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.638393 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cb68adf1-26f2-4b14-9874-dde28826aba3-kube-api-access-gvwsd" (OuterVolumeSpecName: "kube-api-access-gvwsd") pod "cb68adf1-26f2-4b14-9874-dde28826aba3" (UID: "cb68adf1-26f2-4b14-9874-dde28826aba3"). InnerVolumeSpecName "kube-api-access-gvwsd". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.645298 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/388bc903-097f-4f97-8f19-2f7bcdb49cc5-scripts" (OuterVolumeSpecName: "scripts") pod "388bc903-097f-4f97-8f19-2f7bcdb49cc5" (UID: "388bc903-097f-4f97-8f19-2f7bcdb49cc5"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.651246 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/388bc903-097f-4f97-8f19-2f7bcdb49cc5-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.651288 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-gvwsd\" (UniqueName: \"kubernetes.io/projected/cb68adf1-26f2-4b14-9874-dde28826aba3-kube-api-access-gvwsd\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.651299 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/388bc903-097f-4f97-8f19-2f7bcdb49cc5-config-data-custom\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.651309 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/388bc903-097f-4f97-8f19-2f7bcdb49cc5-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.651319 5052 reconciler_common.go:293] "Volume detached for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/388bc903-097f-4f97-8f19-2f7bcdb49cc5-etc-machine-id\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.651327 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-2bxjn\" (UniqueName: \"kubernetes.io/projected/388bc903-097f-4f97-8f19-2f7bcdb49cc5-kube-api-access-2bxjn\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.651336 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/cb68adf1-26f2-4b14-9874-dde28826aba3-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.691532 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="00a9935f-ef2d-4976-97a0-78ee6325433d" path="/var/lib/kubelet/pods/00a9935f-ef2d-4976-97a0-78ee6325433d/volumes" Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.692325 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="16ff7253-a0ad-48a5-92e3-748f8053fc1a" path="/var/lib/kubelet/pods/16ff7253-a0ad-48a5-92e3-748f8053fc1a/volumes" Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.692939 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3fdeea22-1f1b-4237-affe-9368dd14fe6b" path="/var/lib/kubelet/pods/3fdeea22-1f1b-4237-affe-9368dd14fe6b/volumes" Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.694241 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4ef6dc74-a5f4-489c-8527-df175eb265ec" path="/var/lib/kubelet/pods/4ef6dc74-a5f4-489c-8527-df175eb265ec/volumes" Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.696295 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="57bde9b6-d244-4159-840f-25e35a2f8dc6" path="/var/lib/kubelet/pods/57bde9b6-d244-4159-840f-25e35a2f8dc6/volumes" Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.697070 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="58c97692-2e69-4232-ba13-23806ab9fb67" path="/var/lib/kubelet/pods/58c97692-2e69-4232-ba13-23806ab9fb67/volumes" Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.699394 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="59c0dc2a-1f1d-46a1-93c1-6553bf351704" path="/var/lib/kubelet/pods/59c0dc2a-1f1d-46a1-93c1-6553bf351704/volumes" Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.700048 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9e1f00b4-e4b8-4696-aacc-92f6b67f99f4" path="/var/lib/kubelet/pods/9e1f00b4-e4b8-4696-aacc-92f6b67f99f4/volumes" Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.700524 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a15b6144-952c-4d41-a141-5bca5c7fd402" path="/var/lib/kubelet/pods/a15b6144-952c-4d41-a141-5bca5c7fd402/volumes" Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.701531 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a2493bc9-910f-4f3d-9e74-cbd5721c838d" path="/var/lib/kubelet/pods/a2493bc9-910f-4f3d-9e74-cbd5721c838d/volumes" Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.702005 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dfdd8013-ee6e-4ec6-b5d1-fb423f444430" path="/var/lib/kubelet/pods/dfdd8013-ee6e-4ec6-b5d1-fb423f444430/volumes" Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.866827 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cd486343-0aad-4039-a23b-34c60cd1a381-nova-metadata-tls-certs" (OuterVolumeSpecName: "nova-metadata-tls-certs") pod "cd486343-0aad-4039-a23b-34c60cd1a381" (UID: "cd486343-0aad-4039-a23b-34c60cd1a381"). InnerVolumeSpecName "nova-metadata-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.883320 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage20-crc" (UniqueName: "kubernetes.io/local-volume/local-storage20-crc") on node "crc" Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.920088 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cb68adf1-26f2-4b14-9874-dde28826aba3-config-data" (OuterVolumeSpecName: "config-data") pod "cb68adf1-26f2-4b14-9874-dde28826aba3" (UID: "cb68adf1-26f2-4b14-9874-dde28826aba3"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.950877 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d81b09dc-75a1-45f8-ace4-786531ad3bfd-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "d81b09dc-75a1-45f8-ace4-786531ad3bfd" (UID: "d81b09dc-75a1-45f8-ace4-786531ad3bfd"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:15:11 crc kubenswrapper[5052]: E1203 12:15:11.959536 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="33c38af311895b3ea2da41b9880bb47d91d38eadcf28dcf815946f0d92d3f569" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.962972 5052 reconciler_common.go:293] "Volume detached for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/cd486343-0aad-4039-a23b-34c60cd1a381-nova-metadata-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.963016 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage20-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage20-crc\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.963030 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d81b09dc-75a1-45f8-ace4-786531ad3bfd-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.963040 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/cb68adf1-26f2-4b14-9874-dde28826aba3-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:11 crc kubenswrapper[5052]: E1203 12:15:11.964766 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="33c38af311895b3ea2da41b9880bb47d91d38eadcf28dcf815946f0d92d3f569" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:15:11 crc kubenswrapper[5052]: I1203 12:15:11.970476 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cb68adf1-26f2-4b14-9874-dde28826aba3-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "cb68adf1-26f2-4b14-9874-dde28826aba3" (UID: "cb68adf1-26f2-4b14-9874-dde28826aba3"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:15:11 crc kubenswrapper[5052]: E1203 12:15:11.991869 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="33c38af311895b3ea2da41b9880bb47d91d38eadcf28dcf815946f0d92d3f569" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:15:11 crc kubenswrapper[5052]: E1203 12:15:11.991938 5052 prober.go:104] "Probe errored" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" probeType="Readiness" pod="openstack-kuttl-tests/nova-cell0-conductor-0" podUID="14774d58-37e8-4e65-b0b0-d355c0070bca" containerName="nova-cell0-conductor-conductor" Dec 03 12:15:12 crc kubenswrapper[5052]: I1203 12:15:12.010331 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8283de30-8976-47bd-bd09-aab53140f68d-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "8283de30-8976-47bd-bd09-aab53140f68d" (UID: "8283de30-8976-47bd-bd09-aab53140f68d"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:15:12 crc kubenswrapper[5052]: I1203 12:15:12.021100 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/openstack-galera-0" podUID="85f5676f-32e8-4c3d-8729-e1eb2fdc8dc9" containerName="galera" containerID="cri-o://3ba0d39f21ff90002edd3d3fb8c1a2fd07737053623cac5f18054747025f4d6e" gracePeriod=30 Dec 03 12:15:12 crc kubenswrapper[5052]: I1203 12:15:12.031207 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2a349843-a7b7-4dff-a5ae-f513337c5809-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "2a349843-a7b7-4dff-a5ae-f513337c5809" (UID: "2a349843-a7b7-4dff-a5ae-f513337c5809"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:15:12 crc kubenswrapper[5052]: I1203 12:15:12.034689 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ae6940a4-0f4c-46ab-8218-56bf5c5d3084-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "ae6940a4-0f4c-46ab-8218-56bf5c5d3084" (UID: "ae6940a4-0f4c-46ab-8218-56bf5c5d3084"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:15:12 crc kubenswrapper[5052]: I1203 12:15:12.064626 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2a349843-a7b7-4dff-a5ae-f513337c5809-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:12 crc kubenswrapper[5052]: I1203 12:15:12.064658 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8283de30-8976-47bd-bd09-aab53140f68d-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:12 crc kubenswrapper[5052]: I1203 12:15:12.064674 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/cb68adf1-26f2-4b14-9874-dde28826aba3-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:12 crc kubenswrapper[5052]: I1203 12:15:12.064685 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/ae6940a4-0f4c-46ab-8218-56bf5c5d3084-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:12 crc kubenswrapper[5052]: I1203 12:15:12.077282 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d81b09dc-75a1-45f8-ace4-786531ad3bfd-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "d81b09dc-75a1-45f8-ace4-786531ad3bfd" (UID: "d81b09dc-75a1-45f8-ace4-786531ad3bfd"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:15:12 crc kubenswrapper[5052]: I1203 12:15:12.101968 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ae6940a4-0f4c-46ab-8218-56bf5c5d3084-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "ae6940a4-0f4c-46ab-8218-56bf5c5d3084" (UID: "ae6940a4-0f4c-46ab-8218-56bf5c5d3084"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:15:12 crc kubenswrapper[5052]: I1203 12:15:12.111078 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/388bc903-097f-4f97-8f19-2f7bcdb49cc5-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "388bc903-097f-4f97-8f19-2f7bcdb49cc5" (UID: "388bc903-097f-4f97-8f19-2f7bcdb49cc5"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:15:12 crc kubenswrapper[5052]: I1203 12:15:12.117857 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage03-crc" (UniqueName: "kubernetes.io/local-volume/local-storage03-crc") on node "crc" Dec 03 12:15:12 crc kubenswrapper[5052]: I1203 12:15:12.150944 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cd486343-0aad-4039-a23b-34c60cd1a381-config-data" (OuterVolumeSpecName: "config-data") pod "cd486343-0aad-4039-a23b-34c60cd1a381" (UID: "cd486343-0aad-4039-a23b-34c60cd1a381"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:15:12 crc kubenswrapper[5052]: I1203 12:15:12.166498 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/cd486343-0aad-4039-a23b-34c60cd1a381-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:12 crc kubenswrapper[5052]: I1203 12:15:12.166801 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage03-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage03-crc\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:12 crc kubenswrapper[5052]: I1203 12:15:12.166901 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/388bc903-097f-4f97-8f19-2f7bcdb49cc5-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:12 crc kubenswrapper[5052]: I1203 12:15:12.167020 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/ae6940a4-0f4c-46ab-8218-56bf5c5d3084-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:12 crc kubenswrapper[5052]: I1203 12:15:12.167137 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/d81b09dc-75a1-45f8-ace4-786531ad3bfd-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:12 crc kubenswrapper[5052]: I1203 12:15:12.177465 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ae6940a4-0f4c-46ab-8218-56bf5c5d3084-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "ae6940a4-0f4c-46ab-8218-56bf5c5d3084" (UID: "ae6940a4-0f4c-46ab-8218-56bf5c5d3084"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:15:12 crc kubenswrapper[5052]: I1203 12:15:12.187211 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/388bc903-097f-4f97-8f19-2f7bcdb49cc5-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "388bc903-097f-4f97-8f19-2f7bcdb49cc5" (UID: "388bc903-097f-4f97-8f19-2f7bcdb49cc5"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:15:12 crc kubenswrapper[5052]: I1203 12:15:12.188069 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8283de30-8976-47bd-bd09-aab53140f68d-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "8283de30-8976-47bd-bd09-aab53140f68d" (UID: "8283de30-8976-47bd-bd09-aab53140f68d"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:15:12 crc kubenswrapper[5052]: I1203 12:15:12.231140 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2a349843-a7b7-4dff-a5ae-f513337c5809-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "2a349843-a7b7-4dff-a5ae-f513337c5809" (UID: "2a349843-a7b7-4dff-a5ae-f513337c5809"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:15:12 crc kubenswrapper[5052]: I1203 12:15:12.254652 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cd486343-0aad-4039-a23b-34c60cd1a381-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "cd486343-0aad-4039-a23b-34c60cd1a381" (UID: "cd486343-0aad-4039-a23b-34c60cd1a381"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:15:12 crc kubenswrapper[5052]: I1203 12:15:12.259281 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cb68adf1-26f2-4b14-9874-dde28826aba3-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "cb68adf1-26f2-4b14-9874-dde28826aba3" (UID: "cb68adf1-26f2-4b14-9874-dde28826aba3"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:15:12 crc kubenswrapper[5052]: I1203 12:15:12.260096 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ae6940a4-0f4c-46ab-8218-56bf5c5d3084-config-data" (OuterVolumeSpecName: "config-data") pod "ae6940a4-0f4c-46ab-8218-56bf5c5d3084" (UID: "ae6940a4-0f4c-46ab-8218-56bf5c5d3084"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:15:12 crc kubenswrapper[5052]: I1203 12:15:12.268311 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/388bc903-097f-4f97-8f19-2f7bcdb49cc5-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:12 crc kubenswrapper[5052]: I1203 12:15:12.268341 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/cd486343-0aad-4039-a23b-34c60cd1a381-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:12 crc kubenswrapper[5052]: I1203 12:15:12.268351 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/2a349843-a7b7-4dff-a5ae-f513337c5809-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:12 crc kubenswrapper[5052]: I1203 12:15:12.268359 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/8283de30-8976-47bd-bd09-aab53140f68d-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:12 crc kubenswrapper[5052]: I1203 12:15:12.268367 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/ae6940a4-0f4c-46ab-8218-56bf5c5d3084-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:12 crc kubenswrapper[5052]: I1203 12:15:12.268376 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ae6940a4-0f4c-46ab-8218-56bf5c5d3084-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:12 crc kubenswrapper[5052]: I1203 12:15:12.268384 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/cb68adf1-26f2-4b14-9874-dde28826aba3-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:12 crc kubenswrapper[5052]: I1203 12:15:12.275792 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/388bc903-097f-4f97-8f19-2f7bcdb49cc5-config-data" (OuterVolumeSpecName: "config-data") pod "388bc903-097f-4f97-8f19-2f7bcdb49cc5" (UID: "388bc903-097f-4f97-8f19-2f7bcdb49cc5"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:15:12 crc kubenswrapper[5052]: I1203 12:15:12.280651 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d81b09dc-75a1-45f8-ace4-786531ad3bfd-config-data" (OuterVolumeSpecName: "config-data") pod "d81b09dc-75a1-45f8-ace4-786531ad3bfd" (UID: "d81b09dc-75a1-45f8-ace4-786531ad3bfd"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:15:12 crc kubenswrapper[5052]: I1203 12:15:12.284810 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cb68adf1-26f2-4b14-9874-dde28826aba3-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "cb68adf1-26f2-4b14-9874-dde28826aba3" (UID: "cb68adf1-26f2-4b14-9874-dde28826aba3"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:15:12 crc kubenswrapper[5052]: I1203 12:15:12.292423 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2a349843-a7b7-4dff-a5ae-f513337c5809-config-data" (OuterVolumeSpecName: "config-data") pod "2a349843-a7b7-4dff-a5ae-f513337c5809" (UID: "2a349843-a7b7-4dff-a5ae-f513337c5809"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:15:12 crc kubenswrapper[5052]: I1203 12:15:12.297239 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8283de30-8976-47bd-bd09-aab53140f68d-config-data" (OuterVolumeSpecName: "config-data") pod "8283de30-8976-47bd-bd09-aab53140f68d" (UID: "8283de30-8976-47bd-bd09-aab53140f68d"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:15:12 crc kubenswrapper[5052]: I1203 12:15:12.308689 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/388bc903-097f-4f97-8f19-2f7bcdb49cc5-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "388bc903-097f-4f97-8f19-2f7bcdb49cc5" (UID: "388bc903-097f-4f97-8f19-2f7bcdb49cc5"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:15:12 crc kubenswrapper[5052]: I1203 12:15:12.351982 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8283de30-8976-47bd-bd09-aab53140f68d-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "8283de30-8976-47bd-bd09-aab53140f68d" (UID: "8283de30-8976-47bd-bd09-aab53140f68d"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:15:12 crc kubenswrapper[5052]: I1203 12:15:12.370193 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/cb68adf1-26f2-4b14-9874-dde28826aba3-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:12 crc kubenswrapper[5052]: I1203 12:15:12.370231 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/8283de30-8976-47bd-bd09-aab53140f68d-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:12 crc kubenswrapper[5052]: I1203 12:15:12.370242 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/2a349843-a7b7-4dff-a5ae-f513337c5809-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:12 crc kubenswrapper[5052]: I1203 12:15:12.370251 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/388bc903-097f-4f97-8f19-2f7bcdb49cc5-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:12 crc kubenswrapper[5052]: I1203 12:15:12.370261 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/d81b09dc-75a1-45f8-ace4-786531ad3bfd-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:12 crc kubenswrapper[5052]: I1203 12:15:12.370270 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/8283de30-8976-47bd-bd09-aab53140f68d-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:12 crc kubenswrapper[5052]: I1203 12:15:12.370278 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/388bc903-097f-4f97-8f19-2f7bcdb49cc5-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:12 crc kubenswrapper[5052]: I1203 12:15:12.390644 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:15:12 crc kubenswrapper[5052]: I1203 12:15:12.394188 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:15:12 crc kubenswrapper[5052]: I1203 12:15:12.399844 5052 kubelet_pods.go:1007] "Unable to retrieve pull secret, the image pull may not succeed." pod="openstack-kuttl-tests/novacell00bb5-account-delete-r2csc" secret="" err="secret \"galera-openstack-dockercfg-vpq7c\" not found" Dec 03 12:15:12 crc kubenswrapper[5052]: I1203 12:15:12.405208 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-kl2c8" podStartSLOduration=7.405194084 podStartE2EDuration="7.405194084s" podCreationTimestamp="2025-12-03 12:15:05 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:15:12.404255337 +0000 UTC m=+3302.992497101" watchObservedRunningTime="2025-12-03 12:15:12.405194084 +0000 UTC m=+3302.993435848" Dec 03 12:15:12 crc kubenswrapper[5052]: I1203 12:15:12.414929 5052 generic.go:334] "Generic (PLEG): container finished" podID="b28a125a-7966-4f97-91c1-fa52d6062405" containerID="dc5c838958d7fbbc47d97aabd5b021dc0188e172b8b2ed5ab168cef70e171e07" exitCode=0 Dec 03 12:15:12 crc kubenswrapper[5052]: I1203 12:15:12.416582 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/novaapi06eb-account-delete-kvlp2" podUID="72a72aa7-f35b-4b95-b5fd-caaf1272943a" containerName="mariadb-account-delete" containerID="cri-o://1d5931017ee84078964fe579758e4691d11ff7226c17409d0cc037c9caa5d258" gracePeriod=30 Dec 03 12:15:12 crc kubenswrapper[5052]: I1203 12:15:12.429894 5052 kubelet_pods.go:1007] "Unable to retrieve pull secret, the image pull may not succeed." pod="openstack-kuttl-tests/cinder7878-account-delete-jwtvt" secret="" err="secret \"galera-openstack-dockercfg-vpq7c\" not found" Dec 03 12:15:12 crc kubenswrapper[5052]: I1203 12:15:12.432955 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:15:12 crc kubenswrapper[5052]: I1203 12:15:12.435770 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/novacell00bb5-account-delete-r2csc" podStartSLOduration=7.435741626 podStartE2EDuration="7.435741626s" podCreationTimestamp="2025-12-03 12:15:05 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:15:12.424358191 +0000 UTC m=+3303.012599975" watchObservedRunningTime="2025-12-03 12:15:12.435741626 +0000 UTC m=+3303.023983410" Dec 03 12:15:12 crc kubenswrapper[5052]: I1203 12:15:12.465747 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/cinder7878-account-delete-jwtvt" podStartSLOduration=7.465729742 podStartE2EDuration="7.465729742s" podCreationTimestamp="2025-12-03 12:15:05 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:15:12.457644461 +0000 UTC m=+3303.045886245" watchObservedRunningTime="2025-12-03 12:15:12.465729742 +0000 UTC m=+3303.053971506" Dec 03 12:15:12 crc kubenswrapper[5052]: E1203 12:15:12.471915 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/rabbitmq-cell1-config-data: configmap "rabbitmq-cell1-config-data" not found Dec 03 12:15:12 crc kubenswrapper[5052]: E1203 12:15:12.472050 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e63eb1eb-e0b9-46ea-a19b-8f5c018728ff-config-data podName:e63eb1eb-e0b9-46ea-a19b-8f5c018728ff nodeName:}" failed. No retries permitted until 2025-12-03 12:15:20.472030712 +0000 UTC m=+3311.060272476 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "config-data" (UniqueName: "kubernetes.io/configmap/e63eb1eb-e0b9-46ea-a19b-8f5c018728ff-config-data") pod "rabbitmq-cell1-server-0" (UID: "e63eb1eb-e0b9-46ea-a19b-8f5c018728ff") : configmap "rabbitmq-cell1-config-data" not found Dec 03 12:15:12 crc kubenswrapper[5052]: E1203 12:15:12.472390 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:15:12 crc kubenswrapper[5052]: E1203 12:15:12.472459 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/679f17b9-0e40-480b-bfde-4f63899cb972-operator-scripts podName:679f17b9-0e40-480b-bfde-4f63899cb972 nodeName:}" failed. No retries permitted until 2025-12-03 12:15:12.972439684 +0000 UTC m=+3303.560681548 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/679f17b9-0e40-480b-bfde-4f63899cb972-operator-scripts") pod "cinder7878-account-delete-jwtvt" (UID: "679f17b9-0e40-480b-bfde-4f63899cb972") : configmap "openstack-scripts" not found Dec 03 12:15:12 crc kubenswrapper[5052]: E1203 12:15:12.472407 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:15:12 crc kubenswrapper[5052]: E1203 12:15:12.472632 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c200c246-74d3-48dc-8b24-ac4b37611069-operator-scripts podName:c200c246-74d3-48dc-8b24-ac4b37611069 nodeName:}" failed. No retries permitted until 2025-12-03 12:15:12.972621149 +0000 UTC m=+3303.560863003 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/c200c246-74d3-48dc-8b24-ac4b37611069-operator-scripts") pod "novacell00bb5-account-delete-r2csc" (UID: "c200c246-74d3-48dc-8b24-ac4b37611069") : configmap "openstack-scripts" not found Dec 03 12:15:12 crc kubenswrapper[5052]: I1203 12:15:12.487851 5052 scope.go:117] "RemoveContainer" containerID="22c61e9465e05e82a9b75ec36f8899daba6ae848986e069b783febfc60abd647" Dec 03 12:15:12 crc kubenswrapper[5052]: I1203 12:15:12.497785 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/novaapi06eb-account-delete-kvlp2" podStartSLOduration=7.497763027 podStartE2EDuration="7.497763027s" podCreationTimestamp="2025-12-03 12:15:05 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:15:12.472709591 +0000 UTC m=+3303.060951375" watchObservedRunningTime="2025-12-03 12:15:12.497763027 +0000 UTC m=+3303.086004791" Dec 03 12:15:12 crc kubenswrapper[5052]: I1203 12:15:12.504048 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-kl2c8" event={"ID":"ebf52f1d-a87e-4300-a650-4c4cb36e80d2","Type":"ContainerStarted","Data":"fab60ab776551b13a2d2abeee894ccd3d25015754ba7d7788bde10100e3537d9"} Dec 03 12:15:12 crc kubenswrapper[5052]: I1203 12:15:12.504091 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"cb68adf1-26f2-4b14-9874-dde28826aba3","Type":"ContainerDied","Data":"e4952ddc111a31dd387626299292e99f5a098bbfd611559dcea41c8a62d58daf"} Dec 03 12:15:12 crc kubenswrapper[5052]: I1203 12:15:12.504114 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"cd486343-0aad-4039-a23b-34c60cd1a381","Type":"ContainerDied","Data":"5b5a66a88c3807b2a543ebb34939109f35b4a3d23a9dc84017aad6be0ac33835"} Dec 03 12:15:12 crc kubenswrapper[5052]: I1203 12:15:12.504128 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-conductor-0" event={"ID":"c2a65caa-2eec-48af-bdad-8fc19d827867","Type":"ContainerDied","Data":"7d145059d90062ed272404781a290d6f56d30b3f105c3f72c109e13818483a5f"} Dec 03 12:15:12 crc kubenswrapper[5052]: I1203 12:15:12.504152 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7d145059d90062ed272404781a290d6f56d30b3f105c3f72c109e13818483a5f" Dec 03 12:15:12 crc kubenswrapper[5052]: I1203 12:15:12.504169 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-kl2c8" Dec 03 12:15:12 crc kubenswrapper[5052]: I1203 12:15:12.504178 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/novacell00bb5-account-delete-r2csc" event={"ID":"c200c246-74d3-48dc-8b24-ac4b37611069","Type":"ContainerStarted","Data":"c7a74e6ab5fc2e0b601148d2ded033a3a334d1f7ad582da015664c00d9a6e301"} Dec 03 12:15:12 crc kubenswrapper[5052]: I1203 12:15:12.504188 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glancef253-account-delete-cxxlg" event={"ID":"fee7d732-4fa3-40d5-8fff-cb88272db0ce","Type":"ContainerDied","Data":"77f6420de0ee4d7fc7f116d4e9e916a4b2a7b00e8e935caa9e5e9c678683d77e"} Dec 03 12:15:12 crc kubenswrapper[5052]: I1203 12:15:12.504200 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="77f6420de0ee4d7fc7f116d4e9e916a4b2a7b00e8e935caa9e5e9c678683d77e" Dec 03 12:15:12 crc kubenswrapper[5052]: I1203 12:15:12.512696 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/memcached-0" event={"ID":"b28a125a-7966-4f97-91c1-fa52d6062405","Type":"ContainerDied","Data":"dc5c838958d7fbbc47d97aabd5b021dc0188e172b8b2ed5ab168cef70e171e07"} Dec 03 12:15:12 crc kubenswrapper[5052]: I1203 12:15:12.512746 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/memcached-0" event={"ID":"b28a125a-7966-4f97-91c1-fa52d6062405","Type":"ContainerDied","Data":"1947bc52826f5255a366a00d509039b09060cb4706c21b38e5917db37d5e2fc2"} Dec 03 12:15:12 crc kubenswrapper[5052]: I1203 12:15:12.512759 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1947bc52826f5255a366a00d509039b09060cb4706c21b38e5917db37d5e2fc2" Dec 03 12:15:12 crc kubenswrapper[5052]: I1203 12:15:12.512770 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/novaapi06eb-account-delete-kvlp2" event={"ID":"72a72aa7-f35b-4b95-b5fd-caaf1272943a","Type":"ContainerStarted","Data":"1d5931017ee84078964fe579758e4691d11ff7226c17409d0cc037c9caa5d258"} Dec 03 12:15:12 crc kubenswrapper[5052]: I1203 12:15:12.512781 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican8992-account-delete-26lhs" event={"ID":"5856107a-0498-4956-beba-4390f5477ca0","Type":"ContainerDied","Data":"a31833a368a8d17106bbd18312736160d0ab49635a5fa9613d45237fa2a59bfe"} Dec 03 12:15:12 crc kubenswrapper[5052]: I1203 12:15:12.512792 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a31833a368a8d17106bbd18312736160d0ab49635a5fa9613d45237fa2a59bfe" Dec 03 12:15:12 crc kubenswrapper[5052]: I1203 12:15:12.512800 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder7878-account-delete-jwtvt" event={"ID":"679f17b9-0e40-480b-bfde-4f63899cb972","Type":"ContainerStarted","Data":"e9f158233d3479671db2b774df7b5ea49c9fbfe33de68208a166ee13a03145b4"} Dec 03 12:15:12 crc kubenswrapper[5052]: I1203 12:15:12.512812 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"388bc903-097f-4f97-8f19-2f7bcdb49cc5","Type":"ContainerDied","Data":"9b446c712f47a339ef4a4e66b7ebd4d2243479663b879feaf0dd616123564040"} Dec 03 12:15:12 crc kubenswrapper[5052]: I1203 12:15:12.512832 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron1287-account-delete-t8dcs" event={"ID":"f1619fbf-df68-484f-99d4-09b81a7a334e","Type":"ContainerDied","Data":"f44845aa3e508f5ce4fdd2b4e7dcd364803cb5dbfd95eb5cec06972871fb290d"} Dec 03 12:15:12 crc kubenswrapper[5052]: I1203 12:15:12.512844 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f44845aa3e508f5ce4fdd2b4e7dcd364803cb5dbfd95eb5cec06972871fb290d" Dec 03 12:15:12 crc kubenswrapper[5052]: I1203 12:15:12.512853 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"89b056a7-e448-447d-be7d-b69f3d4862a1","Type":"ContainerDied","Data":"0404e3426375893df66b349094edc2e791298eba44431629b420076bbfc5fe1d"} Dec 03 12:15:12 crc kubenswrapper[5052]: I1203 12:15:12.512862 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0404e3426375893df66b349094edc2e791298eba44431629b420076bbfc5fe1d" Dec 03 12:15:12 crc kubenswrapper[5052]: I1203 12:15:12.512868 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-58464649dd-sx8mw" event={"ID":"b25d2a0c-1e93-4b3d-965a-a7c021f9649d","Type":"ContainerDied","Data":"e63247ed7bd80b962fd6a6dd8426c057c8a6295d6246b0f68c805e0f3cb05e2a"} Dec 03 12:15:12 crc kubenswrapper[5052]: I1203 12:15:12.512878 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e63247ed7bd80b962fd6a6dd8426c057c8a6295d6246b0f68c805e0f3cb05e2a" Dec 03 12:15:12 crc kubenswrapper[5052]: I1203 12:15:12.517700 5052 scope.go:117] "RemoveContainer" containerID="f30fe921af5f7b637639949f8977c3720f1ebb48e3b4f39277fcbc953207de23" Dec 03 12:15:12 crc kubenswrapper[5052]: I1203 12:15:12.571341 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:15:12 crc kubenswrapper[5052]: I1203 12:15:12.575560 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:15:12 crc kubenswrapper[5052]: I1203 12:15:12.590289 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:15:12 crc kubenswrapper[5052]: I1203 12:15:12.602510 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:15:12 crc kubenswrapper[5052]: I1203 12:15:12.618571 5052 scope.go:117] "RemoveContainer" containerID="237be9059aceba08b75de8d8ecd8ee5c1eb28f5bd5c20bb5c20a33088beca081" Dec 03 12:15:12 crc kubenswrapper[5052]: I1203 12:15:12.621976 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-api-58464649dd-sx8mw" Dec 03 12:15:12 crc kubenswrapper[5052]: I1203 12:15:12.676213 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/89b056a7-e448-447d-be7d-b69f3d4862a1-combined-ca-bundle\") pod \"89b056a7-e448-447d-be7d-b69f3d4862a1\" (UID: \"89b056a7-e448-447d-be7d-b69f3d4862a1\") " Dec 03 12:15:12 crc kubenswrapper[5052]: I1203 12:15:12.676756 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/b25d2a0c-1e93-4b3d-965a-a7c021f9649d-config-data\") pod \"b25d2a0c-1e93-4b3d-965a-a7c021f9649d\" (UID: \"b25d2a0c-1e93-4b3d-965a-a7c021f9649d\") " Dec 03 12:15:12 crc kubenswrapper[5052]: I1203 12:15:12.676837 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/89b056a7-e448-447d-be7d-b69f3d4862a1-scripts\") pod \"89b056a7-e448-447d-be7d-b69f3d4862a1\" (UID: \"89b056a7-e448-447d-be7d-b69f3d4862a1\") " Dec 03 12:15:12 crc kubenswrapper[5052]: I1203 12:15:12.676929 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b25d2a0c-1e93-4b3d-965a-a7c021f9649d-combined-ca-bundle\") pod \"b25d2a0c-1e93-4b3d-965a-a7c021f9649d\" (UID: \"b25d2a0c-1e93-4b3d-965a-a7c021f9649d\") " Dec 03 12:15:12 crc kubenswrapper[5052]: I1203 12:15:12.677051 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/b25d2a0c-1e93-4b3d-965a-a7c021f9649d-logs\") pod \"b25d2a0c-1e93-4b3d-965a-a7c021f9649d\" (UID: \"b25d2a0c-1e93-4b3d-965a-a7c021f9649d\") " Dec 03 12:15:12 crc kubenswrapper[5052]: I1203 12:15:12.677124 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c2a65caa-2eec-48af-bdad-8fc19d827867-combined-ca-bundle\") pod \"c2a65caa-2eec-48af-bdad-8fc19d827867\" (UID: \"c2a65caa-2eec-48af-bdad-8fc19d827867\") " Dec 03 12:15:12 crc kubenswrapper[5052]: I1203 12:15:12.677204 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r765c\" (UniqueName: \"kubernetes.io/projected/b25d2a0c-1e93-4b3d-965a-a7c021f9649d-kube-api-access-r765c\") pod \"b25d2a0c-1e93-4b3d-965a-a7c021f9649d\" (UID: \"b25d2a0c-1e93-4b3d-965a-a7c021f9649d\") " Dec 03 12:15:12 crc kubenswrapper[5052]: I1203 12:15:12.677296 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/b25d2a0c-1e93-4b3d-965a-a7c021f9649d-public-tls-certs\") pod \"b25d2a0c-1e93-4b3d-965a-a7c021f9649d\" (UID: \"b25d2a0c-1e93-4b3d-965a-a7c021f9649d\") " Dec 03 12:15:12 crc kubenswrapper[5052]: I1203 12:15:12.677377 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/b25d2a0c-1e93-4b3d-965a-a7c021f9649d-config-data-custom\") pod \"b25d2a0c-1e93-4b3d-965a-a7c021f9649d\" (UID: \"b25d2a0c-1e93-4b3d-965a-a7c021f9649d\") " Dec 03 12:15:12 crc kubenswrapper[5052]: I1203 12:15:12.677486 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9qccn\" (UniqueName: \"kubernetes.io/projected/c2a65caa-2eec-48af-bdad-8fc19d827867-kube-api-access-9qccn\") pod \"c2a65caa-2eec-48af-bdad-8fc19d827867\" (UID: \"c2a65caa-2eec-48af-bdad-8fc19d827867\") " Dec 03 12:15:12 crc kubenswrapper[5052]: I1203 12:15:12.677593 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/b25d2a0c-1e93-4b3d-965a-a7c021f9649d-internal-tls-certs\") pod \"b25d2a0c-1e93-4b3d-965a-a7c021f9649d\" (UID: \"b25d2a0c-1e93-4b3d-965a-a7c021f9649d\") " Dec 03 12:15:12 crc kubenswrapper[5052]: I1203 12:15:12.677680 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/89b056a7-e448-447d-be7d-b69f3d4862a1-config-data-custom\") pod \"89b056a7-e448-447d-be7d-b69f3d4862a1\" (UID: \"89b056a7-e448-447d-be7d-b69f3d4862a1\") " Dec 03 12:15:12 crc kubenswrapper[5052]: I1203 12:15:12.677793 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pfjjz\" (UniqueName: \"kubernetes.io/projected/89b056a7-e448-447d-be7d-b69f3d4862a1-kube-api-access-pfjjz\") pod \"89b056a7-e448-447d-be7d-b69f3d4862a1\" (UID: \"89b056a7-e448-447d-be7d-b69f3d4862a1\") " Dec 03 12:15:12 crc kubenswrapper[5052]: I1203 12:15:12.677908 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/89b056a7-e448-447d-be7d-b69f3d4862a1-etc-machine-id\") pod \"89b056a7-e448-447d-be7d-b69f3d4862a1\" (UID: \"89b056a7-e448-447d-be7d-b69f3d4862a1\") " Dec 03 12:15:12 crc kubenswrapper[5052]: I1203 12:15:12.677996 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/c2a65caa-2eec-48af-bdad-8fc19d827867-config-data\") pod \"c2a65caa-2eec-48af-bdad-8fc19d827867\" (UID: \"c2a65caa-2eec-48af-bdad-8fc19d827867\") " Dec 03 12:15:12 crc kubenswrapper[5052]: I1203 12:15:12.678111 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/89b056a7-e448-447d-be7d-b69f3d4862a1-config-data\") pod \"89b056a7-e448-447d-be7d-b69f3d4862a1\" (UID: \"89b056a7-e448-447d-be7d-b69f3d4862a1\") " Dec 03 12:15:12 crc kubenswrapper[5052]: I1203 12:15:12.684992 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b25d2a0c-1e93-4b3d-965a-a7c021f9649d-logs" (OuterVolumeSpecName: "logs") pod "b25d2a0c-1e93-4b3d-965a-a7c021f9649d" (UID: "b25d2a0c-1e93-4b3d-965a-a7c021f9649d"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:15:12 crc kubenswrapper[5052]: I1203 12:15:12.685381 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:15:12 crc kubenswrapper[5052]: I1203 12:15:12.685471 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/89b056a7-e448-447d-be7d-b69f3d4862a1-scripts" (OuterVolumeSpecName: "scripts") pod "89b056a7-e448-447d-be7d-b69f3d4862a1" (UID: "89b056a7-e448-447d-be7d-b69f3d4862a1"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:15:12 crc kubenswrapper[5052]: I1203 12:15:12.685487 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/89b056a7-e448-447d-be7d-b69f3d4862a1-etc-machine-id" (OuterVolumeSpecName: "etc-machine-id") pod "89b056a7-e448-447d-be7d-b69f3d4862a1" (UID: "89b056a7-e448-447d-be7d-b69f3d4862a1"). InnerVolumeSpecName "etc-machine-id". PluginName "kubernetes.io/host-path", VolumeGidValue "" Dec 03 12:15:12 crc kubenswrapper[5052]: I1203 12:15:12.715784 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:15:12 crc kubenswrapper[5052]: I1203 12:15:12.716725 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/89b056a7-e448-447d-be7d-b69f3d4862a1-kube-api-access-pfjjz" (OuterVolumeSpecName: "kube-api-access-pfjjz") pod "89b056a7-e448-447d-be7d-b69f3d4862a1" (UID: "89b056a7-e448-447d-be7d-b69f3d4862a1"). InnerVolumeSpecName "kube-api-access-pfjjz". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:15:12 crc kubenswrapper[5052]: I1203 12:15:12.720935 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b25d2a0c-1e93-4b3d-965a-a7c021f9649d-config-data-custom" (OuterVolumeSpecName: "config-data-custom") pod "b25d2a0c-1e93-4b3d-965a-a7c021f9649d" (UID: "b25d2a0c-1e93-4b3d-965a-a7c021f9649d"). InnerVolumeSpecName "config-data-custom". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:15:12 crc kubenswrapper[5052]: I1203 12:15:12.721973 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c2a65caa-2eec-48af-bdad-8fc19d827867-kube-api-access-9qccn" (OuterVolumeSpecName: "kube-api-access-9qccn") pod "c2a65caa-2eec-48af-bdad-8fc19d827867" (UID: "c2a65caa-2eec-48af-bdad-8fc19d827867"). InnerVolumeSpecName "kube-api-access-9qccn". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:15:12 crc kubenswrapper[5052]: I1203 12:15:12.722257 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/89b056a7-e448-447d-be7d-b69f3d4862a1-config-data-custom" (OuterVolumeSpecName: "config-data-custom") pod "89b056a7-e448-447d-be7d-b69f3d4862a1" (UID: "89b056a7-e448-447d-be7d-b69f3d4862a1"). InnerVolumeSpecName "config-data-custom". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:15:12 crc kubenswrapper[5052]: I1203 12:15:12.741560 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b25d2a0c-1e93-4b3d-965a-a7c021f9649d-kube-api-access-r765c" (OuterVolumeSpecName: "kube-api-access-r765c") pod "b25d2a0c-1e93-4b3d-965a-a7c021f9649d" (UID: "b25d2a0c-1e93-4b3d-965a-a7c021f9649d"). InnerVolumeSpecName "kube-api-access-r765c". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:15:12 crc kubenswrapper[5052]: I1203 12:15:12.780182 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-pfjjz\" (UniqueName: \"kubernetes.io/projected/89b056a7-e448-447d-be7d-b69f3d4862a1-kube-api-access-pfjjz\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:12 crc kubenswrapper[5052]: I1203 12:15:12.780218 5052 reconciler_common.go:293] "Volume detached for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/89b056a7-e448-447d-be7d-b69f3d4862a1-etc-machine-id\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:12 crc kubenswrapper[5052]: I1203 12:15:12.780228 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/89b056a7-e448-447d-be7d-b69f3d4862a1-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:12 crc kubenswrapper[5052]: I1203 12:15:12.780240 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/b25d2a0c-1e93-4b3d-965a-a7c021f9649d-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:12 crc kubenswrapper[5052]: I1203 12:15:12.780251 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-r765c\" (UniqueName: \"kubernetes.io/projected/b25d2a0c-1e93-4b3d-965a-a7c021f9649d-kube-api-access-r765c\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:12 crc kubenswrapper[5052]: I1203 12:15:12.780264 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/b25d2a0c-1e93-4b3d-965a-a7c021f9649d-config-data-custom\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:12 crc kubenswrapper[5052]: I1203 12:15:12.780276 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-9qccn\" (UniqueName: \"kubernetes.io/projected/c2a65caa-2eec-48af-bdad-8fc19d827867-kube-api-access-9qccn\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:12 crc kubenswrapper[5052]: I1203 12:15:12.780287 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/89b056a7-e448-447d-be7d-b69f3d4862a1-config-data-custom\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:12 crc kubenswrapper[5052]: E1203 12:15:12.780354 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/rabbitmq-config-data: configmap "rabbitmq-config-data" not found Dec 03 12:15:12 crc kubenswrapper[5052]: E1203 12:15:12.780409 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f-config-data podName:3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f nodeName:}" failed. No retries permitted until 2025-12-03 12:15:20.780391767 +0000 UTC m=+3311.368633541 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "config-data" (UniqueName: "kubernetes.io/configmap/3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f-config-data") pod "rabbitmq-server-0" (UID: "3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f") : configmap "rabbitmq-config-data" not found Dec 03 12:15:12 crc kubenswrapper[5052]: I1203 12:15:12.787812 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c2a65caa-2eec-48af-bdad-8fc19d827867-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "c2a65caa-2eec-48af-bdad-8fc19d827867" (UID: "c2a65caa-2eec-48af-bdad-8fc19d827867"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:15:12 crc kubenswrapper[5052]: E1203 12:15:12.802308 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="81879a070623f490e700ae5e038359faa2d3ce34a28b8f7e4b856f6c4f95fa7f" cmd=["/usr/bin/pgrep","-r","DRST","nova-scheduler"] Dec 03 12:15:12 crc kubenswrapper[5052]: E1203 12:15:12.805557 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="81879a070623f490e700ae5e038359faa2d3ce34a28b8f7e4b856f6c4f95fa7f" cmd=["/usr/bin/pgrep","-r","DRST","nova-scheduler"] Dec 03 12:15:12 crc kubenswrapper[5052]: E1203 12:15:12.806965 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="81879a070623f490e700ae5e038359faa2d3ce34a28b8f7e4b856f6c4f95fa7f" cmd=["/usr/bin/pgrep","-r","DRST","nova-scheduler"] Dec 03 12:15:12 crc kubenswrapper[5052]: E1203 12:15:12.807007 5052 prober.go:104] "Probe errored" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" probeType="Readiness" pod="openstack-kuttl-tests/nova-scheduler-0" podUID="cf3ed088-9069-40cf-9c9e-ee9bdf7761d9" containerName="nova-scheduler-scheduler" Dec 03 12:15:12 crc kubenswrapper[5052]: I1203 12:15:12.823661 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c2a65caa-2eec-48af-bdad-8fc19d827867-config-data" (OuterVolumeSpecName: "config-data") pod "c2a65caa-2eec-48af-bdad-8fc19d827867" (UID: "c2a65caa-2eec-48af-bdad-8fc19d827867"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:15:12 crc kubenswrapper[5052]: I1203 12:15:12.823691 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b25d2a0c-1e93-4b3d-965a-a7c021f9649d-config-data" (OuterVolumeSpecName: "config-data") pod "b25d2a0c-1e93-4b3d-965a-a7c021f9649d" (UID: "b25d2a0c-1e93-4b3d-965a-a7c021f9649d"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:15:12 crc kubenswrapper[5052]: I1203 12:15:12.831158 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b25d2a0c-1e93-4b3d-965a-a7c021f9649d-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "b25d2a0c-1e93-4b3d-965a-a7c021f9649d" (UID: "b25d2a0c-1e93-4b3d-965a-a7c021f9649d"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:15:12 crc kubenswrapper[5052]: I1203 12:15:12.834340 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/89b056a7-e448-447d-be7d-b69f3d4862a1-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "89b056a7-e448-447d-be7d-b69f3d4862a1" (UID: "89b056a7-e448-447d-be7d-b69f3d4862a1"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:15:12 crc kubenswrapper[5052]: I1203 12:15:12.862215 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b25d2a0c-1e93-4b3d-965a-a7c021f9649d-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "b25d2a0c-1e93-4b3d-965a-a7c021f9649d" (UID: "b25d2a0c-1e93-4b3d-965a-a7c021f9649d"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:15:12 crc kubenswrapper[5052]: I1203 12:15:12.886409 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/c2a65caa-2eec-48af-bdad-8fc19d827867-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:12 crc kubenswrapper[5052]: I1203 12:15:12.886432 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/89b056a7-e448-447d-be7d-b69f3d4862a1-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:12 crc kubenswrapper[5052]: I1203 12:15:12.886443 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/b25d2a0c-1e93-4b3d-965a-a7c021f9649d-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:12 crc kubenswrapper[5052]: I1203 12:15:12.886452 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b25d2a0c-1e93-4b3d-965a-a7c021f9649d-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:12 crc kubenswrapper[5052]: I1203 12:15:12.886460 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c2a65caa-2eec-48af-bdad-8fc19d827867-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:12 crc kubenswrapper[5052]: I1203 12:15:12.886468 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/b25d2a0c-1e93-4b3d-965a-a7c021f9649d-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:12 crc kubenswrapper[5052]: I1203 12:15:12.893814 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b25d2a0c-1e93-4b3d-965a-a7c021f9649d-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "b25d2a0c-1e93-4b3d-965a-a7c021f9649d" (UID: "b25d2a0c-1e93-4b3d-965a-a7c021f9649d"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:15:12 crc kubenswrapper[5052]: I1203 12:15:12.976128 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glancef253-account-delete-cxxlg" Dec 03 12:15:12 crc kubenswrapper[5052]: I1203 12:15:12.983957 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron1287-account-delete-t8dcs" Dec 03 12:15:12 crc kubenswrapper[5052]: I1203 12:15:12.989090 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/b25d2a0c-1e93-4b3d-965a-a7c021f9649d-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:12 crc kubenswrapper[5052]: E1203 12:15:12.989166 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:15:12 crc kubenswrapper[5052]: E1203 12:15:12.989208 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/679f17b9-0e40-480b-bfde-4f63899cb972-operator-scripts podName:679f17b9-0e40-480b-bfde-4f63899cb972 nodeName:}" failed. No retries permitted until 2025-12-03 12:15:13.989194788 +0000 UTC m=+3304.577436552 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/679f17b9-0e40-480b-bfde-4f63899cb972-operator-scripts") pod "cinder7878-account-delete-jwtvt" (UID: "679f17b9-0e40-480b-bfde-4f63899cb972") : configmap "openstack-scripts" not found Dec 03 12:15:12 crc kubenswrapper[5052]: E1203 12:15:12.990388 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:15:12 crc kubenswrapper[5052]: E1203 12:15:12.990568 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c200c246-74d3-48dc-8b24-ac4b37611069-operator-scripts podName:c200c246-74d3-48dc-8b24-ac4b37611069 nodeName:}" failed. No retries permitted until 2025-12-03 12:15:13.990487645 +0000 UTC m=+3304.578729399 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/c200c246-74d3-48dc-8b24-ac4b37611069-operator-scripts") pod "novacell00bb5-account-delete-r2csc" (UID: "c200c246-74d3-48dc-8b24-ac4b37611069") : configmap "openstack-scripts" not found Dec 03 12:15:12 crc kubenswrapper[5052]: I1203 12:15:12.991496 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/89b056a7-e448-447d-be7d-b69f3d4862a1-config-data" (OuterVolumeSpecName: "config-data") pod "89b056a7-e448-447d-be7d-b69f3d4862a1" (UID: "89b056a7-e448-447d-be7d-b69f3d4862a1"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:15:13 crc kubenswrapper[5052]: I1203 12:15:13.004725 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/swift-proxy-6b68c4db5d-g8nvc"] Dec 03 12:15:13 crc kubenswrapper[5052]: I1203 12:15:13.005105 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican8992-account-delete-26lhs" Dec 03 12:15:13 crc kubenswrapper[5052]: I1203 12:15:13.019970 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/memcached-0" Dec 03 12:15:13 crc kubenswrapper[5052]: I1203 12:15:13.045803 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/swift-proxy-6b68c4db5d-g8nvc"] Dec 03 12:15:13 crc kubenswrapper[5052]: I1203 12:15:13.075292 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:15:13 crc kubenswrapper[5052]: I1203 12:15:13.081533 5052 scope.go:117] "RemoveContainer" containerID="2814b190b744e9cf8edd55553cf57bc90eebd7268c2360b6efc06f7b369e975c" Dec 03 12:15:13 crc kubenswrapper[5052]: I1203 12:15:13.095933 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/fee7d732-4fa3-40d5-8fff-cb88272db0ce-operator-scripts\") pod \"fee7d732-4fa3-40d5-8fff-cb88272db0ce\" (UID: \"fee7d732-4fa3-40d5-8fff-cb88272db0ce\") " Dec 03 12:15:13 crc kubenswrapper[5052]: I1203 12:15:13.095988 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b28a125a-7966-4f97-91c1-fa52d6062405-combined-ca-bundle\") pod \"b28a125a-7966-4f97-91c1-fa52d6062405\" (UID: \"b28a125a-7966-4f97-91c1-fa52d6062405\") " Dec 03 12:15:13 crc kubenswrapper[5052]: I1203 12:15:13.096038 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/5856107a-0498-4956-beba-4390f5477ca0-operator-scripts\") pod \"5856107a-0498-4956-beba-4390f5477ca0\" (UID: \"5856107a-0498-4956-beba-4390f5477ca0\") " Dec 03 12:15:13 crc kubenswrapper[5052]: I1203 12:15:13.096063 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"memcached-tls-certs\" (UniqueName: \"kubernetes.io/secret/b28a125a-7966-4f97-91c1-fa52d6062405-memcached-tls-certs\") pod \"b28a125a-7966-4f97-91c1-fa52d6062405\" (UID: \"b28a125a-7966-4f97-91c1-fa52d6062405\") " Dec 03 12:15:13 crc kubenswrapper[5052]: I1203 12:15:13.096107 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kwtt5\" (UniqueName: \"kubernetes.io/projected/5856107a-0498-4956-beba-4390f5477ca0-kube-api-access-kwtt5\") pod \"5856107a-0498-4956-beba-4390f5477ca0\" (UID: \"5856107a-0498-4956-beba-4390f5477ca0\") " Dec 03 12:15:13 crc kubenswrapper[5052]: I1203 12:15:13.096183 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qwfl2\" (UniqueName: \"kubernetes.io/projected/fee7d732-4fa3-40d5-8fff-cb88272db0ce-kube-api-access-qwfl2\") pod \"fee7d732-4fa3-40d5-8fff-cb88272db0ce\" (UID: \"fee7d732-4fa3-40d5-8fff-cb88272db0ce\") " Dec 03 12:15:13 crc kubenswrapper[5052]: I1203 12:15:13.096292 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/f1619fbf-df68-484f-99d4-09b81a7a334e-operator-scripts\") pod \"f1619fbf-df68-484f-99d4-09b81a7a334e\" (UID: \"f1619fbf-df68-484f-99d4-09b81a7a334e\") " Dec 03 12:15:13 crc kubenswrapper[5052]: I1203 12:15:13.096319 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/b28a125a-7966-4f97-91c1-fa52d6062405-config-data\") pod \"b28a125a-7966-4f97-91c1-fa52d6062405\" (UID: \"b28a125a-7966-4f97-91c1-fa52d6062405\") " Dec 03 12:15:13 crc kubenswrapper[5052]: I1203 12:15:13.096399 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xxqfp\" (UniqueName: \"kubernetes.io/projected/f1619fbf-df68-484f-99d4-09b81a7a334e-kube-api-access-xxqfp\") pod \"f1619fbf-df68-484f-99d4-09b81a7a334e\" (UID: \"f1619fbf-df68-484f-99d4-09b81a7a334e\") " Dec 03 12:15:13 crc kubenswrapper[5052]: I1203 12:15:13.096436 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8rz4s\" (UniqueName: \"kubernetes.io/projected/b28a125a-7966-4f97-91c1-fa52d6062405-kube-api-access-8rz4s\") pod \"b28a125a-7966-4f97-91c1-fa52d6062405\" (UID: \"b28a125a-7966-4f97-91c1-fa52d6062405\") " Dec 03 12:15:13 crc kubenswrapper[5052]: I1203 12:15:13.096464 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/b28a125a-7966-4f97-91c1-fa52d6062405-kolla-config\") pod \"b28a125a-7966-4f97-91c1-fa52d6062405\" (UID: \"b28a125a-7966-4f97-91c1-fa52d6062405\") " Dec 03 12:15:13 crc kubenswrapper[5052]: I1203 12:15:13.096926 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/89b056a7-e448-447d-be7d-b69f3d4862a1-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:13 crc kubenswrapper[5052]: I1203 12:15:13.097498 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b28a125a-7966-4f97-91c1-fa52d6062405-kolla-config" (OuterVolumeSpecName: "kolla-config") pod "b28a125a-7966-4f97-91c1-fa52d6062405" (UID: "b28a125a-7966-4f97-91c1-fa52d6062405"). InnerVolumeSpecName "kolla-config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:15:13 crc kubenswrapper[5052]: I1203 12:15:13.098072 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fee7d732-4fa3-40d5-8fff-cb88272db0ce-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "fee7d732-4fa3-40d5-8fff-cb88272db0ce" (UID: "fee7d732-4fa3-40d5-8fff-cb88272db0ce"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:15:13 crc kubenswrapper[5052]: I1203 12:15:13.101102 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:15:13 crc kubenswrapper[5052]: I1203 12:15:13.101556 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fee7d732-4fa3-40d5-8fff-cb88272db0ce-kube-api-access-qwfl2" (OuterVolumeSpecName: "kube-api-access-qwfl2") pod "fee7d732-4fa3-40d5-8fff-cb88272db0ce" (UID: "fee7d732-4fa3-40d5-8fff-cb88272db0ce"). InnerVolumeSpecName "kube-api-access-qwfl2". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:15:13 crc kubenswrapper[5052]: I1203 12:15:13.104646 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5856107a-0498-4956-beba-4390f5477ca0-kube-api-access-kwtt5" (OuterVolumeSpecName: "kube-api-access-kwtt5") pod "5856107a-0498-4956-beba-4390f5477ca0" (UID: "5856107a-0498-4956-beba-4390f5477ca0"). InnerVolumeSpecName "kube-api-access-kwtt5". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:15:13 crc kubenswrapper[5052]: I1203 12:15:13.106665 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b28a125a-7966-4f97-91c1-fa52d6062405-config-data" (OuterVolumeSpecName: "config-data") pod "b28a125a-7966-4f97-91c1-fa52d6062405" (UID: "b28a125a-7966-4f97-91c1-fa52d6062405"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:15:13 crc kubenswrapper[5052]: I1203 12:15:13.107272 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5856107a-0498-4956-beba-4390f5477ca0-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "5856107a-0498-4956-beba-4390f5477ca0" (UID: "5856107a-0498-4956-beba-4390f5477ca0"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:15:13 crc kubenswrapper[5052]: I1203 12:15:13.112322 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f1619fbf-df68-484f-99d4-09b81a7a334e-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "f1619fbf-df68-484f-99d4-09b81a7a334e" (UID: "f1619fbf-df68-484f-99d4-09b81a7a334e"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:15:13 crc kubenswrapper[5052]: I1203 12:15:13.112488 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f1619fbf-df68-484f-99d4-09b81a7a334e-kube-api-access-xxqfp" (OuterVolumeSpecName: "kube-api-access-xxqfp") pod "f1619fbf-df68-484f-99d4-09b81a7a334e" (UID: "f1619fbf-df68-484f-99d4-09b81a7a334e"). InnerVolumeSpecName "kube-api-access-xxqfp". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:15:13 crc kubenswrapper[5052]: I1203 12:15:13.128621 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b28a125a-7966-4f97-91c1-fa52d6062405-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "b28a125a-7966-4f97-91c1-fa52d6062405" (UID: "b28a125a-7966-4f97-91c1-fa52d6062405"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:15:13 crc kubenswrapper[5052]: I1203 12:15:13.145590 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:15:13 crc kubenswrapper[5052]: I1203 12:15:13.145856 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b28a125a-7966-4f97-91c1-fa52d6062405-kube-api-access-8rz4s" (OuterVolumeSpecName: "kube-api-access-8rz4s") pod "b28a125a-7966-4f97-91c1-fa52d6062405" (UID: "b28a125a-7966-4f97-91c1-fa52d6062405"). InnerVolumeSpecName "kube-api-access-8rz4s". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:15:13 crc kubenswrapper[5052]: I1203 12:15:13.157648 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:15:13 crc kubenswrapper[5052]: I1203 12:15:13.182309 5052 scope.go:117] "RemoveContainer" containerID="2b2c845f4fc350fd56a0f9ca2b46def3a0f193a23ad1b9dd526750dac6bdc0e7" Dec 03 12:15:13 crc kubenswrapper[5052]: I1203 12:15:13.191965 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:15:13 crc kubenswrapper[5052]: I1203 12:15:13.198284 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"server-conf\" (UniqueName: \"kubernetes.io/configmap/e63eb1eb-e0b9-46ea-a19b-8f5c018728ff-server-conf\") pod \"e63eb1eb-e0b9-46ea-a19b-8f5c018728ff\" (UID: \"e63eb1eb-e0b9-46ea-a19b-8f5c018728ff\") " Dec 03 12:15:13 crc kubenswrapper[5052]: I1203 12:15:13.198348 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"persistence\" (UniqueName: \"kubernetes.io/local-volume/local-storage04-crc\") pod \"e63eb1eb-e0b9-46ea-a19b-8f5c018728ff\" (UID: \"e63eb1eb-e0b9-46ea-a19b-8f5c018728ff\") " Dec 03 12:15:13 crc kubenswrapper[5052]: I1203 12:15:13.198442 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"rabbitmq-confd\" (UniqueName: \"kubernetes.io/projected/e63eb1eb-e0b9-46ea-a19b-8f5c018728ff-rabbitmq-confd\") pod \"e63eb1eb-e0b9-46ea-a19b-8f5c018728ff\" (UID: \"e63eb1eb-e0b9-46ea-a19b-8f5c018728ff\") " Dec 03 12:15:13 crc kubenswrapper[5052]: I1203 12:15:13.198470 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"pod-info\" (UniqueName: \"kubernetes.io/downward-api/e63eb1eb-e0b9-46ea-a19b-8f5c018728ff-pod-info\") pod \"e63eb1eb-e0b9-46ea-a19b-8f5c018728ff\" (UID: \"e63eb1eb-e0b9-46ea-a19b-8f5c018728ff\") " Dec 03 12:15:13 crc kubenswrapper[5052]: I1203 12:15:13.198502 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/e63eb1eb-e0b9-46ea-a19b-8f5c018728ff-config-data\") pod \"e63eb1eb-e0b9-46ea-a19b-8f5c018728ff\" (UID: \"e63eb1eb-e0b9-46ea-a19b-8f5c018728ff\") " Dec 03 12:15:13 crc kubenswrapper[5052]: I1203 12:15:13.198555 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"erlang-cookie-secret\" (UniqueName: \"kubernetes.io/secret/e63eb1eb-e0b9-46ea-a19b-8f5c018728ff-erlang-cookie-secret\") pod \"e63eb1eb-e0b9-46ea-a19b-8f5c018728ff\" (UID: \"e63eb1eb-e0b9-46ea-a19b-8f5c018728ff\") " Dec 03 12:15:13 crc kubenswrapper[5052]: I1203 12:15:13.198590 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"rabbitmq-erlang-cookie\" (UniqueName: \"kubernetes.io/empty-dir/e63eb1eb-e0b9-46ea-a19b-8f5c018728ff-rabbitmq-erlang-cookie\") pod \"e63eb1eb-e0b9-46ea-a19b-8f5c018728ff\" (UID: \"e63eb1eb-e0b9-46ea-a19b-8f5c018728ff\") " Dec 03 12:15:13 crc kubenswrapper[5052]: I1203 12:15:13.198611 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zlvwl\" (UniqueName: \"kubernetes.io/projected/e63eb1eb-e0b9-46ea-a19b-8f5c018728ff-kube-api-access-zlvwl\") pod \"e63eb1eb-e0b9-46ea-a19b-8f5c018728ff\" (UID: \"e63eb1eb-e0b9-46ea-a19b-8f5c018728ff\") " Dec 03 12:15:13 crc kubenswrapper[5052]: I1203 12:15:13.198674 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"plugins-conf\" (UniqueName: \"kubernetes.io/configmap/e63eb1eb-e0b9-46ea-a19b-8f5c018728ff-plugins-conf\") pod \"e63eb1eb-e0b9-46ea-a19b-8f5c018728ff\" (UID: \"e63eb1eb-e0b9-46ea-a19b-8f5c018728ff\") " Dec 03 12:15:13 crc kubenswrapper[5052]: I1203 12:15:13.198695 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"rabbitmq-plugins\" (UniqueName: \"kubernetes.io/empty-dir/e63eb1eb-e0b9-46ea-a19b-8f5c018728ff-rabbitmq-plugins\") pod \"e63eb1eb-e0b9-46ea-a19b-8f5c018728ff\" (UID: \"e63eb1eb-e0b9-46ea-a19b-8f5c018728ff\") " Dec 03 12:15:13 crc kubenswrapper[5052]: I1203 12:15:13.198715 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"rabbitmq-tls\" (UniqueName: \"kubernetes.io/projected/e63eb1eb-e0b9-46ea-a19b-8f5c018728ff-rabbitmq-tls\") pod \"e63eb1eb-e0b9-46ea-a19b-8f5c018728ff\" (UID: \"e63eb1eb-e0b9-46ea-a19b-8f5c018728ff\") " Dec 03 12:15:13 crc kubenswrapper[5052]: I1203 12:15:13.199026 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-kwtt5\" (UniqueName: \"kubernetes.io/projected/5856107a-0498-4956-beba-4390f5477ca0-kube-api-access-kwtt5\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:13 crc kubenswrapper[5052]: I1203 12:15:13.199048 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-qwfl2\" (UniqueName: \"kubernetes.io/projected/fee7d732-4fa3-40d5-8fff-cb88272db0ce-kube-api-access-qwfl2\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:13 crc kubenswrapper[5052]: I1203 12:15:13.199061 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/f1619fbf-df68-484f-99d4-09b81a7a334e-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:13 crc kubenswrapper[5052]: I1203 12:15:13.199071 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/b28a125a-7966-4f97-91c1-fa52d6062405-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:13 crc kubenswrapper[5052]: I1203 12:15:13.199082 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-xxqfp\" (UniqueName: \"kubernetes.io/projected/f1619fbf-df68-484f-99d4-09b81a7a334e-kube-api-access-xxqfp\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:13 crc kubenswrapper[5052]: I1203 12:15:13.199091 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-8rz4s\" (UniqueName: \"kubernetes.io/projected/b28a125a-7966-4f97-91c1-fa52d6062405-kube-api-access-8rz4s\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:13 crc kubenswrapper[5052]: I1203 12:15:13.199099 5052 reconciler_common.go:293] "Volume detached for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/b28a125a-7966-4f97-91c1-fa52d6062405-kolla-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:13 crc kubenswrapper[5052]: I1203 12:15:13.199110 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/fee7d732-4fa3-40d5-8fff-cb88272db0ce-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:13 crc kubenswrapper[5052]: I1203 12:15:13.199118 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b28a125a-7966-4f97-91c1-fa52d6062405-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:13 crc kubenswrapper[5052]: I1203 12:15:13.199126 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/5856107a-0498-4956-beba-4390f5477ca0-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:13 crc kubenswrapper[5052]: I1203 12:15:13.200265 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e63eb1eb-e0b9-46ea-a19b-8f5c018728ff-plugins-conf" (OuterVolumeSpecName: "plugins-conf") pod "e63eb1eb-e0b9-46ea-a19b-8f5c018728ff" (UID: "e63eb1eb-e0b9-46ea-a19b-8f5c018728ff"). InnerVolumeSpecName "plugins-conf". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:15:13 crc kubenswrapper[5052]: I1203 12:15:13.201143 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e63eb1eb-e0b9-46ea-a19b-8f5c018728ff-rabbitmq-plugins" (OuterVolumeSpecName: "rabbitmq-plugins") pod "e63eb1eb-e0b9-46ea-a19b-8f5c018728ff" (UID: "e63eb1eb-e0b9-46ea-a19b-8f5c018728ff"). InnerVolumeSpecName "rabbitmq-plugins". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:15:13 crc kubenswrapper[5052]: I1203 12:15:13.201612 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e63eb1eb-e0b9-46ea-a19b-8f5c018728ff-rabbitmq-erlang-cookie" (OuterVolumeSpecName: "rabbitmq-erlang-cookie") pod "e63eb1eb-e0b9-46ea-a19b-8f5c018728ff" (UID: "e63eb1eb-e0b9-46ea-a19b-8f5c018728ff"). InnerVolumeSpecName "rabbitmq-erlang-cookie". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:15:13 crc kubenswrapper[5052]: I1203 12:15:13.203165 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b28a125a-7966-4f97-91c1-fa52d6062405-memcached-tls-certs" (OuterVolumeSpecName: "memcached-tls-certs") pod "b28a125a-7966-4f97-91c1-fa52d6062405" (UID: "b28a125a-7966-4f97-91c1-fa52d6062405"). InnerVolumeSpecName "memcached-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:15:13 crc kubenswrapper[5052]: I1203 12:15:13.205809 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e63eb1eb-e0b9-46ea-a19b-8f5c018728ff-rabbitmq-tls" (OuterVolumeSpecName: "rabbitmq-tls") pod "e63eb1eb-e0b9-46ea-a19b-8f5c018728ff" (UID: "e63eb1eb-e0b9-46ea-a19b-8f5c018728ff"). InnerVolumeSpecName "rabbitmq-tls". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:15:13 crc kubenswrapper[5052]: I1203 12:15:13.207350 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/placement-79d5cb4fdb-7h2fn"] Dec 03 12:15:13 crc kubenswrapper[5052]: I1203 12:15:13.211789 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage04-crc" (OuterVolumeSpecName: "persistence") pod "e63eb1eb-e0b9-46ea-a19b-8f5c018728ff" (UID: "e63eb1eb-e0b9-46ea-a19b-8f5c018728ff"). InnerVolumeSpecName "local-storage04-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 12:15:13 crc kubenswrapper[5052]: I1203 12:15:13.211896 5052 scope.go:117] "RemoveContainer" containerID="96dc968ef689e331a6df033c2d763c10c20b7c7df416d6abc28abb2477d6b9ee" Dec 03 12:15:13 crc kubenswrapper[5052]: I1203 12:15:13.221895 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/placement-79d5cb4fdb-7h2fn"] Dec 03 12:15:13 crc kubenswrapper[5052]: I1203 12:15:13.224864 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e63eb1eb-e0b9-46ea-a19b-8f5c018728ff-pod-info" (OuterVolumeSpecName: "pod-info") pod "e63eb1eb-e0b9-46ea-a19b-8f5c018728ff" (UID: "e63eb1eb-e0b9-46ea-a19b-8f5c018728ff"). InnerVolumeSpecName "pod-info". PluginName "kubernetes.io/downward-api", VolumeGidValue "" Dec 03 12:15:13 crc kubenswrapper[5052]: I1203 12:15:13.226099 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e63eb1eb-e0b9-46ea-a19b-8f5c018728ff-erlang-cookie-secret" (OuterVolumeSpecName: "erlang-cookie-secret") pod "e63eb1eb-e0b9-46ea-a19b-8f5c018728ff" (UID: "e63eb1eb-e0b9-46ea-a19b-8f5c018728ff"). InnerVolumeSpecName "erlang-cookie-secret". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:15:13 crc kubenswrapper[5052]: I1203 12:15:13.238034 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e63eb1eb-e0b9-46ea-a19b-8f5c018728ff-kube-api-access-zlvwl" (OuterVolumeSpecName: "kube-api-access-zlvwl") pod "e63eb1eb-e0b9-46ea-a19b-8f5c018728ff" (UID: "e63eb1eb-e0b9-46ea-a19b-8f5c018728ff"). InnerVolumeSpecName "kube-api-access-zlvwl". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:15:13 crc kubenswrapper[5052]: I1203 12:15:13.238433 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e63eb1eb-e0b9-46ea-a19b-8f5c018728ff-config-data" (OuterVolumeSpecName: "config-data") pod "e63eb1eb-e0b9-46ea-a19b-8f5c018728ff" (UID: "e63eb1eb-e0b9-46ea-a19b-8f5c018728ff"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:15:13 crc kubenswrapper[5052]: I1203 12:15:13.238490 5052 scope.go:117] "RemoveContainer" containerID="df21ee50a92cecc1b709d001db731836cf35f7648b75dbb8f7cbdbbb13f8e727" Dec 03 12:15:13 crc kubenswrapper[5052]: I1203 12:15:13.305164 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e63eb1eb-e0b9-46ea-a19b-8f5c018728ff-server-conf" (OuterVolumeSpecName: "server-conf") pod "e63eb1eb-e0b9-46ea-a19b-8f5c018728ff" (UID: "e63eb1eb-e0b9-46ea-a19b-8f5c018728ff"). InnerVolumeSpecName "server-conf". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:15:13 crc kubenswrapper[5052]: I1203 12:15:13.332320 5052 reconciler_common.go:293] "Volume detached for volume \"pod-info\" (UniqueName: \"kubernetes.io/downward-api/e63eb1eb-e0b9-46ea-a19b-8f5c018728ff-pod-info\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:13 crc kubenswrapper[5052]: I1203 12:15:13.332359 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/e63eb1eb-e0b9-46ea-a19b-8f5c018728ff-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:13 crc kubenswrapper[5052]: I1203 12:15:13.332370 5052 reconciler_common.go:293] "Volume detached for volume \"erlang-cookie-secret\" (UniqueName: \"kubernetes.io/secret/e63eb1eb-e0b9-46ea-a19b-8f5c018728ff-erlang-cookie-secret\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:13 crc kubenswrapper[5052]: I1203 12:15:13.332383 5052 reconciler_common.go:293] "Volume detached for volume \"rabbitmq-erlang-cookie\" (UniqueName: \"kubernetes.io/empty-dir/e63eb1eb-e0b9-46ea-a19b-8f5c018728ff-rabbitmq-erlang-cookie\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:13 crc kubenswrapper[5052]: I1203 12:15:13.332398 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-zlvwl\" (UniqueName: \"kubernetes.io/projected/e63eb1eb-e0b9-46ea-a19b-8f5c018728ff-kube-api-access-zlvwl\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:13 crc kubenswrapper[5052]: I1203 12:15:13.332411 5052 reconciler_common.go:293] "Volume detached for volume \"plugins-conf\" (UniqueName: \"kubernetes.io/configmap/e63eb1eb-e0b9-46ea-a19b-8f5c018728ff-plugins-conf\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:13 crc kubenswrapper[5052]: I1203 12:15:13.332423 5052 reconciler_common.go:293] "Volume detached for volume \"rabbitmq-plugins\" (UniqueName: \"kubernetes.io/empty-dir/e63eb1eb-e0b9-46ea-a19b-8f5c018728ff-rabbitmq-plugins\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:13 crc kubenswrapper[5052]: I1203 12:15:13.332433 5052 reconciler_common.go:293] "Volume detached for volume \"rabbitmq-tls\" (UniqueName: \"kubernetes.io/projected/e63eb1eb-e0b9-46ea-a19b-8f5c018728ff-rabbitmq-tls\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:13 crc kubenswrapper[5052]: I1203 12:15:13.332444 5052 reconciler_common.go:293] "Volume detached for volume \"server-conf\" (UniqueName: \"kubernetes.io/configmap/e63eb1eb-e0b9-46ea-a19b-8f5c018728ff-server-conf\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:13 crc kubenswrapper[5052]: I1203 12:15:13.332470 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage04-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage04-crc\") on node \"crc\" " Dec 03 12:15:13 crc kubenswrapper[5052]: I1203 12:15:13.332483 5052 reconciler_common.go:293] "Volume detached for volume \"memcached-tls-certs\" (UniqueName: \"kubernetes.io/secret/b28a125a-7966-4f97-91c1-fa52d6062405-memcached-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:13 crc kubenswrapper[5052]: I1203 12:15:13.383062 5052 scope.go:117] "RemoveContainer" containerID="ec66627f7d357cab05559c6d34ef624f9d761ad2e4531d42cccb0b06d7bb2cba" Dec 03 12:15:13 crc kubenswrapper[5052]: I1203 12:15:13.501045 5052 scope.go:117] "RemoveContainer" containerID="99335f1182dfcd9be7ce5b7f281d04f752ed33796c96039024fa49675032786b" Dec 03 12:15:13 crc kubenswrapper[5052]: I1203 12:15:13.507140 5052 generic.go:334] "Generic (PLEG): container finished" podID="e63eb1eb-e0b9-46ea-a19b-8f5c018728ff" containerID="114f5fc894c9a2d64f9ed86ad645c9cb54b0a4c854a1d0287b2c2a82e3627521" exitCode=0 Dec 03 12:15:13 crc kubenswrapper[5052]: I1203 12:15:13.507315 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" event={"ID":"e63eb1eb-e0b9-46ea-a19b-8f5c018728ff","Type":"ContainerDied","Data":"114f5fc894c9a2d64f9ed86ad645c9cb54b0a4c854a1d0287b2c2a82e3627521"} Dec 03 12:15:13 crc kubenswrapper[5052]: I1203 12:15:13.507416 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" event={"ID":"e63eb1eb-e0b9-46ea-a19b-8f5c018728ff","Type":"ContainerDied","Data":"3fe470592acd81e517aa2c5a619c15d20df1c0a04a8a73f91848113c03fe7642"} Dec 03 12:15:13 crc kubenswrapper[5052]: I1203 12:15:13.507566 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:15:13 crc kubenswrapper[5052]: I1203 12:15:13.512284 5052 generic.go:334] "Generic (PLEG): container finished" podID="3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f" containerID="353f13dfb90cbf0598b28536ad392c1afc49332edb49e65bfaddb985602e41d1" exitCode=0 Dec 03 12:15:13 crc kubenswrapper[5052]: I1203 12:15:13.512469 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-server-0" event={"ID":"3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f","Type":"ContainerDied","Data":"353f13dfb90cbf0598b28536ad392c1afc49332edb49e65bfaddb985602e41d1"} Dec 03 12:15:13 crc kubenswrapper[5052]: I1203 12:15:13.513448 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican8992-account-delete-26lhs" Dec 03 12:15:13 crc kubenswrapper[5052]: I1203 12:15:13.515799 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage04-crc" (UniqueName: "kubernetes.io/local-volume/local-storage04-crc") on node "crc" Dec 03 12:15:13 crc kubenswrapper[5052]: I1203 12:15:13.515908 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron1287-account-delete-t8dcs" Dec 03 12:15:13 crc kubenswrapper[5052]: I1203 12:15:13.519638 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-api-58464649dd-sx8mw" Dec 03 12:15:13 crc kubenswrapper[5052]: I1203 12:15:13.519740 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/novacell00bb5-account-delete-r2csc" podUID="c200c246-74d3-48dc-8b24-ac4b37611069" containerName="mariadb-account-delete" containerID="cri-o://c7a74e6ab5fc2e0b601148d2ded033a3a334d1f7ad582da015664c00d9a6e301" gracePeriod=30 Dec 03 12:15:13 crc kubenswrapper[5052]: I1203 12:15:13.520161 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/memcached-0" Dec 03 12:15:13 crc kubenswrapper[5052]: I1203 12:15:13.520743 5052 kubelet_pods.go:1007] "Unable to retrieve pull secret, the image pull may not succeed." pod="openstack-kuttl-tests/cinder7878-account-delete-jwtvt" secret="" err="secret \"galera-openstack-dockercfg-vpq7c\" not found" Dec 03 12:15:13 crc kubenswrapper[5052]: I1203 12:15:13.521097 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:15:13 crc kubenswrapper[5052]: I1203 12:15:13.521124 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:15:13 crc kubenswrapper[5052]: I1203 12:15:13.525089 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glancef253-account-delete-cxxlg" Dec 03 12:15:13 crc kubenswrapper[5052]: I1203 12:15:13.545715 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage04-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage04-crc\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:13 crc kubenswrapper[5052]: I1203 12:15:13.615157 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e63eb1eb-e0b9-46ea-a19b-8f5c018728ff-rabbitmq-confd" (OuterVolumeSpecName: "rabbitmq-confd") pod "e63eb1eb-e0b9-46ea-a19b-8f5c018728ff" (UID: "e63eb1eb-e0b9-46ea-a19b-8f5c018728ff"). InnerVolumeSpecName "rabbitmq-confd". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:15:13 crc kubenswrapper[5052]: I1203 12:15:13.655144 5052 reconciler_common.go:293] "Volume detached for volume \"rabbitmq-confd\" (UniqueName: \"kubernetes.io/projected/e63eb1eb-e0b9-46ea-a19b-8f5c018728ff-rabbitmq-confd\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:13 crc kubenswrapper[5052]: I1203 12:15:13.685401 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:15:13 crc kubenswrapper[5052]: I1203 12:15:13.687588 5052 scope.go:117] "RemoveContainer" containerID="7024c66624b9b95f71fa0589d2f50417eaa07d7b921b5356f06724331235a546" Dec 03 12:15:13 crc kubenswrapper[5052]: I1203 12:15:13.692227 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2a349843-a7b7-4dff-a5ae-f513337c5809" path="/var/lib/kubelet/pods/2a349843-a7b7-4dff-a5ae-f513337c5809/volumes" Dec 03 12:15:13 crc kubenswrapper[5052]: I1203 12:15:13.694995 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8283de30-8976-47bd-bd09-aab53140f68d" path="/var/lib/kubelet/pods/8283de30-8976-47bd-bd09-aab53140f68d/volumes" Dec 03 12:15:13 crc kubenswrapper[5052]: I1203 12:15:13.697974 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ae6940a4-0f4c-46ab-8218-56bf5c5d3084" path="/var/lib/kubelet/pods/ae6940a4-0f4c-46ab-8218-56bf5c5d3084/volumes" Dec 03 12:15:13 crc kubenswrapper[5052]: I1203 12:15:13.700245 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cb68adf1-26f2-4b14-9874-dde28826aba3" path="/var/lib/kubelet/pods/cb68adf1-26f2-4b14-9874-dde28826aba3/volumes" Dec 03 12:15:13 crc kubenswrapper[5052]: I1203 12:15:13.701961 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cd486343-0aad-4039-a23b-34c60cd1a381" path="/var/lib/kubelet/pods/cd486343-0aad-4039-a23b-34c60cd1a381/volumes" Dec 03 12:15:13 crc kubenswrapper[5052]: I1203 12:15:13.703485 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d81b09dc-75a1-45f8-ace4-786531ad3bfd" path="/var/lib/kubelet/pods/d81b09dc-75a1-45f8-ace4-786531ad3bfd/volumes" Dec 03 12:15:13 crc kubenswrapper[5052]: I1203 12:15:13.756463 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"persistence\" (UniqueName: \"kubernetes.io/local-volume/local-storage05-crc\") pod \"3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f\" (UID: \"3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f\") " Dec 03 12:15:13 crc kubenswrapper[5052]: I1203 12:15:13.756589 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"plugins-conf\" (UniqueName: \"kubernetes.io/configmap/3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f-plugins-conf\") pod \"3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f\" (UID: \"3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f\") " Dec 03 12:15:13 crc kubenswrapper[5052]: I1203 12:15:13.756635 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"rabbitmq-confd\" (UniqueName: \"kubernetes.io/projected/3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f-rabbitmq-confd\") pod \"3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f\" (UID: \"3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f\") " Dec 03 12:15:13 crc kubenswrapper[5052]: I1203 12:15:13.756687 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"rabbitmq-tls\" (UniqueName: \"kubernetes.io/projected/3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f-rabbitmq-tls\") pod \"3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f\" (UID: \"3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f\") " Dec 03 12:15:13 crc kubenswrapper[5052]: I1203 12:15:13.756734 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"erlang-cookie-secret\" (UniqueName: \"kubernetes.io/secret/3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f-erlang-cookie-secret\") pod \"3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f\" (UID: \"3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f\") " Dec 03 12:15:13 crc kubenswrapper[5052]: I1203 12:15:13.756770 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"rabbitmq-plugins\" (UniqueName: \"kubernetes.io/empty-dir/3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f-rabbitmq-plugins\") pod \"3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f\" (UID: \"3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f\") " Dec 03 12:15:13 crc kubenswrapper[5052]: I1203 12:15:13.757385 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"server-conf\" (UniqueName: \"kubernetes.io/configmap/3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f-server-conf\") pod \"3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f\" (UID: \"3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f\") " Dec 03 12:15:13 crc kubenswrapper[5052]: I1203 12:15:13.757419 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"rabbitmq-erlang-cookie\" (UniqueName: \"kubernetes.io/empty-dir/3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f-rabbitmq-erlang-cookie\") pod \"3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f\" (UID: \"3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f\") " Dec 03 12:15:13 crc kubenswrapper[5052]: I1203 12:15:13.757474 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fm9bz\" (UniqueName: \"kubernetes.io/projected/3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f-kube-api-access-fm9bz\") pod \"3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f\" (UID: \"3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f\") " Dec 03 12:15:13 crc kubenswrapper[5052]: I1203 12:15:13.757552 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f-config-data\") pod \"3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f\" (UID: \"3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f\") " Dec 03 12:15:13 crc kubenswrapper[5052]: I1203 12:15:13.757581 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"pod-info\" (UniqueName: \"kubernetes.io/downward-api/3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f-pod-info\") pod \"3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f\" (UID: \"3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f\") " Dec 03 12:15:13 crc kubenswrapper[5052]: I1203 12:15:13.759704 5052 scope.go:117] "RemoveContainer" containerID="c15684b97ff4a9d54a9eeca81697f4cced793ffe157adb3548bedd21bf3249d0" Dec 03 12:15:13 crc kubenswrapper[5052]: I1203 12:15:13.762313 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f-plugins-conf" (OuterVolumeSpecName: "plugins-conf") pod "3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f" (UID: "3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f"). InnerVolumeSpecName "plugins-conf". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:15:13 crc kubenswrapper[5052]: I1203 12:15:13.765580 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-0"] Dec 03 12:15:13 crc kubenswrapper[5052]: I1203 12:15:13.765932 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f-rabbitmq-erlang-cookie" (OuterVolumeSpecName: "rabbitmq-erlang-cookie") pod "3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f" (UID: "3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f"). InnerVolumeSpecName "rabbitmq-erlang-cookie". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:15:13 crc kubenswrapper[5052]: I1203 12:15:13.773108 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage05-crc" (OuterVolumeSpecName: "persistence") pod "3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f" (UID: "3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f"). InnerVolumeSpecName "local-storage05-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 12:15:13 crc kubenswrapper[5052]: I1203 12:15:13.774434 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f-rabbitmq-plugins" (OuterVolumeSpecName: "rabbitmq-plugins") pod "3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f" (UID: "3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f"). InnerVolumeSpecName "rabbitmq-plugins". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:15:13 crc kubenswrapper[5052]: I1203 12:15:13.775196 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f-kube-api-access-fm9bz" (OuterVolumeSpecName: "kube-api-access-fm9bz") pod "3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f" (UID: "3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f"). InnerVolumeSpecName "kube-api-access-fm9bz". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:15:13 crc kubenswrapper[5052]: I1203 12:15:13.781424 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f-erlang-cookie-secret" (OuterVolumeSpecName: "erlang-cookie-secret") pod "3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f" (UID: "3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f"). InnerVolumeSpecName "erlang-cookie-secret". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:15:13 crc kubenswrapper[5052]: I1203 12:15:13.788018 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-0"] Dec 03 12:15:13 crc kubenswrapper[5052]: I1203 12:15:13.791542 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f-rabbitmq-tls" (OuterVolumeSpecName: "rabbitmq-tls") pod "3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f" (UID: "3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f"). InnerVolumeSpecName "rabbitmq-tls". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:15:13 crc kubenswrapper[5052]: I1203 12:15:13.804804 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/memcached-0"] Dec 03 12:15:13 crc kubenswrapper[5052]: I1203 12:15:13.815845 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/memcached-0"] Dec 03 12:15:13 crc kubenswrapper[5052]: I1203 12:15:13.860264 5052 reconciler_common.go:293] "Volume detached for volume \"rabbitmq-erlang-cookie\" (UniqueName: \"kubernetes.io/empty-dir/3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f-rabbitmq-erlang-cookie\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:13 crc kubenswrapper[5052]: I1203 12:15:13.860297 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-fm9bz\" (UniqueName: \"kubernetes.io/projected/3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f-kube-api-access-fm9bz\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:13 crc kubenswrapper[5052]: I1203 12:15:13.860334 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage05-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage05-crc\") on node \"crc\" " Dec 03 12:15:13 crc kubenswrapper[5052]: I1203 12:15:13.860350 5052 reconciler_common.go:293] "Volume detached for volume \"plugins-conf\" (UniqueName: \"kubernetes.io/configmap/3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f-plugins-conf\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:13 crc kubenswrapper[5052]: I1203 12:15:13.860360 5052 reconciler_common.go:293] "Volume detached for volume \"rabbitmq-tls\" (UniqueName: \"kubernetes.io/projected/3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f-rabbitmq-tls\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:13 crc kubenswrapper[5052]: I1203 12:15:13.860370 5052 reconciler_common.go:293] "Volume detached for volume \"erlang-cookie-secret\" (UniqueName: \"kubernetes.io/secret/3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f-erlang-cookie-secret\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:13 crc kubenswrapper[5052]: I1203 12:15:13.860380 5052 reconciler_common.go:293] "Volume detached for volume \"rabbitmq-plugins\" (UniqueName: \"kubernetes.io/empty-dir/3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f-rabbitmq-plugins\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:13 crc kubenswrapper[5052]: I1203 12:15:13.864216 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f-config-data" (OuterVolumeSpecName: "config-data") pod "3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f" (UID: "3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:15:13 crc kubenswrapper[5052]: I1203 12:15:13.864068 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 12:15:13 crc kubenswrapper[5052]: I1203 12:15:13.864678 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f-pod-info" (OuterVolumeSpecName: "pod-info") pod "3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f" (UID: "3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f"). InnerVolumeSpecName "pod-info". PluginName "kubernetes.io/downward-api", VolumeGidValue "" Dec 03 12:15:13 crc kubenswrapper[5052]: I1203 12:15:13.887833 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 12:15:13 crc kubenswrapper[5052]: I1203 12:15:13.899988 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage05-crc" (UniqueName: "kubernetes.io/local-volume/local-storage05-crc") on node "crc" Dec 03 12:15:13 crc kubenswrapper[5052]: I1203 12:15:13.906118 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-api-58464649dd-sx8mw"] Dec 03 12:15:13 crc kubenswrapper[5052]: I1203 12:15:13.917743 5052 scope.go:117] "RemoveContainer" containerID="114f5fc894c9a2d64f9ed86ad645c9cb54b0a4c854a1d0287b2c2a82e3627521" Dec 03 12:15:13 crc kubenswrapper[5052]: I1203 12:15:13.919931 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/barbican-api-58464649dd-sx8mw"] Dec 03 12:15:13 crc kubenswrapper[5052]: I1203 12:15:13.930196 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/rabbitmq-cell1-server-0"] Dec 03 12:15:13 crc kubenswrapper[5052]: I1203 12:15:13.940630 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/rabbitmq-cell1-server-0"] Dec 03 12:15:13 crc kubenswrapper[5052]: I1203 12:15:13.963565 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage05-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage05-crc\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:13 crc kubenswrapper[5052]: I1203 12:15:13.963780 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:13 crc kubenswrapper[5052]: I1203 12:15:13.963792 5052 reconciler_common.go:293] "Volume detached for volume \"pod-info\" (UniqueName: \"kubernetes.io/downward-api/3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f-pod-info\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:13 crc kubenswrapper[5052]: I1203 12:15:13.974089 5052 scope.go:117] "RemoveContainer" containerID="e2d7edf07d19fee0525d0d93f60f9850672844f551c1265e9b21fb9528510d56" Dec 03 12:15:13 crc kubenswrapper[5052]: I1203 12:15:13.994425 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f-rabbitmq-confd" (OuterVolumeSpecName: "rabbitmq-confd") pod "3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f" (UID: "3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f"). InnerVolumeSpecName "rabbitmq-confd". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:15:14 crc kubenswrapper[5052]: I1203 12:15:14.000150 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f-server-conf" (OuterVolumeSpecName: "server-conf") pod "3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f" (UID: "3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f"). InnerVolumeSpecName "server-conf". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:15:14 crc kubenswrapper[5052]: I1203 12:15:14.025520 5052 scope.go:117] "RemoveContainer" containerID="114f5fc894c9a2d64f9ed86ad645c9cb54b0a4c854a1d0287b2c2a82e3627521" Dec 03 12:15:14 crc kubenswrapper[5052]: E1203 12:15:14.025965 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"114f5fc894c9a2d64f9ed86ad645c9cb54b0a4c854a1d0287b2c2a82e3627521\": container with ID starting with 114f5fc894c9a2d64f9ed86ad645c9cb54b0a4c854a1d0287b2c2a82e3627521 not found: ID does not exist" containerID="114f5fc894c9a2d64f9ed86ad645c9cb54b0a4c854a1d0287b2c2a82e3627521" Dec 03 12:15:14 crc kubenswrapper[5052]: I1203 12:15:14.025999 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"114f5fc894c9a2d64f9ed86ad645c9cb54b0a4c854a1d0287b2c2a82e3627521"} err="failed to get container status \"114f5fc894c9a2d64f9ed86ad645c9cb54b0a4c854a1d0287b2c2a82e3627521\": rpc error: code = NotFound desc = could not find container \"114f5fc894c9a2d64f9ed86ad645c9cb54b0a4c854a1d0287b2c2a82e3627521\": container with ID starting with 114f5fc894c9a2d64f9ed86ad645c9cb54b0a4c854a1d0287b2c2a82e3627521 not found: ID does not exist" Dec 03 12:15:14 crc kubenswrapper[5052]: I1203 12:15:14.026027 5052 scope.go:117] "RemoveContainer" containerID="e2d7edf07d19fee0525d0d93f60f9850672844f551c1265e9b21fb9528510d56" Dec 03 12:15:14 crc kubenswrapper[5052]: E1203 12:15:14.026220 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e2d7edf07d19fee0525d0d93f60f9850672844f551c1265e9b21fb9528510d56\": container with ID starting with e2d7edf07d19fee0525d0d93f60f9850672844f551c1265e9b21fb9528510d56 not found: ID does not exist" containerID="e2d7edf07d19fee0525d0d93f60f9850672844f551c1265e9b21fb9528510d56" Dec 03 12:15:14 crc kubenswrapper[5052]: I1203 12:15:14.026253 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e2d7edf07d19fee0525d0d93f60f9850672844f551c1265e9b21fb9528510d56"} err="failed to get container status \"e2d7edf07d19fee0525d0d93f60f9850672844f551c1265e9b21fb9528510d56\": rpc error: code = NotFound desc = could not find container \"e2d7edf07d19fee0525d0d93f60f9850672844f551c1265e9b21fb9528510d56\": container with ID starting with e2d7edf07d19fee0525d0d93f60f9850672844f551c1265e9b21fb9528510d56 not found: ID does not exist" Dec 03 12:15:14 crc kubenswrapper[5052]: I1203 12:15:14.067289 5052 reconciler_common.go:293] "Volume detached for volume \"server-conf\" (UniqueName: \"kubernetes.io/configmap/3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f-server-conf\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:14 crc kubenswrapper[5052]: I1203 12:15:14.067318 5052 reconciler_common.go:293] "Volume detached for volume \"rabbitmq-confd\" (UniqueName: \"kubernetes.io/projected/3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f-rabbitmq-confd\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:14 crc kubenswrapper[5052]: E1203 12:15:14.067327 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:15:14 crc kubenswrapper[5052]: E1203 12:15:14.067361 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:15:14 crc kubenswrapper[5052]: E1203 12:15:14.067391 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/679f17b9-0e40-480b-bfde-4f63899cb972-operator-scripts podName:679f17b9-0e40-480b-bfde-4f63899cb972 nodeName:}" failed. No retries permitted until 2025-12-03 12:15:16.067373813 +0000 UTC m=+3306.655615577 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/679f17b9-0e40-480b-bfde-4f63899cb972-operator-scripts") pod "cinder7878-account-delete-jwtvt" (UID: "679f17b9-0e40-480b-bfde-4f63899cb972") : configmap "openstack-scripts" not found Dec 03 12:15:14 crc kubenswrapper[5052]: E1203 12:15:14.067433 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c200c246-74d3-48dc-8b24-ac4b37611069-operator-scripts podName:c200c246-74d3-48dc-8b24-ac4b37611069 nodeName:}" failed. No retries permitted until 2025-12-03 12:15:16.067422345 +0000 UTC m=+3306.655664109 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/c200c246-74d3-48dc-8b24-ac4b37611069-operator-scripts") pod "novacell00bb5-account-delete-r2csc" (UID: "c200c246-74d3-48dc-8b24-ac4b37611069") : configmap "openstack-scripts" not found Dec 03 12:15:15 crc kubenswrapper[5052]: I1203 12:15:14.541924 5052 generic.go:334] "Generic (PLEG): container finished" podID="cf3ed088-9069-40cf-9c9e-ee9bdf7761d9" containerID="81879a070623f490e700ae5e038359faa2d3ce34a28b8f7e4b856f6c4f95fa7f" exitCode=0 Dec 03 12:15:15 crc kubenswrapper[5052]: I1203 12:15:14.542023 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"cf3ed088-9069-40cf-9c9e-ee9bdf7761d9","Type":"ContainerDied","Data":"81879a070623f490e700ae5e038359faa2d3ce34a28b8f7e4b856f6c4f95fa7f"} Dec 03 12:15:15 crc kubenswrapper[5052]: I1203 12:15:14.548753 5052 generic.go:334] "Generic (PLEG): container finished" podID="1c98e0a9-267b-46b6-be02-fbd163462940" containerID="7c74e4821c6d3bec61337629bd8bf43a58b069eaa5ca6ceb459795bf5d1ec7ae" exitCode=0 Dec 03 12:15:15 crc kubenswrapper[5052]: I1203 12:15:14.548824 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-worker-6ff59c8c65-fsb2k" event={"ID":"1c98e0a9-267b-46b6-be02-fbd163462940","Type":"ContainerDied","Data":"7c74e4821c6d3bec61337629bd8bf43a58b069eaa5ca6ceb459795bf5d1ec7ae"} Dec 03 12:15:15 crc kubenswrapper[5052]: I1203 12:15:14.551947 5052 generic.go:334] "Generic (PLEG): container finished" podID="e14dd2e3-f6e1-450c-a688-1dc4eb2ee622" containerID="6544c0c4bb0b5942eaa383c638931be4bd4915098254d54ab63534949d4fc2c6" exitCode=0 Dec 03 12:15:15 crc kubenswrapper[5052]: I1203 12:15:14.551997 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-6fd4994758-fsw2c" event={"ID":"e14dd2e3-f6e1-450c-a688-1dc4eb2ee622","Type":"ContainerDied","Data":"6544c0c4bb0b5942eaa383c638931be4bd4915098254d54ab63534949d4fc2c6"} Dec 03 12:15:15 crc kubenswrapper[5052]: I1203 12:15:14.555413 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-server-0" event={"ID":"3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f","Type":"ContainerDied","Data":"d41f4f8ea80d58bed016506bfda8d5de94ceed5986cd6ec7e2e81583625c5c68"} Dec 03 12:15:15 crc kubenswrapper[5052]: I1203 12:15:14.555445 5052 scope.go:117] "RemoveContainer" containerID="353f13dfb90cbf0598b28536ad392c1afc49332edb49e65bfaddb985602e41d1" Dec 03 12:15:15 crc kubenswrapper[5052]: I1203 12:15:14.555492 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:15:15 crc kubenswrapper[5052]: I1203 12:15:14.566466 5052 generic.go:334] "Generic (PLEG): container finished" podID="296a2858-22b8-41a4-91ac-a9bf055b2848" containerID="dd65ec28df374ef84fa1a5f87d159edfa4265d92e582b66969d9f54e2878b585" exitCode=0 Dec 03 12:15:15 crc kubenswrapper[5052]: I1203 12:15:14.566560 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-keystone-listener-85f96c8f8-b6m5f" event={"ID":"296a2858-22b8-41a4-91ac-a9bf055b2848","Type":"ContainerDied","Data":"dd65ec28df374ef84fa1a5f87d159edfa4265d92e582b66969d9f54e2878b585"} Dec 03 12:15:15 crc kubenswrapper[5052]: I1203 12:15:14.570178 5052 generic.go:334] "Generic (PLEG): container finished" podID="1661efff-d6ce-479d-bfb9-90849fa6ff40" containerID="aee5b94189aa0133ba610d83708c280ddef3068baf105fae0fbe385e8bb3d748" exitCode=0 Dec 03 12:15:15 crc kubenswrapper[5052]: I1203 12:15:14.570237 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"1661efff-d6ce-479d-bfb9-90849fa6ff40","Type":"ContainerDied","Data":"aee5b94189aa0133ba610d83708c280ddef3068baf105fae0fbe385e8bb3d748"} Dec 03 12:15:15 crc kubenswrapper[5052]: I1203 12:15:14.575699 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_ovn-northd-0_00ab3478-6308-427a-8e34-4133611a90b2/ovn-northd/0.log" Dec 03 12:15:15 crc kubenswrapper[5052]: I1203 12:15:14.575739 5052 generic.go:334] "Generic (PLEG): container finished" podID="00ab3478-6308-427a-8e34-4133611a90b2" containerID="98d14ce0cb9919bb83964fb391e45a00edf1d049296adef59b4e2e0f042c63d2" exitCode=139 Dec 03 12:15:15 crc kubenswrapper[5052]: I1203 12:15:14.575781 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovn-northd-0" event={"ID":"00ab3478-6308-427a-8e34-4133611a90b2","Type":"ContainerDied","Data":"98d14ce0cb9919bb83964fb391e45a00edf1d049296adef59b4e2e0f042c63d2"} Dec 03 12:15:15 crc kubenswrapper[5052]: I1203 12:15:14.577558 5052 generic.go:334] "Generic (PLEG): container finished" podID="14774d58-37e8-4e65-b0b0-d355c0070bca" containerID="33c38af311895b3ea2da41b9880bb47d91d38eadcf28dcf815946f0d92d3f569" exitCode=0 Dec 03 12:15:15 crc kubenswrapper[5052]: I1203 12:15:14.577617 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-conductor-0" event={"ID":"14774d58-37e8-4e65-b0b0-d355c0070bca","Type":"ContainerDied","Data":"33c38af311895b3ea2da41b9880bb47d91d38eadcf28dcf815946f0d92d3f569"} Dec 03 12:15:15 crc kubenswrapper[5052]: I1203 12:15:14.726357 5052 scope.go:117] "RemoveContainer" containerID="a60401c5cce1c04ba232d9ed1392bdc059663fb8d3f36429c2b1a642a6fc8fdf" Dec 03 12:15:15 crc kubenswrapper[5052]: I1203 12:15:14.731366 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-worker-6ff59c8c65-fsb2k" Dec 03 12:15:15 crc kubenswrapper[5052]: I1203 12:15:14.780916 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/rabbitmq-server-0"] Dec 03 12:15:15 crc kubenswrapper[5052]: I1203 12:15:14.788701 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/rabbitmq-server-0"] Dec 03 12:15:15 crc kubenswrapper[5052]: I1203 12:15:14.881534 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/1c98e0a9-267b-46b6-be02-fbd163462940-logs\") pod \"1c98e0a9-267b-46b6-be02-fbd163462940\" (UID: \"1c98e0a9-267b-46b6-be02-fbd163462940\") " Dec 03 12:15:15 crc kubenswrapper[5052]: I1203 12:15:14.881601 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/1c98e0a9-267b-46b6-be02-fbd163462940-config-data\") pod \"1c98e0a9-267b-46b6-be02-fbd163462940\" (UID: \"1c98e0a9-267b-46b6-be02-fbd163462940\") " Dec 03 12:15:15 crc kubenswrapper[5052]: I1203 12:15:14.881648 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fddgs\" (UniqueName: \"kubernetes.io/projected/1c98e0a9-267b-46b6-be02-fbd163462940-kube-api-access-fddgs\") pod \"1c98e0a9-267b-46b6-be02-fbd163462940\" (UID: \"1c98e0a9-267b-46b6-be02-fbd163462940\") " Dec 03 12:15:15 crc kubenswrapper[5052]: I1203 12:15:14.881672 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/1c98e0a9-267b-46b6-be02-fbd163462940-config-data-custom\") pod \"1c98e0a9-267b-46b6-be02-fbd163462940\" (UID: \"1c98e0a9-267b-46b6-be02-fbd163462940\") " Dec 03 12:15:15 crc kubenswrapper[5052]: I1203 12:15:14.881842 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1c98e0a9-267b-46b6-be02-fbd163462940-combined-ca-bundle\") pod \"1c98e0a9-267b-46b6-be02-fbd163462940\" (UID: \"1c98e0a9-267b-46b6-be02-fbd163462940\") " Dec 03 12:15:15 crc kubenswrapper[5052]: I1203 12:15:14.882205 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1c98e0a9-267b-46b6-be02-fbd163462940-logs" (OuterVolumeSpecName: "logs") pod "1c98e0a9-267b-46b6-be02-fbd163462940" (UID: "1c98e0a9-267b-46b6-be02-fbd163462940"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:15:15 crc kubenswrapper[5052]: I1203 12:15:14.886297 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1c98e0a9-267b-46b6-be02-fbd163462940-kube-api-access-fddgs" (OuterVolumeSpecName: "kube-api-access-fddgs") pod "1c98e0a9-267b-46b6-be02-fbd163462940" (UID: "1c98e0a9-267b-46b6-be02-fbd163462940"). InnerVolumeSpecName "kube-api-access-fddgs". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:15:15 crc kubenswrapper[5052]: I1203 12:15:14.887587 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1c98e0a9-267b-46b6-be02-fbd163462940-config-data-custom" (OuterVolumeSpecName: "config-data-custom") pod "1c98e0a9-267b-46b6-be02-fbd163462940" (UID: "1c98e0a9-267b-46b6-be02-fbd163462940"). InnerVolumeSpecName "config-data-custom". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:15:15 crc kubenswrapper[5052]: I1203 12:15:14.909658 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1c98e0a9-267b-46b6-be02-fbd163462940-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "1c98e0a9-267b-46b6-be02-fbd163462940" (UID: "1c98e0a9-267b-46b6-be02-fbd163462940"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:15:15 crc kubenswrapper[5052]: I1203 12:15:14.930858 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1c98e0a9-267b-46b6-be02-fbd163462940-config-data" (OuterVolumeSpecName: "config-data") pod "1c98e0a9-267b-46b6-be02-fbd163462940" (UID: "1c98e0a9-267b-46b6-be02-fbd163462940"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:15:15 crc kubenswrapper[5052]: I1203 12:15:14.983257 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1c98e0a9-267b-46b6-be02-fbd163462940-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:15 crc kubenswrapper[5052]: I1203 12:15:14.983285 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/1c98e0a9-267b-46b6-be02-fbd163462940-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:15 crc kubenswrapper[5052]: I1203 12:15:14.983295 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/1c98e0a9-267b-46b6-be02-fbd163462940-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:15 crc kubenswrapper[5052]: I1203 12:15:14.983309 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-fddgs\" (UniqueName: \"kubernetes.io/projected/1c98e0a9-267b-46b6-be02-fbd163462940-kube-api-access-fddgs\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:15 crc kubenswrapper[5052]: I1203 12:15:14.983325 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/1c98e0a9-267b-46b6-be02-fbd163462940-config-data-custom\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:15 crc kubenswrapper[5052]: I1203 12:15:15.166498 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/placement-db-create-fwcd6"] Dec 03 12:15:15 crc kubenswrapper[5052]: I1203 12:15:15.176190 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/placement-db-create-fwcd6"] Dec 03 12:15:15 crc kubenswrapper[5052]: I1203 12:15:15.188478 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/placement-7cee-account-create-update-rcvs6"] Dec 03 12:15:15 crc kubenswrapper[5052]: I1203 12:15:15.195085 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/placement7cee-account-delete-7wgkx"] Dec 03 12:15:15 crc kubenswrapper[5052]: I1203 12:15:15.202106 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/placement-7cee-account-create-update-rcvs6"] Dec 03 12:15:15 crc kubenswrapper[5052]: I1203 12:15:15.207828 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/placement7cee-account-delete-7wgkx"] Dec 03 12:15:15 crc kubenswrapper[5052]: I1203 12:15:15.477077 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/neutron-db-create-2s5ld"] Dec 03 12:15:15 crc kubenswrapper[5052]: I1203 12:15:15.527551 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/neutron-db-create-2s5ld"] Dec 03 12:15:15 crc kubenswrapper[5052]: I1203 12:15:15.545036 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/neutron-1287-account-create-update-56qp8"] Dec 03 12:15:15 crc kubenswrapper[5052]: I1203 12:15:15.554668 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/neutron1287-account-delete-t8dcs"] Dec 03 12:15:15 crc kubenswrapper[5052]: I1203 12:15:15.563844 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/neutron1287-account-delete-t8dcs"] Dec 03 12:15:15 crc kubenswrapper[5052]: I1203 12:15:15.572321 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/neutron-1287-account-create-update-56qp8"] Dec 03 12:15:15 crc kubenswrapper[5052]: I1203 12:15:15.587067 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-db-create-bn2nj"] Dec 03 12:15:15 crc kubenswrapper[5052]: I1203 12:15:15.596243 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/barbican-db-create-bn2nj"] Dec 03 12:15:15 crc kubenswrapper[5052]: I1203 12:15:15.607252 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-8992-account-create-update-gbt4t"] Dec 03 12:15:15 crc kubenswrapper[5052]: I1203 12:15:15.610804 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-worker-6ff59c8c65-fsb2k" event={"ID":"1c98e0a9-267b-46b6-be02-fbd163462940","Type":"ContainerDied","Data":"7fedb3be91342f96626a4b3d67a9a2fc8957182a74f1741a767f99ef8a035320"} Dec 03 12:15:15 crc kubenswrapper[5052]: I1203 12:15:15.610875 5052 scope.go:117] "RemoveContainer" containerID="7c74e4821c6d3bec61337629bd8bf43a58b069eaa5ca6ceb459795bf5d1ec7ae" Dec 03 12:15:15 crc kubenswrapper[5052]: I1203 12:15:15.611014 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-worker-6ff59c8c65-fsb2k" Dec 03 12:15:15 crc kubenswrapper[5052]: I1203 12:15:15.616165 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/barbican-8992-account-create-update-gbt4t"] Dec 03 12:15:15 crc kubenswrapper[5052]: I1203 12:15:15.626873 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican8992-account-delete-26lhs"] Dec 03 12:15:15 crc kubenswrapper[5052]: I1203 12:15:15.638966 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/barbican8992-account-delete-26lhs"] Dec 03 12:15:15 crc kubenswrapper[5052]: I1203 12:15:15.643953 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-db-create-jr6mb"] Dec 03 12:15:15 crc kubenswrapper[5052]: I1203 12:15:15.649771 5052 scope.go:117] "RemoveContainer" containerID="ba1ddda28f438ca657b63e17167786d2db03d16bf5ae3221146d8c6f9efc02d5" Dec 03 12:15:15 crc kubenswrapper[5052]: I1203 12:15:15.655101 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/glance-db-create-jr6mb"] Dec 03 12:15:15 crc kubenswrapper[5052]: I1203 12:15:15.663211 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glancef253-account-delete-cxxlg"] Dec 03 12:15:15 crc kubenswrapper[5052]: I1203 12:15:15.677048 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-f253-account-create-update-zl642"] Dec 03 12:15:15 crc kubenswrapper[5052]: I1203 12:15:15.677095 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/glancef253-account-delete-cxxlg"] Dec 03 12:15:15 crc kubenswrapper[5052]: I1203 12:15:15.708095 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f" path="/var/lib/kubelet/pods/3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f/volumes" Dec 03 12:15:15 crc kubenswrapper[5052]: I1203 12:15:15.708710 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5093f3f0-cee5-4e84-b99c-6652546075ce" path="/var/lib/kubelet/pods/5093f3f0-cee5-4e84-b99c-6652546075ce/volumes" Dec 03 12:15:15 crc kubenswrapper[5052]: I1203 12:15:15.709179 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5856107a-0498-4956-beba-4390f5477ca0" path="/var/lib/kubelet/pods/5856107a-0498-4956-beba-4390f5477ca0/volumes" Dec 03 12:15:15 crc kubenswrapper[5052]: I1203 12:15:15.711078 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6099fea1-bbb4-4413-a826-539b3dfc3eec" path="/var/lib/kubelet/pods/6099fea1-bbb4-4413-a826-539b3dfc3eec/volumes" Dec 03 12:15:15 crc kubenswrapper[5052]: I1203 12:15:15.712578 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7e802e69-975f-4748-a902-d3715174eefa" path="/var/lib/kubelet/pods/7e802e69-975f-4748-a902-d3715174eefa/volumes" Dec 03 12:15:15 crc kubenswrapper[5052]: I1203 12:15:15.713978 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="81fcccc3-61d8-48d2-9422-92cfbb850d40" path="/var/lib/kubelet/pods/81fcccc3-61d8-48d2-9422-92cfbb850d40/volumes" Dec 03 12:15:15 crc kubenswrapper[5052]: I1203 12:15:15.716329 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="884268a1-ff3a-4aaf-8004-89bb72e79d90" path="/var/lib/kubelet/pods/884268a1-ff3a-4aaf-8004-89bb72e79d90/volumes" Dec 03 12:15:15 crc kubenswrapper[5052]: I1203 12:15:15.717588 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="89b056a7-e448-447d-be7d-b69f3d4862a1" path="/var/lib/kubelet/pods/89b056a7-e448-447d-be7d-b69f3d4862a1/volumes" Dec 03 12:15:15 crc kubenswrapper[5052]: I1203 12:15:15.718639 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="955cd7c4-bd55-4a7e-80e7-75588d3e325a" path="/var/lib/kubelet/pods/955cd7c4-bd55-4a7e-80e7-75588d3e325a/volumes" Dec 03 12:15:15 crc kubenswrapper[5052]: I1203 12:15:15.719580 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b25d2a0c-1e93-4b3d-965a-a7c021f9649d" path="/var/lib/kubelet/pods/b25d2a0c-1e93-4b3d-965a-a7c021f9649d/volumes" Dec 03 12:15:15 crc kubenswrapper[5052]: I1203 12:15:15.720125 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b28a125a-7966-4f97-91c1-fa52d6062405" path="/var/lib/kubelet/pods/b28a125a-7966-4f97-91c1-fa52d6062405/volumes" Dec 03 12:15:15 crc kubenswrapper[5052]: I1203 12:15:15.720735 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c2a65caa-2eec-48af-bdad-8fc19d827867" path="/var/lib/kubelet/pods/c2a65caa-2eec-48af-bdad-8fc19d827867/volumes" Dec 03 12:15:15 crc kubenswrapper[5052]: I1203 12:15:15.721651 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d9613cb1-0439-44fb-a577-c621439a6328" path="/var/lib/kubelet/pods/d9613cb1-0439-44fb-a577-c621439a6328/volumes" Dec 03 12:15:15 crc kubenswrapper[5052]: E1203 12:15:15.722232 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of 98d14ce0cb9919bb83964fb391e45a00edf1d049296adef59b4e2e0f042c63d2 is running failed: container process not found" containerID="98d14ce0cb9919bb83964fb391e45a00edf1d049296adef59b4e2e0f042c63d2" cmd=["/usr/local/bin/container-scripts/status_check.sh"] Dec 03 12:15:15 crc kubenswrapper[5052]: I1203 12:15:15.722764 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e63eb1eb-e0b9-46ea-a19b-8f5c018728ff" path="/var/lib/kubelet/pods/e63eb1eb-e0b9-46ea-a19b-8f5c018728ff/volumes" Dec 03 12:15:15 crc kubenswrapper[5052]: E1203 12:15:15.722839 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of 98d14ce0cb9919bb83964fb391e45a00edf1d049296adef59b4e2e0f042c63d2 is running failed: container process not found" containerID="98d14ce0cb9919bb83964fb391e45a00edf1d049296adef59b4e2e0f042c63d2" cmd=["/usr/local/bin/container-scripts/status_check.sh"] Dec 03 12:15:15 crc kubenswrapper[5052]: E1203 12:15:15.723237 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of 98d14ce0cb9919bb83964fb391e45a00edf1d049296adef59b4e2e0f042c63d2 is running failed: container process not found" containerID="98d14ce0cb9919bb83964fb391e45a00edf1d049296adef59b4e2e0f042c63d2" cmd=["/usr/local/bin/container-scripts/status_check.sh"] Dec 03 12:15:15 crc kubenswrapper[5052]: E1203 12:15:15.723275 5052 prober.go:104] "Probe errored" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of 98d14ce0cb9919bb83964fb391e45a00edf1d049296adef59b4e2e0f042c63d2 is running failed: container process not found" probeType="Readiness" pod="openstack-kuttl-tests/ovn-northd-0" podUID="00ab3478-6308-427a-8e34-4133611a90b2" containerName="ovn-northd" Dec 03 12:15:15 crc kubenswrapper[5052]: I1203 12:15:15.724092 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f1619fbf-df68-484f-99d4-09b81a7a334e" path="/var/lib/kubelet/pods/f1619fbf-df68-484f-99d4-09b81a7a334e/volumes" Dec 03 12:15:15 crc kubenswrapper[5052]: I1203 12:15:15.725484 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fb88d1bf-1334-41ec-8605-e93a852623b3" path="/var/lib/kubelet/pods/fb88d1bf-1334-41ec-8605-e93a852623b3/volumes" Dec 03 12:15:15 crc kubenswrapper[5052]: I1203 12:15:15.726163 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fee7d732-4fa3-40d5-8fff-cb88272db0ce" path="/var/lib/kubelet/pods/fee7d732-4fa3-40d5-8fff-cb88272db0ce/volumes" Dec 03 12:15:15 crc kubenswrapper[5052]: I1203 12:15:15.738184 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/glance-f253-account-create-update-zl642"] Dec 03 12:15:15 crc kubenswrapper[5052]: I1203 12:15:15.738236 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-worker-6ff59c8c65-fsb2k"] Dec 03 12:15:15 crc kubenswrapper[5052]: I1203 12:15:15.738254 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/barbican-worker-6ff59c8c65-fsb2k"] Dec 03 12:15:15 crc kubenswrapper[5052]: I1203 12:15:15.765215 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder-db-create-lqkc5"] Dec 03 12:15:15 crc kubenswrapper[5052]: I1203 12:15:15.771335 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/cinder-db-create-lqkc5"] Dec 03 12:15:15 crc kubenswrapper[5052]: I1203 12:15:15.792400 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-6fd4994758-fsw2c" Dec 03 12:15:15 crc kubenswrapper[5052]: I1203 12:15:15.796811 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder-7878-account-create-update-cxttb"] Dec 03 12:15:15 crc kubenswrapper[5052]: I1203 12:15:15.799898 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:15:15 crc kubenswrapper[5052]: I1203 12:15:15.845237 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/cinder-7878-account-create-update-cxttb"] Dec 03 12:15:15 crc kubenswrapper[5052]: I1203 12:15:15.859737 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder7878-account-delete-jwtvt"] Dec 03 12:15:15 crc kubenswrapper[5052]: I1203 12:15:15.860093 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/cinder7878-account-delete-jwtvt" podUID="679f17b9-0e40-480b-bfde-4f63899cb972" containerName="mariadb-account-delete" containerID="cri-o://e9f158233d3479671db2b774df7b5ea49c9fbfe33de68208a166ee13a03145b4" gracePeriod=30 Dec 03 12:15:15 crc kubenswrapper[5052]: I1203 12:15:15.931773 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-keystone-listener-85f96c8f8-b6m5f" Dec 03 12:15:15 crc kubenswrapper[5052]: I1203 12:15:15.982640 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/e14dd2e3-f6e1-450c-a688-1dc4eb2ee622-internal-tls-certs\") pod \"e14dd2e3-f6e1-450c-a688-1dc4eb2ee622\" (UID: \"e14dd2e3-f6e1-450c-a688-1dc4eb2ee622\") " Dec 03 12:15:15 crc kubenswrapper[5052]: I1203 12:15:15.983036 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/e14dd2e3-f6e1-450c-a688-1dc4eb2ee622-config-data\") pod \"e14dd2e3-f6e1-450c-a688-1dc4eb2ee622\" (UID: \"e14dd2e3-f6e1-450c-a688-1dc4eb2ee622\") " Dec 03 12:15:15 crc kubenswrapper[5052]: I1203 12:15:15.983091 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/14774d58-37e8-4e65-b0b0-d355c0070bca-combined-ca-bundle\") pod \"14774d58-37e8-4e65-b0b0-d355c0070bca\" (UID: \"14774d58-37e8-4e65-b0b0-d355c0070bca\") " Dec 03 12:15:15 crc kubenswrapper[5052]: I1203 12:15:15.983156 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/14774d58-37e8-4e65-b0b0-d355c0070bca-config-data\") pod \"14774d58-37e8-4e65-b0b0-d355c0070bca\" (UID: \"14774d58-37e8-4e65-b0b0-d355c0070bca\") " Dec 03 12:15:15 crc kubenswrapper[5052]: I1203 12:15:15.983178 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/e14dd2e3-f6e1-450c-a688-1dc4eb2ee622-fernet-keys\") pod \"e14dd2e3-f6e1-450c-a688-1dc4eb2ee622\" (UID: \"e14dd2e3-f6e1-450c-a688-1dc4eb2ee622\") " Dec 03 12:15:15 crc kubenswrapper[5052]: I1203 12:15:15.983211 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bv6k4\" (UniqueName: \"kubernetes.io/projected/14774d58-37e8-4e65-b0b0-d355c0070bca-kube-api-access-bv6k4\") pod \"14774d58-37e8-4e65-b0b0-d355c0070bca\" (UID: \"14774d58-37e8-4e65-b0b0-d355c0070bca\") " Dec 03 12:15:15 crc kubenswrapper[5052]: I1203 12:15:15.983234 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/e14dd2e3-f6e1-450c-a688-1dc4eb2ee622-public-tls-certs\") pod \"e14dd2e3-f6e1-450c-a688-1dc4eb2ee622\" (UID: \"e14dd2e3-f6e1-450c-a688-1dc4eb2ee622\") " Dec 03 12:15:15 crc kubenswrapper[5052]: I1203 12:15:15.983265 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e14dd2e3-f6e1-450c-a688-1dc4eb2ee622-combined-ca-bundle\") pod \"e14dd2e3-f6e1-450c-a688-1dc4eb2ee622\" (UID: \"e14dd2e3-f6e1-450c-a688-1dc4eb2ee622\") " Dec 03 12:15:15 crc kubenswrapper[5052]: I1203 12:15:15.983288 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/e14dd2e3-f6e1-450c-a688-1dc4eb2ee622-scripts\") pod \"e14dd2e3-f6e1-450c-a688-1dc4eb2ee622\" (UID: \"e14dd2e3-f6e1-450c-a688-1dc4eb2ee622\") " Dec 03 12:15:15 crc kubenswrapper[5052]: I1203 12:15:15.983377 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gnjbz\" (UniqueName: \"kubernetes.io/projected/e14dd2e3-f6e1-450c-a688-1dc4eb2ee622-kube-api-access-gnjbz\") pod \"e14dd2e3-f6e1-450c-a688-1dc4eb2ee622\" (UID: \"e14dd2e3-f6e1-450c-a688-1dc4eb2ee622\") " Dec 03 12:15:15 crc kubenswrapper[5052]: I1203 12:15:15.983468 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/e14dd2e3-f6e1-450c-a688-1dc4eb2ee622-credential-keys\") pod \"e14dd2e3-f6e1-450c-a688-1dc4eb2ee622\" (UID: \"e14dd2e3-f6e1-450c-a688-1dc4eb2ee622\") " Dec 03 12:15:15 crc kubenswrapper[5052]: I1203 12:15:15.988797 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e14dd2e3-f6e1-450c-a688-1dc4eb2ee622-fernet-keys" (OuterVolumeSpecName: "fernet-keys") pod "e14dd2e3-f6e1-450c-a688-1dc4eb2ee622" (UID: "e14dd2e3-f6e1-450c-a688-1dc4eb2ee622"). InnerVolumeSpecName "fernet-keys". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:15:15 crc kubenswrapper[5052]: I1203 12:15:15.992347 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e14dd2e3-f6e1-450c-a688-1dc4eb2ee622-credential-keys" (OuterVolumeSpecName: "credential-keys") pod "e14dd2e3-f6e1-450c-a688-1dc4eb2ee622" (UID: "e14dd2e3-f6e1-450c-a688-1dc4eb2ee622"). InnerVolumeSpecName "credential-keys". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:15:15 crc kubenswrapper[5052]: I1203 12:15:15.996576 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e14dd2e3-f6e1-450c-a688-1dc4eb2ee622-scripts" (OuterVolumeSpecName: "scripts") pod "e14dd2e3-f6e1-450c-a688-1dc4eb2ee622" (UID: "e14dd2e3-f6e1-450c-a688-1dc4eb2ee622"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:15:16 crc kubenswrapper[5052]: I1203 12:15:16.008623 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/14774d58-37e8-4e65-b0b0-d355c0070bca-kube-api-access-bv6k4" (OuterVolumeSpecName: "kube-api-access-bv6k4") pod "14774d58-37e8-4e65-b0b0-d355c0070bca" (UID: "14774d58-37e8-4e65-b0b0-d355c0070bca"). InnerVolumeSpecName "kube-api-access-bv6k4". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:15:16 crc kubenswrapper[5052]: I1203 12:15:16.015934 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e14dd2e3-f6e1-450c-a688-1dc4eb2ee622-kube-api-access-gnjbz" (OuterVolumeSpecName: "kube-api-access-gnjbz") pod "e14dd2e3-f6e1-450c-a688-1dc4eb2ee622" (UID: "e14dd2e3-f6e1-450c-a688-1dc4eb2ee622"). InnerVolumeSpecName "kube-api-access-gnjbz". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:15:16 crc kubenswrapper[5052]: I1203 12:15:16.024030 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e14dd2e3-f6e1-450c-a688-1dc4eb2ee622-config-data" (OuterVolumeSpecName: "config-data") pod "e14dd2e3-f6e1-450c-a688-1dc4eb2ee622" (UID: "e14dd2e3-f6e1-450c-a688-1dc4eb2ee622"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:15:16 crc kubenswrapper[5052]: I1203 12:15:16.036336 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/14774d58-37e8-4e65-b0b0-d355c0070bca-config-data" (OuterVolumeSpecName: "config-data") pod "14774d58-37e8-4e65-b0b0-d355c0070bca" (UID: "14774d58-37e8-4e65-b0b0-d355c0070bca"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:15:16 crc kubenswrapper[5052]: I1203 12:15:16.036754 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e14dd2e3-f6e1-450c-a688-1dc4eb2ee622-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "e14dd2e3-f6e1-450c-a688-1dc4eb2ee622" (UID: "e14dd2e3-f6e1-450c-a688-1dc4eb2ee622"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:15:16 crc kubenswrapper[5052]: I1203 12:15:16.060778 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/14774d58-37e8-4e65-b0b0-d355c0070bca-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "14774d58-37e8-4e65-b0b0-d355c0070bca" (UID: "14774d58-37e8-4e65-b0b0-d355c0070bca"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:15:16 crc kubenswrapper[5052]: I1203 12:15:16.071230 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e14dd2e3-f6e1-450c-a688-1dc4eb2ee622-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "e14dd2e3-f6e1-450c-a688-1dc4eb2ee622" (UID: "e14dd2e3-f6e1-450c-a688-1dc4eb2ee622"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:15:16 crc kubenswrapper[5052]: I1203 12:15:16.073829 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e14dd2e3-f6e1-450c-a688-1dc4eb2ee622-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "e14dd2e3-f6e1-450c-a688-1dc4eb2ee622" (UID: "e14dd2e3-f6e1-450c-a688-1dc4eb2ee622"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:15:16 crc kubenswrapper[5052]: I1203 12:15:16.084617 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/296a2858-22b8-41a4-91ac-a9bf055b2848-logs\") pod \"296a2858-22b8-41a4-91ac-a9bf055b2848\" (UID: \"296a2858-22b8-41a4-91ac-a9bf055b2848\") " Dec 03 12:15:16 crc kubenswrapper[5052]: I1203 12:15:16.084708 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/296a2858-22b8-41a4-91ac-a9bf055b2848-combined-ca-bundle\") pod \"296a2858-22b8-41a4-91ac-a9bf055b2848\" (UID: \"296a2858-22b8-41a4-91ac-a9bf055b2848\") " Dec 03 12:15:16 crc kubenswrapper[5052]: I1203 12:15:16.084756 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/296a2858-22b8-41a4-91ac-a9bf055b2848-config-data\") pod \"296a2858-22b8-41a4-91ac-a9bf055b2848\" (UID: \"296a2858-22b8-41a4-91ac-a9bf055b2848\") " Dec 03 12:15:16 crc kubenswrapper[5052]: I1203 12:15:16.084784 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dhznr\" (UniqueName: \"kubernetes.io/projected/296a2858-22b8-41a4-91ac-a9bf055b2848-kube-api-access-dhznr\") pod \"296a2858-22b8-41a4-91ac-a9bf055b2848\" (UID: \"296a2858-22b8-41a4-91ac-a9bf055b2848\") " Dec 03 12:15:16 crc kubenswrapper[5052]: I1203 12:15:16.084834 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/296a2858-22b8-41a4-91ac-a9bf055b2848-config-data-custom\") pod \"296a2858-22b8-41a4-91ac-a9bf055b2848\" (UID: \"296a2858-22b8-41a4-91ac-a9bf055b2848\") " Dec 03 12:15:16 crc kubenswrapper[5052]: I1203 12:15:16.085185 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/e14dd2e3-f6e1-450c-a688-1dc4eb2ee622-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:16 crc kubenswrapper[5052]: I1203 12:15:16.085203 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/14774d58-37e8-4e65-b0b0-d355c0070bca-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:16 crc kubenswrapper[5052]: I1203 12:15:16.085214 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/14774d58-37e8-4e65-b0b0-d355c0070bca-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:16 crc kubenswrapper[5052]: I1203 12:15:16.085222 5052 reconciler_common.go:293] "Volume detached for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/e14dd2e3-f6e1-450c-a688-1dc4eb2ee622-fernet-keys\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:16 crc kubenswrapper[5052]: I1203 12:15:16.085233 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-bv6k4\" (UniqueName: \"kubernetes.io/projected/14774d58-37e8-4e65-b0b0-d355c0070bca-kube-api-access-bv6k4\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:16 crc kubenswrapper[5052]: I1203 12:15:16.085243 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/e14dd2e3-f6e1-450c-a688-1dc4eb2ee622-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:16 crc kubenswrapper[5052]: I1203 12:15:16.085251 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e14dd2e3-f6e1-450c-a688-1dc4eb2ee622-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:16 crc kubenswrapper[5052]: I1203 12:15:16.085260 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/e14dd2e3-f6e1-450c-a688-1dc4eb2ee622-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:16 crc kubenswrapper[5052]: I1203 12:15:16.085268 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-gnjbz\" (UniqueName: \"kubernetes.io/projected/e14dd2e3-f6e1-450c-a688-1dc4eb2ee622-kube-api-access-gnjbz\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:16 crc kubenswrapper[5052]: I1203 12:15:16.085278 5052 reconciler_common.go:293] "Volume detached for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/e14dd2e3-f6e1-450c-a688-1dc4eb2ee622-credential-keys\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:16 crc kubenswrapper[5052]: I1203 12:15:16.085286 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/e14dd2e3-f6e1-450c-a688-1dc4eb2ee622-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:16 crc kubenswrapper[5052]: E1203 12:15:16.085330 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:15:16 crc kubenswrapper[5052]: E1203 12:15:16.085353 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:15:16 crc kubenswrapper[5052]: E1203 12:15:16.085407 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c200c246-74d3-48dc-8b24-ac4b37611069-operator-scripts podName:c200c246-74d3-48dc-8b24-ac4b37611069 nodeName:}" failed. No retries permitted until 2025-12-03 12:15:20.085385354 +0000 UTC m=+3310.673627168 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/c200c246-74d3-48dc-8b24-ac4b37611069-operator-scripts") pod "novacell00bb5-account-delete-r2csc" (UID: "c200c246-74d3-48dc-8b24-ac4b37611069") : configmap "openstack-scripts" not found Dec 03 12:15:16 crc kubenswrapper[5052]: E1203 12:15:16.085430 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/679f17b9-0e40-480b-bfde-4f63899cb972-operator-scripts podName:679f17b9-0e40-480b-bfde-4f63899cb972 nodeName:}" failed. No retries permitted until 2025-12-03 12:15:20.085421175 +0000 UTC m=+3310.673663019 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/679f17b9-0e40-480b-bfde-4f63899cb972-operator-scripts") pod "cinder7878-account-delete-jwtvt" (UID: "679f17b9-0e40-480b-bfde-4f63899cb972") : configmap "openstack-scripts" not found Dec 03 12:15:16 crc kubenswrapper[5052]: I1203 12:15:16.085878 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/296a2858-22b8-41a4-91ac-a9bf055b2848-logs" (OuterVolumeSpecName: "logs") pod "296a2858-22b8-41a4-91ac-a9bf055b2848" (UID: "296a2858-22b8-41a4-91ac-a9bf055b2848"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:15:16 crc kubenswrapper[5052]: I1203 12:15:16.088852 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:15:16 crc kubenswrapper[5052]: I1203 12:15:16.090471 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/296a2858-22b8-41a4-91ac-a9bf055b2848-config-data-custom" (OuterVolumeSpecName: "config-data-custom") pod "296a2858-22b8-41a4-91ac-a9bf055b2848" (UID: "296a2858-22b8-41a4-91ac-a9bf055b2848"). InnerVolumeSpecName "config-data-custom". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:15:16 crc kubenswrapper[5052]: I1203 12:15:16.090707 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/296a2858-22b8-41a4-91ac-a9bf055b2848-kube-api-access-dhznr" (OuterVolumeSpecName: "kube-api-access-dhznr") pod "296a2858-22b8-41a4-91ac-a9bf055b2848" (UID: "296a2858-22b8-41a4-91ac-a9bf055b2848"). InnerVolumeSpecName "kube-api-access-dhznr". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:15:16 crc kubenswrapper[5052]: I1203 12:15:16.096538 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:15:16 crc kubenswrapper[5052]: I1203 12:15:16.107471 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_ovn-northd-0_00ab3478-6308-427a-8e34-4133611a90b2/ovn-northd/0.log" Dec 03 12:15:16 crc kubenswrapper[5052]: I1203 12:15:16.107762 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:15:16 crc kubenswrapper[5052]: I1203 12:15:16.114602 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/296a2858-22b8-41a4-91ac-a9bf055b2848-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "296a2858-22b8-41a4-91ac-a9bf055b2848" (UID: "296a2858-22b8-41a4-91ac-a9bf055b2848"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:15:16 crc kubenswrapper[5052]: I1203 12:15:16.156884 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/296a2858-22b8-41a4-91ac-a9bf055b2848-config-data" (OuterVolumeSpecName: "config-data") pod "296a2858-22b8-41a4-91ac-a9bf055b2848" (UID: "296a2858-22b8-41a4-91ac-a9bf055b2848"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:15:16 crc kubenswrapper[5052]: I1203 12:15:16.185893 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9m45m\" (UniqueName: \"kubernetes.io/projected/cf3ed088-9069-40cf-9c9e-ee9bdf7761d9-kube-api-access-9m45m\") pod \"cf3ed088-9069-40cf-9c9e-ee9bdf7761d9\" (UID: \"cf3ed088-9069-40cf-9c9e-ee9bdf7761d9\") " Dec 03 12:15:16 crc kubenswrapper[5052]: I1203 12:15:16.185994 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/cf3ed088-9069-40cf-9c9e-ee9bdf7761d9-config-data\") pod \"cf3ed088-9069-40cf-9c9e-ee9bdf7761d9\" (UID: \"cf3ed088-9069-40cf-9c9e-ee9bdf7761d9\") " Dec 03 12:15:16 crc kubenswrapper[5052]: I1203 12:15:16.186056 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/cf3ed088-9069-40cf-9c9e-ee9bdf7761d9-combined-ca-bundle\") pod \"cf3ed088-9069-40cf-9c9e-ee9bdf7761d9\" (UID: \"cf3ed088-9069-40cf-9c9e-ee9bdf7761d9\") " Dec 03 12:15:16 crc kubenswrapper[5052]: I1203 12:15:16.186417 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/296a2858-22b8-41a4-91ac-a9bf055b2848-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:16 crc kubenswrapper[5052]: I1203 12:15:16.186442 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-dhznr\" (UniqueName: \"kubernetes.io/projected/296a2858-22b8-41a4-91ac-a9bf055b2848-kube-api-access-dhznr\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:16 crc kubenswrapper[5052]: I1203 12:15:16.186455 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/296a2858-22b8-41a4-91ac-a9bf055b2848-config-data-custom\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:16 crc kubenswrapper[5052]: I1203 12:15:16.186467 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/296a2858-22b8-41a4-91ac-a9bf055b2848-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:16 crc kubenswrapper[5052]: I1203 12:15:16.186478 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/296a2858-22b8-41a4-91ac-a9bf055b2848-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:16 crc kubenswrapper[5052]: I1203 12:15:16.189149 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cf3ed088-9069-40cf-9c9e-ee9bdf7761d9-kube-api-access-9m45m" (OuterVolumeSpecName: "kube-api-access-9m45m") pod "cf3ed088-9069-40cf-9c9e-ee9bdf7761d9" (UID: "cf3ed088-9069-40cf-9c9e-ee9bdf7761d9"). InnerVolumeSpecName "kube-api-access-9m45m". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:15:16 crc kubenswrapper[5052]: I1203 12:15:16.205961 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cf3ed088-9069-40cf-9c9e-ee9bdf7761d9-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "cf3ed088-9069-40cf-9c9e-ee9bdf7761d9" (UID: "cf3ed088-9069-40cf-9c9e-ee9bdf7761d9"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:15:16 crc kubenswrapper[5052]: I1203 12:15:16.210841 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cf3ed088-9069-40cf-9c9e-ee9bdf7761d9-config-data" (OuterVolumeSpecName: "config-data") pod "cf3ed088-9069-40cf-9c9e-ee9bdf7761d9" (UID: "cf3ed088-9069-40cf-9c9e-ee9bdf7761d9"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:15:16 crc kubenswrapper[5052]: I1203 12:15:16.287861 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovn-rundir\" (UniqueName: \"kubernetes.io/empty-dir/00ab3478-6308-427a-8e34-4133611a90b2-ovn-rundir\") pod \"00ab3478-6308-427a-8e34-4133611a90b2\" (UID: \"00ab3478-6308-427a-8e34-4133611a90b2\") " Dec 03 12:15:16 crc kubenswrapper[5052]: I1203 12:15:16.287913 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/00ab3478-6308-427a-8e34-4133611a90b2-combined-ca-bundle\") pod \"00ab3478-6308-427a-8e34-4133611a90b2\" (UID: \"00ab3478-6308-427a-8e34-4133611a90b2\") " Dec 03 12:15:16 crc kubenswrapper[5052]: I1203 12:15:16.287958 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/1661efff-d6ce-479d-bfb9-90849fa6ff40-run-httpd\") pod \"1661efff-d6ce-479d-bfb9-90849fa6ff40\" (UID: \"1661efff-d6ce-479d-bfb9-90849fa6ff40\") " Dec 03 12:15:16 crc kubenswrapper[5052]: I1203 12:15:16.288005 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-24whk\" (UniqueName: \"kubernetes.io/projected/00ab3478-6308-427a-8e34-4133611a90b2-kube-api-access-24whk\") pod \"00ab3478-6308-427a-8e34-4133611a90b2\" (UID: \"00ab3478-6308-427a-8e34-4133611a90b2\") " Dec 03 12:15:16 crc kubenswrapper[5052]: I1203 12:15:16.288040 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/1661efff-d6ce-479d-bfb9-90849fa6ff40-log-httpd\") pod \"1661efff-d6ce-479d-bfb9-90849fa6ff40\" (UID: \"1661efff-d6ce-479d-bfb9-90849fa6ff40\") " Dec 03 12:15:16 crc kubenswrapper[5052]: I1203 12:15:16.288062 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/1661efff-d6ce-479d-bfb9-90849fa6ff40-ceilometer-tls-certs\") pod \"1661efff-d6ce-479d-bfb9-90849fa6ff40\" (UID: \"1661efff-d6ce-479d-bfb9-90849fa6ff40\") " Dec 03 12:15:16 crc kubenswrapper[5052]: I1203 12:15:16.288093 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/00ab3478-6308-427a-8e34-4133611a90b2-scripts\") pod \"00ab3478-6308-427a-8e34-4133611a90b2\" (UID: \"00ab3478-6308-427a-8e34-4133611a90b2\") " Dec 03 12:15:16 crc kubenswrapper[5052]: I1203 12:15:16.288129 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1661efff-d6ce-479d-bfb9-90849fa6ff40-combined-ca-bundle\") pod \"1661efff-d6ce-479d-bfb9-90849fa6ff40\" (UID: \"1661efff-d6ce-479d-bfb9-90849fa6ff40\") " Dec 03 12:15:16 crc kubenswrapper[5052]: I1203 12:15:16.288150 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/1661efff-d6ce-479d-bfb9-90849fa6ff40-config-data\") pod \"1661efff-d6ce-479d-bfb9-90849fa6ff40\" (UID: \"1661efff-d6ce-479d-bfb9-90849fa6ff40\") " Dec 03 12:15:16 crc kubenswrapper[5052]: I1203 12:15:16.288167 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/1661efff-d6ce-479d-bfb9-90849fa6ff40-sg-core-conf-yaml\") pod \"1661efff-d6ce-479d-bfb9-90849fa6ff40\" (UID: \"1661efff-d6ce-479d-bfb9-90849fa6ff40\") " Dec 03 12:15:16 crc kubenswrapper[5052]: I1203 12:15:16.288190 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovn-northd-tls-certs\" (UniqueName: \"kubernetes.io/secret/00ab3478-6308-427a-8e34-4133611a90b2-ovn-northd-tls-certs\") pod \"00ab3478-6308-427a-8e34-4133611a90b2\" (UID: \"00ab3478-6308-427a-8e34-4133611a90b2\") " Dec 03 12:15:16 crc kubenswrapper[5052]: I1203 12:15:16.288238 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/00ab3478-6308-427a-8e34-4133611a90b2-config\") pod \"00ab3478-6308-427a-8e34-4133611a90b2\" (UID: \"00ab3478-6308-427a-8e34-4133611a90b2\") " Dec 03 12:15:16 crc kubenswrapper[5052]: I1203 12:15:16.288259 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/00ab3478-6308-427a-8e34-4133611a90b2-metrics-certs-tls-certs\") pod \"00ab3478-6308-427a-8e34-4133611a90b2\" (UID: \"00ab3478-6308-427a-8e34-4133611a90b2\") " Dec 03 12:15:16 crc kubenswrapper[5052]: I1203 12:15:16.288278 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/1661efff-d6ce-479d-bfb9-90849fa6ff40-scripts\") pod \"1661efff-d6ce-479d-bfb9-90849fa6ff40\" (UID: \"1661efff-d6ce-479d-bfb9-90849fa6ff40\") " Dec 03 12:15:16 crc kubenswrapper[5052]: I1203 12:15:16.288332 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zvhkx\" (UniqueName: \"kubernetes.io/projected/1661efff-d6ce-479d-bfb9-90849fa6ff40-kube-api-access-zvhkx\") pod \"1661efff-d6ce-479d-bfb9-90849fa6ff40\" (UID: \"1661efff-d6ce-479d-bfb9-90849fa6ff40\") " Dec 03 12:15:16 crc kubenswrapper[5052]: I1203 12:15:16.288724 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-9m45m\" (UniqueName: \"kubernetes.io/projected/cf3ed088-9069-40cf-9c9e-ee9bdf7761d9-kube-api-access-9m45m\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:16 crc kubenswrapper[5052]: I1203 12:15:16.288742 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/cf3ed088-9069-40cf-9c9e-ee9bdf7761d9-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:16 crc kubenswrapper[5052]: I1203 12:15:16.288751 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/cf3ed088-9069-40cf-9c9e-ee9bdf7761d9-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:16 crc kubenswrapper[5052]: I1203 12:15:16.289185 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1661efff-d6ce-479d-bfb9-90849fa6ff40-run-httpd" (OuterVolumeSpecName: "run-httpd") pod "1661efff-d6ce-479d-bfb9-90849fa6ff40" (UID: "1661efff-d6ce-479d-bfb9-90849fa6ff40"). InnerVolumeSpecName "run-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:15:16 crc kubenswrapper[5052]: I1203 12:15:16.289595 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1661efff-d6ce-479d-bfb9-90849fa6ff40-log-httpd" (OuterVolumeSpecName: "log-httpd") pod "1661efff-d6ce-479d-bfb9-90849fa6ff40" (UID: "1661efff-d6ce-479d-bfb9-90849fa6ff40"). InnerVolumeSpecName "log-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:15:16 crc kubenswrapper[5052]: I1203 12:15:16.290450 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/00ab3478-6308-427a-8e34-4133611a90b2-ovn-rundir" (OuterVolumeSpecName: "ovn-rundir") pod "00ab3478-6308-427a-8e34-4133611a90b2" (UID: "00ab3478-6308-427a-8e34-4133611a90b2"). InnerVolumeSpecName "ovn-rundir". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:15:16 crc kubenswrapper[5052]: I1203 12:15:16.291091 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/00ab3478-6308-427a-8e34-4133611a90b2-config" (OuterVolumeSpecName: "config") pod "00ab3478-6308-427a-8e34-4133611a90b2" (UID: "00ab3478-6308-427a-8e34-4133611a90b2"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:15:16 crc kubenswrapper[5052]: I1203 12:15:16.291695 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1661efff-d6ce-479d-bfb9-90849fa6ff40-kube-api-access-zvhkx" (OuterVolumeSpecName: "kube-api-access-zvhkx") pod "1661efff-d6ce-479d-bfb9-90849fa6ff40" (UID: "1661efff-d6ce-479d-bfb9-90849fa6ff40"). InnerVolumeSpecName "kube-api-access-zvhkx". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:15:16 crc kubenswrapper[5052]: I1203 12:15:16.291825 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/00ab3478-6308-427a-8e34-4133611a90b2-scripts" (OuterVolumeSpecName: "scripts") pod "00ab3478-6308-427a-8e34-4133611a90b2" (UID: "00ab3478-6308-427a-8e34-4133611a90b2"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:15:16 crc kubenswrapper[5052]: I1203 12:15:16.296799 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1661efff-d6ce-479d-bfb9-90849fa6ff40-scripts" (OuterVolumeSpecName: "scripts") pod "1661efff-d6ce-479d-bfb9-90849fa6ff40" (UID: "1661efff-d6ce-479d-bfb9-90849fa6ff40"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:15:16 crc kubenswrapper[5052]: I1203 12:15:16.296881 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/00ab3478-6308-427a-8e34-4133611a90b2-kube-api-access-24whk" (OuterVolumeSpecName: "kube-api-access-24whk") pod "00ab3478-6308-427a-8e34-4133611a90b2" (UID: "00ab3478-6308-427a-8e34-4133611a90b2"). InnerVolumeSpecName "kube-api-access-24whk". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:15:16 crc kubenswrapper[5052]: I1203 12:15:16.314963 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/00ab3478-6308-427a-8e34-4133611a90b2-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "00ab3478-6308-427a-8e34-4133611a90b2" (UID: "00ab3478-6308-427a-8e34-4133611a90b2"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:15:16 crc kubenswrapper[5052]: I1203 12:15:16.333894 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1661efff-d6ce-479d-bfb9-90849fa6ff40-ceilometer-tls-certs" (OuterVolumeSpecName: "ceilometer-tls-certs") pod "1661efff-d6ce-479d-bfb9-90849fa6ff40" (UID: "1661efff-d6ce-479d-bfb9-90849fa6ff40"). InnerVolumeSpecName "ceilometer-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:15:16 crc kubenswrapper[5052]: I1203 12:15:16.340025 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1661efff-d6ce-479d-bfb9-90849fa6ff40-sg-core-conf-yaml" (OuterVolumeSpecName: "sg-core-conf-yaml") pod "1661efff-d6ce-479d-bfb9-90849fa6ff40" (UID: "1661efff-d6ce-479d-bfb9-90849fa6ff40"). InnerVolumeSpecName "sg-core-conf-yaml". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:15:16 crc kubenswrapper[5052]: E1203 12:15:16.351465 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/00ab3478-6308-427a-8e34-4133611a90b2-metrics-certs-tls-certs podName:00ab3478-6308-427a-8e34-4133611a90b2 nodeName:}" failed. No retries permitted until 2025-12-03 12:15:16.851433781 +0000 UTC m=+3307.439675545 (durationBeforeRetry 500ms). Error: error cleaning subPath mounts for volume "metrics-certs-tls-certs" (UniqueName: "kubernetes.io/secret/00ab3478-6308-427a-8e34-4133611a90b2-metrics-certs-tls-certs") pod "00ab3478-6308-427a-8e34-4133611a90b2" (UID: "00ab3478-6308-427a-8e34-4133611a90b2") : error deleting /var/lib/kubelet/pods/00ab3478-6308-427a-8e34-4133611a90b2/volume-subpaths: remove /var/lib/kubelet/pods/00ab3478-6308-427a-8e34-4133611a90b2/volume-subpaths: no such file or directory Dec 03 12:15:16 crc kubenswrapper[5052]: I1203 12:15:16.354278 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/00ab3478-6308-427a-8e34-4133611a90b2-ovn-northd-tls-certs" (OuterVolumeSpecName: "ovn-northd-tls-certs") pod "00ab3478-6308-427a-8e34-4133611a90b2" (UID: "00ab3478-6308-427a-8e34-4133611a90b2"). InnerVolumeSpecName "ovn-northd-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:15:16 crc kubenswrapper[5052]: I1203 12:15:16.355400 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1661efff-d6ce-479d-bfb9-90849fa6ff40-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "1661efff-d6ce-479d-bfb9-90849fa6ff40" (UID: "1661efff-d6ce-479d-bfb9-90849fa6ff40"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:15:16 crc kubenswrapper[5052]: I1203 12:15:16.390120 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1661efff-d6ce-479d-bfb9-90849fa6ff40-config-data" (OuterVolumeSpecName: "config-data") pod "1661efff-d6ce-479d-bfb9-90849fa6ff40" (UID: "1661efff-d6ce-479d-bfb9-90849fa6ff40"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:15:16 crc kubenswrapper[5052]: I1203 12:15:16.390408 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/1661efff-d6ce-479d-bfb9-90849fa6ff40-config-data\") pod \"1661efff-d6ce-479d-bfb9-90849fa6ff40\" (UID: \"1661efff-d6ce-479d-bfb9-90849fa6ff40\") " Dec 03 12:15:16 crc kubenswrapper[5052]: W1203 12:15:16.390558 5052 empty_dir.go:500] Warning: Unmount skipped because path does not exist: /var/lib/kubelet/pods/1661efff-d6ce-479d-bfb9-90849fa6ff40/volumes/kubernetes.io~secret/config-data Dec 03 12:15:16 crc kubenswrapper[5052]: I1203 12:15:16.390574 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1661efff-d6ce-479d-bfb9-90849fa6ff40-config-data" (OuterVolumeSpecName: "config-data") pod "1661efff-d6ce-479d-bfb9-90849fa6ff40" (UID: "1661efff-d6ce-479d-bfb9-90849fa6ff40"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:15:16 crc kubenswrapper[5052]: I1203 12:15:16.390750 5052 reconciler_common.go:293] "Volume detached for volume \"ovn-rundir\" (UniqueName: \"kubernetes.io/empty-dir/00ab3478-6308-427a-8e34-4133611a90b2-ovn-rundir\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:16 crc kubenswrapper[5052]: I1203 12:15:16.390771 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/00ab3478-6308-427a-8e34-4133611a90b2-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:16 crc kubenswrapper[5052]: I1203 12:15:16.390780 5052 reconciler_common.go:293] "Volume detached for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/1661efff-d6ce-479d-bfb9-90849fa6ff40-run-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:16 crc kubenswrapper[5052]: I1203 12:15:16.390789 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-24whk\" (UniqueName: \"kubernetes.io/projected/00ab3478-6308-427a-8e34-4133611a90b2-kube-api-access-24whk\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:16 crc kubenswrapper[5052]: I1203 12:15:16.390798 5052 reconciler_common.go:293] "Volume detached for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/1661efff-d6ce-479d-bfb9-90849fa6ff40-log-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:16 crc kubenswrapper[5052]: I1203 12:15:16.390807 5052 reconciler_common.go:293] "Volume detached for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/1661efff-d6ce-479d-bfb9-90849fa6ff40-ceilometer-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:16 crc kubenswrapper[5052]: I1203 12:15:16.390815 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/00ab3478-6308-427a-8e34-4133611a90b2-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:16 crc kubenswrapper[5052]: I1203 12:15:16.390823 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1661efff-d6ce-479d-bfb9-90849fa6ff40-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:16 crc kubenswrapper[5052]: I1203 12:15:16.390831 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/1661efff-d6ce-479d-bfb9-90849fa6ff40-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:16 crc kubenswrapper[5052]: I1203 12:15:16.390838 5052 reconciler_common.go:293] "Volume detached for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/1661efff-d6ce-479d-bfb9-90849fa6ff40-sg-core-conf-yaml\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:16 crc kubenswrapper[5052]: I1203 12:15:16.390847 5052 reconciler_common.go:293] "Volume detached for volume \"ovn-northd-tls-certs\" (UniqueName: \"kubernetes.io/secret/00ab3478-6308-427a-8e34-4133611a90b2-ovn-northd-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:16 crc kubenswrapper[5052]: I1203 12:15:16.390856 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/00ab3478-6308-427a-8e34-4133611a90b2-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:16 crc kubenswrapper[5052]: I1203 12:15:16.390863 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/1661efff-d6ce-479d-bfb9-90849fa6ff40-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:16 crc kubenswrapper[5052]: I1203 12:15:16.390871 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-zvhkx\" (UniqueName: \"kubernetes.io/projected/1661efff-d6ce-479d-bfb9-90849fa6ff40-kube-api-access-zvhkx\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:16 crc kubenswrapper[5052]: I1203 12:15:16.624381 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-conductor-0" event={"ID":"14774d58-37e8-4e65-b0b0-d355c0070bca","Type":"ContainerDied","Data":"e8a19d510f65a07fc3950e37b6113aaec9bb3cdae16c006c819b91c1bb678de4"} Dec 03 12:15:16 crc kubenswrapper[5052]: I1203 12:15:16.624443 5052 scope.go:117] "RemoveContainer" containerID="33c38af311895b3ea2da41b9880bb47d91d38eadcf28dcf815946f0d92d3f569" Dec 03 12:15:16 crc kubenswrapper[5052]: I1203 12:15:16.624606 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:15:16 crc kubenswrapper[5052]: I1203 12:15:16.633066 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-6fd4994758-fsw2c" event={"ID":"e14dd2e3-f6e1-450c-a688-1dc4eb2ee622","Type":"ContainerDied","Data":"fc4bfab3cfdea7d654ca03ffbcf76a128e0cb31b85de7afed7f10f20633fa5f2"} Dec 03 12:15:16 crc kubenswrapper[5052]: I1203 12:15:16.633146 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-6fd4994758-fsw2c" Dec 03 12:15:16 crc kubenswrapper[5052]: I1203 12:15:16.637213 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-keystone-listener-85f96c8f8-b6m5f" event={"ID":"296a2858-22b8-41a4-91ac-a9bf055b2848","Type":"ContainerDied","Data":"77e4cfb577e8baa1ba5da75e7dd402d3eca4274be0b1d163f90d407d7c6c40f9"} Dec 03 12:15:16 crc kubenswrapper[5052]: I1203 12:15:16.637340 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-keystone-listener-85f96c8f8-b6m5f" Dec 03 12:15:16 crc kubenswrapper[5052]: I1203 12:15:16.639935 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"1661efff-d6ce-479d-bfb9-90849fa6ff40","Type":"ContainerDied","Data":"3b8b6d2940fd19f6bb32b188acbc0cebb086a70566ef76013495a68808a45efd"} Dec 03 12:15:16 crc kubenswrapper[5052]: I1203 12:15:16.640083 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:15:16 crc kubenswrapper[5052]: I1203 12:15:16.644279 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"cf3ed088-9069-40cf-9c9e-ee9bdf7761d9","Type":"ContainerDied","Data":"e071655d60b98a1f28aeee8beb7d104c3a460126999dbdcac767c9abe5620039"} Dec 03 12:15:16 crc kubenswrapper[5052]: I1203 12:15:16.644425 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:15:16 crc kubenswrapper[5052]: I1203 12:15:16.665559 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_ovn-northd-0_00ab3478-6308-427a-8e34-4133611a90b2/ovn-northd/0.log" Dec 03 12:15:16 crc kubenswrapper[5052]: I1203 12:15:16.665697 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:15:16 crc kubenswrapper[5052]: I1203 12:15:16.665695 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovn-northd-0" event={"ID":"00ab3478-6308-427a-8e34-4133611a90b2","Type":"ContainerDied","Data":"78e693d6cab6fe2b4a54716cf3cf7a78b81ff4948a0beca6b155cf2bcbabfa2d"} Dec 03 12:15:16 crc kubenswrapper[5052]: I1203 12:15:16.668633 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-galera-0" event={"ID":"85f5676f-32e8-4c3d-8729-e1eb2fdc8dc9","Type":"ContainerDied","Data":"3ba0d39f21ff90002edd3d3fb8c1a2fd07737053623cac5f18054747025f4d6e"} Dec 03 12:15:16 crc kubenswrapper[5052]: I1203 12:15:16.668531 5052 generic.go:334] "Generic (PLEG): container finished" podID="85f5676f-32e8-4c3d-8729-e1eb2fdc8dc9" containerID="3ba0d39f21ff90002edd3d3fb8c1a2fd07737053623cac5f18054747025f4d6e" exitCode=0 Dec 03 12:15:16 crc kubenswrapper[5052]: I1203 12:15:16.732324 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-0"] Dec 03 12:15:16 crc kubenswrapper[5052]: I1203 12:15:16.735091 5052 scope.go:117] "RemoveContainer" containerID="6544c0c4bb0b5942eaa383c638931be4bd4915098254d54ab63534949d4fc2c6" Dec 03 12:15:16 crc kubenswrapper[5052]: I1203 12:15:16.745575 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-0"] Dec 03 12:15:16 crc kubenswrapper[5052]: I1203 12:15:16.774955 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/keystone-6fd4994758-fsw2c"] Dec 03 12:15:16 crc kubenswrapper[5052]: I1203 12:15:16.782651 5052 scope.go:117] "RemoveContainer" containerID="dd65ec28df374ef84fa1a5f87d159edfa4265d92e582b66969d9f54e2878b585" Dec 03 12:15:16 crc kubenswrapper[5052]: I1203 12:15:16.791167 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/keystone-6fd4994758-fsw2c"] Dec 03 12:15:16 crc kubenswrapper[5052]: I1203 12:15:16.797978 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:15:16 crc kubenswrapper[5052]: I1203 12:15:16.803711 5052 scope.go:117] "RemoveContainer" containerID="88ce6f1697038254e636269a53ace16da9bd7ec94e5543e34a4bab6f9c43e974" Dec 03 12:15:16 crc kubenswrapper[5052]: I1203 12:15:16.804113 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:15:16 crc kubenswrapper[5052]: I1203 12:15:16.809459 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:15:16 crc kubenswrapper[5052]: I1203 12:15:16.814456 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:15:16 crc kubenswrapper[5052]: I1203 12:15:16.819608 5052 scope.go:117] "RemoveContainer" containerID="240bba2f32d305ed667a2f5bc674c5d7f4bda3e122633105398b9c8c9930cfb5" Dec 03 12:15:16 crc kubenswrapper[5052]: I1203 12:15:16.819883 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-keystone-listener-85f96c8f8-b6m5f"] Dec 03 12:15:16 crc kubenswrapper[5052]: I1203 12:15:16.825601 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/barbican-keystone-listener-85f96c8f8-b6m5f"] Dec 03 12:15:16 crc kubenswrapper[5052]: I1203 12:15:16.838616 5052 scope.go:117] "RemoveContainer" containerID="95a8366b935cbf627521e68e3391e0f202e2b033969a53119fa8a475d79c1be3" Dec 03 12:15:16 crc kubenswrapper[5052]: I1203 12:15:16.861773 5052 scope.go:117] "RemoveContainer" containerID="aee5b94189aa0133ba610d83708c280ddef3068baf105fae0fbe385e8bb3d748" Dec 03 12:15:16 crc kubenswrapper[5052]: I1203 12:15:16.899634 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/00ab3478-6308-427a-8e34-4133611a90b2-metrics-certs-tls-certs\") pod \"00ab3478-6308-427a-8e34-4133611a90b2\" (UID: \"00ab3478-6308-427a-8e34-4133611a90b2\") " Dec 03 12:15:16 crc kubenswrapper[5052]: I1203 12:15:16.904590 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/00ab3478-6308-427a-8e34-4133611a90b2-metrics-certs-tls-certs" (OuterVolumeSpecName: "metrics-certs-tls-certs") pod "00ab3478-6308-427a-8e34-4133611a90b2" (UID: "00ab3478-6308-427a-8e34-4133611a90b2"). InnerVolumeSpecName "metrics-certs-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:15:16 crc kubenswrapper[5052]: I1203 12:15:16.963407 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:15:16 crc kubenswrapper[5052]: I1203 12:15:16.977462 5052 scope.go:117] "RemoveContainer" containerID="e964aba7c310e135f899bcb05dcd28df12a6cfb568f657650c49176388d578b0" Dec 03 12:15:17 crc kubenswrapper[5052]: I1203 12:15:17.003556 5052 reconciler_common.go:293] "Volume detached for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/00ab3478-6308-427a-8e34-4133611a90b2-metrics-certs-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:17 crc kubenswrapper[5052]: I1203 12:15:17.006694 5052 scope.go:117] "RemoveContainer" containerID="81879a070623f490e700ae5e038359faa2d3ce34a28b8f7e4b856f6c4f95fa7f" Dec 03 12:15:17 crc kubenswrapper[5052]: I1203 12:15:17.013381 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ovn-northd-0"] Dec 03 12:15:17 crc kubenswrapper[5052]: I1203 12:15:17.019297 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ovn-northd-0"] Dec 03 12:15:17 crc kubenswrapper[5052]: I1203 12:15:17.027094 5052 scope.go:117] "RemoveContainer" containerID="b77011f57a038af73fce8dd2b2cefedfa4b2e4023e9b43da72366d20be9344f4" Dec 03 12:15:17 crc kubenswrapper[5052]: I1203 12:15:17.047595 5052 scope.go:117] "RemoveContainer" containerID="98d14ce0cb9919bb83964fb391e45a00edf1d049296adef59b4e2e0f042c63d2" Dec 03 12:15:17 crc kubenswrapper[5052]: I1203 12:15:17.086683 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-kl2c8" Dec 03 12:15:17 crc kubenswrapper[5052]: I1203 12:15:17.104956 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/85f5676f-32e8-4c3d-8729-e1eb2fdc8dc9-config-data-generated\") pod \"85f5676f-32e8-4c3d-8729-e1eb2fdc8dc9\" (UID: \"85f5676f-32e8-4c3d-8729-e1eb2fdc8dc9\") " Dec 03 12:15:17 crc kubenswrapper[5052]: I1203 12:15:17.105090 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/85f5676f-32e8-4c3d-8729-e1eb2fdc8dc9-galera-tls-certs\") pod \"85f5676f-32e8-4c3d-8729-e1eb2fdc8dc9\" (UID: \"85f5676f-32e8-4c3d-8729-e1eb2fdc8dc9\") " Dec 03 12:15:17 crc kubenswrapper[5052]: I1203 12:15:17.105155 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/85f5676f-32e8-4c3d-8729-e1eb2fdc8dc9-kolla-config\") pod \"85f5676f-32e8-4c3d-8729-e1eb2fdc8dc9\" (UID: \"85f5676f-32e8-4c3d-8729-e1eb2fdc8dc9\") " Dec 03 12:15:17 crc kubenswrapper[5052]: I1203 12:15:17.105183 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/85f5676f-32e8-4c3d-8729-e1eb2fdc8dc9-config-data-default\") pod \"85f5676f-32e8-4c3d-8729-e1eb2fdc8dc9\" (UID: \"85f5676f-32e8-4c3d-8729-e1eb2fdc8dc9\") " Dec 03 12:15:17 crc kubenswrapper[5052]: I1203 12:15:17.105223 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/85f5676f-32e8-4c3d-8729-e1eb2fdc8dc9-operator-scripts\") pod \"85f5676f-32e8-4c3d-8729-e1eb2fdc8dc9\" (UID: \"85f5676f-32e8-4c3d-8729-e1eb2fdc8dc9\") " Dec 03 12:15:17 crc kubenswrapper[5052]: I1203 12:15:17.105295 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"mysql-db\" (UniqueName: \"kubernetes.io/local-volume/local-storage16-crc\") pod \"85f5676f-32e8-4c3d-8729-e1eb2fdc8dc9\" (UID: \"85f5676f-32e8-4c3d-8729-e1eb2fdc8dc9\") " Dec 03 12:15:17 crc kubenswrapper[5052]: I1203 12:15:17.105437 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/85f5676f-32e8-4c3d-8729-e1eb2fdc8dc9-combined-ca-bundle\") pod \"85f5676f-32e8-4c3d-8729-e1eb2fdc8dc9\" (UID: \"85f5676f-32e8-4c3d-8729-e1eb2fdc8dc9\") " Dec 03 12:15:17 crc kubenswrapper[5052]: I1203 12:15:17.105618 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rbmfn\" (UniqueName: \"kubernetes.io/projected/85f5676f-32e8-4c3d-8729-e1eb2fdc8dc9-kube-api-access-rbmfn\") pod \"85f5676f-32e8-4c3d-8729-e1eb2fdc8dc9\" (UID: \"85f5676f-32e8-4c3d-8729-e1eb2fdc8dc9\") " Dec 03 12:15:17 crc kubenswrapper[5052]: I1203 12:15:17.109814 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/85f5676f-32e8-4c3d-8729-e1eb2fdc8dc9-config-data-default" (OuterVolumeSpecName: "config-data-default") pod "85f5676f-32e8-4c3d-8729-e1eb2fdc8dc9" (UID: "85f5676f-32e8-4c3d-8729-e1eb2fdc8dc9"). InnerVolumeSpecName "config-data-default". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:15:17 crc kubenswrapper[5052]: I1203 12:15:17.110814 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/85f5676f-32e8-4c3d-8729-e1eb2fdc8dc9-config-data-generated" (OuterVolumeSpecName: "config-data-generated") pod "85f5676f-32e8-4c3d-8729-e1eb2fdc8dc9" (UID: "85f5676f-32e8-4c3d-8729-e1eb2fdc8dc9"). InnerVolumeSpecName "config-data-generated". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:15:17 crc kubenswrapper[5052]: I1203 12:15:17.114957 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/85f5676f-32e8-4c3d-8729-e1eb2fdc8dc9-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "85f5676f-32e8-4c3d-8729-e1eb2fdc8dc9" (UID: "85f5676f-32e8-4c3d-8729-e1eb2fdc8dc9"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:15:17 crc kubenswrapper[5052]: I1203 12:15:17.119728 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/85f5676f-32e8-4c3d-8729-e1eb2fdc8dc9-kube-api-access-rbmfn" (OuterVolumeSpecName: "kube-api-access-rbmfn") pod "85f5676f-32e8-4c3d-8729-e1eb2fdc8dc9" (UID: "85f5676f-32e8-4c3d-8729-e1eb2fdc8dc9"). InnerVolumeSpecName "kube-api-access-rbmfn". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:15:17 crc kubenswrapper[5052]: I1203 12:15:17.120804 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/85f5676f-32e8-4c3d-8729-e1eb2fdc8dc9-kolla-config" (OuterVolumeSpecName: "kolla-config") pod "85f5676f-32e8-4c3d-8729-e1eb2fdc8dc9" (UID: "85f5676f-32e8-4c3d-8729-e1eb2fdc8dc9"). InnerVolumeSpecName "kolla-config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:15:17 crc kubenswrapper[5052]: I1203 12:15:17.147676 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-6bcdfcd6b5-mhvjp"] Dec 03 12:15:17 crc kubenswrapper[5052]: I1203 12:15:17.148098 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage16-crc" (OuterVolumeSpecName: "mysql-db") pod "85f5676f-32e8-4c3d-8729-e1eb2fdc8dc9" (UID: "85f5676f-32e8-4c3d-8729-e1eb2fdc8dc9"). InnerVolumeSpecName "local-storage16-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 12:15:17 crc kubenswrapper[5052]: I1203 12:15:17.147894 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-6bcdfcd6b5-mhvjp" podUID="9a25d0bb-42c4-4214-b81d-23b29f335155" containerName="dnsmasq-dns" containerID="cri-o://f2236846a20b2c05e69f531a234547ed762dc4e7260c85c9dc2d7a23360fd2cf" gracePeriod=10 Dec 03 12:15:17 crc kubenswrapper[5052]: I1203 12:15:17.162734 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/85f5676f-32e8-4c3d-8729-e1eb2fdc8dc9-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "85f5676f-32e8-4c3d-8729-e1eb2fdc8dc9" (UID: "85f5676f-32e8-4c3d-8729-e1eb2fdc8dc9"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:15:17 crc kubenswrapper[5052]: I1203 12:15:17.177737 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/85f5676f-32e8-4c3d-8729-e1eb2fdc8dc9-galera-tls-certs" (OuterVolumeSpecName: "galera-tls-certs") pod "85f5676f-32e8-4c3d-8729-e1eb2fdc8dc9" (UID: "85f5676f-32e8-4c3d-8729-e1eb2fdc8dc9"). InnerVolumeSpecName "galera-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:15:17 crc kubenswrapper[5052]: I1203 12:15:17.207824 5052 reconciler_common.go:293] "Volume detached for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/85f5676f-32e8-4c3d-8729-e1eb2fdc8dc9-kolla-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:17 crc kubenswrapper[5052]: I1203 12:15:17.207851 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/85f5676f-32e8-4c3d-8729-e1eb2fdc8dc9-config-data-default\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:17 crc kubenswrapper[5052]: I1203 12:15:17.207859 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/85f5676f-32e8-4c3d-8729-e1eb2fdc8dc9-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:17 crc kubenswrapper[5052]: I1203 12:15:17.207878 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage16-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage16-crc\") on node \"crc\" " Dec 03 12:15:17 crc kubenswrapper[5052]: I1203 12:15:17.207887 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/85f5676f-32e8-4c3d-8729-e1eb2fdc8dc9-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:17 crc kubenswrapper[5052]: I1203 12:15:17.207898 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-rbmfn\" (UniqueName: \"kubernetes.io/projected/85f5676f-32e8-4c3d-8729-e1eb2fdc8dc9-kube-api-access-rbmfn\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:17 crc kubenswrapper[5052]: I1203 12:15:17.207908 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/85f5676f-32e8-4c3d-8729-e1eb2fdc8dc9-config-data-generated\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:17 crc kubenswrapper[5052]: I1203 12:15:17.207916 5052 reconciler_common.go:293] "Volume detached for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/85f5676f-32e8-4c3d-8729-e1eb2fdc8dc9-galera-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:17 crc kubenswrapper[5052]: I1203 12:15:17.225189 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage16-crc" (UniqueName: "kubernetes.io/local-volume/local-storage16-crc") on node "crc" Dec 03 12:15:17 crc kubenswrapper[5052]: I1203 12:15:17.309699 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage16-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage16-crc\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:17 crc kubenswrapper[5052]: I1203 12:15:17.604336 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-6bcdfcd6b5-mhvjp" Dec 03 12:15:17 crc kubenswrapper[5052]: I1203 12:15:17.694326 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="00ab3478-6308-427a-8e34-4133611a90b2" path="/var/lib/kubelet/pods/00ab3478-6308-427a-8e34-4133611a90b2/volumes" Dec 03 12:15:17 crc kubenswrapper[5052]: I1203 12:15:17.695100 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="14774d58-37e8-4e65-b0b0-d355c0070bca" path="/var/lib/kubelet/pods/14774d58-37e8-4e65-b0b0-d355c0070bca/volumes" Dec 03 12:15:17 crc kubenswrapper[5052]: I1203 12:15:17.695883 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1661efff-d6ce-479d-bfb9-90849fa6ff40" path="/var/lib/kubelet/pods/1661efff-d6ce-479d-bfb9-90849fa6ff40/volumes" Dec 03 12:15:17 crc kubenswrapper[5052]: I1203 12:15:17.698967 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1c98e0a9-267b-46b6-be02-fbd163462940" path="/var/lib/kubelet/pods/1c98e0a9-267b-46b6-be02-fbd163462940/volumes" Dec 03 12:15:17 crc kubenswrapper[5052]: I1203 12:15:17.699642 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2047705d-ae83-4972-913b-868f2a623022" path="/var/lib/kubelet/pods/2047705d-ae83-4972-913b-868f2a623022/volumes" Dec 03 12:15:17 crc kubenswrapper[5052]: I1203 12:15:17.700767 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="296a2858-22b8-41a4-91ac-a9bf055b2848" path="/var/lib/kubelet/pods/296a2858-22b8-41a4-91ac-a9bf055b2848/volumes" Dec 03 12:15:17 crc kubenswrapper[5052]: I1203 12:15:17.701435 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a81519a6-c5c5-4f47-86de-4d63ebef81fe" path="/var/lib/kubelet/pods/a81519a6-c5c5-4f47-86de-4d63ebef81fe/volumes" Dec 03 12:15:17 crc kubenswrapper[5052]: I1203 12:15:17.702005 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cf3ed088-9069-40cf-9c9e-ee9bdf7761d9" path="/var/lib/kubelet/pods/cf3ed088-9069-40cf-9c9e-ee9bdf7761d9/volumes" Dec 03 12:15:17 crc kubenswrapper[5052]: I1203 12:15:17.702623 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e14dd2e3-f6e1-450c-a688-1dc4eb2ee622" path="/var/lib/kubelet/pods/e14dd2e3-f6e1-450c-a688-1dc4eb2ee622/volumes" Dec 03 12:15:17 crc kubenswrapper[5052]: I1203 12:15:17.704991 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eae2f902-0630-43d2-9a05-cc0d68350068" path="/var/lib/kubelet/pods/eae2f902-0630-43d2-9a05-cc0d68350068/volumes" Dec 03 12:15:17 crc kubenswrapper[5052]: I1203 12:15:17.706172 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:15:17 crc kubenswrapper[5052]: I1203 12:15:17.714104 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-galera-0" event={"ID":"85f5676f-32e8-4c3d-8729-e1eb2fdc8dc9","Type":"ContainerDied","Data":"8079da34f90ae24fe3455ebeaf544cb53b92f6b936c319f989b8c4d8f8ba1026"} Dec 03 12:15:17 crc kubenswrapper[5052]: I1203 12:15:17.714211 5052 scope.go:117] "RemoveContainer" containerID="3ba0d39f21ff90002edd3d3fb8c1a2fd07737053623cac5f18054747025f4d6e" Dec 03 12:15:17 crc kubenswrapper[5052]: I1203 12:15:17.714525 5052 generic.go:334] "Generic (PLEG): container finished" podID="9a25d0bb-42c4-4214-b81d-23b29f335155" containerID="f2236846a20b2c05e69f531a234547ed762dc4e7260c85c9dc2d7a23360fd2cf" exitCode=0 Dec 03 12:15:17 crc kubenswrapper[5052]: I1203 12:15:17.714717 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-6bcdfcd6b5-mhvjp" event={"ID":"9a25d0bb-42c4-4214-b81d-23b29f335155","Type":"ContainerDied","Data":"f2236846a20b2c05e69f531a234547ed762dc4e7260c85c9dc2d7a23360fd2cf"} Dec 03 12:15:17 crc kubenswrapper[5052]: I1203 12:15:17.714950 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-6bcdfcd6b5-mhvjp" Dec 03 12:15:17 crc kubenswrapper[5052]: I1203 12:15:17.714993 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-6bcdfcd6b5-mhvjp" event={"ID":"9a25d0bb-42c4-4214-b81d-23b29f335155","Type":"ContainerDied","Data":"b5f99f908aae3482cc53681f026be2b723bc889ecebc87531a0e82daae1e7264"} Dec 03 12:15:17 crc kubenswrapper[5052]: I1203 12:15:17.717835 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/9a25d0bb-42c4-4214-b81d-23b29f335155-config\") pod \"9a25d0bb-42c4-4214-b81d-23b29f335155\" (UID: \"9a25d0bb-42c4-4214-b81d-23b29f335155\") " Dec 03 12:15:17 crc kubenswrapper[5052]: I1203 12:15:17.717921 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9wblq\" (UniqueName: \"kubernetes.io/projected/9a25d0bb-42c4-4214-b81d-23b29f335155-kube-api-access-9wblq\") pod \"9a25d0bb-42c4-4214-b81d-23b29f335155\" (UID: \"9a25d0bb-42c4-4214-b81d-23b29f335155\") " Dec 03 12:15:17 crc kubenswrapper[5052]: I1203 12:15:17.718019 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/9a25d0bb-42c4-4214-b81d-23b29f335155-dnsmasq-svc\") pod \"9a25d0bb-42c4-4214-b81d-23b29f335155\" (UID: \"9a25d0bb-42c4-4214-b81d-23b29f335155\") " Dec 03 12:15:17 crc kubenswrapper[5052]: I1203 12:15:17.718081 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"dns-swift-storage-0\" (UniqueName: \"kubernetes.io/configmap/9a25d0bb-42c4-4214-b81d-23b29f335155-dns-swift-storage-0\") pod \"9a25d0bb-42c4-4214-b81d-23b29f335155\" (UID: \"9a25d0bb-42c4-4214-b81d-23b29f335155\") " Dec 03 12:15:17 crc kubenswrapper[5052]: I1203 12:15:17.722163 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9a25d0bb-42c4-4214-b81d-23b29f335155-kube-api-access-9wblq" (OuterVolumeSpecName: "kube-api-access-9wblq") pod "9a25d0bb-42c4-4214-b81d-23b29f335155" (UID: "9a25d0bb-42c4-4214-b81d-23b29f335155"). InnerVolumeSpecName "kube-api-access-9wblq". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:15:17 crc kubenswrapper[5052]: I1203 12:15:17.769294 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9a25d0bb-42c4-4214-b81d-23b29f335155-config" (OuterVolumeSpecName: "config") pod "9a25d0bb-42c4-4214-b81d-23b29f335155" (UID: "9a25d0bb-42c4-4214-b81d-23b29f335155"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:15:17 crc kubenswrapper[5052]: I1203 12:15:17.773762 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9a25d0bb-42c4-4214-b81d-23b29f335155-dns-swift-storage-0" (OuterVolumeSpecName: "dns-swift-storage-0") pod "9a25d0bb-42c4-4214-b81d-23b29f335155" (UID: "9a25d0bb-42c4-4214-b81d-23b29f335155"). InnerVolumeSpecName "dns-swift-storage-0". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:15:17 crc kubenswrapper[5052]: I1203 12:15:17.782730 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9a25d0bb-42c4-4214-b81d-23b29f335155-dnsmasq-svc" (OuterVolumeSpecName: "dnsmasq-svc") pod "9a25d0bb-42c4-4214-b81d-23b29f335155" (UID: "9a25d0bb-42c4-4214-b81d-23b29f335155"). InnerVolumeSpecName "dnsmasq-svc". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:15:17 crc kubenswrapper[5052]: I1203 12:15:17.822219 5052 reconciler_common.go:293] "Volume detached for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/9a25d0bb-42c4-4214-b81d-23b29f335155-dnsmasq-svc\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:17 crc kubenswrapper[5052]: I1203 12:15:17.822254 5052 reconciler_common.go:293] "Volume detached for volume \"dns-swift-storage-0\" (UniqueName: \"kubernetes.io/configmap/9a25d0bb-42c4-4214-b81d-23b29f335155-dns-swift-storage-0\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:17 crc kubenswrapper[5052]: I1203 12:15:17.822266 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/9a25d0bb-42c4-4214-b81d-23b29f335155-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:17 crc kubenswrapper[5052]: I1203 12:15:17.822274 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-9wblq\" (UniqueName: \"kubernetes.io/projected/9a25d0bb-42c4-4214-b81d-23b29f335155-kube-api-access-9wblq\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:17 crc kubenswrapper[5052]: I1203 12:15:17.883923 5052 scope.go:117] "RemoveContainer" containerID="83ffd9d45c67b785bed3d072b308795c1b569e98f4500db5effcc832d24bf349" Dec 03 12:15:17 crc kubenswrapper[5052]: I1203 12:15:17.905705 5052 scope.go:117] "RemoveContainer" containerID="f2236846a20b2c05e69f531a234547ed762dc4e7260c85c9dc2d7a23360fd2cf" Dec 03 12:15:17 crc kubenswrapper[5052]: I1203 12:15:17.921812 5052 scope.go:117] "RemoveContainer" containerID="4912275ac49c2d4e0e4d1b6267954dc550f95013e82ac3cf4070fdd4ba7e5da6" Dec 03 12:15:17 crc kubenswrapper[5052]: I1203 12:15:17.939836 5052 scope.go:117] "RemoveContainer" containerID="f2236846a20b2c05e69f531a234547ed762dc4e7260c85c9dc2d7a23360fd2cf" Dec 03 12:15:17 crc kubenswrapper[5052]: E1203 12:15:17.940228 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f2236846a20b2c05e69f531a234547ed762dc4e7260c85c9dc2d7a23360fd2cf\": container with ID starting with f2236846a20b2c05e69f531a234547ed762dc4e7260c85c9dc2d7a23360fd2cf not found: ID does not exist" containerID="f2236846a20b2c05e69f531a234547ed762dc4e7260c85c9dc2d7a23360fd2cf" Dec 03 12:15:17 crc kubenswrapper[5052]: I1203 12:15:17.940263 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f2236846a20b2c05e69f531a234547ed762dc4e7260c85c9dc2d7a23360fd2cf"} err="failed to get container status \"f2236846a20b2c05e69f531a234547ed762dc4e7260c85c9dc2d7a23360fd2cf\": rpc error: code = NotFound desc = could not find container \"f2236846a20b2c05e69f531a234547ed762dc4e7260c85c9dc2d7a23360fd2cf\": container with ID starting with f2236846a20b2c05e69f531a234547ed762dc4e7260c85c9dc2d7a23360fd2cf not found: ID does not exist" Dec 03 12:15:17 crc kubenswrapper[5052]: I1203 12:15:17.940288 5052 scope.go:117] "RemoveContainer" containerID="4912275ac49c2d4e0e4d1b6267954dc550f95013e82ac3cf4070fdd4ba7e5da6" Dec 03 12:15:17 crc kubenswrapper[5052]: E1203 12:15:17.940626 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4912275ac49c2d4e0e4d1b6267954dc550f95013e82ac3cf4070fdd4ba7e5da6\": container with ID starting with 4912275ac49c2d4e0e4d1b6267954dc550f95013e82ac3cf4070fdd4ba7e5da6 not found: ID does not exist" containerID="4912275ac49c2d4e0e4d1b6267954dc550f95013e82ac3cf4070fdd4ba7e5da6" Dec 03 12:15:17 crc kubenswrapper[5052]: I1203 12:15:17.940657 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4912275ac49c2d4e0e4d1b6267954dc550f95013e82ac3cf4070fdd4ba7e5da6"} err="failed to get container status \"4912275ac49c2d4e0e4d1b6267954dc550f95013e82ac3cf4070fdd4ba7e5da6\": rpc error: code = NotFound desc = could not find container \"4912275ac49c2d4e0e4d1b6267954dc550f95013e82ac3cf4070fdd4ba7e5da6\": container with ID starting with 4912275ac49c2d4e0e4d1b6267954dc550f95013e82ac3cf4070fdd4ba7e5da6 not found: ID does not exist" Dec 03 12:15:18 crc kubenswrapper[5052]: I1203 12:15:18.050819 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-6bcdfcd6b5-mhvjp"] Dec 03 12:15:18 crc kubenswrapper[5052]: I1203 12:15:18.056375 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-6bcdfcd6b5-mhvjp"] Dec 03 12:15:19 crc kubenswrapper[5052]: I1203 12:15:19.481074 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openshift-marketplace/redhat-operators-fhsfk"] Dec 03 12:15:19 crc kubenswrapper[5052]: E1203 12:15:19.481628 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="dfdd8013-ee6e-4ec6-b5d1-fb423f444430" containerName="kube-state-metrics" Dec 03 12:15:19 crc kubenswrapper[5052]: I1203 12:15:19.481640 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="dfdd8013-ee6e-4ec6-b5d1-fb423f444430" containerName="kube-state-metrics" Dec 03 12:15:19 crc kubenswrapper[5052]: E1203 12:15:19.481653 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="14774d58-37e8-4e65-b0b0-d355c0070bca" containerName="nova-cell0-conductor-conductor" Dec 03 12:15:19 crc kubenswrapper[5052]: I1203 12:15:19.481659 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="14774d58-37e8-4e65-b0b0-d355c0070bca" containerName="nova-cell0-conductor-conductor" Dec 03 12:15:19 crc kubenswrapper[5052]: E1203 12:15:19.481667 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="c2a65caa-2eec-48af-bdad-8fc19d827867" containerName="nova-cell1-conductor-conductor" Dec 03 12:15:19 crc kubenswrapper[5052]: I1203 12:15:19.481673 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="c2a65caa-2eec-48af-bdad-8fc19d827867" containerName="nova-cell1-conductor-conductor" Dec 03 12:15:19 crc kubenswrapper[5052]: E1203 12:15:19.481678 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="1661efff-d6ce-479d-bfb9-90849fa6ff40" containerName="ceilometer-central-agent" Dec 03 12:15:19 crc kubenswrapper[5052]: I1203 12:15:19.481684 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="1661efff-d6ce-479d-bfb9-90849fa6ff40" containerName="ceilometer-central-agent" Dec 03 12:15:19 crc kubenswrapper[5052]: E1203 12:15:19.481696 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="b25d2a0c-1e93-4b3d-965a-a7c021f9649d" containerName="barbican-api-log" Dec 03 12:15:19 crc kubenswrapper[5052]: I1203 12:15:19.481703 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="b25d2a0c-1e93-4b3d-965a-a7c021f9649d" containerName="barbican-api-log" Dec 03 12:15:19 crc kubenswrapper[5052]: E1203 12:15:19.481709 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="ae6940a4-0f4c-46ab-8218-56bf5c5d3084" containerName="proxy-httpd" Dec 03 12:15:19 crc kubenswrapper[5052]: I1203 12:15:19.481714 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae6940a4-0f4c-46ab-8218-56bf5c5d3084" containerName="proxy-httpd" Dec 03 12:15:19 crc kubenswrapper[5052]: E1203 12:15:19.481726 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="d81b09dc-75a1-45f8-ace4-786531ad3bfd" containerName="glance-httpd" Dec 03 12:15:19 crc kubenswrapper[5052]: I1203 12:15:19.481731 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="d81b09dc-75a1-45f8-ace4-786531ad3bfd" containerName="glance-httpd" Dec 03 12:15:19 crc kubenswrapper[5052]: E1203 12:15:19.481738 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="e63eb1eb-e0b9-46ea-a19b-8f5c018728ff" containerName="setup-container" Dec 03 12:15:19 crc kubenswrapper[5052]: I1203 12:15:19.481744 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="e63eb1eb-e0b9-46ea-a19b-8f5c018728ff" containerName="setup-container" Dec 03 12:15:19 crc kubenswrapper[5052]: E1203 12:15:19.481754 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="8283de30-8976-47bd-bd09-aab53140f68d" containerName="placement-api" Dec 03 12:15:19 crc kubenswrapper[5052]: I1203 12:15:19.481759 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="8283de30-8976-47bd-bd09-aab53140f68d" containerName="placement-api" Dec 03 12:15:19 crc kubenswrapper[5052]: E1203 12:15:19.481774 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="388bc903-097f-4f97-8f19-2f7bcdb49cc5" containerName="cinder-api-log" Dec 03 12:15:19 crc kubenswrapper[5052]: I1203 12:15:19.481780 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="388bc903-097f-4f97-8f19-2f7bcdb49cc5" containerName="cinder-api-log" Dec 03 12:15:19 crc kubenswrapper[5052]: E1203 12:15:19.481793 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="baa1c9d4-5887-47e3-be99-96f718178f58" containerName="ovsdbserver-sb" Dec 03 12:15:19 crc kubenswrapper[5052]: I1203 12:15:19.481800 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="baa1c9d4-5887-47e3-be99-96f718178f58" containerName="ovsdbserver-sb" Dec 03 12:15:19 crc kubenswrapper[5052]: E1203 12:15:19.481811 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="85f5676f-32e8-4c3d-8729-e1eb2fdc8dc9" containerName="mysql-bootstrap" Dec 03 12:15:19 crc kubenswrapper[5052]: I1203 12:15:19.481817 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="85f5676f-32e8-4c3d-8729-e1eb2fdc8dc9" containerName="mysql-bootstrap" Dec 03 12:15:19 crc kubenswrapper[5052]: E1203 12:15:19.481829 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="89b056a7-e448-447d-be7d-b69f3d4862a1" containerName="cinder-scheduler" Dec 03 12:15:19 crc kubenswrapper[5052]: I1203 12:15:19.481836 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="89b056a7-e448-447d-be7d-b69f3d4862a1" containerName="cinder-scheduler" Dec 03 12:15:19 crc kubenswrapper[5052]: E1203 12:15:19.481844 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="e63eb1eb-e0b9-46ea-a19b-8f5c018728ff" containerName="rabbitmq" Dec 03 12:15:19 crc kubenswrapper[5052]: I1203 12:15:19.481849 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="e63eb1eb-e0b9-46ea-a19b-8f5c018728ff" containerName="rabbitmq" Dec 03 12:15:19 crc kubenswrapper[5052]: E1203 12:15:19.481857 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f" containerName="rabbitmq" Dec 03 12:15:19 crc kubenswrapper[5052]: I1203 12:15:19.481863 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f" containerName="rabbitmq" Dec 03 12:15:19 crc kubenswrapper[5052]: E1203 12:15:19.481872 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="e14dd2e3-f6e1-450c-a688-1dc4eb2ee622" containerName="keystone-api" Dec 03 12:15:19 crc kubenswrapper[5052]: I1203 12:15:19.481878 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="e14dd2e3-f6e1-450c-a688-1dc4eb2ee622" containerName="keystone-api" Dec 03 12:15:19 crc kubenswrapper[5052]: E1203 12:15:19.481887 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f" containerName="setup-container" Dec 03 12:15:19 crc kubenswrapper[5052]: I1203 12:15:19.481894 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f" containerName="setup-container" Dec 03 12:15:19 crc kubenswrapper[5052]: E1203 12:15:19.481904 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="cb68adf1-26f2-4b14-9874-dde28826aba3" containerName="nova-api-api" Dec 03 12:15:19 crc kubenswrapper[5052]: I1203 12:15:19.481910 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="cb68adf1-26f2-4b14-9874-dde28826aba3" containerName="nova-api-api" Dec 03 12:15:19 crc kubenswrapper[5052]: E1203 12:15:19.481919 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="f1619fbf-df68-484f-99d4-09b81a7a334e" containerName="mariadb-account-delete" Dec 03 12:15:19 crc kubenswrapper[5052]: I1203 12:15:19.481924 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="f1619fbf-df68-484f-99d4-09b81a7a334e" containerName="mariadb-account-delete" Dec 03 12:15:19 crc kubenswrapper[5052]: E1203 12:15:19.481933 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="cd486343-0aad-4039-a23b-34c60cd1a381" containerName="nova-metadata-metadata" Dec 03 12:15:19 crc kubenswrapper[5052]: I1203 12:15:19.481939 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="cd486343-0aad-4039-a23b-34c60cd1a381" containerName="nova-metadata-metadata" Dec 03 12:15:19 crc kubenswrapper[5052]: E1203 12:15:19.481949 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="d81b09dc-75a1-45f8-ace4-786531ad3bfd" containerName="glance-log" Dec 03 12:15:19 crc kubenswrapper[5052]: I1203 12:15:19.481954 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="d81b09dc-75a1-45f8-ace4-786531ad3bfd" containerName="glance-log" Dec 03 12:15:19 crc kubenswrapper[5052]: E1203 12:15:19.481962 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="b28a125a-7966-4f97-91c1-fa52d6062405" containerName="memcached" Dec 03 12:15:19 crc kubenswrapper[5052]: I1203 12:15:19.481967 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="b28a125a-7966-4f97-91c1-fa52d6062405" containerName="memcached" Dec 03 12:15:19 crc kubenswrapper[5052]: E1203 12:15:19.481975 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="cf3ed088-9069-40cf-9c9e-ee9bdf7761d9" containerName="nova-scheduler-scheduler" Dec 03 12:15:19 crc kubenswrapper[5052]: I1203 12:15:19.481980 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="cf3ed088-9069-40cf-9c9e-ee9bdf7761d9" containerName="nova-scheduler-scheduler" Dec 03 12:15:19 crc kubenswrapper[5052]: E1203 12:15:19.481989 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="a15b6144-952c-4d41-a141-5bca5c7fd402" containerName="nova-cell1-novncproxy-novncproxy" Dec 03 12:15:19 crc kubenswrapper[5052]: I1203 12:15:19.481995 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="a15b6144-952c-4d41-a141-5bca5c7fd402" containerName="nova-cell1-novncproxy-novncproxy" Dec 03 12:15:19 crc kubenswrapper[5052]: E1203 12:15:19.482006 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="00a9935f-ef2d-4976-97a0-78ee6325433d" containerName="galera" Dec 03 12:15:19 crc kubenswrapper[5052]: I1203 12:15:19.482012 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="00a9935f-ef2d-4976-97a0-78ee6325433d" containerName="galera" Dec 03 12:15:19 crc kubenswrapper[5052]: E1203 12:15:19.482017 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="cb68adf1-26f2-4b14-9874-dde28826aba3" containerName="nova-api-log" Dec 03 12:15:19 crc kubenswrapper[5052]: I1203 12:15:19.482022 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="cb68adf1-26f2-4b14-9874-dde28826aba3" containerName="nova-api-log" Dec 03 12:15:19 crc kubenswrapper[5052]: E1203 12:15:19.482028 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="0a2de558-2a4b-46ef-9b53-cada8cd466c4" containerName="openstack-network-exporter" Dec 03 12:15:19 crc kubenswrapper[5052]: I1203 12:15:19.482034 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a2de558-2a4b-46ef-9b53-cada8cd466c4" containerName="openstack-network-exporter" Dec 03 12:15:19 crc kubenswrapper[5052]: E1203 12:15:19.482043 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="9a25d0bb-42c4-4214-b81d-23b29f335155" containerName="dnsmasq-dns" Dec 03 12:15:19 crc kubenswrapper[5052]: I1203 12:15:19.482049 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="9a25d0bb-42c4-4214-b81d-23b29f335155" containerName="dnsmasq-dns" Dec 03 12:15:19 crc kubenswrapper[5052]: E1203 12:15:19.482060 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="8283de30-8976-47bd-bd09-aab53140f68d" containerName="placement-log" Dec 03 12:15:19 crc kubenswrapper[5052]: I1203 12:15:19.482065 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="8283de30-8976-47bd-bd09-aab53140f68d" containerName="placement-log" Dec 03 12:15:19 crc kubenswrapper[5052]: E1203 12:15:19.482076 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="ae6940a4-0f4c-46ab-8218-56bf5c5d3084" containerName="proxy-server" Dec 03 12:15:19 crc kubenswrapper[5052]: I1203 12:15:19.482081 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae6940a4-0f4c-46ab-8218-56bf5c5d3084" containerName="proxy-server" Dec 03 12:15:19 crc kubenswrapper[5052]: E1203 12:15:19.482091 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="1c98e0a9-267b-46b6-be02-fbd163462940" containerName="barbican-worker" Dec 03 12:15:19 crc kubenswrapper[5052]: I1203 12:15:19.482096 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="1c98e0a9-267b-46b6-be02-fbd163462940" containerName="barbican-worker" Dec 03 12:15:19 crc kubenswrapper[5052]: E1203 12:15:19.482105 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="1c98e0a9-267b-46b6-be02-fbd163462940" containerName="barbican-worker-log" Dec 03 12:15:19 crc kubenswrapper[5052]: I1203 12:15:19.482111 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="1c98e0a9-267b-46b6-be02-fbd163462940" containerName="barbican-worker-log" Dec 03 12:15:19 crc kubenswrapper[5052]: E1203 12:15:19.482119 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="1661efff-d6ce-479d-bfb9-90849fa6ff40" containerName="proxy-httpd" Dec 03 12:15:19 crc kubenswrapper[5052]: I1203 12:15:19.482125 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="1661efff-d6ce-479d-bfb9-90849fa6ff40" containerName="proxy-httpd" Dec 03 12:15:19 crc kubenswrapper[5052]: E1203 12:15:19.482133 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="85f5676f-32e8-4c3d-8729-e1eb2fdc8dc9" containerName="galera" Dec 03 12:15:19 crc kubenswrapper[5052]: I1203 12:15:19.482138 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="85f5676f-32e8-4c3d-8729-e1eb2fdc8dc9" containerName="galera" Dec 03 12:15:19 crc kubenswrapper[5052]: E1203 12:15:19.482147 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="00a9935f-ef2d-4976-97a0-78ee6325433d" containerName="mysql-bootstrap" Dec 03 12:15:19 crc kubenswrapper[5052]: I1203 12:15:19.482152 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="00a9935f-ef2d-4976-97a0-78ee6325433d" containerName="mysql-bootstrap" Dec 03 12:15:19 crc kubenswrapper[5052]: E1203 12:15:19.482159 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="6099fea1-bbb4-4413-a826-539b3dfc3eec" containerName="mariadb-account-delete" Dec 03 12:15:19 crc kubenswrapper[5052]: I1203 12:15:19.482165 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="6099fea1-bbb4-4413-a826-539b3dfc3eec" containerName="mariadb-account-delete" Dec 03 12:15:19 crc kubenswrapper[5052]: E1203 12:15:19.482176 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="9a25d0bb-42c4-4214-b81d-23b29f335155" containerName="init" Dec 03 12:15:19 crc kubenswrapper[5052]: I1203 12:15:19.482181 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="9a25d0bb-42c4-4214-b81d-23b29f335155" containerName="init" Dec 03 12:15:19 crc kubenswrapper[5052]: E1203 12:15:19.482188 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="1661efff-d6ce-479d-bfb9-90849fa6ff40" containerName="sg-core" Dec 03 12:15:19 crc kubenswrapper[5052]: I1203 12:15:19.482195 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="1661efff-d6ce-479d-bfb9-90849fa6ff40" containerName="sg-core" Dec 03 12:15:19 crc kubenswrapper[5052]: E1203 12:15:19.482206 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="296a2858-22b8-41a4-91ac-a9bf055b2848" containerName="barbican-keystone-listener-log" Dec 03 12:15:19 crc kubenswrapper[5052]: I1203 12:15:19.482212 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="296a2858-22b8-41a4-91ac-a9bf055b2848" containerName="barbican-keystone-listener-log" Dec 03 12:15:19 crc kubenswrapper[5052]: E1203 12:15:19.482220 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="baa1c9d4-5887-47e3-be99-96f718178f58" containerName="openstack-network-exporter" Dec 03 12:15:19 crc kubenswrapper[5052]: I1203 12:15:19.482227 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="baa1c9d4-5887-47e3-be99-96f718178f58" containerName="openstack-network-exporter" Dec 03 12:15:19 crc kubenswrapper[5052]: E1203 12:15:19.482237 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="89b056a7-e448-447d-be7d-b69f3d4862a1" containerName="probe" Dec 03 12:15:19 crc kubenswrapper[5052]: I1203 12:15:19.482242 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="89b056a7-e448-447d-be7d-b69f3d4862a1" containerName="probe" Dec 03 12:15:19 crc kubenswrapper[5052]: E1203 12:15:19.482250 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="00ab3478-6308-427a-8e34-4133611a90b2" containerName="ovn-northd" Dec 03 12:15:19 crc kubenswrapper[5052]: I1203 12:15:19.482255 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="00ab3478-6308-427a-8e34-4133611a90b2" containerName="ovn-northd" Dec 03 12:15:19 crc kubenswrapper[5052]: E1203 12:15:19.482264 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="2a349843-a7b7-4dff-a5ae-f513337c5809" containerName="glance-log" Dec 03 12:15:19 crc kubenswrapper[5052]: I1203 12:15:19.482270 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="2a349843-a7b7-4dff-a5ae-f513337c5809" containerName="glance-log" Dec 03 12:15:19 crc kubenswrapper[5052]: E1203 12:15:19.482280 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="fee7d732-4fa3-40d5-8fff-cb88272db0ce" containerName="mariadb-account-delete" Dec 03 12:15:19 crc kubenswrapper[5052]: I1203 12:15:19.482286 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="fee7d732-4fa3-40d5-8fff-cb88272db0ce" containerName="mariadb-account-delete" Dec 03 12:15:19 crc kubenswrapper[5052]: E1203 12:15:19.482295 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="388bc903-097f-4f97-8f19-2f7bcdb49cc5" containerName="cinder-api" Dec 03 12:15:19 crc kubenswrapper[5052]: I1203 12:15:19.482301 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="388bc903-097f-4f97-8f19-2f7bcdb49cc5" containerName="cinder-api" Dec 03 12:15:19 crc kubenswrapper[5052]: E1203 12:15:19.482310 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="1661efff-d6ce-479d-bfb9-90849fa6ff40" containerName="ceilometer-notification-agent" Dec 03 12:15:19 crc kubenswrapper[5052]: I1203 12:15:19.482315 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="1661efff-d6ce-479d-bfb9-90849fa6ff40" containerName="ceilometer-notification-agent" Dec 03 12:15:19 crc kubenswrapper[5052]: E1203 12:15:19.482325 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="5856107a-0498-4956-beba-4390f5477ca0" containerName="mariadb-account-delete" Dec 03 12:15:19 crc kubenswrapper[5052]: I1203 12:15:19.482331 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="5856107a-0498-4956-beba-4390f5477ca0" containerName="mariadb-account-delete" Dec 03 12:15:19 crc kubenswrapper[5052]: E1203 12:15:19.482339 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="00ab3478-6308-427a-8e34-4133611a90b2" containerName="openstack-network-exporter" Dec 03 12:15:19 crc kubenswrapper[5052]: I1203 12:15:19.482345 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="00ab3478-6308-427a-8e34-4133611a90b2" containerName="openstack-network-exporter" Dec 03 12:15:19 crc kubenswrapper[5052]: E1203 12:15:19.482354 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="2a349843-a7b7-4dff-a5ae-f513337c5809" containerName="glance-httpd" Dec 03 12:15:19 crc kubenswrapper[5052]: I1203 12:15:19.482359 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="2a349843-a7b7-4dff-a5ae-f513337c5809" containerName="glance-httpd" Dec 03 12:15:19 crc kubenswrapper[5052]: E1203 12:15:19.482367 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="b25d2a0c-1e93-4b3d-965a-a7c021f9649d" containerName="barbican-api" Dec 03 12:15:19 crc kubenswrapper[5052]: I1203 12:15:19.482373 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="b25d2a0c-1e93-4b3d-965a-a7c021f9649d" containerName="barbican-api" Dec 03 12:15:19 crc kubenswrapper[5052]: E1203 12:15:19.482378 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="cd486343-0aad-4039-a23b-34c60cd1a381" containerName="nova-metadata-log" Dec 03 12:15:19 crc kubenswrapper[5052]: I1203 12:15:19.482384 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="cd486343-0aad-4039-a23b-34c60cd1a381" containerName="nova-metadata-log" Dec 03 12:15:19 crc kubenswrapper[5052]: E1203 12:15:19.482391 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="296a2858-22b8-41a4-91ac-a9bf055b2848" containerName="barbican-keystone-listener" Dec 03 12:15:19 crc kubenswrapper[5052]: I1203 12:15:19.482397 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="296a2858-22b8-41a4-91ac-a9bf055b2848" containerName="barbican-keystone-listener" Dec 03 12:15:19 crc kubenswrapper[5052]: E1203 12:15:19.482405 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="0a2de558-2a4b-46ef-9b53-cada8cd466c4" containerName="ovsdbserver-nb" Dec 03 12:15:19 crc kubenswrapper[5052]: I1203 12:15:19.482410 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a2de558-2a4b-46ef-9b53-cada8cd466c4" containerName="ovsdbserver-nb" Dec 03 12:15:19 crc kubenswrapper[5052]: I1203 12:15:19.482566 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="cb68adf1-26f2-4b14-9874-dde28826aba3" containerName="nova-api-log" Dec 03 12:15:19 crc kubenswrapper[5052]: I1203 12:15:19.482582 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="0a2de558-2a4b-46ef-9b53-cada8cd466c4" containerName="openstack-network-exporter" Dec 03 12:15:19 crc kubenswrapper[5052]: I1203 12:15:19.482591 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="1661efff-d6ce-479d-bfb9-90849fa6ff40" containerName="ceilometer-central-agent" Dec 03 12:15:19 crc kubenswrapper[5052]: I1203 12:15:19.482602 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="b25d2a0c-1e93-4b3d-965a-a7c021f9649d" containerName="barbican-api-log" Dec 03 12:15:19 crc kubenswrapper[5052]: I1203 12:15:19.482610 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="1661efff-d6ce-479d-bfb9-90849fa6ff40" containerName="sg-core" Dec 03 12:15:19 crc kubenswrapper[5052]: I1203 12:15:19.482617 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="8283de30-8976-47bd-bd09-aab53140f68d" containerName="placement-api" Dec 03 12:15:19 crc kubenswrapper[5052]: I1203 12:15:19.482626 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="89b056a7-e448-447d-be7d-b69f3d4862a1" containerName="cinder-scheduler" Dec 03 12:15:19 crc kubenswrapper[5052]: I1203 12:15:19.482636 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="89b056a7-e448-447d-be7d-b69f3d4862a1" containerName="probe" Dec 03 12:15:19 crc kubenswrapper[5052]: I1203 12:15:19.482643 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="3fc5d3e4-4943-4621-a6ed-2fa4a3e08d3f" containerName="rabbitmq" Dec 03 12:15:19 crc kubenswrapper[5052]: I1203 12:15:19.482651 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="cf3ed088-9069-40cf-9c9e-ee9bdf7761d9" containerName="nova-scheduler-scheduler" Dec 03 12:15:19 crc kubenswrapper[5052]: I1203 12:15:19.482660 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="b25d2a0c-1e93-4b3d-965a-a7c021f9649d" containerName="barbican-api" Dec 03 12:15:19 crc kubenswrapper[5052]: I1203 12:15:19.482666 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="b28a125a-7966-4f97-91c1-fa52d6062405" containerName="memcached" Dec 03 12:15:19 crc kubenswrapper[5052]: I1203 12:15:19.482674 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="14774d58-37e8-4e65-b0b0-d355c0070bca" containerName="nova-cell0-conductor-conductor" Dec 03 12:15:19 crc kubenswrapper[5052]: I1203 12:15:19.482682 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="a15b6144-952c-4d41-a141-5bca5c7fd402" containerName="nova-cell1-novncproxy-novncproxy" Dec 03 12:15:19 crc kubenswrapper[5052]: I1203 12:15:19.482691 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="296a2858-22b8-41a4-91ac-a9bf055b2848" containerName="barbican-keystone-listener" Dec 03 12:15:19 crc kubenswrapper[5052]: I1203 12:15:19.482696 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="2a349843-a7b7-4dff-a5ae-f513337c5809" containerName="glance-log" Dec 03 12:15:19 crc kubenswrapper[5052]: I1203 12:15:19.482704 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="1661efff-d6ce-479d-bfb9-90849fa6ff40" containerName="proxy-httpd" Dec 03 12:15:19 crc kubenswrapper[5052]: I1203 12:15:19.482713 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="5856107a-0498-4956-beba-4390f5477ca0" containerName="mariadb-account-delete" Dec 03 12:15:19 crc kubenswrapper[5052]: I1203 12:15:19.482722 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="1c98e0a9-267b-46b6-be02-fbd163462940" containerName="barbican-worker-log" Dec 03 12:15:19 crc kubenswrapper[5052]: I1203 12:15:19.482730 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="388bc903-097f-4f97-8f19-2f7bcdb49cc5" containerName="cinder-api-log" Dec 03 12:15:19 crc kubenswrapper[5052]: I1203 12:15:19.482736 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="d81b09dc-75a1-45f8-ace4-786531ad3bfd" containerName="glance-log" Dec 03 12:15:19 crc kubenswrapper[5052]: I1203 12:15:19.482746 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="ae6940a4-0f4c-46ab-8218-56bf5c5d3084" containerName="proxy-httpd" Dec 03 12:15:19 crc kubenswrapper[5052]: I1203 12:15:19.482752 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="fee7d732-4fa3-40d5-8fff-cb88272db0ce" containerName="mariadb-account-delete" Dec 03 12:15:19 crc kubenswrapper[5052]: I1203 12:15:19.482761 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="9a25d0bb-42c4-4214-b81d-23b29f335155" containerName="dnsmasq-dns" Dec 03 12:15:19 crc kubenswrapper[5052]: I1203 12:15:19.482770 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="0a2de558-2a4b-46ef-9b53-cada8cd466c4" containerName="ovsdbserver-nb" Dec 03 12:15:19 crc kubenswrapper[5052]: I1203 12:15:19.482776 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="00ab3478-6308-427a-8e34-4133611a90b2" containerName="ovn-northd" Dec 03 12:15:19 crc kubenswrapper[5052]: I1203 12:15:19.482785 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="1661efff-d6ce-479d-bfb9-90849fa6ff40" containerName="ceilometer-notification-agent" Dec 03 12:15:19 crc kubenswrapper[5052]: I1203 12:15:19.482792 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="2a349843-a7b7-4dff-a5ae-f513337c5809" containerName="glance-httpd" Dec 03 12:15:19 crc kubenswrapper[5052]: I1203 12:15:19.482798 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="c2a65caa-2eec-48af-bdad-8fc19d827867" containerName="nova-cell1-conductor-conductor" Dec 03 12:15:19 crc kubenswrapper[5052]: I1203 12:15:19.482806 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="cd486343-0aad-4039-a23b-34c60cd1a381" containerName="nova-metadata-log" Dec 03 12:15:19 crc kubenswrapper[5052]: I1203 12:15:19.482814 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="cb68adf1-26f2-4b14-9874-dde28826aba3" containerName="nova-api-api" Dec 03 12:15:19 crc kubenswrapper[5052]: I1203 12:15:19.482823 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="296a2858-22b8-41a4-91ac-a9bf055b2848" containerName="barbican-keystone-listener-log" Dec 03 12:15:19 crc kubenswrapper[5052]: I1203 12:15:19.482829 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="00ab3478-6308-427a-8e34-4133611a90b2" containerName="openstack-network-exporter" Dec 03 12:15:19 crc kubenswrapper[5052]: I1203 12:15:19.482835 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="dfdd8013-ee6e-4ec6-b5d1-fb423f444430" containerName="kube-state-metrics" Dec 03 12:15:19 crc kubenswrapper[5052]: I1203 12:15:19.482844 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="f1619fbf-df68-484f-99d4-09b81a7a334e" containerName="mariadb-account-delete" Dec 03 12:15:19 crc kubenswrapper[5052]: I1203 12:15:19.482851 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="1c98e0a9-267b-46b6-be02-fbd163462940" containerName="barbican-worker" Dec 03 12:15:19 crc kubenswrapper[5052]: I1203 12:15:19.482860 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="e63eb1eb-e0b9-46ea-a19b-8f5c018728ff" containerName="rabbitmq" Dec 03 12:15:19 crc kubenswrapper[5052]: I1203 12:15:19.482867 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="cd486343-0aad-4039-a23b-34c60cd1a381" containerName="nova-metadata-metadata" Dec 03 12:15:19 crc kubenswrapper[5052]: I1203 12:15:19.482873 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="e14dd2e3-f6e1-450c-a688-1dc4eb2ee622" containerName="keystone-api" Dec 03 12:15:19 crc kubenswrapper[5052]: I1203 12:15:19.482882 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="388bc903-097f-4f97-8f19-2f7bcdb49cc5" containerName="cinder-api" Dec 03 12:15:19 crc kubenswrapper[5052]: I1203 12:15:19.482891 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="baa1c9d4-5887-47e3-be99-96f718178f58" containerName="openstack-network-exporter" Dec 03 12:15:19 crc kubenswrapper[5052]: I1203 12:15:19.482901 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="00a9935f-ef2d-4976-97a0-78ee6325433d" containerName="galera" Dec 03 12:15:19 crc kubenswrapper[5052]: I1203 12:15:19.482912 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="baa1c9d4-5887-47e3-be99-96f718178f58" containerName="ovsdbserver-sb" Dec 03 12:15:19 crc kubenswrapper[5052]: I1203 12:15:19.482917 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="8283de30-8976-47bd-bd09-aab53140f68d" containerName="placement-log" Dec 03 12:15:19 crc kubenswrapper[5052]: I1203 12:15:19.482923 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="d81b09dc-75a1-45f8-ace4-786531ad3bfd" containerName="glance-httpd" Dec 03 12:15:19 crc kubenswrapper[5052]: I1203 12:15:19.482933 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="6099fea1-bbb4-4413-a826-539b3dfc3eec" containerName="mariadb-account-delete" Dec 03 12:15:19 crc kubenswrapper[5052]: I1203 12:15:19.482944 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="ae6940a4-0f4c-46ab-8218-56bf5c5d3084" containerName="proxy-server" Dec 03 12:15:19 crc kubenswrapper[5052]: I1203 12:15:19.482950 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="85f5676f-32e8-4c3d-8729-e1eb2fdc8dc9" containerName="galera" Dec 03 12:15:19 crc kubenswrapper[5052]: I1203 12:15:19.485252 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/redhat-operators-fhsfk" Dec 03 12:15:19 crc kubenswrapper[5052]: I1203 12:15:19.494659 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/redhat-operators-fhsfk"] Dec 03 12:15:19 crc kubenswrapper[5052]: I1203 12:15:19.653463 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d9nkc\" (UniqueName: \"kubernetes.io/projected/7f722b89-0047-4c97-b76d-3c4faf580781-kube-api-access-d9nkc\") pod \"redhat-operators-fhsfk\" (UID: \"7f722b89-0047-4c97-b76d-3c4faf580781\") " pod="openshift-marketplace/redhat-operators-fhsfk" Dec 03 12:15:19 crc kubenswrapper[5052]: I1203 12:15:19.653539 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/7f722b89-0047-4c97-b76d-3c4faf580781-utilities\") pod \"redhat-operators-fhsfk\" (UID: \"7f722b89-0047-4c97-b76d-3c4faf580781\") " pod="openshift-marketplace/redhat-operators-fhsfk" Dec 03 12:15:19 crc kubenswrapper[5052]: I1203 12:15:19.653617 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/7f722b89-0047-4c97-b76d-3c4faf580781-catalog-content\") pod \"redhat-operators-fhsfk\" (UID: \"7f722b89-0047-4c97-b76d-3c4faf580781\") " pod="openshift-marketplace/redhat-operators-fhsfk" Dec 03 12:15:19 crc kubenswrapper[5052]: I1203 12:15:19.687312 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9a25d0bb-42c4-4214-b81d-23b29f335155" path="/var/lib/kubelet/pods/9a25d0bb-42c4-4214-b81d-23b29f335155/volumes" Dec 03 12:15:19 crc kubenswrapper[5052]: I1203 12:15:19.754642 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-d9nkc\" (UniqueName: \"kubernetes.io/projected/7f722b89-0047-4c97-b76d-3c4faf580781-kube-api-access-d9nkc\") pod \"redhat-operators-fhsfk\" (UID: \"7f722b89-0047-4c97-b76d-3c4faf580781\") " pod="openshift-marketplace/redhat-operators-fhsfk" Dec 03 12:15:19 crc kubenswrapper[5052]: I1203 12:15:19.754695 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/7f722b89-0047-4c97-b76d-3c4faf580781-utilities\") pod \"redhat-operators-fhsfk\" (UID: \"7f722b89-0047-4c97-b76d-3c4faf580781\") " pod="openshift-marketplace/redhat-operators-fhsfk" Dec 03 12:15:19 crc kubenswrapper[5052]: I1203 12:15:19.754745 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/7f722b89-0047-4c97-b76d-3c4faf580781-catalog-content\") pod \"redhat-operators-fhsfk\" (UID: \"7f722b89-0047-4c97-b76d-3c4faf580781\") " pod="openshift-marketplace/redhat-operators-fhsfk" Dec 03 12:15:19 crc kubenswrapper[5052]: I1203 12:15:19.755234 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/7f722b89-0047-4c97-b76d-3c4faf580781-utilities\") pod \"redhat-operators-fhsfk\" (UID: \"7f722b89-0047-4c97-b76d-3c4faf580781\") " pod="openshift-marketplace/redhat-operators-fhsfk" Dec 03 12:15:19 crc kubenswrapper[5052]: I1203 12:15:19.755290 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/7f722b89-0047-4c97-b76d-3c4faf580781-catalog-content\") pod \"redhat-operators-fhsfk\" (UID: \"7f722b89-0047-4c97-b76d-3c4faf580781\") " pod="openshift-marketplace/redhat-operators-fhsfk" Dec 03 12:15:19 crc kubenswrapper[5052]: I1203 12:15:19.774922 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-d9nkc\" (UniqueName: \"kubernetes.io/projected/7f722b89-0047-4c97-b76d-3c4faf580781-kube-api-access-d9nkc\") pod \"redhat-operators-fhsfk\" (UID: \"7f722b89-0047-4c97-b76d-3c4faf580781\") " pod="openshift-marketplace/redhat-operators-fhsfk" Dec 03 12:15:19 crc kubenswrapper[5052]: I1203 12:15:19.824612 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/redhat-operators-fhsfk" Dec 03 12:15:20 crc kubenswrapper[5052]: E1203 12:15:20.164640 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:15:20 crc kubenswrapper[5052]: E1203 12:15:20.164670 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:15:20 crc kubenswrapper[5052]: E1203 12:15:20.164730 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/679f17b9-0e40-480b-bfde-4f63899cb972-operator-scripts podName:679f17b9-0e40-480b-bfde-4f63899cb972 nodeName:}" failed. No retries permitted until 2025-12-03 12:15:28.16471045 +0000 UTC m=+3318.752952214 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/679f17b9-0e40-480b-bfde-4f63899cb972-operator-scripts") pod "cinder7878-account-delete-jwtvt" (UID: "679f17b9-0e40-480b-bfde-4f63899cb972") : configmap "openstack-scripts" not found Dec 03 12:15:20 crc kubenswrapper[5052]: E1203 12:15:20.164748 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c200c246-74d3-48dc-8b24-ac4b37611069-operator-scripts podName:c200c246-74d3-48dc-8b24-ac4b37611069 nodeName:}" failed. No retries permitted until 2025-12-03 12:15:28.164741971 +0000 UTC m=+3318.752983735 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/c200c246-74d3-48dc-8b24-ac4b37611069-operator-scripts") pod "novacell00bb5-account-delete-r2csc" (UID: "c200c246-74d3-48dc-8b24-ac4b37611069") : configmap "openstack-scripts" not found Dec 03 12:15:20 crc kubenswrapper[5052]: I1203 12:15:20.281045 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/redhat-operators-fhsfk"] Dec 03 12:15:20 crc kubenswrapper[5052]: I1203 12:15:20.763421 5052 generic.go:334] "Generic (PLEG): container finished" podID="7f722b89-0047-4c97-b76d-3c4faf580781" containerID="cf77d2f5cf7b52bef64e5e08cef6c48c755a0d1468841ee799e02c7d4a4903c5" exitCode=0 Dec 03 12:15:20 crc kubenswrapper[5052]: I1203 12:15:20.763479 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-operators-fhsfk" event={"ID":"7f722b89-0047-4c97-b76d-3c4faf580781","Type":"ContainerDied","Data":"cf77d2f5cf7b52bef64e5e08cef6c48c755a0d1468841ee799e02c7d4a4903c5"} Dec 03 12:15:20 crc kubenswrapper[5052]: I1203 12:15:20.764866 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-operators-fhsfk" event={"ID":"7f722b89-0047-4c97-b76d-3c4faf580781","Type":"ContainerStarted","Data":"6700e86c169ae13ba41a127f46b3899fa650b29b405069f391982dad54559249"} Dec 03 12:15:20 crc kubenswrapper[5052]: I1203 12:15:20.765002 5052 provider.go:102] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Dec 03 12:15:22 crc kubenswrapper[5052]: I1203 12:15:22.785053 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-operators-fhsfk" event={"ID":"7f722b89-0047-4c97-b76d-3c4faf580781","Type":"ContainerStarted","Data":"7d0d347efadded966fa83aef1520f94181fea705c74c98e60900fc649aeb4650"} Dec 03 12:15:23 crc kubenswrapper[5052]: I1203 12:15:23.795185 5052 generic.go:334] "Generic (PLEG): container finished" podID="7f722b89-0047-4c97-b76d-3c4faf580781" containerID="7d0d347efadded966fa83aef1520f94181fea705c74c98e60900fc649aeb4650" exitCode=0 Dec 03 12:15:23 crc kubenswrapper[5052]: I1203 12:15:23.795233 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-operators-fhsfk" event={"ID":"7f722b89-0047-4c97-b76d-3c4faf580781","Type":"ContainerDied","Data":"7d0d347efadded966fa83aef1520f94181fea705c74c98e60900fc649aeb4650"} Dec 03 12:15:25 crc kubenswrapper[5052]: I1203 12:15:25.282229 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/neutron-5b979cdcb6-6hqqv" podUID="718e2899-d5c3-41a4-a2f9-4830e18355d3" containerName="neutron-httpd" probeResult="failure" output="Get \"https://10.217.1.8:9696/\": dial tcp 10.217.1.8:9696: connect: connection refused" Dec 03 12:15:25 crc kubenswrapper[5052]: I1203 12:15:25.816316 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-operators-fhsfk" event={"ID":"7f722b89-0047-4c97-b76d-3c4faf580781","Type":"ContainerStarted","Data":"9d355688d2c7284f0eb0a0fd9c2f7685a270b9b26ad7915d57029c84f3f526ac"} Dec 03 12:15:25 crc kubenswrapper[5052]: I1203 12:15:25.840326 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-marketplace/redhat-operators-fhsfk" podStartSLOduration=2.100996705 podStartE2EDuration="6.840305946s" podCreationTimestamp="2025-12-03 12:15:19 +0000 UTC" firstStartedPulling="2025-12-03 12:15:20.764775354 +0000 UTC m=+3311.353017118" lastFinishedPulling="2025-12-03 12:15:25.504084595 +0000 UTC m=+3316.092326359" observedRunningTime="2025-12-03 12:15:25.834008816 +0000 UTC m=+3316.422250580" watchObservedRunningTime="2025-12-03 12:15:25.840305946 +0000 UTC m=+3316.428547710" Dec 03 12:15:28 crc kubenswrapper[5052]: E1203 12:15:28.208664 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:15:28 crc kubenswrapper[5052]: E1203 12:15:28.208694 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:15:28 crc kubenswrapper[5052]: E1203 12:15:28.208825 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/679f17b9-0e40-480b-bfde-4f63899cb972-operator-scripts podName:679f17b9-0e40-480b-bfde-4f63899cb972 nodeName:}" failed. No retries permitted until 2025-12-03 12:15:44.208810063 +0000 UTC m=+3334.797051827 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/679f17b9-0e40-480b-bfde-4f63899cb972-operator-scripts") pod "cinder7878-account-delete-jwtvt" (UID: "679f17b9-0e40-480b-bfde-4f63899cb972") : configmap "openstack-scripts" not found Dec 03 12:15:28 crc kubenswrapper[5052]: E1203 12:15:28.209260 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c200c246-74d3-48dc-8b24-ac4b37611069-operator-scripts podName:c200c246-74d3-48dc-8b24-ac4b37611069 nodeName:}" failed. No retries permitted until 2025-12-03 12:15:44.209250906 +0000 UTC m=+3334.797492660 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/c200c246-74d3-48dc-8b24-ac4b37611069-operator-scripts") pod "novacell00bb5-account-delete-r2csc" (UID: "c200c246-74d3-48dc-8b24-ac4b37611069") : configmap "openstack-scripts" not found Dec 03 12:15:29 crc kubenswrapper[5052]: I1203 12:15:29.825027 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-marketplace/redhat-operators-fhsfk" Dec 03 12:15:29 crc kubenswrapper[5052]: I1203 12:15:29.825087 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openshift-marketplace/redhat-operators-fhsfk" Dec 03 12:15:29 crc kubenswrapper[5052]: I1203 12:15:29.882423 5052 generic.go:334] "Generic (PLEG): container finished" podID="718e2899-d5c3-41a4-a2f9-4830e18355d3" containerID="8b2970b007db09e4b4c0137ca8209b26be8338c0d78012060389d6232ffbb06a" exitCode=0 Dec 03 12:15:29 crc kubenswrapper[5052]: I1203 12:15:29.882473 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-5b979cdcb6-6hqqv" event={"ID":"718e2899-d5c3-41a4-a2f9-4830e18355d3","Type":"ContainerDied","Data":"8b2970b007db09e4b4c0137ca8209b26be8338c0d78012060389d6232ffbb06a"} Dec 03 12:15:30 crc kubenswrapper[5052]: I1203 12:15:30.822816 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-5b979cdcb6-6hqqv" Dec 03 12:15:30 crc kubenswrapper[5052]: I1203 12:15:30.868523 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openshift-marketplace/redhat-operators-fhsfk" podUID="7f722b89-0047-4c97-b76d-3c4faf580781" containerName="registry-server" probeResult="failure" output=< Dec 03 12:15:30 crc kubenswrapper[5052]: timeout: failed to connect service ":50051" within 1s Dec 03 12:15:30 crc kubenswrapper[5052]: > Dec 03 12:15:30 crc kubenswrapper[5052]: I1203 12:15:30.895229 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-5b979cdcb6-6hqqv" event={"ID":"718e2899-d5c3-41a4-a2f9-4830e18355d3","Type":"ContainerDied","Data":"005e86831435d920e9fdea739d977c3bb3d174fae83069f9fbb6d758792f23e1"} Dec 03 12:15:30 crc kubenswrapper[5052]: I1203 12:15:30.895284 5052 scope.go:117] "RemoveContainer" containerID="84e05639993606274e28568021cb8fdaf723254416f7db2a86eb1f0830df3b64" Dec 03 12:15:30 crc kubenswrapper[5052]: I1203 12:15:30.895319 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-5b979cdcb6-6hqqv" Dec 03 12:15:30 crc kubenswrapper[5052]: I1203 12:15:30.919375 5052 scope.go:117] "RemoveContainer" containerID="8b2970b007db09e4b4c0137ca8209b26be8338c0d78012060389d6232ffbb06a" Dec 03 12:15:30 crc kubenswrapper[5052]: I1203 12:15:30.945364 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pvvtk\" (UniqueName: \"kubernetes.io/projected/718e2899-d5c3-41a4-a2f9-4830e18355d3-kube-api-access-pvvtk\") pod \"718e2899-d5c3-41a4-a2f9-4830e18355d3\" (UID: \"718e2899-d5c3-41a4-a2f9-4830e18355d3\") " Dec 03 12:15:30 crc kubenswrapper[5052]: I1203 12:15:30.945499 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/718e2899-d5c3-41a4-a2f9-4830e18355d3-combined-ca-bundle\") pod \"718e2899-d5c3-41a4-a2f9-4830e18355d3\" (UID: \"718e2899-d5c3-41a4-a2f9-4830e18355d3\") " Dec 03 12:15:30 crc kubenswrapper[5052]: I1203 12:15:30.945588 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/718e2899-d5c3-41a4-a2f9-4830e18355d3-internal-tls-certs\") pod \"718e2899-d5c3-41a4-a2f9-4830e18355d3\" (UID: \"718e2899-d5c3-41a4-a2f9-4830e18355d3\") " Dec 03 12:15:30 crc kubenswrapper[5052]: I1203 12:15:30.945620 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/718e2899-d5c3-41a4-a2f9-4830e18355d3-ovndb-tls-certs\") pod \"718e2899-d5c3-41a4-a2f9-4830e18355d3\" (UID: \"718e2899-d5c3-41a4-a2f9-4830e18355d3\") " Dec 03 12:15:30 crc kubenswrapper[5052]: I1203 12:15:30.945637 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/718e2899-d5c3-41a4-a2f9-4830e18355d3-httpd-config\") pod \"718e2899-d5c3-41a4-a2f9-4830e18355d3\" (UID: \"718e2899-d5c3-41a4-a2f9-4830e18355d3\") " Dec 03 12:15:30 crc kubenswrapper[5052]: I1203 12:15:30.945665 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/718e2899-d5c3-41a4-a2f9-4830e18355d3-config\") pod \"718e2899-d5c3-41a4-a2f9-4830e18355d3\" (UID: \"718e2899-d5c3-41a4-a2f9-4830e18355d3\") " Dec 03 12:15:30 crc kubenswrapper[5052]: I1203 12:15:30.945707 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/718e2899-d5c3-41a4-a2f9-4830e18355d3-public-tls-certs\") pod \"718e2899-d5c3-41a4-a2f9-4830e18355d3\" (UID: \"718e2899-d5c3-41a4-a2f9-4830e18355d3\") " Dec 03 12:15:30 crc kubenswrapper[5052]: I1203 12:15:30.951294 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/718e2899-d5c3-41a4-a2f9-4830e18355d3-httpd-config" (OuterVolumeSpecName: "httpd-config") pod "718e2899-d5c3-41a4-a2f9-4830e18355d3" (UID: "718e2899-d5c3-41a4-a2f9-4830e18355d3"). InnerVolumeSpecName "httpd-config". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:15:30 crc kubenswrapper[5052]: I1203 12:15:30.953945 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/718e2899-d5c3-41a4-a2f9-4830e18355d3-kube-api-access-pvvtk" (OuterVolumeSpecName: "kube-api-access-pvvtk") pod "718e2899-d5c3-41a4-a2f9-4830e18355d3" (UID: "718e2899-d5c3-41a4-a2f9-4830e18355d3"). InnerVolumeSpecName "kube-api-access-pvvtk". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:15:30 crc kubenswrapper[5052]: I1203 12:15:30.985266 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/718e2899-d5c3-41a4-a2f9-4830e18355d3-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "718e2899-d5c3-41a4-a2f9-4830e18355d3" (UID: "718e2899-d5c3-41a4-a2f9-4830e18355d3"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:15:30 crc kubenswrapper[5052]: I1203 12:15:30.987110 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/718e2899-d5c3-41a4-a2f9-4830e18355d3-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "718e2899-d5c3-41a4-a2f9-4830e18355d3" (UID: "718e2899-d5c3-41a4-a2f9-4830e18355d3"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:15:30 crc kubenswrapper[5052]: I1203 12:15:30.991900 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/718e2899-d5c3-41a4-a2f9-4830e18355d3-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "718e2899-d5c3-41a4-a2f9-4830e18355d3" (UID: "718e2899-d5c3-41a4-a2f9-4830e18355d3"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:15:31 crc kubenswrapper[5052]: I1203 12:15:31.003554 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/718e2899-d5c3-41a4-a2f9-4830e18355d3-config" (OuterVolumeSpecName: "config") pod "718e2899-d5c3-41a4-a2f9-4830e18355d3" (UID: "718e2899-d5c3-41a4-a2f9-4830e18355d3"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:15:31 crc kubenswrapper[5052]: I1203 12:15:31.033740 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/718e2899-d5c3-41a4-a2f9-4830e18355d3-ovndb-tls-certs" (OuterVolumeSpecName: "ovndb-tls-certs") pod "718e2899-d5c3-41a4-a2f9-4830e18355d3" (UID: "718e2899-d5c3-41a4-a2f9-4830e18355d3"). InnerVolumeSpecName "ovndb-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:15:31 crc kubenswrapper[5052]: I1203 12:15:31.047989 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/718e2899-d5c3-41a4-a2f9-4830e18355d3-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:31 crc kubenswrapper[5052]: I1203 12:15:31.048039 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-pvvtk\" (UniqueName: \"kubernetes.io/projected/718e2899-d5c3-41a4-a2f9-4830e18355d3-kube-api-access-pvvtk\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:31 crc kubenswrapper[5052]: I1203 12:15:31.048054 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/718e2899-d5c3-41a4-a2f9-4830e18355d3-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:31 crc kubenswrapper[5052]: I1203 12:15:31.048066 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/718e2899-d5c3-41a4-a2f9-4830e18355d3-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:31 crc kubenswrapper[5052]: I1203 12:15:31.048078 5052 reconciler_common.go:293] "Volume detached for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/718e2899-d5c3-41a4-a2f9-4830e18355d3-ovndb-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:31 crc kubenswrapper[5052]: I1203 12:15:31.048091 5052 reconciler_common.go:293] "Volume detached for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/718e2899-d5c3-41a4-a2f9-4830e18355d3-httpd-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:31 crc kubenswrapper[5052]: I1203 12:15:31.048103 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/718e2899-d5c3-41a4-a2f9-4830e18355d3-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:31 crc kubenswrapper[5052]: I1203 12:15:31.242724 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/neutron-5b979cdcb6-6hqqv"] Dec 03 12:15:31 crc kubenswrapper[5052]: I1203 12:15:31.248831 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/neutron-5b979cdcb6-6hqqv"] Dec 03 12:15:31 crc kubenswrapper[5052]: I1203 12:15:31.684959 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="718e2899-d5c3-41a4-a2f9-4830e18355d3" path="/var/lib/kubelet/pods/718e2899-d5c3-41a4-a2f9-4830e18355d3/volumes" Dec 03 12:15:37 crc kubenswrapper[5052]: I1203 12:15:37.797035 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:15:37 crc kubenswrapper[5052]: I1203 12:15:37.899546 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"cache\" (UniqueName: \"kubernetes.io/empty-dir/694f3a22-38d9-487d-b887-a915fa8f99b0-cache\") pod \"694f3a22-38d9-487d-b887-a915fa8f99b0\" (UID: \"694f3a22-38d9-487d-b887-a915fa8f99b0\") " Dec 03 12:15:37 crc kubenswrapper[5052]: I1203 12:15:37.899622 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/694f3a22-38d9-487d-b887-a915fa8f99b0-etc-swift\") pod \"694f3a22-38d9-487d-b887-a915fa8f99b0\" (UID: \"694f3a22-38d9-487d-b887-a915fa8f99b0\") " Dec 03 12:15:37 crc kubenswrapper[5052]: I1203 12:15:37.899642 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"swift\" (UniqueName: \"kubernetes.io/local-volume/local-storage09-crc\") pod \"694f3a22-38d9-487d-b887-a915fa8f99b0\" (UID: \"694f3a22-38d9-487d-b887-a915fa8f99b0\") " Dec 03 12:15:37 crc kubenswrapper[5052]: I1203 12:15:37.899670 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"lock\" (UniqueName: \"kubernetes.io/empty-dir/694f3a22-38d9-487d-b887-a915fa8f99b0-lock\") pod \"694f3a22-38d9-487d-b887-a915fa8f99b0\" (UID: \"694f3a22-38d9-487d-b887-a915fa8f99b0\") " Dec 03 12:15:37 crc kubenswrapper[5052]: I1203 12:15:37.899691 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dbgxt\" (UniqueName: \"kubernetes.io/projected/694f3a22-38d9-487d-b887-a915fa8f99b0-kube-api-access-dbgxt\") pod \"694f3a22-38d9-487d-b887-a915fa8f99b0\" (UID: \"694f3a22-38d9-487d-b887-a915fa8f99b0\") " Dec 03 12:15:37 crc kubenswrapper[5052]: I1203 12:15:37.900235 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/694f3a22-38d9-487d-b887-a915fa8f99b0-cache" (OuterVolumeSpecName: "cache") pod "694f3a22-38d9-487d-b887-a915fa8f99b0" (UID: "694f3a22-38d9-487d-b887-a915fa8f99b0"). InnerVolumeSpecName "cache". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:15:37 crc kubenswrapper[5052]: I1203 12:15:37.900387 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/694f3a22-38d9-487d-b887-a915fa8f99b0-lock" (OuterVolumeSpecName: "lock") pod "694f3a22-38d9-487d-b887-a915fa8f99b0" (UID: "694f3a22-38d9-487d-b887-a915fa8f99b0"). InnerVolumeSpecName "lock". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:15:37 crc kubenswrapper[5052]: I1203 12:15:37.904366 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/694f3a22-38d9-487d-b887-a915fa8f99b0-kube-api-access-dbgxt" (OuterVolumeSpecName: "kube-api-access-dbgxt") pod "694f3a22-38d9-487d-b887-a915fa8f99b0" (UID: "694f3a22-38d9-487d-b887-a915fa8f99b0"). InnerVolumeSpecName "kube-api-access-dbgxt". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:15:37 crc kubenswrapper[5052]: I1203 12:15:37.904823 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage09-crc" (OuterVolumeSpecName: "swift") pod "694f3a22-38d9-487d-b887-a915fa8f99b0" (UID: "694f3a22-38d9-487d-b887-a915fa8f99b0"). InnerVolumeSpecName "local-storage09-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 12:15:37 crc kubenswrapper[5052]: I1203 12:15:37.905721 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/694f3a22-38d9-487d-b887-a915fa8f99b0-etc-swift" (OuterVolumeSpecName: "etc-swift") pod "694f3a22-38d9-487d-b887-a915fa8f99b0" (UID: "694f3a22-38d9-487d-b887-a915fa8f99b0"). InnerVolumeSpecName "etc-swift". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:15:37 crc kubenswrapper[5052]: I1203 12:15:37.984849 5052 generic.go:334] "Generic (PLEG): container finished" podID="694f3a22-38d9-487d-b887-a915fa8f99b0" containerID="d7754f88aed158a69d3f255fe77907743b8120c66e854103907d2301b4d085dd" exitCode=137 Dec 03 12:15:37 crc kubenswrapper[5052]: I1203 12:15:37.984894 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"694f3a22-38d9-487d-b887-a915fa8f99b0","Type":"ContainerDied","Data":"d7754f88aed158a69d3f255fe77907743b8120c66e854103907d2301b4d085dd"} Dec 03 12:15:37 crc kubenswrapper[5052]: I1203 12:15:37.984952 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"694f3a22-38d9-487d-b887-a915fa8f99b0","Type":"ContainerDied","Data":"cade3940c863bac7871d97d10ee1b99b6179e8e161a970d2eca55f430665b17b"} Dec 03 12:15:37 crc kubenswrapper[5052]: I1203 12:15:37.984972 5052 scope.go:117] "RemoveContainer" containerID="d7754f88aed158a69d3f255fe77907743b8120c66e854103907d2301b4d085dd" Dec 03 12:15:37 crc kubenswrapper[5052]: I1203 12:15:37.985092 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:15:38 crc kubenswrapper[5052]: I1203 12:15:38.001818 5052 reconciler_common.go:293] "Volume detached for volume \"cache\" (UniqueName: \"kubernetes.io/empty-dir/694f3a22-38d9-487d-b887-a915fa8f99b0-cache\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:38 crc kubenswrapper[5052]: I1203 12:15:38.002004 5052 reconciler_common.go:293] "Volume detached for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/694f3a22-38d9-487d-b887-a915fa8f99b0-etc-swift\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:38 crc kubenswrapper[5052]: I1203 12:15:38.002080 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage09-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage09-crc\") on node \"crc\" " Dec 03 12:15:38 crc kubenswrapper[5052]: I1203 12:15:38.002180 5052 reconciler_common.go:293] "Volume detached for volume \"lock\" (UniqueName: \"kubernetes.io/empty-dir/694f3a22-38d9-487d-b887-a915fa8f99b0-lock\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:38 crc kubenswrapper[5052]: I1203 12:15:38.002237 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-dbgxt\" (UniqueName: \"kubernetes.io/projected/694f3a22-38d9-487d-b887-a915fa8f99b0-kube-api-access-dbgxt\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:38 crc kubenswrapper[5052]: I1203 12:15:38.023677 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage09-crc" (UniqueName: "kubernetes.io/local-volume/local-storage09-crc") on node "crc" Dec 03 12:15:38 crc kubenswrapper[5052]: I1203 12:15:38.029290 5052 scope.go:117] "RemoveContainer" containerID="4ca9b25da15ecf1176a45c4f622d417575fd72f1258e4941eaab46adc37d9954" Dec 03 12:15:38 crc kubenswrapper[5052]: I1203 12:15:38.030440 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/swift-storage-0"] Dec 03 12:15:38 crc kubenswrapper[5052]: I1203 12:15:38.036895 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/swift-storage-0"] Dec 03 12:15:38 crc kubenswrapper[5052]: I1203 12:15:38.046756 5052 scope.go:117] "RemoveContainer" containerID="c8a4d6f709c57c2c8c86b3ead5f40b5eed433a688c9bda706475d1a83dc38289" Dec 03 12:15:38 crc kubenswrapper[5052]: I1203 12:15:38.064231 5052 scope.go:117] "RemoveContainer" containerID="2b4fde6dfdf88033dd51e40f592f3092bbadafedc12b51e3dac6d06c7be816e2" Dec 03 12:15:38 crc kubenswrapper[5052]: I1203 12:15:38.082166 5052 scope.go:117] "RemoveContainer" containerID="d34ac8e54a2a8c78fedd6aae86ea4f7c01fa3fb75b7877784aed4c550716ec17" Dec 03 12:15:38 crc kubenswrapper[5052]: I1203 12:15:38.099401 5052 scope.go:117] "RemoveContainer" containerID="edccdb7e982ee726341d6181308a200c1c7f7e489199aae3ff7897c6686d1e53" Dec 03 12:15:38 crc kubenswrapper[5052]: I1203 12:15:38.103538 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage09-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage09-crc\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:38 crc kubenswrapper[5052]: I1203 12:15:38.119175 5052 scope.go:117] "RemoveContainer" containerID="bfc910b016024c52ad21685e4f00e1ebf323cb412cbdac44e180ce0af3b5c974" Dec 03 12:15:38 crc kubenswrapper[5052]: I1203 12:15:38.145159 5052 scope.go:117] "RemoveContainer" containerID="3577cb2e84ebe92b200c0bf67af07582c963a380e9d15f90356b2162da1491e3" Dec 03 12:15:38 crc kubenswrapper[5052]: I1203 12:15:38.165594 5052 scope.go:117] "RemoveContainer" containerID="a9d69281c8f09f62933866859d2a7f994441cb27231474ecb5c4087f61ef9e48" Dec 03 12:15:38 crc kubenswrapper[5052]: I1203 12:15:38.183003 5052 scope.go:117] "RemoveContainer" containerID="4be1f7ec0e5ec2598abc5fccb1e3a336e6d2ce904a40e4470bfebd9a2432aa0d" Dec 03 12:15:38 crc kubenswrapper[5052]: I1203 12:15:38.201711 5052 scope.go:117] "RemoveContainer" containerID="ffacc5bf5e6a05d80223ac48d049c379298b3d4840c2186fb767cc10e1492580" Dec 03 12:15:38 crc kubenswrapper[5052]: I1203 12:15:38.219151 5052 scope.go:117] "RemoveContainer" containerID="9abaf8aac6af7b9078e0f7fa87225ebcfdfa4003e45df36fe94117f331cca24b" Dec 03 12:15:38 crc kubenswrapper[5052]: I1203 12:15:38.237226 5052 scope.go:117] "RemoveContainer" containerID="8de12c6937f84a484a57da6623c0b720805ae449cf86cbf7781810fff5f7dbd6" Dec 03 12:15:38 crc kubenswrapper[5052]: I1203 12:15:38.256752 5052 scope.go:117] "RemoveContainer" containerID="daff85236f8b1825cad56f8cf5a450330943257c15e8a198c3cbd529f86943a9" Dec 03 12:15:38 crc kubenswrapper[5052]: I1203 12:15:38.275801 5052 scope.go:117] "RemoveContainer" containerID="52c9d3753f1d83dc61693ea154f30426c527a127d3d6f00ae6379a1cbd9e5899" Dec 03 12:15:38 crc kubenswrapper[5052]: I1203 12:15:38.295631 5052 scope.go:117] "RemoveContainer" containerID="d7754f88aed158a69d3f255fe77907743b8120c66e854103907d2301b4d085dd" Dec 03 12:15:38 crc kubenswrapper[5052]: E1203 12:15:38.296038 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d7754f88aed158a69d3f255fe77907743b8120c66e854103907d2301b4d085dd\": container with ID starting with d7754f88aed158a69d3f255fe77907743b8120c66e854103907d2301b4d085dd not found: ID does not exist" containerID="d7754f88aed158a69d3f255fe77907743b8120c66e854103907d2301b4d085dd" Dec 03 12:15:38 crc kubenswrapper[5052]: I1203 12:15:38.296078 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d7754f88aed158a69d3f255fe77907743b8120c66e854103907d2301b4d085dd"} err="failed to get container status \"d7754f88aed158a69d3f255fe77907743b8120c66e854103907d2301b4d085dd\": rpc error: code = NotFound desc = could not find container \"d7754f88aed158a69d3f255fe77907743b8120c66e854103907d2301b4d085dd\": container with ID starting with d7754f88aed158a69d3f255fe77907743b8120c66e854103907d2301b4d085dd not found: ID does not exist" Dec 03 12:15:38 crc kubenswrapper[5052]: I1203 12:15:38.296106 5052 scope.go:117] "RemoveContainer" containerID="4ca9b25da15ecf1176a45c4f622d417575fd72f1258e4941eaab46adc37d9954" Dec 03 12:15:38 crc kubenswrapper[5052]: E1203 12:15:38.296395 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4ca9b25da15ecf1176a45c4f622d417575fd72f1258e4941eaab46adc37d9954\": container with ID starting with 4ca9b25da15ecf1176a45c4f622d417575fd72f1258e4941eaab46adc37d9954 not found: ID does not exist" containerID="4ca9b25da15ecf1176a45c4f622d417575fd72f1258e4941eaab46adc37d9954" Dec 03 12:15:38 crc kubenswrapper[5052]: I1203 12:15:38.296452 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4ca9b25da15ecf1176a45c4f622d417575fd72f1258e4941eaab46adc37d9954"} err="failed to get container status \"4ca9b25da15ecf1176a45c4f622d417575fd72f1258e4941eaab46adc37d9954\": rpc error: code = NotFound desc = could not find container \"4ca9b25da15ecf1176a45c4f622d417575fd72f1258e4941eaab46adc37d9954\": container with ID starting with 4ca9b25da15ecf1176a45c4f622d417575fd72f1258e4941eaab46adc37d9954 not found: ID does not exist" Dec 03 12:15:38 crc kubenswrapper[5052]: I1203 12:15:38.296487 5052 scope.go:117] "RemoveContainer" containerID="c8a4d6f709c57c2c8c86b3ead5f40b5eed433a688c9bda706475d1a83dc38289" Dec 03 12:15:38 crc kubenswrapper[5052]: E1203 12:15:38.296996 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c8a4d6f709c57c2c8c86b3ead5f40b5eed433a688c9bda706475d1a83dc38289\": container with ID starting with c8a4d6f709c57c2c8c86b3ead5f40b5eed433a688c9bda706475d1a83dc38289 not found: ID does not exist" containerID="c8a4d6f709c57c2c8c86b3ead5f40b5eed433a688c9bda706475d1a83dc38289" Dec 03 12:15:38 crc kubenswrapper[5052]: I1203 12:15:38.297031 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c8a4d6f709c57c2c8c86b3ead5f40b5eed433a688c9bda706475d1a83dc38289"} err="failed to get container status \"c8a4d6f709c57c2c8c86b3ead5f40b5eed433a688c9bda706475d1a83dc38289\": rpc error: code = NotFound desc = could not find container \"c8a4d6f709c57c2c8c86b3ead5f40b5eed433a688c9bda706475d1a83dc38289\": container with ID starting with c8a4d6f709c57c2c8c86b3ead5f40b5eed433a688c9bda706475d1a83dc38289 not found: ID does not exist" Dec 03 12:15:38 crc kubenswrapper[5052]: I1203 12:15:38.297049 5052 scope.go:117] "RemoveContainer" containerID="2b4fde6dfdf88033dd51e40f592f3092bbadafedc12b51e3dac6d06c7be816e2" Dec 03 12:15:38 crc kubenswrapper[5052]: E1203 12:15:38.297307 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2b4fde6dfdf88033dd51e40f592f3092bbadafedc12b51e3dac6d06c7be816e2\": container with ID starting with 2b4fde6dfdf88033dd51e40f592f3092bbadafedc12b51e3dac6d06c7be816e2 not found: ID does not exist" containerID="2b4fde6dfdf88033dd51e40f592f3092bbadafedc12b51e3dac6d06c7be816e2" Dec 03 12:15:38 crc kubenswrapper[5052]: I1203 12:15:38.297361 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2b4fde6dfdf88033dd51e40f592f3092bbadafedc12b51e3dac6d06c7be816e2"} err="failed to get container status \"2b4fde6dfdf88033dd51e40f592f3092bbadafedc12b51e3dac6d06c7be816e2\": rpc error: code = NotFound desc = could not find container \"2b4fde6dfdf88033dd51e40f592f3092bbadafedc12b51e3dac6d06c7be816e2\": container with ID starting with 2b4fde6dfdf88033dd51e40f592f3092bbadafedc12b51e3dac6d06c7be816e2 not found: ID does not exist" Dec 03 12:15:38 crc kubenswrapper[5052]: I1203 12:15:38.297378 5052 scope.go:117] "RemoveContainer" containerID="d34ac8e54a2a8c78fedd6aae86ea4f7c01fa3fb75b7877784aed4c550716ec17" Dec 03 12:15:38 crc kubenswrapper[5052]: E1203 12:15:38.297703 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d34ac8e54a2a8c78fedd6aae86ea4f7c01fa3fb75b7877784aed4c550716ec17\": container with ID starting with d34ac8e54a2a8c78fedd6aae86ea4f7c01fa3fb75b7877784aed4c550716ec17 not found: ID does not exist" containerID="d34ac8e54a2a8c78fedd6aae86ea4f7c01fa3fb75b7877784aed4c550716ec17" Dec 03 12:15:38 crc kubenswrapper[5052]: I1203 12:15:38.297722 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d34ac8e54a2a8c78fedd6aae86ea4f7c01fa3fb75b7877784aed4c550716ec17"} err="failed to get container status \"d34ac8e54a2a8c78fedd6aae86ea4f7c01fa3fb75b7877784aed4c550716ec17\": rpc error: code = NotFound desc = could not find container \"d34ac8e54a2a8c78fedd6aae86ea4f7c01fa3fb75b7877784aed4c550716ec17\": container with ID starting with d34ac8e54a2a8c78fedd6aae86ea4f7c01fa3fb75b7877784aed4c550716ec17 not found: ID does not exist" Dec 03 12:15:38 crc kubenswrapper[5052]: I1203 12:15:38.297734 5052 scope.go:117] "RemoveContainer" containerID="edccdb7e982ee726341d6181308a200c1c7f7e489199aae3ff7897c6686d1e53" Dec 03 12:15:38 crc kubenswrapper[5052]: E1203 12:15:38.298054 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"edccdb7e982ee726341d6181308a200c1c7f7e489199aae3ff7897c6686d1e53\": container with ID starting with edccdb7e982ee726341d6181308a200c1c7f7e489199aae3ff7897c6686d1e53 not found: ID does not exist" containerID="edccdb7e982ee726341d6181308a200c1c7f7e489199aae3ff7897c6686d1e53" Dec 03 12:15:38 crc kubenswrapper[5052]: I1203 12:15:38.298088 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"edccdb7e982ee726341d6181308a200c1c7f7e489199aae3ff7897c6686d1e53"} err="failed to get container status \"edccdb7e982ee726341d6181308a200c1c7f7e489199aae3ff7897c6686d1e53\": rpc error: code = NotFound desc = could not find container \"edccdb7e982ee726341d6181308a200c1c7f7e489199aae3ff7897c6686d1e53\": container with ID starting with edccdb7e982ee726341d6181308a200c1c7f7e489199aae3ff7897c6686d1e53 not found: ID does not exist" Dec 03 12:15:38 crc kubenswrapper[5052]: I1203 12:15:38.298107 5052 scope.go:117] "RemoveContainer" containerID="bfc910b016024c52ad21685e4f00e1ebf323cb412cbdac44e180ce0af3b5c974" Dec 03 12:15:38 crc kubenswrapper[5052]: E1203 12:15:38.298395 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bfc910b016024c52ad21685e4f00e1ebf323cb412cbdac44e180ce0af3b5c974\": container with ID starting with bfc910b016024c52ad21685e4f00e1ebf323cb412cbdac44e180ce0af3b5c974 not found: ID does not exist" containerID="bfc910b016024c52ad21685e4f00e1ebf323cb412cbdac44e180ce0af3b5c974" Dec 03 12:15:38 crc kubenswrapper[5052]: I1203 12:15:38.298424 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bfc910b016024c52ad21685e4f00e1ebf323cb412cbdac44e180ce0af3b5c974"} err="failed to get container status \"bfc910b016024c52ad21685e4f00e1ebf323cb412cbdac44e180ce0af3b5c974\": rpc error: code = NotFound desc = could not find container \"bfc910b016024c52ad21685e4f00e1ebf323cb412cbdac44e180ce0af3b5c974\": container with ID starting with bfc910b016024c52ad21685e4f00e1ebf323cb412cbdac44e180ce0af3b5c974 not found: ID does not exist" Dec 03 12:15:38 crc kubenswrapper[5052]: I1203 12:15:38.298443 5052 scope.go:117] "RemoveContainer" containerID="3577cb2e84ebe92b200c0bf67af07582c963a380e9d15f90356b2162da1491e3" Dec 03 12:15:38 crc kubenswrapper[5052]: E1203 12:15:38.298748 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3577cb2e84ebe92b200c0bf67af07582c963a380e9d15f90356b2162da1491e3\": container with ID starting with 3577cb2e84ebe92b200c0bf67af07582c963a380e9d15f90356b2162da1491e3 not found: ID does not exist" containerID="3577cb2e84ebe92b200c0bf67af07582c963a380e9d15f90356b2162da1491e3" Dec 03 12:15:38 crc kubenswrapper[5052]: I1203 12:15:38.298774 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3577cb2e84ebe92b200c0bf67af07582c963a380e9d15f90356b2162da1491e3"} err="failed to get container status \"3577cb2e84ebe92b200c0bf67af07582c963a380e9d15f90356b2162da1491e3\": rpc error: code = NotFound desc = could not find container \"3577cb2e84ebe92b200c0bf67af07582c963a380e9d15f90356b2162da1491e3\": container with ID starting with 3577cb2e84ebe92b200c0bf67af07582c963a380e9d15f90356b2162da1491e3 not found: ID does not exist" Dec 03 12:15:38 crc kubenswrapper[5052]: I1203 12:15:38.298790 5052 scope.go:117] "RemoveContainer" containerID="a9d69281c8f09f62933866859d2a7f994441cb27231474ecb5c4087f61ef9e48" Dec 03 12:15:38 crc kubenswrapper[5052]: E1203 12:15:38.299070 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a9d69281c8f09f62933866859d2a7f994441cb27231474ecb5c4087f61ef9e48\": container with ID starting with a9d69281c8f09f62933866859d2a7f994441cb27231474ecb5c4087f61ef9e48 not found: ID does not exist" containerID="a9d69281c8f09f62933866859d2a7f994441cb27231474ecb5c4087f61ef9e48" Dec 03 12:15:38 crc kubenswrapper[5052]: I1203 12:15:38.299099 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a9d69281c8f09f62933866859d2a7f994441cb27231474ecb5c4087f61ef9e48"} err="failed to get container status \"a9d69281c8f09f62933866859d2a7f994441cb27231474ecb5c4087f61ef9e48\": rpc error: code = NotFound desc = could not find container \"a9d69281c8f09f62933866859d2a7f994441cb27231474ecb5c4087f61ef9e48\": container with ID starting with a9d69281c8f09f62933866859d2a7f994441cb27231474ecb5c4087f61ef9e48 not found: ID does not exist" Dec 03 12:15:38 crc kubenswrapper[5052]: I1203 12:15:38.299114 5052 scope.go:117] "RemoveContainer" containerID="4be1f7ec0e5ec2598abc5fccb1e3a336e6d2ce904a40e4470bfebd9a2432aa0d" Dec 03 12:15:38 crc kubenswrapper[5052]: E1203 12:15:38.299444 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4be1f7ec0e5ec2598abc5fccb1e3a336e6d2ce904a40e4470bfebd9a2432aa0d\": container with ID starting with 4be1f7ec0e5ec2598abc5fccb1e3a336e6d2ce904a40e4470bfebd9a2432aa0d not found: ID does not exist" containerID="4be1f7ec0e5ec2598abc5fccb1e3a336e6d2ce904a40e4470bfebd9a2432aa0d" Dec 03 12:15:38 crc kubenswrapper[5052]: I1203 12:15:38.299474 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4be1f7ec0e5ec2598abc5fccb1e3a336e6d2ce904a40e4470bfebd9a2432aa0d"} err="failed to get container status \"4be1f7ec0e5ec2598abc5fccb1e3a336e6d2ce904a40e4470bfebd9a2432aa0d\": rpc error: code = NotFound desc = could not find container \"4be1f7ec0e5ec2598abc5fccb1e3a336e6d2ce904a40e4470bfebd9a2432aa0d\": container with ID starting with 4be1f7ec0e5ec2598abc5fccb1e3a336e6d2ce904a40e4470bfebd9a2432aa0d not found: ID does not exist" Dec 03 12:15:38 crc kubenswrapper[5052]: I1203 12:15:38.299492 5052 scope.go:117] "RemoveContainer" containerID="ffacc5bf5e6a05d80223ac48d049c379298b3d4840c2186fb767cc10e1492580" Dec 03 12:15:38 crc kubenswrapper[5052]: E1203 12:15:38.299800 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ffacc5bf5e6a05d80223ac48d049c379298b3d4840c2186fb767cc10e1492580\": container with ID starting with ffacc5bf5e6a05d80223ac48d049c379298b3d4840c2186fb767cc10e1492580 not found: ID does not exist" containerID="ffacc5bf5e6a05d80223ac48d049c379298b3d4840c2186fb767cc10e1492580" Dec 03 12:15:38 crc kubenswrapper[5052]: I1203 12:15:38.299827 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ffacc5bf5e6a05d80223ac48d049c379298b3d4840c2186fb767cc10e1492580"} err="failed to get container status \"ffacc5bf5e6a05d80223ac48d049c379298b3d4840c2186fb767cc10e1492580\": rpc error: code = NotFound desc = could not find container \"ffacc5bf5e6a05d80223ac48d049c379298b3d4840c2186fb767cc10e1492580\": container with ID starting with ffacc5bf5e6a05d80223ac48d049c379298b3d4840c2186fb767cc10e1492580 not found: ID does not exist" Dec 03 12:15:38 crc kubenswrapper[5052]: I1203 12:15:38.299843 5052 scope.go:117] "RemoveContainer" containerID="9abaf8aac6af7b9078e0f7fa87225ebcfdfa4003e45df36fe94117f331cca24b" Dec 03 12:15:38 crc kubenswrapper[5052]: E1203 12:15:38.300170 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9abaf8aac6af7b9078e0f7fa87225ebcfdfa4003e45df36fe94117f331cca24b\": container with ID starting with 9abaf8aac6af7b9078e0f7fa87225ebcfdfa4003e45df36fe94117f331cca24b not found: ID does not exist" containerID="9abaf8aac6af7b9078e0f7fa87225ebcfdfa4003e45df36fe94117f331cca24b" Dec 03 12:15:38 crc kubenswrapper[5052]: I1203 12:15:38.300196 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9abaf8aac6af7b9078e0f7fa87225ebcfdfa4003e45df36fe94117f331cca24b"} err="failed to get container status \"9abaf8aac6af7b9078e0f7fa87225ebcfdfa4003e45df36fe94117f331cca24b\": rpc error: code = NotFound desc = could not find container \"9abaf8aac6af7b9078e0f7fa87225ebcfdfa4003e45df36fe94117f331cca24b\": container with ID starting with 9abaf8aac6af7b9078e0f7fa87225ebcfdfa4003e45df36fe94117f331cca24b not found: ID does not exist" Dec 03 12:15:38 crc kubenswrapper[5052]: I1203 12:15:38.300213 5052 scope.go:117] "RemoveContainer" containerID="8de12c6937f84a484a57da6623c0b720805ae449cf86cbf7781810fff5f7dbd6" Dec 03 12:15:38 crc kubenswrapper[5052]: E1203 12:15:38.300764 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8de12c6937f84a484a57da6623c0b720805ae449cf86cbf7781810fff5f7dbd6\": container with ID starting with 8de12c6937f84a484a57da6623c0b720805ae449cf86cbf7781810fff5f7dbd6 not found: ID does not exist" containerID="8de12c6937f84a484a57da6623c0b720805ae449cf86cbf7781810fff5f7dbd6" Dec 03 12:15:38 crc kubenswrapper[5052]: I1203 12:15:38.300793 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8de12c6937f84a484a57da6623c0b720805ae449cf86cbf7781810fff5f7dbd6"} err="failed to get container status \"8de12c6937f84a484a57da6623c0b720805ae449cf86cbf7781810fff5f7dbd6\": rpc error: code = NotFound desc = could not find container \"8de12c6937f84a484a57da6623c0b720805ae449cf86cbf7781810fff5f7dbd6\": container with ID starting with 8de12c6937f84a484a57da6623c0b720805ae449cf86cbf7781810fff5f7dbd6 not found: ID does not exist" Dec 03 12:15:38 crc kubenswrapper[5052]: I1203 12:15:38.300810 5052 scope.go:117] "RemoveContainer" containerID="daff85236f8b1825cad56f8cf5a450330943257c15e8a198c3cbd529f86943a9" Dec 03 12:15:38 crc kubenswrapper[5052]: E1203 12:15:38.301215 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"daff85236f8b1825cad56f8cf5a450330943257c15e8a198c3cbd529f86943a9\": container with ID starting with daff85236f8b1825cad56f8cf5a450330943257c15e8a198c3cbd529f86943a9 not found: ID does not exist" containerID="daff85236f8b1825cad56f8cf5a450330943257c15e8a198c3cbd529f86943a9" Dec 03 12:15:38 crc kubenswrapper[5052]: I1203 12:15:38.301241 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"daff85236f8b1825cad56f8cf5a450330943257c15e8a198c3cbd529f86943a9"} err="failed to get container status \"daff85236f8b1825cad56f8cf5a450330943257c15e8a198c3cbd529f86943a9\": rpc error: code = NotFound desc = could not find container \"daff85236f8b1825cad56f8cf5a450330943257c15e8a198c3cbd529f86943a9\": container with ID starting with daff85236f8b1825cad56f8cf5a450330943257c15e8a198c3cbd529f86943a9 not found: ID does not exist" Dec 03 12:15:38 crc kubenswrapper[5052]: I1203 12:15:38.301261 5052 scope.go:117] "RemoveContainer" containerID="52c9d3753f1d83dc61693ea154f30426c527a127d3d6f00ae6379a1cbd9e5899" Dec 03 12:15:38 crc kubenswrapper[5052]: E1203 12:15:38.301548 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"52c9d3753f1d83dc61693ea154f30426c527a127d3d6f00ae6379a1cbd9e5899\": container with ID starting with 52c9d3753f1d83dc61693ea154f30426c527a127d3d6f00ae6379a1cbd9e5899 not found: ID does not exist" containerID="52c9d3753f1d83dc61693ea154f30426c527a127d3d6f00ae6379a1cbd9e5899" Dec 03 12:15:38 crc kubenswrapper[5052]: I1203 12:15:38.301586 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"52c9d3753f1d83dc61693ea154f30426c527a127d3d6f00ae6379a1cbd9e5899"} err="failed to get container status \"52c9d3753f1d83dc61693ea154f30426c527a127d3d6f00ae6379a1cbd9e5899\": rpc error: code = NotFound desc = could not find container \"52c9d3753f1d83dc61693ea154f30426c527a127d3d6f00ae6379a1cbd9e5899\": container with ID starting with 52c9d3753f1d83dc61693ea154f30426c527a127d3d6f00ae6379a1cbd9e5899 not found: ID does not exist" Dec 03 12:15:39 crc kubenswrapper[5052]: I1203 12:15:39.682117 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="694f3a22-38d9-487d-b887-a915fa8f99b0" path="/var/lib/kubelet/pods/694f3a22-38d9-487d-b887-a915fa8f99b0/volumes" Dec 03 12:15:39 crc kubenswrapper[5052]: I1203 12:15:39.878727 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-marketplace/redhat-operators-fhsfk" Dec 03 12:15:39 crc kubenswrapper[5052]: I1203 12:15:39.919842 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-marketplace/redhat-operators-fhsfk" Dec 03 12:15:40 crc kubenswrapper[5052]: I1203 12:15:40.114896 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openshift-marketplace/redhat-operators-fhsfk"] Dec 03 12:15:40 crc kubenswrapper[5052]: I1203 12:15:40.336546 5052 pod_container_manager_linux.go:210] "Failed to delete cgroup paths" cgroupName=["kubepods","besteffort","poddfdd8013-ee6e-4ec6-b5d1-fb423f444430"] err="unable to destroy cgroup paths for cgroup [kubepods besteffort poddfdd8013-ee6e-4ec6-b5d1-fb423f444430] : Timed out while waiting for systemd to remove kubepods-besteffort-poddfdd8013_ee6e_4ec6_b5d1_fb423f444430.slice" Dec 03 12:15:40 crc kubenswrapper[5052]: I1203 12:15:40.344712 5052 pod_container_manager_linux.go:210] "Failed to delete cgroup paths" cgroupName=["kubepods","besteffort","poda15b6144-952c-4d41-a141-5bca5c7fd402"] err="unable to destroy cgroup paths for cgroup [kubepods besteffort poda15b6144-952c-4d41-a141-5bca5c7fd402] : Timed out while waiting for systemd to remove kubepods-besteffort-poda15b6144_952c_4d41_a141_5bca5c7fd402.slice" Dec 03 12:15:41 crc kubenswrapper[5052]: I1203 12:15:41.010880 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openshift-marketplace/redhat-operators-fhsfk" podUID="7f722b89-0047-4c97-b76d-3c4faf580781" containerName="registry-server" containerID="cri-o://9d355688d2c7284f0eb0a0fd9c2f7685a270b9b26ad7915d57029c84f3f526ac" gracePeriod=2 Dec 03 12:15:41 crc kubenswrapper[5052]: I1203 12:15:41.410388 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/redhat-operators-fhsfk" Dec 03 12:15:41 crc kubenswrapper[5052]: I1203 12:15:41.555708 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/7f722b89-0047-4c97-b76d-3c4faf580781-utilities\") pod \"7f722b89-0047-4c97-b76d-3c4faf580781\" (UID: \"7f722b89-0047-4c97-b76d-3c4faf580781\") " Dec 03 12:15:41 crc kubenswrapper[5052]: I1203 12:15:41.555756 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/7f722b89-0047-4c97-b76d-3c4faf580781-catalog-content\") pod \"7f722b89-0047-4c97-b76d-3c4faf580781\" (UID: \"7f722b89-0047-4c97-b76d-3c4faf580781\") " Dec 03 12:15:41 crc kubenswrapper[5052]: I1203 12:15:41.555802 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d9nkc\" (UniqueName: \"kubernetes.io/projected/7f722b89-0047-4c97-b76d-3c4faf580781-kube-api-access-d9nkc\") pod \"7f722b89-0047-4c97-b76d-3c4faf580781\" (UID: \"7f722b89-0047-4c97-b76d-3c4faf580781\") " Dec 03 12:15:41 crc kubenswrapper[5052]: I1203 12:15:41.557373 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f722b89-0047-4c97-b76d-3c4faf580781-utilities" (OuterVolumeSpecName: "utilities") pod "7f722b89-0047-4c97-b76d-3c4faf580781" (UID: "7f722b89-0047-4c97-b76d-3c4faf580781"). InnerVolumeSpecName "utilities". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:15:41 crc kubenswrapper[5052]: I1203 12:15:41.561326 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7f722b89-0047-4c97-b76d-3c4faf580781-kube-api-access-d9nkc" (OuterVolumeSpecName: "kube-api-access-d9nkc") pod "7f722b89-0047-4c97-b76d-3c4faf580781" (UID: "7f722b89-0047-4c97-b76d-3c4faf580781"). InnerVolumeSpecName "kube-api-access-d9nkc". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:15:41 crc kubenswrapper[5052]: I1203 12:15:41.657639 5052 reconciler_common.go:293] "Volume detached for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/7f722b89-0047-4c97-b76d-3c4faf580781-utilities\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:41 crc kubenswrapper[5052]: I1203 12:15:41.657682 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-d9nkc\" (UniqueName: \"kubernetes.io/projected/7f722b89-0047-4c97-b76d-3c4faf580781-kube-api-access-d9nkc\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:41 crc kubenswrapper[5052]: I1203 12:15:41.680461 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f722b89-0047-4c97-b76d-3c4faf580781-catalog-content" (OuterVolumeSpecName: "catalog-content") pod "7f722b89-0047-4c97-b76d-3c4faf580781" (UID: "7f722b89-0047-4c97-b76d-3c4faf580781"). InnerVolumeSpecName "catalog-content". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:15:41 crc kubenswrapper[5052]: I1203 12:15:41.759674 5052 reconciler_common.go:293] "Volume detached for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/7f722b89-0047-4c97-b76d-3c4faf580781-catalog-content\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:42 crc kubenswrapper[5052]: I1203 12:15:42.021810 5052 generic.go:334] "Generic (PLEG): container finished" podID="7f722b89-0047-4c97-b76d-3c4faf580781" containerID="9d355688d2c7284f0eb0a0fd9c2f7685a270b9b26ad7915d57029c84f3f526ac" exitCode=0 Dec 03 12:15:42 crc kubenswrapper[5052]: I1203 12:15:42.021867 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-operators-fhsfk" event={"ID":"7f722b89-0047-4c97-b76d-3c4faf580781","Type":"ContainerDied","Data":"9d355688d2c7284f0eb0a0fd9c2f7685a270b9b26ad7915d57029c84f3f526ac"} Dec 03 12:15:42 crc kubenswrapper[5052]: I1203 12:15:42.021903 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-operators-fhsfk" event={"ID":"7f722b89-0047-4c97-b76d-3c4faf580781","Type":"ContainerDied","Data":"6700e86c169ae13ba41a127f46b3899fa650b29b405069f391982dad54559249"} Dec 03 12:15:42 crc kubenswrapper[5052]: I1203 12:15:42.021925 5052 scope.go:117] "RemoveContainer" containerID="9d355688d2c7284f0eb0a0fd9c2f7685a270b9b26ad7915d57029c84f3f526ac" Dec 03 12:15:42 crc kubenswrapper[5052]: I1203 12:15:42.022051 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/redhat-operators-fhsfk" Dec 03 12:15:42 crc kubenswrapper[5052]: I1203 12:15:42.045729 5052 scope.go:117] "RemoveContainer" containerID="7d0d347efadded966fa83aef1520f94181fea705c74c98e60900fc649aeb4650" Dec 03 12:15:42 crc kubenswrapper[5052]: I1203 12:15:42.046780 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openshift-marketplace/redhat-operators-fhsfk"] Dec 03 12:15:42 crc kubenswrapper[5052]: I1203 12:15:42.058923 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openshift-marketplace/redhat-operators-fhsfk"] Dec 03 12:15:42 crc kubenswrapper[5052]: I1203 12:15:42.068097 5052 scope.go:117] "RemoveContainer" containerID="cf77d2f5cf7b52bef64e5e08cef6c48c755a0d1468841ee799e02c7d4a4903c5" Dec 03 12:15:42 crc kubenswrapper[5052]: I1203 12:15:42.092316 5052 scope.go:117] "RemoveContainer" containerID="9d355688d2c7284f0eb0a0fd9c2f7685a270b9b26ad7915d57029c84f3f526ac" Dec 03 12:15:42 crc kubenswrapper[5052]: E1203 12:15:42.092831 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9d355688d2c7284f0eb0a0fd9c2f7685a270b9b26ad7915d57029c84f3f526ac\": container with ID starting with 9d355688d2c7284f0eb0a0fd9c2f7685a270b9b26ad7915d57029c84f3f526ac not found: ID does not exist" containerID="9d355688d2c7284f0eb0a0fd9c2f7685a270b9b26ad7915d57029c84f3f526ac" Dec 03 12:15:42 crc kubenswrapper[5052]: I1203 12:15:42.092861 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9d355688d2c7284f0eb0a0fd9c2f7685a270b9b26ad7915d57029c84f3f526ac"} err="failed to get container status \"9d355688d2c7284f0eb0a0fd9c2f7685a270b9b26ad7915d57029c84f3f526ac\": rpc error: code = NotFound desc = could not find container \"9d355688d2c7284f0eb0a0fd9c2f7685a270b9b26ad7915d57029c84f3f526ac\": container with ID starting with 9d355688d2c7284f0eb0a0fd9c2f7685a270b9b26ad7915d57029c84f3f526ac not found: ID does not exist" Dec 03 12:15:42 crc kubenswrapper[5052]: I1203 12:15:42.092880 5052 scope.go:117] "RemoveContainer" containerID="7d0d347efadded966fa83aef1520f94181fea705c74c98e60900fc649aeb4650" Dec 03 12:15:42 crc kubenswrapper[5052]: E1203 12:15:42.093134 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7d0d347efadded966fa83aef1520f94181fea705c74c98e60900fc649aeb4650\": container with ID starting with 7d0d347efadded966fa83aef1520f94181fea705c74c98e60900fc649aeb4650 not found: ID does not exist" containerID="7d0d347efadded966fa83aef1520f94181fea705c74c98e60900fc649aeb4650" Dec 03 12:15:42 crc kubenswrapper[5052]: I1203 12:15:42.093158 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7d0d347efadded966fa83aef1520f94181fea705c74c98e60900fc649aeb4650"} err="failed to get container status \"7d0d347efadded966fa83aef1520f94181fea705c74c98e60900fc649aeb4650\": rpc error: code = NotFound desc = could not find container \"7d0d347efadded966fa83aef1520f94181fea705c74c98e60900fc649aeb4650\": container with ID starting with 7d0d347efadded966fa83aef1520f94181fea705c74c98e60900fc649aeb4650 not found: ID does not exist" Dec 03 12:15:42 crc kubenswrapper[5052]: I1203 12:15:42.093172 5052 scope.go:117] "RemoveContainer" containerID="cf77d2f5cf7b52bef64e5e08cef6c48c755a0d1468841ee799e02c7d4a4903c5" Dec 03 12:15:42 crc kubenswrapper[5052]: E1203 12:15:42.093418 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cf77d2f5cf7b52bef64e5e08cef6c48c755a0d1468841ee799e02c7d4a4903c5\": container with ID starting with cf77d2f5cf7b52bef64e5e08cef6c48c755a0d1468841ee799e02c7d4a4903c5 not found: ID does not exist" containerID="cf77d2f5cf7b52bef64e5e08cef6c48c755a0d1468841ee799e02c7d4a4903c5" Dec 03 12:15:42 crc kubenswrapper[5052]: I1203 12:15:42.093443 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cf77d2f5cf7b52bef64e5e08cef6c48c755a0d1468841ee799e02c7d4a4903c5"} err="failed to get container status \"cf77d2f5cf7b52bef64e5e08cef6c48c755a0d1468841ee799e02c7d4a4903c5\": rpc error: code = NotFound desc = could not find container \"cf77d2f5cf7b52bef64e5e08cef6c48c755a0d1468841ee799e02c7d4a4903c5\": container with ID starting with cf77d2f5cf7b52bef64e5e08cef6c48c755a0d1468841ee799e02c7d4a4903c5 not found: ID does not exist" Dec 03 12:15:42 crc kubenswrapper[5052]: I1203 12:15:42.552389 5052 pod_container_manager_linux.go:210] "Failed to delete cgroup paths" cgroupName=["kubepods","besteffort","pod388bc903-097f-4f97-8f19-2f7bcdb49cc5"] err="unable to destroy cgroup paths for cgroup [kubepods besteffort pod388bc903-097f-4f97-8f19-2f7bcdb49cc5] : Timed out while waiting for systemd to remove kubepods-besteffort-pod388bc903_097f_4f97_8f19_2f7bcdb49cc5.slice" Dec 03 12:15:42 crc kubenswrapper[5052]: E1203 12:15:42.552436 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to delete cgroup paths for [kubepods besteffort pod388bc903-097f-4f97-8f19-2f7bcdb49cc5] : unable to destroy cgroup paths for cgroup [kubepods besteffort pod388bc903-097f-4f97-8f19-2f7bcdb49cc5] : Timed out while waiting for systemd to remove kubepods-besteffort-pod388bc903_097f_4f97_8f19_2f7bcdb49cc5.slice" pod="openstack-kuttl-tests/cinder-api-0" podUID="388bc903-097f-4f97-8f19-2f7bcdb49cc5" Dec 03 12:15:42 crc kubenswrapper[5052]: I1203 12:15:42.573152 5052 pod_container_manager_linux.go:210] "Failed to delete cgroup paths" cgroupName=["kubepods","besteffort","podae6940a4-0f4c-46ab-8218-56bf5c5d3084"] err="unable to destroy cgroup paths for cgroup [kubepods besteffort podae6940a4-0f4c-46ab-8218-56bf5c5d3084] : Timed out while waiting for systemd to remove kubepods-besteffort-podae6940a4_0f4c_46ab_8218_56bf5c5d3084.slice" Dec 03 12:15:42 crc kubenswrapper[5052]: I1203 12:15:42.752387 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/novaapi06eb-account-delete-kvlp2" Dec 03 12:15:42 crc kubenswrapper[5052]: I1203 12:15:42.879130 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/72a72aa7-f35b-4b95-b5fd-caaf1272943a-operator-scripts\") pod \"72a72aa7-f35b-4b95-b5fd-caaf1272943a\" (UID: \"72a72aa7-f35b-4b95-b5fd-caaf1272943a\") " Dec 03 12:15:42 crc kubenswrapper[5052]: I1203 12:15:42.879912 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t9b6r\" (UniqueName: \"kubernetes.io/projected/72a72aa7-f35b-4b95-b5fd-caaf1272943a-kube-api-access-t9b6r\") pod \"72a72aa7-f35b-4b95-b5fd-caaf1272943a\" (UID: \"72a72aa7-f35b-4b95-b5fd-caaf1272943a\") " Dec 03 12:15:42 crc kubenswrapper[5052]: I1203 12:15:42.880069 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/72a72aa7-f35b-4b95-b5fd-caaf1272943a-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "72a72aa7-f35b-4b95-b5fd-caaf1272943a" (UID: "72a72aa7-f35b-4b95-b5fd-caaf1272943a"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:15:42 crc kubenswrapper[5052]: I1203 12:15:42.880297 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/72a72aa7-f35b-4b95-b5fd-caaf1272943a-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:42 crc kubenswrapper[5052]: I1203 12:15:42.884829 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/72a72aa7-f35b-4b95-b5fd-caaf1272943a-kube-api-access-t9b6r" (OuterVolumeSpecName: "kube-api-access-t9b6r") pod "72a72aa7-f35b-4b95-b5fd-caaf1272943a" (UID: "72a72aa7-f35b-4b95-b5fd-caaf1272943a"). InnerVolumeSpecName "kube-api-access-t9b6r". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:15:42 crc kubenswrapper[5052]: I1203 12:15:42.981549 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-t9b6r\" (UniqueName: \"kubernetes.io/projected/72a72aa7-f35b-4b95-b5fd-caaf1272943a-kube-api-access-t9b6r\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:43 crc kubenswrapper[5052]: I1203 12:15:43.033121 5052 generic.go:334] "Generic (PLEG): container finished" podID="72a72aa7-f35b-4b95-b5fd-caaf1272943a" containerID="1d5931017ee84078964fe579758e4691d11ff7226c17409d0cc037c9caa5d258" exitCode=137 Dec 03 12:15:43 crc kubenswrapper[5052]: I1203 12:15:43.033190 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/novaapi06eb-account-delete-kvlp2" Dec 03 12:15:43 crc kubenswrapper[5052]: I1203 12:15:43.033204 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/novaapi06eb-account-delete-kvlp2" event={"ID":"72a72aa7-f35b-4b95-b5fd-caaf1272943a","Type":"ContainerDied","Data":"1d5931017ee84078964fe579758e4691d11ff7226c17409d0cc037c9caa5d258"} Dec 03 12:15:43 crc kubenswrapper[5052]: I1203 12:15:43.033226 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/novaapi06eb-account-delete-kvlp2" event={"ID":"72a72aa7-f35b-4b95-b5fd-caaf1272943a","Type":"ContainerDied","Data":"1b718b05fd3c5f8e7a2449b1d22b045b8adb4bf91592bb2a833f1416926ad385"} Dec 03 12:15:43 crc kubenswrapper[5052]: I1203 12:15:43.033241 5052 scope.go:117] "RemoveContainer" containerID="1d5931017ee84078964fe579758e4691d11ff7226c17409d0cc037c9caa5d258" Dec 03 12:15:43 crc kubenswrapper[5052]: I1203 12:15:43.033194 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:15:43 crc kubenswrapper[5052]: I1203 12:15:43.058749 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 12:15:43 crc kubenswrapper[5052]: I1203 12:15:43.070884 5052 scope.go:117] "RemoveContainer" containerID="1d5931017ee84078964fe579758e4691d11ff7226c17409d0cc037c9caa5d258" Dec 03 12:15:43 crc kubenswrapper[5052]: E1203 12:15:43.071540 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1d5931017ee84078964fe579758e4691d11ff7226c17409d0cc037c9caa5d258\": container with ID starting with 1d5931017ee84078964fe579758e4691d11ff7226c17409d0cc037c9caa5d258 not found: ID does not exist" containerID="1d5931017ee84078964fe579758e4691d11ff7226c17409d0cc037c9caa5d258" Dec 03 12:15:43 crc kubenswrapper[5052]: I1203 12:15:43.071600 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1d5931017ee84078964fe579758e4691d11ff7226c17409d0cc037c9caa5d258"} err="failed to get container status \"1d5931017ee84078964fe579758e4691d11ff7226c17409d0cc037c9caa5d258\": rpc error: code = NotFound desc = could not find container \"1d5931017ee84078964fe579758e4691d11ff7226c17409d0cc037c9caa5d258\": container with ID starting with 1d5931017ee84078964fe579758e4691d11ff7226c17409d0cc037c9caa5d258 not found: ID does not exist" Dec 03 12:15:43 crc kubenswrapper[5052]: I1203 12:15:43.073938 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 12:15:43 crc kubenswrapper[5052]: I1203 12:15:43.083358 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/novaapi06eb-account-delete-kvlp2"] Dec 03 12:15:43 crc kubenswrapper[5052]: I1203 12:15:43.090942 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/novaapi06eb-account-delete-kvlp2"] Dec 03 12:15:43 crc kubenswrapper[5052]: I1203 12:15:43.683017 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="388bc903-097f-4f97-8f19-2f7bcdb49cc5" path="/var/lib/kubelet/pods/388bc903-097f-4f97-8f19-2f7bcdb49cc5/volumes" Dec 03 12:15:43 crc kubenswrapper[5052]: I1203 12:15:43.683864 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="72a72aa7-f35b-4b95-b5fd-caaf1272943a" path="/var/lib/kubelet/pods/72a72aa7-f35b-4b95-b5fd-caaf1272943a/volumes" Dec 03 12:15:43 crc kubenswrapper[5052]: I1203 12:15:43.684312 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7f722b89-0047-4c97-b76d-3c4faf580781" path="/var/lib/kubelet/pods/7f722b89-0047-4c97-b76d-3c4faf580781/volumes" Dec 03 12:15:43 crc kubenswrapper[5052]: I1203 12:15:43.845377 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/novacell00bb5-account-delete-r2csc" Dec 03 12:15:43 crc kubenswrapper[5052]: I1203 12:15:43.995709 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/c200c246-74d3-48dc-8b24-ac4b37611069-operator-scripts\") pod \"c200c246-74d3-48dc-8b24-ac4b37611069\" (UID: \"c200c246-74d3-48dc-8b24-ac4b37611069\") " Dec 03 12:15:43 crc kubenswrapper[5052]: I1203 12:15:43.996279 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b7zkq\" (UniqueName: \"kubernetes.io/projected/c200c246-74d3-48dc-8b24-ac4b37611069-kube-api-access-b7zkq\") pod \"c200c246-74d3-48dc-8b24-ac4b37611069\" (UID: \"c200c246-74d3-48dc-8b24-ac4b37611069\") " Dec 03 12:15:43 crc kubenswrapper[5052]: I1203 12:15:43.996298 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c200c246-74d3-48dc-8b24-ac4b37611069-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "c200c246-74d3-48dc-8b24-ac4b37611069" (UID: "c200c246-74d3-48dc-8b24-ac4b37611069"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:15:43 crc kubenswrapper[5052]: I1203 12:15:43.996639 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/c200c246-74d3-48dc-8b24-ac4b37611069-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:44 crc kubenswrapper[5052]: I1203 12:15:43.999945 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c200c246-74d3-48dc-8b24-ac4b37611069-kube-api-access-b7zkq" (OuterVolumeSpecName: "kube-api-access-b7zkq") pod "c200c246-74d3-48dc-8b24-ac4b37611069" (UID: "c200c246-74d3-48dc-8b24-ac4b37611069"). InnerVolumeSpecName "kube-api-access-b7zkq". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:15:44 crc kubenswrapper[5052]: I1203 12:15:44.054754 5052 generic.go:334] "Generic (PLEG): container finished" podID="c200c246-74d3-48dc-8b24-ac4b37611069" containerID="c7a74e6ab5fc2e0b601148d2ded033a3a334d1f7ad582da015664c00d9a6e301" exitCode=137 Dec 03 12:15:44 crc kubenswrapper[5052]: I1203 12:15:44.054870 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/novacell00bb5-account-delete-r2csc" event={"ID":"c200c246-74d3-48dc-8b24-ac4b37611069","Type":"ContainerDied","Data":"c7a74e6ab5fc2e0b601148d2ded033a3a334d1f7ad582da015664c00d9a6e301"} Dec 03 12:15:44 crc kubenswrapper[5052]: I1203 12:15:44.054941 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/novacell00bb5-account-delete-r2csc" event={"ID":"c200c246-74d3-48dc-8b24-ac4b37611069","Type":"ContainerDied","Data":"b30037926737b8288cb289a95633b6703674d732f8671e54cacf83c371b2ed55"} Dec 03 12:15:44 crc kubenswrapper[5052]: I1203 12:15:44.054962 5052 scope.go:117] "RemoveContainer" containerID="c7a74e6ab5fc2e0b601148d2ded033a3a334d1f7ad582da015664c00d9a6e301" Dec 03 12:15:44 crc kubenswrapper[5052]: I1203 12:15:44.055390 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/novacell00bb5-account-delete-r2csc" Dec 03 12:15:44 crc kubenswrapper[5052]: I1203 12:15:44.089652 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/novacell00bb5-account-delete-r2csc"] Dec 03 12:15:44 crc kubenswrapper[5052]: I1203 12:15:44.095042 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/novacell00bb5-account-delete-r2csc"] Dec 03 12:15:44 crc kubenswrapper[5052]: I1203 12:15:44.095676 5052 scope.go:117] "RemoveContainer" containerID="c7a74e6ab5fc2e0b601148d2ded033a3a334d1f7ad582da015664c00d9a6e301" Dec 03 12:15:44 crc kubenswrapper[5052]: E1203 12:15:44.096175 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c7a74e6ab5fc2e0b601148d2ded033a3a334d1f7ad582da015664c00d9a6e301\": container with ID starting with c7a74e6ab5fc2e0b601148d2ded033a3a334d1f7ad582da015664c00d9a6e301 not found: ID does not exist" containerID="c7a74e6ab5fc2e0b601148d2ded033a3a334d1f7ad582da015664c00d9a6e301" Dec 03 12:15:44 crc kubenswrapper[5052]: I1203 12:15:44.096208 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c7a74e6ab5fc2e0b601148d2ded033a3a334d1f7ad582da015664c00d9a6e301"} err="failed to get container status \"c7a74e6ab5fc2e0b601148d2ded033a3a334d1f7ad582da015664c00d9a6e301\": rpc error: code = NotFound desc = could not find container \"c7a74e6ab5fc2e0b601148d2ded033a3a334d1f7ad582da015664c00d9a6e301\": container with ID starting with c7a74e6ab5fc2e0b601148d2ded033a3a334d1f7ad582da015664c00d9a6e301 not found: ID does not exist" Dec 03 12:15:44 crc kubenswrapper[5052]: I1203 12:15:44.098106 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-b7zkq\" (UniqueName: \"kubernetes.io/projected/c200c246-74d3-48dc-8b24-ac4b37611069-kube-api-access-b7zkq\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:44 crc kubenswrapper[5052]: E1203 12:15:44.302248 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:15:44 crc kubenswrapper[5052]: E1203 12:15:44.302374 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/679f17b9-0e40-480b-bfde-4f63899cb972-operator-scripts podName:679f17b9-0e40-480b-bfde-4f63899cb972 nodeName:}" failed. No retries permitted until 2025-12-03 12:16:16.302350433 +0000 UTC m=+3366.890592197 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/679f17b9-0e40-480b-bfde-4f63899cb972-operator-scripts") pod "cinder7878-account-delete-jwtvt" (UID: "679f17b9-0e40-480b-bfde-4f63899cb972") : configmap "openstack-scripts" not found Dec 03 12:15:45 crc kubenswrapper[5052]: I1203 12:15:45.681885 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c200c246-74d3-48dc-8b24-ac4b37611069" path="/var/lib/kubelet/pods/c200c246-74d3-48dc-8b24-ac4b37611069/volumes" Dec 03 12:15:46 crc kubenswrapper[5052]: I1203 12:15:46.080398 5052 generic.go:334] "Generic (PLEG): container finished" podID="679f17b9-0e40-480b-bfde-4f63899cb972" containerID="e9f158233d3479671db2b774df7b5ea49c9fbfe33de68208a166ee13a03145b4" exitCode=137 Dec 03 12:15:46 crc kubenswrapper[5052]: I1203 12:15:46.080435 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder7878-account-delete-jwtvt" event={"ID":"679f17b9-0e40-480b-bfde-4f63899cb972","Type":"ContainerDied","Data":"e9f158233d3479671db2b774df7b5ea49c9fbfe33de68208a166ee13a03145b4"} Dec 03 12:15:46 crc kubenswrapper[5052]: I1203 12:15:46.172300 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder7878-account-delete-jwtvt" Dec 03 12:15:46 crc kubenswrapper[5052]: I1203 12:15:46.337348 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/679f17b9-0e40-480b-bfde-4f63899cb972-operator-scripts\") pod \"679f17b9-0e40-480b-bfde-4f63899cb972\" (UID: \"679f17b9-0e40-480b-bfde-4f63899cb972\") " Dec 03 12:15:46 crc kubenswrapper[5052]: I1203 12:15:46.337432 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v2r2s\" (UniqueName: \"kubernetes.io/projected/679f17b9-0e40-480b-bfde-4f63899cb972-kube-api-access-v2r2s\") pod \"679f17b9-0e40-480b-bfde-4f63899cb972\" (UID: \"679f17b9-0e40-480b-bfde-4f63899cb972\") " Dec 03 12:15:46 crc kubenswrapper[5052]: I1203 12:15:46.339049 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/679f17b9-0e40-480b-bfde-4f63899cb972-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "679f17b9-0e40-480b-bfde-4f63899cb972" (UID: "679f17b9-0e40-480b-bfde-4f63899cb972"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:15:46 crc kubenswrapper[5052]: I1203 12:15:46.359651 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/679f17b9-0e40-480b-bfde-4f63899cb972-kube-api-access-v2r2s" (OuterVolumeSpecName: "kube-api-access-v2r2s") pod "679f17b9-0e40-480b-bfde-4f63899cb972" (UID: "679f17b9-0e40-480b-bfde-4f63899cb972"). InnerVolumeSpecName "kube-api-access-v2r2s". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:15:46 crc kubenswrapper[5052]: I1203 12:15:46.439436 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/679f17b9-0e40-480b-bfde-4f63899cb972-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:46 crc kubenswrapper[5052]: I1203 12:15:46.439492 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-v2r2s\" (UniqueName: \"kubernetes.io/projected/679f17b9-0e40-480b-bfde-4f63899cb972-kube-api-access-v2r2s\") on node \"crc\" DevicePath \"\"" Dec 03 12:15:47 crc kubenswrapper[5052]: I1203 12:15:47.092821 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder7878-account-delete-jwtvt" event={"ID":"679f17b9-0e40-480b-bfde-4f63899cb972","Type":"ContainerDied","Data":"bb3bc4ace76df2b60f85d9bcc20c9ae03f8246692b64143eca974b94cb127a47"} Dec 03 12:15:47 crc kubenswrapper[5052]: I1203 12:15:47.092889 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder7878-account-delete-jwtvt" Dec 03 12:15:47 crc kubenswrapper[5052]: I1203 12:15:47.093162 5052 scope.go:117] "RemoveContainer" containerID="e9f158233d3479671db2b774df7b5ea49c9fbfe33de68208a166ee13a03145b4" Dec 03 12:15:47 crc kubenswrapper[5052]: I1203 12:15:47.123420 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder7878-account-delete-jwtvt"] Dec 03 12:15:47 crc kubenswrapper[5052]: I1203 12:15:47.129210 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/cinder7878-account-delete-jwtvt"] Dec 03 12:15:47 crc kubenswrapper[5052]: I1203 12:15:47.682407 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="679f17b9-0e40-480b-bfde-4f63899cb972" path="/var/lib/kubelet/pods/679f17b9-0e40-480b-bfde-4f63899cb972/volumes" Dec 03 12:15:47 crc kubenswrapper[5052]: I1203 12:15:47.868092 5052 pod_container_manager_linux.go:210] "Failed to delete cgroup paths" cgroupName=["kubepods","besteffort","pod85f5676f-32e8-4c3d-8729-e1eb2fdc8dc9"] err="unable to destroy cgroup paths for cgroup [kubepods besteffort pod85f5676f-32e8-4c3d-8729-e1eb2fdc8dc9] : Timed out while waiting for systemd to remove kubepods-besteffort-pod85f5676f_32e8_4c3d_8729_e1eb2fdc8dc9.slice" Dec 03 12:15:47 crc kubenswrapper[5052]: E1203 12:15:47.868183 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to delete cgroup paths for [kubepods besteffort pod85f5676f-32e8-4c3d-8729-e1eb2fdc8dc9] : unable to destroy cgroup paths for cgroup [kubepods besteffort pod85f5676f-32e8-4c3d-8729-e1eb2fdc8dc9] : Timed out while waiting for systemd to remove kubepods-besteffort-pod85f5676f_32e8_4c3d_8729_e1eb2fdc8dc9.slice" pod="openstack-kuttl-tests/openstack-galera-0" podUID="85f5676f-32e8-4c3d-8729-e1eb2fdc8dc9" Dec 03 12:15:48 crc kubenswrapper[5052]: I1203 12:15:48.102235 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:15:48 crc kubenswrapper[5052]: I1203 12:15:48.124735 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/openstack-galera-0"] Dec 03 12:15:48 crc kubenswrapper[5052]: I1203 12:15:48.130964 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/openstack-galera-0"] Dec 03 12:15:49 crc kubenswrapper[5052]: I1203 12:15:49.682832 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="85f5676f-32e8-4c3d-8729-e1eb2fdc8dc9" path="/var/lib/kubelet/pods/85f5676f-32e8-4c3d-8729-e1eb2fdc8dc9/volumes" Dec 03 12:16:00 crc kubenswrapper[5052]: I1203 12:16:00.865787 5052 scope.go:117] "RemoveContainer" containerID="015b047fdd13acbbfc273d6c1ab47adc405079b2a4c2c1faa5e8cf0065aec661" Dec 03 12:16:00 crc kubenswrapper[5052]: I1203 12:16:00.898702 5052 scope.go:117] "RemoveContainer" containerID="cb976b53b838bf480cfbd836377b628ef3ca7feb9864a7f13f8fe243b48ea276" Dec 03 12:16:05 crc kubenswrapper[5052]: I1203 12:16:05.782404 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["crc-storage/crc-storage-crc-nrblr"] Dec 03 12:16:05 crc kubenswrapper[5052]: I1203 12:16:05.788874 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["crc-storage/crc-storage-crc-nrblr"] Dec 03 12:16:05 crc kubenswrapper[5052]: I1203 12:16:05.933625 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["crc-storage/crc-storage-crc-2vwmq"] Dec 03 12:16:05 crc kubenswrapper[5052]: E1203 12:16:05.934183 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="7f722b89-0047-4c97-b76d-3c4faf580781" containerName="extract-content" Dec 03 12:16:05 crc kubenswrapper[5052]: I1203 12:16:05.934214 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f722b89-0047-4c97-b76d-3c4faf580781" containerName="extract-content" Dec 03 12:16:05 crc kubenswrapper[5052]: E1203 12:16:05.934227 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="694f3a22-38d9-487d-b887-a915fa8f99b0" containerName="object-expirer" Dec 03 12:16:05 crc kubenswrapper[5052]: I1203 12:16:05.934237 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="694f3a22-38d9-487d-b887-a915fa8f99b0" containerName="object-expirer" Dec 03 12:16:05 crc kubenswrapper[5052]: E1203 12:16:05.934258 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="694f3a22-38d9-487d-b887-a915fa8f99b0" containerName="container-updater" Dec 03 12:16:05 crc kubenswrapper[5052]: I1203 12:16:05.934266 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="694f3a22-38d9-487d-b887-a915fa8f99b0" containerName="container-updater" Dec 03 12:16:05 crc kubenswrapper[5052]: E1203 12:16:05.934276 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="718e2899-d5c3-41a4-a2f9-4830e18355d3" containerName="neutron-httpd" Dec 03 12:16:05 crc kubenswrapper[5052]: I1203 12:16:05.934283 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="718e2899-d5c3-41a4-a2f9-4830e18355d3" containerName="neutron-httpd" Dec 03 12:16:05 crc kubenswrapper[5052]: E1203 12:16:05.934295 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="694f3a22-38d9-487d-b887-a915fa8f99b0" containerName="container-server" Dec 03 12:16:05 crc kubenswrapper[5052]: I1203 12:16:05.934302 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="694f3a22-38d9-487d-b887-a915fa8f99b0" containerName="container-server" Dec 03 12:16:05 crc kubenswrapper[5052]: E1203 12:16:05.934321 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="694f3a22-38d9-487d-b887-a915fa8f99b0" containerName="account-replicator" Dec 03 12:16:05 crc kubenswrapper[5052]: I1203 12:16:05.934329 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="694f3a22-38d9-487d-b887-a915fa8f99b0" containerName="account-replicator" Dec 03 12:16:05 crc kubenswrapper[5052]: E1203 12:16:05.934340 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="679f17b9-0e40-480b-bfde-4f63899cb972" containerName="mariadb-account-delete" Dec 03 12:16:05 crc kubenswrapper[5052]: I1203 12:16:05.934348 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="679f17b9-0e40-480b-bfde-4f63899cb972" containerName="mariadb-account-delete" Dec 03 12:16:05 crc kubenswrapper[5052]: E1203 12:16:05.934361 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="694f3a22-38d9-487d-b887-a915fa8f99b0" containerName="object-replicator" Dec 03 12:16:05 crc kubenswrapper[5052]: I1203 12:16:05.934369 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="694f3a22-38d9-487d-b887-a915fa8f99b0" containerName="object-replicator" Dec 03 12:16:05 crc kubenswrapper[5052]: E1203 12:16:05.934379 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="7f722b89-0047-4c97-b76d-3c4faf580781" containerName="registry-server" Dec 03 12:16:05 crc kubenswrapper[5052]: I1203 12:16:05.934387 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f722b89-0047-4c97-b76d-3c4faf580781" containerName="registry-server" Dec 03 12:16:05 crc kubenswrapper[5052]: E1203 12:16:05.934400 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="694f3a22-38d9-487d-b887-a915fa8f99b0" containerName="swift-recon-cron" Dec 03 12:16:05 crc kubenswrapper[5052]: I1203 12:16:05.934408 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="694f3a22-38d9-487d-b887-a915fa8f99b0" containerName="swift-recon-cron" Dec 03 12:16:05 crc kubenswrapper[5052]: E1203 12:16:05.934417 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="694f3a22-38d9-487d-b887-a915fa8f99b0" containerName="rsync" Dec 03 12:16:05 crc kubenswrapper[5052]: I1203 12:16:05.934425 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="694f3a22-38d9-487d-b887-a915fa8f99b0" containerName="rsync" Dec 03 12:16:05 crc kubenswrapper[5052]: E1203 12:16:05.934432 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="7f722b89-0047-4c97-b76d-3c4faf580781" containerName="extract-utilities" Dec 03 12:16:05 crc kubenswrapper[5052]: I1203 12:16:05.934439 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f722b89-0047-4c97-b76d-3c4faf580781" containerName="extract-utilities" Dec 03 12:16:05 crc kubenswrapper[5052]: E1203 12:16:05.934454 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="c200c246-74d3-48dc-8b24-ac4b37611069" containerName="mariadb-account-delete" Dec 03 12:16:05 crc kubenswrapper[5052]: I1203 12:16:05.934462 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="c200c246-74d3-48dc-8b24-ac4b37611069" containerName="mariadb-account-delete" Dec 03 12:16:05 crc kubenswrapper[5052]: E1203 12:16:05.934477 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="718e2899-d5c3-41a4-a2f9-4830e18355d3" containerName="neutron-api" Dec 03 12:16:05 crc kubenswrapper[5052]: I1203 12:16:05.934484 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="718e2899-d5c3-41a4-a2f9-4830e18355d3" containerName="neutron-api" Dec 03 12:16:05 crc kubenswrapper[5052]: E1203 12:16:05.934496 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="694f3a22-38d9-487d-b887-a915fa8f99b0" containerName="account-auditor" Dec 03 12:16:05 crc kubenswrapper[5052]: I1203 12:16:05.934503 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="694f3a22-38d9-487d-b887-a915fa8f99b0" containerName="account-auditor" Dec 03 12:16:05 crc kubenswrapper[5052]: E1203 12:16:05.934544 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="694f3a22-38d9-487d-b887-a915fa8f99b0" containerName="container-auditor" Dec 03 12:16:05 crc kubenswrapper[5052]: I1203 12:16:05.934551 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="694f3a22-38d9-487d-b887-a915fa8f99b0" containerName="container-auditor" Dec 03 12:16:05 crc kubenswrapper[5052]: E1203 12:16:05.934563 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="694f3a22-38d9-487d-b887-a915fa8f99b0" containerName="account-server" Dec 03 12:16:05 crc kubenswrapper[5052]: I1203 12:16:05.934569 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="694f3a22-38d9-487d-b887-a915fa8f99b0" containerName="account-server" Dec 03 12:16:05 crc kubenswrapper[5052]: E1203 12:16:05.934583 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="694f3a22-38d9-487d-b887-a915fa8f99b0" containerName="account-reaper" Dec 03 12:16:05 crc kubenswrapper[5052]: I1203 12:16:05.934590 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="694f3a22-38d9-487d-b887-a915fa8f99b0" containerName="account-reaper" Dec 03 12:16:05 crc kubenswrapper[5052]: E1203 12:16:05.934600 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="694f3a22-38d9-487d-b887-a915fa8f99b0" containerName="object-server" Dec 03 12:16:05 crc kubenswrapper[5052]: I1203 12:16:05.934607 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="694f3a22-38d9-487d-b887-a915fa8f99b0" containerName="object-server" Dec 03 12:16:05 crc kubenswrapper[5052]: E1203 12:16:05.934620 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="694f3a22-38d9-487d-b887-a915fa8f99b0" containerName="container-replicator" Dec 03 12:16:05 crc kubenswrapper[5052]: I1203 12:16:05.934628 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="694f3a22-38d9-487d-b887-a915fa8f99b0" containerName="container-replicator" Dec 03 12:16:05 crc kubenswrapper[5052]: E1203 12:16:05.934639 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="72a72aa7-f35b-4b95-b5fd-caaf1272943a" containerName="mariadb-account-delete" Dec 03 12:16:05 crc kubenswrapper[5052]: I1203 12:16:05.934646 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="72a72aa7-f35b-4b95-b5fd-caaf1272943a" containerName="mariadb-account-delete" Dec 03 12:16:05 crc kubenswrapper[5052]: E1203 12:16:05.934663 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="694f3a22-38d9-487d-b887-a915fa8f99b0" containerName="object-auditor" Dec 03 12:16:05 crc kubenswrapper[5052]: I1203 12:16:05.934674 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="694f3a22-38d9-487d-b887-a915fa8f99b0" containerName="object-auditor" Dec 03 12:16:05 crc kubenswrapper[5052]: E1203 12:16:05.934689 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="694f3a22-38d9-487d-b887-a915fa8f99b0" containerName="object-updater" Dec 03 12:16:05 crc kubenswrapper[5052]: I1203 12:16:05.934702 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="694f3a22-38d9-487d-b887-a915fa8f99b0" containerName="object-updater" Dec 03 12:16:05 crc kubenswrapper[5052]: I1203 12:16:05.934899 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="718e2899-d5c3-41a4-a2f9-4830e18355d3" containerName="neutron-httpd" Dec 03 12:16:05 crc kubenswrapper[5052]: I1203 12:16:05.934921 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="694f3a22-38d9-487d-b887-a915fa8f99b0" containerName="object-replicator" Dec 03 12:16:05 crc kubenswrapper[5052]: I1203 12:16:05.934933 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="694f3a22-38d9-487d-b887-a915fa8f99b0" containerName="container-auditor" Dec 03 12:16:05 crc kubenswrapper[5052]: I1203 12:16:05.934946 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="694f3a22-38d9-487d-b887-a915fa8f99b0" containerName="container-updater" Dec 03 12:16:05 crc kubenswrapper[5052]: I1203 12:16:05.934958 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="679f17b9-0e40-480b-bfde-4f63899cb972" containerName="mariadb-account-delete" Dec 03 12:16:05 crc kubenswrapper[5052]: I1203 12:16:05.934969 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="72a72aa7-f35b-4b95-b5fd-caaf1272943a" containerName="mariadb-account-delete" Dec 03 12:16:05 crc kubenswrapper[5052]: I1203 12:16:05.934982 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="694f3a22-38d9-487d-b887-a915fa8f99b0" containerName="account-replicator" Dec 03 12:16:05 crc kubenswrapper[5052]: I1203 12:16:05.934992 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="718e2899-d5c3-41a4-a2f9-4830e18355d3" containerName="neutron-api" Dec 03 12:16:05 crc kubenswrapper[5052]: I1203 12:16:05.935003 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="c200c246-74d3-48dc-8b24-ac4b37611069" containerName="mariadb-account-delete" Dec 03 12:16:05 crc kubenswrapper[5052]: I1203 12:16:05.935012 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="694f3a22-38d9-487d-b887-a915fa8f99b0" containerName="object-updater" Dec 03 12:16:05 crc kubenswrapper[5052]: I1203 12:16:05.935024 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="694f3a22-38d9-487d-b887-a915fa8f99b0" containerName="container-server" Dec 03 12:16:05 crc kubenswrapper[5052]: I1203 12:16:05.935035 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="694f3a22-38d9-487d-b887-a915fa8f99b0" containerName="rsync" Dec 03 12:16:05 crc kubenswrapper[5052]: I1203 12:16:05.935043 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="694f3a22-38d9-487d-b887-a915fa8f99b0" containerName="container-replicator" Dec 03 12:16:05 crc kubenswrapper[5052]: I1203 12:16:05.935052 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="694f3a22-38d9-487d-b887-a915fa8f99b0" containerName="account-auditor" Dec 03 12:16:05 crc kubenswrapper[5052]: I1203 12:16:05.935065 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="694f3a22-38d9-487d-b887-a915fa8f99b0" containerName="account-reaper" Dec 03 12:16:05 crc kubenswrapper[5052]: I1203 12:16:05.935075 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="694f3a22-38d9-487d-b887-a915fa8f99b0" containerName="object-auditor" Dec 03 12:16:05 crc kubenswrapper[5052]: I1203 12:16:05.935089 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="694f3a22-38d9-487d-b887-a915fa8f99b0" containerName="object-server" Dec 03 12:16:05 crc kubenswrapper[5052]: I1203 12:16:05.935101 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="694f3a22-38d9-487d-b887-a915fa8f99b0" containerName="swift-recon-cron" Dec 03 12:16:05 crc kubenswrapper[5052]: I1203 12:16:05.935113 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="7f722b89-0047-4c97-b76d-3c4faf580781" containerName="registry-server" Dec 03 12:16:05 crc kubenswrapper[5052]: I1203 12:16:05.935127 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="694f3a22-38d9-487d-b887-a915fa8f99b0" containerName="account-server" Dec 03 12:16:05 crc kubenswrapper[5052]: I1203 12:16:05.935137 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="694f3a22-38d9-487d-b887-a915fa8f99b0" containerName="object-expirer" Dec 03 12:16:05 crc kubenswrapper[5052]: I1203 12:16:05.936863 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="crc-storage/crc-storage-crc-2vwmq" Dec 03 12:16:05 crc kubenswrapper[5052]: I1203 12:16:05.941130 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"crc-storage"/"kube-root-ca.crt" Dec 03 12:16:05 crc kubenswrapper[5052]: I1203 12:16:05.941167 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"crc-storage"/"crc-storage" Dec 03 12:16:05 crc kubenswrapper[5052]: I1203 12:16:05.941400 5052 reflector.go:368] Caches populated for *v1.Secret from object-"crc-storage"/"crc-storage-dockercfg-mf76g" Dec 03 12:16:05 crc kubenswrapper[5052]: I1203 12:16:05.941755 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"crc-storage"/"openshift-service-ca.crt" Dec 03 12:16:05 crc kubenswrapper[5052]: I1203 12:16:05.952900 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["crc-storage/crc-storage-crc-2vwmq"] Dec 03 12:16:06 crc kubenswrapper[5052]: I1203 12:16:06.040214 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/13415b4d-4d47-4358-8b8e-9caf074193b6-node-mnt\") pod \"crc-storage-crc-2vwmq\" (UID: \"13415b4d-4d47-4358-8b8e-9caf074193b6\") " pod="crc-storage/crc-storage-crc-2vwmq" Dec 03 12:16:06 crc kubenswrapper[5052]: I1203 12:16:06.040809 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/13415b4d-4d47-4358-8b8e-9caf074193b6-crc-storage\") pod \"crc-storage-crc-2vwmq\" (UID: \"13415b4d-4d47-4358-8b8e-9caf074193b6\") " pod="crc-storage/crc-storage-crc-2vwmq" Dec 03 12:16:06 crc kubenswrapper[5052]: I1203 12:16:06.041204 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bq429\" (UniqueName: \"kubernetes.io/projected/13415b4d-4d47-4358-8b8e-9caf074193b6-kube-api-access-bq429\") pod \"crc-storage-crc-2vwmq\" (UID: \"13415b4d-4d47-4358-8b8e-9caf074193b6\") " pod="crc-storage/crc-storage-crc-2vwmq" Dec 03 12:16:06 crc kubenswrapper[5052]: I1203 12:16:06.142524 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/13415b4d-4d47-4358-8b8e-9caf074193b6-node-mnt\") pod \"crc-storage-crc-2vwmq\" (UID: \"13415b4d-4d47-4358-8b8e-9caf074193b6\") " pod="crc-storage/crc-storage-crc-2vwmq" Dec 03 12:16:06 crc kubenswrapper[5052]: I1203 12:16:06.142644 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/13415b4d-4d47-4358-8b8e-9caf074193b6-crc-storage\") pod \"crc-storage-crc-2vwmq\" (UID: \"13415b4d-4d47-4358-8b8e-9caf074193b6\") " pod="crc-storage/crc-storage-crc-2vwmq" Dec 03 12:16:06 crc kubenswrapper[5052]: I1203 12:16:06.142716 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-bq429\" (UniqueName: \"kubernetes.io/projected/13415b4d-4d47-4358-8b8e-9caf074193b6-kube-api-access-bq429\") pod \"crc-storage-crc-2vwmq\" (UID: \"13415b4d-4d47-4358-8b8e-9caf074193b6\") " pod="crc-storage/crc-storage-crc-2vwmq" Dec 03 12:16:06 crc kubenswrapper[5052]: I1203 12:16:06.142869 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/13415b4d-4d47-4358-8b8e-9caf074193b6-node-mnt\") pod \"crc-storage-crc-2vwmq\" (UID: \"13415b4d-4d47-4358-8b8e-9caf074193b6\") " pod="crc-storage/crc-storage-crc-2vwmq" Dec 03 12:16:06 crc kubenswrapper[5052]: I1203 12:16:06.143892 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/13415b4d-4d47-4358-8b8e-9caf074193b6-crc-storage\") pod \"crc-storage-crc-2vwmq\" (UID: \"13415b4d-4d47-4358-8b8e-9caf074193b6\") " pod="crc-storage/crc-storage-crc-2vwmq" Dec 03 12:16:06 crc kubenswrapper[5052]: I1203 12:16:06.160886 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-bq429\" (UniqueName: \"kubernetes.io/projected/13415b4d-4d47-4358-8b8e-9caf074193b6-kube-api-access-bq429\") pod \"crc-storage-crc-2vwmq\" (UID: \"13415b4d-4d47-4358-8b8e-9caf074193b6\") " pod="crc-storage/crc-storage-crc-2vwmq" Dec 03 12:16:06 crc kubenswrapper[5052]: I1203 12:16:06.256248 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="crc-storage/crc-storage-crc-2vwmq" Dec 03 12:16:06 crc kubenswrapper[5052]: I1203 12:16:06.540590 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["crc-storage/crc-storage-crc-2vwmq"] Dec 03 12:16:07 crc kubenswrapper[5052]: I1203 12:16:07.289427 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="crc-storage/crc-storage-crc-2vwmq" event={"ID":"13415b4d-4d47-4358-8b8e-9caf074193b6","Type":"ContainerStarted","Data":"1d1c8825e8abc7a0eac36abbb77d15150faeb55a0c3f227698155c78c8161192"} Dec 03 12:16:07 crc kubenswrapper[5052]: I1203 12:16:07.683897 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6c7ba7e0-b39e-432d-bda2-da1fb27e197a" path="/var/lib/kubelet/pods/6c7ba7e0-b39e-432d-bda2-da1fb27e197a/volumes" Dec 03 12:16:08 crc kubenswrapper[5052]: I1203 12:16:08.313098 5052 generic.go:334] "Generic (PLEG): container finished" podID="13415b4d-4d47-4358-8b8e-9caf074193b6" containerID="f6d888d44496d050d0248b653846925f4cb1991f287b175e7c992815d8fc3e03" exitCode=0 Dec 03 12:16:08 crc kubenswrapper[5052]: I1203 12:16:08.313173 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="crc-storage/crc-storage-crc-2vwmq" event={"ID":"13415b4d-4d47-4358-8b8e-9caf074193b6","Type":"ContainerDied","Data":"f6d888d44496d050d0248b653846925f4cb1991f287b175e7c992815d8fc3e03"} Dec 03 12:16:09 crc kubenswrapper[5052]: I1203 12:16:09.654636 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="crc-storage/crc-storage-crc-2vwmq" Dec 03 12:16:09 crc kubenswrapper[5052]: I1203 12:16:09.801649 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bq429\" (UniqueName: \"kubernetes.io/projected/13415b4d-4d47-4358-8b8e-9caf074193b6-kube-api-access-bq429\") pod \"13415b4d-4d47-4358-8b8e-9caf074193b6\" (UID: \"13415b4d-4d47-4358-8b8e-9caf074193b6\") " Dec 03 12:16:09 crc kubenswrapper[5052]: I1203 12:16:09.802052 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/13415b4d-4d47-4358-8b8e-9caf074193b6-crc-storage\") pod \"13415b4d-4d47-4358-8b8e-9caf074193b6\" (UID: \"13415b4d-4d47-4358-8b8e-9caf074193b6\") " Dec 03 12:16:09 crc kubenswrapper[5052]: I1203 12:16:09.802131 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/13415b4d-4d47-4358-8b8e-9caf074193b6-node-mnt\") pod \"13415b4d-4d47-4358-8b8e-9caf074193b6\" (UID: \"13415b4d-4d47-4358-8b8e-9caf074193b6\") " Dec 03 12:16:09 crc kubenswrapper[5052]: I1203 12:16:09.802382 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/13415b4d-4d47-4358-8b8e-9caf074193b6-node-mnt" (OuterVolumeSpecName: "node-mnt") pod "13415b4d-4d47-4358-8b8e-9caf074193b6" (UID: "13415b4d-4d47-4358-8b8e-9caf074193b6"). InnerVolumeSpecName "node-mnt". PluginName "kubernetes.io/host-path", VolumeGidValue "" Dec 03 12:16:09 crc kubenswrapper[5052]: I1203 12:16:09.802678 5052 reconciler_common.go:293] "Volume detached for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/13415b4d-4d47-4358-8b8e-9caf074193b6-node-mnt\") on node \"crc\" DevicePath \"\"" Dec 03 12:16:09 crc kubenswrapper[5052]: I1203 12:16:09.806598 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/13415b4d-4d47-4358-8b8e-9caf074193b6-kube-api-access-bq429" (OuterVolumeSpecName: "kube-api-access-bq429") pod "13415b4d-4d47-4358-8b8e-9caf074193b6" (UID: "13415b4d-4d47-4358-8b8e-9caf074193b6"). InnerVolumeSpecName "kube-api-access-bq429". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:16:09 crc kubenswrapper[5052]: I1203 12:16:09.819709 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/13415b4d-4d47-4358-8b8e-9caf074193b6-crc-storage" (OuterVolumeSpecName: "crc-storage") pod "13415b4d-4d47-4358-8b8e-9caf074193b6" (UID: "13415b4d-4d47-4358-8b8e-9caf074193b6"). InnerVolumeSpecName "crc-storage". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:16:09 crc kubenswrapper[5052]: I1203 12:16:09.903956 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-bq429\" (UniqueName: \"kubernetes.io/projected/13415b4d-4d47-4358-8b8e-9caf074193b6-kube-api-access-bq429\") on node \"crc\" DevicePath \"\"" Dec 03 12:16:09 crc kubenswrapper[5052]: I1203 12:16:09.903996 5052 reconciler_common.go:293] "Volume detached for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/13415b4d-4d47-4358-8b8e-9caf074193b6-crc-storage\") on node \"crc\" DevicePath \"\"" Dec 03 12:16:10 crc kubenswrapper[5052]: I1203 12:16:10.333185 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="crc-storage/crc-storage-crc-2vwmq" event={"ID":"13415b4d-4d47-4358-8b8e-9caf074193b6","Type":"ContainerDied","Data":"1d1c8825e8abc7a0eac36abbb77d15150faeb55a0c3f227698155c78c8161192"} Dec 03 12:16:10 crc kubenswrapper[5052]: I1203 12:16:10.333237 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1d1c8825e8abc7a0eac36abbb77d15150faeb55a0c3f227698155c78c8161192" Dec 03 12:16:10 crc kubenswrapper[5052]: I1203 12:16:10.333274 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="crc-storage/crc-storage-crc-2vwmq" Dec 03 12:16:13 crc kubenswrapper[5052]: I1203 12:16:13.503263 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["crc-storage/crc-storage-crc-2vwmq"] Dec 03 12:16:13 crc kubenswrapper[5052]: I1203 12:16:13.509600 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["crc-storage/crc-storage-crc-2vwmq"] Dec 03 12:16:13 crc kubenswrapper[5052]: I1203 12:16:13.647795 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["crc-storage/crc-storage-crc-xtx4j"] Dec 03 12:16:13 crc kubenswrapper[5052]: E1203 12:16:13.648261 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="13415b4d-4d47-4358-8b8e-9caf074193b6" containerName="storage" Dec 03 12:16:13 crc kubenswrapper[5052]: I1203 12:16:13.648287 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="13415b4d-4d47-4358-8b8e-9caf074193b6" containerName="storage" Dec 03 12:16:13 crc kubenswrapper[5052]: I1203 12:16:13.648495 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="13415b4d-4d47-4358-8b8e-9caf074193b6" containerName="storage" Dec 03 12:16:13 crc kubenswrapper[5052]: I1203 12:16:13.649121 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="crc-storage/crc-storage-crc-xtx4j" Dec 03 12:16:13 crc kubenswrapper[5052]: I1203 12:16:13.650723 5052 reflector.go:368] Caches populated for *v1.Secret from object-"crc-storage"/"crc-storage-dockercfg-mf76g" Dec 03 12:16:13 crc kubenswrapper[5052]: I1203 12:16:13.652087 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"crc-storage"/"crc-storage" Dec 03 12:16:13 crc kubenswrapper[5052]: I1203 12:16:13.652109 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"crc-storage"/"openshift-service-ca.crt" Dec 03 12:16:13 crc kubenswrapper[5052]: I1203 12:16:13.653169 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"crc-storage"/"kube-root-ca.crt" Dec 03 12:16:13 crc kubenswrapper[5052]: I1203 12:16:13.661532 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["crc-storage/crc-storage-crc-xtx4j"] Dec 03 12:16:13 crc kubenswrapper[5052]: I1203 12:16:13.695069 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="13415b4d-4d47-4358-8b8e-9caf074193b6" path="/var/lib/kubelet/pods/13415b4d-4d47-4358-8b8e-9caf074193b6/volumes" Dec 03 12:16:13 crc kubenswrapper[5052]: I1203 12:16:13.775491 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/9ca00ad6-dad7-495c-bfcc-ca4b5739633c-crc-storage\") pod \"crc-storage-crc-xtx4j\" (UID: \"9ca00ad6-dad7-495c-bfcc-ca4b5739633c\") " pod="crc-storage/crc-storage-crc-xtx4j" Dec 03 12:16:13 crc kubenswrapper[5052]: I1203 12:16:13.775694 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4xrg7\" (UniqueName: \"kubernetes.io/projected/9ca00ad6-dad7-495c-bfcc-ca4b5739633c-kube-api-access-4xrg7\") pod \"crc-storage-crc-xtx4j\" (UID: \"9ca00ad6-dad7-495c-bfcc-ca4b5739633c\") " pod="crc-storage/crc-storage-crc-xtx4j" Dec 03 12:16:13 crc kubenswrapper[5052]: I1203 12:16:13.775740 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/9ca00ad6-dad7-495c-bfcc-ca4b5739633c-node-mnt\") pod \"crc-storage-crc-xtx4j\" (UID: \"9ca00ad6-dad7-495c-bfcc-ca4b5739633c\") " pod="crc-storage/crc-storage-crc-xtx4j" Dec 03 12:16:13 crc kubenswrapper[5052]: I1203 12:16:13.877370 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-4xrg7\" (UniqueName: \"kubernetes.io/projected/9ca00ad6-dad7-495c-bfcc-ca4b5739633c-kube-api-access-4xrg7\") pod \"crc-storage-crc-xtx4j\" (UID: \"9ca00ad6-dad7-495c-bfcc-ca4b5739633c\") " pod="crc-storage/crc-storage-crc-xtx4j" Dec 03 12:16:13 crc kubenswrapper[5052]: I1203 12:16:13.877426 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/9ca00ad6-dad7-495c-bfcc-ca4b5739633c-node-mnt\") pod \"crc-storage-crc-xtx4j\" (UID: \"9ca00ad6-dad7-495c-bfcc-ca4b5739633c\") " pod="crc-storage/crc-storage-crc-xtx4j" Dec 03 12:16:13 crc kubenswrapper[5052]: I1203 12:16:13.877505 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/9ca00ad6-dad7-495c-bfcc-ca4b5739633c-crc-storage\") pod \"crc-storage-crc-xtx4j\" (UID: \"9ca00ad6-dad7-495c-bfcc-ca4b5739633c\") " pod="crc-storage/crc-storage-crc-xtx4j" Dec 03 12:16:13 crc kubenswrapper[5052]: I1203 12:16:13.878255 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/9ca00ad6-dad7-495c-bfcc-ca4b5739633c-crc-storage\") pod \"crc-storage-crc-xtx4j\" (UID: \"9ca00ad6-dad7-495c-bfcc-ca4b5739633c\") " pod="crc-storage/crc-storage-crc-xtx4j" Dec 03 12:16:13 crc kubenswrapper[5052]: I1203 12:16:13.878694 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/9ca00ad6-dad7-495c-bfcc-ca4b5739633c-node-mnt\") pod \"crc-storage-crc-xtx4j\" (UID: \"9ca00ad6-dad7-495c-bfcc-ca4b5739633c\") " pod="crc-storage/crc-storage-crc-xtx4j" Dec 03 12:16:13 crc kubenswrapper[5052]: I1203 12:16:13.896802 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-4xrg7\" (UniqueName: \"kubernetes.io/projected/9ca00ad6-dad7-495c-bfcc-ca4b5739633c-kube-api-access-4xrg7\") pod \"crc-storage-crc-xtx4j\" (UID: \"9ca00ad6-dad7-495c-bfcc-ca4b5739633c\") " pod="crc-storage/crc-storage-crc-xtx4j" Dec 03 12:16:13 crc kubenswrapper[5052]: I1203 12:16:13.992811 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="crc-storage/crc-storage-crc-xtx4j" Dec 03 12:16:14 crc kubenswrapper[5052]: I1203 12:16:14.432910 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["crc-storage/crc-storage-crc-xtx4j"] Dec 03 12:16:15 crc kubenswrapper[5052]: E1203 12:16:15.273935 5052 cadvisor_stats_provider.go:516] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9ca00ad6_dad7_495c_bfcc_ca4b5739633c.slice/crio-conmon-64bb7856e52629460376e6405674735686ad3f33769a348e6d4f9ecbb1bdc284.scope\": RecentStats: unable to find data in memory cache]" Dec 03 12:16:15 crc kubenswrapper[5052]: I1203 12:16:15.388037 5052 generic.go:334] "Generic (PLEG): container finished" podID="9ca00ad6-dad7-495c-bfcc-ca4b5739633c" containerID="64bb7856e52629460376e6405674735686ad3f33769a348e6d4f9ecbb1bdc284" exitCode=0 Dec 03 12:16:15 crc kubenswrapper[5052]: I1203 12:16:15.388089 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="crc-storage/crc-storage-crc-xtx4j" event={"ID":"9ca00ad6-dad7-495c-bfcc-ca4b5739633c","Type":"ContainerDied","Data":"64bb7856e52629460376e6405674735686ad3f33769a348e6d4f9ecbb1bdc284"} Dec 03 12:16:15 crc kubenswrapper[5052]: I1203 12:16:15.388127 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="crc-storage/crc-storage-crc-xtx4j" event={"ID":"9ca00ad6-dad7-495c-bfcc-ca4b5739633c","Type":"ContainerStarted","Data":"1e1e4d3a94bed3d8136caa07ad37ec4cb560d77d2b32cce26c20162d6c986d28"} Dec 03 12:16:16 crc kubenswrapper[5052]: I1203 12:16:16.717713 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="crc-storage/crc-storage-crc-xtx4j" Dec 03 12:16:16 crc kubenswrapper[5052]: I1203 12:16:16.816468 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/9ca00ad6-dad7-495c-bfcc-ca4b5739633c-node-mnt\") pod \"9ca00ad6-dad7-495c-bfcc-ca4b5739633c\" (UID: \"9ca00ad6-dad7-495c-bfcc-ca4b5739633c\") " Dec 03 12:16:16 crc kubenswrapper[5052]: I1203 12:16:16.816662 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/9ca00ad6-dad7-495c-bfcc-ca4b5739633c-node-mnt" (OuterVolumeSpecName: "node-mnt") pod "9ca00ad6-dad7-495c-bfcc-ca4b5739633c" (UID: "9ca00ad6-dad7-495c-bfcc-ca4b5739633c"). InnerVolumeSpecName "node-mnt". PluginName "kubernetes.io/host-path", VolumeGidValue "" Dec 03 12:16:16 crc kubenswrapper[5052]: I1203 12:16:16.816626 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4xrg7\" (UniqueName: \"kubernetes.io/projected/9ca00ad6-dad7-495c-bfcc-ca4b5739633c-kube-api-access-4xrg7\") pod \"9ca00ad6-dad7-495c-bfcc-ca4b5739633c\" (UID: \"9ca00ad6-dad7-495c-bfcc-ca4b5739633c\") " Dec 03 12:16:16 crc kubenswrapper[5052]: I1203 12:16:16.816945 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/9ca00ad6-dad7-495c-bfcc-ca4b5739633c-crc-storage\") pod \"9ca00ad6-dad7-495c-bfcc-ca4b5739633c\" (UID: \"9ca00ad6-dad7-495c-bfcc-ca4b5739633c\") " Dec 03 12:16:16 crc kubenswrapper[5052]: I1203 12:16:16.818261 5052 reconciler_common.go:293] "Volume detached for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/9ca00ad6-dad7-495c-bfcc-ca4b5739633c-node-mnt\") on node \"crc\" DevicePath \"\"" Dec 03 12:16:16 crc kubenswrapper[5052]: I1203 12:16:16.822848 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9ca00ad6-dad7-495c-bfcc-ca4b5739633c-kube-api-access-4xrg7" (OuterVolumeSpecName: "kube-api-access-4xrg7") pod "9ca00ad6-dad7-495c-bfcc-ca4b5739633c" (UID: "9ca00ad6-dad7-495c-bfcc-ca4b5739633c"). InnerVolumeSpecName "kube-api-access-4xrg7". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:16:16 crc kubenswrapper[5052]: I1203 12:16:16.840987 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9ca00ad6-dad7-495c-bfcc-ca4b5739633c-crc-storage" (OuterVolumeSpecName: "crc-storage") pod "9ca00ad6-dad7-495c-bfcc-ca4b5739633c" (UID: "9ca00ad6-dad7-495c-bfcc-ca4b5739633c"). InnerVolumeSpecName "crc-storage". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:16:16 crc kubenswrapper[5052]: I1203 12:16:16.920162 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-4xrg7\" (UniqueName: \"kubernetes.io/projected/9ca00ad6-dad7-495c-bfcc-ca4b5739633c-kube-api-access-4xrg7\") on node \"crc\" DevicePath \"\"" Dec 03 12:16:16 crc kubenswrapper[5052]: I1203 12:16:16.920197 5052 reconciler_common.go:293] "Volume detached for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/9ca00ad6-dad7-495c-bfcc-ca4b5739633c-crc-storage\") on node \"crc\" DevicePath \"\"" Dec 03 12:16:17 crc kubenswrapper[5052]: I1203 12:16:17.409141 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="crc-storage/crc-storage-crc-xtx4j" event={"ID":"9ca00ad6-dad7-495c-bfcc-ca4b5739633c","Type":"ContainerDied","Data":"1e1e4d3a94bed3d8136caa07ad37ec4cb560d77d2b32cce26c20162d6c986d28"} Dec 03 12:16:17 crc kubenswrapper[5052]: I1203 12:16:17.409456 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1e1e4d3a94bed3d8136caa07ad37ec4cb560d77d2b32cce26c20162d6c986d28" Dec 03 12:16:17 crc kubenswrapper[5052]: I1203 12:16:17.409196 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="crc-storage/crc-storage-crc-xtx4j" Dec 03 12:16:29 crc kubenswrapper[5052]: I1203 12:16:29.380375 5052 patch_prober.go:28] interesting pod/machine-config-daemon-74bj6 container/machine-config-daemon namespace/openshift-machine-config-operator: Liveness probe status=failure output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" start-of-body= Dec 03 12:16:29 crc kubenswrapper[5052]: I1203 12:16:29.381051 5052 prober.go:107] "Probe failed" probeType="Liveness" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" containerName="machine-config-daemon" probeResult="failure" output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" Dec 03 12:16:30 crc kubenswrapper[5052]: I1203 12:16:30.681912 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/rabbitmq-cell1-server-0"] Dec 03 12:16:30 crc kubenswrapper[5052]: E1203 12:16:30.682646 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="9ca00ad6-dad7-495c-bfcc-ca4b5739633c" containerName="storage" Dec 03 12:16:30 crc kubenswrapper[5052]: I1203 12:16:30.682662 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ca00ad6-dad7-495c-bfcc-ca4b5739633c" containerName="storage" Dec 03 12:16:30 crc kubenswrapper[5052]: I1203 12:16:30.682838 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="9ca00ad6-dad7-495c-bfcc-ca4b5739633c" containerName="storage" Dec 03 12:16:30 crc kubenswrapper[5052]: I1203 12:16:30.683646 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:16:30 crc kubenswrapper[5052]: I1203 12:16:30.685742 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"rabbitmq-cell1-erlang-cookie" Dec 03 12:16:30 crc kubenswrapper[5052]: I1203 12:16:30.686333 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"rabbitmq-cell1-config-data" Dec 03 12:16:30 crc kubenswrapper[5052]: I1203 12:16:30.686365 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"rabbitmq-cell1-default-user" Dec 03 12:16:30 crc kubenswrapper[5052]: I1203 12:16:30.686405 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-rabbitmq-cell1-svc" Dec 03 12:16:30 crc kubenswrapper[5052]: I1203 12:16:30.686442 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"rabbitmq-cell1-server-conf" Dec 03 12:16:30 crc kubenswrapper[5052]: I1203 12:16:30.686524 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"rabbitmq-cell1-plugins-conf" Dec 03 12:16:30 crc kubenswrapper[5052]: I1203 12:16:30.687145 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"rabbitmq-cell1-server-dockercfg-bwtz2" Dec 03 12:16:30 crc kubenswrapper[5052]: I1203 12:16:30.700761 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/rabbitmq-cell1-server-0"] Dec 03 12:16:30 crc kubenswrapper[5052]: I1203 12:16:30.825910 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"rabbitmq-erlang-cookie\" (UniqueName: \"kubernetes.io/empty-dir/1ec6b1f7-5abb-4063-80fa-b5ec907dcb91-rabbitmq-erlang-cookie\") pod \"rabbitmq-cell1-server-0\" (UID: \"1ec6b1f7-5abb-4063-80fa-b5ec907dcb91\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:16:30 crc kubenswrapper[5052]: I1203 12:16:30.826016 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"server-conf\" (UniqueName: \"kubernetes.io/configmap/1ec6b1f7-5abb-4063-80fa-b5ec907dcb91-server-conf\") pod \"rabbitmq-cell1-server-0\" (UID: \"1ec6b1f7-5abb-4063-80fa-b5ec907dcb91\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:16:30 crc kubenswrapper[5052]: I1203 12:16:30.826169 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9zzmt\" (UniqueName: \"kubernetes.io/projected/1ec6b1f7-5abb-4063-80fa-b5ec907dcb91-kube-api-access-9zzmt\") pod \"rabbitmq-cell1-server-0\" (UID: \"1ec6b1f7-5abb-4063-80fa-b5ec907dcb91\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:16:30 crc kubenswrapper[5052]: I1203 12:16:30.826254 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"rabbitmq-tls\" (UniqueName: \"kubernetes.io/projected/1ec6b1f7-5abb-4063-80fa-b5ec907dcb91-rabbitmq-tls\") pod \"rabbitmq-cell1-server-0\" (UID: \"1ec6b1f7-5abb-4063-80fa-b5ec907dcb91\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:16:30 crc kubenswrapper[5052]: I1203 12:16:30.826317 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/1ec6b1f7-5abb-4063-80fa-b5ec907dcb91-config-data\") pod \"rabbitmq-cell1-server-0\" (UID: \"1ec6b1f7-5abb-4063-80fa-b5ec907dcb91\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:16:30 crc kubenswrapper[5052]: I1203 12:16:30.826367 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"rabbitmq-confd\" (UniqueName: \"kubernetes.io/projected/1ec6b1f7-5abb-4063-80fa-b5ec907dcb91-rabbitmq-confd\") pod \"rabbitmq-cell1-server-0\" (UID: \"1ec6b1f7-5abb-4063-80fa-b5ec907dcb91\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:16:30 crc kubenswrapper[5052]: I1203 12:16:30.826395 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-conf\" (UniqueName: \"kubernetes.io/configmap/1ec6b1f7-5abb-4063-80fa-b5ec907dcb91-plugins-conf\") pod \"rabbitmq-cell1-server-0\" (UID: \"1ec6b1f7-5abb-4063-80fa-b5ec907dcb91\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:16:30 crc kubenswrapper[5052]: I1203 12:16:30.826436 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage15-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage15-crc\") pod \"rabbitmq-cell1-server-0\" (UID: \"1ec6b1f7-5abb-4063-80fa-b5ec907dcb91\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:16:30 crc kubenswrapper[5052]: I1203 12:16:30.826466 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"erlang-cookie-secret\" (UniqueName: \"kubernetes.io/secret/1ec6b1f7-5abb-4063-80fa-b5ec907dcb91-erlang-cookie-secret\") pod \"rabbitmq-cell1-server-0\" (UID: \"1ec6b1f7-5abb-4063-80fa-b5ec907dcb91\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:16:30 crc kubenswrapper[5052]: I1203 12:16:30.826486 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"pod-info\" (UniqueName: \"kubernetes.io/downward-api/1ec6b1f7-5abb-4063-80fa-b5ec907dcb91-pod-info\") pod \"rabbitmq-cell1-server-0\" (UID: \"1ec6b1f7-5abb-4063-80fa-b5ec907dcb91\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:16:30 crc kubenswrapper[5052]: I1203 12:16:30.826553 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"rabbitmq-plugins\" (UniqueName: \"kubernetes.io/empty-dir/1ec6b1f7-5abb-4063-80fa-b5ec907dcb91-rabbitmq-plugins\") pod \"rabbitmq-cell1-server-0\" (UID: \"1ec6b1f7-5abb-4063-80fa-b5ec907dcb91\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:16:30 crc kubenswrapper[5052]: I1203 12:16:30.928218 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"rabbitmq-plugins\" (UniqueName: \"kubernetes.io/empty-dir/1ec6b1f7-5abb-4063-80fa-b5ec907dcb91-rabbitmq-plugins\") pod \"rabbitmq-cell1-server-0\" (UID: \"1ec6b1f7-5abb-4063-80fa-b5ec907dcb91\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:16:30 crc kubenswrapper[5052]: I1203 12:16:30.928287 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"rabbitmq-erlang-cookie\" (UniqueName: \"kubernetes.io/empty-dir/1ec6b1f7-5abb-4063-80fa-b5ec907dcb91-rabbitmq-erlang-cookie\") pod \"rabbitmq-cell1-server-0\" (UID: \"1ec6b1f7-5abb-4063-80fa-b5ec907dcb91\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:16:30 crc kubenswrapper[5052]: I1203 12:16:30.928310 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"server-conf\" (UniqueName: \"kubernetes.io/configmap/1ec6b1f7-5abb-4063-80fa-b5ec907dcb91-server-conf\") pod \"rabbitmq-cell1-server-0\" (UID: \"1ec6b1f7-5abb-4063-80fa-b5ec907dcb91\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:16:30 crc kubenswrapper[5052]: I1203 12:16:30.928333 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-9zzmt\" (UniqueName: \"kubernetes.io/projected/1ec6b1f7-5abb-4063-80fa-b5ec907dcb91-kube-api-access-9zzmt\") pod \"rabbitmq-cell1-server-0\" (UID: \"1ec6b1f7-5abb-4063-80fa-b5ec907dcb91\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:16:30 crc kubenswrapper[5052]: I1203 12:16:30.928354 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"rabbitmq-tls\" (UniqueName: \"kubernetes.io/projected/1ec6b1f7-5abb-4063-80fa-b5ec907dcb91-rabbitmq-tls\") pod \"rabbitmq-cell1-server-0\" (UID: \"1ec6b1f7-5abb-4063-80fa-b5ec907dcb91\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:16:30 crc kubenswrapper[5052]: I1203 12:16:30.928376 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/1ec6b1f7-5abb-4063-80fa-b5ec907dcb91-config-data\") pod \"rabbitmq-cell1-server-0\" (UID: \"1ec6b1f7-5abb-4063-80fa-b5ec907dcb91\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:16:30 crc kubenswrapper[5052]: I1203 12:16:30.928395 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"rabbitmq-confd\" (UniqueName: \"kubernetes.io/projected/1ec6b1f7-5abb-4063-80fa-b5ec907dcb91-rabbitmq-confd\") pod \"rabbitmq-cell1-server-0\" (UID: \"1ec6b1f7-5abb-4063-80fa-b5ec907dcb91\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:16:30 crc kubenswrapper[5052]: I1203 12:16:30.928415 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"plugins-conf\" (UniqueName: \"kubernetes.io/configmap/1ec6b1f7-5abb-4063-80fa-b5ec907dcb91-plugins-conf\") pod \"rabbitmq-cell1-server-0\" (UID: \"1ec6b1f7-5abb-4063-80fa-b5ec907dcb91\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:16:30 crc kubenswrapper[5052]: I1203 12:16:30.928451 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage15-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage15-crc\") pod \"rabbitmq-cell1-server-0\" (UID: \"1ec6b1f7-5abb-4063-80fa-b5ec907dcb91\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:16:30 crc kubenswrapper[5052]: I1203 12:16:30.928477 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"erlang-cookie-secret\" (UniqueName: \"kubernetes.io/secret/1ec6b1f7-5abb-4063-80fa-b5ec907dcb91-erlang-cookie-secret\") pod \"rabbitmq-cell1-server-0\" (UID: \"1ec6b1f7-5abb-4063-80fa-b5ec907dcb91\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:16:30 crc kubenswrapper[5052]: I1203 12:16:30.928496 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"pod-info\" (UniqueName: \"kubernetes.io/downward-api/1ec6b1f7-5abb-4063-80fa-b5ec907dcb91-pod-info\") pod \"rabbitmq-cell1-server-0\" (UID: \"1ec6b1f7-5abb-4063-80fa-b5ec907dcb91\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:16:30 crc kubenswrapper[5052]: I1203 12:16:30.928821 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"rabbitmq-plugins\" (UniqueName: \"kubernetes.io/empty-dir/1ec6b1f7-5abb-4063-80fa-b5ec907dcb91-rabbitmq-plugins\") pod \"rabbitmq-cell1-server-0\" (UID: \"1ec6b1f7-5abb-4063-80fa-b5ec907dcb91\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:16:30 crc kubenswrapper[5052]: I1203 12:16:30.929015 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"rabbitmq-erlang-cookie\" (UniqueName: \"kubernetes.io/empty-dir/1ec6b1f7-5abb-4063-80fa-b5ec907dcb91-rabbitmq-erlang-cookie\") pod \"rabbitmq-cell1-server-0\" (UID: \"1ec6b1f7-5abb-4063-80fa-b5ec907dcb91\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:16:30 crc kubenswrapper[5052]: I1203 12:16:30.930110 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/1ec6b1f7-5abb-4063-80fa-b5ec907dcb91-config-data\") pod \"rabbitmq-cell1-server-0\" (UID: \"1ec6b1f7-5abb-4063-80fa-b5ec907dcb91\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:16:30 crc kubenswrapper[5052]: I1203 12:16:30.930298 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage15-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage15-crc\") pod \"rabbitmq-cell1-server-0\" (UID: \"1ec6b1f7-5abb-4063-80fa-b5ec907dcb91\") device mount path \"/mnt/openstack/pv15\"" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:16:30 crc kubenswrapper[5052]: I1203 12:16:30.931313 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"server-conf\" (UniqueName: \"kubernetes.io/configmap/1ec6b1f7-5abb-4063-80fa-b5ec907dcb91-server-conf\") pod \"rabbitmq-cell1-server-0\" (UID: \"1ec6b1f7-5abb-4063-80fa-b5ec907dcb91\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:16:30 crc kubenswrapper[5052]: I1203 12:16:30.933205 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"plugins-conf\" (UniqueName: \"kubernetes.io/configmap/1ec6b1f7-5abb-4063-80fa-b5ec907dcb91-plugins-conf\") pod \"rabbitmq-cell1-server-0\" (UID: \"1ec6b1f7-5abb-4063-80fa-b5ec907dcb91\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:16:30 crc kubenswrapper[5052]: I1203 12:16:30.935868 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"rabbitmq-confd\" (UniqueName: \"kubernetes.io/projected/1ec6b1f7-5abb-4063-80fa-b5ec907dcb91-rabbitmq-confd\") pod \"rabbitmq-cell1-server-0\" (UID: \"1ec6b1f7-5abb-4063-80fa-b5ec907dcb91\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:16:30 crc kubenswrapper[5052]: I1203 12:16:30.942350 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"rabbitmq-tls\" (UniqueName: \"kubernetes.io/projected/1ec6b1f7-5abb-4063-80fa-b5ec907dcb91-rabbitmq-tls\") pod \"rabbitmq-cell1-server-0\" (UID: \"1ec6b1f7-5abb-4063-80fa-b5ec907dcb91\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:16:30 crc kubenswrapper[5052]: I1203 12:16:30.944968 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"erlang-cookie-secret\" (UniqueName: \"kubernetes.io/secret/1ec6b1f7-5abb-4063-80fa-b5ec907dcb91-erlang-cookie-secret\") pod \"rabbitmq-cell1-server-0\" (UID: \"1ec6b1f7-5abb-4063-80fa-b5ec907dcb91\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:16:30 crc kubenswrapper[5052]: I1203 12:16:30.947116 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"pod-info\" (UniqueName: \"kubernetes.io/downward-api/1ec6b1f7-5abb-4063-80fa-b5ec907dcb91-pod-info\") pod \"rabbitmq-cell1-server-0\" (UID: \"1ec6b1f7-5abb-4063-80fa-b5ec907dcb91\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:16:30 crc kubenswrapper[5052]: I1203 12:16:30.947819 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-9zzmt\" (UniqueName: \"kubernetes.io/projected/1ec6b1f7-5abb-4063-80fa-b5ec907dcb91-kube-api-access-9zzmt\") pod \"rabbitmq-cell1-server-0\" (UID: \"1ec6b1f7-5abb-4063-80fa-b5ec907dcb91\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:16:30 crc kubenswrapper[5052]: I1203 12:16:30.954755 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage15-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage15-crc\") pod \"rabbitmq-cell1-server-0\" (UID: \"1ec6b1f7-5abb-4063-80fa-b5ec907dcb91\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:16:31 crc kubenswrapper[5052]: I1203 12:16:31.004944 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:16:31 crc kubenswrapper[5052]: I1203 12:16:31.476672 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/rabbitmq-cell1-server-0"] Dec 03 12:16:31 crc kubenswrapper[5052]: I1203 12:16:31.536229 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" event={"ID":"1ec6b1f7-5abb-4063-80fa-b5ec907dcb91","Type":"ContainerStarted","Data":"95fe4ec675f1eef64da36705d4a890480cb0a106844bd69b52f083b1f0bd392d"} Dec 03 12:16:31 crc kubenswrapper[5052]: I1203 12:16:31.588140 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/rabbitmq-server-0"] Dec 03 12:16:31 crc kubenswrapper[5052]: I1203 12:16:31.591660 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:16:31 crc kubenswrapper[5052]: I1203 12:16:31.599879 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"rabbitmq-plugins-conf" Dec 03 12:16:31 crc kubenswrapper[5052]: I1203 12:16:31.600116 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"rabbitmq-config-data" Dec 03 12:16:31 crc kubenswrapper[5052]: I1203 12:16:31.600245 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-rabbitmq-svc" Dec 03 12:16:31 crc kubenswrapper[5052]: I1203 12:16:31.600294 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"rabbitmq-default-user" Dec 03 12:16:31 crc kubenswrapper[5052]: I1203 12:16:31.600365 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"rabbitmq-server-dockercfg-stb9x" Dec 03 12:16:31 crc kubenswrapper[5052]: I1203 12:16:31.600472 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"rabbitmq-server-conf" Dec 03 12:16:31 crc kubenswrapper[5052]: I1203 12:16:31.600645 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"rabbitmq-erlang-cookie" Dec 03 12:16:31 crc kubenswrapper[5052]: I1203 12:16:31.604781 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/rabbitmq-server-0"] Dec 03 12:16:31 crc kubenswrapper[5052]: I1203 12:16:31.745447 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"server-conf\" (UniqueName: \"kubernetes.io/configmap/683b7d5f-a88a-4dc0-ae83-af3c6444c7ff-server-conf\") pod \"rabbitmq-server-0\" (UID: \"683b7d5f-a88a-4dc0-ae83-af3c6444c7ff\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:16:31 crc kubenswrapper[5052]: I1203 12:16:31.745500 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"rabbitmq-plugins\" (UniqueName: \"kubernetes.io/empty-dir/683b7d5f-a88a-4dc0-ae83-af3c6444c7ff-rabbitmq-plugins\") pod \"rabbitmq-server-0\" (UID: \"683b7d5f-a88a-4dc0-ae83-af3c6444c7ff\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:16:31 crc kubenswrapper[5052]: I1203 12:16:31.745551 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"rabbitmq-confd\" (UniqueName: \"kubernetes.io/projected/683b7d5f-a88a-4dc0-ae83-af3c6444c7ff-rabbitmq-confd\") pod \"rabbitmq-server-0\" (UID: \"683b7d5f-a88a-4dc0-ae83-af3c6444c7ff\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:16:31 crc kubenswrapper[5052]: I1203 12:16:31.745578 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"pod-info\" (UniqueName: \"kubernetes.io/downward-api/683b7d5f-a88a-4dc0-ae83-af3c6444c7ff-pod-info\") pod \"rabbitmq-server-0\" (UID: \"683b7d5f-a88a-4dc0-ae83-af3c6444c7ff\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:16:31 crc kubenswrapper[5052]: I1203 12:16:31.745660 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/683b7d5f-a88a-4dc0-ae83-af3c6444c7ff-config-data\") pod \"rabbitmq-server-0\" (UID: \"683b7d5f-a88a-4dc0-ae83-af3c6444c7ff\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:16:31 crc kubenswrapper[5052]: I1203 12:16:31.745687 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"rabbitmq-tls\" (UniqueName: \"kubernetes.io/projected/683b7d5f-a88a-4dc0-ae83-af3c6444c7ff-rabbitmq-tls\") pod \"rabbitmq-server-0\" (UID: \"683b7d5f-a88a-4dc0-ae83-af3c6444c7ff\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:16:31 crc kubenswrapper[5052]: I1203 12:16:31.745712 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l6665\" (UniqueName: \"kubernetes.io/projected/683b7d5f-a88a-4dc0-ae83-af3c6444c7ff-kube-api-access-l6665\") pod \"rabbitmq-server-0\" (UID: \"683b7d5f-a88a-4dc0-ae83-af3c6444c7ff\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:16:31 crc kubenswrapper[5052]: I1203 12:16:31.745733 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"erlang-cookie-secret\" (UniqueName: \"kubernetes.io/secret/683b7d5f-a88a-4dc0-ae83-af3c6444c7ff-erlang-cookie-secret\") pod \"rabbitmq-server-0\" (UID: \"683b7d5f-a88a-4dc0-ae83-af3c6444c7ff\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:16:31 crc kubenswrapper[5052]: I1203 12:16:31.745879 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"rabbitmq-erlang-cookie\" (UniqueName: \"kubernetes.io/empty-dir/683b7d5f-a88a-4dc0-ae83-af3c6444c7ff-rabbitmq-erlang-cookie\") pod \"rabbitmq-server-0\" (UID: \"683b7d5f-a88a-4dc0-ae83-af3c6444c7ff\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:16:31 crc kubenswrapper[5052]: I1203 12:16:31.745940 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage19-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage19-crc\") pod \"rabbitmq-server-0\" (UID: \"683b7d5f-a88a-4dc0-ae83-af3c6444c7ff\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:16:31 crc kubenswrapper[5052]: I1203 12:16:31.746029 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-conf\" (UniqueName: \"kubernetes.io/configmap/683b7d5f-a88a-4dc0-ae83-af3c6444c7ff-plugins-conf\") pod \"rabbitmq-server-0\" (UID: \"683b7d5f-a88a-4dc0-ae83-af3c6444c7ff\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:16:31 crc kubenswrapper[5052]: I1203 12:16:31.847592 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"rabbitmq-plugins\" (UniqueName: \"kubernetes.io/empty-dir/683b7d5f-a88a-4dc0-ae83-af3c6444c7ff-rabbitmq-plugins\") pod \"rabbitmq-server-0\" (UID: \"683b7d5f-a88a-4dc0-ae83-af3c6444c7ff\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:16:31 crc kubenswrapper[5052]: I1203 12:16:31.847645 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"rabbitmq-confd\" (UniqueName: \"kubernetes.io/projected/683b7d5f-a88a-4dc0-ae83-af3c6444c7ff-rabbitmq-confd\") pod \"rabbitmq-server-0\" (UID: \"683b7d5f-a88a-4dc0-ae83-af3c6444c7ff\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:16:31 crc kubenswrapper[5052]: I1203 12:16:31.847693 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"pod-info\" (UniqueName: \"kubernetes.io/downward-api/683b7d5f-a88a-4dc0-ae83-af3c6444c7ff-pod-info\") pod \"rabbitmq-server-0\" (UID: \"683b7d5f-a88a-4dc0-ae83-af3c6444c7ff\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:16:31 crc kubenswrapper[5052]: I1203 12:16:31.848128 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"rabbitmq-plugins\" (UniqueName: \"kubernetes.io/empty-dir/683b7d5f-a88a-4dc0-ae83-af3c6444c7ff-rabbitmq-plugins\") pod \"rabbitmq-server-0\" (UID: \"683b7d5f-a88a-4dc0-ae83-af3c6444c7ff\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:16:31 crc kubenswrapper[5052]: I1203 12:16:31.848331 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/683b7d5f-a88a-4dc0-ae83-af3c6444c7ff-config-data\") pod \"rabbitmq-server-0\" (UID: \"683b7d5f-a88a-4dc0-ae83-af3c6444c7ff\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:16:31 crc kubenswrapper[5052]: I1203 12:16:31.848371 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"rabbitmq-tls\" (UniqueName: \"kubernetes.io/projected/683b7d5f-a88a-4dc0-ae83-af3c6444c7ff-rabbitmq-tls\") pod \"rabbitmq-server-0\" (UID: \"683b7d5f-a88a-4dc0-ae83-af3c6444c7ff\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:16:31 crc kubenswrapper[5052]: I1203 12:16:31.848564 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-l6665\" (UniqueName: \"kubernetes.io/projected/683b7d5f-a88a-4dc0-ae83-af3c6444c7ff-kube-api-access-l6665\") pod \"rabbitmq-server-0\" (UID: \"683b7d5f-a88a-4dc0-ae83-af3c6444c7ff\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:16:31 crc kubenswrapper[5052]: I1203 12:16:31.848623 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"erlang-cookie-secret\" (UniqueName: \"kubernetes.io/secret/683b7d5f-a88a-4dc0-ae83-af3c6444c7ff-erlang-cookie-secret\") pod \"rabbitmq-server-0\" (UID: \"683b7d5f-a88a-4dc0-ae83-af3c6444c7ff\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:16:31 crc kubenswrapper[5052]: I1203 12:16:31.848744 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"rabbitmq-erlang-cookie\" (UniqueName: \"kubernetes.io/empty-dir/683b7d5f-a88a-4dc0-ae83-af3c6444c7ff-rabbitmq-erlang-cookie\") pod \"rabbitmq-server-0\" (UID: \"683b7d5f-a88a-4dc0-ae83-af3c6444c7ff\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:16:31 crc kubenswrapper[5052]: I1203 12:16:31.848790 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage19-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage19-crc\") pod \"rabbitmq-server-0\" (UID: \"683b7d5f-a88a-4dc0-ae83-af3c6444c7ff\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:16:31 crc kubenswrapper[5052]: I1203 12:16:31.848878 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"plugins-conf\" (UniqueName: \"kubernetes.io/configmap/683b7d5f-a88a-4dc0-ae83-af3c6444c7ff-plugins-conf\") pod \"rabbitmq-server-0\" (UID: \"683b7d5f-a88a-4dc0-ae83-af3c6444c7ff\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:16:31 crc kubenswrapper[5052]: I1203 12:16:31.848962 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"server-conf\" (UniqueName: \"kubernetes.io/configmap/683b7d5f-a88a-4dc0-ae83-af3c6444c7ff-server-conf\") pod \"rabbitmq-server-0\" (UID: \"683b7d5f-a88a-4dc0-ae83-af3c6444c7ff\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:16:31 crc kubenswrapper[5052]: I1203 12:16:31.849219 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"rabbitmq-erlang-cookie\" (UniqueName: \"kubernetes.io/empty-dir/683b7d5f-a88a-4dc0-ae83-af3c6444c7ff-rabbitmq-erlang-cookie\") pod \"rabbitmq-server-0\" (UID: \"683b7d5f-a88a-4dc0-ae83-af3c6444c7ff\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:16:31 crc kubenswrapper[5052]: I1203 12:16:31.850270 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"server-conf\" (UniqueName: \"kubernetes.io/configmap/683b7d5f-a88a-4dc0-ae83-af3c6444c7ff-server-conf\") pod \"rabbitmq-server-0\" (UID: \"683b7d5f-a88a-4dc0-ae83-af3c6444c7ff\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:16:31 crc kubenswrapper[5052]: I1203 12:16:31.850487 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage19-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage19-crc\") pod \"rabbitmq-server-0\" (UID: \"683b7d5f-a88a-4dc0-ae83-af3c6444c7ff\") device mount path \"/mnt/openstack/pv19\"" pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:16:31 crc kubenswrapper[5052]: I1203 12:16:31.858232 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/683b7d5f-a88a-4dc0-ae83-af3c6444c7ff-config-data\") pod \"rabbitmq-server-0\" (UID: \"683b7d5f-a88a-4dc0-ae83-af3c6444c7ff\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:16:31 crc kubenswrapper[5052]: I1203 12:16:31.860193 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"plugins-conf\" (UniqueName: \"kubernetes.io/configmap/683b7d5f-a88a-4dc0-ae83-af3c6444c7ff-plugins-conf\") pod \"rabbitmq-server-0\" (UID: \"683b7d5f-a88a-4dc0-ae83-af3c6444c7ff\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:16:31 crc kubenswrapper[5052]: I1203 12:16:31.862085 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"rabbitmq-confd\" (UniqueName: \"kubernetes.io/projected/683b7d5f-a88a-4dc0-ae83-af3c6444c7ff-rabbitmq-confd\") pod \"rabbitmq-server-0\" (UID: \"683b7d5f-a88a-4dc0-ae83-af3c6444c7ff\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:16:31 crc kubenswrapper[5052]: I1203 12:16:31.869053 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"pod-info\" (UniqueName: \"kubernetes.io/downward-api/683b7d5f-a88a-4dc0-ae83-af3c6444c7ff-pod-info\") pod \"rabbitmq-server-0\" (UID: \"683b7d5f-a88a-4dc0-ae83-af3c6444c7ff\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:16:31 crc kubenswrapper[5052]: I1203 12:16:31.869058 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"erlang-cookie-secret\" (UniqueName: \"kubernetes.io/secret/683b7d5f-a88a-4dc0-ae83-af3c6444c7ff-erlang-cookie-secret\") pod \"rabbitmq-server-0\" (UID: \"683b7d5f-a88a-4dc0-ae83-af3c6444c7ff\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:16:31 crc kubenswrapper[5052]: I1203 12:16:31.869551 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"rabbitmq-tls\" (UniqueName: \"kubernetes.io/projected/683b7d5f-a88a-4dc0-ae83-af3c6444c7ff-rabbitmq-tls\") pod \"rabbitmq-server-0\" (UID: \"683b7d5f-a88a-4dc0-ae83-af3c6444c7ff\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:16:31 crc kubenswrapper[5052]: I1203 12:16:31.889939 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage19-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage19-crc\") pod \"rabbitmq-server-0\" (UID: \"683b7d5f-a88a-4dc0-ae83-af3c6444c7ff\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:16:31 crc kubenswrapper[5052]: I1203 12:16:31.892850 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-l6665\" (UniqueName: \"kubernetes.io/projected/683b7d5f-a88a-4dc0-ae83-af3c6444c7ff-kube-api-access-l6665\") pod \"rabbitmq-server-0\" (UID: \"683b7d5f-a88a-4dc0-ae83-af3c6444c7ff\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:16:31 crc kubenswrapper[5052]: I1203 12:16:31.973886 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:16:32 crc kubenswrapper[5052]: I1203 12:16:32.516418 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/rabbitmq-server-0"] Dec 03 12:16:32 crc kubenswrapper[5052]: I1203 12:16:32.548649 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-server-0" event={"ID":"683b7d5f-a88a-4dc0-ae83-af3c6444c7ff","Type":"ContainerStarted","Data":"4c300067c8cb7310d5b5d8227109abc2a348ce501f13459cfca2ab313a1ece96"} Dec 03 12:16:33 crc kubenswrapper[5052]: I1203 12:16:33.125757 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/openstack-galera-0"] Dec 03 12:16:33 crc kubenswrapper[5052]: I1203 12:16:33.127393 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:16:33 crc kubenswrapper[5052]: I1203 12:16:33.131821 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"openstack-config-data" Dec 03 12:16:33 crc kubenswrapper[5052]: I1203 12:16:33.133506 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"galera-openstack-dockercfg-dlq6n" Dec 03 12:16:33 crc kubenswrapper[5052]: I1203 12:16:33.139071 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-galera-openstack-svc" Dec 03 12:16:33 crc kubenswrapper[5052]: I1203 12:16:33.142665 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"combined-ca-bundle" Dec 03 12:16:33 crc kubenswrapper[5052]: I1203 12:16:33.145840 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"openstack-scripts" Dec 03 12:16:33 crc kubenswrapper[5052]: I1203 12:16:33.147863 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/openstack-galera-0"] Dec 03 12:16:33 crc kubenswrapper[5052]: I1203 12:16:33.272647 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k7znz\" (UniqueName: \"kubernetes.io/projected/5067bb8d-d668-46a6-b9de-8065266cde63-kube-api-access-k7znz\") pod \"openstack-galera-0\" (UID: \"5067bb8d-d668-46a6-b9de-8065266cde63\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:16:33 crc kubenswrapper[5052]: I1203 12:16:33.272910 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5067bb8d-d668-46a6-b9de-8065266cde63-combined-ca-bundle\") pod \"openstack-galera-0\" (UID: \"5067bb8d-d668-46a6-b9de-8065266cde63\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:16:33 crc kubenswrapper[5052]: I1203 12:16:33.272971 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/5067bb8d-d668-46a6-b9de-8065266cde63-galera-tls-certs\") pod \"openstack-galera-0\" (UID: \"5067bb8d-d668-46a6-b9de-8065266cde63\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:16:33 crc kubenswrapper[5052]: I1203 12:16:33.273004 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/5067bb8d-d668-46a6-b9de-8065266cde63-operator-scripts\") pod \"openstack-galera-0\" (UID: \"5067bb8d-d668-46a6-b9de-8065266cde63\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:16:33 crc kubenswrapper[5052]: I1203 12:16:33.273035 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage03-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage03-crc\") pod \"openstack-galera-0\" (UID: \"5067bb8d-d668-46a6-b9de-8065266cde63\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:16:33 crc kubenswrapper[5052]: I1203 12:16:33.273052 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/5067bb8d-d668-46a6-b9de-8065266cde63-kolla-config\") pod \"openstack-galera-0\" (UID: \"5067bb8d-d668-46a6-b9de-8065266cde63\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:16:33 crc kubenswrapper[5052]: I1203 12:16:33.273083 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/5067bb8d-d668-46a6-b9de-8065266cde63-config-data-default\") pod \"openstack-galera-0\" (UID: \"5067bb8d-d668-46a6-b9de-8065266cde63\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:16:33 crc kubenswrapper[5052]: I1203 12:16:33.273100 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/5067bb8d-d668-46a6-b9de-8065266cde63-config-data-generated\") pod \"openstack-galera-0\" (UID: \"5067bb8d-d668-46a6-b9de-8065266cde63\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:16:33 crc kubenswrapper[5052]: I1203 12:16:33.374399 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/5067bb8d-d668-46a6-b9de-8065266cde63-galera-tls-certs\") pod \"openstack-galera-0\" (UID: \"5067bb8d-d668-46a6-b9de-8065266cde63\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:16:33 crc kubenswrapper[5052]: I1203 12:16:33.374456 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/5067bb8d-d668-46a6-b9de-8065266cde63-operator-scripts\") pod \"openstack-galera-0\" (UID: \"5067bb8d-d668-46a6-b9de-8065266cde63\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:16:33 crc kubenswrapper[5052]: I1203 12:16:33.374492 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage03-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage03-crc\") pod \"openstack-galera-0\" (UID: \"5067bb8d-d668-46a6-b9de-8065266cde63\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:16:33 crc kubenswrapper[5052]: I1203 12:16:33.374524 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/5067bb8d-d668-46a6-b9de-8065266cde63-kolla-config\") pod \"openstack-galera-0\" (UID: \"5067bb8d-d668-46a6-b9de-8065266cde63\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:16:33 crc kubenswrapper[5052]: I1203 12:16:33.374557 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/5067bb8d-d668-46a6-b9de-8065266cde63-config-data-default\") pod \"openstack-galera-0\" (UID: \"5067bb8d-d668-46a6-b9de-8065266cde63\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:16:33 crc kubenswrapper[5052]: I1203 12:16:33.374576 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/5067bb8d-d668-46a6-b9de-8065266cde63-config-data-generated\") pod \"openstack-galera-0\" (UID: \"5067bb8d-d668-46a6-b9de-8065266cde63\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:16:33 crc kubenswrapper[5052]: I1203 12:16:33.374619 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-k7znz\" (UniqueName: \"kubernetes.io/projected/5067bb8d-d668-46a6-b9de-8065266cde63-kube-api-access-k7znz\") pod \"openstack-galera-0\" (UID: \"5067bb8d-d668-46a6-b9de-8065266cde63\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:16:33 crc kubenswrapper[5052]: I1203 12:16:33.374643 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5067bb8d-d668-46a6-b9de-8065266cde63-combined-ca-bundle\") pod \"openstack-galera-0\" (UID: \"5067bb8d-d668-46a6-b9de-8065266cde63\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:16:33 crc kubenswrapper[5052]: I1203 12:16:33.375626 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage03-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage03-crc\") pod \"openstack-galera-0\" (UID: \"5067bb8d-d668-46a6-b9de-8065266cde63\") device mount path \"/mnt/openstack/pv03\"" pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:16:33 crc kubenswrapper[5052]: I1203 12:16:33.375791 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/5067bb8d-d668-46a6-b9de-8065266cde63-config-data-default\") pod \"openstack-galera-0\" (UID: \"5067bb8d-d668-46a6-b9de-8065266cde63\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:16:33 crc kubenswrapper[5052]: I1203 12:16:33.376071 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/5067bb8d-d668-46a6-b9de-8065266cde63-operator-scripts\") pod \"openstack-galera-0\" (UID: \"5067bb8d-d668-46a6-b9de-8065266cde63\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:16:33 crc kubenswrapper[5052]: I1203 12:16:33.376172 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/5067bb8d-d668-46a6-b9de-8065266cde63-config-data-generated\") pod \"openstack-galera-0\" (UID: \"5067bb8d-d668-46a6-b9de-8065266cde63\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:16:33 crc kubenswrapper[5052]: I1203 12:16:33.376242 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/5067bb8d-d668-46a6-b9de-8065266cde63-kolla-config\") pod \"openstack-galera-0\" (UID: \"5067bb8d-d668-46a6-b9de-8065266cde63\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:16:33 crc kubenswrapper[5052]: I1203 12:16:33.425088 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5067bb8d-d668-46a6-b9de-8065266cde63-combined-ca-bundle\") pod \"openstack-galera-0\" (UID: \"5067bb8d-d668-46a6-b9de-8065266cde63\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:16:33 crc kubenswrapper[5052]: I1203 12:16:33.428863 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/5067bb8d-d668-46a6-b9de-8065266cde63-galera-tls-certs\") pod \"openstack-galera-0\" (UID: \"5067bb8d-d668-46a6-b9de-8065266cde63\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:16:33 crc kubenswrapper[5052]: I1203 12:16:33.430706 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-k7znz\" (UniqueName: \"kubernetes.io/projected/5067bb8d-d668-46a6-b9de-8065266cde63-kube-api-access-k7znz\") pod \"openstack-galera-0\" (UID: \"5067bb8d-d668-46a6-b9de-8065266cde63\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:16:33 crc kubenswrapper[5052]: I1203 12:16:33.448605 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage03-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage03-crc\") pod \"openstack-galera-0\" (UID: \"5067bb8d-d668-46a6-b9de-8065266cde63\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:16:33 crc kubenswrapper[5052]: I1203 12:16:33.454034 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:16:33 crc kubenswrapper[5052]: W1203 12:16:33.926734 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5067bb8d_d668_46a6_b9de_8065266cde63.slice/crio-0472197b95995bed984ae7255d7900169304c573d7fae477d9858f5fa19b9b6d WatchSource:0}: Error finding container 0472197b95995bed984ae7255d7900169304c573d7fae477d9858f5fa19b9b6d: Status 404 returned error can't find the container with id 0472197b95995bed984ae7255d7900169304c573d7fae477d9858f5fa19b9b6d Dec 03 12:16:33 crc kubenswrapper[5052]: I1203 12:16:33.927460 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/openstack-galera-0"] Dec 03 12:16:34 crc kubenswrapper[5052]: I1203 12:16:34.509636 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/openstack-cell1-galera-0"] Dec 03 12:16:34 crc kubenswrapper[5052]: I1203 12:16:34.511080 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:16:34 crc kubenswrapper[5052]: I1203 12:16:34.513909 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"galera-openstack-cell1-dockercfg-nt4tx" Dec 03 12:16:34 crc kubenswrapper[5052]: I1203 12:16:34.513988 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"openstack-cell1-config-data" Dec 03 12:16:34 crc kubenswrapper[5052]: I1203 12:16:34.516418 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-galera-openstack-cell1-svc" Dec 03 12:16:34 crc kubenswrapper[5052]: I1203 12:16:34.516773 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"openstack-cell1-scripts" Dec 03 12:16:34 crc kubenswrapper[5052]: I1203 12:16:34.525771 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/openstack-cell1-galera-0"] Dec 03 12:16:34 crc kubenswrapper[5052]: I1203 12:16:34.587196 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" event={"ID":"1ec6b1f7-5abb-4063-80fa-b5ec907dcb91","Type":"ContainerStarted","Data":"65247d77d1d600b3063d35a5f6d98e45d0e5800997643936febe03cc72d2cda6"} Dec 03 12:16:34 crc kubenswrapper[5052]: I1203 12:16:34.589790 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-galera-0" event={"ID":"5067bb8d-d668-46a6-b9de-8065266cde63","Type":"ContainerStarted","Data":"17351fd3e1257f4ce4d46a149fe313d7f60e83dec93abd203cf3ef841ce6d2e5"} Dec 03 12:16:34 crc kubenswrapper[5052]: I1203 12:16:34.589869 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-galera-0" event={"ID":"5067bb8d-d668-46a6-b9de-8065266cde63","Type":"ContainerStarted","Data":"0472197b95995bed984ae7255d7900169304c573d7fae477d9858f5fa19b9b6d"} Dec 03 12:16:34 crc kubenswrapper[5052]: I1203 12:16:34.591219 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/a56992d9-4662-444f-8a45-0375d0855946-kolla-config\") pod \"openstack-cell1-galera-0\" (UID: \"a56992d9-4662-444f-8a45-0375d0855946\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:16:34 crc kubenswrapper[5052]: I1203 12:16:34.591285 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/a56992d9-4662-444f-8a45-0375d0855946-config-data-generated\") pod \"openstack-cell1-galera-0\" (UID: \"a56992d9-4662-444f-8a45-0375d0855946\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:16:34 crc kubenswrapper[5052]: I1203 12:16:34.591832 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/a56992d9-4662-444f-8a45-0375d0855946-galera-tls-certs\") pod \"openstack-cell1-galera-0\" (UID: \"a56992d9-4662-444f-8a45-0375d0855946\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:16:34 crc kubenswrapper[5052]: I1203 12:16:34.592019 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4tjl2\" (UniqueName: \"kubernetes.io/projected/a56992d9-4662-444f-8a45-0375d0855946-kube-api-access-4tjl2\") pod \"openstack-cell1-galera-0\" (UID: \"a56992d9-4662-444f-8a45-0375d0855946\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:16:34 crc kubenswrapper[5052]: I1203 12:16:34.592070 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/a56992d9-4662-444f-8a45-0375d0855946-operator-scripts\") pod \"openstack-cell1-galera-0\" (UID: \"a56992d9-4662-444f-8a45-0375d0855946\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:16:34 crc kubenswrapper[5052]: I1203 12:16:34.592119 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a56992d9-4662-444f-8a45-0375d0855946-combined-ca-bundle\") pod \"openstack-cell1-galera-0\" (UID: \"a56992d9-4662-444f-8a45-0375d0855946\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:16:34 crc kubenswrapper[5052]: I1203 12:16:34.592259 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage06-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage06-crc\") pod \"openstack-cell1-galera-0\" (UID: \"a56992d9-4662-444f-8a45-0375d0855946\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:16:34 crc kubenswrapper[5052]: I1203 12:16:34.592344 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/a56992d9-4662-444f-8a45-0375d0855946-config-data-default\") pod \"openstack-cell1-galera-0\" (UID: \"a56992d9-4662-444f-8a45-0375d0855946\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:16:34 crc kubenswrapper[5052]: I1203 12:16:34.594961 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-server-0" event={"ID":"683b7d5f-a88a-4dc0-ae83-af3c6444c7ff","Type":"ContainerStarted","Data":"67a78b1201a5ad7507c89c9217d67878a2b07cc939213e22cfeb1612eb0f4575"} Dec 03 12:16:34 crc kubenswrapper[5052]: I1203 12:16:34.694125 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-4tjl2\" (UniqueName: \"kubernetes.io/projected/a56992d9-4662-444f-8a45-0375d0855946-kube-api-access-4tjl2\") pod \"openstack-cell1-galera-0\" (UID: \"a56992d9-4662-444f-8a45-0375d0855946\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:16:34 crc kubenswrapper[5052]: I1203 12:16:34.694174 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/a56992d9-4662-444f-8a45-0375d0855946-operator-scripts\") pod \"openstack-cell1-galera-0\" (UID: \"a56992d9-4662-444f-8a45-0375d0855946\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:16:34 crc kubenswrapper[5052]: I1203 12:16:34.694198 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a56992d9-4662-444f-8a45-0375d0855946-combined-ca-bundle\") pod \"openstack-cell1-galera-0\" (UID: \"a56992d9-4662-444f-8a45-0375d0855946\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:16:34 crc kubenswrapper[5052]: I1203 12:16:34.694262 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage06-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage06-crc\") pod \"openstack-cell1-galera-0\" (UID: \"a56992d9-4662-444f-8a45-0375d0855946\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:16:34 crc kubenswrapper[5052]: I1203 12:16:34.694323 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/a56992d9-4662-444f-8a45-0375d0855946-config-data-default\") pod \"openstack-cell1-galera-0\" (UID: \"a56992d9-4662-444f-8a45-0375d0855946\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:16:34 crc kubenswrapper[5052]: I1203 12:16:34.694415 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/a56992d9-4662-444f-8a45-0375d0855946-kolla-config\") pod \"openstack-cell1-galera-0\" (UID: \"a56992d9-4662-444f-8a45-0375d0855946\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:16:34 crc kubenswrapper[5052]: I1203 12:16:34.694464 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/a56992d9-4662-444f-8a45-0375d0855946-config-data-generated\") pod \"openstack-cell1-galera-0\" (UID: \"a56992d9-4662-444f-8a45-0375d0855946\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:16:34 crc kubenswrapper[5052]: I1203 12:16:34.694524 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/a56992d9-4662-444f-8a45-0375d0855946-galera-tls-certs\") pod \"openstack-cell1-galera-0\" (UID: \"a56992d9-4662-444f-8a45-0375d0855946\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:16:34 crc kubenswrapper[5052]: I1203 12:16:34.697029 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/a56992d9-4662-444f-8a45-0375d0855946-config-data-generated\") pod \"openstack-cell1-galera-0\" (UID: \"a56992d9-4662-444f-8a45-0375d0855946\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:16:34 crc kubenswrapper[5052]: I1203 12:16:34.698167 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage06-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage06-crc\") pod \"openstack-cell1-galera-0\" (UID: \"a56992d9-4662-444f-8a45-0375d0855946\") device mount path \"/mnt/openstack/pv06\"" pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:16:34 crc kubenswrapper[5052]: I1203 12:16:34.698447 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/a56992d9-4662-444f-8a45-0375d0855946-kolla-config\") pod \"openstack-cell1-galera-0\" (UID: \"a56992d9-4662-444f-8a45-0375d0855946\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:16:34 crc kubenswrapper[5052]: I1203 12:16:34.698491 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/a56992d9-4662-444f-8a45-0375d0855946-config-data-default\") pod \"openstack-cell1-galera-0\" (UID: \"a56992d9-4662-444f-8a45-0375d0855946\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:16:34 crc kubenswrapper[5052]: I1203 12:16:34.699244 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/a56992d9-4662-444f-8a45-0375d0855946-operator-scripts\") pod \"openstack-cell1-galera-0\" (UID: \"a56992d9-4662-444f-8a45-0375d0855946\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:16:34 crc kubenswrapper[5052]: I1203 12:16:34.711338 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a56992d9-4662-444f-8a45-0375d0855946-combined-ca-bundle\") pod \"openstack-cell1-galera-0\" (UID: \"a56992d9-4662-444f-8a45-0375d0855946\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:16:34 crc kubenswrapper[5052]: I1203 12:16:34.711920 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/a56992d9-4662-444f-8a45-0375d0855946-galera-tls-certs\") pod \"openstack-cell1-galera-0\" (UID: \"a56992d9-4662-444f-8a45-0375d0855946\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:16:34 crc kubenswrapper[5052]: I1203 12:16:34.727835 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-4tjl2\" (UniqueName: \"kubernetes.io/projected/a56992d9-4662-444f-8a45-0375d0855946-kube-api-access-4tjl2\") pod \"openstack-cell1-galera-0\" (UID: \"a56992d9-4662-444f-8a45-0375d0855946\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:16:34 crc kubenswrapper[5052]: I1203 12:16:34.731489 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage06-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage06-crc\") pod \"openstack-cell1-galera-0\" (UID: \"a56992d9-4662-444f-8a45-0375d0855946\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:16:34 crc kubenswrapper[5052]: I1203 12:16:34.833162 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:16:34 crc kubenswrapper[5052]: I1203 12:16:34.965621 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/memcached-0"] Dec 03 12:16:34 crc kubenswrapper[5052]: I1203 12:16:34.967549 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/memcached-0" Dec 03 12:16:34 crc kubenswrapper[5052]: I1203 12:16:34.972410 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"memcached-memcached-dockercfg-7smft" Dec 03 12:16:34 crc kubenswrapper[5052]: I1203 12:16:34.973128 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"memcached-config-data" Dec 03 12:16:34 crc kubenswrapper[5052]: I1203 12:16:34.975683 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-memcached-svc" Dec 03 12:16:35 crc kubenswrapper[5052]: I1203 12:16:35.043027 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/memcached-0"] Dec 03 12:16:35 crc kubenswrapper[5052]: I1203 12:16:35.105577 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ml58r\" (UniqueName: \"kubernetes.io/projected/695de243-96a9-4e8b-a0f3-3a4ef2d95efe-kube-api-access-ml58r\") pod \"memcached-0\" (UID: \"695de243-96a9-4e8b-a0f3-3a4ef2d95efe\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:16:35 crc kubenswrapper[5052]: I1203 12:16:35.105641 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/695de243-96a9-4e8b-a0f3-3a4ef2d95efe-combined-ca-bundle\") pod \"memcached-0\" (UID: \"695de243-96a9-4e8b-a0f3-3a4ef2d95efe\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:16:35 crc kubenswrapper[5052]: I1203 12:16:35.105692 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/695de243-96a9-4e8b-a0f3-3a4ef2d95efe-config-data\") pod \"memcached-0\" (UID: \"695de243-96a9-4e8b-a0f3-3a4ef2d95efe\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:16:35 crc kubenswrapper[5052]: I1203 12:16:35.105719 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"memcached-tls-certs\" (UniqueName: \"kubernetes.io/secret/695de243-96a9-4e8b-a0f3-3a4ef2d95efe-memcached-tls-certs\") pod \"memcached-0\" (UID: \"695de243-96a9-4e8b-a0f3-3a4ef2d95efe\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:16:35 crc kubenswrapper[5052]: I1203 12:16:35.105752 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/695de243-96a9-4e8b-a0f3-3a4ef2d95efe-kolla-config\") pod \"memcached-0\" (UID: \"695de243-96a9-4e8b-a0f3-3a4ef2d95efe\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:16:35 crc kubenswrapper[5052]: I1203 12:16:35.207280 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"memcached-tls-certs\" (UniqueName: \"kubernetes.io/secret/695de243-96a9-4e8b-a0f3-3a4ef2d95efe-memcached-tls-certs\") pod \"memcached-0\" (UID: \"695de243-96a9-4e8b-a0f3-3a4ef2d95efe\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:16:35 crc kubenswrapper[5052]: I1203 12:16:35.207365 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/695de243-96a9-4e8b-a0f3-3a4ef2d95efe-kolla-config\") pod \"memcached-0\" (UID: \"695de243-96a9-4e8b-a0f3-3a4ef2d95efe\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:16:35 crc kubenswrapper[5052]: I1203 12:16:35.208317 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/695de243-96a9-4e8b-a0f3-3a4ef2d95efe-kolla-config\") pod \"memcached-0\" (UID: \"695de243-96a9-4e8b-a0f3-3a4ef2d95efe\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:16:35 crc kubenswrapper[5052]: I1203 12:16:35.208496 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-ml58r\" (UniqueName: \"kubernetes.io/projected/695de243-96a9-4e8b-a0f3-3a4ef2d95efe-kube-api-access-ml58r\") pod \"memcached-0\" (UID: \"695de243-96a9-4e8b-a0f3-3a4ef2d95efe\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:16:35 crc kubenswrapper[5052]: I1203 12:16:35.209014 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/695de243-96a9-4e8b-a0f3-3a4ef2d95efe-combined-ca-bundle\") pod \"memcached-0\" (UID: \"695de243-96a9-4e8b-a0f3-3a4ef2d95efe\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:16:35 crc kubenswrapper[5052]: I1203 12:16:35.209421 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/695de243-96a9-4e8b-a0f3-3a4ef2d95efe-config-data\") pod \"memcached-0\" (UID: \"695de243-96a9-4e8b-a0f3-3a4ef2d95efe\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:16:35 crc kubenswrapper[5052]: I1203 12:16:35.210087 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/695de243-96a9-4e8b-a0f3-3a4ef2d95efe-config-data\") pod \"memcached-0\" (UID: \"695de243-96a9-4e8b-a0f3-3a4ef2d95efe\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:16:35 crc kubenswrapper[5052]: I1203 12:16:35.214309 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/695de243-96a9-4e8b-a0f3-3a4ef2d95efe-combined-ca-bundle\") pod \"memcached-0\" (UID: \"695de243-96a9-4e8b-a0f3-3a4ef2d95efe\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:16:35 crc kubenswrapper[5052]: I1203 12:16:35.214449 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"memcached-tls-certs\" (UniqueName: \"kubernetes.io/secret/695de243-96a9-4e8b-a0f3-3a4ef2d95efe-memcached-tls-certs\") pod \"memcached-0\" (UID: \"695de243-96a9-4e8b-a0f3-3a4ef2d95efe\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:16:35 crc kubenswrapper[5052]: I1203 12:16:35.228184 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-ml58r\" (UniqueName: \"kubernetes.io/projected/695de243-96a9-4e8b-a0f3-3a4ef2d95efe-kube-api-access-ml58r\") pod \"memcached-0\" (UID: \"695de243-96a9-4e8b-a0f3-3a4ef2d95efe\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:16:35 crc kubenswrapper[5052]: I1203 12:16:35.318661 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/memcached-0" Dec 03 12:16:35 crc kubenswrapper[5052]: I1203 12:16:35.467393 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/openstack-cell1-galera-0"] Dec 03 12:16:35 crc kubenswrapper[5052]: I1203 12:16:35.611022 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-cell1-galera-0" event={"ID":"a56992d9-4662-444f-8a45-0375d0855946","Type":"ContainerStarted","Data":"6ea97c6fa066d8cad9a0a91f87604b718a8dd9dbe2d43a24094b211b878b10a6"} Dec 03 12:16:35 crc kubenswrapper[5052]: I1203 12:16:35.807249 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/memcached-0"] Dec 03 12:16:36 crc kubenswrapper[5052]: I1203 12:16:36.634779 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/memcached-0" event={"ID":"695de243-96a9-4e8b-a0f3-3a4ef2d95efe","Type":"ContainerStarted","Data":"a0b610780c737434b4d97d2fd99c01273a88681fdc19b3a41fdf3d032c04b664"} Dec 03 12:16:36 crc kubenswrapper[5052]: I1203 12:16:36.634840 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/memcached-0" event={"ID":"695de243-96a9-4e8b-a0f3-3a4ef2d95efe","Type":"ContainerStarted","Data":"2606247e9ea56d0251c3c9bb611d0f13abafeb48627eff1f5e998fe4d509b2c2"} Dec 03 12:16:36 crc kubenswrapper[5052]: I1203 12:16:36.634864 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/memcached-0" Dec 03 12:16:36 crc kubenswrapper[5052]: I1203 12:16:36.637026 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-cell1-galera-0" event={"ID":"a56992d9-4662-444f-8a45-0375d0855946","Type":"ContainerStarted","Data":"44461d2df3c8193c47365f75ad3473316c51e0f906c01100d8592472d3b02232"} Dec 03 12:16:36 crc kubenswrapper[5052]: I1203 12:16:36.681134 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/memcached-0" podStartSLOduration=2.681114645 podStartE2EDuration="2.681114645s" podCreationTimestamp="2025-12-03 12:16:34 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:16:36.656967076 +0000 UTC m=+3387.245208840" watchObservedRunningTime="2025-12-03 12:16:36.681114645 +0000 UTC m=+3387.269356409" Dec 03 12:16:36 crc kubenswrapper[5052]: I1203 12:16:36.780005 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/kube-state-metrics-0"] Dec 03 12:16:36 crc kubenswrapper[5052]: I1203 12:16:36.781244 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:16:36 crc kubenswrapper[5052]: I1203 12:16:36.793295 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/kube-state-metrics-0"] Dec 03 12:16:36 crc kubenswrapper[5052]: I1203 12:16:36.813071 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"telemetry-ceilometer-dockercfg-klj4p" Dec 03 12:16:36 crc kubenswrapper[5052]: I1203 12:16:36.837620 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j9gms\" (UniqueName: \"kubernetes.io/projected/d667797a-cec6-4ec5-a853-b56c8e4dce5d-kube-api-access-j9gms\") pod \"kube-state-metrics-0\" (UID: \"d667797a-cec6-4ec5-a853-b56c8e4dce5d\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:16:36 crc kubenswrapper[5052]: I1203 12:16:36.939336 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-j9gms\" (UniqueName: \"kubernetes.io/projected/d667797a-cec6-4ec5-a853-b56c8e4dce5d-kube-api-access-j9gms\") pod \"kube-state-metrics-0\" (UID: \"d667797a-cec6-4ec5-a853-b56c8e4dce5d\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:16:36 crc kubenswrapper[5052]: I1203 12:16:36.957387 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-j9gms\" (UniqueName: \"kubernetes.io/projected/d667797a-cec6-4ec5-a853-b56c8e4dce5d-kube-api-access-j9gms\") pod \"kube-state-metrics-0\" (UID: \"d667797a-cec6-4ec5-a853-b56c8e4dce5d\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:16:37 crc kubenswrapper[5052]: I1203 12:16:37.117488 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:16:37 crc kubenswrapper[5052]: I1203 12:16:37.553736 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/kube-state-metrics-0"] Dec 03 12:16:37 crc kubenswrapper[5052]: I1203 12:16:37.648453 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/kube-state-metrics-0" event={"ID":"d667797a-cec6-4ec5-a853-b56c8e4dce5d","Type":"ContainerStarted","Data":"a2c8f1be2e4597df103ced6cb923549ec51501d18c686f6e66ae6351cb596b00"} Dec 03 12:16:38 crc kubenswrapper[5052]: I1203 12:16:38.667021 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/kube-state-metrics-0" event={"ID":"d667797a-cec6-4ec5-a853-b56c8e4dce5d","Type":"ContainerStarted","Data":"1fb3da7d65ea115a6ce63ebdae896b5ea20377a51668aa9f9ef49555a99f4e17"} Dec 03 12:16:38 crc kubenswrapper[5052]: I1203 12:16:38.678220 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:16:38 crc kubenswrapper[5052]: I1203 12:16:38.688410 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/kube-state-metrics-0" podStartSLOduration=2.225664367 podStartE2EDuration="2.688384229s" podCreationTimestamp="2025-12-03 12:16:36 +0000 UTC" firstStartedPulling="2025-12-03 12:16:37.56068015 +0000 UTC m=+3388.148921914" lastFinishedPulling="2025-12-03 12:16:38.023400012 +0000 UTC m=+3388.611641776" observedRunningTime="2025-12-03 12:16:38.684035405 +0000 UTC m=+3389.272277169" watchObservedRunningTime="2025-12-03 12:16:38.688384229 +0000 UTC m=+3389.276625993" Dec 03 12:16:39 crc kubenswrapper[5052]: I1203 12:16:39.677364 5052 generic.go:334] "Generic (PLEG): container finished" podID="5067bb8d-d668-46a6-b9de-8065266cde63" containerID="17351fd3e1257f4ce4d46a149fe313d7f60e83dec93abd203cf3ef841ce6d2e5" exitCode=0 Dec 03 12:16:39 crc kubenswrapper[5052]: I1203 12:16:39.682101 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-galera-0" event={"ID":"5067bb8d-d668-46a6-b9de-8065266cde63","Type":"ContainerDied","Data":"17351fd3e1257f4ce4d46a149fe313d7f60e83dec93abd203cf3ef841ce6d2e5"} Dec 03 12:16:40 crc kubenswrapper[5052]: I1203 12:16:40.688011 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-galera-0" event={"ID":"5067bb8d-d668-46a6-b9de-8065266cde63","Type":"ContainerStarted","Data":"7c40f1aa68ee0e0f29ce1ab35070e6d55c530a232bd4bf5a962f9aa5aa656271"} Dec 03 12:16:40 crc kubenswrapper[5052]: I1203 12:16:40.690724 5052 generic.go:334] "Generic (PLEG): container finished" podID="a56992d9-4662-444f-8a45-0375d0855946" containerID="44461d2df3c8193c47365f75ad3473316c51e0f906c01100d8592472d3b02232" exitCode=0 Dec 03 12:16:40 crc kubenswrapper[5052]: I1203 12:16:40.690765 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-cell1-galera-0" event={"ID":"a56992d9-4662-444f-8a45-0375d0855946","Type":"ContainerDied","Data":"44461d2df3c8193c47365f75ad3473316c51e0f906c01100d8592472d3b02232"} Dec 03 12:16:40 crc kubenswrapper[5052]: I1203 12:16:40.715436 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/openstack-galera-0" podStartSLOduration=8.715419167 podStartE2EDuration="8.715419167s" podCreationTimestamp="2025-12-03 12:16:32 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:16:40.707663565 +0000 UTC m=+3391.295905329" watchObservedRunningTime="2025-12-03 12:16:40.715419167 +0000 UTC m=+3391.303660931" Dec 03 12:16:41 crc kubenswrapper[5052]: I1203 12:16:41.229148 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ovsdbserver-nb-0"] Dec 03 12:16:41 crc kubenswrapper[5052]: I1203 12:16:41.230470 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:16:41 crc kubenswrapper[5052]: I1203 12:16:41.235010 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-ovn-metrics" Dec 03 12:16:41 crc kubenswrapper[5052]: I1203 12:16:41.235355 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"ovndbcluster-nb-scripts" Dec 03 12:16:41 crc kubenswrapper[5052]: I1203 12:16:41.235369 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-ovndbcluster-nb-ovndbs" Dec 03 12:16:41 crc kubenswrapper[5052]: I1203 12:16:41.240294 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ovncluster-ovndbcluster-nb-dockercfg-4ncxw" Dec 03 12:16:41 crc kubenswrapper[5052]: I1203 12:16:41.240354 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"ovndbcluster-nb-config" Dec 03 12:16:41 crc kubenswrapper[5052]: I1203 12:16:41.245580 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-nb-0"] Dec 03 12:16:41 crc kubenswrapper[5052]: I1203 12:16:41.310449 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/fcd60977-d995-42ee-b654-ba2947a3316d-config\") pod \"ovsdbserver-nb-0\" (UID: \"fcd60977-d995-42ee-b654-ba2947a3316d\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:16:41 crc kubenswrapper[5052]: I1203 12:16:41.310507 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9pp7j\" (UniqueName: \"kubernetes.io/projected/fcd60977-d995-42ee-b654-ba2947a3316d-kube-api-access-9pp7j\") pod \"ovsdbserver-nb-0\" (UID: \"fcd60977-d995-42ee-b654-ba2947a3316d\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:16:41 crc kubenswrapper[5052]: I1203 12:16:41.310564 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/fcd60977-d995-42ee-b654-ba2947a3316d-ovsdb-rundir\") pod \"ovsdbserver-nb-0\" (UID: \"fcd60977-d995-42ee-b654-ba2947a3316d\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:16:41 crc kubenswrapper[5052]: I1203 12:16:41.310593 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage20-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage20-crc\") pod \"ovsdbserver-nb-0\" (UID: \"fcd60977-d995-42ee-b654-ba2947a3316d\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:16:41 crc kubenswrapper[5052]: I1203 12:16:41.310652 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/fcd60977-d995-42ee-b654-ba2947a3316d-combined-ca-bundle\") pod \"ovsdbserver-nb-0\" (UID: \"fcd60977-d995-42ee-b654-ba2947a3316d\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:16:41 crc kubenswrapper[5052]: I1203 12:16:41.310714 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/fcd60977-d995-42ee-b654-ba2947a3316d-scripts\") pod \"ovsdbserver-nb-0\" (UID: \"fcd60977-d995-42ee-b654-ba2947a3316d\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:16:41 crc kubenswrapper[5052]: I1203 12:16:41.310946 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/fcd60977-d995-42ee-b654-ba2947a3316d-metrics-certs-tls-certs\") pod \"ovsdbserver-nb-0\" (UID: \"fcd60977-d995-42ee-b654-ba2947a3316d\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:16:41 crc kubenswrapper[5052]: I1203 12:16:41.311124 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovsdbserver-nb-tls-certs\" (UniqueName: \"kubernetes.io/secret/fcd60977-d995-42ee-b654-ba2947a3316d-ovsdbserver-nb-tls-certs\") pod \"ovsdbserver-nb-0\" (UID: \"fcd60977-d995-42ee-b654-ba2947a3316d\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:16:41 crc kubenswrapper[5052]: I1203 12:16:41.413012 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/fcd60977-d995-42ee-b654-ba2947a3316d-scripts\") pod \"ovsdbserver-nb-0\" (UID: \"fcd60977-d995-42ee-b654-ba2947a3316d\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:16:41 crc kubenswrapper[5052]: I1203 12:16:41.413111 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/fcd60977-d995-42ee-b654-ba2947a3316d-metrics-certs-tls-certs\") pod \"ovsdbserver-nb-0\" (UID: \"fcd60977-d995-42ee-b654-ba2947a3316d\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:16:41 crc kubenswrapper[5052]: I1203 12:16:41.413175 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovsdbserver-nb-tls-certs\" (UniqueName: \"kubernetes.io/secret/fcd60977-d995-42ee-b654-ba2947a3316d-ovsdbserver-nb-tls-certs\") pod \"ovsdbserver-nb-0\" (UID: \"fcd60977-d995-42ee-b654-ba2947a3316d\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:16:41 crc kubenswrapper[5052]: I1203 12:16:41.413224 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/fcd60977-d995-42ee-b654-ba2947a3316d-config\") pod \"ovsdbserver-nb-0\" (UID: \"fcd60977-d995-42ee-b654-ba2947a3316d\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:16:41 crc kubenswrapper[5052]: I1203 12:16:41.413255 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-9pp7j\" (UniqueName: \"kubernetes.io/projected/fcd60977-d995-42ee-b654-ba2947a3316d-kube-api-access-9pp7j\") pod \"ovsdbserver-nb-0\" (UID: \"fcd60977-d995-42ee-b654-ba2947a3316d\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:16:41 crc kubenswrapper[5052]: I1203 12:16:41.413280 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/fcd60977-d995-42ee-b654-ba2947a3316d-ovsdb-rundir\") pod \"ovsdbserver-nb-0\" (UID: \"fcd60977-d995-42ee-b654-ba2947a3316d\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:16:41 crc kubenswrapper[5052]: I1203 12:16:41.413304 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage20-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage20-crc\") pod \"ovsdbserver-nb-0\" (UID: \"fcd60977-d995-42ee-b654-ba2947a3316d\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:16:41 crc kubenswrapper[5052]: I1203 12:16:41.413339 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/fcd60977-d995-42ee-b654-ba2947a3316d-combined-ca-bundle\") pod \"ovsdbserver-nb-0\" (UID: \"fcd60977-d995-42ee-b654-ba2947a3316d\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:16:41 crc kubenswrapper[5052]: I1203 12:16:41.413742 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage20-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage20-crc\") pod \"ovsdbserver-nb-0\" (UID: \"fcd60977-d995-42ee-b654-ba2947a3316d\") device mount path \"/mnt/openstack/pv20\"" pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:16:41 crc kubenswrapper[5052]: I1203 12:16:41.413902 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/fcd60977-d995-42ee-b654-ba2947a3316d-ovsdb-rundir\") pod \"ovsdbserver-nb-0\" (UID: \"fcd60977-d995-42ee-b654-ba2947a3316d\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:16:41 crc kubenswrapper[5052]: I1203 12:16:41.414299 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/fcd60977-d995-42ee-b654-ba2947a3316d-scripts\") pod \"ovsdbserver-nb-0\" (UID: \"fcd60977-d995-42ee-b654-ba2947a3316d\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:16:41 crc kubenswrapper[5052]: I1203 12:16:41.418050 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovsdbserver-nb-tls-certs\" (UniqueName: \"kubernetes.io/secret/fcd60977-d995-42ee-b654-ba2947a3316d-ovsdbserver-nb-tls-certs\") pod \"ovsdbserver-nb-0\" (UID: \"fcd60977-d995-42ee-b654-ba2947a3316d\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:16:41 crc kubenswrapper[5052]: I1203 12:16:41.418222 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/fcd60977-d995-42ee-b654-ba2947a3316d-config\") pod \"ovsdbserver-nb-0\" (UID: \"fcd60977-d995-42ee-b654-ba2947a3316d\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:16:41 crc kubenswrapper[5052]: I1203 12:16:41.427928 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/fcd60977-d995-42ee-b654-ba2947a3316d-combined-ca-bundle\") pod \"ovsdbserver-nb-0\" (UID: \"fcd60977-d995-42ee-b654-ba2947a3316d\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:16:41 crc kubenswrapper[5052]: I1203 12:16:41.436916 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-9pp7j\" (UniqueName: \"kubernetes.io/projected/fcd60977-d995-42ee-b654-ba2947a3316d-kube-api-access-9pp7j\") pod \"ovsdbserver-nb-0\" (UID: \"fcd60977-d995-42ee-b654-ba2947a3316d\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:16:41 crc kubenswrapper[5052]: I1203 12:16:41.437226 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/fcd60977-d995-42ee-b654-ba2947a3316d-metrics-certs-tls-certs\") pod \"ovsdbserver-nb-0\" (UID: \"fcd60977-d995-42ee-b654-ba2947a3316d\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:16:41 crc kubenswrapper[5052]: I1203 12:16:41.440595 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage20-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage20-crc\") pod \"ovsdbserver-nb-0\" (UID: \"fcd60977-d995-42ee-b654-ba2947a3316d\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:16:41 crc kubenswrapper[5052]: I1203 12:16:41.598798 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:16:41 crc kubenswrapper[5052]: I1203 12:16:41.712122 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-cell1-galera-0" event={"ID":"a56992d9-4662-444f-8a45-0375d0855946","Type":"ContainerStarted","Data":"8dfc8c0fb51ed3361082e5d6a21d3480abe509b00c224f35e46b8ddb4b3e7415"} Dec 03 12:16:41 crc kubenswrapper[5052]: I1203 12:16:41.741448 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/openstack-cell1-galera-0" podStartSLOduration=8.741428152 podStartE2EDuration="8.741428152s" podCreationTimestamp="2025-12-03 12:16:33 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:16:41.732534999 +0000 UTC m=+3392.320776763" watchObservedRunningTime="2025-12-03 12:16:41.741428152 +0000 UTC m=+3392.329669916" Dec 03 12:16:42 crc kubenswrapper[5052]: I1203 12:16:42.033374 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-nb-0"] Dec 03 12:16:42 crc kubenswrapper[5052]: I1203 12:16:42.485165 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ovsdbserver-sb-0"] Dec 03 12:16:42 crc kubenswrapper[5052]: I1203 12:16:42.487416 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:16:42 crc kubenswrapper[5052]: I1203 12:16:42.491257 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-ovndbcluster-sb-ovndbs" Dec 03 12:16:42 crc kubenswrapper[5052]: I1203 12:16:42.491460 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"ovndbcluster-sb-scripts" Dec 03 12:16:42 crc kubenswrapper[5052]: I1203 12:16:42.491562 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ovncluster-ovndbcluster-sb-dockercfg-wqdjd" Dec 03 12:16:42 crc kubenswrapper[5052]: I1203 12:16:42.492939 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"ovndbcluster-sb-config" Dec 03 12:16:42 crc kubenswrapper[5052]: I1203 12:16:42.536908 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-sb-0"] Dec 03 12:16:42 crc kubenswrapper[5052]: I1203 12:16:42.635426 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kmxk2\" (UniqueName: \"kubernetes.io/projected/06dda089-0745-4941-adfe-2de711fce24e-kube-api-access-kmxk2\") pod \"ovsdbserver-sb-0\" (UID: \"06dda089-0745-4941-adfe-2de711fce24e\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:16:42 crc kubenswrapper[5052]: I1203 12:16:42.635494 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/06dda089-0745-4941-adfe-2de711fce24e-ovsdb-rundir\") pod \"ovsdbserver-sb-0\" (UID: \"06dda089-0745-4941-adfe-2de711fce24e\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:16:42 crc kubenswrapper[5052]: I1203 12:16:42.635549 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/06dda089-0745-4941-adfe-2de711fce24e-config\") pod \"ovsdbserver-sb-0\" (UID: \"06dda089-0745-4941-adfe-2de711fce24e\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:16:42 crc kubenswrapper[5052]: I1203 12:16:42.635574 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovsdbserver-sb-tls-certs\" (UniqueName: \"kubernetes.io/secret/06dda089-0745-4941-adfe-2de711fce24e-ovsdbserver-sb-tls-certs\") pod \"ovsdbserver-sb-0\" (UID: \"06dda089-0745-4941-adfe-2de711fce24e\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:16:42 crc kubenswrapper[5052]: I1203 12:16:42.635634 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/06dda089-0745-4941-adfe-2de711fce24e-metrics-certs-tls-certs\") pod \"ovsdbserver-sb-0\" (UID: \"06dda089-0745-4941-adfe-2de711fce24e\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:16:42 crc kubenswrapper[5052]: I1203 12:16:42.635733 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/06dda089-0745-4941-adfe-2de711fce24e-scripts\") pod \"ovsdbserver-sb-0\" (UID: \"06dda089-0745-4941-adfe-2de711fce24e\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:16:42 crc kubenswrapper[5052]: I1203 12:16:42.635774 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage02-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage02-crc\") pod \"ovsdbserver-sb-0\" (UID: \"06dda089-0745-4941-adfe-2de711fce24e\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:16:42 crc kubenswrapper[5052]: I1203 12:16:42.635804 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/06dda089-0745-4941-adfe-2de711fce24e-combined-ca-bundle\") pod \"ovsdbserver-sb-0\" (UID: \"06dda089-0745-4941-adfe-2de711fce24e\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:16:42 crc kubenswrapper[5052]: I1203 12:16:42.724491 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-nb-0" event={"ID":"fcd60977-d995-42ee-b654-ba2947a3316d","Type":"ContainerStarted","Data":"daa77eec7322cd2c65ca8b891da5191f57aec462539227515b3efcddb082df4d"} Dec 03 12:16:42 crc kubenswrapper[5052]: I1203 12:16:42.724578 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-nb-0" event={"ID":"fcd60977-d995-42ee-b654-ba2947a3316d","Type":"ContainerStarted","Data":"87154b4a2a5069ca41b7570b52ae76e123d9dca6d0b7110267bc2f70b48cd720"} Dec 03 12:16:42 crc kubenswrapper[5052]: I1203 12:16:42.724593 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-nb-0" event={"ID":"fcd60977-d995-42ee-b654-ba2947a3316d","Type":"ContainerStarted","Data":"7e8beed27f00db2e91962f4ee2426cdfad0e851c8f641388600870c9ec6f3109"} Dec 03 12:16:42 crc kubenswrapper[5052]: I1203 12:16:42.737936 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-kmxk2\" (UniqueName: \"kubernetes.io/projected/06dda089-0745-4941-adfe-2de711fce24e-kube-api-access-kmxk2\") pod \"ovsdbserver-sb-0\" (UID: \"06dda089-0745-4941-adfe-2de711fce24e\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:16:42 crc kubenswrapper[5052]: I1203 12:16:42.737998 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/06dda089-0745-4941-adfe-2de711fce24e-ovsdb-rundir\") pod \"ovsdbserver-sb-0\" (UID: \"06dda089-0745-4941-adfe-2de711fce24e\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:16:42 crc kubenswrapper[5052]: I1203 12:16:42.738035 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/06dda089-0745-4941-adfe-2de711fce24e-config\") pod \"ovsdbserver-sb-0\" (UID: \"06dda089-0745-4941-adfe-2de711fce24e\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:16:42 crc kubenswrapper[5052]: I1203 12:16:42.738062 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovsdbserver-sb-tls-certs\" (UniqueName: \"kubernetes.io/secret/06dda089-0745-4941-adfe-2de711fce24e-ovsdbserver-sb-tls-certs\") pod \"ovsdbserver-sb-0\" (UID: \"06dda089-0745-4941-adfe-2de711fce24e\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:16:42 crc kubenswrapper[5052]: I1203 12:16:42.738123 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/06dda089-0745-4941-adfe-2de711fce24e-metrics-certs-tls-certs\") pod \"ovsdbserver-sb-0\" (UID: \"06dda089-0745-4941-adfe-2de711fce24e\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:16:42 crc kubenswrapper[5052]: I1203 12:16:42.738170 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/06dda089-0745-4941-adfe-2de711fce24e-scripts\") pod \"ovsdbserver-sb-0\" (UID: \"06dda089-0745-4941-adfe-2de711fce24e\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:16:42 crc kubenswrapper[5052]: I1203 12:16:42.738227 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage02-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage02-crc\") pod \"ovsdbserver-sb-0\" (UID: \"06dda089-0745-4941-adfe-2de711fce24e\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:16:42 crc kubenswrapper[5052]: I1203 12:16:42.738258 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/06dda089-0745-4941-adfe-2de711fce24e-combined-ca-bundle\") pod \"ovsdbserver-sb-0\" (UID: \"06dda089-0745-4941-adfe-2de711fce24e\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:16:42 crc kubenswrapper[5052]: I1203 12:16:42.738631 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/06dda089-0745-4941-adfe-2de711fce24e-ovsdb-rundir\") pod \"ovsdbserver-sb-0\" (UID: \"06dda089-0745-4941-adfe-2de711fce24e\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:16:42 crc kubenswrapper[5052]: I1203 12:16:42.739480 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage02-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage02-crc\") pod \"ovsdbserver-sb-0\" (UID: \"06dda089-0745-4941-adfe-2de711fce24e\") device mount path \"/mnt/openstack/pv02\"" pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:16:42 crc kubenswrapper[5052]: I1203 12:16:42.740064 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/06dda089-0745-4941-adfe-2de711fce24e-config\") pod \"ovsdbserver-sb-0\" (UID: \"06dda089-0745-4941-adfe-2de711fce24e\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:16:42 crc kubenswrapper[5052]: I1203 12:16:42.740645 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/06dda089-0745-4941-adfe-2de711fce24e-scripts\") pod \"ovsdbserver-sb-0\" (UID: \"06dda089-0745-4941-adfe-2de711fce24e\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:16:42 crc kubenswrapper[5052]: I1203 12:16:42.744451 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/06dda089-0745-4941-adfe-2de711fce24e-metrics-certs-tls-certs\") pod \"ovsdbserver-sb-0\" (UID: \"06dda089-0745-4941-adfe-2de711fce24e\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:16:42 crc kubenswrapper[5052]: I1203 12:16:42.747966 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/06dda089-0745-4941-adfe-2de711fce24e-combined-ca-bundle\") pod \"ovsdbserver-sb-0\" (UID: \"06dda089-0745-4941-adfe-2de711fce24e\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:16:42 crc kubenswrapper[5052]: I1203 12:16:42.751742 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovsdbserver-sb-tls-certs\" (UniqueName: \"kubernetes.io/secret/06dda089-0745-4941-adfe-2de711fce24e-ovsdbserver-sb-tls-certs\") pod \"ovsdbserver-sb-0\" (UID: \"06dda089-0745-4941-adfe-2de711fce24e\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:16:42 crc kubenswrapper[5052]: I1203 12:16:42.758267 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-kmxk2\" (UniqueName: \"kubernetes.io/projected/06dda089-0745-4941-adfe-2de711fce24e-kube-api-access-kmxk2\") pod \"ovsdbserver-sb-0\" (UID: \"06dda089-0745-4941-adfe-2de711fce24e\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:16:42 crc kubenswrapper[5052]: I1203 12:16:42.777288 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage02-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage02-crc\") pod \"ovsdbserver-sb-0\" (UID: \"06dda089-0745-4941-adfe-2de711fce24e\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:16:42 crc kubenswrapper[5052]: I1203 12:16:42.901618 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:16:43 crc kubenswrapper[5052]: W1203 12:16:43.341548 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod06dda089_0745_4941_adfe_2de711fce24e.slice/crio-e5b10cce0a408072de6a69243553e83730687f6a07a501c281056171959831aa WatchSource:0}: Error finding container e5b10cce0a408072de6a69243553e83730687f6a07a501c281056171959831aa: Status 404 returned error can't find the container with id e5b10cce0a408072de6a69243553e83730687f6a07a501c281056171959831aa Dec 03 12:16:43 crc kubenswrapper[5052]: I1203 12:16:43.343063 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-sb-0"] Dec 03 12:16:43 crc kubenswrapper[5052]: I1203 12:16:43.455432 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:16:43 crc kubenswrapper[5052]: I1203 12:16:43.455557 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:16:43 crc kubenswrapper[5052]: I1203 12:16:43.742355 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-sb-0" event={"ID":"06dda089-0745-4941-adfe-2de711fce24e","Type":"ContainerStarted","Data":"4ce7e130ceeb669b4057b5b4f5937db4eabc9ded0e28a82765f352a1735b5239"} Dec 03 12:16:43 crc kubenswrapper[5052]: I1203 12:16:43.742763 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-sb-0" event={"ID":"06dda089-0745-4941-adfe-2de711fce24e","Type":"ContainerStarted","Data":"e5b10cce0a408072de6a69243553e83730687f6a07a501c281056171959831aa"} Dec 03 12:16:43 crc kubenswrapper[5052]: I1203 12:16:43.781731 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/ovsdbserver-nb-0" podStartSLOduration=3.781704178 podStartE2EDuration="3.781704178s" podCreationTimestamp="2025-12-03 12:16:40 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:16:43.760146532 +0000 UTC m=+3394.348388296" watchObservedRunningTime="2025-12-03 12:16:43.781704178 +0000 UTC m=+3394.369945942" Dec 03 12:16:44 crc kubenswrapper[5052]: I1203 12:16:44.600096 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:16:44 crc kubenswrapper[5052]: I1203 12:16:44.754956 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-sb-0" event={"ID":"06dda089-0745-4941-adfe-2de711fce24e","Type":"ContainerStarted","Data":"33dc5ae74629ef4a73b20f4631f0f9178eee3dfb4459965f03d5530f56c3e84d"} Dec 03 12:16:44 crc kubenswrapper[5052]: I1203 12:16:44.780348 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/ovsdbserver-sb-0" podStartSLOduration=3.780328242 podStartE2EDuration="3.780328242s" podCreationTimestamp="2025-12-03 12:16:41 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:16:44.775311709 +0000 UTC m=+3395.363553473" watchObservedRunningTime="2025-12-03 12:16:44.780328242 +0000 UTC m=+3395.368570006" Dec 03 12:16:44 crc kubenswrapper[5052]: I1203 12:16:44.833847 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:16:44 crc kubenswrapper[5052]: I1203 12:16:44.833917 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:16:45 crc kubenswrapper[5052]: I1203 12:16:45.321562 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/memcached-0" Dec 03 12:16:45 crc kubenswrapper[5052]: I1203 12:16:45.648116 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:16:45 crc kubenswrapper[5052]: I1203 12:16:45.729761 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:16:45 crc kubenswrapper[5052]: I1203 12:16:45.911138 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:16:46 crc kubenswrapper[5052]: I1203 12:16:46.600186 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:16:47 crc kubenswrapper[5052]: I1203 12:16:47.125629 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:16:47 crc kubenswrapper[5052]: I1203 12:16:47.645912 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:16:47 crc kubenswrapper[5052]: I1203 12:16:47.691295 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:16:47 crc kubenswrapper[5052]: I1203 12:16:47.902812 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:16:48 crc kubenswrapper[5052]: I1203 12:16:48.104187 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/swift-storage-0"] Dec 03 12:16:48 crc kubenswrapper[5052]: I1203 12:16:48.110386 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:16:48 crc kubenswrapper[5052]: I1203 12:16:48.116758 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"swift-conf" Dec 03 12:16:48 crc kubenswrapper[5052]: I1203 12:16:48.116789 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"swift-ring-files" Dec 03 12:16:48 crc kubenswrapper[5052]: I1203 12:16:48.116875 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"swift-swift-dockercfg-6djml" Dec 03 12:16:48 crc kubenswrapper[5052]: I1203 12:16:48.116978 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"swift-storage-config-data" Dec 03 12:16:48 crc kubenswrapper[5052]: I1203 12:16:48.134316 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:16:48 crc kubenswrapper[5052]: I1203 12:16:48.139253 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/swift-storage-0"] Dec 03 12:16:48 crc kubenswrapper[5052]: I1203 12:16:48.214716 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:16:48 crc kubenswrapper[5052]: I1203 12:16:48.231409 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/335ad996-ad3b-4ac8-a488-64e9453f24a4-etc-swift\") pod \"swift-storage-0\" (UID: \"335ad996-ad3b-4ac8-a488-64e9453f24a4\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:16:48 crc kubenswrapper[5052]: I1203 12:16:48.231501 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lock\" (UniqueName: \"kubernetes.io/empty-dir/335ad996-ad3b-4ac8-a488-64e9453f24a4-lock\") pod \"swift-storage-0\" (UID: \"335ad996-ad3b-4ac8-a488-64e9453f24a4\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:16:48 crc kubenswrapper[5052]: I1203 12:16:48.231583 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage05-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage05-crc\") pod \"swift-storage-0\" (UID: \"335ad996-ad3b-4ac8-a488-64e9453f24a4\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:16:48 crc kubenswrapper[5052]: I1203 12:16:48.231682 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n4bgm\" (UniqueName: \"kubernetes.io/projected/335ad996-ad3b-4ac8-a488-64e9453f24a4-kube-api-access-n4bgm\") pod \"swift-storage-0\" (UID: \"335ad996-ad3b-4ac8-a488-64e9453f24a4\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:16:48 crc kubenswrapper[5052]: I1203 12:16:48.231738 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cache\" (UniqueName: \"kubernetes.io/empty-dir/335ad996-ad3b-4ac8-a488-64e9453f24a4-cache\") pod \"swift-storage-0\" (UID: \"335ad996-ad3b-4ac8-a488-64e9453f24a4\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:16:48 crc kubenswrapper[5052]: I1203 12:16:48.333949 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/335ad996-ad3b-4ac8-a488-64e9453f24a4-etc-swift\") pod \"swift-storage-0\" (UID: \"335ad996-ad3b-4ac8-a488-64e9453f24a4\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:16:48 crc kubenswrapper[5052]: I1203 12:16:48.334052 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"lock\" (UniqueName: \"kubernetes.io/empty-dir/335ad996-ad3b-4ac8-a488-64e9453f24a4-lock\") pod \"swift-storage-0\" (UID: \"335ad996-ad3b-4ac8-a488-64e9453f24a4\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:16:48 crc kubenswrapper[5052]: E1203 12:16:48.334147 5052 projected.go:288] Couldn't get configMap openstack-kuttl-tests/swift-ring-files: configmap "swift-ring-files" not found Dec 03 12:16:48 crc kubenswrapper[5052]: E1203 12:16:48.334183 5052 projected.go:194] Error preparing data for projected volume etc-swift for pod openstack-kuttl-tests/swift-storage-0: configmap "swift-ring-files" not found Dec 03 12:16:48 crc kubenswrapper[5052]: E1203 12:16:48.334242 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/335ad996-ad3b-4ac8-a488-64e9453f24a4-etc-swift podName:335ad996-ad3b-4ac8-a488-64e9453f24a4 nodeName:}" failed. No retries permitted until 2025-12-03 12:16:48.834223386 +0000 UTC m=+3399.422465150 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "etc-swift" (UniqueName: "kubernetes.io/projected/335ad996-ad3b-4ac8-a488-64e9453f24a4-etc-swift") pod "swift-storage-0" (UID: "335ad996-ad3b-4ac8-a488-64e9453f24a4") : configmap "swift-ring-files" not found Dec 03 12:16:48 crc kubenswrapper[5052]: I1203 12:16:48.334533 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"lock\" (UniqueName: \"kubernetes.io/empty-dir/335ad996-ad3b-4ac8-a488-64e9453f24a4-lock\") pod \"swift-storage-0\" (UID: \"335ad996-ad3b-4ac8-a488-64e9453f24a4\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:16:48 crc kubenswrapper[5052]: I1203 12:16:48.334090 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage05-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage05-crc\") pod \"swift-storage-0\" (UID: \"335ad996-ad3b-4ac8-a488-64e9453f24a4\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:16:48 crc kubenswrapper[5052]: I1203 12:16:48.334688 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-n4bgm\" (UniqueName: \"kubernetes.io/projected/335ad996-ad3b-4ac8-a488-64e9453f24a4-kube-api-access-n4bgm\") pod \"swift-storage-0\" (UID: \"335ad996-ad3b-4ac8-a488-64e9453f24a4\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:16:48 crc kubenswrapper[5052]: I1203 12:16:48.334851 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage05-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage05-crc\") pod \"swift-storage-0\" (UID: \"335ad996-ad3b-4ac8-a488-64e9453f24a4\") device mount path \"/mnt/openstack/pv05\"" pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:16:48 crc kubenswrapper[5052]: I1203 12:16:48.335150 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"cache\" (UniqueName: \"kubernetes.io/empty-dir/335ad996-ad3b-4ac8-a488-64e9453f24a4-cache\") pod \"swift-storage-0\" (UID: \"335ad996-ad3b-4ac8-a488-64e9453f24a4\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:16:48 crc kubenswrapper[5052]: I1203 12:16:48.335793 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"cache\" (UniqueName: \"kubernetes.io/empty-dir/335ad996-ad3b-4ac8-a488-64e9453f24a4-cache\") pod \"swift-storage-0\" (UID: \"335ad996-ad3b-4ac8-a488-64e9453f24a4\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:16:48 crc kubenswrapper[5052]: I1203 12:16:48.357610 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-n4bgm\" (UniqueName: \"kubernetes.io/projected/335ad996-ad3b-4ac8-a488-64e9453f24a4-kube-api-access-n4bgm\") pod \"swift-storage-0\" (UID: \"335ad996-ad3b-4ac8-a488-64e9453f24a4\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:16:48 crc kubenswrapper[5052]: I1203 12:16:48.359324 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage05-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage05-crc\") pod \"swift-storage-0\" (UID: \"335ad996-ad3b-4ac8-a488-64e9453f24a4\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:16:48 crc kubenswrapper[5052]: I1203 12:16:48.459051 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/swift-ring-rebalance-shz9b"] Dec 03 12:16:48 crc kubenswrapper[5052]: I1203 12:16:48.460110 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-ring-rebalance-shz9b" Dec 03 12:16:48 crc kubenswrapper[5052]: I1203 12:16:48.461826 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"swift-proxy-config-data" Dec 03 12:16:48 crc kubenswrapper[5052]: I1203 12:16:48.463674 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"swift-ring-config-data" Dec 03 12:16:48 crc kubenswrapper[5052]: I1203 12:16:48.463708 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"swift-ring-scripts" Dec 03 12:16:48 crc kubenswrapper[5052]: I1203 12:16:48.473593 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/swift-ring-rebalance-shz9b"] Dec 03 12:16:48 crc kubenswrapper[5052]: I1203 12:16:48.639556 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"swiftconf\" (UniqueName: \"kubernetes.io/secret/33837271-e865-4153-afab-6de6235c8ea8-swiftconf\") pod \"swift-ring-rebalance-shz9b\" (UID: \"33837271-e865-4153-afab-6de6235c8ea8\") " pod="openstack-kuttl-tests/swift-ring-rebalance-shz9b" Dec 03 12:16:48 crc kubenswrapper[5052]: I1203 12:16:48.639654 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ring-data-devices\" (UniqueName: \"kubernetes.io/configmap/33837271-e865-4153-afab-6de6235c8ea8-ring-data-devices\") pod \"swift-ring-rebalance-shz9b\" (UID: \"33837271-e865-4153-afab-6de6235c8ea8\") " pod="openstack-kuttl-tests/swift-ring-rebalance-shz9b" Dec 03 12:16:48 crc kubenswrapper[5052]: I1203 12:16:48.639714 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/empty-dir/33837271-e865-4153-afab-6de6235c8ea8-etc-swift\") pod \"swift-ring-rebalance-shz9b\" (UID: \"33837271-e865-4153-afab-6de6235c8ea8\") " pod="openstack-kuttl-tests/swift-ring-rebalance-shz9b" Dec 03 12:16:48 crc kubenswrapper[5052]: I1203 12:16:48.639765 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nq2db\" (UniqueName: \"kubernetes.io/projected/33837271-e865-4153-afab-6de6235c8ea8-kube-api-access-nq2db\") pod \"swift-ring-rebalance-shz9b\" (UID: \"33837271-e865-4153-afab-6de6235c8ea8\") " pod="openstack-kuttl-tests/swift-ring-rebalance-shz9b" Dec 03 12:16:48 crc kubenswrapper[5052]: I1203 12:16:48.639812 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/33837271-e865-4153-afab-6de6235c8ea8-combined-ca-bundle\") pod \"swift-ring-rebalance-shz9b\" (UID: \"33837271-e865-4153-afab-6de6235c8ea8\") " pod="openstack-kuttl-tests/swift-ring-rebalance-shz9b" Dec 03 12:16:48 crc kubenswrapper[5052]: I1203 12:16:48.639893 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/33837271-e865-4153-afab-6de6235c8ea8-scripts\") pod \"swift-ring-rebalance-shz9b\" (UID: \"33837271-e865-4153-afab-6de6235c8ea8\") " pod="openstack-kuttl-tests/swift-ring-rebalance-shz9b" Dec 03 12:16:48 crc kubenswrapper[5052]: I1203 12:16:48.639921 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dispersionconf\" (UniqueName: \"kubernetes.io/secret/33837271-e865-4153-afab-6de6235c8ea8-dispersionconf\") pod \"swift-ring-rebalance-shz9b\" (UID: \"33837271-e865-4153-afab-6de6235c8ea8\") " pod="openstack-kuttl-tests/swift-ring-rebalance-shz9b" Dec 03 12:16:48 crc kubenswrapper[5052]: I1203 12:16:48.741636 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"swiftconf\" (UniqueName: \"kubernetes.io/secret/33837271-e865-4153-afab-6de6235c8ea8-swiftconf\") pod \"swift-ring-rebalance-shz9b\" (UID: \"33837271-e865-4153-afab-6de6235c8ea8\") " pod="openstack-kuttl-tests/swift-ring-rebalance-shz9b" Dec 03 12:16:48 crc kubenswrapper[5052]: I1203 12:16:48.741732 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ring-data-devices\" (UniqueName: \"kubernetes.io/configmap/33837271-e865-4153-afab-6de6235c8ea8-ring-data-devices\") pod \"swift-ring-rebalance-shz9b\" (UID: \"33837271-e865-4153-afab-6de6235c8ea8\") " pod="openstack-kuttl-tests/swift-ring-rebalance-shz9b" Dec 03 12:16:48 crc kubenswrapper[5052]: I1203 12:16:48.741759 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/empty-dir/33837271-e865-4153-afab-6de6235c8ea8-etc-swift\") pod \"swift-ring-rebalance-shz9b\" (UID: \"33837271-e865-4153-afab-6de6235c8ea8\") " pod="openstack-kuttl-tests/swift-ring-rebalance-shz9b" Dec 03 12:16:48 crc kubenswrapper[5052]: I1203 12:16:48.741791 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-nq2db\" (UniqueName: \"kubernetes.io/projected/33837271-e865-4153-afab-6de6235c8ea8-kube-api-access-nq2db\") pod \"swift-ring-rebalance-shz9b\" (UID: \"33837271-e865-4153-afab-6de6235c8ea8\") " pod="openstack-kuttl-tests/swift-ring-rebalance-shz9b" Dec 03 12:16:48 crc kubenswrapper[5052]: I1203 12:16:48.741826 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/33837271-e865-4153-afab-6de6235c8ea8-combined-ca-bundle\") pod \"swift-ring-rebalance-shz9b\" (UID: \"33837271-e865-4153-afab-6de6235c8ea8\") " pod="openstack-kuttl-tests/swift-ring-rebalance-shz9b" Dec 03 12:16:48 crc kubenswrapper[5052]: I1203 12:16:48.741876 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/33837271-e865-4153-afab-6de6235c8ea8-scripts\") pod \"swift-ring-rebalance-shz9b\" (UID: \"33837271-e865-4153-afab-6de6235c8ea8\") " pod="openstack-kuttl-tests/swift-ring-rebalance-shz9b" Dec 03 12:16:48 crc kubenswrapper[5052]: I1203 12:16:48.741894 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"dispersionconf\" (UniqueName: \"kubernetes.io/secret/33837271-e865-4153-afab-6de6235c8ea8-dispersionconf\") pod \"swift-ring-rebalance-shz9b\" (UID: \"33837271-e865-4153-afab-6de6235c8ea8\") " pod="openstack-kuttl-tests/swift-ring-rebalance-shz9b" Dec 03 12:16:48 crc kubenswrapper[5052]: I1203 12:16:48.742713 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ring-data-devices\" (UniqueName: \"kubernetes.io/configmap/33837271-e865-4153-afab-6de6235c8ea8-ring-data-devices\") pod \"swift-ring-rebalance-shz9b\" (UID: \"33837271-e865-4153-afab-6de6235c8ea8\") " pod="openstack-kuttl-tests/swift-ring-rebalance-shz9b" Dec 03 12:16:48 crc kubenswrapper[5052]: I1203 12:16:48.742768 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/33837271-e865-4153-afab-6de6235c8ea8-scripts\") pod \"swift-ring-rebalance-shz9b\" (UID: \"33837271-e865-4153-afab-6de6235c8ea8\") " pod="openstack-kuttl-tests/swift-ring-rebalance-shz9b" Dec 03 12:16:48 crc kubenswrapper[5052]: I1203 12:16:48.743039 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"etc-swift\" (UniqueName: \"kubernetes.io/empty-dir/33837271-e865-4153-afab-6de6235c8ea8-etc-swift\") pod \"swift-ring-rebalance-shz9b\" (UID: \"33837271-e865-4153-afab-6de6235c8ea8\") " pod="openstack-kuttl-tests/swift-ring-rebalance-shz9b" Dec 03 12:16:48 crc kubenswrapper[5052]: I1203 12:16:48.747677 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"swiftconf\" (UniqueName: \"kubernetes.io/secret/33837271-e865-4153-afab-6de6235c8ea8-swiftconf\") pod \"swift-ring-rebalance-shz9b\" (UID: \"33837271-e865-4153-afab-6de6235c8ea8\") " pod="openstack-kuttl-tests/swift-ring-rebalance-shz9b" Dec 03 12:16:48 crc kubenswrapper[5052]: I1203 12:16:48.748972 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/33837271-e865-4153-afab-6de6235c8ea8-combined-ca-bundle\") pod \"swift-ring-rebalance-shz9b\" (UID: \"33837271-e865-4153-afab-6de6235c8ea8\") " pod="openstack-kuttl-tests/swift-ring-rebalance-shz9b" Dec 03 12:16:48 crc kubenswrapper[5052]: I1203 12:16:48.754836 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"dispersionconf\" (UniqueName: \"kubernetes.io/secret/33837271-e865-4153-afab-6de6235c8ea8-dispersionconf\") pod \"swift-ring-rebalance-shz9b\" (UID: \"33837271-e865-4153-afab-6de6235c8ea8\") " pod="openstack-kuttl-tests/swift-ring-rebalance-shz9b" Dec 03 12:16:48 crc kubenswrapper[5052]: I1203 12:16:48.795524 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-nq2db\" (UniqueName: \"kubernetes.io/projected/33837271-e865-4153-afab-6de6235c8ea8-kube-api-access-nq2db\") pod \"swift-ring-rebalance-shz9b\" (UID: \"33837271-e865-4153-afab-6de6235c8ea8\") " pod="openstack-kuttl-tests/swift-ring-rebalance-shz9b" Dec 03 12:16:48 crc kubenswrapper[5052]: I1203 12:16:48.843693 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/335ad996-ad3b-4ac8-a488-64e9453f24a4-etc-swift\") pod \"swift-storage-0\" (UID: \"335ad996-ad3b-4ac8-a488-64e9453f24a4\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:16:48 crc kubenswrapper[5052]: E1203 12:16:48.843954 5052 projected.go:288] Couldn't get configMap openstack-kuttl-tests/swift-ring-files: configmap "swift-ring-files" not found Dec 03 12:16:48 crc kubenswrapper[5052]: E1203 12:16:48.843988 5052 projected.go:194] Error preparing data for projected volume etc-swift for pod openstack-kuttl-tests/swift-storage-0: configmap "swift-ring-files" not found Dec 03 12:16:48 crc kubenswrapper[5052]: E1203 12:16:48.844051 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/335ad996-ad3b-4ac8-a488-64e9453f24a4-etc-swift podName:335ad996-ad3b-4ac8-a488-64e9453f24a4 nodeName:}" failed. No retries permitted until 2025-12-03 12:16:49.844027593 +0000 UTC m=+3400.432269357 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "etc-swift" (UniqueName: "kubernetes.io/projected/335ad996-ad3b-4ac8-a488-64e9453f24a4-etc-swift") pod "swift-storage-0" (UID: "335ad996-ad3b-4ac8-a488-64e9453f24a4") : configmap "swift-ring-files" not found Dec 03 12:16:48 crc kubenswrapper[5052]: I1203 12:16:48.943826 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:16:48 crc kubenswrapper[5052]: I1203 12:16:48.992548 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:16:49 crc kubenswrapper[5052]: I1203 12:16:49.084786 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-ring-rebalance-shz9b" Dec 03 12:16:49 crc kubenswrapper[5052]: I1203 12:16:49.254982 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ovn-northd-0"] Dec 03 12:16:49 crc kubenswrapper[5052]: I1203 12:16:49.257133 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:16:49 crc kubenswrapper[5052]: I1203 12:16:49.260391 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ovnnorthd-ovnnorthd-dockercfg-6wvg8" Dec 03 12:16:49 crc kubenswrapper[5052]: I1203 12:16:49.261190 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"ovnnorthd-scripts" Dec 03 12:16:49 crc kubenswrapper[5052]: I1203 12:16:49.271207 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"ovnnorthd-config" Dec 03 12:16:49 crc kubenswrapper[5052]: I1203 12:16:49.271468 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-ovnnorthd-ovndbs" Dec 03 12:16:49 crc kubenswrapper[5052]: I1203 12:16:49.283598 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ovn-northd-0"] Dec 03 12:16:49 crc kubenswrapper[5052]: I1203 12:16:49.354345 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/966185e5-f414-46c5-ba48-0e234750795a-metrics-certs-tls-certs\") pod \"ovn-northd-0\" (UID: \"966185e5-f414-46c5-ba48-0e234750795a\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:16:49 crc kubenswrapper[5052]: I1203 12:16:49.354428 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/966185e5-f414-46c5-ba48-0e234750795a-config\") pod \"ovn-northd-0\" (UID: \"966185e5-f414-46c5-ba48-0e234750795a\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:16:49 crc kubenswrapper[5052]: I1203 12:16:49.354474 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-58ns2\" (UniqueName: \"kubernetes.io/projected/966185e5-f414-46c5-ba48-0e234750795a-kube-api-access-58ns2\") pod \"ovn-northd-0\" (UID: \"966185e5-f414-46c5-ba48-0e234750795a\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:16:49 crc kubenswrapper[5052]: I1203 12:16:49.354528 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-northd-tls-certs\" (UniqueName: \"kubernetes.io/secret/966185e5-f414-46c5-ba48-0e234750795a-ovn-northd-tls-certs\") pod \"ovn-northd-0\" (UID: \"966185e5-f414-46c5-ba48-0e234750795a\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:16:49 crc kubenswrapper[5052]: I1203 12:16:49.354547 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-rundir\" (UniqueName: \"kubernetes.io/empty-dir/966185e5-f414-46c5-ba48-0e234750795a-ovn-rundir\") pod \"ovn-northd-0\" (UID: \"966185e5-f414-46c5-ba48-0e234750795a\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:16:49 crc kubenswrapper[5052]: I1203 12:16:49.354563 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/966185e5-f414-46c5-ba48-0e234750795a-scripts\") pod \"ovn-northd-0\" (UID: \"966185e5-f414-46c5-ba48-0e234750795a\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:16:49 crc kubenswrapper[5052]: I1203 12:16:49.354613 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/966185e5-f414-46c5-ba48-0e234750795a-combined-ca-bundle\") pod \"ovn-northd-0\" (UID: \"966185e5-f414-46c5-ba48-0e234750795a\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:16:49 crc kubenswrapper[5052]: I1203 12:16:49.456773 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/966185e5-f414-46c5-ba48-0e234750795a-metrics-certs-tls-certs\") pod \"ovn-northd-0\" (UID: \"966185e5-f414-46c5-ba48-0e234750795a\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:16:49 crc kubenswrapper[5052]: I1203 12:16:49.457450 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/966185e5-f414-46c5-ba48-0e234750795a-config\") pod \"ovn-northd-0\" (UID: \"966185e5-f414-46c5-ba48-0e234750795a\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:16:49 crc kubenswrapper[5052]: I1203 12:16:49.457547 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-58ns2\" (UniqueName: \"kubernetes.io/projected/966185e5-f414-46c5-ba48-0e234750795a-kube-api-access-58ns2\") pod \"ovn-northd-0\" (UID: \"966185e5-f414-46c5-ba48-0e234750795a\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:16:49 crc kubenswrapper[5052]: I1203 12:16:49.457617 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovn-northd-tls-certs\" (UniqueName: \"kubernetes.io/secret/966185e5-f414-46c5-ba48-0e234750795a-ovn-northd-tls-certs\") pod \"ovn-northd-0\" (UID: \"966185e5-f414-46c5-ba48-0e234750795a\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:16:49 crc kubenswrapper[5052]: I1203 12:16:49.457647 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovn-rundir\" (UniqueName: \"kubernetes.io/empty-dir/966185e5-f414-46c5-ba48-0e234750795a-ovn-rundir\") pod \"ovn-northd-0\" (UID: \"966185e5-f414-46c5-ba48-0e234750795a\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:16:49 crc kubenswrapper[5052]: I1203 12:16:49.457671 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/966185e5-f414-46c5-ba48-0e234750795a-scripts\") pod \"ovn-northd-0\" (UID: \"966185e5-f414-46c5-ba48-0e234750795a\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:16:49 crc kubenswrapper[5052]: I1203 12:16:49.457715 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/966185e5-f414-46c5-ba48-0e234750795a-combined-ca-bundle\") pod \"ovn-northd-0\" (UID: \"966185e5-f414-46c5-ba48-0e234750795a\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:16:49 crc kubenswrapper[5052]: I1203 12:16:49.458369 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovn-rundir\" (UniqueName: \"kubernetes.io/empty-dir/966185e5-f414-46c5-ba48-0e234750795a-ovn-rundir\") pod \"ovn-northd-0\" (UID: \"966185e5-f414-46c5-ba48-0e234750795a\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:16:49 crc kubenswrapper[5052]: I1203 12:16:49.458441 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/966185e5-f414-46c5-ba48-0e234750795a-config\") pod \"ovn-northd-0\" (UID: \"966185e5-f414-46c5-ba48-0e234750795a\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:16:49 crc kubenswrapper[5052]: I1203 12:16:49.458729 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/966185e5-f414-46c5-ba48-0e234750795a-scripts\") pod \"ovn-northd-0\" (UID: \"966185e5-f414-46c5-ba48-0e234750795a\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:16:49 crc kubenswrapper[5052]: I1203 12:16:49.464568 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovn-northd-tls-certs\" (UniqueName: \"kubernetes.io/secret/966185e5-f414-46c5-ba48-0e234750795a-ovn-northd-tls-certs\") pod \"ovn-northd-0\" (UID: \"966185e5-f414-46c5-ba48-0e234750795a\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:16:49 crc kubenswrapper[5052]: I1203 12:16:49.464783 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/966185e5-f414-46c5-ba48-0e234750795a-combined-ca-bundle\") pod \"ovn-northd-0\" (UID: \"966185e5-f414-46c5-ba48-0e234750795a\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:16:49 crc kubenswrapper[5052]: I1203 12:16:49.465190 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/966185e5-f414-46c5-ba48-0e234750795a-metrics-certs-tls-certs\") pod \"ovn-northd-0\" (UID: \"966185e5-f414-46c5-ba48-0e234750795a\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:16:49 crc kubenswrapper[5052]: I1203 12:16:49.486473 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-58ns2\" (UniqueName: \"kubernetes.io/projected/966185e5-f414-46c5-ba48-0e234750795a-kube-api-access-58ns2\") pod \"ovn-northd-0\" (UID: \"966185e5-f414-46c5-ba48-0e234750795a\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:16:49 crc kubenswrapper[5052]: I1203 12:16:49.590536 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:16:49 crc kubenswrapper[5052]: I1203 12:16:49.873571 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/335ad996-ad3b-4ac8-a488-64e9453f24a4-etc-swift\") pod \"swift-storage-0\" (UID: \"335ad996-ad3b-4ac8-a488-64e9453f24a4\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:16:49 crc kubenswrapper[5052]: E1203 12:16:49.873911 5052 projected.go:288] Couldn't get configMap openstack-kuttl-tests/swift-ring-files: configmap "swift-ring-files" not found Dec 03 12:16:49 crc kubenswrapper[5052]: E1203 12:16:49.877363 5052 projected.go:194] Error preparing data for projected volume etc-swift for pod openstack-kuttl-tests/swift-storage-0: configmap "swift-ring-files" not found Dec 03 12:16:49 crc kubenswrapper[5052]: E1203 12:16:49.877453 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/335ad996-ad3b-4ac8-a488-64e9453f24a4-etc-swift podName:335ad996-ad3b-4ac8-a488-64e9453f24a4 nodeName:}" failed. No retries permitted until 2025-12-03 12:16:51.877432309 +0000 UTC m=+3402.465674143 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "etc-swift" (UniqueName: "kubernetes.io/projected/335ad996-ad3b-4ac8-a488-64e9453f24a4-etc-swift") pod "swift-storage-0" (UID: "335ad996-ad3b-4ac8-a488-64e9453f24a4") : configmap "swift-ring-files" not found Dec 03 12:16:49 crc kubenswrapper[5052]: I1203 12:16:49.877775 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/swift-ring-rebalance-shz9b"] Dec 03 12:16:49 crc kubenswrapper[5052]: W1203 12:16:49.886682 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod33837271_e865_4153_afab_6de6235c8ea8.slice/crio-e501e08aa953468d0f7140a9c4c2f318ef52b35f5d158db84ed71d90b37e777f WatchSource:0}: Error finding container e501e08aa953468d0f7140a9c4c2f318ef52b35f5d158db84ed71d90b37e777f: Status 404 returned error can't find the container with id e501e08aa953468d0f7140a9c4c2f318ef52b35f5d158db84ed71d90b37e777f Dec 03 12:16:50 crc kubenswrapper[5052]: I1203 12:16:50.117770 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ovn-northd-0"] Dec 03 12:16:50 crc kubenswrapper[5052]: W1203 12:16:50.120595 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod966185e5_f414_46c5_ba48_0e234750795a.slice/crio-d95de27c91a991d126bd6b66ff7ca758270a3709bf07db38f0dd347282027b49 WatchSource:0}: Error finding container d95de27c91a991d126bd6b66ff7ca758270a3709bf07db38f0dd347282027b49: Status 404 returned error can't find the container with id d95de27c91a991d126bd6b66ff7ca758270a3709bf07db38f0dd347282027b49 Dec 03 12:16:50 crc kubenswrapper[5052]: I1203 12:16:50.813780 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-ring-rebalance-shz9b" event={"ID":"33837271-e865-4153-afab-6de6235c8ea8","Type":"ContainerStarted","Data":"231d5f3ea609c0fd0b8f6c9745cde813952d26ceaed8f036d11f3d3a0034c03a"} Dec 03 12:16:50 crc kubenswrapper[5052]: I1203 12:16:50.813833 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-ring-rebalance-shz9b" event={"ID":"33837271-e865-4153-afab-6de6235c8ea8","Type":"ContainerStarted","Data":"e501e08aa953468d0f7140a9c4c2f318ef52b35f5d158db84ed71d90b37e777f"} Dec 03 12:16:50 crc kubenswrapper[5052]: I1203 12:16:50.814761 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovn-northd-0" event={"ID":"966185e5-f414-46c5-ba48-0e234750795a","Type":"ContainerStarted","Data":"d95de27c91a991d126bd6b66ff7ca758270a3709bf07db38f0dd347282027b49"} Dec 03 12:16:51 crc kubenswrapper[5052]: I1203 12:16:51.826199 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovn-northd-0" event={"ID":"966185e5-f414-46c5-ba48-0e234750795a","Type":"ContainerStarted","Data":"da2e79425aa667a3b6c86223531989383a59d7905be7837190a7bae4c94575bc"} Dec 03 12:16:51 crc kubenswrapper[5052]: I1203 12:16:51.826631 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovn-northd-0" event={"ID":"966185e5-f414-46c5-ba48-0e234750795a","Type":"ContainerStarted","Data":"ef532f5fb83b3ed8c9243e5c20875e68a9ee6b646ee961f30ffe04c5b504e0f8"} Dec 03 12:16:51 crc kubenswrapper[5052]: I1203 12:16:51.851311 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/swift-ring-rebalance-shz9b" podStartSLOduration=3.851286739 podStartE2EDuration="3.851286739s" podCreationTimestamp="2025-12-03 12:16:48 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:16:51.840335016 +0000 UTC m=+3402.428576790" watchObservedRunningTime="2025-12-03 12:16:51.851286739 +0000 UTC m=+3402.439528503" Dec 03 12:16:51 crc kubenswrapper[5052]: I1203 12:16:51.914497 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/335ad996-ad3b-4ac8-a488-64e9453f24a4-etc-swift\") pod \"swift-storage-0\" (UID: \"335ad996-ad3b-4ac8-a488-64e9453f24a4\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:16:51 crc kubenswrapper[5052]: E1203 12:16:51.914646 5052 projected.go:288] Couldn't get configMap openstack-kuttl-tests/swift-ring-files: configmap "swift-ring-files" not found Dec 03 12:16:51 crc kubenswrapper[5052]: E1203 12:16:51.914668 5052 projected.go:194] Error preparing data for projected volume etc-swift for pod openstack-kuttl-tests/swift-storage-0: configmap "swift-ring-files" not found Dec 03 12:16:51 crc kubenswrapper[5052]: E1203 12:16:51.914718 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/335ad996-ad3b-4ac8-a488-64e9453f24a4-etc-swift podName:335ad996-ad3b-4ac8-a488-64e9453f24a4 nodeName:}" failed. No retries permitted until 2025-12-03 12:16:55.914702009 +0000 UTC m=+3406.502943773 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "etc-swift" (UniqueName: "kubernetes.io/projected/335ad996-ad3b-4ac8-a488-64e9453f24a4-etc-swift") pod "swift-storage-0" (UID: "335ad996-ad3b-4ac8-a488-64e9453f24a4") : configmap "swift-ring-files" not found Dec 03 12:16:52 crc kubenswrapper[5052]: I1203 12:16:52.834881 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:16:54 crc kubenswrapper[5052]: I1203 12:16:54.838441 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/ovn-northd-0" podStartSLOduration=5.83842312 podStartE2EDuration="5.83842312s" podCreationTimestamp="2025-12-03 12:16:49 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:16:52.85507968 +0000 UTC m=+3403.443321454" watchObservedRunningTime="2025-12-03 12:16:54.83842312 +0000 UTC m=+3405.426664884" Dec 03 12:16:54 crc kubenswrapper[5052]: I1203 12:16:54.839226 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/keystone-db-create-59gmf"] Dec 03 12:16:54 crc kubenswrapper[5052]: I1203 12:16:54.840424 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-db-create-59gmf" Dec 03 12:16:54 crc kubenswrapper[5052]: I1203 12:16:54.852464 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-db-create-59gmf"] Dec 03 12:16:54 crc kubenswrapper[5052]: I1203 12:16:54.864007 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/fc3fb6e4-34b1-403a-8761-f03573c6d993-operator-scripts\") pod \"keystone-db-create-59gmf\" (UID: \"fc3fb6e4-34b1-403a-8761-f03573c6d993\") " pod="openstack-kuttl-tests/keystone-db-create-59gmf" Dec 03 12:16:54 crc kubenswrapper[5052]: I1203 12:16:54.864092 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j9kmf\" (UniqueName: \"kubernetes.io/projected/fc3fb6e4-34b1-403a-8761-f03573c6d993-kube-api-access-j9kmf\") pod \"keystone-db-create-59gmf\" (UID: \"fc3fb6e4-34b1-403a-8761-f03573c6d993\") " pod="openstack-kuttl-tests/keystone-db-create-59gmf" Dec 03 12:16:54 crc kubenswrapper[5052]: I1203 12:16:54.954728 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/keystone-28d1-account-create-update-f58gf"] Dec 03 12:16:54 crc kubenswrapper[5052]: I1203 12:16:54.955851 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-28d1-account-create-update-f58gf" Dec 03 12:16:54 crc kubenswrapper[5052]: I1203 12:16:54.958489 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone-db-secret" Dec 03 12:16:54 crc kubenswrapper[5052]: I1203 12:16:54.964321 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-28d1-account-create-update-f58gf"] Dec 03 12:16:54 crc kubenswrapper[5052]: I1203 12:16:54.965163 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-j9kmf\" (UniqueName: \"kubernetes.io/projected/fc3fb6e4-34b1-403a-8761-f03573c6d993-kube-api-access-j9kmf\") pod \"keystone-db-create-59gmf\" (UID: \"fc3fb6e4-34b1-403a-8761-f03573c6d993\") " pod="openstack-kuttl-tests/keystone-db-create-59gmf" Dec 03 12:16:54 crc kubenswrapper[5052]: I1203 12:16:54.965313 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/fc3fb6e4-34b1-403a-8761-f03573c6d993-operator-scripts\") pod \"keystone-db-create-59gmf\" (UID: \"fc3fb6e4-34b1-403a-8761-f03573c6d993\") " pod="openstack-kuttl-tests/keystone-db-create-59gmf" Dec 03 12:16:54 crc kubenswrapper[5052]: I1203 12:16:54.966141 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/fc3fb6e4-34b1-403a-8761-f03573c6d993-operator-scripts\") pod \"keystone-db-create-59gmf\" (UID: \"fc3fb6e4-34b1-403a-8761-f03573c6d993\") " pod="openstack-kuttl-tests/keystone-db-create-59gmf" Dec 03 12:16:54 crc kubenswrapper[5052]: I1203 12:16:54.992693 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-j9kmf\" (UniqueName: \"kubernetes.io/projected/fc3fb6e4-34b1-403a-8761-f03573c6d993-kube-api-access-j9kmf\") pod \"keystone-db-create-59gmf\" (UID: \"fc3fb6e4-34b1-403a-8761-f03573c6d993\") " pod="openstack-kuttl-tests/keystone-db-create-59gmf" Dec 03 12:16:55 crc kubenswrapper[5052]: I1203 12:16:55.067083 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5wb5l\" (UniqueName: \"kubernetes.io/projected/23c58e9d-3154-4887-8a51-1601ad306abf-kube-api-access-5wb5l\") pod \"keystone-28d1-account-create-update-f58gf\" (UID: \"23c58e9d-3154-4887-8a51-1601ad306abf\") " pod="openstack-kuttl-tests/keystone-28d1-account-create-update-f58gf" Dec 03 12:16:55 crc kubenswrapper[5052]: I1203 12:16:55.067539 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/23c58e9d-3154-4887-8a51-1601ad306abf-operator-scripts\") pod \"keystone-28d1-account-create-update-f58gf\" (UID: \"23c58e9d-3154-4887-8a51-1601ad306abf\") " pod="openstack-kuttl-tests/keystone-28d1-account-create-update-f58gf" Dec 03 12:16:55 crc kubenswrapper[5052]: I1203 12:16:55.168694 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/placement-db-create-r5z4q"] Dec 03 12:16:55 crc kubenswrapper[5052]: I1203 12:16:55.169597 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/23c58e9d-3154-4887-8a51-1601ad306abf-operator-scripts\") pod \"keystone-28d1-account-create-update-f58gf\" (UID: \"23c58e9d-3154-4887-8a51-1601ad306abf\") " pod="openstack-kuttl-tests/keystone-28d1-account-create-update-f58gf" Dec 03 12:16:55 crc kubenswrapper[5052]: I1203 12:16:55.169780 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-5wb5l\" (UniqueName: \"kubernetes.io/projected/23c58e9d-3154-4887-8a51-1601ad306abf-kube-api-access-5wb5l\") pod \"keystone-28d1-account-create-update-f58gf\" (UID: \"23c58e9d-3154-4887-8a51-1601ad306abf\") " pod="openstack-kuttl-tests/keystone-28d1-account-create-update-f58gf" Dec 03 12:16:55 crc kubenswrapper[5052]: I1203 12:16:55.170343 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-db-create-r5z4q" Dec 03 12:16:55 crc kubenswrapper[5052]: I1203 12:16:55.170672 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/23c58e9d-3154-4887-8a51-1601ad306abf-operator-scripts\") pod \"keystone-28d1-account-create-update-f58gf\" (UID: \"23c58e9d-3154-4887-8a51-1601ad306abf\") " pod="openstack-kuttl-tests/keystone-28d1-account-create-update-f58gf" Dec 03 12:16:55 crc kubenswrapper[5052]: I1203 12:16:55.172607 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-db-create-59gmf" Dec 03 12:16:55 crc kubenswrapper[5052]: I1203 12:16:55.182286 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/placement-db-create-r5z4q"] Dec 03 12:16:55 crc kubenswrapper[5052]: I1203 12:16:55.219121 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-5wb5l\" (UniqueName: \"kubernetes.io/projected/23c58e9d-3154-4887-8a51-1601ad306abf-kube-api-access-5wb5l\") pod \"keystone-28d1-account-create-update-f58gf\" (UID: \"23c58e9d-3154-4887-8a51-1601ad306abf\") " pod="openstack-kuttl-tests/keystone-28d1-account-create-update-f58gf" Dec 03 12:16:55 crc kubenswrapper[5052]: I1203 12:16:55.271615 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fb6jz\" (UniqueName: \"kubernetes.io/projected/9b7bd5ab-a50e-4a66-ade7-c09c913c481f-kube-api-access-fb6jz\") pod \"placement-db-create-r5z4q\" (UID: \"9b7bd5ab-a50e-4a66-ade7-c09c913c481f\") " pod="openstack-kuttl-tests/placement-db-create-r5z4q" Dec 03 12:16:55 crc kubenswrapper[5052]: I1203 12:16:55.271938 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/9b7bd5ab-a50e-4a66-ade7-c09c913c481f-operator-scripts\") pod \"placement-db-create-r5z4q\" (UID: \"9b7bd5ab-a50e-4a66-ade7-c09c913c481f\") " pod="openstack-kuttl-tests/placement-db-create-r5z4q" Dec 03 12:16:55 crc kubenswrapper[5052]: I1203 12:16:55.275802 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-28d1-account-create-update-f58gf" Dec 03 12:16:55 crc kubenswrapper[5052]: I1203 12:16:55.297335 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/placement-9152-account-create-update-sgs4t"] Dec 03 12:16:55 crc kubenswrapper[5052]: I1203 12:16:55.299417 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-9152-account-create-update-sgs4t" Dec 03 12:16:55 crc kubenswrapper[5052]: I1203 12:16:55.307052 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"placement-db-secret" Dec 03 12:16:55 crc kubenswrapper[5052]: I1203 12:16:55.320669 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/placement-9152-account-create-update-sgs4t"] Dec 03 12:16:55 crc kubenswrapper[5052]: I1203 12:16:55.374467 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bf2m2\" (UniqueName: \"kubernetes.io/projected/5e64bff0-44a3-472d-8b94-210a31a0e4e6-kube-api-access-bf2m2\") pod \"placement-9152-account-create-update-sgs4t\" (UID: \"5e64bff0-44a3-472d-8b94-210a31a0e4e6\") " pod="openstack-kuttl-tests/placement-9152-account-create-update-sgs4t" Dec 03 12:16:55 crc kubenswrapper[5052]: I1203 12:16:55.375005 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/5e64bff0-44a3-472d-8b94-210a31a0e4e6-operator-scripts\") pod \"placement-9152-account-create-update-sgs4t\" (UID: \"5e64bff0-44a3-472d-8b94-210a31a0e4e6\") " pod="openstack-kuttl-tests/placement-9152-account-create-update-sgs4t" Dec 03 12:16:55 crc kubenswrapper[5052]: I1203 12:16:55.375127 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-fb6jz\" (UniqueName: \"kubernetes.io/projected/9b7bd5ab-a50e-4a66-ade7-c09c913c481f-kube-api-access-fb6jz\") pod \"placement-db-create-r5z4q\" (UID: \"9b7bd5ab-a50e-4a66-ade7-c09c913c481f\") " pod="openstack-kuttl-tests/placement-db-create-r5z4q" Dec 03 12:16:55 crc kubenswrapper[5052]: I1203 12:16:55.375175 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/9b7bd5ab-a50e-4a66-ade7-c09c913c481f-operator-scripts\") pod \"placement-db-create-r5z4q\" (UID: \"9b7bd5ab-a50e-4a66-ade7-c09c913c481f\") " pod="openstack-kuttl-tests/placement-db-create-r5z4q" Dec 03 12:16:55 crc kubenswrapper[5052]: I1203 12:16:55.376157 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/9b7bd5ab-a50e-4a66-ade7-c09c913c481f-operator-scripts\") pod \"placement-db-create-r5z4q\" (UID: \"9b7bd5ab-a50e-4a66-ade7-c09c913c481f\") " pod="openstack-kuttl-tests/placement-db-create-r5z4q" Dec 03 12:16:55 crc kubenswrapper[5052]: I1203 12:16:55.394965 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-fb6jz\" (UniqueName: \"kubernetes.io/projected/9b7bd5ab-a50e-4a66-ade7-c09c913c481f-kube-api-access-fb6jz\") pod \"placement-db-create-r5z4q\" (UID: \"9b7bd5ab-a50e-4a66-ade7-c09c913c481f\") " pod="openstack-kuttl-tests/placement-db-create-r5z4q" Dec 03 12:16:55 crc kubenswrapper[5052]: I1203 12:16:55.473475 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/glance-db-create-d6shg"] Dec 03 12:16:55 crc kubenswrapper[5052]: I1203 12:16:55.474604 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-db-create-d6shg" Dec 03 12:16:55 crc kubenswrapper[5052]: I1203 12:16:55.477119 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-bf2m2\" (UniqueName: \"kubernetes.io/projected/5e64bff0-44a3-472d-8b94-210a31a0e4e6-kube-api-access-bf2m2\") pod \"placement-9152-account-create-update-sgs4t\" (UID: \"5e64bff0-44a3-472d-8b94-210a31a0e4e6\") " pod="openstack-kuttl-tests/placement-9152-account-create-update-sgs4t" Dec 03 12:16:55 crc kubenswrapper[5052]: I1203 12:16:55.477223 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/5e64bff0-44a3-472d-8b94-210a31a0e4e6-operator-scripts\") pod \"placement-9152-account-create-update-sgs4t\" (UID: \"5e64bff0-44a3-472d-8b94-210a31a0e4e6\") " pod="openstack-kuttl-tests/placement-9152-account-create-update-sgs4t" Dec 03 12:16:55 crc kubenswrapper[5052]: I1203 12:16:55.478031 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/5e64bff0-44a3-472d-8b94-210a31a0e4e6-operator-scripts\") pod \"placement-9152-account-create-update-sgs4t\" (UID: \"5e64bff0-44a3-472d-8b94-210a31a0e4e6\") " pod="openstack-kuttl-tests/placement-9152-account-create-update-sgs4t" Dec 03 12:16:55 crc kubenswrapper[5052]: I1203 12:16:55.494940 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-bf2m2\" (UniqueName: \"kubernetes.io/projected/5e64bff0-44a3-472d-8b94-210a31a0e4e6-kube-api-access-bf2m2\") pod \"placement-9152-account-create-update-sgs4t\" (UID: \"5e64bff0-44a3-472d-8b94-210a31a0e4e6\") " pod="openstack-kuttl-tests/placement-9152-account-create-update-sgs4t" Dec 03 12:16:55 crc kubenswrapper[5052]: I1203 12:16:55.497046 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-db-create-d6shg"] Dec 03 12:16:55 crc kubenswrapper[5052]: I1203 12:16:55.578820 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/b3c91987-f9ac-46b0-a634-33c314ab4576-operator-scripts\") pod \"glance-db-create-d6shg\" (UID: \"b3c91987-f9ac-46b0-a634-33c314ab4576\") " pod="openstack-kuttl-tests/glance-db-create-d6shg" Dec 03 12:16:55 crc kubenswrapper[5052]: I1203 12:16:55.578882 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ps25q\" (UniqueName: \"kubernetes.io/projected/b3c91987-f9ac-46b0-a634-33c314ab4576-kube-api-access-ps25q\") pod \"glance-db-create-d6shg\" (UID: \"b3c91987-f9ac-46b0-a634-33c314ab4576\") " pod="openstack-kuttl-tests/glance-db-create-d6shg" Dec 03 12:16:55 crc kubenswrapper[5052]: I1203 12:16:55.584542 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-db-create-r5z4q" Dec 03 12:16:55 crc kubenswrapper[5052]: I1203 12:16:55.606840 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/glance-73ca-account-create-update-c8r5x"] Dec 03 12:16:55 crc kubenswrapper[5052]: I1203 12:16:55.607926 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-73ca-account-create-update-c8r5x" Dec 03 12:16:55 crc kubenswrapper[5052]: I1203 12:16:55.614473 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-73ca-account-create-update-c8r5x"] Dec 03 12:16:55 crc kubenswrapper[5052]: I1203 12:16:55.615895 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-db-secret" Dec 03 12:16:55 crc kubenswrapper[5052]: I1203 12:16:55.622951 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-9152-account-create-update-sgs4t" Dec 03 12:16:55 crc kubenswrapper[5052]: I1203 12:16:55.652408 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-db-create-59gmf"] Dec 03 12:16:55 crc kubenswrapper[5052]: I1203 12:16:55.681461 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/6b4e9ce4-5733-4df7-a4f8-f3d23e906b53-operator-scripts\") pod \"glance-73ca-account-create-update-c8r5x\" (UID: \"6b4e9ce4-5733-4df7-a4f8-f3d23e906b53\") " pod="openstack-kuttl-tests/glance-73ca-account-create-update-c8r5x" Dec 03 12:16:55 crc kubenswrapper[5052]: I1203 12:16:55.681502 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h289z\" (UniqueName: \"kubernetes.io/projected/6b4e9ce4-5733-4df7-a4f8-f3d23e906b53-kube-api-access-h289z\") pod \"glance-73ca-account-create-update-c8r5x\" (UID: \"6b4e9ce4-5733-4df7-a4f8-f3d23e906b53\") " pod="openstack-kuttl-tests/glance-73ca-account-create-update-c8r5x" Dec 03 12:16:55 crc kubenswrapper[5052]: I1203 12:16:55.681551 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-ps25q\" (UniqueName: \"kubernetes.io/projected/b3c91987-f9ac-46b0-a634-33c314ab4576-kube-api-access-ps25q\") pod \"glance-db-create-d6shg\" (UID: \"b3c91987-f9ac-46b0-a634-33c314ab4576\") " pod="openstack-kuttl-tests/glance-db-create-d6shg" Dec 03 12:16:55 crc kubenswrapper[5052]: I1203 12:16:55.681674 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/b3c91987-f9ac-46b0-a634-33c314ab4576-operator-scripts\") pod \"glance-db-create-d6shg\" (UID: \"b3c91987-f9ac-46b0-a634-33c314ab4576\") " pod="openstack-kuttl-tests/glance-db-create-d6shg" Dec 03 12:16:55 crc kubenswrapper[5052]: I1203 12:16:55.682393 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/b3c91987-f9ac-46b0-a634-33c314ab4576-operator-scripts\") pod \"glance-db-create-d6shg\" (UID: \"b3c91987-f9ac-46b0-a634-33c314ab4576\") " pod="openstack-kuttl-tests/glance-db-create-d6shg" Dec 03 12:16:55 crc kubenswrapper[5052]: I1203 12:16:55.709442 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-ps25q\" (UniqueName: \"kubernetes.io/projected/b3c91987-f9ac-46b0-a634-33c314ab4576-kube-api-access-ps25q\") pod \"glance-db-create-d6shg\" (UID: \"b3c91987-f9ac-46b0-a634-33c314ab4576\") " pod="openstack-kuttl-tests/glance-db-create-d6shg" Dec 03 12:16:55 crc kubenswrapper[5052]: I1203 12:16:55.782634 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-h289z\" (UniqueName: \"kubernetes.io/projected/6b4e9ce4-5733-4df7-a4f8-f3d23e906b53-kube-api-access-h289z\") pod \"glance-73ca-account-create-update-c8r5x\" (UID: \"6b4e9ce4-5733-4df7-a4f8-f3d23e906b53\") " pod="openstack-kuttl-tests/glance-73ca-account-create-update-c8r5x" Dec 03 12:16:55 crc kubenswrapper[5052]: I1203 12:16:55.783075 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/6b4e9ce4-5733-4df7-a4f8-f3d23e906b53-operator-scripts\") pod \"glance-73ca-account-create-update-c8r5x\" (UID: \"6b4e9ce4-5733-4df7-a4f8-f3d23e906b53\") " pod="openstack-kuttl-tests/glance-73ca-account-create-update-c8r5x" Dec 03 12:16:55 crc kubenswrapper[5052]: I1203 12:16:55.783747 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/6b4e9ce4-5733-4df7-a4f8-f3d23e906b53-operator-scripts\") pod \"glance-73ca-account-create-update-c8r5x\" (UID: \"6b4e9ce4-5733-4df7-a4f8-f3d23e906b53\") " pod="openstack-kuttl-tests/glance-73ca-account-create-update-c8r5x" Dec 03 12:16:55 crc kubenswrapper[5052]: I1203 12:16:55.810581 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-28d1-account-create-update-f58gf"] Dec 03 12:16:55 crc kubenswrapper[5052]: I1203 12:16:55.811247 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-h289z\" (UniqueName: \"kubernetes.io/projected/6b4e9ce4-5733-4df7-a4f8-f3d23e906b53-kube-api-access-h289z\") pod \"glance-73ca-account-create-update-c8r5x\" (UID: \"6b4e9ce4-5733-4df7-a4f8-f3d23e906b53\") " pod="openstack-kuttl-tests/glance-73ca-account-create-update-c8r5x" Dec 03 12:16:55 crc kubenswrapper[5052]: I1203 12:16:55.812214 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-db-create-d6shg" Dec 03 12:16:55 crc kubenswrapper[5052]: I1203 12:16:55.873247 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-db-create-59gmf" event={"ID":"fc3fb6e4-34b1-403a-8761-f03573c6d993","Type":"ContainerStarted","Data":"acabd11f67c578a289ff1b5572b82ec5c14a08efbfe4929ff13289c867543646"} Dec 03 12:16:55 crc kubenswrapper[5052]: I1203 12:16:55.873327 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-db-create-59gmf" event={"ID":"fc3fb6e4-34b1-403a-8761-f03573c6d993","Type":"ContainerStarted","Data":"0c5d43144974a8ab2a9483ef123db712d5adefd5b51c7a5d0024c4ae175f8f8d"} Dec 03 12:16:55 crc kubenswrapper[5052]: I1203 12:16:55.875346 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-28d1-account-create-update-f58gf" event={"ID":"23c58e9d-3154-4887-8a51-1601ad306abf","Type":"ContainerStarted","Data":"f5c3aa9aeea711c350aad6d9bdcc6e38bad899dc8ee79cb94fa9ba2369ae35df"} Dec 03 12:16:55 crc kubenswrapper[5052]: I1203 12:16:55.897984 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/keystone-db-create-59gmf" podStartSLOduration=1.8979611539999999 podStartE2EDuration="1.897961154s" podCreationTimestamp="2025-12-03 12:16:54 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:16:55.889183483 +0000 UTC m=+3406.477425247" watchObservedRunningTime="2025-12-03 12:16:55.897961154 +0000 UTC m=+3406.486202928" Dec 03 12:16:55 crc kubenswrapper[5052]: I1203 12:16:55.935459 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-73ca-account-create-update-c8r5x" Dec 03 12:16:55 crc kubenswrapper[5052]: E1203 12:16:55.986396 5052 projected.go:288] Couldn't get configMap openstack-kuttl-tests/swift-ring-files: configmap "swift-ring-files" not found Dec 03 12:16:55 crc kubenswrapper[5052]: E1203 12:16:55.987032 5052 projected.go:194] Error preparing data for projected volume etc-swift for pod openstack-kuttl-tests/swift-storage-0: configmap "swift-ring-files" not found Dec 03 12:16:55 crc kubenswrapper[5052]: E1203 12:16:55.987181 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/335ad996-ad3b-4ac8-a488-64e9453f24a4-etc-swift podName:335ad996-ad3b-4ac8-a488-64e9453f24a4 nodeName:}" failed. No retries permitted until 2025-12-03 12:17:03.987158571 +0000 UTC m=+3414.575400335 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "etc-swift" (UniqueName: "kubernetes.io/projected/335ad996-ad3b-4ac8-a488-64e9453f24a4-etc-swift") pod "swift-storage-0" (UID: "335ad996-ad3b-4ac8-a488-64e9453f24a4") : configmap "swift-ring-files" not found Dec 03 12:16:55 crc kubenswrapper[5052]: I1203 12:16:55.987286 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/335ad996-ad3b-4ac8-a488-64e9453f24a4-etc-swift\") pod \"swift-storage-0\" (UID: \"335ad996-ad3b-4ac8-a488-64e9453f24a4\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:16:56 crc kubenswrapper[5052]: I1203 12:16:56.119114 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/placement-db-create-r5z4q"] Dec 03 12:16:56 crc kubenswrapper[5052]: W1203 12:16:56.121344 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9b7bd5ab_a50e_4a66_ade7_c09c913c481f.slice/crio-adb9bd77ff9efaae3b888524341cc97c528336b172031b46b2f1b6bcc5d35c37 WatchSource:0}: Error finding container adb9bd77ff9efaae3b888524341cc97c528336b172031b46b2f1b6bcc5d35c37: Status 404 returned error can't find the container with id adb9bd77ff9efaae3b888524341cc97c528336b172031b46b2f1b6bcc5d35c37 Dec 03 12:16:56 crc kubenswrapper[5052]: I1203 12:16:56.227102 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/placement-9152-account-create-update-sgs4t"] Dec 03 12:16:56 crc kubenswrapper[5052]: I1203 12:16:56.352540 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-db-create-d6shg"] Dec 03 12:16:56 crc kubenswrapper[5052]: W1203 12:16:56.355493 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb3c91987_f9ac_46b0_a634_33c314ab4576.slice/crio-6a4a959f80f5dd51b84d092cd4128214a091bd8db45dde31c7b9ce98eab516fd WatchSource:0}: Error finding container 6a4a959f80f5dd51b84d092cd4128214a091bd8db45dde31c7b9ce98eab516fd: Status 404 returned error can't find the container with id 6a4a959f80f5dd51b84d092cd4128214a091bd8db45dde31c7b9ce98eab516fd Dec 03 12:16:56 crc kubenswrapper[5052]: I1203 12:16:56.474316 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-73ca-account-create-update-c8r5x"] Dec 03 12:16:56 crc kubenswrapper[5052]: I1203 12:16:56.887417 5052 generic.go:334] "Generic (PLEG): container finished" podID="23c58e9d-3154-4887-8a51-1601ad306abf" containerID="c96eef4a8dc590f60d0cb12973141986e22e627a28dcd22a83015316fc20efc1" exitCode=0 Dec 03 12:16:56 crc kubenswrapper[5052]: I1203 12:16:56.887488 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-28d1-account-create-update-f58gf" event={"ID":"23c58e9d-3154-4887-8a51-1601ad306abf","Type":"ContainerDied","Data":"c96eef4a8dc590f60d0cb12973141986e22e627a28dcd22a83015316fc20efc1"} Dec 03 12:16:56 crc kubenswrapper[5052]: I1203 12:16:56.890315 5052 generic.go:334] "Generic (PLEG): container finished" podID="6b4e9ce4-5733-4df7-a4f8-f3d23e906b53" containerID="4dd5e22ebf83ff299dc4da047909204239d8d079333b89c5c5d39c2221025070" exitCode=0 Dec 03 12:16:56 crc kubenswrapper[5052]: I1203 12:16:56.890373 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-73ca-account-create-update-c8r5x" event={"ID":"6b4e9ce4-5733-4df7-a4f8-f3d23e906b53","Type":"ContainerDied","Data":"4dd5e22ebf83ff299dc4da047909204239d8d079333b89c5c5d39c2221025070"} Dec 03 12:16:56 crc kubenswrapper[5052]: I1203 12:16:56.890422 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-73ca-account-create-update-c8r5x" event={"ID":"6b4e9ce4-5733-4df7-a4f8-f3d23e906b53","Type":"ContainerStarted","Data":"d04dae741526c8d558528fd3146889a6ad3834653df995c0dec0cdffc0a87bd0"} Dec 03 12:16:56 crc kubenswrapper[5052]: I1203 12:16:56.892706 5052 generic.go:334] "Generic (PLEG): container finished" podID="9b7bd5ab-a50e-4a66-ade7-c09c913c481f" containerID="4cf218ecb3b4f85168c67eef87ddb54d85c03143ceee481f6d32d5b217a05b13" exitCode=0 Dec 03 12:16:56 crc kubenswrapper[5052]: I1203 12:16:56.892780 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-db-create-r5z4q" event={"ID":"9b7bd5ab-a50e-4a66-ade7-c09c913c481f","Type":"ContainerDied","Data":"4cf218ecb3b4f85168c67eef87ddb54d85c03143ceee481f6d32d5b217a05b13"} Dec 03 12:16:56 crc kubenswrapper[5052]: I1203 12:16:56.892808 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-db-create-r5z4q" event={"ID":"9b7bd5ab-a50e-4a66-ade7-c09c913c481f","Type":"ContainerStarted","Data":"adb9bd77ff9efaae3b888524341cc97c528336b172031b46b2f1b6bcc5d35c37"} Dec 03 12:16:56 crc kubenswrapper[5052]: I1203 12:16:56.894471 5052 generic.go:334] "Generic (PLEG): container finished" podID="b3c91987-f9ac-46b0-a634-33c314ab4576" containerID="ba753a10be8fb7f00ba30d6776de7a5083ef01995b6ab3a6a792f9958569ab15" exitCode=0 Dec 03 12:16:56 crc kubenswrapper[5052]: I1203 12:16:56.894554 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-db-create-d6shg" event={"ID":"b3c91987-f9ac-46b0-a634-33c314ab4576","Type":"ContainerDied","Data":"ba753a10be8fb7f00ba30d6776de7a5083ef01995b6ab3a6a792f9958569ab15"} Dec 03 12:16:56 crc kubenswrapper[5052]: I1203 12:16:56.894580 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-db-create-d6shg" event={"ID":"b3c91987-f9ac-46b0-a634-33c314ab4576","Type":"ContainerStarted","Data":"6a4a959f80f5dd51b84d092cd4128214a091bd8db45dde31c7b9ce98eab516fd"} Dec 03 12:16:56 crc kubenswrapper[5052]: I1203 12:16:56.897100 5052 generic.go:334] "Generic (PLEG): container finished" podID="5e64bff0-44a3-472d-8b94-210a31a0e4e6" containerID="096ac7f05d80e97fa0a9f7155bad8e1122f8bb53f0932b4d4a2ff75baa41a0e8" exitCode=0 Dec 03 12:16:56 crc kubenswrapper[5052]: I1203 12:16:56.897176 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-9152-account-create-update-sgs4t" event={"ID":"5e64bff0-44a3-472d-8b94-210a31a0e4e6","Type":"ContainerDied","Data":"096ac7f05d80e97fa0a9f7155bad8e1122f8bb53f0932b4d4a2ff75baa41a0e8"} Dec 03 12:16:56 crc kubenswrapper[5052]: I1203 12:16:56.897202 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-9152-account-create-update-sgs4t" event={"ID":"5e64bff0-44a3-472d-8b94-210a31a0e4e6","Type":"ContainerStarted","Data":"9de7b4b92f46600d12d32b12b314e286f65664c6091aee2a7d5db6895cc7d32f"} Dec 03 12:16:56 crc kubenswrapper[5052]: I1203 12:16:56.898986 5052 generic.go:334] "Generic (PLEG): container finished" podID="fc3fb6e4-34b1-403a-8761-f03573c6d993" containerID="acabd11f67c578a289ff1b5572b82ec5c14a08efbfe4929ff13289c867543646" exitCode=0 Dec 03 12:16:56 crc kubenswrapper[5052]: I1203 12:16:56.899021 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-db-create-59gmf" event={"ID":"fc3fb6e4-34b1-403a-8761-f03573c6d993","Type":"ContainerDied","Data":"acabd11f67c578a289ff1b5572b82ec5c14a08efbfe4929ff13289c867543646"} Dec 03 12:16:58 crc kubenswrapper[5052]: I1203 12:16:58.293999 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-db-create-59gmf" Dec 03 12:16:58 crc kubenswrapper[5052]: I1203 12:16:58.328685 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j9kmf\" (UniqueName: \"kubernetes.io/projected/fc3fb6e4-34b1-403a-8761-f03573c6d993-kube-api-access-j9kmf\") pod \"fc3fb6e4-34b1-403a-8761-f03573c6d993\" (UID: \"fc3fb6e4-34b1-403a-8761-f03573c6d993\") " Dec 03 12:16:58 crc kubenswrapper[5052]: I1203 12:16:58.328892 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/fc3fb6e4-34b1-403a-8761-f03573c6d993-operator-scripts\") pod \"fc3fb6e4-34b1-403a-8761-f03573c6d993\" (UID: \"fc3fb6e4-34b1-403a-8761-f03573c6d993\") " Dec 03 12:16:58 crc kubenswrapper[5052]: I1203 12:16:58.330437 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fc3fb6e4-34b1-403a-8761-f03573c6d993-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "fc3fb6e4-34b1-403a-8761-f03573c6d993" (UID: "fc3fb6e4-34b1-403a-8761-f03573c6d993"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:16:58 crc kubenswrapper[5052]: I1203 12:16:58.341620 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fc3fb6e4-34b1-403a-8761-f03573c6d993-kube-api-access-j9kmf" (OuterVolumeSpecName: "kube-api-access-j9kmf") pod "fc3fb6e4-34b1-403a-8761-f03573c6d993" (UID: "fc3fb6e4-34b1-403a-8761-f03573c6d993"). InnerVolumeSpecName "kube-api-access-j9kmf". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:16:58 crc kubenswrapper[5052]: I1203 12:16:58.430823 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-j9kmf\" (UniqueName: \"kubernetes.io/projected/fc3fb6e4-34b1-403a-8761-f03573c6d993-kube-api-access-j9kmf\") on node \"crc\" DevicePath \"\"" Dec 03 12:16:58 crc kubenswrapper[5052]: I1203 12:16:58.431128 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/fc3fb6e4-34b1-403a-8761-f03573c6d993-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:16:58 crc kubenswrapper[5052]: I1203 12:16:58.570983 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-28d1-account-create-update-f58gf" Dec 03 12:16:58 crc kubenswrapper[5052]: I1203 12:16:58.582493 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-db-create-d6shg" Dec 03 12:16:58 crc kubenswrapper[5052]: I1203 12:16:58.596585 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-9152-account-create-update-sgs4t" Dec 03 12:16:58 crc kubenswrapper[5052]: I1203 12:16:58.609301 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-db-create-r5z4q" Dec 03 12:16:58 crc kubenswrapper[5052]: I1203 12:16:58.622698 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-73ca-account-create-update-c8r5x" Dec 03 12:16:58 crc kubenswrapper[5052]: I1203 12:16:58.641152 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/9b7bd5ab-a50e-4a66-ade7-c09c913c481f-operator-scripts\") pod \"9b7bd5ab-a50e-4a66-ade7-c09c913c481f\" (UID: \"9b7bd5ab-a50e-4a66-ade7-c09c913c481f\") " Dec 03 12:16:58 crc kubenswrapper[5052]: I1203 12:16:58.641229 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bf2m2\" (UniqueName: \"kubernetes.io/projected/5e64bff0-44a3-472d-8b94-210a31a0e4e6-kube-api-access-bf2m2\") pod \"5e64bff0-44a3-472d-8b94-210a31a0e4e6\" (UID: \"5e64bff0-44a3-472d-8b94-210a31a0e4e6\") " Dec 03 12:16:58 crc kubenswrapper[5052]: I1203 12:16:58.641293 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5wb5l\" (UniqueName: \"kubernetes.io/projected/23c58e9d-3154-4887-8a51-1601ad306abf-kube-api-access-5wb5l\") pod \"23c58e9d-3154-4887-8a51-1601ad306abf\" (UID: \"23c58e9d-3154-4887-8a51-1601ad306abf\") " Dec 03 12:16:58 crc kubenswrapper[5052]: I1203 12:16:58.641374 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fb6jz\" (UniqueName: \"kubernetes.io/projected/9b7bd5ab-a50e-4a66-ade7-c09c913c481f-kube-api-access-fb6jz\") pod \"9b7bd5ab-a50e-4a66-ade7-c09c913c481f\" (UID: \"9b7bd5ab-a50e-4a66-ade7-c09c913c481f\") " Dec 03 12:16:58 crc kubenswrapper[5052]: I1203 12:16:58.641415 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/23c58e9d-3154-4887-8a51-1601ad306abf-operator-scripts\") pod \"23c58e9d-3154-4887-8a51-1601ad306abf\" (UID: \"23c58e9d-3154-4887-8a51-1601ad306abf\") " Dec 03 12:16:58 crc kubenswrapper[5052]: I1203 12:16:58.641563 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ps25q\" (UniqueName: \"kubernetes.io/projected/b3c91987-f9ac-46b0-a634-33c314ab4576-kube-api-access-ps25q\") pod \"b3c91987-f9ac-46b0-a634-33c314ab4576\" (UID: \"b3c91987-f9ac-46b0-a634-33c314ab4576\") " Dec 03 12:16:58 crc kubenswrapper[5052]: I1203 12:16:58.641616 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/b3c91987-f9ac-46b0-a634-33c314ab4576-operator-scripts\") pod \"b3c91987-f9ac-46b0-a634-33c314ab4576\" (UID: \"b3c91987-f9ac-46b0-a634-33c314ab4576\") " Dec 03 12:16:58 crc kubenswrapper[5052]: I1203 12:16:58.641858 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/5e64bff0-44a3-472d-8b94-210a31a0e4e6-operator-scripts\") pod \"5e64bff0-44a3-472d-8b94-210a31a0e4e6\" (UID: \"5e64bff0-44a3-472d-8b94-210a31a0e4e6\") " Dec 03 12:16:58 crc kubenswrapper[5052]: I1203 12:16:58.643171 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5e64bff0-44a3-472d-8b94-210a31a0e4e6-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "5e64bff0-44a3-472d-8b94-210a31a0e4e6" (UID: "5e64bff0-44a3-472d-8b94-210a31a0e4e6"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:16:58 crc kubenswrapper[5052]: I1203 12:16:58.643678 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/23c58e9d-3154-4887-8a51-1601ad306abf-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "23c58e9d-3154-4887-8a51-1601ad306abf" (UID: "23c58e9d-3154-4887-8a51-1601ad306abf"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:16:58 crc kubenswrapper[5052]: I1203 12:16:58.645120 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9b7bd5ab-a50e-4a66-ade7-c09c913c481f-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "9b7bd5ab-a50e-4a66-ade7-c09c913c481f" (UID: "9b7bd5ab-a50e-4a66-ade7-c09c913c481f"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:16:58 crc kubenswrapper[5052]: I1203 12:16:58.645428 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b3c91987-f9ac-46b0-a634-33c314ab4576-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "b3c91987-f9ac-46b0-a634-33c314ab4576" (UID: "b3c91987-f9ac-46b0-a634-33c314ab4576"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:16:58 crc kubenswrapper[5052]: I1203 12:16:58.650299 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9b7bd5ab-a50e-4a66-ade7-c09c913c481f-kube-api-access-fb6jz" (OuterVolumeSpecName: "kube-api-access-fb6jz") pod "9b7bd5ab-a50e-4a66-ade7-c09c913c481f" (UID: "9b7bd5ab-a50e-4a66-ade7-c09c913c481f"). InnerVolumeSpecName "kube-api-access-fb6jz". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:16:58 crc kubenswrapper[5052]: I1203 12:16:58.651374 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b3c91987-f9ac-46b0-a634-33c314ab4576-kube-api-access-ps25q" (OuterVolumeSpecName: "kube-api-access-ps25q") pod "b3c91987-f9ac-46b0-a634-33c314ab4576" (UID: "b3c91987-f9ac-46b0-a634-33c314ab4576"). InnerVolumeSpecName "kube-api-access-ps25q". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:16:58 crc kubenswrapper[5052]: I1203 12:16:58.663681 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5e64bff0-44a3-472d-8b94-210a31a0e4e6-kube-api-access-bf2m2" (OuterVolumeSpecName: "kube-api-access-bf2m2") pod "5e64bff0-44a3-472d-8b94-210a31a0e4e6" (UID: "5e64bff0-44a3-472d-8b94-210a31a0e4e6"). InnerVolumeSpecName "kube-api-access-bf2m2". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:16:58 crc kubenswrapper[5052]: I1203 12:16:58.674768 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/23c58e9d-3154-4887-8a51-1601ad306abf-kube-api-access-5wb5l" (OuterVolumeSpecName: "kube-api-access-5wb5l") pod "23c58e9d-3154-4887-8a51-1601ad306abf" (UID: "23c58e9d-3154-4887-8a51-1601ad306abf"). InnerVolumeSpecName "kube-api-access-5wb5l". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:16:58 crc kubenswrapper[5052]: I1203 12:16:58.744327 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h289z\" (UniqueName: \"kubernetes.io/projected/6b4e9ce4-5733-4df7-a4f8-f3d23e906b53-kube-api-access-h289z\") pod \"6b4e9ce4-5733-4df7-a4f8-f3d23e906b53\" (UID: \"6b4e9ce4-5733-4df7-a4f8-f3d23e906b53\") " Dec 03 12:16:58 crc kubenswrapper[5052]: I1203 12:16:58.744490 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/6b4e9ce4-5733-4df7-a4f8-f3d23e906b53-operator-scripts\") pod \"6b4e9ce4-5733-4df7-a4f8-f3d23e906b53\" (UID: \"6b4e9ce4-5733-4df7-a4f8-f3d23e906b53\") " Dec 03 12:16:58 crc kubenswrapper[5052]: I1203 12:16:58.745033 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/5e64bff0-44a3-472d-8b94-210a31a0e4e6-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:16:58 crc kubenswrapper[5052]: I1203 12:16:58.745057 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/9b7bd5ab-a50e-4a66-ade7-c09c913c481f-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:16:58 crc kubenswrapper[5052]: I1203 12:16:58.745068 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-bf2m2\" (UniqueName: \"kubernetes.io/projected/5e64bff0-44a3-472d-8b94-210a31a0e4e6-kube-api-access-bf2m2\") on node \"crc\" DevicePath \"\"" Dec 03 12:16:58 crc kubenswrapper[5052]: I1203 12:16:58.745080 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-5wb5l\" (UniqueName: \"kubernetes.io/projected/23c58e9d-3154-4887-8a51-1601ad306abf-kube-api-access-5wb5l\") on node \"crc\" DevicePath \"\"" Dec 03 12:16:58 crc kubenswrapper[5052]: I1203 12:16:58.745090 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-fb6jz\" (UniqueName: \"kubernetes.io/projected/9b7bd5ab-a50e-4a66-ade7-c09c913c481f-kube-api-access-fb6jz\") on node \"crc\" DevicePath \"\"" Dec 03 12:16:58 crc kubenswrapper[5052]: I1203 12:16:58.745101 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/23c58e9d-3154-4887-8a51-1601ad306abf-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:16:58 crc kubenswrapper[5052]: I1203 12:16:58.745097 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6b4e9ce4-5733-4df7-a4f8-f3d23e906b53-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "6b4e9ce4-5733-4df7-a4f8-f3d23e906b53" (UID: "6b4e9ce4-5733-4df7-a4f8-f3d23e906b53"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:16:58 crc kubenswrapper[5052]: I1203 12:16:58.745111 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-ps25q\" (UniqueName: \"kubernetes.io/projected/b3c91987-f9ac-46b0-a634-33c314ab4576-kube-api-access-ps25q\") on node \"crc\" DevicePath \"\"" Dec 03 12:16:58 crc kubenswrapper[5052]: I1203 12:16:58.745146 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/b3c91987-f9ac-46b0-a634-33c314ab4576-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:16:58 crc kubenswrapper[5052]: I1203 12:16:58.747597 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6b4e9ce4-5733-4df7-a4f8-f3d23e906b53-kube-api-access-h289z" (OuterVolumeSpecName: "kube-api-access-h289z") pod "6b4e9ce4-5733-4df7-a4f8-f3d23e906b53" (UID: "6b4e9ce4-5733-4df7-a4f8-f3d23e906b53"). InnerVolumeSpecName "kube-api-access-h289z". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:16:58 crc kubenswrapper[5052]: I1203 12:16:58.847126 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-h289z\" (UniqueName: \"kubernetes.io/projected/6b4e9ce4-5733-4df7-a4f8-f3d23e906b53-kube-api-access-h289z\") on node \"crc\" DevicePath \"\"" Dec 03 12:16:58 crc kubenswrapper[5052]: I1203 12:16:58.847169 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/6b4e9ce4-5733-4df7-a4f8-f3d23e906b53-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:16:58 crc kubenswrapper[5052]: I1203 12:16:58.933086 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-db-create-59gmf" event={"ID":"fc3fb6e4-34b1-403a-8761-f03573c6d993","Type":"ContainerDied","Data":"0c5d43144974a8ab2a9483ef123db712d5adefd5b51c7a5d0024c4ae175f8f8d"} Dec 03 12:16:58 crc kubenswrapper[5052]: I1203 12:16:58.933413 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0c5d43144974a8ab2a9483ef123db712d5adefd5b51c7a5d0024c4ae175f8f8d" Dec 03 12:16:58 crc kubenswrapper[5052]: I1203 12:16:58.933185 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-db-create-59gmf" Dec 03 12:16:58 crc kubenswrapper[5052]: I1203 12:16:58.935534 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-28d1-account-create-update-f58gf" event={"ID":"23c58e9d-3154-4887-8a51-1601ad306abf","Type":"ContainerDied","Data":"f5c3aa9aeea711c350aad6d9bdcc6e38bad899dc8ee79cb94fa9ba2369ae35df"} Dec 03 12:16:58 crc kubenswrapper[5052]: I1203 12:16:58.935572 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-28d1-account-create-update-f58gf" Dec 03 12:16:58 crc kubenswrapper[5052]: I1203 12:16:58.935580 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f5c3aa9aeea711c350aad6d9bdcc6e38bad899dc8ee79cb94fa9ba2369ae35df" Dec 03 12:16:58 crc kubenswrapper[5052]: I1203 12:16:58.938154 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-73ca-account-create-update-c8r5x" Dec 03 12:16:58 crc kubenswrapper[5052]: I1203 12:16:58.938145 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-73ca-account-create-update-c8r5x" event={"ID":"6b4e9ce4-5733-4df7-a4f8-f3d23e906b53","Type":"ContainerDied","Data":"d04dae741526c8d558528fd3146889a6ad3834653df995c0dec0cdffc0a87bd0"} Dec 03 12:16:58 crc kubenswrapper[5052]: I1203 12:16:58.938316 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d04dae741526c8d558528fd3146889a6ad3834653df995c0dec0cdffc0a87bd0" Dec 03 12:16:58 crc kubenswrapper[5052]: I1203 12:16:58.939824 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-db-create-r5z4q" Dec 03 12:16:58 crc kubenswrapper[5052]: I1203 12:16:58.939825 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-db-create-r5z4q" event={"ID":"9b7bd5ab-a50e-4a66-ade7-c09c913c481f","Type":"ContainerDied","Data":"adb9bd77ff9efaae3b888524341cc97c528336b172031b46b2f1b6bcc5d35c37"} Dec 03 12:16:58 crc kubenswrapper[5052]: I1203 12:16:58.939982 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="adb9bd77ff9efaae3b888524341cc97c528336b172031b46b2f1b6bcc5d35c37" Dec 03 12:16:58 crc kubenswrapper[5052]: I1203 12:16:58.941551 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-db-create-d6shg" event={"ID":"b3c91987-f9ac-46b0-a634-33c314ab4576","Type":"ContainerDied","Data":"6a4a959f80f5dd51b84d092cd4128214a091bd8db45dde31c7b9ce98eab516fd"} Dec 03 12:16:58 crc kubenswrapper[5052]: I1203 12:16:58.941595 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6a4a959f80f5dd51b84d092cd4128214a091bd8db45dde31c7b9ce98eab516fd" Dec 03 12:16:58 crc kubenswrapper[5052]: I1203 12:16:58.941621 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-db-create-d6shg" Dec 03 12:16:58 crc kubenswrapper[5052]: I1203 12:16:58.943491 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-9152-account-create-update-sgs4t" Dec 03 12:16:58 crc kubenswrapper[5052]: I1203 12:16:58.943495 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-9152-account-create-update-sgs4t" event={"ID":"5e64bff0-44a3-472d-8b94-210a31a0e4e6","Type":"ContainerDied","Data":"9de7b4b92f46600d12d32b12b314e286f65664c6091aee2a7d5db6895cc7d32f"} Dec 03 12:16:58 crc kubenswrapper[5052]: I1203 12:16:58.943559 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9de7b4b92f46600d12d32b12b314e286f65664c6091aee2a7d5db6895cc7d32f" Dec 03 12:16:58 crc kubenswrapper[5052]: I1203 12:16:58.945180 5052 generic.go:334] "Generic (PLEG): container finished" podID="33837271-e865-4153-afab-6de6235c8ea8" containerID="231d5f3ea609c0fd0b8f6c9745cde813952d26ceaed8f036d11f3d3a0034c03a" exitCode=0 Dec 03 12:16:58 crc kubenswrapper[5052]: I1203 12:16:58.945212 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-ring-rebalance-shz9b" event={"ID":"33837271-e865-4153-afab-6de6235c8ea8","Type":"ContainerDied","Data":"231d5f3ea609c0fd0b8f6c9745cde813952d26ceaed8f036d11f3d3a0034c03a"} Dec 03 12:16:59 crc kubenswrapper[5052]: I1203 12:16:59.379709 5052 patch_prober.go:28] interesting pod/machine-config-daemon-74bj6 container/machine-config-daemon namespace/openshift-machine-config-operator: Liveness probe status=failure output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" start-of-body= Dec 03 12:16:59 crc kubenswrapper[5052]: I1203 12:16:59.379762 5052 prober.go:107] "Probe failed" probeType="Liveness" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" containerName="machine-config-daemon" probeResult="failure" output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" Dec 03 12:17:00 crc kubenswrapper[5052]: I1203 12:17:00.430174 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-ring-rebalance-shz9b" Dec 03 12:17:00 crc kubenswrapper[5052]: I1203 12:17:00.477224 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ring-data-devices\" (UniqueName: \"kubernetes.io/configmap/33837271-e865-4153-afab-6de6235c8ea8-ring-data-devices\") pod \"33837271-e865-4153-afab-6de6235c8ea8\" (UID: \"33837271-e865-4153-afab-6de6235c8ea8\") " Dec 03 12:17:00 crc kubenswrapper[5052]: I1203 12:17:00.477377 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"dispersionconf\" (UniqueName: \"kubernetes.io/secret/33837271-e865-4153-afab-6de6235c8ea8-dispersionconf\") pod \"33837271-e865-4153-afab-6de6235c8ea8\" (UID: \"33837271-e865-4153-afab-6de6235c8ea8\") " Dec 03 12:17:00 crc kubenswrapper[5052]: I1203 12:17:00.477481 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/33837271-e865-4153-afab-6de6235c8ea8-combined-ca-bundle\") pod \"33837271-e865-4153-afab-6de6235c8ea8\" (UID: \"33837271-e865-4153-afab-6de6235c8ea8\") " Dec 03 12:17:00 crc kubenswrapper[5052]: I1203 12:17:00.477838 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nq2db\" (UniqueName: \"kubernetes.io/projected/33837271-e865-4153-afab-6de6235c8ea8-kube-api-access-nq2db\") pod \"33837271-e865-4153-afab-6de6235c8ea8\" (UID: \"33837271-e865-4153-afab-6de6235c8ea8\") " Dec 03 12:17:00 crc kubenswrapper[5052]: I1203 12:17:00.477873 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"swiftconf\" (UniqueName: \"kubernetes.io/secret/33837271-e865-4153-afab-6de6235c8ea8-swiftconf\") pod \"33837271-e865-4153-afab-6de6235c8ea8\" (UID: \"33837271-e865-4153-afab-6de6235c8ea8\") " Dec 03 12:17:00 crc kubenswrapper[5052]: I1203 12:17:00.477906 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/empty-dir/33837271-e865-4153-afab-6de6235c8ea8-etc-swift\") pod \"33837271-e865-4153-afab-6de6235c8ea8\" (UID: \"33837271-e865-4153-afab-6de6235c8ea8\") " Dec 03 12:17:00 crc kubenswrapper[5052]: I1203 12:17:00.477934 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/33837271-e865-4153-afab-6de6235c8ea8-scripts\") pod \"33837271-e865-4153-afab-6de6235c8ea8\" (UID: \"33837271-e865-4153-afab-6de6235c8ea8\") " Dec 03 12:17:00 crc kubenswrapper[5052]: I1203 12:17:00.478111 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/33837271-e865-4153-afab-6de6235c8ea8-ring-data-devices" (OuterVolumeSpecName: "ring-data-devices") pod "33837271-e865-4153-afab-6de6235c8ea8" (UID: "33837271-e865-4153-afab-6de6235c8ea8"). InnerVolumeSpecName "ring-data-devices". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:17:00 crc kubenswrapper[5052]: I1203 12:17:00.478452 5052 reconciler_common.go:293] "Volume detached for volume \"ring-data-devices\" (UniqueName: \"kubernetes.io/configmap/33837271-e865-4153-afab-6de6235c8ea8-ring-data-devices\") on node \"crc\" DevicePath \"\"" Dec 03 12:17:00 crc kubenswrapper[5052]: I1203 12:17:00.478993 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33837271-e865-4153-afab-6de6235c8ea8-etc-swift" (OuterVolumeSpecName: "etc-swift") pod "33837271-e865-4153-afab-6de6235c8ea8" (UID: "33837271-e865-4153-afab-6de6235c8ea8"). InnerVolumeSpecName "etc-swift". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:17:00 crc kubenswrapper[5052]: I1203 12:17:00.482698 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/33837271-e865-4153-afab-6de6235c8ea8-kube-api-access-nq2db" (OuterVolumeSpecName: "kube-api-access-nq2db") pod "33837271-e865-4153-afab-6de6235c8ea8" (UID: "33837271-e865-4153-afab-6de6235c8ea8"). InnerVolumeSpecName "kube-api-access-nq2db". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:17:00 crc kubenswrapper[5052]: I1203 12:17:00.485843 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/33837271-e865-4153-afab-6de6235c8ea8-dispersionconf" (OuterVolumeSpecName: "dispersionconf") pod "33837271-e865-4153-afab-6de6235c8ea8" (UID: "33837271-e865-4153-afab-6de6235c8ea8"). InnerVolumeSpecName "dispersionconf". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:17:00 crc kubenswrapper[5052]: I1203 12:17:00.501777 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/33837271-e865-4153-afab-6de6235c8ea8-scripts" (OuterVolumeSpecName: "scripts") pod "33837271-e865-4153-afab-6de6235c8ea8" (UID: "33837271-e865-4153-afab-6de6235c8ea8"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:17:00 crc kubenswrapper[5052]: I1203 12:17:00.502388 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/33837271-e865-4153-afab-6de6235c8ea8-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "33837271-e865-4153-afab-6de6235c8ea8" (UID: "33837271-e865-4153-afab-6de6235c8ea8"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:17:00 crc kubenswrapper[5052]: I1203 12:17:00.502427 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/33837271-e865-4153-afab-6de6235c8ea8-swiftconf" (OuterVolumeSpecName: "swiftconf") pod "33837271-e865-4153-afab-6de6235c8ea8" (UID: "33837271-e865-4153-afab-6de6235c8ea8"). InnerVolumeSpecName "swiftconf". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:17:00 crc kubenswrapper[5052]: I1203 12:17:00.581788 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-nq2db\" (UniqueName: \"kubernetes.io/projected/33837271-e865-4153-afab-6de6235c8ea8-kube-api-access-nq2db\") on node \"crc\" DevicePath \"\"" Dec 03 12:17:00 crc kubenswrapper[5052]: I1203 12:17:00.582072 5052 reconciler_common.go:293] "Volume detached for volume \"swiftconf\" (UniqueName: \"kubernetes.io/secret/33837271-e865-4153-afab-6de6235c8ea8-swiftconf\") on node \"crc\" DevicePath \"\"" Dec 03 12:17:00 crc kubenswrapper[5052]: I1203 12:17:00.582159 5052 reconciler_common.go:293] "Volume detached for volume \"etc-swift\" (UniqueName: \"kubernetes.io/empty-dir/33837271-e865-4153-afab-6de6235c8ea8-etc-swift\") on node \"crc\" DevicePath \"\"" Dec 03 12:17:00 crc kubenswrapper[5052]: I1203 12:17:00.582249 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/33837271-e865-4153-afab-6de6235c8ea8-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:17:00 crc kubenswrapper[5052]: I1203 12:17:00.583553 5052 reconciler_common.go:293] "Volume detached for volume \"dispersionconf\" (UniqueName: \"kubernetes.io/secret/33837271-e865-4153-afab-6de6235c8ea8-dispersionconf\") on node \"crc\" DevicePath \"\"" Dec 03 12:17:00 crc kubenswrapper[5052]: I1203 12:17:00.583641 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/33837271-e865-4153-afab-6de6235c8ea8-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:17:00 crc kubenswrapper[5052]: I1203 12:17:00.780902 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/glance-db-sync-qstzv"] Dec 03 12:17:00 crc kubenswrapper[5052]: E1203 12:17:00.781261 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="fc3fb6e4-34b1-403a-8761-f03573c6d993" containerName="mariadb-database-create" Dec 03 12:17:00 crc kubenswrapper[5052]: I1203 12:17:00.781279 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="fc3fb6e4-34b1-403a-8761-f03573c6d993" containerName="mariadb-database-create" Dec 03 12:17:00 crc kubenswrapper[5052]: E1203 12:17:00.781292 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="23c58e9d-3154-4887-8a51-1601ad306abf" containerName="mariadb-account-create-update" Dec 03 12:17:00 crc kubenswrapper[5052]: I1203 12:17:00.781299 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="23c58e9d-3154-4887-8a51-1601ad306abf" containerName="mariadb-account-create-update" Dec 03 12:17:00 crc kubenswrapper[5052]: E1203 12:17:00.781313 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="33837271-e865-4153-afab-6de6235c8ea8" containerName="swift-ring-rebalance" Dec 03 12:17:00 crc kubenswrapper[5052]: I1203 12:17:00.781319 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="33837271-e865-4153-afab-6de6235c8ea8" containerName="swift-ring-rebalance" Dec 03 12:17:00 crc kubenswrapper[5052]: E1203 12:17:00.781337 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="9b7bd5ab-a50e-4a66-ade7-c09c913c481f" containerName="mariadb-database-create" Dec 03 12:17:00 crc kubenswrapper[5052]: I1203 12:17:00.781343 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b7bd5ab-a50e-4a66-ade7-c09c913c481f" containerName="mariadb-database-create" Dec 03 12:17:00 crc kubenswrapper[5052]: E1203 12:17:00.781358 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="b3c91987-f9ac-46b0-a634-33c314ab4576" containerName="mariadb-database-create" Dec 03 12:17:00 crc kubenswrapper[5052]: I1203 12:17:00.781363 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="b3c91987-f9ac-46b0-a634-33c314ab4576" containerName="mariadb-database-create" Dec 03 12:17:00 crc kubenswrapper[5052]: E1203 12:17:00.781376 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="5e64bff0-44a3-472d-8b94-210a31a0e4e6" containerName="mariadb-account-create-update" Dec 03 12:17:00 crc kubenswrapper[5052]: I1203 12:17:00.781381 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="5e64bff0-44a3-472d-8b94-210a31a0e4e6" containerName="mariadb-account-create-update" Dec 03 12:17:00 crc kubenswrapper[5052]: E1203 12:17:00.781394 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="6b4e9ce4-5733-4df7-a4f8-f3d23e906b53" containerName="mariadb-account-create-update" Dec 03 12:17:00 crc kubenswrapper[5052]: I1203 12:17:00.781401 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="6b4e9ce4-5733-4df7-a4f8-f3d23e906b53" containerName="mariadb-account-create-update" Dec 03 12:17:00 crc kubenswrapper[5052]: I1203 12:17:00.781557 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="fc3fb6e4-34b1-403a-8761-f03573c6d993" containerName="mariadb-database-create" Dec 03 12:17:00 crc kubenswrapper[5052]: I1203 12:17:00.781574 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="6b4e9ce4-5733-4df7-a4f8-f3d23e906b53" containerName="mariadb-account-create-update" Dec 03 12:17:00 crc kubenswrapper[5052]: I1203 12:17:00.781586 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="b3c91987-f9ac-46b0-a634-33c314ab4576" containerName="mariadb-database-create" Dec 03 12:17:00 crc kubenswrapper[5052]: I1203 12:17:00.781594 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="5e64bff0-44a3-472d-8b94-210a31a0e4e6" containerName="mariadb-account-create-update" Dec 03 12:17:00 crc kubenswrapper[5052]: I1203 12:17:00.781613 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="9b7bd5ab-a50e-4a66-ade7-c09c913c481f" containerName="mariadb-database-create" Dec 03 12:17:00 crc kubenswrapper[5052]: I1203 12:17:00.781624 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="33837271-e865-4153-afab-6de6235c8ea8" containerName="swift-ring-rebalance" Dec 03 12:17:00 crc kubenswrapper[5052]: I1203 12:17:00.781635 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="23c58e9d-3154-4887-8a51-1601ad306abf" containerName="mariadb-account-create-update" Dec 03 12:17:00 crc kubenswrapper[5052]: I1203 12:17:00.782160 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-db-sync-qstzv" Dec 03 12:17:00 crc kubenswrapper[5052]: I1203 12:17:00.784633 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-config-data" Dec 03 12:17:00 crc kubenswrapper[5052]: I1203 12:17:00.788060 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-glance-dockercfg-rsctn" Dec 03 12:17:00 crc kubenswrapper[5052]: I1203 12:17:00.790487 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-db-sync-qstzv"] Dec 03 12:17:00 crc kubenswrapper[5052]: I1203 12:17:00.888419 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hbgd6\" (UniqueName: \"kubernetes.io/projected/33b07d8d-aae0-436c-831b-aa13ceff481b-kube-api-access-hbgd6\") pod \"glance-db-sync-qstzv\" (UID: \"33b07d8d-aae0-436c-831b-aa13ceff481b\") " pod="openstack-kuttl-tests/glance-db-sync-qstzv" Dec 03 12:17:00 crc kubenswrapper[5052]: I1203 12:17:00.888462 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/33b07d8d-aae0-436c-831b-aa13ceff481b-db-sync-config-data\") pod \"glance-db-sync-qstzv\" (UID: \"33b07d8d-aae0-436c-831b-aa13ceff481b\") " pod="openstack-kuttl-tests/glance-db-sync-qstzv" Dec 03 12:17:00 crc kubenswrapper[5052]: I1203 12:17:00.888851 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/33b07d8d-aae0-436c-831b-aa13ceff481b-combined-ca-bundle\") pod \"glance-db-sync-qstzv\" (UID: \"33b07d8d-aae0-436c-831b-aa13ceff481b\") " pod="openstack-kuttl-tests/glance-db-sync-qstzv" Dec 03 12:17:00 crc kubenswrapper[5052]: I1203 12:17:00.888908 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/33b07d8d-aae0-436c-831b-aa13ceff481b-config-data\") pod \"glance-db-sync-qstzv\" (UID: \"33b07d8d-aae0-436c-831b-aa13ceff481b\") " pod="openstack-kuttl-tests/glance-db-sync-qstzv" Dec 03 12:17:00 crc kubenswrapper[5052]: I1203 12:17:00.968135 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-ring-rebalance-shz9b" event={"ID":"33837271-e865-4153-afab-6de6235c8ea8","Type":"ContainerDied","Data":"e501e08aa953468d0f7140a9c4c2f318ef52b35f5d158db84ed71d90b37e777f"} Dec 03 12:17:00 crc kubenswrapper[5052]: I1203 12:17:00.968437 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e501e08aa953468d0f7140a9c4c2f318ef52b35f5d158db84ed71d90b37e777f" Dec 03 12:17:00 crc kubenswrapper[5052]: I1203 12:17:00.968406 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-ring-rebalance-shz9b" Dec 03 12:17:00 crc kubenswrapper[5052]: I1203 12:17:00.990266 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-hbgd6\" (UniqueName: \"kubernetes.io/projected/33b07d8d-aae0-436c-831b-aa13ceff481b-kube-api-access-hbgd6\") pod \"glance-db-sync-qstzv\" (UID: \"33b07d8d-aae0-436c-831b-aa13ceff481b\") " pod="openstack-kuttl-tests/glance-db-sync-qstzv" Dec 03 12:17:00 crc kubenswrapper[5052]: I1203 12:17:00.990317 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/33b07d8d-aae0-436c-831b-aa13ceff481b-db-sync-config-data\") pod \"glance-db-sync-qstzv\" (UID: \"33b07d8d-aae0-436c-831b-aa13ceff481b\") " pod="openstack-kuttl-tests/glance-db-sync-qstzv" Dec 03 12:17:00 crc kubenswrapper[5052]: I1203 12:17:00.990436 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/33b07d8d-aae0-436c-831b-aa13ceff481b-combined-ca-bundle\") pod \"glance-db-sync-qstzv\" (UID: \"33b07d8d-aae0-436c-831b-aa13ceff481b\") " pod="openstack-kuttl-tests/glance-db-sync-qstzv" Dec 03 12:17:00 crc kubenswrapper[5052]: I1203 12:17:00.990464 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/33b07d8d-aae0-436c-831b-aa13ceff481b-config-data\") pod \"glance-db-sync-qstzv\" (UID: \"33b07d8d-aae0-436c-831b-aa13ceff481b\") " pod="openstack-kuttl-tests/glance-db-sync-qstzv" Dec 03 12:17:00 crc kubenswrapper[5052]: I1203 12:17:00.996152 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/33b07d8d-aae0-436c-831b-aa13ceff481b-combined-ca-bundle\") pod \"glance-db-sync-qstzv\" (UID: \"33b07d8d-aae0-436c-831b-aa13ceff481b\") " pod="openstack-kuttl-tests/glance-db-sync-qstzv" Dec 03 12:17:00 crc kubenswrapper[5052]: I1203 12:17:00.996249 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/33b07d8d-aae0-436c-831b-aa13ceff481b-db-sync-config-data\") pod \"glance-db-sync-qstzv\" (UID: \"33b07d8d-aae0-436c-831b-aa13ceff481b\") " pod="openstack-kuttl-tests/glance-db-sync-qstzv" Dec 03 12:17:00 crc kubenswrapper[5052]: I1203 12:17:00.997415 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/33b07d8d-aae0-436c-831b-aa13ceff481b-config-data\") pod \"glance-db-sync-qstzv\" (UID: \"33b07d8d-aae0-436c-831b-aa13ceff481b\") " pod="openstack-kuttl-tests/glance-db-sync-qstzv" Dec 03 12:17:01 crc kubenswrapper[5052]: I1203 12:17:01.010312 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-hbgd6\" (UniqueName: \"kubernetes.io/projected/33b07d8d-aae0-436c-831b-aa13ceff481b-kube-api-access-hbgd6\") pod \"glance-db-sync-qstzv\" (UID: \"33b07d8d-aae0-436c-831b-aa13ceff481b\") " pod="openstack-kuttl-tests/glance-db-sync-qstzv" Dec 03 12:17:01 crc kubenswrapper[5052]: I1203 12:17:01.098835 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-db-sync-qstzv" Dec 03 12:17:01 crc kubenswrapper[5052]: I1203 12:17:01.436135 5052 scope.go:117] "RemoveContainer" containerID="50e44a430f57ac6bd96da39c820b96952a8f8446aba9b14209511df5a891cac2" Dec 03 12:17:01 crc kubenswrapper[5052]: I1203 12:17:01.460870 5052 scope.go:117] "RemoveContainer" containerID="90d318c8258cb1c966ff66e10dc4b544bb9db909d657651ffa26b1e4b73924f6" Dec 03 12:17:01 crc kubenswrapper[5052]: I1203 12:17:01.494165 5052 scope.go:117] "RemoveContainer" containerID="3a6988a519408f10e2c7e85d6879bc627c5cf388c6b01b100a31df75d3b69f87" Dec 03 12:17:01 crc kubenswrapper[5052]: I1203 12:17:01.527867 5052 scope.go:117] "RemoveContainer" containerID="2daa465a549ba89fd54fb30d8b03b5b9fae793709253f37943466cd74fa59032" Dec 03 12:17:01 crc kubenswrapper[5052]: I1203 12:17:01.564877 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-db-sync-qstzv"] Dec 03 12:17:01 crc kubenswrapper[5052]: W1203 12:17:01.570251 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod33b07d8d_aae0_436c_831b_aa13ceff481b.slice/crio-9dc821a1a790aeafdf0f867616a2b996745d8ed42e3ef4577b367547c87e356d WatchSource:0}: Error finding container 9dc821a1a790aeafdf0f867616a2b996745d8ed42e3ef4577b367547c87e356d: Status 404 returned error can't find the container with id 9dc821a1a790aeafdf0f867616a2b996745d8ed42e3ef4577b367547c87e356d Dec 03 12:17:01 crc kubenswrapper[5052]: I1203 12:17:01.638724 5052 scope.go:117] "RemoveContainer" containerID="44f6c3de6269728aee0300a683f398adb92ca35b09bbc01daff9bd5bfff2bcd3" Dec 03 12:17:01 crc kubenswrapper[5052]: I1203 12:17:01.661830 5052 scope.go:117] "RemoveContainer" containerID="fd4788c2774ac207384c1a4ed34e2dd899d31e1787be5216e43104b4382b3dca" Dec 03 12:17:01 crc kubenswrapper[5052]: I1203 12:17:01.698573 5052 scope.go:117] "RemoveContainer" containerID="7bf695872e862a8fb97b7b705d1ea5a2d8ed02b1b073a00b69dc98aeab010dbd" Dec 03 12:17:01 crc kubenswrapper[5052]: I1203 12:17:01.721869 5052 scope.go:117] "RemoveContainer" containerID="dc5c838958d7fbbc47d97aabd5b021dc0188e172b8b2ed5ab168cef70e171e07" Dec 03 12:17:01 crc kubenswrapper[5052]: I1203 12:17:01.741341 5052 scope.go:117] "RemoveContainer" containerID="5a2cb3d5fc8ac42d6634d0be3922a93e66ea5b5349f071f28e904e46567ec72d" Dec 03 12:17:01 crc kubenswrapper[5052]: I1203 12:17:01.760385 5052 scope.go:117] "RemoveContainer" containerID="0ce8d3aa98265aafb7a39d170fa6017ca01e2e308bce70bae94d80594915a43f" Dec 03 12:17:01 crc kubenswrapper[5052]: I1203 12:17:01.795637 5052 scope.go:117] "RemoveContainer" containerID="087bf897c860ad2b3bd2cd4f2aa80759a612a4adda5ec15314f2e7579f48e9e2" Dec 03 12:17:01 crc kubenswrapper[5052]: I1203 12:17:01.819197 5052 scope.go:117] "RemoveContainer" containerID="5546110e826607fda4ef25f17d2fb70bb3bbf733803298d3030e388c70a537d9" Dec 03 12:17:01 crc kubenswrapper[5052]: I1203 12:17:01.865806 5052 scope.go:117] "RemoveContainer" containerID="5b4ba199f7649c85533a15076c1e1e29d06aba5da6d519402cfa536c066e590b" Dec 03 12:17:01 crc kubenswrapper[5052]: I1203 12:17:01.886367 5052 scope.go:117] "RemoveContainer" containerID="2148ec305b2c6455e0a7a297728c85e7be875c005f08c3e77fff33cf4d3bde2d" Dec 03 12:17:01 crc kubenswrapper[5052]: I1203 12:17:01.903303 5052 scope.go:117] "RemoveContainer" containerID="e9482eb27c41eb5dac4cd251777a2b7ad32fe9cc1d2f868c13cb8db63c2720b0" Dec 03 12:17:01 crc kubenswrapper[5052]: I1203 12:17:01.926032 5052 scope.go:117] "RemoveContainer" containerID="de161b1412d41d5ec322763c71b1221e12135067ced1a67a7f68542d387c4abb" Dec 03 12:17:01 crc kubenswrapper[5052]: I1203 12:17:01.950932 5052 scope.go:117] "RemoveContainer" containerID="c9905267b6558bc3332728a3891c7daa7a01ebb18411fc152bcfc514ff6b7b6e" Dec 03 12:17:01 crc kubenswrapper[5052]: I1203 12:17:01.979315 5052 scope.go:117] "RemoveContainer" containerID="328b5a10a350b22ad4abac8cb75074633d0d2442e8c1278b06e290b169197ede" Dec 03 12:17:01 crc kubenswrapper[5052]: I1203 12:17:01.984615 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-db-sync-qstzv" event={"ID":"33b07d8d-aae0-436c-831b-aa13ceff481b","Type":"ContainerStarted","Data":"9dc821a1a790aeafdf0f867616a2b996745d8ed42e3ef4577b367547c87e356d"} Dec 03 12:17:02 crc kubenswrapper[5052]: I1203 12:17:02.013226 5052 scope.go:117] "RemoveContainer" containerID="a3147e7344c54bf91a2b2c3ace19353f08a7f75ada18932e0399780c89d82766" Dec 03 12:17:03 crc kubenswrapper[5052]: I1203 12:17:03.017960 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-db-sync-qstzv" event={"ID":"33b07d8d-aae0-436c-831b-aa13ceff481b","Type":"ContainerStarted","Data":"9d59b98890d62735c1c3c35a6531d9c3479675d06cd1783c7c1c65a8112e60ca"} Dec 03 12:17:03 crc kubenswrapper[5052]: I1203 12:17:03.038316 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/glance-db-sync-qstzv" podStartSLOduration=3.038297382 podStartE2EDuration="3.038297382s" podCreationTimestamp="2025-12-03 12:17:00 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:17:03.035927224 +0000 UTC m=+3413.624168998" watchObservedRunningTime="2025-12-03 12:17:03.038297382 +0000 UTC m=+3413.626539146" Dec 03 12:17:04 crc kubenswrapper[5052]: I1203 12:17:04.039578 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/335ad996-ad3b-4ac8-a488-64e9453f24a4-etc-swift\") pod \"swift-storage-0\" (UID: \"335ad996-ad3b-4ac8-a488-64e9453f24a4\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:17:04 crc kubenswrapper[5052]: I1203 12:17:04.051404 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/335ad996-ad3b-4ac8-a488-64e9453f24a4-etc-swift\") pod \"swift-storage-0\" (UID: \"335ad996-ad3b-4ac8-a488-64e9453f24a4\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:17:04 crc kubenswrapper[5052]: I1203 12:17:04.333641 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:17:04 crc kubenswrapper[5052]: I1203 12:17:04.648462 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:17:04 crc kubenswrapper[5052]: I1203 12:17:04.778539 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/swift-storage-0"] Dec 03 12:17:05 crc kubenswrapper[5052]: I1203 12:17:05.034934 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"335ad996-ad3b-4ac8-a488-64e9453f24a4","Type":"ContainerStarted","Data":"dc35354b0a80d09200fdddb37c6af8fefe0bfea68ba76cd38f4455c24df918d1"} Dec 03 12:17:06 crc kubenswrapper[5052]: I1203 12:17:06.052896 5052 generic.go:334] "Generic (PLEG): container finished" podID="683b7d5f-a88a-4dc0-ae83-af3c6444c7ff" containerID="67a78b1201a5ad7507c89c9217d67878a2b07cc939213e22cfeb1612eb0f4575" exitCode=0 Dec 03 12:17:06 crc kubenswrapper[5052]: I1203 12:17:06.052949 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-server-0" event={"ID":"683b7d5f-a88a-4dc0-ae83-af3c6444c7ff","Type":"ContainerDied","Data":"67a78b1201a5ad7507c89c9217d67878a2b07cc939213e22cfeb1612eb0f4575"} Dec 03 12:17:06 crc kubenswrapper[5052]: I1203 12:17:06.056413 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"335ad996-ad3b-4ac8-a488-64e9453f24a4","Type":"ContainerStarted","Data":"006d1e418590929322462448d037f47e0e4031553a716f0f323b0ceb29ae6857"} Dec 03 12:17:06 crc kubenswrapper[5052]: I1203 12:17:06.060904 5052 generic.go:334] "Generic (PLEG): container finished" podID="1ec6b1f7-5abb-4063-80fa-b5ec907dcb91" containerID="65247d77d1d600b3063d35a5f6d98e45d0e5800997643936febe03cc72d2cda6" exitCode=0 Dec 03 12:17:06 crc kubenswrapper[5052]: I1203 12:17:06.061867 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" event={"ID":"1ec6b1f7-5abb-4063-80fa-b5ec907dcb91","Type":"ContainerDied","Data":"65247d77d1d600b3063d35a5f6d98e45d0e5800997643936febe03cc72d2cda6"} Dec 03 12:17:07 crc kubenswrapper[5052]: I1203 12:17:07.071612 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-server-0" event={"ID":"683b7d5f-a88a-4dc0-ae83-af3c6444c7ff","Type":"ContainerStarted","Data":"82c91d3bad330d4f21f39bfba31fbc4e7bfd4508da6e46af97efbf075afd08f3"} Dec 03 12:17:07 crc kubenswrapper[5052]: I1203 12:17:07.075439 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"335ad996-ad3b-4ac8-a488-64e9453f24a4","Type":"ContainerStarted","Data":"d801a1f1f45c17b5422af1ea12c95b38c64a258c49f3953222cbb94387e19d2c"} Dec 03 12:17:07 crc kubenswrapper[5052]: I1203 12:17:07.075470 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"335ad996-ad3b-4ac8-a488-64e9453f24a4","Type":"ContainerStarted","Data":"df73d6886a1afb1e6d09c9481252bc2871f9a8608386e4ea20b13852f0881a4b"} Dec 03 12:17:07 crc kubenswrapper[5052]: I1203 12:17:07.077584 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" event={"ID":"1ec6b1f7-5abb-4063-80fa-b5ec907dcb91","Type":"ContainerStarted","Data":"50d90ae2ffca4d0cc92a43c260b1dec75f6540c0f3b6a2f388e0315ce52b8fdd"} Dec 03 12:17:08 crc kubenswrapper[5052]: I1203 12:17:08.090534 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"335ad996-ad3b-4ac8-a488-64e9453f24a4","Type":"ContainerStarted","Data":"55c422ce2a089bbc2d4cce09c586d3b79a3abf94fbea99b43ef8ff9d803269d4"} Dec 03 12:17:08 crc kubenswrapper[5052]: I1203 12:17:08.090607 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"335ad996-ad3b-4ac8-a488-64e9453f24a4","Type":"ContainerStarted","Data":"000974bff3024c4dd8ae265f8fc3dc2cd023ab0097daaa4fd6b13c365cb5dd62"} Dec 03 12:17:08 crc kubenswrapper[5052]: I1203 12:17:08.090618 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"335ad996-ad3b-4ac8-a488-64e9453f24a4","Type":"ContainerStarted","Data":"e1a2b505ab3bea37930a1c222d85a3ab5421377ce504e97bef52966f349f5e26"} Dec 03 12:17:08 crc kubenswrapper[5052]: I1203 12:17:08.090629 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"335ad996-ad3b-4ac8-a488-64e9453f24a4","Type":"ContainerStarted","Data":"6af612b56d9c2f4b731475f24619368f730783d7ef872ccc130c33f5a189e57f"} Dec 03 12:17:08 crc kubenswrapper[5052]: I1203 12:17:08.090644 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"335ad996-ad3b-4ac8-a488-64e9453f24a4","Type":"ContainerStarted","Data":"f7f12231dd030c2c9c3f43d8493e609bd58faa9dd7c14aed345ea597010d1451"} Dec 03 12:17:08 crc kubenswrapper[5052]: I1203 12:17:08.091001 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:17:08 crc kubenswrapper[5052]: I1203 12:17:08.125070 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" podStartSLOduration=39.125051833 podStartE2EDuration="39.125051833s" podCreationTimestamp="2025-12-03 12:16:29 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:17:08.12353786 +0000 UTC m=+3418.711779644" watchObservedRunningTime="2025-12-03 12:17:08.125051833 +0000 UTC m=+3418.713293627" Dec 03 12:17:08 crc kubenswrapper[5052]: I1203 12:17:08.172625 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/rabbitmq-server-0" podStartSLOduration=38.172606661 podStartE2EDuration="38.172606661s" podCreationTimestamp="2025-12-03 12:16:30 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:17:08.170710577 +0000 UTC m=+3418.758952341" watchObservedRunningTime="2025-12-03 12:17:08.172606661 +0000 UTC m=+3418.760848415" Dec 03 12:17:09 crc kubenswrapper[5052]: I1203 12:17:09.107287 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"335ad996-ad3b-4ac8-a488-64e9453f24a4","Type":"ContainerStarted","Data":"fb2b07929be7f2dd4d6fd56e12955638b93484338cfedea07147fa092b946832"} Dec 03 12:17:09 crc kubenswrapper[5052]: I1203 12:17:09.107347 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"335ad996-ad3b-4ac8-a488-64e9453f24a4","Type":"ContainerStarted","Data":"9ca86e437e38c5642f8640cedce80dfac23a56655784a6b7769c0317c467d085"} Dec 03 12:17:09 crc kubenswrapper[5052]: I1203 12:17:09.107362 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"335ad996-ad3b-4ac8-a488-64e9453f24a4","Type":"ContainerStarted","Data":"2660887ea0a6145f5154899fe6bb77bb7f4491a21de6c7447f3c126adc776305"} Dec 03 12:17:09 crc kubenswrapper[5052]: I1203 12:17:09.107373 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"335ad996-ad3b-4ac8-a488-64e9453f24a4","Type":"ContainerStarted","Data":"486c4efda615830f80b3292087835a1464d89e6438a42bab394e0f23416c0683"} Dec 03 12:17:09 crc kubenswrapper[5052]: I1203 12:17:09.107383 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"335ad996-ad3b-4ac8-a488-64e9453f24a4","Type":"ContainerStarted","Data":"2b57d07d8ebd7db5a1f0aa4b4ebb75c36047390ad674b3a41f27874f7e01c7b5"} Dec 03 12:17:09 crc kubenswrapper[5052]: I1203 12:17:09.107395 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"335ad996-ad3b-4ac8-a488-64e9453f24a4","Type":"ContainerStarted","Data":"2d0cbf82dccb7f04f14e9a8c02dce57f3fe56aa4bea37bafafd0b0e8ef2825ff"} Dec 03 12:17:09 crc kubenswrapper[5052]: I1203 12:17:09.107405 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"335ad996-ad3b-4ac8-a488-64e9453f24a4","Type":"ContainerStarted","Data":"2591ce792243cc1fe842d92ce9a53f946994c1f42c26f9657e9e29db8d53f707"} Dec 03 12:17:09 crc kubenswrapper[5052]: I1203 12:17:09.336127 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/swift-storage-0" podStartSLOduration=22.336103923 podStartE2EDuration="22.336103923s" podCreationTimestamp="2025-12-03 12:16:47 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:17:09.156881245 +0000 UTC m=+3419.745123039" watchObservedRunningTime="2025-12-03 12:17:09.336103923 +0000 UTC m=+3419.924345687" Dec 03 12:17:09 crc kubenswrapper[5052]: I1203 12:17:09.337297 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-7598b67757-jsb6x"] Dec 03 12:17:09 crc kubenswrapper[5052]: I1203 12:17:09.338710 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-7598b67757-jsb6x" Dec 03 12:17:09 crc kubenswrapper[5052]: I1203 12:17:09.345053 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"dns-swift-storage-0" Dec 03 12:17:09 crc kubenswrapper[5052]: I1203 12:17:09.360225 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-7598b67757-jsb6x"] Dec 03 12:17:09 crc kubenswrapper[5052]: I1203 12:17:09.426106 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dns-swift-storage-0\" (UniqueName: \"kubernetes.io/configmap/20831b4d-3e7e-45c3-accd-406b36c385fa-dns-swift-storage-0\") pod \"dnsmasq-dnsmasq-7598b67757-jsb6x\" (UID: \"20831b4d-3e7e-45c3-accd-406b36c385fa\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-7598b67757-jsb6x" Dec 03 12:17:09 crc kubenswrapper[5052]: I1203 12:17:09.426431 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/20831b4d-3e7e-45c3-accd-406b36c385fa-config\") pod \"dnsmasq-dnsmasq-7598b67757-jsb6x\" (UID: \"20831b4d-3e7e-45c3-accd-406b36c385fa\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-7598b67757-jsb6x" Dec 03 12:17:09 crc kubenswrapper[5052]: I1203 12:17:09.426531 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mhqrl\" (UniqueName: \"kubernetes.io/projected/20831b4d-3e7e-45c3-accd-406b36c385fa-kube-api-access-mhqrl\") pod \"dnsmasq-dnsmasq-7598b67757-jsb6x\" (UID: \"20831b4d-3e7e-45c3-accd-406b36c385fa\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-7598b67757-jsb6x" Dec 03 12:17:09 crc kubenswrapper[5052]: I1203 12:17:09.426564 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/20831b4d-3e7e-45c3-accd-406b36c385fa-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-7598b67757-jsb6x\" (UID: \"20831b4d-3e7e-45c3-accd-406b36c385fa\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-7598b67757-jsb6x" Dec 03 12:17:09 crc kubenswrapper[5052]: I1203 12:17:09.528265 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/20831b4d-3e7e-45c3-accd-406b36c385fa-config\") pod \"dnsmasq-dnsmasq-7598b67757-jsb6x\" (UID: \"20831b4d-3e7e-45c3-accd-406b36c385fa\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-7598b67757-jsb6x" Dec 03 12:17:09 crc kubenswrapper[5052]: I1203 12:17:09.528329 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-mhqrl\" (UniqueName: \"kubernetes.io/projected/20831b4d-3e7e-45c3-accd-406b36c385fa-kube-api-access-mhqrl\") pod \"dnsmasq-dnsmasq-7598b67757-jsb6x\" (UID: \"20831b4d-3e7e-45c3-accd-406b36c385fa\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-7598b67757-jsb6x" Dec 03 12:17:09 crc kubenswrapper[5052]: I1203 12:17:09.528347 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/20831b4d-3e7e-45c3-accd-406b36c385fa-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-7598b67757-jsb6x\" (UID: \"20831b4d-3e7e-45c3-accd-406b36c385fa\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-7598b67757-jsb6x" Dec 03 12:17:09 crc kubenswrapper[5052]: I1203 12:17:09.528387 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"dns-swift-storage-0\" (UniqueName: \"kubernetes.io/configmap/20831b4d-3e7e-45c3-accd-406b36c385fa-dns-swift-storage-0\") pod \"dnsmasq-dnsmasq-7598b67757-jsb6x\" (UID: \"20831b4d-3e7e-45c3-accd-406b36c385fa\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-7598b67757-jsb6x" Dec 03 12:17:09 crc kubenswrapper[5052]: I1203 12:17:09.529140 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/20831b4d-3e7e-45c3-accd-406b36c385fa-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-7598b67757-jsb6x\" (UID: \"20831b4d-3e7e-45c3-accd-406b36c385fa\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-7598b67757-jsb6x" Dec 03 12:17:09 crc kubenswrapper[5052]: I1203 12:17:09.529246 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"dns-swift-storage-0\" (UniqueName: \"kubernetes.io/configmap/20831b4d-3e7e-45c3-accd-406b36c385fa-dns-swift-storage-0\") pod \"dnsmasq-dnsmasq-7598b67757-jsb6x\" (UID: \"20831b4d-3e7e-45c3-accd-406b36c385fa\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-7598b67757-jsb6x" Dec 03 12:17:09 crc kubenswrapper[5052]: I1203 12:17:09.529279 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/20831b4d-3e7e-45c3-accd-406b36c385fa-config\") pod \"dnsmasq-dnsmasq-7598b67757-jsb6x\" (UID: \"20831b4d-3e7e-45c3-accd-406b36c385fa\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-7598b67757-jsb6x" Dec 03 12:17:09 crc kubenswrapper[5052]: I1203 12:17:09.564342 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-mhqrl\" (UniqueName: \"kubernetes.io/projected/20831b4d-3e7e-45c3-accd-406b36c385fa-kube-api-access-mhqrl\") pod \"dnsmasq-dnsmasq-7598b67757-jsb6x\" (UID: \"20831b4d-3e7e-45c3-accd-406b36c385fa\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-7598b67757-jsb6x" Dec 03 12:17:09 crc kubenswrapper[5052]: I1203 12:17:09.657574 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-7598b67757-jsb6x" Dec 03 12:17:10 crc kubenswrapper[5052]: I1203 12:17:10.136690 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-7598b67757-jsb6x"] Dec 03 12:17:10 crc kubenswrapper[5052]: W1203 12:17:10.148540 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod20831b4d_3e7e_45c3_accd_406b36c385fa.slice/crio-e405a3ca4acb967d47085b9658620a8dda399136e2cdcbcdbf9f34cef149ac5f WatchSource:0}: Error finding container e405a3ca4acb967d47085b9658620a8dda399136e2cdcbcdbf9f34cef149ac5f: Status 404 returned error can't find the container with id e405a3ca4acb967d47085b9658620a8dda399136e2cdcbcdbf9f34cef149ac5f Dec 03 12:17:11 crc kubenswrapper[5052]: I1203 12:17:11.124842 5052 generic.go:334] "Generic (PLEG): container finished" podID="20831b4d-3e7e-45c3-accd-406b36c385fa" containerID="c0afa390492cf53e06b476785e0f501d2d08b04be1d1d0bea16a94ce268161a2" exitCode=0 Dec 03 12:17:11 crc kubenswrapper[5052]: I1203 12:17:11.124949 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-7598b67757-jsb6x" event={"ID":"20831b4d-3e7e-45c3-accd-406b36c385fa","Type":"ContainerDied","Data":"c0afa390492cf53e06b476785e0f501d2d08b04be1d1d0bea16a94ce268161a2"} Dec 03 12:17:11 crc kubenswrapper[5052]: I1203 12:17:11.125247 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-7598b67757-jsb6x" event={"ID":"20831b4d-3e7e-45c3-accd-406b36c385fa","Type":"ContainerStarted","Data":"e405a3ca4acb967d47085b9658620a8dda399136e2cdcbcdbf9f34cef149ac5f"} Dec 03 12:17:11 crc kubenswrapper[5052]: I1203 12:17:11.975282 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:17:12 crc kubenswrapper[5052]: I1203 12:17:12.135561 5052 generic.go:334] "Generic (PLEG): container finished" podID="33b07d8d-aae0-436c-831b-aa13ceff481b" containerID="9d59b98890d62735c1c3c35a6531d9c3479675d06cd1783c7c1c65a8112e60ca" exitCode=0 Dec 03 12:17:12 crc kubenswrapper[5052]: I1203 12:17:12.135640 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-db-sync-qstzv" event={"ID":"33b07d8d-aae0-436c-831b-aa13ceff481b","Type":"ContainerDied","Data":"9d59b98890d62735c1c3c35a6531d9c3479675d06cd1783c7c1c65a8112e60ca"} Dec 03 12:17:12 crc kubenswrapper[5052]: I1203 12:17:12.137671 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-7598b67757-jsb6x" event={"ID":"20831b4d-3e7e-45c3-accd-406b36c385fa","Type":"ContainerStarted","Data":"332f879d8fbb05e0197024edb0f25a3acd752356221f62a3cae8691bc75f6f17"} Dec 03 12:17:12 crc kubenswrapper[5052]: I1203 12:17:12.137838 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-7598b67757-jsb6x" Dec 03 12:17:12 crc kubenswrapper[5052]: I1203 12:17:12.167574 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-7598b67757-jsb6x" podStartSLOduration=3.167559269 podStartE2EDuration="3.167559269s" podCreationTimestamp="2025-12-03 12:17:09 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:17:12.165576692 +0000 UTC m=+3422.753818456" watchObservedRunningTime="2025-12-03 12:17:12.167559269 +0000 UTC m=+3422.755801033" Dec 03 12:17:13 crc kubenswrapper[5052]: I1203 12:17:13.478463 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-db-sync-qstzv" Dec 03 12:17:13 crc kubenswrapper[5052]: I1203 12:17:13.515754 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/33b07d8d-aae0-436c-831b-aa13ceff481b-config-data\") pod \"33b07d8d-aae0-436c-831b-aa13ceff481b\" (UID: \"33b07d8d-aae0-436c-831b-aa13ceff481b\") " Dec 03 12:17:13 crc kubenswrapper[5052]: I1203 12:17:13.516214 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/33b07d8d-aae0-436c-831b-aa13ceff481b-db-sync-config-data\") pod \"33b07d8d-aae0-436c-831b-aa13ceff481b\" (UID: \"33b07d8d-aae0-436c-831b-aa13ceff481b\") " Dec 03 12:17:13 crc kubenswrapper[5052]: I1203 12:17:13.516335 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/33b07d8d-aae0-436c-831b-aa13ceff481b-combined-ca-bundle\") pod \"33b07d8d-aae0-436c-831b-aa13ceff481b\" (UID: \"33b07d8d-aae0-436c-831b-aa13ceff481b\") " Dec 03 12:17:13 crc kubenswrapper[5052]: I1203 12:17:13.516373 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hbgd6\" (UniqueName: \"kubernetes.io/projected/33b07d8d-aae0-436c-831b-aa13ceff481b-kube-api-access-hbgd6\") pod \"33b07d8d-aae0-436c-831b-aa13ceff481b\" (UID: \"33b07d8d-aae0-436c-831b-aa13ceff481b\") " Dec 03 12:17:13 crc kubenswrapper[5052]: I1203 12:17:13.523768 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/33b07d8d-aae0-436c-831b-aa13ceff481b-kube-api-access-hbgd6" (OuterVolumeSpecName: "kube-api-access-hbgd6") pod "33b07d8d-aae0-436c-831b-aa13ceff481b" (UID: "33b07d8d-aae0-436c-831b-aa13ceff481b"). InnerVolumeSpecName "kube-api-access-hbgd6". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:17:13 crc kubenswrapper[5052]: I1203 12:17:13.531526 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/33b07d8d-aae0-436c-831b-aa13ceff481b-db-sync-config-data" (OuterVolumeSpecName: "db-sync-config-data") pod "33b07d8d-aae0-436c-831b-aa13ceff481b" (UID: "33b07d8d-aae0-436c-831b-aa13ceff481b"). InnerVolumeSpecName "db-sync-config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:17:13 crc kubenswrapper[5052]: I1203 12:17:13.558482 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/33b07d8d-aae0-436c-831b-aa13ceff481b-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "33b07d8d-aae0-436c-831b-aa13ceff481b" (UID: "33b07d8d-aae0-436c-831b-aa13ceff481b"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:17:13 crc kubenswrapper[5052]: I1203 12:17:13.571968 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/33b07d8d-aae0-436c-831b-aa13ceff481b-config-data" (OuterVolumeSpecName: "config-data") pod "33b07d8d-aae0-436c-831b-aa13ceff481b" (UID: "33b07d8d-aae0-436c-831b-aa13ceff481b"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:17:13 crc kubenswrapper[5052]: I1203 12:17:13.618936 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/33b07d8d-aae0-436c-831b-aa13ceff481b-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:17:13 crc kubenswrapper[5052]: I1203 12:17:13.618988 5052 reconciler_common.go:293] "Volume detached for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/33b07d8d-aae0-436c-831b-aa13ceff481b-db-sync-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:17:13 crc kubenswrapper[5052]: I1203 12:17:13.619000 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/33b07d8d-aae0-436c-831b-aa13ceff481b-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:17:13 crc kubenswrapper[5052]: I1203 12:17:13.619014 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-hbgd6\" (UniqueName: \"kubernetes.io/projected/33b07d8d-aae0-436c-831b-aa13ceff481b-kube-api-access-hbgd6\") on node \"crc\" DevicePath \"\"" Dec 03 12:17:14 crc kubenswrapper[5052]: I1203 12:17:14.157661 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-db-sync-qstzv" event={"ID":"33b07d8d-aae0-436c-831b-aa13ceff481b","Type":"ContainerDied","Data":"9dc821a1a790aeafdf0f867616a2b996745d8ed42e3ef4577b367547c87e356d"} Dec 03 12:17:14 crc kubenswrapper[5052]: I1203 12:17:14.158315 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9dc821a1a790aeafdf0f867616a2b996745d8ed42e3ef4577b367547c87e356d" Dec 03 12:17:14 crc kubenswrapper[5052]: I1203 12:17:14.157742 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-db-sync-qstzv" Dec 03 12:17:16 crc kubenswrapper[5052]: I1203 12:17:16.370381 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openshift-marketplace/certified-operators-528ct"] Dec 03 12:17:16 crc kubenswrapper[5052]: E1203 12:17:16.371075 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="33b07d8d-aae0-436c-831b-aa13ceff481b" containerName="glance-db-sync" Dec 03 12:17:16 crc kubenswrapper[5052]: I1203 12:17:16.371089 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="33b07d8d-aae0-436c-831b-aa13ceff481b" containerName="glance-db-sync" Dec 03 12:17:16 crc kubenswrapper[5052]: I1203 12:17:16.371261 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="33b07d8d-aae0-436c-831b-aa13ceff481b" containerName="glance-db-sync" Dec 03 12:17:16 crc kubenswrapper[5052]: I1203 12:17:16.372450 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/certified-operators-528ct" Dec 03 12:17:16 crc kubenswrapper[5052]: I1203 12:17:16.394769 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/certified-operators-528ct"] Dec 03 12:17:16 crc kubenswrapper[5052]: I1203 12:17:16.462226 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/e033aacf-0878-47d9-8609-e5b83d7c4da7-utilities\") pod \"certified-operators-528ct\" (UID: \"e033aacf-0878-47d9-8609-e5b83d7c4da7\") " pod="openshift-marketplace/certified-operators-528ct" Dec 03 12:17:16 crc kubenswrapper[5052]: I1203 12:17:16.462348 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-frwf9\" (UniqueName: \"kubernetes.io/projected/e033aacf-0878-47d9-8609-e5b83d7c4da7-kube-api-access-frwf9\") pod \"certified-operators-528ct\" (UID: \"e033aacf-0878-47d9-8609-e5b83d7c4da7\") " pod="openshift-marketplace/certified-operators-528ct" Dec 03 12:17:16 crc kubenswrapper[5052]: I1203 12:17:16.462476 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/e033aacf-0878-47d9-8609-e5b83d7c4da7-catalog-content\") pod \"certified-operators-528ct\" (UID: \"e033aacf-0878-47d9-8609-e5b83d7c4da7\") " pod="openshift-marketplace/certified-operators-528ct" Dec 03 12:17:16 crc kubenswrapper[5052]: I1203 12:17:16.563661 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-frwf9\" (UniqueName: \"kubernetes.io/projected/e033aacf-0878-47d9-8609-e5b83d7c4da7-kube-api-access-frwf9\") pod \"certified-operators-528ct\" (UID: \"e033aacf-0878-47d9-8609-e5b83d7c4da7\") " pod="openshift-marketplace/certified-operators-528ct" Dec 03 12:17:16 crc kubenswrapper[5052]: I1203 12:17:16.563797 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/e033aacf-0878-47d9-8609-e5b83d7c4da7-catalog-content\") pod \"certified-operators-528ct\" (UID: \"e033aacf-0878-47d9-8609-e5b83d7c4da7\") " pod="openshift-marketplace/certified-operators-528ct" Dec 03 12:17:16 crc kubenswrapper[5052]: I1203 12:17:16.563858 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/e033aacf-0878-47d9-8609-e5b83d7c4da7-utilities\") pod \"certified-operators-528ct\" (UID: \"e033aacf-0878-47d9-8609-e5b83d7c4da7\") " pod="openshift-marketplace/certified-operators-528ct" Dec 03 12:17:16 crc kubenswrapper[5052]: I1203 12:17:16.565456 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/e033aacf-0878-47d9-8609-e5b83d7c4da7-utilities\") pod \"certified-operators-528ct\" (UID: \"e033aacf-0878-47d9-8609-e5b83d7c4da7\") " pod="openshift-marketplace/certified-operators-528ct" Dec 03 12:17:16 crc kubenswrapper[5052]: I1203 12:17:16.565673 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/e033aacf-0878-47d9-8609-e5b83d7c4da7-catalog-content\") pod \"certified-operators-528ct\" (UID: \"e033aacf-0878-47d9-8609-e5b83d7c4da7\") " pod="openshift-marketplace/certified-operators-528ct" Dec 03 12:17:16 crc kubenswrapper[5052]: I1203 12:17:16.602352 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-frwf9\" (UniqueName: \"kubernetes.io/projected/e033aacf-0878-47d9-8609-e5b83d7c4da7-kube-api-access-frwf9\") pod \"certified-operators-528ct\" (UID: \"e033aacf-0878-47d9-8609-e5b83d7c4da7\") " pod="openshift-marketplace/certified-operators-528ct" Dec 03 12:17:16 crc kubenswrapper[5052]: I1203 12:17:16.693495 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/certified-operators-528ct" Dec 03 12:17:17 crc kubenswrapper[5052]: I1203 12:17:17.223665 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/certified-operators-528ct"] Dec 03 12:17:17 crc kubenswrapper[5052]: W1203 12:17:17.228457 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode033aacf_0878_47d9_8609_e5b83d7c4da7.slice/crio-348a1a8ed17db9bb843f64b0f489d2acaca48522fd7ed8b6c0a2c888265206fc WatchSource:0}: Error finding container 348a1a8ed17db9bb843f64b0f489d2acaca48522fd7ed8b6c0a2c888265206fc: Status 404 returned error can't find the container with id 348a1a8ed17db9bb843f64b0f489d2acaca48522fd7ed8b6c0a2c888265206fc Dec 03 12:17:18 crc kubenswrapper[5052]: I1203 12:17:18.193931 5052 generic.go:334] "Generic (PLEG): container finished" podID="e033aacf-0878-47d9-8609-e5b83d7c4da7" containerID="ffd0e4085392b43898db74aa4780b5303c92e9fd536364fce4e47d8961eba90f" exitCode=0 Dec 03 12:17:18 crc kubenswrapper[5052]: I1203 12:17:18.193977 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/certified-operators-528ct" event={"ID":"e033aacf-0878-47d9-8609-e5b83d7c4da7","Type":"ContainerDied","Data":"ffd0e4085392b43898db74aa4780b5303c92e9fd536364fce4e47d8961eba90f"} Dec 03 12:17:18 crc kubenswrapper[5052]: I1203 12:17:18.194279 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/certified-operators-528ct" event={"ID":"e033aacf-0878-47d9-8609-e5b83d7c4da7","Type":"ContainerStarted","Data":"348a1a8ed17db9bb843f64b0f489d2acaca48522fd7ed8b6c0a2c888265206fc"} Dec 03 12:17:19 crc kubenswrapper[5052]: I1203 12:17:19.659735 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-7598b67757-jsb6x" Dec 03 12:17:19 crc kubenswrapper[5052]: I1203 12:17:19.768109 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-kl2c8"] Dec 03 12:17:19 crc kubenswrapper[5052]: I1203 12:17:19.768388 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-kl2c8" podUID="ebf52f1d-a87e-4300-a650-4c4cb36e80d2" containerName="dnsmasq-dns" containerID="cri-o://fab60ab776551b13a2d2abeee894ccd3d25015754ba7d7788bde10100e3537d9" gracePeriod=10 Dec 03 12:17:20 crc kubenswrapper[5052]: I1203 12:17:20.214630 5052 generic.go:334] "Generic (PLEG): container finished" podID="e033aacf-0878-47d9-8609-e5b83d7c4da7" containerID="533fdaad2d9ffe27d02f9304317e986a4c536afef3275352126b53d6a14a62e1" exitCode=0 Dec 03 12:17:20 crc kubenswrapper[5052]: I1203 12:17:20.214668 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/certified-operators-528ct" event={"ID":"e033aacf-0878-47d9-8609-e5b83d7c4da7","Type":"ContainerDied","Data":"533fdaad2d9ffe27d02f9304317e986a4c536afef3275352126b53d6a14a62e1"} Dec 03 12:17:20 crc kubenswrapper[5052]: I1203 12:17:20.217103 5052 generic.go:334] "Generic (PLEG): container finished" podID="ebf52f1d-a87e-4300-a650-4c4cb36e80d2" containerID="fab60ab776551b13a2d2abeee894ccd3d25015754ba7d7788bde10100e3537d9" exitCode=0 Dec 03 12:17:20 crc kubenswrapper[5052]: I1203 12:17:20.217129 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-kl2c8" event={"ID":"ebf52f1d-a87e-4300-a650-4c4cb36e80d2","Type":"ContainerDied","Data":"fab60ab776551b13a2d2abeee894ccd3d25015754ba7d7788bde10100e3537d9"} Dec 03 12:17:20 crc kubenswrapper[5052]: I1203 12:17:20.217143 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-kl2c8" event={"ID":"ebf52f1d-a87e-4300-a650-4c4cb36e80d2","Type":"ContainerDied","Data":"255fcc6be8986f3475fa6b9c263e74d466e135cd4ba0b85c82b314b0abb90ee9"} Dec 03 12:17:20 crc kubenswrapper[5052]: I1203 12:17:20.217153 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="255fcc6be8986f3475fa6b9c263e74d466e135cd4ba0b85c82b314b0abb90ee9" Dec 03 12:17:20 crc kubenswrapper[5052]: I1203 12:17:20.231164 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-kl2c8" Dec 03 12:17:20 crc kubenswrapper[5052]: I1203 12:17:20.350006 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fknkw\" (UniqueName: \"kubernetes.io/projected/ebf52f1d-a87e-4300-a650-4c4cb36e80d2-kube-api-access-fknkw\") pod \"ebf52f1d-a87e-4300-a650-4c4cb36e80d2\" (UID: \"ebf52f1d-a87e-4300-a650-4c4cb36e80d2\") " Dec 03 12:17:20 crc kubenswrapper[5052]: I1203 12:17:20.350075 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ebf52f1d-a87e-4300-a650-4c4cb36e80d2-config\") pod \"ebf52f1d-a87e-4300-a650-4c4cb36e80d2\" (UID: \"ebf52f1d-a87e-4300-a650-4c4cb36e80d2\") " Dec 03 12:17:20 crc kubenswrapper[5052]: I1203 12:17:20.350223 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/ebf52f1d-a87e-4300-a650-4c4cb36e80d2-dnsmasq-svc\") pod \"ebf52f1d-a87e-4300-a650-4c4cb36e80d2\" (UID: \"ebf52f1d-a87e-4300-a650-4c4cb36e80d2\") " Dec 03 12:17:20 crc kubenswrapper[5052]: I1203 12:17:20.355389 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ebf52f1d-a87e-4300-a650-4c4cb36e80d2-kube-api-access-fknkw" (OuterVolumeSpecName: "kube-api-access-fknkw") pod "ebf52f1d-a87e-4300-a650-4c4cb36e80d2" (UID: "ebf52f1d-a87e-4300-a650-4c4cb36e80d2"). InnerVolumeSpecName "kube-api-access-fknkw". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:17:20 crc kubenswrapper[5052]: I1203 12:17:20.392301 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ebf52f1d-a87e-4300-a650-4c4cb36e80d2-dnsmasq-svc" (OuterVolumeSpecName: "dnsmasq-svc") pod "ebf52f1d-a87e-4300-a650-4c4cb36e80d2" (UID: "ebf52f1d-a87e-4300-a650-4c4cb36e80d2"). InnerVolumeSpecName "dnsmasq-svc". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:17:20 crc kubenswrapper[5052]: I1203 12:17:20.405259 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ebf52f1d-a87e-4300-a650-4c4cb36e80d2-config" (OuterVolumeSpecName: "config") pod "ebf52f1d-a87e-4300-a650-4c4cb36e80d2" (UID: "ebf52f1d-a87e-4300-a650-4c4cb36e80d2"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:17:20 crc kubenswrapper[5052]: I1203 12:17:20.452151 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ebf52f1d-a87e-4300-a650-4c4cb36e80d2-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:17:20 crc kubenswrapper[5052]: I1203 12:17:20.452179 5052 reconciler_common.go:293] "Volume detached for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/ebf52f1d-a87e-4300-a650-4c4cb36e80d2-dnsmasq-svc\") on node \"crc\" DevicePath \"\"" Dec 03 12:17:20 crc kubenswrapper[5052]: I1203 12:17:20.452190 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-fknkw\" (UniqueName: \"kubernetes.io/projected/ebf52f1d-a87e-4300-a650-4c4cb36e80d2-kube-api-access-fknkw\") on node \"crc\" DevicePath \"\"" Dec 03 12:17:21 crc kubenswrapper[5052]: I1203 12:17:21.008745 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:17:21 crc kubenswrapper[5052]: I1203 12:17:21.233063 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/certified-operators-528ct" event={"ID":"e033aacf-0878-47d9-8609-e5b83d7c4da7","Type":"ContainerStarted","Data":"7591deb9124e0ebafd61d96d640ba57880c8a9bc88b609126c48597769c06ff8"} Dec 03 12:17:21 crc kubenswrapper[5052]: I1203 12:17:21.233095 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-kl2c8" Dec 03 12:17:21 crc kubenswrapper[5052]: I1203 12:17:21.271913 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-marketplace/certified-operators-528ct" podStartSLOduration=2.487437531 podStartE2EDuration="5.271886435s" podCreationTimestamp="2025-12-03 12:17:16 +0000 UTC" firstStartedPulling="2025-12-03 12:17:18.196598767 +0000 UTC m=+3428.784840531" lastFinishedPulling="2025-12-03 12:17:20.981047671 +0000 UTC m=+3431.569289435" observedRunningTime="2025-12-03 12:17:21.256170227 +0000 UTC m=+3431.844411991" watchObservedRunningTime="2025-12-03 12:17:21.271886435 +0000 UTC m=+3431.860128209" Dec 03 12:17:21 crc kubenswrapper[5052]: I1203 12:17:21.281208 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-kl2c8"] Dec 03 12:17:21 crc kubenswrapper[5052]: I1203 12:17:21.287464 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-kl2c8"] Dec 03 12:17:21 crc kubenswrapper[5052]: I1203 12:17:21.684443 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ebf52f1d-a87e-4300-a650-4c4cb36e80d2" path="/var/lib/kubelet/pods/ebf52f1d-a87e-4300-a650-4c4cb36e80d2/volumes" Dec 03 12:17:21 crc kubenswrapper[5052]: I1203 12:17:21.977747 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:17:23 crc kubenswrapper[5052]: I1203 12:17:23.103261 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/cinder-db-create-d6bcr"] Dec 03 12:17:23 crc kubenswrapper[5052]: E1203 12:17:23.103912 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="ebf52f1d-a87e-4300-a650-4c4cb36e80d2" containerName="dnsmasq-dns" Dec 03 12:17:23 crc kubenswrapper[5052]: I1203 12:17:23.103926 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="ebf52f1d-a87e-4300-a650-4c4cb36e80d2" containerName="dnsmasq-dns" Dec 03 12:17:23 crc kubenswrapper[5052]: E1203 12:17:23.103937 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="ebf52f1d-a87e-4300-a650-4c4cb36e80d2" containerName="init" Dec 03 12:17:23 crc kubenswrapper[5052]: I1203 12:17:23.103943 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="ebf52f1d-a87e-4300-a650-4c4cb36e80d2" containerName="init" Dec 03 12:17:23 crc kubenswrapper[5052]: I1203 12:17:23.104109 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="ebf52f1d-a87e-4300-a650-4c4cb36e80d2" containerName="dnsmasq-dns" Dec 03 12:17:23 crc kubenswrapper[5052]: I1203 12:17:23.104695 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-db-create-d6bcr" Dec 03 12:17:23 crc kubenswrapper[5052]: I1203 12:17:23.115299 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-db-create-d6bcr"] Dec 03 12:17:23 crc kubenswrapper[5052]: I1203 12:17:23.204661 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/334ceeeb-df59-4f9b-b9e8-46e44c9c515a-operator-scripts\") pod \"cinder-db-create-d6bcr\" (UID: \"334ceeeb-df59-4f9b-b9e8-46e44c9c515a\") " pod="openstack-kuttl-tests/cinder-db-create-d6bcr" Dec 03 12:17:23 crc kubenswrapper[5052]: I1203 12:17:23.204716 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ldfmb\" (UniqueName: \"kubernetes.io/projected/334ceeeb-df59-4f9b-b9e8-46e44c9c515a-kube-api-access-ldfmb\") pod \"cinder-db-create-d6bcr\" (UID: \"334ceeeb-df59-4f9b-b9e8-46e44c9c515a\") " pod="openstack-kuttl-tests/cinder-db-create-d6bcr" Dec 03 12:17:23 crc kubenswrapper[5052]: I1203 12:17:23.205495 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/barbican-db-create-sj945"] Dec 03 12:17:23 crc kubenswrapper[5052]: I1203 12:17:23.207308 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-db-create-sj945" Dec 03 12:17:23 crc kubenswrapper[5052]: I1203 12:17:23.213940 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/barbican-0e0e-account-create-update-cscpf"] Dec 03 12:17:23 crc kubenswrapper[5052]: I1203 12:17:23.215828 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-0e0e-account-create-update-cscpf" Dec 03 12:17:23 crc kubenswrapper[5052]: I1203 12:17:23.218437 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"barbican-db-secret" Dec 03 12:17:23 crc kubenswrapper[5052]: I1203 12:17:23.224946 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-db-create-sj945"] Dec 03 12:17:23 crc kubenswrapper[5052]: I1203 12:17:23.239266 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-0e0e-account-create-update-cscpf"] Dec 03 12:17:23 crc kubenswrapper[5052]: I1203 12:17:23.306271 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/96e36cf6-43a6-43e9-b8dd-64f1424477e8-operator-scripts\") pod \"barbican-db-create-sj945\" (UID: \"96e36cf6-43a6-43e9-b8dd-64f1424477e8\") " pod="openstack-kuttl-tests/barbican-db-create-sj945" Dec 03 12:17:23 crc kubenswrapper[5052]: I1203 12:17:23.306331 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ddjvk\" (UniqueName: \"kubernetes.io/projected/2c08972f-630a-48c4-b7a6-c98c1b0f1a20-kube-api-access-ddjvk\") pod \"barbican-0e0e-account-create-update-cscpf\" (UID: \"2c08972f-630a-48c4-b7a6-c98c1b0f1a20\") " pod="openstack-kuttl-tests/barbican-0e0e-account-create-update-cscpf" Dec 03 12:17:23 crc kubenswrapper[5052]: I1203 12:17:23.306403 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/2c08972f-630a-48c4-b7a6-c98c1b0f1a20-operator-scripts\") pod \"barbican-0e0e-account-create-update-cscpf\" (UID: \"2c08972f-630a-48c4-b7a6-c98c1b0f1a20\") " pod="openstack-kuttl-tests/barbican-0e0e-account-create-update-cscpf" Dec 03 12:17:23 crc kubenswrapper[5052]: I1203 12:17:23.306429 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ccw6d\" (UniqueName: \"kubernetes.io/projected/96e36cf6-43a6-43e9-b8dd-64f1424477e8-kube-api-access-ccw6d\") pod \"barbican-db-create-sj945\" (UID: \"96e36cf6-43a6-43e9-b8dd-64f1424477e8\") " pod="openstack-kuttl-tests/barbican-db-create-sj945" Dec 03 12:17:23 crc kubenswrapper[5052]: I1203 12:17:23.306482 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/334ceeeb-df59-4f9b-b9e8-46e44c9c515a-operator-scripts\") pod \"cinder-db-create-d6bcr\" (UID: \"334ceeeb-df59-4f9b-b9e8-46e44c9c515a\") " pod="openstack-kuttl-tests/cinder-db-create-d6bcr" Dec 03 12:17:23 crc kubenswrapper[5052]: I1203 12:17:23.306531 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-ldfmb\" (UniqueName: \"kubernetes.io/projected/334ceeeb-df59-4f9b-b9e8-46e44c9c515a-kube-api-access-ldfmb\") pod \"cinder-db-create-d6bcr\" (UID: \"334ceeeb-df59-4f9b-b9e8-46e44c9c515a\") " pod="openstack-kuttl-tests/cinder-db-create-d6bcr" Dec 03 12:17:23 crc kubenswrapper[5052]: I1203 12:17:23.307885 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/334ceeeb-df59-4f9b-b9e8-46e44c9c515a-operator-scripts\") pod \"cinder-db-create-d6bcr\" (UID: \"334ceeeb-df59-4f9b-b9e8-46e44c9c515a\") " pod="openstack-kuttl-tests/cinder-db-create-d6bcr" Dec 03 12:17:23 crc kubenswrapper[5052]: I1203 12:17:23.326565 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/cinder-2067-account-create-update-9rsn5"] Dec 03 12:17:23 crc kubenswrapper[5052]: I1203 12:17:23.328265 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-2067-account-create-update-9rsn5" Dec 03 12:17:23 crc kubenswrapper[5052]: I1203 12:17:23.335622 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cinder-db-secret" Dec 03 12:17:23 crc kubenswrapper[5052]: I1203 12:17:23.337824 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-2067-account-create-update-9rsn5"] Dec 03 12:17:23 crc kubenswrapper[5052]: I1203 12:17:23.340954 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-ldfmb\" (UniqueName: \"kubernetes.io/projected/334ceeeb-df59-4f9b-b9e8-46e44c9c515a-kube-api-access-ldfmb\") pod \"cinder-db-create-d6bcr\" (UID: \"334ceeeb-df59-4f9b-b9e8-46e44c9c515a\") " pod="openstack-kuttl-tests/cinder-db-create-d6bcr" Dec 03 12:17:23 crc kubenswrapper[5052]: I1203 12:17:23.397282 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/neutron-db-create-g75vx"] Dec 03 12:17:23 crc kubenswrapper[5052]: I1203 12:17:23.399474 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-db-create-g75vx" Dec 03 12:17:23 crc kubenswrapper[5052]: I1203 12:17:23.430673 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-db-create-d6bcr" Dec 03 12:17:23 crc kubenswrapper[5052]: I1203 12:17:23.432786 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/27756338-429b-45dd-b0c6-9f7ad69f9806-operator-scripts\") pod \"cinder-2067-account-create-update-9rsn5\" (UID: \"27756338-429b-45dd-b0c6-9f7ad69f9806\") " pod="openstack-kuttl-tests/cinder-2067-account-create-update-9rsn5" Dec 03 12:17:23 crc kubenswrapper[5052]: I1203 12:17:23.432867 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/96e36cf6-43a6-43e9-b8dd-64f1424477e8-operator-scripts\") pod \"barbican-db-create-sj945\" (UID: \"96e36cf6-43a6-43e9-b8dd-64f1424477e8\") " pod="openstack-kuttl-tests/barbican-db-create-sj945" Dec 03 12:17:23 crc kubenswrapper[5052]: I1203 12:17:23.433023 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-ddjvk\" (UniqueName: \"kubernetes.io/projected/2c08972f-630a-48c4-b7a6-c98c1b0f1a20-kube-api-access-ddjvk\") pod \"barbican-0e0e-account-create-update-cscpf\" (UID: \"2c08972f-630a-48c4-b7a6-c98c1b0f1a20\") " pod="openstack-kuttl-tests/barbican-0e0e-account-create-update-cscpf" Dec 03 12:17:23 crc kubenswrapper[5052]: I1203 12:17:23.433220 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/2c08972f-630a-48c4-b7a6-c98c1b0f1a20-operator-scripts\") pod \"barbican-0e0e-account-create-update-cscpf\" (UID: \"2c08972f-630a-48c4-b7a6-c98c1b0f1a20\") " pod="openstack-kuttl-tests/barbican-0e0e-account-create-update-cscpf" Dec 03 12:17:23 crc kubenswrapper[5052]: I1203 12:17:23.433271 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-ccw6d\" (UniqueName: \"kubernetes.io/projected/96e36cf6-43a6-43e9-b8dd-64f1424477e8-kube-api-access-ccw6d\") pod \"barbican-db-create-sj945\" (UID: \"96e36cf6-43a6-43e9-b8dd-64f1424477e8\") " pod="openstack-kuttl-tests/barbican-db-create-sj945" Dec 03 12:17:23 crc kubenswrapper[5052]: I1203 12:17:23.433372 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4bd86\" (UniqueName: \"kubernetes.io/projected/27756338-429b-45dd-b0c6-9f7ad69f9806-kube-api-access-4bd86\") pod \"cinder-2067-account-create-update-9rsn5\" (UID: \"27756338-429b-45dd-b0c6-9f7ad69f9806\") " pod="openstack-kuttl-tests/cinder-2067-account-create-update-9rsn5" Dec 03 12:17:23 crc kubenswrapper[5052]: I1203 12:17:23.433677 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/96e36cf6-43a6-43e9-b8dd-64f1424477e8-operator-scripts\") pod \"barbican-db-create-sj945\" (UID: \"96e36cf6-43a6-43e9-b8dd-64f1424477e8\") " pod="openstack-kuttl-tests/barbican-db-create-sj945" Dec 03 12:17:23 crc kubenswrapper[5052]: I1203 12:17:23.434294 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/2c08972f-630a-48c4-b7a6-c98c1b0f1a20-operator-scripts\") pod \"barbican-0e0e-account-create-update-cscpf\" (UID: \"2c08972f-630a-48c4-b7a6-c98c1b0f1a20\") " pod="openstack-kuttl-tests/barbican-0e0e-account-create-update-cscpf" Dec 03 12:17:23 crc kubenswrapper[5052]: I1203 12:17:23.437091 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-db-create-g75vx"] Dec 03 12:17:23 crc kubenswrapper[5052]: I1203 12:17:23.472313 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-ccw6d\" (UniqueName: \"kubernetes.io/projected/96e36cf6-43a6-43e9-b8dd-64f1424477e8-kube-api-access-ccw6d\") pod \"barbican-db-create-sj945\" (UID: \"96e36cf6-43a6-43e9-b8dd-64f1424477e8\") " pod="openstack-kuttl-tests/barbican-db-create-sj945" Dec 03 12:17:23 crc kubenswrapper[5052]: I1203 12:17:23.498646 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/neutron-734a-account-create-update-vxxfj"] Dec 03 12:17:23 crc kubenswrapper[5052]: I1203 12:17:23.501416 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-734a-account-create-update-vxxfj" Dec 03 12:17:23 crc kubenswrapper[5052]: I1203 12:17:23.505313 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-ddjvk\" (UniqueName: \"kubernetes.io/projected/2c08972f-630a-48c4-b7a6-c98c1b0f1a20-kube-api-access-ddjvk\") pod \"barbican-0e0e-account-create-update-cscpf\" (UID: \"2c08972f-630a-48c4-b7a6-c98c1b0f1a20\") " pod="openstack-kuttl-tests/barbican-0e0e-account-create-update-cscpf" Dec 03 12:17:23 crc kubenswrapper[5052]: I1203 12:17:23.506908 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"neutron-db-secret" Dec 03 12:17:23 crc kubenswrapper[5052]: I1203 12:17:23.526946 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-db-create-sj945" Dec 03 12:17:23 crc kubenswrapper[5052]: I1203 12:17:23.537444 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/ee933fcd-48c9-46a1-badd-bc87089bc1e9-operator-scripts\") pod \"neutron-db-create-g75vx\" (UID: \"ee933fcd-48c9-46a1-badd-bc87089bc1e9\") " pod="openstack-kuttl-tests/neutron-db-create-g75vx" Dec 03 12:17:23 crc kubenswrapper[5052]: I1203 12:17:23.537567 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-4bd86\" (UniqueName: \"kubernetes.io/projected/27756338-429b-45dd-b0c6-9f7ad69f9806-kube-api-access-4bd86\") pod \"cinder-2067-account-create-update-9rsn5\" (UID: \"27756338-429b-45dd-b0c6-9f7ad69f9806\") " pod="openstack-kuttl-tests/cinder-2067-account-create-update-9rsn5" Dec 03 12:17:23 crc kubenswrapper[5052]: I1203 12:17:23.537621 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d4kj5\" (UniqueName: \"kubernetes.io/projected/ee933fcd-48c9-46a1-badd-bc87089bc1e9-kube-api-access-d4kj5\") pod \"neutron-db-create-g75vx\" (UID: \"ee933fcd-48c9-46a1-badd-bc87089bc1e9\") " pod="openstack-kuttl-tests/neutron-db-create-g75vx" Dec 03 12:17:23 crc kubenswrapper[5052]: I1203 12:17:23.537688 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/27756338-429b-45dd-b0c6-9f7ad69f9806-operator-scripts\") pod \"cinder-2067-account-create-update-9rsn5\" (UID: \"27756338-429b-45dd-b0c6-9f7ad69f9806\") " pod="openstack-kuttl-tests/cinder-2067-account-create-update-9rsn5" Dec 03 12:17:23 crc kubenswrapper[5052]: I1203 12:17:23.538871 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/27756338-429b-45dd-b0c6-9f7ad69f9806-operator-scripts\") pod \"cinder-2067-account-create-update-9rsn5\" (UID: \"27756338-429b-45dd-b0c6-9f7ad69f9806\") " pod="openstack-kuttl-tests/cinder-2067-account-create-update-9rsn5" Dec 03 12:17:23 crc kubenswrapper[5052]: I1203 12:17:23.539034 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-0e0e-account-create-update-cscpf" Dec 03 12:17:23 crc kubenswrapper[5052]: I1203 12:17:23.550808 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-734a-account-create-update-vxxfj"] Dec 03 12:17:23 crc kubenswrapper[5052]: I1203 12:17:23.557094 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-4bd86\" (UniqueName: \"kubernetes.io/projected/27756338-429b-45dd-b0c6-9f7ad69f9806-kube-api-access-4bd86\") pod \"cinder-2067-account-create-update-9rsn5\" (UID: \"27756338-429b-45dd-b0c6-9f7ad69f9806\") " pod="openstack-kuttl-tests/cinder-2067-account-create-update-9rsn5" Dec 03 12:17:23 crc kubenswrapper[5052]: I1203 12:17:23.566603 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/keystone-db-sync-4tktw"] Dec 03 12:17:23 crc kubenswrapper[5052]: I1203 12:17:23.567908 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-db-sync-4tktw" Dec 03 12:17:23 crc kubenswrapper[5052]: I1203 12:17:23.572058 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone-keystone-dockercfg-kkks7" Dec 03 12:17:23 crc kubenswrapper[5052]: I1203 12:17:23.572386 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone" Dec 03 12:17:23 crc kubenswrapper[5052]: I1203 12:17:23.572576 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone-config-data" Dec 03 12:17:23 crc kubenswrapper[5052]: I1203 12:17:23.572745 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone-scripts" Dec 03 12:17:23 crc kubenswrapper[5052]: I1203 12:17:23.576580 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-db-sync-4tktw"] Dec 03 12:17:23 crc kubenswrapper[5052]: I1203 12:17:23.642181 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qkcds\" (UniqueName: \"kubernetes.io/projected/9d4a23eb-93e4-4fa3-a10a-4317c891692f-kube-api-access-qkcds\") pod \"keystone-db-sync-4tktw\" (UID: \"9d4a23eb-93e4-4fa3-a10a-4317c891692f\") " pod="openstack-kuttl-tests/keystone-db-sync-4tktw" Dec 03 12:17:23 crc kubenswrapper[5052]: I1203 12:17:23.642765 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9d4a23eb-93e4-4fa3-a10a-4317c891692f-combined-ca-bundle\") pod \"keystone-db-sync-4tktw\" (UID: \"9d4a23eb-93e4-4fa3-a10a-4317c891692f\") " pod="openstack-kuttl-tests/keystone-db-sync-4tktw" Dec 03 12:17:23 crc kubenswrapper[5052]: I1203 12:17:23.642815 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/9d4a23eb-93e4-4fa3-a10a-4317c891692f-config-data\") pod \"keystone-db-sync-4tktw\" (UID: \"9d4a23eb-93e4-4fa3-a10a-4317c891692f\") " pod="openstack-kuttl-tests/keystone-db-sync-4tktw" Dec 03 12:17:23 crc kubenswrapper[5052]: I1203 12:17:23.642924 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-d4kj5\" (UniqueName: \"kubernetes.io/projected/ee933fcd-48c9-46a1-badd-bc87089bc1e9-kube-api-access-d4kj5\") pod \"neutron-db-create-g75vx\" (UID: \"ee933fcd-48c9-46a1-badd-bc87089bc1e9\") " pod="openstack-kuttl-tests/neutron-db-create-g75vx" Dec 03 12:17:23 crc kubenswrapper[5052]: I1203 12:17:23.643064 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z52h6\" (UniqueName: \"kubernetes.io/projected/f30ededb-4c5b-4874-97c8-f65e4fe0421d-kube-api-access-z52h6\") pod \"neutron-734a-account-create-update-vxxfj\" (UID: \"f30ededb-4c5b-4874-97c8-f65e4fe0421d\") " pod="openstack-kuttl-tests/neutron-734a-account-create-update-vxxfj" Dec 03 12:17:23 crc kubenswrapper[5052]: I1203 12:17:23.643155 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/f30ededb-4c5b-4874-97c8-f65e4fe0421d-operator-scripts\") pod \"neutron-734a-account-create-update-vxxfj\" (UID: \"f30ededb-4c5b-4874-97c8-f65e4fe0421d\") " pod="openstack-kuttl-tests/neutron-734a-account-create-update-vxxfj" Dec 03 12:17:23 crc kubenswrapper[5052]: I1203 12:17:23.643398 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/ee933fcd-48c9-46a1-badd-bc87089bc1e9-operator-scripts\") pod \"neutron-db-create-g75vx\" (UID: \"ee933fcd-48c9-46a1-badd-bc87089bc1e9\") " pod="openstack-kuttl-tests/neutron-db-create-g75vx" Dec 03 12:17:23 crc kubenswrapper[5052]: I1203 12:17:23.644431 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/ee933fcd-48c9-46a1-badd-bc87089bc1e9-operator-scripts\") pod \"neutron-db-create-g75vx\" (UID: \"ee933fcd-48c9-46a1-badd-bc87089bc1e9\") " pod="openstack-kuttl-tests/neutron-db-create-g75vx" Dec 03 12:17:23 crc kubenswrapper[5052]: I1203 12:17:23.665018 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-d4kj5\" (UniqueName: \"kubernetes.io/projected/ee933fcd-48c9-46a1-badd-bc87089bc1e9-kube-api-access-d4kj5\") pod \"neutron-db-create-g75vx\" (UID: \"ee933fcd-48c9-46a1-badd-bc87089bc1e9\") " pod="openstack-kuttl-tests/neutron-db-create-g75vx" Dec 03 12:17:23 crc kubenswrapper[5052]: I1203 12:17:23.696481 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-2067-account-create-update-9rsn5" Dec 03 12:17:23 crc kubenswrapper[5052]: I1203 12:17:23.748733 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-z52h6\" (UniqueName: \"kubernetes.io/projected/f30ededb-4c5b-4874-97c8-f65e4fe0421d-kube-api-access-z52h6\") pod \"neutron-734a-account-create-update-vxxfj\" (UID: \"f30ededb-4c5b-4874-97c8-f65e4fe0421d\") " pod="openstack-kuttl-tests/neutron-734a-account-create-update-vxxfj" Dec 03 12:17:23 crc kubenswrapper[5052]: I1203 12:17:23.748822 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/f30ededb-4c5b-4874-97c8-f65e4fe0421d-operator-scripts\") pod \"neutron-734a-account-create-update-vxxfj\" (UID: \"f30ededb-4c5b-4874-97c8-f65e4fe0421d\") " pod="openstack-kuttl-tests/neutron-734a-account-create-update-vxxfj" Dec 03 12:17:23 crc kubenswrapper[5052]: I1203 12:17:23.748919 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-qkcds\" (UniqueName: \"kubernetes.io/projected/9d4a23eb-93e4-4fa3-a10a-4317c891692f-kube-api-access-qkcds\") pod \"keystone-db-sync-4tktw\" (UID: \"9d4a23eb-93e4-4fa3-a10a-4317c891692f\") " pod="openstack-kuttl-tests/keystone-db-sync-4tktw" Dec 03 12:17:23 crc kubenswrapper[5052]: I1203 12:17:23.748945 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9d4a23eb-93e4-4fa3-a10a-4317c891692f-combined-ca-bundle\") pod \"keystone-db-sync-4tktw\" (UID: \"9d4a23eb-93e4-4fa3-a10a-4317c891692f\") " pod="openstack-kuttl-tests/keystone-db-sync-4tktw" Dec 03 12:17:23 crc kubenswrapper[5052]: I1203 12:17:23.748968 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/9d4a23eb-93e4-4fa3-a10a-4317c891692f-config-data\") pod \"keystone-db-sync-4tktw\" (UID: \"9d4a23eb-93e4-4fa3-a10a-4317c891692f\") " pod="openstack-kuttl-tests/keystone-db-sync-4tktw" Dec 03 12:17:23 crc kubenswrapper[5052]: I1203 12:17:23.751378 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/f30ededb-4c5b-4874-97c8-f65e4fe0421d-operator-scripts\") pod \"neutron-734a-account-create-update-vxxfj\" (UID: \"f30ededb-4c5b-4874-97c8-f65e4fe0421d\") " pod="openstack-kuttl-tests/neutron-734a-account-create-update-vxxfj" Dec 03 12:17:23 crc kubenswrapper[5052]: I1203 12:17:23.754712 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/9d4a23eb-93e4-4fa3-a10a-4317c891692f-config-data\") pod \"keystone-db-sync-4tktw\" (UID: \"9d4a23eb-93e4-4fa3-a10a-4317c891692f\") " pod="openstack-kuttl-tests/keystone-db-sync-4tktw" Dec 03 12:17:23 crc kubenswrapper[5052]: I1203 12:17:23.755764 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9d4a23eb-93e4-4fa3-a10a-4317c891692f-combined-ca-bundle\") pod \"keystone-db-sync-4tktw\" (UID: \"9d4a23eb-93e4-4fa3-a10a-4317c891692f\") " pod="openstack-kuttl-tests/keystone-db-sync-4tktw" Dec 03 12:17:23 crc kubenswrapper[5052]: I1203 12:17:23.766792 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-z52h6\" (UniqueName: \"kubernetes.io/projected/f30ededb-4c5b-4874-97c8-f65e4fe0421d-kube-api-access-z52h6\") pod \"neutron-734a-account-create-update-vxxfj\" (UID: \"f30ededb-4c5b-4874-97c8-f65e4fe0421d\") " pod="openstack-kuttl-tests/neutron-734a-account-create-update-vxxfj" Dec 03 12:17:23 crc kubenswrapper[5052]: I1203 12:17:23.767038 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-qkcds\" (UniqueName: \"kubernetes.io/projected/9d4a23eb-93e4-4fa3-a10a-4317c891692f-kube-api-access-qkcds\") pod \"keystone-db-sync-4tktw\" (UID: \"9d4a23eb-93e4-4fa3-a10a-4317c891692f\") " pod="openstack-kuttl-tests/keystone-db-sync-4tktw" Dec 03 12:17:23 crc kubenswrapper[5052]: I1203 12:17:23.931973 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-db-create-g75vx" Dec 03 12:17:23 crc kubenswrapper[5052]: I1203 12:17:23.948713 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-734a-account-create-update-vxxfj" Dec 03 12:17:23 crc kubenswrapper[5052]: I1203 12:17:23.977113 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-db-sync-4tktw" Dec 03 12:17:24 crc kubenswrapper[5052]: I1203 12:17:24.036384 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-db-create-d6bcr"] Dec 03 12:17:24 crc kubenswrapper[5052]: I1203 12:17:24.171996 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-db-create-sj945"] Dec 03 12:17:24 crc kubenswrapper[5052]: I1203 12:17:24.206430 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-0e0e-account-create-update-cscpf"] Dec 03 12:17:24 crc kubenswrapper[5052]: W1203 12:17:24.216482 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2c08972f_630a_48c4_b7a6_c98c1b0f1a20.slice/crio-32f8b294c4ce8c07898295547a182d2a90155c8f3a8a7784f3210a73d10c7a6b WatchSource:0}: Error finding container 32f8b294c4ce8c07898295547a182d2a90155c8f3a8a7784f3210a73d10c7a6b: Status 404 returned error can't find the container with id 32f8b294c4ce8c07898295547a182d2a90155c8f3a8a7784f3210a73d10c7a6b Dec 03 12:17:24 crc kubenswrapper[5052]: I1203 12:17:24.338932 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-0e0e-account-create-update-cscpf" event={"ID":"2c08972f-630a-48c4-b7a6-c98c1b0f1a20","Type":"ContainerStarted","Data":"32f8b294c4ce8c07898295547a182d2a90155c8f3a8a7784f3210a73d10c7a6b"} Dec 03 12:17:24 crc kubenswrapper[5052]: I1203 12:17:24.348202 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-db-create-d6bcr" event={"ID":"334ceeeb-df59-4f9b-b9e8-46e44c9c515a","Type":"ContainerStarted","Data":"602556e6db94f43c532ff1aeaa67b1f80695063c213a4ead258df27b0d8e82e8"} Dec 03 12:17:24 crc kubenswrapper[5052]: I1203 12:17:24.352868 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-db-create-sj945" event={"ID":"96e36cf6-43a6-43e9-b8dd-64f1424477e8","Type":"ContainerStarted","Data":"34ed2a1cbd0f47d45bf8f699e77e39ba90214690e5b40ec18b6264826378dfab"} Dec 03 12:17:24 crc kubenswrapper[5052]: I1203 12:17:24.509161 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-2067-account-create-update-9rsn5"] Dec 03 12:17:24 crc kubenswrapper[5052]: I1203 12:17:24.644291 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-db-create-g75vx"] Dec 03 12:17:24 crc kubenswrapper[5052]: I1203 12:17:24.664940 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-734a-account-create-update-vxxfj"] Dec 03 12:17:24 crc kubenswrapper[5052]: I1203 12:17:24.961554 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-db-sync-4tktw"] Dec 03 12:17:25 crc kubenswrapper[5052]: I1203 12:17:25.364015 5052 generic.go:334] "Generic (PLEG): container finished" podID="f30ededb-4c5b-4874-97c8-f65e4fe0421d" containerID="7ee346888c18afd8eeba9670f320334edd272704bcbcc941c871d902c641c5ec" exitCode=0 Dec 03 12:17:25 crc kubenswrapper[5052]: I1203 12:17:25.364170 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-734a-account-create-update-vxxfj" event={"ID":"f30ededb-4c5b-4874-97c8-f65e4fe0421d","Type":"ContainerDied","Data":"7ee346888c18afd8eeba9670f320334edd272704bcbcc941c871d902c641c5ec"} Dec 03 12:17:25 crc kubenswrapper[5052]: I1203 12:17:25.364381 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-734a-account-create-update-vxxfj" event={"ID":"f30ededb-4c5b-4874-97c8-f65e4fe0421d","Type":"ContainerStarted","Data":"e867be337371146955ccfa3d35a5213d6d7a8a19cc053dd5ccebff3a154f5979"} Dec 03 12:17:25 crc kubenswrapper[5052]: I1203 12:17:25.366399 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-2067-account-create-update-9rsn5" event={"ID":"27756338-429b-45dd-b0c6-9f7ad69f9806","Type":"ContainerStarted","Data":"4432cd8497f4f957b73472527d488a26bff3ce7f4676d54c429f0ab40edb3759"} Dec 03 12:17:25 crc kubenswrapper[5052]: I1203 12:17:25.366462 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-2067-account-create-update-9rsn5" event={"ID":"27756338-429b-45dd-b0c6-9f7ad69f9806","Type":"ContainerStarted","Data":"f36300bc2d052550e949af8ebd432f484341bb9965031f6fb73f012a2fe9c605"} Dec 03 12:17:25 crc kubenswrapper[5052]: I1203 12:17:25.374418 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-db-sync-4tktw" event={"ID":"9d4a23eb-93e4-4fa3-a10a-4317c891692f","Type":"ContainerStarted","Data":"b646684ca873f921091ec3cfe94a27d3c5b4b13b7ebda3dd17e9b77bd7ef3b12"} Dec 03 12:17:25 crc kubenswrapper[5052]: I1203 12:17:25.374474 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-db-sync-4tktw" event={"ID":"9d4a23eb-93e4-4fa3-a10a-4317c891692f","Type":"ContainerStarted","Data":"f163fb22ec43a0d88e85dd629f2cb28027498e6ba826aa7159e764ebc2af913c"} Dec 03 12:17:25 crc kubenswrapper[5052]: I1203 12:17:25.385038 5052 generic.go:334] "Generic (PLEG): container finished" podID="2c08972f-630a-48c4-b7a6-c98c1b0f1a20" containerID="4d3f21da798b8c70e54adbb15ba838c217865bc978d044ed0c58d262e9c8b5e3" exitCode=0 Dec 03 12:17:25 crc kubenswrapper[5052]: I1203 12:17:25.385136 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-0e0e-account-create-update-cscpf" event={"ID":"2c08972f-630a-48c4-b7a6-c98c1b0f1a20","Type":"ContainerDied","Data":"4d3f21da798b8c70e54adbb15ba838c217865bc978d044ed0c58d262e9c8b5e3"} Dec 03 12:17:25 crc kubenswrapper[5052]: I1203 12:17:25.386811 5052 generic.go:334] "Generic (PLEG): container finished" podID="334ceeeb-df59-4f9b-b9e8-46e44c9c515a" containerID="0152fe83dd8c9d0afd4f66464b63a8a659d1d2dddf64d378d7b1c0444939feca" exitCode=0 Dec 03 12:17:25 crc kubenswrapper[5052]: I1203 12:17:25.386873 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-db-create-d6bcr" event={"ID":"334ceeeb-df59-4f9b-b9e8-46e44c9c515a","Type":"ContainerDied","Data":"0152fe83dd8c9d0afd4f66464b63a8a659d1d2dddf64d378d7b1c0444939feca"} Dec 03 12:17:25 crc kubenswrapper[5052]: I1203 12:17:25.388009 5052 generic.go:334] "Generic (PLEG): container finished" podID="96e36cf6-43a6-43e9-b8dd-64f1424477e8" containerID="e4692207d1b05926e8b6ed27de18dc9d4ea3c35cfccd32439b2b9da23da7ddaf" exitCode=0 Dec 03 12:17:25 crc kubenswrapper[5052]: I1203 12:17:25.388061 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-db-create-sj945" event={"ID":"96e36cf6-43a6-43e9-b8dd-64f1424477e8","Type":"ContainerDied","Data":"e4692207d1b05926e8b6ed27de18dc9d4ea3c35cfccd32439b2b9da23da7ddaf"} Dec 03 12:17:25 crc kubenswrapper[5052]: I1203 12:17:25.389170 5052 generic.go:334] "Generic (PLEG): container finished" podID="ee933fcd-48c9-46a1-badd-bc87089bc1e9" containerID="05e702674cf68c3b05c75966c10349cb828cdd967a55c92bca4816aad25891d7" exitCode=0 Dec 03 12:17:25 crc kubenswrapper[5052]: I1203 12:17:25.389207 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-db-create-g75vx" event={"ID":"ee933fcd-48c9-46a1-badd-bc87089bc1e9","Type":"ContainerDied","Data":"05e702674cf68c3b05c75966c10349cb828cdd967a55c92bca4816aad25891d7"} Dec 03 12:17:25 crc kubenswrapper[5052]: I1203 12:17:25.389224 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-db-create-g75vx" event={"ID":"ee933fcd-48c9-46a1-badd-bc87089bc1e9","Type":"ContainerStarted","Data":"f1e1b82b5c1ba5752029b05f0b87d12431dce2a4a997d70a37546a726b70abcf"} Dec 03 12:17:25 crc kubenswrapper[5052]: I1203 12:17:25.399408 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/cinder-2067-account-create-update-9rsn5" podStartSLOduration=2.399388167 podStartE2EDuration="2.399388167s" podCreationTimestamp="2025-12-03 12:17:23 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:17:25.39494413 +0000 UTC m=+3435.983185894" watchObservedRunningTime="2025-12-03 12:17:25.399388167 +0000 UTC m=+3435.987629931" Dec 03 12:17:25 crc kubenswrapper[5052]: I1203 12:17:25.422033 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/keystone-db-sync-4tktw" podStartSLOduration=2.422011833 podStartE2EDuration="2.422011833s" podCreationTimestamp="2025-12-03 12:17:23 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:17:25.415660432 +0000 UTC m=+3436.003902206" watchObservedRunningTime="2025-12-03 12:17:25.422011833 +0000 UTC m=+3436.010253597" Dec 03 12:17:26 crc kubenswrapper[5052]: I1203 12:17:26.400247 5052 generic.go:334] "Generic (PLEG): container finished" podID="27756338-429b-45dd-b0c6-9f7ad69f9806" containerID="4432cd8497f4f957b73472527d488a26bff3ce7f4676d54c429f0ab40edb3759" exitCode=0 Dec 03 12:17:26 crc kubenswrapper[5052]: I1203 12:17:26.400298 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-2067-account-create-update-9rsn5" event={"ID":"27756338-429b-45dd-b0c6-9f7ad69f9806","Type":"ContainerDied","Data":"4432cd8497f4f957b73472527d488a26bff3ce7f4676d54c429f0ab40edb3759"} Dec 03 12:17:26 crc kubenswrapper[5052]: I1203 12:17:26.695811 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-marketplace/certified-operators-528ct" Dec 03 12:17:26 crc kubenswrapper[5052]: I1203 12:17:26.696084 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openshift-marketplace/certified-operators-528ct" Dec 03 12:17:26 crc kubenswrapper[5052]: I1203 12:17:26.767435 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-marketplace/certified-operators-528ct" Dec 03 12:17:26 crc kubenswrapper[5052]: I1203 12:17:26.795622 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-db-create-sj945" Dec 03 12:17:26 crc kubenswrapper[5052]: I1203 12:17:26.929422 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/96e36cf6-43a6-43e9-b8dd-64f1424477e8-operator-scripts\") pod \"96e36cf6-43a6-43e9-b8dd-64f1424477e8\" (UID: \"96e36cf6-43a6-43e9-b8dd-64f1424477e8\") " Dec 03 12:17:26 crc kubenswrapper[5052]: I1203 12:17:26.929849 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ccw6d\" (UniqueName: \"kubernetes.io/projected/96e36cf6-43a6-43e9-b8dd-64f1424477e8-kube-api-access-ccw6d\") pod \"96e36cf6-43a6-43e9-b8dd-64f1424477e8\" (UID: \"96e36cf6-43a6-43e9-b8dd-64f1424477e8\") " Dec 03 12:17:26 crc kubenswrapper[5052]: I1203 12:17:26.929851 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/96e36cf6-43a6-43e9-b8dd-64f1424477e8-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "96e36cf6-43a6-43e9-b8dd-64f1424477e8" (UID: "96e36cf6-43a6-43e9-b8dd-64f1424477e8"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:17:26 crc kubenswrapper[5052]: I1203 12:17:26.930315 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/96e36cf6-43a6-43e9-b8dd-64f1424477e8-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:17:26 crc kubenswrapper[5052]: I1203 12:17:26.934381 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/96e36cf6-43a6-43e9-b8dd-64f1424477e8-kube-api-access-ccw6d" (OuterVolumeSpecName: "kube-api-access-ccw6d") pod "96e36cf6-43a6-43e9-b8dd-64f1424477e8" (UID: "96e36cf6-43a6-43e9-b8dd-64f1424477e8"). InnerVolumeSpecName "kube-api-access-ccw6d". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:17:27 crc kubenswrapper[5052]: I1203 12:17:27.003434 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-0e0e-account-create-update-cscpf" Dec 03 12:17:27 crc kubenswrapper[5052]: I1203 12:17:27.014612 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-734a-account-create-update-vxxfj" Dec 03 12:17:27 crc kubenswrapper[5052]: I1203 12:17:27.021178 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-db-create-g75vx" Dec 03 12:17:27 crc kubenswrapper[5052]: I1203 12:17:27.031401 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-ccw6d\" (UniqueName: \"kubernetes.io/projected/96e36cf6-43a6-43e9-b8dd-64f1424477e8-kube-api-access-ccw6d\") on node \"crc\" DevicePath \"\"" Dec 03 12:17:27 crc kubenswrapper[5052]: I1203 12:17:27.031849 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-db-create-d6bcr" Dec 03 12:17:27 crc kubenswrapper[5052]: I1203 12:17:27.132737 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ldfmb\" (UniqueName: \"kubernetes.io/projected/334ceeeb-df59-4f9b-b9e8-46e44c9c515a-kube-api-access-ldfmb\") pod \"334ceeeb-df59-4f9b-b9e8-46e44c9c515a\" (UID: \"334ceeeb-df59-4f9b-b9e8-46e44c9c515a\") " Dec 03 12:17:27 crc kubenswrapper[5052]: I1203 12:17:27.132825 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d4kj5\" (UniqueName: \"kubernetes.io/projected/ee933fcd-48c9-46a1-badd-bc87089bc1e9-kube-api-access-d4kj5\") pod \"ee933fcd-48c9-46a1-badd-bc87089bc1e9\" (UID: \"ee933fcd-48c9-46a1-badd-bc87089bc1e9\") " Dec 03 12:17:27 crc kubenswrapper[5052]: I1203 12:17:27.132887 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z52h6\" (UniqueName: \"kubernetes.io/projected/f30ededb-4c5b-4874-97c8-f65e4fe0421d-kube-api-access-z52h6\") pod \"f30ededb-4c5b-4874-97c8-f65e4fe0421d\" (UID: \"f30ededb-4c5b-4874-97c8-f65e4fe0421d\") " Dec 03 12:17:27 crc kubenswrapper[5052]: I1203 12:17:27.132931 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ddjvk\" (UniqueName: \"kubernetes.io/projected/2c08972f-630a-48c4-b7a6-c98c1b0f1a20-kube-api-access-ddjvk\") pod \"2c08972f-630a-48c4-b7a6-c98c1b0f1a20\" (UID: \"2c08972f-630a-48c4-b7a6-c98c1b0f1a20\") " Dec 03 12:17:27 crc kubenswrapper[5052]: I1203 12:17:27.133000 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/ee933fcd-48c9-46a1-badd-bc87089bc1e9-operator-scripts\") pod \"ee933fcd-48c9-46a1-badd-bc87089bc1e9\" (UID: \"ee933fcd-48c9-46a1-badd-bc87089bc1e9\") " Dec 03 12:17:27 crc kubenswrapper[5052]: I1203 12:17:27.133018 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/f30ededb-4c5b-4874-97c8-f65e4fe0421d-operator-scripts\") pod \"f30ededb-4c5b-4874-97c8-f65e4fe0421d\" (UID: \"f30ededb-4c5b-4874-97c8-f65e4fe0421d\") " Dec 03 12:17:27 crc kubenswrapper[5052]: I1203 12:17:27.133085 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/2c08972f-630a-48c4-b7a6-c98c1b0f1a20-operator-scripts\") pod \"2c08972f-630a-48c4-b7a6-c98c1b0f1a20\" (UID: \"2c08972f-630a-48c4-b7a6-c98c1b0f1a20\") " Dec 03 12:17:27 crc kubenswrapper[5052]: I1203 12:17:27.133125 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/334ceeeb-df59-4f9b-b9e8-46e44c9c515a-operator-scripts\") pod \"334ceeeb-df59-4f9b-b9e8-46e44c9c515a\" (UID: \"334ceeeb-df59-4f9b-b9e8-46e44c9c515a\") " Dec 03 12:17:27 crc kubenswrapper[5052]: I1203 12:17:27.133567 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f30ededb-4c5b-4874-97c8-f65e4fe0421d-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "f30ededb-4c5b-4874-97c8-f65e4fe0421d" (UID: "f30ededb-4c5b-4874-97c8-f65e4fe0421d"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:17:27 crc kubenswrapper[5052]: I1203 12:17:27.133888 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ee933fcd-48c9-46a1-badd-bc87089bc1e9-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "ee933fcd-48c9-46a1-badd-bc87089bc1e9" (UID: "ee933fcd-48c9-46a1-badd-bc87089bc1e9"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:17:27 crc kubenswrapper[5052]: I1203 12:17:27.133892 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2c08972f-630a-48c4-b7a6-c98c1b0f1a20-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "2c08972f-630a-48c4-b7a6-c98c1b0f1a20" (UID: "2c08972f-630a-48c4-b7a6-c98c1b0f1a20"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:17:27 crc kubenswrapper[5052]: I1203 12:17:27.134009 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/334ceeeb-df59-4f9b-b9e8-46e44c9c515a-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "334ceeeb-df59-4f9b-b9e8-46e44c9c515a" (UID: "334ceeeb-df59-4f9b-b9e8-46e44c9c515a"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:17:27 crc kubenswrapper[5052]: I1203 12:17:27.137271 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f30ededb-4c5b-4874-97c8-f65e4fe0421d-kube-api-access-z52h6" (OuterVolumeSpecName: "kube-api-access-z52h6") pod "f30ededb-4c5b-4874-97c8-f65e4fe0421d" (UID: "f30ededb-4c5b-4874-97c8-f65e4fe0421d"). InnerVolumeSpecName "kube-api-access-z52h6". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:17:27 crc kubenswrapper[5052]: I1203 12:17:27.137353 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ee933fcd-48c9-46a1-badd-bc87089bc1e9-kube-api-access-d4kj5" (OuterVolumeSpecName: "kube-api-access-d4kj5") pod "ee933fcd-48c9-46a1-badd-bc87089bc1e9" (UID: "ee933fcd-48c9-46a1-badd-bc87089bc1e9"). InnerVolumeSpecName "kube-api-access-d4kj5". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:17:27 crc kubenswrapper[5052]: I1203 12:17:27.137406 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2c08972f-630a-48c4-b7a6-c98c1b0f1a20-kube-api-access-ddjvk" (OuterVolumeSpecName: "kube-api-access-ddjvk") pod "2c08972f-630a-48c4-b7a6-c98c1b0f1a20" (UID: "2c08972f-630a-48c4-b7a6-c98c1b0f1a20"). InnerVolumeSpecName "kube-api-access-ddjvk". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:17:27 crc kubenswrapper[5052]: I1203 12:17:27.137746 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/334ceeeb-df59-4f9b-b9e8-46e44c9c515a-kube-api-access-ldfmb" (OuterVolumeSpecName: "kube-api-access-ldfmb") pod "334ceeeb-df59-4f9b-b9e8-46e44c9c515a" (UID: "334ceeeb-df59-4f9b-b9e8-46e44c9c515a"). InnerVolumeSpecName "kube-api-access-ldfmb". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:17:27 crc kubenswrapper[5052]: I1203 12:17:27.235420 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/2c08972f-630a-48c4-b7a6-c98c1b0f1a20-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:17:27 crc kubenswrapper[5052]: I1203 12:17:27.235466 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/334ceeeb-df59-4f9b-b9e8-46e44c9c515a-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:17:27 crc kubenswrapper[5052]: I1203 12:17:27.235483 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-ldfmb\" (UniqueName: \"kubernetes.io/projected/334ceeeb-df59-4f9b-b9e8-46e44c9c515a-kube-api-access-ldfmb\") on node \"crc\" DevicePath \"\"" Dec 03 12:17:27 crc kubenswrapper[5052]: I1203 12:17:27.235496 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-d4kj5\" (UniqueName: \"kubernetes.io/projected/ee933fcd-48c9-46a1-badd-bc87089bc1e9-kube-api-access-d4kj5\") on node \"crc\" DevicePath \"\"" Dec 03 12:17:27 crc kubenswrapper[5052]: I1203 12:17:27.235511 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-z52h6\" (UniqueName: \"kubernetes.io/projected/f30ededb-4c5b-4874-97c8-f65e4fe0421d-kube-api-access-z52h6\") on node \"crc\" DevicePath \"\"" Dec 03 12:17:27 crc kubenswrapper[5052]: I1203 12:17:27.235575 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-ddjvk\" (UniqueName: \"kubernetes.io/projected/2c08972f-630a-48c4-b7a6-c98c1b0f1a20-kube-api-access-ddjvk\") on node \"crc\" DevicePath \"\"" Dec 03 12:17:27 crc kubenswrapper[5052]: I1203 12:17:27.235586 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/ee933fcd-48c9-46a1-badd-bc87089bc1e9-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:17:27 crc kubenswrapper[5052]: I1203 12:17:27.235597 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/f30ededb-4c5b-4874-97c8-f65e4fe0421d-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:17:27 crc kubenswrapper[5052]: I1203 12:17:27.411112 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-734a-account-create-update-vxxfj" Dec 03 12:17:27 crc kubenswrapper[5052]: I1203 12:17:27.411125 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-734a-account-create-update-vxxfj" event={"ID":"f30ededb-4c5b-4874-97c8-f65e4fe0421d","Type":"ContainerDied","Data":"e867be337371146955ccfa3d35a5213d6d7a8a19cc053dd5ccebff3a154f5979"} Dec 03 12:17:27 crc kubenswrapper[5052]: I1203 12:17:27.411162 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e867be337371146955ccfa3d35a5213d6d7a8a19cc053dd5ccebff3a154f5979" Dec 03 12:17:27 crc kubenswrapper[5052]: I1203 12:17:27.412819 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-0e0e-account-create-update-cscpf" Dec 03 12:17:27 crc kubenswrapper[5052]: I1203 12:17:27.412822 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-0e0e-account-create-update-cscpf" event={"ID":"2c08972f-630a-48c4-b7a6-c98c1b0f1a20","Type":"ContainerDied","Data":"32f8b294c4ce8c07898295547a182d2a90155c8f3a8a7784f3210a73d10c7a6b"} Dec 03 12:17:27 crc kubenswrapper[5052]: I1203 12:17:27.412934 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="32f8b294c4ce8c07898295547a182d2a90155c8f3a8a7784f3210a73d10c7a6b" Dec 03 12:17:27 crc kubenswrapper[5052]: I1203 12:17:27.414442 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-db-create-d6bcr" event={"ID":"334ceeeb-df59-4f9b-b9e8-46e44c9c515a","Type":"ContainerDied","Data":"602556e6db94f43c532ff1aeaa67b1f80695063c213a4ead258df27b0d8e82e8"} Dec 03 12:17:27 crc kubenswrapper[5052]: I1203 12:17:27.414477 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="602556e6db94f43c532ff1aeaa67b1f80695063c213a4ead258df27b0d8e82e8" Dec 03 12:17:27 crc kubenswrapper[5052]: I1203 12:17:27.414553 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-db-create-d6bcr" Dec 03 12:17:27 crc kubenswrapper[5052]: I1203 12:17:27.418617 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-db-create-sj945" event={"ID":"96e36cf6-43a6-43e9-b8dd-64f1424477e8","Type":"ContainerDied","Data":"34ed2a1cbd0f47d45bf8f699e77e39ba90214690e5b40ec18b6264826378dfab"} Dec 03 12:17:27 crc kubenswrapper[5052]: I1203 12:17:27.418674 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="34ed2a1cbd0f47d45bf8f699e77e39ba90214690e5b40ec18b6264826378dfab" Dec 03 12:17:27 crc kubenswrapper[5052]: I1203 12:17:27.418800 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-db-create-sj945" Dec 03 12:17:27 crc kubenswrapper[5052]: I1203 12:17:27.425601 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-db-create-g75vx" event={"ID":"ee933fcd-48c9-46a1-badd-bc87089bc1e9","Type":"ContainerDied","Data":"f1e1b82b5c1ba5752029b05f0b87d12431dce2a4a997d70a37546a726b70abcf"} Dec 03 12:17:27 crc kubenswrapper[5052]: I1203 12:17:27.425649 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f1e1b82b5c1ba5752029b05f0b87d12431dce2a4a997d70a37546a726b70abcf" Dec 03 12:17:27 crc kubenswrapper[5052]: I1203 12:17:27.425805 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-db-create-g75vx" Dec 03 12:17:27 crc kubenswrapper[5052]: I1203 12:17:27.474806 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-marketplace/certified-operators-528ct" Dec 03 12:17:27 crc kubenswrapper[5052]: I1203 12:17:27.538288 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openshift-marketplace/certified-operators-528ct"] Dec 03 12:17:27 crc kubenswrapper[5052]: I1203 12:17:27.774442 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-2067-account-create-update-9rsn5" Dec 03 12:17:27 crc kubenswrapper[5052]: I1203 12:17:27.848550 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4bd86\" (UniqueName: \"kubernetes.io/projected/27756338-429b-45dd-b0c6-9f7ad69f9806-kube-api-access-4bd86\") pod \"27756338-429b-45dd-b0c6-9f7ad69f9806\" (UID: \"27756338-429b-45dd-b0c6-9f7ad69f9806\") " Dec 03 12:17:27 crc kubenswrapper[5052]: I1203 12:17:27.849576 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/27756338-429b-45dd-b0c6-9f7ad69f9806-operator-scripts\") pod \"27756338-429b-45dd-b0c6-9f7ad69f9806\" (UID: \"27756338-429b-45dd-b0c6-9f7ad69f9806\") " Dec 03 12:17:27 crc kubenswrapper[5052]: I1203 12:17:27.850496 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/27756338-429b-45dd-b0c6-9f7ad69f9806-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "27756338-429b-45dd-b0c6-9f7ad69f9806" (UID: "27756338-429b-45dd-b0c6-9f7ad69f9806"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:17:27 crc kubenswrapper[5052]: I1203 12:17:27.853698 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/27756338-429b-45dd-b0c6-9f7ad69f9806-kube-api-access-4bd86" (OuterVolumeSpecName: "kube-api-access-4bd86") pod "27756338-429b-45dd-b0c6-9f7ad69f9806" (UID: "27756338-429b-45dd-b0c6-9f7ad69f9806"). InnerVolumeSpecName "kube-api-access-4bd86". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:17:27 crc kubenswrapper[5052]: I1203 12:17:27.951274 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/27756338-429b-45dd-b0c6-9f7ad69f9806-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:17:27 crc kubenswrapper[5052]: I1203 12:17:27.951310 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-4bd86\" (UniqueName: \"kubernetes.io/projected/27756338-429b-45dd-b0c6-9f7ad69f9806-kube-api-access-4bd86\") on node \"crc\" DevicePath \"\"" Dec 03 12:17:28 crc kubenswrapper[5052]: I1203 12:17:28.436065 5052 generic.go:334] "Generic (PLEG): container finished" podID="9d4a23eb-93e4-4fa3-a10a-4317c891692f" containerID="b646684ca873f921091ec3cfe94a27d3c5b4b13b7ebda3dd17e9b77bd7ef3b12" exitCode=0 Dec 03 12:17:28 crc kubenswrapper[5052]: I1203 12:17:28.436095 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-db-sync-4tktw" event={"ID":"9d4a23eb-93e4-4fa3-a10a-4317c891692f","Type":"ContainerDied","Data":"b646684ca873f921091ec3cfe94a27d3c5b4b13b7ebda3dd17e9b77bd7ef3b12"} Dec 03 12:17:28 crc kubenswrapper[5052]: I1203 12:17:28.438254 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-2067-account-create-update-9rsn5" Dec 03 12:17:28 crc kubenswrapper[5052]: I1203 12:17:28.438668 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-2067-account-create-update-9rsn5" event={"ID":"27756338-429b-45dd-b0c6-9f7ad69f9806","Type":"ContainerDied","Data":"f36300bc2d052550e949af8ebd432f484341bb9965031f6fb73f012a2fe9c605"} Dec 03 12:17:28 crc kubenswrapper[5052]: I1203 12:17:28.438712 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f36300bc2d052550e949af8ebd432f484341bb9965031f6fb73f012a2fe9c605" Dec 03 12:17:29 crc kubenswrapper[5052]: I1203 12:17:29.379160 5052 patch_prober.go:28] interesting pod/machine-config-daemon-74bj6 container/machine-config-daemon namespace/openshift-machine-config-operator: Liveness probe status=failure output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" start-of-body= Dec 03 12:17:29 crc kubenswrapper[5052]: I1203 12:17:29.379228 5052 prober.go:107] "Probe failed" probeType="Liveness" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" containerName="machine-config-daemon" probeResult="failure" output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" Dec 03 12:17:29 crc kubenswrapper[5052]: I1203 12:17:29.379270 5052 kubelet.go:2542] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" Dec 03 12:17:29 crc kubenswrapper[5052]: I1203 12:17:29.379912 5052 kuberuntime_manager.go:1027] "Message for Container of pod" containerName="machine-config-daemon" containerStatusID={"Type":"cri-o","ID":"79e965387252796514d44c1fd52f83c66501c2dc3200a49ddb7dacbcbcf015ce"} pod="openshift-machine-config-operator/machine-config-daemon-74bj6" containerMessage="Container machine-config-daemon failed liveness probe, will be restarted" Dec 03 12:17:29 crc kubenswrapper[5052]: I1203 12:17:29.379967 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" containerName="machine-config-daemon" containerID="cri-o://79e965387252796514d44c1fd52f83c66501c2dc3200a49ddb7dacbcbcf015ce" gracePeriod=600 Dec 03 12:17:29 crc kubenswrapper[5052]: I1203 12:17:29.445916 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openshift-marketplace/certified-operators-528ct" podUID="e033aacf-0878-47d9-8609-e5b83d7c4da7" containerName="registry-server" containerID="cri-o://7591deb9124e0ebafd61d96d640ba57880c8a9bc88b609126c48597769c06ff8" gracePeriod=2 Dec 03 12:17:29 crc kubenswrapper[5052]: I1203 12:17:29.761190 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-db-sync-4tktw" Dec 03 12:17:29 crc kubenswrapper[5052]: I1203 12:17:29.888575 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9d4a23eb-93e4-4fa3-a10a-4317c891692f-combined-ca-bundle\") pod \"9d4a23eb-93e4-4fa3-a10a-4317c891692f\" (UID: \"9d4a23eb-93e4-4fa3-a10a-4317c891692f\") " Dec 03 12:17:29 crc kubenswrapper[5052]: I1203 12:17:29.888754 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/9d4a23eb-93e4-4fa3-a10a-4317c891692f-config-data\") pod \"9d4a23eb-93e4-4fa3-a10a-4317c891692f\" (UID: \"9d4a23eb-93e4-4fa3-a10a-4317c891692f\") " Dec 03 12:17:29 crc kubenswrapper[5052]: I1203 12:17:29.888793 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qkcds\" (UniqueName: \"kubernetes.io/projected/9d4a23eb-93e4-4fa3-a10a-4317c891692f-kube-api-access-qkcds\") pod \"9d4a23eb-93e4-4fa3-a10a-4317c891692f\" (UID: \"9d4a23eb-93e4-4fa3-a10a-4317c891692f\") " Dec 03 12:17:29 crc kubenswrapper[5052]: I1203 12:17:29.894754 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9d4a23eb-93e4-4fa3-a10a-4317c891692f-kube-api-access-qkcds" (OuterVolumeSpecName: "kube-api-access-qkcds") pod "9d4a23eb-93e4-4fa3-a10a-4317c891692f" (UID: "9d4a23eb-93e4-4fa3-a10a-4317c891692f"). InnerVolumeSpecName "kube-api-access-qkcds". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:17:29 crc kubenswrapper[5052]: I1203 12:17:29.907769 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/certified-operators-528ct" Dec 03 12:17:29 crc kubenswrapper[5052]: I1203 12:17:29.911602 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9d4a23eb-93e4-4fa3-a10a-4317c891692f-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "9d4a23eb-93e4-4fa3-a10a-4317c891692f" (UID: "9d4a23eb-93e4-4fa3-a10a-4317c891692f"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:17:29 crc kubenswrapper[5052]: I1203 12:17:29.950921 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9d4a23eb-93e4-4fa3-a10a-4317c891692f-config-data" (OuterVolumeSpecName: "config-data") pod "9d4a23eb-93e4-4fa3-a10a-4317c891692f" (UID: "9d4a23eb-93e4-4fa3-a10a-4317c891692f"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:17:29 crc kubenswrapper[5052]: I1203 12:17:29.989815 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/e033aacf-0878-47d9-8609-e5b83d7c4da7-utilities\") pod \"e033aacf-0878-47d9-8609-e5b83d7c4da7\" (UID: \"e033aacf-0878-47d9-8609-e5b83d7c4da7\") " Dec 03 12:17:29 crc kubenswrapper[5052]: I1203 12:17:29.990149 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-frwf9\" (UniqueName: \"kubernetes.io/projected/e033aacf-0878-47d9-8609-e5b83d7c4da7-kube-api-access-frwf9\") pod \"e033aacf-0878-47d9-8609-e5b83d7c4da7\" (UID: \"e033aacf-0878-47d9-8609-e5b83d7c4da7\") " Dec 03 12:17:29 crc kubenswrapper[5052]: I1203 12:17:29.990806 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/e033aacf-0878-47d9-8609-e5b83d7c4da7-catalog-content\") pod \"e033aacf-0878-47d9-8609-e5b83d7c4da7\" (UID: \"e033aacf-0878-47d9-8609-e5b83d7c4da7\") " Dec 03 12:17:29 crc kubenswrapper[5052]: I1203 12:17:29.990917 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e033aacf-0878-47d9-8609-e5b83d7c4da7-utilities" (OuterVolumeSpecName: "utilities") pod "e033aacf-0878-47d9-8609-e5b83d7c4da7" (UID: "e033aacf-0878-47d9-8609-e5b83d7c4da7"). InnerVolumeSpecName "utilities". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:17:29 crc kubenswrapper[5052]: I1203 12:17:29.993453 5052 reconciler_common.go:293] "Volume detached for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/e033aacf-0878-47d9-8609-e5b83d7c4da7-utilities\") on node \"crc\" DevicePath \"\"" Dec 03 12:17:29 crc kubenswrapper[5052]: I1203 12:17:29.993492 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/9d4a23eb-93e4-4fa3-a10a-4317c891692f-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:17:29 crc kubenswrapper[5052]: I1203 12:17:29.993521 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-qkcds\" (UniqueName: \"kubernetes.io/projected/9d4a23eb-93e4-4fa3-a10a-4317c891692f-kube-api-access-qkcds\") on node \"crc\" DevicePath \"\"" Dec 03 12:17:29 crc kubenswrapper[5052]: I1203 12:17:29.993535 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9d4a23eb-93e4-4fa3-a10a-4317c891692f-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:17:29 crc kubenswrapper[5052]: I1203 12:17:29.993701 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e033aacf-0878-47d9-8609-e5b83d7c4da7-kube-api-access-frwf9" (OuterVolumeSpecName: "kube-api-access-frwf9") pod "e033aacf-0878-47d9-8609-e5b83d7c4da7" (UID: "e033aacf-0878-47d9-8609-e5b83d7c4da7"). InnerVolumeSpecName "kube-api-access-frwf9". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:17:30 crc kubenswrapper[5052]: I1203 12:17:30.050699 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e033aacf-0878-47d9-8609-e5b83d7c4da7-catalog-content" (OuterVolumeSpecName: "catalog-content") pod "e033aacf-0878-47d9-8609-e5b83d7c4da7" (UID: "e033aacf-0878-47d9-8609-e5b83d7c4da7"). InnerVolumeSpecName "catalog-content". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:17:30 crc kubenswrapper[5052]: I1203 12:17:30.094860 5052 reconciler_common.go:293] "Volume detached for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/e033aacf-0878-47d9-8609-e5b83d7c4da7-catalog-content\") on node \"crc\" DevicePath \"\"" Dec 03 12:17:30 crc kubenswrapper[5052]: I1203 12:17:30.095173 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-frwf9\" (UniqueName: \"kubernetes.io/projected/e033aacf-0878-47d9-8609-e5b83d7c4da7-kube-api-access-frwf9\") on node \"crc\" DevicePath \"\"" Dec 03 12:17:30 crc kubenswrapper[5052]: I1203 12:17:30.457686 5052 generic.go:334] "Generic (PLEG): container finished" podID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" containerID="79e965387252796514d44c1fd52f83c66501c2dc3200a49ddb7dacbcbcf015ce" exitCode=0 Dec 03 12:17:30 crc kubenswrapper[5052]: I1203 12:17:30.457718 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" event={"ID":"10c237bc-72e7-4da3-9f2c-b1688395bf6e","Type":"ContainerDied","Data":"79e965387252796514d44c1fd52f83c66501c2dc3200a49ddb7dacbcbcf015ce"} Dec 03 12:17:30 crc kubenswrapper[5052]: I1203 12:17:30.457789 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" event={"ID":"10c237bc-72e7-4da3-9f2c-b1688395bf6e","Type":"ContainerStarted","Data":"09b4a0072741f60499d24f40cc4ff3406d5d22671ced431363ab865172786300"} Dec 03 12:17:30 crc kubenswrapper[5052]: I1203 12:17:30.457815 5052 scope.go:117] "RemoveContainer" containerID="63298107b3d090deff9ed92f3d613e5b1c86d9137684d117500257db8ee1c805" Dec 03 12:17:30 crc kubenswrapper[5052]: I1203 12:17:30.460245 5052 generic.go:334] "Generic (PLEG): container finished" podID="e033aacf-0878-47d9-8609-e5b83d7c4da7" containerID="7591deb9124e0ebafd61d96d640ba57880c8a9bc88b609126c48597769c06ff8" exitCode=0 Dec 03 12:17:30 crc kubenswrapper[5052]: I1203 12:17:30.460314 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/certified-operators-528ct" Dec 03 12:17:30 crc kubenswrapper[5052]: I1203 12:17:30.460380 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/certified-operators-528ct" event={"ID":"e033aacf-0878-47d9-8609-e5b83d7c4da7","Type":"ContainerDied","Data":"7591deb9124e0ebafd61d96d640ba57880c8a9bc88b609126c48597769c06ff8"} Dec 03 12:17:30 crc kubenswrapper[5052]: I1203 12:17:30.460454 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/certified-operators-528ct" event={"ID":"e033aacf-0878-47d9-8609-e5b83d7c4da7","Type":"ContainerDied","Data":"348a1a8ed17db9bb843f64b0f489d2acaca48522fd7ed8b6c0a2c888265206fc"} Dec 03 12:17:30 crc kubenswrapper[5052]: I1203 12:17:30.462477 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-db-sync-4tktw" event={"ID":"9d4a23eb-93e4-4fa3-a10a-4317c891692f","Type":"ContainerDied","Data":"f163fb22ec43a0d88e85dd629f2cb28027498e6ba826aa7159e764ebc2af913c"} Dec 03 12:17:30 crc kubenswrapper[5052]: I1203 12:17:30.462547 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f163fb22ec43a0d88e85dd629f2cb28027498e6ba826aa7159e764ebc2af913c" Dec 03 12:17:30 crc kubenswrapper[5052]: I1203 12:17:30.462572 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-db-sync-4tktw" Dec 03 12:17:30 crc kubenswrapper[5052]: I1203 12:17:30.510287 5052 scope.go:117] "RemoveContainer" containerID="7591deb9124e0ebafd61d96d640ba57880c8a9bc88b609126c48597769c06ff8" Dec 03 12:17:30 crc kubenswrapper[5052]: I1203 12:17:30.519843 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openshift-marketplace/certified-operators-528ct"] Dec 03 12:17:30 crc kubenswrapper[5052]: I1203 12:17:30.533124 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openshift-marketplace/certified-operators-528ct"] Dec 03 12:17:30 crc kubenswrapper[5052]: I1203 12:17:30.593341 5052 scope.go:117] "RemoveContainer" containerID="533fdaad2d9ffe27d02f9304317e986a4c536afef3275352126b53d6a14a62e1" Dec 03 12:17:30 crc kubenswrapper[5052]: I1203 12:17:30.629560 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/keystone-bootstrap-5qw5w"] Dec 03 12:17:30 crc kubenswrapper[5052]: E1203 12:17:30.630883 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="2c08972f-630a-48c4-b7a6-c98c1b0f1a20" containerName="mariadb-account-create-update" Dec 03 12:17:30 crc kubenswrapper[5052]: I1203 12:17:30.630902 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c08972f-630a-48c4-b7a6-c98c1b0f1a20" containerName="mariadb-account-create-update" Dec 03 12:17:30 crc kubenswrapper[5052]: E1203 12:17:30.630924 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="e033aacf-0878-47d9-8609-e5b83d7c4da7" containerName="extract-utilities" Dec 03 12:17:30 crc kubenswrapper[5052]: I1203 12:17:30.630931 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="e033aacf-0878-47d9-8609-e5b83d7c4da7" containerName="extract-utilities" Dec 03 12:17:30 crc kubenswrapper[5052]: E1203 12:17:30.630952 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="96e36cf6-43a6-43e9-b8dd-64f1424477e8" containerName="mariadb-database-create" Dec 03 12:17:30 crc kubenswrapper[5052]: I1203 12:17:30.630958 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="96e36cf6-43a6-43e9-b8dd-64f1424477e8" containerName="mariadb-database-create" Dec 03 12:17:30 crc kubenswrapper[5052]: E1203 12:17:30.630966 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="27756338-429b-45dd-b0c6-9f7ad69f9806" containerName="mariadb-account-create-update" Dec 03 12:17:30 crc kubenswrapper[5052]: I1203 12:17:30.630972 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="27756338-429b-45dd-b0c6-9f7ad69f9806" containerName="mariadb-account-create-update" Dec 03 12:17:30 crc kubenswrapper[5052]: E1203 12:17:30.630982 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="334ceeeb-df59-4f9b-b9e8-46e44c9c515a" containerName="mariadb-database-create" Dec 03 12:17:30 crc kubenswrapper[5052]: I1203 12:17:30.630988 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="334ceeeb-df59-4f9b-b9e8-46e44c9c515a" containerName="mariadb-database-create" Dec 03 12:17:30 crc kubenswrapper[5052]: E1203 12:17:30.631000 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="9d4a23eb-93e4-4fa3-a10a-4317c891692f" containerName="keystone-db-sync" Dec 03 12:17:30 crc kubenswrapper[5052]: I1203 12:17:30.631005 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d4a23eb-93e4-4fa3-a10a-4317c891692f" containerName="keystone-db-sync" Dec 03 12:17:30 crc kubenswrapper[5052]: E1203 12:17:30.631019 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="e033aacf-0878-47d9-8609-e5b83d7c4da7" containerName="registry-server" Dec 03 12:17:30 crc kubenswrapper[5052]: I1203 12:17:30.631025 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="e033aacf-0878-47d9-8609-e5b83d7c4da7" containerName="registry-server" Dec 03 12:17:30 crc kubenswrapper[5052]: E1203 12:17:30.631032 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="ee933fcd-48c9-46a1-badd-bc87089bc1e9" containerName="mariadb-database-create" Dec 03 12:17:30 crc kubenswrapper[5052]: I1203 12:17:30.631039 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee933fcd-48c9-46a1-badd-bc87089bc1e9" containerName="mariadb-database-create" Dec 03 12:17:30 crc kubenswrapper[5052]: E1203 12:17:30.631048 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="e033aacf-0878-47d9-8609-e5b83d7c4da7" containerName="extract-content" Dec 03 12:17:30 crc kubenswrapper[5052]: I1203 12:17:30.631053 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="e033aacf-0878-47d9-8609-e5b83d7c4da7" containerName="extract-content" Dec 03 12:17:30 crc kubenswrapper[5052]: E1203 12:17:30.631064 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="f30ededb-4c5b-4874-97c8-f65e4fe0421d" containerName="mariadb-account-create-update" Dec 03 12:17:30 crc kubenswrapper[5052]: I1203 12:17:30.631069 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="f30ededb-4c5b-4874-97c8-f65e4fe0421d" containerName="mariadb-account-create-update" Dec 03 12:17:30 crc kubenswrapper[5052]: I1203 12:17:30.633656 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="27756338-429b-45dd-b0c6-9f7ad69f9806" containerName="mariadb-account-create-update" Dec 03 12:17:30 crc kubenswrapper[5052]: I1203 12:17:30.633687 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="ee933fcd-48c9-46a1-badd-bc87089bc1e9" containerName="mariadb-database-create" Dec 03 12:17:30 crc kubenswrapper[5052]: I1203 12:17:30.633698 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="2c08972f-630a-48c4-b7a6-c98c1b0f1a20" containerName="mariadb-account-create-update" Dec 03 12:17:30 crc kubenswrapper[5052]: I1203 12:17:30.633708 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="334ceeeb-df59-4f9b-b9e8-46e44c9c515a" containerName="mariadb-database-create" Dec 03 12:17:30 crc kubenswrapper[5052]: I1203 12:17:30.633721 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="f30ededb-4c5b-4874-97c8-f65e4fe0421d" containerName="mariadb-account-create-update" Dec 03 12:17:30 crc kubenswrapper[5052]: I1203 12:17:30.633755 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="96e36cf6-43a6-43e9-b8dd-64f1424477e8" containerName="mariadb-database-create" Dec 03 12:17:30 crc kubenswrapper[5052]: I1203 12:17:30.633764 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="e033aacf-0878-47d9-8609-e5b83d7c4da7" containerName="registry-server" Dec 03 12:17:30 crc kubenswrapper[5052]: I1203 12:17:30.633776 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="9d4a23eb-93e4-4fa3-a10a-4317c891692f" containerName="keystone-db-sync" Dec 03 12:17:30 crc kubenswrapper[5052]: I1203 12:17:30.634404 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-bootstrap-5qw5w" Dec 03 12:17:30 crc kubenswrapper[5052]: I1203 12:17:30.637225 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone-keystone-dockercfg-kkks7" Dec 03 12:17:30 crc kubenswrapper[5052]: I1203 12:17:30.637385 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"osp-secret" Dec 03 12:17:30 crc kubenswrapper[5052]: I1203 12:17:30.637537 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone" Dec 03 12:17:30 crc kubenswrapper[5052]: I1203 12:17:30.637646 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone-scripts" Dec 03 12:17:30 crc kubenswrapper[5052]: I1203 12:17:30.637827 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone-config-data" Dec 03 12:17:30 crc kubenswrapper[5052]: I1203 12:17:30.664368 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-bootstrap-5qw5w"] Dec 03 12:17:30 crc kubenswrapper[5052]: I1203 12:17:30.685060 5052 scope.go:117] "RemoveContainer" containerID="ffd0e4085392b43898db74aa4780b5303c92e9fd536364fce4e47d8961eba90f" Dec 03 12:17:30 crc kubenswrapper[5052]: I1203 12:17:30.708431 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/38a5d722-dbe5-4d1e-9aec-bf6513db82fb-scripts\") pod \"keystone-bootstrap-5qw5w\" (UID: \"38a5d722-dbe5-4d1e-9aec-bf6513db82fb\") " pod="openstack-kuttl-tests/keystone-bootstrap-5qw5w" Dec 03 12:17:30 crc kubenswrapper[5052]: I1203 12:17:30.708609 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/38a5d722-dbe5-4d1e-9aec-bf6513db82fb-credential-keys\") pod \"keystone-bootstrap-5qw5w\" (UID: \"38a5d722-dbe5-4d1e-9aec-bf6513db82fb\") " pod="openstack-kuttl-tests/keystone-bootstrap-5qw5w" Dec 03 12:17:30 crc kubenswrapper[5052]: I1203 12:17:30.708668 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/38a5d722-dbe5-4d1e-9aec-bf6513db82fb-config-data\") pod \"keystone-bootstrap-5qw5w\" (UID: \"38a5d722-dbe5-4d1e-9aec-bf6513db82fb\") " pod="openstack-kuttl-tests/keystone-bootstrap-5qw5w" Dec 03 12:17:30 crc kubenswrapper[5052]: I1203 12:17:30.708705 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/38a5d722-dbe5-4d1e-9aec-bf6513db82fb-combined-ca-bundle\") pod \"keystone-bootstrap-5qw5w\" (UID: \"38a5d722-dbe5-4d1e-9aec-bf6513db82fb\") " pod="openstack-kuttl-tests/keystone-bootstrap-5qw5w" Dec 03 12:17:30 crc kubenswrapper[5052]: I1203 12:17:30.708833 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bz22k\" (UniqueName: \"kubernetes.io/projected/38a5d722-dbe5-4d1e-9aec-bf6513db82fb-kube-api-access-bz22k\") pod \"keystone-bootstrap-5qw5w\" (UID: \"38a5d722-dbe5-4d1e-9aec-bf6513db82fb\") " pod="openstack-kuttl-tests/keystone-bootstrap-5qw5w" Dec 03 12:17:30 crc kubenswrapper[5052]: I1203 12:17:30.708885 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/38a5d722-dbe5-4d1e-9aec-bf6513db82fb-fernet-keys\") pod \"keystone-bootstrap-5qw5w\" (UID: \"38a5d722-dbe5-4d1e-9aec-bf6513db82fb\") " pod="openstack-kuttl-tests/keystone-bootstrap-5qw5w" Dec 03 12:17:30 crc kubenswrapper[5052]: I1203 12:17:30.735179 5052 scope.go:117] "RemoveContainer" containerID="7591deb9124e0ebafd61d96d640ba57880c8a9bc88b609126c48597769c06ff8" Dec 03 12:17:30 crc kubenswrapper[5052]: E1203 12:17:30.735846 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7591deb9124e0ebafd61d96d640ba57880c8a9bc88b609126c48597769c06ff8\": container with ID starting with 7591deb9124e0ebafd61d96d640ba57880c8a9bc88b609126c48597769c06ff8 not found: ID does not exist" containerID="7591deb9124e0ebafd61d96d640ba57880c8a9bc88b609126c48597769c06ff8" Dec 03 12:17:30 crc kubenswrapper[5052]: I1203 12:17:30.735885 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7591deb9124e0ebafd61d96d640ba57880c8a9bc88b609126c48597769c06ff8"} err="failed to get container status \"7591deb9124e0ebafd61d96d640ba57880c8a9bc88b609126c48597769c06ff8\": rpc error: code = NotFound desc = could not find container \"7591deb9124e0ebafd61d96d640ba57880c8a9bc88b609126c48597769c06ff8\": container with ID starting with 7591deb9124e0ebafd61d96d640ba57880c8a9bc88b609126c48597769c06ff8 not found: ID does not exist" Dec 03 12:17:30 crc kubenswrapper[5052]: I1203 12:17:30.735911 5052 scope.go:117] "RemoveContainer" containerID="533fdaad2d9ffe27d02f9304317e986a4c536afef3275352126b53d6a14a62e1" Dec 03 12:17:30 crc kubenswrapper[5052]: E1203 12:17:30.736721 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"533fdaad2d9ffe27d02f9304317e986a4c536afef3275352126b53d6a14a62e1\": container with ID starting with 533fdaad2d9ffe27d02f9304317e986a4c536afef3275352126b53d6a14a62e1 not found: ID does not exist" containerID="533fdaad2d9ffe27d02f9304317e986a4c536afef3275352126b53d6a14a62e1" Dec 03 12:17:30 crc kubenswrapper[5052]: I1203 12:17:30.736758 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"533fdaad2d9ffe27d02f9304317e986a4c536afef3275352126b53d6a14a62e1"} err="failed to get container status \"533fdaad2d9ffe27d02f9304317e986a4c536afef3275352126b53d6a14a62e1\": rpc error: code = NotFound desc = could not find container \"533fdaad2d9ffe27d02f9304317e986a4c536afef3275352126b53d6a14a62e1\": container with ID starting with 533fdaad2d9ffe27d02f9304317e986a4c536afef3275352126b53d6a14a62e1 not found: ID does not exist" Dec 03 12:17:30 crc kubenswrapper[5052]: I1203 12:17:30.736780 5052 scope.go:117] "RemoveContainer" containerID="ffd0e4085392b43898db74aa4780b5303c92e9fd536364fce4e47d8961eba90f" Dec 03 12:17:30 crc kubenswrapper[5052]: E1203 12:17:30.738924 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ffd0e4085392b43898db74aa4780b5303c92e9fd536364fce4e47d8961eba90f\": container with ID starting with ffd0e4085392b43898db74aa4780b5303c92e9fd536364fce4e47d8961eba90f not found: ID does not exist" containerID="ffd0e4085392b43898db74aa4780b5303c92e9fd536364fce4e47d8961eba90f" Dec 03 12:17:30 crc kubenswrapper[5052]: I1203 12:17:30.738970 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ffd0e4085392b43898db74aa4780b5303c92e9fd536364fce4e47d8961eba90f"} err="failed to get container status \"ffd0e4085392b43898db74aa4780b5303c92e9fd536364fce4e47d8961eba90f\": rpc error: code = NotFound desc = could not find container \"ffd0e4085392b43898db74aa4780b5303c92e9fd536364fce4e47d8961eba90f\": container with ID starting with ffd0e4085392b43898db74aa4780b5303c92e9fd536364fce4e47d8961eba90f not found: ID does not exist" Dec 03 12:17:30 crc kubenswrapper[5052]: I1203 12:17:30.810448 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-bz22k\" (UniqueName: \"kubernetes.io/projected/38a5d722-dbe5-4d1e-9aec-bf6513db82fb-kube-api-access-bz22k\") pod \"keystone-bootstrap-5qw5w\" (UID: \"38a5d722-dbe5-4d1e-9aec-bf6513db82fb\") " pod="openstack-kuttl-tests/keystone-bootstrap-5qw5w" Dec 03 12:17:30 crc kubenswrapper[5052]: I1203 12:17:30.810562 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/38a5d722-dbe5-4d1e-9aec-bf6513db82fb-fernet-keys\") pod \"keystone-bootstrap-5qw5w\" (UID: \"38a5d722-dbe5-4d1e-9aec-bf6513db82fb\") " pod="openstack-kuttl-tests/keystone-bootstrap-5qw5w" Dec 03 12:17:30 crc kubenswrapper[5052]: I1203 12:17:30.810609 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/38a5d722-dbe5-4d1e-9aec-bf6513db82fb-scripts\") pod \"keystone-bootstrap-5qw5w\" (UID: \"38a5d722-dbe5-4d1e-9aec-bf6513db82fb\") " pod="openstack-kuttl-tests/keystone-bootstrap-5qw5w" Dec 03 12:17:30 crc kubenswrapper[5052]: I1203 12:17:30.810664 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/38a5d722-dbe5-4d1e-9aec-bf6513db82fb-credential-keys\") pod \"keystone-bootstrap-5qw5w\" (UID: \"38a5d722-dbe5-4d1e-9aec-bf6513db82fb\") " pod="openstack-kuttl-tests/keystone-bootstrap-5qw5w" Dec 03 12:17:30 crc kubenswrapper[5052]: I1203 12:17:30.810706 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/38a5d722-dbe5-4d1e-9aec-bf6513db82fb-config-data\") pod \"keystone-bootstrap-5qw5w\" (UID: \"38a5d722-dbe5-4d1e-9aec-bf6513db82fb\") " pod="openstack-kuttl-tests/keystone-bootstrap-5qw5w" Dec 03 12:17:30 crc kubenswrapper[5052]: I1203 12:17:30.810746 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/38a5d722-dbe5-4d1e-9aec-bf6513db82fb-combined-ca-bundle\") pod \"keystone-bootstrap-5qw5w\" (UID: \"38a5d722-dbe5-4d1e-9aec-bf6513db82fb\") " pod="openstack-kuttl-tests/keystone-bootstrap-5qw5w" Dec 03 12:17:30 crc kubenswrapper[5052]: I1203 12:17:30.816904 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/38a5d722-dbe5-4d1e-9aec-bf6513db82fb-scripts\") pod \"keystone-bootstrap-5qw5w\" (UID: \"38a5d722-dbe5-4d1e-9aec-bf6513db82fb\") " pod="openstack-kuttl-tests/keystone-bootstrap-5qw5w" Dec 03 12:17:30 crc kubenswrapper[5052]: I1203 12:17:30.817084 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/38a5d722-dbe5-4d1e-9aec-bf6513db82fb-credential-keys\") pod \"keystone-bootstrap-5qw5w\" (UID: \"38a5d722-dbe5-4d1e-9aec-bf6513db82fb\") " pod="openstack-kuttl-tests/keystone-bootstrap-5qw5w" Dec 03 12:17:30 crc kubenswrapper[5052]: I1203 12:17:30.817411 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/38a5d722-dbe5-4d1e-9aec-bf6513db82fb-fernet-keys\") pod \"keystone-bootstrap-5qw5w\" (UID: \"38a5d722-dbe5-4d1e-9aec-bf6513db82fb\") " pod="openstack-kuttl-tests/keystone-bootstrap-5qw5w" Dec 03 12:17:30 crc kubenswrapper[5052]: I1203 12:17:30.818783 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/38a5d722-dbe5-4d1e-9aec-bf6513db82fb-combined-ca-bundle\") pod \"keystone-bootstrap-5qw5w\" (UID: \"38a5d722-dbe5-4d1e-9aec-bf6513db82fb\") " pod="openstack-kuttl-tests/keystone-bootstrap-5qw5w" Dec 03 12:17:30 crc kubenswrapper[5052]: I1203 12:17:30.820178 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/38a5d722-dbe5-4d1e-9aec-bf6513db82fb-config-data\") pod \"keystone-bootstrap-5qw5w\" (UID: \"38a5d722-dbe5-4d1e-9aec-bf6513db82fb\") " pod="openstack-kuttl-tests/keystone-bootstrap-5qw5w" Dec 03 12:17:30 crc kubenswrapper[5052]: I1203 12:17:30.841200 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:17:30 crc kubenswrapper[5052]: I1203 12:17:30.843075 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:17:30 crc kubenswrapper[5052]: I1203 12:17:30.843943 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-bz22k\" (UniqueName: \"kubernetes.io/projected/38a5d722-dbe5-4d1e-9aec-bf6513db82fb-kube-api-access-bz22k\") pod \"keystone-bootstrap-5qw5w\" (UID: \"38a5d722-dbe5-4d1e-9aec-bf6513db82fb\") " pod="openstack-kuttl-tests/keystone-bootstrap-5qw5w" Dec 03 12:17:30 crc kubenswrapper[5052]: I1203 12:17:30.845839 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-scripts" Dec 03 12:17:30 crc kubenswrapper[5052]: I1203 12:17:30.846356 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-config-data" Dec 03 12:17:30 crc kubenswrapper[5052]: I1203 12:17:30.860704 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/neutron-db-sync-kgtt9"] Dec 03 12:17:30 crc kubenswrapper[5052]: I1203 12:17:30.862031 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-db-sync-kgtt9" Dec 03 12:17:30 crc kubenswrapper[5052]: I1203 12:17:30.866268 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"neutron-neutron-dockercfg-vpfh5" Dec 03 12:17:30 crc kubenswrapper[5052]: I1203 12:17:30.867881 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"neutron-config" Dec 03 12:17:30 crc kubenswrapper[5052]: I1203 12:17:30.871121 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"neutron-httpd-config" Dec 03 12:17:30 crc kubenswrapper[5052]: I1203 12:17:30.879454 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-db-sync-kgtt9"] Dec 03 12:17:30 crc kubenswrapper[5052]: I1203 12:17:30.921953 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/614c796f-690e-4c8e-833f-bf7ba7c47d1a-config-data\") pod \"ceilometer-0\" (UID: \"614c796f-690e-4c8e-833f-bf7ba7c47d1a\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:17:30 crc kubenswrapper[5052]: I1203 12:17:30.922009 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/614c796f-690e-4c8e-833f-bf7ba7c47d1a-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"614c796f-690e-4c8e-833f-bf7ba7c47d1a\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:17:30 crc kubenswrapper[5052]: I1203 12:17:30.922051 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/614c796f-690e-4c8e-833f-bf7ba7c47d1a-run-httpd\") pod \"ceilometer-0\" (UID: \"614c796f-690e-4c8e-833f-bf7ba7c47d1a\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:17:30 crc kubenswrapper[5052]: I1203 12:17:30.922077 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hkvxn\" (UniqueName: \"kubernetes.io/projected/614c796f-690e-4c8e-833f-bf7ba7c47d1a-kube-api-access-hkvxn\") pod \"ceilometer-0\" (UID: \"614c796f-690e-4c8e-833f-bf7ba7c47d1a\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:17:30 crc kubenswrapper[5052]: I1203 12:17:30.922127 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/614c796f-690e-4c8e-833f-bf7ba7c47d1a-log-httpd\") pod \"ceilometer-0\" (UID: \"614c796f-690e-4c8e-833f-bf7ba7c47d1a\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:17:30 crc kubenswrapper[5052]: I1203 12:17:30.922173 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/cb69fca5-361d-4273-bee8-27c54648309c-config\") pod \"neutron-db-sync-kgtt9\" (UID: \"cb69fca5-361d-4273-bee8-27c54648309c\") " pod="openstack-kuttl-tests/neutron-db-sync-kgtt9" Dec 03 12:17:30 crc kubenswrapper[5052]: I1203 12:17:30.922204 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-22pgz\" (UniqueName: \"kubernetes.io/projected/cb69fca5-361d-4273-bee8-27c54648309c-kube-api-access-22pgz\") pod \"neutron-db-sync-kgtt9\" (UID: \"cb69fca5-361d-4273-bee8-27c54648309c\") " pod="openstack-kuttl-tests/neutron-db-sync-kgtt9" Dec 03 12:17:30 crc kubenswrapper[5052]: I1203 12:17:30.922223 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/cb69fca5-361d-4273-bee8-27c54648309c-combined-ca-bundle\") pod \"neutron-db-sync-kgtt9\" (UID: \"cb69fca5-361d-4273-bee8-27c54648309c\") " pod="openstack-kuttl-tests/neutron-db-sync-kgtt9" Dec 03 12:17:30 crc kubenswrapper[5052]: I1203 12:17:30.922244 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/614c796f-690e-4c8e-833f-bf7ba7c47d1a-scripts\") pod \"ceilometer-0\" (UID: \"614c796f-690e-4c8e-833f-bf7ba7c47d1a\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:17:30 crc kubenswrapper[5052]: I1203 12:17:30.922284 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/614c796f-690e-4c8e-833f-bf7ba7c47d1a-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"614c796f-690e-4c8e-833f-bf7ba7c47d1a\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:17:30 crc kubenswrapper[5052]: I1203 12:17:30.963335 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:17:30 crc kubenswrapper[5052]: I1203 12:17:30.991282 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-bootstrap-5qw5w" Dec 03 12:17:30 crc kubenswrapper[5052]: I1203 12:17:30.995501 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/barbican-db-sync-pnhqf"] Dec 03 12:17:30 crc kubenswrapper[5052]: I1203 12:17:30.998504 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-db-sync-pnhqf" Dec 03 12:17:31 crc kubenswrapper[5052]: I1203 12:17:31.001873 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"barbican-barbican-dockercfg-jqhml" Dec 03 12:17:31 crc kubenswrapper[5052]: I1203 12:17:31.002466 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"barbican-config-data" Dec 03 12:17:31 crc kubenswrapper[5052]: I1203 12:17:31.021414 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-db-sync-pnhqf"] Dec 03 12:17:31 crc kubenswrapper[5052]: I1203 12:17:31.024041 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/cb69fca5-361d-4273-bee8-27c54648309c-config\") pod \"neutron-db-sync-kgtt9\" (UID: \"cb69fca5-361d-4273-bee8-27c54648309c\") " pod="openstack-kuttl-tests/neutron-db-sync-kgtt9" Dec 03 12:17:31 crc kubenswrapper[5052]: I1203 12:17:31.024110 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-22pgz\" (UniqueName: \"kubernetes.io/projected/cb69fca5-361d-4273-bee8-27c54648309c-kube-api-access-22pgz\") pod \"neutron-db-sync-kgtt9\" (UID: \"cb69fca5-361d-4273-bee8-27c54648309c\") " pod="openstack-kuttl-tests/neutron-db-sync-kgtt9" Dec 03 12:17:31 crc kubenswrapper[5052]: I1203 12:17:31.024138 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/cb69fca5-361d-4273-bee8-27c54648309c-combined-ca-bundle\") pod \"neutron-db-sync-kgtt9\" (UID: \"cb69fca5-361d-4273-bee8-27c54648309c\") " pod="openstack-kuttl-tests/neutron-db-sync-kgtt9" Dec 03 12:17:31 crc kubenswrapper[5052]: I1203 12:17:31.024165 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/614c796f-690e-4c8e-833f-bf7ba7c47d1a-scripts\") pod \"ceilometer-0\" (UID: \"614c796f-690e-4c8e-833f-bf7ba7c47d1a\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:17:31 crc kubenswrapper[5052]: I1203 12:17:31.024213 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/614c796f-690e-4c8e-833f-bf7ba7c47d1a-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"614c796f-690e-4c8e-833f-bf7ba7c47d1a\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:17:31 crc kubenswrapper[5052]: I1203 12:17:31.024243 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/614c796f-690e-4c8e-833f-bf7ba7c47d1a-config-data\") pod \"ceilometer-0\" (UID: \"614c796f-690e-4c8e-833f-bf7ba7c47d1a\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:17:31 crc kubenswrapper[5052]: I1203 12:17:31.024270 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/614c796f-690e-4c8e-833f-bf7ba7c47d1a-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"614c796f-690e-4c8e-833f-bf7ba7c47d1a\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:17:31 crc kubenswrapper[5052]: I1203 12:17:31.024316 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/614c796f-690e-4c8e-833f-bf7ba7c47d1a-run-httpd\") pod \"ceilometer-0\" (UID: \"614c796f-690e-4c8e-833f-bf7ba7c47d1a\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:17:31 crc kubenswrapper[5052]: I1203 12:17:31.024341 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-hkvxn\" (UniqueName: \"kubernetes.io/projected/614c796f-690e-4c8e-833f-bf7ba7c47d1a-kube-api-access-hkvxn\") pod \"ceilometer-0\" (UID: \"614c796f-690e-4c8e-833f-bf7ba7c47d1a\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:17:31 crc kubenswrapper[5052]: I1203 12:17:31.024395 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/614c796f-690e-4c8e-833f-bf7ba7c47d1a-log-httpd\") pod \"ceilometer-0\" (UID: \"614c796f-690e-4c8e-833f-bf7ba7c47d1a\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:17:31 crc kubenswrapper[5052]: I1203 12:17:31.024913 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/614c796f-690e-4c8e-833f-bf7ba7c47d1a-log-httpd\") pod \"ceilometer-0\" (UID: \"614c796f-690e-4c8e-833f-bf7ba7c47d1a\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:17:31 crc kubenswrapper[5052]: I1203 12:17:31.026702 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/614c796f-690e-4c8e-833f-bf7ba7c47d1a-run-httpd\") pod \"ceilometer-0\" (UID: \"614c796f-690e-4c8e-833f-bf7ba7c47d1a\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:17:31 crc kubenswrapper[5052]: I1203 12:17:31.035792 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/614c796f-690e-4c8e-833f-bf7ba7c47d1a-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"614c796f-690e-4c8e-833f-bf7ba7c47d1a\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:17:31 crc kubenswrapper[5052]: I1203 12:17:31.036404 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/cb69fca5-361d-4273-bee8-27c54648309c-combined-ca-bundle\") pod \"neutron-db-sync-kgtt9\" (UID: \"cb69fca5-361d-4273-bee8-27c54648309c\") " pod="openstack-kuttl-tests/neutron-db-sync-kgtt9" Dec 03 12:17:31 crc kubenswrapper[5052]: I1203 12:17:31.036979 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/614c796f-690e-4c8e-833f-bf7ba7c47d1a-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"614c796f-690e-4c8e-833f-bf7ba7c47d1a\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:17:31 crc kubenswrapper[5052]: I1203 12:17:31.038159 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/placement-db-sync-hh2rh"] Dec 03 12:17:31 crc kubenswrapper[5052]: I1203 12:17:31.039544 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-db-sync-hh2rh" Dec 03 12:17:31 crc kubenswrapper[5052]: I1203 12:17:31.042365 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/cb69fca5-361d-4273-bee8-27c54648309c-config\") pod \"neutron-db-sync-kgtt9\" (UID: \"cb69fca5-361d-4273-bee8-27c54648309c\") " pod="openstack-kuttl-tests/neutron-db-sync-kgtt9" Dec 03 12:17:31 crc kubenswrapper[5052]: I1203 12:17:31.044668 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"placement-placement-dockercfg-j6csr" Dec 03 12:17:31 crc kubenswrapper[5052]: I1203 12:17:31.044819 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"placement-config-data" Dec 03 12:17:31 crc kubenswrapper[5052]: I1203 12:17:31.044958 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"placement-scripts" Dec 03 12:17:31 crc kubenswrapper[5052]: I1203 12:17:31.047749 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/614c796f-690e-4c8e-833f-bf7ba7c47d1a-scripts\") pod \"ceilometer-0\" (UID: \"614c796f-690e-4c8e-833f-bf7ba7c47d1a\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:17:31 crc kubenswrapper[5052]: I1203 12:17:31.052813 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/614c796f-690e-4c8e-833f-bf7ba7c47d1a-config-data\") pod \"ceilometer-0\" (UID: \"614c796f-690e-4c8e-833f-bf7ba7c47d1a\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:17:31 crc kubenswrapper[5052]: I1203 12:17:31.058085 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/placement-db-sync-hh2rh"] Dec 03 12:17:31 crc kubenswrapper[5052]: I1203 12:17:31.058632 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-22pgz\" (UniqueName: \"kubernetes.io/projected/cb69fca5-361d-4273-bee8-27c54648309c-kube-api-access-22pgz\") pod \"neutron-db-sync-kgtt9\" (UID: \"cb69fca5-361d-4273-bee8-27c54648309c\") " pod="openstack-kuttl-tests/neutron-db-sync-kgtt9" Dec 03 12:17:31 crc kubenswrapper[5052]: I1203 12:17:31.073968 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-hkvxn\" (UniqueName: \"kubernetes.io/projected/614c796f-690e-4c8e-833f-bf7ba7c47d1a-kube-api-access-hkvxn\") pod \"ceilometer-0\" (UID: \"614c796f-690e-4c8e-833f-bf7ba7c47d1a\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:17:31 crc kubenswrapper[5052]: I1203 12:17:31.125906 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/9f563ce4-a788-44eb-bdab-9fc4c3423176-config-data\") pod \"placement-db-sync-hh2rh\" (UID: \"9f563ce4-a788-44eb-bdab-9fc4c3423176\") " pod="openstack-kuttl-tests/placement-db-sync-hh2rh" Dec 03 12:17:31 crc kubenswrapper[5052]: I1203 12:17:31.125974 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/2f98071b-1e7d-4cf3-976e-ad9073a2f914-db-sync-config-data\") pod \"barbican-db-sync-pnhqf\" (UID: \"2f98071b-1e7d-4cf3-976e-ad9073a2f914\") " pod="openstack-kuttl-tests/barbican-db-sync-pnhqf" Dec 03 12:17:31 crc kubenswrapper[5052]: I1203 12:17:31.126005 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2f98071b-1e7d-4cf3-976e-ad9073a2f914-combined-ca-bundle\") pod \"barbican-db-sync-pnhqf\" (UID: \"2f98071b-1e7d-4cf3-976e-ad9073a2f914\") " pod="openstack-kuttl-tests/barbican-db-sync-pnhqf" Dec 03 12:17:31 crc kubenswrapper[5052]: I1203 12:17:31.126052 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/9f563ce4-a788-44eb-bdab-9fc4c3423176-logs\") pod \"placement-db-sync-hh2rh\" (UID: \"9f563ce4-a788-44eb-bdab-9fc4c3423176\") " pod="openstack-kuttl-tests/placement-db-sync-hh2rh" Dec 03 12:17:31 crc kubenswrapper[5052]: I1203 12:17:31.126073 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nzhhv\" (UniqueName: \"kubernetes.io/projected/2f98071b-1e7d-4cf3-976e-ad9073a2f914-kube-api-access-nzhhv\") pod \"barbican-db-sync-pnhqf\" (UID: \"2f98071b-1e7d-4cf3-976e-ad9073a2f914\") " pod="openstack-kuttl-tests/barbican-db-sync-pnhqf" Dec 03 12:17:31 crc kubenswrapper[5052]: I1203 12:17:31.126094 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sbhzv\" (UniqueName: \"kubernetes.io/projected/9f563ce4-a788-44eb-bdab-9fc4c3423176-kube-api-access-sbhzv\") pod \"placement-db-sync-hh2rh\" (UID: \"9f563ce4-a788-44eb-bdab-9fc4c3423176\") " pod="openstack-kuttl-tests/placement-db-sync-hh2rh" Dec 03 12:17:31 crc kubenswrapper[5052]: I1203 12:17:31.126145 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/9f563ce4-a788-44eb-bdab-9fc4c3423176-scripts\") pod \"placement-db-sync-hh2rh\" (UID: \"9f563ce4-a788-44eb-bdab-9fc4c3423176\") " pod="openstack-kuttl-tests/placement-db-sync-hh2rh" Dec 03 12:17:31 crc kubenswrapper[5052]: I1203 12:17:31.126185 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9f563ce4-a788-44eb-bdab-9fc4c3423176-combined-ca-bundle\") pod \"placement-db-sync-hh2rh\" (UID: \"9f563ce4-a788-44eb-bdab-9fc4c3423176\") " pod="openstack-kuttl-tests/placement-db-sync-hh2rh" Dec 03 12:17:31 crc kubenswrapper[5052]: I1203 12:17:31.228080 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/9f563ce4-a788-44eb-bdab-9fc4c3423176-scripts\") pod \"placement-db-sync-hh2rh\" (UID: \"9f563ce4-a788-44eb-bdab-9fc4c3423176\") " pod="openstack-kuttl-tests/placement-db-sync-hh2rh" Dec 03 12:17:31 crc kubenswrapper[5052]: I1203 12:17:31.228455 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9f563ce4-a788-44eb-bdab-9fc4c3423176-combined-ca-bundle\") pod \"placement-db-sync-hh2rh\" (UID: \"9f563ce4-a788-44eb-bdab-9fc4c3423176\") " pod="openstack-kuttl-tests/placement-db-sync-hh2rh" Dec 03 12:17:31 crc kubenswrapper[5052]: I1203 12:17:31.228481 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/9f563ce4-a788-44eb-bdab-9fc4c3423176-config-data\") pod \"placement-db-sync-hh2rh\" (UID: \"9f563ce4-a788-44eb-bdab-9fc4c3423176\") " pod="openstack-kuttl-tests/placement-db-sync-hh2rh" Dec 03 12:17:31 crc kubenswrapper[5052]: I1203 12:17:31.228534 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/2f98071b-1e7d-4cf3-976e-ad9073a2f914-db-sync-config-data\") pod \"barbican-db-sync-pnhqf\" (UID: \"2f98071b-1e7d-4cf3-976e-ad9073a2f914\") " pod="openstack-kuttl-tests/barbican-db-sync-pnhqf" Dec 03 12:17:31 crc kubenswrapper[5052]: I1203 12:17:31.228561 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2f98071b-1e7d-4cf3-976e-ad9073a2f914-combined-ca-bundle\") pod \"barbican-db-sync-pnhqf\" (UID: \"2f98071b-1e7d-4cf3-976e-ad9073a2f914\") " pod="openstack-kuttl-tests/barbican-db-sync-pnhqf" Dec 03 12:17:31 crc kubenswrapper[5052]: I1203 12:17:31.228608 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/9f563ce4-a788-44eb-bdab-9fc4c3423176-logs\") pod \"placement-db-sync-hh2rh\" (UID: \"9f563ce4-a788-44eb-bdab-9fc4c3423176\") " pod="openstack-kuttl-tests/placement-db-sync-hh2rh" Dec 03 12:17:31 crc kubenswrapper[5052]: I1203 12:17:31.228627 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-nzhhv\" (UniqueName: \"kubernetes.io/projected/2f98071b-1e7d-4cf3-976e-ad9073a2f914-kube-api-access-nzhhv\") pod \"barbican-db-sync-pnhqf\" (UID: \"2f98071b-1e7d-4cf3-976e-ad9073a2f914\") " pod="openstack-kuttl-tests/barbican-db-sync-pnhqf" Dec 03 12:17:31 crc kubenswrapper[5052]: I1203 12:17:31.228655 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-sbhzv\" (UniqueName: \"kubernetes.io/projected/9f563ce4-a788-44eb-bdab-9fc4c3423176-kube-api-access-sbhzv\") pod \"placement-db-sync-hh2rh\" (UID: \"9f563ce4-a788-44eb-bdab-9fc4c3423176\") " pod="openstack-kuttl-tests/placement-db-sync-hh2rh" Dec 03 12:17:31 crc kubenswrapper[5052]: I1203 12:17:31.231409 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/9f563ce4-a788-44eb-bdab-9fc4c3423176-logs\") pod \"placement-db-sync-hh2rh\" (UID: \"9f563ce4-a788-44eb-bdab-9fc4c3423176\") " pod="openstack-kuttl-tests/placement-db-sync-hh2rh" Dec 03 12:17:31 crc kubenswrapper[5052]: I1203 12:17:31.238237 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/9f563ce4-a788-44eb-bdab-9fc4c3423176-config-data\") pod \"placement-db-sync-hh2rh\" (UID: \"9f563ce4-a788-44eb-bdab-9fc4c3423176\") " pod="openstack-kuttl-tests/placement-db-sync-hh2rh" Dec 03 12:17:31 crc kubenswrapper[5052]: I1203 12:17:31.238374 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/9f563ce4-a788-44eb-bdab-9fc4c3423176-scripts\") pod \"placement-db-sync-hh2rh\" (UID: \"9f563ce4-a788-44eb-bdab-9fc4c3423176\") " pod="openstack-kuttl-tests/placement-db-sync-hh2rh" Dec 03 12:17:31 crc kubenswrapper[5052]: I1203 12:17:31.238789 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/2f98071b-1e7d-4cf3-976e-ad9073a2f914-db-sync-config-data\") pod \"barbican-db-sync-pnhqf\" (UID: \"2f98071b-1e7d-4cf3-976e-ad9073a2f914\") " pod="openstack-kuttl-tests/barbican-db-sync-pnhqf" Dec 03 12:17:31 crc kubenswrapper[5052]: I1203 12:17:31.239085 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2f98071b-1e7d-4cf3-976e-ad9073a2f914-combined-ca-bundle\") pod \"barbican-db-sync-pnhqf\" (UID: \"2f98071b-1e7d-4cf3-976e-ad9073a2f914\") " pod="openstack-kuttl-tests/barbican-db-sync-pnhqf" Dec 03 12:17:31 crc kubenswrapper[5052]: I1203 12:17:31.244966 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9f563ce4-a788-44eb-bdab-9fc4c3423176-combined-ca-bundle\") pod \"placement-db-sync-hh2rh\" (UID: \"9f563ce4-a788-44eb-bdab-9fc4c3423176\") " pod="openstack-kuttl-tests/placement-db-sync-hh2rh" Dec 03 12:17:31 crc kubenswrapper[5052]: I1203 12:17:31.248657 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:17:31 crc kubenswrapper[5052]: I1203 12:17:31.256277 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-nzhhv\" (UniqueName: \"kubernetes.io/projected/2f98071b-1e7d-4cf3-976e-ad9073a2f914-kube-api-access-nzhhv\") pod \"barbican-db-sync-pnhqf\" (UID: \"2f98071b-1e7d-4cf3-976e-ad9073a2f914\") " pod="openstack-kuttl-tests/barbican-db-sync-pnhqf" Dec 03 12:17:31 crc kubenswrapper[5052]: I1203 12:17:31.258841 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-sbhzv\" (UniqueName: \"kubernetes.io/projected/9f563ce4-a788-44eb-bdab-9fc4c3423176-kube-api-access-sbhzv\") pod \"placement-db-sync-hh2rh\" (UID: \"9f563ce4-a788-44eb-bdab-9fc4c3423176\") " pod="openstack-kuttl-tests/placement-db-sync-hh2rh" Dec 03 12:17:31 crc kubenswrapper[5052]: I1203 12:17:31.280736 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-db-sync-kgtt9" Dec 03 12:17:31 crc kubenswrapper[5052]: I1203 12:17:31.332104 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-db-sync-pnhqf" Dec 03 12:17:31 crc kubenswrapper[5052]: I1203 12:17:31.375956 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-db-sync-hh2rh" Dec 03 12:17:31 crc kubenswrapper[5052]: I1203 12:17:31.570327 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-bootstrap-5qw5w"] Dec 03 12:17:31 crc kubenswrapper[5052]: I1203 12:17:31.713402 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e033aacf-0878-47d9-8609-e5b83d7c4da7" path="/var/lib/kubelet/pods/e033aacf-0878-47d9-8609-e5b83d7c4da7/volumes" Dec 03 12:17:31 crc kubenswrapper[5052]: I1203 12:17:31.753481 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:17:31 crc kubenswrapper[5052]: I1203 12:17:31.755391 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:17:31 crc kubenswrapper[5052]: I1203 12:17:31.762163 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-scripts" Dec 03 12:17:31 crc kubenswrapper[5052]: I1203 12:17:31.763417 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-default-external-config-data" Dec 03 12:17:31 crc kubenswrapper[5052]: I1203 12:17:31.763654 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-glance-default-public-svc" Dec 03 12:17:31 crc kubenswrapper[5052]: I1203 12:17:31.763862 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-glance-dockercfg-rsctn" Dec 03 12:17:31 crc kubenswrapper[5052]: I1203 12:17:31.768002 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:17:31 crc kubenswrapper[5052]: I1203 12:17:31.777041 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:17:31 crc kubenswrapper[5052]: W1203 12:17:31.779068 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod614c796f_690e_4c8e_833f_bf7ba7c47d1a.slice/crio-baec3061547d2412c64a59a314db756ac34d9a235835cd5adbe0fdfa0faf5bbf WatchSource:0}: Error finding container baec3061547d2412c64a59a314db756ac34d9a235835cd5adbe0fdfa0faf5bbf: Status 404 returned error can't find the container with id baec3061547d2412c64a59a314db756ac34d9a235835cd5adbe0fdfa0faf5bbf Dec 03 12:17:31 crc kubenswrapper[5052]: I1203 12:17:31.843001 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:17:31 crc kubenswrapper[5052]: I1203 12:17:31.843159 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage09-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage09-crc\") pod \"glance-default-external-api-0\" (UID: \"408d4a98-4cc6-4a13-a2f3-940747dfb910\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:17:31 crc kubenswrapper[5052]: I1203 12:17:31.843233 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/408d4a98-4cc6-4a13-a2f3-940747dfb910-logs\") pod \"glance-default-external-api-0\" (UID: \"408d4a98-4cc6-4a13-a2f3-940747dfb910\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:17:31 crc kubenswrapper[5052]: I1203 12:17:31.843262 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/408d4a98-4cc6-4a13-a2f3-940747dfb910-public-tls-certs\") pod \"glance-default-external-api-0\" (UID: \"408d4a98-4cc6-4a13-a2f3-940747dfb910\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:17:31 crc kubenswrapper[5052]: I1203 12:17:31.843304 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/408d4a98-4cc6-4a13-a2f3-940747dfb910-httpd-run\") pod \"glance-default-external-api-0\" (UID: \"408d4a98-4cc6-4a13-a2f3-940747dfb910\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:17:31 crc kubenswrapper[5052]: I1203 12:17:31.843740 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/408d4a98-4cc6-4a13-a2f3-940747dfb910-scripts\") pod \"glance-default-external-api-0\" (UID: \"408d4a98-4cc6-4a13-a2f3-940747dfb910\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:17:31 crc kubenswrapper[5052]: I1203 12:17:31.843822 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fvlmk\" (UniqueName: \"kubernetes.io/projected/408d4a98-4cc6-4a13-a2f3-940747dfb910-kube-api-access-fvlmk\") pod \"glance-default-external-api-0\" (UID: \"408d4a98-4cc6-4a13-a2f3-940747dfb910\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:17:31 crc kubenswrapper[5052]: I1203 12:17:31.843881 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/408d4a98-4cc6-4a13-a2f3-940747dfb910-config-data\") pod \"glance-default-external-api-0\" (UID: \"408d4a98-4cc6-4a13-a2f3-940747dfb910\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:17:31 crc kubenswrapper[5052]: I1203 12:17:31.843928 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/408d4a98-4cc6-4a13-a2f3-940747dfb910-combined-ca-bundle\") pod \"glance-default-external-api-0\" (UID: \"408d4a98-4cc6-4a13-a2f3-940747dfb910\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:17:31 crc kubenswrapper[5052]: I1203 12:17:31.844631 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:17:31 crc kubenswrapper[5052]: I1203 12:17:31.848372 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-glance-default-internal-svc" Dec 03 12:17:31 crc kubenswrapper[5052]: I1203 12:17:31.848664 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-default-internal-config-data" Dec 03 12:17:31 crc kubenswrapper[5052]: I1203 12:17:31.857150 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:17:31 crc kubenswrapper[5052]: I1203 12:17:31.902624 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-db-sync-kgtt9"] Dec 03 12:17:31 crc kubenswrapper[5052]: W1203 12:17:31.905342 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcb69fca5_361d_4273_bee8_27c54648309c.slice/crio-37723aa45e613d2653462e3c8d41df055a7a9bbe5732538efa703b6871bd789e WatchSource:0}: Error finding container 37723aa45e613d2653462e3c8d41df055a7a9bbe5732538efa703b6871bd789e: Status 404 returned error can't find the container with id 37723aa45e613d2653462e3c8d41df055a7a9bbe5732538efa703b6871bd789e Dec 03 12:17:31 crc kubenswrapper[5052]: I1203 12:17:31.915719 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-db-sync-pnhqf"] Dec 03 12:17:31 crc kubenswrapper[5052]: W1203 12:17:31.934075 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2f98071b_1e7d_4cf3_976e_ad9073a2f914.slice/crio-ed66b602b37675ef4806ee0a6e812de42767f2a4c5c1228a1e553b32060605b4 WatchSource:0}: Error finding container ed66b602b37675ef4806ee0a6e812de42767f2a4c5c1228a1e553b32060605b4: Status 404 returned error can't find the container with id ed66b602b37675ef4806ee0a6e812de42767f2a4c5c1228a1e553b32060605b4 Dec 03 12:17:31 crc kubenswrapper[5052]: I1203 12:17:31.945086 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-fvlmk\" (UniqueName: \"kubernetes.io/projected/408d4a98-4cc6-4a13-a2f3-940747dfb910-kube-api-access-fvlmk\") pod \"glance-default-external-api-0\" (UID: \"408d4a98-4cc6-4a13-a2f3-940747dfb910\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:17:31 crc kubenswrapper[5052]: I1203 12:17:31.945120 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/408d4a98-4cc6-4a13-a2f3-940747dfb910-config-data\") pod \"glance-default-external-api-0\" (UID: \"408d4a98-4cc6-4a13-a2f3-940747dfb910\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:17:31 crc kubenswrapper[5052]: I1203 12:17:31.945147 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/e51b5737-6907-4db8-a9a6-1019da0e64d2-internal-tls-certs\") pod \"glance-default-internal-api-0\" (UID: \"e51b5737-6907-4db8-a9a6-1019da0e64d2\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:17:31 crc kubenswrapper[5052]: I1203 12:17:31.945175 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/408d4a98-4cc6-4a13-a2f3-940747dfb910-combined-ca-bundle\") pod \"glance-default-external-api-0\" (UID: \"408d4a98-4cc6-4a13-a2f3-940747dfb910\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:17:31 crc kubenswrapper[5052]: I1203 12:17:31.945193 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/e51b5737-6907-4db8-a9a6-1019da0e64d2-logs\") pod \"glance-default-internal-api-0\" (UID: \"e51b5737-6907-4db8-a9a6-1019da0e64d2\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:17:31 crc kubenswrapper[5052]: I1203 12:17:31.945216 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/e51b5737-6907-4db8-a9a6-1019da0e64d2-httpd-run\") pod \"glance-default-internal-api-0\" (UID: \"e51b5737-6907-4db8-a9a6-1019da0e64d2\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:17:31 crc kubenswrapper[5052]: I1203 12:17:31.945255 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage09-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage09-crc\") pod \"glance-default-external-api-0\" (UID: \"408d4a98-4cc6-4a13-a2f3-940747dfb910\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:17:31 crc kubenswrapper[5052]: I1203 12:17:31.945272 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e51b5737-6907-4db8-a9a6-1019da0e64d2-combined-ca-bundle\") pod \"glance-default-internal-api-0\" (UID: \"e51b5737-6907-4db8-a9a6-1019da0e64d2\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:17:31 crc kubenswrapper[5052]: I1203 12:17:31.945295 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/408d4a98-4cc6-4a13-a2f3-940747dfb910-logs\") pod \"glance-default-external-api-0\" (UID: \"408d4a98-4cc6-4a13-a2f3-940747dfb910\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:17:31 crc kubenswrapper[5052]: I1203 12:17:31.945308 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/408d4a98-4cc6-4a13-a2f3-940747dfb910-public-tls-certs\") pod \"glance-default-external-api-0\" (UID: \"408d4a98-4cc6-4a13-a2f3-940747dfb910\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:17:31 crc kubenswrapper[5052]: I1203 12:17:31.945338 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/e51b5737-6907-4db8-a9a6-1019da0e64d2-config-data\") pod \"glance-default-internal-api-0\" (UID: \"e51b5737-6907-4db8-a9a6-1019da0e64d2\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:17:31 crc kubenswrapper[5052]: I1203 12:17:31.945356 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/408d4a98-4cc6-4a13-a2f3-940747dfb910-httpd-run\") pod \"glance-default-external-api-0\" (UID: \"408d4a98-4cc6-4a13-a2f3-940747dfb910\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:17:31 crc kubenswrapper[5052]: I1203 12:17:31.945397 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/e51b5737-6907-4db8-a9a6-1019da0e64d2-scripts\") pod \"glance-default-internal-api-0\" (UID: \"e51b5737-6907-4db8-a9a6-1019da0e64d2\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:17:31 crc kubenswrapper[5052]: I1203 12:17:31.945417 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage17-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage17-crc\") pod \"glance-default-internal-api-0\" (UID: \"e51b5737-6907-4db8-a9a6-1019da0e64d2\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:17:31 crc kubenswrapper[5052]: I1203 12:17:31.945434 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/408d4a98-4cc6-4a13-a2f3-940747dfb910-scripts\") pod \"glance-default-external-api-0\" (UID: \"408d4a98-4cc6-4a13-a2f3-940747dfb910\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:17:31 crc kubenswrapper[5052]: I1203 12:17:31.945448 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v979t\" (UniqueName: \"kubernetes.io/projected/e51b5737-6907-4db8-a9a6-1019da0e64d2-kube-api-access-v979t\") pod \"glance-default-internal-api-0\" (UID: \"e51b5737-6907-4db8-a9a6-1019da0e64d2\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:17:31 crc kubenswrapper[5052]: I1203 12:17:31.946112 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage09-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage09-crc\") pod \"glance-default-external-api-0\" (UID: \"408d4a98-4cc6-4a13-a2f3-940747dfb910\") device mount path \"/mnt/openstack/pv09\"" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:17:31 crc kubenswrapper[5052]: I1203 12:17:31.946266 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/408d4a98-4cc6-4a13-a2f3-940747dfb910-logs\") pod \"glance-default-external-api-0\" (UID: \"408d4a98-4cc6-4a13-a2f3-940747dfb910\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:17:31 crc kubenswrapper[5052]: I1203 12:17:31.946278 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/408d4a98-4cc6-4a13-a2f3-940747dfb910-httpd-run\") pod \"glance-default-external-api-0\" (UID: \"408d4a98-4cc6-4a13-a2f3-940747dfb910\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:17:31 crc kubenswrapper[5052]: I1203 12:17:31.950166 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/408d4a98-4cc6-4a13-a2f3-940747dfb910-config-data\") pod \"glance-default-external-api-0\" (UID: \"408d4a98-4cc6-4a13-a2f3-940747dfb910\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:17:31 crc kubenswrapper[5052]: I1203 12:17:31.952528 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/408d4a98-4cc6-4a13-a2f3-940747dfb910-public-tls-certs\") pod \"glance-default-external-api-0\" (UID: \"408d4a98-4cc6-4a13-a2f3-940747dfb910\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:17:31 crc kubenswrapper[5052]: I1203 12:17:31.952664 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/408d4a98-4cc6-4a13-a2f3-940747dfb910-scripts\") pod \"glance-default-external-api-0\" (UID: \"408d4a98-4cc6-4a13-a2f3-940747dfb910\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:17:31 crc kubenswrapper[5052]: I1203 12:17:31.953436 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/408d4a98-4cc6-4a13-a2f3-940747dfb910-combined-ca-bundle\") pod \"glance-default-external-api-0\" (UID: \"408d4a98-4cc6-4a13-a2f3-940747dfb910\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:17:31 crc kubenswrapper[5052]: I1203 12:17:31.963635 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-fvlmk\" (UniqueName: \"kubernetes.io/projected/408d4a98-4cc6-4a13-a2f3-940747dfb910-kube-api-access-fvlmk\") pod \"glance-default-external-api-0\" (UID: \"408d4a98-4cc6-4a13-a2f3-940747dfb910\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:17:31 crc kubenswrapper[5052]: I1203 12:17:31.985396 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage09-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage09-crc\") pod \"glance-default-external-api-0\" (UID: \"408d4a98-4cc6-4a13-a2f3-940747dfb910\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:17:32 crc kubenswrapper[5052]: I1203 12:17:32.046812 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/e51b5737-6907-4db8-a9a6-1019da0e64d2-internal-tls-certs\") pod \"glance-default-internal-api-0\" (UID: \"e51b5737-6907-4db8-a9a6-1019da0e64d2\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:17:32 crc kubenswrapper[5052]: I1203 12:17:32.046877 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/e51b5737-6907-4db8-a9a6-1019da0e64d2-logs\") pod \"glance-default-internal-api-0\" (UID: \"e51b5737-6907-4db8-a9a6-1019da0e64d2\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:17:32 crc kubenswrapper[5052]: I1203 12:17:32.046923 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/e51b5737-6907-4db8-a9a6-1019da0e64d2-httpd-run\") pod \"glance-default-internal-api-0\" (UID: \"e51b5737-6907-4db8-a9a6-1019da0e64d2\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:17:32 crc kubenswrapper[5052]: I1203 12:17:32.046978 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e51b5737-6907-4db8-a9a6-1019da0e64d2-combined-ca-bundle\") pod \"glance-default-internal-api-0\" (UID: \"e51b5737-6907-4db8-a9a6-1019da0e64d2\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:17:32 crc kubenswrapper[5052]: I1203 12:17:32.047043 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/e51b5737-6907-4db8-a9a6-1019da0e64d2-config-data\") pod \"glance-default-internal-api-0\" (UID: \"e51b5737-6907-4db8-a9a6-1019da0e64d2\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:17:32 crc kubenswrapper[5052]: I1203 12:17:32.047120 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/e51b5737-6907-4db8-a9a6-1019da0e64d2-scripts\") pod \"glance-default-internal-api-0\" (UID: \"e51b5737-6907-4db8-a9a6-1019da0e64d2\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:17:32 crc kubenswrapper[5052]: I1203 12:17:32.047162 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage17-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage17-crc\") pod \"glance-default-internal-api-0\" (UID: \"e51b5737-6907-4db8-a9a6-1019da0e64d2\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:17:32 crc kubenswrapper[5052]: I1203 12:17:32.047191 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-v979t\" (UniqueName: \"kubernetes.io/projected/e51b5737-6907-4db8-a9a6-1019da0e64d2-kube-api-access-v979t\") pod \"glance-default-internal-api-0\" (UID: \"e51b5737-6907-4db8-a9a6-1019da0e64d2\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:17:32 crc kubenswrapper[5052]: I1203 12:17:32.047654 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/e51b5737-6907-4db8-a9a6-1019da0e64d2-logs\") pod \"glance-default-internal-api-0\" (UID: \"e51b5737-6907-4db8-a9a6-1019da0e64d2\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:17:32 crc kubenswrapper[5052]: I1203 12:17:32.047771 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/e51b5737-6907-4db8-a9a6-1019da0e64d2-httpd-run\") pod \"glance-default-internal-api-0\" (UID: \"e51b5737-6907-4db8-a9a6-1019da0e64d2\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:17:32 crc kubenswrapper[5052]: I1203 12:17:32.047814 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage17-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage17-crc\") pod \"glance-default-internal-api-0\" (UID: \"e51b5737-6907-4db8-a9a6-1019da0e64d2\") device mount path \"/mnt/openstack/pv17\"" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:17:32 crc kubenswrapper[5052]: I1203 12:17:32.054260 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e51b5737-6907-4db8-a9a6-1019da0e64d2-combined-ca-bundle\") pod \"glance-default-internal-api-0\" (UID: \"e51b5737-6907-4db8-a9a6-1019da0e64d2\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:17:32 crc kubenswrapper[5052]: I1203 12:17:32.055027 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/e51b5737-6907-4db8-a9a6-1019da0e64d2-scripts\") pod \"glance-default-internal-api-0\" (UID: \"e51b5737-6907-4db8-a9a6-1019da0e64d2\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:17:32 crc kubenswrapper[5052]: I1203 12:17:32.066501 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/e51b5737-6907-4db8-a9a6-1019da0e64d2-internal-tls-certs\") pod \"glance-default-internal-api-0\" (UID: \"e51b5737-6907-4db8-a9a6-1019da0e64d2\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:17:32 crc kubenswrapper[5052]: W1203 12:17:32.067389 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9f563ce4_a788_44eb_bdab_9fc4c3423176.slice/crio-e1a5483ee61db6af9e8058349f002642831d4d2623b7c0ded4999b576d55b614 WatchSource:0}: Error finding container e1a5483ee61db6af9e8058349f002642831d4d2623b7c0ded4999b576d55b614: Status 404 returned error can't find the container with id e1a5483ee61db6af9e8058349f002642831d4d2623b7c0ded4999b576d55b614 Dec 03 12:17:32 crc kubenswrapper[5052]: I1203 12:17:32.068300 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/placement-db-sync-hh2rh"] Dec 03 12:17:32 crc kubenswrapper[5052]: I1203 12:17:32.069273 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/e51b5737-6907-4db8-a9a6-1019da0e64d2-config-data\") pod \"glance-default-internal-api-0\" (UID: \"e51b5737-6907-4db8-a9a6-1019da0e64d2\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:17:32 crc kubenswrapper[5052]: I1203 12:17:32.075697 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-v979t\" (UniqueName: \"kubernetes.io/projected/e51b5737-6907-4db8-a9a6-1019da0e64d2-kube-api-access-v979t\") pod \"glance-default-internal-api-0\" (UID: \"e51b5737-6907-4db8-a9a6-1019da0e64d2\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:17:32 crc kubenswrapper[5052]: I1203 12:17:32.091140 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage17-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage17-crc\") pod \"glance-default-internal-api-0\" (UID: \"e51b5737-6907-4db8-a9a6-1019da0e64d2\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:17:32 crc kubenswrapper[5052]: I1203 12:17:32.144177 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:17:32 crc kubenswrapper[5052]: I1203 12:17:32.167555 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:17:32 crc kubenswrapper[5052]: I1203 12:17:32.522347 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-db-sync-pnhqf" event={"ID":"2f98071b-1e7d-4cf3-976e-ad9073a2f914","Type":"ContainerStarted","Data":"518459180ad34e42e3d4890b691c685fdb215176ae41fec275f3e19b1b7907ae"} Dec 03 12:17:32 crc kubenswrapper[5052]: I1203 12:17:32.522685 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-db-sync-pnhqf" event={"ID":"2f98071b-1e7d-4cf3-976e-ad9073a2f914","Type":"ContainerStarted","Data":"ed66b602b37675ef4806ee0a6e812de42767f2a4c5c1228a1e553b32060605b4"} Dec 03 12:17:32 crc kubenswrapper[5052]: I1203 12:17:32.525651 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"614c796f-690e-4c8e-833f-bf7ba7c47d1a","Type":"ContainerStarted","Data":"baec3061547d2412c64a59a314db756ac34d9a235835cd5adbe0fdfa0faf5bbf"} Dec 03 12:17:32 crc kubenswrapper[5052]: I1203 12:17:32.533767 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-db-sync-kgtt9" event={"ID":"cb69fca5-361d-4273-bee8-27c54648309c","Type":"ContainerStarted","Data":"0e69c8cda10283dde61afaf75a7b595710a8bdadd8fdf004e7e5e8d3fa479050"} Dec 03 12:17:32 crc kubenswrapper[5052]: I1203 12:17:32.533821 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-db-sync-kgtt9" event={"ID":"cb69fca5-361d-4273-bee8-27c54648309c","Type":"ContainerStarted","Data":"37723aa45e613d2653462e3c8d41df055a7a9bbe5732538efa703b6871bd789e"} Dec 03 12:17:32 crc kubenswrapper[5052]: I1203 12:17:32.538151 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-db-sync-hh2rh" event={"ID":"9f563ce4-a788-44eb-bdab-9fc4c3423176","Type":"ContainerStarted","Data":"815f97f7cb31b7b78a79807aa95002614e45122852e5f05422072d06574a839a"} Dec 03 12:17:32 crc kubenswrapper[5052]: I1203 12:17:32.538200 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-db-sync-hh2rh" event={"ID":"9f563ce4-a788-44eb-bdab-9fc4c3423176","Type":"ContainerStarted","Data":"e1a5483ee61db6af9e8058349f002642831d4d2623b7c0ded4999b576d55b614"} Dec 03 12:17:32 crc kubenswrapper[5052]: I1203 12:17:32.561497 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-bootstrap-5qw5w" event={"ID":"38a5d722-dbe5-4d1e-9aec-bf6513db82fb","Type":"ContainerStarted","Data":"1cfce437c4e384a34c57bced804c64f40f25e45c8f65ee39ad0277b7e37ec7f0"} Dec 03 12:17:32 crc kubenswrapper[5052]: I1203 12:17:32.561597 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-bootstrap-5qw5w" event={"ID":"38a5d722-dbe5-4d1e-9aec-bf6513db82fb","Type":"ContainerStarted","Data":"c3c0ae4cd1db927c0fa2fc6e45fe8e973f1dae47b5bad44a95215a9c11ce0c61"} Dec 03 12:17:32 crc kubenswrapper[5052]: I1203 12:17:32.624355 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/barbican-db-sync-pnhqf" podStartSLOduration=2.624338903 podStartE2EDuration="2.624338903s" podCreationTimestamp="2025-12-03 12:17:30 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:17:32.551314839 +0000 UTC m=+3443.139556603" watchObservedRunningTime="2025-12-03 12:17:32.624338903 +0000 UTC m=+3443.212580657" Dec 03 12:17:32 crc kubenswrapper[5052]: I1203 12:17:32.667530 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/placement-db-sync-hh2rh" podStartSLOduration=2.6674801649999997 podStartE2EDuration="2.667480165s" podCreationTimestamp="2025-12-03 12:17:30 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:17:32.624051455 +0000 UTC m=+3443.212293219" watchObservedRunningTime="2025-12-03 12:17:32.667480165 +0000 UTC m=+3443.255721939" Dec 03 12:17:32 crc kubenswrapper[5052]: W1203 12:17:32.698716 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod408d4a98_4cc6_4a13_a2f3_940747dfb910.slice/crio-5ab9ad5b949bcad9a1e2a7995184951af8606e4811a924968d05980a82c461e4 WatchSource:0}: Error finding container 5ab9ad5b949bcad9a1e2a7995184951af8606e4811a924968d05980a82c461e4: Status 404 returned error can't find the container with id 5ab9ad5b949bcad9a1e2a7995184951af8606e4811a924968d05980a82c461e4 Dec 03 12:17:32 crc kubenswrapper[5052]: I1203 12:17:32.716349 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:17:32 crc kubenswrapper[5052]: I1203 12:17:32.719884 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/neutron-db-sync-kgtt9" podStartSLOduration=2.71985952 podStartE2EDuration="2.71985952s" podCreationTimestamp="2025-12-03 12:17:30 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:17:32.658249001 +0000 UTC m=+3443.246490785" watchObservedRunningTime="2025-12-03 12:17:32.71985952 +0000 UTC m=+3443.308101284" Dec 03 12:17:32 crc kubenswrapper[5052]: I1203 12:17:32.734717 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/keystone-bootstrap-5qw5w" podStartSLOduration=2.734693263 podStartE2EDuration="2.734693263s" podCreationTimestamp="2025-12-03 12:17:30 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:17:32.684947643 +0000 UTC m=+3443.273189407" watchObservedRunningTime="2025-12-03 12:17:32.734693263 +0000 UTC m=+3443.322935047" Dec 03 12:17:32 crc kubenswrapper[5052]: I1203 12:17:32.775681 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:17:33 crc kubenswrapper[5052]: I1203 12:17:33.009833 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:17:33 crc kubenswrapper[5052]: I1203 12:17:33.052138 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:17:33 crc kubenswrapper[5052]: I1203 12:17:33.128874 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:17:33 crc kubenswrapper[5052]: I1203 12:17:33.578502 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"e51b5737-6907-4db8-a9a6-1019da0e64d2","Type":"ContainerStarted","Data":"6695052548a2eb4feacbdcfb8ed6828bbe4bdbb48ef281893bc6842868d4ed6d"} Dec 03 12:17:33 crc kubenswrapper[5052]: I1203 12:17:33.589576 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"408d4a98-4cc6-4a13-a2f3-940747dfb910","Type":"ContainerStarted","Data":"a18bca6774b9cb4d63b87a8d0ab0c88739a06f3003f911475911f5498b1e1501"} Dec 03 12:17:33 crc kubenswrapper[5052]: I1203 12:17:33.589626 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"408d4a98-4cc6-4a13-a2f3-940747dfb910","Type":"ContainerStarted","Data":"5ab9ad5b949bcad9a1e2a7995184951af8606e4811a924968d05980a82c461e4"} Dec 03 12:17:33 crc kubenswrapper[5052]: I1203 12:17:33.607049 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"614c796f-690e-4c8e-833f-bf7ba7c47d1a","Type":"ContainerStarted","Data":"ecb8cd2f428a35b04fce42b144a6f3606e98b3dcaa31eb3623b92627a69a0813"} Dec 03 12:17:33 crc kubenswrapper[5052]: I1203 12:17:33.607090 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"614c796f-690e-4c8e-833f-bf7ba7c47d1a","Type":"ContainerStarted","Data":"d26b41a42104863894f831fd9b945032f3ccd5ce0630c9e4673313dfb9e41580"} Dec 03 12:17:33 crc kubenswrapper[5052]: I1203 12:17:33.632999 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/cinder-db-sync-wp26m"] Dec 03 12:17:33 crc kubenswrapper[5052]: I1203 12:17:33.634137 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-db-sync-wp26m" Dec 03 12:17:33 crc kubenswrapper[5052]: I1203 12:17:33.638122 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cinder-cinder-dockercfg-znp6c" Dec 03 12:17:33 crc kubenswrapper[5052]: I1203 12:17:33.640092 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cinder-scripts" Dec 03 12:17:33 crc kubenswrapper[5052]: I1203 12:17:33.640173 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cinder-config-data" Dec 03 12:17:33 crc kubenswrapper[5052]: I1203 12:17:33.663263 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-db-sync-wp26m"] Dec 03 12:17:33 crc kubenswrapper[5052]: I1203 12:17:33.702917 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/e5091121-ce68-4b86-baae-2fbd5b22d537-config-data\") pod \"cinder-db-sync-wp26m\" (UID: \"e5091121-ce68-4b86-baae-2fbd5b22d537\") " pod="openstack-kuttl-tests/cinder-db-sync-wp26m" Dec 03 12:17:33 crc kubenswrapper[5052]: I1203 12:17:33.702998 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/e5091121-ce68-4b86-baae-2fbd5b22d537-etc-machine-id\") pod \"cinder-db-sync-wp26m\" (UID: \"e5091121-ce68-4b86-baae-2fbd5b22d537\") " pod="openstack-kuttl-tests/cinder-db-sync-wp26m" Dec 03 12:17:33 crc kubenswrapper[5052]: I1203 12:17:33.703017 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e5091121-ce68-4b86-baae-2fbd5b22d537-combined-ca-bundle\") pod \"cinder-db-sync-wp26m\" (UID: \"e5091121-ce68-4b86-baae-2fbd5b22d537\") " pod="openstack-kuttl-tests/cinder-db-sync-wp26m" Dec 03 12:17:33 crc kubenswrapper[5052]: I1203 12:17:33.703125 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5qtdl\" (UniqueName: \"kubernetes.io/projected/e5091121-ce68-4b86-baae-2fbd5b22d537-kube-api-access-5qtdl\") pod \"cinder-db-sync-wp26m\" (UID: \"e5091121-ce68-4b86-baae-2fbd5b22d537\") " pod="openstack-kuttl-tests/cinder-db-sync-wp26m" Dec 03 12:17:33 crc kubenswrapper[5052]: I1203 12:17:33.703150 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/e5091121-ce68-4b86-baae-2fbd5b22d537-scripts\") pod \"cinder-db-sync-wp26m\" (UID: \"e5091121-ce68-4b86-baae-2fbd5b22d537\") " pod="openstack-kuttl-tests/cinder-db-sync-wp26m" Dec 03 12:17:33 crc kubenswrapper[5052]: I1203 12:17:33.703215 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/e5091121-ce68-4b86-baae-2fbd5b22d537-db-sync-config-data\") pod \"cinder-db-sync-wp26m\" (UID: \"e5091121-ce68-4b86-baae-2fbd5b22d537\") " pod="openstack-kuttl-tests/cinder-db-sync-wp26m" Dec 03 12:17:33 crc kubenswrapper[5052]: I1203 12:17:33.804361 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e5091121-ce68-4b86-baae-2fbd5b22d537-combined-ca-bundle\") pod \"cinder-db-sync-wp26m\" (UID: \"e5091121-ce68-4b86-baae-2fbd5b22d537\") " pod="openstack-kuttl-tests/cinder-db-sync-wp26m" Dec 03 12:17:33 crc kubenswrapper[5052]: I1203 12:17:33.804447 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-5qtdl\" (UniqueName: \"kubernetes.io/projected/e5091121-ce68-4b86-baae-2fbd5b22d537-kube-api-access-5qtdl\") pod \"cinder-db-sync-wp26m\" (UID: \"e5091121-ce68-4b86-baae-2fbd5b22d537\") " pod="openstack-kuttl-tests/cinder-db-sync-wp26m" Dec 03 12:17:33 crc kubenswrapper[5052]: I1203 12:17:33.804474 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/e5091121-ce68-4b86-baae-2fbd5b22d537-scripts\") pod \"cinder-db-sync-wp26m\" (UID: \"e5091121-ce68-4b86-baae-2fbd5b22d537\") " pod="openstack-kuttl-tests/cinder-db-sync-wp26m" Dec 03 12:17:33 crc kubenswrapper[5052]: I1203 12:17:33.804563 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/e5091121-ce68-4b86-baae-2fbd5b22d537-db-sync-config-data\") pod \"cinder-db-sync-wp26m\" (UID: \"e5091121-ce68-4b86-baae-2fbd5b22d537\") " pod="openstack-kuttl-tests/cinder-db-sync-wp26m" Dec 03 12:17:33 crc kubenswrapper[5052]: I1203 12:17:33.804648 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/e5091121-ce68-4b86-baae-2fbd5b22d537-config-data\") pod \"cinder-db-sync-wp26m\" (UID: \"e5091121-ce68-4b86-baae-2fbd5b22d537\") " pod="openstack-kuttl-tests/cinder-db-sync-wp26m" Dec 03 12:17:33 crc kubenswrapper[5052]: I1203 12:17:33.804697 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/e5091121-ce68-4b86-baae-2fbd5b22d537-etc-machine-id\") pod \"cinder-db-sync-wp26m\" (UID: \"e5091121-ce68-4b86-baae-2fbd5b22d537\") " pod="openstack-kuttl-tests/cinder-db-sync-wp26m" Dec 03 12:17:33 crc kubenswrapper[5052]: I1203 12:17:33.804779 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/e5091121-ce68-4b86-baae-2fbd5b22d537-etc-machine-id\") pod \"cinder-db-sync-wp26m\" (UID: \"e5091121-ce68-4b86-baae-2fbd5b22d537\") " pod="openstack-kuttl-tests/cinder-db-sync-wp26m" Dec 03 12:17:33 crc kubenswrapper[5052]: I1203 12:17:33.811870 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/e5091121-ce68-4b86-baae-2fbd5b22d537-config-data\") pod \"cinder-db-sync-wp26m\" (UID: \"e5091121-ce68-4b86-baae-2fbd5b22d537\") " pod="openstack-kuttl-tests/cinder-db-sync-wp26m" Dec 03 12:17:33 crc kubenswrapper[5052]: I1203 12:17:33.815187 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e5091121-ce68-4b86-baae-2fbd5b22d537-combined-ca-bundle\") pod \"cinder-db-sync-wp26m\" (UID: \"e5091121-ce68-4b86-baae-2fbd5b22d537\") " pod="openstack-kuttl-tests/cinder-db-sync-wp26m" Dec 03 12:17:33 crc kubenswrapper[5052]: I1203 12:17:33.819894 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/e5091121-ce68-4b86-baae-2fbd5b22d537-scripts\") pod \"cinder-db-sync-wp26m\" (UID: \"e5091121-ce68-4b86-baae-2fbd5b22d537\") " pod="openstack-kuttl-tests/cinder-db-sync-wp26m" Dec 03 12:17:33 crc kubenswrapper[5052]: I1203 12:17:33.827063 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-5qtdl\" (UniqueName: \"kubernetes.io/projected/e5091121-ce68-4b86-baae-2fbd5b22d537-kube-api-access-5qtdl\") pod \"cinder-db-sync-wp26m\" (UID: \"e5091121-ce68-4b86-baae-2fbd5b22d537\") " pod="openstack-kuttl-tests/cinder-db-sync-wp26m" Dec 03 12:17:33 crc kubenswrapper[5052]: I1203 12:17:33.827167 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/e5091121-ce68-4b86-baae-2fbd5b22d537-db-sync-config-data\") pod \"cinder-db-sync-wp26m\" (UID: \"e5091121-ce68-4b86-baae-2fbd5b22d537\") " pod="openstack-kuttl-tests/cinder-db-sync-wp26m" Dec 03 12:17:34 crc kubenswrapper[5052]: I1203 12:17:34.036087 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-db-sync-wp26m" Dec 03 12:17:34 crc kubenswrapper[5052]: I1203 12:17:34.603834 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-db-sync-wp26m"] Dec 03 12:17:34 crc kubenswrapper[5052]: W1203 12:17:34.619882 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode5091121_ce68_4b86_baae_2fbd5b22d537.slice/crio-ba9b87636c0720bfc27c02fade6459540a85cd2254622312ecec3919fdf9857a WatchSource:0}: Error finding container ba9b87636c0720bfc27c02fade6459540a85cd2254622312ecec3919fdf9857a: Status 404 returned error can't find the container with id ba9b87636c0720bfc27c02fade6459540a85cd2254622312ecec3919fdf9857a Dec 03 12:17:34 crc kubenswrapper[5052]: I1203 12:17:34.682821 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"e51b5737-6907-4db8-a9a6-1019da0e64d2","Type":"ContainerStarted","Data":"fe340f30c1795c46a94d71508d1527680e9438f839d1587b7040eab22d8367f5"} Dec 03 12:17:34 crc kubenswrapper[5052]: I1203 12:17:34.686905 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"408d4a98-4cc6-4a13-a2f3-940747dfb910","Type":"ContainerStarted","Data":"0d95395b91d1fb7847abdd91e269f7b3149789f5d1f1b4daa4b55b65a0c39ac1"} Dec 03 12:17:34 crc kubenswrapper[5052]: I1203 12:17:34.687103 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-external-api-0" podUID="408d4a98-4cc6-4a13-a2f3-940747dfb910" containerName="glance-log" containerID="cri-o://a18bca6774b9cb4d63b87a8d0ab0c88739a06f3003f911475911f5498b1e1501" gracePeriod=30 Dec 03 12:17:34 crc kubenswrapper[5052]: I1203 12:17:34.687759 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-external-api-0" podUID="408d4a98-4cc6-4a13-a2f3-940747dfb910" containerName="glance-httpd" containerID="cri-o://0d95395b91d1fb7847abdd91e269f7b3149789f5d1f1b4daa4b55b65a0c39ac1" gracePeriod=30 Dec 03 12:17:34 crc kubenswrapper[5052]: I1203 12:17:34.697195 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"614c796f-690e-4c8e-833f-bf7ba7c47d1a","Type":"ContainerStarted","Data":"c8beb7f8c222fb4c94ee6aadf2ded6eb09f46f942e334eb694d5a1683fcf80f9"} Dec 03 12:17:34 crc kubenswrapper[5052]: I1203 12:17:34.717107 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/glance-default-external-api-0" podStartSLOduration=4.717080345 podStartE2EDuration="4.717080345s" podCreationTimestamp="2025-12-03 12:17:30 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:17:34.715235193 +0000 UTC m=+3445.303476957" watchObservedRunningTime="2025-12-03 12:17:34.717080345 +0000 UTC m=+3445.305322109" Dec 03 12:17:35 crc kubenswrapper[5052]: I1203 12:17:35.208121 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:17:35 crc kubenswrapper[5052]: I1203 12:17:35.347097 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/408d4a98-4cc6-4a13-a2f3-940747dfb910-combined-ca-bundle\") pod \"408d4a98-4cc6-4a13-a2f3-940747dfb910\" (UID: \"408d4a98-4cc6-4a13-a2f3-940747dfb910\") " Dec 03 12:17:35 crc kubenswrapper[5052]: I1203 12:17:35.347178 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"glance\" (UniqueName: \"kubernetes.io/local-volume/local-storage09-crc\") pod \"408d4a98-4cc6-4a13-a2f3-940747dfb910\" (UID: \"408d4a98-4cc6-4a13-a2f3-940747dfb910\") " Dec 03 12:17:35 crc kubenswrapper[5052]: I1203 12:17:35.347203 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fvlmk\" (UniqueName: \"kubernetes.io/projected/408d4a98-4cc6-4a13-a2f3-940747dfb910-kube-api-access-fvlmk\") pod \"408d4a98-4cc6-4a13-a2f3-940747dfb910\" (UID: \"408d4a98-4cc6-4a13-a2f3-940747dfb910\") " Dec 03 12:17:35 crc kubenswrapper[5052]: I1203 12:17:35.347251 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/408d4a98-4cc6-4a13-a2f3-940747dfb910-logs\") pod \"408d4a98-4cc6-4a13-a2f3-940747dfb910\" (UID: \"408d4a98-4cc6-4a13-a2f3-940747dfb910\") " Dec 03 12:17:35 crc kubenswrapper[5052]: I1203 12:17:35.347463 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/408d4a98-4cc6-4a13-a2f3-940747dfb910-scripts\") pod \"408d4a98-4cc6-4a13-a2f3-940747dfb910\" (UID: \"408d4a98-4cc6-4a13-a2f3-940747dfb910\") " Dec 03 12:17:35 crc kubenswrapper[5052]: I1203 12:17:35.347589 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/408d4a98-4cc6-4a13-a2f3-940747dfb910-public-tls-certs\") pod \"408d4a98-4cc6-4a13-a2f3-940747dfb910\" (UID: \"408d4a98-4cc6-4a13-a2f3-940747dfb910\") " Dec 03 12:17:35 crc kubenswrapper[5052]: I1203 12:17:35.347679 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/408d4a98-4cc6-4a13-a2f3-940747dfb910-config-data\") pod \"408d4a98-4cc6-4a13-a2f3-940747dfb910\" (UID: \"408d4a98-4cc6-4a13-a2f3-940747dfb910\") " Dec 03 12:17:35 crc kubenswrapper[5052]: I1203 12:17:35.347707 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/408d4a98-4cc6-4a13-a2f3-940747dfb910-httpd-run\") pod \"408d4a98-4cc6-4a13-a2f3-940747dfb910\" (UID: \"408d4a98-4cc6-4a13-a2f3-940747dfb910\") " Dec 03 12:17:35 crc kubenswrapper[5052]: I1203 12:17:35.349647 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/408d4a98-4cc6-4a13-a2f3-940747dfb910-httpd-run" (OuterVolumeSpecName: "httpd-run") pod "408d4a98-4cc6-4a13-a2f3-940747dfb910" (UID: "408d4a98-4cc6-4a13-a2f3-940747dfb910"). InnerVolumeSpecName "httpd-run". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:17:35 crc kubenswrapper[5052]: I1203 12:17:35.351637 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/408d4a98-4cc6-4a13-a2f3-940747dfb910-logs" (OuterVolumeSpecName: "logs") pod "408d4a98-4cc6-4a13-a2f3-940747dfb910" (UID: "408d4a98-4cc6-4a13-a2f3-940747dfb910"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:17:35 crc kubenswrapper[5052]: I1203 12:17:35.380564 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/408d4a98-4cc6-4a13-a2f3-940747dfb910-scripts" (OuterVolumeSpecName: "scripts") pod "408d4a98-4cc6-4a13-a2f3-940747dfb910" (UID: "408d4a98-4cc6-4a13-a2f3-940747dfb910"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:17:35 crc kubenswrapper[5052]: I1203 12:17:35.380596 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage09-crc" (OuterVolumeSpecName: "glance") pod "408d4a98-4cc6-4a13-a2f3-940747dfb910" (UID: "408d4a98-4cc6-4a13-a2f3-940747dfb910"). InnerVolumeSpecName "local-storage09-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 12:17:35 crc kubenswrapper[5052]: I1203 12:17:35.380784 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/408d4a98-4cc6-4a13-a2f3-940747dfb910-kube-api-access-fvlmk" (OuterVolumeSpecName: "kube-api-access-fvlmk") pod "408d4a98-4cc6-4a13-a2f3-940747dfb910" (UID: "408d4a98-4cc6-4a13-a2f3-940747dfb910"). InnerVolumeSpecName "kube-api-access-fvlmk". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:17:35 crc kubenswrapper[5052]: I1203 12:17:35.393314 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/408d4a98-4cc6-4a13-a2f3-940747dfb910-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "408d4a98-4cc6-4a13-a2f3-940747dfb910" (UID: "408d4a98-4cc6-4a13-a2f3-940747dfb910"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:17:35 crc kubenswrapper[5052]: I1203 12:17:35.438286 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/408d4a98-4cc6-4a13-a2f3-940747dfb910-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "408d4a98-4cc6-4a13-a2f3-940747dfb910" (UID: "408d4a98-4cc6-4a13-a2f3-940747dfb910"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:17:35 crc kubenswrapper[5052]: I1203 12:17:35.443492 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/408d4a98-4cc6-4a13-a2f3-940747dfb910-config-data" (OuterVolumeSpecName: "config-data") pod "408d4a98-4cc6-4a13-a2f3-940747dfb910" (UID: "408d4a98-4cc6-4a13-a2f3-940747dfb910"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:17:35 crc kubenswrapper[5052]: I1203 12:17:35.449834 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/408d4a98-4cc6-4a13-a2f3-940747dfb910-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:17:35 crc kubenswrapper[5052]: I1203 12:17:35.449874 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/408d4a98-4cc6-4a13-a2f3-940747dfb910-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:17:35 crc kubenswrapper[5052]: I1203 12:17:35.449887 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/408d4a98-4cc6-4a13-a2f3-940747dfb910-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:17:35 crc kubenswrapper[5052]: I1203 12:17:35.449899 5052 reconciler_common.go:293] "Volume detached for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/408d4a98-4cc6-4a13-a2f3-940747dfb910-httpd-run\") on node \"crc\" DevicePath \"\"" Dec 03 12:17:35 crc kubenswrapper[5052]: I1203 12:17:35.449910 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/408d4a98-4cc6-4a13-a2f3-940747dfb910-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:17:35 crc kubenswrapper[5052]: I1203 12:17:35.450016 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage09-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage09-crc\") on node \"crc\" " Dec 03 12:17:35 crc kubenswrapper[5052]: I1203 12:17:35.450058 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-fvlmk\" (UniqueName: \"kubernetes.io/projected/408d4a98-4cc6-4a13-a2f3-940747dfb910-kube-api-access-fvlmk\") on node \"crc\" DevicePath \"\"" Dec 03 12:17:35 crc kubenswrapper[5052]: I1203 12:17:35.450091 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/408d4a98-4cc6-4a13-a2f3-940747dfb910-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:17:35 crc kubenswrapper[5052]: I1203 12:17:35.472363 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage09-crc" (UniqueName: "kubernetes.io/local-volume/local-storage09-crc") on node "crc" Dec 03 12:17:35 crc kubenswrapper[5052]: I1203 12:17:35.555603 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage09-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage09-crc\") on node \"crc\" DevicePath \"\"" Dec 03 12:17:35 crc kubenswrapper[5052]: I1203 12:17:35.719310 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"e51b5737-6907-4db8-a9a6-1019da0e64d2","Type":"ContainerStarted","Data":"9f6ab18d411ec6d144b75557c3a32ac72e2d81183efdfb6a5aee9bdbf7117475"} Dec 03 12:17:35 crc kubenswrapper[5052]: I1203 12:17:35.719443 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-internal-api-0" podUID="e51b5737-6907-4db8-a9a6-1019da0e64d2" containerName="glance-httpd" containerID="cri-o://9f6ab18d411ec6d144b75557c3a32ac72e2d81183efdfb6a5aee9bdbf7117475" gracePeriod=30 Dec 03 12:17:35 crc kubenswrapper[5052]: I1203 12:17:35.719417 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-internal-api-0" podUID="e51b5737-6907-4db8-a9a6-1019da0e64d2" containerName="glance-log" containerID="cri-o://fe340f30c1795c46a94d71508d1527680e9438f839d1587b7040eab22d8367f5" gracePeriod=30 Dec 03 12:17:35 crc kubenswrapper[5052]: I1203 12:17:35.723685 5052 generic.go:334] "Generic (PLEG): container finished" podID="408d4a98-4cc6-4a13-a2f3-940747dfb910" containerID="0d95395b91d1fb7847abdd91e269f7b3149789f5d1f1b4daa4b55b65a0c39ac1" exitCode=143 Dec 03 12:17:35 crc kubenswrapper[5052]: I1203 12:17:35.723714 5052 generic.go:334] "Generic (PLEG): container finished" podID="408d4a98-4cc6-4a13-a2f3-940747dfb910" containerID="a18bca6774b9cb4d63b87a8d0ab0c88739a06f3003f911475911f5498b1e1501" exitCode=143 Dec 03 12:17:35 crc kubenswrapper[5052]: I1203 12:17:35.723764 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"408d4a98-4cc6-4a13-a2f3-940747dfb910","Type":"ContainerDied","Data":"0d95395b91d1fb7847abdd91e269f7b3149789f5d1f1b4daa4b55b65a0c39ac1"} Dec 03 12:17:35 crc kubenswrapper[5052]: I1203 12:17:35.723797 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"408d4a98-4cc6-4a13-a2f3-940747dfb910","Type":"ContainerDied","Data":"a18bca6774b9cb4d63b87a8d0ab0c88739a06f3003f911475911f5498b1e1501"} Dec 03 12:17:35 crc kubenswrapper[5052]: I1203 12:17:35.723810 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"408d4a98-4cc6-4a13-a2f3-940747dfb910","Type":"ContainerDied","Data":"5ab9ad5b949bcad9a1e2a7995184951af8606e4811a924968d05980a82c461e4"} Dec 03 12:17:35 crc kubenswrapper[5052]: I1203 12:17:35.723830 5052 scope.go:117] "RemoveContainer" containerID="0d95395b91d1fb7847abdd91e269f7b3149789f5d1f1b4daa4b55b65a0c39ac1" Dec 03 12:17:35 crc kubenswrapper[5052]: I1203 12:17:35.724013 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:17:35 crc kubenswrapper[5052]: I1203 12:17:35.733531 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-db-sync-wp26m" event={"ID":"e5091121-ce68-4b86-baae-2fbd5b22d537","Type":"ContainerStarted","Data":"7d8e7144eee1a253836215f98a2a034d3756074819c1433459b21b25e1261483"} Dec 03 12:17:35 crc kubenswrapper[5052]: I1203 12:17:35.733633 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-db-sync-wp26m" event={"ID":"e5091121-ce68-4b86-baae-2fbd5b22d537","Type":"ContainerStarted","Data":"ba9b87636c0720bfc27c02fade6459540a85cd2254622312ecec3919fdf9857a"} Dec 03 12:17:35 crc kubenswrapper[5052]: I1203 12:17:35.752869 5052 generic.go:334] "Generic (PLEG): container finished" podID="9f563ce4-a788-44eb-bdab-9fc4c3423176" containerID="815f97f7cb31b7b78a79807aa95002614e45122852e5f05422072d06574a839a" exitCode=0 Dec 03 12:17:35 crc kubenswrapper[5052]: I1203 12:17:35.752907 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-db-sync-hh2rh" event={"ID":"9f563ce4-a788-44eb-bdab-9fc4c3423176","Type":"ContainerDied","Data":"815f97f7cb31b7b78a79807aa95002614e45122852e5f05422072d06574a839a"} Dec 03 12:17:35 crc kubenswrapper[5052]: I1203 12:17:35.772142 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/glance-default-internal-api-0" podStartSLOduration=5.772120149 podStartE2EDuration="5.772120149s" podCreationTimestamp="2025-12-03 12:17:30 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:17:35.740410224 +0000 UTC m=+3446.328651988" watchObservedRunningTime="2025-12-03 12:17:35.772120149 +0000 UTC m=+3446.360361913" Dec 03 12:17:35 crc kubenswrapper[5052]: I1203 12:17:35.781608 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/cinder-db-sync-wp26m" podStartSLOduration=2.781590929 podStartE2EDuration="2.781590929s" podCreationTimestamp="2025-12-03 12:17:33 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:17:35.774761034 +0000 UTC m=+3446.363002818" watchObservedRunningTime="2025-12-03 12:17:35.781590929 +0000 UTC m=+3446.369832693" Dec 03 12:17:35 crc kubenswrapper[5052]: I1203 12:17:35.830959 5052 scope.go:117] "RemoveContainer" containerID="a18bca6774b9cb4d63b87a8d0ab0c88739a06f3003f911475911f5498b1e1501" Dec 03 12:17:35 crc kubenswrapper[5052]: I1203 12:17:35.909485 5052 scope.go:117] "RemoveContainer" containerID="0d95395b91d1fb7847abdd91e269f7b3149789f5d1f1b4daa4b55b65a0c39ac1" Dec 03 12:17:35 crc kubenswrapper[5052]: I1203 12:17:35.921565 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:17:35 crc kubenswrapper[5052]: E1203 12:17:35.921850 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0d95395b91d1fb7847abdd91e269f7b3149789f5d1f1b4daa4b55b65a0c39ac1\": container with ID starting with 0d95395b91d1fb7847abdd91e269f7b3149789f5d1f1b4daa4b55b65a0c39ac1 not found: ID does not exist" containerID="0d95395b91d1fb7847abdd91e269f7b3149789f5d1f1b4daa4b55b65a0c39ac1" Dec 03 12:17:35 crc kubenswrapper[5052]: I1203 12:17:35.921884 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0d95395b91d1fb7847abdd91e269f7b3149789f5d1f1b4daa4b55b65a0c39ac1"} err="failed to get container status \"0d95395b91d1fb7847abdd91e269f7b3149789f5d1f1b4daa4b55b65a0c39ac1\": rpc error: code = NotFound desc = could not find container \"0d95395b91d1fb7847abdd91e269f7b3149789f5d1f1b4daa4b55b65a0c39ac1\": container with ID starting with 0d95395b91d1fb7847abdd91e269f7b3149789f5d1f1b4daa4b55b65a0c39ac1 not found: ID does not exist" Dec 03 12:17:35 crc kubenswrapper[5052]: I1203 12:17:35.921907 5052 scope.go:117] "RemoveContainer" containerID="a18bca6774b9cb4d63b87a8d0ab0c88739a06f3003f911475911f5498b1e1501" Dec 03 12:17:35 crc kubenswrapper[5052]: E1203 12:17:35.924549 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a18bca6774b9cb4d63b87a8d0ab0c88739a06f3003f911475911f5498b1e1501\": container with ID starting with a18bca6774b9cb4d63b87a8d0ab0c88739a06f3003f911475911f5498b1e1501 not found: ID does not exist" containerID="a18bca6774b9cb4d63b87a8d0ab0c88739a06f3003f911475911f5498b1e1501" Dec 03 12:17:35 crc kubenswrapper[5052]: I1203 12:17:35.924576 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a18bca6774b9cb4d63b87a8d0ab0c88739a06f3003f911475911f5498b1e1501"} err="failed to get container status \"a18bca6774b9cb4d63b87a8d0ab0c88739a06f3003f911475911f5498b1e1501\": rpc error: code = NotFound desc = could not find container \"a18bca6774b9cb4d63b87a8d0ab0c88739a06f3003f911475911f5498b1e1501\": container with ID starting with a18bca6774b9cb4d63b87a8d0ab0c88739a06f3003f911475911f5498b1e1501 not found: ID does not exist" Dec 03 12:17:35 crc kubenswrapper[5052]: I1203 12:17:35.924598 5052 scope.go:117] "RemoveContainer" containerID="0d95395b91d1fb7847abdd91e269f7b3149789f5d1f1b4daa4b55b65a0c39ac1" Dec 03 12:17:35 crc kubenswrapper[5052]: I1203 12:17:35.925309 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0d95395b91d1fb7847abdd91e269f7b3149789f5d1f1b4daa4b55b65a0c39ac1"} err="failed to get container status \"0d95395b91d1fb7847abdd91e269f7b3149789f5d1f1b4daa4b55b65a0c39ac1\": rpc error: code = NotFound desc = could not find container \"0d95395b91d1fb7847abdd91e269f7b3149789f5d1f1b4daa4b55b65a0c39ac1\": container with ID starting with 0d95395b91d1fb7847abdd91e269f7b3149789f5d1f1b4daa4b55b65a0c39ac1 not found: ID does not exist" Dec 03 12:17:35 crc kubenswrapper[5052]: I1203 12:17:35.925329 5052 scope.go:117] "RemoveContainer" containerID="a18bca6774b9cb4d63b87a8d0ab0c88739a06f3003f911475911f5498b1e1501" Dec 03 12:17:35 crc kubenswrapper[5052]: I1203 12:17:35.925875 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a18bca6774b9cb4d63b87a8d0ab0c88739a06f3003f911475911f5498b1e1501"} err="failed to get container status \"a18bca6774b9cb4d63b87a8d0ab0c88739a06f3003f911475911f5498b1e1501\": rpc error: code = NotFound desc = could not find container \"a18bca6774b9cb4d63b87a8d0ab0c88739a06f3003f911475911f5498b1e1501\": container with ID starting with a18bca6774b9cb4d63b87a8d0ab0c88739a06f3003f911475911f5498b1e1501 not found: ID does not exist" Dec 03 12:17:35 crc kubenswrapper[5052]: I1203 12:17:35.945997 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:17:35 crc kubenswrapper[5052]: I1203 12:17:35.953457 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:17:35 crc kubenswrapper[5052]: E1203 12:17:35.953995 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="408d4a98-4cc6-4a13-a2f3-940747dfb910" containerName="glance-log" Dec 03 12:17:35 crc kubenswrapper[5052]: I1203 12:17:35.954014 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="408d4a98-4cc6-4a13-a2f3-940747dfb910" containerName="glance-log" Dec 03 12:17:35 crc kubenswrapper[5052]: E1203 12:17:35.954046 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="408d4a98-4cc6-4a13-a2f3-940747dfb910" containerName="glance-httpd" Dec 03 12:17:35 crc kubenswrapper[5052]: I1203 12:17:35.954055 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="408d4a98-4cc6-4a13-a2f3-940747dfb910" containerName="glance-httpd" Dec 03 12:17:35 crc kubenswrapper[5052]: I1203 12:17:35.954273 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="408d4a98-4cc6-4a13-a2f3-940747dfb910" containerName="glance-log" Dec 03 12:17:35 crc kubenswrapper[5052]: I1203 12:17:35.954310 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="408d4a98-4cc6-4a13-a2f3-940747dfb910" containerName="glance-httpd" Dec 03 12:17:35 crc kubenswrapper[5052]: I1203 12:17:35.955548 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:17:35 crc kubenswrapper[5052]: I1203 12:17:35.962224 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:17:35 crc kubenswrapper[5052]: I1203 12:17:35.964745 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-default-external-config-data" Dec 03 12:17:35 crc kubenswrapper[5052]: I1203 12:17:35.964935 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-glance-default-public-svc" Dec 03 12:17:36 crc kubenswrapper[5052]: I1203 12:17:36.070209 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lfv2c\" (UniqueName: \"kubernetes.io/projected/cd9a6dc3-a814-4a29-924a-d8ada756e7c7-kube-api-access-lfv2c\") pod \"glance-default-external-api-0\" (UID: \"cd9a6dc3-a814-4a29-924a-d8ada756e7c7\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:17:36 crc kubenswrapper[5052]: I1203 12:17:36.070271 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/cd9a6dc3-a814-4a29-924a-d8ada756e7c7-config-data\") pod \"glance-default-external-api-0\" (UID: \"cd9a6dc3-a814-4a29-924a-d8ada756e7c7\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:17:36 crc kubenswrapper[5052]: I1203 12:17:36.070338 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/cd9a6dc3-a814-4a29-924a-d8ada756e7c7-logs\") pod \"glance-default-external-api-0\" (UID: \"cd9a6dc3-a814-4a29-924a-d8ada756e7c7\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:17:36 crc kubenswrapper[5052]: I1203 12:17:36.070379 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/cd9a6dc3-a814-4a29-924a-d8ada756e7c7-httpd-run\") pod \"glance-default-external-api-0\" (UID: \"cd9a6dc3-a814-4a29-924a-d8ada756e7c7\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:17:36 crc kubenswrapper[5052]: I1203 12:17:36.070445 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/cd9a6dc3-a814-4a29-924a-d8ada756e7c7-scripts\") pod \"glance-default-external-api-0\" (UID: \"cd9a6dc3-a814-4a29-924a-d8ada756e7c7\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:17:36 crc kubenswrapper[5052]: I1203 12:17:36.070472 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage09-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage09-crc\") pod \"glance-default-external-api-0\" (UID: \"cd9a6dc3-a814-4a29-924a-d8ada756e7c7\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:17:36 crc kubenswrapper[5052]: I1203 12:17:36.070589 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/cd9a6dc3-a814-4a29-924a-d8ada756e7c7-public-tls-certs\") pod \"glance-default-external-api-0\" (UID: \"cd9a6dc3-a814-4a29-924a-d8ada756e7c7\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:17:36 crc kubenswrapper[5052]: I1203 12:17:36.070643 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/cd9a6dc3-a814-4a29-924a-d8ada756e7c7-combined-ca-bundle\") pod \"glance-default-external-api-0\" (UID: \"cd9a6dc3-a814-4a29-924a-d8ada756e7c7\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:17:36 crc kubenswrapper[5052]: I1203 12:17:36.172600 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/cd9a6dc3-a814-4a29-924a-d8ada756e7c7-logs\") pod \"glance-default-external-api-0\" (UID: \"cd9a6dc3-a814-4a29-924a-d8ada756e7c7\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:17:36 crc kubenswrapper[5052]: I1203 12:17:36.172653 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/cd9a6dc3-a814-4a29-924a-d8ada756e7c7-httpd-run\") pod \"glance-default-external-api-0\" (UID: \"cd9a6dc3-a814-4a29-924a-d8ada756e7c7\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:17:36 crc kubenswrapper[5052]: I1203 12:17:36.172690 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/cd9a6dc3-a814-4a29-924a-d8ada756e7c7-scripts\") pod \"glance-default-external-api-0\" (UID: \"cd9a6dc3-a814-4a29-924a-d8ada756e7c7\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:17:36 crc kubenswrapper[5052]: I1203 12:17:36.172709 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage09-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage09-crc\") pod \"glance-default-external-api-0\" (UID: \"cd9a6dc3-a814-4a29-924a-d8ada756e7c7\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:17:36 crc kubenswrapper[5052]: I1203 12:17:36.173228 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage09-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage09-crc\") pod \"glance-default-external-api-0\" (UID: \"cd9a6dc3-a814-4a29-924a-d8ada756e7c7\") device mount path \"/mnt/openstack/pv09\"" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:17:36 crc kubenswrapper[5052]: I1203 12:17:36.173333 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/cd9a6dc3-a814-4a29-924a-d8ada756e7c7-logs\") pod \"glance-default-external-api-0\" (UID: \"cd9a6dc3-a814-4a29-924a-d8ada756e7c7\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:17:36 crc kubenswrapper[5052]: I1203 12:17:36.173358 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/cd9a6dc3-a814-4a29-924a-d8ada756e7c7-httpd-run\") pod \"glance-default-external-api-0\" (UID: \"cd9a6dc3-a814-4a29-924a-d8ada756e7c7\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:17:36 crc kubenswrapper[5052]: I1203 12:17:36.175681 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/cd9a6dc3-a814-4a29-924a-d8ada756e7c7-public-tls-certs\") pod \"glance-default-external-api-0\" (UID: \"cd9a6dc3-a814-4a29-924a-d8ada756e7c7\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:17:36 crc kubenswrapper[5052]: I1203 12:17:36.175810 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/cd9a6dc3-a814-4a29-924a-d8ada756e7c7-combined-ca-bundle\") pod \"glance-default-external-api-0\" (UID: \"cd9a6dc3-a814-4a29-924a-d8ada756e7c7\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:17:36 crc kubenswrapper[5052]: I1203 12:17:36.175894 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-lfv2c\" (UniqueName: \"kubernetes.io/projected/cd9a6dc3-a814-4a29-924a-d8ada756e7c7-kube-api-access-lfv2c\") pod \"glance-default-external-api-0\" (UID: \"cd9a6dc3-a814-4a29-924a-d8ada756e7c7\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:17:36 crc kubenswrapper[5052]: I1203 12:17:36.175940 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/cd9a6dc3-a814-4a29-924a-d8ada756e7c7-config-data\") pod \"glance-default-external-api-0\" (UID: \"cd9a6dc3-a814-4a29-924a-d8ada756e7c7\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:17:36 crc kubenswrapper[5052]: I1203 12:17:36.180183 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/cd9a6dc3-a814-4a29-924a-d8ada756e7c7-scripts\") pod \"glance-default-external-api-0\" (UID: \"cd9a6dc3-a814-4a29-924a-d8ada756e7c7\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:17:36 crc kubenswrapper[5052]: I1203 12:17:36.182385 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/cd9a6dc3-a814-4a29-924a-d8ada756e7c7-config-data\") pod \"glance-default-external-api-0\" (UID: \"cd9a6dc3-a814-4a29-924a-d8ada756e7c7\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:17:36 crc kubenswrapper[5052]: I1203 12:17:36.182459 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/cd9a6dc3-a814-4a29-924a-d8ada756e7c7-public-tls-certs\") pod \"glance-default-external-api-0\" (UID: \"cd9a6dc3-a814-4a29-924a-d8ada756e7c7\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:17:36 crc kubenswrapper[5052]: I1203 12:17:36.192538 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/cd9a6dc3-a814-4a29-924a-d8ada756e7c7-combined-ca-bundle\") pod \"glance-default-external-api-0\" (UID: \"cd9a6dc3-a814-4a29-924a-d8ada756e7c7\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:17:36 crc kubenswrapper[5052]: I1203 12:17:36.204237 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-lfv2c\" (UniqueName: \"kubernetes.io/projected/cd9a6dc3-a814-4a29-924a-d8ada756e7c7-kube-api-access-lfv2c\") pod \"glance-default-external-api-0\" (UID: \"cd9a6dc3-a814-4a29-924a-d8ada756e7c7\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:17:36 crc kubenswrapper[5052]: I1203 12:17:36.209813 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage09-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage09-crc\") pod \"glance-default-external-api-0\" (UID: \"cd9a6dc3-a814-4a29-924a-d8ada756e7c7\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:17:36 crc kubenswrapper[5052]: I1203 12:17:36.282065 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:17:36 crc kubenswrapper[5052]: I1203 12:17:36.330026 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:17:36 crc kubenswrapper[5052]: I1203 12:17:36.379674 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/e51b5737-6907-4db8-a9a6-1019da0e64d2-config-data\") pod \"e51b5737-6907-4db8-a9a6-1019da0e64d2\" (UID: \"e51b5737-6907-4db8-a9a6-1019da0e64d2\") " Dec 03 12:17:36 crc kubenswrapper[5052]: I1203 12:17:36.379742 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/e51b5737-6907-4db8-a9a6-1019da0e64d2-logs\") pod \"e51b5737-6907-4db8-a9a6-1019da0e64d2\" (UID: \"e51b5737-6907-4db8-a9a6-1019da0e64d2\") " Dec 03 12:17:36 crc kubenswrapper[5052]: I1203 12:17:36.379798 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/e51b5737-6907-4db8-a9a6-1019da0e64d2-scripts\") pod \"e51b5737-6907-4db8-a9a6-1019da0e64d2\" (UID: \"e51b5737-6907-4db8-a9a6-1019da0e64d2\") " Dec 03 12:17:36 crc kubenswrapper[5052]: I1203 12:17:36.379844 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/e51b5737-6907-4db8-a9a6-1019da0e64d2-httpd-run\") pod \"e51b5737-6907-4db8-a9a6-1019da0e64d2\" (UID: \"e51b5737-6907-4db8-a9a6-1019da0e64d2\") " Dec 03 12:17:36 crc kubenswrapper[5052]: I1203 12:17:36.379902 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e51b5737-6907-4db8-a9a6-1019da0e64d2-combined-ca-bundle\") pod \"e51b5737-6907-4db8-a9a6-1019da0e64d2\" (UID: \"e51b5737-6907-4db8-a9a6-1019da0e64d2\") " Dec 03 12:17:36 crc kubenswrapper[5052]: I1203 12:17:36.379948 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/e51b5737-6907-4db8-a9a6-1019da0e64d2-internal-tls-certs\") pod \"e51b5737-6907-4db8-a9a6-1019da0e64d2\" (UID: \"e51b5737-6907-4db8-a9a6-1019da0e64d2\") " Dec 03 12:17:36 crc kubenswrapper[5052]: I1203 12:17:36.379991 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"glance\" (UniqueName: \"kubernetes.io/local-volume/local-storage17-crc\") pod \"e51b5737-6907-4db8-a9a6-1019da0e64d2\" (UID: \"e51b5737-6907-4db8-a9a6-1019da0e64d2\") " Dec 03 12:17:36 crc kubenswrapper[5052]: I1203 12:17:36.380025 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v979t\" (UniqueName: \"kubernetes.io/projected/e51b5737-6907-4db8-a9a6-1019da0e64d2-kube-api-access-v979t\") pod \"e51b5737-6907-4db8-a9a6-1019da0e64d2\" (UID: \"e51b5737-6907-4db8-a9a6-1019da0e64d2\") " Dec 03 12:17:36 crc kubenswrapper[5052]: I1203 12:17:36.380546 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e51b5737-6907-4db8-a9a6-1019da0e64d2-httpd-run" (OuterVolumeSpecName: "httpd-run") pod "e51b5737-6907-4db8-a9a6-1019da0e64d2" (UID: "e51b5737-6907-4db8-a9a6-1019da0e64d2"). InnerVolumeSpecName "httpd-run". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:17:36 crc kubenswrapper[5052]: I1203 12:17:36.380627 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e51b5737-6907-4db8-a9a6-1019da0e64d2-logs" (OuterVolumeSpecName: "logs") pod "e51b5737-6907-4db8-a9a6-1019da0e64d2" (UID: "e51b5737-6907-4db8-a9a6-1019da0e64d2"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:17:36 crc kubenswrapper[5052]: I1203 12:17:36.385741 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e51b5737-6907-4db8-a9a6-1019da0e64d2-kube-api-access-v979t" (OuterVolumeSpecName: "kube-api-access-v979t") pod "e51b5737-6907-4db8-a9a6-1019da0e64d2" (UID: "e51b5737-6907-4db8-a9a6-1019da0e64d2"). InnerVolumeSpecName "kube-api-access-v979t". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:17:36 crc kubenswrapper[5052]: I1203 12:17:36.385721 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e51b5737-6907-4db8-a9a6-1019da0e64d2-scripts" (OuterVolumeSpecName: "scripts") pod "e51b5737-6907-4db8-a9a6-1019da0e64d2" (UID: "e51b5737-6907-4db8-a9a6-1019da0e64d2"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:17:36 crc kubenswrapper[5052]: I1203 12:17:36.400304 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage17-crc" (OuterVolumeSpecName: "glance") pod "e51b5737-6907-4db8-a9a6-1019da0e64d2" (UID: "e51b5737-6907-4db8-a9a6-1019da0e64d2"). InnerVolumeSpecName "local-storage17-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 12:17:36 crc kubenswrapper[5052]: I1203 12:17:36.410722 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e51b5737-6907-4db8-a9a6-1019da0e64d2-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "e51b5737-6907-4db8-a9a6-1019da0e64d2" (UID: "e51b5737-6907-4db8-a9a6-1019da0e64d2"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:17:36 crc kubenswrapper[5052]: I1203 12:17:36.458063 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e51b5737-6907-4db8-a9a6-1019da0e64d2-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "e51b5737-6907-4db8-a9a6-1019da0e64d2" (UID: "e51b5737-6907-4db8-a9a6-1019da0e64d2"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:17:36 crc kubenswrapper[5052]: I1203 12:17:36.482259 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e51b5737-6907-4db8-a9a6-1019da0e64d2-config-data" (OuterVolumeSpecName: "config-data") pod "e51b5737-6907-4db8-a9a6-1019da0e64d2" (UID: "e51b5737-6907-4db8-a9a6-1019da0e64d2"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:17:36 crc kubenswrapper[5052]: I1203 12:17:36.482805 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/e51b5737-6907-4db8-a9a6-1019da0e64d2-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:17:36 crc kubenswrapper[5052]: I1203 12:17:36.482836 5052 reconciler_common.go:293] "Volume detached for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/e51b5737-6907-4db8-a9a6-1019da0e64d2-httpd-run\") on node \"crc\" DevicePath \"\"" Dec 03 12:17:36 crc kubenswrapper[5052]: I1203 12:17:36.482850 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e51b5737-6907-4db8-a9a6-1019da0e64d2-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:17:36 crc kubenswrapper[5052]: I1203 12:17:36.482864 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/e51b5737-6907-4db8-a9a6-1019da0e64d2-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:17:36 crc kubenswrapper[5052]: I1203 12:17:36.482901 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage17-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage17-crc\") on node \"crc\" " Dec 03 12:17:36 crc kubenswrapper[5052]: I1203 12:17:36.482918 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-v979t\" (UniqueName: \"kubernetes.io/projected/e51b5737-6907-4db8-a9a6-1019da0e64d2-kube-api-access-v979t\") on node \"crc\" DevicePath \"\"" Dec 03 12:17:36 crc kubenswrapper[5052]: I1203 12:17:36.482933 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/e51b5737-6907-4db8-a9a6-1019da0e64d2-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:17:36 crc kubenswrapper[5052]: I1203 12:17:36.482944 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/e51b5737-6907-4db8-a9a6-1019da0e64d2-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:17:36 crc kubenswrapper[5052]: I1203 12:17:36.507074 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage17-crc" (UniqueName: "kubernetes.io/local-volume/local-storage17-crc") on node "crc" Dec 03 12:17:36 crc kubenswrapper[5052]: I1203 12:17:36.584920 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage17-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage17-crc\") on node \"crc\" DevicePath \"\"" Dec 03 12:17:36 crc kubenswrapper[5052]: I1203 12:17:36.766960 5052 generic.go:334] "Generic (PLEG): container finished" podID="38a5d722-dbe5-4d1e-9aec-bf6513db82fb" containerID="1cfce437c4e384a34c57bced804c64f40f25e45c8f65ee39ad0277b7e37ec7f0" exitCode=0 Dec 03 12:17:36 crc kubenswrapper[5052]: I1203 12:17:36.767121 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-bootstrap-5qw5w" event={"ID":"38a5d722-dbe5-4d1e-9aec-bf6513db82fb","Type":"ContainerDied","Data":"1cfce437c4e384a34c57bced804c64f40f25e45c8f65ee39ad0277b7e37ec7f0"} Dec 03 12:17:36 crc kubenswrapper[5052]: I1203 12:17:36.771783 5052 generic.go:334] "Generic (PLEG): container finished" podID="2f98071b-1e7d-4cf3-976e-ad9073a2f914" containerID="518459180ad34e42e3d4890b691c685fdb215176ae41fec275f3e19b1b7907ae" exitCode=0 Dec 03 12:17:36 crc kubenswrapper[5052]: I1203 12:17:36.771860 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-db-sync-pnhqf" event={"ID":"2f98071b-1e7d-4cf3-976e-ad9073a2f914","Type":"ContainerDied","Data":"518459180ad34e42e3d4890b691c685fdb215176ae41fec275f3e19b1b7907ae"} Dec 03 12:17:36 crc kubenswrapper[5052]: I1203 12:17:36.774718 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"614c796f-690e-4c8e-833f-bf7ba7c47d1a","Type":"ContainerStarted","Data":"e152d21042dfbb59504e298835f095942b27ff95b7e38c26f308eec742685448"} Dec 03 12:17:36 crc kubenswrapper[5052]: I1203 12:17:36.774737 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="614c796f-690e-4c8e-833f-bf7ba7c47d1a" containerName="ceilometer-central-agent" containerID="cri-o://d26b41a42104863894f831fd9b945032f3ccd5ce0630c9e4673313dfb9e41580" gracePeriod=30 Dec 03 12:17:36 crc kubenswrapper[5052]: I1203 12:17:36.774905 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:17:36 crc kubenswrapper[5052]: I1203 12:17:36.774942 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="614c796f-690e-4c8e-833f-bf7ba7c47d1a" containerName="sg-core" containerID="cri-o://c8beb7f8c222fb4c94ee6aadf2ded6eb09f46f942e334eb694d5a1683fcf80f9" gracePeriod=30 Dec 03 12:17:36 crc kubenswrapper[5052]: I1203 12:17:36.774987 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="614c796f-690e-4c8e-833f-bf7ba7c47d1a" containerName="proxy-httpd" containerID="cri-o://e152d21042dfbb59504e298835f095942b27ff95b7e38c26f308eec742685448" gracePeriod=30 Dec 03 12:17:36 crc kubenswrapper[5052]: I1203 12:17:36.775006 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="614c796f-690e-4c8e-833f-bf7ba7c47d1a" containerName="ceilometer-notification-agent" containerID="cri-o://ecb8cd2f428a35b04fce42b144a6f3606e98b3dcaa31eb3623b92627a69a0813" gracePeriod=30 Dec 03 12:17:36 crc kubenswrapper[5052]: I1203 12:17:36.779042 5052 generic.go:334] "Generic (PLEG): container finished" podID="e51b5737-6907-4db8-a9a6-1019da0e64d2" containerID="9f6ab18d411ec6d144b75557c3a32ac72e2d81183efdfb6a5aee9bdbf7117475" exitCode=0 Dec 03 12:17:36 crc kubenswrapper[5052]: I1203 12:17:36.779084 5052 generic.go:334] "Generic (PLEG): container finished" podID="e51b5737-6907-4db8-a9a6-1019da0e64d2" containerID="fe340f30c1795c46a94d71508d1527680e9438f839d1587b7040eab22d8367f5" exitCode=143 Dec 03 12:17:36 crc kubenswrapper[5052]: I1203 12:17:36.779102 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:17:36 crc kubenswrapper[5052]: I1203 12:17:36.779185 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"e51b5737-6907-4db8-a9a6-1019da0e64d2","Type":"ContainerDied","Data":"9f6ab18d411ec6d144b75557c3a32ac72e2d81183efdfb6a5aee9bdbf7117475"} Dec 03 12:17:36 crc kubenswrapper[5052]: I1203 12:17:36.779218 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"e51b5737-6907-4db8-a9a6-1019da0e64d2","Type":"ContainerDied","Data":"fe340f30c1795c46a94d71508d1527680e9438f839d1587b7040eab22d8367f5"} Dec 03 12:17:36 crc kubenswrapper[5052]: I1203 12:17:36.779230 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"e51b5737-6907-4db8-a9a6-1019da0e64d2","Type":"ContainerDied","Data":"6695052548a2eb4feacbdcfb8ed6828bbe4bdbb48ef281893bc6842868d4ed6d"} Dec 03 12:17:36 crc kubenswrapper[5052]: I1203 12:17:36.779248 5052 scope.go:117] "RemoveContainer" containerID="9f6ab18d411ec6d144b75557c3a32ac72e2d81183efdfb6a5aee9bdbf7117475" Dec 03 12:17:36 crc kubenswrapper[5052]: I1203 12:17:36.842367 5052 scope.go:117] "RemoveContainer" containerID="fe340f30c1795c46a94d71508d1527680e9438f839d1587b7040eab22d8367f5" Dec 03 12:17:36 crc kubenswrapper[5052]: W1203 12:17:36.853071 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcd9a6dc3_a814_4a29_924a_d8ada756e7c7.slice/crio-bb61d38d070167697e611c63d76232f8d5328df5151f316456068d66eb87f80a WatchSource:0}: Error finding container bb61d38d070167697e611c63d76232f8d5328df5151f316456068d66eb87f80a: Status 404 returned error can't find the container with id bb61d38d070167697e611c63d76232f8d5328df5151f316456068d66eb87f80a Dec 03 12:17:36 crc kubenswrapper[5052]: I1203 12:17:36.878225 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:17:36 crc kubenswrapper[5052]: I1203 12:17:36.893241 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/ceilometer-0" podStartSLOduration=3.039683838 podStartE2EDuration="6.893216776s" podCreationTimestamp="2025-12-03 12:17:30 +0000 UTC" firstStartedPulling="2025-12-03 12:17:31.78684483 +0000 UTC m=+3442.375086594" lastFinishedPulling="2025-12-03 12:17:35.640377758 +0000 UTC m=+3446.228619532" observedRunningTime="2025-12-03 12:17:36.825555185 +0000 UTC m=+3447.413796959" watchObservedRunningTime="2025-12-03 12:17:36.893216776 +0000 UTC m=+3447.481458560" Dec 03 12:17:36 crc kubenswrapper[5052]: I1203 12:17:36.928442 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:17:36 crc kubenswrapper[5052]: I1203 12:17:36.969621 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:17:36 crc kubenswrapper[5052]: I1203 12:17:36.999016 5052 scope.go:117] "RemoveContainer" containerID="9f6ab18d411ec6d144b75557c3a32ac72e2d81183efdfb6a5aee9bdbf7117475" Dec 03 12:17:37 crc kubenswrapper[5052]: E1203 12:17:36.999498 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9f6ab18d411ec6d144b75557c3a32ac72e2d81183efdfb6a5aee9bdbf7117475\": container with ID starting with 9f6ab18d411ec6d144b75557c3a32ac72e2d81183efdfb6a5aee9bdbf7117475 not found: ID does not exist" containerID="9f6ab18d411ec6d144b75557c3a32ac72e2d81183efdfb6a5aee9bdbf7117475" Dec 03 12:17:37 crc kubenswrapper[5052]: I1203 12:17:36.999558 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9f6ab18d411ec6d144b75557c3a32ac72e2d81183efdfb6a5aee9bdbf7117475"} err="failed to get container status \"9f6ab18d411ec6d144b75557c3a32ac72e2d81183efdfb6a5aee9bdbf7117475\": rpc error: code = NotFound desc = could not find container \"9f6ab18d411ec6d144b75557c3a32ac72e2d81183efdfb6a5aee9bdbf7117475\": container with ID starting with 9f6ab18d411ec6d144b75557c3a32ac72e2d81183efdfb6a5aee9bdbf7117475 not found: ID does not exist" Dec 03 12:17:37 crc kubenswrapper[5052]: I1203 12:17:36.999582 5052 scope.go:117] "RemoveContainer" containerID="fe340f30c1795c46a94d71508d1527680e9438f839d1587b7040eab22d8367f5" Dec 03 12:17:37 crc kubenswrapper[5052]: I1203 12:17:37.002308 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:17:37 crc kubenswrapper[5052]: E1203 12:17:37.003030 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="e51b5737-6907-4db8-a9a6-1019da0e64d2" containerName="glance-httpd" Dec 03 12:17:37 crc kubenswrapper[5052]: I1203 12:17:37.003054 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="e51b5737-6907-4db8-a9a6-1019da0e64d2" containerName="glance-httpd" Dec 03 12:17:37 crc kubenswrapper[5052]: E1203 12:17:37.003069 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="e51b5737-6907-4db8-a9a6-1019da0e64d2" containerName="glance-log" Dec 03 12:17:37 crc kubenswrapper[5052]: I1203 12:17:37.003077 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="e51b5737-6907-4db8-a9a6-1019da0e64d2" containerName="glance-log" Dec 03 12:17:37 crc kubenswrapper[5052]: I1203 12:17:37.003253 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="e51b5737-6907-4db8-a9a6-1019da0e64d2" containerName="glance-httpd" Dec 03 12:17:37 crc kubenswrapper[5052]: I1203 12:17:37.003280 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="e51b5737-6907-4db8-a9a6-1019da0e64d2" containerName="glance-log" Dec 03 12:17:37 crc kubenswrapper[5052]: I1203 12:17:37.004304 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:17:37 crc kubenswrapper[5052]: E1203 12:17:37.006842 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fe340f30c1795c46a94d71508d1527680e9438f839d1587b7040eab22d8367f5\": container with ID starting with fe340f30c1795c46a94d71508d1527680e9438f839d1587b7040eab22d8367f5 not found: ID does not exist" containerID="fe340f30c1795c46a94d71508d1527680e9438f839d1587b7040eab22d8367f5" Dec 03 12:17:37 crc kubenswrapper[5052]: I1203 12:17:37.006900 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fe340f30c1795c46a94d71508d1527680e9438f839d1587b7040eab22d8367f5"} err="failed to get container status \"fe340f30c1795c46a94d71508d1527680e9438f839d1587b7040eab22d8367f5\": rpc error: code = NotFound desc = could not find container \"fe340f30c1795c46a94d71508d1527680e9438f839d1587b7040eab22d8367f5\": container with ID starting with fe340f30c1795c46a94d71508d1527680e9438f839d1587b7040eab22d8367f5 not found: ID does not exist" Dec 03 12:17:37 crc kubenswrapper[5052]: I1203 12:17:37.006933 5052 scope.go:117] "RemoveContainer" containerID="9f6ab18d411ec6d144b75557c3a32ac72e2d81183efdfb6a5aee9bdbf7117475" Dec 03 12:17:37 crc kubenswrapper[5052]: I1203 12:17:37.007566 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9f6ab18d411ec6d144b75557c3a32ac72e2d81183efdfb6a5aee9bdbf7117475"} err="failed to get container status \"9f6ab18d411ec6d144b75557c3a32ac72e2d81183efdfb6a5aee9bdbf7117475\": rpc error: code = NotFound desc = could not find container \"9f6ab18d411ec6d144b75557c3a32ac72e2d81183efdfb6a5aee9bdbf7117475\": container with ID starting with 9f6ab18d411ec6d144b75557c3a32ac72e2d81183efdfb6a5aee9bdbf7117475 not found: ID does not exist" Dec 03 12:17:37 crc kubenswrapper[5052]: I1203 12:17:37.007595 5052 scope.go:117] "RemoveContainer" containerID="fe340f30c1795c46a94d71508d1527680e9438f839d1587b7040eab22d8367f5" Dec 03 12:17:37 crc kubenswrapper[5052]: I1203 12:17:37.007779 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-glance-default-internal-svc" Dec 03 12:17:37 crc kubenswrapper[5052]: I1203 12:17:37.007949 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fe340f30c1795c46a94d71508d1527680e9438f839d1587b7040eab22d8367f5"} err="failed to get container status \"fe340f30c1795c46a94d71508d1527680e9438f839d1587b7040eab22d8367f5\": rpc error: code = NotFound desc = could not find container \"fe340f30c1795c46a94d71508d1527680e9438f839d1587b7040eab22d8367f5\": container with ID starting with fe340f30c1795c46a94d71508d1527680e9438f839d1587b7040eab22d8367f5 not found: ID does not exist" Dec 03 12:17:37 crc kubenswrapper[5052]: I1203 12:17:37.008288 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-default-internal-config-data" Dec 03 12:17:37 crc kubenswrapper[5052]: I1203 12:17:37.016356 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:17:37 crc kubenswrapper[5052]: I1203 12:17:37.098255 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/f65abb68-4704-4c08-b1eb-da1ac582a4d8-httpd-run\") pod \"glance-default-internal-api-0\" (UID: \"f65abb68-4704-4c08-b1eb-da1ac582a4d8\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:17:37 crc kubenswrapper[5052]: I1203 12:17:37.102742 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f65abb68-4704-4c08-b1eb-da1ac582a4d8-combined-ca-bundle\") pod \"glance-default-internal-api-0\" (UID: \"f65abb68-4704-4c08-b1eb-da1ac582a4d8\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:17:37 crc kubenswrapper[5052]: I1203 12:17:37.102788 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage17-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage17-crc\") pod \"glance-default-internal-api-0\" (UID: \"f65abb68-4704-4c08-b1eb-da1ac582a4d8\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:17:37 crc kubenswrapper[5052]: I1203 12:17:37.102812 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nlm8s\" (UniqueName: \"kubernetes.io/projected/f65abb68-4704-4c08-b1eb-da1ac582a4d8-kube-api-access-nlm8s\") pod \"glance-default-internal-api-0\" (UID: \"f65abb68-4704-4c08-b1eb-da1ac582a4d8\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:17:37 crc kubenswrapper[5052]: I1203 12:17:37.102883 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/f65abb68-4704-4c08-b1eb-da1ac582a4d8-internal-tls-certs\") pod \"glance-default-internal-api-0\" (UID: \"f65abb68-4704-4c08-b1eb-da1ac582a4d8\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:17:37 crc kubenswrapper[5052]: I1203 12:17:37.102907 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/f65abb68-4704-4c08-b1eb-da1ac582a4d8-logs\") pod \"glance-default-internal-api-0\" (UID: \"f65abb68-4704-4c08-b1eb-da1ac582a4d8\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:17:37 crc kubenswrapper[5052]: I1203 12:17:37.102989 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/f65abb68-4704-4c08-b1eb-da1ac582a4d8-config-data\") pod \"glance-default-internal-api-0\" (UID: \"f65abb68-4704-4c08-b1eb-da1ac582a4d8\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:17:37 crc kubenswrapper[5052]: I1203 12:17:37.103056 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/f65abb68-4704-4c08-b1eb-da1ac582a4d8-scripts\") pod \"glance-default-internal-api-0\" (UID: \"f65abb68-4704-4c08-b1eb-da1ac582a4d8\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:17:37 crc kubenswrapper[5052]: I1203 12:17:37.205882 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f65abb68-4704-4c08-b1eb-da1ac582a4d8-combined-ca-bundle\") pod \"glance-default-internal-api-0\" (UID: \"f65abb68-4704-4c08-b1eb-da1ac582a4d8\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:17:37 crc kubenswrapper[5052]: I1203 12:17:37.206194 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage17-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage17-crc\") pod \"glance-default-internal-api-0\" (UID: \"f65abb68-4704-4c08-b1eb-da1ac582a4d8\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:17:37 crc kubenswrapper[5052]: I1203 12:17:37.206215 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-nlm8s\" (UniqueName: \"kubernetes.io/projected/f65abb68-4704-4c08-b1eb-da1ac582a4d8-kube-api-access-nlm8s\") pod \"glance-default-internal-api-0\" (UID: \"f65abb68-4704-4c08-b1eb-da1ac582a4d8\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:17:37 crc kubenswrapper[5052]: I1203 12:17:37.206275 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/f65abb68-4704-4c08-b1eb-da1ac582a4d8-internal-tls-certs\") pod \"glance-default-internal-api-0\" (UID: \"f65abb68-4704-4c08-b1eb-da1ac582a4d8\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:17:37 crc kubenswrapper[5052]: I1203 12:17:37.206295 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/f65abb68-4704-4c08-b1eb-da1ac582a4d8-logs\") pod \"glance-default-internal-api-0\" (UID: \"f65abb68-4704-4c08-b1eb-da1ac582a4d8\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:17:37 crc kubenswrapper[5052]: I1203 12:17:37.206357 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/f65abb68-4704-4c08-b1eb-da1ac582a4d8-config-data\") pod \"glance-default-internal-api-0\" (UID: \"f65abb68-4704-4c08-b1eb-da1ac582a4d8\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:17:37 crc kubenswrapper[5052]: I1203 12:17:37.206416 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/f65abb68-4704-4c08-b1eb-da1ac582a4d8-scripts\") pod \"glance-default-internal-api-0\" (UID: \"f65abb68-4704-4c08-b1eb-da1ac582a4d8\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:17:37 crc kubenswrapper[5052]: I1203 12:17:37.206453 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/f65abb68-4704-4c08-b1eb-da1ac582a4d8-httpd-run\") pod \"glance-default-internal-api-0\" (UID: \"f65abb68-4704-4c08-b1eb-da1ac582a4d8\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:17:37 crc kubenswrapper[5052]: I1203 12:17:37.206893 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/f65abb68-4704-4c08-b1eb-da1ac582a4d8-httpd-run\") pod \"glance-default-internal-api-0\" (UID: \"f65abb68-4704-4c08-b1eb-da1ac582a4d8\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:17:37 crc kubenswrapper[5052]: I1203 12:17:37.207206 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage17-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage17-crc\") pod \"glance-default-internal-api-0\" (UID: \"f65abb68-4704-4c08-b1eb-da1ac582a4d8\") device mount path \"/mnt/openstack/pv17\"" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:17:37 crc kubenswrapper[5052]: I1203 12:17:37.208523 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/f65abb68-4704-4c08-b1eb-da1ac582a4d8-logs\") pod \"glance-default-internal-api-0\" (UID: \"f65abb68-4704-4c08-b1eb-da1ac582a4d8\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:17:37 crc kubenswrapper[5052]: I1203 12:17:37.216156 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/f65abb68-4704-4c08-b1eb-da1ac582a4d8-scripts\") pod \"glance-default-internal-api-0\" (UID: \"f65abb68-4704-4c08-b1eb-da1ac582a4d8\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:17:37 crc kubenswrapper[5052]: I1203 12:17:37.216468 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/f65abb68-4704-4c08-b1eb-da1ac582a4d8-config-data\") pod \"glance-default-internal-api-0\" (UID: \"f65abb68-4704-4c08-b1eb-da1ac582a4d8\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:17:37 crc kubenswrapper[5052]: I1203 12:17:37.216744 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/f65abb68-4704-4c08-b1eb-da1ac582a4d8-internal-tls-certs\") pod \"glance-default-internal-api-0\" (UID: \"f65abb68-4704-4c08-b1eb-da1ac582a4d8\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:17:37 crc kubenswrapper[5052]: I1203 12:17:37.216856 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f65abb68-4704-4c08-b1eb-da1ac582a4d8-combined-ca-bundle\") pod \"glance-default-internal-api-0\" (UID: \"f65abb68-4704-4c08-b1eb-da1ac582a4d8\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:17:37 crc kubenswrapper[5052]: I1203 12:17:37.228053 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-nlm8s\" (UniqueName: \"kubernetes.io/projected/f65abb68-4704-4c08-b1eb-da1ac582a4d8-kube-api-access-nlm8s\") pod \"glance-default-internal-api-0\" (UID: \"f65abb68-4704-4c08-b1eb-da1ac582a4d8\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:17:37 crc kubenswrapper[5052]: I1203 12:17:37.239423 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage17-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage17-crc\") pod \"glance-default-internal-api-0\" (UID: \"f65abb68-4704-4c08-b1eb-da1ac582a4d8\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:17:37 crc kubenswrapper[5052]: I1203 12:17:37.241304 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-db-sync-hh2rh" Dec 03 12:17:37 crc kubenswrapper[5052]: I1203 12:17:37.307222 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/9f563ce4-a788-44eb-bdab-9fc4c3423176-config-data\") pod \"9f563ce4-a788-44eb-bdab-9fc4c3423176\" (UID: \"9f563ce4-a788-44eb-bdab-9fc4c3423176\") " Dec 03 12:17:37 crc kubenswrapper[5052]: I1203 12:17:37.307322 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sbhzv\" (UniqueName: \"kubernetes.io/projected/9f563ce4-a788-44eb-bdab-9fc4c3423176-kube-api-access-sbhzv\") pod \"9f563ce4-a788-44eb-bdab-9fc4c3423176\" (UID: \"9f563ce4-a788-44eb-bdab-9fc4c3423176\") " Dec 03 12:17:37 crc kubenswrapper[5052]: I1203 12:17:37.307409 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/9f563ce4-a788-44eb-bdab-9fc4c3423176-scripts\") pod \"9f563ce4-a788-44eb-bdab-9fc4c3423176\" (UID: \"9f563ce4-a788-44eb-bdab-9fc4c3423176\") " Dec 03 12:17:37 crc kubenswrapper[5052]: I1203 12:17:37.307584 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/9f563ce4-a788-44eb-bdab-9fc4c3423176-logs\") pod \"9f563ce4-a788-44eb-bdab-9fc4c3423176\" (UID: \"9f563ce4-a788-44eb-bdab-9fc4c3423176\") " Dec 03 12:17:37 crc kubenswrapper[5052]: I1203 12:17:37.307610 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9f563ce4-a788-44eb-bdab-9fc4c3423176-combined-ca-bundle\") pod \"9f563ce4-a788-44eb-bdab-9fc4c3423176\" (UID: \"9f563ce4-a788-44eb-bdab-9fc4c3423176\") " Dec 03 12:17:37 crc kubenswrapper[5052]: I1203 12:17:37.309721 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f563ce4-a788-44eb-bdab-9fc4c3423176-logs" (OuterVolumeSpecName: "logs") pod "9f563ce4-a788-44eb-bdab-9fc4c3423176" (UID: "9f563ce4-a788-44eb-bdab-9fc4c3423176"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:17:37 crc kubenswrapper[5052]: I1203 12:17:37.311112 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9f563ce4-a788-44eb-bdab-9fc4c3423176-kube-api-access-sbhzv" (OuterVolumeSpecName: "kube-api-access-sbhzv") pod "9f563ce4-a788-44eb-bdab-9fc4c3423176" (UID: "9f563ce4-a788-44eb-bdab-9fc4c3423176"). InnerVolumeSpecName "kube-api-access-sbhzv". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:17:37 crc kubenswrapper[5052]: I1203 12:17:37.312044 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9f563ce4-a788-44eb-bdab-9fc4c3423176-scripts" (OuterVolumeSpecName: "scripts") pod "9f563ce4-a788-44eb-bdab-9fc4c3423176" (UID: "9f563ce4-a788-44eb-bdab-9fc4c3423176"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:17:37 crc kubenswrapper[5052]: I1203 12:17:37.331399 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9f563ce4-a788-44eb-bdab-9fc4c3423176-config-data" (OuterVolumeSpecName: "config-data") pod "9f563ce4-a788-44eb-bdab-9fc4c3423176" (UID: "9f563ce4-a788-44eb-bdab-9fc4c3423176"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:17:37 crc kubenswrapper[5052]: I1203 12:17:37.332557 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9f563ce4-a788-44eb-bdab-9fc4c3423176-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "9f563ce4-a788-44eb-bdab-9fc4c3423176" (UID: "9f563ce4-a788-44eb-bdab-9fc4c3423176"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:17:37 crc kubenswrapper[5052]: I1203 12:17:37.409093 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/9f563ce4-a788-44eb-bdab-9fc4c3423176-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:17:37 crc kubenswrapper[5052]: I1203 12:17:37.409129 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9f563ce4-a788-44eb-bdab-9fc4c3423176-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:17:37 crc kubenswrapper[5052]: I1203 12:17:37.409141 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/9f563ce4-a788-44eb-bdab-9fc4c3423176-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:17:37 crc kubenswrapper[5052]: I1203 12:17:37.409151 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-sbhzv\" (UniqueName: \"kubernetes.io/projected/9f563ce4-a788-44eb-bdab-9fc4c3423176-kube-api-access-sbhzv\") on node \"crc\" DevicePath \"\"" Dec 03 12:17:37 crc kubenswrapper[5052]: I1203 12:17:37.409159 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/9f563ce4-a788-44eb-bdab-9fc4c3423176-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:17:37 crc kubenswrapper[5052]: I1203 12:17:37.411490 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:17:37 crc kubenswrapper[5052]: I1203 12:17:37.703349 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="408d4a98-4cc6-4a13-a2f3-940747dfb910" path="/var/lib/kubelet/pods/408d4a98-4cc6-4a13-a2f3-940747dfb910/volumes" Dec 03 12:17:37 crc kubenswrapper[5052]: I1203 12:17:37.707623 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e51b5737-6907-4db8-a9a6-1019da0e64d2" path="/var/lib/kubelet/pods/e51b5737-6907-4db8-a9a6-1019da0e64d2/volumes" Dec 03 12:17:37 crc kubenswrapper[5052]: I1203 12:17:37.847258 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-db-sync-hh2rh" event={"ID":"9f563ce4-a788-44eb-bdab-9fc4c3423176","Type":"ContainerDied","Data":"e1a5483ee61db6af9e8058349f002642831d4d2623b7c0ded4999b576d55b614"} Dec 03 12:17:37 crc kubenswrapper[5052]: I1203 12:17:37.847302 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e1a5483ee61db6af9e8058349f002642831d4d2623b7c0ded4999b576d55b614" Dec 03 12:17:37 crc kubenswrapper[5052]: I1203 12:17:37.847363 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-db-sync-hh2rh" Dec 03 12:17:37 crc kubenswrapper[5052]: I1203 12:17:37.854713 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"cd9a6dc3-a814-4a29-924a-d8ada756e7c7","Type":"ContainerStarted","Data":"621073c1609087e450aafc5127e1c89c8e8d8de48573af61d7ee441bcd0685c7"} Dec 03 12:17:37 crc kubenswrapper[5052]: I1203 12:17:37.854776 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"cd9a6dc3-a814-4a29-924a-d8ada756e7c7","Type":"ContainerStarted","Data":"bb61d38d070167697e611c63d76232f8d5328df5151f316456068d66eb87f80a"} Dec 03 12:17:37 crc kubenswrapper[5052]: I1203 12:17:37.858324 5052 generic.go:334] "Generic (PLEG): container finished" podID="614c796f-690e-4c8e-833f-bf7ba7c47d1a" containerID="e152d21042dfbb59504e298835f095942b27ff95b7e38c26f308eec742685448" exitCode=0 Dec 03 12:17:37 crc kubenswrapper[5052]: I1203 12:17:37.858352 5052 generic.go:334] "Generic (PLEG): container finished" podID="614c796f-690e-4c8e-833f-bf7ba7c47d1a" containerID="c8beb7f8c222fb4c94ee6aadf2ded6eb09f46f942e334eb694d5a1683fcf80f9" exitCode=2 Dec 03 12:17:37 crc kubenswrapper[5052]: I1203 12:17:37.858361 5052 generic.go:334] "Generic (PLEG): container finished" podID="614c796f-690e-4c8e-833f-bf7ba7c47d1a" containerID="ecb8cd2f428a35b04fce42b144a6f3606e98b3dcaa31eb3623b92627a69a0813" exitCode=0 Dec 03 12:17:37 crc kubenswrapper[5052]: I1203 12:17:37.858371 5052 generic.go:334] "Generic (PLEG): container finished" podID="614c796f-690e-4c8e-833f-bf7ba7c47d1a" containerID="d26b41a42104863894f831fd9b945032f3ccd5ce0630c9e4673313dfb9e41580" exitCode=0 Dec 03 12:17:37 crc kubenswrapper[5052]: I1203 12:17:37.858643 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"614c796f-690e-4c8e-833f-bf7ba7c47d1a","Type":"ContainerDied","Data":"e152d21042dfbb59504e298835f095942b27ff95b7e38c26f308eec742685448"} Dec 03 12:17:37 crc kubenswrapper[5052]: I1203 12:17:37.858744 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"614c796f-690e-4c8e-833f-bf7ba7c47d1a","Type":"ContainerDied","Data":"c8beb7f8c222fb4c94ee6aadf2ded6eb09f46f942e334eb694d5a1683fcf80f9"} Dec 03 12:17:37 crc kubenswrapper[5052]: I1203 12:17:37.858758 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"614c796f-690e-4c8e-833f-bf7ba7c47d1a","Type":"ContainerDied","Data":"ecb8cd2f428a35b04fce42b144a6f3606e98b3dcaa31eb3623b92627a69a0813"} Dec 03 12:17:37 crc kubenswrapper[5052]: I1203 12:17:37.858770 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"614c796f-690e-4c8e-833f-bf7ba7c47d1a","Type":"ContainerDied","Data":"d26b41a42104863894f831fd9b945032f3ccd5ce0630c9e4673313dfb9e41580"} Dec 03 12:17:37 crc kubenswrapper[5052]: I1203 12:17:37.858780 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"614c796f-690e-4c8e-833f-bf7ba7c47d1a","Type":"ContainerDied","Data":"baec3061547d2412c64a59a314db756ac34d9a235835cd5adbe0fdfa0faf5bbf"} Dec 03 12:17:37 crc kubenswrapper[5052]: I1203 12:17:37.858791 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="baec3061547d2412c64a59a314db756ac34d9a235835cd5adbe0fdfa0faf5bbf" Dec 03 12:17:37 crc kubenswrapper[5052]: I1203 12:17:37.906304 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:17:37 crc kubenswrapper[5052]: I1203 12:17:37.914685 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/placement-7d6bd9d7c9-n2lr6"] Dec 03 12:17:37 crc kubenswrapper[5052]: E1203 12:17:37.915106 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="614c796f-690e-4c8e-833f-bf7ba7c47d1a" containerName="ceilometer-notification-agent" Dec 03 12:17:37 crc kubenswrapper[5052]: I1203 12:17:37.915122 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="614c796f-690e-4c8e-833f-bf7ba7c47d1a" containerName="ceilometer-notification-agent" Dec 03 12:17:37 crc kubenswrapper[5052]: E1203 12:17:37.915140 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="9f563ce4-a788-44eb-bdab-9fc4c3423176" containerName="placement-db-sync" Dec 03 12:17:37 crc kubenswrapper[5052]: I1203 12:17:37.915146 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="9f563ce4-a788-44eb-bdab-9fc4c3423176" containerName="placement-db-sync" Dec 03 12:17:37 crc kubenswrapper[5052]: E1203 12:17:37.915171 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="614c796f-690e-4c8e-833f-bf7ba7c47d1a" containerName="sg-core" Dec 03 12:17:37 crc kubenswrapper[5052]: I1203 12:17:37.915178 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="614c796f-690e-4c8e-833f-bf7ba7c47d1a" containerName="sg-core" Dec 03 12:17:37 crc kubenswrapper[5052]: E1203 12:17:37.915189 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="614c796f-690e-4c8e-833f-bf7ba7c47d1a" containerName="proxy-httpd" Dec 03 12:17:37 crc kubenswrapper[5052]: I1203 12:17:37.915194 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="614c796f-690e-4c8e-833f-bf7ba7c47d1a" containerName="proxy-httpd" Dec 03 12:17:37 crc kubenswrapper[5052]: E1203 12:17:37.915205 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="614c796f-690e-4c8e-833f-bf7ba7c47d1a" containerName="ceilometer-central-agent" Dec 03 12:17:37 crc kubenswrapper[5052]: I1203 12:17:37.915211 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="614c796f-690e-4c8e-833f-bf7ba7c47d1a" containerName="ceilometer-central-agent" Dec 03 12:17:37 crc kubenswrapper[5052]: I1203 12:17:37.915369 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="614c796f-690e-4c8e-833f-bf7ba7c47d1a" containerName="sg-core" Dec 03 12:17:37 crc kubenswrapper[5052]: I1203 12:17:37.915385 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="614c796f-690e-4c8e-833f-bf7ba7c47d1a" containerName="ceilometer-central-agent" Dec 03 12:17:37 crc kubenswrapper[5052]: I1203 12:17:37.915399 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="9f563ce4-a788-44eb-bdab-9fc4c3423176" containerName="placement-db-sync" Dec 03 12:17:37 crc kubenswrapper[5052]: I1203 12:17:37.915410 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="614c796f-690e-4c8e-833f-bf7ba7c47d1a" containerName="ceilometer-notification-agent" Dec 03 12:17:37 crc kubenswrapper[5052]: I1203 12:17:37.915421 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="614c796f-690e-4c8e-833f-bf7ba7c47d1a" containerName="proxy-httpd" Dec 03 12:17:37 crc kubenswrapper[5052]: I1203 12:17:37.916568 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-7d6bd9d7c9-n2lr6" Dec 03 12:17:37 crc kubenswrapper[5052]: I1203 12:17:37.931335 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/placement-7d6bd9d7c9-n2lr6"] Dec 03 12:17:37 crc kubenswrapper[5052]: I1203 12:17:37.937000 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-placement-internal-svc" Dec 03 12:17:37 crc kubenswrapper[5052]: I1203 12:17:37.937449 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"placement-config-data" Dec 03 12:17:37 crc kubenswrapper[5052]: I1203 12:17:37.937701 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-placement-public-svc" Dec 03 12:17:37 crc kubenswrapper[5052]: I1203 12:17:37.937929 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"placement-placement-dockercfg-j6csr" Dec 03 12:17:37 crc kubenswrapper[5052]: I1203 12:17:37.938209 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"placement-scripts" Dec 03 12:17:38 crc kubenswrapper[5052]: I1203 12:17:38.025768 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:17:38 crc kubenswrapper[5052]: W1203 12:17:38.029270 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf65abb68_4704_4c08_b1eb_da1ac582a4d8.slice/crio-e52bb54c85721046eec5069fe89bc4080b3807de715a2779ff1e00ebf84e55df WatchSource:0}: Error finding container e52bb54c85721046eec5069fe89bc4080b3807de715a2779ff1e00ebf84e55df: Status 404 returned error can't find the container with id e52bb54c85721046eec5069fe89bc4080b3807de715a2779ff1e00ebf84e55df Dec 03 12:17:38 crc kubenswrapper[5052]: I1203 12:17:38.034136 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/614c796f-690e-4c8e-833f-bf7ba7c47d1a-run-httpd\") pod \"614c796f-690e-4c8e-833f-bf7ba7c47d1a\" (UID: \"614c796f-690e-4c8e-833f-bf7ba7c47d1a\") " Dec 03 12:17:38 crc kubenswrapper[5052]: I1203 12:17:38.034313 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hkvxn\" (UniqueName: \"kubernetes.io/projected/614c796f-690e-4c8e-833f-bf7ba7c47d1a-kube-api-access-hkvxn\") pod \"614c796f-690e-4c8e-833f-bf7ba7c47d1a\" (UID: \"614c796f-690e-4c8e-833f-bf7ba7c47d1a\") " Dec 03 12:17:38 crc kubenswrapper[5052]: I1203 12:17:38.034380 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/614c796f-690e-4c8e-833f-bf7ba7c47d1a-sg-core-conf-yaml\") pod \"614c796f-690e-4c8e-833f-bf7ba7c47d1a\" (UID: \"614c796f-690e-4c8e-833f-bf7ba7c47d1a\") " Dec 03 12:17:38 crc kubenswrapper[5052]: I1203 12:17:38.034400 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/614c796f-690e-4c8e-833f-bf7ba7c47d1a-config-data\") pod \"614c796f-690e-4c8e-833f-bf7ba7c47d1a\" (UID: \"614c796f-690e-4c8e-833f-bf7ba7c47d1a\") " Dec 03 12:17:38 crc kubenswrapper[5052]: I1203 12:17:38.034418 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/614c796f-690e-4c8e-833f-bf7ba7c47d1a-log-httpd\") pod \"614c796f-690e-4c8e-833f-bf7ba7c47d1a\" (UID: \"614c796f-690e-4c8e-833f-bf7ba7c47d1a\") " Dec 03 12:17:38 crc kubenswrapper[5052]: I1203 12:17:38.034439 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/614c796f-690e-4c8e-833f-bf7ba7c47d1a-scripts\") pod \"614c796f-690e-4c8e-833f-bf7ba7c47d1a\" (UID: \"614c796f-690e-4c8e-833f-bf7ba7c47d1a\") " Dec 03 12:17:38 crc kubenswrapper[5052]: I1203 12:17:38.034482 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/614c796f-690e-4c8e-833f-bf7ba7c47d1a-combined-ca-bundle\") pod \"614c796f-690e-4c8e-833f-bf7ba7c47d1a\" (UID: \"614c796f-690e-4c8e-833f-bf7ba7c47d1a\") " Dec 03 12:17:38 crc kubenswrapper[5052]: I1203 12:17:38.034699 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/803e4fd4-6dec-4eca-b15b-10391318f545-scripts\") pod \"placement-7d6bd9d7c9-n2lr6\" (UID: \"803e4fd4-6dec-4eca-b15b-10391318f545\") " pod="openstack-kuttl-tests/placement-7d6bd9d7c9-n2lr6" Dec 03 12:17:38 crc kubenswrapper[5052]: I1203 12:17:38.034755 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/803e4fd4-6dec-4eca-b15b-10391318f545-public-tls-certs\") pod \"placement-7d6bd9d7c9-n2lr6\" (UID: \"803e4fd4-6dec-4eca-b15b-10391318f545\") " pod="openstack-kuttl-tests/placement-7d6bd9d7c9-n2lr6" Dec 03 12:17:38 crc kubenswrapper[5052]: I1203 12:17:38.034783 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/803e4fd4-6dec-4eca-b15b-10391318f545-config-data\") pod \"placement-7d6bd9d7c9-n2lr6\" (UID: \"803e4fd4-6dec-4eca-b15b-10391318f545\") " pod="openstack-kuttl-tests/placement-7d6bd9d7c9-n2lr6" Dec 03 12:17:38 crc kubenswrapper[5052]: I1203 12:17:38.034821 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/803e4fd4-6dec-4eca-b15b-10391318f545-logs\") pod \"placement-7d6bd9d7c9-n2lr6\" (UID: \"803e4fd4-6dec-4eca-b15b-10391318f545\") " pod="openstack-kuttl-tests/placement-7d6bd9d7c9-n2lr6" Dec 03 12:17:38 crc kubenswrapper[5052]: I1203 12:17:38.034852 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/803e4fd4-6dec-4eca-b15b-10391318f545-internal-tls-certs\") pod \"placement-7d6bd9d7c9-n2lr6\" (UID: \"803e4fd4-6dec-4eca-b15b-10391318f545\") " pod="openstack-kuttl-tests/placement-7d6bd9d7c9-n2lr6" Dec 03 12:17:38 crc kubenswrapper[5052]: I1203 12:17:38.034872 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/803e4fd4-6dec-4eca-b15b-10391318f545-combined-ca-bundle\") pod \"placement-7d6bd9d7c9-n2lr6\" (UID: \"803e4fd4-6dec-4eca-b15b-10391318f545\") " pod="openstack-kuttl-tests/placement-7d6bd9d7c9-n2lr6" Dec 03 12:17:38 crc kubenswrapper[5052]: I1203 12:17:38.034922 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jqc74\" (UniqueName: \"kubernetes.io/projected/803e4fd4-6dec-4eca-b15b-10391318f545-kube-api-access-jqc74\") pod \"placement-7d6bd9d7c9-n2lr6\" (UID: \"803e4fd4-6dec-4eca-b15b-10391318f545\") " pod="openstack-kuttl-tests/placement-7d6bd9d7c9-n2lr6" Dec 03 12:17:38 crc kubenswrapper[5052]: I1203 12:17:38.035579 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/614c796f-690e-4c8e-833f-bf7ba7c47d1a-log-httpd" (OuterVolumeSpecName: "log-httpd") pod "614c796f-690e-4c8e-833f-bf7ba7c47d1a" (UID: "614c796f-690e-4c8e-833f-bf7ba7c47d1a"). InnerVolumeSpecName "log-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:17:38 crc kubenswrapper[5052]: I1203 12:17:38.035773 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/614c796f-690e-4c8e-833f-bf7ba7c47d1a-run-httpd" (OuterVolumeSpecName: "run-httpd") pod "614c796f-690e-4c8e-833f-bf7ba7c47d1a" (UID: "614c796f-690e-4c8e-833f-bf7ba7c47d1a"). InnerVolumeSpecName "run-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:17:38 crc kubenswrapper[5052]: I1203 12:17:38.040045 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/614c796f-690e-4c8e-833f-bf7ba7c47d1a-scripts" (OuterVolumeSpecName: "scripts") pod "614c796f-690e-4c8e-833f-bf7ba7c47d1a" (UID: "614c796f-690e-4c8e-833f-bf7ba7c47d1a"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:17:38 crc kubenswrapper[5052]: I1203 12:17:38.042604 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/614c796f-690e-4c8e-833f-bf7ba7c47d1a-kube-api-access-hkvxn" (OuterVolumeSpecName: "kube-api-access-hkvxn") pod "614c796f-690e-4c8e-833f-bf7ba7c47d1a" (UID: "614c796f-690e-4c8e-833f-bf7ba7c47d1a"). InnerVolumeSpecName "kube-api-access-hkvxn". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:17:38 crc kubenswrapper[5052]: I1203 12:17:38.092319 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/614c796f-690e-4c8e-833f-bf7ba7c47d1a-sg-core-conf-yaml" (OuterVolumeSpecName: "sg-core-conf-yaml") pod "614c796f-690e-4c8e-833f-bf7ba7c47d1a" (UID: "614c796f-690e-4c8e-833f-bf7ba7c47d1a"). InnerVolumeSpecName "sg-core-conf-yaml". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:17:38 crc kubenswrapper[5052]: I1203 12:17:38.130614 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/614c796f-690e-4c8e-833f-bf7ba7c47d1a-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "614c796f-690e-4c8e-833f-bf7ba7c47d1a" (UID: "614c796f-690e-4c8e-833f-bf7ba7c47d1a"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:17:38 crc kubenswrapper[5052]: I1203 12:17:38.141764 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/803e4fd4-6dec-4eca-b15b-10391318f545-public-tls-certs\") pod \"placement-7d6bd9d7c9-n2lr6\" (UID: \"803e4fd4-6dec-4eca-b15b-10391318f545\") " pod="openstack-kuttl-tests/placement-7d6bd9d7c9-n2lr6" Dec 03 12:17:38 crc kubenswrapper[5052]: I1203 12:17:38.141827 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/803e4fd4-6dec-4eca-b15b-10391318f545-config-data\") pod \"placement-7d6bd9d7c9-n2lr6\" (UID: \"803e4fd4-6dec-4eca-b15b-10391318f545\") " pod="openstack-kuttl-tests/placement-7d6bd9d7c9-n2lr6" Dec 03 12:17:38 crc kubenswrapper[5052]: I1203 12:17:38.141886 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/803e4fd4-6dec-4eca-b15b-10391318f545-logs\") pod \"placement-7d6bd9d7c9-n2lr6\" (UID: \"803e4fd4-6dec-4eca-b15b-10391318f545\") " pod="openstack-kuttl-tests/placement-7d6bd9d7c9-n2lr6" Dec 03 12:17:38 crc kubenswrapper[5052]: I1203 12:17:38.141932 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/803e4fd4-6dec-4eca-b15b-10391318f545-internal-tls-certs\") pod \"placement-7d6bd9d7c9-n2lr6\" (UID: \"803e4fd4-6dec-4eca-b15b-10391318f545\") " pod="openstack-kuttl-tests/placement-7d6bd9d7c9-n2lr6" Dec 03 12:17:38 crc kubenswrapper[5052]: I1203 12:17:38.141961 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/803e4fd4-6dec-4eca-b15b-10391318f545-combined-ca-bundle\") pod \"placement-7d6bd9d7c9-n2lr6\" (UID: \"803e4fd4-6dec-4eca-b15b-10391318f545\") " pod="openstack-kuttl-tests/placement-7d6bd9d7c9-n2lr6" Dec 03 12:17:38 crc kubenswrapper[5052]: I1203 12:17:38.142346 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-jqc74\" (UniqueName: \"kubernetes.io/projected/803e4fd4-6dec-4eca-b15b-10391318f545-kube-api-access-jqc74\") pod \"placement-7d6bd9d7c9-n2lr6\" (UID: \"803e4fd4-6dec-4eca-b15b-10391318f545\") " pod="openstack-kuttl-tests/placement-7d6bd9d7c9-n2lr6" Dec 03 12:17:38 crc kubenswrapper[5052]: I1203 12:17:38.142804 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/803e4fd4-6dec-4eca-b15b-10391318f545-logs\") pod \"placement-7d6bd9d7c9-n2lr6\" (UID: \"803e4fd4-6dec-4eca-b15b-10391318f545\") " pod="openstack-kuttl-tests/placement-7d6bd9d7c9-n2lr6" Dec 03 12:17:38 crc kubenswrapper[5052]: I1203 12:17:38.143026 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/803e4fd4-6dec-4eca-b15b-10391318f545-scripts\") pod \"placement-7d6bd9d7c9-n2lr6\" (UID: \"803e4fd4-6dec-4eca-b15b-10391318f545\") " pod="openstack-kuttl-tests/placement-7d6bd9d7c9-n2lr6" Dec 03 12:17:38 crc kubenswrapper[5052]: I1203 12:17:38.143378 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-hkvxn\" (UniqueName: \"kubernetes.io/projected/614c796f-690e-4c8e-833f-bf7ba7c47d1a-kube-api-access-hkvxn\") on node \"crc\" DevicePath \"\"" Dec 03 12:17:38 crc kubenswrapper[5052]: I1203 12:17:38.145360 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/803e4fd4-6dec-4eca-b15b-10391318f545-config-data\") pod \"placement-7d6bd9d7c9-n2lr6\" (UID: \"803e4fd4-6dec-4eca-b15b-10391318f545\") " pod="openstack-kuttl-tests/placement-7d6bd9d7c9-n2lr6" Dec 03 12:17:38 crc kubenswrapper[5052]: I1203 12:17:38.145425 5052 reconciler_common.go:293] "Volume detached for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/614c796f-690e-4c8e-833f-bf7ba7c47d1a-sg-core-conf-yaml\") on node \"crc\" DevicePath \"\"" Dec 03 12:17:38 crc kubenswrapper[5052]: I1203 12:17:38.145442 5052 reconciler_common.go:293] "Volume detached for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/614c796f-690e-4c8e-833f-bf7ba7c47d1a-log-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:17:38 crc kubenswrapper[5052]: I1203 12:17:38.145453 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/614c796f-690e-4c8e-833f-bf7ba7c47d1a-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:17:38 crc kubenswrapper[5052]: I1203 12:17:38.145465 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/614c796f-690e-4c8e-833f-bf7ba7c47d1a-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:17:38 crc kubenswrapper[5052]: I1203 12:17:38.145477 5052 reconciler_common.go:293] "Volume detached for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/614c796f-690e-4c8e-833f-bf7ba7c47d1a-run-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:17:38 crc kubenswrapper[5052]: I1203 12:17:38.146542 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/803e4fd4-6dec-4eca-b15b-10391318f545-public-tls-certs\") pod \"placement-7d6bd9d7c9-n2lr6\" (UID: \"803e4fd4-6dec-4eca-b15b-10391318f545\") " pod="openstack-kuttl-tests/placement-7d6bd9d7c9-n2lr6" Dec 03 12:17:38 crc kubenswrapper[5052]: I1203 12:17:38.148919 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/803e4fd4-6dec-4eca-b15b-10391318f545-internal-tls-certs\") pod \"placement-7d6bd9d7c9-n2lr6\" (UID: \"803e4fd4-6dec-4eca-b15b-10391318f545\") " pod="openstack-kuttl-tests/placement-7d6bd9d7c9-n2lr6" Dec 03 12:17:38 crc kubenswrapper[5052]: I1203 12:17:38.164131 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/803e4fd4-6dec-4eca-b15b-10391318f545-combined-ca-bundle\") pod \"placement-7d6bd9d7c9-n2lr6\" (UID: \"803e4fd4-6dec-4eca-b15b-10391318f545\") " pod="openstack-kuttl-tests/placement-7d6bd9d7c9-n2lr6" Dec 03 12:17:38 crc kubenswrapper[5052]: I1203 12:17:38.164365 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/803e4fd4-6dec-4eca-b15b-10391318f545-scripts\") pod \"placement-7d6bd9d7c9-n2lr6\" (UID: \"803e4fd4-6dec-4eca-b15b-10391318f545\") " pod="openstack-kuttl-tests/placement-7d6bd9d7c9-n2lr6" Dec 03 12:17:38 crc kubenswrapper[5052]: I1203 12:17:38.177968 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-jqc74\" (UniqueName: \"kubernetes.io/projected/803e4fd4-6dec-4eca-b15b-10391318f545-kube-api-access-jqc74\") pod \"placement-7d6bd9d7c9-n2lr6\" (UID: \"803e4fd4-6dec-4eca-b15b-10391318f545\") " pod="openstack-kuttl-tests/placement-7d6bd9d7c9-n2lr6" Dec 03 12:17:38 crc kubenswrapper[5052]: I1203 12:17:38.189801 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/614c796f-690e-4c8e-833f-bf7ba7c47d1a-config-data" (OuterVolumeSpecName: "config-data") pod "614c796f-690e-4c8e-833f-bf7ba7c47d1a" (UID: "614c796f-690e-4c8e-833f-bf7ba7c47d1a"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:17:38 crc kubenswrapper[5052]: I1203 12:17:38.247658 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/614c796f-690e-4c8e-833f-bf7ba7c47d1a-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:17:38 crc kubenswrapper[5052]: I1203 12:17:38.267747 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-7d6bd9d7c9-n2lr6" Dec 03 12:17:38 crc kubenswrapper[5052]: I1203 12:17:38.320493 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-db-sync-pnhqf" Dec 03 12:17:38 crc kubenswrapper[5052]: I1203 12:17:38.384961 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-bootstrap-5qw5w" Dec 03 12:17:38 crc kubenswrapper[5052]: I1203 12:17:38.450088 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bz22k\" (UniqueName: \"kubernetes.io/projected/38a5d722-dbe5-4d1e-9aec-bf6513db82fb-kube-api-access-bz22k\") pod \"38a5d722-dbe5-4d1e-9aec-bf6513db82fb\" (UID: \"38a5d722-dbe5-4d1e-9aec-bf6513db82fb\") " Dec 03 12:17:38 crc kubenswrapper[5052]: I1203 12:17:38.450563 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nzhhv\" (UniqueName: \"kubernetes.io/projected/2f98071b-1e7d-4cf3-976e-ad9073a2f914-kube-api-access-nzhhv\") pod \"2f98071b-1e7d-4cf3-976e-ad9073a2f914\" (UID: \"2f98071b-1e7d-4cf3-976e-ad9073a2f914\") " Dec 03 12:17:38 crc kubenswrapper[5052]: I1203 12:17:38.450609 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/38a5d722-dbe5-4d1e-9aec-bf6513db82fb-scripts\") pod \"38a5d722-dbe5-4d1e-9aec-bf6513db82fb\" (UID: \"38a5d722-dbe5-4d1e-9aec-bf6513db82fb\") " Dec 03 12:17:38 crc kubenswrapper[5052]: I1203 12:17:38.450636 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2f98071b-1e7d-4cf3-976e-ad9073a2f914-combined-ca-bundle\") pod \"2f98071b-1e7d-4cf3-976e-ad9073a2f914\" (UID: \"2f98071b-1e7d-4cf3-976e-ad9073a2f914\") " Dec 03 12:17:38 crc kubenswrapper[5052]: I1203 12:17:38.450655 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/38a5d722-dbe5-4d1e-9aec-bf6513db82fb-credential-keys\") pod \"38a5d722-dbe5-4d1e-9aec-bf6513db82fb\" (UID: \"38a5d722-dbe5-4d1e-9aec-bf6513db82fb\") " Dec 03 12:17:38 crc kubenswrapper[5052]: I1203 12:17:38.450689 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/38a5d722-dbe5-4d1e-9aec-bf6513db82fb-config-data\") pod \"38a5d722-dbe5-4d1e-9aec-bf6513db82fb\" (UID: \"38a5d722-dbe5-4d1e-9aec-bf6513db82fb\") " Dec 03 12:17:38 crc kubenswrapper[5052]: I1203 12:17:38.450718 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/38a5d722-dbe5-4d1e-9aec-bf6513db82fb-combined-ca-bundle\") pod \"38a5d722-dbe5-4d1e-9aec-bf6513db82fb\" (UID: \"38a5d722-dbe5-4d1e-9aec-bf6513db82fb\") " Dec 03 12:17:38 crc kubenswrapper[5052]: I1203 12:17:38.450745 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/38a5d722-dbe5-4d1e-9aec-bf6513db82fb-fernet-keys\") pod \"38a5d722-dbe5-4d1e-9aec-bf6513db82fb\" (UID: \"38a5d722-dbe5-4d1e-9aec-bf6513db82fb\") " Dec 03 12:17:38 crc kubenswrapper[5052]: I1203 12:17:38.450805 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/2f98071b-1e7d-4cf3-976e-ad9073a2f914-db-sync-config-data\") pod \"2f98071b-1e7d-4cf3-976e-ad9073a2f914\" (UID: \"2f98071b-1e7d-4cf3-976e-ad9073a2f914\") " Dec 03 12:17:38 crc kubenswrapper[5052]: I1203 12:17:38.462416 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/38a5d722-dbe5-4d1e-9aec-bf6513db82fb-fernet-keys" (OuterVolumeSpecName: "fernet-keys") pod "38a5d722-dbe5-4d1e-9aec-bf6513db82fb" (UID: "38a5d722-dbe5-4d1e-9aec-bf6513db82fb"). InnerVolumeSpecName "fernet-keys". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:17:38 crc kubenswrapper[5052]: I1203 12:17:38.465731 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/38a5d722-dbe5-4d1e-9aec-bf6513db82fb-scripts" (OuterVolumeSpecName: "scripts") pod "38a5d722-dbe5-4d1e-9aec-bf6513db82fb" (UID: "38a5d722-dbe5-4d1e-9aec-bf6513db82fb"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:17:38 crc kubenswrapper[5052]: I1203 12:17:38.469715 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/38a5d722-dbe5-4d1e-9aec-bf6513db82fb-kube-api-access-bz22k" (OuterVolumeSpecName: "kube-api-access-bz22k") pod "38a5d722-dbe5-4d1e-9aec-bf6513db82fb" (UID: "38a5d722-dbe5-4d1e-9aec-bf6513db82fb"). InnerVolumeSpecName "kube-api-access-bz22k". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:17:38 crc kubenswrapper[5052]: I1203 12:17:38.469817 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2f98071b-1e7d-4cf3-976e-ad9073a2f914-db-sync-config-data" (OuterVolumeSpecName: "db-sync-config-data") pod "2f98071b-1e7d-4cf3-976e-ad9073a2f914" (UID: "2f98071b-1e7d-4cf3-976e-ad9073a2f914"). InnerVolumeSpecName "db-sync-config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:17:38 crc kubenswrapper[5052]: I1203 12:17:38.469714 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/38a5d722-dbe5-4d1e-9aec-bf6513db82fb-credential-keys" (OuterVolumeSpecName: "credential-keys") pod "38a5d722-dbe5-4d1e-9aec-bf6513db82fb" (UID: "38a5d722-dbe5-4d1e-9aec-bf6513db82fb"). InnerVolumeSpecName "credential-keys". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:17:38 crc kubenswrapper[5052]: I1203 12:17:38.469805 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2f98071b-1e7d-4cf3-976e-ad9073a2f914-kube-api-access-nzhhv" (OuterVolumeSpecName: "kube-api-access-nzhhv") pod "2f98071b-1e7d-4cf3-976e-ad9073a2f914" (UID: "2f98071b-1e7d-4cf3-976e-ad9073a2f914"). InnerVolumeSpecName "kube-api-access-nzhhv". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:17:38 crc kubenswrapper[5052]: I1203 12:17:38.489648 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/38a5d722-dbe5-4d1e-9aec-bf6513db82fb-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "38a5d722-dbe5-4d1e-9aec-bf6513db82fb" (UID: "38a5d722-dbe5-4d1e-9aec-bf6513db82fb"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:17:38 crc kubenswrapper[5052]: I1203 12:17:38.494785 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2f98071b-1e7d-4cf3-976e-ad9073a2f914-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "2f98071b-1e7d-4cf3-976e-ad9073a2f914" (UID: "2f98071b-1e7d-4cf3-976e-ad9073a2f914"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:17:38 crc kubenswrapper[5052]: I1203 12:17:38.496692 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/38a5d722-dbe5-4d1e-9aec-bf6513db82fb-config-data" (OuterVolumeSpecName: "config-data") pod "38a5d722-dbe5-4d1e-9aec-bf6513db82fb" (UID: "38a5d722-dbe5-4d1e-9aec-bf6513db82fb"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:17:38 crc kubenswrapper[5052]: I1203 12:17:38.552909 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/38a5d722-dbe5-4d1e-9aec-bf6513db82fb-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:17:38 crc kubenswrapper[5052]: I1203 12:17:38.552936 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/38a5d722-dbe5-4d1e-9aec-bf6513db82fb-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:17:38 crc kubenswrapper[5052]: I1203 12:17:38.552945 5052 reconciler_common.go:293] "Volume detached for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/38a5d722-dbe5-4d1e-9aec-bf6513db82fb-fernet-keys\") on node \"crc\" DevicePath \"\"" Dec 03 12:17:38 crc kubenswrapper[5052]: I1203 12:17:38.552954 5052 reconciler_common.go:293] "Volume detached for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/2f98071b-1e7d-4cf3-976e-ad9073a2f914-db-sync-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:17:38 crc kubenswrapper[5052]: I1203 12:17:38.552964 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-bz22k\" (UniqueName: \"kubernetes.io/projected/38a5d722-dbe5-4d1e-9aec-bf6513db82fb-kube-api-access-bz22k\") on node \"crc\" DevicePath \"\"" Dec 03 12:17:38 crc kubenswrapper[5052]: I1203 12:17:38.552974 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-nzhhv\" (UniqueName: \"kubernetes.io/projected/2f98071b-1e7d-4cf3-976e-ad9073a2f914-kube-api-access-nzhhv\") on node \"crc\" DevicePath \"\"" Dec 03 12:17:38 crc kubenswrapper[5052]: I1203 12:17:38.552982 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/38a5d722-dbe5-4d1e-9aec-bf6513db82fb-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:17:38 crc kubenswrapper[5052]: I1203 12:17:38.552990 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2f98071b-1e7d-4cf3-976e-ad9073a2f914-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:17:38 crc kubenswrapper[5052]: I1203 12:17:38.553000 5052 reconciler_common.go:293] "Volume detached for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/38a5d722-dbe5-4d1e-9aec-bf6513db82fb-credential-keys\") on node \"crc\" DevicePath \"\"" Dec 03 12:17:38 crc kubenswrapper[5052]: I1203 12:17:38.766876 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/placement-7d6bd9d7c9-n2lr6"] Dec 03 12:17:38 crc kubenswrapper[5052]: I1203 12:17:38.883352 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-7d6bd9d7c9-n2lr6" event={"ID":"803e4fd4-6dec-4eca-b15b-10391318f545","Type":"ContainerStarted","Data":"fa6ca5da9d1fcb97c4f48bcd0336274c8f5c21c365e96c8637b319169134d35e"} Dec 03 12:17:38 crc kubenswrapper[5052]: I1203 12:17:38.909773 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"cd9a6dc3-a814-4a29-924a-d8ada756e7c7","Type":"ContainerStarted","Data":"9d3ac55b9014620975c68f93418691e10f6505723457d823dbfb3d7265965d17"} Dec 03 12:17:38 crc kubenswrapper[5052]: I1203 12:17:38.916386 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-bootstrap-5qw5w" event={"ID":"38a5d722-dbe5-4d1e-9aec-bf6513db82fb","Type":"ContainerDied","Data":"c3c0ae4cd1db927c0fa2fc6e45fe8e973f1dae47b5bad44a95215a9c11ce0c61"} Dec 03 12:17:38 crc kubenswrapper[5052]: I1203 12:17:38.916424 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c3c0ae4cd1db927c0fa2fc6e45fe8e973f1dae47b5bad44a95215a9c11ce0c61" Dec 03 12:17:38 crc kubenswrapper[5052]: I1203 12:17:38.916486 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-bootstrap-5qw5w" Dec 03 12:17:38 crc kubenswrapper[5052]: I1203 12:17:38.927565 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"f65abb68-4704-4c08-b1eb-da1ac582a4d8","Type":"ContainerStarted","Data":"28597fb29454145069e00cd7412e37a205ba7c5070c6f525254956aa0e9f8ea8"} Dec 03 12:17:38 crc kubenswrapper[5052]: I1203 12:17:38.927633 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"f65abb68-4704-4c08-b1eb-da1ac582a4d8","Type":"ContainerStarted","Data":"e52bb54c85721046eec5069fe89bc4080b3807de715a2779ff1e00ebf84e55df"} Dec 03 12:17:38 crc kubenswrapper[5052]: I1203 12:17:38.933111 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:17:38 crc kubenswrapper[5052]: I1203 12:17:38.939322 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-db-sync-pnhqf" event={"ID":"2f98071b-1e7d-4cf3-976e-ad9073a2f914","Type":"ContainerDied","Data":"ed66b602b37675ef4806ee0a6e812de42767f2a4c5c1228a1e553b32060605b4"} Dec 03 12:17:38 crc kubenswrapper[5052]: I1203 12:17:38.939366 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ed66b602b37675ef4806ee0a6e812de42767f2a4c5c1228a1e553b32060605b4" Dec 03 12:17:38 crc kubenswrapper[5052]: I1203 12:17:38.939436 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-db-sync-pnhqf" Dec 03 12:17:38 crc kubenswrapper[5052]: I1203 12:17:38.958886 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/glance-default-external-api-0" podStartSLOduration=3.958868025 podStartE2EDuration="3.958868025s" podCreationTimestamp="2025-12-03 12:17:35 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:17:38.947745157 +0000 UTC m=+3449.535986911" watchObservedRunningTime="2025-12-03 12:17:38.958868025 +0000 UTC m=+3449.547109789" Dec 03 12:17:39 crc kubenswrapper[5052]: I1203 12:17:39.056350 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/keystone-bootstrap-5qw5w"] Dec 03 12:17:39 crc kubenswrapper[5052]: I1203 12:17:39.116375 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/keystone-bootstrap-5qw5w"] Dec 03 12:17:39 crc kubenswrapper[5052]: I1203 12:17:39.175968 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/barbican-worker-76d6bdd486-g9c2p"] Dec 03 12:17:39 crc kubenswrapper[5052]: E1203 12:17:39.176969 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="38a5d722-dbe5-4d1e-9aec-bf6513db82fb" containerName="keystone-bootstrap" Dec 03 12:17:39 crc kubenswrapper[5052]: I1203 12:17:39.176999 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="38a5d722-dbe5-4d1e-9aec-bf6513db82fb" containerName="keystone-bootstrap" Dec 03 12:17:39 crc kubenswrapper[5052]: E1203 12:17:39.177041 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="2f98071b-1e7d-4cf3-976e-ad9073a2f914" containerName="barbican-db-sync" Dec 03 12:17:39 crc kubenswrapper[5052]: I1203 12:17:39.177052 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f98071b-1e7d-4cf3-976e-ad9073a2f914" containerName="barbican-db-sync" Dec 03 12:17:39 crc kubenswrapper[5052]: I1203 12:17:39.177462 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="38a5d722-dbe5-4d1e-9aec-bf6513db82fb" containerName="keystone-bootstrap" Dec 03 12:17:39 crc kubenswrapper[5052]: I1203 12:17:39.177507 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="2f98071b-1e7d-4cf3-976e-ad9073a2f914" containerName="barbican-db-sync" Dec 03 12:17:39 crc kubenswrapper[5052]: I1203 12:17:39.179638 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-worker-76d6bdd486-g9c2p" Dec 03 12:17:39 crc kubenswrapper[5052]: I1203 12:17:39.190116 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"barbican-worker-config-data" Dec 03 12:17:39 crc kubenswrapper[5052]: I1203 12:17:39.190468 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"barbican-barbican-dockercfg-jqhml" Dec 03 12:17:39 crc kubenswrapper[5052]: I1203 12:17:39.190748 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"barbican-config-data" Dec 03 12:17:39 crc kubenswrapper[5052]: I1203 12:17:39.209809 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/barbican-keystone-listener-55bfbc4797-wjrjr"] Dec 03 12:17:39 crc kubenswrapper[5052]: I1203 12:17:39.213104 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-keystone-listener-55bfbc4797-wjrjr" Dec 03 12:17:39 crc kubenswrapper[5052]: I1203 12:17:39.216062 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"barbican-keystone-listener-config-data" Dec 03 12:17:39 crc kubenswrapper[5052]: I1203 12:17:39.234199 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/keystone-bootstrap-cz9xr"] Dec 03 12:17:39 crc kubenswrapper[5052]: I1203 12:17:39.237156 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-bootstrap-cz9xr" Dec 03 12:17:39 crc kubenswrapper[5052]: I1203 12:17:39.241670 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone-scripts" Dec 03 12:17:39 crc kubenswrapper[5052]: I1203 12:17:39.241722 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone-keystone-dockercfg-kkks7" Dec 03 12:17:39 crc kubenswrapper[5052]: I1203 12:17:39.241670 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"osp-secret" Dec 03 12:17:39 crc kubenswrapper[5052]: I1203 12:17:39.242193 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone-config-data" Dec 03 12:17:39 crc kubenswrapper[5052]: I1203 12:17:39.242292 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone" Dec 03 12:17:39 crc kubenswrapper[5052]: I1203 12:17:39.258816 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-worker-76d6bdd486-g9c2p"] Dec 03 12:17:39 crc kubenswrapper[5052]: I1203 12:17:39.281732 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-bootstrap-cz9xr"] Dec 03 12:17:39 crc kubenswrapper[5052]: I1203 12:17:39.289660 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-keystone-listener-55bfbc4797-wjrjr"] Dec 03 12:17:39 crc kubenswrapper[5052]: I1203 12:17:39.299668 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:17:39 crc kubenswrapper[5052]: I1203 12:17:39.300101 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/fcc1e62d-c1cc-49a0-93f4-5112fddf89bf-config-data-custom\") pod \"barbican-keystone-listener-55bfbc4797-wjrjr\" (UID: \"fcc1e62d-c1cc-49a0-93f4-5112fddf89bf\") " pod="openstack-kuttl-tests/barbican-keystone-listener-55bfbc4797-wjrjr" Dec 03 12:17:39 crc kubenswrapper[5052]: I1203 12:17:39.300146 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/311e25fa-0785-45ce-ae3c-00d32adbf3f4-config-data\") pod \"barbican-worker-76d6bdd486-g9c2p\" (UID: \"311e25fa-0785-45ce-ae3c-00d32adbf3f4\") " pod="openstack-kuttl-tests/barbican-worker-76d6bdd486-g9c2p" Dec 03 12:17:39 crc kubenswrapper[5052]: I1203 12:17:39.300177 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hn8ds\" (UniqueName: \"kubernetes.io/projected/cf5d77fc-8424-4c37-8b5d-de2cf2ffd886-kube-api-access-hn8ds\") pod \"keystone-bootstrap-cz9xr\" (UID: \"cf5d77fc-8424-4c37-8b5d-de2cf2ffd886\") " pod="openstack-kuttl-tests/keystone-bootstrap-cz9xr" Dec 03 12:17:39 crc kubenswrapper[5052]: I1203 12:17:39.300232 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/fcc1e62d-c1cc-49a0-93f4-5112fddf89bf-logs\") pod \"barbican-keystone-listener-55bfbc4797-wjrjr\" (UID: \"fcc1e62d-c1cc-49a0-93f4-5112fddf89bf\") " pod="openstack-kuttl-tests/barbican-keystone-listener-55bfbc4797-wjrjr" Dec 03 12:17:39 crc kubenswrapper[5052]: I1203 12:17:39.300336 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/cf5d77fc-8424-4c37-8b5d-de2cf2ffd886-config-data\") pod \"keystone-bootstrap-cz9xr\" (UID: \"cf5d77fc-8424-4c37-8b5d-de2cf2ffd886\") " pod="openstack-kuttl-tests/keystone-bootstrap-cz9xr" Dec 03 12:17:39 crc kubenswrapper[5052]: I1203 12:17:39.300395 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ckhjp\" (UniqueName: \"kubernetes.io/projected/fcc1e62d-c1cc-49a0-93f4-5112fddf89bf-kube-api-access-ckhjp\") pod \"barbican-keystone-listener-55bfbc4797-wjrjr\" (UID: \"fcc1e62d-c1cc-49a0-93f4-5112fddf89bf\") " pod="openstack-kuttl-tests/barbican-keystone-listener-55bfbc4797-wjrjr" Dec 03 12:17:39 crc kubenswrapper[5052]: I1203 12:17:39.300420 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/fcc1e62d-c1cc-49a0-93f4-5112fddf89bf-combined-ca-bundle\") pod \"barbican-keystone-listener-55bfbc4797-wjrjr\" (UID: \"fcc1e62d-c1cc-49a0-93f4-5112fddf89bf\") " pod="openstack-kuttl-tests/barbican-keystone-listener-55bfbc4797-wjrjr" Dec 03 12:17:39 crc kubenswrapper[5052]: I1203 12:17:39.300457 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/cf5d77fc-8424-4c37-8b5d-de2cf2ffd886-combined-ca-bundle\") pod \"keystone-bootstrap-cz9xr\" (UID: \"cf5d77fc-8424-4c37-8b5d-de2cf2ffd886\") " pod="openstack-kuttl-tests/keystone-bootstrap-cz9xr" Dec 03 12:17:39 crc kubenswrapper[5052]: I1203 12:17:39.300542 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/311e25fa-0785-45ce-ae3c-00d32adbf3f4-logs\") pod \"barbican-worker-76d6bdd486-g9c2p\" (UID: \"311e25fa-0785-45ce-ae3c-00d32adbf3f4\") " pod="openstack-kuttl-tests/barbican-worker-76d6bdd486-g9c2p" Dec 03 12:17:39 crc kubenswrapper[5052]: I1203 12:17:39.300587 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/cf5d77fc-8424-4c37-8b5d-de2cf2ffd886-fernet-keys\") pod \"keystone-bootstrap-cz9xr\" (UID: \"cf5d77fc-8424-4c37-8b5d-de2cf2ffd886\") " pod="openstack-kuttl-tests/keystone-bootstrap-cz9xr" Dec 03 12:17:39 crc kubenswrapper[5052]: I1203 12:17:39.300624 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/cf5d77fc-8424-4c37-8b5d-de2cf2ffd886-credential-keys\") pod \"keystone-bootstrap-cz9xr\" (UID: \"cf5d77fc-8424-4c37-8b5d-de2cf2ffd886\") " pod="openstack-kuttl-tests/keystone-bootstrap-cz9xr" Dec 03 12:17:39 crc kubenswrapper[5052]: I1203 12:17:39.300673 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/cf5d77fc-8424-4c37-8b5d-de2cf2ffd886-scripts\") pod \"keystone-bootstrap-cz9xr\" (UID: \"cf5d77fc-8424-4c37-8b5d-de2cf2ffd886\") " pod="openstack-kuttl-tests/keystone-bootstrap-cz9xr" Dec 03 12:17:39 crc kubenswrapper[5052]: I1203 12:17:39.300713 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/311e25fa-0785-45ce-ae3c-00d32adbf3f4-combined-ca-bundle\") pod \"barbican-worker-76d6bdd486-g9c2p\" (UID: \"311e25fa-0785-45ce-ae3c-00d32adbf3f4\") " pod="openstack-kuttl-tests/barbican-worker-76d6bdd486-g9c2p" Dec 03 12:17:39 crc kubenswrapper[5052]: I1203 12:17:39.300771 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s6scm\" (UniqueName: \"kubernetes.io/projected/311e25fa-0785-45ce-ae3c-00d32adbf3f4-kube-api-access-s6scm\") pod \"barbican-worker-76d6bdd486-g9c2p\" (UID: \"311e25fa-0785-45ce-ae3c-00d32adbf3f4\") " pod="openstack-kuttl-tests/barbican-worker-76d6bdd486-g9c2p" Dec 03 12:17:39 crc kubenswrapper[5052]: I1203 12:17:39.300803 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/fcc1e62d-c1cc-49a0-93f4-5112fddf89bf-config-data\") pod \"barbican-keystone-listener-55bfbc4797-wjrjr\" (UID: \"fcc1e62d-c1cc-49a0-93f4-5112fddf89bf\") " pod="openstack-kuttl-tests/barbican-keystone-listener-55bfbc4797-wjrjr" Dec 03 12:17:39 crc kubenswrapper[5052]: I1203 12:17:39.300829 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/311e25fa-0785-45ce-ae3c-00d32adbf3f4-config-data-custom\") pod \"barbican-worker-76d6bdd486-g9c2p\" (UID: \"311e25fa-0785-45ce-ae3c-00d32adbf3f4\") " pod="openstack-kuttl-tests/barbican-worker-76d6bdd486-g9c2p" Dec 03 12:17:39 crc kubenswrapper[5052]: I1203 12:17:39.314893 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:17:39 crc kubenswrapper[5052]: I1203 12:17:39.331399 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:17:39 crc kubenswrapper[5052]: I1203 12:17:39.333883 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:17:39 crc kubenswrapper[5052]: I1203 12:17:39.337076 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-scripts" Dec 03 12:17:39 crc kubenswrapper[5052]: I1203 12:17:39.339148 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-config-data" Dec 03 12:17:39 crc kubenswrapper[5052]: I1203 12:17:39.342341 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:17:39 crc kubenswrapper[5052]: I1203 12:17:39.355148 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/barbican-api-547ff4f8ff-dpw9k"] Dec 03 12:17:39 crc kubenswrapper[5052]: I1203 12:17:39.357606 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-api-547ff4f8ff-dpw9k" Dec 03 12:17:39 crc kubenswrapper[5052]: I1203 12:17:39.361052 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"barbican-api-config-data" Dec 03 12:17:39 crc kubenswrapper[5052]: I1203 12:17:39.372060 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-api-547ff4f8ff-dpw9k"] Dec 03 12:17:39 crc kubenswrapper[5052]: I1203 12:17:39.402112 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/e36b4a02-2b71-4e70-b890-e91695ec53f3-log-httpd\") pod \"ceilometer-0\" (UID: \"e36b4a02-2b71-4e70-b890-e91695ec53f3\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:17:39 crc kubenswrapper[5052]: I1203 12:17:39.402174 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/fcc1e62d-c1cc-49a0-93f4-5112fddf89bf-logs\") pod \"barbican-keystone-listener-55bfbc4797-wjrjr\" (UID: \"fcc1e62d-c1cc-49a0-93f4-5112fddf89bf\") " pod="openstack-kuttl-tests/barbican-keystone-listener-55bfbc4797-wjrjr" Dec 03 12:17:39 crc kubenswrapper[5052]: I1203 12:17:39.402209 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/e36b4a02-2b71-4e70-b890-e91695ec53f3-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"e36b4a02-2b71-4e70-b890-e91695ec53f3\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:17:39 crc kubenswrapper[5052]: I1203 12:17:39.402243 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tllbc\" (UniqueName: \"kubernetes.io/projected/e36b4a02-2b71-4e70-b890-e91695ec53f3-kube-api-access-tllbc\") pod \"ceilometer-0\" (UID: \"e36b4a02-2b71-4e70-b890-e91695ec53f3\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:17:39 crc kubenswrapper[5052]: I1203 12:17:39.402273 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9k4zc\" (UniqueName: \"kubernetes.io/projected/888ed75c-2d05-4199-b51a-123ac3e44e31-kube-api-access-9k4zc\") pod \"barbican-api-547ff4f8ff-dpw9k\" (UID: \"888ed75c-2d05-4199-b51a-123ac3e44e31\") " pod="openstack-kuttl-tests/barbican-api-547ff4f8ff-dpw9k" Dec 03 12:17:39 crc kubenswrapper[5052]: I1203 12:17:39.402300 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/cf5d77fc-8424-4c37-8b5d-de2cf2ffd886-config-data\") pod \"keystone-bootstrap-cz9xr\" (UID: \"cf5d77fc-8424-4c37-8b5d-de2cf2ffd886\") " pod="openstack-kuttl-tests/keystone-bootstrap-cz9xr" Dec 03 12:17:39 crc kubenswrapper[5052]: I1203 12:17:39.402325 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/888ed75c-2d05-4199-b51a-123ac3e44e31-logs\") pod \"barbican-api-547ff4f8ff-dpw9k\" (UID: \"888ed75c-2d05-4199-b51a-123ac3e44e31\") " pod="openstack-kuttl-tests/barbican-api-547ff4f8ff-dpw9k" Dec 03 12:17:39 crc kubenswrapper[5052]: I1203 12:17:39.402352 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/888ed75c-2d05-4199-b51a-123ac3e44e31-config-data\") pod \"barbican-api-547ff4f8ff-dpw9k\" (UID: \"888ed75c-2d05-4199-b51a-123ac3e44e31\") " pod="openstack-kuttl-tests/barbican-api-547ff4f8ff-dpw9k" Dec 03 12:17:39 crc kubenswrapper[5052]: I1203 12:17:39.402418 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-ckhjp\" (UniqueName: \"kubernetes.io/projected/fcc1e62d-c1cc-49a0-93f4-5112fddf89bf-kube-api-access-ckhjp\") pod \"barbican-keystone-listener-55bfbc4797-wjrjr\" (UID: \"fcc1e62d-c1cc-49a0-93f4-5112fddf89bf\") " pod="openstack-kuttl-tests/barbican-keystone-listener-55bfbc4797-wjrjr" Dec 03 12:17:39 crc kubenswrapper[5052]: I1203 12:17:39.402446 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/fcc1e62d-c1cc-49a0-93f4-5112fddf89bf-combined-ca-bundle\") pod \"barbican-keystone-listener-55bfbc4797-wjrjr\" (UID: \"fcc1e62d-c1cc-49a0-93f4-5112fddf89bf\") " pod="openstack-kuttl-tests/barbican-keystone-listener-55bfbc4797-wjrjr" Dec 03 12:17:39 crc kubenswrapper[5052]: I1203 12:17:39.402476 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e36b4a02-2b71-4e70-b890-e91695ec53f3-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"e36b4a02-2b71-4e70-b890-e91695ec53f3\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:17:39 crc kubenswrapper[5052]: I1203 12:17:39.402500 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/cf5d77fc-8424-4c37-8b5d-de2cf2ffd886-combined-ca-bundle\") pod \"keystone-bootstrap-cz9xr\" (UID: \"cf5d77fc-8424-4c37-8b5d-de2cf2ffd886\") " pod="openstack-kuttl-tests/keystone-bootstrap-cz9xr" Dec 03 12:17:39 crc kubenswrapper[5052]: I1203 12:17:39.402564 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/e36b4a02-2b71-4e70-b890-e91695ec53f3-config-data\") pod \"ceilometer-0\" (UID: \"e36b4a02-2b71-4e70-b890-e91695ec53f3\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:17:39 crc kubenswrapper[5052]: I1203 12:17:39.402586 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/e36b4a02-2b71-4e70-b890-e91695ec53f3-run-httpd\") pod \"ceilometer-0\" (UID: \"e36b4a02-2b71-4e70-b890-e91695ec53f3\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:17:39 crc kubenswrapper[5052]: I1203 12:17:39.402613 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/311e25fa-0785-45ce-ae3c-00d32adbf3f4-logs\") pod \"barbican-worker-76d6bdd486-g9c2p\" (UID: \"311e25fa-0785-45ce-ae3c-00d32adbf3f4\") " pod="openstack-kuttl-tests/barbican-worker-76d6bdd486-g9c2p" Dec 03 12:17:39 crc kubenswrapper[5052]: I1203 12:17:39.402649 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/cf5d77fc-8424-4c37-8b5d-de2cf2ffd886-fernet-keys\") pod \"keystone-bootstrap-cz9xr\" (UID: \"cf5d77fc-8424-4c37-8b5d-de2cf2ffd886\") " pod="openstack-kuttl-tests/keystone-bootstrap-cz9xr" Dec 03 12:17:39 crc kubenswrapper[5052]: I1203 12:17:39.402681 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/cf5d77fc-8424-4c37-8b5d-de2cf2ffd886-credential-keys\") pod \"keystone-bootstrap-cz9xr\" (UID: \"cf5d77fc-8424-4c37-8b5d-de2cf2ffd886\") " pod="openstack-kuttl-tests/keystone-bootstrap-cz9xr" Dec 03 12:17:39 crc kubenswrapper[5052]: I1203 12:17:39.402721 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/cf5d77fc-8424-4c37-8b5d-de2cf2ffd886-scripts\") pod \"keystone-bootstrap-cz9xr\" (UID: \"cf5d77fc-8424-4c37-8b5d-de2cf2ffd886\") " pod="openstack-kuttl-tests/keystone-bootstrap-cz9xr" Dec 03 12:17:39 crc kubenswrapper[5052]: I1203 12:17:39.402939 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/e36b4a02-2b71-4e70-b890-e91695ec53f3-scripts\") pod \"ceilometer-0\" (UID: \"e36b4a02-2b71-4e70-b890-e91695ec53f3\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:17:39 crc kubenswrapper[5052]: I1203 12:17:39.403001 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/311e25fa-0785-45ce-ae3c-00d32adbf3f4-combined-ca-bundle\") pod \"barbican-worker-76d6bdd486-g9c2p\" (UID: \"311e25fa-0785-45ce-ae3c-00d32adbf3f4\") " pod="openstack-kuttl-tests/barbican-worker-76d6bdd486-g9c2p" Dec 03 12:17:39 crc kubenswrapper[5052]: I1203 12:17:39.403047 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/888ed75c-2d05-4199-b51a-123ac3e44e31-config-data-custom\") pod \"barbican-api-547ff4f8ff-dpw9k\" (UID: \"888ed75c-2d05-4199-b51a-123ac3e44e31\") " pod="openstack-kuttl-tests/barbican-api-547ff4f8ff-dpw9k" Dec 03 12:17:39 crc kubenswrapper[5052]: I1203 12:17:39.403104 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-s6scm\" (UniqueName: \"kubernetes.io/projected/311e25fa-0785-45ce-ae3c-00d32adbf3f4-kube-api-access-s6scm\") pod \"barbican-worker-76d6bdd486-g9c2p\" (UID: \"311e25fa-0785-45ce-ae3c-00d32adbf3f4\") " pod="openstack-kuttl-tests/barbican-worker-76d6bdd486-g9c2p" Dec 03 12:17:39 crc kubenswrapper[5052]: I1203 12:17:39.403151 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/fcc1e62d-c1cc-49a0-93f4-5112fddf89bf-config-data\") pod \"barbican-keystone-listener-55bfbc4797-wjrjr\" (UID: \"fcc1e62d-c1cc-49a0-93f4-5112fddf89bf\") " pod="openstack-kuttl-tests/barbican-keystone-listener-55bfbc4797-wjrjr" Dec 03 12:17:39 crc kubenswrapper[5052]: I1203 12:17:39.403174 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/888ed75c-2d05-4199-b51a-123ac3e44e31-combined-ca-bundle\") pod \"barbican-api-547ff4f8ff-dpw9k\" (UID: \"888ed75c-2d05-4199-b51a-123ac3e44e31\") " pod="openstack-kuttl-tests/barbican-api-547ff4f8ff-dpw9k" Dec 03 12:17:39 crc kubenswrapper[5052]: I1203 12:17:39.403203 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/311e25fa-0785-45ce-ae3c-00d32adbf3f4-config-data-custom\") pod \"barbican-worker-76d6bdd486-g9c2p\" (UID: \"311e25fa-0785-45ce-ae3c-00d32adbf3f4\") " pod="openstack-kuttl-tests/barbican-worker-76d6bdd486-g9c2p" Dec 03 12:17:39 crc kubenswrapper[5052]: I1203 12:17:39.403255 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/fcc1e62d-c1cc-49a0-93f4-5112fddf89bf-config-data-custom\") pod \"barbican-keystone-listener-55bfbc4797-wjrjr\" (UID: \"fcc1e62d-c1cc-49a0-93f4-5112fddf89bf\") " pod="openstack-kuttl-tests/barbican-keystone-listener-55bfbc4797-wjrjr" Dec 03 12:17:39 crc kubenswrapper[5052]: I1203 12:17:39.403285 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/311e25fa-0785-45ce-ae3c-00d32adbf3f4-config-data\") pod \"barbican-worker-76d6bdd486-g9c2p\" (UID: \"311e25fa-0785-45ce-ae3c-00d32adbf3f4\") " pod="openstack-kuttl-tests/barbican-worker-76d6bdd486-g9c2p" Dec 03 12:17:39 crc kubenswrapper[5052]: I1203 12:17:39.403334 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-hn8ds\" (UniqueName: \"kubernetes.io/projected/cf5d77fc-8424-4c37-8b5d-de2cf2ffd886-kube-api-access-hn8ds\") pod \"keystone-bootstrap-cz9xr\" (UID: \"cf5d77fc-8424-4c37-8b5d-de2cf2ffd886\") " pod="openstack-kuttl-tests/keystone-bootstrap-cz9xr" Dec 03 12:17:39 crc kubenswrapper[5052]: I1203 12:17:39.404753 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/fcc1e62d-c1cc-49a0-93f4-5112fddf89bf-logs\") pod \"barbican-keystone-listener-55bfbc4797-wjrjr\" (UID: \"fcc1e62d-c1cc-49a0-93f4-5112fddf89bf\") " pod="openstack-kuttl-tests/barbican-keystone-listener-55bfbc4797-wjrjr" Dec 03 12:17:39 crc kubenswrapper[5052]: I1203 12:17:39.405362 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/311e25fa-0785-45ce-ae3c-00d32adbf3f4-logs\") pod \"barbican-worker-76d6bdd486-g9c2p\" (UID: \"311e25fa-0785-45ce-ae3c-00d32adbf3f4\") " pod="openstack-kuttl-tests/barbican-worker-76d6bdd486-g9c2p" Dec 03 12:17:39 crc kubenswrapper[5052]: I1203 12:17:39.413994 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/311e25fa-0785-45ce-ae3c-00d32adbf3f4-combined-ca-bundle\") pod \"barbican-worker-76d6bdd486-g9c2p\" (UID: \"311e25fa-0785-45ce-ae3c-00d32adbf3f4\") " pod="openstack-kuttl-tests/barbican-worker-76d6bdd486-g9c2p" Dec 03 12:17:39 crc kubenswrapper[5052]: I1203 12:17:39.414244 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/cf5d77fc-8424-4c37-8b5d-de2cf2ffd886-config-data\") pod \"keystone-bootstrap-cz9xr\" (UID: \"cf5d77fc-8424-4c37-8b5d-de2cf2ffd886\") " pod="openstack-kuttl-tests/keystone-bootstrap-cz9xr" Dec 03 12:17:39 crc kubenswrapper[5052]: I1203 12:17:39.414687 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/fcc1e62d-c1cc-49a0-93f4-5112fddf89bf-config-data\") pod \"barbican-keystone-listener-55bfbc4797-wjrjr\" (UID: \"fcc1e62d-c1cc-49a0-93f4-5112fddf89bf\") " pod="openstack-kuttl-tests/barbican-keystone-listener-55bfbc4797-wjrjr" Dec 03 12:17:39 crc kubenswrapper[5052]: I1203 12:17:39.415202 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/cf5d77fc-8424-4c37-8b5d-de2cf2ffd886-combined-ca-bundle\") pod \"keystone-bootstrap-cz9xr\" (UID: \"cf5d77fc-8424-4c37-8b5d-de2cf2ffd886\") " pod="openstack-kuttl-tests/keystone-bootstrap-cz9xr" Dec 03 12:17:39 crc kubenswrapper[5052]: I1203 12:17:39.419549 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-s6scm\" (UniqueName: \"kubernetes.io/projected/311e25fa-0785-45ce-ae3c-00d32adbf3f4-kube-api-access-s6scm\") pod \"barbican-worker-76d6bdd486-g9c2p\" (UID: \"311e25fa-0785-45ce-ae3c-00d32adbf3f4\") " pod="openstack-kuttl-tests/barbican-worker-76d6bdd486-g9c2p" Dec 03 12:17:39 crc kubenswrapper[5052]: I1203 12:17:39.422576 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/cf5d77fc-8424-4c37-8b5d-de2cf2ffd886-credential-keys\") pod \"keystone-bootstrap-cz9xr\" (UID: \"cf5d77fc-8424-4c37-8b5d-de2cf2ffd886\") " pod="openstack-kuttl-tests/keystone-bootstrap-cz9xr" Dec 03 12:17:39 crc kubenswrapper[5052]: I1203 12:17:39.423233 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/311e25fa-0785-45ce-ae3c-00d32adbf3f4-config-data\") pod \"barbican-worker-76d6bdd486-g9c2p\" (UID: \"311e25fa-0785-45ce-ae3c-00d32adbf3f4\") " pod="openstack-kuttl-tests/barbican-worker-76d6bdd486-g9c2p" Dec 03 12:17:39 crc kubenswrapper[5052]: I1203 12:17:39.423336 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/cf5d77fc-8424-4c37-8b5d-de2cf2ffd886-fernet-keys\") pod \"keystone-bootstrap-cz9xr\" (UID: \"cf5d77fc-8424-4c37-8b5d-de2cf2ffd886\") " pod="openstack-kuttl-tests/keystone-bootstrap-cz9xr" Dec 03 12:17:39 crc kubenswrapper[5052]: I1203 12:17:39.423776 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/fcc1e62d-c1cc-49a0-93f4-5112fddf89bf-combined-ca-bundle\") pod \"barbican-keystone-listener-55bfbc4797-wjrjr\" (UID: \"fcc1e62d-c1cc-49a0-93f4-5112fddf89bf\") " pod="openstack-kuttl-tests/barbican-keystone-listener-55bfbc4797-wjrjr" Dec 03 12:17:39 crc kubenswrapper[5052]: I1203 12:17:39.424822 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/fcc1e62d-c1cc-49a0-93f4-5112fddf89bf-config-data-custom\") pod \"barbican-keystone-listener-55bfbc4797-wjrjr\" (UID: \"fcc1e62d-c1cc-49a0-93f4-5112fddf89bf\") " pod="openstack-kuttl-tests/barbican-keystone-listener-55bfbc4797-wjrjr" Dec 03 12:17:39 crc kubenswrapper[5052]: I1203 12:17:39.428799 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/cf5d77fc-8424-4c37-8b5d-de2cf2ffd886-scripts\") pod \"keystone-bootstrap-cz9xr\" (UID: \"cf5d77fc-8424-4c37-8b5d-de2cf2ffd886\") " pod="openstack-kuttl-tests/keystone-bootstrap-cz9xr" Dec 03 12:17:39 crc kubenswrapper[5052]: I1203 12:17:39.428923 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-ckhjp\" (UniqueName: \"kubernetes.io/projected/fcc1e62d-c1cc-49a0-93f4-5112fddf89bf-kube-api-access-ckhjp\") pod \"barbican-keystone-listener-55bfbc4797-wjrjr\" (UID: \"fcc1e62d-c1cc-49a0-93f4-5112fddf89bf\") " pod="openstack-kuttl-tests/barbican-keystone-listener-55bfbc4797-wjrjr" Dec 03 12:17:39 crc kubenswrapper[5052]: I1203 12:17:39.429625 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/311e25fa-0785-45ce-ae3c-00d32adbf3f4-config-data-custom\") pod \"barbican-worker-76d6bdd486-g9c2p\" (UID: \"311e25fa-0785-45ce-ae3c-00d32adbf3f4\") " pod="openstack-kuttl-tests/barbican-worker-76d6bdd486-g9c2p" Dec 03 12:17:39 crc kubenswrapper[5052]: I1203 12:17:39.430300 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-hn8ds\" (UniqueName: \"kubernetes.io/projected/cf5d77fc-8424-4c37-8b5d-de2cf2ffd886-kube-api-access-hn8ds\") pod \"keystone-bootstrap-cz9xr\" (UID: \"cf5d77fc-8424-4c37-8b5d-de2cf2ffd886\") " pod="openstack-kuttl-tests/keystone-bootstrap-cz9xr" Dec 03 12:17:39 crc kubenswrapper[5052]: I1203 12:17:39.505137 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/e36b4a02-2b71-4e70-b890-e91695ec53f3-config-data\") pod \"ceilometer-0\" (UID: \"e36b4a02-2b71-4e70-b890-e91695ec53f3\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:17:39 crc kubenswrapper[5052]: I1203 12:17:39.505191 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/e36b4a02-2b71-4e70-b890-e91695ec53f3-run-httpd\") pod \"ceilometer-0\" (UID: \"e36b4a02-2b71-4e70-b890-e91695ec53f3\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:17:39 crc kubenswrapper[5052]: I1203 12:17:39.505389 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/e36b4a02-2b71-4e70-b890-e91695ec53f3-scripts\") pod \"ceilometer-0\" (UID: \"e36b4a02-2b71-4e70-b890-e91695ec53f3\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:17:39 crc kubenswrapper[5052]: I1203 12:17:39.505479 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/888ed75c-2d05-4199-b51a-123ac3e44e31-config-data-custom\") pod \"barbican-api-547ff4f8ff-dpw9k\" (UID: \"888ed75c-2d05-4199-b51a-123ac3e44e31\") " pod="openstack-kuttl-tests/barbican-api-547ff4f8ff-dpw9k" Dec 03 12:17:39 crc kubenswrapper[5052]: I1203 12:17:39.505559 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/888ed75c-2d05-4199-b51a-123ac3e44e31-combined-ca-bundle\") pod \"barbican-api-547ff4f8ff-dpw9k\" (UID: \"888ed75c-2d05-4199-b51a-123ac3e44e31\") " pod="openstack-kuttl-tests/barbican-api-547ff4f8ff-dpw9k" Dec 03 12:17:39 crc kubenswrapper[5052]: I1203 12:17:39.505656 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/e36b4a02-2b71-4e70-b890-e91695ec53f3-log-httpd\") pod \"ceilometer-0\" (UID: \"e36b4a02-2b71-4e70-b890-e91695ec53f3\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:17:39 crc kubenswrapper[5052]: I1203 12:17:39.505728 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/e36b4a02-2b71-4e70-b890-e91695ec53f3-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"e36b4a02-2b71-4e70-b890-e91695ec53f3\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:17:39 crc kubenswrapper[5052]: I1203 12:17:39.505795 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-tllbc\" (UniqueName: \"kubernetes.io/projected/e36b4a02-2b71-4e70-b890-e91695ec53f3-kube-api-access-tllbc\") pod \"ceilometer-0\" (UID: \"e36b4a02-2b71-4e70-b890-e91695ec53f3\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:17:39 crc kubenswrapper[5052]: I1203 12:17:39.505860 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-9k4zc\" (UniqueName: \"kubernetes.io/projected/888ed75c-2d05-4199-b51a-123ac3e44e31-kube-api-access-9k4zc\") pod \"barbican-api-547ff4f8ff-dpw9k\" (UID: \"888ed75c-2d05-4199-b51a-123ac3e44e31\") " pod="openstack-kuttl-tests/barbican-api-547ff4f8ff-dpw9k" Dec 03 12:17:39 crc kubenswrapper[5052]: I1203 12:17:39.505907 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/888ed75c-2d05-4199-b51a-123ac3e44e31-logs\") pod \"barbican-api-547ff4f8ff-dpw9k\" (UID: \"888ed75c-2d05-4199-b51a-123ac3e44e31\") " pod="openstack-kuttl-tests/barbican-api-547ff4f8ff-dpw9k" Dec 03 12:17:39 crc kubenswrapper[5052]: I1203 12:17:39.505960 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/888ed75c-2d05-4199-b51a-123ac3e44e31-config-data\") pod \"barbican-api-547ff4f8ff-dpw9k\" (UID: \"888ed75c-2d05-4199-b51a-123ac3e44e31\") " pod="openstack-kuttl-tests/barbican-api-547ff4f8ff-dpw9k" Dec 03 12:17:39 crc kubenswrapper[5052]: I1203 12:17:39.506048 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e36b4a02-2b71-4e70-b890-e91695ec53f3-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"e36b4a02-2b71-4e70-b890-e91695ec53f3\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:17:39 crc kubenswrapper[5052]: I1203 12:17:39.510262 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/e36b4a02-2b71-4e70-b890-e91695ec53f3-log-httpd\") pod \"ceilometer-0\" (UID: \"e36b4a02-2b71-4e70-b890-e91695ec53f3\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:17:39 crc kubenswrapper[5052]: I1203 12:17:39.510918 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/888ed75c-2d05-4199-b51a-123ac3e44e31-config-data-custom\") pod \"barbican-api-547ff4f8ff-dpw9k\" (UID: \"888ed75c-2d05-4199-b51a-123ac3e44e31\") " pod="openstack-kuttl-tests/barbican-api-547ff4f8ff-dpw9k" Dec 03 12:17:39 crc kubenswrapper[5052]: I1203 12:17:39.510954 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/888ed75c-2d05-4199-b51a-123ac3e44e31-combined-ca-bundle\") pod \"barbican-api-547ff4f8ff-dpw9k\" (UID: \"888ed75c-2d05-4199-b51a-123ac3e44e31\") " pod="openstack-kuttl-tests/barbican-api-547ff4f8ff-dpw9k" Dec 03 12:17:39 crc kubenswrapper[5052]: I1203 12:17:39.511182 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/888ed75c-2d05-4199-b51a-123ac3e44e31-logs\") pod \"barbican-api-547ff4f8ff-dpw9k\" (UID: \"888ed75c-2d05-4199-b51a-123ac3e44e31\") " pod="openstack-kuttl-tests/barbican-api-547ff4f8ff-dpw9k" Dec 03 12:17:39 crc kubenswrapper[5052]: I1203 12:17:39.513024 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/e36b4a02-2b71-4e70-b890-e91695ec53f3-run-httpd\") pod \"ceilometer-0\" (UID: \"e36b4a02-2b71-4e70-b890-e91695ec53f3\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:17:39 crc kubenswrapper[5052]: I1203 12:17:39.515424 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/e36b4a02-2b71-4e70-b890-e91695ec53f3-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"e36b4a02-2b71-4e70-b890-e91695ec53f3\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:17:39 crc kubenswrapper[5052]: I1203 12:17:39.516237 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/e36b4a02-2b71-4e70-b890-e91695ec53f3-config-data\") pod \"ceilometer-0\" (UID: \"e36b4a02-2b71-4e70-b890-e91695ec53f3\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:17:39 crc kubenswrapper[5052]: I1203 12:17:39.516313 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/e36b4a02-2b71-4e70-b890-e91695ec53f3-scripts\") pod \"ceilometer-0\" (UID: \"e36b4a02-2b71-4e70-b890-e91695ec53f3\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:17:39 crc kubenswrapper[5052]: I1203 12:17:39.516682 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/888ed75c-2d05-4199-b51a-123ac3e44e31-config-data\") pod \"barbican-api-547ff4f8ff-dpw9k\" (UID: \"888ed75c-2d05-4199-b51a-123ac3e44e31\") " pod="openstack-kuttl-tests/barbican-api-547ff4f8ff-dpw9k" Dec 03 12:17:39 crc kubenswrapper[5052]: I1203 12:17:39.518897 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e36b4a02-2b71-4e70-b890-e91695ec53f3-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"e36b4a02-2b71-4e70-b890-e91695ec53f3\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:17:39 crc kubenswrapper[5052]: I1203 12:17:39.524693 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-9k4zc\" (UniqueName: \"kubernetes.io/projected/888ed75c-2d05-4199-b51a-123ac3e44e31-kube-api-access-9k4zc\") pod \"barbican-api-547ff4f8ff-dpw9k\" (UID: \"888ed75c-2d05-4199-b51a-123ac3e44e31\") " pod="openstack-kuttl-tests/barbican-api-547ff4f8ff-dpw9k" Dec 03 12:17:39 crc kubenswrapper[5052]: I1203 12:17:39.524961 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-tllbc\" (UniqueName: \"kubernetes.io/projected/e36b4a02-2b71-4e70-b890-e91695ec53f3-kube-api-access-tllbc\") pod \"ceilometer-0\" (UID: \"e36b4a02-2b71-4e70-b890-e91695ec53f3\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:17:39 crc kubenswrapper[5052]: I1203 12:17:39.534053 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-worker-76d6bdd486-g9c2p" Dec 03 12:17:39 crc kubenswrapper[5052]: I1203 12:17:39.542367 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-keystone-listener-55bfbc4797-wjrjr" Dec 03 12:17:39 crc kubenswrapper[5052]: I1203 12:17:39.570753 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-bootstrap-cz9xr" Dec 03 12:17:39 crc kubenswrapper[5052]: I1203 12:17:39.666241 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:17:39 crc kubenswrapper[5052]: I1203 12:17:39.690186 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-api-547ff4f8ff-dpw9k" Dec 03 12:17:39 crc kubenswrapper[5052]: I1203 12:17:39.943718 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="38a5d722-dbe5-4d1e-9aec-bf6513db82fb" path="/var/lib/kubelet/pods/38a5d722-dbe5-4d1e-9aec-bf6513db82fb/volumes" Dec 03 12:17:39 crc kubenswrapper[5052]: I1203 12:17:39.944643 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="614c796f-690e-4c8e-833f-bf7ba7c47d1a" path="/var/lib/kubelet/pods/614c796f-690e-4c8e-833f-bf7ba7c47d1a/volumes" Dec 03 12:17:40 crc kubenswrapper[5052]: I1203 12:17:40.002242 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-7d6bd9d7c9-n2lr6" event={"ID":"803e4fd4-6dec-4eca-b15b-10391318f545","Type":"ContainerStarted","Data":"62a6906785e64de4b321a53b5b04ed62907021345e45c53972baa6f045c1f746"} Dec 03 12:17:40 crc kubenswrapper[5052]: I1203 12:17:40.002820 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/placement-7d6bd9d7c9-n2lr6" Dec 03 12:17:40 crc kubenswrapper[5052]: I1203 12:17:40.002880 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-7d6bd9d7c9-n2lr6" event={"ID":"803e4fd4-6dec-4eca-b15b-10391318f545","Type":"ContainerStarted","Data":"4b811fae861808e9515fee8304fd3b59a0c3b044a28ee334526f62da3c3a9d74"} Dec 03 12:17:40 crc kubenswrapper[5052]: I1203 12:17:40.002906 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/placement-7d6bd9d7c9-n2lr6" Dec 03 12:17:40 crc kubenswrapper[5052]: I1203 12:17:40.036189 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/placement-7d6bd9d7c9-n2lr6" podStartSLOduration=3.036171273 podStartE2EDuration="3.036171273s" podCreationTimestamp="2025-12-03 12:17:37 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:17:40.034996919 +0000 UTC m=+3450.623238683" watchObservedRunningTime="2025-12-03 12:17:40.036171273 +0000 UTC m=+3450.624413037" Dec 03 12:17:40 crc kubenswrapper[5052]: I1203 12:17:40.349120 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-worker-76d6bdd486-g9c2p"] Dec 03 12:17:40 crc kubenswrapper[5052]: I1203 12:17:40.422601 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-keystone-listener-55bfbc4797-wjrjr"] Dec 03 12:17:40 crc kubenswrapper[5052]: I1203 12:17:40.516341 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:17:40 crc kubenswrapper[5052]: I1203 12:17:40.523545 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-bootstrap-cz9xr"] Dec 03 12:17:40 crc kubenswrapper[5052]: W1203 12:17:40.531134 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode36b4a02_2b71_4e70_b890_e91695ec53f3.slice/crio-ce923a9f194e70589709036bf9f1201218ffb9db8b0b58c984d5c96a3014fb5b WatchSource:0}: Error finding container ce923a9f194e70589709036bf9f1201218ffb9db8b0b58c984d5c96a3014fb5b: Status 404 returned error can't find the container with id ce923a9f194e70589709036bf9f1201218ffb9db8b0b58c984d5c96a3014fb5b Dec 03 12:17:40 crc kubenswrapper[5052]: I1203 12:17:40.682355 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-api-547ff4f8ff-dpw9k"] Dec 03 12:17:41 crc kubenswrapper[5052]: I1203 12:17:41.026161 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-bootstrap-cz9xr" event={"ID":"cf5d77fc-8424-4c37-8b5d-de2cf2ffd886","Type":"ContainerStarted","Data":"c181c3bdf14ee9b20cbdd2f9d53262cf57598b2c6bebd77c669f32d823697e1b"} Dec 03 12:17:41 crc kubenswrapper[5052]: I1203 12:17:41.026224 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-bootstrap-cz9xr" event={"ID":"cf5d77fc-8424-4c37-8b5d-de2cf2ffd886","Type":"ContainerStarted","Data":"8fb3a8fad238826f1dfe601696b70ef88cf10f3bffa97819718c8aefdad07af3"} Dec 03 12:17:41 crc kubenswrapper[5052]: I1203 12:17:41.030161 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-547ff4f8ff-dpw9k" event={"ID":"888ed75c-2d05-4199-b51a-123ac3e44e31","Type":"ContainerStarted","Data":"3ac703aa86fa7f1d905e3dee4314f3f9a9da4082370ed127c2c324867f9c0e59"} Dec 03 12:17:41 crc kubenswrapper[5052]: I1203 12:17:41.030226 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-547ff4f8ff-dpw9k" event={"ID":"888ed75c-2d05-4199-b51a-123ac3e44e31","Type":"ContainerStarted","Data":"2aa0092570f6a58d45f8600287cfbd593ba372e185ca4e33fc4dace9ebb34b31"} Dec 03 12:17:41 crc kubenswrapper[5052]: I1203 12:17:41.032577 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-keystone-listener-55bfbc4797-wjrjr" event={"ID":"fcc1e62d-c1cc-49a0-93f4-5112fddf89bf","Type":"ContainerStarted","Data":"8cbc2d5f8732cb847f4e0ef3c58849a3aac9e0fab347ee31905e53398f6a0f69"} Dec 03 12:17:41 crc kubenswrapper[5052]: I1203 12:17:41.032648 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-keystone-listener-55bfbc4797-wjrjr" event={"ID":"fcc1e62d-c1cc-49a0-93f4-5112fddf89bf","Type":"ContainerStarted","Data":"954e646bee69b2119ee9b7b6cb8846a3b7d5373dd70ba72d80b286777fa3abfd"} Dec 03 12:17:41 crc kubenswrapper[5052]: I1203 12:17:41.032658 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-keystone-listener-55bfbc4797-wjrjr" event={"ID":"fcc1e62d-c1cc-49a0-93f4-5112fddf89bf","Type":"ContainerStarted","Data":"5da345bb4422f25990625e27cdecce254f7d0c0ff9033154534628e92a5a94f3"} Dec 03 12:17:41 crc kubenswrapper[5052]: I1203 12:17:41.034915 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-worker-76d6bdd486-g9c2p" event={"ID":"311e25fa-0785-45ce-ae3c-00d32adbf3f4","Type":"ContainerStarted","Data":"b7e1028dbd6661eea3ae89a9c9f561e69d762520ab29fe9c2720e56e95408280"} Dec 03 12:17:41 crc kubenswrapper[5052]: I1203 12:17:41.034941 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-worker-76d6bdd486-g9c2p" event={"ID":"311e25fa-0785-45ce-ae3c-00d32adbf3f4","Type":"ContainerStarted","Data":"522e7f93cdd131cc57ae189abb416b435302969048510ba07854b6d472d399d4"} Dec 03 12:17:41 crc kubenswrapper[5052]: I1203 12:17:41.034951 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-worker-76d6bdd486-g9c2p" event={"ID":"311e25fa-0785-45ce-ae3c-00d32adbf3f4","Type":"ContainerStarted","Data":"7aeaad7bc91a3ea84cb0f054a2df5f62f36ffac7995e71358e1f6f08838a3afb"} Dec 03 12:17:41 crc kubenswrapper[5052]: I1203 12:17:41.040036 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"f65abb68-4704-4c08-b1eb-da1ac582a4d8","Type":"ContainerStarted","Data":"343ed046f4e6ee0353b2e1c5ae14819d623cdc87eab25cae67ba467f7d93a548"} Dec 03 12:17:41 crc kubenswrapper[5052]: I1203 12:17:41.052482 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/keystone-bootstrap-cz9xr" podStartSLOduration=3.052449319 podStartE2EDuration="3.052449319s" podCreationTimestamp="2025-12-03 12:17:38 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:17:41.050161324 +0000 UTC m=+3451.638403078" watchObservedRunningTime="2025-12-03 12:17:41.052449319 +0000 UTC m=+3451.640691083" Dec 03 12:17:41 crc kubenswrapper[5052]: I1203 12:17:41.053162 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"e36b4a02-2b71-4e70-b890-e91695ec53f3","Type":"ContainerStarted","Data":"ce923a9f194e70589709036bf9f1201218ffb9db8b0b58c984d5c96a3014fb5b"} Dec 03 12:17:41 crc kubenswrapper[5052]: I1203 12:17:41.090491 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/glance-default-internal-api-0" podStartSLOduration=5.090465285 podStartE2EDuration="5.090465285s" podCreationTimestamp="2025-12-03 12:17:36 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:17:41.075412685 +0000 UTC m=+3451.663654459" watchObservedRunningTime="2025-12-03 12:17:41.090465285 +0000 UTC m=+3451.678707049" Dec 03 12:17:41 crc kubenswrapper[5052]: I1203 12:17:41.099405 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/barbican-keystone-listener-55bfbc4797-wjrjr" podStartSLOduration=2.099384609 podStartE2EDuration="2.099384609s" podCreationTimestamp="2025-12-03 12:17:39 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:17:41.099021919 +0000 UTC m=+3451.687263703" watchObservedRunningTime="2025-12-03 12:17:41.099384609 +0000 UTC m=+3451.687626383" Dec 03 12:17:41 crc kubenswrapper[5052]: I1203 12:17:41.130882 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/barbican-worker-76d6bdd486-g9c2p" podStartSLOduration=3.130859087 podStartE2EDuration="3.130859087s" podCreationTimestamp="2025-12-03 12:17:38 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:17:41.123913479 +0000 UTC m=+3451.712155243" watchObservedRunningTime="2025-12-03 12:17:41.130859087 +0000 UTC m=+3451.719100851" Dec 03 12:17:42 crc kubenswrapper[5052]: I1203 12:17:42.083681 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"e36b4a02-2b71-4e70-b890-e91695ec53f3","Type":"ContainerStarted","Data":"ec4c6e5477ad824e01b795f69608fed547c19bcae844fe2fa93dbe97916d866e"} Dec 03 12:17:42 crc kubenswrapper[5052]: I1203 12:17:42.083939 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"e36b4a02-2b71-4e70-b890-e91695ec53f3","Type":"ContainerStarted","Data":"7ac693835aaf1e9e855735211f4334f714699a1693c265e3b741d201b9e78b8f"} Dec 03 12:17:42 crc kubenswrapper[5052]: I1203 12:17:42.087561 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-547ff4f8ff-dpw9k" event={"ID":"888ed75c-2d05-4199-b51a-123ac3e44e31","Type":"ContainerStarted","Data":"cf5bcf38e7380e5571b79442bbf2eb53e25771afb2633483fb9da0003a4bb84d"} Dec 03 12:17:42 crc kubenswrapper[5052]: I1203 12:17:42.111024 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/barbican-api-547ff4f8ff-dpw9k" podStartSLOduration=3.110873529 podStartE2EDuration="3.110873529s" podCreationTimestamp="2025-12-03 12:17:39 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:17:42.10563889 +0000 UTC m=+3452.693880674" watchObservedRunningTime="2025-12-03 12:17:42.110873529 +0000 UTC m=+3452.699115293" Dec 03 12:17:42 crc kubenswrapper[5052]: I1203 12:17:42.335654 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/barbican-api-64fc65b95f-tl6vk"] Dec 03 12:17:42 crc kubenswrapper[5052]: I1203 12:17:42.337439 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-api-64fc65b95f-tl6vk" Dec 03 12:17:42 crc kubenswrapper[5052]: I1203 12:17:42.342298 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-barbican-internal-svc" Dec 03 12:17:42 crc kubenswrapper[5052]: I1203 12:17:42.342549 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-barbican-public-svc" Dec 03 12:17:42 crc kubenswrapper[5052]: I1203 12:17:42.359190 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-api-64fc65b95f-tl6vk"] Dec 03 12:17:42 crc kubenswrapper[5052]: I1203 12:17:42.383589 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/7076e341-78ad-4783-8a0e-2f3914f92a5b-config-data\") pod \"barbican-api-64fc65b95f-tl6vk\" (UID: \"7076e341-78ad-4783-8a0e-2f3914f92a5b\") " pod="openstack-kuttl-tests/barbican-api-64fc65b95f-tl6vk" Dec 03 12:17:42 crc kubenswrapper[5052]: I1203 12:17:42.383649 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/7076e341-78ad-4783-8a0e-2f3914f92a5b-combined-ca-bundle\") pod \"barbican-api-64fc65b95f-tl6vk\" (UID: \"7076e341-78ad-4783-8a0e-2f3914f92a5b\") " pod="openstack-kuttl-tests/barbican-api-64fc65b95f-tl6vk" Dec 03 12:17:42 crc kubenswrapper[5052]: I1203 12:17:42.383747 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/7076e341-78ad-4783-8a0e-2f3914f92a5b-logs\") pod \"barbican-api-64fc65b95f-tl6vk\" (UID: \"7076e341-78ad-4783-8a0e-2f3914f92a5b\") " pod="openstack-kuttl-tests/barbican-api-64fc65b95f-tl6vk" Dec 03 12:17:42 crc kubenswrapper[5052]: I1203 12:17:42.383778 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dgggj\" (UniqueName: \"kubernetes.io/projected/7076e341-78ad-4783-8a0e-2f3914f92a5b-kube-api-access-dgggj\") pod \"barbican-api-64fc65b95f-tl6vk\" (UID: \"7076e341-78ad-4783-8a0e-2f3914f92a5b\") " pod="openstack-kuttl-tests/barbican-api-64fc65b95f-tl6vk" Dec 03 12:17:42 crc kubenswrapper[5052]: I1203 12:17:42.383952 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/7076e341-78ad-4783-8a0e-2f3914f92a5b-internal-tls-certs\") pod \"barbican-api-64fc65b95f-tl6vk\" (UID: \"7076e341-78ad-4783-8a0e-2f3914f92a5b\") " pod="openstack-kuttl-tests/barbican-api-64fc65b95f-tl6vk" Dec 03 12:17:42 crc kubenswrapper[5052]: I1203 12:17:42.384048 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/7076e341-78ad-4783-8a0e-2f3914f92a5b-public-tls-certs\") pod \"barbican-api-64fc65b95f-tl6vk\" (UID: \"7076e341-78ad-4783-8a0e-2f3914f92a5b\") " pod="openstack-kuttl-tests/barbican-api-64fc65b95f-tl6vk" Dec 03 12:17:42 crc kubenswrapper[5052]: I1203 12:17:42.384141 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/7076e341-78ad-4783-8a0e-2f3914f92a5b-config-data-custom\") pod \"barbican-api-64fc65b95f-tl6vk\" (UID: \"7076e341-78ad-4783-8a0e-2f3914f92a5b\") " pod="openstack-kuttl-tests/barbican-api-64fc65b95f-tl6vk" Dec 03 12:17:42 crc kubenswrapper[5052]: I1203 12:17:42.486362 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/7076e341-78ad-4783-8a0e-2f3914f92a5b-public-tls-certs\") pod \"barbican-api-64fc65b95f-tl6vk\" (UID: \"7076e341-78ad-4783-8a0e-2f3914f92a5b\") " pod="openstack-kuttl-tests/barbican-api-64fc65b95f-tl6vk" Dec 03 12:17:42 crc kubenswrapper[5052]: I1203 12:17:42.486678 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/7076e341-78ad-4783-8a0e-2f3914f92a5b-config-data-custom\") pod \"barbican-api-64fc65b95f-tl6vk\" (UID: \"7076e341-78ad-4783-8a0e-2f3914f92a5b\") " pod="openstack-kuttl-tests/barbican-api-64fc65b95f-tl6vk" Dec 03 12:17:42 crc kubenswrapper[5052]: I1203 12:17:42.486785 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/7076e341-78ad-4783-8a0e-2f3914f92a5b-config-data\") pod \"barbican-api-64fc65b95f-tl6vk\" (UID: \"7076e341-78ad-4783-8a0e-2f3914f92a5b\") " pod="openstack-kuttl-tests/barbican-api-64fc65b95f-tl6vk" Dec 03 12:17:42 crc kubenswrapper[5052]: I1203 12:17:42.486890 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/7076e341-78ad-4783-8a0e-2f3914f92a5b-combined-ca-bundle\") pod \"barbican-api-64fc65b95f-tl6vk\" (UID: \"7076e341-78ad-4783-8a0e-2f3914f92a5b\") " pod="openstack-kuttl-tests/barbican-api-64fc65b95f-tl6vk" Dec 03 12:17:42 crc kubenswrapper[5052]: I1203 12:17:42.487096 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/7076e341-78ad-4783-8a0e-2f3914f92a5b-logs\") pod \"barbican-api-64fc65b95f-tl6vk\" (UID: \"7076e341-78ad-4783-8a0e-2f3914f92a5b\") " pod="openstack-kuttl-tests/barbican-api-64fc65b95f-tl6vk" Dec 03 12:17:42 crc kubenswrapper[5052]: I1203 12:17:42.487181 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-dgggj\" (UniqueName: \"kubernetes.io/projected/7076e341-78ad-4783-8a0e-2f3914f92a5b-kube-api-access-dgggj\") pod \"barbican-api-64fc65b95f-tl6vk\" (UID: \"7076e341-78ad-4783-8a0e-2f3914f92a5b\") " pod="openstack-kuttl-tests/barbican-api-64fc65b95f-tl6vk" Dec 03 12:17:42 crc kubenswrapper[5052]: I1203 12:17:42.487310 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/7076e341-78ad-4783-8a0e-2f3914f92a5b-internal-tls-certs\") pod \"barbican-api-64fc65b95f-tl6vk\" (UID: \"7076e341-78ad-4783-8a0e-2f3914f92a5b\") " pod="openstack-kuttl-tests/barbican-api-64fc65b95f-tl6vk" Dec 03 12:17:42 crc kubenswrapper[5052]: I1203 12:17:42.487369 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/7076e341-78ad-4783-8a0e-2f3914f92a5b-logs\") pod \"barbican-api-64fc65b95f-tl6vk\" (UID: \"7076e341-78ad-4783-8a0e-2f3914f92a5b\") " pod="openstack-kuttl-tests/barbican-api-64fc65b95f-tl6vk" Dec 03 12:17:42 crc kubenswrapper[5052]: I1203 12:17:42.493012 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/7076e341-78ad-4783-8a0e-2f3914f92a5b-combined-ca-bundle\") pod \"barbican-api-64fc65b95f-tl6vk\" (UID: \"7076e341-78ad-4783-8a0e-2f3914f92a5b\") " pod="openstack-kuttl-tests/barbican-api-64fc65b95f-tl6vk" Dec 03 12:17:42 crc kubenswrapper[5052]: I1203 12:17:42.493306 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/7076e341-78ad-4783-8a0e-2f3914f92a5b-internal-tls-certs\") pod \"barbican-api-64fc65b95f-tl6vk\" (UID: \"7076e341-78ad-4783-8a0e-2f3914f92a5b\") " pod="openstack-kuttl-tests/barbican-api-64fc65b95f-tl6vk" Dec 03 12:17:42 crc kubenswrapper[5052]: I1203 12:17:42.493988 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/7076e341-78ad-4783-8a0e-2f3914f92a5b-public-tls-certs\") pod \"barbican-api-64fc65b95f-tl6vk\" (UID: \"7076e341-78ad-4783-8a0e-2f3914f92a5b\") " pod="openstack-kuttl-tests/barbican-api-64fc65b95f-tl6vk" Dec 03 12:17:42 crc kubenswrapper[5052]: I1203 12:17:42.494690 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/7076e341-78ad-4783-8a0e-2f3914f92a5b-config-data\") pod \"barbican-api-64fc65b95f-tl6vk\" (UID: \"7076e341-78ad-4783-8a0e-2f3914f92a5b\") " pod="openstack-kuttl-tests/barbican-api-64fc65b95f-tl6vk" Dec 03 12:17:42 crc kubenswrapper[5052]: I1203 12:17:42.505550 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-dgggj\" (UniqueName: \"kubernetes.io/projected/7076e341-78ad-4783-8a0e-2f3914f92a5b-kube-api-access-dgggj\") pod \"barbican-api-64fc65b95f-tl6vk\" (UID: \"7076e341-78ad-4783-8a0e-2f3914f92a5b\") " pod="openstack-kuttl-tests/barbican-api-64fc65b95f-tl6vk" Dec 03 12:17:42 crc kubenswrapper[5052]: I1203 12:17:42.505811 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/7076e341-78ad-4783-8a0e-2f3914f92a5b-config-data-custom\") pod \"barbican-api-64fc65b95f-tl6vk\" (UID: \"7076e341-78ad-4783-8a0e-2f3914f92a5b\") " pod="openstack-kuttl-tests/barbican-api-64fc65b95f-tl6vk" Dec 03 12:17:42 crc kubenswrapper[5052]: I1203 12:17:42.653315 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-api-64fc65b95f-tl6vk" Dec 03 12:17:43 crc kubenswrapper[5052]: I1203 12:17:43.097385 5052 generic.go:334] "Generic (PLEG): container finished" podID="e5091121-ce68-4b86-baae-2fbd5b22d537" containerID="7d8e7144eee1a253836215f98a2a034d3756074819c1433459b21b25e1261483" exitCode=0 Dec 03 12:17:43 crc kubenswrapper[5052]: I1203 12:17:43.097433 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-db-sync-wp26m" event={"ID":"e5091121-ce68-4b86-baae-2fbd5b22d537","Type":"ContainerDied","Data":"7d8e7144eee1a253836215f98a2a034d3756074819c1433459b21b25e1261483"} Dec 03 12:17:43 crc kubenswrapper[5052]: I1203 12:17:43.100822 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"e36b4a02-2b71-4e70-b890-e91695ec53f3","Type":"ContainerStarted","Data":"f6833798ca88a77a606fd1470b8377553dfe1ed45d731e4a1f9fce2597d061a8"} Dec 03 12:17:43 crc kubenswrapper[5052]: I1203 12:17:43.101034 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/barbican-api-547ff4f8ff-dpw9k" Dec 03 12:17:43 crc kubenswrapper[5052]: I1203 12:17:43.101065 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/barbican-api-547ff4f8ff-dpw9k" Dec 03 12:17:43 crc kubenswrapper[5052]: I1203 12:17:43.150313 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-api-64fc65b95f-tl6vk"] Dec 03 12:17:43 crc kubenswrapper[5052]: W1203 12:17:43.155614 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7076e341_78ad_4783_8a0e_2f3914f92a5b.slice/crio-3d0cb3c33636c943ed49faf4eb8517b934ecd6d4fbd3df9e62306709fc2bce64 WatchSource:0}: Error finding container 3d0cb3c33636c943ed49faf4eb8517b934ecd6d4fbd3df9e62306709fc2bce64: Status 404 returned error can't find the container with id 3d0cb3c33636c943ed49faf4eb8517b934ecd6d4fbd3df9e62306709fc2bce64 Dec 03 12:17:44 crc kubenswrapper[5052]: I1203 12:17:44.112236 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-64fc65b95f-tl6vk" event={"ID":"7076e341-78ad-4783-8a0e-2f3914f92a5b","Type":"ContainerStarted","Data":"b42993f3a56100f0e34428fc72958d399bed23bdc2ca879c6c31d4ce0a789544"} Dec 03 12:17:44 crc kubenswrapper[5052]: I1203 12:17:44.112886 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/barbican-api-64fc65b95f-tl6vk" Dec 03 12:17:44 crc kubenswrapper[5052]: I1203 12:17:44.112907 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/barbican-api-64fc65b95f-tl6vk" Dec 03 12:17:44 crc kubenswrapper[5052]: I1203 12:17:44.112918 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-64fc65b95f-tl6vk" event={"ID":"7076e341-78ad-4783-8a0e-2f3914f92a5b","Type":"ContainerStarted","Data":"6f32818c01018bb4abae02721a800dd04ca46b072f663e382163bc70ee748e43"} Dec 03 12:17:44 crc kubenswrapper[5052]: I1203 12:17:44.112930 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-64fc65b95f-tl6vk" event={"ID":"7076e341-78ad-4783-8a0e-2f3914f92a5b","Type":"ContainerStarted","Data":"3d0cb3c33636c943ed49faf4eb8517b934ecd6d4fbd3df9e62306709fc2bce64"} Dec 03 12:17:44 crc kubenswrapper[5052]: I1203 12:17:44.142937 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/barbican-api-64fc65b95f-tl6vk" podStartSLOduration=2.142914957 podStartE2EDuration="2.142914957s" podCreationTimestamp="2025-12-03 12:17:42 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:17:44.132573652 +0000 UTC m=+3454.720815416" watchObservedRunningTime="2025-12-03 12:17:44.142914957 +0000 UTC m=+3454.731156741" Dec 03 12:17:44 crc kubenswrapper[5052]: I1203 12:17:44.460644 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-db-sync-wp26m" Dec 03 12:17:44 crc kubenswrapper[5052]: I1203 12:17:44.523907 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/e5091121-ce68-4b86-baae-2fbd5b22d537-db-sync-config-data\") pod \"e5091121-ce68-4b86-baae-2fbd5b22d537\" (UID: \"e5091121-ce68-4b86-baae-2fbd5b22d537\") " Dec 03 12:17:44 crc kubenswrapper[5052]: I1203 12:17:44.524012 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e5091121-ce68-4b86-baae-2fbd5b22d537-combined-ca-bundle\") pod \"e5091121-ce68-4b86-baae-2fbd5b22d537\" (UID: \"e5091121-ce68-4b86-baae-2fbd5b22d537\") " Dec 03 12:17:44 crc kubenswrapper[5052]: I1203 12:17:44.524057 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/e5091121-ce68-4b86-baae-2fbd5b22d537-config-data\") pod \"e5091121-ce68-4b86-baae-2fbd5b22d537\" (UID: \"e5091121-ce68-4b86-baae-2fbd5b22d537\") " Dec 03 12:17:44 crc kubenswrapper[5052]: I1203 12:17:44.524107 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5qtdl\" (UniqueName: \"kubernetes.io/projected/e5091121-ce68-4b86-baae-2fbd5b22d537-kube-api-access-5qtdl\") pod \"e5091121-ce68-4b86-baae-2fbd5b22d537\" (UID: \"e5091121-ce68-4b86-baae-2fbd5b22d537\") " Dec 03 12:17:44 crc kubenswrapper[5052]: I1203 12:17:44.524165 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/e5091121-ce68-4b86-baae-2fbd5b22d537-etc-machine-id\") pod \"e5091121-ce68-4b86-baae-2fbd5b22d537\" (UID: \"e5091121-ce68-4b86-baae-2fbd5b22d537\") " Dec 03 12:17:44 crc kubenswrapper[5052]: I1203 12:17:44.524307 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/e5091121-ce68-4b86-baae-2fbd5b22d537-etc-machine-id" (OuterVolumeSpecName: "etc-machine-id") pod "e5091121-ce68-4b86-baae-2fbd5b22d537" (UID: "e5091121-ce68-4b86-baae-2fbd5b22d537"). InnerVolumeSpecName "etc-machine-id". PluginName "kubernetes.io/host-path", VolumeGidValue "" Dec 03 12:17:44 crc kubenswrapper[5052]: I1203 12:17:44.524255 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/e5091121-ce68-4b86-baae-2fbd5b22d537-scripts\") pod \"e5091121-ce68-4b86-baae-2fbd5b22d537\" (UID: \"e5091121-ce68-4b86-baae-2fbd5b22d537\") " Dec 03 12:17:44 crc kubenswrapper[5052]: I1203 12:17:44.524894 5052 reconciler_common.go:293] "Volume detached for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/e5091121-ce68-4b86-baae-2fbd5b22d537-etc-machine-id\") on node \"crc\" DevicePath \"\"" Dec 03 12:17:44 crc kubenswrapper[5052]: I1203 12:17:44.529898 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e5091121-ce68-4b86-baae-2fbd5b22d537-scripts" (OuterVolumeSpecName: "scripts") pod "e5091121-ce68-4b86-baae-2fbd5b22d537" (UID: "e5091121-ce68-4b86-baae-2fbd5b22d537"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:17:44 crc kubenswrapper[5052]: I1203 12:17:44.529932 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e5091121-ce68-4b86-baae-2fbd5b22d537-kube-api-access-5qtdl" (OuterVolumeSpecName: "kube-api-access-5qtdl") pod "e5091121-ce68-4b86-baae-2fbd5b22d537" (UID: "e5091121-ce68-4b86-baae-2fbd5b22d537"). InnerVolumeSpecName "kube-api-access-5qtdl". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:17:44 crc kubenswrapper[5052]: I1203 12:17:44.530698 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e5091121-ce68-4b86-baae-2fbd5b22d537-db-sync-config-data" (OuterVolumeSpecName: "db-sync-config-data") pod "e5091121-ce68-4b86-baae-2fbd5b22d537" (UID: "e5091121-ce68-4b86-baae-2fbd5b22d537"). InnerVolumeSpecName "db-sync-config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:17:44 crc kubenswrapper[5052]: I1203 12:17:44.558272 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e5091121-ce68-4b86-baae-2fbd5b22d537-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "e5091121-ce68-4b86-baae-2fbd5b22d537" (UID: "e5091121-ce68-4b86-baae-2fbd5b22d537"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:17:44 crc kubenswrapper[5052]: I1203 12:17:44.579549 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e5091121-ce68-4b86-baae-2fbd5b22d537-config-data" (OuterVolumeSpecName: "config-data") pod "e5091121-ce68-4b86-baae-2fbd5b22d537" (UID: "e5091121-ce68-4b86-baae-2fbd5b22d537"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:17:44 crc kubenswrapper[5052]: I1203 12:17:44.626473 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/e5091121-ce68-4b86-baae-2fbd5b22d537-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:17:44 crc kubenswrapper[5052]: I1203 12:17:44.626535 5052 reconciler_common.go:293] "Volume detached for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/e5091121-ce68-4b86-baae-2fbd5b22d537-db-sync-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:17:44 crc kubenswrapper[5052]: I1203 12:17:44.626551 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e5091121-ce68-4b86-baae-2fbd5b22d537-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:17:44 crc kubenswrapper[5052]: I1203 12:17:44.626562 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/e5091121-ce68-4b86-baae-2fbd5b22d537-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:17:44 crc kubenswrapper[5052]: I1203 12:17:44.626575 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-5qtdl\" (UniqueName: \"kubernetes.io/projected/e5091121-ce68-4b86-baae-2fbd5b22d537-kube-api-access-5qtdl\") on node \"crc\" DevicePath \"\"" Dec 03 12:17:45 crc kubenswrapper[5052]: I1203 12:17:45.122943 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-db-sync-wp26m" Dec 03 12:17:45 crc kubenswrapper[5052]: I1203 12:17:45.122935 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-db-sync-wp26m" event={"ID":"e5091121-ce68-4b86-baae-2fbd5b22d537","Type":"ContainerDied","Data":"ba9b87636c0720bfc27c02fade6459540a85cd2254622312ecec3919fdf9857a"} Dec 03 12:17:45 crc kubenswrapper[5052]: I1203 12:17:45.123057 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ba9b87636c0720bfc27c02fade6459540a85cd2254622312ecec3919fdf9857a" Dec 03 12:17:45 crc kubenswrapper[5052]: I1203 12:17:45.126923 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"e36b4a02-2b71-4e70-b890-e91695ec53f3","Type":"ContainerStarted","Data":"6d1dc86dca302c34d13335215a72604630127727141b8b1de8e5259bd7e5f5fe"} Dec 03 12:17:45 crc kubenswrapper[5052]: I1203 12:17:45.127076 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:17:45 crc kubenswrapper[5052]: I1203 12:17:45.131613 5052 generic.go:334] "Generic (PLEG): container finished" podID="cf5d77fc-8424-4c37-8b5d-de2cf2ffd886" containerID="c181c3bdf14ee9b20cbdd2f9d53262cf57598b2c6bebd77c669f32d823697e1b" exitCode=0 Dec 03 12:17:45 crc kubenswrapper[5052]: I1203 12:17:45.131879 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-bootstrap-cz9xr" event={"ID":"cf5d77fc-8424-4c37-8b5d-de2cf2ffd886","Type":"ContainerDied","Data":"c181c3bdf14ee9b20cbdd2f9d53262cf57598b2c6bebd77c669f32d823697e1b"} Dec 03 12:17:45 crc kubenswrapper[5052]: I1203 12:17:45.159914 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/ceilometer-0" podStartSLOduration=2.740353354 podStartE2EDuration="6.159891385s" podCreationTimestamp="2025-12-03 12:17:39 +0000 UTC" firstStartedPulling="2025-12-03 12:17:40.537309706 +0000 UTC m=+3451.125551470" lastFinishedPulling="2025-12-03 12:17:43.956847707 +0000 UTC m=+3454.545089501" observedRunningTime="2025-12-03 12:17:45.154226513 +0000 UTC m=+3455.742468287" watchObservedRunningTime="2025-12-03 12:17:45.159891385 +0000 UTC m=+3455.748133159" Dec 03 12:17:45 crc kubenswrapper[5052]: I1203 12:17:45.466011 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 12:17:45 crc kubenswrapper[5052]: E1203 12:17:45.466888 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="e5091121-ce68-4b86-baae-2fbd5b22d537" containerName="cinder-db-sync" Dec 03 12:17:45 crc kubenswrapper[5052]: I1203 12:17:45.467005 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="e5091121-ce68-4b86-baae-2fbd5b22d537" containerName="cinder-db-sync" Dec 03 12:17:45 crc kubenswrapper[5052]: I1203 12:17:45.467276 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="e5091121-ce68-4b86-baae-2fbd5b22d537" containerName="cinder-db-sync" Dec 03 12:17:45 crc kubenswrapper[5052]: I1203 12:17:45.468482 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:17:45 crc kubenswrapper[5052]: I1203 12:17:45.471889 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cinder-scheduler-config-data" Dec 03 12:17:45 crc kubenswrapper[5052]: I1203 12:17:45.472075 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cinder-cinder-dockercfg-znp6c" Dec 03 12:17:45 crc kubenswrapper[5052]: I1203 12:17:45.472269 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cinder-config-data" Dec 03 12:17:45 crc kubenswrapper[5052]: I1203 12:17:45.472378 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cinder-scripts" Dec 03 12:17:45 crc kubenswrapper[5052]: I1203 12:17:45.479621 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 12:17:45 crc kubenswrapper[5052]: I1203 12:17:45.575440 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 12:17:45 crc kubenswrapper[5052]: I1203 12:17:45.577168 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:17:45 crc kubenswrapper[5052]: I1203 12:17:45.579198 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cinder-api-config-data" Dec 03 12:17:45 crc kubenswrapper[5052]: I1203 12:17:45.589799 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 12:17:45 crc kubenswrapper[5052]: I1203 12:17:45.592532 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qx4fm\" (UniqueName: \"kubernetes.io/projected/321c0d14-7d66-4b3a-8989-f6a161421d2e-kube-api-access-qx4fm\") pod \"cinder-scheduler-0\" (UID: \"321c0d14-7d66-4b3a-8989-f6a161421d2e\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:17:45 crc kubenswrapper[5052]: I1203 12:17:45.592597 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/321c0d14-7d66-4b3a-8989-f6a161421d2e-scripts\") pod \"cinder-scheduler-0\" (UID: \"321c0d14-7d66-4b3a-8989-f6a161421d2e\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:17:45 crc kubenswrapper[5052]: I1203 12:17:45.592617 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/321c0d14-7d66-4b3a-8989-f6a161421d2e-config-data\") pod \"cinder-scheduler-0\" (UID: \"321c0d14-7d66-4b3a-8989-f6a161421d2e\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:17:45 crc kubenswrapper[5052]: I1203 12:17:45.592654 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/321c0d14-7d66-4b3a-8989-f6a161421d2e-config-data-custom\") pod \"cinder-scheduler-0\" (UID: \"321c0d14-7d66-4b3a-8989-f6a161421d2e\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:17:45 crc kubenswrapper[5052]: I1203 12:17:45.592673 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/321c0d14-7d66-4b3a-8989-f6a161421d2e-combined-ca-bundle\") pod \"cinder-scheduler-0\" (UID: \"321c0d14-7d66-4b3a-8989-f6a161421d2e\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:17:45 crc kubenswrapper[5052]: I1203 12:17:45.592704 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/321c0d14-7d66-4b3a-8989-f6a161421d2e-etc-machine-id\") pod \"cinder-scheduler-0\" (UID: \"321c0d14-7d66-4b3a-8989-f6a161421d2e\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:17:45 crc kubenswrapper[5052]: I1203 12:17:45.693778 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/908e450d-211f-4b2f-8f21-332e6a18e0c1-combined-ca-bundle\") pod \"cinder-api-0\" (UID: \"908e450d-211f-4b2f-8f21-332e6a18e0c1\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:17:45 crc kubenswrapper[5052]: I1203 12:17:45.694137 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/908e450d-211f-4b2f-8f21-332e6a18e0c1-logs\") pod \"cinder-api-0\" (UID: \"908e450d-211f-4b2f-8f21-332e6a18e0c1\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:17:45 crc kubenswrapper[5052]: I1203 12:17:45.694321 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-qx4fm\" (UniqueName: \"kubernetes.io/projected/321c0d14-7d66-4b3a-8989-f6a161421d2e-kube-api-access-qx4fm\") pod \"cinder-scheduler-0\" (UID: \"321c0d14-7d66-4b3a-8989-f6a161421d2e\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:17:45 crc kubenswrapper[5052]: I1203 12:17:45.694454 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/321c0d14-7d66-4b3a-8989-f6a161421d2e-scripts\") pod \"cinder-scheduler-0\" (UID: \"321c0d14-7d66-4b3a-8989-f6a161421d2e\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:17:45 crc kubenswrapper[5052]: I1203 12:17:45.694589 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/321c0d14-7d66-4b3a-8989-f6a161421d2e-config-data\") pod \"cinder-scheduler-0\" (UID: \"321c0d14-7d66-4b3a-8989-f6a161421d2e\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:17:45 crc kubenswrapper[5052]: I1203 12:17:45.694721 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/908e450d-211f-4b2f-8f21-332e6a18e0c1-etc-machine-id\") pod \"cinder-api-0\" (UID: \"908e450d-211f-4b2f-8f21-332e6a18e0c1\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:17:45 crc kubenswrapper[5052]: I1203 12:17:45.694866 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/321c0d14-7d66-4b3a-8989-f6a161421d2e-config-data-custom\") pod \"cinder-scheduler-0\" (UID: \"321c0d14-7d66-4b3a-8989-f6a161421d2e\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:17:45 crc kubenswrapper[5052]: I1203 12:17:45.694955 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/908e450d-211f-4b2f-8f21-332e6a18e0c1-config-data-custom\") pod \"cinder-api-0\" (UID: \"908e450d-211f-4b2f-8f21-332e6a18e0c1\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:17:45 crc kubenswrapper[5052]: I1203 12:17:45.695039 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/321c0d14-7d66-4b3a-8989-f6a161421d2e-combined-ca-bundle\") pod \"cinder-scheduler-0\" (UID: \"321c0d14-7d66-4b3a-8989-f6a161421d2e\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:17:45 crc kubenswrapper[5052]: I1203 12:17:45.695134 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/908e450d-211f-4b2f-8f21-332e6a18e0c1-scripts\") pod \"cinder-api-0\" (UID: \"908e450d-211f-4b2f-8f21-332e6a18e0c1\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:17:45 crc kubenswrapper[5052]: I1203 12:17:45.695249 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/321c0d14-7d66-4b3a-8989-f6a161421d2e-etc-machine-id\") pod \"cinder-scheduler-0\" (UID: \"321c0d14-7d66-4b3a-8989-f6a161421d2e\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:17:45 crc kubenswrapper[5052]: I1203 12:17:45.695409 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5fnj5\" (UniqueName: \"kubernetes.io/projected/908e450d-211f-4b2f-8f21-332e6a18e0c1-kube-api-access-5fnj5\") pod \"cinder-api-0\" (UID: \"908e450d-211f-4b2f-8f21-332e6a18e0c1\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:17:45 crc kubenswrapper[5052]: I1203 12:17:45.695619 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/908e450d-211f-4b2f-8f21-332e6a18e0c1-config-data\") pod \"cinder-api-0\" (UID: \"908e450d-211f-4b2f-8f21-332e6a18e0c1\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:17:45 crc kubenswrapper[5052]: I1203 12:17:45.695428 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/321c0d14-7d66-4b3a-8989-f6a161421d2e-etc-machine-id\") pod \"cinder-scheduler-0\" (UID: \"321c0d14-7d66-4b3a-8989-f6a161421d2e\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:17:45 crc kubenswrapper[5052]: I1203 12:17:45.698861 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/321c0d14-7d66-4b3a-8989-f6a161421d2e-config-data-custom\") pod \"cinder-scheduler-0\" (UID: \"321c0d14-7d66-4b3a-8989-f6a161421d2e\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:17:45 crc kubenswrapper[5052]: I1203 12:17:45.699495 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/321c0d14-7d66-4b3a-8989-f6a161421d2e-scripts\") pod \"cinder-scheduler-0\" (UID: \"321c0d14-7d66-4b3a-8989-f6a161421d2e\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:17:45 crc kubenswrapper[5052]: I1203 12:17:45.700181 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/321c0d14-7d66-4b3a-8989-f6a161421d2e-config-data\") pod \"cinder-scheduler-0\" (UID: \"321c0d14-7d66-4b3a-8989-f6a161421d2e\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:17:45 crc kubenswrapper[5052]: I1203 12:17:45.700609 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/321c0d14-7d66-4b3a-8989-f6a161421d2e-combined-ca-bundle\") pod \"cinder-scheduler-0\" (UID: \"321c0d14-7d66-4b3a-8989-f6a161421d2e\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:17:45 crc kubenswrapper[5052]: I1203 12:17:45.713084 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-qx4fm\" (UniqueName: \"kubernetes.io/projected/321c0d14-7d66-4b3a-8989-f6a161421d2e-kube-api-access-qx4fm\") pod \"cinder-scheduler-0\" (UID: \"321c0d14-7d66-4b3a-8989-f6a161421d2e\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:17:45 crc kubenswrapper[5052]: I1203 12:17:45.786169 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:17:45 crc kubenswrapper[5052]: I1203 12:17:45.797091 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-5fnj5\" (UniqueName: \"kubernetes.io/projected/908e450d-211f-4b2f-8f21-332e6a18e0c1-kube-api-access-5fnj5\") pod \"cinder-api-0\" (UID: \"908e450d-211f-4b2f-8f21-332e6a18e0c1\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:17:45 crc kubenswrapper[5052]: I1203 12:17:45.797190 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/908e450d-211f-4b2f-8f21-332e6a18e0c1-config-data\") pod \"cinder-api-0\" (UID: \"908e450d-211f-4b2f-8f21-332e6a18e0c1\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:17:45 crc kubenswrapper[5052]: I1203 12:17:45.797237 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/908e450d-211f-4b2f-8f21-332e6a18e0c1-combined-ca-bundle\") pod \"cinder-api-0\" (UID: \"908e450d-211f-4b2f-8f21-332e6a18e0c1\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:17:45 crc kubenswrapper[5052]: I1203 12:17:45.797262 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/908e450d-211f-4b2f-8f21-332e6a18e0c1-logs\") pod \"cinder-api-0\" (UID: \"908e450d-211f-4b2f-8f21-332e6a18e0c1\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:17:45 crc kubenswrapper[5052]: I1203 12:17:45.797325 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/908e450d-211f-4b2f-8f21-332e6a18e0c1-etc-machine-id\") pod \"cinder-api-0\" (UID: \"908e450d-211f-4b2f-8f21-332e6a18e0c1\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:17:45 crc kubenswrapper[5052]: I1203 12:17:45.797372 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/908e450d-211f-4b2f-8f21-332e6a18e0c1-config-data-custom\") pod \"cinder-api-0\" (UID: \"908e450d-211f-4b2f-8f21-332e6a18e0c1\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:17:45 crc kubenswrapper[5052]: I1203 12:17:45.797406 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/908e450d-211f-4b2f-8f21-332e6a18e0c1-scripts\") pod \"cinder-api-0\" (UID: \"908e450d-211f-4b2f-8f21-332e6a18e0c1\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:17:45 crc kubenswrapper[5052]: I1203 12:17:45.797939 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/908e450d-211f-4b2f-8f21-332e6a18e0c1-logs\") pod \"cinder-api-0\" (UID: \"908e450d-211f-4b2f-8f21-332e6a18e0c1\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:17:45 crc kubenswrapper[5052]: I1203 12:17:45.798651 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/908e450d-211f-4b2f-8f21-332e6a18e0c1-etc-machine-id\") pod \"cinder-api-0\" (UID: \"908e450d-211f-4b2f-8f21-332e6a18e0c1\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:17:45 crc kubenswrapper[5052]: I1203 12:17:45.802266 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/908e450d-211f-4b2f-8f21-332e6a18e0c1-config-data-custom\") pod \"cinder-api-0\" (UID: \"908e450d-211f-4b2f-8f21-332e6a18e0c1\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:17:45 crc kubenswrapper[5052]: I1203 12:17:45.807425 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/908e450d-211f-4b2f-8f21-332e6a18e0c1-config-data\") pod \"cinder-api-0\" (UID: \"908e450d-211f-4b2f-8f21-332e6a18e0c1\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:17:45 crc kubenswrapper[5052]: I1203 12:17:45.809055 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/908e450d-211f-4b2f-8f21-332e6a18e0c1-scripts\") pod \"cinder-api-0\" (UID: \"908e450d-211f-4b2f-8f21-332e6a18e0c1\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:17:45 crc kubenswrapper[5052]: I1203 12:17:45.811644 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/908e450d-211f-4b2f-8f21-332e6a18e0c1-combined-ca-bundle\") pod \"cinder-api-0\" (UID: \"908e450d-211f-4b2f-8f21-332e6a18e0c1\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:17:45 crc kubenswrapper[5052]: I1203 12:17:45.818668 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-5fnj5\" (UniqueName: \"kubernetes.io/projected/908e450d-211f-4b2f-8f21-332e6a18e0c1-kube-api-access-5fnj5\") pod \"cinder-api-0\" (UID: \"908e450d-211f-4b2f-8f21-332e6a18e0c1\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:17:45 crc kubenswrapper[5052]: I1203 12:17:45.896482 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:17:46 crc kubenswrapper[5052]: I1203 12:17:46.282643 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:17:46 crc kubenswrapper[5052]: I1203 12:17:46.283034 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:17:46 crc kubenswrapper[5052]: I1203 12:17:46.285443 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 12:17:46 crc kubenswrapper[5052]: I1203 12:17:46.314878 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:17:46 crc kubenswrapper[5052]: I1203 12:17:46.347825 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:17:46 crc kubenswrapper[5052]: I1203 12:17:46.511394 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 12:17:46 crc kubenswrapper[5052]: I1203 12:17:46.625930 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-bootstrap-cz9xr" Dec 03 12:17:46 crc kubenswrapper[5052]: I1203 12:17:46.758654 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/cf5d77fc-8424-4c37-8b5d-de2cf2ffd886-config-data\") pod \"cf5d77fc-8424-4c37-8b5d-de2cf2ffd886\" (UID: \"cf5d77fc-8424-4c37-8b5d-de2cf2ffd886\") " Dec 03 12:17:46 crc kubenswrapper[5052]: I1203 12:17:46.758802 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/cf5d77fc-8424-4c37-8b5d-de2cf2ffd886-credential-keys\") pod \"cf5d77fc-8424-4c37-8b5d-de2cf2ffd886\" (UID: \"cf5d77fc-8424-4c37-8b5d-de2cf2ffd886\") " Dec 03 12:17:46 crc kubenswrapper[5052]: I1203 12:17:46.758851 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/cf5d77fc-8424-4c37-8b5d-de2cf2ffd886-fernet-keys\") pod \"cf5d77fc-8424-4c37-8b5d-de2cf2ffd886\" (UID: \"cf5d77fc-8424-4c37-8b5d-de2cf2ffd886\") " Dec 03 12:17:46 crc kubenswrapper[5052]: I1203 12:17:46.758906 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hn8ds\" (UniqueName: \"kubernetes.io/projected/cf5d77fc-8424-4c37-8b5d-de2cf2ffd886-kube-api-access-hn8ds\") pod \"cf5d77fc-8424-4c37-8b5d-de2cf2ffd886\" (UID: \"cf5d77fc-8424-4c37-8b5d-de2cf2ffd886\") " Dec 03 12:17:46 crc kubenswrapper[5052]: I1203 12:17:46.758929 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/cf5d77fc-8424-4c37-8b5d-de2cf2ffd886-scripts\") pod \"cf5d77fc-8424-4c37-8b5d-de2cf2ffd886\" (UID: \"cf5d77fc-8424-4c37-8b5d-de2cf2ffd886\") " Dec 03 12:17:46 crc kubenswrapper[5052]: I1203 12:17:46.758986 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/cf5d77fc-8424-4c37-8b5d-de2cf2ffd886-combined-ca-bundle\") pod \"cf5d77fc-8424-4c37-8b5d-de2cf2ffd886\" (UID: \"cf5d77fc-8424-4c37-8b5d-de2cf2ffd886\") " Dec 03 12:17:46 crc kubenswrapper[5052]: I1203 12:17:46.765495 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cf5d77fc-8424-4c37-8b5d-de2cf2ffd886-scripts" (OuterVolumeSpecName: "scripts") pod "cf5d77fc-8424-4c37-8b5d-de2cf2ffd886" (UID: "cf5d77fc-8424-4c37-8b5d-de2cf2ffd886"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:17:46 crc kubenswrapper[5052]: I1203 12:17:46.770722 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cf5d77fc-8424-4c37-8b5d-de2cf2ffd886-fernet-keys" (OuterVolumeSpecName: "fernet-keys") pod "cf5d77fc-8424-4c37-8b5d-de2cf2ffd886" (UID: "cf5d77fc-8424-4c37-8b5d-de2cf2ffd886"). InnerVolumeSpecName "fernet-keys". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:17:46 crc kubenswrapper[5052]: I1203 12:17:46.770763 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cf5d77fc-8424-4c37-8b5d-de2cf2ffd886-credential-keys" (OuterVolumeSpecName: "credential-keys") pod "cf5d77fc-8424-4c37-8b5d-de2cf2ffd886" (UID: "cf5d77fc-8424-4c37-8b5d-de2cf2ffd886"). InnerVolumeSpecName "credential-keys". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:17:46 crc kubenswrapper[5052]: I1203 12:17:46.774609 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cf5d77fc-8424-4c37-8b5d-de2cf2ffd886-kube-api-access-hn8ds" (OuterVolumeSpecName: "kube-api-access-hn8ds") pod "cf5d77fc-8424-4c37-8b5d-de2cf2ffd886" (UID: "cf5d77fc-8424-4c37-8b5d-de2cf2ffd886"). InnerVolumeSpecName "kube-api-access-hn8ds". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:17:46 crc kubenswrapper[5052]: I1203 12:17:46.804576 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cf5d77fc-8424-4c37-8b5d-de2cf2ffd886-config-data" (OuterVolumeSpecName: "config-data") pod "cf5d77fc-8424-4c37-8b5d-de2cf2ffd886" (UID: "cf5d77fc-8424-4c37-8b5d-de2cf2ffd886"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:17:46 crc kubenswrapper[5052]: I1203 12:17:46.827275 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cf5d77fc-8424-4c37-8b5d-de2cf2ffd886-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "cf5d77fc-8424-4c37-8b5d-de2cf2ffd886" (UID: "cf5d77fc-8424-4c37-8b5d-de2cf2ffd886"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:17:46 crc kubenswrapper[5052]: I1203 12:17:46.862441 5052 reconciler_common.go:293] "Volume detached for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/cf5d77fc-8424-4c37-8b5d-de2cf2ffd886-credential-keys\") on node \"crc\" DevicePath \"\"" Dec 03 12:17:46 crc kubenswrapper[5052]: I1203 12:17:46.862501 5052 reconciler_common.go:293] "Volume detached for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/cf5d77fc-8424-4c37-8b5d-de2cf2ffd886-fernet-keys\") on node \"crc\" DevicePath \"\"" Dec 03 12:17:46 crc kubenswrapper[5052]: I1203 12:17:46.862532 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-hn8ds\" (UniqueName: \"kubernetes.io/projected/cf5d77fc-8424-4c37-8b5d-de2cf2ffd886-kube-api-access-hn8ds\") on node \"crc\" DevicePath \"\"" Dec 03 12:17:46 crc kubenswrapper[5052]: I1203 12:17:46.862546 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/cf5d77fc-8424-4c37-8b5d-de2cf2ffd886-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:17:46 crc kubenswrapper[5052]: I1203 12:17:46.862556 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/cf5d77fc-8424-4c37-8b5d-de2cf2ffd886-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:17:46 crc kubenswrapper[5052]: I1203 12:17:46.862586 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/cf5d77fc-8424-4c37-8b5d-de2cf2ffd886-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:17:47 crc kubenswrapper[5052]: I1203 12:17:47.168839 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-bootstrap-cz9xr" event={"ID":"cf5d77fc-8424-4c37-8b5d-de2cf2ffd886","Type":"ContainerDied","Data":"8fb3a8fad238826f1dfe601696b70ef88cf10f3bffa97819718c8aefdad07af3"} Dec 03 12:17:47 crc kubenswrapper[5052]: I1203 12:17:47.169376 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8fb3a8fad238826f1dfe601696b70ef88cf10f3bffa97819718c8aefdad07af3" Dec 03 12:17:47 crc kubenswrapper[5052]: I1203 12:17:47.169171 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-bootstrap-cz9xr" Dec 03 12:17:47 crc kubenswrapper[5052]: I1203 12:17:47.190153 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"321c0d14-7d66-4b3a-8989-f6a161421d2e","Type":"ContainerStarted","Data":"8d6a07cb12d95665a4437d877dc9f8c2c418b523de12dc71840ac557d95e3c39"} Dec 03 12:17:47 crc kubenswrapper[5052]: I1203 12:17:47.192150 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"908e450d-211f-4b2f-8f21-332e6a18e0c1","Type":"ContainerStarted","Data":"f945ed9ca1d6ac3eeaa5a79754ae1e40a268ea6c472f4545af321737a203d3c4"} Dec 03 12:17:47 crc kubenswrapper[5052]: I1203 12:17:47.192467 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:17:47 crc kubenswrapper[5052]: I1203 12:17:47.192493 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:17:47 crc kubenswrapper[5052]: I1203 12:17:47.305409 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/keystone-78dd58875-slm9r"] Dec 03 12:17:47 crc kubenswrapper[5052]: E1203 12:17:47.308651 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="cf5d77fc-8424-4c37-8b5d-de2cf2ffd886" containerName="keystone-bootstrap" Dec 03 12:17:47 crc kubenswrapper[5052]: I1203 12:17:47.308684 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="cf5d77fc-8424-4c37-8b5d-de2cf2ffd886" containerName="keystone-bootstrap" Dec 03 12:17:47 crc kubenswrapper[5052]: I1203 12:17:47.308855 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="cf5d77fc-8424-4c37-8b5d-de2cf2ffd886" containerName="keystone-bootstrap" Dec 03 12:17:47 crc kubenswrapper[5052]: I1203 12:17:47.310203 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-78dd58875-slm9r" Dec 03 12:17:47 crc kubenswrapper[5052]: I1203 12:17:47.314358 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone-config-data" Dec 03 12:17:47 crc kubenswrapper[5052]: I1203 12:17:47.319410 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-keystone-public-svc" Dec 03 12:17:47 crc kubenswrapper[5052]: I1203 12:17:47.327749 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone-scripts" Dec 03 12:17:47 crc kubenswrapper[5052]: I1203 12:17:47.328128 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone" Dec 03 12:17:47 crc kubenswrapper[5052]: I1203 12:17:47.328186 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-keystone-internal-svc" Dec 03 12:17:47 crc kubenswrapper[5052]: I1203 12:17:47.328396 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone-keystone-dockercfg-kkks7" Dec 03 12:17:47 crc kubenswrapper[5052]: I1203 12:17:47.361663 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-78dd58875-slm9r"] Dec 03 12:17:47 crc kubenswrapper[5052]: I1203 12:17:47.376906 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4107ef62-fa32-4d24-a7ad-3e2eb59884de-combined-ca-bundle\") pod \"keystone-78dd58875-slm9r\" (UID: \"4107ef62-fa32-4d24-a7ad-3e2eb59884de\") " pod="openstack-kuttl-tests/keystone-78dd58875-slm9r" Dec 03 12:17:47 crc kubenswrapper[5052]: I1203 12:17:47.377128 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/4107ef62-fa32-4d24-a7ad-3e2eb59884de-internal-tls-certs\") pod \"keystone-78dd58875-slm9r\" (UID: \"4107ef62-fa32-4d24-a7ad-3e2eb59884de\") " pod="openstack-kuttl-tests/keystone-78dd58875-slm9r" Dec 03 12:17:47 crc kubenswrapper[5052]: I1203 12:17:47.377274 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/4107ef62-fa32-4d24-a7ad-3e2eb59884de-fernet-keys\") pod \"keystone-78dd58875-slm9r\" (UID: \"4107ef62-fa32-4d24-a7ad-3e2eb59884de\") " pod="openstack-kuttl-tests/keystone-78dd58875-slm9r" Dec 03 12:17:47 crc kubenswrapper[5052]: I1203 12:17:47.377308 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pl8gv\" (UniqueName: \"kubernetes.io/projected/4107ef62-fa32-4d24-a7ad-3e2eb59884de-kube-api-access-pl8gv\") pod \"keystone-78dd58875-slm9r\" (UID: \"4107ef62-fa32-4d24-a7ad-3e2eb59884de\") " pod="openstack-kuttl-tests/keystone-78dd58875-slm9r" Dec 03 12:17:47 crc kubenswrapper[5052]: I1203 12:17:47.377351 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/4107ef62-fa32-4d24-a7ad-3e2eb59884de-scripts\") pod \"keystone-78dd58875-slm9r\" (UID: \"4107ef62-fa32-4d24-a7ad-3e2eb59884de\") " pod="openstack-kuttl-tests/keystone-78dd58875-slm9r" Dec 03 12:17:47 crc kubenswrapper[5052]: I1203 12:17:47.377394 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/4107ef62-fa32-4d24-a7ad-3e2eb59884de-credential-keys\") pod \"keystone-78dd58875-slm9r\" (UID: \"4107ef62-fa32-4d24-a7ad-3e2eb59884de\") " pod="openstack-kuttl-tests/keystone-78dd58875-slm9r" Dec 03 12:17:47 crc kubenswrapper[5052]: I1203 12:17:47.377419 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/4107ef62-fa32-4d24-a7ad-3e2eb59884de-public-tls-certs\") pod \"keystone-78dd58875-slm9r\" (UID: \"4107ef62-fa32-4d24-a7ad-3e2eb59884de\") " pod="openstack-kuttl-tests/keystone-78dd58875-slm9r" Dec 03 12:17:47 crc kubenswrapper[5052]: I1203 12:17:47.377484 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/4107ef62-fa32-4d24-a7ad-3e2eb59884de-config-data\") pod \"keystone-78dd58875-slm9r\" (UID: \"4107ef62-fa32-4d24-a7ad-3e2eb59884de\") " pod="openstack-kuttl-tests/keystone-78dd58875-slm9r" Dec 03 12:17:47 crc kubenswrapper[5052]: I1203 12:17:47.411846 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:17:47 crc kubenswrapper[5052]: I1203 12:17:47.411991 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:17:47 crc kubenswrapper[5052]: I1203 12:17:47.480733 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/4107ef62-fa32-4d24-a7ad-3e2eb59884de-fernet-keys\") pod \"keystone-78dd58875-slm9r\" (UID: \"4107ef62-fa32-4d24-a7ad-3e2eb59884de\") " pod="openstack-kuttl-tests/keystone-78dd58875-slm9r" Dec 03 12:17:47 crc kubenswrapper[5052]: I1203 12:17:47.480853 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-pl8gv\" (UniqueName: \"kubernetes.io/projected/4107ef62-fa32-4d24-a7ad-3e2eb59884de-kube-api-access-pl8gv\") pod \"keystone-78dd58875-slm9r\" (UID: \"4107ef62-fa32-4d24-a7ad-3e2eb59884de\") " pod="openstack-kuttl-tests/keystone-78dd58875-slm9r" Dec 03 12:17:47 crc kubenswrapper[5052]: I1203 12:17:47.480937 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/4107ef62-fa32-4d24-a7ad-3e2eb59884de-scripts\") pod \"keystone-78dd58875-slm9r\" (UID: \"4107ef62-fa32-4d24-a7ad-3e2eb59884de\") " pod="openstack-kuttl-tests/keystone-78dd58875-slm9r" Dec 03 12:17:47 crc kubenswrapper[5052]: I1203 12:17:47.481018 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/4107ef62-fa32-4d24-a7ad-3e2eb59884de-credential-keys\") pod \"keystone-78dd58875-slm9r\" (UID: \"4107ef62-fa32-4d24-a7ad-3e2eb59884de\") " pod="openstack-kuttl-tests/keystone-78dd58875-slm9r" Dec 03 12:17:47 crc kubenswrapper[5052]: I1203 12:17:47.481080 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/4107ef62-fa32-4d24-a7ad-3e2eb59884de-public-tls-certs\") pod \"keystone-78dd58875-slm9r\" (UID: \"4107ef62-fa32-4d24-a7ad-3e2eb59884de\") " pod="openstack-kuttl-tests/keystone-78dd58875-slm9r" Dec 03 12:17:47 crc kubenswrapper[5052]: I1203 12:17:47.481112 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/4107ef62-fa32-4d24-a7ad-3e2eb59884de-config-data\") pod \"keystone-78dd58875-slm9r\" (UID: \"4107ef62-fa32-4d24-a7ad-3e2eb59884de\") " pod="openstack-kuttl-tests/keystone-78dd58875-slm9r" Dec 03 12:17:47 crc kubenswrapper[5052]: I1203 12:17:47.481379 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4107ef62-fa32-4d24-a7ad-3e2eb59884de-combined-ca-bundle\") pod \"keystone-78dd58875-slm9r\" (UID: \"4107ef62-fa32-4d24-a7ad-3e2eb59884de\") " pod="openstack-kuttl-tests/keystone-78dd58875-slm9r" Dec 03 12:17:47 crc kubenswrapper[5052]: I1203 12:17:47.481427 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/4107ef62-fa32-4d24-a7ad-3e2eb59884de-internal-tls-certs\") pod \"keystone-78dd58875-slm9r\" (UID: \"4107ef62-fa32-4d24-a7ad-3e2eb59884de\") " pod="openstack-kuttl-tests/keystone-78dd58875-slm9r" Dec 03 12:17:47 crc kubenswrapper[5052]: I1203 12:17:47.503062 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:17:47 crc kubenswrapper[5052]: I1203 12:17:47.504435 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:17:47 crc kubenswrapper[5052]: I1203 12:17:47.508138 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/4107ef62-fa32-4d24-a7ad-3e2eb59884de-credential-keys\") pod \"keystone-78dd58875-slm9r\" (UID: \"4107ef62-fa32-4d24-a7ad-3e2eb59884de\") " pod="openstack-kuttl-tests/keystone-78dd58875-slm9r" Dec 03 12:17:47 crc kubenswrapper[5052]: I1203 12:17:47.508340 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/4107ef62-fa32-4d24-a7ad-3e2eb59884de-fernet-keys\") pod \"keystone-78dd58875-slm9r\" (UID: \"4107ef62-fa32-4d24-a7ad-3e2eb59884de\") " pod="openstack-kuttl-tests/keystone-78dd58875-slm9r" Dec 03 12:17:47 crc kubenswrapper[5052]: I1203 12:17:47.513718 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-pl8gv\" (UniqueName: \"kubernetes.io/projected/4107ef62-fa32-4d24-a7ad-3e2eb59884de-kube-api-access-pl8gv\") pod \"keystone-78dd58875-slm9r\" (UID: \"4107ef62-fa32-4d24-a7ad-3e2eb59884de\") " pod="openstack-kuttl-tests/keystone-78dd58875-slm9r" Dec 03 12:17:47 crc kubenswrapper[5052]: I1203 12:17:47.513854 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/4107ef62-fa32-4d24-a7ad-3e2eb59884de-scripts\") pod \"keystone-78dd58875-slm9r\" (UID: \"4107ef62-fa32-4d24-a7ad-3e2eb59884de\") " pod="openstack-kuttl-tests/keystone-78dd58875-slm9r" Dec 03 12:17:47 crc kubenswrapper[5052]: I1203 12:17:47.514337 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/4107ef62-fa32-4d24-a7ad-3e2eb59884de-public-tls-certs\") pod \"keystone-78dd58875-slm9r\" (UID: \"4107ef62-fa32-4d24-a7ad-3e2eb59884de\") " pod="openstack-kuttl-tests/keystone-78dd58875-slm9r" Dec 03 12:17:47 crc kubenswrapper[5052]: I1203 12:17:47.514819 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/4107ef62-fa32-4d24-a7ad-3e2eb59884de-config-data\") pod \"keystone-78dd58875-slm9r\" (UID: \"4107ef62-fa32-4d24-a7ad-3e2eb59884de\") " pod="openstack-kuttl-tests/keystone-78dd58875-slm9r" Dec 03 12:17:47 crc kubenswrapper[5052]: I1203 12:17:47.516885 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/4107ef62-fa32-4d24-a7ad-3e2eb59884de-internal-tls-certs\") pod \"keystone-78dd58875-slm9r\" (UID: \"4107ef62-fa32-4d24-a7ad-3e2eb59884de\") " pod="openstack-kuttl-tests/keystone-78dd58875-slm9r" Dec 03 12:17:47 crc kubenswrapper[5052]: I1203 12:17:47.517002 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4107ef62-fa32-4d24-a7ad-3e2eb59884de-combined-ca-bundle\") pod \"keystone-78dd58875-slm9r\" (UID: \"4107ef62-fa32-4d24-a7ad-3e2eb59884de\") " pod="openstack-kuttl-tests/keystone-78dd58875-slm9r" Dec 03 12:17:47 crc kubenswrapper[5052]: I1203 12:17:47.705340 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-78dd58875-slm9r" Dec 03 12:17:47 crc kubenswrapper[5052]: I1203 12:17:47.985870 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 12:17:48 crc kubenswrapper[5052]: I1203 12:17:48.204471 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"321c0d14-7d66-4b3a-8989-f6a161421d2e","Type":"ContainerStarted","Data":"19aa0a6013cd529547763657024a5ae0ee17db18c9cc49526b6066bb7c447d1b"} Dec 03 12:17:48 crc kubenswrapper[5052]: I1203 12:17:48.208265 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"908e450d-211f-4b2f-8f21-332e6a18e0c1","Type":"ContainerStarted","Data":"6f8e865a7a757567cbf456b45f1930f531b007aa607fbb497b7ea9420fe0ff69"} Dec 03 12:17:48 crc kubenswrapper[5052]: I1203 12:17:48.209486 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:17:48 crc kubenswrapper[5052]: I1203 12:17:48.209649 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:17:48 crc kubenswrapper[5052]: I1203 12:17:48.461548 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-78dd58875-slm9r"] Dec 03 12:17:49 crc kubenswrapper[5052]: I1203 12:17:49.247585 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"321c0d14-7d66-4b3a-8989-f6a161421d2e","Type":"ContainerStarted","Data":"b800ece057aa8281071c80450e628f23c676cb4fc4228db9b5d9cd509984d054"} Dec 03 12:17:49 crc kubenswrapper[5052]: I1203 12:17:49.256826 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"908e450d-211f-4b2f-8f21-332e6a18e0c1","Type":"ContainerStarted","Data":"f5598fa3249017b0a3d8e71f1580576adea34395145eccefa85117608e9f04e3"} Dec 03 12:17:49 crc kubenswrapper[5052]: I1203 12:17:49.257175 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/cinder-api-0" podUID="908e450d-211f-4b2f-8f21-332e6a18e0c1" containerName="cinder-api-log" containerID="cri-o://6f8e865a7a757567cbf456b45f1930f531b007aa607fbb497b7ea9420fe0ff69" gracePeriod=30 Dec 03 12:17:49 crc kubenswrapper[5052]: I1203 12:17:49.257747 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:17:49 crc kubenswrapper[5052]: I1203 12:17:49.257798 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/cinder-api-0" podUID="908e450d-211f-4b2f-8f21-332e6a18e0c1" containerName="cinder-api" containerID="cri-o://f5598fa3249017b0a3d8e71f1580576adea34395145eccefa85117608e9f04e3" gracePeriod=30 Dec 03 12:17:49 crc kubenswrapper[5052]: I1203 12:17:49.271624 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-78dd58875-slm9r" event={"ID":"4107ef62-fa32-4d24-a7ad-3e2eb59884de","Type":"ContainerStarted","Data":"e93ca0a9fbc6106e1c59be08c9244f1adaba71b37a9be94ca1173d57e88ad162"} Dec 03 12:17:49 crc kubenswrapper[5052]: I1203 12:17:49.271694 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-78dd58875-slm9r" event={"ID":"4107ef62-fa32-4d24-a7ad-3e2eb59884de","Type":"ContainerStarted","Data":"1d45c10dc60a6974996075c035507884e5f15a3d87050e09496d080d220b5d2b"} Dec 03 12:17:49 crc kubenswrapper[5052]: I1203 12:17:49.271921 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/keystone-78dd58875-slm9r" Dec 03 12:17:49 crc kubenswrapper[5052]: I1203 12:17:49.328292 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/cinder-scheduler-0" podStartSLOduration=4.32826848 podStartE2EDuration="4.32826848s" podCreationTimestamp="2025-12-03 12:17:45 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:17:49.289531484 +0000 UTC m=+3459.877773248" watchObservedRunningTime="2025-12-03 12:17:49.32826848 +0000 UTC m=+3459.916510254" Dec 03 12:17:49 crc kubenswrapper[5052]: I1203 12:17:49.331558 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/keystone-78dd58875-slm9r" podStartSLOduration=2.331537773 podStartE2EDuration="2.331537773s" podCreationTimestamp="2025-12-03 12:17:47 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:17:49.309127523 +0000 UTC m=+3459.897369297" watchObservedRunningTime="2025-12-03 12:17:49.331537773 +0000 UTC m=+3459.919779547" Dec 03 12:17:49 crc kubenswrapper[5052]: I1203 12:17:49.351551 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/cinder-api-0" podStartSLOduration=4.351502453 podStartE2EDuration="4.351502453s" podCreationTimestamp="2025-12-03 12:17:45 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:17:49.339587043 +0000 UTC m=+3459.927828807" watchObservedRunningTime="2025-12-03 12:17:49.351502453 +0000 UTC m=+3459.939744217" Dec 03 12:17:49 crc kubenswrapper[5052]: I1203 12:17:49.904153 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:17:49 crc kubenswrapper[5052]: I1203 12:17:49.904475 5052 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Dec 03 12:17:50 crc kubenswrapper[5052]: I1203 12:17:50.109380 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:17:50 crc kubenswrapper[5052]: I1203 12:17:50.255571 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/908e450d-211f-4b2f-8f21-332e6a18e0c1-combined-ca-bundle\") pod \"908e450d-211f-4b2f-8f21-332e6a18e0c1\" (UID: \"908e450d-211f-4b2f-8f21-332e6a18e0c1\") " Dec 03 12:17:50 crc kubenswrapper[5052]: I1203 12:17:50.255696 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/908e450d-211f-4b2f-8f21-332e6a18e0c1-logs\") pod \"908e450d-211f-4b2f-8f21-332e6a18e0c1\" (UID: \"908e450d-211f-4b2f-8f21-332e6a18e0c1\") " Dec 03 12:17:50 crc kubenswrapper[5052]: I1203 12:17:50.255790 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/908e450d-211f-4b2f-8f21-332e6a18e0c1-scripts\") pod \"908e450d-211f-4b2f-8f21-332e6a18e0c1\" (UID: \"908e450d-211f-4b2f-8f21-332e6a18e0c1\") " Dec 03 12:17:50 crc kubenswrapper[5052]: I1203 12:17:50.255817 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/908e450d-211f-4b2f-8f21-332e6a18e0c1-config-data\") pod \"908e450d-211f-4b2f-8f21-332e6a18e0c1\" (UID: \"908e450d-211f-4b2f-8f21-332e6a18e0c1\") " Dec 03 12:17:50 crc kubenswrapper[5052]: I1203 12:17:50.255865 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5fnj5\" (UniqueName: \"kubernetes.io/projected/908e450d-211f-4b2f-8f21-332e6a18e0c1-kube-api-access-5fnj5\") pod \"908e450d-211f-4b2f-8f21-332e6a18e0c1\" (UID: \"908e450d-211f-4b2f-8f21-332e6a18e0c1\") " Dec 03 12:17:50 crc kubenswrapper[5052]: I1203 12:17:50.255904 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/908e450d-211f-4b2f-8f21-332e6a18e0c1-config-data-custom\") pod \"908e450d-211f-4b2f-8f21-332e6a18e0c1\" (UID: \"908e450d-211f-4b2f-8f21-332e6a18e0c1\") " Dec 03 12:17:50 crc kubenswrapper[5052]: I1203 12:17:50.255940 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/908e450d-211f-4b2f-8f21-332e6a18e0c1-etc-machine-id\") pod \"908e450d-211f-4b2f-8f21-332e6a18e0c1\" (UID: \"908e450d-211f-4b2f-8f21-332e6a18e0c1\") " Dec 03 12:17:50 crc kubenswrapper[5052]: I1203 12:17:50.256316 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/908e450d-211f-4b2f-8f21-332e6a18e0c1-etc-machine-id" (OuterVolumeSpecName: "etc-machine-id") pod "908e450d-211f-4b2f-8f21-332e6a18e0c1" (UID: "908e450d-211f-4b2f-8f21-332e6a18e0c1"). InnerVolumeSpecName "etc-machine-id". PluginName "kubernetes.io/host-path", VolumeGidValue "" Dec 03 12:17:50 crc kubenswrapper[5052]: I1203 12:17:50.257700 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/908e450d-211f-4b2f-8f21-332e6a18e0c1-logs" (OuterVolumeSpecName: "logs") pod "908e450d-211f-4b2f-8f21-332e6a18e0c1" (UID: "908e450d-211f-4b2f-8f21-332e6a18e0c1"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:17:50 crc kubenswrapper[5052]: I1203 12:17:50.269701 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/908e450d-211f-4b2f-8f21-332e6a18e0c1-kube-api-access-5fnj5" (OuterVolumeSpecName: "kube-api-access-5fnj5") pod "908e450d-211f-4b2f-8f21-332e6a18e0c1" (UID: "908e450d-211f-4b2f-8f21-332e6a18e0c1"). InnerVolumeSpecName "kube-api-access-5fnj5". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:17:50 crc kubenswrapper[5052]: I1203 12:17:50.269723 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/908e450d-211f-4b2f-8f21-332e6a18e0c1-scripts" (OuterVolumeSpecName: "scripts") pod "908e450d-211f-4b2f-8f21-332e6a18e0c1" (UID: "908e450d-211f-4b2f-8f21-332e6a18e0c1"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:17:50 crc kubenswrapper[5052]: I1203 12:17:50.269740 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/908e450d-211f-4b2f-8f21-332e6a18e0c1-config-data-custom" (OuterVolumeSpecName: "config-data-custom") pod "908e450d-211f-4b2f-8f21-332e6a18e0c1" (UID: "908e450d-211f-4b2f-8f21-332e6a18e0c1"). InnerVolumeSpecName "config-data-custom". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:17:50 crc kubenswrapper[5052]: I1203 12:17:50.293258 5052 generic.go:334] "Generic (PLEG): container finished" podID="908e450d-211f-4b2f-8f21-332e6a18e0c1" containerID="f5598fa3249017b0a3d8e71f1580576adea34395145eccefa85117608e9f04e3" exitCode=0 Dec 03 12:17:50 crc kubenswrapper[5052]: I1203 12:17:50.293291 5052 generic.go:334] "Generic (PLEG): container finished" podID="908e450d-211f-4b2f-8f21-332e6a18e0c1" containerID="6f8e865a7a757567cbf456b45f1930f531b007aa607fbb497b7ea9420fe0ff69" exitCode=143 Dec 03 12:17:50 crc kubenswrapper[5052]: I1203 12:17:50.293361 5052 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Dec 03 12:17:50 crc kubenswrapper[5052]: I1203 12:17:50.293368 5052 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Dec 03 12:17:50 crc kubenswrapper[5052]: I1203 12:17:50.293615 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"908e450d-211f-4b2f-8f21-332e6a18e0c1","Type":"ContainerDied","Data":"f5598fa3249017b0a3d8e71f1580576adea34395145eccefa85117608e9f04e3"} Dec 03 12:17:50 crc kubenswrapper[5052]: I1203 12:17:50.293652 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"908e450d-211f-4b2f-8f21-332e6a18e0c1","Type":"ContainerDied","Data":"6f8e865a7a757567cbf456b45f1930f531b007aa607fbb497b7ea9420fe0ff69"} Dec 03 12:17:50 crc kubenswrapper[5052]: I1203 12:17:50.293666 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"908e450d-211f-4b2f-8f21-332e6a18e0c1","Type":"ContainerDied","Data":"f945ed9ca1d6ac3eeaa5a79754ae1e40a268ea6c472f4545af321737a203d3c4"} Dec 03 12:17:50 crc kubenswrapper[5052]: I1203 12:17:50.293686 5052 scope.go:117] "RemoveContainer" containerID="f5598fa3249017b0a3d8e71f1580576adea34395145eccefa85117608e9f04e3" Dec 03 12:17:50 crc kubenswrapper[5052]: I1203 12:17:50.293820 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:17:50 crc kubenswrapper[5052]: I1203 12:17:50.313825 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/908e450d-211f-4b2f-8f21-332e6a18e0c1-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "908e450d-211f-4b2f-8f21-332e6a18e0c1" (UID: "908e450d-211f-4b2f-8f21-332e6a18e0c1"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:17:50 crc kubenswrapper[5052]: I1203 12:17:50.361296 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/908e450d-211f-4b2f-8f21-332e6a18e0c1-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:17:50 crc kubenswrapper[5052]: I1203 12:17:50.361358 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-5fnj5\" (UniqueName: \"kubernetes.io/projected/908e450d-211f-4b2f-8f21-332e6a18e0c1-kube-api-access-5fnj5\") on node \"crc\" DevicePath \"\"" Dec 03 12:17:50 crc kubenswrapper[5052]: I1203 12:17:50.361375 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/908e450d-211f-4b2f-8f21-332e6a18e0c1-config-data-custom\") on node \"crc\" DevicePath \"\"" Dec 03 12:17:50 crc kubenswrapper[5052]: I1203 12:17:50.361387 5052 reconciler_common.go:293] "Volume detached for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/908e450d-211f-4b2f-8f21-332e6a18e0c1-etc-machine-id\") on node \"crc\" DevicePath \"\"" Dec 03 12:17:50 crc kubenswrapper[5052]: I1203 12:17:50.361399 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/908e450d-211f-4b2f-8f21-332e6a18e0c1-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:17:50 crc kubenswrapper[5052]: I1203 12:17:50.361412 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/908e450d-211f-4b2f-8f21-332e6a18e0c1-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:17:50 crc kubenswrapper[5052]: I1203 12:17:50.366341 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/908e450d-211f-4b2f-8f21-332e6a18e0c1-config-data" (OuterVolumeSpecName: "config-data") pod "908e450d-211f-4b2f-8f21-332e6a18e0c1" (UID: "908e450d-211f-4b2f-8f21-332e6a18e0c1"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:17:50 crc kubenswrapper[5052]: I1203 12:17:50.401063 5052 scope.go:117] "RemoveContainer" containerID="6f8e865a7a757567cbf456b45f1930f531b007aa607fbb497b7ea9420fe0ff69" Dec 03 12:17:50 crc kubenswrapper[5052]: I1203 12:17:50.437481 5052 scope.go:117] "RemoveContainer" containerID="f5598fa3249017b0a3d8e71f1580576adea34395145eccefa85117608e9f04e3" Dec 03 12:17:50 crc kubenswrapper[5052]: E1203 12:17:50.440163 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f5598fa3249017b0a3d8e71f1580576adea34395145eccefa85117608e9f04e3\": container with ID starting with f5598fa3249017b0a3d8e71f1580576adea34395145eccefa85117608e9f04e3 not found: ID does not exist" containerID="f5598fa3249017b0a3d8e71f1580576adea34395145eccefa85117608e9f04e3" Dec 03 12:17:50 crc kubenswrapper[5052]: I1203 12:17:50.440200 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f5598fa3249017b0a3d8e71f1580576adea34395145eccefa85117608e9f04e3"} err="failed to get container status \"f5598fa3249017b0a3d8e71f1580576adea34395145eccefa85117608e9f04e3\": rpc error: code = NotFound desc = could not find container \"f5598fa3249017b0a3d8e71f1580576adea34395145eccefa85117608e9f04e3\": container with ID starting with f5598fa3249017b0a3d8e71f1580576adea34395145eccefa85117608e9f04e3 not found: ID does not exist" Dec 03 12:17:50 crc kubenswrapper[5052]: I1203 12:17:50.440221 5052 scope.go:117] "RemoveContainer" containerID="6f8e865a7a757567cbf456b45f1930f531b007aa607fbb497b7ea9420fe0ff69" Dec 03 12:17:50 crc kubenswrapper[5052]: E1203 12:17:50.443824 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6f8e865a7a757567cbf456b45f1930f531b007aa607fbb497b7ea9420fe0ff69\": container with ID starting with 6f8e865a7a757567cbf456b45f1930f531b007aa607fbb497b7ea9420fe0ff69 not found: ID does not exist" containerID="6f8e865a7a757567cbf456b45f1930f531b007aa607fbb497b7ea9420fe0ff69" Dec 03 12:17:50 crc kubenswrapper[5052]: I1203 12:17:50.443857 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f8e865a7a757567cbf456b45f1930f531b007aa607fbb497b7ea9420fe0ff69"} err="failed to get container status \"6f8e865a7a757567cbf456b45f1930f531b007aa607fbb497b7ea9420fe0ff69\": rpc error: code = NotFound desc = could not find container \"6f8e865a7a757567cbf456b45f1930f531b007aa607fbb497b7ea9420fe0ff69\": container with ID starting with 6f8e865a7a757567cbf456b45f1930f531b007aa607fbb497b7ea9420fe0ff69 not found: ID does not exist" Dec 03 12:17:50 crc kubenswrapper[5052]: I1203 12:17:50.443876 5052 scope.go:117] "RemoveContainer" containerID="f5598fa3249017b0a3d8e71f1580576adea34395145eccefa85117608e9f04e3" Dec 03 12:17:50 crc kubenswrapper[5052]: I1203 12:17:50.447747 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f5598fa3249017b0a3d8e71f1580576adea34395145eccefa85117608e9f04e3"} err="failed to get container status \"f5598fa3249017b0a3d8e71f1580576adea34395145eccefa85117608e9f04e3\": rpc error: code = NotFound desc = could not find container \"f5598fa3249017b0a3d8e71f1580576adea34395145eccefa85117608e9f04e3\": container with ID starting with f5598fa3249017b0a3d8e71f1580576adea34395145eccefa85117608e9f04e3 not found: ID does not exist" Dec 03 12:17:50 crc kubenswrapper[5052]: I1203 12:17:50.447788 5052 scope.go:117] "RemoveContainer" containerID="6f8e865a7a757567cbf456b45f1930f531b007aa607fbb497b7ea9420fe0ff69" Dec 03 12:17:50 crc kubenswrapper[5052]: I1203 12:17:50.451835 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f8e865a7a757567cbf456b45f1930f531b007aa607fbb497b7ea9420fe0ff69"} err="failed to get container status \"6f8e865a7a757567cbf456b45f1930f531b007aa607fbb497b7ea9420fe0ff69\": rpc error: code = NotFound desc = could not find container \"6f8e865a7a757567cbf456b45f1930f531b007aa607fbb497b7ea9420fe0ff69\": container with ID starting with 6f8e865a7a757567cbf456b45f1930f531b007aa607fbb497b7ea9420fe0ff69 not found: ID does not exist" Dec 03 12:17:50 crc kubenswrapper[5052]: I1203 12:17:50.463110 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/908e450d-211f-4b2f-8f21-332e6a18e0c1-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:17:50 crc kubenswrapper[5052]: I1203 12:17:50.506189 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:17:50 crc kubenswrapper[5052]: I1203 12:17:50.652578 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 12:17:50 crc kubenswrapper[5052]: I1203 12:17:50.659722 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 12:17:50 crc kubenswrapper[5052]: I1203 12:17:50.698556 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 12:17:50 crc kubenswrapper[5052]: E1203 12:17:50.699118 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="908e450d-211f-4b2f-8f21-332e6a18e0c1" containerName="cinder-api-log" Dec 03 12:17:50 crc kubenswrapper[5052]: I1203 12:17:50.699140 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="908e450d-211f-4b2f-8f21-332e6a18e0c1" containerName="cinder-api-log" Dec 03 12:17:50 crc kubenswrapper[5052]: E1203 12:17:50.699154 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="908e450d-211f-4b2f-8f21-332e6a18e0c1" containerName="cinder-api" Dec 03 12:17:50 crc kubenswrapper[5052]: I1203 12:17:50.699161 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="908e450d-211f-4b2f-8f21-332e6a18e0c1" containerName="cinder-api" Dec 03 12:17:50 crc kubenswrapper[5052]: I1203 12:17:50.699355 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="908e450d-211f-4b2f-8f21-332e6a18e0c1" containerName="cinder-api-log" Dec 03 12:17:50 crc kubenswrapper[5052]: I1203 12:17:50.699373 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="908e450d-211f-4b2f-8f21-332e6a18e0c1" containerName="cinder-api" Dec 03 12:17:50 crc kubenswrapper[5052]: I1203 12:17:50.700278 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:17:50 crc kubenswrapper[5052]: I1203 12:17:50.706023 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-cinder-internal-svc" Dec 03 12:17:50 crc kubenswrapper[5052]: I1203 12:17:50.706276 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-cinder-public-svc" Dec 03 12:17:50 crc kubenswrapper[5052]: I1203 12:17:50.706777 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cinder-api-config-data" Dec 03 12:17:50 crc kubenswrapper[5052]: I1203 12:17:50.732211 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 12:17:50 crc kubenswrapper[5052]: I1203 12:17:50.773476 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/99a08445-670e-4194-bdb1-1e7b79dc00f3-public-tls-certs\") pod \"cinder-api-0\" (UID: \"99a08445-670e-4194-bdb1-1e7b79dc00f3\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:17:50 crc kubenswrapper[5052]: I1203 12:17:50.773586 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/99a08445-670e-4194-bdb1-1e7b79dc00f3-config-data-custom\") pod \"cinder-api-0\" (UID: \"99a08445-670e-4194-bdb1-1e7b79dc00f3\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:17:50 crc kubenswrapper[5052]: I1203 12:17:50.773617 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/99a08445-670e-4194-bdb1-1e7b79dc00f3-internal-tls-certs\") pod \"cinder-api-0\" (UID: \"99a08445-670e-4194-bdb1-1e7b79dc00f3\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:17:50 crc kubenswrapper[5052]: I1203 12:17:50.773646 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/99a08445-670e-4194-bdb1-1e7b79dc00f3-config-data\") pod \"cinder-api-0\" (UID: \"99a08445-670e-4194-bdb1-1e7b79dc00f3\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:17:50 crc kubenswrapper[5052]: I1203 12:17:50.773675 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/99a08445-670e-4194-bdb1-1e7b79dc00f3-logs\") pod \"cinder-api-0\" (UID: \"99a08445-670e-4194-bdb1-1e7b79dc00f3\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:17:50 crc kubenswrapper[5052]: I1203 12:17:50.773703 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/99a08445-670e-4194-bdb1-1e7b79dc00f3-combined-ca-bundle\") pod \"cinder-api-0\" (UID: \"99a08445-670e-4194-bdb1-1e7b79dc00f3\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:17:50 crc kubenswrapper[5052]: I1203 12:17:50.773735 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/99a08445-670e-4194-bdb1-1e7b79dc00f3-scripts\") pod \"cinder-api-0\" (UID: \"99a08445-670e-4194-bdb1-1e7b79dc00f3\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:17:50 crc kubenswrapper[5052]: I1203 12:17:50.773764 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/99a08445-670e-4194-bdb1-1e7b79dc00f3-etc-machine-id\") pod \"cinder-api-0\" (UID: \"99a08445-670e-4194-bdb1-1e7b79dc00f3\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:17:50 crc kubenswrapper[5052]: I1203 12:17:50.773791 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cs5ln\" (UniqueName: \"kubernetes.io/projected/99a08445-670e-4194-bdb1-1e7b79dc00f3-kube-api-access-cs5ln\") pod \"cinder-api-0\" (UID: \"99a08445-670e-4194-bdb1-1e7b79dc00f3\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:17:50 crc kubenswrapper[5052]: I1203 12:17:50.787245 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:17:50 crc kubenswrapper[5052]: I1203 12:17:50.874796 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/99a08445-670e-4194-bdb1-1e7b79dc00f3-public-tls-certs\") pod \"cinder-api-0\" (UID: \"99a08445-670e-4194-bdb1-1e7b79dc00f3\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:17:50 crc kubenswrapper[5052]: I1203 12:17:50.874864 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/99a08445-670e-4194-bdb1-1e7b79dc00f3-config-data-custom\") pod \"cinder-api-0\" (UID: \"99a08445-670e-4194-bdb1-1e7b79dc00f3\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:17:50 crc kubenswrapper[5052]: I1203 12:17:50.874885 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/99a08445-670e-4194-bdb1-1e7b79dc00f3-internal-tls-certs\") pod \"cinder-api-0\" (UID: \"99a08445-670e-4194-bdb1-1e7b79dc00f3\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:17:50 crc kubenswrapper[5052]: I1203 12:17:50.874916 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/99a08445-670e-4194-bdb1-1e7b79dc00f3-config-data\") pod \"cinder-api-0\" (UID: \"99a08445-670e-4194-bdb1-1e7b79dc00f3\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:17:50 crc kubenswrapper[5052]: I1203 12:17:50.874965 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/99a08445-670e-4194-bdb1-1e7b79dc00f3-logs\") pod \"cinder-api-0\" (UID: \"99a08445-670e-4194-bdb1-1e7b79dc00f3\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:17:50 crc kubenswrapper[5052]: I1203 12:17:50.874994 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/99a08445-670e-4194-bdb1-1e7b79dc00f3-combined-ca-bundle\") pod \"cinder-api-0\" (UID: \"99a08445-670e-4194-bdb1-1e7b79dc00f3\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:17:50 crc kubenswrapper[5052]: I1203 12:17:50.875027 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/99a08445-670e-4194-bdb1-1e7b79dc00f3-scripts\") pod \"cinder-api-0\" (UID: \"99a08445-670e-4194-bdb1-1e7b79dc00f3\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:17:50 crc kubenswrapper[5052]: I1203 12:17:50.875056 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/99a08445-670e-4194-bdb1-1e7b79dc00f3-etc-machine-id\") pod \"cinder-api-0\" (UID: \"99a08445-670e-4194-bdb1-1e7b79dc00f3\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:17:50 crc kubenswrapper[5052]: I1203 12:17:50.875084 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-cs5ln\" (UniqueName: \"kubernetes.io/projected/99a08445-670e-4194-bdb1-1e7b79dc00f3-kube-api-access-cs5ln\") pod \"cinder-api-0\" (UID: \"99a08445-670e-4194-bdb1-1e7b79dc00f3\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:17:50 crc kubenswrapper[5052]: I1203 12:17:50.875166 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/99a08445-670e-4194-bdb1-1e7b79dc00f3-etc-machine-id\") pod \"cinder-api-0\" (UID: \"99a08445-670e-4194-bdb1-1e7b79dc00f3\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:17:50 crc kubenswrapper[5052]: I1203 12:17:50.875490 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/99a08445-670e-4194-bdb1-1e7b79dc00f3-logs\") pod \"cinder-api-0\" (UID: \"99a08445-670e-4194-bdb1-1e7b79dc00f3\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:17:50 crc kubenswrapper[5052]: I1203 12:17:50.880710 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/99a08445-670e-4194-bdb1-1e7b79dc00f3-combined-ca-bundle\") pod \"cinder-api-0\" (UID: \"99a08445-670e-4194-bdb1-1e7b79dc00f3\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:17:50 crc kubenswrapper[5052]: I1203 12:17:50.882403 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/99a08445-670e-4194-bdb1-1e7b79dc00f3-config-data\") pod \"cinder-api-0\" (UID: \"99a08445-670e-4194-bdb1-1e7b79dc00f3\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:17:50 crc kubenswrapper[5052]: I1203 12:17:50.883207 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/99a08445-670e-4194-bdb1-1e7b79dc00f3-scripts\") pod \"cinder-api-0\" (UID: \"99a08445-670e-4194-bdb1-1e7b79dc00f3\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:17:50 crc kubenswrapper[5052]: I1203 12:17:50.884171 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/99a08445-670e-4194-bdb1-1e7b79dc00f3-internal-tls-certs\") pod \"cinder-api-0\" (UID: \"99a08445-670e-4194-bdb1-1e7b79dc00f3\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:17:50 crc kubenswrapper[5052]: I1203 12:17:50.884267 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/99a08445-670e-4194-bdb1-1e7b79dc00f3-public-tls-certs\") pod \"cinder-api-0\" (UID: \"99a08445-670e-4194-bdb1-1e7b79dc00f3\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:17:50 crc kubenswrapper[5052]: I1203 12:17:50.888126 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/99a08445-670e-4194-bdb1-1e7b79dc00f3-config-data-custom\") pod \"cinder-api-0\" (UID: \"99a08445-670e-4194-bdb1-1e7b79dc00f3\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:17:50 crc kubenswrapper[5052]: I1203 12:17:50.893454 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-cs5ln\" (UniqueName: \"kubernetes.io/projected/99a08445-670e-4194-bdb1-1e7b79dc00f3-kube-api-access-cs5ln\") pod \"cinder-api-0\" (UID: \"99a08445-670e-4194-bdb1-1e7b79dc00f3\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:17:51 crc kubenswrapper[5052]: I1203 12:17:51.034246 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:17:51 crc kubenswrapper[5052]: I1203 12:17:51.402939 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:17:51 crc kubenswrapper[5052]: I1203 12:17:51.403357 5052 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Dec 03 12:17:51 crc kubenswrapper[5052]: I1203 12:17:51.502404 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 12:17:51 crc kubenswrapper[5052]: I1203 12:17:51.505082 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/barbican-api-547ff4f8ff-dpw9k" Dec 03 12:17:51 crc kubenswrapper[5052]: I1203 12:17:51.539709 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:17:51 crc kubenswrapper[5052]: I1203 12:17:51.650677 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/barbican-api-547ff4f8ff-dpw9k" Dec 03 12:17:51 crc kubenswrapper[5052]: I1203 12:17:51.707121 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="908e450d-211f-4b2f-8f21-332e6a18e0c1" path="/var/lib/kubelet/pods/908e450d-211f-4b2f-8f21-332e6a18e0c1/volumes" Dec 03 12:17:52 crc kubenswrapper[5052]: I1203 12:17:52.346568 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"99a08445-670e-4194-bdb1-1e7b79dc00f3","Type":"ContainerStarted","Data":"c5c1aee8f74d3c51ad667b122373af80c11a540d038db6ff4ad7b44a5a8c4f6e"} Dec 03 12:17:52 crc kubenswrapper[5052]: I1203 12:17:52.346904 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"99a08445-670e-4194-bdb1-1e7b79dc00f3","Type":"ContainerStarted","Data":"e21752e5b4419159e2a808d3724582e2d6ba1c284136034ea5acf8e21a04d757"} Dec 03 12:17:53 crc kubenswrapper[5052]: I1203 12:17:53.359433 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"99a08445-670e-4194-bdb1-1e7b79dc00f3","Type":"ContainerStarted","Data":"e09e55a6c45ce4ed5ad18b47c220ac1df2d781056bacefff348ec37669d05115"} Dec 03 12:17:53 crc kubenswrapper[5052]: I1203 12:17:53.360110 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:17:53 crc kubenswrapper[5052]: I1203 12:17:53.387891 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/cinder-api-0" podStartSLOduration=3.38787018 podStartE2EDuration="3.38787018s" podCreationTimestamp="2025-12-03 12:17:50 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:17:53.382940379 +0000 UTC m=+3463.971182153" watchObservedRunningTime="2025-12-03 12:17:53.38787018 +0000 UTC m=+3463.976111944" Dec 03 12:17:54 crc kubenswrapper[5052]: I1203 12:17:54.115648 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/barbican-api-64fc65b95f-tl6vk" Dec 03 12:17:54 crc kubenswrapper[5052]: I1203 12:17:54.372127 5052 generic.go:334] "Generic (PLEG): container finished" podID="cb69fca5-361d-4273-bee8-27c54648309c" containerID="0e69c8cda10283dde61afaf75a7b595710a8bdadd8fdf004e7e5e8d3fa479050" exitCode=0 Dec 03 12:17:54 crc kubenswrapper[5052]: I1203 12:17:54.372240 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-db-sync-kgtt9" event={"ID":"cb69fca5-361d-4273-bee8-27c54648309c","Type":"ContainerDied","Data":"0e69c8cda10283dde61afaf75a7b595710a8bdadd8fdf004e7e5e8d3fa479050"} Dec 03 12:17:54 crc kubenswrapper[5052]: I1203 12:17:54.410572 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/barbican-api-64fc65b95f-tl6vk" Dec 03 12:17:54 crc kubenswrapper[5052]: I1203 12:17:54.478243 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-api-547ff4f8ff-dpw9k"] Dec 03 12:17:54 crc kubenswrapper[5052]: I1203 12:17:54.478522 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/barbican-api-547ff4f8ff-dpw9k" podUID="888ed75c-2d05-4199-b51a-123ac3e44e31" containerName="barbican-api-log" containerID="cri-o://3ac703aa86fa7f1d905e3dee4314f3f9a9da4082370ed127c2c324867f9c0e59" gracePeriod=30 Dec 03 12:17:54 crc kubenswrapper[5052]: I1203 12:17:54.479030 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/barbican-api-547ff4f8ff-dpw9k" podUID="888ed75c-2d05-4199-b51a-123ac3e44e31" containerName="barbican-api" containerID="cri-o://cf5bcf38e7380e5571b79442bbf2eb53e25771afb2633483fb9da0003a4bb84d" gracePeriod=30 Dec 03 12:17:55 crc kubenswrapper[5052]: I1203 12:17:55.387679 5052 generic.go:334] "Generic (PLEG): container finished" podID="888ed75c-2d05-4199-b51a-123ac3e44e31" containerID="3ac703aa86fa7f1d905e3dee4314f3f9a9da4082370ed127c2c324867f9c0e59" exitCode=143 Dec 03 12:17:55 crc kubenswrapper[5052]: I1203 12:17:55.387942 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-547ff4f8ff-dpw9k" event={"ID":"888ed75c-2d05-4199-b51a-123ac3e44e31","Type":"ContainerDied","Data":"3ac703aa86fa7f1d905e3dee4314f3f9a9da4082370ed127c2c324867f9c0e59"} Dec 03 12:17:55 crc kubenswrapper[5052]: I1203 12:17:55.799859 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-db-sync-kgtt9" Dec 03 12:17:55 crc kubenswrapper[5052]: I1203 12:17:55.922979 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-22pgz\" (UniqueName: \"kubernetes.io/projected/cb69fca5-361d-4273-bee8-27c54648309c-kube-api-access-22pgz\") pod \"cb69fca5-361d-4273-bee8-27c54648309c\" (UID: \"cb69fca5-361d-4273-bee8-27c54648309c\") " Dec 03 12:17:55 crc kubenswrapper[5052]: I1203 12:17:55.923645 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/cb69fca5-361d-4273-bee8-27c54648309c-combined-ca-bundle\") pod \"cb69fca5-361d-4273-bee8-27c54648309c\" (UID: \"cb69fca5-361d-4273-bee8-27c54648309c\") " Dec 03 12:17:55 crc kubenswrapper[5052]: I1203 12:17:55.923720 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/cb69fca5-361d-4273-bee8-27c54648309c-config\") pod \"cb69fca5-361d-4273-bee8-27c54648309c\" (UID: \"cb69fca5-361d-4273-bee8-27c54648309c\") " Dec 03 12:17:55 crc kubenswrapper[5052]: I1203 12:17:55.931492 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cb69fca5-361d-4273-bee8-27c54648309c-kube-api-access-22pgz" (OuterVolumeSpecName: "kube-api-access-22pgz") pod "cb69fca5-361d-4273-bee8-27c54648309c" (UID: "cb69fca5-361d-4273-bee8-27c54648309c"). InnerVolumeSpecName "kube-api-access-22pgz". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:17:55 crc kubenswrapper[5052]: I1203 12:17:55.961839 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cb69fca5-361d-4273-bee8-27c54648309c-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "cb69fca5-361d-4273-bee8-27c54648309c" (UID: "cb69fca5-361d-4273-bee8-27c54648309c"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:17:55 crc kubenswrapper[5052]: I1203 12:17:55.975398 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cb69fca5-361d-4273-bee8-27c54648309c-config" (OuterVolumeSpecName: "config") pod "cb69fca5-361d-4273-bee8-27c54648309c" (UID: "cb69fca5-361d-4273-bee8-27c54648309c"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:17:55 crc kubenswrapper[5052]: I1203 12:17:55.998722 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:17:56 crc kubenswrapper[5052]: I1203 12:17:56.026187 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-22pgz\" (UniqueName: \"kubernetes.io/projected/cb69fca5-361d-4273-bee8-27c54648309c-kube-api-access-22pgz\") on node \"crc\" DevicePath \"\"" Dec 03 12:17:56 crc kubenswrapper[5052]: I1203 12:17:56.026218 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/cb69fca5-361d-4273-bee8-27c54648309c-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:17:56 crc kubenswrapper[5052]: I1203 12:17:56.026230 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/cb69fca5-361d-4273-bee8-27c54648309c-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:17:56 crc kubenswrapper[5052]: I1203 12:17:56.046225 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 12:17:56 crc kubenswrapper[5052]: I1203 12:17:56.403837 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/cinder-scheduler-0" podUID="321c0d14-7d66-4b3a-8989-f6a161421d2e" containerName="cinder-scheduler" containerID="cri-o://19aa0a6013cd529547763657024a5ae0ee17db18c9cc49526b6066bb7c447d1b" gracePeriod=30 Dec 03 12:17:56 crc kubenswrapper[5052]: I1203 12:17:56.403914 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-db-sync-kgtt9" Dec 03 12:17:56 crc kubenswrapper[5052]: I1203 12:17:56.403922 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/cinder-scheduler-0" podUID="321c0d14-7d66-4b3a-8989-f6a161421d2e" containerName="probe" containerID="cri-o://b800ece057aa8281071c80450e628f23c676cb4fc4228db9b5d9cd509984d054" gracePeriod=30 Dec 03 12:17:56 crc kubenswrapper[5052]: I1203 12:17:56.403835 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-db-sync-kgtt9" event={"ID":"cb69fca5-361d-4273-bee8-27c54648309c","Type":"ContainerDied","Data":"37723aa45e613d2653462e3c8d41df055a7a9bbe5732538efa703b6871bd789e"} Dec 03 12:17:56 crc kubenswrapper[5052]: I1203 12:17:56.404378 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="37723aa45e613d2653462e3c8d41df055a7a9bbe5732538efa703b6871bd789e" Dec 03 12:17:56 crc kubenswrapper[5052]: I1203 12:17:56.717599 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/neutron-54c879c996-kc674"] Dec 03 12:17:56 crc kubenswrapper[5052]: E1203 12:17:56.718022 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="cb69fca5-361d-4273-bee8-27c54648309c" containerName="neutron-db-sync" Dec 03 12:17:56 crc kubenswrapper[5052]: I1203 12:17:56.718063 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="cb69fca5-361d-4273-bee8-27c54648309c" containerName="neutron-db-sync" Dec 03 12:17:56 crc kubenswrapper[5052]: I1203 12:17:56.718322 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="cb69fca5-361d-4273-bee8-27c54648309c" containerName="neutron-db-sync" Dec 03 12:17:56 crc kubenswrapper[5052]: I1203 12:17:56.719598 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-54c879c996-kc674" Dec 03 12:17:56 crc kubenswrapper[5052]: I1203 12:17:56.725038 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"neutron-httpd-config" Dec 03 12:17:56 crc kubenswrapper[5052]: I1203 12:17:56.725227 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-neutron-ovndbs" Dec 03 12:17:56 crc kubenswrapper[5052]: I1203 12:17:56.725265 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"neutron-neutron-dockercfg-vpfh5" Dec 03 12:17:56 crc kubenswrapper[5052]: I1203 12:17:56.726361 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"neutron-config" Dec 03 12:17:56 crc kubenswrapper[5052]: I1203 12:17:56.729862 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-54c879c996-kc674"] Dec 03 12:17:56 crc kubenswrapper[5052]: I1203 12:17:56.840780 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d19ca0f1-4df8-41e7-bab7-dd6d03050486-combined-ca-bundle\") pod \"neutron-54c879c996-kc674\" (UID: \"d19ca0f1-4df8-41e7-bab7-dd6d03050486\") " pod="openstack-kuttl-tests/neutron-54c879c996-kc674" Dec 03 12:17:56 crc kubenswrapper[5052]: I1203 12:17:56.840835 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mkqtz\" (UniqueName: \"kubernetes.io/projected/d19ca0f1-4df8-41e7-bab7-dd6d03050486-kube-api-access-mkqtz\") pod \"neutron-54c879c996-kc674\" (UID: \"d19ca0f1-4df8-41e7-bab7-dd6d03050486\") " pod="openstack-kuttl-tests/neutron-54c879c996-kc674" Dec 03 12:17:56 crc kubenswrapper[5052]: I1203 12:17:56.840863 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/d19ca0f1-4df8-41e7-bab7-dd6d03050486-httpd-config\") pod \"neutron-54c879c996-kc674\" (UID: \"d19ca0f1-4df8-41e7-bab7-dd6d03050486\") " pod="openstack-kuttl-tests/neutron-54c879c996-kc674" Dec 03 12:17:56 crc kubenswrapper[5052]: I1203 12:17:56.840888 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/d19ca0f1-4df8-41e7-bab7-dd6d03050486-config\") pod \"neutron-54c879c996-kc674\" (UID: \"d19ca0f1-4df8-41e7-bab7-dd6d03050486\") " pod="openstack-kuttl-tests/neutron-54c879c996-kc674" Dec 03 12:17:56 crc kubenswrapper[5052]: I1203 12:17:56.841242 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/d19ca0f1-4df8-41e7-bab7-dd6d03050486-ovndb-tls-certs\") pod \"neutron-54c879c996-kc674\" (UID: \"d19ca0f1-4df8-41e7-bab7-dd6d03050486\") " pod="openstack-kuttl-tests/neutron-54c879c996-kc674" Dec 03 12:17:56 crc kubenswrapper[5052]: I1203 12:17:56.942980 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d19ca0f1-4df8-41e7-bab7-dd6d03050486-combined-ca-bundle\") pod \"neutron-54c879c996-kc674\" (UID: \"d19ca0f1-4df8-41e7-bab7-dd6d03050486\") " pod="openstack-kuttl-tests/neutron-54c879c996-kc674" Dec 03 12:17:56 crc kubenswrapper[5052]: I1203 12:17:56.943031 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-mkqtz\" (UniqueName: \"kubernetes.io/projected/d19ca0f1-4df8-41e7-bab7-dd6d03050486-kube-api-access-mkqtz\") pod \"neutron-54c879c996-kc674\" (UID: \"d19ca0f1-4df8-41e7-bab7-dd6d03050486\") " pod="openstack-kuttl-tests/neutron-54c879c996-kc674" Dec 03 12:17:56 crc kubenswrapper[5052]: I1203 12:17:56.943060 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/d19ca0f1-4df8-41e7-bab7-dd6d03050486-httpd-config\") pod \"neutron-54c879c996-kc674\" (UID: \"d19ca0f1-4df8-41e7-bab7-dd6d03050486\") " pod="openstack-kuttl-tests/neutron-54c879c996-kc674" Dec 03 12:17:56 crc kubenswrapper[5052]: I1203 12:17:56.943087 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/d19ca0f1-4df8-41e7-bab7-dd6d03050486-config\") pod \"neutron-54c879c996-kc674\" (UID: \"d19ca0f1-4df8-41e7-bab7-dd6d03050486\") " pod="openstack-kuttl-tests/neutron-54c879c996-kc674" Dec 03 12:17:56 crc kubenswrapper[5052]: I1203 12:17:56.943232 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/d19ca0f1-4df8-41e7-bab7-dd6d03050486-ovndb-tls-certs\") pod \"neutron-54c879c996-kc674\" (UID: \"d19ca0f1-4df8-41e7-bab7-dd6d03050486\") " pod="openstack-kuttl-tests/neutron-54c879c996-kc674" Dec 03 12:17:56 crc kubenswrapper[5052]: I1203 12:17:56.948388 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/d19ca0f1-4df8-41e7-bab7-dd6d03050486-httpd-config\") pod \"neutron-54c879c996-kc674\" (UID: \"d19ca0f1-4df8-41e7-bab7-dd6d03050486\") " pod="openstack-kuttl-tests/neutron-54c879c996-kc674" Dec 03 12:17:56 crc kubenswrapper[5052]: I1203 12:17:56.948405 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d19ca0f1-4df8-41e7-bab7-dd6d03050486-combined-ca-bundle\") pod \"neutron-54c879c996-kc674\" (UID: \"d19ca0f1-4df8-41e7-bab7-dd6d03050486\") " pod="openstack-kuttl-tests/neutron-54c879c996-kc674" Dec 03 12:17:56 crc kubenswrapper[5052]: I1203 12:17:56.950170 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/d19ca0f1-4df8-41e7-bab7-dd6d03050486-ovndb-tls-certs\") pod \"neutron-54c879c996-kc674\" (UID: \"d19ca0f1-4df8-41e7-bab7-dd6d03050486\") " pod="openstack-kuttl-tests/neutron-54c879c996-kc674" Dec 03 12:17:56 crc kubenswrapper[5052]: I1203 12:17:56.950553 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/d19ca0f1-4df8-41e7-bab7-dd6d03050486-config\") pod \"neutron-54c879c996-kc674\" (UID: \"d19ca0f1-4df8-41e7-bab7-dd6d03050486\") " pod="openstack-kuttl-tests/neutron-54c879c996-kc674" Dec 03 12:17:56 crc kubenswrapper[5052]: I1203 12:17:56.961131 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-mkqtz\" (UniqueName: \"kubernetes.io/projected/d19ca0f1-4df8-41e7-bab7-dd6d03050486-kube-api-access-mkqtz\") pod \"neutron-54c879c996-kc674\" (UID: \"d19ca0f1-4df8-41e7-bab7-dd6d03050486\") " pod="openstack-kuttl-tests/neutron-54c879c996-kc674" Dec 03 12:17:57 crc kubenswrapper[5052]: I1203 12:17:57.040479 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-54c879c996-kc674" Dec 03 12:17:57 crc kubenswrapper[5052]: I1203 12:17:57.414055 5052 generic.go:334] "Generic (PLEG): container finished" podID="321c0d14-7d66-4b3a-8989-f6a161421d2e" containerID="b800ece057aa8281071c80450e628f23c676cb4fc4228db9b5d9cd509984d054" exitCode=0 Dec 03 12:17:57 crc kubenswrapper[5052]: I1203 12:17:57.414144 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"321c0d14-7d66-4b3a-8989-f6a161421d2e","Type":"ContainerDied","Data":"b800ece057aa8281071c80450e628f23c676cb4fc4228db9b5d9cd509984d054"} Dec 03 12:17:57 crc kubenswrapper[5052]: I1203 12:17:57.545121 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-54c879c996-kc674"] Dec 03 12:17:57 crc kubenswrapper[5052]: W1203 12:17:57.545372 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd19ca0f1_4df8_41e7_bab7_dd6d03050486.slice/crio-4a5654a157a731272c7f243ca48712c7c12b204dd4c16ee08f96ba5d0490c985 WatchSource:0}: Error finding container 4a5654a157a731272c7f243ca48712c7c12b204dd4c16ee08f96ba5d0490c985: Status 404 returned error can't find the container with id 4a5654a157a731272c7f243ca48712c7c12b204dd4c16ee08f96ba5d0490c985 Dec 03 12:17:57 crc kubenswrapper[5052]: I1203 12:17:57.619919 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/barbican-api-547ff4f8ff-dpw9k" podUID="888ed75c-2d05-4199-b51a-123ac3e44e31" containerName="barbican-api" probeResult="failure" output="Get \"http://10.217.1.101:9311/healthcheck\": read tcp 10.217.0.2:57448->10.217.1.101:9311: read: connection reset by peer" Dec 03 12:17:57 crc kubenswrapper[5052]: I1203 12:17:57.620079 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/barbican-api-547ff4f8ff-dpw9k" podUID="888ed75c-2d05-4199-b51a-123ac3e44e31" containerName="barbican-api-log" probeResult="failure" output="Get \"http://10.217.1.101:9311/healthcheck\": read tcp 10.217.0.2:57456->10.217.1.101:9311: read: connection reset by peer" Dec 03 12:17:58 crc kubenswrapper[5052]: I1203 12:17:58.217307 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-api-547ff4f8ff-dpw9k" Dec 03 12:17:58 crc kubenswrapper[5052]: I1203 12:17:58.376755 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9k4zc\" (UniqueName: \"kubernetes.io/projected/888ed75c-2d05-4199-b51a-123ac3e44e31-kube-api-access-9k4zc\") pod \"888ed75c-2d05-4199-b51a-123ac3e44e31\" (UID: \"888ed75c-2d05-4199-b51a-123ac3e44e31\") " Dec 03 12:17:58 crc kubenswrapper[5052]: I1203 12:17:58.376843 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/888ed75c-2d05-4199-b51a-123ac3e44e31-config-data-custom\") pod \"888ed75c-2d05-4199-b51a-123ac3e44e31\" (UID: \"888ed75c-2d05-4199-b51a-123ac3e44e31\") " Dec 03 12:17:58 crc kubenswrapper[5052]: I1203 12:17:58.377124 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/888ed75c-2d05-4199-b51a-123ac3e44e31-config-data\") pod \"888ed75c-2d05-4199-b51a-123ac3e44e31\" (UID: \"888ed75c-2d05-4199-b51a-123ac3e44e31\") " Dec 03 12:17:58 crc kubenswrapper[5052]: I1203 12:17:58.377168 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/888ed75c-2d05-4199-b51a-123ac3e44e31-logs\") pod \"888ed75c-2d05-4199-b51a-123ac3e44e31\" (UID: \"888ed75c-2d05-4199-b51a-123ac3e44e31\") " Dec 03 12:17:58 crc kubenswrapper[5052]: I1203 12:17:58.377242 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/888ed75c-2d05-4199-b51a-123ac3e44e31-combined-ca-bundle\") pod \"888ed75c-2d05-4199-b51a-123ac3e44e31\" (UID: \"888ed75c-2d05-4199-b51a-123ac3e44e31\") " Dec 03 12:17:58 crc kubenswrapper[5052]: I1203 12:17:58.379150 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/888ed75c-2d05-4199-b51a-123ac3e44e31-logs" (OuterVolumeSpecName: "logs") pod "888ed75c-2d05-4199-b51a-123ac3e44e31" (UID: "888ed75c-2d05-4199-b51a-123ac3e44e31"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:17:58 crc kubenswrapper[5052]: I1203 12:17:58.385914 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/888ed75c-2d05-4199-b51a-123ac3e44e31-config-data-custom" (OuterVolumeSpecName: "config-data-custom") pod "888ed75c-2d05-4199-b51a-123ac3e44e31" (UID: "888ed75c-2d05-4199-b51a-123ac3e44e31"). InnerVolumeSpecName "config-data-custom". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:17:58 crc kubenswrapper[5052]: I1203 12:17:58.386227 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/888ed75c-2d05-4199-b51a-123ac3e44e31-kube-api-access-9k4zc" (OuterVolumeSpecName: "kube-api-access-9k4zc") pod "888ed75c-2d05-4199-b51a-123ac3e44e31" (UID: "888ed75c-2d05-4199-b51a-123ac3e44e31"). InnerVolumeSpecName "kube-api-access-9k4zc". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:17:58 crc kubenswrapper[5052]: I1203 12:17:58.413025 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/888ed75c-2d05-4199-b51a-123ac3e44e31-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "888ed75c-2d05-4199-b51a-123ac3e44e31" (UID: "888ed75c-2d05-4199-b51a-123ac3e44e31"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:17:58 crc kubenswrapper[5052]: I1203 12:17:58.426736 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-54c879c996-kc674" event={"ID":"d19ca0f1-4df8-41e7-bab7-dd6d03050486","Type":"ContainerStarted","Data":"36bd66e285f10587a82a74ea7546e511ef627d378d46a797d9f30d0c1d508790"} Dec 03 12:17:58 crc kubenswrapper[5052]: I1203 12:17:58.426818 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-54c879c996-kc674" event={"ID":"d19ca0f1-4df8-41e7-bab7-dd6d03050486","Type":"ContainerStarted","Data":"10244003ad298cfe03dc6285b75d835bfc6b5409e5af2803e4f440d356990d96"} Dec 03 12:17:58 crc kubenswrapper[5052]: I1203 12:17:58.426832 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-54c879c996-kc674" event={"ID":"d19ca0f1-4df8-41e7-bab7-dd6d03050486","Type":"ContainerStarted","Data":"4a5654a157a731272c7f243ca48712c7c12b204dd4c16ee08f96ba5d0490c985"} Dec 03 12:17:58 crc kubenswrapper[5052]: I1203 12:17:58.428240 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/neutron-54c879c996-kc674" Dec 03 12:17:58 crc kubenswrapper[5052]: I1203 12:17:58.431551 5052 generic.go:334] "Generic (PLEG): container finished" podID="888ed75c-2d05-4199-b51a-123ac3e44e31" containerID="cf5bcf38e7380e5571b79442bbf2eb53e25771afb2633483fb9da0003a4bb84d" exitCode=0 Dec 03 12:17:58 crc kubenswrapper[5052]: I1203 12:17:58.431740 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-547ff4f8ff-dpw9k" event={"ID":"888ed75c-2d05-4199-b51a-123ac3e44e31","Type":"ContainerDied","Data":"cf5bcf38e7380e5571b79442bbf2eb53e25771afb2633483fb9da0003a4bb84d"} Dec 03 12:17:58 crc kubenswrapper[5052]: I1203 12:17:58.432532 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-547ff4f8ff-dpw9k" event={"ID":"888ed75c-2d05-4199-b51a-123ac3e44e31","Type":"ContainerDied","Data":"2aa0092570f6a58d45f8600287cfbd593ba372e185ca4e33fc4dace9ebb34b31"} Dec 03 12:17:58 crc kubenswrapper[5052]: I1203 12:17:58.431834 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-api-547ff4f8ff-dpw9k" Dec 03 12:17:58 crc kubenswrapper[5052]: I1203 12:17:58.432617 5052 scope.go:117] "RemoveContainer" containerID="cf5bcf38e7380e5571b79442bbf2eb53e25771afb2633483fb9da0003a4bb84d" Dec 03 12:17:58 crc kubenswrapper[5052]: I1203 12:17:58.457133 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/888ed75c-2d05-4199-b51a-123ac3e44e31-config-data" (OuterVolumeSpecName: "config-data") pod "888ed75c-2d05-4199-b51a-123ac3e44e31" (UID: "888ed75c-2d05-4199-b51a-123ac3e44e31"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:17:58 crc kubenswrapper[5052]: I1203 12:17:58.476615 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/neutron-54c879c996-kc674" podStartSLOduration=2.476595042 podStartE2EDuration="2.476595042s" podCreationTimestamp="2025-12-03 12:17:56 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:17:58.449578701 +0000 UTC m=+3469.037820475" watchObservedRunningTime="2025-12-03 12:17:58.476595042 +0000 UTC m=+3469.064836806" Dec 03 12:17:58 crc kubenswrapper[5052]: I1203 12:17:58.479547 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/888ed75c-2d05-4199-b51a-123ac3e44e31-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:17:58 crc kubenswrapper[5052]: I1203 12:17:58.479694 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/888ed75c-2d05-4199-b51a-123ac3e44e31-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:17:58 crc kubenswrapper[5052]: I1203 12:17:58.479756 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/888ed75c-2d05-4199-b51a-123ac3e44e31-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:17:58 crc kubenswrapper[5052]: I1203 12:17:58.479828 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-9k4zc\" (UniqueName: \"kubernetes.io/projected/888ed75c-2d05-4199-b51a-123ac3e44e31-kube-api-access-9k4zc\") on node \"crc\" DevicePath \"\"" Dec 03 12:17:58 crc kubenswrapper[5052]: I1203 12:17:58.479881 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/888ed75c-2d05-4199-b51a-123ac3e44e31-config-data-custom\") on node \"crc\" DevicePath \"\"" Dec 03 12:17:58 crc kubenswrapper[5052]: I1203 12:17:58.491840 5052 scope.go:117] "RemoveContainer" containerID="3ac703aa86fa7f1d905e3dee4314f3f9a9da4082370ed127c2c324867f9c0e59" Dec 03 12:17:58 crc kubenswrapper[5052]: I1203 12:17:58.511019 5052 scope.go:117] "RemoveContainer" containerID="cf5bcf38e7380e5571b79442bbf2eb53e25771afb2633483fb9da0003a4bb84d" Dec 03 12:17:58 crc kubenswrapper[5052]: E1203 12:17:58.511857 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cf5bcf38e7380e5571b79442bbf2eb53e25771afb2633483fb9da0003a4bb84d\": container with ID starting with cf5bcf38e7380e5571b79442bbf2eb53e25771afb2633483fb9da0003a4bb84d not found: ID does not exist" containerID="cf5bcf38e7380e5571b79442bbf2eb53e25771afb2633483fb9da0003a4bb84d" Dec 03 12:17:58 crc kubenswrapper[5052]: I1203 12:17:58.511906 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cf5bcf38e7380e5571b79442bbf2eb53e25771afb2633483fb9da0003a4bb84d"} err="failed to get container status \"cf5bcf38e7380e5571b79442bbf2eb53e25771afb2633483fb9da0003a4bb84d\": rpc error: code = NotFound desc = could not find container \"cf5bcf38e7380e5571b79442bbf2eb53e25771afb2633483fb9da0003a4bb84d\": container with ID starting with cf5bcf38e7380e5571b79442bbf2eb53e25771afb2633483fb9da0003a4bb84d not found: ID does not exist" Dec 03 12:17:58 crc kubenswrapper[5052]: I1203 12:17:58.511935 5052 scope.go:117] "RemoveContainer" containerID="3ac703aa86fa7f1d905e3dee4314f3f9a9da4082370ed127c2c324867f9c0e59" Dec 03 12:17:58 crc kubenswrapper[5052]: E1203 12:17:58.512423 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3ac703aa86fa7f1d905e3dee4314f3f9a9da4082370ed127c2c324867f9c0e59\": container with ID starting with 3ac703aa86fa7f1d905e3dee4314f3f9a9da4082370ed127c2c324867f9c0e59 not found: ID does not exist" containerID="3ac703aa86fa7f1d905e3dee4314f3f9a9da4082370ed127c2c324867f9c0e59" Dec 03 12:17:58 crc kubenswrapper[5052]: I1203 12:17:58.512555 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3ac703aa86fa7f1d905e3dee4314f3f9a9da4082370ed127c2c324867f9c0e59"} err="failed to get container status \"3ac703aa86fa7f1d905e3dee4314f3f9a9da4082370ed127c2c324867f9c0e59\": rpc error: code = NotFound desc = could not find container \"3ac703aa86fa7f1d905e3dee4314f3f9a9da4082370ed127c2c324867f9c0e59\": container with ID starting with 3ac703aa86fa7f1d905e3dee4314f3f9a9da4082370ed127c2c324867f9c0e59 not found: ID does not exist" Dec 03 12:17:58 crc kubenswrapper[5052]: I1203 12:17:58.780094 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-api-547ff4f8ff-dpw9k"] Dec 03 12:17:58 crc kubenswrapper[5052]: I1203 12:17:58.791009 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/barbican-api-547ff4f8ff-dpw9k"] Dec 03 12:17:58 crc kubenswrapper[5052]: I1203 12:17:58.935116 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/neutron-54579c6749-xzrwm"] Dec 03 12:17:58 crc kubenswrapper[5052]: E1203 12:17:58.935579 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="888ed75c-2d05-4199-b51a-123ac3e44e31" containerName="barbican-api" Dec 03 12:17:58 crc kubenswrapper[5052]: I1203 12:17:58.935598 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="888ed75c-2d05-4199-b51a-123ac3e44e31" containerName="barbican-api" Dec 03 12:17:58 crc kubenswrapper[5052]: E1203 12:17:58.935619 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="888ed75c-2d05-4199-b51a-123ac3e44e31" containerName="barbican-api-log" Dec 03 12:17:58 crc kubenswrapper[5052]: I1203 12:17:58.935626 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="888ed75c-2d05-4199-b51a-123ac3e44e31" containerName="barbican-api-log" Dec 03 12:17:58 crc kubenswrapper[5052]: I1203 12:17:58.935797 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="888ed75c-2d05-4199-b51a-123ac3e44e31" containerName="barbican-api-log" Dec 03 12:17:58 crc kubenswrapper[5052]: I1203 12:17:58.935818 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="888ed75c-2d05-4199-b51a-123ac3e44e31" containerName="barbican-api" Dec 03 12:17:58 crc kubenswrapper[5052]: I1203 12:17:58.936888 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-54579c6749-xzrwm" Dec 03 12:17:58 crc kubenswrapper[5052]: I1203 12:17:58.948958 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-54579c6749-xzrwm"] Dec 03 12:17:58 crc kubenswrapper[5052]: I1203 12:17:58.949943 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-neutron-internal-svc" Dec 03 12:17:58 crc kubenswrapper[5052]: I1203 12:17:58.950412 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-neutron-public-svc" Dec 03 12:17:59 crc kubenswrapper[5052]: I1203 12:17:59.090519 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/d1660f09-947f-433a-973f-0cd19d97331f-httpd-config\") pod \"neutron-54579c6749-xzrwm\" (UID: \"d1660f09-947f-433a-973f-0cd19d97331f\") " pod="openstack-kuttl-tests/neutron-54579c6749-xzrwm" Dec 03 12:17:59 crc kubenswrapper[5052]: I1203 12:17:59.090574 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/d1660f09-947f-433a-973f-0cd19d97331f-public-tls-certs\") pod \"neutron-54579c6749-xzrwm\" (UID: \"d1660f09-947f-433a-973f-0cd19d97331f\") " pod="openstack-kuttl-tests/neutron-54579c6749-xzrwm" Dec 03 12:17:59 crc kubenswrapper[5052]: I1203 12:17:59.090598 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/d1660f09-947f-433a-973f-0cd19d97331f-ovndb-tls-certs\") pod \"neutron-54579c6749-xzrwm\" (UID: \"d1660f09-947f-433a-973f-0cd19d97331f\") " pod="openstack-kuttl-tests/neutron-54579c6749-xzrwm" Dec 03 12:17:59 crc kubenswrapper[5052]: I1203 12:17:59.090626 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b6wjl\" (UniqueName: \"kubernetes.io/projected/d1660f09-947f-433a-973f-0cd19d97331f-kube-api-access-b6wjl\") pod \"neutron-54579c6749-xzrwm\" (UID: \"d1660f09-947f-433a-973f-0cd19d97331f\") " pod="openstack-kuttl-tests/neutron-54579c6749-xzrwm" Dec 03 12:17:59 crc kubenswrapper[5052]: I1203 12:17:59.090794 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d1660f09-947f-433a-973f-0cd19d97331f-combined-ca-bundle\") pod \"neutron-54579c6749-xzrwm\" (UID: \"d1660f09-947f-433a-973f-0cd19d97331f\") " pod="openstack-kuttl-tests/neutron-54579c6749-xzrwm" Dec 03 12:17:59 crc kubenswrapper[5052]: I1203 12:17:59.090830 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/d1660f09-947f-433a-973f-0cd19d97331f-config\") pod \"neutron-54579c6749-xzrwm\" (UID: \"d1660f09-947f-433a-973f-0cd19d97331f\") " pod="openstack-kuttl-tests/neutron-54579c6749-xzrwm" Dec 03 12:17:59 crc kubenswrapper[5052]: I1203 12:17:59.090848 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/d1660f09-947f-433a-973f-0cd19d97331f-internal-tls-certs\") pod \"neutron-54579c6749-xzrwm\" (UID: \"d1660f09-947f-433a-973f-0cd19d97331f\") " pod="openstack-kuttl-tests/neutron-54579c6749-xzrwm" Dec 03 12:17:59 crc kubenswrapper[5052]: I1203 12:17:59.192315 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/d1660f09-947f-433a-973f-0cd19d97331f-ovndb-tls-certs\") pod \"neutron-54579c6749-xzrwm\" (UID: \"d1660f09-947f-433a-973f-0cd19d97331f\") " pod="openstack-kuttl-tests/neutron-54579c6749-xzrwm" Dec 03 12:17:59 crc kubenswrapper[5052]: I1203 12:17:59.192358 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-b6wjl\" (UniqueName: \"kubernetes.io/projected/d1660f09-947f-433a-973f-0cd19d97331f-kube-api-access-b6wjl\") pod \"neutron-54579c6749-xzrwm\" (UID: \"d1660f09-947f-433a-973f-0cd19d97331f\") " pod="openstack-kuttl-tests/neutron-54579c6749-xzrwm" Dec 03 12:17:59 crc kubenswrapper[5052]: I1203 12:17:59.192470 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d1660f09-947f-433a-973f-0cd19d97331f-combined-ca-bundle\") pod \"neutron-54579c6749-xzrwm\" (UID: \"d1660f09-947f-433a-973f-0cd19d97331f\") " pod="openstack-kuttl-tests/neutron-54579c6749-xzrwm" Dec 03 12:17:59 crc kubenswrapper[5052]: I1203 12:17:59.192504 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/d1660f09-947f-433a-973f-0cd19d97331f-config\") pod \"neutron-54579c6749-xzrwm\" (UID: \"d1660f09-947f-433a-973f-0cd19d97331f\") " pod="openstack-kuttl-tests/neutron-54579c6749-xzrwm" Dec 03 12:17:59 crc kubenswrapper[5052]: I1203 12:17:59.193109 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/d1660f09-947f-433a-973f-0cd19d97331f-internal-tls-certs\") pod \"neutron-54579c6749-xzrwm\" (UID: \"d1660f09-947f-433a-973f-0cd19d97331f\") " pod="openstack-kuttl-tests/neutron-54579c6749-xzrwm" Dec 03 12:17:59 crc kubenswrapper[5052]: I1203 12:17:59.193162 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/d1660f09-947f-433a-973f-0cd19d97331f-httpd-config\") pod \"neutron-54579c6749-xzrwm\" (UID: \"d1660f09-947f-433a-973f-0cd19d97331f\") " pod="openstack-kuttl-tests/neutron-54579c6749-xzrwm" Dec 03 12:17:59 crc kubenswrapper[5052]: I1203 12:17:59.193196 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/d1660f09-947f-433a-973f-0cd19d97331f-public-tls-certs\") pod \"neutron-54579c6749-xzrwm\" (UID: \"d1660f09-947f-433a-973f-0cd19d97331f\") " pod="openstack-kuttl-tests/neutron-54579c6749-xzrwm" Dec 03 12:17:59 crc kubenswrapper[5052]: I1203 12:17:59.196632 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/d1660f09-947f-433a-973f-0cd19d97331f-public-tls-certs\") pod \"neutron-54579c6749-xzrwm\" (UID: \"d1660f09-947f-433a-973f-0cd19d97331f\") " pod="openstack-kuttl-tests/neutron-54579c6749-xzrwm" Dec 03 12:17:59 crc kubenswrapper[5052]: I1203 12:17:59.196904 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d1660f09-947f-433a-973f-0cd19d97331f-combined-ca-bundle\") pod \"neutron-54579c6749-xzrwm\" (UID: \"d1660f09-947f-433a-973f-0cd19d97331f\") " pod="openstack-kuttl-tests/neutron-54579c6749-xzrwm" Dec 03 12:17:59 crc kubenswrapper[5052]: I1203 12:17:59.197037 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/d1660f09-947f-433a-973f-0cd19d97331f-ovndb-tls-certs\") pod \"neutron-54579c6749-xzrwm\" (UID: \"d1660f09-947f-433a-973f-0cd19d97331f\") " pod="openstack-kuttl-tests/neutron-54579c6749-xzrwm" Dec 03 12:17:59 crc kubenswrapper[5052]: I1203 12:17:59.197915 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/d1660f09-947f-433a-973f-0cd19d97331f-config\") pod \"neutron-54579c6749-xzrwm\" (UID: \"d1660f09-947f-433a-973f-0cd19d97331f\") " pod="openstack-kuttl-tests/neutron-54579c6749-xzrwm" Dec 03 12:17:59 crc kubenswrapper[5052]: I1203 12:17:59.199361 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/d1660f09-947f-433a-973f-0cd19d97331f-httpd-config\") pod \"neutron-54579c6749-xzrwm\" (UID: \"d1660f09-947f-433a-973f-0cd19d97331f\") " pod="openstack-kuttl-tests/neutron-54579c6749-xzrwm" Dec 03 12:17:59 crc kubenswrapper[5052]: I1203 12:17:59.200219 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/d1660f09-947f-433a-973f-0cd19d97331f-internal-tls-certs\") pod \"neutron-54579c6749-xzrwm\" (UID: \"d1660f09-947f-433a-973f-0cd19d97331f\") " pod="openstack-kuttl-tests/neutron-54579c6749-xzrwm" Dec 03 12:17:59 crc kubenswrapper[5052]: I1203 12:17:59.217579 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-b6wjl\" (UniqueName: \"kubernetes.io/projected/d1660f09-947f-433a-973f-0cd19d97331f-kube-api-access-b6wjl\") pod \"neutron-54579c6749-xzrwm\" (UID: \"d1660f09-947f-433a-973f-0cd19d97331f\") " pod="openstack-kuttl-tests/neutron-54579c6749-xzrwm" Dec 03 12:17:59 crc kubenswrapper[5052]: I1203 12:17:59.291993 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-54579c6749-xzrwm" Dec 03 12:17:59 crc kubenswrapper[5052]: I1203 12:17:59.683055 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="888ed75c-2d05-4199-b51a-123ac3e44e31" path="/var/lib/kubelet/pods/888ed75c-2d05-4199-b51a-123ac3e44e31/volumes" Dec 03 12:17:59 crc kubenswrapper[5052]: I1203 12:17:59.770128 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-54579c6749-xzrwm"] Dec 03 12:18:00 crc kubenswrapper[5052]: I1203 12:18:00.462972 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-54579c6749-xzrwm" event={"ID":"d1660f09-947f-433a-973f-0cd19d97331f","Type":"ContainerStarted","Data":"99d0e325b776b63e2cdf1bb10d165091f0981effadb9947661ddb77345f58080"} Dec 03 12:18:00 crc kubenswrapper[5052]: I1203 12:18:00.463358 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-54579c6749-xzrwm" event={"ID":"d1660f09-947f-433a-973f-0cd19d97331f","Type":"ContainerStarted","Data":"165f8dc84d45c1316b53403bab7a57ec38af31a5ca7fec5ae240ccfd5404eb90"} Dec 03 12:18:00 crc kubenswrapper[5052]: I1203 12:18:00.463380 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-54579c6749-xzrwm" event={"ID":"d1660f09-947f-433a-973f-0cd19d97331f","Type":"ContainerStarted","Data":"b136ec2626a76cf3a482455fc91da29f61b314a12e6e80ea4523ea4cc1bb249e"} Dec 03 12:18:00 crc kubenswrapper[5052]: I1203 12:18:00.463685 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/neutron-54579c6749-xzrwm" Dec 03 12:18:00 crc kubenswrapper[5052]: I1203 12:18:00.486506 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/neutron-54579c6749-xzrwm" podStartSLOduration=2.486480309 podStartE2EDuration="2.486480309s" podCreationTimestamp="2025-12-03 12:17:58 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:18:00.481676451 +0000 UTC m=+3471.069918245" watchObservedRunningTime="2025-12-03 12:18:00.486480309 +0000 UTC m=+3471.074722073" Dec 03 12:18:01 crc kubenswrapper[5052]: I1203 12:18:01.446830 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:18:01 crc kubenswrapper[5052]: I1203 12:18:01.492078 5052 generic.go:334] "Generic (PLEG): container finished" podID="321c0d14-7d66-4b3a-8989-f6a161421d2e" containerID="19aa0a6013cd529547763657024a5ae0ee17db18c9cc49526b6066bb7c447d1b" exitCode=0 Dec 03 12:18:01 crc kubenswrapper[5052]: I1203 12:18:01.492190 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"321c0d14-7d66-4b3a-8989-f6a161421d2e","Type":"ContainerDied","Data":"19aa0a6013cd529547763657024a5ae0ee17db18c9cc49526b6066bb7c447d1b"} Dec 03 12:18:01 crc kubenswrapper[5052]: I1203 12:18:01.492240 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:18:01 crc kubenswrapper[5052]: I1203 12:18:01.492263 5052 scope.go:117] "RemoveContainer" containerID="b800ece057aa8281071c80450e628f23c676cb4fc4228db9b5d9cd509984d054" Dec 03 12:18:01 crc kubenswrapper[5052]: I1203 12:18:01.492246 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"321c0d14-7d66-4b3a-8989-f6a161421d2e","Type":"ContainerDied","Data":"8d6a07cb12d95665a4437d877dc9f8c2c418b523de12dc71840ac557d95e3c39"} Dec 03 12:18:01 crc kubenswrapper[5052]: I1203 12:18:01.525577 5052 scope.go:117] "RemoveContainer" containerID="19aa0a6013cd529547763657024a5ae0ee17db18c9cc49526b6066bb7c447d1b" Dec 03 12:18:01 crc kubenswrapper[5052]: I1203 12:18:01.557861 5052 scope.go:117] "RemoveContainer" containerID="b800ece057aa8281071c80450e628f23c676cb4fc4228db9b5d9cd509984d054" Dec 03 12:18:01 crc kubenswrapper[5052]: E1203 12:18:01.558713 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b800ece057aa8281071c80450e628f23c676cb4fc4228db9b5d9cd509984d054\": container with ID starting with b800ece057aa8281071c80450e628f23c676cb4fc4228db9b5d9cd509984d054 not found: ID does not exist" containerID="b800ece057aa8281071c80450e628f23c676cb4fc4228db9b5d9cd509984d054" Dec 03 12:18:01 crc kubenswrapper[5052]: I1203 12:18:01.558762 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b800ece057aa8281071c80450e628f23c676cb4fc4228db9b5d9cd509984d054"} err="failed to get container status \"b800ece057aa8281071c80450e628f23c676cb4fc4228db9b5d9cd509984d054\": rpc error: code = NotFound desc = could not find container \"b800ece057aa8281071c80450e628f23c676cb4fc4228db9b5d9cd509984d054\": container with ID starting with b800ece057aa8281071c80450e628f23c676cb4fc4228db9b5d9cd509984d054 not found: ID does not exist" Dec 03 12:18:01 crc kubenswrapper[5052]: I1203 12:18:01.558803 5052 scope.go:117] "RemoveContainer" containerID="19aa0a6013cd529547763657024a5ae0ee17db18c9cc49526b6066bb7c447d1b" Dec 03 12:18:01 crc kubenswrapper[5052]: E1203 12:18:01.561699 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"19aa0a6013cd529547763657024a5ae0ee17db18c9cc49526b6066bb7c447d1b\": container with ID starting with 19aa0a6013cd529547763657024a5ae0ee17db18c9cc49526b6066bb7c447d1b not found: ID does not exist" containerID="19aa0a6013cd529547763657024a5ae0ee17db18c9cc49526b6066bb7c447d1b" Dec 03 12:18:01 crc kubenswrapper[5052]: I1203 12:18:01.561727 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"19aa0a6013cd529547763657024a5ae0ee17db18c9cc49526b6066bb7c447d1b"} err="failed to get container status \"19aa0a6013cd529547763657024a5ae0ee17db18c9cc49526b6066bb7c447d1b\": rpc error: code = NotFound desc = could not find container \"19aa0a6013cd529547763657024a5ae0ee17db18c9cc49526b6066bb7c447d1b\": container with ID starting with 19aa0a6013cd529547763657024a5ae0ee17db18c9cc49526b6066bb7c447d1b not found: ID does not exist" Dec 03 12:18:01 crc kubenswrapper[5052]: I1203 12:18:01.637416 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/321c0d14-7d66-4b3a-8989-f6a161421d2e-etc-machine-id\") pod \"321c0d14-7d66-4b3a-8989-f6a161421d2e\" (UID: \"321c0d14-7d66-4b3a-8989-f6a161421d2e\") " Dec 03 12:18:01 crc kubenswrapper[5052]: I1203 12:18:01.637782 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/321c0d14-7d66-4b3a-8989-f6a161421d2e-scripts\") pod \"321c0d14-7d66-4b3a-8989-f6a161421d2e\" (UID: \"321c0d14-7d66-4b3a-8989-f6a161421d2e\") " Dec 03 12:18:01 crc kubenswrapper[5052]: I1203 12:18:01.637846 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qx4fm\" (UniqueName: \"kubernetes.io/projected/321c0d14-7d66-4b3a-8989-f6a161421d2e-kube-api-access-qx4fm\") pod \"321c0d14-7d66-4b3a-8989-f6a161421d2e\" (UID: \"321c0d14-7d66-4b3a-8989-f6a161421d2e\") " Dec 03 12:18:01 crc kubenswrapper[5052]: I1203 12:18:01.637869 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/321c0d14-7d66-4b3a-8989-f6a161421d2e-config-data\") pod \"321c0d14-7d66-4b3a-8989-f6a161421d2e\" (UID: \"321c0d14-7d66-4b3a-8989-f6a161421d2e\") " Dec 03 12:18:01 crc kubenswrapper[5052]: I1203 12:18:01.638042 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/321c0d14-7d66-4b3a-8989-f6a161421d2e-combined-ca-bundle\") pod \"321c0d14-7d66-4b3a-8989-f6a161421d2e\" (UID: \"321c0d14-7d66-4b3a-8989-f6a161421d2e\") " Dec 03 12:18:01 crc kubenswrapper[5052]: I1203 12:18:01.638121 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/321c0d14-7d66-4b3a-8989-f6a161421d2e-config-data-custom\") pod \"321c0d14-7d66-4b3a-8989-f6a161421d2e\" (UID: \"321c0d14-7d66-4b3a-8989-f6a161421d2e\") " Dec 03 12:18:01 crc kubenswrapper[5052]: I1203 12:18:01.637496 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/321c0d14-7d66-4b3a-8989-f6a161421d2e-etc-machine-id" (OuterVolumeSpecName: "etc-machine-id") pod "321c0d14-7d66-4b3a-8989-f6a161421d2e" (UID: "321c0d14-7d66-4b3a-8989-f6a161421d2e"). InnerVolumeSpecName "etc-machine-id". PluginName "kubernetes.io/host-path", VolumeGidValue "" Dec 03 12:18:01 crc kubenswrapper[5052]: I1203 12:18:01.638781 5052 reconciler_common.go:293] "Volume detached for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/321c0d14-7d66-4b3a-8989-f6a161421d2e-etc-machine-id\") on node \"crc\" DevicePath \"\"" Dec 03 12:18:01 crc kubenswrapper[5052]: I1203 12:18:01.644333 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/321c0d14-7d66-4b3a-8989-f6a161421d2e-config-data-custom" (OuterVolumeSpecName: "config-data-custom") pod "321c0d14-7d66-4b3a-8989-f6a161421d2e" (UID: "321c0d14-7d66-4b3a-8989-f6a161421d2e"). InnerVolumeSpecName "config-data-custom". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:18:01 crc kubenswrapper[5052]: I1203 12:18:01.645028 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/321c0d14-7d66-4b3a-8989-f6a161421d2e-kube-api-access-qx4fm" (OuterVolumeSpecName: "kube-api-access-qx4fm") pod "321c0d14-7d66-4b3a-8989-f6a161421d2e" (UID: "321c0d14-7d66-4b3a-8989-f6a161421d2e"). InnerVolumeSpecName "kube-api-access-qx4fm". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:18:01 crc kubenswrapper[5052]: I1203 12:18:01.649452 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/321c0d14-7d66-4b3a-8989-f6a161421d2e-scripts" (OuterVolumeSpecName: "scripts") pod "321c0d14-7d66-4b3a-8989-f6a161421d2e" (UID: "321c0d14-7d66-4b3a-8989-f6a161421d2e"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:18:01 crc kubenswrapper[5052]: I1203 12:18:01.739883 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/321c0d14-7d66-4b3a-8989-f6a161421d2e-config-data-custom\") on node \"crc\" DevicePath \"\"" Dec 03 12:18:01 crc kubenswrapper[5052]: I1203 12:18:01.739914 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/321c0d14-7d66-4b3a-8989-f6a161421d2e-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:18:01 crc kubenswrapper[5052]: I1203 12:18:01.739926 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-qx4fm\" (UniqueName: \"kubernetes.io/projected/321c0d14-7d66-4b3a-8989-f6a161421d2e-kube-api-access-qx4fm\") on node \"crc\" DevicePath \"\"" Dec 03 12:18:01 crc kubenswrapper[5052]: I1203 12:18:01.745622 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/321c0d14-7d66-4b3a-8989-f6a161421d2e-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "321c0d14-7d66-4b3a-8989-f6a161421d2e" (UID: "321c0d14-7d66-4b3a-8989-f6a161421d2e"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:18:01 crc kubenswrapper[5052]: I1203 12:18:01.801644 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/321c0d14-7d66-4b3a-8989-f6a161421d2e-config-data" (OuterVolumeSpecName: "config-data") pod "321c0d14-7d66-4b3a-8989-f6a161421d2e" (UID: "321c0d14-7d66-4b3a-8989-f6a161421d2e"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:18:01 crc kubenswrapper[5052]: I1203 12:18:01.841920 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/321c0d14-7d66-4b3a-8989-f6a161421d2e-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:18:01 crc kubenswrapper[5052]: I1203 12:18:01.841961 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/321c0d14-7d66-4b3a-8989-f6a161421d2e-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:18:02 crc kubenswrapper[5052]: I1203 12:18:02.139565 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 12:18:02 crc kubenswrapper[5052]: I1203 12:18:02.156054 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 12:18:02 crc kubenswrapper[5052]: I1203 12:18:02.170551 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 12:18:02 crc kubenswrapper[5052]: E1203 12:18:02.170942 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="321c0d14-7d66-4b3a-8989-f6a161421d2e" containerName="probe" Dec 03 12:18:02 crc kubenswrapper[5052]: I1203 12:18:02.170960 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="321c0d14-7d66-4b3a-8989-f6a161421d2e" containerName="probe" Dec 03 12:18:02 crc kubenswrapper[5052]: E1203 12:18:02.170972 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="321c0d14-7d66-4b3a-8989-f6a161421d2e" containerName="cinder-scheduler" Dec 03 12:18:02 crc kubenswrapper[5052]: I1203 12:18:02.170978 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="321c0d14-7d66-4b3a-8989-f6a161421d2e" containerName="cinder-scheduler" Dec 03 12:18:02 crc kubenswrapper[5052]: I1203 12:18:02.171168 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="321c0d14-7d66-4b3a-8989-f6a161421d2e" containerName="cinder-scheduler" Dec 03 12:18:02 crc kubenswrapper[5052]: I1203 12:18:02.171187 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="321c0d14-7d66-4b3a-8989-f6a161421d2e" containerName="probe" Dec 03 12:18:02 crc kubenswrapper[5052]: I1203 12:18:02.172142 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:18:02 crc kubenswrapper[5052]: I1203 12:18:02.175203 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cinder-scheduler-config-data" Dec 03 12:18:02 crc kubenswrapper[5052]: I1203 12:18:02.181490 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 12:18:02 crc kubenswrapper[5052]: I1203 12:18:02.288068 5052 scope.go:117] "RemoveContainer" containerID="6908da1783700654af60d79f711ce4360ed719f4f83031effe704c70de2ab47f" Dec 03 12:18:02 crc kubenswrapper[5052]: I1203 12:18:02.326338 5052 scope.go:117] "RemoveContainer" containerID="0519af1e24d904a84f986fb58221ec301343653caf8111b439f0966c5ec6f27b" Dec 03 12:18:02 crc kubenswrapper[5052]: I1203 12:18:02.351216 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9h52h\" (UniqueName: \"kubernetes.io/projected/d5750ef1-ce21-4092-9612-13d6f6c923df-kube-api-access-9h52h\") pod \"cinder-scheduler-0\" (UID: \"d5750ef1-ce21-4092-9612-13d6f6c923df\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:18:02 crc kubenswrapper[5052]: I1203 12:18:02.351270 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d5750ef1-ce21-4092-9612-13d6f6c923df-combined-ca-bundle\") pod \"cinder-scheduler-0\" (UID: \"d5750ef1-ce21-4092-9612-13d6f6c923df\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:18:02 crc kubenswrapper[5052]: I1203 12:18:02.351349 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/d5750ef1-ce21-4092-9612-13d6f6c923df-config-data\") pod \"cinder-scheduler-0\" (UID: \"d5750ef1-ce21-4092-9612-13d6f6c923df\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:18:02 crc kubenswrapper[5052]: I1203 12:18:02.351380 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/d5750ef1-ce21-4092-9612-13d6f6c923df-config-data-custom\") pod \"cinder-scheduler-0\" (UID: \"d5750ef1-ce21-4092-9612-13d6f6c923df\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:18:02 crc kubenswrapper[5052]: I1203 12:18:02.351407 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/d5750ef1-ce21-4092-9612-13d6f6c923df-scripts\") pod \"cinder-scheduler-0\" (UID: \"d5750ef1-ce21-4092-9612-13d6f6c923df\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:18:02 crc kubenswrapper[5052]: I1203 12:18:02.351469 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/d5750ef1-ce21-4092-9612-13d6f6c923df-etc-machine-id\") pod \"cinder-scheduler-0\" (UID: \"d5750ef1-ce21-4092-9612-13d6f6c923df\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:18:02 crc kubenswrapper[5052]: I1203 12:18:02.440666 5052 scope.go:117] "RemoveContainer" containerID="b2b84bb90c0815c25cd39fbb85a799acfc5bc3d5a578b8bf8f3a88a0db65844c" Dec 03 12:18:02 crc kubenswrapper[5052]: I1203 12:18:02.453758 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-9h52h\" (UniqueName: \"kubernetes.io/projected/d5750ef1-ce21-4092-9612-13d6f6c923df-kube-api-access-9h52h\") pod \"cinder-scheduler-0\" (UID: \"d5750ef1-ce21-4092-9612-13d6f6c923df\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:18:02 crc kubenswrapper[5052]: I1203 12:18:02.453815 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d5750ef1-ce21-4092-9612-13d6f6c923df-combined-ca-bundle\") pod \"cinder-scheduler-0\" (UID: \"d5750ef1-ce21-4092-9612-13d6f6c923df\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:18:02 crc kubenswrapper[5052]: I1203 12:18:02.453889 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/d5750ef1-ce21-4092-9612-13d6f6c923df-config-data\") pod \"cinder-scheduler-0\" (UID: \"d5750ef1-ce21-4092-9612-13d6f6c923df\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:18:02 crc kubenswrapper[5052]: I1203 12:18:02.453917 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/d5750ef1-ce21-4092-9612-13d6f6c923df-config-data-custom\") pod \"cinder-scheduler-0\" (UID: \"d5750ef1-ce21-4092-9612-13d6f6c923df\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:18:02 crc kubenswrapper[5052]: I1203 12:18:02.453940 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/d5750ef1-ce21-4092-9612-13d6f6c923df-scripts\") pod \"cinder-scheduler-0\" (UID: \"d5750ef1-ce21-4092-9612-13d6f6c923df\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:18:02 crc kubenswrapper[5052]: I1203 12:18:02.454002 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/d5750ef1-ce21-4092-9612-13d6f6c923df-etc-machine-id\") pod \"cinder-scheduler-0\" (UID: \"d5750ef1-ce21-4092-9612-13d6f6c923df\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:18:02 crc kubenswrapper[5052]: I1203 12:18:02.454125 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/d5750ef1-ce21-4092-9612-13d6f6c923df-etc-machine-id\") pod \"cinder-scheduler-0\" (UID: \"d5750ef1-ce21-4092-9612-13d6f6c923df\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:18:02 crc kubenswrapper[5052]: I1203 12:18:02.459829 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d5750ef1-ce21-4092-9612-13d6f6c923df-combined-ca-bundle\") pod \"cinder-scheduler-0\" (UID: \"d5750ef1-ce21-4092-9612-13d6f6c923df\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:18:02 crc kubenswrapper[5052]: I1203 12:18:02.459864 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/d5750ef1-ce21-4092-9612-13d6f6c923df-scripts\") pod \"cinder-scheduler-0\" (UID: \"d5750ef1-ce21-4092-9612-13d6f6c923df\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:18:02 crc kubenswrapper[5052]: I1203 12:18:02.460593 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/d5750ef1-ce21-4092-9612-13d6f6c923df-config-data-custom\") pod \"cinder-scheduler-0\" (UID: \"d5750ef1-ce21-4092-9612-13d6f6c923df\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:18:02 crc kubenswrapper[5052]: I1203 12:18:02.460708 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/d5750ef1-ce21-4092-9612-13d6f6c923df-config-data\") pod \"cinder-scheduler-0\" (UID: \"d5750ef1-ce21-4092-9612-13d6f6c923df\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:18:02 crc kubenswrapper[5052]: I1203 12:18:02.472737 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-9h52h\" (UniqueName: \"kubernetes.io/projected/d5750ef1-ce21-4092-9612-13d6f6c923df-kube-api-access-9h52h\") pod \"cinder-scheduler-0\" (UID: \"d5750ef1-ce21-4092-9612-13d6f6c923df\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:18:02 crc kubenswrapper[5052]: I1203 12:18:02.481962 5052 scope.go:117] "RemoveContainer" containerID="425e1d453b59af4c319776f6782cfa58108a322f2cd520919a3ce1749cbf5b9a" Dec 03 12:18:02 crc kubenswrapper[5052]: I1203 12:18:02.493983 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:18:02 crc kubenswrapper[5052]: I1203 12:18:02.605388 5052 scope.go:117] "RemoveContainer" containerID="f020c0314d37f1ebed5754d085ae61111bf03a1246b93ef4ff86bfaf736db918" Dec 03 12:18:02 crc kubenswrapper[5052]: I1203 12:18:02.649874 5052 scope.go:117] "RemoveContainer" containerID="d25bb92aee772ca0fd8734997ea7b7df542c4bbee1daf0a9eae2d669f91bf473" Dec 03 12:18:02 crc kubenswrapper[5052]: I1203 12:18:02.681432 5052 scope.go:117] "RemoveContainer" containerID="d4bb4f5c0829cb278a445845849850d561fa98757159ecf40155346ec8896bde" Dec 03 12:18:02 crc kubenswrapper[5052]: I1203 12:18:02.706477 5052 scope.go:117] "RemoveContainer" containerID="f360d4e646d5e381237bbb8eca54754e311f0f72353048f82b3866d760e09157" Dec 03 12:18:02 crc kubenswrapper[5052]: I1203 12:18:02.961855 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 12:18:02 crc kubenswrapper[5052]: W1203 12:18:02.985150 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd5750ef1_ce21_4092_9612_13d6f6c923df.slice/crio-f56f669cd30eb6c41c01ec084a9fe903bb582700fc2651cbab96ee3a78bb81d8 WatchSource:0}: Error finding container f56f669cd30eb6c41c01ec084a9fe903bb582700fc2651cbab96ee3a78bb81d8: Status 404 returned error can't find the container with id f56f669cd30eb6c41c01ec084a9fe903bb582700fc2651cbab96ee3a78bb81d8 Dec 03 12:18:03 crc kubenswrapper[5052]: I1203 12:18:03.127191 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:18:03 crc kubenswrapper[5052]: I1203 12:18:03.539893 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"d5750ef1-ce21-4092-9612-13d6f6c923df","Type":"ContainerStarted","Data":"f56f669cd30eb6c41c01ec084a9fe903bb582700fc2651cbab96ee3a78bb81d8"} Dec 03 12:18:03 crc kubenswrapper[5052]: I1203 12:18:03.685791 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="321c0d14-7d66-4b3a-8989-f6a161421d2e" path="/var/lib/kubelet/pods/321c0d14-7d66-4b3a-8989-f6a161421d2e/volumes" Dec 03 12:18:04 crc kubenswrapper[5052]: I1203 12:18:04.551178 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"d5750ef1-ce21-4092-9612-13d6f6c923df","Type":"ContainerStarted","Data":"82bddf0f905668e21ca0acf3ebd30d75b07c6a6534dd49912da54363cd369335"} Dec 03 12:18:04 crc kubenswrapper[5052]: I1203 12:18:04.559732 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"d5750ef1-ce21-4092-9612-13d6f6c923df","Type":"ContainerStarted","Data":"66ff70190e4169f7ceded95ca82e7cd10045ed2cce8eb37b9d1a20b3e28c1110"} Dec 03 12:18:04 crc kubenswrapper[5052]: I1203 12:18:04.576368 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/cinder-scheduler-0" podStartSLOduration=2.576343902 podStartE2EDuration="2.576343902s" podCreationTimestamp="2025-12-03 12:18:02 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:18:04.568631041 +0000 UTC m=+3475.156872825" watchObservedRunningTime="2025-12-03 12:18:04.576343902 +0000 UTC m=+3475.164585666" Dec 03 12:18:05 crc kubenswrapper[5052]: I1203 12:18:05.799852 5052 pod_container_manager_linux.go:210] "Failed to delete cgroup paths" cgroupName=["kubepods","besteffort","pod408d4a98-4cc6-4a13-a2f3-940747dfb910"] err="unable to destroy cgroup paths for cgroup [kubepods besteffort pod408d4a98-4cc6-4a13-a2f3-940747dfb910] : Timed out while waiting for systemd to remove kubepods-besteffort-pod408d4a98_4cc6_4a13_a2f3_940747dfb910.slice" Dec 03 12:18:07 crc kubenswrapper[5052]: I1203 12:18:07.494866 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:18:09 crc kubenswrapper[5052]: I1203 12:18:09.460770 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/placement-7d6bd9d7c9-n2lr6" Dec 03 12:18:09 crc kubenswrapper[5052]: I1203 12:18:09.700865 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:18:10 crc kubenswrapper[5052]: I1203 12:18:10.370245 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/placement-7d6bd9d7c9-n2lr6" Dec 03 12:18:12 crc kubenswrapper[5052]: I1203 12:18:12.706715 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:18:19 crc kubenswrapper[5052]: I1203 12:18:19.304936 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/keystone-78dd58875-slm9r" Dec 03 12:18:21 crc kubenswrapper[5052]: I1203 12:18:21.495419 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/openstackclient"] Dec 03 12:18:21 crc kubenswrapper[5052]: I1203 12:18:21.496794 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstackclient" Dec 03 12:18:21 crc kubenswrapper[5052]: I1203 12:18:21.499361 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"openstack-config-secret" Dec 03 12:18:21 crc kubenswrapper[5052]: I1203 12:18:21.500450 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"openstackclient-openstackclient-dockercfg-rddjd" Dec 03 12:18:21 crc kubenswrapper[5052]: I1203 12:18:21.500627 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"openstack-config" Dec 03 12:18:21 crc kubenswrapper[5052]: I1203 12:18:21.511183 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/openstackclient"] Dec 03 12:18:21 crc kubenswrapper[5052]: I1203 12:18:21.648246 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/357bc965-8633-4d67-992e-baa770a0defd-combined-ca-bundle\") pod \"openstackclient\" (UID: \"357bc965-8633-4d67-992e-baa770a0defd\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 12:18:21 crc kubenswrapper[5052]: I1203 12:18:21.648320 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openstack-config-secret\" (UniqueName: \"kubernetes.io/secret/357bc965-8633-4d67-992e-baa770a0defd-openstack-config-secret\") pod \"openstackclient\" (UID: \"357bc965-8633-4d67-992e-baa770a0defd\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 12:18:21 crc kubenswrapper[5052]: I1203 12:18:21.648362 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openstack-config\" (UniqueName: \"kubernetes.io/configmap/357bc965-8633-4d67-992e-baa770a0defd-openstack-config\") pod \"openstackclient\" (UID: \"357bc965-8633-4d67-992e-baa770a0defd\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 12:18:21 crc kubenswrapper[5052]: I1203 12:18:21.648430 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p4cv8\" (UniqueName: \"kubernetes.io/projected/357bc965-8633-4d67-992e-baa770a0defd-kube-api-access-p4cv8\") pod \"openstackclient\" (UID: \"357bc965-8633-4d67-992e-baa770a0defd\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 12:18:21 crc kubenswrapper[5052]: I1203 12:18:21.750172 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/357bc965-8633-4d67-992e-baa770a0defd-combined-ca-bundle\") pod \"openstackclient\" (UID: \"357bc965-8633-4d67-992e-baa770a0defd\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 12:18:21 crc kubenswrapper[5052]: I1203 12:18:21.750579 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"openstack-config-secret\" (UniqueName: \"kubernetes.io/secret/357bc965-8633-4d67-992e-baa770a0defd-openstack-config-secret\") pod \"openstackclient\" (UID: \"357bc965-8633-4d67-992e-baa770a0defd\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 12:18:21 crc kubenswrapper[5052]: I1203 12:18:21.750625 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"openstack-config\" (UniqueName: \"kubernetes.io/configmap/357bc965-8633-4d67-992e-baa770a0defd-openstack-config\") pod \"openstackclient\" (UID: \"357bc965-8633-4d67-992e-baa770a0defd\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 12:18:21 crc kubenswrapper[5052]: I1203 12:18:21.750715 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-p4cv8\" (UniqueName: \"kubernetes.io/projected/357bc965-8633-4d67-992e-baa770a0defd-kube-api-access-p4cv8\") pod \"openstackclient\" (UID: \"357bc965-8633-4d67-992e-baa770a0defd\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 12:18:21 crc kubenswrapper[5052]: I1203 12:18:21.752539 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"openstack-config\" (UniqueName: \"kubernetes.io/configmap/357bc965-8633-4d67-992e-baa770a0defd-openstack-config\") pod \"openstackclient\" (UID: \"357bc965-8633-4d67-992e-baa770a0defd\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 12:18:21 crc kubenswrapper[5052]: I1203 12:18:21.756757 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"openstack-config-secret\" (UniqueName: \"kubernetes.io/secret/357bc965-8633-4d67-992e-baa770a0defd-openstack-config-secret\") pod \"openstackclient\" (UID: \"357bc965-8633-4d67-992e-baa770a0defd\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 12:18:21 crc kubenswrapper[5052]: I1203 12:18:21.764790 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/357bc965-8633-4d67-992e-baa770a0defd-combined-ca-bundle\") pod \"openstackclient\" (UID: \"357bc965-8633-4d67-992e-baa770a0defd\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 12:18:21 crc kubenswrapper[5052]: I1203 12:18:21.770797 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-p4cv8\" (UniqueName: \"kubernetes.io/projected/357bc965-8633-4d67-992e-baa770a0defd-kube-api-access-p4cv8\") pod \"openstackclient\" (UID: \"357bc965-8633-4d67-992e-baa770a0defd\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 12:18:21 crc kubenswrapper[5052]: I1203 12:18:21.858803 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstackclient" Dec 03 12:18:22 crc kubenswrapper[5052]: I1203 12:18:22.318501 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/openstackclient"] Dec 03 12:18:22 crc kubenswrapper[5052]: I1203 12:18:22.742782 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstackclient" event={"ID":"357bc965-8633-4d67-992e-baa770a0defd","Type":"ContainerStarted","Data":"97eb19cad24849878c7614c23d646abe200c279ae260c2f8f7bae07110bd2a2b"} Dec 03 12:18:22 crc kubenswrapper[5052]: I1203 12:18:22.743161 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstackclient" event={"ID":"357bc965-8633-4d67-992e-baa770a0defd","Type":"ContainerStarted","Data":"00f5191a5b2048454ee6e431bb224c1eafd448387a1cfcf173d2e65ca2a76343"} Dec 03 12:18:25 crc kubenswrapper[5052]: I1203 12:18:25.217443 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/openstackclient" podStartSLOduration=4.217417431 podStartE2EDuration="4.217417431s" podCreationTimestamp="2025-12-03 12:18:21 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:18:22.771955831 +0000 UTC m=+3493.360197605" watchObservedRunningTime="2025-12-03 12:18:25.217417431 +0000 UTC m=+3495.805659195" Dec 03 12:18:25 crc kubenswrapper[5052]: I1203 12:18:25.225569 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:18:25 crc kubenswrapper[5052]: I1203 12:18:25.225862 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="e36b4a02-2b71-4e70-b890-e91695ec53f3" containerName="ceilometer-central-agent" containerID="cri-o://7ac693835aaf1e9e855735211f4334f714699a1693c265e3b741d201b9e78b8f" gracePeriod=30 Dec 03 12:18:25 crc kubenswrapper[5052]: I1203 12:18:25.225932 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="e36b4a02-2b71-4e70-b890-e91695ec53f3" containerName="sg-core" containerID="cri-o://f6833798ca88a77a606fd1470b8377553dfe1ed45d731e4a1f9fce2597d061a8" gracePeriod=30 Dec 03 12:18:25 crc kubenswrapper[5052]: I1203 12:18:25.225929 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="e36b4a02-2b71-4e70-b890-e91695ec53f3" containerName="proxy-httpd" containerID="cri-o://6d1dc86dca302c34d13335215a72604630127727141b8b1de8e5259bd7e5f5fe" gracePeriod=30 Dec 03 12:18:25 crc kubenswrapper[5052]: I1203 12:18:25.225949 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="e36b4a02-2b71-4e70-b890-e91695ec53f3" containerName="ceilometer-notification-agent" containerID="cri-o://ec4c6e5477ad824e01b795f69608fed547c19bcae844fe2fa93dbe97916d866e" gracePeriod=30 Dec 03 12:18:25 crc kubenswrapper[5052]: I1203 12:18:25.778073 5052 generic.go:334] "Generic (PLEG): container finished" podID="e36b4a02-2b71-4e70-b890-e91695ec53f3" containerID="6d1dc86dca302c34d13335215a72604630127727141b8b1de8e5259bd7e5f5fe" exitCode=0 Dec 03 12:18:25 crc kubenswrapper[5052]: I1203 12:18:25.778348 5052 generic.go:334] "Generic (PLEG): container finished" podID="e36b4a02-2b71-4e70-b890-e91695ec53f3" containerID="f6833798ca88a77a606fd1470b8377553dfe1ed45d731e4a1f9fce2597d061a8" exitCode=2 Dec 03 12:18:25 crc kubenswrapper[5052]: I1203 12:18:25.778358 5052 generic.go:334] "Generic (PLEG): container finished" podID="e36b4a02-2b71-4e70-b890-e91695ec53f3" containerID="7ac693835aaf1e9e855735211f4334f714699a1693c265e3b741d201b9e78b8f" exitCode=0 Dec 03 12:18:25 crc kubenswrapper[5052]: I1203 12:18:25.778139 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"e36b4a02-2b71-4e70-b890-e91695ec53f3","Type":"ContainerDied","Data":"6d1dc86dca302c34d13335215a72604630127727141b8b1de8e5259bd7e5f5fe"} Dec 03 12:18:25 crc kubenswrapper[5052]: I1203 12:18:25.778393 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"e36b4a02-2b71-4e70-b890-e91695ec53f3","Type":"ContainerDied","Data":"f6833798ca88a77a606fd1470b8377553dfe1ed45d731e4a1f9fce2597d061a8"} Dec 03 12:18:25 crc kubenswrapper[5052]: I1203 12:18:25.778407 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"e36b4a02-2b71-4e70-b890-e91695ec53f3","Type":"ContainerDied","Data":"7ac693835aaf1e9e855735211f4334f714699a1693c265e3b741d201b9e78b8f"} Dec 03 12:18:26 crc kubenswrapper[5052]: I1203 12:18:26.228762 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/swift-proxy-d4747d79f-wtbhv"] Dec 03 12:18:26 crc kubenswrapper[5052]: I1203 12:18:26.233945 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-proxy-d4747d79f-wtbhv" Dec 03 12:18:26 crc kubenswrapper[5052]: I1203 12:18:26.240673 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"swift-proxy-config-data" Dec 03 12:18:26 crc kubenswrapper[5052]: I1203 12:18:26.240991 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-swift-public-svc" Dec 03 12:18:26 crc kubenswrapper[5052]: I1203 12:18:26.252741 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-swift-internal-svc" Dec 03 12:18:26 crc kubenswrapper[5052]: I1203 12:18:26.276617 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/swift-proxy-d4747d79f-wtbhv"] Dec 03 12:18:26 crc kubenswrapper[5052]: I1203 12:18:26.341588 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/7e109538-a2d2-43bb-8cef-bf99d7d0f1e9-config-data\") pod \"swift-proxy-d4747d79f-wtbhv\" (UID: \"7e109538-a2d2-43bb-8cef-bf99d7d0f1e9\") " pod="openstack-kuttl-tests/swift-proxy-d4747d79f-wtbhv" Dec 03 12:18:26 crc kubenswrapper[5052]: I1203 12:18:26.341657 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ddvrk\" (UniqueName: \"kubernetes.io/projected/7e109538-a2d2-43bb-8cef-bf99d7d0f1e9-kube-api-access-ddvrk\") pod \"swift-proxy-d4747d79f-wtbhv\" (UID: \"7e109538-a2d2-43bb-8cef-bf99d7d0f1e9\") " pod="openstack-kuttl-tests/swift-proxy-d4747d79f-wtbhv" Dec 03 12:18:26 crc kubenswrapper[5052]: I1203 12:18:26.341685 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/7e109538-a2d2-43bb-8cef-bf99d7d0f1e9-combined-ca-bundle\") pod \"swift-proxy-d4747d79f-wtbhv\" (UID: \"7e109538-a2d2-43bb-8cef-bf99d7d0f1e9\") " pod="openstack-kuttl-tests/swift-proxy-d4747d79f-wtbhv" Dec 03 12:18:26 crc kubenswrapper[5052]: I1203 12:18:26.341744 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/7e109538-a2d2-43bb-8cef-bf99d7d0f1e9-run-httpd\") pod \"swift-proxy-d4747d79f-wtbhv\" (UID: \"7e109538-a2d2-43bb-8cef-bf99d7d0f1e9\") " pod="openstack-kuttl-tests/swift-proxy-d4747d79f-wtbhv" Dec 03 12:18:26 crc kubenswrapper[5052]: I1203 12:18:26.341769 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/7e109538-a2d2-43bb-8cef-bf99d7d0f1e9-public-tls-certs\") pod \"swift-proxy-d4747d79f-wtbhv\" (UID: \"7e109538-a2d2-43bb-8cef-bf99d7d0f1e9\") " pod="openstack-kuttl-tests/swift-proxy-d4747d79f-wtbhv" Dec 03 12:18:26 crc kubenswrapper[5052]: I1203 12:18:26.341800 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/7e109538-a2d2-43bb-8cef-bf99d7d0f1e9-etc-swift\") pod \"swift-proxy-d4747d79f-wtbhv\" (UID: \"7e109538-a2d2-43bb-8cef-bf99d7d0f1e9\") " pod="openstack-kuttl-tests/swift-proxy-d4747d79f-wtbhv" Dec 03 12:18:26 crc kubenswrapper[5052]: I1203 12:18:26.341815 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/7e109538-a2d2-43bb-8cef-bf99d7d0f1e9-internal-tls-certs\") pod \"swift-proxy-d4747d79f-wtbhv\" (UID: \"7e109538-a2d2-43bb-8cef-bf99d7d0f1e9\") " pod="openstack-kuttl-tests/swift-proxy-d4747d79f-wtbhv" Dec 03 12:18:26 crc kubenswrapper[5052]: I1203 12:18:26.341856 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/7e109538-a2d2-43bb-8cef-bf99d7d0f1e9-log-httpd\") pod \"swift-proxy-d4747d79f-wtbhv\" (UID: \"7e109538-a2d2-43bb-8cef-bf99d7d0f1e9\") " pod="openstack-kuttl-tests/swift-proxy-d4747d79f-wtbhv" Dec 03 12:18:26 crc kubenswrapper[5052]: I1203 12:18:26.443435 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/7e109538-a2d2-43bb-8cef-bf99d7d0f1e9-run-httpd\") pod \"swift-proxy-d4747d79f-wtbhv\" (UID: \"7e109538-a2d2-43bb-8cef-bf99d7d0f1e9\") " pod="openstack-kuttl-tests/swift-proxy-d4747d79f-wtbhv" Dec 03 12:18:26 crc kubenswrapper[5052]: I1203 12:18:26.444000 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/7e109538-a2d2-43bb-8cef-bf99d7d0f1e9-run-httpd\") pod \"swift-proxy-d4747d79f-wtbhv\" (UID: \"7e109538-a2d2-43bb-8cef-bf99d7d0f1e9\") " pod="openstack-kuttl-tests/swift-proxy-d4747d79f-wtbhv" Dec 03 12:18:26 crc kubenswrapper[5052]: I1203 12:18:26.444149 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/7e109538-a2d2-43bb-8cef-bf99d7d0f1e9-public-tls-certs\") pod \"swift-proxy-d4747d79f-wtbhv\" (UID: \"7e109538-a2d2-43bb-8cef-bf99d7d0f1e9\") " pod="openstack-kuttl-tests/swift-proxy-d4747d79f-wtbhv" Dec 03 12:18:26 crc kubenswrapper[5052]: I1203 12:18:26.444233 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/7e109538-a2d2-43bb-8cef-bf99d7d0f1e9-etc-swift\") pod \"swift-proxy-d4747d79f-wtbhv\" (UID: \"7e109538-a2d2-43bb-8cef-bf99d7d0f1e9\") " pod="openstack-kuttl-tests/swift-proxy-d4747d79f-wtbhv" Dec 03 12:18:26 crc kubenswrapper[5052]: I1203 12:18:26.444264 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/7e109538-a2d2-43bb-8cef-bf99d7d0f1e9-internal-tls-certs\") pod \"swift-proxy-d4747d79f-wtbhv\" (UID: \"7e109538-a2d2-43bb-8cef-bf99d7d0f1e9\") " pod="openstack-kuttl-tests/swift-proxy-d4747d79f-wtbhv" Dec 03 12:18:26 crc kubenswrapper[5052]: I1203 12:18:26.444450 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/7e109538-a2d2-43bb-8cef-bf99d7d0f1e9-log-httpd\") pod \"swift-proxy-d4747d79f-wtbhv\" (UID: \"7e109538-a2d2-43bb-8cef-bf99d7d0f1e9\") " pod="openstack-kuttl-tests/swift-proxy-d4747d79f-wtbhv" Dec 03 12:18:26 crc kubenswrapper[5052]: I1203 12:18:26.445258 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/7e109538-a2d2-43bb-8cef-bf99d7d0f1e9-config-data\") pod \"swift-proxy-d4747d79f-wtbhv\" (UID: \"7e109538-a2d2-43bb-8cef-bf99d7d0f1e9\") " pod="openstack-kuttl-tests/swift-proxy-d4747d79f-wtbhv" Dec 03 12:18:26 crc kubenswrapper[5052]: I1203 12:18:26.444946 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/7e109538-a2d2-43bb-8cef-bf99d7d0f1e9-log-httpd\") pod \"swift-proxy-d4747d79f-wtbhv\" (UID: \"7e109538-a2d2-43bb-8cef-bf99d7d0f1e9\") " pod="openstack-kuttl-tests/swift-proxy-d4747d79f-wtbhv" Dec 03 12:18:26 crc kubenswrapper[5052]: I1203 12:18:26.445306 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-ddvrk\" (UniqueName: \"kubernetes.io/projected/7e109538-a2d2-43bb-8cef-bf99d7d0f1e9-kube-api-access-ddvrk\") pod \"swift-proxy-d4747d79f-wtbhv\" (UID: \"7e109538-a2d2-43bb-8cef-bf99d7d0f1e9\") " pod="openstack-kuttl-tests/swift-proxy-d4747d79f-wtbhv" Dec 03 12:18:26 crc kubenswrapper[5052]: I1203 12:18:26.445429 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/7e109538-a2d2-43bb-8cef-bf99d7d0f1e9-combined-ca-bundle\") pod \"swift-proxy-d4747d79f-wtbhv\" (UID: \"7e109538-a2d2-43bb-8cef-bf99d7d0f1e9\") " pod="openstack-kuttl-tests/swift-proxy-d4747d79f-wtbhv" Dec 03 12:18:26 crc kubenswrapper[5052]: I1203 12:18:26.453385 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/7e109538-a2d2-43bb-8cef-bf99d7d0f1e9-public-tls-certs\") pod \"swift-proxy-d4747d79f-wtbhv\" (UID: \"7e109538-a2d2-43bb-8cef-bf99d7d0f1e9\") " pod="openstack-kuttl-tests/swift-proxy-d4747d79f-wtbhv" Dec 03 12:18:26 crc kubenswrapper[5052]: I1203 12:18:26.453527 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/7e109538-a2d2-43bb-8cef-bf99d7d0f1e9-combined-ca-bundle\") pod \"swift-proxy-d4747d79f-wtbhv\" (UID: \"7e109538-a2d2-43bb-8cef-bf99d7d0f1e9\") " pod="openstack-kuttl-tests/swift-proxy-d4747d79f-wtbhv" Dec 03 12:18:26 crc kubenswrapper[5052]: I1203 12:18:26.453761 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/7e109538-a2d2-43bb-8cef-bf99d7d0f1e9-internal-tls-certs\") pod \"swift-proxy-d4747d79f-wtbhv\" (UID: \"7e109538-a2d2-43bb-8cef-bf99d7d0f1e9\") " pod="openstack-kuttl-tests/swift-proxy-d4747d79f-wtbhv" Dec 03 12:18:26 crc kubenswrapper[5052]: I1203 12:18:26.453787 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/7e109538-a2d2-43bb-8cef-bf99d7d0f1e9-etc-swift\") pod \"swift-proxy-d4747d79f-wtbhv\" (UID: \"7e109538-a2d2-43bb-8cef-bf99d7d0f1e9\") " pod="openstack-kuttl-tests/swift-proxy-d4747d79f-wtbhv" Dec 03 12:18:26 crc kubenswrapper[5052]: I1203 12:18:26.466128 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-ddvrk\" (UniqueName: \"kubernetes.io/projected/7e109538-a2d2-43bb-8cef-bf99d7d0f1e9-kube-api-access-ddvrk\") pod \"swift-proxy-d4747d79f-wtbhv\" (UID: \"7e109538-a2d2-43bb-8cef-bf99d7d0f1e9\") " pod="openstack-kuttl-tests/swift-proxy-d4747d79f-wtbhv" Dec 03 12:18:26 crc kubenswrapper[5052]: I1203 12:18:26.466643 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/7e109538-a2d2-43bb-8cef-bf99d7d0f1e9-config-data\") pod \"swift-proxy-d4747d79f-wtbhv\" (UID: \"7e109538-a2d2-43bb-8cef-bf99d7d0f1e9\") " pod="openstack-kuttl-tests/swift-proxy-d4747d79f-wtbhv" Dec 03 12:18:26 crc kubenswrapper[5052]: I1203 12:18:26.585406 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-proxy-d4747d79f-wtbhv" Dec 03 12:18:27 crc kubenswrapper[5052]: I1203 12:18:27.054331 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/neutron-54c879c996-kc674" Dec 03 12:18:27 crc kubenswrapper[5052]: I1203 12:18:27.083029 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/swift-proxy-d4747d79f-wtbhv"] Dec 03 12:18:27 crc kubenswrapper[5052]: I1203 12:18:27.271436 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:18:27 crc kubenswrapper[5052]: I1203 12:18:27.378235 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/e36b4a02-2b71-4e70-b890-e91695ec53f3-scripts\") pod \"e36b4a02-2b71-4e70-b890-e91695ec53f3\" (UID: \"e36b4a02-2b71-4e70-b890-e91695ec53f3\") " Dec 03 12:18:27 crc kubenswrapper[5052]: I1203 12:18:27.378319 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/e36b4a02-2b71-4e70-b890-e91695ec53f3-sg-core-conf-yaml\") pod \"e36b4a02-2b71-4e70-b890-e91695ec53f3\" (UID: \"e36b4a02-2b71-4e70-b890-e91695ec53f3\") " Dec 03 12:18:27 crc kubenswrapper[5052]: I1203 12:18:27.378408 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/e36b4a02-2b71-4e70-b890-e91695ec53f3-log-httpd\") pod \"e36b4a02-2b71-4e70-b890-e91695ec53f3\" (UID: \"e36b4a02-2b71-4e70-b890-e91695ec53f3\") " Dec 03 12:18:27 crc kubenswrapper[5052]: I1203 12:18:27.378437 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/e36b4a02-2b71-4e70-b890-e91695ec53f3-run-httpd\") pod \"e36b4a02-2b71-4e70-b890-e91695ec53f3\" (UID: \"e36b4a02-2b71-4e70-b890-e91695ec53f3\") " Dec 03 12:18:27 crc kubenswrapper[5052]: I1203 12:18:27.378476 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tllbc\" (UniqueName: \"kubernetes.io/projected/e36b4a02-2b71-4e70-b890-e91695ec53f3-kube-api-access-tllbc\") pod \"e36b4a02-2b71-4e70-b890-e91695ec53f3\" (UID: \"e36b4a02-2b71-4e70-b890-e91695ec53f3\") " Dec 03 12:18:27 crc kubenswrapper[5052]: I1203 12:18:27.378651 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/e36b4a02-2b71-4e70-b890-e91695ec53f3-config-data\") pod \"e36b4a02-2b71-4e70-b890-e91695ec53f3\" (UID: \"e36b4a02-2b71-4e70-b890-e91695ec53f3\") " Dec 03 12:18:27 crc kubenswrapper[5052]: I1203 12:18:27.378677 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e36b4a02-2b71-4e70-b890-e91695ec53f3-combined-ca-bundle\") pod \"e36b4a02-2b71-4e70-b890-e91695ec53f3\" (UID: \"e36b4a02-2b71-4e70-b890-e91695ec53f3\") " Dec 03 12:18:27 crc kubenswrapper[5052]: I1203 12:18:27.380540 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e36b4a02-2b71-4e70-b890-e91695ec53f3-run-httpd" (OuterVolumeSpecName: "run-httpd") pod "e36b4a02-2b71-4e70-b890-e91695ec53f3" (UID: "e36b4a02-2b71-4e70-b890-e91695ec53f3"). InnerVolumeSpecName "run-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:18:27 crc kubenswrapper[5052]: I1203 12:18:27.382058 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e36b4a02-2b71-4e70-b890-e91695ec53f3-log-httpd" (OuterVolumeSpecName: "log-httpd") pod "e36b4a02-2b71-4e70-b890-e91695ec53f3" (UID: "e36b4a02-2b71-4e70-b890-e91695ec53f3"). InnerVolumeSpecName "log-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:18:27 crc kubenswrapper[5052]: I1203 12:18:27.383667 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e36b4a02-2b71-4e70-b890-e91695ec53f3-scripts" (OuterVolumeSpecName: "scripts") pod "e36b4a02-2b71-4e70-b890-e91695ec53f3" (UID: "e36b4a02-2b71-4e70-b890-e91695ec53f3"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:18:27 crc kubenswrapper[5052]: I1203 12:18:27.385709 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e36b4a02-2b71-4e70-b890-e91695ec53f3-kube-api-access-tllbc" (OuterVolumeSpecName: "kube-api-access-tllbc") pod "e36b4a02-2b71-4e70-b890-e91695ec53f3" (UID: "e36b4a02-2b71-4e70-b890-e91695ec53f3"). InnerVolumeSpecName "kube-api-access-tllbc". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:18:27 crc kubenswrapper[5052]: I1203 12:18:27.408848 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e36b4a02-2b71-4e70-b890-e91695ec53f3-sg-core-conf-yaml" (OuterVolumeSpecName: "sg-core-conf-yaml") pod "e36b4a02-2b71-4e70-b890-e91695ec53f3" (UID: "e36b4a02-2b71-4e70-b890-e91695ec53f3"). InnerVolumeSpecName "sg-core-conf-yaml". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:18:27 crc kubenswrapper[5052]: I1203 12:18:27.484184 5052 reconciler_common.go:293] "Volume detached for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/e36b4a02-2b71-4e70-b890-e91695ec53f3-run-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:18:27 crc kubenswrapper[5052]: I1203 12:18:27.484225 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-tllbc\" (UniqueName: \"kubernetes.io/projected/e36b4a02-2b71-4e70-b890-e91695ec53f3-kube-api-access-tllbc\") on node \"crc\" DevicePath \"\"" Dec 03 12:18:27 crc kubenswrapper[5052]: I1203 12:18:27.484237 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/e36b4a02-2b71-4e70-b890-e91695ec53f3-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:18:27 crc kubenswrapper[5052]: I1203 12:18:27.484248 5052 reconciler_common.go:293] "Volume detached for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/e36b4a02-2b71-4e70-b890-e91695ec53f3-sg-core-conf-yaml\") on node \"crc\" DevicePath \"\"" Dec 03 12:18:27 crc kubenswrapper[5052]: I1203 12:18:27.484259 5052 reconciler_common.go:293] "Volume detached for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/e36b4a02-2b71-4e70-b890-e91695ec53f3-log-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:18:27 crc kubenswrapper[5052]: I1203 12:18:27.485105 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e36b4a02-2b71-4e70-b890-e91695ec53f3-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "e36b4a02-2b71-4e70-b890-e91695ec53f3" (UID: "e36b4a02-2b71-4e70-b890-e91695ec53f3"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:18:27 crc kubenswrapper[5052]: I1203 12:18:27.523644 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e36b4a02-2b71-4e70-b890-e91695ec53f3-config-data" (OuterVolumeSpecName: "config-data") pod "e36b4a02-2b71-4e70-b890-e91695ec53f3" (UID: "e36b4a02-2b71-4e70-b890-e91695ec53f3"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:18:27 crc kubenswrapper[5052]: I1203 12:18:27.586357 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/e36b4a02-2b71-4e70-b890-e91695ec53f3-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:18:27 crc kubenswrapper[5052]: I1203 12:18:27.586638 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e36b4a02-2b71-4e70-b890-e91695ec53f3-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:18:27 crc kubenswrapper[5052]: I1203 12:18:27.810685 5052 generic.go:334] "Generic (PLEG): container finished" podID="e36b4a02-2b71-4e70-b890-e91695ec53f3" containerID="ec4c6e5477ad824e01b795f69608fed547c19bcae844fe2fa93dbe97916d866e" exitCode=0 Dec 03 12:18:27 crc kubenswrapper[5052]: I1203 12:18:27.810750 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"e36b4a02-2b71-4e70-b890-e91695ec53f3","Type":"ContainerDied","Data":"ec4c6e5477ad824e01b795f69608fed547c19bcae844fe2fa93dbe97916d866e"} Dec 03 12:18:27 crc kubenswrapper[5052]: I1203 12:18:27.810824 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"e36b4a02-2b71-4e70-b890-e91695ec53f3","Type":"ContainerDied","Data":"ce923a9f194e70589709036bf9f1201218ffb9db8b0b58c984d5c96a3014fb5b"} Dec 03 12:18:27 crc kubenswrapper[5052]: I1203 12:18:27.810844 5052 scope.go:117] "RemoveContainer" containerID="6d1dc86dca302c34d13335215a72604630127727141b8b1de8e5259bd7e5f5fe" Dec 03 12:18:27 crc kubenswrapper[5052]: I1203 12:18:27.811127 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:18:27 crc kubenswrapper[5052]: I1203 12:18:27.822176 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-proxy-d4747d79f-wtbhv" event={"ID":"7e109538-a2d2-43bb-8cef-bf99d7d0f1e9","Type":"ContainerStarted","Data":"ef9fabe8451cd9b904301777198ce63bf6cefd87c577b4cad8746d28b098af4a"} Dec 03 12:18:27 crc kubenswrapper[5052]: I1203 12:18:27.822221 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-proxy-d4747d79f-wtbhv" event={"ID":"7e109538-a2d2-43bb-8cef-bf99d7d0f1e9","Type":"ContainerStarted","Data":"17114cefe25d8670369fc24018b4320e72bd1b81aa69b034a694652ddb010caa"} Dec 03 12:18:27 crc kubenswrapper[5052]: I1203 12:18:27.822231 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-proxy-d4747d79f-wtbhv" event={"ID":"7e109538-a2d2-43bb-8cef-bf99d7d0f1e9","Type":"ContainerStarted","Data":"c16bc0acd0b2a80765797511bb2ab0ad881fee07b3ee755104460e0f97a88f66"} Dec 03 12:18:27 crc kubenswrapper[5052]: I1203 12:18:27.822454 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/swift-proxy-d4747d79f-wtbhv" Dec 03 12:18:27 crc kubenswrapper[5052]: I1203 12:18:27.822658 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/swift-proxy-d4747d79f-wtbhv" Dec 03 12:18:27 crc kubenswrapper[5052]: I1203 12:18:27.845070 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:18:27 crc kubenswrapper[5052]: I1203 12:18:27.851702 5052 scope.go:117] "RemoveContainer" containerID="f6833798ca88a77a606fd1470b8377553dfe1ed45d731e4a1f9fce2597d061a8" Dec 03 12:18:27 crc kubenswrapper[5052]: I1203 12:18:27.855827 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:18:27 crc kubenswrapper[5052]: I1203 12:18:27.865439 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:18:27 crc kubenswrapper[5052]: E1203 12:18:27.870026 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="e36b4a02-2b71-4e70-b890-e91695ec53f3" containerName="ceilometer-central-agent" Dec 03 12:18:27 crc kubenswrapper[5052]: I1203 12:18:27.870090 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="e36b4a02-2b71-4e70-b890-e91695ec53f3" containerName="ceilometer-central-agent" Dec 03 12:18:27 crc kubenswrapper[5052]: E1203 12:18:27.870166 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="e36b4a02-2b71-4e70-b890-e91695ec53f3" containerName="proxy-httpd" Dec 03 12:18:27 crc kubenswrapper[5052]: I1203 12:18:27.870174 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="e36b4a02-2b71-4e70-b890-e91695ec53f3" containerName="proxy-httpd" Dec 03 12:18:27 crc kubenswrapper[5052]: E1203 12:18:27.870194 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="e36b4a02-2b71-4e70-b890-e91695ec53f3" containerName="ceilometer-notification-agent" Dec 03 12:18:27 crc kubenswrapper[5052]: I1203 12:18:27.870230 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="e36b4a02-2b71-4e70-b890-e91695ec53f3" containerName="ceilometer-notification-agent" Dec 03 12:18:27 crc kubenswrapper[5052]: E1203 12:18:27.870250 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="e36b4a02-2b71-4e70-b890-e91695ec53f3" containerName="sg-core" Dec 03 12:18:27 crc kubenswrapper[5052]: I1203 12:18:27.870255 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="e36b4a02-2b71-4e70-b890-e91695ec53f3" containerName="sg-core" Dec 03 12:18:27 crc kubenswrapper[5052]: I1203 12:18:27.870661 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="e36b4a02-2b71-4e70-b890-e91695ec53f3" containerName="ceilometer-notification-agent" Dec 03 12:18:27 crc kubenswrapper[5052]: I1203 12:18:27.870719 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="e36b4a02-2b71-4e70-b890-e91695ec53f3" containerName="ceilometer-central-agent" Dec 03 12:18:27 crc kubenswrapper[5052]: I1203 12:18:27.870741 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="e36b4a02-2b71-4e70-b890-e91695ec53f3" containerName="sg-core" Dec 03 12:18:27 crc kubenswrapper[5052]: I1203 12:18:27.870752 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="e36b4a02-2b71-4e70-b890-e91695ec53f3" containerName="proxy-httpd" Dec 03 12:18:27 crc kubenswrapper[5052]: I1203 12:18:27.875440 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:18:27 crc kubenswrapper[5052]: I1203 12:18:27.884224 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-config-data" Dec 03 12:18:27 crc kubenswrapper[5052]: I1203 12:18:27.884725 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-scripts" Dec 03 12:18:27 crc kubenswrapper[5052]: I1203 12:18:27.886325 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/swift-proxy-d4747d79f-wtbhv" podStartSLOduration=1.8863045760000001 podStartE2EDuration="1.886304576s" podCreationTimestamp="2025-12-03 12:18:26 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:18:27.884172965 +0000 UTC m=+3498.472414749" watchObservedRunningTime="2025-12-03 12:18:27.886304576 +0000 UTC m=+3498.474546350" Dec 03 12:18:27 crc kubenswrapper[5052]: I1203 12:18:27.916848 5052 scope.go:117] "RemoveContainer" containerID="ec4c6e5477ad824e01b795f69608fed547c19bcae844fe2fa93dbe97916d866e" Dec 03 12:18:27 crc kubenswrapper[5052]: I1203 12:18:27.922791 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:18:27 crc kubenswrapper[5052]: I1203 12:18:27.946787 5052 scope.go:117] "RemoveContainer" containerID="7ac693835aaf1e9e855735211f4334f714699a1693c265e3b741d201b9e78b8f" Dec 03 12:18:27 crc kubenswrapper[5052]: I1203 12:18:27.978196 5052 scope.go:117] "RemoveContainer" containerID="6d1dc86dca302c34d13335215a72604630127727141b8b1de8e5259bd7e5f5fe" Dec 03 12:18:27 crc kubenswrapper[5052]: E1203 12:18:27.978752 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6d1dc86dca302c34d13335215a72604630127727141b8b1de8e5259bd7e5f5fe\": container with ID starting with 6d1dc86dca302c34d13335215a72604630127727141b8b1de8e5259bd7e5f5fe not found: ID does not exist" containerID="6d1dc86dca302c34d13335215a72604630127727141b8b1de8e5259bd7e5f5fe" Dec 03 12:18:27 crc kubenswrapper[5052]: I1203 12:18:27.978808 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6d1dc86dca302c34d13335215a72604630127727141b8b1de8e5259bd7e5f5fe"} err="failed to get container status \"6d1dc86dca302c34d13335215a72604630127727141b8b1de8e5259bd7e5f5fe\": rpc error: code = NotFound desc = could not find container \"6d1dc86dca302c34d13335215a72604630127727141b8b1de8e5259bd7e5f5fe\": container with ID starting with 6d1dc86dca302c34d13335215a72604630127727141b8b1de8e5259bd7e5f5fe not found: ID does not exist" Dec 03 12:18:27 crc kubenswrapper[5052]: I1203 12:18:27.978843 5052 scope.go:117] "RemoveContainer" containerID="f6833798ca88a77a606fd1470b8377553dfe1ed45d731e4a1f9fce2597d061a8" Dec 03 12:18:27 crc kubenswrapper[5052]: E1203 12:18:27.982633 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f6833798ca88a77a606fd1470b8377553dfe1ed45d731e4a1f9fce2597d061a8\": container with ID starting with f6833798ca88a77a606fd1470b8377553dfe1ed45d731e4a1f9fce2597d061a8 not found: ID does not exist" containerID="f6833798ca88a77a606fd1470b8377553dfe1ed45d731e4a1f9fce2597d061a8" Dec 03 12:18:27 crc kubenswrapper[5052]: I1203 12:18:27.982674 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f6833798ca88a77a606fd1470b8377553dfe1ed45d731e4a1f9fce2597d061a8"} err="failed to get container status \"f6833798ca88a77a606fd1470b8377553dfe1ed45d731e4a1f9fce2597d061a8\": rpc error: code = NotFound desc = could not find container \"f6833798ca88a77a606fd1470b8377553dfe1ed45d731e4a1f9fce2597d061a8\": container with ID starting with f6833798ca88a77a606fd1470b8377553dfe1ed45d731e4a1f9fce2597d061a8 not found: ID does not exist" Dec 03 12:18:27 crc kubenswrapper[5052]: I1203 12:18:27.982713 5052 scope.go:117] "RemoveContainer" containerID="ec4c6e5477ad824e01b795f69608fed547c19bcae844fe2fa93dbe97916d866e" Dec 03 12:18:27 crc kubenswrapper[5052]: E1203 12:18:27.983251 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ec4c6e5477ad824e01b795f69608fed547c19bcae844fe2fa93dbe97916d866e\": container with ID starting with ec4c6e5477ad824e01b795f69608fed547c19bcae844fe2fa93dbe97916d866e not found: ID does not exist" containerID="ec4c6e5477ad824e01b795f69608fed547c19bcae844fe2fa93dbe97916d866e" Dec 03 12:18:27 crc kubenswrapper[5052]: I1203 12:18:27.983300 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ec4c6e5477ad824e01b795f69608fed547c19bcae844fe2fa93dbe97916d866e"} err="failed to get container status \"ec4c6e5477ad824e01b795f69608fed547c19bcae844fe2fa93dbe97916d866e\": rpc error: code = NotFound desc = could not find container \"ec4c6e5477ad824e01b795f69608fed547c19bcae844fe2fa93dbe97916d866e\": container with ID starting with ec4c6e5477ad824e01b795f69608fed547c19bcae844fe2fa93dbe97916d866e not found: ID does not exist" Dec 03 12:18:27 crc kubenswrapper[5052]: I1203 12:18:27.983331 5052 scope.go:117] "RemoveContainer" containerID="7ac693835aaf1e9e855735211f4334f714699a1693c265e3b741d201b9e78b8f" Dec 03 12:18:27 crc kubenswrapper[5052]: E1203 12:18:27.983689 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7ac693835aaf1e9e855735211f4334f714699a1693c265e3b741d201b9e78b8f\": container with ID starting with 7ac693835aaf1e9e855735211f4334f714699a1693c265e3b741d201b9e78b8f not found: ID does not exist" containerID="7ac693835aaf1e9e855735211f4334f714699a1693c265e3b741d201b9e78b8f" Dec 03 12:18:27 crc kubenswrapper[5052]: I1203 12:18:27.983741 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7ac693835aaf1e9e855735211f4334f714699a1693c265e3b741d201b9e78b8f"} err="failed to get container status \"7ac693835aaf1e9e855735211f4334f714699a1693c265e3b741d201b9e78b8f\": rpc error: code = NotFound desc = could not find container \"7ac693835aaf1e9e855735211f4334f714699a1693c265e3b741d201b9e78b8f\": container with ID starting with 7ac693835aaf1e9e855735211f4334f714699a1693c265e3b741d201b9e78b8f not found: ID does not exist" Dec 03 12:18:28 crc kubenswrapper[5052]: I1203 12:18:28.007886 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2wh88\" (UniqueName: \"kubernetes.io/projected/d91d7c4a-99fe-4ab8-af6f-43aa89679220-kube-api-access-2wh88\") pod \"ceilometer-0\" (UID: \"d91d7c4a-99fe-4ab8-af6f-43aa89679220\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:18:28 crc kubenswrapper[5052]: I1203 12:18:28.007974 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/d91d7c4a-99fe-4ab8-af6f-43aa89679220-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"d91d7c4a-99fe-4ab8-af6f-43aa89679220\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:18:28 crc kubenswrapper[5052]: I1203 12:18:28.008005 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d91d7c4a-99fe-4ab8-af6f-43aa89679220-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"d91d7c4a-99fe-4ab8-af6f-43aa89679220\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:18:28 crc kubenswrapper[5052]: I1203 12:18:28.008055 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/d91d7c4a-99fe-4ab8-af6f-43aa89679220-scripts\") pod \"ceilometer-0\" (UID: \"d91d7c4a-99fe-4ab8-af6f-43aa89679220\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:18:28 crc kubenswrapper[5052]: I1203 12:18:28.008073 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/d91d7c4a-99fe-4ab8-af6f-43aa89679220-config-data\") pod \"ceilometer-0\" (UID: \"d91d7c4a-99fe-4ab8-af6f-43aa89679220\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:18:28 crc kubenswrapper[5052]: I1203 12:18:28.008087 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/d91d7c4a-99fe-4ab8-af6f-43aa89679220-log-httpd\") pod \"ceilometer-0\" (UID: \"d91d7c4a-99fe-4ab8-af6f-43aa89679220\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:18:28 crc kubenswrapper[5052]: I1203 12:18:28.008103 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/d91d7c4a-99fe-4ab8-af6f-43aa89679220-run-httpd\") pod \"ceilometer-0\" (UID: \"d91d7c4a-99fe-4ab8-af6f-43aa89679220\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:18:28 crc kubenswrapper[5052]: I1203 12:18:28.013806 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:18:28 crc kubenswrapper[5052]: E1203 12:18:28.014586 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[combined-ca-bundle config-data kube-api-access-2wh88 log-httpd run-httpd scripts sg-core-conf-yaml], unattached volumes=[], failed to process volumes=[]: context canceled" pod="openstack-kuttl-tests/ceilometer-0" podUID="d91d7c4a-99fe-4ab8-af6f-43aa89679220" Dec 03 12:18:28 crc kubenswrapper[5052]: I1203 12:18:28.109621 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-2wh88\" (UniqueName: \"kubernetes.io/projected/d91d7c4a-99fe-4ab8-af6f-43aa89679220-kube-api-access-2wh88\") pod \"ceilometer-0\" (UID: \"d91d7c4a-99fe-4ab8-af6f-43aa89679220\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:18:28 crc kubenswrapper[5052]: I1203 12:18:28.109689 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/d91d7c4a-99fe-4ab8-af6f-43aa89679220-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"d91d7c4a-99fe-4ab8-af6f-43aa89679220\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:18:28 crc kubenswrapper[5052]: I1203 12:18:28.109716 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d91d7c4a-99fe-4ab8-af6f-43aa89679220-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"d91d7c4a-99fe-4ab8-af6f-43aa89679220\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:18:28 crc kubenswrapper[5052]: I1203 12:18:28.109752 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/d91d7c4a-99fe-4ab8-af6f-43aa89679220-scripts\") pod \"ceilometer-0\" (UID: \"d91d7c4a-99fe-4ab8-af6f-43aa89679220\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:18:28 crc kubenswrapper[5052]: I1203 12:18:28.109772 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/d91d7c4a-99fe-4ab8-af6f-43aa89679220-config-data\") pod \"ceilometer-0\" (UID: \"d91d7c4a-99fe-4ab8-af6f-43aa89679220\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:18:28 crc kubenswrapper[5052]: I1203 12:18:28.109786 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/d91d7c4a-99fe-4ab8-af6f-43aa89679220-log-httpd\") pod \"ceilometer-0\" (UID: \"d91d7c4a-99fe-4ab8-af6f-43aa89679220\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:18:28 crc kubenswrapper[5052]: I1203 12:18:28.109800 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/d91d7c4a-99fe-4ab8-af6f-43aa89679220-run-httpd\") pod \"ceilometer-0\" (UID: \"d91d7c4a-99fe-4ab8-af6f-43aa89679220\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:18:28 crc kubenswrapper[5052]: I1203 12:18:28.110221 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/d91d7c4a-99fe-4ab8-af6f-43aa89679220-run-httpd\") pod \"ceilometer-0\" (UID: \"d91d7c4a-99fe-4ab8-af6f-43aa89679220\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:18:28 crc kubenswrapper[5052]: I1203 12:18:28.110736 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/d91d7c4a-99fe-4ab8-af6f-43aa89679220-log-httpd\") pod \"ceilometer-0\" (UID: \"d91d7c4a-99fe-4ab8-af6f-43aa89679220\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:18:28 crc kubenswrapper[5052]: I1203 12:18:28.115096 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/d91d7c4a-99fe-4ab8-af6f-43aa89679220-config-data\") pod \"ceilometer-0\" (UID: \"d91d7c4a-99fe-4ab8-af6f-43aa89679220\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:18:28 crc kubenswrapper[5052]: I1203 12:18:28.115181 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d91d7c4a-99fe-4ab8-af6f-43aa89679220-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"d91d7c4a-99fe-4ab8-af6f-43aa89679220\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:18:28 crc kubenswrapper[5052]: I1203 12:18:28.115741 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/d91d7c4a-99fe-4ab8-af6f-43aa89679220-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"d91d7c4a-99fe-4ab8-af6f-43aa89679220\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:18:28 crc kubenswrapper[5052]: I1203 12:18:28.122379 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/d91d7c4a-99fe-4ab8-af6f-43aa89679220-scripts\") pod \"ceilometer-0\" (UID: \"d91d7c4a-99fe-4ab8-af6f-43aa89679220\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:18:28 crc kubenswrapper[5052]: I1203 12:18:28.125719 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-2wh88\" (UniqueName: \"kubernetes.io/projected/d91d7c4a-99fe-4ab8-af6f-43aa89679220-kube-api-access-2wh88\") pod \"ceilometer-0\" (UID: \"d91d7c4a-99fe-4ab8-af6f-43aa89679220\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:18:28 crc kubenswrapper[5052]: I1203 12:18:28.837969 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:18:28 crc kubenswrapper[5052]: I1203 12:18:28.889493 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:18:29 crc kubenswrapper[5052]: I1203 12:18:29.026417 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/d91d7c4a-99fe-4ab8-af6f-43aa89679220-log-httpd\") pod \"d91d7c4a-99fe-4ab8-af6f-43aa89679220\" (UID: \"d91d7c4a-99fe-4ab8-af6f-43aa89679220\") " Dec 03 12:18:29 crc kubenswrapper[5052]: I1203 12:18:29.026479 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/d91d7c4a-99fe-4ab8-af6f-43aa89679220-sg-core-conf-yaml\") pod \"d91d7c4a-99fe-4ab8-af6f-43aa89679220\" (UID: \"d91d7c4a-99fe-4ab8-af6f-43aa89679220\") " Dec 03 12:18:29 crc kubenswrapper[5052]: I1203 12:18:29.026569 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/d91d7c4a-99fe-4ab8-af6f-43aa89679220-config-data\") pod \"d91d7c4a-99fe-4ab8-af6f-43aa89679220\" (UID: \"d91d7c4a-99fe-4ab8-af6f-43aa89679220\") " Dec 03 12:18:29 crc kubenswrapper[5052]: I1203 12:18:29.026600 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/d91d7c4a-99fe-4ab8-af6f-43aa89679220-scripts\") pod \"d91d7c4a-99fe-4ab8-af6f-43aa89679220\" (UID: \"d91d7c4a-99fe-4ab8-af6f-43aa89679220\") " Dec 03 12:18:29 crc kubenswrapper[5052]: I1203 12:18:29.026675 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d91d7c4a-99fe-4ab8-af6f-43aa89679220-combined-ca-bundle\") pod \"d91d7c4a-99fe-4ab8-af6f-43aa89679220\" (UID: \"d91d7c4a-99fe-4ab8-af6f-43aa89679220\") " Dec 03 12:18:29 crc kubenswrapper[5052]: I1203 12:18:29.026745 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2wh88\" (UniqueName: \"kubernetes.io/projected/d91d7c4a-99fe-4ab8-af6f-43aa89679220-kube-api-access-2wh88\") pod \"d91d7c4a-99fe-4ab8-af6f-43aa89679220\" (UID: \"d91d7c4a-99fe-4ab8-af6f-43aa89679220\") " Dec 03 12:18:29 crc kubenswrapper[5052]: I1203 12:18:29.026820 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/d91d7c4a-99fe-4ab8-af6f-43aa89679220-run-httpd\") pod \"d91d7c4a-99fe-4ab8-af6f-43aa89679220\" (UID: \"d91d7c4a-99fe-4ab8-af6f-43aa89679220\") " Dec 03 12:18:29 crc kubenswrapper[5052]: I1203 12:18:29.027400 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d91d7c4a-99fe-4ab8-af6f-43aa89679220-run-httpd" (OuterVolumeSpecName: "run-httpd") pod "d91d7c4a-99fe-4ab8-af6f-43aa89679220" (UID: "d91d7c4a-99fe-4ab8-af6f-43aa89679220"). InnerVolumeSpecName "run-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:18:29 crc kubenswrapper[5052]: I1203 12:18:29.027544 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d91d7c4a-99fe-4ab8-af6f-43aa89679220-log-httpd" (OuterVolumeSpecName: "log-httpd") pod "d91d7c4a-99fe-4ab8-af6f-43aa89679220" (UID: "d91d7c4a-99fe-4ab8-af6f-43aa89679220"). InnerVolumeSpecName "log-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:18:29 crc kubenswrapper[5052]: I1203 12:18:29.040042 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d91d7c4a-99fe-4ab8-af6f-43aa89679220-sg-core-conf-yaml" (OuterVolumeSpecName: "sg-core-conf-yaml") pod "d91d7c4a-99fe-4ab8-af6f-43aa89679220" (UID: "d91d7c4a-99fe-4ab8-af6f-43aa89679220"). InnerVolumeSpecName "sg-core-conf-yaml". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:18:29 crc kubenswrapper[5052]: I1203 12:18:29.040356 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d91d7c4a-99fe-4ab8-af6f-43aa89679220-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "d91d7c4a-99fe-4ab8-af6f-43aa89679220" (UID: "d91d7c4a-99fe-4ab8-af6f-43aa89679220"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:18:29 crc kubenswrapper[5052]: I1203 12:18:29.040651 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d91d7c4a-99fe-4ab8-af6f-43aa89679220-config-data" (OuterVolumeSpecName: "config-data") pod "d91d7c4a-99fe-4ab8-af6f-43aa89679220" (UID: "d91d7c4a-99fe-4ab8-af6f-43aa89679220"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:18:29 crc kubenswrapper[5052]: I1203 12:18:29.044711 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d91d7c4a-99fe-4ab8-af6f-43aa89679220-kube-api-access-2wh88" (OuterVolumeSpecName: "kube-api-access-2wh88") pod "d91d7c4a-99fe-4ab8-af6f-43aa89679220" (UID: "d91d7c4a-99fe-4ab8-af6f-43aa89679220"). InnerVolumeSpecName "kube-api-access-2wh88". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:18:29 crc kubenswrapper[5052]: I1203 12:18:29.051429 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d91d7c4a-99fe-4ab8-af6f-43aa89679220-scripts" (OuterVolumeSpecName: "scripts") pod "d91d7c4a-99fe-4ab8-af6f-43aa89679220" (UID: "d91d7c4a-99fe-4ab8-af6f-43aa89679220"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:18:29 crc kubenswrapper[5052]: I1203 12:18:29.129089 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d91d7c4a-99fe-4ab8-af6f-43aa89679220-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:18:29 crc kubenswrapper[5052]: I1203 12:18:29.129142 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-2wh88\" (UniqueName: \"kubernetes.io/projected/d91d7c4a-99fe-4ab8-af6f-43aa89679220-kube-api-access-2wh88\") on node \"crc\" DevicePath \"\"" Dec 03 12:18:29 crc kubenswrapper[5052]: I1203 12:18:29.129157 5052 reconciler_common.go:293] "Volume detached for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/d91d7c4a-99fe-4ab8-af6f-43aa89679220-run-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:18:29 crc kubenswrapper[5052]: I1203 12:18:29.129170 5052 reconciler_common.go:293] "Volume detached for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/d91d7c4a-99fe-4ab8-af6f-43aa89679220-log-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:18:29 crc kubenswrapper[5052]: I1203 12:18:29.129182 5052 reconciler_common.go:293] "Volume detached for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/d91d7c4a-99fe-4ab8-af6f-43aa89679220-sg-core-conf-yaml\") on node \"crc\" DevicePath \"\"" Dec 03 12:18:29 crc kubenswrapper[5052]: I1203 12:18:29.129193 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/d91d7c4a-99fe-4ab8-af6f-43aa89679220-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:18:29 crc kubenswrapper[5052]: I1203 12:18:29.129204 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/d91d7c4a-99fe-4ab8-af6f-43aa89679220-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:18:29 crc kubenswrapper[5052]: I1203 12:18:29.462086 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/neutron-54579c6749-xzrwm" Dec 03 12:18:29 crc kubenswrapper[5052]: I1203 12:18:29.548175 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/neutron-54c879c996-kc674"] Dec 03 12:18:29 crc kubenswrapper[5052]: I1203 12:18:29.548430 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/neutron-54c879c996-kc674" podUID="d19ca0f1-4df8-41e7-bab7-dd6d03050486" containerName="neutron-api" containerID="cri-o://10244003ad298cfe03dc6285b75d835bfc6b5409e5af2803e4f440d356990d96" gracePeriod=30 Dec 03 12:18:29 crc kubenswrapper[5052]: I1203 12:18:29.548918 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/neutron-54c879c996-kc674" podUID="d19ca0f1-4df8-41e7-bab7-dd6d03050486" containerName="neutron-httpd" containerID="cri-o://36bd66e285f10587a82a74ea7546e511ef627d378d46a797d9f30d0c1d508790" gracePeriod=30 Dec 03 12:18:29 crc kubenswrapper[5052]: I1203 12:18:29.685633 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e36b4a02-2b71-4e70-b890-e91695ec53f3" path="/var/lib/kubelet/pods/e36b4a02-2b71-4e70-b890-e91695ec53f3/volumes" Dec 03 12:18:29 crc kubenswrapper[5052]: I1203 12:18:29.854795 5052 generic.go:334] "Generic (PLEG): container finished" podID="d19ca0f1-4df8-41e7-bab7-dd6d03050486" containerID="36bd66e285f10587a82a74ea7546e511ef627d378d46a797d9f30d0c1d508790" exitCode=0 Dec 03 12:18:29 crc kubenswrapper[5052]: I1203 12:18:29.854891 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:18:29 crc kubenswrapper[5052]: I1203 12:18:29.854907 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-54c879c996-kc674" event={"ID":"d19ca0f1-4df8-41e7-bab7-dd6d03050486","Type":"ContainerDied","Data":"36bd66e285f10587a82a74ea7546e511ef627d378d46a797d9f30d0c1d508790"} Dec 03 12:18:29 crc kubenswrapper[5052]: I1203 12:18:29.909119 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:18:29 crc kubenswrapper[5052]: I1203 12:18:29.925940 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:18:29 crc kubenswrapper[5052]: I1203 12:18:29.953201 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:18:29 crc kubenswrapper[5052]: I1203 12:18:29.956165 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:18:29 crc kubenswrapper[5052]: I1203 12:18:29.957795 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:18:29 crc kubenswrapper[5052]: I1203 12:18:29.959854 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-scripts" Dec 03 12:18:29 crc kubenswrapper[5052]: I1203 12:18:29.960115 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-config-data" Dec 03 12:18:30 crc kubenswrapper[5052]: I1203 12:18:30.051541 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/abb41fad-6d53-475d-b66a-0f7a4d2e7398-log-httpd\") pod \"ceilometer-0\" (UID: \"abb41fad-6d53-475d-b66a-0f7a4d2e7398\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:18:30 crc kubenswrapper[5052]: I1203 12:18:30.051634 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/abb41fad-6d53-475d-b66a-0f7a4d2e7398-config-data\") pod \"ceilometer-0\" (UID: \"abb41fad-6d53-475d-b66a-0f7a4d2e7398\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:18:30 crc kubenswrapper[5052]: I1203 12:18:30.051693 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/abb41fad-6d53-475d-b66a-0f7a4d2e7398-scripts\") pod \"ceilometer-0\" (UID: \"abb41fad-6d53-475d-b66a-0f7a4d2e7398\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:18:30 crc kubenswrapper[5052]: I1203 12:18:30.051718 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/abb41fad-6d53-475d-b66a-0f7a4d2e7398-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"abb41fad-6d53-475d-b66a-0f7a4d2e7398\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:18:30 crc kubenswrapper[5052]: I1203 12:18:30.051741 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/abb41fad-6d53-475d-b66a-0f7a4d2e7398-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"abb41fad-6d53-475d-b66a-0f7a4d2e7398\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:18:30 crc kubenswrapper[5052]: I1203 12:18:30.051773 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/abb41fad-6d53-475d-b66a-0f7a4d2e7398-run-httpd\") pod \"ceilometer-0\" (UID: \"abb41fad-6d53-475d-b66a-0f7a4d2e7398\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:18:30 crc kubenswrapper[5052]: I1203 12:18:30.051826 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hk2lc\" (UniqueName: \"kubernetes.io/projected/abb41fad-6d53-475d-b66a-0f7a4d2e7398-kube-api-access-hk2lc\") pod \"ceilometer-0\" (UID: \"abb41fad-6d53-475d-b66a-0f7a4d2e7398\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:18:30 crc kubenswrapper[5052]: I1203 12:18:30.153617 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/abb41fad-6d53-475d-b66a-0f7a4d2e7398-log-httpd\") pod \"ceilometer-0\" (UID: \"abb41fad-6d53-475d-b66a-0f7a4d2e7398\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:18:30 crc kubenswrapper[5052]: I1203 12:18:30.153709 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/abb41fad-6d53-475d-b66a-0f7a4d2e7398-config-data\") pod \"ceilometer-0\" (UID: \"abb41fad-6d53-475d-b66a-0f7a4d2e7398\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:18:30 crc kubenswrapper[5052]: I1203 12:18:30.153761 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/abb41fad-6d53-475d-b66a-0f7a4d2e7398-scripts\") pod \"ceilometer-0\" (UID: \"abb41fad-6d53-475d-b66a-0f7a4d2e7398\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:18:30 crc kubenswrapper[5052]: I1203 12:18:30.153785 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/abb41fad-6d53-475d-b66a-0f7a4d2e7398-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"abb41fad-6d53-475d-b66a-0f7a4d2e7398\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:18:30 crc kubenswrapper[5052]: I1203 12:18:30.153817 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/abb41fad-6d53-475d-b66a-0f7a4d2e7398-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"abb41fad-6d53-475d-b66a-0f7a4d2e7398\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:18:30 crc kubenswrapper[5052]: I1203 12:18:30.153846 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/abb41fad-6d53-475d-b66a-0f7a4d2e7398-run-httpd\") pod \"ceilometer-0\" (UID: \"abb41fad-6d53-475d-b66a-0f7a4d2e7398\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:18:30 crc kubenswrapper[5052]: I1203 12:18:30.153890 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-hk2lc\" (UniqueName: \"kubernetes.io/projected/abb41fad-6d53-475d-b66a-0f7a4d2e7398-kube-api-access-hk2lc\") pod \"ceilometer-0\" (UID: \"abb41fad-6d53-475d-b66a-0f7a4d2e7398\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:18:30 crc kubenswrapper[5052]: I1203 12:18:30.154754 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/abb41fad-6d53-475d-b66a-0f7a4d2e7398-log-httpd\") pod \"ceilometer-0\" (UID: \"abb41fad-6d53-475d-b66a-0f7a4d2e7398\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:18:30 crc kubenswrapper[5052]: I1203 12:18:30.156696 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/abb41fad-6d53-475d-b66a-0f7a4d2e7398-run-httpd\") pod \"ceilometer-0\" (UID: \"abb41fad-6d53-475d-b66a-0f7a4d2e7398\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:18:30 crc kubenswrapper[5052]: I1203 12:18:30.162661 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/abb41fad-6d53-475d-b66a-0f7a4d2e7398-scripts\") pod \"ceilometer-0\" (UID: \"abb41fad-6d53-475d-b66a-0f7a4d2e7398\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:18:30 crc kubenswrapper[5052]: I1203 12:18:30.163934 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/abb41fad-6d53-475d-b66a-0f7a4d2e7398-config-data\") pod \"ceilometer-0\" (UID: \"abb41fad-6d53-475d-b66a-0f7a4d2e7398\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:18:30 crc kubenswrapper[5052]: I1203 12:18:30.164462 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/abb41fad-6d53-475d-b66a-0f7a4d2e7398-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"abb41fad-6d53-475d-b66a-0f7a4d2e7398\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:18:30 crc kubenswrapper[5052]: I1203 12:18:30.177314 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/abb41fad-6d53-475d-b66a-0f7a4d2e7398-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"abb41fad-6d53-475d-b66a-0f7a4d2e7398\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:18:30 crc kubenswrapper[5052]: I1203 12:18:30.192046 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-hk2lc\" (UniqueName: \"kubernetes.io/projected/abb41fad-6d53-475d-b66a-0f7a4d2e7398-kube-api-access-hk2lc\") pod \"ceilometer-0\" (UID: \"abb41fad-6d53-475d-b66a-0f7a4d2e7398\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:18:30 crc kubenswrapper[5052]: I1203 12:18:30.274789 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:18:30 crc kubenswrapper[5052]: I1203 12:18:30.758067 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:18:30 crc kubenswrapper[5052]: I1203 12:18:30.868086 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"abb41fad-6d53-475d-b66a-0f7a4d2e7398","Type":"ContainerStarted","Data":"9fec43b69a45c59afdfb6def7ba7073e8865907a8fe9902649442b8377a42c0b"} Dec 03 12:18:31 crc kubenswrapper[5052]: I1203 12:18:31.683721 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d91d7c4a-99fe-4ab8-af6f-43aa89679220" path="/var/lib/kubelet/pods/d91d7c4a-99fe-4ab8-af6f-43aa89679220/volumes" Dec 03 12:18:31 crc kubenswrapper[5052]: I1203 12:18:31.879063 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"abb41fad-6d53-475d-b66a-0f7a4d2e7398","Type":"ContainerStarted","Data":"9ca3901fae35712fbd4bcdeebb868762d7c28b079fea9ebe4579eacbbf811496"} Dec 03 12:18:32 crc kubenswrapper[5052]: I1203 12:18:32.428097 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:18:32 crc kubenswrapper[5052]: I1203 12:18:32.429161 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-internal-api-0" podUID="f65abb68-4704-4c08-b1eb-da1ac582a4d8" containerName="glance-log" containerID="cri-o://28597fb29454145069e00cd7412e37a205ba7c5070c6f525254956aa0e9f8ea8" gracePeriod=30 Dec 03 12:18:32 crc kubenswrapper[5052]: I1203 12:18:32.429225 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-internal-api-0" podUID="f65abb68-4704-4c08-b1eb-da1ac582a4d8" containerName="glance-httpd" containerID="cri-o://343ed046f4e6ee0353b2e1c5ae14819d623cdc87eab25cae67ba467f7d93a548" gracePeriod=30 Dec 03 12:18:32 crc kubenswrapper[5052]: I1203 12:18:32.890698 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"abb41fad-6d53-475d-b66a-0f7a4d2e7398","Type":"ContainerStarted","Data":"427897f1d97e18aab7d9841c595373b5e9f2d193755b53cfd7157a780214e315"} Dec 03 12:18:32 crc kubenswrapper[5052]: I1203 12:18:32.893174 5052 generic.go:334] "Generic (PLEG): container finished" podID="f65abb68-4704-4c08-b1eb-da1ac582a4d8" containerID="28597fb29454145069e00cd7412e37a205ba7c5070c6f525254956aa0e9f8ea8" exitCode=143 Dec 03 12:18:32 crc kubenswrapper[5052]: I1203 12:18:32.893212 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"f65abb68-4704-4c08-b1eb-da1ac582a4d8","Type":"ContainerDied","Data":"28597fb29454145069e00cd7412e37a205ba7c5070c6f525254956aa0e9f8ea8"} Dec 03 12:18:33 crc kubenswrapper[5052]: I1203 12:18:33.904398 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"abb41fad-6d53-475d-b66a-0f7a4d2e7398","Type":"ContainerStarted","Data":"4a0df3ab9421105cfa8a9dd1190b05910b9c73c0e595f295afaeb3ff6b493d8c"} Dec 03 12:18:34 crc kubenswrapper[5052]: I1203 12:18:34.615057 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:18:34 crc kubenswrapper[5052]: I1203 12:18:34.913914 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"abb41fad-6d53-475d-b66a-0f7a4d2e7398","Type":"ContainerStarted","Data":"ff4a8d315be0c18a9396d7b65e791846daa6c4cea304826e510a593a7972bdb7"} Dec 03 12:18:34 crc kubenswrapper[5052]: I1203 12:18:34.914245 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:18:34 crc kubenswrapper[5052]: I1203 12:18:34.937391 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/ceilometer-0" podStartSLOduration=2.6109413249999998 podStartE2EDuration="5.937373608s" podCreationTimestamp="2025-12-03 12:18:29 +0000 UTC" firstStartedPulling="2025-12-03 12:18:30.767648065 +0000 UTC m=+3501.355889829" lastFinishedPulling="2025-12-03 12:18:34.094080348 +0000 UTC m=+3504.682322112" observedRunningTime="2025-12-03 12:18:34.929839203 +0000 UTC m=+3505.518080967" watchObservedRunningTime="2025-12-03 12:18:34.937373608 +0000 UTC m=+3505.525615372" Dec 03 12:18:35 crc kubenswrapper[5052]: I1203 12:18:35.425883 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-54c879c996-kc674" Dec 03 12:18:35 crc kubenswrapper[5052]: I1203 12:18:35.554451 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/d19ca0f1-4df8-41e7-bab7-dd6d03050486-config\") pod \"d19ca0f1-4df8-41e7-bab7-dd6d03050486\" (UID: \"d19ca0f1-4df8-41e7-bab7-dd6d03050486\") " Dec 03 12:18:35 crc kubenswrapper[5052]: I1203 12:18:35.554555 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/d19ca0f1-4df8-41e7-bab7-dd6d03050486-ovndb-tls-certs\") pod \"d19ca0f1-4df8-41e7-bab7-dd6d03050486\" (UID: \"d19ca0f1-4df8-41e7-bab7-dd6d03050486\") " Dec 03 12:18:35 crc kubenswrapper[5052]: I1203 12:18:35.554662 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/d19ca0f1-4df8-41e7-bab7-dd6d03050486-httpd-config\") pod \"d19ca0f1-4df8-41e7-bab7-dd6d03050486\" (UID: \"d19ca0f1-4df8-41e7-bab7-dd6d03050486\") " Dec 03 12:18:35 crc kubenswrapper[5052]: I1203 12:18:35.554729 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d19ca0f1-4df8-41e7-bab7-dd6d03050486-combined-ca-bundle\") pod \"d19ca0f1-4df8-41e7-bab7-dd6d03050486\" (UID: \"d19ca0f1-4df8-41e7-bab7-dd6d03050486\") " Dec 03 12:18:35 crc kubenswrapper[5052]: I1203 12:18:35.554766 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mkqtz\" (UniqueName: \"kubernetes.io/projected/d19ca0f1-4df8-41e7-bab7-dd6d03050486-kube-api-access-mkqtz\") pod \"d19ca0f1-4df8-41e7-bab7-dd6d03050486\" (UID: \"d19ca0f1-4df8-41e7-bab7-dd6d03050486\") " Dec 03 12:18:35 crc kubenswrapper[5052]: I1203 12:18:35.573798 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d19ca0f1-4df8-41e7-bab7-dd6d03050486-kube-api-access-mkqtz" (OuterVolumeSpecName: "kube-api-access-mkqtz") pod "d19ca0f1-4df8-41e7-bab7-dd6d03050486" (UID: "d19ca0f1-4df8-41e7-bab7-dd6d03050486"). InnerVolumeSpecName "kube-api-access-mkqtz". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:18:35 crc kubenswrapper[5052]: I1203 12:18:35.574313 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d19ca0f1-4df8-41e7-bab7-dd6d03050486-httpd-config" (OuterVolumeSpecName: "httpd-config") pod "d19ca0f1-4df8-41e7-bab7-dd6d03050486" (UID: "d19ca0f1-4df8-41e7-bab7-dd6d03050486"). InnerVolumeSpecName "httpd-config". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:18:35 crc kubenswrapper[5052]: I1203 12:18:35.629013 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d19ca0f1-4df8-41e7-bab7-dd6d03050486-config" (OuterVolumeSpecName: "config") pod "d19ca0f1-4df8-41e7-bab7-dd6d03050486" (UID: "d19ca0f1-4df8-41e7-bab7-dd6d03050486"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:18:35 crc kubenswrapper[5052]: I1203 12:18:35.634313 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d19ca0f1-4df8-41e7-bab7-dd6d03050486-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "d19ca0f1-4df8-41e7-bab7-dd6d03050486" (UID: "d19ca0f1-4df8-41e7-bab7-dd6d03050486"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:18:35 crc kubenswrapper[5052]: I1203 12:18:35.658140 5052 reconciler_common.go:293] "Volume detached for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/d19ca0f1-4df8-41e7-bab7-dd6d03050486-httpd-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:18:35 crc kubenswrapper[5052]: I1203 12:18:35.658181 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d19ca0f1-4df8-41e7-bab7-dd6d03050486-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:18:35 crc kubenswrapper[5052]: I1203 12:18:35.658196 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-mkqtz\" (UniqueName: \"kubernetes.io/projected/d19ca0f1-4df8-41e7-bab7-dd6d03050486-kube-api-access-mkqtz\") on node \"crc\" DevicePath \"\"" Dec 03 12:18:35 crc kubenswrapper[5052]: I1203 12:18:35.658210 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/d19ca0f1-4df8-41e7-bab7-dd6d03050486-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:18:35 crc kubenswrapper[5052]: I1203 12:18:35.667562 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d19ca0f1-4df8-41e7-bab7-dd6d03050486-ovndb-tls-certs" (OuterVolumeSpecName: "ovndb-tls-certs") pod "d19ca0f1-4df8-41e7-bab7-dd6d03050486" (UID: "d19ca0f1-4df8-41e7-bab7-dd6d03050486"). InnerVolumeSpecName "ovndb-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:18:35 crc kubenswrapper[5052]: I1203 12:18:35.760984 5052 reconciler_common.go:293] "Volume detached for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/d19ca0f1-4df8-41e7-bab7-dd6d03050486-ovndb-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:18:35 crc kubenswrapper[5052]: I1203 12:18:35.932643 5052 generic.go:334] "Generic (PLEG): container finished" podID="d19ca0f1-4df8-41e7-bab7-dd6d03050486" containerID="10244003ad298cfe03dc6285b75d835bfc6b5409e5af2803e4f440d356990d96" exitCode=0 Dec 03 12:18:35 crc kubenswrapper[5052]: I1203 12:18:35.932715 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-54c879c996-kc674" event={"ID":"d19ca0f1-4df8-41e7-bab7-dd6d03050486","Type":"ContainerDied","Data":"10244003ad298cfe03dc6285b75d835bfc6b5409e5af2803e4f440d356990d96"} Dec 03 12:18:35 crc kubenswrapper[5052]: I1203 12:18:35.932744 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-54c879c996-kc674" event={"ID":"d19ca0f1-4df8-41e7-bab7-dd6d03050486","Type":"ContainerDied","Data":"4a5654a157a731272c7f243ca48712c7c12b204dd4c16ee08f96ba5d0490c985"} Dec 03 12:18:35 crc kubenswrapper[5052]: I1203 12:18:35.932762 5052 scope.go:117] "RemoveContainer" containerID="36bd66e285f10587a82a74ea7546e511ef627d378d46a797d9f30d0c1d508790" Dec 03 12:18:35 crc kubenswrapper[5052]: I1203 12:18:35.932902 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-54c879c996-kc674" Dec 03 12:18:35 crc kubenswrapper[5052]: I1203 12:18:35.941082 5052 generic.go:334] "Generic (PLEG): container finished" podID="f65abb68-4704-4c08-b1eb-da1ac582a4d8" containerID="343ed046f4e6ee0353b2e1c5ae14819d623cdc87eab25cae67ba467f7d93a548" exitCode=0 Dec 03 12:18:35 crc kubenswrapper[5052]: I1203 12:18:35.941401 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="abb41fad-6d53-475d-b66a-0f7a4d2e7398" containerName="ceilometer-central-agent" containerID="cri-o://9ca3901fae35712fbd4bcdeebb868762d7c28b079fea9ebe4579eacbbf811496" gracePeriod=30 Dec 03 12:18:35 crc kubenswrapper[5052]: I1203 12:18:35.942083 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"f65abb68-4704-4c08-b1eb-da1ac582a4d8","Type":"ContainerDied","Data":"343ed046f4e6ee0353b2e1c5ae14819d623cdc87eab25cae67ba467f7d93a548"} Dec 03 12:18:35 crc kubenswrapper[5052]: I1203 12:18:35.942155 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="abb41fad-6d53-475d-b66a-0f7a4d2e7398" containerName="proxy-httpd" containerID="cri-o://ff4a8d315be0c18a9396d7b65e791846daa6c4cea304826e510a593a7972bdb7" gracePeriod=30 Dec 03 12:18:35 crc kubenswrapper[5052]: I1203 12:18:35.942206 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="abb41fad-6d53-475d-b66a-0f7a4d2e7398" containerName="sg-core" containerID="cri-o://4a0df3ab9421105cfa8a9dd1190b05910b9c73c0e595f295afaeb3ff6b493d8c" gracePeriod=30 Dec 03 12:18:35 crc kubenswrapper[5052]: I1203 12:18:35.942252 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="abb41fad-6d53-475d-b66a-0f7a4d2e7398" containerName="ceilometer-notification-agent" containerID="cri-o://427897f1d97e18aab7d9841c595373b5e9f2d193755b53cfd7157a780214e315" gracePeriod=30 Dec 03 12:18:35 crc kubenswrapper[5052]: I1203 12:18:35.962174 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/neutron-54c879c996-kc674"] Dec 03 12:18:35 crc kubenswrapper[5052]: I1203 12:18:35.982307 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/neutron-54c879c996-kc674"] Dec 03 12:18:35 crc kubenswrapper[5052]: I1203 12:18:35.982743 5052 scope.go:117] "RemoveContainer" containerID="10244003ad298cfe03dc6285b75d835bfc6b5409e5af2803e4f440d356990d96" Dec 03 12:18:35 crc kubenswrapper[5052]: I1203 12:18:35.998177 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:18:36 crc kubenswrapper[5052]: I1203 12:18:36.010812 5052 scope.go:117] "RemoveContainer" containerID="36bd66e285f10587a82a74ea7546e511ef627d378d46a797d9f30d0c1d508790" Dec 03 12:18:36 crc kubenswrapper[5052]: E1203 12:18:36.013257 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"36bd66e285f10587a82a74ea7546e511ef627d378d46a797d9f30d0c1d508790\": container with ID starting with 36bd66e285f10587a82a74ea7546e511ef627d378d46a797d9f30d0c1d508790 not found: ID does not exist" containerID="36bd66e285f10587a82a74ea7546e511ef627d378d46a797d9f30d0c1d508790" Dec 03 12:18:36 crc kubenswrapper[5052]: I1203 12:18:36.013398 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"36bd66e285f10587a82a74ea7546e511ef627d378d46a797d9f30d0c1d508790"} err="failed to get container status \"36bd66e285f10587a82a74ea7546e511ef627d378d46a797d9f30d0c1d508790\": rpc error: code = NotFound desc = could not find container \"36bd66e285f10587a82a74ea7546e511ef627d378d46a797d9f30d0c1d508790\": container with ID starting with 36bd66e285f10587a82a74ea7546e511ef627d378d46a797d9f30d0c1d508790 not found: ID does not exist" Dec 03 12:18:36 crc kubenswrapper[5052]: I1203 12:18:36.013495 5052 scope.go:117] "RemoveContainer" containerID="10244003ad298cfe03dc6285b75d835bfc6b5409e5af2803e4f440d356990d96" Dec 03 12:18:36 crc kubenswrapper[5052]: E1203 12:18:36.014209 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"10244003ad298cfe03dc6285b75d835bfc6b5409e5af2803e4f440d356990d96\": container with ID starting with 10244003ad298cfe03dc6285b75d835bfc6b5409e5af2803e4f440d356990d96 not found: ID does not exist" containerID="10244003ad298cfe03dc6285b75d835bfc6b5409e5af2803e4f440d356990d96" Dec 03 12:18:36 crc kubenswrapper[5052]: I1203 12:18:36.014243 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"10244003ad298cfe03dc6285b75d835bfc6b5409e5af2803e4f440d356990d96"} err="failed to get container status \"10244003ad298cfe03dc6285b75d835bfc6b5409e5af2803e4f440d356990d96\": rpc error: code = NotFound desc = could not find container \"10244003ad298cfe03dc6285b75d835bfc6b5409e5af2803e4f440d356990d96\": container with ID starting with 10244003ad298cfe03dc6285b75d835bfc6b5409e5af2803e4f440d356990d96 not found: ID does not exist" Dec 03 12:18:36 crc kubenswrapper[5052]: I1203 12:18:36.070193 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/f65abb68-4704-4c08-b1eb-da1ac582a4d8-logs\") pod \"f65abb68-4704-4c08-b1eb-da1ac582a4d8\" (UID: \"f65abb68-4704-4c08-b1eb-da1ac582a4d8\") " Dec 03 12:18:36 crc kubenswrapper[5052]: I1203 12:18:36.070279 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/f65abb68-4704-4c08-b1eb-da1ac582a4d8-internal-tls-certs\") pod \"f65abb68-4704-4c08-b1eb-da1ac582a4d8\" (UID: \"f65abb68-4704-4c08-b1eb-da1ac582a4d8\") " Dec 03 12:18:36 crc kubenswrapper[5052]: I1203 12:18:36.070297 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f65abb68-4704-4c08-b1eb-da1ac582a4d8-combined-ca-bundle\") pod \"f65abb68-4704-4c08-b1eb-da1ac582a4d8\" (UID: \"f65abb68-4704-4c08-b1eb-da1ac582a4d8\") " Dec 03 12:18:36 crc kubenswrapper[5052]: I1203 12:18:36.070334 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"glance\" (UniqueName: \"kubernetes.io/local-volume/local-storage17-crc\") pod \"f65abb68-4704-4c08-b1eb-da1ac582a4d8\" (UID: \"f65abb68-4704-4c08-b1eb-da1ac582a4d8\") " Dec 03 12:18:36 crc kubenswrapper[5052]: I1203 12:18:36.070379 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nlm8s\" (UniqueName: \"kubernetes.io/projected/f65abb68-4704-4c08-b1eb-da1ac582a4d8-kube-api-access-nlm8s\") pod \"f65abb68-4704-4c08-b1eb-da1ac582a4d8\" (UID: \"f65abb68-4704-4c08-b1eb-da1ac582a4d8\") " Dec 03 12:18:36 crc kubenswrapper[5052]: I1203 12:18:36.070410 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/f65abb68-4704-4c08-b1eb-da1ac582a4d8-config-data\") pod \"f65abb68-4704-4c08-b1eb-da1ac582a4d8\" (UID: \"f65abb68-4704-4c08-b1eb-da1ac582a4d8\") " Dec 03 12:18:36 crc kubenswrapper[5052]: I1203 12:18:36.070428 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/f65abb68-4704-4c08-b1eb-da1ac582a4d8-httpd-run\") pod \"f65abb68-4704-4c08-b1eb-da1ac582a4d8\" (UID: \"f65abb68-4704-4c08-b1eb-da1ac582a4d8\") " Dec 03 12:18:36 crc kubenswrapper[5052]: I1203 12:18:36.070485 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/f65abb68-4704-4c08-b1eb-da1ac582a4d8-scripts\") pod \"f65abb68-4704-4c08-b1eb-da1ac582a4d8\" (UID: \"f65abb68-4704-4c08-b1eb-da1ac582a4d8\") " Dec 03 12:18:36 crc kubenswrapper[5052]: I1203 12:18:36.081891 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f65abb68-4704-4c08-b1eb-da1ac582a4d8-httpd-run" (OuterVolumeSpecName: "httpd-run") pod "f65abb68-4704-4c08-b1eb-da1ac582a4d8" (UID: "f65abb68-4704-4c08-b1eb-da1ac582a4d8"). InnerVolumeSpecName "httpd-run". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:18:36 crc kubenswrapper[5052]: I1203 12:18:36.088750 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage17-crc" (OuterVolumeSpecName: "glance") pod "f65abb68-4704-4c08-b1eb-da1ac582a4d8" (UID: "f65abb68-4704-4c08-b1eb-da1ac582a4d8"). InnerVolumeSpecName "local-storage17-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 12:18:36 crc kubenswrapper[5052]: I1203 12:18:36.094662 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f65abb68-4704-4c08-b1eb-da1ac582a4d8-scripts" (OuterVolumeSpecName: "scripts") pod "f65abb68-4704-4c08-b1eb-da1ac582a4d8" (UID: "f65abb68-4704-4c08-b1eb-da1ac582a4d8"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:18:36 crc kubenswrapper[5052]: I1203 12:18:36.095606 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f65abb68-4704-4c08-b1eb-da1ac582a4d8-logs" (OuterVolumeSpecName: "logs") pod "f65abb68-4704-4c08-b1eb-da1ac582a4d8" (UID: "f65abb68-4704-4c08-b1eb-da1ac582a4d8"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:18:36 crc kubenswrapper[5052]: I1203 12:18:36.113927 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f65abb68-4704-4c08-b1eb-da1ac582a4d8-kube-api-access-nlm8s" (OuterVolumeSpecName: "kube-api-access-nlm8s") pod "f65abb68-4704-4c08-b1eb-da1ac582a4d8" (UID: "f65abb68-4704-4c08-b1eb-da1ac582a4d8"). InnerVolumeSpecName "kube-api-access-nlm8s". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:18:36 crc kubenswrapper[5052]: I1203 12:18:36.149002 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f65abb68-4704-4c08-b1eb-da1ac582a4d8-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "f65abb68-4704-4c08-b1eb-da1ac582a4d8" (UID: "f65abb68-4704-4c08-b1eb-da1ac582a4d8"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:18:36 crc kubenswrapper[5052]: I1203 12:18:36.173500 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/f65abb68-4704-4c08-b1eb-da1ac582a4d8-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:18:36 crc kubenswrapper[5052]: I1203 12:18:36.173553 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f65abb68-4704-4c08-b1eb-da1ac582a4d8-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:18:36 crc kubenswrapper[5052]: I1203 12:18:36.173603 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage17-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage17-crc\") on node \"crc\" " Dec 03 12:18:36 crc kubenswrapper[5052]: I1203 12:18:36.173639 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-nlm8s\" (UniqueName: \"kubernetes.io/projected/f65abb68-4704-4c08-b1eb-da1ac582a4d8-kube-api-access-nlm8s\") on node \"crc\" DevicePath \"\"" Dec 03 12:18:36 crc kubenswrapper[5052]: I1203 12:18:36.173651 5052 reconciler_common.go:293] "Volume detached for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/f65abb68-4704-4c08-b1eb-da1ac582a4d8-httpd-run\") on node \"crc\" DevicePath \"\"" Dec 03 12:18:36 crc kubenswrapper[5052]: I1203 12:18:36.173661 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/f65abb68-4704-4c08-b1eb-da1ac582a4d8-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:18:36 crc kubenswrapper[5052]: I1203 12:18:36.261876 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f65abb68-4704-4c08-b1eb-da1ac582a4d8-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "f65abb68-4704-4c08-b1eb-da1ac582a4d8" (UID: "f65abb68-4704-4c08-b1eb-da1ac582a4d8"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:18:36 crc kubenswrapper[5052]: I1203 12:18:36.268157 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage17-crc" (UniqueName: "kubernetes.io/local-volume/local-storage17-crc") on node "crc" Dec 03 12:18:36 crc kubenswrapper[5052]: I1203 12:18:36.272147 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f65abb68-4704-4c08-b1eb-da1ac582a4d8-config-data" (OuterVolumeSpecName: "config-data") pod "f65abb68-4704-4c08-b1eb-da1ac582a4d8" (UID: "f65abb68-4704-4c08-b1eb-da1ac582a4d8"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:18:36 crc kubenswrapper[5052]: I1203 12:18:36.277487 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/f65abb68-4704-4c08-b1eb-da1ac582a4d8-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:18:36 crc kubenswrapper[5052]: I1203 12:18:36.277557 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage17-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage17-crc\") on node \"crc\" DevicePath \"\"" Dec 03 12:18:36 crc kubenswrapper[5052]: I1203 12:18:36.277567 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/f65abb68-4704-4c08-b1eb-da1ac582a4d8-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:18:36 crc kubenswrapper[5052]: I1203 12:18:36.590087 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/swift-proxy-d4747d79f-wtbhv" Dec 03 12:18:36 crc kubenswrapper[5052]: I1203 12:18:36.594083 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/swift-proxy-d4747d79f-wtbhv" Dec 03 12:18:36 crc kubenswrapper[5052]: I1203 12:18:36.954318 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"f65abb68-4704-4c08-b1eb-da1ac582a4d8","Type":"ContainerDied","Data":"e52bb54c85721046eec5069fe89bc4080b3807de715a2779ff1e00ebf84e55df"} Dec 03 12:18:36 crc kubenswrapper[5052]: I1203 12:18:36.954354 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:18:36 crc kubenswrapper[5052]: I1203 12:18:36.955117 5052 scope.go:117] "RemoveContainer" containerID="343ed046f4e6ee0353b2e1c5ae14819d623cdc87eab25cae67ba467f7d93a548" Dec 03 12:18:36 crc kubenswrapper[5052]: I1203 12:18:36.961183 5052 generic.go:334] "Generic (PLEG): container finished" podID="abb41fad-6d53-475d-b66a-0f7a4d2e7398" containerID="ff4a8d315be0c18a9396d7b65e791846daa6c4cea304826e510a593a7972bdb7" exitCode=0 Dec 03 12:18:36 crc kubenswrapper[5052]: I1203 12:18:36.961396 5052 generic.go:334] "Generic (PLEG): container finished" podID="abb41fad-6d53-475d-b66a-0f7a4d2e7398" containerID="4a0df3ab9421105cfa8a9dd1190b05910b9c73c0e595f295afaeb3ff6b493d8c" exitCode=2 Dec 03 12:18:36 crc kubenswrapper[5052]: I1203 12:18:36.961480 5052 generic.go:334] "Generic (PLEG): container finished" podID="abb41fad-6d53-475d-b66a-0f7a4d2e7398" containerID="427897f1d97e18aab7d9841c595373b5e9f2d193755b53cfd7157a780214e315" exitCode=0 Dec 03 12:18:36 crc kubenswrapper[5052]: I1203 12:18:36.961263 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"abb41fad-6d53-475d-b66a-0f7a4d2e7398","Type":"ContainerDied","Data":"ff4a8d315be0c18a9396d7b65e791846daa6c4cea304826e510a593a7972bdb7"} Dec 03 12:18:36 crc kubenswrapper[5052]: I1203 12:18:36.961627 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"abb41fad-6d53-475d-b66a-0f7a4d2e7398","Type":"ContainerDied","Data":"4a0df3ab9421105cfa8a9dd1190b05910b9c73c0e595f295afaeb3ff6b493d8c"} Dec 03 12:18:36 crc kubenswrapper[5052]: I1203 12:18:36.961643 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"abb41fad-6d53-475d-b66a-0f7a4d2e7398","Type":"ContainerDied","Data":"427897f1d97e18aab7d9841c595373b5e9f2d193755b53cfd7157a780214e315"} Dec 03 12:18:36 crc kubenswrapper[5052]: I1203 12:18:36.987722 5052 scope.go:117] "RemoveContainer" containerID="28597fb29454145069e00cd7412e37a205ba7c5070c6f525254956aa0e9f8ea8" Dec 03 12:18:36 crc kubenswrapper[5052]: I1203 12:18:36.998725 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:18:37 crc kubenswrapper[5052]: I1203 12:18:37.011876 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:18:37 crc kubenswrapper[5052]: I1203 12:18:37.031082 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:18:37 crc kubenswrapper[5052]: E1203 12:18:37.031472 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="d19ca0f1-4df8-41e7-bab7-dd6d03050486" containerName="neutron-api" Dec 03 12:18:37 crc kubenswrapper[5052]: I1203 12:18:37.031492 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="d19ca0f1-4df8-41e7-bab7-dd6d03050486" containerName="neutron-api" Dec 03 12:18:37 crc kubenswrapper[5052]: E1203 12:18:37.031715 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="d19ca0f1-4df8-41e7-bab7-dd6d03050486" containerName="neutron-httpd" Dec 03 12:18:37 crc kubenswrapper[5052]: I1203 12:18:37.031744 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="d19ca0f1-4df8-41e7-bab7-dd6d03050486" containerName="neutron-httpd" Dec 03 12:18:37 crc kubenswrapper[5052]: E1203 12:18:37.031766 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="f65abb68-4704-4c08-b1eb-da1ac582a4d8" containerName="glance-httpd" Dec 03 12:18:37 crc kubenswrapper[5052]: I1203 12:18:37.031774 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="f65abb68-4704-4c08-b1eb-da1ac582a4d8" containerName="glance-httpd" Dec 03 12:18:37 crc kubenswrapper[5052]: E1203 12:18:37.031804 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="f65abb68-4704-4c08-b1eb-da1ac582a4d8" containerName="glance-log" Dec 03 12:18:37 crc kubenswrapper[5052]: I1203 12:18:37.031813 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="f65abb68-4704-4c08-b1eb-da1ac582a4d8" containerName="glance-log" Dec 03 12:18:37 crc kubenswrapper[5052]: I1203 12:18:37.032029 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="d19ca0f1-4df8-41e7-bab7-dd6d03050486" containerName="neutron-httpd" Dec 03 12:18:37 crc kubenswrapper[5052]: I1203 12:18:37.032055 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="f65abb68-4704-4c08-b1eb-da1ac582a4d8" containerName="glance-log" Dec 03 12:18:37 crc kubenswrapper[5052]: I1203 12:18:37.032070 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="f65abb68-4704-4c08-b1eb-da1ac582a4d8" containerName="glance-httpd" Dec 03 12:18:37 crc kubenswrapper[5052]: I1203 12:18:37.032087 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="d19ca0f1-4df8-41e7-bab7-dd6d03050486" containerName="neutron-api" Dec 03 12:18:37 crc kubenswrapper[5052]: I1203 12:18:37.033258 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:18:37 crc kubenswrapper[5052]: I1203 12:18:37.040350 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-default-internal-config-data" Dec 03 12:18:37 crc kubenswrapper[5052]: I1203 12:18:37.040822 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-glance-default-internal-svc" Dec 03 12:18:37 crc kubenswrapper[5052]: I1203 12:18:37.046279 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:18:37 crc kubenswrapper[5052]: I1203 12:18:37.089853 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/941d2a2e-d51f-4b67-9b4a-6ff075a2eb0f-config-data\") pod \"glance-default-internal-api-0\" (UID: \"941d2a2e-d51f-4b67-9b4a-6ff075a2eb0f\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:18:37 crc kubenswrapper[5052]: I1203 12:18:37.089914 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/941d2a2e-d51f-4b67-9b4a-6ff075a2eb0f-httpd-run\") pod \"glance-default-internal-api-0\" (UID: \"941d2a2e-d51f-4b67-9b4a-6ff075a2eb0f\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:18:37 crc kubenswrapper[5052]: I1203 12:18:37.089997 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/941d2a2e-d51f-4b67-9b4a-6ff075a2eb0f-combined-ca-bundle\") pod \"glance-default-internal-api-0\" (UID: \"941d2a2e-d51f-4b67-9b4a-6ff075a2eb0f\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:18:37 crc kubenswrapper[5052]: I1203 12:18:37.090037 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qt7nf\" (UniqueName: \"kubernetes.io/projected/941d2a2e-d51f-4b67-9b4a-6ff075a2eb0f-kube-api-access-qt7nf\") pod \"glance-default-internal-api-0\" (UID: \"941d2a2e-d51f-4b67-9b4a-6ff075a2eb0f\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:18:37 crc kubenswrapper[5052]: I1203 12:18:37.090090 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage17-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage17-crc\") pod \"glance-default-internal-api-0\" (UID: \"941d2a2e-d51f-4b67-9b4a-6ff075a2eb0f\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:18:37 crc kubenswrapper[5052]: I1203 12:18:37.090129 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/941d2a2e-d51f-4b67-9b4a-6ff075a2eb0f-scripts\") pod \"glance-default-internal-api-0\" (UID: \"941d2a2e-d51f-4b67-9b4a-6ff075a2eb0f\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:18:37 crc kubenswrapper[5052]: I1203 12:18:37.090191 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/941d2a2e-d51f-4b67-9b4a-6ff075a2eb0f-logs\") pod \"glance-default-internal-api-0\" (UID: \"941d2a2e-d51f-4b67-9b4a-6ff075a2eb0f\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:18:37 crc kubenswrapper[5052]: I1203 12:18:37.090221 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/941d2a2e-d51f-4b67-9b4a-6ff075a2eb0f-internal-tls-certs\") pod \"glance-default-internal-api-0\" (UID: \"941d2a2e-d51f-4b67-9b4a-6ff075a2eb0f\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:18:37 crc kubenswrapper[5052]: I1203 12:18:37.191530 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/941d2a2e-d51f-4b67-9b4a-6ff075a2eb0f-scripts\") pod \"glance-default-internal-api-0\" (UID: \"941d2a2e-d51f-4b67-9b4a-6ff075a2eb0f\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:18:37 crc kubenswrapper[5052]: I1203 12:18:37.191590 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/941d2a2e-d51f-4b67-9b4a-6ff075a2eb0f-logs\") pod \"glance-default-internal-api-0\" (UID: \"941d2a2e-d51f-4b67-9b4a-6ff075a2eb0f\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:18:37 crc kubenswrapper[5052]: I1203 12:18:37.191621 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/941d2a2e-d51f-4b67-9b4a-6ff075a2eb0f-internal-tls-certs\") pod \"glance-default-internal-api-0\" (UID: \"941d2a2e-d51f-4b67-9b4a-6ff075a2eb0f\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:18:37 crc kubenswrapper[5052]: I1203 12:18:37.191654 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/941d2a2e-d51f-4b67-9b4a-6ff075a2eb0f-config-data\") pod \"glance-default-internal-api-0\" (UID: \"941d2a2e-d51f-4b67-9b4a-6ff075a2eb0f\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:18:37 crc kubenswrapper[5052]: I1203 12:18:37.191690 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/941d2a2e-d51f-4b67-9b4a-6ff075a2eb0f-httpd-run\") pod \"glance-default-internal-api-0\" (UID: \"941d2a2e-d51f-4b67-9b4a-6ff075a2eb0f\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:18:37 crc kubenswrapper[5052]: I1203 12:18:37.191726 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/941d2a2e-d51f-4b67-9b4a-6ff075a2eb0f-combined-ca-bundle\") pod \"glance-default-internal-api-0\" (UID: \"941d2a2e-d51f-4b67-9b4a-6ff075a2eb0f\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:18:37 crc kubenswrapper[5052]: I1203 12:18:37.191751 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-qt7nf\" (UniqueName: \"kubernetes.io/projected/941d2a2e-d51f-4b67-9b4a-6ff075a2eb0f-kube-api-access-qt7nf\") pod \"glance-default-internal-api-0\" (UID: \"941d2a2e-d51f-4b67-9b4a-6ff075a2eb0f\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:18:37 crc kubenswrapper[5052]: I1203 12:18:37.191779 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage17-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage17-crc\") pod \"glance-default-internal-api-0\" (UID: \"941d2a2e-d51f-4b67-9b4a-6ff075a2eb0f\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:18:37 crc kubenswrapper[5052]: I1203 12:18:37.192002 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage17-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage17-crc\") pod \"glance-default-internal-api-0\" (UID: \"941d2a2e-d51f-4b67-9b4a-6ff075a2eb0f\") device mount path \"/mnt/openstack/pv17\"" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:18:37 crc kubenswrapper[5052]: I1203 12:18:37.196568 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/941d2a2e-d51f-4b67-9b4a-6ff075a2eb0f-logs\") pod \"glance-default-internal-api-0\" (UID: \"941d2a2e-d51f-4b67-9b4a-6ff075a2eb0f\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:18:37 crc kubenswrapper[5052]: I1203 12:18:37.199241 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/941d2a2e-d51f-4b67-9b4a-6ff075a2eb0f-httpd-run\") pod \"glance-default-internal-api-0\" (UID: \"941d2a2e-d51f-4b67-9b4a-6ff075a2eb0f\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:18:37 crc kubenswrapper[5052]: I1203 12:18:37.205166 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/941d2a2e-d51f-4b67-9b4a-6ff075a2eb0f-internal-tls-certs\") pod \"glance-default-internal-api-0\" (UID: \"941d2a2e-d51f-4b67-9b4a-6ff075a2eb0f\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:18:37 crc kubenswrapper[5052]: I1203 12:18:37.205208 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/941d2a2e-d51f-4b67-9b4a-6ff075a2eb0f-scripts\") pod \"glance-default-internal-api-0\" (UID: \"941d2a2e-d51f-4b67-9b4a-6ff075a2eb0f\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:18:37 crc kubenswrapper[5052]: I1203 12:18:37.205631 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/941d2a2e-d51f-4b67-9b4a-6ff075a2eb0f-config-data\") pod \"glance-default-internal-api-0\" (UID: \"941d2a2e-d51f-4b67-9b4a-6ff075a2eb0f\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:18:37 crc kubenswrapper[5052]: I1203 12:18:37.210662 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/941d2a2e-d51f-4b67-9b4a-6ff075a2eb0f-combined-ca-bundle\") pod \"glance-default-internal-api-0\" (UID: \"941d2a2e-d51f-4b67-9b4a-6ff075a2eb0f\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:18:37 crc kubenswrapper[5052]: I1203 12:18:37.224942 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-qt7nf\" (UniqueName: \"kubernetes.io/projected/941d2a2e-d51f-4b67-9b4a-6ff075a2eb0f-kube-api-access-qt7nf\") pod \"glance-default-internal-api-0\" (UID: \"941d2a2e-d51f-4b67-9b4a-6ff075a2eb0f\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:18:37 crc kubenswrapper[5052]: I1203 12:18:37.226673 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage17-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage17-crc\") pod \"glance-default-internal-api-0\" (UID: \"941d2a2e-d51f-4b67-9b4a-6ff075a2eb0f\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:18:37 crc kubenswrapper[5052]: I1203 12:18:37.362266 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:18:37 crc kubenswrapper[5052]: I1203 12:18:37.685438 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d19ca0f1-4df8-41e7-bab7-dd6d03050486" path="/var/lib/kubelet/pods/d19ca0f1-4df8-41e7-bab7-dd6d03050486/volumes" Dec 03 12:18:37 crc kubenswrapper[5052]: I1203 12:18:37.686437 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f65abb68-4704-4c08-b1eb-da1ac582a4d8" path="/var/lib/kubelet/pods/f65abb68-4704-4c08-b1eb-da1ac582a4d8/volumes" Dec 03 12:18:37 crc kubenswrapper[5052]: I1203 12:18:37.817675 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:18:37 crc kubenswrapper[5052]: W1203 12:18:37.826993 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod941d2a2e_d51f_4b67_9b4a_6ff075a2eb0f.slice/crio-2be741a0e4b2f878aa142c8486c08a4543804d17a7967a78a28ddef3fb9df795 WatchSource:0}: Error finding container 2be741a0e4b2f878aa142c8486c08a4543804d17a7967a78a28ddef3fb9df795: Status 404 returned error can't find the container with id 2be741a0e4b2f878aa142c8486c08a4543804d17a7967a78a28ddef3fb9df795 Dec 03 12:18:37 crc kubenswrapper[5052]: I1203 12:18:37.978329 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"941d2a2e-d51f-4b67-9b4a-6ff075a2eb0f","Type":"ContainerStarted","Data":"2be741a0e4b2f878aa142c8486c08a4543804d17a7967a78a28ddef3fb9df795"} Dec 03 12:18:38 crc kubenswrapper[5052]: I1203 12:18:38.364028 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-api-db-create-76nlc"] Dec 03 12:18:38 crc kubenswrapper[5052]: I1203 12:18:38.367759 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-db-create-76nlc" Dec 03 12:18:38 crc kubenswrapper[5052]: I1203 12:18:38.377445 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-db-create-76nlc"] Dec 03 12:18:38 crc kubenswrapper[5052]: I1203 12:18:38.412759 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vwfq5\" (UniqueName: \"kubernetes.io/projected/0f021810-dfcd-4695-8330-380dfc297701-kube-api-access-vwfq5\") pod \"nova-api-db-create-76nlc\" (UID: \"0f021810-dfcd-4695-8330-380dfc297701\") " pod="openstack-kuttl-tests/nova-api-db-create-76nlc" Dec 03 12:18:38 crc kubenswrapper[5052]: I1203 12:18:38.412949 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/0f021810-dfcd-4695-8330-380dfc297701-operator-scripts\") pod \"nova-api-db-create-76nlc\" (UID: \"0f021810-dfcd-4695-8330-380dfc297701\") " pod="openstack-kuttl-tests/nova-api-db-create-76nlc" Dec 03 12:18:38 crc kubenswrapper[5052]: I1203 12:18:38.486302 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell0-db-create-2vxt6"] Dec 03 12:18:38 crc kubenswrapper[5052]: I1203 12:18:38.495184 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-db-create-2vxt6" Dec 03 12:18:38 crc kubenswrapper[5052]: I1203 12:18:38.500755 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-api-8f5b-account-create-update-89qfl"] Dec 03 12:18:38 crc kubenswrapper[5052]: I1203 12:18:38.502294 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-8f5b-account-create-update-89qfl" Dec 03 12:18:38 crc kubenswrapper[5052]: I1203 12:18:38.503784 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-api-db-secret" Dec 03 12:18:38 crc kubenswrapper[5052]: I1203 12:18:38.515753 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s8flq\" (UniqueName: \"kubernetes.io/projected/dfbfcab9-41af-42e1-9ddf-5a05ffedb6c7-kube-api-access-s8flq\") pod \"nova-cell0-db-create-2vxt6\" (UID: \"dfbfcab9-41af-42e1-9ddf-5a05ffedb6c7\") " pod="openstack-kuttl-tests/nova-cell0-db-create-2vxt6" Dec 03 12:18:38 crc kubenswrapper[5052]: I1203 12:18:38.515820 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/0f021810-dfcd-4695-8330-380dfc297701-operator-scripts\") pod \"nova-api-db-create-76nlc\" (UID: \"0f021810-dfcd-4695-8330-380dfc297701\") " pod="openstack-kuttl-tests/nova-api-db-create-76nlc" Dec 03 12:18:38 crc kubenswrapper[5052]: I1203 12:18:38.515979 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/dfbfcab9-41af-42e1-9ddf-5a05ffedb6c7-operator-scripts\") pod \"nova-cell0-db-create-2vxt6\" (UID: \"dfbfcab9-41af-42e1-9ddf-5a05ffedb6c7\") " pod="openstack-kuttl-tests/nova-cell0-db-create-2vxt6" Dec 03 12:18:38 crc kubenswrapper[5052]: I1203 12:18:38.516065 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-vwfq5\" (UniqueName: \"kubernetes.io/projected/0f021810-dfcd-4695-8330-380dfc297701-kube-api-access-vwfq5\") pod \"nova-api-db-create-76nlc\" (UID: \"0f021810-dfcd-4695-8330-380dfc297701\") " pod="openstack-kuttl-tests/nova-api-db-create-76nlc" Dec 03 12:18:38 crc kubenswrapper[5052]: I1203 12:18:38.516322 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/03fcb214-234e-4e5a-8d9f-bbd2bbda66aa-operator-scripts\") pod \"nova-api-8f5b-account-create-update-89qfl\" (UID: \"03fcb214-234e-4e5a-8d9f-bbd2bbda66aa\") " pod="openstack-kuttl-tests/nova-api-8f5b-account-create-update-89qfl" Dec 03 12:18:38 crc kubenswrapper[5052]: I1203 12:18:38.516405 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dzrdf\" (UniqueName: \"kubernetes.io/projected/03fcb214-234e-4e5a-8d9f-bbd2bbda66aa-kube-api-access-dzrdf\") pod \"nova-api-8f5b-account-create-update-89qfl\" (UID: \"03fcb214-234e-4e5a-8d9f-bbd2bbda66aa\") " pod="openstack-kuttl-tests/nova-api-8f5b-account-create-update-89qfl" Dec 03 12:18:38 crc kubenswrapper[5052]: I1203 12:18:38.516572 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/0f021810-dfcd-4695-8330-380dfc297701-operator-scripts\") pod \"nova-api-db-create-76nlc\" (UID: \"0f021810-dfcd-4695-8330-380dfc297701\") " pod="openstack-kuttl-tests/nova-api-db-create-76nlc" Dec 03 12:18:38 crc kubenswrapper[5052]: I1203 12:18:38.519792 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell0-db-create-2vxt6"] Dec 03 12:18:38 crc kubenswrapper[5052]: I1203 12:18:38.545193 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-vwfq5\" (UniqueName: \"kubernetes.io/projected/0f021810-dfcd-4695-8330-380dfc297701-kube-api-access-vwfq5\") pod \"nova-api-db-create-76nlc\" (UID: \"0f021810-dfcd-4695-8330-380dfc297701\") " pod="openstack-kuttl-tests/nova-api-db-create-76nlc" Dec 03 12:18:38 crc kubenswrapper[5052]: I1203 12:18:38.567850 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-8f5b-account-create-update-89qfl"] Dec 03 12:18:38 crc kubenswrapper[5052]: I1203 12:18:38.600801 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell1-db-create-tczzp"] Dec 03 12:18:38 crc kubenswrapper[5052]: I1203 12:18:38.602036 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-db-create-tczzp" Dec 03 12:18:38 crc kubenswrapper[5052]: I1203 12:18:38.614671 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-db-create-tczzp"] Dec 03 12:18:38 crc kubenswrapper[5052]: I1203 12:18:38.633928 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/03fcb214-234e-4e5a-8d9f-bbd2bbda66aa-operator-scripts\") pod \"nova-api-8f5b-account-create-update-89qfl\" (UID: \"03fcb214-234e-4e5a-8d9f-bbd2bbda66aa\") " pod="openstack-kuttl-tests/nova-api-8f5b-account-create-update-89qfl" Dec 03 12:18:38 crc kubenswrapper[5052]: I1203 12:18:38.633985 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-dzrdf\" (UniqueName: \"kubernetes.io/projected/03fcb214-234e-4e5a-8d9f-bbd2bbda66aa-kube-api-access-dzrdf\") pod \"nova-api-8f5b-account-create-update-89qfl\" (UID: \"03fcb214-234e-4e5a-8d9f-bbd2bbda66aa\") " pod="openstack-kuttl-tests/nova-api-8f5b-account-create-update-89qfl" Dec 03 12:18:38 crc kubenswrapper[5052]: I1203 12:18:38.634023 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-s8flq\" (UniqueName: \"kubernetes.io/projected/dfbfcab9-41af-42e1-9ddf-5a05ffedb6c7-kube-api-access-s8flq\") pod \"nova-cell0-db-create-2vxt6\" (UID: \"dfbfcab9-41af-42e1-9ddf-5a05ffedb6c7\") " pod="openstack-kuttl-tests/nova-cell0-db-create-2vxt6" Dec 03 12:18:38 crc kubenswrapper[5052]: I1203 12:18:38.634073 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/dfbfcab9-41af-42e1-9ddf-5a05ffedb6c7-operator-scripts\") pod \"nova-cell0-db-create-2vxt6\" (UID: \"dfbfcab9-41af-42e1-9ddf-5a05ffedb6c7\") " pod="openstack-kuttl-tests/nova-cell0-db-create-2vxt6" Dec 03 12:18:38 crc kubenswrapper[5052]: I1203 12:18:38.634756 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/dfbfcab9-41af-42e1-9ddf-5a05ffedb6c7-operator-scripts\") pod \"nova-cell0-db-create-2vxt6\" (UID: \"dfbfcab9-41af-42e1-9ddf-5a05ffedb6c7\") " pod="openstack-kuttl-tests/nova-cell0-db-create-2vxt6" Dec 03 12:18:38 crc kubenswrapper[5052]: I1203 12:18:38.635885 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/03fcb214-234e-4e5a-8d9f-bbd2bbda66aa-operator-scripts\") pod \"nova-api-8f5b-account-create-update-89qfl\" (UID: \"03fcb214-234e-4e5a-8d9f-bbd2bbda66aa\") " pod="openstack-kuttl-tests/nova-api-8f5b-account-create-update-89qfl" Dec 03 12:18:38 crc kubenswrapper[5052]: I1203 12:18:38.658777 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-s8flq\" (UniqueName: \"kubernetes.io/projected/dfbfcab9-41af-42e1-9ddf-5a05ffedb6c7-kube-api-access-s8flq\") pod \"nova-cell0-db-create-2vxt6\" (UID: \"dfbfcab9-41af-42e1-9ddf-5a05ffedb6c7\") " pod="openstack-kuttl-tests/nova-cell0-db-create-2vxt6" Dec 03 12:18:38 crc kubenswrapper[5052]: I1203 12:18:38.659397 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-dzrdf\" (UniqueName: \"kubernetes.io/projected/03fcb214-234e-4e5a-8d9f-bbd2bbda66aa-kube-api-access-dzrdf\") pod \"nova-api-8f5b-account-create-update-89qfl\" (UID: \"03fcb214-234e-4e5a-8d9f-bbd2bbda66aa\") " pod="openstack-kuttl-tests/nova-api-8f5b-account-create-update-89qfl" Dec 03 12:18:38 crc kubenswrapper[5052]: I1203 12:18:38.670089 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-8f5b-account-create-update-89qfl" Dec 03 12:18:38 crc kubenswrapper[5052]: I1203 12:18:38.684170 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-db-create-76nlc" Dec 03 12:18:38 crc kubenswrapper[5052]: I1203 12:18:38.732595 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell0-0e20-account-create-update-5hn8s"] Dec 03 12:18:38 crc kubenswrapper[5052]: I1203 12:18:38.734072 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-0e20-account-create-update-5hn8s" Dec 03 12:18:38 crc kubenswrapper[5052]: I1203 12:18:38.735742 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/c2e338a2-8424-4325-a580-55618dcd3c29-operator-scripts\") pod \"nova-cell1-db-create-tczzp\" (UID: \"c2e338a2-8424-4325-a580-55618dcd3c29\") " pod="openstack-kuttl-tests/nova-cell1-db-create-tczzp" Dec 03 12:18:38 crc kubenswrapper[5052]: I1203 12:18:38.737796 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell0-db-secret" Dec 03 12:18:38 crc kubenswrapper[5052]: I1203 12:18:38.739840 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5l8j8\" (UniqueName: \"kubernetes.io/projected/c2e338a2-8424-4325-a580-55618dcd3c29-kube-api-access-5l8j8\") pod \"nova-cell1-db-create-tczzp\" (UID: \"c2e338a2-8424-4325-a580-55618dcd3c29\") " pod="openstack-kuttl-tests/nova-cell1-db-create-tczzp" Dec 03 12:18:38 crc kubenswrapper[5052]: I1203 12:18:38.748021 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell0-0e20-account-create-update-5hn8s"] Dec 03 12:18:38 crc kubenswrapper[5052]: I1203 12:18:38.844831 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/8d0ffc74-5f47-4195-8091-b4890e55c32f-operator-scripts\") pod \"nova-cell0-0e20-account-create-update-5hn8s\" (UID: \"8d0ffc74-5f47-4195-8091-b4890e55c32f\") " pod="openstack-kuttl-tests/nova-cell0-0e20-account-create-update-5hn8s" Dec 03 12:18:38 crc kubenswrapper[5052]: I1203 12:18:38.845408 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jwvdv\" (UniqueName: \"kubernetes.io/projected/8d0ffc74-5f47-4195-8091-b4890e55c32f-kube-api-access-jwvdv\") pod \"nova-cell0-0e20-account-create-update-5hn8s\" (UID: \"8d0ffc74-5f47-4195-8091-b4890e55c32f\") " pod="openstack-kuttl-tests/nova-cell0-0e20-account-create-update-5hn8s" Dec 03 12:18:38 crc kubenswrapper[5052]: I1203 12:18:38.845477 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/c2e338a2-8424-4325-a580-55618dcd3c29-operator-scripts\") pod \"nova-cell1-db-create-tczzp\" (UID: \"c2e338a2-8424-4325-a580-55618dcd3c29\") " pod="openstack-kuttl-tests/nova-cell1-db-create-tczzp" Dec 03 12:18:38 crc kubenswrapper[5052]: I1203 12:18:38.845802 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-5l8j8\" (UniqueName: \"kubernetes.io/projected/c2e338a2-8424-4325-a580-55618dcd3c29-kube-api-access-5l8j8\") pod \"nova-cell1-db-create-tczzp\" (UID: \"c2e338a2-8424-4325-a580-55618dcd3c29\") " pod="openstack-kuttl-tests/nova-cell1-db-create-tczzp" Dec 03 12:18:38 crc kubenswrapper[5052]: I1203 12:18:38.846616 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/c2e338a2-8424-4325-a580-55618dcd3c29-operator-scripts\") pod \"nova-cell1-db-create-tczzp\" (UID: \"c2e338a2-8424-4325-a580-55618dcd3c29\") " pod="openstack-kuttl-tests/nova-cell1-db-create-tczzp" Dec 03 12:18:38 crc kubenswrapper[5052]: I1203 12:18:38.886707 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-5l8j8\" (UniqueName: \"kubernetes.io/projected/c2e338a2-8424-4325-a580-55618dcd3c29-kube-api-access-5l8j8\") pod \"nova-cell1-db-create-tczzp\" (UID: \"c2e338a2-8424-4325-a580-55618dcd3c29\") " pod="openstack-kuttl-tests/nova-cell1-db-create-tczzp" Dec 03 12:18:38 crc kubenswrapper[5052]: I1203 12:18:38.890595 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell1-3fba-account-create-update-7dlvh"] Dec 03 12:18:38 crc kubenswrapper[5052]: I1203 12:18:38.892062 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-3fba-account-create-update-7dlvh" Dec 03 12:18:38 crc kubenswrapper[5052]: I1203 12:18:38.894420 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell1-db-secret" Dec 03 12:18:38 crc kubenswrapper[5052]: I1203 12:18:38.909205 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-3fba-account-create-update-7dlvh"] Dec 03 12:18:38 crc kubenswrapper[5052]: I1203 12:18:38.946793 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/8d0ffc74-5f47-4195-8091-b4890e55c32f-operator-scripts\") pod \"nova-cell0-0e20-account-create-update-5hn8s\" (UID: \"8d0ffc74-5f47-4195-8091-b4890e55c32f\") " pod="openstack-kuttl-tests/nova-cell0-0e20-account-create-update-5hn8s" Dec 03 12:18:38 crc kubenswrapper[5052]: I1203 12:18:38.946887 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-jwvdv\" (UniqueName: \"kubernetes.io/projected/8d0ffc74-5f47-4195-8091-b4890e55c32f-kube-api-access-jwvdv\") pod \"nova-cell0-0e20-account-create-update-5hn8s\" (UID: \"8d0ffc74-5f47-4195-8091-b4890e55c32f\") " pod="openstack-kuttl-tests/nova-cell0-0e20-account-create-update-5hn8s" Dec 03 12:18:38 crc kubenswrapper[5052]: I1203 12:18:38.946946 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pd6mt\" (UniqueName: \"kubernetes.io/projected/f560f1ba-b0e0-4fb6-9802-bf1c9b726592-kube-api-access-pd6mt\") pod \"nova-cell1-3fba-account-create-update-7dlvh\" (UID: \"f560f1ba-b0e0-4fb6-9802-bf1c9b726592\") " pod="openstack-kuttl-tests/nova-cell1-3fba-account-create-update-7dlvh" Dec 03 12:18:38 crc kubenswrapper[5052]: I1203 12:18:38.946992 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/f560f1ba-b0e0-4fb6-9802-bf1c9b726592-operator-scripts\") pod \"nova-cell1-3fba-account-create-update-7dlvh\" (UID: \"f560f1ba-b0e0-4fb6-9802-bf1c9b726592\") " pod="openstack-kuttl-tests/nova-cell1-3fba-account-create-update-7dlvh" Dec 03 12:18:38 crc kubenswrapper[5052]: I1203 12:18:38.948090 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/8d0ffc74-5f47-4195-8091-b4890e55c32f-operator-scripts\") pod \"nova-cell0-0e20-account-create-update-5hn8s\" (UID: \"8d0ffc74-5f47-4195-8091-b4890e55c32f\") " pod="openstack-kuttl-tests/nova-cell0-0e20-account-create-update-5hn8s" Dec 03 12:18:38 crc kubenswrapper[5052]: I1203 12:18:38.948675 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-db-create-2vxt6" Dec 03 12:18:38 crc kubenswrapper[5052]: I1203 12:18:38.973148 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-jwvdv\" (UniqueName: \"kubernetes.io/projected/8d0ffc74-5f47-4195-8091-b4890e55c32f-kube-api-access-jwvdv\") pod \"nova-cell0-0e20-account-create-update-5hn8s\" (UID: \"8d0ffc74-5f47-4195-8091-b4890e55c32f\") " pod="openstack-kuttl-tests/nova-cell0-0e20-account-create-update-5hn8s" Dec 03 12:18:38 crc kubenswrapper[5052]: I1203 12:18:38.983893 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-db-create-tczzp" Dec 03 12:18:38 crc kubenswrapper[5052]: I1203 12:18:38.995839 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"941d2a2e-d51f-4b67-9b4a-6ff075a2eb0f","Type":"ContainerStarted","Data":"9f17cf626b688bdd38aa8737ff46f9c764ddd68d168ed599986a4ac6dc81223a"} Dec 03 12:18:39 crc kubenswrapper[5052]: I1203 12:18:39.049950 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-pd6mt\" (UniqueName: \"kubernetes.io/projected/f560f1ba-b0e0-4fb6-9802-bf1c9b726592-kube-api-access-pd6mt\") pod \"nova-cell1-3fba-account-create-update-7dlvh\" (UID: \"f560f1ba-b0e0-4fb6-9802-bf1c9b726592\") " pod="openstack-kuttl-tests/nova-cell1-3fba-account-create-update-7dlvh" Dec 03 12:18:39 crc kubenswrapper[5052]: I1203 12:18:39.050063 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/f560f1ba-b0e0-4fb6-9802-bf1c9b726592-operator-scripts\") pod \"nova-cell1-3fba-account-create-update-7dlvh\" (UID: \"f560f1ba-b0e0-4fb6-9802-bf1c9b726592\") " pod="openstack-kuttl-tests/nova-cell1-3fba-account-create-update-7dlvh" Dec 03 12:18:39 crc kubenswrapper[5052]: I1203 12:18:39.051082 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/f560f1ba-b0e0-4fb6-9802-bf1c9b726592-operator-scripts\") pod \"nova-cell1-3fba-account-create-update-7dlvh\" (UID: \"f560f1ba-b0e0-4fb6-9802-bf1c9b726592\") " pod="openstack-kuttl-tests/nova-cell1-3fba-account-create-update-7dlvh" Dec 03 12:18:39 crc kubenswrapper[5052]: I1203 12:18:39.063637 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-0e20-account-create-update-5hn8s" Dec 03 12:18:39 crc kubenswrapper[5052]: I1203 12:18:39.072928 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-pd6mt\" (UniqueName: \"kubernetes.io/projected/f560f1ba-b0e0-4fb6-9802-bf1c9b726592-kube-api-access-pd6mt\") pod \"nova-cell1-3fba-account-create-update-7dlvh\" (UID: \"f560f1ba-b0e0-4fb6-9802-bf1c9b726592\") " pod="openstack-kuttl-tests/nova-cell1-3fba-account-create-update-7dlvh" Dec 03 12:18:39 crc kubenswrapper[5052]: I1203 12:18:39.218858 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-3fba-account-create-update-7dlvh" Dec 03 12:18:39 crc kubenswrapper[5052]: I1203 12:18:39.304682 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-db-create-76nlc"] Dec 03 12:18:39 crc kubenswrapper[5052]: I1203 12:18:39.329660 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-8f5b-account-create-update-89qfl"] Dec 03 12:18:39 crc kubenswrapper[5052]: I1203 12:18:39.553369 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell0-db-create-2vxt6"] Dec 03 12:18:39 crc kubenswrapper[5052]: I1203 12:18:39.619476 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-db-create-tczzp"] Dec 03 12:18:39 crc kubenswrapper[5052]: I1203 12:18:39.734012 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell0-0e20-account-create-update-5hn8s"] Dec 03 12:18:39 crc kubenswrapper[5052]: I1203 12:18:39.852505 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-3fba-account-create-update-7dlvh"] Dec 03 12:18:40 crc kubenswrapper[5052]: I1203 12:18:40.010131 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-db-create-tczzp" event={"ID":"c2e338a2-8424-4325-a580-55618dcd3c29","Type":"ContainerStarted","Data":"a41a6c3524bcee68d48ead6fe403ee2cd332f3a948dfc6434e4cfddf5ded599e"} Dec 03 12:18:40 crc kubenswrapper[5052]: I1203 12:18:40.010193 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-db-create-tczzp" event={"ID":"c2e338a2-8424-4325-a580-55618dcd3c29","Type":"ContainerStarted","Data":"334c3cbe8b938b7b5cb33c3d19eed183bd8a7a667983ac2827c11aa0646a7016"} Dec 03 12:18:40 crc kubenswrapper[5052]: I1203 12:18:40.016261 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-db-create-76nlc" event={"ID":"0f021810-dfcd-4695-8330-380dfc297701","Type":"ContainerStarted","Data":"b9ffc180c65833f54ac4d801441e9df4db9322afb9d77cb2f4775dc868d1a03d"} Dec 03 12:18:40 crc kubenswrapper[5052]: I1203 12:18:40.016310 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-db-create-76nlc" event={"ID":"0f021810-dfcd-4695-8330-380dfc297701","Type":"ContainerStarted","Data":"1c8ce63748029a279f5acdafd57a217ba217362c3b50ed7b7df5b5a4e3ad8199"} Dec 03 12:18:40 crc kubenswrapper[5052]: I1203 12:18:40.019114 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-0e20-account-create-update-5hn8s" event={"ID":"8d0ffc74-5f47-4195-8091-b4890e55c32f","Type":"ContainerStarted","Data":"9b34bcef1b760b4f3f7417ac9222e8595707f6e3124b14e2613276d75db6ce45"} Dec 03 12:18:40 crc kubenswrapper[5052]: I1203 12:18:40.019167 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-0e20-account-create-update-5hn8s" event={"ID":"8d0ffc74-5f47-4195-8091-b4890e55c32f","Type":"ContainerStarted","Data":"c6fa814155ff7b6ba99d54e314f75cdfebb6064aef56c5a95556bc35e1f5bf15"} Dec 03 12:18:40 crc kubenswrapper[5052]: I1203 12:18:40.022424 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-8f5b-account-create-update-89qfl" event={"ID":"03fcb214-234e-4e5a-8d9f-bbd2bbda66aa","Type":"ContainerStarted","Data":"3a2ca9d08031e2ebddb639a9dacfa6094d35ce57e118c4a4e882deff84684149"} Dec 03 12:18:40 crc kubenswrapper[5052]: I1203 12:18:40.022466 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-8f5b-account-create-update-89qfl" event={"ID":"03fcb214-234e-4e5a-8d9f-bbd2bbda66aa","Type":"ContainerStarted","Data":"0bd942fc8effe4e0620e0505e0189496ea2479a523dac707a0b380f8edeedb9e"} Dec 03 12:18:40 crc kubenswrapper[5052]: I1203 12:18:40.029644 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-db-create-2vxt6" event={"ID":"dfbfcab9-41af-42e1-9ddf-5a05ffedb6c7","Type":"ContainerStarted","Data":"6b10b360bd6b52778b72608626846a9f746ffaa71ad3eb4acd15aceb0aa6bf5d"} Dec 03 12:18:40 crc kubenswrapper[5052]: I1203 12:18:40.029695 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-db-create-2vxt6" event={"ID":"dfbfcab9-41af-42e1-9ddf-5a05ffedb6c7","Type":"ContainerStarted","Data":"bc8c9a489e15f709ad4c30887d493242b3303b777dd3d56ec320c387eef889a4"} Dec 03 12:18:40 crc kubenswrapper[5052]: I1203 12:18:40.033229 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-cell1-db-create-tczzp" podStartSLOduration=2.033211034 podStartE2EDuration="2.033211034s" podCreationTimestamp="2025-12-03 12:18:38 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:18:40.027669616 +0000 UTC m=+3510.615911390" watchObservedRunningTime="2025-12-03 12:18:40.033211034 +0000 UTC m=+3510.621452798" Dec 03 12:18:40 crc kubenswrapper[5052]: I1203 12:18:40.033256 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"941d2a2e-d51f-4b67-9b4a-6ff075a2eb0f","Type":"ContainerStarted","Data":"35b431787e68c1ec9b78768c06706d4afad176de2f59bf713fd162cab978b071"} Dec 03 12:18:40 crc kubenswrapper[5052]: I1203 12:18:40.035582 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-3fba-account-create-update-7dlvh" event={"ID":"f560f1ba-b0e0-4fb6-9802-bf1c9b726592","Type":"ContainerStarted","Data":"54f74b5c06bf6339c0008a439dac9d4008cf3b2fce0dc494d7a4273258b5878d"} Dec 03 12:18:40 crc kubenswrapper[5052]: I1203 12:18:40.064082 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-api-db-create-76nlc" podStartSLOduration=2.064054084 podStartE2EDuration="2.064054084s" podCreationTimestamp="2025-12-03 12:18:38 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:18:40.041732577 +0000 UTC m=+3510.629974341" watchObservedRunningTime="2025-12-03 12:18:40.064054084 +0000 UTC m=+3510.652295848" Dec 03 12:18:40 crc kubenswrapper[5052]: I1203 12:18:40.129454 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-cell0-0e20-account-create-update-5hn8s" podStartSLOduration=2.12942839 podStartE2EDuration="2.12942839s" podCreationTimestamp="2025-12-03 12:18:38 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:18:40.075016857 +0000 UTC m=+3510.663258621" watchObservedRunningTime="2025-12-03 12:18:40.12942839 +0000 UTC m=+3510.717670154" Dec 03 12:18:40 crc kubenswrapper[5052]: I1203 12:18:40.135472 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-api-8f5b-account-create-update-89qfl" podStartSLOduration=2.135460492 podStartE2EDuration="2.135460492s" podCreationTimestamp="2025-12-03 12:18:38 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:18:40.09194336 +0000 UTC m=+3510.680185124" watchObservedRunningTime="2025-12-03 12:18:40.135460492 +0000 UTC m=+3510.723702256" Dec 03 12:18:40 crc kubenswrapper[5052]: I1203 12:18:40.161858 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-cell0-db-create-2vxt6" podStartSLOduration=2.1618366350000002 podStartE2EDuration="2.161836635s" podCreationTimestamp="2025-12-03 12:18:38 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:18:40.114286958 +0000 UTC m=+3510.702528722" watchObservedRunningTime="2025-12-03 12:18:40.161836635 +0000 UTC m=+3510.750078399" Dec 03 12:18:40 crc kubenswrapper[5052]: I1203 12:18:40.178081 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/glance-default-internal-api-0" podStartSLOduration=3.178056508 podStartE2EDuration="3.178056508s" podCreationTimestamp="2025-12-03 12:18:37 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:18:40.134308619 +0000 UTC m=+3510.722550383" watchObservedRunningTime="2025-12-03 12:18:40.178056508 +0000 UTC m=+3510.766298272" Dec 03 12:18:40 crc kubenswrapper[5052]: I1203 12:18:40.721188 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:18:40 crc kubenswrapper[5052]: I1203 12:18:40.721772 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-external-api-0" podUID="cd9a6dc3-a814-4a29-924a-d8ada756e7c7" containerName="glance-log" containerID="cri-o://621073c1609087e450aafc5127e1c89c8e8d8de48573af61d7ee441bcd0685c7" gracePeriod=30 Dec 03 12:18:40 crc kubenswrapper[5052]: I1203 12:18:40.721824 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-external-api-0" podUID="cd9a6dc3-a814-4a29-924a-d8ada756e7c7" containerName="glance-httpd" containerID="cri-o://9d3ac55b9014620975c68f93418691e10f6505723457d823dbfb3d7265965d17" gracePeriod=30 Dec 03 12:18:41 crc kubenswrapper[5052]: I1203 12:18:41.066250 5052 generic.go:334] "Generic (PLEG): container finished" podID="f560f1ba-b0e0-4fb6-9802-bf1c9b726592" containerID="3ef54de746a038fe4206f2d3e720519abdadde19963a1e8b1d6c80e85e395799" exitCode=0 Dec 03 12:18:41 crc kubenswrapper[5052]: I1203 12:18:41.066313 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-3fba-account-create-update-7dlvh" event={"ID":"f560f1ba-b0e0-4fb6-9802-bf1c9b726592","Type":"ContainerDied","Data":"3ef54de746a038fe4206f2d3e720519abdadde19963a1e8b1d6c80e85e395799"} Dec 03 12:18:41 crc kubenswrapper[5052]: I1203 12:18:41.068601 5052 generic.go:334] "Generic (PLEG): container finished" podID="cd9a6dc3-a814-4a29-924a-d8ada756e7c7" containerID="621073c1609087e450aafc5127e1c89c8e8d8de48573af61d7ee441bcd0685c7" exitCode=143 Dec 03 12:18:41 crc kubenswrapper[5052]: I1203 12:18:41.068677 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"cd9a6dc3-a814-4a29-924a-d8ada756e7c7","Type":"ContainerDied","Data":"621073c1609087e450aafc5127e1c89c8e8d8de48573af61d7ee441bcd0685c7"} Dec 03 12:18:41 crc kubenswrapper[5052]: I1203 12:18:41.070056 5052 generic.go:334] "Generic (PLEG): container finished" podID="c2e338a2-8424-4325-a580-55618dcd3c29" containerID="a41a6c3524bcee68d48ead6fe403ee2cd332f3a948dfc6434e4cfddf5ded599e" exitCode=0 Dec 03 12:18:41 crc kubenswrapper[5052]: I1203 12:18:41.070118 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-db-create-tczzp" event={"ID":"c2e338a2-8424-4325-a580-55618dcd3c29","Type":"ContainerDied","Data":"a41a6c3524bcee68d48ead6fe403ee2cd332f3a948dfc6434e4cfddf5ded599e"} Dec 03 12:18:41 crc kubenswrapper[5052]: I1203 12:18:41.071319 5052 generic.go:334] "Generic (PLEG): container finished" podID="0f021810-dfcd-4695-8330-380dfc297701" containerID="b9ffc180c65833f54ac4d801441e9df4db9322afb9d77cb2f4775dc868d1a03d" exitCode=0 Dec 03 12:18:41 crc kubenswrapper[5052]: I1203 12:18:41.071389 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-db-create-76nlc" event={"ID":"0f021810-dfcd-4695-8330-380dfc297701","Type":"ContainerDied","Data":"b9ffc180c65833f54ac4d801441e9df4db9322afb9d77cb2f4775dc868d1a03d"} Dec 03 12:18:41 crc kubenswrapper[5052]: I1203 12:18:41.072856 5052 generic.go:334] "Generic (PLEG): container finished" podID="8d0ffc74-5f47-4195-8091-b4890e55c32f" containerID="9b34bcef1b760b4f3f7417ac9222e8595707f6e3124b14e2613276d75db6ce45" exitCode=0 Dec 03 12:18:41 crc kubenswrapper[5052]: I1203 12:18:41.072939 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-0e20-account-create-update-5hn8s" event={"ID":"8d0ffc74-5f47-4195-8091-b4890e55c32f","Type":"ContainerDied","Data":"9b34bcef1b760b4f3f7417ac9222e8595707f6e3124b14e2613276d75db6ce45"} Dec 03 12:18:41 crc kubenswrapper[5052]: I1203 12:18:41.075124 5052 generic.go:334] "Generic (PLEG): container finished" podID="03fcb214-234e-4e5a-8d9f-bbd2bbda66aa" containerID="3a2ca9d08031e2ebddb639a9dacfa6094d35ce57e118c4a4e882deff84684149" exitCode=0 Dec 03 12:18:41 crc kubenswrapper[5052]: I1203 12:18:41.075175 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-8f5b-account-create-update-89qfl" event={"ID":"03fcb214-234e-4e5a-8d9f-bbd2bbda66aa","Type":"ContainerDied","Data":"3a2ca9d08031e2ebddb639a9dacfa6094d35ce57e118c4a4e882deff84684149"} Dec 03 12:18:41 crc kubenswrapper[5052]: I1203 12:18:41.077384 5052 generic.go:334] "Generic (PLEG): container finished" podID="dfbfcab9-41af-42e1-9ddf-5a05ffedb6c7" containerID="6b10b360bd6b52778b72608626846a9f746ffaa71ad3eb4acd15aceb0aa6bf5d" exitCode=0 Dec 03 12:18:41 crc kubenswrapper[5052]: I1203 12:18:41.077634 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-db-create-2vxt6" event={"ID":"dfbfcab9-41af-42e1-9ddf-5a05ffedb6c7","Type":"ContainerDied","Data":"6b10b360bd6b52778b72608626846a9f746ffaa71ad3eb4acd15aceb0aa6bf5d"} Dec 03 12:18:41 crc kubenswrapper[5052]: I1203 12:18:41.830101 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:18:41 crc kubenswrapper[5052]: I1203 12:18:41.919956 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/abb41fad-6d53-475d-b66a-0f7a4d2e7398-scripts\") pod \"abb41fad-6d53-475d-b66a-0f7a4d2e7398\" (UID: \"abb41fad-6d53-475d-b66a-0f7a4d2e7398\") " Dec 03 12:18:41 crc kubenswrapper[5052]: I1203 12:18:41.920019 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/abb41fad-6d53-475d-b66a-0f7a4d2e7398-log-httpd\") pod \"abb41fad-6d53-475d-b66a-0f7a4d2e7398\" (UID: \"abb41fad-6d53-475d-b66a-0f7a4d2e7398\") " Dec 03 12:18:41 crc kubenswrapper[5052]: I1203 12:18:41.920051 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/abb41fad-6d53-475d-b66a-0f7a4d2e7398-run-httpd\") pod \"abb41fad-6d53-475d-b66a-0f7a4d2e7398\" (UID: \"abb41fad-6d53-475d-b66a-0f7a4d2e7398\") " Dec 03 12:18:41 crc kubenswrapper[5052]: I1203 12:18:41.920101 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/abb41fad-6d53-475d-b66a-0f7a4d2e7398-combined-ca-bundle\") pod \"abb41fad-6d53-475d-b66a-0f7a4d2e7398\" (UID: \"abb41fad-6d53-475d-b66a-0f7a4d2e7398\") " Dec 03 12:18:41 crc kubenswrapper[5052]: I1203 12:18:41.920166 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hk2lc\" (UniqueName: \"kubernetes.io/projected/abb41fad-6d53-475d-b66a-0f7a4d2e7398-kube-api-access-hk2lc\") pod \"abb41fad-6d53-475d-b66a-0f7a4d2e7398\" (UID: \"abb41fad-6d53-475d-b66a-0f7a4d2e7398\") " Dec 03 12:18:41 crc kubenswrapper[5052]: I1203 12:18:41.920554 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/abb41fad-6d53-475d-b66a-0f7a4d2e7398-log-httpd" (OuterVolumeSpecName: "log-httpd") pod "abb41fad-6d53-475d-b66a-0f7a4d2e7398" (UID: "abb41fad-6d53-475d-b66a-0f7a4d2e7398"). InnerVolumeSpecName "log-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:18:41 crc kubenswrapper[5052]: I1203 12:18:41.920588 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/abb41fad-6d53-475d-b66a-0f7a4d2e7398-run-httpd" (OuterVolumeSpecName: "run-httpd") pod "abb41fad-6d53-475d-b66a-0f7a4d2e7398" (UID: "abb41fad-6d53-475d-b66a-0f7a4d2e7398"). InnerVolumeSpecName "run-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:18:41 crc kubenswrapper[5052]: I1203 12:18:41.920886 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/abb41fad-6d53-475d-b66a-0f7a4d2e7398-sg-core-conf-yaml\") pod \"abb41fad-6d53-475d-b66a-0f7a4d2e7398\" (UID: \"abb41fad-6d53-475d-b66a-0f7a4d2e7398\") " Dec 03 12:18:41 crc kubenswrapper[5052]: I1203 12:18:41.921003 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/abb41fad-6d53-475d-b66a-0f7a4d2e7398-config-data\") pod \"abb41fad-6d53-475d-b66a-0f7a4d2e7398\" (UID: \"abb41fad-6d53-475d-b66a-0f7a4d2e7398\") " Dec 03 12:18:41 crc kubenswrapper[5052]: I1203 12:18:41.921483 5052 reconciler_common.go:293] "Volume detached for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/abb41fad-6d53-475d-b66a-0f7a4d2e7398-log-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:18:41 crc kubenswrapper[5052]: I1203 12:18:41.921523 5052 reconciler_common.go:293] "Volume detached for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/abb41fad-6d53-475d-b66a-0f7a4d2e7398-run-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:18:41 crc kubenswrapper[5052]: I1203 12:18:41.925708 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/abb41fad-6d53-475d-b66a-0f7a4d2e7398-kube-api-access-hk2lc" (OuterVolumeSpecName: "kube-api-access-hk2lc") pod "abb41fad-6d53-475d-b66a-0f7a4d2e7398" (UID: "abb41fad-6d53-475d-b66a-0f7a4d2e7398"). InnerVolumeSpecName "kube-api-access-hk2lc". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:18:41 crc kubenswrapper[5052]: I1203 12:18:41.925803 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/abb41fad-6d53-475d-b66a-0f7a4d2e7398-scripts" (OuterVolumeSpecName: "scripts") pod "abb41fad-6d53-475d-b66a-0f7a4d2e7398" (UID: "abb41fad-6d53-475d-b66a-0f7a4d2e7398"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:18:41 crc kubenswrapper[5052]: I1203 12:18:41.961693 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/abb41fad-6d53-475d-b66a-0f7a4d2e7398-sg-core-conf-yaml" (OuterVolumeSpecName: "sg-core-conf-yaml") pod "abb41fad-6d53-475d-b66a-0f7a4d2e7398" (UID: "abb41fad-6d53-475d-b66a-0f7a4d2e7398"). InnerVolumeSpecName "sg-core-conf-yaml". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:18:42 crc kubenswrapper[5052]: I1203 12:18:42.019276 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/abb41fad-6d53-475d-b66a-0f7a4d2e7398-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "abb41fad-6d53-475d-b66a-0f7a4d2e7398" (UID: "abb41fad-6d53-475d-b66a-0f7a4d2e7398"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:18:42 crc kubenswrapper[5052]: I1203 12:18:42.023926 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/abb41fad-6d53-475d-b66a-0f7a4d2e7398-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:18:42 crc kubenswrapper[5052]: I1203 12:18:42.023946 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/abb41fad-6d53-475d-b66a-0f7a4d2e7398-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:18:42 crc kubenswrapper[5052]: I1203 12:18:42.023958 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-hk2lc\" (UniqueName: \"kubernetes.io/projected/abb41fad-6d53-475d-b66a-0f7a4d2e7398-kube-api-access-hk2lc\") on node \"crc\" DevicePath \"\"" Dec 03 12:18:42 crc kubenswrapper[5052]: I1203 12:18:42.023970 5052 reconciler_common.go:293] "Volume detached for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/abb41fad-6d53-475d-b66a-0f7a4d2e7398-sg-core-conf-yaml\") on node \"crc\" DevicePath \"\"" Dec 03 12:18:42 crc kubenswrapper[5052]: I1203 12:18:42.036643 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/abb41fad-6d53-475d-b66a-0f7a4d2e7398-config-data" (OuterVolumeSpecName: "config-data") pod "abb41fad-6d53-475d-b66a-0f7a4d2e7398" (UID: "abb41fad-6d53-475d-b66a-0f7a4d2e7398"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:18:42 crc kubenswrapper[5052]: I1203 12:18:42.096169 5052 generic.go:334] "Generic (PLEG): container finished" podID="abb41fad-6d53-475d-b66a-0f7a4d2e7398" containerID="9ca3901fae35712fbd4bcdeebb868762d7c28b079fea9ebe4579eacbbf811496" exitCode=0 Dec 03 12:18:42 crc kubenswrapper[5052]: I1203 12:18:42.096265 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:18:42 crc kubenswrapper[5052]: I1203 12:18:42.096338 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"abb41fad-6d53-475d-b66a-0f7a4d2e7398","Type":"ContainerDied","Data":"9ca3901fae35712fbd4bcdeebb868762d7c28b079fea9ebe4579eacbbf811496"} Dec 03 12:18:42 crc kubenswrapper[5052]: I1203 12:18:42.096392 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"abb41fad-6d53-475d-b66a-0f7a4d2e7398","Type":"ContainerDied","Data":"9fec43b69a45c59afdfb6def7ba7073e8865907a8fe9902649442b8377a42c0b"} Dec 03 12:18:42 crc kubenswrapper[5052]: I1203 12:18:42.096416 5052 scope.go:117] "RemoveContainer" containerID="ff4a8d315be0c18a9396d7b65e791846daa6c4cea304826e510a593a7972bdb7" Dec 03 12:18:42 crc kubenswrapper[5052]: I1203 12:18:42.127784 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/abb41fad-6d53-475d-b66a-0f7a4d2e7398-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:18:42 crc kubenswrapper[5052]: I1203 12:18:42.137841 5052 scope.go:117] "RemoveContainer" containerID="4a0df3ab9421105cfa8a9dd1190b05910b9c73c0e595f295afaeb3ff6b493d8c" Dec 03 12:18:42 crc kubenswrapper[5052]: I1203 12:18:42.148479 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:18:42 crc kubenswrapper[5052]: I1203 12:18:42.161189 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:18:42 crc kubenswrapper[5052]: I1203 12:18:42.174347 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:18:42 crc kubenswrapper[5052]: E1203 12:18:42.174840 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="abb41fad-6d53-475d-b66a-0f7a4d2e7398" containerName="sg-core" Dec 03 12:18:42 crc kubenswrapper[5052]: I1203 12:18:42.174862 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="abb41fad-6d53-475d-b66a-0f7a4d2e7398" containerName="sg-core" Dec 03 12:18:42 crc kubenswrapper[5052]: E1203 12:18:42.174881 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="abb41fad-6d53-475d-b66a-0f7a4d2e7398" containerName="ceilometer-notification-agent" Dec 03 12:18:42 crc kubenswrapper[5052]: I1203 12:18:42.174891 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="abb41fad-6d53-475d-b66a-0f7a4d2e7398" containerName="ceilometer-notification-agent" Dec 03 12:18:42 crc kubenswrapper[5052]: E1203 12:18:42.174920 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="abb41fad-6d53-475d-b66a-0f7a4d2e7398" containerName="ceilometer-central-agent" Dec 03 12:18:42 crc kubenswrapper[5052]: I1203 12:18:42.174929 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="abb41fad-6d53-475d-b66a-0f7a4d2e7398" containerName="ceilometer-central-agent" Dec 03 12:18:42 crc kubenswrapper[5052]: E1203 12:18:42.174960 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="abb41fad-6d53-475d-b66a-0f7a4d2e7398" containerName="proxy-httpd" Dec 03 12:18:42 crc kubenswrapper[5052]: I1203 12:18:42.174969 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="abb41fad-6d53-475d-b66a-0f7a4d2e7398" containerName="proxy-httpd" Dec 03 12:18:42 crc kubenswrapper[5052]: I1203 12:18:42.175194 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="abb41fad-6d53-475d-b66a-0f7a4d2e7398" containerName="sg-core" Dec 03 12:18:42 crc kubenswrapper[5052]: I1203 12:18:42.175215 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="abb41fad-6d53-475d-b66a-0f7a4d2e7398" containerName="proxy-httpd" Dec 03 12:18:42 crc kubenswrapper[5052]: I1203 12:18:42.175251 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="abb41fad-6d53-475d-b66a-0f7a4d2e7398" containerName="ceilometer-central-agent" Dec 03 12:18:42 crc kubenswrapper[5052]: I1203 12:18:42.175269 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="abb41fad-6d53-475d-b66a-0f7a4d2e7398" containerName="ceilometer-notification-agent" Dec 03 12:18:42 crc kubenswrapper[5052]: I1203 12:18:42.177445 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:18:42 crc kubenswrapper[5052]: I1203 12:18:42.182937 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-config-data" Dec 03 12:18:42 crc kubenswrapper[5052]: I1203 12:18:42.183036 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-scripts" Dec 03 12:18:42 crc kubenswrapper[5052]: I1203 12:18:42.188329 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:18:42 crc kubenswrapper[5052]: I1203 12:18:42.207707 5052 scope.go:117] "RemoveContainer" containerID="427897f1d97e18aab7d9841c595373b5e9f2d193755b53cfd7157a780214e315" Dec 03 12:18:42 crc kubenswrapper[5052]: I1203 12:18:42.246909 5052 scope.go:117] "RemoveContainer" containerID="9ca3901fae35712fbd4bcdeebb868762d7c28b079fea9ebe4579eacbbf811496" Dec 03 12:18:42 crc kubenswrapper[5052]: I1203 12:18:42.285857 5052 scope.go:117] "RemoveContainer" containerID="ff4a8d315be0c18a9396d7b65e791846daa6c4cea304826e510a593a7972bdb7" Dec 03 12:18:42 crc kubenswrapper[5052]: E1203 12:18:42.286565 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ff4a8d315be0c18a9396d7b65e791846daa6c4cea304826e510a593a7972bdb7\": container with ID starting with ff4a8d315be0c18a9396d7b65e791846daa6c4cea304826e510a593a7972bdb7 not found: ID does not exist" containerID="ff4a8d315be0c18a9396d7b65e791846daa6c4cea304826e510a593a7972bdb7" Dec 03 12:18:42 crc kubenswrapper[5052]: I1203 12:18:42.286616 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ff4a8d315be0c18a9396d7b65e791846daa6c4cea304826e510a593a7972bdb7"} err="failed to get container status \"ff4a8d315be0c18a9396d7b65e791846daa6c4cea304826e510a593a7972bdb7\": rpc error: code = NotFound desc = could not find container \"ff4a8d315be0c18a9396d7b65e791846daa6c4cea304826e510a593a7972bdb7\": container with ID starting with ff4a8d315be0c18a9396d7b65e791846daa6c4cea304826e510a593a7972bdb7 not found: ID does not exist" Dec 03 12:18:42 crc kubenswrapper[5052]: I1203 12:18:42.286644 5052 scope.go:117] "RemoveContainer" containerID="4a0df3ab9421105cfa8a9dd1190b05910b9c73c0e595f295afaeb3ff6b493d8c" Dec 03 12:18:42 crc kubenswrapper[5052]: E1203 12:18:42.286944 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4a0df3ab9421105cfa8a9dd1190b05910b9c73c0e595f295afaeb3ff6b493d8c\": container with ID starting with 4a0df3ab9421105cfa8a9dd1190b05910b9c73c0e595f295afaeb3ff6b493d8c not found: ID does not exist" containerID="4a0df3ab9421105cfa8a9dd1190b05910b9c73c0e595f295afaeb3ff6b493d8c" Dec 03 12:18:42 crc kubenswrapper[5052]: I1203 12:18:42.286962 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4a0df3ab9421105cfa8a9dd1190b05910b9c73c0e595f295afaeb3ff6b493d8c"} err="failed to get container status \"4a0df3ab9421105cfa8a9dd1190b05910b9c73c0e595f295afaeb3ff6b493d8c\": rpc error: code = NotFound desc = could not find container \"4a0df3ab9421105cfa8a9dd1190b05910b9c73c0e595f295afaeb3ff6b493d8c\": container with ID starting with 4a0df3ab9421105cfa8a9dd1190b05910b9c73c0e595f295afaeb3ff6b493d8c not found: ID does not exist" Dec 03 12:18:42 crc kubenswrapper[5052]: I1203 12:18:42.286996 5052 scope.go:117] "RemoveContainer" containerID="427897f1d97e18aab7d9841c595373b5e9f2d193755b53cfd7157a780214e315" Dec 03 12:18:42 crc kubenswrapper[5052]: E1203 12:18:42.287289 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"427897f1d97e18aab7d9841c595373b5e9f2d193755b53cfd7157a780214e315\": container with ID starting with 427897f1d97e18aab7d9841c595373b5e9f2d193755b53cfd7157a780214e315 not found: ID does not exist" containerID="427897f1d97e18aab7d9841c595373b5e9f2d193755b53cfd7157a780214e315" Dec 03 12:18:42 crc kubenswrapper[5052]: I1203 12:18:42.287325 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"427897f1d97e18aab7d9841c595373b5e9f2d193755b53cfd7157a780214e315"} err="failed to get container status \"427897f1d97e18aab7d9841c595373b5e9f2d193755b53cfd7157a780214e315\": rpc error: code = NotFound desc = could not find container \"427897f1d97e18aab7d9841c595373b5e9f2d193755b53cfd7157a780214e315\": container with ID starting with 427897f1d97e18aab7d9841c595373b5e9f2d193755b53cfd7157a780214e315 not found: ID does not exist" Dec 03 12:18:42 crc kubenswrapper[5052]: I1203 12:18:42.287352 5052 scope.go:117] "RemoveContainer" containerID="9ca3901fae35712fbd4bcdeebb868762d7c28b079fea9ebe4579eacbbf811496" Dec 03 12:18:42 crc kubenswrapper[5052]: E1203 12:18:42.287646 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9ca3901fae35712fbd4bcdeebb868762d7c28b079fea9ebe4579eacbbf811496\": container with ID starting with 9ca3901fae35712fbd4bcdeebb868762d7c28b079fea9ebe4579eacbbf811496 not found: ID does not exist" containerID="9ca3901fae35712fbd4bcdeebb868762d7c28b079fea9ebe4579eacbbf811496" Dec 03 12:18:42 crc kubenswrapper[5052]: I1203 12:18:42.287668 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9ca3901fae35712fbd4bcdeebb868762d7c28b079fea9ebe4579eacbbf811496"} err="failed to get container status \"9ca3901fae35712fbd4bcdeebb868762d7c28b079fea9ebe4579eacbbf811496\": rpc error: code = NotFound desc = could not find container \"9ca3901fae35712fbd4bcdeebb868762d7c28b079fea9ebe4579eacbbf811496\": container with ID starting with 9ca3901fae35712fbd4bcdeebb868762d7c28b079fea9ebe4579eacbbf811496 not found: ID does not exist" Dec 03 12:18:42 crc kubenswrapper[5052]: I1203 12:18:42.331157 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e8433c1d-9f85-444b-b7bf-c2bd62fdbe8d-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"e8433c1d-9f85-444b-b7bf-c2bd62fdbe8d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:18:42 crc kubenswrapper[5052]: I1203 12:18:42.331538 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/e8433c1d-9f85-444b-b7bf-c2bd62fdbe8d-run-httpd\") pod \"ceilometer-0\" (UID: \"e8433c1d-9f85-444b-b7bf-c2bd62fdbe8d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:18:42 crc kubenswrapper[5052]: I1203 12:18:42.331669 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/e8433c1d-9f85-444b-b7bf-c2bd62fdbe8d-config-data\") pod \"ceilometer-0\" (UID: \"e8433c1d-9f85-444b-b7bf-c2bd62fdbe8d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:18:42 crc kubenswrapper[5052]: I1203 12:18:42.331776 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-flxgv\" (UniqueName: \"kubernetes.io/projected/e8433c1d-9f85-444b-b7bf-c2bd62fdbe8d-kube-api-access-flxgv\") pod \"ceilometer-0\" (UID: \"e8433c1d-9f85-444b-b7bf-c2bd62fdbe8d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:18:42 crc kubenswrapper[5052]: I1203 12:18:42.331819 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/e8433c1d-9f85-444b-b7bf-c2bd62fdbe8d-scripts\") pod \"ceilometer-0\" (UID: \"e8433c1d-9f85-444b-b7bf-c2bd62fdbe8d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:18:42 crc kubenswrapper[5052]: I1203 12:18:42.331865 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/e8433c1d-9f85-444b-b7bf-c2bd62fdbe8d-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"e8433c1d-9f85-444b-b7bf-c2bd62fdbe8d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:18:42 crc kubenswrapper[5052]: I1203 12:18:42.331990 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/e8433c1d-9f85-444b-b7bf-c2bd62fdbe8d-log-httpd\") pod \"ceilometer-0\" (UID: \"e8433c1d-9f85-444b-b7bf-c2bd62fdbe8d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:18:42 crc kubenswrapper[5052]: I1203 12:18:42.434319 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/e8433c1d-9f85-444b-b7bf-c2bd62fdbe8d-run-httpd\") pod \"ceilometer-0\" (UID: \"e8433c1d-9f85-444b-b7bf-c2bd62fdbe8d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:18:42 crc kubenswrapper[5052]: I1203 12:18:42.434408 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/e8433c1d-9f85-444b-b7bf-c2bd62fdbe8d-config-data\") pod \"ceilometer-0\" (UID: \"e8433c1d-9f85-444b-b7bf-c2bd62fdbe8d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:18:42 crc kubenswrapper[5052]: I1203 12:18:42.434465 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-flxgv\" (UniqueName: \"kubernetes.io/projected/e8433c1d-9f85-444b-b7bf-c2bd62fdbe8d-kube-api-access-flxgv\") pod \"ceilometer-0\" (UID: \"e8433c1d-9f85-444b-b7bf-c2bd62fdbe8d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:18:42 crc kubenswrapper[5052]: I1203 12:18:42.434496 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/e8433c1d-9f85-444b-b7bf-c2bd62fdbe8d-scripts\") pod \"ceilometer-0\" (UID: \"e8433c1d-9f85-444b-b7bf-c2bd62fdbe8d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:18:42 crc kubenswrapper[5052]: I1203 12:18:42.434548 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/e8433c1d-9f85-444b-b7bf-c2bd62fdbe8d-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"e8433c1d-9f85-444b-b7bf-c2bd62fdbe8d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:18:42 crc kubenswrapper[5052]: I1203 12:18:42.434583 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/e8433c1d-9f85-444b-b7bf-c2bd62fdbe8d-log-httpd\") pod \"ceilometer-0\" (UID: \"e8433c1d-9f85-444b-b7bf-c2bd62fdbe8d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:18:42 crc kubenswrapper[5052]: I1203 12:18:42.434679 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e8433c1d-9f85-444b-b7bf-c2bd62fdbe8d-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"e8433c1d-9f85-444b-b7bf-c2bd62fdbe8d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:18:42 crc kubenswrapper[5052]: I1203 12:18:42.435135 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/e8433c1d-9f85-444b-b7bf-c2bd62fdbe8d-run-httpd\") pod \"ceilometer-0\" (UID: \"e8433c1d-9f85-444b-b7bf-c2bd62fdbe8d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:18:42 crc kubenswrapper[5052]: I1203 12:18:42.435530 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/e8433c1d-9f85-444b-b7bf-c2bd62fdbe8d-log-httpd\") pod \"ceilometer-0\" (UID: \"e8433c1d-9f85-444b-b7bf-c2bd62fdbe8d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:18:42 crc kubenswrapper[5052]: I1203 12:18:42.439548 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/e8433c1d-9f85-444b-b7bf-c2bd62fdbe8d-scripts\") pod \"ceilometer-0\" (UID: \"e8433c1d-9f85-444b-b7bf-c2bd62fdbe8d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:18:42 crc kubenswrapper[5052]: I1203 12:18:42.444934 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/e8433c1d-9f85-444b-b7bf-c2bd62fdbe8d-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"e8433c1d-9f85-444b-b7bf-c2bd62fdbe8d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:18:42 crc kubenswrapper[5052]: I1203 12:18:42.445652 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e8433c1d-9f85-444b-b7bf-c2bd62fdbe8d-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"e8433c1d-9f85-444b-b7bf-c2bd62fdbe8d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:18:42 crc kubenswrapper[5052]: I1203 12:18:42.449331 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/e8433c1d-9f85-444b-b7bf-c2bd62fdbe8d-config-data\") pod \"ceilometer-0\" (UID: \"e8433c1d-9f85-444b-b7bf-c2bd62fdbe8d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:18:42 crc kubenswrapper[5052]: I1203 12:18:42.453490 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-flxgv\" (UniqueName: \"kubernetes.io/projected/e8433c1d-9f85-444b-b7bf-c2bd62fdbe8d-kube-api-access-flxgv\") pod \"ceilometer-0\" (UID: \"e8433c1d-9f85-444b-b7bf-c2bd62fdbe8d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:18:42 crc kubenswrapper[5052]: I1203 12:18:42.500133 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:18:42 crc kubenswrapper[5052]: I1203 12:18:42.662971 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-0e20-account-create-update-5hn8s" Dec 03 12:18:42 crc kubenswrapper[5052]: I1203 12:18:42.742139 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jwvdv\" (UniqueName: \"kubernetes.io/projected/8d0ffc74-5f47-4195-8091-b4890e55c32f-kube-api-access-jwvdv\") pod \"8d0ffc74-5f47-4195-8091-b4890e55c32f\" (UID: \"8d0ffc74-5f47-4195-8091-b4890e55c32f\") " Dec 03 12:18:42 crc kubenswrapper[5052]: I1203 12:18:42.742262 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/8d0ffc74-5f47-4195-8091-b4890e55c32f-operator-scripts\") pod \"8d0ffc74-5f47-4195-8091-b4890e55c32f\" (UID: \"8d0ffc74-5f47-4195-8091-b4890e55c32f\") " Dec 03 12:18:42 crc kubenswrapper[5052]: I1203 12:18:42.745146 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8d0ffc74-5f47-4195-8091-b4890e55c32f-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "8d0ffc74-5f47-4195-8091-b4890e55c32f" (UID: "8d0ffc74-5f47-4195-8091-b4890e55c32f"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:18:42 crc kubenswrapper[5052]: I1203 12:18:42.751306 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8d0ffc74-5f47-4195-8091-b4890e55c32f-kube-api-access-jwvdv" (OuterVolumeSpecName: "kube-api-access-jwvdv") pod "8d0ffc74-5f47-4195-8091-b4890e55c32f" (UID: "8d0ffc74-5f47-4195-8091-b4890e55c32f"). InnerVolumeSpecName "kube-api-access-jwvdv". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:18:42 crc kubenswrapper[5052]: I1203 12:18:42.811180 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-8f5b-account-create-update-89qfl" Dec 03 12:18:42 crc kubenswrapper[5052]: I1203 12:18:42.822834 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-db-create-tczzp" Dec 03 12:18:42 crc kubenswrapper[5052]: I1203 12:18:42.833613 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-3fba-account-create-update-7dlvh" Dec 03 12:18:42 crc kubenswrapper[5052]: I1203 12:18:42.858619 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-db-create-76nlc" Dec 03 12:18:42 crc kubenswrapper[5052]: I1203 12:18:42.860400 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-db-create-2vxt6" Dec 03 12:18:42 crc kubenswrapper[5052]: I1203 12:18:42.863037 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-jwvdv\" (UniqueName: \"kubernetes.io/projected/8d0ffc74-5f47-4195-8091-b4890e55c32f-kube-api-access-jwvdv\") on node \"crc\" DevicePath \"\"" Dec 03 12:18:42 crc kubenswrapper[5052]: I1203 12:18:42.863079 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/8d0ffc74-5f47-4195-8091-b4890e55c32f-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:18:42 crc kubenswrapper[5052]: I1203 12:18:42.964299 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/0f021810-dfcd-4695-8330-380dfc297701-operator-scripts\") pod \"0f021810-dfcd-4695-8330-380dfc297701\" (UID: \"0f021810-dfcd-4695-8330-380dfc297701\") " Dec 03 12:18:42 crc kubenswrapper[5052]: I1203 12:18:42.964352 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vwfq5\" (UniqueName: \"kubernetes.io/projected/0f021810-dfcd-4695-8330-380dfc297701-kube-api-access-vwfq5\") pod \"0f021810-dfcd-4695-8330-380dfc297701\" (UID: \"0f021810-dfcd-4695-8330-380dfc297701\") " Dec 03 12:18:42 crc kubenswrapper[5052]: I1203 12:18:42.964423 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pd6mt\" (UniqueName: \"kubernetes.io/projected/f560f1ba-b0e0-4fb6-9802-bf1c9b726592-kube-api-access-pd6mt\") pod \"f560f1ba-b0e0-4fb6-9802-bf1c9b726592\" (UID: \"f560f1ba-b0e0-4fb6-9802-bf1c9b726592\") " Dec 03 12:18:42 crc kubenswrapper[5052]: I1203 12:18:42.964473 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5l8j8\" (UniqueName: \"kubernetes.io/projected/c2e338a2-8424-4325-a580-55618dcd3c29-kube-api-access-5l8j8\") pod \"c2e338a2-8424-4325-a580-55618dcd3c29\" (UID: \"c2e338a2-8424-4325-a580-55618dcd3c29\") " Dec 03 12:18:42 crc kubenswrapper[5052]: I1203 12:18:42.964557 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s8flq\" (UniqueName: \"kubernetes.io/projected/dfbfcab9-41af-42e1-9ddf-5a05ffedb6c7-kube-api-access-s8flq\") pod \"dfbfcab9-41af-42e1-9ddf-5a05ffedb6c7\" (UID: \"dfbfcab9-41af-42e1-9ddf-5a05ffedb6c7\") " Dec 03 12:18:42 crc kubenswrapper[5052]: I1203 12:18:42.964623 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/f560f1ba-b0e0-4fb6-9802-bf1c9b726592-operator-scripts\") pod \"f560f1ba-b0e0-4fb6-9802-bf1c9b726592\" (UID: \"f560f1ba-b0e0-4fb6-9802-bf1c9b726592\") " Dec 03 12:18:42 crc kubenswrapper[5052]: I1203 12:18:42.964655 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/dfbfcab9-41af-42e1-9ddf-5a05ffedb6c7-operator-scripts\") pod \"dfbfcab9-41af-42e1-9ddf-5a05ffedb6c7\" (UID: \"dfbfcab9-41af-42e1-9ddf-5a05ffedb6c7\") " Dec 03 12:18:42 crc kubenswrapper[5052]: I1203 12:18:42.964732 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dzrdf\" (UniqueName: \"kubernetes.io/projected/03fcb214-234e-4e5a-8d9f-bbd2bbda66aa-kube-api-access-dzrdf\") pod \"03fcb214-234e-4e5a-8d9f-bbd2bbda66aa\" (UID: \"03fcb214-234e-4e5a-8d9f-bbd2bbda66aa\") " Dec 03 12:18:42 crc kubenswrapper[5052]: I1203 12:18:42.964786 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/03fcb214-234e-4e5a-8d9f-bbd2bbda66aa-operator-scripts\") pod \"03fcb214-234e-4e5a-8d9f-bbd2bbda66aa\" (UID: \"03fcb214-234e-4e5a-8d9f-bbd2bbda66aa\") " Dec 03 12:18:42 crc kubenswrapper[5052]: I1203 12:18:42.964822 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/c2e338a2-8424-4325-a580-55618dcd3c29-operator-scripts\") pod \"c2e338a2-8424-4325-a580-55618dcd3c29\" (UID: \"c2e338a2-8424-4325-a580-55618dcd3c29\") " Dec 03 12:18:42 crc kubenswrapper[5052]: I1203 12:18:42.964990 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0f021810-dfcd-4695-8330-380dfc297701-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "0f021810-dfcd-4695-8330-380dfc297701" (UID: "0f021810-dfcd-4695-8330-380dfc297701"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:18:42 crc kubenswrapper[5052]: I1203 12:18:42.965223 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f560f1ba-b0e0-4fb6-9802-bf1c9b726592-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "f560f1ba-b0e0-4fb6-9802-bf1c9b726592" (UID: "f560f1ba-b0e0-4fb6-9802-bf1c9b726592"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:18:42 crc kubenswrapper[5052]: I1203 12:18:42.965304 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/0f021810-dfcd-4695-8330-380dfc297701-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:18:42 crc kubenswrapper[5052]: I1203 12:18:42.965818 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dfbfcab9-41af-42e1-9ddf-5a05ffedb6c7-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "dfbfcab9-41af-42e1-9ddf-5a05ffedb6c7" (UID: "dfbfcab9-41af-42e1-9ddf-5a05ffedb6c7"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:18:42 crc kubenswrapper[5052]: I1203 12:18:42.968305 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c2e338a2-8424-4325-a580-55618dcd3c29-kube-api-access-5l8j8" (OuterVolumeSpecName: "kube-api-access-5l8j8") pod "c2e338a2-8424-4325-a580-55618dcd3c29" (UID: "c2e338a2-8424-4325-a580-55618dcd3c29"). InnerVolumeSpecName "kube-api-access-5l8j8". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:18:42 crc kubenswrapper[5052]: I1203 12:18:42.968345 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0f021810-dfcd-4695-8330-380dfc297701-kube-api-access-vwfq5" (OuterVolumeSpecName: "kube-api-access-vwfq5") pod "0f021810-dfcd-4695-8330-380dfc297701" (UID: "0f021810-dfcd-4695-8330-380dfc297701"). InnerVolumeSpecName "kube-api-access-vwfq5". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:18:42 crc kubenswrapper[5052]: I1203 12:18:42.968538 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/03fcb214-234e-4e5a-8d9f-bbd2bbda66aa-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "03fcb214-234e-4e5a-8d9f-bbd2bbda66aa" (UID: "03fcb214-234e-4e5a-8d9f-bbd2bbda66aa"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:18:42 crc kubenswrapper[5052]: I1203 12:18:42.968666 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c2e338a2-8424-4325-a580-55618dcd3c29-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "c2e338a2-8424-4325-a580-55618dcd3c29" (UID: "c2e338a2-8424-4325-a580-55618dcd3c29"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:18:42 crc kubenswrapper[5052]: I1203 12:18:42.968803 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/03fcb214-234e-4e5a-8d9f-bbd2bbda66aa-kube-api-access-dzrdf" (OuterVolumeSpecName: "kube-api-access-dzrdf") pod "03fcb214-234e-4e5a-8d9f-bbd2bbda66aa" (UID: "03fcb214-234e-4e5a-8d9f-bbd2bbda66aa"). InnerVolumeSpecName "kube-api-access-dzrdf". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:18:42 crc kubenswrapper[5052]: I1203 12:18:42.969548 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f560f1ba-b0e0-4fb6-9802-bf1c9b726592-kube-api-access-pd6mt" (OuterVolumeSpecName: "kube-api-access-pd6mt") pod "f560f1ba-b0e0-4fb6-9802-bf1c9b726592" (UID: "f560f1ba-b0e0-4fb6-9802-bf1c9b726592"). InnerVolumeSpecName "kube-api-access-pd6mt". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:18:42 crc kubenswrapper[5052]: I1203 12:18:42.970029 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dfbfcab9-41af-42e1-9ddf-5a05ffedb6c7-kube-api-access-s8flq" (OuterVolumeSpecName: "kube-api-access-s8flq") pod "dfbfcab9-41af-42e1-9ddf-5a05ffedb6c7" (UID: "dfbfcab9-41af-42e1-9ddf-5a05ffedb6c7"). InnerVolumeSpecName "kube-api-access-s8flq". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:18:43 crc kubenswrapper[5052]: I1203 12:18:43.067252 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-vwfq5\" (UniqueName: \"kubernetes.io/projected/0f021810-dfcd-4695-8330-380dfc297701-kube-api-access-vwfq5\") on node \"crc\" DevicePath \"\"" Dec 03 12:18:43 crc kubenswrapper[5052]: I1203 12:18:43.067304 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-pd6mt\" (UniqueName: \"kubernetes.io/projected/f560f1ba-b0e0-4fb6-9802-bf1c9b726592-kube-api-access-pd6mt\") on node \"crc\" DevicePath \"\"" Dec 03 12:18:43 crc kubenswrapper[5052]: I1203 12:18:43.067320 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-5l8j8\" (UniqueName: \"kubernetes.io/projected/c2e338a2-8424-4325-a580-55618dcd3c29-kube-api-access-5l8j8\") on node \"crc\" DevicePath \"\"" Dec 03 12:18:43 crc kubenswrapper[5052]: I1203 12:18:43.067336 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-s8flq\" (UniqueName: \"kubernetes.io/projected/dfbfcab9-41af-42e1-9ddf-5a05ffedb6c7-kube-api-access-s8flq\") on node \"crc\" DevicePath \"\"" Dec 03 12:18:43 crc kubenswrapper[5052]: I1203 12:18:43.067351 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/f560f1ba-b0e0-4fb6-9802-bf1c9b726592-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:18:43 crc kubenswrapper[5052]: I1203 12:18:43.067363 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/dfbfcab9-41af-42e1-9ddf-5a05ffedb6c7-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:18:43 crc kubenswrapper[5052]: I1203 12:18:43.067375 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-dzrdf\" (UniqueName: \"kubernetes.io/projected/03fcb214-234e-4e5a-8d9f-bbd2bbda66aa-kube-api-access-dzrdf\") on node \"crc\" DevicePath \"\"" Dec 03 12:18:43 crc kubenswrapper[5052]: I1203 12:18:43.067387 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/03fcb214-234e-4e5a-8d9f-bbd2bbda66aa-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:18:43 crc kubenswrapper[5052]: I1203 12:18:43.067398 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/c2e338a2-8424-4325-a580-55618dcd3c29-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:18:43 crc kubenswrapper[5052]: I1203 12:18:43.071323 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:18:43 crc kubenswrapper[5052]: I1203 12:18:43.126398 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-db-create-76nlc" event={"ID":"0f021810-dfcd-4695-8330-380dfc297701","Type":"ContainerDied","Data":"1c8ce63748029a279f5acdafd57a217ba217362c3b50ed7b7df5b5a4e3ad8199"} Dec 03 12:18:43 crc kubenswrapper[5052]: I1203 12:18:43.126451 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1c8ce63748029a279f5acdafd57a217ba217362c3b50ed7b7df5b5a4e3ad8199" Dec 03 12:18:43 crc kubenswrapper[5052]: I1203 12:18:43.126417 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-db-create-76nlc" Dec 03 12:18:43 crc kubenswrapper[5052]: I1203 12:18:43.129116 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-0e20-account-create-update-5hn8s" event={"ID":"8d0ffc74-5f47-4195-8091-b4890e55c32f","Type":"ContainerDied","Data":"c6fa814155ff7b6ba99d54e314f75cdfebb6064aef56c5a95556bc35e1f5bf15"} Dec 03 12:18:43 crc kubenswrapper[5052]: I1203 12:18:43.129150 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c6fa814155ff7b6ba99d54e314f75cdfebb6064aef56c5a95556bc35e1f5bf15" Dec 03 12:18:43 crc kubenswrapper[5052]: I1203 12:18:43.129273 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-0e20-account-create-update-5hn8s" Dec 03 12:18:43 crc kubenswrapper[5052]: I1203 12:18:43.139180 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-8f5b-account-create-update-89qfl" event={"ID":"03fcb214-234e-4e5a-8d9f-bbd2bbda66aa","Type":"ContainerDied","Data":"0bd942fc8effe4e0620e0505e0189496ea2479a523dac707a0b380f8edeedb9e"} Dec 03 12:18:43 crc kubenswrapper[5052]: I1203 12:18:43.139230 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0bd942fc8effe4e0620e0505e0189496ea2479a523dac707a0b380f8edeedb9e" Dec 03 12:18:43 crc kubenswrapper[5052]: I1203 12:18:43.139239 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-8f5b-account-create-update-89qfl" Dec 03 12:18:43 crc kubenswrapper[5052]: I1203 12:18:43.141304 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-db-create-2vxt6" event={"ID":"dfbfcab9-41af-42e1-9ddf-5a05ffedb6c7","Type":"ContainerDied","Data":"bc8c9a489e15f709ad4c30887d493242b3303b777dd3d56ec320c387eef889a4"} Dec 03 12:18:43 crc kubenswrapper[5052]: I1203 12:18:43.141886 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bc8c9a489e15f709ad4c30887d493242b3303b777dd3d56ec320c387eef889a4" Dec 03 12:18:43 crc kubenswrapper[5052]: I1203 12:18:43.141370 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-db-create-2vxt6" Dec 03 12:18:43 crc kubenswrapper[5052]: I1203 12:18:43.143343 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-3fba-account-create-update-7dlvh" event={"ID":"f560f1ba-b0e0-4fb6-9802-bf1c9b726592","Type":"ContainerDied","Data":"54f74b5c06bf6339c0008a439dac9d4008cf3b2fce0dc494d7a4273258b5878d"} Dec 03 12:18:43 crc kubenswrapper[5052]: I1203 12:18:43.143373 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="54f74b5c06bf6339c0008a439dac9d4008cf3b2fce0dc494d7a4273258b5878d" Dec 03 12:18:43 crc kubenswrapper[5052]: I1203 12:18:43.143443 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-3fba-account-create-update-7dlvh" Dec 03 12:18:43 crc kubenswrapper[5052]: I1203 12:18:43.145598 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"e8433c1d-9f85-444b-b7bf-c2bd62fdbe8d","Type":"ContainerStarted","Data":"7e9652ea1e1261d5b6c7936e3301027974dfa87b069906329efcaa0f7a1206b5"} Dec 03 12:18:43 crc kubenswrapper[5052]: I1203 12:18:43.148405 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-db-create-tczzp" event={"ID":"c2e338a2-8424-4325-a580-55618dcd3c29","Type":"ContainerDied","Data":"334c3cbe8b938b7b5cb33c3d19eed183bd8a7a667983ac2827c11aa0646a7016"} Dec 03 12:18:43 crc kubenswrapper[5052]: I1203 12:18:43.148430 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="334c3cbe8b938b7b5cb33c3d19eed183bd8a7a667983ac2827c11aa0646a7016" Dec 03 12:18:43 crc kubenswrapper[5052]: I1203 12:18:43.148479 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-db-create-tczzp" Dec 03 12:18:43 crc kubenswrapper[5052]: I1203 12:18:43.683362 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="abb41fad-6d53-475d-b66a-0f7a4d2e7398" path="/var/lib/kubelet/pods/abb41fad-6d53-475d-b66a-0f7a4d2e7398/volumes" Dec 03 12:18:44 crc kubenswrapper[5052]: I1203 12:18:44.161325 5052 generic.go:334] "Generic (PLEG): container finished" podID="cd9a6dc3-a814-4a29-924a-d8ada756e7c7" containerID="9d3ac55b9014620975c68f93418691e10f6505723457d823dbfb3d7265965d17" exitCode=0 Dec 03 12:18:44 crc kubenswrapper[5052]: I1203 12:18:44.161659 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"cd9a6dc3-a814-4a29-924a-d8ada756e7c7","Type":"ContainerDied","Data":"9d3ac55b9014620975c68f93418691e10f6505723457d823dbfb3d7265965d17"} Dec 03 12:18:44 crc kubenswrapper[5052]: I1203 12:18:44.163474 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"e8433c1d-9f85-444b-b7bf-c2bd62fdbe8d","Type":"ContainerStarted","Data":"7bcf29528a2630e2584876eb33663faf73c9660396ef93d28e491b29c246425e"} Dec 03 12:18:44 crc kubenswrapper[5052]: I1203 12:18:44.326006 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:18:44 crc kubenswrapper[5052]: I1203 12:18:44.501180 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/cd9a6dc3-a814-4a29-924a-d8ada756e7c7-httpd-run\") pod \"cd9a6dc3-a814-4a29-924a-d8ada756e7c7\" (UID: \"cd9a6dc3-a814-4a29-924a-d8ada756e7c7\") " Dec 03 12:18:44 crc kubenswrapper[5052]: I1203 12:18:44.501274 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/cd9a6dc3-a814-4a29-924a-d8ada756e7c7-scripts\") pod \"cd9a6dc3-a814-4a29-924a-d8ada756e7c7\" (UID: \"cd9a6dc3-a814-4a29-924a-d8ada756e7c7\") " Dec 03 12:18:44 crc kubenswrapper[5052]: I1203 12:18:44.501305 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/cd9a6dc3-a814-4a29-924a-d8ada756e7c7-config-data\") pod \"cd9a6dc3-a814-4a29-924a-d8ada756e7c7\" (UID: \"cd9a6dc3-a814-4a29-924a-d8ada756e7c7\") " Dec 03 12:18:44 crc kubenswrapper[5052]: I1203 12:18:44.501390 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/cd9a6dc3-a814-4a29-924a-d8ada756e7c7-combined-ca-bundle\") pod \"cd9a6dc3-a814-4a29-924a-d8ada756e7c7\" (UID: \"cd9a6dc3-a814-4a29-924a-d8ada756e7c7\") " Dec 03 12:18:44 crc kubenswrapper[5052]: I1203 12:18:44.501425 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"glance\" (UniqueName: \"kubernetes.io/local-volume/local-storage09-crc\") pod \"cd9a6dc3-a814-4a29-924a-d8ada756e7c7\" (UID: \"cd9a6dc3-a814-4a29-924a-d8ada756e7c7\") " Dec 03 12:18:44 crc kubenswrapper[5052]: I1203 12:18:44.501479 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lfv2c\" (UniqueName: \"kubernetes.io/projected/cd9a6dc3-a814-4a29-924a-d8ada756e7c7-kube-api-access-lfv2c\") pod \"cd9a6dc3-a814-4a29-924a-d8ada756e7c7\" (UID: \"cd9a6dc3-a814-4a29-924a-d8ada756e7c7\") " Dec 03 12:18:44 crc kubenswrapper[5052]: I1203 12:18:44.501505 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/cd9a6dc3-a814-4a29-924a-d8ada756e7c7-logs\") pod \"cd9a6dc3-a814-4a29-924a-d8ada756e7c7\" (UID: \"cd9a6dc3-a814-4a29-924a-d8ada756e7c7\") " Dec 03 12:18:44 crc kubenswrapper[5052]: I1203 12:18:44.501550 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/cd9a6dc3-a814-4a29-924a-d8ada756e7c7-public-tls-certs\") pod \"cd9a6dc3-a814-4a29-924a-d8ada756e7c7\" (UID: \"cd9a6dc3-a814-4a29-924a-d8ada756e7c7\") " Dec 03 12:18:44 crc kubenswrapper[5052]: I1203 12:18:44.503470 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd9a6dc3-a814-4a29-924a-d8ada756e7c7-httpd-run" (OuterVolumeSpecName: "httpd-run") pod "cd9a6dc3-a814-4a29-924a-d8ada756e7c7" (UID: "cd9a6dc3-a814-4a29-924a-d8ada756e7c7"). InnerVolumeSpecName "httpd-run". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:18:44 crc kubenswrapper[5052]: I1203 12:18:44.503984 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd9a6dc3-a814-4a29-924a-d8ada756e7c7-logs" (OuterVolumeSpecName: "logs") pod "cd9a6dc3-a814-4a29-924a-d8ada756e7c7" (UID: "cd9a6dc3-a814-4a29-924a-d8ada756e7c7"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:18:44 crc kubenswrapper[5052]: I1203 12:18:44.508374 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cd9a6dc3-a814-4a29-924a-d8ada756e7c7-kube-api-access-lfv2c" (OuterVolumeSpecName: "kube-api-access-lfv2c") pod "cd9a6dc3-a814-4a29-924a-d8ada756e7c7" (UID: "cd9a6dc3-a814-4a29-924a-d8ada756e7c7"). InnerVolumeSpecName "kube-api-access-lfv2c". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:18:44 crc kubenswrapper[5052]: I1203 12:18:44.508391 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cd9a6dc3-a814-4a29-924a-d8ada756e7c7-scripts" (OuterVolumeSpecName: "scripts") pod "cd9a6dc3-a814-4a29-924a-d8ada756e7c7" (UID: "cd9a6dc3-a814-4a29-924a-d8ada756e7c7"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:18:44 crc kubenswrapper[5052]: I1203 12:18:44.510645 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage09-crc" (OuterVolumeSpecName: "glance") pod "cd9a6dc3-a814-4a29-924a-d8ada756e7c7" (UID: "cd9a6dc3-a814-4a29-924a-d8ada756e7c7"). InnerVolumeSpecName "local-storage09-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 12:18:44 crc kubenswrapper[5052]: I1203 12:18:44.548698 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cd9a6dc3-a814-4a29-924a-d8ada756e7c7-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "cd9a6dc3-a814-4a29-924a-d8ada756e7c7" (UID: "cd9a6dc3-a814-4a29-924a-d8ada756e7c7"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:18:44 crc kubenswrapper[5052]: I1203 12:18:44.576662 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cd9a6dc3-a814-4a29-924a-d8ada756e7c7-config-data" (OuterVolumeSpecName: "config-data") pod "cd9a6dc3-a814-4a29-924a-d8ada756e7c7" (UID: "cd9a6dc3-a814-4a29-924a-d8ada756e7c7"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:18:44 crc kubenswrapper[5052]: I1203 12:18:44.577934 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cd9a6dc3-a814-4a29-924a-d8ada756e7c7-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "cd9a6dc3-a814-4a29-924a-d8ada756e7c7" (UID: "cd9a6dc3-a814-4a29-924a-d8ada756e7c7"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:18:44 crc kubenswrapper[5052]: I1203 12:18:44.604987 5052 reconciler_common.go:293] "Volume detached for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/cd9a6dc3-a814-4a29-924a-d8ada756e7c7-httpd-run\") on node \"crc\" DevicePath \"\"" Dec 03 12:18:44 crc kubenswrapper[5052]: I1203 12:18:44.605025 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/cd9a6dc3-a814-4a29-924a-d8ada756e7c7-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:18:44 crc kubenswrapper[5052]: I1203 12:18:44.605036 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/cd9a6dc3-a814-4a29-924a-d8ada756e7c7-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:18:44 crc kubenswrapper[5052]: I1203 12:18:44.605049 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/cd9a6dc3-a814-4a29-924a-d8ada756e7c7-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:18:44 crc kubenswrapper[5052]: I1203 12:18:44.605096 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage09-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage09-crc\") on node \"crc\" " Dec 03 12:18:44 crc kubenswrapper[5052]: I1203 12:18:44.605109 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-lfv2c\" (UniqueName: \"kubernetes.io/projected/cd9a6dc3-a814-4a29-924a-d8ada756e7c7-kube-api-access-lfv2c\") on node \"crc\" DevicePath \"\"" Dec 03 12:18:44 crc kubenswrapper[5052]: I1203 12:18:44.605123 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/cd9a6dc3-a814-4a29-924a-d8ada756e7c7-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:18:44 crc kubenswrapper[5052]: I1203 12:18:44.605134 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/cd9a6dc3-a814-4a29-924a-d8ada756e7c7-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:18:44 crc kubenswrapper[5052]: I1203 12:18:44.629763 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage09-crc" (UniqueName: "kubernetes.io/local-volume/local-storage09-crc") on node "crc" Dec 03 12:18:44 crc kubenswrapper[5052]: I1203 12:18:44.706977 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage09-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage09-crc\") on node \"crc\" DevicePath \"\"" Dec 03 12:18:45 crc kubenswrapper[5052]: I1203 12:18:45.194553 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"cd9a6dc3-a814-4a29-924a-d8ada756e7c7","Type":"ContainerDied","Data":"bb61d38d070167697e611c63d76232f8d5328df5151f316456068d66eb87f80a"} Dec 03 12:18:45 crc kubenswrapper[5052]: I1203 12:18:45.194612 5052 scope.go:117] "RemoveContainer" containerID="9d3ac55b9014620975c68f93418691e10f6505723457d823dbfb3d7265965d17" Dec 03 12:18:45 crc kubenswrapper[5052]: I1203 12:18:45.194573 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:18:45 crc kubenswrapper[5052]: I1203 12:18:45.197659 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"e8433c1d-9f85-444b-b7bf-c2bd62fdbe8d","Type":"ContainerStarted","Data":"7166565c8875f3c04747c70ec6700450c93e64697c1138c561ef9fbe4d0bf894"} Dec 03 12:18:45 crc kubenswrapper[5052]: I1203 12:18:45.224003 5052 scope.go:117] "RemoveContainer" containerID="621073c1609087e450aafc5127e1c89c8e8d8de48573af61d7ee441bcd0685c7" Dec 03 12:18:45 crc kubenswrapper[5052]: I1203 12:18:45.235359 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:18:45 crc kubenswrapper[5052]: I1203 12:18:45.251071 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:18:45 crc kubenswrapper[5052]: I1203 12:18:45.263829 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:18:45 crc kubenswrapper[5052]: E1203 12:18:45.264316 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="cd9a6dc3-a814-4a29-924a-d8ada756e7c7" containerName="glance-log" Dec 03 12:18:45 crc kubenswrapper[5052]: I1203 12:18:45.264334 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="cd9a6dc3-a814-4a29-924a-d8ada756e7c7" containerName="glance-log" Dec 03 12:18:45 crc kubenswrapper[5052]: E1203 12:18:45.264354 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="8d0ffc74-5f47-4195-8091-b4890e55c32f" containerName="mariadb-account-create-update" Dec 03 12:18:45 crc kubenswrapper[5052]: I1203 12:18:45.264362 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="8d0ffc74-5f47-4195-8091-b4890e55c32f" containerName="mariadb-account-create-update" Dec 03 12:18:45 crc kubenswrapper[5052]: E1203 12:18:45.264379 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="0f021810-dfcd-4695-8330-380dfc297701" containerName="mariadb-database-create" Dec 03 12:18:45 crc kubenswrapper[5052]: I1203 12:18:45.264387 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="0f021810-dfcd-4695-8330-380dfc297701" containerName="mariadb-database-create" Dec 03 12:18:45 crc kubenswrapper[5052]: E1203 12:18:45.264400 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="cd9a6dc3-a814-4a29-924a-d8ada756e7c7" containerName="glance-httpd" Dec 03 12:18:45 crc kubenswrapper[5052]: I1203 12:18:45.264407 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="cd9a6dc3-a814-4a29-924a-d8ada756e7c7" containerName="glance-httpd" Dec 03 12:18:45 crc kubenswrapper[5052]: E1203 12:18:45.264423 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="03fcb214-234e-4e5a-8d9f-bbd2bbda66aa" containerName="mariadb-account-create-update" Dec 03 12:18:45 crc kubenswrapper[5052]: I1203 12:18:45.264432 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="03fcb214-234e-4e5a-8d9f-bbd2bbda66aa" containerName="mariadb-account-create-update" Dec 03 12:18:45 crc kubenswrapper[5052]: E1203 12:18:45.264453 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="dfbfcab9-41af-42e1-9ddf-5a05ffedb6c7" containerName="mariadb-database-create" Dec 03 12:18:45 crc kubenswrapper[5052]: I1203 12:18:45.264460 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="dfbfcab9-41af-42e1-9ddf-5a05ffedb6c7" containerName="mariadb-database-create" Dec 03 12:18:45 crc kubenswrapper[5052]: E1203 12:18:45.264481 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="f560f1ba-b0e0-4fb6-9802-bf1c9b726592" containerName="mariadb-account-create-update" Dec 03 12:18:45 crc kubenswrapper[5052]: I1203 12:18:45.264488 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="f560f1ba-b0e0-4fb6-9802-bf1c9b726592" containerName="mariadb-account-create-update" Dec 03 12:18:45 crc kubenswrapper[5052]: E1203 12:18:45.264498 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="c2e338a2-8424-4325-a580-55618dcd3c29" containerName="mariadb-database-create" Dec 03 12:18:45 crc kubenswrapper[5052]: I1203 12:18:45.264521 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="c2e338a2-8424-4325-a580-55618dcd3c29" containerName="mariadb-database-create" Dec 03 12:18:45 crc kubenswrapper[5052]: I1203 12:18:45.264740 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="0f021810-dfcd-4695-8330-380dfc297701" containerName="mariadb-database-create" Dec 03 12:18:45 crc kubenswrapper[5052]: I1203 12:18:45.264761 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="dfbfcab9-41af-42e1-9ddf-5a05ffedb6c7" containerName="mariadb-database-create" Dec 03 12:18:45 crc kubenswrapper[5052]: I1203 12:18:45.264776 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="f560f1ba-b0e0-4fb6-9802-bf1c9b726592" containerName="mariadb-account-create-update" Dec 03 12:18:45 crc kubenswrapper[5052]: I1203 12:18:45.264790 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="03fcb214-234e-4e5a-8d9f-bbd2bbda66aa" containerName="mariadb-account-create-update" Dec 03 12:18:45 crc kubenswrapper[5052]: I1203 12:18:45.264803 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="cd9a6dc3-a814-4a29-924a-d8ada756e7c7" containerName="glance-httpd" Dec 03 12:18:45 crc kubenswrapper[5052]: I1203 12:18:45.264815 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="8d0ffc74-5f47-4195-8091-b4890e55c32f" containerName="mariadb-account-create-update" Dec 03 12:18:45 crc kubenswrapper[5052]: I1203 12:18:45.264824 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="c2e338a2-8424-4325-a580-55618dcd3c29" containerName="mariadb-database-create" Dec 03 12:18:45 crc kubenswrapper[5052]: I1203 12:18:45.264838 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="cd9a6dc3-a814-4a29-924a-d8ada756e7c7" containerName="glance-log" Dec 03 12:18:45 crc kubenswrapper[5052]: I1203 12:18:45.266373 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:18:45 crc kubenswrapper[5052]: I1203 12:18:45.270016 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-glance-default-public-svc" Dec 03 12:18:45 crc kubenswrapper[5052]: I1203 12:18:45.270328 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-default-external-config-data" Dec 03 12:18:45 crc kubenswrapper[5052]: I1203 12:18:45.280792 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:18:45 crc kubenswrapper[5052]: I1203 12:18:45.419565 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/736b6178-1b32-4f5e-a207-931b03aed990-public-tls-certs\") pod \"glance-default-external-api-0\" (UID: \"736b6178-1b32-4f5e-a207-931b03aed990\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:18:45 crc kubenswrapper[5052]: I1203 12:18:45.419628 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fb5tp\" (UniqueName: \"kubernetes.io/projected/736b6178-1b32-4f5e-a207-931b03aed990-kube-api-access-fb5tp\") pod \"glance-default-external-api-0\" (UID: \"736b6178-1b32-4f5e-a207-931b03aed990\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:18:45 crc kubenswrapper[5052]: I1203 12:18:45.419650 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/736b6178-1b32-4f5e-a207-931b03aed990-scripts\") pod \"glance-default-external-api-0\" (UID: \"736b6178-1b32-4f5e-a207-931b03aed990\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:18:45 crc kubenswrapper[5052]: I1203 12:18:45.419679 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/736b6178-1b32-4f5e-a207-931b03aed990-config-data\") pod \"glance-default-external-api-0\" (UID: \"736b6178-1b32-4f5e-a207-931b03aed990\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:18:45 crc kubenswrapper[5052]: I1203 12:18:45.419710 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/736b6178-1b32-4f5e-a207-931b03aed990-logs\") pod \"glance-default-external-api-0\" (UID: \"736b6178-1b32-4f5e-a207-931b03aed990\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:18:45 crc kubenswrapper[5052]: I1203 12:18:45.419733 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/736b6178-1b32-4f5e-a207-931b03aed990-httpd-run\") pod \"glance-default-external-api-0\" (UID: \"736b6178-1b32-4f5e-a207-931b03aed990\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:18:45 crc kubenswrapper[5052]: I1203 12:18:45.419764 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/736b6178-1b32-4f5e-a207-931b03aed990-combined-ca-bundle\") pod \"glance-default-external-api-0\" (UID: \"736b6178-1b32-4f5e-a207-931b03aed990\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:18:45 crc kubenswrapper[5052]: I1203 12:18:45.419809 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage09-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage09-crc\") pod \"glance-default-external-api-0\" (UID: \"736b6178-1b32-4f5e-a207-931b03aed990\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:18:45 crc kubenswrapper[5052]: I1203 12:18:45.522107 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/736b6178-1b32-4f5e-a207-931b03aed990-logs\") pod \"glance-default-external-api-0\" (UID: \"736b6178-1b32-4f5e-a207-931b03aed990\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:18:45 crc kubenswrapper[5052]: I1203 12:18:45.522161 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/736b6178-1b32-4f5e-a207-931b03aed990-httpd-run\") pod \"glance-default-external-api-0\" (UID: \"736b6178-1b32-4f5e-a207-931b03aed990\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:18:45 crc kubenswrapper[5052]: I1203 12:18:45.522195 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/736b6178-1b32-4f5e-a207-931b03aed990-combined-ca-bundle\") pod \"glance-default-external-api-0\" (UID: \"736b6178-1b32-4f5e-a207-931b03aed990\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:18:45 crc kubenswrapper[5052]: I1203 12:18:45.522240 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage09-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage09-crc\") pod \"glance-default-external-api-0\" (UID: \"736b6178-1b32-4f5e-a207-931b03aed990\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:18:45 crc kubenswrapper[5052]: I1203 12:18:45.522390 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/736b6178-1b32-4f5e-a207-931b03aed990-public-tls-certs\") pod \"glance-default-external-api-0\" (UID: \"736b6178-1b32-4f5e-a207-931b03aed990\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:18:45 crc kubenswrapper[5052]: I1203 12:18:45.522422 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-fb5tp\" (UniqueName: \"kubernetes.io/projected/736b6178-1b32-4f5e-a207-931b03aed990-kube-api-access-fb5tp\") pod \"glance-default-external-api-0\" (UID: \"736b6178-1b32-4f5e-a207-931b03aed990\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:18:45 crc kubenswrapper[5052]: I1203 12:18:45.522447 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/736b6178-1b32-4f5e-a207-931b03aed990-scripts\") pod \"glance-default-external-api-0\" (UID: \"736b6178-1b32-4f5e-a207-931b03aed990\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:18:45 crc kubenswrapper[5052]: I1203 12:18:45.522477 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/736b6178-1b32-4f5e-a207-931b03aed990-config-data\") pod \"glance-default-external-api-0\" (UID: \"736b6178-1b32-4f5e-a207-931b03aed990\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:18:45 crc kubenswrapper[5052]: I1203 12:18:45.522766 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage09-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage09-crc\") pod \"glance-default-external-api-0\" (UID: \"736b6178-1b32-4f5e-a207-931b03aed990\") device mount path \"/mnt/openstack/pv09\"" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:18:45 crc kubenswrapper[5052]: I1203 12:18:45.523036 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/736b6178-1b32-4f5e-a207-931b03aed990-logs\") pod \"glance-default-external-api-0\" (UID: \"736b6178-1b32-4f5e-a207-931b03aed990\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:18:45 crc kubenswrapper[5052]: I1203 12:18:45.523409 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/736b6178-1b32-4f5e-a207-931b03aed990-httpd-run\") pod \"glance-default-external-api-0\" (UID: \"736b6178-1b32-4f5e-a207-931b03aed990\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:18:45 crc kubenswrapper[5052]: I1203 12:18:45.527233 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/736b6178-1b32-4f5e-a207-931b03aed990-scripts\") pod \"glance-default-external-api-0\" (UID: \"736b6178-1b32-4f5e-a207-931b03aed990\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:18:45 crc kubenswrapper[5052]: I1203 12:18:45.527777 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/736b6178-1b32-4f5e-a207-931b03aed990-public-tls-certs\") pod \"glance-default-external-api-0\" (UID: \"736b6178-1b32-4f5e-a207-931b03aed990\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:18:45 crc kubenswrapper[5052]: I1203 12:18:45.527947 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/736b6178-1b32-4f5e-a207-931b03aed990-combined-ca-bundle\") pod \"glance-default-external-api-0\" (UID: \"736b6178-1b32-4f5e-a207-931b03aed990\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:18:45 crc kubenswrapper[5052]: I1203 12:18:45.531624 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/736b6178-1b32-4f5e-a207-931b03aed990-config-data\") pod \"glance-default-external-api-0\" (UID: \"736b6178-1b32-4f5e-a207-931b03aed990\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:18:45 crc kubenswrapper[5052]: I1203 12:18:45.541354 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-fb5tp\" (UniqueName: \"kubernetes.io/projected/736b6178-1b32-4f5e-a207-931b03aed990-kube-api-access-fb5tp\") pod \"glance-default-external-api-0\" (UID: \"736b6178-1b32-4f5e-a207-931b03aed990\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:18:45 crc kubenswrapper[5052]: I1203 12:18:45.552547 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage09-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage09-crc\") pod \"glance-default-external-api-0\" (UID: \"736b6178-1b32-4f5e-a207-931b03aed990\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:18:45 crc kubenswrapper[5052]: I1203 12:18:45.626165 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:18:45 crc kubenswrapper[5052]: I1203 12:18:45.686076 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cd9a6dc3-a814-4a29-924a-d8ada756e7c7" path="/var/lib/kubelet/pods/cd9a6dc3-a814-4a29-924a-d8ada756e7c7/volumes" Dec 03 12:18:46 crc kubenswrapper[5052]: I1203 12:18:46.087217 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:18:46 crc kubenswrapper[5052]: W1203 12:18:46.091779 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod736b6178_1b32_4f5e_a207_931b03aed990.slice/crio-eaad6a6e236c45540e07e3ae81242576acf2408c9a149279047c9fa917b7baee WatchSource:0}: Error finding container eaad6a6e236c45540e07e3ae81242576acf2408c9a149279047c9fa917b7baee: Status 404 returned error can't find the container with id eaad6a6e236c45540e07e3ae81242576acf2408c9a149279047c9fa917b7baee Dec 03 12:18:46 crc kubenswrapper[5052]: I1203 12:18:46.213412 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"736b6178-1b32-4f5e-a207-931b03aed990","Type":"ContainerStarted","Data":"eaad6a6e236c45540e07e3ae81242576acf2408c9a149279047c9fa917b7baee"} Dec 03 12:18:46 crc kubenswrapper[5052]: I1203 12:18:46.218580 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"e8433c1d-9f85-444b-b7bf-c2bd62fdbe8d","Type":"ContainerStarted","Data":"01a88ea407817720df9018415facbf62f01f27ca78e05870124a9055f16b2dc8"} Dec 03 12:18:47 crc kubenswrapper[5052]: I1203 12:18:47.234797 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"736b6178-1b32-4f5e-a207-931b03aed990","Type":"ContainerStarted","Data":"08ba12ac062944881ab39ae9fbebc472a569ed1780e7c9436d3f4c6a4444d057"} Dec 03 12:18:47 crc kubenswrapper[5052]: I1203 12:18:47.234850 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"736b6178-1b32-4f5e-a207-931b03aed990","Type":"ContainerStarted","Data":"e480cf5faa382d0f7dc902f72f5fa5a78a13cd4fac390962daf40b8d0bee88d7"} Dec 03 12:18:47 crc kubenswrapper[5052]: I1203 12:18:47.260337 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/glance-default-external-api-0" podStartSLOduration=2.26031438 podStartE2EDuration="2.26031438s" podCreationTimestamp="2025-12-03 12:18:45 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:18:47.257824769 +0000 UTC m=+3517.846066533" watchObservedRunningTime="2025-12-03 12:18:47.26031438 +0000 UTC m=+3517.848556154" Dec 03 12:18:47 crc kubenswrapper[5052]: I1203 12:18:47.362629 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:18:47 crc kubenswrapper[5052]: I1203 12:18:47.362703 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:18:47 crc kubenswrapper[5052]: I1203 12:18:47.403637 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:18:47 crc kubenswrapper[5052]: I1203 12:18:47.406394 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:18:48 crc kubenswrapper[5052]: I1203 12:18:48.012786 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:18:48 crc kubenswrapper[5052]: I1203 12:18:48.246626 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"e8433c1d-9f85-444b-b7bf-c2bd62fdbe8d","Type":"ContainerStarted","Data":"9851d96050f063049f438f33dbc80fc83617385636dd5d0105a048a483ddd04c"} Dec 03 12:18:48 crc kubenswrapper[5052]: I1203 12:18:48.247059 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:18:48 crc kubenswrapper[5052]: I1203 12:18:48.247073 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:18:48 crc kubenswrapper[5052]: I1203 12:18:48.277588 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/ceilometer-0" podStartSLOduration=2.213352252 podStartE2EDuration="6.277570894s" podCreationTimestamp="2025-12-03 12:18:42 +0000 UTC" firstStartedPulling="2025-12-03 12:18:43.077523944 +0000 UTC m=+3513.665765708" lastFinishedPulling="2025-12-03 12:18:47.141742586 +0000 UTC m=+3517.729984350" observedRunningTime="2025-12-03 12:18:48.268933847 +0000 UTC m=+3518.857175611" watchObservedRunningTime="2025-12-03 12:18:48.277570894 +0000 UTC m=+3518.865812658" Dec 03 12:18:48 crc kubenswrapper[5052]: I1203 12:18:48.690965 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-db-sync-j4cfx"] Dec 03 12:18:48 crc kubenswrapper[5052]: I1203 12:18:48.692134 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-j4cfx" Dec 03 12:18:48 crc kubenswrapper[5052]: I1203 12:18:48.694391 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell0-conductor-scripts" Dec 03 12:18:48 crc kubenswrapper[5052]: I1203 12:18:48.703626 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell0-conductor-config-data" Dec 03 12:18:48 crc kubenswrapper[5052]: I1203 12:18:48.704612 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-nova-dockercfg-6vqct" Dec 03 12:18:48 crc kubenswrapper[5052]: I1203 12:18:48.714749 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-db-sync-j4cfx"] Dec 03 12:18:48 crc kubenswrapper[5052]: I1203 12:18:48.785766 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/8a3bbadd-37b5-4328-ae01-657aa8eb3028-config-data\") pod \"nova-cell0-conductor-db-sync-j4cfx\" (UID: \"8a3bbadd-37b5-4328-ae01-657aa8eb3028\") " pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-j4cfx" Dec 03 12:18:48 crc kubenswrapper[5052]: I1203 12:18:48.785853 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h8hcl\" (UniqueName: \"kubernetes.io/projected/8a3bbadd-37b5-4328-ae01-657aa8eb3028-kube-api-access-h8hcl\") pod \"nova-cell0-conductor-db-sync-j4cfx\" (UID: \"8a3bbadd-37b5-4328-ae01-657aa8eb3028\") " pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-j4cfx" Dec 03 12:18:48 crc kubenswrapper[5052]: I1203 12:18:48.785878 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/8a3bbadd-37b5-4328-ae01-657aa8eb3028-scripts\") pod \"nova-cell0-conductor-db-sync-j4cfx\" (UID: \"8a3bbadd-37b5-4328-ae01-657aa8eb3028\") " pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-j4cfx" Dec 03 12:18:48 crc kubenswrapper[5052]: I1203 12:18:48.785936 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8a3bbadd-37b5-4328-ae01-657aa8eb3028-combined-ca-bundle\") pod \"nova-cell0-conductor-db-sync-j4cfx\" (UID: \"8a3bbadd-37b5-4328-ae01-657aa8eb3028\") " pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-j4cfx" Dec 03 12:18:48 crc kubenswrapper[5052]: I1203 12:18:48.887352 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-h8hcl\" (UniqueName: \"kubernetes.io/projected/8a3bbadd-37b5-4328-ae01-657aa8eb3028-kube-api-access-h8hcl\") pod \"nova-cell0-conductor-db-sync-j4cfx\" (UID: \"8a3bbadd-37b5-4328-ae01-657aa8eb3028\") " pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-j4cfx" Dec 03 12:18:48 crc kubenswrapper[5052]: I1203 12:18:48.887401 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/8a3bbadd-37b5-4328-ae01-657aa8eb3028-scripts\") pod \"nova-cell0-conductor-db-sync-j4cfx\" (UID: \"8a3bbadd-37b5-4328-ae01-657aa8eb3028\") " pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-j4cfx" Dec 03 12:18:48 crc kubenswrapper[5052]: I1203 12:18:48.887489 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8a3bbadd-37b5-4328-ae01-657aa8eb3028-combined-ca-bundle\") pod \"nova-cell0-conductor-db-sync-j4cfx\" (UID: \"8a3bbadd-37b5-4328-ae01-657aa8eb3028\") " pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-j4cfx" Dec 03 12:18:48 crc kubenswrapper[5052]: I1203 12:18:48.888532 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/8a3bbadd-37b5-4328-ae01-657aa8eb3028-config-data\") pod \"nova-cell0-conductor-db-sync-j4cfx\" (UID: \"8a3bbadd-37b5-4328-ae01-657aa8eb3028\") " pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-j4cfx" Dec 03 12:18:48 crc kubenswrapper[5052]: I1203 12:18:48.894249 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8a3bbadd-37b5-4328-ae01-657aa8eb3028-combined-ca-bundle\") pod \"nova-cell0-conductor-db-sync-j4cfx\" (UID: \"8a3bbadd-37b5-4328-ae01-657aa8eb3028\") " pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-j4cfx" Dec 03 12:18:48 crc kubenswrapper[5052]: I1203 12:18:48.895445 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/8a3bbadd-37b5-4328-ae01-657aa8eb3028-config-data\") pod \"nova-cell0-conductor-db-sync-j4cfx\" (UID: \"8a3bbadd-37b5-4328-ae01-657aa8eb3028\") " pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-j4cfx" Dec 03 12:18:48 crc kubenswrapper[5052]: I1203 12:18:48.896981 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/8a3bbadd-37b5-4328-ae01-657aa8eb3028-scripts\") pod \"nova-cell0-conductor-db-sync-j4cfx\" (UID: \"8a3bbadd-37b5-4328-ae01-657aa8eb3028\") " pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-j4cfx" Dec 03 12:18:48 crc kubenswrapper[5052]: I1203 12:18:48.912031 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-h8hcl\" (UniqueName: \"kubernetes.io/projected/8a3bbadd-37b5-4328-ae01-657aa8eb3028-kube-api-access-h8hcl\") pod \"nova-cell0-conductor-db-sync-j4cfx\" (UID: \"8a3bbadd-37b5-4328-ae01-657aa8eb3028\") " pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-j4cfx" Dec 03 12:18:49 crc kubenswrapper[5052]: I1203 12:18:49.014637 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-j4cfx" Dec 03 12:18:49 crc kubenswrapper[5052]: I1203 12:18:49.262537 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:18:49 crc kubenswrapper[5052]: I1203 12:18:49.262827 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="e8433c1d-9f85-444b-b7bf-c2bd62fdbe8d" containerName="ceilometer-central-agent" containerID="cri-o://7bcf29528a2630e2584876eb33663faf73c9660396ef93d28e491b29c246425e" gracePeriod=30 Dec 03 12:18:49 crc kubenswrapper[5052]: I1203 12:18:49.263088 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="e8433c1d-9f85-444b-b7bf-c2bd62fdbe8d" containerName="proxy-httpd" containerID="cri-o://9851d96050f063049f438f33dbc80fc83617385636dd5d0105a048a483ddd04c" gracePeriod=30 Dec 03 12:18:49 crc kubenswrapper[5052]: I1203 12:18:49.263139 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="e8433c1d-9f85-444b-b7bf-c2bd62fdbe8d" containerName="sg-core" containerID="cri-o://01a88ea407817720df9018415facbf62f01f27ca78e05870124a9055f16b2dc8" gracePeriod=30 Dec 03 12:18:49 crc kubenswrapper[5052]: I1203 12:18:49.263184 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="e8433c1d-9f85-444b-b7bf-c2bd62fdbe8d" containerName="ceilometer-notification-agent" containerID="cri-o://7166565c8875f3c04747c70ec6700450c93e64697c1138c561ef9fbe4d0bf894" gracePeriod=30 Dec 03 12:18:49 crc kubenswrapper[5052]: I1203 12:18:49.451059 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-db-sync-j4cfx"] Dec 03 12:18:49 crc kubenswrapper[5052]: W1203 12:18:49.458736 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8a3bbadd_37b5_4328_ae01_657aa8eb3028.slice/crio-ba7b32c7b0f64968162590dda93cc4c915be936affbe9eefd18f8d408c226f63 WatchSource:0}: Error finding container ba7b32c7b0f64968162590dda93cc4c915be936affbe9eefd18f8d408c226f63: Status 404 returned error can't find the container with id ba7b32c7b0f64968162590dda93cc4c915be936affbe9eefd18f8d408c226f63 Dec 03 12:18:49 crc kubenswrapper[5052]: E1203 12:18:49.672978 5052 cadvisor_stats_provider.go:516] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode8433c1d_9f85_444b_b7bf_c2bd62fdbe8d.slice/crio-9851d96050f063049f438f33dbc80fc83617385636dd5d0105a048a483ddd04c.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode8433c1d_9f85_444b_b7bf_c2bd62fdbe8d.slice/crio-conmon-01a88ea407817720df9018415facbf62f01f27ca78e05870124a9055f16b2dc8.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode8433c1d_9f85_444b_b7bf_c2bd62fdbe8d.slice/crio-7166565c8875f3c04747c70ec6700450c93e64697c1138c561ef9fbe4d0bf894.scope\": RecentStats: unable to find data in memory cache]" Dec 03 12:18:50 crc kubenswrapper[5052]: I1203 12:18:50.286219 5052 generic.go:334] "Generic (PLEG): container finished" podID="e8433c1d-9f85-444b-b7bf-c2bd62fdbe8d" containerID="9851d96050f063049f438f33dbc80fc83617385636dd5d0105a048a483ddd04c" exitCode=0 Dec 03 12:18:50 crc kubenswrapper[5052]: I1203 12:18:50.286537 5052 generic.go:334] "Generic (PLEG): container finished" podID="e8433c1d-9f85-444b-b7bf-c2bd62fdbe8d" containerID="01a88ea407817720df9018415facbf62f01f27ca78e05870124a9055f16b2dc8" exitCode=2 Dec 03 12:18:50 crc kubenswrapper[5052]: I1203 12:18:50.286548 5052 generic.go:334] "Generic (PLEG): container finished" podID="e8433c1d-9f85-444b-b7bf-c2bd62fdbe8d" containerID="7166565c8875f3c04747c70ec6700450c93e64697c1138c561ef9fbe4d0bf894" exitCode=0 Dec 03 12:18:50 crc kubenswrapper[5052]: I1203 12:18:50.286328 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"e8433c1d-9f85-444b-b7bf-c2bd62fdbe8d","Type":"ContainerDied","Data":"9851d96050f063049f438f33dbc80fc83617385636dd5d0105a048a483ddd04c"} Dec 03 12:18:50 crc kubenswrapper[5052]: I1203 12:18:50.286638 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"e8433c1d-9f85-444b-b7bf-c2bd62fdbe8d","Type":"ContainerDied","Data":"01a88ea407817720df9018415facbf62f01f27ca78e05870124a9055f16b2dc8"} Dec 03 12:18:50 crc kubenswrapper[5052]: I1203 12:18:50.286662 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"e8433c1d-9f85-444b-b7bf-c2bd62fdbe8d","Type":"ContainerDied","Data":"7166565c8875f3c04747c70ec6700450c93e64697c1138c561ef9fbe4d0bf894"} Dec 03 12:18:50 crc kubenswrapper[5052]: I1203 12:18:50.288284 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-j4cfx" event={"ID":"8a3bbadd-37b5-4328-ae01-657aa8eb3028","Type":"ContainerStarted","Data":"6c1ba05ae9fa6187bf00e4c104305fd60275127701d6a185a2709d3cdff50ab1"} Dec 03 12:18:50 crc kubenswrapper[5052]: I1203 12:18:50.288344 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-j4cfx" event={"ID":"8a3bbadd-37b5-4328-ae01-657aa8eb3028","Type":"ContainerStarted","Data":"ba7b32c7b0f64968162590dda93cc4c915be936affbe9eefd18f8d408c226f63"} Dec 03 12:18:50 crc kubenswrapper[5052]: I1203 12:18:50.308391 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-j4cfx" podStartSLOduration=2.308367908 podStartE2EDuration="2.308367908s" podCreationTimestamp="2025-12-03 12:18:48 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:18:50.304313142 +0000 UTC m=+3520.892554906" watchObservedRunningTime="2025-12-03 12:18:50.308367908 +0000 UTC m=+3520.896609692" Dec 03 12:18:50 crc kubenswrapper[5052]: I1203 12:18:50.373240 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:18:50 crc kubenswrapper[5052]: I1203 12:18:50.373358 5052 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Dec 03 12:18:50 crc kubenswrapper[5052]: I1203 12:18:50.377114 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:18:54 crc kubenswrapper[5052]: I1203 12:18:54.134024 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:18:54 crc kubenswrapper[5052]: I1203 12:18:54.205788 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e8433c1d-9f85-444b-b7bf-c2bd62fdbe8d-combined-ca-bundle\") pod \"e8433c1d-9f85-444b-b7bf-c2bd62fdbe8d\" (UID: \"e8433c1d-9f85-444b-b7bf-c2bd62fdbe8d\") " Dec 03 12:18:54 crc kubenswrapper[5052]: I1203 12:18:54.206930 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/e8433c1d-9f85-444b-b7bf-c2bd62fdbe8d-log-httpd\") pod \"e8433c1d-9f85-444b-b7bf-c2bd62fdbe8d\" (UID: \"e8433c1d-9f85-444b-b7bf-c2bd62fdbe8d\") " Dec 03 12:18:54 crc kubenswrapper[5052]: I1203 12:18:54.207081 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/e8433c1d-9f85-444b-b7bf-c2bd62fdbe8d-run-httpd\") pod \"e8433c1d-9f85-444b-b7bf-c2bd62fdbe8d\" (UID: \"e8433c1d-9f85-444b-b7bf-c2bd62fdbe8d\") " Dec 03 12:18:54 crc kubenswrapper[5052]: I1203 12:18:54.207218 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/e8433c1d-9f85-444b-b7bf-c2bd62fdbe8d-scripts\") pod \"e8433c1d-9f85-444b-b7bf-c2bd62fdbe8d\" (UID: \"e8433c1d-9f85-444b-b7bf-c2bd62fdbe8d\") " Dec 03 12:18:54 crc kubenswrapper[5052]: I1203 12:18:54.207317 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/e8433c1d-9f85-444b-b7bf-c2bd62fdbe8d-config-data\") pod \"e8433c1d-9f85-444b-b7bf-c2bd62fdbe8d\" (UID: \"e8433c1d-9f85-444b-b7bf-c2bd62fdbe8d\") " Dec 03 12:18:54 crc kubenswrapper[5052]: I1203 12:18:54.207407 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-flxgv\" (UniqueName: \"kubernetes.io/projected/e8433c1d-9f85-444b-b7bf-c2bd62fdbe8d-kube-api-access-flxgv\") pod \"e8433c1d-9f85-444b-b7bf-c2bd62fdbe8d\" (UID: \"e8433c1d-9f85-444b-b7bf-c2bd62fdbe8d\") " Dec 03 12:18:54 crc kubenswrapper[5052]: I1203 12:18:54.207745 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/e8433c1d-9f85-444b-b7bf-c2bd62fdbe8d-sg-core-conf-yaml\") pod \"e8433c1d-9f85-444b-b7bf-c2bd62fdbe8d\" (UID: \"e8433c1d-9f85-444b-b7bf-c2bd62fdbe8d\") " Dec 03 12:18:54 crc kubenswrapper[5052]: I1203 12:18:54.207745 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e8433c1d-9f85-444b-b7bf-c2bd62fdbe8d-log-httpd" (OuterVolumeSpecName: "log-httpd") pod "e8433c1d-9f85-444b-b7bf-c2bd62fdbe8d" (UID: "e8433c1d-9f85-444b-b7bf-c2bd62fdbe8d"). InnerVolumeSpecName "log-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:18:54 crc kubenswrapper[5052]: I1203 12:18:54.208023 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e8433c1d-9f85-444b-b7bf-c2bd62fdbe8d-run-httpd" (OuterVolumeSpecName: "run-httpd") pod "e8433c1d-9f85-444b-b7bf-c2bd62fdbe8d" (UID: "e8433c1d-9f85-444b-b7bf-c2bd62fdbe8d"). InnerVolumeSpecName "run-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:18:54 crc kubenswrapper[5052]: I1203 12:18:54.215783 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e8433c1d-9f85-444b-b7bf-c2bd62fdbe8d-kube-api-access-flxgv" (OuterVolumeSpecName: "kube-api-access-flxgv") pod "e8433c1d-9f85-444b-b7bf-c2bd62fdbe8d" (UID: "e8433c1d-9f85-444b-b7bf-c2bd62fdbe8d"). InnerVolumeSpecName "kube-api-access-flxgv". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:18:54 crc kubenswrapper[5052]: I1203 12:18:54.229649 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e8433c1d-9f85-444b-b7bf-c2bd62fdbe8d-scripts" (OuterVolumeSpecName: "scripts") pod "e8433c1d-9f85-444b-b7bf-c2bd62fdbe8d" (UID: "e8433c1d-9f85-444b-b7bf-c2bd62fdbe8d"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:18:54 crc kubenswrapper[5052]: I1203 12:18:54.239839 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e8433c1d-9f85-444b-b7bf-c2bd62fdbe8d-sg-core-conf-yaml" (OuterVolumeSpecName: "sg-core-conf-yaml") pod "e8433c1d-9f85-444b-b7bf-c2bd62fdbe8d" (UID: "e8433c1d-9f85-444b-b7bf-c2bd62fdbe8d"). InnerVolumeSpecName "sg-core-conf-yaml". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:18:54 crc kubenswrapper[5052]: I1203 12:18:54.282259 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e8433c1d-9f85-444b-b7bf-c2bd62fdbe8d-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "e8433c1d-9f85-444b-b7bf-c2bd62fdbe8d" (UID: "e8433c1d-9f85-444b-b7bf-c2bd62fdbe8d"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:18:54 crc kubenswrapper[5052]: I1203 12:18:54.310018 5052 reconciler_common.go:293] "Volume detached for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/e8433c1d-9f85-444b-b7bf-c2bd62fdbe8d-log-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:18:54 crc kubenswrapper[5052]: I1203 12:18:54.310049 5052 reconciler_common.go:293] "Volume detached for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/e8433c1d-9f85-444b-b7bf-c2bd62fdbe8d-run-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:18:54 crc kubenswrapper[5052]: I1203 12:18:54.310057 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/e8433c1d-9f85-444b-b7bf-c2bd62fdbe8d-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:18:54 crc kubenswrapper[5052]: I1203 12:18:54.310068 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-flxgv\" (UniqueName: \"kubernetes.io/projected/e8433c1d-9f85-444b-b7bf-c2bd62fdbe8d-kube-api-access-flxgv\") on node \"crc\" DevicePath \"\"" Dec 03 12:18:54 crc kubenswrapper[5052]: I1203 12:18:54.310077 5052 reconciler_common.go:293] "Volume detached for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/e8433c1d-9f85-444b-b7bf-c2bd62fdbe8d-sg-core-conf-yaml\") on node \"crc\" DevicePath \"\"" Dec 03 12:18:54 crc kubenswrapper[5052]: I1203 12:18:54.310086 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e8433c1d-9f85-444b-b7bf-c2bd62fdbe8d-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:18:54 crc kubenswrapper[5052]: I1203 12:18:54.314931 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e8433c1d-9f85-444b-b7bf-c2bd62fdbe8d-config-data" (OuterVolumeSpecName: "config-data") pod "e8433c1d-9f85-444b-b7bf-c2bd62fdbe8d" (UID: "e8433c1d-9f85-444b-b7bf-c2bd62fdbe8d"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:18:54 crc kubenswrapper[5052]: I1203 12:18:54.333250 5052 generic.go:334] "Generic (PLEG): container finished" podID="e8433c1d-9f85-444b-b7bf-c2bd62fdbe8d" containerID="7bcf29528a2630e2584876eb33663faf73c9660396ef93d28e491b29c246425e" exitCode=0 Dec 03 12:18:54 crc kubenswrapper[5052]: I1203 12:18:54.333287 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:18:54 crc kubenswrapper[5052]: I1203 12:18:54.333281 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"e8433c1d-9f85-444b-b7bf-c2bd62fdbe8d","Type":"ContainerDied","Data":"7bcf29528a2630e2584876eb33663faf73c9660396ef93d28e491b29c246425e"} Dec 03 12:18:54 crc kubenswrapper[5052]: I1203 12:18:54.333599 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"e8433c1d-9f85-444b-b7bf-c2bd62fdbe8d","Type":"ContainerDied","Data":"7e9652ea1e1261d5b6c7936e3301027974dfa87b069906329efcaa0f7a1206b5"} Dec 03 12:18:54 crc kubenswrapper[5052]: I1203 12:18:54.333638 5052 scope.go:117] "RemoveContainer" containerID="9851d96050f063049f438f33dbc80fc83617385636dd5d0105a048a483ddd04c" Dec 03 12:18:54 crc kubenswrapper[5052]: I1203 12:18:54.355339 5052 scope.go:117] "RemoveContainer" containerID="01a88ea407817720df9018415facbf62f01f27ca78e05870124a9055f16b2dc8" Dec 03 12:18:54 crc kubenswrapper[5052]: I1203 12:18:54.388837 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:18:54 crc kubenswrapper[5052]: I1203 12:18:54.399046 5052 scope.go:117] "RemoveContainer" containerID="7166565c8875f3c04747c70ec6700450c93e64697c1138c561ef9fbe4d0bf894" Dec 03 12:18:54 crc kubenswrapper[5052]: I1203 12:18:54.405290 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:18:54 crc kubenswrapper[5052]: I1203 12:18:54.414788 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/e8433c1d-9f85-444b-b7bf-c2bd62fdbe8d-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:18:54 crc kubenswrapper[5052]: I1203 12:18:54.417246 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:18:54 crc kubenswrapper[5052]: E1203 12:18:54.417723 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="e8433c1d-9f85-444b-b7bf-c2bd62fdbe8d" containerName="proxy-httpd" Dec 03 12:18:54 crc kubenswrapper[5052]: I1203 12:18:54.417736 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="e8433c1d-9f85-444b-b7bf-c2bd62fdbe8d" containerName="proxy-httpd" Dec 03 12:18:54 crc kubenswrapper[5052]: E1203 12:18:54.417754 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="e8433c1d-9f85-444b-b7bf-c2bd62fdbe8d" containerName="ceilometer-notification-agent" Dec 03 12:18:54 crc kubenswrapper[5052]: I1203 12:18:54.417761 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="e8433c1d-9f85-444b-b7bf-c2bd62fdbe8d" containerName="ceilometer-notification-agent" Dec 03 12:18:54 crc kubenswrapper[5052]: E1203 12:18:54.417782 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="e8433c1d-9f85-444b-b7bf-c2bd62fdbe8d" containerName="sg-core" Dec 03 12:18:54 crc kubenswrapper[5052]: I1203 12:18:54.417790 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="e8433c1d-9f85-444b-b7bf-c2bd62fdbe8d" containerName="sg-core" Dec 03 12:18:54 crc kubenswrapper[5052]: E1203 12:18:54.417814 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="e8433c1d-9f85-444b-b7bf-c2bd62fdbe8d" containerName="ceilometer-central-agent" Dec 03 12:18:54 crc kubenswrapper[5052]: I1203 12:18:54.417820 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="e8433c1d-9f85-444b-b7bf-c2bd62fdbe8d" containerName="ceilometer-central-agent" Dec 03 12:18:54 crc kubenswrapper[5052]: I1203 12:18:54.418003 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="e8433c1d-9f85-444b-b7bf-c2bd62fdbe8d" containerName="ceilometer-notification-agent" Dec 03 12:18:54 crc kubenswrapper[5052]: I1203 12:18:54.418012 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="e8433c1d-9f85-444b-b7bf-c2bd62fdbe8d" containerName="proxy-httpd" Dec 03 12:18:54 crc kubenswrapper[5052]: I1203 12:18:54.418026 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="e8433c1d-9f85-444b-b7bf-c2bd62fdbe8d" containerName="ceilometer-central-agent" Dec 03 12:18:54 crc kubenswrapper[5052]: I1203 12:18:54.418039 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="e8433c1d-9f85-444b-b7bf-c2bd62fdbe8d" containerName="sg-core" Dec 03 12:18:54 crc kubenswrapper[5052]: I1203 12:18:54.425149 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:18:54 crc kubenswrapper[5052]: I1203 12:18:54.427847 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:18:54 crc kubenswrapper[5052]: I1203 12:18:54.431055 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-scripts" Dec 03 12:18:54 crc kubenswrapper[5052]: I1203 12:18:54.431353 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-config-data" Dec 03 12:18:54 crc kubenswrapper[5052]: I1203 12:18:54.431637 5052 scope.go:117] "RemoveContainer" containerID="7bcf29528a2630e2584876eb33663faf73c9660396ef93d28e491b29c246425e" Dec 03 12:18:54 crc kubenswrapper[5052]: I1203 12:18:54.465727 5052 scope.go:117] "RemoveContainer" containerID="9851d96050f063049f438f33dbc80fc83617385636dd5d0105a048a483ddd04c" Dec 03 12:18:54 crc kubenswrapper[5052]: E1203 12:18:54.467051 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9851d96050f063049f438f33dbc80fc83617385636dd5d0105a048a483ddd04c\": container with ID starting with 9851d96050f063049f438f33dbc80fc83617385636dd5d0105a048a483ddd04c not found: ID does not exist" containerID="9851d96050f063049f438f33dbc80fc83617385636dd5d0105a048a483ddd04c" Dec 03 12:18:54 crc kubenswrapper[5052]: I1203 12:18:54.467149 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9851d96050f063049f438f33dbc80fc83617385636dd5d0105a048a483ddd04c"} err="failed to get container status \"9851d96050f063049f438f33dbc80fc83617385636dd5d0105a048a483ddd04c\": rpc error: code = NotFound desc = could not find container \"9851d96050f063049f438f33dbc80fc83617385636dd5d0105a048a483ddd04c\": container with ID starting with 9851d96050f063049f438f33dbc80fc83617385636dd5d0105a048a483ddd04c not found: ID does not exist" Dec 03 12:18:54 crc kubenswrapper[5052]: I1203 12:18:54.467210 5052 scope.go:117] "RemoveContainer" containerID="01a88ea407817720df9018415facbf62f01f27ca78e05870124a9055f16b2dc8" Dec 03 12:18:54 crc kubenswrapper[5052]: E1203 12:18:54.467689 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"01a88ea407817720df9018415facbf62f01f27ca78e05870124a9055f16b2dc8\": container with ID starting with 01a88ea407817720df9018415facbf62f01f27ca78e05870124a9055f16b2dc8 not found: ID does not exist" containerID="01a88ea407817720df9018415facbf62f01f27ca78e05870124a9055f16b2dc8" Dec 03 12:18:54 crc kubenswrapper[5052]: I1203 12:18:54.467723 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"01a88ea407817720df9018415facbf62f01f27ca78e05870124a9055f16b2dc8"} err="failed to get container status \"01a88ea407817720df9018415facbf62f01f27ca78e05870124a9055f16b2dc8\": rpc error: code = NotFound desc = could not find container \"01a88ea407817720df9018415facbf62f01f27ca78e05870124a9055f16b2dc8\": container with ID starting with 01a88ea407817720df9018415facbf62f01f27ca78e05870124a9055f16b2dc8 not found: ID does not exist" Dec 03 12:18:54 crc kubenswrapper[5052]: I1203 12:18:54.467741 5052 scope.go:117] "RemoveContainer" containerID="7166565c8875f3c04747c70ec6700450c93e64697c1138c561ef9fbe4d0bf894" Dec 03 12:18:54 crc kubenswrapper[5052]: E1203 12:18:54.468004 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7166565c8875f3c04747c70ec6700450c93e64697c1138c561ef9fbe4d0bf894\": container with ID starting with 7166565c8875f3c04747c70ec6700450c93e64697c1138c561ef9fbe4d0bf894 not found: ID does not exist" containerID="7166565c8875f3c04747c70ec6700450c93e64697c1138c561ef9fbe4d0bf894" Dec 03 12:18:54 crc kubenswrapper[5052]: I1203 12:18:54.468029 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7166565c8875f3c04747c70ec6700450c93e64697c1138c561ef9fbe4d0bf894"} err="failed to get container status \"7166565c8875f3c04747c70ec6700450c93e64697c1138c561ef9fbe4d0bf894\": rpc error: code = NotFound desc = could not find container \"7166565c8875f3c04747c70ec6700450c93e64697c1138c561ef9fbe4d0bf894\": container with ID starting with 7166565c8875f3c04747c70ec6700450c93e64697c1138c561ef9fbe4d0bf894 not found: ID does not exist" Dec 03 12:18:54 crc kubenswrapper[5052]: I1203 12:18:54.468045 5052 scope.go:117] "RemoveContainer" containerID="7bcf29528a2630e2584876eb33663faf73c9660396ef93d28e491b29c246425e" Dec 03 12:18:54 crc kubenswrapper[5052]: E1203 12:18:54.468251 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7bcf29528a2630e2584876eb33663faf73c9660396ef93d28e491b29c246425e\": container with ID starting with 7bcf29528a2630e2584876eb33663faf73c9660396ef93d28e491b29c246425e not found: ID does not exist" containerID="7bcf29528a2630e2584876eb33663faf73c9660396ef93d28e491b29c246425e" Dec 03 12:18:54 crc kubenswrapper[5052]: I1203 12:18:54.468279 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7bcf29528a2630e2584876eb33663faf73c9660396ef93d28e491b29c246425e"} err="failed to get container status \"7bcf29528a2630e2584876eb33663faf73c9660396ef93d28e491b29c246425e\": rpc error: code = NotFound desc = could not find container \"7bcf29528a2630e2584876eb33663faf73c9660396ef93d28e491b29c246425e\": container with ID starting with 7bcf29528a2630e2584876eb33663faf73c9660396ef93d28e491b29c246425e not found: ID does not exist" Dec 03 12:18:54 crc kubenswrapper[5052]: I1203 12:18:54.516735 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/32685390-2027-41c0-8b2b-2ef4fe93b87d-scripts\") pod \"ceilometer-0\" (UID: \"32685390-2027-41c0-8b2b-2ef4fe93b87d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:18:54 crc kubenswrapper[5052]: I1203 12:18:54.516783 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/32685390-2027-41c0-8b2b-2ef4fe93b87d-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"32685390-2027-41c0-8b2b-2ef4fe93b87d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:18:54 crc kubenswrapper[5052]: I1203 12:18:54.516822 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/32685390-2027-41c0-8b2b-2ef4fe93b87d-run-httpd\") pod \"ceilometer-0\" (UID: \"32685390-2027-41c0-8b2b-2ef4fe93b87d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:18:54 crc kubenswrapper[5052]: I1203 12:18:54.516842 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gg8xd\" (UniqueName: \"kubernetes.io/projected/32685390-2027-41c0-8b2b-2ef4fe93b87d-kube-api-access-gg8xd\") pod \"ceilometer-0\" (UID: \"32685390-2027-41c0-8b2b-2ef4fe93b87d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:18:54 crc kubenswrapper[5052]: I1203 12:18:54.516876 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/32685390-2027-41c0-8b2b-2ef4fe93b87d-config-data\") pod \"ceilometer-0\" (UID: \"32685390-2027-41c0-8b2b-2ef4fe93b87d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:18:54 crc kubenswrapper[5052]: I1203 12:18:54.516913 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/32685390-2027-41c0-8b2b-2ef4fe93b87d-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"32685390-2027-41c0-8b2b-2ef4fe93b87d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:18:54 crc kubenswrapper[5052]: I1203 12:18:54.516988 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/32685390-2027-41c0-8b2b-2ef4fe93b87d-log-httpd\") pod \"ceilometer-0\" (UID: \"32685390-2027-41c0-8b2b-2ef4fe93b87d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:18:54 crc kubenswrapper[5052]: I1203 12:18:54.618188 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/32685390-2027-41c0-8b2b-2ef4fe93b87d-run-httpd\") pod \"ceilometer-0\" (UID: \"32685390-2027-41c0-8b2b-2ef4fe93b87d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:18:54 crc kubenswrapper[5052]: I1203 12:18:54.618241 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-gg8xd\" (UniqueName: \"kubernetes.io/projected/32685390-2027-41c0-8b2b-2ef4fe93b87d-kube-api-access-gg8xd\") pod \"ceilometer-0\" (UID: \"32685390-2027-41c0-8b2b-2ef4fe93b87d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:18:54 crc kubenswrapper[5052]: I1203 12:18:54.618285 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/32685390-2027-41c0-8b2b-2ef4fe93b87d-config-data\") pod \"ceilometer-0\" (UID: \"32685390-2027-41c0-8b2b-2ef4fe93b87d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:18:54 crc kubenswrapper[5052]: I1203 12:18:54.618331 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/32685390-2027-41c0-8b2b-2ef4fe93b87d-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"32685390-2027-41c0-8b2b-2ef4fe93b87d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:18:54 crc kubenswrapper[5052]: I1203 12:18:54.618436 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/32685390-2027-41c0-8b2b-2ef4fe93b87d-log-httpd\") pod \"ceilometer-0\" (UID: \"32685390-2027-41c0-8b2b-2ef4fe93b87d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:18:54 crc kubenswrapper[5052]: I1203 12:18:54.618479 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/32685390-2027-41c0-8b2b-2ef4fe93b87d-scripts\") pod \"ceilometer-0\" (UID: \"32685390-2027-41c0-8b2b-2ef4fe93b87d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:18:54 crc kubenswrapper[5052]: I1203 12:18:54.618535 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/32685390-2027-41c0-8b2b-2ef4fe93b87d-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"32685390-2027-41c0-8b2b-2ef4fe93b87d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:18:54 crc kubenswrapper[5052]: I1203 12:18:54.618806 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/32685390-2027-41c0-8b2b-2ef4fe93b87d-run-httpd\") pod \"ceilometer-0\" (UID: \"32685390-2027-41c0-8b2b-2ef4fe93b87d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:18:54 crc kubenswrapper[5052]: I1203 12:18:54.619383 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/32685390-2027-41c0-8b2b-2ef4fe93b87d-log-httpd\") pod \"ceilometer-0\" (UID: \"32685390-2027-41c0-8b2b-2ef4fe93b87d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:18:54 crc kubenswrapper[5052]: I1203 12:18:54.623252 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/32685390-2027-41c0-8b2b-2ef4fe93b87d-config-data\") pod \"ceilometer-0\" (UID: \"32685390-2027-41c0-8b2b-2ef4fe93b87d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:18:54 crc kubenswrapper[5052]: I1203 12:18:54.623325 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/32685390-2027-41c0-8b2b-2ef4fe93b87d-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"32685390-2027-41c0-8b2b-2ef4fe93b87d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:18:54 crc kubenswrapper[5052]: I1203 12:18:54.623689 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/32685390-2027-41c0-8b2b-2ef4fe93b87d-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"32685390-2027-41c0-8b2b-2ef4fe93b87d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:18:54 crc kubenswrapper[5052]: I1203 12:18:54.625287 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/32685390-2027-41c0-8b2b-2ef4fe93b87d-scripts\") pod \"ceilometer-0\" (UID: \"32685390-2027-41c0-8b2b-2ef4fe93b87d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:18:54 crc kubenswrapper[5052]: I1203 12:18:54.639248 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-gg8xd\" (UniqueName: \"kubernetes.io/projected/32685390-2027-41c0-8b2b-2ef4fe93b87d-kube-api-access-gg8xd\") pod \"ceilometer-0\" (UID: \"32685390-2027-41c0-8b2b-2ef4fe93b87d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:18:54 crc kubenswrapper[5052]: I1203 12:18:54.752530 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:18:55 crc kubenswrapper[5052]: I1203 12:18:55.264593 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:18:55 crc kubenswrapper[5052]: W1203 12:18:55.267757 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod32685390_2027_41c0_8b2b_2ef4fe93b87d.slice/crio-1d0f99cb6a437d7a439d8b69b6989bd98cb77031794a9da0749187064296d151 WatchSource:0}: Error finding container 1d0f99cb6a437d7a439d8b69b6989bd98cb77031794a9da0749187064296d151: Status 404 returned error can't find the container with id 1d0f99cb6a437d7a439d8b69b6989bd98cb77031794a9da0749187064296d151 Dec 03 12:18:55 crc kubenswrapper[5052]: I1203 12:18:55.346568 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"32685390-2027-41c0-8b2b-2ef4fe93b87d","Type":"ContainerStarted","Data":"1d0f99cb6a437d7a439d8b69b6989bd98cb77031794a9da0749187064296d151"} Dec 03 12:18:55 crc kubenswrapper[5052]: I1203 12:18:55.627035 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:18:55 crc kubenswrapper[5052]: I1203 12:18:55.627298 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:18:55 crc kubenswrapper[5052]: I1203 12:18:55.658379 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:18:55 crc kubenswrapper[5052]: I1203 12:18:55.737251 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e8433c1d-9f85-444b-b7bf-c2bd62fdbe8d" path="/var/lib/kubelet/pods/e8433c1d-9f85-444b-b7bf-c2bd62fdbe8d/volumes" Dec 03 12:18:55 crc kubenswrapper[5052]: I1203 12:18:55.738532 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:18:56 crc kubenswrapper[5052]: I1203 12:18:56.357284 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"32685390-2027-41c0-8b2b-2ef4fe93b87d","Type":"ContainerStarted","Data":"e2fe63f6c08e62f1c3cb7c9f6ef97cca4730134a56deda34d146e0d302018e6f"} Dec 03 12:18:56 crc kubenswrapper[5052]: I1203 12:18:56.357692 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:18:56 crc kubenswrapper[5052]: I1203 12:18:56.357716 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:18:56 crc kubenswrapper[5052]: I1203 12:18:56.554034 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:18:57 crc kubenswrapper[5052]: I1203 12:18:57.387611 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"32685390-2027-41c0-8b2b-2ef4fe93b87d","Type":"ContainerStarted","Data":"8b320bb589098497d50132ff2fb0e750bc6ab1e43890c26583f35d536a09d456"} Dec 03 12:18:57 crc kubenswrapper[5052]: I1203 12:18:57.388151 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"32685390-2027-41c0-8b2b-2ef4fe93b87d","Type":"ContainerStarted","Data":"5ce74171b7f323f7cd0c029db691936ab9775259c6a662bcd6f05d4703e6d14b"} Dec 03 12:18:58 crc kubenswrapper[5052]: I1203 12:18:58.454800 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:18:58 crc kubenswrapper[5052]: I1203 12:18:58.455565 5052 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Dec 03 12:18:58 crc kubenswrapper[5052]: I1203 12:18:58.460228 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:18:59 crc kubenswrapper[5052]: I1203 12:18:59.417594 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="32685390-2027-41c0-8b2b-2ef4fe93b87d" containerName="ceilometer-central-agent" containerID="cri-o://e2fe63f6c08e62f1c3cb7c9f6ef97cca4730134a56deda34d146e0d302018e6f" gracePeriod=30 Dec 03 12:18:59 crc kubenswrapper[5052]: I1203 12:18:59.418054 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"32685390-2027-41c0-8b2b-2ef4fe93b87d","Type":"ContainerStarted","Data":"a32a078875e6c2bb3af0f0a5347b4d64ca07c3d92c5365873e915b1b15fc6fd9"} Dec 03 12:18:59 crc kubenswrapper[5052]: I1203 12:18:59.418136 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:18:59 crc kubenswrapper[5052]: I1203 12:18:59.418378 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="32685390-2027-41c0-8b2b-2ef4fe93b87d" containerName="proxy-httpd" containerID="cri-o://a32a078875e6c2bb3af0f0a5347b4d64ca07c3d92c5365873e915b1b15fc6fd9" gracePeriod=30 Dec 03 12:18:59 crc kubenswrapper[5052]: I1203 12:18:59.418639 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="32685390-2027-41c0-8b2b-2ef4fe93b87d" containerName="sg-core" containerID="cri-o://8b320bb589098497d50132ff2fb0e750bc6ab1e43890c26583f35d536a09d456" gracePeriod=30 Dec 03 12:18:59 crc kubenswrapper[5052]: I1203 12:18:59.418748 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="32685390-2027-41c0-8b2b-2ef4fe93b87d" containerName="ceilometer-notification-agent" containerID="cri-o://5ce74171b7f323f7cd0c029db691936ab9775259c6a662bcd6f05d4703e6d14b" gracePeriod=30 Dec 03 12:18:59 crc kubenswrapper[5052]: I1203 12:18:59.467155 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/ceilometer-0" podStartSLOduration=2.273369161 podStartE2EDuration="5.467130278s" podCreationTimestamp="2025-12-03 12:18:54 +0000 UTC" firstStartedPulling="2025-12-03 12:18:55.269987592 +0000 UTC m=+3525.858229356" lastFinishedPulling="2025-12-03 12:18:58.463748709 +0000 UTC m=+3529.051990473" observedRunningTime="2025-12-03 12:18:59.454852698 +0000 UTC m=+3530.043094472" watchObservedRunningTime="2025-12-03 12:18:59.467130278 +0000 UTC m=+3530.055372052" Dec 03 12:18:59 crc kubenswrapper[5052]: E1203 12:18:59.956166 5052 cadvisor_stats_provider.go:516] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod32685390_2027_41c0_8b2b_2ef4fe93b87d.slice/crio-conmon-5ce74171b7f323f7cd0c029db691936ab9775259c6a662bcd6f05d4703e6d14b.scope\": RecentStats: unable to find data in memory cache]" Dec 03 12:19:00 crc kubenswrapper[5052]: I1203 12:19:00.433239 5052 generic.go:334] "Generic (PLEG): container finished" podID="32685390-2027-41c0-8b2b-2ef4fe93b87d" containerID="a32a078875e6c2bb3af0f0a5347b4d64ca07c3d92c5365873e915b1b15fc6fd9" exitCode=0 Dec 03 12:19:00 crc kubenswrapper[5052]: I1203 12:19:00.433279 5052 generic.go:334] "Generic (PLEG): container finished" podID="32685390-2027-41c0-8b2b-2ef4fe93b87d" containerID="8b320bb589098497d50132ff2fb0e750bc6ab1e43890c26583f35d536a09d456" exitCode=2 Dec 03 12:19:00 crc kubenswrapper[5052]: I1203 12:19:00.433288 5052 generic.go:334] "Generic (PLEG): container finished" podID="32685390-2027-41c0-8b2b-2ef4fe93b87d" containerID="5ce74171b7f323f7cd0c029db691936ab9775259c6a662bcd6f05d4703e6d14b" exitCode=0 Dec 03 12:19:00 crc kubenswrapper[5052]: I1203 12:19:00.433299 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"32685390-2027-41c0-8b2b-2ef4fe93b87d","Type":"ContainerDied","Data":"a32a078875e6c2bb3af0f0a5347b4d64ca07c3d92c5365873e915b1b15fc6fd9"} Dec 03 12:19:00 crc kubenswrapper[5052]: I1203 12:19:00.433349 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"32685390-2027-41c0-8b2b-2ef4fe93b87d","Type":"ContainerDied","Data":"8b320bb589098497d50132ff2fb0e750bc6ab1e43890c26583f35d536a09d456"} Dec 03 12:19:00 crc kubenswrapper[5052]: I1203 12:19:00.433359 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"32685390-2027-41c0-8b2b-2ef4fe93b87d","Type":"ContainerDied","Data":"5ce74171b7f323f7cd0c029db691936ab9775259c6a662bcd6f05d4703e6d14b"} Dec 03 12:19:01 crc kubenswrapper[5052]: I1203 12:19:01.447035 5052 generic.go:334] "Generic (PLEG): container finished" podID="8a3bbadd-37b5-4328-ae01-657aa8eb3028" containerID="6c1ba05ae9fa6187bf00e4c104305fd60275127701d6a185a2709d3cdff50ab1" exitCode=0 Dec 03 12:19:01 crc kubenswrapper[5052]: I1203 12:19:01.447132 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-j4cfx" event={"ID":"8a3bbadd-37b5-4328-ae01-657aa8eb3028","Type":"ContainerDied","Data":"6c1ba05ae9fa6187bf00e4c104305fd60275127701d6a185a2709d3cdff50ab1"} Dec 03 12:19:02 crc kubenswrapper[5052]: I1203 12:19:02.906840 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-j4cfx" Dec 03 12:19:03 crc kubenswrapper[5052]: I1203 12:19:03.006474 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8a3bbadd-37b5-4328-ae01-657aa8eb3028-combined-ca-bundle\") pod \"8a3bbadd-37b5-4328-ae01-657aa8eb3028\" (UID: \"8a3bbadd-37b5-4328-ae01-657aa8eb3028\") " Dec 03 12:19:03 crc kubenswrapper[5052]: I1203 12:19:03.006610 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/8a3bbadd-37b5-4328-ae01-657aa8eb3028-config-data\") pod \"8a3bbadd-37b5-4328-ae01-657aa8eb3028\" (UID: \"8a3bbadd-37b5-4328-ae01-657aa8eb3028\") " Dec 03 12:19:03 crc kubenswrapper[5052]: I1203 12:19:03.006642 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h8hcl\" (UniqueName: \"kubernetes.io/projected/8a3bbadd-37b5-4328-ae01-657aa8eb3028-kube-api-access-h8hcl\") pod \"8a3bbadd-37b5-4328-ae01-657aa8eb3028\" (UID: \"8a3bbadd-37b5-4328-ae01-657aa8eb3028\") " Dec 03 12:19:03 crc kubenswrapper[5052]: I1203 12:19:03.006661 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/8a3bbadd-37b5-4328-ae01-657aa8eb3028-scripts\") pod \"8a3bbadd-37b5-4328-ae01-657aa8eb3028\" (UID: \"8a3bbadd-37b5-4328-ae01-657aa8eb3028\") " Dec 03 12:19:03 crc kubenswrapper[5052]: I1203 12:19:03.012736 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8a3bbadd-37b5-4328-ae01-657aa8eb3028-kube-api-access-h8hcl" (OuterVolumeSpecName: "kube-api-access-h8hcl") pod "8a3bbadd-37b5-4328-ae01-657aa8eb3028" (UID: "8a3bbadd-37b5-4328-ae01-657aa8eb3028"). InnerVolumeSpecName "kube-api-access-h8hcl". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:19:03 crc kubenswrapper[5052]: I1203 12:19:03.019297 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8a3bbadd-37b5-4328-ae01-657aa8eb3028-scripts" (OuterVolumeSpecName: "scripts") pod "8a3bbadd-37b5-4328-ae01-657aa8eb3028" (UID: "8a3bbadd-37b5-4328-ae01-657aa8eb3028"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:19:03 crc kubenswrapper[5052]: I1203 12:19:03.034202 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8a3bbadd-37b5-4328-ae01-657aa8eb3028-config-data" (OuterVolumeSpecName: "config-data") pod "8a3bbadd-37b5-4328-ae01-657aa8eb3028" (UID: "8a3bbadd-37b5-4328-ae01-657aa8eb3028"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:19:03 crc kubenswrapper[5052]: I1203 12:19:03.045891 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8a3bbadd-37b5-4328-ae01-657aa8eb3028-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "8a3bbadd-37b5-4328-ae01-657aa8eb3028" (UID: "8a3bbadd-37b5-4328-ae01-657aa8eb3028"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:19:03 crc kubenswrapper[5052]: I1203 12:19:03.056033 5052 scope.go:117] "RemoveContainer" containerID="d7514d8a22f7396ecebd2be340bf89d19d446e00e42b69441a979d5db4b2df63" Dec 03 12:19:03 crc kubenswrapper[5052]: I1203 12:19:03.108134 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8a3bbadd-37b5-4328-ae01-657aa8eb3028-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:19:03 crc kubenswrapper[5052]: I1203 12:19:03.108175 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/8a3bbadd-37b5-4328-ae01-657aa8eb3028-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:19:03 crc kubenswrapper[5052]: I1203 12:19:03.108187 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-h8hcl\" (UniqueName: \"kubernetes.io/projected/8a3bbadd-37b5-4328-ae01-657aa8eb3028-kube-api-access-h8hcl\") on node \"crc\" DevicePath \"\"" Dec 03 12:19:03 crc kubenswrapper[5052]: I1203 12:19:03.108198 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/8a3bbadd-37b5-4328-ae01-657aa8eb3028-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:19:03 crc kubenswrapper[5052]: I1203 12:19:03.111631 5052 scope.go:117] "RemoveContainer" containerID="c51412b43d032effc161eda7a5d9e1847c358086f4766840050e85e65b009a7b" Dec 03 12:19:03 crc kubenswrapper[5052]: I1203 12:19:03.149321 5052 scope.go:117] "RemoveContainer" containerID="a128294473b2ae65eb81c532b3040ade4790b89f3cfa21e3d64a4929843efc65" Dec 03 12:19:03 crc kubenswrapper[5052]: I1203 12:19:03.169924 5052 scope.go:117] "RemoveContainer" containerID="ac8820d4342df6998e1122358c0ed26bd8debe54ca84f5a0d380bb533f8c00eb" Dec 03 12:19:03 crc kubenswrapper[5052]: I1203 12:19:03.190904 5052 scope.go:117] "RemoveContainer" containerID="256120295e912e4fd3dc859c0d55655b280c1f38dc2bf15611be732d0cf7213d" Dec 03 12:19:03 crc kubenswrapper[5052]: I1203 12:19:03.208967 5052 scope.go:117] "RemoveContainer" containerID="ec2fa1dad0e9bf5a1f5fbb88480a091dc02ea8944490375c6dd24553d51aadee" Dec 03 12:19:03 crc kubenswrapper[5052]: I1203 12:19:03.241927 5052 scope.go:117] "RemoveContainer" containerID="6f3d99a733e885f4083824dc0ca93f91a9d364725590a1230f9348d03355db05" Dec 03 12:19:03 crc kubenswrapper[5052]: I1203 12:19:03.261182 5052 scope.go:117] "RemoveContainer" containerID="6b5e16bdf4a9b75d7e426aac04d9e0f302e3e926580d5ea1db33ccce8cef8eb9" Dec 03 12:19:03 crc kubenswrapper[5052]: I1203 12:19:03.280373 5052 scope.go:117] "RemoveContainer" containerID="f570e33c32f271c1641da025fe74e75e6be9f91e37adc2f545d4947ca1189d6b" Dec 03 12:19:03 crc kubenswrapper[5052]: I1203 12:19:03.300868 5052 scope.go:117] "RemoveContainer" containerID="734003ce7f213657dd0c54361c6ddb117b5dc376cf9d9bfc93668c52a0c7900e" Dec 03 12:19:03 crc kubenswrapper[5052]: I1203 12:19:03.322199 5052 scope.go:117] "RemoveContainer" containerID="aa0bc6f7ead0eddf2779c1b4d8c7ab91d106b7722a51650dfdf4c53883d3f5aa" Dec 03 12:19:03 crc kubenswrapper[5052]: I1203 12:19:03.343902 5052 scope.go:117] "RemoveContainer" containerID="86aeba625efdf4888a66dbb87161495cc7e53e5502649308eea48c73eebda680" Dec 03 12:19:03 crc kubenswrapper[5052]: I1203 12:19:03.362958 5052 scope.go:117] "RemoveContainer" containerID="598592c093c893935a6c3dd121e70f0606c05f1732db9c2404c00c490b70c675" Dec 03 12:19:03 crc kubenswrapper[5052]: I1203 12:19:03.478693 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-j4cfx" event={"ID":"8a3bbadd-37b5-4328-ae01-657aa8eb3028","Type":"ContainerDied","Data":"ba7b32c7b0f64968162590dda93cc4c915be936affbe9eefd18f8d408c226f63"} Dec 03 12:19:03 crc kubenswrapper[5052]: I1203 12:19:03.478739 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ba7b32c7b0f64968162590dda93cc4c915be936affbe9eefd18f8d408c226f63" Dec 03 12:19:03 crc kubenswrapper[5052]: I1203 12:19:03.478807 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-j4cfx" Dec 03 12:19:03 crc kubenswrapper[5052]: I1203 12:19:03.552540 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-0"] Dec 03 12:19:03 crc kubenswrapper[5052]: E1203 12:19:03.553013 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="8a3bbadd-37b5-4328-ae01-657aa8eb3028" containerName="nova-cell0-conductor-db-sync" Dec 03 12:19:03 crc kubenswrapper[5052]: I1203 12:19:03.553033 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="8a3bbadd-37b5-4328-ae01-657aa8eb3028" containerName="nova-cell0-conductor-db-sync" Dec 03 12:19:03 crc kubenswrapper[5052]: I1203 12:19:03.553220 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="8a3bbadd-37b5-4328-ae01-657aa8eb3028" containerName="nova-cell0-conductor-db-sync" Dec 03 12:19:03 crc kubenswrapper[5052]: I1203 12:19:03.553970 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:19:03 crc kubenswrapper[5052]: I1203 12:19:03.557110 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-nova-dockercfg-6vqct" Dec 03 12:19:03 crc kubenswrapper[5052]: I1203 12:19:03.557429 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell0-conductor-config-data" Dec 03 12:19:03 crc kubenswrapper[5052]: I1203 12:19:03.561614 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-0"] Dec 03 12:19:03 crc kubenswrapper[5052]: I1203 12:19:03.617310 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/e80169f9-5480-4318-8ab2-1af4394fe76a-config-data\") pod \"nova-cell0-conductor-0\" (UID: \"e80169f9-5480-4318-8ab2-1af4394fe76a\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:19:03 crc kubenswrapper[5052]: I1203 12:19:03.617414 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e80169f9-5480-4318-8ab2-1af4394fe76a-combined-ca-bundle\") pod \"nova-cell0-conductor-0\" (UID: \"e80169f9-5480-4318-8ab2-1af4394fe76a\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:19:03 crc kubenswrapper[5052]: I1203 12:19:03.617458 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tsh5z\" (UniqueName: \"kubernetes.io/projected/e80169f9-5480-4318-8ab2-1af4394fe76a-kube-api-access-tsh5z\") pod \"nova-cell0-conductor-0\" (UID: \"e80169f9-5480-4318-8ab2-1af4394fe76a\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:19:03 crc kubenswrapper[5052]: I1203 12:19:03.718438 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/e80169f9-5480-4318-8ab2-1af4394fe76a-config-data\") pod \"nova-cell0-conductor-0\" (UID: \"e80169f9-5480-4318-8ab2-1af4394fe76a\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:19:03 crc kubenswrapper[5052]: I1203 12:19:03.719373 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e80169f9-5480-4318-8ab2-1af4394fe76a-combined-ca-bundle\") pod \"nova-cell0-conductor-0\" (UID: \"e80169f9-5480-4318-8ab2-1af4394fe76a\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:19:03 crc kubenswrapper[5052]: I1203 12:19:03.719557 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-tsh5z\" (UniqueName: \"kubernetes.io/projected/e80169f9-5480-4318-8ab2-1af4394fe76a-kube-api-access-tsh5z\") pod \"nova-cell0-conductor-0\" (UID: \"e80169f9-5480-4318-8ab2-1af4394fe76a\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:19:03 crc kubenswrapper[5052]: I1203 12:19:03.723845 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e80169f9-5480-4318-8ab2-1af4394fe76a-combined-ca-bundle\") pod \"nova-cell0-conductor-0\" (UID: \"e80169f9-5480-4318-8ab2-1af4394fe76a\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:19:03 crc kubenswrapper[5052]: I1203 12:19:03.723892 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/e80169f9-5480-4318-8ab2-1af4394fe76a-config-data\") pod \"nova-cell0-conductor-0\" (UID: \"e80169f9-5480-4318-8ab2-1af4394fe76a\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:19:03 crc kubenswrapper[5052]: I1203 12:19:03.735936 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-tsh5z\" (UniqueName: \"kubernetes.io/projected/e80169f9-5480-4318-8ab2-1af4394fe76a-kube-api-access-tsh5z\") pod \"nova-cell0-conductor-0\" (UID: \"e80169f9-5480-4318-8ab2-1af4394fe76a\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:19:03 crc kubenswrapper[5052]: I1203 12:19:03.884484 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:19:04 crc kubenswrapper[5052]: I1203 12:19:04.126465 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:19:04 crc kubenswrapper[5052]: I1203 12:19:04.137272 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/32685390-2027-41c0-8b2b-2ef4fe93b87d-combined-ca-bundle\") pod \"32685390-2027-41c0-8b2b-2ef4fe93b87d\" (UID: \"32685390-2027-41c0-8b2b-2ef4fe93b87d\") " Dec 03 12:19:04 crc kubenswrapper[5052]: I1203 12:19:04.137418 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/32685390-2027-41c0-8b2b-2ef4fe93b87d-run-httpd\") pod \"32685390-2027-41c0-8b2b-2ef4fe93b87d\" (UID: \"32685390-2027-41c0-8b2b-2ef4fe93b87d\") " Dec 03 12:19:04 crc kubenswrapper[5052]: I1203 12:19:04.137493 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/32685390-2027-41c0-8b2b-2ef4fe93b87d-scripts\") pod \"32685390-2027-41c0-8b2b-2ef4fe93b87d\" (UID: \"32685390-2027-41c0-8b2b-2ef4fe93b87d\") " Dec 03 12:19:04 crc kubenswrapper[5052]: I1203 12:19:04.137563 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/32685390-2027-41c0-8b2b-2ef4fe93b87d-log-httpd\") pod \"32685390-2027-41c0-8b2b-2ef4fe93b87d\" (UID: \"32685390-2027-41c0-8b2b-2ef4fe93b87d\") " Dec 03 12:19:04 crc kubenswrapper[5052]: I1203 12:19:04.137700 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/32685390-2027-41c0-8b2b-2ef4fe93b87d-config-data\") pod \"32685390-2027-41c0-8b2b-2ef4fe93b87d\" (UID: \"32685390-2027-41c0-8b2b-2ef4fe93b87d\") " Dec 03 12:19:04 crc kubenswrapper[5052]: I1203 12:19:04.137758 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gg8xd\" (UniqueName: \"kubernetes.io/projected/32685390-2027-41c0-8b2b-2ef4fe93b87d-kube-api-access-gg8xd\") pod \"32685390-2027-41c0-8b2b-2ef4fe93b87d\" (UID: \"32685390-2027-41c0-8b2b-2ef4fe93b87d\") " Dec 03 12:19:04 crc kubenswrapper[5052]: I1203 12:19:04.137793 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/32685390-2027-41c0-8b2b-2ef4fe93b87d-sg-core-conf-yaml\") pod \"32685390-2027-41c0-8b2b-2ef4fe93b87d\" (UID: \"32685390-2027-41c0-8b2b-2ef4fe93b87d\") " Dec 03 12:19:04 crc kubenswrapper[5052]: I1203 12:19:04.139144 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/32685390-2027-41c0-8b2b-2ef4fe93b87d-run-httpd" (OuterVolumeSpecName: "run-httpd") pod "32685390-2027-41c0-8b2b-2ef4fe93b87d" (UID: "32685390-2027-41c0-8b2b-2ef4fe93b87d"). InnerVolumeSpecName "run-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:19:04 crc kubenswrapper[5052]: I1203 12:19:04.139542 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/32685390-2027-41c0-8b2b-2ef4fe93b87d-log-httpd" (OuterVolumeSpecName: "log-httpd") pod "32685390-2027-41c0-8b2b-2ef4fe93b87d" (UID: "32685390-2027-41c0-8b2b-2ef4fe93b87d"). InnerVolumeSpecName "log-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:19:04 crc kubenswrapper[5052]: I1203 12:19:04.144660 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/32685390-2027-41c0-8b2b-2ef4fe93b87d-kube-api-access-gg8xd" (OuterVolumeSpecName: "kube-api-access-gg8xd") pod "32685390-2027-41c0-8b2b-2ef4fe93b87d" (UID: "32685390-2027-41c0-8b2b-2ef4fe93b87d"). InnerVolumeSpecName "kube-api-access-gg8xd". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:19:04 crc kubenswrapper[5052]: I1203 12:19:04.147128 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/32685390-2027-41c0-8b2b-2ef4fe93b87d-scripts" (OuterVolumeSpecName: "scripts") pod "32685390-2027-41c0-8b2b-2ef4fe93b87d" (UID: "32685390-2027-41c0-8b2b-2ef4fe93b87d"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:19:04 crc kubenswrapper[5052]: I1203 12:19:04.173985 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/32685390-2027-41c0-8b2b-2ef4fe93b87d-sg-core-conf-yaml" (OuterVolumeSpecName: "sg-core-conf-yaml") pod "32685390-2027-41c0-8b2b-2ef4fe93b87d" (UID: "32685390-2027-41c0-8b2b-2ef4fe93b87d"). InnerVolumeSpecName "sg-core-conf-yaml". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:19:04 crc kubenswrapper[5052]: I1203 12:19:04.219534 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/32685390-2027-41c0-8b2b-2ef4fe93b87d-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "32685390-2027-41c0-8b2b-2ef4fe93b87d" (UID: "32685390-2027-41c0-8b2b-2ef4fe93b87d"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:19:04 crc kubenswrapper[5052]: I1203 12:19:04.240398 5052 reconciler_common.go:293] "Volume detached for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/32685390-2027-41c0-8b2b-2ef4fe93b87d-sg-core-conf-yaml\") on node \"crc\" DevicePath \"\"" Dec 03 12:19:04 crc kubenswrapper[5052]: I1203 12:19:04.240428 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/32685390-2027-41c0-8b2b-2ef4fe93b87d-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:19:04 crc kubenswrapper[5052]: I1203 12:19:04.240438 5052 reconciler_common.go:293] "Volume detached for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/32685390-2027-41c0-8b2b-2ef4fe93b87d-run-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:19:04 crc kubenswrapper[5052]: I1203 12:19:04.240448 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/32685390-2027-41c0-8b2b-2ef4fe93b87d-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:19:04 crc kubenswrapper[5052]: I1203 12:19:04.240457 5052 reconciler_common.go:293] "Volume detached for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/32685390-2027-41c0-8b2b-2ef4fe93b87d-log-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:19:04 crc kubenswrapper[5052]: I1203 12:19:04.240466 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-gg8xd\" (UniqueName: \"kubernetes.io/projected/32685390-2027-41c0-8b2b-2ef4fe93b87d-kube-api-access-gg8xd\") on node \"crc\" DevicePath \"\"" Dec 03 12:19:04 crc kubenswrapper[5052]: I1203 12:19:04.244654 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/32685390-2027-41c0-8b2b-2ef4fe93b87d-config-data" (OuterVolumeSpecName: "config-data") pod "32685390-2027-41c0-8b2b-2ef4fe93b87d" (UID: "32685390-2027-41c0-8b2b-2ef4fe93b87d"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:19:04 crc kubenswrapper[5052]: I1203 12:19:04.342094 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/32685390-2027-41c0-8b2b-2ef4fe93b87d-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:19:04 crc kubenswrapper[5052]: I1203 12:19:04.363208 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-0"] Dec 03 12:19:04 crc kubenswrapper[5052]: I1203 12:19:04.523304 5052 generic.go:334] "Generic (PLEG): container finished" podID="32685390-2027-41c0-8b2b-2ef4fe93b87d" containerID="e2fe63f6c08e62f1c3cb7c9f6ef97cca4730134a56deda34d146e0d302018e6f" exitCode=0 Dec 03 12:19:04 crc kubenswrapper[5052]: I1203 12:19:04.523363 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:19:04 crc kubenswrapper[5052]: I1203 12:19:04.523407 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"32685390-2027-41c0-8b2b-2ef4fe93b87d","Type":"ContainerDied","Data":"e2fe63f6c08e62f1c3cb7c9f6ef97cca4730134a56deda34d146e0d302018e6f"} Dec 03 12:19:04 crc kubenswrapper[5052]: I1203 12:19:04.523441 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"32685390-2027-41c0-8b2b-2ef4fe93b87d","Type":"ContainerDied","Data":"1d0f99cb6a437d7a439d8b69b6989bd98cb77031794a9da0749187064296d151"} Dec 03 12:19:04 crc kubenswrapper[5052]: I1203 12:19:04.523472 5052 scope.go:117] "RemoveContainer" containerID="a32a078875e6c2bb3af0f0a5347b4d64ca07c3d92c5365873e915b1b15fc6fd9" Dec 03 12:19:04 crc kubenswrapper[5052]: I1203 12:19:04.525972 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-conductor-0" event={"ID":"e80169f9-5480-4318-8ab2-1af4394fe76a","Type":"ContainerStarted","Data":"bc5d8b8512b0c0b501fde049ef6154dff4775672351180bc35390815b2d7c8a2"} Dec 03 12:19:04 crc kubenswrapper[5052]: I1203 12:19:04.565768 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:19:04 crc kubenswrapper[5052]: I1203 12:19:04.576203 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:19:04 crc kubenswrapper[5052]: I1203 12:19:04.601645 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:19:04 crc kubenswrapper[5052]: E1203 12:19:04.602355 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="32685390-2027-41c0-8b2b-2ef4fe93b87d" containerName="ceilometer-notification-agent" Dec 03 12:19:04 crc kubenswrapper[5052]: I1203 12:19:04.602381 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="32685390-2027-41c0-8b2b-2ef4fe93b87d" containerName="ceilometer-notification-agent" Dec 03 12:19:04 crc kubenswrapper[5052]: E1203 12:19:04.602409 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="32685390-2027-41c0-8b2b-2ef4fe93b87d" containerName="sg-core" Dec 03 12:19:04 crc kubenswrapper[5052]: I1203 12:19:04.602418 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="32685390-2027-41c0-8b2b-2ef4fe93b87d" containerName="sg-core" Dec 03 12:19:04 crc kubenswrapper[5052]: E1203 12:19:04.602454 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="32685390-2027-41c0-8b2b-2ef4fe93b87d" containerName="ceilometer-central-agent" Dec 03 12:19:04 crc kubenswrapper[5052]: I1203 12:19:04.602464 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="32685390-2027-41c0-8b2b-2ef4fe93b87d" containerName="ceilometer-central-agent" Dec 03 12:19:04 crc kubenswrapper[5052]: E1203 12:19:04.602483 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="32685390-2027-41c0-8b2b-2ef4fe93b87d" containerName="proxy-httpd" Dec 03 12:19:04 crc kubenswrapper[5052]: I1203 12:19:04.602491 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="32685390-2027-41c0-8b2b-2ef4fe93b87d" containerName="proxy-httpd" Dec 03 12:19:04 crc kubenswrapper[5052]: I1203 12:19:04.602883 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="32685390-2027-41c0-8b2b-2ef4fe93b87d" containerName="proxy-httpd" Dec 03 12:19:04 crc kubenswrapper[5052]: I1203 12:19:04.602905 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="32685390-2027-41c0-8b2b-2ef4fe93b87d" containerName="ceilometer-central-agent" Dec 03 12:19:04 crc kubenswrapper[5052]: I1203 12:19:04.602924 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="32685390-2027-41c0-8b2b-2ef4fe93b87d" containerName="ceilometer-notification-agent" Dec 03 12:19:04 crc kubenswrapper[5052]: I1203 12:19:04.602952 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="32685390-2027-41c0-8b2b-2ef4fe93b87d" containerName="sg-core" Dec 03 12:19:04 crc kubenswrapper[5052]: I1203 12:19:04.605181 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:19:04 crc kubenswrapper[5052]: I1203 12:19:04.608470 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-scripts" Dec 03 12:19:04 crc kubenswrapper[5052]: I1203 12:19:04.608784 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-config-data" Dec 03 12:19:04 crc kubenswrapper[5052]: I1203 12:19:04.611753 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:19:04 crc kubenswrapper[5052]: I1203 12:19:04.641481 5052 scope.go:117] "RemoveContainer" containerID="8b320bb589098497d50132ff2fb0e750bc6ab1e43890c26583f35d536a09d456" Dec 03 12:19:04 crc kubenswrapper[5052]: I1203 12:19:04.648522 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/ee5afa22-9b03-4637-a06d-0a2e50ba8acd-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"ee5afa22-9b03-4637-a06d-0a2e50ba8acd\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:19:04 crc kubenswrapper[5052]: I1203 12:19:04.648584 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9gwww\" (UniqueName: \"kubernetes.io/projected/ee5afa22-9b03-4637-a06d-0a2e50ba8acd-kube-api-access-9gwww\") pod \"ceilometer-0\" (UID: \"ee5afa22-9b03-4637-a06d-0a2e50ba8acd\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:19:04 crc kubenswrapper[5052]: I1203 12:19:04.648636 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/ee5afa22-9b03-4637-a06d-0a2e50ba8acd-log-httpd\") pod \"ceilometer-0\" (UID: \"ee5afa22-9b03-4637-a06d-0a2e50ba8acd\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:19:04 crc kubenswrapper[5052]: I1203 12:19:04.649007 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/ee5afa22-9b03-4637-a06d-0a2e50ba8acd-scripts\") pod \"ceilometer-0\" (UID: \"ee5afa22-9b03-4637-a06d-0a2e50ba8acd\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:19:04 crc kubenswrapper[5052]: I1203 12:19:04.649566 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ee5afa22-9b03-4637-a06d-0a2e50ba8acd-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"ee5afa22-9b03-4637-a06d-0a2e50ba8acd\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:19:04 crc kubenswrapper[5052]: I1203 12:19:04.649628 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/ee5afa22-9b03-4637-a06d-0a2e50ba8acd-config-data\") pod \"ceilometer-0\" (UID: \"ee5afa22-9b03-4637-a06d-0a2e50ba8acd\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:19:04 crc kubenswrapper[5052]: I1203 12:19:04.649711 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/ee5afa22-9b03-4637-a06d-0a2e50ba8acd-run-httpd\") pod \"ceilometer-0\" (UID: \"ee5afa22-9b03-4637-a06d-0a2e50ba8acd\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:19:04 crc kubenswrapper[5052]: I1203 12:19:04.746661 5052 scope.go:117] "RemoveContainer" containerID="5ce74171b7f323f7cd0c029db691936ab9775259c6a662bcd6f05d4703e6d14b" Dec 03 12:19:04 crc kubenswrapper[5052]: I1203 12:19:04.751525 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/ee5afa22-9b03-4637-a06d-0a2e50ba8acd-run-httpd\") pod \"ceilometer-0\" (UID: \"ee5afa22-9b03-4637-a06d-0a2e50ba8acd\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:19:04 crc kubenswrapper[5052]: I1203 12:19:04.751819 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/ee5afa22-9b03-4637-a06d-0a2e50ba8acd-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"ee5afa22-9b03-4637-a06d-0a2e50ba8acd\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:19:04 crc kubenswrapper[5052]: I1203 12:19:04.752877 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-9gwww\" (UniqueName: \"kubernetes.io/projected/ee5afa22-9b03-4637-a06d-0a2e50ba8acd-kube-api-access-9gwww\") pod \"ceilometer-0\" (UID: \"ee5afa22-9b03-4637-a06d-0a2e50ba8acd\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:19:04 crc kubenswrapper[5052]: I1203 12:19:04.753011 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/ee5afa22-9b03-4637-a06d-0a2e50ba8acd-log-httpd\") pod \"ceilometer-0\" (UID: \"ee5afa22-9b03-4637-a06d-0a2e50ba8acd\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:19:04 crc kubenswrapper[5052]: I1203 12:19:04.753093 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/ee5afa22-9b03-4637-a06d-0a2e50ba8acd-scripts\") pod \"ceilometer-0\" (UID: \"ee5afa22-9b03-4637-a06d-0a2e50ba8acd\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:19:04 crc kubenswrapper[5052]: I1203 12:19:04.752025 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/ee5afa22-9b03-4637-a06d-0a2e50ba8acd-run-httpd\") pod \"ceilometer-0\" (UID: \"ee5afa22-9b03-4637-a06d-0a2e50ba8acd\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:19:04 crc kubenswrapper[5052]: I1203 12:19:04.753496 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/ee5afa22-9b03-4637-a06d-0a2e50ba8acd-log-httpd\") pod \"ceilometer-0\" (UID: \"ee5afa22-9b03-4637-a06d-0a2e50ba8acd\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:19:04 crc kubenswrapper[5052]: I1203 12:19:04.754195 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ee5afa22-9b03-4637-a06d-0a2e50ba8acd-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"ee5afa22-9b03-4637-a06d-0a2e50ba8acd\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:19:04 crc kubenswrapper[5052]: I1203 12:19:04.754240 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/ee5afa22-9b03-4637-a06d-0a2e50ba8acd-config-data\") pod \"ceilometer-0\" (UID: \"ee5afa22-9b03-4637-a06d-0a2e50ba8acd\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:19:04 crc kubenswrapper[5052]: I1203 12:19:04.757892 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/ee5afa22-9b03-4637-a06d-0a2e50ba8acd-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"ee5afa22-9b03-4637-a06d-0a2e50ba8acd\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:19:04 crc kubenswrapper[5052]: I1203 12:19:04.758399 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/ee5afa22-9b03-4637-a06d-0a2e50ba8acd-config-data\") pod \"ceilometer-0\" (UID: \"ee5afa22-9b03-4637-a06d-0a2e50ba8acd\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:19:04 crc kubenswrapper[5052]: I1203 12:19:04.759020 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/ee5afa22-9b03-4637-a06d-0a2e50ba8acd-scripts\") pod \"ceilometer-0\" (UID: \"ee5afa22-9b03-4637-a06d-0a2e50ba8acd\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:19:04 crc kubenswrapper[5052]: I1203 12:19:04.760302 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ee5afa22-9b03-4637-a06d-0a2e50ba8acd-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"ee5afa22-9b03-4637-a06d-0a2e50ba8acd\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:19:04 crc kubenswrapper[5052]: I1203 12:19:04.780306 5052 scope.go:117] "RemoveContainer" containerID="e2fe63f6c08e62f1c3cb7c9f6ef97cca4730134a56deda34d146e0d302018e6f" Dec 03 12:19:04 crc kubenswrapper[5052]: I1203 12:19:04.785422 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-9gwww\" (UniqueName: \"kubernetes.io/projected/ee5afa22-9b03-4637-a06d-0a2e50ba8acd-kube-api-access-9gwww\") pod \"ceilometer-0\" (UID: \"ee5afa22-9b03-4637-a06d-0a2e50ba8acd\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:19:04 crc kubenswrapper[5052]: I1203 12:19:04.806855 5052 scope.go:117] "RemoveContainer" containerID="a32a078875e6c2bb3af0f0a5347b4d64ca07c3d92c5365873e915b1b15fc6fd9" Dec 03 12:19:04 crc kubenswrapper[5052]: E1203 12:19:04.807824 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a32a078875e6c2bb3af0f0a5347b4d64ca07c3d92c5365873e915b1b15fc6fd9\": container with ID starting with a32a078875e6c2bb3af0f0a5347b4d64ca07c3d92c5365873e915b1b15fc6fd9 not found: ID does not exist" containerID="a32a078875e6c2bb3af0f0a5347b4d64ca07c3d92c5365873e915b1b15fc6fd9" Dec 03 12:19:04 crc kubenswrapper[5052]: I1203 12:19:04.807864 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a32a078875e6c2bb3af0f0a5347b4d64ca07c3d92c5365873e915b1b15fc6fd9"} err="failed to get container status \"a32a078875e6c2bb3af0f0a5347b4d64ca07c3d92c5365873e915b1b15fc6fd9\": rpc error: code = NotFound desc = could not find container \"a32a078875e6c2bb3af0f0a5347b4d64ca07c3d92c5365873e915b1b15fc6fd9\": container with ID starting with a32a078875e6c2bb3af0f0a5347b4d64ca07c3d92c5365873e915b1b15fc6fd9 not found: ID does not exist" Dec 03 12:19:04 crc kubenswrapper[5052]: I1203 12:19:04.807889 5052 scope.go:117] "RemoveContainer" containerID="8b320bb589098497d50132ff2fb0e750bc6ab1e43890c26583f35d536a09d456" Dec 03 12:19:04 crc kubenswrapper[5052]: E1203 12:19:04.808140 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8b320bb589098497d50132ff2fb0e750bc6ab1e43890c26583f35d536a09d456\": container with ID starting with 8b320bb589098497d50132ff2fb0e750bc6ab1e43890c26583f35d536a09d456 not found: ID does not exist" containerID="8b320bb589098497d50132ff2fb0e750bc6ab1e43890c26583f35d536a09d456" Dec 03 12:19:04 crc kubenswrapper[5052]: I1203 12:19:04.808163 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8b320bb589098497d50132ff2fb0e750bc6ab1e43890c26583f35d536a09d456"} err="failed to get container status \"8b320bb589098497d50132ff2fb0e750bc6ab1e43890c26583f35d536a09d456\": rpc error: code = NotFound desc = could not find container \"8b320bb589098497d50132ff2fb0e750bc6ab1e43890c26583f35d536a09d456\": container with ID starting with 8b320bb589098497d50132ff2fb0e750bc6ab1e43890c26583f35d536a09d456 not found: ID does not exist" Dec 03 12:19:04 crc kubenswrapper[5052]: I1203 12:19:04.808189 5052 scope.go:117] "RemoveContainer" containerID="5ce74171b7f323f7cd0c029db691936ab9775259c6a662bcd6f05d4703e6d14b" Dec 03 12:19:04 crc kubenswrapper[5052]: E1203 12:19:04.808462 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5ce74171b7f323f7cd0c029db691936ab9775259c6a662bcd6f05d4703e6d14b\": container with ID starting with 5ce74171b7f323f7cd0c029db691936ab9775259c6a662bcd6f05d4703e6d14b not found: ID does not exist" containerID="5ce74171b7f323f7cd0c029db691936ab9775259c6a662bcd6f05d4703e6d14b" Dec 03 12:19:04 crc kubenswrapper[5052]: I1203 12:19:04.808488 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5ce74171b7f323f7cd0c029db691936ab9775259c6a662bcd6f05d4703e6d14b"} err="failed to get container status \"5ce74171b7f323f7cd0c029db691936ab9775259c6a662bcd6f05d4703e6d14b\": rpc error: code = NotFound desc = could not find container \"5ce74171b7f323f7cd0c029db691936ab9775259c6a662bcd6f05d4703e6d14b\": container with ID starting with 5ce74171b7f323f7cd0c029db691936ab9775259c6a662bcd6f05d4703e6d14b not found: ID does not exist" Dec 03 12:19:04 crc kubenswrapper[5052]: I1203 12:19:04.808504 5052 scope.go:117] "RemoveContainer" containerID="e2fe63f6c08e62f1c3cb7c9f6ef97cca4730134a56deda34d146e0d302018e6f" Dec 03 12:19:04 crc kubenswrapper[5052]: E1203 12:19:04.808791 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e2fe63f6c08e62f1c3cb7c9f6ef97cca4730134a56deda34d146e0d302018e6f\": container with ID starting with e2fe63f6c08e62f1c3cb7c9f6ef97cca4730134a56deda34d146e0d302018e6f not found: ID does not exist" containerID="e2fe63f6c08e62f1c3cb7c9f6ef97cca4730134a56deda34d146e0d302018e6f" Dec 03 12:19:04 crc kubenswrapper[5052]: I1203 12:19:04.808810 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e2fe63f6c08e62f1c3cb7c9f6ef97cca4730134a56deda34d146e0d302018e6f"} err="failed to get container status \"e2fe63f6c08e62f1c3cb7c9f6ef97cca4730134a56deda34d146e0d302018e6f\": rpc error: code = NotFound desc = could not find container \"e2fe63f6c08e62f1c3cb7c9f6ef97cca4730134a56deda34d146e0d302018e6f\": container with ID starting with e2fe63f6c08e62f1c3cb7c9f6ef97cca4730134a56deda34d146e0d302018e6f not found: ID does not exist" Dec 03 12:19:04 crc kubenswrapper[5052]: I1203 12:19:04.926666 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:19:05 crc kubenswrapper[5052]: I1203 12:19:05.372486 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:19:05 crc kubenswrapper[5052]: I1203 12:19:05.542459 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"ee5afa22-9b03-4637-a06d-0a2e50ba8acd","Type":"ContainerStarted","Data":"2583a9197f09e727adfbc00687ed4cf69590f3cc3f3f93e9b2ccc85d86aa6c52"} Dec 03 12:19:05 crc kubenswrapper[5052]: I1203 12:19:05.548458 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-conductor-0" event={"ID":"e80169f9-5480-4318-8ab2-1af4394fe76a","Type":"ContainerStarted","Data":"a96261519a88986377772f2eba818435423e35e4613b257b006f4e1d712a712b"} Dec 03 12:19:05 crc kubenswrapper[5052]: I1203 12:19:05.550010 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:19:05 crc kubenswrapper[5052]: I1203 12:19:05.573294 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-cell0-conductor-0" podStartSLOduration=2.57327803 podStartE2EDuration="2.57327803s" podCreationTimestamp="2025-12-03 12:19:03 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:19:05.567062723 +0000 UTC m=+3536.155304487" watchObservedRunningTime="2025-12-03 12:19:05.57327803 +0000 UTC m=+3536.161519794" Dec 03 12:19:05 crc kubenswrapper[5052]: I1203 12:19:05.685069 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="32685390-2027-41c0-8b2b-2ef4fe93b87d" path="/var/lib/kubelet/pods/32685390-2027-41c0-8b2b-2ef4fe93b87d/volumes" Dec 03 12:19:06 crc kubenswrapper[5052]: I1203 12:19:06.558122 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"ee5afa22-9b03-4637-a06d-0a2e50ba8acd","Type":"ContainerStarted","Data":"5f2bdd98b42e88a6875dcf0312cee327ad8b12fe0c0406943ebc0874b76993a2"} Dec 03 12:19:07 crc kubenswrapper[5052]: I1203 12:19:07.570353 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"ee5afa22-9b03-4637-a06d-0a2e50ba8acd","Type":"ContainerStarted","Data":"32ed87e381ed568d90156f9ddbc55bf9e8a3e9ece1e4240b61e1888a410caa87"} Dec 03 12:19:08 crc kubenswrapper[5052]: I1203 12:19:08.582599 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"ee5afa22-9b03-4637-a06d-0a2e50ba8acd","Type":"ContainerStarted","Data":"c2a8b0a8115d9df6f7308e52f5f02b46717547b22569fa41657b485815150f85"} Dec 03 12:19:09 crc kubenswrapper[5052]: I1203 12:19:09.605109 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"ee5afa22-9b03-4637-a06d-0a2e50ba8acd","Type":"ContainerStarted","Data":"38388ebd248f12e273f88c8011cab32758fafbd0c18c6044b9721ab2ef11e6ad"} Dec 03 12:19:09 crc kubenswrapper[5052]: I1203 12:19:09.606091 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:19:09 crc kubenswrapper[5052]: I1203 12:19:09.634045 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/ceilometer-0" podStartSLOduration=2.001041699 podStartE2EDuration="5.634023141s" podCreationTimestamp="2025-12-03 12:19:04 +0000 UTC" firstStartedPulling="2025-12-03 12:19:05.394986362 +0000 UTC m=+3535.983228126" lastFinishedPulling="2025-12-03 12:19:09.027967774 +0000 UTC m=+3539.616209568" observedRunningTime="2025-12-03 12:19:09.628866804 +0000 UTC m=+3540.217108578" watchObservedRunningTime="2025-12-03 12:19:09.634023141 +0000 UTC m=+3540.222264915" Dec 03 12:19:13 crc kubenswrapper[5052]: I1203 12:19:13.911864 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:19:14 crc kubenswrapper[5052]: I1203 12:19:14.755981 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell0-cell-mapping-pn79k"] Dec 03 12:19:14 crc kubenswrapper[5052]: I1203 12:19:14.758053 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-cell-mapping-pn79k" Dec 03 12:19:14 crc kubenswrapper[5052]: I1203 12:19:14.761397 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell0-manage-config-data" Dec 03 12:19:14 crc kubenswrapper[5052]: I1203 12:19:14.761676 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell0-manage-scripts" Dec 03 12:19:14 crc kubenswrapper[5052]: I1203 12:19:14.764565 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell0-cell-mapping-pn79k"] Dec 03 12:19:14 crc kubenswrapper[5052]: I1203 12:19:14.919798 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/aaf39e45-e5a6-4ead-82e3-035c43cbc1d7-scripts\") pod \"nova-cell0-cell-mapping-pn79k\" (UID: \"aaf39e45-e5a6-4ead-82e3-035c43cbc1d7\") " pod="openstack-kuttl-tests/nova-cell0-cell-mapping-pn79k" Dec 03 12:19:14 crc kubenswrapper[5052]: I1203 12:19:14.919860 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l2xwb\" (UniqueName: \"kubernetes.io/projected/aaf39e45-e5a6-4ead-82e3-035c43cbc1d7-kube-api-access-l2xwb\") pod \"nova-cell0-cell-mapping-pn79k\" (UID: \"aaf39e45-e5a6-4ead-82e3-035c43cbc1d7\") " pod="openstack-kuttl-tests/nova-cell0-cell-mapping-pn79k" Dec 03 12:19:14 crc kubenswrapper[5052]: I1203 12:19:14.919946 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/aaf39e45-e5a6-4ead-82e3-035c43cbc1d7-combined-ca-bundle\") pod \"nova-cell0-cell-mapping-pn79k\" (UID: \"aaf39e45-e5a6-4ead-82e3-035c43cbc1d7\") " pod="openstack-kuttl-tests/nova-cell0-cell-mapping-pn79k" Dec 03 12:19:14 crc kubenswrapper[5052]: I1203 12:19:14.920003 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/aaf39e45-e5a6-4ead-82e3-035c43cbc1d7-config-data\") pod \"nova-cell0-cell-mapping-pn79k\" (UID: \"aaf39e45-e5a6-4ead-82e3-035c43cbc1d7\") " pod="openstack-kuttl-tests/nova-cell0-cell-mapping-pn79k" Dec 03 12:19:14 crc kubenswrapper[5052]: I1203 12:19:14.921237 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:19:14 crc kubenswrapper[5052]: I1203 12:19:14.923685 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:19:14 crc kubenswrapper[5052]: I1203 12:19:14.954322 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:19:14 crc kubenswrapper[5052]: I1203 12:19:14.956883 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-api-config-data" Dec 03 12:19:15 crc kubenswrapper[5052]: I1203 12:19:15.023674 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/aaf39e45-e5a6-4ead-82e3-035c43cbc1d7-config-data\") pod \"nova-cell0-cell-mapping-pn79k\" (UID: \"aaf39e45-e5a6-4ead-82e3-035c43cbc1d7\") " pod="openstack-kuttl-tests/nova-cell0-cell-mapping-pn79k" Dec 03 12:19:15 crc kubenswrapper[5052]: I1203 12:19:15.023820 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/aaf39e45-e5a6-4ead-82e3-035c43cbc1d7-scripts\") pod \"nova-cell0-cell-mapping-pn79k\" (UID: \"aaf39e45-e5a6-4ead-82e3-035c43cbc1d7\") " pod="openstack-kuttl-tests/nova-cell0-cell-mapping-pn79k" Dec 03 12:19:15 crc kubenswrapper[5052]: I1203 12:19:15.023846 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-l2xwb\" (UniqueName: \"kubernetes.io/projected/aaf39e45-e5a6-4ead-82e3-035c43cbc1d7-kube-api-access-l2xwb\") pod \"nova-cell0-cell-mapping-pn79k\" (UID: \"aaf39e45-e5a6-4ead-82e3-035c43cbc1d7\") " pod="openstack-kuttl-tests/nova-cell0-cell-mapping-pn79k" Dec 03 12:19:15 crc kubenswrapper[5052]: I1203 12:19:15.023931 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/aaf39e45-e5a6-4ead-82e3-035c43cbc1d7-combined-ca-bundle\") pod \"nova-cell0-cell-mapping-pn79k\" (UID: \"aaf39e45-e5a6-4ead-82e3-035c43cbc1d7\") " pod="openstack-kuttl-tests/nova-cell0-cell-mapping-pn79k" Dec 03 12:19:15 crc kubenswrapper[5052]: I1203 12:19:15.047498 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 12:19:15 crc kubenswrapper[5052]: I1203 12:19:15.048271 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/aaf39e45-e5a6-4ead-82e3-035c43cbc1d7-config-data\") pod \"nova-cell0-cell-mapping-pn79k\" (UID: \"aaf39e45-e5a6-4ead-82e3-035c43cbc1d7\") " pod="openstack-kuttl-tests/nova-cell0-cell-mapping-pn79k" Dec 03 12:19:15 crc kubenswrapper[5052]: I1203 12:19:15.049865 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:19:15 crc kubenswrapper[5052]: I1203 12:19:15.051336 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/aaf39e45-e5a6-4ead-82e3-035c43cbc1d7-scripts\") pod \"nova-cell0-cell-mapping-pn79k\" (UID: \"aaf39e45-e5a6-4ead-82e3-035c43cbc1d7\") " pod="openstack-kuttl-tests/nova-cell0-cell-mapping-pn79k" Dec 03 12:19:15 crc kubenswrapper[5052]: I1203 12:19:15.054425 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell1-novncproxy-config-data" Dec 03 12:19:15 crc kubenswrapper[5052]: I1203 12:19:15.059406 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/aaf39e45-e5a6-4ead-82e3-035c43cbc1d7-combined-ca-bundle\") pod \"nova-cell0-cell-mapping-pn79k\" (UID: \"aaf39e45-e5a6-4ead-82e3-035c43cbc1d7\") " pod="openstack-kuttl-tests/nova-cell0-cell-mapping-pn79k" Dec 03 12:19:15 crc kubenswrapper[5052]: I1203 12:19:15.090750 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 12:19:15 crc kubenswrapper[5052]: I1203 12:19:15.097552 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-l2xwb\" (UniqueName: \"kubernetes.io/projected/aaf39e45-e5a6-4ead-82e3-035c43cbc1d7-kube-api-access-l2xwb\") pod \"nova-cell0-cell-mapping-pn79k\" (UID: \"aaf39e45-e5a6-4ead-82e3-035c43cbc1d7\") " pod="openstack-kuttl-tests/nova-cell0-cell-mapping-pn79k" Dec 03 12:19:15 crc kubenswrapper[5052]: I1203 12:19:15.113690 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-cell-mapping-pn79k" Dec 03 12:19:15 crc kubenswrapper[5052]: I1203 12:19:15.133601 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/b85d1543-4d68-4980-97d2-4168381db238-config-data\") pod \"nova-api-0\" (UID: \"b85d1543-4d68-4980-97d2-4168381db238\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:19:15 crc kubenswrapper[5052]: I1203 12:19:15.133686 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/b85d1543-4d68-4980-97d2-4168381db238-logs\") pod \"nova-api-0\" (UID: \"b85d1543-4d68-4980-97d2-4168381db238\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:19:15 crc kubenswrapper[5052]: I1203 12:19:15.133997 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8bjpw\" (UniqueName: \"kubernetes.io/projected/b85d1543-4d68-4980-97d2-4168381db238-kube-api-access-8bjpw\") pod \"nova-api-0\" (UID: \"b85d1543-4d68-4980-97d2-4168381db238\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:19:15 crc kubenswrapper[5052]: I1203 12:19:15.134066 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b85d1543-4d68-4980-97d2-4168381db238-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"b85d1543-4d68-4980-97d2-4168381db238\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:19:15 crc kubenswrapper[5052]: I1203 12:19:15.188942 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:19:15 crc kubenswrapper[5052]: I1203 12:19:15.190544 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:19:15 crc kubenswrapper[5052]: I1203 12:19:15.194591 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-scheduler-config-data" Dec 03 12:19:15 crc kubenswrapper[5052]: I1203 12:19:15.205487 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:19:15 crc kubenswrapper[5052]: I1203 12:19:15.240862 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6b6f7757-7421-4af9-87ce-e0265ee64160-config-data\") pod \"nova-cell1-novncproxy-0\" (UID: \"6b6f7757-7421-4af9-87ce-e0265ee64160\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:19:15 crc kubenswrapper[5052]: I1203 12:19:15.241131 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/b85d1543-4d68-4980-97d2-4168381db238-config-data\") pod \"nova-api-0\" (UID: \"b85d1543-4d68-4980-97d2-4168381db238\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:19:15 crc kubenswrapper[5052]: I1203 12:19:15.241184 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qqw2l\" (UniqueName: \"kubernetes.io/projected/6b6f7757-7421-4af9-87ce-e0265ee64160-kube-api-access-qqw2l\") pod \"nova-cell1-novncproxy-0\" (UID: \"6b6f7757-7421-4af9-87ce-e0265ee64160\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:19:15 crc kubenswrapper[5052]: I1203 12:19:15.241327 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/b85d1543-4d68-4980-97d2-4168381db238-logs\") pod \"nova-api-0\" (UID: \"b85d1543-4d68-4980-97d2-4168381db238\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:19:15 crc kubenswrapper[5052]: I1203 12:19:15.247953 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-8bjpw\" (UniqueName: \"kubernetes.io/projected/b85d1543-4d68-4980-97d2-4168381db238-kube-api-access-8bjpw\") pod \"nova-api-0\" (UID: \"b85d1543-4d68-4980-97d2-4168381db238\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:19:15 crc kubenswrapper[5052]: I1203 12:19:15.248751 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/b85d1543-4d68-4980-97d2-4168381db238-logs\") pod \"nova-api-0\" (UID: \"b85d1543-4d68-4980-97d2-4168381db238\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:19:15 crc kubenswrapper[5052]: I1203 12:19:15.249563 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6b6f7757-7421-4af9-87ce-e0265ee64160-combined-ca-bundle\") pod \"nova-cell1-novncproxy-0\" (UID: \"6b6f7757-7421-4af9-87ce-e0265ee64160\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:19:15 crc kubenswrapper[5052]: I1203 12:19:15.249617 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b85d1543-4d68-4980-97d2-4168381db238-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"b85d1543-4d68-4980-97d2-4168381db238\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:19:15 crc kubenswrapper[5052]: I1203 12:19:15.249966 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:19:15 crc kubenswrapper[5052]: I1203 12:19:15.251998 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:19:15 crc kubenswrapper[5052]: I1203 12:19:15.256780 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-metadata-config-data" Dec 03 12:19:15 crc kubenswrapper[5052]: I1203 12:19:15.256823 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b85d1543-4d68-4980-97d2-4168381db238-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"b85d1543-4d68-4980-97d2-4168381db238\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:19:15 crc kubenswrapper[5052]: I1203 12:19:15.309000 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/b85d1543-4d68-4980-97d2-4168381db238-config-data\") pod \"nova-api-0\" (UID: \"b85d1543-4d68-4980-97d2-4168381db238\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:19:15 crc kubenswrapper[5052]: I1203 12:19:15.320272 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:19:15 crc kubenswrapper[5052]: I1203 12:19:15.338368 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-8bjpw\" (UniqueName: \"kubernetes.io/projected/b85d1543-4d68-4980-97d2-4168381db238-kube-api-access-8bjpw\") pod \"nova-api-0\" (UID: \"b85d1543-4d68-4980-97d2-4168381db238\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:19:15 crc kubenswrapper[5052]: I1203 12:19:15.350633 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-qqw2l\" (UniqueName: \"kubernetes.io/projected/6b6f7757-7421-4af9-87ce-e0265ee64160-kube-api-access-qqw2l\") pod \"nova-cell1-novncproxy-0\" (UID: \"6b6f7757-7421-4af9-87ce-e0265ee64160\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:19:15 crc kubenswrapper[5052]: I1203 12:19:15.350692 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/986b933a-6cbe-42d6-ab39-7233c006e9f5-combined-ca-bundle\") pod \"nova-scheduler-0\" (UID: \"986b933a-6cbe-42d6-ab39-7233c006e9f5\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:19:15 crc kubenswrapper[5052]: I1203 12:19:15.350722 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6b6f7757-7421-4af9-87ce-e0265ee64160-combined-ca-bundle\") pod \"nova-cell1-novncproxy-0\" (UID: \"6b6f7757-7421-4af9-87ce-e0265ee64160\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:19:15 crc kubenswrapper[5052]: I1203 12:19:15.350794 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6b6f7757-7421-4af9-87ce-e0265ee64160-config-data\") pod \"nova-cell1-novncproxy-0\" (UID: \"6b6f7757-7421-4af9-87ce-e0265ee64160\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:19:15 crc kubenswrapper[5052]: I1203 12:19:15.350844 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7f5sz\" (UniqueName: \"kubernetes.io/projected/986b933a-6cbe-42d6-ab39-7233c006e9f5-kube-api-access-7f5sz\") pod \"nova-scheduler-0\" (UID: \"986b933a-6cbe-42d6-ab39-7233c006e9f5\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:19:15 crc kubenswrapper[5052]: I1203 12:19:15.350867 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/986b933a-6cbe-42d6-ab39-7233c006e9f5-config-data\") pod \"nova-scheduler-0\" (UID: \"986b933a-6cbe-42d6-ab39-7233c006e9f5\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:19:15 crc kubenswrapper[5052]: I1203 12:19:15.355301 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6b6f7757-7421-4af9-87ce-e0265ee64160-config-data\") pod \"nova-cell1-novncproxy-0\" (UID: \"6b6f7757-7421-4af9-87ce-e0265ee64160\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:19:15 crc kubenswrapper[5052]: I1203 12:19:15.357008 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6b6f7757-7421-4af9-87ce-e0265ee64160-combined-ca-bundle\") pod \"nova-cell1-novncproxy-0\" (UID: \"6b6f7757-7421-4af9-87ce-e0265ee64160\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:19:15 crc kubenswrapper[5052]: I1203 12:19:15.383342 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-qqw2l\" (UniqueName: \"kubernetes.io/projected/6b6f7757-7421-4af9-87ce-e0265ee64160-kube-api-access-qqw2l\") pod \"nova-cell1-novncproxy-0\" (UID: \"6b6f7757-7421-4af9-87ce-e0265ee64160\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:19:15 crc kubenswrapper[5052]: I1203 12:19:15.474010 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/bd6ff54a-35e6-4c80-bc3f-caab63ce5239-logs\") pod \"nova-metadata-0\" (UID: \"bd6ff54a-35e6-4c80-bc3f-caab63ce5239\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:19:15 crc kubenswrapper[5052]: I1203 12:19:15.474083 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/986b933a-6cbe-42d6-ab39-7233c006e9f5-combined-ca-bundle\") pod \"nova-scheduler-0\" (UID: \"986b933a-6cbe-42d6-ab39-7233c006e9f5\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:19:15 crc kubenswrapper[5052]: I1203 12:19:15.474202 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/bd6ff54a-35e6-4c80-bc3f-caab63ce5239-config-data\") pod \"nova-metadata-0\" (UID: \"bd6ff54a-35e6-4c80-bc3f-caab63ce5239\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:19:15 crc kubenswrapper[5052]: I1203 12:19:15.474350 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nd8v4\" (UniqueName: \"kubernetes.io/projected/bd6ff54a-35e6-4c80-bc3f-caab63ce5239-kube-api-access-nd8v4\") pod \"nova-metadata-0\" (UID: \"bd6ff54a-35e6-4c80-bc3f-caab63ce5239\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:19:15 crc kubenswrapper[5052]: I1203 12:19:15.474376 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-7f5sz\" (UniqueName: \"kubernetes.io/projected/986b933a-6cbe-42d6-ab39-7233c006e9f5-kube-api-access-7f5sz\") pod \"nova-scheduler-0\" (UID: \"986b933a-6cbe-42d6-ab39-7233c006e9f5\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:19:15 crc kubenswrapper[5052]: I1203 12:19:15.474408 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/986b933a-6cbe-42d6-ab39-7233c006e9f5-config-data\") pod \"nova-scheduler-0\" (UID: \"986b933a-6cbe-42d6-ab39-7233c006e9f5\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:19:15 crc kubenswrapper[5052]: I1203 12:19:15.474630 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/bd6ff54a-35e6-4c80-bc3f-caab63ce5239-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"bd6ff54a-35e6-4c80-bc3f-caab63ce5239\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:19:15 crc kubenswrapper[5052]: I1203 12:19:15.479232 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/986b933a-6cbe-42d6-ab39-7233c006e9f5-combined-ca-bundle\") pod \"nova-scheduler-0\" (UID: \"986b933a-6cbe-42d6-ab39-7233c006e9f5\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:19:15 crc kubenswrapper[5052]: I1203 12:19:15.479613 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/986b933a-6cbe-42d6-ab39-7233c006e9f5-config-data\") pod \"nova-scheduler-0\" (UID: \"986b933a-6cbe-42d6-ab39-7233c006e9f5\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:19:15 crc kubenswrapper[5052]: I1203 12:19:15.498273 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-7f5sz\" (UniqueName: \"kubernetes.io/projected/986b933a-6cbe-42d6-ab39-7233c006e9f5-kube-api-access-7f5sz\") pod \"nova-scheduler-0\" (UID: \"986b933a-6cbe-42d6-ab39-7233c006e9f5\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:19:15 crc kubenswrapper[5052]: I1203 12:19:15.572721 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:19:15 crc kubenswrapper[5052]: I1203 12:19:15.576203 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/bd6ff54a-35e6-4c80-bc3f-caab63ce5239-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"bd6ff54a-35e6-4c80-bc3f-caab63ce5239\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:19:15 crc kubenswrapper[5052]: I1203 12:19:15.576258 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/bd6ff54a-35e6-4c80-bc3f-caab63ce5239-logs\") pod \"nova-metadata-0\" (UID: \"bd6ff54a-35e6-4c80-bc3f-caab63ce5239\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:19:15 crc kubenswrapper[5052]: I1203 12:19:15.576335 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/bd6ff54a-35e6-4c80-bc3f-caab63ce5239-config-data\") pod \"nova-metadata-0\" (UID: \"bd6ff54a-35e6-4c80-bc3f-caab63ce5239\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:19:15 crc kubenswrapper[5052]: I1203 12:19:15.576423 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-nd8v4\" (UniqueName: \"kubernetes.io/projected/bd6ff54a-35e6-4c80-bc3f-caab63ce5239-kube-api-access-nd8v4\") pod \"nova-metadata-0\" (UID: \"bd6ff54a-35e6-4c80-bc3f-caab63ce5239\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:19:15 crc kubenswrapper[5052]: I1203 12:19:15.577688 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/bd6ff54a-35e6-4c80-bc3f-caab63ce5239-logs\") pod \"nova-metadata-0\" (UID: \"bd6ff54a-35e6-4c80-bc3f-caab63ce5239\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:19:15 crc kubenswrapper[5052]: I1203 12:19:15.583235 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/bd6ff54a-35e6-4c80-bc3f-caab63ce5239-config-data\") pod \"nova-metadata-0\" (UID: \"bd6ff54a-35e6-4c80-bc3f-caab63ce5239\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:19:15 crc kubenswrapper[5052]: I1203 12:19:15.583811 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/bd6ff54a-35e6-4c80-bc3f-caab63ce5239-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"bd6ff54a-35e6-4c80-bc3f-caab63ce5239\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:19:15 crc kubenswrapper[5052]: I1203 12:19:15.603287 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-nd8v4\" (UniqueName: \"kubernetes.io/projected/bd6ff54a-35e6-4c80-bc3f-caab63ce5239-kube-api-access-nd8v4\") pod \"nova-metadata-0\" (UID: \"bd6ff54a-35e6-4c80-bc3f-caab63ce5239\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:19:15 crc kubenswrapper[5052]: I1203 12:19:15.652936 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:19:15 crc kubenswrapper[5052]: I1203 12:19:15.664773 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:19:15 crc kubenswrapper[5052]: I1203 12:19:15.687422 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:19:15 crc kubenswrapper[5052]: I1203 12:19:15.818200 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell0-cell-mapping-pn79k"] Dec 03 12:19:15 crc kubenswrapper[5052]: I1203 12:19:15.912582 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:19:15 crc kubenswrapper[5052]: I1203 12:19:15.951562 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-db-sync-zlxvc"] Dec 03 12:19:15 crc kubenswrapper[5052]: I1203 12:19:15.953771 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-zlxvc" Dec 03 12:19:15 crc kubenswrapper[5052]: I1203 12:19:15.958407 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell1-conductor-scripts" Dec 03 12:19:15 crc kubenswrapper[5052]: I1203 12:19:15.965533 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell1-conductor-config-data" Dec 03 12:19:15 crc kubenswrapper[5052]: I1203 12:19:15.980763 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-db-sync-zlxvc"] Dec 03 12:19:15 crc kubenswrapper[5052]: I1203 12:19:15.991921 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c4d2abd6-6282-4c8f-b24d-49ffd64b0ce2-combined-ca-bundle\") pod \"nova-cell1-conductor-db-sync-zlxvc\" (UID: \"c4d2abd6-6282-4c8f-b24d-49ffd64b0ce2\") " pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-zlxvc" Dec 03 12:19:15 crc kubenswrapper[5052]: I1203 12:19:15.991974 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/c4d2abd6-6282-4c8f-b24d-49ffd64b0ce2-config-data\") pod \"nova-cell1-conductor-db-sync-zlxvc\" (UID: \"c4d2abd6-6282-4c8f-b24d-49ffd64b0ce2\") " pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-zlxvc" Dec 03 12:19:15 crc kubenswrapper[5052]: I1203 12:19:15.992032 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7g2mq\" (UniqueName: \"kubernetes.io/projected/c4d2abd6-6282-4c8f-b24d-49ffd64b0ce2-kube-api-access-7g2mq\") pod \"nova-cell1-conductor-db-sync-zlxvc\" (UID: \"c4d2abd6-6282-4c8f-b24d-49ffd64b0ce2\") " pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-zlxvc" Dec 03 12:19:15 crc kubenswrapper[5052]: I1203 12:19:15.992072 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/c4d2abd6-6282-4c8f-b24d-49ffd64b0ce2-scripts\") pod \"nova-cell1-conductor-db-sync-zlxvc\" (UID: \"c4d2abd6-6282-4c8f-b24d-49ffd64b0ce2\") " pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-zlxvc" Dec 03 12:19:16 crc kubenswrapper[5052]: I1203 12:19:16.096628 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c4d2abd6-6282-4c8f-b24d-49ffd64b0ce2-combined-ca-bundle\") pod \"nova-cell1-conductor-db-sync-zlxvc\" (UID: \"c4d2abd6-6282-4c8f-b24d-49ffd64b0ce2\") " pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-zlxvc" Dec 03 12:19:16 crc kubenswrapper[5052]: I1203 12:19:16.096708 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/c4d2abd6-6282-4c8f-b24d-49ffd64b0ce2-config-data\") pod \"nova-cell1-conductor-db-sync-zlxvc\" (UID: \"c4d2abd6-6282-4c8f-b24d-49ffd64b0ce2\") " pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-zlxvc" Dec 03 12:19:16 crc kubenswrapper[5052]: I1203 12:19:16.096807 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-7g2mq\" (UniqueName: \"kubernetes.io/projected/c4d2abd6-6282-4c8f-b24d-49ffd64b0ce2-kube-api-access-7g2mq\") pod \"nova-cell1-conductor-db-sync-zlxvc\" (UID: \"c4d2abd6-6282-4c8f-b24d-49ffd64b0ce2\") " pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-zlxvc" Dec 03 12:19:16 crc kubenswrapper[5052]: I1203 12:19:16.102377 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c4d2abd6-6282-4c8f-b24d-49ffd64b0ce2-combined-ca-bundle\") pod \"nova-cell1-conductor-db-sync-zlxvc\" (UID: \"c4d2abd6-6282-4c8f-b24d-49ffd64b0ce2\") " pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-zlxvc" Dec 03 12:19:16 crc kubenswrapper[5052]: I1203 12:19:16.103719 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/c4d2abd6-6282-4c8f-b24d-49ffd64b0ce2-scripts\") pod \"nova-cell1-conductor-db-sync-zlxvc\" (UID: \"c4d2abd6-6282-4c8f-b24d-49ffd64b0ce2\") " pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-zlxvc" Dec 03 12:19:16 crc kubenswrapper[5052]: I1203 12:19:16.103958 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/c4d2abd6-6282-4c8f-b24d-49ffd64b0ce2-config-data\") pod \"nova-cell1-conductor-db-sync-zlxvc\" (UID: \"c4d2abd6-6282-4c8f-b24d-49ffd64b0ce2\") " pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-zlxvc" Dec 03 12:19:16 crc kubenswrapper[5052]: I1203 12:19:16.113171 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/c4d2abd6-6282-4c8f-b24d-49ffd64b0ce2-scripts\") pod \"nova-cell1-conductor-db-sync-zlxvc\" (UID: \"c4d2abd6-6282-4c8f-b24d-49ffd64b0ce2\") " pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-zlxvc" Dec 03 12:19:16 crc kubenswrapper[5052]: I1203 12:19:16.124333 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-7g2mq\" (UniqueName: \"kubernetes.io/projected/c4d2abd6-6282-4c8f-b24d-49ffd64b0ce2-kube-api-access-7g2mq\") pod \"nova-cell1-conductor-db-sync-zlxvc\" (UID: \"c4d2abd6-6282-4c8f-b24d-49ffd64b0ce2\") " pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-zlxvc" Dec 03 12:19:16 crc kubenswrapper[5052]: I1203 12:19:16.291237 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-zlxvc" Dec 03 12:19:16 crc kubenswrapper[5052]: I1203 12:19:16.389588 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:19:16 crc kubenswrapper[5052]: I1203 12:19:16.400390 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 12:19:16 crc kubenswrapper[5052]: W1203 12:19:16.421994 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6b6f7757_7421_4af9_87ce_e0265ee64160.slice/crio-1d0fd17eeba92c661a29a8fca6d6497ac3f2a6d2db34ab542bf22f21f7d1bf44 WatchSource:0}: Error finding container 1d0fd17eeba92c661a29a8fca6d6497ac3f2a6d2db34ab542bf22f21f7d1bf44: Status 404 returned error can't find the container with id 1d0fd17eeba92c661a29a8fca6d6497ac3f2a6d2db34ab542bf22f21f7d1bf44 Dec 03 12:19:16 crc kubenswrapper[5052]: I1203 12:19:16.620034 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:19:16 crc kubenswrapper[5052]: W1203 12:19:16.641810 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbd6ff54a_35e6_4c80_bc3f_caab63ce5239.slice/crio-9f4c7cee0c5acc6b88d333a976b6c5292d3436b7e4224e88fe69f963658dccbe WatchSource:0}: Error finding container 9f4c7cee0c5acc6b88d333a976b6c5292d3436b7e4224e88fe69f963658dccbe: Status 404 returned error can't find the container with id 9f4c7cee0c5acc6b88d333a976b6c5292d3436b7e4224e88fe69f963658dccbe Dec 03 12:19:16 crc kubenswrapper[5052]: I1203 12:19:16.739875 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-cell-mapping-pn79k" event={"ID":"aaf39e45-e5a6-4ead-82e3-035c43cbc1d7","Type":"ContainerStarted","Data":"9ef2ea9519f50d4f2765e00b600cfe8f29f447a34e20aa72760861bd5ab6b0b0"} Dec 03 12:19:16 crc kubenswrapper[5052]: I1203 12:19:16.740223 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-cell-mapping-pn79k" event={"ID":"aaf39e45-e5a6-4ead-82e3-035c43cbc1d7","Type":"ContainerStarted","Data":"23244b815e34569395e1ce5df42dad06cb1c676c383b536d2fc97d53a27452d9"} Dec 03 12:19:16 crc kubenswrapper[5052]: I1203 12:19:16.746910 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"bd6ff54a-35e6-4c80-bc3f-caab63ce5239","Type":"ContainerStarted","Data":"9f4c7cee0c5acc6b88d333a976b6c5292d3436b7e4224e88fe69f963658dccbe"} Dec 03 12:19:16 crc kubenswrapper[5052]: I1203 12:19:16.770432 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-cell0-cell-mapping-pn79k" podStartSLOduration=2.770412209 podStartE2EDuration="2.770412209s" podCreationTimestamp="2025-12-03 12:19:14 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:19:16.756386908 +0000 UTC m=+3547.344628692" watchObservedRunningTime="2025-12-03 12:19:16.770412209 +0000 UTC m=+3547.358653973" Dec 03 12:19:16 crc kubenswrapper[5052]: I1203 12:19:16.785303 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" event={"ID":"6b6f7757-7421-4af9-87ce-e0265ee64160","Type":"ContainerStarted","Data":"1d0fd17eeba92c661a29a8fca6d6497ac3f2a6d2db34ab542bf22f21f7d1bf44"} Dec 03 12:19:16 crc kubenswrapper[5052]: I1203 12:19:16.789667 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"b85d1543-4d68-4980-97d2-4168381db238","Type":"ContainerStarted","Data":"d91c19496f184f5f6be8562c5b5ecd26605542c98135cd177562baf0dce4ffb1"} Dec 03 12:19:16 crc kubenswrapper[5052]: I1203 12:19:16.789716 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"b85d1543-4d68-4980-97d2-4168381db238","Type":"ContainerStarted","Data":"8967c542141b2c8c6329afc40c6cae659b60319e60f2b2f334ff176ed3cc9b64"} Dec 03 12:19:16 crc kubenswrapper[5052]: I1203 12:19:16.795179 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"986b933a-6cbe-42d6-ab39-7233c006e9f5","Type":"ContainerStarted","Data":"2a0541c3b6039a96df991332853628579e08c340a094b8c1d44a58e13bc07005"} Dec 03 12:19:16 crc kubenswrapper[5052]: I1203 12:19:16.795231 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"986b933a-6cbe-42d6-ab39-7233c006e9f5","Type":"ContainerStarted","Data":"1f73e25ff25b51aefbdccb7c39c529ff8e799702ffdb649fec1fa98ffdbf8b1f"} Dec 03 12:19:16 crc kubenswrapper[5052]: I1203 12:19:16.807656 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-db-sync-zlxvc"] Dec 03 12:19:16 crc kubenswrapper[5052]: I1203 12:19:16.815056 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-scheduler-0" podStartSLOduration=1.815034332 podStartE2EDuration="1.815034332s" podCreationTimestamp="2025-12-03 12:19:15 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:19:16.810823482 +0000 UTC m=+3547.399065246" watchObservedRunningTime="2025-12-03 12:19:16.815034332 +0000 UTC m=+3547.403276096" Dec 03 12:19:16 crc kubenswrapper[5052]: W1203 12:19:16.826283 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc4d2abd6_6282_4c8f_b24d_49ffd64b0ce2.slice/crio-8d1e3ef27cf0fa25f7232f45f631c831dde4a79f18a801b9e28d7daead6bf96a WatchSource:0}: Error finding container 8d1e3ef27cf0fa25f7232f45f631c831dde4a79f18a801b9e28d7daead6bf96a: Status 404 returned error can't find the container with id 8d1e3ef27cf0fa25f7232f45f631c831dde4a79f18a801b9e28d7daead6bf96a Dec 03 12:19:17 crc kubenswrapper[5052]: I1203 12:19:17.821947 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"b85d1543-4d68-4980-97d2-4168381db238","Type":"ContainerStarted","Data":"f47c386e9ed42113922ee7779b6fcd345651a75d010d2ed9f2e0908c03e6b7e3"} Dec 03 12:19:17 crc kubenswrapper[5052]: I1203 12:19:17.829240 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-zlxvc" event={"ID":"c4d2abd6-6282-4c8f-b24d-49ffd64b0ce2","Type":"ContainerStarted","Data":"d2dbcc7e2fdee4a46c5830b20391717220b63cfcd058c8f7699961d14ed09d4e"} Dec 03 12:19:17 crc kubenswrapper[5052]: I1203 12:19:17.829286 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-zlxvc" event={"ID":"c4d2abd6-6282-4c8f-b24d-49ffd64b0ce2","Type":"ContainerStarted","Data":"8d1e3ef27cf0fa25f7232f45f631c831dde4a79f18a801b9e28d7daead6bf96a"} Dec 03 12:19:17 crc kubenswrapper[5052]: I1203 12:19:17.840607 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"bd6ff54a-35e6-4c80-bc3f-caab63ce5239","Type":"ContainerStarted","Data":"0ccd9f77159f708bb0c9e38592a52b58156ee68c539a70c1a8cb2b34e63828bf"} Dec 03 12:19:17 crc kubenswrapper[5052]: I1203 12:19:17.840662 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"bd6ff54a-35e6-4c80-bc3f-caab63ce5239","Type":"ContainerStarted","Data":"fff5e81f4e61e210898a9efd838676120b71112c24cdf2b3d314324c91c024eb"} Dec 03 12:19:17 crc kubenswrapper[5052]: I1203 12:19:17.856269 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" event={"ID":"6b6f7757-7421-4af9-87ce-e0265ee64160","Type":"ContainerStarted","Data":"7f84c127b73452ac5b99f8b132e0617ddd12fa40640f55107cde1118446d6ad1"} Dec 03 12:19:17 crc kubenswrapper[5052]: I1203 12:19:17.859177 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-api-0" podStartSLOduration=3.859143164 podStartE2EDuration="3.859143164s" podCreationTimestamp="2025-12-03 12:19:14 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:19:17.853007578 +0000 UTC m=+3548.441249342" watchObservedRunningTime="2025-12-03 12:19:17.859143164 +0000 UTC m=+3548.447384948" Dec 03 12:19:17 crc kubenswrapper[5052]: I1203 12:19:17.884746 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-zlxvc" podStartSLOduration=2.884725744 podStartE2EDuration="2.884725744s" podCreationTimestamp="2025-12-03 12:19:15 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:19:17.874299186 +0000 UTC m=+3548.462540950" watchObservedRunningTime="2025-12-03 12:19:17.884725744 +0000 UTC m=+3548.472967508" Dec 03 12:19:17 crc kubenswrapper[5052]: I1203 12:19:17.905168 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:19:17 crc kubenswrapper[5052]: I1203 12:19:17.919345 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 12:19:17 crc kubenswrapper[5052]: I1203 12:19:17.924978 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-metadata-0" podStartSLOduration=2.9249548020000002 podStartE2EDuration="2.924954802s" podCreationTimestamp="2025-12-03 12:19:15 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:19:17.914167974 +0000 UTC m=+3548.502409758" watchObservedRunningTime="2025-12-03 12:19:17.924954802 +0000 UTC m=+3548.513196566" Dec 03 12:19:17 crc kubenswrapper[5052]: I1203 12:19:17.951316 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" podStartSLOduration=2.9512969140000003 podStartE2EDuration="2.951296914s" podCreationTimestamp="2025-12-03 12:19:15 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:19:17.940951609 +0000 UTC m=+3548.529193383" watchObservedRunningTime="2025-12-03 12:19:17.951296914 +0000 UTC m=+3548.539538678" Dec 03 12:19:19 crc kubenswrapper[5052]: I1203 12:19:19.873928 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" podUID="6b6f7757-7421-4af9-87ce-e0265ee64160" containerName="nova-cell1-novncproxy-novncproxy" containerID="cri-o://7f84c127b73452ac5b99f8b132e0617ddd12fa40640f55107cde1118446d6ad1" gracePeriod=30 Dec 03 12:19:19 crc kubenswrapper[5052]: I1203 12:19:19.875397 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-metadata-0" podUID="bd6ff54a-35e6-4c80-bc3f-caab63ce5239" containerName="nova-metadata-log" containerID="cri-o://fff5e81f4e61e210898a9efd838676120b71112c24cdf2b3d314324c91c024eb" gracePeriod=30 Dec 03 12:19:19 crc kubenswrapper[5052]: I1203 12:19:19.875610 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-metadata-0" podUID="bd6ff54a-35e6-4c80-bc3f-caab63ce5239" containerName="nova-metadata-metadata" containerID="cri-o://0ccd9f77159f708bb0c9e38592a52b58156ee68c539a70c1a8cb2b34e63828bf" gracePeriod=30 Dec 03 12:19:20 crc kubenswrapper[5052]: I1203 12:19:20.467795 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:19:20 crc kubenswrapper[5052]: I1203 12:19:20.611548 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/bd6ff54a-35e6-4c80-bc3f-caab63ce5239-config-data\") pod \"bd6ff54a-35e6-4c80-bc3f-caab63ce5239\" (UID: \"bd6ff54a-35e6-4c80-bc3f-caab63ce5239\") " Dec 03 12:19:20 crc kubenswrapper[5052]: I1203 12:19:20.612311 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nd8v4\" (UniqueName: \"kubernetes.io/projected/bd6ff54a-35e6-4c80-bc3f-caab63ce5239-kube-api-access-nd8v4\") pod \"bd6ff54a-35e6-4c80-bc3f-caab63ce5239\" (UID: \"bd6ff54a-35e6-4c80-bc3f-caab63ce5239\") " Dec 03 12:19:20 crc kubenswrapper[5052]: I1203 12:19:20.612377 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/bd6ff54a-35e6-4c80-bc3f-caab63ce5239-combined-ca-bundle\") pod \"bd6ff54a-35e6-4c80-bc3f-caab63ce5239\" (UID: \"bd6ff54a-35e6-4c80-bc3f-caab63ce5239\") " Dec 03 12:19:20 crc kubenswrapper[5052]: I1203 12:19:20.612442 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/bd6ff54a-35e6-4c80-bc3f-caab63ce5239-logs\") pod \"bd6ff54a-35e6-4c80-bc3f-caab63ce5239\" (UID: \"bd6ff54a-35e6-4c80-bc3f-caab63ce5239\") " Dec 03 12:19:20 crc kubenswrapper[5052]: I1203 12:19:20.614005 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd6ff54a-35e6-4c80-bc3f-caab63ce5239-logs" (OuterVolumeSpecName: "logs") pod "bd6ff54a-35e6-4c80-bc3f-caab63ce5239" (UID: "bd6ff54a-35e6-4c80-bc3f-caab63ce5239"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:19:20 crc kubenswrapper[5052]: I1203 12:19:20.628880 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bd6ff54a-35e6-4c80-bc3f-caab63ce5239-kube-api-access-nd8v4" (OuterVolumeSpecName: "kube-api-access-nd8v4") pod "bd6ff54a-35e6-4c80-bc3f-caab63ce5239" (UID: "bd6ff54a-35e6-4c80-bc3f-caab63ce5239"). InnerVolumeSpecName "kube-api-access-nd8v4". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:19:20 crc kubenswrapper[5052]: I1203 12:19:20.653114 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:19:20 crc kubenswrapper[5052]: I1203 12:19:20.666963 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:19:20 crc kubenswrapper[5052]: I1203 12:19:20.689328 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bd6ff54a-35e6-4c80-bc3f-caab63ce5239-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "bd6ff54a-35e6-4c80-bc3f-caab63ce5239" (UID: "bd6ff54a-35e6-4c80-bc3f-caab63ce5239"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:19:20 crc kubenswrapper[5052]: I1203 12:19:20.704326 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bd6ff54a-35e6-4c80-bc3f-caab63ce5239-config-data" (OuterVolumeSpecName: "config-data") pod "bd6ff54a-35e6-4c80-bc3f-caab63ce5239" (UID: "bd6ff54a-35e6-4c80-bc3f-caab63ce5239"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:19:20 crc kubenswrapper[5052]: I1203 12:19:20.715763 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/bd6ff54a-35e6-4c80-bc3f-caab63ce5239-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:19:20 crc kubenswrapper[5052]: I1203 12:19:20.715808 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-nd8v4\" (UniqueName: \"kubernetes.io/projected/bd6ff54a-35e6-4c80-bc3f-caab63ce5239-kube-api-access-nd8v4\") on node \"crc\" DevicePath \"\"" Dec 03 12:19:20 crc kubenswrapper[5052]: I1203 12:19:20.715825 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/bd6ff54a-35e6-4c80-bc3f-caab63ce5239-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:19:20 crc kubenswrapper[5052]: I1203 12:19:20.715840 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/bd6ff54a-35e6-4c80-bc3f-caab63ce5239-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:19:20 crc kubenswrapper[5052]: I1203 12:19:20.896692 5052 generic.go:334] "Generic (PLEG): container finished" podID="bd6ff54a-35e6-4c80-bc3f-caab63ce5239" containerID="0ccd9f77159f708bb0c9e38592a52b58156ee68c539a70c1a8cb2b34e63828bf" exitCode=0 Dec 03 12:19:20 crc kubenswrapper[5052]: I1203 12:19:20.896729 5052 generic.go:334] "Generic (PLEG): container finished" podID="bd6ff54a-35e6-4c80-bc3f-caab63ce5239" containerID="fff5e81f4e61e210898a9efd838676120b71112c24cdf2b3d314324c91c024eb" exitCode=143 Dec 03 12:19:20 crc kubenswrapper[5052]: I1203 12:19:20.896758 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"bd6ff54a-35e6-4c80-bc3f-caab63ce5239","Type":"ContainerDied","Data":"0ccd9f77159f708bb0c9e38592a52b58156ee68c539a70c1a8cb2b34e63828bf"} Dec 03 12:19:20 crc kubenswrapper[5052]: I1203 12:19:20.896827 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"bd6ff54a-35e6-4c80-bc3f-caab63ce5239","Type":"ContainerDied","Data":"fff5e81f4e61e210898a9efd838676120b71112c24cdf2b3d314324c91c024eb"} Dec 03 12:19:20 crc kubenswrapper[5052]: I1203 12:19:20.896840 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"bd6ff54a-35e6-4c80-bc3f-caab63ce5239","Type":"ContainerDied","Data":"9f4c7cee0c5acc6b88d333a976b6c5292d3436b7e4224e88fe69f963658dccbe"} Dec 03 12:19:20 crc kubenswrapper[5052]: I1203 12:19:20.896856 5052 scope.go:117] "RemoveContainer" containerID="0ccd9f77159f708bb0c9e38592a52b58156ee68c539a70c1a8cb2b34e63828bf" Dec 03 12:19:20 crc kubenswrapper[5052]: I1203 12:19:20.896774 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:19:20 crc kubenswrapper[5052]: I1203 12:19:20.900676 5052 generic.go:334] "Generic (PLEG): container finished" podID="6b6f7757-7421-4af9-87ce-e0265ee64160" containerID="7f84c127b73452ac5b99f8b132e0617ddd12fa40640f55107cde1118446d6ad1" exitCode=0 Dec 03 12:19:20 crc kubenswrapper[5052]: I1203 12:19:20.900777 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" event={"ID":"6b6f7757-7421-4af9-87ce-e0265ee64160","Type":"ContainerDied","Data":"7f84c127b73452ac5b99f8b132e0617ddd12fa40640f55107cde1118446d6ad1"} Dec 03 12:19:20 crc kubenswrapper[5052]: I1203 12:19:20.934036 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:19:20 crc kubenswrapper[5052]: I1203 12:19:20.937897 5052 scope.go:117] "RemoveContainer" containerID="fff5e81f4e61e210898a9efd838676120b71112c24cdf2b3d314324c91c024eb" Dec 03 12:19:20 crc kubenswrapper[5052]: I1203 12:19:20.947106 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:19:20 crc kubenswrapper[5052]: I1203 12:19:20.968799 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:19:20 crc kubenswrapper[5052]: E1203 12:19:20.970312 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="bd6ff54a-35e6-4c80-bc3f-caab63ce5239" containerName="nova-metadata-log" Dec 03 12:19:20 crc kubenswrapper[5052]: I1203 12:19:20.970339 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd6ff54a-35e6-4c80-bc3f-caab63ce5239" containerName="nova-metadata-log" Dec 03 12:19:20 crc kubenswrapper[5052]: E1203 12:19:20.970357 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="bd6ff54a-35e6-4c80-bc3f-caab63ce5239" containerName="nova-metadata-metadata" Dec 03 12:19:20 crc kubenswrapper[5052]: I1203 12:19:20.970365 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd6ff54a-35e6-4c80-bc3f-caab63ce5239" containerName="nova-metadata-metadata" Dec 03 12:19:20 crc kubenswrapper[5052]: I1203 12:19:20.970634 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="bd6ff54a-35e6-4c80-bc3f-caab63ce5239" containerName="nova-metadata-metadata" Dec 03 12:19:20 crc kubenswrapper[5052]: I1203 12:19:20.970663 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="bd6ff54a-35e6-4c80-bc3f-caab63ce5239" containerName="nova-metadata-log" Dec 03 12:19:20 crc kubenswrapper[5052]: I1203 12:19:20.972567 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:19:20 crc kubenswrapper[5052]: I1203 12:19:20.979340 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-metadata-config-data" Dec 03 12:19:20 crc kubenswrapper[5052]: I1203 12:19:20.979894 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-nova-metadata-internal-svc" Dec 03 12:19:20 crc kubenswrapper[5052]: I1203 12:19:20.987700 5052 scope.go:117] "RemoveContainer" containerID="0ccd9f77159f708bb0c9e38592a52b58156ee68c539a70c1a8cb2b34e63828bf" Dec 03 12:19:20 crc kubenswrapper[5052]: E1203 12:19:20.998356 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0ccd9f77159f708bb0c9e38592a52b58156ee68c539a70c1a8cb2b34e63828bf\": container with ID starting with 0ccd9f77159f708bb0c9e38592a52b58156ee68c539a70c1a8cb2b34e63828bf not found: ID does not exist" containerID="0ccd9f77159f708bb0c9e38592a52b58156ee68c539a70c1a8cb2b34e63828bf" Dec 03 12:19:20 crc kubenswrapper[5052]: I1203 12:19:20.998432 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0ccd9f77159f708bb0c9e38592a52b58156ee68c539a70c1a8cb2b34e63828bf"} err="failed to get container status \"0ccd9f77159f708bb0c9e38592a52b58156ee68c539a70c1a8cb2b34e63828bf\": rpc error: code = NotFound desc = could not find container \"0ccd9f77159f708bb0c9e38592a52b58156ee68c539a70c1a8cb2b34e63828bf\": container with ID starting with 0ccd9f77159f708bb0c9e38592a52b58156ee68c539a70c1a8cb2b34e63828bf not found: ID does not exist" Dec 03 12:19:20 crc kubenswrapper[5052]: I1203 12:19:20.998464 5052 scope.go:117] "RemoveContainer" containerID="fff5e81f4e61e210898a9efd838676120b71112c24cdf2b3d314324c91c024eb" Dec 03 12:19:21 crc kubenswrapper[5052]: E1203 12:19:21.003946 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fff5e81f4e61e210898a9efd838676120b71112c24cdf2b3d314324c91c024eb\": container with ID starting with fff5e81f4e61e210898a9efd838676120b71112c24cdf2b3d314324c91c024eb not found: ID does not exist" containerID="fff5e81f4e61e210898a9efd838676120b71112c24cdf2b3d314324c91c024eb" Dec 03 12:19:21 crc kubenswrapper[5052]: I1203 12:19:21.004022 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fff5e81f4e61e210898a9efd838676120b71112c24cdf2b3d314324c91c024eb"} err="failed to get container status \"fff5e81f4e61e210898a9efd838676120b71112c24cdf2b3d314324c91c024eb\": rpc error: code = NotFound desc = could not find container \"fff5e81f4e61e210898a9efd838676120b71112c24cdf2b3d314324c91c024eb\": container with ID starting with fff5e81f4e61e210898a9efd838676120b71112c24cdf2b3d314324c91c024eb not found: ID does not exist" Dec 03 12:19:21 crc kubenswrapper[5052]: I1203 12:19:21.004052 5052 scope.go:117] "RemoveContainer" containerID="0ccd9f77159f708bb0c9e38592a52b58156ee68c539a70c1a8cb2b34e63828bf" Dec 03 12:19:21 crc kubenswrapper[5052]: I1203 12:19:21.004613 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0ccd9f77159f708bb0c9e38592a52b58156ee68c539a70c1a8cb2b34e63828bf"} err="failed to get container status \"0ccd9f77159f708bb0c9e38592a52b58156ee68c539a70c1a8cb2b34e63828bf\": rpc error: code = NotFound desc = could not find container \"0ccd9f77159f708bb0c9e38592a52b58156ee68c539a70c1a8cb2b34e63828bf\": container with ID starting with 0ccd9f77159f708bb0c9e38592a52b58156ee68c539a70c1a8cb2b34e63828bf not found: ID does not exist" Dec 03 12:19:21 crc kubenswrapper[5052]: I1203 12:19:21.004641 5052 scope.go:117] "RemoveContainer" containerID="fff5e81f4e61e210898a9efd838676120b71112c24cdf2b3d314324c91c024eb" Dec 03 12:19:21 crc kubenswrapper[5052]: I1203 12:19:21.005162 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fff5e81f4e61e210898a9efd838676120b71112c24cdf2b3d314324c91c024eb"} err="failed to get container status \"fff5e81f4e61e210898a9efd838676120b71112c24cdf2b3d314324c91c024eb\": rpc error: code = NotFound desc = could not find container \"fff5e81f4e61e210898a9efd838676120b71112c24cdf2b3d314324c91c024eb\": container with ID starting with fff5e81f4e61e210898a9efd838676120b71112c24cdf2b3d314324c91c024eb not found: ID does not exist" Dec 03 12:19:21 crc kubenswrapper[5052]: I1203 12:19:21.006085 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:19:21 crc kubenswrapper[5052]: I1203 12:19:21.094532 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:19:21 crc kubenswrapper[5052]: I1203 12:19:21.143575 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/2f59acc0-b4cd-4648-8203-171a401c87c5-config-data\") pod \"nova-metadata-0\" (UID: \"2f59acc0-b4cd-4648-8203-171a401c87c5\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:19:21 crc kubenswrapper[5052]: I1203 12:19:21.143665 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/2f59acc0-b4cd-4648-8203-171a401c87c5-logs\") pod \"nova-metadata-0\" (UID: \"2f59acc0-b4cd-4648-8203-171a401c87c5\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:19:21 crc kubenswrapper[5052]: I1203 12:19:21.143694 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q9srd\" (UniqueName: \"kubernetes.io/projected/2f59acc0-b4cd-4648-8203-171a401c87c5-kube-api-access-q9srd\") pod \"nova-metadata-0\" (UID: \"2f59acc0-b4cd-4648-8203-171a401c87c5\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:19:21 crc kubenswrapper[5052]: I1203 12:19:21.143911 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2f59acc0-b4cd-4648-8203-171a401c87c5-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"2f59acc0-b4cd-4648-8203-171a401c87c5\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:19:21 crc kubenswrapper[5052]: I1203 12:19:21.144001 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/2f59acc0-b4cd-4648-8203-171a401c87c5-nova-metadata-tls-certs\") pod \"nova-metadata-0\" (UID: \"2f59acc0-b4cd-4648-8203-171a401c87c5\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:19:21 crc kubenswrapper[5052]: I1203 12:19:21.245486 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qqw2l\" (UniqueName: \"kubernetes.io/projected/6b6f7757-7421-4af9-87ce-e0265ee64160-kube-api-access-qqw2l\") pod \"6b6f7757-7421-4af9-87ce-e0265ee64160\" (UID: \"6b6f7757-7421-4af9-87ce-e0265ee64160\") " Dec 03 12:19:21 crc kubenswrapper[5052]: I1203 12:19:21.245762 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6b6f7757-7421-4af9-87ce-e0265ee64160-config-data\") pod \"6b6f7757-7421-4af9-87ce-e0265ee64160\" (UID: \"6b6f7757-7421-4af9-87ce-e0265ee64160\") " Dec 03 12:19:21 crc kubenswrapper[5052]: I1203 12:19:21.245874 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6b6f7757-7421-4af9-87ce-e0265ee64160-combined-ca-bundle\") pod \"6b6f7757-7421-4af9-87ce-e0265ee64160\" (UID: \"6b6f7757-7421-4af9-87ce-e0265ee64160\") " Dec 03 12:19:21 crc kubenswrapper[5052]: I1203 12:19:21.246199 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/2f59acc0-b4cd-4648-8203-171a401c87c5-config-data\") pod \"nova-metadata-0\" (UID: \"2f59acc0-b4cd-4648-8203-171a401c87c5\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:19:21 crc kubenswrapper[5052]: I1203 12:19:21.246261 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/2f59acc0-b4cd-4648-8203-171a401c87c5-logs\") pod \"nova-metadata-0\" (UID: \"2f59acc0-b4cd-4648-8203-171a401c87c5\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:19:21 crc kubenswrapper[5052]: I1203 12:19:21.246293 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-q9srd\" (UniqueName: \"kubernetes.io/projected/2f59acc0-b4cd-4648-8203-171a401c87c5-kube-api-access-q9srd\") pod \"nova-metadata-0\" (UID: \"2f59acc0-b4cd-4648-8203-171a401c87c5\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:19:21 crc kubenswrapper[5052]: I1203 12:19:21.246365 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2f59acc0-b4cd-4648-8203-171a401c87c5-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"2f59acc0-b4cd-4648-8203-171a401c87c5\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:19:21 crc kubenswrapper[5052]: I1203 12:19:21.246412 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/2f59acc0-b4cd-4648-8203-171a401c87c5-nova-metadata-tls-certs\") pod \"nova-metadata-0\" (UID: \"2f59acc0-b4cd-4648-8203-171a401c87c5\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:19:21 crc kubenswrapper[5052]: I1203 12:19:21.247076 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/2f59acc0-b4cd-4648-8203-171a401c87c5-logs\") pod \"nova-metadata-0\" (UID: \"2f59acc0-b4cd-4648-8203-171a401c87c5\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:19:21 crc kubenswrapper[5052]: I1203 12:19:21.250930 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2f59acc0-b4cd-4648-8203-171a401c87c5-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"2f59acc0-b4cd-4648-8203-171a401c87c5\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:19:21 crc kubenswrapper[5052]: I1203 12:19:21.251441 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/2f59acc0-b4cd-4648-8203-171a401c87c5-nova-metadata-tls-certs\") pod \"nova-metadata-0\" (UID: \"2f59acc0-b4cd-4648-8203-171a401c87c5\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:19:21 crc kubenswrapper[5052]: I1203 12:19:21.252722 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/2f59acc0-b4cd-4648-8203-171a401c87c5-config-data\") pod \"nova-metadata-0\" (UID: \"2f59acc0-b4cd-4648-8203-171a401c87c5\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:19:21 crc kubenswrapper[5052]: I1203 12:19:21.259473 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6b6f7757-7421-4af9-87ce-e0265ee64160-kube-api-access-qqw2l" (OuterVolumeSpecName: "kube-api-access-qqw2l") pod "6b6f7757-7421-4af9-87ce-e0265ee64160" (UID: "6b6f7757-7421-4af9-87ce-e0265ee64160"). InnerVolumeSpecName "kube-api-access-qqw2l". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:19:21 crc kubenswrapper[5052]: I1203 12:19:21.261719 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-q9srd\" (UniqueName: \"kubernetes.io/projected/2f59acc0-b4cd-4648-8203-171a401c87c5-kube-api-access-q9srd\") pod \"nova-metadata-0\" (UID: \"2f59acc0-b4cd-4648-8203-171a401c87c5\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:19:21 crc kubenswrapper[5052]: I1203 12:19:21.276785 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6b6f7757-7421-4af9-87ce-e0265ee64160-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "6b6f7757-7421-4af9-87ce-e0265ee64160" (UID: "6b6f7757-7421-4af9-87ce-e0265ee64160"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:19:21 crc kubenswrapper[5052]: I1203 12:19:21.290577 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6b6f7757-7421-4af9-87ce-e0265ee64160-config-data" (OuterVolumeSpecName: "config-data") pod "6b6f7757-7421-4af9-87ce-e0265ee64160" (UID: "6b6f7757-7421-4af9-87ce-e0265ee64160"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:19:21 crc kubenswrapper[5052]: I1203 12:19:21.347836 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6b6f7757-7421-4af9-87ce-e0265ee64160-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:19:21 crc kubenswrapper[5052]: I1203 12:19:21.347866 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-qqw2l\" (UniqueName: \"kubernetes.io/projected/6b6f7757-7421-4af9-87ce-e0265ee64160-kube-api-access-qqw2l\") on node \"crc\" DevicePath \"\"" Dec 03 12:19:21 crc kubenswrapper[5052]: I1203 12:19:21.347878 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6b6f7757-7421-4af9-87ce-e0265ee64160-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:19:21 crc kubenswrapper[5052]: I1203 12:19:21.391150 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:19:21 crc kubenswrapper[5052]: I1203 12:19:21.686778 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bd6ff54a-35e6-4c80-bc3f-caab63ce5239" path="/var/lib/kubelet/pods/bd6ff54a-35e6-4c80-bc3f-caab63ce5239/volumes" Dec 03 12:19:21 crc kubenswrapper[5052]: I1203 12:19:21.840075 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:19:21 crc kubenswrapper[5052]: W1203 12:19:21.844303 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2f59acc0_b4cd_4648_8203_171a401c87c5.slice/crio-07e823909162b7d55ddef29b1c0a1da42be085dad1babf359a11f5fdf94ea42b WatchSource:0}: Error finding container 07e823909162b7d55ddef29b1c0a1da42be085dad1babf359a11f5fdf94ea42b: Status 404 returned error can't find the container with id 07e823909162b7d55ddef29b1c0a1da42be085dad1babf359a11f5fdf94ea42b Dec 03 12:19:21 crc kubenswrapper[5052]: I1203 12:19:21.913192 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"2f59acc0-b4cd-4648-8203-171a401c87c5","Type":"ContainerStarted","Data":"07e823909162b7d55ddef29b1c0a1da42be085dad1babf359a11f5fdf94ea42b"} Dec 03 12:19:21 crc kubenswrapper[5052]: I1203 12:19:21.920421 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" event={"ID":"6b6f7757-7421-4af9-87ce-e0265ee64160","Type":"ContainerDied","Data":"1d0fd17eeba92c661a29a8fca6d6497ac3f2a6d2db34ab542bf22f21f7d1bf44"} Dec 03 12:19:21 crc kubenswrapper[5052]: I1203 12:19:21.920475 5052 scope.go:117] "RemoveContainer" containerID="7f84c127b73452ac5b99f8b132e0617ddd12fa40640f55107cde1118446d6ad1" Dec 03 12:19:21 crc kubenswrapper[5052]: I1203 12:19:21.920635 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:19:21 crc kubenswrapper[5052]: I1203 12:19:21.956402 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 12:19:21 crc kubenswrapper[5052]: I1203 12:19:21.984741 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 12:19:22 crc kubenswrapper[5052]: I1203 12:19:22.000891 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 12:19:22 crc kubenswrapper[5052]: E1203 12:19:22.001316 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="6b6f7757-7421-4af9-87ce-e0265ee64160" containerName="nova-cell1-novncproxy-novncproxy" Dec 03 12:19:22 crc kubenswrapper[5052]: I1203 12:19:22.001336 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="6b6f7757-7421-4af9-87ce-e0265ee64160" containerName="nova-cell1-novncproxy-novncproxy" Dec 03 12:19:22 crc kubenswrapper[5052]: I1203 12:19:22.003425 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="6b6f7757-7421-4af9-87ce-e0265ee64160" containerName="nova-cell1-novncproxy-novncproxy" Dec 03 12:19:22 crc kubenswrapper[5052]: I1203 12:19:22.009164 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 12:19:22 crc kubenswrapper[5052]: I1203 12:19:22.009276 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:19:22 crc kubenswrapper[5052]: I1203 12:19:22.012874 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-nova-novncproxy-cell1-public-svc" Dec 03 12:19:22 crc kubenswrapper[5052]: I1203 12:19:22.013702 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-nova-novncproxy-cell1-vencrypt" Dec 03 12:19:22 crc kubenswrapper[5052]: I1203 12:19:22.018290 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell1-novncproxy-config-data" Dec 03 12:19:22 crc kubenswrapper[5052]: I1203 12:19:22.161633 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/42ebb0e3-9a37-4898-a1e4-0822b4bd9e1e-combined-ca-bundle\") pod \"nova-cell1-novncproxy-0\" (UID: \"42ebb0e3-9a37-4898-a1e4-0822b4bd9e1e\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:19:22 crc kubenswrapper[5052]: I1203 12:19:22.161707 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kfc9r\" (UniqueName: \"kubernetes.io/projected/42ebb0e3-9a37-4898-a1e4-0822b4bd9e1e-kube-api-access-kfc9r\") pod \"nova-cell1-novncproxy-0\" (UID: \"42ebb0e3-9a37-4898-a1e4-0822b4bd9e1e\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:19:22 crc kubenswrapper[5052]: I1203 12:19:22.161772 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/42ebb0e3-9a37-4898-a1e4-0822b4bd9e1e-config-data\") pod \"nova-cell1-novncproxy-0\" (UID: \"42ebb0e3-9a37-4898-a1e4-0822b4bd9e1e\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:19:22 crc kubenswrapper[5052]: I1203 12:19:22.161823 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nova-novncproxy-tls-certs\" (UniqueName: \"kubernetes.io/secret/42ebb0e3-9a37-4898-a1e4-0822b4bd9e1e-nova-novncproxy-tls-certs\") pod \"nova-cell1-novncproxy-0\" (UID: \"42ebb0e3-9a37-4898-a1e4-0822b4bd9e1e\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:19:22 crc kubenswrapper[5052]: I1203 12:19:22.161868 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"vencrypt-tls-certs\" (UniqueName: \"kubernetes.io/secret/42ebb0e3-9a37-4898-a1e4-0822b4bd9e1e-vencrypt-tls-certs\") pod \"nova-cell1-novncproxy-0\" (UID: \"42ebb0e3-9a37-4898-a1e4-0822b4bd9e1e\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:19:22 crc kubenswrapper[5052]: I1203 12:19:22.263829 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/42ebb0e3-9a37-4898-a1e4-0822b4bd9e1e-combined-ca-bundle\") pod \"nova-cell1-novncproxy-0\" (UID: \"42ebb0e3-9a37-4898-a1e4-0822b4bd9e1e\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:19:22 crc kubenswrapper[5052]: I1203 12:19:22.263913 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-kfc9r\" (UniqueName: \"kubernetes.io/projected/42ebb0e3-9a37-4898-a1e4-0822b4bd9e1e-kube-api-access-kfc9r\") pod \"nova-cell1-novncproxy-0\" (UID: \"42ebb0e3-9a37-4898-a1e4-0822b4bd9e1e\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:19:22 crc kubenswrapper[5052]: I1203 12:19:22.263958 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/42ebb0e3-9a37-4898-a1e4-0822b4bd9e1e-config-data\") pod \"nova-cell1-novncproxy-0\" (UID: \"42ebb0e3-9a37-4898-a1e4-0822b4bd9e1e\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:19:22 crc kubenswrapper[5052]: I1203 12:19:22.264023 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"nova-novncproxy-tls-certs\" (UniqueName: \"kubernetes.io/secret/42ebb0e3-9a37-4898-a1e4-0822b4bd9e1e-nova-novncproxy-tls-certs\") pod \"nova-cell1-novncproxy-0\" (UID: \"42ebb0e3-9a37-4898-a1e4-0822b4bd9e1e\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:19:22 crc kubenswrapper[5052]: I1203 12:19:22.264079 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"vencrypt-tls-certs\" (UniqueName: \"kubernetes.io/secret/42ebb0e3-9a37-4898-a1e4-0822b4bd9e1e-vencrypt-tls-certs\") pod \"nova-cell1-novncproxy-0\" (UID: \"42ebb0e3-9a37-4898-a1e4-0822b4bd9e1e\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:19:22 crc kubenswrapper[5052]: I1203 12:19:22.282874 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/42ebb0e3-9a37-4898-a1e4-0822b4bd9e1e-combined-ca-bundle\") pod \"nova-cell1-novncproxy-0\" (UID: \"42ebb0e3-9a37-4898-a1e4-0822b4bd9e1e\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:19:22 crc kubenswrapper[5052]: I1203 12:19:22.282891 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"vencrypt-tls-certs\" (UniqueName: \"kubernetes.io/secret/42ebb0e3-9a37-4898-a1e4-0822b4bd9e1e-vencrypt-tls-certs\") pod \"nova-cell1-novncproxy-0\" (UID: \"42ebb0e3-9a37-4898-a1e4-0822b4bd9e1e\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:19:22 crc kubenswrapper[5052]: I1203 12:19:22.284056 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"nova-novncproxy-tls-certs\" (UniqueName: \"kubernetes.io/secret/42ebb0e3-9a37-4898-a1e4-0822b4bd9e1e-nova-novncproxy-tls-certs\") pod \"nova-cell1-novncproxy-0\" (UID: \"42ebb0e3-9a37-4898-a1e4-0822b4bd9e1e\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:19:22 crc kubenswrapper[5052]: I1203 12:19:22.284575 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/42ebb0e3-9a37-4898-a1e4-0822b4bd9e1e-config-data\") pod \"nova-cell1-novncproxy-0\" (UID: \"42ebb0e3-9a37-4898-a1e4-0822b4bd9e1e\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:19:22 crc kubenswrapper[5052]: I1203 12:19:22.286382 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-kfc9r\" (UniqueName: \"kubernetes.io/projected/42ebb0e3-9a37-4898-a1e4-0822b4bd9e1e-kube-api-access-kfc9r\") pod \"nova-cell1-novncproxy-0\" (UID: \"42ebb0e3-9a37-4898-a1e4-0822b4bd9e1e\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:19:22 crc kubenswrapper[5052]: I1203 12:19:22.501413 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:19:22 crc kubenswrapper[5052]: I1203 12:19:22.933571 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"2f59acc0-b4cd-4648-8203-171a401c87c5","Type":"ContainerStarted","Data":"59214da017963229e1532e88e84fa02f6eac072120b8a11de52c0a16e29bb692"} Dec 03 12:19:22 crc kubenswrapper[5052]: I1203 12:19:22.933978 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"2f59acc0-b4cd-4648-8203-171a401c87c5","Type":"ContainerStarted","Data":"344865ad3061207e03450b813558d8694ba4d0a92d637189861aef405e7d8730"} Dec 03 12:19:23 crc kubenswrapper[5052]: I1203 12:19:23.001539 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 12:19:23 crc kubenswrapper[5052]: I1203 12:19:23.034488 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-metadata-0" podStartSLOduration=3.034467968 podStartE2EDuration="3.034467968s" podCreationTimestamp="2025-12-03 12:19:20 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:19:22.979822039 +0000 UTC m=+3553.568063833" watchObservedRunningTime="2025-12-03 12:19:23.034467968 +0000 UTC m=+3553.622709722" Dec 03 12:19:23 crc kubenswrapper[5052]: I1203 12:19:23.683996 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6b6f7757-7421-4af9-87ce-e0265ee64160" path="/var/lib/kubelet/pods/6b6f7757-7421-4af9-87ce-e0265ee64160/volumes" Dec 03 12:19:23 crc kubenswrapper[5052]: I1203 12:19:23.950933 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" event={"ID":"42ebb0e3-9a37-4898-a1e4-0822b4bd9e1e","Type":"ContainerStarted","Data":"1a78431f63106b64cc7a1e9a2171a9a34a9537578480657d5f058f3cfe1f49bd"} Dec 03 12:19:23 crc kubenswrapper[5052]: I1203 12:19:23.951240 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" event={"ID":"42ebb0e3-9a37-4898-a1e4-0822b4bd9e1e","Type":"ContainerStarted","Data":"3f856f59f54e0aa64029214150a2dd788c92dbcde263e65e41d3708a5652e283"} Dec 03 12:19:23 crc kubenswrapper[5052]: I1203 12:19:23.953057 5052 generic.go:334] "Generic (PLEG): container finished" podID="aaf39e45-e5a6-4ead-82e3-035c43cbc1d7" containerID="9ef2ea9519f50d4f2765e00b600cfe8f29f447a34e20aa72760861bd5ab6b0b0" exitCode=0 Dec 03 12:19:23 crc kubenswrapper[5052]: I1203 12:19:23.953137 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-cell-mapping-pn79k" event={"ID":"aaf39e45-e5a6-4ead-82e3-035c43cbc1d7","Type":"ContainerDied","Data":"9ef2ea9519f50d4f2765e00b600cfe8f29f447a34e20aa72760861bd5ab6b0b0"} Dec 03 12:19:23 crc kubenswrapper[5052]: I1203 12:19:23.977159 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" podStartSLOduration=2.977140923 podStartE2EDuration="2.977140923s" podCreationTimestamp="2025-12-03 12:19:21 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:19:23.967421366 +0000 UTC m=+3554.555663130" watchObservedRunningTime="2025-12-03 12:19:23.977140923 +0000 UTC m=+3554.565382687" Dec 03 12:19:24 crc kubenswrapper[5052]: I1203 12:19:24.983280 5052 generic.go:334] "Generic (PLEG): container finished" podID="c4d2abd6-6282-4c8f-b24d-49ffd64b0ce2" containerID="d2dbcc7e2fdee4a46c5830b20391717220b63cfcd058c8f7699961d14ed09d4e" exitCode=0 Dec 03 12:19:24 crc kubenswrapper[5052]: I1203 12:19:24.983456 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-zlxvc" event={"ID":"c4d2abd6-6282-4c8f-b24d-49ffd64b0ce2","Type":"ContainerDied","Data":"d2dbcc7e2fdee4a46c5830b20391717220b63cfcd058c8f7699961d14ed09d4e"} Dec 03 12:19:25 crc kubenswrapper[5052]: I1203 12:19:25.360999 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-cell-mapping-pn79k" Dec 03 12:19:25 crc kubenswrapper[5052]: I1203 12:19:25.448086 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/aaf39e45-e5a6-4ead-82e3-035c43cbc1d7-config-data\") pod \"aaf39e45-e5a6-4ead-82e3-035c43cbc1d7\" (UID: \"aaf39e45-e5a6-4ead-82e3-035c43cbc1d7\") " Dec 03 12:19:25 crc kubenswrapper[5052]: I1203 12:19:25.448175 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/aaf39e45-e5a6-4ead-82e3-035c43cbc1d7-scripts\") pod \"aaf39e45-e5a6-4ead-82e3-035c43cbc1d7\" (UID: \"aaf39e45-e5a6-4ead-82e3-035c43cbc1d7\") " Dec 03 12:19:25 crc kubenswrapper[5052]: I1203 12:19:25.448232 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/aaf39e45-e5a6-4ead-82e3-035c43cbc1d7-combined-ca-bundle\") pod \"aaf39e45-e5a6-4ead-82e3-035c43cbc1d7\" (UID: \"aaf39e45-e5a6-4ead-82e3-035c43cbc1d7\") " Dec 03 12:19:25 crc kubenswrapper[5052]: I1203 12:19:25.448274 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l2xwb\" (UniqueName: \"kubernetes.io/projected/aaf39e45-e5a6-4ead-82e3-035c43cbc1d7-kube-api-access-l2xwb\") pod \"aaf39e45-e5a6-4ead-82e3-035c43cbc1d7\" (UID: \"aaf39e45-e5a6-4ead-82e3-035c43cbc1d7\") " Dec 03 12:19:25 crc kubenswrapper[5052]: I1203 12:19:25.453767 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aaf39e45-e5a6-4ead-82e3-035c43cbc1d7-kube-api-access-l2xwb" (OuterVolumeSpecName: "kube-api-access-l2xwb") pod "aaf39e45-e5a6-4ead-82e3-035c43cbc1d7" (UID: "aaf39e45-e5a6-4ead-82e3-035c43cbc1d7"). InnerVolumeSpecName "kube-api-access-l2xwb". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:19:25 crc kubenswrapper[5052]: I1203 12:19:25.454020 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aaf39e45-e5a6-4ead-82e3-035c43cbc1d7-scripts" (OuterVolumeSpecName: "scripts") pod "aaf39e45-e5a6-4ead-82e3-035c43cbc1d7" (UID: "aaf39e45-e5a6-4ead-82e3-035c43cbc1d7"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:19:25 crc kubenswrapper[5052]: I1203 12:19:25.476847 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aaf39e45-e5a6-4ead-82e3-035c43cbc1d7-config-data" (OuterVolumeSpecName: "config-data") pod "aaf39e45-e5a6-4ead-82e3-035c43cbc1d7" (UID: "aaf39e45-e5a6-4ead-82e3-035c43cbc1d7"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:19:25 crc kubenswrapper[5052]: I1203 12:19:25.484768 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aaf39e45-e5a6-4ead-82e3-035c43cbc1d7-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "aaf39e45-e5a6-4ead-82e3-035c43cbc1d7" (UID: "aaf39e45-e5a6-4ead-82e3-035c43cbc1d7"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:19:25 crc kubenswrapper[5052]: I1203 12:19:25.550758 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/aaf39e45-e5a6-4ead-82e3-035c43cbc1d7-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:19:25 crc kubenswrapper[5052]: I1203 12:19:25.550795 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/aaf39e45-e5a6-4ead-82e3-035c43cbc1d7-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:19:25 crc kubenswrapper[5052]: I1203 12:19:25.550806 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/aaf39e45-e5a6-4ead-82e3-035c43cbc1d7-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:19:25 crc kubenswrapper[5052]: I1203 12:19:25.550819 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-l2xwb\" (UniqueName: \"kubernetes.io/projected/aaf39e45-e5a6-4ead-82e3-035c43cbc1d7-kube-api-access-l2xwb\") on node \"crc\" DevicePath \"\"" Dec 03 12:19:25 crc kubenswrapper[5052]: I1203 12:19:25.573866 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:19:25 crc kubenswrapper[5052]: I1203 12:19:25.574676 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:19:25 crc kubenswrapper[5052]: I1203 12:19:25.666157 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:19:25 crc kubenswrapper[5052]: I1203 12:19:25.702172 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:19:25 crc kubenswrapper[5052]: I1203 12:19:25.994656 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-cell-mapping-pn79k" Dec 03 12:19:25 crc kubenswrapper[5052]: I1203 12:19:25.994653 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-cell-mapping-pn79k" event={"ID":"aaf39e45-e5a6-4ead-82e3-035c43cbc1d7","Type":"ContainerDied","Data":"23244b815e34569395e1ce5df42dad06cb1c676c383b536d2fc97d53a27452d9"} Dec 03 12:19:25 crc kubenswrapper[5052]: I1203 12:19:25.994724 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="23244b815e34569395e1ce5df42dad06cb1c676c383b536d2fc97d53a27452d9" Dec 03 12:19:26 crc kubenswrapper[5052]: I1203 12:19:26.033354 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:19:26 crc kubenswrapper[5052]: I1203 12:19:26.176438 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:19:26 crc kubenswrapper[5052]: I1203 12:19:26.176678 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-api-0" podUID="b85d1543-4d68-4980-97d2-4168381db238" containerName="nova-api-log" containerID="cri-o://d91c19496f184f5f6be8562c5b5ecd26605542c98135cd177562baf0dce4ffb1" gracePeriod=30 Dec 03 12:19:26 crc kubenswrapper[5052]: I1203 12:19:26.176722 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-api-0" podUID="b85d1543-4d68-4980-97d2-4168381db238" containerName="nova-api-api" containerID="cri-o://f47c386e9ed42113922ee7779b6fcd345651a75d010d2ed9f2e0908c03e6b7e3" gracePeriod=30 Dec 03 12:19:26 crc kubenswrapper[5052]: I1203 12:19:26.202382 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-api-0" podUID="b85d1543-4d68-4980-97d2-4168381db238" containerName="nova-api-api" probeResult="failure" output="Get \"http://10.217.1.128:8774/\": EOF" Dec 03 12:19:26 crc kubenswrapper[5052]: I1203 12:19:26.206400 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-api-0" podUID="b85d1543-4d68-4980-97d2-4168381db238" containerName="nova-api-log" probeResult="failure" output="Get \"http://10.217.1.128:8774/\": EOF" Dec 03 12:19:26 crc kubenswrapper[5052]: I1203 12:19:26.214364 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:19:26 crc kubenswrapper[5052]: I1203 12:19:26.214816 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-metadata-0" podUID="2f59acc0-b4cd-4648-8203-171a401c87c5" containerName="nova-metadata-metadata" containerID="cri-o://59214da017963229e1532e88e84fa02f6eac072120b8a11de52c0a16e29bb692" gracePeriod=30 Dec 03 12:19:26 crc kubenswrapper[5052]: I1203 12:19:26.214968 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-metadata-0" podUID="2f59acc0-b4cd-4648-8203-171a401c87c5" containerName="nova-metadata-log" containerID="cri-o://344865ad3061207e03450b813558d8694ba4d0a92d637189861aef405e7d8730" gracePeriod=30 Dec 03 12:19:26 crc kubenswrapper[5052]: I1203 12:19:26.391900 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:19:26 crc kubenswrapper[5052]: I1203 12:19:26.391942 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:19:26 crc kubenswrapper[5052]: I1203 12:19:26.490565 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-zlxvc" Dec 03 12:19:26 crc kubenswrapper[5052]: I1203 12:19:26.571632 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/c4d2abd6-6282-4c8f-b24d-49ffd64b0ce2-scripts\") pod \"c4d2abd6-6282-4c8f-b24d-49ffd64b0ce2\" (UID: \"c4d2abd6-6282-4c8f-b24d-49ffd64b0ce2\") " Dec 03 12:19:26 crc kubenswrapper[5052]: I1203 12:19:26.571702 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c4d2abd6-6282-4c8f-b24d-49ffd64b0ce2-combined-ca-bundle\") pod \"c4d2abd6-6282-4c8f-b24d-49ffd64b0ce2\" (UID: \"c4d2abd6-6282-4c8f-b24d-49ffd64b0ce2\") " Dec 03 12:19:26 crc kubenswrapper[5052]: I1203 12:19:26.571837 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7g2mq\" (UniqueName: \"kubernetes.io/projected/c4d2abd6-6282-4c8f-b24d-49ffd64b0ce2-kube-api-access-7g2mq\") pod \"c4d2abd6-6282-4c8f-b24d-49ffd64b0ce2\" (UID: \"c4d2abd6-6282-4c8f-b24d-49ffd64b0ce2\") " Dec 03 12:19:26 crc kubenswrapper[5052]: I1203 12:19:26.571951 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/c4d2abd6-6282-4c8f-b24d-49ffd64b0ce2-config-data\") pod \"c4d2abd6-6282-4c8f-b24d-49ffd64b0ce2\" (UID: \"c4d2abd6-6282-4c8f-b24d-49ffd64b0ce2\") " Dec 03 12:19:26 crc kubenswrapper[5052]: I1203 12:19:26.575325 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:19:26 crc kubenswrapper[5052]: I1203 12:19:26.590840 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c4d2abd6-6282-4c8f-b24d-49ffd64b0ce2-scripts" (OuterVolumeSpecName: "scripts") pod "c4d2abd6-6282-4c8f-b24d-49ffd64b0ce2" (UID: "c4d2abd6-6282-4c8f-b24d-49ffd64b0ce2"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:19:26 crc kubenswrapper[5052]: I1203 12:19:26.590953 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c4d2abd6-6282-4c8f-b24d-49ffd64b0ce2-kube-api-access-7g2mq" (OuterVolumeSpecName: "kube-api-access-7g2mq") pod "c4d2abd6-6282-4c8f-b24d-49ffd64b0ce2" (UID: "c4d2abd6-6282-4c8f-b24d-49ffd64b0ce2"). InnerVolumeSpecName "kube-api-access-7g2mq". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:19:26 crc kubenswrapper[5052]: I1203 12:19:26.615040 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c4d2abd6-6282-4c8f-b24d-49ffd64b0ce2-config-data" (OuterVolumeSpecName: "config-data") pod "c4d2abd6-6282-4c8f-b24d-49ffd64b0ce2" (UID: "c4d2abd6-6282-4c8f-b24d-49ffd64b0ce2"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:19:26 crc kubenswrapper[5052]: I1203 12:19:26.615574 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c4d2abd6-6282-4c8f-b24d-49ffd64b0ce2-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "c4d2abd6-6282-4c8f-b24d-49ffd64b0ce2" (UID: "c4d2abd6-6282-4c8f-b24d-49ffd64b0ce2"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:19:26 crc kubenswrapper[5052]: I1203 12:19:26.676921 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/c4d2abd6-6282-4c8f-b24d-49ffd64b0ce2-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:19:26 crc kubenswrapper[5052]: I1203 12:19:26.677881 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c4d2abd6-6282-4c8f-b24d-49ffd64b0ce2-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:19:26 crc kubenswrapper[5052]: I1203 12:19:26.677918 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-7g2mq\" (UniqueName: \"kubernetes.io/projected/c4d2abd6-6282-4c8f-b24d-49ffd64b0ce2-kube-api-access-7g2mq\") on node \"crc\" DevicePath \"\"" Dec 03 12:19:26 crc kubenswrapper[5052]: I1203 12:19:26.677934 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/c4d2abd6-6282-4c8f-b24d-49ffd64b0ce2-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:19:26 crc kubenswrapper[5052]: I1203 12:19:26.772055 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:19:26 crc kubenswrapper[5052]: I1203 12:19:26.880964 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/2f59acc0-b4cd-4648-8203-171a401c87c5-nova-metadata-tls-certs\") pod \"2f59acc0-b4cd-4648-8203-171a401c87c5\" (UID: \"2f59acc0-b4cd-4648-8203-171a401c87c5\") " Dec 03 12:19:26 crc kubenswrapper[5052]: I1203 12:19:26.881228 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q9srd\" (UniqueName: \"kubernetes.io/projected/2f59acc0-b4cd-4648-8203-171a401c87c5-kube-api-access-q9srd\") pod \"2f59acc0-b4cd-4648-8203-171a401c87c5\" (UID: \"2f59acc0-b4cd-4648-8203-171a401c87c5\") " Dec 03 12:19:26 crc kubenswrapper[5052]: I1203 12:19:26.881257 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2f59acc0-b4cd-4648-8203-171a401c87c5-combined-ca-bundle\") pod \"2f59acc0-b4cd-4648-8203-171a401c87c5\" (UID: \"2f59acc0-b4cd-4648-8203-171a401c87c5\") " Dec 03 12:19:26 crc kubenswrapper[5052]: I1203 12:19:26.881323 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/2f59acc0-b4cd-4648-8203-171a401c87c5-config-data\") pod \"2f59acc0-b4cd-4648-8203-171a401c87c5\" (UID: \"2f59acc0-b4cd-4648-8203-171a401c87c5\") " Dec 03 12:19:26 crc kubenswrapper[5052]: I1203 12:19:26.881358 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/2f59acc0-b4cd-4648-8203-171a401c87c5-logs\") pod \"2f59acc0-b4cd-4648-8203-171a401c87c5\" (UID: \"2f59acc0-b4cd-4648-8203-171a401c87c5\") " Dec 03 12:19:26 crc kubenswrapper[5052]: I1203 12:19:26.882277 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f59acc0-b4cd-4648-8203-171a401c87c5-logs" (OuterVolumeSpecName: "logs") pod "2f59acc0-b4cd-4648-8203-171a401c87c5" (UID: "2f59acc0-b4cd-4648-8203-171a401c87c5"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:19:26 crc kubenswrapper[5052]: I1203 12:19:26.885872 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2f59acc0-b4cd-4648-8203-171a401c87c5-kube-api-access-q9srd" (OuterVolumeSpecName: "kube-api-access-q9srd") pod "2f59acc0-b4cd-4648-8203-171a401c87c5" (UID: "2f59acc0-b4cd-4648-8203-171a401c87c5"). InnerVolumeSpecName "kube-api-access-q9srd". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:19:26 crc kubenswrapper[5052]: I1203 12:19:26.907737 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2f59acc0-b4cd-4648-8203-171a401c87c5-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "2f59acc0-b4cd-4648-8203-171a401c87c5" (UID: "2f59acc0-b4cd-4648-8203-171a401c87c5"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:19:26 crc kubenswrapper[5052]: I1203 12:19:26.908121 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2f59acc0-b4cd-4648-8203-171a401c87c5-config-data" (OuterVolumeSpecName: "config-data") pod "2f59acc0-b4cd-4648-8203-171a401c87c5" (UID: "2f59acc0-b4cd-4648-8203-171a401c87c5"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:19:26 crc kubenswrapper[5052]: I1203 12:19:26.937841 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2f59acc0-b4cd-4648-8203-171a401c87c5-nova-metadata-tls-certs" (OuterVolumeSpecName: "nova-metadata-tls-certs") pod "2f59acc0-b4cd-4648-8203-171a401c87c5" (UID: "2f59acc0-b4cd-4648-8203-171a401c87c5"). InnerVolumeSpecName "nova-metadata-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:19:26 crc kubenswrapper[5052]: I1203 12:19:26.984405 5052 reconciler_common.go:293] "Volume detached for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/2f59acc0-b4cd-4648-8203-171a401c87c5-nova-metadata-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:19:26 crc kubenswrapper[5052]: I1203 12:19:26.984444 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-q9srd\" (UniqueName: \"kubernetes.io/projected/2f59acc0-b4cd-4648-8203-171a401c87c5-kube-api-access-q9srd\") on node \"crc\" DevicePath \"\"" Dec 03 12:19:26 crc kubenswrapper[5052]: I1203 12:19:26.984460 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2f59acc0-b4cd-4648-8203-171a401c87c5-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:19:26 crc kubenswrapper[5052]: I1203 12:19:26.984470 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/2f59acc0-b4cd-4648-8203-171a401c87c5-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:19:26 crc kubenswrapper[5052]: I1203 12:19:26.984480 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/2f59acc0-b4cd-4648-8203-171a401c87c5-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:19:27 crc kubenswrapper[5052]: I1203 12:19:27.010062 5052 generic.go:334] "Generic (PLEG): container finished" podID="2f59acc0-b4cd-4648-8203-171a401c87c5" containerID="59214da017963229e1532e88e84fa02f6eac072120b8a11de52c0a16e29bb692" exitCode=0 Dec 03 12:19:27 crc kubenswrapper[5052]: I1203 12:19:27.010105 5052 generic.go:334] "Generic (PLEG): container finished" podID="2f59acc0-b4cd-4648-8203-171a401c87c5" containerID="344865ad3061207e03450b813558d8694ba4d0a92d637189861aef405e7d8730" exitCode=143 Dec 03 12:19:27 crc kubenswrapper[5052]: I1203 12:19:27.010132 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:19:27 crc kubenswrapper[5052]: I1203 12:19:27.010208 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"2f59acc0-b4cd-4648-8203-171a401c87c5","Type":"ContainerDied","Data":"59214da017963229e1532e88e84fa02f6eac072120b8a11de52c0a16e29bb692"} Dec 03 12:19:27 crc kubenswrapper[5052]: I1203 12:19:27.010255 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"2f59acc0-b4cd-4648-8203-171a401c87c5","Type":"ContainerDied","Data":"344865ad3061207e03450b813558d8694ba4d0a92d637189861aef405e7d8730"} Dec 03 12:19:27 crc kubenswrapper[5052]: I1203 12:19:27.010271 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"2f59acc0-b4cd-4648-8203-171a401c87c5","Type":"ContainerDied","Data":"07e823909162b7d55ddef29b1c0a1da42be085dad1babf359a11f5fdf94ea42b"} Dec 03 12:19:27 crc kubenswrapper[5052]: I1203 12:19:27.010293 5052 scope.go:117] "RemoveContainer" containerID="59214da017963229e1532e88e84fa02f6eac072120b8a11de52c0a16e29bb692" Dec 03 12:19:27 crc kubenswrapper[5052]: I1203 12:19:27.035620 5052 generic.go:334] "Generic (PLEG): container finished" podID="b85d1543-4d68-4980-97d2-4168381db238" containerID="d91c19496f184f5f6be8562c5b5ecd26605542c98135cd177562baf0dce4ffb1" exitCode=143 Dec 03 12:19:27 crc kubenswrapper[5052]: I1203 12:19:27.035702 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"b85d1543-4d68-4980-97d2-4168381db238","Type":"ContainerDied","Data":"d91c19496f184f5f6be8562c5b5ecd26605542c98135cd177562baf0dce4ffb1"} Dec 03 12:19:27 crc kubenswrapper[5052]: I1203 12:19:27.038816 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-zlxvc" Dec 03 12:19:27 crc kubenswrapper[5052]: I1203 12:19:27.039119 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-zlxvc" event={"ID":"c4d2abd6-6282-4c8f-b24d-49ffd64b0ce2","Type":"ContainerDied","Data":"8d1e3ef27cf0fa25f7232f45f631c831dde4a79f18a801b9e28d7daead6bf96a"} Dec 03 12:19:27 crc kubenswrapper[5052]: I1203 12:19:27.039147 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8d1e3ef27cf0fa25f7232f45f631c831dde4a79f18a801b9e28d7daead6bf96a" Dec 03 12:19:27 crc kubenswrapper[5052]: I1203 12:19:27.040394 5052 scope.go:117] "RemoveContainer" containerID="344865ad3061207e03450b813558d8694ba4d0a92d637189861aef405e7d8730" Dec 03 12:19:27 crc kubenswrapper[5052]: I1203 12:19:27.071710 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:19:27 crc kubenswrapper[5052]: I1203 12:19:27.073005 5052 scope.go:117] "RemoveContainer" containerID="59214da017963229e1532e88e84fa02f6eac072120b8a11de52c0a16e29bb692" Dec 03 12:19:27 crc kubenswrapper[5052]: E1203 12:19:27.073853 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"59214da017963229e1532e88e84fa02f6eac072120b8a11de52c0a16e29bb692\": container with ID starting with 59214da017963229e1532e88e84fa02f6eac072120b8a11de52c0a16e29bb692 not found: ID does not exist" containerID="59214da017963229e1532e88e84fa02f6eac072120b8a11de52c0a16e29bb692" Dec 03 12:19:27 crc kubenswrapper[5052]: I1203 12:19:27.074019 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"59214da017963229e1532e88e84fa02f6eac072120b8a11de52c0a16e29bb692"} err="failed to get container status \"59214da017963229e1532e88e84fa02f6eac072120b8a11de52c0a16e29bb692\": rpc error: code = NotFound desc = could not find container \"59214da017963229e1532e88e84fa02f6eac072120b8a11de52c0a16e29bb692\": container with ID starting with 59214da017963229e1532e88e84fa02f6eac072120b8a11de52c0a16e29bb692 not found: ID does not exist" Dec 03 12:19:27 crc kubenswrapper[5052]: I1203 12:19:27.074139 5052 scope.go:117] "RemoveContainer" containerID="344865ad3061207e03450b813558d8694ba4d0a92d637189861aef405e7d8730" Dec 03 12:19:27 crc kubenswrapper[5052]: E1203 12:19:27.074556 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"344865ad3061207e03450b813558d8694ba4d0a92d637189861aef405e7d8730\": container with ID starting with 344865ad3061207e03450b813558d8694ba4d0a92d637189861aef405e7d8730 not found: ID does not exist" containerID="344865ad3061207e03450b813558d8694ba4d0a92d637189861aef405e7d8730" Dec 03 12:19:27 crc kubenswrapper[5052]: I1203 12:19:27.074692 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"344865ad3061207e03450b813558d8694ba4d0a92d637189861aef405e7d8730"} err="failed to get container status \"344865ad3061207e03450b813558d8694ba4d0a92d637189861aef405e7d8730\": rpc error: code = NotFound desc = could not find container \"344865ad3061207e03450b813558d8694ba4d0a92d637189861aef405e7d8730\": container with ID starting with 344865ad3061207e03450b813558d8694ba4d0a92d637189861aef405e7d8730 not found: ID does not exist" Dec 03 12:19:27 crc kubenswrapper[5052]: I1203 12:19:27.077470 5052 scope.go:117] "RemoveContainer" containerID="59214da017963229e1532e88e84fa02f6eac072120b8a11de52c0a16e29bb692" Dec 03 12:19:27 crc kubenswrapper[5052]: I1203 12:19:27.078304 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"59214da017963229e1532e88e84fa02f6eac072120b8a11de52c0a16e29bb692"} err="failed to get container status \"59214da017963229e1532e88e84fa02f6eac072120b8a11de52c0a16e29bb692\": rpc error: code = NotFound desc = could not find container \"59214da017963229e1532e88e84fa02f6eac072120b8a11de52c0a16e29bb692\": container with ID starting with 59214da017963229e1532e88e84fa02f6eac072120b8a11de52c0a16e29bb692 not found: ID does not exist" Dec 03 12:19:27 crc kubenswrapper[5052]: I1203 12:19:27.078357 5052 scope.go:117] "RemoveContainer" containerID="344865ad3061207e03450b813558d8694ba4d0a92d637189861aef405e7d8730" Dec 03 12:19:27 crc kubenswrapper[5052]: I1203 12:19:27.080758 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:19:27 crc kubenswrapper[5052]: I1203 12:19:27.081633 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"344865ad3061207e03450b813558d8694ba4d0a92d637189861aef405e7d8730"} err="failed to get container status \"344865ad3061207e03450b813558d8694ba4d0a92d637189861aef405e7d8730\": rpc error: code = NotFound desc = could not find container \"344865ad3061207e03450b813558d8694ba4d0a92d637189861aef405e7d8730\": container with ID starting with 344865ad3061207e03450b813558d8694ba4d0a92d637189861aef405e7d8730 not found: ID does not exist" Dec 03 12:19:27 crc kubenswrapper[5052]: I1203 12:19:27.098557 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:19:27 crc kubenswrapper[5052]: E1203 12:19:27.098972 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="2f59acc0-b4cd-4648-8203-171a401c87c5" containerName="nova-metadata-log" Dec 03 12:19:27 crc kubenswrapper[5052]: I1203 12:19:27.098988 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f59acc0-b4cd-4648-8203-171a401c87c5" containerName="nova-metadata-log" Dec 03 12:19:27 crc kubenswrapper[5052]: E1203 12:19:27.099006 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="2f59acc0-b4cd-4648-8203-171a401c87c5" containerName="nova-metadata-metadata" Dec 03 12:19:27 crc kubenswrapper[5052]: I1203 12:19:27.099012 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f59acc0-b4cd-4648-8203-171a401c87c5" containerName="nova-metadata-metadata" Dec 03 12:19:27 crc kubenswrapper[5052]: E1203 12:19:27.099034 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="aaf39e45-e5a6-4ead-82e3-035c43cbc1d7" containerName="nova-manage" Dec 03 12:19:27 crc kubenswrapper[5052]: I1203 12:19:27.099040 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="aaf39e45-e5a6-4ead-82e3-035c43cbc1d7" containerName="nova-manage" Dec 03 12:19:27 crc kubenswrapper[5052]: E1203 12:19:27.099060 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="c4d2abd6-6282-4c8f-b24d-49ffd64b0ce2" containerName="nova-cell1-conductor-db-sync" Dec 03 12:19:27 crc kubenswrapper[5052]: I1203 12:19:27.099066 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="c4d2abd6-6282-4c8f-b24d-49ffd64b0ce2" containerName="nova-cell1-conductor-db-sync" Dec 03 12:19:27 crc kubenswrapper[5052]: I1203 12:19:27.099234 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="aaf39e45-e5a6-4ead-82e3-035c43cbc1d7" containerName="nova-manage" Dec 03 12:19:27 crc kubenswrapper[5052]: I1203 12:19:27.099244 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="2f59acc0-b4cd-4648-8203-171a401c87c5" containerName="nova-metadata-metadata" Dec 03 12:19:27 crc kubenswrapper[5052]: I1203 12:19:27.099257 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="c4d2abd6-6282-4c8f-b24d-49ffd64b0ce2" containerName="nova-cell1-conductor-db-sync" Dec 03 12:19:27 crc kubenswrapper[5052]: I1203 12:19:27.099270 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="2f59acc0-b4cd-4648-8203-171a401c87c5" containerName="nova-metadata-log" Dec 03 12:19:27 crc kubenswrapper[5052]: I1203 12:19:27.100283 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:19:27 crc kubenswrapper[5052]: I1203 12:19:27.103195 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-nova-metadata-internal-svc" Dec 03 12:19:27 crc kubenswrapper[5052]: I1203 12:19:27.103196 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-metadata-config-data" Dec 03 12:19:27 crc kubenswrapper[5052]: I1203 12:19:27.142335 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:19:27 crc kubenswrapper[5052]: I1203 12:19:27.152443 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-0"] Dec 03 12:19:27 crc kubenswrapper[5052]: I1203 12:19:27.153849 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:19:27 crc kubenswrapper[5052]: I1203 12:19:27.157892 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell1-conductor-config-data" Dec 03 12:19:27 crc kubenswrapper[5052]: I1203 12:19:27.171274 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-0"] Dec 03 12:19:27 crc kubenswrapper[5052]: I1203 12:19:27.187788 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/e159b4fe-4531-4a5e-8a7c-e759dab05223-logs\") pod \"nova-metadata-0\" (UID: \"e159b4fe-4531-4a5e-8a7c-e759dab05223\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:19:27 crc kubenswrapper[5052]: I1203 12:19:27.187936 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/e159b4fe-4531-4a5e-8a7c-e759dab05223-config-data\") pod \"nova-metadata-0\" (UID: \"e159b4fe-4531-4a5e-8a7c-e759dab05223\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:19:27 crc kubenswrapper[5052]: I1203 12:19:27.187966 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/e159b4fe-4531-4a5e-8a7c-e759dab05223-nova-metadata-tls-certs\") pod \"nova-metadata-0\" (UID: \"e159b4fe-4531-4a5e-8a7c-e759dab05223\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:19:27 crc kubenswrapper[5052]: I1203 12:19:27.187998 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cl2bm\" (UniqueName: \"kubernetes.io/projected/e159b4fe-4531-4a5e-8a7c-e759dab05223-kube-api-access-cl2bm\") pod \"nova-metadata-0\" (UID: \"e159b4fe-4531-4a5e-8a7c-e759dab05223\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:19:27 crc kubenswrapper[5052]: I1203 12:19:27.188058 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e159b4fe-4531-4a5e-8a7c-e759dab05223-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"e159b4fe-4531-4a5e-8a7c-e759dab05223\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:19:27 crc kubenswrapper[5052]: I1203 12:19:27.289783 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/aa2b4c07-e5c7-4543-a0a5-f15b8e1b7f83-combined-ca-bundle\") pod \"nova-cell1-conductor-0\" (UID: \"aa2b4c07-e5c7-4543-a0a5-f15b8e1b7f83\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:19:27 crc kubenswrapper[5052]: I1203 12:19:27.289832 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/e159b4fe-4531-4a5e-8a7c-e759dab05223-config-data\") pod \"nova-metadata-0\" (UID: \"e159b4fe-4531-4a5e-8a7c-e759dab05223\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:19:27 crc kubenswrapper[5052]: I1203 12:19:27.289858 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/e159b4fe-4531-4a5e-8a7c-e759dab05223-nova-metadata-tls-certs\") pod \"nova-metadata-0\" (UID: \"e159b4fe-4531-4a5e-8a7c-e759dab05223\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:19:27 crc kubenswrapper[5052]: I1203 12:19:27.289880 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tlncn\" (UniqueName: \"kubernetes.io/projected/aa2b4c07-e5c7-4543-a0a5-f15b8e1b7f83-kube-api-access-tlncn\") pod \"nova-cell1-conductor-0\" (UID: \"aa2b4c07-e5c7-4543-a0a5-f15b8e1b7f83\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:19:27 crc kubenswrapper[5052]: I1203 12:19:27.289905 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-cl2bm\" (UniqueName: \"kubernetes.io/projected/e159b4fe-4531-4a5e-8a7c-e759dab05223-kube-api-access-cl2bm\") pod \"nova-metadata-0\" (UID: \"e159b4fe-4531-4a5e-8a7c-e759dab05223\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:19:27 crc kubenswrapper[5052]: I1203 12:19:27.289923 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/aa2b4c07-e5c7-4543-a0a5-f15b8e1b7f83-config-data\") pod \"nova-cell1-conductor-0\" (UID: \"aa2b4c07-e5c7-4543-a0a5-f15b8e1b7f83\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:19:27 crc kubenswrapper[5052]: I1203 12:19:27.289944 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e159b4fe-4531-4a5e-8a7c-e759dab05223-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"e159b4fe-4531-4a5e-8a7c-e759dab05223\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:19:27 crc kubenswrapper[5052]: I1203 12:19:27.290086 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/e159b4fe-4531-4a5e-8a7c-e759dab05223-logs\") pod \"nova-metadata-0\" (UID: \"e159b4fe-4531-4a5e-8a7c-e759dab05223\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:19:27 crc kubenswrapper[5052]: I1203 12:19:27.290490 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/e159b4fe-4531-4a5e-8a7c-e759dab05223-logs\") pod \"nova-metadata-0\" (UID: \"e159b4fe-4531-4a5e-8a7c-e759dab05223\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:19:27 crc kubenswrapper[5052]: I1203 12:19:27.294305 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/e159b4fe-4531-4a5e-8a7c-e759dab05223-config-data\") pod \"nova-metadata-0\" (UID: \"e159b4fe-4531-4a5e-8a7c-e759dab05223\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:19:27 crc kubenswrapper[5052]: I1203 12:19:27.294334 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e159b4fe-4531-4a5e-8a7c-e759dab05223-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"e159b4fe-4531-4a5e-8a7c-e759dab05223\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:19:27 crc kubenswrapper[5052]: I1203 12:19:27.296088 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/e159b4fe-4531-4a5e-8a7c-e759dab05223-nova-metadata-tls-certs\") pod \"nova-metadata-0\" (UID: \"e159b4fe-4531-4a5e-8a7c-e759dab05223\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:19:27 crc kubenswrapper[5052]: I1203 12:19:27.306694 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-cl2bm\" (UniqueName: \"kubernetes.io/projected/e159b4fe-4531-4a5e-8a7c-e759dab05223-kube-api-access-cl2bm\") pod \"nova-metadata-0\" (UID: \"e159b4fe-4531-4a5e-8a7c-e759dab05223\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:19:27 crc kubenswrapper[5052]: I1203 12:19:27.392508 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/aa2b4c07-e5c7-4543-a0a5-f15b8e1b7f83-combined-ca-bundle\") pod \"nova-cell1-conductor-0\" (UID: \"aa2b4c07-e5c7-4543-a0a5-f15b8e1b7f83\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:19:27 crc kubenswrapper[5052]: I1203 12:19:27.392573 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-tlncn\" (UniqueName: \"kubernetes.io/projected/aa2b4c07-e5c7-4543-a0a5-f15b8e1b7f83-kube-api-access-tlncn\") pod \"nova-cell1-conductor-0\" (UID: \"aa2b4c07-e5c7-4543-a0a5-f15b8e1b7f83\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:19:27 crc kubenswrapper[5052]: I1203 12:19:27.392598 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/aa2b4c07-e5c7-4543-a0a5-f15b8e1b7f83-config-data\") pod \"nova-cell1-conductor-0\" (UID: \"aa2b4c07-e5c7-4543-a0a5-f15b8e1b7f83\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:19:27 crc kubenswrapper[5052]: I1203 12:19:27.396551 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/aa2b4c07-e5c7-4543-a0a5-f15b8e1b7f83-combined-ca-bundle\") pod \"nova-cell1-conductor-0\" (UID: \"aa2b4c07-e5c7-4543-a0a5-f15b8e1b7f83\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:19:27 crc kubenswrapper[5052]: I1203 12:19:27.396580 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/aa2b4c07-e5c7-4543-a0a5-f15b8e1b7f83-config-data\") pod \"nova-cell1-conductor-0\" (UID: \"aa2b4c07-e5c7-4543-a0a5-f15b8e1b7f83\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:19:27 crc kubenswrapper[5052]: I1203 12:19:27.410667 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-tlncn\" (UniqueName: \"kubernetes.io/projected/aa2b4c07-e5c7-4543-a0a5-f15b8e1b7f83-kube-api-access-tlncn\") pod \"nova-cell1-conductor-0\" (UID: \"aa2b4c07-e5c7-4543-a0a5-f15b8e1b7f83\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:19:27 crc kubenswrapper[5052]: I1203 12:19:27.437244 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:19:27 crc kubenswrapper[5052]: I1203 12:19:27.480783 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:19:27 crc kubenswrapper[5052]: I1203 12:19:27.502704 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:19:27 crc kubenswrapper[5052]: I1203 12:19:27.687314 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2f59acc0-b4cd-4648-8203-171a401c87c5" path="/var/lib/kubelet/pods/2f59acc0-b4cd-4648-8203-171a401c87c5/volumes" Dec 03 12:19:27 crc kubenswrapper[5052]: I1203 12:19:27.880335 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-0"] Dec 03 12:19:27 crc kubenswrapper[5052]: I1203 12:19:27.952464 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:19:27 crc kubenswrapper[5052]: W1203 12:19:27.954773 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode159b4fe_4531_4a5e_8a7c_e759dab05223.slice/crio-3eb8343e1e26e1dfb24da9b063349cd8b52440c17a32313eee01848827ddd286 WatchSource:0}: Error finding container 3eb8343e1e26e1dfb24da9b063349cd8b52440c17a32313eee01848827ddd286: Status 404 returned error can't find the container with id 3eb8343e1e26e1dfb24da9b063349cd8b52440c17a32313eee01848827ddd286 Dec 03 12:19:28 crc kubenswrapper[5052]: I1203 12:19:28.050064 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"e159b4fe-4531-4a5e-8a7c-e759dab05223","Type":"ContainerStarted","Data":"3eb8343e1e26e1dfb24da9b063349cd8b52440c17a32313eee01848827ddd286"} Dec 03 12:19:28 crc kubenswrapper[5052]: I1203 12:19:28.052329 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-conductor-0" event={"ID":"aa2b4c07-e5c7-4543-a0a5-f15b8e1b7f83","Type":"ContainerStarted","Data":"272469651cbe8cb9d64d3d9e777fcd39aedc76908ef2aebc1005c9492df664a4"} Dec 03 12:19:28 crc kubenswrapper[5052]: I1203 12:19:28.052438 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-scheduler-0" podUID="986b933a-6cbe-42d6-ab39-7233c006e9f5" containerName="nova-scheduler-scheduler" containerID="cri-o://2a0541c3b6039a96df991332853628579e08c340a094b8c1d44a58e13bc07005" gracePeriod=30 Dec 03 12:19:29 crc kubenswrapper[5052]: I1203 12:19:29.064061 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"e159b4fe-4531-4a5e-8a7c-e759dab05223","Type":"ContainerStarted","Data":"9de7a2c8a8faa15e1d495b079f4635c3cc16c762477ca929b4d0221cf84d7066"} Dec 03 12:19:29 crc kubenswrapper[5052]: I1203 12:19:29.064499 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"e159b4fe-4531-4a5e-8a7c-e759dab05223","Type":"ContainerStarted","Data":"710ef1f3716939feefec91cafc54638d3bd676f9ff55da8928f8ad595f171b67"} Dec 03 12:19:29 crc kubenswrapper[5052]: I1203 12:19:29.066031 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-conductor-0" event={"ID":"aa2b4c07-e5c7-4543-a0a5-f15b8e1b7f83","Type":"ContainerStarted","Data":"4c12bd41f4c701be825345e793ba798c2e3b6ecd4b951f8fb1ee1579f6953903"} Dec 03 12:19:29 crc kubenswrapper[5052]: I1203 12:19:29.066163 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:19:29 crc kubenswrapper[5052]: I1203 12:19:29.087973 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-metadata-0" podStartSLOduration=2.087940747 podStartE2EDuration="2.087940747s" podCreationTimestamp="2025-12-03 12:19:27 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:19:29.078663182 +0000 UTC m=+3559.666904966" watchObservedRunningTime="2025-12-03 12:19:29.087940747 +0000 UTC m=+3559.676182521" Dec 03 12:19:29 crc kubenswrapper[5052]: I1203 12:19:29.101538 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-cell1-conductor-0" podStartSLOduration=2.101502804 podStartE2EDuration="2.101502804s" podCreationTimestamp="2025-12-03 12:19:27 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:19:29.096585833 +0000 UTC m=+3559.684827627" watchObservedRunningTime="2025-12-03 12:19:29.101502804 +0000 UTC m=+3559.689744568" Dec 03 12:19:29 crc kubenswrapper[5052]: I1203 12:19:29.378935 5052 patch_prober.go:28] interesting pod/machine-config-daemon-74bj6 container/machine-config-daemon namespace/openshift-machine-config-operator: Liveness probe status=failure output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" start-of-body= Dec 03 12:19:29 crc kubenswrapper[5052]: I1203 12:19:29.378996 5052 prober.go:107] "Probe failed" probeType="Liveness" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" containerName="machine-config-daemon" probeResult="failure" output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" Dec 03 12:19:30 crc kubenswrapper[5052]: E1203 12:19:30.667980 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="2a0541c3b6039a96df991332853628579e08c340a094b8c1d44a58e13bc07005" cmd=["/usr/bin/pgrep","-r","DRST","nova-scheduler"] Dec 03 12:19:30 crc kubenswrapper[5052]: E1203 12:19:30.670755 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="2a0541c3b6039a96df991332853628579e08c340a094b8c1d44a58e13bc07005" cmd=["/usr/bin/pgrep","-r","DRST","nova-scheduler"] Dec 03 12:19:30 crc kubenswrapper[5052]: E1203 12:19:30.672217 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="2a0541c3b6039a96df991332853628579e08c340a094b8c1d44a58e13bc07005" cmd=["/usr/bin/pgrep","-r","DRST","nova-scheduler"] Dec 03 12:19:30 crc kubenswrapper[5052]: E1203 12:19:30.672263 5052 prober.go:104] "Probe errored" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" probeType="Readiness" pod="openstack-kuttl-tests/nova-scheduler-0" podUID="986b933a-6cbe-42d6-ab39-7233c006e9f5" containerName="nova-scheduler-scheduler" Dec 03 12:19:32 crc kubenswrapper[5052]: I1203 12:19:32.438088 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:19:32 crc kubenswrapper[5052]: I1203 12:19:32.438377 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:19:32 crc kubenswrapper[5052]: I1203 12:19:32.502197 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:19:32 crc kubenswrapper[5052]: I1203 12:19:32.519452 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:19:33 crc kubenswrapper[5052]: I1203 12:19:33.104340 5052 generic.go:334] "Generic (PLEG): container finished" podID="b85d1543-4d68-4980-97d2-4168381db238" containerID="f47c386e9ed42113922ee7779b6fcd345651a75d010d2ed9f2e0908c03e6b7e3" exitCode=0 Dec 03 12:19:33 crc kubenswrapper[5052]: I1203 12:19:33.104459 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"b85d1543-4d68-4980-97d2-4168381db238","Type":"ContainerDied","Data":"f47c386e9ed42113922ee7779b6fcd345651a75d010d2ed9f2e0908c03e6b7e3"} Dec 03 12:19:33 crc kubenswrapper[5052]: I1203 12:19:33.104780 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"b85d1543-4d68-4980-97d2-4168381db238","Type":"ContainerDied","Data":"8967c542141b2c8c6329afc40c6cae659b60319e60f2b2f334ff176ed3cc9b64"} Dec 03 12:19:33 crc kubenswrapper[5052]: I1203 12:19:33.104816 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8967c542141b2c8c6329afc40c6cae659b60319e60f2b2f334ff176ed3cc9b64" Dec 03 12:19:33 crc kubenswrapper[5052]: I1203 12:19:33.109454 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:19:33 crc kubenswrapper[5052]: I1203 12:19:33.124915 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:19:33 crc kubenswrapper[5052]: I1203 12:19:33.233528 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8bjpw\" (UniqueName: \"kubernetes.io/projected/b85d1543-4d68-4980-97d2-4168381db238-kube-api-access-8bjpw\") pod \"b85d1543-4d68-4980-97d2-4168381db238\" (UID: \"b85d1543-4d68-4980-97d2-4168381db238\") " Dec 03 12:19:33 crc kubenswrapper[5052]: I1203 12:19:33.233601 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/b85d1543-4d68-4980-97d2-4168381db238-logs\") pod \"b85d1543-4d68-4980-97d2-4168381db238\" (UID: \"b85d1543-4d68-4980-97d2-4168381db238\") " Dec 03 12:19:33 crc kubenswrapper[5052]: I1203 12:19:33.233818 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/b85d1543-4d68-4980-97d2-4168381db238-config-data\") pod \"b85d1543-4d68-4980-97d2-4168381db238\" (UID: \"b85d1543-4d68-4980-97d2-4168381db238\") " Dec 03 12:19:33 crc kubenswrapper[5052]: I1203 12:19:33.233870 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b85d1543-4d68-4980-97d2-4168381db238-combined-ca-bundle\") pod \"b85d1543-4d68-4980-97d2-4168381db238\" (UID: \"b85d1543-4d68-4980-97d2-4168381db238\") " Dec 03 12:19:33 crc kubenswrapper[5052]: I1203 12:19:33.234077 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b85d1543-4d68-4980-97d2-4168381db238-logs" (OuterVolumeSpecName: "logs") pod "b85d1543-4d68-4980-97d2-4168381db238" (UID: "b85d1543-4d68-4980-97d2-4168381db238"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:19:33 crc kubenswrapper[5052]: I1203 12:19:33.234713 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/b85d1543-4d68-4980-97d2-4168381db238-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:19:33 crc kubenswrapper[5052]: I1203 12:19:33.240041 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b85d1543-4d68-4980-97d2-4168381db238-kube-api-access-8bjpw" (OuterVolumeSpecName: "kube-api-access-8bjpw") pod "b85d1543-4d68-4980-97d2-4168381db238" (UID: "b85d1543-4d68-4980-97d2-4168381db238"). InnerVolumeSpecName "kube-api-access-8bjpw". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:19:33 crc kubenswrapper[5052]: I1203 12:19:33.264675 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b85d1543-4d68-4980-97d2-4168381db238-config-data" (OuterVolumeSpecName: "config-data") pod "b85d1543-4d68-4980-97d2-4168381db238" (UID: "b85d1543-4d68-4980-97d2-4168381db238"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:19:33 crc kubenswrapper[5052]: I1203 12:19:33.264723 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b85d1543-4d68-4980-97d2-4168381db238-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "b85d1543-4d68-4980-97d2-4168381db238" (UID: "b85d1543-4d68-4980-97d2-4168381db238"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:19:33 crc kubenswrapper[5052]: I1203 12:19:33.336918 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/b85d1543-4d68-4980-97d2-4168381db238-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:19:33 crc kubenswrapper[5052]: I1203 12:19:33.336974 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b85d1543-4d68-4980-97d2-4168381db238-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:19:33 crc kubenswrapper[5052]: I1203 12:19:33.336985 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-8bjpw\" (UniqueName: \"kubernetes.io/projected/b85d1543-4d68-4980-97d2-4168381db238-kube-api-access-8bjpw\") on node \"crc\" DevicePath \"\"" Dec 03 12:19:33 crc kubenswrapper[5052]: I1203 12:19:33.828829 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:19:33 crc kubenswrapper[5052]: I1203 12:19:33.950224 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7f5sz\" (UniqueName: \"kubernetes.io/projected/986b933a-6cbe-42d6-ab39-7233c006e9f5-kube-api-access-7f5sz\") pod \"986b933a-6cbe-42d6-ab39-7233c006e9f5\" (UID: \"986b933a-6cbe-42d6-ab39-7233c006e9f5\") " Dec 03 12:19:33 crc kubenswrapper[5052]: I1203 12:19:33.950454 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/986b933a-6cbe-42d6-ab39-7233c006e9f5-config-data\") pod \"986b933a-6cbe-42d6-ab39-7233c006e9f5\" (UID: \"986b933a-6cbe-42d6-ab39-7233c006e9f5\") " Dec 03 12:19:33 crc kubenswrapper[5052]: I1203 12:19:33.950552 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/986b933a-6cbe-42d6-ab39-7233c006e9f5-combined-ca-bundle\") pod \"986b933a-6cbe-42d6-ab39-7233c006e9f5\" (UID: \"986b933a-6cbe-42d6-ab39-7233c006e9f5\") " Dec 03 12:19:33 crc kubenswrapper[5052]: I1203 12:19:33.956243 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/986b933a-6cbe-42d6-ab39-7233c006e9f5-kube-api-access-7f5sz" (OuterVolumeSpecName: "kube-api-access-7f5sz") pod "986b933a-6cbe-42d6-ab39-7233c006e9f5" (UID: "986b933a-6cbe-42d6-ab39-7233c006e9f5"). InnerVolumeSpecName "kube-api-access-7f5sz". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:19:33 crc kubenswrapper[5052]: I1203 12:19:33.976667 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/986b933a-6cbe-42d6-ab39-7233c006e9f5-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "986b933a-6cbe-42d6-ab39-7233c006e9f5" (UID: "986b933a-6cbe-42d6-ab39-7233c006e9f5"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:19:33 crc kubenswrapper[5052]: I1203 12:19:33.976855 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/986b933a-6cbe-42d6-ab39-7233c006e9f5-config-data" (OuterVolumeSpecName: "config-data") pod "986b933a-6cbe-42d6-ab39-7233c006e9f5" (UID: "986b933a-6cbe-42d6-ab39-7233c006e9f5"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:19:34 crc kubenswrapper[5052]: I1203 12:19:34.052419 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-7f5sz\" (UniqueName: \"kubernetes.io/projected/986b933a-6cbe-42d6-ab39-7233c006e9f5-kube-api-access-7f5sz\") on node \"crc\" DevicePath \"\"" Dec 03 12:19:34 crc kubenswrapper[5052]: I1203 12:19:34.052797 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/986b933a-6cbe-42d6-ab39-7233c006e9f5-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:19:34 crc kubenswrapper[5052]: I1203 12:19:34.052809 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/986b933a-6cbe-42d6-ab39-7233c006e9f5-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:19:34 crc kubenswrapper[5052]: I1203 12:19:34.116241 5052 generic.go:334] "Generic (PLEG): container finished" podID="986b933a-6cbe-42d6-ab39-7233c006e9f5" containerID="2a0541c3b6039a96df991332853628579e08c340a094b8c1d44a58e13bc07005" exitCode=0 Dec 03 12:19:34 crc kubenswrapper[5052]: I1203 12:19:34.116321 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:19:34 crc kubenswrapper[5052]: I1203 12:19:34.116346 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"986b933a-6cbe-42d6-ab39-7233c006e9f5","Type":"ContainerDied","Data":"2a0541c3b6039a96df991332853628579e08c340a094b8c1d44a58e13bc07005"} Dec 03 12:19:34 crc kubenswrapper[5052]: I1203 12:19:34.116404 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"986b933a-6cbe-42d6-ab39-7233c006e9f5","Type":"ContainerDied","Data":"1f73e25ff25b51aefbdccb7c39c529ff8e799702ffdb649fec1fa98ffdbf8b1f"} Dec 03 12:19:34 crc kubenswrapper[5052]: I1203 12:19:34.116425 5052 scope.go:117] "RemoveContainer" containerID="2a0541c3b6039a96df991332853628579e08c340a094b8c1d44a58e13bc07005" Dec 03 12:19:34 crc kubenswrapper[5052]: I1203 12:19:34.116577 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:19:34 crc kubenswrapper[5052]: I1203 12:19:34.140387 5052 scope.go:117] "RemoveContainer" containerID="2a0541c3b6039a96df991332853628579e08c340a094b8c1d44a58e13bc07005" Dec 03 12:19:34 crc kubenswrapper[5052]: E1203 12:19:34.140860 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2a0541c3b6039a96df991332853628579e08c340a094b8c1d44a58e13bc07005\": container with ID starting with 2a0541c3b6039a96df991332853628579e08c340a094b8c1d44a58e13bc07005 not found: ID does not exist" containerID="2a0541c3b6039a96df991332853628579e08c340a094b8c1d44a58e13bc07005" Dec 03 12:19:34 crc kubenswrapper[5052]: I1203 12:19:34.140900 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2a0541c3b6039a96df991332853628579e08c340a094b8c1d44a58e13bc07005"} err="failed to get container status \"2a0541c3b6039a96df991332853628579e08c340a094b8c1d44a58e13bc07005\": rpc error: code = NotFound desc = could not find container \"2a0541c3b6039a96df991332853628579e08c340a094b8c1d44a58e13bc07005\": container with ID starting with 2a0541c3b6039a96df991332853628579e08c340a094b8c1d44a58e13bc07005 not found: ID does not exist" Dec 03 12:19:34 crc kubenswrapper[5052]: I1203 12:19:34.147894 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:19:34 crc kubenswrapper[5052]: I1203 12:19:34.170370 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:19:34 crc kubenswrapper[5052]: I1203 12:19:34.184643 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:19:34 crc kubenswrapper[5052]: I1203 12:19:34.203432 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:19:34 crc kubenswrapper[5052]: I1203 12:19:34.215406 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:19:34 crc kubenswrapper[5052]: E1203 12:19:34.215911 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="b85d1543-4d68-4980-97d2-4168381db238" containerName="nova-api-log" Dec 03 12:19:34 crc kubenswrapper[5052]: I1203 12:19:34.215934 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="b85d1543-4d68-4980-97d2-4168381db238" containerName="nova-api-log" Dec 03 12:19:34 crc kubenswrapper[5052]: E1203 12:19:34.215965 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="986b933a-6cbe-42d6-ab39-7233c006e9f5" containerName="nova-scheduler-scheduler" Dec 03 12:19:34 crc kubenswrapper[5052]: I1203 12:19:34.215973 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="986b933a-6cbe-42d6-ab39-7233c006e9f5" containerName="nova-scheduler-scheduler" Dec 03 12:19:34 crc kubenswrapper[5052]: E1203 12:19:34.215991 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="b85d1543-4d68-4980-97d2-4168381db238" containerName="nova-api-api" Dec 03 12:19:34 crc kubenswrapper[5052]: I1203 12:19:34.215999 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="b85d1543-4d68-4980-97d2-4168381db238" containerName="nova-api-api" Dec 03 12:19:34 crc kubenswrapper[5052]: I1203 12:19:34.216833 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="986b933a-6cbe-42d6-ab39-7233c006e9f5" containerName="nova-scheduler-scheduler" Dec 03 12:19:34 crc kubenswrapper[5052]: I1203 12:19:34.216863 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="b85d1543-4d68-4980-97d2-4168381db238" containerName="nova-api-log" Dec 03 12:19:34 crc kubenswrapper[5052]: I1203 12:19:34.216910 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="b85d1543-4d68-4980-97d2-4168381db238" containerName="nova-api-api" Dec 03 12:19:34 crc kubenswrapper[5052]: I1203 12:19:34.218396 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:19:34 crc kubenswrapper[5052]: I1203 12:19:34.221250 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-api-config-data" Dec 03 12:19:34 crc kubenswrapper[5052]: I1203 12:19:34.237646 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:19:34 crc kubenswrapper[5052]: I1203 12:19:34.239119 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:19:34 crc kubenswrapper[5052]: I1203 12:19:34.241614 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-scheduler-config-data" Dec 03 12:19:34 crc kubenswrapper[5052]: I1203 12:19:34.250797 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:19:34 crc kubenswrapper[5052]: I1203 12:19:34.266108 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:19:34 crc kubenswrapper[5052]: I1203 12:19:34.358831 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/d77f169d-276f-4432-9f68-053d75e3c33b-config-data\") pod \"nova-api-0\" (UID: \"d77f169d-276f-4432-9f68-053d75e3c33b\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:19:34 crc kubenswrapper[5052]: I1203 12:19:34.358882 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/d77f169d-276f-4432-9f68-053d75e3c33b-logs\") pod \"nova-api-0\" (UID: \"d77f169d-276f-4432-9f68-053d75e3c33b\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:19:34 crc kubenswrapper[5052]: I1203 12:19:34.358902 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d77f169d-276f-4432-9f68-053d75e3c33b-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"d77f169d-276f-4432-9f68-053d75e3c33b\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:19:34 crc kubenswrapper[5052]: I1203 12:19:34.359099 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d67e60e3-f995-4ff3-82ca-9c7399852935-combined-ca-bundle\") pod \"nova-scheduler-0\" (UID: \"d67e60e3-f995-4ff3-82ca-9c7399852935\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:19:34 crc kubenswrapper[5052]: I1203 12:19:34.359134 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-54dxk\" (UniqueName: \"kubernetes.io/projected/d67e60e3-f995-4ff3-82ca-9c7399852935-kube-api-access-54dxk\") pod \"nova-scheduler-0\" (UID: \"d67e60e3-f995-4ff3-82ca-9c7399852935\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:19:34 crc kubenswrapper[5052]: I1203 12:19:34.359218 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6vz5w\" (UniqueName: \"kubernetes.io/projected/d77f169d-276f-4432-9f68-053d75e3c33b-kube-api-access-6vz5w\") pod \"nova-api-0\" (UID: \"d77f169d-276f-4432-9f68-053d75e3c33b\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:19:34 crc kubenswrapper[5052]: I1203 12:19:34.359468 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/d67e60e3-f995-4ff3-82ca-9c7399852935-config-data\") pod \"nova-scheduler-0\" (UID: \"d67e60e3-f995-4ff3-82ca-9c7399852935\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:19:34 crc kubenswrapper[5052]: I1203 12:19:34.461231 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/d67e60e3-f995-4ff3-82ca-9c7399852935-config-data\") pod \"nova-scheduler-0\" (UID: \"d67e60e3-f995-4ff3-82ca-9c7399852935\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:19:34 crc kubenswrapper[5052]: I1203 12:19:34.461380 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/d77f169d-276f-4432-9f68-053d75e3c33b-config-data\") pod \"nova-api-0\" (UID: \"d77f169d-276f-4432-9f68-053d75e3c33b\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:19:34 crc kubenswrapper[5052]: I1203 12:19:34.461407 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/d77f169d-276f-4432-9f68-053d75e3c33b-logs\") pod \"nova-api-0\" (UID: \"d77f169d-276f-4432-9f68-053d75e3c33b\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:19:34 crc kubenswrapper[5052]: I1203 12:19:34.461434 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d77f169d-276f-4432-9f68-053d75e3c33b-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"d77f169d-276f-4432-9f68-053d75e3c33b\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:19:34 crc kubenswrapper[5052]: I1203 12:19:34.461486 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d67e60e3-f995-4ff3-82ca-9c7399852935-combined-ca-bundle\") pod \"nova-scheduler-0\" (UID: \"d67e60e3-f995-4ff3-82ca-9c7399852935\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:19:34 crc kubenswrapper[5052]: I1203 12:19:34.461512 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-54dxk\" (UniqueName: \"kubernetes.io/projected/d67e60e3-f995-4ff3-82ca-9c7399852935-kube-api-access-54dxk\") pod \"nova-scheduler-0\" (UID: \"d67e60e3-f995-4ff3-82ca-9c7399852935\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:19:34 crc kubenswrapper[5052]: I1203 12:19:34.461560 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-6vz5w\" (UniqueName: \"kubernetes.io/projected/d77f169d-276f-4432-9f68-053d75e3c33b-kube-api-access-6vz5w\") pod \"nova-api-0\" (UID: \"d77f169d-276f-4432-9f68-053d75e3c33b\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:19:34 crc kubenswrapper[5052]: I1203 12:19:34.462375 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/d77f169d-276f-4432-9f68-053d75e3c33b-logs\") pod \"nova-api-0\" (UID: \"d77f169d-276f-4432-9f68-053d75e3c33b\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:19:34 crc kubenswrapper[5052]: I1203 12:19:34.467222 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d67e60e3-f995-4ff3-82ca-9c7399852935-combined-ca-bundle\") pod \"nova-scheduler-0\" (UID: \"d67e60e3-f995-4ff3-82ca-9c7399852935\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:19:34 crc kubenswrapper[5052]: I1203 12:19:34.467393 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/d77f169d-276f-4432-9f68-053d75e3c33b-config-data\") pod \"nova-api-0\" (UID: \"d77f169d-276f-4432-9f68-053d75e3c33b\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:19:34 crc kubenswrapper[5052]: I1203 12:19:34.467401 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/d67e60e3-f995-4ff3-82ca-9c7399852935-config-data\") pod \"nova-scheduler-0\" (UID: \"d67e60e3-f995-4ff3-82ca-9c7399852935\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:19:34 crc kubenswrapper[5052]: I1203 12:19:34.468006 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d77f169d-276f-4432-9f68-053d75e3c33b-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"d77f169d-276f-4432-9f68-053d75e3c33b\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:19:34 crc kubenswrapper[5052]: I1203 12:19:34.482742 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-6vz5w\" (UniqueName: \"kubernetes.io/projected/d77f169d-276f-4432-9f68-053d75e3c33b-kube-api-access-6vz5w\") pod \"nova-api-0\" (UID: \"d77f169d-276f-4432-9f68-053d75e3c33b\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:19:34 crc kubenswrapper[5052]: I1203 12:19:34.483386 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-54dxk\" (UniqueName: \"kubernetes.io/projected/d67e60e3-f995-4ff3-82ca-9c7399852935-kube-api-access-54dxk\") pod \"nova-scheduler-0\" (UID: \"d67e60e3-f995-4ff3-82ca-9c7399852935\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:19:34 crc kubenswrapper[5052]: I1203 12:19:34.540440 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:19:34 crc kubenswrapper[5052]: I1203 12:19:34.562662 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:19:34 crc kubenswrapper[5052]: I1203 12:19:34.931437 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:19:35 crc kubenswrapper[5052]: I1203 12:19:35.053261 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:19:35 crc kubenswrapper[5052]: I1203 12:19:35.089007 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:19:35 crc kubenswrapper[5052]: W1203 12:19:35.094423 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd67e60e3_f995_4ff3_82ca_9c7399852935.slice/crio-81431d536c30f923487d8ab9e8efec22c6be48e6dc51bf50cdd7c81fbf8561c4 WatchSource:0}: Error finding container 81431d536c30f923487d8ab9e8efec22c6be48e6dc51bf50cdd7c81fbf8561c4: Status 404 returned error can't find the container with id 81431d536c30f923487d8ab9e8efec22c6be48e6dc51bf50cdd7c81fbf8561c4 Dec 03 12:19:35 crc kubenswrapper[5052]: I1203 12:19:35.126632 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"d67e60e3-f995-4ff3-82ca-9c7399852935","Type":"ContainerStarted","Data":"81431d536c30f923487d8ab9e8efec22c6be48e6dc51bf50cdd7c81fbf8561c4"} Dec 03 12:19:35 crc kubenswrapper[5052]: I1203 12:19:35.127971 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"d77f169d-276f-4432-9f68-053d75e3c33b","Type":"ContainerStarted","Data":"e54c66fb912393fd3579d21628cb87c7d25600200db17c8d377bf175d5eecd61"} Dec 03 12:19:35 crc kubenswrapper[5052]: I1203 12:19:35.686584 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="986b933a-6cbe-42d6-ab39-7233c006e9f5" path="/var/lib/kubelet/pods/986b933a-6cbe-42d6-ab39-7233c006e9f5/volumes" Dec 03 12:19:35 crc kubenswrapper[5052]: I1203 12:19:35.687168 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b85d1543-4d68-4980-97d2-4168381db238" path="/var/lib/kubelet/pods/b85d1543-4d68-4980-97d2-4168381db238/volumes" Dec 03 12:19:36 crc kubenswrapper[5052]: I1203 12:19:36.163348 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"d67e60e3-f995-4ff3-82ca-9c7399852935","Type":"ContainerStarted","Data":"674698fe5ddff2b2393abcebeef8a9ab776dcd00b5becc26958d8ae3e309669f"} Dec 03 12:19:36 crc kubenswrapper[5052]: I1203 12:19:36.167565 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"d77f169d-276f-4432-9f68-053d75e3c33b","Type":"ContainerStarted","Data":"33737803c5825dc350d3435a9a20cb8c3cb5d557acdac44fc09fe47863149942"} Dec 03 12:19:36 crc kubenswrapper[5052]: I1203 12:19:36.167616 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"d77f169d-276f-4432-9f68-053d75e3c33b","Type":"ContainerStarted","Data":"92b44cc750b48900a9f7dabd65ea43267f4dc1e842bf4ba16982de87b54589a5"} Dec 03 12:19:36 crc kubenswrapper[5052]: I1203 12:19:36.199751 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-api-0" podStartSLOduration=2.199733851 podStartE2EDuration="2.199733851s" podCreationTimestamp="2025-12-03 12:19:34 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:19:36.198613029 +0000 UTC m=+3566.786854793" watchObservedRunningTime="2025-12-03 12:19:36.199733851 +0000 UTC m=+3566.787975615" Dec 03 12:19:36 crc kubenswrapper[5052]: I1203 12:19:36.204792 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-scheduler-0" podStartSLOduration=2.204778365 podStartE2EDuration="2.204778365s" podCreationTimestamp="2025-12-03 12:19:34 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:19:36.183039745 +0000 UTC m=+3566.771281499" watchObservedRunningTime="2025-12-03 12:19:36.204778365 +0000 UTC m=+3566.793020119" Dec 03 12:19:37 crc kubenswrapper[5052]: I1203 12:19:37.438475 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:19:37 crc kubenswrapper[5052]: I1203 12:19:37.438554 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:19:37 crc kubenswrapper[5052]: I1203 12:19:37.512472 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:19:38 crc kubenswrapper[5052]: I1203 12:19:38.082027 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell1-cell-mapping-86wh2"] Dec 03 12:19:38 crc kubenswrapper[5052]: I1203 12:19:38.083777 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-cell-mapping-86wh2" Dec 03 12:19:38 crc kubenswrapper[5052]: I1203 12:19:38.086423 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell1-manage-config-data" Dec 03 12:19:38 crc kubenswrapper[5052]: I1203 12:19:38.087345 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell1-manage-scripts" Dec 03 12:19:38 crc kubenswrapper[5052]: I1203 12:19:38.095623 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-cell-mapping-86wh2"] Dec 03 12:19:38 crc kubenswrapper[5052]: I1203 12:19:38.163605 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z7glj\" (UniqueName: \"kubernetes.io/projected/896a13e1-d614-4be4-9a4a-03b438952ac9-kube-api-access-z7glj\") pod \"nova-cell1-cell-mapping-86wh2\" (UID: \"896a13e1-d614-4be4-9a4a-03b438952ac9\") " pod="openstack-kuttl-tests/nova-cell1-cell-mapping-86wh2" Dec 03 12:19:38 crc kubenswrapper[5052]: I1203 12:19:38.163687 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/896a13e1-d614-4be4-9a4a-03b438952ac9-combined-ca-bundle\") pod \"nova-cell1-cell-mapping-86wh2\" (UID: \"896a13e1-d614-4be4-9a4a-03b438952ac9\") " pod="openstack-kuttl-tests/nova-cell1-cell-mapping-86wh2" Dec 03 12:19:38 crc kubenswrapper[5052]: I1203 12:19:38.163728 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/896a13e1-d614-4be4-9a4a-03b438952ac9-config-data\") pod \"nova-cell1-cell-mapping-86wh2\" (UID: \"896a13e1-d614-4be4-9a4a-03b438952ac9\") " pod="openstack-kuttl-tests/nova-cell1-cell-mapping-86wh2" Dec 03 12:19:38 crc kubenswrapper[5052]: I1203 12:19:38.163763 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/896a13e1-d614-4be4-9a4a-03b438952ac9-scripts\") pod \"nova-cell1-cell-mapping-86wh2\" (UID: \"896a13e1-d614-4be4-9a4a-03b438952ac9\") " pod="openstack-kuttl-tests/nova-cell1-cell-mapping-86wh2" Dec 03 12:19:38 crc kubenswrapper[5052]: I1203 12:19:38.265203 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/896a13e1-d614-4be4-9a4a-03b438952ac9-combined-ca-bundle\") pod \"nova-cell1-cell-mapping-86wh2\" (UID: \"896a13e1-d614-4be4-9a4a-03b438952ac9\") " pod="openstack-kuttl-tests/nova-cell1-cell-mapping-86wh2" Dec 03 12:19:38 crc kubenswrapper[5052]: I1203 12:19:38.265277 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/896a13e1-d614-4be4-9a4a-03b438952ac9-config-data\") pod \"nova-cell1-cell-mapping-86wh2\" (UID: \"896a13e1-d614-4be4-9a4a-03b438952ac9\") " pod="openstack-kuttl-tests/nova-cell1-cell-mapping-86wh2" Dec 03 12:19:38 crc kubenswrapper[5052]: I1203 12:19:38.265317 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/896a13e1-d614-4be4-9a4a-03b438952ac9-scripts\") pod \"nova-cell1-cell-mapping-86wh2\" (UID: \"896a13e1-d614-4be4-9a4a-03b438952ac9\") " pod="openstack-kuttl-tests/nova-cell1-cell-mapping-86wh2" Dec 03 12:19:38 crc kubenswrapper[5052]: I1203 12:19:38.265478 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-z7glj\" (UniqueName: \"kubernetes.io/projected/896a13e1-d614-4be4-9a4a-03b438952ac9-kube-api-access-z7glj\") pod \"nova-cell1-cell-mapping-86wh2\" (UID: \"896a13e1-d614-4be4-9a4a-03b438952ac9\") " pod="openstack-kuttl-tests/nova-cell1-cell-mapping-86wh2" Dec 03 12:19:38 crc kubenswrapper[5052]: I1203 12:19:38.271362 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/896a13e1-d614-4be4-9a4a-03b438952ac9-combined-ca-bundle\") pod \"nova-cell1-cell-mapping-86wh2\" (UID: \"896a13e1-d614-4be4-9a4a-03b438952ac9\") " pod="openstack-kuttl-tests/nova-cell1-cell-mapping-86wh2" Dec 03 12:19:38 crc kubenswrapper[5052]: I1203 12:19:38.273999 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/896a13e1-d614-4be4-9a4a-03b438952ac9-scripts\") pod \"nova-cell1-cell-mapping-86wh2\" (UID: \"896a13e1-d614-4be4-9a4a-03b438952ac9\") " pod="openstack-kuttl-tests/nova-cell1-cell-mapping-86wh2" Dec 03 12:19:38 crc kubenswrapper[5052]: I1203 12:19:38.277392 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/896a13e1-d614-4be4-9a4a-03b438952ac9-config-data\") pod \"nova-cell1-cell-mapping-86wh2\" (UID: \"896a13e1-d614-4be4-9a4a-03b438952ac9\") " pod="openstack-kuttl-tests/nova-cell1-cell-mapping-86wh2" Dec 03 12:19:38 crc kubenswrapper[5052]: I1203 12:19:38.287808 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-z7glj\" (UniqueName: \"kubernetes.io/projected/896a13e1-d614-4be4-9a4a-03b438952ac9-kube-api-access-z7glj\") pod \"nova-cell1-cell-mapping-86wh2\" (UID: \"896a13e1-d614-4be4-9a4a-03b438952ac9\") " pod="openstack-kuttl-tests/nova-cell1-cell-mapping-86wh2" Dec 03 12:19:38 crc kubenswrapper[5052]: I1203 12:19:38.408416 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-cell-mapping-86wh2" Dec 03 12:19:38 crc kubenswrapper[5052]: I1203 12:19:38.451696 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-metadata-0" podUID="e159b4fe-4531-4a5e-8a7c-e759dab05223" containerName="nova-metadata-log" probeResult="failure" output="Get \"https://10.217.1.135:8775/\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Dec 03 12:19:38 crc kubenswrapper[5052]: I1203 12:19:38.451738 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-metadata-0" podUID="e159b4fe-4531-4a5e-8a7c-e759dab05223" containerName="nova-metadata-metadata" probeResult="failure" output="Get \"https://10.217.1.135:8775/\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Dec 03 12:19:38 crc kubenswrapper[5052]: I1203 12:19:38.634891 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/kube-state-metrics-0"] Dec 03 12:19:38 crc kubenswrapper[5052]: I1203 12:19:38.635463 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/kube-state-metrics-0" podUID="d667797a-cec6-4ec5-a853-b56c8e4dce5d" containerName="kube-state-metrics" containerID="cri-o://1fb3da7d65ea115a6ce63ebdae896b5ea20377a51668aa9f9ef49555a99f4e17" gracePeriod=30 Dec 03 12:19:38 crc kubenswrapper[5052]: I1203 12:19:38.872766 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-cell-mapping-86wh2"] Dec 03 12:19:39 crc kubenswrapper[5052]: I1203 12:19:39.029933 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:19:39 crc kubenswrapper[5052]: I1203 12:19:39.180467 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j9gms\" (UniqueName: \"kubernetes.io/projected/d667797a-cec6-4ec5-a853-b56c8e4dce5d-kube-api-access-j9gms\") pod \"d667797a-cec6-4ec5-a853-b56c8e4dce5d\" (UID: \"d667797a-cec6-4ec5-a853-b56c8e4dce5d\") " Dec 03 12:19:39 crc kubenswrapper[5052]: I1203 12:19:39.184593 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d667797a-cec6-4ec5-a853-b56c8e4dce5d-kube-api-access-j9gms" (OuterVolumeSpecName: "kube-api-access-j9gms") pod "d667797a-cec6-4ec5-a853-b56c8e4dce5d" (UID: "d667797a-cec6-4ec5-a853-b56c8e4dce5d"). InnerVolumeSpecName "kube-api-access-j9gms". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:19:39 crc kubenswrapper[5052]: I1203 12:19:39.196416 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-cell-mapping-86wh2" event={"ID":"896a13e1-d614-4be4-9a4a-03b438952ac9","Type":"ContainerStarted","Data":"d99cc4f4b6ecb6e1a9c7ee843b98d6486c6ecec28b0e9d8a368da66230b5fbcc"} Dec 03 12:19:39 crc kubenswrapper[5052]: I1203 12:19:39.196466 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-cell-mapping-86wh2" event={"ID":"896a13e1-d614-4be4-9a4a-03b438952ac9","Type":"ContainerStarted","Data":"596111de5d1e3c7604c47c76e6b20a5dad7526b2bd46490c1721e6ae1994c0b2"} Dec 03 12:19:39 crc kubenswrapper[5052]: I1203 12:19:39.199209 5052 generic.go:334] "Generic (PLEG): container finished" podID="d667797a-cec6-4ec5-a853-b56c8e4dce5d" containerID="1fb3da7d65ea115a6ce63ebdae896b5ea20377a51668aa9f9ef49555a99f4e17" exitCode=2 Dec 03 12:19:39 crc kubenswrapper[5052]: I1203 12:19:39.199263 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/kube-state-metrics-0" event={"ID":"d667797a-cec6-4ec5-a853-b56c8e4dce5d","Type":"ContainerDied","Data":"1fb3da7d65ea115a6ce63ebdae896b5ea20377a51668aa9f9ef49555a99f4e17"} Dec 03 12:19:39 crc kubenswrapper[5052]: I1203 12:19:39.199327 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/kube-state-metrics-0" event={"ID":"d667797a-cec6-4ec5-a853-b56c8e4dce5d","Type":"ContainerDied","Data":"a2c8f1be2e4597df103ced6cb923549ec51501d18c686f6e66ae6351cb596b00"} Dec 03 12:19:39 crc kubenswrapper[5052]: I1203 12:19:39.199374 5052 scope.go:117] "RemoveContainer" containerID="1fb3da7d65ea115a6ce63ebdae896b5ea20377a51668aa9f9ef49555a99f4e17" Dec 03 12:19:39 crc kubenswrapper[5052]: I1203 12:19:39.199285 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:19:39 crc kubenswrapper[5052]: I1203 12:19:39.220486 5052 scope.go:117] "RemoveContainer" containerID="1fb3da7d65ea115a6ce63ebdae896b5ea20377a51668aa9f9ef49555a99f4e17" Dec 03 12:19:39 crc kubenswrapper[5052]: E1203 12:19:39.221081 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1fb3da7d65ea115a6ce63ebdae896b5ea20377a51668aa9f9ef49555a99f4e17\": container with ID starting with 1fb3da7d65ea115a6ce63ebdae896b5ea20377a51668aa9f9ef49555a99f4e17 not found: ID does not exist" containerID="1fb3da7d65ea115a6ce63ebdae896b5ea20377a51668aa9f9ef49555a99f4e17" Dec 03 12:19:39 crc kubenswrapper[5052]: I1203 12:19:39.221166 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1fb3da7d65ea115a6ce63ebdae896b5ea20377a51668aa9f9ef49555a99f4e17"} err="failed to get container status \"1fb3da7d65ea115a6ce63ebdae896b5ea20377a51668aa9f9ef49555a99f4e17\": rpc error: code = NotFound desc = could not find container \"1fb3da7d65ea115a6ce63ebdae896b5ea20377a51668aa9f9ef49555a99f4e17\": container with ID starting with 1fb3da7d65ea115a6ce63ebdae896b5ea20377a51668aa9f9ef49555a99f4e17 not found: ID does not exist" Dec 03 12:19:39 crc kubenswrapper[5052]: I1203 12:19:39.229866 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-cell1-cell-mapping-86wh2" podStartSLOduration=1.229843517 podStartE2EDuration="1.229843517s" podCreationTimestamp="2025-12-03 12:19:38 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:19:39.216265009 +0000 UTC m=+3569.804506773" watchObservedRunningTime="2025-12-03 12:19:39.229843517 +0000 UTC m=+3569.818085271" Dec 03 12:19:39 crc kubenswrapper[5052]: I1203 12:19:39.255351 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/kube-state-metrics-0"] Dec 03 12:19:39 crc kubenswrapper[5052]: I1203 12:19:39.267670 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/kube-state-metrics-0"] Dec 03 12:19:39 crc kubenswrapper[5052]: I1203 12:19:39.284154 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-j9gms\" (UniqueName: \"kubernetes.io/projected/d667797a-cec6-4ec5-a853-b56c8e4dce5d-kube-api-access-j9gms\") on node \"crc\" DevicePath \"\"" Dec 03 12:19:39 crc kubenswrapper[5052]: I1203 12:19:39.307788 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/kube-state-metrics-0"] Dec 03 12:19:39 crc kubenswrapper[5052]: E1203 12:19:39.308721 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="d667797a-cec6-4ec5-a853-b56c8e4dce5d" containerName="kube-state-metrics" Dec 03 12:19:39 crc kubenswrapper[5052]: I1203 12:19:39.308748 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="d667797a-cec6-4ec5-a853-b56c8e4dce5d" containerName="kube-state-metrics" Dec 03 12:19:39 crc kubenswrapper[5052]: I1203 12:19:39.309003 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="d667797a-cec6-4ec5-a853-b56c8e4dce5d" containerName="kube-state-metrics" Dec 03 12:19:39 crc kubenswrapper[5052]: I1203 12:19:39.309839 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:19:39 crc kubenswrapper[5052]: I1203 12:19:39.314341 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/kube-state-metrics-0"] Dec 03 12:19:39 crc kubenswrapper[5052]: I1203 12:19:39.315907 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"kube-state-metrics-tls-config" Dec 03 12:19:39 crc kubenswrapper[5052]: I1203 12:19:39.316159 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-kube-state-metrics-svc" Dec 03 12:19:39 crc kubenswrapper[5052]: I1203 12:19:39.394267 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kvvr6\" (UniqueName: \"kubernetes.io/projected/42e93b9b-3a58-4ca6-abc6-6e359b15e7cc-kube-api-access-kvvr6\") pod \"kube-state-metrics-0\" (UID: \"42e93b9b-3a58-4ca6-abc6-6e359b15e7cc\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:19:39 crc kubenswrapper[5052]: I1203 12:19:39.394653 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/42e93b9b-3a58-4ca6-abc6-6e359b15e7cc-combined-ca-bundle\") pod \"kube-state-metrics-0\" (UID: \"42e93b9b-3a58-4ca6-abc6-6e359b15e7cc\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:19:39 crc kubenswrapper[5052]: I1203 12:19:39.395788 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls-certs\" (UniqueName: \"kubernetes.io/secret/42e93b9b-3a58-4ca6-abc6-6e359b15e7cc-kube-state-metrics-tls-certs\") pod \"kube-state-metrics-0\" (UID: \"42e93b9b-3a58-4ca6-abc6-6e359b15e7cc\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:19:39 crc kubenswrapper[5052]: I1203 12:19:39.396137 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls-config\" (UniqueName: \"kubernetes.io/secret/42e93b9b-3a58-4ca6-abc6-6e359b15e7cc-kube-state-metrics-tls-config\") pod \"kube-state-metrics-0\" (UID: \"42e93b9b-3a58-4ca6-abc6-6e359b15e7cc\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:19:39 crc kubenswrapper[5052]: I1203 12:19:39.499004 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls-certs\" (UniqueName: \"kubernetes.io/secret/42e93b9b-3a58-4ca6-abc6-6e359b15e7cc-kube-state-metrics-tls-certs\") pod \"kube-state-metrics-0\" (UID: \"42e93b9b-3a58-4ca6-abc6-6e359b15e7cc\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:19:39 crc kubenswrapper[5052]: I1203 12:19:39.499205 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls-config\" (UniqueName: \"kubernetes.io/secret/42e93b9b-3a58-4ca6-abc6-6e359b15e7cc-kube-state-metrics-tls-config\") pod \"kube-state-metrics-0\" (UID: \"42e93b9b-3a58-4ca6-abc6-6e359b15e7cc\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:19:39 crc kubenswrapper[5052]: I1203 12:19:39.499377 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-kvvr6\" (UniqueName: \"kubernetes.io/projected/42e93b9b-3a58-4ca6-abc6-6e359b15e7cc-kube-api-access-kvvr6\") pod \"kube-state-metrics-0\" (UID: \"42e93b9b-3a58-4ca6-abc6-6e359b15e7cc\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:19:39 crc kubenswrapper[5052]: I1203 12:19:39.499472 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/42e93b9b-3a58-4ca6-abc6-6e359b15e7cc-combined-ca-bundle\") pod \"kube-state-metrics-0\" (UID: \"42e93b9b-3a58-4ca6-abc6-6e359b15e7cc\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:19:39 crc kubenswrapper[5052]: I1203 12:19:39.504673 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls-certs\" (UniqueName: \"kubernetes.io/secret/42e93b9b-3a58-4ca6-abc6-6e359b15e7cc-kube-state-metrics-tls-certs\") pod \"kube-state-metrics-0\" (UID: \"42e93b9b-3a58-4ca6-abc6-6e359b15e7cc\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:19:39 crc kubenswrapper[5052]: I1203 12:19:39.507149 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/42e93b9b-3a58-4ca6-abc6-6e359b15e7cc-combined-ca-bundle\") pod \"kube-state-metrics-0\" (UID: \"42e93b9b-3a58-4ca6-abc6-6e359b15e7cc\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:19:39 crc kubenswrapper[5052]: I1203 12:19:39.507391 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls-config\" (UniqueName: \"kubernetes.io/secret/42e93b9b-3a58-4ca6-abc6-6e359b15e7cc-kube-state-metrics-tls-config\") pod \"kube-state-metrics-0\" (UID: \"42e93b9b-3a58-4ca6-abc6-6e359b15e7cc\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:19:39 crc kubenswrapper[5052]: I1203 12:19:39.517757 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-kvvr6\" (UniqueName: \"kubernetes.io/projected/42e93b9b-3a58-4ca6-abc6-6e359b15e7cc-kube-api-access-kvvr6\") pod \"kube-state-metrics-0\" (UID: \"42e93b9b-3a58-4ca6-abc6-6e359b15e7cc\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:19:39 crc kubenswrapper[5052]: I1203 12:19:39.563329 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:19:39 crc kubenswrapper[5052]: I1203 12:19:39.644252 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:19:39 crc kubenswrapper[5052]: I1203 12:19:39.689191 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d667797a-cec6-4ec5-a853-b56c8e4dce5d" path="/var/lib/kubelet/pods/d667797a-cec6-4ec5-a853-b56c8e4dce5d/volumes" Dec 03 12:19:40 crc kubenswrapper[5052]: W1203 12:19:40.097223 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod42e93b9b_3a58_4ca6_abc6_6e359b15e7cc.slice/crio-6e5c5ec695992cefa702e431184c66bb5a174101a594a1f0d3057db0a724a8ad WatchSource:0}: Error finding container 6e5c5ec695992cefa702e431184c66bb5a174101a594a1f0d3057db0a724a8ad: Status 404 returned error can't find the container with id 6e5c5ec695992cefa702e431184c66bb5a174101a594a1f0d3057db0a724a8ad Dec 03 12:19:40 crc kubenswrapper[5052]: I1203 12:19:40.107077 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/kube-state-metrics-0"] Dec 03 12:19:40 crc kubenswrapper[5052]: I1203 12:19:40.215568 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/kube-state-metrics-0" event={"ID":"42e93b9b-3a58-4ca6-abc6-6e359b15e7cc","Type":"ContainerStarted","Data":"6e5c5ec695992cefa702e431184c66bb5a174101a594a1f0d3057db0a724a8ad"} Dec 03 12:19:40 crc kubenswrapper[5052]: I1203 12:19:40.650568 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:19:40 crc kubenswrapper[5052]: I1203 12:19:40.650899 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="ee5afa22-9b03-4637-a06d-0a2e50ba8acd" containerName="ceilometer-central-agent" containerID="cri-o://5f2bdd98b42e88a6875dcf0312cee327ad8b12fe0c0406943ebc0874b76993a2" gracePeriod=30 Dec 03 12:19:40 crc kubenswrapper[5052]: I1203 12:19:40.651055 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="ee5afa22-9b03-4637-a06d-0a2e50ba8acd" containerName="proxy-httpd" containerID="cri-o://38388ebd248f12e273f88c8011cab32758fafbd0c18c6044b9721ab2ef11e6ad" gracePeriod=30 Dec 03 12:19:40 crc kubenswrapper[5052]: I1203 12:19:40.651106 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="ee5afa22-9b03-4637-a06d-0a2e50ba8acd" containerName="sg-core" containerID="cri-o://c2a8b0a8115d9df6f7308e52f5f02b46717547b22569fa41657b485815150f85" gracePeriod=30 Dec 03 12:19:40 crc kubenswrapper[5052]: I1203 12:19:40.651152 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="ee5afa22-9b03-4637-a06d-0a2e50ba8acd" containerName="ceilometer-notification-agent" containerID="cri-o://32ed87e381ed568d90156f9ddbc55bf9e8a3e9ece1e4240b61e1888a410caa87" gracePeriod=30 Dec 03 12:19:41 crc kubenswrapper[5052]: I1203 12:19:41.226843 5052 generic.go:334] "Generic (PLEG): container finished" podID="ee5afa22-9b03-4637-a06d-0a2e50ba8acd" containerID="38388ebd248f12e273f88c8011cab32758fafbd0c18c6044b9721ab2ef11e6ad" exitCode=0 Dec 03 12:19:41 crc kubenswrapper[5052]: I1203 12:19:41.227820 5052 generic.go:334] "Generic (PLEG): container finished" podID="ee5afa22-9b03-4637-a06d-0a2e50ba8acd" containerID="c2a8b0a8115d9df6f7308e52f5f02b46717547b22569fa41657b485815150f85" exitCode=2 Dec 03 12:19:41 crc kubenswrapper[5052]: I1203 12:19:41.227839 5052 generic.go:334] "Generic (PLEG): container finished" podID="ee5afa22-9b03-4637-a06d-0a2e50ba8acd" containerID="5f2bdd98b42e88a6875dcf0312cee327ad8b12fe0c0406943ebc0874b76993a2" exitCode=0 Dec 03 12:19:41 crc kubenswrapper[5052]: I1203 12:19:41.227892 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"ee5afa22-9b03-4637-a06d-0a2e50ba8acd","Type":"ContainerDied","Data":"38388ebd248f12e273f88c8011cab32758fafbd0c18c6044b9721ab2ef11e6ad"} Dec 03 12:19:41 crc kubenswrapper[5052]: I1203 12:19:41.227939 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"ee5afa22-9b03-4637-a06d-0a2e50ba8acd","Type":"ContainerDied","Data":"c2a8b0a8115d9df6f7308e52f5f02b46717547b22569fa41657b485815150f85"} Dec 03 12:19:41 crc kubenswrapper[5052]: I1203 12:19:41.227950 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"ee5afa22-9b03-4637-a06d-0a2e50ba8acd","Type":"ContainerDied","Data":"5f2bdd98b42e88a6875dcf0312cee327ad8b12fe0c0406943ebc0874b76993a2"} Dec 03 12:19:41 crc kubenswrapper[5052]: I1203 12:19:41.230078 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/kube-state-metrics-0" event={"ID":"42e93b9b-3a58-4ca6-abc6-6e359b15e7cc","Type":"ContainerStarted","Data":"c572ac5c2d472c1297b5500f42d80ab53aad7c4f8d35a45f1ad28447e9dc6639"} Dec 03 12:19:41 crc kubenswrapper[5052]: I1203 12:19:41.230245 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:19:41 crc kubenswrapper[5052]: I1203 12:19:41.250502 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/kube-state-metrics-0" podStartSLOduration=1.859068358 podStartE2EDuration="2.2504798s" podCreationTimestamp="2025-12-03 12:19:39 +0000 UTC" firstStartedPulling="2025-12-03 12:19:40.102483144 +0000 UTC m=+3570.690724908" lastFinishedPulling="2025-12-03 12:19:40.493894586 +0000 UTC m=+3571.082136350" observedRunningTime="2025-12-03 12:19:41.247219837 +0000 UTC m=+3571.835461601" watchObservedRunningTime="2025-12-03 12:19:41.2504798 +0000 UTC m=+3571.838721564" Dec 03 12:19:44 crc kubenswrapper[5052]: I1203 12:19:44.540735 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:19:44 crc kubenswrapper[5052]: I1203 12:19:44.541106 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:19:44 crc kubenswrapper[5052]: I1203 12:19:44.562889 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:19:44 crc kubenswrapper[5052]: I1203 12:19:44.597008 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:19:45 crc kubenswrapper[5052]: I1203 12:19:45.004818 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:19:45 crc kubenswrapper[5052]: I1203 12:19:45.044346 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ee5afa22-9b03-4637-a06d-0a2e50ba8acd-combined-ca-bundle\") pod \"ee5afa22-9b03-4637-a06d-0a2e50ba8acd\" (UID: \"ee5afa22-9b03-4637-a06d-0a2e50ba8acd\") " Dec 03 12:19:45 crc kubenswrapper[5052]: I1203 12:19:45.044426 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/ee5afa22-9b03-4637-a06d-0a2e50ba8acd-run-httpd\") pod \"ee5afa22-9b03-4637-a06d-0a2e50ba8acd\" (UID: \"ee5afa22-9b03-4637-a06d-0a2e50ba8acd\") " Dec 03 12:19:45 crc kubenswrapper[5052]: I1203 12:19:45.044471 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/ee5afa22-9b03-4637-a06d-0a2e50ba8acd-scripts\") pod \"ee5afa22-9b03-4637-a06d-0a2e50ba8acd\" (UID: \"ee5afa22-9b03-4637-a06d-0a2e50ba8acd\") " Dec 03 12:19:45 crc kubenswrapper[5052]: I1203 12:19:45.044578 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/ee5afa22-9b03-4637-a06d-0a2e50ba8acd-config-data\") pod \"ee5afa22-9b03-4637-a06d-0a2e50ba8acd\" (UID: \"ee5afa22-9b03-4637-a06d-0a2e50ba8acd\") " Dec 03 12:19:45 crc kubenswrapper[5052]: I1203 12:19:45.044660 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/ee5afa22-9b03-4637-a06d-0a2e50ba8acd-log-httpd\") pod \"ee5afa22-9b03-4637-a06d-0a2e50ba8acd\" (UID: \"ee5afa22-9b03-4637-a06d-0a2e50ba8acd\") " Dec 03 12:19:45 crc kubenswrapper[5052]: I1203 12:19:45.044712 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9gwww\" (UniqueName: \"kubernetes.io/projected/ee5afa22-9b03-4637-a06d-0a2e50ba8acd-kube-api-access-9gwww\") pod \"ee5afa22-9b03-4637-a06d-0a2e50ba8acd\" (UID: \"ee5afa22-9b03-4637-a06d-0a2e50ba8acd\") " Dec 03 12:19:45 crc kubenswrapper[5052]: I1203 12:19:45.044746 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/ee5afa22-9b03-4637-a06d-0a2e50ba8acd-sg-core-conf-yaml\") pod \"ee5afa22-9b03-4637-a06d-0a2e50ba8acd\" (UID: \"ee5afa22-9b03-4637-a06d-0a2e50ba8acd\") " Dec 03 12:19:45 crc kubenswrapper[5052]: I1203 12:19:45.045200 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ee5afa22-9b03-4637-a06d-0a2e50ba8acd-log-httpd" (OuterVolumeSpecName: "log-httpd") pod "ee5afa22-9b03-4637-a06d-0a2e50ba8acd" (UID: "ee5afa22-9b03-4637-a06d-0a2e50ba8acd"). InnerVolumeSpecName "log-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:19:45 crc kubenswrapper[5052]: I1203 12:19:45.045596 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ee5afa22-9b03-4637-a06d-0a2e50ba8acd-run-httpd" (OuterVolumeSpecName: "run-httpd") pod "ee5afa22-9b03-4637-a06d-0a2e50ba8acd" (UID: "ee5afa22-9b03-4637-a06d-0a2e50ba8acd"). InnerVolumeSpecName "run-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:19:45 crc kubenswrapper[5052]: I1203 12:19:45.051687 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ee5afa22-9b03-4637-a06d-0a2e50ba8acd-scripts" (OuterVolumeSpecName: "scripts") pod "ee5afa22-9b03-4637-a06d-0a2e50ba8acd" (UID: "ee5afa22-9b03-4637-a06d-0a2e50ba8acd"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:19:45 crc kubenswrapper[5052]: I1203 12:19:45.062977 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ee5afa22-9b03-4637-a06d-0a2e50ba8acd-kube-api-access-9gwww" (OuterVolumeSpecName: "kube-api-access-9gwww") pod "ee5afa22-9b03-4637-a06d-0a2e50ba8acd" (UID: "ee5afa22-9b03-4637-a06d-0a2e50ba8acd"). InnerVolumeSpecName "kube-api-access-9gwww". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:19:45 crc kubenswrapper[5052]: I1203 12:19:45.099937 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ee5afa22-9b03-4637-a06d-0a2e50ba8acd-sg-core-conf-yaml" (OuterVolumeSpecName: "sg-core-conf-yaml") pod "ee5afa22-9b03-4637-a06d-0a2e50ba8acd" (UID: "ee5afa22-9b03-4637-a06d-0a2e50ba8acd"). InnerVolumeSpecName "sg-core-conf-yaml". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:19:45 crc kubenswrapper[5052]: I1203 12:19:45.147758 5052 reconciler_common.go:293] "Volume detached for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/ee5afa22-9b03-4637-a06d-0a2e50ba8acd-log-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:19:45 crc kubenswrapper[5052]: I1203 12:19:45.147835 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-9gwww\" (UniqueName: \"kubernetes.io/projected/ee5afa22-9b03-4637-a06d-0a2e50ba8acd-kube-api-access-9gwww\") on node \"crc\" DevicePath \"\"" Dec 03 12:19:45 crc kubenswrapper[5052]: I1203 12:19:45.147855 5052 reconciler_common.go:293] "Volume detached for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/ee5afa22-9b03-4637-a06d-0a2e50ba8acd-sg-core-conf-yaml\") on node \"crc\" DevicePath \"\"" Dec 03 12:19:45 crc kubenswrapper[5052]: I1203 12:19:45.147873 5052 reconciler_common.go:293] "Volume detached for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/ee5afa22-9b03-4637-a06d-0a2e50ba8acd-run-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:19:45 crc kubenswrapper[5052]: I1203 12:19:45.147886 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/ee5afa22-9b03-4637-a06d-0a2e50ba8acd-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:19:45 crc kubenswrapper[5052]: I1203 12:19:45.155102 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ee5afa22-9b03-4637-a06d-0a2e50ba8acd-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "ee5afa22-9b03-4637-a06d-0a2e50ba8acd" (UID: "ee5afa22-9b03-4637-a06d-0a2e50ba8acd"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:19:45 crc kubenswrapper[5052]: I1203 12:19:45.191693 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ee5afa22-9b03-4637-a06d-0a2e50ba8acd-config-data" (OuterVolumeSpecName: "config-data") pod "ee5afa22-9b03-4637-a06d-0a2e50ba8acd" (UID: "ee5afa22-9b03-4637-a06d-0a2e50ba8acd"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:19:45 crc kubenswrapper[5052]: I1203 12:19:45.251810 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ee5afa22-9b03-4637-a06d-0a2e50ba8acd-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:19:45 crc kubenswrapper[5052]: I1203 12:19:45.252089 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/ee5afa22-9b03-4637-a06d-0a2e50ba8acd-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:19:45 crc kubenswrapper[5052]: I1203 12:19:45.269601 5052 generic.go:334] "Generic (PLEG): container finished" podID="ee5afa22-9b03-4637-a06d-0a2e50ba8acd" containerID="32ed87e381ed568d90156f9ddbc55bf9e8a3e9ece1e4240b61e1888a410caa87" exitCode=0 Dec 03 12:19:45 crc kubenswrapper[5052]: I1203 12:19:45.269670 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:19:45 crc kubenswrapper[5052]: I1203 12:19:45.269704 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"ee5afa22-9b03-4637-a06d-0a2e50ba8acd","Type":"ContainerDied","Data":"32ed87e381ed568d90156f9ddbc55bf9e8a3e9ece1e4240b61e1888a410caa87"} Dec 03 12:19:45 crc kubenswrapper[5052]: I1203 12:19:45.270281 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"ee5afa22-9b03-4637-a06d-0a2e50ba8acd","Type":"ContainerDied","Data":"2583a9197f09e727adfbc00687ed4cf69590f3cc3f3f93e9b2ccc85d86aa6c52"} Dec 03 12:19:45 crc kubenswrapper[5052]: I1203 12:19:45.270302 5052 scope.go:117] "RemoveContainer" containerID="38388ebd248f12e273f88c8011cab32758fafbd0c18c6044b9721ab2ef11e6ad" Dec 03 12:19:45 crc kubenswrapper[5052]: I1203 12:19:45.272487 5052 generic.go:334] "Generic (PLEG): container finished" podID="896a13e1-d614-4be4-9a4a-03b438952ac9" containerID="d99cc4f4b6ecb6e1a9c7ee843b98d6486c6ecec28b0e9d8a368da66230b5fbcc" exitCode=0 Dec 03 12:19:45 crc kubenswrapper[5052]: I1203 12:19:45.272717 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-cell-mapping-86wh2" event={"ID":"896a13e1-d614-4be4-9a4a-03b438952ac9","Type":"ContainerDied","Data":"d99cc4f4b6ecb6e1a9c7ee843b98d6486c6ecec28b0e9d8a368da66230b5fbcc"} Dec 03 12:19:45 crc kubenswrapper[5052]: I1203 12:19:45.319895 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:19:45 crc kubenswrapper[5052]: I1203 12:19:45.320910 5052 scope.go:117] "RemoveContainer" containerID="c2a8b0a8115d9df6f7308e52f5f02b46717547b22569fa41657b485815150f85" Dec 03 12:19:45 crc kubenswrapper[5052]: I1203 12:19:45.338045 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:19:45 crc kubenswrapper[5052]: I1203 12:19:45.350106 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:19:45 crc kubenswrapper[5052]: I1203 12:19:45.381569 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:19:45 crc kubenswrapper[5052]: E1203 12:19:45.382071 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="ee5afa22-9b03-4637-a06d-0a2e50ba8acd" containerName="ceilometer-central-agent" Dec 03 12:19:45 crc kubenswrapper[5052]: I1203 12:19:45.382085 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee5afa22-9b03-4637-a06d-0a2e50ba8acd" containerName="ceilometer-central-agent" Dec 03 12:19:45 crc kubenswrapper[5052]: E1203 12:19:45.382106 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="ee5afa22-9b03-4637-a06d-0a2e50ba8acd" containerName="proxy-httpd" Dec 03 12:19:45 crc kubenswrapper[5052]: I1203 12:19:45.382113 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee5afa22-9b03-4637-a06d-0a2e50ba8acd" containerName="proxy-httpd" Dec 03 12:19:45 crc kubenswrapper[5052]: E1203 12:19:45.382144 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="ee5afa22-9b03-4637-a06d-0a2e50ba8acd" containerName="ceilometer-notification-agent" Dec 03 12:19:45 crc kubenswrapper[5052]: I1203 12:19:45.382152 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee5afa22-9b03-4637-a06d-0a2e50ba8acd" containerName="ceilometer-notification-agent" Dec 03 12:19:45 crc kubenswrapper[5052]: E1203 12:19:45.382172 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="ee5afa22-9b03-4637-a06d-0a2e50ba8acd" containerName="sg-core" Dec 03 12:19:45 crc kubenswrapper[5052]: I1203 12:19:45.382179 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee5afa22-9b03-4637-a06d-0a2e50ba8acd" containerName="sg-core" Dec 03 12:19:45 crc kubenswrapper[5052]: I1203 12:19:45.382413 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="ee5afa22-9b03-4637-a06d-0a2e50ba8acd" containerName="ceilometer-notification-agent" Dec 03 12:19:45 crc kubenswrapper[5052]: I1203 12:19:45.382426 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="ee5afa22-9b03-4637-a06d-0a2e50ba8acd" containerName="proxy-httpd" Dec 03 12:19:45 crc kubenswrapper[5052]: I1203 12:19:45.382439 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="ee5afa22-9b03-4637-a06d-0a2e50ba8acd" containerName="sg-core" Dec 03 12:19:45 crc kubenswrapper[5052]: I1203 12:19:45.382452 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="ee5afa22-9b03-4637-a06d-0a2e50ba8acd" containerName="ceilometer-central-agent" Dec 03 12:19:45 crc kubenswrapper[5052]: I1203 12:19:45.384636 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:19:45 crc kubenswrapper[5052]: I1203 12:19:45.388053 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-config-data" Dec 03 12:19:45 crc kubenswrapper[5052]: I1203 12:19:45.388835 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-scripts" Dec 03 12:19:45 crc kubenswrapper[5052]: I1203 12:19:45.390630 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-ceilometer-internal-svc" Dec 03 12:19:45 crc kubenswrapper[5052]: I1203 12:19:45.398682 5052 scope.go:117] "RemoveContainer" containerID="32ed87e381ed568d90156f9ddbc55bf9e8a3e9ece1e4240b61e1888a410caa87" Dec 03 12:19:45 crc kubenswrapper[5052]: I1203 12:19:45.420787 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:19:45 crc kubenswrapper[5052]: I1203 12:19:45.448716 5052 scope.go:117] "RemoveContainer" containerID="5f2bdd98b42e88a6875dcf0312cee327ad8b12fe0c0406943ebc0874b76993a2" Dec 03 12:19:45 crc kubenswrapper[5052]: I1203 12:19:45.456243 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/9e9cf358-eca4-4774-b1a1-1667774e900f-run-httpd\") pod \"ceilometer-0\" (UID: \"9e9cf358-eca4-4774-b1a1-1667774e900f\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:19:45 crc kubenswrapper[5052]: I1203 12:19:45.456679 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/9e9cf358-eca4-4774-b1a1-1667774e900f-scripts\") pod \"ceilometer-0\" (UID: \"9e9cf358-eca4-4774-b1a1-1667774e900f\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:19:45 crc kubenswrapper[5052]: I1203 12:19:45.456784 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vsqgq\" (UniqueName: \"kubernetes.io/projected/9e9cf358-eca4-4774-b1a1-1667774e900f-kube-api-access-vsqgq\") pod \"ceilometer-0\" (UID: \"9e9cf358-eca4-4774-b1a1-1667774e900f\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:19:45 crc kubenswrapper[5052]: I1203 12:19:45.456877 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9e9cf358-eca4-4774-b1a1-1667774e900f-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"9e9cf358-eca4-4774-b1a1-1667774e900f\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:19:45 crc kubenswrapper[5052]: I1203 12:19:45.456931 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/9e9cf358-eca4-4774-b1a1-1667774e900f-log-httpd\") pod \"ceilometer-0\" (UID: \"9e9cf358-eca4-4774-b1a1-1667774e900f\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:19:45 crc kubenswrapper[5052]: I1203 12:19:45.456954 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/9e9cf358-eca4-4774-b1a1-1667774e900f-config-data\") pod \"ceilometer-0\" (UID: \"9e9cf358-eca4-4774-b1a1-1667774e900f\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:19:45 crc kubenswrapper[5052]: I1203 12:19:45.456983 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/9e9cf358-eca4-4774-b1a1-1667774e900f-ceilometer-tls-certs\") pod \"ceilometer-0\" (UID: \"9e9cf358-eca4-4774-b1a1-1667774e900f\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:19:45 crc kubenswrapper[5052]: I1203 12:19:45.457122 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/9e9cf358-eca4-4774-b1a1-1667774e900f-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"9e9cf358-eca4-4774-b1a1-1667774e900f\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:19:45 crc kubenswrapper[5052]: I1203 12:19:45.473620 5052 scope.go:117] "RemoveContainer" containerID="38388ebd248f12e273f88c8011cab32758fafbd0c18c6044b9721ab2ef11e6ad" Dec 03 12:19:45 crc kubenswrapper[5052]: E1203 12:19:45.474069 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"38388ebd248f12e273f88c8011cab32758fafbd0c18c6044b9721ab2ef11e6ad\": container with ID starting with 38388ebd248f12e273f88c8011cab32758fafbd0c18c6044b9721ab2ef11e6ad not found: ID does not exist" containerID="38388ebd248f12e273f88c8011cab32758fafbd0c18c6044b9721ab2ef11e6ad" Dec 03 12:19:45 crc kubenswrapper[5052]: I1203 12:19:45.474103 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"38388ebd248f12e273f88c8011cab32758fafbd0c18c6044b9721ab2ef11e6ad"} err="failed to get container status \"38388ebd248f12e273f88c8011cab32758fafbd0c18c6044b9721ab2ef11e6ad\": rpc error: code = NotFound desc = could not find container \"38388ebd248f12e273f88c8011cab32758fafbd0c18c6044b9721ab2ef11e6ad\": container with ID starting with 38388ebd248f12e273f88c8011cab32758fafbd0c18c6044b9721ab2ef11e6ad not found: ID does not exist" Dec 03 12:19:45 crc kubenswrapper[5052]: I1203 12:19:45.474124 5052 scope.go:117] "RemoveContainer" containerID="c2a8b0a8115d9df6f7308e52f5f02b46717547b22569fa41657b485815150f85" Dec 03 12:19:45 crc kubenswrapper[5052]: E1203 12:19:45.474477 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c2a8b0a8115d9df6f7308e52f5f02b46717547b22569fa41657b485815150f85\": container with ID starting with c2a8b0a8115d9df6f7308e52f5f02b46717547b22569fa41657b485815150f85 not found: ID does not exist" containerID="c2a8b0a8115d9df6f7308e52f5f02b46717547b22569fa41657b485815150f85" Dec 03 12:19:45 crc kubenswrapper[5052]: I1203 12:19:45.474528 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c2a8b0a8115d9df6f7308e52f5f02b46717547b22569fa41657b485815150f85"} err="failed to get container status \"c2a8b0a8115d9df6f7308e52f5f02b46717547b22569fa41657b485815150f85\": rpc error: code = NotFound desc = could not find container \"c2a8b0a8115d9df6f7308e52f5f02b46717547b22569fa41657b485815150f85\": container with ID starting with c2a8b0a8115d9df6f7308e52f5f02b46717547b22569fa41657b485815150f85 not found: ID does not exist" Dec 03 12:19:45 crc kubenswrapper[5052]: I1203 12:19:45.474552 5052 scope.go:117] "RemoveContainer" containerID="32ed87e381ed568d90156f9ddbc55bf9e8a3e9ece1e4240b61e1888a410caa87" Dec 03 12:19:45 crc kubenswrapper[5052]: E1203 12:19:45.474736 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"32ed87e381ed568d90156f9ddbc55bf9e8a3e9ece1e4240b61e1888a410caa87\": container with ID starting with 32ed87e381ed568d90156f9ddbc55bf9e8a3e9ece1e4240b61e1888a410caa87 not found: ID does not exist" containerID="32ed87e381ed568d90156f9ddbc55bf9e8a3e9ece1e4240b61e1888a410caa87" Dec 03 12:19:45 crc kubenswrapper[5052]: I1203 12:19:45.474798 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"32ed87e381ed568d90156f9ddbc55bf9e8a3e9ece1e4240b61e1888a410caa87"} err="failed to get container status \"32ed87e381ed568d90156f9ddbc55bf9e8a3e9ece1e4240b61e1888a410caa87\": rpc error: code = NotFound desc = could not find container \"32ed87e381ed568d90156f9ddbc55bf9e8a3e9ece1e4240b61e1888a410caa87\": container with ID starting with 32ed87e381ed568d90156f9ddbc55bf9e8a3e9ece1e4240b61e1888a410caa87 not found: ID does not exist" Dec 03 12:19:45 crc kubenswrapper[5052]: I1203 12:19:45.474813 5052 scope.go:117] "RemoveContainer" containerID="5f2bdd98b42e88a6875dcf0312cee327ad8b12fe0c0406943ebc0874b76993a2" Dec 03 12:19:45 crc kubenswrapper[5052]: E1203 12:19:45.474999 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5f2bdd98b42e88a6875dcf0312cee327ad8b12fe0c0406943ebc0874b76993a2\": container with ID starting with 5f2bdd98b42e88a6875dcf0312cee327ad8b12fe0c0406943ebc0874b76993a2 not found: ID does not exist" containerID="5f2bdd98b42e88a6875dcf0312cee327ad8b12fe0c0406943ebc0874b76993a2" Dec 03 12:19:45 crc kubenswrapper[5052]: I1203 12:19:45.475064 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5f2bdd98b42e88a6875dcf0312cee327ad8b12fe0c0406943ebc0874b76993a2"} err="failed to get container status \"5f2bdd98b42e88a6875dcf0312cee327ad8b12fe0c0406943ebc0874b76993a2\": rpc error: code = NotFound desc = could not find container \"5f2bdd98b42e88a6875dcf0312cee327ad8b12fe0c0406943ebc0874b76993a2\": container with ID starting with 5f2bdd98b42e88a6875dcf0312cee327ad8b12fe0c0406943ebc0874b76993a2 not found: ID does not exist" Dec 03 12:19:45 crc kubenswrapper[5052]: I1203 12:19:45.559614 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/9e9cf358-eca4-4774-b1a1-1667774e900f-log-httpd\") pod \"ceilometer-0\" (UID: \"9e9cf358-eca4-4774-b1a1-1667774e900f\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:19:45 crc kubenswrapper[5052]: I1203 12:19:45.559680 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/9e9cf358-eca4-4774-b1a1-1667774e900f-config-data\") pod \"ceilometer-0\" (UID: \"9e9cf358-eca4-4774-b1a1-1667774e900f\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:19:45 crc kubenswrapper[5052]: I1203 12:19:45.559763 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/9e9cf358-eca4-4774-b1a1-1667774e900f-ceilometer-tls-certs\") pod \"ceilometer-0\" (UID: \"9e9cf358-eca4-4774-b1a1-1667774e900f\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:19:45 crc kubenswrapper[5052]: I1203 12:19:45.560080 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/9e9cf358-eca4-4774-b1a1-1667774e900f-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"9e9cf358-eca4-4774-b1a1-1667774e900f\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:19:45 crc kubenswrapper[5052]: I1203 12:19:45.560526 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/9e9cf358-eca4-4774-b1a1-1667774e900f-run-httpd\") pod \"ceilometer-0\" (UID: \"9e9cf358-eca4-4774-b1a1-1667774e900f\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:19:45 crc kubenswrapper[5052]: I1203 12:19:45.560605 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/9e9cf358-eca4-4774-b1a1-1667774e900f-scripts\") pod \"ceilometer-0\" (UID: \"9e9cf358-eca4-4774-b1a1-1667774e900f\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:19:45 crc kubenswrapper[5052]: I1203 12:19:45.560640 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-vsqgq\" (UniqueName: \"kubernetes.io/projected/9e9cf358-eca4-4774-b1a1-1667774e900f-kube-api-access-vsqgq\") pod \"ceilometer-0\" (UID: \"9e9cf358-eca4-4774-b1a1-1667774e900f\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:19:45 crc kubenswrapper[5052]: I1203 12:19:45.560663 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9e9cf358-eca4-4774-b1a1-1667774e900f-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"9e9cf358-eca4-4774-b1a1-1667774e900f\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:19:45 crc kubenswrapper[5052]: I1203 12:19:45.560700 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/9e9cf358-eca4-4774-b1a1-1667774e900f-log-httpd\") pod \"ceilometer-0\" (UID: \"9e9cf358-eca4-4774-b1a1-1667774e900f\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:19:45 crc kubenswrapper[5052]: I1203 12:19:45.560774 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/9e9cf358-eca4-4774-b1a1-1667774e900f-run-httpd\") pod \"ceilometer-0\" (UID: \"9e9cf358-eca4-4774-b1a1-1667774e900f\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:19:45 crc kubenswrapper[5052]: I1203 12:19:45.564277 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/9e9cf358-eca4-4774-b1a1-1667774e900f-ceilometer-tls-certs\") pod \"ceilometer-0\" (UID: \"9e9cf358-eca4-4774-b1a1-1667774e900f\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:19:45 crc kubenswrapper[5052]: I1203 12:19:45.564428 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/9e9cf358-eca4-4774-b1a1-1667774e900f-scripts\") pod \"ceilometer-0\" (UID: \"9e9cf358-eca4-4774-b1a1-1667774e900f\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:19:45 crc kubenswrapper[5052]: I1203 12:19:45.564932 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/9e9cf358-eca4-4774-b1a1-1667774e900f-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"9e9cf358-eca4-4774-b1a1-1667774e900f\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:19:45 crc kubenswrapper[5052]: I1203 12:19:45.564934 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/9e9cf358-eca4-4774-b1a1-1667774e900f-config-data\") pod \"ceilometer-0\" (UID: \"9e9cf358-eca4-4774-b1a1-1667774e900f\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:19:45 crc kubenswrapper[5052]: I1203 12:19:45.574898 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9e9cf358-eca4-4774-b1a1-1667774e900f-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"9e9cf358-eca4-4774-b1a1-1667774e900f\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:19:45 crc kubenswrapper[5052]: I1203 12:19:45.579564 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-vsqgq\" (UniqueName: \"kubernetes.io/projected/9e9cf358-eca4-4774-b1a1-1667774e900f-kube-api-access-vsqgq\") pod \"ceilometer-0\" (UID: \"9e9cf358-eca4-4774-b1a1-1667774e900f\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:19:45 crc kubenswrapper[5052]: I1203 12:19:45.622873 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-api-0" podUID="d77f169d-276f-4432-9f68-053d75e3c33b" containerName="nova-api-log" probeResult="failure" output="Get \"http://10.217.1.137:8774/\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Dec 03 12:19:45 crc kubenswrapper[5052]: I1203 12:19:45.622874 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-api-0" podUID="d77f169d-276f-4432-9f68-053d75e3c33b" containerName="nova-api-api" probeResult="failure" output="Get \"http://10.217.1.137:8774/\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Dec 03 12:19:45 crc kubenswrapper[5052]: I1203 12:19:45.689126 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ee5afa22-9b03-4637-a06d-0a2e50ba8acd" path="/var/lib/kubelet/pods/ee5afa22-9b03-4637-a06d-0a2e50ba8acd/volumes" Dec 03 12:19:45 crc kubenswrapper[5052]: I1203 12:19:45.707688 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:19:46 crc kubenswrapper[5052]: I1203 12:19:46.239294 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:19:46 crc kubenswrapper[5052]: W1203 12:19:46.241336 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9e9cf358_eca4_4774_b1a1_1667774e900f.slice/crio-324894c5addce3772545fa5d883bb38c905b52ac439d4f5c51fa71a3d37ba150 WatchSource:0}: Error finding container 324894c5addce3772545fa5d883bb38c905b52ac439d4f5c51fa71a3d37ba150: Status 404 returned error can't find the container with id 324894c5addce3772545fa5d883bb38c905b52ac439d4f5c51fa71a3d37ba150 Dec 03 12:19:46 crc kubenswrapper[5052]: I1203 12:19:46.296606 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"9e9cf358-eca4-4774-b1a1-1667774e900f","Type":"ContainerStarted","Data":"324894c5addce3772545fa5d883bb38c905b52ac439d4f5c51fa71a3d37ba150"} Dec 03 12:19:46 crc kubenswrapper[5052]: I1203 12:19:46.683628 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-cell-mapping-86wh2" Dec 03 12:19:46 crc kubenswrapper[5052]: I1203 12:19:46.816475 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/896a13e1-d614-4be4-9a4a-03b438952ac9-combined-ca-bundle\") pod \"896a13e1-d614-4be4-9a4a-03b438952ac9\" (UID: \"896a13e1-d614-4be4-9a4a-03b438952ac9\") " Dec 03 12:19:46 crc kubenswrapper[5052]: I1203 12:19:46.816624 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/896a13e1-d614-4be4-9a4a-03b438952ac9-config-data\") pod \"896a13e1-d614-4be4-9a4a-03b438952ac9\" (UID: \"896a13e1-d614-4be4-9a4a-03b438952ac9\") " Dec 03 12:19:46 crc kubenswrapper[5052]: I1203 12:19:46.816675 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/896a13e1-d614-4be4-9a4a-03b438952ac9-scripts\") pod \"896a13e1-d614-4be4-9a4a-03b438952ac9\" (UID: \"896a13e1-d614-4be4-9a4a-03b438952ac9\") " Dec 03 12:19:46 crc kubenswrapper[5052]: I1203 12:19:46.816732 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z7glj\" (UniqueName: \"kubernetes.io/projected/896a13e1-d614-4be4-9a4a-03b438952ac9-kube-api-access-z7glj\") pod \"896a13e1-d614-4be4-9a4a-03b438952ac9\" (UID: \"896a13e1-d614-4be4-9a4a-03b438952ac9\") " Dec 03 12:19:46 crc kubenswrapper[5052]: I1203 12:19:46.826853 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/896a13e1-d614-4be4-9a4a-03b438952ac9-kube-api-access-z7glj" (OuterVolumeSpecName: "kube-api-access-z7glj") pod "896a13e1-d614-4be4-9a4a-03b438952ac9" (UID: "896a13e1-d614-4be4-9a4a-03b438952ac9"). InnerVolumeSpecName "kube-api-access-z7glj". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:19:46 crc kubenswrapper[5052]: I1203 12:19:46.836703 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/896a13e1-d614-4be4-9a4a-03b438952ac9-scripts" (OuterVolumeSpecName: "scripts") pod "896a13e1-d614-4be4-9a4a-03b438952ac9" (UID: "896a13e1-d614-4be4-9a4a-03b438952ac9"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:19:46 crc kubenswrapper[5052]: I1203 12:19:46.850575 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/896a13e1-d614-4be4-9a4a-03b438952ac9-config-data" (OuterVolumeSpecName: "config-data") pod "896a13e1-d614-4be4-9a4a-03b438952ac9" (UID: "896a13e1-d614-4be4-9a4a-03b438952ac9"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:19:46 crc kubenswrapper[5052]: I1203 12:19:46.851187 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/896a13e1-d614-4be4-9a4a-03b438952ac9-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "896a13e1-d614-4be4-9a4a-03b438952ac9" (UID: "896a13e1-d614-4be4-9a4a-03b438952ac9"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:19:46 crc kubenswrapper[5052]: I1203 12:19:46.919601 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/896a13e1-d614-4be4-9a4a-03b438952ac9-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:19:46 crc kubenswrapper[5052]: I1203 12:19:46.919649 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/896a13e1-d614-4be4-9a4a-03b438952ac9-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:19:46 crc kubenswrapper[5052]: I1203 12:19:46.919663 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-z7glj\" (UniqueName: \"kubernetes.io/projected/896a13e1-d614-4be4-9a4a-03b438952ac9-kube-api-access-z7glj\") on node \"crc\" DevicePath \"\"" Dec 03 12:19:46 crc kubenswrapper[5052]: I1203 12:19:46.919677 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/896a13e1-d614-4be4-9a4a-03b438952ac9-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:19:47 crc kubenswrapper[5052]: I1203 12:19:47.319364 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-cell-mapping-86wh2" Dec 03 12:19:47 crc kubenswrapper[5052]: I1203 12:19:47.319612 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-cell-mapping-86wh2" event={"ID":"896a13e1-d614-4be4-9a4a-03b438952ac9","Type":"ContainerDied","Data":"596111de5d1e3c7604c47c76e6b20a5dad7526b2bd46490c1721e6ae1994c0b2"} Dec 03 12:19:47 crc kubenswrapper[5052]: I1203 12:19:47.319669 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="596111de5d1e3c7604c47c76e6b20a5dad7526b2bd46490c1721e6ae1994c0b2" Dec 03 12:19:47 crc kubenswrapper[5052]: I1203 12:19:47.446910 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:19:47 crc kubenswrapper[5052]: I1203 12:19:47.449363 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:19:47 crc kubenswrapper[5052]: I1203 12:19:47.452696 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:19:47 crc kubenswrapper[5052]: I1203 12:19:47.493491 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:19:47 crc kubenswrapper[5052]: I1203 12:19:47.493762 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-api-0" podUID="d77f169d-276f-4432-9f68-053d75e3c33b" containerName="nova-api-log" containerID="cri-o://92b44cc750b48900a9f7dabd65ea43267f4dc1e842bf4ba16982de87b54589a5" gracePeriod=30 Dec 03 12:19:47 crc kubenswrapper[5052]: I1203 12:19:47.493802 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-api-0" podUID="d77f169d-276f-4432-9f68-053d75e3c33b" containerName="nova-api-api" containerID="cri-o://33737803c5825dc350d3435a9a20cb8c3cb5d557acdac44fc09fe47863149942" gracePeriod=30 Dec 03 12:19:47 crc kubenswrapper[5052]: I1203 12:19:47.562898 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:19:47 crc kubenswrapper[5052]: I1203 12:19:47.563117 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-scheduler-0" podUID="d67e60e3-f995-4ff3-82ca-9c7399852935" containerName="nova-scheduler-scheduler" containerID="cri-o://674698fe5ddff2b2393abcebeef8a9ab776dcd00b5becc26958d8ae3e309669f" gracePeriod=30 Dec 03 12:19:47 crc kubenswrapper[5052]: I1203 12:19:47.580198 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:19:48 crc kubenswrapper[5052]: I1203 12:19:48.330783 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"9e9cf358-eca4-4774-b1a1-1667774e900f","Type":"ContainerStarted","Data":"648a05aa3b83b4428ca8cb8790ec8c4c9aa4881d0992e41989e59db80706d517"} Dec 03 12:19:48 crc kubenswrapper[5052]: I1203 12:19:48.331141 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"9e9cf358-eca4-4774-b1a1-1667774e900f","Type":"ContainerStarted","Data":"b1d5d717d0fcd013d58f887c25d6c34fe306a5816962f6a70d25cd36f488c6a3"} Dec 03 12:19:48 crc kubenswrapper[5052]: I1203 12:19:48.333750 5052 generic.go:334] "Generic (PLEG): container finished" podID="d77f169d-276f-4432-9f68-053d75e3c33b" containerID="92b44cc750b48900a9f7dabd65ea43267f4dc1e842bf4ba16982de87b54589a5" exitCode=143 Dec 03 12:19:48 crc kubenswrapper[5052]: I1203 12:19:48.333994 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"d77f169d-276f-4432-9f68-053d75e3c33b","Type":"ContainerDied","Data":"92b44cc750b48900a9f7dabd65ea43267f4dc1e842bf4ba16982de87b54589a5"} Dec 03 12:19:48 crc kubenswrapper[5052]: I1203 12:19:48.343169 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:19:49 crc kubenswrapper[5052]: I1203 12:19:49.345931 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"9e9cf358-eca4-4774-b1a1-1667774e900f","Type":"ContainerStarted","Data":"6ad4071deb854da06386e9a499db2d3dac5c11cb33dde601cceab73c2f94d29b"} Dec 03 12:19:49 crc kubenswrapper[5052]: I1203 12:19:49.346042 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-metadata-0" podUID="e159b4fe-4531-4a5e-8a7c-e759dab05223" containerName="nova-metadata-log" containerID="cri-o://710ef1f3716939feefec91cafc54638d3bd676f9ff55da8928f8ad595f171b67" gracePeriod=30 Dec 03 12:19:49 crc kubenswrapper[5052]: I1203 12:19:49.346453 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-metadata-0" podUID="e159b4fe-4531-4a5e-8a7c-e759dab05223" containerName="nova-metadata-metadata" containerID="cri-o://9de7a2c8a8faa15e1d495b079f4635c3cc16c762477ca929b4d0221cf84d7066" gracePeriod=30 Dec 03 12:19:49 crc kubenswrapper[5052]: E1203 12:19:49.565271 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="674698fe5ddff2b2393abcebeef8a9ab776dcd00b5becc26958d8ae3e309669f" cmd=["/usr/bin/pgrep","-r","DRST","nova-scheduler"] Dec 03 12:19:49 crc kubenswrapper[5052]: E1203 12:19:49.568429 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="674698fe5ddff2b2393abcebeef8a9ab776dcd00b5becc26958d8ae3e309669f" cmd=["/usr/bin/pgrep","-r","DRST","nova-scheduler"] Dec 03 12:19:49 crc kubenswrapper[5052]: E1203 12:19:49.570316 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="674698fe5ddff2b2393abcebeef8a9ab776dcd00b5becc26958d8ae3e309669f" cmd=["/usr/bin/pgrep","-r","DRST","nova-scheduler"] Dec 03 12:19:49 crc kubenswrapper[5052]: E1203 12:19:49.570378 5052 prober.go:104] "Probe errored" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" probeType="Readiness" pod="openstack-kuttl-tests/nova-scheduler-0" podUID="d67e60e3-f995-4ff3-82ca-9c7399852935" containerName="nova-scheduler-scheduler" Dec 03 12:19:49 crc kubenswrapper[5052]: I1203 12:19:49.661406 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:19:50 crc kubenswrapper[5052]: I1203 12:19:50.375561 5052 generic.go:334] "Generic (PLEG): container finished" podID="e159b4fe-4531-4a5e-8a7c-e759dab05223" containerID="710ef1f3716939feefec91cafc54638d3bd676f9ff55da8928f8ad595f171b67" exitCode=143 Dec 03 12:19:50 crc kubenswrapper[5052]: I1203 12:19:50.375970 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"e159b4fe-4531-4a5e-8a7c-e759dab05223","Type":"ContainerDied","Data":"710ef1f3716939feefec91cafc54638d3bd676f9ff55da8928f8ad595f171b67"} Dec 03 12:19:50 crc kubenswrapper[5052]: I1203 12:19:50.380814 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"9e9cf358-eca4-4774-b1a1-1667774e900f","Type":"ContainerStarted","Data":"88e82ab460c499ca6f48460cef37af029831a0965950387bc54369ec6d4ca508"} Dec 03 12:19:50 crc kubenswrapper[5052]: I1203 12:19:50.381854 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:19:50 crc kubenswrapper[5052]: I1203 12:19:50.415228 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/ceilometer-0" podStartSLOduration=1.860482961 podStartE2EDuration="5.41520305s" podCreationTimestamp="2025-12-03 12:19:45 +0000 UTC" firstStartedPulling="2025-12-03 12:19:46.2434832 +0000 UTC m=+3576.831724964" lastFinishedPulling="2025-12-03 12:19:49.798203289 +0000 UTC m=+3580.386445053" observedRunningTime="2025-12-03 12:19:50.404038921 +0000 UTC m=+3580.992280685" watchObservedRunningTime="2025-12-03 12:19:50.41520305 +0000 UTC m=+3581.003444824" Dec 03 12:19:51 crc kubenswrapper[5052]: E1203 12:19:51.306763 5052 cadvisor_stats_provider.go:516] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd77f169d_276f_4432_9f68_053d75e3c33b.slice/crio-33737803c5825dc350d3435a9a20cb8c3cb5d557acdac44fc09fe47863149942.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd77f169d_276f_4432_9f68_053d75e3c33b.slice/crio-conmon-33737803c5825dc350d3435a9a20cb8c3cb5d557acdac44fc09fe47863149942.scope\": RecentStats: unable to find data in memory cache]" Dec 03 12:19:51 crc kubenswrapper[5052]: I1203 12:19:51.430279 5052 generic.go:334] "Generic (PLEG): container finished" podID="d77f169d-276f-4432-9f68-053d75e3c33b" containerID="33737803c5825dc350d3435a9a20cb8c3cb5d557acdac44fc09fe47863149942" exitCode=0 Dec 03 12:19:51 crc kubenswrapper[5052]: I1203 12:19:51.431375 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"d77f169d-276f-4432-9f68-053d75e3c33b","Type":"ContainerDied","Data":"33737803c5825dc350d3435a9a20cb8c3cb5d557acdac44fc09fe47863149942"} Dec 03 12:19:51 crc kubenswrapper[5052]: I1203 12:19:51.650102 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:19:51 crc kubenswrapper[5052]: I1203 12:19:51.724590 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/d77f169d-276f-4432-9f68-053d75e3c33b-config-data\") pod \"d77f169d-276f-4432-9f68-053d75e3c33b\" (UID: \"d77f169d-276f-4432-9f68-053d75e3c33b\") " Dec 03 12:19:51 crc kubenswrapper[5052]: I1203 12:19:51.724919 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6vz5w\" (UniqueName: \"kubernetes.io/projected/d77f169d-276f-4432-9f68-053d75e3c33b-kube-api-access-6vz5w\") pod \"d77f169d-276f-4432-9f68-053d75e3c33b\" (UID: \"d77f169d-276f-4432-9f68-053d75e3c33b\") " Dec 03 12:19:51 crc kubenswrapper[5052]: I1203 12:19:51.725074 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/d77f169d-276f-4432-9f68-053d75e3c33b-logs\") pod \"d77f169d-276f-4432-9f68-053d75e3c33b\" (UID: \"d77f169d-276f-4432-9f68-053d75e3c33b\") " Dec 03 12:19:51 crc kubenswrapper[5052]: I1203 12:19:51.725245 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d77f169d-276f-4432-9f68-053d75e3c33b-combined-ca-bundle\") pod \"d77f169d-276f-4432-9f68-053d75e3c33b\" (UID: \"d77f169d-276f-4432-9f68-053d75e3c33b\") " Dec 03 12:19:51 crc kubenswrapper[5052]: I1203 12:19:51.725553 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d77f169d-276f-4432-9f68-053d75e3c33b-logs" (OuterVolumeSpecName: "logs") pod "d77f169d-276f-4432-9f68-053d75e3c33b" (UID: "d77f169d-276f-4432-9f68-053d75e3c33b"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:19:51 crc kubenswrapper[5052]: I1203 12:19:51.726033 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/d77f169d-276f-4432-9f68-053d75e3c33b-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:19:51 crc kubenswrapper[5052]: I1203 12:19:51.734461 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d77f169d-276f-4432-9f68-053d75e3c33b-kube-api-access-6vz5w" (OuterVolumeSpecName: "kube-api-access-6vz5w") pod "d77f169d-276f-4432-9f68-053d75e3c33b" (UID: "d77f169d-276f-4432-9f68-053d75e3c33b"). InnerVolumeSpecName "kube-api-access-6vz5w". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:19:51 crc kubenswrapper[5052]: I1203 12:19:51.753774 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d77f169d-276f-4432-9f68-053d75e3c33b-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "d77f169d-276f-4432-9f68-053d75e3c33b" (UID: "d77f169d-276f-4432-9f68-053d75e3c33b"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:19:51 crc kubenswrapper[5052]: I1203 12:19:51.759240 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d77f169d-276f-4432-9f68-053d75e3c33b-config-data" (OuterVolumeSpecName: "config-data") pod "d77f169d-276f-4432-9f68-053d75e3c33b" (UID: "d77f169d-276f-4432-9f68-053d75e3c33b"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:19:51 crc kubenswrapper[5052]: I1203 12:19:51.827731 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d77f169d-276f-4432-9f68-053d75e3c33b-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:19:51 crc kubenswrapper[5052]: I1203 12:19:51.827771 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/d77f169d-276f-4432-9f68-053d75e3c33b-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:19:51 crc kubenswrapper[5052]: I1203 12:19:51.827786 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-6vz5w\" (UniqueName: \"kubernetes.io/projected/d77f169d-276f-4432-9f68-053d75e3c33b-kube-api-access-6vz5w\") on node \"crc\" DevicePath \"\"" Dec 03 12:19:52 crc kubenswrapper[5052]: I1203 12:19:52.182120 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:19:52 crc kubenswrapper[5052]: I1203 12:19:52.237930 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-54dxk\" (UniqueName: \"kubernetes.io/projected/d67e60e3-f995-4ff3-82ca-9c7399852935-kube-api-access-54dxk\") pod \"d67e60e3-f995-4ff3-82ca-9c7399852935\" (UID: \"d67e60e3-f995-4ff3-82ca-9c7399852935\") " Dec 03 12:19:52 crc kubenswrapper[5052]: I1203 12:19:52.238092 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/d67e60e3-f995-4ff3-82ca-9c7399852935-config-data\") pod \"d67e60e3-f995-4ff3-82ca-9c7399852935\" (UID: \"d67e60e3-f995-4ff3-82ca-9c7399852935\") " Dec 03 12:19:52 crc kubenswrapper[5052]: I1203 12:19:52.238140 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d67e60e3-f995-4ff3-82ca-9c7399852935-combined-ca-bundle\") pod \"d67e60e3-f995-4ff3-82ca-9c7399852935\" (UID: \"d67e60e3-f995-4ff3-82ca-9c7399852935\") " Dec 03 12:19:52 crc kubenswrapper[5052]: I1203 12:19:52.245641 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d67e60e3-f995-4ff3-82ca-9c7399852935-kube-api-access-54dxk" (OuterVolumeSpecName: "kube-api-access-54dxk") pod "d67e60e3-f995-4ff3-82ca-9c7399852935" (UID: "d67e60e3-f995-4ff3-82ca-9c7399852935"). InnerVolumeSpecName "kube-api-access-54dxk". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:19:52 crc kubenswrapper[5052]: I1203 12:19:52.267745 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d67e60e3-f995-4ff3-82ca-9c7399852935-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "d67e60e3-f995-4ff3-82ca-9c7399852935" (UID: "d67e60e3-f995-4ff3-82ca-9c7399852935"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:19:52 crc kubenswrapper[5052]: I1203 12:19:52.286890 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d67e60e3-f995-4ff3-82ca-9c7399852935-config-data" (OuterVolumeSpecName: "config-data") pod "d67e60e3-f995-4ff3-82ca-9c7399852935" (UID: "d67e60e3-f995-4ff3-82ca-9c7399852935"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:19:52 crc kubenswrapper[5052]: I1203 12:19:52.341520 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-54dxk\" (UniqueName: \"kubernetes.io/projected/d67e60e3-f995-4ff3-82ca-9c7399852935-kube-api-access-54dxk\") on node \"crc\" DevicePath \"\"" Dec 03 12:19:52 crc kubenswrapper[5052]: I1203 12:19:52.341560 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/d67e60e3-f995-4ff3-82ca-9c7399852935-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:19:52 crc kubenswrapper[5052]: I1203 12:19:52.341569 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d67e60e3-f995-4ff3-82ca-9c7399852935-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:19:52 crc kubenswrapper[5052]: I1203 12:19:52.441915 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"d77f169d-276f-4432-9f68-053d75e3c33b","Type":"ContainerDied","Data":"e54c66fb912393fd3579d21628cb87c7d25600200db17c8d377bf175d5eecd61"} Dec 03 12:19:52 crc kubenswrapper[5052]: I1203 12:19:52.442309 5052 scope.go:117] "RemoveContainer" containerID="33737803c5825dc350d3435a9a20cb8c3cb5d557acdac44fc09fe47863149942" Dec 03 12:19:52 crc kubenswrapper[5052]: I1203 12:19:52.441993 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:19:52 crc kubenswrapper[5052]: I1203 12:19:52.443994 5052 generic.go:334] "Generic (PLEG): container finished" podID="d67e60e3-f995-4ff3-82ca-9c7399852935" containerID="674698fe5ddff2b2393abcebeef8a9ab776dcd00b5becc26958d8ae3e309669f" exitCode=0 Dec 03 12:19:52 crc kubenswrapper[5052]: I1203 12:19:52.444023 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"d67e60e3-f995-4ff3-82ca-9c7399852935","Type":"ContainerDied","Data":"674698fe5ddff2b2393abcebeef8a9ab776dcd00b5becc26958d8ae3e309669f"} Dec 03 12:19:52 crc kubenswrapper[5052]: I1203 12:19:52.444046 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"d67e60e3-f995-4ff3-82ca-9c7399852935","Type":"ContainerDied","Data":"81431d536c30f923487d8ab9e8efec22c6be48e6dc51bf50cdd7c81fbf8561c4"} Dec 03 12:19:52 crc kubenswrapper[5052]: I1203 12:19:52.444077 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:19:52 crc kubenswrapper[5052]: I1203 12:19:52.474654 5052 scope.go:117] "RemoveContainer" containerID="92b44cc750b48900a9f7dabd65ea43267f4dc1e842bf4ba16982de87b54589a5" Dec 03 12:19:52 crc kubenswrapper[5052]: I1203 12:19:52.499909 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:19:52 crc kubenswrapper[5052]: I1203 12:19:52.505390 5052 scope.go:117] "RemoveContainer" containerID="674698fe5ddff2b2393abcebeef8a9ab776dcd00b5becc26958d8ae3e309669f" Dec 03 12:19:52 crc kubenswrapper[5052]: I1203 12:19:52.527411 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:19:52 crc kubenswrapper[5052]: I1203 12:19:52.538962 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/nova-metadata-0" podUID="e159b4fe-4531-4a5e-8a7c-e759dab05223" containerName="nova-metadata-metadata" probeResult="failure" output="Get \"https://10.217.1.135:8775/\": read tcp 10.217.0.2:52912->10.217.1.135:8775: read: connection reset by peer" Dec 03 12:19:52 crc kubenswrapper[5052]: I1203 12:19:52.539321 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/nova-metadata-0" podUID="e159b4fe-4531-4a5e-8a7c-e759dab05223" containerName="nova-metadata-log" probeResult="failure" output="Get \"https://10.217.1.135:8775/\": read tcp 10.217.0.2:52926->10.217.1.135:8775: read: connection reset by peer" Dec 03 12:19:52 crc kubenswrapper[5052]: I1203 12:19:52.541422 5052 scope.go:117] "RemoveContainer" containerID="674698fe5ddff2b2393abcebeef8a9ab776dcd00b5becc26958d8ae3e309669f" Dec 03 12:19:52 crc kubenswrapper[5052]: E1203 12:19:52.543080 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"674698fe5ddff2b2393abcebeef8a9ab776dcd00b5becc26958d8ae3e309669f\": container with ID starting with 674698fe5ddff2b2393abcebeef8a9ab776dcd00b5becc26958d8ae3e309669f not found: ID does not exist" containerID="674698fe5ddff2b2393abcebeef8a9ab776dcd00b5becc26958d8ae3e309669f" Dec 03 12:19:52 crc kubenswrapper[5052]: I1203 12:19:52.543135 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"674698fe5ddff2b2393abcebeef8a9ab776dcd00b5becc26958d8ae3e309669f"} err="failed to get container status \"674698fe5ddff2b2393abcebeef8a9ab776dcd00b5becc26958d8ae3e309669f\": rpc error: code = NotFound desc = could not find container \"674698fe5ddff2b2393abcebeef8a9ab776dcd00b5becc26958d8ae3e309669f\": container with ID starting with 674698fe5ddff2b2393abcebeef8a9ab776dcd00b5becc26958d8ae3e309669f not found: ID does not exist" Dec 03 12:19:52 crc kubenswrapper[5052]: I1203 12:19:52.546435 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:19:52 crc kubenswrapper[5052]: I1203 12:19:52.570392 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:19:52 crc kubenswrapper[5052]: I1203 12:19:52.595005 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:19:52 crc kubenswrapper[5052]: E1203 12:19:52.595529 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="d67e60e3-f995-4ff3-82ca-9c7399852935" containerName="nova-scheduler-scheduler" Dec 03 12:19:52 crc kubenswrapper[5052]: I1203 12:19:52.595548 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="d67e60e3-f995-4ff3-82ca-9c7399852935" containerName="nova-scheduler-scheduler" Dec 03 12:19:52 crc kubenswrapper[5052]: E1203 12:19:52.595589 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="896a13e1-d614-4be4-9a4a-03b438952ac9" containerName="nova-manage" Dec 03 12:19:52 crc kubenswrapper[5052]: I1203 12:19:52.595600 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="896a13e1-d614-4be4-9a4a-03b438952ac9" containerName="nova-manage" Dec 03 12:19:52 crc kubenswrapper[5052]: E1203 12:19:52.595618 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="d77f169d-276f-4432-9f68-053d75e3c33b" containerName="nova-api-api" Dec 03 12:19:52 crc kubenswrapper[5052]: I1203 12:19:52.595626 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="d77f169d-276f-4432-9f68-053d75e3c33b" containerName="nova-api-api" Dec 03 12:19:52 crc kubenswrapper[5052]: E1203 12:19:52.595643 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="d77f169d-276f-4432-9f68-053d75e3c33b" containerName="nova-api-log" Dec 03 12:19:52 crc kubenswrapper[5052]: I1203 12:19:52.595652 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="d77f169d-276f-4432-9f68-053d75e3c33b" containerName="nova-api-log" Dec 03 12:19:52 crc kubenswrapper[5052]: I1203 12:19:52.595981 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="d77f169d-276f-4432-9f68-053d75e3c33b" containerName="nova-api-log" Dec 03 12:19:52 crc kubenswrapper[5052]: I1203 12:19:52.596013 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="d77f169d-276f-4432-9f68-053d75e3c33b" containerName="nova-api-api" Dec 03 12:19:52 crc kubenswrapper[5052]: I1203 12:19:52.596032 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="d67e60e3-f995-4ff3-82ca-9c7399852935" containerName="nova-scheduler-scheduler" Dec 03 12:19:52 crc kubenswrapper[5052]: I1203 12:19:52.596053 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="896a13e1-d614-4be4-9a4a-03b438952ac9" containerName="nova-manage" Dec 03 12:19:52 crc kubenswrapper[5052]: I1203 12:19:52.598186 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:19:52 crc kubenswrapper[5052]: I1203 12:19:52.602329 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-scheduler-config-data" Dec 03 12:19:52 crc kubenswrapper[5052]: I1203 12:19:52.624225 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:19:52 crc kubenswrapper[5052]: I1203 12:19:52.634068 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:19:52 crc kubenswrapper[5052]: I1203 12:19:52.636088 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:19:52 crc kubenswrapper[5052]: I1203 12:19:52.639908 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-api-config-data" Dec 03 12:19:52 crc kubenswrapper[5052]: I1203 12:19:52.645640 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:19:52 crc kubenswrapper[5052]: I1203 12:19:52.648096 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bq5cv\" (UniqueName: \"kubernetes.io/projected/9d3ab728-8f1c-48ac-808c-dfbf80b8f792-kube-api-access-bq5cv\") pod \"nova-scheduler-0\" (UID: \"9d3ab728-8f1c-48ac-808c-dfbf80b8f792\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:19:52 crc kubenswrapper[5052]: I1203 12:19:52.648226 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/9d3ab728-8f1c-48ac-808c-dfbf80b8f792-config-data\") pod \"nova-scheduler-0\" (UID: \"9d3ab728-8f1c-48ac-808c-dfbf80b8f792\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:19:52 crc kubenswrapper[5052]: I1203 12:19:52.648316 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9d3ab728-8f1c-48ac-808c-dfbf80b8f792-combined-ca-bundle\") pod \"nova-scheduler-0\" (UID: \"9d3ab728-8f1c-48ac-808c-dfbf80b8f792\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:19:52 crc kubenswrapper[5052]: I1203 12:19:52.750088 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/9d3ab728-8f1c-48ac-808c-dfbf80b8f792-config-data\") pod \"nova-scheduler-0\" (UID: \"9d3ab728-8f1c-48ac-808c-dfbf80b8f792\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:19:52 crc kubenswrapper[5052]: I1203 12:19:52.750168 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/1fa646b0-2860-4eb4-9465-277b573e8833-logs\") pod \"nova-api-0\" (UID: \"1fa646b0-2860-4eb4-9465-277b573e8833\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:19:52 crc kubenswrapper[5052]: I1203 12:19:52.750647 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cn8p6\" (UniqueName: \"kubernetes.io/projected/1fa646b0-2860-4eb4-9465-277b573e8833-kube-api-access-cn8p6\") pod \"nova-api-0\" (UID: \"1fa646b0-2860-4eb4-9465-277b573e8833\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:19:52 crc kubenswrapper[5052]: I1203 12:19:52.750673 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9d3ab728-8f1c-48ac-808c-dfbf80b8f792-combined-ca-bundle\") pod \"nova-scheduler-0\" (UID: \"9d3ab728-8f1c-48ac-808c-dfbf80b8f792\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:19:52 crc kubenswrapper[5052]: I1203 12:19:52.750742 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-bq5cv\" (UniqueName: \"kubernetes.io/projected/9d3ab728-8f1c-48ac-808c-dfbf80b8f792-kube-api-access-bq5cv\") pod \"nova-scheduler-0\" (UID: \"9d3ab728-8f1c-48ac-808c-dfbf80b8f792\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:19:52 crc kubenswrapper[5052]: I1203 12:19:52.750763 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1fa646b0-2860-4eb4-9465-277b573e8833-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"1fa646b0-2860-4eb4-9465-277b573e8833\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:19:52 crc kubenswrapper[5052]: I1203 12:19:52.750784 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/1fa646b0-2860-4eb4-9465-277b573e8833-config-data\") pod \"nova-api-0\" (UID: \"1fa646b0-2860-4eb4-9465-277b573e8833\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:19:52 crc kubenswrapper[5052]: I1203 12:19:52.775195 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/9d3ab728-8f1c-48ac-808c-dfbf80b8f792-config-data\") pod \"nova-scheduler-0\" (UID: \"9d3ab728-8f1c-48ac-808c-dfbf80b8f792\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:19:52 crc kubenswrapper[5052]: I1203 12:19:52.775262 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9d3ab728-8f1c-48ac-808c-dfbf80b8f792-combined-ca-bundle\") pod \"nova-scheduler-0\" (UID: \"9d3ab728-8f1c-48ac-808c-dfbf80b8f792\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:19:52 crc kubenswrapper[5052]: I1203 12:19:52.783094 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-bq5cv\" (UniqueName: \"kubernetes.io/projected/9d3ab728-8f1c-48ac-808c-dfbf80b8f792-kube-api-access-bq5cv\") pod \"nova-scheduler-0\" (UID: \"9d3ab728-8f1c-48ac-808c-dfbf80b8f792\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:19:52 crc kubenswrapper[5052]: I1203 12:19:52.867103 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-cn8p6\" (UniqueName: \"kubernetes.io/projected/1fa646b0-2860-4eb4-9465-277b573e8833-kube-api-access-cn8p6\") pod \"nova-api-0\" (UID: \"1fa646b0-2860-4eb4-9465-277b573e8833\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:19:52 crc kubenswrapper[5052]: I1203 12:19:52.867290 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1fa646b0-2860-4eb4-9465-277b573e8833-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"1fa646b0-2860-4eb4-9465-277b573e8833\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:19:52 crc kubenswrapper[5052]: I1203 12:19:52.867334 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/1fa646b0-2860-4eb4-9465-277b573e8833-config-data\") pod \"nova-api-0\" (UID: \"1fa646b0-2860-4eb4-9465-277b573e8833\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:19:52 crc kubenswrapper[5052]: I1203 12:19:52.867463 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/1fa646b0-2860-4eb4-9465-277b573e8833-logs\") pod \"nova-api-0\" (UID: \"1fa646b0-2860-4eb4-9465-277b573e8833\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:19:52 crc kubenswrapper[5052]: I1203 12:19:52.867992 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/1fa646b0-2860-4eb4-9465-277b573e8833-logs\") pod \"nova-api-0\" (UID: \"1fa646b0-2860-4eb4-9465-277b573e8833\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:19:52 crc kubenswrapper[5052]: I1203 12:19:52.882209 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/1fa646b0-2860-4eb4-9465-277b573e8833-config-data\") pod \"nova-api-0\" (UID: \"1fa646b0-2860-4eb4-9465-277b573e8833\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:19:52 crc kubenswrapper[5052]: I1203 12:19:52.886264 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1fa646b0-2860-4eb4-9465-277b573e8833-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"1fa646b0-2860-4eb4-9465-277b573e8833\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:19:52 crc kubenswrapper[5052]: I1203 12:19:52.891058 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-cn8p6\" (UniqueName: \"kubernetes.io/projected/1fa646b0-2860-4eb4-9465-277b573e8833-kube-api-access-cn8p6\") pod \"nova-api-0\" (UID: \"1fa646b0-2860-4eb4-9465-277b573e8833\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:19:52 crc kubenswrapper[5052]: I1203 12:19:52.900709 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:19:52 crc kubenswrapper[5052]: I1203 12:19:52.970128 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/e159b4fe-4531-4a5e-8a7c-e759dab05223-logs\") pod \"e159b4fe-4531-4a5e-8a7c-e759dab05223\" (UID: \"e159b4fe-4531-4a5e-8a7c-e759dab05223\") " Dec 03 12:19:52 crc kubenswrapper[5052]: I1203 12:19:52.970291 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/e159b4fe-4531-4a5e-8a7c-e759dab05223-config-data\") pod \"e159b4fe-4531-4a5e-8a7c-e759dab05223\" (UID: \"e159b4fe-4531-4a5e-8a7c-e759dab05223\") " Dec 03 12:19:52 crc kubenswrapper[5052]: I1203 12:19:52.970326 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cl2bm\" (UniqueName: \"kubernetes.io/projected/e159b4fe-4531-4a5e-8a7c-e759dab05223-kube-api-access-cl2bm\") pod \"e159b4fe-4531-4a5e-8a7c-e759dab05223\" (UID: \"e159b4fe-4531-4a5e-8a7c-e759dab05223\") " Dec 03 12:19:52 crc kubenswrapper[5052]: I1203 12:19:52.970355 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e159b4fe-4531-4a5e-8a7c-e759dab05223-combined-ca-bundle\") pod \"e159b4fe-4531-4a5e-8a7c-e759dab05223\" (UID: \"e159b4fe-4531-4a5e-8a7c-e759dab05223\") " Dec 03 12:19:52 crc kubenswrapper[5052]: I1203 12:19:52.970419 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/e159b4fe-4531-4a5e-8a7c-e759dab05223-nova-metadata-tls-certs\") pod \"e159b4fe-4531-4a5e-8a7c-e759dab05223\" (UID: \"e159b4fe-4531-4a5e-8a7c-e759dab05223\") " Dec 03 12:19:52 crc kubenswrapper[5052]: I1203 12:19:52.971110 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e159b4fe-4531-4a5e-8a7c-e759dab05223-logs" (OuterVolumeSpecName: "logs") pod "e159b4fe-4531-4a5e-8a7c-e759dab05223" (UID: "e159b4fe-4531-4a5e-8a7c-e759dab05223"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:19:52 crc kubenswrapper[5052]: I1203 12:19:52.974996 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e159b4fe-4531-4a5e-8a7c-e759dab05223-kube-api-access-cl2bm" (OuterVolumeSpecName: "kube-api-access-cl2bm") pod "e159b4fe-4531-4a5e-8a7c-e759dab05223" (UID: "e159b4fe-4531-4a5e-8a7c-e759dab05223"). InnerVolumeSpecName "kube-api-access-cl2bm". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:19:52 crc kubenswrapper[5052]: I1203 12:19:52.986280 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:19:52 crc kubenswrapper[5052]: I1203 12:19:52.999622 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:19:53 crc kubenswrapper[5052]: I1203 12:19:53.007873 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e159b4fe-4531-4a5e-8a7c-e759dab05223-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "e159b4fe-4531-4a5e-8a7c-e759dab05223" (UID: "e159b4fe-4531-4a5e-8a7c-e759dab05223"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:19:53 crc kubenswrapper[5052]: I1203 12:19:53.023752 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e159b4fe-4531-4a5e-8a7c-e759dab05223-config-data" (OuterVolumeSpecName: "config-data") pod "e159b4fe-4531-4a5e-8a7c-e759dab05223" (UID: "e159b4fe-4531-4a5e-8a7c-e759dab05223"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:19:53 crc kubenswrapper[5052]: I1203 12:19:53.054485 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e159b4fe-4531-4a5e-8a7c-e759dab05223-nova-metadata-tls-certs" (OuterVolumeSpecName: "nova-metadata-tls-certs") pod "e159b4fe-4531-4a5e-8a7c-e759dab05223" (UID: "e159b4fe-4531-4a5e-8a7c-e759dab05223"). InnerVolumeSpecName "nova-metadata-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:19:53 crc kubenswrapper[5052]: I1203 12:19:53.072995 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/e159b4fe-4531-4a5e-8a7c-e759dab05223-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:19:53 crc kubenswrapper[5052]: I1203 12:19:53.073032 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-cl2bm\" (UniqueName: \"kubernetes.io/projected/e159b4fe-4531-4a5e-8a7c-e759dab05223-kube-api-access-cl2bm\") on node \"crc\" DevicePath \"\"" Dec 03 12:19:53 crc kubenswrapper[5052]: I1203 12:19:53.073044 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e159b4fe-4531-4a5e-8a7c-e759dab05223-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:19:53 crc kubenswrapper[5052]: I1203 12:19:53.073052 5052 reconciler_common.go:293] "Volume detached for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/e159b4fe-4531-4a5e-8a7c-e759dab05223-nova-metadata-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:19:53 crc kubenswrapper[5052]: I1203 12:19:53.073061 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/e159b4fe-4531-4a5e-8a7c-e759dab05223-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:19:53 crc kubenswrapper[5052]: I1203 12:19:53.455557 5052 generic.go:334] "Generic (PLEG): container finished" podID="e159b4fe-4531-4a5e-8a7c-e759dab05223" containerID="9de7a2c8a8faa15e1d495b079f4635c3cc16c762477ca929b4d0221cf84d7066" exitCode=0 Dec 03 12:19:53 crc kubenswrapper[5052]: I1203 12:19:53.455599 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:19:53 crc kubenswrapper[5052]: I1203 12:19:53.455619 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"e159b4fe-4531-4a5e-8a7c-e759dab05223","Type":"ContainerDied","Data":"9de7a2c8a8faa15e1d495b079f4635c3cc16c762477ca929b4d0221cf84d7066"} Dec 03 12:19:53 crc kubenswrapper[5052]: I1203 12:19:53.457241 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"e159b4fe-4531-4a5e-8a7c-e759dab05223","Type":"ContainerDied","Data":"3eb8343e1e26e1dfb24da9b063349cd8b52440c17a32313eee01848827ddd286"} Dec 03 12:19:53 crc kubenswrapper[5052]: I1203 12:19:53.457270 5052 scope.go:117] "RemoveContainer" containerID="9de7a2c8a8faa15e1d495b079f4635c3cc16c762477ca929b4d0221cf84d7066" Dec 03 12:19:53 crc kubenswrapper[5052]: I1203 12:19:53.502568 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:19:53 crc kubenswrapper[5052]: I1203 12:19:53.515234 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:19:53 crc kubenswrapper[5052]: I1203 12:19:53.527378 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:19:53 crc kubenswrapper[5052]: I1203 12:19:53.529488 5052 scope.go:117] "RemoveContainer" containerID="710ef1f3716939feefec91cafc54638d3bd676f9ff55da8928f8ad595f171b67" Dec 03 12:19:53 crc kubenswrapper[5052]: I1203 12:19:53.585549 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:19:53 crc kubenswrapper[5052]: E1203 12:19:53.586019 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="e159b4fe-4531-4a5e-8a7c-e759dab05223" containerName="nova-metadata-metadata" Dec 03 12:19:53 crc kubenswrapper[5052]: I1203 12:19:53.586043 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="e159b4fe-4531-4a5e-8a7c-e759dab05223" containerName="nova-metadata-metadata" Dec 03 12:19:53 crc kubenswrapper[5052]: E1203 12:19:53.586065 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="e159b4fe-4531-4a5e-8a7c-e759dab05223" containerName="nova-metadata-log" Dec 03 12:19:53 crc kubenswrapper[5052]: I1203 12:19:53.586073 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="e159b4fe-4531-4a5e-8a7c-e759dab05223" containerName="nova-metadata-log" Dec 03 12:19:53 crc kubenswrapper[5052]: I1203 12:19:53.586298 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="e159b4fe-4531-4a5e-8a7c-e759dab05223" containerName="nova-metadata-log" Dec 03 12:19:53 crc kubenswrapper[5052]: I1203 12:19:53.586332 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="e159b4fe-4531-4a5e-8a7c-e759dab05223" containerName="nova-metadata-metadata" Dec 03 12:19:53 crc kubenswrapper[5052]: I1203 12:19:53.587459 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:19:53 crc kubenswrapper[5052]: I1203 12:19:53.589746 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-nova-metadata-internal-svc" Dec 03 12:19:53 crc kubenswrapper[5052]: I1203 12:19:53.590580 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-metadata-config-data" Dec 03 12:19:53 crc kubenswrapper[5052]: I1203 12:19:53.624825 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:19:53 crc kubenswrapper[5052]: I1203 12:19:53.640913 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:19:53 crc kubenswrapper[5052]: I1203 12:19:53.688378 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d67e60e3-f995-4ff3-82ca-9c7399852935" path="/var/lib/kubelet/pods/d67e60e3-f995-4ff3-82ca-9c7399852935/volumes" Dec 03 12:19:53 crc kubenswrapper[5052]: I1203 12:19:53.689275 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d77f169d-276f-4432-9f68-053d75e3c33b" path="/var/lib/kubelet/pods/d77f169d-276f-4432-9f68-053d75e3c33b/volumes" Dec 03 12:19:53 crc kubenswrapper[5052]: I1203 12:19:53.690144 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e159b4fe-4531-4a5e-8a7c-e759dab05223" path="/var/lib/kubelet/pods/e159b4fe-4531-4a5e-8a7c-e759dab05223/volumes" Dec 03 12:19:53 crc kubenswrapper[5052]: I1203 12:19:53.694491 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/5bda226d-9d40-42d8-ac3e-11719e896649-nova-metadata-tls-certs\") pod \"nova-metadata-0\" (UID: \"5bda226d-9d40-42d8-ac3e-11719e896649\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:19:53 crc kubenswrapper[5052]: I1203 12:19:53.694639 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/5bda226d-9d40-42d8-ac3e-11719e896649-config-data\") pod \"nova-metadata-0\" (UID: \"5bda226d-9d40-42d8-ac3e-11719e896649\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:19:53 crc kubenswrapper[5052]: I1203 12:19:53.694890 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zvm9j\" (UniqueName: \"kubernetes.io/projected/5bda226d-9d40-42d8-ac3e-11719e896649-kube-api-access-zvm9j\") pod \"nova-metadata-0\" (UID: \"5bda226d-9d40-42d8-ac3e-11719e896649\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:19:53 crc kubenswrapper[5052]: I1203 12:19:53.695020 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/5bda226d-9d40-42d8-ac3e-11719e896649-logs\") pod \"nova-metadata-0\" (UID: \"5bda226d-9d40-42d8-ac3e-11719e896649\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:19:53 crc kubenswrapper[5052]: I1203 12:19:53.695150 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5bda226d-9d40-42d8-ac3e-11719e896649-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"5bda226d-9d40-42d8-ac3e-11719e896649\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:19:53 crc kubenswrapper[5052]: I1203 12:19:53.702726 5052 scope.go:117] "RemoveContainer" containerID="9de7a2c8a8faa15e1d495b079f4635c3cc16c762477ca929b4d0221cf84d7066" Dec 03 12:19:53 crc kubenswrapper[5052]: E1203 12:19:53.703211 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9de7a2c8a8faa15e1d495b079f4635c3cc16c762477ca929b4d0221cf84d7066\": container with ID starting with 9de7a2c8a8faa15e1d495b079f4635c3cc16c762477ca929b4d0221cf84d7066 not found: ID does not exist" containerID="9de7a2c8a8faa15e1d495b079f4635c3cc16c762477ca929b4d0221cf84d7066" Dec 03 12:19:53 crc kubenswrapper[5052]: I1203 12:19:53.703244 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9de7a2c8a8faa15e1d495b079f4635c3cc16c762477ca929b4d0221cf84d7066"} err="failed to get container status \"9de7a2c8a8faa15e1d495b079f4635c3cc16c762477ca929b4d0221cf84d7066\": rpc error: code = NotFound desc = could not find container \"9de7a2c8a8faa15e1d495b079f4635c3cc16c762477ca929b4d0221cf84d7066\": container with ID starting with 9de7a2c8a8faa15e1d495b079f4635c3cc16c762477ca929b4d0221cf84d7066 not found: ID does not exist" Dec 03 12:19:53 crc kubenswrapper[5052]: I1203 12:19:53.703270 5052 scope.go:117] "RemoveContainer" containerID="710ef1f3716939feefec91cafc54638d3bd676f9ff55da8928f8ad595f171b67" Dec 03 12:19:53 crc kubenswrapper[5052]: E1203 12:19:53.703531 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"710ef1f3716939feefec91cafc54638d3bd676f9ff55da8928f8ad595f171b67\": container with ID starting with 710ef1f3716939feefec91cafc54638d3bd676f9ff55da8928f8ad595f171b67 not found: ID does not exist" containerID="710ef1f3716939feefec91cafc54638d3bd676f9ff55da8928f8ad595f171b67" Dec 03 12:19:53 crc kubenswrapper[5052]: I1203 12:19:53.703565 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"710ef1f3716939feefec91cafc54638d3bd676f9ff55da8928f8ad595f171b67"} err="failed to get container status \"710ef1f3716939feefec91cafc54638d3bd676f9ff55da8928f8ad595f171b67\": rpc error: code = NotFound desc = could not find container \"710ef1f3716939feefec91cafc54638d3bd676f9ff55da8928f8ad595f171b67\": container with ID starting with 710ef1f3716939feefec91cafc54638d3bd676f9ff55da8928f8ad595f171b67 not found: ID does not exist" Dec 03 12:19:53 crc kubenswrapper[5052]: I1203 12:19:53.797213 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/5bda226d-9d40-42d8-ac3e-11719e896649-config-data\") pod \"nova-metadata-0\" (UID: \"5bda226d-9d40-42d8-ac3e-11719e896649\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:19:53 crc kubenswrapper[5052]: I1203 12:19:53.797757 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-zvm9j\" (UniqueName: \"kubernetes.io/projected/5bda226d-9d40-42d8-ac3e-11719e896649-kube-api-access-zvm9j\") pod \"nova-metadata-0\" (UID: \"5bda226d-9d40-42d8-ac3e-11719e896649\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:19:53 crc kubenswrapper[5052]: I1203 12:19:53.797793 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/5bda226d-9d40-42d8-ac3e-11719e896649-logs\") pod \"nova-metadata-0\" (UID: \"5bda226d-9d40-42d8-ac3e-11719e896649\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:19:53 crc kubenswrapper[5052]: I1203 12:19:53.797825 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5bda226d-9d40-42d8-ac3e-11719e896649-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"5bda226d-9d40-42d8-ac3e-11719e896649\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:19:53 crc kubenswrapper[5052]: I1203 12:19:53.797891 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/5bda226d-9d40-42d8-ac3e-11719e896649-nova-metadata-tls-certs\") pod \"nova-metadata-0\" (UID: \"5bda226d-9d40-42d8-ac3e-11719e896649\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:19:53 crc kubenswrapper[5052]: I1203 12:19:53.799095 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/5bda226d-9d40-42d8-ac3e-11719e896649-logs\") pod \"nova-metadata-0\" (UID: \"5bda226d-9d40-42d8-ac3e-11719e896649\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:19:53 crc kubenswrapper[5052]: I1203 12:19:53.807075 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/5bda226d-9d40-42d8-ac3e-11719e896649-nova-metadata-tls-certs\") pod \"nova-metadata-0\" (UID: \"5bda226d-9d40-42d8-ac3e-11719e896649\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:19:53 crc kubenswrapper[5052]: I1203 12:19:53.807096 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/5bda226d-9d40-42d8-ac3e-11719e896649-config-data\") pod \"nova-metadata-0\" (UID: \"5bda226d-9d40-42d8-ac3e-11719e896649\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:19:53 crc kubenswrapper[5052]: I1203 12:19:53.809215 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5bda226d-9d40-42d8-ac3e-11719e896649-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"5bda226d-9d40-42d8-ac3e-11719e896649\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:19:53 crc kubenswrapper[5052]: I1203 12:19:53.818925 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-zvm9j\" (UniqueName: \"kubernetes.io/projected/5bda226d-9d40-42d8-ac3e-11719e896649-kube-api-access-zvm9j\") pod \"nova-metadata-0\" (UID: \"5bda226d-9d40-42d8-ac3e-11719e896649\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:19:54 crc kubenswrapper[5052]: I1203 12:19:54.056960 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:19:54 crc kubenswrapper[5052]: I1203 12:19:54.473630 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"9d3ab728-8f1c-48ac-808c-dfbf80b8f792","Type":"ContainerStarted","Data":"20a9921d6ffcb85977689eb24ca64a251931e0e59f572a888d9ee16492bf0b00"} Dec 03 12:19:54 crc kubenswrapper[5052]: I1203 12:19:54.473965 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"9d3ab728-8f1c-48ac-808c-dfbf80b8f792","Type":"ContainerStarted","Data":"d111d7c254f77ac4200c9f7d6366b103bdf3c9690d8c2ffcdaa56adf1c7365b7"} Dec 03 12:19:54 crc kubenswrapper[5052]: I1203 12:19:54.477428 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"1fa646b0-2860-4eb4-9465-277b573e8833","Type":"ContainerStarted","Data":"f78f5c744f6b85db55f83c169645f4e175fea630e53c4631ba813ec516db849b"} Dec 03 12:19:54 crc kubenswrapper[5052]: I1203 12:19:54.477476 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"1fa646b0-2860-4eb4-9465-277b573e8833","Type":"ContainerStarted","Data":"3c9b1c00020081fb6aaa180232f9cccfc11821fc69ae178b2e202e185ce313c4"} Dec 03 12:19:54 crc kubenswrapper[5052]: I1203 12:19:54.477489 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"1fa646b0-2860-4eb4-9465-277b573e8833","Type":"ContainerStarted","Data":"2aab4e3407e68b8be7dd0db246d395be26f96b302471d6ae99bd370ad54c036b"} Dec 03 12:19:54 crc kubenswrapper[5052]: I1203 12:19:54.493484 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-scheduler-0" podStartSLOduration=2.493457792 podStartE2EDuration="2.493457792s" podCreationTimestamp="2025-12-03 12:19:52 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:19:54.491754723 +0000 UTC m=+3585.079996507" watchObservedRunningTime="2025-12-03 12:19:54.493457792 +0000 UTC m=+3585.081699556" Dec 03 12:19:54 crc kubenswrapper[5052]: I1203 12:19:54.516908 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-api-0" podStartSLOduration=2.516889601 podStartE2EDuration="2.516889601s" podCreationTimestamp="2025-12-03 12:19:52 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:19:54.513847014 +0000 UTC m=+3585.102088778" watchObservedRunningTime="2025-12-03 12:19:54.516889601 +0000 UTC m=+3585.105131365" Dec 03 12:19:54 crc kubenswrapper[5052]: I1203 12:19:54.553754 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:19:55 crc kubenswrapper[5052]: I1203 12:19:55.488881 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"5bda226d-9d40-42d8-ac3e-11719e896649","Type":"ContainerStarted","Data":"67d4dbc28757901384966bfa33590ec06891feae45053aaf9e8e283f4b17ad15"} Dec 03 12:19:55 crc kubenswrapper[5052]: I1203 12:19:55.489552 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"5bda226d-9d40-42d8-ac3e-11719e896649","Type":"ContainerStarted","Data":"9818701fe2f0f4ff6dc2b33cbd5c557f191bfe07f31eb908a06a4fcecfe3dcd0"} Dec 03 12:19:55 crc kubenswrapper[5052]: I1203 12:19:55.489574 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"5bda226d-9d40-42d8-ac3e-11719e896649","Type":"ContainerStarted","Data":"457c34f728bf076a5e41aa3b8eff71a377813b1768597cae052035c793de82a9"} Dec 03 12:19:55 crc kubenswrapper[5052]: I1203 12:19:55.524291 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-metadata-0" podStartSLOduration=2.524261723 podStartE2EDuration="2.524261723s" podCreationTimestamp="2025-12-03 12:19:53 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:19:55.514612588 +0000 UTC m=+3586.102854352" watchObservedRunningTime="2025-12-03 12:19:55.524261723 +0000 UTC m=+3586.112503487" Dec 03 12:19:57 crc kubenswrapper[5052]: I1203 12:19:57.988044 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:19:59 crc kubenswrapper[5052]: I1203 12:19:59.057500 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:19:59 crc kubenswrapper[5052]: I1203 12:19:59.057571 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:19:59 crc kubenswrapper[5052]: I1203 12:19:59.379831 5052 patch_prober.go:28] interesting pod/machine-config-daemon-74bj6 container/machine-config-daemon namespace/openshift-machine-config-operator: Liveness probe status=failure output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" start-of-body= Dec 03 12:19:59 crc kubenswrapper[5052]: I1203 12:19:59.380200 5052 prober.go:107] "Probe failed" probeType="Liveness" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" containerName="machine-config-daemon" probeResult="failure" output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" Dec 03 12:20:02 crc kubenswrapper[5052]: I1203 12:20:02.987854 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:20:03 crc kubenswrapper[5052]: I1203 12:20:03.000867 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:20:03 crc kubenswrapper[5052]: I1203 12:20:03.000914 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:20:03 crc kubenswrapper[5052]: I1203 12:20:03.024159 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:20:03 crc kubenswrapper[5052]: I1203 12:20:03.592212 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:20:03 crc kubenswrapper[5052]: I1203 12:20:03.679768 5052 scope.go:117] "RemoveContainer" containerID="624a18da80d875d17af9c7c2a3b638755eddc32b1144b57ac4834a2414178324" Dec 03 12:20:03 crc kubenswrapper[5052]: I1203 12:20:03.733465 5052 scope.go:117] "RemoveContainer" containerID="e2af5ecd95d6d829c6c19029b5a86f5c757439ed343b6c5d710eff37b7fddd69" Dec 03 12:20:03 crc kubenswrapper[5052]: I1203 12:20:03.791006 5052 scope.go:117] "RemoveContainer" containerID="28049a9b7828f990e7cdf628c7744bd85e07ead3fc6256992ef7d5bf23e045a3" Dec 03 12:20:03 crc kubenswrapper[5052]: I1203 12:20:03.830535 5052 scope.go:117] "RemoveContainer" containerID="63ba8a0e2bdcd79bccfbfa1aced2e45ca9e753ae90253a074759197725efe5f1" Dec 03 12:20:04 crc kubenswrapper[5052]: I1203 12:20:04.057561 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:20:04 crc kubenswrapper[5052]: I1203 12:20:04.057604 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:20:04 crc kubenswrapper[5052]: I1203 12:20:04.087732 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-api-0" podUID="1fa646b0-2860-4eb4-9465-277b573e8833" containerName="nova-api-log" probeResult="failure" output="Get \"http://10.217.1.143:8774/\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Dec 03 12:20:04 crc kubenswrapper[5052]: I1203 12:20:04.087735 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-api-0" podUID="1fa646b0-2860-4eb4-9465-277b573e8833" containerName="nova-api-api" probeResult="failure" output="Get \"http://10.217.1.143:8774/\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Dec 03 12:20:05 crc kubenswrapper[5052]: I1203 12:20:05.070684 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-metadata-0" podUID="5bda226d-9d40-42d8-ac3e-11719e896649" containerName="nova-metadata-log" probeResult="failure" output="Get \"https://10.217.1.144:8775/\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Dec 03 12:20:05 crc kubenswrapper[5052]: I1203 12:20:05.070695 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-metadata-0" podUID="5bda226d-9d40-42d8-ac3e-11719e896649" containerName="nova-metadata-metadata" probeResult="failure" output="Get \"https://10.217.1.144:8775/\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Dec 03 12:20:13 crc kubenswrapper[5052]: I1203 12:20:13.003999 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:20:13 crc kubenswrapper[5052]: I1203 12:20:13.005493 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:20:13 crc kubenswrapper[5052]: I1203 12:20:13.006484 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:20:13 crc kubenswrapper[5052]: I1203 12:20:13.013202 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:20:13 crc kubenswrapper[5052]: I1203 12:20:13.684519 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:20:13 crc kubenswrapper[5052]: I1203 12:20:13.810447 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:20:14 crc kubenswrapper[5052]: I1203 12:20:14.065977 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:20:14 crc kubenswrapper[5052]: I1203 12:20:14.067148 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:20:14 crc kubenswrapper[5052]: I1203 12:20:14.077458 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:20:14 crc kubenswrapper[5052]: I1203 12:20:14.697660 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:20:15 crc kubenswrapper[5052]: I1203 12:20:15.714465 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:20:15 crc kubenswrapper[5052]: I1203 12:20:15.982303 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:20:16 crc kubenswrapper[5052]: I1203 12:20:16.242690 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:20:16 crc kubenswrapper[5052]: I1203 12:20:16.703604 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-api-0" podUID="1fa646b0-2860-4eb4-9465-277b573e8833" containerName="nova-api-log" containerID="cri-o://3c9b1c00020081fb6aaa180232f9cccfc11821fc69ae178b2e202e185ce313c4" gracePeriod=30 Dec 03 12:20:16 crc kubenswrapper[5052]: I1203 12:20:16.703684 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-api-0" podUID="1fa646b0-2860-4eb4-9465-277b573e8833" containerName="nova-api-api" containerID="cri-o://f78f5c744f6b85db55f83c169645f4e175fea630e53c4631ba813ec516db849b" gracePeriod=30 Dec 03 12:20:16 crc kubenswrapper[5052]: I1203 12:20:16.703847 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="9e9cf358-eca4-4774-b1a1-1667774e900f" containerName="ceilometer-central-agent" containerID="cri-o://b1d5d717d0fcd013d58f887c25d6c34fe306a5816962f6a70d25cd36f488c6a3" gracePeriod=30 Dec 03 12:20:16 crc kubenswrapper[5052]: I1203 12:20:16.703879 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="9e9cf358-eca4-4774-b1a1-1667774e900f" containerName="ceilometer-notification-agent" containerID="cri-o://648a05aa3b83b4428ca8cb8790ec8c4c9aa4881d0992e41989e59db80706d517" gracePeriod=30 Dec 03 12:20:16 crc kubenswrapper[5052]: I1203 12:20:16.703884 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="9e9cf358-eca4-4774-b1a1-1667774e900f" containerName="sg-core" containerID="cri-o://6ad4071deb854da06386e9a499db2d3dac5c11cb33dde601cceab73c2f94d29b" gracePeriod=30 Dec 03 12:20:16 crc kubenswrapper[5052]: I1203 12:20:16.703911 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="9e9cf358-eca4-4774-b1a1-1667774e900f" containerName="proxy-httpd" containerID="cri-o://88e82ab460c499ca6f48460cef37af029831a0965950387bc54369ec6d4ca508" gracePeriod=30 Dec 03 12:20:17 crc kubenswrapper[5052]: I1203 12:20:17.715839 5052 generic.go:334] "Generic (PLEG): container finished" podID="1fa646b0-2860-4eb4-9465-277b573e8833" containerID="3c9b1c00020081fb6aaa180232f9cccfc11821fc69ae178b2e202e185ce313c4" exitCode=143 Dec 03 12:20:17 crc kubenswrapper[5052]: I1203 12:20:17.715913 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"1fa646b0-2860-4eb4-9465-277b573e8833","Type":"ContainerDied","Data":"3c9b1c00020081fb6aaa180232f9cccfc11821fc69ae178b2e202e185ce313c4"} Dec 03 12:20:17 crc kubenswrapper[5052]: I1203 12:20:17.719491 5052 generic.go:334] "Generic (PLEG): container finished" podID="9e9cf358-eca4-4774-b1a1-1667774e900f" containerID="88e82ab460c499ca6f48460cef37af029831a0965950387bc54369ec6d4ca508" exitCode=0 Dec 03 12:20:17 crc kubenswrapper[5052]: I1203 12:20:17.719533 5052 generic.go:334] "Generic (PLEG): container finished" podID="9e9cf358-eca4-4774-b1a1-1667774e900f" containerID="6ad4071deb854da06386e9a499db2d3dac5c11cb33dde601cceab73c2f94d29b" exitCode=2 Dec 03 12:20:17 crc kubenswrapper[5052]: I1203 12:20:17.719542 5052 generic.go:334] "Generic (PLEG): container finished" podID="9e9cf358-eca4-4774-b1a1-1667774e900f" containerID="b1d5d717d0fcd013d58f887c25d6c34fe306a5816962f6a70d25cd36f488c6a3" exitCode=0 Dec 03 12:20:17 crc kubenswrapper[5052]: I1203 12:20:17.719556 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"9e9cf358-eca4-4774-b1a1-1667774e900f","Type":"ContainerDied","Data":"88e82ab460c499ca6f48460cef37af029831a0965950387bc54369ec6d4ca508"} Dec 03 12:20:17 crc kubenswrapper[5052]: I1203 12:20:17.719571 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"9e9cf358-eca4-4774-b1a1-1667774e900f","Type":"ContainerDied","Data":"6ad4071deb854da06386e9a499db2d3dac5c11cb33dde601cceab73c2f94d29b"} Dec 03 12:20:17 crc kubenswrapper[5052]: I1203 12:20:17.719587 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"9e9cf358-eca4-4774-b1a1-1667774e900f","Type":"ContainerDied","Data":"b1d5d717d0fcd013d58f887c25d6c34fe306a5816962f6a70d25cd36f488c6a3"} Dec 03 12:20:20 crc kubenswrapper[5052]: I1203 12:20:20.332730 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:20:20 crc kubenswrapper[5052]: I1203 12:20:20.411140 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cn8p6\" (UniqueName: \"kubernetes.io/projected/1fa646b0-2860-4eb4-9465-277b573e8833-kube-api-access-cn8p6\") pod \"1fa646b0-2860-4eb4-9465-277b573e8833\" (UID: \"1fa646b0-2860-4eb4-9465-277b573e8833\") " Dec 03 12:20:20 crc kubenswrapper[5052]: I1203 12:20:20.411569 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/1fa646b0-2860-4eb4-9465-277b573e8833-config-data\") pod \"1fa646b0-2860-4eb4-9465-277b573e8833\" (UID: \"1fa646b0-2860-4eb4-9465-277b573e8833\") " Dec 03 12:20:20 crc kubenswrapper[5052]: I1203 12:20:20.411587 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1fa646b0-2860-4eb4-9465-277b573e8833-combined-ca-bundle\") pod \"1fa646b0-2860-4eb4-9465-277b573e8833\" (UID: \"1fa646b0-2860-4eb4-9465-277b573e8833\") " Dec 03 12:20:20 crc kubenswrapper[5052]: I1203 12:20:20.411661 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/1fa646b0-2860-4eb4-9465-277b573e8833-logs\") pod \"1fa646b0-2860-4eb4-9465-277b573e8833\" (UID: \"1fa646b0-2860-4eb4-9465-277b573e8833\") " Dec 03 12:20:20 crc kubenswrapper[5052]: I1203 12:20:20.412449 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1fa646b0-2860-4eb4-9465-277b573e8833-logs" (OuterVolumeSpecName: "logs") pod "1fa646b0-2860-4eb4-9465-277b573e8833" (UID: "1fa646b0-2860-4eb4-9465-277b573e8833"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:20:20 crc kubenswrapper[5052]: I1203 12:20:20.422740 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1fa646b0-2860-4eb4-9465-277b573e8833-kube-api-access-cn8p6" (OuterVolumeSpecName: "kube-api-access-cn8p6") pod "1fa646b0-2860-4eb4-9465-277b573e8833" (UID: "1fa646b0-2860-4eb4-9465-277b573e8833"). InnerVolumeSpecName "kube-api-access-cn8p6". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:20:20 crc kubenswrapper[5052]: I1203 12:20:20.443807 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1fa646b0-2860-4eb4-9465-277b573e8833-config-data" (OuterVolumeSpecName: "config-data") pod "1fa646b0-2860-4eb4-9465-277b573e8833" (UID: "1fa646b0-2860-4eb4-9465-277b573e8833"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:20:20 crc kubenswrapper[5052]: I1203 12:20:20.443828 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1fa646b0-2860-4eb4-9465-277b573e8833-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "1fa646b0-2860-4eb4-9465-277b573e8833" (UID: "1fa646b0-2860-4eb4-9465-277b573e8833"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:20:20 crc kubenswrapper[5052]: I1203 12:20:20.513376 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/1fa646b0-2860-4eb4-9465-277b573e8833-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:20 crc kubenswrapper[5052]: I1203 12:20:20.513699 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1fa646b0-2860-4eb4-9465-277b573e8833-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:20 crc kubenswrapper[5052]: I1203 12:20:20.513795 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/1fa646b0-2860-4eb4-9465-277b573e8833-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:20 crc kubenswrapper[5052]: I1203 12:20:20.513874 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-cn8p6\" (UniqueName: \"kubernetes.io/projected/1fa646b0-2860-4eb4-9465-277b573e8833-kube-api-access-cn8p6\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:20 crc kubenswrapper[5052]: I1203 12:20:20.746920 5052 generic.go:334] "Generic (PLEG): container finished" podID="1fa646b0-2860-4eb4-9465-277b573e8833" containerID="f78f5c744f6b85db55f83c169645f4e175fea630e53c4631ba813ec516db849b" exitCode=0 Dec 03 12:20:20 crc kubenswrapper[5052]: I1203 12:20:20.746994 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:20:20 crc kubenswrapper[5052]: I1203 12:20:20.747011 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"1fa646b0-2860-4eb4-9465-277b573e8833","Type":"ContainerDied","Data":"f78f5c744f6b85db55f83c169645f4e175fea630e53c4631ba813ec516db849b"} Dec 03 12:20:20 crc kubenswrapper[5052]: I1203 12:20:20.747146 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"1fa646b0-2860-4eb4-9465-277b573e8833","Type":"ContainerDied","Data":"2aab4e3407e68b8be7dd0db246d395be26f96b302471d6ae99bd370ad54c036b"} Dec 03 12:20:20 crc kubenswrapper[5052]: I1203 12:20:20.747177 5052 scope.go:117] "RemoveContainer" containerID="f78f5c744f6b85db55f83c169645f4e175fea630e53c4631ba813ec516db849b" Dec 03 12:20:20 crc kubenswrapper[5052]: I1203 12:20:20.751835 5052 generic.go:334] "Generic (PLEG): container finished" podID="9e9cf358-eca4-4774-b1a1-1667774e900f" containerID="648a05aa3b83b4428ca8cb8790ec8c4c9aa4881d0992e41989e59db80706d517" exitCode=0 Dec 03 12:20:20 crc kubenswrapper[5052]: I1203 12:20:20.751884 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"9e9cf358-eca4-4774-b1a1-1667774e900f","Type":"ContainerDied","Data":"648a05aa3b83b4428ca8cb8790ec8c4c9aa4881d0992e41989e59db80706d517"} Dec 03 12:20:20 crc kubenswrapper[5052]: I1203 12:20:20.791754 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:20:20 crc kubenswrapper[5052]: I1203 12:20:20.800624 5052 scope.go:117] "RemoveContainer" containerID="3c9b1c00020081fb6aaa180232f9cccfc11821fc69ae178b2e202e185ce313c4" Dec 03 12:20:20 crc kubenswrapper[5052]: I1203 12:20:20.806316 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:20:20 crc kubenswrapper[5052]: I1203 12:20:20.815883 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:20:20 crc kubenswrapper[5052]: I1203 12:20:20.855747 5052 scope.go:117] "RemoveContainer" containerID="f78f5c744f6b85db55f83c169645f4e175fea630e53c4631ba813ec516db849b" Dec 03 12:20:20 crc kubenswrapper[5052]: E1203 12:20:20.856211 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f78f5c744f6b85db55f83c169645f4e175fea630e53c4631ba813ec516db849b\": container with ID starting with f78f5c744f6b85db55f83c169645f4e175fea630e53c4631ba813ec516db849b not found: ID does not exist" containerID="f78f5c744f6b85db55f83c169645f4e175fea630e53c4631ba813ec516db849b" Dec 03 12:20:20 crc kubenswrapper[5052]: I1203 12:20:20.856245 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f78f5c744f6b85db55f83c169645f4e175fea630e53c4631ba813ec516db849b"} err="failed to get container status \"f78f5c744f6b85db55f83c169645f4e175fea630e53c4631ba813ec516db849b\": rpc error: code = NotFound desc = could not find container \"f78f5c744f6b85db55f83c169645f4e175fea630e53c4631ba813ec516db849b\": container with ID starting with f78f5c744f6b85db55f83c169645f4e175fea630e53c4631ba813ec516db849b not found: ID does not exist" Dec 03 12:20:20 crc kubenswrapper[5052]: I1203 12:20:20.856270 5052 scope.go:117] "RemoveContainer" containerID="3c9b1c00020081fb6aaa180232f9cccfc11821fc69ae178b2e202e185ce313c4" Dec 03 12:20:20 crc kubenswrapper[5052]: I1203 12:20:20.859180 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:20:20 crc kubenswrapper[5052]: E1203 12:20:20.859745 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="9e9cf358-eca4-4774-b1a1-1667774e900f" containerName="ceilometer-notification-agent" Dec 03 12:20:20 crc kubenswrapper[5052]: I1203 12:20:20.859844 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="9e9cf358-eca4-4774-b1a1-1667774e900f" containerName="ceilometer-notification-agent" Dec 03 12:20:20 crc kubenswrapper[5052]: E1203 12:20:20.859932 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="9e9cf358-eca4-4774-b1a1-1667774e900f" containerName="ceilometer-central-agent" Dec 03 12:20:20 crc kubenswrapper[5052]: I1203 12:20:20.859995 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="9e9cf358-eca4-4774-b1a1-1667774e900f" containerName="ceilometer-central-agent" Dec 03 12:20:20 crc kubenswrapper[5052]: E1203 12:20:20.860056 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="9e9cf358-eca4-4774-b1a1-1667774e900f" containerName="sg-core" Dec 03 12:20:20 crc kubenswrapper[5052]: I1203 12:20:20.860115 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="9e9cf358-eca4-4774-b1a1-1667774e900f" containerName="sg-core" Dec 03 12:20:20 crc kubenswrapper[5052]: E1203 12:20:20.860186 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="9e9cf358-eca4-4774-b1a1-1667774e900f" containerName="proxy-httpd" Dec 03 12:20:20 crc kubenswrapper[5052]: I1203 12:20:20.860240 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="9e9cf358-eca4-4774-b1a1-1667774e900f" containerName="proxy-httpd" Dec 03 12:20:20 crc kubenswrapper[5052]: E1203 12:20:20.860313 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="1fa646b0-2860-4eb4-9465-277b573e8833" containerName="nova-api-log" Dec 03 12:20:20 crc kubenswrapper[5052]: I1203 12:20:20.860365 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="1fa646b0-2860-4eb4-9465-277b573e8833" containerName="nova-api-log" Dec 03 12:20:20 crc kubenswrapper[5052]: E1203 12:20:20.860422 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="1fa646b0-2860-4eb4-9465-277b573e8833" containerName="nova-api-api" Dec 03 12:20:20 crc kubenswrapper[5052]: I1203 12:20:20.860484 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="1fa646b0-2860-4eb4-9465-277b573e8833" containerName="nova-api-api" Dec 03 12:20:20 crc kubenswrapper[5052]: I1203 12:20:20.860781 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="9e9cf358-eca4-4774-b1a1-1667774e900f" containerName="sg-core" Dec 03 12:20:20 crc kubenswrapper[5052]: I1203 12:20:20.860855 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="1fa646b0-2860-4eb4-9465-277b573e8833" containerName="nova-api-api" Dec 03 12:20:20 crc kubenswrapper[5052]: I1203 12:20:20.860938 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="9e9cf358-eca4-4774-b1a1-1667774e900f" containerName="ceilometer-notification-agent" Dec 03 12:20:20 crc kubenswrapper[5052]: I1203 12:20:20.861030 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="9e9cf358-eca4-4774-b1a1-1667774e900f" containerName="ceilometer-central-agent" Dec 03 12:20:20 crc kubenswrapper[5052]: I1203 12:20:20.861114 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="1fa646b0-2860-4eb4-9465-277b573e8833" containerName="nova-api-log" Dec 03 12:20:20 crc kubenswrapper[5052]: I1203 12:20:20.861196 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="9e9cf358-eca4-4774-b1a1-1667774e900f" containerName="proxy-httpd" Dec 03 12:20:20 crc kubenswrapper[5052]: I1203 12:20:20.862326 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:20:20 crc kubenswrapper[5052]: E1203 12:20:20.862825 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3c9b1c00020081fb6aaa180232f9cccfc11821fc69ae178b2e202e185ce313c4\": container with ID starting with 3c9b1c00020081fb6aaa180232f9cccfc11821fc69ae178b2e202e185ce313c4 not found: ID does not exist" containerID="3c9b1c00020081fb6aaa180232f9cccfc11821fc69ae178b2e202e185ce313c4" Dec 03 12:20:20 crc kubenswrapper[5052]: I1203 12:20:20.864452 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3c9b1c00020081fb6aaa180232f9cccfc11821fc69ae178b2e202e185ce313c4"} err="failed to get container status \"3c9b1c00020081fb6aaa180232f9cccfc11821fc69ae178b2e202e185ce313c4\": rpc error: code = NotFound desc = could not find container \"3c9b1c00020081fb6aaa180232f9cccfc11821fc69ae178b2e202e185ce313c4\": container with ID starting with 3c9b1c00020081fb6aaa180232f9cccfc11821fc69ae178b2e202e185ce313c4 not found: ID does not exist" Dec 03 12:20:20 crc kubenswrapper[5052]: I1203 12:20:20.866855 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-nova-internal-svc" Dec 03 12:20:20 crc kubenswrapper[5052]: I1203 12:20:20.867014 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-nova-public-svc" Dec 03 12:20:20 crc kubenswrapper[5052]: I1203 12:20:20.867197 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-api-config-data" Dec 03 12:20:20 crc kubenswrapper[5052]: I1203 12:20:20.887561 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:20:20 crc kubenswrapper[5052]: I1203 12:20:20.935098 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/9e9cf358-eca4-4774-b1a1-1667774e900f-config-data\") pod \"9e9cf358-eca4-4774-b1a1-1667774e900f\" (UID: \"9e9cf358-eca4-4774-b1a1-1667774e900f\") " Dec 03 12:20:20 crc kubenswrapper[5052]: I1203 12:20:20.935179 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/9e9cf358-eca4-4774-b1a1-1667774e900f-ceilometer-tls-certs\") pod \"9e9cf358-eca4-4774-b1a1-1667774e900f\" (UID: \"9e9cf358-eca4-4774-b1a1-1667774e900f\") " Dec 03 12:20:20 crc kubenswrapper[5052]: I1203 12:20:20.935225 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vsqgq\" (UniqueName: \"kubernetes.io/projected/9e9cf358-eca4-4774-b1a1-1667774e900f-kube-api-access-vsqgq\") pod \"9e9cf358-eca4-4774-b1a1-1667774e900f\" (UID: \"9e9cf358-eca4-4774-b1a1-1667774e900f\") " Dec 03 12:20:20 crc kubenswrapper[5052]: I1203 12:20:20.935298 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/9e9cf358-eca4-4774-b1a1-1667774e900f-log-httpd\") pod \"9e9cf358-eca4-4774-b1a1-1667774e900f\" (UID: \"9e9cf358-eca4-4774-b1a1-1667774e900f\") " Dec 03 12:20:20 crc kubenswrapper[5052]: I1203 12:20:20.935417 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/9e9cf358-eca4-4774-b1a1-1667774e900f-run-httpd\") pod \"9e9cf358-eca4-4774-b1a1-1667774e900f\" (UID: \"9e9cf358-eca4-4774-b1a1-1667774e900f\") " Dec 03 12:20:20 crc kubenswrapper[5052]: I1203 12:20:20.935489 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9e9cf358-eca4-4774-b1a1-1667774e900f-combined-ca-bundle\") pod \"9e9cf358-eca4-4774-b1a1-1667774e900f\" (UID: \"9e9cf358-eca4-4774-b1a1-1667774e900f\") " Dec 03 12:20:20 crc kubenswrapper[5052]: I1203 12:20:20.935555 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/9e9cf358-eca4-4774-b1a1-1667774e900f-sg-core-conf-yaml\") pod \"9e9cf358-eca4-4774-b1a1-1667774e900f\" (UID: \"9e9cf358-eca4-4774-b1a1-1667774e900f\") " Dec 03 12:20:20 crc kubenswrapper[5052]: I1203 12:20:20.935727 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/9e9cf358-eca4-4774-b1a1-1667774e900f-scripts\") pod \"9e9cf358-eca4-4774-b1a1-1667774e900f\" (UID: \"9e9cf358-eca4-4774-b1a1-1667774e900f\") " Dec 03 12:20:20 crc kubenswrapper[5052]: I1203 12:20:20.936038 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7qgpg\" (UniqueName: \"kubernetes.io/projected/ce16d54b-ef04-429b-8795-2865395f7774-kube-api-access-7qgpg\") pod \"nova-api-0\" (UID: \"ce16d54b-ef04-429b-8795-2865395f7774\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:20:20 crc kubenswrapper[5052]: I1203 12:20:20.936097 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/ce16d54b-ef04-429b-8795-2865395f7774-public-tls-certs\") pod \"nova-api-0\" (UID: \"ce16d54b-ef04-429b-8795-2865395f7774\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:20:20 crc kubenswrapper[5052]: I1203 12:20:20.936160 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ce16d54b-ef04-429b-8795-2865395f7774-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"ce16d54b-ef04-429b-8795-2865395f7774\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:20:20 crc kubenswrapper[5052]: I1203 12:20:20.936256 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/ce16d54b-ef04-429b-8795-2865395f7774-config-data\") pod \"nova-api-0\" (UID: \"ce16d54b-ef04-429b-8795-2865395f7774\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:20:20 crc kubenswrapper[5052]: I1203 12:20:20.936307 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/ce16d54b-ef04-429b-8795-2865395f7774-internal-tls-certs\") pod \"nova-api-0\" (UID: \"ce16d54b-ef04-429b-8795-2865395f7774\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:20:20 crc kubenswrapper[5052]: I1203 12:20:20.936335 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/ce16d54b-ef04-429b-8795-2865395f7774-logs\") pod \"nova-api-0\" (UID: \"ce16d54b-ef04-429b-8795-2865395f7774\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:20:20 crc kubenswrapper[5052]: I1203 12:20:20.937990 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9e9cf358-eca4-4774-b1a1-1667774e900f-run-httpd" (OuterVolumeSpecName: "run-httpd") pod "9e9cf358-eca4-4774-b1a1-1667774e900f" (UID: "9e9cf358-eca4-4774-b1a1-1667774e900f"). InnerVolumeSpecName "run-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:20:20 crc kubenswrapper[5052]: I1203 12:20:20.938434 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9e9cf358-eca4-4774-b1a1-1667774e900f-log-httpd" (OuterVolumeSpecName: "log-httpd") pod "9e9cf358-eca4-4774-b1a1-1667774e900f" (UID: "9e9cf358-eca4-4774-b1a1-1667774e900f"). InnerVolumeSpecName "log-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:20:20 crc kubenswrapper[5052]: I1203 12:20:20.941449 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9e9cf358-eca4-4774-b1a1-1667774e900f-scripts" (OuterVolumeSpecName: "scripts") pod "9e9cf358-eca4-4774-b1a1-1667774e900f" (UID: "9e9cf358-eca4-4774-b1a1-1667774e900f"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:20:20 crc kubenswrapper[5052]: I1203 12:20:20.943751 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9e9cf358-eca4-4774-b1a1-1667774e900f-kube-api-access-vsqgq" (OuterVolumeSpecName: "kube-api-access-vsqgq") pod "9e9cf358-eca4-4774-b1a1-1667774e900f" (UID: "9e9cf358-eca4-4774-b1a1-1667774e900f"). InnerVolumeSpecName "kube-api-access-vsqgq". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:20:20 crc kubenswrapper[5052]: I1203 12:20:20.981830 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9e9cf358-eca4-4774-b1a1-1667774e900f-sg-core-conf-yaml" (OuterVolumeSpecName: "sg-core-conf-yaml") pod "9e9cf358-eca4-4774-b1a1-1667774e900f" (UID: "9e9cf358-eca4-4774-b1a1-1667774e900f"). InnerVolumeSpecName "sg-core-conf-yaml". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:20:20 crc kubenswrapper[5052]: I1203 12:20:20.989546 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9e9cf358-eca4-4774-b1a1-1667774e900f-ceilometer-tls-certs" (OuterVolumeSpecName: "ceilometer-tls-certs") pod "9e9cf358-eca4-4774-b1a1-1667774e900f" (UID: "9e9cf358-eca4-4774-b1a1-1667774e900f"). InnerVolumeSpecName "ceilometer-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:20:21 crc kubenswrapper[5052]: I1203 12:20:21.014696 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9e9cf358-eca4-4774-b1a1-1667774e900f-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "9e9cf358-eca4-4774-b1a1-1667774e900f" (UID: "9e9cf358-eca4-4774-b1a1-1667774e900f"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:20:21 crc kubenswrapper[5052]: I1203 12:20:21.039225 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-7qgpg\" (UniqueName: \"kubernetes.io/projected/ce16d54b-ef04-429b-8795-2865395f7774-kube-api-access-7qgpg\") pod \"nova-api-0\" (UID: \"ce16d54b-ef04-429b-8795-2865395f7774\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:20:21 crc kubenswrapper[5052]: I1203 12:20:21.039286 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/ce16d54b-ef04-429b-8795-2865395f7774-public-tls-certs\") pod \"nova-api-0\" (UID: \"ce16d54b-ef04-429b-8795-2865395f7774\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:20:21 crc kubenswrapper[5052]: I1203 12:20:21.039330 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ce16d54b-ef04-429b-8795-2865395f7774-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"ce16d54b-ef04-429b-8795-2865395f7774\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:20:21 crc kubenswrapper[5052]: I1203 12:20:21.039405 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/ce16d54b-ef04-429b-8795-2865395f7774-config-data\") pod \"nova-api-0\" (UID: \"ce16d54b-ef04-429b-8795-2865395f7774\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:20:21 crc kubenswrapper[5052]: I1203 12:20:21.039440 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/ce16d54b-ef04-429b-8795-2865395f7774-internal-tls-certs\") pod \"nova-api-0\" (UID: \"ce16d54b-ef04-429b-8795-2865395f7774\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:20:21 crc kubenswrapper[5052]: I1203 12:20:21.039454 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/ce16d54b-ef04-429b-8795-2865395f7774-logs\") pod \"nova-api-0\" (UID: \"ce16d54b-ef04-429b-8795-2865395f7774\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:20:21 crc kubenswrapper[5052]: I1203 12:20:21.039540 5052 reconciler_common.go:293] "Volume detached for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/9e9cf358-eca4-4774-b1a1-1667774e900f-run-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:21 crc kubenswrapper[5052]: I1203 12:20:21.039551 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9e9cf358-eca4-4774-b1a1-1667774e900f-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:21 crc kubenswrapper[5052]: I1203 12:20:21.039560 5052 reconciler_common.go:293] "Volume detached for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/9e9cf358-eca4-4774-b1a1-1667774e900f-sg-core-conf-yaml\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:21 crc kubenswrapper[5052]: I1203 12:20:21.039568 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/9e9cf358-eca4-4774-b1a1-1667774e900f-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:21 crc kubenswrapper[5052]: I1203 12:20:21.039575 5052 reconciler_common.go:293] "Volume detached for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/9e9cf358-eca4-4774-b1a1-1667774e900f-ceilometer-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:21 crc kubenswrapper[5052]: I1203 12:20:21.039584 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-vsqgq\" (UniqueName: \"kubernetes.io/projected/9e9cf358-eca4-4774-b1a1-1667774e900f-kube-api-access-vsqgq\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:21 crc kubenswrapper[5052]: I1203 12:20:21.039592 5052 reconciler_common.go:293] "Volume detached for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/9e9cf358-eca4-4774-b1a1-1667774e900f-log-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:21 crc kubenswrapper[5052]: I1203 12:20:21.039915 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/ce16d54b-ef04-429b-8795-2865395f7774-logs\") pod \"nova-api-0\" (UID: \"ce16d54b-ef04-429b-8795-2865395f7774\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:20:21 crc kubenswrapper[5052]: I1203 12:20:21.045233 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/ce16d54b-ef04-429b-8795-2865395f7774-public-tls-certs\") pod \"nova-api-0\" (UID: \"ce16d54b-ef04-429b-8795-2865395f7774\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:20:21 crc kubenswrapper[5052]: I1203 12:20:21.045535 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/ce16d54b-ef04-429b-8795-2865395f7774-config-data\") pod \"nova-api-0\" (UID: \"ce16d54b-ef04-429b-8795-2865395f7774\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:20:21 crc kubenswrapper[5052]: I1203 12:20:21.048454 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/ce16d54b-ef04-429b-8795-2865395f7774-internal-tls-certs\") pod \"nova-api-0\" (UID: \"ce16d54b-ef04-429b-8795-2865395f7774\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:20:21 crc kubenswrapper[5052]: I1203 12:20:21.051934 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ce16d54b-ef04-429b-8795-2865395f7774-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"ce16d54b-ef04-429b-8795-2865395f7774\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:20:21 crc kubenswrapper[5052]: I1203 12:20:21.058272 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9e9cf358-eca4-4774-b1a1-1667774e900f-config-data" (OuterVolumeSpecName: "config-data") pod "9e9cf358-eca4-4774-b1a1-1667774e900f" (UID: "9e9cf358-eca4-4774-b1a1-1667774e900f"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:20:21 crc kubenswrapper[5052]: I1203 12:20:21.059449 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-7qgpg\" (UniqueName: \"kubernetes.io/projected/ce16d54b-ef04-429b-8795-2865395f7774-kube-api-access-7qgpg\") pod \"nova-api-0\" (UID: \"ce16d54b-ef04-429b-8795-2865395f7774\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:20:21 crc kubenswrapper[5052]: I1203 12:20:21.141890 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/9e9cf358-eca4-4774-b1a1-1667774e900f-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:21 crc kubenswrapper[5052]: I1203 12:20:21.190623 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:20:21 crc kubenswrapper[5052]: I1203 12:20:21.642076 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/openstack-galera-0"] Dec 03 12:20:21 crc kubenswrapper[5052]: I1203 12:20:21.688925 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1fa646b0-2860-4eb4-9465-277b573e8833" path="/var/lib/kubelet/pods/1fa646b0-2860-4eb4-9465-277b573e8833/volumes" Dec 03 12:20:21 crc kubenswrapper[5052]: I1203 12:20:21.691149 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/memcached-0"] Dec 03 12:20:21 crc kubenswrapper[5052]: I1203 12:20:21.692405 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/memcached-0" podUID="695de243-96a9-4e8b-a0f3-3a4ef2d95efe" containerName="memcached" containerID="cri-o://a0b610780c737434b4d97d2fd99c01273a88681fdc19b3a41fdf3d032c04b664" gracePeriod=30 Dec 03 12:20:21 crc kubenswrapper[5052]: I1203 12:20:21.722452 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/rabbitmq-server-0"] Dec 03 12:20:21 crc kubenswrapper[5052]: I1203 12:20:21.746527 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:20:21 crc kubenswrapper[5052]: I1203 12:20:21.791086 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/openstack-cell1-galera-0"] Dec 03 12:20:21 crc kubenswrapper[5052]: I1203 12:20:21.799829 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"9e9cf358-eca4-4774-b1a1-1667774e900f","Type":"ContainerDied","Data":"324894c5addce3772545fa5d883bb38c905b52ac439d4f5c51fa71a3d37ba150"} Dec 03 12:20:21 crc kubenswrapper[5052]: I1203 12:20:21.799884 5052 scope.go:117] "RemoveContainer" containerID="88e82ab460c499ca6f48460cef37af029831a0965950387bc54369ec6d4ca508" Dec 03 12:20:21 crc kubenswrapper[5052]: I1203 12:20:21.800094 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:20:21 crc kubenswrapper[5052]: I1203 12:20:21.854886 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/rabbitmq-cell1-server-0"] Dec 03 12:20:21 crc kubenswrapper[5052]: I1203 12:20:21.884549 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:20:21 crc kubenswrapper[5052]: I1203 12:20:21.915240 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:20:21 crc kubenswrapper[5052]: I1203 12:20:21.986643 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:20:21 crc kubenswrapper[5052]: I1203 12:20:21.996648 5052 scope.go:117] "RemoveContainer" containerID="6ad4071deb854da06386e9a499db2d3dac5c11cb33dde601cceab73c2f94d29b" Dec 03 12:20:22 crc kubenswrapper[5052]: I1203 12:20:22.008445 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/placement-db-sync-hh2rh"] Dec 03 12:20:22 crc kubenswrapper[5052]: I1203 12:20:22.008633 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:20:22 crc kubenswrapper[5052]: I1203 12:20:22.016160 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-scripts" Dec 03 12:20:22 crc kubenswrapper[5052]: I1203 12:20:22.018032 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-ceilometer-internal-svc" Dec 03 12:20:22 crc kubenswrapper[5052]: I1203 12:20:22.018352 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-config-data" Dec 03 12:20:22 crc kubenswrapper[5052]: I1203 12:20:22.024766 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:20:22 crc kubenswrapper[5052]: I1203 12:20:22.047364 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/keystone-db-sync-4tktw"] Dec 03 12:20:22 crc kubenswrapper[5052]: I1203 12:20:22.063286 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/placement-db-sync-hh2rh"] Dec 03 12:20:22 crc kubenswrapper[5052]: I1203 12:20:22.073555 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/ae23fe34-7bf4-47fc-addc-5edda0399c77-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"ae23fe34-7bf4-47fc-addc-5edda0399c77\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:20:22 crc kubenswrapper[5052]: I1203 12:20:22.073636 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/ae23fe34-7bf4-47fc-addc-5edda0399c77-ceilometer-tls-certs\") pod \"ceilometer-0\" (UID: \"ae23fe34-7bf4-47fc-addc-5edda0399c77\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:20:22 crc kubenswrapper[5052]: I1203 12:20:22.073677 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xbhgc\" (UniqueName: \"kubernetes.io/projected/ae23fe34-7bf4-47fc-addc-5edda0399c77-kube-api-access-xbhgc\") pod \"ceilometer-0\" (UID: \"ae23fe34-7bf4-47fc-addc-5edda0399c77\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:20:22 crc kubenswrapper[5052]: I1203 12:20:22.073726 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/ae23fe34-7bf4-47fc-addc-5edda0399c77-scripts\") pod \"ceilometer-0\" (UID: \"ae23fe34-7bf4-47fc-addc-5edda0399c77\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:20:22 crc kubenswrapper[5052]: I1203 12:20:22.073793 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ae23fe34-7bf4-47fc-addc-5edda0399c77-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"ae23fe34-7bf4-47fc-addc-5edda0399c77\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:20:22 crc kubenswrapper[5052]: I1203 12:20:22.073822 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/ae23fe34-7bf4-47fc-addc-5edda0399c77-config-data\") pod \"ceilometer-0\" (UID: \"ae23fe34-7bf4-47fc-addc-5edda0399c77\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:20:22 crc kubenswrapper[5052]: I1203 12:20:22.073853 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/ae23fe34-7bf4-47fc-addc-5edda0399c77-run-httpd\") pod \"ceilometer-0\" (UID: \"ae23fe34-7bf4-47fc-addc-5edda0399c77\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:20:22 crc kubenswrapper[5052]: I1203 12:20:22.073915 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/ae23fe34-7bf4-47fc-addc-5edda0399c77-log-httpd\") pod \"ceilometer-0\" (UID: \"ae23fe34-7bf4-47fc-addc-5edda0399c77\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:20:22 crc kubenswrapper[5052]: I1203 12:20:22.078720 5052 scope.go:117] "RemoveContainer" containerID="648a05aa3b83b4428ca8cb8790ec8c4c9aa4881d0992e41989e59db80706d517" Dec 03 12:20:22 crc kubenswrapper[5052]: I1203 12:20:22.078834 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/keystone-db-sync-4tktw"] Dec 03 12:20:22 crc kubenswrapper[5052]: I1203 12:20:22.127618 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/placement-db-sync-gzzkp"] Dec 03 12:20:22 crc kubenswrapper[5052]: I1203 12:20:22.129592 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-db-sync-gzzkp" Dec 03 12:20:22 crc kubenswrapper[5052]: I1203 12:20:22.153647 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/placement-db-sync-gzzkp"] Dec 03 12:20:22 crc kubenswrapper[5052]: I1203 12:20:22.178923 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/ae23fe34-7bf4-47fc-addc-5edda0399c77-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"ae23fe34-7bf4-47fc-addc-5edda0399c77\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:20:22 crc kubenswrapper[5052]: I1203 12:20:22.178980 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/ae23fe34-7bf4-47fc-addc-5edda0399c77-ceilometer-tls-certs\") pod \"ceilometer-0\" (UID: \"ae23fe34-7bf4-47fc-addc-5edda0399c77\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:20:22 crc kubenswrapper[5052]: I1203 12:20:22.179018 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-xbhgc\" (UniqueName: \"kubernetes.io/projected/ae23fe34-7bf4-47fc-addc-5edda0399c77-kube-api-access-xbhgc\") pod \"ceilometer-0\" (UID: \"ae23fe34-7bf4-47fc-addc-5edda0399c77\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:20:22 crc kubenswrapper[5052]: I1203 12:20:22.179069 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/ae23fe34-7bf4-47fc-addc-5edda0399c77-scripts\") pod \"ceilometer-0\" (UID: \"ae23fe34-7bf4-47fc-addc-5edda0399c77\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:20:22 crc kubenswrapper[5052]: I1203 12:20:22.179138 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ae23fe34-7bf4-47fc-addc-5edda0399c77-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"ae23fe34-7bf4-47fc-addc-5edda0399c77\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:20:22 crc kubenswrapper[5052]: I1203 12:20:22.179167 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/ae23fe34-7bf4-47fc-addc-5edda0399c77-config-data\") pod \"ceilometer-0\" (UID: \"ae23fe34-7bf4-47fc-addc-5edda0399c77\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:20:22 crc kubenswrapper[5052]: I1203 12:20:22.179225 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/ae23fe34-7bf4-47fc-addc-5edda0399c77-run-httpd\") pod \"ceilometer-0\" (UID: \"ae23fe34-7bf4-47fc-addc-5edda0399c77\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:20:22 crc kubenswrapper[5052]: I1203 12:20:22.179271 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/ae23fe34-7bf4-47fc-addc-5edda0399c77-log-httpd\") pod \"ceilometer-0\" (UID: \"ae23fe34-7bf4-47fc-addc-5edda0399c77\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:20:22 crc kubenswrapper[5052]: I1203 12:20:22.179865 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/ae23fe34-7bf4-47fc-addc-5edda0399c77-log-httpd\") pod \"ceilometer-0\" (UID: \"ae23fe34-7bf4-47fc-addc-5edda0399c77\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:20:22 crc kubenswrapper[5052]: I1203 12:20:22.183695 5052 scope.go:117] "RemoveContainer" containerID="b1d5d717d0fcd013d58f887c25d6c34fe306a5816962f6a70d25cd36f488c6a3" Dec 03 12:20:22 crc kubenswrapper[5052]: I1203 12:20:22.185787 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/ae23fe34-7bf4-47fc-addc-5edda0399c77-run-httpd\") pod \"ceilometer-0\" (UID: \"ae23fe34-7bf4-47fc-addc-5edda0399c77\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:20:22 crc kubenswrapper[5052]: I1203 12:20:22.209234 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/ae23fe34-7bf4-47fc-addc-5edda0399c77-ceilometer-tls-certs\") pod \"ceilometer-0\" (UID: \"ae23fe34-7bf4-47fc-addc-5edda0399c77\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:20:22 crc kubenswrapper[5052]: I1203 12:20:22.231215 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/keystone-db-sync-9jf9q"] Dec 03 12:20:22 crc kubenswrapper[5052]: I1203 12:20:22.235262 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-db-sync-9jf9q" Dec 03 12:20:22 crc kubenswrapper[5052]: I1203 12:20:22.254666 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/ae23fe34-7bf4-47fc-addc-5edda0399c77-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"ae23fe34-7bf4-47fc-addc-5edda0399c77\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:20:22 crc kubenswrapper[5052]: I1203 12:20:22.255148 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/ae23fe34-7bf4-47fc-addc-5edda0399c77-config-data\") pod \"ceilometer-0\" (UID: \"ae23fe34-7bf4-47fc-addc-5edda0399c77\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:20:22 crc kubenswrapper[5052]: I1203 12:20:22.260226 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/ae23fe34-7bf4-47fc-addc-5edda0399c77-scripts\") pod \"ceilometer-0\" (UID: \"ae23fe34-7bf4-47fc-addc-5edda0399c77\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:20:22 crc kubenswrapper[5052]: I1203 12:20:22.268629 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-xbhgc\" (UniqueName: \"kubernetes.io/projected/ae23fe34-7bf4-47fc-addc-5edda0399c77-kube-api-access-xbhgc\") pod \"ceilometer-0\" (UID: \"ae23fe34-7bf4-47fc-addc-5edda0399c77\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:20:22 crc kubenswrapper[5052]: I1203 12:20:22.278125 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ae23fe34-7bf4-47fc-addc-5edda0399c77-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"ae23fe34-7bf4-47fc-addc-5edda0399c77\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:20:22 crc kubenswrapper[5052]: I1203 12:20:22.282430 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r54l6\" (UniqueName: \"kubernetes.io/projected/50ae31b2-a1c0-4afb-8e23-d7856c55af1d-kube-api-access-r54l6\") pod \"placement-db-sync-gzzkp\" (UID: \"50ae31b2-a1c0-4afb-8e23-d7856c55af1d\") " pod="openstack-kuttl-tests/placement-db-sync-gzzkp" Dec 03 12:20:22 crc kubenswrapper[5052]: I1203 12:20:22.282538 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/50ae31b2-a1c0-4afb-8e23-d7856c55af1d-scripts\") pod \"placement-db-sync-gzzkp\" (UID: \"50ae31b2-a1c0-4afb-8e23-d7856c55af1d\") " pod="openstack-kuttl-tests/placement-db-sync-gzzkp" Dec 03 12:20:22 crc kubenswrapper[5052]: I1203 12:20:22.282625 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/50ae31b2-a1c0-4afb-8e23-d7856c55af1d-logs\") pod \"placement-db-sync-gzzkp\" (UID: \"50ae31b2-a1c0-4afb-8e23-d7856c55af1d\") " pod="openstack-kuttl-tests/placement-db-sync-gzzkp" Dec 03 12:20:22 crc kubenswrapper[5052]: I1203 12:20:22.282737 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/50ae31b2-a1c0-4afb-8e23-d7856c55af1d-config-data\") pod \"placement-db-sync-gzzkp\" (UID: \"50ae31b2-a1c0-4afb-8e23-d7856c55af1d\") " pod="openstack-kuttl-tests/placement-db-sync-gzzkp" Dec 03 12:20:22 crc kubenswrapper[5052]: I1203 12:20:22.282762 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/50ae31b2-a1c0-4afb-8e23-d7856c55af1d-combined-ca-bundle\") pod \"placement-db-sync-gzzkp\" (UID: \"50ae31b2-a1c0-4afb-8e23-d7856c55af1d\") " pod="openstack-kuttl-tests/placement-db-sync-gzzkp" Dec 03 12:20:22 crc kubenswrapper[5052]: I1203 12:20:22.321949 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-db-sync-9jf9q"] Dec 03 12:20:22 crc kubenswrapper[5052]: I1203 12:20:22.369092 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:20:22 crc kubenswrapper[5052]: I1203 12:20:22.375166 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-db-sync-qstzv"] Dec 03 12:20:22 crc kubenswrapper[5052]: I1203 12:20:22.386902 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b852a7d7-8510-440d-8e87-0755fac2a324-combined-ca-bundle\") pod \"keystone-db-sync-9jf9q\" (UID: \"b852a7d7-8510-440d-8e87-0755fac2a324\") " pod="openstack-kuttl-tests/keystone-db-sync-9jf9q" Dec 03 12:20:22 crc kubenswrapper[5052]: I1203 12:20:22.386966 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/50ae31b2-a1c0-4afb-8e23-d7856c55af1d-scripts\") pod \"placement-db-sync-gzzkp\" (UID: \"50ae31b2-a1c0-4afb-8e23-d7856c55af1d\") " pod="openstack-kuttl-tests/placement-db-sync-gzzkp" Dec 03 12:20:22 crc kubenswrapper[5052]: I1203 12:20:22.387054 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/50ae31b2-a1c0-4afb-8e23-d7856c55af1d-logs\") pod \"placement-db-sync-gzzkp\" (UID: \"50ae31b2-a1c0-4afb-8e23-d7856c55af1d\") " pod="openstack-kuttl-tests/placement-db-sync-gzzkp" Dec 03 12:20:22 crc kubenswrapper[5052]: I1203 12:20:22.387144 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/50ae31b2-a1c0-4afb-8e23-d7856c55af1d-config-data\") pod \"placement-db-sync-gzzkp\" (UID: \"50ae31b2-a1c0-4afb-8e23-d7856c55af1d\") " pod="openstack-kuttl-tests/placement-db-sync-gzzkp" Dec 03 12:20:22 crc kubenswrapper[5052]: I1203 12:20:22.387176 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/50ae31b2-a1c0-4afb-8e23-d7856c55af1d-combined-ca-bundle\") pod \"placement-db-sync-gzzkp\" (UID: \"50ae31b2-a1c0-4afb-8e23-d7856c55af1d\") " pod="openstack-kuttl-tests/placement-db-sync-gzzkp" Dec 03 12:20:22 crc kubenswrapper[5052]: I1203 12:20:22.387205 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6pcrx\" (UniqueName: \"kubernetes.io/projected/b852a7d7-8510-440d-8e87-0755fac2a324-kube-api-access-6pcrx\") pod \"keystone-db-sync-9jf9q\" (UID: \"b852a7d7-8510-440d-8e87-0755fac2a324\") " pod="openstack-kuttl-tests/keystone-db-sync-9jf9q" Dec 03 12:20:22 crc kubenswrapper[5052]: I1203 12:20:22.387232 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/b852a7d7-8510-440d-8e87-0755fac2a324-config-data\") pod \"keystone-db-sync-9jf9q\" (UID: \"b852a7d7-8510-440d-8e87-0755fac2a324\") " pod="openstack-kuttl-tests/keystone-db-sync-9jf9q" Dec 03 12:20:22 crc kubenswrapper[5052]: I1203 12:20:22.387282 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-r54l6\" (UniqueName: \"kubernetes.io/projected/50ae31b2-a1c0-4afb-8e23-d7856c55af1d-kube-api-access-r54l6\") pod \"placement-db-sync-gzzkp\" (UID: \"50ae31b2-a1c0-4afb-8e23-d7856c55af1d\") " pod="openstack-kuttl-tests/placement-db-sync-gzzkp" Dec 03 12:20:22 crc kubenswrapper[5052]: I1203 12:20:22.396539 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/50ae31b2-a1c0-4afb-8e23-d7856c55af1d-logs\") pod \"placement-db-sync-gzzkp\" (UID: \"50ae31b2-a1c0-4afb-8e23-d7856c55af1d\") " pod="openstack-kuttl-tests/placement-db-sync-gzzkp" Dec 03 12:20:22 crc kubenswrapper[5052]: I1203 12:20:22.408502 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/50ae31b2-a1c0-4afb-8e23-d7856c55af1d-scripts\") pod \"placement-db-sync-gzzkp\" (UID: \"50ae31b2-a1c0-4afb-8e23-d7856c55af1d\") " pod="openstack-kuttl-tests/placement-db-sync-gzzkp" Dec 03 12:20:22 crc kubenswrapper[5052]: I1203 12:20:22.408559 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/50ae31b2-a1c0-4afb-8e23-d7856c55af1d-combined-ca-bundle\") pod \"placement-db-sync-gzzkp\" (UID: \"50ae31b2-a1c0-4afb-8e23-d7856c55af1d\") " pod="openstack-kuttl-tests/placement-db-sync-gzzkp" Dec 03 12:20:22 crc kubenswrapper[5052]: I1203 12:20:22.409275 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/50ae31b2-a1c0-4afb-8e23-d7856c55af1d-config-data\") pod \"placement-db-sync-gzzkp\" (UID: \"50ae31b2-a1c0-4afb-8e23-d7856c55af1d\") " pod="openstack-kuttl-tests/placement-db-sync-gzzkp" Dec 03 12:20:22 crc kubenswrapper[5052]: I1203 12:20:22.414197 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-r54l6\" (UniqueName: \"kubernetes.io/projected/50ae31b2-a1c0-4afb-8e23-d7856c55af1d-kube-api-access-r54l6\") pod \"placement-db-sync-gzzkp\" (UID: \"50ae31b2-a1c0-4afb-8e23-d7856c55af1d\") " pod="openstack-kuttl-tests/placement-db-sync-gzzkp" Dec 03 12:20:22 crc kubenswrapper[5052]: I1203 12:20:22.414677 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/glance-db-sync-qstzv"] Dec 03 12:20:22 crc kubenswrapper[5052]: I1203 12:20:22.486624 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-db-sync-gzzkp" Dec 03 12:20:22 crc kubenswrapper[5052]: I1203 12:20:22.488994 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-6pcrx\" (UniqueName: \"kubernetes.io/projected/b852a7d7-8510-440d-8e87-0755fac2a324-kube-api-access-6pcrx\") pod \"keystone-db-sync-9jf9q\" (UID: \"b852a7d7-8510-440d-8e87-0755fac2a324\") " pod="openstack-kuttl-tests/keystone-db-sync-9jf9q" Dec 03 12:20:22 crc kubenswrapper[5052]: I1203 12:20:22.489062 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/b852a7d7-8510-440d-8e87-0755fac2a324-config-data\") pod \"keystone-db-sync-9jf9q\" (UID: \"b852a7d7-8510-440d-8e87-0755fac2a324\") " pod="openstack-kuttl-tests/keystone-db-sync-9jf9q" Dec 03 12:20:22 crc kubenswrapper[5052]: I1203 12:20:22.489146 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b852a7d7-8510-440d-8e87-0755fac2a324-combined-ca-bundle\") pod \"keystone-db-sync-9jf9q\" (UID: \"b852a7d7-8510-440d-8e87-0755fac2a324\") " pod="openstack-kuttl-tests/keystone-db-sync-9jf9q" Dec 03 12:20:22 crc kubenswrapper[5052]: I1203 12:20:22.496789 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/b852a7d7-8510-440d-8e87-0755fac2a324-config-data\") pod \"keystone-db-sync-9jf9q\" (UID: \"b852a7d7-8510-440d-8e87-0755fac2a324\") " pod="openstack-kuttl-tests/keystone-db-sync-9jf9q" Dec 03 12:20:22 crc kubenswrapper[5052]: I1203 12:20:22.497745 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b852a7d7-8510-440d-8e87-0755fac2a324-combined-ca-bundle\") pod \"keystone-db-sync-9jf9q\" (UID: \"b852a7d7-8510-440d-8e87-0755fac2a324\") " pod="openstack-kuttl-tests/keystone-db-sync-9jf9q" Dec 03 12:20:22 crc kubenswrapper[5052]: I1203 12:20:22.515589 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-nb-0"] Dec 03 12:20:22 crc kubenswrapper[5052]: I1203 12:20:22.517024 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ovsdbserver-nb-0" podUID="fcd60977-d995-42ee-b654-ba2947a3316d" containerName="openstack-network-exporter" containerID="cri-o://daa77eec7322cd2c65ca8b891da5191f57aec462539227515b3efcddb082df4d" gracePeriod=300 Dec 03 12:20:22 crc kubenswrapper[5052]: I1203 12:20:22.561680 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-6pcrx\" (UniqueName: \"kubernetes.io/projected/b852a7d7-8510-440d-8e87-0755fac2a324-kube-api-access-6pcrx\") pod \"keystone-db-sync-9jf9q\" (UID: \"b852a7d7-8510-440d-8e87-0755fac2a324\") " pod="openstack-kuttl-tests/keystone-db-sync-9jf9q" Dec 03 12:20:22 crc kubenswrapper[5052]: I1203 12:20:22.563095 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-db-sync-9jf9q" Dec 03 12:20:22 crc kubenswrapper[5052]: I1203 12:20:22.584170 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/glance-db-sync-pvgq9"] Dec 03 12:20:22 crc kubenswrapper[5052]: I1203 12:20:22.586717 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-db-sync-pvgq9" Dec 03 12:20:22 crc kubenswrapper[5052]: I1203 12:20:22.598326 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-db-sync-pvgq9"] Dec 03 12:20:22 crc kubenswrapper[5052]: I1203 12:20:22.610896 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-config-data" Dec 03 12:20:22 crc kubenswrapper[5052]: I1203 12:20:22.644748 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder-db-sync-wp26m"] Dec 03 12:20:22 crc kubenswrapper[5052]: I1203 12:20:22.680120 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/cinder-db-sync-wp26m"] Dec 03 12:20:22 crc kubenswrapper[5052]: I1203 12:20:22.686908 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ovsdbserver-nb-0" podUID="fcd60977-d995-42ee-b654-ba2947a3316d" containerName="ovsdbserver-nb" containerID="cri-o://87154b4a2a5069ca41b7570b52ae76e123d9dca6d0b7110267bc2f70b48cd720" gracePeriod=300 Dec 03 12:20:22 crc kubenswrapper[5052]: I1203 12:20:22.704856 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/66f5168a-1ee5-4de9-8e50-90ef25186ee8-combined-ca-bundle\") pod \"glance-db-sync-pvgq9\" (UID: \"66f5168a-1ee5-4de9-8e50-90ef25186ee8\") " pod="openstack-kuttl-tests/glance-db-sync-pvgq9" Dec 03 12:20:22 crc kubenswrapper[5052]: I1203 12:20:22.705082 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/66f5168a-1ee5-4de9-8e50-90ef25186ee8-db-sync-config-data\") pod \"glance-db-sync-pvgq9\" (UID: \"66f5168a-1ee5-4de9-8e50-90ef25186ee8\") " pod="openstack-kuttl-tests/glance-db-sync-pvgq9" Dec 03 12:20:22 crc kubenswrapper[5052]: I1203 12:20:22.705352 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wlsk7\" (UniqueName: \"kubernetes.io/projected/66f5168a-1ee5-4de9-8e50-90ef25186ee8-kube-api-access-wlsk7\") pod \"glance-db-sync-pvgq9\" (UID: \"66f5168a-1ee5-4de9-8e50-90ef25186ee8\") " pod="openstack-kuttl-tests/glance-db-sync-pvgq9" Dec 03 12:20:22 crc kubenswrapper[5052]: I1203 12:20:22.705460 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/66f5168a-1ee5-4de9-8e50-90ef25186ee8-config-data\") pod \"glance-db-sync-pvgq9\" (UID: \"66f5168a-1ee5-4de9-8e50-90ef25186ee8\") " pod="openstack-kuttl-tests/glance-db-sync-pvgq9" Dec 03 12:20:22 crc kubenswrapper[5052]: I1203 12:20:22.801707 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ovn-northd-0"] Dec 03 12:20:22 crc kubenswrapper[5052]: I1203 12:20:22.802735 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ovn-northd-0" podUID="966185e5-f414-46c5-ba48-0e234750795a" containerName="ovn-northd" containerID="cri-o://ef532f5fb83b3ed8c9243e5c20875e68a9ee6b646ee961f30ffe04c5b504e0f8" gracePeriod=30 Dec 03 12:20:22 crc kubenswrapper[5052]: I1203 12:20:22.802934 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ovn-northd-0" podUID="966185e5-f414-46c5-ba48-0e234750795a" containerName="openstack-network-exporter" containerID="cri-o://da2e79425aa667a3b6c86223531989383a59d7905be7837190a7bae4c94575bc" gracePeriod=30 Dec 03 12:20:22 crc kubenswrapper[5052]: I1203 12:20:22.809762 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/66f5168a-1ee5-4de9-8e50-90ef25186ee8-db-sync-config-data\") pod \"glance-db-sync-pvgq9\" (UID: \"66f5168a-1ee5-4de9-8e50-90ef25186ee8\") " pod="openstack-kuttl-tests/glance-db-sync-pvgq9" Dec 03 12:20:22 crc kubenswrapper[5052]: I1203 12:20:22.809852 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-wlsk7\" (UniqueName: \"kubernetes.io/projected/66f5168a-1ee5-4de9-8e50-90ef25186ee8-kube-api-access-wlsk7\") pod \"glance-db-sync-pvgq9\" (UID: \"66f5168a-1ee5-4de9-8e50-90ef25186ee8\") " pod="openstack-kuttl-tests/glance-db-sync-pvgq9" Dec 03 12:20:22 crc kubenswrapper[5052]: I1203 12:20:22.809906 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/66f5168a-1ee5-4de9-8e50-90ef25186ee8-config-data\") pod \"glance-db-sync-pvgq9\" (UID: \"66f5168a-1ee5-4de9-8e50-90ef25186ee8\") " pod="openstack-kuttl-tests/glance-db-sync-pvgq9" Dec 03 12:20:22 crc kubenswrapper[5052]: I1203 12:20:22.809973 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/66f5168a-1ee5-4de9-8e50-90ef25186ee8-combined-ca-bundle\") pod \"glance-db-sync-pvgq9\" (UID: \"66f5168a-1ee5-4de9-8e50-90ef25186ee8\") " pod="openstack-kuttl-tests/glance-db-sync-pvgq9" Dec 03 12:20:22 crc kubenswrapper[5052]: I1203 12:20:22.829683 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/cinder-db-sync-gqt2p"] Dec 03 12:20:22 crc kubenswrapper[5052]: I1203 12:20:22.832149 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-db-sync-gqt2p" Dec 03 12:20:22 crc kubenswrapper[5052]: I1203 12:20:22.835085 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/66f5168a-1ee5-4de9-8e50-90ef25186ee8-db-sync-config-data\") pod \"glance-db-sync-pvgq9\" (UID: \"66f5168a-1ee5-4de9-8e50-90ef25186ee8\") " pod="openstack-kuttl-tests/glance-db-sync-pvgq9" Dec 03 12:20:22 crc kubenswrapper[5052]: I1203 12:20:22.836726 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/66f5168a-1ee5-4de9-8e50-90ef25186ee8-combined-ca-bundle\") pod \"glance-db-sync-pvgq9\" (UID: \"66f5168a-1ee5-4de9-8e50-90ef25186ee8\") " pod="openstack-kuttl-tests/glance-db-sync-pvgq9" Dec 03 12:20:22 crc kubenswrapper[5052]: I1203 12:20:22.840644 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/66f5168a-1ee5-4de9-8e50-90ef25186ee8-config-data\") pod \"glance-db-sync-pvgq9\" (UID: \"66f5168a-1ee5-4de9-8e50-90ef25186ee8\") " pod="openstack-kuttl-tests/glance-db-sync-pvgq9" Dec 03 12:20:22 crc kubenswrapper[5052]: I1203 12:20:22.864345 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-wlsk7\" (UniqueName: \"kubernetes.io/projected/66f5168a-1ee5-4de9-8e50-90ef25186ee8-kube-api-access-wlsk7\") pod \"glance-db-sync-pvgq9\" (UID: \"66f5168a-1ee5-4de9-8e50-90ef25186ee8\") " pod="openstack-kuttl-tests/glance-db-sync-pvgq9" Dec 03 12:20:22 crc kubenswrapper[5052]: I1203 12:20:22.872405 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-db-sync-gqt2p"] Dec 03 12:20:22 crc kubenswrapper[5052]: I1203 12:20:22.879429 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"ce16d54b-ef04-429b-8795-2865395f7774","Type":"ContainerStarted","Data":"8531d325ffe575ccabb11f725af2c0fca6093a5c058282b15cfeabec695cc2f0"} Dec 03 12:20:22 crc kubenswrapper[5052]: I1203 12:20:22.879477 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"ce16d54b-ef04-429b-8795-2865395f7774","Type":"ContainerStarted","Data":"dfb12dbb2d7b94c4c5755dc4e198ee9048c925b8d89483a14be0f89b600fa089"} Dec 03 12:20:22 crc kubenswrapper[5052]: I1203 12:20:22.904085 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-sb-0"] Dec 03 12:20:22 crc kubenswrapper[5052]: I1203 12:20:22.904419 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ovsdbserver-sb-0" podUID="06dda089-0745-4941-adfe-2de711fce24e" containerName="openstack-network-exporter" containerID="cri-o://33dc5ae74629ef4a73b20f4631f0f9178eee3dfb4459965f03d5530f56c3e84d" gracePeriod=300 Dec 03 12:20:22 crc kubenswrapper[5052]: I1203 12:20:22.913785 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/c4487b92-6d46-4cde-b1c3-056ac6fa9753-db-sync-config-data\") pod \"cinder-db-sync-gqt2p\" (UID: \"c4487b92-6d46-4cde-b1c3-056ac6fa9753\") " pod="openstack-kuttl-tests/cinder-db-sync-gqt2p" Dec 03 12:20:22 crc kubenswrapper[5052]: I1203 12:20:22.913855 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c4487b92-6d46-4cde-b1c3-056ac6fa9753-combined-ca-bundle\") pod \"cinder-db-sync-gqt2p\" (UID: \"c4487b92-6d46-4cde-b1c3-056ac6fa9753\") " pod="openstack-kuttl-tests/cinder-db-sync-gqt2p" Dec 03 12:20:22 crc kubenswrapper[5052]: I1203 12:20:22.913877 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/c4487b92-6d46-4cde-b1c3-056ac6fa9753-config-data\") pod \"cinder-db-sync-gqt2p\" (UID: \"c4487b92-6d46-4cde-b1c3-056ac6fa9753\") " pod="openstack-kuttl-tests/cinder-db-sync-gqt2p" Dec 03 12:20:22 crc kubenswrapper[5052]: I1203 12:20:22.913916 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/c4487b92-6d46-4cde-b1c3-056ac6fa9753-scripts\") pod \"cinder-db-sync-gqt2p\" (UID: \"c4487b92-6d46-4cde-b1c3-056ac6fa9753\") " pod="openstack-kuttl-tests/cinder-db-sync-gqt2p" Dec 03 12:20:22 crc kubenswrapper[5052]: I1203 12:20:22.913941 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/c4487b92-6d46-4cde-b1c3-056ac6fa9753-etc-machine-id\") pod \"cinder-db-sync-gqt2p\" (UID: \"c4487b92-6d46-4cde-b1c3-056ac6fa9753\") " pod="openstack-kuttl-tests/cinder-db-sync-gqt2p" Dec 03 12:20:22 crc kubenswrapper[5052]: I1203 12:20:22.913974 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bplcb\" (UniqueName: \"kubernetes.io/projected/c4487b92-6d46-4cde-b1c3-056ac6fa9753-kube-api-access-bplcb\") pod \"cinder-db-sync-gqt2p\" (UID: \"c4487b92-6d46-4cde-b1c3-056ac6fa9753\") " pod="openstack-kuttl-tests/cinder-db-sync-gqt2p" Dec 03 12:20:22 crc kubenswrapper[5052]: I1203 12:20:22.964628 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/neutron-db-sync-kgtt9"] Dec 03 12:20:22 crc kubenswrapper[5052]: I1203 12:20:22.966045 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-db-sync-pvgq9" Dec 03 12:20:23 crc kubenswrapper[5052]: I1203 12:20:23.016638 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-bplcb\" (UniqueName: \"kubernetes.io/projected/c4487b92-6d46-4cde-b1c3-056ac6fa9753-kube-api-access-bplcb\") pod \"cinder-db-sync-gqt2p\" (UID: \"c4487b92-6d46-4cde-b1c3-056ac6fa9753\") " pod="openstack-kuttl-tests/cinder-db-sync-gqt2p" Dec 03 12:20:23 crc kubenswrapper[5052]: I1203 12:20:23.016766 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/c4487b92-6d46-4cde-b1c3-056ac6fa9753-db-sync-config-data\") pod \"cinder-db-sync-gqt2p\" (UID: \"c4487b92-6d46-4cde-b1c3-056ac6fa9753\") " pod="openstack-kuttl-tests/cinder-db-sync-gqt2p" Dec 03 12:20:23 crc kubenswrapper[5052]: I1203 12:20:23.016851 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c4487b92-6d46-4cde-b1c3-056ac6fa9753-combined-ca-bundle\") pod \"cinder-db-sync-gqt2p\" (UID: \"c4487b92-6d46-4cde-b1c3-056ac6fa9753\") " pod="openstack-kuttl-tests/cinder-db-sync-gqt2p" Dec 03 12:20:23 crc kubenswrapper[5052]: I1203 12:20:23.016881 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/c4487b92-6d46-4cde-b1c3-056ac6fa9753-config-data\") pod \"cinder-db-sync-gqt2p\" (UID: \"c4487b92-6d46-4cde-b1c3-056ac6fa9753\") " pod="openstack-kuttl-tests/cinder-db-sync-gqt2p" Dec 03 12:20:23 crc kubenswrapper[5052]: I1203 12:20:23.016928 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/c4487b92-6d46-4cde-b1c3-056ac6fa9753-scripts\") pod \"cinder-db-sync-gqt2p\" (UID: \"c4487b92-6d46-4cde-b1c3-056ac6fa9753\") " pod="openstack-kuttl-tests/cinder-db-sync-gqt2p" Dec 03 12:20:23 crc kubenswrapper[5052]: I1203 12:20:23.016959 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/c4487b92-6d46-4cde-b1c3-056ac6fa9753-etc-machine-id\") pod \"cinder-db-sync-gqt2p\" (UID: \"c4487b92-6d46-4cde-b1c3-056ac6fa9753\") " pod="openstack-kuttl-tests/cinder-db-sync-gqt2p" Dec 03 12:20:23 crc kubenswrapper[5052]: I1203 12:20:23.017108 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/c4487b92-6d46-4cde-b1c3-056ac6fa9753-etc-machine-id\") pod \"cinder-db-sync-gqt2p\" (UID: \"c4487b92-6d46-4cde-b1c3-056ac6fa9753\") " pod="openstack-kuttl-tests/cinder-db-sync-gqt2p" Dec 03 12:20:23 crc kubenswrapper[5052]: I1203 12:20:23.023185 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/c4487b92-6d46-4cde-b1c3-056ac6fa9753-db-sync-config-data\") pod \"cinder-db-sync-gqt2p\" (UID: \"c4487b92-6d46-4cde-b1c3-056ac6fa9753\") " pod="openstack-kuttl-tests/cinder-db-sync-gqt2p" Dec 03 12:20:23 crc kubenswrapper[5052]: I1203 12:20:23.044528 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/c4487b92-6d46-4cde-b1c3-056ac6fa9753-config-data\") pod \"cinder-db-sync-gqt2p\" (UID: \"c4487b92-6d46-4cde-b1c3-056ac6fa9753\") " pod="openstack-kuttl-tests/cinder-db-sync-gqt2p" Dec 03 12:20:23 crc kubenswrapper[5052]: I1203 12:20:23.047368 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/neutron-db-sync-kgtt9"] Dec 03 12:20:23 crc kubenswrapper[5052]: I1203 12:20:23.049076 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/c4487b92-6d46-4cde-b1c3-056ac6fa9753-scripts\") pod \"cinder-db-sync-gqt2p\" (UID: \"c4487b92-6d46-4cde-b1c3-056ac6fa9753\") " pod="openstack-kuttl-tests/cinder-db-sync-gqt2p" Dec 03 12:20:23 crc kubenswrapper[5052]: I1203 12:20:23.065751 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c4487b92-6d46-4cde-b1c3-056ac6fa9753-combined-ca-bundle\") pod \"cinder-db-sync-gqt2p\" (UID: \"c4487b92-6d46-4cde-b1c3-056ac6fa9753\") " pod="openstack-kuttl-tests/cinder-db-sync-gqt2p" Dec 03 12:20:23 crc kubenswrapper[5052]: I1203 12:20:23.161845 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-bplcb\" (UniqueName: \"kubernetes.io/projected/c4487b92-6d46-4cde-b1c3-056ac6fa9753-kube-api-access-bplcb\") pod \"cinder-db-sync-gqt2p\" (UID: \"c4487b92-6d46-4cde-b1c3-056ac6fa9753\") " pod="openstack-kuttl-tests/cinder-db-sync-gqt2p" Dec 03 12:20:23 crc kubenswrapper[5052]: I1203 12:20:23.184943 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-db-sync-gqt2p" Dec 03 12:20:23 crc kubenswrapper[5052]: I1203 12:20:23.264899 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/neutron-db-sync-lrqq6"] Dec 03 12:20:23 crc kubenswrapper[5052]: I1203 12:20:23.266392 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-db-sync-lrqq6" Dec 03 12:20:23 crc kubenswrapper[5052]: I1203 12:20:23.316839 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-db-sync-lrqq6"] Dec 03 12:20:23 crc kubenswrapper[5052]: I1203 12:20:23.331546 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/94ac1a3b-14db-445c-8fb8-8bc3069bdb57-combined-ca-bundle\") pod \"neutron-db-sync-lrqq6\" (UID: \"94ac1a3b-14db-445c-8fb8-8bc3069bdb57\") " pod="openstack-kuttl-tests/neutron-db-sync-lrqq6" Dec 03 12:20:23 crc kubenswrapper[5052]: I1203 12:20:23.331598 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/94ac1a3b-14db-445c-8fb8-8bc3069bdb57-config\") pod \"neutron-db-sync-lrqq6\" (UID: \"94ac1a3b-14db-445c-8fb8-8bc3069bdb57\") " pod="openstack-kuttl-tests/neutron-db-sync-lrqq6" Dec 03 12:20:23 crc kubenswrapper[5052]: I1203 12:20:23.331660 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b4j7k\" (UniqueName: \"kubernetes.io/projected/94ac1a3b-14db-445c-8fb8-8bc3069bdb57-kube-api-access-b4j7k\") pod \"neutron-db-sync-lrqq6\" (UID: \"94ac1a3b-14db-445c-8fb8-8bc3069bdb57\") " pod="openstack-kuttl-tests/neutron-db-sync-lrqq6" Dec 03 12:20:23 crc kubenswrapper[5052]: I1203 12:20:23.340929 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ovsdbserver-sb-0" podUID="06dda089-0745-4941-adfe-2de711fce24e" containerName="ovsdbserver-sb" containerID="cri-o://4ce7e130ceeb669b4057b5b4f5937db4eabc9ded0e28a82765f352a1735b5239" gracePeriod=300 Dec 03 12:20:23 crc kubenswrapper[5052]: I1203 12:20:23.374281 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:20:23 crc kubenswrapper[5052]: I1203 12:20:23.433303 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/94ac1a3b-14db-445c-8fb8-8bc3069bdb57-combined-ca-bundle\") pod \"neutron-db-sync-lrqq6\" (UID: \"94ac1a3b-14db-445c-8fb8-8bc3069bdb57\") " pod="openstack-kuttl-tests/neutron-db-sync-lrqq6" Dec 03 12:20:23 crc kubenswrapper[5052]: I1203 12:20:23.433674 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/94ac1a3b-14db-445c-8fb8-8bc3069bdb57-config\") pod \"neutron-db-sync-lrqq6\" (UID: \"94ac1a3b-14db-445c-8fb8-8bc3069bdb57\") " pod="openstack-kuttl-tests/neutron-db-sync-lrqq6" Dec 03 12:20:23 crc kubenswrapper[5052]: I1203 12:20:23.433795 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-b4j7k\" (UniqueName: \"kubernetes.io/projected/94ac1a3b-14db-445c-8fb8-8bc3069bdb57-kube-api-access-b4j7k\") pod \"neutron-db-sync-lrqq6\" (UID: \"94ac1a3b-14db-445c-8fb8-8bc3069bdb57\") " pod="openstack-kuttl-tests/neutron-db-sync-lrqq6" Dec 03 12:20:23 crc kubenswrapper[5052]: W1203 12:20:23.434887 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podae23fe34_7bf4_47fc_addc_5edda0399c77.slice/crio-b5504923aab1fd3aa9e6334f88826548fb2609a9f099f5f7ca6488b1e5ce9a6a WatchSource:0}: Error finding container b5504923aab1fd3aa9e6334f88826548fb2609a9f099f5f7ca6488b1e5ce9a6a: Status 404 returned error can't find the container with id b5504923aab1fd3aa9e6334f88826548fb2609a9f099f5f7ca6488b1e5ce9a6a Dec 03 12:20:23 crc kubenswrapper[5052]: I1203 12:20:23.473038 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/94ac1a3b-14db-445c-8fb8-8bc3069bdb57-config\") pod \"neutron-db-sync-lrqq6\" (UID: \"94ac1a3b-14db-445c-8fb8-8bc3069bdb57\") " pod="openstack-kuttl-tests/neutron-db-sync-lrqq6" Dec 03 12:20:23 crc kubenswrapper[5052]: I1203 12:20:23.473109 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/94ac1a3b-14db-445c-8fb8-8bc3069bdb57-combined-ca-bundle\") pod \"neutron-db-sync-lrqq6\" (UID: \"94ac1a3b-14db-445c-8fb8-8bc3069bdb57\") " pod="openstack-kuttl-tests/neutron-db-sync-lrqq6" Dec 03 12:20:23 crc kubenswrapper[5052]: I1203 12:20:23.479025 5052 provider.go:102] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Dec 03 12:20:23 crc kubenswrapper[5052]: I1203 12:20:23.491504 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-b4j7k\" (UniqueName: \"kubernetes.io/projected/94ac1a3b-14db-445c-8fb8-8bc3069bdb57-kube-api-access-b4j7k\") pod \"neutron-db-sync-lrqq6\" (UID: \"94ac1a3b-14db-445c-8fb8-8bc3069bdb57\") " pod="openstack-kuttl-tests/neutron-db-sync-lrqq6" Dec 03 12:20:23 crc kubenswrapper[5052]: I1203 12:20:23.517835 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/placement-db-sync-gzzkp"] Dec 03 12:20:23 crc kubenswrapper[5052]: I1203 12:20:23.555771 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 12:20:23 crc kubenswrapper[5052]: I1203 12:20:23.556063 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" podUID="42ebb0e3-9a37-4898-a1e4-0822b4bd9e1e" containerName="nova-cell1-novncproxy-novncproxy" containerID="cri-o://1a78431f63106b64cc7a1e9a2171a9a34a9537578480657d5f058f3cfe1f49bd" gracePeriod=30 Dec 03 12:20:23 crc kubenswrapper[5052]: I1203 12:20:23.577242 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-db-sync-pnhqf"] Dec 03 12:20:23 crc kubenswrapper[5052]: I1203 12:20:23.609684 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/barbican-db-sync-pnhqf"] Dec 03 12:20:23 crc kubenswrapper[5052]: I1203 12:20:23.659160 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-db-sync-9jf9q"] Dec 03 12:20:23 crc kubenswrapper[5052]: I1203 12:20:23.664695 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/openstack-cell1-galera-0" podUID="a56992d9-4662-444f-8a45-0375d0855946" containerName="galera" containerID="cri-o://8dfc8c0fb51ed3361082e5d6a21d3480abe509b00c224f35e46b8ddb4b3e7415" gracePeriod=29 Dec 03 12:20:23 crc kubenswrapper[5052]: I1203 12:20:23.669208 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/openstack-galera-0" podUID="5067bb8d-d668-46a6-b9de-8065266cde63" containerName="galera" containerID="cri-o://7c40f1aa68ee0e0f29ce1ab35070e6d55c530a232bd4bf5a962f9aa5aa656271" gracePeriod=28 Dec 03 12:20:23 crc kubenswrapper[5052]: I1203 12:20:23.679590 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-db-sync-lrqq6" Dec 03 12:20:23 crc kubenswrapper[5052]: I1203 12:20:23.707235 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2f98071b-1e7d-4cf3-976e-ad9073a2f914" path="/var/lib/kubelet/pods/2f98071b-1e7d-4cf3-976e-ad9073a2f914/volumes" Dec 03 12:20:23 crc kubenswrapper[5052]: I1203 12:20:23.708213 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="33b07d8d-aae0-436c-831b-aa13ceff481b" path="/var/lib/kubelet/pods/33b07d8d-aae0-436c-831b-aa13ceff481b/volumes" Dec 03 12:20:23 crc kubenswrapper[5052]: I1203 12:20:23.717111 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9d4a23eb-93e4-4fa3-a10a-4317c891692f" path="/var/lib/kubelet/pods/9d4a23eb-93e4-4fa3-a10a-4317c891692f/volumes" Dec 03 12:20:23 crc kubenswrapper[5052]: I1203 12:20:23.717718 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9e9cf358-eca4-4774-b1a1-1667774e900f" path="/var/lib/kubelet/pods/9e9cf358-eca4-4774-b1a1-1667774e900f/volumes" Dec 03 12:20:23 crc kubenswrapper[5052]: I1203 12:20:23.724996 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9f563ce4-a788-44eb-bdab-9fc4c3423176" path="/var/lib/kubelet/pods/9f563ce4-a788-44eb-bdab-9fc4c3423176/volumes" Dec 03 12:20:23 crc kubenswrapper[5052]: I1203 12:20:23.725833 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cb69fca5-361d-4273-bee8-27c54648309c" path="/var/lib/kubelet/pods/cb69fca5-361d-4273-bee8-27c54648309c/volumes" Dec 03 12:20:23 crc kubenswrapper[5052]: I1203 12:20:23.726374 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e5091121-ce68-4b86-baae-2fbd5b22d537" path="/var/lib/kubelet/pods/e5091121-ce68-4b86-baae-2fbd5b22d537/volumes" Dec 03 12:20:23 crc kubenswrapper[5052]: I1203 12:20:23.727096 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/barbican-db-sync-92z8b"] Dec 03 12:20:23 crc kubenswrapper[5052]: I1203 12:20:23.728672 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-db-sync-92z8b" Dec 03 12:20:23 crc kubenswrapper[5052]: I1203 12:20:23.738736 5052 prober.go:107] "Probe failed" probeType="Liveness" pod="openstack-kuttl-tests/cinder-scheduler-0" podUID="d5750ef1-ce21-4092-9612-13d6f6c923df" containerName="cinder-scheduler" probeResult="failure" output="HTTP probe failed with statuscode: 500" Dec 03 12:20:23 crc kubenswrapper[5052]: I1203 12:20:23.747466 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/7a9497b1-89e2-4ebc-97e6-2bdaeefab95f-combined-ca-bundle\") pod \"barbican-db-sync-92z8b\" (UID: \"7a9497b1-89e2-4ebc-97e6-2bdaeefab95f\") " pod="openstack-kuttl-tests/barbican-db-sync-92z8b" Dec 03 12:20:23 crc kubenswrapper[5052]: I1203 12:20:23.747589 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/7a9497b1-89e2-4ebc-97e6-2bdaeefab95f-db-sync-config-data\") pod \"barbican-db-sync-92z8b\" (UID: \"7a9497b1-89e2-4ebc-97e6-2bdaeefab95f\") " pod="openstack-kuttl-tests/barbican-db-sync-92z8b" Dec 03 12:20:23 crc kubenswrapper[5052]: I1203 12:20:23.747663 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dkl27\" (UniqueName: \"kubernetes.io/projected/7a9497b1-89e2-4ebc-97e6-2bdaeefab95f-kube-api-access-dkl27\") pod \"barbican-db-sync-92z8b\" (UID: \"7a9497b1-89e2-4ebc-97e6-2bdaeefab95f\") " pod="openstack-kuttl-tests/barbican-db-sync-92z8b" Dec 03 12:20:23 crc kubenswrapper[5052]: I1203 12:20:23.789653 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-db-sync-92z8b"] Dec 03 12:20:23 crc kubenswrapper[5052]: I1203 12:20:23.849280 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-dkl27\" (UniqueName: \"kubernetes.io/projected/7a9497b1-89e2-4ebc-97e6-2bdaeefab95f-kube-api-access-dkl27\") pod \"barbican-db-sync-92z8b\" (UID: \"7a9497b1-89e2-4ebc-97e6-2bdaeefab95f\") " pod="openstack-kuttl-tests/barbican-db-sync-92z8b" Dec 03 12:20:23 crc kubenswrapper[5052]: I1203 12:20:23.849399 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/7a9497b1-89e2-4ebc-97e6-2bdaeefab95f-combined-ca-bundle\") pod \"barbican-db-sync-92z8b\" (UID: \"7a9497b1-89e2-4ebc-97e6-2bdaeefab95f\") " pod="openstack-kuttl-tests/barbican-db-sync-92z8b" Dec 03 12:20:23 crc kubenswrapper[5052]: I1203 12:20:23.849475 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/7a9497b1-89e2-4ebc-97e6-2bdaeefab95f-db-sync-config-data\") pod \"barbican-db-sync-92z8b\" (UID: \"7a9497b1-89e2-4ebc-97e6-2bdaeefab95f\") " pod="openstack-kuttl-tests/barbican-db-sync-92z8b" Dec 03 12:20:23 crc kubenswrapper[5052]: I1203 12:20:23.861602 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/swift-ring-rebalance-shz9b"] Dec 03 12:20:23 crc kubenswrapper[5052]: I1203 12:20:23.865655 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/7a9497b1-89e2-4ebc-97e6-2bdaeefab95f-combined-ca-bundle\") pod \"barbican-db-sync-92z8b\" (UID: \"7a9497b1-89e2-4ebc-97e6-2bdaeefab95f\") " pod="openstack-kuttl-tests/barbican-db-sync-92z8b" Dec 03 12:20:23 crc kubenswrapper[5052]: I1203 12:20:23.866503 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/7a9497b1-89e2-4ebc-97e6-2bdaeefab95f-db-sync-config-data\") pod \"barbican-db-sync-92z8b\" (UID: \"7a9497b1-89e2-4ebc-97e6-2bdaeefab95f\") " pod="openstack-kuttl-tests/barbican-db-sync-92z8b" Dec 03 12:20:23 crc kubenswrapper[5052]: I1203 12:20:23.915179 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-dkl27\" (UniqueName: \"kubernetes.io/projected/7a9497b1-89e2-4ebc-97e6-2bdaeefab95f-kube-api-access-dkl27\") pod \"barbican-db-sync-92z8b\" (UID: \"7a9497b1-89e2-4ebc-97e6-2bdaeefab95f\") " pod="openstack-kuttl-tests/barbican-db-sync-92z8b" Dec 03 12:20:23 crc kubenswrapper[5052]: I1203 12:20:23.941498 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-db-sync-92z8b" Dec 03 12:20:24 crc kubenswrapper[5052]: I1203 12:20:24.049928 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/swift-ring-rebalance-shz9b"] Dec 03 12:20:24 crc kubenswrapper[5052]: I1203 12:20:24.103609 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-db-sync-9jf9q" event={"ID":"b852a7d7-8510-440d-8e87-0755fac2a324","Type":"ContainerStarted","Data":"2b94be6b43835c8d9230bf06afb4a758df81f4bb7a901387bd220d7526304993"} Dec 03 12:20:24 crc kubenswrapper[5052]: I1203 12:20:24.137861 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/swift-ring-rebalance-z7lpm"] Dec 03 12:20:24 crc kubenswrapper[5052]: I1203 12:20:24.139887 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-ring-rebalance-z7lpm" Dec 03 12:20:24 crc kubenswrapper[5052]: I1203 12:20:24.145044 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"swift-ring-config-data" Dec 03 12:20:24 crc kubenswrapper[5052]: I1203 12:20:24.145594 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"swift-ring-scripts" Dec 03 12:20:24 crc kubenswrapper[5052]: I1203 12:20:24.165118 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_ovsdbserver-sb-0_06dda089-0745-4941-adfe-2de711fce24e/ovsdbserver-sb/0.log" Dec 03 12:20:24 crc kubenswrapper[5052]: I1203 12:20:24.165180 5052 generic.go:334] "Generic (PLEG): container finished" podID="06dda089-0745-4941-adfe-2de711fce24e" containerID="33dc5ae74629ef4a73b20f4631f0f9178eee3dfb4459965f03d5530f56c3e84d" exitCode=2 Dec 03 12:20:24 crc kubenswrapper[5052]: I1203 12:20:24.165199 5052 generic.go:334] "Generic (PLEG): container finished" podID="06dda089-0745-4941-adfe-2de711fce24e" containerID="4ce7e130ceeb669b4057b5b4f5937db4eabc9ded0e28a82765f352a1735b5239" exitCode=143 Dec 03 12:20:24 crc kubenswrapper[5052]: I1203 12:20:24.165249 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-sb-0" event={"ID":"06dda089-0745-4941-adfe-2de711fce24e","Type":"ContainerDied","Data":"33dc5ae74629ef4a73b20f4631f0f9178eee3dfb4459965f03d5530f56c3e84d"} Dec 03 12:20:24 crc kubenswrapper[5052]: I1203 12:20:24.165283 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-sb-0" event={"ID":"06dda089-0745-4941-adfe-2de711fce24e","Type":"ContainerDied","Data":"4ce7e130ceeb669b4057b5b4f5937db4eabc9ded0e28a82765f352a1735b5239"} Dec 03 12:20:24 crc kubenswrapper[5052]: I1203 12:20:24.177702 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"ae23fe34-7bf4-47fc-addc-5edda0399c77","Type":"ContainerStarted","Data":"b5504923aab1fd3aa9e6334f88826548fb2609a9f099f5f7ca6488b1e5ce9a6a"} Dec 03 12:20:24 crc kubenswrapper[5052]: I1203 12:20:24.179686 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/swift-ring-rebalance-z7lpm"] Dec 03 12:20:24 crc kubenswrapper[5052]: I1203 12:20:24.181111 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-db-sync-gzzkp" event={"ID":"50ae31b2-a1c0-4afb-8e23-d7856c55af1d","Type":"ContainerStarted","Data":"dabc6c08dd8bb953a7aa6c3562c635fe665e3ba4b101b7c62fb03488e221a8b4"} Dec 03 12:20:24 crc kubenswrapper[5052]: I1203 12:20:24.195476 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_ovsdbserver-nb-0_fcd60977-d995-42ee-b654-ba2947a3316d/ovsdbserver-nb/0.log" Dec 03 12:20:24 crc kubenswrapper[5052]: I1203 12:20:24.195560 5052 generic.go:334] "Generic (PLEG): container finished" podID="fcd60977-d995-42ee-b654-ba2947a3316d" containerID="daa77eec7322cd2c65ca8b891da5191f57aec462539227515b3efcddb082df4d" exitCode=2 Dec 03 12:20:24 crc kubenswrapper[5052]: I1203 12:20:24.195582 5052 generic.go:334] "Generic (PLEG): container finished" podID="fcd60977-d995-42ee-b654-ba2947a3316d" containerID="87154b4a2a5069ca41b7570b52ae76e123d9dca6d0b7110267bc2f70b48cd720" exitCode=143 Dec 03 12:20:24 crc kubenswrapper[5052]: I1203 12:20:24.195632 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-nb-0" event={"ID":"fcd60977-d995-42ee-b654-ba2947a3316d","Type":"ContainerDied","Data":"daa77eec7322cd2c65ca8b891da5191f57aec462539227515b3efcddb082df4d"} Dec 03 12:20:24 crc kubenswrapper[5052]: I1203 12:20:24.195664 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-nb-0" event={"ID":"fcd60977-d995-42ee-b654-ba2947a3316d","Type":"ContainerDied","Data":"87154b4a2a5069ca41b7570b52ae76e123d9dca6d0b7110267bc2f70b48cd720"} Dec 03 12:20:24 crc kubenswrapper[5052]: I1203 12:20:24.211956 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/swift-storage-0"] Dec 03 12:20:24 crc kubenswrapper[5052]: I1203 12:20:24.212385 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="335ad996-ad3b-4ac8-a488-64e9453f24a4" containerName="account-server" containerID="cri-o://006d1e418590929322462448d037f47e0e4031553a716f0f323b0ceb29ae6857" gracePeriod=30 Dec 03 12:20:24 crc kubenswrapper[5052]: I1203 12:20:24.212823 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="335ad996-ad3b-4ac8-a488-64e9453f24a4" containerName="swift-recon-cron" containerID="cri-o://fb2b07929be7f2dd4d6fd56e12955638b93484338cfedea07147fa092b946832" gracePeriod=30 Dec 03 12:20:24 crc kubenswrapper[5052]: I1203 12:20:24.212886 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="335ad996-ad3b-4ac8-a488-64e9453f24a4" containerName="rsync" containerID="cri-o://9ca86e437e38c5642f8640cedce80dfac23a56655784a6b7769c0317c467d085" gracePeriod=30 Dec 03 12:20:24 crc kubenswrapper[5052]: I1203 12:20:24.212918 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="335ad996-ad3b-4ac8-a488-64e9453f24a4" containerName="object-expirer" containerID="cri-o://2660887ea0a6145f5154899fe6bb77bb7f4491a21de6c7447f3c126adc776305" gracePeriod=30 Dec 03 12:20:24 crc kubenswrapper[5052]: I1203 12:20:24.212949 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="335ad996-ad3b-4ac8-a488-64e9453f24a4" containerName="object-updater" containerID="cri-o://486c4efda615830f80b3292087835a1464d89e6438a42bab394e0f23416c0683" gracePeriod=30 Dec 03 12:20:24 crc kubenswrapper[5052]: I1203 12:20:24.212979 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="335ad996-ad3b-4ac8-a488-64e9453f24a4" containerName="object-auditor" containerID="cri-o://2b57d07d8ebd7db5a1f0aa4b4ebb75c36047390ad674b3a41f27874f7e01c7b5" gracePeriod=30 Dec 03 12:20:24 crc kubenswrapper[5052]: I1203 12:20:24.213008 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="335ad996-ad3b-4ac8-a488-64e9453f24a4" containerName="object-replicator" containerID="cri-o://2d0cbf82dccb7f04f14e9a8c02dce57f3fe56aa4bea37bafafd0b0e8ef2825ff" gracePeriod=30 Dec 03 12:20:24 crc kubenswrapper[5052]: I1203 12:20:24.213039 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="335ad996-ad3b-4ac8-a488-64e9453f24a4" containerName="object-server" containerID="cri-o://2591ce792243cc1fe842d92ce9a53f946994c1f42c26f9657e9e29db8d53f707" gracePeriod=30 Dec 03 12:20:24 crc kubenswrapper[5052]: I1203 12:20:24.213079 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="335ad996-ad3b-4ac8-a488-64e9453f24a4" containerName="account-reaper" containerID="cri-o://f7f12231dd030c2c9c3f43d8493e609bd58faa9dd7c14aed345ea597010d1451" gracePeriod=30 Dec 03 12:20:24 crc kubenswrapper[5052]: I1203 12:20:24.213111 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="335ad996-ad3b-4ac8-a488-64e9453f24a4" containerName="account-auditor" containerID="cri-o://d801a1f1f45c17b5422af1ea12c95b38c64a258c49f3953222cbb94387e19d2c" gracePeriod=30 Dec 03 12:20:24 crc kubenswrapper[5052]: I1203 12:20:24.213091 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="335ad996-ad3b-4ac8-a488-64e9453f24a4" containerName="container-server" containerID="cri-o://6af612b56d9c2f4b731475f24619368f730783d7ef872ccc130c33f5a189e57f" gracePeriod=30 Dec 03 12:20:24 crc kubenswrapper[5052]: I1203 12:20:24.213147 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="335ad996-ad3b-4ac8-a488-64e9453f24a4" containerName="account-replicator" containerID="cri-o://df73d6886a1afb1e6d09c9481252bc2871f9a8608386e4ea20b13852f0881a4b" gracePeriod=30 Dec 03 12:20:24 crc kubenswrapper[5052]: I1203 12:20:24.213196 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="335ad996-ad3b-4ac8-a488-64e9453f24a4" containerName="container-updater" containerID="cri-o://55c422ce2a089bbc2d4cce09c586d3b79a3abf94fbea99b43ef8ff9d803269d4" gracePeriod=30 Dec 03 12:20:24 crc kubenswrapper[5052]: I1203 12:20:24.213241 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="335ad996-ad3b-4ac8-a488-64e9453f24a4" containerName="container-auditor" containerID="cri-o://000974bff3024c4dd8ae265f8fc3dc2cd023ab0097daaa4fd6b13c365cb5dd62" gracePeriod=30 Dec 03 12:20:24 crc kubenswrapper[5052]: I1203 12:20:24.213273 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="335ad996-ad3b-4ac8-a488-64e9453f24a4" containerName="container-replicator" containerID="cri-o://e1a2b505ab3bea37930a1c222d85a3ab5421377ce504e97bef52966f349f5e26" gracePeriod=30 Dec 03 12:20:24 crc kubenswrapper[5052]: I1203 12:20:24.242946 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_ovn-northd-0_966185e5-f414-46c5-ba48-0e234750795a/ovn-northd/0.log" Dec 03 12:20:24 crc kubenswrapper[5052]: I1203 12:20:24.242993 5052 generic.go:334] "Generic (PLEG): container finished" podID="966185e5-f414-46c5-ba48-0e234750795a" containerID="da2e79425aa667a3b6c86223531989383a59d7905be7837190a7bae4c94575bc" exitCode=2 Dec 03 12:20:24 crc kubenswrapper[5052]: I1203 12:20:24.243031 5052 generic.go:334] "Generic (PLEG): container finished" podID="966185e5-f414-46c5-ba48-0e234750795a" containerID="ef532f5fb83b3ed8c9243e5c20875e68a9ee6b646ee961f30ffe04c5b504e0f8" exitCode=143 Dec 03 12:20:24 crc kubenswrapper[5052]: I1203 12:20:24.243055 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovn-northd-0" event={"ID":"966185e5-f414-46c5-ba48-0e234750795a","Type":"ContainerDied","Data":"da2e79425aa667a3b6c86223531989383a59d7905be7837190a7bae4c94575bc"} Dec 03 12:20:24 crc kubenswrapper[5052]: I1203 12:20:24.243096 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovn-northd-0" event={"ID":"966185e5-f414-46c5-ba48-0e234750795a","Type":"ContainerDied","Data":"ef532f5fb83b3ed8c9243e5c20875e68a9ee6b646ee961f30ffe04c5b504e0f8"} Dec 03 12:20:24 crc kubenswrapper[5052]: I1203 12:20:24.262657 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ring-data-devices\" (UniqueName: \"kubernetes.io/configmap/318545bf-5895-48f9-9649-67207bc4b207-ring-data-devices\") pod \"swift-ring-rebalance-z7lpm\" (UID: \"318545bf-5895-48f9-9649-67207bc4b207\") " pod="openstack-kuttl-tests/swift-ring-rebalance-z7lpm" Dec 03 12:20:24 crc kubenswrapper[5052]: I1203 12:20:24.262699 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dispersionconf\" (UniqueName: \"kubernetes.io/secret/318545bf-5895-48f9-9649-67207bc4b207-dispersionconf\") pod \"swift-ring-rebalance-z7lpm\" (UID: \"318545bf-5895-48f9-9649-67207bc4b207\") " pod="openstack-kuttl-tests/swift-ring-rebalance-z7lpm" Dec 03 12:20:24 crc kubenswrapper[5052]: I1203 12:20:24.262747 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"swiftconf\" (UniqueName: \"kubernetes.io/secret/318545bf-5895-48f9-9649-67207bc4b207-swiftconf\") pod \"swift-ring-rebalance-z7lpm\" (UID: \"318545bf-5895-48f9-9649-67207bc4b207\") " pod="openstack-kuttl-tests/swift-ring-rebalance-z7lpm" Dec 03 12:20:24 crc kubenswrapper[5052]: I1203 12:20:24.262765 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/empty-dir/318545bf-5895-48f9-9649-67207bc4b207-etc-swift\") pod \"swift-ring-rebalance-z7lpm\" (UID: \"318545bf-5895-48f9-9649-67207bc4b207\") " pod="openstack-kuttl-tests/swift-ring-rebalance-z7lpm" Dec 03 12:20:24 crc kubenswrapper[5052]: I1203 12:20:24.262837 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pn692\" (UniqueName: \"kubernetes.io/projected/318545bf-5895-48f9-9649-67207bc4b207-kube-api-access-pn692\") pod \"swift-ring-rebalance-z7lpm\" (UID: \"318545bf-5895-48f9-9649-67207bc4b207\") " pod="openstack-kuttl-tests/swift-ring-rebalance-z7lpm" Dec 03 12:20:24 crc kubenswrapper[5052]: I1203 12:20:24.262873 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/318545bf-5895-48f9-9649-67207bc4b207-combined-ca-bundle\") pod \"swift-ring-rebalance-z7lpm\" (UID: \"318545bf-5895-48f9-9649-67207bc4b207\") " pod="openstack-kuttl-tests/swift-ring-rebalance-z7lpm" Dec 03 12:20:24 crc kubenswrapper[5052]: I1203 12:20:24.262906 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/318545bf-5895-48f9-9649-67207bc4b207-scripts\") pod \"swift-ring-rebalance-z7lpm\" (UID: \"318545bf-5895-48f9-9649-67207bc4b207\") " pod="openstack-kuttl-tests/swift-ring-rebalance-z7lpm" Dec 03 12:20:24 crc kubenswrapper[5052]: I1203 12:20:24.364256 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/318545bf-5895-48f9-9649-67207bc4b207-combined-ca-bundle\") pod \"swift-ring-rebalance-z7lpm\" (UID: \"318545bf-5895-48f9-9649-67207bc4b207\") " pod="openstack-kuttl-tests/swift-ring-rebalance-z7lpm" Dec 03 12:20:24 crc kubenswrapper[5052]: I1203 12:20:24.364320 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/318545bf-5895-48f9-9649-67207bc4b207-scripts\") pod \"swift-ring-rebalance-z7lpm\" (UID: \"318545bf-5895-48f9-9649-67207bc4b207\") " pod="openstack-kuttl-tests/swift-ring-rebalance-z7lpm" Dec 03 12:20:24 crc kubenswrapper[5052]: I1203 12:20:24.364382 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ring-data-devices\" (UniqueName: \"kubernetes.io/configmap/318545bf-5895-48f9-9649-67207bc4b207-ring-data-devices\") pod \"swift-ring-rebalance-z7lpm\" (UID: \"318545bf-5895-48f9-9649-67207bc4b207\") " pod="openstack-kuttl-tests/swift-ring-rebalance-z7lpm" Dec 03 12:20:24 crc kubenswrapper[5052]: I1203 12:20:24.364401 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"dispersionconf\" (UniqueName: \"kubernetes.io/secret/318545bf-5895-48f9-9649-67207bc4b207-dispersionconf\") pod \"swift-ring-rebalance-z7lpm\" (UID: \"318545bf-5895-48f9-9649-67207bc4b207\") " pod="openstack-kuttl-tests/swift-ring-rebalance-z7lpm" Dec 03 12:20:24 crc kubenswrapper[5052]: I1203 12:20:24.364441 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"swiftconf\" (UniqueName: \"kubernetes.io/secret/318545bf-5895-48f9-9649-67207bc4b207-swiftconf\") pod \"swift-ring-rebalance-z7lpm\" (UID: \"318545bf-5895-48f9-9649-67207bc4b207\") " pod="openstack-kuttl-tests/swift-ring-rebalance-z7lpm" Dec 03 12:20:24 crc kubenswrapper[5052]: I1203 12:20:24.364459 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/empty-dir/318545bf-5895-48f9-9649-67207bc4b207-etc-swift\") pod \"swift-ring-rebalance-z7lpm\" (UID: \"318545bf-5895-48f9-9649-67207bc4b207\") " pod="openstack-kuttl-tests/swift-ring-rebalance-z7lpm" Dec 03 12:20:24 crc kubenswrapper[5052]: I1203 12:20:24.364559 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-pn692\" (UniqueName: \"kubernetes.io/projected/318545bf-5895-48f9-9649-67207bc4b207-kube-api-access-pn692\") pod \"swift-ring-rebalance-z7lpm\" (UID: \"318545bf-5895-48f9-9649-67207bc4b207\") " pod="openstack-kuttl-tests/swift-ring-rebalance-z7lpm" Dec 03 12:20:24 crc kubenswrapper[5052]: I1203 12:20:24.371346 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/318545bf-5895-48f9-9649-67207bc4b207-scripts\") pod \"swift-ring-rebalance-z7lpm\" (UID: \"318545bf-5895-48f9-9649-67207bc4b207\") " pod="openstack-kuttl-tests/swift-ring-rebalance-z7lpm" Dec 03 12:20:24 crc kubenswrapper[5052]: I1203 12:20:24.371822 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ring-data-devices\" (UniqueName: \"kubernetes.io/configmap/318545bf-5895-48f9-9649-67207bc4b207-ring-data-devices\") pod \"swift-ring-rebalance-z7lpm\" (UID: \"318545bf-5895-48f9-9649-67207bc4b207\") " pod="openstack-kuttl-tests/swift-ring-rebalance-z7lpm" Dec 03 12:20:24 crc kubenswrapper[5052]: I1203 12:20:24.372034 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"etc-swift\" (UniqueName: \"kubernetes.io/empty-dir/318545bf-5895-48f9-9649-67207bc4b207-etc-swift\") pod \"swift-ring-rebalance-z7lpm\" (UID: \"318545bf-5895-48f9-9649-67207bc4b207\") " pod="openstack-kuttl-tests/swift-ring-rebalance-z7lpm" Dec 03 12:20:24 crc kubenswrapper[5052]: I1203 12:20:24.389384 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/318545bf-5895-48f9-9649-67207bc4b207-combined-ca-bundle\") pod \"swift-ring-rebalance-z7lpm\" (UID: \"318545bf-5895-48f9-9649-67207bc4b207\") " pod="openstack-kuttl-tests/swift-ring-rebalance-z7lpm" Dec 03 12:20:24 crc kubenswrapper[5052]: I1203 12:20:24.403356 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"swiftconf\" (UniqueName: \"kubernetes.io/secret/318545bf-5895-48f9-9649-67207bc4b207-swiftconf\") pod \"swift-ring-rebalance-z7lpm\" (UID: \"318545bf-5895-48f9-9649-67207bc4b207\") " pod="openstack-kuttl-tests/swift-ring-rebalance-z7lpm" Dec 03 12:20:24 crc kubenswrapper[5052]: I1203 12:20:24.428864 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-pn692\" (UniqueName: \"kubernetes.io/projected/318545bf-5895-48f9-9649-67207bc4b207-kube-api-access-pn692\") pod \"swift-ring-rebalance-z7lpm\" (UID: \"318545bf-5895-48f9-9649-67207bc4b207\") " pod="openstack-kuttl-tests/swift-ring-rebalance-z7lpm" Dec 03 12:20:24 crc kubenswrapper[5052]: I1203 12:20:24.444263 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"dispersionconf\" (UniqueName: \"kubernetes.io/secret/318545bf-5895-48f9-9649-67207bc4b207-dispersionconf\") pod \"swift-ring-rebalance-z7lpm\" (UID: \"318545bf-5895-48f9-9649-67207bc4b207\") " pod="openstack-kuttl-tests/swift-ring-rebalance-z7lpm" Dec 03 12:20:24 crc kubenswrapper[5052]: I1203 12:20:24.445952 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-db-sync-gqt2p"] Dec 03 12:20:24 crc kubenswrapper[5052]: I1203 12:20:24.507606 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-ring-rebalance-z7lpm" Dec 03 12:20:24 crc kubenswrapper[5052]: I1203 12:20:24.597562 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-db-sync-zlxvc"] Dec 03 12:20:24 crc kubenswrapper[5052]: I1203 12:20:24.625203 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-db-sync-zlxvc"] Dec 03 12:20:24 crc kubenswrapper[5052]: I1203 12:20:24.652966 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-db-sync-j4cfx"] Dec 03 12:20:24 crc kubenswrapper[5052]: W1203 12:20:24.653311 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc4487b92_6d46_4cde_b1c3_056ac6fa9753.slice/crio-d474ae25d3177b1d9fe0ff4b2420f4c2baccc1af8926ffad706922bffd92dadd WatchSource:0}: Error finding container d474ae25d3177b1d9fe0ff4b2420f4c2baccc1af8926ffad706922bffd92dadd: Status 404 returned error can't find the container with id d474ae25d3177b1d9fe0ff4b2420f4c2baccc1af8926ffad706922bffd92dadd Dec 03 12:20:24 crc kubenswrapper[5052]: E1203 12:20:24.653386 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of ef532f5fb83b3ed8c9243e5c20875e68a9ee6b646ee961f30ffe04c5b504e0f8 is running failed: container process not found" containerID="ef532f5fb83b3ed8c9243e5c20875e68a9ee6b646ee961f30ffe04c5b504e0f8" cmd=["/usr/local/bin/container-scripts/status_check.sh"] Dec 03 12:20:24 crc kubenswrapper[5052]: E1203 12:20:24.666570 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of ef532f5fb83b3ed8c9243e5c20875e68a9ee6b646ee961f30ffe04c5b504e0f8 is running failed: container process not found" containerID="ef532f5fb83b3ed8c9243e5c20875e68a9ee6b646ee961f30ffe04c5b504e0f8" cmd=["/usr/local/bin/container-scripts/status_check.sh"] Dec 03 12:20:24 crc kubenswrapper[5052]: I1203 12:20:24.679402 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-db-sync-j4cfx"] Dec 03 12:20:24 crc kubenswrapper[5052]: I1203 12:20:24.699550 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-db-sync-r444h"] Dec 03 12:20:24 crc kubenswrapper[5052]: I1203 12:20:24.700947 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-r444h" Dec 03 12:20:24 crc kubenswrapper[5052]: I1203 12:20:24.707193 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell1-conductor-scripts" Dec 03 12:20:24 crc kubenswrapper[5052]: E1203 12:20:24.720555 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of ef532f5fb83b3ed8c9243e5c20875e68a9ee6b646ee961f30ffe04c5b504e0f8 is running failed: container process not found" containerID="ef532f5fb83b3ed8c9243e5c20875e68a9ee6b646ee961f30ffe04c5b504e0f8" cmd=["/usr/local/bin/container-scripts/status_check.sh"] Dec 03 12:20:24 crc kubenswrapper[5052]: E1203 12:20:24.720606 5052 prober.go:104] "Probe errored" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of ef532f5fb83b3ed8c9243e5c20875e68a9ee6b646ee961f30ffe04c5b504e0f8 is running failed: container process not found" probeType="Readiness" pod="openstack-kuttl-tests/ovn-northd-0" podUID="966185e5-f414-46c5-ba48-0e234750795a" containerName="ovn-northd" Dec 03 12:20:24 crc kubenswrapper[5052]: I1203 12:20:24.729005 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/openstack-galera-0" podUID="5067bb8d-d668-46a6-b9de-8065266cde63" containerName="galera" probeResult="failure" output="command timed out" Dec 03 12:20:24 crc kubenswrapper[5052]: I1203 12:20:24.776293 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-db-sync-r444h"] Dec 03 12:20:24 crc kubenswrapper[5052]: I1203 12:20:24.778839 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/cb824463-7efe-49ee-a052-acd1fd8f02d3-config-data\") pod \"nova-cell1-conductor-db-sync-r444h\" (UID: \"cb824463-7efe-49ee-a052-acd1fd8f02d3\") " pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-r444h" Dec 03 12:20:24 crc kubenswrapper[5052]: I1203 12:20:24.778926 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/cb824463-7efe-49ee-a052-acd1fd8f02d3-combined-ca-bundle\") pod \"nova-cell1-conductor-db-sync-r444h\" (UID: \"cb824463-7efe-49ee-a052-acd1fd8f02d3\") " pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-r444h" Dec 03 12:20:24 crc kubenswrapper[5052]: I1203 12:20:24.779045 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8rmgs\" (UniqueName: \"kubernetes.io/projected/cb824463-7efe-49ee-a052-acd1fd8f02d3-kube-api-access-8rmgs\") pod \"nova-cell1-conductor-db-sync-r444h\" (UID: \"cb824463-7efe-49ee-a052-acd1fd8f02d3\") " pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-r444h" Dec 03 12:20:24 crc kubenswrapper[5052]: I1203 12:20:24.779070 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/cb824463-7efe-49ee-a052-acd1fd8f02d3-scripts\") pod \"nova-cell1-conductor-db-sync-r444h\" (UID: \"cb824463-7efe-49ee-a052-acd1fd8f02d3\") " pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-r444h" Dec 03 12:20:24 crc kubenswrapper[5052]: E1203 12:20:24.853027 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="8dfc8c0fb51ed3361082e5d6a21d3480abe509b00c224f35e46b8ddb4b3e7415" cmd=["/bin/bash","/var/lib/operator-scripts/mysql_probe.sh","readiness"] Dec 03 12:20:24 crc kubenswrapper[5052]: I1203 12:20:24.860650 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-db-sync-hjwg6"] Dec 03 12:20:24 crc kubenswrapper[5052]: I1203 12:20:24.862035 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-hjwg6" Dec 03 12:20:24 crc kubenswrapper[5052]: E1203 12:20:24.872149 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="8dfc8c0fb51ed3361082e5d6a21d3480abe509b00c224f35e46b8ddb4b3e7415" cmd=["/bin/bash","/var/lib/operator-scripts/mysql_probe.sh","readiness"] Dec 03 12:20:24 crc kubenswrapper[5052]: I1203 12:20:24.872983 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell0-conductor-scripts" Dec 03 12:20:24 crc kubenswrapper[5052]: I1203 12:20:24.882590 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-8rmgs\" (UniqueName: \"kubernetes.io/projected/cb824463-7efe-49ee-a052-acd1fd8f02d3-kube-api-access-8rmgs\") pod \"nova-cell1-conductor-db-sync-r444h\" (UID: \"cb824463-7efe-49ee-a052-acd1fd8f02d3\") " pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-r444h" Dec 03 12:20:24 crc kubenswrapper[5052]: I1203 12:20:24.882678 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/cb824463-7efe-49ee-a052-acd1fd8f02d3-scripts\") pod \"nova-cell1-conductor-db-sync-r444h\" (UID: \"cb824463-7efe-49ee-a052-acd1fd8f02d3\") " pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-r444h" Dec 03 12:20:24 crc kubenswrapper[5052]: I1203 12:20:24.882773 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/cb824463-7efe-49ee-a052-acd1fd8f02d3-config-data\") pod \"nova-cell1-conductor-db-sync-r444h\" (UID: \"cb824463-7efe-49ee-a052-acd1fd8f02d3\") " pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-r444h" Dec 03 12:20:24 crc kubenswrapper[5052]: I1203 12:20:24.882841 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/cb824463-7efe-49ee-a052-acd1fd8f02d3-combined-ca-bundle\") pod \"nova-cell1-conductor-db-sync-r444h\" (UID: \"cb824463-7efe-49ee-a052-acd1fd8f02d3\") " pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-r444h" Dec 03 12:20:24 crc kubenswrapper[5052]: I1203 12:20:24.889653 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/cb824463-7efe-49ee-a052-acd1fd8f02d3-scripts\") pod \"nova-cell1-conductor-db-sync-r444h\" (UID: \"cb824463-7efe-49ee-a052-acd1fd8f02d3\") " pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-r444h" Dec 03 12:20:24 crc kubenswrapper[5052]: I1203 12:20:24.893528 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/cb824463-7efe-49ee-a052-acd1fd8f02d3-combined-ca-bundle\") pod \"nova-cell1-conductor-db-sync-r444h\" (UID: \"cb824463-7efe-49ee-a052-acd1fd8f02d3\") " pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-r444h" Dec 03 12:20:24 crc kubenswrapper[5052]: E1203 12:20:24.893848 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="8dfc8c0fb51ed3361082e5d6a21d3480abe509b00c224f35e46b8ddb4b3e7415" cmd=["/bin/bash","/var/lib/operator-scripts/mysql_probe.sh","readiness"] Dec 03 12:20:24 crc kubenswrapper[5052]: E1203 12:20:24.893906 5052 prober.go:104] "Probe errored" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" probeType="Readiness" pod="openstack-kuttl-tests/openstack-cell1-galera-0" podUID="a56992d9-4662-444f-8a45-0375d0855946" containerName="galera" Dec 03 12:20:24 crc kubenswrapper[5052]: I1203 12:20:24.906105 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/cb824463-7efe-49ee-a052-acd1fd8f02d3-config-data\") pod \"nova-cell1-conductor-db-sync-r444h\" (UID: \"cb824463-7efe-49ee-a052-acd1fd8f02d3\") " pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-r444h" Dec 03 12:20:24 crc kubenswrapper[5052]: I1203 12:20:24.926270 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_ovsdbserver-nb-0_fcd60977-d995-42ee-b654-ba2947a3316d/ovsdbserver-nb/0.log" Dec 03 12:20:24 crc kubenswrapper[5052]: I1203 12:20:24.926641 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-db-sync-hjwg6"] Dec 03 12:20:24 crc kubenswrapper[5052]: I1203 12:20:24.926742 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:20:24 crc kubenswrapper[5052]: I1203 12:20:24.952408 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-8rmgs\" (UniqueName: \"kubernetes.io/projected/cb824463-7efe-49ee-a052-acd1fd8f02d3-kube-api-access-8rmgs\") pod \"nova-cell1-conductor-db-sync-r444h\" (UID: \"cb824463-7efe-49ee-a052-acd1fd8f02d3\") " pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-r444h" Dec 03 12:20:24 crc kubenswrapper[5052]: I1203 12:20:24.984137 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/9ea78435-8fd9-4321-9e48-3aaae833dcf2-scripts\") pod \"nova-cell0-conductor-db-sync-hjwg6\" (UID: \"9ea78435-8fd9-4321-9e48-3aaae833dcf2\") " pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-hjwg6" Dec 03 12:20:24 crc kubenswrapper[5052]: I1203 12:20:24.984217 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9ea78435-8fd9-4321-9e48-3aaae833dcf2-combined-ca-bundle\") pod \"nova-cell0-conductor-db-sync-hjwg6\" (UID: \"9ea78435-8fd9-4321-9e48-3aaae833dcf2\") " pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-hjwg6" Dec 03 12:20:24 crc kubenswrapper[5052]: I1203 12:20:24.984243 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/9ea78435-8fd9-4321-9e48-3aaae833dcf2-config-data\") pod \"nova-cell0-conductor-db-sync-hjwg6\" (UID: \"9ea78435-8fd9-4321-9e48-3aaae833dcf2\") " pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-hjwg6" Dec 03 12:20:24 crc kubenswrapper[5052]: I1203 12:20:24.984271 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5rfwf\" (UniqueName: \"kubernetes.io/projected/9ea78435-8fd9-4321-9e48-3aaae833dcf2-kube-api-access-5rfwf\") pod \"nova-cell0-conductor-db-sync-hjwg6\" (UID: \"9ea78435-8fd9-4321-9e48-3aaae833dcf2\") " pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-hjwg6" Dec 03 12:20:25 crc kubenswrapper[5052]: I1203 12:20:24.998862 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-db-sync-pvgq9"] Dec 03 12:20:25 crc kubenswrapper[5052]: I1203 12:20:25.049017 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-r444h" Dec 03 12:20:25 crc kubenswrapper[5052]: I1203 12:20:25.050017 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/kube-state-metrics-0"] Dec 03 12:20:25 crc kubenswrapper[5052]: I1203 12:20:25.050337 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/kube-state-metrics-0" podUID="42e93b9b-3a58-4ca6-abc6-6e359b15e7cc" containerName="kube-state-metrics" containerID="cri-o://c572ac5c2d472c1297b5500f42d80ab53aad7c4f8d35a45f1ad28447e9dc6639" gracePeriod=30 Dec 03 12:20:25 crc kubenswrapper[5052]: I1203 12:20:25.100854 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9pp7j\" (UniqueName: \"kubernetes.io/projected/fcd60977-d995-42ee-b654-ba2947a3316d-kube-api-access-9pp7j\") pod \"fcd60977-d995-42ee-b654-ba2947a3316d\" (UID: \"fcd60977-d995-42ee-b654-ba2947a3316d\") " Dec 03 12:20:25 crc kubenswrapper[5052]: I1203 12:20:25.100956 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovndbcluster-nb-etc-ovn\" (UniqueName: \"kubernetes.io/local-volume/local-storage20-crc\") pod \"fcd60977-d995-42ee-b654-ba2947a3316d\" (UID: \"fcd60977-d995-42ee-b654-ba2947a3316d\") " Dec 03 12:20:25 crc kubenswrapper[5052]: I1203 12:20:25.101008 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovsdbserver-nb-tls-certs\" (UniqueName: \"kubernetes.io/secret/fcd60977-d995-42ee-b654-ba2947a3316d-ovsdbserver-nb-tls-certs\") pod \"fcd60977-d995-42ee-b654-ba2947a3316d\" (UID: \"fcd60977-d995-42ee-b654-ba2947a3316d\") " Dec 03 12:20:25 crc kubenswrapper[5052]: I1203 12:20:25.101080 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/fcd60977-d995-42ee-b654-ba2947a3316d-config\") pod \"fcd60977-d995-42ee-b654-ba2947a3316d\" (UID: \"fcd60977-d995-42ee-b654-ba2947a3316d\") " Dec 03 12:20:25 crc kubenswrapper[5052]: I1203 12:20:25.101158 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/fcd60977-d995-42ee-b654-ba2947a3316d-scripts\") pod \"fcd60977-d995-42ee-b654-ba2947a3316d\" (UID: \"fcd60977-d995-42ee-b654-ba2947a3316d\") " Dec 03 12:20:25 crc kubenswrapper[5052]: I1203 12:20:25.101197 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/fcd60977-d995-42ee-b654-ba2947a3316d-metrics-certs-tls-certs\") pod \"fcd60977-d995-42ee-b654-ba2947a3316d\" (UID: \"fcd60977-d995-42ee-b654-ba2947a3316d\") " Dec 03 12:20:25 crc kubenswrapper[5052]: I1203 12:20:25.101336 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/fcd60977-d995-42ee-b654-ba2947a3316d-combined-ca-bundle\") pod \"fcd60977-d995-42ee-b654-ba2947a3316d\" (UID: \"fcd60977-d995-42ee-b654-ba2947a3316d\") " Dec 03 12:20:25 crc kubenswrapper[5052]: I1203 12:20:25.101392 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/fcd60977-d995-42ee-b654-ba2947a3316d-ovsdb-rundir\") pod \"fcd60977-d995-42ee-b654-ba2947a3316d\" (UID: \"fcd60977-d995-42ee-b654-ba2947a3316d\") " Dec 03 12:20:25 crc kubenswrapper[5052]: I1203 12:20:25.102282 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fcd60977-d995-42ee-b654-ba2947a3316d-config" (OuterVolumeSpecName: "config") pod "fcd60977-d995-42ee-b654-ba2947a3316d" (UID: "fcd60977-d995-42ee-b654-ba2947a3316d"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:20:25 crc kubenswrapper[5052]: I1203 12:20:25.102392 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/9ea78435-8fd9-4321-9e48-3aaae833dcf2-scripts\") pod \"nova-cell0-conductor-db-sync-hjwg6\" (UID: \"9ea78435-8fd9-4321-9e48-3aaae833dcf2\") " pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-hjwg6" Dec 03 12:20:25 crc kubenswrapper[5052]: I1203 12:20:25.102717 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9ea78435-8fd9-4321-9e48-3aaae833dcf2-combined-ca-bundle\") pod \"nova-cell0-conductor-db-sync-hjwg6\" (UID: \"9ea78435-8fd9-4321-9e48-3aaae833dcf2\") " pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-hjwg6" Dec 03 12:20:25 crc kubenswrapper[5052]: I1203 12:20:25.102774 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/9ea78435-8fd9-4321-9e48-3aaae833dcf2-config-data\") pod \"nova-cell0-conductor-db-sync-hjwg6\" (UID: \"9ea78435-8fd9-4321-9e48-3aaae833dcf2\") " pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-hjwg6" Dec 03 12:20:25 crc kubenswrapper[5052]: I1203 12:20:25.102858 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-5rfwf\" (UniqueName: \"kubernetes.io/projected/9ea78435-8fd9-4321-9e48-3aaae833dcf2-kube-api-access-5rfwf\") pod \"nova-cell0-conductor-db-sync-hjwg6\" (UID: \"9ea78435-8fd9-4321-9e48-3aaae833dcf2\") " pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-hjwg6" Dec 03 12:20:25 crc kubenswrapper[5052]: I1203 12:20:25.103464 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/fcd60977-d995-42ee-b654-ba2947a3316d-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:25 crc kubenswrapper[5052]: I1203 12:20:25.105794 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fcd60977-d995-42ee-b654-ba2947a3316d-ovsdb-rundir" (OuterVolumeSpecName: "ovsdb-rundir") pod "fcd60977-d995-42ee-b654-ba2947a3316d" (UID: "fcd60977-d995-42ee-b654-ba2947a3316d"). InnerVolumeSpecName "ovsdb-rundir". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:20:25 crc kubenswrapper[5052]: I1203 12:20:25.106572 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fcd60977-d995-42ee-b654-ba2947a3316d-scripts" (OuterVolumeSpecName: "scripts") pod "fcd60977-d995-42ee-b654-ba2947a3316d" (UID: "fcd60977-d995-42ee-b654-ba2947a3316d"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:20:25 crc kubenswrapper[5052]: I1203 12:20:25.121276 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/9ea78435-8fd9-4321-9e48-3aaae833dcf2-config-data\") pod \"nova-cell0-conductor-db-sync-hjwg6\" (UID: \"9ea78435-8fd9-4321-9e48-3aaae833dcf2\") " pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-hjwg6" Dec 03 12:20:25 crc kubenswrapper[5052]: I1203 12:20:25.126618 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fcd60977-d995-42ee-b654-ba2947a3316d-kube-api-access-9pp7j" (OuterVolumeSpecName: "kube-api-access-9pp7j") pod "fcd60977-d995-42ee-b654-ba2947a3316d" (UID: "fcd60977-d995-42ee-b654-ba2947a3316d"). InnerVolumeSpecName "kube-api-access-9pp7j". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:20:25 crc kubenswrapper[5052]: I1203 12:20:25.132643 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage20-crc" (OuterVolumeSpecName: "ovndbcluster-nb-etc-ovn") pod "fcd60977-d995-42ee-b654-ba2947a3316d" (UID: "fcd60977-d995-42ee-b654-ba2947a3316d"). InnerVolumeSpecName "local-storage20-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 12:20:25 crc kubenswrapper[5052]: I1203 12:20:25.135161 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-5rfwf\" (UniqueName: \"kubernetes.io/projected/9ea78435-8fd9-4321-9e48-3aaae833dcf2-kube-api-access-5rfwf\") pod \"nova-cell0-conductor-db-sync-hjwg6\" (UID: \"9ea78435-8fd9-4321-9e48-3aaae833dcf2\") " pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-hjwg6" Dec 03 12:20:25 crc kubenswrapper[5052]: I1203 12:20:25.137095 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/9ea78435-8fd9-4321-9e48-3aaae833dcf2-scripts\") pod \"nova-cell0-conductor-db-sync-hjwg6\" (UID: \"9ea78435-8fd9-4321-9e48-3aaae833dcf2\") " pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-hjwg6" Dec 03 12:20:25 crc kubenswrapper[5052]: I1203 12:20:25.169253 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9ea78435-8fd9-4321-9e48-3aaae833dcf2-combined-ca-bundle\") pod \"nova-cell0-conductor-db-sync-hjwg6\" (UID: \"9ea78435-8fd9-4321-9e48-3aaae833dcf2\") " pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-hjwg6" Dec 03 12:20:25 crc kubenswrapper[5052]: I1203 12:20:25.213397 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/fcd60977-d995-42ee-b654-ba2947a3316d-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:25 crc kubenswrapper[5052]: I1203 12:20:25.213449 5052 reconciler_common.go:293] "Volume detached for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/fcd60977-d995-42ee-b654-ba2947a3316d-ovsdb-rundir\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:25 crc kubenswrapper[5052]: I1203 12:20:25.213466 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-9pp7j\" (UniqueName: \"kubernetes.io/projected/fcd60977-d995-42ee-b654-ba2947a3316d-kube-api-access-9pp7j\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:25 crc kubenswrapper[5052]: I1203 12:20:25.213853 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage20-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage20-crc\") on node \"crc\" " Dec 03 12:20:25 crc kubenswrapper[5052]: I1203 12:20:25.246339 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage20-crc" (UniqueName: "kubernetes.io/local-volume/local-storage20-crc") on node "crc" Dec 03 12:20:25 crc kubenswrapper[5052]: I1203 12:20:25.267694 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-hjwg6" Dec 03 12:20:25 crc kubenswrapper[5052]: I1203 12:20:25.320241 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage20-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage20-crc\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:25 crc kubenswrapper[5052]: I1203 12:20:25.320715 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/memcached-0" podUID="695de243-96a9-4e8b-a0f3-3a4ef2d95efe" containerName="memcached" probeResult="failure" output="dial tcp 10.217.1.63:11211: connect: connection refused" Dec 03 12:20:25 crc kubenswrapper[5052]: I1203 12:20:25.324767 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_ovsdbserver-nb-0_fcd60977-d995-42ee-b654-ba2947a3316d/ovsdbserver-nb/0.log" Dec 03 12:20:25 crc kubenswrapper[5052]: I1203 12:20:25.324843 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-nb-0" event={"ID":"fcd60977-d995-42ee-b654-ba2947a3316d","Type":"ContainerDied","Data":"7e8beed27f00db2e91962f4ee2426cdfad0e851c8f641388600870c9ec6f3109"} Dec 03 12:20:25 crc kubenswrapper[5052]: I1203 12:20:25.324884 5052 scope.go:117] "RemoveContainer" containerID="daa77eec7322cd2c65ca8b891da5191f57aec462539227515b3efcddb082df4d" Dec 03 12:20:25 crc kubenswrapper[5052]: I1203 12:20:25.325047 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:20:25 crc kubenswrapper[5052]: I1203 12:20:25.331022 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-db-sync-pvgq9" event={"ID":"66f5168a-1ee5-4de9-8e50-90ef25186ee8","Type":"ContainerStarted","Data":"52fde149577b09c34ef00f62971a8730a9ea8cb1951a6aaf40880c92177d564e"} Dec 03 12:20:25 crc kubenswrapper[5052]: I1203 12:20:25.333486 5052 generic.go:334] "Generic (PLEG): container finished" podID="42e93b9b-3a58-4ca6-abc6-6e359b15e7cc" containerID="c572ac5c2d472c1297b5500f42d80ab53aad7c4f8d35a45f1ad28447e9dc6639" exitCode=2 Dec 03 12:20:25 crc kubenswrapper[5052]: I1203 12:20:25.333549 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/kube-state-metrics-0" event={"ID":"42e93b9b-3a58-4ca6-abc6-6e359b15e7cc","Type":"ContainerDied","Data":"c572ac5c2d472c1297b5500f42d80ab53aad7c4f8d35a45f1ad28447e9dc6639"} Dec 03 12:20:25 crc kubenswrapper[5052]: I1203 12:20:25.335081 5052 generic.go:334] "Generic (PLEG): container finished" podID="695de243-96a9-4e8b-a0f3-3a4ef2d95efe" containerID="a0b610780c737434b4d97d2fd99c01273a88681fdc19b3a41fdf3d032c04b664" exitCode=0 Dec 03 12:20:25 crc kubenswrapper[5052]: I1203 12:20:25.335117 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/memcached-0" event={"ID":"695de243-96a9-4e8b-a0f3-3a4ef2d95efe","Type":"ContainerDied","Data":"a0b610780c737434b4d97d2fd99c01273a88681fdc19b3a41fdf3d032c04b664"} Dec 03 12:20:25 crc kubenswrapper[5052]: I1203 12:20:25.339285 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-db-sync-gqt2p" event={"ID":"c4487b92-6d46-4cde-b1c3-056ac6fa9753","Type":"ContainerStarted","Data":"d474ae25d3177b1d9fe0ff4b2420f4c2baccc1af8926ffad706922bffd92dadd"} Dec 03 12:20:25 crc kubenswrapper[5052]: I1203 12:20:25.359375 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-db-sync-lrqq6"] Dec 03 12:20:25 crc kubenswrapper[5052]: I1203 12:20:25.364939 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"ce16d54b-ef04-429b-8795-2865395f7774","Type":"ContainerStarted","Data":"0a4ad14776400bd11635febb15632b857ffe75a1ed4b078b2eb0c7621d98c834"} Dec 03 12:20:25 crc kubenswrapper[5052]: I1203 12:20:25.385739 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-db-sync-gzzkp" event={"ID":"50ae31b2-a1c0-4afb-8e23-d7856c55af1d","Type":"ContainerStarted","Data":"739b5f17c284ab8823d07f252de61cfc9a034a60b12ad3eafd6d9d840e4cf76d"} Dec 03 12:20:25 crc kubenswrapper[5052]: I1203 12:20:25.389791 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fcd60977-d995-42ee-b654-ba2947a3316d-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "fcd60977-d995-42ee-b654-ba2947a3316d" (UID: "fcd60977-d995-42ee-b654-ba2947a3316d"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:20:25 crc kubenswrapper[5052]: I1203 12:20:25.414956 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-api-0" podStartSLOduration=5.414936675 podStartE2EDuration="5.414936675s" podCreationTimestamp="2025-12-03 12:20:20 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:20:25.413100992 +0000 UTC m=+3616.001342756" watchObservedRunningTime="2025-12-03 12:20:25.414936675 +0000 UTC m=+3616.003178439" Dec 03 12:20:25 crc kubenswrapper[5052]: I1203 12:20:25.422313 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/fcd60977-d995-42ee-b654-ba2947a3316d-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:25 crc kubenswrapper[5052]: I1203 12:20:25.447879 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fcd60977-d995-42ee-b654-ba2947a3316d-metrics-certs-tls-certs" (OuterVolumeSpecName: "metrics-certs-tls-certs") pod "fcd60977-d995-42ee-b654-ba2947a3316d" (UID: "fcd60977-d995-42ee-b654-ba2947a3316d"). InnerVolumeSpecName "metrics-certs-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:20:25 crc kubenswrapper[5052]: I1203 12:20:25.451866 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/placement-db-sync-gzzkp" podStartSLOduration=4.451849918 podStartE2EDuration="4.451849918s" podCreationTimestamp="2025-12-03 12:20:21 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:20:25.449925104 +0000 UTC m=+3616.038166868" watchObservedRunningTime="2025-12-03 12:20:25.451849918 +0000 UTC m=+3616.040091682" Dec 03 12:20:25 crc kubenswrapper[5052]: I1203 12:20:25.475177 5052 generic.go:334] "Generic (PLEG): container finished" podID="335ad996-ad3b-4ac8-a488-64e9453f24a4" containerID="9ca86e437e38c5642f8640cedce80dfac23a56655784a6b7769c0317c467d085" exitCode=0 Dec 03 12:20:25 crc kubenswrapper[5052]: I1203 12:20:25.475209 5052 generic.go:334] "Generic (PLEG): container finished" podID="335ad996-ad3b-4ac8-a488-64e9453f24a4" containerID="2660887ea0a6145f5154899fe6bb77bb7f4491a21de6c7447f3c126adc776305" exitCode=0 Dec 03 12:20:25 crc kubenswrapper[5052]: I1203 12:20:25.475223 5052 generic.go:334] "Generic (PLEG): container finished" podID="335ad996-ad3b-4ac8-a488-64e9453f24a4" containerID="486c4efda615830f80b3292087835a1464d89e6438a42bab394e0f23416c0683" exitCode=0 Dec 03 12:20:25 crc kubenswrapper[5052]: I1203 12:20:25.475232 5052 generic.go:334] "Generic (PLEG): container finished" podID="335ad996-ad3b-4ac8-a488-64e9453f24a4" containerID="2b57d07d8ebd7db5a1f0aa4b4ebb75c36047390ad674b3a41f27874f7e01c7b5" exitCode=0 Dec 03 12:20:25 crc kubenswrapper[5052]: I1203 12:20:25.475240 5052 generic.go:334] "Generic (PLEG): container finished" podID="335ad996-ad3b-4ac8-a488-64e9453f24a4" containerID="2d0cbf82dccb7f04f14e9a8c02dce57f3fe56aa4bea37bafafd0b0e8ef2825ff" exitCode=0 Dec 03 12:20:25 crc kubenswrapper[5052]: I1203 12:20:25.475249 5052 generic.go:334] "Generic (PLEG): container finished" podID="335ad996-ad3b-4ac8-a488-64e9453f24a4" containerID="2591ce792243cc1fe842d92ce9a53f946994c1f42c26f9657e9e29db8d53f707" exitCode=0 Dec 03 12:20:25 crc kubenswrapper[5052]: I1203 12:20:25.475259 5052 generic.go:334] "Generic (PLEG): container finished" podID="335ad996-ad3b-4ac8-a488-64e9453f24a4" containerID="55c422ce2a089bbc2d4cce09c586d3b79a3abf94fbea99b43ef8ff9d803269d4" exitCode=0 Dec 03 12:20:25 crc kubenswrapper[5052]: I1203 12:20:25.475268 5052 generic.go:334] "Generic (PLEG): container finished" podID="335ad996-ad3b-4ac8-a488-64e9453f24a4" containerID="000974bff3024c4dd8ae265f8fc3dc2cd023ab0097daaa4fd6b13c365cb5dd62" exitCode=0 Dec 03 12:20:25 crc kubenswrapper[5052]: I1203 12:20:25.475277 5052 generic.go:334] "Generic (PLEG): container finished" podID="335ad996-ad3b-4ac8-a488-64e9453f24a4" containerID="e1a2b505ab3bea37930a1c222d85a3ab5421377ce504e97bef52966f349f5e26" exitCode=0 Dec 03 12:20:25 crc kubenswrapper[5052]: I1203 12:20:25.475285 5052 generic.go:334] "Generic (PLEG): container finished" podID="335ad996-ad3b-4ac8-a488-64e9453f24a4" containerID="6af612b56d9c2f4b731475f24619368f730783d7ef872ccc130c33f5a189e57f" exitCode=0 Dec 03 12:20:25 crc kubenswrapper[5052]: I1203 12:20:25.475295 5052 generic.go:334] "Generic (PLEG): container finished" podID="335ad996-ad3b-4ac8-a488-64e9453f24a4" containerID="f7f12231dd030c2c9c3f43d8493e609bd58faa9dd7c14aed345ea597010d1451" exitCode=0 Dec 03 12:20:25 crc kubenswrapper[5052]: I1203 12:20:25.475304 5052 generic.go:334] "Generic (PLEG): container finished" podID="335ad996-ad3b-4ac8-a488-64e9453f24a4" containerID="d801a1f1f45c17b5422af1ea12c95b38c64a258c49f3953222cbb94387e19d2c" exitCode=0 Dec 03 12:20:25 crc kubenswrapper[5052]: I1203 12:20:25.475315 5052 generic.go:334] "Generic (PLEG): container finished" podID="335ad996-ad3b-4ac8-a488-64e9453f24a4" containerID="df73d6886a1afb1e6d09c9481252bc2871f9a8608386e4ea20b13852f0881a4b" exitCode=0 Dec 03 12:20:25 crc kubenswrapper[5052]: I1203 12:20:25.475324 5052 generic.go:334] "Generic (PLEG): container finished" podID="335ad996-ad3b-4ac8-a488-64e9453f24a4" containerID="006d1e418590929322462448d037f47e0e4031553a716f0f323b0ceb29ae6857" exitCode=0 Dec 03 12:20:25 crc kubenswrapper[5052]: I1203 12:20:25.475432 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"335ad996-ad3b-4ac8-a488-64e9453f24a4","Type":"ContainerDied","Data":"9ca86e437e38c5642f8640cedce80dfac23a56655784a6b7769c0317c467d085"} Dec 03 12:20:25 crc kubenswrapper[5052]: I1203 12:20:25.475466 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"335ad996-ad3b-4ac8-a488-64e9453f24a4","Type":"ContainerDied","Data":"2660887ea0a6145f5154899fe6bb77bb7f4491a21de6c7447f3c126adc776305"} Dec 03 12:20:25 crc kubenswrapper[5052]: I1203 12:20:25.475481 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"335ad996-ad3b-4ac8-a488-64e9453f24a4","Type":"ContainerDied","Data":"486c4efda615830f80b3292087835a1464d89e6438a42bab394e0f23416c0683"} Dec 03 12:20:25 crc kubenswrapper[5052]: I1203 12:20:25.475494 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"335ad996-ad3b-4ac8-a488-64e9453f24a4","Type":"ContainerDied","Data":"2b57d07d8ebd7db5a1f0aa4b4ebb75c36047390ad674b3a41f27874f7e01c7b5"} Dec 03 12:20:25 crc kubenswrapper[5052]: I1203 12:20:25.475536 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"335ad996-ad3b-4ac8-a488-64e9453f24a4","Type":"ContainerDied","Data":"2d0cbf82dccb7f04f14e9a8c02dce57f3fe56aa4bea37bafafd0b0e8ef2825ff"} Dec 03 12:20:25 crc kubenswrapper[5052]: I1203 12:20:25.475554 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"335ad996-ad3b-4ac8-a488-64e9453f24a4","Type":"ContainerDied","Data":"2591ce792243cc1fe842d92ce9a53f946994c1f42c26f9657e9e29db8d53f707"} Dec 03 12:20:25 crc kubenswrapper[5052]: I1203 12:20:25.475568 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"335ad996-ad3b-4ac8-a488-64e9453f24a4","Type":"ContainerDied","Data":"55c422ce2a089bbc2d4cce09c586d3b79a3abf94fbea99b43ef8ff9d803269d4"} Dec 03 12:20:25 crc kubenswrapper[5052]: I1203 12:20:25.475582 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"335ad996-ad3b-4ac8-a488-64e9453f24a4","Type":"ContainerDied","Data":"000974bff3024c4dd8ae265f8fc3dc2cd023ab0097daaa4fd6b13c365cb5dd62"} Dec 03 12:20:25 crc kubenswrapper[5052]: I1203 12:20:25.475595 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"335ad996-ad3b-4ac8-a488-64e9453f24a4","Type":"ContainerDied","Data":"e1a2b505ab3bea37930a1c222d85a3ab5421377ce504e97bef52966f349f5e26"} Dec 03 12:20:25 crc kubenswrapper[5052]: I1203 12:20:25.475609 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"335ad996-ad3b-4ac8-a488-64e9453f24a4","Type":"ContainerDied","Data":"6af612b56d9c2f4b731475f24619368f730783d7ef872ccc130c33f5a189e57f"} Dec 03 12:20:25 crc kubenswrapper[5052]: I1203 12:20:25.475621 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"335ad996-ad3b-4ac8-a488-64e9453f24a4","Type":"ContainerDied","Data":"f7f12231dd030c2c9c3f43d8493e609bd58faa9dd7c14aed345ea597010d1451"} Dec 03 12:20:25 crc kubenswrapper[5052]: I1203 12:20:25.475634 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"335ad996-ad3b-4ac8-a488-64e9453f24a4","Type":"ContainerDied","Data":"d801a1f1f45c17b5422af1ea12c95b38c64a258c49f3953222cbb94387e19d2c"} Dec 03 12:20:25 crc kubenswrapper[5052]: I1203 12:20:25.475648 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"335ad996-ad3b-4ac8-a488-64e9453f24a4","Type":"ContainerDied","Data":"df73d6886a1afb1e6d09c9481252bc2871f9a8608386e4ea20b13852f0881a4b"} Dec 03 12:20:25 crc kubenswrapper[5052]: I1203 12:20:25.475666 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"335ad996-ad3b-4ac8-a488-64e9453f24a4","Type":"ContainerDied","Data":"006d1e418590929322462448d037f47e0e4031553a716f0f323b0ceb29ae6857"} Dec 03 12:20:25 crc kubenswrapper[5052]: I1203 12:20:25.480113 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fcd60977-d995-42ee-b654-ba2947a3316d-ovsdbserver-nb-tls-certs" (OuterVolumeSpecName: "ovsdbserver-nb-tls-certs") pod "fcd60977-d995-42ee-b654-ba2947a3316d" (UID: "fcd60977-d995-42ee-b654-ba2947a3316d"). InnerVolumeSpecName "ovsdbserver-nb-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:20:25 crc kubenswrapper[5052]: I1203 12:20:25.481036 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-db-sync-9jf9q" event={"ID":"b852a7d7-8510-440d-8e87-0755fac2a324","Type":"ContainerStarted","Data":"065aebd7d9dcbe855a40abb9f1fcec166d09f5a018bf0567b9d6ef02a6577350"} Dec 03 12:20:25 crc kubenswrapper[5052]: I1203 12:20:25.525955 5052 reconciler_common.go:293] "Volume detached for volume \"ovsdbserver-nb-tls-certs\" (UniqueName: \"kubernetes.io/secret/fcd60977-d995-42ee-b654-ba2947a3316d-ovsdbserver-nb-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:25 crc kubenswrapper[5052]: I1203 12:20:25.525989 5052 reconciler_common.go:293] "Volume detached for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/fcd60977-d995-42ee-b654-ba2947a3316d-metrics-certs-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:25 crc kubenswrapper[5052]: I1203 12:20:25.546718 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/keystone-db-sync-9jf9q" podStartSLOduration=3.546659465 podStartE2EDuration="3.546659465s" podCreationTimestamp="2025-12-03 12:20:22 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:20:25.515794484 +0000 UTC m=+3616.104036258" watchObservedRunningTime="2025-12-03 12:20:25.546659465 +0000 UTC m=+3616.134901229" Dec 03 12:20:25 crc kubenswrapper[5052]: I1203 12:20:25.718936 5052 scope.go:117] "RemoveContainer" containerID="87154b4a2a5069ca41b7570b52ae76e123d9dca6d0b7110267bc2f70b48cd720" Dec 03 12:20:25 crc kubenswrapper[5052]: I1203 12:20:25.853112 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="33837271-e865-4153-afab-6de6235c8ea8" path="/var/lib/kubelet/pods/33837271-e865-4153-afab-6de6235c8ea8/volumes" Dec 03 12:20:25 crc kubenswrapper[5052]: I1203 12:20:25.854674 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8a3bbadd-37b5-4328-ae01-657aa8eb3028" path="/var/lib/kubelet/pods/8a3bbadd-37b5-4328-ae01-657aa8eb3028/volumes" Dec 03 12:20:25 crc kubenswrapper[5052]: I1203 12:20:25.855401 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c4d2abd6-6282-4c8f-b24d-49ffd64b0ce2" path="/var/lib/kubelet/pods/c4d2abd6-6282-4c8f-b24d-49ffd64b0ce2/volumes" Dec 03 12:20:25 crc kubenswrapper[5052]: I1203 12:20:25.862253 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:20:25 crc kubenswrapper[5052]: I1203 12:20:25.892853 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_ovsdbserver-sb-0_06dda089-0745-4941-adfe-2de711fce24e/ovsdbserver-sb/0.log" Dec 03 12:20:25 crc kubenswrapper[5052]: I1203 12:20:25.893097 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_ovn-northd-0_966185e5-f414-46c5-ba48-0e234750795a/ovn-northd/0.log" Dec 03 12:20:25 crc kubenswrapper[5052]: I1203 12:20:25.893162 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:20:25 crc kubenswrapper[5052]: I1203 12:20:25.893758 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:20:25 crc kubenswrapper[5052]: I1203 12:20:25.925703 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-nb-0"] Dec 03 12:20:25 crc kubenswrapper[5052]: I1203 12:20:25.976709 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-nb-0"] Dec 03 12:20:26 crc kubenswrapper[5052]: I1203 12:20:26.057574 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ovsdbserver-nb-0"] Dec 03 12:20:26 crc kubenswrapper[5052]: E1203 12:20:26.057980 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="966185e5-f414-46c5-ba48-0e234750795a" containerName="ovn-northd" Dec 03 12:20:26 crc kubenswrapper[5052]: I1203 12:20:26.057995 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="966185e5-f414-46c5-ba48-0e234750795a" containerName="ovn-northd" Dec 03 12:20:26 crc kubenswrapper[5052]: E1203 12:20:26.058003 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="966185e5-f414-46c5-ba48-0e234750795a" containerName="openstack-network-exporter" Dec 03 12:20:26 crc kubenswrapper[5052]: I1203 12:20:26.058008 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="966185e5-f414-46c5-ba48-0e234750795a" containerName="openstack-network-exporter" Dec 03 12:20:26 crc kubenswrapper[5052]: E1203 12:20:26.058024 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="fcd60977-d995-42ee-b654-ba2947a3316d" containerName="openstack-network-exporter" Dec 03 12:20:26 crc kubenswrapper[5052]: I1203 12:20:26.058030 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="fcd60977-d995-42ee-b654-ba2947a3316d" containerName="openstack-network-exporter" Dec 03 12:20:26 crc kubenswrapper[5052]: E1203 12:20:26.058051 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="fcd60977-d995-42ee-b654-ba2947a3316d" containerName="ovsdbserver-nb" Dec 03 12:20:26 crc kubenswrapper[5052]: I1203 12:20:26.058057 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="fcd60977-d995-42ee-b654-ba2947a3316d" containerName="ovsdbserver-nb" Dec 03 12:20:26 crc kubenswrapper[5052]: E1203 12:20:26.058071 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="06dda089-0745-4941-adfe-2de711fce24e" containerName="ovsdbserver-sb" Dec 03 12:20:26 crc kubenswrapper[5052]: I1203 12:20:26.058077 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="06dda089-0745-4941-adfe-2de711fce24e" containerName="ovsdbserver-sb" Dec 03 12:20:26 crc kubenswrapper[5052]: E1203 12:20:26.058090 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="06dda089-0745-4941-adfe-2de711fce24e" containerName="openstack-network-exporter" Dec 03 12:20:26 crc kubenswrapper[5052]: I1203 12:20:26.058096 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="06dda089-0745-4941-adfe-2de711fce24e" containerName="openstack-network-exporter" Dec 03 12:20:26 crc kubenswrapper[5052]: I1203 12:20:26.058270 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="fcd60977-d995-42ee-b654-ba2947a3316d" containerName="openstack-network-exporter" Dec 03 12:20:26 crc kubenswrapper[5052]: I1203 12:20:26.058290 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="fcd60977-d995-42ee-b654-ba2947a3316d" containerName="ovsdbserver-nb" Dec 03 12:20:26 crc kubenswrapper[5052]: I1203 12:20:26.058306 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="966185e5-f414-46c5-ba48-0e234750795a" containerName="ovn-northd" Dec 03 12:20:26 crc kubenswrapper[5052]: I1203 12:20:26.058319 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="966185e5-f414-46c5-ba48-0e234750795a" containerName="openstack-network-exporter" Dec 03 12:20:26 crc kubenswrapper[5052]: I1203 12:20:26.058331 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="06dda089-0745-4941-adfe-2de711fce24e" containerName="openstack-network-exporter" Dec 03 12:20:26 crc kubenswrapper[5052]: I1203 12:20:26.058339 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="06dda089-0745-4941-adfe-2de711fce24e" containerName="ovsdbserver-sb" Dec 03 12:20:26 crc kubenswrapper[5052]: I1203 12:20:26.059278 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:20:26 crc kubenswrapper[5052]: I1203 12:20:26.064300 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ovncluster-ovndbcluster-nb-dockercfg-4ncxw" Dec 03 12:20:26 crc kubenswrapper[5052]: I1203 12:20:26.064626 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"ovndbcluster-nb-config" Dec 03 12:20:26 crc kubenswrapper[5052]: I1203 12:20:26.064813 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-ovndbcluster-nb-ovndbs" Dec 03 12:20:26 crc kubenswrapper[5052]: I1203 12:20:26.064966 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"ovndbcluster-nb-scripts" Dec 03 12:20:26 crc kubenswrapper[5052]: I1203 12:20:26.078589 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-nb-0"] Dec 03 12:20:26 crc kubenswrapper[5052]: I1203 12:20:26.095890 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovsdbserver-sb-tls-certs\" (UniqueName: \"kubernetes.io/secret/06dda089-0745-4941-adfe-2de711fce24e-ovsdbserver-sb-tls-certs\") pod \"06dda089-0745-4941-adfe-2de711fce24e\" (UID: \"06dda089-0745-4941-adfe-2de711fce24e\") " Dec 03 12:20:26 crc kubenswrapper[5052]: I1203 12:20:26.096020 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovndbcluster-sb-etc-ovn\" (UniqueName: \"kubernetes.io/local-volume/local-storage02-crc\") pod \"06dda089-0745-4941-adfe-2de711fce24e\" (UID: \"06dda089-0745-4941-adfe-2de711fce24e\") " Dec 03 12:20:26 crc kubenswrapper[5052]: I1203 12:20:26.096056 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/06dda089-0745-4941-adfe-2de711fce24e-ovsdb-rundir\") pod \"06dda089-0745-4941-adfe-2de711fce24e\" (UID: \"06dda089-0745-4941-adfe-2de711fce24e\") " Dec 03 12:20:26 crc kubenswrapper[5052]: I1203 12:20:26.096107 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovn-northd-tls-certs\" (UniqueName: \"kubernetes.io/secret/966185e5-f414-46c5-ba48-0e234750795a-ovn-northd-tls-certs\") pod \"966185e5-f414-46c5-ba48-0e234750795a\" (UID: \"966185e5-f414-46c5-ba48-0e234750795a\") " Dec 03 12:20:26 crc kubenswrapper[5052]: I1203 12:20:26.096163 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/966185e5-f414-46c5-ba48-0e234750795a-config\") pod \"966185e5-f414-46c5-ba48-0e234750795a\" (UID: \"966185e5-f414-46c5-ba48-0e234750795a\") " Dec 03 12:20:26 crc kubenswrapper[5052]: I1203 12:20:26.096207 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovn-rundir\" (UniqueName: \"kubernetes.io/empty-dir/966185e5-f414-46c5-ba48-0e234750795a-ovn-rundir\") pod \"966185e5-f414-46c5-ba48-0e234750795a\" (UID: \"966185e5-f414-46c5-ba48-0e234750795a\") " Dec 03 12:20:26 crc kubenswrapper[5052]: I1203 12:20:26.096248 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kmxk2\" (UniqueName: \"kubernetes.io/projected/06dda089-0745-4941-adfe-2de711fce24e-kube-api-access-kmxk2\") pod \"06dda089-0745-4941-adfe-2de711fce24e\" (UID: \"06dda089-0745-4941-adfe-2de711fce24e\") " Dec 03 12:20:26 crc kubenswrapper[5052]: I1203 12:20:26.096269 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/966185e5-f414-46c5-ba48-0e234750795a-metrics-certs-tls-certs\") pod \"966185e5-f414-46c5-ba48-0e234750795a\" (UID: \"966185e5-f414-46c5-ba48-0e234750795a\") " Dec 03 12:20:26 crc kubenswrapper[5052]: I1203 12:20:26.096298 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/966185e5-f414-46c5-ba48-0e234750795a-combined-ca-bundle\") pod \"966185e5-f414-46c5-ba48-0e234750795a\" (UID: \"966185e5-f414-46c5-ba48-0e234750795a\") " Dec 03 12:20:26 crc kubenswrapper[5052]: I1203 12:20:26.096400 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/06dda089-0745-4941-adfe-2de711fce24e-combined-ca-bundle\") pod \"06dda089-0745-4941-adfe-2de711fce24e\" (UID: \"06dda089-0745-4941-adfe-2de711fce24e\") " Dec 03 12:20:26 crc kubenswrapper[5052]: I1203 12:20:26.096435 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-58ns2\" (UniqueName: \"kubernetes.io/projected/966185e5-f414-46c5-ba48-0e234750795a-kube-api-access-58ns2\") pod \"966185e5-f414-46c5-ba48-0e234750795a\" (UID: \"966185e5-f414-46c5-ba48-0e234750795a\") " Dec 03 12:20:26 crc kubenswrapper[5052]: I1203 12:20:26.096454 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/966185e5-f414-46c5-ba48-0e234750795a-scripts\") pod \"966185e5-f414-46c5-ba48-0e234750795a\" (UID: \"966185e5-f414-46c5-ba48-0e234750795a\") " Dec 03 12:20:26 crc kubenswrapper[5052]: I1203 12:20:26.096485 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/06dda089-0745-4941-adfe-2de711fce24e-config\") pod \"06dda089-0745-4941-adfe-2de711fce24e\" (UID: \"06dda089-0745-4941-adfe-2de711fce24e\") " Dec 03 12:20:26 crc kubenswrapper[5052]: I1203 12:20:26.096550 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/06dda089-0745-4941-adfe-2de711fce24e-metrics-certs-tls-certs\") pod \"06dda089-0745-4941-adfe-2de711fce24e\" (UID: \"06dda089-0745-4941-adfe-2de711fce24e\") " Dec 03 12:20:26 crc kubenswrapper[5052]: I1203 12:20:26.096588 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/06dda089-0745-4941-adfe-2de711fce24e-scripts\") pod \"06dda089-0745-4941-adfe-2de711fce24e\" (UID: \"06dda089-0745-4941-adfe-2de711fce24e\") " Dec 03 12:20:26 crc kubenswrapper[5052]: I1203 12:20:26.099202 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/06dda089-0745-4941-adfe-2de711fce24e-scripts" (OuterVolumeSpecName: "scripts") pod "06dda089-0745-4941-adfe-2de711fce24e" (UID: "06dda089-0745-4941-adfe-2de711fce24e"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:20:26 crc kubenswrapper[5052]: I1203 12:20:26.103976 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/966185e5-f414-46c5-ba48-0e234750795a-kube-api-access-58ns2" (OuterVolumeSpecName: "kube-api-access-58ns2") pod "966185e5-f414-46c5-ba48-0e234750795a" (UID: "966185e5-f414-46c5-ba48-0e234750795a"). InnerVolumeSpecName "kube-api-access-58ns2". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:20:26 crc kubenswrapper[5052]: I1203 12:20:26.111639 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/06dda089-0745-4941-adfe-2de711fce24e-ovsdb-rundir" (OuterVolumeSpecName: "ovsdb-rundir") pod "06dda089-0745-4941-adfe-2de711fce24e" (UID: "06dda089-0745-4941-adfe-2de711fce24e"). InnerVolumeSpecName "ovsdb-rundir". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:20:26 crc kubenswrapper[5052]: I1203 12:20:26.121206 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/966185e5-f414-46c5-ba48-0e234750795a-scripts" (OuterVolumeSpecName: "scripts") pod "966185e5-f414-46c5-ba48-0e234750795a" (UID: "966185e5-f414-46c5-ba48-0e234750795a"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:20:26 crc kubenswrapper[5052]: I1203 12:20:26.127042 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/966185e5-f414-46c5-ba48-0e234750795a-ovn-rundir" (OuterVolumeSpecName: "ovn-rundir") pod "966185e5-f414-46c5-ba48-0e234750795a" (UID: "966185e5-f414-46c5-ba48-0e234750795a"). InnerVolumeSpecName "ovn-rundir". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:20:26 crc kubenswrapper[5052]: I1203 12:20:26.129140 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/06dda089-0745-4941-adfe-2de711fce24e-config" (OuterVolumeSpecName: "config") pod "06dda089-0745-4941-adfe-2de711fce24e" (UID: "06dda089-0745-4941-adfe-2de711fce24e"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:20:26 crc kubenswrapper[5052]: I1203 12:20:26.132613 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/966185e5-f414-46c5-ba48-0e234750795a-config" (OuterVolumeSpecName: "config") pod "966185e5-f414-46c5-ba48-0e234750795a" (UID: "966185e5-f414-46c5-ba48-0e234750795a"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:20:26 crc kubenswrapper[5052]: I1203 12:20:26.149835 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/06dda089-0745-4941-adfe-2de711fce24e-kube-api-access-kmxk2" (OuterVolumeSpecName: "kube-api-access-kmxk2") pod "06dda089-0745-4941-adfe-2de711fce24e" (UID: "06dda089-0745-4941-adfe-2de711fce24e"). InnerVolumeSpecName "kube-api-access-kmxk2". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:20:26 crc kubenswrapper[5052]: I1203 12:20:26.179419 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage02-crc" (OuterVolumeSpecName: "ovndbcluster-sb-etc-ovn") pod "06dda089-0745-4941-adfe-2de711fce24e" (UID: "06dda089-0745-4941-adfe-2de711fce24e"). InnerVolumeSpecName "local-storage02-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 12:20:26 crc kubenswrapper[5052]: I1203 12:20:26.206654 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/70cedf3b-c9b9-4e5c-81a2-ba134b150997-ovsdb-rundir\") pod \"ovsdbserver-nb-0\" (UID: \"70cedf3b-c9b9-4e5c-81a2-ba134b150997\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:20:26 crc kubenswrapper[5052]: I1203 12:20:26.206713 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage20-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage20-crc\") pod \"ovsdbserver-nb-0\" (UID: \"70cedf3b-c9b9-4e5c-81a2-ba134b150997\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:20:26 crc kubenswrapper[5052]: I1203 12:20:26.219617 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovsdbserver-nb-tls-certs\" (UniqueName: \"kubernetes.io/secret/70cedf3b-c9b9-4e5c-81a2-ba134b150997-ovsdbserver-nb-tls-certs\") pod \"ovsdbserver-nb-0\" (UID: \"70cedf3b-c9b9-4e5c-81a2-ba134b150997\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:20:26 crc kubenswrapper[5052]: I1203 12:20:26.219732 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/70cedf3b-c9b9-4e5c-81a2-ba134b150997-metrics-certs-tls-certs\") pod \"ovsdbserver-nb-0\" (UID: \"70cedf3b-c9b9-4e5c-81a2-ba134b150997\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:20:26 crc kubenswrapper[5052]: I1203 12:20:26.219791 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/70cedf3b-c9b9-4e5c-81a2-ba134b150997-scripts\") pod \"ovsdbserver-nb-0\" (UID: \"70cedf3b-c9b9-4e5c-81a2-ba134b150997\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:20:26 crc kubenswrapper[5052]: I1203 12:20:26.219824 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/70cedf3b-c9b9-4e5c-81a2-ba134b150997-combined-ca-bundle\") pod \"ovsdbserver-nb-0\" (UID: \"70cedf3b-c9b9-4e5c-81a2-ba134b150997\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:20:26 crc kubenswrapper[5052]: I1203 12:20:26.219843 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/70cedf3b-c9b9-4e5c-81a2-ba134b150997-config\") pod \"ovsdbserver-nb-0\" (UID: \"70cedf3b-c9b9-4e5c-81a2-ba134b150997\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:20:26 crc kubenswrapper[5052]: I1203 12:20:26.219999 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kt8jm\" (UniqueName: \"kubernetes.io/projected/70cedf3b-c9b9-4e5c-81a2-ba134b150997-kube-api-access-kt8jm\") pod \"ovsdbserver-nb-0\" (UID: \"70cedf3b-c9b9-4e5c-81a2-ba134b150997\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:20:26 crc kubenswrapper[5052]: I1203 12:20:26.220106 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage02-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage02-crc\") on node \"crc\" " Dec 03 12:20:26 crc kubenswrapper[5052]: I1203 12:20:26.220118 5052 reconciler_common.go:293] "Volume detached for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/06dda089-0745-4941-adfe-2de711fce24e-ovsdb-rundir\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:26 crc kubenswrapper[5052]: I1203 12:20:26.220128 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/966185e5-f414-46c5-ba48-0e234750795a-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:26 crc kubenswrapper[5052]: I1203 12:20:26.220137 5052 reconciler_common.go:293] "Volume detached for volume \"ovn-rundir\" (UniqueName: \"kubernetes.io/empty-dir/966185e5-f414-46c5-ba48-0e234750795a-ovn-rundir\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:26 crc kubenswrapper[5052]: I1203 12:20:26.220146 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-kmxk2\" (UniqueName: \"kubernetes.io/projected/06dda089-0745-4941-adfe-2de711fce24e-kube-api-access-kmxk2\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:26 crc kubenswrapper[5052]: I1203 12:20:26.220157 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-58ns2\" (UniqueName: \"kubernetes.io/projected/966185e5-f414-46c5-ba48-0e234750795a-kube-api-access-58ns2\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:26 crc kubenswrapper[5052]: I1203 12:20:26.220166 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/966185e5-f414-46c5-ba48-0e234750795a-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:26 crc kubenswrapper[5052]: I1203 12:20:26.220175 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/06dda089-0745-4941-adfe-2de711fce24e-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:26 crc kubenswrapper[5052]: I1203 12:20:26.220183 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/06dda089-0745-4941-adfe-2de711fce24e-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:26 crc kubenswrapper[5052]: I1203 12:20:26.261015 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage02-crc" (UniqueName: "kubernetes.io/local-volume/local-storage02-crc") on node "crc" Dec 03 12:20:26 crc kubenswrapper[5052]: I1203 12:20:26.278393 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-db-sync-92z8b"] Dec 03 12:20:26 crc kubenswrapper[5052]: I1203 12:20:26.321727 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-kt8jm\" (UniqueName: \"kubernetes.io/projected/70cedf3b-c9b9-4e5c-81a2-ba134b150997-kube-api-access-kt8jm\") pod \"ovsdbserver-nb-0\" (UID: \"70cedf3b-c9b9-4e5c-81a2-ba134b150997\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:20:26 crc kubenswrapper[5052]: I1203 12:20:26.321790 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/70cedf3b-c9b9-4e5c-81a2-ba134b150997-ovsdb-rundir\") pod \"ovsdbserver-nb-0\" (UID: \"70cedf3b-c9b9-4e5c-81a2-ba134b150997\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:20:26 crc kubenswrapper[5052]: I1203 12:20:26.321819 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage20-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage20-crc\") pod \"ovsdbserver-nb-0\" (UID: \"70cedf3b-c9b9-4e5c-81a2-ba134b150997\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:20:26 crc kubenswrapper[5052]: I1203 12:20:26.321867 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovsdbserver-nb-tls-certs\" (UniqueName: \"kubernetes.io/secret/70cedf3b-c9b9-4e5c-81a2-ba134b150997-ovsdbserver-nb-tls-certs\") pod \"ovsdbserver-nb-0\" (UID: \"70cedf3b-c9b9-4e5c-81a2-ba134b150997\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:20:26 crc kubenswrapper[5052]: I1203 12:20:26.321905 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/70cedf3b-c9b9-4e5c-81a2-ba134b150997-metrics-certs-tls-certs\") pod \"ovsdbserver-nb-0\" (UID: \"70cedf3b-c9b9-4e5c-81a2-ba134b150997\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:20:26 crc kubenswrapper[5052]: I1203 12:20:26.321934 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/70cedf3b-c9b9-4e5c-81a2-ba134b150997-scripts\") pod \"ovsdbserver-nb-0\" (UID: \"70cedf3b-c9b9-4e5c-81a2-ba134b150997\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:20:26 crc kubenswrapper[5052]: I1203 12:20:26.321953 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/70cedf3b-c9b9-4e5c-81a2-ba134b150997-combined-ca-bundle\") pod \"ovsdbserver-nb-0\" (UID: \"70cedf3b-c9b9-4e5c-81a2-ba134b150997\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:20:26 crc kubenswrapper[5052]: I1203 12:20:26.321971 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/70cedf3b-c9b9-4e5c-81a2-ba134b150997-config\") pod \"ovsdbserver-nb-0\" (UID: \"70cedf3b-c9b9-4e5c-81a2-ba134b150997\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:20:26 crc kubenswrapper[5052]: I1203 12:20:26.322037 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage02-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage02-crc\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:26 crc kubenswrapper[5052]: I1203 12:20:26.322951 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/70cedf3b-c9b9-4e5c-81a2-ba134b150997-config\") pod \"ovsdbserver-nb-0\" (UID: \"70cedf3b-c9b9-4e5c-81a2-ba134b150997\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:20:26 crc kubenswrapper[5052]: I1203 12:20:26.328026 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/966185e5-f414-46c5-ba48-0e234750795a-ovn-northd-tls-certs" (OuterVolumeSpecName: "ovn-northd-tls-certs") pod "966185e5-f414-46c5-ba48-0e234750795a" (UID: "966185e5-f414-46c5-ba48-0e234750795a"). InnerVolumeSpecName "ovn-northd-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:20:26 crc kubenswrapper[5052]: I1203 12:20:26.328581 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/70cedf3b-c9b9-4e5c-81a2-ba134b150997-ovsdb-rundir\") pod \"ovsdbserver-nb-0\" (UID: \"70cedf3b-c9b9-4e5c-81a2-ba134b150997\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:20:26 crc kubenswrapper[5052]: I1203 12:20:26.326493 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/70cedf3b-c9b9-4e5c-81a2-ba134b150997-scripts\") pod \"ovsdbserver-nb-0\" (UID: \"70cedf3b-c9b9-4e5c-81a2-ba134b150997\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:20:26 crc kubenswrapper[5052]: I1203 12:20:26.329628 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage20-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage20-crc\") pod \"ovsdbserver-nb-0\" (UID: \"70cedf3b-c9b9-4e5c-81a2-ba134b150997\") device mount path \"/mnt/openstack/pv20\"" pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:20:26 crc kubenswrapper[5052]: I1203 12:20:26.343545 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/70cedf3b-c9b9-4e5c-81a2-ba134b150997-combined-ca-bundle\") pod \"ovsdbserver-nb-0\" (UID: \"70cedf3b-c9b9-4e5c-81a2-ba134b150997\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:20:26 crc kubenswrapper[5052]: I1203 12:20:26.344369 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/70cedf3b-c9b9-4e5c-81a2-ba134b150997-metrics-certs-tls-certs\") pod \"ovsdbserver-nb-0\" (UID: \"70cedf3b-c9b9-4e5c-81a2-ba134b150997\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:20:26 crc kubenswrapper[5052]: I1203 12:20:26.351992 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovsdbserver-nb-tls-certs\" (UniqueName: \"kubernetes.io/secret/70cedf3b-c9b9-4e5c-81a2-ba134b150997-ovsdbserver-nb-tls-certs\") pod \"ovsdbserver-nb-0\" (UID: \"70cedf3b-c9b9-4e5c-81a2-ba134b150997\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:20:26 crc kubenswrapper[5052]: I1203 12:20:26.368042 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/06dda089-0745-4941-adfe-2de711fce24e-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "06dda089-0745-4941-adfe-2de711fce24e" (UID: "06dda089-0745-4941-adfe-2de711fce24e"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:20:26 crc kubenswrapper[5052]: I1203 12:20:26.386558 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-kt8jm\" (UniqueName: \"kubernetes.io/projected/70cedf3b-c9b9-4e5c-81a2-ba134b150997-kube-api-access-kt8jm\") pod \"ovsdbserver-nb-0\" (UID: \"70cedf3b-c9b9-4e5c-81a2-ba134b150997\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:20:26 crc kubenswrapper[5052]: I1203 12:20:26.426323 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/06dda089-0745-4941-adfe-2de711fce24e-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:26 crc kubenswrapper[5052]: I1203 12:20:26.426367 5052 reconciler_common.go:293] "Volume detached for volume \"ovn-northd-tls-certs\" (UniqueName: \"kubernetes.io/secret/966185e5-f414-46c5-ba48-0e234750795a-ovn-northd-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:26 crc kubenswrapper[5052]: I1203 12:20:26.507884 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_ovn-northd-0_966185e5-f414-46c5-ba48-0e234750795a/ovn-northd/0.log" Dec 03 12:20:26 crc kubenswrapper[5052]: I1203 12:20:26.508445 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovn-northd-0" event={"ID":"966185e5-f414-46c5-ba48-0e234750795a","Type":"ContainerDied","Data":"d95de27c91a991d126bd6b66ff7ca758270a3709bf07db38f0dd347282027b49"} Dec 03 12:20:26 crc kubenswrapper[5052]: I1203 12:20:26.508487 5052 scope.go:117] "RemoveContainer" containerID="da2e79425aa667a3b6c86223531989383a59d7905be7837190a7bae4c94575bc" Dec 03 12:20:26 crc kubenswrapper[5052]: I1203 12:20:26.508657 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:20:26 crc kubenswrapper[5052]: I1203 12:20:26.526689 5052 prober.go:107] "Probe failed" probeType="Liveness" pod="openstack-kuttl-tests/cinder-scheduler-0" podUID="d5750ef1-ce21-4092-9612-13d6f6c923df" containerName="cinder-scheduler" probeResult="failure" output="HTTP probe failed with statuscode: 500" Dec 03 12:20:26 crc kubenswrapper[5052]: I1203 12:20:26.531520 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-db-sync-lrqq6" event={"ID":"94ac1a3b-14db-445c-8fb8-8bc3069bdb57","Type":"ContainerStarted","Data":"3aec26badb36af30719176457ba5cb8ca7a4acad4e10744afee0c09a24f1da70"} Dec 03 12:20:26 crc kubenswrapper[5052]: I1203 12:20:26.546107 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/memcached-0" event={"ID":"695de243-96a9-4e8b-a0f3-3a4ef2d95efe","Type":"ContainerDied","Data":"2606247e9ea56d0251c3c9bb611d0f13abafeb48627eff1f5e998fe4d509b2c2"} Dec 03 12:20:26 crc kubenswrapper[5052]: I1203 12:20:26.546151 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2606247e9ea56d0251c3c9bb611d0f13abafeb48627eff1f5e998fe4d509b2c2" Dec 03 12:20:26 crc kubenswrapper[5052]: I1203 12:20:26.572945 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_ovsdbserver-sb-0_06dda089-0745-4941-adfe-2de711fce24e/ovsdbserver-sb/0.log" Dec 03 12:20:26 crc kubenswrapper[5052]: I1203 12:20:26.573062 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-sb-0" event={"ID":"06dda089-0745-4941-adfe-2de711fce24e","Type":"ContainerDied","Data":"e5b10cce0a408072de6a69243553e83730687f6a07a501c281056171959831aa"} Dec 03 12:20:26 crc kubenswrapper[5052]: I1203 12:20:26.573166 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:20:26 crc kubenswrapper[5052]: I1203 12:20:26.604801 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-db-sync-92z8b" event={"ID":"7a9497b1-89e2-4ebc-97e6-2bdaeefab95f","Type":"ContainerStarted","Data":"bdb59561dc5e2ad5e9132a7b7a181f019391ce3c7182ce50b77e73b222297b71"} Dec 03 12:20:26 crc kubenswrapper[5052]: I1203 12:20:26.620693 5052 generic.go:334] "Generic (PLEG): container finished" podID="a56992d9-4662-444f-8a45-0375d0855946" containerID="8dfc8c0fb51ed3361082e5d6a21d3480abe509b00c224f35e46b8ddb4b3e7415" exitCode=0 Dec 03 12:20:26 crc kubenswrapper[5052]: I1203 12:20:26.620952 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-cell1-galera-0" event={"ID":"a56992d9-4662-444f-8a45-0375d0855946","Type":"ContainerDied","Data":"8dfc8c0fb51ed3361082e5d6a21d3480abe509b00c224f35e46b8ddb4b3e7415"} Dec 03 12:20:26 crc kubenswrapper[5052]: I1203 12:20:26.623131 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/swift-ring-rebalance-z7lpm"] Dec 03 12:20:26 crc kubenswrapper[5052]: I1203 12:20:26.634889 5052 generic.go:334] "Generic (PLEG): container finished" podID="42ebb0e3-9a37-4898-a1e4-0822b4bd9e1e" containerID="1a78431f63106b64cc7a1e9a2171a9a34a9537578480657d5f058f3cfe1f49bd" exitCode=0 Dec 03 12:20:26 crc kubenswrapper[5052]: I1203 12:20:26.635110 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" event={"ID":"42ebb0e3-9a37-4898-a1e4-0822b4bd9e1e","Type":"ContainerDied","Data":"1a78431f63106b64cc7a1e9a2171a9a34a9537578480657d5f058f3cfe1f49bd"} Dec 03 12:20:26 crc kubenswrapper[5052]: I1203 12:20:26.716548 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/06dda089-0745-4941-adfe-2de711fce24e-ovsdbserver-sb-tls-certs" (OuterVolumeSpecName: "ovsdbserver-sb-tls-certs") pod "06dda089-0745-4941-adfe-2de711fce24e" (UID: "06dda089-0745-4941-adfe-2de711fce24e"). InnerVolumeSpecName "ovsdbserver-sb-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:20:26 crc kubenswrapper[5052]: I1203 12:20:26.734163 5052 reconciler_common.go:293] "Volume detached for volume \"ovsdbserver-sb-tls-certs\" (UniqueName: \"kubernetes.io/secret/06dda089-0745-4941-adfe-2de711fce24e-ovsdbserver-sb-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:26 crc kubenswrapper[5052]: I1203 12:20:26.758122 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage20-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage20-crc\") pod \"ovsdbserver-nb-0\" (UID: \"70cedf3b-c9b9-4e5c-81a2-ba134b150997\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:20:26 crc kubenswrapper[5052]: I1203 12:20:26.793364 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/966185e5-f414-46c5-ba48-0e234750795a-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "966185e5-f414-46c5-ba48-0e234750795a" (UID: "966185e5-f414-46c5-ba48-0e234750795a"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:20:26 crc kubenswrapper[5052]: I1203 12:20:26.833477 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-db-sync-r444h"] Dec 03 12:20:26 crc kubenswrapper[5052]: I1203 12:20:26.836022 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/966185e5-f414-46c5-ba48-0e234750795a-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:26 crc kubenswrapper[5052]: I1203 12:20:26.856074 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/06dda089-0745-4941-adfe-2de711fce24e-metrics-certs-tls-certs" (OuterVolumeSpecName: "metrics-certs-tls-certs") pod "06dda089-0745-4941-adfe-2de711fce24e" (UID: "06dda089-0745-4941-adfe-2de711fce24e"). InnerVolumeSpecName "metrics-certs-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:20:26 crc kubenswrapper[5052]: I1203 12:20:26.874541 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/966185e5-f414-46c5-ba48-0e234750795a-metrics-certs-tls-certs" (OuterVolumeSpecName: "metrics-certs-tls-certs") pod "966185e5-f414-46c5-ba48-0e234750795a" (UID: "966185e5-f414-46c5-ba48-0e234750795a"). InnerVolumeSpecName "metrics-certs-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:20:26 crc kubenswrapper[5052]: I1203 12:20:26.901045 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/memcached-0" Dec 03 12:20:26 crc kubenswrapper[5052]: I1203 12:20:26.920710 5052 scope.go:117] "RemoveContainer" containerID="ef532f5fb83b3ed8c9243e5c20875e68a9ee6b646ee961f30ffe04c5b504e0f8" Dec 03 12:20:26 crc kubenswrapper[5052]: I1203 12:20:26.942661 5052 reconciler_common.go:293] "Volume detached for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/06dda089-0745-4941-adfe-2de711fce24e-metrics-certs-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:26 crc kubenswrapper[5052]: I1203 12:20:26.942708 5052 reconciler_common.go:293] "Volume detached for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/966185e5-f414-46c5-ba48-0e234750795a-metrics-certs-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:26 crc kubenswrapper[5052]: I1203 12:20:26.943626 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:20:26 crc kubenswrapper[5052]: I1203 12:20:26.959927 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:20:26 crc kubenswrapper[5052]: I1203 12:20:26.991675 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:20:27 crc kubenswrapper[5052]: I1203 12:20:27.055176 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/695de243-96a9-4e8b-a0f3-3a4ef2d95efe-config-data\") pod \"695de243-96a9-4e8b-a0f3-3a4ef2d95efe\" (UID: \"695de243-96a9-4e8b-a0f3-3a4ef2d95efe\") " Dec 03 12:20:27 crc kubenswrapper[5052]: I1203 12:20:27.055277 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-state-metrics-tls-certs\" (UniqueName: \"kubernetes.io/secret/42e93b9b-3a58-4ca6-abc6-6e359b15e7cc-kube-state-metrics-tls-certs\") pod \"42e93b9b-3a58-4ca6-abc6-6e359b15e7cc\" (UID: \"42e93b9b-3a58-4ca6-abc6-6e359b15e7cc\") " Dec 03 12:20:27 crc kubenswrapper[5052]: I1203 12:20:27.055348 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"memcached-tls-certs\" (UniqueName: \"kubernetes.io/secret/695de243-96a9-4e8b-a0f3-3a4ef2d95efe-memcached-tls-certs\") pod \"695de243-96a9-4e8b-a0f3-3a4ef2d95efe\" (UID: \"695de243-96a9-4e8b-a0f3-3a4ef2d95efe\") " Dec 03 12:20:27 crc kubenswrapper[5052]: I1203 12:20:27.055393 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"nova-novncproxy-tls-certs\" (UniqueName: \"kubernetes.io/secret/42ebb0e3-9a37-4898-a1e4-0822b4bd9e1e-nova-novncproxy-tls-certs\") pod \"42ebb0e3-9a37-4898-a1e4-0822b4bd9e1e\" (UID: \"42ebb0e3-9a37-4898-a1e4-0822b4bd9e1e\") " Dec 03 12:20:27 crc kubenswrapper[5052]: I1203 12:20:27.055433 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kfc9r\" (UniqueName: \"kubernetes.io/projected/42ebb0e3-9a37-4898-a1e4-0822b4bd9e1e-kube-api-access-kfc9r\") pod \"42ebb0e3-9a37-4898-a1e4-0822b4bd9e1e\" (UID: \"42ebb0e3-9a37-4898-a1e4-0822b4bd9e1e\") " Dec 03 12:20:27 crc kubenswrapper[5052]: I1203 12:20:27.055469 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-state-metrics-tls-config\" (UniqueName: \"kubernetes.io/secret/42e93b9b-3a58-4ca6-abc6-6e359b15e7cc-kube-state-metrics-tls-config\") pod \"42e93b9b-3a58-4ca6-abc6-6e359b15e7cc\" (UID: \"42e93b9b-3a58-4ca6-abc6-6e359b15e7cc\") " Dec 03 12:20:27 crc kubenswrapper[5052]: I1203 12:20:27.055532 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ml58r\" (UniqueName: \"kubernetes.io/projected/695de243-96a9-4e8b-a0f3-3a4ef2d95efe-kube-api-access-ml58r\") pod \"695de243-96a9-4e8b-a0f3-3a4ef2d95efe\" (UID: \"695de243-96a9-4e8b-a0f3-3a4ef2d95efe\") " Dec 03 12:20:27 crc kubenswrapper[5052]: I1203 12:20:27.055577 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/42ebb0e3-9a37-4898-a1e4-0822b4bd9e1e-config-data\") pod \"42ebb0e3-9a37-4898-a1e4-0822b4bd9e1e\" (UID: \"42ebb0e3-9a37-4898-a1e4-0822b4bd9e1e\") " Dec 03 12:20:27 crc kubenswrapper[5052]: I1203 12:20:27.055604 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/695de243-96a9-4e8b-a0f3-3a4ef2d95efe-kolla-config\") pod \"695de243-96a9-4e8b-a0f3-3a4ef2d95efe\" (UID: \"695de243-96a9-4e8b-a0f3-3a4ef2d95efe\") " Dec 03 12:20:27 crc kubenswrapper[5052]: I1203 12:20:27.055635 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kvvr6\" (UniqueName: \"kubernetes.io/projected/42e93b9b-3a58-4ca6-abc6-6e359b15e7cc-kube-api-access-kvvr6\") pod \"42e93b9b-3a58-4ca6-abc6-6e359b15e7cc\" (UID: \"42e93b9b-3a58-4ca6-abc6-6e359b15e7cc\") " Dec 03 12:20:27 crc kubenswrapper[5052]: I1203 12:20:27.055683 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/42ebb0e3-9a37-4898-a1e4-0822b4bd9e1e-combined-ca-bundle\") pod \"42ebb0e3-9a37-4898-a1e4-0822b4bd9e1e\" (UID: \"42ebb0e3-9a37-4898-a1e4-0822b4bd9e1e\") " Dec 03 12:20:27 crc kubenswrapper[5052]: I1203 12:20:27.055743 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"vencrypt-tls-certs\" (UniqueName: \"kubernetes.io/secret/42ebb0e3-9a37-4898-a1e4-0822b4bd9e1e-vencrypt-tls-certs\") pod \"42ebb0e3-9a37-4898-a1e4-0822b4bd9e1e\" (UID: \"42ebb0e3-9a37-4898-a1e4-0822b4bd9e1e\") " Dec 03 12:20:27 crc kubenswrapper[5052]: I1203 12:20:27.055781 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/695de243-96a9-4e8b-a0f3-3a4ef2d95efe-combined-ca-bundle\") pod \"695de243-96a9-4e8b-a0f3-3a4ef2d95efe\" (UID: \"695de243-96a9-4e8b-a0f3-3a4ef2d95efe\") " Dec 03 12:20:27 crc kubenswrapper[5052]: I1203 12:20:27.055823 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/42e93b9b-3a58-4ca6-abc6-6e359b15e7cc-combined-ca-bundle\") pod \"42e93b9b-3a58-4ca6-abc6-6e359b15e7cc\" (UID: \"42e93b9b-3a58-4ca6-abc6-6e359b15e7cc\") " Dec 03 12:20:27 crc kubenswrapper[5052]: I1203 12:20:27.057626 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/695de243-96a9-4e8b-a0f3-3a4ef2d95efe-config-data" (OuterVolumeSpecName: "config-data") pod "695de243-96a9-4e8b-a0f3-3a4ef2d95efe" (UID: "695de243-96a9-4e8b-a0f3-3a4ef2d95efe"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:20:27 crc kubenswrapper[5052]: I1203 12:20:27.076961 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/695de243-96a9-4e8b-a0f3-3a4ef2d95efe-kolla-config" (OuterVolumeSpecName: "kolla-config") pod "695de243-96a9-4e8b-a0f3-3a4ef2d95efe" (UID: "695de243-96a9-4e8b-a0f3-3a4ef2d95efe"). InnerVolumeSpecName "kolla-config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:20:27 crc kubenswrapper[5052]: I1203 12:20:27.115986 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/695de243-96a9-4e8b-a0f3-3a4ef2d95efe-kube-api-access-ml58r" (OuterVolumeSpecName: "kube-api-access-ml58r") pod "695de243-96a9-4e8b-a0f3-3a4ef2d95efe" (UID: "695de243-96a9-4e8b-a0f3-3a4ef2d95efe"). InnerVolumeSpecName "kube-api-access-ml58r". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:20:27 crc kubenswrapper[5052]: I1203 12:20:27.129854 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/42e93b9b-3a58-4ca6-abc6-6e359b15e7cc-kube-api-access-kvvr6" (OuterVolumeSpecName: "kube-api-access-kvvr6") pod "42e93b9b-3a58-4ca6-abc6-6e359b15e7cc" (UID: "42e93b9b-3a58-4ca6-abc6-6e359b15e7cc"). InnerVolumeSpecName "kube-api-access-kvvr6". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:20:27 crc kubenswrapper[5052]: I1203 12:20:27.157916 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-ml58r\" (UniqueName: \"kubernetes.io/projected/695de243-96a9-4e8b-a0f3-3a4ef2d95efe-kube-api-access-ml58r\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:27 crc kubenswrapper[5052]: I1203 12:20:27.158149 5052 reconciler_common.go:293] "Volume detached for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/695de243-96a9-4e8b-a0f3-3a4ef2d95efe-kolla-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:27 crc kubenswrapper[5052]: I1203 12:20:27.158230 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-kvvr6\" (UniqueName: \"kubernetes.io/projected/42e93b9b-3a58-4ca6-abc6-6e359b15e7cc-kube-api-access-kvvr6\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:27 crc kubenswrapper[5052]: I1203 12:20:27.158334 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/695de243-96a9-4e8b-a0f3-3a4ef2d95efe-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:27 crc kubenswrapper[5052]: I1203 12:20:27.178238 5052 scope.go:117] "RemoveContainer" containerID="33dc5ae74629ef4a73b20f4631f0f9178eee3dfb4459965f03d5530f56c3e84d" Dec 03 12:20:27 crc kubenswrapper[5052]: I1203 12:20:27.209590 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-db-sync-hjwg6"] Dec 03 12:20:27 crc kubenswrapper[5052]: I1203 12:20:27.386723 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/42ebb0e3-9a37-4898-a1e4-0822b4bd9e1e-kube-api-access-kfc9r" (OuterVolumeSpecName: "kube-api-access-kfc9r") pod "42ebb0e3-9a37-4898-a1e4-0822b4bd9e1e" (UID: "42ebb0e3-9a37-4898-a1e4-0822b4bd9e1e"). InnerVolumeSpecName "kube-api-access-kfc9r". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:20:27 crc kubenswrapper[5052]: I1203 12:20:27.469984 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-kfc9r\" (UniqueName: \"kubernetes.io/projected/42ebb0e3-9a37-4898-a1e4-0822b4bd9e1e-kube-api-access-kfc9r\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:27 crc kubenswrapper[5052]: W1203 12:20:27.530942 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9ea78435_8fd9_4321_9e48_3aaae833dcf2.slice/crio-2d570bd0295bdf63ea3bd8c425b3cd03f0e2e31ab814b356062540ad27c5917b WatchSource:0}: Error finding container 2d570bd0295bdf63ea3bd8c425b3cd03f0e2e31ab814b356062540ad27c5917b: Status 404 returned error can't find the container with id 2d570bd0295bdf63ea3bd8c425b3cd03f0e2e31ab814b356062540ad27c5917b Dec 03 12:20:27 crc kubenswrapper[5052]: I1203 12:20:27.704836 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fcd60977-d995-42ee-b654-ba2947a3316d" path="/var/lib/kubelet/pods/fcd60977-d995-42ee-b654-ba2947a3316d/volumes" Dec 03 12:20:27 crc kubenswrapper[5052]: I1203 12:20:27.705688 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:20:27 crc kubenswrapper[5052]: I1203 12:20:27.712332 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/memcached-0" Dec 03 12:20:27 crc kubenswrapper[5052]: I1203 12:20:27.712676 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:20:28 crc kubenswrapper[5052]: I1203 12:20:28.012254 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/42e93b9b-3a58-4ca6-abc6-6e359b15e7cc-kube-state-metrics-tls-config" (OuterVolumeSpecName: "kube-state-metrics-tls-config") pod "42e93b9b-3a58-4ca6-abc6-6e359b15e7cc" (UID: "42e93b9b-3a58-4ca6-abc6-6e359b15e7cc"). InnerVolumeSpecName "kube-state-metrics-tls-config". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:20:28 crc kubenswrapper[5052]: I1203 12:20:28.095429 5052 reconciler_common.go:293] "Volume detached for volume \"kube-state-metrics-tls-config\" (UniqueName: \"kubernetes.io/secret/42e93b9b-3a58-4ca6-abc6-6e359b15e7cc-kube-state-metrics-tls-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:28 crc kubenswrapper[5052]: I1203 12:20:28.133754 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/42ebb0e3-9a37-4898-a1e4-0822b4bd9e1e-config-data" (OuterVolumeSpecName: "config-data") pod "42ebb0e3-9a37-4898-a1e4-0822b4bd9e1e" (UID: "42ebb0e3-9a37-4898-a1e4-0822b4bd9e1e"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:20:28 crc kubenswrapper[5052]: I1203 12:20:28.199561 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/42ebb0e3-9a37-4898-a1e4-0822b4bd9e1e-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:28 crc kubenswrapper[5052]: I1203 12:20:28.258641 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/695de243-96a9-4e8b-a0f3-3a4ef2d95efe-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "695de243-96a9-4e8b-a0f3-3a4ef2d95efe" (UID: "695de243-96a9-4e8b-a0f3-3a4ef2d95efe"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:20:28 crc kubenswrapper[5052]: I1203 12:20:28.293864 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/42ebb0e3-9a37-4898-a1e4-0822b4bd9e1e-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "42ebb0e3-9a37-4898-a1e4-0822b4bd9e1e" (UID: "42ebb0e3-9a37-4898-a1e4-0822b4bd9e1e"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:20:28 crc kubenswrapper[5052]: I1203 12:20:28.297762 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/42e93b9b-3a58-4ca6-abc6-6e359b15e7cc-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "42e93b9b-3a58-4ca6-abc6-6e359b15e7cc" (UID: "42e93b9b-3a58-4ca6-abc6-6e359b15e7cc"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:20:28 crc kubenswrapper[5052]: I1203 12:20:28.303619 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/42ebb0e3-9a37-4898-a1e4-0822b4bd9e1e-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:28 crc kubenswrapper[5052]: I1203 12:20:28.303646 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/695de243-96a9-4e8b-a0f3-3a4ef2d95efe-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:28 crc kubenswrapper[5052]: I1203 12:20:28.303655 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/42e93b9b-3a58-4ca6-abc6-6e359b15e7cc-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:28 crc kubenswrapper[5052]: I1203 12:20:28.305300 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/42ebb0e3-9a37-4898-a1e4-0822b4bd9e1e-vencrypt-tls-certs" (OuterVolumeSpecName: "vencrypt-tls-certs") pod "42ebb0e3-9a37-4898-a1e4-0822b4bd9e1e" (UID: "42ebb0e3-9a37-4898-a1e4-0822b4bd9e1e"). InnerVolumeSpecName "vencrypt-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:20:28 crc kubenswrapper[5052]: I1203 12:20:28.320738 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/42e93b9b-3a58-4ca6-abc6-6e359b15e7cc-kube-state-metrics-tls-certs" (OuterVolumeSpecName: "kube-state-metrics-tls-certs") pod "42e93b9b-3a58-4ca6-abc6-6e359b15e7cc" (UID: "42e93b9b-3a58-4ca6-abc6-6e359b15e7cc"). InnerVolumeSpecName "kube-state-metrics-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:20:28 crc kubenswrapper[5052]: I1203 12:20:28.355286 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/695de243-96a9-4e8b-a0f3-3a4ef2d95efe-memcached-tls-certs" (OuterVolumeSpecName: "memcached-tls-certs") pod "695de243-96a9-4e8b-a0f3-3a4ef2d95efe" (UID: "695de243-96a9-4e8b-a0f3-3a4ef2d95efe"). InnerVolumeSpecName "memcached-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:20:28 crc kubenswrapper[5052]: I1203 12:20:28.375002 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/42ebb0e3-9a37-4898-a1e4-0822b4bd9e1e-nova-novncproxy-tls-certs" (OuterVolumeSpecName: "nova-novncproxy-tls-certs") pod "42ebb0e3-9a37-4898-a1e4-0822b4bd9e1e" (UID: "42ebb0e3-9a37-4898-a1e4-0822b4bd9e1e"). InnerVolumeSpecName "nova-novncproxy-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:20:28 crc kubenswrapper[5052]: I1203 12:20:28.407854 5052 reconciler_common.go:293] "Volume detached for volume \"vencrypt-tls-certs\" (UniqueName: \"kubernetes.io/secret/42ebb0e3-9a37-4898-a1e4-0822b4bd9e1e-vencrypt-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:28 crc kubenswrapper[5052]: I1203 12:20:28.408124 5052 reconciler_common.go:293] "Volume detached for volume \"kube-state-metrics-tls-certs\" (UniqueName: \"kubernetes.io/secret/42e93b9b-3a58-4ca6-abc6-6e359b15e7cc-kube-state-metrics-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:28 crc kubenswrapper[5052]: I1203 12:20:28.408202 5052 reconciler_common.go:293] "Volume detached for volume \"memcached-tls-certs\" (UniqueName: \"kubernetes.io/secret/695de243-96a9-4e8b-a0f3-3a4ef2d95efe-memcached-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:28 crc kubenswrapper[5052]: I1203 12:20:28.408307 5052 reconciler_common.go:293] "Volume detached for volume \"nova-novncproxy-tls-certs\" (UniqueName: \"kubernetes.io/secret/42ebb0e3-9a37-4898-a1e4-0822b4bd9e1e-nova-novncproxy-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:28 crc kubenswrapper[5052]: I1203 12:20:28.548026 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-hjwg6" event={"ID":"9ea78435-8fd9-4321-9e48-3aaae833dcf2","Type":"ContainerStarted","Data":"2d570bd0295bdf63ea3bd8c425b3cd03f0e2e31ab814b356062540ad27c5917b"} Dec 03 12:20:28 crc kubenswrapper[5052]: I1203 12:20:28.548088 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-nb-0"] Dec 03 12:20:28 crc kubenswrapper[5052]: I1203 12:20:28.548124 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-cell1-galera-0" event={"ID":"a56992d9-4662-444f-8a45-0375d0855946","Type":"ContainerDied","Data":"6ea97c6fa066d8cad9a0a91f87604b718a8dd9dbe2d43a24094b211b878b10a6"} Dec 03 12:20:28 crc kubenswrapper[5052]: I1203 12:20:28.548167 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6ea97c6fa066d8cad9a0a91f87604b718a8dd9dbe2d43a24094b211b878b10a6" Dec 03 12:20:28 crc kubenswrapper[5052]: I1203 12:20:28.548182 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"ae23fe34-7bf4-47fc-addc-5edda0399c77","Type":"ContainerStarted","Data":"da65d715c37dc90aaa5503b209873b3a777e78c8e5cca7b7707528a1f1fc1b70"} Dec 03 12:20:28 crc kubenswrapper[5052]: I1203 12:20:28.548197 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-r444h" event={"ID":"cb824463-7efe-49ee-a052-acd1fd8f02d3","Type":"ContainerStarted","Data":"d1c0b9b946484cb4ba9ab688208b0b3936161b2c4dead77bc73efe64622e358b"} Dec 03 12:20:28 crc kubenswrapper[5052]: I1203 12:20:28.548209 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" event={"ID":"42ebb0e3-9a37-4898-a1e4-0822b4bd9e1e","Type":"ContainerDied","Data":"3f856f59f54e0aa64029214150a2dd788c92dbcde263e65e41d3708a5652e283"} Dec 03 12:20:28 crc kubenswrapper[5052]: I1203 12:20:28.548227 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-ring-rebalance-z7lpm" event={"ID":"318545bf-5895-48f9-9649-67207bc4b207","Type":"ContainerStarted","Data":"f696cb70d675a723254603c90b2d5ef021170f5f1bf709cae94ef8a50cfa63b1"} Dec 03 12:20:28 crc kubenswrapper[5052]: I1203 12:20:28.548240 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/kube-state-metrics-0" event={"ID":"42e93b9b-3a58-4ca6-abc6-6e359b15e7cc","Type":"ContainerDied","Data":"6e5c5ec695992cefa702e431184c66bb5a174101a594a1f0d3057db0a724a8ad"} Dec 03 12:20:28 crc kubenswrapper[5052]: I1203 12:20:28.575459 5052 scope.go:117] "RemoveContainer" containerID="4ce7e130ceeb669b4057b5b4f5937db4eabc9ded0e28a82765f352a1735b5239" Dec 03 12:20:28 crc kubenswrapper[5052]: I1203 12:20:28.733838 5052 generic.go:334] "Generic (PLEG): container finished" podID="5067bb8d-d668-46a6-b9de-8065266cde63" containerID="7c40f1aa68ee0e0f29ce1ab35070e6d55c530a232bd4bf5a962f9aa5aa656271" exitCode=0 Dec 03 12:20:28 crc kubenswrapper[5052]: I1203 12:20:28.733881 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-galera-0" event={"ID":"5067bb8d-d668-46a6-b9de-8065266cde63","Type":"ContainerDied","Data":"7c40f1aa68ee0e0f29ce1ab35070e6d55c530a232bd4bf5a962f9aa5aa656271"} Dec 03 12:20:28 crc kubenswrapper[5052]: I1203 12:20:28.734215 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-galera-0" event={"ID":"5067bb8d-d668-46a6-b9de-8065266cde63","Type":"ContainerDied","Data":"0472197b95995bed984ae7255d7900169304c573d7fae477d9858f5fa19b9b6d"} Dec 03 12:20:28 crc kubenswrapper[5052]: I1203 12:20:28.734228 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0472197b95995bed984ae7255d7900169304c573d7fae477d9858f5fa19b9b6d" Dec 03 12:20:28 crc kubenswrapper[5052]: I1203 12:20:28.740966 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-nb-0" event={"ID":"70cedf3b-c9b9-4e5c-81a2-ba134b150997","Type":"ContainerStarted","Data":"b64aba72338acf63ff50b518818d5d916ca0c4e6b7b9793a282a7970c1cba540"} Dec 03 12:20:28 crc kubenswrapper[5052]: I1203 12:20:28.750881 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-db-sync-pvgq9" event={"ID":"66f5168a-1ee5-4de9-8e50-90ef25186ee8","Type":"ContainerStarted","Data":"fcd7a92adc3ca1e7e96f3a9c20674d72f6b8c11f7f6f67a8af2cf51365e8fcb1"} Dec 03 12:20:28 crc kubenswrapper[5052]: I1203 12:20:28.771325 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-db-sync-lrqq6" event={"ID":"94ac1a3b-14db-445c-8fb8-8bc3069bdb57","Type":"ContainerStarted","Data":"e47bb47cae9cbe2f35ecd613fb6be4fc2447cc9124af569f2f18e065e8af2171"} Dec 03 12:20:28 crc kubenswrapper[5052]: I1203 12:20:28.777632 5052 scope.go:117] "RemoveContainer" containerID="1a78431f63106b64cc7a1e9a2171a9a34a9537578480657d5f058f3cfe1f49bd" Dec 03 12:20:28 crc kubenswrapper[5052]: E1203 12:20:28.781940 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4ce7e130ceeb669b4057b5b4f5937db4eabc9ded0e28a82765f352a1735b5239\": container with ID starting with 4ce7e130ceeb669b4057b5b4f5937db4eabc9ded0e28a82765f352a1735b5239 not found: ID does not exist" containerID="4ce7e130ceeb669b4057b5b4f5937db4eabc9ded0e28a82765f352a1735b5239" Dec 03 12:20:28 crc kubenswrapper[5052]: I1203 12:20:28.785791 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-db-sync-gqt2p" event={"ID":"c4487b92-6d46-4cde-b1c3-056ac6fa9753","Type":"ContainerStarted","Data":"d21edbe94c25709bf834fc9a057d971ffaeed3e9a181d6a8bd2dc198291496be"} Dec 03 12:20:28 crc kubenswrapper[5052]: I1203 12:20:28.786368 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/glance-db-sync-pvgq9" podStartSLOduration=6.786352562 podStartE2EDuration="6.786352562s" podCreationTimestamp="2025-12-03 12:20:22 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:20:28.778049095 +0000 UTC m=+3619.366290859" watchObservedRunningTime="2025-12-03 12:20:28.786352562 +0000 UTC m=+3619.374594326" Dec 03 12:20:28 crc kubenswrapper[5052]: I1203 12:20:28.805931 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:20:28 crc kubenswrapper[5052]: I1203 12:20:28.846874 5052 scope.go:117] "RemoveContainer" containerID="c572ac5c2d472c1297b5500f42d80ab53aad7c4f8d35a45f1ad28447e9dc6639" Dec 03 12:20:28 crc kubenswrapper[5052]: I1203 12:20:28.856065 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/neutron-db-sync-lrqq6" podStartSLOduration=6.856043111 podStartE2EDuration="6.856043111s" podCreationTimestamp="2025-12-03 12:20:22 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:20:28.81643401 +0000 UTC m=+3619.404675774" watchObservedRunningTime="2025-12-03 12:20:28.856043111 +0000 UTC m=+3619.444284875" Dec 03 12:20:28 crc kubenswrapper[5052]: I1203 12:20:28.877569 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ovn-northd-0"] Dec 03 12:20:28 crc kubenswrapper[5052]: I1203 12:20:28.908876 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ovn-northd-0"] Dec 03 12:20:28 crc kubenswrapper[5052]: I1203 12:20:28.926754 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ovn-northd-0"] Dec 03 12:20:28 crc kubenswrapper[5052]: E1203 12:20:28.927364 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="42e93b9b-3a58-4ca6-abc6-6e359b15e7cc" containerName="kube-state-metrics" Dec 03 12:20:28 crc kubenswrapper[5052]: I1203 12:20:28.927386 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="42e93b9b-3a58-4ca6-abc6-6e359b15e7cc" containerName="kube-state-metrics" Dec 03 12:20:28 crc kubenswrapper[5052]: E1203 12:20:28.927459 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="42ebb0e3-9a37-4898-a1e4-0822b4bd9e1e" containerName="nova-cell1-novncproxy-novncproxy" Dec 03 12:20:28 crc kubenswrapper[5052]: I1203 12:20:28.927470 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="42ebb0e3-9a37-4898-a1e4-0822b4bd9e1e" containerName="nova-cell1-novncproxy-novncproxy" Dec 03 12:20:28 crc kubenswrapper[5052]: E1203 12:20:28.927527 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="a56992d9-4662-444f-8a45-0375d0855946" containerName="galera" Dec 03 12:20:28 crc kubenswrapper[5052]: I1203 12:20:28.927537 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="a56992d9-4662-444f-8a45-0375d0855946" containerName="galera" Dec 03 12:20:28 crc kubenswrapper[5052]: E1203 12:20:28.927560 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="a56992d9-4662-444f-8a45-0375d0855946" containerName="mysql-bootstrap" Dec 03 12:20:28 crc kubenswrapper[5052]: I1203 12:20:28.927573 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="a56992d9-4662-444f-8a45-0375d0855946" containerName="mysql-bootstrap" Dec 03 12:20:28 crc kubenswrapper[5052]: E1203 12:20:28.927583 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="695de243-96a9-4e8b-a0f3-3a4ef2d95efe" containerName="memcached" Dec 03 12:20:28 crc kubenswrapper[5052]: I1203 12:20:28.927594 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="695de243-96a9-4e8b-a0f3-3a4ef2d95efe" containerName="memcached" Dec 03 12:20:28 crc kubenswrapper[5052]: I1203 12:20:28.927921 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="695de243-96a9-4e8b-a0f3-3a4ef2d95efe" containerName="memcached" Dec 03 12:20:28 crc kubenswrapper[5052]: I1203 12:20:28.927953 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="42e93b9b-3a58-4ca6-abc6-6e359b15e7cc" containerName="kube-state-metrics" Dec 03 12:20:28 crc kubenswrapper[5052]: I1203 12:20:28.927963 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="a56992d9-4662-444f-8a45-0375d0855946" containerName="galera" Dec 03 12:20:28 crc kubenswrapper[5052]: I1203 12:20:28.927982 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="42ebb0e3-9a37-4898-a1e4-0822b4bd9e1e" containerName="nova-cell1-novncproxy-novncproxy" Dec 03 12:20:28 crc kubenswrapper[5052]: I1203 12:20:28.929317 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:20:28 crc kubenswrapper[5052]: I1203 12:20:28.934944 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/a56992d9-4662-444f-8a45-0375d0855946-config-data-default\") pod \"a56992d9-4662-444f-8a45-0375d0855946\" (UID: \"a56992d9-4662-444f-8a45-0375d0855946\") " Dec 03 12:20:28 crc kubenswrapper[5052]: I1203 12:20:28.935020 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/a56992d9-4662-444f-8a45-0375d0855946-config-data-generated\") pod \"a56992d9-4662-444f-8a45-0375d0855946\" (UID: \"a56992d9-4662-444f-8a45-0375d0855946\") " Dec 03 12:20:28 crc kubenswrapper[5052]: I1203 12:20:28.935075 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/a56992d9-4662-444f-8a45-0375d0855946-operator-scripts\") pod \"a56992d9-4662-444f-8a45-0375d0855946\" (UID: \"a56992d9-4662-444f-8a45-0375d0855946\") " Dec 03 12:20:28 crc kubenswrapper[5052]: I1203 12:20:28.935109 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a56992d9-4662-444f-8a45-0375d0855946-combined-ca-bundle\") pod \"a56992d9-4662-444f-8a45-0375d0855946\" (UID: \"a56992d9-4662-444f-8a45-0375d0855946\") " Dec 03 12:20:28 crc kubenswrapper[5052]: I1203 12:20:28.935169 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/a56992d9-4662-444f-8a45-0375d0855946-kolla-config\") pod \"a56992d9-4662-444f-8a45-0375d0855946\" (UID: \"a56992d9-4662-444f-8a45-0375d0855946\") " Dec 03 12:20:28 crc kubenswrapper[5052]: I1203 12:20:28.935209 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4tjl2\" (UniqueName: \"kubernetes.io/projected/a56992d9-4662-444f-8a45-0375d0855946-kube-api-access-4tjl2\") pod \"a56992d9-4662-444f-8a45-0375d0855946\" (UID: \"a56992d9-4662-444f-8a45-0375d0855946\") " Dec 03 12:20:28 crc kubenswrapper[5052]: I1203 12:20:28.935311 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/a56992d9-4662-444f-8a45-0375d0855946-galera-tls-certs\") pod \"a56992d9-4662-444f-8a45-0375d0855946\" (UID: \"a56992d9-4662-444f-8a45-0375d0855946\") " Dec 03 12:20:28 crc kubenswrapper[5052]: I1203 12:20:28.935340 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"mysql-db\" (UniqueName: \"kubernetes.io/local-volume/local-storage06-crc\") pod \"a56992d9-4662-444f-8a45-0375d0855946\" (UID: \"a56992d9-4662-444f-8a45-0375d0855946\") " Dec 03 12:20:28 crc kubenswrapper[5052]: I1203 12:20:28.937025 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a56992d9-4662-444f-8a45-0375d0855946-config-data-default" (OuterVolumeSpecName: "config-data-default") pod "a56992d9-4662-444f-8a45-0375d0855946" (UID: "a56992d9-4662-444f-8a45-0375d0855946"). InnerVolumeSpecName "config-data-default". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:20:28 crc kubenswrapper[5052]: I1203 12:20:28.937683 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a56992d9-4662-444f-8a45-0375d0855946-config-data-generated" (OuterVolumeSpecName: "config-data-generated") pod "a56992d9-4662-444f-8a45-0375d0855946" (UID: "a56992d9-4662-444f-8a45-0375d0855946"). InnerVolumeSpecName "config-data-generated". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:20:28 crc kubenswrapper[5052]: I1203 12:20:28.938388 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a56992d9-4662-444f-8a45-0375d0855946-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "a56992d9-4662-444f-8a45-0375d0855946" (UID: "a56992d9-4662-444f-8a45-0375d0855946"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:20:28 crc kubenswrapper[5052]: I1203 12:20:28.954152 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a56992d9-4662-444f-8a45-0375d0855946-kolla-config" (OuterVolumeSpecName: "kolla-config") pod "a56992d9-4662-444f-8a45-0375d0855946" (UID: "a56992d9-4662-444f-8a45-0375d0855946"). InnerVolumeSpecName "kolla-config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:20:28 crc kubenswrapper[5052]: I1203 12:20:28.954935 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-ovnnorthd-ovndbs" Dec 03 12:20:28 crc kubenswrapper[5052]: I1203 12:20:28.954984 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"ovnnorthd-config" Dec 03 12:20:28 crc kubenswrapper[5052]: I1203 12:20:28.955284 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"ovnnorthd-scripts" Dec 03 12:20:28 crc kubenswrapper[5052]: I1203 12:20:28.968795 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ovnnorthd-ovnnorthd-dockercfg-6wvg8" Dec 03 12:20:28 crc kubenswrapper[5052]: I1203 12:20:28.975254 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ovn-northd-0"] Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.000388 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.006326 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-sb-0"] Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.014702 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a56992d9-4662-444f-8a45-0375d0855946-kube-api-access-4tjl2" (OuterVolumeSpecName: "kube-api-access-4tjl2") pod "a56992d9-4662-444f-8a45-0375d0855946" (UID: "a56992d9-4662-444f-8a45-0375d0855946"). InnerVolumeSpecName "kube-api-access-4tjl2". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.036115 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage06-crc" (OuterVolumeSpecName: "mysql-db") pod "a56992d9-4662-444f-8a45-0375d0855946" (UID: "a56992d9-4662-444f-8a45-0375d0855946"). InnerVolumeSpecName "local-storage06-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.036397 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-sb-0"] Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.037248 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tgrmq\" (UniqueName: \"kubernetes.io/projected/a77df94c-3e83-47a5-9b5f-eb0aeb41bc3d-kube-api-access-tgrmq\") pod \"ovn-northd-0\" (UID: \"a77df94c-3e83-47a5-9b5f-eb0aeb41bc3d\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.037328 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a77df94c-3e83-47a5-9b5f-eb0aeb41bc3d-combined-ca-bundle\") pod \"ovn-northd-0\" (UID: \"a77df94c-3e83-47a5-9b5f-eb0aeb41bc3d\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.037374 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a77df94c-3e83-47a5-9b5f-eb0aeb41bc3d-config\") pod \"ovn-northd-0\" (UID: \"a77df94c-3e83-47a5-9b5f-eb0aeb41bc3d\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.037425 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-rundir\" (UniqueName: \"kubernetes.io/empty-dir/a77df94c-3e83-47a5-9b5f-eb0aeb41bc3d-ovn-rundir\") pod \"ovn-northd-0\" (UID: \"a77df94c-3e83-47a5-9b5f-eb0aeb41bc3d\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.037469 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/a77df94c-3e83-47a5-9b5f-eb0aeb41bc3d-scripts\") pod \"ovn-northd-0\" (UID: \"a77df94c-3e83-47a5-9b5f-eb0aeb41bc3d\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.037497 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-northd-tls-certs\" (UniqueName: \"kubernetes.io/secret/a77df94c-3e83-47a5-9b5f-eb0aeb41bc3d-ovn-northd-tls-certs\") pod \"ovn-northd-0\" (UID: \"a77df94c-3e83-47a5-9b5f-eb0aeb41bc3d\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.037553 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/a77df94c-3e83-47a5-9b5f-eb0aeb41bc3d-metrics-certs-tls-certs\") pod \"ovn-northd-0\" (UID: \"a77df94c-3e83-47a5-9b5f-eb0aeb41bc3d\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.037601 5052 reconciler_common.go:293] "Volume detached for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/a56992d9-4662-444f-8a45-0375d0855946-kolla-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.037615 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-4tjl2\" (UniqueName: \"kubernetes.io/projected/a56992d9-4662-444f-8a45-0375d0855946-kube-api-access-4tjl2\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.037638 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage06-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage06-crc\") on node \"crc\" " Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.037665 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/a56992d9-4662-444f-8a45-0375d0855946-config-data-default\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.037674 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/a56992d9-4662-444f-8a45-0375d0855946-config-data-generated\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.037683 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/a56992d9-4662-444f-8a45-0375d0855946-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.076501 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a56992d9-4662-444f-8a45-0375d0855946-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "a56992d9-4662-444f-8a45-0375d0855946" (UID: "a56992d9-4662-444f-8a45-0375d0855946"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.079108 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ovsdbserver-sb-0"] Dec 03 12:20:29 crc kubenswrapper[5052]: E1203 12:20:29.080235 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="5067bb8d-d668-46a6-b9de-8065266cde63" containerName="galera" Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.080251 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="5067bb8d-d668-46a6-b9de-8065266cde63" containerName="galera" Dec 03 12:20:29 crc kubenswrapper[5052]: E1203 12:20:29.080258 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="5067bb8d-d668-46a6-b9de-8065266cde63" containerName="mysql-bootstrap" Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.080265 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="5067bb8d-d668-46a6-b9de-8065266cde63" containerName="mysql-bootstrap" Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.080444 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="5067bb8d-d668-46a6-b9de-8065266cde63" containerName="galera" Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.081447 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.090667 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/cinder-db-sync-gqt2p" podStartSLOduration=7.090649357 podStartE2EDuration="7.090649357s" podCreationTimestamp="2025-12-03 12:20:22 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:20:28.99965167 +0000 UTC m=+3619.587893434" watchObservedRunningTime="2025-12-03 12:20:29.090649357 +0000 UTC m=+3619.678891121" Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.095029 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ovncluster-ovndbcluster-sb-dockercfg-wqdjd" Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.095170 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-ovndbcluster-sb-ovndbs" Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.095280 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"ovndbcluster-sb-config" Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.103192 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"ovndbcluster-sb-scripts" Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.128797 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a56992d9-4662-444f-8a45-0375d0855946-galera-tls-certs" (OuterVolumeSpecName: "galera-tls-certs") pod "a56992d9-4662-444f-8a45-0375d0855946" (UID: "a56992d9-4662-444f-8a45-0375d0855946"). InnerVolumeSpecName "galera-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.133827 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage06-crc" (UniqueName: "kubernetes.io/local-volume/local-storage06-crc") on node "crc" Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.146429 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"mysql-db\" (UniqueName: \"kubernetes.io/local-volume/local-storage03-crc\") pod \"5067bb8d-d668-46a6-b9de-8065266cde63\" (UID: \"5067bb8d-d668-46a6-b9de-8065266cde63\") " Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.146588 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/5067bb8d-d668-46a6-b9de-8065266cde63-galera-tls-certs\") pod \"5067bb8d-d668-46a6-b9de-8065266cde63\" (UID: \"5067bb8d-d668-46a6-b9de-8065266cde63\") " Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.146692 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/5067bb8d-d668-46a6-b9de-8065266cde63-config-data-generated\") pod \"5067bb8d-d668-46a6-b9de-8065266cde63\" (UID: \"5067bb8d-d668-46a6-b9de-8065266cde63\") " Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.146742 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5067bb8d-d668-46a6-b9de-8065266cde63-combined-ca-bundle\") pod \"5067bb8d-d668-46a6-b9de-8065266cde63\" (UID: \"5067bb8d-d668-46a6-b9de-8065266cde63\") " Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.146780 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/5067bb8d-d668-46a6-b9de-8065266cde63-kolla-config\") pod \"5067bb8d-d668-46a6-b9de-8065266cde63\" (UID: \"5067bb8d-d668-46a6-b9de-8065266cde63\") " Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.146810 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/5067bb8d-d668-46a6-b9de-8065266cde63-operator-scripts\") pod \"5067bb8d-d668-46a6-b9de-8065266cde63\" (UID: \"5067bb8d-d668-46a6-b9de-8065266cde63\") " Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.146840 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k7znz\" (UniqueName: \"kubernetes.io/projected/5067bb8d-d668-46a6-b9de-8065266cde63-kube-api-access-k7znz\") pod \"5067bb8d-d668-46a6-b9de-8065266cde63\" (UID: \"5067bb8d-d668-46a6-b9de-8065266cde63\") " Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.146866 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/5067bb8d-d668-46a6-b9de-8065266cde63-config-data-default\") pod \"5067bb8d-d668-46a6-b9de-8065266cde63\" (UID: \"5067bb8d-d668-46a6-b9de-8065266cde63\") " Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.147060 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/a77df94c-3e83-47a5-9b5f-eb0aeb41bc3d-scripts\") pod \"ovn-northd-0\" (UID: \"a77df94c-3e83-47a5-9b5f-eb0aeb41bc3d\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.147101 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovn-northd-tls-certs\" (UniqueName: \"kubernetes.io/secret/a77df94c-3e83-47a5-9b5f-eb0aeb41bc3d-ovn-northd-tls-certs\") pod \"ovn-northd-0\" (UID: \"a77df94c-3e83-47a5-9b5f-eb0aeb41bc3d\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.147144 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/a77df94c-3e83-47a5-9b5f-eb0aeb41bc3d-metrics-certs-tls-certs\") pod \"ovn-northd-0\" (UID: \"a77df94c-3e83-47a5-9b5f-eb0aeb41bc3d\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.147210 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-tgrmq\" (UniqueName: \"kubernetes.io/projected/a77df94c-3e83-47a5-9b5f-eb0aeb41bc3d-kube-api-access-tgrmq\") pod \"ovn-northd-0\" (UID: \"a77df94c-3e83-47a5-9b5f-eb0aeb41bc3d\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.147254 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a77df94c-3e83-47a5-9b5f-eb0aeb41bc3d-combined-ca-bundle\") pod \"ovn-northd-0\" (UID: \"a77df94c-3e83-47a5-9b5f-eb0aeb41bc3d\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.147306 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a77df94c-3e83-47a5-9b5f-eb0aeb41bc3d-config\") pod \"ovn-northd-0\" (UID: \"a77df94c-3e83-47a5-9b5f-eb0aeb41bc3d\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.147378 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovn-rundir\" (UniqueName: \"kubernetes.io/empty-dir/a77df94c-3e83-47a5-9b5f-eb0aeb41bc3d-ovn-rundir\") pod \"ovn-northd-0\" (UID: \"a77df94c-3e83-47a5-9b5f-eb0aeb41bc3d\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.147461 5052 reconciler_common.go:293] "Volume detached for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/a56992d9-4662-444f-8a45-0375d0855946-galera-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.147477 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage06-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage06-crc\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.147489 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a56992d9-4662-444f-8a45-0375d0855946-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.147925 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovn-rundir\" (UniqueName: \"kubernetes.io/empty-dir/a77df94c-3e83-47a5-9b5f-eb0aeb41bc3d-ovn-rundir\") pod \"ovn-northd-0\" (UID: \"a77df94c-3e83-47a5-9b5f-eb0aeb41bc3d\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.150060 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5067bb8d-d668-46a6-b9de-8065266cde63-config-data-default" (OuterVolumeSpecName: "config-data-default") pod "5067bb8d-d668-46a6-b9de-8065266cde63" (UID: "5067bb8d-d668-46a6-b9de-8065266cde63"). InnerVolumeSpecName "config-data-default". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.150480 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5067bb8d-d668-46a6-b9de-8065266cde63-kolla-config" (OuterVolumeSpecName: "kolla-config") pod "5067bb8d-d668-46a6-b9de-8065266cde63" (UID: "5067bb8d-d668-46a6-b9de-8065266cde63"). InnerVolumeSpecName "kolla-config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.151024 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5067bb8d-d668-46a6-b9de-8065266cde63-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "5067bb8d-d668-46a6-b9de-8065266cde63" (UID: "5067bb8d-d668-46a6-b9de-8065266cde63"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.155009 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a77df94c-3e83-47a5-9b5f-eb0aeb41bc3d-config\") pod \"ovn-northd-0\" (UID: \"a77df94c-3e83-47a5-9b5f-eb0aeb41bc3d\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.155593 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/a77df94c-3e83-47a5-9b5f-eb0aeb41bc3d-scripts\") pod \"ovn-northd-0\" (UID: \"a77df94c-3e83-47a5-9b5f-eb0aeb41bc3d\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.164568 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5067bb8d-d668-46a6-b9de-8065266cde63-config-data-generated" (OuterVolumeSpecName: "config-data-generated") pod "5067bb8d-d668-46a6-b9de-8065266cde63" (UID: "5067bb8d-d668-46a6-b9de-8065266cde63"). InnerVolumeSpecName "config-data-generated". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.171618 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-sb-0"] Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.202293 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-tgrmq\" (UniqueName: \"kubernetes.io/projected/a77df94c-3e83-47a5-9b5f-eb0aeb41bc3d-kube-api-access-tgrmq\") pod \"ovn-northd-0\" (UID: \"a77df94c-3e83-47a5-9b5f-eb0aeb41bc3d\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.236061 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/a77df94c-3e83-47a5-9b5f-eb0aeb41bc3d-metrics-certs-tls-certs\") pod \"ovn-northd-0\" (UID: \"a77df94c-3e83-47a5-9b5f-eb0aeb41bc3d\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.236250 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a77df94c-3e83-47a5-9b5f-eb0aeb41bc3d-combined-ca-bundle\") pod \"ovn-northd-0\" (UID: \"a77df94c-3e83-47a5-9b5f-eb0aeb41bc3d\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.240520 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovn-northd-tls-certs\" (UniqueName: \"kubernetes.io/secret/a77df94c-3e83-47a5-9b5f-eb0aeb41bc3d-ovn-northd-tls-certs\") pod \"ovn-northd-0\" (UID: \"a77df94c-3e83-47a5-9b5f-eb0aeb41bc3d\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.252979 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/19a09467-4c91-40f5-b83a-d43a8eb5f2dc-combined-ca-bundle\") pod \"ovsdbserver-sb-0\" (UID: \"19a09467-4c91-40f5-b83a-d43a8eb5f2dc\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.253033 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-48j4f\" (UniqueName: \"kubernetes.io/projected/19a09467-4c91-40f5-b83a-d43a8eb5f2dc-kube-api-access-48j4f\") pod \"ovsdbserver-sb-0\" (UID: \"19a09467-4c91-40f5-b83a-d43a8eb5f2dc\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.253056 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/19a09467-4c91-40f5-b83a-d43a8eb5f2dc-metrics-certs-tls-certs\") pod \"ovsdbserver-sb-0\" (UID: \"19a09467-4c91-40f5-b83a-d43a8eb5f2dc\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.253130 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/19a09467-4c91-40f5-b83a-d43a8eb5f2dc-config\") pod \"ovsdbserver-sb-0\" (UID: \"19a09467-4c91-40f5-b83a-d43a8eb5f2dc\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.253154 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovsdbserver-sb-tls-certs\" (UniqueName: \"kubernetes.io/secret/19a09467-4c91-40f5-b83a-d43a8eb5f2dc-ovsdbserver-sb-tls-certs\") pod \"ovsdbserver-sb-0\" (UID: \"19a09467-4c91-40f5-b83a-d43a8eb5f2dc\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.253172 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage02-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage02-crc\") pod \"ovsdbserver-sb-0\" (UID: \"19a09467-4c91-40f5-b83a-d43a8eb5f2dc\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.253193 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/19a09467-4c91-40f5-b83a-d43a8eb5f2dc-scripts\") pod \"ovsdbserver-sb-0\" (UID: \"19a09467-4c91-40f5-b83a-d43a8eb5f2dc\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.253226 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/19a09467-4c91-40f5-b83a-d43a8eb5f2dc-ovsdb-rundir\") pod \"ovsdbserver-sb-0\" (UID: \"19a09467-4c91-40f5-b83a-d43a8eb5f2dc\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.253308 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/5067bb8d-d668-46a6-b9de-8065266cde63-config-data-default\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.253322 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/5067bb8d-d668-46a6-b9de-8065266cde63-config-data-generated\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.253332 5052 reconciler_common.go:293] "Volume detached for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/5067bb8d-d668-46a6-b9de-8065266cde63-kolla-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.253342 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/5067bb8d-d668-46a6-b9de-8065266cde63-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.263935 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5067bb8d-d668-46a6-b9de-8065266cde63-kube-api-access-k7znz" (OuterVolumeSpecName: "kube-api-access-k7znz") pod "5067bb8d-d668-46a6-b9de-8065266cde63" (UID: "5067bb8d-d668-46a6-b9de-8065266cde63"). InnerVolumeSpecName "kube-api-access-k7znz". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.269583 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/memcached-0"] Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.290572 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/memcached-0"] Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.302558 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage03-crc" (OuterVolumeSpecName: "mysql-db") pod "5067bb8d-d668-46a6-b9de-8065266cde63" (UID: "5067bb8d-d668-46a6-b9de-8065266cde63"). InnerVolumeSpecName "local-storage03-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.311232 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/memcached-0"] Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.312415 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/memcached-0" Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.317084 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"memcached-memcached-dockercfg-7smft" Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.317298 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"memcached-config-data" Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.317694 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-memcached-svc" Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.340780 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.382288 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.382951 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/19a09467-4c91-40f5-b83a-d43a8eb5f2dc-ovsdb-rundir\") pod \"ovsdbserver-sb-0\" (UID: \"19a09467-4c91-40f5-b83a-d43a8eb5f2dc\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.384784 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sc7n7\" (UniqueName: \"kubernetes.io/projected/adcea736-eb15-4768-b12c-edb4c810beb3-kube-api-access-sc7n7\") pod \"memcached-0\" (UID: \"adcea736-eb15-4768-b12c-edb4c810beb3\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.384838 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/19a09467-4c91-40f5-b83a-d43a8eb5f2dc-combined-ca-bundle\") pod \"ovsdbserver-sb-0\" (UID: \"19a09467-4c91-40f5-b83a-d43a8eb5f2dc\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.384898 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-48j4f\" (UniqueName: \"kubernetes.io/projected/19a09467-4c91-40f5-b83a-d43a8eb5f2dc-kube-api-access-48j4f\") pod \"ovsdbserver-sb-0\" (UID: \"19a09467-4c91-40f5-b83a-d43a8eb5f2dc\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.384939 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/19a09467-4c91-40f5-b83a-d43a8eb5f2dc-metrics-certs-tls-certs\") pod \"ovsdbserver-sb-0\" (UID: \"19a09467-4c91-40f5-b83a-d43a8eb5f2dc\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.385023 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"memcached-tls-certs\" (UniqueName: \"kubernetes.io/secret/adcea736-eb15-4768-b12c-edb4c810beb3-memcached-tls-certs\") pod \"memcached-0\" (UID: \"adcea736-eb15-4768-b12c-edb4c810beb3\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.385078 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/adcea736-eb15-4768-b12c-edb4c810beb3-combined-ca-bundle\") pod \"memcached-0\" (UID: \"adcea736-eb15-4768-b12c-edb4c810beb3\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.385161 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/19a09467-4c91-40f5-b83a-d43a8eb5f2dc-config\") pod \"ovsdbserver-sb-0\" (UID: \"19a09467-4c91-40f5-b83a-d43a8eb5f2dc\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.385189 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/adcea736-eb15-4768-b12c-edb4c810beb3-kolla-config\") pod \"memcached-0\" (UID: \"adcea736-eb15-4768-b12c-edb4c810beb3\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.385228 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovsdbserver-sb-tls-certs\" (UniqueName: \"kubernetes.io/secret/19a09467-4c91-40f5-b83a-d43a8eb5f2dc-ovsdbserver-sb-tls-certs\") pod \"ovsdbserver-sb-0\" (UID: \"19a09467-4c91-40f5-b83a-d43a8eb5f2dc\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.385252 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage02-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage02-crc\") pod \"ovsdbserver-sb-0\" (UID: \"19a09467-4c91-40f5-b83a-d43a8eb5f2dc\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.385273 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/19a09467-4c91-40f5-b83a-d43a8eb5f2dc-scripts\") pod \"ovsdbserver-sb-0\" (UID: \"19a09467-4c91-40f5-b83a-d43a8eb5f2dc\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.385291 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/adcea736-eb15-4768-b12c-edb4c810beb3-config-data\") pod \"memcached-0\" (UID: \"adcea736-eb15-4768-b12c-edb4c810beb3\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.385456 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-k7znz\" (UniqueName: \"kubernetes.io/projected/5067bb8d-d668-46a6-b9de-8065266cde63-kube-api-access-k7znz\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.385486 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage03-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage03-crc\") on node \"crc\" " Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.385602 5052 patch_prober.go:28] interesting pod/machine-config-daemon-74bj6 container/machine-config-daemon namespace/openshift-machine-config-operator: Liveness probe status=failure output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" start-of-body= Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.385682 5052 prober.go:107] "Probe failed" probeType="Liveness" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" containerName="machine-config-daemon" probeResult="failure" output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.385735 5052 kubelet.go:2542] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.386792 5052 kuberuntime_manager.go:1027] "Message for Container of pod" containerName="machine-config-daemon" containerStatusID={"Type":"cri-o","ID":"09b4a0072741f60499d24f40cc4ff3406d5d22671ced431363ab865172786300"} pod="openshift-machine-config-operator/machine-config-daemon-74bj6" containerMessage="Container machine-config-daemon failed liveness probe, will be restarted" Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.386850 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" containerName="machine-config-daemon" containerID="cri-o://09b4a0072741f60499d24f40cc4ff3406d5d22671ced431363ab865172786300" gracePeriod=600 Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.387789 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/19a09467-4c91-40f5-b83a-d43a8eb5f2dc-ovsdb-rundir\") pod \"ovsdbserver-sb-0\" (UID: \"19a09467-4c91-40f5-b83a-d43a8eb5f2dc\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.388417 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage02-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage02-crc\") pod \"ovsdbserver-sb-0\" (UID: \"19a09467-4c91-40f5-b83a-d43a8eb5f2dc\") device mount path \"/mnt/openstack/pv02\"" pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.388815 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/19a09467-4c91-40f5-b83a-d43a8eb5f2dc-config\") pod \"ovsdbserver-sb-0\" (UID: \"19a09467-4c91-40f5-b83a-d43a8eb5f2dc\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.391833 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.393457 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/19a09467-4c91-40f5-b83a-d43a8eb5f2dc-scripts\") pod \"ovsdbserver-sb-0\" (UID: \"19a09467-4c91-40f5-b83a-d43a8eb5f2dc\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.442904 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/19a09467-4c91-40f5-b83a-d43a8eb5f2dc-combined-ca-bundle\") pod \"ovsdbserver-sb-0\" (UID: \"19a09467-4c91-40f5-b83a-d43a8eb5f2dc\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.445772 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovsdbserver-sb-tls-certs\" (UniqueName: \"kubernetes.io/secret/19a09467-4c91-40f5-b83a-d43a8eb5f2dc-ovsdbserver-sb-tls-certs\") pod \"ovsdbserver-sb-0\" (UID: \"19a09467-4c91-40f5-b83a-d43a8eb5f2dc\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.449761 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" podUID="1ec6b1f7-5abb-4063-80fa-b5ec907dcb91" containerName="rabbitmq" containerID="cri-o://50d90ae2ffca4d0cc92a43c260b1dec75f6540c0f3b6a2f388e0315ce52b8fdd" gracePeriod=604793 Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.462621 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.464142 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.471250 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/memcached-0"] Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.476134 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-48j4f\" (UniqueName: \"kubernetes.io/projected/19a09467-4c91-40f5-b83a-d43a8eb5f2dc-kube-api-access-48j4f\") pod \"ovsdbserver-sb-0\" (UID: \"19a09467-4c91-40f5-b83a-d43a8eb5f2dc\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.478720 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-nova-novncproxy-cell1-public-svc" Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.478895 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/19a09467-4c91-40f5-b83a-d43a8eb5f2dc-metrics-certs-tls-certs\") pod \"ovsdbserver-sb-0\" (UID: \"19a09467-4c91-40f5-b83a-d43a8eb5f2dc\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.478994 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-nova-novncproxy-cell1-vencrypt" Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.479111 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell1-novncproxy-config-data" Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.485480 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.495836 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-sc7n7\" (UniqueName: \"kubernetes.io/projected/adcea736-eb15-4768-b12c-edb4c810beb3-kube-api-access-sc7n7\") pod \"memcached-0\" (UID: \"adcea736-eb15-4768-b12c-edb4c810beb3\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.495929 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"memcached-tls-certs\" (UniqueName: \"kubernetes.io/secret/adcea736-eb15-4768-b12c-edb4c810beb3-memcached-tls-certs\") pod \"memcached-0\" (UID: \"adcea736-eb15-4768-b12c-edb4c810beb3\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.495965 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/adcea736-eb15-4768-b12c-edb4c810beb3-combined-ca-bundle\") pod \"memcached-0\" (UID: \"adcea736-eb15-4768-b12c-edb4c810beb3\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.496023 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/adcea736-eb15-4768-b12c-edb4c810beb3-kolla-config\") pod \"memcached-0\" (UID: \"adcea736-eb15-4768-b12c-edb4c810beb3\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.496066 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/adcea736-eb15-4768-b12c-edb4c810beb3-config-data\") pod \"memcached-0\" (UID: \"adcea736-eb15-4768-b12c-edb4c810beb3\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.499597 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/adcea736-eb15-4768-b12c-edb4c810beb3-kolla-config\") pod \"memcached-0\" (UID: \"adcea736-eb15-4768-b12c-edb4c810beb3\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.499888 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/adcea736-eb15-4768-b12c-edb4c810beb3-config-data\") pod \"memcached-0\" (UID: \"adcea736-eb15-4768-b12c-edb4c810beb3\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.510924 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage02-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage02-crc\") pod \"ovsdbserver-sb-0\" (UID: \"19a09467-4c91-40f5-b83a-d43a8eb5f2dc\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.520947 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"memcached-tls-certs\" (UniqueName: \"kubernetes.io/secret/adcea736-eb15-4768-b12c-edb4c810beb3-memcached-tls-certs\") pod \"memcached-0\" (UID: \"adcea736-eb15-4768-b12c-edb4c810beb3\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.521308 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/adcea736-eb15-4768-b12c-edb4c810beb3-combined-ca-bundle\") pod \"memcached-0\" (UID: \"adcea736-eb15-4768-b12c-edb4c810beb3\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.522879 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/kube-state-metrics-0"] Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.531227 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/kube-state-metrics-0"] Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.541259 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/kube-state-metrics-0"] Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.542938 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.549868 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-kube-state-metrics-svc" Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.550134 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"kube-state-metrics-tls-config" Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.551038 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-sc7n7\" (UniqueName: \"kubernetes.io/projected/adcea736-eb15-4768-b12c-edb4c810beb3-kube-api-access-sc7n7\") pod \"memcached-0\" (UID: \"adcea736-eb15-4768-b12c-edb4c810beb3\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.557831 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/kube-state-metrics-0"] Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.578765 5052 prober.go:107] "Probe failed" probeType="Liveness" pod="openstack-kuttl-tests/cinder-scheduler-0" podUID="d5750ef1-ce21-4092-9612-13d6f6c923df" containerName="cinder-scheduler" probeResult="failure" output="HTTP probe failed with statuscode: 500" Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.578836 5052 kubelet.go:2542] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.579648 5052 kuberuntime_manager.go:1027] "Message for Container of pod" containerName="cinder-scheduler" containerStatusID={"Type":"cri-o","ID":"66ff70190e4169f7ceded95ca82e7cd10045ed2cce8eb37b9d1a20b3e28c1110"} pod="openstack-kuttl-tests/cinder-scheduler-0" containerMessage="Container cinder-scheduler failed liveness probe, will be restarted" Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.579702 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/cinder-scheduler-0" podUID="d5750ef1-ce21-4092-9612-13d6f6c923df" containerName="cinder-scheduler" containerID="cri-o://66ff70190e4169f7ceded95ca82e7cd10045ed2cce8eb37b9d1a20b3e28c1110" gracePeriod=30 Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.598468 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9e3a0650-c101-4bfa-8137-9909bf8ed4c7-combined-ca-bundle\") pod \"nova-cell1-novncproxy-0\" (UID: \"9e3a0650-c101-4bfa-8137-9909bf8ed4c7\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.598550 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"vencrypt-tls-certs\" (UniqueName: \"kubernetes.io/secret/9e3a0650-c101-4bfa-8137-9909bf8ed4c7-vencrypt-tls-certs\") pod \"nova-cell1-novncproxy-0\" (UID: \"9e3a0650-c101-4bfa-8137-9909bf8ed4c7\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.598584 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nova-novncproxy-tls-certs\" (UniqueName: \"kubernetes.io/secret/9e3a0650-c101-4bfa-8137-9909bf8ed4c7-nova-novncproxy-tls-certs\") pod \"nova-cell1-novncproxy-0\" (UID: \"9e3a0650-c101-4bfa-8137-9909bf8ed4c7\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.610313 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/9e3a0650-c101-4bfa-8137-9909bf8ed4c7-config-data\") pod \"nova-cell1-novncproxy-0\" (UID: \"9e3a0650-c101-4bfa-8137-9909bf8ed4c7\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.610941 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rtdc4\" (UniqueName: \"kubernetes.io/projected/9e3a0650-c101-4bfa-8137-9909bf8ed4c7-kube-api-access-rtdc4\") pod \"nova-cell1-novncproxy-0\" (UID: \"9e3a0650-c101-4bfa-8137-9909bf8ed4c7\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.649449 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage03-crc" (UniqueName: "kubernetes.io/local-volume/local-storage03-crc") on node "crc" Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.700993 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="06dda089-0745-4941-adfe-2de711fce24e" path="/var/lib/kubelet/pods/06dda089-0745-4941-adfe-2de711fce24e/volumes" Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.701949 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="42e93b9b-3a58-4ca6-abc6-6e359b15e7cc" path="/var/lib/kubelet/pods/42e93b9b-3a58-4ca6-abc6-6e359b15e7cc/volumes" Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.702582 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="42ebb0e3-9a37-4898-a1e4-0822b4bd9e1e" path="/var/lib/kubelet/pods/42ebb0e3-9a37-4898-a1e4-0822b4bd9e1e/volumes" Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.709462 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="695de243-96a9-4e8b-a0f3-3a4ef2d95efe" path="/var/lib/kubelet/pods/695de243-96a9-4e8b-a0f3-3a4ef2d95efe/volumes" Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.710233 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="966185e5-f414-46c5-ba48-0e234750795a" path="/var/lib/kubelet/pods/966185e5-f414-46c5-ba48-0e234750795a/volumes" Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.712708 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls-certs\" (UniqueName: \"kubernetes.io/secret/b657a259-eb7a-48e8-8f41-2a3c077aff2a-kube-state-metrics-tls-certs\") pod \"kube-state-metrics-0\" (UID: \"b657a259-eb7a-48e8-8f41-2a3c077aff2a\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.712898 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-rtdc4\" (UniqueName: \"kubernetes.io/projected/9e3a0650-c101-4bfa-8137-9909bf8ed4c7-kube-api-access-rtdc4\") pod \"nova-cell1-novncproxy-0\" (UID: \"9e3a0650-c101-4bfa-8137-9909bf8ed4c7\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.712927 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b657a259-eb7a-48e8-8f41-2a3c077aff2a-combined-ca-bundle\") pod \"kube-state-metrics-0\" (UID: \"b657a259-eb7a-48e8-8f41-2a3c077aff2a\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.713064 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9e3a0650-c101-4bfa-8137-9909bf8ed4c7-combined-ca-bundle\") pod \"nova-cell1-novncproxy-0\" (UID: \"9e3a0650-c101-4bfa-8137-9909bf8ed4c7\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.713103 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"vencrypt-tls-certs\" (UniqueName: \"kubernetes.io/secret/9e3a0650-c101-4bfa-8137-9909bf8ed4c7-vencrypt-tls-certs\") pod \"nova-cell1-novncproxy-0\" (UID: \"9e3a0650-c101-4bfa-8137-9909bf8ed4c7\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.713127 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls-config\" (UniqueName: \"kubernetes.io/secret/b657a259-eb7a-48e8-8f41-2a3c077aff2a-kube-state-metrics-tls-config\") pod \"kube-state-metrics-0\" (UID: \"b657a259-eb7a-48e8-8f41-2a3c077aff2a\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.713149 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"nova-novncproxy-tls-certs\" (UniqueName: \"kubernetes.io/secret/9e3a0650-c101-4bfa-8137-9909bf8ed4c7-nova-novncproxy-tls-certs\") pod \"nova-cell1-novncproxy-0\" (UID: \"9e3a0650-c101-4bfa-8137-9909bf8ed4c7\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.713176 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zz6rj\" (UniqueName: \"kubernetes.io/projected/b657a259-eb7a-48e8-8f41-2a3c077aff2a-kube-api-access-zz6rj\") pod \"kube-state-metrics-0\" (UID: \"b657a259-eb7a-48e8-8f41-2a3c077aff2a\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.713226 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/9e3a0650-c101-4bfa-8137-9909bf8ed4c7-config-data\") pod \"nova-cell1-novncproxy-0\" (UID: \"9e3a0650-c101-4bfa-8137-9909bf8ed4c7\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.713277 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage03-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage03-crc\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.758604 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/9e3a0650-c101-4bfa-8137-9909bf8ed4c7-config-data\") pod \"nova-cell1-novncproxy-0\" (UID: \"9e3a0650-c101-4bfa-8137-9909bf8ed4c7\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.760357 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9e3a0650-c101-4bfa-8137-9909bf8ed4c7-combined-ca-bundle\") pod \"nova-cell1-novncproxy-0\" (UID: \"9e3a0650-c101-4bfa-8137-9909bf8ed4c7\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.760786 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/rabbitmq-server-0" podUID="683b7d5f-a88a-4dc0-ae83-af3c6444c7ff" containerName="rabbitmq" containerID="cri-o://82c91d3bad330d4f21f39bfba31fbc4e7bfd4508da6e46af97efbf075afd08f3" gracePeriod=604792 Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.761740 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"nova-novncproxy-tls-certs\" (UniqueName: \"kubernetes.io/secret/9e3a0650-c101-4bfa-8137-9909bf8ed4c7-nova-novncproxy-tls-certs\") pod \"nova-cell1-novncproxy-0\" (UID: \"9e3a0650-c101-4bfa-8137-9909bf8ed4c7\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.771875 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"vencrypt-tls-certs\" (UniqueName: \"kubernetes.io/secret/9e3a0650-c101-4bfa-8137-9909bf8ed4c7-vencrypt-tls-certs\") pod \"nova-cell1-novncproxy-0\" (UID: \"9e3a0650-c101-4bfa-8137-9909bf8ed4c7\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.774605 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-rtdc4\" (UniqueName: \"kubernetes.io/projected/9e3a0650-c101-4bfa-8137-9909bf8ed4c7-kube-api-access-rtdc4\") pod \"nova-cell1-novncproxy-0\" (UID: \"9e3a0650-c101-4bfa-8137-9909bf8ed4c7\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.816238 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b657a259-eb7a-48e8-8f41-2a3c077aff2a-combined-ca-bundle\") pod \"kube-state-metrics-0\" (UID: \"b657a259-eb7a-48e8-8f41-2a3c077aff2a\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.816296 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls-config\" (UniqueName: \"kubernetes.io/secret/b657a259-eb7a-48e8-8f41-2a3c077aff2a-kube-state-metrics-tls-config\") pod \"kube-state-metrics-0\" (UID: \"b657a259-eb7a-48e8-8f41-2a3c077aff2a\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.816328 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-zz6rj\" (UniqueName: \"kubernetes.io/projected/b657a259-eb7a-48e8-8f41-2a3c077aff2a-kube-api-access-zz6rj\") pod \"kube-state-metrics-0\" (UID: \"b657a259-eb7a-48e8-8f41-2a3c077aff2a\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.816379 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls-certs\" (UniqueName: \"kubernetes.io/secret/b657a259-eb7a-48e8-8f41-2a3c077aff2a-kube-state-metrics-tls-certs\") pod \"kube-state-metrics-0\" (UID: \"b657a259-eb7a-48e8-8f41-2a3c077aff2a\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.842874 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5067bb8d-d668-46a6-b9de-8065266cde63-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "5067bb8d-d668-46a6-b9de-8065266cde63" (UID: "5067bb8d-d668-46a6-b9de-8065266cde63"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.854039 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b657a259-eb7a-48e8-8f41-2a3c077aff2a-combined-ca-bundle\") pod \"kube-state-metrics-0\" (UID: \"b657a259-eb7a-48e8-8f41-2a3c077aff2a\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.858618 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls-certs\" (UniqueName: \"kubernetes.io/secret/b657a259-eb7a-48e8-8f41-2a3c077aff2a-kube-state-metrics-tls-certs\") pod \"kube-state-metrics-0\" (UID: \"b657a259-eb7a-48e8-8f41-2a3c077aff2a\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.872105 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-zz6rj\" (UniqueName: \"kubernetes.io/projected/b657a259-eb7a-48e8-8f41-2a3c077aff2a-kube-api-access-zz6rj\") pod \"kube-state-metrics-0\" (UID: \"b657a259-eb7a-48e8-8f41-2a3c077aff2a\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.894083 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls-config\" (UniqueName: \"kubernetes.io/secret/b657a259-eb7a-48e8-8f41-2a3c077aff2a-kube-state-metrics-tls-config\") pod \"kube-state-metrics-0\" (UID: \"b657a259-eb7a-48e8-8f41-2a3c077aff2a\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.917790 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5067bb8d-d668-46a6-b9de-8065266cde63-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:29 crc kubenswrapper[5052]: I1203 12:20:29.978965 5052 generic.go:334] "Generic (PLEG): container finished" podID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" containerID="09b4a0072741f60499d24f40cc4ff3406d5d22671ced431363ab865172786300" exitCode=0 Dec 03 12:20:29 crc kubenswrapper[5052]: W1203 12:20:29.994695 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda77df94c_3e83_47a5_9b5f_eb0aeb41bc3d.slice/crio-c5318b2d6cbe69bd088c706a2833c45418c4ceb09d5906c6f5dc5ce8834b0109 WatchSource:0}: Error finding container c5318b2d6cbe69bd088c706a2833c45418c4ceb09d5906c6f5dc5ce8834b0109: Status 404 returned error can't find the container with id c5318b2d6cbe69bd088c706a2833c45418c4ceb09d5906c6f5dc5ce8834b0109 Dec 03 12:20:30 crc kubenswrapper[5052]: I1203 12:20:30.017754 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:20:30 crc kubenswrapper[5052]: I1203 12:20:30.019017 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:20:30 crc kubenswrapper[5052]: I1203 12:20:30.039259 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/barbican-db-sync-92z8b" podStartSLOduration=7.039240872 podStartE2EDuration="7.039240872s" podCreationTimestamp="2025-12-03 12:20:23 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:20:30.024185702 +0000 UTC m=+3620.612427466" watchObservedRunningTime="2025-12-03 12:20:30.039240872 +0000 UTC m=+3620.627482636" Dec 03 12:20:30 crc kubenswrapper[5052]: I1203 12:20:30.063160 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5067bb8d-d668-46a6-b9de-8065266cde63-galera-tls-certs" (OuterVolumeSpecName: "galera-tls-certs") pod "5067bb8d-d668-46a6-b9de-8065266cde63" (UID: "5067bb8d-d668-46a6-b9de-8065266cde63"). InnerVolumeSpecName "galera-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:20:30 crc kubenswrapper[5052]: I1203 12:20:30.069992 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/swift-ring-rebalance-z7lpm" podStartSLOduration=7.069969209 podStartE2EDuration="7.069969209s" podCreationTimestamp="2025-12-03 12:20:23 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:20:30.065489991 +0000 UTC m=+3620.653731785" watchObservedRunningTime="2025-12-03 12:20:30.069969209 +0000 UTC m=+3620.658210973" Dec 03 12:20:30 crc kubenswrapper[5052]: I1203 12:20:30.087481 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-ring-rebalance-z7lpm" event={"ID":"318545bf-5895-48f9-9649-67207bc4b207","Type":"ContainerStarted","Data":"e3b8749f931ef525570e1e11172cba2d27c70a94440e0ea98f51fe1e60b90b6d"} Dec 03 12:20:30 crc kubenswrapper[5052]: I1203 12:20:30.087547 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-nb-0" event={"ID":"70cedf3b-c9b9-4e5c-81a2-ba134b150997","Type":"ContainerStarted","Data":"f84d0df84d3cb24dd97faa8c986cf4682cc548000edcdfa7e202ea04d4fd81e2"} Dec 03 12:20:30 crc kubenswrapper[5052]: I1203 12:20:30.087565 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-db-sync-92z8b" event={"ID":"7a9497b1-89e2-4ebc-97e6-2bdaeefab95f","Type":"ContainerStarted","Data":"d8891df0b213dbaf1fa63a8450de0cdab0ef7ec799eb7a4b97714ab88cffe2ab"} Dec 03 12:20:30 crc kubenswrapper[5052]: I1203 12:20:30.087580 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" event={"ID":"10c237bc-72e7-4da3-9f2c-b1688395bf6e","Type":"ContainerDied","Data":"09b4a0072741f60499d24f40cc4ff3406d5d22671ced431363ab865172786300"} Dec 03 12:20:30 crc kubenswrapper[5052]: I1203 12:20:30.087603 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-r444h" event={"ID":"cb824463-7efe-49ee-a052-acd1fd8f02d3","Type":"ContainerStarted","Data":"2f07afc7faedc91462a894d76d86fe113f7f5b41d2ce70d3b0e31378e31a2ea3"} Dec 03 12:20:30 crc kubenswrapper[5052]: I1203 12:20:30.087645 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ovn-northd-0"] Dec 03 12:20:30 crc kubenswrapper[5052]: I1203 12:20:30.089277 5052 scope.go:117] "RemoveContainer" containerID="79e965387252796514d44c1fd52f83c66501c2dc3200a49ddb7dacbcbcf015ce" Dec 03 12:20:30 crc kubenswrapper[5052]: I1203 12:20:30.098608 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-r444h" podStartSLOduration=6.098587046 podStartE2EDuration="6.098587046s" podCreationTimestamp="2025-12-03 12:20:24 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:20:30.095260441 +0000 UTC m=+3620.683502205" watchObservedRunningTime="2025-12-03 12:20:30.098587046 +0000 UTC m=+3620.686828810" Dec 03 12:20:30 crc kubenswrapper[5052]: I1203 12:20:30.139990 5052 reconciler_common.go:293] "Volume detached for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/5067bb8d-d668-46a6-b9de-8065266cde63-galera-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:30 crc kubenswrapper[5052]: I1203 12:20:30.141584 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/openstack-cell1-galera-0"] Dec 03 12:20:30 crc kubenswrapper[5052]: I1203 12:20:30.189137 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/openstack-cell1-galera-0"] Dec 03 12:20:30 crc kubenswrapper[5052]: I1203 12:20:30.198628 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:20:30 crc kubenswrapper[5052]: I1203 12:20:30.199818 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/memcached-0" Dec 03 12:20:30 crc kubenswrapper[5052]: I1203 12:20:30.247138 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/openstack-cell1-galera-0"] Dec 03 12:20:30 crc kubenswrapper[5052]: I1203 12:20:30.250308 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:20:30 crc kubenswrapper[5052]: E1203 12:20:30.258086 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 12:20:30 crc kubenswrapper[5052]: I1203 12:20:30.261210 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:20:30 crc kubenswrapper[5052]: I1203 12:20:30.277252 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-galera-openstack-cell1-svc" Dec 03 12:20:30 crc kubenswrapper[5052]: I1203 12:20:30.281894 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"openstack-cell1-scripts" Dec 03 12:20:30 crc kubenswrapper[5052]: I1203 12:20:30.287931 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"openstack-cell1-config-data" Dec 03 12:20:30 crc kubenswrapper[5052]: I1203 12:20:30.288208 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"galera-openstack-cell1-dockercfg-nt4tx" Dec 03 12:20:30 crc kubenswrapper[5052]: I1203 12:20:30.355405 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/openstack-cell1-galera-0"] Dec 03 12:20:30 crc kubenswrapper[5052]: I1203 12:20:30.359795 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/47aa63e7-b7b2-4c1d-a73d-348b50cc5d2b-galera-tls-certs\") pod \"openstack-cell1-galera-0\" (UID: \"47aa63e7-b7b2-4c1d-a73d-348b50cc5d2b\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:20:30 crc kubenswrapper[5052]: I1203 12:20:30.367209 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/47aa63e7-b7b2-4c1d-a73d-348b50cc5d2b-operator-scripts\") pod \"openstack-cell1-galera-0\" (UID: \"47aa63e7-b7b2-4c1d-a73d-348b50cc5d2b\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:20:30 crc kubenswrapper[5052]: I1203 12:20:30.367336 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/47aa63e7-b7b2-4c1d-a73d-348b50cc5d2b-combined-ca-bundle\") pod \"openstack-cell1-galera-0\" (UID: \"47aa63e7-b7b2-4c1d-a73d-348b50cc5d2b\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:20:30 crc kubenswrapper[5052]: I1203 12:20:30.367803 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/47aa63e7-b7b2-4c1d-a73d-348b50cc5d2b-kolla-config\") pod \"openstack-cell1-galera-0\" (UID: \"47aa63e7-b7b2-4c1d-a73d-348b50cc5d2b\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:20:30 crc kubenswrapper[5052]: I1203 12:20:30.367903 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-njjpc\" (UniqueName: \"kubernetes.io/projected/47aa63e7-b7b2-4c1d-a73d-348b50cc5d2b-kube-api-access-njjpc\") pod \"openstack-cell1-galera-0\" (UID: \"47aa63e7-b7b2-4c1d-a73d-348b50cc5d2b\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:20:30 crc kubenswrapper[5052]: I1203 12:20:30.368106 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/47aa63e7-b7b2-4c1d-a73d-348b50cc5d2b-config-data-default\") pod \"openstack-cell1-galera-0\" (UID: \"47aa63e7-b7b2-4c1d-a73d-348b50cc5d2b\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:20:30 crc kubenswrapper[5052]: I1203 12:20:30.371548 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage06-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage06-crc\") pod \"openstack-cell1-galera-0\" (UID: \"47aa63e7-b7b2-4c1d-a73d-348b50cc5d2b\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:20:30 crc kubenswrapper[5052]: I1203 12:20:30.371721 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/47aa63e7-b7b2-4c1d-a73d-348b50cc5d2b-config-data-generated\") pod \"openstack-cell1-galera-0\" (UID: \"47aa63e7-b7b2-4c1d-a73d-348b50cc5d2b\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:20:30 crc kubenswrapper[5052]: I1203 12:20:30.482916 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/47aa63e7-b7b2-4c1d-a73d-348b50cc5d2b-config-data-default\") pod \"openstack-cell1-galera-0\" (UID: \"47aa63e7-b7b2-4c1d-a73d-348b50cc5d2b\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:20:30 crc kubenswrapper[5052]: I1203 12:20:30.482976 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage06-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage06-crc\") pod \"openstack-cell1-galera-0\" (UID: \"47aa63e7-b7b2-4c1d-a73d-348b50cc5d2b\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:20:30 crc kubenswrapper[5052]: I1203 12:20:30.483000 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/47aa63e7-b7b2-4c1d-a73d-348b50cc5d2b-config-data-generated\") pod \"openstack-cell1-galera-0\" (UID: \"47aa63e7-b7b2-4c1d-a73d-348b50cc5d2b\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:20:30 crc kubenswrapper[5052]: I1203 12:20:30.483044 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/47aa63e7-b7b2-4c1d-a73d-348b50cc5d2b-galera-tls-certs\") pod \"openstack-cell1-galera-0\" (UID: \"47aa63e7-b7b2-4c1d-a73d-348b50cc5d2b\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:20:30 crc kubenswrapper[5052]: I1203 12:20:30.483077 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/47aa63e7-b7b2-4c1d-a73d-348b50cc5d2b-operator-scripts\") pod \"openstack-cell1-galera-0\" (UID: \"47aa63e7-b7b2-4c1d-a73d-348b50cc5d2b\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:20:30 crc kubenswrapper[5052]: I1203 12:20:30.483108 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/47aa63e7-b7b2-4c1d-a73d-348b50cc5d2b-combined-ca-bundle\") pod \"openstack-cell1-galera-0\" (UID: \"47aa63e7-b7b2-4c1d-a73d-348b50cc5d2b\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:20:30 crc kubenswrapper[5052]: I1203 12:20:30.483243 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/47aa63e7-b7b2-4c1d-a73d-348b50cc5d2b-kolla-config\") pod \"openstack-cell1-galera-0\" (UID: \"47aa63e7-b7b2-4c1d-a73d-348b50cc5d2b\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:20:30 crc kubenswrapper[5052]: I1203 12:20:30.483268 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-njjpc\" (UniqueName: \"kubernetes.io/projected/47aa63e7-b7b2-4c1d-a73d-348b50cc5d2b-kube-api-access-njjpc\") pod \"openstack-cell1-galera-0\" (UID: \"47aa63e7-b7b2-4c1d-a73d-348b50cc5d2b\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:20:30 crc kubenswrapper[5052]: I1203 12:20:30.485451 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/47aa63e7-b7b2-4c1d-a73d-348b50cc5d2b-config-data-default\") pod \"openstack-cell1-galera-0\" (UID: \"47aa63e7-b7b2-4c1d-a73d-348b50cc5d2b\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:20:30 crc kubenswrapper[5052]: I1203 12:20:30.486845 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/47aa63e7-b7b2-4c1d-a73d-348b50cc5d2b-operator-scripts\") pod \"openstack-cell1-galera-0\" (UID: \"47aa63e7-b7b2-4c1d-a73d-348b50cc5d2b\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:20:30 crc kubenswrapper[5052]: I1203 12:20:30.487022 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage06-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage06-crc\") pod \"openstack-cell1-galera-0\" (UID: \"47aa63e7-b7b2-4c1d-a73d-348b50cc5d2b\") device mount path \"/mnt/openstack/pv06\"" pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:20:30 crc kubenswrapper[5052]: I1203 12:20:30.493474 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/47aa63e7-b7b2-4c1d-a73d-348b50cc5d2b-combined-ca-bundle\") pod \"openstack-cell1-galera-0\" (UID: \"47aa63e7-b7b2-4c1d-a73d-348b50cc5d2b\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:20:30 crc kubenswrapper[5052]: I1203 12:20:30.493813 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/47aa63e7-b7b2-4c1d-a73d-348b50cc5d2b-config-data-generated\") pod \"openstack-cell1-galera-0\" (UID: \"47aa63e7-b7b2-4c1d-a73d-348b50cc5d2b\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:20:30 crc kubenswrapper[5052]: I1203 12:20:30.497140 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/47aa63e7-b7b2-4c1d-a73d-348b50cc5d2b-galera-tls-certs\") pod \"openstack-cell1-galera-0\" (UID: \"47aa63e7-b7b2-4c1d-a73d-348b50cc5d2b\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:20:30 crc kubenswrapper[5052]: I1203 12:20:30.503375 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:20:30 crc kubenswrapper[5052]: I1203 12:20:30.504659 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/47aa63e7-b7b2-4c1d-a73d-348b50cc5d2b-kolla-config\") pod \"openstack-cell1-galera-0\" (UID: \"47aa63e7-b7b2-4c1d-a73d-348b50cc5d2b\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:20:30 crc kubenswrapper[5052]: I1203 12:20:30.516062 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-njjpc\" (UniqueName: \"kubernetes.io/projected/47aa63e7-b7b2-4c1d-a73d-348b50cc5d2b-kube-api-access-njjpc\") pod \"openstack-cell1-galera-0\" (UID: \"47aa63e7-b7b2-4c1d-a73d-348b50cc5d2b\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:20:30 crc kubenswrapper[5052]: I1203 12:20:30.553649 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage06-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage06-crc\") pod \"openstack-cell1-galera-0\" (UID: \"47aa63e7-b7b2-4c1d-a73d-348b50cc5d2b\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:20:30 crc kubenswrapper[5052]: I1203 12:20:30.559867 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/openstack-galera-0"] Dec 03 12:20:30 crc kubenswrapper[5052]: I1203 12:20:30.585570 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/openstack-galera-0"] Dec 03 12:20:30 crc kubenswrapper[5052]: I1203 12:20:30.617365 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/openstack-galera-0"] Dec 03 12:20:30 crc kubenswrapper[5052]: I1203 12:20:30.619658 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:20:30 crc kubenswrapper[5052]: I1203 12:20:30.633167 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"openstack-config-data" Dec 03 12:20:30 crc kubenswrapper[5052]: I1203 12:20:30.633398 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"galera-openstack-dockercfg-dlq6n" Dec 03 12:20:30 crc kubenswrapper[5052]: I1203 12:20:30.633429 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"openstack-scripts" Dec 03 12:20:30 crc kubenswrapper[5052]: I1203 12:20:30.633640 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-galera-openstack-svc" Dec 03 12:20:30 crc kubenswrapper[5052]: I1203 12:20:30.640266 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/openstack-galera-0"] Dec 03 12:20:30 crc kubenswrapper[5052]: I1203 12:20:30.826090 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:20:30 crc kubenswrapper[5052]: I1203 12:20:30.832927 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage03-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage03-crc\") pod \"openstack-galera-0\" (UID: \"0988bafc-df97-4a53-8308-9e992be58d9a\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:20:30 crc kubenswrapper[5052]: I1203 12:20:30.834490 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/0988bafc-df97-4a53-8308-9e992be58d9a-operator-scripts\") pod \"openstack-galera-0\" (UID: \"0988bafc-df97-4a53-8308-9e992be58d9a\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:20:30 crc kubenswrapper[5052]: I1203 12:20:30.836680 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/0988bafc-df97-4a53-8308-9e992be58d9a-config-data-default\") pod \"openstack-galera-0\" (UID: \"0988bafc-df97-4a53-8308-9e992be58d9a\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:20:30 crc kubenswrapper[5052]: I1203 12:20:30.836937 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/0988bafc-df97-4a53-8308-9e992be58d9a-config-data-generated\") pod \"openstack-galera-0\" (UID: \"0988bafc-df97-4a53-8308-9e992be58d9a\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:20:30 crc kubenswrapper[5052]: I1203 12:20:30.837650 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6zp88\" (UniqueName: \"kubernetes.io/projected/0988bafc-df97-4a53-8308-9e992be58d9a-kube-api-access-6zp88\") pod \"openstack-galera-0\" (UID: \"0988bafc-df97-4a53-8308-9e992be58d9a\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:20:30 crc kubenswrapper[5052]: I1203 12:20:30.837868 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/0988bafc-df97-4a53-8308-9e992be58d9a-galera-tls-certs\") pod \"openstack-galera-0\" (UID: \"0988bafc-df97-4a53-8308-9e992be58d9a\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:20:30 crc kubenswrapper[5052]: I1203 12:20:30.838010 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0988bafc-df97-4a53-8308-9e992be58d9a-combined-ca-bundle\") pod \"openstack-galera-0\" (UID: \"0988bafc-df97-4a53-8308-9e992be58d9a\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:20:30 crc kubenswrapper[5052]: I1203 12:20:30.838936 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/0988bafc-df97-4a53-8308-9e992be58d9a-kolla-config\") pod \"openstack-galera-0\" (UID: \"0988bafc-df97-4a53-8308-9e992be58d9a\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:20:30 crc kubenswrapper[5052]: I1203 12:20:30.940861 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/0988bafc-df97-4a53-8308-9e992be58d9a-config-data-default\") pod \"openstack-galera-0\" (UID: \"0988bafc-df97-4a53-8308-9e992be58d9a\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:20:30 crc kubenswrapper[5052]: I1203 12:20:30.940989 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/0988bafc-df97-4a53-8308-9e992be58d9a-config-data-generated\") pod \"openstack-galera-0\" (UID: \"0988bafc-df97-4a53-8308-9e992be58d9a\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:20:30 crc kubenswrapper[5052]: I1203 12:20:30.941035 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-6zp88\" (UniqueName: \"kubernetes.io/projected/0988bafc-df97-4a53-8308-9e992be58d9a-kube-api-access-6zp88\") pod \"openstack-galera-0\" (UID: \"0988bafc-df97-4a53-8308-9e992be58d9a\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:20:30 crc kubenswrapper[5052]: I1203 12:20:30.941063 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/0988bafc-df97-4a53-8308-9e992be58d9a-galera-tls-certs\") pod \"openstack-galera-0\" (UID: \"0988bafc-df97-4a53-8308-9e992be58d9a\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:20:30 crc kubenswrapper[5052]: I1203 12:20:30.941097 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0988bafc-df97-4a53-8308-9e992be58d9a-combined-ca-bundle\") pod \"openstack-galera-0\" (UID: \"0988bafc-df97-4a53-8308-9e992be58d9a\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:20:30 crc kubenswrapper[5052]: I1203 12:20:30.941126 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/0988bafc-df97-4a53-8308-9e992be58d9a-kolla-config\") pod \"openstack-galera-0\" (UID: \"0988bafc-df97-4a53-8308-9e992be58d9a\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:20:30 crc kubenswrapper[5052]: I1203 12:20:30.941599 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage03-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage03-crc\") pod \"openstack-galera-0\" (UID: \"0988bafc-df97-4a53-8308-9e992be58d9a\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:20:30 crc kubenswrapper[5052]: I1203 12:20:30.941675 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/0988bafc-df97-4a53-8308-9e992be58d9a-operator-scripts\") pod \"openstack-galera-0\" (UID: \"0988bafc-df97-4a53-8308-9e992be58d9a\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:20:30 crc kubenswrapper[5052]: I1203 12:20:30.943682 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/0988bafc-df97-4a53-8308-9e992be58d9a-config-data-default\") pod \"openstack-galera-0\" (UID: \"0988bafc-df97-4a53-8308-9e992be58d9a\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:20:30 crc kubenswrapper[5052]: I1203 12:20:30.944029 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/0988bafc-df97-4a53-8308-9e992be58d9a-config-data-generated\") pod \"openstack-galera-0\" (UID: \"0988bafc-df97-4a53-8308-9e992be58d9a\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:20:30 crc kubenswrapper[5052]: I1203 12:20:30.944314 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/0988bafc-df97-4a53-8308-9e992be58d9a-operator-scripts\") pod \"openstack-galera-0\" (UID: \"0988bafc-df97-4a53-8308-9e992be58d9a\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:20:30 crc kubenswrapper[5052]: I1203 12:20:30.944582 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/0988bafc-df97-4a53-8308-9e992be58d9a-kolla-config\") pod \"openstack-galera-0\" (UID: \"0988bafc-df97-4a53-8308-9e992be58d9a\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:20:30 crc kubenswrapper[5052]: I1203 12:20:30.944833 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage03-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage03-crc\") pod \"openstack-galera-0\" (UID: \"0988bafc-df97-4a53-8308-9e992be58d9a\") device mount path \"/mnt/openstack/pv03\"" pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:20:30 crc kubenswrapper[5052]: I1203 12:20:30.945413 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0988bafc-df97-4a53-8308-9e992be58d9a-combined-ca-bundle\") pod \"openstack-galera-0\" (UID: \"0988bafc-df97-4a53-8308-9e992be58d9a\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:20:30 crc kubenswrapper[5052]: I1203 12:20:30.947007 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/0988bafc-df97-4a53-8308-9e992be58d9a-galera-tls-certs\") pod \"openstack-galera-0\" (UID: \"0988bafc-df97-4a53-8308-9e992be58d9a\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:20:30 crc kubenswrapper[5052]: I1203 12:20:30.971072 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-6zp88\" (UniqueName: \"kubernetes.io/projected/0988bafc-df97-4a53-8308-9e992be58d9a-kube-api-access-6zp88\") pod \"openstack-galera-0\" (UID: \"0988bafc-df97-4a53-8308-9e992be58d9a\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:20:31 crc kubenswrapper[5052]: I1203 12:20:31.009626 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" podUID="1ec6b1f7-5abb-4063-80fa-b5ec907dcb91" containerName="rabbitmq" probeResult="failure" output="dial tcp 10.217.1.59:5671: connect: connection refused" Dec 03 12:20:31 crc kubenswrapper[5052]: I1203 12:20:31.045866 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage03-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage03-crc\") pod \"openstack-galera-0\" (UID: \"0988bafc-df97-4a53-8308-9e992be58d9a\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:20:31 crc kubenswrapper[5052]: I1203 12:20:31.064493 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-nb-0" event={"ID":"70cedf3b-c9b9-4e5c-81a2-ba134b150997","Type":"ContainerStarted","Data":"c7645a9210d2fc1e2e3f163388ea3b91c8a787603d06989bb0272c5699b638c7"} Dec 03 12:20:31 crc kubenswrapper[5052]: I1203 12:20:31.100763 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"ae23fe34-7bf4-47fc-addc-5edda0399c77","Type":"ContainerStarted","Data":"2c50059422ef609ba1f9e4b0f091ebdd6434fb16741e9f966d2f651d24ee2458"} Dec 03 12:20:31 crc kubenswrapper[5052]: I1203 12:20:31.120785 5052 scope.go:117] "RemoveContainer" containerID="09b4a0072741f60499d24f40cc4ff3406d5d22671ced431363ab865172786300" Dec 03 12:20:31 crc kubenswrapper[5052]: E1203 12:20:31.121367 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 12:20:31 crc kubenswrapper[5052]: I1203 12:20:31.127281 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/ovsdbserver-nb-0" podStartSLOduration=6.127258367 podStartE2EDuration="6.127258367s" podCreationTimestamp="2025-12-03 12:20:25 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:20:31.111724953 +0000 UTC m=+3621.699966707" watchObservedRunningTime="2025-12-03 12:20:31.127258367 +0000 UTC m=+3621.715500131" Dec 03 12:20:31 crc kubenswrapper[5052]: I1203 12:20:31.139807 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovn-northd-0" event={"ID":"a77df94c-3e83-47a5-9b5f-eb0aeb41bc3d","Type":"ContainerStarted","Data":"97a957b8c33e1b7a5cef18f4383a4faea413910406298074a6052c04396cec18"} Dec 03 12:20:31 crc kubenswrapper[5052]: I1203 12:20:31.139884 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovn-northd-0" event={"ID":"a77df94c-3e83-47a5-9b5f-eb0aeb41bc3d","Type":"ContainerStarted","Data":"c5318b2d6cbe69bd088c706a2833c45418c4ceb09d5906c6f5dc5ce8834b0109"} Dec 03 12:20:31 crc kubenswrapper[5052]: I1203 12:20:31.143867 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-sb-0"] Dec 03 12:20:31 crc kubenswrapper[5052]: I1203 12:20:31.148535 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-hjwg6" event={"ID":"9ea78435-8fd9-4321-9e48-3aaae833dcf2","Type":"ContainerStarted","Data":"4acfccfaa6f110a7544ac7a681df22aceab18e0f00a4e818bfb5f8c9536032f3"} Dec 03 12:20:31 crc kubenswrapper[5052]: I1203 12:20:31.193059 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:20:31 crc kubenswrapper[5052]: I1203 12:20:31.194415 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:20:31 crc kubenswrapper[5052]: I1203 12:20:31.197861 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-hjwg6" podStartSLOduration=7.197846191 podStartE2EDuration="7.197846191s" podCreationTimestamp="2025-12-03 12:20:24 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:20:31.173684392 +0000 UTC m=+3621.761926176" watchObservedRunningTime="2025-12-03 12:20:31.197846191 +0000 UTC m=+3621.786087955" Dec 03 12:20:31 crc kubenswrapper[5052]: I1203 12:20:31.290186 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:20:31 crc kubenswrapper[5052]: I1203 12:20:31.463281 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 12:20:31 crc kubenswrapper[5052]: I1203 12:20:31.544457 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/memcached-0"] Dec 03 12:20:31 crc kubenswrapper[5052]: W1203 12:20:31.629993 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podadcea736_eb15_4768_b12c_edb4c810beb3.slice/crio-e4968308844574087d5174a9ee90bc9f9ed1b29332de5a1f79f4ced53f213edb WatchSource:0}: Error finding container e4968308844574087d5174a9ee90bc9f9ed1b29332de5a1f79f4ced53f213edb: Status 404 returned error can't find the container with id e4968308844574087d5174a9ee90bc9f9ed1b29332de5a1f79f4ced53f213edb Dec 03 12:20:31 crc kubenswrapper[5052]: I1203 12:20:31.754780 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5067bb8d-d668-46a6-b9de-8065266cde63" path="/var/lib/kubelet/pods/5067bb8d-d668-46a6-b9de-8065266cde63/volumes" Dec 03 12:20:31 crc kubenswrapper[5052]: I1203 12:20:31.755946 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a56992d9-4662-444f-8a45-0375d0855946" path="/var/lib/kubelet/pods/a56992d9-4662-444f-8a45-0375d0855946/volumes" Dec 03 12:20:31 crc kubenswrapper[5052]: I1203 12:20:31.757469 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/kube-state-metrics-0"] Dec 03 12:20:31 crc kubenswrapper[5052]: I1203 12:20:31.946185 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:20:31 crc kubenswrapper[5052]: I1203 12:20:31.972771 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/openstack-cell1-galera-0"] Dec 03 12:20:31 crc kubenswrapper[5052]: I1203 12:20:31.995398 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/rabbitmq-server-0" podUID="683b7d5f-a88a-4dc0-ae83-af3c6444c7ff" containerName="rabbitmq" probeResult="failure" output="dial tcp 10.217.1.60:5671: connect: connection refused" Dec 03 12:20:32 crc kubenswrapper[5052]: I1203 12:20:32.196112 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-sb-0" event={"ID":"19a09467-4c91-40f5-b83a-d43a8eb5f2dc","Type":"ContainerStarted","Data":"0642a54f4a79fd94ab1beba706cc6d89bfd392723aef5b869ebade6546d4c912"} Dec 03 12:20:32 crc kubenswrapper[5052]: I1203 12:20:32.205217 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovn-northd-0" event={"ID":"a77df94c-3e83-47a5-9b5f-eb0aeb41bc3d","Type":"ContainerStarted","Data":"cf3832a4b1ebcd0697f163ca295f373357a42dc4f1fd311347b02c08c7239df4"} Dec 03 12:20:32 crc kubenswrapper[5052]: I1203 12:20:32.207485 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:20:32 crc kubenswrapper[5052]: I1203 12:20:32.213422 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/memcached-0" event={"ID":"adcea736-eb15-4768-b12c-edb4c810beb3","Type":"ContainerStarted","Data":"7eee2cae018afa667fc80fb6f8e55b05f28d5f2bfff1ccd7ad972013dda2c344"} Dec 03 12:20:32 crc kubenswrapper[5052]: I1203 12:20:32.213858 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/memcached-0" event={"ID":"adcea736-eb15-4768-b12c-edb4c810beb3","Type":"ContainerStarted","Data":"e4968308844574087d5174a9ee90bc9f9ed1b29332de5a1f79f4ced53f213edb"} Dec 03 12:20:32 crc kubenswrapper[5052]: I1203 12:20:32.218113 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/memcached-0" Dec 03 12:20:32 crc kubenswrapper[5052]: I1203 12:20:32.220814 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-api-0" podUID="ce16d54b-ef04-429b-8795-2865395f7774" containerName="nova-api-api" probeResult="failure" output="Get \"https://10.217.1.145:8774/\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Dec 03 12:20:32 crc kubenswrapper[5052]: I1203 12:20:32.221156 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-cell1-galera-0" event={"ID":"47aa63e7-b7b2-4c1d-a73d-348b50cc5d2b","Type":"ContainerStarted","Data":"c5698971234998dd0322ba0867ba6b5c54ebfdcf6ae1aa262542b9b50030540b"} Dec 03 12:20:32 crc kubenswrapper[5052]: I1203 12:20:32.220918 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-api-0" podUID="ce16d54b-ef04-429b-8795-2865395f7774" containerName="nova-api-log" probeResult="failure" output="Get \"https://10.217.1.145:8774/\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Dec 03 12:20:32 crc kubenswrapper[5052]: I1203 12:20:32.236629 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" event={"ID":"9e3a0650-c101-4bfa-8137-9909bf8ed4c7","Type":"ContainerStarted","Data":"c5e90a99765d5931a5d6cd2673f9bb42cd7a8b05426a276c6f134ae97869e630"} Dec 03 12:20:32 crc kubenswrapper[5052]: I1203 12:20:32.236704 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" event={"ID":"9e3a0650-c101-4bfa-8137-9909bf8ed4c7","Type":"ContainerStarted","Data":"95666e9bd25d9c51049cde002d933bd54c91313b21de8911bf336e7370a0f3b3"} Dec 03 12:20:32 crc kubenswrapper[5052]: I1203 12:20:32.242109 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/kube-state-metrics-0" event={"ID":"b657a259-eb7a-48e8-8f41-2a3c077aff2a","Type":"ContainerStarted","Data":"103ec0f2998cccccbbff8045e265d938a66f7f6517e99492386e58e8b92e443f"} Dec 03 12:20:32 crc kubenswrapper[5052]: I1203 12:20:32.260995 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/ovn-northd-0" podStartSLOduration=4.260974685 podStartE2EDuration="4.260974685s" podCreationTimestamp="2025-12-03 12:20:28 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:20:32.241461118 +0000 UTC m=+3622.829702892" watchObservedRunningTime="2025-12-03 12:20:32.260974685 +0000 UTC m=+3622.849216449" Dec 03 12:20:32 crc kubenswrapper[5052]: I1203 12:20:32.279491 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"ae23fe34-7bf4-47fc-addc-5edda0399c77","Type":"ContainerStarted","Data":"e2e14fd70f01beb40a6d9c1089a3d22a3f55f619c64d96c8e07abaa9aceec7cc"} Dec 03 12:20:32 crc kubenswrapper[5052]: I1203 12:20:32.283158 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" podStartSLOduration=3.283138157 podStartE2EDuration="3.283138157s" podCreationTimestamp="2025-12-03 12:20:29 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:20:32.274574373 +0000 UTC m=+3622.862816137" watchObservedRunningTime="2025-12-03 12:20:32.283138157 +0000 UTC m=+3622.871379921" Dec 03 12:20:32 crc kubenswrapper[5052]: I1203 12:20:32.332154 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/openstack-galera-0"] Dec 03 12:20:32 crc kubenswrapper[5052]: I1203 12:20:32.335096 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/memcached-0" podStartSLOduration=3.33508474 podStartE2EDuration="3.33508474s" podCreationTimestamp="2025-12-03 12:20:29 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:20:32.314882503 +0000 UTC m=+3622.903124267" watchObservedRunningTime="2025-12-03 12:20:32.33508474 +0000 UTC m=+3622.923326504" Dec 03 12:20:32 crc kubenswrapper[5052]: W1203 12:20:32.352477 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0988bafc_df97_4a53_8308_9e992be58d9a.slice/crio-479380ef7f22d0dd218de093b97f58b18735e0c4321f78d3eb90de6a30bf5d82 WatchSource:0}: Error finding container 479380ef7f22d0dd218de093b97f58b18735e0c4321f78d3eb90de6a30bf5d82: Status 404 returned error can't find the container with id 479380ef7f22d0dd218de093b97f58b18735e0c4321f78d3eb90de6a30bf5d82 Dec 03 12:20:32 crc kubenswrapper[5052]: E1203 12:20:32.913761 5052 cadvisor_stats_provider.go:516] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod47aa63e7_b7b2_4c1d_a73d_348b50cc5d2b.slice/crio-conmon-4b6df714a3fd21bc70f3269c776fc01e4b5e78532c5feda754673b2ff6304a5e.scope\": RecentStats: unable to find data in memory cache]" Dec 03 12:20:32 crc kubenswrapper[5052]: I1203 12:20:32.946469 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:20:33 crc kubenswrapper[5052]: I1203 12:20:33.000089 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:20:33 crc kubenswrapper[5052]: I1203 12:20:33.288254 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-sb-0" event={"ID":"19a09467-4c91-40f5-b83a-d43a8eb5f2dc","Type":"ContainerStarted","Data":"5c0846c17b1ab7bae05ee7ab757a5ac9bfa9ad58f76d6d21d76a677a190e3f91"} Dec 03 12:20:33 crc kubenswrapper[5052]: I1203 12:20:33.288304 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-sb-0" event={"ID":"19a09467-4c91-40f5-b83a-d43a8eb5f2dc","Type":"ContainerStarted","Data":"81e8dead6ae4e8433ea10d47201edd08f411e7f097f98bf35c829bcf51a695f9"} Dec 03 12:20:33 crc kubenswrapper[5052]: I1203 12:20:33.293132 5052 generic.go:334] "Generic (PLEG): container finished" podID="47aa63e7-b7b2-4c1d-a73d-348b50cc5d2b" containerID="4b6df714a3fd21bc70f3269c776fc01e4b5e78532c5feda754673b2ff6304a5e" exitCode=0 Dec 03 12:20:33 crc kubenswrapper[5052]: I1203 12:20:33.293211 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-cell1-galera-0" event={"ID":"47aa63e7-b7b2-4c1d-a73d-348b50cc5d2b","Type":"ContainerDied","Data":"4b6df714a3fd21bc70f3269c776fc01e4b5e78532c5feda754673b2ff6304a5e"} Dec 03 12:20:33 crc kubenswrapper[5052]: I1203 12:20:33.299122 5052 generic.go:334] "Generic (PLEG): container finished" podID="0988bafc-df97-4a53-8308-9e992be58d9a" containerID="d9cc6d1bf947eaf3ffcfecf84ec2f0efe75eb932ba489b21a3991f6ce0828bc4" exitCode=0 Dec 03 12:20:33 crc kubenswrapper[5052]: I1203 12:20:33.299222 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-galera-0" event={"ID":"0988bafc-df97-4a53-8308-9e992be58d9a","Type":"ContainerDied","Data":"d9cc6d1bf947eaf3ffcfecf84ec2f0efe75eb932ba489b21a3991f6ce0828bc4"} Dec 03 12:20:33 crc kubenswrapper[5052]: I1203 12:20:33.299249 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-galera-0" event={"ID":"0988bafc-df97-4a53-8308-9e992be58d9a","Type":"ContainerStarted","Data":"479380ef7f22d0dd218de093b97f58b18735e0c4321f78d3eb90de6a30bf5d82"} Dec 03 12:20:33 crc kubenswrapper[5052]: I1203 12:20:33.307829 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/kube-state-metrics-0" event={"ID":"b657a259-eb7a-48e8-8f41-2a3c077aff2a","Type":"ContainerStarted","Data":"9569b49a51e3eb2f02cd20195f982384cb40ad5f6b6fd5f65cf8a8a8fff4d592"} Dec 03 12:20:33 crc kubenswrapper[5052]: I1203 12:20:33.308771 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:20:33 crc kubenswrapper[5052]: I1203 12:20:33.330391 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/ovsdbserver-sb-0" podStartSLOduration=5.330372208 podStartE2EDuration="5.330372208s" podCreationTimestamp="2025-12-03 12:20:28 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:20:33.322174144 +0000 UTC m=+3623.910415898" watchObservedRunningTime="2025-12-03 12:20:33.330372208 +0000 UTC m=+3623.918613972" Dec 03 12:20:33 crc kubenswrapper[5052]: I1203 12:20:33.407818 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/kube-state-metrics-0" podStartSLOduration=3.8966972699999998 podStartE2EDuration="4.407801318s" podCreationTimestamp="2025-12-03 12:20:29 +0000 UTC" firstStartedPulling="2025-12-03 12:20:31.769820186 +0000 UTC m=+3622.358061950" lastFinishedPulling="2025-12-03 12:20:32.280924234 +0000 UTC m=+3622.869165998" observedRunningTime="2025-12-03 12:20:33.356864664 +0000 UTC m=+3623.945106438" watchObservedRunningTime="2025-12-03 12:20:33.407801318 +0000 UTC m=+3623.996043082" Dec 03 12:20:34 crc kubenswrapper[5052]: I1203 12:20:34.323451 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"ae23fe34-7bf4-47fc-addc-5edda0399c77","Type":"ContainerStarted","Data":"92acee0db3ee456b921b9f9d70e9053b7851f29b4430b86d5f9ff5646f3a41fe"} Dec 03 12:20:34 crc kubenswrapper[5052]: I1203 12:20:34.323854 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="ae23fe34-7bf4-47fc-addc-5edda0399c77" containerName="ceilometer-central-agent" containerID="cri-o://da65d715c37dc90aaa5503b209873b3a777e78c8e5cca7b7707528a1f1fc1b70" gracePeriod=30 Dec 03 12:20:34 crc kubenswrapper[5052]: I1203 12:20:34.324181 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:20:34 crc kubenswrapper[5052]: I1203 12:20:34.324659 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="ae23fe34-7bf4-47fc-addc-5edda0399c77" containerName="sg-core" containerID="cri-o://e2e14fd70f01beb40a6d9c1089a3d22a3f55f619c64d96c8e07abaa9aceec7cc" gracePeriod=30 Dec 03 12:20:34 crc kubenswrapper[5052]: I1203 12:20:34.324758 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="ae23fe34-7bf4-47fc-addc-5edda0399c77" containerName="proxy-httpd" containerID="cri-o://92acee0db3ee456b921b9f9d70e9053b7851f29b4430b86d5f9ff5646f3a41fe" gracePeriod=30 Dec 03 12:20:34 crc kubenswrapper[5052]: I1203 12:20:34.324835 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="ae23fe34-7bf4-47fc-addc-5edda0399c77" containerName="ceilometer-notification-agent" containerID="cri-o://2c50059422ef609ba1f9e4b0f091ebdd6434fb16741e9f966d2f651d24ee2458" gracePeriod=30 Dec 03 12:20:34 crc kubenswrapper[5052]: I1203 12:20:34.327790 5052 generic.go:334] "Generic (PLEG): container finished" podID="d5750ef1-ce21-4092-9612-13d6f6c923df" containerID="66ff70190e4169f7ceded95ca82e7cd10045ed2cce8eb37b9d1a20b3e28c1110" exitCode=0 Dec 03 12:20:34 crc kubenswrapper[5052]: I1203 12:20:34.327862 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"d5750ef1-ce21-4092-9612-13d6f6c923df","Type":"ContainerDied","Data":"66ff70190e4169f7ceded95ca82e7cd10045ed2cce8eb37b9d1a20b3e28c1110"} Dec 03 12:20:34 crc kubenswrapper[5052]: I1203 12:20:34.327892 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"d5750ef1-ce21-4092-9612-13d6f6c923df","Type":"ContainerStarted","Data":"58cdd95d026e2885f81c339ec294204441c3b8e523ecb43a19f7a3d140811ca5"} Dec 03 12:20:34 crc kubenswrapper[5052]: I1203 12:20:34.341603 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-cell1-galera-0" event={"ID":"47aa63e7-b7b2-4c1d-a73d-348b50cc5d2b","Type":"ContainerStarted","Data":"37fd4ad16311c4c25724489aee98ab54bb95696bace9c0196b6418d1b39c27b1"} Dec 03 12:20:34 crc kubenswrapper[5052]: I1203 12:20:34.362167 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-galera-0" event={"ID":"0988bafc-df97-4a53-8308-9e992be58d9a","Type":"ContainerStarted","Data":"8a1128866a1e3dbe3fe50f0d8c53319b6d90d894631644af6969f5f9926f2eaa"} Dec 03 12:20:34 crc kubenswrapper[5052]: I1203 12:20:34.382418 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/ceilometer-0" podStartSLOduration=3.565736358 podStartE2EDuration="13.382375234s" podCreationTimestamp="2025-12-03 12:20:21 +0000 UTC" firstStartedPulling="2025-12-03 12:20:23.478685491 +0000 UTC m=+3614.066927255" lastFinishedPulling="2025-12-03 12:20:33.295324367 +0000 UTC m=+3623.883566131" observedRunningTime="2025-12-03 12:20:34.359530782 +0000 UTC m=+3624.947772556" watchObservedRunningTime="2025-12-03 12:20:34.382375234 +0000 UTC m=+3624.970617018" Dec 03 12:20:34 crc kubenswrapper[5052]: I1203 12:20:34.424875 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/openstack-cell1-galera-0" podStartSLOduration=4.424854927 podStartE2EDuration="4.424854927s" podCreationTimestamp="2025-12-03 12:20:30 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:20:34.414076029 +0000 UTC m=+3625.002317823" watchObservedRunningTime="2025-12-03 12:20:34.424854927 +0000 UTC m=+3625.013096691" Dec 03 12:20:34 crc kubenswrapper[5052]: I1203 12:20:34.429929 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:20:34 crc kubenswrapper[5052]: I1203 12:20:34.451601 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/openstack-galera-0" podStartSLOduration=4.45158144 podStartE2EDuration="4.45158144s" podCreationTimestamp="2025-12-03 12:20:30 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:20:34.450023195 +0000 UTC m=+3625.038264979" watchObservedRunningTime="2025-12-03 12:20:34.45158144 +0000 UTC m=+3625.039823204" Dec 03 12:20:35 crc kubenswrapper[5052]: I1203 12:20:35.190344 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:20:35 crc kubenswrapper[5052]: I1203 12:20:35.200501 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:20:35 crc kubenswrapper[5052]: I1203 12:20:35.250809 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:20:35 crc kubenswrapper[5052]: I1203 12:20:35.295296 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/ae23fe34-7bf4-47fc-addc-5edda0399c77-ceilometer-tls-certs\") pod \"ae23fe34-7bf4-47fc-addc-5edda0399c77\" (UID: \"ae23fe34-7bf4-47fc-addc-5edda0399c77\") " Dec 03 12:20:35 crc kubenswrapper[5052]: I1203 12:20:35.295849 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/ae23fe34-7bf4-47fc-addc-5edda0399c77-run-httpd\") pod \"ae23fe34-7bf4-47fc-addc-5edda0399c77\" (UID: \"ae23fe34-7bf4-47fc-addc-5edda0399c77\") " Dec 03 12:20:35 crc kubenswrapper[5052]: I1203 12:20:35.295886 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ae23fe34-7bf4-47fc-addc-5edda0399c77-combined-ca-bundle\") pod \"ae23fe34-7bf4-47fc-addc-5edda0399c77\" (UID: \"ae23fe34-7bf4-47fc-addc-5edda0399c77\") " Dec 03 12:20:35 crc kubenswrapper[5052]: I1203 12:20:35.295951 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/ae23fe34-7bf4-47fc-addc-5edda0399c77-sg-core-conf-yaml\") pod \"ae23fe34-7bf4-47fc-addc-5edda0399c77\" (UID: \"ae23fe34-7bf4-47fc-addc-5edda0399c77\") " Dec 03 12:20:35 crc kubenswrapper[5052]: I1203 12:20:35.296041 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xbhgc\" (UniqueName: \"kubernetes.io/projected/ae23fe34-7bf4-47fc-addc-5edda0399c77-kube-api-access-xbhgc\") pod \"ae23fe34-7bf4-47fc-addc-5edda0399c77\" (UID: \"ae23fe34-7bf4-47fc-addc-5edda0399c77\") " Dec 03 12:20:35 crc kubenswrapper[5052]: I1203 12:20:35.296140 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/ae23fe34-7bf4-47fc-addc-5edda0399c77-scripts\") pod \"ae23fe34-7bf4-47fc-addc-5edda0399c77\" (UID: \"ae23fe34-7bf4-47fc-addc-5edda0399c77\") " Dec 03 12:20:35 crc kubenswrapper[5052]: I1203 12:20:35.296178 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/ae23fe34-7bf4-47fc-addc-5edda0399c77-log-httpd\") pod \"ae23fe34-7bf4-47fc-addc-5edda0399c77\" (UID: \"ae23fe34-7bf4-47fc-addc-5edda0399c77\") " Dec 03 12:20:35 crc kubenswrapper[5052]: I1203 12:20:35.296675 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/ae23fe34-7bf4-47fc-addc-5edda0399c77-config-data\") pod \"ae23fe34-7bf4-47fc-addc-5edda0399c77\" (UID: \"ae23fe34-7bf4-47fc-addc-5edda0399c77\") " Dec 03 12:20:35 crc kubenswrapper[5052]: I1203 12:20:35.296894 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae23fe34-7bf4-47fc-addc-5edda0399c77-run-httpd" (OuterVolumeSpecName: "run-httpd") pod "ae23fe34-7bf4-47fc-addc-5edda0399c77" (UID: "ae23fe34-7bf4-47fc-addc-5edda0399c77"). InnerVolumeSpecName "run-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:20:35 crc kubenswrapper[5052]: I1203 12:20:35.297414 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae23fe34-7bf4-47fc-addc-5edda0399c77-log-httpd" (OuterVolumeSpecName: "log-httpd") pod "ae23fe34-7bf4-47fc-addc-5edda0399c77" (UID: "ae23fe34-7bf4-47fc-addc-5edda0399c77"). InnerVolumeSpecName "log-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:20:35 crc kubenswrapper[5052]: I1203 12:20:35.298192 5052 reconciler_common.go:293] "Volume detached for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/ae23fe34-7bf4-47fc-addc-5edda0399c77-log-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:35 crc kubenswrapper[5052]: I1203 12:20:35.298213 5052 reconciler_common.go:293] "Volume detached for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/ae23fe34-7bf4-47fc-addc-5edda0399c77-run-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:35 crc kubenswrapper[5052]: I1203 12:20:35.302062 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ae23fe34-7bf4-47fc-addc-5edda0399c77-kube-api-access-xbhgc" (OuterVolumeSpecName: "kube-api-access-xbhgc") pod "ae23fe34-7bf4-47fc-addc-5edda0399c77" (UID: "ae23fe34-7bf4-47fc-addc-5edda0399c77"). InnerVolumeSpecName "kube-api-access-xbhgc". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:20:35 crc kubenswrapper[5052]: I1203 12:20:35.320248 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ae23fe34-7bf4-47fc-addc-5edda0399c77-scripts" (OuterVolumeSpecName: "scripts") pod "ae23fe34-7bf4-47fc-addc-5edda0399c77" (UID: "ae23fe34-7bf4-47fc-addc-5edda0399c77"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:20:35 crc kubenswrapper[5052]: I1203 12:20:35.335607 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ae23fe34-7bf4-47fc-addc-5edda0399c77-sg-core-conf-yaml" (OuterVolumeSpecName: "sg-core-conf-yaml") pod "ae23fe34-7bf4-47fc-addc-5edda0399c77" (UID: "ae23fe34-7bf4-47fc-addc-5edda0399c77"). InnerVolumeSpecName "sg-core-conf-yaml". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:20:35 crc kubenswrapper[5052]: I1203 12:20:35.370678 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ae23fe34-7bf4-47fc-addc-5edda0399c77-ceilometer-tls-certs" (OuterVolumeSpecName: "ceilometer-tls-certs") pod "ae23fe34-7bf4-47fc-addc-5edda0399c77" (UID: "ae23fe34-7bf4-47fc-addc-5edda0399c77"). InnerVolumeSpecName "ceilometer-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:20:35 crc kubenswrapper[5052]: I1203 12:20:35.378577 5052 generic.go:334] "Generic (PLEG): container finished" podID="ae23fe34-7bf4-47fc-addc-5edda0399c77" containerID="92acee0db3ee456b921b9f9d70e9053b7851f29b4430b86d5f9ff5646f3a41fe" exitCode=0 Dec 03 12:20:35 crc kubenswrapper[5052]: I1203 12:20:35.378610 5052 generic.go:334] "Generic (PLEG): container finished" podID="ae23fe34-7bf4-47fc-addc-5edda0399c77" containerID="e2e14fd70f01beb40a6d9c1089a3d22a3f55f619c64d96c8e07abaa9aceec7cc" exitCode=2 Dec 03 12:20:35 crc kubenswrapper[5052]: I1203 12:20:35.378618 5052 generic.go:334] "Generic (PLEG): container finished" podID="ae23fe34-7bf4-47fc-addc-5edda0399c77" containerID="2c50059422ef609ba1f9e4b0f091ebdd6434fb16741e9f966d2f651d24ee2458" exitCode=0 Dec 03 12:20:35 crc kubenswrapper[5052]: I1203 12:20:35.378625 5052 generic.go:334] "Generic (PLEG): container finished" podID="ae23fe34-7bf4-47fc-addc-5edda0399c77" containerID="da65d715c37dc90aaa5503b209873b3a777e78c8e5cca7b7707528a1f1fc1b70" exitCode=0 Dec 03 12:20:35 crc kubenswrapper[5052]: I1203 12:20:35.379801 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:20:35 crc kubenswrapper[5052]: I1203 12:20:35.380470 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"ae23fe34-7bf4-47fc-addc-5edda0399c77","Type":"ContainerDied","Data":"92acee0db3ee456b921b9f9d70e9053b7851f29b4430b86d5f9ff5646f3a41fe"} Dec 03 12:20:35 crc kubenswrapper[5052]: I1203 12:20:35.380497 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"ae23fe34-7bf4-47fc-addc-5edda0399c77","Type":"ContainerDied","Data":"e2e14fd70f01beb40a6d9c1089a3d22a3f55f619c64d96c8e07abaa9aceec7cc"} Dec 03 12:20:35 crc kubenswrapper[5052]: I1203 12:20:35.380522 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"ae23fe34-7bf4-47fc-addc-5edda0399c77","Type":"ContainerDied","Data":"2c50059422ef609ba1f9e4b0f091ebdd6434fb16741e9f966d2f651d24ee2458"} Dec 03 12:20:35 crc kubenswrapper[5052]: I1203 12:20:35.380531 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"ae23fe34-7bf4-47fc-addc-5edda0399c77","Type":"ContainerDied","Data":"da65d715c37dc90aaa5503b209873b3a777e78c8e5cca7b7707528a1f1fc1b70"} Dec 03 12:20:35 crc kubenswrapper[5052]: I1203 12:20:35.380540 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"ae23fe34-7bf4-47fc-addc-5edda0399c77","Type":"ContainerDied","Data":"b5504923aab1fd3aa9e6334f88826548fb2609a9f099f5f7ca6488b1e5ce9a6a"} Dec 03 12:20:35 crc kubenswrapper[5052]: I1203 12:20:35.380555 5052 scope.go:117] "RemoveContainer" containerID="92acee0db3ee456b921b9f9d70e9053b7851f29b4430b86d5f9ff5646f3a41fe" Dec 03 12:20:35 crc kubenswrapper[5052]: I1203 12:20:35.403723 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-xbhgc\" (UniqueName: \"kubernetes.io/projected/ae23fe34-7bf4-47fc-addc-5edda0399c77-kube-api-access-xbhgc\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:35 crc kubenswrapper[5052]: I1203 12:20:35.403760 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/ae23fe34-7bf4-47fc-addc-5edda0399c77-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:35 crc kubenswrapper[5052]: I1203 12:20:35.403772 5052 reconciler_common.go:293] "Volume detached for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/ae23fe34-7bf4-47fc-addc-5edda0399c77-ceilometer-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:35 crc kubenswrapper[5052]: I1203 12:20:35.403780 5052 reconciler_common.go:293] "Volume detached for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/ae23fe34-7bf4-47fc-addc-5edda0399c77-sg-core-conf-yaml\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:35 crc kubenswrapper[5052]: I1203 12:20:35.422211 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ae23fe34-7bf4-47fc-addc-5edda0399c77-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "ae23fe34-7bf4-47fc-addc-5edda0399c77" (UID: "ae23fe34-7bf4-47fc-addc-5edda0399c77"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:20:35 crc kubenswrapper[5052]: I1203 12:20:35.473247 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ae23fe34-7bf4-47fc-addc-5edda0399c77-config-data" (OuterVolumeSpecName: "config-data") pod "ae23fe34-7bf4-47fc-addc-5edda0399c77" (UID: "ae23fe34-7bf4-47fc-addc-5edda0399c77"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:20:35 crc kubenswrapper[5052]: I1203 12:20:35.507178 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/ae23fe34-7bf4-47fc-addc-5edda0399c77-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:35 crc kubenswrapper[5052]: I1203 12:20:35.507204 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ae23fe34-7bf4-47fc-addc-5edda0399c77-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:35 crc kubenswrapper[5052]: I1203 12:20:35.587497 5052 scope.go:117] "RemoveContainer" containerID="e2e14fd70f01beb40a6d9c1089a3d22a3f55f619c64d96c8e07abaa9aceec7cc" Dec 03 12:20:35 crc kubenswrapper[5052]: I1203 12:20:35.615361 5052 scope.go:117] "RemoveContainer" containerID="2c50059422ef609ba1f9e4b0f091ebdd6434fb16741e9f966d2f651d24ee2458" Dec 03 12:20:35 crc kubenswrapper[5052]: I1203 12:20:35.661730 5052 scope.go:117] "RemoveContainer" containerID="da65d715c37dc90aaa5503b209873b3a777e78c8e5cca7b7707528a1f1fc1b70" Dec 03 12:20:35 crc kubenswrapper[5052]: I1203 12:20:35.737908 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:20:35 crc kubenswrapper[5052]: I1203 12:20:35.758592 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:20:35 crc kubenswrapper[5052]: I1203 12:20:35.771592 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:20:35 crc kubenswrapper[5052]: E1203 12:20:35.772393 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="ae23fe34-7bf4-47fc-addc-5edda0399c77" containerName="ceilometer-central-agent" Dec 03 12:20:35 crc kubenswrapper[5052]: I1203 12:20:35.772418 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae23fe34-7bf4-47fc-addc-5edda0399c77" containerName="ceilometer-central-agent" Dec 03 12:20:35 crc kubenswrapper[5052]: E1203 12:20:35.772430 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="ae23fe34-7bf4-47fc-addc-5edda0399c77" containerName="sg-core" Dec 03 12:20:35 crc kubenswrapper[5052]: I1203 12:20:35.772438 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae23fe34-7bf4-47fc-addc-5edda0399c77" containerName="sg-core" Dec 03 12:20:35 crc kubenswrapper[5052]: E1203 12:20:35.772449 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="ae23fe34-7bf4-47fc-addc-5edda0399c77" containerName="ceilometer-notification-agent" Dec 03 12:20:35 crc kubenswrapper[5052]: I1203 12:20:35.772456 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae23fe34-7bf4-47fc-addc-5edda0399c77" containerName="ceilometer-notification-agent" Dec 03 12:20:35 crc kubenswrapper[5052]: E1203 12:20:35.772486 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="ae23fe34-7bf4-47fc-addc-5edda0399c77" containerName="proxy-httpd" Dec 03 12:20:35 crc kubenswrapper[5052]: I1203 12:20:35.772492 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae23fe34-7bf4-47fc-addc-5edda0399c77" containerName="proxy-httpd" Dec 03 12:20:35 crc kubenswrapper[5052]: I1203 12:20:35.772747 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="ae23fe34-7bf4-47fc-addc-5edda0399c77" containerName="sg-core" Dec 03 12:20:35 crc kubenswrapper[5052]: I1203 12:20:35.772763 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="ae23fe34-7bf4-47fc-addc-5edda0399c77" containerName="proxy-httpd" Dec 03 12:20:35 crc kubenswrapper[5052]: I1203 12:20:35.772784 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="ae23fe34-7bf4-47fc-addc-5edda0399c77" containerName="ceilometer-notification-agent" Dec 03 12:20:35 crc kubenswrapper[5052]: I1203 12:20:35.772801 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="ae23fe34-7bf4-47fc-addc-5edda0399c77" containerName="ceilometer-central-agent" Dec 03 12:20:35 crc kubenswrapper[5052]: I1203 12:20:35.775109 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:20:35 crc kubenswrapper[5052]: I1203 12:20:35.778855 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-config-data" Dec 03 12:20:35 crc kubenswrapper[5052]: I1203 12:20:35.779016 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-ceilometer-internal-svc" Dec 03 12:20:35 crc kubenswrapper[5052]: I1203 12:20:35.779545 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-scripts" Dec 03 12:20:35 crc kubenswrapper[5052]: I1203 12:20:35.789746 5052 scope.go:117] "RemoveContainer" containerID="92acee0db3ee456b921b9f9d70e9053b7851f29b4430b86d5f9ff5646f3a41fe" Dec 03 12:20:35 crc kubenswrapper[5052]: E1203 12:20:35.792716 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"92acee0db3ee456b921b9f9d70e9053b7851f29b4430b86d5f9ff5646f3a41fe\": container with ID starting with 92acee0db3ee456b921b9f9d70e9053b7851f29b4430b86d5f9ff5646f3a41fe not found: ID does not exist" containerID="92acee0db3ee456b921b9f9d70e9053b7851f29b4430b86d5f9ff5646f3a41fe" Dec 03 12:20:35 crc kubenswrapper[5052]: I1203 12:20:35.792799 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"92acee0db3ee456b921b9f9d70e9053b7851f29b4430b86d5f9ff5646f3a41fe"} err="failed to get container status \"92acee0db3ee456b921b9f9d70e9053b7851f29b4430b86d5f9ff5646f3a41fe\": rpc error: code = NotFound desc = could not find container \"92acee0db3ee456b921b9f9d70e9053b7851f29b4430b86d5f9ff5646f3a41fe\": container with ID starting with 92acee0db3ee456b921b9f9d70e9053b7851f29b4430b86d5f9ff5646f3a41fe not found: ID does not exist" Dec 03 12:20:35 crc kubenswrapper[5052]: I1203 12:20:35.792836 5052 scope.go:117] "RemoveContainer" containerID="e2e14fd70f01beb40a6d9c1089a3d22a3f55f619c64d96c8e07abaa9aceec7cc" Dec 03 12:20:35 crc kubenswrapper[5052]: E1203 12:20:35.796641 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e2e14fd70f01beb40a6d9c1089a3d22a3f55f619c64d96c8e07abaa9aceec7cc\": container with ID starting with e2e14fd70f01beb40a6d9c1089a3d22a3f55f619c64d96c8e07abaa9aceec7cc not found: ID does not exist" containerID="e2e14fd70f01beb40a6d9c1089a3d22a3f55f619c64d96c8e07abaa9aceec7cc" Dec 03 12:20:35 crc kubenswrapper[5052]: I1203 12:20:35.796674 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e2e14fd70f01beb40a6d9c1089a3d22a3f55f619c64d96c8e07abaa9aceec7cc"} err="failed to get container status \"e2e14fd70f01beb40a6d9c1089a3d22a3f55f619c64d96c8e07abaa9aceec7cc\": rpc error: code = NotFound desc = could not find container \"e2e14fd70f01beb40a6d9c1089a3d22a3f55f619c64d96c8e07abaa9aceec7cc\": container with ID starting with e2e14fd70f01beb40a6d9c1089a3d22a3f55f619c64d96c8e07abaa9aceec7cc not found: ID does not exist" Dec 03 12:20:35 crc kubenswrapper[5052]: I1203 12:20:35.796694 5052 scope.go:117] "RemoveContainer" containerID="2c50059422ef609ba1f9e4b0f091ebdd6434fb16741e9f966d2f651d24ee2458" Dec 03 12:20:35 crc kubenswrapper[5052]: E1203 12:20:35.801810 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2c50059422ef609ba1f9e4b0f091ebdd6434fb16741e9f966d2f651d24ee2458\": container with ID starting with 2c50059422ef609ba1f9e4b0f091ebdd6434fb16741e9f966d2f651d24ee2458 not found: ID does not exist" containerID="2c50059422ef609ba1f9e4b0f091ebdd6434fb16741e9f966d2f651d24ee2458" Dec 03 12:20:35 crc kubenswrapper[5052]: I1203 12:20:35.801872 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2c50059422ef609ba1f9e4b0f091ebdd6434fb16741e9f966d2f651d24ee2458"} err="failed to get container status \"2c50059422ef609ba1f9e4b0f091ebdd6434fb16741e9f966d2f651d24ee2458\": rpc error: code = NotFound desc = could not find container \"2c50059422ef609ba1f9e4b0f091ebdd6434fb16741e9f966d2f651d24ee2458\": container with ID starting with 2c50059422ef609ba1f9e4b0f091ebdd6434fb16741e9f966d2f651d24ee2458 not found: ID does not exist" Dec 03 12:20:35 crc kubenswrapper[5052]: I1203 12:20:35.801928 5052 scope.go:117] "RemoveContainer" containerID="da65d715c37dc90aaa5503b209873b3a777e78c8e5cca7b7707528a1f1fc1b70" Dec 03 12:20:35 crc kubenswrapper[5052]: E1203 12:20:35.804244 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"da65d715c37dc90aaa5503b209873b3a777e78c8e5cca7b7707528a1f1fc1b70\": container with ID starting with da65d715c37dc90aaa5503b209873b3a777e78c8e5cca7b7707528a1f1fc1b70 not found: ID does not exist" containerID="da65d715c37dc90aaa5503b209873b3a777e78c8e5cca7b7707528a1f1fc1b70" Dec 03 12:20:35 crc kubenswrapper[5052]: I1203 12:20:35.804305 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"da65d715c37dc90aaa5503b209873b3a777e78c8e5cca7b7707528a1f1fc1b70"} err="failed to get container status \"da65d715c37dc90aaa5503b209873b3a777e78c8e5cca7b7707528a1f1fc1b70\": rpc error: code = NotFound desc = could not find container \"da65d715c37dc90aaa5503b209873b3a777e78c8e5cca7b7707528a1f1fc1b70\": container with ID starting with da65d715c37dc90aaa5503b209873b3a777e78c8e5cca7b7707528a1f1fc1b70 not found: ID does not exist" Dec 03 12:20:35 crc kubenswrapper[5052]: I1203 12:20:35.804347 5052 scope.go:117] "RemoveContainer" containerID="92acee0db3ee456b921b9f9d70e9053b7851f29b4430b86d5f9ff5646f3a41fe" Dec 03 12:20:35 crc kubenswrapper[5052]: I1203 12:20:35.809368 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"92acee0db3ee456b921b9f9d70e9053b7851f29b4430b86d5f9ff5646f3a41fe"} err="failed to get container status \"92acee0db3ee456b921b9f9d70e9053b7851f29b4430b86d5f9ff5646f3a41fe\": rpc error: code = NotFound desc = could not find container \"92acee0db3ee456b921b9f9d70e9053b7851f29b4430b86d5f9ff5646f3a41fe\": container with ID starting with 92acee0db3ee456b921b9f9d70e9053b7851f29b4430b86d5f9ff5646f3a41fe not found: ID does not exist" Dec 03 12:20:35 crc kubenswrapper[5052]: I1203 12:20:35.809611 5052 scope.go:117] "RemoveContainer" containerID="e2e14fd70f01beb40a6d9c1089a3d22a3f55f619c64d96c8e07abaa9aceec7cc" Dec 03 12:20:35 crc kubenswrapper[5052]: I1203 12:20:35.810610 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e2e14fd70f01beb40a6d9c1089a3d22a3f55f619c64d96c8e07abaa9aceec7cc"} err="failed to get container status \"e2e14fd70f01beb40a6d9c1089a3d22a3f55f619c64d96c8e07abaa9aceec7cc\": rpc error: code = NotFound desc = could not find container \"e2e14fd70f01beb40a6d9c1089a3d22a3f55f619c64d96c8e07abaa9aceec7cc\": container with ID starting with e2e14fd70f01beb40a6d9c1089a3d22a3f55f619c64d96c8e07abaa9aceec7cc not found: ID does not exist" Dec 03 12:20:35 crc kubenswrapper[5052]: I1203 12:20:35.810668 5052 scope.go:117] "RemoveContainer" containerID="2c50059422ef609ba1f9e4b0f091ebdd6434fb16741e9f966d2f651d24ee2458" Dec 03 12:20:35 crc kubenswrapper[5052]: I1203 12:20:35.811206 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2c50059422ef609ba1f9e4b0f091ebdd6434fb16741e9f966d2f651d24ee2458"} err="failed to get container status \"2c50059422ef609ba1f9e4b0f091ebdd6434fb16741e9f966d2f651d24ee2458\": rpc error: code = NotFound desc = could not find container \"2c50059422ef609ba1f9e4b0f091ebdd6434fb16741e9f966d2f651d24ee2458\": container with ID starting with 2c50059422ef609ba1f9e4b0f091ebdd6434fb16741e9f966d2f651d24ee2458 not found: ID does not exist" Dec 03 12:20:35 crc kubenswrapper[5052]: I1203 12:20:35.811255 5052 scope.go:117] "RemoveContainer" containerID="da65d715c37dc90aaa5503b209873b3a777e78c8e5cca7b7707528a1f1fc1b70" Dec 03 12:20:35 crc kubenswrapper[5052]: I1203 12:20:35.812750 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/f50a3c6a-8630-4ca5-99cd-4cf76e55fef8-scripts\") pod \"ceilometer-0\" (UID: \"f50a3c6a-8630-4ca5-99cd-4cf76e55fef8\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:20:35 crc kubenswrapper[5052]: I1203 12:20:35.812863 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f50a3c6a-8630-4ca5-99cd-4cf76e55fef8-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"f50a3c6a-8630-4ca5-99cd-4cf76e55fef8\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:20:35 crc kubenswrapper[5052]: I1203 12:20:35.812912 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/f50a3c6a-8630-4ca5-99cd-4cf76e55fef8-log-httpd\") pod \"ceilometer-0\" (UID: \"f50a3c6a-8630-4ca5-99cd-4cf76e55fef8\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:20:35 crc kubenswrapper[5052]: I1203 12:20:35.812946 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/f50a3c6a-8630-4ca5-99cd-4cf76e55fef8-config-data\") pod \"ceilometer-0\" (UID: \"f50a3c6a-8630-4ca5-99cd-4cf76e55fef8\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:20:35 crc kubenswrapper[5052]: I1203 12:20:35.812973 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/f50a3c6a-8630-4ca5-99cd-4cf76e55fef8-ceilometer-tls-certs\") pod \"ceilometer-0\" (UID: \"f50a3c6a-8630-4ca5-99cd-4cf76e55fef8\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:20:35 crc kubenswrapper[5052]: I1203 12:20:35.813053 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/f50a3c6a-8630-4ca5-99cd-4cf76e55fef8-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"f50a3c6a-8630-4ca5-99cd-4cf76e55fef8\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:20:35 crc kubenswrapper[5052]: I1203 12:20:35.813091 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m9ffd\" (UniqueName: \"kubernetes.io/projected/f50a3c6a-8630-4ca5-99cd-4cf76e55fef8-kube-api-access-m9ffd\") pod \"ceilometer-0\" (UID: \"f50a3c6a-8630-4ca5-99cd-4cf76e55fef8\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:20:35 crc kubenswrapper[5052]: I1203 12:20:35.813138 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/f50a3c6a-8630-4ca5-99cd-4cf76e55fef8-run-httpd\") pod \"ceilometer-0\" (UID: \"f50a3c6a-8630-4ca5-99cd-4cf76e55fef8\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:20:35 crc kubenswrapper[5052]: I1203 12:20:35.813321 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"da65d715c37dc90aaa5503b209873b3a777e78c8e5cca7b7707528a1f1fc1b70"} err="failed to get container status \"da65d715c37dc90aaa5503b209873b3a777e78c8e5cca7b7707528a1f1fc1b70\": rpc error: code = NotFound desc = could not find container \"da65d715c37dc90aaa5503b209873b3a777e78c8e5cca7b7707528a1f1fc1b70\": container with ID starting with da65d715c37dc90aaa5503b209873b3a777e78c8e5cca7b7707528a1f1fc1b70 not found: ID does not exist" Dec 03 12:20:35 crc kubenswrapper[5052]: I1203 12:20:35.813420 5052 scope.go:117] "RemoveContainer" containerID="92acee0db3ee456b921b9f9d70e9053b7851f29b4430b86d5f9ff5646f3a41fe" Dec 03 12:20:35 crc kubenswrapper[5052]: I1203 12:20:35.817025 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:20:35 crc kubenswrapper[5052]: I1203 12:20:35.819916 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"92acee0db3ee456b921b9f9d70e9053b7851f29b4430b86d5f9ff5646f3a41fe"} err="failed to get container status \"92acee0db3ee456b921b9f9d70e9053b7851f29b4430b86d5f9ff5646f3a41fe\": rpc error: code = NotFound desc = could not find container \"92acee0db3ee456b921b9f9d70e9053b7851f29b4430b86d5f9ff5646f3a41fe\": container with ID starting with 92acee0db3ee456b921b9f9d70e9053b7851f29b4430b86d5f9ff5646f3a41fe not found: ID does not exist" Dec 03 12:20:35 crc kubenswrapper[5052]: I1203 12:20:35.819968 5052 scope.go:117] "RemoveContainer" containerID="e2e14fd70f01beb40a6d9c1089a3d22a3f55f619c64d96c8e07abaa9aceec7cc" Dec 03 12:20:35 crc kubenswrapper[5052]: I1203 12:20:35.820380 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e2e14fd70f01beb40a6d9c1089a3d22a3f55f619c64d96c8e07abaa9aceec7cc"} err="failed to get container status \"e2e14fd70f01beb40a6d9c1089a3d22a3f55f619c64d96c8e07abaa9aceec7cc\": rpc error: code = NotFound desc = could not find container \"e2e14fd70f01beb40a6d9c1089a3d22a3f55f619c64d96c8e07abaa9aceec7cc\": container with ID starting with e2e14fd70f01beb40a6d9c1089a3d22a3f55f619c64d96c8e07abaa9aceec7cc not found: ID does not exist" Dec 03 12:20:35 crc kubenswrapper[5052]: I1203 12:20:35.820418 5052 scope.go:117] "RemoveContainer" containerID="2c50059422ef609ba1f9e4b0f091ebdd6434fb16741e9f966d2f651d24ee2458" Dec 03 12:20:35 crc kubenswrapper[5052]: I1203 12:20:35.820771 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2c50059422ef609ba1f9e4b0f091ebdd6434fb16741e9f966d2f651d24ee2458"} err="failed to get container status \"2c50059422ef609ba1f9e4b0f091ebdd6434fb16741e9f966d2f651d24ee2458\": rpc error: code = NotFound desc = could not find container \"2c50059422ef609ba1f9e4b0f091ebdd6434fb16741e9f966d2f651d24ee2458\": container with ID starting with 2c50059422ef609ba1f9e4b0f091ebdd6434fb16741e9f966d2f651d24ee2458 not found: ID does not exist" Dec 03 12:20:35 crc kubenswrapper[5052]: I1203 12:20:35.820801 5052 scope.go:117] "RemoveContainer" containerID="da65d715c37dc90aaa5503b209873b3a777e78c8e5cca7b7707528a1f1fc1b70" Dec 03 12:20:35 crc kubenswrapper[5052]: I1203 12:20:35.821056 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"da65d715c37dc90aaa5503b209873b3a777e78c8e5cca7b7707528a1f1fc1b70"} err="failed to get container status \"da65d715c37dc90aaa5503b209873b3a777e78c8e5cca7b7707528a1f1fc1b70\": rpc error: code = NotFound desc = could not find container \"da65d715c37dc90aaa5503b209873b3a777e78c8e5cca7b7707528a1f1fc1b70\": container with ID starting with da65d715c37dc90aaa5503b209873b3a777e78c8e5cca7b7707528a1f1fc1b70 not found: ID does not exist" Dec 03 12:20:35 crc kubenswrapper[5052]: I1203 12:20:35.821082 5052 scope.go:117] "RemoveContainer" containerID="92acee0db3ee456b921b9f9d70e9053b7851f29b4430b86d5f9ff5646f3a41fe" Dec 03 12:20:35 crc kubenswrapper[5052]: I1203 12:20:35.821342 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"92acee0db3ee456b921b9f9d70e9053b7851f29b4430b86d5f9ff5646f3a41fe"} err="failed to get container status \"92acee0db3ee456b921b9f9d70e9053b7851f29b4430b86d5f9ff5646f3a41fe\": rpc error: code = NotFound desc = could not find container \"92acee0db3ee456b921b9f9d70e9053b7851f29b4430b86d5f9ff5646f3a41fe\": container with ID starting with 92acee0db3ee456b921b9f9d70e9053b7851f29b4430b86d5f9ff5646f3a41fe not found: ID does not exist" Dec 03 12:20:35 crc kubenswrapper[5052]: I1203 12:20:35.821366 5052 scope.go:117] "RemoveContainer" containerID="e2e14fd70f01beb40a6d9c1089a3d22a3f55f619c64d96c8e07abaa9aceec7cc" Dec 03 12:20:35 crc kubenswrapper[5052]: I1203 12:20:35.821635 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e2e14fd70f01beb40a6d9c1089a3d22a3f55f619c64d96c8e07abaa9aceec7cc"} err="failed to get container status \"e2e14fd70f01beb40a6d9c1089a3d22a3f55f619c64d96c8e07abaa9aceec7cc\": rpc error: code = NotFound desc = could not find container \"e2e14fd70f01beb40a6d9c1089a3d22a3f55f619c64d96c8e07abaa9aceec7cc\": container with ID starting with e2e14fd70f01beb40a6d9c1089a3d22a3f55f619c64d96c8e07abaa9aceec7cc not found: ID does not exist" Dec 03 12:20:35 crc kubenswrapper[5052]: I1203 12:20:35.821658 5052 scope.go:117] "RemoveContainer" containerID="2c50059422ef609ba1f9e4b0f091ebdd6434fb16741e9f966d2f651d24ee2458" Dec 03 12:20:35 crc kubenswrapper[5052]: I1203 12:20:35.822027 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2c50059422ef609ba1f9e4b0f091ebdd6434fb16741e9f966d2f651d24ee2458"} err="failed to get container status \"2c50059422ef609ba1f9e4b0f091ebdd6434fb16741e9f966d2f651d24ee2458\": rpc error: code = NotFound desc = could not find container \"2c50059422ef609ba1f9e4b0f091ebdd6434fb16741e9f966d2f651d24ee2458\": container with ID starting with 2c50059422ef609ba1f9e4b0f091ebdd6434fb16741e9f966d2f651d24ee2458 not found: ID does not exist" Dec 03 12:20:35 crc kubenswrapper[5052]: I1203 12:20:35.822050 5052 scope.go:117] "RemoveContainer" containerID="da65d715c37dc90aaa5503b209873b3a777e78c8e5cca7b7707528a1f1fc1b70" Dec 03 12:20:35 crc kubenswrapper[5052]: I1203 12:20:35.822583 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"da65d715c37dc90aaa5503b209873b3a777e78c8e5cca7b7707528a1f1fc1b70"} err="failed to get container status \"da65d715c37dc90aaa5503b209873b3a777e78c8e5cca7b7707528a1f1fc1b70\": rpc error: code = NotFound desc = could not find container \"da65d715c37dc90aaa5503b209873b3a777e78c8e5cca7b7707528a1f1fc1b70\": container with ID starting with da65d715c37dc90aaa5503b209873b3a777e78c8e5cca7b7707528a1f1fc1b70 not found: ID does not exist" Dec 03 12:20:35 crc kubenswrapper[5052]: I1203 12:20:35.915431 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f50a3c6a-8630-4ca5-99cd-4cf76e55fef8-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"f50a3c6a-8630-4ca5-99cd-4cf76e55fef8\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:20:35 crc kubenswrapper[5052]: I1203 12:20:35.915532 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/f50a3c6a-8630-4ca5-99cd-4cf76e55fef8-log-httpd\") pod \"ceilometer-0\" (UID: \"f50a3c6a-8630-4ca5-99cd-4cf76e55fef8\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:20:35 crc kubenswrapper[5052]: I1203 12:20:35.915561 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/f50a3c6a-8630-4ca5-99cd-4cf76e55fef8-config-data\") pod \"ceilometer-0\" (UID: \"f50a3c6a-8630-4ca5-99cd-4cf76e55fef8\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:20:35 crc kubenswrapper[5052]: I1203 12:20:35.915583 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/f50a3c6a-8630-4ca5-99cd-4cf76e55fef8-ceilometer-tls-certs\") pod \"ceilometer-0\" (UID: \"f50a3c6a-8630-4ca5-99cd-4cf76e55fef8\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:20:35 crc kubenswrapper[5052]: I1203 12:20:35.915640 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/f50a3c6a-8630-4ca5-99cd-4cf76e55fef8-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"f50a3c6a-8630-4ca5-99cd-4cf76e55fef8\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:20:35 crc kubenswrapper[5052]: I1203 12:20:35.915664 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-m9ffd\" (UniqueName: \"kubernetes.io/projected/f50a3c6a-8630-4ca5-99cd-4cf76e55fef8-kube-api-access-m9ffd\") pod \"ceilometer-0\" (UID: \"f50a3c6a-8630-4ca5-99cd-4cf76e55fef8\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:20:35 crc kubenswrapper[5052]: I1203 12:20:35.915698 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/f50a3c6a-8630-4ca5-99cd-4cf76e55fef8-run-httpd\") pod \"ceilometer-0\" (UID: \"f50a3c6a-8630-4ca5-99cd-4cf76e55fef8\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:20:35 crc kubenswrapper[5052]: I1203 12:20:35.915756 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/f50a3c6a-8630-4ca5-99cd-4cf76e55fef8-scripts\") pod \"ceilometer-0\" (UID: \"f50a3c6a-8630-4ca5-99cd-4cf76e55fef8\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:20:35 crc kubenswrapper[5052]: I1203 12:20:35.918154 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/f50a3c6a-8630-4ca5-99cd-4cf76e55fef8-log-httpd\") pod \"ceilometer-0\" (UID: \"f50a3c6a-8630-4ca5-99cd-4cf76e55fef8\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:20:35 crc kubenswrapper[5052]: I1203 12:20:35.918341 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/f50a3c6a-8630-4ca5-99cd-4cf76e55fef8-run-httpd\") pod \"ceilometer-0\" (UID: \"f50a3c6a-8630-4ca5-99cd-4cf76e55fef8\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:20:35 crc kubenswrapper[5052]: I1203 12:20:35.920890 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/f50a3c6a-8630-4ca5-99cd-4cf76e55fef8-scripts\") pod \"ceilometer-0\" (UID: \"f50a3c6a-8630-4ca5-99cd-4cf76e55fef8\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:20:35 crc kubenswrapper[5052]: I1203 12:20:35.921257 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/f50a3c6a-8630-4ca5-99cd-4cf76e55fef8-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"f50a3c6a-8630-4ca5-99cd-4cf76e55fef8\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:20:35 crc kubenswrapper[5052]: I1203 12:20:35.922975 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/f50a3c6a-8630-4ca5-99cd-4cf76e55fef8-ceilometer-tls-certs\") pod \"ceilometer-0\" (UID: \"f50a3c6a-8630-4ca5-99cd-4cf76e55fef8\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:20:35 crc kubenswrapper[5052]: I1203 12:20:35.931303 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/f50a3c6a-8630-4ca5-99cd-4cf76e55fef8-config-data\") pod \"ceilometer-0\" (UID: \"f50a3c6a-8630-4ca5-99cd-4cf76e55fef8\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:20:35 crc kubenswrapper[5052]: I1203 12:20:35.934487 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f50a3c6a-8630-4ca5-99cd-4cf76e55fef8-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"f50a3c6a-8630-4ca5-99cd-4cf76e55fef8\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:20:35 crc kubenswrapper[5052]: I1203 12:20:35.936802 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-m9ffd\" (UniqueName: \"kubernetes.io/projected/f50a3c6a-8630-4ca5-99cd-4cf76e55fef8-kube-api-access-m9ffd\") pod \"ceilometer-0\" (UID: \"f50a3c6a-8630-4ca5-99cd-4cf76e55fef8\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:20:36 crc kubenswrapper[5052]: I1203 12:20:36.103835 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:20:36 crc kubenswrapper[5052]: I1203 12:20:36.200429 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:20:36 crc kubenswrapper[5052]: I1203 12:20:36.252833 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:20:36 crc kubenswrapper[5052]: I1203 12:20:36.312699 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:20:36 crc kubenswrapper[5052]: I1203 12:20:36.440966 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"server-conf\" (UniqueName: \"kubernetes.io/configmap/1ec6b1f7-5abb-4063-80fa-b5ec907dcb91-server-conf\") pod \"1ec6b1f7-5abb-4063-80fa-b5ec907dcb91\" (UID: \"1ec6b1f7-5abb-4063-80fa-b5ec907dcb91\") " Dec 03 12:20:36 crc kubenswrapper[5052]: I1203 12:20:36.441228 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/1ec6b1f7-5abb-4063-80fa-b5ec907dcb91-config-data\") pod \"1ec6b1f7-5abb-4063-80fa-b5ec907dcb91\" (UID: \"1ec6b1f7-5abb-4063-80fa-b5ec907dcb91\") " Dec 03 12:20:36 crc kubenswrapper[5052]: I1203 12:20:36.441267 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"erlang-cookie-secret\" (UniqueName: \"kubernetes.io/secret/1ec6b1f7-5abb-4063-80fa-b5ec907dcb91-erlang-cookie-secret\") pod \"1ec6b1f7-5abb-4063-80fa-b5ec907dcb91\" (UID: \"1ec6b1f7-5abb-4063-80fa-b5ec907dcb91\") " Dec 03 12:20:36 crc kubenswrapper[5052]: I1203 12:20:36.441374 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"persistence\" (UniqueName: \"kubernetes.io/local-volume/local-storage15-crc\") pod \"1ec6b1f7-5abb-4063-80fa-b5ec907dcb91\" (UID: \"1ec6b1f7-5abb-4063-80fa-b5ec907dcb91\") " Dec 03 12:20:36 crc kubenswrapper[5052]: I1203 12:20:36.441404 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"rabbitmq-confd\" (UniqueName: \"kubernetes.io/projected/1ec6b1f7-5abb-4063-80fa-b5ec907dcb91-rabbitmq-confd\") pod \"1ec6b1f7-5abb-4063-80fa-b5ec907dcb91\" (UID: \"1ec6b1f7-5abb-4063-80fa-b5ec907dcb91\") " Dec 03 12:20:36 crc kubenswrapper[5052]: I1203 12:20:36.441482 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"pod-info\" (UniqueName: \"kubernetes.io/downward-api/1ec6b1f7-5abb-4063-80fa-b5ec907dcb91-pod-info\") pod \"1ec6b1f7-5abb-4063-80fa-b5ec907dcb91\" (UID: \"1ec6b1f7-5abb-4063-80fa-b5ec907dcb91\") " Dec 03 12:20:36 crc kubenswrapper[5052]: I1203 12:20:36.441543 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9zzmt\" (UniqueName: \"kubernetes.io/projected/1ec6b1f7-5abb-4063-80fa-b5ec907dcb91-kube-api-access-9zzmt\") pod \"1ec6b1f7-5abb-4063-80fa-b5ec907dcb91\" (UID: \"1ec6b1f7-5abb-4063-80fa-b5ec907dcb91\") " Dec 03 12:20:36 crc kubenswrapper[5052]: I1203 12:20:36.441600 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"rabbitmq-erlang-cookie\" (UniqueName: \"kubernetes.io/empty-dir/1ec6b1f7-5abb-4063-80fa-b5ec907dcb91-rabbitmq-erlang-cookie\") pod \"1ec6b1f7-5abb-4063-80fa-b5ec907dcb91\" (UID: \"1ec6b1f7-5abb-4063-80fa-b5ec907dcb91\") " Dec 03 12:20:36 crc kubenswrapper[5052]: I1203 12:20:36.441625 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"rabbitmq-plugins\" (UniqueName: \"kubernetes.io/empty-dir/1ec6b1f7-5abb-4063-80fa-b5ec907dcb91-rabbitmq-plugins\") pod \"1ec6b1f7-5abb-4063-80fa-b5ec907dcb91\" (UID: \"1ec6b1f7-5abb-4063-80fa-b5ec907dcb91\") " Dec 03 12:20:36 crc kubenswrapper[5052]: I1203 12:20:36.441663 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"plugins-conf\" (UniqueName: \"kubernetes.io/configmap/1ec6b1f7-5abb-4063-80fa-b5ec907dcb91-plugins-conf\") pod \"1ec6b1f7-5abb-4063-80fa-b5ec907dcb91\" (UID: \"1ec6b1f7-5abb-4063-80fa-b5ec907dcb91\") " Dec 03 12:20:36 crc kubenswrapper[5052]: I1203 12:20:36.441691 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"rabbitmq-tls\" (UniqueName: \"kubernetes.io/projected/1ec6b1f7-5abb-4063-80fa-b5ec907dcb91-rabbitmq-tls\") pod \"1ec6b1f7-5abb-4063-80fa-b5ec907dcb91\" (UID: \"1ec6b1f7-5abb-4063-80fa-b5ec907dcb91\") " Dec 03 12:20:36 crc kubenswrapper[5052]: I1203 12:20:36.458653 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1ec6b1f7-5abb-4063-80fa-b5ec907dcb91-pod-info" (OuterVolumeSpecName: "pod-info") pod "1ec6b1f7-5abb-4063-80fa-b5ec907dcb91" (UID: "1ec6b1f7-5abb-4063-80fa-b5ec907dcb91"). InnerVolumeSpecName "pod-info". PluginName "kubernetes.io/downward-api", VolumeGidValue "" Dec 03 12:20:36 crc kubenswrapper[5052]: I1203 12:20:36.474748 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1ec6b1f7-5abb-4063-80fa-b5ec907dcb91-kube-api-access-9zzmt" (OuterVolumeSpecName: "kube-api-access-9zzmt") pod "1ec6b1f7-5abb-4063-80fa-b5ec907dcb91" (UID: "1ec6b1f7-5abb-4063-80fa-b5ec907dcb91"). InnerVolumeSpecName "kube-api-access-9zzmt". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:20:36 crc kubenswrapper[5052]: I1203 12:20:36.475195 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ec6b1f7-5abb-4063-80fa-b5ec907dcb91-rabbitmq-erlang-cookie" (OuterVolumeSpecName: "rabbitmq-erlang-cookie") pod "1ec6b1f7-5abb-4063-80fa-b5ec907dcb91" (UID: "1ec6b1f7-5abb-4063-80fa-b5ec907dcb91"). InnerVolumeSpecName "rabbitmq-erlang-cookie". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:20:36 crc kubenswrapper[5052]: I1203 12:20:36.475575 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ec6b1f7-5abb-4063-80fa-b5ec907dcb91-rabbitmq-plugins" (OuterVolumeSpecName: "rabbitmq-plugins") pod "1ec6b1f7-5abb-4063-80fa-b5ec907dcb91" (UID: "1ec6b1f7-5abb-4063-80fa-b5ec907dcb91"). InnerVolumeSpecName "rabbitmq-plugins". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:20:36 crc kubenswrapper[5052]: I1203 12:20:36.476078 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1ec6b1f7-5abb-4063-80fa-b5ec907dcb91-plugins-conf" (OuterVolumeSpecName: "plugins-conf") pod "1ec6b1f7-5abb-4063-80fa-b5ec907dcb91" (UID: "1ec6b1f7-5abb-4063-80fa-b5ec907dcb91"). InnerVolumeSpecName "plugins-conf". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:20:36 crc kubenswrapper[5052]: I1203 12:20:36.484493 5052 generic.go:334] "Generic (PLEG): container finished" podID="1ec6b1f7-5abb-4063-80fa-b5ec907dcb91" containerID="50d90ae2ffca4d0cc92a43c260b1dec75f6540c0f3b6a2f388e0315ce52b8fdd" exitCode=0 Dec 03 12:20:36 crc kubenswrapper[5052]: I1203 12:20:36.484591 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" event={"ID":"1ec6b1f7-5abb-4063-80fa-b5ec907dcb91","Type":"ContainerDied","Data":"50d90ae2ffca4d0cc92a43c260b1dec75f6540c0f3b6a2f388e0315ce52b8fdd"} Dec 03 12:20:36 crc kubenswrapper[5052]: I1203 12:20:36.484615 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" event={"ID":"1ec6b1f7-5abb-4063-80fa-b5ec907dcb91","Type":"ContainerDied","Data":"95fe4ec675f1eef64da36705d4a890480cb0a106844bd69b52f083b1f0bd392d"} Dec 03 12:20:36 crc kubenswrapper[5052]: I1203 12:20:36.484630 5052 scope.go:117] "RemoveContainer" containerID="50d90ae2ffca4d0cc92a43c260b1dec75f6540c0f3b6a2f388e0315ce52b8fdd" Dec 03 12:20:36 crc kubenswrapper[5052]: I1203 12:20:36.484771 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:20:36 crc kubenswrapper[5052]: I1203 12:20:36.495716 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1ec6b1f7-5abb-4063-80fa-b5ec907dcb91-erlang-cookie-secret" (OuterVolumeSpecName: "erlang-cookie-secret") pod "1ec6b1f7-5abb-4063-80fa-b5ec907dcb91" (UID: "1ec6b1f7-5abb-4063-80fa-b5ec907dcb91"). InnerVolumeSpecName "erlang-cookie-secret". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:20:36 crc kubenswrapper[5052]: I1203 12:20:36.500569 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage15-crc" (OuterVolumeSpecName: "persistence") pod "1ec6b1f7-5abb-4063-80fa-b5ec907dcb91" (UID: "1ec6b1f7-5abb-4063-80fa-b5ec907dcb91"). InnerVolumeSpecName "local-storage15-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 12:20:36 crc kubenswrapper[5052]: I1203 12:20:36.520506 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1ec6b1f7-5abb-4063-80fa-b5ec907dcb91-rabbitmq-tls" (OuterVolumeSpecName: "rabbitmq-tls") pod "1ec6b1f7-5abb-4063-80fa-b5ec907dcb91" (UID: "1ec6b1f7-5abb-4063-80fa-b5ec907dcb91"). InnerVolumeSpecName "rabbitmq-tls". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:20:36 crc kubenswrapper[5052]: I1203 12:20:36.521719 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1ec6b1f7-5abb-4063-80fa-b5ec907dcb91-config-data" (OuterVolumeSpecName: "config-data") pod "1ec6b1f7-5abb-4063-80fa-b5ec907dcb91" (UID: "1ec6b1f7-5abb-4063-80fa-b5ec907dcb91"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:20:36 crc kubenswrapper[5052]: I1203 12:20:36.544431 5052 reconciler_common.go:293] "Volume detached for volume \"rabbitmq-tls\" (UniqueName: \"kubernetes.io/projected/1ec6b1f7-5abb-4063-80fa-b5ec907dcb91-rabbitmq-tls\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:36 crc kubenswrapper[5052]: I1203 12:20:36.544849 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/1ec6b1f7-5abb-4063-80fa-b5ec907dcb91-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:36 crc kubenswrapper[5052]: I1203 12:20:36.544935 5052 reconciler_common.go:293] "Volume detached for volume \"erlang-cookie-secret\" (UniqueName: \"kubernetes.io/secret/1ec6b1f7-5abb-4063-80fa-b5ec907dcb91-erlang-cookie-secret\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:36 crc kubenswrapper[5052]: I1203 12:20:36.545045 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage15-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage15-crc\") on node \"crc\" " Dec 03 12:20:36 crc kubenswrapper[5052]: I1203 12:20:36.545122 5052 reconciler_common.go:293] "Volume detached for volume \"pod-info\" (UniqueName: \"kubernetes.io/downward-api/1ec6b1f7-5abb-4063-80fa-b5ec907dcb91-pod-info\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:36 crc kubenswrapper[5052]: I1203 12:20:36.545219 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-9zzmt\" (UniqueName: \"kubernetes.io/projected/1ec6b1f7-5abb-4063-80fa-b5ec907dcb91-kube-api-access-9zzmt\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:36 crc kubenswrapper[5052]: I1203 12:20:36.545304 5052 reconciler_common.go:293] "Volume detached for volume \"rabbitmq-erlang-cookie\" (UniqueName: \"kubernetes.io/empty-dir/1ec6b1f7-5abb-4063-80fa-b5ec907dcb91-rabbitmq-erlang-cookie\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:36 crc kubenswrapper[5052]: I1203 12:20:36.545382 5052 reconciler_common.go:293] "Volume detached for volume \"rabbitmq-plugins\" (UniqueName: \"kubernetes.io/empty-dir/1ec6b1f7-5abb-4063-80fa-b5ec907dcb91-rabbitmq-plugins\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:36 crc kubenswrapper[5052]: I1203 12:20:36.545469 5052 reconciler_common.go:293] "Volume detached for volume \"plugins-conf\" (UniqueName: \"kubernetes.io/configmap/1ec6b1f7-5abb-4063-80fa-b5ec907dcb91-plugins-conf\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:36 crc kubenswrapper[5052]: I1203 12:20:36.548603 5052 generic.go:334] "Generic (PLEG): container finished" podID="683b7d5f-a88a-4dc0-ae83-af3c6444c7ff" containerID="82c91d3bad330d4f21f39bfba31fbc4e7bfd4508da6e46af97efbf075afd08f3" exitCode=0 Dec 03 12:20:36 crc kubenswrapper[5052]: I1203 12:20:36.549696 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-server-0" event={"ID":"683b7d5f-a88a-4dc0-ae83-af3c6444c7ff","Type":"ContainerDied","Data":"82c91d3bad330d4f21f39bfba31fbc4e7bfd4508da6e46af97efbf075afd08f3"} Dec 03 12:20:36 crc kubenswrapper[5052]: I1203 12:20:36.585073 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1ec6b1f7-5abb-4063-80fa-b5ec907dcb91-server-conf" (OuterVolumeSpecName: "server-conf") pod "1ec6b1f7-5abb-4063-80fa-b5ec907dcb91" (UID: "1ec6b1f7-5abb-4063-80fa-b5ec907dcb91"). InnerVolumeSpecName "server-conf". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:20:36 crc kubenswrapper[5052]: I1203 12:20:36.630692 5052 scope.go:117] "RemoveContainer" containerID="65247d77d1d600b3063d35a5f6d98e45d0e5800997643936febe03cc72d2cda6" Dec 03 12:20:36 crc kubenswrapper[5052]: I1203 12:20:36.636918 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage15-crc" (UniqueName: "kubernetes.io/local-volume/local-storage15-crc") on node "crc" Dec 03 12:20:36 crc kubenswrapper[5052]: I1203 12:20:36.649339 5052 reconciler_common.go:293] "Volume detached for volume \"server-conf\" (UniqueName: \"kubernetes.io/configmap/1ec6b1f7-5abb-4063-80fa-b5ec907dcb91-server-conf\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:36 crc kubenswrapper[5052]: I1203 12:20:36.649360 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage15-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage15-crc\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:36 crc kubenswrapper[5052]: I1203 12:20:36.678741 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1ec6b1f7-5abb-4063-80fa-b5ec907dcb91-rabbitmq-confd" (OuterVolumeSpecName: "rabbitmq-confd") pod "1ec6b1f7-5abb-4063-80fa-b5ec907dcb91" (UID: "1ec6b1f7-5abb-4063-80fa-b5ec907dcb91"). InnerVolumeSpecName "rabbitmq-confd". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:20:36 crc kubenswrapper[5052]: I1203 12:20:36.707710 5052 scope.go:117] "RemoveContainer" containerID="50d90ae2ffca4d0cc92a43c260b1dec75f6540c0f3b6a2f388e0315ce52b8fdd" Dec 03 12:20:36 crc kubenswrapper[5052]: E1203 12:20:36.714971 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"50d90ae2ffca4d0cc92a43c260b1dec75f6540c0f3b6a2f388e0315ce52b8fdd\": container with ID starting with 50d90ae2ffca4d0cc92a43c260b1dec75f6540c0f3b6a2f388e0315ce52b8fdd not found: ID does not exist" containerID="50d90ae2ffca4d0cc92a43c260b1dec75f6540c0f3b6a2f388e0315ce52b8fdd" Dec 03 12:20:36 crc kubenswrapper[5052]: I1203 12:20:36.715028 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"50d90ae2ffca4d0cc92a43c260b1dec75f6540c0f3b6a2f388e0315ce52b8fdd"} err="failed to get container status \"50d90ae2ffca4d0cc92a43c260b1dec75f6540c0f3b6a2f388e0315ce52b8fdd\": rpc error: code = NotFound desc = could not find container \"50d90ae2ffca4d0cc92a43c260b1dec75f6540c0f3b6a2f388e0315ce52b8fdd\": container with ID starting with 50d90ae2ffca4d0cc92a43c260b1dec75f6540c0f3b6a2f388e0315ce52b8fdd not found: ID does not exist" Dec 03 12:20:36 crc kubenswrapper[5052]: I1203 12:20:36.715060 5052 scope.go:117] "RemoveContainer" containerID="65247d77d1d600b3063d35a5f6d98e45d0e5800997643936febe03cc72d2cda6" Dec 03 12:20:36 crc kubenswrapper[5052]: E1203 12:20:36.716353 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"65247d77d1d600b3063d35a5f6d98e45d0e5800997643936febe03cc72d2cda6\": container with ID starting with 65247d77d1d600b3063d35a5f6d98e45d0e5800997643936febe03cc72d2cda6 not found: ID does not exist" containerID="65247d77d1d600b3063d35a5f6d98e45d0e5800997643936febe03cc72d2cda6" Dec 03 12:20:36 crc kubenswrapper[5052]: I1203 12:20:36.716381 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"65247d77d1d600b3063d35a5f6d98e45d0e5800997643936febe03cc72d2cda6"} err="failed to get container status \"65247d77d1d600b3063d35a5f6d98e45d0e5800997643936febe03cc72d2cda6\": rpc error: code = NotFound desc = could not find container \"65247d77d1d600b3063d35a5f6d98e45d0e5800997643936febe03cc72d2cda6\": container with ID starting with 65247d77d1d600b3063d35a5f6d98e45d0e5800997643936febe03cc72d2cda6 not found: ID does not exist" Dec 03 12:20:36 crc kubenswrapper[5052]: I1203 12:20:36.758282 5052 reconciler_common.go:293] "Volume detached for volume \"rabbitmq-confd\" (UniqueName: \"kubernetes.io/projected/1ec6b1f7-5abb-4063-80fa-b5ec907dcb91-rabbitmq-confd\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:36 crc kubenswrapper[5052]: I1203 12:20:36.774874 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:20:36 crc kubenswrapper[5052]: I1203 12:20:36.860041 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"pod-info\" (UniqueName: \"kubernetes.io/downward-api/683b7d5f-a88a-4dc0-ae83-af3c6444c7ff-pod-info\") pod \"683b7d5f-a88a-4dc0-ae83-af3c6444c7ff\" (UID: \"683b7d5f-a88a-4dc0-ae83-af3c6444c7ff\") " Dec 03 12:20:36 crc kubenswrapper[5052]: I1203 12:20:36.860145 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"rabbitmq-erlang-cookie\" (UniqueName: \"kubernetes.io/empty-dir/683b7d5f-a88a-4dc0-ae83-af3c6444c7ff-rabbitmq-erlang-cookie\") pod \"683b7d5f-a88a-4dc0-ae83-af3c6444c7ff\" (UID: \"683b7d5f-a88a-4dc0-ae83-af3c6444c7ff\") " Dec 03 12:20:36 crc kubenswrapper[5052]: I1203 12:20:36.860198 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l6665\" (UniqueName: \"kubernetes.io/projected/683b7d5f-a88a-4dc0-ae83-af3c6444c7ff-kube-api-access-l6665\") pod \"683b7d5f-a88a-4dc0-ae83-af3c6444c7ff\" (UID: \"683b7d5f-a88a-4dc0-ae83-af3c6444c7ff\") " Dec 03 12:20:36 crc kubenswrapper[5052]: I1203 12:20:36.860250 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"plugins-conf\" (UniqueName: \"kubernetes.io/configmap/683b7d5f-a88a-4dc0-ae83-af3c6444c7ff-plugins-conf\") pod \"683b7d5f-a88a-4dc0-ae83-af3c6444c7ff\" (UID: \"683b7d5f-a88a-4dc0-ae83-af3c6444c7ff\") " Dec 03 12:20:36 crc kubenswrapper[5052]: I1203 12:20:36.860338 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"rabbitmq-tls\" (UniqueName: \"kubernetes.io/projected/683b7d5f-a88a-4dc0-ae83-af3c6444c7ff-rabbitmq-tls\") pod \"683b7d5f-a88a-4dc0-ae83-af3c6444c7ff\" (UID: \"683b7d5f-a88a-4dc0-ae83-af3c6444c7ff\") " Dec 03 12:20:36 crc kubenswrapper[5052]: I1203 12:20:36.860392 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"server-conf\" (UniqueName: \"kubernetes.io/configmap/683b7d5f-a88a-4dc0-ae83-af3c6444c7ff-server-conf\") pod \"683b7d5f-a88a-4dc0-ae83-af3c6444c7ff\" (UID: \"683b7d5f-a88a-4dc0-ae83-af3c6444c7ff\") " Dec 03 12:20:36 crc kubenswrapper[5052]: I1203 12:20:36.860433 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"rabbitmq-plugins\" (UniqueName: \"kubernetes.io/empty-dir/683b7d5f-a88a-4dc0-ae83-af3c6444c7ff-rabbitmq-plugins\") pod \"683b7d5f-a88a-4dc0-ae83-af3c6444c7ff\" (UID: \"683b7d5f-a88a-4dc0-ae83-af3c6444c7ff\") " Dec 03 12:20:36 crc kubenswrapper[5052]: I1203 12:20:36.860475 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"erlang-cookie-secret\" (UniqueName: \"kubernetes.io/secret/683b7d5f-a88a-4dc0-ae83-af3c6444c7ff-erlang-cookie-secret\") pod \"683b7d5f-a88a-4dc0-ae83-af3c6444c7ff\" (UID: \"683b7d5f-a88a-4dc0-ae83-af3c6444c7ff\") " Dec 03 12:20:36 crc kubenswrapper[5052]: I1203 12:20:36.860526 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"rabbitmq-confd\" (UniqueName: \"kubernetes.io/projected/683b7d5f-a88a-4dc0-ae83-af3c6444c7ff-rabbitmq-confd\") pod \"683b7d5f-a88a-4dc0-ae83-af3c6444c7ff\" (UID: \"683b7d5f-a88a-4dc0-ae83-af3c6444c7ff\") " Dec 03 12:20:36 crc kubenswrapper[5052]: I1203 12:20:36.860604 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"persistence\" (UniqueName: \"kubernetes.io/local-volume/local-storage19-crc\") pod \"683b7d5f-a88a-4dc0-ae83-af3c6444c7ff\" (UID: \"683b7d5f-a88a-4dc0-ae83-af3c6444c7ff\") " Dec 03 12:20:36 crc kubenswrapper[5052]: I1203 12:20:36.860636 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/683b7d5f-a88a-4dc0-ae83-af3c6444c7ff-config-data\") pod \"683b7d5f-a88a-4dc0-ae83-af3c6444c7ff\" (UID: \"683b7d5f-a88a-4dc0-ae83-af3c6444c7ff\") " Dec 03 12:20:36 crc kubenswrapper[5052]: I1203 12:20:36.867650 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/rabbitmq-cell1-server-0"] Dec 03 12:20:36 crc kubenswrapper[5052]: I1203 12:20:36.869354 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/683b7d5f-a88a-4dc0-ae83-af3c6444c7ff-rabbitmq-erlang-cookie" (OuterVolumeSpecName: "rabbitmq-erlang-cookie") pod "683b7d5f-a88a-4dc0-ae83-af3c6444c7ff" (UID: "683b7d5f-a88a-4dc0-ae83-af3c6444c7ff"). InnerVolumeSpecName "rabbitmq-erlang-cookie". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:20:36 crc kubenswrapper[5052]: I1203 12:20:36.875452 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/683b7d5f-a88a-4dc0-ae83-af3c6444c7ff-rabbitmq-plugins" (OuterVolumeSpecName: "rabbitmq-plugins") pod "683b7d5f-a88a-4dc0-ae83-af3c6444c7ff" (UID: "683b7d5f-a88a-4dc0-ae83-af3c6444c7ff"). InnerVolumeSpecName "rabbitmq-plugins". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:20:36 crc kubenswrapper[5052]: I1203 12:20:36.877005 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/683b7d5f-a88a-4dc0-ae83-af3c6444c7ff-plugins-conf" (OuterVolumeSpecName: "plugins-conf") pod "683b7d5f-a88a-4dc0-ae83-af3c6444c7ff" (UID: "683b7d5f-a88a-4dc0-ae83-af3c6444c7ff"). InnerVolumeSpecName "plugins-conf". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:20:36 crc kubenswrapper[5052]: I1203 12:20:36.885725 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage19-crc" (OuterVolumeSpecName: "persistence") pod "683b7d5f-a88a-4dc0-ae83-af3c6444c7ff" (UID: "683b7d5f-a88a-4dc0-ae83-af3c6444c7ff"). InnerVolumeSpecName "local-storage19-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 12:20:36 crc kubenswrapper[5052]: I1203 12:20:36.889053 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/683b7d5f-a88a-4dc0-ae83-af3c6444c7ff-erlang-cookie-secret" (OuterVolumeSpecName: "erlang-cookie-secret") pod "683b7d5f-a88a-4dc0-ae83-af3c6444c7ff" (UID: "683b7d5f-a88a-4dc0-ae83-af3c6444c7ff"). InnerVolumeSpecName "erlang-cookie-secret". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:20:36 crc kubenswrapper[5052]: I1203 12:20:36.889526 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/683b7d5f-a88a-4dc0-ae83-af3c6444c7ff-kube-api-access-l6665" (OuterVolumeSpecName: "kube-api-access-l6665") pod "683b7d5f-a88a-4dc0-ae83-af3c6444c7ff" (UID: "683b7d5f-a88a-4dc0-ae83-af3c6444c7ff"). InnerVolumeSpecName "kube-api-access-l6665". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:20:36 crc kubenswrapper[5052]: I1203 12:20:36.897373 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/683b7d5f-a88a-4dc0-ae83-af3c6444c7ff-pod-info" (OuterVolumeSpecName: "pod-info") pod "683b7d5f-a88a-4dc0-ae83-af3c6444c7ff" (UID: "683b7d5f-a88a-4dc0-ae83-af3c6444c7ff"). InnerVolumeSpecName "pod-info". PluginName "kubernetes.io/downward-api", VolumeGidValue "" Dec 03 12:20:36 crc kubenswrapper[5052]: I1203 12:20:36.909238 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/rabbitmq-cell1-server-0"] Dec 03 12:20:36 crc kubenswrapper[5052]: I1203 12:20:36.921713 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/683b7d5f-a88a-4dc0-ae83-af3c6444c7ff-rabbitmq-tls" (OuterVolumeSpecName: "rabbitmq-tls") pod "683b7d5f-a88a-4dc0-ae83-af3c6444c7ff" (UID: "683b7d5f-a88a-4dc0-ae83-af3c6444c7ff"). InnerVolumeSpecName "rabbitmq-tls". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:20:36 crc kubenswrapper[5052]: I1203 12:20:36.950698 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/683b7d5f-a88a-4dc0-ae83-af3c6444c7ff-config-data" (OuterVolumeSpecName: "config-data") pod "683b7d5f-a88a-4dc0-ae83-af3c6444c7ff" (UID: "683b7d5f-a88a-4dc0-ae83-af3c6444c7ff"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:20:36 crc kubenswrapper[5052]: I1203 12:20:36.965269 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage19-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage19-crc\") on node \"crc\" " Dec 03 12:20:36 crc kubenswrapper[5052]: I1203 12:20:36.965312 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/683b7d5f-a88a-4dc0-ae83-af3c6444c7ff-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:36 crc kubenswrapper[5052]: I1203 12:20:36.965323 5052 reconciler_common.go:293] "Volume detached for volume \"pod-info\" (UniqueName: \"kubernetes.io/downward-api/683b7d5f-a88a-4dc0-ae83-af3c6444c7ff-pod-info\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:36 crc kubenswrapper[5052]: I1203 12:20:36.965334 5052 reconciler_common.go:293] "Volume detached for volume \"rabbitmq-erlang-cookie\" (UniqueName: \"kubernetes.io/empty-dir/683b7d5f-a88a-4dc0-ae83-af3c6444c7ff-rabbitmq-erlang-cookie\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:36 crc kubenswrapper[5052]: I1203 12:20:36.965347 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-l6665\" (UniqueName: \"kubernetes.io/projected/683b7d5f-a88a-4dc0-ae83-af3c6444c7ff-kube-api-access-l6665\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:36 crc kubenswrapper[5052]: I1203 12:20:36.965358 5052 reconciler_common.go:293] "Volume detached for volume \"plugins-conf\" (UniqueName: \"kubernetes.io/configmap/683b7d5f-a88a-4dc0-ae83-af3c6444c7ff-plugins-conf\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:36 crc kubenswrapper[5052]: I1203 12:20:36.965368 5052 reconciler_common.go:293] "Volume detached for volume \"rabbitmq-tls\" (UniqueName: \"kubernetes.io/projected/683b7d5f-a88a-4dc0-ae83-af3c6444c7ff-rabbitmq-tls\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:36 crc kubenswrapper[5052]: I1203 12:20:36.965382 5052 reconciler_common.go:293] "Volume detached for volume \"rabbitmq-plugins\" (UniqueName: \"kubernetes.io/empty-dir/683b7d5f-a88a-4dc0-ae83-af3c6444c7ff-rabbitmq-plugins\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:36 crc kubenswrapper[5052]: I1203 12:20:36.965392 5052 reconciler_common.go:293] "Volume detached for volume \"erlang-cookie-secret\" (UniqueName: \"kubernetes.io/secret/683b7d5f-a88a-4dc0-ae83-af3c6444c7ff-erlang-cookie-secret\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:36 crc kubenswrapper[5052]: I1203 12:20:36.984673 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/rabbitmq-cell1-server-0"] Dec 03 12:20:36 crc kubenswrapper[5052]: E1203 12:20:36.985221 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="1ec6b1f7-5abb-4063-80fa-b5ec907dcb91" containerName="setup-container" Dec 03 12:20:36 crc kubenswrapper[5052]: I1203 12:20:36.985242 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="1ec6b1f7-5abb-4063-80fa-b5ec907dcb91" containerName="setup-container" Dec 03 12:20:36 crc kubenswrapper[5052]: E1203 12:20:36.985254 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="683b7d5f-a88a-4dc0-ae83-af3c6444c7ff" containerName="rabbitmq" Dec 03 12:20:36 crc kubenswrapper[5052]: I1203 12:20:36.985263 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="683b7d5f-a88a-4dc0-ae83-af3c6444c7ff" containerName="rabbitmq" Dec 03 12:20:36 crc kubenswrapper[5052]: E1203 12:20:36.985281 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="1ec6b1f7-5abb-4063-80fa-b5ec907dcb91" containerName="rabbitmq" Dec 03 12:20:36 crc kubenswrapper[5052]: I1203 12:20:36.985288 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="1ec6b1f7-5abb-4063-80fa-b5ec907dcb91" containerName="rabbitmq" Dec 03 12:20:36 crc kubenswrapper[5052]: E1203 12:20:36.985330 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="683b7d5f-a88a-4dc0-ae83-af3c6444c7ff" containerName="setup-container" Dec 03 12:20:36 crc kubenswrapper[5052]: I1203 12:20:36.985337 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="683b7d5f-a88a-4dc0-ae83-af3c6444c7ff" containerName="setup-container" Dec 03 12:20:36 crc kubenswrapper[5052]: I1203 12:20:36.985610 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="1ec6b1f7-5abb-4063-80fa-b5ec907dcb91" containerName="rabbitmq" Dec 03 12:20:36 crc kubenswrapper[5052]: I1203 12:20:36.985632 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="683b7d5f-a88a-4dc0-ae83-af3c6444c7ff" containerName="rabbitmq" Dec 03 12:20:36 crc kubenswrapper[5052]: I1203 12:20:36.986973 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:20:36 crc kubenswrapper[5052]: I1203 12:20:36.997025 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"rabbitmq-cell1-server-conf" Dec 03 12:20:36 crc kubenswrapper[5052]: I1203 12:20:36.997203 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-rabbitmq-cell1-svc" Dec 03 12:20:36 crc kubenswrapper[5052]: I1203 12:20:36.997368 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"rabbitmq-cell1-config-data" Dec 03 12:20:36 crc kubenswrapper[5052]: I1203 12:20:36.997492 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"rabbitmq-cell1-default-user" Dec 03 12:20:36 crc kubenswrapper[5052]: I1203 12:20:36.997626 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"rabbitmq-cell1-server-dockercfg-bwtz2" Dec 03 12:20:36 crc kubenswrapper[5052]: I1203 12:20:36.997738 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"rabbitmq-cell1-erlang-cookie" Dec 03 12:20:36 crc kubenswrapper[5052]: I1203 12:20:36.997842 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"rabbitmq-cell1-plugins-conf" Dec 03 12:20:37 crc kubenswrapper[5052]: I1203 12:20:37.000411 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/rabbitmq-cell1-server-0"] Dec 03 12:20:37 crc kubenswrapper[5052]: I1203 12:20:37.012806 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage19-crc" (UniqueName: "kubernetes.io/local-volume/local-storage19-crc") on node "crc" Dec 03 12:20:37 crc kubenswrapper[5052]: I1203 12:20:37.013778 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:20:37 crc kubenswrapper[5052]: I1203 12:20:37.062421 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/683b7d5f-a88a-4dc0-ae83-af3c6444c7ff-server-conf" (OuterVolumeSpecName: "server-conf") pod "683b7d5f-a88a-4dc0-ae83-af3c6444c7ff" (UID: "683b7d5f-a88a-4dc0-ae83-af3c6444c7ff"). InnerVolumeSpecName "server-conf". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:20:37 crc kubenswrapper[5052]: I1203 12:20:37.067059 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"rabbitmq-confd\" (UniqueName: \"kubernetes.io/projected/c279eaf1-2ee2-46fe-88be-9a5a65d4897e-rabbitmq-confd\") pod \"rabbitmq-cell1-server-0\" (UID: \"c279eaf1-2ee2-46fe-88be-9a5a65d4897e\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:20:37 crc kubenswrapper[5052]: I1203 12:20:37.067107 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"server-conf\" (UniqueName: \"kubernetes.io/configmap/c279eaf1-2ee2-46fe-88be-9a5a65d4897e-server-conf\") pod \"rabbitmq-cell1-server-0\" (UID: \"c279eaf1-2ee2-46fe-88be-9a5a65d4897e\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:20:37 crc kubenswrapper[5052]: I1203 12:20:37.067134 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"rabbitmq-plugins\" (UniqueName: \"kubernetes.io/empty-dir/c279eaf1-2ee2-46fe-88be-9a5a65d4897e-rabbitmq-plugins\") pod \"rabbitmq-cell1-server-0\" (UID: \"c279eaf1-2ee2-46fe-88be-9a5a65d4897e\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:20:37 crc kubenswrapper[5052]: I1203 12:20:37.067214 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q2n96\" (UniqueName: \"kubernetes.io/projected/c279eaf1-2ee2-46fe-88be-9a5a65d4897e-kube-api-access-q2n96\") pod \"rabbitmq-cell1-server-0\" (UID: \"c279eaf1-2ee2-46fe-88be-9a5a65d4897e\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:20:37 crc kubenswrapper[5052]: I1203 12:20:37.067251 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"rabbitmq-erlang-cookie\" (UniqueName: \"kubernetes.io/empty-dir/c279eaf1-2ee2-46fe-88be-9a5a65d4897e-rabbitmq-erlang-cookie\") pod \"rabbitmq-cell1-server-0\" (UID: \"c279eaf1-2ee2-46fe-88be-9a5a65d4897e\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:20:37 crc kubenswrapper[5052]: I1203 12:20:37.067310 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"erlang-cookie-secret\" (UniqueName: \"kubernetes.io/secret/c279eaf1-2ee2-46fe-88be-9a5a65d4897e-erlang-cookie-secret\") pod \"rabbitmq-cell1-server-0\" (UID: \"c279eaf1-2ee2-46fe-88be-9a5a65d4897e\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:20:37 crc kubenswrapper[5052]: I1203 12:20:37.067377 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/c279eaf1-2ee2-46fe-88be-9a5a65d4897e-config-data\") pod \"rabbitmq-cell1-server-0\" (UID: \"c279eaf1-2ee2-46fe-88be-9a5a65d4897e\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:20:37 crc kubenswrapper[5052]: I1203 12:20:37.067403 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-conf\" (UniqueName: \"kubernetes.io/configmap/c279eaf1-2ee2-46fe-88be-9a5a65d4897e-plugins-conf\") pod \"rabbitmq-cell1-server-0\" (UID: \"c279eaf1-2ee2-46fe-88be-9a5a65d4897e\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:20:37 crc kubenswrapper[5052]: I1203 12:20:37.067452 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"pod-info\" (UniqueName: \"kubernetes.io/downward-api/c279eaf1-2ee2-46fe-88be-9a5a65d4897e-pod-info\") pod \"rabbitmq-cell1-server-0\" (UID: \"c279eaf1-2ee2-46fe-88be-9a5a65d4897e\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:20:37 crc kubenswrapper[5052]: I1203 12:20:37.067546 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage15-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage15-crc\") pod \"rabbitmq-cell1-server-0\" (UID: \"c279eaf1-2ee2-46fe-88be-9a5a65d4897e\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:20:37 crc kubenswrapper[5052]: I1203 12:20:37.067648 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"rabbitmq-tls\" (UniqueName: \"kubernetes.io/projected/c279eaf1-2ee2-46fe-88be-9a5a65d4897e-rabbitmq-tls\") pod \"rabbitmq-cell1-server-0\" (UID: \"c279eaf1-2ee2-46fe-88be-9a5a65d4897e\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:20:37 crc kubenswrapper[5052]: I1203 12:20:37.067927 5052 reconciler_common.go:293] "Volume detached for volume \"server-conf\" (UniqueName: \"kubernetes.io/configmap/683b7d5f-a88a-4dc0-ae83-af3c6444c7ff-server-conf\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:37 crc kubenswrapper[5052]: I1203 12:20:37.067947 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage19-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage19-crc\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:37 crc kubenswrapper[5052]: I1203 12:20:37.108712 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/683b7d5f-a88a-4dc0-ae83-af3c6444c7ff-rabbitmq-confd" (OuterVolumeSpecName: "rabbitmq-confd") pod "683b7d5f-a88a-4dc0-ae83-af3c6444c7ff" (UID: "683b7d5f-a88a-4dc0-ae83-af3c6444c7ff"). InnerVolumeSpecName "rabbitmq-confd". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:20:37 crc kubenswrapper[5052]: I1203 12:20:37.169193 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"plugins-conf\" (UniqueName: \"kubernetes.io/configmap/c279eaf1-2ee2-46fe-88be-9a5a65d4897e-plugins-conf\") pod \"rabbitmq-cell1-server-0\" (UID: \"c279eaf1-2ee2-46fe-88be-9a5a65d4897e\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:20:37 crc kubenswrapper[5052]: I1203 12:20:37.169317 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"pod-info\" (UniqueName: \"kubernetes.io/downward-api/c279eaf1-2ee2-46fe-88be-9a5a65d4897e-pod-info\") pod \"rabbitmq-cell1-server-0\" (UID: \"c279eaf1-2ee2-46fe-88be-9a5a65d4897e\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:20:37 crc kubenswrapper[5052]: I1203 12:20:37.169351 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage15-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage15-crc\") pod \"rabbitmq-cell1-server-0\" (UID: \"c279eaf1-2ee2-46fe-88be-9a5a65d4897e\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:20:37 crc kubenswrapper[5052]: I1203 12:20:37.169386 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"rabbitmq-tls\" (UniqueName: \"kubernetes.io/projected/c279eaf1-2ee2-46fe-88be-9a5a65d4897e-rabbitmq-tls\") pod \"rabbitmq-cell1-server-0\" (UID: \"c279eaf1-2ee2-46fe-88be-9a5a65d4897e\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:20:37 crc kubenswrapper[5052]: I1203 12:20:37.169411 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"rabbitmq-confd\" (UniqueName: \"kubernetes.io/projected/c279eaf1-2ee2-46fe-88be-9a5a65d4897e-rabbitmq-confd\") pod \"rabbitmq-cell1-server-0\" (UID: \"c279eaf1-2ee2-46fe-88be-9a5a65d4897e\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:20:37 crc kubenswrapper[5052]: I1203 12:20:37.169426 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"server-conf\" (UniqueName: \"kubernetes.io/configmap/c279eaf1-2ee2-46fe-88be-9a5a65d4897e-server-conf\") pod \"rabbitmq-cell1-server-0\" (UID: \"c279eaf1-2ee2-46fe-88be-9a5a65d4897e\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:20:37 crc kubenswrapper[5052]: I1203 12:20:37.169450 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"rabbitmq-plugins\" (UniqueName: \"kubernetes.io/empty-dir/c279eaf1-2ee2-46fe-88be-9a5a65d4897e-rabbitmq-plugins\") pod \"rabbitmq-cell1-server-0\" (UID: \"c279eaf1-2ee2-46fe-88be-9a5a65d4897e\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:20:37 crc kubenswrapper[5052]: I1203 12:20:37.169551 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-q2n96\" (UniqueName: \"kubernetes.io/projected/c279eaf1-2ee2-46fe-88be-9a5a65d4897e-kube-api-access-q2n96\") pod \"rabbitmq-cell1-server-0\" (UID: \"c279eaf1-2ee2-46fe-88be-9a5a65d4897e\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:20:37 crc kubenswrapper[5052]: I1203 12:20:37.169593 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"rabbitmq-erlang-cookie\" (UniqueName: \"kubernetes.io/empty-dir/c279eaf1-2ee2-46fe-88be-9a5a65d4897e-rabbitmq-erlang-cookie\") pod \"rabbitmq-cell1-server-0\" (UID: \"c279eaf1-2ee2-46fe-88be-9a5a65d4897e\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:20:37 crc kubenswrapper[5052]: I1203 12:20:37.169641 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"erlang-cookie-secret\" (UniqueName: \"kubernetes.io/secret/c279eaf1-2ee2-46fe-88be-9a5a65d4897e-erlang-cookie-secret\") pod \"rabbitmq-cell1-server-0\" (UID: \"c279eaf1-2ee2-46fe-88be-9a5a65d4897e\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:20:37 crc kubenswrapper[5052]: I1203 12:20:37.169680 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/c279eaf1-2ee2-46fe-88be-9a5a65d4897e-config-data\") pod \"rabbitmq-cell1-server-0\" (UID: \"c279eaf1-2ee2-46fe-88be-9a5a65d4897e\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:20:37 crc kubenswrapper[5052]: I1203 12:20:37.169741 5052 reconciler_common.go:293] "Volume detached for volume \"rabbitmq-confd\" (UniqueName: \"kubernetes.io/projected/683b7d5f-a88a-4dc0-ae83-af3c6444c7ff-rabbitmq-confd\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:37 crc kubenswrapper[5052]: I1203 12:20:37.170248 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"plugins-conf\" (UniqueName: \"kubernetes.io/configmap/c279eaf1-2ee2-46fe-88be-9a5a65d4897e-plugins-conf\") pod \"rabbitmq-cell1-server-0\" (UID: \"c279eaf1-2ee2-46fe-88be-9a5a65d4897e\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:20:37 crc kubenswrapper[5052]: I1203 12:20:37.170427 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/c279eaf1-2ee2-46fe-88be-9a5a65d4897e-config-data\") pod \"rabbitmq-cell1-server-0\" (UID: \"c279eaf1-2ee2-46fe-88be-9a5a65d4897e\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:20:37 crc kubenswrapper[5052]: I1203 12:20:37.171573 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"rabbitmq-plugins\" (UniqueName: \"kubernetes.io/empty-dir/c279eaf1-2ee2-46fe-88be-9a5a65d4897e-rabbitmq-plugins\") pod \"rabbitmq-cell1-server-0\" (UID: \"c279eaf1-2ee2-46fe-88be-9a5a65d4897e\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:20:37 crc kubenswrapper[5052]: I1203 12:20:37.172129 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"rabbitmq-erlang-cookie\" (UniqueName: \"kubernetes.io/empty-dir/c279eaf1-2ee2-46fe-88be-9a5a65d4897e-rabbitmq-erlang-cookie\") pod \"rabbitmq-cell1-server-0\" (UID: \"c279eaf1-2ee2-46fe-88be-9a5a65d4897e\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:20:37 crc kubenswrapper[5052]: I1203 12:20:37.175068 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"rabbitmq-tls\" (UniqueName: \"kubernetes.io/projected/c279eaf1-2ee2-46fe-88be-9a5a65d4897e-rabbitmq-tls\") pod \"rabbitmq-cell1-server-0\" (UID: \"c279eaf1-2ee2-46fe-88be-9a5a65d4897e\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:20:37 crc kubenswrapper[5052]: I1203 12:20:37.175913 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage15-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage15-crc\") pod \"rabbitmq-cell1-server-0\" (UID: \"c279eaf1-2ee2-46fe-88be-9a5a65d4897e\") device mount path \"/mnt/openstack/pv15\"" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:20:37 crc kubenswrapper[5052]: I1203 12:20:37.178125 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"erlang-cookie-secret\" (UniqueName: \"kubernetes.io/secret/c279eaf1-2ee2-46fe-88be-9a5a65d4897e-erlang-cookie-secret\") pod \"rabbitmq-cell1-server-0\" (UID: \"c279eaf1-2ee2-46fe-88be-9a5a65d4897e\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:20:37 crc kubenswrapper[5052]: I1203 12:20:37.178249 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"rabbitmq-confd\" (UniqueName: \"kubernetes.io/projected/c279eaf1-2ee2-46fe-88be-9a5a65d4897e-rabbitmq-confd\") pod \"rabbitmq-cell1-server-0\" (UID: \"c279eaf1-2ee2-46fe-88be-9a5a65d4897e\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:20:37 crc kubenswrapper[5052]: I1203 12:20:37.180629 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"server-conf\" (UniqueName: \"kubernetes.io/configmap/c279eaf1-2ee2-46fe-88be-9a5a65d4897e-server-conf\") pod \"rabbitmq-cell1-server-0\" (UID: \"c279eaf1-2ee2-46fe-88be-9a5a65d4897e\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:20:37 crc kubenswrapper[5052]: I1203 12:20:37.181136 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"pod-info\" (UniqueName: \"kubernetes.io/downward-api/c279eaf1-2ee2-46fe-88be-9a5a65d4897e-pod-info\") pod \"rabbitmq-cell1-server-0\" (UID: \"c279eaf1-2ee2-46fe-88be-9a5a65d4897e\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:20:37 crc kubenswrapper[5052]: I1203 12:20:37.189181 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-q2n96\" (UniqueName: \"kubernetes.io/projected/c279eaf1-2ee2-46fe-88be-9a5a65d4897e-kube-api-access-q2n96\") pod \"rabbitmq-cell1-server-0\" (UID: \"c279eaf1-2ee2-46fe-88be-9a5a65d4897e\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:20:37 crc kubenswrapper[5052]: I1203 12:20:37.203840 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage15-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage15-crc\") pod \"rabbitmq-cell1-server-0\" (UID: \"c279eaf1-2ee2-46fe-88be-9a5a65d4897e\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:20:37 crc kubenswrapper[5052]: I1203 12:20:37.335142 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:20:37 crc kubenswrapper[5052]: I1203 12:20:37.494702 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:20:37 crc kubenswrapper[5052]: I1203 12:20:37.562693 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"f50a3c6a-8630-4ca5-99cd-4cf76e55fef8","Type":"ContainerStarted","Data":"1233ff3b35b92293a94cb57661d5131e67c6e740e238a5179328cce848e33741"} Dec 03 12:20:37 crc kubenswrapper[5052]: I1203 12:20:37.566822 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:20:37 crc kubenswrapper[5052]: I1203 12:20:37.569531 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-server-0" event={"ID":"683b7d5f-a88a-4dc0-ae83-af3c6444c7ff","Type":"ContainerDied","Data":"4c300067c8cb7310d5b5d8227109abc2a348ce501f13459cfca2ab313a1ece96"} Dec 03 12:20:37 crc kubenswrapper[5052]: I1203 12:20:37.569601 5052 scope.go:117] "RemoveContainer" containerID="82c91d3bad330d4f21f39bfba31fbc4e7bfd4508da6e46af97efbf075afd08f3" Dec 03 12:20:37 crc kubenswrapper[5052]: I1203 12:20:37.612038 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/rabbitmq-server-0"] Dec 03 12:20:37 crc kubenswrapper[5052]: I1203 12:20:37.620318 5052 scope.go:117] "RemoveContainer" containerID="67a78b1201a5ad7507c89c9217d67878a2b07cc939213e22cfeb1612eb0f4575" Dec 03 12:20:37 crc kubenswrapper[5052]: I1203 12:20:37.620820 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/rabbitmq-server-0"] Dec 03 12:20:37 crc kubenswrapper[5052]: I1203 12:20:37.656230 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/rabbitmq-server-0"] Dec 03 12:20:37 crc kubenswrapper[5052]: I1203 12:20:37.657997 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:20:37 crc kubenswrapper[5052]: I1203 12:20:37.660544 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:20:37 crc kubenswrapper[5052]: I1203 12:20:37.669395 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"rabbitmq-erlang-cookie" Dec 03 12:20:37 crc kubenswrapper[5052]: I1203 12:20:37.669786 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"rabbitmq-default-user" Dec 03 12:20:37 crc kubenswrapper[5052]: I1203 12:20:37.669981 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"rabbitmq-plugins-conf" Dec 03 12:20:37 crc kubenswrapper[5052]: I1203 12:20:37.670179 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"rabbitmq-server-dockercfg-stb9x" Dec 03 12:20:37 crc kubenswrapper[5052]: I1203 12:20:37.670409 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"rabbitmq-server-conf" Dec 03 12:20:37 crc kubenswrapper[5052]: I1203 12:20:37.670491 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-rabbitmq-svc" Dec 03 12:20:37 crc kubenswrapper[5052]: I1203 12:20:37.670901 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"rabbitmq-config-data" Dec 03 12:20:37 crc kubenswrapper[5052]: I1203 12:20:37.705484 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1ec6b1f7-5abb-4063-80fa-b5ec907dcb91" path="/var/lib/kubelet/pods/1ec6b1f7-5abb-4063-80fa-b5ec907dcb91/volumes" Dec 03 12:20:37 crc kubenswrapper[5052]: I1203 12:20:37.710908 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="683b7d5f-a88a-4dc0-ae83-af3c6444c7ff" path="/var/lib/kubelet/pods/683b7d5f-a88a-4dc0-ae83-af3c6444c7ff/volumes" Dec 03 12:20:37 crc kubenswrapper[5052]: I1203 12:20:37.712076 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ae23fe34-7bf4-47fc-addc-5edda0399c77" path="/var/lib/kubelet/pods/ae23fe34-7bf4-47fc-addc-5edda0399c77/volumes" Dec 03 12:20:37 crc kubenswrapper[5052]: I1203 12:20:37.739804 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/rabbitmq-server-0"] Dec 03 12:20:37 crc kubenswrapper[5052]: I1203 12:20:37.805238 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-conf\" (UniqueName: \"kubernetes.io/configmap/c14a906c-f19d-4806-9110-8c47b38c5ae0-plugins-conf\") pod \"rabbitmq-server-0\" (UID: \"c14a906c-f19d-4806-9110-8c47b38c5ae0\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:20:37 crc kubenswrapper[5052]: I1203 12:20:37.805303 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qljvf\" (UniqueName: \"kubernetes.io/projected/c14a906c-f19d-4806-9110-8c47b38c5ae0-kube-api-access-qljvf\") pod \"rabbitmq-server-0\" (UID: \"c14a906c-f19d-4806-9110-8c47b38c5ae0\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:20:37 crc kubenswrapper[5052]: I1203 12:20:37.805336 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/c14a906c-f19d-4806-9110-8c47b38c5ae0-config-data\") pod \"rabbitmq-server-0\" (UID: \"c14a906c-f19d-4806-9110-8c47b38c5ae0\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:20:37 crc kubenswrapper[5052]: I1203 12:20:37.805360 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"rabbitmq-plugins\" (UniqueName: \"kubernetes.io/empty-dir/c14a906c-f19d-4806-9110-8c47b38c5ae0-rabbitmq-plugins\") pod \"rabbitmq-server-0\" (UID: \"c14a906c-f19d-4806-9110-8c47b38c5ae0\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:20:37 crc kubenswrapper[5052]: I1203 12:20:37.805387 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"erlang-cookie-secret\" (UniqueName: \"kubernetes.io/secret/c14a906c-f19d-4806-9110-8c47b38c5ae0-erlang-cookie-secret\") pod \"rabbitmq-server-0\" (UID: \"c14a906c-f19d-4806-9110-8c47b38c5ae0\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:20:37 crc kubenswrapper[5052]: I1203 12:20:37.805566 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"rabbitmq-tls\" (UniqueName: \"kubernetes.io/projected/c14a906c-f19d-4806-9110-8c47b38c5ae0-rabbitmq-tls\") pod \"rabbitmq-server-0\" (UID: \"c14a906c-f19d-4806-9110-8c47b38c5ae0\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:20:37 crc kubenswrapper[5052]: I1203 12:20:37.805638 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"server-conf\" (UniqueName: \"kubernetes.io/configmap/c14a906c-f19d-4806-9110-8c47b38c5ae0-server-conf\") pod \"rabbitmq-server-0\" (UID: \"c14a906c-f19d-4806-9110-8c47b38c5ae0\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:20:37 crc kubenswrapper[5052]: I1203 12:20:37.805687 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage19-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage19-crc\") pod \"rabbitmq-server-0\" (UID: \"c14a906c-f19d-4806-9110-8c47b38c5ae0\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:20:37 crc kubenswrapper[5052]: I1203 12:20:37.805723 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"rabbitmq-confd\" (UniqueName: \"kubernetes.io/projected/c14a906c-f19d-4806-9110-8c47b38c5ae0-rabbitmq-confd\") pod \"rabbitmq-server-0\" (UID: \"c14a906c-f19d-4806-9110-8c47b38c5ae0\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:20:37 crc kubenswrapper[5052]: I1203 12:20:37.805744 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"rabbitmq-erlang-cookie\" (UniqueName: \"kubernetes.io/empty-dir/c14a906c-f19d-4806-9110-8c47b38c5ae0-rabbitmq-erlang-cookie\") pod \"rabbitmq-server-0\" (UID: \"c14a906c-f19d-4806-9110-8c47b38c5ae0\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:20:37 crc kubenswrapper[5052]: I1203 12:20:37.805764 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"pod-info\" (UniqueName: \"kubernetes.io/downward-api/c14a906c-f19d-4806-9110-8c47b38c5ae0-pod-info\") pod \"rabbitmq-server-0\" (UID: \"c14a906c-f19d-4806-9110-8c47b38c5ae0\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:20:37 crc kubenswrapper[5052]: I1203 12:20:37.810414 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/rabbitmq-cell1-server-0"] Dec 03 12:20:37 crc kubenswrapper[5052]: I1203 12:20:37.907308 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"rabbitmq-tls\" (UniqueName: \"kubernetes.io/projected/c14a906c-f19d-4806-9110-8c47b38c5ae0-rabbitmq-tls\") pod \"rabbitmq-server-0\" (UID: \"c14a906c-f19d-4806-9110-8c47b38c5ae0\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:20:37 crc kubenswrapper[5052]: I1203 12:20:37.907693 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"server-conf\" (UniqueName: \"kubernetes.io/configmap/c14a906c-f19d-4806-9110-8c47b38c5ae0-server-conf\") pod \"rabbitmq-server-0\" (UID: \"c14a906c-f19d-4806-9110-8c47b38c5ae0\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:20:37 crc kubenswrapper[5052]: I1203 12:20:37.907734 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage19-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage19-crc\") pod \"rabbitmq-server-0\" (UID: \"c14a906c-f19d-4806-9110-8c47b38c5ae0\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:20:37 crc kubenswrapper[5052]: I1203 12:20:37.907765 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"rabbitmq-confd\" (UniqueName: \"kubernetes.io/projected/c14a906c-f19d-4806-9110-8c47b38c5ae0-rabbitmq-confd\") pod \"rabbitmq-server-0\" (UID: \"c14a906c-f19d-4806-9110-8c47b38c5ae0\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:20:37 crc kubenswrapper[5052]: I1203 12:20:37.907789 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"rabbitmq-erlang-cookie\" (UniqueName: \"kubernetes.io/empty-dir/c14a906c-f19d-4806-9110-8c47b38c5ae0-rabbitmq-erlang-cookie\") pod \"rabbitmq-server-0\" (UID: \"c14a906c-f19d-4806-9110-8c47b38c5ae0\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:20:37 crc kubenswrapper[5052]: I1203 12:20:37.907813 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"pod-info\" (UniqueName: \"kubernetes.io/downward-api/c14a906c-f19d-4806-9110-8c47b38c5ae0-pod-info\") pod \"rabbitmq-server-0\" (UID: \"c14a906c-f19d-4806-9110-8c47b38c5ae0\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:20:37 crc kubenswrapper[5052]: I1203 12:20:37.907855 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"plugins-conf\" (UniqueName: \"kubernetes.io/configmap/c14a906c-f19d-4806-9110-8c47b38c5ae0-plugins-conf\") pod \"rabbitmq-server-0\" (UID: \"c14a906c-f19d-4806-9110-8c47b38c5ae0\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:20:37 crc kubenswrapper[5052]: I1203 12:20:37.907892 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-qljvf\" (UniqueName: \"kubernetes.io/projected/c14a906c-f19d-4806-9110-8c47b38c5ae0-kube-api-access-qljvf\") pod \"rabbitmq-server-0\" (UID: \"c14a906c-f19d-4806-9110-8c47b38c5ae0\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:20:37 crc kubenswrapper[5052]: I1203 12:20:37.907918 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/c14a906c-f19d-4806-9110-8c47b38c5ae0-config-data\") pod \"rabbitmq-server-0\" (UID: \"c14a906c-f19d-4806-9110-8c47b38c5ae0\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:20:37 crc kubenswrapper[5052]: I1203 12:20:37.907956 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"rabbitmq-plugins\" (UniqueName: \"kubernetes.io/empty-dir/c14a906c-f19d-4806-9110-8c47b38c5ae0-rabbitmq-plugins\") pod \"rabbitmq-server-0\" (UID: \"c14a906c-f19d-4806-9110-8c47b38c5ae0\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:20:37 crc kubenswrapper[5052]: I1203 12:20:37.907983 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"erlang-cookie-secret\" (UniqueName: \"kubernetes.io/secret/c14a906c-f19d-4806-9110-8c47b38c5ae0-erlang-cookie-secret\") pod \"rabbitmq-server-0\" (UID: \"c14a906c-f19d-4806-9110-8c47b38c5ae0\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:20:37 crc kubenswrapper[5052]: I1203 12:20:37.908405 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"rabbitmq-erlang-cookie\" (UniqueName: \"kubernetes.io/empty-dir/c14a906c-f19d-4806-9110-8c47b38c5ae0-rabbitmq-erlang-cookie\") pod \"rabbitmq-server-0\" (UID: \"c14a906c-f19d-4806-9110-8c47b38c5ae0\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:20:37 crc kubenswrapper[5052]: I1203 12:20:37.908793 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage19-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage19-crc\") pod \"rabbitmq-server-0\" (UID: \"c14a906c-f19d-4806-9110-8c47b38c5ae0\") device mount path \"/mnt/openstack/pv19\"" pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:20:37 crc kubenswrapper[5052]: I1203 12:20:37.908822 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"rabbitmq-plugins\" (UniqueName: \"kubernetes.io/empty-dir/c14a906c-f19d-4806-9110-8c47b38c5ae0-rabbitmq-plugins\") pod \"rabbitmq-server-0\" (UID: \"c14a906c-f19d-4806-9110-8c47b38c5ae0\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:20:37 crc kubenswrapper[5052]: I1203 12:20:37.909277 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/c14a906c-f19d-4806-9110-8c47b38c5ae0-config-data\") pod \"rabbitmq-server-0\" (UID: \"c14a906c-f19d-4806-9110-8c47b38c5ae0\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:20:37 crc kubenswrapper[5052]: I1203 12:20:37.909383 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"server-conf\" (UniqueName: \"kubernetes.io/configmap/c14a906c-f19d-4806-9110-8c47b38c5ae0-server-conf\") pod \"rabbitmq-server-0\" (UID: \"c14a906c-f19d-4806-9110-8c47b38c5ae0\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:20:37 crc kubenswrapper[5052]: I1203 12:20:37.909793 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"plugins-conf\" (UniqueName: \"kubernetes.io/configmap/c14a906c-f19d-4806-9110-8c47b38c5ae0-plugins-conf\") pod \"rabbitmq-server-0\" (UID: \"c14a906c-f19d-4806-9110-8c47b38c5ae0\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:20:37 crc kubenswrapper[5052]: I1203 12:20:37.911118 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"rabbitmq-tls\" (UniqueName: \"kubernetes.io/projected/c14a906c-f19d-4806-9110-8c47b38c5ae0-rabbitmq-tls\") pod \"rabbitmq-server-0\" (UID: \"c14a906c-f19d-4806-9110-8c47b38c5ae0\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:20:37 crc kubenswrapper[5052]: I1203 12:20:37.911830 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"erlang-cookie-secret\" (UniqueName: \"kubernetes.io/secret/c14a906c-f19d-4806-9110-8c47b38c5ae0-erlang-cookie-secret\") pod \"rabbitmq-server-0\" (UID: \"c14a906c-f19d-4806-9110-8c47b38c5ae0\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:20:37 crc kubenswrapper[5052]: I1203 12:20:37.912752 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"rabbitmq-confd\" (UniqueName: \"kubernetes.io/projected/c14a906c-f19d-4806-9110-8c47b38c5ae0-rabbitmq-confd\") pod \"rabbitmq-server-0\" (UID: \"c14a906c-f19d-4806-9110-8c47b38c5ae0\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:20:37 crc kubenswrapper[5052]: I1203 12:20:37.913075 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"pod-info\" (UniqueName: \"kubernetes.io/downward-api/c14a906c-f19d-4806-9110-8c47b38c5ae0-pod-info\") pod \"rabbitmq-server-0\" (UID: \"c14a906c-f19d-4806-9110-8c47b38c5ae0\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:20:37 crc kubenswrapper[5052]: I1203 12:20:37.931857 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-qljvf\" (UniqueName: \"kubernetes.io/projected/c14a906c-f19d-4806-9110-8c47b38c5ae0-kube-api-access-qljvf\") pod \"rabbitmq-server-0\" (UID: \"c14a906c-f19d-4806-9110-8c47b38c5ae0\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:20:37 crc kubenswrapper[5052]: I1203 12:20:37.953667 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage19-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage19-crc\") pod \"rabbitmq-server-0\" (UID: \"c14a906c-f19d-4806-9110-8c47b38c5ae0\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:20:38 crc kubenswrapper[5052]: I1203 12:20:38.015497 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:20:38 crc kubenswrapper[5052]: I1203 12:20:38.463501 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/rabbitmq-server-0"] Dec 03 12:20:38 crc kubenswrapper[5052]: I1203 12:20:38.578432 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-server-0" event={"ID":"c14a906c-f19d-4806-9110-8c47b38c5ae0","Type":"ContainerStarted","Data":"b21e77f593b12d889c25fb56626d0100aa2f2aaed9d36690637f8aa4e39de474"} Dec 03 12:20:38 crc kubenswrapper[5052]: I1203 12:20:38.592763 5052 generic.go:334] "Generic (PLEG): container finished" podID="318545bf-5895-48f9-9649-67207bc4b207" containerID="e3b8749f931ef525570e1e11172cba2d27c70a94440e0ea98f51fe1e60b90b6d" exitCode=0 Dec 03 12:20:38 crc kubenswrapper[5052]: I1203 12:20:38.592833 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-ring-rebalance-z7lpm" event={"ID":"318545bf-5895-48f9-9649-67207bc4b207","Type":"ContainerDied","Data":"e3b8749f931ef525570e1e11172cba2d27c70a94440e0ea98f51fe1e60b90b6d"} Dec 03 12:20:38 crc kubenswrapper[5052]: I1203 12:20:38.603723 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"f50a3c6a-8630-4ca5-99cd-4cf76e55fef8","Type":"ContainerStarted","Data":"54ee1af59e7c0b81f5383ecda09a3919eb885e9052b0a39e29cc34a49666c7cb"} Dec 03 12:20:38 crc kubenswrapper[5052]: I1203 12:20:38.605729 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" event={"ID":"c279eaf1-2ee2-46fe-88be-9a5a65d4897e","Type":"ContainerStarted","Data":"aa080ded14dd4e75e83d60cd422e78ffb3eaef58b0f54feabd6bb11cbe1dde7a"} Dec 03 12:20:40 crc kubenswrapper[5052]: I1203 12:20:40.201079 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/memcached-0" Dec 03 12:20:40 crc kubenswrapper[5052]: I1203 12:20:40.251572 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:20:40 crc kubenswrapper[5052]: I1203 12:20:40.272463 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-ring-rebalance-z7lpm" Dec 03 12:20:40 crc kubenswrapper[5052]: I1203 12:20:40.310971 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:20:40 crc kubenswrapper[5052]: I1203 12:20:40.459850 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/318545bf-5895-48f9-9649-67207bc4b207-scripts\") pod \"318545bf-5895-48f9-9649-67207bc4b207\" (UID: \"318545bf-5895-48f9-9649-67207bc4b207\") " Dec 03 12:20:40 crc kubenswrapper[5052]: I1203 12:20:40.459919 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/empty-dir/318545bf-5895-48f9-9649-67207bc4b207-etc-swift\") pod \"318545bf-5895-48f9-9649-67207bc4b207\" (UID: \"318545bf-5895-48f9-9649-67207bc4b207\") " Dec 03 12:20:40 crc kubenswrapper[5052]: I1203 12:20:40.459950 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pn692\" (UniqueName: \"kubernetes.io/projected/318545bf-5895-48f9-9649-67207bc4b207-kube-api-access-pn692\") pod \"318545bf-5895-48f9-9649-67207bc4b207\" (UID: \"318545bf-5895-48f9-9649-67207bc4b207\") " Dec 03 12:20:40 crc kubenswrapper[5052]: I1203 12:20:40.460011 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/318545bf-5895-48f9-9649-67207bc4b207-combined-ca-bundle\") pod \"318545bf-5895-48f9-9649-67207bc4b207\" (UID: \"318545bf-5895-48f9-9649-67207bc4b207\") " Dec 03 12:20:40 crc kubenswrapper[5052]: I1203 12:20:40.460089 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"swiftconf\" (UniqueName: \"kubernetes.io/secret/318545bf-5895-48f9-9649-67207bc4b207-swiftconf\") pod \"318545bf-5895-48f9-9649-67207bc4b207\" (UID: \"318545bf-5895-48f9-9649-67207bc4b207\") " Dec 03 12:20:40 crc kubenswrapper[5052]: I1203 12:20:40.460152 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"dispersionconf\" (UniqueName: \"kubernetes.io/secret/318545bf-5895-48f9-9649-67207bc4b207-dispersionconf\") pod \"318545bf-5895-48f9-9649-67207bc4b207\" (UID: \"318545bf-5895-48f9-9649-67207bc4b207\") " Dec 03 12:20:40 crc kubenswrapper[5052]: I1203 12:20:40.460198 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ring-data-devices\" (UniqueName: \"kubernetes.io/configmap/318545bf-5895-48f9-9649-67207bc4b207-ring-data-devices\") pod \"318545bf-5895-48f9-9649-67207bc4b207\" (UID: \"318545bf-5895-48f9-9649-67207bc4b207\") " Dec 03 12:20:40 crc kubenswrapper[5052]: I1203 12:20:40.461124 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/318545bf-5895-48f9-9649-67207bc4b207-ring-data-devices" (OuterVolumeSpecName: "ring-data-devices") pod "318545bf-5895-48f9-9649-67207bc4b207" (UID: "318545bf-5895-48f9-9649-67207bc4b207"). InnerVolumeSpecName "ring-data-devices". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:20:40 crc kubenswrapper[5052]: I1203 12:20:40.462154 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/318545bf-5895-48f9-9649-67207bc4b207-etc-swift" (OuterVolumeSpecName: "etc-swift") pod "318545bf-5895-48f9-9649-67207bc4b207" (UID: "318545bf-5895-48f9-9649-67207bc4b207"). InnerVolumeSpecName "etc-swift". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:20:40 crc kubenswrapper[5052]: I1203 12:20:40.470027 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/318545bf-5895-48f9-9649-67207bc4b207-kube-api-access-pn692" (OuterVolumeSpecName: "kube-api-access-pn692") pod "318545bf-5895-48f9-9649-67207bc4b207" (UID: "318545bf-5895-48f9-9649-67207bc4b207"). InnerVolumeSpecName "kube-api-access-pn692". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:20:40 crc kubenswrapper[5052]: I1203 12:20:40.499372 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/318545bf-5895-48f9-9649-67207bc4b207-swiftconf" (OuterVolumeSpecName: "swiftconf") pod "318545bf-5895-48f9-9649-67207bc4b207" (UID: "318545bf-5895-48f9-9649-67207bc4b207"). InnerVolumeSpecName "swiftconf". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:20:40 crc kubenswrapper[5052]: I1203 12:20:40.499592 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/318545bf-5895-48f9-9649-67207bc4b207-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "318545bf-5895-48f9-9649-67207bc4b207" (UID: "318545bf-5895-48f9-9649-67207bc4b207"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:20:40 crc kubenswrapper[5052]: I1203 12:20:40.500106 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/318545bf-5895-48f9-9649-67207bc4b207-scripts" (OuterVolumeSpecName: "scripts") pod "318545bf-5895-48f9-9649-67207bc4b207" (UID: "318545bf-5895-48f9-9649-67207bc4b207"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:20:40 crc kubenswrapper[5052]: I1203 12:20:40.509951 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/318545bf-5895-48f9-9649-67207bc4b207-dispersionconf" (OuterVolumeSpecName: "dispersionconf") pod "318545bf-5895-48f9-9649-67207bc4b207" (UID: "318545bf-5895-48f9-9649-67207bc4b207"). InnerVolumeSpecName "dispersionconf". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:20:40 crc kubenswrapper[5052]: I1203 12:20:40.512625 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:20:40 crc kubenswrapper[5052]: I1203 12:20:40.562654 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/318545bf-5895-48f9-9649-67207bc4b207-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:40 crc kubenswrapper[5052]: I1203 12:20:40.562960 5052 reconciler_common.go:293] "Volume detached for volume \"etc-swift\" (UniqueName: \"kubernetes.io/empty-dir/318545bf-5895-48f9-9649-67207bc4b207-etc-swift\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:40 crc kubenswrapper[5052]: I1203 12:20:40.562977 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-pn692\" (UniqueName: \"kubernetes.io/projected/318545bf-5895-48f9-9649-67207bc4b207-kube-api-access-pn692\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:40 crc kubenswrapper[5052]: I1203 12:20:40.562989 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/318545bf-5895-48f9-9649-67207bc4b207-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:40 crc kubenswrapper[5052]: I1203 12:20:40.563003 5052 reconciler_common.go:293] "Volume detached for volume \"swiftconf\" (UniqueName: \"kubernetes.io/secret/318545bf-5895-48f9-9649-67207bc4b207-swiftconf\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:40 crc kubenswrapper[5052]: I1203 12:20:40.563015 5052 reconciler_common.go:293] "Volume detached for volume \"dispersionconf\" (UniqueName: \"kubernetes.io/secret/318545bf-5895-48f9-9649-67207bc4b207-dispersionconf\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:40 crc kubenswrapper[5052]: I1203 12:20:40.563026 5052 reconciler_common.go:293] "Volume detached for volume \"ring-data-devices\" (UniqueName: \"kubernetes.io/configmap/318545bf-5895-48f9-9649-67207bc4b207-ring-data-devices\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:40 crc kubenswrapper[5052]: I1203 12:20:40.660606 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-server-0" event={"ID":"c14a906c-f19d-4806-9110-8c47b38c5ae0","Type":"ContainerStarted","Data":"4eb8d3dd99172fe5ebfc87ffeaf97722130ac6901117c3bdc435b6f6a054e23f"} Dec 03 12:20:40 crc kubenswrapper[5052]: I1203 12:20:40.667047 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-ring-rebalance-z7lpm" event={"ID":"318545bf-5895-48f9-9649-67207bc4b207","Type":"ContainerDied","Data":"f696cb70d675a723254603c90b2d5ef021170f5f1bf709cae94ef8a50cfa63b1"} Dec 03 12:20:40 crc kubenswrapper[5052]: I1203 12:20:40.667170 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f696cb70d675a723254603c90b2d5ef021170f5f1bf709cae94ef8a50cfa63b1" Dec 03 12:20:40 crc kubenswrapper[5052]: I1203 12:20:40.667488 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-ring-rebalance-z7lpm" Dec 03 12:20:40 crc kubenswrapper[5052]: I1203 12:20:40.691730 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"f50a3c6a-8630-4ca5-99cd-4cf76e55fef8","Type":"ContainerStarted","Data":"089b7b9d3316173ba08748410a75144e1893dcfa6896898974faf640f2c1a05d"} Dec 03 12:20:40 crc kubenswrapper[5052]: I1203 12:20:40.694620 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" event={"ID":"c279eaf1-2ee2-46fe-88be-9a5a65d4897e","Type":"ContainerStarted","Data":"461a75cc1c9b99044785a22ff9dae21bbe4db12db4d1af47322cef18e6d1565c"} Dec 03 12:20:40 crc kubenswrapper[5052]: I1203 12:20:40.715230 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:20:40 crc kubenswrapper[5052]: I1203 12:20:40.826956 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:20:40 crc kubenswrapper[5052]: I1203 12:20:40.827005 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:20:41 crc kubenswrapper[5052]: I1203 12:20:41.186416 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:20:41 crc kubenswrapper[5052]: I1203 12:20:41.290945 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:20:41 crc kubenswrapper[5052]: I1203 12:20:41.292063 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:20:41 crc kubenswrapper[5052]: I1203 12:20:41.377926 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:20:41 crc kubenswrapper[5052]: I1203 12:20:41.710979 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"f50a3c6a-8630-4ca5-99cd-4cf76e55fef8","Type":"ContainerStarted","Data":"8e4cce4b3c05ecd1298240aed31657ec3eef471ac5a79fd4cc2c083140955492"} Dec 03 12:20:41 crc kubenswrapper[5052]: I1203 12:20:41.801540 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:20:41 crc kubenswrapper[5052]: I1203 12:20:41.802461 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:20:42 crc kubenswrapper[5052]: I1203 12:20:42.194842 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-api-0" podUID="ce16d54b-ef04-429b-8795-2865395f7774" containerName="nova-api-log" probeResult="failure" output="Get \"https://10.217.1.145:8774/\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Dec 03 12:20:42 crc kubenswrapper[5052]: I1203 12:20:42.199690 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-api-0" podUID="ce16d54b-ef04-429b-8795-2865395f7774" containerName="nova-api-api" probeResult="failure" output="Get \"https://10.217.1.145:8774/\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Dec 03 12:20:42 crc kubenswrapper[5052]: I1203 12:20:42.535572 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:20:42 crc kubenswrapper[5052]: I1203 12:20:42.727963 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"f50a3c6a-8630-4ca5-99cd-4cf76e55fef8","Type":"ContainerStarted","Data":"7e235b1c981a983fbce29278e319efa581e7cc12336981b70ac5f4f2252d9ea9"} Dec 03 12:20:42 crc kubenswrapper[5052]: I1203 12:20:42.728442 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:20:42 crc kubenswrapper[5052]: I1203 12:20:42.757277 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/ceilometer-0" podStartSLOduration=2.5250878439999997 podStartE2EDuration="7.75726285s" podCreationTimestamp="2025-12-03 12:20:35 +0000 UTC" firstStartedPulling="2025-12-03 12:20:37.039775632 +0000 UTC m=+3627.628017396" lastFinishedPulling="2025-12-03 12:20:42.271950628 +0000 UTC m=+3632.860192402" observedRunningTime="2025-12-03 12:20:42.754961894 +0000 UTC m=+3633.343203648" watchObservedRunningTime="2025-12-03 12:20:42.75726285 +0000 UTC m=+3633.345504614" Dec 03 12:20:43 crc kubenswrapper[5052]: I1203 12:20:43.737823 5052 generic.go:334] "Generic (PLEG): container finished" podID="94ac1a3b-14db-445c-8fb8-8bc3069bdb57" containerID="e47bb47cae9cbe2f35ecd613fb6be4fc2447cc9124af569f2f18e065e8af2171" exitCode=0 Dec 03 12:20:43 crc kubenswrapper[5052]: I1203 12:20:43.737919 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-db-sync-lrqq6" event={"ID":"94ac1a3b-14db-445c-8fb8-8bc3069bdb57","Type":"ContainerDied","Data":"e47bb47cae9cbe2f35ecd613fb6be4fc2447cc9124af569f2f18e065e8af2171"} Dec 03 12:20:43 crc kubenswrapper[5052]: I1203 12:20:43.743003 5052 generic.go:334] "Generic (PLEG): container finished" podID="c4487b92-6d46-4cde-b1c3-056ac6fa9753" containerID="d21edbe94c25709bf834fc9a057d971ffaeed3e9a181d6a8bd2dc198291496be" exitCode=0 Dec 03 12:20:43 crc kubenswrapper[5052]: I1203 12:20:43.743101 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-db-sync-gqt2p" event={"ID":"c4487b92-6d46-4cde-b1c3-056ac6fa9753","Type":"ContainerDied","Data":"d21edbe94c25709bf834fc9a057d971ffaeed3e9a181d6a8bd2dc198291496be"} Dec 03 12:20:44 crc kubenswrapper[5052]: I1203 12:20:44.298876 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:20:44 crc kubenswrapper[5052]: I1203 12:20:44.451098 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:20:44 crc kubenswrapper[5052]: I1203 12:20:44.756503 5052 generic.go:334] "Generic (PLEG): container finished" podID="cb824463-7efe-49ee-a052-acd1fd8f02d3" containerID="2f07afc7faedc91462a894d76d86fe113f7f5b41d2ce70d3b0e31378e31a2ea3" exitCode=0 Dec 03 12:20:44 crc kubenswrapper[5052]: I1203 12:20:44.756705 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-r444h" event={"ID":"cb824463-7efe-49ee-a052-acd1fd8f02d3","Type":"ContainerDied","Data":"2f07afc7faedc91462a894d76d86fe113f7f5b41d2ce70d3b0e31378e31a2ea3"} Dec 03 12:20:45 crc kubenswrapper[5052]: I1203 12:20:45.229431 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-db-sync-gqt2p" Dec 03 12:20:45 crc kubenswrapper[5052]: I1203 12:20:45.235339 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-db-sync-lrqq6" Dec 03 12:20:45 crc kubenswrapper[5052]: I1203 12:20:45.287959 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/94ac1a3b-14db-445c-8fb8-8bc3069bdb57-combined-ca-bundle\") pod \"94ac1a3b-14db-445c-8fb8-8bc3069bdb57\" (UID: \"94ac1a3b-14db-445c-8fb8-8bc3069bdb57\") " Dec 03 12:20:45 crc kubenswrapper[5052]: I1203 12:20:45.288066 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/94ac1a3b-14db-445c-8fb8-8bc3069bdb57-config\") pod \"94ac1a3b-14db-445c-8fb8-8bc3069bdb57\" (UID: \"94ac1a3b-14db-445c-8fb8-8bc3069bdb57\") " Dec 03 12:20:45 crc kubenswrapper[5052]: I1203 12:20:45.288135 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/c4487b92-6d46-4cde-b1c3-056ac6fa9753-config-data\") pod \"c4487b92-6d46-4cde-b1c3-056ac6fa9753\" (UID: \"c4487b92-6d46-4cde-b1c3-056ac6fa9753\") " Dec 03 12:20:45 crc kubenswrapper[5052]: I1203 12:20:45.288172 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/c4487b92-6d46-4cde-b1c3-056ac6fa9753-etc-machine-id\") pod \"c4487b92-6d46-4cde-b1c3-056ac6fa9753\" (UID: \"c4487b92-6d46-4cde-b1c3-056ac6fa9753\") " Dec 03 12:20:45 crc kubenswrapper[5052]: I1203 12:20:45.288218 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/c4487b92-6d46-4cde-b1c3-056ac6fa9753-scripts\") pod \"c4487b92-6d46-4cde-b1c3-056ac6fa9753\" (UID: \"c4487b92-6d46-4cde-b1c3-056ac6fa9753\") " Dec 03 12:20:45 crc kubenswrapper[5052]: I1203 12:20:45.288264 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bplcb\" (UniqueName: \"kubernetes.io/projected/c4487b92-6d46-4cde-b1c3-056ac6fa9753-kube-api-access-bplcb\") pod \"c4487b92-6d46-4cde-b1c3-056ac6fa9753\" (UID: \"c4487b92-6d46-4cde-b1c3-056ac6fa9753\") " Dec 03 12:20:45 crc kubenswrapper[5052]: I1203 12:20:45.288312 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c4487b92-6d46-4cde-b1c3-056ac6fa9753-combined-ca-bundle\") pod \"c4487b92-6d46-4cde-b1c3-056ac6fa9753\" (UID: \"c4487b92-6d46-4cde-b1c3-056ac6fa9753\") " Dec 03 12:20:45 crc kubenswrapper[5052]: I1203 12:20:45.288367 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b4j7k\" (UniqueName: \"kubernetes.io/projected/94ac1a3b-14db-445c-8fb8-8bc3069bdb57-kube-api-access-b4j7k\") pod \"94ac1a3b-14db-445c-8fb8-8bc3069bdb57\" (UID: \"94ac1a3b-14db-445c-8fb8-8bc3069bdb57\") " Dec 03 12:20:45 crc kubenswrapper[5052]: I1203 12:20:45.288391 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/c4487b92-6d46-4cde-b1c3-056ac6fa9753-db-sync-config-data\") pod \"c4487b92-6d46-4cde-b1c3-056ac6fa9753\" (UID: \"c4487b92-6d46-4cde-b1c3-056ac6fa9753\") " Dec 03 12:20:45 crc kubenswrapper[5052]: I1203 12:20:45.289646 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/c4487b92-6d46-4cde-b1c3-056ac6fa9753-etc-machine-id" (OuterVolumeSpecName: "etc-machine-id") pod "c4487b92-6d46-4cde-b1c3-056ac6fa9753" (UID: "c4487b92-6d46-4cde-b1c3-056ac6fa9753"). InnerVolumeSpecName "etc-machine-id". PluginName "kubernetes.io/host-path", VolumeGidValue "" Dec 03 12:20:45 crc kubenswrapper[5052]: I1203 12:20:45.295433 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/94ac1a3b-14db-445c-8fb8-8bc3069bdb57-kube-api-access-b4j7k" (OuterVolumeSpecName: "kube-api-access-b4j7k") pod "94ac1a3b-14db-445c-8fb8-8bc3069bdb57" (UID: "94ac1a3b-14db-445c-8fb8-8bc3069bdb57"). InnerVolumeSpecName "kube-api-access-b4j7k". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:20:45 crc kubenswrapper[5052]: I1203 12:20:45.297628 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c4487b92-6d46-4cde-b1c3-056ac6fa9753-scripts" (OuterVolumeSpecName: "scripts") pod "c4487b92-6d46-4cde-b1c3-056ac6fa9753" (UID: "c4487b92-6d46-4cde-b1c3-056ac6fa9753"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:20:45 crc kubenswrapper[5052]: I1203 12:20:45.297727 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c4487b92-6d46-4cde-b1c3-056ac6fa9753-kube-api-access-bplcb" (OuterVolumeSpecName: "kube-api-access-bplcb") pod "c4487b92-6d46-4cde-b1c3-056ac6fa9753" (UID: "c4487b92-6d46-4cde-b1c3-056ac6fa9753"). InnerVolumeSpecName "kube-api-access-bplcb". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:20:45 crc kubenswrapper[5052]: I1203 12:20:45.331726 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c4487b92-6d46-4cde-b1c3-056ac6fa9753-db-sync-config-data" (OuterVolumeSpecName: "db-sync-config-data") pod "c4487b92-6d46-4cde-b1c3-056ac6fa9753" (UID: "c4487b92-6d46-4cde-b1c3-056ac6fa9753"). InnerVolumeSpecName "db-sync-config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:20:45 crc kubenswrapper[5052]: I1203 12:20:45.341648 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c4487b92-6d46-4cde-b1c3-056ac6fa9753-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "c4487b92-6d46-4cde-b1c3-056ac6fa9753" (UID: "c4487b92-6d46-4cde-b1c3-056ac6fa9753"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:20:45 crc kubenswrapper[5052]: I1203 12:20:45.376656 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/94ac1a3b-14db-445c-8fb8-8bc3069bdb57-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "94ac1a3b-14db-445c-8fb8-8bc3069bdb57" (UID: "94ac1a3b-14db-445c-8fb8-8bc3069bdb57"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:20:45 crc kubenswrapper[5052]: I1203 12:20:45.382245 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/94ac1a3b-14db-445c-8fb8-8bc3069bdb57-config" (OuterVolumeSpecName: "config") pod "94ac1a3b-14db-445c-8fb8-8bc3069bdb57" (UID: "94ac1a3b-14db-445c-8fb8-8bc3069bdb57"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:20:45 crc kubenswrapper[5052]: I1203 12:20:45.384373 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c4487b92-6d46-4cde-b1c3-056ac6fa9753-config-data" (OuterVolumeSpecName: "config-data") pod "c4487b92-6d46-4cde-b1c3-056ac6fa9753" (UID: "c4487b92-6d46-4cde-b1c3-056ac6fa9753"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:20:45 crc kubenswrapper[5052]: I1203 12:20:45.395203 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/c4487b92-6d46-4cde-b1c3-056ac6fa9753-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:45 crc kubenswrapper[5052]: I1203 12:20:45.395373 5052 reconciler_common.go:293] "Volume detached for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/c4487b92-6d46-4cde-b1c3-056ac6fa9753-etc-machine-id\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:45 crc kubenswrapper[5052]: I1203 12:20:45.395491 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/c4487b92-6d46-4cde-b1c3-056ac6fa9753-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:45 crc kubenswrapper[5052]: I1203 12:20:45.395605 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-bplcb\" (UniqueName: \"kubernetes.io/projected/c4487b92-6d46-4cde-b1c3-056ac6fa9753-kube-api-access-bplcb\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:45 crc kubenswrapper[5052]: I1203 12:20:45.395729 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c4487b92-6d46-4cde-b1c3-056ac6fa9753-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:45 crc kubenswrapper[5052]: I1203 12:20:45.395820 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-b4j7k\" (UniqueName: \"kubernetes.io/projected/94ac1a3b-14db-445c-8fb8-8bc3069bdb57-kube-api-access-b4j7k\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:45 crc kubenswrapper[5052]: I1203 12:20:45.395893 5052 reconciler_common.go:293] "Volume detached for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/c4487b92-6d46-4cde-b1c3-056ac6fa9753-db-sync-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:45 crc kubenswrapper[5052]: I1203 12:20:45.395971 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/94ac1a3b-14db-445c-8fb8-8bc3069bdb57-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:45 crc kubenswrapper[5052]: I1203 12:20:45.396052 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/94ac1a3b-14db-445c-8fb8-8bc3069bdb57-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:45 crc kubenswrapper[5052]: I1203 12:20:45.673175 5052 scope.go:117] "RemoveContainer" containerID="09b4a0072741f60499d24f40cc4ff3406d5d22671ced431363ab865172786300" Dec 03 12:20:45 crc kubenswrapper[5052]: E1203 12:20:45.673578 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 12:20:45 crc kubenswrapper[5052]: I1203 12:20:45.784684 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-db-sync-lrqq6" event={"ID":"94ac1a3b-14db-445c-8fb8-8bc3069bdb57","Type":"ContainerDied","Data":"3aec26badb36af30719176457ba5cb8ca7a4acad4e10744afee0c09a24f1da70"} Dec 03 12:20:45 crc kubenswrapper[5052]: I1203 12:20:45.784759 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3aec26badb36af30719176457ba5cb8ca7a4acad4e10744afee0c09a24f1da70" Dec 03 12:20:45 crc kubenswrapper[5052]: I1203 12:20:45.784797 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-db-sync-lrqq6" Dec 03 12:20:45 crc kubenswrapper[5052]: I1203 12:20:45.794278 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="f50a3c6a-8630-4ca5-99cd-4cf76e55fef8" containerName="ceilometer-central-agent" containerID="cri-o://54ee1af59e7c0b81f5383ecda09a3919eb885e9052b0a39e29cc34a49666c7cb" gracePeriod=30 Dec 03 12:20:45 crc kubenswrapper[5052]: I1203 12:20:45.794462 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-db-sync-gqt2p" Dec 03 12:20:45 crc kubenswrapper[5052]: I1203 12:20:45.794548 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-db-sync-gqt2p" event={"ID":"c4487b92-6d46-4cde-b1c3-056ac6fa9753","Type":"ContainerDied","Data":"d474ae25d3177b1d9fe0ff4b2420f4c2baccc1af8926ffad706922bffd92dadd"} Dec 03 12:20:45 crc kubenswrapper[5052]: I1203 12:20:45.794614 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d474ae25d3177b1d9fe0ff4b2420f4c2baccc1af8926ffad706922bffd92dadd" Dec 03 12:20:45 crc kubenswrapper[5052]: I1203 12:20:45.794698 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="f50a3c6a-8630-4ca5-99cd-4cf76e55fef8" containerName="sg-core" containerID="cri-o://8e4cce4b3c05ecd1298240aed31657ec3eef471ac5a79fd4cc2c083140955492" gracePeriod=30 Dec 03 12:20:45 crc kubenswrapper[5052]: I1203 12:20:45.794796 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="f50a3c6a-8630-4ca5-99cd-4cf76e55fef8" containerName="proxy-httpd" containerID="cri-o://7e235b1c981a983fbce29278e319efa581e7cc12336981b70ac5f4f2252d9ea9" gracePeriod=30 Dec 03 12:20:45 crc kubenswrapper[5052]: I1203 12:20:45.794871 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="f50a3c6a-8630-4ca5-99cd-4cf76e55fef8" containerName="ceilometer-notification-agent" containerID="cri-o://089b7b9d3316173ba08748410a75144e1893dcfa6896898974faf640f2c1a05d" gracePeriod=30 Dec 03 12:20:46 crc kubenswrapper[5052]: I1203 12:20:46.034084 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 12:20:46 crc kubenswrapper[5052]: I1203 12:20:46.034664 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/cinder-scheduler-0" podUID="d5750ef1-ce21-4092-9612-13d6f6c923df" containerName="probe" containerID="cri-o://82bddf0f905668e21ca0acf3ebd30d75b07c6a6534dd49912da54363cd369335" gracePeriod=30 Dec 03 12:20:46 crc kubenswrapper[5052]: I1203 12:20:46.035430 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/cinder-scheduler-0" podUID="d5750ef1-ce21-4092-9612-13d6f6c923df" containerName="cinder-scheduler" containerID="cri-o://58cdd95d026e2885f81c339ec294204441c3b8e523ecb43a19f7a3d140811ca5" gracePeriod=30 Dec 03 12:20:46 crc kubenswrapper[5052]: I1203 12:20:46.050414 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/neutron-69c9545844-xf6zm"] Dec 03 12:20:46 crc kubenswrapper[5052]: E1203 12:20:46.051195 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="318545bf-5895-48f9-9649-67207bc4b207" containerName="swift-ring-rebalance" Dec 03 12:20:46 crc kubenswrapper[5052]: I1203 12:20:46.051227 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="318545bf-5895-48f9-9649-67207bc4b207" containerName="swift-ring-rebalance" Dec 03 12:20:46 crc kubenswrapper[5052]: E1203 12:20:46.051257 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="94ac1a3b-14db-445c-8fb8-8bc3069bdb57" containerName="neutron-db-sync" Dec 03 12:20:46 crc kubenswrapper[5052]: I1203 12:20:46.051266 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="94ac1a3b-14db-445c-8fb8-8bc3069bdb57" containerName="neutron-db-sync" Dec 03 12:20:46 crc kubenswrapper[5052]: E1203 12:20:46.051296 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="c4487b92-6d46-4cde-b1c3-056ac6fa9753" containerName="cinder-db-sync" Dec 03 12:20:46 crc kubenswrapper[5052]: I1203 12:20:46.051306 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="c4487b92-6d46-4cde-b1c3-056ac6fa9753" containerName="cinder-db-sync" Dec 03 12:20:46 crc kubenswrapper[5052]: I1203 12:20:46.051729 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="318545bf-5895-48f9-9649-67207bc4b207" containerName="swift-ring-rebalance" Dec 03 12:20:46 crc kubenswrapper[5052]: I1203 12:20:46.051761 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="c4487b92-6d46-4cde-b1c3-056ac6fa9753" containerName="cinder-db-sync" Dec 03 12:20:46 crc kubenswrapper[5052]: I1203 12:20:46.051778 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="94ac1a3b-14db-445c-8fb8-8bc3069bdb57" containerName="neutron-db-sync" Dec 03 12:20:46 crc kubenswrapper[5052]: I1203 12:20:46.053782 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-69c9545844-xf6zm" Dec 03 12:20:46 crc kubenswrapper[5052]: I1203 12:20:46.095563 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-69c9545844-xf6zm"] Dec 03 12:20:46 crc kubenswrapper[5052]: I1203 12:20:46.115619 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/03bb65ef-e9a8-4cfb-9536-001e316b1f72-internal-tls-certs\") pod \"neutron-69c9545844-xf6zm\" (UID: \"03bb65ef-e9a8-4cfb-9536-001e316b1f72\") " pod="openstack-kuttl-tests/neutron-69c9545844-xf6zm" Dec 03 12:20:46 crc kubenswrapper[5052]: I1203 12:20:46.115677 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/03bb65ef-e9a8-4cfb-9536-001e316b1f72-config\") pod \"neutron-69c9545844-xf6zm\" (UID: \"03bb65ef-e9a8-4cfb-9536-001e316b1f72\") " pod="openstack-kuttl-tests/neutron-69c9545844-xf6zm" Dec 03 12:20:46 crc kubenswrapper[5052]: I1203 12:20:46.115764 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/03bb65ef-e9a8-4cfb-9536-001e316b1f72-ovndb-tls-certs\") pod \"neutron-69c9545844-xf6zm\" (UID: \"03bb65ef-e9a8-4cfb-9536-001e316b1f72\") " pod="openstack-kuttl-tests/neutron-69c9545844-xf6zm" Dec 03 12:20:46 crc kubenswrapper[5052]: I1203 12:20:46.115825 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/03bb65ef-e9a8-4cfb-9536-001e316b1f72-httpd-config\") pod \"neutron-69c9545844-xf6zm\" (UID: \"03bb65ef-e9a8-4cfb-9536-001e316b1f72\") " pod="openstack-kuttl-tests/neutron-69c9545844-xf6zm" Dec 03 12:20:46 crc kubenswrapper[5052]: I1203 12:20:46.115878 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/03bb65ef-e9a8-4cfb-9536-001e316b1f72-combined-ca-bundle\") pod \"neutron-69c9545844-xf6zm\" (UID: \"03bb65ef-e9a8-4cfb-9536-001e316b1f72\") " pod="openstack-kuttl-tests/neutron-69c9545844-xf6zm" Dec 03 12:20:46 crc kubenswrapper[5052]: I1203 12:20:46.115926 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ck8dz\" (UniqueName: \"kubernetes.io/projected/03bb65ef-e9a8-4cfb-9536-001e316b1f72-kube-api-access-ck8dz\") pod \"neutron-69c9545844-xf6zm\" (UID: \"03bb65ef-e9a8-4cfb-9536-001e316b1f72\") " pod="openstack-kuttl-tests/neutron-69c9545844-xf6zm" Dec 03 12:20:46 crc kubenswrapper[5052]: I1203 12:20:46.115971 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/03bb65ef-e9a8-4cfb-9536-001e316b1f72-public-tls-certs\") pod \"neutron-69c9545844-xf6zm\" (UID: \"03bb65ef-e9a8-4cfb-9536-001e316b1f72\") " pod="openstack-kuttl-tests/neutron-69c9545844-xf6zm" Dec 03 12:20:46 crc kubenswrapper[5052]: I1203 12:20:46.121970 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 12:20:46 crc kubenswrapper[5052]: I1203 12:20:46.122313 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/cinder-api-0" podUID="99a08445-670e-4194-bdb1-1e7b79dc00f3" containerName="cinder-api-log" containerID="cri-o://c5c1aee8f74d3c51ad667b122373af80c11a540d038db6ff4ad7b44a5a8c4f6e" gracePeriod=30 Dec 03 12:20:46 crc kubenswrapper[5052]: I1203 12:20:46.122915 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/cinder-api-0" podUID="99a08445-670e-4194-bdb1-1e7b79dc00f3" containerName="cinder-api" containerID="cri-o://e09e55a6c45ce4ed5ad18b47c220ac1df2d781056bacefff348ec37669d05115" gracePeriod=30 Dec 03 12:20:46 crc kubenswrapper[5052]: I1203 12:20:46.217551 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/03bb65ef-e9a8-4cfb-9536-001e316b1f72-httpd-config\") pod \"neutron-69c9545844-xf6zm\" (UID: \"03bb65ef-e9a8-4cfb-9536-001e316b1f72\") " pod="openstack-kuttl-tests/neutron-69c9545844-xf6zm" Dec 03 12:20:46 crc kubenswrapper[5052]: I1203 12:20:46.217616 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/03bb65ef-e9a8-4cfb-9536-001e316b1f72-combined-ca-bundle\") pod \"neutron-69c9545844-xf6zm\" (UID: \"03bb65ef-e9a8-4cfb-9536-001e316b1f72\") " pod="openstack-kuttl-tests/neutron-69c9545844-xf6zm" Dec 03 12:20:46 crc kubenswrapper[5052]: I1203 12:20:46.217654 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-ck8dz\" (UniqueName: \"kubernetes.io/projected/03bb65ef-e9a8-4cfb-9536-001e316b1f72-kube-api-access-ck8dz\") pod \"neutron-69c9545844-xf6zm\" (UID: \"03bb65ef-e9a8-4cfb-9536-001e316b1f72\") " pod="openstack-kuttl-tests/neutron-69c9545844-xf6zm" Dec 03 12:20:46 crc kubenswrapper[5052]: I1203 12:20:46.217689 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/03bb65ef-e9a8-4cfb-9536-001e316b1f72-public-tls-certs\") pod \"neutron-69c9545844-xf6zm\" (UID: \"03bb65ef-e9a8-4cfb-9536-001e316b1f72\") " pod="openstack-kuttl-tests/neutron-69c9545844-xf6zm" Dec 03 12:20:46 crc kubenswrapper[5052]: I1203 12:20:46.217717 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/03bb65ef-e9a8-4cfb-9536-001e316b1f72-internal-tls-certs\") pod \"neutron-69c9545844-xf6zm\" (UID: \"03bb65ef-e9a8-4cfb-9536-001e316b1f72\") " pod="openstack-kuttl-tests/neutron-69c9545844-xf6zm" Dec 03 12:20:46 crc kubenswrapper[5052]: I1203 12:20:46.217739 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/03bb65ef-e9a8-4cfb-9536-001e316b1f72-config\") pod \"neutron-69c9545844-xf6zm\" (UID: \"03bb65ef-e9a8-4cfb-9536-001e316b1f72\") " pod="openstack-kuttl-tests/neutron-69c9545844-xf6zm" Dec 03 12:20:46 crc kubenswrapper[5052]: I1203 12:20:46.217792 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/03bb65ef-e9a8-4cfb-9536-001e316b1f72-ovndb-tls-certs\") pod \"neutron-69c9545844-xf6zm\" (UID: \"03bb65ef-e9a8-4cfb-9536-001e316b1f72\") " pod="openstack-kuttl-tests/neutron-69c9545844-xf6zm" Dec 03 12:20:46 crc kubenswrapper[5052]: I1203 12:20:46.237173 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/03bb65ef-e9a8-4cfb-9536-001e316b1f72-public-tls-certs\") pod \"neutron-69c9545844-xf6zm\" (UID: \"03bb65ef-e9a8-4cfb-9536-001e316b1f72\") " pod="openstack-kuttl-tests/neutron-69c9545844-xf6zm" Dec 03 12:20:46 crc kubenswrapper[5052]: I1203 12:20:46.237362 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/03bb65ef-e9a8-4cfb-9536-001e316b1f72-combined-ca-bundle\") pod \"neutron-69c9545844-xf6zm\" (UID: \"03bb65ef-e9a8-4cfb-9536-001e316b1f72\") " pod="openstack-kuttl-tests/neutron-69c9545844-xf6zm" Dec 03 12:20:46 crc kubenswrapper[5052]: I1203 12:20:46.239065 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/03bb65ef-e9a8-4cfb-9536-001e316b1f72-config\") pod \"neutron-69c9545844-xf6zm\" (UID: \"03bb65ef-e9a8-4cfb-9536-001e316b1f72\") " pod="openstack-kuttl-tests/neutron-69c9545844-xf6zm" Dec 03 12:20:46 crc kubenswrapper[5052]: I1203 12:20:46.245659 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-ck8dz\" (UniqueName: \"kubernetes.io/projected/03bb65ef-e9a8-4cfb-9536-001e316b1f72-kube-api-access-ck8dz\") pod \"neutron-69c9545844-xf6zm\" (UID: \"03bb65ef-e9a8-4cfb-9536-001e316b1f72\") " pod="openstack-kuttl-tests/neutron-69c9545844-xf6zm" Dec 03 12:20:46 crc kubenswrapper[5052]: I1203 12:20:46.245915 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/03bb65ef-e9a8-4cfb-9536-001e316b1f72-internal-tls-certs\") pod \"neutron-69c9545844-xf6zm\" (UID: \"03bb65ef-e9a8-4cfb-9536-001e316b1f72\") " pod="openstack-kuttl-tests/neutron-69c9545844-xf6zm" Dec 03 12:20:46 crc kubenswrapper[5052]: I1203 12:20:46.246607 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/03bb65ef-e9a8-4cfb-9536-001e316b1f72-httpd-config\") pod \"neutron-69c9545844-xf6zm\" (UID: \"03bb65ef-e9a8-4cfb-9536-001e316b1f72\") " pod="openstack-kuttl-tests/neutron-69c9545844-xf6zm" Dec 03 12:20:46 crc kubenswrapper[5052]: I1203 12:20:46.250000 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/03bb65ef-e9a8-4cfb-9536-001e316b1f72-ovndb-tls-certs\") pod \"neutron-69c9545844-xf6zm\" (UID: \"03bb65ef-e9a8-4cfb-9536-001e316b1f72\") " pod="openstack-kuttl-tests/neutron-69c9545844-xf6zm" Dec 03 12:20:46 crc kubenswrapper[5052]: I1203 12:20:46.378073 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-r444h" Dec 03 12:20:46 crc kubenswrapper[5052]: I1203 12:20:46.417305 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-69c9545844-xf6zm" Dec 03 12:20:46 crc kubenswrapper[5052]: I1203 12:20:46.421769 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/cb824463-7efe-49ee-a052-acd1fd8f02d3-config-data\") pod \"cb824463-7efe-49ee-a052-acd1fd8f02d3\" (UID: \"cb824463-7efe-49ee-a052-acd1fd8f02d3\") " Dec 03 12:20:46 crc kubenswrapper[5052]: I1203 12:20:46.421811 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/cb824463-7efe-49ee-a052-acd1fd8f02d3-combined-ca-bundle\") pod \"cb824463-7efe-49ee-a052-acd1fd8f02d3\" (UID: \"cb824463-7efe-49ee-a052-acd1fd8f02d3\") " Dec 03 12:20:46 crc kubenswrapper[5052]: I1203 12:20:46.421928 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8rmgs\" (UniqueName: \"kubernetes.io/projected/cb824463-7efe-49ee-a052-acd1fd8f02d3-kube-api-access-8rmgs\") pod \"cb824463-7efe-49ee-a052-acd1fd8f02d3\" (UID: \"cb824463-7efe-49ee-a052-acd1fd8f02d3\") " Dec 03 12:20:46 crc kubenswrapper[5052]: I1203 12:20:46.422038 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/cb824463-7efe-49ee-a052-acd1fd8f02d3-scripts\") pod \"cb824463-7efe-49ee-a052-acd1fd8f02d3\" (UID: \"cb824463-7efe-49ee-a052-acd1fd8f02d3\") " Dec 03 12:20:46 crc kubenswrapper[5052]: I1203 12:20:46.427706 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cb824463-7efe-49ee-a052-acd1fd8f02d3-scripts" (OuterVolumeSpecName: "scripts") pod "cb824463-7efe-49ee-a052-acd1fd8f02d3" (UID: "cb824463-7efe-49ee-a052-acd1fd8f02d3"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:20:46 crc kubenswrapper[5052]: I1203 12:20:46.440848 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cb824463-7efe-49ee-a052-acd1fd8f02d3-kube-api-access-8rmgs" (OuterVolumeSpecName: "kube-api-access-8rmgs") pod "cb824463-7efe-49ee-a052-acd1fd8f02d3" (UID: "cb824463-7efe-49ee-a052-acd1fd8f02d3"). InnerVolumeSpecName "kube-api-access-8rmgs". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:20:46 crc kubenswrapper[5052]: I1203 12:20:46.450154 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cb824463-7efe-49ee-a052-acd1fd8f02d3-config-data" (OuterVolumeSpecName: "config-data") pod "cb824463-7efe-49ee-a052-acd1fd8f02d3" (UID: "cb824463-7efe-49ee-a052-acd1fd8f02d3"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:20:46 crc kubenswrapper[5052]: I1203 12:20:46.458913 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cb824463-7efe-49ee-a052-acd1fd8f02d3-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "cb824463-7efe-49ee-a052-acd1fd8f02d3" (UID: "cb824463-7efe-49ee-a052-acd1fd8f02d3"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:20:46 crc kubenswrapper[5052]: I1203 12:20:46.524243 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-8rmgs\" (UniqueName: \"kubernetes.io/projected/cb824463-7efe-49ee-a052-acd1fd8f02d3-kube-api-access-8rmgs\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:46 crc kubenswrapper[5052]: I1203 12:20:46.524582 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/cb824463-7efe-49ee-a052-acd1fd8f02d3-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:46 crc kubenswrapper[5052]: I1203 12:20:46.524593 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/cb824463-7efe-49ee-a052-acd1fd8f02d3-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:46 crc kubenswrapper[5052]: I1203 12:20:46.524603 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/cb824463-7efe-49ee-a052-acd1fd8f02d3-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:46 crc kubenswrapper[5052]: I1203 12:20:46.811156 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-r444h" Dec 03 12:20:46 crc kubenswrapper[5052]: I1203 12:20:46.812630 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-r444h" event={"ID":"cb824463-7efe-49ee-a052-acd1fd8f02d3","Type":"ContainerDied","Data":"d1c0b9b946484cb4ba9ab688208b0b3936161b2c4dead77bc73efe64622e358b"} Dec 03 12:20:46 crc kubenswrapper[5052]: I1203 12:20:46.812682 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d1c0b9b946484cb4ba9ab688208b0b3936161b2c4dead77bc73efe64622e358b" Dec 03 12:20:46 crc kubenswrapper[5052]: I1203 12:20:46.821578 5052 generic.go:334] "Generic (PLEG): container finished" podID="99a08445-670e-4194-bdb1-1e7b79dc00f3" containerID="c5c1aee8f74d3c51ad667b122373af80c11a540d038db6ff4ad7b44a5a8c4f6e" exitCode=143 Dec 03 12:20:46 crc kubenswrapper[5052]: I1203 12:20:46.821693 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"99a08445-670e-4194-bdb1-1e7b79dc00f3","Type":"ContainerDied","Data":"c5c1aee8f74d3c51ad667b122373af80c11a540d038db6ff4ad7b44a5a8c4f6e"} Dec 03 12:20:46 crc kubenswrapper[5052]: I1203 12:20:46.828444 5052 generic.go:334] "Generic (PLEG): container finished" podID="f50a3c6a-8630-4ca5-99cd-4cf76e55fef8" containerID="7e235b1c981a983fbce29278e319efa581e7cc12336981b70ac5f4f2252d9ea9" exitCode=0 Dec 03 12:20:46 crc kubenswrapper[5052]: I1203 12:20:46.828483 5052 generic.go:334] "Generic (PLEG): container finished" podID="f50a3c6a-8630-4ca5-99cd-4cf76e55fef8" containerID="8e4cce4b3c05ecd1298240aed31657ec3eef471ac5a79fd4cc2c083140955492" exitCode=2 Dec 03 12:20:46 crc kubenswrapper[5052]: I1203 12:20:46.828493 5052 generic.go:334] "Generic (PLEG): container finished" podID="f50a3c6a-8630-4ca5-99cd-4cf76e55fef8" containerID="089b7b9d3316173ba08748410a75144e1893dcfa6896898974faf640f2c1a05d" exitCode=0 Dec 03 12:20:46 crc kubenswrapper[5052]: I1203 12:20:46.828501 5052 generic.go:334] "Generic (PLEG): container finished" podID="f50a3c6a-8630-4ca5-99cd-4cf76e55fef8" containerID="54ee1af59e7c0b81f5383ecda09a3919eb885e9052b0a39e29cc34a49666c7cb" exitCode=0 Dec 03 12:20:46 crc kubenswrapper[5052]: I1203 12:20:46.828540 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"f50a3c6a-8630-4ca5-99cd-4cf76e55fef8","Type":"ContainerDied","Data":"7e235b1c981a983fbce29278e319efa581e7cc12336981b70ac5f4f2252d9ea9"} Dec 03 12:20:46 crc kubenswrapper[5052]: I1203 12:20:46.828573 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"f50a3c6a-8630-4ca5-99cd-4cf76e55fef8","Type":"ContainerDied","Data":"8e4cce4b3c05ecd1298240aed31657ec3eef471ac5a79fd4cc2c083140955492"} Dec 03 12:20:46 crc kubenswrapper[5052]: I1203 12:20:46.828587 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"f50a3c6a-8630-4ca5-99cd-4cf76e55fef8","Type":"ContainerDied","Data":"089b7b9d3316173ba08748410a75144e1893dcfa6896898974faf640f2c1a05d"} Dec 03 12:20:46 crc kubenswrapper[5052]: I1203 12:20:46.828600 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"f50a3c6a-8630-4ca5-99cd-4cf76e55fef8","Type":"ContainerDied","Data":"54ee1af59e7c0b81f5383ecda09a3919eb885e9052b0a39e29cc34a49666c7cb"} Dec 03 12:20:46 crc kubenswrapper[5052]: I1203 12:20:46.847877 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-0"] Dec 03 12:20:46 crc kubenswrapper[5052]: I1203 12:20:46.848107 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-cell1-conductor-0" podUID="aa2b4c07-e5c7-4543-a0a5-f15b8e1b7f83" containerName="nova-cell1-conductor-conductor" containerID="cri-o://4c12bd41f4c701be825345e793ba798c2e3b6ecd4b951f8fb1ee1579f6953903" gracePeriod=30 Dec 03 12:20:46 crc kubenswrapper[5052]: I1203 12:20:46.852741 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:20:46 crc kubenswrapper[5052]: I1203 12:20:46.934459 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m9ffd\" (UniqueName: \"kubernetes.io/projected/f50a3c6a-8630-4ca5-99cd-4cf76e55fef8-kube-api-access-m9ffd\") pod \"f50a3c6a-8630-4ca5-99cd-4cf76e55fef8\" (UID: \"f50a3c6a-8630-4ca5-99cd-4cf76e55fef8\") " Dec 03 12:20:46 crc kubenswrapper[5052]: I1203 12:20:46.934530 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/f50a3c6a-8630-4ca5-99cd-4cf76e55fef8-scripts\") pod \"f50a3c6a-8630-4ca5-99cd-4cf76e55fef8\" (UID: \"f50a3c6a-8630-4ca5-99cd-4cf76e55fef8\") " Dec 03 12:20:46 crc kubenswrapper[5052]: I1203 12:20:46.934607 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/f50a3c6a-8630-4ca5-99cd-4cf76e55fef8-config-data\") pod \"f50a3c6a-8630-4ca5-99cd-4cf76e55fef8\" (UID: \"f50a3c6a-8630-4ca5-99cd-4cf76e55fef8\") " Dec 03 12:20:46 crc kubenswrapper[5052]: I1203 12:20:46.934664 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/f50a3c6a-8630-4ca5-99cd-4cf76e55fef8-log-httpd\") pod \"f50a3c6a-8630-4ca5-99cd-4cf76e55fef8\" (UID: \"f50a3c6a-8630-4ca5-99cd-4cf76e55fef8\") " Dec 03 12:20:46 crc kubenswrapper[5052]: I1203 12:20:46.934797 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f50a3c6a-8630-4ca5-99cd-4cf76e55fef8-combined-ca-bundle\") pod \"f50a3c6a-8630-4ca5-99cd-4cf76e55fef8\" (UID: \"f50a3c6a-8630-4ca5-99cd-4cf76e55fef8\") " Dec 03 12:20:46 crc kubenswrapper[5052]: I1203 12:20:46.934821 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/f50a3c6a-8630-4ca5-99cd-4cf76e55fef8-sg-core-conf-yaml\") pod \"f50a3c6a-8630-4ca5-99cd-4cf76e55fef8\" (UID: \"f50a3c6a-8630-4ca5-99cd-4cf76e55fef8\") " Dec 03 12:20:46 crc kubenswrapper[5052]: I1203 12:20:46.934842 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/f50a3c6a-8630-4ca5-99cd-4cf76e55fef8-ceilometer-tls-certs\") pod \"f50a3c6a-8630-4ca5-99cd-4cf76e55fef8\" (UID: \"f50a3c6a-8630-4ca5-99cd-4cf76e55fef8\") " Dec 03 12:20:46 crc kubenswrapper[5052]: I1203 12:20:46.934883 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/f50a3c6a-8630-4ca5-99cd-4cf76e55fef8-run-httpd\") pod \"f50a3c6a-8630-4ca5-99cd-4cf76e55fef8\" (UID: \"f50a3c6a-8630-4ca5-99cd-4cf76e55fef8\") " Dec 03 12:20:46 crc kubenswrapper[5052]: I1203 12:20:46.935718 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f50a3c6a-8630-4ca5-99cd-4cf76e55fef8-run-httpd" (OuterVolumeSpecName: "run-httpd") pod "f50a3c6a-8630-4ca5-99cd-4cf76e55fef8" (UID: "f50a3c6a-8630-4ca5-99cd-4cf76e55fef8"). InnerVolumeSpecName "run-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:20:46 crc kubenswrapper[5052]: I1203 12:20:46.936944 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f50a3c6a-8630-4ca5-99cd-4cf76e55fef8-log-httpd" (OuterVolumeSpecName: "log-httpd") pod "f50a3c6a-8630-4ca5-99cd-4cf76e55fef8" (UID: "f50a3c6a-8630-4ca5-99cd-4cf76e55fef8"). InnerVolumeSpecName "log-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:20:46 crc kubenswrapper[5052]: I1203 12:20:46.940597 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f50a3c6a-8630-4ca5-99cd-4cf76e55fef8-scripts" (OuterVolumeSpecName: "scripts") pod "f50a3c6a-8630-4ca5-99cd-4cf76e55fef8" (UID: "f50a3c6a-8630-4ca5-99cd-4cf76e55fef8"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:20:46 crc kubenswrapper[5052]: I1203 12:20:46.940683 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f50a3c6a-8630-4ca5-99cd-4cf76e55fef8-kube-api-access-m9ffd" (OuterVolumeSpecName: "kube-api-access-m9ffd") pod "f50a3c6a-8630-4ca5-99cd-4cf76e55fef8" (UID: "f50a3c6a-8630-4ca5-99cd-4cf76e55fef8"). InnerVolumeSpecName "kube-api-access-m9ffd". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:20:46 crc kubenswrapper[5052]: I1203 12:20:46.974200 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f50a3c6a-8630-4ca5-99cd-4cf76e55fef8-sg-core-conf-yaml" (OuterVolumeSpecName: "sg-core-conf-yaml") pod "f50a3c6a-8630-4ca5-99cd-4cf76e55fef8" (UID: "f50a3c6a-8630-4ca5-99cd-4cf76e55fef8"). InnerVolumeSpecName "sg-core-conf-yaml". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:20:47 crc kubenswrapper[5052]: I1203 12:20:47.022702 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f50a3c6a-8630-4ca5-99cd-4cf76e55fef8-ceilometer-tls-certs" (OuterVolumeSpecName: "ceilometer-tls-certs") pod "f50a3c6a-8630-4ca5-99cd-4cf76e55fef8" (UID: "f50a3c6a-8630-4ca5-99cd-4cf76e55fef8"). InnerVolumeSpecName "ceilometer-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:20:47 crc kubenswrapper[5052]: I1203 12:20:47.037644 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/f50a3c6a-8630-4ca5-99cd-4cf76e55fef8-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:47 crc kubenswrapper[5052]: I1203 12:20:47.037760 5052 reconciler_common.go:293] "Volume detached for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/f50a3c6a-8630-4ca5-99cd-4cf76e55fef8-log-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:47 crc kubenswrapper[5052]: I1203 12:20:47.037822 5052 reconciler_common.go:293] "Volume detached for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/f50a3c6a-8630-4ca5-99cd-4cf76e55fef8-sg-core-conf-yaml\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:47 crc kubenswrapper[5052]: I1203 12:20:47.037897 5052 reconciler_common.go:293] "Volume detached for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/f50a3c6a-8630-4ca5-99cd-4cf76e55fef8-ceilometer-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:47 crc kubenswrapper[5052]: I1203 12:20:47.037957 5052 reconciler_common.go:293] "Volume detached for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/f50a3c6a-8630-4ca5-99cd-4cf76e55fef8-run-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:47 crc kubenswrapper[5052]: I1203 12:20:47.038017 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-m9ffd\" (UniqueName: \"kubernetes.io/projected/f50a3c6a-8630-4ca5-99cd-4cf76e55fef8-kube-api-access-m9ffd\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:47 crc kubenswrapper[5052]: I1203 12:20:47.039503 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-69c9545844-xf6zm"] Dec 03 12:20:47 crc kubenswrapper[5052]: I1203 12:20:47.073249 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f50a3c6a-8630-4ca5-99cd-4cf76e55fef8-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "f50a3c6a-8630-4ca5-99cd-4cf76e55fef8" (UID: "f50a3c6a-8630-4ca5-99cd-4cf76e55fef8"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:20:47 crc kubenswrapper[5052]: I1203 12:20:47.087073 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f50a3c6a-8630-4ca5-99cd-4cf76e55fef8-config-data" (OuterVolumeSpecName: "config-data") pod "f50a3c6a-8630-4ca5-99cd-4cf76e55fef8" (UID: "f50a3c6a-8630-4ca5-99cd-4cf76e55fef8"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:20:47 crc kubenswrapper[5052]: I1203 12:20:47.144190 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f50a3c6a-8630-4ca5-99cd-4cf76e55fef8-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:47 crc kubenswrapper[5052]: I1203 12:20:47.144529 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/f50a3c6a-8630-4ca5-99cd-4cf76e55fef8-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:47 crc kubenswrapper[5052]: E1203 12:20:47.483225 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="4c12bd41f4c701be825345e793ba798c2e3b6ecd4b951f8fb1ee1579f6953903" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:20:47 crc kubenswrapper[5052]: E1203 12:20:47.487265 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="4c12bd41f4c701be825345e793ba798c2e3b6ecd4b951f8fb1ee1579f6953903" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:20:47 crc kubenswrapper[5052]: E1203 12:20:47.488677 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="4c12bd41f4c701be825345e793ba798c2e3b6ecd4b951f8fb1ee1579f6953903" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:20:47 crc kubenswrapper[5052]: E1203 12:20:47.488709 5052 prober.go:104] "Probe errored" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" probeType="Readiness" pod="openstack-kuttl-tests/nova-cell1-conductor-0" podUID="aa2b4c07-e5c7-4543-a0a5-f15b8e1b7f83" containerName="nova-cell1-conductor-conductor" Dec 03 12:20:47 crc kubenswrapper[5052]: I1203 12:20:47.842457 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-69c9545844-xf6zm" event={"ID":"03bb65ef-e9a8-4cfb-9536-001e316b1f72","Type":"ContainerStarted","Data":"ece24b1db9c0709fd516d1a80d474c2937fdeb49dd573043245a6e0d4d27d04a"} Dec 03 12:20:47 crc kubenswrapper[5052]: I1203 12:20:47.843611 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/neutron-69c9545844-xf6zm" Dec 03 12:20:47 crc kubenswrapper[5052]: I1203 12:20:47.843691 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-69c9545844-xf6zm" event={"ID":"03bb65ef-e9a8-4cfb-9536-001e316b1f72","Type":"ContainerStarted","Data":"48fa168d5d7691a3b2d8508087753330d79877c968cc2b4e0f021b0441800e44"} Dec 03 12:20:47 crc kubenswrapper[5052]: I1203 12:20:47.843767 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-69c9545844-xf6zm" event={"ID":"03bb65ef-e9a8-4cfb-9536-001e316b1f72","Type":"ContainerStarted","Data":"9f16a2482848753a3055f65a88879c86ee646460485035f648588f1468241420"} Dec 03 12:20:47 crc kubenswrapper[5052]: I1203 12:20:47.846662 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"f50a3c6a-8630-4ca5-99cd-4cf76e55fef8","Type":"ContainerDied","Data":"1233ff3b35b92293a94cb57661d5131e67c6e740e238a5179328cce848e33741"} Dec 03 12:20:47 crc kubenswrapper[5052]: I1203 12:20:47.846725 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:20:47 crc kubenswrapper[5052]: I1203 12:20:47.846926 5052 scope.go:117] "RemoveContainer" containerID="7e235b1c981a983fbce29278e319efa581e7cc12336981b70ac5f4f2252d9ea9" Dec 03 12:20:47 crc kubenswrapper[5052]: I1203 12:20:47.850428 5052 generic.go:334] "Generic (PLEG): container finished" podID="d5750ef1-ce21-4092-9612-13d6f6c923df" containerID="82bddf0f905668e21ca0acf3ebd30d75b07c6a6534dd49912da54363cd369335" exitCode=0 Dec 03 12:20:47 crc kubenswrapper[5052]: I1203 12:20:47.850565 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"d5750ef1-ce21-4092-9612-13d6f6c923df","Type":"ContainerDied","Data":"82bddf0f905668e21ca0acf3ebd30d75b07c6a6534dd49912da54363cd369335"} Dec 03 12:20:47 crc kubenswrapper[5052]: I1203 12:20:47.852616 5052 generic.go:334] "Generic (PLEG): container finished" podID="9ea78435-8fd9-4321-9e48-3aaae833dcf2" containerID="4acfccfaa6f110a7544ac7a681df22aceab18e0f00a4e818bfb5f8c9536032f3" exitCode=0 Dec 03 12:20:47 crc kubenswrapper[5052]: I1203 12:20:47.852663 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-hjwg6" event={"ID":"9ea78435-8fd9-4321-9e48-3aaae833dcf2","Type":"ContainerDied","Data":"4acfccfaa6f110a7544ac7a681df22aceab18e0f00a4e818bfb5f8c9536032f3"} Dec 03 12:20:47 crc kubenswrapper[5052]: I1203 12:20:47.878018 5052 scope.go:117] "RemoveContainer" containerID="8e4cce4b3c05ecd1298240aed31657ec3eef471ac5a79fd4cc2c083140955492" Dec 03 12:20:47 crc kubenswrapper[5052]: I1203 12:20:47.893058 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/neutron-69c9545844-xf6zm" podStartSLOduration=2.893038406 podStartE2EDuration="2.893038406s" podCreationTimestamp="2025-12-03 12:20:45 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:20:47.87670802 +0000 UTC m=+3638.464949784" watchObservedRunningTime="2025-12-03 12:20:47.893038406 +0000 UTC m=+3638.481280170" Dec 03 12:20:47 crc kubenswrapper[5052]: I1203 12:20:47.901783 5052 scope.go:117] "RemoveContainer" containerID="089b7b9d3316173ba08748410a75144e1893dcfa6896898974faf640f2c1a05d" Dec 03 12:20:47 crc kubenswrapper[5052]: I1203 12:20:47.902919 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:20:47 crc kubenswrapper[5052]: I1203 12:20:47.918952 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:20:47 crc kubenswrapper[5052]: I1203 12:20:47.954023 5052 scope.go:117] "RemoveContainer" containerID="54ee1af59e7c0b81f5383ecda09a3919eb885e9052b0a39e29cc34a49666c7cb" Dec 03 12:20:47 crc kubenswrapper[5052]: I1203 12:20:47.954182 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:20:47 crc kubenswrapper[5052]: E1203 12:20:47.954732 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="cb824463-7efe-49ee-a052-acd1fd8f02d3" containerName="nova-cell1-conductor-db-sync" Dec 03 12:20:47 crc kubenswrapper[5052]: I1203 12:20:47.954748 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="cb824463-7efe-49ee-a052-acd1fd8f02d3" containerName="nova-cell1-conductor-db-sync" Dec 03 12:20:47 crc kubenswrapper[5052]: E1203 12:20:47.954769 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="f50a3c6a-8630-4ca5-99cd-4cf76e55fef8" containerName="proxy-httpd" Dec 03 12:20:47 crc kubenswrapper[5052]: I1203 12:20:47.954777 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="f50a3c6a-8630-4ca5-99cd-4cf76e55fef8" containerName="proxy-httpd" Dec 03 12:20:47 crc kubenswrapper[5052]: E1203 12:20:47.954797 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="f50a3c6a-8630-4ca5-99cd-4cf76e55fef8" containerName="sg-core" Dec 03 12:20:47 crc kubenswrapper[5052]: I1203 12:20:47.954804 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="f50a3c6a-8630-4ca5-99cd-4cf76e55fef8" containerName="sg-core" Dec 03 12:20:47 crc kubenswrapper[5052]: E1203 12:20:47.954824 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="f50a3c6a-8630-4ca5-99cd-4cf76e55fef8" containerName="ceilometer-central-agent" Dec 03 12:20:47 crc kubenswrapper[5052]: I1203 12:20:47.954832 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="f50a3c6a-8630-4ca5-99cd-4cf76e55fef8" containerName="ceilometer-central-agent" Dec 03 12:20:47 crc kubenswrapper[5052]: E1203 12:20:47.954849 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="f50a3c6a-8630-4ca5-99cd-4cf76e55fef8" containerName="ceilometer-notification-agent" Dec 03 12:20:47 crc kubenswrapper[5052]: I1203 12:20:47.954857 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="f50a3c6a-8630-4ca5-99cd-4cf76e55fef8" containerName="ceilometer-notification-agent" Dec 03 12:20:47 crc kubenswrapper[5052]: I1203 12:20:47.955129 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="f50a3c6a-8630-4ca5-99cd-4cf76e55fef8" containerName="ceilometer-notification-agent" Dec 03 12:20:47 crc kubenswrapper[5052]: I1203 12:20:47.955151 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="f50a3c6a-8630-4ca5-99cd-4cf76e55fef8" containerName="ceilometer-central-agent" Dec 03 12:20:47 crc kubenswrapper[5052]: I1203 12:20:47.955172 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="f50a3c6a-8630-4ca5-99cd-4cf76e55fef8" containerName="sg-core" Dec 03 12:20:47 crc kubenswrapper[5052]: I1203 12:20:47.955189 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="f50a3c6a-8630-4ca5-99cd-4cf76e55fef8" containerName="proxy-httpd" Dec 03 12:20:47 crc kubenswrapper[5052]: I1203 12:20:47.955202 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="cb824463-7efe-49ee-a052-acd1fd8f02d3" containerName="nova-cell1-conductor-db-sync" Dec 03 12:20:47 crc kubenswrapper[5052]: I1203 12:20:47.972485 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:20:47 crc kubenswrapper[5052]: I1203 12:20:47.972595 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:20:47 crc kubenswrapper[5052]: I1203 12:20:47.980160 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-config-data" Dec 03 12:20:47 crc kubenswrapper[5052]: I1203 12:20:47.981889 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-ceilometer-internal-svc" Dec 03 12:20:48 crc kubenswrapper[5052]: I1203 12:20:47.999756 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-scripts" Dec 03 12:20:48 crc kubenswrapper[5052]: I1203 12:20:48.079905 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2d94bf00-7a64-4e90-a48c-b206ec54da6d-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"2d94bf00-7a64-4e90-a48c-b206ec54da6d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:20:48 crc kubenswrapper[5052]: I1203 12:20:48.079981 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q9kzq\" (UniqueName: \"kubernetes.io/projected/2d94bf00-7a64-4e90-a48c-b206ec54da6d-kube-api-access-q9kzq\") pod \"ceilometer-0\" (UID: \"2d94bf00-7a64-4e90-a48c-b206ec54da6d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:20:48 crc kubenswrapper[5052]: I1203 12:20:48.080198 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/2d94bf00-7a64-4e90-a48c-b206ec54da6d-scripts\") pod \"ceilometer-0\" (UID: \"2d94bf00-7a64-4e90-a48c-b206ec54da6d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:20:48 crc kubenswrapper[5052]: I1203 12:20:48.080347 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/2d94bf00-7a64-4e90-a48c-b206ec54da6d-config-data\") pod \"ceilometer-0\" (UID: \"2d94bf00-7a64-4e90-a48c-b206ec54da6d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:20:48 crc kubenswrapper[5052]: I1203 12:20:48.080377 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/2d94bf00-7a64-4e90-a48c-b206ec54da6d-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"2d94bf00-7a64-4e90-a48c-b206ec54da6d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:20:48 crc kubenswrapper[5052]: I1203 12:20:48.080570 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/2d94bf00-7a64-4e90-a48c-b206ec54da6d-run-httpd\") pod \"ceilometer-0\" (UID: \"2d94bf00-7a64-4e90-a48c-b206ec54da6d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:20:48 crc kubenswrapper[5052]: I1203 12:20:48.080677 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/2d94bf00-7a64-4e90-a48c-b206ec54da6d-ceilometer-tls-certs\") pod \"ceilometer-0\" (UID: \"2d94bf00-7a64-4e90-a48c-b206ec54da6d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:20:48 crc kubenswrapper[5052]: I1203 12:20:48.080800 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/2d94bf00-7a64-4e90-a48c-b206ec54da6d-log-httpd\") pod \"ceilometer-0\" (UID: \"2d94bf00-7a64-4e90-a48c-b206ec54da6d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:20:48 crc kubenswrapper[5052]: I1203 12:20:48.182718 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-q9kzq\" (UniqueName: \"kubernetes.io/projected/2d94bf00-7a64-4e90-a48c-b206ec54da6d-kube-api-access-q9kzq\") pod \"ceilometer-0\" (UID: \"2d94bf00-7a64-4e90-a48c-b206ec54da6d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:20:48 crc kubenswrapper[5052]: I1203 12:20:48.182789 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/2d94bf00-7a64-4e90-a48c-b206ec54da6d-scripts\") pod \"ceilometer-0\" (UID: \"2d94bf00-7a64-4e90-a48c-b206ec54da6d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:20:48 crc kubenswrapper[5052]: I1203 12:20:48.182833 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/2d94bf00-7a64-4e90-a48c-b206ec54da6d-config-data\") pod \"ceilometer-0\" (UID: \"2d94bf00-7a64-4e90-a48c-b206ec54da6d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:20:48 crc kubenswrapper[5052]: I1203 12:20:48.182850 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/2d94bf00-7a64-4e90-a48c-b206ec54da6d-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"2d94bf00-7a64-4e90-a48c-b206ec54da6d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:20:48 crc kubenswrapper[5052]: I1203 12:20:48.182897 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/2d94bf00-7a64-4e90-a48c-b206ec54da6d-run-httpd\") pod \"ceilometer-0\" (UID: \"2d94bf00-7a64-4e90-a48c-b206ec54da6d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:20:48 crc kubenswrapper[5052]: I1203 12:20:48.182934 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/2d94bf00-7a64-4e90-a48c-b206ec54da6d-ceilometer-tls-certs\") pod \"ceilometer-0\" (UID: \"2d94bf00-7a64-4e90-a48c-b206ec54da6d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:20:48 crc kubenswrapper[5052]: I1203 12:20:48.182976 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/2d94bf00-7a64-4e90-a48c-b206ec54da6d-log-httpd\") pod \"ceilometer-0\" (UID: \"2d94bf00-7a64-4e90-a48c-b206ec54da6d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:20:48 crc kubenswrapper[5052]: I1203 12:20:48.183012 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2d94bf00-7a64-4e90-a48c-b206ec54da6d-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"2d94bf00-7a64-4e90-a48c-b206ec54da6d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:20:48 crc kubenswrapper[5052]: I1203 12:20:48.184330 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/2d94bf00-7a64-4e90-a48c-b206ec54da6d-run-httpd\") pod \"ceilometer-0\" (UID: \"2d94bf00-7a64-4e90-a48c-b206ec54da6d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:20:48 crc kubenswrapper[5052]: I1203 12:20:48.184413 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/2d94bf00-7a64-4e90-a48c-b206ec54da6d-log-httpd\") pod \"ceilometer-0\" (UID: \"2d94bf00-7a64-4e90-a48c-b206ec54da6d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:20:48 crc kubenswrapper[5052]: I1203 12:20:48.189795 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/2d94bf00-7a64-4e90-a48c-b206ec54da6d-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"2d94bf00-7a64-4e90-a48c-b206ec54da6d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:20:48 crc kubenswrapper[5052]: I1203 12:20:48.190058 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2d94bf00-7a64-4e90-a48c-b206ec54da6d-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"2d94bf00-7a64-4e90-a48c-b206ec54da6d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:20:48 crc kubenswrapper[5052]: I1203 12:20:48.190497 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/2d94bf00-7a64-4e90-a48c-b206ec54da6d-config-data\") pod \"ceilometer-0\" (UID: \"2d94bf00-7a64-4e90-a48c-b206ec54da6d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:20:48 crc kubenswrapper[5052]: I1203 12:20:48.190752 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/2d94bf00-7a64-4e90-a48c-b206ec54da6d-scripts\") pod \"ceilometer-0\" (UID: \"2d94bf00-7a64-4e90-a48c-b206ec54da6d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:20:48 crc kubenswrapper[5052]: I1203 12:20:48.197190 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/2d94bf00-7a64-4e90-a48c-b206ec54da6d-ceilometer-tls-certs\") pod \"ceilometer-0\" (UID: \"2d94bf00-7a64-4e90-a48c-b206ec54da6d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:20:48 crc kubenswrapper[5052]: I1203 12:20:48.206821 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-q9kzq\" (UniqueName: \"kubernetes.io/projected/2d94bf00-7a64-4e90-a48c-b206ec54da6d-kube-api-access-q9kzq\") pod \"ceilometer-0\" (UID: \"2d94bf00-7a64-4e90-a48c-b206ec54da6d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:20:48 crc kubenswrapper[5052]: I1203 12:20:48.293915 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:20:48 crc kubenswrapper[5052]: I1203 12:20:48.810798 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:20:48 crc kubenswrapper[5052]: I1203 12:20:48.863384 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"2d94bf00-7a64-4e90-a48c-b206ec54da6d","Type":"ContainerStarted","Data":"e15733fe2ecaff5b6cb0fc9bc23ab8123d3d773b2cda2b3b92b9af2f689a159d"} Dec 03 12:20:48 crc kubenswrapper[5052]: I1203 12:20:48.866426 5052 generic.go:334] "Generic (PLEG): container finished" podID="50ae31b2-a1c0-4afb-8e23-d7856c55af1d" containerID="739b5f17c284ab8823d07f252de61cfc9a034a60b12ad3eafd6d9d840e4cf76d" exitCode=0 Dec 03 12:20:48 crc kubenswrapper[5052]: I1203 12:20:48.866659 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-db-sync-gzzkp" event={"ID":"50ae31b2-a1c0-4afb-8e23-d7856c55af1d","Type":"ContainerDied","Data":"739b5f17c284ab8823d07f252de61cfc9a034a60b12ad3eafd6d9d840e4cf76d"} Dec 03 12:20:49 crc kubenswrapper[5052]: I1203 12:20:49.136969 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-hjwg6" Dec 03 12:20:49 crc kubenswrapper[5052]: I1203 12:20:49.204065 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5rfwf\" (UniqueName: \"kubernetes.io/projected/9ea78435-8fd9-4321-9e48-3aaae833dcf2-kube-api-access-5rfwf\") pod \"9ea78435-8fd9-4321-9e48-3aaae833dcf2\" (UID: \"9ea78435-8fd9-4321-9e48-3aaae833dcf2\") " Dec 03 12:20:49 crc kubenswrapper[5052]: I1203 12:20:49.204189 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/9ea78435-8fd9-4321-9e48-3aaae833dcf2-scripts\") pod \"9ea78435-8fd9-4321-9e48-3aaae833dcf2\" (UID: \"9ea78435-8fd9-4321-9e48-3aaae833dcf2\") " Dec 03 12:20:49 crc kubenswrapper[5052]: I1203 12:20:49.204234 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9ea78435-8fd9-4321-9e48-3aaae833dcf2-combined-ca-bundle\") pod \"9ea78435-8fd9-4321-9e48-3aaae833dcf2\" (UID: \"9ea78435-8fd9-4321-9e48-3aaae833dcf2\") " Dec 03 12:20:49 crc kubenswrapper[5052]: I1203 12:20:49.204266 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/9ea78435-8fd9-4321-9e48-3aaae833dcf2-config-data\") pod \"9ea78435-8fd9-4321-9e48-3aaae833dcf2\" (UID: \"9ea78435-8fd9-4321-9e48-3aaae833dcf2\") " Dec 03 12:20:49 crc kubenswrapper[5052]: I1203 12:20:49.212684 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9ea78435-8fd9-4321-9e48-3aaae833dcf2-scripts" (OuterVolumeSpecName: "scripts") pod "9ea78435-8fd9-4321-9e48-3aaae833dcf2" (UID: "9ea78435-8fd9-4321-9e48-3aaae833dcf2"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:20:49 crc kubenswrapper[5052]: I1203 12:20:49.213148 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9ea78435-8fd9-4321-9e48-3aaae833dcf2-kube-api-access-5rfwf" (OuterVolumeSpecName: "kube-api-access-5rfwf") pod "9ea78435-8fd9-4321-9e48-3aaae833dcf2" (UID: "9ea78435-8fd9-4321-9e48-3aaae833dcf2"). InnerVolumeSpecName "kube-api-access-5rfwf". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:20:49 crc kubenswrapper[5052]: I1203 12:20:49.234707 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9ea78435-8fd9-4321-9e48-3aaae833dcf2-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "9ea78435-8fd9-4321-9e48-3aaae833dcf2" (UID: "9ea78435-8fd9-4321-9e48-3aaae833dcf2"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:20:49 crc kubenswrapper[5052]: I1203 12:20:49.235823 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9ea78435-8fd9-4321-9e48-3aaae833dcf2-config-data" (OuterVolumeSpecName: "config-data") pod "9ea78435-8fd9-4321-9e48-3aaae833dcf2" (UID: "9ea78435-8fd9-4321-9e48-3aaae833dcf2"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:20:49 crc kubenswrapper[5052]: I1203 12:20:49.307103 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/9ea78435-8fd9-4321-9e48-3aaae833dcf2-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:49 crc kubenswrapper[5052]: I1203 12:20:49.307136 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9ea78435-8fd9-4321-9e48-3aaae833dcf2-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:49 crc kubenswrapper[5052]: I1203 12:20:49.307151 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/9ea78435-8fd9-4321-9e48-3aaae833dcf2-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:49 crc kubenswrapper[5052]: I1203 12:20:49.307162 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-5rfwf\" (UniqueName: \"kubernetes.io/projected/9ea78435-8fd9-4321-9e48-3aaae833dcf2-kube-api-access-5rfwf\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:49 crc kubenswrapper[5052]: I1203 12:20:49.693383 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f50a3c6a-8630-4ca5-99cd-4cf76e55fef8" path="/var/lib/kubelet/pods/f50a3c6a-8630-4ca5-99cd-4cf76e55fef8/volumes" Dec 03 12:20:49 crc kubenswrapper[5052]: I1203 12:20:49.885679 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-hjwg6" event={"ID":"9ea78435-8fd9-4321-9e48-3aaae833dcf2","Type":"ContainerDied","Data":"2d570bd0295bdf63ea3bd8c425b3cd03f0e2e31ab814b356062540ad27c5917b"} Dec 03 12:20:49 crc kubenswrapper[5052]: I1203 12:20:49.885729 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2d570bd0295bdf63ea3bd8c425b3cd03f0e2e31ab814b356062540ad27c5917b" Dec 03 12:20:49 crc kubenswrapper[5052]: I1203 12:20:49.885794 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-hjwg6" Dec 03 12:20:49 crc kubenswrapper[5052]: I1203 12:20:49.901711 5052 generic.go:334] "Generic (PLEG): container finished" podID="99a08445-670e-4194-bdb1-1e7b79dc00f3" containerID="e09e55a6c45ce4ed5ad18b47c220ac1df2d781056bacefff348ec37669d05115" exitCode=0 Dec 03 12:20:49 crc kubenswrapper[5052]: I1203 12:20:49.901793 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"99a08445-670e-4194-bdb1-1e7b79dc00f3","Type":"ContainerDied","Data":"e09e55a6c45ce4ed5ad18b47c220ac1df2d781056bacefff348ec37669d05115"} Dec 03 12:20:49 crc kubenswrapper[5052]: I1203 12:20:49.903544 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"2d94bf00-7a64-4e90-a48c-b206ec54da6d","Type":"ContainerStarted","Data":"cf05124bcf6f5c1b9dc38517547ce3e4342c12cea5c84ee53adeabf6d90e42cf"} Dec 03 12:20:49 crc kubenswrapper[5052]: I1203 12:20:49.980123 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:20:49 crc kubenswrapper[5052]: I1203 12:20:49.981727 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-0"] Dec 03 12:20:49 crc kubenswrapper[5052]: I1203 12:20:49.981999 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-cell0-conductor-0" podUID="e80169f9-5480-4318-8ab2-1af4394fe76a" containerName="nova-cell0-conductor-conductor" containerID="cri-o://a96261519a88986377772f2eba818435423e35e4613b257b006f4e1d712a712b" gracePeriod=30 Dec 03 12:20:50 crc kubenswrapper[5052]: I1203 12:20:50.127980 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/99a08445-670e-4194-bdb1-1e7b79dc00f3-scripts\") pod \"99a08445-670e-4194-bdb1-1e7b79dc00f3\" (UID: \"99a08445-670e-4194-bdb1-1e7b79dc00f3\") " Dec 03 12:20:50 crc kubenswrapper[5052]: I1203 12:20:50.128083 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/99a08445-670e-4194-bdb1-1e7b79dc00f3-combined-ca-bundle\") pod \"99a08445-670e-4194-bdb1-1e7b79dc00f3\" (UID: \"99a08445-670e-4194-bdb1-1e7b79dc00f3\") " Dec 03 12:20:50 crc kubenswrapper[5052]: I1203 12:20:50.128120 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cs5ln\" (UniqueName: \"kubernetes.io/projected/99a08445-670e-4194-bdb1-1e7b79dc00f3-kube-api-access-cs5ln\") pod \"99a08445-670e-4194-bdb1-1e7b79dc00f3\" (UID: \"99a08445-670e-4194-bdb1-1e7b79dc00f3\") " Dec 03 12:20:50 crc kubenswrapper[5052]: I1203 12:20:50.128327 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/99a08445-670e-4194-bdb1-1e7b79dc00f3-internal-tls-certs\") pod \"99a08445-670e-4194-bdb1-1e7b79dc00f3\" (UID: \"99a08445-670e-4194-bdb1-1e7b79dc00f3\") " Dec 03 12:20:50 crc kubenswrapper[5052]: I1203 12:20:50.128359 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/99a08445-670e-4194-bdb1-1e7b79dc00f3-logs\") pod \"99a08445-670e-4194-bdb1-1e7b79dc00f3\" (UID: \"99a08445-670e-4194-bdb1-1e7b79dc00f3\") " Dec 03 12:20:50 crc kubenswrapper[5052]: I1203 12:20:50.128438 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/99a08445-670e-4194-bdb1-1e7b79dc00f3-config-data-custom\") pod \"99a08445-670e-4194-bdb1-1e7b79dc00f3\" (UID: \"99a08445-670e-4194-bdb1-1e7b79dc00f3\") " Dec 03 12:20:50 crc kubenswrapper[5052]: I1203 12:20:50.128561 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/99a08445-670e-4194-bdb1-1e7b79dc00f3-config-data\") pod \"99a08445-670e-4194-bdb1-1e7b79dc00f3\" (UID: \"99a08445-670e-4194-bdb1-1e7b79dc00f3\") " Dec 03 12:20:50 crc kubenswrapper[5052]: I1203 12:20:50.128621 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/99a08445-670e-4194-bdb1-1e7b79dc00f3-public-tls-certs\") pod \"99a08445-670e-4194-bdb1-1e7b79dc00f3\" (UID: \"99a08445-670e-4194-bdb1-1e7b79dc00f3\") " Dec 03 12:20:50 crc kubenswrapper[5052]: I1203 12:20:50.129635 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/99a08445-670e-4194-bdb1-1e7b79dc00f3-etc-machine-id\") pod \"99a08445-670e-4194-bdb1-1e7b79dc00f3\" (UID: \"99a08445-670e-4194-bdb1-1e7b79dc00f3\") " Dec 03 12:20:50 crc kubenswrapper[5052]: I1203 12:20:50.131220 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/99a08445-670e-4194-bdb1-1e7b79dc00f3-etc-machine-id" (OuterVolumeSpecName: "etc-machine-id") pod "99a08445-670e-4194-bdb1-1e7b79dc00f3" (UID: "99a08445-670e-4194-bdb1-1e7b79dc00f3"). InnerVolumeSpecName "etc-machine-id". PluginName "kubernetes.io/host-path", VolumeGidValue "" Dec 03 12:20:50 crc kubenswrapper[5052]: I1203 12:20:50.132133 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/99a08445-670e-4194-bdb1-1e7b79dc00f3-logs" (OuterVolumeSpecName: "logs") pod "99a08445-670e-4194-bdb1-1e7b79dc00f3" (UID: "99a08445-670e-4194-bdb1-1e7b79dc00f3"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:20:50 crc kubenswrapper[5052]: I1203 12:20:50.138359 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/99a08445-670e-4194-bdb1-1e7b79dc00f3-config-data-custom" (OuterVolumeSpecName: "config-data-custom") pod "99a08445-670e-4194-bdb1-1e7b79dc00f3" (UID: "99a08445-670e-4194-bdb1-1e7b79dc00f3"). InnerVolumeSpecName "config-data-custom". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:20:50 crc kubenswrapper[5052]: I1203 12:20:50.141122 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/99a08445-670e-4194-bdb1-1e7b79dc00f3-kube-api-access-cs5ln" (OuterVolumeSpecName: "kube-api-access-cs5ln") pod "99a08445-670e-4194-bdb1-1e7b79dc00f3" (UID: "99a08445-670e-4194-bdb1-1e7b79dc00f3"). InnerVolumeSpecName "kube-api-access-cs5ln". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:20:50 crc kubenswrapper[5052]: I1203 12:20:50.143748 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/99a08445-670e-4194-bdb1-1e7b79dc00f3-scripts" (OuterVolumeSpecName: "scripts") pod "99a08445-670e-4194-bdb1-1e7b79dc00f3" (UID: "99a08445-670e-4194-bdb1-1e7b79dc00f3"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:20:50 crc kubenswrapper[5052]: I1203 12:20:50.207691 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/99a08445-670e-4194-bdb1-1e7b79dc00f3-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "99a08445-670e-4194-bdb1-1e7b79dc00f3" (UID: "99a08445-670e-4194-bdb1-1e7b79dc00f3"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:20:50 crc kubenswrapper[5052]: I1203 12:20:50.219684 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/99a08445-670e-4194-bdb1-1e7b79dc00f3-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "99a08445-670e-4194-bdb1-1e7b79dc00f3" (UID: "99a08445-670e-4194-bdb1-1e7b79dc00f3"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:20:50 crc kubenswrapper[5052]: I1203 12:20:50.232342 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/99a08445-670e-4194-bdb1-1e7b79dc00f3-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:50 crc kubenswrapper[5052]: I1203 12:20:50.232385 5052 reconciler_common.go:293] "Volume detached for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/99a08445-670e-4194-bdb1-1e7b79dc00f3-etc-machine-id\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:50 crc kubenswrapper[5052]: I1203 12:20:50.232425 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/99a08445-670e-4194-bdb1-1e7b79dc00f3-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:50 crc kubenswrapper[5052]: I1203 12:20:50.232439 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/99a08445-670e-4194-bdb1-1e7b79dc00f3-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:50 crc kubenswrapper[5052]: I1203 12:20:50.232452 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-cs5ln\" (UniqueName: \"kubernetes.io/projected/99a08445-670e-4194-bdb1-1e7b79dc00f3-kube-api-access-cs5ln\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:50 crc kubenswrapper[5052]: I1203 12:20:50.232465 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/99a08445-670e-4194-bdb1-1e7b79dc00f3-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:50 crc kubenswrapper[5052]: I1203 12:20:50.232475 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/99a08445-670e-4194-bdb1-1e7b79dc00f3-config-data-custom\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:50 crc kubenswrapper[5052]: I1203 12:20:50.236014 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/99a08445-670e-4194-bdb1-1e7b79dc00f3-config-data" (OuterVolumeSpecName: "config-data") pod "99a08445-670e-4194-bdb1-1e7b79dc00f3" (UID: "99a08445-670e-4194-bdb1-1e7b79dc00f3"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:20:50 crc kubenswrapper[5052]: I1203 12:20:50.268962 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/99a08445-670e-4194-bdb1-1e7b79dc00f3-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "99a08445-670e-4194-bdb1-1e7b79dc00f3" (UID: "99a08445-670e-4194-bdb1-1e7b79dc00f3"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:20:50 crc kubenswrapper[5052]: I1203 12:20:50.334928 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/99a08445-670e-4194-bdb1-1e7b79dc00f3-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:50 crc kubenswrapper[5052]: I1203 12:20:50.335140 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/99a08445-670e-4194-bdb1-1e7b79dc00f3-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:50 crc kubenswrapper[5052]: I1203 12:20:50.358701 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-db-sync-gzzkp" Dec 03 12:20:50 crc kubenswrapper[5052]: I1203 12:20:50.436166 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/50ae31b2-a1c0-4afb-8e23-d7856c55af1d-logs\") pod \"50ae31b2-a1c0-4afb-8e23-d7856c55af1d\" (UID: \"50ae31b2-a1c0-4afb-8e23-d7856c55af1d\") " Dec 03 12:20:50 crc kubenswrapper[5052]: I1203 12:20:50.436271 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/50ae31b2-a1c0-4afb-8e23-d7856c55af1d-scripts\") pod \"50ae31b2-a1c0-4afb-8e23-d7856c55af1d\" (UID: \"50ae31b2-a1c0-4afb-8e23-d7856c55af1d\") " Dec 03 12:20:50 crc kubenswrapper[5052]: I1203 12:20:50.436351 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r54l6\" (UniqueName: \"kubernetes.io/projected/50ae31b2-a1c0-4afb-8e23-d7856c55af1d-kube-api-access-r54l6\") pod \"50ae31b2-a1c0-4afb-8e23-d7856c55af1d\" (UID: \"50ae31b2-a1c0-4afb-8e23-d7856c55af1d\") " Dec 03 12:20:50 crc kubenswrapper[5052]: I1203 12:20:50.436388 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/50ae31b2-a1c0-4afb-8e23-d7856c55af1d-config-data\") pod \"50ae31b2-a1c0-4afb-8e23-d7856c55af1d\" (UID: \"50ae31b2-a1c0-4afb-8e23-d7856c55af1d\") " Dec 03 12:20:50 crc kubenswrapper[5052]: I1203 12:20:50.436480 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/50ae31b2-a1c0-4afb-8e23-d7856c55af1d-combined-ca-bundle\") pod \"50ae31b2-a1c0-4afb-8e23-d7856c55af1d\" (UID: \"50ae31b2-a1c0-4afb-8e23-d7856c55af1d\") " Dec 03 12:20:50 crc kubenswrapper[5052]: I1203 12:20:50.436707 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/50ae31b2-a1c0-4afb-8e23-d7856c55af1d-logs" (OuterVolumeSpecName: "logs") pod "50ae31b2-a1c0-4afb-8e23-d7856c55af1d" (UID: "50ae31b2-a1c0-4afb-8e23-d7856c55af1d"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:20:50 crc kubenswrapper[5052]: I1203 12:20:50.437090 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/50ae31b2-a1c0-4afb-8e23-d7856c55af1d-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:50 crc kubenswrapper[5052]: I1203 12:20:50.440877 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/50ae31b2-a1c0-4afb-8e23-d7856c55af1d-kube-api-access-r54l6" (OuterVolumeSpecName: "kube-api-access-r54l6") pod "50ae31b2-a1c0-4afb-8e23-d7856c55af1d" (UID: "50ae31b2-a1c0-4afb-8e23-d7856c55af1d"). InnerVolumeSpecName "kube-api-access-r54l6". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:20:50 crc kubenswrapper[5052]: I1203 12:20:50.441148 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/50ae31b2-a1c0-4afb-8e23-d7856c55af1d-scripts" (OuterVolumeSpecName: "scripts") pod "50ae31b2-a1c0-4afb-8e23-d7856c55af1d" (UID: "50ae31b2-a1c0-4afb-8e23-d7856c55af1d"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:20:50 crc kubenswrapper[5052]: I1203 12:20:50.464728 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/50ae31b2-a1c0-4afb-8e23-d7856c55af1d-config-data" (OuterVolumeSpecName: "config-data") pod "50ae31b2-a1c0-4afb-8e23-d7856c55af1d" (UID: "50ae31b2-a1c0-4afb-8e23-d7856c55af1d"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:20:50 crc kubenswrapper[5052]: I1203 12:20:50.482036 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/50ae31b2-a1c0-4afb-8e23-d7856c55af1d-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "50ae31b2-a1c0-4afb-8e23-d7856c55af1d" (UID: "50ae31b2-a1c0-4afb-8e23-d7856c55af1d"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:20:50 crc kubenswrapper[5052]: I1203 12:20:50.539104 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/50ae31b2-a1c0-4afb-8e23-d7856c55af1d-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:50 crc kubenswrapper[5052]: I1203 12:20:50.539302 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/50ae31b2-a1c0-4afb-8e23-d7856c55af1d-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:50 crc kubenswrapper[5052]: I1203 12:20:50.539355 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-r54l6\" (UniqueName: \"kubernetes.io/projected/50ae31b2-a1c0-4afb-8e23-d7856c55af1d-kube-api-access-r54l6\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:50 crc kubenswrapper[5052]: I1203 12:20:50.539405 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/50ae31b2-a1c0-4afb-8e23-d7856c55af1d-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:50 crc kubenswrapper[5052]: I1203 12:20:50.916163 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-db-sync-gzzkp" event={"ID":"50ae31b2-a1c0-4afb-8e23-d7856c55af1d","Type":"ContainerDied","Data":"dabc6c08dd8bb953a7aa6c3562c635fe665e3ba4b101b7c62fb03488e221a8b4"} Dec 03 12:20:50 crc kubenswrapper[5052]: I1203 12:20:50.916222 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="dabc6c08dd8bb953a7aa6c3562c635fe665e3ba4b101b7c62fb03488e221a8b4" Dec 03 12:20:50 crc kubenswrapper[5052]: I1203 12:20:50.916183 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-db-sync-gzzkp" Dec 03 12:20:50 crc kubenswrapper[5052]: I1203 12:20:50.918325 5052 generic.go:334] "Generic (PLEG): container finished" podID="b852a7d7-8510-440d-8e87-0755fac2a324" containerID="065aebd7d9dcbe855a40abb9f1fcec166d09f5a018bf0567b9d6ef02a6577350" exitCode=0 Dec 03 12:20:50 crc kubenswrapper[5052]: I1203 12:20:50.918388 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-db-sync-9jf9q" event={"ID":"b852a7d7-8510-440d-8e87-0755fac2a324","Type":"ContainerDied","Data":"065aebd7d9dcbe855a40abb9f1fcec166d09f5a018bf0567b9d6ef02a6577350"} Dec 03 12:20:50 crc kubenswrapper[5052]: I1203 12:20:50.922100 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"99a08445-670e-4194-bdb1-1e7b79dc00f3","Type":"ContainerDied","Data":"e21752e5b4419159e2a808d3724582e2d6ba1c284136034ea5acf8e21a04d757"} Dec 03 12:20:50 crc kubenswrapper[5052]: I1203 12:20:50.922142 5052 scope.go:117] "RemoveContainer" containerID="e09e55a6c45ce4ed5ad18b47c220ac1df2d781056bacefff348ec37669d05115" Dec 03 12:20:50 crc kubenswrapper[5052]: I1203 12:20:50.922253 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:20:50 crc kubenswrapper[5052]: I1203 12:20:50.939537 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"2d94bf00-7a64-4e90-a48c-b206ec54da6d","Type":"ContainerStarted","Data":"e0441d5665d194615504d1f41084afce1caa468996e04b063d890632f490acca"} Dec 03 12:20:51 crc kubenswrapper[5052]: I1203 12:20:51.025809 5052 scope.go:117] "RemoveContainer" containerID="c5c1aee8f74d3c51ad667b122373af80c11a540d038db6ff4ad7b44a5a8c4f6e" Dec 03 12:20:51 crc kubenswrapper[5052]: I1203 12:20:51.081462 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/placement-5447868864-mfdcg"] Dec 03 12:20:51 crc kubenswrapper[5052]: E1203 12:20:51.081928 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="50ae31b2-a1c0-4afb-8e23-d7856c55af1d" containerName="placement-db-sync" Dec 03 12:20:51 crc kubenswrapper[5052]: I1203 12:20:51.081951 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="50ae31b2-a1c0-4afb-8e23-d7856c55af1d" containerName="placement-db-sync" Dec 03 12:20:51 crc kubenswrapper[5052]: E1203 12:20:51.081968 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="99a08445-670e-4194-bdb1-1e7b79dc00f3" containerName="cinder-api-log" Dec 03 12:20:51 crc kubenswrapper[5052]: I1203 12:20:51.081974 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="99a08445-670e-4194-bdb1-1e7b79dc00f3" containerName="cinder-api-log" Dec 03 12:20:51 crc kubenswrapper[5052]: E1203 12:20:51.081999 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="99a08445-670e-4194-bdb1-1e7b79dc00f3" containerName="cinder-api" Dec 03 12:20:51 crc kubenswrapper[5052]: I1203 12:20:51.082005 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="99a08445-670e-4194-bdb1-1e7b79dc00f3" containerName="cinder-api" Dec 03 12:20:51 crc kubenswrapper[5052]: E1203 12:20:51.082019 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="9ea78435-8fd9-4321-9e48-3aaae833dcf2" containerName="nova-cell0-conductor-db-sync" Dec 03 12:20:51 crc kubenswrapper[5052]: I1203 12:20:51.082024 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ea78435-8fd9-4321-9e48-3aaae833dcf2" containerName="nova-cell0-conductor-db-sync" Dec 03 12:20:51 crc kubenswrapper[5052]: I1203 12:20:51.082192 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="99a08445-670e-4194-bdb1-1e7b79dc00f3" containerName="cinder-api-log" Dec 03 12:20:51 crc kubenswrapper[5052]: I1203 12:20:51.082202 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="9ea78435-8fd9-4321-9e48-3aaae833dcf2" containerName="nova-cell0-conductor-db-sync" Dec 03 12:20:51 crc kubenswrapper[5052]: I1203 12:20:51.082213 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="50ae31b2-a1c0-4afb-8e23-d7856c55af1d" containerName="placement-db-sync" Dec 03 12:20:51 crc kubenswrapper[5052]: I1203 12:20:51.082233 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="99a08445-670e-4194-bdb1-1e7b79dc00f3" containerName="cinder-api" Dec 03 12:20:51 crc kubenswrapper[5052]: I1203 12:20:51.083376 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-5447868864-mfdcg" Dec 03 12:20:51 crc kubenswrapper[5052]: I1203 12:20:51.093189 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/placement-5447868864-mfdcg"] Dec 03 12:20:51 crc kubenswrapper[5052]: I1203 12:20:51.115638 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 12:20:51 crc kubenswrapper[5052]: I1203 12:20:51.127488 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 12:20:51 crc kubenswrapper[5052]: I1203 12:20:51.150270 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 12:20:51 crc kubenswrapper[5052]: I1203 12:20:51.152294 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:20:51 crc kubenswrapper[5052]: I1203 12:20:51.155762 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8f26f\" (UniqueName: \"kubernetes.io/projected/90814ea7-1abb-4760-bbfa-1190f23ab38a-kube-api-access-8f26f\") pod \"placement-5447868864-mfdcg\" (UID: \"90814ea7-1abb-4760-bbfa-1190f23ab38a\") " pod="openstack-kuttl-tests/placement-5447868864-mfdcg" Dec 03 12:20:51 crc kubenswrapper[5052]: I1203 12:20:51.155842 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/90814ea7-1abb-4760-bbfa-1190f23ab38a-config-data\") pod \"placement-5447868864-mfdcg\" (UID: \"90814ea7-1abb-4760-bbfa-1190f23ab38a\") " pod="openstack-kuttl-tests/placement-5447868864-mfdcg" Dec 03 12:20:51 crc kubenswrapper[5052]: I1203 12:20:51.155913 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/90814ea7-1abb-4760-bbfa-1190f23ab38a-internal-tls-certs\") pod \"placement-5447868864-mfdcg\" (UID: \"90814ea7-1abb-4760-bbfa-1190f23ab38a\") " pod="openstack-kuttl-tests/placement-5447868864-mfdcg" Dec 03 12:20:51 crc kubenswrapper[5052]: I1203 12:20:51.155941 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/90814ea7-1abb-4760-bbfa-1190f23ab38a-logs\") pod \"placement-5447868864-mfdcg\" (UID: \"90814ea7-1abb-4760-bbfa-1190f23ab38a\") " pod="openstack-kuttl-tests/placement-5447868864-mfdcg" Dec 03 12:20:51 crc kubenswrapper[5052]: I1203 12:20:51.155960 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/90814ea7-1abb-4760-bbfa-1190f23ab38a-public-tls-certs\") pod \"placement-5447868864-mfdcg\" (UID: \"90814ea7-1abb-4760-bbfa-1190f23ab38a\") " pod="openstack-kuttl-tests/placement-5447868864-mfdcg" Dec 03 12:20:51 crc kubenswrapper[5052]: I1203 12:20:51.155979 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/90814ea7-1abb-4760-bbfa-1190f23ab38a-combined-ca-bundle\") pod \"placement-5447868864-mfdcg\" (UID: \"90814ea7-1abb-4760-bbfa-1190f23ab38a\") " pod="openstack-kuttl-tests/placement-5447868864-mfdcg" Dec 03 12:20:51 crc kubenswrapper[5052]: I1203 12:20:51.156019 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/90814ea7-1abb-4760-bbfa-1190f23ab38a-scripts\") pod \"placement-5447868864-mfdcg\" (UID: \"90814ea7-1abb-4760-bbfa-1190f23ab38a\") " pod="openstack-kuttl-tests/placement-5447868864-mfdcg" Dec 03 12:20:51 crc kubenswrapper[5052]: I1203 12:20:51.156241 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cinder-api-config-data" Dec 03 12:20:51 crc kubenswrapper[5052]: I1203 12:20:51.156536 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-cinder-internal-svc" Dec 03 12:20:51 crc kubenswrapper[5052]: I1203 12:20:51.157114 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-cinder-public-svc" Dec 03 12:20:51 crc kubenswrapper[5052]: I1203 12:20:51.161349 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 12:20:51 crc kubenswrapper[5052]: I1203 12:20:51.192161 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:20:51 crc kubenswrapper[5052]: I1203 12:20:51.193411 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:20:51 crc kubenswrapper[5052]: I1203 12:20:51.199951 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:20:51 crc kubenswrapper[5052]: I1203 12:20:51.200469 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:20:51 crc kubenswrapper[5052]: I1203 12:20:51.257619 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/90814ea7-1abb-4760-bbfa-1190f23ab38a-logs\") pod \"placement-5447868864-mfdcg\" (UID: \"90814ea7-1abb-4760-bbfa-1190f23ab38a\") " pod="openstack-kuttl-tests/placement-5447868864-mfdcg" Dec 03 12:20:51 crc kubenswrapper[5052]: I1203 12:20:51.257888 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/90814ea7-1abb-4760-bbfa-1190f23ab38a-public-tls-certs\") pod \"placement-5447868864-mfdcg\" (UID: \"90814ea7-1abb-4760-bbfa-1190f23ab38a\") " pod="openstack-kuttl-tests/placement-5447868864-mfdcg" Dec 03 12:20:51 crc kubenswrapper[5052]: I1203 12:20:51.257909 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/90814ea7-1abb-4760-bbfa-1190f23ab38a-combined-ca-bundle\") pod \"placement-5447868864-mfdcg\" (UID: \"90814ea7-1abb-4760-bbfa-1190f23ab38a\") " pod="openstack-kuttl-tests/placement-5447868864-mfdcg" Dec 03 12:20:51 crc kubenswrapper[5052]: I1203 12:20:51.257937 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/bb41ed7e-c96b-44dd-8c4e-ffb702de3f87-config-data-custom\") pod \"cinder-api-0\" (UID: \"bb41ed7e-c96b-44dd-8c4e-ffb702de3f87\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:20:51 crc kubenswrapper[5052]: I1203 12:20:51.257970 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/bb41ed7e-c96b-44dd-8c4e-ffb702de3f87-public-tls-certs\") pod \"cinder-api-0\" (UID: \"bb41ed7e-c96b-44dd-8c4e-ffb702de3f87\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:20:51 crc kubenswrapper[5052]: I1203 12:20:51.257989 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/bb41ed7e-c96b-44dd-8c4e-ffb702de3f87-combined-ca-bundle\") pod \"cinder-api-0\" (UID: \"bb41ed7e-c96b-44dd-8c4e-ffb702de3f87\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:20:51 crc kubenswrapper[5052]: I1203 12:20:51.258006 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/90814ea7-1abb-4760-bbfa-1190f23ab38a-scripts\") pod \"placement-5447868864-mfdcg\" (UID: \"90814ea7-1abb-4760-bbfa-1190f23ab38a\") " pod="openstack-kuttl-tests/placement-5447868864-mfdcg" Dec 03 12:20:51 crc kubenswrapper[5052]: I1203 12:20:51.258045 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/bb41ed7e-c96b-44dd-8c4e-ffb702de3f87-etc-machine-id\") pod \"cinder-api-0\" (UID: \"bb41ed7e-c96b-44dd-8c4e-ffb702de3f87\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:20:51 crc kubenswrapper[5052]: I1203 12:20:51.258070 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-8f26f\" (UniqueName: \"kubernetes.io/projected/90814ea7-1abb-4760-bbfa-1190f23ab38a-kube-api-access-8f26f\") pod \"placement-5447868864-mfdcg\" (UID: \"90814ea7-1abb-4760-bbfa-1190f23ab38a\") " pod="openstack-kuttl-tests/placement-5447868864-mfdcg" Dec 03 12:20:51 crc kubenswrapper[5052]: I1203 12:20:51.258101 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/bb41ed7e-c96b-44dd-8c4e-ffb702de3f87-logs\") pod \"cinder-api-0\" (UID: \"bb41ed7e-c96b-44dd-8c4e-ffb702de3f87\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:20:51 crc kubenswrapper[5052]: I1203 12:20:51.258134 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kjtr9\" (UniqueName: \"kubernetes.io/projected/bb41ed7e-c96b-44dd-8c4e-ffb702de3f87-kube-api-access-kjtr9\") pod \"cinder-api-0\" (UID: \"bb41ed7e-c96b-44dd-8c4e-ffb702de3f87\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:20:51 crc kubenswrapper[5052]: I1203 12:20:51.258150 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/90814ea7-1abb-4760-bbfa-1190f23ab38a-logs\") pod \"placement-5447868864-mfdcg\" (UID: \"90814ea7-1abb-4760-bbfa-1190f23ab38a\") " pod="openstack-kuttl-tests/placement-5447868864-mfdcg" Dec 03 12:20:51 crc kubenswrapper[5052]: I1203 12:20:51.258175 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/90814ea7-1abb-4760-bbfa-1190f23ab38a-config-data\") pod \"placement-5447868864-mfdcg\" (UID: \"90814ea7-1abb-4760-bbfa-1190f23ab38a\") " pod="openstack-kuttl-tests/placement-5447868864-mfdcg" Dec 03 12:20:51 crc kubenswrapper[5052]: I1203 12:20:51.258198 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/bb41ed7e-c96b-44dd-8c4e-ffb702de3f87-internal-tls-certs\") pod \"cinder-api-0\" (UID: \"bb41ed7e-c96b-44dd-8c4e-ffb702de3f87\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:20:51 crc kubenswrapper[5052]: I1203 12:20:51.258229 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/bb41ed7e-c96b-44dd-8c4e-ffb702de3f87-config-data\") pod \"cinder-api-0\" (UID: \"bb41ed7e-c96b-44dd-8c4e-ffb702de3f87\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:20:51 crc kubenswrapper[5052]: I1203 12:20:51.258264 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/bb41ed7e-c96b-44dd-8c4e-ffb702de3f87-scripts\") pod \"cinder-api-0\" (UID: \"bb41ed7e-c96b-44dd-8c4e-ffb702de3f87\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:20:51 crc kubenswrapper[5052]: I1203 12:20:51.258305 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/90814ea7-1abb-4760-bbfa-1190f23ab38a-internal-tls-certs\") pod \"placement-5447868864-mfdcg\" (UID: \"90814ea7-1abb-4760-bbfa-1190f23ab38a\") " pod="openstack-kuttl-tests/placement-5447868864-mfdcg" Dec 03 12:20:51 crc kubenswrapper[5052]: I1203 12:20:51.262499 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/90814ea7-1abb-4760-bbfa-1190f23ab38a-combined-ca-bundle\") pod \"placement-5447868864-mfdcg\" (UID: \"90814ea7-1abb-4760-bbfa-1190f23ab38a\") " pod="openstack-kuttl-tests/placement-5447868864-mfdcg" Dec 03 12:20:51 crc kubenswrapper[5052]: I1203 12:20:51.262902 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/90814ea7-1abb-4760-bbfa-1190f23ab38a-public-tls-certs\") pod \"placement-5447868864-mfdcg\" (UID: \"90814ea7-1abb-4760-bbfa-1190f23ab38a\") " pod="openstack-kuttl-tests/placement-5447868864-mfdcg" Dec 03 12:20:51 crc kubenswrapper[5052]: I1203 12:20:51.263285 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/90814ea7-1abb-4760-bbfa-1190f23ab38a-scripts\") pod \"placement-5447868864-mfdcg\" (UID: \"90814ea7-1abb-4760-bbfa-1190f23ab38a\") " pod="openstack-kuttl-tests/placement-5447868864-mfdcg" Dec 03 12:20:51 crc kubenswrapper[5052]: I1203 12:20:51.264360 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/90814ea7-1abb-4760-bbfa-1190f23ab38a-config-data\") pod \"placement-5447868864-mfdcg\" (UID: \"90814ea7-1abb-4760-bbfa-1190f23ab38a\") " pod="openstack-kuttl-tests/placement-5447868864-mfdcg" Dec 03 12:20:51 crc kubenswrapper[5052]: I1203 12:20:51.270419 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/90814ea7-1abb-4760-bbfa-1190f23ab38a-internal-tls-certs\") pod \"placement-5447868864-mfdcg\" (UID: \"90814ea7-1abb-4760-bbfa-1190f23ab38a\") " pod="openstack-kuttl-tests/placement-5447868864-mfdcg" Dec 03 12:20:51 crc kubenswrapper[5052]: I1203 12:20:51.275953 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-8f26f\" (UniqueName: \"kubernetes.io/projected/90814ea7-1abb-4760-bbfa-1190f23ab38a-kube-api-access-8f26f\") pod \"placement-5447868864-mfdcg\" (UID: \"90814ea7-1abb-4760-bbfa-1190f23ab38a\") " pod="openstack-kuttl-tests/placement-5447868864-mfdcg" Dec 03 12:20:51 crc kubenswrapper[5052]: I1203 12:20:51.359766 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/bb41ed7e-c96b-44dd-8c4e-ffb702de3f87-combined-ca-bundle\") pod \"cinder-api-0\" (UID: \"bb41ed7e-c96b-44dd-8c4e-ffb702de3f87\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:20:51 crc kubenswrapper[5052]: I1203 12:20:51.359826 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/bb41ed7e-c96b-44dd-8c4e-ffb702de3f87-etc-machine-id\") pod \"cinder-api-0\" (UID: \"bb41ed7e-c96b-44dd-8c4e-ffb702de3f87\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:20:51 crc kubenswrapper[5052]: I1203 12:20:51.359864 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/bb41ed7e-c96b-44dd-8c4e-ffb702de3f87-logs\") pod \"cinder-api-0\" (UID: \"bb41ed7e-c96b-44dd-8c4e-ffb702de3f87\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:20:51 crc kubenswrapper[5052]: I1203 12:20:51.359901 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-kjtr9\" (UniqueName: \"kubernetes.io/projected/bb41ed7e-c96b-44dd-8c4e-ffb702de3f87-kube-api-access-kjtr9\") pod \"cinder-api-0\" (UID: \"bb41ed7e-c96b-44dd-8c4e-ffb702de3f87\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:20:51 crc kubenswrapper[5052]: I1203 12:20:51.359937 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/bb41ed7e-c96b-44dd-8c4e-ffb702de3f87-internal-tls-certs\") pod \"cinder-api-0\" (UID: \"bb41ed7e-c96b-44dd-8c4e-ffb702de3f87\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:20:51 crc kubenswrapper[5052]: I1203 12:20:51.359968 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/bb41ed7e-c96b-44dd-8c4e-ffb702de3f87-config-data\") pod \"cinder-api-0\" (UID: \"bb41ed7e-c96b-44dd-8c4e-ffb702de3f87\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:20:51 crc kubenswrapper[5052]: I1203 12:20:51.359994 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/bb41ed7e-c96b-44dd-8c4e-ffb702de3f87-scripts\") pod \"cinder-api-0\" (UID: \"bb41ed7e-c96b-44dd-8c4e-ffb702de3f87\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:20:51 crc kubenswrapper[5052]: I1203 12:20:51.360048 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/bb41ed7e-c96b-44dd-8c4e-ffb702de3f87-config-data-custom\") pod \"cinder-api-0\" (UID: \"bb41ed7e-c96b-44dd-8c4e-ffb702de3f87\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:20:51 crc kubenswrapper[5052]: I1203 12:20:51.360073 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/bb41ed7e-c96b-44dd-8c4e-ffb702de3f87-public-tls-certs\") pod \"cinder-api-0\" (UID: \"bb41ed7e-c96b-44dd-8c4e-ffb702de3f87\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:20:51 crc kubenswrapper[5052]: I1203 12:20:51.360607 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/bb41ed7e-c96b-44dd-8c4e-ffb702de3f87-etc-machine-id\") pod \"cinder-api-0\" (UID: \"bb41ed7e-c96b-44dd-8c4e-ffb702de3f87\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:20:51 crc kubenswrapper[5052]: I1203 12:20:51.360998 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/bb41ed7e-c96b-44dd-8c4e-ffb702de3f87-logs\") pod \"cinder-api-0\" (UID: \"bb41ed7e-c96b-44dd-8c4e-ffb702de3f87\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:20:51 crc kubenswrapper[5052]: I1203 12:20:51.363801 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/bb41ed7e-c96b-44dd-8c4e-ffb702de3f87-combined-ca-bundle\") pod \"cinder-api-0\" (UID: \"bb41ed7e-c96b-44dd-8c4e-ffb702de3f87\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:20:51 crc kubenswrapper[5052]: I1203 12:20:51.364302 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/bb41ed7e-c96b-44dd-8c4e-ffb702de3f87-public-tls-certs\") pod \"cinder-api-0\" (UID: \"bb41ed7e-c96b-44dd-8c4e-ffb702de3f87\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:20:51 crc kubenswrapper[5052]: I1203 12:20:51.365199 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/bb41ed7e-c96b-44dd-8c4e-ffb702de3f87-scripts\") pod \"cinder-api-0\" (UID: \"bb41ed7e-c96b-44dd-8c4e-ffb702de3f87\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:20:51 crc kubenswrapper[5052]: I1203 12:20:51.372391 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/bb41ed7e-c96b-44dd-8c4e-ffb702de3f87-internal-tls-certs\") pod \"cinder-api-0\" (UID: \"bb41ed7e-c96b-44dd-8c4e-ffb702de3f87\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:20:51 crc kubenswrapper[5052]: I1203 12:20:51.372402 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/bb41ed7e-c96b-44dd-8c4e-ffb702de3f87-config-data-custom\") pod \"cinder-api-0\" (UID: \"bb41ed7e-c96b-44dd-8c4e-ffb702de3f87\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:20:51 crc kubenswrapper[5052]: I1203 12:20:51.373601 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/bb41ed7e-c96b-44dd-8c4e-ffb702de3f87-config-data\") pod \"cinder-api-0\" (UID: \"bb41ed7e-c96b-44dd-8c4e-ffb702de3f87\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:20:51 crc kubenswrapper[5052]: I1203 12:20:51.379368 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-kjtr9\" (UniqueName: \"kubernetes.io/projected/bb41ed7e-c96b-44dd-8c4e-ffb702de3f87-kube-api-access-kjtr9\") pod \"cinder-api-0\" (UID: \"bb41ed7e-c96b-44dd-8c4e-ffb702de3f87\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:20:51 crc kubenswrapper[5052]: I1203 12:20:51.408287 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-5447868864-mfdcg" Dec 03 12:20:51 crc kubenswrapper[5052]: I1203 12:20:51.472115 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:20:51 crc kubenswrapper[5052]: I1203 12:20:51.687566 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="99a08445-670e-4194-bdb1-1e7b79dc00f3" path="/var/lib/kubelet/pods/99a08445-670e-4194-bdb1-1e7b79dc00f3/volumes" Dec 03 12:20:51 crc kubenswrapper[5052]: I1203 12:20:51.926657 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/placement-5447868864-mfdcg"] Dec 03 12:20:51 crc kubenswrapper[5052]: I1203 12:20:51.957949 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-5447868864-mfdcg" event={"ID":"90814ea7-1abb-4760-bbfa-1190f23ab38a","Type":"ContainerStarted","Data":"be3444a0d99f7a4b2d30e89d0c25a76e9126e091283a312597b11200cc324aa1"} Dec 03 12:20:51 crc kubenswrapper[5052]: I1203 12:20:51.959930 5052 generic.go:334] "Generic (PLEG): container finished" podID="e80169f9-5480-4318-8ab2-1af4394fe76a" containerID="a96261519a88986377772f2eba818435423e35e4613b257b006f4e1d712a712b" exitCode=0 Dec 03 12:20:51 crc kubenswrapper[5052]: I1203 12:20:51.959980 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-conductor-0" event={"ID":"e80169f9-5480-4318-8ab2-1af4394fe76a","Type":"ContainerDied","Data":"a96261519a88986377772f2eba818435423e35e4613b257b006f4e1d712a712b"} Dec 03 12:20:51 crc kubenswrapper[5052]: I1203 12:20:51.961053 5052 generic.go:334] "Generic (PLEG): container finished" podID="7a9497b1-89e2-4ebc-97e6-2bdaeefab95f" containerID="d8891df0b213dbaf1fa63a8450de0cdab0ef7ec799eb7a4b97714ab88cffe2ab" exitCode=0 Dec 03 12:20:51 crc kubenswrapper[5052]: I1203 12:20:51.961203 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-db-sync-92z8b" event={"ID":"7a9497b1-89e2-4ebc-97e6-2bdaeefab95f","Type":"ContainerDied","Data":"d8891df0b213dbaf1fa63a8450de0cdab0ef7ec799eb7a4b97714ab88cffe2ab"} Dec 03 12:20:51 crc kubenswrapper[5052]: I1203 12:20:51.974763 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:20:51 crc kubenswrapper[5052]: I1203 12:20:51.974822 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:20:52 crc kubenswrapper[5052]: I1203 12:20:52.037649 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 12:20:52 crc kubenswrapper[5052]: E1203 12:20:52.496204 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="4c12bd41f4c701be825345e793ba798c2e3b6ecd4b951f8fb1ee1579f6953903" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:20:52 crc kubenswrapper[5052]: E1203 12:20:52.498705 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="4c12bd41f4c701be825345e793ba798c2e3b6ecd4b951f8fb1ee1579f6953903" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:20:52 crc kubenswrapper[5052]: E1203 12:20:52.500808 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="4c12bd41f4c701be825345e793ba798c2e3b6ecd4b951f8fb1ee1579f6953903" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:20:52 crc kubenswrapper[5052]: E1203 12:20:52.500840 5052 prober.go:104] "Probe errored" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" probeType="Readiness" pod="openstack-kuttl-tests/nova-cell1-conductor-0" podUID="aa2b4c07-e5c7-4543-a0a5-f15b8e1b7f83" containerName="nova-cell1-conductor-conductor" Dec 03 12:20:52 crc kubenswrapper[5052]: I1203 12:20:52.508001 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-db-sync-9jf9q" Dec 03 12:20:52 crc kubenswrapper[5052]: I1203 12:20:52.599150 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6pcrx\" (UniqueName: \"kubernetes.io/projected/b852a7d7-8510-440d-8e87-0755fac2a324-kube-api-access-6pcrx\") pod \"b852a7d7-8510-440d-8e87-0755fac2a324\" (UID: \"b852a7d7-8510-440d-8e87-0755fac2a324\") " Dec 03 12:20:52 crc kubenswrapper[5052]: I1203 12:20:52.599258 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/b852a7d7-8510-440d-8e87-0755fac2a324-config-data\") pod \"b852a7d7-8510-440d-8e87-0755fac2a324\" (UID: \"b852a7d7-8510-440d-8e87-0755fac2a324\") " Dec 03 12:20:52 crc kubenswrapper[5052]: I1203 12:20:52.599453 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b852a7d7-8510-440d-8e87-0755fac2a324-combined-ca-bundle\") pod \"b852a7d7-8510-440d-8e87-0755fac2a324\" (UID: \"b852a7d7-8510-440d-8e87-0755fac2a324\") " Dec 03 12:20:52 crc kubenswrapper[5052]: I1203 12:20:52.603042 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b852a7d7-8510-440d-8e87-0755fac2a324-kube-api-access-6pcrx" (OuterVolumeSpecName: "kube-api-access-6pcrx") pod "b852a7d7-8510-440d-8e87-0755fac2a324" (UID: "b852a7d7-8510-440d-8e87-0755fac2a324"). InnerVolumeSpecName "kube-api-access-6pcrx". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:20:52 crc kubenswrapper[5052]: I1203 12:20:52.631846 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b852a7d7-8510-440d-8e87-0755fac2a324-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "b852a7d7-8510-440d-8e87-0755fac2a324" (UID: "b852a7d7-8510-440d-8e87-0755fac2a324"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:20:52 crc kubenswrapper[5052]: I1203 12:20:52.632029 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:20:52 crc kubenswrapper[5052]: I1203 12:20:52.676890 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b852a7d7-8510-440d-8e87-0755fac2a324-config-data" (OuterVolumeSpecName: "config-data") pod "b852a7d7-8510-440d-8e87-0755fac2a324" (UID: "b852a7d7-8510-440d-8e87-0755fac2a324"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:20:52 crc kubenswrapper[5052]: I1203 12:20:52.702554 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tsh5z\" (UniqueName: \"kubernetes.io/projected/e80169f9-5480-4318-8ab2-1af4394fe76a-kube-api-access-tsh5z\") pod \"e80169f9-5480-4318-8ab2-1af4394fe76a\" (UID: \"e80169f9-5480-4318-8ab2-1af4394fe76a\") " Dec 03 12:20:52 crc kubenswrapper[5052]: I1203 12:20:52.702695 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/e80169f9-5480-4318-8ab2-1af4394fe76a-config-data\") pod \"e80169f9-5480-4318-8ab2-1af4394fe76a\" (UID: \"e80169f9-5480-4318-8ab2-1af4394fe76a\") " Dec 03 12:20:52 crc kubenswrapper[5052]: I1203 12:20:52.702730 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e80169f9-5480-4318-8ab2-1af4394fe76a-combined-ca-bundle\") pod \"e80169f9-5480-4318-8ab2-1af4394fe76a\" (UID: \"e80169f9-5480-4318-8ab2-1af4394fe76a\") " Dec 03 12:20:52 crc kubenswrapper[5052]: I1203 12:20:52.703299 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/b852a7d7-8510-440d-8e87-0755fac2a324-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:52 crc kubenswrapper[5052]: I1203 12:20:52.703316 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b852a7d7-8510-440d-8e87-0755fac2a324-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:52 crc kubenswrapper[5052]: I1203 12:20:52.703330 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-6pcrx\" (UniqueName: \"kubernetes.io/projected/b852a7d7-8510-440d-8e87-0755fac2a324-kube-api-access-6pcrx\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:52 crc kubenswrapper[5052]: I1203 12:20:52.712704 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e80169f9-5480-4318-8ab2-1af4394fe76a-kube-api-access-tsh5z" (OuterVolumeSpecName: "kube-api-access-tsh5z") pod "e80169f9-5480-4318-8ab2-1af4394fe76a" (UID: "e80169f9-5480-4318-8ab2-1af4394fe76a"). InnerVolumeSpecName "kube-api-access-tsh5z". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:20:52 crc kubenswrapper[5052]: I1203 12:20:52.744628 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e80169f9-5480-4318-8ab2-1af4394fe76a-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "e80169f9-5480-4318-8ab2-1af4394fe76a" (UID: "e80169f9-5480-4318-8ab2-1af4394fe76a"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:20:52 crc kubenswrapper[5052]: I1203 12:20:52.744712 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e80169f9-5480-4318-8ab2-1af4394fe76a-config-data" (OuterVolumeSpecName: "config-data") pod "e80169f9-5480-4318-8ab2-1af4394fe76a" (UID: "e80169f9-5480-4318-8ab2-1af4394fe76a"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:20:52 crc kubenswrapper[5052]: I1203 12:20:52.804785 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-tsh5z\" (UniqueName: \"kubernetes.io/projected/e80169f9-5480-4318-8ab2-1af4394fe76a-kube-api-access-tsh5z\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:52 crc kubenswrapper[5052]: I1203 12:20:52.804815 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/e80169f9-5480-4318-8ab2-1af4394fe76a-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:52 crc kubenswrapper[5052]: I1203 12:20:52.804827 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e80169f9-5480-4318-8ab2-1af4394fe76a-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:52 crc kubenswrapper[5052]: I1203 12:20:52.993598 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-5447868864-mfdcg" event={"ID":"90814ea7-1abb-4760-bbfa-1190f23ab38a","Type":"ContainerStarted","Data":"0c61528c202083e822a6d60926f5531661737d72572cc3938c0bb3db2f2fcd35"} Dec 03 12:20:52 crc kubenswrapper[5052]: I1203 12:20:52.994774 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/placement-5447868864-mfdcg" Dec 03 12:20:52 crc kubenswrapper[5052]: I1203 12:20:52.994818 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/placement-5447868864-mfdcg" Dec 03 12:20:52 crc kubenswrapper[5052]: I1203 12:20:52.994862 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-5447868864-mfdcg" event={"ID":"90814ea7-1abb-4760-bbfa-1190f23ab38a","Type":"ContainerStarted","Data":"f82b984cb5939c510f6703add68625e1e534cdf59488ddc53cb91e355ac65330"} Dec 03 12:20:52 crc kubenswrapper[5052]: I1203 12:20:52.996531 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"bb41ed7e-c96b-44dd-8c4e-ffb702de3f87","Type":"ContainerStarted","Data":"19ad1a127fb47796fec06e903f60d68c562fce37f707be6c98c93f3e0d2d25cd"} Dec 03 12:20:52 crc kubenswrapper[5052]: I1203 12:20:52.996556 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"bb41ed7e-c96b-44dd-8c4e-ffb702de3f87","Type":"ContainerStarted","Data":"e685a7366b76c078153cbfe61a3a3a16b3be6c0dffdc41b2a7513817c10b1140"} Dec 03 12:20:53 crc kubenswrapper[5052]: I1203 12:20:52.998750 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-db-sync-9jf9q" event={"ID":"b852a7d7-8510-440d-8e87-0755fac2a324","Type":"ContainerDied","Data":"2b94be6b43835c8d9230bf06afb4a758df81f4bb7a901387bd220d7526304993"} Dec 03 12:20:53 crc kubenswrapper[5052]: I1203 12:20:52.998770 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2b94be6b43835c8d9230bf06afb4a758df81f4bb7a901387bd220d7526304993" Dec 03 12:20:53 crc kubenswrapper[5052]: I1203 12:20:52.998817 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-db-sync-9jf9q" Dec 03 12:20:53 crc kubenswrapper[5052]: I1203 12:20:53.010236 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"2d94bf00-7a64-4e90-a48c-b206ec54da6d","Type":"ContainerStarted","Data":"7a966caf1545e10cd03293bfda99f2c9475e9dcbe187dcc16f09580784759143"} Dec 03 12:20:53 crc kubenswrapper[5052]: I1203 12:20:53.012532 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-conductor-0" event={"ID":"e80169f9-5480-4318-8ab2-1af4394fe76a","Type":"ContainerDied","Data":"bc5d8b8512b0c0b501fde049ef6154dff4775672351180bc35390815b2d7c8a2"} Dec 03 12:20:53 crc kubenswrapper[5052]: I1203 12:20:53.012582 5052 scope.go:117] "RemoveContainer" containerID="a96261519a88986377772f2eba818435423e35e4613b257b006f4e1d712a712b" Dec 03 12:20:53 crc kubenswrapper[5052]: I1203 12:20:53.012832 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:20:53 crc kubenswrapper[5052]: I1203 12:20:53.025629 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/placement-5447868864-mfdcg" podStartSLOduration=3.025607689 podStartE2EDuration="3.025607689s" podCreationTimestamp="2025-12-03 12:20:50 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:20:53.02073647 +0000 UTC m=+3643.608978234" watchObservedRunningTime="2025-12-03 12:20:53.025607689 +0000 UTC m=+3643.613849453" Dec 03 12:20:53 crc kubenswrapper[5052]: I1203 12:20:53.136938 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-0"] Dec 03 12:20:53 crc kubenswrapper[5052]: I1203 12:20:53.187872 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-0"] Dec 03 12:20:53 crc kubenswrapper[5052]: I1203 12:20:53.282595 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-0"] Dec 03 12:20:53 crc kubenswrapper[5052]: E1203 12:20:53.283170 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="b852a7d7-8510-440d-8e87-0755fac2a324" containerName="keystone-db-sync" Dec 03 12:20:53 crc kubenswrapper[5052]: I1203 12:20:53.283192 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="b852a7d7-8510-440d-8e87-0755fac2a324" containerName="keystone-db-sync" Dec 03 12:20:53 crc kubenswrapper[5052]: E1203 12:20:53.283221 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="e80169f9-5480-4318-8ab2-1af4394fe76a" containerName="nova-cell0-conductor-conductor" Dec 03 12:20:53 crc kubenswrapper[5052]: I1203 12:20:53.283230 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="e80169f9-5480-4318-8ab2-1af4394fe76a" containerName="nova-cell0-conductor-conductor" Dec 03 12:20:53 crc kubenswrapper[5052]: I1203 12:20:53.283473 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="e80169f9-5480-4318-8ab2-1af4394fe76a" containerName="nova-cell0-conductor-conductor" Dec 03 12:20:53 crc kubenswrapper[5052]: I1203 12:20:53.283499 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="b852a7d7-8510-440d-8e87-0755fac2a324" containerName="keystone-db-sync" Dec 03 12:20:53 crc kubenswrapper[5052]: I1203 12:20:53.284414 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:20:53 crc kubenswrapper[5052]: I1203 12:20:53.287975 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell0-conductor-config-data" Dec 03 12:20:53 crc kubenswrapper[5052]: I1203 12:20:53.328568 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bbz9b\" (UniqueName: \"kubernetes.io/projected/efe32106-ca1b-4d84-80a2-c0cddd3541b9-kube-api-access-bbz9b\") pod \"nova-cell0-conductor-0\" (UID: \"efe32106-ca1b-4d84-80a2-c0cddd3541b9\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:20:53 crc kubenswrapper[5052]: I1203 12:20:53.328656 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/efe32106-ca1b-4d84-80a2-c0cddd3541b9-combined-ca-bundle\") pod \"nova-cell0-conductor-0\" (UID: \"efe32106-ca1b-4d84-80a2-c0cddd3541b9\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:20:53 crc kubenswrapper[5052]: I1203 12:20:53.328759 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/efe32106-ca1b-4d84-80a2-c0cddd3541b9-config-data\") pod \"nova-cell0-conductor-0\" (UID: \"efe32106-ca1b-4d84-80a2-c0cddd3541b9\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:20:53 crc kubenswrapper[5052]: I1203 12:20:53.358875 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-0"] Dec 03 12:20:53 crc kubenswrapper[5052]: I1203 12:20:53.377663 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/keystone-bootstrap-cz9xr"] Dec 03 12:20:53 crc kubenswrapper[5052]: I1203 12:20:53.384156 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/keystone-bootstrap-cz9xr"] Dec 03 12:20:53 crc kubenswrapper[5052]: I1203 12:20:53.399307 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/keystone-bootstrap-dqvmm"] Dec 03 12:20:53 crc kubenswrapper[5052]: I1203 12:20:53.400962 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-bootstrap-dqvmm" Dec 03 12:20:53 crc kubenswrapper[5052]: I1203 12:20:53.404294 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-bootstrap-dqvmm"] Dec 03 12:20:53 crc kubenswrapper[5052]: I1203 12:20:53.404800 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"osp-secret" Dec 03 12:20:53 crc kubenswrapper[5052]: I1203 12:20:53.432103 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/63c18817-7663-466c-845d-bc998c5b7093-fernet-keys\") pod \"keystone-bootstrap-dqvmm\" (UID: \"63c18817-7663-466c-845d-bc998c5b7093\") " pod="openstack-kuttl-tests/keystone-bootstrap-dqvmm" Dec 03 12:20:53 crc kubenswrapper[5052]: I1203 12:20:53.432146 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rpgv8\" (UniqueName: \"kubernetes.io/projected/63c18817-7663-466c-845d-bc998c5b7093-kube-api-access-rpgv8\") pod \"keystone-bootstrap-dqvmm\" (UID: \"63c18817-7663-466c-845d-bc998c5b7093\") " pod="openstack-kuttl-tests/keystone-bootstrap-dqvmm" Dec 03 12:20:53 crc kubenswrapper[5052]: I1203 12:20:53.432191 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/63c18817-7663-466c-845d-bc998c5b7093-config-data\") pod \"keystone-bootstrap-dqvmm\" (UID: \"63c18817-7663-466c-845d-bc998c5b7093\") " pod="openstack-kuttl-tests/keystone-bootstrap-dqvmm" Dec 03 12:20:53 crc kubenswrapper[5052]: I1203 12:20:53.432220 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/efe32106-ca1b-4d84-80a2-c0cddd3541b9-config-data\") pod \"nova-cell0-conductor-0\" (UID: \"efe32106-ca1b-4d84-80a2-c0cddd3541b9\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:20:53 crc kubenswrapper[5052]: I1203 12:20:53.432240 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/63c18817-7663-466c-845d-bc998c5b7093-scripts\") pod \"keystone-bootstrap-dqvmm\" (UID: \"63c18817-7663-466c-845d-bc998c5b7093\") " pod="openstack-kuttl-tests/keystone-bootstrap-dqvmm" Dec 03 12:20:53 crc kubenswrapper[5052]: I1203 12:20:53.432293 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/63c18817-7663-466c-845d-bc998c5b7093-combined-ca-bundle\") pod \"keystone-bootstrap-dqvmm\" (UID: \"63c18817-7663-466c-845d-bc998c5b7093\") " pod="openstack-kuttl-tests/keystone-bootstrap-dqvmm" Dec 03 12:20:53 crc kubenswrapper[5052]: I1203 12:20:53.432324 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/63c18817-7663-466c-845d-bc998c5b7093-credential-keys\") pod \"keystone-bootstrap-dqvmm\" (UID: \"63c18817-7663-466c-845d-bc998c5b7093\") " pod="openstack-kuttl-tests/keystone-bootstrap-dqvmm" Dec 03 12:20:53 crc kubenswrapper[5052]: I1203 12:20:53.432354 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-bbz9b\" (UniqueName: \"kubernetes.io/projected/efe32106-ca1b-4d84-80a2-c0cddd3541b9-kube-api-access-bbz9b\") pod \"nova-cell0-conductor-0\" (UID: \"efe32106-ca1b-4d84-80a2-c0cddd3541b9\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:20:53 crc kubenswrapper[5052]: I1203 12:20:53.432402 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/efe32106-ca1b-4d84-80a2-c0cddd3541b9-combined-ca-bundle\") pod \"nova-cell0-conductor-0\" (UID: \"efe32106-ca1b-4d84-80a2-c0cddd3541b9\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:20:53 crc kubenswrapper[5052]: I1203 12:20:53.460535 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/efe32106-ca1b-4d84-80a2-c0cddd3541b9-config-data\") pod \"nova-cell0-conductor-0\" (UID: \"efe32106-ca1b-4d84-80a2-c0cddd3541b9\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:20:53 crc kubenswrapper[5052]: I1203 12:20:53.463181 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/efe32106-ca1b-4d84-80a2-c0cddd3541b9-combined-ca-bundle\") pod \"nova-cell0-conductor-0\" (UID: \"efe32106-ca1b-4d84-80a2-c0cddd3541b9\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:20:53 crc kubenswrapper[5052]: I1203 12:20:53.477255 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-bbz9b\" (UniqueName: \"kubernetes.io/projected/efe32106-ca1b-4d84-80a2-c0cddd3541b9-kube-api-access-bbz9b\") pod \"nova-cell0-conductor-0\" (UID: \"efe32106-ca1b-4d84-80a2-c0cddd3541b9\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:20:53 crc kubenswrapper[5052]: I1203 12:20:53.536240 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/63c18817-7663-466c-845d-bc998c5b7093-config-data\") pod \"keystone-bootstrap-dqvmm\" (UID: \"63c18817-7663-466c-845d-bc998c5b7093\") " pod="openstack-kuttl-tests/keystone-bootstrap-dqvmm" Dec 03 12:20:53 crc kubenswrapper[5052]: I1203 12:20:53.536286 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/63c18817-7663-466c-845d-bc998c5b7093-scripts\") pod \"keystone-bootstrap-dqvmm\" (UID: \"63c18817-7663-466c-845d-bc998c5b7093\") " pod="openstack-kuttl-tests/keystone-bootstrap-dqvmm" Dec 03 12:20:53 crc kubenswrapper[5052]: I1203 12:20:53.536339 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/63c18817-7663-466c-845d-bc998c5b7093-combined-ca-bundle\") pod \"keystone-bootstrap-dqvmm\" (UID: \"63c18817-7663-466c-845d-bc998c5b7093\") " pod="openstack-kuttl-tests/keystone-bootstrap-dqvmm" Dec 03 12:20:53 crc kubenswrapper[5052]: I1203 12:20:53.536368 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/63c18817-7663-466c-845d-bc998c5b7093-credential-keys\") pod \"keystone-bootstrap-dqvmm\" (UID: \"63c18817-7663-466c-845d-bc998c5b7093\") " pod="openstack-kuttl-tests/keystone-bootstrap-dqvmm" Dec 03 12:20:53 crc kubenswrapper[5052]: I1203 12:20:53.536445 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/63c18817-7663-466c-845d-bc998c5b7093-fernet-keys\") pod \"keystone-bootstrap-dqvmm\" (UID: \"63c18817-7663-466c-845d-bc998c5b7093\") " pod="openstack-kuttl-tests/keystone-bootstrap-dqvmm" Dec 03 12:20:53 crc kubenswrapper[5052]: I1203 12:20:53.536464 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-rpgv8\" (UniqueName: \"kubernetes.io/projected/63c18817-7663-466c-845d-bc998c5b7093-kube-api-access-rpgv8\") pod \"keystone-bootstrap-dqvmm\" (UID: \"63c18817-7663-466c-845d-bc998c5b7093\") " pod="openstack-kuttl-tests/keystone-bootstrap-dqvmm" Dec 03 12:20:53 crc kubenswrapper[5052]: I1203 12:20:53.590494 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/63c18817-7663-466c-845d-bc998c5b7093-combined-ca-bundle\") pod \"keystone-bootstrap-dqvmm\" (UID: \"63c18817-7663-466c-845d-bc998c5b7093\") " pod="openstack-kuttl-tests/keystone-bootstrap-dqvmm" Dec 03 12:20:53 crc kubenswrapper[5052]: I1203 12:20:53.590875 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/63c18817-7663-466c-845d-bc998c5b7093-scripts\") pod \"keystone-bootstrap-dqvmm\" (UID: \"63c18817-7663-466c-845d-bc998c5b7093\") " pod="openstack-kuttl-tests/keystone-bootstrap-dqvmm" Dec 03 12:20:53 crc kubenswrapper[5052]: I1203 12:20:53.594344 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-rpgv8\" (UniqueName: \"kubernetes.io/projected/63c18817-7663-466c-845d-bc998c5b7093-kube-api-access-rpgv8\") pod \"keystone-bootstrap-dqvmm\" (UID: \"63c18817-7663-466c-845d-bc998c5b7093\") " pod="openstack-kuttl-tests/keystone-bootstrap-dqvmm" Dec 03 12:20:53 crc kubenswrapper[5052]: I1203 12:20:53.603982 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/63c18817-7663-466c-845d-bc998c5b7093-config-data\") pod \"keystone-bootstrap-dqvmm\" (UID: \"63c18817-7663-466c-845d-bc998c5b7093\") " pod="openstack-kuttl-tests/keystone-bootstrap-dqvmm" Dec 03 12:20:53 crc kubenswrapper[5052]: I1203 12:20:53.617121 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/63c18817-7663-466c-845d-bc998c5b7093-credential-keys\") pod \"keystone-bootstrap-dqvmm\" (UID: \"63c18817-7663-466c-845d-bc998c5b7093\") " pod="openstack-kuttl-tests/keystone-bootstrap-dqvmm" Dec 03 12:20:53 crc kubenswrapper[5052]: I1203 12:20:53.617899 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/63c18817-7663-466c-845d-bc998c5b7093-fernet-keys\") pod \"keystone-bootstrap-dqvmm\" (UID: \"63c18817-7663-466c-845d-bc998c5b7093\") " pod="openstack-kuttl-tests/keystone-bootstrap-dqvmm" Dec 03 12:20:53 crc kubenswrapper[5052]: I1203 12:20:53.710298 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cf5d77fc-8424-4c37-8b5d-de2cf2ffd886" path="/var/lib/kubelet/pods/cf5d77fc-8424-4c37-8b5d-de2cf2ffd886/volumes" Dec 03 12:20:53 crc kubenswrapper[5052]: I1203 12:20:53.718669 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e80169f9-5480-4318-8ab2-1af4394fe76a" path="/var/lib/kubelet/pods/e80169f9-5480-4318-8ab2-1af4394fe76a/volumes" Dec 03 12:20:53 crc kubenswrapper[5052]: I1203 12:20:53.787373 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:20:53 crc kubenswrapper[5052]: I1203 12:20:53.791616 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-bootstrap-dqvmm" Dec 03 12:20:53 crc kubenswrapper[5052]: I1203 12:20:53.793777 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-db-sync-92z8b" Dec 03 12:20:53 crc kubenswrapper[5052]: E1203 12:20:53.828547 5052 cadvisor_stats_provider.go:516] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod66f5168a_1ee5_4de9_8e50_90ef25186ee8.slice/crio-conmon-fcd7a92adc3ca1e7e96f3a9c20674d72f6b8c11f7f6f67a8af2cf51365e8fcb1.scope\": RecentStats: unable to find data in memory cache]" Dec 03 12:20:53 crc kubenswrapper[5052]: I1203 12:20:53.851886 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/7a9497b1-89e2-4ebc-97e6-2bdaeefab95f-db-sync-config-data\") pod \"7a9497b1-89e2-4ebc-97e6-2bdaeefab95f\" (UID: \"7a9497b1-89e2-4ebc-97e6-2bdaeefab95f\") " Dec 03 12:20:53 crc kubenswrapper[5052]: I1203 12:20:53.851961 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/7a9497b1-89e2-4ebc-97e6-2bdaeefab95f-combined-ca-bundle\") pod \"7a9497b1-89e2-4ebc-97e6-2bdaeefab95f\" (UID: \"7a9497b1-89e2-4ebc-97e6-2bdaeefab95f\") " Dec 03 12:20:53 crc kubenswrapper[5052]: I1203 12:20:53.852175 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dkl27\" (UniqueName: \"kubernetes.io/projected/7a9497b1-89e2-4ebc-97e6-2bdaeefab95f-kube-api-access-dkl27\") pod \"7a9497b1-89e2-4ebc-97e6-2bdaeefab95f\" (UID: \"7a9497b1-89e2-4ebc-97e6-2bdaeefab95f\") " Dec 03 12:20:53 crc kubenswrapper[5052]: I1203 12:20:53.859832 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7a9497b1-89e2-4ebc-97e6-2bdaeefab95f-db-sync-config-data" (OuterVolumeSpecName: "db-sync-config-data") pod "7a9497b1-89e2-4ebc-97e6-2bdaeefab95f" (UID: "7a9497b1-89e2-4ebc-97e6-2bdaeefab95f"). InnerVolumeSpecName "db-sync-config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:20:53 crc kubenswrapper[5052]: I1203 12:20:53.887006 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7a9497b1-89e2-4ebc-97e6-2bdaeefab95f-kube-api-access-dkl27" (OuterVolumeSpecName: "kube-api-access-dkl27") pod "7a9497b1-89e2-4ebc-97e6-2bdaeefab95f" (UID: "7a9497b1-89e2-4ebc-97e6-2bdaeefab95f"). InnerVolumeSpecName "kube-api-access-dkl27". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:20:53 crc kubenswrapper[5052]: I1203 12:20:53.899144 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7a9497b1-89e2-4ebc-97e6-2bdaeefab95f-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "7a9497b1-89e2-4ebc-97e6-2bdaeefab95f" (UID: "7a9497b1-89e2-4ebc-97e6-2bdaeefab95f"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:20:53 crc kubenswrapper[5052]: I1203 12:20:53.954278 5052 reconciler_common.go:293] "Volume detached for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/7a9497b1-89e2-4ebc-97e6-2bdaeefab95f-db-sync-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:53 crc kubenswrapper[5052]: I1203 12:20:53.954314 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/7a9497b1-89e2-4ebc-97e6-2bdaeefab95f-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:53 crc kubenswrapper[5052]: I1203 12:20:53.954328 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-dkl27\" (UniqueName: \"kubernetes.io/projected/7a9497b1-89e2-4ebc-97e6-2bdaeefab95f-kube-api-access-dkl27\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:54 crc kubenswrapper[5052]: I1203 12:20:54.047763 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"bb41ed7e-c96b-44dd-8c4e-ffb702de3f87","Type":"ContainerStarted","Data":"2a7af42a9086e0868a44334065634bf0f3e6cbd7e859d0ffd6a5df7b8b1b5555"} Dec 03 12:20:54 crc kubenswrapper[5052]: I1203 12:20:54.048121 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:20:54 crc kubenswrapper[5052]: I1203 12:20:54.073611 5052 generic.go:334] "Generic (PLEG): container finished" podID="66f5168a-1ee5-4de9-8e50-90ef25186ee8" containerID="fcd7a92adc3ca1e7e96f3a9c20674d72f6b8c11f7f6f67a8af2cf51365e8fcb1" exitCode=0 Dec 03 12:20:54 crc kubenswrapper[5052]: I1203 12:20:54.073765 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-db-sync-pvgq9" event={"ID":"66f5168a-1ee5-4de9-8e50-90ef25186ee8","Type":"ContainerDied","Data":"fcd7a92adc3ca1e7e96f3a9c20674d72f6b8c11f7f6f67a8af2cf51365e8fcb1"} Dec 03 12:20:54 crc kubenswrapper[5052]: I1203 12:20:54.092894 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-db-sync-92z8b" event={"ID":"7a9497b1-89e2-4ebc-97e6-2bdaeefab95f","Type":"ContainerDied","Data":"bdb59561dc5e2ad5e9132a7b7a181f019391ce3c7182ce50b77e73b222297b71"} Dec 03 12:20:54 crc kubenswrapper[5052]: I1203 12:20:54.092932 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bdb59561dc5e2ad5e9132a7b7a181f019391ce3c7182ce50b77e73b222297b71" Dec 03 12:20:54 crc kubenswrapper[5052]: I1203 12:20:54.092998 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-db-sync-92z8b" Dec 03 12:20:54 crc kubenswrapper[5052]: I1203 12:20:54.093786 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/cinder-api-0" podStartSLOduration=3.093770697 podStartE2EDuration="3.093770697s" podCreationTimestamp="2025-12-03 12:20:51 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:20:54.073021905 +0000 UTC m=+3644.661263689" watchObservedRunningTime="2025-12-03 12:20:54.093770697 +0000 UTC m=+3644.682012461" Dec 03 12:20:54 crc kubenswrapper[5052]: I1203 12:20:54.189974 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/barbican-worker-6b4674c647-56v84"] Dec 03 12:20:54 crc kubenswrapper[5052]: E1203 12:20:54.190398 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="7a9497b1-89e2-4ebc-97e6-2bdaeefab95f" containerName="barbican-db-sync" Dec 03 12:20:54 crc kubenswrapper[5052]: I1203 12:20:54.190408 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="7a9497b1-89e2-4ebc-97e6-2bdaeefab95f" containerName="barbican-db-sync" Dec 03 12:20:54 crc kubenswrapper[5052]: I1203 12:20:54.190645 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="7a9497b1-89e2-4ebc-97e6-2bdaeefab95f" containerName="barbican-db-sync" Dec 03 12:20:54 crc kubenswrapper[5052]: I1203 12:20:54.191663 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-worker-6b4674c647-56v84" Dec 03 12:20:54 crc kubenswrapper[5052]: I1203 12:20:54.220724 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-worker-6b4674c647-56v84"] Dec 03 12:20:54 crc kubenswrapper[5052]: I1203 12:20:54.242917 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/barbican-keystone-listener-6fb665b666-ftpjr"] Dec 03 12:20:54 crc kubenswrapper[5052]: I1203 12:20:54.244611 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-keystone-listener-6fb665b666-ftpjr" Dec 03 12:20:54 crc kubenswrapper[5052]: I1203 12:20:54.265996 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/484e6cd1-fb57-4416-9a1a-cf24b4693764-logs\") pod \"barbican-worker-6b4674c647-56v84\" (UID: \"484e6cd1-fb57-4416-9a1a-cf24b4693764\") " pod="openstack-kuttl-tests/barbican-worker-6b4674c647-56v84" Dec 03 12:20:54 crc kubenswrapper[5052]: I1203 12:20:54.266065 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/484e6cd1-fb57-4416-9a1a-cf24b4693764-combined-ca-bundle\") pod \"barbican-worker-6b4674c647-56v84\" (UID: \"484e6cd1-fb57-4416-9a1a-cf24b4693764\") " pod="openstack-kuttl-tests/barbican-worker-6b4674c647-56v84" Dec 03 12:20:54 crc kubenswrapper[5052]: I1203 12:20:54.266129 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/484e6cd1-fb57-4416-9a1a-cf24b4693764-config-data-custom\") pod \"barbican-worker-6b4674c647-56v84\" (UID: \"484e6cd1-fb57-4416-9a1a-cf24b4693764\") " pod="openstack-kuttl-tests/barbican-worker-6b4674c647-56v84" Dec 03 12:20:54 crc kubenswrapper[5052]: I1203 12:20:54.266145 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s5rx8\" (UniqueName: \"kubernetes.io/projected/484e6cd1-fb57-4416-9a1a-cf24b4693764-kube-api-access-s5rx8\") pod \"barbican-worker-6b4674c647-56v84\" (UID: \"484e6cd1-fb57-4416-9a1a-cf24b4693764\") " pod="openstack-kuttl-tests/barbican-worker-6b4674c647-56v84" Dec 03 12:20:54 crc kubenswrapper[5052]: I1203 12:20:54.266290 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/484e6cd1-fb57-4416-9a1a-cf24b4693764-config-data\") pod \"barbican-worker-6b4674c647-56v84\" (UID: \"484e6cd1-fb57-4416-9a1a-cf24b4693764\") " pod="openstack-kuttl-tests/barbican-worker-6b4674c647-56v84" Dec 03 12:20:54 crc kubenswrapper[5052]: I1203 12:20:54.291611 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/barbican-api-644f564b46-cjtkx"] Dec 03 12:20:54 crc kubenswrapper[5052]: I1203 12:20:54.293318 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-api-644f564b46-cjtkx" Dec 03 12:20:54 crc kubenswrapper[5052]: I1203 12:20:54.336474 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-keystone-listener-6fb665b666-ftpjr"] Dec 03 12:20:54 crc kubenswrapper[5052]: I1203 12:20:54.369070 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/af38eb0d-3924-4c4b-a22e-7e934bdcf172-config-data\") pod \"barbican-api-644f564b46-cjtkx\" (UID: \"af38eb0d-3924-4c4b-a22e-7e934bdcf172\") " pod="openstack-kuttl-tests/barbican-api-644f564b46-cjtkx" Dec 03 12:20:54 crc kubenswrapper[5052]: I1203 12:20:54.369123 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9119635e-cb55-4428-9c61-b93ef2afd799-combined-ca-bundle\") pod \"barbican-keystone-listener-6fb665b666-ftpjr\" (UID: \"9119635e-cb55-4428-9c61-b93ef2afd799\") " pod="openstack-kuttl-tests/barbican-keystone-listener-6fb665b666-ftpjr" Dec 03 12:20:54 crc kubenswrapper[5052]: I1203 12:20:54.369169 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/af38eb0d-3924-4c4b-a22e-7e934bdcf172-logs\") pod \"barbican-api-644f564b46-cjtkx\" (UID: \"af38eb0d-3924-4c4b-a22e-7e934bdcf172\") " pod="openstack-kuttl-tests/barbican-api-644f564b46-cjtkx" Dec 03 12:20:54 crc kubenswrapper[5052]: I1203 12:20:54.369205 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f25j9\" (UniqueName: \"kubernetes.io/projected/9119635e-cb55-4428-9c61-b93ef2afd799-kube-api-access-f25j9\") pod \"barbican-keystone-listener-6fb665b666-ftpjr\" (UID: \"9119635e-cb55-4428-9c61-b93ef2afd799\") " pod="openstack-kuttl-tests/barbican-keystone-listener-6fb665b666-ftpjr" Dec 03 12:20:54 crc kubenswrapper[5052]: I1203 12:20:54.369241 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fqcvl\" (UniqueName: \"kubernetes.io/projected/af38eb0d-3924-4c4b-a22e-7e934bdcf172-kube-api-access-fqcvl\") pod \"barbican-api-644f564b46-cjtkx\" (UID: \"af38eb0d-3924-4c4b-a22e-7e934bdcf172\") " pod="openstack-kuttl-tests/barbican-api-644f564b46-cjtkx" Dec 03 12:20:54 crc kubenswrapper[5052]: I1203 12:20:54.369299 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/484e6cd1-fb57-4416-9a1a-cf24b4693764-config-data\") pod \"barbican-worker-6b4674c647-56v84\" (UID: \"484e6cd1-fb57-4416-9a1a-cf24b4693764\") " pod="openstack-kuttl-tests/barbican-worker-6b4674c647-56v84" Dec 03 12:20:54 crc kubenswrapper[5052]: I1203 12:20:54.369333 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/af38eb0d-3924-4c4b-a22e-7e934bdcf172-config-data-custom\") pod \"barbican-api-644f564b46-cjtkx\" (UID: \"af38eb0d-3924-4c4b-a22e-7e934bdcf172\") " pod="openstack-kuttl-tests/barbican-api-644f564b46-cjtkx" Dec 03 12:20:54 crc kubenswrapper[5052]: I1203 12:20:54.369361 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/9119635e-cb55-4428-9c61-b93ef2afd799-config-data\") pod \"barbican-keystone-listener-6fb665b666-ftpjr\" (UID: \"9119635e-cb55-4428-9c61-b93ef2afd799\") " pod="openstack-kuttl-tests/barbican-keystone-listener-6fb665b666-ftpjr" Dec 03 12:20:54 crc kubenswrapper[5052]: I1203 12:20:54.369398 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/af38eb0d-3924-4c4b-a22e-7e934bdcf172-public-tls-certs\") pod \"barbican-api-644f564b46-cjtkx\" (UID: \"af38eb0d-3924-4c4b-a22e-7e934bdcf172\") " pod="openstack-kuttl-tests/barbican-api-644f564b46-cjtkx" Dec 03 12:20:54 crc kubenswrapper[5052]: I1203 12:20:54.369422 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/484e6cd1-fb57-4416-9a1a-cf24b4693764-logs\") pod \"barbican-worker-6b4674c647-56v84\" (UID: \"484e6cd1-fb57-4416-9a1a-cf24b4693764\") " pod="openstack-kuttl-tests/barbican-worker-6b4674c647-56v84" Dec 03 12:20:54 crc kubenswrapper[5052]: I1203 12:20:54.369456 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/484e6cd1-fb57-4416-9a1a-cf24b4693764-combined-ca-bundle\") pod \"barbican-worker-6b4674c647-56v84\" (UID: \"484e6cd1-fb57-4416-9a1a-cf24b4693764\") " pod="openstack-kuttl-tests/barbican-worker-6b4674c647-56v84" Dec 03 12:20:54 crc kubenswrapper[5052]: I1203 12:20:54.369479 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/9119635e-cb55-4428-9c61-b93ef2afd799-logs\") pod \"barbican-keystone-listener-6fb665b666-ftpjr\" (UID: \"9119635e-cb55-4428-9c61-b93ef2afd799\") " pod="openstack-kuttl-tests/barbican-keystone-listener-6fb665b666-ftpjr" Dec 03 12:20:54 crc kubenswrapper[5052]: I1203 12:20:54.369500 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/9119635e-cb55-4428-9c61-b93ef2afd799-config-data-custom\") pod \"barbican-keystone-listener-6fb665b666-ftpjr\" (UID: \"9119635e-cb55-4428-9c61-b93ef2afd799\") " pod="openstack-kuttl-tests/barbican-keystone-listener-6fb665b666-ftpjr" Dec 03 12:20:54 crc kubenswrapper[5052]: I1203 12:20:54.386815 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/af38eb0d-3924-4c4b-a22e-7e934bdcf172-internal-tls-certs\") pod \"barbican-api-644f564b46-cjtkx\" (UID: \"af38eb0d-3924-4c4b-a22e-7e934bdcf172\") " pod="openstack-kuttl-tests/barbican-api-644f564b46-cjtkx" Dec 03 12:20:54 crc kubenswrapper[5052]: I1203 12:20:54.387189 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/af38eb0d-3924-4c4b-a22e-7e934bdcf172-combined-ca-bundle\") pod \"barbican-api-644f564b46-cjtkx\" (UID: \"af38eb0d-3924-4c4b-a22e-7e934bdcf172\") " pod="openstack-kuttl-tests/barbican-api-644f564b46-cjtkx" Dec 03 12:20:54 crc kubenswrapper[5052]: I1203 12:20:54.387264 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/484e6cd1-fb57-4416-9a1a-cf24b4693764-config-data-custom\") pod \"barbican-worker-6b4674c647-56v84\" (UID: \"484e6cd1-fb57-4416-9a1a-cf24b4693764\") " pod="openstack-kuttl-tests/barbican-worker-6b4674c647-56v84" Dec 03 12:20:54 crc kubenswrapper[5052]: I1203 12:20:54.387299 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-s5rx8\" (UniqueName: \"kubernetes.io/projected/484e6cd1-fb57-4416-9a1a-cf24b4693764-kube-api-access-s5rx8\") pod \"barbican-worker-6b4674c647-56v84\" (UID: \"484e6cd1-fb57-4416-9a1a-cf24b4693764\") " pod="openstack-kuttl-tests/barbican-worker-6b4674c647-56v84" Dec 03 12:20:54 crc kubenswrapper[5052]: I1203 12:20:54.390167 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/484e6cd1-fb57-4416-9a1a-cf24b4693764-logs\") pod \"barbican-worker-6b4674c647-56v84\" (UID: \"484e6cd1-fb57-4416-9a1a-cf24b4693764\") " pod="openstack-kuttl-tests/barbican-worker-6b4674c647-56v84" Dec 03 12:20:54 crc kubenswrapper[5052]: I1203 12:20:54.399965 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/484e6cd1-fb57-4416-9a1a-cf24b4693764-combined-ca-bundle\") pod \"barbican-worker-6b4674c647-56v84\" (UID: \"484e6cd1-fb57-4416-9a1a-cf24b4693764\") " pod="openstack-kuttl-tests/barbican-worker-6b4674c647-56v84" Dec 03 12:20:54 crc kubenswrapper[5052]: I1203 12:20:54.400925 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/484e6cd1-fb57-4416-9a1a-cf24b4693764-config-data-custom\") pod \"barbican-worker-6b4674c647-56v84\" (UID: \"484e6cd1-fb57-4416-9a1a-cf24b4693764\") " pod="openstack-kuttl-tests/barbican-worker-6b4674c647-56v84" Dec 03 12:20:54 crc kubenswrapper[5052]: I1203 12:20:54.421928 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/484e6cd1-fb57-4416-9a1a-cf24b4693764-config-data\") pod \"barbican-worker-6b4674c647-56v84\" (UID: \"484e6cd1-fb57-4416-9a1a-cf24b4693764\") " pod="openstack-kuttl-tests/barbican-worker-6b4674c647-56v84" Dec 03 12:20:54 crc kubenswrapper[5052]: I1203 12:20:54.430227 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-s5rx8\" (UniqueName: \"kubernetes.io/projected/484e6cd1-fb57-4416-9a1a-cf24b4693764-kube-api-access-s5rx8\") pod \"barbican-worker-6b4674c647-56v84\" (UID: \"484e6cd1-fb57-4416-9a1a-cf24b4693764\") " pod="openstack-kuttl-tests/barbican-worker-6b4674c647-56v84" Dec 03 12:20:54 crc kubenswrapper[5052]: I1203 12:20:54.453095 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-api-644f564b46-cjtkx"] Dec 03 12:20:54 crc kubenswrapper[5052]: I1203 12:20:54.490254 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/af38eb0d-3924-4c4b-a22e-7e934bdcf172-public-tls-certs\") pod \"barbican-api-644f564b46-cjtkx\" (UID: \"af38eb0d-3924-4c4b-a22e-7e934bdcf172\") " pod="openstack-kuttl-tests/barbican-api-644f564b46-cjtkx" Dec 03 12:20:54 crc kubenswrapper[5052]: I1203 12:20:54.490342 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/9119635e-cb55-4428-9c61-b93ef2afd799-logs\") pod \"barbican-keystone-listener-6fb665b666-ftpjr\" (UID: \"9119635e-cb55-4428-9c61-b93ef2afd799\") " pod="openstack-kuttl-tests/barbican-keystone-listener-6fb665b666-ftpjr" Dec 03 12:20:54 crc kubenswrapper[5052]: I1203 12:20:54.490366 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/9119635e-cb55-4428-9c61-b93ef2afd799-config-data-custom\") pod \"barbican-keystone-listener-6fb665b666-ftpjr\" (UID: \"9119635e-cb55-4428-9c61-b93ef2afd799\") " pod="openstack-kuttl-tests/barbican-keystone-listener-6fb665b666-ftpjr" Dec 03 12:20:54 crc kubenswrapper[5052]: I1203 12:20:54.490401 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/af38eb0d-3924-4c4b-a22e-7e934bdcf172-internal-tls-certs\") pod \"barbican-api-644f564b46-cjtkx\" (UID: \"af38eb0d-3924-4c4b-a22e-7e934bdcf172\") " pod="openstack-kuttl-tests/barbican-api-644f564b46-cjtkx" Dec 03 12:20:54 crc kubenswrapper[5052]: I1203 12:20:54.490426 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/af38eb0d-3924-4c4b-a22e-7e934bdcf172-combined-ca-bundle\") pod \"barbican-api-644f564b46-cjtkx\" (UID: \"af38eb0d-3924-4c4b-a22e-7e934bdcf172\") " pod="openstack-kuttl-tests/barbican-api-644f564b46-cjtkx" Dec 03 12:20:54 crc kubenswrapper[5052]: I1203 12:20:54.490495 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/af38eb0d-3924-4c4b-a22e-7e934bdcf172-config-data\") pod \"barbican-api-644f564b46-cjtkx\" (UID: \"af38eb0d-3924-4c4b-a22e-7e934bdcf172\") " pod="openstack-kuttl-tests/barbican-api-644f564b46-cjtkx" Dec 03 12:20:54 crc kubenswrapper[5052]: I1203 12:20:54.490529 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9119635e-cb55-4428-9c61-b93ef2afd799-combined-ca-bundle\") pod \"barbican-keystone-listener-6fb665b666-ftpjr\" (UID: \"9119635e-cb55-4428-9c61-b93ef2afd799\") " pod="openstack-kuttl-tests/barbican-keystone-listener-6fb665b666-ftpjr" Dec 03 12:20:54 crc kubenswrapper[5052]: I1203 12:20:54.490566 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/af38eb0d-3924-4c4b-a22e-7e934bdcf172-logs\") pod \"barbican-api-644f564b46-cjtkx\" (UID: \"af38eb0d-3924-4c4b-a22e-7e934bdcf172\") " pod="openstack-kuttl-tests/barbican-api-644f564b46-cjtkx" Dec 03 12:20:54 crc kubenswrapper[5052]: I1203 12:20:54.490617 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-f25j9\" (UniqueName: \"kubernetes.io/projected/9119635e-cb55-4428-9c61-b93ef2afd799-kube-api-access-f25j9\") pod \"barbican-keystone-listener-6fb665b666-ftpjr\" (UID: \"9119635e-cb55-4428-9c61-b93ef2afd799\") " pod="openstack-kuttl-tests/barbican-keystone-listener-6fb665b666-ftpjr" Dec 03 12:20:54 crc kubenswrapper[5052]: I1203 12:20:54.490649 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-fqcvl\" (UniqueName: \"kubernetes.io/projected/af38eb0d-3924-4c4b-a22e-7e934bdcf172-kube-api-access-fqcvl\") pod \"barbican-api-644f564b46-cjtkx\" (UID: \"af38eb0d-3924-4c4b-a22e-7e934bdcf172\") " pod="openstack-kuttl-tests/barbican-api-644f564b46-cjtkx" Dec 03 12:20:54 crc kubenswrapper[5052]: I1203 12:20:54.490708 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/af38eb0d-3924-4c4b-a22e-7e934bdcf172-config-data-custom\") pod \"barbican-api-644f564b46-cjtkx\" (UID: \"af38eb0d-3924-4c4b-a22e-7e934bdcf172\") " pod="openstack-kuttl-tests/barbican-api-644f564b46-cjtkx" Dec 03 12:20:54 crc kubenswrapper[5052]: I1203 12:20:54.490731 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/9119635e-cb55-4428-9c61-b93ef2afd799-config-data\") pod \"barbican-keystone-listener-6fb665b666-ftpjr\" (UID: \"9119635e-cb55-4428-9c61-b93ef2afd799\") " pod="openstack-kuttl-tests/barbican-keystone-listener-6fb665b666-ftpjr" Dec 03 12:20:54 crc kubenswrapper[5052]: I1203 12:20:54.493450 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/9119635e-cb55-4428-9c61-b93ef2afd799-logs\") pod \"barbican-keystone-listener-6fb665b666-ftpjr\" (UID: \"9119635e-cb55-4428-9c61-b93ef2afd799\") " pod="openstack-kuttl-tests/barbican-keystone-listener-6fb665b666-ftpjr" Dec 03 12:20:54 crc kubenswrapper[5052]: I1203 12:20:54.495026 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/af38eb0d-3924-4c4b-a22e-7e934bdcf172-logs\") pod \"barbican-api-644f564b46-cjtkx\" (UID: \"af38eb0d-3924-4c4b-a22e-7e934bdcf172\") " pod="openstack-kuttl-tests/barbican-api-644f564b46-cjtkx" Dec 03 12:20:54 crc kubenswrapper[5052]: I1203 12:20:54.496424 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/9119635e-cb55-4428-9c61-b93ef2afd799-config-data\") pod \"barbican-keystone-listener-6fb665b666-ftpjr\" (UID: \"9119635e-cb55-4428-9c61-b93ef2afd799\") " pod="openstack-kuttl-tests/barbican-keystone-listener-6fb665b666-ftpjr" Dec 03 12:20:54 crc kubenswrapper[5052]: I1203 12:20:54.497651 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/9119635e-cb55-4428-9c61-b93ef2afd799-config-data-custom\") pod \"barbican-keystone-listener-6fb665b666-ftpjr\" (UID: \"9119635e-cb55-4428-9c61-b93ef2afd799\") " pod="openstack-kuttl-tests/barbican-keystone-listener-6fb665b666-ftpjr" Dec 03 12:20:54 crc kubenswrapper[5052]: I1203 12:20:54.498851 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/af38eb0d-3924-4c4b-a22e-7e934bdcf172-internal-tls-certs\") pod \"barbican-api-644f564b46-cjtkx\" (UID: \"af38eb0d-3924-4c4b-a22e-7e934bdcf172\") " pod="openstack-kuttl-tests/barbican-api-644f564b46-cjtkx" Dec 03 12:20:54 crc kubenswrapper[5052]: I1203 12:20:54.499215 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9119635e-cb55-4428-9c61-b93ef2afd799-combined-ca-bundle\") pod \"barbican-keystone-listener-6fb665b666-ftpjr\" (UID: \"9119635e-cb55-4428-9c61-b93ef2afd799\") " pod="openstack-kuttl-tests/barbican-keystone-listener-6fb665b666-ftpjr" Dec 03 12:20:54 crc kubenswrapper[5052]: I1203 12:20:54.499551 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/af38eb0d-3924-4c4b-a22e-7e934bdcf172-config-data\") pod \"barbican-api-644f564b46-cjtkx\" (UID: \"af38eb0d-3924-4c4b-a22e-7e934bdcf172\") " pod="openstack-kuttl-tests/barbican-api-644f564b46-cjtkx" Dec 03 12:20:54 crc kubenswrapper[5052]: I1203 12:20:54.499885 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/af38eb0d-3924-4c4b-a22e-7e934bdcf172-public-tls-certs\") pod \"barbican-api-644f564b46-cjtkx\" (UID: \"af38eb0d-3924-4c4b-a22e-7e934bdcf172\") " pod="openstack-kuttl-tests/barbican-api-644f564b46-cjtkx" Dec 03 12:20:54 crc kubenswrapper[5052]: I1203 12:20:54.506885 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/af38eb0d-3924-4c4b-a22e-7e934bdcf172-config-data-custom\") pod \"barbican-api-644f564b46-cjtkx\" (UID: \"af38eb0d-3924-4c4b-a22e-7e934bdcf172\") " pod="openstack-kuttl-tests/barbican-api-644f564b46-cjtkx" Dec 03 12:20:54 crc kubenswrapper[5052]: I1203 12:20:54.510074 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/af38eb0d-3924-4c4b-a22e-7e934bdcf172-combined-ca-bundle\") pod \"barbican-api-644f564b46-cjtkx\" (UID: \"af38eb0d-3924-4c4b-a22e-7e934bdcf172\") " pod="openstack-kuttl-tests/barbican-api-644f564b46-cjtkx" Dec 03 12:20:54 crc kubenswrapper[5052]: I1203 12:20:54.526045 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-fqcvl\" (UniqueName: \"kubernetes.io/projected/af38eb0d-3924-4c4b-a22e-7e934bdcf172-kube-api-access-fqcvl\") pod \"barbican-api-644f564b46-cjtkx\" (UID: \"af38eb0d-3924-4c4b-a22e-7e934bdcf172\") " pod="openstack-kuttl-tests/barbican-api-644f564b46-cjtkx" Dec 03 12:20:54 crc kubenswrapper[5052]: I1203 12:20:54.541582 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-worker-6b4674c647-56v84" Dec 03 12:20:54 crc kubenswrapper[5052]: I1203 12:20:54.544574 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-f25j9\" (UniqueName: \"kubernetes.io/projected/9119635e-cb55-4428-9c61-b93ef2afd799-kube-api-access-f25j9\") pod \"barbican-keystone-listener-6fb665b666-ftpjr\" (UID: \"9119635e-cb55-4428-9c61-b93ef2afd799\") " pod="openstack-kuttl-tests/barbican-keystone-listener-6fb665b666-ftpjr" Dec 03 12:20:54 crc kubenswrapper[5052]: I1203 12:20:54.578416 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-bootstrap-dqvmm"] Dec 03 12:20:54 crc kubenswrapper[5052]: I1203 12:20:54.601763 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-0"] Dec 03 12:20:54 crc kubenswrapper[5052]: I1203 12:20:54.700050 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-keystone-listener-6fb665b666-ftpjr" Dec 03 12:20:54 crc kubenswrapper[5052]: I1203 12:20:54.793545 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-api-644f564b46-cjtkx" Dec 03 12:20:54 crc kubenswrapper[5052]: I1203 12:20:54.798158 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:20:54 crc kubenswrapper[5052]: I1203 12:20:54.845278 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:20:54 crc kubenswrapper[5052]: I1203 12:20:54.898470 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/aa2b4c07-e5c7-4543-a0a5-f15b8e1b7f83-config-data\") pod \"aa2b4c07-e5c7-4543-a0a5-f15b8e1b7f83\" (UID: \"aa2b4c07-e5c7-4543-a0a5-f15b8e1b7f83\") " Dec 03 12:20:54 crc kubenswrapper[5052]: I1203 12:20:54.898526 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/aa2b4c07-e5c7-4543-a0a5-f15b8e1b7f83-combined-ca-bundle\") pod \"aa2b4c07-e5c7-4543-a0a5-f15b8e1b7f83\" (UID: \"aa2b4c07-e5c7-4543-a0a5-f15b8e1b7f83\") " Dec 03 12:20:54 crc kubenswrapper[5052]: I1203 12:20:54.898567 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"lock\" (UniqueName: \"kubernetes.io/empty-dir/335ad996-ad3b-4ac8-a488-64e9453f24a4-lock\") pod \"335ad996-ad3b-4ac8-a488-64e9453f24a4\" (UID: \"335ad996-ad3b-4ac8-a488-64e9453f24a4\") " Dec 03 12:20:54 crc kubenswrapper[5052]: I1203 12:20:54.898604 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n4bgm\" (UniqueName: \"kubernetes.io/projected/335ad996-ad3b-4ac8-a488-64e9453f24a4-kube-api-access-n4bgm\") pod \"335ad996-ad3b-4ac8-a488-64e9453f24a4\" (UID: \"335ad996-ad3b-4ac8-a488-64e9453f24a4\") " Dec 03 12:20:54 crc kubenswrapper[5052]: I1203 12:20:54.898633 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"swift\" (UniqueName: \"kubernetes.io/local-volume/local-storage05-crc\") pod \"335ad996-ad3b-4ac8-a488-64e9453f24a4\" (UID: \"335ad996-ad3b-4ac8-a488-64e9453f24a4\") " Dec 03 12:20:54 crc kubenswrapper[5052]: I1203 12:20:54.898743 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tlncn\" (UniqueName: \"kubernetes.io/projected/aa2b4c07-e5c7-4543-a0a5-f15b8e1b7f83-kube-api-access-tlncn\") pod \"aa2b4c07-e5c7-4543-a0a5-f15b8e1b7f83\" (UID: \"aa2b4c07-e5c7-4543-a0a5-f15b8e1b7f83\") " Dec 03 12:20:54 crc kubenswrapper[5052]: I1203 12:20:54.898819 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"cache\" (UniqueName: \"kubernetes.io/empty-dir/335ad996-ad3b-4ac8-a488-64e9453f24a4-cache\") pod \"335ad996-ad3b-4ac8-a488-64e9453f24a4\" (UID: \"335ad996-ad3b-4ac8-a488-64e9453f24a4\") " Dec 03 12:20:54 crc kubenswrapper[5052]: I1203 12:20:54.900843 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/335ad996-ad3b-4ac8-a488-64e9453f24a4-lock" (OuterVolumeSpecName: "lock") pod "335ad996-ad3b-4ac8-a488-64e9453f24a4" (UID: "335ad996-ad3b-4ac8-a488-64e9453f24a4"). InnerVolumeSpecName "lock". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:20:54 crc kubenswrapper[5052]: I1203 12:20:54.905350 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aa2b4c07-e5c7-4543-a0a5-f15b8e1b7f83-kube-api-access-tlncn" (OuterVolumeSpecName: "kube-api-access-tlncn") pod "aa2b4c07-e5c7-4543-a0a5-f15b8e1b7f83" (UID: "aa2b4c07-e5c7-4543-a0a5-f15b8e1b7f83"). InnerVolumeSpecName "kube-api-access-tlncn". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:20:54 crc kubenswrapper[5052]: I1203 12:20:54.906111 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/335ad996-ad3b-4ac8-a488-64e9453f24a4-cache" (OuterVolumeSpecName: "cache") pod "335ad996-ad3b-4ac8-a488-64e9453f24a4" (UID: "335ad996-ad3b-4ac8-a488-64e9453f24a4"). InnerVolumeSpecName "cache". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:20:54 crc kubenswrapper[5052]: I1203 12:20:54.907279 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage05-crc" (OuterVolumeSpecName: "swift") pod "335ad996-ad3b-4ac8-a488-64e9453f24a4" (UID: "335ad996-ad3b-4ac8-a488-64e9453f24a4"). InnerVolumeSpecName "local-storage05-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 12:20:54 crc kubenswrapper[5052]: I1203 12:20:54.907668 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/335ad996-ad3b-4ac8-a488-64e9453f24a4-etc-swift\") pod \"335ad996-ad3b-4ac8-a488-64e9453f24a4\" (UID: \"335ad996-ad3b-4ac8-a488-64e9453f24a4\") " Dec 03 12:20:54 crc kubenswrapper[5052]: I1203 12:20:54.909287 5052 reconciler_common.go:293] "Volume detached for volume \"cache\" (UniqueName: \"kubernetes.io/empty-dir/335ad996-ad3b-4ac8-a488-64e9453f24a4-cache\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:54 crc kubenswrapper[5052]: I1203 12:20:54.909310 5052 reconciler_common.go:293] "Volume detached for volume \"lock\" (UniqueName: \"kubernetes.io/empty-dir/335ad996-ad3b-4ac8-a488-64e9453f24a4-lock\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:54 crc kubenswrapper[5052]: I1203 12:20:54.909335 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage05-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage05-crc\") on node \"crc\" " Dec 03 12:20:54 crc kubenswrapper[5052]: I1203 12:20:54.909348 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-tlncn\" (UniqueName: \"kubernetes.io/projected/aa2b4c07-e5c7-4543-a0a5-f15b8e1b7f83-kube-api-access-tlncn\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:54 crc kubenswrapper[5052]: I1203 12:20:54.912098 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/335ad996-ad3b-4ac8-a488-64e9453f24a4-kube-api-access-n4bgm" (OuterVolumeSpecName: "kube-api-access-n4bgm") pod "335ad996-ad3b-4ac8-a488-64e9453f24a4" (UID: "335ad996-ad3b-4ac8-a488-64e9453f24a4"). InnerVolumeSpecName "kube-api-access-n4bgm". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:20:54 crc kubenswrapper[5052]: I1203 12:20:54.918346 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/335ad996-ad3b-4ac8-a488-64e9453f24a4-etc-swift" (OuterVolumeSpecName: "etc-swift") pod "335ad996-ad3b-4ac8-a488-64e9453f24a4" (UID: "335ad996-ad3b-4ac8-a488-64e9453f24a4"). InnerVolumeSpecName "etc-swift". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:20:54 crc kubenswrapper[5052]: I1203 12:20:54.939832 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aa2b4c07-e5c7-4543-a0a5-f15b8e1b7f83-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "aa2b4c07-e5c7-4543-a0a5-f15b8e1b7f83" (UID: "aa2b4c07-e5c7-4543-a0a5-f15b8e1b7f83"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:20:54 crc kubenswrapper[5052]: I1203 12:20:54.953374 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aa2b4c07-e5c7-4543-a0a5-f15b8e1b7f83-config-data" (OuterVolumeSpecName: "config-data") pod "aa2b4c07-e5c7-4543-a0a5-f15b8e1b7f83" (UID: "aa2b4c07-e5c7-4543-a0a5-f15b8e1b7f83"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:20:54 crc kubenswrapper[5052]: I1203 12:20:54.989597 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage05-crc" (UniqueName: "kubernetes.io/local-volume/local-storage05-crc") on node "crc" Dec 03 12:20:55 crc kubenswrapper[5052]: I1203 12:20:55.010885 5052 reconciler_common.go:293] "Volume detached for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/335ad996-ad3b-4ac8-a488-64e9453f24a4-etc-swift\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:55 crc kubenswrapper[5052]: I1203 12:20:55.010923 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/aa2b4c07-e5c7-4543-a0a5-f15b8e1b7f83-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:55 crc kubenswrapper[5052]: I1203 12:20:55.010937 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/aa2b4c07-e5c7-4543-a0a5-f15b8e1b7f83-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:55 crc kubenswrapper[5052]: I1203 12:20:55.010950 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-n4bgm\" (UniqueName: \"kubernetes.io/projected/335ad996-ad3b-4ac8-a488-64e9453f24a4-kube-api-access-n4bgm\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:55 crc kubenswrapper[5052]: I1203 12:20:55.010961 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage05-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage05-crc\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:55 crc kubenswrapper[5052]: I1203 12:20:55.132688 5052 generic.go:334] "Generic (PLEG): container finished" podID="335ad996-ad3b-4ac8-a488-64e9453f24a4" containerID="fb2b07929be7f2dd4d6fd56e12955638b93484338cfedea07147fa092b946832" exitCode=137 Dec 03 12:20:55 crc kubenswrapper[5052]: I1203 12:20:55.132761 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"335ad996-ad3b-4ac8-a488-64e9453f24a4","Type":"ContainerDied","Data":"fb2b07929be7f2dd4d6fd56e12955638b93484338cfedea07147fa092b946832"} Dec 03 12:20:55 crc kubenswrapper[5052]: I1203 12:20:55.132793 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"335ad996-ad3b-4ac8-a488-64e9453f24a4","Type":"ContainerDied","Data":"dc35354b0a80d09200fdddb37c6af8fefe0bfea68ba76cd38f4455c24df918d1"} Dec 03 12:20:55 crc kubenswrapper[5052]: I1203 12:20:55.132815 5052 scope.go:117] "RemoveContainer" containerID="fb2b07929be7f2dd4d6fd56e12955638b93484338cfedea07147fa092b946832" Dec 03 12:20:55 crc kubenswrapper[5052]: I1203 12:20:55.132959 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:20:55 crc kubenswrapper[5052]: I1203 12:20:55.136264 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-conductor-0" event={"ID":"efe32106-ca1b-4d84-80a2-c0cddd3541b9","Type":"ContainerStarted","Data":"75bc26794aa58f8bf478527bc4377138449cad0841fd60348ba9d8b17243a1e4"} Dec 03 12:20:55 crc kubenswrapper[5052]: I1203 12:20:55.136311 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-conductor-0" event={"ID":"efe32106-ca1b-4d84-80a2-c0cddd3541b9","Type":"ContainerStarted","Data":"13bb5b7012087ab429759406030370af76979b95cc58559676341cb6a1ce0b66"} Dec 03 12:20:55 crc kubenswrapper[5052]: I1203 12:20:55.136372 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:20:55 crc kubenswrapper[5052]: I1203 12:20:55.139562 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-worker-6b4674c647-56v84"] Dec 03 12:20:55 crc kubenswrapper[5052]: W1203 12:20:55.139567 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod484e6cd1_fb57_4416_9a1a_cf24b4693764.slice/crio-192121d8e29554479b65c83a0f69371b4815e1374a9f01070f02296033b3c0de WatchSource:0}: Error finding container 192121d8e29554479b65c83a0f69371b4815e1374a9f01070f02296033b3c0de: Status 404 returned error can't find the container with id 192121d8e29554479b65c83a0f69371b4815e1374a9f01070f02296033b3c0de Dec 03 12:20:55 crc kubenswrapper[5052]: I1203 12:20:55.140592 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"2d94bf00-7a64-4e90-a48c-b206ec54da6d","Type":"ContainerStarted","Data":"3d06201ab438e62b35c336b3ff97f18ab1544e471098deb210f49f6607c474fe"} Dec 03 12:20:55 crc kubenswrapper[5052]: I1203 12:20:55.141391 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:20:55 crc kubenswrapper[5052]: I1203 12:20:55.147822 5052 generic.go:334] "Generic (PLEG): container finished" podID="aa2b4c07-e5c7-4543-a0a5-f15b8e1b7f83" containerID="4c12bd41f4c701be825345e793ba798c2e3b6ecd4b951f8fb1ee1579f6953903" exitCode=0 Dec 03 12:20:55 crc kubenswrapper[5052]: I1203 12:20:55.147931 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-conductor-0" event={"ID":"aa2b4c07-e5c7-4543-a0a5-f15b8e1b7f83","Type":"ContainerDied","Data":"4c12bd41f4c701be825345e793ba798c2e3b6ecd4b951f8fb1ee1579f6953903"} Dec 03 12:20:55 crc kubenswrapper[5052]: I1203 12:20:55.147968 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-conductor-0" event={"ID":"aa2b4c07-e5c7-4543-a0a5-f15b8e1b7f83","Type":"ContainerDied","Data":"272469651cbe8cb9d64d3d9e777fcd39aedc76908ef2aebc1005c9492df664a4"} Dec 03 12:20:55 crc kubenswrapper[5052]: I1203 12:20:55.148026 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:20:55 crc kubenswrapper[5052]: I1203 12:20:55.163816 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-bootstrap-dqvmm" event={"ID":"63c18817-7663-466c-845d-bc998c5b7093","Type":"ContainerStarted","Data":"4d363d9441eef4d9d65d1bcd8bc7f04f2fe0d45b88e46676a88a9cb0748fb83e"} Dec 03 12:20:55 crc kubenswrapper[5052]: I1203 12:20:55.163912 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-bootstrap-dqvmm" event={"ID":"63c18817-7663-466c-845d-bc998c5b7093","Type":"ContainerStarted","Data":"ab04e7b8ee1e1ecddfd758a69caeb2cfdd73c7299872f1df1c46bd77dbb02492"} Dec 03 12:20:55 crc kubenswrapper[5052]: I1203 12:20:55.166362 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-cell0-conductor-0" podStartSLOduration=2.166349681 podStartE2EDuration="2.166349681s" podCreationTimestamp="2025-12-03 12:20:53 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:20:55.163890001 +0000 UTC m=+3645.752131765" watchObservedRunningTime="2025-12-03 12:20:55.166349681 +0000 UTC m=+3645.754591445" Dec 03 12:20:55 crc kubenswrapper[5052]: I1203 12:20:55.197431 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/ceilometer-0" podStartSLOduration=3.256189776 podStartE2EDuration="8.197412658s" podCreationTimestamp="2025-12-03 12:20:47 +0000 UTC" firstStartedPulling="2025-12-03 12:20:48.813194439 +0000 UTC m=+3639.401436203" lastFinishedPulling="2025-12-03 12:20:53.754417321 +0000 UTC m=+3644.342659085" observedRunningTime="2025-12-03 12:20:55.187388842 +0000 UTC m=+3645.775630616" watchObservedRunningTime="2025-12-03 12:20:55.197412658 +0000 UTC m=+3645.785654422" Dec 03 12:20:55 crc kubenswrapper[5052]: I1203 12:20:55.248996 5052 scope.go:117] "RemoveContainer" containerID="9ca86e437e38c5642f8640cedce80dfac23a56655784a6b7769c0317c467d085" Dec 03 12:20:55 crc kubenswrapper[5052]: I1203 12:20:55.252854 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/swift-storage-0"] Dec 03 12:20:55 crc kubenswrapper[5052]: I1203 12:20:55.355653 5052 scope.go:117] "RemoveContainer" containerID="2660887ea0a6145f5154899fe6bb77bb7f4491a21de6c7447f3c126adc776305" Dec 03 12:20:55 crc kubenswrapper[5052]: I1203 12:20:55.355850 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/swift-storage-0"] Dec 03 12:20:55 crc kubenswrapper[5052]: I1203 12:20:55.368814 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/keystone-bootstrap-dqvmm" podStartSLOduration=2.3687896889999998 podStartE2EDuration="2.368789689s" podCreationTimestamp="2025-12-03 12:20:53 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:20:55.250175454 +0000 UTC m=+3645.838417238" watchObservedRunningTime="2025-12-03 12:20:55.368789689 +0000 UTC m=+3645.957031463" Dec 03 12:20:55 crc kubenswrapper[5052]: I1203 12:20:55.420890 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/swift-storage-0"] Dec 03 12:20:55 crc kubenswrapper[5052]: E1203 12:20:55.421611 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="335ad996-ad3b-4ac8-a488-64e9453f24a4" containerName="container-replicator" Dec 03 12:20:55 crc kubenswrapper[5052]: I1203 12:20:55.421711 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="335ad996-ad3b-4ac8-a488-64e9453f24a4" containerName="container-replicator" Dec 03 12:20:55 crc kubenswrapper[5052]: E1203 12:20:55.421799 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="335ad996-ad3b-4ac8-a488-64e9453f24a4" containerName="object-expirer" Dec 03 12:20:55 crc kubenswrapper[5052]: I1203 12:20:55.421876 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="335ad996-ad3b-4ac8-a488-64e9453f24a4" containerName="object-expirer" Dec 03 12:20:55 crc kubenswrapper[5052]: E1203 12:20:55.421951 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="335ad996-ad3b-4ac8-a488-64e9453f24a4" containerName="object-auditor" Dec 03 12:20:55 crc kubenswrapper[5052]: I1203 12:20:55.422049 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="335ad996-ad3b-4ac8-a488-64e9453f24a4" containerName="object-auditor" Dec 03 12:20:55 crc kubenswrapper[5052]: E1203 12:20:55.422129 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="335ad996-ad3b-4ac8-a488-64e9453f24a4" containerName="container-auditor" Dec 03 12:20:55 crc kubenswrapper[5052]: I1203 12:20:55.422210 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="335ad996-ad3b-4ac8-a488-64e9453f24a4" containerName="container-auditor" Dec 03 12:20:55 crc kubenswrapper[5052]: E1203 12:20:55.422304 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="335ad996-ad3b-4ac8-a488-64e9453f24a4" containerName="account-replicator" Dec 03 12:20:55 crc kubenswrapper[5052]: I1203 12:20:55.422378 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="335ad996-ad3b-4ac8-a488-64e9453f24a4" containerName="account-replicator" Dec 03 12:20:55 crc kubenswrapper[5052]: E1203 12:20:55.422458 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="335ad996-ad3b-4ac8-a488-64e9453f24a4" containerName="object-replicator" Dec 03 12:20:55 crc kubenswrapper[5052]: I1203 12:20:55.422599 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="335ad996-ad3b-4ac8-a488-64e9453f24a4" containerName="object-replicator" Dec 03 12:20:55 crc kubenswrapper[5052]: E1203 12:20:55.422686 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="335ad996-ad3b-4ac8-a488-64e9453f24a4" containerName="object-updater" Dec 03 12:20:55 crc kubenswrapper[5052]: I1203 12:20:55.422763 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="335ad996-ad3b-4ac8-a488-64e9453f24a4" containerName="object-updater" Dec 03 12:20:55 crc kubenswrapper[5052]: E1203 12:20:55.422840 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="335ad996-ad3b-4ac8-a488-64e9453f24a4" containerName="account-auditor" Dec 03 12:20:55 crc kubenswrapper[5052]: I1203 12:20:55.422914 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="335ad996-ad3b-4ac8-a488-64e9453f24a4" containerName="account-auditor" Dec 03 12:20:55 crc kubenswrapper[5052]: E1203 12:20:55.423027 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="335ad996-ad3b-4ac8-a488-64e9453f24a4" containerName="object-server" Dec 03 12:20:55 crc kubenswrapper[5052]: I1203 12:20:55.423103 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="335ad996-ad3b-4ac8-a488-64e9453f24a4" containerName="object-server" Dec 03 12:20:55 crc kubenswrapper[5052]: E1203 12:20:55.423267 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="335ad996-ad3b-4ac8-a488-64e9453f24a4" containerName="container-server" Dec 03 12:20:55 crc kubenswrapper[5052]: I1203 12:20:55.423339 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="335ad996-ad3b-4ac8-a488-64e9453f24a4" containerName="container-server" Dec 03 12:20:55 crc kubenswrapper[5052]: E1203 12:20:55.423419 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="335ad996-ad3b-4ac8-a488-64e9453f24a4" containerName="rsync" Dec 03 12:20:55 crc kubenswrapper[5052]: I1203 12:20:55.423485 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="335ad996-ad3b-4ac8-a488-64e9453f24a4" containerName="rsync" Dec 03 12:20:55 crc kubenswrapper[5052]: E1203 12:20:55.423616 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="335ad996-ad3b-4ac8-a488-64e9453f24a4" containerName="swift-recon-cron" Dec 03 12:20:55 crc kubenswrapper[5052]: I1203 12:20:55.423692 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="335ad996-ad3b-4ac8-a488-64e9453f24a4" containerName="swift-recon-cron" Dec 03 12:20:55 crc kubenswrapper[5052]: E1203 12:20:55.423788 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="335ad996-ad3b-4ac8-a488-64e9453f24a4" containerName="account-server" Dec 03 12:20:55 crc kubenswrapper[5052]: I1203 12:20:55.423859 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="335ad996-ad3b-4ac8-a488-64e9453f24a4" containerName="account-server" Dec 03 12:20:55 crc kubenswrapper[5052]: E1203 12:20:55.423933 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="aa2b4c07-e5c7-4543-a0a5-f15b8e1b7f83" containerName="nova-cell1-conductor-conductor" Dec 03 12:20:55 crc kubenswrapper[5052]: I1203 12:20:55.424025 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa2b4c07-e5c7-4543-a0a5-f15b8e1b7f83" containerName="nova-cell1-conductor-conductor" Dec 03 12:20:55 crc kubenswrapper[5052]: E1203 12:20:55.424098 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="335ad996-ad3b-4ac8-a488-64e9453f24a4" containerName="account-reaper" Dec 03 12:20:55 crc kubenswrapper[5052]: I1203 12:20:55.424169 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="335ad996-ad3b-4ac8-a488-64e9453f24a4" containerName="account-reaper" Dec 03 12:20:55 crc kubenswrapper[5052]: E1203 12:20:55.424252 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="335ad996-ad3b-4ac8-a488-64e9453f24a4" containerName="container-updater" Dec 03 12:20:55 crc kubenswrapper[5052]: I1203 12:20:55.424328 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="335ad996-ad3b-4ac8-a488-64e9453f24a4" containerName="container-updater" Dec 03 12:20:55 crc kubenswrapper[5052]: I1203 12:20:55.424708 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="aa2b4c07-e5c7-4543-a0a5-f15b8e1b7f83" containerName="nova-cell1-conductor-conductor" Dec 03 12:20:55 crc kubenswrapper[5052]: I1203 12:20:55.424809 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="335ad996-ad3b-4ac8-a488-64e9453f24a4" containerName="rsync" Dec 03 12:20:55 crc kubenswrapper[5052]: I1203 12:20:55.424908 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="335ad996-ad3b-4ac8-a488-64e9453f24a4" containerName="account-auditor" Dec 03 12:20:55 crc kubenswrapper[5052]: I1203 12:20:55.424990 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="335ad996-ad3b-4ac8-a488-64e9453f24a4" containerName="container-server" Dec 03 12:20:55 crc kubenswrapper[5052]: I1203 12:20:55.425062 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="335ad996-ad3b-4ac8-a488-64e9453f24a4" containerName="object-server" Dec 03 12:20:55 crc kubenswrapper[5052]: I1203 12:20:55.425148 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="335ad996-ad3b-4ac8-a488-64e9453f24a4" containerName="container-replicator" Dec 03 12:20:55 crc kubenswrapper[5052]: I1203 12:20:55.425229 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="335ad996-ad3b-4ac8-a488-64e9453f24a4" containerName="account-reaper" Dec 03 12:20:55 crc kubenswrapper[5052]: I1203 12:20:55.425304 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="335ad996-ad3b-4ac8-a488-64e9453f24a4" containerName="object-expirer" Dec 03 12:20:55 crc kubenswrapper[5052]: I1203 12:20:55.425385 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="335ad996-ad3b-4ac8-a488-64e9453f24a4" containerName="account-server" Dec 03 12:20:55 crc kubenswrapper[5052]: I1203 12:20:55.425457 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="335ad996-ad3b-4ac8-a488-64e9453f24a4" containerName="object-auditor" Dec 03 12:20:55 crc kubenswrapper[5052]: I1203 12:20:55.425558 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="335ad996-ad3b-4ac8-a488-64e9453f24a4" containerName="container-updater" Dec 03 12:20:55 crc kubenswrapper[5052]: I1203 12:20:55.425655 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="335ad996-ad3b-4ac8-a488-64e9453f24a4" containerName="object-replicator" Dec 03 12:20:55 crc kubenswrapper[5052]: I1203 12:20:55.425758 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="335ad996-ad3b-4ac8-a488-64e9453f24a4" containerName="swift-recon-cron" Dec 03 12:20:55 crc kubenswrapper[5052]: I1203 12:20:55.425846 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="335ad996-ad3b-4ac8-a488-64e9453f24a4" containerName="account-replicator" Dec 03 12:20:55 crc kubenswrapper[5052]: I1203 12:20:55.425923 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="335ad996-ad3b-4ac8-a488-64e9453f24a4" containerName="object-updater" Dec 03 12:20:55 crc kubenswrapper[5052]: I1203 12:20:55.426001 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="335ad996-ad3b-4ac8-a488-64e9453f24a4" containerName="container-auditor" Dec 03 12:20:55 crc kubenswrapper[5052]: I1203 12:20:55.435859 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/swift-storage-0"] Dec 03 12:20:55 crc kubenswrapper[5052]: I1203 12:20:55.436214 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:20:55 crc kubenswrapper[5052]: W1203 12:20:55.449890 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9119635e_cb55_4428_9c61_b93ef2afd799.slice/crio-4e1a1d86a4b1a8e7ad13fe35c40a03629e8393c18875d941b6004f1f25ed058e WatchSource:0}: Error finding container 4e1a1d86a4b1a8e7ad13fe35c40a03629e8393c18875d941b6004f1f25ed058e: Status 404 returned error can't find the container with id 4e1a1d86a4b1a8e7ad13fe35c40a03629e8393c18875d941b6004f1f25ed058e Dec 03 12:20:55 crc kubenswrapper[5052]: I1203 12:20:55.451154 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"swift-storage-config-data" Dec 03 12:20:55 crc kubenswrapper[5052]: I1203 12:20:55.453431 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-0"] Dec 03 12:20:55 crc kubenswrapper[5052]: I1203 12:20:55.466352 5052 scope.go:117] "RemoveContainer" containerID="486c4efda615830f80b3292087835a1464d89e6438a42bab394e0f23416c0683" Dec 03 12:20:55 crc kubenswrapper[5052]: I1203 12:20:55.467731 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-0"] Dec 03 12:20:55 crc kubenswrapper[5052]: I1203 12:20:55.486780 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-0"] Dec 03 12:20:55 crc kubenswrapper[5052]: I1203 12:20:55.489714 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:20:55 crc kubenswrapper[5052]: I1203 12:20:55.492049 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell1-conductor-config-data" Dec 03 12:20:55 crc kubenswrapper[5052]: I1203 12:20:55.509829 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-0"] Dec 03 12:20:55 crc kubenswrapper[5052]: I1203 12:20:55.529706 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cache\" (UniqueName: \"kubernetes.io/empty-dir/0b121234-d24f-4307-9409-59a7dad3df4c-cache\") pod \"swift-storage-0\" (UID: \"0b121234-d24f-4307-9409-59a7dad3df4c\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:20:55 crc kubenswrapper[5052]: I1203 12:20:55.529746 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/0b121234-d24f-4307-9409-59a7dad3df4c-etc-swift\") pod \"swift-storage-0\" (UID: \"0b121234-d24f-4307-9409-59a7dad3df4c\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:20:55 crc kubenswrapper[5052]: I1203 12:20:55.529775 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage05-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage05-crc\") pod \"swift-storage-0\" (UID: \"0b121234-d24f-4307-9409-59a7dad3df4c\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:20:55 crc kubenswrapper[5052]: I1203 12:20:55.529845 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kq22z\" (UniqueName: \"kubernetes.io/projected/a7907bc2-8e73-4272-b779-fd71d4acb1f4-kube-api-access-kq22z\") pod \"nova-cell1-conductor-0\" (UID: \"a7907bc2-8e73-4272-b779-fd71d4acb1f4\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:20:55 crc kubenswrapper[5052]: I1203 12:20:55.529876 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-88lpb\" (UniqueName: \"kubernetes.io/projected/0b121234-d24f-4307-9409-59a7dad3df4c-kube-api-access-88lpb\") pod \"swift-storage-0\" (UID: \"0b121234-d24f-4307-9409-59a7dad3df4c\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:20:55 crc kubenswrapper[5052]: I1203 12:20:55.529900 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lock\" (UniqueName: \"kubernetes.io/empty-dir/0b121234-d24f-4307-9409-59a7dad3df4c-lock\") pod \"swift-storage-0\" (UID: \"0b121234-d24f-4307-9409-59a7dad3df4c\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:20:55 crc kubenswrapper[5052]: I1203 12:20:55.529932 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a7907bc2-8e73-4272-b779-fd71d4acb1f4-config-data\") pod \"nova-cell1-conductor-0\" (UID: \"a7907bc2-8e73-4272-b779-fd71d4acb1f4\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:20:55 crc kubenswrapper[5052]: I1203 12:20:55.529961 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a7907bc2-8e73-4272-b779-fd71d4acb1f4-combined-ca-bundle\") pod \"nova-cell1-conductor-0\" (UID: \"a7907bc2-8e73-4272-b779-fd71d4acb1f4\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:20:55 crc kubenswrapper[5052]: I1203 12:20:55.545814 5052 scope.go:117] "RemoveContainer" containerID="2b57d07d8ebd7db5a1f0aa4b4ebb75c36047390ad674b3a41f27874f7e01c7b5" Dec 03 12:20:55 crc kubenswrapper[5052]: I1203 12:20:55.545810 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-keystone-listener-6fb665b666-ftpjr"] Dec 03 12:20:55 crc kubenswrapper[5052]: I1203 12:20:55.634319 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a7907bc2-8e73-4272-b779-fd71d4acb1f4-combined-ca-bundle\") pod \"nova-cell1-conductor-0\" (UID: \"a7907bc2-8e73-4272-b779-fd71d4acb1f4\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:20:55 crc kubenswrapper[5052]: I1203 12:20:55.634559 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"cache\" (UniqueName: \"kubernetes.io/empty-dir/0b121234-d24f-4307-9409-59a7dad3df4c-cache\") pod \"swift-storage-0\" (UID: \"0b121234-d24f-4307-9409-59a7dad3df4c\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:20:55 crc kubenswrapper[5052]: I1203 12:20:55.634643 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/0b121234-d24f-4307-9409-59a7dad3df4c-etc-swift\") pod \"swift-storage-0\" (UID: \"0b121234-d24f-4307-9409-59a7dad3df4c\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:20:55 crc kubenswrapper[5052]: I1203 12:20:55.634716 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage05-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage05-crc\") pod \"swift-storage-0\" (UID: \"0b121234-d24f-4307-9409-59a7dad3df4c\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:20:55 crc kubenswrapper[5052]: I1203 12:20:55.635387 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage05-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage05-crc\") pod \"swift-storage-0\" (UID: \"0b121234-d24f-4307-9409-59a7dad3df4c\") device mount path \"/mnt/openstack/pv05\"" pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:20:55 crc kubenswrapper[5052]: I1203 12:20:55.637186 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"cache\" (UniqueName: \"kubernetes.io/empty-dir/0b121234-d24f-4307-9409-59a7dad3df4c-cache\") pod \"swift-storage-0\" (UID: \"0b121234-d24f-4307-9409-59a7dad3df4c\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:20:55 crc kubenswrapper[5052]: I1203 12:20:55.639313 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a7907bc2-8e73-4272-b779-fd71d4acb1f4-combined-ca-bundle\") pod \"nova-cell1-conductor-0\" (UID: \"a7907bc2-8e73-4272-b779-fd71d4acb1f4\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:20:55 crc kubenswrapper[5052]: I1203 12:20:55.640380 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-kq22z\" (UniqueName: \"kubernetes.io/projected/a7907bc2-8e73-4272-b779-fd71d4acb1f4-kube-api-access-kq22z\") pod \"nova-cell1-conductor-0\" (UID: \"a7907bc2-8e73-4272-b779-fd71d4acb1f4\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:20:55 crc kubenswrapper[5052]: I1203 12:20:55.640506 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-88lpb\" (UniqueName: \"kubernetes.io/projected/0b121234-d24f-4307-9409-59a7dad3df4c-kube-api-access-88lpb\") pod \"swift-storage-0\" (UID: \"0b121234-d24f-4307-9409-59a7dad3df4c\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:20:55 crc kubenswrapper[5052]: I1203 12:20:55.640663 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"lock\" (UniqueName: \"kubernetes.io/empty-dir/0b121234-d24f-4307-9409-59a7dad3df4c-lock\") pod \"swift-storage-0\" (UID: \"0b121234-d24f-4307-9409-59a7dad3df4c\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:20:55 crc kubenswrapper[5052]: I1203 12:20:55.640792 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a7907bc2-8e73-4272-b779-fd71d4acb1f4-config-data\") pod \"nova-cell1-conductor-0\" (UID: \"a7907bc2-8e73-4272-b779-fd71d4acb1f4\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:20:55 crc kubenswrapper[5052]: I1203 12:20:55.656381 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"lock\" (UniqueName: \"kubernetes.io/empty-dir/0b121234-d24f-4307-9409-59a7dad3df4c-lock\") pod \"swift-storage-0\" (UID: \"0b121234-d24f-4307-9409-59a7dad3df4c\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:20:55 crc kubenswrapper[5052]: I1203 12:20:55.666226 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/0b121234-d24f-4307-9409-59a7dad3df4c-etc-swift\") pod \"swift-storage-0\" (UID: \"0b121234-d24f-4307-9409-59a7dad3df4c\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:20:55 crc kubenswrapper[5052]: I1203 12:20:55.667450 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-88lpb\" (UniqueName: \"kubernetes.io/projected/0b121234-d24f-4307-9409-59a7dad3df4c-kube-api-access-88lpb\") pod \"swift-storage-0\" (UID: \"0b121234-d24f-4307-9409-59a7dad3df4c\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:20:55 crc kubenswrapper[5052]: I1203 12:20:55.671168 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-api-644f564b46-cjtkx"] Dec 03 12:20:55 crc kubenswrapper[5052]: I1203 12:20:55.671755 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a7907bc2-8e73-4272-b779-fd71d4acb1f4-config-data\") pod \"nova-cell1-conductor-0\" (UID: \"a7907bc2-8e73-4272-b779-fd71d4acb1f4\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:20:55 crc kubenswrapper[5052]: I1203 12:20:55.689717 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-kq22z\" (UniqueName: \"kubernetes.io/projected/a7907bc2-8e73-4272-b779-fd71d4acb1f4-kube-api-access-kq22z\") pod \"nova-cell1-conductor-0\" (UID: \"a7907bc2-8e73-4272-b779-fd71d4acb1f4\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:20:55 crc kubenswrapper[5052]: I1203 12:20:55.698095 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="335ad996-ad3b-4ac8-a488-64e9453f24a4" path="/var/lib/kubelet/pods/335ad996-ad3b-4ac8-a488-64e9453f24a4/volumes" Dec 03 12:20:55 crc kubenswrapper[5052]: I1203 12:20:55.700045 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aa2b4c07-e5c7-4543-a0a5-f15b8e1b7f83" path="/var/lib/kubelet/pods/aa2b4c07-e5c7-4543-a0a5-f15b8e1b7f83/volumes" Dec 03 12:20:55 crc kubenswrapper[5052]: I1203 12:20:55.720882 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage05-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage05-crc\") pod \"swift-storage-0\" (UID: \"0b121234-d24f-4307-9409-59a7dad3df4c\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:20:55 crc kubenswrapper[5052]: I1203 12:20:55.762380 5052 scope.go:117] "RemoveContainer" containerID="2d0cbf82dccb7f04f14e9a8c02dce57f3fe56aa4bea37bafafd0b0e8ef2825ff" Dec 03 12:20:55 crc kubenswrapper[5052]: I1203 12:20:55.781256 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:20:55 crc kubenswrapper[5052]: I1203 12:20:55.782161 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-db-sync-pvgq9" Dec 03 12:20:55 crc kubenswrapper[5052]: I1203 12:20:55.810200 5052 scope.go:117] "RemoveContainer" containerID="2591ce792243cc1fe842d92ce9a53f946994c1f42c26f9657e9e29db8d53f707" Dec 03 12:20:55 crc kubenswrapper[5052]: I1203 12:20:55.829196 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:20:55 crc kubenswrapper[5052]: I1203 12:20:55.905905 5052 scope.go:117] "RemoveContainer" containerID="55c422ce2a089bbc2d4cce09c586d3b79a3abf94fbea99b43ef8ff9d803269d4" Dec 03 12:20:55 crc kubenswrapper[5052]: I1203 12:20:55.945802 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/66f5168a-1ee5-4de9-8e50-90ef25186ee8-db-sync-config-data\") pod \"66f5168a-1ee5-4de9-8e50-90ef25186ee8\" (UID: \"66f5168a-1ee5-4de9-8e50-90ef25186ee8\") " Dec 03 12:20:55 crc kubenswrapper[5052]: I1203 12:20:55.946101 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wlsk7\" (UniqueName: \"kubernetes.io/projected/66f5168a-1ee5-4de9-8e50-90ef25186ee8-kube-api-access-wlsk7\") pod \"66f5168a-1ee5-4de9-8e50-90ef25186ee8\" (UID: \"66f5168a-1ee5-4de9-8e50-90ef25186ee8\") " Dec 03 12:20:55 crc kubenswrapper[5052]: I1203 12:20:55.946142 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/66f5168a-1ee5-4de9-8e50-90ef25186ee8-config-data\") pod \"66f5168a-1ee5-4de9-8e50-90ef25186ee8\" (UID: \"66f5168a-1ee5-4de9-8e50-90ef25186ee8\") " Dec 03 12:20:55 crc kubenswrapper[5052]: I1203 12:20:55.946279 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/66f5168a-1ee5-4de9-8e50-90ef25186ee8-combined-ca-bundle\") pod \"66f5168a-1ee5-4de9-8e50-90ef25186ee8\" (UID: \"66f5168a-1ee5-4de9-8e50-90ef25186ee8\") " Dec 03 12:20:55 crc kubenswrapper[5052]: I1203 12:20:55.956970 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/66f5168a-1ee5-4de9-8e50-90ef25186ee8-db-sync-config-data" (OuterVolumeSpecName: "db-sync-config-data") pod "66f5168a-1ee5-4de9-8e50-90ef25186ee8" (UID: "66f5168a-1ee5-4de9-8e50-90ef25186ee8"). InnerVolumeSpecName "db-sync-config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:20:55 crc kubenswrapper[5052]: I1203 12:20:55.959774 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/66f5168a-1ee5-4de9-8e50-90ef25186ee8-kube-api-access-wlsk7" (OuterVolumeSpecName: "kube-api-access-wlsk7") pod "66f5168a-1ee5-4de9-8e50-90ef25186ee8" (UID: "66f5168a-1ee5-4de9-8e50-90ef25186ee8"). InnerVolumeSpecName "kube-api-access-wlsk7". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:20:55 crc kubenswrapper[5052]: I1203 12:20:55.973703 5052 scope.go:117] "RemoveContainer" containerID="000974bff3024c4dd8ae265f8fc3dc2cd023ab0097daaa4fd6b13c365cb5dd62" Dec 03 12:20:55 crc kubenswrapper[5052]: I1203 12:20:55.988013 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/66f5168a-1ee5-4de9-8e50-90ef25186ee8-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "66f5168a-1ee5-4de9-8e50-90ef25186ee8" (UID: "66f5168a-1ee5-4de9-8e50-90ef25186ee8"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:20:56 crc kubenswrapper[5052]: I1203 12:20:56.041717 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/66f5168a-1ee5-4de9-8e50-90ef25186ee8-config-data" (OuterVolumeSpecName: "config-data") pod "66f5168a-1ee5-4de9-8e50-90ef25186ee8" (UID: "66f5168a-1ee5-4de9-8e50-90ef25186ee8"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:20:56 crc kubenswrapper[5052]: I1203 12:20:56.048276 5052 reconciler_common.go:293] "Volume detached for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/66f5168a-1ee5-4de9-8e50-90ef25186ee8-db-sync-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:56 crc kubenswrapper[5052]: I1203 12:20:56.048305 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-wlsk7\" (UniqueName: \"kubernetes.io/projected/66f5168a-1ee5-4de9-8e50-90ef25186ee8-kube-api-access-wlsk7\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:56 crc kubenswrapper[5052]: I1203 12:20:56.048316 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/66f5168a-1ee5-4de9-8e50-90ef25186ee8-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:56 crc kubenswrapper[5052]: I1203 12:20:56.048324 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/66f5168a-1ee5-4de9-8e50-90ef25186ee8-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:20:56 crc kubenswrapper[5052]: I1203 12:20:56.057745 5052 scope.go:117] "RemoveContainer" containerID="e1a2b505ab3bea37930a1c222d85a3ab5421377ce504e97bef52966f349f5e26" Dec 03 12:20:56 crc kubenswrapper[5052]: I1203 12:20:56.142600 5052 scope.go:117] "RemoveContainer" containerID="6af612b56d9c2f4b731475f24619368f730783d7ef872ccc130c33f5a189e57f" Dec 03 12:20:56 crc kubenswrapper[5052]: I1203 12:20:56.195890 5052 scope.go:117] "RemoveContainer" containerID="f7f12231dd030c2c9c3f43d8493e609bd58faa9dd7c14aed345ea597010d1451" Dec 03 12:20:56 crc kubenswrapper[5052]: I1203 12:20:56.267191 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-db-sync-pvgq9" event={"ID":"66f5168a-1ee5-4de9-8e50-90ef25186ee8","Type":"ContainerDied","Data":"52fde149577b09c34ef00f62971a8730a9ea8cb1951a6aaf40880c92177d564e"} Dec 03 12:20:56 crc kubenswrapper[5052]: I1203 12:20:56.267387 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="52fde149577b09c34ef00f62971a8730a9ea8cb1951a6aaf40880c92177d564e" Dec 03 12:20:56 crc kubenswrapper[5052]: I1203 12:20:56.267458 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:20:56 crc kubenswrapper[5052]: I1203 12:20:56.267731 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-external-api-0" podUID="736b6178-1b32-4f5e-a207-931b03aed990" containerName="glance-log" containerID="cri-o://e480cf5faa382d0f7dc902f72f5fa5a78a13cd4fac390962daf40b8d0bee88d7" gracePeriod=30 Dec 03 12:20:56 crc kubenswrapper[5052]: I1203 12:20:56.268077 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-db-sync-pvgq9" Dec 03 12:20:56 crc kubenswrapper[5052]: I1203 12:20:56.269614 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-external-api-0" podUID="736b6178-1b32-4f5e-a207-931b03aed990" containerName="glance-httpd" containerID="cri-o://08ba12ac062944881ab39ae9fbebc472a569ed1780e7c9436d3f4c6a4444d057" gracePeriod=30 Dec 03 12:20:56 crc kubenswrapper[5052]: I1203 12:20:56.275418 5052 scope.go:117] "RemoveContainer" containerID="d801a1f1f45c17b5422af1ea12c95b38c64a258c49f3953222cbb94387e19d2c" Dec 03 12:20:56 crc kubenswrapper[5052]: I1203 12:20:56.286420 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-644f564b46-cjtkx" event={"ID":"af38eb0d-3924-4c4b-a22e-7e934bdcf172","Type":"ContainerStarted","Data":"e699b862f9953c2e25b19ab9d64356e9c8a7220759bf33ca05a10010b2639fc3"} Dec 03 12:20:56 crc kubenswrapper[5052]: I1203 12:20:56.286460 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-644f564b46-cjtkx" event={"ID":"af38eb0d-3924-4c4b-a22e-7e934bdcf172","Type":"ContainerStarted","Data":"f8398124a351883182a32783ea01603696a9bae2ae988e76ddf3651290914f19"} Dec 03 12:20:56 crc kubenswrapper[5052]: I1203 12:20:56.288551 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:20:56 crc kubenswrapper[5052]: I1203 12:20:56.288739 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-internal-api-0" podUID="941d2a2e-d51f-4b67-9b4a-6ff075a2eb0f" containerName="glance-log" containerID="cri-o://9f17cf626b688bdd38aa8737ff46f9c764ddd68d168ed599986a4ac6dc81223a" gracePeriod=30 Dec 03 12:20:56 crc kubenswrapper[5052]: I1203 12:20:56.288821 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-internal-api-0" podUID="941d2a2e-d51f-4b67-9b4a-6ff075a2eb0f" containerName="glance-httpd" containerID="cri-o://35b431787e68c1ec9b78768c06706d4afad176de2f59bf713fd162cab978b071" gracePeriod=30 Dec 03 12:20:56 crc kubenswrapper[5052]: I1203 12:20:56.293576 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-keystone-listener-6fb665b666-ftpjr" event={"ID":"9119635e-cb55-4428-9c61-b93ef2afd799","Type":"ContainerStarted","Data":"d006b275534cfdfd7a634f938440db7eeb0d006446de410abf688af6391508cb"} Dec 03 12:20:56 crc kubenswrapper[5052]: I1203 12:20:56.293613 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-keystone-listener-6fb665b666-ftpjr" event={"ID":"9119635e-cb55-4428-9c61-b93ef2afd799","Type":"ContainerStarted","Data":"4e1a1d86a4b1a8e7ad13fe35c40a03629e8393c18875d941b6004f1f25ed058e"} Dec 03 12:20:56 crc kubenswrapper[5052]: I1203 12:20:56.296825 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-worker-6b4674c647-56v84" event={"ID":"484e6cd1-fb57-4416-9a1a-cf24b4693764","Type":"ContainerStarted","Data":"e00bcd0d914f7fbd5f88db32fed6a7e8a0000e44d444198b8f7f1ed7f6ecdb89"} Dec 03 12:20:56 crc kubenswrapper[5052]: I1203 12:20:56.296869 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-worker-6b4674c647-56v84" event={"ID":"484e6cd1-fb57-4416-9a1a-cf24b4693764","Type":"ContainerStarted","Data":"01a0969c60e54d821f713b7a30fadf577164bb47e2f3fc851bc640b779f911d6"} Dec 03 12:20:56 crc kubenswrapper[5052]: I1203 12:20:56.296886 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-worker-6b4674c647-56v84" event={"ID":"484e6cd1-fb57-4416-9a1a-cf24b4693764","Type":"ContainerStarted","Data":"192121d8e29554479b65c83a0f69371b4815e1374a9f01070f02296033b3c0de"} Dec 03 12:20:56 crc kubenswrapper[5052]: I1203 12:20:56.332996 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/barbican-worker-6b4674c647-56v84" podStartSLOduration=2.332975699 podStartE2EDuration="2.332975699s" podCreationTimestamp="2025-12-03 12:20:54 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:20:56.326737901 +0000 UTC m=+3646.914979655" watchObservedRunningTime="2025-12-03 12:20:56.332975699 +0000 UTC m=+3646.921217463" Dec 03 12:20:56 crc kubenswrapper[5052]: I1203 12:20:56.425964 5052 scope.go:117] "RemoveContainer" containerID="df73d6886a1afb1e6d09c9481252bc2871f9a8608386e4ea20b13852f0881a4b" Dec 03 12:20:56 crc kubenswrapper[5052]: I1203 12:20:56.427459 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-worker-76d6bdd486-g9c2p"] Dec 03 12:20:56 crc kubenswrapper[5052]: I1203 12:20:56.428627 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/barbican-worker-76d6bdd486-g9c2p" podUID="311e25fa-0785-45ce-ae3c-00d32adbf3f4" containerName="barbican-worker-log" containerID="cri-o://522e7f93cdd131cc57ae189abb416b435302969048510ba07854b6d472d399d4" gracePeriod=30 Dec 03 12:20:56 crc kubenswrapper[5052]: I1203 12:20:56.429001 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/barbican-worker-76d6bdd486-g9c2p" podUID="311e25fa-0785-45ce-ae3c-00d32adbf3f4" containerName="barbican-worker" containerID="cri-o://b7e1028dbd6661eea3ae89a9c9f561e69d762520ab29fe9c2720e56e95408280" gracePeriod=30 Dec 03 12:20:56 crc kubenswrapper[5052]: I1203 12:20:56.471000 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-0"] Dec 03 12:20:56 crc kubenswrapper[5052]: W1203 12:20:56.531368 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda7907bc2_8e73_4272_b779_fd71d4acb1f4.slice/crio-5c18996aaae6a85aceef22fa6d8699403c2e1d986214568e260b342877e4010d WatchSource:0}: Error finding container 5c18996aaae6a85aceef22fa6d8699403c2e1d986214568e260b342877e4010d: Status 404 returned error can't find the container with id 5c18996aaae6a85aceef22fa6d8699403c2e1d986214568e260b342877e4010d Dec 03 12:20:56 crc kubenswrapper[5052]: I1203 12:20:56.639674 5052 scope.go:117] "RemoveContainer" containerID="006d1e418590929322462448d037f47e0e4031553a716f0f323b0ceb29ae6857" Dec 03 12:20:56 crc kubenswrapper[5052]: I1203 12:20:56.691112 5052 scope.go:117] "RemoveContainer" containerID="fb2b07929be7f2dd4d6fd56e12955638b93484338cfedea07147fa092b946832" Dec 03 12:20:56 crc kubenswrapper[5052]: E1203 12:20:56.697962 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fb2b07929be7f2dd4d6fd56e12955638b93484338cfedea07147fa092b946832\": container with ID starting with fb2b07929be7f2dd4d6fd56e12955638b93484338cfedea07147fa092b946832 not found: ID does not exist" containerID="fb2b07929be7f2dd4d6fd56e12955638b93484338cfedea07147fa092b946832" Dec 03 12:20:56 crc kubenswrapper[5052]: I1203 12:20:56.701811 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fb2b07929be7f2dd4d6fd56e12955638b93484338cfedea07147fa092b946832"} err="failed to get container status \"fb2b07929be7f2dd4d6fd56e12955638b93484338cfedea07147fa092b946832\": rpc error: code = NotFound desc = could not find container \"fb2b07929be7f2dd4d6fd56e12955638b93484338cfedea07147fa092b946832\": container with ID starting with fb2b07929be7f2dd4d6fd56e12955638b93484338cfedea07147fa092b946832 not found: ID does not exist" Dec 03 12:20:56 crc kubenswrapper[5052]: I1203 12:20:56.702005 5052 scope.go:117] "RemoveContainer" containerID="9ca86e437e38c5642f8640cedce80dfac23a56655784a6b7769c0317c467d085" Dec 03 12:20:56 crc kubenswrapper[5052]: E1203 12:20:56.706337 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9ca86e437e38c5642f8640cedce80dfac23a56655784a6b7769c0317c467d085\": container with ID starting with 9ca86e437e38c5642f8640cedce80dfac23a56655784a6b7769c0317c467d085 not found: ID does not exist" containerID="9ca86e437e38c5642f8640cedce80dfac23a56655784a6b7769c0317c467d085" Dec 03 12:20:56 crc kubenswrapper[5052]: I1203 12:20:56.706549 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9ca86e437e38c5642f8640cedce80dfac23a56655784a6b7769c0317c467d085"} err="failed to get container status \"9ca86e437e38c5642f8640cedce80dfac23a56655784a6b7769c0317c467d085\": rpc error: code = NotFound desc = could not find container \"9ca86e437e38c5642f8640cedce80dfac23a56655784a6b7769c0317c467d085\": container with ID starting with 9ca86e437e38c5642f8640cedce80dfac23a56655784a6b7769c0317c467d085 not found: ID does not exist" Dec 03 12:20:56 crc kubenswrapper[5052]: I1203 12:20:56.706649 5052 scope.go:117] "RemoveContainer" containerID="2660887ea0a6145f5154899fe6bb77bb7f4491a21de6c7447f3c126adc776305" Dec 03 12:20:56 crc kubenswrapper[5052]: E1203 12:20:56.707097 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2660887ea0a6145f5154899fe6bb77bb7f4491a21de6c7447f3c126adc776305\": container with ID starting with 2660887ea0a6145f5154899fe6bb77bb7f4491a21de6c7447f3c126adc776305 not found: ID does not exist" containerID="2660887ea0a6145f5154899fe6bb77bb7f4491a21de6c7447f3c126adc776305" Dec 03 12:20:56 crc kubenswrapper[5052]: I1203 12:20:56.707225 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2660887ea0a6145f5154899fe6bb77bb7f4491a21de6c7447f3c126adc776305"} err="failed to get container status \"2660887ea0a6145f5154899fe6bb77bb7f4491a21de6c7447f3c126adc776305\": rpc error: code = NotFound desc = could not find container \"2660887ea0a6145f5154899fe6bb77bb7f4491a21de6c7447f3c126adc776305\": container with ID starting with 2660887ea0a6145f5154899fe6bb77bb7f4491a21de6c7447f3c126adc776305 not found: ID does not exist" Dec 03 12:20:56 crc kubenswrapper[5052]: I1203 12:20:56.707310 5052 scope.go:117] "RemoveContainer" containerID="486c4efda615830f80b3292087835a1464d89e6438a42bab394e0f23416c0683" Dec 03 12:20:56 crc kubenswrapper[5052]: E1203 12:20:56.707625 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"486c4efda615830f80b3292087835a1464d89e6438a42bab394e0f23416c0683\": container with ID starting with 486c4efda615830f80b3292087835a1464d89e6438a42bab394e0f23416c0683 not found: ID does not exist" containerID="486c4efda615830f80b3292087835a1464d89e6438a42bab394e0f23416c0683" Dec 03 12:20:56 crc kubenswrapper[5052]: I1203 12:20:56.707717 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"486c4efda615830f80b3292087835a1464d89e6438a42bab394e0f23416c0683"} err="failed to get container status \"486c4efda615830f80b3292087835a1464d89e6438a42bab394e0f23416c0683\": rpc error: code = NotFound desc = could not find container \"486c4efda615830f80b3292087835a1464d89e6438a42bab394e0f23416c0683\": container with ID starting with 486c4efda615830f80b3292087835a1464d89e6438a42bab394e0f23416c0683 not found: ID does not exist" Dec 03 12:20:56 crc kubenswrapper[5052]: I1203 12:20:56.707798 5052 scope.go:117] "RemoveContainer" containerID="2b57d07d8ebd7db5a1f0aa4b4ebb75c36047390ad674b3a41f27874f7e01c7b5" Dec 03 12:20:56 crc kubenswrapper[5052]: E1203 12:20:56.708178 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2b57d07d8ebd7db5a1f0aa4b4ebb75c36047390ad674b3a41f27874f7e01c7b5\": container with ID starting with 2b57d07d8ebd7db5a1f0aa4b4ebb75c36047390ad674b3a41f27874f7e01c7b5 not found: ID does not exist" containerID="2b57d07d8ebd7db5a1f0aa4b4ebb75c36047390ad674b3a41f27874f7e01c7b5" Dec 03 12:20:56 crc kubenswrapper[5052]: I1203 12:20:56.708276 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2b57d07d8ebd7db5a1f0aa4b4ebb75c36047390ad674b3a41f27874f7e01c7b5"} err="failed to get container status \"2b57d07d8ebd7db5a1f0aa4b4ebb75c36047390ad674b3a41f27874f7e01c7b5\": rpc error: code = NotFound desc = could not find container \"2b57d07d8ebd7db5a1f0aa4b4ebb75c36047390ad674b3a41f27874f7e01c7b5\": container with ID starting with 2b57d07d8ebd7db5a1f0aa4b4ebb75c36047390ad674b3a41f27874f7e01c7b5 not found: ID does not exist" Dec 03 12:20:56 crc kubenswrapper[5052]: I1203 12:20:56.708354 5052 scope.go:117] "RemoveContainer" containerID="2d0cbf82dccb7f04f14e9a8c02dce57f3fe56aa4bea37bafafd0b0e8ef2825ff" Dec 03 12:20:56 crc kubenswrapper[5052]: E1203 12:20:56.708634 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2d0cbf82dccb7f04f14e9a8c02dce57f3fe56aa4bea37bafafd0b0e8ef2825ff\": container with ID starting with 2d0cbf82dccb7f04f14e9a8c02dce57f3fe56aa4bea37bafafd0b0e8ef2825ff not found: ID does not exist" containerID="2d0cbf82dccb7f04f14e9a8c02dce57f3fe56aa4bea37bafafd0b0e8ef2825ff" Dec 03 12:20:56 crc kubenswrapper[5052]: I1203 12:20:56.708724 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d0cbf82dccb7f04f14e9a8c02dce57f3fe56aa4bea37bafafd0b0e8ef2825ff"} err="failed to get container status \"2d0cbf82dccb7f04f14e9a8c02dce57f3fe56aa4bea37bafafd0b0e8ef2825ff\": rpc error: code = NotFound desc = could not find container \"2d0cbf82dccb7f04f14e9a8c02dce57f3fe56aa4bea37bafafd0b0e8ef2825ff\": container with ID starting with 2d0cbf82dccb7f04f14e9a8c02dce57f3fe56aa4bea37bafafd0b0e8ef2825ff not found: ID does not exist" Dec 03 12:20:56 crc kubenswrapper[5052]: I1203 12:20:56.708815 5052 scope.go:117] "RemoveContainer" containerID="2591ce792243cc1fe842d92ce9a53f946994c1f42c26f9657e9e29db8d53f707" Dec 03 12:20:56 crc kubenswrapper[5052]: E1203 12:20:56.713724 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2591ce792243cc1fe842d92ce9a53f946994c1f42c26f9657e9e29db8d53f707\": container with ID starting with 2591ce792243cc1fe842d92ce9a53f946994c1f42c26f9657e9e29db8d53f707 not found: ID does not exist" containerID="2591ce792243cc1fe842d92ce9a53f946994c1f42c26f9657e9e29db8d53f707" Dec 03 12:20:56 crc kubenswrapper[5052]: I1203 12:20:56.713914 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2591ce792243cc1fe842d92ce9a53f946994c1f42c26f9657e9e29db8d53f707"} err="failed to get container status \"2591ce792243cc1fe842d92ce9a53f946994c1f42c26f9657e9e29db8d53f707\": rpc error: code = NotFound desc = could not find container \"2591ce792243cc1fe842d92ce9a53f946994c1f42c26f9657e9e29db8d53f707\": container with ID starting with 2591ce792243cc1fe842d92ce9a53f946994c1f42c26f9657e9e29db8d53f707 not found: ID does not exist" Dec 03 12:20:56 crc kubenswrapper[5052]: I1203 12:20:56.714008 5052 scope.go:117] "RemoveContainer" containerID="55c422ce2a089bbc2d4cce09c586d3b79a3abf94fbea99b43ef8ff9d803269d4" Dec 03 12:20:56 crc kubenswrapper[5052]: E1203 12:20:56.715527 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"55c422ce2a089bbc2d4cce09c586d3b79a3abf94fbea99b43ef8ff9d803269d4\": container with ID starting with 55c422ce2a089bbc2d4cce09c586d3b79a3abf94fbea99b43ef8ff9d803269d4 not found: ID does not exist" containerID="55c422ce2a089bbc2d4cce09c586d3b79a3abf94fbea99b43ef8ff9d803269d4" Dec 03 12:20:56 crc kubenswrapper[5052]: I1203 12:20:56.715647 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"55c422ce2a089bbc2d4cce09c586d3b79a3abf94fbea99b43ef8ff9d803269d4"} err="failed to get container status \"55c422ce2a089bbc2d4cce09c586d3b79a3abf94fbea99b43ef8ff9d803269d4\": rpc error: code = NotFound desc = could not find container \"55c422ce2a089bbc2d4cce09c586d3b79a3abf94fbea99b43ef8ff9d803269d4\": container with ID starting with 55c422ce2a089bbc2d4cce09c586d3b79a3abf94fbea99b43ef8ff9d803269d4 not found: ID does not exist" Dec 03 12:20:56 crc kubenswrapper[5052]: I1203 12:20:56.715731 5052 scope.go:117] "RemoveContainer" containerID="000974bff3024c4dd8ae265f8fc3dc2cd023ab0097daaa4fd6b13c365cb5dd62" Dec 03 12:20:56 crc kubenswrapper[5052]: E1203 12:20:56.716020 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"000974bff3024c4dd8ae265f8fc3dc2cd023ab0097daaa4fd6b13c365cb5dd62\": container with ID starting with 000974bff3024c4dd8ae265f8fc3dc2cd023ab0097daaa4fd6b13c365cb5dd62 not found: ID does not exist" containerID="000974bff3024c4dd8ae265f8fc3dc2cd023ab0097daaa4fd6b13c365cb5dd62" Dec 03 12:20:56 crc kubenswrapper[5052]: I1203 12:20:56.716094 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"000974bff3024c4dd8ae265f8fc3dc2cd023ab0097daaa4fd6b13c365cb5dd62"} err="failed to get container status \"000974bff3024c4dd8ae265f8fc3dc2cd023ab0097daaa4fd6b13c365cb5dd62\": rpc error: code = NotFound desc = could not find container \"000974bff3024c4dd8ae265f8fc3dc2cd023ab0097daaa4fd6b13c365cb5dd62\": container with ID starting with 000974bff3024c4dd8ae265f8fc3dc2cd023ab0097daaa4fd6b13c365cb5dd62 not found: ID does not exist" Dec 03 12:20:56 crc kubenswrapper[5052]: I1203 12:20:56.716154 5052 scope.go:117] "RemoveContainer" containerID="e1a2b505ab3bea37930a1c222d85a3ab5421377ce504e97bef52966f349f5e26" Dec 03 12:20:56 crc kubenswrapper[5052]: E1203 12:20:56.716392 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e1a2b505ab3bea37930a1c222d85a3ab5421377ce504e97bef52966f349f5e26\": container with ID starting with e1a2b505ab3bea37930a1c222d85a3ab5421377ce504e97bef52966f349f5e26 not found: ID does not exist" containerID="e1a2b505ab3bea37930a1c222d85a3ab5421377ce504e97bef52966f349f5e26" Dec 03 12:20:56 crc kubenswrapper[5052]: I1203 12:20:56.716472 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e1a2b505ab3bea37930a1c222d85a3ab5421377ce504e97bef52966f349f5e26"} err="failed to get container status \"e1a2b505ab3bea37930a1c222d85a3ab5421377ce504e97bef52966f349f5e26\": rpc error: code = NotFound desc = could not find container \"e1a2b505ab3bea37930a1c222d85a3ab5421377ce504e97bef52966f349f5e26\": container with ID starting with e1a2b505ab3bea37930a1c222d85a3ab5421377ce504e97bef52966f349f5e26 not found: ID does not exist" Dec 03 12:20:56 crc kubenswrapper[5052]: I1203 12:20:56.716646 5052 scope.go:117] "RemoveContainer" containerID="6af612b56d9c2f4b731475f24619368f730783d7ef872ccc130c33f5a189e57f" Dec 03 12:20:56 crc kubenswrapper[5052]: E1203 12:20:56.716959 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6af612b56d9c2f4b731475f24619368f730783d7ef872ccc130c33f5a189e57f\": container with ID starting with 6af612b56d9c2f4b731475f24619368f730783d7ef872ccc130c33f5a189e57f not found: ID does not exist" containerID="6af612b56d9c2f4b731475f24619368f730783d7ef872ccc130c33f5a189e57f" Dec 03 12:20:56 crc kubenswrapper[5052]: I1203 12:20:56.717036 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6af612b56d9c2f4b731475f24619368f730783d7ef872ccc130c33f5a189e57f"} err="failed to get container status \"6af612b56d9c2f4b731475f24619368f730783d7ef872ccc130c33f5a189e57f\": rpc error: code = NotFound desc = could not find container \"6af612b56d9c2f4b731475f24619368f730783d7ef872ccc130c33f5a189e57f\": container with ID starting with 6af612b56d9c2f4b731475f24619368f730783d7ef872ccc130c33f5a189e57f not found: ID does not exist" Dec 03 12:20:56 crc kubenswrapper[5052]: I1203 12:20:56.717100 5052 scope.go:117] "RemoveContainer" containerID="f7f12231dd030c2c9c3f43d8493e609bd58faa9dd7c14aed345ea597010d1451" Dec 03 12:20:56 crc kubenswrapper[5052]: E1203 12:20:56.717356 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f7f12231dd030c2c9c3f43d8493e609bd58faa9dd7c14aed345ea597010d1451\": container with ID starting with f7f12231dd030c2c9c3f43d8493e609bd58faa9dd7c14aed345ea597010d1451 not found: ID does not exist" containerID="f7f12231dd030c2c9c3f43d8493e609bd58faa9dd7c14aed345ea597010d1451" Dec 03 12:20:56 crc kubenswrapper[5052]: I1203 12:20:56.717429 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f7f12231dd030c2c9c3f43d8493e609bd58faa9dd7c14aed345ea597010d1451"} err="failed to get container status \"f7f12231dd030c2c9c3f43d8493e609bd58faa9dd7c14aed345ea597010d1451\": rpc error: code = NotFound desc = could not find container \"f7f12231dd030c2c9c3f43d8493e609bd58faa9dd7c14aed345ea597010d1451\": container with ID starting with f7f12231dd030c2c9c3f43d8493e609bd58faa9dd7c14aed345ea597010d1451 not found: ID does not exist" Dec 03 12:20:56 crc kubenswrapper[5052]: I1203 12:20:56.717488 5052 scope.go:117] "RemoveContainer" containerID="d801a1f1f45c17b5422af1ea12c95b38c64a258c49f3953222cbb94387e19d2c" Dec 03 12:20:56 crc kubenswrapper[5052]: E1203 12:20:56.717884 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d801a1f1f45c17b5422af1ea12c95b38c64a258c49f3953222cbb94387e19d2c\": container with ID starting with d801a1f1f45c17b5422af1ea12c95b38c64a258c49f3953222cbb94387e19d2c not found: ID does not exist" containerID="d801a1f1f45c17b5422af1ea12c95b38c64a258c49f3953222cbb94387e19d2c" Dec 03 12:20:56 crc kubenswrapper[5052]: I1203 12:20:56.718363 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d801a1f1f45c17b5422af1ea12c95b38c64a258c49f3953222cbb94387e19d2c"} err="failed to get container status \"d801a1f1f45c17b5422af1ea12c95b38c64a258c49f3953222cbb94387e19d2c\": rpc error: code = NotFound desc = could not find container \"d801a1f1f45c17b5422af1ea12c95b38c64a258c49f3953222cbb94387e19d2c\": container with ID starting with d801a1f1f45c17b5422af1ea12c95b38c64a258c49f3953222cbb94387e19d2c not found: ID does not exist" Dec 03 12:20:56 crc kubenswrapper[5052]: I1203 12:20:56.718436 5052 scope.go:117] "RemoveContainer" containerID="df73d6886a1afb1e6d09c9481252bc2871f9a8608386e4ea20b13852f0881a4b" Dec 03 12:20:56 crc kubenswrapper[5052]: E1203 12:20:56.718720 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"df73d6886a1afb1e6d09c9481252bc2871f9a8608386e4ea20b13852f0881a4b\": container with ID starting with df73d6886a1afb1e6d09c9481252bc2871f9a8608386e4ea20b13852f0881a4b not found: ID does not exist" containerID="df73d6886a1afb1e6d09c9481252bc2871f9a8608386e4ea20b13852f0881a4b" Dec 03 12:20:56 crc kubenswrapper[5052]: I1203 12:20:56.718795 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"df73d6886a1afb1e6d09c9481252bc2871f9a8608386e4ea20b13852f0881a4b"} err="failed to get container status \"df73d6886a1afb1e6d09c9481252bc2871f9a8608386e4ea20b13852f0881a4b\": rpc error: code = NotFound desc = could not find container \"df73d6886a1afb1e6d09c9481252bc2871f9a8608386e4ea20b13852f0881a4b\": container with ID starting with df73d6886a1afb1e6d09c9481252bc2871f9a8608386e4ea20b13852f0881a4b not found: ID does not exist" Dec 03 12:20:56 crc kubenswrapper[5052]: I1203 12:20:56.718867 5052 scope.go:117] "RemoveContainer" containerID="006d1e418590929322462448d037f47e0e4031553a716f0f323b0ceb29ae6857" Dec 03 12:20:56 crc kubenswrapper[5052]: E1203 12:20:56.719112 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"006d1e418590929322462448d037f47e0e4031553a716f0f323b0ceb29ae6857\": container with ID starting with 006d1e418590929322462448d037f47e0e4031553a716f0f323b0ceb29ae6857 not found: ID does not exist" containerID="006d1e418590929322462448d037f47e0e4031553a716f0f323b0ceb29ae6857" Dec 03 12:20:56 crc kubenswrapper[5052]: I1203 12:20:56.719192 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"006d1e418590929322462448d037f47e0e4031553a716f0f323b0ceb29ae6857"} err="failed to get container status \"006d1e418590929322462448d037f47e0e4031553a716f0f323b0ceb29ae6857\": rpc error: code = NotFound desc = could not find container \"006d1e418590929322462448d037f47e0e4031553a716f0f323b0ceb29ae6857\": container with ID starting with 006d1e418590929322462448d037f47e0e4031553a716f0f323b0ceb29ae6857 not found: ID does not exist" Dec 03 12:20:56 crc kubenswrapper[5052]: I1203 12:20:56.719250 5052 scope.go:117] "RemoveContainer" containerID="4c12bd41f4c701be825345e793ba798c2e3b6ecd4b951f8fb1ee1579f6953903" Dec 03 12:20:56 crc kubenswrapper[5052]: I1203 12:20:56.752244 5052 scope.go:117] "RemoveContainer" containerID="4c12bd41f4c701be825345e793ba798c2e3b6ecd4b951f8fb1ee1579f6953903" Dec 03 12:20:56 crc kubenswrapper[5052]: E1203 12:20:56.756039 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4c12bd41f4c701be825345e793ba798c2e3b6ecd4b951f8fb1ee1579f6953903\": container with ID starting with 4c12bd41f4c701be825345e793ba798c2e3b6ecd4b951f8fb1ee1579f6953903 not found: ID does not exist" containerID="4c12bd41f4c701be825345e793ba798c2e3b6ecd4b951f8fb1ee1579f6953903" Dec 03 12:20:56 crc kubenswrapper[5052]: I1203 12:20:56.756082 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4c12bd41f4c701be825345e793ba798c2e3b6ecd4b951f8fb1ee1579f6953903"} err="failed to get container status \"4c12bd41f4c701be825345e793ba798c2e3b6ecd4b951f8fb1ee1579f6953903\": rpc error: code = NotFound desc = could not find container \"4c12bd41f4c701be825345e793ba798c2e3b6ecd4b951f8fb1ee1579f6953903\": container with ID starting with 4c12bd41f4c701be825345e793ba798c2e3b6ecd4b951f8fb1ee1579f6953903 not found: ID does not exist" Dec 03 12:20:56 crc kubenswrapper[5052]: I1203 12:20:56.773873 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/swift-storage-0"] Dec 03 12:20:56 crc kubenswrapper[5052]: W1203 12:20:56.793684 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0b121234_d24f_4307_9409_59a7dad3df4c.slice/crio-0406d9b3dde0ec3ac2702085cbbb703bed2f65a26230fed9c8d699555d0cadb8 WatchSource:0}: Error finding container 0406d9b3dde0ec3ac2702085cbbb703bed2f65a26230fed9c8d699555d0cadb8: Status 404 returned error can't find the container with id 0406d9b3dde0ec3ac2702085cbbb703bed2f65a26230fed9c8d699555d0cadb8 Dec 03 12:20:57 crc kubenswrapper[5052]: I1203 12:20:57.311520 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-conductor-0" event={"ID":"a7907bc2-8e73-4272-b779-fd71d4acb1f4","Type":"ContainerStarted","Data":"938a22611459fa789b24b03d2a21853b69071aad6d79f25384679595984908d8"} Dec 03 12:20:57 crc kubenswrapper[5052]: I1203 12:20:57.311560 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-conductor-0" event={"ID":"a7907bc2-8e73-4272-b779-fd71d4acb1f4","Type":"ContainerStarted","Data":"5c18996aaae6a85aceef22fa6d8699403c2e1d986214568e260b342877e4010d"} Dec 03 12:20:57 crc kubenswrapper[5052]: I1203 12:20:57.311726 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:20:57 crc kubenswrapper[5052]: I1203 12:20:57.315872 5052 generic.go:334] "Generic (PLEG): container finished" podID="311e25fa-0785-45ce-ae3c-00d32adbf3f4" containerID="522e7f93cdd131cc57ae189abb416b435302969048510ba07854b6d472d399d4" exitCode=143 Dec 03 12:20:57 crc kubenswrapper[5052]: I1203 12:20:57.315946 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-worker-76d6bdd486-g9c2p" event={"ID":"311e25fa-0785-45ce-ae3c-00d32adbf3f4","Type":"ContainerDied","Data":"522e7f93cdd131cc57ae189abb416b435302969048510ba07854b6d472d399d4"} Dec 03 12:20:57 crc kubenswrapper[5052]: I1203 12:20:57.324252 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"0b121234-d24f-4307-9409-59a7dad3df4c","Type":"ContainerStarted","Data":"914125fe1587ea7c446a5334d9e2828cf4834d6ad4f2053f842f07ad24a0e67d"} Dec 03 12:20:57 crc kubenswrapper[5052]: I1203 12:20:57.324307 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"0b121234-d24f-4307-9409-59a7dad3df4c","Type":"ContainerStarted","Data":"6b7c8f662acc40a3137f223890e54394fd6013bb9203453b7285ac28a63f277d"} Dec 03 12:20:57 crc kubenswrapper[5052]: I1203 12:20:57.324317 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"0b121234-d24f-4307-9409-59a7dad3df4c","Type":"ContainerStarted","Data":"0406d9b3dde0ec3ac2702085cbbb703bed2f65a26230fed9c8d699555d0cadb8"} Dec 03 12:20:57 crc kubenswrapper[5052]: I1203 12:20:57.331203 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-cell1-conductor-0" podStartSLOduration=2.33118261 podStartE2EDuration="2.33118261s" podCreationTimestamp="2025-12-03 12:20:55 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:20:57.328816492 +0000 UTC m=+3647.917058256" watchObservedRunningTime="2025-12-03 12:20:57.33118261 +0000 UTC m=+3647.919424374" Dec 03 12:20:57 crc kubenswrapper[5052]: I1203 12:20:57.335241 5052 generic.go:334] "Generic (PLEG): container finished" podID="736b6178-1b32-4f5e-a207-931b03aed990" containerID="e480cf5faa382d0f7dc902f72f5fa5a78a13cd4fac390962daf40b8d0bee88d7" exitCode=143 Dec 03 12:20:57 crc kubenswrapper[5052]: I1203 12:20:57.335324 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"736b6178-1b32-4f5e-a207-931b03aed990","Type":"ContainerDied","Data":"e480cf5faa382d0f7dc902f72f5fa5a78a13cd4fac390962daf40b8d0bee88d7"} Dec 03 12:20:57 crc kubenswrapper[5052]: I1203 12:20:57.338328 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-keystone-listener-6fb665b666-ftpjr" event={"ID":"9119635e-cb55-4428-9c61-b93ef2afd799","Type":"ContainerStarted","Data":"eaa2b6fb75e11891ca6e0ce7eb90ce65bb570e469fdab944929138063436ea33"} Dec 03 12:20:57 crc kubenswrapper[5052]: I1203 12:20:57.348080 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-644f564b46-cjtkx" event={"ID":"af38eb0d-3924-4c4b-a22e-7e934bdcf172","Type":"ContainerStarted","Data":"0b9c5ec46decf17edaf9cf870fd66992b0a9f87a5bcb4a208561e9aa1a0d41be"} Dec 03 12:20:57 crc kubenswrapper[5052]: I1203 12:20:57.348168 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/barbican-api-644f564b46-cjtkx" Dec 03 12:20:57 crc kubenswrapper[5052]: I1203 12:20:57.348206 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/barbican-api-644f564b46-cjtkx" Dec 03 12:20:57 crc kubenswrapper[5052]: I1203 12:20:57.351503 5052 generic.go:334] "Generic (PLEG): container finished" podID="941d2a2e-d51f-4b67-9b4a-6ff075a2eb0f" containerID="9f17cf626b688bdd38aa8737ff46f9c764ddd68d168ed599986a4ac6dc81223a" exitCode=143 Dec 03 12:20:57 crc kubenswrapper[5052]: I1203 12:20:57.352482 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"941d2a2e-d51f-4b67-9b4a-6ff075a2eb0f","Type":"ContainerDied","Data":"9f17cf626b688bdd38aa8737ff46f9c764ddd68d168ed599986a4ac6dc81223a"} Dec 03 12:20:57 crc kubenswrapper[5052]: I1203 12:20:57.364346 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/barbican-keystone-listener-6fb665b666-ftpjr" podStartSLOduration=3.364327426 podStartE2EDuration="3.364327426s" podCreationTimestamp="2025-12-03 12:20:54 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:20:57.362003169 +0000 UTC m=+3647.950244933" watchObservedRunningTime="2025-12-03 12:20:57.364327426 +0000 UTC m=+3647.952569190" Dec 03 12:20:57 crc kubenswrapper[5052]: I1203 12:20:57.408342 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-keystone-listener-55bfbc4797-wjrjr"] Dec 03 12:20:57 crc kubenswrapper[5052]: I1203 12:20:57.408826 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/barbican-keystone-listener-55bfbc4797-wjrjr" podUID="fcc1e62d-c1cc-49a0-93f4-5112fddf89bf" containerName="barbican-keystone-listener-log" containerID="cri-o://954e646bee69b2119ee9b7b6cb8846a3b7d5373dd70ba72d80b286777fa3abfd" gracePeriod=30 Dec 03 12:20:57 crc kubenswrapper[5052]: I1203 12:20:57.409167 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/barbican-keystone-listener-55bfbc4797-wjrjr" podUID="fcc1e62d-c1cc-49a0-93f4-5112fddf89bf" containerName="barbican-keystone-listener" containerID="cri-o://8cbc2d5f8732cb847f4e0ef3c58849a3aac9e0fab347ee31905e53398f6a0f69" gracePeriod=30 Dec 03 12:20:57 crc kubenswrapper[5052]: I1203 12:20:57.425369 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/barbican-api-644f564b46-cjtkx" podStartSLOduration=3.425345107 podStartE2EDuration="3.425345107s" podCreationTimestamp="2025-12-03 12:20:54 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:20:57.387809736 +0000 UTC m=+3647.976051500" watchObservedRunningTime="2025-12-03 12:20:57.425345107 +0000 UTC m=+3648.013586871" Dec 03 12:20:57 crc kubenswrapper[5052]: I1203 12:20:57.674667 5052 scope.go:117] "RemoveContainer" containerID="09b4a0072741f60499d24f40cc4ff3406d5d22671ced431363ab865172786300" Dec 03 12:20:57 crc kubenswrapper[5052]: E1203 12:20:57.675105 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 12:20:58 crc kubenswrapper[5052]: I1203 12:20:58.375489 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"0b121234-d24f-4307-9409-59a7dad3df4c","Type":"ContainerStarted","Data":"1fb344079bdd4d88652a84cde52ac882ef46f1e1b483946b91fa06958b9781b1"} Dec 03 12:20:58 crc kubenswrapper[5052]: I1203 12:20:58.375820 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"0b121234-d24f-4307-9409-59a7dad3df4c","Type":"ContainerStarted","Data":"7448a246780facb87faed946ab976ad223f836a90fd0ac6cbf876c7f11c6c2d8"} Dec 03 12:20:58 crc kubenswrapper[5052]: I1203 12:20:58.375835 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"0b121234-d24f-4307-9409-59a7dad3df4c","Type":"ContainerStarted","Data":"70f7ea9905786e034fff8dcf46682040188eb7ca7903f795173a38888c5b11d9"} Dec 03 12:20:58 crc kubenswrapper[5052]: I1203 12:20:58.375844 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"0b121234-d24f-4307-9409-59a7dad3df4c","Type":"ContainerStarted","Data":"1de28481ebd24dff85a8213026dad78dd91bf4d0b6c4d77c2b5231b95244a4cb"} Dec 03 12:20:58 crc kubenswrapper[5052]: I1203 12:20:58.378740 5052 generic.go:334] "Generic (PLEG): container finished" podID="fcc1e62d-c1cc-49a0-93f4-5112fddf89bf" containerID="954e646bee69b2119ee9b7b6cb8846a3b7d5373dd70ba72d80b286777fa3abfd" exitCode=143 Dec 03 12:20:58 crc kubenswrapper[5052]: I1203 12:20:58.379458 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-keystone-listener-55bfbc4797-wjrjr" event={"ID":"fcc1e62d-c1cc-49a0-93f4-5112fddf89bf","Type":"ContainerDied","Data":"954e646bee69b2119ee9b7b6cb8846a3b7d5373dd70ba72d80b286777fa3abfd"} Dec 03 12:20:59 crc kubenswrapper[5052]: I1203 12:20:59.391704 5052 generic.go:334] "Generic (PLEG): container finished" podID="63c18817-7663-466c-845d-bc998c5b7093" containerID="4d363d9441eef4d9d65d1bcd8bc7f04f2fe0d45b88e46676a88a9cb0748fb83e" exitCode=0 Dec 03 12:20:59 crc kubenswrapper[5052]: I1203 12:20:59.391925 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-bootstrap-dqvmm" event={"ID":"63c18817-7663-466c-845d-bc998c5b7093","Type":"ContainerDied","Data":"4d363d9441eef4d9d65d1bcd8bc7f04f2fe0d45b88e46676a88a9cb0748fb83e"} Dec 03 12:20:59 crc kubenswrapper[5052]: I1203 12:20:59.398900 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"0b121234-d24f-4307-9409-59a7dad3df4c","Type":"ContainerStarted","Data":"38ac8f54d7f675400878ab50b896bde2d4d40ee63497af23ebdca0f75d215cd7"} Dec 03 12:20:59 crc kubenswrapper[5052]: I1203 12:20:59.398937 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"0b121234-d24f-4307-9409-59a7dad3df4c","Type":"ContainerStarted","Data":"a108f5ccd3086ba9eada83058d904fc18a5b2e825c993e540cb30d0f9ced71e9"} Dec 03 12:20:59 crc kubenswrapper[5052]: I1203 12:20:59.398947 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"0b121234-d24f-4307-9409-59a7dad3df4c","Type":"ContainerStarted","Data":"a7be0eed6906ff248e67daeb0cb8433a1ab73cf3e47159ff1cbfe4029900aa36"} Dec 03 12:20:59 crc kubenswrapper[5052]: I1203 12:20:59.398956 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"0b121234-d24f-4307-9409-59a7dad3df4c","Type":"ContainerStarted","Data":"93d4d6d903155179da2cc815475bcc4c3e646bd25173aec67f526949d47734f0"} Dec 03 12:20:59 crc kubenswrapper[5052]: I1203 12:20:59.398964 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"0b121234-d24f-4307-9409-59a7dad3df4c","Type":"ContainerStarted","Data":"280a6f9eeba9635ba75d3f89698a54d959866a87a84e1c34632aae63f6162f88"} Dec 03 12:20:59 crc kubenswrapper[5052]: I1203 12:20:59.398973 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"0b121234-d24f-4307-9409-59a7dad3df4c","Type":"ContainerStarted","Data":"76aa0661476cea8be8d11604d15492437f317021e6ae641e33f6ca912d72fe2e"} Dec 03 12:20:59 crc kubenswrapper[5052]: I1203 12:20:59.972023 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.043859 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/736b6178-1b32-4f5e-a207-931b03aed990-scripts\") pod \"736b6178-1b32-4f5e-a207-931b03aed990\" (UID: \"736b6178-1b32-4f5e-a207-931b03aed990\") " Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.044308 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/736b6178-1b32-4f5e-a207-931b03aed990-httpd-run\") pod \"736b6178-1b32-4f5e-a207-931b03aed990\" (UID: \"736b6178-1b32-4f5e-a207-931b03aed990\") " Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.044368 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/736b6178-1b32-4f5e-a207-931b03aed990-public-tls-certs\") pod \"736b6178-1b32-4f5e-a207-931b03aed990\" (UID: \"736b6178-1b32-4f5e-a207-931b03aed990\") " Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.044406 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/736b6178-1b32-4f5e-a207-931b03aed990-logs\") pod \"736b6178-1b32-4f5e-a207-931b03aed990\" (UID: \"736b6178-1b32-4f5e-a207-931b03aed990\") " Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.044535 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/736b6178-1b32-4f5e-a207-931b03aed990-combined-ca-bundle\") pod \"736b6178-1b32-4f5e-a207-931b03aed990\" (UID: \"736b6178-1b32-4f5e-a207-931b03aed990\") " Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.044578 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/736b6178-1b32-4f5e-a207-931b03aed990-config-data\") pod \"736b6178-1b32-4f5e-a207-931b03aed990\" (UID: \"736b6178-1b32-4f5e-a207-931b03aed990\") " Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.044649 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fb5tp\" (UniqueName: \"kubernetes.io/projected/736b6178-1b32-4f5e-a207-931b03aed990-kube-api-access-fb5tp\") pod \"736b6178-1b32-4f5e-a207-931b03aed990\" (UID: \"736b6178-1b32-4f5e-a207-931b03aed990\") " Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.044707 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"glance\" (UniqueName: \"kubernetes.io/local-volume/local-storage09-crc\") pod \"736b6178-1b32-4f5e-a207-931b03aed990\" (UID: \"736b6178-1b32-4f5e-a207-931b03aed990\") " Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.046325 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/736b6178-1b32-4f5e-a207-931b03aed990-httpd-run" (OuterVolumeSpecName: "httpd-run") pod "736b6178-1b32-4f5e-a207-931b03aed990" (UID: "736b6178-1b32-4f5e-a207-931b03aed990"). InnerVolumeSpecName "httpd-run". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.046745 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/736b6178-1b32-4f5e-a207-931b03aed990-logs" (OuterVolumeSpecName: "logs") pod "736b6178-1b32-4f5e-a207-931b03aed990" (UID: "736b6178-1b32-4f5e-a207-931b03aed990"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.051735 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/736b6178-1b32-4f5e-a207-931b03aed990-scripts" (OuterVolumeSpecName: "scripts") pod "736b6178-1b32-4f5e-a207-931b03aed990" (UID: "736b6178-1b32-4f5e-a207-931b03aed990"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.052349 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage09-crc" (OuterVolumeSpecName: "glance") pod "736b6178-1b32-4f5e-a207-931b03aed990" (UID: "736b6178-1b32-4f5e-a207-931b03aed990"). InnerVolumeSpecName "local-storage09-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.052739 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/736b6178-1b32-4f5e-a207-931b03aed990-kube-api-access-fb5tp" (OuterVolumeSpecName: "kube-api-access-fb5tp") pod "736b6178-1b32-4f5e-a207-931b03aed990" (UID: "736b6178-1b32-4f5e-a207-931b03aed990"). InnerVolumeSpecName "kube-api-access-fb5tp". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.091750 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/736b6178-1b32-4f5e-a207-931b03aed990-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "736b6178-1b32-4f5e-a207-931b03aed990" (UID: "736b6178-1b32-4f5e-a207-931b03aed990"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.147856 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-fb5tp\" (UniqueName: \"kubernetes.io/projected/736b6178-1b32-4f5e-a207-931b03aed990-kube-api-access-fb5tp\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.147950 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage09-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage09-crc\") on node \"crc\" " Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.147969 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/736b6178-1b32-4f5e-a207-931b03aed990-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.147984 5052 reconciler_common.go:293] "Volume detached for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/736b6178-1b32-4f5e-a207-931b03aed990-httpd-run\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.147997 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/736b6178-1b32-4f5e-a207-931b03aed990-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.148011 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/736b6178-1b32-4f5e-a207-931b03aed990-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.149889 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-keystone-listener-55bfbc4797-wjrjr" Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.184992 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/736b6178-1b32-4f5e-a207-931b03aed990-config-data" (OuterVolumeSpecName: "config-data") pod "736b6178-1b32-4f5e-a207-931b03aed990" (UID: "736b6178-1b32-4f5e-a207-931b03aed990"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.186960 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage09-crc" (UniqueName: "kubernetes.io/local-volume/local-storage09-crc") on node "crc" Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.199361 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/736b6178-1b32-4f5e-a207-931b03aed990-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "736b6178-1b32-4f5e-a207-931b03aed990" (UID: "736b6178-1b32-4f5e-a207-931b03aed990"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.249073 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/fcc1e62d-c1cc-49a0-93f4-5112fddf89bf-config-data\") pod \"fcc1e62d-c1cc-49a0-93f4-5112fddf89bf\" (UID: \"fcc1e62d-c1cc-49a0-93f4-5112fddf89bf\") " Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.249243 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/fcc1e62d-c1cc-49a0-93f4-5112fddf89bf-logs\") pod \"fcc1e62d-c1cc-49a0-93f4-5112fddf89bf\" (UID: \"fcc1e62d-c1cc-49a0-93f4-5112fddf89bf\") " Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.249319 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ckhjp\" (UniqueName: \"kubernetes.io/projected/fcc1e62d-c1cc-49a0-93f4-5112fddf89bf-kube-api-access-ckhjp\") pod \"fcc1e62d-c1cc-49a0-93f4-5112fddf89bf\" (UID: \"fcc1e62d-c1cc-49a0-93f4-5112fddf89bf\") " Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.249382 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/fcc1e62d-c1cc-49a0-93f4-5112fddf89bf-combined-ca-bundle\") pod \"fcc1e62d-c1cc-49a0-93f4-5112fddf89bf\" (UID: \"fcc1e62d-c1cc-49a0-93f4-5112fddf89bf\") " Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.249421 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/fcc1e62d-c1cc-49a0-93f4-5112fddf89bf-config-data-custom\") pod \"fcc1e62d-c1cc-49a0-93f4-5112fddf89bf\" (UID: \"fcc1e62d-c1cc-49a0-93f4-5112fddf89bf\") " Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.249968 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/736b6178-1b32-4f5e-a207-931b03aed990-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.249986 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage09-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage09-crc\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.249995 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/736b6178-1b32-4f5e-a207-931b03aed990-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.251323 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fcc1e62d-c1cc-49a0-93f4-5112fddf89bf-logs" (OuterVolumeSpecName: "logs") pod "fcc1e62d-c1cc-49a0-93f4-5112fddf89bf" (UID: "fcc1e62d-c1cc-49a0-93f4-5112fddf89bf"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.254366 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fcc1e62d-c1cc-49a0-93f4-5112fddf89bf-kube-api-access-ckhjp" (OuterVolumeSpecName: "kube-api-access-ckhjp") pod "fcc1e62d-c1cc-49a0-93f4-5112fddf89bf" (UID: "fcc1e62d-c1cc-49a0-93f4-5112fddf89bf"). InnerVolumeSpecName "kube-api-access-ckhjp". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.256286 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fcc1e62d-c1cc-49a0-93f4-5112fddf89bf-config-data-custom" (OuterVolumeSpecName: "config-data-custom") pod "fcc1e62d-c1cc-49a0-93f4-5112fddf89bf" (UID: "fcc1e62d-c1cc-49a0-93f4-5112fddf89bf"). InnerVolumeSpecName "config-data-custom". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.260907 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.317878 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fcc1e62d-c1cc-49a0-93f4-5112fddf89bf-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "fcc1e62d-c1cc-49a0-93f4-5112fddf89bf" (UID: "fcc1e62d-c1cc-49a0-93f4-5112fddf89bf"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.351350 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"glance\" (UniqueName: \"kubernetes.io/local-volume/local-storage17-crc\") pod \"941d2a2e-d51f-4b67-9b4a-6ff075a2eb0f\" (UID: \"941d2a2e-d51f-4b67-9b4a-6ff075a2eb0f\") " Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.351556 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/941d2a2e-d51f-4b67-9b4a-6ff075a2eb0f-combined-ca-bundle\") pod \"941d2a2e-d51f-4b67-9b4a-6ff075a2eb0f\" (UID: \"941d2a2e-d51f-4b67-9b4a-6ff075a2eb0f\") " Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.351628 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/941d2a2e-d51f-4b67-9b4a-6ff075a2eb0f-internal-tls-certs\") pod \"941d2a2e-d51f-4b67-9b4a-6ff075a2eb0f\" (UID: \"941d2a2e-d51f-4b67-9b4a-6ff075a2eb0f\") " Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.351709 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/941d2a2e-d51f-4b67-9b4a-6ff075a2eb0f-httpd-run\") pod \"941d2a2e-d51f-4b67-9b4a-6ff075a2eb0f\" (UID: \"941d2a2e-d51f-4b67-9b4a-6ff075a2eb0f\") " Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.351795 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/941d2a2e-d51f-4b67-9b4a-6ff075a2eb0f-scripts\") pod \"941d2a2e-d51f-4b67-9b4a-6ff075a2eb0f\" (UID: \"941d2a2e-d51f-4b67-9b4a-6ff075a2eb0f\") " Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.351906 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/941d2a2e-d51f-4b67-9b4a-6ff075a2eb0f-logs\") pod \"941d2a2e-d51f-4b67-9b4a-6ff075a2eb0f\" (UID: \"941d2a2e-d51f-4b67-9b4a-6ff075a2eb0f\") " Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.351989 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/941d2a2e-d51f-4b67-9b4a-6ff075a2eb0f-config-data\") pod \"941d2a2e-d51f-4b67-9b4a-6ff075a2eb0f\" (UID: \"941d2a2e-d51f-4b67-9b4a-6ff075a2eb0f\") " Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.352062 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qt7nf\" (UniqueName: \"kubernetes.io/projected/941d2a2e-d51f-4b67-9b4a-6ff075a2eb0f-kube-api-access-qt7nf\") pod \"941d2a2e-d51f-4b67-9b4a-6ff075a2eb0f\" (UID: \"941d2a2e-d51f-4b67-9b4a-6ff075a2eb0f\") " Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.353014 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-ckhjp\" (UniqueName: \"kubernetes.io/projected/fcc1e62d-c1cc-49a0-93f4-5112fddf89bf-kube-api-access-ckhjp\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.353040 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/fcc1e62d-c1cc-49a0-93f4-5112fddf89bf-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.353054 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/fcc1e62d-c1cc-49a0-93f4-5112fddf89bf-config-data-custom\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.353097 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/fcc1e62d-c1cc-49a0-93f4-5112fddf89bf-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.356062 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/941d2a2e-d51f-4b67-9b4a-6ff075a2eb0f-httpd-run" (OuterVolumeSpecName: "httpd-run") pod "941d2a2e-d51f-4b67-9b4a-6ff075a2eb0f" (UID: "941d2a2e-d51f-4b67-9b4a-6ff075a2eb0f"). InnerVolumeSpecName "httpd-run". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.359054 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/941d2a2e-d51f-4b67-9b4a-6ff075a2eb0f-logs" (OuterVolumeSpecName: "logs") pod "941d2a2e-d51f-4b67-9b4a-6ff075a2eb0f" (UID: "941d2a2e-d51f-4b67-9b4a-6ff075a2eb0f"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.370248 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/941d2a2e-d51f-4b67-9b4a-6ff075a2eb0f-kube-api-access-qt7nf" (OuterVolumeSpecName: "kube-api-access-qt7nf") pod "941d2a2e-d51f-4b67-9b4a-6ff075a2eb0f" (UID: "941d2a2e-d51f-4b67-9b4a-6ff075a2eb0f"). InnerVolumeSpecName "kube-api-access-qt7nf". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.379069 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage17-crc" (OuterVolumeSpecName: "glance") pod "941d2a2e-d51f-4b67-9b4a-6ff075a2eb0f" (UID: "941d2a2e-d51f-4b67-9b4a-6ff075a2eb0f"). InnerVolumeSpecName "local-storage17-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.381065 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/941d2a2e-d51f-4b67-9b4a-6ff075a2eb0f-scripts" (OuterVolumeSpecName: "scripts") pod "941d2a2e-d51f-4b67-9b4a-6ff075a2eb0f" (UID: "941d2a2e-d51f-4b67-9b4a-6ff075a2eb0f"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.396688 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/941d2a2e-d51f-4b67-9b4a-6ff075a2eb0f-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "941d2a2e-d51f-4b67-9b4a-6ff075a2eb0f" (UID: "941d2a2e-d51f-4b67-9b4a-6ff075a2eb0f"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.415446 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fcc1e62d-c1cc-49a0-93f4-5112fddf89bf-config-data" (OuterVolumeSpecName: "config-data") pod "fcc1e62d-c1cc-49a0-93f4-5112fddf89bf" (UID: "fcc1e62d-c1cc-49a0-93f4-5112fddf89bf"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.416716 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.416533 5052 generic.go:334] "Generic (PLEG): container finished" podID="736b6178-1b32-4f5e-a207-931b03aed990" containerID="08ba12ac062944881ab39ae9fbebc472a569ed1780e7c9436d3f4c6a4444d057" exitCode=0 Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.417441 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"736b6178-1b32-4f5e-a207-931b03aed990","Type":"ContainerDied","Data":"08ba12ac062944881ab39ae9fbebc472a569ed1780e7c9436d3f4c6a4444d057"} Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.417559 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"736b6178-1b32-4f5e-a207-931b03aed990","Type":"ContainerDied","Data":"eaad6a6e236c45540e07e3ae81242576acf2408c9a149279047c9fa917b7baee"} Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.417658 5052 scope.go:117] "RemoveContainer" containerID="08ba12ac062944881ab39ae9fbebc472a569ed1780e7c9436d3f4c6a4444d057" Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.423597 5052 generic.go:334] "Generic (PLEG): container finished" podID="941d2a2e-d51f-4b67-9b4a-6ff075a2eb0f" containerID="35b431787e68c1ec9b78768c06706d4afad176de2f59bf713fd162cab978b071" exitCode=0 Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.423822 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.424434 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"941d2a2e-d51f-4b67-9b4a-6ff075a2eb0f","Type":"ContainerDied","Data":"35b431787e68c1ec9b78768c06706d4afad176de2f59bf713fd162cab978b071"} Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.425369 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"941d2a2e-d51f-4b67-9b4a-6ff075a2eb0f","Type":"ContainerDied","Data":"2be741a0e4b2f878aa142c8486c08a4543804d17a7967a78a28ddef3fb9df795"} Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.428168 5052 generic.go:334] "Generic (PLEG): container finished" podID="fcc1e62d-c1cc-49a0-93f4-5112fddf89bf" containerID="8cbc2d5f8732cb847f4e0ef3c58849a3aac9e0fab347ee31905e53398f6a0f69" exitCode=0 Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.428213 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-keystone-listener-55bfbc4797-wjrjr" Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.428277 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-keystone-listener-55bfbc4797-wjrjr" event={"ID":"fcc1e62d-c1cc-49a0-93f4-5112fddf89bf","Type":"ContainerDied","Data":"8cbc2d5f8732cb847f4e0ef3c58849a3aac9e0fab347ee31905e53398f6a0f69"} Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.428327 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-keystone-listener-55bfbc4797-wjrjr" event={"ID":"fcc1e62d-c1cc-49a0-93f4-5112fddf89bf","Type":"ContainerDied","Data":"5da345bb4422f25990625e27cdecce254f7d0c0ff9033154534628e92a5a94f3"} Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.435479 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"0b121234-d24f-4307-9409-59a7dad3df4c","Type":"ContainerStarted","Data":"e63a7b1f0f328636f34f8466d7114142477116b8ba39ff4c63590f6d3d2ecb61"} Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.436041 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"0b121234-d24f-4307-9409-59a7dad3df4c","Type":"ContainerStarted","Data":"eac0ccbf31e29898c07277ff8b3f25e720d684db3945841e058ea63b8f2fc50e"} Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.436250 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"0b121234-d24f-4307-9409-59a7dad3df4c","Type":"ContainerStarted","Data":"9826c564d51675cc0d19bfb280f54c6cdb8da70e11a2735f3456a7c16c5b0331"} Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.449177 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/941d2a2e-d51f-4b67-9b4a-6ff075a2eb0f-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "941d2a2e-d51f-4b67-9b4a-6ff075a2eb0f" (UID: "941d2a2e-d51f-4b67-9b4a-6ff075a2eb0f"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.455000 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage17-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage17-crc\") on node \"crc\" " Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.455029 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/941d2a2e-d51f-4b67-9b4a-6ff075a2eb0f-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.455042 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/941d2a2e-d51f-4b67-9b4a-6ff075a2eb0f-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.455053 5052 reconciler_common.go:293] "Volume detached for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/941d2a2e-d51f-4b67-9b4a-6ff075a2eb0f-httpd-run\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.455066 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/941d2a2e-d51f-4b67-9b4a-6ff075a2eb0f-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.455078 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/941d2a2e-d51f-4b67-9b4a-6ff075a2eb0f-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.455089 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/fcc1e62d-c1cc-49a0-93f4-5112fddf89bf-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.455101 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-qt7nf\" (UniqueName: \"kubernetes.io/projected/941d2a2e-d51f-4b67-9b4a-6ff075a2eb0f-kube-api-access-qt7nf\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.475320 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/swift-storage-0" podStartSLOduration=5.475296009 podStartE2EDuration="5.475296009s" podCreationTimestamp="2025-12-03 12:20:55 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:21:00.467236739 +0000 UTC m=+3651.055478503" watchObservedRunningTime="2025-12-03 12:21:00.475296009 +0000 UTC m=+3651.063537773" Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.477809 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/941d2a2e-d51f-4b67-9b4a-6ff075a2eb0f-config-data" (OuterVolumeSpecName: "config-data") pod "941d2a2e-d51f-4b67-9b4a-6ff075a2eb0f" (UID: "941d2a2e-d51f-4b67-9b4a-6ff075a2eb0f"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.540375 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage17-crc" (UniqueName: "kubernetes.io/local-volume/local-storage17-crc") on node "crc" Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.558243 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/941d2a2e-d51f-4b67-9b4a-6ff075a2eb0f-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.558269 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage17-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage17-crc\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.575056 5052 scope.go:117] "RemoveContainer" containerID="e480cf5faa382d0f7dc902f72f5fa5a78a13cd4fac390962daf40b8d0bee88d7" Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.622913 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-keystone-listener-55bfbc4797-wjrjr"] Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.656924 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/barbican-keystone-listener-55bfbc4797-wjrjr"] Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.668654 5052 scope.go:117] "RemoveContainer" containerID="08ba12ac062944881ab39ae9fbebc472a569ed1780e7c9436d3f4c6a4444d057" Dec 03 12:21:00 crc kubenswrapper[5052]: E1203 12:21:00.670915 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"08ba12ac062944881ab39ae9fbebc472a569ed1780e7c9436d3f4c6a4444d057\": container with ID starting with 08ba12ac062944881ab39ae9fbebc472a569ed1780e7c9436d3f4c6a4444d057 not found: ID does not exist" containerID="08ba12ac062944881ab39ae9fbebc472a569ed1780e7c9436d3f4c6a4444d057" Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.670969 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"08ba12ac062944881ab39ae9fbebc472a569ed1780e7c9436d3f4c6a4444d057"} err="failed to get container status \"08ba12ac062944881ab39ae9fbebc472a569ed1780e7c9436d3f4c6a4444d057\": rpc error: code = NotFound desc = could not find container \"08ba12ac062944881ab39ae9fbebc472a569ed1780e7c9436d3f4c6a4444d057\": container with ID starting with 08ba12ac062944881ab39ae9fbebc472a569ed1780e7c9436d3f4c6a4444d057 not found: ID does not exist" Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.670999 5052 scope.go:117] "RemoveContainer" containerID="e480cf5faa382d0f7dc902f72f5fa5a78a13cd4fac390962daf40b8d0bee88d7" Dec 03 12:21:00 crc kubenswrapper[5052]: E1203 12:21:00.674877 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e480cf5faa382d0f7dc902f72f5fa5a78a13cd4fac390962daf40b8d0bee88d7\": container with ID starting with e480cf5faa382d0f7dc902f72f5fa5a78a13cd4fac390962daf40b8d0bee88d7 not found: ID does not exist" containerID="e480cf5faa382d0f7dc902f72f5fa5a78a13cd4fac390962daf40b8d0bee88d7" Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.674915 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e480cf5faa382d0f7dc902f72f5fa5a78a13cd4fac390962daf40b8d0bee88d7"} err="failed to get container status \"e480cf5faa382d0f7dc902f72f5fa5a78a13cd4fac390962daf40b8d0bee88d7\": rpc error: code = NotFound desc = could not find container \"e480cf5faa382d0f7dc902f72f5fa5a78a13cd4fac390962daf40b8d0bee88d7\": container with ID starting with e480cf5faa382d0f7dc902f72f5fa5a78a13cd4fac390962daf40b8d0bee88d7 not found: ID does not exist" Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.674940 5052 scope.go:117] "RemoveContainer" containerID="35b431787e68c1ec9b78768c06706d4afad176de2f59bf713fd162cab978b071" Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.686099 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.713058 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.725682 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:21:00 crc kubenswrapper[5052]: E1203 12:21:00.726094 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="941d2a2e-d51f-4b67-9b4a-6ff075a2eb0f" containerName="glance-log" Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.726105 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="941d2a2e-d51f-4b67-9b4a-6ff075a2eb0f" containerName="glance-log" Dec 03 12:21:00 crc kubenswrapper[5052]: E1203 12:21:00.726118 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="736b6178-1b32-4f5e-a207-931b03aed990" containerName="glance-httpd" Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.726124 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="736b6178-1b32-4f5e-a207-931b03aed990" containerName="glance-httpd" Dec 03 12:21:00 crc kubenswrapper[5052]: E1203 12:21:00.726140 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="66f5168a-1ee5-4de9-8e50-90ef25186ee8" containerName="glance-db-sync" Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.726146 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="66f5168a-1ee5-4de9-8e50-90ef25186ee8" containerName="glance-db-sync" Dec 03 12:21:00 crc kubenswrapper[5052]: E1203 12:21:00.726154 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="fcc1e62d-c1cc-49a0-93f4-5112fddf89bf" containerName="barbican-keystone-listener" Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.726160 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="fcc1e62d-c1cc-49a0-93f4-5112fddf89bf" containerName="barbican-keystone-listener" Dec 03 12:21:00 crc kubenswrapper[5052]: E1203 12:21:00.726177 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="736b6178-1b32-4f5e-a207-931b03aed990" containerName="glance-log" Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.726183 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="736b6178-1b32-4f5e-a207-931b03aed990" containerName="glance-log" Dec 03 12:21:00 crc kubenswrapper[5052]: E1203 12:21:00.726194 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="fcc1e62d-c1cc-49a0-93f4-5112fddf89bf" containerName="barbican-keystone-listener-log" Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.726200 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="fcc1e62d-c1cc-49a0-93f4-5112fddf89bf" containerName="barbican-keystone-listener-log" Dec 03 12:21:00 crc kubenswrapper[5052]: E1203 12:21:00.726210 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="941d2a2e-d51f-4b67-9b4a-6ff075a2eb0f" containerName="glance-httpd" Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.726216 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="941d2a2e-d51f-4b67-9b4a-6ff075a2eb0f" containerName="glance-httpd" Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.726377 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="941d2a2e-d51f-4b67-9b4a-6ff075a2eb0f" containerName="glance-httpd" Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.726395 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="fcc1e62d-c1cc-49a0-93f4-5112fddf89bf" containerName="barbican-keystone-listener-log" Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.726405 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="736b6178-1b32-4f5e-a207-931b03aed990" containerName="glance-log" Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.726412 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="736b6178-1b32-4f5e-a207-931b03aed990" containerName="glance-httpd" Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.726425 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="941d2a2e-d51f-4b67-9b4a-6ff075a2eb0f" containerName="glance-log" Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.726442 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="66f5168a-1ee5-4de9-8e50-90ef25186ee8" containerName="glance-db-sync" Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.726451 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="fcc1e62d-c1cc-49a0-93f4-5112fddf89bf" containerName="barbican-keystone-listener" Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.727456 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.730315 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-glance-dockercfg-rsctn" Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.731272 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-default-external-config-data" Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.731404 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-scripts" Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.738232 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.740882 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-glance-default-public-svc" Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.758139 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-5bbbb9b4b9-m4ttn"] Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.770874 5052 scope.go:117] "RemoveContainer" containerID="9f17cf626b688bdd38aa8737ff46f9c764ddd68d168ed599986a4ac6dc81223a" Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.776289 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-5bbbb9b4b9-m4ttn"] Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.777496 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-5bbbb9b4b9-m4ttn" Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.824014 5052 scope.go:117] "RemoveContainer" containerID="35b431787e68c1ec9b78768c06706d4afad176de2f59bf713fd162cab978b071" Dec 03 12:21:00 crc kubenswrapper[5052]: E1203 12:21:00.825436 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"35b431787e68c1ec9b78768c06706d4afad176de2f59bf713fd162cab978b071\": container with ID starting with 35b431787e68c1ec9b78768c06706d4afad176de2f59bf713fd162cab978b071 not found: ID does not exist" containerID="35b431787e68c1ec9b78768c06706d4afad176de2f59bf713fd162cab978b071" Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.825476 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"35b431787e68c1ec9b78768c06706d4afad176de2f59bf713fd162cab978b071"} err="failed to get container status \"35b431787e68c1ec9b78768c06706d4afad176de2f59bf713fd162cab978b071\": rpc error: code = NotFound desc = could not find container \"35b431787e68c1ec9b78768c06706d4afad176de2f59bf713fd162cab978b071\": container with ID starting with 35b431787e68c1ec9b78768c06706d4afad176de2f59bf713fd162cab978b071 not found: ID does not exist" Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.825501 5052 scope.go:117] "RemoveContainer" containerID="9f17cf626b688bdd38aa8737ff46f9c764ddd68d168ed599986a4ac6dc81223a" Dec 03 12:21:00 crc kubenswrapper[5052]: E1203 12:21:00.826095 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9f17cf626b688bdd38aa8737ff46f9c764ddd68d168ed599986a4ac6dc81223a\": container with ID starting with 9f17cf626b688bdd38aa8737ff46f9c764ddd68d168ed599986a4ac6dc81223a not found: ID does not exist" containerID="9f17cf626b688bdd38aa8737ff46f9c764ddd68d168ed599986a4ac6dc81223a" Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.826152 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9f17cf626b688bdd38aa8737ff46f9c764ddd68d168ed599986a4ac6dc81223a"} err="failed to get container status \"9f17cf626b688bdd38aa8737ff46f9c764ddd68d168ed599986a4ac6dc81223a\": rpc error: code = NotFound desc = could not find container \"9f17cf626b688bdd38aa8737ff46f9c764ddd68d168ed599986a4ac6dc81223a\": container with ID starting with 9f17cf626b688bdd38aa8737ff46f9c764ddd68d168ed599986a4ac6dc81223a not found: ID does not exist" Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.826192 5052 scope.go:117] "RemoveContainer" containerID="8cbc2d5f8732cb847f4e0ef3c58849a3aac9e0fab347ee31905e53398f6a0f69" Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.877793 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dmhw4\" (UniqueName: \"kubernetes.io/projected/72b87ae1-6854-475b-a31d-976acc913623-kube-api-access-dmhw4\") pod \"glance-default-external-api-0\" (UID: \"72b87ae1-6854-475b-a31d-976acc913623\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.877843 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/72b87ae1-6854-475b-a31d-976acc913623-public-tls-certs\") pod \"glance-default-external-api-0\" (UID: \"72b87ae1-6854-475b-a31d-976acc913623\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.877877 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/72b87ae1-6854-475b-a31d-976acc913623-scripts\") pod \"glance-default-external-api-0\" (UID: \"72b87ae1-6854-475b-a31d-976acc913623\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.877915 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dns-swift-storage-0\" (UniqueName: \"kubernetes.io/configmap/f12eef70-f626-42ce-b4cf-2a07adec9657-dns-swift-storage-0\") pod \"dnsmasq-dnsmasq-5bbbb9b4b9-m4ttn\" (UID: \"f12eef70-f626-42ce-b4cf-2a07adec9657\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-5bbbb9b4b9-m4ttn" Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.877936 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage09-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage09-crc\") pod \"glance-default-external-api-0\" (UID: \"72b87ae1-6854-475b-a31d-976acc913623\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.877974 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/72b87ae1-6854-475b-a31d-976acc913623-logs\") pod \"glance-default-external-api-0\" (UID: \"72b87ae1-6854-475b-a31d-976acc913623\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.878001 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/72b87ae1-6854-475b-a31d-976acc913623-combined-ca-bundle\") pod \"glance-default-external-api-0\" (UID: \"72b87ae1-6854-475b-a31d-976acc913623\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.878041 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/72b87ae1-6854-475b-a31d-976acc913623-httpd-run\") pod \"glance-default-external-api-0\" (UID: \"72b87ae1-6854-475b-a31d-976acc913623\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.878073 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f12eef70-f626-42ce-b4cf-2a07adec9657-config\") pod \"dnsmasq-dnsmasq-5bbbb9b4b9-m4ttn\" (UID: \"f12eef70-f626-42ce-b4cf-2a07adec9657\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-5bbbb9b4b9-m4ttn" Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.878093 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-llb4x\" (UniqueName: \"kubernetes.io/projected/f12eef70-f626-42ce-b4cf-2a07adec9657-kube-api-access-llb4x\") pod \"dnsmasq-dnsmasq-5bbbb9b4b9-m4ttn\" (UID: \"f12eef70-f626-42ce-b4cf-2a07adec9657\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-5bbbb9b4b9-m4ttn" Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.878129 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/f12eef70-f626-42ce-b4cf-2a07adec9657-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-5bbbb9b4b9-m4ttn\" (UID: \"f12eef70-f626-42ce-b4cf-2a07adec9657\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-5bbbb9b4b9-m4ttn" Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.878154 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/72b87ae1-6854-475b-a31d-976acc913623-config-data\") pod \"glance-default-external-api-0\" (UID: \"72b87ae1-6854-475b-a31d-976acc913623\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.903280 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.937095 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-bootstrap-dqvmm" Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.945872 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.979712 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/63c18817-7663-466c-845d-bc998c5b7093-scripts\") pod \"63c18817-7663-466c-845d-bc998c5b7093\" (UID: \"63c18817-7663-466c-845d-bc998c5b7093\") " Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.980040 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/63c18817-7663-466c-845d-bc998c5b7093-fernet-keys\") pod \"63c18817-7663-466c-845d-bc998c5b7093\" (UID: \"63c18817-7663-466c-845d-bc998c5b7093\") " Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.980061 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/63c18817-7663-466c-845d-bc998c5b7093-combined-ca-bundle\") pod \"63c18817-7663-466c-845d-bc998c5b7093\" (UID: \"63c18817-7663-466c-845d-bc998c5b7093\") " Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.980096 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/63c18817-7663-466c-845d-bc998c5b7093-credential-keys\") pod \"63c18817-7663-466c-845d-bc998c5b7093\" (UID: \"63c18817-7663-466c-845d-bc998c5b7093\") " Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.980131 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rpgv8\" (UniqueName: \"kubernetes.io/projected/63c18817-7663-466c-845d-bc998c5b7093-kube-api-access-rpgv8\") pod \"63c18817-7663-466c-845d-bc998c5b7093\" (UID: \"63c18817-7663-466c-845d-bc998c5b7093\") " Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.980147 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/63c18817-7663-466c-845d-bc998c5b7093-config-data\") pod \"63c18817-7663-466c-845d-bc998c5b7093\" (UID: \"63c18817-7663-466c-845d-bc998c5b7093\") " Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.980732 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"dns-swift-storage-0\" (UniqueName: \"kubernetes.io/configmap/f12eef70-f626-42ce-b4cf-2a07adec9657-dns-swift-storage-0\") pod \"dnsmasq-dnsmasq-5bbbb9b4b9-m4ttn\" (UID: \"f12eef70-f626-42ce-b4cf-2a07adec9657\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-5bbbb9b4b9-m4ttn" Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.980780 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage09-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage09-crc\") pod \"glance-default-external-api-0\" (UID: \"72b87ae1-6854-475b-a31d-976acc913623\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.980912 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/72b87ae1-6854-475b-a31d-976acc913623-logs\") pod \"glance-default-external-api-0\" (UID: \"72b87ae1-6854-475b-a31d-976acc913623\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.980994 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/72b87ae1-6854-475b-a31d-976acc913623-combined-ca-bundle\") pod \"glance-default-external-api-0\" (UID: \"72b87ae1-6854-475b-a31d-976acc913623\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.981129 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/72b87ae1-6854-475b-a31d-976acc913623-httpd-run\") pod \"glance-default-external-api-0\" (UID: \"72b87ae1-6854-475b-a31d-976acc913623\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.981237 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f12eef70-f626-42ce-b4cf-2a07adec9657-config\") pod \"dnsmasq-dnsmasq-5bbbb9b4b9-m4ttn\" (UID: \"f12eef70-f626-42ce-b4cf-2a07adec9657\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-5bbbb9b4b9-m4ttn" Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.981247 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage09-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage09-crc\") pod \"glance-default-external-api-0\" (UID: \"72b87ae1-6854-475b-a31d-976acc913623\") device mount path \"/mnt/openstack/pv09\"" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.981303 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-llb4x\" (UniqueName: \"kubernetes.io/projected/f12eef70-f626-42ce-b4cf-2a07adec9657-kube-api-access-llb4x\") pod \"dnsmasq-dnsmasq-5bbbb9b4b9-m4ttn\" (UID: \"f12eef70-f626-42ce-b4cf-2a07adec9657\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-5bbbb9b4b9-m4ttn" Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.981388 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/f12eef70-f626-42ce-b4cf-2a07adec9657-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-5bbbb9b4b9-m4ttn\" (UID: \"f12eef70-f626-42ce-b4cf-2a07adec9657\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-5bbbb9b4b9-m4ttn" Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.981423 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/72b87ae1-6854-475b-a31d-976acc913623-config-data\") pod \"glance-default-external-api-0\" (UID: \"72b87ae1-6854-475b-a31d-976acc913623\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.981577 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-dmhw4\" (UniqueName: \"kubernetes.io/projected/72b87ae1-6854-475b-a31d-976acc913623-kube-api-access-dmhw4\") pod \"glance-default-external-api-0\" (UID: \"72b87ae1-6854-475b-a31d-976acc913623\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.981616 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/72b87ae1-6854-475b-a31d-976acc913623-public-tls-certs\") pod \"glance-default-external-api-0\" (UID: \"72b87ae1-6854-475b-a31d-976acc913623\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.981659 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/72b87ae1-6854-475b-a31d-976acc913623-scripts\") pod \"glance-default-external-api-0\" (UID: \"72b87ae1-6854-475b-a31d-976acc913623\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.982755 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f12eef70-f626-42ce-b4cf-2a07adec9657-config\") pod \"dnsmasq-dnsmasq-5bbbb9b4b9-m4ttn\" (UID: \"f12eef70-f626-42ce-b4cf-2a07adec9657\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-5bbbb9b4b9-m4ttn" Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.982969 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"dns-swift-storage-0\" (UniqueName: \"kubernetes.io/configmap/f12eef70-f626-42ce-b4cf-2a07adec9657-dns-swift-storage-0\") pod \"dnsmasq-dnsmasq-5bbbb9b4b9-m4ttn\" (UID: \"f12eef70-f626-42ce-b4cf-2a07adec9657\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-5bbbb9b4b9-m4ttn" Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.983079 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/72b87ae1-6854-475b-a31d-976acc913623-logs\") pod \"glance-default-external-api-0\" (UID: \"72b87ae1-6854-475b-a31d-976acc913623\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.984502 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/72b87ae1-6854-475b-a31d-976acc913623-httpd-run\") pod \"glance-default-external-api-0\" (UID: \"72b87ae1-6854-475b-a31d-976acc913623\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.985216 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/f12eef70-f626-42ce-b4cf-2a07adec9657-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-5bbbb9b4b9-m4ttn\" (UID: \"f12eef70-f626-42ce-b4cf-2a07adec9657\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-5bbbb9b4b9-m4ttn" Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.989687 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/63c18817-7663-466c-845d-bc998c5b7093-fernet-keys" (OuterVolumeSpecName: "fernet-keys") pod "63c18817-7663-466c-845d-bc998c5b7093" (UID: "63c18817-7663-466c-845d-bc998c5b7093"). InnerVolumeSpecName "fernet-keys". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:00 crc kubenswrapper[5052]: I1203 12:21:00.989893 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:21:01 crc kubenswrapper[5052]: I1203 12:21:00.992396 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/63c18817-7663-466c-845d-bc998c5b7093-credential-keys" (OuterVolumeSpecName: "credential-keys") pod "63c18817-7663-466c-845d-bc998c5b7093" (UID: "63c18817-7663-466c-845d-bc998c5b7093"). InnerVolumeSpecName "credential-keys". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:01 crc kubenswrapper[5052]: I1203 12:21:01.002716 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-dmhw4\" (UniqueName: \"kubernetes.io/projected/72b87ae1-6854-475b-a31d-976acc913623-kube-api-access-dmhw4\") pod \"glance-default-external-api-0\" (UID: \"72b87ae1-6854-475b-a31d-976acc913623\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:21:01 crc kubenswrapper[5052]: I1203 12:21:01.003241 5052 scope.go:117] "RemoveContainer" containerID="954e646bee69b2119ee9b7b6cb8846a3b7d5373dd70ba72d80b286777fa3abfd" Dec 03 12:21:01 crc kubenswrapper[5052]: I1203 12:21:01.007124 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/63c18817-7663-466c-845d-bc998c5b7093-scripts" (OuterVolumeSpecName: "scripts") pod "63c18817-7663-466c-845d-bc998c5b7093" (UID: "63c18817-7663-466c-845d-bc998c5b7093"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:01 crc kubenswrapper[5052]: E1203 12:21:01.007271 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="63c18817-7663-466c-845d-bc998c5b7093" containerName="keystone-bootstrap" Dec 03 12:21:01 crc kubenswrapper[5052]: I1203 12:21:01.007300 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="63c18817-7663-466c-845d-bc998c5b7093" containerName="keystone-bootstrap" Dec 03 12:21:01 crc kubenswrapper[5052]: I1203 12:21:01.007646 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="63c18817-7663-466c-845d-bc998c5b7093" containerName="keystone-bootstrap" Dec 03 12:21:01 crc kubenswrapper[5052]: I1203 12:21:01.008232 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-llb4x\" (UniqueName: \"kubernetes.io/projected/f12eef70-f626-42ce-b4cf-2a07adec9657-kube-api-access-llb4x\") pod \"dnsmasq-dnsmasq-5bbbb9b4b9-m4ttn\" (UID: \"f12eef70-f626-42ce-b4cf-2a07adec9657\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-5bbbb9b4b9-m4ttn" Dec 03 12:21:01 crc kubenswrapper[5052]: I1203 12:21:01.009073 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:21:01 crc kubenswrapper[5052]: I1203 12:21:01.013103 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-default-internal-config-data" Dec 03 12:21:01 crc kubenswrapper[5052]: I1203 12:21:01.013482 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-glance-default-internal-svc" Dec 03 12:21:01 crc kubenswrapper[5052]: I1203 12:21:01.022960 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/72b87ae1-6854-475b-a31d-976acc913623-scripts\") pod \"glance-default-external-api-0\" (UID: \"72b87ae1-6854-475b-a31d-976acc913623\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:21:01 crc kubenswrapper[5052]: I1203 12:21:01.024632 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:21:01 crc kubenswrapper[5052]: I1203 12:21:01.033748 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/63c18817-7663-466c-845d-bc998c5b7093-kube-api-access-rpgv8" (OuterVolumeSpecName: "kube-api-access-rpgv8") pod "63c18817-7663-466c-845d-bc998c5b7093" (UID: "63c18817-7663-466c-845d-bc998c5b7093"). InnerVolumeSpecName "kube-api-access-rpgv8". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:21:01 crc kubenswrapper[5052]: I1203 12:21:01.038424 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/72b87ae1-6854-475b-a31d-976acc913623-public-tls-certs\") pod \"glance-default-external-api-0\" (UID: \"72b87ae1-6854-475b-a31d-976acc913623\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:21:01 crc kubenswrapper[5052]: I1203 12:21:01.048814 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/72b87ae1-6854-475b-a31d-976acc913623-config-data\") pod \"glance-default-external-api-0\" (UID: \"72b87ae1-6854-475b-a31d-976acc913623\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:21:01 crc kubenswrapper[5052]: I1203 12:21:01.050615 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/72b87ae1-6854-475b-a31d-976acc913623-combined-ca-bundle\") pod \"glance-default-external-api-0\" (UID: \"72b87ae1-6854-475b-a31d-976acc913623\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:21:01 crc kubenswrapper[5052]: I1203 12:21:01.058631 5052 scope.go:117] "RemoveContainer" containerID="8cbc2d5f8732cb847f4e0ef3c58849a3aac9e0fab347ee31905e53398f6a0f69" Dec 03 12:21:01 crc kubenswrapper[5052]: E1203 12:21:01.060547 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8cbc2d5f8732cb847f4e0ef3c58849a3aac9e0fab347ee31905e53398f6a0f69\": container with ID starting with 8cbc2d5f8732cb847f4e0ef3c58849a3aac9e0fab347ee31905e53398f6a0f69 not found: ID does not exist" containerID="8cbc2d5f8732cb847f4e0ef3c58849a3aac9e0fab347ee31905e53398f6a0f69" Dec 03 12:21:01 crc kubenswrapper[5052]: I1203 12:21:01.060583 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8cbc2d5f8732cb847f4e0ef3c58849a3aac9e0fab347ee31905e53398f6a0f69"} err="failed to get container status \"8cbc2d5f8732cb847f4e0ef3c58849a3aac9e0fab347ee31905e53398f6a0f69\": rpc error: code = NotFound desc = could not find container \"8cbc2d5f8732cb847f4e0ef3c58849a3aac9e0fab347ee31905e53398f6a0f69\": container with ID starting with 8cbc2d5f8732cb847f4e0ef3c58849a3aac9e0fab347ee31905e53398f6a0f69 not found: ID does not exist" Dec 03 12:21:01 crc kubenswrapper[5052]: I1203 12:21:01.060609 5052 scope.go:117] "RemoveContainer" containerID="954e646bee69b2119ee9b7b6cb8846a3b7d5373dd70ba72d80b286777fa3abfd" Dec 03 12:21:01 crc kubenswrapper[5052]: E1203 12:21:01.061989 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"954e646bee69b2119ee9b7b6cb8846a3b7d5373dd70ba72d80b286777fa3abfd\": container with ID starting with 954e646bee69b2119ee9b7b6cb8846a3b7d5373dd70ba72d80b286777fa3abfd not found: ID does not exist" containerID="954e646bee69b2119ee9b7b6cb8846a3b7d5373dd70ba72d80b286777fa3abfd" Dec 03 12:21:01 crc kubenswrapper[5052]: I1203 12:21:01.062043 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"954e646bee69b2119ee9b7b6cb8846a3b7d5373dd70ba72d80b286777fa3abfd"} err="failed to get container status \"954e646bee69b2119ee9b7b6cb8846a3b7d5373dd70ba72d80b286777fa3abfd\": rpc error: code = NotFound desc = could not find container \"954e646bee69b2119ee9b7b6cb8846a3b7d5373dd70ba72d80b286777fa3abfd\": container with ID starting with 954e646bee69b2119ee9b7b6cb8846a3b7d5373dd70ba72d80b286777fa3abfd not found: ID does not exist" Dec 03 12:21:01 crc kubenswrapper[5052]: I1203 12:21:01.062756 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/63c18817-7663-466c-845d-bc998c5b7093-config-data" (OuterVolumeSpecName: "config-data") pod "63c18817-7663-466c-845d-bc998c5b7093" (UID: "63c18817-7663-466c-845d-bc998c5b7093"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:01 crc kubenswrapper[5052]: I1203 12:21:01.069481 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage09-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage09-crc\") pod \"glance-default-external-api-0\" (UID: \"72b87ae1-6854-475b-a31d-976acc913623\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:21:01 crc kubenswrapper[5052]: I1203 12:21:01.076341 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:21:01 crc kubenswrapper[5052]: I1203 12:21:01.084325 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/2d217a2f-3478-41e0-936a-6b1d01530007-httpd-run\") pod \"glance-default-internal-api-0\" (UID: \"2d217a2f-3478-41e0-936a-6b1d01530007\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:21:01 crc kubenswrapper[5052]: I1203 12:21:01.084399 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/2d217a2f-3478-41e0-936a-6b1d01530007-internal-tls-certs\") pod \"glance-default-internal-api-0\" (UID: \"2d217a2f-3478-41e0-936a-6b1d01530007\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:21:01 crc kubenswrapper[5052]: I1203 12:21:01.084457 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/2d217a2f-3478-41e0-936a-6b1d01530007-config-data\") pod \"glance-default-internal-api-0\" (UID: \"2d217a2f-3478-41e0-936a-6b1d01530007\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:21:01 crc kubenswrapper[5052]: I1203 12:21:01.084524 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2d217a2f-3478-41e0-936a-6b1d01530007-combined-ca-bundle\") pod \"glance-default-internal-api-0\" (UID: \"2d217a2f-3478-41e0-936a-6b1d01530007\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:21:01 crc kubenswrapper[5052]: I1203 12:21:01.084600 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-77z56\" (UniqueName: \"kubernetes.io/projected/2d217a2f-3478-41e0-936a-6b1d01530007-kube-api-access-77z56\") pod \"glance-default-internal-api-0\" (UID: \"2d217a2f-3478-41e0-936a-6b1d01530007\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:21:01 crc kubenswrapper[5052]: I1203 12:21:01.084664 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage17-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage17-crc\") pod \"glance-default-internal-api-0\" (UID: \"2d217a2f-3478-41e0-936a-6b1d01530007\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:21:01 crc kubenswrapper[5052]: I1203 12:21:01.084721 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/2d217a2f-3478-41e0-936a-6b1d01530007-logs\") pod \"glance-default-internal-api-0\" (UID: \"2d217a2f-3478-41e0-936a-6b1d01530007\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:21:01 crc kubenswrapper[5052]: I1203 12:21:01.084829 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/2d217a2f-3478-41e0-936a-6b1d01530007-scripts\") pod \"glance-default-internal-api-0\" (UID: \"2d217a2f-3478-41e0-936a-6b1d01530007\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:21:01 crc kubenswrapper[5052]: I1203 12:21:01.084932 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/63c18817-7663-466c-845d-bc998c5b7093-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:01 crc kubenswrapper[5052]: I1203 12:21:01.084949 5052 reconciler_common.go:293] "Volume detached for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/63c18817-7663-466c-845d-bc998c5b7093-fernet-keys\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:01 crc kubenswrapper[5052]: I1203 12:21:01.084961 5052 reconciler_common.go:293] "Volume detached for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/63c18817-7663-466c-845d-bc998c5b7093-credential-keys\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:01 crc kubenswrapper[5052]: I1203 12:21:01.084982 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-rpgv8\" (UniqueName: \"kubernetes.io/projected/63c18817-7663-466c-845d-bc998c5b7093-kube-api-access-rpgv8\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:01 crc kubenswrapper[5052]: I1203 12:21:01.084995 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/63c18817-7663-466c-845d-bc998c5b7093-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:01 crc kubenswrapper[5052]: I1203 12:21:01.090081 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/63c18817-7663-466c-845d-bc998c5b7093-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "63c18817-7663-466c-845d-bc998c5b7093" (UID: "63c18817-7663-466c-845d-bc998c5b7093"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:01 crc kubenswrapper[5052]: I1203 12:21:01.123858 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-5bbbb9b4b9-m4ttn" Dec 03 12:21:01 crc kubenswrapper[5052]: I1203 12:21:01.186372 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/2d217a2f-3478-41e0-936a-6b1d01530007-scripts\") pod \"glance-default-internal-api-0\" (UID: \"2d217a2f-3478-41e0-936a-6b1d01530007\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:21:01 crc kubenswrapper[5052]: I1203 12:21:01.186437 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/2d217a2f-3478-41e0-936a-6b1d01530007-httpd-run\") pod \"glance-default-internal-api-0\" (UID: \"2d217a2f-3478-41e0-936a-6b1d01530007\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:21:01 crc kubenswrapper[5052]: I1203 12:21:01.186470 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/2d217a2f-3478-41e0-936a-6b1d01530007-internal-tls-certs\") pod \"glance-default-internal-api-0\" (UID: \"2d217a2f-3478-41e0-936a-6b1d01530007\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:21:01 crc kubenswrapper[5052]: I1203 12:21:01.186502 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/2d217a2f-3478-41e0-936a-6b1d01530007-config-data\") pod \"glance-default-internal-api-0\" (UID: \"2d217a2f-3478-41e0-936a-6b1d01530007\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:21:01 crc kubenswrapper[5052]: I1203 12:21:01.186549 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2d217a2f-3478-41e0-936a-6b1d01530007-combined-ca-bundle\") pod \"glance-default-internal-api-0\" (UID: \"2d217a2f-3478-41e0-936a-6b1d01530007\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:21:01 crc kubenswrapper[5052]: I1203 12:21:01.186566 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-77z56\" (UniqueName: \"kubernetes.io/projected/2d217a2f-3478-41e0-936a-6b1d01530007-kube-api-access-77z56\") pod \"glance-default-internal-api-0\" (UID: \"2d217a2f-3478-41e0-936a-6b1d01530007\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:21:01 crc kubenswrapper[5052]: I1203 12:21:01.186608 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage17-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage17-crc\") pod \"glance-default-internal-api-0\" (UID: \"2d217a2f-3478-41e0-936a-6b1d01530007\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:21:01 crc kubenswrapper[5052]: I1203 12:21:01.186631 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/2d217a2f-3478-41e0-936a-6b1d01530007-logs\") pod \"glance-default-internal-api-0\" (UID: \"2d217a2f-3478-41e0-936a-6b1d01530007\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:21:01 crc kubenswrapper[5052]: I1203 12:21:01.186701 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/63c18817-7663-466c-845d-bc998c5b7093-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:01 crc kubenswrapper[5052]: I1203 12:21:01.187455 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/2d217a2f-3478-41e0-936a-6b1d01530007-logs\") pod \"glance-default-internal-api-0\" (UID: \"2d217a2f-3478-41e0-936a-6b1d01530007\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:21:01 crc kubenswrapper[5052]: I1203 12:21:01.195937 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/2d217a2f-3478-41e0-936a-6b1d01530007-httpd-run\") pod \"glance-default-internal-api-0\" (UID: \"2d217a2f-3478-41e0-936a-6b1d01530007\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:21:01 crc kubenswrapper[5052]: I1203 12:21:01.196205 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/2d217a2f-3478-41e0-936a-6b1d01530007-scripts\") pod \"glance-default-internal-api-0\" (UID: \"2d217a2f-3478-41e0-936a-6b1d01530007\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:21:01 crc kubenswrapper[5052]: I1203 12:21:01.196493 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage17-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage17-crc\") pod \"glance-default-internal-api-0\" (UID: \"2d217a2f-3478-41e0-936a-6b1d01530007\") device mount path \"/mnt/openstack/pv17\"" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:21:01 crc kubenswrapper[5052]: I1203 12:21:01.199052 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/2d217a2f-3478-41e0-936a-6b1d01530007-config-data\") pod \"glance-default-internal-api-0\" (UID: \"2d217a2f-3478-41e0-936a-6b1d01530007\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:21:01 crc kubenswrapper[5052]: I1203 12:21:01.206732 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2d217a2f-3478-41e0-936a-6b1d01530007-combined-ca-bundle\") pod \"glance-default-internal-api-0\" (UID: \"2d217a2f-3478-41e0-936a-6b1d01530007\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:21:01 crc kubenswrapper[5052]: I1203 12:21:01.215300 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-77z56\" (UniqueName: \"kubernetes.io/projected/2d217a2f-3478-41e0-936a-6b1d01530007-kube-api-access-77z56\") pod \"glance-default-internal-api-0\" (UID: \"2d217a2f-3478-41e0-936a-6b1d01530007\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:21:01 crc kubenswrapper[5052]: I1203 12:21:01.217302 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/2d217a2f-3478-41e0-936a-6b1d01530007-internal-tls-certs\") pod \"glance-default-internal-api-0\" (UID: \"2d217a2f-3478-41e0-936a-6b1d01530007\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:21:01 crc kubenswrapper[5052]: I1203 12:21:01.238777 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage17-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage17-crc\") pod \"glance-default-internal-api-0\" (UID: \"2d217a2f-3478-41e0-936a-6b1d01530007\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:21:01 crc kubenswrapper[5052]: I1203 12:21:01.342676 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:21:01 crc kubenswrapper[5052]: I1203 12:21:01.478937 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-bootstrap-dqvmm" Dec 03 12:21:01 crc kubenswrapper[5052]: I1203 12:21:01.479921 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-bootstrap-dqvmm" event={"ID":"63c18817-7663-466c-845d-bc998c5b7093","Type":"ContainerDied","Data":"ab04e7b8ee1e1ecddfd758a69caeb2cfdd73c7299872f1df1c46bd77dbb02492"} Dec 03 12:21:01 crc kubenswrapper[5052]: I1203 12:21:01.479958 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ab04e7b8ee1e1ecddfd758a69caeb2cfdd73c7299872f1df1c46bd77dbb02492" Dec 03 12:21:01 crc kubenswrapper[5052]: I1203 12:21:01.534389 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/keystone-796c4bc57b-zsjsh"] Dec 03 12:21:01 crc kubenswrapper[5052]: I1203 12:21:01.537227 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-796c4bc57b-zsjsh" Dec 03 12:21:01 crc kubenswrapper[5052]: I1203 12:21:01.573668 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-796c4bc57b-zsjsh"] Dec 03 12:21:01 crc kubenswrapper[5052]: I1203 12:21:01.594977 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/540ee320-be53-46e0-90a3-079a5804124d-fernet-keys\") pod \"keystone-796c4bc57b-zsjsh\" (UID: \"540ee320-be53-46e0-90a3-079a5804124d\") " pod="openstack-kuttl-tests/keystone-796c4bc57b-zsjsh" Dec 03 12:21:01 crc kubenswrapper[5052]: I1203 12:21:01.595050 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/540ee320-be53-46e0-90a3-079a5804124d-config-data\") pod \"keystone-796c4bc57b-zsjsh\" (UID: \"540ee320-be53-46e0-90a3-079a5804124d\") " pod="openstack-kuttl-tests/keystone-796c4bc57b-zsjsh" Dec 03 12:21:01 crc kubenswrapper[5052]: I1203 12:21:01.595087 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/540ee320-be53-46e0-90a3-079a5804124d-combined-ca-bundle\") pod \"keystone-796c4bc57b-zsjsh\" (UID: \"540ee320-be53-46e0-90a3-079a5804124d\") " pod="openstack-kuttl-tests/keystone-796c4bc57b-zsjsh" Dec 03 12:21:01 crc kubenswrapper[5052]: I1203 12:21:01.595115 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/540ee320-be53-46e0-90a3-079a5804124d-public-tls-certs\") pod \"keystone-796c4bc57b-zsjsh\" (UID: \"540ee320-be53-46e0-90a3-079a5804124d\") " pod="openstack-kuttl-tests/keystone-796c4bc57b-zsjsh" Dec 03 12:21:01 crc kubenswrapper[5052]: I1203 12:21:01.595162 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/540ee320-be53-46e0-90a3-079a5804124d-internal-tls-certs\") pod \"keystone-796c4bc57b-zsjsh\" (UID: \"540ee320-be53-46e0-90a3-079a5804124d\") " pod="openstack-kuttl-tests/keystone-796c4bc57b-zsjsh" Dec 03 12:21:01 crc kubenswrapper[5052]: I1203 12:21:01.595206 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/540ee320-be53-46e0-90a3-079a5804124d-scripts\") pod \"keystone-796c4bc57b-zsjsh\" (UID: \"540ee320-be53-46e0-90a3-079a5804124d\") " pod="openstack-kuttl-tests/keystone-796c4bc57b-zsjsh" Dec 03 12:21:01 crc kubenswrapper[5052]: I1203 12:21:01.595243 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/540ee320-be53-46e0-90a3-079a5804124d-credential-keys\") pod \"keystone-796c4bc57b-zsjsh\" (UID: \"540ee320-be53-46e0-90a3-079a5804124d\") " pod="openstack-kuttl-tests/keystone-796c4bc57b-zsjsh" Dec 03 12:21:01 crc kubenswrapper[5052]: I1203 12:21:01.595278 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dl5rg\" (UniqueName: \"kubernetes.io/projected/540ee320-be53-46e0-90a3-079a5804124d-kube-api-access-dl5rg\") pod \"keystone-796c4bc57b-zsjsh\" (UID: \"540ee320-be53-46e0-90a3-079a5804124d\") " pod="openstack-kuttl-tests/keystone-796c4bc57b-zsjsh" Dec 03 12:21:01 crc kubenswrapper[5052]: I1203 12:21:01.614796 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:21:01 crc kubenswrapper[5052]: W1203 12:21:01.632936 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod72b87ae1_6854_475b_a31d_976acc913623.slice/crio-c933b10a6bd214d4066d9a4b73c83f47e712f44c414c9cca8b30ee7c39c1ec9d WatchSource:0}: Error finding container c933b10a6bd214d4066d9a4b73c83f47e712f44c414c9cca8b30ee7c39c1ec9d: Status 404 returned error can't find the container with id c933b10a6bd214d4066d9a4b73c83f47e712f44c414c9cca8b30ee7c39c1ec9d Dec 03 12:21:01 crc kubenswrapper[5052]: I1203 12:21:01.696383 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/540ee320-be53-46e0-90a3-079a5804124d-credential-keys\") pod \"keystone-796c4bc57b-zsjsh\" (UID: \"540ee320-be53-46e0-90a3-079a5804124d\") " pod="openstack-kuttl-tests/keystone-796c4bc57b-zsjsh" Dec 03 12:21:01 crc kubenswrapper[5052]: I1203 12:21:01.696443 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-dl5rg\" (UniqueName: \"kubernetes.io/projected/540ee320-be53-46e0-90a3-079a5804124d-kube-api-access-dl5rg\") pod \"keystone-796c4bc57b-zsjsh\" (UID: \"540ee320-be53-46e0-90a3-079a5804124d\") " pod="openstack-kuttl-tests/keystone-796c4bc57b-zsjsh" Dec 03 12:21:01 crc kubenswrapper[5052]: I1203 12:21:01.696472 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/540ee320-be53-46e0-90a3-079a5804124d-fernet-keys\") pod \"keystone-796c4bc57b-zsjsh\" (UID: \"540ee320-be53-46e0-90a3-079a5804124d\") " pod="openstack-kuttl-tests/keystone-796c4bc57b-zsjsh" Dec 03 12:21:01 crc kubenswrapper[5052]: I1203 12:21:01.696536 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/540ee320-be53-46e0-90a3-079a5804124d-config-data\") pod \"keystone-796c4bc57b-zsjsh\" (UID: \"540ee320-be53-46e0-90a3-079a5804124d\") " pod="openstack-kuttl-tests/keystone-796c4bc57b-zsjsh" Dec 03 12:21:01 crc kubenswrapper[5052]: I1203 12:21:01.696567 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/540ee320-be53-46e0-90a3-079a5804124d-combined-ca-bundle\") pod \"keystone-796c4bc57b-zsjsh\" (UID: \"540ee320-be53-46e0-90a3-079a5804124d\") " pod="openstack-kuttl-tests/keystone-796c4bc57b-zsjsh" Dec 03 12:21:01 crc kubenswrapper[5052]: I1203 12:21:01.696590 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/540ee320-be53-46e0-90a3-079a5804124d-public-tls-certs\") pod \"keystone-796c4bc57b-zsjsh\" (UID: \"540ee320-be53-46e0-90a3-079a5804124d\") " pod="openstack-kuttl-tests/keystone-796c4bc57b-zsjsh" Dec 03 12:21:01 crc kubenswrapper[5052]: I1203 12:21:01.696628 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/540ee320-be53-46e0-90a3-079a5804124d-internal-tls-certs\") pod \"keystone-796c4bc57b-zsjsh\" (UID: \"540ee320-be53-46e0-90a3-079a5804124d\") " pod="openstack-kuttl-tests/keystone-796c4bc57b-zsjsh" Dec 03 12:21:01 crc kubenswrapper[5052]: I1203 12:21:01.697663 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/540ee320-be53-46e0-90a3-079a5804124d-scripts\") pod \"keystone-796c4bc57b-zsjsh\" (UID: \"540ee320-be53-46e0-90a3-079a5804124d\") " pod="openstack-kuttl-tests/keystone-796c4bc57b-zsjsh" Dec 03 12:21:01 crc kubenswrapper[5052]: I1203 12:21:01.703527 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/540ee320-be53-46e0-90a3-079a5804124d-internal-tls-certs\") pod \"keystone-796c4bc57b-zsjsh\" (UID: \"540ee320-be53-46e0-90a3-079a5804124d\") " pod="openstack-kuttl-tests/keystone-796c4bc57b-zsjsh" Dec 03 12:21:01 crc kubenswrapper[5052]: I1203 12:21:01.703534 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/540ee320-be53-46e0-90a3-079a5804124d-public-tls-certs\") pod \"keystone-796c4bc57b-zsjsh\" (UID: \"540ee320-be53-46e0-90a3-079a5804124d\") " pod="openstack-kuttl-tests/keystone-796c4bc57b-zsjsh" Dec 03 12:21:01 crc kubenswrapper[5052]: I1203 12:21:01.703976 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/540ee320-be53-46e0-90a3-079a5804124d-combined-ca-bundle\") pod \"keystone-796c4bc57b-zsjsh\" (UID: \"540ee320-be53-46e0-90a3-079a5804124d\") " pod="openstack-kuttl-tests/keystone-796c4bc57b-zsjsh" Dec 03 12:21:01 crc kubenswrapper[5052]: I1203 12:21:01.704314 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/540ee320-be53-46e0-90a3-079a5804124d-config-data\") pod \"keystone-796c4bc57b-zsjsh\" (UID: \"540ee320-be53-46e0-90a3-079a5804124d\") " pod="openstack-kuttl-tests/keystone-796c4bc57b-zsjsh" Dec 03 12:21:01 crc kubenswrapper[5052]: I1203 12:21:01.707900 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/540ee320-be53-46e0-90a3-079a5804124d-scripts\") pod \"keystone-796c4bc57b-zsjsh\" (UID: \"540ee320-be53-46e0-90a3-079a5804124d\") " pod="openstack-kuttl-tests/keystone-796c4bc57b-zsjsh" Dec 03 12:21:01 crc kubenswrapper[5052]: I1203 12:21:01.714545 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/540ee320-be53-46e0-90a3-079a5804124d-credential-keys\") pod \"keystone-796c4bc57b-zsjsh\" (UID: \"540ee320-be53-46e0-90a3-079a5804124d\") " pod="openstack-kuttl-tests/keystone-796c4bc57b-zsjsh" Dec 03 12:21:01 crc kubenswrapper[5052]: I1203 12:21:01.714985 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="736b6178-1b32-4f5e-a207-931b03aed990" path="/var/lib/kubelet/pods/736b6178-1b32-4f5e-a207-931b03aed990/volumes" Dec 03 12:21:01 crc kubenswrapper[5052]: I1203 12:21:01.716028 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-dl5rg\" (UniqueName: \"kubernetes.io/projected/540ee320-be53-46e0-90a3-079a5804124d-kube-api-access-dl5rg\") pod \"keystone-796c4bc57b-zsjsh\" (UID: \"540ee320-be53-46e0-90a3-079a5804124d\") " pod="openstack-kuttl-tests/keystone-796c4bc57b-zsjsh" Dec 03 12:21:01 crc kubenswrapper[5052]: I1203 12:21:01.716387 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="941d2a2e-d51f-4b67-9b4a-6ff075a2eb0f" path="/var/lib/kubelet/pods/941d2a2e-d51f-4b67-9b4a-6ff075a2eb0f/volumes" Dec 03 12:21:01 crc kubenswrapper[5052]: I1203 12:21:01.716560 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/540ee320-be53-46e0-90a3-079a5804124d-fernet-keys\") pod \"keystone-796c4bc57b-zsjsh\" (UID: \"540ee320-be53-46e0-90a3-079a5804124d\") " pod="openstack-kuttl-tests/keystone-796c4bc57b-zsjsh" Dec 03 12:21:01 crc kubenswrapper[5052]: I1203 12:21:01.717314 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fcc1e62d-c1cc-49a0-93f4-5112fddf89bf" path="/var/lib/kubelet/pods/fcc1e62d-c1cc-49a0-93f4-5112fddf89bf/volumes" Dec 03 12:21:01 crc kubenswrapper[5052]: I1203 12:21:01.737261 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-5bbbb9b4b9-m4ttn"] Dec 03 12:21:01 crc kubenswrapper[5052]: W1203 12:21:01.756453 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf12eef70_f626_42ce_b4cf_2a07adec9657.slice/crio-81629af8a7959b1b67cf8533bb2a47a92e3d1553cbe762f77235d9042375cb75 WatchSource:0}: Error finding container 81629af8a7959b1b67cf8533bb2a47a92e3d1553cbe762f77235d9042375cb75: Status 404 returned error can't find the container with id 81629af8a7959b1b67cf8533bb2a47a92e3d1553cbe762f77235d9042375cb75 Dec 03 12:21:01 crc kubenswrapper[5052]: I1203 12:21:01.889179 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-worker-76d6bdd486-g9c2p" Dec 03 12:21:01 crc kubenswrapper[5052]: I1203 12:21:01.937868 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-796c4bc57b-zsjsh" Dec 03 12:21:02 crc kubenswrapper[5052]: I1203 12:21:02.005691 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/311e25fa-0785-45ce-ae3c-00d32adbf3f4-config-data-custom\") pod \"311e25fa-0785-45ce-ae3c-00d32adbf3f4\" (UID: \"311e25fa-0785-45ce-ae3c-00d32adbf3f4\") " Dec 03 12:21:02 crc kubenswrapper[5052]: I1203 12:21:02.005811 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/311e25fa-0785-45ce-ae3c-00d32adbf3f4-combined-ca-bundle\") pod \"311e25fa-0785-45ce-ae3c-00d32adbf3f4\" (UID: \"311e25fa-0785-45ce-ae3c-00d32adbf3f4\") " Dec 03 12:21:02 crc kubenswrapper[5052]: I1203 12:21:02.006102 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s6scm\" (UniqueName: \"kubernetes.io/projected/311e25fa-0785-45ce-ae3c-00d32adbf3f4-kube-api-access-s6scm\") pod \"311e25fa-0785-45ce-ae3c-00d32adbf3f4\" (UID: \"311e25fa-0785-45ce-ae3c-00d32adbf3f4\") " Dec 03 12:21:02 crc kubenswrapper[5052]: I1203 12:21:02.006132 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/311e25fa-0785-45ce-ae3c-00d32adbf3f4-logs\") pod \"311e25fa-0785-45ce-ae3c-00d32adbf3f4\" (UID: \"311e25fa-0785-45ce-ae3c-00d32adbf3f4\") " Dec 03 12:21:02 crc kubenswrapper[5052]: I1203 12:21:02.006263 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/311e25fa-0785-45ce-ae3c-00d32adbf3f4-config-data\") pod \"311e25fa-0785-45ce-ae3c-00d32adbf3f4\" (UID: \"311e25fa-0785-45ce-ae3c-00d32adbf3f4\") " Dec 03 12:21:02 crc kubenswrapper[5052]: I1203 12:21:02.009153 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/311e25fa-0785-45ce-ae3c-00d32adbf3f4-logs" (OuterVolumeSpecName: "logs") pod "311e25fa-0785-45ce-ae3c-00d32adbf3f4" (UID: "311e25fa-0785-45ce-ae3c-00d32adbf3f4"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:21:02 crc kubenswrapper[5052]: I1203 12:21:02.012836 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/311e25fa-0785-45ce-ae3c-00d32adbf3f4-kube-api-access-s6scm" (OuterVolumeSpecName: "kube-api-access-s6scm") pod "311e25fa-0785-45ce-ae3c-00d32adbf3f4" (UID: "311e25fa-0785-45ce-ae3c-00d32adbf3f4"). InnerVolumeSpecName "kube-api-access-s6scm". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:21:02 crc kubenswrapper[5052]: I1203 12:21:02.012926 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/311e25fa-0785-45ce-ae3c-00d32adbf3f4-config-data-custom" (OuterVolumeSpecName: "config-data-custom") pod "311e25fa-0785-45ce-ae3c-00d32adbf3f4" (UID: "311e25fa-0785-45ce-ae3c-00d32adbf3f4"). InnerVolumeSpecName "config-data-custom". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:02 crc kubenswrapper[5052]: I1203 12:21:02.041632 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:21:02 crc kubenswrapper[5052]: W1203 12:21:02.061672 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2d217a2f_3478_41e0_936a_6b1d01530007.slice/crio-1f0abc5d306c36679cbe12f40d8c06b80fd913c661b1eb429a18cf30c0656ca5 WatchSource:0}: Error finding container 1f0abc5d306c36679cbe12f40d8c06b80fd913c661b1eb429a18cf30c0656ca5: Status 404 returned error can't find the container with id 1f0abc5d306c36679cbe12f40d8c06b80fd913c661b1eb429a18cf30c0656ca5 Dec 03 12:21:02 crc kubenswrapper[5052]: I1203 12:21:02.073713 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/311e25fa-0785-45ce-ae3c-00d32adbf3f4-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "311e25fa-0785-45ce-ae3c-00d32adbf3f4" (UID: "311e25fa-0785-45ce-ae3c-00d32adbf3f4"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:02 crc kubenswrapper[5052]: I1203 12:21:02.099313 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/311e25fa-0785-45ce-ae3c-00d32adbf3f4-config-data" (OuterVolumeSpecName: "config-data") pod "311e25fa-0785-45ce-ae3c-00d32adbf3f4" (UID: "311e25fa-0785-45ce-ae3c-00d32adbf3f4"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:02 crc kubenswrapper[5052]: I1203 12:21:02.108858 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-s6scm\" (UniqueName: \"kubernetes.io/projected/311e25fa-0785-45ce-ae3c-00d32adbf3f4-kube-api-access-s6scm\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:02 crc kubenswrapper[5052]: I1203 12:21:02.108900 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/311e25fa-0785-45ce-ae3c-00d32adbf3f4-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:02 crc kubenswrapper[5052]: I1203 12:21:02.108914 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/311e25fa-0785-45ce-ae3c-00d32adbf3f4-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:02 crc kubenswrapper[5052]: I1203 12:21:02.108928 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/311e25fa-0785-45ce-ae3c-00d32adbf3f4-config-data-custom\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:02 crc kubenswrapper[5052]: I1203 12:21:02.108940 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/311e25fa-0785-45ce-ae3c-00d32adbf3f4-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:02 crc kubenswrapper[5052]: I1203 12:21:02.463727 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-796c4bc57b-zsjsh"] Dec 03 12:21:02 crc kubenswrapper[5052]: I1203 12:21:02.493438 5052 generic.go:334] "Generic (PLEG): container finished" podID="311e25fa-0785-45ce-ae3c-00d32adbf3f4" containerID="b7e1028dbd6661eea3ae89a9c9f561e69d762520ab29fe9c2720e56e95408280" exitCode=0 Dec 03 12:21:02 crc kubenswrapper[5052]: I1203 12:21:02.493542 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-worker-76d6bdd486-g9c2p" event={"ID":"311e25fa-0785-45ce-ae3c-00d32adbf3f4","Type":"ContainerDied","Data":"b7e1028dbd6661eea3ae89a9c9f561e69d762520ab29fe9c2720e56e95408280"} Dec 03 12:21:02 crc kubenswrapper[5052]: I1203 12:21:02.493576 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-worker-76d6bdd486-g9c2p" event={"ID":"311e25fa-0785-45ce-ae3c-00d32adbf3f4","Type":"ContainerDied","Data":"7aeaad7bc91a3ea84cb0f054a2df5f62f36ffac7995e71358e1f6f08838a3afb"} Dec 03 12:21:02 crc kubenswrapper[5052]: I1203 12:21:02.493596 5052 scope.go:117] "RemoveContainer" containerID="b7e1028dbd6661eea3ae89a9c9f561e69d762520ab29fe9c2720e56e95408280" Dec 03 12:21:02 crc kubenswrapper[5052]: I1203 12:21:02.493759 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-worker-76d6bdd486-g9c2p" Dec 03 12:21:02 crc kubenswrapper[5052]: I1203 12:21:02.505794 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"2d217a2f-3478-41e0-936a-6b1d01530007","Type":"ContainerStarted","Data":"1f0abc5d306c36679cbe12f40d8c06b80fd913c661b1eb429a18cf30c0656ca5"} Dec 03 12:21:02 crc kubenswrapper[5052]: I1203 12:21:02.510085 5052 generic.go:334] "Generic (PLEG): container finished" podID="f12eef70-f626-42ce-b4cf-2a07adec9657" containerID="e7e7dd0b02628c62826b4852f91d8f2be59cbd148aede2a98c85fb101457d775" exitCode=0 Dec 03 12:21:02 crc kubenswrapper[5052]: I1203 12:21:02.510165 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-5bbbb9b4b9-m4ttn" event={"ID":"f12eef70-f626-42ce-b4cf-2a07adec9657","Type":"ContainerDied","Data":"e7e7dd0b02628c62826b4852f91d8f2be59cbd148aede2a98c85fb101457d775"} Dec 03 12:21:02 crc kubenswrapper[5052]: I1203 12:21:02.510207 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-5bbbb9b4b9-m4ttn" event={"ID":"f12eef70-f626-42ce-b4cf-2a07adec9657","Type":"ContainerStarted","Data":"81629af8a7959b1b67cf8533bb2a47a92e3d1553cbe762f77235d9042375cb75"} Dec 03 12:21:02 crc kubenswrapper[5052]: I1203 12:21:02.517484 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-796c4bc57b-zsjsh" event={"ID":"540ee320-be53-46e0-90a3-079a5804124d","Type":"ContainerStarted","Data":"734b8cd29c4f2dc46d2026b98bc155fc68f0b7742c2c1a028e385b827e359fe2"} Dec 03 12:21:02 crc kubenswrapper[5052]: I1203 12:21:02.520821 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"72b87ae1-6854-475b-a31d-976acc913623","Type":"ContainerStarted","Data":"c933b10a6bd214d4066d9a4b73c83f47e712f44c414c9cca8b30ee7c39c1ec9d"} Dec 03 12:21:02 crc kubenswrapper[5052]: I1203 12:21:02.678643 5052 scope.go:117] "RemoveContainer" containerID="522e7f93cdd131cc57ae189abb416b435302969048510ba07854b6d472d399d4" Dec 03 12:21:02 crc kubenswrapper[5052]: I1203 12:21:02.711161 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-worker-76d6bdd486-g9c2p"] Dec 03 12:21:02 crc kubenswrapper[5052]: I1203 12:21:02.727809 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/barbican-worker-76d6bdd486-g9c2p"] Dec 03 12:21:02 crc kubenswrapper[5052]: I1203 12:21:02.735182 5052 scope.go:117] "RemoveContainer" containerID="b7e1028dbd6661eea3ae89a9c9f561e69d762520ab29fe9c2720e56e95408280" Dec 03 12:21:02 crc kubenswrapper[5052]: E1203 12:21:02.735684 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b7e1028dbd6661eea3ae89a9c9f561e69d762520ab29fe9c2720e56e95408280\": container with ID starting with b7e1028dbd6661eea3ae89a9c9f561e69d762520ab29fe9c2720e56e95408280 not found: ID does not exist" containerID="b7e1028dbd6661eea3ae89a9c9f561e69d762520ab29fe9c2720e56e95408280" Dec 03 12:21:02 crc kubenswrapper[5052]: I1203 12:21:02.735736 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b7e1028dbd6661eea3ae89a9c9f561e69d762520ab29fe9c2720e56e95408280"} err="failed to get container status \"b7e1028dbd6661eea3ae89a9c9f561e69d762520ab29fe9c2720e56e95408280\": rpc error: code = NotFound desc = could not find container \"b7e1028dbd6661eea3ae89a9c9f561e69d762520ab29fe9c2720e56e95408280\": container with ID starting with b7e1028dbd6661eea3ae89a9c9f561e69d762520ab29fe9c2720e56e95408280 not found: ID does not exist" Dec 03 12:21:02 crc kubenswrapper[5052]: I1203 12:21:02.735767 5052 scope.go:117] "RemoveContainer" containerID="522e7f93cdd131cc57ae189abb416b435302969048510ba07854b6d472d399d4" Dec 03 12:21:02 crc kubenswrapper[5052]: E1203 12:21:02.736182 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"522e7f93cdd131cc57ae189abb416b435302969048510ba07854b6d472d399d4\": container with ID starting with 522e7f93cdd131cc57ae189abb416b435302969048510ba07854b6d472d399d4 not found: ID does not exist" containerID="522e7f93cdd131cc57ae189abb416b435302969048510ba07854b6d472d399d4" Dec 03 12:21:02 crc kubenswrapper[5052]: I1203 12:21:02.736212 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"522e7f93cdd131cc57ae189abb416b435302969048510ba07854b6d472d399d4"} err="failed to get container status \"522e7f93cdd131cc57ae189abb416b435302969048510ba07854b6d472d399d4\": rpc error: code = NotFound desc = could not find container \"522e7f93cdd131cc57ae189abb416b435302969048510ba07854b6d472d399d4\": container with ID starting with 522e7f93cdd131cc57ae189abb416b435302969048510ba07854b6d472d399d4 not found: ID does not exist" Dec 03 12:21:03 crc kubenswrapper[5052]: I1203 12:21:03.502469 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:21:03 crc kubenswrapper[5052]: I1203 12:21:03.567323 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-796c4bc57b-zsjsh" event={"ID":"540ee320-be53-46e0-90a3-079a5804124d","Type":"ContainerStarted","Data":"b80b6e8c4ad454ae227b95729ae24f6a9c106e69b758ed3c06b5fe2924584a0c"} Dec 03 12:21:03 crc kubenswrapper[5052]: I1203 12:21:03.567647 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/keystone-796c4bc57b-zsjsh" Dec 03 12:21:03 crc kubenswrapper[5052]: I1203 12:21:03.570986 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"72b87ae1-6854-475b-a31d-976acc913623","Type":"ContainerStarted","Data":"dc3b9cb9559bad706ad6fd805545ec2569385c158301c0e04856678a08e6dfe0"} Dec 03 12:21:03 crc kubenswrapper[5052]: I1203 12:21:03.571034 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"72b87ae1-6854-475b-a31d-976acc913623","Type":"ContainerStarted","Data":"ace907d0e00f36ffb29d30e119e8630da96914e43d3d15c7fdb73b39b73ccfd8"} Dec 03 12:21:03 crc kubenswrapper[5052]: I1203 12:21:03.575051 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"2d217a2f-3478-41e0-936a-6b1d01530007","Type":"ContainerStarted","Data":"c432832de2a3a0521fe28359f14c05384338a7557e1245553a8611f5a30ee883"} Dec 03 12:21:03 crc kubenswrapper[5052]: I1203 12:21:03.575113 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"2d217a2f-3478-41e0-936a-6b1d01530007","Type":"ContainerStarted","Data":"e124f853c8070429efcc9a6778d70b3a8c1c60fccd3f62bb22a54362a2148f4e"} Dec 03 12:21:03 crc kubenswrapper[5052]: I1203 12:21:03.577669 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-5bbbb9b4b9-m4ttn" event={"ID":"f12eef70-f626-42ce-b4cf-2a07adec9657","Type":"ContainerStarted","Data":"1e7096c07ede142e1bcdfe92a44f991a0ced70d36647bcc1f6c50501e03a2b5d"} Dec 03 12:21:03 crc kubenswrapper[5052]: I1203 12:21:03.577871 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-5bbbb9b4b9-m4ttn" Dec 03 12:21:03 crc kubenswrapper[5052]: I1203 12:21:03.606942 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/keystone-796c4bc57b-zsjsh" podStartSLOduration=2.606915393 podStartE2EDuration="2.606915393s" podCreationTimestamp="2025-12-03 12:21:01 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:21:03.583992548 +0000 UTC m=+3654.172234322" watchObservedRunningTime="2025-12-03 12:21:03.606915393 +0000 UTC m=+3654.195157157" Dec 03 12:21:03 crc kubenswrapper[5052]: I1203 12:21:03.610220 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-5bbbb9b4b9-m4ttn" podStartSLOduration=3.610206366 podStartE2EDuration="3.610206366s" podCreationTimestamp="2025-12-03 12:21:00 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:21:03.607795638 +0000 UTC m=+3654.196037402" watchObservedRunningTime="2025-12-03 12:21:03.610206366 +0000 UTC m=+3654.198448130" Dec 03 12:21:03 crc kubenswrapper[5052]: I1203 12:21:03.633082 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/glance-default-internal-api-0" podStartSLOduration=3.633063769 podStartE2EDuration="3.633063769s" podCreationTimestamp="2025-12-03 12:21:00 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:21:03.629371294 +0000 UTC m=+3654.217613058" watchObservedRunningTime="2025-12-03 12:21:03.633063769 +0000 UTC m=+3654.221305533" Dec 03 12:21:03 crc kubenswrapper[5052]: I1203 12:21:03.662269 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/glance-default-external-api-0" podStartSLOduration=3.662252392 podStartE2EDuration="3.662252392s" podCreationTimestamp="2025-12-03 12:21:00 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:21:03.652172674 +0000 UTC m=+3654.240414438" watchObservedRunningTime="2025-12-03 12:21:03.662252392 +0000 UTC m=+3654.250494156" Dec 03 12:21:03 crc kubenswrapper[5052]: I1203 12:21:03.684585 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="311e25fa-0785-45ce-ae3c-00d32adbf3f4" path="/var/lib/kubelet/pods/311e25fa-0785-45ce-ae3c-00d32adbf3f4/volumes" Dec 03 12:21:03 crc kubenswrapper[5052]: I1203 12:21:03.819817 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:21:05 crc kubenswrapper[5052]: I1203 12:21:05.810603 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:21:06 crc kubenswrapper[5052]: I1203 12:21:06.253734 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/barbican-api-644f564b46-cjtkx" Dec 03 12:21:06 crc kubenswrapper[5052]: I1203 12:21:06.270043 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/barbican-api-644f564b46-cjtkx" Dec 03 12:21:06 crc kubenswrapper[5052]: I1203 12:21:06.350652 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-api-64fc65b95f-tl6vk"] Dec 03 12:21:06 crc kubenswrapper[5052]: I1203 12:21:06.350917 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/barbican-api-64fc65b95f-tl6vk" podUID="7076e341-78ad-4783-8a0e-2f3914f92a5b" containerName="barbican-api-log" containerID="cri-o://6f32818c01018bb4abae02721a800dd04ca46b072f663e382163bc70ee748e43" gracePeriod=30 Dec 03 12:21:06 crc kubenswrapper[5052]: I1203 12:21:06.351005 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/barbican-api-64fc65b95f-tl6vk" podUID="7076e341-78ad-4783-8a0e-2f3914f92a5b" containerName="barbican-api" containerID="cri-o://b42993f3a56100f0e34428fc72958d399bed23bdc2ca879c6c31d4ce0a789544" gracePeriod=30 Dec 03 12:21:06 crc kubenswrapper[5052]: I1203 12:21:06.618201 5052 generic.go:334] "Generic (PLEG): container finished" podID="7076e341-78ad-4783-8a0e-2f3914f92a5b" containerID="6f32818c01018bb4abae02721a800dd04ca46b072f663e382163bc70ee748e43" exitCode=143 Dec 03 12:21:06 crc kubenswrapper[5052]: I1203 12:21:06.618426 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-64fc65b95f-tl6vk" event={"ID":"7076e341-78ad-4783-8a0e-2f3914f92a5b","Type":"ContainerDied","Data":"6f32818c01018bb4abae02721a800dd04ca46b072f663e382163bc70ee748e43"} Dec 03 12:21:08 crc kubenswrapper[5052]: I1203 12:21:08.673740 5052 scope.go:117] "RemoveContainer" containerID="09b4a0072741f60499d24f40cc4ff3406d5d22671ced431363ab865172786300" Dec 03 12:21:08 crc kubenswrapper[5052]: E1203 12:21:08.674286 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 12:21:09 crc kubenswrapper[5052]: I1203 12:21:09.508902 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/barbican-api-64fc65b95f-tl6vk" podUID="7076e341-78ad-4783-8a0e-2f3914f92a5b" containerName="barbican-api-log" probeResult="failure" output="Get \"https://10.217.1.102:9311/healthcheck\": read tcp 10.217.0.2:47156->10.217.1.102:9311: read: connection reset by peer" Dec 03 12:21:09 crc kubenswrapper[5052]: I1203 12:21:09.508985 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/barbican-api-64fc65b95f-tl6vk" podUID="7076e341-78ad-4783-8a0e-2f3914f92a5b" containerName="barbican-api" probeResult="failure" output="Get \"https://10.217.1.102:9311/healthcheck\": read tcp 10.217.0.2:47166->10.217.1.102:9311: read: connection reset by peer" Dec 03 12:21:09 crc kubenswrapper[5052]: I1203 12:21:09.653213 5052 generic.go:334] "Generic (PLEG): container finished" podID="7076e341-78ad-4783-8a0e-2f3914f92a5b" containerID="b42993f3a56100f0e34428fc72958d399bed23bdc2ca879c6c31d4ce0a789544" exitCode=0 Dec 03 12:21:09 crc kubenswrapper[5052]: I1203 12:21:09.653267 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-64fc65b95f-tl6vk" event={"ID":"7076e341-78ad-4783-8a0e-2f3914f92a5b","Type":"ContainerDied","Data":"b42993f3a56100f0e34428fc72958d399bed23bdc2ca879c6c31d4ce0a789544"} Dec 03 12:21:09 crc kubenswrapper[5052]: I1203 12:21:09.932755 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-api-64fc65b95f-tl6vk" Dec 03 12:21:10 crc kubenswrapper[5052]: I1203 12:21:10.113574 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/7076e341-78ad-4783-8a0e-2f3914f92a5b-config-data\") pod \"7076e341-78ad-4783-8a0e-2f3914f92a5b\" (UID: \"7076e341-78ad-4783-8a0e-2f3914f92a5b\") " Dec 03 12:21:10 crc kubenswrapper[5052]: I1203 12:21:10.113693 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/7076e341-78ad-4783-8a0e-2f3914f92a5b-logs\") pod \"7076e341-78ad-4783-8a0e-2f3914f92a5b\" (UID: \"7076e341-78ad-4783-8a0e-2f3914f92a5b\") " Dec 03 12:21:10 crc kubenswrapper[5052]: I1203 12:21:10.113744 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/7076e341-78ad-4783-8a0e-2f3914f92a5b-combined-ca-bundle\") pod \"7076e341-78ad-4783-8a0e-2f3914f92a5b\" (UID: \"7076e341-78ad-4783-8a0e-2f3914f92a5b\") " Dec 03 12:21:10 crc kubenswrapper[5052]: I1203 12:21:10.113766 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/7076e341-78ad-4783-8a0e-2f3914f92a5b-internal-tls-certs\") pod \"7076e341-78ad-4783-8a0e-2f3914f92a5b\" (UID: \"7076e341-78ad-4783-8a0e-2f3914f92a5b\") " Dec 03 12:21:10 crc kubenswrapper[5052]: I1203 12:21:10.113804 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dgggj\" (UniqueName: \"kubernetes.io/projected/7076e341-78ad-4783-8a0e-2f3914f92a5b-kube-api-access-dgggj\") pod \"7076e341-78ad-4783-8a0e-2f3914f92a5b\" (UID: \"7076e341-78ad-4783-8a0e-2f3914f92a5b\") " Dec 03 12:21:10 crc kubenswrapper[5052]: I1203 12:21:10.113856 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/7076e341-78ad-4783-8a0e-2f3914f92a5b-public-tls-certs\") pod \"7076e341-78ad-4783-8a0e-2f3914f92a5b\" (UID: \"7076e341-78ad-4783-8a0e-2f3914f92a5b\") " Dec 03 12:21:10 crc kubenswrapper[5052]: I1203 12:21:10.113969 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/7076e341-78ad-4783-8a0e-2f3914f92a5b-config-data-custom\") pod \"7076e341-78ad-4783-8a0e-2f3914f92a5b\" (UID: \"7076e341-78ad-4783-8a0e-2f3914f92a5b\") " Dec 03 12:21:10 crc kubenswrapper[5052]: I1203 12:21:10.114986 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7076e341-78ad-4783-8a0e-2f3914f92a5b-logs" (OuterVolumeSpecName: "logs") pod "7076e341-78ad-4783-8a0e-2f3914f92a5b" (UID: "7076e341-78ad-4783-8a0e-2f3914f92a5b"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:21:10 crc kubenswrapper[5052]: I1203 12:21:10.124809 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7076e341-78ad-4783-8a0e-2f3914f92a5b-kube-api-access-dgggj" (OuterVolumeSpecName: "kube-api-access-dgggj") pod "7076e341-78ad-4783-8a0e-2f3914f92a5b" (UID: "7076e341-78ad-4783-8a0e-2f3914f92a5b"). InnerVolumeSpecName "kube-api-access-dgggj". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:21:10 crc kubenswrapper[5052]: I1203 12:21:10.131922 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7076e341-78ad-4783-8a0e-2f3914f92a5b-config-data-custom" (OuterVolumeSpecName: "config-data-custom") pod "7076e341-78ad-4783-8a0e-2f3914f92a5b" (UID: "7076e341-78ad-4783-8a0e-2f3914f92a5b"). InnerVolumeSpecName "config-data-custom". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:10 crc kubenswrapper[5052]: I1203 12:21:10.148894 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7076e341-78ad-4783-8a0e-2f3914f92a5b-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "7076e341-78ad-4783-8a0e-2f3914f92a5b" (UID: "7076e341-78ad-4783-8a0e-2f3914f92a5b"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:10 crc kubenswrapper[5052]: I1203 12:21:10.175112 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7076e341-78ad-4783-8a0e-2f3914f92a5b-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "7076e341-78ad-4783-8a0e-2f3914f92a5b" (UID: "7076e341-78ad-4783-8a0e-2f3914f92a5b"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:10 crc kubenswrapper[5052]: I1203 12:21:10.176005 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7076e341-78ad-4783-8a0e-2f3914f92a5b-config-data" (OuterVolumeSpecName: "config-data") pod "7076e341-78ad-4783-8a0e-2f3914f92a5b" (UID: "7076e341-78ad-4783-8a0e-2f3914f92a5b"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:10 crc kubenswrapper[5052]: I1203 12:21:10.180380 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7076e341-78ad-4783-8a0e-2f3914f92a5b-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "7076e341-78ad-4783-8a0e-2f3914f92a5b" (UID: "7076e341-78ad-4783-8a0e-2f3914f92a5b"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:10 crc kubenswrapper[5052]: I1203 12:21:10.217454 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/7076e341-78ad-4783-8a0e-2f3914f92a5b-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:10 crc kubenswrapper[5052]: I1203 12:21:10.217541 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/7076e341-78ad-4783-8a0e-2f3914f92a5b-config-data-custom\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:10 crc kubenswrapper[5052]: I1203 12:21:10.217554 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/7076e341-78ad-4783-8a0e-2f3914f92a5b-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:10 crc kubenswrapper[5052]: I1203 12:21:10.217565 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/7076e341-78ad-4783-8a0e-2f3914f92a5b-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:10 crc kubenswrapper[5052]: I1203 12:21:10.217577 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/7076e341-78ad-4783-8a0e-2f3914f92a5b-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:10 crc kubenswrapper[5052]: I1203 12:21:10.217589 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/7076e341-78ad-4783-8a0e-2f3914f92a5b-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:10 crc kubenswrapper[5052]: I1203 12:21:10.217604 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-dgggj\" (UniqueName: \"kubernetes.io/projected/7076e341-78ad-4783-8a0e-2f3914f92a5b-kube-api-access-dgggj\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:10 crc kubenswrapper[5052]: I1203 12:21:10.666168 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-64fc65b95f-tl6vk" event={"ID":"7076e341-78ad-4783-8a0e-2f3914f92a5b","Type":"ContainerDied","Data":"3d0cb3c33636c943ed49faf4eb8517b934ecd6d4fbd3df9e62306709fc2bce64"} Dec 03 12:21:10 crc kubenswrapper[5052]: I1203 12:21:10.666208 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-api-64fc65b95f-tl6vk" Dec 03 12:21:10 crc kubenswrapper[5052]: I1203 12:21:10.666838 5052 scope.go:117] "RemoveContainer" containerID="b42993f3a56100f0e34428fc72958d399bed23bdc2ca879c6c31d4ce0a789544" Dec 03 12:21:10 crc kubenswrapper[5052]: I1203 12:21:10.691794 5052 scope.go:117] "RemoveContainer" containerID="6f32818c01018bb4abae02721a800dd04ca46b072f663e382163bc70ee748e43" Dec 03 12:21:10 crc kubenswrapper[5052]: I1203 12:21:10.712131 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-api-64fc65b95f-tl6vk"] Dec 03 12:21:10 crc kubenswrapper[5052]: I1203 12:21:10.720542 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/barbican-api-64fc65b95f-tl6vk"] Dec 03 12:21:11 crc kubenswrapper[5052]: I1203 12:21:11.078445 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:21:11 crc kubenswrapper[5052]: I1203 12:21:11.078495 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:21:11 crc kubenswrapper[5052]: I1203 12:21:11.121155 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:21:11 crc kubenswrapper[5052]: I1203 12:21:11.122775 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:21:11 crc kubenswrapper[5052]: I1203 12:21:11.125938 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-5bbbb9b4b9-m4ttn" Dec 03 12:21:11 crc kubenswrapper[5052]: I1203 12:21:11.276323 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-7598b67757-jsb6x"] Dec 03 12:21:11 crc kubenswrapper[5052]: I1203 12:21:11.276561 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-7598b67757-jsb6x" podUID="20831b4d-3e7e-45c3-accd-406b36c385fa" containerName="dnsmasq-dns" containerID="cri-o://332f879d8fbb05e0197024edb0f25a3acd752356221f62a3cae8691bc75f6f17" gracePeriod=10 Dec 03 12:21:11 crc kubenswrapper[5052]: I1203 12:21:11.343369 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:21:11 crc kubenswrapper[5052]: I1203 12:21:11.343420 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:21:11 crc kubenswrapper[5052]: I1203 12:21:11.375030 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:21:11 crc kubenswrapper[5052]: I1203 12:21:11.390321 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:21:11 crc kubenswrapper[5052]: I1203 12:21:11.682096 5052 generic.go:334] "Generic (PLEG): container finished" podID="20831b4d-3e7e-45c3-accd-406b36c385fa" containerID="332f879d8fbb05e0197024edb0f25a3acd752356221f62a3cae8691bc75f6f17" exitCode=0 Dec 03 12:21:11 crc kubenswrapper[5052]: I1203 12:21:11.687870 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7076e341-78ad-4783-8a0e-2f3914f92a5b" path="/var/lib/kubelet/pods/7076e341-78ad-4783-8a0e-2f3914f92a5b/volumes" Dec 03 12:21:11 crc kubenswrapper[5052]: I1203 12:21:11.688649 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-7598b67757-jsb6x" event={"ID":"20831b4d-3e7e-45c3-accd-406b36c385fa","Type":"ContainerDied","Data":"332f879d8fbb05e0197024edb0f25a3acd752356221f62a3cae8691bc75f6f17"} Dec 03 12:21:11 crc kubenswrapper[5052]: I1203 12:21:11.688687 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:21:11 crc kubenswrapper[5052]: I1203 12:21:11.688704 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:21:11 crc kubenswrapper[5052]: I1203 12:21:11.688715 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:21:11 crc kubenswrapper[5052]: I1203 12:21:11.688726 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:21:11 crc kubenswrapper[5052]: I1203 12:21:11.772661 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-7598b67757-jsb6x" Dec 03 12:21:11 crc kubenswrapper[5052]: I1203 12:21:11.862834 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"dns-swift-storage-0\" (UniqueName: \"kubernetes.io/configmap/20831b4d-3e7e-45c3-accd-406b36c385fa-dns-swift-storage-0\") pod \"20831b4d-3e7e-45c3-accd-406b36c385fa\" (UID: \"20831b4d-3e7e-45c3-accd-406b36c385fa\") " Dec 03 12:21:11 crc kubenswrapper[5052]: I1203 12:21:11.863231 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/20831b4d-3e7e-45c3-accd-406b36c385fa-dnsmasq-svc\") pod \"20831b4d-3e7e-45c3-accd-406b36c385fa\" (UID: \"20831b4d-3e7e-45c3-accd-406b36c385fa\") " Dec 03 12:21:11 crc kubenswrapper[5052]: I1203 12:21:11.863371 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/20831b4d-3e7e-45c3-accd-406b36c385fa-config\") pod \"20831b4d-3e7e-45c3-accd-406b36c385fa\" (UID: \"20831b4d-3e7e-45c3-accd-406b36c385fa\") " Dec 03 12:21:11 crc kubenswrapper[5052]: I1203 12:21:11.863436 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mhqrl\" (UniqueName: \"kubernetes.io/projected/20831b4d-3e7e-45c3-accd-406b36c385fa-kube-api-access-mhqrl\") pod \"20831b4d-3e7e-45c3-accd-406b36c385fa\" (UID: \"20831b4d-3e7e-45c3-accd-406b36c385fa\") " Dec 03 12:21:11 crc kubenswrapper[5052]: I1203 12:21:11.870362 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/20831b4d-3e7e-45c3-accd-406b36c385fa-kube-api-access-mhqrl" (OuterVolumeSpecName: "kube-api-access-mhqrl") pod "20831b4d-3e7e-45c3-accd-406b36c385fa" (UID: "20831b4d-3e7e-45c3-accd-406b36c385fa"). InnerVolumeSpecName "kube-api-access-mhqrl". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:21:11 crc kubenswrapper[5052]: I1203 12:21:11.915059 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/20831b4d-3e7e-45c3-accd-406b36c385fa-config" (OuterVolumeSpecName: "config") pod "20831b4d-3e7e-45c3-accd-406b36c385fa" (UID: "20831b4d-3e7e-45c3-accd-406b36c385fa"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:21:11 crc kubenswrapper[5052]: I1203 12:21:11.915130 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/20831b4d-3e7e-45c3-accd-406b36c385fa-dnsmasq-svc" (OuterVolumeSpecName: "dnsmasq-svc") pod "20831b4d-3e7e-45c3-accd-406b36c385fa" (UID: "20831b4d-3e7e-45c3-accd-406b36c385fa"). InnerVolumeSpecName "dnsmasq-svc". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:21:11 crc kubenswrapper[5052]: I1203 12:21:11.919245 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/20831b4d-3e7e-45c3-accd-406b36c385fa-dns-swift-storage-0" (OuterVolumeSpecName: "dns-swift-storage-0") pod "20831b4d-3e7e-45c3-accd-406b36c385fa" (UID: "20831b4d-3e7e-45c3-accd-406b36c385fa"). InnerVolumeSpecName "dns-swift-storage-0". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:21:11 crc kubenswrapper[5052]: I1203 12:21:11.967145 5052 reconciler_common.go:293] "Volume detached for volume \"dns-swift-storage-0\" (UniqueName: \"kubernetes.io/configmap/20831b4d-3e7e-45c3-accd-406b36c385fa-dns-swift-storage-0\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:11 crc kubenswrapper[5052]: I1203 12:21:11.967193 5052 reconciler_common.go:293] "Volume detached for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/20831b4d-3e7e-45c3-accd-406b36c385fa-dnsmasq-svc\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:11 crc kubenswrapper[5052]: I1203 12:21:11.967205 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/20831b4d-3e7e-45c3-accd-406b36c385fa-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:11 crc kubenswrapper[5052]: I1203 12:21:11.967220 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-mhqrl\" (UniqueName: \"kubernetes.io/projected/20831b4d-3e7e-45c3-accd-406b36c385fa-kube-api-access-mhqrl\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:12 crc kubenswrapper[5052]: I1203 12:21:12.698672 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-7598b67757-jsb6x" event={"ID":"20831b4d-3e7e-45c3-accd-406b36c385fa","Type":"ContainerDied","Data":"e405a3ca4acb967d47085b9658620a8dda399136e2cdcbcdbf9f34cef149ac5f"} Dec 03 12:21:12 crc kubenswrapper[5052]: I1203 12:21:12.698723 5052 scope.go:117] "RemoveContainer" containerID="332f879d8fbb05e0197024edb0f25a3acd752356221f62a3cae8691bc75f6f17" Dec 03 12:21:12 crc kubenswrapper[5052]: I1203 12:21:12.699783 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-7598b67757-jsb6x" Dec 03 12:21:12 crc kubenswrapper[5052]: I1203 12:21:12.710013 5052 generic.go:334] "Generic (PLEG): container finished" podID="c279eaf1-2ee2-46fe-88be-9a5a65d4897e" containerID="461a75cc1c9b99044785a22ff9dae21bbe4db12db4d1af47322cef18e6d1565c" exitCode=0 Dec 03 12:21:12 crc kubenswrapper[5052]: I1203 12:21:12.710295 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" event={"ID":"c279eaf1-2ee2-46fe-88be-9a5a65d4897e","Type":"ContainerDied","Data":"461a75cc1c9b99044785a22ff9dae21bbe4db12db4d1af47322cef18e6d1565c"} Dec 03 12:21:12 crc kubenswrapper[5052]: I1203 12:21:12.714323 5052 generic.go:334] "Generic (PLEG): container finished" podID="c14a906c-f19d-4806-9110-8c47b38c5ae0" containerID="4eb8d3dd99172fe5ebfc87ffeaf97722130ac6901117c3bdc435b6f6a054e23f" exitCode=0 Dec 03 12:21:12 crc kubenswrapper[5052]: I1203 12:21:12.715446 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-server-0" event={"ID":"c14a906c-f19d-4806-9110-8c47b38c5ae0","Type":"ContainerDied","Data":"4eb8d3dd99172fe5ebfc87ffeaf97722130ac6901117c3bdc435b6f6a054e23f"} Dec 03 12:21:12 crc kubenswrapper[5052]: I1203 12:21:12.725882 5052 scope.go:117] "RemoveContainer" containerID="c0afa390492cf53e06b476785e0f501d2d08b04be1d1d0bea16a94ce268161a2" Dec 03 12:21:12 crc kubenswrapper[5052]: I1203 12:21:12.814972 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-7598b67757-jsb6x"] Dec 03 12:21:12 crc kubenswrapper[5052]: I1203 12:21:12.839698 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-7598b67757-jsb6x"] Dec 03 12:21:13 crc kubenswrapper[5052]: I1203 12:21:13.686898 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="20831b4d-3e7e-45c3-accd-406b36c385fa" path="/var/lib/kubelet/pods/20831b4d-3e7e-45c3-accd-406b36c385fa/volumes" Dec 03 12:21:13 crc kubenswrapper[5052]: I1203 12:21:13.688306 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:21:13 crc kubenswrapper[5052]: I1203 12:21:13.727320 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:21:13 crc kubenswrapper[5052]: I1203 12:21:13.727752 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-server-0" event={"ID":"c14a906c-f19d-4806-9110-8c47b38c5ae0","Type":"ContainerStarted","Data":"9217efd68f2e5602eadef89a260d023bd85316f53943b906f69ea6a06b762ecd"} Dec 03 12:21:13 crc kubenswrapper[5052]: I1203 12:21:13.727943 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:21:13 crc kubenswrapper[5052]: I1203 12:21:13.732095 5052 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Dec 03 12:21:13 crc kubenswrapper[5052]: I1203 12:21:13.732944 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" event={"ID":"c279eaf1-2ee2-46fe-88be-9a5a65d4897e","Type":"ContainerStarted","Data":"9e557ebf23292119d2650d0f7e9abf49644bff9d991b5bf5077147a45b27b74b"} Dec 03 12:21:13 crc kubenswrapper[5052]: I1203 12:21:13.733546 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:21:13 crc kubenswrapper[5052]: I1203 12:21:13.733601 5052 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Dec 03 12:21:13 crc kubenswrapper[5052]: I1203 12:21:13.792314 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" podStartSLOduration=37.792287854 podStartE2EDuration="37.792287854s" podCreationTimestamp="2025-12-03 12:20:36 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:21:13.779391246 +0000 UTC m=+3664.367633010" watchObservedRunningTime="2025-12-03 12:21:13.792287854 +0000 UTC m=+3664.380529618" Dec 03 12:21:13 crc kubenswrapper[5052]: I1203 12:21:13.886964 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:21:13 crc kubenswrapper[5052]: I1203 12:21:13.930179 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/rabbitmq-server-0" podStartSLOduration=36.930159939 podStartE2EDuration="36.930159939s" podCreationTimestamp="2025-12-03 12:20:37 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:21:13.81387662 +0000 UTC m=+3664.402118394" watchObservedRunningTime="2025-12-03 12:21:13.930159939 +0000 UTC m=+3664.518401703" Dec 03 12:21:14 crc kubenswrapper[5052]: I1203 12:21:14.227245 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:21:16 crc kubenswrapper[5052]: I1203 12:21:16.435157 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/neutron-69c9545844-xf6zm" Dec 03 12:21:16 crc kubenswrapper[5052]: I1203 12:21:16.498842 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/neutron-54579c6749-xzrwm"] Dec 03 12:21:16 crc kubenswrapper[5052]: I1203 12:21:16.499119 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/neutron-54579c6749-xzrwm" podUID="d1660f09-947f-433a-973f-0cd19d97331f" containerName="neutron-api" containerID="cri-o://165f8dc84d45c1316b53403bab7a57ec38af31a5ca7fec5ae240ccfd5404eb90" gracePeriod=30 Dec 03 12:21:16 crc kubenswrapper[5052]: I1203 12:21:16.499215 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/neutron-54579c6749-xzrwm" podUID="d1660f09-947f-433a-973f-0cd19d97331f" containerName="neutron-httpd" containerID="cri-o://99d0e325b776b63e2cdf1bb10d165091f0981effadb9947661ddb77345f58080" gracePeriod=30 Dec 03 12:21:16 crc kubenswrapper[5052]: I1203 12:21:16.710847 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:21:16 crc kubenswrapper[5052]: I1203 12:21:16.767772 5052 generic.go:334] "Generic (PLEG): container finished" podID="d5750ef1-ce21-4092-9612-13d6f6c923df" containerID="58cdd95d026e2885f81c339ec294204441c3b8e523ecb43a19f7a3d140811ca5" exitCode=137 Dec 03 12:21:16 crc kubenswrapper[5052]: I1203 12:21:16.767811 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"d5750ef1-ce21-4092-9612-13d6f6c923df","Type":"ContainerDied","Data":"58cdd95d026e2885f81c339ec294204441c3b8e523ecb43a19f7a3d140811ca5"} Dec 03 12:21:16 crc kubenswrapper[5052]: I1203 12:21:16.767835 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"d5750ef1-ce21-4092-9612-13d6f6c923df","Type":"ContainerDied","Data":"f56f669cd30eb6c41c01ec084a9fe903bb582700fc2651cbab96ee3a78bb81d8"} Dec 03 12:21:16 crc kubenswrapper[5052]: I1203 12:21:16.767880 5052 scope.go:117] "RemoveContainer" containerID="58cdd95d026e2885f81c339ec294204441c3b8e523ecb43a19f7a3d140811ca5" Dec 03 12:21:16 crc kubenswrapper[5052]: I1203 12:21:16.767875 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:21:16 crc kubenswrapper[5052]: I1203 12:21:16.793778 5052 scope.go:117] "RemoveContainer" containerID="82bddf0f905668e21ca0acf3ebd30d75b07c6a6534dd49912da54363cd369335" Dec 03 12:21:16 crc kubenswrapper[5052]: I1203 12:21:16.809613 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/d5750ef1-ce21-4092-9612-13d6f6c923df-scripts\") pod \"d5750ef1-ce21-4092-9612-13d6f6c923df\" (UID: \"d5750ef1-ce21-4092-9612-13d6f6c923df\") " Dec 03 12:21:16 crc kubenswrapper[5052]: I1203 12:21:16.809702 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d5750ef1-ce21-4092-9612-13d6f6c923df-combined-ca-bundle\") pod \"d5750ef1-ce21-4092-9612-13d6f6c923df\" (UID: \"d5750ef1-ce21-4092-9612-13d6f6c923df\") " Dec 03 12:21:16 crc kubenswrapper[5052]: I1203 12:21:16.809757 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/d5750ef1-ce21-4092-9612-13d6f6c923df-etc-machine-id\") pod \"d5750ef1-ce21-4092-9612-13d6f6c923df\" (UID: \"d5750ef1-ce21-4092-9612-13d6f6c923df\") " Dec 03 12:21:16 crc kubenswrapper[5052]: I1203 12:21:16.809872 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/d5750ef1-ce21-4092-9612-13d6f6c923df-config-data\") pod \"d5750ef1-ce21-4092-9612-13d6f6c923df\" (UID: \"d5750ef1-ce21-4092-9612-13d6f6c923df\") " Dec 03 12:21:16 crc kubenswrapper[5052]: I1203 12:21:16.809937 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9h52h\" (UniqueName: \"kubernetes.io/projected/d5750ef1-ce21-4092-9612-13d6f6c923df-kube-api-access-9h52h\") pod \"d5750ef1-ce21-4092-9612-13d6f6c923df\" (UID: \"d5750ef1-ce21-4092-9612-13d6f6c923df\") " Dec 03 12:21:16 crc kubenswrapper[5052]: I1203 12:21:16.810001 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/d5750ef1-ce21-4092-9612-13d6f6c923df-config-data-custom\") pod \"d5750ef1-ce21-4092-9612-13d6f6c923df\" (UID: \"d5750ef1-ce21-4092-9612-13d6f6c923df\") " Dec 03 12:21:16 crc kubenswrapper[5052]: I1203 12:21:16.809927 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/d5750ef1-ce21-4092-9612-13d6f6c923df-etc-machine-id" (OuterVolumeSpecName: "etc-machine-id") pod "d5750ef1-ce21-4092-9612-13d6f6c923df" (UID: "d5750ef1-ce21-4092-9612-13d6f6c923df"). InnerVolumeSpecName "etc-machine-id". PluginName "kubernetes.io/host-path", VolumeGidValue "" Dec 03 12:21:16 crc kubenswrapper[5052]: I1203 12:21:16.810551 5052 reconciler_common.go:293] "Volume detached for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/d5750ef1-ce21-4092-9612-13d6f6c923df-etc-machine-id\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:16 crc kubenswrapper[5052]: I1203 12:21:16.813144 5052 scope.go:117] "RemoveContainer" containerID="66ff70190e4169f7ceded95ca82e7cd10045ed2cce8eb37b9d1a20b3e28c1110" Dec 03 12:21:16 crc kubenswrapper[5052]: I1203 12:21:16.815010 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d5750ef1-ce21-4092-9612-13d6f6c923df-kube-api-access-9h52h" (OuterVolumeSpecName: "kube-api-access-9h52h") pod "d5750ef1-ce21-4092-9612-13d6f6c923df" (UID: "d5750ef1-ce21-4092-9612-13d6f6c923df"). InnerVolumeSpecName "kube-api-access-9h52h". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:21:16 crc kubenswrapper[5052]: I1203 12:21:16.815338 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d5750ef1-ce21-4092-9612-13d6f6c923df-config-data-custom" (OuterVolumeSpecName: "config-data-custom") pod "d5750ef1-ce21-4092-9612-13d6f6c923df" (UID: "d5750ef1-ce21-4092-9612-13d6f6c923df"). InnerVolumeSpecName "config-data-custom". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:16 crc kubenswrapper[5052]: I1203 12:21:16.815973 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d5750ef1-ce21-4092-9612-13d6f6c923df-scripts" (OuterVolumeSpecName: "scripts") pod "d5750ef1-ce21-4092-9612-13d6f6c923df" (UID: "d5750ef1-ce21-4092-9612-13d6f6c923df"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:16 crc kubenswrapper[5052]: I1203 12:21:16.891820 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d5750ef1-ce21-4092-9612-13d6f6c923df-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "d5750ef1-ce21-4092-9612-13d6f6c923df" (UID: "d5750ef1-ce21-4092-9612-13d6f6c923df"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:16 crc kubenswrapper[5052]: I1203 12:21:16.913781 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d5750ef1-ce21-4092-9612-13d6f6c923df-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:16 crc kubenswrapper[5052]: I1203 12:21:16.913825 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-9h52h\" (UniqueName: \"kubernetes.io/projected/d5750ef1-ce21-4092-9612-13d6f6c923df-kube-api-access-9h52h\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:16 crc kubenswrapper[5052]: I1203 12:21:16.913840 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/d5750ef1-ce21-4092-9612-13d6f6c923df-config-data-custom\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:16 crc kubenswrapper[5052]: I1203 12:21:16.913854 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/d5750ef1-ce21-4092-9612-13d6f6c923df-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:16 crc kubenswrapper[5052]: I1203 12:21:16.921305 5052 scope.go:117] "RemoveContainer" containerID="58cdd95d026e2885f81c339ec294204441c3b8e523ecb43a19f7a3d140811ca5" Dec 03 12:21:16 crc kubenswrapper[5052]: E1203 12:21:16.921791 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"58cdd95d026e2885f81c339ec294204441c3b8e523ecb43a19f7a3d140811ca5\": container with ID starting with 58cdd95d026e2885f81c339ec294204441c3b8e523ecb43a19f7a3d140811ca5 not found: ID does not exist" containerID="58cdd95d026e2885f81c339ec294204441c3b8e523ecb43a19f7a3d140811ca5" Dec 03 12:21:16 crc kubenswrapper[5052]: I1203 12:21:16.921820 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"58cdd95d026e2885f81c339ec294204441c3b8e523ecb43a19f7a3d140811ca5"} err="failed to get container status \"58cdd95d026e2885f81c339ec294204441c3b8e523ecb43a19f7a3d140811ca5\": rpc error: code = NotFound desc = could not find container \"58cdd95d026e2885f81c339ec294204441c3b8e523ecb43a19f7a3d140811ca5\": container with ID starting with 58cdd95d026e2885f81c339ec294204441c3b8e523ecb43a19f7a3d140811ca5 not found: ID does not exist" Dec 03 12:21:16 crc kubenswrapper[5052]: I1203 12:21:16.921839 5052 scope.go:117] "RemoveContainer" containerID="82bddf0f905668e21ca0acf3ebd30d75b07c6a6534dd49912da54363cd369335" Dec 03 12:21:16 crc kubenswrapper[5052]: E1203 12:21:16.922358 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"82bddf0f905668e21ca0acf3ebd30d75b07c6a6534dd49912da54363cd369335\": container with ID starting with 82bddf0f905668e21ca0acf3ebd30d75b07c6a6534dd49912da54363cd369335 not found: ID does not exist" containerID="82bddf0f905668e21ca0acf3ebd30d75b07c6a6534dd49912da54363cd369335" Dec 03 12:21:16 crc kubenswrapper[5052]: I1203 12:21:16.922380 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"82bddf0f905668e21ca0acf3ebd30d75b07c6a6534dd49912da54363cd369335"} err="failed to get container status \"82bddf0f905668e21ca0acf3ebd30d75b07c6a6534dd49912da54363cd369335\": rpc error: code = NotFound desc = could not find container \"82bddf0f905668e21ca0acf3ebd30d75b07c6a6534dd49912da54363cd369335\": container with ID starting with 82bddf0f905668e21ca0acf3ebd30d75b07c6a6534dd49912da54363cd369335 not found: ID does not exist" Dec 03 12:21:16 crc kubenswrapper[5052]: I1203 12:21:16.922393 5052 scope.go:117] "RemoveContainer" containerID="66ff70190e4169f7ceded95ca82e7cd10045ed2cce8eb37b9d1a20b3e28c1110" Dec 03 12:21:16 crc kubenswrapper[5052]: E1203 12:21:16.922882 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"66ff70190e4169f7ceded95ca82e7cd10045ed2cce8eb37b9d1a20b3e28c1110\": container with ID starting with 66ff70190e4169f7ceded95ca82e7cd10045ed2cce8eb37b9d1a20b3e28c1110 not found: ID does not exist" containerID="66ff70190e4169f7ceded95ca82e7cd10045ed2cce8eb37b9d1a20b3e28c1110" Dec 03 12:21:16 crc kubenswrapper[5052]: I1203 12:21:16.922928 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"66ff70190e4169f7ceded95ca82e7cd10045ed2cce8eb37b9d1a20b3e28c1110"} err="failed to get container status \"66ff70190e4169f7ceded95ca82e7cd10045ed2cce8eb37b9d1a20b3e28c1110\": rpc error: code = NotFound desc = could not find container \"66ff70190e4169f7ceded95ca82e7cd10045ed2cce8eb37b9d1a20b3e28c1110\": container with ID starting with 66ff70190e4169f7ceded95ca82e7cd10045ed2cce8eb37b9d1a20b3e28c1110 not found: ID does not exist" Dec 03 12:21:16 crc kubenswrapper[5052]: I1203 12:21:16.924321 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d5750ef1-ce21-4092-9612-13d6f6c923df-config-data" (OuterVolumeSpecName: "config-data") pod "d5750ef1-ce21-4092-9612-13d6f6c923df" (UID: "d5750ef1-ce21-4092-9612-13d6f6c923df"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:17 crc kubenswrapper[5052]: I1203 12:21:17.015518 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/d5750ef1-ce21-4092-9612-13d6f6c923df-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:17 crc kubenswrapper[5052]: I1203 12:21:17.119782 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 12:21:17 crc kubenswrapper[5052]: I1203 12:21:17.148337 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 12:21:17 crc kubenswrapper[5052]: I1203 12:21:17.164995 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 12:21:17 crc kubenswrapper[5052]: E1203 12:21:17.165644 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="d5750ef1-ce21-4092-9612-13d6f6c923df" containerName="probe" Dec 03 12:21:17 crc kubenswrapper[5052]: I1203 12:21:17.165667 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="d5750ef1-ce21-4092-9612-13d6f6c923df" containerName="probe" Dec 03 12:21:17 crc kubenswrapper[5052]: E1203 12:21:17.165681 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="20831b4d-3e7e-45c3-accd-406b36c385fa" containerName="init" Dec 03 12:21:17 crc kubenswrapper[5052]: I1203 12:21:17.165687 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="20831b4d-3e7e-45c3-accd-406b36c385fa" containerName="init" Dec 03 12:21:17 crc kubenswrapper[5052]: E1203 12:21:17.165699 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="d5750ef1-ce21-4092-9612-13d6f6c923df" containerName="cinder-scheduler" Dec 03 12:21:17 crc kubenswrapper[5052]: I1203 12:21:17.165705 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="d5750ef1-ce21-4092-9612-13d6f6c923df" containerName="cinder-scheduler" Dec 03 12:21:17 crc kubenswrapper[5052]: E1203 12:21:17.165721 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="311e25fa-0785-45ce-ae3c-00d32adbf3f4" containerName="barbican-worker" Dec 03 12:21:17 crc kubenswrapper[5052]: I1203 12:21:17.165728 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="311e25fa-0785-45ce-ae3c-00d32adbf3f4" containerName="barbican-worker" Dec 03 12:21:17 crc kubenswrapper[5052]: E1203 12:21:17.165740 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="7076e341-78ad-4783-8a0e-2f3914f92a5b" containerName="barbican-api" Dec 03 12:21:17 crc kubenswrapper[5052]: I1203 12:21:17.165746 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="7076e341-78ad-4783-8a0e-2f3914f92a5b" containerName="barbican-api" Dec 03 12:21:17 crc kubenswrapper[5052]: E1203 12:21:17.165756 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="20831b4d-3e7e-45c3-accd-406b36c385fa" containerName="dnsmasq-dns" Dec 03 12:21:17 crc kubenswrapper[5052]: I1203 12:21:17.165761 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="20831b4d-3e7e-45c3-accd-406b36c385fa" containerName="dnsmasq-dns" Dec 03 12:21:17 crc kubenswrapper[5052]: E1203 12:21:17.165772 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="7076e341-78ad-4783-8a0e-2f3914f92a5b" containerName="barbican-api-log" Dec 03 12:21:17 crc kubenswrapper[5052]: I1203 12:21:17.165778 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="7076e341-78ad-4783-8a0e-2f3914f92a5b" containerName="barbican-api-log" Dec 03 12:21:17 crc kubenswrapper[5052]: E1203 12:21:17.165792 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="311e25fa-0785-45ce-ae3c-00d32adbf3f4" containerName="barbican-worker-log" Dec 03 12:21:17 crc kubenswrapper[5052]: I1203 12:21:17.165798 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="311e25fa-0785-45ce-ae3c-00d32adbf3f4" containerName="barbican-worker-log" Dec 03 12:21:17 crc kubenswrapper[5052]: I1203 12:21:17.165990 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="d5750ef1-ce21-4092-9612-13d6f6c923df" containerName="cinder-scheduler" Dec 03 12:21:17 crc kubenswrapper[5052]: I1203 12:21:17.166006 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="d5750ef1-ce21-4092-9612-13d6f6c923df" containerName="probe" Dec 03 12:21:17 crc kubenswrapper[5052]: I1203 12:21:17.166022 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="7076e341-78ad-4783-8a0e-2f3914f92a5b" containerName="barbican-api-log" Dec 03 12:21:17 crc kubenswrapper[5052]: I1203 12:21:17.166036 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="20831b4d-3e7e-45c3-accd-406b36c385fa" containerName="dnsmasq-dns" Dec 03 12:21:17 crc kubenswrapper[5052]: I1203 12:21:17.166046 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="311e25fa-0785-45ce-ae3c-00d32adbf3f4" containerName="barbican-worker" Dec 03 12:21:17 crc kubenswrapper[5052]: I1203 12:21:17.166058 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="311e25fa-0785-45ce-ae3c-00d32adbf3f4" containerName="barbican-worker-log" Dec 03 12:21:17 crc kubenswrapper[5052]: I1203 12:21:17.166067 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="7076e341-78ad-4783-8a0e-2f3914f92a5b" containerName="barbican-api" Dec 03 12:21:17 crc kubenswrapper[5052]: E1203 12:21:17.166285 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="d5750ef1-ce21-4092-9612-13d6f6c923df" containerName="cinder-scheduler" Dec 03 12:21:17 crc kubenswrapper[5052]: I1203 12:21:17.166294 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="d5750ef1-ce21-4092-9612-13d6f6c923df" containerName="cinder-scheduler" Dec 03 12:21:17 crc kubenswrapper[5052]: I1203 12:21:17.166491 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="d5750ef1-ce21-4092-9612-13d6f6c923df" containerName="cinder-scheduler" Dec 03 12:21:17 crc kubenswrapper[5052]: I1203 12:21:17.167324 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:21:17 crc kubenswrapper[5052]: I1203 12:21:17.169433 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cinder-scheduler-config-data" Dec 03 12:21:17 crc kubenswrapper[5052]: I1203 12:21:17.176293 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 12:21:17 crc kubenswrapper[5052]: I1203 12:21:17.325196 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4a55b492-d57f-40dc-b719-1294fb24ef13-combined-ca-bundle\") pod \"cinder-scheduler-0\" (UID: \"4a55b492-d57f-40dc-b719-1294fb24ef13\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:21:17 crc kubenswrapper[5052]: I1203 12:21:17.325272 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z4ktp\" (UniqueName: \"kubernetes.io/projected/4a55b492-d57f-40dc-b719-1294fb24ef13-kube-api-access-z4ktp\") pod \"cinder-scheduler-0\" (UID: \"4a55b492-d57f-40dc-b719-1294fb24ef13\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:21:17 crc kubenswrapper[5052]: I1203 12:21:17.325360 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/4a55b492-d57f-40dc-b719-1294fb24ef13-scripts\") pod \"cinder-scheduler-0\" (UID: \"4a55b492-d57f-40dc-b719-1294fb24ef13\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:21:17 crc kubenswrapper[5052]: I1203 12:21:17.325393 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/4a55b492-d57f-40dc-b719-1294fb24ef13-config-data\") pod \"cinder-scheduler-0\" (UID: \"4a55b492-d57f-40dc-b719-1294fb24ef13\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:21:17 crc kubenswrapper[5052]: I1203 12:21:17.325490 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/4a55b492-d57f-40dc-b719-1294fb24ef13-config-data-custom\") pod \"cinder-scheduler-0\" (UID: \"4a55b492-d57f-40dc-b719-1294fb24ef13\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:21:17 crc kubenswrapper[5052]: I1203 12:21:17.325668 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/4a55b492-d57f-40dc-b719-1294fb24ef13-etc-machine-id\") pod \"cinder-scheduler-0\" (UID: \"4a55b492-d57f-40dc-b719-1294fb24ef13\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:21:17 crc kubenswrapper[5052]: I1203 12:21:17.427961 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/4a55b492-d57f-40dc-b719-1294fb24ef13-etc-machine-id\") pod \"cinder-scheduler-0\" (UID: \"4a55b492-d57f-40dc-b719-1294fb24ef13\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:21:17 crc kubenswrapper[5052]: I1203 12:21:17.428032 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4a55b492-d57f-40dc-b719-1294fb24ef13-combined-ca-bundle\") pod \"cinder-scheduler-0\" (UID: \"4a55b492-d57f-40dc-b719-1294fb24ef13\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:21:17 crc kubenswrapper[5052]: I1203 12:21:17.428052 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-z4ktp\" (UniqueName: \"kubernetes.io/projected/4a55b492-d57f-40dc-b719-1294fb24ef13-kube-api-access-z4ktp\") pod \"cinder-scheduler-0\" (UID: \"4a55b492-d57f-40dc-b719-1294fb24ef13\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:21:17 crc kubenswrapper[5052]: I1203 12:21:17.428084 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/4a55b492-d57f-40dc-b719-1294fb24ef13-scripts\") pod \"cinder-scheduler-0\" (UID: \"4a55b492-d57f-40dc-b719-1294fb24ef13\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:21:17 crc kubenswrapper[5052]: I1203 12:21:17.428108 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/4a55b492-d57f-40dc-b719-1294fb24ef13-config-data\") pod \"cinder-scheduler-0\" (UID: \"4a55b492-d57f-40dc-b719-1294fb24ef13\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:21:17 crc kubenswrapper[5052]: I1203 12:21:17.428145 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/4a55b492-d57f-40dc-b719-1294fb24ef13-config-data-custom\") pod \"cinder-scheduler-0\" (UID: \"4a55b492-d57f-40dc-b719-1294fb24ef13\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:21:17 crc kubenswrapper[5052]: I1203 12:21:17.428671 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/4a55b492-d57f-40dc-b719-1294fb24ef13-etc-machine-id\") pod \"cinder-scheduler-0\" (UID: \"4a55b492-d57f-40dc-b719-1294fb24ef13\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:21:17 crc kubenswrapper[5052]: I1203 12:21:17.433591 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4a55b492-d57f-40dc-b719-1294fb24ef13-combined-ca-bundle\") pod \"cinder-scheduler-0\" (UID: \"4a55b492-d57f-40dc-b719-1294fb24ef13\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:21:17 crc kubenswrapper[5052]: I1203 12:21:17.434013 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/4a55b492-d57f-40dc-b719-1294fb24ef13-scripts\") pod \"cinder-scheduler-0\" (UID: \"4a55b492-d57f-40dc-b719-1294fb24ef13\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:21:17 crc kubenswrapper[5052]: I1203 12:21:17.435657 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/4a55b492-d57f-40dc-b719-1294fb24ef13-config-data-custom\") pod \"cinder-scheduler-0\" (UID: \"4a55b492-d57f-40dc-b719-1294fb24ef13\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:21:17 crc kubenswrapper[5052]: I1203 12:21:17.438088 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/4a55b492-d57f-40dc-b719-1294fb24ef13-config-data\") pod \"cinder-scheduler-0\" (UID: \"4a55b492-d57f-40dc-b719-1294fb24ef13\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:21:17 crc kubenswrapper[5052]: I1203 12:21:17.446957 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-z4ktp\" (UniqueName: \"kubernetes.io/projected/4a55b492-d57f-40dc-b719-1294fb24ef13-kube-api-access-z4ktp\") pod \"cinder-scheduler-0\" (UID: \"4a55b492-d57f-40dc-b719-1294fb24ef13\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:21:17 crc kubenswrapper[5052]: I1203 12:21:17.532718 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:21:17 crc kubenswrapper[5052]: I1203 12:21:17.686903 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d5750ef1-ce21-4092-9612-13d6f6c923df" path="/var/lib/kubelet/pods/d5750ef1-ce21-4092-9612-13d6f6c923df/volumes" Dec 03 12:21:17 crc kubenswrapper[5052]: I1203 12:21:17.793678 5052 generic.go:334] "Generic (PLEG): container finished" podID="d1660f09-947f-433a-973f-0cd19d97331f" containerID="99d0e325b776b63e2cdf1bb10d165091f0981effadb9947661ddb77345f58080" exitCode=0 Dec 03 12:21:17 crc kubenswrapper[5052]: I1203 12:21:17.793741 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-54579c6749-xzrwm" event={"ID":"d1660f09-947f-433a-973f-0cd19d97331f","Type":"ContainerDied","Data":"99d0e325b776b63e2cdf1bb10d165091f0981effadb9947661ddb77345f58080"} Dec 03 12:21:18 crc kubenswrapper[5052]: I1203 12:21:18.026441 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 12:21:18 crc kubenswrapper[5052]: I1203 12:21:18.307832 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:21:18 crc kubenswrapper[5052]: I1203 12:21:18.805481 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"4a55b492-d57f-40dc-b719-1294fb24ef13","Type":"ContainerStarted","Data":"84022ca2f2789fb89a8aa147aadbca38260a93e0dfa5f25cefce8f679ca30454"} Dec 03 12:21:18 crc kubenswrapper[5052]: I1203 12:21:18.805534 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"4a55b492-d57f-40dc-b719-1294fb24ef13","Type":"ContainerStarted","Data":"ec5ded814017ddd24b3aff8929f5844220d80b4ac8f9878cfd10a94a254e0ca6"} Dec 03 12:21:19 crc kubenswrapper[5052]: I1203 12:21:19.819172 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"4a55b492-d57f-40dc-b719-1294fb24ef13","Type":"ContainerStarted","Data":"4fdd95681555772f2043566122b6cc7d5d50f1b3bc46512f2b2e8fedf1c6d999"} Dec 03 12:21:19 crc kubenswrapper[5052]: I1203 12:21:19.839424 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/cinder-scheduler-0" podStartSLOduration=2.839402531 podStartE2EDuration="2.839402531s" podCreationTimestamp="2025-12-03 12:21:17 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:21:19.838393402 +0000 UTC m=+3670.426635166" watchObservedRunningTime="2025-12-03 12:21:19.839402531 +0000 UTC m=+3670.427644295" Dec 03 12:21:22 crc kubenswrapper[5052]: I1203 12:21:22.533000 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:21:22 crc kubenswrapper[5052]: I1203 12:21:22.569316 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/placement-5447868864-mfdcg" Dec 03 12:21:22 crc kubenswrapper[5052]: I1203 12:21:22.595649 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/placement-5447868864-mfdcg" Dec 03 12:21:22 crc kubenswrapper[5052]: I1203 12:21:22.663461 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/placement-7d6bd9d7c9-n2lr6"] Dec 03 12:21:22 crc kubenswrapper[5052]: I1203 12:21:22.663745 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/placement-7d6bd9d7c9-n2lr6" podUID="803e4fd4-6dec-4eca-b15b-10391318f545" containerName="placement-log" containerID="cri-o://4b811fae861808e9515fee8304fd3b59a0c3b044a28ee334526f62da3c3a9d74" gracePeriod=30 Dec 03 12:21:22 crc kubenswrapper[5052]: I1203 12:21:22.663856 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/placement-7d6bd9d7c9-n2lr6" podUID="803e4fd4-6dec-4eca-b15b-10391318f545" containerName="placement-api" containerID="cri-o://62a6906785e64de4b321a53b5b04ed62907021345e45c53972baa6f045c1f746" gracePeriod=30 Dec 03 12:21:22 crc kubenswrapper[5052]: I1203 12:21:22.857880 5052 generic.go:334] "Generic (PLEG): container finished" podID="803e4fd4-6dec-4eca-b15b-10391318f545" containerID="4b811fae861808e9515fee8304fd3b59a0c3b044a28ee334526f62da3c3a9d74" exitCode=143 Dec 03 12:21:22 crc kubenswrapper[5052]: I1203 12:21:22.858668 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-7d6bd9d7c9-n2lr6" event={"ID":"803e4fd4-6dec-4eca-b15b-10391318f545","Type":"ContainerDied","Data":"4b811fae861808e9515fee8304fd3b59a0c3b044a28ee334526f62da3c3a9d74"} Dec 03 12:21:23 crc kubenswrapper[5052]: I1203 12:21:23.672709 5052 scope.go:117] "RemoveContainer" containerID="09b4a0072741f60499d24f40cc4ff3406d5d22671ced431363ab865172786300" Dec 03 12:21:23 crc kubenswrapper[5052]: E1203 12:21:23.672940 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 12:21:25 crc kubenswrapper[5052]: I1203 12:21:25.916486 5052 generic.go:334] "Generic (PLEG): container finished" podID="803e4fd4-6dec-4eca-b15b-10391318f545" containerID="62a6906785e64de4b321a53b5b04ed62907021345e45c53972baa6f045c1f746" exitCode=0 Dec 03 12:21:25 crc kubenswrapper[5052]: I1203 12:21:25.916551 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-7d6bd9d7c9-n2lr6" event={"ID":"803e4fd4-6dec-4eca-b15b-10391318f545","Type":"ContainerDied","Data":"62a6906785e64de4b321a53b5b04ed62907021345e45c53972baa6f045c1f746"} Dec 03 12:21:26 crc kubenswrapper[5052]: I1203 12:21:26.256219 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-7d6bd9d7c9-n2lr6" Dec 03 12:21:26 crc kubenswrapper[5052]: I1203 12:21:26.351184 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/803e4fd4-6dec-4eca-b15b-10391318f545-logs\") pod \"803e4fd4-6dec-4eca-b15b-10391318f545\" (UID: \"803e4fd4-6dec-4eca-b15b-10391318f545\") " Dec 03 12:21:26 crc kubenswrapper[5052]: I1203 12:21:26.351283 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/803e4fd4-6dec-4eca-b15b-10391318f545-internal-tls-certs\") pod \"803e4fd4-6dec-4eca-b15b-10391318f545\" (UID: \"803e4fd4-6dec-4eca-b15b-10391318f545\") " Dec 03 12:21:26 crc kubenswrapper[5052]: I1203 12:21:26.351343 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/803e4fd4-6dec-4eca-b15b-10391318f545-config-data\") pod \"803e4fd4-6dec-4eca-b15b-10391318f545\" (UID: \"803e4fd4-6dec-4eca-b15b-10391318f545\") " Dec 03 12:21:26 crc kubenswrapper[5052]: I1203 12:21:26.351402 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/803e4fd4-6dec-4eca-b15b-10391318f545-public-tls-certs\") pod \"803e4fd4-6dec-4eca-b15b-10391318f545\" (UID: \"803e4fd4-6dec-4eca-b15b-10391318f545\") " Dec 03 12:21:26 crc kubenswrapper[5052]: I1203 12:21:26.351429 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jqc74\" (UniqueName: \"kubernetes.io/projected/803e4fd4-6dec-4eca-b15b-10391318f545-kube-api-access-jqc74\") pod \"803e4fd4-6dec-4eca-b15b-10391318f545\" (UID: \"803e4fd4-6dec-4eca-b15b-10391318f545\") " Dec 03 12:21:26 crc kubenswrapper[5052]: I1203 12:21:26.351455 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/803e4fd4-6dec-4eca-b15b-10391318f545-scripts\") pod \"803e4fd4-6dec-4eca-b15b-10391318f545\" (UID: \"803e4fd4-6dec-4eca-b15b-10391318f545\") " Dec 03 12:21:26 crc kubenswrapper[5052]: I1203 12:21:26.351527 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/803e4fd4-6dec-4eca-b15b-10391318f545-combined-ca-bundle\") pod \"803e4fd4-6dec-4eca-b15b-10391318f545\" (UID: \"803e4fd4-6dec-4eca-b15b-10391318f545\") " Dec 03 12:21:26 crc kubenswrapper[5052]: I1203 12:21:26.351965 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/803e4fd4-6dec-4eca-b15b-10391318f545-logs" (OuterVolumeSpecName: "logs") pod "803e4fd4-6dec-4eca-b15b-10391318f545" (UID: "803e4fd4-6dec-4eca-b15b-10391318f545"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:21:26 crc kubenswrapper[5052]: I1203 12:21:26.358283 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/803e4fd4-6dec-4eca-b15b-10391318f545-kube-api-access-jqc74" (OuterVolumeSpecName: "kube-api-access-jqc74") pod "803e4fd4-6dec-4eca-b15b-10391318f545" (UID: "803e4fd4-6dec-4eca-b15b-10391318f545"). InnerVolumeSpecName "kube-api-access-jqc74". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:21:26 crc kubenswrapper[5052]: I1203 12:21:26.360497 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/803e4fd4-6dec-4eca-b15b-10391318f545-scripts" (OuterVolumeSpecName: "scripts") pod "803e4fd4-6dec-4eca-b15b-10391318f545" (UID: "803e4fd4-6dec-4eca-b15b-10391318f545"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:26 crc kubenswrapper[5052]: I1203 12:21:26.412459 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/803e4fd4-6dec-4eca-b15b-10391318f545-config-data" (OuterVolumeSpecName: "config-data") pod "803e4fd4-6dec-4eca-b15b-10391318f545" (UID: "803e4fd4-6dec-4eca-b15b-10391318f545"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:26 crc kubenswrapper[5052]: I1203 12:21:26.414014 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/803e4fd4-6dec-4eca-b15b-10391318f545-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "803e4fd4-6dec-4eca-b15b-10391318f545" (UID: "803e4fd4-6dec-4eca-b15b-10391318f545"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:26 crc kubenswrapper[5052]: I1203 12:21:26.453780 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/803e4fd4-6dec-4eca-b15b-10391318f545-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:26 crc kubenswrapper[5052]: I1203 12:21:26.453826 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/803e4fd4-6dec-4eca-b15b-10391318f545-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:26 crc kubenswrapper[5052]: I1203 12:21:26.453841 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-jqc74\" (UniqueName: \"kubernetes.io/projected/803e4fd4-6dec-4eca-b15b-10391318f545-kube-api-access-jqc74\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:26 crc kubenswrapper[5052]: I1203 12:21:26.453851 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/803e4fd4-6dec-4eca-b15b-10391318f545-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:26 crc kubenswrapper[5052]: I1203 12:21:26.453859 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/803e4fd4-6dec-4eca-b15b-10391318f545-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:26 crc kubenswrapper[5052]: I1203 12:21:26.457874 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/803e4fd4-6dec-4eca-b15b-10391318f545-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "803e4fd4-6dec-4eca-b15b-10391318f545" (UID: "803e4fd4-6dec-4eca-b15b-10391318f545"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:26 crc kubenswrapper[5052]: I1203 12:21:26.461097 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/803e4fd4-6dec-4eca-b15b-10391318f545-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "803e4fd4-6dec-4eca-b15b-10391318f545" (UID: "803e4fd4-6dec-4eca-b15b-10391318f545"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:26 crc kubenswrapper[5052]: I1203 12:21:26.555812 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/803e4fd4-6dec-4eca-b15b-10391318f545-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:26 crc kubenswrapper[5052]: I1203 12:21:26.555851 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/803e4fd4-6dec-4eca-b15b-10391318f545-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:26 crc kubenswrapper[5052]: I1203 12:21:26.933321 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-7d6bd9d7c9-n2lr6" event={"ID":"803e4fd4-6dec-4eca-b15b-10391318f545","Type":"ContainerDied","Data":"fa6ca5da9d1fcb97c4f48bcd0336274c8f5c21c365e96c8637b319169134d35e"} Dec 03 12:21:26 crc kubenswrapper[5052]: I1203 12:21:26.933386 5052 scope.go:117] "RemoveContainer" containerID="62a6906785e64de4b321a53b5b04ed62907021345e45c53972baa6f045c1f746" Dec 03 12:21:26 crc kubenswrapper[5052]: I1203 12:21:26.933539 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-7d6bd9d7c9-n2lr6" Dec 03 12:21:26 crc kubenswrapper[5052]: I1203 12:21:26.967923 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/placement-7d6bd9d7c9-n2lr6"] Dec 03 12:21:26 crc kubenswrapper[5052]: I1203 12:21:26.968331 5052 scope.go:117] "RemoveContainer" containerID="4b811fae861808e9515fee8304fd3b59a0c3b044a28ee334526f62da3c3a9d74" Dec 03 12:21:26 crc kubenswrapper[5052]: I1203 12:21:26.978909 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/placement-7d6bd9d7c9-n2lr6"] Dec 03 12:21:27 crc kubenswrapper[5052]: I1203 12:21:27.337756 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:21:27 crc kubenswrapper[5052]: I1203 12:21:27.686145 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="803e4fd4-6dec-4eca-b15b-10391318f545" path="/var/lib/kubelet/pods/803e4fd4-6dec-4eca-b15b-10391318f545/volumes" Dec 03 12:21:27 crc kubenswrapper[5052]: I1203 12:21:27.747426 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:21:28 crc kubenswrapper[5052]: I1203 12:21:28.019763 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:21:28 crc kubenswrapper[5052]: I1203 12:21:28.294528 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell1-cell-mapping-86wh2"] Dec 03 12:21:28 crc kubenswrapper[5052]: I1203 12:21:28.315983 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell0-cell-mapping-pn79k"] Dec 03 12:21:28 crc kubenswrapper[5052]: I1203 12:21:28.329104 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell1-cell-mapping-86wh2"] Dec 03 12:21:28 crc kubenswrapper[5052]: I1203 12:21:28.339416 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell0-cell-mapping-pn79k"] Dec 03 12:21:28 crc kubenswrapper[5052]: I1203 12:21:28.349203 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:21:28 crc kubenswrapper[5052]: I1203 12:21:28.349473 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-api-0" podUID="ce16d54b-ef04-429b-8795-2865395f7774" containerName="nova-api-log" containerID="cri-o://8531d325ffe575ccabb11f725af2c0fca6093a5c058282b15cfeabec695cc2f0" gracePeriod=30 Dec 03 12:21:28 crc kubenswrapper[5052]: I1203 12:21:28.349658 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-api-0" podUID="ce16d54b-ef04-429b-8795-2865395f7774" containerName="nova-api-api" containerID="cri-o://0a4ad14776400bd11635febb15632b857ffe75a1ed4b078b2eb0c7621d98c834" gracePeriod=30 Dec 03 12:21:28 crc kubenswrapper[5052]: I1203 12:21:28.375532 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:21:28 crc kubenswrapper[5052]: I1203 12:21:28.375831 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-scheduler-0" podUID="9d3ab728-8f1c-48ac-808c-dfbf80b8f792" containerName="nova-scheduler-scheduler" containerID="cri-o://20a9921d6ffcb85977689eb24ca64a251931e0e59f572a888d9ee16492bf0b00" gracePeriod=30 Dec 03 12:21:28 crc kubenswrapper[5052]: I1203 12:21:28.410681 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:21:28 crc kubenswrapper[5052]: I1203 12:21:28.410906 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-metadata-0" podUID="5bda226d-9d40-42d8-ac3e-11719e896649" containerName="nova-metadata-log" containerID="cri-o://9818701fe2f0f4ff6dc2b33cbd5c557f191bfe07f31eb908a06a4fcecfe3dcd0" gracePeriod=30 Dec 03 12:21:28 crc kubenswrapper[5052]: I1203 12:21:28.411181 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-metadata-0" podUID="5bda226d-9d40-42d8-ac3e-11719e896649" containerName="nova-metadata-metadata" containerID="cri-o://67d4dbc28757901384966bfa33590ec06891feae45053aaf9e8e283f4b17ad15" gracePeriod=30 Dec 03 12:21:28 crc kubenswrapper[5052]: I1203 12:21:28.490271 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell0-cell-mapping-6dkwd"] Dec 03 12:21:28 crc kubenswrapper[5052]: E1203 12:21:28.491017 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="803e4fd4-6dec-4eca-b15b-10391318f545" containerName="placement-log" Dec 03 12:21:28 crc kubenswrapper[5052]: I1203 12:21:28.491124 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="803e4fd4-6dec-4eca-b15b-10391318f545" containerName="placement-log" Dec 03 12:21:28 crc kubenswrapper[5052]: E1203 12:21:28.491226 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="803e4fd4-6dec-4eca-b15b-10391318f545" containerName="placement-api" Dec 03 12:21:28 crc kubenswrapper[5052]: I1203 12:21:28.491300 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="803e4fd4-6dec-4eca-b15b-10391318f545" containerName="placement-api" Dec 03 12:21:28 crc kubenswrapper[5052]: I1203 12:21:28.491640 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="803e4fd4-6dec-4eca-b15b-10391318f545" containerName="placement-log" Dec 03 12:21:28 crc kubenswrapper[5052]: I1203 12:21:28.491726 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="803e4fd4-6dec-4eca-b15b-10391318f545" containerName="placement-api" Dec 03 12:21:28 crc kubenswrapper[5052]: I1203 12:21:28.492627 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-cell-mapping-6dkwd" Dec 03 12:21:28 crc kubenswrapper[5052]: I1203 12:21:28.495619 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell0-manage-scripts" Dec 03 12:21:28 crc kubenswrapper[5052]: I1203 12:21:28.495855 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell0-manage-config-data" Dec 03 12:21:28 crc kubenswrapper[5052]: I1203 12:21:28.511962 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell0-cell-mapping-6dkwd"] Dec 03 12:21:28 crc kubenswrapper[5052]: I1203 12:21:28.560697 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell1-cell-mapping-qr65w"] Dec 03 12:21:28 crc kubenswrapper[5052]: I1203 12:21:28.562580 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-cell-mapping-qr65w" Dec 03 12:21:28 crc kubenswrapper[5052]: I1203 12:21:28.565201 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell1-manage-scripts" Dec 03 12:21:28 crc kubenswrapper[5052]: I1203 12:21:28.565488 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell1-manage-config-data" Dec 03 12:21:28 crc kubenswrapper[5052]: I1203 12:21:28.605633 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rr4br\" (UniqueName: \"kubernetes.io/projected/27d134d0-a0b9-457b-8c8c-92cfb104cba1-kube-api-access-rr4br\") pod \"nova-cell1-cell-mapping-qr65w\" (UID: \"27d134d0-a0b9-457b-8c8c-92cfb104cba1\") " pod="openstack-kuttl-tests/nova-cell1-cell-mapping-qr65w" Dec 03 12:21:28 crc kubenswrapper[5052]: I1203 12:21:28.606001 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/27d134d0-a0b9-457b-8c8c-92cfb104cba1-combined-ca-bundle\") pod \"nova-cell1-cell-mapping-qr65w\" (UID: \"27d134d0-a0b9-457b-8c8c-92cfb104cba1\") " pod="openstack-kuttl-tests/nova-cell1-cell-mapping-qr65w" Dec 03 12:21:28 crc kubenswrapper[5052]: I1203 12:21:28.606048 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/0bbf844e-3419-4f88-b375-ca77df3007af-config-data\") pod \"nova-cell0-cell-mapping-6dkwd\" (UID: \"0bbf844e-3419-4f88-b375-ca77df3007af\") " pod="openstack-kuttl-tests/nova-cell0-cell-mapping-6dkwd" Dec 03 12:21:28 crc kubenswrapper[5052]: I1203 12:21:28.606087 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xcscw\" (UniqueName: \"kubernetes.io/projected/0bbf844e-3419-4f88-b375-ca77df3007af-kube-api-access-xcscw\") pod \"nova-cell0-cell-mapping-6dkwd\" (UID: \"0bbf844e-3419-4f88-b375-ca77df3007af\") " pod="openstack-kuttl-tests/nova-cell0-cell-mapping-6dkwd" Dec 03 12:21:28 crc kubenswrapper[5052]: I1203 12:21:28.606161 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/27d134d0-a0b9-457b-8c8c-92cfb104cba1-scripts\") pod \"nova-cell1-cell-mapping-qr65w\" (UID: \"27d134d0-a0b9-457b-8c8c-92cfb104cba1\") " pod="openstack-kuttl-tests/nova-cell1-cell-mapping-qr65w" Dec 03 12:21:28 crc kubenswrapper[5052]: I1203 12:21:28.606244 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/27d134d0-a0b9-457b-8c8c-92cfb104cba1-config-data\") pod \"nova-cell1-cell-mapping-qr65w\" (UID: \"27d134d0-a0b9-457b-8c8c-92cfb104cba1\") " pod="openstack-kuttl-tests/nova-cell1-cell-mapping-qr65w" Dec 03 12:21:28 crc kubenswrapper[5052]: I1203 12:21:28.606279 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/0bbf844e-3419-4f88-b375-ca77df3007af-scripts\") pod \"nova-cell0-cell-mapping-6dkwd\" (UID: \"0bbf844e-3419-4f88-b375-ca77df3007af\") " pod="openstack-kuttl-tests/nova-cell0-cell-mapping-6dkwd" Dec 03 12:21:28 crc kubenswrapper[5052]: I1203 12:21:28.606363 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0bbf844e-3419-4f88-b375-ca77df3007af-combined-ca-bundle\") pod \"nova-cell0-cell-mapping-6dkwd\" (UID: \"0bbf844e-3419-4f88-b375-ca77df3007af\") " pod="openstack-kuttl-tests/nova-cell0-cell-mapping-6dkwd" Dec 03 12:21:28 crc kubenswrapper[5052]: I1203 12:21:28.606363 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-cell-mapping-qr65w"] Dec 03 12:21:28 crc kubenswrapper[5052]: I1203 12:21:28.708572 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/27d134d0-a0b9-457b-8c8c-92cfb104cba1-config-data\") pod \"nova-cell1-cell-mapping-qr65w\" (UID: \"27d134d0-a0b9-457b-8c8c-92cfb104cba1\") " pod="openstack-kuttl-tests/nova-cell1-cell-mapping-qr65w" Dec 03 12:21:28 crc kubenswrapper[5052]: I1203 12:21:28.708631 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/0bbf844e-3419-4f88-b375-ca77df3007af-scripts\") pod \"nova-cell0-cell-mapping-6dkwd\" (UID: \"0bbf844e-3419-4f88-b375-ca77df3007af\") " pod="openstack-kuttl-tests/nova-cell0-cell-mapping-6dkwd" Dec 03 12:21:28 crc kubenswrapper[5052]: I1203 12:21:28.708681 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0bbf844e-3419-4f88-b375-ca77df3007af-combined-ca-bundle\") pod \"nova-cell0-cell-mapping-6dkwd\" (UID: \"0bbf844e-3419-4f88-b375-ca77df3007af\") " pod="openstack-kuttl-tests/nova-cell0-cell-mapping-6dkwd" Dec 03 12:21:28 crc kubenswrapper[5052]: I1203 12:21:28.708775 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/27d134d0-a0b9-457b-8c8c-92cfb104cba1-combined-ca-bundle\") pod \"nova-cell1-cell-mapping-qr65w\" (UID: \"27d134d0-a0b9-457b-8c8c-92cfb104cba1\") " pod="openstack-kuttl-tests/nova-cell1-cell-mapping-qr65w" Dec 03 12:21:28 crc kubenswrapper[5052]: I1203 12:21:28.708797 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-rr4br\" (UniqueName: \"kubernetes.io/projected/27d134d0-a0b9-457b-8c8c-92cfb104cba1-kube-api-access-rr4br\") pod \"nova-cell1-cell-mapping-qr65w\" (UID: \"27d134d0-a0b9-457b-8c8c-92cfb104cba1\") " pod="openstack-kuttl-tests/nova-cell1-cell-mapping-qr65w" Dec 03 12:21:28 crc kubenswrapper[5052]: I1203 12:21:28.708826 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/0bbf844e-3419-4f88-b375-ca77df3007af-config-data\") pod \"nova-cell0-cell-mapping-6dkwd\" (UID: \"0bbf844e-3419-4f88-b375-ca77df3007af\") " pod="openstack-kuttl-tests/nova-cell0-cell-mapping-6dkwd" Dec 03 12:21:28 crc kubenswrapper[5052]: I1203 12:21:28.708857 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-xcscw\" (UniqueName: \"kubernetes.io/projected/0bbf844e-3419-4f88-b375-ca77df3007af-kube-api-access-xcscw\") pod \"nova-cell0-cell-mapping-6dkwd\" (UID: \"0bbf844e-3419-4f88-b375-ca77df3007af\") " pod="openstack-kuttl-tests/nova-cell0-cell-mapping-6dkwd" Dec 03 12:21:28 crc kubenswrapper[5052]: I1203 12:21:28.708923 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/27d134d0-a0b9-457b-8c8c-92cfb104cba1-scripts\") pod \"nova-cell1-cell-mapping-qr65w\" (UID: \"27d134d0-a0b9-457b-8c8c-92cfb104cba1\") " pod="openstack-kuttl-tests/nova-cell1-cell-mapping-qr65w" Dec 03 12:21:28 crc kubenswrapper[5052]: I1203 12:21:28.714979 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/0bbf844e-3419-4f88-b375-ca77df3007af-scripts\") pod \"nova-cell0-cell-mapping-6dkwd\" (UID: \"0bbf844e-3419-4f88-b375-ca77df3007af\") " pod="openstack-kuttl-tests/nova-cell0-cell-mapping-6dkwd" Dec 03 12:21:28 crc kubenswrapper[5052]: I1203 12:21:28.714978 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0bbf844e-3419-4f88-b375-ca77df3007af-combined-ca-bundle\") pod \"nova-cell0-cell-mapping-6dkwd\" (UID: \"0bbf844e-3419-4f88-b375-ca77df3007af\") " pod="openstack-kuttl-tests/nova-cell0-cell-mapping-6dkwd" Dec 03 12:21:28 crc kubenswrapper[5052]: I1203 12:21:28.724040 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/27d134d0-a0b9-457b-8c8c-92cfb104cba1-combined-ca-bundle\") pod \"nova-cell1-cell-mapping-qr65w\" (UID: \"27d134d0-a0b9-457b-8c8c-92cfb104cba1\") " pod="openstack-kuttl-tests/nova-cell1-cell-mapping-qr65w" Dec 03 12:21:28 crc kubenswrapper[5052]: I1203 12:21:28.724073 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/0bbf844e-3419-4f88-b375-ca77df3007af-config-data\") pod \"nova-cell0-cell-mapping-6dkwd\" (UID: \"0bbf844e-3419-4f88-b375-ca77df3007af\") " pod="openstack-kuttl-tests/nova-cell0-cell-mapping-6dkwd" Dec 03 12:21:28 crc kubenswrapper[5052]: I1203 12:21:28.724171 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/27d134d0-a0b9-457b-8c8c-92cfb104cba1-scripts\") pod \"nova-cell1-cell-mapping-qr65w\" (UID: \"27d134d0-a0b9-457b-8c8c-92cfb104cba1\") " pod="openstack-kuttl-tests/nova-cell1-cell-mapping-qr65w" Dec 03 12:21:28 crc kubenswrapper[5052]: I1203 12:21:28.725865 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/27d134d0-a0b9-457b-8c8c-92cfb104cba1-config-data\") pod \"nova-cell1-cell-mapping-qr65w\" (UID: \"27d134d0-a0b9-457b-8c8c-92cfb104cba1\") " pod="openstack-kuttl-tests/nova-cell1-cell-mapping-qr65w" Dec 03 12:21:28 crc kubenswrapper[5052]: I1203 12:21:28.727061 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-xcscw\" (UniqueName: \"kubernetes.io/projected/0bbf844e-3419-4f88-b375-ca77df3007af-kube-api-access-xcscw\") pod \"nova-cell0-cell-mapping-6dkwd\" (UID: \"0bbf844e-3419-4f88-b375-ca77df3007af\") " pod="openstack-kuttl-tests/nova-cell0-cell-mapping-6dkwd" Dec 03 12:21:28 crc kubenswrapper[5052]: I1203 12:21:28.727481 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-rr4br\" (UniqueName: \"kubernetes.io/projected/27d134d0-a0b9-457b-8c8c-92cfb104cba1-kube-api-access-rr4br\") pod \"nova-cell1-cell-mapping-qr65w\" (UID: \"27d134d0-a0b9-457b-8c8c-92cfb104cba1\") " pod="openstack-kuttl-tests/nova-cell1-cell-mapping-qr65w" Dec 03 12:21:28 crc kubenswrapper[5052]: I1203 12:21:28.897449 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-cell-mapping-6dkwd" Dec 03 12:21:28 crc kubenswrapper[5052]: I1203 12:21:28.905861 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-cell-mapping-qr65w" Dec 03 12:21:29 crc kubenswrapper[5052]: I1203 12:21:29.017745 5052 generic.go:334] "Generic (PLEG): container finished" podID="5bda226d-9d40-42d8-ac3e-11719e896649" containerID="9818701fe2f0f4ff6dc2b33cbd5c557f191bfe07f31eb908a06a4fcecfe3dcd0" exitCode=143 Dec 03 12:21:29 crc kubenswrapper[5052]: I1203 12:21:29.018042 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"5bda226d-9d40-42d8-ac3e-11719e896649","Type":"ContainerDied","Data":"9818701fe2f0f4ff6dc2b33cbd5c557f191bfe07f31eb908a06a4fcecfe3dcd0"} Dec 03 12:21:29 crc kubenswrapper[5052]: I1203 12:21:29.024979 5052 generic.go:334] "Generic (PLEG): container finished" podID="ce16d54b-ef04-429b-8795-2865395f7774" containerID="8531d325ffe575ccabb11f725af2c0fca6093a5c058282b15cfeabec695cc2f0" exitCode=143 Dec 03 12:21:29 crc kubenswrapper[5052]: I1203 12:21:29.025030 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"ce16d54b-ef04-429b-8795-2865395f7774","Type":"ContainerDied","Data":"8531d325ffe575ccabb11f725af2c0fca6093a5c058282b15cfeabec695cc2f0"} Dec 03 12:21:29 crc kubenswrapper[5052]: I1203 12:21:29.296341 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/neutron-54579c6749-xzrwm" podUID="d1660f09-947f-433a-973f-0cd19d97331f" containerName="neutron-httpd" probeResult="failure" output="Get \"https://10.217.1.108:9696/\": dial tcp 10.217.1.108:9696: connect: connection refused" Dec 03 12:21:29 crc kubenswrapper[5052]: I1203 12:21:29.410961 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell0-cell-mapping-6dkwd"] Dec 03 12:21:29 crc kubenswrapper[5052]: I1203 12:21:29.488990 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-cell-mapping-qr65w"] Dec 03 12:21:29 crc kubenswrapper[5052]: I1203 12:21:29.720322 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="896a13e1-d614-4be4-9a4a-03b438952ac9" path="/var/lib/kubelet/pods/896a13e1-d614-4be4-9a4a-03b438952ac9/volumes" Dec 03 12:21:29 crc kubenswrapper[5052]: I1203 12:21:29.721590 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aaf39e45-e5a6-4ead-82e3-035c43cbc1d7" path="/var/lib/kubelet/pods/aaf39e45-e5a6-4ead-82e3-035c43cbc1d7/volumes" Dec 03 12:21:29 crc kubenswrapper[5052]: I1203 12:21:29.743313 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:21:29 crc kubenswrapper[5052]: I1203 12:21:29.747413 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-54579c6749-xzrwm" Dec 03 12:21:29 crc kubenswrapper[5052]: I1203 12:21:29.836494 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/d1660f09-947f-433a-973f-0cd19d97331f-httpd-config\") pod \"d1660f09-947f-433a-973f-0cd19d97331f\" (UID: \"d1660f09-947f-433a-973f-0cd19d97331f\") " Dec 03 12:21:29 crc kubenswrapper[5052]: I1203 12:21:29.836964 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d1660f09-947f-433a-973f-0cd19d97331f-combined-ca-bundle\") pod \"d1660f09-947f-433a-973f-0cd19d97331f\" (UID: \"d1660f09-947f-433a-973f-0cd19d97331f\") " Dec 03 12:21:29 crc kubenswrapper[5052]: I1203 12:21:29.836996 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/9d3ab728-8f1c-48ac-808c-dfbf80b8f792-config-data\") pod \"9d3ab728-8f1c-48ac-808c-dfbf80b8f792\" (UID: \"9d3ab728-8f1c-48ac-808c-dfbf80b8f792\") " Dec 03 12:21:29 crc kubenswrapper[5052]: I1203 12:21:29.837041 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b6wjl\" (UniqueName: \"kubernetes.io/projected/d1660f09-947f-433a-973f-0cd19d97331f-kube-api-access-b6wjl\") pod \"d1660f09-947f-433a-973f-0cd19d97331f\" (UID: \"d1660f09-947f-433a-973f-0cd19d97331f\") " Dec 03 12:21:29 crc kubenswrapper[5052]: I1203 12:21:29.837068 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bq5cv\" (UniqueName: \"kubernetes.io/projected/9d3ab728-8f1c-48ac-808c-dfbf80b8f792-kube-api-access-bq5cv\") pod \"9d3ab728-8f1c-48ac-808c-dfbf80b8f792\" (UID: \"9d3ab728-8f1c-48ac-808c-dfbf80b8f792\") " Dec 03 12:21:29 crc kubenswrapper[5052]: I1203 12:21:29.837143 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/d1660f09-947f-433a-973f-0cd19d97331f-internal-tls-certs\") pod \"d1660f09-947f-433a-973f-0cd19d97331f\" (UID: \"d1660f09-947f-433a-973f-0cd19d97331f\") " Dec 03 12:21:29 crc kubenswrapper[5052]: I1203 12:21:29.837184 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9d3ab728-8f1c-48ac-808c-dfbf80b8f792-combined-ca-bundle\") pod \"9d3ab728-8f1c-48ac-808c-dfbf80b8f792\" (UID: \"9d3ab728-8f1c-48ac-808c-dfbf80b8f792\") " Dec 03 12:21:29 crc kubenswrapper[5052]: I1203 12:21:29.837228 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/d1660f09-947f-433a-973f-0cd19d97331f-ovndb-tls-certs\") pod \"d1660f09-947f-433a-973f-0cd19d97331f\" (UID: \"d1660f09-947f-433a-973f-0cd19d97331f\") " Dec 03 12:21:29 crc kubenswrapper[5052]: I1203 12:21:29.837277 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/d1660f09-947f-433a-973f-0cd19d97331f-public-tls-certs\") pod \"d1660f09-947f-433a-973f-0cd19d97331f\" (UID: \"d1660f09-947f-433a-973f-0cd19d97331f\") " Dec 03 12:21:29 crc kubenswrapper[5052]: I1203 12:21:29.837304 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/d1660f09-947f-433a-973f-0cd19d97331f-config\") pod \"d1660f09-947f-433a-973f-0cd19d97331f\" (UID: \"d1660f09-947f-433a-973f-0cd19d97331f\") " Dec 03 12:21:29 crc kubenswrapper[5052]: I1203 12:21:29.841778 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d1660f09-947f-433a-973f-0cd19d97331f-httpd-config" (OuterVolumeSpecName: "httpd-config") pod "d1660f09-947f-433a-973f-0cd19d97331f" (UID: "d1660f09-947f-433a-973f-0cd19d97331f"). InnerVolumeSpecName "httpd-config". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:29 crc kubenswrapper[5052]: I1203 12:21:29.844781 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d1660f09-947f-433a-973f-0cd19d97331f-kube-api-access-b6wjl" (OuterVolumeSpecName: "kube-api-access-b6wjl") pod "d1660f09-947f-433a-973f-0cd19d97331f" (UID: "d1660f09-947f-433a-973f-0cd19d97331f"). InnerVolumeSpecName "kube-api-access-b6wjl". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:21:29 crc kubenswrapper[5052]: I1203 12:21:29.847202 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9d3ab728-8f1c-48ac-808c-dfbf80b8f792-kube-api-access-bq5cv" (OuterVolumeSpecName: "kube-api-access-bq5cv") pod "9d3ab728-8f1c-48ac-808c-dfbf80b8f792" (UID: "9d3ab728-8f1c-48ac-808c-dfbf80b8f792"). InnerVolumeSpecName "kube-api-access-bq5cv". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:21:29 crc kubenswrapper[5052]: I1203 12:21:29.865549 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9d3ab728-8f1c-48ac-808c-dfbf80b8f792-config-data" (OuterVolumeSpecName: "config-data") pod "9d3ab728-8f1c-48ac-808c-dfbf80b8f792" (UID: "9d3ab728-8f1c-48ac-808c-dfbf80b8f792"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:29 crc kubenswrapper[5052]: I1203 12:21:29.884426 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9d3ab728-8f1c-48ac-808c-dfbf80b8f792-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "9d3ab728-8f1c-48ac-808c-dfbf80b8f792" (UID: "9d3ab728-8f1c-48ac-808c-dfbf80b8f792"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:29 crc kubenswrapper[5052]: I1203 12:21:29.907802 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d1660f09-947f-433a-973f-0cd19d97331f-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "d1660f09-947f-433a-973f-0cd19d97331f" (UID: "d1660f09-947f-433a-973f-0cd19d97331f"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:29 crc kubenswrapper[5052]: I1203 12:21:29.910038 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d1660f09-947f-433a-973f-0cd19d97331f-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "d1660f09-947f-433a-973f-0cd19d97331f" (UID: "d1660f09-947f-433a-973f-0cd19d97331f"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:29 crc kubenswrapper[5052]: I1203 12:21:29.914691 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d1660f09-947f-433a-973f-0cd19d97331f-config" (OuterVolumeSpecName: "config") pod "d1660f09-947f-433a-973f-0cd19d97331f" (UID: "d1660f09-947f-433a-973f-0cd19d97331f"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:29 crc kubenswrapper[5052]: I1203 12:21:29.923974 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d1660f09-947f-433a-973f-0cd19d97331f-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "d1660f09-947f-433a-973f-0cd19d97331f" (UID: "d1660f09-947f-433a-973f-0cd19d97331f"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:29 crc kubenswrapper[5052]: I1203 12:21:29.939955 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/d1660f09-947f-433a-973f-0cd19d97331f-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:29 crc kubenswrapper[5052]: I1203 12:21:29.939997 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/d1660f09-947f-433a-973f-0cd19d97331f-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:29 crc kubenswrapper[5052]: I1203 12:21:29.940009 5052 reconciler_common.go:293] "Volume detached for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/d1660f09-947f-433a-973f-0cd19d97331f-httpd-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:29 crc kubenswrapper[5052]: I1203 12:21:29.940021 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d1660f09-947f-433a-973f-0cd19d97331f-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:29 crc kubenswrapper[5052]: I1203 12:21:29.940033 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/9d3ab728-8f1c-48ac-808c-dfbf80b8f792-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:29 crc kubenswrapper[5052]: I1203 12:21:29.940047 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-b6wjl\" (UniqueName: \"kubernetes.io/projected/d1660f09-947f-433a-973f-0cd19d97331f-kube-api-access-b6wjl\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:29 crc kubenswrapper[5052]: I1203 12:21:29.940061 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-bq5cv\" (UniqueName: \"kubernetes.io/projected/9d3ab728-8f1c-48ac-808c-dfbf80b8f792-kube-api-access-bq5cv\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:29 crc kubenswrapper[5052]: I1203 12:21:29.940071 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/d1660f09-947f-433a-973f-0cd19d97331f-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:29 crc kubenswrapper[5052]: I1203 12:21:29.940081 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9d3ab728-8f1c-48ac-808c-dfbf80b8f792-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:29 crc kubenswrapper[5052]: I1203 12:21:29.949198 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d1660f09-947f-433a-973f-0cd19d97331f-ovndb-tls-certs" (OuterVolumeSpecName: "ovndb-tls-certs") pod "d1660f09-947f-433a-973f-0cd19d97331f" (UID: "d1660f09-947f-433a-973f-0cd19d97331f"). InnerVolumeSpecName "ovndb-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:30 crc kubenswrapper[5052]: I1203 12:21:30.038331 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-cell-mapping-6dkwd" event={"ID":"0bbf844e-3419-4f88-b375-ca77df3007af","Type":"ContainerStarted","Data":"d090e68312aedebe296e1c2b129632e2019e3685d8daf2a12e3a6ceb18d11578"} Dec 03 12:21:30 crc kubenswrapper[5052]: I1203 12:21:30.038383 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-cell-mapping-6dkwd" event={"ID":"0bbf844e-3419-4f88-b375-ca77df3007af","Type":"ContainerStarted","Data":"108d3bff9d752c0eb795849f42f7174dddd9e36267357acf3faa96c4252f424b"} Dec 03 12:21:30 crc kubenswrapper[5052]: I1203 12:21:30.039678 5052 generic.go:334] "Generic (PLEG): container finished" podID="9d3ab728-8f1c-48ac-808c-dfbf80b8f792" containerID="20a9921d6ffcb85977689eb24ca64a251931e0e59f572a888d9ee16492bf0b00" exitCode=0 Dec 03 12:21:30 crc kubenswrapper[5052]: I1203 12:21:30.039725 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"9d3ab728-8f1c-48ac-808c-dfbf80b8f792","Type":"ContainerDied","Data":"20a9921d6ffcb85977689eb24ca64a251931e0e59f572a888d9ee16492bf0b00"} Dec 03 12:21:30 crc kubenswrapper[5052]: I1203 12:21:30.039745 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"9d3ab728-8f1c-48ac-808c-dfbf80b8f792","Type":"ContainerDied","Data":"d111d7c254f77ac4200c9f7d6366b103bdf3c9690d8c2ffcdaa56adf1c7365b7"} Dec 03 12:21:30 crc kubenswrapper[5052]: I1203 12:21:30.039762 5052 scope.go:117] "RemoveContainer" containerID="20a9921d6ffcb85977689eb24ca64a251931e0e59f572a888d9ee16492bf0b00" Dec 03 12:21:30 crc kubenswrapper[5052]: I1203 12:21:30.039886 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:21:30 crc kubenswrapper[5052]: I1203 12:21:30.041641 5052 reconciler_common.go:293] "Volume detached for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/d1660f09-947f-433a-973f-0cd19d97331f-ovndb-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:30 crc kubenswrapper[5052]: I1203 12:21:30.042934 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-cell-mapping-qr65w" event={"ID":"27d134d0-a0b9-457b-8c8c-92cfb104cba1","Type":"ContainerStarted","Data":"45053dc66310b77c52cb144a03e6e50f334cdd69a31d2716d30553971203e459"} Dec 03 12:21:30 crc kubenswrapper[5052]: I1203 12:21:30.042989 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-cell-mapping-qr65w" event={"ID":"27d134d0-a0b9-457b-8c8c-92cfb104cba1","Type":"ContainerStarted","Data":"1343620f921627b260159205a761d6971c73d314e261ab38caceac9aedbe0c86"} Dec 03 12:21:30 crc kubenswrapper[5052]: I1203 12:21:30.045953 5052 generic.go:334] "Generic (PLEG): container finished" podID="d1660f09-947f-433a-973f-0cd19d97331f" containerID="165f8dc84d45c1316b53403bab7a57ec38af31a5ca7fec5ae240ccfd5404eb90" exitCode=0 Dec 03 12:21:30 crc kubenswrapper[5052]: I1203 12:21:30.046049 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-54579c6749-xzrwm" Dec 03 12:21:30 crc kubenswrapper[5052]: I1203 12:21:30.046111 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-54579c6749-xzrwm" event={"ID":"d1660f09-947f-433a-973f-0cd19d97331f","Type":"ContainerDied","Data":"165f8dc84d45c1316b53403bab7a57ec38af31a5ca7fec5ae240ccfd5404eb90"} Dec 03 12:21:30 crc kubenswrapper[5052]: I1203 12:21:30.047281 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-54579c6749-xzrwm" event={"ID":"d1660f09-947f-433a-973f-0cd19d97331f","Type":"ContainerDied","Data":"b136ec2626a76cf3a482455fc91da29f61b314a12e6e80ea4523ea4cc1bb249e"} Dec 03 12:21:30 crc kubenswrapper[5052]: I1203 12:21:30.056613 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-cell0-cell-mapping-6dkwd" podStartSLOduration=2.05658185 podStartE2EDuration="2.05658185s" podCreationTimestamp="2025-12-03 12:21:28 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:21:30.055096367 +0000 UTC m=+3680.643338151" watchObservedRunningTime="2025-12-03 12:21:30.05658185 +0000 UTC m=+3680.644823644" Dec 03 12:21:30 crc kubenswrapper[5052]: I1203 12:21:30.066687 5052 scope.go:117] "RemoveContainer" containerID="20a9921d6ffcb85977689eb24ca64a251931e0e59f572a888d9ee16492bf0b00" Dec 03 12:21:30 crc kubenswrapper[5052]: E1203 12:21:30.067082 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"20a9921d6ffcb85977689eb24ca64a251931e0e59f572a888d9ee16492bf0b00\": container with ID starting with 20a9921d6ffcb85977689eb24ca64a251931e0e59f572a888d9ee16492bf0b00 not found: ID does not exist" containerID="20a9921d6ffcb85977689eb24ca64a251931e0e59f572a888d9ee16492bf0b00" Dec 03 12:21:30 crc kubenswrapper[5052]: I1203 12:21:30.067114 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"20a9921d6ffcb85977689eb24ca64a251931e0e59f572a888d9ee16492bf0b00"} err="failed to get container status \"20a9921d6ffcb85977689eb24ca64a251931e0e59f572a888d9ee16492bf0b00\": rpc error: code = NotFound desc = could not find container \"20a9921d6ffcb85977689eb24ca64a251931e0e59f572a888d9ee16492bf0b00\": container with ID starting with 20a9921d6ffcb85977689eb24ca64a251931e0e59f572a888d9ee16492bf0b00 not found: ID does not exist" Dec 03 12:21:30 crc kubenswrapper[5052]: I1203 12:21:30.067143 5052 scope.go:117] "RemoveContainer" containerID="99d0e325b776b63e2cdf1bb10d165091f0981effadb9947661ddb77345f58080" Dec 03 12:21:30 crc kubenswrapper[5052]: I1203 12:21:30.082151 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-cell1-cell-mapping-qr65w" podStartSLOduration=2.082130929 podStartE2EDuration="2.082130929s" podCreationTimestamp="2025-12-03 12:21:28 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:21:30.071906917 +0000 UTC m=+3680.660148691" watchObservedRunningTime="2025-12-03 12:21:30.082130929 +0000 UTC m=+3680.670372693" Dec 03 12:21:30 crc kubenswrapper[5052]: I1203 12:21:30.107455 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:21:30 crc kubenswrapper[5052]: I1203 12:21:30.119801 5052 scope.go:117] "RemoveContainer" containerID="165f8dc84d45c1316b53403bab7a57ec38af31a5ca7fec5ae240ccfd5404eb90" Dec 03 12:21:30 crc kubenswrapper[5052]: I1203 12:21:30.120925 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:21:30 crc kubenswrapper[5052]: I1203 12:21:30.131686 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/neutron-54579c6749-xzrwm"] Dec 03 12:21:30 crc kubenswrapper[5052]: I1203 12:21:30.146387 5052 scope.go:117] "RemoveContainer" containerID="99d0e325b776b63e2cdf1bb10d165091f0981effadb9947661ddb77345f58080" Dec 03 12:21:30 crc kubenswrapper[5052]: I1203 12:21:30.146502 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:21:30 crc kubenswrapper[5052]: E1203 12:21:30.146980 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="d1660f09-947f-433a-973f-0cd19d97331f" containerName="neutron-api" Dec 03 12:21:30 crc kubenswrapper[5052]: E1203 12:21:30.146989 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"99d0e325b776b63e2cdf1bb10d165091f0981effadb9947661ddb77345f58080\": container with ID starting with 99d0e325b776b63e2cdf1bb10d165091f0981effadb9947661ddb77345f58080 not found: ID does not exist" containerID="99d0e325b776b63e2cdf1bb10d165091f0981effadb9947661ddb77345f58080" Dec 03 12:21:30 crc kubenswrapper[5052]: I1203 12:21:30.147031 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"99d0e325b776b63e2cdf1bb10d165091f0981effadb9947661ddb77345f58080"} err="failed to get container status \"99d0e325b776b63e2cdf1bb10d165091f0981effadb9947661ddb77345f58080\": rpc error: code = NotFound desc = could not find container \"99d0e325b776b63e2cdf1bb10d165091f0981effadb9947661ddb77345f58080\": container with ID starting with 99d0e325b776b63e2cdf1bb10d165091f0981effadb9947661ddb77345f58080 not found: ID does not exist" Dec 03 12:21:30 crc kubenswrapper[5052]: I1203 12:21:30.147057 5052 scope.go:117] "RemoveContainer" containerID="165f8dc84d45c1316b53403bab7a57ec38af31a5ca7fec5ae240ccfd5404eb90" Dec 03 12:21:30 crc kubenswrapper[5052]: I1203 12:21:30.147002 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="d1660f09-947f-433a-973f-0cd19d97331f" containerName="neutron-api" Dec 03 12:21:30 crc kubenswrapper[5052]: E1203 12:21:30.147138 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="d1660f09-947f-433a-973f-0cd19d97331f" containerName="neutron-httpd" Dec 03 12:21:30 crc kubenswrapper[5052]: I1203 12:21:30.147149 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="d1660f09-947f-433a-973f-0cd19d97331f" containerName="neutron-httpd" Dec 03 12:21:30 crc kubenswrapper[5052]: E1203 12:21:30.147181 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="9d3ab728-8f1c-48ac-808c-dfbf80b8f792" containerName="nova-scheduler-scheduler" Dec 03 12:21:30 crc kubenswrapper[5052]: I1203 12:21:30.147189 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d3ab728-8f1c-48ac-808c-dfbf80b8f792" containerName="nova-scheduler-scheduler" Dec 03 12:21:30 crc kubenswrapper[5052]: E1203 12:21:30.147285 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"165f8dc84d45c1316b53403bab7a57ec38af31a5ca7fec5ae240ccfd5404eb90\": container with ID starting with 165f8dc84d45c1316b53403bab7a57ec38af31a5ca7fec5ae240ccfd5404eb90 not found: ID does not exist" containerID="165f8dc84d45c1316b53403bab7a57ec38af31a5ca7fec5ae240ccfd5404eb90" Dec 03 12:21:30 crc kubenswrapper[5052]: I1203 12:21:30.147302 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"165f8dc84d45c1316b53403bab7a57ec38af31a5ca7fec5ae240ccfd5404eb90"} err="failed to get container status \"165f8dc84d45c1316b53403bab7a57ec38af31a5ca7fec5ae240ccfd5404eb90\": rpc error: code = NotFound desc = could not find container \"165f8dc84d45c1316b53403bab7a57ec38af31a5ca7fec5ae240ccfd5404eb90\": container with ID starting with 165f8dc84d45c1316b53403bab7a57ec38af31a5ca7fec5ae240ccfd5404eb90 not found: ID does not exist" Dec 03 12:21:30 crc kubenswrapper[5052]: I1203 12:21:30.147399 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="d1660f09-947f-433a-973f-0cd19d97331f" containerName="neutron-api" Dec 03 12:21:30 crc kubenswrapper[5052]: I1203 12:21:30.147426 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="9d3ab728-8f1c-48ac-808c-dfbf80b8f792" containerName="nova-scheduler-scheduler" Dec 03 12:21:30 crc kubenswrapper[5052]: I1203 12:21:30.147440 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="d1660f09-947f-433a-973f-0cd19d97331f" containerName="neutron-httpd" Dec 03 12:21:30 crc kubenswrapper[5052]: I1203 12:21:30.148192 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:21:30 crc kubenswrapper[5052]: I1203 12:21:30.150595 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-scheduler-config-data" Dec 03 12:21:30 crc kubenswrapper[5052]: I1203 12:21:30.154430 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/neutron-54579c6749-xzrwm"] Dec 03 12:21:30 crc kubenswrapper[5052]: I1203 12:21:30.163949 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:21:30 crc kubenswrapper[5052]: I1203 12:21:30.245068 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q4cg5\" (UniqueName: \"kubernetes.io/projected/779772f8-a0bd-491f-9153-33614d19e423-kube-api-access-q4cg5\") pod \"nova-scheduler-0\" (UID: \"779772f8-a0bd-491f-9153-33614d19e423\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:21:30 crc kubenswrapper[5052]: I1203 12:21:30.245139 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/779772f8-a0bd-491f-9153-33614d19e423-combined-ca-bundle\") pod \"nova-scheduler-0\" (UID: \"779772f8-a0bd-491f-9153-33614d19e423\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:21:30 crc kubenswrapper[5052]: I1203 12:21:30.245225 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/779772f8-a0bd-491f-9153-33614d19e423-config-data\") pod \"nova-scheduler-0\" (UID: \"779772f8-a0bd-491f-9153-33614d19e423\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:21:30 crc kubenswrapper[5052]: I1203 12:21:30.346521 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-q4cg5\" (UniqueName: \"kubernetes.io/projected/779772f8-a0bd-491f-9153-33614d19e423-kube-api-access-q4cg5\") pod \"nova-scheduler-0\" (UID: \"779772f8-a0bd-491f-9153-33614d19e423\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:21:30 crc kubenswrapper[5052]: I1203 12:21:30.346584 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/779772f8-a0bd-491f-9153-33614d19e423-combined-ca-bundle\") pod \"nova-scheduler-0\" (UID: \"779772f8-a0bd-491f-9153-33614d19e423\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:21:30 crc kubenswrapper[5052]: I1203 12:21:30.346659 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/779772f8-a0bd-491f-9153-33614d19e423-config-data\") pod \"nova-scheduler-0\" (UID: \"779772f8-a0bd-491f-9153-33614d19e423\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:21:30 crc kubenswrapper[5052]: I1203 12:21:30.350773 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/779772f8-a0bd-491f-9153-33614d19e423-config-data\") pod \"nova-scheduler-0\" (UID: \"779772f8-a0bd-491f-9153-33614d19e423\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:21:30 crc kubenswrapper[5052]: I1203 12:21:30.350891 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/779772f8-a0bd-491f-9153-33614d19e423-combined-ca-bundle\") pod \"nova-scheduler-0\" (UID: \"779772f8-a0bd-491f-9153-33614d19e423\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:21:30 crc kubenswrapper[5052]: I1203 12:21:30.362715 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-q4cg5\" (UniqueName: \"kubernetes.io/projected/779772f8-a0bd-491f-9153-33614d19e423-kube-api-access-q4cg5\") pod \"nova-scheduler-0\" (UID: \"779772f8-a0bd-491f-9153-33614d19e423\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:21:30 crc kubenswrapper[5052]: I1203 12:21:30.476113 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:21:30 crc kubenswrapper[5052]: I1203 12:21:30.971449 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:21:30 crc kubenswrapper[5052]: W1203 12:21:30.974178 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod779772f8_a0bd_491f_9153_33614d19e423.slice/crio-95fd26e67c32872c11067f163126b665a158781319a98d2158a54a7bd7d0390d WatchSource:0}: Error finding container 95fd26e67c32872c11067f163126b665a158781319a98d2158a54a7bd7d0390d: Status 404 returned error can't find the container with id 95fd26e67c32872c11067f163126b665a158781319a98d2158a54a7bd7d0390d Dec 03 12:21:31 crc kubenswrapper[5052]: I1203 12:21:31.059359 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"779772f8-a0bd-491f-9153-33614d19e423","Type":"ContainerStarted","Data":"95fd26e67c32872c11067f163126b665a158781319a98d2158a54a7bd7d0390d"} Dec 03 12:21:31 crc kubenswrapper[5052]: I1203 12:21:31.563574 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/nova-metadata-0" podUID="5bda226d-9d40-42d8-ac3e-11719e896649" containerName="nova-metadata-log" probeResult="failure" output="Get \"https://10.217.1.144:8775/\": read tcp 10.217.0.2:56864->10.217.1.144:8775: read: connection reset by peer" Dec 03 12:21:31 crc kubenswrapper[5052]: I1203 12:21:31.563651 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/nova-metadata-0" podUID="5bda226d-9d40-42d8-ac3e-11719e896649" containerName="nova-metadata-metadata" probeResult="failure" output="Get \"https://10.217.1.144:8775/\": read tcp 10.217.0.2:56852->10.217.1.144:8775: read: connection reset by peer" Dec 03 12:21:31 crc kubenswrapper[5052]: I1203 12:21:31.682462 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9d3ab728-8f1c-48ac-808c-dfbf80b8f792" path="/var/lib/kubelet/pods/9d3ab728-8f1c-48ac-808c-dfbf80b8f792/volumes" Dec 03 12:21:31 crc kubenswrapper[5052]: I1203 12:21:31.683224 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d1660f09-947f-433a-973f-0cd19d97331f" path="/var/lib/kubelet/pods/d1660f09-947f-433a-973f-0cd19d97331f/volumes" Dec 03 12:21:32 crc kubenswrapper[5052]: I1203 12:21:32.078548 5052 generic.go:334] "Generic (PLEG): container finished" podID="ce16d54b-ef04-429b-8795-2865395f7774" containerID="0a4ad14776400bd11635febb15632b857ffe75a1ed4b078b2eb0c7621d98c834" exitCode=0 Dec 03 12:21:32 crc kubenswrapper[5052]: I1203 12:21:32.078608 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"ce16d54b-ef04-429b-8795-2865395f7774","Type":"ContainerDied","Data":"0a4ad14776400bd11635febb15632b857ffe75a1ed4b078b2eb0c7621d98c834"} Dec 03 12:21:32 crc kubenswrapper[5052]: I1203 12:21:32.079742 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"779772f8-a0bd-491f-9153-33614d19e423","Type":"ContainerStarted","Data":"db51cc7950f2c5a12d1aa905e418ed04e400dcebb362b0f45a28636e8440122e"} Dec 03 12:21:32 crc kubenswrapper[5052]: I1203 12:21:32.082396 5052 generic.go:334] "Generic (PLEG): container finished" podID="5bda226d-9d40-42d8-ac3e-11719e896649" containerID="67d4dbc28757901384966bfa33590ec06891feae45053aaf9e8e283f4b17ad15" exitCode=0 Dec 03 12:21:32 crc kubenswrapper[5052]: I1203 12:21:32.082426 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"5bda226d-9d40-42d8-ac3e-11719e896649","Type":"ContainerDied","Data":"67d4dbc28757901384966bfa33590ec06891feae45053aaf9e8e283f4b17ad15"} Dec 03 12:21:32 crc kubenswrapper[5052]: I1203 12:21:32.098223 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-scheduler-0" podStartSLOduration=2.098204323 podStartE2EDuration="2.098204323s" podCreationTimestamp="2025-12-03 12:21:30 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:21:32.093818907 +0000 UTC m=+3682.682060671" watchObservedRunningTime="2025-12-03 12:21:32.098204323 +0000 UTC m=+3682.686446087" Dec 03 12:21:32 crc kubenswrapper[5052]: I1203 12:21:32.249119 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:21:32 crc kubenswrapper[5052]: I1203 12:21:32.282159 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7qgpg\" (UniqueName: \"kubernetes.io/projected/ce16d54b-ef04-429b-8795-2865395f7774-kube-api-access-7qgpg\") pod \"ce16d54b-ef04-429b-8795-2865395f7774\" (UID: \"ce16d54b-ef04-429b-8795-2865395f7774\") " Dec 03 12:21:32 crc kubenswrapper[5052]: I1203 12:21:32.282545 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/ce16d54b-ef04-429b-8795-2865395f7774-config-data\") pod \"ce16d54b-ef04-429b-8795-2865395f7774\" (UID: \"ce16d54b-ef04-429b-8795-2865395f7774\") " Dec 03 12:21:32 crc kubenswrapper[5052]: I1203 12:21:32.282634 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ce16d54b-ef04-429b-8795-2865395f7774-combined-ca-bundle\") pod \"ce16d54b-ef04-429b-8795-2865395f7774\" (UID: \"ce16d54b-ef04-429b-8795-2865395f7774\") " Dec 03 12:21:32 crc kubenswrapper[5052]: I1203 12:21:32.282671 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/ce16d54b-ef04-429b-8795-2865395f7774-internal-tls-certs\") pod \"ce16d54b-ef04-429b-8795-2865395f7774\" (UID: \"ce16d54b-ef04-429b-8795-2865395f7774\") " Dec 03 12:21:32 crc kubenswrapper[5052]: I1203 12:21:32.282839 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/ce16d54b-ef04-429b-8795-2865395f7774-public-tls-certs\") pod \"ce16d54b-ef04-429b-8795-2865395f7774\" (UID: \"ce16d54b-ef04-429b-8795-2865395f7774\") " Dec 03 12:21:32 crc kubenswrapper[5052]: I1203 12:21:32.282869 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/ce16d54b-ef04-429b-8795-2865395f7774-logs\") pod \"ce16d54b-ef04-429b-8795-2865395f7774\" (UID: \"ce16d54b-ef04-429b-8795-2865395f7774\") " Dec 03 12:21:32 crc kubenswrapper[5052]: I1203 12:21:32.283760 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ce16d54b-ef04-429b-8795-2865395f7774-logs" (OuterVolumeSpecName: "logs") pod "ce16d54b-ef04-429b-8795-2865395f7774" (UID: "ce16d54b-ef04-429b-8795-2865395f7774"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:21:32 crc kubenswrapper[5052]: I1203 12:21:32.320091 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ce16d54b-ef04-429b-8795-2865395f7774-kube-api-access-7qgpg" (OuterVolumeSpecName: "kube-api-access-7qgpg") pod "ce16d54b-ef04-429b-8795-2865395f7774" (UID: "ce16d54b-ef04-429b-8795-2865395f7774"). InnerVolumeSpecName "kube-api-access-7qgpg". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:21:32 crc kubenswrapper[5052]: I1203 12:21:32.333792 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ce16d54b-ef04-429b-8795-2865395f7774-config-data" (OuterVolumeSpecName: "config-data") pod "ce16d54b-ef04-429b-8795-2865395f7774" (UID: "ce16d54b-ef04-429b-8795-2865395f7774"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:32 crc kubenswrapper[5052]: I1203 12:21:32.354602 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ce16d54b-ef04-429b-8795-2865395f7774-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "ce16d54b-ef04-429b-8795-2865395f7774" (UID: "ce16d54b-ef04-429b-8795-2865395f7774"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:32 crc kubenswrapper[5052]: I1203 12:21:32.371975 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ce16d54b-ef04-429b-8795-2865395f7774-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "ce16d54b-ef04-429b-8795-2865395f7774" (UID: "ce16d54b-ef04-429b-8795-2865395f7774"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:32 crc kubenswrapper[5052]: I1203 12:21:32.384991 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-7qgpg\" (UniqueName: \"kubernetes.io/projected/ce16d54b-ef04-429b-8795-2865395f7774-kube-api-access-7qgpg\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:32 crc kubenswrapper[5052]: I1203 12:21:32.385055 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/ce16d54b-ef04-429b-8795-2865395f7774-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:32 crc kubenswrapper[5052]: I1203 12:21:32.385067 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ce16d54b-ef04-429b-8795-2865395f7774-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:32 crc kubenswrapper[5052]: I1203 12:21:32.385110 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/ce16d54b-ef04-429b-8795-2865395f7774-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:32 crc kubenswrapper[5052]: I1203 12:21:32.385125 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/ce16d54b-ef04-429b-8795-2865395f7774-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:32 crc kubenswrapper[5052]: I1203 12:21:32.387721 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ce16d54b-ef04-429b-8795-2865395f7774-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "ce16d54b-ef04-429b-8795-2865395f7774" (UID: "ce16d54b-ef04-429b-8795-2865395f7774"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:32 crc kubenswrapper[5052]: I1203 12:21:32.488027 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/ce16d54b-ef04-429b-8795-2865395f7774-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:32 crc kubenswrapper[5052]: I1203 12:21:32.596268 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:21:32 crc kubenswrapper[5052]: I1203 12:21:32.694748 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/5bda226d-9d40-42d8-ac3e-11719e896649-nova-metadata-tls-certs\") pod \"5bda226d-9d40-42d8-ac3e-11719e896649\" (UID: \"5bda226d-9d40-42d8-ac3e-11719e896649\") " Dec 03 12:21:32 crc kubenswrapper[5052]: I1203 12:21:32.694799 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5bda226d-9d40-42d8-ac3e-11719e896649-combined-ca-bundle\") pod \"5bda226d-9d40-42d8-ac3e-11719e896649\" (UID: \"5bda226d-9d40-42d8-ac3e-11719e896649\") " Dec 03 12:21:32 crc kubenswrapper[5052]: I1203 12:21:32.696985 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/5bda226d-9d40-42d8-ac3e-11719e896649-logs\") pod \"5bda226d-9d40-42d8-ac3e-11719e896649\" (UID: \"5bda226d-9d40-42d8-ac3e-11719e896649\") " Dec 03 12:21:32 crc kubenswrapper[5052]: I1203 12:21:32.697033 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zvm9j\" (UniqueName: \"kubernetes.io/projected/5bda226d-9d40-42d8-ac3e-11719e896649-kube-api-access-zvm9j\") pod \"5bda226d-9d40-42d8-ac3e-11719e896649\" (UID: \"5bda226d-9d40-42d8-ac3e-11719e896649\") " Dec 03 12:21:32 crc kubenswrapper[5052]: I1203 12:21:32.697105 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/5bda226d-9d40-42d8-ac3e-11719e896649-config-data\") pod \"5bda226d-9d40-42d8-ac3e-11719e896649\" (UID: \"5bda226d-9d40-42d8-ac3e-11719e896649\") " Dec 03 12:21:32 crc kubenswrapper[5052]: I1203 12:21:32.698646 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5bda226d-9d40-42d8-ac3e-11719e896649-logs" (OuterVolumeSpecName: "logs") pod "5bda226d-9d40-42d8-ac3e-11719e896649" (UID: "5bda226d-9d40-42d8-ac3e-11719e896649"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:21:32 crc kubenswrapper[5052]: I1203 12:21:32.706670 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5bda226d-9d40-42d8-ac3e-11719e896649-kube-api-access-zvm9j" (OuterVolumeSpecName: "kube-api-access-zvm9j") pod "5bda226d-9d40-42d8-ac3e-11719e896649" (UID: "5bda226d-9d40-42d8-ac3e-11719e896649"). InnerVolumeSpecName "kube-api-access-zvm9j". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:21:32 crc kubenswrapper[5052]: I1203 12:21:32.754788 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5bda226d-9d40-42d8-ac3e-11719e896649-config-data" (OuterVolumeSpecName: "config-data") pod "5bda226d-9d40-42d8-ac3e-11719e896649" (UID: "5bda226d-9d40-42d8-ac3e-11719e896649"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:32 crc kubenswrapper[5052]: I1203 12:21:32.761875 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5bda226d-9d40-42d8-ac3e-11719e896649-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "5bda226d-9d40-42d8-ac3e-11719e896649" (UID: "5bda226d-9d40-42d8-ac3e-11719e896649"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:32 crc kubenswrapper[5052]: I1203 12:21:32.781606 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5bda226d-9d40-42d8-ac3e-11719e896649-nova-metadata-tls-certs" (OuterVolumeSpecName: "nova-metadata-tls-certs") pod "5bda226d-9d40-42d8-ac3e-11719e896649" (UID: "5bda226d-9d40-42d8-ac3e-11719e896649"). InnerVolumeSpecName "nova-metadata-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:32 crc kubenswrapper[5052]: I1203 12:21:32.800979 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/5bda226d-9d40-42d8-ac3e-11719e896649-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:32 crc kubenswrapper[5052]: I1203 12:21:32.801221 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-zvm9j\" (UniqueName: \"kubernetes.io/projected/5bda226d-9d40-42d8-ac3e-11719e896649-kube-api-access-zvm9j\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:32 crc kubenswrapper[5052]: I1203 12:21:32.801300 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/5bda226d-9d40-42d8-ac3e-11719e896649-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:32 crc kubenswrapper[5052]: I1203 12:21:32.801393 5052 reconciler_common.go:293] "Volume detached for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/5bda226d-9d40-42d8-ac3e-11719e896649-nova-metadata-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:32 crc kubenswrapper[5052]: I1203 12:21:32.801465 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5bda226d-9d40-42d8-ac3e-11719e896649-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:33 crc kubenswrapper[5052]: I1203 12:21:33.101614 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:21:33 crc kubenswrapper[5052]: I1203 12:21:33.101449 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"5bda226d-9d40-42d8-ac3e-11719e896649","Type":"ContainerDied","Data":"457c34f728bf076a5e41aa3b8eff71a377813b1768597cae052035c793de82a9"} Dec 03 12:21:33 crc kubenswrapper[5052]: I1203 12:21:33.102338 5052 scope.go:117] "RemoveContainer" containerID="67d4dbc28757901384966bfa33590ec06891feae45053aaf9e8e283f4b17ad15" Dec 03 12:21:33 crc kubenswrapper[5052]: I1203 12:21:33.104412 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:21:33 crc kubenswrapper[5052]: I1203 12:21:33.104772 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"ce16d54b-ef04-429b-8795-2865395f7774","Type":"ContainerDied","Data":"dfb12dbb2d7b94c4c5755dc4e198ee9048c925b8d89483a14be0f89b600fa089"} Dec 03 12:21:33 crc kubenswrapper[5052]: I1203 12:21:33.146722 5052 scope.go:117] "RemoveContainer" containerID="9818701fe2f0f4ff6dc2b33cbd5c557f191bfe07f31eb908a06a4fcecfe3dcd0" Dec 03 12:21:33 crc kubenswrapper[5052]: I1203 12:21:33.180268 5052 scope.go:117] "RemoveContainer" containerID="0a4ad14776400bd11635febb15632b857ffe75a1ed4b078b2eb0c7621d98c834" Dec 03 12:21:33 crc kubenswrapper[5052]: I1203 12:21:33.199061 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:21:33 crc kubenswrapper[5052]: I1203 12:21:33.262652 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:21:33 crc kubenswrapper[5052]: I1203 12:21:33.264442 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:21:33 crc kubenswrapper[5052]: I1203 12:21:33.269243 5052 scope.go:117] "RemoveContainer" containerID="8531d325ffe575ccabb11f725af2c0fca6093a5c058282b15cfeabec695cc2f0" Dec 03 12:21:33 crc kubenswrapper[5052]: I1203 12:21:33.279979 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:21:33 crc kubenswrapper[5052]: I1203 12:21:33.285838 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:21:33 crc kubenswrapper[5052]: E1203 12:21:33.286684 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="5bda226d-9d40-42d8-ac3e-11719e896649" containerName="nova-metadata-log" Dec 03 12:21:33 crc kubenswrapper[5052]: I1203 12:21:33.286702 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="5bda226d-9d40-42d8-ac3e-11719e896649" containerName="nova-metadata-log" Dec 03 12:21:33 crc kubenswrapper[5052]: E1203 12:21:33.286722 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="5bda226d-9d40-42d8-ac3e-11719e896649" containerName="nova-metadata-metadata" Dec 03 12:21:33 crc kubenswrapper[5052]: I1203 12:21:33.286730 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="5bda226d-9d40-42d8-ac3e-11719e896649" containerName="nova-metadata-metadata" Dec 03 12:21:33 crc kubenswrapper[5052]: E1203 12:21:33.286745 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="ce16d54b-ef04-429b-8795-2865395f7774" containerName="nova-api-log" Dec 03 12:21:33 crc kubenswrapper[5052]: I1203 12:21:33.286753 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="ce16d54b-ef04-429b-8795-2865395f7774" containerName="nova-api-log" Dec 03 12:21:33 crc kubenswrapper[5052]: E1203 12:21:33.286801 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="ce16d54b-ef04-429b-8795-2865395f7774" containerName="nova-api-api" Dec 03 12:21:33 crc kubenswrapper[5052]: I1203 12:21:33.286811 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="ce16d54b-ef04-429b-8795-2865395f7774" containerName="nova-api-api" Dec 03 12:21:33 crc kubenswrapper[5052]: I1203 12:21:33.287048 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="ce16d54b-ef04-429b-8795-2865395f7774" containerName="nova-api-api" Dec 03 12:21:33 crc kubenswrapper[5052]: I1203 12:21:33.287067 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="5bda226d-9d40-42d8-ac3e-11719e896649" containerName="nova-metadata-metadata" Dec 03 12:21:33 crc kubenswrapper[5052]: I1203 12:21:33.287087 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="5bda226d-9d40-42d8-ac3e-11719e896649" containerName="nova-metadata-log" Dec 03 12:21:33 crc kubenswrapper[5052]: I1203 12:21:33.287099 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="ce16d54b-ef04-429b-8795-2865395f7774" containerName="nova-api-log" Dec 03 12:21:33 crc kubenswrapper[5052]: I1203 12:21:33.288499 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:21:33 crc kubenswrapper[5052]: I1203 12:21:33.294192 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-nova-public-svc" Dec 03 12:21:33 crc kubenswrapper[5052]: I1203 12:21:33.294253 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-nova-internal-svc" Dec 03 12:21:33 crc kubenswrapper[5052]: I1203 12:21:33.294358 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-api-config-data" Dec 03 12:21:33 crc kubenswrapper[5052]: I1203 12:21:33.299346 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:21:33 crc kubenswrapper[5052]: I1203 12:21:33.312027 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2cbpl\" (UniqueName: \"kubernetes.io/projected/cd50414b-e062-408f-9ce0-052489596b5b-kube-api-access-2cbpl\") pod \"nova-api-0\" (UID: \"cd50414b-e062-408f-9ce0-052489596b5b\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:21:33 crc kubenswrapper[5052]: I1203 12:21:33.312091 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/cd50414b-e062-408f-9ce0-052489596b5b-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"cd50414b-e062-408f-9ce0-052489596b5b\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:21:33 crc kubenswrapper[5052]: I1203 12:21:33.312118 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/cd50414b-e062-408f-9ce0-052489596b5b-config-data\") pod \"nova-api-0\" (UID: \"cd50414b-e062-408f-9ce0-052489596b5b\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:21:33 crc kubenswrapper[5052]: I1203 12:21:33.312174 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/cd50414b-e062-408f-9ce0-052489596b5b-public-tls-certs\") pod \"nova-api-0\" (UID: \"cd50414b-e062-408f-9ce0-052489596b5b\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:21:33 crc kubenswrapper[5052]: I1203 12:21:33.312211 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/cd50414b-e062-408f-9ce0-052489596b5b-logs\") pod \"nova-api-0\" (UID: \"cd50414b-e062-408f-9ce0-052489596b5b\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:21:33 crc kubenswrapper[5052]: I1203 12:21:33.312230 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/cd50414b-e062-408f-9ce0-052489596b5b-internal-tls-certs\") pod \"nova-api-0\" (UID: \"cd50414b-e062-408f-9ce0-052489596b5b\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:21:33 crc kubenswrapper[5052]: I1203 12:21:33.316078 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:21:33 crc kubenswrapper[5052]: I1203 12:21:33.319427 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:21:33 crc kubenswrapper[5052]: I1203 12:21:33.323633 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:21:33 crc kubenswrapper[5052]: I1203 12:21:33.329881 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-metadata-config-data" Dec 03 12:21:33 crc kubenswrapper[5052]: I1203 12:21:33.330484 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-nova-metadata-internal-svc" Dec 03 12:21:33 crc kubenswrapper[5052]: I1203 12:21:33.414954 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ffbd4\" (UniqueName: \"kubernetes.io/projected/e77ccc47-f7cc-4543-8650-6c03f3764778-kube-api-access-ffbd4\") pod \"nova-metadata-0\" (UID: \"e77ccc47-f7cc-4543-8650-6c03f3764778\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:21:33 crc kubenswrapper[5052]: I1203 12:21:33.415314 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/e77ccc47-f7cc-4543-8650-6c03f3764778-config-data\") pod \"nova-metadata-0\" (UID: \"e77ccc47-f7cc-4543-8650-6c03f3764778\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:21:33 crc kubenswrapper[5052]: I1203 12:21:33.415438 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/e77ccc47-f7cc-4543-8650-6c03f3764778-nova-metadata-tls-certs\") pod \"nova-metadata-0\" (UID: \"e77ccc47-f7cc-4543-8650-6c03f3764778\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:21:33 crc kubenswrapper[5052]: I1203 12:21:33.415630 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-2cbpl\" (UniqueName: \"kubernetes.io/projected/cd50414b-e062-408f-9ce0-052489596b5b-kube-api-access-2cbpl\") pod \"nova-api-0\" (UID: \"cd50414b-e062-408f-9ce0-052489596b5b\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:21:33 crc kubenswrapper[5052]: I1203 12:21:33.415764 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/cd50414b-e062-408f-9ce0-052489596b5b-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"cd50414b-e062-408f-9ce0-052489596b5b\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:21:33 crc kubenswrapper[5052]: I1203 12:21:33.415853 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/e77ccc47-f7cc-4543-8650-6c03f3764778-logs\") pod \"nova-metadata-0\" (UID: \"e77ccc47-f7cc-4543-8650-6c03f3764778\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:21:33 crc kubenswrapper[5052]: I1203 12:21:33.415948 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/cd50414b-e062-408f-9ce0-052489596b5b-config-data\") pod \"nova-api-0\" (UID: \"cd50414b-e062-408f-9ce0-052489596b5b\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:21:33 crc kubenswrapper[5052]: I1203 12:21:33.416061 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e77ccc47-f7cc-4543-8650-6c03f3764778-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"e77ccc47-f7cc-4543-8650-6c03f3764778\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:21:33 crc kubenswrapper[5052]: I1203 12:21:33.416276 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/cd50414b-e062-408f-9ce0-052489596b5b-public-tls-certs\") pod \"nova-api-0\" (UID: \"cd50414b-e062-408f-9ce0-052489596b5b\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:21:33 crc kubenswrapper[5052]: I1203 12:21:33.416435 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/cd50414b-e062-408f-9ce0-052489596b5b-logs\") pod \"nova-api-0\" (UID: \"cd50414b-e062-408f-9ce0-052489596b5b\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:21:33 crc kubenswrapper[5052]: I1203 12:21:33.416587 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/cd50414b-e062-408f-9ce0-052489596b5b-internal-tls-certs\") pod \"nova-api-0\" (UID: \"cd50414b-e062-408f-9ce0-052489596b5b\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:21:33 crc kubenswrapper[5052]: I1203 12:21:33.418368 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/cd50414b-e062-408f-9ce0-052489596b5b-logs\") pod \"nova-api-0\" (UID: \"cd50414b-e062-408f-9ce0-052489596b5b\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:21:33 crc kubenswrapper[5052]: I1203 12:21:33.422487 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/cd50414b-e062-408f-9ce0-052489596b5b-public-tls-certs\") pod \"nova-api-0\" (UID: \"cd50414b-e062-408f-9ce0-052489596b5b\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:21:33 crc kubenswrapper[5052]: I1203 12:21:33.422799 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/cd50414b-e062-408f-9ce0-052489596b5b-internal-tls-certs\") pod \"nova-api-0\" (UID: \"cd50414b-e062-408f-9ce0-052489596b5b\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:21:33 crc kubenswrapper[5052]: I1203 12:21:33.423241 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/cd50414b-e062-408f-9ce0-052489596b5b-config-data\") pod \"nova-api-0\" (UID: \"cd50414b-e062-408f-9ce0-052489596b5b\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:21:33 crc kubenswrapper[5052]: I1203 12:21:33.424207 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/cd50414b-e062-408f-9ce0-052489596b5b-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"cd50414b-e062-408f-9ce0-052489596b5b\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:21:33 crc kubenswrapper[5052]: I1203 12:21:33.444030 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-2cbpl\" (UniqueName: \"kubernetes.io/projected/cd50414b-e062-408f-9ce0-052489596b5b-kube-api-access-2cbpl\") pod \"nova-api-0\" (UID: \"cd50414b-e062-408f-9ce0-052489596b5b\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:21:33 crc kubenswrapper[5052]: I1203 12:21:33.518387 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/e77ccc47-f7cc-4543-8650-6c03f3764778-logs\") pod \"nova-metadata-0\" (UID: \"e77ccc47-f7cc-4543-8650-6c03f3764778\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:21:33 crc kubenswrapper[5052]: I1203 12:21:33.519161 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e77ccc47-f7cc-4543-8650-6c03f3764778-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"e77ccc47-f7cc-4543-8650-6c03f3764778\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:21:33 crc kubenswrapper[5052]: I1203 12:21:33.519095 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/e77ccc47-f7cc-4543-8650-6c03f3764778-logs\") pod \"nova-metadata-0\" (UID: \"e77ccc47-f7cc-4543-8650-6c03f3764778\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:21:33 crc kubenswrapper[5052]: I1203 12:21:33.520066 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-ffbd4\" (UniqueName: \"kubernetes.io/projected/e77ccc47-f7cc-4543-8650-6c03f3764778-kube-api-access-ffbd4\") pod \"nova-metadata-0\" (UID: \"e77ccc47-f7cc-4543-8650-6c03f3764778\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:21:33 crc kubenswrapper[5052]: I1203 12:21:33.520199 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/e77ccc47-f7cc-4543-8650-6c03f3764778-config-data\") pod \"nova-metadata-0\" (UID: \"e77ccc47-f7cc-4543-8650-6c03f3764778\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:21:33 crc kubenswrapper[5052]: I1203 12:21:33.520315 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/e77ccc47-f7cc-4543-8650-6c03f3764778-nova-metadata-tls-certs\") pod \"nova-metadata-0\" (UID: \"e77ccc47-f7cc-4543-8650-6c03f3764778\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:21:33 crc kubenswrapper[5052]: I1203 12:21:33.522696 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e77ccc47-f7cc-4543-8650-6c03f3764778-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"e77ccc47-f7cc-4543-8650-6c03f3764778\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:21:33 crc kubenswrapper[5052]: I1203 12:21:33.523200 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/e77ccc47-f7cc-4543-8650-6c03f3764778-nova-metadata-tls-certs\") pod \"nova-metadata-0\" (UID: \"e77ccc47-f7cc-4543-8650-6c03f3764778\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:21:33 crc kubenswrapper[5052]: I1203 12:21:33.523773 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/e77ccc47-f7cc-4543-8650-6c03f3764778-config-data\") pod \"nova-metadata-0\" (UID: \"e77ccc47-f7cc-4543-8650-6c03f3764778\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:21:33 crc kubenswrapper[5052]: I1203 12:21:33.545298 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-ffbd4\" (UniqueName: \"kubernetes.io/projected/e77ccc47-f7cc-4543-8650-6c03f3764778-kube-api-access-ffbd4\") pod \"nova-metadata-0\" (UID: \"e77ccc47-f7cc-4543-8650-6c03f3764778\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:21:33 crc kubenswrapper[5052]: I1203 12:21:33.627268 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:21:33 crc kubenswrapper[5052]: I1203 12:21:33.669614 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:21:33 crc kubenswrapper[5052]: I1203 12:21:33.692179 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5bda226d-9d40-42d8-ac3e-11719e896649" path="/var/lib/kubelet/pods/5bda226d-9d40-42d8-ac3e-11719e896649/volumes" Dec 03 12:21:33 crc kubenswrapper[5052]: I1203 12:21:33.693171 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ce16d54b-ef04-429b-8795-2865395f7774" path="/var/lib/kubelet/pods/ce16d54b-ef04-429b-8795-2865395f7774/volumes" Dec 03 12:21:33 crc kubenswrapper[5052]: I1203 12:21:33.781233 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/keystone-796c4bc57b-zsjsh" Dec 03 12:21:33 crc kubenswrapper[5052]: I1203 12:21:33.860285 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/keystone-78dd58875-slm9r"] Dec 03 12:21:33 crc kubenswrapper[5052]: I1203 12:21:33.862735 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/keystone-78dd58875-slm9r" podUID="4107ef62-fa32-4d24-a7ad-3e2eb59884de" containerName="keystone-api" containerID="cri-o://e93ca0a9fbc6106e1c59be08c9244f1adaba71b37a9be94ca1173d57e88ad162" gracePeriod=30 Dec 03 12:21:34 crc kubenswrapper[5052]: I1203 12:21:34.204269 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:21:34 crc kubenswrapper[5052]: I1203 12:21:34.297757 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:21:34 crc kubenswrapper[5052]: W1203 12:21:34.298928 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode77ccc47_f7cc_4543_8650_6c03f3764778.slice/crio-b8892b7fd9cdbc94bbf9610ada60454ec616f6a3093334f07463f58c6e00620d WatchSource:0}: Error finding container b8892b7fd9cdbc94bbf9610ada60454ec616f6a3093334f07463f58c6e00620d: Status 404 returned error can't find the container with id b8892b7fd9cdbc94bbf9610ada60454ec616f6a3093334f07463f58c6e00620d Dec 03 12:21:34 crc kubenswrapper[5052]: E1203 12:21:34.987040 5052 cadvisor_stats_provider.go:516] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod27d134d0_a0b9_457b_8c8c_92cfb104cba1.slice/crio-45053dc66310b77c52cb144a03e6e50f334cdd69a31d2716d30553971203e459.scope\": RecentStats: unable to find data in memory cache]" Dec 03 12:21:35 crc kubenswrapper[5052]: I1203 12:21:35.132716 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"cd50414b-e062-408f-9ce0-052489596b5b","Type":"ContainerStarted","Data":"6ea34547ae7e1c8cfc3acb2770d1230c67f6ec0e4c220fe214a5a88afa39571c"} Dec 03 12:21:35 crc kubenswrapper[5052]: I1203 12:21:35.132782 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"cd50414b-e062-408f-9ce0-052489596b5b","Type":"ContainerStarted","Data":"0cdc7315abd0e95c41207cdd0a71a4ba5caee6de1636c4f55eaac360444a3a2d"} Dec 03 12:21:35 crc kubenswrapper[5052]: I1203 12:21:35.132793 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"cd50414b-e062-408f-9ce0-052489596b5b","Type":"ContainerStarted","Data":"563788af792ed6e17d075863e6aa4a5ac24aa618e78a98ce12dcc0c54053afac"} Dec 03 12:21:35 crc kubenswrapper[5052]: I1203 12:21:35.150891 5052 generic.go:334] "Generic (PLEG): container finished" podID="779772f8-a0bd-491f-9153-33614d19e423" containerID="db51cc7950f2c5a12d1aa905e418ed04e400dcebb362b0f45a28636e8440122e" exitCode=1 Dec 03 12:21:35 crc kubenswrapper[5052]: I1203 12:21:35.150964 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"779772f8-a0bd-491f-9153-33614d19e423","Type":"ContainerDied","Data":"db51cc7950f2c5a12d1aa905e418ed04e400dcebb362b0f45a28636e8440122e"} Dec 03 12:21:35 crc kubenswrapper[5052]: I1203 12:21:35.151884 5052 scope.go:117] "RemoveContainer" containerID="db51cc7950f2c5a12d1aa905e418ed04e400dcebb362b0f45a28636e8440122e" Dec 03 12:21:35 crc kubenswrapper[5052]: I1203 12:21:35.153823 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"e77ccc47-f7cc-4543-8650-6c03f3764778","Type":"ContainerStarted","Data":"e56be8de24b8902adb455f81974dad4fb51a1e35d0517e98a360cf0479b92cca"} Dec 03 12:21:35 crc kubenswrapper[5052]: I1203 12:21:35.153859 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"e77ccc47-f7cc-4543-8650-6c03f3764778","Type":"ContainerStarted","Data":"b128132fcc73ebdf762c298ede4fe69e106df8231adcc8687c7d9484c3a8ec8e"} Dec 03 12:21:35 crc kubenswrapper[5052]: I1203 12:21:35.153872 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"e77ccc47-f7cc-4543-8650-6c03f3764778","Type":"ContainerStarted","Data":"b8892b7fd9cdbc94bbf9610ada60454ec616f6a3093334f07463f58c6e00620d"} Dec 03 12:21:35 crc kubenswrapper[5052]: I1203 12:21:35.156390 5052 generic.go:334] "Generic (PLEG): container finished" podID="27d134d0-a0b9-457b-8c8c-92cfb104cba1" containerID="45053dc66310b77c52cb144a03e6e50f334cdd69a31d2716d30553971203e459" exitCode=0 Dec 03 12:21:35 crc kubenswrapper[5052]: I1203 12:21:35.156420 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-cell-mapping-qr65w" event={"ID":"27d134d0-a0b9-457b-8c8c-92cfb104cba1","Type":"ContainerDied","Data":"45053dc66310b77c52cb144a03e6e50f334cdd69a31d2716d30553971203e459"} Dec 03 12:21:35 crc kubenswrapper[5052]: I1203 12:21:35.183348 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-api-0" podStartSLOduration=2.183328498 podStartE2EDuration="2.183328498s" podCreationTimestamp="2025-12-03 12:21:33 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:21:35.176452002 +0000 UTC m=+3685.764693766" watchObservedRunningTime="2025-12-03 12:21:35.183328498 +0000 UTC m=+3685.771570262" Dec 03 12:21:35 crc kubenswrapper[5052]: I1203 12:21:35.248311 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-metadata-0" podStartSLOduration=2.248287612 podStartE2EDuration="2.248287612s" podCreationTimestamp="2025-12-03 12:21:33 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:21:35.230595837 +0000 UTC m=+3685.818837621" watchObservedRunningTime="2025-12-03 12:21:35.248287612 +0000 UTC m=+3685.836529376" Dec 03 12:21:35 crc kubenswrapper[5052]: I1203 12:21:35.476811 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:21:36 crc kubenswrapper[5052]: I1203 12:21:36.169639 5052 generic.go:334] "Generic (PLEG): container finished" podID="0bbf844e-3419-4f88-b375-ca77df3007af" containerID="d090e68312aedebe296e1c2b129632e2019e3685d8daf2a12e3a6ceb18d11578" exitCode=0 Dec 03 12:21:36 crc kubenswrapper[5052]: I1203 12:21:36.169721 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-cell-mapping-6dkwd" event={"ID":"0bbf844e-3419-4f88-b375-ca77df3007af","Type":"ContainerDied","Data":"d090e68312aedebe296e1c2b129632e2019e3685d8daf2a12e3a6ceb18d11578"} Dec 03 12:21:36 crc kubenswrapper[5052]: I1203 12:21:36.173479 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"779772f8-a0bd-491f-9153-33614d19e423","Type":"ContainerStarted","Data":"0e513984b9212a78bfe53fea749394bea6e44fa33a9bc3a945f96fe05465097a"} Dec 03 12:21:36 crc kubenswrapper[5052]: I1203 12:21:36.523449 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-cell-mapping-qr65w" Dec 03 12:21:36 crc kubenswrapper[5052]: I1203 12:21:36.598650 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/27d134d0-a0b9-457b-8c8c-92cfb104cba1-combined-ca-bundle\") pod \"27d134d0-a0b9-457b-8c8c-92cfb104cba1\" (UID: \"27d134d0-a0b9-457b-8c8c-92cfb104cba1\") " Dec 03 12:21:36 crc kubenswrapper[5052]: I1203 12:21:36.598721 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/27d134d0-a0b9-457b-8c8c-92cfb104cba1-config-data\") pod \"27d134d0-a0b9-457b-8c8c-92cfb104cba1\" (UID: \"27d134d0-a0b9-457b-8c8c-92cfb104cba1\") " Dec 03 12:21:36 crc kubenswrapper[5052]: I1203 12:21:36.598981 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rr4br\" (UniqueName: \"kubernetes.io/projected/27d134d0-a0b9-457b-8c8c-92cfb104cba1-kube-api-access-rr4br\") pod \"27d134d0-a0b9-457b-8c8c-92cfb104cba1\" (UID: \"27d134d0-a0b9-457b-8c8c-92cfb104cba1\") " Dec 03 12:21:36 crc kubenswrapper[5052]: I1203 12:21:36.599077 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/27d134d0-a0b9-457b-8c8c-92cfb104cba1-scripts\") pod \"27d134d0-a0b9-457b-8c8c-92cfb104cba1\" (UID: \"27d134d0-a0b9-457b-8c8c-92cfb104cba1\") " Dec 03 12:21:36 crc kubenswrapper[5052]: I1203 12:21:36.616840 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/27d134d0-a0b9-457b-8c8c-92cfb104cba1-kube-api-access-rr4br" (OuterVolumeSpecName: "kube-api-access-rr4br") pod "27d134d0-a0b9-457b-8c8c-92cfb104cba1" (UID: "27d134d0-a0b9-457b-8c8c-92cfb104cba1"). InnerVolumeSpecName "kube-api-access-rr4br". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:21:36 crc kubenswrapper[5052]: I1203 12:21:36.617757 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/27d134d0-a0b9-457b-8c8c-92cfb104cba1-scripts" (OuterVolumeSpecName: "scripts") pod "27d134d0-a0b9-457b-8c8c-92cfb104cba1" (UID: "27d134d0-a0b9-457b-8c8c-92cfb104cba1"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:36 crc kubenswrapper[5052]: I1203 12:21:36.629782 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/27d134d0-a0b9-457b-8c8c-92cfb104cba1-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "27d134d0-a0b9-457b-8c8c-92cfb104cba1" (UID: "27d134d0-a0b9-457b-8c8c-92cfb104cba1"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:36 crc kubenswrapper[5052]: I1203 12:21:36.641171 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/27d134d0-a0b9-457b-8c8c-92cfb104cba1-config-data" (OuterVolumeSpecName: "config-data") pod "27d134d0-a0b9-457b-8c8c-92cfb104cba1" (UID: "27d134d0-a0b9-457b-8c8c-92cfb104cba1"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:36 crc kubenswrapper[5052]: I1203 12:21:36.673275 5052 scope.go:117] "RemoveContainer" containerID="09b4a0072741f60499d24f40cc4ff3406d5d22671ced431363ab865172786300" Dec 03 12:21:36 crc kubenswrapper[5052]: E1203 12:21:36.673698 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 12:21:36 crc kubenswrapper[5052]: I1203 12:21:36.701322 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-rr4br\" (UniqueName: \"kubernetes.io/projected/27d134d0-a0b9-457b-8c8c-92cfb104cba1-kube-api-access-rr4br\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:36 crc kubenswrapper[5052]: I1203 12:21:36.701367 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/27d134d0-a0b9-457b-8c8c-92cfb104cba1-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:36 crc kubenswrapper[5052]: I1203 12:21:36.701382 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/27d134d0-a0b9-457b-8c8c-92cfb104cba1-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:36 crc kubenswrapper[5052]: I1203 12:21:36.701394 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/27d134d0-a0b9-457b-8c8c-92cfb104cba1-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:37 crc kubenswrapper[5052]: I1203 12:21:37.189857 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-cell-mapping-qr65w" event={"ID":"27d134d0-a0b9-457b-8c8c-92cfb104cba1","Type":"ContainerDied","Data":"1343620f921627b260159205a761d6971c73d314e261ab38caceac9aedbe0c86"} Dec 03 12:21:37 crc kubenswrapper[5052]: I1203 12:21:37.189915 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1343620f921627b260159205a761d6971c73d314e261ab38caceac9aedbe0c86" Dec 03 12:21:37 crc kubenswrapper[5052]: I1203 12:21:37.189881 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-cell-mapping-qr65w" Dec 03 12:21:37 crc kubenswrapper[5052]: I1203 12:21:37.404469 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:21:37 crc kubenswrapper[5052]: I1203 12:21:37.405005 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-api-0" podUID="cd50414b-e062-408f-9ce0-052489596b5b" containerName="nova-api-log" containerID="cri-o://0cdc7315abd0e95c41207cdd0a71a4ba5caee6de1636c4f55eaac360444a3a2d" gracePeriod=30 Dec 03 12:21:37 crc kubenswrapper[5052]: I1203 12:21:37.405112 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-api-0" podUID="cd50414b-e062-408f-9ce0-052489596b5b" containerName="nova-api-api" containerID="cri-o://6ea34547ae7e1c8cfc3acb2770d1230c67f6ec0e4c220fe214a5a88afa39571c" gracePeriod=30 Dec 03 12:21:37 crc kubenswrapper[5052]: I1203 12:21:37.421118 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:21:37 crc kubenswrapper[5052]: I1203 12:21:37.479182 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:21:37 crc kubenswrapper[5052]: I1203 12:21:37.479447 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-metadata-0" podUID="e77ccc47-f7cc-4543-8650-6c03f3764778" containerName="nova-metadata-log" containerID="cri-o://b128132fcc73ebdf762c298ede4fe69e106df8231adcc8687c7d9484c3a8ec8e" gracePeriod=30 Dec 03 12:21:37 crc kubenswrapper[5052]: I1203 12:21:37.479992 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-metadata-0" podUID="e77ccc47-f7cc-4543-8650-6c03f3764778" containerName="nova-metadata-metadata" containerID="cri-o://e56be8de24b8902adb455f81974dad4fb51a1e35d0517e98a360cf0479b92cca" gracePeriod=30 Dec 03 12:21:37 crc kubenswrapper[5052]: I1203 12:21:37.618743 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/swift-proxy-6b9f9968d-5zjp2"] Dec 03 12:21:37 crc kubenswrapper[5052]: E1203 12:21:37.619236 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="27d134d0-a0b9-457b-8c8c-92cfb104cba1" containerName="nova-manage" Dec 03 12:21:37 crc kubenswrapper[5052]: I1203 12:21:37.619255 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="27d134d0-a0b9-457b-8c8c-92cfb104cba1" containerName="nova-manage" Dec 03 12:21:37 crc kubenswrapper[5052]: I1203 12:21:37.619487 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="27d134d0-a0b9-457b-8c8c-92cfb104cba1" containerName="nova-manage" Dec 03 12:21:37 crc kubenswrapper[5052]: I1203 12:21:37.625123 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-proxy-6b9f9968d-5zjp2" Dec 03 12:21:37 crc kubenswrapper[5052]: I1203 12:21:37.643232 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/swift-proxy-6b9f9968d-5zjp2"] Dec 03 12:21:37 crc kubenswrapper[5052]: I1203 12:21:37.675895 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-cell-mapping-6dkwd" Dec 03 12:21:37 crc kubenswrapper[5052]: I1203 12:21:37.724308 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/0bbf844e-3419-4f88-b375-ca77df3007af-scripts\") pod \"0bbf844e-3419-4f88-b375-ca77df3007af\" (UID: \"0bbf844e-3419-4f88-b375-ca77df3007af\") " Dec 03 12:21:37 crc kubenswrapper[5052]: I1203 12:21:37.724386 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xcscw\" (UniqueName: \"kubernetes.io/projected/0bbf844e-3419-4f88-b375-ca77df3007af-kube-api-access-xcscw\") pod \"0bbf844e-3419-4f88-b375-ca77df3007af\" (UID: \"0bbf844e-3419-4f88-b375-ca77df3007af\") " Dec 03 12:21:37 crc kubenswrapper[5052]: I1203 12:21:37.724465 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/0bbf844e-3419-4f88-b375-ca77df3007af-config-data\") pod \"0bbf844e-3419-4f88-b375-ca77df3007af\" (UID: \"0bbf844e-3419-4f88-b375-ca77df3007af\") " Dec 03 12:21:37 crc kubenswrapper[5052]: I1203 12:21:37.724530 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0bbf844e-3419-4f88-b375-ca77df3007af-combined-ca-bundle\") pod \"0bbf844e-3419-4f88-b375-ca77df3007af\" (UID: \"0bbf844e-3419-4f88-b375-ca77df3007af\") " Dec 03 12:21:37 crc kubenswrapper[5052]: I1203 12:21:37.724817 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/fb73cc9a-89cc-4cb8-87ce-2731e5dab0f7-internal-tls-certs\") pod \"swift-proxy-6b9f9968d-5zjp2\" (UID: \"fb73cc9a-89cc-4cb8-87ce-2731e5dab0f7\") " pod="openstack-kuttl-tests/swift-proxy-6b9f9968d-5zjp2" Dec 03 12:21:37 crc kubenswrapper[5052]: I1203 12:21:37.724860 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/fb73cc9a-89cc-4cb8-87ce-2731e5dab0f7-run-httpd\") pod \"swift-proxy-6b9f9968d-5zjp2\" (UID: \"fb73cc9a-89cc-4cb8-87ce-2731e5dab0f7\") " pod="openstack-kuttl-tests/swift-proxy-6b9f9968d-5zjp2" Dec 03 12:21:37 crc kubenswrapper[5052]: I1203 12:21:37.724877 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/fb73cc9a-89cc-4cb8-87ce-2731e5dab0f7-public-tls-certs\") pod \"swift-proxy-6b9f9968d-5zjp2\" (UID: \"fb73cc9a-89cc-4cb8-87ce-2731e5dab0f7\") " pod="openstack-kuttl-tests/swift-proxy-6b9f9968d-5zjp2" Dec 03 12:21:37 crc kubenswrapper[5052]: I1203 12:21:37.724904 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gppn6\" (UniqueName: \"kubernetes.io/projected/fb73cc9a-89cc-4cb8-87ce-2731e5dab0f7-kube-api-access-gppn6\") pod \"swift-proxy-6b9f9968d-5zjp2\" (UID: \"fb73cc9a-89cc-4cb8-87ce-2731e5dab0f7\") " pod="openstack-kuttl-tests/swift-proxy-6b9f9968d-5zjp2" Dec 03 12:21:37 crc kubenswrapper[5052]: I1203 12:21:37.724933 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/fb73cc9a-89cc-4cb8-87ce-2731e5dab0f7-log-httpd\") pod \"swift-proxy-6b9f9968d-5zjp2\" (UID: \"fb73cc9a-89cc-4cb8-87ce-2731e5dab0f7\") " pod="openstack-kuttl-tests/swift-proxy-6b9f9968d-5zjp2" Dec 03 12:21:37 crc kubenswrapper[5052]: I1203 12:21:37.724966 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/fb73cc9a-89cc-4cb8-87ce-2731e5dab0f7-combined-ca-bundle\") pod \"swift-proxy-6b9f9968d-5zjp2\" (UID: \"fb73cc9a-89cc-4cb8-87ce-2731e5dab0f7\") " pod="openstack-kuttl-tests/swift-proxy-6b9f9968d-5zjp2" Dec 03 12:21:37 crc kubenswrapper[5052]: I1203 12:21:37.725057 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/fb73cc9a-89cc-4cb8-87ce-2731e5dab0f7-config-data\") pod \"swift-proxy-6b9f9968d-5zjp2\" (UID: \"fb73cc9a-89cc-4cb8-87ce-2731e5dab0f7\") " pod="openstack-kuttl-tests/swift-proxy-6b9f9968d-5zjp2" Dec 03 12:21:37 crc kubenswrapper[5052]: I1203 12:21:37.725123 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/fb73cc9a-89cc-4cb8-87ce-2731e5dab0f7-etc-swift\") pod \"swift-proxy-6b9f9968d-5zjp2\" (UID: \"fb73cc9a-89cc-4cb8-87ce-2731e5dab0f7\") " pod="openstack-kuttl-tests/swift-proxy-6b9f9968d-5zjp2" Dec 03 12:21:37 crc kubenswrapper[5052]: I1203 12:21:37.731155 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0bbf844e-3419-4f88-b375-ca77df3007af-scripts" (OuterVolumeSpecName: "scripts") pod "0bbf844e-3419-4f88-b375-ca77df3007af" (UID: "0bbf844e-3419-4f88-b375-ca77df3007af"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:37 crc kubenswrapper[5052]: I1203 12:21:37.755280 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0bbf844e-3419-4f88-b375-ca77df3007af-kube-api-access-xcscw" (OuterVolumeSpecName: "kube-api-access-xcscw") pod "0bbf844e-3419-4f88-b375-ca77df3007af" (UID: "0bbf844e-3419-4f88-b375-ca77df3007af"). InnerVolumeSpecName "kube-api-access-xcscw". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:21:37 crc kubenswrapper[5052]: I1203 12:21:37.803823 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0bbf844e-3419-4f88-b375-ca77df3007af-config-data" (OuterVolumeSpecName: "config-data") pod "0bbf844e-3419-4f88-b375-ca77df3007af" (UID: "0bbf844e-3419-4f88-b375-ca77df3007af"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:37 crc kubenswrapper[5052]: I1203 12:21:37.807730 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0bbf844e-3419-4f88-b375-ca77df3007af-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "0bbf844e-3419-4f88-b375-ca77df3007af" (UID: "0bbf844e-3419-4f88-b375-ca77df3007af"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:37 crc kubenswrapper[5052]: I1203 12:21:37.826929 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/fb73cc9a-89cc-4cb8-87ce-2731e5dab0f7-config-data\") pod \"swift-proxy-6b9f9968d-5zjp2\" (UID: \"fb73cc9a-89cc-4cb8-87ce-2731e5dab0f7\") " pod="openstack-kuttl-tests/swift-proxy-6b9f9968d-5zjp2" Dec 03 12:21:37 crc kubenswrapper[5052]: I1203 12:21:37.827300 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/fb73cc9a-89cc-4cb8-87ce-2731e5dab0f7-etc-swift\") pod \"swift-proxy-6b9f9968d-5zjp2\" (UID: \"fb73cc9a-89cc-4cb8-87ce-2731e5dab0f7\") " pod="openstack-kuttl-tests/swift-proxy-6b9f9968d-5zjp2" Dec 03 12:21:37 crc kubenswrapper[5052]: I1203 12:21:37.827351 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/fb73cc9a-89cc-4cb8-87ce-2731e5dab0f7-internal-tls-certs\") pod \"swift-proxy-6b9f9968d-5zjp2\" (UID: \"fb73cc9a-89cc-4cb8-87ce-2731e5dab0f7\") " pod="openstack-kuttl-tests/swift-proxy-6b9f9968d-5zjp2" Dec 03 12:21:37 crc kubenswrapper[5052]: I1203 12:21:37.827380 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/fb73cc9a-89cc-4cb8-87ce-2731e5dab0f7-run-httpd\") pod \"swift-proxy-6b9f9968d-5zjp2\" (UID: \"fb73cc9a-89cc-4cb8-87ce-2731e5dab0f7\") " pod="openstack-kuttl-tests/swift-proxy-6b9f9968d-5zjp2" Dec 03 12:21:37 crc kubenswrapper[5052]: I1203 12:21:37.827398 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/fb73cc9a-89cc-4cb8-87ce-2731e5dab0f7-public-tls-certs\") pod \"swift-proxy-6b9f9968d-5zjp2\" (UID: \"fb73cc9a-89cc-4cb8-87ce-2731e5dab0f7\") " pod="openstack-kuttl-tests/swift-proxy-6b9f9968d-5zjp2" Dec 03 12:21:37 crc kubenswrapper[5052]: I1203 12:21:37.827418 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-gppn6\" (UniqueName: \"kubernetes.io/projected/fb73cc9a-89cc-4cb8-87ce-2731e5dab0f7-kube-api-access-gppn6\") pod \"swift-proxy-6b9f9968d-5zjp2\" (UID: \"fb73cc9a-89cc-4cb8-87ce-2731e5dab0f7\") " pod="openstack-kuttl-tests/swift-proxy-6b9f9968d-5zjp2" Dec 03 12:21:37 crc kubenswrapper[5052]: I1203 12:21:37.827441 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/fb73cc9a-89cc-4cb8-87ce-2731e5dab0f7-log-httpd\") pod \"swift-proxy-6b9f9968d-5zjp2\" (UID: \"fb73cc9a-89cc-4cb8-87ce-2731e5dab0f7\") " pod="openstack-kuttl-tests/swift-proxy-6b9f9968d-5zjp2" Dec 03 12:21:37 crc kubenswrapper[5052]: I1203 12:21:37.827466 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/fb73cc9a-89cc-4cb8-87ce-2731e5dab0f7-combined-ca-bundle\") pod \"swift-proxy-6b9f9968d-5zjp2\" (UID: \"fb73cc9a-89cc-4cb8-87ce-2731e5dab0f7\") " pod="openstack-kuttl-tests/swift-proxy-6b9f9968d-5zjp2" Dec 03 12:21:37 crc kubenswrapper[5052]: I1203 12:21:37.827689 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-xcscw\" (UniqueName: \"kubernetes.io/projected/0bbf844e-3419-4f88-b375-ca77df3007af-kube-api-access-xcscw\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:37 crc kubenswrapper[5052]: I1203 12:21:37.827709 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/0bbf844e-3419-4f88-b375-ca77df3007af-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:37 crc kubenswrapper[5052]: I1203 12:21:37.827718 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0bbf844e-3419-4f88-b375-ca77df3007af-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:37 crc kubenswrapper[5052]: I1203 12:21:37.827728 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/0bbf844e-3419-4f88-b375-ca77df3007af-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:37 crc kubenswrapper[5052]: I1203 12:21:37.828312 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/fb73cc9a-89cc-4cb8-87ce-2731e5dab0f7-log-httpd\") pod \"swift-proxy-6b9f9968d-5zjp2\" (UID: \"fb73cc9a-89cc-4cb8-87ce-2731e5dab0f7\") " pod="openstack-kuttl-tests/swift-proxy-6b9f9968d-5zjp2" Dec 03 12:21:37 crc kubenswrapper[5052]: I1203 12:21:37.828601 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/fb73cc9a-89cc-4cb8-87ce-2731e5dab0f7-run-httpd\") pod \"swift-proxy-6b9f9968d-5zjp2\" (UID: \"fb73cc9a-89cc-4cb8-87ce-2731e5dab0f7\") " pod="openstack-kuttl-tests/swift-proxy-6b9f9968d-5zjp2" Dec 03 12:21:37 crc kubenswrapper[5052]: I1203 12:21:37.831440 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/fb73cc9a-89cc-4cb8-87ce-2731e5dab0f7-internal-tls-certs\") pod \"swift-proxy-6b9f9968d-5zjp2\" (UID: \"fb73cc9a-89cc-4cb8-87ce-2731e5dab0f7\") " pod="openstack-kuttl-tests/swift-proxy-6b9f9968d-5zjp2" Dec 03 12:21:37 crc kubenswrapper[5052]: I1203 12:21:37.831686 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/fb73cc9a-89cc-4cb8-87ce-2731e5dab0f7-combined-ca-bundle\") pod \"swift-proxy-6b9f9968d-5zjp2\" (UID: \"fb73cc9a-89cc-4cb8-87ce-2731e5dab0f7\") " pod="openstack-kuttl-tests/swift-proxy-6b9f9968d-5zjp2" Dec 03 12:21:37 crc kubenswrapper[5052]: I1203 12:21:37.832535 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/fb73cc9a-89cc-4cb8-87ce-2731e5dab0f7-config-data\") pod \"swift-proxy-6b9f9968d-5zjp2\" (UID: \"fb73cc9a-89cc-4cb8-87ce-2731e5dab0f7\") " pod="openstack-kuttl-tests/swift-proxy-6b9f9968d-5zjp2" Dec 03 12:21:37 crc kubenswrapper[5052]: I1203 12:21:37.832658 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/fb73cc9a-89cc-4cb8-87ce-2731e5dab0f7-etc-swift\") pod \"swift-proxy-6b9f9968d-5zjp2\" (UID: \"fb73cc9a-89cc-4cb8-87ce-2731e5dab0f7\") " pod="openstack-kuttl-tests/swift-proxy-6b9f9968d-5zjp2" Dec 03 12:21:37 crc kubenswrapper[5052]: I1203 12:21:37.833153 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/fb73cc9a-89cc-4cb8-87ce-2731e5dab0f7-public-tls-certs\") pod \"swift-proxy-6b9f9968d-5zjp2\" (UID: \"fb73cc9a-89cc-4cb8-87ce-2731e5dab0f7\") " pod="openstack-kuttl-tests/swift-proxy-6b9f9968d-5zjp2" Dec 03 12:21:37 crc kubenswrapper[5052]: I1203 12:21:37.853682 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-gppn6\" (UniqueName: \"kubernetes.io/projected/fb73cc9a-89cc-4cb8-87ce-2731e5dab0f7-kube-api-access-gppn6\") pod \"swift-proxy-6b9f9968d-5zjp2\" (UID: \"fb73cc9a-89cc-4cb8-87ce-2731e5dab0f7\") " pod="openstack-kuttl-tests/swift-proxy-6b9f9968d-5zjp2" Dec 03 12:21:37 crc kubenswrapper[5052]: I1203 12:21:37.967212 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-78dd58875-slm9r" Dec 03 12:21:37 crc kubenswrapper[5052]: I1203 12:21:37.969275 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-proxy-6b9f9968d-5zjp2" Dec 03 12:21:37 crc kubenswrapper[5052]: I1203 12:21:37.977329 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/openstackclient"] Dec 03 12:21:37 crc kubenswrapper[5052]: I1203 12:21:37.977841 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/openstackclient" podUID="357bc965-8633-4d67-992e-baa770a0defd" containerName="openstackclient" containerID="cri-o://97eb19cad24849878c7614c23d646abe200c279ae260c2f8f7bae07110bd2a2b" gracePeriod=2 Dec 03 12:21:37 crc kubenswrapper[5052]: I1203 12:21:37.991919 5052 status_manager.go:861] "Pod was deleted and then recreated, skipping status update" pod="openstack-kuttl-tests/openstackclient" oldPodUID="357bc965-8633-4d67-992e-baa770a0defd" podUID="892f0ab2-8602-4485-a11c-5171252e2a7b" Dec 03 12:21:38 crc kubenswrapper[5052]: I1203 12:21:38.003753 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/openstackclient"] Dec 03 12:21:38 crc kubenswrapper[5052]: I1203 12:21:38.023115 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/openstackclient"] Dec 03 12:21:38 crc kubenswrapper[5052]: E1203 12:21:38.023574 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="357bc965-8633-4d67-992e-baa770a0defd" containerName="openstackclient" Dec 03 12:21:38 crc kubenswrapper[5052]: I1203 12:21:38.023595 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="357bc965-8633-4d67-992e-baa770a0defd" containerName="openstackclient" Dec 03 12:21:38 crc kubenswrapper[5052]: E1203 12:21:38.023636 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="4107ef62-fa32-4d24-a7ad-3e2eb59884de" containerName="keystone-api" Dec 03 12:21:38 crc kubenswrapper[5052]: I1203 12:21:38.023644 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="4107ef62-fa32-4d24-a7ad-3e2eb59884de" containerName="keystone-api" Dec 03 12:21:38 crc kubenswrapper[5052]: E1203 12:21:38.023666 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="0bbf844e-3419-4f88-b375-ca77df3007af" containerName="nova-manage" Dec 03 12:21:38 crc kubenswrapper[5052]: I1203 12:21:38.023674 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="0bbf844e-3419-4f88-b375-ca77df3007af" containerName="nova-manage" Dec 03 12:21:38 crc kubenswrapper[5052]: I1203 12:21:38.023890 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="4107ef62-fa32-4d24-a7ad-3e2eb59884de" containerName="keystone-api" Dec 03 12:21:38 crc kubenswrapper[5052]: I1203 12:21:38.023910 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="0bbf844e-3419-4f88-b375-ca77df3007af" containerName="nova-manage" Dec 03 12:21:38 crc kubenswrapper[5052]: I1203 12:21:38.023926 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="357bc965-8633-4d67-992e-baa770a0defd" containerName="openstackclient" Dec 03 12:21:38 crc kubenswrapper[5052]: I1203 12:21:38.024708 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstackclient" Dec 03 12:21:38 crc kubenswrapper[5052]: I1203 12:21:38.030894 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/4107ef62-fa32-4d24-a7ad-3e2eb59884de-credential-keys\") pod \"4107ef62-fa32-4d24-a7ad-3e2eb59884de\" (UID: \"4107ef62-fa32-4d24-a7ad-3e2eb59884de\") " Dec 03 12:21:38 crc kubenswrapper[5052]: I1203 12:21:38.030985 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pl8gv\" (UniqueName: \"kubernetes.io/projected/4107ef62-fa32-4d24-a7ad-3e2eb59884de-kube-api-access-pl8gv\") pod \"4107ef62-fa32-4d24-a7ad-3e2eb59884de\" (UID: \"4107ef62-fa32-4d24-a7ad-3e2eb59884de\") " Dec 03 12:21:38 crc kubenswrapper[5052]: I1203 12:21:38.031095 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/4107ef62-fa32-4d24-a7ad-3e2eb59884de-scripts\") pod \"4107ef62-fa32-4d24-a7ad-3e2eb59884de\" (UID: \"4107ef62-fa32-4d24-a7ad-3e2eb59884de\") " Dec 03 12:21:38 crc kubenswrapper[5052]: I1203 12:21:38.031231 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/4107ef62-fa32-4d24-a7ad-3e2eb59884de-internal-tls-certs\") pod \"4107ef62-fa32-4d24-a7ad-3e2eb59884de\" (UID: \"4107ef62-fa32-4d24-a7ad-3e2eb59884de\") " Dec 03 12:21:38 crc kubenswrapper[5052]: I1203 12:21:38.031368 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/4107ef62-fa32-4d24-a7ad-3e2eb59884de-fernet-keys\") pod \"4107ef62-fa32-4d24-a7ad-3e2eb59884de\" (UID: \"4107ef62-fa32-4d24-a7ad-3e2eb59884de\") " Dec 03 12:21:38 crc kubenswrapper[5052]: I1203 12:21:38.031473 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4107ef62-fa32-4d24-a7ad-3e2eb59884de-combined-ca-bundle\") pod \"4107ef62-fa32-4d24-a7ad-3e2eb59884de\" (UID: \"4107ef62-fa32-4d24-a7ad-3e2eb59884de\") " Dec 03 12:21:38 crc kubenswrapper[5052]: I1203 12:21:38.031579 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/4107ef62-fa32-4d24-a7ad-3e2eb59884de-public-tls-certs\") pod \"4107ef62-fa32-4d24-a7ad-3e2eb59884de\" (UID: \"4107ef62-fa32-4d24-a7ad-3e2eb59884de\") " Dec 03 12:21:38 crc kubenswrapper[5052]: I1203 12:21:38.031621 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/4107ef62-fa32-4d24-a7ad-3e2eb59884de-config-data\") pod \"4107ef62-fa32-4d24-a7ad-3e2eb59884de\" (UID: \"4107ef62-fa32-4d24-a7ad-3e2eb59884de\") " Dec 03 12:21:38 crc kubenswrapper[5052]: I1203 12:21:38.045324 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4107ef62-fa32-4d24-a7ad-3e2eb59884de-scripts" (OuterVolumeSpecName: "scripts") pod "4107ef62-fa32-4d24-a7ad-3e2eb59884de" (UID: "4107ef62-fa32-4d24-a7ad-3e2eb59884de"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:38 crc kubenswrapper[5052]: I1203 12:21:38.047270 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4107ef62-fa32-4d24-a7ad-3e2eb59884de-fernet-keys" (OuterVolumeSpecName: "fernet-keys") pod "4107ef62-fa32-4d24-a7ad-3e2eb59884de" (UID: "4107ef62-fa32-4d24-a7ad-3e2eb59884de"). InnerVolumeSpecName "fernet-keys". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:38 crc kubenswrapper[5052]: I1203 12:21:38.048719 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4107ef62-fa32-4d24-a7ad-3e2eb59884de-credential-keys" (OuterVolumeSpecName: "credential-keys") pod "4107ef62-fa32-4d24-a7ad-3e2eb59884de" (UID: "4107ef62-fa32-4d24-a7ad-3e2eb59884de"). InnerVolumeSpecName "credential-keys". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:38 crc kubenswrapper[5052]: I1203 12:21:38.058721 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4107ef62-fa32-4d24-a7ad-3e2eb59884de-kube-api-access-pl8gv" (OuterVolumeSpecName: "kube-api-access-pl8gv") pod "4107ef62-fa32-4d24-a7ad-3e2eb59884de" (UID: "4107ef62-fa32-4d24-a7ad-3e2eb59884de"). InnerVolumeSpecName "kube-api-access-pl8gv". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:21:38 crc kubenswrapper[5052]: I1203 12:21:38.061277 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/openstackclient"] Dec 03 12:21:38 crc kubenswrapper[5052]: I1203 12:21:38.063913 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:21:38 crc kubenswrapper[5052]: I1203 12:21:38.081529 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4107ef62-fa32-4d24-a7ad-3e2eb59884de-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "4107ef62-fa32-4d24-a7ad-3e2eb59884de" (UID: "4107ef62-fa32-4d24-a7ad-3e2eb59884de"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:38 crc kubenswrapper[5052]: I1203 12:21:38.105405 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4107ef62-fa32-4d24-a7ad-3e2eb59884de-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "4107ef62-fa32-4d24-a7ad-3e2eb59884de" (UID: "4107ef62-fa32-4d24-a7ad-3e2eb59884de"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:38 crc kubenswrapper[5052]: I1203 12:21:38.109985 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4107ef62-fa32-4d24-a7ad-3e2eb59884de-config-data" (OuterVolumeSpecName: "config-data") pod "4107ef62-fa32-4d24-a7ad-3e2eb59884de" (UID: "4107ef62-fa32-4d24-a7ad-3e2eb59884de"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:38 crc kubenswrapper[5052]: I1203 12:21:38.132940 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4107ef62-fa32-4d24-a7ad-3e2eb59884de-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "4107ef62-fa32-4d24-a7ad-3e2eb59884de" (UID: "4107ef62-fa32-4d24-a7ad-3e2eb59884de"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:38 crc kubenswrapper[5052]: I1203 12:21:38.133849 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/e77ccc47-f7cc-4543-8650-6c03f3764778-config-data\") pod \"e77ccc47-f7cc-4543-8650-6c03f3764778\" (UID: \"e77ccc47-f7cc-4543-8650-6c03f3764778\") " Dec 03 12:21:38 crc kubenswrapper[5052]: I1203 12:21:38.134111 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/e77ccc47-f7cc-4543-8650-6c03f3764778-logs\") pod \"e77ccc47-f7cc-4543-8650-6c03f3764778\" (UID: \"e77ccc47-f7cc-4543-8650-6c03f3764778\") " Dec 03 12:21:38 crc kubenswrapper[5052]: I1203 12:21:38.134246 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ffbd4\" (UniqueName: \"kubernetes.io/projected/e77ccc47-f7cc-4543-8650-6c03f3764778-kube-api-access-ffbd4\") pod \"e77ccc47-f7cc-4543-8650-6c03f3764778\" (UID: \"e77ccc47-f7cc-4543-8650-6c03f3764778\") " Dec 03 12:21:38 crc kubenswrapper[5052]: I1203 12:21:38.134353 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e77ccc47-f7cc-4543-8650-6c03f3764778-combined-ca-bundle\") pod \"e77ccc47-f7cc-4543-8650-6c03f3764778\" (UID: \"e77ccc47-f7cc-4543-8650-6c03f3764778\") " Dec 03 12:21:38 crc kubenswrapper[5052]: I1203 12:21:38.134396 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/4107ef62-fa32-4d24-a7ad-3e2eb59884de-internal-tls-certs\") pod \"4107ef62-fa32-4d24-a7ad-3e2eb59884de\" (UID: \"4107ef62-fa32-4d24-a7ad-3e2eb59884de\") " Dec 03 12:21:38 crc kubenswrapper[5052]: I1203 12:21:38.134442 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/e77ccc47-f7cc-4543-8650-6c03f3764778-nova-metadata-tls-certs\") pod \"e77ccc47-f7cc-4543-8650-6c03f3764778\" (UID: \"e77ccc47-f7cc-4543-8650-6c03f3764778\") " Dec 03 12:21:38 crc kubenswrapper[5052]: I1203 12:21:38.134536 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e77ccc47-f7cc-4543-8650-6c03f3764778-logs" (OuterVolumeSpecName: "logs") pod "e77ccc47-f7cc-4543-8650-6c03f3764778" (UID: "e77ccc47-f7cc-4543-8650-6c03f3764778"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:21:38 crc kubenswrapper[5052]: W1203 12:21:38.134695 5052 empty_dir.go:500] Warning: Unmount skipped because path does not exist: /var/lib/kubelet/pods/4107ef62-fa32-4d24-a7ad-3e2eb59884de/volumes/kubernetes.io~secret/internal-tls-certs Dec 03 12:21:38 crc kubenswrapper[5052]: I1203 12:21:38.134719 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4107ef62-fa32-4d24-a7ad-3e2eb59884de-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "4107ef62-fa32-4d24-a7ad-3e2eb59884de" (UID: "4107ef62-fa32-4d24-a7ad-3e2eb59884de"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:38 crc kubenswrapper[5052]: I1203 12:21:38.135235 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openstack-config\" (UniqueName: \"kubernetes.io/configmap/892f0ab2-8602-4485-a11c-5171252e2a7b-openstack-config\") pod \"openstackclient\" (UID: \"892f0ab2-8602-4485-a11c-5171252e2a7b\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 12:21:38 crc kubenswrapper[5052]: I1203 12:21:38.135339 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openstack-config-secret\" (UniqueName: \"kubernetes.io/secret/892f0ab2-8602-4485-a11c-5171252e2a7b-openstack-config-secret\") pod \"openstackclient\" (UID: \"892f0ab2-8602-4485-a11c-5171252e2a7b\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 12:21:38 crc kubenswrapper[5052]: I1203 12:21:38.135489 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/892f0ab2-8602-4485-a11c-5171252e2a7b-combined-ca-bundle\") pod \"openstackclient\" (UID: \"892f0ab2-8602-4485-a11c-5171252e2a7b\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 12:21:38 crc kubenswrapper[5052]: I1203 12:21:38.135652 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4xxw2\" (UniqueName: \"kubernetes.io/projected/892f0ab2-8602-4485-a11c-5171252e2a7b-kube-api-access-4xxw2\") pod \"openstackclient\" (UID: \"892f0ab2-8602-4485-a11c-5171252e2a7b\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 12:21:38 crc kubenswrapper[5052]: I1203 12:21:38.135761 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/4107ef62-fa32-4d24-a7ad-3e2eb59884de-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:38 crc kubenswrapper[5052]: I1203 12:21:38.135795 5052 reconciler_common.go:293] "Volume detached for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/4107ef62-fa32-4d24-a7ad-3e2eb59884de-fernet-keys\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:38 crc kubenswrapper[5052]: I1203 12:21:38.135812 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4107ef62-fa32-4d24-a7ad-3e2eb59884de-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:38 crc kubenswrapper[5052]: I1203 12:21:38.135825 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/4107ef62-fa32-4d24-a7ad-3e2eb59884de-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:38 crc kubenswrapper[5052]: I1203 12:21:38.135836 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/4107ef62-fa32-4d24-a7ad-3e2eb59884de-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:38 crc kubenswrapper[5052]: I1203 12:21:38.135851 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/e77ccc47-f7cc-4543-8650-6c03f3764778-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:38 crc kubenswrapper[5052]: I1203 12:21:38.135864 5052 reconciler_common.go:293] "Volume detached for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/4107ef62-fa32-4d24-a7ad-3e2eb59884de-credential-keys\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:38 crc kubenswrapper[5052]: I1203 12:21:38.135876 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-pl8gv\" (UniqueName: \"kubernetes.io/projected/4107ef62-fa32-4d24-a7ad-3e2eb59884de-kube-api-access-pl8gv\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:38 crc kubenswrapper[5052]: I1203 12:21:38.135891 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/4107ef62-fa32-4d24-a7ad-3e2eb59884de-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:38 crc kubenswrapper[5052]: I1203 12:21:38.140179 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e77ccc47-f7cc-4543-8650-6c03f3764778-kube-api-access-ffbd4" (OuterVolumeSpecName: "kube-api-access-ffbd4") pod "e77ccc47-f7cc-4543-8650-6c03f3764778" (UID: "e77ccc47-f7cc-4543-8650-6c03f3764778"). InnerVolumeSpecName "kube-api-access-ffbd4". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:21:38 crc kubenswrapper[5052]: I1203 12:21:38.187821 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e77ccc47-f7cc-4543-8650-6c03f3764778-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "e77ccc47-f7cc-4543-8650-6c03f3764778" (UID: "e77ccc47-f7cc-4543-8650-6c03f3764778"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:38 crc kubenswrapper[5052]: I1203 12:21:38.207229 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e77ccc47-f7cc-4543-8650-6c03f3764778-config-data" (OuterVolumeSpecName: "config-data") pod "e77ccc47-f7cc-4543-8650-6c03f3764778" (UID: "e77ccc47-f7cc-4543-8650-6c03f3764778"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:38 crc kubenswrapper[5052]: I1203 12:21:38.222387 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e77ccc47-f7cc-4543-8650-6c03f3764778-nova-metadata-tls-certs" (OuterVolumeSpecName: "nova-metadata-tls-certs") pod "e77ccc47-f7cc-4543-8650-6c03f3764778" (UID: "e77ccc47-f7cc-4543-8650-6c03f3764778"). InnerVolumeSpecName "nova-metadata-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:38 crc kubenswrapper[5052]: I1203 12:21:38.232037 5052 generic.go:334] "Generic (PLEG): container finished" podID="4107ef62-fa32-4d24-a7ad-3e2eb59884de" containerID="e93ca0a9fbc6106e1c59be08c9244f1adaba71b37a9be94ca1173d57e88ad162" exitCode=0 Dec 03 12:21:38 crc kubenswrapper[5052]: I1203 12:21:38.232150 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-78dd58875-slm9r" Dec 03 12:21:38 crc kubenswrapper[5052]: I1203 12:21:38.232163 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-78dd58875-slm9r" event={"ID":"4107ef62-fa32-4d24-a7ad-3e2eb59884de","Type":"ContainerDied","Data":"e93ca0a9fbc6106e1c59be08c9244f1adaba71b37a9be94ca1173d57e88ad162"} Dec 03 12:21:38 crc kubenswrapper[5052]: I1203 12:21:38.232196 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-78dd58875-slm9r" event={"ID":"4107ef62-fa32-4d24-a7ad-3e2eb59884de","Type":"ContainerDied","Data":"1d45c10dc60a6974996075c035507884e5f15a3d87050e09496d080d220b5d2b"} Dec 03 12:21:38 crc kubenswrapper[5052]: I1203 12:21:38.232217 5052 scope.go:117] "RemoveContainer" containerID="e93ca0a9fbc6106e1c59be08c9244f1adaba71b37a9be94ca1173d57e88ad162" Dec 03 12:21:38 crc kubenswrapper[5052]: I1203 12:21:38.251817 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"openstack-config\" (UniqueName: \"kubernetes.io/configmap/892f0ab2-8602-4485-a11c-5171252e2a7b-openstack-config\") pod \"openstackclient\" (UID: \"892f0ab2-8602-4485-a11c-5171252e2a7b\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 12:21:38 crc kubenswrapper[5052]: I1203 12:21:38.252025 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"openstack-config-secret\" (UniqueName: \"kubernetes.io/secret/892f0ab2-8602-4485-a11c-5171252e2a7b-openstack-config-secret\") pod \"openstackclient\" (UID: \"892f0ab2-8602-4485-a11c-5171252e2a7b\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 12:21:38 crc kubenswrapper[5052]: I1203 12:21:38.255268 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/892f0ab2-8602-4485-a11c-5171252e2a7b-combined-ca-bundle\") pod \"openstackclient\" (UID: \"892f0ab2-8602-4485-a11c-5171252e2a7b\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 12:21:38 crc kubenswrapper[5052]: I1203 12:21:38.255482 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-4xxw2\" (UniqueName: \"kubernetes.io/projected/892f0ab2-8602-4485-a11c-5171252e2a7b-kube-api-access-4xxw2\") pod \"openstackclient\" (UID: \"892f0ab2-8602-4485-a11c-5171252e2a7b\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 12:21:38 crc kubenswrapper[5052]: I1203 12:21:38.259767 5052 generic.go:334] "Generic (PLEG): container finished" podID="cd50414b-e062-408f-9ce0-052489596b5b" containerID="6ea34547ae7e1c8cfc3acb2770d1230c67f6ec0e4c220fe214a5a88afa39571c" exitCode=0 Dec 03 12:21:38 crc kubenswrapper[5052]: I1203 12:21:38.259810 5052 generic.go:334] "Generic (PLEG): container finished" podID="cd50414b-e062-408f-9ce0-052489596b5b" containerID="0cdc7315abd0e95c41207cdd0a71a4ba5caee6de1636c4f55eaac360444a3a2d" exitCode=143 Dec 03 12:21:38 crc kubenswrapper[5052]: I1203 12:21:38.260195 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"cd50414b-e062-408f-9ce0-052489596b5b","Type":"ContainerDied","Data":"6ea34547ae7e1c8cfc3acb2770d1230c67f6ec0e4c220fe214a5a88afa39571c"} Dec 03 12:21:38 crc kubenswrapper[5052]: I1203 12:21:38.260394 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"cd50414b-e062-408f-9ce0-052489596b5b","Type":"ContainerDied","Data":"0cdc7315abd0e95c41207cdd0a71a4ba5caee6de1636c4f55eaac360444a3a2d"} Dec 03 12:21:38 crc kubenswrapper[5052]: I1203 12:21:38.263752 5052 reconciler_common.go:293] "Volume detached for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/e77ccc47-f7cc-4543-8650-6c03f3764778-nova-metadata-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:38 crc kubenswrapper[5052]: I1203 12:21:38.263808 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/e77ccc47-f7cc-4543-8650-6c03f3764778-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:38 crc kubenswrapper[5052]: I1203 12:21:38.263828 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-ffbd4\" (UniqueName: \"kubernetes.io/projected/e77ccc47-f7cc-4543-8650-6c03f3764778-kube-api-access-ffbd4\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:38 crc kubenswrapper[5052]: I1203 12:21:38.263852 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e77ccc47-f7cc-4543-8650-6c03f3764778-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:38 crc kubenswrapper[5052]: I1203 12:21:38.269443 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-cell-mapping-6dkwd" event={"ID":"0bbf844e-3419-4f88-b375-ca77df3007af","Type":"ContainerDied","Data":"108d3bff9d752c0eb795849f42f7174dddd9e36267357acf3faa96c4252f424b"} Dec 03 12:21:38 crc kubenswrapper[5052]: I1203 12:21:38.269495 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="108d3bff9d752c0eb795849f42f7174dddd9e36267357acf3faa96c4252f424b" Dec 03 12:21:38 crc kubenswrapper[5052]: I1203 12:21:38.269462 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-cell-mapping-6dkwd" Dec 03 12:21:38 crc kubenswrapper[5052]: I1203 12:21:38.273863 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"openstack-config\" (UniqueName: \"kubernetes.io/configmap/892f0ab2-8602-4485-a11c-5171252e2a7b-openstack-config\") pod \"openstackclient\" (UID: \"892f0ab2-8602-4485-a11c-5171252e2a7b\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 12:21:38 crc kubenswrapper[5052]: I1203 12:21:38.278524 5052 generic.go:334] "Generic (PLEG): container finished" podID="e77ccc47-f7cc-4543-8650-6c03f3764778" containerID="e56be8de24b8902adb455f81974dad4fb51a1e35d0517e98a360cf0479b92cca" exitCode=0 Dec 03 12:21:38 crc kubenswrapper[5052]: I1203 12:21:38.278559 5052 generic.go:334] "Generic (PLEG): container finished" podID="e77ccc47-f7cc-4543-8650-6c03f3764778" containerID="b128132fcc73ebdf762c298ede4fe69e106df8231adcc8687c7d9484c3a8ec8e" exitCode=143 Dec 03 12:21:38 crc kubenswrapper[5052]: I1203 12:21:38.278708 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"e77ccc47-f7cc-4543-8650-6c03f3764778","Type":"ContainerDied","Data":"e56be8de24b8902adb455f81974dad4fb51a1e35d0517e98a360cf0479b92cca"} Dec 03 12:21:38 crc kubenswrapper[5052]: I1203 12:21:38.278799 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"e77ccc47-f7cc-4543-8650-6c03f3764778","Type":"ContainerDied","Data":"b128132fcc73ebdf762c298ede4fe69e106df8231adcc8687c7d9484c3a8ec8e"} Dec 03 12:21:38 crc kubenswrapper[5052]: I1203 12:21:38.278818 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"e77ccc47-f7cc-4543-8650-6c03f3764778","Type":"ContainerDied","Data":"b8892b7fd9cdbc94bbf9610ada60454ec616f6a3093334f07463f58c6e00620d"} Dec 03 12:21:38 crc kubenswrapper[5052]: I1203 12:21:38.278745 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:21:38 crc kubenswrapper[5052]: I1203 12:21:38.282036 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"openstack-config-secret\" (UniqueName: \"kubernetes.io/secret/892f0ab2-8602-4485-a11c-5171252e2a7b-openstack-config-secret\") pod \"openstackclient\" (UID: \"892f0ab2-8602-4485-a11c-5171252e2a7b\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 12:21:38 crc kubenswrapper[5052]: I1203 12:21:38.282175 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/892f0ab2-8602-4485-a11c-5171252e2a7b-combined-ca-bundle\") pod \"openstackclient\" (UID: \"892f0ab2-8602-4485-a11c-5171252e2a7b\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 12:21:38 crc kubenswrapper[5052]: I1203 12:21:38.284798 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-scheduler-0" podUID="779772f8-a0bd-491f-9153-33614d19e423" containerName="nova-scheduler-scheduler" containerID="cri-o://0e513984b9212a78bfe53fea749394bea6e44fa33a9bc3a945f96fe05465097a" gracePeriod=30 Dec 03 12:21:38 crc kubenswrapper[5052]: I1203 12:21:38.285370 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-4xxw2\" (UniqueName: \"kubernetes.io/projected/892f0ab2-8602-4485-a11c-5171252e2a7b-kube-api-access-4xxw2\") pod \"openstackclient\" (UID: \"892f0ab2-8602-4485-a11c-5171252e2a7b\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 12:21:38 crc kubenswrapper[5052]: I1203 12:21:38.305473 5052 scope.go:117] "RemoveContainer" containerID="e93ca0a9fbc6106e1c59be08c9244f1adaba71b37a9be94ca1173d57e88ad162" Dec 03 12:21:38 crc kubenswrapper[5052]: E1203 12:21:38.308755 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e93ca0a9fbc6106e1c59be08c9244f1adaba71b37a9be94ca1173d57e88ad162\": container with ID starting with e93ca0a9fbc6106e1c59be08c9244f1adaba71b37a9be94ca1173d57e88ad162 not found: ID does not exist" containerID="e93ca0a9fbc6106e1c59be08c9244f1adaba71b37a9be94ca1173d57e88ad162" Dec 03 12:21:38 crc kubenswrapper[5052]: I1203 12:21:38.308822 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e93ca0a9fbc6106e1c59be08c9244f1adaba71b37a9be94ca1173d57e88ad162"} err="failed to get container status \"e93ca0a9fbc6106e1c59be08c9244f1adaba71b37a9be94ca1173d57e88ad162\": rpc error: code = NotFound desc = could not find container \"e93ca0a9fbc6106e1c59be08c9244f1adaba71b37a9be94ca1173d57e88ad162\": container with ID starting with e93ca0a9fbc6106e1c59be08c9244f1adaba71b37a9be94ca1173d57e88ad162 not found: ID does not exist" Dec 03 12:21:38 crc kubenswrapper[5052]: I1203 12:21:38.308856 5052 scope.go:117] "RemoveContainer" containerID="e56be8de24b8902adb455f81974dad4fb51a1e35d0517e98a360cf0479b92cca" Dec 03 12:21:38 crc kubenswrapper[5052]: I1203 12:21:38.319889 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:21:38 crc kubenswrapper[5052]: I1203 12:21:38.348938 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstackclient" Dec 03 12:21:38 crc kubenswrapper[5052]: I1203 12:21:38.356910 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/keystone-78dd58875-slm9r"] Dec 03 12:21:38 crc kubenswrapper[5052]: I1203 12:21:38.365243 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/keystone-78dd58875-slm9r"] Dec 03 12:21:38 crc kubenswrapper[5052]: I1203 12:21:38.369395 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/cd50414b-e062-408f-9ce0-052489596b5b-logs\") pod \"cd50414b-e062-408f-9ce0-052489596b5b\" (UID: \"cd50414b-e062-408f-9ce0-052489596b5b\") " Dec 03 12:21:38 crc kubenswrapper[5052]: I1203 12:21:38.369577 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/cd50414b-e062-408f-9ce0-052489596b5b-config-data\") pod \"cd50414b-e062-408f-9ce0-052489596b5b\" (UID: \"cd50414b-e062-408f-9ce0-052489596b5b\") " Dec 03 12:21:38 crc kubenswrapper[5052]: I1203 12:21:38.369639 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/cd50414b-e062-408f-9ce0-052489596b5b-internal-tls-certs\") pod \"cd50414b-e062-408f-9ce0-052489596b5b\" (UID: \"cd50414b-e062-408f-9ce0-052489596b5b\") " Dec 03 12:21:38 crc kubenswrapper[5052]: I1203 12:21:38.369673 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/cd50414b-e062-408f-9ce0-052489596b5b-combined-ca-bundle\") pod \"cd50414b-e062-408f-9ce0-052489596b5b\" (UID: \"cd50414b-e062-408f-9ce0-052489596b5b\") " Dec 03 12:21:38 crc kubenswrapper[5052]: I1203 12:21:38.369710 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2cbpl\" (UniqueName: \"kubernetes.io/projected/cd50414b-e062-408f-9ce0-052489596b5b-kube-api-access-2cbpl\") pod \"cd50414b-e062-408f-9ce0-052489596b5b\" (UID: \"cd50414b-e062-408f-9ce0-052489596b5b\") " Dec 03 12:21:38 crc kubenswrapper[5052]: I1203 12:21:38.369739 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/cd50414b-e062-408f-9ce0-052489596b5b-public-tls-certs\") pod \"cd50414b-e062-408f-9ce0-052489596b5b\" (UID: \"cd50414b-e062-408f-9ce0-052489596b5b\") " Dec 03 12:21:38 crc kubenswrapper[5052]: I1203 12:21:38.370902 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd50414b-e062-408f-9ce0-052489596b5b-logs" (OuterVolumeSpecName: "logs") pod "cd50414b-e062-408f-9ce0-052489596b5b" (UID: "cd50414b-e062-408f-9ce0-052489596b5b"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:21:38 crc kubenswrapper[5052]: I1203 12:21:38.390217 5052 scope.go:117] "RemoveContainer" containerID="b128132fcc73ebdf762c298ede4fe69e106df8231adcc8687c7d9484c3a8ec8e" Dec 03 12:21:38 crc kubenswrapper[5052]: I1203 12:21:38.398566 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cd50414b-e062-408f-9ce0-052489596b5b-kube-api-access-2cbpl" (OuterVolumeSpecName: "kube-api-access-2cbpl") pod "cd50414b-e062-408f-9ce0-052489596b5b" (UID: "cd50414b-e062-408f-9ce0-052489596b5b"). InnerVolumeSpecName "kube-api-access-2cbpl". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:21:38 crc kubenswrapper[5052]: I1203 12:21:38.454780 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cd50414b-e062-408f-9ce0-052489596b5b-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "cd50414b-e062-408f-9ce0-052489596b5b" (UID: "cd50414b-e062-408f-9ce0-052489596b5b"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:38 crc kubenswrapper[5052]: I1203 12:21:38.454851 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/swift-proxy-6b9f9968d-5zjp2"] Dec 03 12:21:38 crc kubenswrapper[5052]: I1203 12:21:38.468640 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cd50414b-e062-408f-9ce0-052489596b5b-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "cd50414b-e062-408f-9ce0-052489596b5b" (UID: "cd50414b-e062-408f-9ce0-052489596b5b"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:38 crc kubenswrapper[5052]: I1203 12:21:38.471205 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:21:38 crc kubenswrapper[5052]: I1203 12:21:38.471807 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/cd50414b-e062-408f-9ce0-052489596b5b-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:38 crc kubenswrapper[5052]: I1203 12:21:38.471831 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-2cbpl\" (UniqueName: \"kubernetes.io/projected/cd50414b-e062-408f-9ce0-052489596b5b-kube-api-access-2cbpl\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:38 crc kubenswrapper[5052]: I1203 12:21:38.471863 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/cd50414b-e062-408f-9ce0-052489596b5b-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:38 crc kubenswrapper[5052]: I1203 12:21:38.471871 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/cd50414b-e062-408f-9ce0-052489596b5b-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:38 crc kubenswrapper[5052]: I1203 12:21:38.478414 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cd50414b-e062-408f-9ce0-052489596b5b-config-data" (OuterVolumeSpecName: "config-data") pod "cd50414b-e062-408f-9ce0-052489596b5b" (UID: "cd50414b-e062-408f-9ce0-052489596b5b"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:38 crc kubenswrapper[5052]: I1203 12:21:38.480451 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cd50414b-e062-408f-9ce0-052489596b5b-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "cd50414b-e062-408f-9ce0-052489596b5b" (UID: "cd50414b-e062-408f-9ce0-052489596b5b"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:38 crc kubenswrapper[5052]: I1203 12:21:38.504264 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:21:38 crc kubenswrapper[5052]: I1203 12:21:38.513273 5052 scope.go:117] "RemoveContainer" containerID="e56be8de24b8902adb455f81974dad4fb51a1e35d0517e98a360cf0479b92cca" Dec 03 12:21:38 crc kubenswrapper[5052]: I1203 12:21:38.513929 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:21:38 crc kubenswrapper[5052]: E1203 12:21:38.514404 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="e77ccc47-f7cc-4543-8650-6c03f3764778" containerName="nova-metadata-log" Dec 03 12:21:38 crc kubenswrapper[5052]: I1203 12:21:38.514420 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="e77ccc47-f7cc-4543-8650-6c03f3764778" containerName="nova-metadata-log" Dec 03 12:21:38 crc kubenswrapper[5052]: E1203 12:21:38.514438 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="cd50414b-e062-408f-9ce0-052489596b5b" containerName="nova-api-log" Dec 03 12:21:38 crc kubenswrapper[5052]: I1203 12:21:38.514445 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="cd50414b-e062-408f-9ce0-052489596b5b" containerName="nova-api-log" Dec 03 12:21:38 crc kubenswrapper[5052]: E1203 12:21:38.514456 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="cd50414b-e062-408f-9ce0-052489596b5b" containerName="nova-api-api" Dec 03 12:21:38 crc kubenswrapper[5052]: I1203 12:21:38.514462 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="cd50414b-e062-408f-9ce0-052489596b5b" containerName="nova-api-api" Dec 03 12:21:38 crc kubenswrapper[5052]: E1203 12:21:38.514522 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="e77ccc47-f7cc-4543-8650-6c03f3764778" containerName="nova-metadata-metadata" Dec 03 12:21:38 crc kubenswrapper[5052]: I1203 12:21:38.514529 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="e77ccc47-f7cc-4543-8650-6c03f3764778" containerName="nova-metadata-metadata" Dec 03 12:21:38 crc kubenswrapper[5052]: I1203 12:21:38.514720 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="cd50414b-e062-408f-9ce0-052489596b5b" containerName="nova-api-api" Dec 03 12:21:38 crc kubenswrapper[5052]: I1203 12:21:38.514735 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="cd50414b-e062-408f-9ce0-052489596b5b" containerName="nova-api-log" Dec 03 12:21:38 crc kubenswrapper[5052]: I1203 12:21:38.514753 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="e77ccc47-f7cc-4543-8650-6c03f3764778" containerName="nova-metadata-log" Dec 03 12:21:38 crc kubenswrapper[5052]: I1203 12:21:38.514779 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="e77ccc47-f7cc-4543-8650-6c03f3764778" containerName="nova-metadata-metadata" Dec 03 12:21:38 crc kubenswrapper[5052]: I1203 12:21:38.516206 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:21:38 crc kubenswrapper[5052]: E1203 12:21:38.516607 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e56be8de24b8902adb455f81974dad4fb51a1e35d0517e98a360cf0479b92cca\": container with ID starting with e56be8de24b8902adb455f81974dad4fb51a1e35d0517e98a360cf0479b92cca not found: ID does not exist" containerID="e56be8de24b8902adb455f81974dad4fb51a1e35d0517e98a360cf0479b92cca" Dec 03 12:21:38 crc kubenswrapper[5052]: I1203 12:21:38.516646 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e56be8de24b8902adb455f81974dad4fb51a1e35d0517e98a360cf0479b92cca"} err="failed to get container status \"e56be8de24b8902adb455f81974dad4fb51a1e35d0517e98a360cf0479b92cca\": rpc error: code = NotFound desc = could not find container \"e56be8de24b8902adb455f81974dad4fb51a1e35d0517e98a360cf0479b92cca\": container with ID starting with e56be8de24b8902adb455f81974dad4fb51a1e35d0517e98a360cf0479b92cca not found: ID does not exist" Dec 03 12:21:38 crc kubenswrapper[5052]: I1203 12:21:38.516669 5052 scope.go:117] "RemoveContainer" containerID="b128132fcc73ebdf762c298ede4fe69e106df8231adcc8687c7d9484c3a8ec8e" Dec 03 12:21:38 crc kubenswrapper[5052]: E1203 12:21:38.517212 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b128132fcc73ebdf762c298ede4fe69e106df8231adcc8687c7d9484c3a8ec8e\": container with ID starting with b128132fcc73ebdf762c298ede4fe69e106df8231adcc8687c7d9484c3a8ec8e not found: ID does not exist" containerID="b128132fcc73ebdf762c298ede4fe69e106df8231adcc8687c7d9484c3a8ec8e" Dec 03 12:21:38 crc kubenswrapper[5052]: I1203 12:21:38.517256 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b128132fcc73ebdf762c298ede4fe69e106df8231adcc8687c7d9484c3a8ec8e"} err="failed to get container status \"b128132fcc73ebdf762c298ede4fe69e106df8231adcc8687c7d9484c3a8ec8e\": rpc error: code = NotFound desc = could not find container \"b128132fcc73ebdf762c298ede4fe69e106df8231adcc8687c7d9484c3a8ec8e\": container with ID starting with b128132fcc73ebdf762c298ede4fe69e106df8231adcc8687c7d9484c3a8ec8e not found: ID does not exist" Dec 03 12:21:38 crc kubenswrapper[5052]: I1203 12:21:38.517286 5052 scope.go:117] "RemoveContainer" containerID="e56be8de24b8902adb455f81974dad4fb51a1e35d0517e98a360cf0479b92cca" Dec 03 12:21:38 crc kubenswrapper[5052]: I1203 12:21:38.519251 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e56be8de24b8902adb455f81974dad4fb51a1e35d0517e98a360cf0479b92cca"} err="failed to get container status \"e56be8de24b8902adb455f81974dad4fb51a1e35d0517e98a360cf0479b92cca\": rpc error: code = NotFound desc = could not find container \"e56be8de24b8902adb455f81974dad4fb51a1e35d0517e98a360cf0479b92cca\": container with ID starting with e56be8de24b8902adb455f81974dad4fb51a1e35d0517e98a360cf0479b92cca not found: ID does not exist" Dec 03 12:21:38 crc kubenswrapper[5052]: I1203 12:21:38.519281 5052 scope.go:117] "RemoveContainer" containerID="b128132fcc73ebdf762c298ede4fe69e106df8231adcc8687c7d9484c3a8ec8e" Dec 03 12:21:38 crc kubenswrapper[5052]: I1203 12:21:38.519486 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-nova-metadata-internal-svc" Dec 03 12:21:38 crc kubenswrapper[5052]: I1203 12:21:38.519699 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-metadata-config-data" Dec 03 12:21:38 crc kubenswrapper[5052]: I1203 12:21:38.519832 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b128132fcc73ebdf762c298ede4fe69e106df8231adcc8687c7d9484c3a8ec8e"} err="failed to get container status \"b128132fcc73ebdf762c298ede4fe69e106df8231adcc8687c7d9484c3a8ec8e\": rpc error: code = NotFound desc = could not find container \"b128132fcc73ebdf762c298ede4fe69e106df8231adcc8687c7d9484c3a8ec8e\": container with ID starting with b128132fcc73ebdf762c298ede4fe69e106df8231adcc8687c7d9484c3a8ec8e not found: ID does not exist" Dec 03 12:21:38 crc kubenswrapper[5052]: I1203 12:21:38.524196 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:21:38 crc kubenswrapper[5052]: I1203 12:21:38.574010 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4dd73a47-cf2a-43b8-ba1c-0d47332edaa4-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"4dd73a47-cf2a-43b8-ba1c-0d47332edaa4\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:21:38 crc kubenswrapper[5052]: I1203 12:21:38.574215 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mdj9b\" (UniqueName: \"kubernetes.io/projected/4dd73a47-cf2a-43b8-ba1c-0d47332edaa4-kube-api-access-mdj9b\") pod \"nova-metadata-0\" (UID: \"4dd73a47-cf2a-43b8-ba1c-0d47332edaa4\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:21:38 crc kubenswrapper[5052]: I1203 12:21:38.574321 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/4dd73a47-cf2a-43b8-ba1c-0d47332edaa4-config-data\") pod \"nova-metadata-0\" (UID: \"4dd73a47-cf2a-43b8-ba1c-0d47332edaa4\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:21:38 crc kubenswrapper[5052]: I1203 12:21:38.574358 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/4dd73a47-cf2a-43b8-ba1c-0d47332edaa4-nova-metadata-tls-certs\") pod \"nova-metadata-0\" (UID: \"4dd73a47-cf2a-43b8-ba1c-0d47332edaa4\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:21:38 crc kubenswrapper[5052]: I1203 12:21:38.574411 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/4dd73a47-cf2a-43b8-ba1c-0d47332edaa4-logs\") pod \"nova-metadata-0\" (UID: \"4dd73a47-cf2a-43b8-ba1c-0d47332edaa4\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:21:38 crc kubenswrapper[5052]: I1203 12:21:38.574472 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/cd50414b-e062-408f-9ce0-052489596b5b-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:38 crc kubenswrapper[5052]: I1203 12:21:38.574482 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/cd50414b-e062-408f-9ce0-052489596b5b-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:38 crc kubenswrapper[5052]: I1203 12:21:38.592657 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:21:38 crc kubenswrapper[5052]: I1203 12:21:38.675676 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-mdj9b\" (UniqueName: \"kubernetes.io/projected/4dd73a47-cf2a-43b8-ba1c-0d47332edaa4-kube-api-access-mdj9b\") pod \"nova-metadata-0\" (UID: \"4dd73a47-cf2a-43b8-ba1c-0d47332edaa4\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:21:38 crc kubenswrapper[5052]: I1203 12:21:38.675944 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/4dd73a47-cf2a-43b8-ba1c-0d47332edaa4-config-data\") pod \"nova-metadata-0\" (UID: \"4dd73a47-cf2a-43b8-ba1c-0d47332edaa4\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:21:38 crc kubenswrapper[5052]: I1203 12:21:38.675970 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/4dd73a47-cf2a-43b8-ba1c-0d47332edaa4-nova-metadata-tls-certs\") pod \"nova-metadata-0\" (UID: \"4dd73a47-cf2a-43b8-ba1c-0d47332edaa4\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:21:38 crc kubenswrapper[5052]: I1203 12:21:38.676008 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/4dd73a47-cf2a-43b8-ba1c-0d47332edaa4-logs\") pod \"nova-metadata-0\" (UID: \"4dd73a47-cf2a-43b8-ba1c-0d47332edaa4\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:21:38 crc kubenswrapper[5052]: I1203 12:21:38.676043 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4dd73a47-cf2a-43b8-ba1c-0d47332edaa4-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"4dd73a47-cf2a-43b8-ba1c-0d47332edaa4\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:21:38 crc kubenswrapper[5052]: I1203 12:21:38.676856 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/4dd73a47-cf2a-43b8-ba1c-0d47332edaa4-logs\") pod \"nova-metadata-0\" (UID: \"4dd73a47-cf2a-43b8-ba1c-0d47332edaa4\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:21:38 crc kubenswrapper[5052]: I1203 12:21:38.679931 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/4dd73a47-cf2a-43b8-ba1c-0d47332edaa4-nova-metadata-tls-certs\") pod \"nova-metadata-0\" (UID: \"4dd73a47-cf2a-43b8-ba1c-0d47332edaa4\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:21:38 crc kubenswrapper[5052]: I1203 12:21:38.680422 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/4dd73a47-cf2a-43b8-ba1c-0d47332edaa4-config-data\") pod \"nova-metadata-0\" (UID: \"4dd73a47-cf2a-43b8-ba1c-0d47332edaa4\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:21:38 crc kubenswrapper[5052]: I1203 12:21:38.683184 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4dd73a47-cf2a-43b8-ba1c-0d47332edaa4-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"4dd73a47-cf2a-43b8-ba1c-0d47332edaa4\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:21:38 crc kubenswrapper[5052]: I1203 12:21:38.698450 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-mdj9b\" (UniqueName: \"kubernetes.io/projected/4dd73a47-cf2a-43b8-ba1c-0d47332edaa4-kube-api-access-mdj9b\") pod \"nova-metadata-0\" (UID: \"4dd73a47-cf2a-43b8-ba1c-0d47332edaa4\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:21:38 crc kubenswrapper[5052]: I1203 12:21:38.821134 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:21:39 crc kubenswrapper[5052]: I1203 12:21:39.049053 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/openstackclient"] Dec 03 12:21:39 crc kubenswrapper[5052]: W1203 12:21:39.052198 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod892f0ab2_8602_4485_a11c_5171252e2a7b.slice/crio-cc76584824a603479ce037ec6c21062c9c0c113b6499686abd9a92f9e99cd63e WatchSource:0}: Error finding container cc76584824a603479ce037ec6c21062c9c0c113b6499686abd9a92f9e99cd63e: Status 404 returned error can't find the container with id cc76584824a603479ce037ec6c21062c9c0c113b6499686abd9a92f9e99cd63e Dec 03 12:21:39 crc kubenswrapper[5052]: I1203 12:21:39.302593 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:21:39 crc kubenswrapper[5052]: I1203 12:21:39.302736 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"cd50414b-e062-408f-9ce0-052489596b5b","Type":"ContainerDied","Data":"563788af792ed6e17d075863e6aa4a5ac24aa618e78a98ce12dcc0c54053afac"} Dec 03 12:21:39 crc kubenswrapper[5052]: I1203 12:21:39.303229 5052 scope.go:117] "RemoveContainer" containerID="6ea34547ae7e1c8cfc3acb2770d1230c67f6ec0e4c220fe214a5a88afa39571c" Dec 03 12:21:39 crc kubenswrapper[5052]: I1203 12:21:39.323076 5052 generic.go:334] "Generic (PLEG): container finished" podID="779772f8-a0bd-491f-9153-33614d19e423" containerID="0e513984b9212a78bfe53fea749394bea6e44fa33a9bc3a945f96fe05465097a" exitCode=0 Dec 03 12:21:39 crc kubenswrapper[5052]: I1203 12:21:39.323146 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"779772f8-a0bd-491f-9153-33614d19e423","Type":"ContainerDied","Data":"0e513984b9212a78bfe53fea749394bea6e44fa33a9bc3a945f96fe05465097a"} Dec 03 12:21:39 crc kubenswrapper[5052]: I1203 12:21:39.328776 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstackclient" event={"ID":"892f0ab2-8602-4485-a11c-5171252e2a7b","Type":"ContainerStarted","Data":"cc76584824a603479ce037ec6c21062c9c0c113b6499686abd9a92f9e99cd63e"} Dec 03 12:21:39 crc kubenswrapper[5052]: W1203 12:21:39.335270 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4dd73a47_cf2a_43b8_ba1c_0d47332edaa4.slice/crio-76b4d90822b2cea4dc663e9fb413320e7954c906bd78785f2f60c7e955624e00 WatchSource:0}: Error finding container 76b4d90822b2cea4dc663e9fb413320e7954c906bd78785f2f60c7e955624e00: Status 404 returned error can't find the container with id 76b4d90822b2cea4dc663e9fb413320e7954c906bd78785f2f60c7e955624e00 Dec 03 12:21:39 crc kubenswrapper[5052]: I1203 12:21:39.337113 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:21:39 crc kubenswrapper[5052]: I1203 12:21:39.343388 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-proxy-6b9f9968d-5zjp2" event={"ID":"fb73cc9a-89cc-4cb8-87ce-2731e5dab0f7","Type":"ContainerStarted","Data":"370f335a2865f2cdbcf3d953581d2819c13f1b7d88a0ac612179ca394589c0e9"} Dec 03 12:21:39 crc kubenswrapper[5052]: I1203 12:21:39.343475 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-proxy-6b9f9968d-5zjp2" event={"ID":"fb73cc9a-89cc-4cb8-87ce-2731e5dab0f7","Type":"ContainerStarted","Data":"a1d1e7cece7f6bab44f530f29cd0b9119aee96b8c725b66c1bb8c782f4ccd637"} Dec 03 12:21:39 crc kubenswrapper[5052]: I1203 12:21:39.343499 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-proxy-6b9f9968d-5zjp2" event={"ID":"fb73cc9a-89cc-4cb8-87ce-2731e5dab0f7","Type":"ContainerStarted","Data":"626262d6ee8fb51930f0b1e62c74d74c26de191482f371f01edff5823037b3bf"} Dec 03 12:21:39 crc kubenswrapper[5052]: I1203 12:21:39.343570 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/swift-proxy-6b9f9968d-5zjp2" Dec 03 12:21:39 crc kubenswrapper[5052]: I1203 12:21:39.343618 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/swift-proxy-6b9f9968d-5zjp2" Dec 03 12:21:39 crc kubenswrapper[5052]: I1203 12:21:39.350475 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:21:39 crc kubenswrapper[5052]: I1203 12:21:39.366241 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:21:39 crc kubenswrapper[5052]: I1203 12:21:39.375745 5052 scope.go:117] "RemoveContainer" containerID="0cdc7315abd0e95c41207cdd0a71a4ba5caee6de1636c4f55eaac360444a3a2d" Dec 03 12:21:39 crc kubenswrapper[5052]: I1203 12:21:39.375992 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:21:39 crc kubenswrapper[5052]: I1203 12:21:39.395594 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/779772f8-a0bd-491f-9153-33614d19e423-config-data\") pod \"779772f8-a0bd-491f-9153-33614d19e423\" (UID: \"779772f8-a0bd-491f-9153-33614d19e423\") " Dec 03 12:21:39 crc kubenswrapper[5052]: I1203 12:21:39.395649 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/779772f8-a0bd-491f-9153-33614d19e423-combined-ca-bundle\") pod \"779772f8-a0bd-491f-9153-33614d19e423\" (UID: \"779772f8-a0bd-491f-9153-33614d19e423\") " Dec 03 12:21:39 crc kubenswrapper[5052]: I1203 12:21:39.395781 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q4cg5\" (UniqueName: \"kubernetes.io/projected/779772f8-a0bd-491f-9153-33614d19e423-kube-api-access-q4cg5\") pod \"779772f8-a0bd-491f-9153-33614d19e423\" (UID: \"779772f8-a0bd-491f-9153-33614d19e423\") " Dec 03 12:21:39 crc kubenswrapper[5052]: I1203 12:21:39.402380 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/779772f8-a0bd-491f-9153-33614d19e423-kube-api-access-q4cg5" (OuterVolumeSpecName: "kube-api-access-q4cg5") pod "779772f8-a0bd-491f-9153-33614d19e423" (UID: "779772f8-a0bd-491f-9153-33614d19e423"). InnerVolumeSpecName "kube-api-access-q4cg5". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:21:39 crc kubenswrapper[5052]: I1203 12:21:39.408731 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:21:39 crc kubenswrapper[5052]: E1203 12:21:39.409534 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="779772f8-a0bd-491f-9153-33614d19e423" containerName="nova-scheduler-scheduler" Dec 03 12:21:39 crc kubenswrapper[5052]: I1203 12:21:39.409559 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="779772f8-a0bd-491f-9153-33614d19e423" containerName="nova-scheduler-scheduler" Dec 03 12:21:39 crc kubenswrapper[5052]: E1203 12:21:39.409625 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="779772f8-a0bd-491f-9153-33614d19e423" containerName="nova-scheduler-scheduler" Dec 03 12:21:39 crc kubenswrapper[5052]: I1203 12:21:39.409637 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="779772f8-a0bd-491f-9153-33614d19e423" containerName="nova-scheduler-scheduler" Dec 03 12:21:39 crc kubenswrapper[5052]: I1203 12:21:39.410097 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="779772f8-a0bd-491f-9153-33614d19e423" containerName="nova-scheduler-scheduler" Dec 03 12:21:39 crc kubenswrapper[5052]: I1203 12:21:39.410136 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="779772f8-a0bd-491f-9153-33614d19e423" containerName="nova-scheduler-scheduler" Dec 03 12:21:39 crc kubenswrapper[5052]: I1203 12:21:39.411550 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:21:39 crc kubenswrapper[5052]: I1203 12:21:39.440434 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-nova-internal-svc" Dec 03 12:21:39 crc kubenswrapper[5052]: I1203 12:21:39.440684 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-api-config-data" Dec 03 12:21:39 crc kubenswrapper[5052]: I1203 12:21:39.440976 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-nova-public-svc" Dec 03 12:21:39 crc kubenswrapper[5052]: I1203 12:21:39.446934 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/779772f8-a0bd-491f-9153-33614d19e423-config-data" (OuterVolumeSpecName: "config-data") pod "779772f8-a0bd-491f-9153-33614d19e423" (UID: "779772f8-a0bd-491f-9153-33614d19e423"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:39 crc kubenswrapper[5052]: I1203 12:21:39.469604 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:21:39 crc kubenswrapper[5052]: I1203 12:21:39.473638 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/swift-proxy-6b9f9968d-5zjp2" podStartSLOduration=2.473620252 podStartE2EDuration="2.473620252s" podCreationTimestamp="2025-12-03 12:21:37 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:21:39.440440595 +0000 UTC m=+3690.028682359" watchObservedRunningTime="2025-12-03 12:21:39.473620252 +0000 UTC m=+3690.061862016" Dec 03 12:21:39 crc kubenswrapper[5052]: I1203 12:21:39.479263 5052 scope.go:117] "RemoveContainer" containerID="db51cc7950f2c5a12d1aa905e418ed04e400dcebb362b0f45a28636e8440122e" Dec 03 12:21:39 crc kubenswrapper[5052]: I1203 12:21:39.490941 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/779772f8-a0bd-491f-9153-33614d19e423-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "779772f8-a0bd-491f-9153-33614d19e423" (UID: "779772f8-a0bd-491f-9153-33614d19e423"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:39 crc kubenswrapper[5052]: I1203 12:21:39.502306 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/9dec0a31-cba3-4c38-8b5d-4358f16b3704-public-tls-certs\") pod \"nova-api-0\" (UID: \"9dec0a31-cba3-4c38-8b5d-4358f16b3704\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:21:39 crc kubenswrapper[5052]: I1203 12:21:39.502347 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/9dec0a31-cba3-4c38-8b5d-4358f16b3704-logs\") pod \"nova-api-0\" (UID: \"9dec0a31-cba3-4c38-8b5d-4358f16b3704\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:21:39 crc kubenswrapper[5052]: I1203 12:21:39.502400 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9dec0a31-cba3-4c38-8b5d-4358f16b3704-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"9dec0a31-cba3-4c38-8b5d-4358f16b3704\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:21:39 crc kubenswrapper[5052]: I1203 12:21:39.502627 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/9dec0a31-cba3-4c38-8b5d-4358f16b3704-config-data\") pod \"nova-api-0\" (UID: \"9dec0a31-cba3-4c38-8b5d-4358f16b3704\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:21:39 crc kubenswrapper[5052]: I1203 12:21:39.502672 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/9dec0a31-cba3-4c38-8b5d-4358f16b3704-internal-tls-certs\") pod \"nova-api-0\" (UID: \"9dec0a31-cba3-4c38-8b5d-4358f16b3704\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:21:39 crc kubenswrapper[5052]: I1203 12:21:39.502778 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8t4hw\" (UniqueName: \"kubernetes.io/projected/9dec0a31-cba3-4c38-8b5d-4358f16b3704-kube-api-access-8t4hw\") pod \"nova-api-0\" (UID: \"9dec0a31-cba3-4c38-8b5d-4358f16b3704\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:21:39 crc kubenswrapper[5052]: I1203 12:21:39.502888 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/779772f8-a0bd-491f-9153-33614d19e423-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:39 crc kubenswrapper[5052]: I1203 12:21:39.502901 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-q4cg5\" (UniqueName: \"kubernetes.io/projected/779772f8-a0bd-491f-9153-33614d19e423-kube-api-access-q4cg5\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:39 crc kubenswrapper[5052]: I1203 12:21:39.502911 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/779772f8-a0bd-491f-9153-33614d19e423-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:39 crc kubenswrapper[5052]: I1203 12:21:39.604377 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9dec0a31-cba3-4c38-8b5d-4358f16b3704-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"9dec0a31-cba3-4c38-8b5d-4358f16b3704\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:21:39 crc kubenswrapper[5052]: I1203 12:21:39.604729 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/9dec0a31-cba3-4c38-8b5d-4358f16b3704-config-data\") pod \"nova-api-0\" (UID: \"9dec0a31-cba3-4c38-8b5d-4358f16b3704\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:21:39 crc kubenswrapper[5052]: I1203 12:21:39.604755 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/9dec0a31-cba3-4c38-8b5d-4358f16b3704-internal-tls-certs\") pod \"nova-api-0\" (UID: \"9dec0a31-cba3-4c38-8b5d-4358f16b3704\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:21:39 crc kubenswrapper[5052]: I1203 12:21:39.604801 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-8t4hw\" (UniqueName: \"kubernetes.io/projected/9dec0a31-cba3-4c38-8b5d-4358f16b3704-kube-api-access-8t4hw\") pod \"nova-api-0\" (UID: \"9dec0a31-cba3-4c38-8b5d-4358f16b3704\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:21:39 crc kubenswrapper[5052]: I1203 12:21:39.604820 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/9dec0a31-cba3-4c38-8b5d-4358f16b3704-public-tls-certs\") pod \"nova-api-0\" (UID: \"9dec0a31-cba3-4c38-8b5d-4358f16b3704\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:21:39 crc kubenswrapper[5052]: I1203 12:21:39.604845 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/9dec0a31-cba3-4c38-8b5d-4358f16b3704-logs\") pod \"nova-api-0\" (UID: \"9dec0a31-cba3-4c38-8b5d-4358f16b3704\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:21:39 crc kubenswrapper[5052]: I1203 12:21:39.605202 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/9dec0a31-cba3-4c38-8b5d-4358f16b3704-logs\") pod \"nova-api-0\" (UID: \"9dec0a31-cba3-4c38-8b5d-4358f16b3704\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:21:39 crc kubenswrapper[5052]: I1203 12:21:39.609528 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/9dec0a31-cba3-4c38-8b5d-4358f16b3704-public-tls-certs\") pod \"nova-api-0\" (UID: \"9dec0a31-cba3-4c38-8b5d-4358f16b3704\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:21:39 crc kubenswrapper[5052]: I1203 12:21:39.610050 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9dec0a31-cba3-4c38-8b5d-4358f16b3704-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"9dec0a31-cba3-4c38-8b5d-4358f16b3704\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:21:39 crc kubenswrapper[5052]: I1203 12:21:39.610181 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/9dec0a31-cba3-4c38-8b5d-4358f16b3704-config-data\") pod \"nova-api-0\" (UID: \"9dec0a31-cba3-4c38-8b5d-4358f16b3704\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:21:39 crc kubenswrapper[5052]: I1203 12:21:39.611974 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/9dec0a31-cba3-4c38-8b5d-4358f16b3704-internal-tls-certs\") pod \"nova-api-0\" (UID: \"9dec0a31-cba3-4c38-8b5d-4358f16b3704\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:21:39 crc kubenswrapper[5052]: I1203 12:21:39.622563 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-8t4hw\" (UniqueName: \"kubernetes.io/projected/9dec0a31-cba3-4c38-8b5d-4358f16b3704-kube-api-access-8t4hw\") pod \"nova-api-0\" (UID: \"9dec0a31-cba3-4c38-8b5d-4358f16b3704\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:21:39 crc kubenswrapper[5052]: I1203 12:21:39.692400 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4107ef62-fa32-4d24-a7ad-3e2eb59884de" path="/var/lib/kubelet/pods/4107ef62-fa32-4d24-a7ad-3e2eb59884de/volumes" Dec 03 12:21:39 crc kubenswrapper[5052]: I1203 12:21:39.693362 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cd50414b-e062-408f-9ce0-052489596b5b" path="/var/lib/kubelet/pods/cd50414b-e062-408f-9ce0-052489596b5b/volumes" Dec 03 12:21:39 crc kubenswrapper[5052]: I1203 12:21:39.695272 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e77ccc47-f7cc-4543-8650-6c03f3764778" path="/var/lib/kubelet/pods/e77ccc47-f7cc-4543-8650-6c03f3764778/volumes" Dec 03 12:21:39 crc kubenswrapper[5052]: I1203 12:21:39.774722 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:21:40 crc kubenswrapper[5052]: I1203 12:21:40.100452 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/neutron-734a-account-create-update-vxxfj"] Dec 03 12:21:40 crc kubenswrapper[5052]: I1203 12:21:40.139772 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/neutron-734a-account-create-update-vxxfj"] Dec 03 12:21:40 crc kubenswrapper[5052]: I1203 12:21:40.322109 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/openstackclient"] Dec 03 12:21:40 crc kubenswrapper[5052]: I1203 12:21:40.386577 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/openstackclient"] Dec 03 12:21:40 crc kubenswrapper[5052]: I1203 12:21:40.464275 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/rabbitmq-cell1-server-0"] Dec 03 12:21:40 crc kubenswrapper[5052]: I1203 12:21:40.497716 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/neutron-734a-account-create-update-qcgft"] Dec 03 12:21:40 crc kubenswrapper[5052]: E1203 12:21:40.498212 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="892f0ab2-8602-4485-a11c-5171252e2a7b" containerName="openstackclient" Dec 03 12:21:40 crc kubenswrapper[5052]: I1203 12:21:40.498228 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="892f0ab2-8602-4485-a11c-5171252e2a7b" containerName="openstackclient" Dec 03 12:21:40 crc kubenswrapper[5052]: I1203 12:21:40.498499 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="892f0ab2-8602-4485-a11c-5171252e2a7b" containerName="openstackclient" Dec 03 12:21:40 crc kubenswrapper[5052]: I1203 12:21:40.499847 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-734a-account-create-update-qcgft" Dec 03 12:21:40 crc kubenswrapper[5052]: E1203 12:21:40.501948 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/rabbitmq-cell1-config-data: configmap "rabbitmq-cell1-config-data" not found Dec 03 12:21:40 crc kubenswrapper[5052]: E1203 12:21:40.502001 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c279eaf1-2ee2-46fe-88be-9a5a65d4897e-config-data podName:c279eaf1-2ee2-46fe-88be-9a5a65d4897e nodeName:}" failed. No retries permitted until 2025-12-03 12:21:41.001981983 +0000 UTC m=+3691.590223777 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "config-data" (UniqueName: "kubernetes.io/configmap/c279eaf1-2ee2-46fe-88be-9a5a65d4897e-config-data") pod "rabbitmq-cell1-server-0" (UID: "c279eaf1-2ee2-46fe-88be-9a5a65d4897e") : configmap "rabbitmq-cell1-config-data" not found Dec 03 12:21:40 crc kubenswrapper[5052]: I1203 12:21:40.502083 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-metadata-0" podUID="4dd73a47-cf2a-43b8-ba1c-0d47332edaa4" containerName="nova-metadata-log" containerID="cri-o://dc90f1007187fa83ea0bb677d36177c1605dca45111209c9532edc07fddefddd" gracePeriod=30 Dec 03 12:21:40 crc kubenswrapper[5052]: I1203 12:21:40.502201 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-metadata-0" podUID="4dd73a47-cf2a-43b8-ba1c-0d47332edaa4" containerName="nova-metadata-metadata" containerID="cri-o://c9ed4b9b88de467e9c8e8554fc978077cfb8bd825b23cfae311340cd81ff7333" gracePeriod=30 Dec 03 12:21:40 crc kubenswrapper[5052]: I1203 12:21:40.501934 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"4dd73a47-cf2a-43b8-ba1c-0d47332edaa4","Type":"ContainerStarted","Data":"c9ed4b9b88de467e9c8e8554fc978077cfb8bd825b23cfae311340cd81ff7333"} Dec 03 12:21:40 crc kubenswrapper[5052]: I1203 12:21:40.502590 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"4dd73a47-cf2a-43b8-ba1c-0d47332edaa4","Type":"ContainerStarted","Data":"dc90f1007187fa83ea0bb677d36177c1605dca45111209c9532edc07fddefddd"} Dec 03 12:21:40 crc kubenswrapper[5052]: I1203 12:21:40.502611 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"4dd73a47-cf2a-43b8-ba1c-0d47332edaa4","Type":"ContainerStarted","Data":"76b4d90822b2cea4dc663e9fb413320e7954c906bd78785f2f60c7e955624e00"} Dec 03 12:21:40 crc kubenswrapper[5052]: I1203 12:21:40.504377 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"neutron-db-secret" Dec 03 12:21:40 crc kubenswrapper[5052]: I1203 12:21:40.557803 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-734a-account-create-update-qcgft"] Dec 03 12:21:40 crc kubenswrapper[5052]: I1203 12:21:40.604891 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/817345c8-3a42-4dea-8158-75973d9c41d9-operator-scripts\") pod \"neutron-734a-account-create-update-qcgft\" (UID: \"817345c8-3a42-4dea-8158-75973d9c41d9\") " pod="openstack-kuttl-tests/neutron-734a-account-create-update-qcgft" Dec 03 12:21:40 crc kubenswrapper[5052]: I1203 12:21:40.604953 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7p76h\" (UniqueName: \"kubernetes.io/projected/817345c8-3a42-4dea-8158-75973d9c41d9-kube-api-access-7p76h\") pod \"neutron-734a-account-create-update-qcgft\" (UID: \"817345c8-3a42-4dea-8158-75973d9c41d9\") " pod="openstack-kuttl-tests/neutron-734a-account-create-update-qcgft" Dec 03 12:21:40 crc kubenswrapper[5052]: I1203 12:21:40.605495 5052 generic.go:334] "Generic (PLEG): container finished" podID="357bc965-8633-4d67-992e-baa770a0defd" containerID="97eb19cad24849878c7614c23d646abe200c279ae260c2f8f7bae07110bd2a2b" exitCode=137 Dec 03 12:21:40 crc kubenswrapper[5052]: I1203 12:21:40.641455 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/placement-db-create-r5z4q"] Dec 03 12:21:40 crc kubenswrapper[5052]: I1203 12:21:40.674850 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/placement-db-create-r5z4q"] Dec 03 12:21:40 crc kubenswrapper[5052]: I1203 12:21:40.692944 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:21:40 crc kubenswrapper[5052]: I1203 12:21:40.694000 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/placement-9152-account-create-update-sgs4t"] Dec 03 12:21:40 crc kubenswrapper[5052]: I1203 12:21:40.694031 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"779772f8-a0bd-491f-9153-33614d19e423","Type":"ContainerDied","Data":"95fd26e67c32872c11067f163126b665a158781319a98d2158a54a7bd7d0390d"} Dec 03 12:21:40 crc kubenswrapper[5052]: I1203 12:21:40.694068 5052 scope.go:117] "RemoveContainer" containerID="0e513984b9212a78bfe53fea749394bea6e44fa33a9bc3a945f96fe05465097a" Dec 03 12:21:40 crc kubenswrapper[5052]: I1203 12:21:40.707304 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/817345c8-3a42-4dea-8158-75973d9c41d9-operator-scripts\") pod \"neutron-734a-account-create-update-qcgft\" (UID: \"817345c8-3a42-4dea-8158-75973d9c41d9\") " pod="openstack-kuttl-tests/neutron-734a-account-create-update-qcgft" Dec 03 12:21:40 crc kubenswrapper[5052]: I1203 12:21:40.707393 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-7p76h\" (UniqueName: \"kubernetes.io/projected/817345c8-3a42-4dea-8158-75973d9c41d9-kube-api-access-7p76h\") pod \"neutron-734a-account-create-update-qcgft\" (UID: \"817345c8-3a42-4dea-8158-75973d9c41d9\") " pod="openstack-kuttl-tests/neutron-734a-account-create-update-qcgft" Dec 03 12:21:40 crc kubenswrapper[5052]: I1203 12:21:40.708290 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/817345c8-3a42-4dea-8158-75973d9c41d9-operator-scripts\") pod \"neutron-734a-account-create-update-qcgft\" (UID: \"817345c8-3a42-4dea-8158-75973d9c41d9\") " pod="openstack-kuttl-tests/neutron-734a-account-create-update-qcgft" Dec 03 12:21:40 crc kubenswrapper[5052]: I1203 12:21:40.713635 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-metadata-0" podStartSLOduration=2.7136158630000002 podStartE2EDuration="2.713615863s" podCreationTimestamp="2025-12-03 12:21:38 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:21:40.644897762 +0000 UTC m=+3691.233139536" watchObservedRunningTime="2025-12-03 12:21:40.713615863 +0000 UTC m=+3691.301857627" Dec 03 12:21:40 crc kubenswrapper[5052]: I1203 12:21:40.715463 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/placement-9152-account-create-update-sgs4t"] Dec 03 12:21:40 crc kubenswrapper[5052]: I1203 12:21:40.781477 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-7p76h\" (UniqueName: \"kubernetes.io/projected/817345c8-3a42-4dea-8158-75973d9c41d9-kube-api-access-7p76h\") pod \"neutron-734a-account-create-update-qcgft\" (UID: \"817345c8-3a42-4dea-8158-75973d9c41d9\") " pod="openstack-kuttl-tests/neutron-734a-account-create-update-qcgft" Dec 03 12:21:40 crc kubenswrapper[5052]: E1203 12:21:40.810634 5052 projected.go:263] Couldn't get secret openstack-kuttl-tests/swift-conf: secret "swift-conf" not found Dec 03 12:21:40 crc kubenswrapper[5052]: E1203 12:21:40.810678 5052 projected.go:194] Error preparing data for projected volume etc-swift for pod openstack-kuttl-tests/swift-proxy-6b9f9968d-5zjp2: secret "swift-conf" not found Dec 03 12:21:40 crc kubenswrapper[5052]: E1203 12:21:40.810730 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fb73cc9a-89cc-4cb8-87ce-2731e5dab0f7-etc-swift podName:fb73cc9a-89cc-4cb8-87ce-2731e5dab0f7 nodeName:}" failed. No retries permitted until 2025-12-03 12:21:41.310712445 +0000 UTC m=+3691.898954209 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "etc-swift" (UniqueName: "kubernetes.io/projected/fb73cc9a-89cc-4cb8-87ce-2731e5dab0f7-etc-swift") pod "swift-proxy-6b9f9968d-5zjp2" (UID: "fb73cc9a-89cc-4cb8-87ce-2731e5dab0f7") : secret "swift-conf" not found Dec 03 12:21:40 crc kubenswrapper[5052]: E1203 12:21:40.811179 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/combined-ca-bundle: secret "combined-ca-bundle" not found Dec 03 12:21:40 crc kubenswrapper[5052]: E1203 12:21:40.811221 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fb73cc9a-89cc-4cb8-87ce-2731e5dab0f7-combined-ca-bundle podName:fb73cc9a-89cc-4cb8-87ce-2731e5dab0f7 nodeName:}" failed. No retries permitted until 2025-12-03 12:21:41.311211329 +0000 UTC m=+3691.899453093 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "combined-ca-bundle" (UniqueName: "kubernetes.io/secret/fb73cc9a-89cc-4cb8-87ce-2731e5dab0f7-combined-ca-bundle") pod "swift-proxy-6b9f9968d-5zjp2" (UID: "fb73cc9a-89cc-4cb8-87ce-2731e5dab0f7") : secret "combined-ca-bundle" not found Dec 03 12:21:40 crc kubenswrapper[5052]: I1203 12:21:40.837346 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:21:40 crc kubenswrapper[5052]: I1203 12:21:40.858721 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/rabbitmq-server-0"] Dec 03 12:21:40 crc kubenswrapper[5052]: I1203 12:21:40.893332 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-734a-account-create-update-qcgft" Dec 03 12:21:40 crc kubenswrapper[5052]: I1203 12:21:40.905875 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:21:40 crc kubenswrapper[5052]: I1203 12:21:40.981268 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/neutron-db-create-g75vx"] Dec 03 12:21:41 crc kubenswrapper[5052]: I1203 12:21:41.002550 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/placement9152-account-delete-x7q46"] Dec 03 12:21:41 crc kubenswrapper[5052]: I1203 12:21:41.004189 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement9152-account-delete-x7q46" Dec 03 12:21:41 crc kubenswrapper[5052]: E1203 12:21:41.028179 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/rabbitmq-cell1-config-data: configmap "rabbitmq-cell1-config-data" not found Dec 03 12:21:41 crc kubenswrapper[5052]: E1203 12:21:41.028253 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c279eaf1-2ee2-46fe-88be-9a5a65d4897e-config-data podName:c279eaf1-2ee2-46fe-88be-9a5a65d4897e nodeName:}" failed. No retries permitted until 2025-12-03 12:21:42.028219953 +0000 UTC m=+3692.616461707 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "config-data" (UniqueName: "kubernetes.io/configmap/c279eaf1-2ee2-46fe-88be-9a5a65d4897e-config-data") pod "rabbitmq-cell1-server-0" (UID: "c279eaf1-2ee2-46fe-88be-9a5a65d4897e") : configmap "rabbitmq-cell1-config-data" not found Dec 03 12:21:41 crc kubenswrapper[5052]: E1203 12:21:41.029931 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/rabbitmq-config-data: configmap "rabbitmq-config-data" not found Dec 03 12:21:41 crc kubenswrapper[5052]: E1203 12:21:41.029959 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c14a906c-f19d-4806-9110-8c47b38c5ae0-config-data podName:c14a906c-f19d-4806-9110-8c47b38c5ae0 nodeName:}" failed. No retries permitted until 2025-12-03 12:21:41.529950312 +0000 UTC m=+3692.118192076 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "config-data" (UniqueName: "kubernetes.io/configmap/c14a906c-f19d-4806-9110-8c47b38c5ae0-config-data") pod "rabbitmq-server-0" (UID: "c14a906c-f19d-4806-9110-8c47b38c5ae0") : configmap "rabbitmq-config-data" not found Dec 03 12:21:41 crc kubenswrapper[5052]: I1203 12:21:41.040607 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/neutron-db-create-g75vx"] Dec 03 12:21:41 crc kubenswrapper[5052]: I1203 12:21:41.071376 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:21:41 crc kubenswrapper[5052]: I1203 12:21:41.072929 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:21:41 crc kubenswrapper[5052]: I1203 12:21:41.097737 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-scheduler-config-data" Dec 03 12:21:41 crc kubenswrapper[5052]: I1203 12:21:41.110244 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/placement9152-account-delete-x7q46"] Dec 03 12:21:41 crc kubenswrapper[5052]: I1203 12:21:41.123340 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/e3a73fa0-68ef-4e03-81f9-2a143ed805bf-config-data\") pod \"nova-scheduler-0\" (UID: \"e3a73fa0-68ef-4e03-81f9-2a143ed805bf\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:21:41 crc kubenswrapper[5052]: I1203 12:21:41.123399 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gtxmr\" (UniqueName: \"kubernetes.io/projected/e3a73fa0-68ef-4e03-81f9-2a143ed805bf-kube-api-access-gtxmr\") pod \"nova-scheduler-0\" (UID: \"e3a73fa0-68ef-4e03-81f9-2a143ed805bf\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:21:41 crc kubenswrapper[5052]: I1203 12:21:41.123445 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/4f7ab0ec-52a7-4be2-9886-04bf98989b63-operator-scripts\") pod \"placement9152-account-delete-x7q46\" (UID: \"4f7ab0ec-52a7-4be2-9886-04bf98989b63\") " pod="openstack-kuttl-tests/placement9152-account-delete-x7q46" Dec 03 12:21:41 crc kubenswrapper[5052]: I1203 12:21:41.123471 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qrtcw\" (UniqueName: \"kubernetes.io/projected/4f7ab0ec-52a7-4be2-9886-04bf98989b63-kube-api-access-qrtcw\") pod \"placement9152-account-delete-x7q46\" (UID: \"4f7ab0ec-52a7-4be2-9886-04bf98989b63\") " pod="openstack-kuttl-tests/placement9152-account-delete-x7q46" Dec 03 12:21:41 crc kubenswrapper[5052]: I1203 12:21:41.123786 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e3a73fa0-68ef-4e03-81f9-2a143ed805bf-combined-ca-bundle\") pod \"nova-scheduler-0\" (UID: \"e3a73fa0-68ef-4e03-81f9-2a143ed805bf\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:21:41 crc kubenswrapper[5052]: I1203 12:21:41.131251 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:21:41 crc kubenswrapper[5052]: I1203 12:21:41.196060 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-db-create-d6shg"] Dec 03 12:21:41 crc kubenswrapper[5052]: I1203 12:21:41.222589 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-73ca-account-create-update-c8r5x"] Dec 03 12:21:41 crc kubenswrapper[5052]: I1203 12:21:41.225793 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/e3a73fa0-68ef-4e03-81f9-2a143ed805bf-config-data\") pod \"nova-scheduler-0\" (UID: \"e3a73fa0-68ef-4e03-81f9-2a143ed805bf\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:21:41 crc kubenswrapper[5052]: I1203 12:21:41.225851 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-gtxmr\" (UniqueName: \"kubernetes.io/projected/e3a73fa0-68ef-4e03-81f9-2a143ed805bf-kube-api-access-gtxmr\") pod \"nova-scheduler-0\" (UID: \"e3a73fa0-68ef-4e03-81f9-2a143ed805bf\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:21:41 crc kubenswrapper[5052]: I1203 12:21:41.225899 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/4f7ab0ec-52a7-4be2-9886-04bf98989b63-operator-scripts\") pod \"placement9152-account-delete-x7q46\" (UID: \"4f7ab0ec-52a7-4be2-9886-04bf98989b63\") " pod="openstack-kuttl-tests/placement9152-account-delete-x7q46" Dec 03 12:21:41 crc kubenswrapper[5052]: I1203 12:21:41.225945 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-qrtcw\" (UniqueName: \"kubernetes.io/projected/4f7ab0ec-52a7-4be2-9886-04bf98989b63-kube-api-access-qrtcw\") pod \"placement9152-account-delete-x7q46\" (UID: \"4f7ab0ec-52a7-4be2-9886-04bf98989b63\") " pod="openstack-kuttl-tests/placement9152-account-delete-x7q46" Dec 03 12:21:41 crc kubenswrapper[5052]: I1203 12:21:41.226046 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e3a73fa0-68ef-4e03-81f9-2a143ed805bf-combined-ca-bundle\") pod \"nova-scheduler-0\" (UID: \"e3a73fa0-68ef-4e03-81f9-2a143ed805bf\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:21:41 crc kubenswrapper[5052]: E1203 12:21:41.226183 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/combined-ca-bundle: secret "combined-ca-bundle" not found Dec 03 12:21:41 crc kubenswrapper[5052]: E1203 12:21:41.226229 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3a73fa0-68ef-4e03-81f9-2a143ed805bf-combined-ca-bundle podName:e3a73fa0-68ef-4e03-81f9-2a143ed805bf nodeName:}" failed. No retries permitted until 2025-12-03 12:21:41.726214724 +0000 UTC m=+3692.314456488 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "combined-ca-bundle" (UniqueName: "kubernetes.io/secret/e3a73fa0-68ef-4e03-81f9-2a143ed805bf-combined-ca-bundle") pod "nova-scheduler-0" (UID: "e3a73fa0-68ef-4e03-81f9-2a143ed805bf") : secret "combined-ca-bundle" not found Dec 03 12:21:41 crc kubenswrapper[5052]: I1203 12:21:41.228561 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/4f7ab0ec-52a7-4be2-9886-04bf98989b63-operator-scripts\") pod \"placement9152-account-delete-x7q46\" (UID: \"4f7ab0ec-52a7-4be2-9886-04bf98989b63\") " pod="openstack-kuttl-tests/placement9152-account-delete-x7q46" Dec 03 12:21:41 crc kubenswrapper[5052]: I1203 12:21:41.232139 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/e3a73fa0-68ef-4e03-81f9-2a143ed805bf-config-data\") pod \"nova-scheduler-0\" (UID: \"e3a73fa0-68ef-4e03-81f9-2a143ed805bf\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:21:41 crc kubenswrapper[5052]: I1203 12:21:41.267196 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:21:41 crc kubenswrapper[5052]: I1203 12:21:41.298252 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/glance-73ca-account-create-update-c8r5x"] Dec 03 12:21:41 crc kubenswrapper[5052]: I1203 12:21:41.316641 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/glance-db-create-d6shg"] Dec 03 12:21:41 crc kubenswrapper[5052]: I1203 12:21:41.329095 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-qrtcw\" (UniqueName: \"kubernetes.io/projected/4f7ab0ec-52a7-4be2-9886-04bf98989b63-kube-api-access-qrtcw\") pod \"placement9152-account-delete-x7q46\" (UID: \"4f7ab0ec-52a7-4be2-9886-04bf98989b63\") " pod="openstack-kuttl-tests/placement9152-account-delete-x7q46" Dec 03 12:21:41 crc kubenswrapper[5052]: E1203 12:21:41.329236 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/combined-ca-bundle: secret "combined-ca-bundle" not found Dec 03 12:21:41 crc kubenswrapper[5052]: E1203 12:21:41.329290 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fb73cc9a-89cc-4cb8-87ce-2731e5dab0f7-combined-ca-bundle podName:fb73cc9a-89cc-4cb8-87ce-2731e5dab0f7 nodeName:}" failed. No retries permitted until 2025-12-03 12:21:42.329271295 +0000 UTC m=+3692.917513149 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "combined-ca-bundle" (UniqueName: "kubernetes.io/secret/fb73cc9a-89cc-4cb8-87ce-2731e5dab0f7-combined-ca-bundle") pod "swift-proxy-6b9f9968d-5zjp2" (UID: "fb73cc9a-89cc-4cb8-87ce-2731e5dab0f7") : secret "combined-ca-bundle" not found Dec 03 12:21:41 crc kubenswrapper[5052]: E1203 12:21:41.329584 5052 projected.go:263] Couldn't get secret openstack-kuttl-tests/swift-conf: secret "swift-conf" not found Dec 03 12:21:41 crc kubenswrapper[5052]: E1203 12:21:41.329599 5052 projected.go:288] Couldn't get configMap openstack-kuttl-tests/swift-ring-files: configmap "swift-ring-files" not found Dec 03 12:21:41 crc kubenswrapper[5052]: E1203 12:21:41.329610 5052 projected.go:194] Error preparing data for projected volume etc-swift for pod openstack-kuttl-tests/swift-proxy-6b9f9968d-5zjp2: [secret "swift-conf" not found, configmap "swift-ring-files" not found] Dec 03 12:21:41 crc kubenswrapper[5052]: E1203 12:21:41.329637 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fb73cc9a-89cc-4cb8-87ce-2731e5dab0f7-etc-swift podName:fb73cc9a-89cc-4cb8-87ce-2731e5dab0f7 nodeName:}" failed. No retries permitted until 2025-12-03 12:21:42.329628966 +0000 UTC m=+3692.917870830 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "etc-swift" (UniqueName: "kubernetes.io/projected/fb73cc9a-89cc-4cb8-87ce-2731e5dab0f7-etc-swift") pod "swift-proxy-6b9f9968d-5zjp2" (UID: "fb73cc9a-89cc-4cb8-87ce-2731e5dab0f7") : [secret "swift-conf" not found, configmap "swift-ring-files" not found] Dec 03 12:21:41 crc kubenswrapper[5052]: I1203 12:21:41.339499 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-gtxmr\" (UniqueName: \"kubernetes.io/projected/e3a73fa0-68ef-4e03-81f9-2a143ed805bf-kube-api-access-gtxmr\") pod \"nova-scheduler-0\" (UID: \"e3a73fa0-68ef-4e03-81f9-2a143ed805bf\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:21:41 crc kubenswrapper[5052]: I1203 12:21:41.352621 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-nb-0"] Dec 03 12:21:41 crc kubenswrapper[5052]: I1203 12:21:41.353263 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ovsdbserver-nb-0" podUID="70cedf3b-c9b9-4e5c-81a2-ba134b150997" containerName="openstack-network-exporter" containerID="cri-o://c7645a9210d2fc1e2e3f163388ea3b91c8a787603d06989bb0272c5699b638c7" gracePeriod=300 Dec 03 12:21:41 crc kubenswrapper[5052]: I1203 12:21:41.377576 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-api-8f5b-account-create-update-89qfl"] Dec 03 12:21:41 crc kubenswrapper[5052]: I1203 12:21:41.403765 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-api-8f5b-account-create-update-89qfl"] Dec 03 12:21:41 crc kubenswrapper[5052]: I1203 12:21:41.424141 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/glance73ca-account-delete-vpqhp"] Dec 03 12:21:41 crc kubenswrapper[5052]: I1203 12:21:41.425771 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance73ca-account-delete-vpqhp" Dec 03 12:21:41 crc kubenswrapper[5052]: I1203 12:21:41.443957 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance73ca-account-delete-vpqhp"] Dec 03 12:21:41 crc kubenswrapper[5052]: I1203 12:21:41.558800 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gg8jj\" (UniqueName: \"kubernetes.io/projected/db31774e-8f50-4862-86f3-93cb93f645a0-kube-api-access-gg8jj\") pod \"glance73ca-account-delete-vpqhp\" (UID: \"db31774e-8f50-4862-86f3-93cb93f645a0\") " pod="openstack-kuttl-tests/glance73ca-account-delete-vpqhp" Dec 03 12:21:41 crc kubenswrapper[5052]: I1203 12:21:41.558872 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/db31774e-8f50-4862-86f3-93cb93f645a0-operator-scripts\") pod \"glance73ca-account-delete-vpqhp\" (UID: \"db31774e-8f50-4862-86f3-93cb93f645a0\") " pod="openstack-kuttl-tests/glance73ca-account-delete-vpqhp" Dec 03 12:21:41 crc kubenswrapper[5052]: E1203 12:21:41.559187 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/rabbitmq-config-data: configmap "rabbitmq-config-data" not found Dec 03 12:21:41 crc kubenswrapper[5052]: E1203 12:21:41.559240 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c14a906c-f19d-4806-9110-8c47b38c5ae0-config-data podName:c14a906c-f19d-4806-9110-8c47b38c5ae0 nodeName:}" failed. No retries permitted until 2025-12-03 12:21:42.559221369 +0000 UTC m=+3693.147463133 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "config-data" (UniqueName: "kubernetes.io/configmap/c14a906c-f19d-4806-9110-8c47b38c5ae0-config-data") pod "rabbitmq-server-0" (UID: "c14a906c-f19d-4806-9110-8c47b38c5ae0") : configmap "rabbitmq-config-data" not found Dec 03 12:21:41 crc kubenswrapper[5052]: I1203 12:21:41.560069 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement9152-account-delete-x7q46" Dec 03 12:21:42 crc kubenswrapper[5052]: I1203 12:21:41.634202 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-api-8f5b-account-create-update-m9q79"] Dec 03 12:21:42 crc kubenswrapper[5052]: I1203 12:21:41.653452 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-8f5b-account-create-update-m9q79" Dec 03 12:21:42 crc kubenswrapper[5052]: I1203 12:21:41.668940 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-api-db-secret" Dec 03 12:21:42 crc kubenswrapper[5052]: I1203 12:21:41.674453 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-gg8jj\" (UniqueName: \"kubernetes.io/projected/db31774e-8f50-4862-86f3-93cb93f645a0-kube-api-access-gg8jj\") pod \"glance73ca-account-delete-vpqhp\" (UID: \"db31774e-8f50-4862-86f3-93cb93f645a0\") " pod="openstack-kuttl-tests/glance73ca-account-delete-vpqhp" Dec 03 12:21:42 crc kubenswrapper[5052]: I1203 12:21:41.674518 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/db31774e-8f50-4862-86f3-93cb93f645a0-operator-scripts\") pod \"glance73ca-account-delete-vpqhp\" (UID: \"db31774e-8f50-4862-86f3-93cb93f645a0\") " pod="openstack-kuttl-tests/glance73ca-account-delete-vpqhp" Dec 03 12:21:42 crc kubenswrapper[5052]: I1203 12:21:41.679787 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/db31774e-8f50-4862-86f3-93cb93f645a0-operator-scripts\") pod \"glance73ca-account-delete-vpqhp\" (UID: \"db31774e-8f50-4862-86f3-93cb93f645a0\") " pod="openstack-kuttl-tests/glance73ca-account-delete-vpqhp" Dec 03 12:21:42 crc kubenswrapper[5052]: I1203 12:21:41.769225 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-gg8jj\" (UniqueName: \"kubernetes.io/projected/db31774e-8f50-4862-86f3-93cb93f645a0-kube-api-access-gg8jj\") pod \"glance73ca-account-delete-vpqhp\" (UID: \"db31774e-8f50-4862-86f3-93cb93f645a0\") " pod="openstack-kuttl-tests/glance73ca-account-delete-vpqhp" Dec 03 12:21:42 crc kubenswrapper[5052]: I1203 12:21:41.776021 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hdxhc\" (UniqueName: \"kubernetes.io/projected/66954cf4-1c7b-4ca6-a6cf-b73bde8d670f-kube-api-access-hdxhc\") pod \"nova-api-8f5b-account-create-update-m9q79\" (UID: \"66954cf4-1c7b-4ca6-a6cf-b73bde8d670f\") " pod="openstack-kuttl-tests/nova-api-8f5b-account-create-update-m9q79" Dec 03 12:21:42 crc kubenswrapper[5052]: I1203 12:21:41.776173 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/66954cf4-1c7b-4ca6-a6cf-b73bde8d670f-operator-scripts\") pod \"nova-api-8f5b-account-create-update-m9q79\" (UID: \"66954cf4-1c7b-4ca6-a6cf-b73bde8d670f\") " pod="openstack-kuttl-tests/nova-api-8f5b-account-create-update-m9q79" Dec 03 12:21:42 crc kubenswrapper[5052]: I1203 12:21:41.776294 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e3a73fa0-68ef-4e03-81f9-2a143ed805bf-combined-ca-bundle\") pod \"nova-scheduler-0\" (UID: \"e3a73fa0-68ef-4e03-81f9-2a143ed805bf\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:21:42 crc kubenswrapper[5052]: E1203 12:21:41.776466 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/combined-ca-bundle: secret "combined-ca-bundle" not found Dec 03 12:21:42 crc kubenswrapper[5052]: E1203 12:21:41.776548 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3a73fa0-68ef-4e03-81f9-2a143ed805bf-combined-ca-bundle podName:e3a73fa0-68ef-4e03-81f9-2a143ed805bf nodeName:}" failed. No retries permitted until 2025-12-03 12:21:42.776506901 +0000 UTC m=+3693.364748665 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "combined-ca-bundle" (UniqueName: "kubernetes.io/secret/e3a73fa0-68ef-4e03-81f9-2a143ed805bf-combined-ca-bundle") pod "nova-scheduler-0" (UID: "e3a73fa0-68ef-4e03-81f9-2a143ed805bf") : secret "combined-ca-bundle" not found Dec 03 12:21:42 crc kubenswrapper[5052]: I1203 12:21:41.782134 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance73ca-account-delete-vpqhp" Dec 03 12:21:42 crc kubenswrapper[5052]: I1203 12:21:41.824978 5052 prober.go:107] "Probe failed" probeType="Liveness" pod="openstack-kuttl-tests/openstack-cell1-galera-0" podUID="47aa63e7-b7b2-4c1d-a73d-348b50cc5d2b" containerName="galera" probeResult="failure" output="command timed out" Dec 03 12:21:42 crc kubenswrapper[5052]: I1203 12:21:41.879797 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-hdxhc\" (UniqueName: \"kubernetes.io/projected/66954cf4-1c7b-4ca6-a6cf-b73bde8d670f-kube-api-access-hdxhc\") pod \"nova-api-8f5b-account-create-update-m9q79\" (UID: \"66954cf4-1c7b-4ca6-a6cf-b73bde8d670f\") " pod="openstack-kuttl-tests/nova-api-8f5b-account-create-update-m9q79" Dec 03 12:21:42 crc kubenswrapper[5052]: I1203 12:21:41.880029 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/66954cf4-1c7b-4ca6-a6cf-b73bde8d670f-operator-scripts\") pod \"nova-api-8f5b-account-create-update-m9q79\" (UID: \"66954cf4-1c7b-4ca6-a6cf-b73bde8d670f\") " pod="openstack-kuttl-tests/nova-api-8f5b-account-create-update-m9q79" Dec 03 12:21:42 crc kubenswrapper[5052]: E1203 12:21:41.880169 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/combined-ca-bundle: secret "combined-ca-bundle" not found Dec 03 12:21:42 crc kubenswrapper[5052]: E1203 12:21:41.880218 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/adcea736-eb15-4768-b12c-edb4c810beb3-combined-ca-bundle podName:adcea736-eb15-4768-b12c-edb4c810beb3 nodeName:}" failed. No retries permitted until 2025-12-03 12:21:42.38020185 +0000 UTC m=+3692.968443614 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "combined-ca-bundle" (UniqueName: "kubernetes.io/secret/adcea736-eb15-4768-b12c-edb4c810beb3-combined-ca-bundle") pod "memcached-0" (UID: "adcea736-eb15-4768-b12c-edb4c810beb3") : secret "combined-ca-bundle" not found Dec 03 12:21:42 crc kubenswrapper[5052]: E1203 12:21:41.880536 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-memcached-svc: secret "cert-memcached-svc" not found Dec 03 12:21:42 crc kubenswrapper[5052]: E1203 12:21:41.880565 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/adcea736-eb15-4768-b12c-edb4c810beb3-memcached-tls-certs podName:adcea736-eb15-4768-b12c-edb4c810beb3 nodeName:}" failed. No retries permitted until 2025-12-03 12:21:42.38055773 +0000 UTC m=+3692.968799554 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "memcached-tls-certs" (UniqueName: "kubernetes.io/secret/adcea736-eb15-4768-b12c-edb4c810beb3-memcached-tls-certs") pod "memcached-0" (UID: "adcea736-eb15-4768-b12c-edb4c810beb3") : secret "cert-memcached-svc" not found Dec 03 12:21:42 crc kubenswrapper[5052]: I1203 12:21:41.881863 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/66954cf4-1c7b-4ca6-a6cf-b73bde8d670f-operator-scripts\") pod \"nova-api-8f5b-account-create-update-m9q79\" (UID: \"66954cf4-1c7b-4ca6-a6cf-b73bde8d670f\") " pod="openstack-kuttl-tests/nova-api-8f5b-account-create-update-m9q79" Dec 03 12:21:42 crc kubenswrapper[5052]: I1203 12:21:41.958135 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-hdxhc\" (UniqueName: \"kubernetes.io/projected/66954cf4-1c7b-4ca6-a6cf-b73bde8d670f-kube-api-access-hdxhc\") pod \"nova-api-8f5b-account-create-update-m9q79\" (UID: \"66954cf4-1c7b-4ca6-a6cf-b73bde8d670f\") " pod="openstack-kuttl-tests/nova-api-8f5b-account-create-update-m9q79" Dec 03 12:21:42 crc kubenswrapper[5052]: I1203 12:21:41.974264 5052 generic.go:334] "Generic (PLEG): container finished" podID="4dd73a47-cf2a-43b8-ba1c-0d47332edaa4" containerID="dc90f1007187fa83ea0bb677d36177c1605dca45111209c9532edc07fddefddd" exitCode=143 Dec 03 12:21:42 crc kubenswrapper[5052]: I1203 12:21:41.974458 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/openstackclient" podUID="892f0ab2-8602-4485-a11c-5171252e2a7b" containerName="openstackclient" containerID="cri-o://5bdabbc1acf777d8b66b937a8a2bfb67eb3f1e9fde05ff4a60b0028fe9914ce7" gracePeriod=2 Dec 03 12:21:42 crc kubenswrapper[5052]: I1203 12:21:42.081146 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="03fcb214-234e-4e5a-8d9f-bbd2bbda66aa" path="/var/lib/kubelet/pods/03fcb214-234e-4e5a-8d9f-bbd2bbda66aa/volumes" Dec 03 12:21:42 crc kubenswrapper[5052]: E1203 12:21:42.086814 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/rabbitmq-cell1-config-data: configmap "rabbitmq-cell1-config-data" not found Dec 03 12:21:42 crc kubenswrapper[5052]: E1203 12:21:42.086877 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c279eaf1-2ee2-46fe-88be-9a5a65d4897e-config-data podName:c279eaf1-2ee2-46fe-88be-9a5a65d4897e nodeName:}" failed. No retries permitted until 2025-12-03 12:21:44.086862229 +0000 UTC m=+3694.675103983 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "config-data" (UniqueName: "kubernetes.io/configmap/c279eaf1-2ee2-46fe-88be-9a5a65d4897e-config-data") pod "rabbitmq-cell1-server-0" (UID: "c279eaf1-2ee2-46fe-88be-9a5a65d4897e") : configmap "rabbitmq-cell1-config-data" not found Dec 03 12:21:42 crc kubenswrapper[5052]: I1203 12:21:42.094683 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ovsdbserver-nb-0" podUID="70cedf3b-c9b9-4e5c-81a2-ba134b150997" containerName="ovsdbserver-nb" containerID="cri-o://f84d0df84d3cb24dd97faa8c986cf4682cc548000edcdfa7e202ea04d4fd81e2" gracePeriod=300 Dec 03 12:21:42 crc kubenswrapper[5052]: I1203 12:21:42.102918 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5e64bff0-44a3-472d-8b94-210a31a0e4e6" path="/var/lib/kubelet/pods/5e64bff0-44a3-472d-8b94-210a31a0e4e6/volumes" Dec 03 12:21:42 crc kubenswrapper[5052]: I1203 12:21:42.132182 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6b4e9ce4-5733-4df7-a4f8-f3d23e906b53" path="/var/lib/kubelet/pods/6b4e9ce4-5733-4df7-a4f8-f3d23e906b53/volumes" Dec 03 12:21:42 crc kubenswrapper[5052]: I1203 12:21:42.132953 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="779772f8-a0bd-491f-9153-33614d19e423" path="/var/lib/kubelet/pods/779772f8-a0bd-491f-9153-33614d19e423/volumes" Dec 03 12:21:42 crc kubenswrapper[5052]: I1203 12:21:42.133659 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9b7bd5ab-a50e-4a66-ade7-c09c913c481f" path="/var/lib/kubelet/pods/9b7bd5ab-a50e-4a66-ade7-c09c913c481f/volumes" Dec 03 12:21:42 crc kubenswrapper[5052]: I1203 12:21:42.141544 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b3c91987-f9ac-46b0-a634-33c314ab4576" path="/var/lib/kubelet/pods/b3c91987-f9ac-46b0-a634-33c314ab4576/volumes" Dec 03 12:21:42 crc kubenswrapper[5052]: I1203 12:21:42.142246 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ee933fcd-48c9-46a1-badd-bc87089bc1e9" path="/var/lib/kubelet/pods/ee933fcd-48c9-46a1-badd-bc87089bc1e9/volumes" Dec 03 12:21:42 crc kubenswrapper[5052]: I1203 12:21:42.142904 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f30ededb-4c5b-4874-97c8-f65e4fe0421d" path="/var/lib/kubelet/pods/f30ededb-4c5b-4874-97c8-f65e4fe0421d/volumes" Dec 03 12:21:42 crc kubenswrapper[5052]: I1203 12:21:42.147043 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/placement-db-sync-gzzkp"] Dec 03 12:21:42 crc kubenswrapper[5052]: I1203 12:21:42.147090 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"9dec0a31-cba3-4c38-8b5d-4358f16b3704","Type":"ContainerStarted","Data":"301bbf49f5f1e1267a4886e2478668c30d4e45b89f48d5139fe9429743fa8ca6"} Dec 03 12:21:42 crc kubenswrapper[5052]: I1203 12:21:42.147118 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"4dd73a47-cf2a-43b8-ba1c-0d47332edaa4","Type":"ContainerDied","Data":"dc90f1007187fa83ea0bb677d36177c1605dca45111209c9532edc07fddefddd"} Dec 03 12:21:42 crc kubenswrapper[5052]: I1203 12:21:42.147138 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-api-db-create-76nlc"] Dec 03 12:21:42 crc kubenswrapper[5052]: I1203 12:21:42.147151 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/neutron-db-sync-lrqq6"] Dec 03 12:21:42 crc kubenswrapper[5052]: I1203 12:21:42.147162 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder-2067-account-create-update-9rsn5"] Dec 03 12:21:42 crc kubenswrapper[5052]: I1203 12:21:42.147174 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder-db-create-d6bcr"] Dec 03 12:21:42 crc kubenswrapper[5052]: I1203 12:21:42.147187 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-api-db-create-76nlc"] Dec 03 12:21:42 crc kubenswrapper[5052]: I1203 12:21:42.147205 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/cinder-2067-account-create-update-9rsn5"] Dec 03 12:21:42 crc kubenswrapper[5052]: I1203 12:21:42.147218 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/cinder-db-create-d6bcr"] Dec 03 12:21:42 crc kubenswrapper[5052]: I1203 12:21:42.147230 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/neutron-db-sync-lrqq6"] Dec 03 12:21:42 crc kubenswrapper[5052]: I1203 12:21:42.147243 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-8f5b-account-create-update-m9q79"] Dec 03 12:21:42 crc kubenswrapper[5052]: I1203 12:21:42.147258 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/placement-db-sync-gzzkp"] Dec 03 12:21:42 crc kubenswrapper[5052]: I1203 12:21:42.147271 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-0e0e-account-create-update-cscpf"] Dec 03 12:21:42 crc kubenswrapper[5052]: I1203 12:21:42.147282 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/barbican-0e0e-account-create-update-cscpf"] Dec 03 12:21:42 crc kubenswrapper[5052]: E1203 12:21:42.159174 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: EOF, stdout: , stderr: , exit code -1" containerID="f84d0df84d3cb24dd97faa8c986cf4682cc548000edcdfa7e202ea04d4fd81e2" cmd=["/usr/bin/pidof","ovsdb-server"] Dec 03 12:21:42 crc kubenswrapper[5052]: I1203 12:21:42.160726 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-db-create-sj945"] Dec 03 12:21:42 crc kubenswrapper[5052]: E1203 12:21:42.165217 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of f84d0df84d3cb24dd97faa8c986cf4682cc548000edcdfa7e202ea04d4fd81e2 is running failed: container process not found" containerID="f84d0df84d3cb24dd97faa8c986cf4682cc548000edcdfa7e202ea04d4fd81e2" cmd=["/usr/bin/pidof","ovsdb-server"] Dec 03 12:21:42 crc kubenswrapper[5052]: E1203 12:21:42.167486 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of f84d0df84d3cb24dd97faa8c986cf4682cc548000edcdfa7e202ea04d4fd81e2 is running failed: container process not found" containerID="f84d0df84d3cb24dd97faa8c986cf4682cc548000edcdfa7e202ea04d4fd81e2" cmd=["/usr/bin/pidof","ovsdb-server"] Dec 03 12:21:42 crc kubenswrapper[5052]: E1203 12:21:42.167576 5052 prober.go:104] "Probe errored" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of f84d0df84d3cb24dd97faa8c986cf4682cc548000edcdfa7e202ea04d4fd81e2 is running failed: container process not found" probeType="Readiness" pod="openstack-kuttl-tests/ovsdbserver-nb-0" podUID="70cedf3b-c9b9-4e5c-81a2-ba134b150997" containerName="ovsdbserver-nb" Dec 03 12:21:42 crc kubenswrapper[5052]: I1203 12:21:42.177877 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/barbican-db-create-sj945"] Dec 03 12:21:42 crc kubenswrapper[5052]: I1203 12:21:42.193024 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-sb-0"] Dec 03 12:21:42 crc kubenswrapper[5052]: I1203 12:21:42.193671 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ovsdbserver-sb-0" podUID="19a09467-4c91-40f5-b83a-d43a8eb5f2dc" containerName="openstack-network-exporter" containerID="cri-o://81e8dead6ae4e8433ea10d47201edd08f411e7f097f98bf35c829bcf51a695f9" gracePeriod=300 Dec 03 12:21:42 crc kubenswrapper[5052]: I1203 12:21:42.210589 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder-db-sync-gqt2p"] Dec 03 12:21:42 crc kubenswrapper[5052]: I1203 12:21:42.237029 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/cinder-db-sync-gqt2p"] Dec 03 12:21:42 crc kubenswrapper[5052]: I1203 12:21:42.263612 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell0-0e20-account-create-update-5hn8s"] Dec 03 12:21:42 crc kubenswrapper[5052]: I1203 12:21:42.288326 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell0-0e20-account-create-update-5hn8s"] Dec 03 12:21:42 crc kubenswrapper[5052]: I1203 12:21:42.293231 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ovsdbserver-sb-0" podUID="19a09467-4c91-40f5-b83a-d43a8eb5f2dc" containerName="ovsdbserver-sb" containerID="cri-o://5c0846c17b1ab7bae05ee7ab757a5ac9bfa9ad58f76d6d21d76a677a190e3f91" gracePeriod=300 Dec 03 12:21:42 crc kubenswrapper[5052]: I1203 12:21:42.300203 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/neutron734a-account-delete-sxpn9"] Dec 03 12:21:42 crc kubenswrapper[5052]: I1203 12:21:42.301869 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron734a-account-delete-sxpn9" Dec 03 12:21:42 crc kubenswrapper[5052]: I1203 12:21:42.320437 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron734a-account-delete-sxpn9"] Dec 03 12:21:42 crc kubenswrapper[5052]: I1203 12:21:42.360650 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-db-sync-92z8b"] Dec 03 12:21:42 crc kubenswrapper[5052]: I1203 12:21:42.377878 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-db-sync-pvgq9"] Dec 03 12:21:42 crc kubenswrapper[5052]: I1203 12:21:42.397211 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ovn-northd-0"] Dec 03 12:21:42 crc kubenswrapper[5052]: I1203 12:21:42.397493 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ovn-northd-0" podUID="a77df94c-3e83-47a5-9b5f-eb0aeb41bc3d" containerName="ovn-northd" containerID="cri-o://97a957b8c33e1b7a5cef18f4383a4faea413910406298074a6052c04396cec18" gracePeriod=30 Dec 03 12:21:42 crc kubenswrapper[5052]: I1203 12:21:42.397889 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell0-db-create-2vxt6"] Dec 03 12:21:42 crc kubenswrapper[5052]: I1203 12:21:42.398126 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ovn-northd-0" podUID="a77df94c-3e83-47a5-9b5f-eb0aeb41bc3d" containerName="openstack-network-exporter" containerID="cri-o://cf3832a4b1ebcd0697f163ca295f373357a42dc4f1fd311347b02c08c7239df4" gracePeriod=30 Dec 03 12:21:42 crc kubenswrapper[5052]: I1203 12:21:42.409425 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/38427938-9a58-4f07-9f57-061ee6693a0d-operator-scripts\") pod \"neutron734a-account-delete-sxpn9\" (UID: \"38427938-9a58-4f07-9f57-061ee6693a0d\") " pod="openstack-kuttl-tests/neutron734a-account-delete-sxpn9" Dec 03 12:21:42 crc kubenswrapper[5052]: I1203 12:21:42.409477 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bx7p4\" (UniqueName: \"kubernetes.io/projected/38427938-9a58-4f07-9f57-061ee6693a0d-kube-api-access-bx7p4\") pod \"neutron734a-account-delete-sxpn9\" (UID: \"38427938-9a58-4f07-9f57-061ee6693a0d\") " pod="openstack-kuttl-tests/neutron734a-account-delete-sxpn9" Dec 03 12:21:42 crc kubenswrapper[5052]: E1203 12:21:42.409480 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-memcached-svc: secret "cert-memcached-svc" not found Dec 03 12:21:42 crc kubenswrapper[5052]: E1203 12:21:42.409571 5052 projected.go:263] Couldn't get secret openstack-kuttl-tests/swift-conf: secret "swift-conf" not found Dec 03 12:21:42 crc kubenswrapper[5052]: E1203 12:21:42.409693 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/adcea736-eb15-4768-b12c-edb4c810beb3-memcached-tls-certs podName:adcea736-eb15-4768-b12c-edb4c810beb3 nodeName:}" failed. No retries permitted until 2025-12-03 12:21:43.409667032 +0000 UTC m=+3693.997908796 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "memcached-tls-certs" (UniqueName: "kubernetes.io/secret/adcea736-eb15-4768-b12c-edb4c810beb3-memcached-tls-certs") pod "memcached-0" (UID: "adcea736-eb15-4768-b12c-edb4c810beb3") : secret "cert-memcached-svc" not found Dec 03 12:21:42 crc kubenswrapper[5052]: E1203 12:21:42.409702 5052 projected.go:288] Couldn't get configMap openstack-kuttl-tests/swift-ring-files: configmap "swift-ring-files" not found Dec 03 12:21:42 crc kubenswrapper[5052]: E1203 12:21:42.409722 5052 projected.go:194] Error preparing data for projected volume etc-swift for pod openstack-kuttl-tests/swift-proxy-6b9f9968d-5zjp2: [secret "swift-conf" not found, configmap "swift-ring-files" not found] Dec 03 12:21:42 crc kubenswrapper[5052]: E1203 12:21:42.409786 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fb73cc9a-89cc-4cb8-87ce-2731e5dab0f7-etc-swift podName:fb73cc9a-89cc-4cb8-87ce-2731e5dab0f7 nodeName:}" failed. No retries permitted until 2025-12-03 12:21:44.409765655 +0000 UTC m=+3694.998007419 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "etc-swift" (UniqueName: "kubernetes.io/projected/fb73cc9a-89cc-4cb8-87ce-2731e5dab0f7-etc-swift") pod "swift-proxy-6b9f9968d-5zjp2" (UID: "fb73cc9a-89cc-4cb8-87ce-2731e5dab0f7") : [secret "swift-conf" not found, configmap "swift-ring-files" not found] Dec 03 12:21:42 crc kubenswrapper[5052]: E1203 12:21:42.409612 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/combined-ca-bundle: secret "combined-ca-bundle" not found Dec 03 12:21:42 crc kubenswrapper[5052]: E1203 12:21:42.409824 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/adcea736-eb15-4768-b12c-edb4c810beb3-combined-ca-bundle podName:adcea736-eb15-4768-b12c-edb4c810beb3 nodeName:}" failed. No retries permitted until 2025-12-03 12:21:43.409819297 +0000 UTC m=+3693.998061061 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "combined-ca-bundle" (UniqueName: "kubernetes.io/secret/adcea736-eb15-4768-b12c-edb4c810beb3-combined-ca-bundle") pod "memcached-0" (UID: "adcea736-eb15-4768-b12c-edb4c810beb3") : secret "combined-ca-bundle" not found Dec 03 12:21:42 crc kubenswrapper[5052]: E1203 12:21:42.409827 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/combined-ca-bundle: secret "combined-ca-bundle" not found Dec 03 12:21:42 crc kubenswrapper[5052]: E1203 12:21:42.409873 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fb73cc9a-89cc-4cb8-87ce-2731e5dab0f7-combined-ca-bundle podName:fb73cc9a-89cc-4cb8-87ce-2731e5dab0f7 nodeName:}" failed. No retries permitted until 2025-12-03 12:21:44.409851438 +0000 UTC m=+3694.998093292 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "combined-ca-bundle" (UniqueName: "kubernetes.io/secret/fb73cc9a-89cc-4cb8-87ce-2731e5dab0f7-combined-ca-bundle") pod "swift-proxy-6b9f9968d-5zjp2" (UID: "fb73cc9a-89cc-4cb8-87ce-2731e5dab0f7") : secret "combined-ca-bundle" not found Dec 03 12:21:42 crc kubenswrapper[5052]: I1203 12:21:42.423600 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/glance-db-sync-pvgq9"] Dec 03 12:21:42 crc kubenswrapper[5052]: I1203 12:21:42.432568 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/barbican-db-sync-92z8b"] Dec 03 12:21:42 crc kubenswrapper[5052]: I1203 12:21:42.441175 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell0-db-create-2vxt6"] Dec 03 12:21:42 crc kubenswrapper[5052]: I1203 12:21:42.448019 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell1-3fba-account-create-update-7dlvh"] Dec 03 12:21:42 crc kubenswrapper[5052]: I1203 12:21:42.456499 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell1-3fba-account-create-update-7dlvh"] Dec 03 12:21:42 crc kubenswrapper[5052]: I1203 12:21:42.468646 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/swift-ring-rebalance-z7lpm"] Dec 03 12:21:42 crc kubenswrapper[5052]: I1203 12:21:42.477914 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/swift-ring-rebalance-z7lpm"] Dec 03 12:21:42 crc kubenswrapper[5052]: I1203 12:21:42.487708 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/placement-5447868864-mfdcg"] Dec 03 12:21:42 crc kubenswrapper[5052]: I1203 12:21:42.488015 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/placement-5447868864-mfdcg" podUID="90814ea7-1abb-4760-bbfa-1190f23ab38a" containerName="placement-log" containerID="cri-o://f82b984cb5939c510f6703add68625e1e534cdf59488ddc53cb91e355ac65330" gracePeriod=30 Dec 03 12:21:42 crc kubenswrapper[5052]: I1203 12:21:42.488566 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/placement-5447868864-mfdcg" podUID="90814ea7-1abb-4760-bbfa-1190f23ab38a" containerName="placement-api" containerID="cri-o://0c61528c202083e822a6d60926f5531661737d72572cc3938c0bb3db2f2fcd35" gracePeriod=30 Dec 03 12:21:42 crc kubenswrapper[5052]: I1203 12:21:42.498450 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/cinder2067-account-delete-n2tf7"] Dec 03 12:21:42 crc kubenswrapper[5052]: I1203 12:21:42.500125 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder2067-account-delete-n2tf7" Dec 03 12:21:42 crc kubenswrapper[5052]: I1203 12:21:42.511730 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/38427938-9a58-4f07-9f57-061ee6693a0d-operator-scripts\") pod \"neutron734a-account-delete-sxpn9\" (UID: \"38427938-9a58-4f07-9f57-061ee6693a0d\") " pod="openstack-kuttl-tests/neutron734a-account-delete-sxpn9" Dec 03 12:21:42 crc kubenswrapper[5052]: I1203 12:21:42.511763 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-bx7p4\" (UniqueName: \"kubernetes.io/projected/38427938-9a58-4f07-9f57-061ee6693a0d-kube-api-access-bx7p4\") pod \"neutron734a-account-delete-sxpn9\" (UID: \"38427938-9a58-4f07-9f57-061ee6693a0d\") " pod="openstack-kuttl-tests/neutron734a-account-delete-sxpn9" Dec 03 12:21:42 crc kubenswrapper[5052]: I1203 12:21:42.511839 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/neutron-69c9545844-xf6zm"] Dec 03 12:21:42 crc kubenswrapper[5052]: I1203 12:21:42.512089 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/neutron-69c9545844-xf6zm" podUID="03bb65ef-e9a8-4cfb-9536-001e316b1f72" containerName="neutron-api" containerID="cri-o://48fa168d5d7691a3b2d8508087753330d79877c968cc2b4e0f021b0441800e44" gracePeriod=30 Dec 03 12:21:42 crc kubenswrapper[5052]: I1203 12:21:42.512611 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/neutron-69c9545844-xf6zm" podUID="03bb65ef-e9a8-4cfb-9536-001e316b1f72" containerName="neutron-httpd" containerID="cri-o://ece24b1db9c0709fd516d1a80d474c2937fdeb49dd573043245a6e0d4d27d04a" gracePeriod=30 Dec 03 12:21:42 crc kubenswrapper[5052]: I1203 12:21:42.514154 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/38427938-9a58-4f07-9f57-061ee6693a0d-operator-scripts\") pod \"neutron734a-account-delete-sxpn9\" (UID: \"38427938-9a58-4f07-9f57-061ee6693a0d\") " pod="openstack-kuttl-tests/neutron734a-account-delete-sxpn9" Dec 03 12:21:42 crc kubenswrapper[5052]: I1203 12:21:42.535162 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-bx7p4\" (UniqueName: \"kubernetes.io/projected/38427938-9a58-4f07-9f57-061ee6693a0d-kube-api-access-bx7p4\") pod \"neutron734a-account-delete-sxpn9\" (UID: \"38427938-9a58-4f07-9f57-061ee6693a0d\") " pod="openstack-kuttl-tests/neutron734a-account-delete-sxpn9" Dec 03 12:21:42 crc kubenswrapper[5052]: I1203 12:21:42.556067 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder2067-account-delete-n2tf7"] Dec 03 12:21:42 crc kubenswrapper[5052]: I1203 12:21:42.588571 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell1-cell-mapping-qr65w"] Dec 03 12:21:42 crc kubenswrapper[5052]: I1203 12:21:42.625556 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pd777\" (UniqueName: \"kubernetes.io/projected/f4b213b4-e7bf-475f-8a2a-8c28437c32e0-kube-api-access-pd777\") pod \"cinder2067-account-delete-n2tf7\" (UID: \"f4b213b4-e7bf-475f-8a2a-8c28437c32e0\") " pod="openstack-kuttl-tests/cinder2067-account-delete-n2tf7" Dec 03 12:21:42 crc kubenswrapper[5052]: I1203 12:21:42.625671 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/f4b213b4-e7bf-475f-8a2a-8c28437c32e0-operator-scripts\") pod \"cinder2067-account-delete-n2tf7\" (UID: \"f4b213b4-e7bf-475f-8a2a-8c28437c32e0\") " pod="openstack-kuttl-tests/cinder2067-account-delete-n2tf7" Dec 03 12:21:42 crc kubenswrapper[5052]: E1203 12:21:42.626279 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/rabbitmq-config-data: configmap "rabbitmq-config-data" not found Dec 03 12:21:42 crc kubenswrapper[5052]: E1203 12:21:42.626354 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c14a906c-f19d-4806-9110-8c47b38c5ae0-config-data podName:c14a906c-f19d-4806-9110-8c47b38c5ae0 nodeName:}" failed. No retries permitted until 2025-12-03 12:21:44.626327586 +0000 UTC m=+3695.214569350 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "config-data" (UniqueName: "kubernetes.io/configmap/c14a906c-f19d-4806-9110-8c47b38c5ae0-config-data") pod "rabbitmq-server-0" (UID: "c14a906c-f19d-4806-9110-8c47b38c5ae0") : configmap "rabbitmq-config-data" not found Dec 03 12:21:42 crc kubenswrapper[5052]: I1203 12:21:42.652932 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell0-cell-mapping-6dkwd"] Dec 03 12:21:42 crc kubenswrapper[5052]: I1203 12:21:42.723543 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell1-cell-mapping-qr65w"] Dec 03 12:21:42 crc kubenswrapper[5052]: I1203 12:21:42.739939 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-pd777\" (UniqueName: \"kubernetes.io/projected/f4b213b4-e7bf-475f-8a2a-8c28437c32e0-kube-api-access-pd777\") pod \"cinder2067-account-delete-n2tf7\" (UID: \"f4b213b4-e7bf-475f-8a2a-8c28437c32e0\") " pod="openstack-kuttl-tests/cinder2067-account-delete-n2tf7" Dec 03 12:21:42 crc kubenswrapper[5052]: I1203 12:21:42.740049 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/f4b213b4-e7bf-475f-8a2a-8c28437c32e0-operator-scripts\") pod \"cinder2067-account-delete-n2tf7\" (UID: \"f4b213b4-e7bf-475f-8a2a-8c28437c32e0\") " pod="openstack-kuttl-tests/cinder2067-account-delete-n2tf7" Dec 03 12:21:42 crc kubenswrapper[5052]: I1203 12:21:42.741439 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/f4b213b4-e7bf-475f-8a2a-8c28437c32e0-operator-scripts\") pod \"cinder2067-account-delete-n2tf7\" (UID: \"f4b213b4-e7bf-475f-8a2a-8c28437c32e0\") " pod="openstack-kuttl-tests/cinder2067-account-delete-n2tf7" Dec 03 12:21:42 crc kubenswrapper[5052]: I1203 12:21:42.780289 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-pd777\" (UniqueName: \"kubernetes.io/projected/f4b213b4-e7bf-475f-8a2a-8c28437c32e0-kube-api-access-pd777\") pod \"cinder2067-account-delete-n2tf7\" (UID: \"f4b213b4-e7bf-475f-8a2a-8c28437c32e0\") " pod="openstack-kuttl-tests/cinder2067-account-delete-n2tf7" Dec 03 12:21:42 crc kubenswrapper[5052]: I1203 12:21:42.798751 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell1-db-create-tczzp"] Dec 03 12:21:42 crc kubenswrapper[5052]: I1203 12:21:42.825707 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell1-db-create-tczzp"] Dec 03 12:21:42 crc kubenswrapper[5052]: W1203 12:21:42.833676 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddb31774e_8f50_4862_86f3_93cb93f645a0.slice/crio-d52fa6a791c4db9bacf26c89f59fbe21fefb28a88af961341f4ea8108f037da9 WatchSource:0}: Error finding container d52fa6a791c4db9bacf26c89f59fbe21fefb28a88af961341f4ea8108f037da9: Status 404 returned error can't find the container with id d52fa6a791c4db9bacf26c89f59fbe21fefb28a88af961341f4ea8108f037da9 Dec 03 12:21:42 crc kubenswrapper[5052]: I1203 12:21:42.841465 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e3a73fa0-68ef-4e03-81f9-2a143ed805bf-combined-ca-bundle\") pod \"nova-scheduler-0\" (UID: \"e3a73fa0-68ef-4e03-81f9-2a143ed805bf\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:21:42 crc kubenswrapper[5052]: E1203 12:21:42.841685 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/combined-ca-bundle: secret "combined-ca-bundle" not found Dec 03 12:21:42 crc kubenswrapper[5052]: I1203 12:21:42.841793 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/swift-storage-0"] Dec 03 12:21:42 crc kubenswrapper[5052]: E1203 12:21:42.841887 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3a73fa0-68ef-4e03-81f9-2a143ed805bf-combined-ca-bundle podName:e3a73fa0-68ef-4e03-81f9-2a143ed805bf nodeName:}" failed. No retries permitted until 2025-12-03 12:21:44.841869989 +0000 UTC m=+3695.430111753 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "combined-ca-bundle" (UniqueName: "kubernetes.io/secret/e3a73fa0-68ef-4e03-81f9-2a143ed805bf-combined-ca-bundle") pod "nova-scheduler-0" (UID: "e3a73fa0-68ef-4e03-81f9-2a143ed805bf") : secret "combined-ca-bundle" not found Dec 03 12:21:42 crc kubenswrapper[5052]: I1203 12:21:42.842287 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="0b121234-d24f-4307-9409-59a7dad3df4c" containerName="account-server" containerID="cri-o://6b7c8f662acc40a3137f223890e54394fd6013bb9203453b7285ac28a63f277d" gracePeriod=30 Dec 03 12:21:42 crc kubenswrapper[5052]: I1203 12:21:42.842774 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="0b121234-d24f-4307-9409-59a7dad3df4c" containerName="swift-recon-cron" containerID="cri-o://e63a7b1f0f328636f34f8466d7114142477116b8ba39ff4c63590f6d3d2ecb61" gracePeriod=30 Dec 03 12:21:42 crc kubenswrapper[5052]: I1203 12:21:42.842825 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="0b121234-d24f-4307-9409-59a7dad3df4c" containerName="rsync" containerID="cri-o://eac0ccbf31e29898c07277ff8b3f25e720d684db3945841e058ea63b8f2fc50e" gracePeriod=30 Dec 03 12:21:42 crc kubenswrapper[5052]: I1203 12:21:42.842856 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="0b121234-d24f-4307-9409-59a7dad3df4c" containerName="object-expirer" containerID="cri-o://9826c564d51675cc0d19bfb280f54c6cdb8da70e11a2735f3456a7c16c5b0331" gracePeriod=30 Dec 03 12:21:42 crc kubenswrapper[5052]: I1203 12:21:42.842887 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="0b121234-d24f-4307-9409-59a7dad3df4c" containerName="object-updater" containerID="cri-o://38ac8f54d7f675400878ab50b896bde2d4d40ee63497af23ebdca0f75d215cd7" gracePeriod=30 Dec 03 12:21:42 crc kubenswrapper[5052]: I1203 12:21:42.842921 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="0b121234-d24f-4307-9409-59a7dad3df4c" containerName="object-auditor" containerID="cri-o://a108f5ccd3086ba9eada83058d904fc18a5b2e825c993e540cb30d0f9ced71e9" gracePeriod=30 Dec 03 12:21:42 crc kubenswrapper[5052]: I1203 12:21:42.842951 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="0b121234-d24f-4307-9409-59a7dad3df4c" containerName="object-replicator" containerID="cri-o://a7be0eed6906ff248e67daeb0cb8433a1ab73cf3e47159ff1cbfe4029900aa36" gracePeriod=30 Dec 03 12:21:42 crc kubenswrapper[5052]: I1203 12:21:42.842990 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="0b121234-d24f-4307-9409-59a7dad3df4c" containerName="object-server" containerID="cri-o://93d4d6d903155179da2cc815475bcc4c3e646bd25173aec67f526949d47734f0" gracePeriod=30 Dec 03 12:21:42 crc kubenswrapper[5052]: I1203 12:21:42.843021 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="0b121234-d24f-4307-9409-59a7dad3df4c" containerName="container-updater" containerID="cri-o://280a6f9eeba9635ba75d3f89698a54d959866a87a84e1c34632aae63f6162f88" gracePeriod=30 Dec 03 12:21:42 crc kubenswrapper[5052]: I1203 12:21:42.843052 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="0b121234-d24f-4307-9409-59a7dad3df4c" containerName="container-auditor" containerID="cri-o://76aa0661476cea8be8d11604d15492437f317021e6ae641e33f6ca912d72fe2e" gracePeriod=30 Dec 03 12:21:42 crc kubenswrapper[5052]: I1203 12:21:42.843084 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="0b121234-d24f-4307-9409-59a7dad3df4c" containerName="container-replicator" containerID="cri-o://1fb344079bdd4d88652a84cde52ac882ef46f1e1b483946b91fa06958b9781b1" gracePeriod=30 Dec 03 12:21:42 crc kubenswrapper[5052]: I1203 12:21:42.843112 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="0b121234-d24f-4307-9409-59a7dad3df4c" containerName="container-server" containerID="cri-o://7448a246780facb87faed946ab976ad223f836a90fd0ac6cbf876c7f11c6c2d8" gracePeriod=30 Dec 03 12:21:42 crc kubenswrapper[5052]: I1203 12:21:42.843146 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="0b121234-d24f-4307-9409-59a7dad3df4c" containerName="account-reaper" containerID="cri-o://70f7ea9905786e034fff8dcf46682040188eb7ca7903f795173a38888c5b11d9" gracePeriod=30 Dec 03 12:21:42 crc kubenswrapper[5052]: I1203 12:21:42.843183 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="0b121234-d24f-4307-9409-59a7dad3df4c" containerName="account-auditor" containerID="cri-o://1de28481ebd24dff85a8213026dad78dd91bf4d0b6c4d77c2b5231b95244a4cb" gracePeriod=30 Dec 03 12:21:42 crc kubenswrapper[5052]: I1203 12:21:42.843222 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="0b121234-d24f-4307-9409-59a7dad3df4c" containerName="account-replicator" containerID="cri-o://914125fe1587ea7c446a5334d9e2828cf4834d6ad4f2053f842f07ad24a0e67d" gracePeriod=30 Dec 03 12:21:42 crc kubenswrapper[5052]: I1203 12:21:42.853370 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/rabbitmq-cell1-server-0"] Dec 03 12:21:42 crc kubenswrapper[5052]: I1203 12:21:42.875544 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell0-cell-mapping-6dkwd"] Dec 03 12:21:42 crc kubenswrapper[5052]: I1203 12:21:42.885637 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 12:21:42 crc kubenswrapper[5052]: I1203 12:21:42.885917 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/cinder-scheduler-0" podUID="4a55b492-d57f-40dc-b719-1294fb24ef13" containerName="cinder-scheduler" containerID="cri-o://84022ca2f2789fb89a8aa147aadbca38260a93e0dfa5f25cefce8f679ca30454" gracePeriod=30 Dec 03 12:21:42 crc kubenswrapper[5052]: I1203 12:21:42.886327 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/cinder-scheduler-0" podUID="4a55b492-d57f-40dc-b719-1294fb24ef13" containerName="probe" containerID="cri-o://4fdd95681555772f2043566122b6cc7d5d50f1b3bc46512f2b2e8fedf1c6d999" gracePeriod=30 Dec 03 12:21:42 crc kubenswrapper[5052]: I1203 12:21:42.904002 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/barbican0e0e-account-delete-fvkr2"] Dec 03 12:21:42 crc kubenswrapper[5052]: I1203 12:21:42.906602 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican0e0e-account-delete-fvkr2" Dec 03 12:21:42 crc kubenswrapper[5052]: I1203 12:21:42.923900 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 12:21:42 crc kubenswrapper[5052]: I1203 12:21:42.924171 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/cinder-api-0" podUID="bb41ed7e-c96b-44dd-8c4e-ffb702de3f87" containerName="cinder-api-log" containerID="cri-o://19ad1a127fb47796fec06e903f60d68c562fce37f707be6c98c93f3e0d2d25cd" gracePeriod=30 Dec 03 12:21:42 crc kubenswrapper[5052]: I1203 12:21:42.924358 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/cinder-api-0" podUID="bb41ed7e-c96b-44dd-8c4e-ffb702de3f87" containerName="cinder-api" containerID="cri-o://2a7af42a9086e0868a44334065634bf0f3e6cbd7e859d0ffd6a5df7b8b1b5555" gracePeriod=30 Dec 03 12:21:42 crc kubenswrapper[5052]: I1203 12:21:42.944824 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4c9xg\" (UniqueName: \"kubernetes.io/projected/fc2a6873-8741-472a-864d-84651abd4c1c-kube-api-access-4c9xg\") pod \"barbican0e0e-account-delete-fvkr2\" (UID: \"fc2a6873-8741-472a-864d-84651abd4c1c\") " pod="openstack-kuttl-tests/barbican0e0e-account-delete-fvkr2" Dec 03 12:21:42 crc kubenswrapper[5052]: I1203 12:21:42.944908 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/fc2a6873-8741-472a-864d-84651abd4c1c-operator-scripts\") pod \"barbican0e0e-account-delete-fvkr2\" (UID: \"fc2a6873-8741-472a-864d-84651abd4c1c\") " pod="openstack-kuttl-tests/barbican0e0e-account-delete-fvkr2" Dec 03 12:21:42 crc kubenswrapper[5052]: I1203 12:21:42.953646 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican0e0e-account-delete-fvkr2"] Dec 03 12:21:42 crc kubenswrapper[5052]: I1203 12:21:42.964586 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/novacell00e20-account-delete-ktxhg"] Dec 03 12:21:42 crc kubenswrapper[5052]: I1203 12:21:42.965896 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/novacell00e20-account-delete-ktxhg" Dec 03 12:21:42 crc kubenswrapper[5052]: I1203 12:21:42.985582 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:21:42 crc kubenswrapper[5052]: I1203 12:21:42.985907 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-external-api-0" podUID="72b87ae1-6854-475b-a31d-976acc913623" containerName="glance-log" containerID="cri-o://dc3b9cb9559bad706ad6fd805545ec2569385c158301c0e04856678a08e6dfe0" gracePeriod=30 Dec 03 12:21:42 crc kubenswrapper[5052]: I1203 12:21:42.986118 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-external-api-0" podUID="72b87ae1-6854-475b-a31d-976acc913623" containerName="glance-httpd" containerID="cri-o://ace907d0e00f36ffb29d30e119e8630da96914e43d3d15c7fdb73b39b73ccfd8" gracePeriod=30 Dec 03 12:21:42 crc kubenswrapper[5052]: I1203 12:21:42.994754 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/novacell00e20-account-delete-ktxhg"] Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.010529 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/rabbitmq-server-0"] Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.023974 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.024219 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-internal-api-0" podUID="2d217a2f-3478-41e0-936a-6b1d01530007" containerName="glance-log" containerID="cri-o://e124f853c8070429efcc9a6778d70b3a8c1c60fccd3f62bb22a54362a2148f4e" gracePeriod=30 Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.024635 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-internal-api-0" podUID="2d217a2f-3478-41e0-936a-6b1d01530007" containerName="glance-httpd" containerID="cri-o://c432832de2a3a0521fe28359f14c05384338a7557e1245553a8611f5a30ee883" gracePeriod=30 Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.035960 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-7mq7s"] Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.037683 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-7mq7s" Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.047781 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/fc2a6873-8741-472a-864d-84651abd4c1c-operator-scripts\") pod \"barbican0e0e-account-delete-fvkr2\" (UID: \"fc2a6873-8741-472a-864d-84651abd4c1c\") " pod="openstack-kuttl-tests/barbican0e0e-account-delete-fvkr2" Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.047846 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/4edc0513-31af-4aa4-b507-f1181c549be9-operator-scripts\") pod \"novacell00e20-account-delete-ktxhg\" (UID: \"4edc0513-31af-4aa4-b507-f1181c549be9\") " pod="openstack-kuttl-tests/novacell00e20-account-delete-ktxhg" Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.047967 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jxpwq\" (UniqueName: \"kubernetes.io/projected/4edc0513-31af-4aa4-b507-f1181c549be9-kube-api-access-jxpwq\") pod \"novacell00e20-account-delete-ktxhg\" (UID: \"4edc0513-31af-4aa4-b507-f1181c549be9\") " pod="openstack-kuttl-tests/novacell00e20-account-delete-ktxhg" Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.048086 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-4c9xg\" (UniqueName: \"kubernetes.io/projected/fc2a6873-8741-472a-864d-84651abd4c1c-kube-api-access-4c9xg\") pod \"barbican0e0e-account-delete-fvkr2\" (UID: \"fc2a6873-8741-472a-864d-84651abd4c1c\") " pod="openstack-kuttl-tests/barbican0e0e-account-delete-fvkr2" Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.053937 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/novacell13fba-account-delete-hvzdt"] Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.055291 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/novacell13fba-account-delete-hvzdt" Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.057233 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/fc2a6873-8741-472a-864d-84651abd4c1c-operator-scripts\") pod \"barbican0e0e-account-delete-fvkr2\" (UID: \"fc2a6873-8741-472a-864d-84651abd4c1c\") " pod="openstack-kuttl-tests/barbican0e0e-account-delete-fvkr2" Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.083716 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/novacell13fba-account-delete-hvzdt"] Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.108406 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-4c9xg\" (UniqueName: \"kubernetes.io/projected/fc2a6873-8741-472a-864d-84651abd4c1c-kube-api-access-4c9xg\") pod \"barbican0e0e-account-delete-fvkr2\" (UID: \"fc2a6873-8741-472a-864d-84651abd4c1c\") " pod="openstack-kuttl-tests/barbican0e0e-account-delete-fvkr2" Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.123794 5052 generic.go:334] "Generic (PLEG): container finished" podID="90814ea7-1abb-4760-bbfa-1190f23ab38a" containerID="f82b984cb5939c510f6703add68625e1e534cdf59488ddc53cb91e355ac65330" exitCode=143 Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.123858 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-5447868864-mfdcg" event={"ID":"90814ea7-1abb-4760-bbfa-1190f23ab38a","Type":"ContainerDied","Data":"f82b984cb5939c510f6703add68625e1e534cdf59488ddc53cb91e355ac65330"} Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.124693 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-7mq7s"] Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.125335 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" podUID="c279eaf1-2ee2-46fe-88be-9a5a65d4897e" containerName="rabbitmq" containerID="cri-o://9e557ebf23292119d2650d0f7e9abf49644bff9d991b5bf5077147a45b27b74b" gracePeriod=604800 Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.127732 5052 generic.go:334] "Generic (PLEG): container finished" podID="a77df94c-3e83-47a5-9b5f-eb0aeb41bc3d" containerID="cf3832a4b1ebcd0697f163ca295f373357a42dc4f1fd311347b02c08c7239df4" exitCode=2 Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.127812 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovn-northd-0" event={"ID":"a77df94c-3e83-47a5-9b5f-eb0aeb41bc3d","Type":"ContainerDied","Data":"cf3832a4b1ebcd0697f163ca295f373357a42dc4f1fd311347b02c08c7239df4"} Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.145736 5052 generic.go:334] "Generic (PLEG): container finished" podID="03bb65ef-e9a8-4cfb-9536-001e316b1f72" containerID="ece24b1db9c0709fd516d1a80d474c2937fdeb49dd573043245a6e0d4d27d04a" exitCode=0 Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.145828 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-69c9545844-xf6zm" event={"ID":"03bb65ef-e9a8-4cfb-9536-001e316b1f72","Type":"ContainerDied","Data":"ece24b1db9c0709fd516d1a80d474c2937fdeb49dd573043245a6e0d4d27d04a"} Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.151547 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/cdf67382-3145-4280-a5a5-41b0185cb07f-config\") pod \"dnsmasq-dnsmasq-84b9f45d47-7mq7s\" (UID: \"cdf67382-3145-4280-a5a5-41b0185cb07f\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-7mq7s" Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.151658 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-jxpwq\" (UniqueName: \"kubernetes.io/projected/4edc0513-31af-4aa4-b507-f1181c549be9-kube-api-access-jxpwq\") pod \"novacell00e20-account-delete-ktxhg\" (UID: \"4edc0513-31af-4aa4-b507-f1181c549be9\") " pod="openstack-kuttl-tests/novacell00e20-account-delete-ktxhg" Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.151679 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nkzwn\" (UniqueName: \"kubernetes.io/projected/cdf67382-3145-4280-a5a5-41b0185cb07f-kube-api-access-nkzwn\") pod \"dnsmasq-dnsmasq-84b9f45d47-7mq7s\" (UID: \"cdf67382-3145-4280-a5a5-41b0185cb07f\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-7mq7s" Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.151699 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/0f00d4de-f49c-4155-98e7-00be8bc4ddc7-operator-scripts\") pod \"novacell13fba-account-delete-hvzdt\" (UID: \"0f00d4de-f49c-4155-98e7-00be8bc4ddc7\") " pod="openstack-kuttl-tests/novacell13fba-account-delete-hvzdt" Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.151774 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/cdf67382-3145-4280-a5a5-41b0185cb07f-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-84b9f45d47-7mq7s\" (UID: \"cdf67382-3145-4280-a5a5-41b0185cb07f\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-7mq7s" Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.151826 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/4edc0513-31af-4aa4-b507-f1181c549be9-operator-scripts\") pod \"novacell00e20-account-delete-ktxhg\" (UID: \"4edc0513-31af-4aa4-b507-f1181c549be9\") " pod="openstack-kuttl-tests/novacell00e20-account-delete-ktxhg" Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.151859 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vvdpm\" (UniqueName: \"kubernetes.io/projected/0f00d4de-f49c-4155-98e7-00be8bc4ddc7-kube-api-access-vvdpm\") pod \"novacell13fba-account-delete-hvzdt\" (UID: \"0f00d4de-f49c-4155-98e7-00be8bc4ddc7\") " pod="openstack-kuttl-tests/novacell13fba-account-delete-hvzdt" Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.153098 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/4edc0513-31af-4aa4-b507-f1181c549be9-operator-scripts\") pod \"novacell00e20-account-delete-ktxhg\" (UID: \"4edc0513-31af-4aa4-b507-f1181c549be9\") " pod="openstack-kuttl-tests/novacell00e20-account-delete-ktxhg" Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.156811 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-worker-6b4674c647-56v84"] Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.157054 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/barbican-worker-6b4674c647-56v84" podUID="484e6cd1-fb57-4416-9a1a-cf24b4693764" containerName="barbican-worker-log" containerID="cri-o://01a0969c60e54d821f713b7a30fadf577164bb47e2f3fc851bc640b779f911d6" gracePeriod=30 Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.157445 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/barbican-worker-6b4674c647-56v84" podUID="484e6cd1-fb57-4416-9a1a-cf24b4693764" containerName="barbican-worker" containerID="cri-o://e00bcd0d914f7fbd5f88db32fed6a7e8a0000e44d444198b8f7f1ed7f6ecdb89" gracePeriod=30 Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.182925 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-8f5b-account-create-update-m9q79" Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.186213 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_ovsdbserver-sb-0_19a09467-4c91-40f5-b83a-d43a8eb5f2dc/ovsdbserver-sb/0.log" Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.186253 5052 generic.go:334] "Generic (PLEG): container finished" podID="19a09467-4c91-40f5-b83a-d43a8eb5f2dc" containerID="81e8dead6ae4e8433ea10d47201edd08f411e7f097f98bf35c829bcf51a695f9" exitCode=2 Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.186273 5052 generic.go:334] "Generic (PLEG): container finished" podID="19a09467-4c91-40f5-b83a-d43a8eb5f2dc" containerID="5c0846c17b1ab7bae05ee7ab757a5ac9bfa9ad58f76d6d21d76a677a190e3f91" exitCode=143 Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.186310 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-sb-0" event={"ID":"19a09467-4c91-40f5-b83a-d43a8eb5f2dc","Type":"ContainerDied","Data":"81e8dead6ae4e8433ea10d47201edd08f411e7f097f98bf35c829bcf51a695f9"} Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.186334 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-sb-0" event={"ID":"19a09467-4c91-40f5-b83a-d43a8eb5f2dc","Type":"ContainerDied","Data":"5c0846c17b1ab7bae05ee7ab757a5ac9bfa9ad58f76d6d21d76a677a190e3f91"} Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.189299 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-keystone-listener-6fb665b666-ftpjr"] Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.189639 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/barbican-keystone-listener-6fb665b666-ftpjr" podUID="9119635e-cb55-4428-9c61-b93ef2afd799" containerName="barbican-keystone-listener-log" containerID="cri-o://d006b275534cfdfd7a634f938440db7eeb0d006446de410abf688af6391508cb" gracePeriod=30 Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.189855 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/barbican-keystone-listener-6fb665b666-ftpjr" podUID="9119635e-cb55-4428-9c61-b93ef2afd799" containerName="barbican-keystone-listener" containerID="cri-o://eaa2b6fb75e11891ca6e0ce7eb90ce65bb570e469fdab944929138063436ea33" gracePeriod=30 Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.191285 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-jxpwq\" (UniqueName: \"kubernetes.io/projected/4edc0513-31af-4aa4-b507-f1181c549be9-kube-api-access-jxpwq\") pod \"novacell00e20-account-delete-ktxhg\" (UID: \"4edc0513-31af-4aa4-b507-f1181c549be9\") " pod="openstack-kuttl-tests/novacell00e20-account-delete-ktxhg" Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.197670 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstackclient" Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.204428 5052 scope.go:117] "RemoveContainer" containerID="97eb19cad24849878c7614c23d646abe200c279ae260c2f8f7bae07110bd2a2b" Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.212812 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-734a-account-create-update-qcgft" event={"ID":"817345c8-3a42-4dea-8158-75973d9c41d9","Type":"ContainerStarted","Data":"1820e85036ca53b44f2a3156a713a2d04b85ce5d904896d2651e6d8304d19baa"} Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.213997 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:21:43 crc kubenswrapper[5052]: E1203 12:21:43.214942 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[combined-ca-bundle], unattached volumes=[], failed to process volumes=[]: context canceled" pod="openstack-kuttl-tests/nova-scheduler-0" podUID="e3a73fa0-68ef-4e03-81f9-2a143ed805bf" Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.220006 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance73ca-account-delete-vpqhp" event={"ID":"db31774e-8f50-4862-86f3-93cb93f645a0","Type":"ContainerStarted","Data":"d52fa6a791c4db9bacf26c89f59fbe21fefb28a88af961341f4ea8108f037da9"} Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.237145 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_ovsdbserver-nb-0_70cedf3b-c9b9-4e5c-81a2-ba134b150997/ovsdbserver-nb/0.log" Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.237220 5052 generic.go:334] "Generic (PLEG): container finished" podID="70cedf3b-c9b9-4e5c-81a2-ba134b150997" containerID="c7645a9210d2fc1e2e3f163388ea3b91c8a787603d06989bb0272c5699b638c7" exitCode=2 Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.237237 5052 generic.go:334] "Generic (PLEG): container finished" podID="70cedf3b-c9b9-4e5c-81a2-ba134b150997" containerID="f84d0df84d3cb24dd97faa8c986cf4682cc548000edcdfa7e202ea04d4fd81e2" exitCode=143 Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.237322 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-nb-0" event={"ID":"70cedf3b-c9b9-4e5c-81a2-ba134b150997","Type":"ContainerDied","Data":"c7645a9210d2fc1e2e3f163388ea3b91c8a787603d06989bb0272c5699b638c7"} Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.237369 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-nb-0" event={"ID":"70cedf3b-c9b9-4e5c-81a2-ba134b150997","Type":"ContainerDied","Data":"f84d0df84d3cb24dd97faa8c986cf4682cc548000edcdfa7e202ea04d4fd81e2"} Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.237732 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-api-644f564b46-cjtkx"] Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.238959 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/barbican-api-644f564b46-cjtkx" podUID="af38eb0d-3924-4c4b-a22e-7e934bdcf172" containerName="barbican-api-log" containerID="cri-o://e699b862f9953c2e25b19ab9d64356e9c8a7220759bf33ca05a10010b2639fc3" gracePeriod=30 Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.239988 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/barbican-api-644f564b46-cjtkx" podUID="af38eb0d-3924-4c4b-a22e-7e934bdcf172" containerName="barbican-api" containerID="cri-o://0b9c5ec46decf17edaf9cf870fd66992b0a9f87a5bcb4a208561e9aa1a0d41be" gracePeriod=30 Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.255112 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.255326 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" podUID="9e3a0650-c101-4bfa-8137-9909bf8ed4c7" containerName="nova-cell1-novncproxy-novncproxy" containerID="cri-o://c5e90a99765d5931a5d6cd2673f9bb42cd7a8b05426a276c6f134ae97869e630" gracePeriod=30 Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.256163 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"openstack-config-secret\" (UniqueName: \"kubernetes.io/secret/357bc965-8633-4d67-992e-baa770a0defd-openstack-config-secret\") pod \"357bc965-8633-4d67-992e-baa770a0defd\" (UID: \"357bc965-8633-4d67-992e-baa770a0defd\") " Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.256241 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/357bc965-8633-4d67-992e-baa770a0defd-combined-ca-bundle\") pod \"357bc965-8633-4d67-992e-baa770a0defd\" (UID: \"357bc965-8633-4d67-992e-baa770a0defd\") " Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.256351 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"openstack-config\" (UniqueName: \"kubernetes.io/configmap/357bc965-8633-4d67-992e-baa770a0defd-openstack-config\") pod \"357bc965-8633-4d67-992e-baa770a0defd\" (UID: \"357bc965-8633-4d67-992e-baa770a0defd\") " Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.256572 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p4cv8\" (UniqueName: \"kubernetes.io/projected/357bc965-8633-4d67-992e-baa770a0defd-kube-api-access-p4cv8\") pod \"357bc965-8633-4d67-992e-baa770a0defd\" (UID: \"357bc965-8633-4d67-992e-baa770a0defd\") " Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.256869 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/cdf67382-3145-4280-a5a5-41b0185cb07f-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-84b9f45d47-7mq7s\" (UID: \"cdf67382-3145-4280-a5a5-41b0185cb07f\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-7mq7s" Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.256983 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-vvdpm\" (UniqueName: \"kubernetes.io/projected/0f00d4de-f49c-4155-98e7-00be8bc4ddc7-kube-api-access-vvdpm\") pod \"novacell13fba-account-delete-hvzdt\" (UID: \"0f00d4de-f49c-4155-98e7-00be8bc4ddc7\") " pod="openstack-kuttl-tests/novacell13fba-account-delete-hvzdt" Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.257030 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/cdf67382-3145-4280-a5a5-41b0185cb07f-config\") pod \"dnsmasq-dnsmasq-84b9f45d47-7mq7s\" (UID: \"cdf67382-3145-4280-a5a5-41b0185cb07f\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-7mq7s" Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.257139 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-nkzwn\" (UniqueName: \"kubernetes.io/projected/cdf67382-3145-4280-a5a5-41b0185cb07f-kube-api-access-nkzwn\") pod \"dnsmasq-dnsmasq-84b9f45d47-7mq7s\" (UID: \"cdf67382-3145-4280-a5a5-41b0185cb07f\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-7mq7s" Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.257167 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/0f00d4de-f49c-4155-98e7-00be8bc4ddc7-operator-scripts\") pod \"novacell13fba-account-delete-hvzdt\" (UID: \"0f00d4de-f49c-4155-98e7-00be8bc4ddc7\") " pod="openstack-kuttl-tests/novacell13fba-account-delete-hvzdt" Dec 03 12:21:43 crc kubenswrapper[5052]: E1203 12:21:43.257350 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-cell1-scripts: configmap "openstack-cell1-scripts" not found Dec 03 12:21:43 crc kubenswrapper[5052]: E1203 12:21:43.257392 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0f00d4de-f49c-4155-98e7-00be8bc4ddc7-operator-scripts podName:0f00d4de-f49c-4155-98e7-00be8bc4ddc7 nodeName:}" failed. No retries permitted until 2025-12-03 12:21:43.757378007 +0000 UTC m=+3694.345619771 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/0f00d4de-f49c-4155-98e7-00be8bc4ddc7-operator-scripts") pod "novacell13fba-account-delete-hvzdt" (UID: "0f00d4de-f49c-4155-98e7-00be8bc4ddc7") : configmap "openstack-cell1-scripts" not found Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.265885 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/cdf67382-3145-4280-a5a5-41b0185cb07f-config\") pod \"dnsmasq-dnsmasq-84b9f45d47-7mq7s\" (UID: \"cdf67382-3145-4280-a5a5-41b0185cb07f\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-7mq7s" Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.274147 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/cdf67382-3145-4280-a5a5-41b0185cb07f-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-84b9f45d47-7mq7s\" (UID: \"cdf67382-3145-4280-a5a5-41b0185cb07f\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-7mq7s" Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.280779 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"9dec0a31-cba3-4c38-8b5d-4358f16b3704","Type":"ContainerStarted","Data":"72c6e09b72f57c6f1f19ff9c617f85acc4ceb1d16fb1a62383bab5dea1e10644"} Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.291388 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/novaapi8f5b-account-delete-86k67"] Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.293110 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/novaapi8f5b-account-delete-86k67" Dec 03 12:21:43 crc kubenswrapper[5052]: E1203 12:21:43.300640 5052 projected.go:194] Error preparing data for projected volume kube-api-access-vvdpm for pod openstack-kuttl-tests/novacell13fba-account-delete-hvzdt: failed to fetch token: serviceaccounts "galera-openstack-cell1" not found Dec 03 12:21:43 crc kubenswrapper[5052]: E1203 12:21:43.300720 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0f00d4de-f49c-4155-98e7-00be8bc4ddc7-kube-api-access-vvdpm podName:0f00d4de-f49c-4155-98e7-00be8bc4ddc7 nodeName:}" failed. No retries permitted until 2025-12-03 12:21:43.800698854 +0000 UTC m=+3694.388940618 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-vvdpm" (UniqueName: "kubernetes.io/projected/0f00d4de-f49c-4155-98e7-00be8bc4ddc7-kube-api-access-vvdpm") pod "novacell13fba-account-delete-hvzdt" (UID: "0f00d4de-f49c-4155-98e7-00be8bc4ddc7") : failed to fetch token: serviceaccounts "galera-openstack-cell1" not found Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.306183 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/novaapi8f5b-account-delete-86k67"] Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.314439 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-nkzwn\" (UniqueName: \"kubernetes.io/projected/cdf67382-3145-4280-a5a5-41b0185cb07f-kube-api-access-nkzwn\") pod \"dnsmasq-dnsmasq-84b9f45d47-7mq7s\" (UID: \"cdf67382-3145-4280-a5a5-41b0185cb07f\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-7mq7s" Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.318184 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.325747 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/357bc965-8633-4d67-992e-baa770a0defd-kube-api-access-p4cv8" (OuterVolumeSpecName: "kube-api-access-p4cv8") pod "357bc965-8633-4d67-992e-baa770a0defd" (UID: "357bc965-8633-4d67-992e-baa770a0defd"). InnerVolumeSpecName "kube-api-access-p4cv8". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.333732 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/openstack-cell1-galera-0"] Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.357440 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron734a-account-delete-sxpn9" Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.363707 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5m76k\" (UniqueName: \"kubernetes.io/projected/bb65017d-6c06-4870-946b-80b9b220df36-kube-api-access-5m76k\") pod \"novaapi8f5b-account-delete-86k67\" (UID: \"bb65017d-6c06-4870-946b-80b9b220df36\") " pod="openstack-kuttl-tests/novaapi8f5b-account-delete-86k67" Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.364530 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/bb65017d-6c06-4870-946b-80b9b220df36-operator-scripts\") pod \"novaapi8f5b-account-delete-86k67\" (UID: \"bb65017d-6c06-4870-946b-80b9b220df36\") " pod="openstack-kuttl-tests/novaapi8f5b-account-delete-86k67" Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.364897 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-p4cv8\" (UniqueName: \"kubernetes.io/projected/357bc965-8633-4d67-992e-baa770a0defd-kube-api-access-p4cv8\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.371255 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder2067-account-delete-n2tf7" Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.371249 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/357bc965-8633-4d67-992e-baa770a0defd-openstack-config" (OuterVolumeSpecName: "openstack-config") pod "357bc965-8633-4d67-992e-baa770a0defd" (UID: "357bc965-8633-4d67-992e-baa770a0defd"). InnerVolumeSpecName "openstack-config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.378545 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/novacell13fba-account-delete-hvzdt"] Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.393072 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-734a-account-create-update-qcgft"] Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.396908 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/rabbitmq-server-0" podUID="c14a906c-f19d-4806-9110-8c47b38c5ae0" containerName="rabbitmq" containerID="cri-o://9217efd68f2e5602eadef89a260d023bd85316f53943b906f69ea6a06b762ecd" gracePeriod=604800 Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.405019 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican0e0e-account-delete-fvkr2" Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.417886 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance73ca-account-delete-vpqhp"] Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.437194 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.437582 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="2d94bf00-7a64-4e90-a48c-b206ec54da6d" containerName="ceilometer-central-agent" containerID="cri-o://cf05124bcf6f5c1b9dc38517547ce3e4342c12cea5c84ee53adeabf6d90e42cf" gracePeriod=30 Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.438167 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="2d94bf00-7a64-4e90-a48c-b206ec54da6d" containerName="proxy-httpd" containerID="cri-o://3d06201ab438e62b35c336b3ff97f18ab1544e471098deb210f49f6607c474fe" gracePeriod=30 Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.438230 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="2d94bf00-7a64-4e90-a48c-b206ec54da6d" containerName="sg-core" containerID="cri-o://7a966caf1545e10cd03293bfda99f2c9475e9dcbe187dcc16f09580784759143" gracePeriod=30 Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.438271 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="2d94bf00-7a64-4e90-a48c-b206ec54da6d" containerName="ceilometer-notification-agent" containerID="cri-o://e0441d5665d194615504d1f41084afce1caa468996e04b063d890632f490acca" gracePeriod=30 Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.445849 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/novacell00e20-account-delete-ktxhg" Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.460499 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-7mq7s" Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.465975 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/kube-state-metrics-0"] Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.465985 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-5m76k\" (UniqueName: \"kubernetes.io/projected/bb65017d-6c06-4870-946b-80b9b220df36-kube-api-access-5m76k\") pod \"novaapi8f5b-account-delete-86k67\" (UID: \"bb65017d-6c06-4870-946b-80b9b220df36\") " pod="openstack-kuttl-tests/novaapi8f5b-account-delete-86k67" Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.466152 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/bb65017d-6c06-4870-946b-80b9b220df36-operator-scripts\") pod \"novaapi8f5b-account-delete-86k67\" (UID: \"bb65017d-6c06-4870-946b-80b9b220df36\") " pod="openstack-kuttl-tests/novaapi8f5b-account-delete-86k67" Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.466188 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/kube-state-metrics-0" podUID="b657a259-eb7a-48e8-8f41-2a3c077aff2a" containerName="kube-state-metrics" containerID="cri-o://9569b49a51e3eb2f02cd20195f982384cb40ad5f6b6fd5f65cf8a8a8fff4d592" gracePeriod=30 Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.466812 5052 reconciler_common.go:293] "Volume detached for volume \"openstack-config\" (UniqueName: \"kubernetes.io/configmap/357bc965-8633-4d67-992e-baa770a0defd-openstack-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:43 crc kubenswrapper[5052]: E1203 12:21:43.468004 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-memcached-svc: secret "cert-memcached-svc" not found Dec 03 12:21:43 crc kubenswrapper[5052]: E1203 12:21:43.468047 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/adcea736-eb15-4768-b12c-edb4c810beb3-memcached-tls-certs podName:adcea736-eb15-4768-b12c-edb4c810beb3 nodeName:}" failed. No retries permitted until 2025-12-03 12:21:45.46803318 +0000 UTC m=+3696.056274944 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "memcached-tls-certs" (UniqueName: "kubernetes.io/secret/adcea736-eb15-4768-b12c-edb4c810beb3-memcached-tls-certs") pod "memcached-0" (UID: "adcea736-eb15-4768-b12c-edb4c810beb3") : secret "cert-memcached-svc" not found Dec 03 12:21:43 crc kubenswrapper[5052]: E1203 12:21:43.468999 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/combined-ca-bundle: secret "combined-ca-bundle" not found Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.469053 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/bb65017d-6c06-4870-946b-80b9b220df36-operator-scripts\") pod \"novaapi8f5b-account-delete-86k67\" (UID: \"bb65017d-6c06-4870-946b-80b9b220df36\") " pod="openstack-kuttl-tests/novaapi8f5b-account-delete-86k67" Dec 03 12:21:43 crc kubenswrapper[5052]: E1203 12:21:43.469068 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/adcea736-eb15-4768-b12c-edb4c810beb3-combined-ca-bundle podName:adcea736-eb15-4768-b12c-edb4c810beb3 nodeName:}" failed. No retries permitted until 2025-12-03 12:21:45.469049359 +0000 UTC m=+3696.057291213 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "combined-ca-bundle" (UniqueName: "kubernetes.io/secret/adcea736-eb15-4768-b12c-edb4c810beb3-combined-ca-bundle") pod "memcached-0" (UID: "adcea736-eb15-4768-b12c-edb4c810beb3") : secret "combined-ca-bundle" not found Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.479808 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/placement9152-account-delete-x7q46"] Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.503353 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-5m76k\" (UniqueName: \"kubernetes.io/projected/bb65017d-6c06-4870-946b-80b9b220df36-kube-api-access-5m76k\") pod \"novaapi8f5b-account-delete-86k67\" (UID: \"bb65017d-6c06-4870-946b-80b9b220df36\") " pod="openstack-kuttl-tests/novaapi8f5b-account-delete-86k67" Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.551893 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-db-sync-r444h"] Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.565755 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/novaapi8f5b-account-delete-86k67" Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.591033 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-db-sync-r444h"] Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.643225 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-0"] Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.643433 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-cell1-conductor-0" podUID="a7907bc2-8e73-4272-b779-fd71d4acb1f4" containerName="nova-cell1-conductor-conductor" containerID="cri-o://938a22611459fa789b24b03d2a21853b69071aad6d79f25384679595984908d8" gracePeriod=30 Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.671872 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-0"] Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.675252 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-cell0-conductor-0" podUID="efe32106-ca1b-4d84-80a2-c0cddd3541b9" containerName="nova-cell0-conductor-conductor" containerID="cri-o://75bc26794aa58f8bf478527bc4377138449cad0841fd60348ba9d8b17243a1e4" gracePeriod=30 Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.710438 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0bbf844e-3419-4f88-b375-ca77df3007af" path="/var/lib/kubelet/pods/0bbf844e-3419-4f88-b375-ca77df3007af/volumes" Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.711787 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0f021810-dfcd-4695-8330-380dfc297701" path="/var/lib/kubelet/pods/0f021810-dfcd-4695-8330-380dfc297701/volumes" Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.712486 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="27756338-429b-45dd-b0c6-9f7ad69f9806" path="/var/lib/kubelet/pods/27756338-429b-45dd-b0c6-9f7ad69f9806/volumes" Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.725547 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="27d134d0-a0b9-457b-8c8c-92cfb104cba1" path="/var/lib/kubelet/pods/27d134d0-a0b9-457b-8c8c-92cfb104cba1/volumes" Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.728365 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2c08972f-630a-48c4-b7a6-c98c1b0f1a20" path="/var/lib/kubelet/pods/2c08972f-630a-48c4-b7a6-c98c1b0f1a20/volumes" Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.730219 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="318545bf-5895-48f9-9649-67207bc4b207" path="/var/lib/kubelet/pods/318545bf-5895-48f9-9649-67207bc4b207/volumes" Dec 03 12:21:43 crc kubenswrapper[5052]: E1203 12:21:43.749887 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-vvdpm operator-scripts], unattached volumes=[], failed to process volumes=[]: context canceled" pod="openstack-kuttl-tests/novacell13fba-account-delete-hvzdt" podUID="0f00d4de-f49c-4155-98e7-00be8bc4ddc7" Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.768103 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/357bc965-8633-4d67-992e-baa770a0defd-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "357bc965-8633-4d67-992e-baa770a0defd" (UID: "357bc965-8633-4d67-992e-baa770a0defd"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.768786 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="334ceeeb-df59-4f9b-b9e8-46e44c9c515a" path="/var/lib/kubelet/pods/334ceeeb-df59-4f9b-b9e8-46e44c9c515a/volumes" Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.769444 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="50ae31b2-a1c0-4afb-8e23-d7856c55af1d" path="/var/lib/kubelet/pods/50ae31b2-a1c0-4afb-8e23-d7856c55af1d/volumes" Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.769919 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="66f5168a-1ee5-4de9-8e50-90ef25186ee8" path="/var/lib/kubelet/pods/66f5168a-1ee5-4de9-8e50-90ef25186ee8/volumes" Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.776219 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7a9497b1-89e2-4ebc-97e6-2bdaeefab95f" path="/var/lib/kubelet/pods/7a9497b1-89e2-4ebc-97e6-2bdaeefab95f/volumes" Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.777358 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_ovsdbserver-sb-0_19a09467-4c91-40f5-b83a-d43a8eb5f2dc/ovsdbserver-sb/0.log" Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.777444 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.779314 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8d0ffc74-5f47-4195-8091-b4890e55c32f" path="/var/lib/kubelet/pods/8d0ffc74-5f47-4195-8091-b4890e55c32f/volumes" Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.783935 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="94ac1a3b-14db-445c-8fb8-8bc3069bdb57" path="/var/lib/kubelet/pods/94ac1a3b-14db-445c-8fb8-8bc3069bdb57/volumes" Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.784705 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="96e36cf6-43a6-43e9-b8dd-64f1424477e8" path="/var/lib/kubelet/pods/96e36cf6-43a6-43e9-b8dd-64f1424477e8/volumes" Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.785729 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c2e338a2-8424-4325-a580-55618dcd3c29" path="/var/lib/kubelet/pods/c2e338a2-8424-4325-a580-55618dcd3c29/volumes" Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.786665 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/19a09467-4c91-40f5-b83a-d43a8eb5f2dc-combined-ca-bundle\") pod \"19a09467-4c91-40f5-b83a-d43a8eb5f2dc\" (UID: \"19a09467-4c91-40f5-b83a-d43a8eb5f2dc\") " Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.786775 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovsdbserver-sb-tls-certs\" (UniqueName: \"kubernetes.io/secret/19a09467-4c91-40f5-b83a-d43a8eb5f2dc-ovsdbserver-sb-tls-certs\") pod \"19a09467-4c91-40f5-b83a-d43a8eb5f2dc\" (UID: \"19a09467-4c91-40f5-b83a-d43a8eb5f2dc\") " Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.786849 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/19a09467-4c91-40f5-b83a-d43a8eb5f2dc-config\") pod \"19a09467-4c91-40f5-b83a-d43a8eb5f2dc\" (UID: \"19a09467-4c91-40f5-b83a-d43a8eb5f2dc\") " Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.786880 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovndbcluster-sb-etc-ovn\" (UniqueName: \"kubernetes.io/local-volume/local-storage02-crc\") pod \"19a09467-4c91-40f5-b83a-d43a8eb5f2dc\" (UID: \"19a09467-4c91-40f5-b83a-d43a8eb5f2dc\") " Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.786936 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/19a09467-4c91-40f5-b83a-d43a8eb5f2dc-scripts\") pod \"19a09467-4c91-40f5-b83a-d43a8eb5f2dc\" (UID: \"19a09467-4c91-40f5-b83a-d43a8eb5f2dc\") " Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.786963 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/19a09467-4c91-40f5-b83a-d43a8eb5f2dc-metrics-certs-tls-certs\") pod \"19a09467-4c91-40f5-b83a-d43a8eb5f2dc\" (UID: \"19a09467-4c91-40f5-b83a-d43a8eb5f2dc\") " Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.786999 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-48j4f\" (UniqueName: \"kubernetes.io/projected/19a09467-4c91-40f5-b83a-d43a8eb5f2dc-kube-api-access-48j4f\") pod \"19a09467-4c91-40f5-b83a-d43a8eb5f2dc\" (UID: \"19a09467-4c91-40f5-b83a-d43a8eb5f2dc\") " Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.787030 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/19a09467-4c91-40f5-b83a-d43a8eb5f2dc-ovsdb-rundir\") pod \"19a09467-4c91-40f5-b83a-d43a8eb5f2dc\" (UID: \"19a09467-4c91-40f5-b83a-d43a8eb5f2dc\") " Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.787196 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/0f00d4de-f49c-4155-98e7-00be8bc4ddc7-operator-scripts\") pod \"novacell13fba-account-delete-hvzdt\" (UID: \"0f00d4de-f49c-4155-98e7-00be8bc4ddc7\") " pod="openstack-kuttl-tests/novacell13fba-account-delete-hvzdt" Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.787482 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/357bc965-8633-4d67-992e-baa770a0defd-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:43 crc kubenswrapper[5052]: E1203 12:21:43.787878 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-cell1-scripts: configmap "openstack-cell1-scripts" not found Dec 03 12:21:43 crc kubenswrapper[5052]: E1203 12:21:43.787954 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0f00d4de-f49c-4155-98e7-00be8bc4ddc7-operator-scripts podName:0f00d4de-f49c-4155-98e7-00be8bc4ddc7 nodeName:}" failed. No retries permitted until 2025-12-03 12:21:44.78792256 +0000 UTC m=+3695.376164324 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/0f00d4de-f49c-4155-98e7-00be8bc4ddc7-operator-scripts") pod "novacell13fba-account-delete-hvzdt" (UID: "0f00d4de-f49c-4155-98e7-00be8bc4ddc7") : configmap "openstack-cell1-scripts" not found Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.790138 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c4487b92-6d46-4cde-b1c3-056ac6fa9753" path="/var/lib/kubelet/pods/c4487b92-6d46-4cde-b1c3-056ac6fa9753/volumes" Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.791450 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/19a09467-4c91-40f5-b83a-d43a8eb5f2dc-config" (OuterVolumeSpecName: "config") pod "19a09467-4c91-40f5-b83a-d43a8eb5f2dc" (UID: "19a09467-4c91-40f5-b83a-d43a8eb5f2dc"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.792020 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/19a09467-4c91-40f5-b83a-d43a8eb5f2dc-scripts" (OuterVolumeSpecName: "scripts") pod "19a09467-4c91-40f5-b83a-d43a8eb5f2dc" (UID: "19a09467-4c91-40f5-b83a-d43a8eb5f2dc"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.792346 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19a09467-4c91-40f5-b83a-d43a8eb5f2dc-ovsdb-rundir" (OuterVolumeSpecName: "ovsdb-rundir") pod "19a09467-4c91-40f5-b83a-d43a8eb5f2dc" (UID: "19a09467-4c91-40f5-b83a-d43a8eb5f2dc"). InnerVolumeSpecName "ovsdb-rundir". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.793330 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cb824463-7efe-49ee-a052-acd1fd8f02d3" path="/var/lib/kubelet/pods/cb824463-7efe-49ee-a052-acd1fd8f02d3/volumes" Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.793986 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dfbfcab9-41af-42e1-9ddf-5a05ffedb6c7" path="/var/lib/kubelet/pods/dfbfcab9-41af-42e1-9ddf-5a05ffedb6c7/volumes" Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.794642 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f560f1ba-b0e0-4fb6-9802-bf1c9b726592" path="/var/lib/kubelet/pods/f560f1ba-b0e0-4fb6-9802-bf1c9b726592/volumes" Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.821715 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage02-crc" (OuterVolumeSpecName: "ovndbcluster-sb-etc-ovn") pod "19a09467-4c91-40f5-b83a-d43a8eb5f2dc" (UID: "19a09467-4c91-40f5-b83a-d43a8eb5f2dc"). InnerVolumeSpecName "local-storage02-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.821856 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/19a09467-4c91-40f5-b83a-d43a8eb5f2dc-kube-api-access-48j4f" (OuterVolumeSpecName: "kube-api-access-48j4f") pod "19a09467-4c91-40f5-b83a-d43a8eb5f2dc" (UID: "19a09467-4c91-40f5-b83a-d43a8eb5f2dc"). InnerVolumeSpecName "kube-api-access-48j4f". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:21:43 crc kubenswrapper[5052]: E1203 12:21:43.822091 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="75bc26794aa58f8bf478527bc4377138449cad0841fd60348ba9d8b17243a1e4" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:21:43 crc kubenswrapper[5052]: E1203 12:21:43.830658 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="75bc26794aa58f8bf478527bc4377138449cad0841fd60348ba9d8b17243a1e4" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.830804 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/357bc965-8633-4d67-992e-baa770a0defd-openstack-config-secret" (OuterVolumeSpecName: "openstack-config-secret") pod "357bc965-8633-4d67-992e-baa770a0defd" (UID: "357bc965-8633-4d67-992e-baa770a0defd"). InnerVolumeSpecName "openstack-config-secret". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:43 crc kubenswrapper[5052]: E1203 12:21:43.861696 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="75bc26794aa58f8bf478527bc4377138449cad0841fd60348ba9d8b17243a1e4" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:21:43 crc kubenswrapper[5052]: E1203 12:21:43.861760 5052 prober.go:104] "Probe errored" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" probeType="Readiness" pod="openstack-kuttl-tests/nova-cell0-conductor-0" podUID="efe32106-ca1b-4d84-80a2-c0cddd3541b9" containerName="nova-cell0-conductor-conductor" Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.890019 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-vvdpm\" (UniqueName: \"kubernetes.io/projected/0f00d4de-f49c-4155-98e7-00be8bc4ddc7-kube-api-access-vvdpm\") pod \"novacell13fba-account-delete-hvzdt\" (UID: \"0f00d4de-f49c-4155-98e7-00be8bc4ddc7\") " pod="openstack-kuttl-tests/novacell13fba-account-delete-hvzdt" Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.890181 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-48j4f\" (UniqueName: \"kubernetes.io/projected/19a09467-4c91-40f5-b83a-d43a8eb5f2dc-kube-api-access-48j4f\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.890194 5052 reconciler_common.go:293] "Volume detached for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/19a09467-4c91-40f5-b83a-d43a8eb5f2dc-ovsdb-rundir\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.890203 5052 reconciler_common.go:293] "Volume detached for volume \"openstack-config-secret\" (UniqueName: \"kubernetes.io/secret/357bc965-8633-4d67-992e-baa770a0defd-openstack-config-secret\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.890212 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/19a09467-4c91-40f5-b83a-d43a8eb5f2dc-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.890231 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage02-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage02-crc\") on node \"crc\" " Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.890240 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/19a09467-4c91-40f5-b83a-d43a8eb5f2dc-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.907665 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.907695 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-db-sync-hjwg6"] Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.907710 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.907720 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-db-sync-hjwg6"] Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.907738 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/swift-proxy-d4747d79f-wtbhv"] Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.907882 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/swift-proxy-6b9f9968d-5zjp2"] Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.907901 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/memcached-0"] Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.908314 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/memcached-0" podUID="adcea736-eb15-4768-b12c-edb4c810beb3" containerName="memcached" containerID="cri-o://7eee2cae018afa667fc80fb6f8e55b05f28d5f2bfff1ccd7ad972013dda2c344" gracePeriod=30 Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.910105 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-proxy-d4747d79f-wtbhv" podUID="7e109538-a2d2-43bb-8cef-bf99d7d0f1e9" containerName="proxy-httpd" containerID="cri-o://17114cefe25d8670369fc24018b4320e72bd1b81aa69b034a694652ddb010caa" gracePeriod=30 Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.910249 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-proxy-6b9f9968d-5zjp2" podUID="fb73cc9a-89cc-4cb8-87ce-2731e5dab0f7" containerName="proxy-httpd" containerID="cri-o://a1d1e7cece7f6bab44f530f29cd0b9119aee96b8c725b66c1bb8c782f4ccd637" gracePeriod=30 Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.910689 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-proxy-d4747d79f-wtbhv" podUID="7e109538-a2d2-43bb-8cef-bf99d7d0f1e9" containerName="proxy-server" containerID="cri-o://ef9fabe8451cd9b904301777198ce63bf6cefd87c577b4cad8746d28b098af4a" gracePeriod=30 Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.911036 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-proxy-6b9f9968d-5zjp2" podUID="fb73cc9a-89cc-4cb8-87ce-2731e5dab0f7" containerName="proxy-server" containerID="cri-o://370f335a2865f2cdbcf3d953581d2819c13f1b7d88a0ac612179ca394589c0e9" gracePeriod=30 Dec 03 12:21:43 crc kubenswrapper[5052]: E1203 12:21:43.943544 5052 projected.go:194] Error preparing data for projected volume kube-api-access-vvdpm for pod openstack-kuttl-tests/novacell13fba-account-delete-hvzdt: failed to fetch token: serviceaccounts "galera-openstack-cell1" not found Dec 03 12:21:43 crc kubenswrapper[5052]: E1203 12:21:43.943646 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0f00d4de-f49c-4155-98e7-00be8bc4ddc7-kube-api-access-vvdpm podName:0f00d4de-f49c-4155-98e7-00be8bc4ddc7 nodeName:}" failed. No retries permitted until 2025-12-03 12:21:44.943624924 +0000 UTC m=+3695.531866688 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-vvdpm" (UniqueName: "kubernetes.io/projected/0f00d4de-f49c-4155-98e7-00be8bc4ddc7-kube-api-access-vvdpm") pod "novacell13fba-account-delete-hvzdt" (UID: "0f00d4de-f49c-4155-98e7-00be8bc4ddc7") : failed to fetch token: serviceaccounts "galera-openstack-cell1" not found Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.973643 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/keystone-28d1-account-create-update-f58gf"] Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.986601 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_ovsdbserver-nb-0_70cedf3b-c9b9-4e5c-81a2-ba134b150997/ovsdbserver-nb/0.log" Dec 03 12:21:43 crc kubenswrapper[5052]: I1203 12:21:43.986674 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:21:44 crc kubenswrapper[5052]: I1203 12:21:44.009897 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage02-crc" (UniqueName: "kubernetes.io/local-volume/local-storage02-crc") on node "crc" Dec 03 12:21:44 crc kubenswrapper[5052]: I1203 12:21:44.071722 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/19a09467-4c91-40f5-b83a-d43a8eb5f2dc-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "19a09467-4c91-40f5-b83a-d43a8eb5f2dc" (UID: "19a09467-4c91-40f5-b83a-d43a8eb5f2dc"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:44 crc kubenswrapper[5052]: I1203 12:21:44.092578 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/keystone-28d1-account-create-update-f58gf"] Dec 03 12:21:44 crc kubenswrapper[5052]: I1203 12:21:44.093937 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/70cedf3b-c9b9-4e5c-81a2-ba134b150997-ovsdb-rundir\") pod \"70cedf3b-c9b9-4e5c-81a2-ba134b150997\" (UID: \"70cedf3b-c9b9-4e5c-81a2-ba134b150997\") " Dec 03 12:21:44 crc kubenswrapper[5052]: I1203 12:21:44.094030 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/70cedf3b-c9b9-4e5c-81a2-ba134b150997-combined-ca-bundle\") pod \"70cedf3b-c9b9-4e5c-81a2-ba134b150997\" (UID: \"70cedf3b-c9b9-4e5c-81a2-ba134b150997\") " Dec 03 12:21:44 crc kubenswrapper[5052]: I1203 12:21:44.094089 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovsdbserver-nb-tls-certs\" (UniqueName: \"kubernetes.io/secret/70cedf3b-c9b9-4e5c-81a2-ba134b150997-ovsdbserver-nb-tls-certs\") pod \"70cedf3b-c9b9-4e5c-81a2-ba134b150997\" (UID: \"70cedf3b-c9b9-4e5c-81a2-ba134b150997\") " Dec 03 12:21:44 crc kubenswrapper[5052]: I1203 12:21:44.094185 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/70cedf3b-c9b9-4e5c-81a2-ba134b150997-scripts\") pod \"70cedf3b-c9b9-4e5c-81a2-ba134b150997\" (UID: \"70cedf3b-c9b9-4e5c-81a2-ba134b150997\") " Dec 03 12:21:44 crc kubenswrapper[5052]: I1203 12:21:44.094232 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovndbcluster-nb-etc-ovn\" (UniqueName: \"kubernetes.io/local-volume/local-storage20-crc\") pod \"70cedf3b-c9b9-4e5c-81a2-ba134b150997\" (UID: \"70cedf3b-c9b9-4e5c-81a2-ba134b150997\") " Dec 03 12:21:44 crc kubenswrapper[5052]: I1203 12:21:44.094284 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/70cedf3b-c9b9-4e5c-81a2-ba134b150997-metrics-certs-tls-certs\") pod \"70cedf3b-c9b9-4e5c-81a2-ba134b150997\" (UID: \"70cedf3b-c9b9-4e5c-81a2-ba134b150997\") " Dec 03 12:21:44 crc kubenswrapper[5052]: I1203 12:21:44.094413 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/70cedf3b-c9b9-4e5c-81a2-ba134b150997-config\") pod \"70cedf3b-c9b9-4e5c-81a2-ba134b150997\" (UID: \"70cedf3b-c9b9-4e5c-81a2-ba134b150997\") " Dec 03 12:21:44 crc kubenswrapper[5052]: I1203 12:21:44.094475 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kt8jm\" (UniqueName: \"kubernetes.io/projected/70cedf3b-c9b9-4e5c-81a2-ba134b150997-kube-api-access-kt8jm\") pod \"70cedf3b-c9b9-4e5c-81a2-ba134b150997\" (UID: \"70cedf3b-c9b9-4e5c-81a2-ba134b150997\") " Dec 03 12:21:44 crc kubenswrapper[5052]: I1203 12:21:44.095332 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/19a09467-4c91-40f5-b83a-d43a8eb5f2dc-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:44 crc kubenswrapper[5052]: I1203 12:21:44.095357 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage02-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage02-crc\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:44 crc kubenswrapper[5052]: E1203 12:21:44.095426 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/rabbitmq-cell1-config-data: configmap "rabbitmq-cell1-config-data" not found Dec 03 12:21:44 crc kubenswrapper[5052]: E1203 12:21:44.095623 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c279eaf1-2ee2-46fe-88be-9a5a65d4897e-config-data podName:c279eaf1-2ee2-46fe-88be-9a5a65d4897e nodeName:}" failed. No retries permitted until 2025-12-03 12:21:48.095602732 +0000 UTC m=+3698.683844496 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "config-data" (UniqueName: "kubernetes.io/configmap/c279eaf1-2ee2-46fe-88be-9a5a65d4897e-config-data") pod "rabbitmq-cell1-server-0" (UID: "c279eaf1-2ee2-46fe-88be-9a5a65d4897e") : configmap "rabbitmq-cell1-config-data" not found Dec 03 12:21:44 crc kubenswrapper[5052]: I1203 12:21:44.096342 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/70cedf3b-c9b9-4e5c-81a2-ba134b150997-ovsdb-rundir" (OuterVolumeSpecName: "ovsdb-rundir") pod "70cedf3b-c9b9-4e5c-81a2-ba134b150997" (UID: "70cedf3b-c9b9-4e5c-81a2-ba134b150997"). InnerVolumeSpecName "ovsdb-rundir". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:21:44 crc kubenswrapper[5052]: I1203 12:21:44.108275 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/70cedf3b-c9b9-4e5c-81a2-ba134b150997-config" (OuterVolumeSpecName: "config") pod "70cedf3b-c9b9-4e5c-81a2-ba134b150997" (UID: "70cedf3b-c9b9-4e5c-81a2-ba134b150997"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:21:44 crc kubenswrapper[5052]: I1203 12:21:44.108359 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/70cedf3b-c9b9-4e5c-81a2-ba134b150997-scripts" (OuterVolumeSpecName: "scripts") pod "70cedf3b-c9b9-4e5c-81a2-ba134b150997" (UID: "70cedf3b-c9b9-4e5c-81a2-ba134b150997"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:21:44 crc kubenswrapper[5052]: I1203 12:21:44.116610 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage20-crc" (OuterVolumeSpecName: "ovndbcluster-nb-etc-ovn") pod "70cedf3b-c9b9-4e5c-81a2-ba134b150997" (UID: "70cedf3b-c9b9-4e5c-81a2-ba134b150997"). InnerVolumeSpecName "local-storage20-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 12:21:44 crc kubenswrapper[5052]: I1203 12:21:44.117764 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/70cedf3b-c9b9-4e5c-81a2-ba134b150997-kube-api-access-kt8jm" (OuterVolumeSpecName: "kube-api-access-kt8jm") pod "70cedf3b-c9b9-4e5c-81a2-ba134b150997" (UID: "70cedf3b-c9b9-4e5c-81a2-ba134b150997"). InnerVolumeSpecName "kube-api-access-kt8jm". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:21:44 crc kubenswrapper[5052]: I1203 12:21:44.125583 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/keystone-db-create-59gmf"] Dec 03 12:21:44 crc kubenswrapper[5052]: I1203 12:21:44.132583 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/keystone-28d1-account-create-update-f27hx"] Dec 03 12:21:44 crc kubenswrapper[5052]: E1203 12:21:44.133028 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="19a09467-4c91-40f5-b83a-d43a8eb5f2dc" containerName="ovsdbserver-sb" Dec 03 12:21:44 crc kubenswrapper[5052]: I1203 12:21:44.133043 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="19a09467-4c91-40f5-b83a-d43a8eb5f2dc" containerName="ovsdbserver-sb" Dec 03 12:21:44 crc kubenswrapper[5052]: E1203 12:21:44.133067 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="70cedf3b-c9b9-4e5c-81a2-ba134b150997" containerName="openstack-network-exporter" Dec 03 12:21:44 crc kubenswrapper[5052]: I1203 12:21:44.133075 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="70cedf3b-c9b9-4e5c-81a2-ba134b150997" containerName="openstack-network-exporter" Dec 03 12:21:44 crc kubenswrapper[5052]: E1203 12:21:44.133103 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="70cedf3b-c9b9-4e5c-81a2-ba134b150997" containerName="ovsdbserver-nb" Dec 03 12:21:44 crc kubenswrapper[5052]: I1203 12:21:44.133110 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="70cedf3b-c9b9-4e5c-81a2-ba134b150997" containerName="ovsdbserver-nb" Dec 03 12:21:44 crc kubenswrapper[5052]: E1203 12:21:44.133122 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="19a09467-4c91-40f5-b83a-d43a8eb5f2dc" containerName="openstack-network-exporter" Dec 03 12:21:44 crc kubenswrapper[5052]: I1203 12:21:44.133128 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="19a09467-4c91-40f5-b83a-d43a8eb5f2dc" containerName="openstack-network-exporter" Dec 03 12:21:44 crc kubenswrapper[5052]: I1203 12:21:44.133309 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="19a09467-4c91-40f5-b83a-d43a8eb5f2dc" containerName="openstack-network-exporter" Dec 03 12:21:44 crc kubenswrapper[5052]: I1203 12:21:44.133325 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="70cedf3b-c9b9-4e5c-81a2-ba134b150997" containerName="openstack-network-exporter" Dec 03 12:21:44 crc kubenswrapper[5052]: I1203 12:21:44.133339 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="70cedf3b-c9b9-4e5c-81a2-ba134b150997" containerName="ovsdbserver-nb" Dec 03 12:21:44 crc kubenswrapper[5052]: I1203 12:21:44.133357 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="19a09467-4c91-40f5-b83a-d43a8eb5f2dc" containerName="ovsdbserver-sb" Dec 03 12:21:44 crc kubenswrapper[5052]: I1203 12:21:44.133978 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-28d1-account-create-update-f27hx" Dec 03 12:21:44 crc kubenswrapper[5052]: I1203 12:21:44.139123 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone-db-secret" Dec 03 12:21:44 crc kubenswrapper[5052]: I1203 12:21:44.144131 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/keystone-bootstrap-dqvmm"] Dec 03 12:21:44 crc kubenswrapper[5052]: I1203 12:21:44.155801 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/keystone-db-sync-9jf9q"] Dec 03 12:21:44 crc kubenswrapper[5052]: I1203 12:21:44.161472 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/keystone-db-create-59gmf"] Dec 03 12:21:44 crc kubenswrapper[5052]: I1203 12:21:44.168662 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/keystone-bootstrap-dqvmm"] Dec 03 12:21:44 crc kubenswrapper[5052]: I1203 12:21:44.174690 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/keystone-db-sync-9jf9q"] Dec 03 12:21:44 crc kubenswrapper[5052]: I1203 12:21:44.181539 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/keystone-796c4bc57b-zsjsh"] Dec 03 12:21:44 crc kubenswrapper[5052]: I1203 12:21:44.181742 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/keystone-796c4bc57b-zsjsh" podUID="540ee320-be53-46e0-90a3-079a5804124d" containerName="keystone-api" containerID="cri-o://b80b6e8c4ad454ae227b95729ae24f6a9c106e69b758ed3c06b5fe2924584a0c" gracePeriod=30 Dec 03 12:21:44 crc kubenswrapper[5052]: I1203 12:21:44.194843 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-28d1-account-create-update-f27hx"] Dec 03 12:21:44 crc kubenswrapper[5052]: I1203 12:21:44.198491 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/70cedf3b-c9b9-4e5c-81a2-ba134b150997-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:44 crc kubenswrapper[5052]: I1203 12:21:44.198543 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage20-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage20-crc\") on node \"crc\" " Dec 03 12:21:44 crc kubenswrapper[5052]: I1203 12:21:44.198555 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/70cedf3b-c9b9-4e5c-81a2-ba134b150997-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:44 crc kubenswrapper[5052]: I1203 12:21:44.198565 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-kt8jm\" (UniqueName: \"kubernetes.io/projected/70cedf3b-c9b9-4e5c-81a2-ba134b150997-kube-api-access-kt8jm\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:44 crc kubenswrapper[5052]: I1203 12:21:44.198574 5052 reconciler_common.go:293] "Volume detached for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/70cedf3b-c9b9-4e5c-81a2-ba134b150997-ovsdb-rundir\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:44 crc kubenswrapper[5052]: I1203 12:21:44.201403 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/keystone28d1-account-delete-c9vmm"] Dec 03 12:21:44 crc kubenswrapper[5052]: I1203 12:21:44.203691 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone28d1-account-delete-c9vmm" Dec 03 12:21:44 crc kubenswrapper[5052]: I1203 12:21:44.219184 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone28d1-account-delete-c9vmm"] Dec 03 12:21:44 crc kubenswrapper[5052]: I1203 12:21:44.228060 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage20-crc" (UniqueName: "kubernetes.io/local-volume/local-storage20-crc") on node "crc" Dec 03 12:21:44 crc kubenswrapper[5052]: I1203 12:21:44.242034 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/openstack-galera-0"] Dec 03 12:21:44 crc kubenswrapper[5052]: I1203 12:21:44.275552 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/keystone28d1-account-delete-c9vmm"] Dec 03 12:21:44 crc kubenswrapper[5052]: E1203 12:21:44.276736 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-5h59r operator-scripts], unattached volumes=[], failed to process volumes=[]: context canceled" pod="openstack-kuttl-tests/keystone28d1-account-delete-c9vmm" podUID="9165ef68-b76d-472e-99af-06cc8c8a2d3e" Dec 03 12:21:44 crc kubenswrapper[5052]: I1203 12:21:44.280940 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/keystone-28d1-account-create-update-f27hx"] Dec 03 12:21:44 crc kubenswrapper[5052]: E1203 12:21:44.281636 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-75s94 operator-scripts], unattached volumes=[], failed to process volumes=[]: context canceled" pod="openstack-kuttl-tests/keystone-28d1-account-create-update-f27hx" podUID="bfc242a2-aca4-458b-812e-018787bdbcab" Dec 03 12:21:44 crc kubenswrapper[5052]: I1203 12:21:44.289559 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/19a09467-4c91-40f5-b83a-d43a8eb5f2dc-metrics-certs-tls-certs" (OuterVolumeSpecName: "metrics-certs-tls-certs") pod "19a09467-4c91-40f5-b83a-d43a8eb5f2dc" (UID: "19a09467-4c91-40f5-b83a-d43a8eb5f2dc"). InnerVolumeSpecName "metrics-certs-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:44 crc kubenswrapper[5052]: I1203 12:21:44.301993 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/swift-proxy-6b9f9968d-5zjp2" podUID="fb73cc9a-89cc-4cb8-87ce-2731e5dab0f7" containerName="proxy-httpd" probeResult="failure" output="Get \"https://10.217.1.193:8080/healthcheck\": read tcp 10.217.0.2:57890->10.217.1.193:8080: read: connection reset by peer" Dec 03 12:21:44 crc kubenswrapper[5052]: I1203 12:21:44.302987 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/9165ef68-b76d-472e-99af-06cc8c8a2d3e-operator-scripts\") pod \"keystone28d1-account-delete-c9vmm\" (UID: \"9165ef68-b76d-472e-99af-06cc8c8a2d3e\") " pod="openstack-kuttl-tests/keystone28d1-account-delete-c9vmm" Dec 03 12:21:44 crc kubenswrapper[5052]: I1203 12:21:44.303025 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/bfc242a2-aca4-458b-812e-018787bdbcab-operator-scripts\") pod \"keystone-28d1-account-create-update-f27hx\" (UID: \"bfc242a2-aca4-458b-812e-018787bdbcab\") " pod="openstack-kuttl-tests/keystone-28d1-account-create-update-f27hx" Dec 03 12:21:44 crc kubenswrapper[5052]: I1203 12:21:44.303163 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-75s94\" (UniqueName: \"kubernetes.io/projected/bfc242a2-aca4-458b-812e-018787bdbcab-kube-api-access-75s94\") pod \"keystone-28d1-account-create-update-f27hx\" (UID: \"bfc242a2-aca4-458b-812e-018787bdbcab\") " pod="openstack-kuttl-tests/keystone-28d1-account-create-update-f27hx" Dec 03 12:21:44 crc kubenswrapper[5052]: I1203 12:21:44.303182 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5h59r\" (UniqueName: \"kubernetes.io/projected/9165ef68-b76d-472e-99af-06cc8c8a2d3e-kube-api-access-5h59r\") pod \"keystone28d1-account-delete-c9vmm\" (UID: \"9165ef68-b76d-472e-99af-06cc8c8a2d3e\") " pod="openstack-kuttl-tests/keystone28d1-account-delete-c9vmm" Dec 03 12:21:44 crc kubenswrapper[5052]: I1203 12:21:44.303261 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage20-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage20-crc\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:44 crc kubenswrapper[5052]: I1203 12:21:44.303271 5052 reconciler_common.go:293] "Volume detached for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/19a09467-4c91-40f5-b83a-d43a8eb5f2dc-metrics-certs-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:44 crc kubenswrapper[5052]: I1203 12:21:44.304699 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/openstack-cell1-galera-0" podUID="47aa63e7-b7b2-4c1d-a73d-348b50cc5d2b" containerName="galera" containerID="cri-o://37fd4ad16311c4c25724489aee98ab54bb95696bace9c0196b6418d1b39c27b1" gracePeriod=30 Dec 03 12:21:44 crc kubenswrapper[5052]: I1203 12:21:44.325193 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/70cedf3b-c9b9-4e5c-81a2-ba134b150997-ovsdbserver-nb-tls-certs" (OuterVolumeSpecName: "ovsdbserver-nb-tls-certs") pod "70cedf3b-c9b9-4e5c-81a2-ba134b150997" (UID: "70cedf3b-c9b9-4e5c-81a2-ba134b150997"). InnerVolumeSpecName "ovsdbserver-nb-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:44 crc kubenswrapper[5052]: I1203 12:21:44.329155 5052 generic.go:334] "Generic (PLEG): container finished" podID="4a55b492-d57f-40dc-b719-1294fb24ef13" containerID="4fdd95681555772f2043566122b6cc7d5d50f1b3bc46512f2b2e8fedf1c6d999" exitCode=0 Dec 03 12:21:44 crc kubenswrapper[5052]: I1203 12:21:44.329271 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"4a55b492-d57f-40dc-b719-1294fb24ef13","Type":"ContainerDied","Data":"4fdd95681555772f2043566122b6cc7d5d50f1b3bc46512f2b2e8fedf1c6d999"} Dec 03 12:21:44 crc kubenswrapper[5052]: I1203 12:21:44.332259 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/19a09467-4c91-40f5-b83a-d43a8eb5f2dc-ovsdbserver-sb-tls-certs" (OuterVolumeSpecName: "ovsdbserver-sb-tls-certs") pod "19a09467-4c91-40f5-b83a-d43a8eb5f2dc" (UID: "19a09467-4c91-40f5-b83a-d43a8eb5f2dc"). InnerVolumeSpecName "ovsdbserver-sb-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:44 crc kubenswrapper[5052]: I1203 12:21:44.338123 5052 generic.go:334] "Generic (PLEG): container finished" podID="9119635e-cb55-4428-9c61-b93ef2afd799" containerID="d006b275534cfdfd7a634f938440db7eeb0d006446de410abf688af6391508cb" exitCode=143 Dec 03 12:21:44 crc kubenswrapper[5052]: I1203 12:21:44.338193 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-keystone-listener-6fb665b666-ftpjr" event={"ID":"9119635e-cb55-4428-9c61-b93ef2afd799","Type":"ContainerDied","Data":"d006b275534cfdfd7a634f938440db7eeb0d006446de410abf688af6391508cb"} Dec 03 12:21:44 crc kubenswrapper[5052]: I1203 12:21:44.348345 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-734a-account-create-update-qcgft" event={"ID":"817345c8-3a42-4dea-8158-75973d9c41d9","Type":"ContainerStarted","Data":"609a1a60c18e09d802dbabfa7e561327519116592dcd204cb6b24fec223b9b18"} Dec 03 12:21:44 crc kubenswrapper[5052]: I1203 12:21:44.349213 5052 kubelet_pods.go:1007] "Unable to retrieve pull secret, the image pull may not succeed." pod="openstack-kuttl-tests/neutron-734a-account-create-update-qcgft" secret="" err="secret \"galera-openstack-dockercfg-dlq6n\" not found" Dec 03 12:21:44 crc kubenswrapper[5052]: I1203 12:21:44.355694 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/70cedf3b-c9b9-4e5c-81a2-ba134b150997-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "70cedf3b-c9b9-4e5c-81a2-ba134b150997" (UID: "70cedf3b-c9b9-4e5c-81a2-ba134b150997"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:44 crc kubenswrapper[5052]: I1203 12:21:44.361693 5052 generic.go:334] "Generic (PLEG): container finished" podID="2d217a2f-3478-41e0-936a-6b1d01530007" containerID="e124f853c8070429efcc9a6778d70b3a8c1c60fccd3f62bb22a54362a2148f4e" exitCode=143 Dec 03 12:21:44 crc kubenswrapper[5052]: I1203 12:21:44.361768 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"2d217a2f-3478-41e0-936a-6b1d01530007","Type":"ContainerDied","Data":"e124f853c8070429efcc9a6778d70b3a8c1c60fccd3f62bb22a54362a2148f4e"} Dec 03 12:21:44 crc kubenswrapper[5052]: I1203 12:21:44.368621 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/neutron-734a-account-create-update-qcgft" podStartSLOduration=4.368601884 podStartE2EDuration="4.368601884s" podCreationTimestamp="2025-12-03 12:21:40 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:21:44.364106746 +0000 UTC m=+3694.952348510" watchObservedRunningTime="2025-12-03 12:21:44.368601884 +0000 UTC m=+3694.956843648" Dec 03 12:21:44 crc kubenswrapper[5052]: E1203 12:21:44.391885 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="97a957b8c33e1b7a5cef18f4383a4faea413910406298074a6052c04396cec18" cmd=["/usr/local/bin/container-scripts/status_check.sh"] Dec 03 12:21:44 crc kubenswrapper[5052]: E1203 12:21:44.404392 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="97a957b8c33e1b7a5cef18f4383a4faea413910406298074a6052c04396cec18" cmd=["/usr/local/bin/container-scripts/status_check.sh"] Dec 03 12:21:44 crc kubenswrapper[5052]: E1203 12:21:44.407633 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="97a957b8c33e1b7a5cef18f4383a4faea413910406298074a6052c04396cec18" cmd=["/usr/local/bin/container-scripts/status_check.sh"] Dec 03 12:21:44 crc kubenswrapper[5052]: E1203 12:21:44.407673 5052 prober.go:104] "Probe errored" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" probeType="Readiness" pod="openstack-kuttl-tests/ovn-northd-0" podUID="a77df94c-3e83-47a5-9b5f-eb0aeb41bc3d" containerName="ovn-northd" Dec 03 12:21:44 crc kubenswrapper[5052]: I1203 12:21:44.413916 5052 generic.go:334] "Generic (PLEG): container finished" podID="2d94bf00-7a64-4e90-a48c-b206ec54da6d" containerID="3d06201ab438e62b35c336b3ff97f18ab1544e471098deb210f49f6607c474fe" exitCode=0 Dec 03 12:21:44 crc kubenswrapper[5052]: I1203 12:21:44.413948 5052 generic.go:334] "Generic (PLEG): container finished" podID="2d94bf00-7a64-4e90-a48c-b206ec54da6d" containerID="7a966caf1545e10cd03293bfda99f2c9475e9dcbe187dcc16f09580784759143" exitCode=2 Dec 03 12:21:44 crc kubenswrapper[5052]: I1203 12:21:44.414004 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"2d94bf00-7a64-4e90-a48c-b206ec54da6d","Type":"ContainerDied","Data":"3d06201ab438e62b35c336b3ff97f18ab1544e471098deb210f49f6607c474fe"} Dec 03 12:21:44 crc kubenswrapper[5052]: I1203 12:21:44.414039 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"2d94bf00-7a64-4e90-a48c-b206ec54da6d","Type":"ContainerDied","Data":"7a966caf1545e10cd03293bfda99f2c9475e9dcbe187dcc16f09580784759143"} Dec 03 12:21:44 crc kubenswrapper[5052]: I1203 12:21:44.421597 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-75s94\" (UniqueName: \"kubernetes.io/projected/bfc242a2-aca4-458b-812e-018787bdbcab-kube-api-access-75s94\") pod \"keystone-28d1-account-create-update-f27hx\" (UID: \"bfc242a2-aca4-458b-812e-018787bdbcab\") " pod="openstack-kuttl-tests/keystone-28d1-account-create-update-f27hx" Dec 03 12:21:44 crc kubenswrapper[5052]: I1203 12:21:44.421656 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-5h59r\" (UniqueName: \"kubernetes.io/projected/9165ef68-b76d-472e-99af-06cc8c8a2d3e-kube-api-access-5h59r\") pod \"keystone28d1-account-delete-c9vmm\" (UID: \"9165ef68-b76d-472e-99af-06cc8c8a2d3e\") " pod="openstack-kuttl-tests/keystone28d1-account-delete-c9vmm" Dec 03 12:21:44 crc kubenswrapper[5052]: I1203 12:21:44.421865 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/9165ef68-b76d-472e-99af-06cc8c8a2d3e-operator-scripts\") pod \"keystone28d1-account-delete-c9vmm\" (UID: \"9165ef68-b76d-472e-99af-06cc8c8a2d3e\") " pod="openstack-kuttl-tests/keystone28d1-account-delete-c9vmm" Dec 03 12:21:44 crc kubenswrapper[5052]: I1203 12:21:44.421903 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/bfc242a2-aca4-458b-812e-018787bdbcab-operator-scripts\") pod \"keystone-28d1-account-create-update-f27hx\" (UID: \"bfc242a2-aca4-458b-812e-018787bdbcab\") " pod="openstack-kuttl-tests/keystone-28d1-account-create-update-f27hx" Dec 03 12:21:44 crc kubenswrapper[5052]: I1203 12:21:44.422051 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/70cedf3b-c9b9-4e5c-81a2-ba134b150997-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:44 crc kubenswrapper[5052]: I1203 12:21:44.422064 5052 reconciler_common.go:293] "Volume detached for volume \"ovsdbserver-sb-tls-certs\" (UniqueName: \"kubernetes.io/secret/19a09467-4c91-40f5-b83a-d43a8eb5f2dc-ovsdbserver-sb-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:44 crc kubenswrapper[5052]: I1203 12:21:44.422079 5052 reconciler_common.go:293] "Volume detached for volume \"ovsdbserver-nb-tls-certs\" (UniqueName: \"kubernetes.io/secret/70cedf3b-c9b9-4e5c-81a2-ba134b150997-ovsdbserver-nb-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:44 crc kubenswrapper[5052]: I1203 12:21:44.422063 5052 generic.go:334] "Generic (PLEG): container finished" podID="b657a259-eb7a-48e8-8f41-2a3c077aff2a" containerID="9569b49a51e3eb2f02cd20195f982384cb40ad5f6b6fd5f65cf8a8a8fff4d592" exitCode=2 Dec 03 12:21:44 crc kubenswrapper[5052]: E1203 12:21:44.422145 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:21:44 crc kubenswrapper[5052]: I1203 12:21:44.422149 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/kube-state-metrics-0" event={"ID":"b657a259-eb7a-48e8-8f41-2a3c077aff2a","Type":"ContainerDied","Data":"9569b49a51e3eb2f02cd20195f982384cb40ad5f6b6fd5f65cf8a8a8fff4d592"} Dec 03 12:21:44 crc kubenswrapper[5052]: E1203 12:21:44.422194 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/bfc242a2-aca4-458b-812e-018787bdbcab-operator-scripts podName:bfc242a2-aca4-458b-812e-018787bdbcab nodeName:}" failed. No retries permitted until 2025-12-03 12:21:44.922180133 +0000 UTC m=+3695.510421887 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/bfc242a2-aca4-458b-812e-018787bdbcab-operator-scripts") pod "keystone-28d1-account-create-update-f27hx" (UID: "bfc242a2-aca4-458b-812e-018787bdbcab") : configmap "openstack-scripts" not found Dec 03 12:21:44 crc kubenswrapper[5052]: E1203 12:21:44.422441 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/combined-ca-bundle: secret "combined-ca-bundle" not found Dec 03 12:21:44 crc kubenswrapper[5052]: E1203 12:21:44.422474 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fb73cc9a-89cc-4cb8-87ce-2731e5dab0f7-combined-ca-bundle podName:fb73cc9a-89cc-4cb8-87ce-2731e5dab0f7 nodeName:}" failed. No retries permitted until 2025-12-03 12:21:48.422467231 +0000 UTC m=+3699.010708995 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "combined-ca-bundle" (UniqueName: "kubernetes.io/secret/fb73cc9a-89cc-4cb8-87ce-2731e5dab0f7-combined-ca-bundle") pod "swift-proxy-6b9f9968d-5zjp2" (UID: "fb73cc9a-89cc-4cb8-87ce-2731e5dab0f7") : secret "combined-ca-bundle" not found Dec 03 12:21:44 crc kubenswrapper[5052]: E1203 12:21:44.422538 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:21:44 crc kubenswrapper[5052]: E1203 12:21:44.422562 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9165ef68-b76d-472e-99af-06cc8c8a2d3e-operator-scripts podName:9165ef68-b76d-472e-99af-06cc8c8a2d3e nodeName:}" failed. No retries permitted until 2025-12-03 12:21:44.922556104 +0000 UTC m=+3695.510797868 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/9165ef68-b76d-472e-99af-06cc8c8a2d3e-operator-scripts") pod "keystone28d1-account-delete-c9vmm" (UID: "9165ef68-b76d-472e-99af-06cc8c8a2d3e") : configmap "openstack-scripts" not found Dec 03 12:21:44 crc kubenswrapper[5052]: E1203 12:21:44.422616 5052 projected.go:263] Couldn't get secret openstack-kuttl-tests/swift-proxy-config-data: secret "swift-proxy-config-data" not found Dec 03 12:21:44 crc kubenswrapper[5052]: E1203 12:21:44.422627 5052 projected.go:263] Couldn't get secret openstack-kuttl-tests/swift-conf: secret "swift-conf" not found Dec 03 12:21:44 crc kubenswrapper[5052]: E1203 12:21:44.422983 5052 projected.go:288] Couldn't get configMap openstack-kuttl-tests/swift-ring-files: configmap "swift-ring-files" not found Dec 03 12:21:44 crc kubenswrapper[5052]: E1203 12:21:44.423002 5052 projected.go:194] Error preparing data for projected volume etc-swift for pod openstack-kuttl-tests/swift-proxy-6b9f9968d-5zjp2: [secret "swift-proxy-config-data" not found, secret "swift-conf" not found, configmap "swift-ring-files" not found] Dec 03 12:21:44 crc kubenswrapper[5052]: E1203 12:21:44.423032 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fb73cc9a-89cc-4cb8-87ce-2731e5dab0f7-etc-swift podName:fb73cc9a-89cc-4cb8-87ce-2731e5dab0f7 nodeName:}" failed. No retries permitted until 2025-12-03 12:21:48.423024577 +0000 UTC m=+3699.011266341 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "etc-swift" (UniqueName: "kubernetes.io/projected/fb73cc9a-89cc-4cb8-87ce-2731e5dab0f7-etc-swift") pod "swift-proxy-6b9f9968d-5zjp2" (UID: "fb73cc9a-89cc-4cb8-87ce-2731e5dab0f7") : [secret "swift-proxy-config-data" not found, secret "swift-conf" not found, configmap "swift-ring-files" not found] Dec 03 12:21:44 crc kubenswrapper[5052]: E1203 12:21:44.425501 5052 projected.go:194] Error preparing data for projected volume kube-api-access-5h59r for pod openstack-kuttl-tests/keystone28d1-account-delete-c9vmm: failed to fetch token: serviceaccounts "galera-openstack" not found Dec 03 12:21:44 crc kubenswrapper[5052]: E1203 12:21:44.425599 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9165ef68-b76d-472e-99af-06cc8c8a2d3e-kube-api-access-5h59r podName:9165ef68-b76d-472e-99af-06cc8c8a2d3e nodeName:}" failed. No retries permitted until 2025-12-03 12:21:44.92557534 +0000 UTC m=+3695.513817104 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-5h59r" (UniqueName: "kubernetes.io/projected/9165ef68-b76d-472e-99af-06cc8c8a2d3e-kube-api-access-5h59r") pod "keystone28d1-account-delete-c9vmm" (UID: "9165ef68-b76d-472e-99af-06cc8c8a2d3e") : failed to fetch token: serviceaccounts "galera-openstack" not found Dec 03 12:21:44 crc kubenswrapper[5052]: E1203 12:21:44.426115 5052 projected.go:194] Error preparing data for projected volume kube-api-access-75s94 for pod openstack-kuttl-tests/keystone-28d1-account-create-update-f27hx: failed to fetch token: serviceaccounts "galera-openstack" not found Dec 03 12:21:44 crc kubenswrapper[5052]: E1203 12:21:44.426167 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bfc242a2-aca4-458b-812e-018787bdbcab-kube-api-access-75s94 podName:bfc242a2-aca4-458b-812e-018787bdbcab nodeName:}" failed. No retries permitted until 2025-12-03 12:21:44.926149857 +0000 UTC m=+3695.514391631 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-75s94" (UniqueName: "kubernetes.io/projected/bfc242a2-aca4-458b-812e-018787bdbcab-kube-api-access-75s94") pod "keystone-28d1-account-create-update-f27hx" (UID: "bfc242a2-aca4-458b-812e-018787bdbcab") : failed to fetch token: serviceaccounts "galera-openstack" not found Dec 03 12:21:44 crc kubenswrapper[5052]: I1203 12:21:44.432702 5052 generic.go:334] "Generic (PLEG): container finished" podID="72b87ae1-6854-475b-a31d-976acc913623" containerID="dc3b9cb9559bad706ad6fd805545ec2569385c158301c0e04856678a08e6dfe0" exitCode=143 Dec 03 12:21:44 crc kubenswrapper[5052]: I1203 12:21:44.432798 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"72b87ae1-6854-475b-a31d-976acc913623","Type":"ContainerDied","Data":"dc3b9cb9559bad706ad6fd805545ec2569385c158301c0e04856678a08e6dfe0"} Dec 03 12:21:44 crc kubenswrapper[5052]: I1203 12:21:44.445012 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_ovsdbserver-sb-0_19a09467-4c91-40f5-b83a-d43a8eb5f2dc/ovsdbserver-sb/0.log" Dec 03 12:21:44 crc kubenswrapper[5052]: I1203 12:21:44.445830 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-sb-0" event={"ID":"19a09467-4c91-40f5-b83a-d43a8eb5f2dc","Type":"ContainerDied","Data":"0642a54f4a79fd94ab1beba706cc6d89bfd392723aef5b869ebade6546d4c912"} Dec 03 12:21:44 crc kubenswrapper[5052]: I1203 12:21:44.445874 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:21:44 crc kubenswrapper[5052]: I1203 12:21:44.451683 5052 generic.go:334] "Generic (PLEG): container finished" podID="484e6cd1-fb57-4416-9a1a-cf24b4693764" containerID="01a0969c60e54d821f713b7a30fadf577164bb47e2f3fc851bc640b779f911d6" exitCode=143 Dec 03 12:21:44 crc kubenswrapper[5052]: I1203 12:21:44.451789 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-worker-6b4674c647-56v84" event={"ID":"484e6cd1-fb57-4416-9a1a-cf24b4693764","Type":"ContainerDied","Data":"01a0969c60e54d821f713b7a30fadf577164bb47e2f3fc851bc640b779f911d6"} Dec 03 12:21:44 crc kubenswrapper[5052]: I1203 12:21:44.464518 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance73ca-account-delete-vpqhp" event={"ID":"db31774e-8f50-4862-86f3-93cb93f645a0","Type":"ContainerStarted","Data":"9661574f87aebd235d130b4c19903ffb15d5afc4025f54b00c2da496a34e0aad"} Dec 03 12:21:44 crc kubenswrapper[5052]: I1203 12:21:44.465715 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/70cedf3b-c9b9-4e5c-81a2-ba134b150997-metrics-certs-tls-certs" (OuterVolumeSpecName: "metrics-certs-tls-certs") pod "70cedf3b-c9b9-4e5c-81a2-ba134b150997" (UID: "70cedf3b-c9b9-4e5c-81a2-ba134b150997"). InnerVolumeSpecName "metrics-certs-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:44 crc kubenswrapper[5052]: I1203 12:21:44.467216 5052 kubelet_pods.go:1007] "Unable to retrieve pull secret, the image pull may not succeed." pod="openstack-kuttl-tests/glance73ca-account-delete-vpqhp" secret="" err="secret \"galera-openstack-dockercfg-dlq6n\" not found" Dec 03 12:21:44 crc kubenswrapper[5052]: I1203 12:21:44.503277 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/glance73ca-account-delete-vpqhp" podStartSLOduration=3.5032498370000003 podStartE2EDuration="3.503249837s" podCreationTimestamp="2025-12-03 12:21:41 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:21:44.478726827 +0000 UTC m=+3695.066968591" watchObservedRunningTime="2025-12-03 12:21:44.503249837 +0000 UTC m=+3695.091491601" Dec 03 12:21:44 crc kubenswrapper[5052]: I1203 12:21:44.513383 5052 generic.go:334] "Generic (PLEG): container finished" podID="0b121234-d24f-4307-9409-59a7dad3df4c" containerID="eac0ccbf31e29898c07277ff8b3f25e720d684db3945841e058ea63b8f2fc50e" exitCode=0 Dec 03 12:21:44 crc kubenswrapper[5052]: I1203 12:21:44.513415 5052 generic.go:334] "Generic (PLEG): container finished" podID="0b121234-d24f-4307-9409-59a7dad3df4c" containerID="9826c564d51675cc0d19bfb280f54c6cdb8da70e11a2735f3456a7c16c5b0331" exitCode=0 Dec 03 12:21:44 crc kubenswrapper[5052]: I1203 12:21:44.513426 5052 generic.go:334] "Generic (PLEG): container finished" podID="0b121234-d24f-4307-9409-59a7dad3df4c" containerID="38ac8f54d7f675400878ab50b896bde2d4d40ee63497af23ebdca0f75d215cd7" exitCode=0 Dec 03 12:21:44 crc kubenswrapper[5052]: I1203 12:21:44.513434 5052 generic.go:334] "Generic (PLEG): container finished" podID="0b121234-d24f-4307-9409-59a7dad3df4c" containerID="a108f5ccd3086ba9eada83058d904fc18a5b2e825c993e540cb30d0f9ced71e9" exitCode=0 Dec 03 12:21:44 crc kubenswrapper[5052]: I1203 12:21:44.513442 5052 generic.go:334] "Generic (PLEG): container finished" podID="0b121234-d24f-4307-9409-59a7dad3df4c" containerID="a7be0eed6906ff248e67daeb0cb8433a1ab73cf3e47159ff1cbfe4029900aa36" exitCode=0 Dec 03 12:21:44 crc kubenswrapper[5052]: I1203 12:21:44.513450 5052 generic.go:334] "Generic (PLEG): container finished" podID="0b121234-d24f-4307-9409-59a7dad3df4c" containerID="93d4d6d903155179da2cc815475bcc4c3e646bd25173aec67f526949d47734f0" exitCode=0 Dec 03 12:21:44 crc kubenswrapper[5052]: I1203 12:21:44.513458 5052 generic.go:334] "Generic (PLEG): container finished" podID="0b121234-d24f-4307-9409-59a7dad3df4c" containerID="280a6f9eeba9635ba75d3f89698a54d959866a87a84e1c34632aae63f6162f88" exitCode=0 Dec 03 12:21:44 crc kubenswrapper[5052]: I1203 12:21:44.513467 5052 generic.go:334] "Generic (PLEG): container finished" podID="0b121234-d24f-4307-9409-59a7dad3df4c" containerID="76aa0661476cea8be8d11604d15492437f317021e6ae641e33f6ca912d72fe2e" exitCode=0 Dec 03 12:21:44 crc kubenswrapper[5052]: I1203 12:21:44.513475 5052 generic.go:334] "Generic (PLEG): container finished" podID="0b121234-d24f-4307-9409-59a7dad3df4c" containerID="1fb344079bdd4d88652a84cde52ac882ef46f1e1b483946b91fa06958b9781b1" exitCode=0 Dec 03 12:21:44 crc kubenswrapper[5052]: I1203 12:21:44.513483 5052 generic.go:334] "Generic (PLEG): container finished" podID="0b121234-d24f-4307-9409-59a7dad3df4c" containerID="7448a246780facb87faed946ab976ad223f836a90fd0ac6cbf876c7f11c6c2d8" exitCode=0 Dec 03 12:21:44 crc kubenswrapper[5052]: I1203 12:21:44.513491 5052 generic.go:334] "Generic (PLEG): container finished" podID="0b121234-d24f-4307-9409-59a7dad3df4c" containerID="70f7ea9905786e034fff8dcf46682040188eb7ca7903f795173a38888c5b11d9" exitCode=0 Dec 03 12:21:44 crc kubenswrapper[5052]: I1203 12:21:44.513499 5052 generic.go:334] "Generic (PLEG): container finished" podID="0b121234-d24f-4307-9409-59a7dad3df4c" containerID="1de28481ebd24dff85a8213026dad78dd91bf4d0b6c4d77c2b5231b95244a4cb" exitCode=0 Dec 03 12:21:44 crc kubenswrapper[5052]: I1203 12:21:44.513529 5052 generic.go:334] "Generic (PLEG): container finished" podID="0b121234-d24f-4307-9409-59a7dad3df4c" containerID="914125fe1587ea7c446a5334d9e2828cf4834d6ad4f2053f842f07ad24a0e67d" exitCode=0 Dec 03 12:21:44 crc kubenswrapper[5052]: I1203 12:21:44.513530 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"0b121234-d24f-4307-9409-59a7dad3df4c","Type":"ContainerDied","Data":"eac0ccbf31e29898c07277ff8b3f25e720d684db3945841e058ea63b8f2fc50e"} Dec 03 12:21:44 crc kubenswrapper[5052]: I1203 12:21:44.513557 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"0b121234-d24f-4307-9409-59a7dad3df4c","Type":"ContainerDied","Data":"9826c564d51675cc0d19bfb280f54c6cdb8da70e11a2735f3456a7c16c5b0331"} Dec 03 12:21:44 crc kubenswrapper[5052]: I1203 12:21:44.513567 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"0b121234-d24f-4307-9409-59a7dad3df4c","Type":"ContainerDied","Data":"38ac8f54d7f675400878ab50b896bde2d4d40ee63497af23ebdca0f75d215cd7"} Dec 03 12:21:44 crc kubenswrapper[5052]: I1203 12:21:44.513576 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"0b121234-d24f-4307-9409-59a7dad3df4c","Type":"ContainerDied","Data":"a108f5ccd3086ba9eada83058d904fc18a5b2e825c993e540cb30d0f9ced71e9"} Dec 03 12:21:44 crc kubenswrapper[5052]: I1203 12:21:44.513584 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"0b121234-d24f-4307-9409-59a7dad3df4c","Type":"ContainerDied","Data":"a7be0eed6906ff248e67daeb0cb8433a1ab73cf3e47159ff1cbfe4029900aa36"} Dec 03 12:21:44 crc kubenswrapper[5052]: I1203 12:21:44.513592 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"0b121234-d24f-4307-9409-59a7dad3df4c","Type":"ContainerDied","Data":"93d4d6d903155179da2cc815475bcc4c3e646bd25173aec67f526949d47734f0"} Dec 03 12:21:44 crc kubenswrapper[5052]: I1203 12:21:44.513601 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"0b121234-d24f-4307-9409-59a7dad3df4c","Type":"ContainerDied","Data":"280a6f9eeba9635ba75d3f89698a54d959866a87a84e1c34632aae63f6162f88"} Dec 03 12:21:44 crc kubenswrapper[5052]: I1203 12:21:44.513609 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"0b121234-d24f-4307-9409-59a7dad3df4c","Type":"ContainerDied","Data":"76aa0661476cea8be8d11604d15492437f317021e6ae641e33f6ca912d72fe2e"} Dec 03 12:21:44 crc kubenswrapper[5052]: I1203 12:21:44.513617 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"0b121234-d24f-4307-9409-59a7dad3df4c","Type":"ContainerDied","Data":"1fb344079bdd4d88652a84cde52ac882ef46f1e1b483946b91fa06958b9781b1"} Dec 03 12:21:44 crc kubenswrapper[5052]: I1203 12:21:44.513540 5052 generic.go:334] "Generic (PLEG): container finished" podID="0b121234-d24f-4307-9409-59a7dad3df4c" containerID="6b7c8f662acc40a3137f223890e54394fd6013bb9203453b7285ac28a63f277d" exitCode=0 Dec 03 12:21:44 crc kubenswrapper[5052]: I1203 12:21:44.513625 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"0b121234-d24f-4307-9409-59a7dad3df4c","Type":"ContainerDied","Data":"7448a246780facb87faed946ab976ad223f836a90fd0ac6cbf876c7f11c6c2d8"} Dec 03 12:21:44 crc kubenswrapper[5052]: I1203 12:21:44.513712 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"0b121234-d24f-4307-9409-59a7dad3df4c","Type":"ContainerDied","Data":"70f7ea9905786e034fff8dcf46682040188eb7ca7903f795173a38888c5b11d9"} Dec 03 12:21:44 crc kubenswrapper[5052]: I1203 12:21:44.513748 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"0b121234-d24f-4307-9409-59a7dad3df4c","Type":"ContainerDied","Data":"1de28481ebd24dff85a8213026dad78dd91bf4d0b6c4d77c2b5231b95244a4cb"} Dec 03 12:21:44 crc kubenswrapper[5052]: I1203 12:21:44.513761 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"0b121234-d24f-4307-9409-59a7dad3df4c","Type":"ContainerDied","Data":"914125fe1587ea7c446a5334d9e2828cf4834d6ad4f2053f842f07ad24a0e67d"} Dec 03 12:21:44 crc kubenswrapper[5052]: I1203 12:21:44.513773 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"0b121234-d24f-4307-9409-59a7dad3df4c","Type":"ContainerDied","Data":"6b7c8f662acc40a3137f223890e54394fd6013bb9203453b7285ac28a63f277d"} Dec 03 12:21:44 crc kubenswrapper[5052]: I1203 12:21:44.524653 5052 reconciler_common.go:293] "Volume detached for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/70cedf3b-c9b9-4e5c-81a2-ba134b150997-metrics-certs-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:44 crc kubenswrapper[5052]: E1203 12:21:44.525003 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:21:44 crc kubenswrapper[5052]: E1203 12:21:44.525079 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/817345c8-3a42-4dea-8158-75973d9c41d9-operator-scripts podName:817345c8-3a42-4dea-8158-75973d9c41d9 nodeName:}" failed. No retries permitted until 2025-12-03 12:21:45.02506138 +0000 UTC m=+3695.613303144 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/817345c8-3a42-4dea-8158-75973d9c41d9-operator-scripts") pod "neutron-734a-account-create-update-qcgft" (UID: "817345c8-3a42-4dea-8158-75973d9c41d9") : configmap "openstack-scripts" not found Dec 03 12:21:44 crc kubenswrapper[5052]: I1203 12:21:44.525329 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-sb-0"] Dec 03 12:21:44 crc kubenswrapper[5052]: I1203 12:21:44.529424 5052 generic.go:334] "Generic (PLEG): container finished" podID="7e109538-a2d2-43bb-8cef-bf99d7d0f1e9" containerID="17114cefe25d8670369fc24018b4320e72bd1b81aa69b034a694652ddb010caa" exitCode=0 Dec 03 12:21:44 crc kubenswrapper[5052]: I1203 12:21:44.529534 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-proxy-d4747d79f-wtbhv" event={"ID":"7e109538-a2d2-43bb-8cef-bf99d7d0f1e9","Type":"ContainerDied","Data":"17114cefe25d8670369fc24018b4320e72bd1b81aa69b034a694652ddb010caa"} Dec 03 12:21:44 crc kubenswrapper[5052]: I1203 12:21:44.535300 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-sb-0"] Dec 03 12:21:44 crc kubenswrapper[5052]: I1203 12:21:44.539153 5052 generic.go:334] "Generic (PLEG): container finished" podID="bb41ed7e-c96b-44dd-8c4e-ffb702de3f87" containerID="19ad1a127fb47796fec06e903f60d68c562fce37f707be6c98c93f3e0d2d25cd" exitCode=143 Dec 03 12:21:44 crc kubenswrapper[5052]: I1203 12:21:44.539306 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"bb41ed7e-c96b-44dd-8c4e-ffb702de3f87","Type":"ContainerDied","Data":"19ad1a127fb47796fec06e903f60d68c562fce37f707be6c98c93f3e0d2d25cd"} Dec 03 12:21:44 crc kubenswrapper[5052]: I1203 12:21:44.551603 5052 scope.go:117] "RemoveContainer" containerID="81e8dead6ae4e8433ea10d47201edd08f411e7f097f98bf35c829bcf51a695f9" Dec 03 12:21:44 crc kubenswrapper[5052]: I1203 12:21:44.564224 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/openstack-galera-0" podUID="0988bafc-df97-4a53-8308-9e992be58d9a" containerName="galera" containerID="cri-o://8a1128866a1e3dbe3fe50f0d8c53319b6d90d894631644af6969f5f9926f2eaa" gracePeriod=30 Dec 03 12:21:44 crc kubenswrapper[5052]: I1203 12:21:44.569007 5052 generic.go:334] "Generic (PLEG): container finished" podID="af38eb0d-3924-4c4b-a22e-7e934bdcf172" containerID="e699b862f9953c2e25b19ab9d64356e9c8a7220759bf33ca05a10010b2639fc3" exitCode=143 Dec 03 12:21:44 crc kubenswrapper[5052]: I1203 12:21:44.569059 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-644f564b46-cjtkx" event={"ID":"af38eb0d-3924-4c4b-a22e-7e934bdcf172","Type":"ContainerDied","Data":"e699b862f9953c2e25b19ab9d64356e9c8a7220759bf33ca05a10010b2639fc3"} Dec 03 12:21:44 crc kubenswrapper[5052]: I1203 12:21:44.577727 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement9152-account-delete-x7q46" event={"ID":"4f7ab0ec-52a7-4be2-9886-04bf98989b63","Type":"ContainerStarted","Data":"0c28dc5ab7c2ba469dfbfce79e80be331f7d22e6d6ea521fdd0d4304a576ca02"} Dec 03 12:21:44 crc kubenswrapper[5052]: I1203 12:21:44.594710 5052 generic.go:334] "Generic (PLEG): container finished" podID="892f0ab2-8602-4485-a11c-5171252e2a7b" containerID="5bdabbc1acf777d8b66b937a8a2bfb67eb3f1e9fde05ff4a60b0028fe9914ce7" exitCode=137 Dec 03 12:21:44 crc kubenswrapper[5052]: I1203 12:21:44.623557 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_ovsdbserver-nb-0_70cedf3b-c9b9-4e5c-81a2-ba134b150997/ovsdbserver-nb/0.log" Dec 03 12:21:44 crc kubenswrapper[5052]: I1203 12:21:44.623659 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-nb-0" event={"ID":"70cedf3b-c9b9-4e5c-81a2-ba134b150997","Type":"ContainerDied","Data":"b64aba72338acf63ff50b518818d5d916ca0c4e6b7b9793a282a7970c1cba540"} Dec 03 12:21:44 crc kubenswrapper[5052]: I1203 12:21:44.623778 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:21:44 crc kubenswrapper[5052]: E1203 12:21:44.626444 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:21:44 crc kubenswrapper[5052]: E1203 12:21:44.626530 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/db31774e-8f50-4862-86f3-93cb93f645a0-operator-scripts podName:db31774e-8f50-4862-86f3-93cb93f645a0 nodeName:}" failed. No retries permitted until 2025-12-03 12:21:45.126496265 +0000 UTC m=+3695.714738159 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/db31774e-8f50-4862-86f3-93cb93f645a0-operator-scripts") pod "glance73ca-account-delete-vpqhp" (UID: "db31774e-8f50-4862-86f3-93cb93f645a0") : configmap "openstack-scripts" not found Dec 03 12:21:44 crc kubenswrapper[5052]: I1203 12:21:44.644863 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstackclient" Dec 03 12:21:44 crc kubenswrapper[5052]: I1203 12:21:44.645181 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone28d1-account-delete-c9vmm" Dec 03 12:21:44 crc kubenswrapper[5052]: I1203 12:21:44.645274 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-api-0" podUID="9dec0a31-cba3-4c38-8b5d-4358f16b3704" containerName="nova-api-log" containerID="cri-o://72c6e09b72f57c6f1f19ff9c617f85acc4ceb1d16fb1a62383bab5dea1e10644" gracePeriod=30 Dec 03 12:21:44 crc kubenswrapper[5052]: I1203 12:21:44.645433 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-api-0" podUID="9dec0a31-cba3-4c38-8b5d-4358f16b3704" containerName="nova-api-api" containerID="cri-o://7f8633f82aa0da6af5e1f867996abc38506c65e6d6ca569fc68ad7d8ed0d1cf1" gracePeriod=30 Dec 03 12:21:44 crc kubenswrapper[5052]: I1203 12:21:44.645747 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"9dec0a31-cba3-4c38-8b5d-4358f16b3704","Type":"ContainerStarted","Data":"7f8633f82aa0da6af5e1f867996abc38506c65e6d6ca569fc68ad7d8ed0d1cf1"} Dec 03 12:21:44 crc kubenswrapper[5052]: I1203 12:21:44.645808 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/novacell13fba-account-delete-hvzdt" Dec 03 12:21:44 crc kubenswrapper[5052]: I1203 12:21:44.645896 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-28d1-account-create-update-f27hx" Dec 03 12:21:44 crc kubenswrapper[5052]: I1203 12:21:44.645957 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:21:44 crc kubenswrapper[5052]: I1203 12:21:44.689879 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-api-0" podStartSLOduration=5.689856643 podStartE2EDuration="5.689856643s" podCreationTimestamp="2025-12-03 12:21:39 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:21:44.677874761 +0000 UTC m=+3695.266116535" watchObservedRunningTime="2025-12-03 12:21:44.689856643 +0000 UTC m=+3695.278098407" Dec 03 12:21:44 crc kubenswrapper[5052]: E1203 12:21:44.728039 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/rabbitmq-config-data: configmap "rabbitmq-config-data" not found Dec 03 12:21:44 crc kubenswrapper[5052]: E1203 12:21:44.728101 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c14a906c-f19d-4806-9110-8c47b38c5ae0-config-data podName:c14a906c-f19d-4806-9110-8c47b38c5ae0 nodeName:}" failed. No retries permitted until 2025-12-03 12:21:48.728087525 +0000 UTC m=+3699.316329289 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "config-data" (UniqueName: "kubernetes.io/configmap/c14a906c-f19d-4806-9110-8c47b38c5ae0-config-data") pod "rabbitmq-server-0" (UID: "c14a906c-f19d-4806-9110-8c47b38c5ae0") : configmap "rabbitmq-config-data" not found Dec 03 12:21:44 crc kubenswrapper[5052]: E1203 12:21:44.835393 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-cell1-scripts: configmap "openstack-cell1-scripts" not found Dec 03 12:21:44 crc kubenswrapper[5052]: I1203 12:21:44.835221 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/0f00d4de-f49c-4155-98e7-00be8bc4ddc7-operator-scripts\") pod \"novacell13fba-account-delete-hvzdt\" (UID: \"0f00d4de-f49c-4155-98e7-00be8bc4ddc7\") " pod="openstack-kuttl-tests/novacell13fba-account-delete-hvzdt" Dec 03 12:21:44 crc kubenswrapper[5052]: E1203 12:21:44.835737 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0f00d4de-f49c-4155-98e7-00be8bc4ddc7-operator-scripts podName:0f00d4de-f49c-4155-98e7-00be8bc4ddc7 nodeName:}" failed. No retries permitted until 2025-12-03 12:21:46.835713606 +0000 UTC m=+3697.423955410 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/0f00d4de-f49c-4155-98e7-00be8bc4ddc7-operator-scripts") pod "novacell13fba-account-delete-hvzdt" (UID: "0f00d4de-f49c-4155-98e7-00be8bc4ddc7") : configmap "openstack-cell1-scripts" not found Dec 03 12:21:44 crc kubenswrapper[5052]: I1203 12:21:44.953626 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/bfc242a2-aca4-458b-812e-018787bdbcab-operator-scripts\") pod \"keystone-28d1-account-create-update-f27hx\" (UID: \"bfc242a2-aca4-458b-812e-018787bdbcab\") " pod="openstack-kuttl-tests/keystone-28d1-account-create-update-f27hx" Dec 03 12:21:44 crc kubenswrapper[5052]: I1203 12:21:44.953727 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-vvdpm\" (UniqueName: \"kubernetes.io/projected/0f00d4de-f49c-4155-98e7-00be8bc4ddc7-kube-api-access-vvdpm\") pod \"novacell13fba-account-delete-hvzdt\" (UID: \"0f00d4de-f49c-4155-98e7-00be8bc4ddc7\") " pod="openstack-kuttl-tests/novacell13fba-account-delete-hvzdt" Dec 03 12:21:44 crc kubenswrapper[5052]: I1203 12:21:44.953894 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-75s94\" (UniqueName: \"kubernetes.io/projected/bfc242a2-aca4-458b-812e-018787bdbcab-kube-api-access-75s94\") pod \"keystone-28d1-account-create-update-f27hx\" (UID: \"bfc242a2-aca4-458b-812e-018787bdbcab\") " pod="openstack-kuttl-tests/keystone-28d1-account-create-update-f27hx" Dec 03 12:21:44 crc kubenswrapper[5052]: I1203 12:21:44.953920 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-5h59r\" (UniqueName: \"kubernetes.io/projected/9165ef68-b76d-472e-99af-06cc8c8a2d3e-kube-api-access-5h59r\") pod \"keystone28d1-account-delete-c9vmm\" (UID: \"9165ef68-b76d-472e-99af-06cc8c8a2d3e\") " pod="openstack-kuttl-tests/keystone28d1-account-delete-c9vmm" Dec 03 12:21:44 crc kubenswrapper[5052]: I1203 12:21:44.954015 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e3a73fa0-68ef-4e03-81f9-2a143ed805bf-combined-ca-bundle\") pod \"nova-scheduler-0\" (UID: \"e3a73fa0-68ef-4e03-81f9-2a143ed805bf\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:21:44 crc kubenswrapper[5052]: I1203 12:21:44.954082 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/9165ef68-b76d-472e-99af-06cc8c8a2d3e-operator-scripts\") pod \"keystone28d1-account-delete-c9vmm\" (UID: \"9165ef68-b76d-472e-99af-06cc8c8a2d3e\") " pod="openstack-kuttl-tests/keystone28d1-account-delete-c9vmm" Dec 03 12:21:44 crc kubenswrapper[5052]: E1203 12:21:44.954213 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:21:44 crc kubenswrapper[5052]: E1203 12:21:44.954271 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9165ef68-b76d-472e-99af-06cc8c8a2d3e-operator-scripts podName:9165ef68-b76d-472e-99af-06cc8c8a2d3e nodeName:}" failed. No retries permitted until 2025-12-03 12:21:45.95425377 +0000 UTC m=+3696.542495534 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/9165ef68-b76d-472e-99af-06cc8c8a2d3e-operator-scripts") pod "keystone28d1-account-delete-c9vmm" (UID: "9165ef68-b76d-472e-99af-06cc8c8a2d3e") : configmap "openstack-scripts" not found Dec 03 12:21:44 crc kubenswrapper[5052]: E1203 12:21:44.955586 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/combined-ca-bundle: secret "combined-ca-bundle" not found Dec 03 12:21:44 crc kubenswrapper[5052]: E1203 12:21:44.955645 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3a73fa0-68ef-4e03-81f9-2a143ed805bf-combined-ca-bundle podName:e3a73fa0-68ef-4e03-81f9-2a143ed805bf nodeName:}" failed. No retries permitted until 2025-12-03 12:21:48.955628919 +0000 UTC m=+3699.543870683 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "combined-ca-bundle" (UniqueName: "kubernetes.io/secret/e3a73fa0-68ef-4e03-81f9-2a143ed805bf-combined-ca-bundle") pod "nova-scheduler-0" (UID: "e3a73fa0-68ef-4e03-81f9-2a143ed805bf") : secret "combined-ca-bundle" not found Dec 03 12:21:44 crc kubenswrapper[5052]: E1203 12:21:44.955713 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:21:44 crc kubenswrapper[5052]: E1203 12:21:44.955784 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/bfc242a2-aca4-458b-812e-018787bdbcab-operator-scripts podName:bfc242a2-aca4-458b-812e-018787bdbcab nodeName:}" failed. No retries permitted until 2025-12-03 12:21:45.955732932 +0000 UTC m=+3696.543974696 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/bfc242a2-aca4-458b-812e-018787bdbcab-operator-scripts") pod "keystone-28d1-account-create-update-f27hx" (UID: "bfc242a2-aca4-458b-812e-018787bdbcab") : configmap "openstack-scripts" not found Dec 03 12:21:44 crc kubenswrapper[5052]: E1203 12:21:44.959481 5052 projected.go:194] Error preparing data for projected volume kube-api-access-5h59r for pod openstack-kuttl-tests/keystone28d1-account-delete-c9vmm: failed to fetch token: serviceaccounts "galera-openstack" not found Dec 03 12:21:44 crc kubenswrapper[5052]: E1203 12:21:44.959570 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9165ef68-b76d-472e-99af-06cc8c8a2d3e-kube-api-access-5h59r podName:9165ef68-b76d-472e-99af-06cc8c8a2d3e nodeName:}" failed. No retries permitted until 2025-12-03 12:21:45.959551151 +0000 UTC m=+3696.547792975 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-5h59r" (UniqueName: "kubernetes.io/projected/9165ef68-b76d-472e-99af-06cc8c8a2d3e-kube-api-access-5h59r") pod "keystone28d1-account-delete-c9vmm" (UID: "9165ef68-b76d-472e-99af-06cc8c8a2d3e") : failed to fetch token: serviceaccounts "galera-openstack" not found Dec 03 12:21:44 crc kubenswrapper[5052]: E1203 12:21:44.959729 5052 projected.go:194] Error preparing data for projected volume kube-api-access-vvdpm for pod openstack-kuttl-tests/novacell13fba-account-delete-hvzdt: failed to fetch token: serviceaccounts "galera-openstack-cell1" not found Dec 03 12:21:44 crc kubenswrapper[5052]: E1203 12:21:44.959772 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0f00d4de-f49c-4155-98e7-00be8bc4ddc7-kube-api-access-vvdpm podName:0f00d4de-f49c-4155-98e7-00be8bc4ddc7 nodeName:}" failed. No retries permitted until 2025-12-03 12:21:46.959764627 +0000 UTC m=+3697.548006391 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-vvdpm" (UniqueName: "kubernetes.io/projected/0f00d4de-f49c-4155-98e7-00be8bc4ddc7-kube-api-access-vvdpm") pod "novacell13fba-account-delete-hvzdt" (UID: "0f00d4de-f49c-4155-98e7-00be8bc4ddc7") : failed to fetch token: serviceaccounts "galera-openstack-cell1" not found Dec 03 12:21:44 crc kubenswrapper[5052]: E1203 12:21:44.959893 5052 projected.go:194] Error preparing data for projected volume kube-api-access-75s94 for pod openstack-kuttl-tests/keystone-28d1-account-create-update-f27hx: failed to fetch token: serviceaccounts "galera-openstack" not found Dec 03 12:21:44 crc kubenswrapper[5052]: E1203 12:21:44.959930 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bfc242a2-aca4-458b-812e-018787bdbcab-kube-api-access-75s94 podName:bfc242a2-aca4-458b-812e-018787bdbcab nodeName:}" failed. No retries permitted until 2025-12-03 12:21:45.959922752 +0000 UTC m=+3696.548164516 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-75s94" (UniqueName: "kubernetes.io/projected/bfc242a2-aca4-458b-812e-018787bdbcab-kube-api-access-75s94") pod "keystone-28d1-account-create-update-f27hx" (UID: "bfc242a2-aca4-458b-812e-018787bdbcab") : failed to fetch token: serviceaccounts "galera-openstack" not found Dec 03 12:21:44 crc kubenswrapper[5052]: I1203 12:21:44.974897 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/neutron-734a-account-create-update-qcgft"] Dec 03 12:21:44 crc kubenswrapper[5052]: I1203 12:21:44.992613 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/neutron734a-account-delete-sxpn9"] Dec 03 12:21:45 crc kubenswrapper[5052]: I1203 12:21:45.003608 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone28d1-account-delete-c9vmm" Dec 03 12:21:45 crc kubenswrapper[5052]: I1203 12:21:45.005865 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/novacell13fba-account-delete-hvzdt" Dec 03 12:21:45 crc kubenswrapper[5052]: I1203 12:21:45.037305 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-28d1-account-create-update-f27hx" Dec 03 12:21:45 crc kubenswrapper[5052]: E1203 12:21:45.062097 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:21:45 crc kubenswrapper[5052]: E1203 12:21:45.062182 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/817345c8-3a42-4dea-8158-75973d9c41d9-operator-scripts podName:817345c8-3a42-4dea-8158-75973d9c41d9 nodeName:}" failed. No retries permitted until 2025-12-03 12:21:46.06216532 +0000 UTC m=+3696.650407084 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/817345c8-3a42-4dea-8158-75973d9c41d9-operator-scripts") pod "neutron-734a-account-create-update-qcgft" (UID: "817345c8-3a42-4dea-8158-75973d9c41d9") : configmap "openstack-scripts" not found Dec 03 12:21:45 crc kubenswrapper[5052]: I1203 12:21:45.062404 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-nb-0"] Dec 03 12:21:45 crc kubenswrapper[5052]: I1203 12:21:45.073394 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-nb-0"] Dec 03 12:21:45 crc kubenswrapper[5052]: I1203 12:21:45.081260 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/placement9152-account-delete-x7q46"] Dec 03 12:21:45 crc kubenswrapper[5052]: I1203 12:21:45.113025 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:21:45 crc kubenswrapper[5052]: I1203 12:21:45.163759 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/e3a73fa0-68ef-4e03-81f9-2a143ed805bf-config-data\") pod \"e3a73fa0-68ef-4e03-81f9-2a143ed805bf\" (UID: \"e3a73fa0-68ef-4e03-81f9-2a143ed805bf\") " Dec 03 12:21:45 crc kubenswrapper[5052]: I1203 12:21:45.163914 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gtxmr\" (UniqueName: \"kubernetes.io/projected/e3a73fa0-68ef-4e03-81f9-2a143ed805bf-kube-api-access-gtxmr\") pod \"e3a73fa0-68ef-4e03-81f9-2a143ed805bf\" (UID: \"e3a73fa0-68ef-4e03-81f9-2a143ed805bf\") " Dec 03 12:21:45 crc kubenswrapper[5052]: E1203 12:21:45.164686 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:21:45 crc kubenswrapper[5052]: E1203 12:21:45.164739 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/db31774e-8f50-4862-86f3-93cb93f645a0-operator-scripts podName:db31774e-8f50-4862-86f3-93cb93f645a0 nodeName:}" failed. No retries permitted until 2025-12-03 12:21:46.164725407 +0000 UTC m=+3696.752967171 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/db31774e-8f50-4862-86f3-93cb93f645a0-operator-scripts") pod "glance73ca-account-delete-vpqhp" (UID: "db31774e-8f50-4862-86f3-93cb93f645a0") : configmap "openstack-scripts" not found Dec 03 12:21:45 crc kubenswrapper[5052]: I1203 12:21:45.179056 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e3a73fa0-68ef-4e03-81f9-2a143ed805bf-kube-api-access-gtxmr" (OuterVolumeSpecName: "kube-api-access-gtxmr") pod "e3a73fa0-68ef-4e03-81f9-2a143ed805bf" (UID: "e3a73fa0-68ef-4e03-81f9-2a143ed805bf"). InnerVolumeSpecName "kube-api-access-gtxmr". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:21:45 crc kubenswrapper[5052]: I1203 12:21:45.183569 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e3a73fa0-68ef-4e03-81f9-2a143ed805bf-config-data" (OuterVolumeSpecName: "config-data") pod "e3a73fa0-68ef-4e03-81f9-2a143ed805bf" (UID: "e3a73fa0-68ef-4e03-81f9-2a143ed805bf"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:45 crc kubenswrapper[5052]: I1203 12:21:45.202024 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/memcached-0" podUID="adcea736-eb15-4768-b12c-edb4c810beb3" containerName="memcached" probeResult="failure" output="dial tcp 10.217.1.164:11211: connect: connection refused" Dec 03 12:21:45 crc kubenswrapper[5052]: I1203 12:21:45.217076 5052 scope.go:117] "RemoveContainer" containerID="5c0846c17b1ab7bae05ee7ab757a5ac9bfa9ad58f76d6d21d76a677a190e3f91" Dec 03 12:21:45 crc kubenswrapper[5052]: I1203 12:21:45.264944 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:21:45 crc kubenswrapper[5052]: I1203 12:21:45.267465 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-gtxmr\" (UniqueName: \"kubernetes.io/projected/e3a73fa0-68ef-4e03-81f9-2a143ed805bf-kube-api-access-gtxmr\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:45 crc kubenswrapper[5052]: I1203 12:21:45.267538 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/e3a73fa0-68ef-4e03-81f9-2a143ed805bf-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:45 crc kubenswrapper[5052]: I1203 12:21:45.298635 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstackclient" Dec 03 12:21:45 crc kubenswrapper[5052]: I1203 12:21:45.304132 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:21:45 crc kubenswrapper[5052]: I1203 12:21:45.307594 5052 scope.go:117] "RemoveContainer" containerID="c7645a9210d2fc1e2e3f163388ea3b91c8a787603d06989bb0272c5699b638c7" Dec 03 12:21:45 crc kubenswrapper[5052]: I1203 12:21:45.327945 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance73ca-account-delete-vpqhp"] Dec 03 12:21:45 crc kubenswrapper[5052]: I1203 12:21:45.379891 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/892f0ab2-8602-4485-a11c-5171252e2a7b-combined-ca-bundle\") pod \"892f0ab2-8602-4485-a11c-5171252e2a7b\" (UID: \"892f0ab2-8602-4485-a11c-5171252e2a7b\") " Dec 03 12:21:45 crc kubenswrapper[5052]: I1203 12:21:45.379954 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"openstack-config-secret\" (UniqueName: \"kubernetes.io/secret/892f0ab2-8602-4485-a11c-5171252e2a7b-openstack-config-secret\") pod \"892f0ab2-8602-4485-a11c-5171252e2a7b\" (UID: \"892f0ab2-8602-4485-a11c-5171252e2a7b\") " Dec 03 12:21:45 crc kubenswrapper[5052]: I1203 12:21:45.379993 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zz6rj\" (UniqueName: \"kubernetes.io/projected/b657a259-eb7a-48e8-8f41-2a3c077aff2a-kube-api-access-zz6rj\") pod \"b657a259-eb7a-48e8-8f41-2a3c077aff2a\" (UID: \"b657a259-eb7a-48e8-8f41-2a3c077aff2a\") " Dec 03 12:21:45 crc kubenswrapper[5052]: I1203 12:21:45.380025 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-state-metrics-tls-certs\" (UniqueName: \"kubernetes.io/secret/b657a259-eb7a-48e8-8f41-2a3c077aff2a-kube-state-metrics-tls-certs\") pod \"b657a259-eb7a-48e8-8f41-2a3c077aff2a\" (UID: \"b657a259-eb7a-48e8-8f41-2a3c077aff2a\") " Dec 03 12:21:45 crc kubenswrapper[5052]: I1203 12:21:45.380058 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rtdc4\" (UniqueName: \"kubernetes.io/projected/9e3a0650-c101-4bfa-8137-9909bf8ed4c7-kube-api-access-rtdc4\") pod \"9e3a0650-c101-4bfa-8137-9909bf8ed4c7\" (UID: \"9e3a0650-c101-4bfa-8137-9909bf8ed4c7\") " Dec 03 12:21:45 crc kubenswrapper[5052]: I1203 12:21:45.380083 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"openstack-config\" (UniqueName: \"kubernetes.io/configmap/892f0ab2-8602-4485-a11c-5171252e2a7b-openstack-config\") pod \"892f0ab2-8602-4485-a11c-5171252e2a7b\" (UID: \"892f0ab2-8602-4485-a11c-5171252e2a7b\") " Dec 03 12:21:45 crc kubenswrapper[5052]: I1203 12:21:45.380200 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"vencrypt-tls-certs\" (UniqueName: \"kubernetes.io/secret/9e3a0650-c101-4bfa-8137-9909bf8ed4c7-vencrypt-tls-certs\") pod \"9e3a0650-c101-4bfa-8137-9909bf8ed4c7\" (UID: \"9e3a0650-c101-4bfa-8137-9909bf8ed4c7\") " Dec 03 12:21:45 crc kubenswrapper[5052]: I1203 12:21:45.380280 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4xxw2\" (UniqueName: \"kubernetes.io/projected/892f0ab2-8602-4485-a11c-5171252e2a7b-kube-api-access-4xxw2\") pod \"892f0ab2-8602-4485-a11c-5171252e2a7b\" (UID: \"892f0ab2-8602-4485-a11c-5171252e2a7b\") " Dec 03 12:21:45 crc kubenswrapper[5052]: I1203 12:21:45.380324 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/9e3a0650-c101-4bfa-8137-9909bf8ed4c7-config-data\") pod \"9e3a0650-c101-4bfa-8137-9909bf8ed4c7\" (UID: \"9e3a0650-c101-4bfa-8137-9909bf8ed4c7\") " Dec 03 12:21:45 crc kubenswrapper[5052]: I1203 12:21:45.380349 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b657a259-eb7a-48e8-8f41-2a3c077aff2a-combined-ca-bundle\") pod \"b657a259-eb7a-48e8-8f41-2a3c077aff2a\" (UID: \"b657a259-eb7a-48e8-8f41-2a3c077aff2a\") " Dec 03 12:21:45 crc kubenswrapper[5052]: I1203 12:21:45.380369 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9e3a0650-c101-4bfa-8137-9909bf8ed4c7-combined-ca-bundle\") pod \"9e3a0650-c101-4bfa-8137-9909bf8ed4c7\" (UID: \"9e3a0650-c101-4bfa-8137-9909bf8ed4c7\") " Dec 03 12:21:45 crc kubenswrapper[5052]: I1203 12:21:45.380405 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-state-metrics-tls-config\" (UniqueName: \"kubernetes.io/secret/b657a259-eb7a-48e8-8f41-2a3c077aff2a-kube-state-metrics-tls-config\") pod \"b657a259-eb7a-48e8-8f41-2a3c077aff2a\" (UID: \"b657a259-eb7a-48e8-8f41-2a3c077aff2a\") " Dec 03 12:21:45 crc kubenswrapper[5052]: I1203 12:21:45.380424 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"nova-novncproxy-tls-certs\" (UniqueName: \"kubernetes.io/secret/9e3a0650-c101-4bfa-8137-9909bf8ed4c7-nova-novncproxy-tls-certs\") pod \"9e3a0650-c101-4bfa-8137-9909bf8ed4c7\" (UID: \"9e3a0650-c101-4bfa-8137-9909bf8ed4c7\") " Dec 03 12:21:45 crc kubenswrapper[5052]: I1203 12:21:45.393829 5052 scope.go:117] "RemoveContainer" containerID="f84d0df84d3cb24dd97faa8c986cf4682cc548000edcdfa7e202ea04d4fd81e2" Dec 03 12:21:45 crc kubenswrapper[5052]: I1203 12:21:45.395471 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b657a259-eb7a-48e8-8f41-2a3c077aff2a-kube-api-access-zz6rj" (OuterVolumeSpecName: "kube-api-access-zz6rj") pod "b657a259-eb7a-48e8-8f41-2a3c077aff2a" (UID: "b657a259-eb7a-48e8-8f41-2a3c077aff2a"). InnerVolumeSpecName "kube-api-access-zz6rj". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:21:45 crc kubenswrapper[5052]: I1203 12:21:45.412881 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9e3a0650-c101-4bfa-8137-9909bf8ed4c7-kube-api-access-rtdc4" (OuterVolumeSpecName: "kube-api-access-rtdc4") pod "9e3a0650-c101-4bfa-8137-9909bf8ed4c7" (UID: "9e3a0650-c101-4bfa-8137-9909bf8ed4c7"). InnerVolumeSpecName "kube-api-access-rtdc4". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:21:45 crc kubenswrapper[5052]: I1203 12:21:45.413848 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/892f0ab2-8602-4485-a11c-5171252e2a7b-kube-api-access-4xxw2" (OuterVolumeSpecName: "kube-api-access-4xxw2") pod "892f0ab2-8602-4485-a11c-5171252e2a7b" (UID: "892f0ab2-8602-4485-a11c-5171252e2a7b"). InnerVolumeSpecName "kube-api-access-4xxw2". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:21:45 crc kubenswrapper[5052]: I1203 12:21:45.415616 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-api-8f5b-account-create-update-m9q79"] Dec 03 12:21:45 crc kubenswrapper[5052]: I1203 12:21:45.426444 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/novaapi8f5b-account-delete-86k67"] Dec 03 12:21:45 crc kubenswrapper[5052]: I1203 12:21:45.431307 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/892f0ab2-8602-4485-a11c-5171252e2a7b-openstack-config" (OuterVolumeSpecName: "openstack-config") pod "892f0ab2-8602-4485-a11c-5171252e2a7b" (UID: "892f0ab2-8602-4485-a11c-5171252e2a7b"). InnerVolumeSpecName "openstack-config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:21:45 crc kubenswrapper[5052]: I1203 12:21:45.462559 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b657a259-eb7a-48e8-8f41-2a3c077aff2a-kube-state-metrics-tls-config" (OuterVolumeSpecName: "kube-state-metrics-tls-config") pod "b657a259-eb7a-48e8-8f41-2a3c077aff2a" (UID: "b657a259-eb7a-48e8-8f41-2a3c077aff2a"). InnerVolumeSpecName "kube-state-metrics-tls-config". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:45 crc kubenswrapper[5052]: E1203 12:21:45.474995 5052 cadvisor_stats_provider.go:516] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4a55b492_d57f_40dc_b719_1294fb24ef13.slice/crio-conmon-84022ca2f2789fb89a8aa147aadbca38260a93e0dfa5f25cefce8f679ca30454.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9dec0a31_cba3_4c38_8b5d_4358f16b3704.slice/crio-conmon-7f8633f82aa0da6af5e1f867996abc38506c65e6d6ca569fc68ad7d8ed0d1cf1.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0988bafc_df97_4a53_8308_9e992be58d9a.slice/crio-8a1128866a1e3dbe3fe50f0d8c53319b6d90d894631644af6969f5f9926f2eaa.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4a55b492_d57f_40dc_b719_1294fb24ef13.slice/crio-84022ca2f2789fb89a8aa147aadbca38260a93e0dfa5f25cefce8f679ca30454.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9dec0a31_cba3_4c38_8b5d_4358f16b3704.slice/crio-7f8633f82aa0da6af5e1f867996abc38506c65e6d6ca569fc68ad7d8ed0d1cf1.scope\": RecentStats: unable to find data in memory cache]" Dec 03 12:21:45 crc kubenswrapper[5052]: I1203 12:21:45.477037 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/892f0ab2-8602-4485-a11c-5171252e2a7b-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "892f0ab2-8602-4485-a11c-5171252e2a7b" (UID: "892f0ab2-8602-4485-a11c-5171252e2a7b"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:45 crc kubenswrapper[5052]: I1203 12:21:45.483855 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/892f0ab2-8602-4485-a11c-5171252e2a7b-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:45 crc kubenswrapper[5052]: I1203 12:21:45.483883 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-zz6rj\" (UniqueName: \"kubernetes.io/projected/b657a259-eb7a-48e8-8f41-2a3c077aff2a-kube-api-access-zz6rj\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:45 crc kubenswrapper[5052]: I1203 12:21:45.483894 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-rtdc4\" (UniqueName: \"kubernetes.io/projected/9e3a0650-c101-4bfa-8137-9909bf8ed4c7-kube-api-access-rtdc4\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:45 crc kubenswrapper[5052]: I1203 12:21:45.483907 5052 reconciler_common.go:293] "Volume detached for volume \"openstack-config\" (UniqueName: \"kubernetes.io/configmap/892f0ab2-8602-4485-a11c-5171252e2a7b-openstack-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:45 crc kubenswrapper[5052]: I1203 12:21:45.483915 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-4xxw2\" (UniqueName: \"kubernetes.io/projected/892f0ab2-8602-4485-a11c-5171252e2a7b-kube-api-access-4xxw2\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:45 crc kubenswrapper[5052]: I1203 12:21:45.483925 5052 reconciler_common.go:293] "Volume detached for volume \"kube-state-metrics-tls-config\" (UniqueName: \"kubernetes.io/secret/b657a259-eb7a-48e8-8f41-2a3c077aff2a-kube-state-metrics-tls-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:45 crc kubenswrapper[5052]: E1203 12:21:45.483858 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-memcached-svc: secret "cert-memcached-svc" not found Dec 03 12:21:45 crc kubenswrapper[5052]: E1203 12:21:45.483911 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/combined-ca-bundle: secret "combined-ca-bundle" not found Dec 03 12:21:45 crc kubenswrapper[5052]: E1203 12:21:45.483991 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/adcea736-eb15-4768-b12c-edb4c810beb3-memcached-tls-certs podName:adcea736-eb15-4768-b12c-edb4c810beb3 nodeName:}" failed. No retries permitted until 2025-12-03 12:21:49.483972439 +0000 UTC m=+3700.072214283 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "memcached-tls-certs" (UniqueName: "kubernetes.io/secret/adcea736-eb15-4768-b12c-edb4c810beb3-memcached-tls-certs") pod "memcached-0" (UID: "adcea736-eb15-4768-b12c-edb4c810beb3") : secret "cert-memcached-svc" not found Dec 03 12:21:45 crc kubenswrapper[5052]: E1203 12:21:45.484074 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/adcea736-eb15-4768-b12c-edb4c810beb3-combined-ca-bundle podName:adcea736-eb15-4768-b12c-edb4c810beb3 nodeName:}" failed. No retries permitted until 2025-12-03 12:21:49.484053452 +0000 UTC m=+3700.072295286 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "combined-ca-bundle" (UniqueName: "kubernetes.io/secret/adcea736-eb15-4768-b12c-edb4c810beb3-combined-ca-bundle") pod "memcached-0" (UID: "adcea736-eb15-4768-b12c-edb4c810beb3") : secret "combined-ca-bundle" not found Dec 03 12:21:45 crc kubenswrapper[5052]: I1203 12:21:45.488601 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b657a259-eb7a-48e8-8f41-2a3c077aff2a-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "b657a259-eb7a-48e8-8f41-2a3c077aff2a" (UID: "b657a259-eb7a-48e8-8f41-2a3c077aff2a"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:45 crc kubenswrapper[5052]: I1203 12:21:45.564923 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder2067-account-delete-n2tf7"] Dec 03 12:21:45 crc kubenswrapper[5052]: I1203 12:21:45.572605 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9e3a0650-c101-4bfa-8137-9909bf8ed4c7-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "9e3a0650-c101-4bfa-8137-9909bf8ed4c7" (UID: "9e3a0650-c101-4bfa-8137-9909bf8ed4c7"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:45 crc kubenswrapper[5052]: I1203 12:21:45.585720 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b657a259-eb7a-48e8-8f41-2a3c077aff2a-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:45 crc kubenswrapper[5052]: I1203 12:21:45.585760 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9e3a0650-c101-4bfa-8137-9909bf8ed4c7-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:45 crc kubenswrapper[5052]: I1203 12:21:45.616880 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9e3a0650-c101-4bfa-8137-9909bf8ed4c7-nova-novncproxy-tls-certs" (OuterVolumeSpecName: "nova-novncproxy-tls-certs") pod "9e3a0650-c101-4bfa-8137-9909bf8ed4c7" (UID: "9e3a0650-c101-4bfa-8137-9909bf8ed4c7"). InnerVolumeSpecName "nova-novncproxy-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:45 crc kubenswrapper[5052]: I1203 12:21:45.625761 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9e3a0650-c101-4bfa-8137-9909bf8ed4c7-config-data" (OuterVolumeSpecName: "config-data") pod "9e3a0650-c101-4bfa-8137-9909bf8ed4c7" (UID: "9e3a0650-c101-4bfa-8137-9909bf8ed4c7"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:45 crc kubenswrapper[5052]: I1203 12:21:45.636582 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b657a259-eb7a-48e8-8f41-2a3c077aff2a-kube-state-metrics-tls-certs" (OuterVolumeSpecName: "kube-state-metrics-tls-certs") pod "b657a259-eb7a-48e8-8f41-2a3c077aff2a" (UID: "b657a259-eb7a-48e8-8f41-2a3c077aff2a"). InnerVolumeSpecName "kube-state-metrics-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:45 crc kubenswrapper[5052]: I1203 12:21:45.651377 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9e3a0650-c101-4bfa-8137-9909bf8ed4c7-vencrypt-tls-certs" (OuterVolumeSpecName: "vencrypt-tls-certs") pod "9e3a0650-c101-4bfa-8137-9909bf8ed4c7" (UID: "9e3a0650-c101-4bfa-8137-9909bf8ed4c7"). InnerVolumeSpecName "vencrypt-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:45 crc kubenswrapper[5052]: I1203 12:21:45.679072 5052 generic.go:334] "Generic (PLEG): container finished" podID="4a55b492-d57f-40dc-b719-1294fb24ef13" containerID="84022ca2f2789fb89a8aa147aadbca38260a93e0dfa5f25cefce8f679ca30454" exitCode=0 Dec 03 12:21:45 crc kubenswrapper[5052]: I1203 12:21:45.683754 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/892f0ab2-8602-4485-a11c-5171252e2a7b-openstack-config-secret" (OuterVolumeSpecName: "openstack-config-secret") pod "892f0ab2-8602-4485-a11c-5171252e2a7b" (UID: "892f0ab2-8602-4485-a11c-5171252e2a7b"). InnerVolumeSpecName "openstack-config-secret". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:45 crc kubenswrapper[5052]: I1203 12:21:45.684941 5052 generic.go:334] "Generic (PLEG): container finished" podID="0988bafc-df97-4a53-8308-9e992be58d9a" containerID="8a1128866a1e3dbe3fe50f0d8c53319b6d90d894631644af6969f5f9926f2eaa" exitCode=0 Dec 03 12:21:45 crc kubenswrapper[5052]: I1203 12:21:45.687272 5052 reconciler_common.go:293] "Volume detached for volume \"vencrypt-tls-certs\" (UniqueName: \"kubernetes.io/secret/9e3a0650-c101-4bfa-8137-9909bf8ed4c7-vencrypt-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:45 crc kubenswrapper[5052]: I1203 12:21:45.687299 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/9e3a0650-c101-4bfa-8137-9909bf8ed4c7-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:45 crc kubenswrapper[5052]: I1203 12:21:45.687309 5052 reconciler_common.go:293] "Volume detached for volume \"nova-novncproxy-tls-certs\" (UniqueName: \"kubernetes.io/secret/9e3a0650-c101-4bfa-8137-9909bf8ed4c7-nova-novncproxy-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:45 crc kubenswrapper[5052]: I1203 12:21:45.687319 5052 reconciler_common.go:293] "Volume detached for volume \"openstack-config-secret\" (UniqueName: \"kubernetes.io/secret/892f0ab2-8602-4485-a11c-5171252e2a7b-openstack-config-secret\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:45 crc kubenswrapper[5052]: I1203 12:21:45.687330 5052 reconciler_common.go:293] "Volume detached for volume \"kube-state-metrics-tls-certs\" (UniqueName: \"kubernetes.io/secret/b657a259-eb7a-48e8-8f41-2a3c077aff2a-kube-state-metrics-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:45 crc kubenswrapper[5052]: I1203 12:21:45.687978 5052 generic.go:334] "Generic (PLEG): container finished" podID="9dec0a31-cba3-4c38-8b5d-4358f16b3704" containerID="7f8633f82aa0da6af5e1f867996abc38506c65e6d6ca569fc68ad7d8ed0d1cf1" exitCode=0 Dec 03 12:21:45 crc kubenswrapper[5052]: I1203 12:21:45.688004 5052 generic.go:334] "Generic (PLEG): container finished" podID="9dec0a31-cba3-4c38-8b5d-4358f16b3704" containerID="72c6e09b72f57c6f1f19ff9c617f85acc4ceb1d16fb1a62383bab5dea1e10644" exitCode=143 Dec 03 12:21:45 crc kubenswrapper[5052]: I1203 12:21:45.689778 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/placement9152-account-delete-x7q46" podUID="4f7ab0ec-52a7-4be2-9886-04bf98989b63" containerName="mariadb-account-delete" containerID="cri-o://1eb47db985e554c7f897dc0656ec6f7756b165606954fa8a078d98e25f051bc1" gracePeriod=30 Dec 03 12:21:45 crc kubenswrapper[5052]: I1203 12:21:45.694307 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="19a09467-4c91-40f5-b83a-d43a8eb5f2dc" path="/var/lib/kubelet/pods/19a09467-4c91-40f5-b83a-d43a8eb5f2dc/volumes" Dec 03 12:21:45 crc kubenswrapper[5052]: I1203 12:21:45.694865 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstackclient" Dec 03 12:21:45 crc kubenswrapper[5052]: I1203 12:21:45.701213 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="23c58e9d-3154-4887-8a51-1601ad306abf" path="/var/lib/kubelet/pods/23c58e9d-3154-4887-8a51-1601ad306abf/volumes" Dec 03 12:21:45 crc kubenswrapper[5052]: I1203 12:21:45.701972 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="357bc965-8633-4d67-992e-baa770a0defd" path="/var/lib/kubelet/pods/357bc965-8633-4d67-992e-baa770a0defd/volumes" Dec 03 12:21:45 crc kubenswrapper[5052]: I1203 12:21:45.702925 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="63c18817-7663-466c-845d-bc998c5b7093" path="/var/lib/kubelet/pods/63c18817-7663-466c-845d-bc998c5b7093/volumes" Dec 03 12:21:45 crc kubenswrapper[5052]: I1203 12:21:45.703874 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:21:45 crc kubenswrapper[5052]: I1203 12:21:45.704319 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="70cedf3b-c9b9-4e5c-81a2-ba134b150997" path="/var/lib/kubelet/pods/70cedf3b-c9b9-4e5c-81a2-ba134b150997/volumes" Dec 03 12:21:45 crc kubenswrapper[5052]: I1203 12:21:45.710323 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/placement9152-account-delete-x7q46" podStartSLOduration=5.710303069 podStartE2EDuration="5.710303069s" podCreationTimestamp="2025-12-03 12:21:40 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:21:45.710002331 +0000 UTC m=+3696.298244115" watchObservedRunningTime="2025-12-03 12:21:45.710303069 +0000 UTC m=+3696.298544833" Dec 03 12:21:45 crc kubenswrapper[5052]: I1203 12:21:45.737694 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="892f0ab2-8602-4485-a11c-5171252e2a7b" path="/var/lib/kubelet/pods/892f0ab2-8602-4485-a11c-5171252e2a7b/volumes" Dec 03 12:21:45 crc kubenswrapper[5052]: I1203 12:21:45.739442 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9ea78435-8fd9-4321-9e48-3aaae833dcf2" path="/var/lib/kubelet/pods/9ea78435-8fd9-4321-9e48-3aaae833dcf2/volumes" Dec 03 12:21:45 crc kubenswrapper[5052]: I1203 12:21:45.741228 5052 generic.go:334] "Generic (PLEG): container finished" podID="fb73cc9a-89cc-4cb8-87ce-2731e5dab0f7" containerID="370f335a2865f2cdbcf3d953581d2819c13f1b7d88a0ac612179ca394589c0e9" exitCode=0 Dec 03 12:21:45 crc kubenswrapper[5052]: I1203 12:21:45.741252 5052 generic.go:334] "Generic (PLEG): container finished" podID="fb73cc9a-89cc-4cb8-87ce-2731e5dab0f7" containerID="a1d1e7cece7f6bab44f530f29cd0b9119aee96b8c725b66c1bb8c782f4ccd637" exitCode=0 Dec 03 12:21:45 crc kubenswrapper[5052]: I1203 12:21:45.757366 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b852a7d7-8510-440d-8e87-0755fac2a324" path="/var/lib/kubelet/pods/b852a7d7-8510-440d-8e87-0755fac2a324/volumes" Dec 03 12:21:45 crc kubenswrapper[5052]: I1203 12:21:45.758635 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fc3fb6e4-34b1-403a-8761-f03573c6d993" path="/var/lib/kubelet/pods/fc3fb6e4-34b1-403a-8761-f03573c6d993/volumes" Dec 03 12:21:45 crc kubenswrapper[5052]: I1203 12:21:45.783156 5052 generic.go:334] "Generic (PLEG): container finished" podID="7e109538-a2d2-43bb-8cef-bf99d7d0f1e9" containerID="ef9fabe8451cd9b904301777198ce63bf6cefd87c577b4cad8746d28b098af4a" exitCode=0 Dec 03 12:21:45 crc kubenswrapper[5052]: I1203 12:21:45.810923 5052 generic.go:334] "Generic (PLEG): container finished" podID="adcea736-eb15-4768-b12c-edb4c810beb3" containerID="7eee2cae018afa667fc80fb6f8e55b05f28d5f2bfff1ccd7ad972013dda2c344" exitCode=0 Dec 03 12:21:45 crc kubenswrapper[5052]: I1203 12:21:45.817603 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican0e0e-account-delete-fvkr2"] Dec 03 12:21:45 crc kubenswrapper[5052]: I1203 12:21:45.817665 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"4a55b492-d57f-40dc-b719-1294fb24ef13","Type":"ContainerDied","Data":"84022ca2f2789fb89a8aa147aadbca38260a93e0dfa5f25cefce8f679ca30454"} Dec 03 12:21:45 crc kubenswrapper[5052]: I1203 12:21:45.817704 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-galera-0" event={"ID":"0988bafc-df97-4a53-8308-9e992be58d9a","Type":"ContainerDied","Data":"8a1128866a1e3dbe3fe50f0d8c53319b6d90d894631644af6969f5f9926f2eaa"} Dec 03 12:21:45 crc kubenswrapper[5052]: I1203 12:21:45.817723 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"9dec0a31-cba3-4c38-8b5d-4358f16b3704","Type":"ContainerDied","Data":"7f8633f82aa0da6af5e1f867996abc38506c65e6d6ca569fc68ad7d8ed0d1cf1"} Dec 03 12:21:45 crc kubenswrapper[5052]: I1203 12:21:45.817738 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"9dec0a31-cba3-4c38-8b5d-4358f16b3704","Type":"ContainerDied","Data":"72c6e09b72f57c6f1f19ff9c617f85acc4ceb1d16fb1a62383bab5dea1e10644"} Dec 03 12:21:45 crc kubenswrapper[5052]: I1203 12:21:45.817751 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement9152-account-delete-x7q46" event={"ID":"4f7ab0ec-52a7-4be2-9886-04bf98989b63","Type":"ContainerStarted","Data":"1eb47db985e554c7f897dc0656ec6f7756b165606954fa8a078d98e25f051bc1"} Dec 03 12:21:45 crc kubenswrapper[5052]: I1203 12:21:45.817779 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/kube-state-metrics-0" event={"ID":"b657a259-eb7a-48e8-8f41-2a3c077aff2a","Type":"ContainerDied","Data":"103ec0f2998cccccbbff8045e265d938a66f7f6517e99492386e58e8b92e443f"} Dec 03 12:21:45 crc kubenswrapper[5052]: I1203 12:21:45.817797 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-proxy-6b9f9968d-5zjp2" event={"ID":"fb73cc9a-89cc-4cb8-87ce-2731e5dab0f7","Type":"ContainerDied","Data":"370f335a2865f2cdbcf3d953581d2819c13f1b7d88a0ac612179ca394589c0e9"} Dec 03 12:21:45 crc kubenswrapper[5052]: I1203 12:21:45.817810 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-proxy-6b9f9968d-5zjp2" event={"ID":"fb73cc9a-89cc-4cb8-87ce-2731e5dab0f7","Type":"ContainerDied","Data":"a1d1e7cece7f6bab44f530f29cd0b9119aee96b8c725b66c1bb8c782f4ccd637"} Dec 03 12:21:45 crc kubenswrapper[5052]: I1203 12:21:45.817821 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-proxy-d4747d79f-wtbhv" event={"ID":"7e109538-a2d2-43bb-8cef-bf99d7d0f1e9","Type":"ContainerDied","Data":"ef9fabe8451cd9b904301777198ce63bf6cefd87c577b4cad8746d28b098af4a"} Dec 03 12:21:45 crc kubenswrapper[5052]: I1203 12:21:45.817833 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/memcached-0" event={"ID":"adcea736-eb15-4768-b12c-edb4c810beb3","Type":"ContainerDied","Data":"7eee2cae018afa667fc80fb6f8e55b05f28d5f2bfff1ccd7ad972013dda2c344"} Dec 03 12:21:45 crc kubenswrapper[5052]: I1203 12:21:45.817865 5052 scope.go:117] "RemoveContainer" containerID="5bdabbc1acf777d8b66b937a8a2bfb67eb3f1e9fde05ff4a60b0028fe9914ce7" Dec 03 12:21:45 crc kubenswrapper[5052]: E1203 12:21:45.833356 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="938a22611459fa789b24b03d2a21853b69071aad6d79f25384679595984908d8" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:21:45 crc kubenswrapper[5052]: I1203 12:21:45.850218 5052 generic.go:334] "Generic (PLEG): container finished" podID="47aa63e7-b7b2-4c1d-a73d-348b50cc5d2b" containerID="37fd4ad16311c4c25724489aee98ab54bb95696bace9c0196b6418d1b39c27b1" exitCode=0 Dec 03 12:21:45 crc kubenswrapper[5052]: I1203 12:21:45.850315 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-cell1-galera-0" event={"ID":"47aa63e7-b7b2-4c1d-a73d-348b50cc5d2b","Type":"ContainerDied","Data":"37fd4ad16311c4c25724489aee98ab54bb95696bace9c0196b6418d1b39c27b1"} Dec 03 12:21:45 crc kubenswrapper[5052]: I1203 12:21:45.854566 5052 generic.go:334] "Generic (PLEG): container finished" podID="9e3a0650-c101-4bfa-8137-9909bf8ed4c7" containerID="c5e90a99765d5931a5d6cd2673f9bb42cd7a8b05426a276c6f134ae97869e630" exitCode=0 Dec 03 12:21:45 crc kubenswrapper[5052]: I1203 12:21:45.854721 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:21:45 crc kubenswrapper[5052]: I1203 12:21:45.854890 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" event={"ID":"9e3a0650-c101-4bfa-8137-9909bf8ed4c7","Type":"ContainerDied","Data":"c5e90a99765d5931a5d6cd2673f9bb42cd7a8b05426a276c6f134ae97869e630"} Dec 03 12:21:45 crc kubenswrapper[5052]: I1203 12:21:45.855040 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" event={"ID":"9e3a0650-c101-4bfa-8137-9909bf8ed4c7","Type":"ContainerDied","Data":"95666e9bd25d9c51049cde002d933bd54c91313b21de8911bf336e7370a0f3b3"} Dec 03 12:21:45 crc kubenswrapper[5052]: E1203 12:21:45.909181 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="938a22611459fa789b24b03d2a21853b69071aad6d79f25384679595984908d8" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:21:45 crc kubenswrapper[5052]: E1203 12:21:45.918901 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="938a22611459fa789b24b03d2a21853b69071aad6d79f25384679595984908d8" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:21:45 crc kubenswrapper[5052]: E1203 12:21:45.919004 5052 prober.go:104] "Probe errored" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" probeType="Readiness" pod="openstack-kuttl-tests/nova-cell1-conductor-0" podUID="a7907bc2-8e73-4272-b779-fd71d4acb1f4" containerName="nova-cell1-conductor-conductor" Dec 03 12:21:45 crc kubenswrapper[5052]: I1203 12:21:45.926120 5052 generic.go:334] "Generic (PLEG): container finished" podID="2d94bf00-7a64-4e90-a48c-b206ec54da6d" containerID="cf05124bcf6f5c1b9dc38517547ce3e4342c12cea5c84ee53adeabf6d90e42cf" exitCode=0 Dec 03 12:21:45 crc kubenswrapper[5052]: I1203 12:21:45.926221 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"2d94bf00-7a64-4e90-a48c-b206ec54da6d","Type":"ContainerDied","Data":"cf05124bcf6f5c1b9dc38517547ce3e4342c12cea5c84ee53adeabf6d90e42cf"} Dec 03 12:21:45 crc kubenswrapper[5052]: I1203 12:21:45.934480 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/novacell00e20-account-delete-ktxhg"] Dec 03 12:21:45 crc kubenswrapper[5052]: I1203 12:21:45.937692 5052 kubelet_pods.go:1007] "Unable to retrieve pull secret, the image pull may not succeed." pod="openstack-kuttl-tests/neutron-734a-account-create-update-qcgft" secret="" err="secret \"galera-openstack-dockercfg-dlq6n\" not found" Dec 03 12:21:45 crc kubenswrapper[5052]: I1203 12:21:45.938315 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-28d1-account-create-update-f27hx" Dec 03 12:21:45 crc kubenswrapper[5052]: I1203 12:21:45.938825 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone28d1-account-delete-c9vmm" Dec 03 12:21:45 crc kubenswrapper[5052]: I1203 12:21:45.940046 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:21:45 crc kubenswrapper[5052]: I1203 12:21:45.941063 5052 kubelet_pods.go:1007] "Unable to retrieve pull secret, the image pull may not succeed." pod="openstack-kuttl-tests/glance73ca-account-delete-vpqhp" secret="" err="secret \"galera-openstack-dockercfg-dlq6n\" not found" Dec 03 12:21:45 crc kubenswrapper[5052]: I1203 12:21:45.941844 5052 scope.go:117] "RemoveContainer" containerID="9569b49a51e3eb2f02cd20195f982384cb40ad5f6b6fd5f65cf8a8a8fff4d592" Dec 03 12:21:45 crc kubenswrapper[5052]: I1203 12:21:45.941976 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/novacell13fba-account-delete-hvzdt" Dec 03 12:21:45 crc kubenswrapper[5052]: I1203 12:21:45.961984 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-proxy-6b9f9968d-5zjp2" Dec 03 12:21:45 crc kubenswrapper[5052]: I1203 12:21:45.967642 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/kube-state-metrics-0"] Dec 03 12:21:45 crc kubenswrapper[5052]: I1203 12:21:45.978686 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/kube-state-metrics-0"] Dec 03 12:21:45 crc kubenswrapper[5052]: I1203 12:21:45.987151 5052 scope.go:117] "RemoveContainer" containerID="c5e90a99765d5931a5d6cd2673f9bb42cd7a8b05426a276c6f134ae97869e630" Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.022118 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/fb73cc9a-89cc-4cb8-87ce-2731e5dab0f7-log-httpd\") pod \"fb73cc9a-89cc-4cb8-87ce-2731e5dab0f7\" (UID: \"fb73cc9a-89cc-4cb8-87ce-2731e5dab0f7\") " Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.022213 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/fb73cc9a-89cc-4cb8-87ce-2731e5dab0f7-run-httpd\") pod \"fb73cc9a-89cc-4cb8-87ce-2731e5dab0f7\" (UID: \"fb73cc9a-89cc-4cb8-87ce-2731e5dab0f7\") " Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.022253 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/fb73cc9a-89cc-4cb8-87ce-2731e5dab0f7-public-tls-certs\") pod \"fb73cc9a-89cc-4cb8-87ce-2731e5dab0f7\" (UID: \"fb73cc9a-89cc-4cb8-87ce-2731e5dab0f7\") " Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.022330 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/fb73cc9a-89cc-4cb8-87ce-2731e5dab0f7-combined-ca-bundle\") pod \"fb73cc9a-89cc-4cb8-87ce-2731e5dab0f7\" (UID: \"fb73cc9a-89cc-4cb8-87ce-2731e5dab0f7\") " Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.022392 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/fb73cc9a-89cc-4cb8-87ce-2731e5dab0f7-internal-tls-certs\") pod \"fb73cc9a-89cc-4cb8-87ce-2731e5dab0f7\" (UID: \"fb73cc9a-89cc-4cb8-87ce-2731e5dab0f7\") " Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.022414 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gppn6\" (UniqueName: \"kubernetes.io/projected/fb73cc9a-89cc-4cb8-87ce-2731e5dab0f7-kube-api-access-gppn6\") pod \"fb73cc9a-89cc-4cb8-87ce-2731e5dab0f7\" (UID: \"fb73cc9a-89cc-4cb8-87ce-2731e5dab0f7\") " Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.022523 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/fb73cc9a-89cc-4cb8-87ce-2731e5dab0f7-etc-swift\") pod \"fb73cc9a-89cc-4cb8-87ce-2731e5dab0f7\" (UID: \"fb73cc9a-89cc-4cb8-87ce-2731e5dab0f7\") " Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.022555 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/fb73cc9a-89cc-4cb8-87ce-2731e5dab0f7-config-data\") pod \"fb73cc9a-89cc-4cb8-87ce-2731e5dab0f7\" (UID: \"fb73cc9a-89cc-4cb8-87ce-2731e5dab0f7\") " Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.022822 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-75s94\" (UniqueName: \"kubernetes.io/projected/bfc242a2-aca4-458b-812e-018787bdbcab-kube-api-access-75s94\") pod \"keystone-28d1-account-create-update-f27hx\" (UID: \"bfc242a2-aca4-458b-812e-018787bdbcab\") " pod="openstack-kuttl-tests/keystone-28d1-account-create-update-f27hx" Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.022856 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-5h59r\" (UniqueName: \"kubernetes.io/projected/9165ef68-b76d-472e-99af-06cc8c8a2d3e-kube-api-access-5h59r\") pod \"keystone28d1-account-delete-c9vmm\" (UID: \"9165ef68-b76d-472e-99af-06cc8c8a2d3e\") " pod="openstack-kuttl-tests/keystone28d1-account-delete-c9vmm" Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.022990 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/9165ef68-b76d-472e-99af-06cc8c8a2d3e-operator-scripts\") pod \"keystone28d1-account-delete-c9vmm\" (UID: \"9165ef68-b76d-472e-99af-06cc8c8a2d3e\") " pod="openstack-kuttl-tests/keystone28d1-account-delete-c9vmm" Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.023022 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/bfc242a2-aca4-458b-812e-018787bdbcab-operator-scripts\") pod \"keystone-28d1-account-create-update-f27hx\" (UID: \"bfc242a2-aca4-458b-812e-018787bdbcab\") " pod="openstack-kuttl-tests/keystone-28d1-account-create-update-f27hx" Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.027152 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb73cc9a-89cc-4cb8-87ce-2731e5dab0f7-run-httpd" (OuterVolumeSpecName: "run-httpd") pod "fb73cc9a-89cc-4cb8-87ce-2731e5dab0f7" (UID: "fb73cc9a-89cc-4cb8-87ce-2731e5dab0f7"). InnerVolumeSpecName "run-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.027637 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb73cc9a-89cc-4cb8-87ce-2731e5dab0f7-log-httpd" (OuterVolumeSpecName: "log-httpd") pod "fb73cc9a-89cc-4cb8-87ce-2731e5dab0f7" (UID: "fb73cc9a-89cc-4cb8-87ce-2731e5dab0f7"). InnerVolumeSpecName "log-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:21:46 crc kubenswrapper[5052]: E1203 12:21:46.037523 5052 projected.go:194] Error preparing data for projected volume kube-api-access-75s94 for pod openstack-kuttl-tests/keystone-28d1-account-create-update-f27hx: failed to fetch token: serviceaccounts "galera-openstack" not found Dec 03 12:21:46 crc kubenswrapper[5052]: E1203 12:21:46.037589 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bfc242a2-aca4-458b-812e-018787bdbcab-kube-api-access-75s94 podName:bfc242a2-aca4-458b-812e-018787bdbcab nodeName:}" failed. No retries permitted until 2025-12-03 12:21:48.03757099 +0000 UTC m=+3698.625812764 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-75s94" (UniqueName: "kubernetes.io/projected/bfc242a2-aca4-458b-812e-018787bdbcab-kube-api-access-75s94") pod "keystone-28d1-account-create-update-f27hx" (UID: "bfc242a2-aca4-458b-812e-018787bdbcab") : failed to fetch token: serviceaccounts "galera-openstack" not found Dec 03 12:21:46 crc kubenswrapper[5052]: E1203 12:21:46.039993 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:21:46 crc kubenswrapper[5052]: E1203 12:21:46.040022 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:21:46 crc kubenswrapper[5052]: E1203 12:21:46.040034 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9165ef68-b76d-472e-99af-06cc8c8a2d3e-operator-scripts podName:9165ef68-b76d-472e-99af-06cc8c8a2d3e nodeName:}" failed. No retries permitted until 2025-12-03 12:21:48.04002276 +0000 UTC m=+3698.628264524 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/9165ef68-b76d-472e-99af-06cc8c8a2d3e-operator-scripts") pod "keystone28d1-account-delete-c9vmm" (UID: "9165ef68-b76d-472e-99af-06cc8c8a2d3e") : configmap "openstack-scripts" not found Dec 03 12:21:46 crc kubenswrapper[5052]: E1203 12:21:46.040063 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/bfc242a2-aca4-458b-812e-018787bdbcab-operator-scripts podName:bfc242a2-aca4-458b-812e-018787bdbcab nodeName:}" failed. No retries permitted until 2025-12-03 12:21:48.040050321 +0000 UTC m=+3698.628292085 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/bfc242a2-aca4-458b-812e-018787bdbcab-operator-scripts") pod "keystone-28d1-account-create-update-f27hx" (UID: "bfc242a2-aca4-458b-812e-018787bdbcab") : configmap "openstack-scripts" not found Dec 03 12:21:46 crc kubenswrapper[5052]: E1203 12:21:46.046581 5052 projected.go:194] Error preparing data for projected volume kube-api-access-5h59r for pod openstack-kuttl-tests/keystone28d1-account-delete-c9vmm: failed to fetch token: serviceaccounts "galera-openstack" not found Dec 03 12:21:46 crc kubenswrapper[5052]: E1203 12:21:46.046657 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9165ef68-b76d-472e-99af-06cc8c8a2d3e-kube-api-access-5h59r podName:9165ef68-b76d-472e-99af-06cc8c8a2d3e nodeName:}" failed. No retries permitted until 2025-12-03 12:21:48.046637249 +0000 UTC m=+3698.634879013 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-5h59r" (UniqueName: "kubernetes.io/projected/9165ef68-b76d-472e-99af-06cc8c8a2d3e-kube-api-access-5h59r") pod "keystone28d1-account-delete-c9vmm" (UID: "9165ef68-b76d-472e-99af-06cc8c8a2d3e") : failed to fetch token: serviceaccounts "galera-openstack" not found Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.046838 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fb73cc9a-89cc-4cb8-87ce-2731e5dab0f7-etc-swift" (OuterVolumeSpecName: "etc-swift") pod "fb73cc9a-89cc-4cb8-87ce-2731e5dab0f7" (UID: "fb73cc9a-89cc-4cb8-87ce-2731e5dab0f7"). InnerVolumeSpecName "etc-swift". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.065951 5052 scope.go:117] "RemoveContainer" containerID="c5e90a99765d5931a5d6cd2673f9bb42cd7a8b05426a276c6f134ae97869e630" Dec 03 12:21:46 crc kubenswrapper[5052]: E1203 12:21:46.080689 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c5e90a99765d5931a5d6cd2673f9bb42cd7a8b05426a276c6f134ae97869e630\": container with ID starting with c5e90a99765d5931a5d6cd2673f9bb42cd7a8b05426a276c6f134ae97869e630 not found: ID does not exist" containerID="c5e90a99765d5931a5d6cd2673f9bb42cd7a8b05426a276c6f134ae97869e630" Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.080741 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c5e90a99765d5931a5d6cd2673f9bb42cd7a8b05426a276c6f134ae97869e630"} err="failed to get container status \"c5e90a99765d5931a5d6cd2673f9bb42cd7a8b05426a276c6f134ae97869e630\": rpc error: code = NotFound desc = could not find container \"c5e90a99765d5931a5d6cd2673f9bb42cd7a8b05426a276c6f134ae97869e630\": container with ID starting with c5e90a99765d5931a5d6cd2673f9bb42cd7a8b05426a276c6f134ae97869e630 not found: ID does not exist" Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.096409 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fb73cc9a-89cc-4cb8-87ce-2731e5dab0f7-kube-api-access-gppn6" (OuterVolumeSpecName: "kube-api-access-gppn6") pod "fb73cc9a-89cc-4cb8-87ce-2731e5dab0f7" (UID: "fb73cc9a-89cc-4cb8-87ce-2731e5dab0f7"). InnerVolumeSpecName "kube-api-access-gppn6". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.120827 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.125728 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-gppn6\" (UniqueName: \"kubernetes.io/projected/fb73cc9a-89cc-4cb8-87ce-2731e5dab0f7-kube-api-access-gppn6\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.125929 5052 reconciler_common.go:293] "Volume detached for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/fb73cc9a-89cc-4cb8-87ce-2731e5dab0f7-etc-swift\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.125987 5052 reconciler_common.go:293] "Volume detached for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/fb73cc9a-89cc-4cb8-87ce-2731e5dab0f7-log-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.126057 5052 reconciler_common.go:293] "Volume detached for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/fb73cc9a-89cc-4cb8-87ce-2731e5dab0f7-run-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:46 crc kubenswrapper[5052]: E1203 12:21:46.126461 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:21:46 crc kubenswrapper[5052]: E1203 12:21:46.126557 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/817345c8-3a42-4dea-8158-75973d9c41d9-operator-scripts podName:817345c8-3a42-4dea-8158-75973d9c41d9 nodeName:}" failed. No retries permitted until 2025-12-03 12:21:48.126493578 +0000 UTC m=+3698.714735342 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/817345c8-3a42-4dea-8158-75973d9c41d9-operator-scripts") pod "neutron-734a-account-create-update-qcgft" (UID: "817345c8-3a42-4dea-8158-75973d9c41d9") : configmap "openstack-scripts" not found Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.131717 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.137855 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.165067 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fb73cc9a-89cc-4cb8-87ce-2731e5dab0f7-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "fb73cc9a-89cc-4cb8-87ce-2731e5dab0f7" (UID: "fb73cc9a-89cc-4cb8-87ce-2731e5dab0f7"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.168141 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/keystone28d1-account-delete-c9vmm"] Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.172781 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fb73cc9a-89cc-4cb8-87ce-2731e5dab0f7-config-data" (OuterVolumeSpecName: "config-data") pod "fb73cc9a-89cc-4cb8-87ce-2731e5dab0f7" (UID: "fb73cc9a-89cc-4cb8-87ce-2731e5dab0f7"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.180167 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/keystone28d1-account-delete-c9vmm"] Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.180236 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fb73cc9a-89cc-4cb8-87ce-2731e5dab0f7-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "fb73cc9a-89cc-4cb8-87ce-2731e5dab0f7" (UID: "fb73cc9a-89cc-4cb8-87ce-2731e5dab0f7"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.181262 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.186818 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-proxy-d4747d79f-wtbhv" Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.187410 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fb73cc9a-89cc-4cb8-87ce-2731e5dab0f7-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "fb73cc9a-89cc-4cb8-87ce-2731e5dab0f7" (UID: "fb73cc9a-89cc-4cb8-87ce-2731e5dab0f7"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.191842 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.202264 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.211303 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.219208 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.227677 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"mysql-db\" (UniqueName: \"kubernetes.io/local-volume/local-storage03-crc\") pod \"0988bafc-df97-4a53-8308-9e992be58d9a\" (UID: \"0988bafc-df97-4a53-8308-9e992be58d9a\") " Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.227744 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/4a55b492-d57f-40dc-b719-1294fb24ef13-config-data-custom\") pod \"4a55b492-d57f-40dc-b719-1294fb24ef13\" (UID: \"4a55b492-d57f-40dc-b719-1294fb24ef13\") " Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.227791 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z4ktp\" (UniqueName: \"kubernetes.io/projected/4a55b492-d57f-40dc-b719-1294fb24ef13-kube-api-access-z4ktp\") pod \"4a55b492-d57f-40dc-b719-1294fb24ef13\" (UID: \"4a55b492-d57f-40dc-b719-1294fb24ef13\") " Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.227821 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/9dec0a31-cba3-4c38-8b5d-4358f16b3704-internal-tls-certs\") pod \"9dec0a31-cba3-4c38-8b5d-4358f16b3704\" (UID: \"9dec0a31-cba3-4c38-8b5d-4358f16b3704\") " Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.227857 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-njjpc\" (UniqueName: \"kubernetes.io/projected/47aa63e7-b7b2-4c1d-a73d-348b50cc5d2b-kube-api-access-njjpc\") pod \"47aa63e7-b7b2-4c1d-a73d-348b50cc5d2b\" (UID: \"47aa63e7-b7b2-4c1d-a73d-348b50cc5d2b\") " Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.227892 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/0988bafc-df97-4a53-8308-9e992be58d9a-config-data-generated\") pod \"0988bafc-df97-4a53-8308-9e992be58d9a\" (UID: \"0988bafc-df97-4a53-8308-9e992be58d9a\") " Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.227923 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/0988bafc-df97-4a53-8308-9e992be58d9a-galera-tls-certs\") pod \"0988bafc-df97-4a53-8308-9e992be58d9a\" (UID: \"0988bafc-df97-4a53-8308-9e992be58d9a\") " Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.227962 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/7e109538-a2d2-43bb-8cef-bf99d7d0f1e9-public-tls-certs\") pod \"7e109538-a2d2-43bb-8cef-bf99d7d0f1e9\" (UID: \"7e109538-a2d2-43bb-8cef-bf99d7d0f1e9\") " Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.228008 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0988bafc-df97-4a53-8308-9e992be58d9a-combined-ca-bundle\") pod \"0988bafc-df97-4a53-8308-9e992be58d9a\" (UID: \"0988bafc-df97-4a53-8308-9e992be58d9a\") " Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.228029 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/4a55b492-d57f-40dc-b719-1294fb24ef13-etc-machine-id\") pod \"4a55b492-d57f-40dc-b719-1294fb24ef13\" (UID: \"4a55b492-d57f-40dc-b719-1294fb24ef13\") " Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.228064 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/0988bafc-df97-4a53-8308-9e992be58d9a-kolla-config\") pod \"0988bafc-df97-4a53-8308-9e992be58d9a\" (UID: \"0988bafc-df97-4a53-8308-9e992be58d9a\") " Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.228095 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/7e109538-a2d2-43bb-8cef-bf99d7d0f1e9-combined-ca-bundle\") pod \"7e109538-a2d2-43bb-8cef-bf99d7d0f1e9\" (UID: \"7e109538-a2d2-43bb-8cef-bf99d7d0f1e9\") " Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.228117 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9dec0a31-cba3-4c38-8b5d-4358f16b3704-combined-ca-bundle\") pod \"9dec0a31-cba3-4c38-8b5d-4358f16b3704\" (UID: \"9dec0a31-cba3-4c38-8b5d-4358f16b3704\") " Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.228145 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"mysql-db\" (UniqueName: \"kubernetes.io/local-volume/local-storage06-crc\") pod \"47aa63e7-b7b2-4c1d-a73d-348b50cc5d2b\" (UID: \"47aa63e7-b7b2-4c1d-a73d-348b50cc5d2b\") " Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.228165 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/47aa63e7-b7b2-4c1d-a73d-348b50cc5d2b-galera-tls-certs\") pod \"47aa63e7-b7b2-4c1d-a73d-348b50cc5d2b\" (UID: \"47aa63e7-b7b2-4c1d-a73d-348b50cc5d2b\") " Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.228187 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/7e109538-a2d2-43bb-8cef-bf99d7d0f1e9-etc-swift\") pod \"7e109538-a2d2-43bb-8cef-bf99d7d0f1e9\" (UID: \"7e109538-a2d2-43bb-8cef-bf99d7d0f1e9\") " Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.228219 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8t4hw\" (UniqueName: \"kubernetes.io/projected/9dec0a31-cba3-4c38-8b5d-4358f16b3704-kube-api-access-8t4hw\") pod \"9dec0a31-cba3-4c38-8b5d-4358f16b3704\" (UID: \"9dec0a31-cba3-4c38-8b5d-4358f16b3704\") " Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.228242 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/7e109538-a2d2-43bb-8cef-bf99d7d0f1e9-internal-tls-certs\") pod \"7e109538-a2d2-43bb-8cef-bf99d7d0f1e9\" (UID: \"7e109538-a2d2-43bb-8cef-bf99d7d0f1e9\") " Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.228309 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/9dec0a31-cba3-4c38-8b5d-4358f16b3704-config-data\") pod \"9dec0a31-cba3-4c38-8b5d-4358f16b3704\" (UID: \"9dec0a31-cba3-4c38-8b5d-4358f16b3704\") " Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.228328 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/7e109538-a2d2-43bb-8cef-bf99d7d0f1e9-log-httpd\") pod \"7e109538-a2d2-43bb-8cef-bf99d7d0f1e9\" (UID: \"7e109538-a2d2-43bb-8cef-bf99d7d0f1e9\") " Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.228390 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/9dec0a31-cba3-4c38-8b5d-4358f16b3704-public-tls-certs\") pod \"9dec0a31-cba3-4c38-8b5d-4358f16b3704\" (UID: \"9dec0a31-cba3-4c38-8b5d-4358f16b3704\") " Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.228419 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/7e109538-a2d2-43bb-8cef-bf99d7d0f1e9-config-data\") pod \"7e109538-a2d2-43bb-8cef-bf99d7d0f1e9\" (UID: \"7e109538-a2d2-43bb-8cef-bf99d7d0f1e9\") " Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.228473 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/47aa63e7-b7b2-4c1d-a73d-348b50cc5d2b-config-data-generated\") pod \"47aa63e7-b7b2-4c1d-a73d-348b50cc5d2b\" (UID: \"47aa63e7-b7b2-4c1d-a73d-348b50cc5d2b\") " Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.228547 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/47aa63e7-b7b2-4c1d-a73d-348b50cc5d2b-combined-ca-bundle\") pod \"47aa63e7-b7b2-4c1d-a73d-348b50cc5d2b\" (UID: \"47aa63e7-b7b2-4c1d-a73d-348b50cc5d2b\") " Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.228579 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/47aa63e7-b7b2-4c1d-a73d-348b50cc5d2b-operator-scripts\") pod \"47aa63e7-b7b2-4c1d-a73d-348b50cc5d2b\" (UID: \"47aa63e7-b7b2-4c1d-a73d-348b50cc5d2b\") " Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.228646 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/0988bafc-df97-4a53-8308-9e992be58d9a-config-data-default\") pod \"0988bafc-df97-4a53-8308-9e992be58d9a\" (UID: \"0988bafc-df97-4a53-8308-9e992be58d9a\") " Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.228670 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6zp88\" (UniqueName: \"kubernetes.io/projected/0988bafc-df97-4a53-8308-9e992be58d9a-kube-api-access-6zp88\") pod \"0988bafc-df97-4a53-8308-9e992be58d9a\" (UID: \"0988bafc-df97-4a53-8308-9e992be58d9a\") " Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.228739 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/7e109538-a2d2-43bb-8cef-bf99d7d0f1e9-run-httpd\") pod \"7e109538-a2d2-43bb-8cef-bf99d7d0f1e9\" (UID: \"7e109538-a2d2-43bb-8cef-bf99d7d0f1e9\") " Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.228766 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/0988bafc-df97-4a53-8308-9e992be58d9a-operator-scripts\") pod \"0988bafc-df97-4a53-8308-9e992be58d9a\" (UID: \"0988bafc-df97-4a53-8308-9e992be58d9a\") " Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.228816 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/9dec0a31-cba3-4c38-8b5d-4358f16b3704-logs\") pod \"9dec0a31-cba3-4c38-8b5d-4358f16b3704\" (UID: \"9dec0a31-cba3-4c38-8b5d-4358f16b3704\") " Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.228841 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/47aa63e7-b7b2-4c1d-a73d-348b50cc5d2b-kolla-config\") pod \"47aa63e7-b7b2-4c1d-a73d-348b50cc5d2b\" (UID: \"47aa63e7-b7b2-4c1d-a73d-348b50cc5d2b\") " Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.228883 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ddvrk\" (UniqueName: \"kubernetes.io/projected/7e109538-a2d2-43bb-8cef-bf99d7d0f1e9-kube-api-access-ddvrk\") pod \"7e109538-a2d2-43bb-8cef-bf99d7d0f1e9\" (UID: \"7e109538-a2d2-43bb-8cef-bf99d7d0f1e9\") " Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.228908 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/4a55b492-d57f-40dc-b719-1294fb24ef13-config-data\") pod \"4a55b492-d57f-40dc-b719-1294fb24ef13\" (UID: \"4a55b492-d57f-40dc-b719-1294fb24ef13\") " Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.228961 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/47aa63e7-b7b2-4c1d-a73d-348b50cc5d2b-config-data-default\") pod \"47aa63e7-b7b2-4c1d-a73d-348b50cc5d2b\" (UID: \"47aa63e7-b7b2-4c1d-a73d-348b50cc5d2b\") " Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.228988 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/4a55b492-d57f-40dc-b719-1294fb24ef13-scripts\") pod \"4a55b492-d57f-40dc-b719-1294fb24ef13\" (UID: \"4a55b492-d57f-40dc-b719-1294fb24ef13\") " Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.229038 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4a55b492-d57f-40dc-b719-1294fb24ef13-combined-ca-bundle\") pod \"4a55b492-d57f-40dc-b719-1294fb24ef13\" (UID: \"4a55b492-d57f-40dc-b719-1294fb24ef13\") " Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.229933 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/fb73cc9a-89cc-4cb8-87ce-2731e5dab0f7-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.229953 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e3a73fa0-68ef-4e03-81f9-2a143ed805bf-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.229965 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/fb73cc9a-89cc-4cb8-87ce-2731e5dab0f7-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.229977 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-5h59r\" (UniqueName: \"kubernetes.io/projected/9165ef68-b76d-472e-99af-06cc8c8a2d3e-kube-api-access-5h59r\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.230016 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/fb73cc9a-89cc-4cb8-87ce-2731e5dab0f7-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.230032 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/9165ef68-b76d-472e-99af-06cc8c8a2d3e-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.230044 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/fb73cc9a-89cc-4cb8-87ce-2731e5dab0f7-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:46 crc kubenswrapper[5052]: E1203 12:21:46.230135 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:21:46 crc kubenswrapper[5052]: E1203 12:21:46.230279 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/db31774e-8f50-4862-86f3-93cb93f645a0-operator-scripts podName:db31774e-8f50-4862-86f3-93cb93f645a0 nodeName:}" failed. No retries permitted until 2025-12-03 12:21:48.23025983 +0000 UTC m=+3698.818501594 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/db31774e-8f50-4862-86f3-93cb93f645a0-operator-scripts") pod "glance73ca-account-delete-vpqhp" (UID: "db31774e-8f50-4862-86f3-93cb93f645a0") : configmap "openstack-scripts" not found Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.232604 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e109538-a2d2-43bb-8cef-bf99d7d0f1e9-run-httpd" (OuterVolumeSpecName: "run-httpd") pod "7e109538-a2d2-43bb-8cef-bf99d7d0f1e9" (UID: "7e109538-a2d2-43bb-8cef-bf99d7d0f1e9"). InnerVolumeSpecName "run-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.233914 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e109538-a2d2-43bb-8cef-bf99d7d0f1e9-log-httpd" (OuterVolumeSpecName: "log-httpd") pod "7e109538-a2d2-43bb-8cef-bf99d7d0f1e9" (UID: "7e109538-a2d2-43bb-8cef-bf99d7d0f1e9"). InnerVolumeSpecName "log-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.234010 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9dec0a31-cba3-4c38-8b5d-4358f16b3704-logs" (OuterVolumeSpecName: "logs") pod "9dec0a31-cba3-4c38-8b5d-4358f16b3704" (UID: "9dec0a31-cba3-4c38-8b5d-4358f16b3704"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.235008 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0988bafc-df97-4a53-8308-9e992be58d9a-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "0988bafc-df97-4a53-8308-9e992be58d9a" (UID: "0988bafc-df97-4a53-8308-9e992be58d9a"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.235359 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/keystone-28d1-account-create-update-f27hx"] Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.236215 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/47aa63e7-b7b2-4c1d-a73d-348b50cc5d2b-kolla-config" (OuterVolumeSpecName: "kolla-config") pod "47aa63e7-b7b2-4c1d-a73d-348b50cc5d2b" (UID: "47aa63e7-b7b2-4c1d-a73d-348b50cc5d2b"). InnerVolumeSpecName "kolla-config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.242631 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/47aa63e7-b7b2-4c1d-a73d-348b50cc5d2b-config-data-generated" (OuterVolumeSpecName: "config-data-generated") pod "47aa63e7-b7b2-4c1d-a73d-348b50cc5d2b" (UID: "47aa63e7-b7b2-4c1d-a73d-348b50cc5d2b"). InnerVolumeSpecName "config-data-generated". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.245838 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0988bafc-df97-4a53-8308-9e992be58d9a-config-data-default" (OuterVolumeSpecName: "config-data-default") pod "0988bafc-df97-4a53-8308-9e992be58d9a" (UID: "0988bafc-df97-4a53-8308-9e992be58d9a"). InnerVolumeSpecName "config-data-default". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.246755 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/4a55b492-d57f-40dc-b719-1294fb24ef13-etc-machine-id" (OuterVolumeSpecName: "etc-machine-id") pod "4a55b492-d57f-40dc-b719-1294fb24ef13" (UID: "4a55b492-d57f-40dc-b719-1294fb24ef13"). InnerVolumeSpecName "etc-machine-id". PluginName "kubernetes.io/host-path", VolumeGidValue "" Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.247218 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0988bafc-df97-4a53-8308-9e992be58d9a-kolla-config" (OuterVolumeSpecName: "kolla-config") pod "0988bafc-df97-4a53-8308-9e992be58d9a" (UID: "0988bafc-df97-4a53-8308-9e992be58d9a"). InnerVolumeSpecName "kolla-config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.248309 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0988bafc-df97-4a53-8308-9e992be58d9a-config-data-generated" (OuterVolumeSpecName: "config-data-generated") pod "0988bafc-df97-4a53-8308-9e992be58d9a" (UID: "0988bafc-df97-4a53-8308-9e992be58d9a"). InnerVolumeSpecName "config-data-generated". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.248630 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/memcached-0" Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.251285 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/47aa63e7-b7b2-4c1d-a73d-348b50cc5d2b-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "47aa63e7-b7b2-4c1d-a73d-348b50cc5d2b" (UID: "47aa63e7-b7b2-4c1d-a73d-348b50cc5d2b"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.260280 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/keystone-28d1-account-create-update-f27hx"] Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.260614 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/47aa63e7-b7b2-4c1d-a73d-348b50cc5d2b-config-data-default" (OuterVolumeSpecName: "config-data-default") pod "47aa63e7-b7b2-4c1d-a73d-348b50cc5d2b" (UID: "47aa63e7-b7b2-4c1d-a73d-348b50cc5d2b"). InnerVolumeSpecName "config-data-default". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.269773 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9dec0a31-cba3-4c38-8b5d-4358f16b3704-kube-api-access-8t4hw" (OuterVolumeSpecName: "kube-api-access-8t4hw") pod "9dec0a31-cba3-4c38-8b5d-4358f16b3704" (UID: "9dec0a31-cba3-4c38-8b5d-4358f16b3704"). InnerVolumeSpecName "kube-api-access-8t4hw". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.274566 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7e109538-a2d2-43bb-8cef-bf99d7d0f1e9-kube-api-access-ddvrk" (OuterVolumeSpecName: "kube-api-access-ddvrk") pod "7e109538-a2d2-43bb-8cef-bf99d7d0f1e9" (UID: "7e109538-a2d2-43bb-8cef-bf99d7d0f1e9"). InnerVolumeSpecName "kube-api-access-ddvrk". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.277198 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/novacell13fba-account-delete-hvzdt"] Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.282057 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4a55b492-d57f-40dc-b719-1294fb24ef13-scripts" (OuterVolumeSpecName: "scripts") pod "4a55b492-d57f-40dc-b719-1294fb24ef13" (UID: "4a55b492-d57f-40dc-b719-1294fb24ef13"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.285162 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/novacell13fba-account-delete-hvzdt"] Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.293905 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4a55b492-d57f-40dc-b719-1294fb24ef13-config-data-custom" (OuterVolumeSpecName: "config-data-custom") pod "4a55b492-d57f-40dc-b719-1294fb24ef13" (UID: "4a55b492-d57f-40dc-b719-1294fb24ef13"). InnerVolumeSpecName "config-data-custom". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.298271 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4a55b492-d57f-40dc-b719-1294fb24ef13-kube-api-access-z4ktp" (OuterVolumeSpecName: "kube-api-access-z4ktp") pod "4a55b492-d57f-40dc-b719-1294fb24ef13" (UID: "4a55b492-d57f-40dc-b719-1294fb24ef13"). InnerVolumeSpecName "kube-api-access-z4ktp". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.301706 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7e109538-a2d2-43bb-8cef-bf99d7d0f1e9-etc-swift" (OuterVolumeSpecName: "etc-swift") pod "7e109538-a2d2-43bb-8cef-bf99d7d0f1e9" (UID: "7e109538-a2d2-43bb-8cef-bf99d7d0f1e9"). InnerVolumeSpecName "etc-swift". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.304310 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/47aa63e7-b7b2-4c1d-a73d-348b50cc5d2b-kube-api-access-njjpc" (OuterVolumeSpecName: "kube-api-access-njjpc") pod "47aa63e7-b7b2-4c1d-a73d-348b50cc5d2b" (UID: "47aa63e7-b7b2-4c1d-a73d-348b50cc5d2b"). InnerVolumeSpecName "kube-api-access-njjpc". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.304385 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0988bafc-df97-4a53-8308-9e992be58d9a-kube-api-access-6zp88" (OuterVolumeSpecName: "kube-api-access-6zp88") pod "0988bafc-df97-4a53-8308-9e992be58d9a" (UID: "0988bafc-df97-4a53-8308-9e992be58d9a"). InnerVolumeSpecName "kube-api-access-6zp88". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.333903 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/adcea736-eb15-4768-b12c-edb4c810beb3-kolla-config\") pod \"adcea736-eb15-4768-b12c-edb4c810beb3\" (UID: \"adcea736-eb15-4768-b12c-edb4c810beb3\") " Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.333964 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/adcea736-eb15-4768-b12c-edb4c810beb3-combined-ca-bundle\") pod \"adcea736-eb15-4768-b12c-edb4c810beb3\" (UID: \"adcea736-eb15-4768-b12c-edb4c810beb3\") " Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.334098 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sc7n7\" (UniqueName: \"kubernetes.io/projected/adcea736-eb15-4768-b12c-edb4c810beb3-kube-api-access-sc7n7\") pod \"adcea736-eb15-4768-b12c-edb4c810beb3\" (UID: \"adcea736-eb15-4768-b12c-edb4c810beb3\") " Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.334221 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/adcea736-eb15-4768-b12c-edb4c810beb3-config-data\") pod \"adcea736-eb15-4768-b12c-edb4c810beb3\" (UID: \"adcea736-eb15-4768-b12c-edb4c810beb3\") " Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.334301 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"memcached-tls-certs\" (UniqueName: \"kubernetes.io/secret/adcea736-eb15-4768-b12c-edb4c810beb3-memcached-tls-certs\") pod \"adcea736-eb15-4768-b12c-edb4c810beb3\" (UID: \"adcea736-eb15-4768-b12c-edb4c810beb3\") " Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.335013 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-ddvrk\" (UniqueName: \"kubernetes.io/projected/7e109538-a2d2-43bb-8cef-bf99d7d0f1e9-kube-api-access-ddvrk\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.335040 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/47aa63e7-b7b2-4c1d-a73d-348b50cc5d2b-config-data-default\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.335055 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/4a55b492-d57f-40dc-b719-1294fb24ef13-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.335066 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/4a55b492-d57f-40dc-b719-1294fb24ef13-config-data-custom\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.335077 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-z4ktp\" (UniqueName: \"kubernetes.io/projected/4a55b492-d57f-40dc-b719-1294fb24ef13-kube-api-access-z4ktp\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.335090 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-njjpc\" (UniqueName: \"kubernetes.io/projected/47aa63e7-b7b2-4c1d-a73d-348b50cc5d2b-kube-api-access-njjpc\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.335103 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/0988bafc-df97-4a53-8308-9e992be58d9a-config-data-generated\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.335114 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-75s94\" (UniqueName: \"kubernetes.io/projected/bfc242a2-aca4-458b-812e-018787bdbcab-kube-api-access-75s94\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.335127 5052 reconciler_common.go:293] "Volume detached for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/4a55b492-d57f-40dc-b719-1294fb24ef13-etc-machine-id\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.335139 5052 reconciler_common.go:293] "Volume detached for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/0988bafc-df97-4a53-8308-9e992be58d9a-kolla-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.335153 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/bfc242a2-aca4-458b-812e-018787bdbcab-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.335164 5052 reconciler_common.go:293] "Volume detached for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/7e109538-a2d2-43bb-8cef-bf99d7d0f1e9-etc-swift\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.335174 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-8t4hw\" (UniqueName: \"kubernetes.io/projected/9dec0a31-cba3-4c38-8b5d-4358f16b3704-kube-api-access-8t4hw\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.335187 5052 reconciler_common.go:293] "Volume detached for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/7e109538-a2d2-43bb-8cef-bf99d7d0f1e9-log-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.335198 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/47aa63e7-b7b2-4c1d-a73d-348b50cc5d2b-config-data-generated\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.335209 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-vvdpm\" (UniqueName: \"kubernetes.io/projected/0f00d4de-f49c-4155-98e7-00be8bc4ddc7-kube-api-access-vvdpm\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.335221 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/47aa63e7-b7b2-4c1d-a73d-348b50cc5d2b-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.335232 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/0f00d4de-f49c-4155-98e7-00be8bc4ddc7-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.335245 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/0988bafc-df97-4a53-8308-9e992be58d9a-config-data-default\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.335257 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-6zp88\" (UniqueName: \"kubernetes.io/projected/0988bafc-df97-4a53-8308-9e992be58d9a-kube-api-access-6zp88\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.335268 5052 reconciler_common.go:293] "Volume detached for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/7e109538-a2d2-43bb-8cef-bf99d7d0f1e9-run-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.335279 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/0988bafc-df97-4a53-8308-9e992be58d9a-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.335290 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/9dec0a31-cba3-4c38-8b5d-4358f16b3704-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.335300 5052 reconciler_common.go:293] "Volume detached for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/47aa63e7-b7b2-4c1d-a73d-348b50cc5d2b-kolla-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.335758 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage03-crc" (OuterVolumeSpecName: "mysql-db") pod "0988bafc-df97-4a53-8308-9e992be58d9a" (UID: "0988bafc-df97-4a53-8308-9e992be58d9a"). InnerVolumeSpecName "local-storage03-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.338982 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/adcea736-eb15-4768-b12c-edb4c810beb3-kolla-config" (OuterVolumeSpecName: "kolla-config") pod "adcea736-eb15-4768-b12c-edb4c810beb3" (UID: "adcea736-eb15-4768-b12c-edb4c810beb3"). InnerVolumeSpecName "kolla-config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.339133 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/adcea736-eb15-4768-b12c-edb4c810beb3-config-data" (OuterVolumeSpecName: "config-data") pod "adcea736-eb15-4768-b12c-edb4c810beb3" (UID: "adcea736-eb15-4768-b12c-edb4c810beb3"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.342967 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/adcea736-eb15-4768-b12c-edb4c810beb3-kube-api-access-sc7n7" (OuterVolumeSpecName: "kube-api-access-sc7n7") pod "adcea736-eb15-4768-b12c-edb4c810beb3" (UID: "adcea736-eb15-4768-b12c-edb4c810beb3"). InnerVolumeSpecName "kube-api-access-sc7n7". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.348924 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage06-crc" (OuterVolumeSpecName: "mysql-db") pod "47aa63e7-b7b2-4c1d-a73d-348b50cc5d2b" (UID: "47aa63e7-b7b2-4c1d-a73d-348b50cc5d2b"). InnerVolumeSpecName "local-storage06-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.383962 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/47aa63e7-b7b2-4c1d-a73d-348b50cc5d2b-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "47aa63e7-b7b2-4c1d-a73d-348b50cc5d2b" (UID: "47aa63e7-b7b2-4c1d-a73d-348b50cc5d2b"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.384701 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9dec0a31-cba3-4c38-8b5d-4358f16b3704-config-data" (OuterVolumeSpecName: "config-data") pod "9dec0a31-cba3-4c38-8b5d-4358f16b3704" (UID: "9dec0a31-cba3-4c38-8b5d-4358f16b3704"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.416986 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican0e0e-account-delete-fvkr2"] Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.424042 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/neutron-69c9545844-xf6zm" podUID="03bb65ef-e9a8-4cfb-9536-001e316b1f72" containerName="neutron-httpd" probeResult="failure" output="Get \"https://10.217.1.172:9696/\": dial tcp 10.217.1.172:9696: connect: connection refused" Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.429756 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0988bafc-df97-4a53-8308-9e992be58d9a-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "0988bafc-df97-4a53-8308-9e992be58d9a" (UID: "0988bafc-df97-4a53-8308-9e992be58d9a"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.430857 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9dec0a31-cba3-4c38-8b5d-4358f16b3704-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "9dec0a31-cba3-4c38-8b5d-4358f16b3704" (UID: "9dec0a31-cba3-4c38-8b5d-4358f16b3704"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.437065 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/adcea736-eb15-4768-b12c-edb4c810beb3-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.439182 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0988bafc-df97-4a53-8308-9e992be58d9a-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.439287 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9dec0a31-cba3-4c38-8b5d-4358f16b3704-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.439397 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage06-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage06-crc\") on node \"crc\" " Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.439482 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/9dec0a31-cba3-4c38-8b5d-4358f16b3704-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.439584 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/47aa63e7-b7b2-4c1d-a73d-348b50cc5d2b-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.439660 5052 reconciler_common.go:293] "Volume detached for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/adcea736-eb15-4768-b12c-edb4c810beb3-kolla-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.439736 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-sc7n7\" (UniqueName: \"kubernetes.io/projected/adcea736-eb15-4768-b12c-edb4c810beb3-kube-api-access-sc7n7\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.439833 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage03-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage03-crc\") on node \"crc\" " Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.483679 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/cinder-api-0" podUID="bb41ed7e-c96b-44dd-8c4e-ffb702de3f87" containerName="cinder-api" probeResult="failure" output="Get \"https://10.217.1.175:8776/healthcheck\": dial tcp 10.217.1.175:8776: connect: connection refused" Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.524085 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/adcea736-eb15-4768-b12c-edb4c810beb3-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "adcea736-eb15-4768-b12c-edb4c810beb3" (UID: "adcea736-eb15-4768-b12c-edb4c810beb3"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.527310 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7e109538-a2d2-43bb-8cef-bf99d7d0f1e9-config-data" (OuterVolumeSpecName: "config-data") pod "7e109538-a2d2-43bb-8cef-bf99d7d0f1e9" (UID: "7e109538-a2d2-43bb-8cef-bf99d7d0f1e9"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.532876 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7e109538-a2d2-43bb-8cef-bf99d7d0f1e9-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "7e109538-a2d2-43bb-8cef-bf99d7d0f1e9" (UID: "7e109538-a2d2-43bb-8cef-bf99d7d0f1e9"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.540960 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/7e109538-a2d2-43bb-8cef-bf99d7d0f1e9-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.540998 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/7e109538-a2d2-43bb-8cef-bf99d7d0f1e9-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.541010 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/adcea736-eb15-4768-b12c-edb4c810beb3-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.542084 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/barbican-api-644f564b46-cjtkx" podUID="af38eb0d-3924-4c4b-a22e-7e934bdcf172" containerName="barbican-api" probeResult="failure" output="Get \"https://10.217.1.180:9311/healthcheck\": read tcp 10.217.0.2:43594->10.217.1.180:9311: read: connection reset by peer" Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.542624 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/barbican-api-644f564b46-cjtkx" podUID="af38eb0d-3924-4c4b-a22e-7e934bdcf172" containerName="barbican-api-log" probeResult="failure" output="Get \"https://10.217.1.180:9311/healthcheck\": read tcp 10.217.0.2:43580->10.217.1.180:9311: read: connection reset by peer" Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.544855 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage06-crc" (UniqueName: "kubernetes.io/local-volume/local-storage06-crc") on node "crc" Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.551736 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7e109538-a2d2-43bb-8cef-bf99d7d0f1e9-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "7e109538-a2d2-43bb-8cef-bf99d7d0f1e9" (UID: "7e109538-a2d2-43bb-8cef-bf99d7d0f1e9"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.584090 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/47aa63e7-b7b2-4c1d-a73d-348b50cc5d2b-galera-tls-certs" (OuterVolumeSpecName: "galera-tls-certs") pod "47aa63e7-b7b2-4c1d-a73d-348b50cc5d2b" (UID: "47aa63e7-b7b2-4c1d-a73d-348b50cc5d2b"). InnerVolumeSpecName "galera-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.584858 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0988bafc-df97-4a53-8308-9e992be58d9a-galera-tls-certs" (OuterVolumeSpecName: "galera-tls-certs") pod "0988bafc-df97-4a53-8308-9e992be58d9a" (UID: "0988bafc-df97-4a53-8308-9e992be58d9a"). InnerVolumeSpecName "galera-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.599709 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9dec0a31-cba3-4c38-8b5d-4358f16b3704-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "9dec0a31-cba3-4c38-8b5d-4358f16b3704" (UID: "9dec0a31-cba3-4c38-8b5d-4358f16b3704"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.602435 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/adcea736-eb15-4768-b12c-edb4c810beb3-memcached-tls-certs" (OuterVolumeSpecName: "memcached-tls-certs") pod "adcea736-eb15-4768-b12c-edb4c810beb3" (UID: "adcea736-eb15-4768-b12c-edb4c810beb3"). InnerVolumeSpecName "memcached-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.617263 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage03-crc" (UniqueName: "kubernetes.io/local-volume/local-storage03-crc") on node "crc" Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.654914 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/7e109538-a2d2-43bb-8cef-bf99d7d0f1e9-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.654954 5052 reconciler_common.go:293] "Volume detached for volume \"memcached-tls-certs\" (UniqueName: \"kubernetes.io/secret/adcea736-eb15-4768-b12c-edb4c810beb3-memcached-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.654964 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage06-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage06-crc\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.654976 5052 reconciler_common.go:293] "Volume detached for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/47aa63e7-b7b2-4c1d-a73d-348b50cc5d2b-galera-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.654987 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage03-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage03-crc\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.655000 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/9dec0a31-cba3-4c38-8b5d-4358f16b3704-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.655012 5052 reconciler_common.go:293] "Volume detached for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/0988bafc-df97-4a53-8308-9e992be58d9a-galera-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.703486 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7e109538-a2d2-43bb-8cef-bf99d7d0f1e9-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "7e109538-a2d2-43bb-8cef-bf99d7d0f1e9" (UID: "7e109538-a2d2-43bb-8cef-bf99d7d0f1e9"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.712596 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9dec0a31-cba3-4c38-8b5d-4358f16b3704-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "9dec0a31-cba3-4c38-8b5d-4358f16b3704" (UID: "9dec0a31-cba3-4c38-8b5d-4358f16b3704"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.757240 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/7e109538-a2d2-43bb-8cef-bf99d7d0f1e9-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.757283 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/9dec0a31-cba3-4c38-8b5d-4358f16b3704-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.822456 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4a55b492-d57f-40dc-b719-1294fb24ef13-config-data" (OuterVolumeSpecName: "config-data") pod "4a55b492-d57f-40dc-b719-1294fb24ef13" (UID: "4a55b492-d57f-40dc-b719-1294fb24ef13"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.825270 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4a55b492-d57f-40dc-b719-1294fb24ef13-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "4a55b492-d57f-40dc-b719-1294fb24ef13" (UID: "4a55b492-d57f-40dc-b719-1294fb24ef13"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.858376 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/4a55b492-d57f-40dc-b719-1294fb24ef13-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:46 crc kubenswrapper[5052]: I1203 12:21:46.858408 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4a55b492-d57f-40dc-b719-1294fb24ef13-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.028266 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-proxy-6b9f9968d-5zjp2" event={"ID":"fb73cc9a-89cc-4cb8-87ce-2731e5dab0f7","Type":"ContainerDied","Data":"626262d6ee8fb51930f0b1e62c74d74c26de191482f371f01edff5823037b3bf"} Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.028327 5052 scope.go:117] "RemoveContainer" containerID="370f335a2865f2cdbcf3d953581d2819c13f1b7d88a0ac612179ca394589c0e9" Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.028473 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-proxy-6b9f9968d-5zjp2" Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.036809 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-proxy-d4747d79f-wtbhv" event={"ID":"7e109538-a2d2-43bb-8cef-bf99d7d0f1e9","Type":"ContainerDied","Data":"c16bc0acd0b2a80765797511bb2ab0ad881fee07b3ee755104460e0f97a88f66"} Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.036837 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-proxy-d4747d79f-wtbhv" Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.052835 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder2067-account-delete-n2tf7"] Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.065008 5052 generic.go:334] "Generic (PLEG): container finished" podID="af38eb0d-3924-4c4b-a22e-7e934bdcf172" containerID="0b9c5ec46decf17edaf9cf870fd66992b0a9f87a5bcb4a208561e9aa1a0d41be" exitCode=0 Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.065097 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-644f564b46-cjtkx" event={"ID":"af38eb0d-3924-4c4b-a22e-7e934bdcf172","Type":"ContainerDied","Data":"0b9c5ec46decf17edaf9cf870fd66992b0a9f87a5bcb4a208561e9aa1a0d41be"} Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.065640 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/novaapi8f5b-account-delete-86k67"] Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.078407 5052 generic.go:334] "Generic (PLEG): container finished" podID="2d94bf00-7a64-4e90-a48c-b206ec54da6d" containerID="e0441d5665d194615504d1f41084afce1caa468996e04b063d890632f490acca" exitCode=0 Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.078494 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"2d94bf00-7a64-4e90-a48c-b206ec54da6d","Type":"ContainerDied","Data":"e0441d5665d194615504d1f41084afce1caa468996e04b063d890632f490acca"} Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.081623 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"9dec0a31-cba3-4c38-8b5d-4358f16b3704","Type":"ContainerDied","Data":"301bbf49f5f1e1267a4886e2478668c30d4e45b89f48d5139fe9429743fa8ca6"} Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.081723 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.091575 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-7mq7s"] Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.094878 5052 generic.go:334] "Generic (PLEG): container finished" podID="a7907bc2-8e73-4272-b779-fd71d4acb1f4" containerID="938a22611459fa789b24b03d2a21853b69071aad6d79f25384679595984908d8" exitCode=0 Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.095012 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-conductor-0" event={"ID":"a7907bc2-8e73-4272-b779-fd71d4acb1f4","Type":"ContainerDied","Data":"938a22611459fa789b24b03d2a21853b69071aad6d79f25384679595984908d8"} Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.096665 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-api-8f5b-account-create-update-m9q79"] Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.101557 5052 generic.go:334] "Generic (PLEG): container finished" podID="72b87ae1-6854-475b-a31d-976acc913623" containerID="ace907d0e00f36ffb29d30e119e8630da96914e43d3d15c7fdb73b39b73ccfd8" exitCode=0 Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.101607 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"72b87ae1-6854-475b-a31d-976acc913623","Type":"ContainerDied","Data":"ace907d0e00f36ffb29d30e119e8630da96914e43d3d15c7fdb73b39b73ccfd8"} Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.102233 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/novacell00e20-account-delete-ktxhg"] Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.103780 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican0e0e-account-delete-fvkr2" event={"ID":"fc2a6873-8741-472a-864d-84651abd4c1c","Type":"ContainerStarted","Data":"7876e19d880dc70ae433b6bd5a052ca9f95be159fc29a72a9c4c95cb85db0c42"} Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.107756 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/neutron734a-account-delete-sxpn9"] Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.112245 5052 generic.go:334] "Generic (PLEG): container finished" podID="bb41ed7e-c96b-44dd-8c4e-ffb702de3f87" containerID="2a7af42a9086e0868a44334065634bf0f3e6cbd7e859d0ffd6a5df7b8b1b5555" exitCode=0 Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.112321 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"bb41ed7e-c96b-44dd-8c4e-ffb702de3f87","Type":"ContainerDied","Data":"2a7af42a9086e0868a44334065634bf0f3e6cbd7e859d0ffd6a5df7b8b1b5555"} Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.114607 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"4a55b492-d57f-40dc-b719-1294fb24ef13","Type":"ContainerDied","Data":"ec5ded814017ddd24b3aff8929f5844220d80b4ac8f9878cfd10a94a254e0ca6"} Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.114706 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.120396 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-cell1-galera-0" event={"ID":"47aa63e7-b7b2-4c1d-a73d-348b50cc5d2b","Type":"ContainerDied","Data":"c5698971234998dd0322ba0867ba6b5c54ebfdcf6ae1aa262542b9b50030540b"} Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.120522 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.126546 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-galera-0" event={"ID":"0988bafc-df97-4a53-8308-9e992be58d9a","Type":"ContainerDied","Data":"479380ef7f22d0dd218de093b97f58b18735e0c4321f78d3eb90de6a30bf5d82"} Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.126707 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.134863 5052 generic.go:334] "Generic (PLEG): container finished" podID="90814ea7-1abb-4760-bbfa-1190f23ab38a" containerID="0c61528c202083e822a6d60926f5531661737d72572cc3938c0bb3db2f2fcd35" exitCode=0 Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.134953 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-5447868864-mfdcg" event={"ID":"90814ea7-1abb-4760-bbfa-1190f23ab38a","Type":"ContainerDied","Data":"0c61528c202083e822a6d60926f5531661737d72572cc3938c0bb3db2f2fcd35"} Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.134984 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-5447868864-mfdcg" event={"ID":"90814ea7-1abb-4760-bbfa-1190f23ab38a","Type":"ContainerDied","Data":"be3444a0d99f7a4b2d30e89d0c25a76e9126e091283a312597b11200cc324aa1"} Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.134999 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="be3444a0d99f7a4b2d30e89d0c25a76e9126e091283a312597b11200cc324aa1" Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.149129 5052 scope.go:117] "RemoveContainer" containerID="a1d1e7cece7f6bab44f530f29cd0b9119aee96b8c725b66c1bb8c782f4ccd637" Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.152743 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-5447868864-mfdcg" Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.161920 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/memcached-0" event={"ID":"adcea736-eb15-4768-b12c-edb4c810beb3","Type":"ContainerDied","Data":"e4968308844574087d5174a9ee90bc9f9ed1b29332de5a1f79f4ced53f213edb"} Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.161946 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/memcached-0" Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.171450 5052 generic.go:334] "Generic (PLEG): container finished" podID="2d217a2f-3478-41e0-936a-6b1d01530007" containerID="c432832de2a3a0521fe28359f14c05384338a7557e1245553a8611f5a30ee883" exitCode=0 Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.171639 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"2d217a2f-3478-41e0-936a-6b1d01530007","Type":"ContainerDied","Data":"c432832de2a3a0521fe28359f14c05384338a7557e1245553a8611f5a30ee883"} Dec 03 12:21:47 crc kubenswrapper[5052]: E1203 12:21:47.179100 5052 kuberuntime_manager.go:1274] "Unhandled Error" err=< Dec 03 12:21:47 crc kubenswrapper[5052]: container &Container{Name:mariadb-account-create-update,Image:quay.io/podified-antelope-centos9/openstack-mariadb@sha256:d9a2d8b19d3de4658dd26a2e781d00002e937738bfe6a1d0cf6c68c015085f4a,Command:[/bin/sh -c #!/bin/bash Dec 03 12:21:47 crc kubenswrapper[5052]: Dec 03 12:21:47 crc kubenswrapper[5052]: MYSQL_REMOTE_HOST=openstack.openstack-kuttl-tests.svc source /var/lib/operator-scripts/mysql_root_auth.sh Dec 03 12:21:47 crc kubenswrapper[5052]: Dec 03 12:21:47 crc kubenswrapper[5052]: export DatabasePassword=${DatabasePassword:?"Please specify a DatabasePassword variable."} Dec 03 12:21:47 crc kubenswrapper[5052]: Dec 03 12:21:47 crc kubenswrapper[5052]: MYSQL_CMD="mysql -h openstack.openstack-kuttl-tests.svc -u root -P 3306" Dec 03 12:21:47 crc kubenswrapper[5052]: Dec 03 12:21:47 crc kubenswrapper[5052]: if [ -n "nova_api" ]; then Dec 03 12:21:47 crc kubenswrapper[5052]: GRANT_DATABASE="nova_api" Dec 03 12:21:47 crc kubenswrapper[5052]: else Dec 03 12:21:47 crc kubenswrapper[5052]: GRANT_DATABASE="*" Dec 03 12:21:47 crc kubenswrapper[5052]: fi Dec 03 12:21:47 crc kubenswrapper[5052]: Dec 03 12:21:47 crc kubenswrapper[5052]: # going for maximum compatibility here: Dec 03 12:21:47 crc kubenswrapper[5052]: # 1. MySQL 8 no longer allows implicit create user when GRANT is used Dec 03 12:21:47 crc kubenswrapper[5052]: # 2. MariaDB has "CREATE OR REPLACE", but MySQL does not Dec 03 12:21:47 crc kubenswrapper[5052]: # 3. create user with CREATE but then do all password and TLS with ALTER to Dec 03 12:21:47 crc kubenswrapper[5052]: # support updates Dec 03 12:21:47 crc kubenswrapper[5052]: Dec 03 12:21:47 crc kubenswrapper[5052]: $MYSQL_CMD < logger="UnhandledError" Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.179635 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/neutron-734a-account-create-update-qcgft" podUID="817345c8-3a42-4dea-8158-75973d9c41d9" containerName="mariadb-account-create-update" containerID="cri-o://609a1a60c18e09d802dbabfa7e561327519116592dcd204cb6b24fec223b9b18" gracePeriod=30 Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.179852 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance73ca-account-delete-vpqhp" podUID="db31774e-8f50-4862-86f3-93cb93f645a0" containerName="mariadb-account-delete" containerID="cri-o://9661574f87aebd235d130b4c19903ffb15d5afc4025f54b00c2da496a34e0aad" gracePeriod=30 Dec 03 12:21:47 crc kubenswrapper[5052]: E1203 12:21:47.181103 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"mariadb-account-create-update\" with CreateContainerConfigError: \"secret \\\"nova-api-db-secret\\\" not found\"" pod="openstack-kuttl-tests/nova-api-8f5b-account-create-update-m9q79" podUID="66954cf4-1c7b-4ca6-a6cf-b73bde8d670f" Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.226236 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:21:47 crc kubenswrapper[5052]: W1203 12:21:47.227307 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcdf67382_3145_4280_a5a5_41b0185cb07f.slice/crio-3f984b732b7d3c63737bf25b10c931347b0989361ec2884621e57852bcae454a WatchSource:0}: Error finding container 3f984b732b7d3c63737bf25b10c931347b0989361ec2884621e57852bcae454a: Status 404 returned error can't find the container with id 3f984b732b7d3c63737bf25b10c931347b0989361ec2884621e57852bcae454a Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.261127 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.261169 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.261538 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.269125 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8f26f\" (UniqueName: \"kubernetes.io/projected/90814ea7-1abb-4760-bbfa-1190f23ab38a-kube-api-access-8f26f\") pod \"90814ea7-1abb-4760-bbfa-1190f23ab38a\" (UID: \"90814ea7-1abb-4760-bbfa-1190f23ab38a\") " Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.269267 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/90814ea7-1abb-4760-bbfa-1190f23ab38a-logs\") pod \"90814ea7-1abb-4760-bbfa-1190f23ab38a\" (UID: \"90814ea7-1abb-4760-bbfa-1190f23ab38a\") " Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.269333 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/90814ea7-1abb-4760-bbfa-1190f23ab38a-internal-tls-certs\") pod \"90814ea7-1abb-4760-bbfa-1190f23ab38a\" (UID: \"90814ea7-1abb-4760-bbfa-1190f23ab38a\") " Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.269380 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/90814ea7-1abb-4760-bbfa-1190f23ab38a-combined-ca-bundle\") pod \"90814ea7-1abb-4760-bbfa-1190f23ab38a\" (UID: \"90814ea7-1abb-4760-bbfa-1190f23ab38a\") " Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.269412 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/90814ea7-1abb-4760-bbfa-1190f23ab38a-config-data\") pod \"90814ea7-1abb-4760-bbfa-1190f23ab38a\" (UID: \"90814ea7-1abb-4760-bbfa-1190f23ab38a\") " Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.269464 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/90814ea7-1abb-4760-bbfa-1190f23ab38a-public-tls-certs\") pod \"90814ea7-1abb-4760-bbfa-1190f23ab38a\" (UID: \"90814ea7-1abb-4760-bbfa-1190f23ab38a\") " Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.269492 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/90814ea7-1abb-4760-bbfa-1190f23ab38a-scripts\") pod \"90814ea7-1abb-4760-bbfa-1190f23ab38a\" (UID: \"90814ea7-1abb-4760-bbfa-1190f23ab38a\") " Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.272500 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/90814ea7-1abb-4760-bbfa-1190f23ab38a-logs" (OuterVolumeSpecName: "logs") pod "90814ea7-1abb-4760-bbfa-1190f23ab38a" (UID: "90814ea7-1abb-4760-bbfa-1190f23ab38a"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.283554 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/swift-proxy-d4747d79f-wtbhv"] Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.311795 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/swift-proxy-d4747d79f-wtbhv"] Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.314599 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.322638 5052 scope.go:117] "RemoveContainer" containerID="ef9fabe8451cd9b904301777198ce63bf6cefd87c577b4cad8746d28b098af4a" Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.325158 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/90814ea7-1abb-4760-bbfa-1190f23ab38a-kube-api-access-8f26f" (OuterVolumeSpecName: "kube-api-access-8f26f") pod "90814ea7-1abb-4760-bbfa-1190f23ab38a" (UID: "90814ea7-1abb-4760-bbfa-1190f23ab38a"). InnerVolumeSpecName "kube-api-access-8f26f". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.328009 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/90814ea7-1abb-4760-bbfa-1190f23ab38a-scripts" (OuterVolumeSpecName: "scripts") pod "90814ea7-1abb-4760-bbfa-1190f23ab38a" (UID: "90814ea7-1abb-4760-bbfa-1190f23ab38a"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.334362 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-api-644f564b46-cjtkx" Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.335991 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" podUID="c279eaf1-2ee2-46fe-88be-9a5a65d4897e" containerName="rabbitmq" probeResult="failure" output="dial tcp 10.217.1.170:5671: connect: connection refused" Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.361654 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/swift-proxy-6b9f9968d-5zjp2"] Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.371425 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/72b87ae1-6854-475b-a31d-976acc913623-httpd-run\") pod \"72b87ae1-6854-475b-a31d-976acc913623\" (UID: \"72b87ae1-6854-475b-a31d-976acc913623\") " Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.371492 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dmhw4\" (UniqueName: \"kubernetes.io/projected/72b87ae1-6854-475b-a31d-976acc913623-kube-api-access-dmhw4\") pod \"72b87ae1-6854-475b-a31d-976acc913623\" (UID: \"72b87ae1-6854-475b-a31d-976acc913623\") " Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.371549 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kjtr9\" (UniqueName: \"kubernetes.io/projected/bb41ed7e-c96b-44dd-8c4e-ffb702de3f87-kube-api-access-kjtr9\") pod \"bb41ed7e-c96b-44dd-8c4e-ffb702de3f87\" (UID: \"bb41ed7e-c96b-44dd-8c4e-ffb702de3f87\") " Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.371574 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/bb41ed7e-c96b-44dd-8c4e-ffb702de3f87-etc-machine-id\") pod \"bb41ed7e-c96b-44dd-8c4e-ffb702de3f87\" (UID: \"bb41ed7e-c96b-44dd-8c4e-ffb702de3f87\") " Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.371611 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"glance\" (UniqueName: \"kubernetes.io/local-volume/local-storage09-crc\") pod \"72b87ae1-6854-475b-a31d-976acc913623\" (UID: \"72b87ae1-6854-475b-a31d-976acc913623\") " Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.371656 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/bb41ed7e-c96b-44dd-8c4e-ffb702de3f87-combined-ca-bundle\") pod \"bb41ed7e-c96b-44dd-8c4e-ffb702de3f87\" (UID: \"bb41ed7e-c96b-44dd-8c4e-ffb702de3f87\") " Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.371681 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/bb41ed7e-c96b-44dd-8c4e-ffb702de3f87-internal-tls-certs\") pod \"bb41ed7e-c96b-44dd-8c4e-ffb702de3f87\" (UID: \"bb41ed7e-c96b-44dd-8c4e-ffb702de3f87\") " Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.371708 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/72b87ae1-6854-475b-a31d-976acc913623-combined-ca-bundle\") pod \"72b87ae1-6854-475b-a31d-976acc913623\" (UID: \"72b87ae1-6854-475b-a31d-976acc913623\") " Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.371734 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/72b87ae1-6854-475b-a31d-976acc913623-config-data\") pod \"72b87ae1-6854-475b-a31d-976acc913623\" (UID: \"72b87ae1-6854-475b-a31d-976acc913623\") " Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.372498 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/bb41ed7e-c96b-44dd-8c4e-ffb702de3f87-etc-machine-id" (OuterVolumeSpecName: "etc-machine-id") pod "bb41ed7e-c96b-44dd-8c4e-ffb702de3f87" (UID: "bb41ed7e-c96b-44dd-8c4e-ffb702de3f87"). InnerVolumeSpecName "etc-machine-id". PluginName "kubernetes.io/host-path", VolumeGidValue "" Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.375130 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/72b87ae1-6854-475b-a31d-976acc913623-httpd-run" (OuterVolumeSpecName: "httpd-run") pod "72b87ae1-6854-475b-a31d-976acc913623" (UID: "72b87ae1-6854-475b-a31d-976acc913623"). InnerVolumeSpecName "httpd-run". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.377749 5052 scope.go:117] "RemoveContainer" containerID="17114cefe25d8670369fc24018b4320e72bd1b81aa69b034a694652ddb010caa" Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.378150 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/swift-proxy-6b9f9968d-5zjp2"] Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.379411 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/72b87ae1-6854-475b-a31d-976acc913623-scripts\") pod \"72b87ae1-6854-475b-a31d-976acc913623\" (UID: \"72b87ae1-6854-475b-a31d-976acc913623\") " Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.379460 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/72b87ae1-6854-475b-a31d-976acc913623-public-tls-certs\") pod \"72b87ae1-6854-475b-a31d-976acc913623\" (UID: \"72b87ae1-6854-475b-a31d-976acc913623\") " Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.379490 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/bb41ed7e-c96b-44dd-8c4e-ffb702de3f87-config-data\") pod \"bb41ed7e-c96b-44dd-8c4e-ffb702de3f87\" (UID: \"bb41ed7e-c96b-44dd-8c4e-ffb702de3f87\") " Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.379574 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/bb41ed7e-c96b-44dd-8c4e-ffb702de3f87-public-tls-certs\") pod \"bb41ed7e-c96b-44dd-8c4e-ffb702de3f87\" (UID: \"bb41ed7e-c96b-44dd-8c4e-ffb702de3f87\") " Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.379605 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/bb41ed7e-c96b-44dd-8c4e-ffb702de3f87-scripts\") pod \"bb41ed7e-c96b-44dd-8c4e-ffb702de3f87\" (UID: \"bb41ed7e-c96b-44dd-8c4e-ffb702de3f87\") " Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.379645 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/bb41ed7e-c96b-44dd-8c4e-ffb702de3f87-logs\") pod \"bb41ed7e-c96b-44dd-8c4e-ffb702de3f87\" (UID: \"bb41ed7e-c96b-44dd-8c4e-ffb702de3f87\") " Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.379696 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/72b87ae1-6854-475b-a31d-976acc913623-logs\") pod \"72b87ae1-6854-475b-a31d-976acc913623\" (UID: \"72b87ae1-6854-475b-a31d-976acc913623\") " Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.379735 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/bb41ed7e-c96b-44dd-8c4e-ffb702de3f87-config-data-custom\") pod \"bb41ed7e-c96b-44dd-8c4e-ffb702de3f87\" (UID: \"bb41ed7e-c96b-44dd-8c4e-ffb702de3f87\") " Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.380723 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-8f26f\" (UniqueName: \"kubernetes.io/projected/90814ea7-1abb-4760-bbfa-1190f23ab38a-kube-api-access-8f26f\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.380859 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/90814ea7-1abb-4760-bbfa-1190f23ab38a-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.380886 5052 reconciler_common.go:293] "Volume detached for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/72b87ae1-6854-475b-a31d-976acc913623-httpd-run\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.380901 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/90814ea7-1abb-4760-bbfa-1190f23ab38a-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.380963 5052 reconciler_common.go:293] "Volume detached for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/bb41ed7e-c96b-44dd-8c4e-ffb702de3f87-etc-machine-id\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.381408 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bb41ed7e-c96b-44dd-8c4e-ffb702de3f87-logs" (OuterVolumeSpecName: "logs") pod "bb41ed7e-c96b-44dd-8c4e-ffb702de3f87" (UID: "bb41ed7e-c96b-44dd-8c4e-ffb702de3f87"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.381894 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/72b87ae1-6854-475b-a31d-976acc913623-logs" (OuterVolumeSpecName: "logs") pod "72b87ae1-6854-475b-a31d-976acc913623" (UID: "72b87ae1-6854-475b-a31d-976acc913623"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.385758 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bb41ed7e-c96b-44dd-8c4e-ffb702de3f87-kube-api-access-kjtr9" (OuterVolumeSpecName: "kube-api-access-kjtr9") pod "bb41ed7e-c96b-44dd-8c4e-ffb702de3f87" (UID: "bb41ed7e-c96b-44dd-8c4e-ffb702de3f87"). InnerVolumeSpecName "kube-api-access-kjtr9". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.386170 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/72b87ae1-6854-475b-a31d-976acc913623-kube-api-access-dmhw4" (OuterVolumeSpecName: "kube-api-access-dmhw4") pod "72b87ae1-6854-475b-a31d-976acc913623" (UID: "72b87ae1-6854-475b-a31d-976acc913623"). InnerVolumeSpecName "kube-api-access-dmhw4". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.388416 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/openstack-cell1-galera-0"] Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.396131 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/openstack-cell1-galera-0"] Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.406647 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bb41ed7e-c96b-44dd-8c4e-ffb702de3f87-scripts" (OuterVolumeSpecName: "scripts") pod "bb41ed7e-c96b-44dd-8c4e-ffb702de3f87" (UID: "bb41ed7e-c96b-44dd-8c4e-ffb702de3f87"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.406671 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bb41ed7e-c96b-44dd-8c4e-ffb702de3f87-config-data-custom" (OuterVolumeSpecName: "config-data-custom") pod "bb41ed7e-c96b-44dd-8c4e-ffb702de3f87" (UID: "bb41ed7e-c96b-44dd-8c4e-ffb702de3f87"). InnerVolumeSpecName "config-data-custom". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.407425 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/72b87ae1-6854-475b-a31d-976acc913623-scripts" (OuterVolumeSpecName: "scripts") pod "72b87ae1-6854-475b-a31d-976acc913623" (UID: "72b87ae1-6854-475b-a31d-976acc913623"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.408715 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage09-crc" (OuterVolumeSpecName: "glance") pod "72b87ae1-6854-475b-a31d-976acc913623" (UID: "72b87ae1-6854-475b-a31d-976acc913623"). InnerVolumeSpecName "local-storage09-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.409440 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/openstack-galera-0"] Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.426804 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/openstack-galera-0"] Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.431417 5052 scope.go:117] "RemoveContainer" containerID="7f8633f82aa0da6af5e1f867996abc38506c65e6d6ca569fc68ad7d8ed0d1cf1" Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.460263 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/memcached-0"] Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.464112 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/memcached-0"] Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.470893 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.482723 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/af38eb0d-3924-4c4b-a22e-7e934bdcf172-public-tls-certs\") pod \"af38eb0d-3924-4c4b-a22e-7e934bdcf172\" (UID: \"af38eb0d-3924-4c4b-a22e-7e934bdcf172\") " Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.482830 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2d217a2f-3478-41e0-936a-6b1d01530007-combined-ca-bundle\") pod \"2d217a2f-3478-41e0-936a-6b1d01530007\" (UID: \"2d217a2f-3478-41e0-936a-6b1d01530007\") " Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.482879 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/af38eb0d-3924-4c4b-a22e-7e934bdcf172-logs\") pod \"af38eb0d-3924-4c4b-a22e-7e934bdcf172\" (UID: \"af38eb0d-3924-4c4b-a22e-7e934bdcf172\") " Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.482924 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-77z56\" (UniqueName: \"kubernetes.io/projected/2d217a2f-3478-41e0-936a-6b1d01530007-kube-api-access-77z56\") pod \"2d217a2f-3478-41e0-936a-6b1d01530007\" (UID: \"2d217a2f-3478-41e0-936a-6b1d01530007\") " Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.482968 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/2d217a2f-3478-41e0-936a-6b1d01530007-internal-tls-certs\") pod \"2d217a2f-3478-41e0-936a-6b1d01530007\" (UID: \"2d217a2f-3478-41e0-936a-6b1d01530007\") " Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.483003 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/2d217a2f-3478-41e0-936a-6b1d01530007-logs\") pod \"2d217a2f-3478-41e0-936a-6b1d01530007\" (UID: \"2d217a2f-3478-41e0-936a-6b1d01530007\") " Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.483067 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/af38eb0d-3924-4c4b-a22e-7e934bdcf172-internal-tls-certs\") pod \"af38eb0d-3924-4c4b-a22e-7e934bdcf172\" (UID: \"af38eb0d-3924-4c4b-a22e-7e934bdcf172\") " Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.483107 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/2d217a2f-3478-41e0-936a-6b1d01530007-httpd-run\") pod \"2d217a2f-3478-41e0-936a-6b1d01530007\" (UID: \"2d217a2f-3478-41e0-936a-6b1d01530007\") " Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.483131 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/af38eb0d-3924-4c4b-a22e-7e934bdcf172-config-data\") pod \"af38eb0d-3924-4c4b-a22e-7e934bdcf172\" (UID: \"af38eb0d-3924-4c4b-a22e-7e934bdcf172\") " Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.483291 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"glance\" (UniqueName: \"kubernetes.io/local-volume/local-storage17-crc\") pod \"2d217a2f-3478-41e0-936a-6b1d01530007\" (UID: \"2d217a2f-3478-41e0-936a-6b1d01530007\") " Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.483335 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/2d217a2f-3478-41e0-936a-6b1d01530007-config-data\") pod \"2d217a2f-3478-41e0-936a-6b1d01530007\" (UID: \"2d217a2f-3478-41e0-936a-6b1d01530007\") " Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.483361 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fqcvl\" (UniqueName: \"kubernetes.io/projected/af38eb0d-3924-4c4b-a22e-7e934bdcf172-kube-api-access-fqcvl\") pod \"af38eb0d-3924-4c4b-a22e-7e934bdcf172\" (UID: \"af38eb0d-3924-4c4b-a22e-7e934bdcf172\") " Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.483387 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/af38eb0d-3924-4c4b-a22e-7e934bdcf172-combined-ca-bundle\") pod \"af38eb0d-3924-4c4b-a22e-7e934bdcf172\" (UID: \"af38eb0d-3924-4c4b-a22e-7e934bdcf172\") " Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.483421 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/af38eb0d-3924-4c4b-a22e-7e934bdcf172-config-data-custom\") pod \"af38eb0d-3924-4c4b-a22e-7e934bdcf172\" (UID: \"af38eb0d-3924-4c4b-a22e-7e934bdcf172\") " Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.483448 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/2d217a2f-3478-41e0-936a-6b1d01530007-scripts\") pod \"2d217a2f-3478-41e0-936a-6b1d01530007\" (UID: \"2d217a2f-3478-41e0-936a-6b1d01530007\") " Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.483768 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2d217a2f-3478-41e0-936a-6b1d01530007-logs" (OuterVolumeSpecName: "logs") pod "2d217a2f-3478-41e0-936a-6b1d01530007" (UID: "2d217a2f-3478-41e0-936a-6b1d01530007"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.484100 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/72b87ae1-6854-475b-a31d-976acc913623-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.484118 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/bb41ed7e-c96b-44dd-8c4e-ffb702de3f87-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.484128 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/bb41ed7e-c96b-44dd-8c4e-ffb702de3f87-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.484137 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/72b87ae1-6854-475b-a31d-976acc913623-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.484147 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/bb41ed7e-c96b-44dd-8c4e-ffb702de3f87-config-data-custom\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.484159 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-dmhw4\" (UniqueName: \"kubernetes.io/projected/72b87ae1-6854-475b-a31d-976acc913623-kube-api-access-dmhw4\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.484168 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-kjtr9\" (UniqueName: \"kubernetes.io/projected/bb41ed7e-c96b-44dd-8c4e-ffb702de3f87-kube-api-access-kjtr9\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.484189 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage09-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage09-crc\") on node \"crc\" " Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.484199 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/2d217a2f-3478-41e0-936a-6b1d01530007-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.485797 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/af38eb0d-3924-4c4b-a22e-7e934bdcf172-logs" (OuterVolumeSpecName: "logs") pod "af38eb0d-3924-4c4b-a22e-7e934bdcf172" (UID: "af38eb0d-3924-4c4b-a22e-7e934bdcf172"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.486141 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2d217a2f-3478-41e0-936a-6b1d01530007-httpd-run" (OuterVolumeSpecName: "httpd-run") pod "2d217a2f-3478-41e0-936a-6b1d01530007" (UID: "2d217a2f-3478-41e0-936a-6b1d01530007"). InnerVolumeSpecName "httpd-run". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.487024 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.490182 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.495116 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2d217a2f-3478-41e0-936a-6b1d01530007-kube-api-access-77z56" (OuterVolumeSpecName: "kube-api-access-77z56") pod "2d217a2f-3478-41e0-936a-6b1d01530007" (UID: "2d217a2f-3478-41e0-936a-6b1d01530007"). InnerVolumeSpecName "kube-api-access-77z56". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.499467 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/af38eb0d-3924-4c4b-a22e-7e934bdcf172-kube-api-access-fqcvl" (OuterVolumeSpecName: "kube-api-access-fqcvl") pod "af38eb0d-3924-4c4b-a22e-7e934bdcf172" (UID: "af38eb0d-3924-4c4b-a22e-7e934bdcf172"). InnerVolumeSpecName "kube-api-access-fqcvl". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.500572 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage17-crc" (OuterVolumeSpecName: "glance") pod "2d217a2f-3478-41e0-936a-6b1d01530007" (UID: "2d217a2f-3478-41e0-936a-6b1d01530007"). InnerVolumeSpecName "local-storage17-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.500591 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2d217a2f-3478-41e0-936a-6b1d01530007-scripts" (OuterVolumeSpecName: "scripts") pod "2d217a2f-3478-41e0-936a-6b1d01530007" (UID: "2d217a2f-3478-41e0-936a-6b1d01530007"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.501683 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.505258 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/72b87ae1-6854-475b-a31d-976acc913623-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "72b87ae1-6854-475b-a31d-976acc913623" (UID: "72b87ae1-6854-475b-a31d-976acc913623"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.513694 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/af38eb0d-3924-4c4b-a22e-7e934bdcf172-config-data-custom" (OuterVolumeSpecName: "config-data-custom") pod "af38eb0d-3924-4c4b-a22e-7e934bdcf172" (UID: "af38eb0d-3924-4c4b-a22e-7e934bdcf172"). InnerVolumeSpecName "config-data-custom". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.523718 5052 scope.go:117] "RemoveContainer" containerID="72c6e09b72f57c6f1f19ff9c617f85acc4ceb1d16fb1a62383bab5dea1e10644" Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.570015 5052 scope.go:117] "RemoveContainer" containerID="4fdd95681555772f2043566122b6cc7d5d50f1b3bc46512f2b2e8fedf1c6d999" Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.585347 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/2d94bf00-7a64-4e90-a48c-b206ec54da6d-log-httpd\") pod \"2d94bf00-7a64-4e90-a48c-b206ec54da6d\" (UID: \"2d94bf00-7a64-4e90-a48c-b206ec54da6d\") " Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.585533 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2d94bf00-7a64-4e90-a48c-b206ec54da6d-combined-ca-bundle\") pod \"2d94bf00-7a64-4e90-a48c-b206ec54da6d\" (UID: \"2d94bf00-7a64-4e90-a48c-b206ec54da6d\") " Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.585564 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/2d94bf00-7a64-4e90-a48c-b206ec54da6d-run-httpd\") pod \"2d94bf00-7a64-4e90-a48c-b206ec54da6d\" (UID: \"2d94bf00-7a64-4e90-a48c-b206ec54da6d\") " Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.585615 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/2d94bf00-7a64-4e90-a48c-b206ec54da6d-sg-core-conf-yaml\") pod \"2d94bf00-7a64-4e90-a48c-b206ec54da6d\" (UID: \"2d94bf00-7a64-4e90-a48c-b206ec54da6d\") " Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.585668 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/2d94bf00-7a64-4e90-a48c-b206ec54da6d-ceilometer-tls-certs\") pod \"2d94bf00-7a64-4e90-a48c-b206ec54da6d\" (UID: \"2d94bf00-7a64-4e90-a48c-b206ec54da6d\") " Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.585694 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/2d94bf00-7a64-4e90-a48c-b206ec54da6d-config-data\") pod \"2d94bf00-7a64-4e90-a48c-b206ec54da6d\" (UID: \"2d94bf00-7a64-4e90-a48c-b206ec54da6d\") " Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.585713 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/2d94bf00-7a64-4e90-a48c-b206ec54da6d-scripts\") pod \"2d94bf00-7a64-4e90-a48c-b206ec54da6d\" (UID: \"2d94bf00-7a64-4e90-a48c-b206ec54da6d\") " Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.586032 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2d94bf00-7a64-4e90-a48c-b206ec54da6d-run-httpd" (OuterVolumeSpecName: "run-httpd") pod "2d94bf00-7a64-4e90-a48c-b206ec54da6d" (UID: "2d94bf00-7a64-4e90-a48c-b206ec54da6d"). InnerVolumeSpecName "run-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.586326 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2d94bf00-7a64-4e90-a48c-b206ec54da6d-log-httpd" (OuterVolumeSpecName: "log-httpd") pod "2d94bf00-7a64-4e90-a48c-b206ec54da6d" (UID: "2d94bf00-7a64-4e90-a48c-b206ec54da6d"). InnerVolumeSpecName "log-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.586925 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/af38eb0d-3924-4c4b-a22e-7e934bdcf172-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.586968 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/72b87ae1-6854-475b-a31d-976acc913623-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.586988 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-77z56\" (UniqueName: \"kubernetes.io/projected/2d217a2f-3478-41e0-936a-6b1d01530007-kube-api-access-77z56\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.587000 5052 reconciler_common.go:293] "Volume detached for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/2d217a2f-3478-41e0-936a-6b1d01530007-httpd-run\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.587014 5052 reconciler_common.go:293] "Volume detached for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/2d94bf00-7a64-4e90-a48c-b206ec54da6d-run-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.587042 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage17-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage17-crc\") on node \"crc\" " Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.587058 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-fqcvl\" (UniqueName: \"kubernetes.io/projected/af38eb0d-3924-4c4b-a22e-7e934bdcf172-kube-api-access-fqcvl\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.587071 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/af38eb0d-3924-4c4b-a22e-7e934bdcf172-config-data-custom\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.587083 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/2d217a2f-3478-41e0-936a-6b1d01530007-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.587095 5052 reconciler_common.go:293] "Volume detached for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/2d94bf00-7a64-4e90-a48c-b206ec54da6d-log-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.611155 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2d94bf00-7a64-4e90-a48c-b206ec54da6d-scripts" (OuterVolumeSpecName: "scripts") pod "2d94bf00-7a64-4e90-a48c-b206ec54da6d" (UID: "2d94bf00-7a64-4e90-a48c-b206ec54da6d"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.629173 5052 scope.go:117] "RemoveContainer" containerID="84022ca2f2789fb89a8aa147aadbca38260a93e0dfa5f25cefce8f679ca30454" Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.677230 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage09-crc" (UniqueName: "kubernetes.io/local-volume/local-storage09-crc") on node "crc" Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.687944 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a7907bc2-8e73-4272-b779-fd71d4acb1f4-combined-ca-bundle\") pod \"a7907bc2-8e73-4272-b779-fd71d4acb1f4\" (UID: \"a7907bc2-8e73-4272-b779-fd71d4acb1f4\") " Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.688020 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q9kzq\" (UniqueName: \"kubernetes.io/projected/2d94bf00-7a64-4e90-a48c-b206ec54da6d-kube-api-access-q9kzq\") pod \"2d94bf00-7a64-4e90-a48c-b206ec54da6d\" (UID: \"2d94bf00-7a64-4e90-a48c-b206ec54da6d\") " Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.688268 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kq22z\" (UniqueName: \"kubernetes.io/projected/a7907bc2-8e73-4272-b779-fd71d4acb1f4-kube-api-access-kq22z\") pod \"a7907bc2-8e73-4272-b779-fd71d4acb1f4\" (UID: \"a7907bc2-8e73-4272-b779-fd71d4acb1f4\") " Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.688300 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a7907bc2-8e73-4272-b779-fd71d4acb1f4-config-data\") pod \"a7907bc2-8e73-4272-b779-fd71d4acb1f4\" (UID: \"a7907bc2-8e73-4272-b779-fd71d4acb1f4\") " Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.688822 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage09-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage09-crc\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.688846 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/2d94bf00-7a64-4e90-a48c-b206ec54da6d-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.693231 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0988bafc-df97-4a53-8308-9e992be58d9a" path="/var/lib/kubelet/pods/0988bafc-df97-4a53-8308-9e992be58d9a/volumes" Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.693845 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0f00d4de-f49c-4155-98e7-00be8bc4ddc7" path="/var/lib/kubelet/pods/0f00d4de-f49c-4155-98e7-00be8bc4ddc7/volumes" Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.694339 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="47aa63e7-b7b2-4c1d-a73d-348b50cc5d2b" path="/var/lib/kubelet/pods/47aa63e7-b7b2-4c1d-a73d-348b50cc5d2b/volumes" Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.695811 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4a55b492-d57f-40dc-b719-1294fb24ef13" path="/var/lib/kubelet/pods/4a55b492-d57f-40dc-b719-1294fb24ef13/volumes" Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.696805 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7e109538-a2d2-43bb-8cef-bf99d7d0f1e9" path="/var/lib/kubelet/pods/7e109538-a2d2-43bb-8cef-bf99d7d0f1e9/volumes" Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.697357 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9165ef68-b76d-472e-99af-06cc8c8a2d3e" path="/var/lib/kubelet/pods/9165ef68-b76d-472e-99af-06cc8c8a2d3e/volumes" Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.698485 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9dec0a31-cba3-4c38-8b5d-4358f16b3704" path="/var/lib/kubelet/pods/9dec0a31-cba3-4c38-8b5d-4358f16b3704/volumes" Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.699164 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9e3a0650-c101-4bfa-8137-9909bf8ed4c7" path="/var/lib/kubelet/pods/9e3a0650-c101-4bfa-8137-9909bf8ed4c7/volumes" Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.699788 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="adcea736-eb15-4768-b12c-edb4c810beb3" path="/var/lib/kubelet/pods/adcea736-eb15-4768-b12c-edb4c810beb3/volumes" Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.701066 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b657a259-eb7a-48e8-8f41-2a3c077aff2a" path="/var/lib/kubelet/pods/b657a259-eb7a-48e8-8f41-2a3c077aff2a/volumes" Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.701998 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bfc242a2-aca4-458b-812e-018787bdbcab" path="/var/lib/kubelet/pods/bfc242a2-aca4-458b-812e-018787bdbcab/volumes" Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.702863 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e3a73fa0-68ef-4e03-81f9-2a143ed805bf" path="/var/lib/kubelet/pods/e3a73fa0-68ef-4e03-81f9-2a143ed805bf/volumes" Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.706579 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fb73cc9a-89cc-4cb8-87ce-2731e5dab0f7" path="/var/lib/kubelet/pods/fb73cc9a-89cc-4cb8-87ce-2731e5dab0f7/volumes" Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.707229 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2d94bf00-7a64-4e90-a48c-b206ec54da6d-kube-api-access-q9kzq" (OuterVolumeSpecName: "kube-api-access-q9kzq") pod "2d94bf00-7a64-4e90-a48c-b206ec54da6d" (UID: "2d94bf00-7a64-4e90-a48c-b206ec54da6d"). InnerVolumeSpecName "kube-api-access-q9kzq". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.708527 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a7907bc2-8e73-4272-b779-fd71d4acb1f4-kube-api-access-kq22z" (OuterVolumeSpecName: "kube-api-access-kq22z") pod "a7907bc2-8e73-4272-b779-fd71d4acb1f4" (UID: "a7907bc2-8e73-4272-b779-fd71d4acb1f4"). InnerVolumeSpecName "kube-api-access-kq22z". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.790231 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-kq22z\" (UniqueName: \"kubernetes.io/projected/a7907bc2-8e73-4272-b779-fd71d4acb1f4-kube-api-access-kq22z\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.790263 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-q9kzq\" (UniqueName: \"kubernetes.io/projected/2d94bf00-7a64-4e90-a48c-b206ec54da6d-kube-api-access-q9kzq\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.908049 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bb41ed7e-c96b-44dd-8c4e-ffb702de3f87-config-data" (OuterVolumeSpecName: "config-data") pod "bb41ed7e-c96b-44dd-8c4e-ffb702de3f87" (UID: "bb41ed7e-c96b-44dd-8c4e-ffb702de3f87"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.909667 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/af38eb0d-3924-4c4b-a22e-7e934bdcf172-config-data" (OuterVolumeSpecName: "config-data") pod "af38eb0d-3924-4c4b-a22e-7e934bdcf172" (UID: "af38eb0d-3924-4c4b-a22e-7e934bdcf172"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.914948 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/90814ea7-1abb-4760-bbfa-1190f23ab38a-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "90814ea7-1abb-4760-bbfa-1190f23ab38a" (UID: "90814ea7-1abb-4760-bbfa-1190f23ab38a"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.919440 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage17-crc" (UniqueName: "kubernetes.io/local-volume/local-storage17-crc") on node "crc" Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.934956 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bb41ed7e-c96b-44dd-8c4e-ffb702de3f87-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "bb41ed7e-c96b-44dd-8c4e-ffb702de3f87" (UID: "bb41ed7e-c96b-44dd-8c4e-ffb702de3f87"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.959575 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2d217a2f-3478-41e0-936a-6b1d01530007-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "2d217a2f-3478-41e0-936a-6b1d01530007" (UID: "2d217a2f-3478-41e0-936a-6b1d01530007"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.965942 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/72b87ae1-6854-475b-a31d-976acc913623-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "72b87ae1-6854-475b-a31d-976acc913623" (UID: "72b87ae1-6854-475b-a31d-976acc913623"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:47 crc kubenswrapper[5052]: I1203 12:21:47.977283 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bb41ed7e-c96b-44dd-8c4e-ffb702de3f87-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "bb41ed7e-c96b-44dd-8c4e-ffb702de3f87" (UID: "bb41ed7e-c96b-44dd-8c4e-ffb702de3f87"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:48 crc kubenswrapper[5052]: I1203 12:21:48.004091 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/72b87ae1-6854-475b-a31d-976acc913623-config-data" (OuterVolumeSpecName: "config-data") pod "72b87ae1-6854-475b-a31d-976acc913623" (UID: "72b87ae1-6854-475b-a31d-976acc913623"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:48 crc kubenswrapper[5052]: I1203 12:21:48.008871 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2d217a2f-3478-41e0-936a-6b1d01530007-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:48 crc kubenswrapper[5052]: I1203 12:21:48.008970 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/bb41ed7e-c96b-44dd-8c4e-ffb702de3f87-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:48 crc kubenswrapper[5052]: I1203 12:21:48.008988 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/72b87ae1-6854-475b-a31d-976acc913623-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:48 crc kubenswrapper[5052]: I1203 12:21:48.009001 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/72b87ae1-6854-475b-a31d-976acc913623-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:48 crc kubenswrapper[5052]: I1203 12:21:48.009013 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/af38eb0d-3924-4c4b-a22e-7e934bdcf172-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:48 crc kubenswrapper[5052]: I1203 12:21:48.009023 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/bb41ed7e-c96b-44dd-8c4e-ffb702de3f87-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:48 crc kubenswrapper[5052]: I1203 12:21:48.009035 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/90814ea7-1abb-4760-bbfa-1190f23ab38a-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:48 crc kubenswrapper[5052]: I1203 12:21:48.009047 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage17-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage17-crc\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:48 crc kubenswrapper[5052]: I1203 12:21:48.009061 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/bb41ed7e-c96b-44dd-8c4e-ffb702de3f87-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:48 crc kubenswrapper[5052]: I1203 12:21:48.017576 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/rabbitmq-server-0" podUID="c14a906c-f19d-4806-9110-8c47b38c5ae0" containerName="rabbitmq" probeResult="failure" output="dial tcp 10.217.1.171:5671: connect: connection refused" Dec 03 12:21:48 crc kubenswrapper[5052]: I1203 12:21:48.029329 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2d94bf00-7a64-4e90-a48c-b206ec54da6d-sg-core-conf-yaml" (OuterVolumeSpecName: "sg-core-conf-yaml") pod "2d94bf00-7a64-4e90-a48c-b206ec54da6d" (UID: "2d94bf00-7a64-4e90-a48c-b206ec54da6d"). InnerVolumeSpecName "sg-core-conf-yaml". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:48 crc kubenswrapper[5052]: I1203 12:21:48.041141 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/90814ea7-1abb-4760-bbfa-1190f23ab38a-config-data" (OuterVolumeSpecName: "config-data") pod "90814ea7-1abb-4760-bbfa-1190f23ab38a" (UID: "90814ea7-1abb-4760-bbfa-1190f23ab38a"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:48 crc kubenswrapper[5052]: I1203 12:21:48.044660 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2d217a2f-3478-41e0-936a-6b1d01530007-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "2d217a2f-3478-41e0-936a-6b1d01530007" (UID: "2d217a2f-3478-41e0-936a-6b1d01530007"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:48 crc kubenswrapper[5052]: I1203 12:21:48.045335 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a7907bc2-8e73-4272-b779-fd71d4acb1f4-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "a7907bc2-8e73-4272-b779-fd71d4acb1f4" (UID: "a7907bc2-8e73-4272-b779-fd71d4acb1f4"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:48 crc kubenswrapper[5052]: I1203 12:21:48.054658 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a7907bc2-8e73-4272-b779-fd71d4acb1f4-config-data" (OuterVolumeSpecName: "config-data") pod "a7907bc2-8e73-4272-b779-fd71d4acb1f4" (UID: "a7907bc2-8e73-4272-b779-fd71d4acb1f4"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:48 crc kubenswrapper[5052]: I1203 12:21:48.054760 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/af38eb0d-3924-4c4b-a22e-7e934bdcf172-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "af38eb0d-3924-4c4b-a22e-7e934bdcf172" (UID: "af38eb0d-3924-4c4b-a22e-7e934bdcf172"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:48 crc kubenswrapper[5052]: I1203 12:21:48.062997 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/af38eb0d-3924-4c4b-a22e-7e934bdcf172-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "af38eb0d-3924-4c4b-a22e-7e934bdcf172" (UID: "af38eb0d-3924-4c4b-a22e-7e934bdcf172"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:48 crc kubenswrapper[5052]: I1203 12:21:48.070858 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bb41ed7e-c96b-44dd-8c4e-ffb702de3f87-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "bb41ed7e-c96b-44dd-8c4e-ffb702de3f87" (UID: "bb41ed7e-c96b-44dd-8c4e-ffb702de3f87"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:48 crc kubenswrapper[5052]: I1203 12:21:48.083898 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2d217a2f-3478-41e0-936a-6b1d01530007-config-data" (OuterVolumeSpecName: "config-data") pod "2d217a2f-3478-41e0-936a-6b1d01530007" (UID: "2d217a2f-3478-41e0-936a-6b1d01530007"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:48 crc kubenswrapper[5052]: I1203 12:21:48.084521 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2d94bf00-7a64-4e90-a48c-b206ec54da6d-ceilometer-tls-certs" (OuterVolumeSpecName: "ceilometer-tls-certs") pod "2d94bf00-7a64-4e90-a48c-b206ec54da6d" (UID: "2d94bf00-7a64-4e90-a48c-b206ec54da6d"). InnerVolumeSpecName "ceilometer-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:48 crc kubenswrapper[5052]: I1203 12:21:48.086716 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/90814ea7-1abb-4760-bbfa-1190f23ab38a-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "90814ea7-1abb-4760-bbfa-1190f23ab38a" (UID: "90814ea7-1abb-4760-bbfa-1190f23ab38a"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:48 crc kubenswrapper[5052]: I1203 12:21:48.097020 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/af38eb0d-3924-4c4b-a22e-7e934bdcf172-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "af38eb0d-3924-4c4b-a22e-7e934bdcf172" (UID: "af38eb0d-3924-4c4b-a22e-7e934bdcf172"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:48 crc kubenswrapper[5052]: I1203 12:21:48.097034 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2d94bf00-7a64-4e90-a48c-b206ec54da6d-config-data" (OuterVolumeSpecName: "config-data") pod "2d94bf00-7a64-4e90-a48c-b206ec54da6d" (UID: "2d94bf00-7a64-4e90-a48c-b206ec54da6d"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:48 crc kubenswrapper[5052]: I1203 12:21:48.110452 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/af38eb0d-3924-4c4b-a22e-7e934bdcf172-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:48 crc kubenswrapper[5052]: I1203 12:21:48.110495 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/bb41ed7e-c96b-44dd-8c4e-ffb702de3f87-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:48 crc kubenswrapper[5052]: I1203 12:21:48.110522 5052 reconciler_common.go:293] "Volume detached for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/2d94bf00-7a64-4e90-a48c-b206ec54da6d-sg-core-conf-yaml\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:48 crc kubenswrapper[5052]: I1203 12:21:48.110535 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/90814ea7-1abb-4760-bbfa-1190f23ab38a-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:48 crc kubenswrapper[5052]: I1203 12:21:48.110547 5052 reconciler_common.go:293] "Volume detached for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/2d94bf00-7a64-4e90-a48c-b206ec54da6d-ceilometer-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:48 crc kubenswrapper[5052]: I1203 12:21:48.110560 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/2d94bf00-7a64-4e90-a48c-b206ec54da6d-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:48 crc kubenswrapper[5052]: I1203 12:21:48.110572 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/90814ea7-1abb-4760-bbfa-1190f23ab38a-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:48 crc kubenswrapper[5052]: I1203 12:21:48.110584 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/2d217a2f-3478-41e0-936a-6b1d01530007-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:48 crc kubenswrapper[5052]: I1203 12:21:48.110596 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/af38eb0d-3924-4c4b-a22e-7e934bdcf172-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:48 crc kubenswrapper[5052]: I1203 12:21:48.110609 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/af38eb0d-3924-4c4b-a22e-7e934bdcf172-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:48 crc kubenswrapper[5052]: I1203 12:21:48.110620 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a7907bc2-8e73-4272-b779-fd71d4acb1f4-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:48 crc kubenswrapper[5052]: I1203 12:21:48.110632 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a7907bc2-8e73-4272-b779-fd71d4acb1f4-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:48 crc kubenswrapper[5052]: I1203 12:21:48.110644 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/2d217a2f-3478-41e0-936a-6b1d01530007-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:48 crc kubenswrapper[5052]: E1203 12:21:48.110715 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/rabbitmq-cell1-config-data: configmap "rabbitmq-cell1-config-data" not found Dec 03 12:21:48 crc kubenswrapper[5052]: E1203 12:21:48.110777 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c279eaf1-2ee2-46fe-88be-9a5a65d4897e-config-data podName:c279eaf1-2ee2-46fe-88be-9a5a65d4897e nodeName:}" failed. No retries permitted until 2025-12-03 12:21:56.110759373 +0000 UTC m=+3706.699001137 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "config-data" (UniqueName: "kubernetes.io/configmap/c279eaf1-2ee2-46fe-88be-9a5a65d4897e-config-data") pod "rabbitmq-cell1-server-0" (UID: "c279eaf1-2ee2-46fe-88be-9a5a65d4897e") : configmap "rabbitmq-cell1-config-data" not found Dec 03 12:21:48 crc kubenswrapper[5052]: I1203 12:21:48.111816 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/90814ea7-1abb-4760-bbfa-1190f23ab38a-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "90814ea7-1abb-4760-bbfa-1190f23ab38a" (UID: "90814ea7-1abb-4760-bbfa-1190f23ab38a"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:48 crc kubenswrapper[5052]: I1203 12:21:48.119083 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2d94bf00-7a64-4e90-a48c-b206ec54da6d-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "2d94bf00-7a64-4e90-a48c-b206ec54da6d" (UID: "2d94bf00-7a64-4e90-a48c-b206ec54da6d"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:48 crc kubenswrapper[5052]: I1203 12:21:48.198789 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/cinder2067-account-delete-n2tf7" podUID="f4b213b4-e7bf-475f-8a2a-8c28437c32e0" containerName="mariadb-account-delete" containerID="cri-o://d897f2aa9cfac4051595e006bc6f7f2f03fa68c4d3c838fcc1749c0905fd2e2c" gracePeriod=30 Dec 03 12:21:48 crc kubenswrapper[5052]: I1203 12:21:48.206657 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/neutron734a-account-delete-sxpn9" podUID="38427938-9a58-4f07-9f57-061ee6693a0d" containerName="mariadb-account-delete" containerID="cri-o://2e5420fa5b52ae593a6a78576f5b291b1d9e69b7a745d45d21824b617230f090" gracePeriod=30 Dec 03 12:21:48 crc kubenswrapper[5052]: I1203 12:21:48.213865 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2d94bf00-7a64-4e90-a48c-b206ec54da6d-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:48 crc kubenswrapper[5052]: E1203 12:21:48.213978 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:21:48 crc kubenswrapper[5052]: E1203 12:21:48.214068 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/817345c8-3a42-4dea-8158-75973d9c41d9-operator-scripts podName:817345c8-3a42-4dea-8158-75973d9c41d9 nodeName:}" failed. No retries permitted until 2025-12-03 12:21:52.214049281 +0000 UTC m=+3702.802291045 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/817345c8-3a42-4dea-8158-75973d9c41d9-operator-scripts") pod "neutron-734a-account-create-update-qcgft" (UID: "817345c8-3a42-4dea-8158-75973d9c41d9") : configmap "openstack-scripts" not found Dec 03 12:21:48 crc kubenswrapper[5052]: I1203 12:21:48.213994 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/90814ea7-1abb-4760-bbfa-1190f23ab38a-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:48 crc kubenswrapper[5052]: I1203 12:21:48.215459 5052 generic.go:334] "Generic (PLEG): container finished" podID="540ee320-be53-46e0-90a3-079a5804124d" containerID="b80b6e8c4ad454ae227b95729ae24f6a9c106e69b758ed3c06b5fe2924584a0c" exitCode=0 Dec 03 12:21:48 crc kubenswrapper[5052]: I1203 12:21:48.229446 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/cinder2067-account-delete-n2tf7" podStartSLOduration=7.22942747 podStartE2EDuration="7.22942747s" podCreationTimestamp="2025-12-03 12:21:41 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:21:48.222483182 +0000 UTC m=+3698.810724956" watchObservedRunningTime="2025-12-03 12:21:48.22942747 +0000 UTC m=+3698.817669234" Dec 03 12:21:48 crc kubenswrapper[5052]: I1203 12:21:48.231733 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/novaapi8f5b-account-delete-86k67" podUID="bb65017d-6c06-4870-946b-80b9b220df36" containerName="mariadb-account-delete" containerID="cri-o://e22b78f35edac4a97928b1a8902e875b71b34760d326c1e0d71c57d0735bd8b6" gracePeriod=30 Dec 03 12:21:48 crc kubenswrapper[5052]: I1203 12:21:48.239813 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/barbican0e0e-account-delete-fvkr2" podUID="fc2a6873-8741-472a-864d-84651abd4c1c" containerName="mariadb-account-delete" containerID="cri-o://742fdf1f4f0dcabc5efaae2c5ad0dc3207f2a2b46274caf8b3413d161d2c0942" gracePeriod=30 Dec 03 12:21:48 crc kubenswrapper[5052]: I1203 12:21:48.245142 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/neutron734a-account-delete-sxpn9" podStartSLOduration=7.245124598 podStartE2EDuration="7.245124598s" podCreationTimestamp="2025-12-03 12:21:41 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:21:48.237259603 +0000 UTC m=+3698.825501367" watchObservedRunningTime="2025-12-03 12:21:48.245124598 +0000 UTC m=+3698.833366362" Dec 03 12:21:48 crc kubenswrapper[5052]: I1203 12:21:48.252966 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:21:48 crc kubenswrapper[5052]: I1203 12:21:48.263476 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-api-644f564b46-cjtkx" Dec 03 12:21:48 crc kubenswrapper[5052]: I1203 12:21:48.264902 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/novaapi8f5b-account-delete-86k67" podStartSLOduration=6.264884182 podStartE2EDuration="6.264884182s" podCreationTimestamp="2025-12-03 12:21:42 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:21:48.248238907 +0000 UTC m=+3698.836480671" watchObservedRunningTime="2025-12-03 12:21:48.264884182 +0000 UTC m=+3698.853125946" Dec 03 12:21:48 crc kubenswrapper[5052]: I1203 12:21:48.265625 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/barbican0e0e-account-delete-fvkr2" podStartSLOduration=7.265588392 podStartE2EDuration="7.265588392s" podCreationTimestamp="2025-12-03 12:21:41 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:21:48.262595807 +0000 UTC m=+3698.850837571" watchObservedRunningTime="2025-12-03 12:21:48.265588392 +0000 UTC m=+3698.853830156" Dec 03 12:21:48 crc kubenswrapper[5052]: I1203 12:21:48.267995 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder2067-account-delete-n2tf7" event={"ID":"f4b213b4-e7bf-475f-8a2a-8c28437c32e0","Type":"ContainerStarted","Data":"d897f2aa9cfac4051595e006bc6f7f2f03fa68c4d3c838fcc1749c0905fd2e2c"} Dec 03 12:21:48 crc kubenswrapper[5052]: I1203 12:21:48.268054 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder2067-account-delete-n2tf7" event={"ID":"f4b213b4-e7bf-475f-8a2a-8c28437c32e0","Type":"ContainerStarted","Data":"e23864fbc5f6730ba921194f728f7723b38a4e96fe4d99763788140f4fa13fd1"} Dec 03 12:21:48 crc kubenswrapper[5052]: I1203 12:21:48.268071 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/novacell00e20-account-delete-ktxhg" event={"ID":"4edc0513-31af-4aa4-b507-f1181c549be9","Type":"ContainerStarted","Data":"d1e1ec1a0605156f381e4a7e6d168dd2ec758e912d75200346b9ee5705cab0e0"} Dec 03 12:21:48 crc kubenswrapper[5052]: I1203 12:21:48.268084 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron734a-account-delete-sxpn9" event={"ID":"38427938-9a58-4f07-9f57-061ee6693a0d","Type":"ContainerStarted","Data":"2e5420fa5b52ae593a6a78576f5b291b1d9e69b7a745d45d21824b617230f090"} Dec 03 12:21:48 crc kubenswrapper[5052]: I1203 12:21:48.268093 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron734a-account-delete-sxpn9" event={"ID":"38427938-9a58-4f07-9f57-061ee6693a0d","Type":"ContainerStarted","Data":"430b11412ac94c2f1d55891a970f2e4f53e8276437ac26967b0689285cd56856"} Dec 03 12:21:48 crc kubenswrapper[5052]: I1203 12:21:48.268107 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-796c4bc57b-zsjsh" event={"ID":"540ee320-be53-46e0-90a3-079a5804124d","Type":"ContainerDied","Data":"b80b6e8c4ad454ae227b95729ae24f6a9c106e69b758ed3c06b5fe2924584a0c"} Dec 03 12:21:48 crc kubenswrapper[5052]: I1203 12:21:48.268122 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-796c4bc57b-zsjsh" event={"ID":"540ee320-be53-46e0-90a3-079a5804124d","Type":"ContainerDied","Data":"734b8cd29c4f2dc46d2026b98bc155fc68f0b7742c2c1a028e385b827e359fe2"} Dec 03 12:21:48 crc kubenswrapper[5052]: I1203 12:21:48.268132 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="734b8cd29c4f2dc46d2026b98bc155fc68f0b7742c2c1a028e385b827e359fe2" Dec 03 12:21:48 crc kubenswrapper[5052]: I1203 12:21:48.268141 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-7mq7s" event={"ID":"cdf67382-3145-4280-a5a5-41b0185cb07f","Type":"ContainerStarted","Data":"3f984b732b7d3c63737bf25b10c931347b0989361ec2884621e57852bcae454a"} Dec 03 12:21:48 crc kubenswrapper[5052]: I1203 12:21:48.268153 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/novaapi8f5b-account-delete-86k67" event={"ID":"bb65017d-6c06-4870-946b-80b9b220df36","Type":"ContainerStarted","Data":"e22b78f35edac4a97928b1a8902e875b71b34760d326c1e0d71c57d0735bd8b6"} Dec 03 12:21:48 crc kubenswrapper[5052]: I1203 12:21:48.268166 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/novaapi8f5b-account-delete-86k67" event={"ID":"bb65017d-6c06-4870-946b-80b9b220df36","Type":"ContainerStarted","Data":"1d24e0398e0fad5786bfe85a0daeebc30f86ef9a4fda453eb30c0cf2745d1ebe"} Dec 03 12:21:48 crc kubenswrapper[5052]: I1203 12:21:48.268178 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican0e0e-account-delete-fvkr2" event={"ID":"fc2a6873-8741-472a-864d-84651abd4c1c","Type":"ContainerStarted","Data":"742fdf1f4f0dcabc5efaae2c5ad0dc3207f2a2b46274caf8b3413d161d2c0942"} Dec 03 12:21:48 crc kubenswrapper[5052]: I1203 12:21:48.268190 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-8f5b-account-create-update-m9q79" event={"ID":"66954cf4-1c7b-4ca6-a6cf-b73bde8d670f","Type":"ContainerStarted","Data":"3a174f5ac8b79234788da4bd83a7272aa338a418da3aaf77385b4d9a22cec8a9"} Dec 03 12:21:48 crc kubenswrapper[5052]: I1203 12:21:48.268204 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"2d217a2f-3478-41e0-936a-6b1d01530007","Type":"ContainerDied","Data":"1f0abc5d306c36679cbe12f40d8c06b80fd913c661b1eb429a18cf30c0656ca5"} Dec 03 12:21:48 crc kubenswrapper[5052]: I1203 12:21:48.268221 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-644f564b46-cjtkx" event={"ID":"af38eb0d-3924-4c4b-a22e-7e934bdcf172","Type":"ContainerDied","Data":"f8398124a351883182a32783ea01603696a9bae2ae988e76ddf3651290914f19"} Dec 03 12:21:48 crc kubenswrapper[5052]: I1203 12:21:48.290224 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:21:48 crc kubenswrapper[5052]: I1203 12:21:48.290813 5052 scope.go:117] "RemoveContainer" containerID="37fd4ad16311c4c25724489aee98ab54bb95696bace9c0196b6418d1b39c27b1" Dec 03 12:21:48 crc kubenswrapper[5052]: I1203 12:21:48.291096 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"bb41ed7e-c96b-44dd-8c4e-ffb702de3f87","Type":"ContainerDied","Data":"e685a7366b76c078153cbfe61a3a3a16b3be6c0dffdc41b2a7513817c10b1140"} Dec 03 12:21:48 crc kubenswrapper[5052]: I1203 12:21:48.294426 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"72b87ae1-6854-475b-a31d-976acc913623","Type":"ContainerDied","Data":"c933b10a6bd214d4066d9a4b73c83f47e712f44c414c9cca8b30ee7c39c1ec9d"} Dec 03 12:21:48 crc kubenswrapper[5052]: I1203 12:21:48.294467 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:21:48 crc kubenswrapper[5052]: I1203 12:21:48.299247 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"2d94bf00-7a64-4e90-a48c-b206ec54da6d","Type":"ContainerDied","Data":"e15733fe2ecaff5b6cb0fc9bc23ab8123d3d773b2cda2b3b92b9af2f689a159d"} Dec 03 12:21:48 crc kubenswrapper[5052]: I1203 12:21:48.299496 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:21:48 crc kubenswrapper[5052]: I1203 12:21:48.302339 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-5447868864-mfdcg" Dec 03 12:21:48 crc kubenswrapper[5052]: I1203 12:21:48.302392 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-conductor-0" event={"ID":"a7907bc2-8e73-4272-b779-fd71d4acb1f4","Type":"ContainerDied","Data":"5c18996aaae6a85aceef22fa6d8699403c2e1d986214568e260b342877e4010d"} Dec 03 12:21:48 crc kubenswrapper[5052]: I1203 12:21:48.302359 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:21:48 crc kubenswrapper[5052]: E1203 12:21:48.319816 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:21:48 crc kubenswrapper[5052]: E1203 12:21:48.319894 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/db31774e-8f50-4862-86f3-93cb93f645a0-operator-scripts podName:db31774e-8f50-4862-86f3-93cb93f645a0 nodeName:}" failed. No retries permitted until 2025-12-03 12:21:52.319871771 +0000 UTC m=+3702.908113535 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/db31774e-8f50-4862-86f3-93cb93f645a0-operator-scripts") pod "glance73ca-account-delete-vpqhp" (UID: "db31774e-8f50-4862-86f3-93cb93f645a0") : configmap "openstack-scripts" not found Dec 03 12:21:48 crc kubenswrapper[5052]: I1203 12:21:48.366040 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-796c4bc57b-zsjsh" Dec 03 12:21:48 crc kubenswrapper[5052]: I1203 12:21:48.409707 5052 scope.go:117] "RemoveContainer" containerID="4b6df714a3fd21bc70f3269c776fc01e4b5e78532c5feda754673b2ff6304a5e" Dec 03 12:21:48 crc kubenswrapper[5052]: I1203 12:21:48.485771 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:21:48 crc kubenswrapper[5052]: I1203 12:21:48.487308 5052 scope.go:117] "RemoveContainer" containerID="8a1128866a1e3dbe3fe50f0d8c53319b6d90d894631644af6969f5f9926f2eaa" Dec 03 12:21:48 crc kubenswrapper[5052]: I1203 12:21:48.494402 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:21:48 crc kubenswrapper[5052]: I1203 12:21:48.508255 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:21:48 crc kubenswrapper[5052]: I1203 12:21:48.523663 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/540ee320-be53-46e0-90a3-079a5804124d-scripts\") pod \"540ee320-be53-46e0-90a3-079a5804124d\" (UID: \"540ee320-be53-46e0-90a3-079a5804124d\") " Dec 03 12:21:48 crc kubenswrapper[5052]: I1203 12:21:48.524043 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/540ee320-be53-46e0-90a3-079a5804124d-fernet-keys\") pod \"540ee320-be53-46e0-90a3-079a5804124d\" (UID: \"540ee320-be53-46e0-90a3-079a5804124d\") " Dec 03 12:21:48 crc kubenswrapper[5052]: I1203 12:21:48.524162 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/540ee320-be53-46e0-90a3-079a5804124d-config-data\") pod \"540ee320-be53-46e0-90a3-079a5804124d\" (UID: \"540ee320-be53-46e0-90a3-079a5804124d\") " Dec 03 12:21:48 crc kubenswrapper[5052]: I1203 12:21:48.524297 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/540ee320-be53-46e0-90a3-079a5804124d-combined-ca-bundle\") pod \"540ee320-be53-46e0-90a3-079a5804124d\" (UID: \"540ee320-be53-46e0-90a3-079a5804124d\") " Dec 03 12:21:48 crc kubenswrapper[5052]: I1203 12:21:48.524489 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/540ee320-be53-46e0-90a3-079a5804124d-public-tls-certs\") pod \"540ee320-be53-46e0-90a3-079a5804124d\" (UID: \"540ee320-be53-46e0-90a3-079a5804124d\") " Dec 03 12:21:48 crc kubenswrapper[5052]: I1203 12:21:48.524713 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dl5rg\" (UniqueName: \"kubernetes.io/projected/540ee320-be53-46e0-90a3-079a5804124d-kube-api-access-dl5rg\") pod \"540ee320-be53-46e0-90a3-079a5804124d\" (UID: \"540ee320-be53-46e0-90a3-079a5804124d\") " Dec 03 12:21:48 crc kubenswrapper[5052]: I1203 12:21:48.524878 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/540ee320-be53-46e0-90a3-079a5804124d-internal-tls-certs\") pod \"540ee320-be53-46e0-90a3-079a5804124d\" (UID: \"540ee320-be53-46e0-90a3-079a5804124d\") " Dec 03 12:21:48 crc kubenswrapper[5052]: I1203 12:21:48.525069 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/540ee320-be53-46e0-90a3-079a5804124d-credential-keys\") pod \"540ee320-be53-46e0-90a3-079a5804124d\" (UID: \"540ee320-be53-46e0-90a3-079a5804124d\") " Dec 03 12:21:48 crc kubenswrapper[5052]: I1203 12:21:48.525912 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:21:48 crc kubenswrapper[5052]: I1203 12:21:48.532895 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/540ee320-be53-46e0-90a3-079a5804124d-kube-api-access-dl5rg" (OuterVolumeSpecName: "kube-api-access-dl5rg") pod "540ee320-be53-46e0-90a3-079a5804124d" (UID: "540ee320-be53-46e0-90a3-079a5804124d"). InnerVolumeSpecName "kube-api-access-dl5rg". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:21:48 crc kubenswrapper[5052]: I1203 12:21:48.535475 5052 scope.go:117] "RemoveContainer" containerID="d9cc6d1bf947eaf3ffcfecf84ec2f0efe75eb932ba489b21a3991f6ce0828bc4" Dec 03 12:21:48 crc kubenswrapper[5052]: I1203 12:21:48.535949 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/540ee320-be53-46e0-90a3-079a5804124d-scripts" (OuterVolumeSpecName: "scripts") pod "540ee320-be53-46e0-90a3-079a5804124d" (UID: "540ee320-be53-46e0-90a3-079a5804124d"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:48 crc kubenswrapper[5052]: I1203 12:21:48.541294 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/placement-5447868864-mfdcg"] Dec 03 12:21:48 crc kubenswrapper[5052]: I1203 12:21:48.545157 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/540ee320-be53-46e0-90a3-079a5804124d-credential-keys" (OuterVolumeSpecName: "credential-keys") pod "540ee320-be53-46e0-90a3-079a5804124d" (UID: "540ee320-be53-46e0-90a3-079a5804124d"). InnerVolumeSpecName "credential-keys". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:48 crc kubenswrapper[5052]: I1203 12:21:48.548000 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/placement-5447868864-mfdcg"] Dec 03 12:21:48 crc kubenswrapper[5052]: I1203 12:21:48.560435 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-api-644f564b46-cjtkx"] Dec 03 12:21:48 crc kubenswrapper[5052]: I1203 12:21:48.560814 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/540ee320-be53-46e0-90a3-079a5804124d-fernet-keys" (OuterVolumeSpecName: "fernet-keys") pod "540ee320-be53-46e0-90a3-079a5804124d" (UID: "540ee320-be53-46e0-90a3-079a5804124d"). InnerVolumeSpecName "fernet-keys". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:48 crc kubenswrapper[5052]: I1203 12:21:48.560918 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/540ee320-be53-46e0-90a3-079a5804124d-config-data" (OuterVolumeSpecName: "config-data") pod "540ee320-be53-46e0-90a3-079a5804124d" (UID: "540ee320-be53-46e0-90a3-079a5804124d"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:48 crc kubenswrapper[5052]: I1203 12:21:48.566807 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/540ee320-be53-46e0-90a3-079a5804124d-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "540ee320-be53-46e0-90a3-079a5804124d" (UID: "540ee320-be53-46e0-90a3-079a5804124d"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:48 crc kubenswrapper[5052]: I1203 12:21:48.574206 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/barbican-api-644f564b46-cjtkx"] Dec 03 12:21:48 crc kubenswrapper[5052]: I1203 12:21:48.578872 5052 scope.go:117] "RemoveContainer" containerID="7eee2cae018afa667fc80fb6f8e55b05f28d5f2bfff1ccd7ad972013dda2c344" Dec 03 12:21:48 crc kubenswrapper[5052]: I1203 12:21:48.585109 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/540ee320-be53-46e0-90a3-079a5804124d-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "540ee320-be53-46e0-90a3-079a5804124d" (UID: "540ee320-be53-46e0-90a3-079a5804124d"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:48 crc kubenswrapper[5052]: I1203 12:21:48.595444 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/540ee320-be53-46e0-90a3-079a5804124d-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "540ee320-be53-46e0-90a3-079a5804124d" (UID: "540ee320-be53-46e0-90a3-079a5804124d"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:48 crc kubenswrapper[5052]: I1203 12:21:48.597582 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-8f5b-account-create-update-m9q79" Dec 03 12:21:48 crc kubenswrapper[5052]: I1203 12:21:48.613890 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:21:48 crc kubenswrapper[5052]: I1203 12:21:48.625806 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:21:48 crc kubenswrapper[5052]: I1203 12:21:48.627154 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-dl5rg\" (UniqueName: \"kubernetes.io/projected/540ee320-be53-46e0-90a3-079a5804124d-kube-api-access-dl5rg\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:48 crc kubenswrapper[5052]: I1203 12:21:48.627187 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/540ee320-be53-46e0-90a3-079a5804124d-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:48 crc kubenswrapper[5052]: I1203 12:21:48.627199 5052 reconciler_common.go:293] "Volume detached for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/540ee320-be53-46e0-90a3-079a5804124d-credential-keys\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:48 crc kubenswrapper[5052]: I1203 12:21:48.627208 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/540ee320-be53-46e0-90a3-079a5804124d-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:48 crc kubenswrapper[5052]: I1203 12:21:48.627217 5052 reconciler_common.go:293] "Volume detached for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/540ee320-be53-46e0-90a3-079a5804124d-fernet-keys\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:48 crc kubenswrapper[5052]: I1203 12:21:48.627227 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/540ee320-be53-46e0-90a3-079a5804124d-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:48 crc kubenswrapper[5052]: I1203 12:21:48.627235 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/540ee320-be53-46e0-90a3-079a5804124d-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:48 crc kubenswrapper[5052]: I1203 12:21:48.627245 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/540ee320-be53-46e0-90a3-079a5804124d-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:48 crc kubenswrapper[5052]: I1203 12:21:48.645992 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 12:21:48 crc kubenswrapper[5052]: I1203 12:21:48.651850 5052 scope.go:117] "RemoveContainer" containerID="c432832de2a3a0521fe28359f14c05384338a7557e1245553a8611f5a30ee883" Dec 03 12:21:48 crc kubenswrapper[5052]: I1203 12:21:48.657265 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 12:21:48 crc kubenswrapper[5052]: I1203 12:21:48.666766 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-0"] Dec 03 12:21:48 crc kubenswrapper[5052]: I1203 12:21:48.673183 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-0"] Dec 03 12:21:48 crc kubenswrapper[5052]: I1203 12:21:48.673916 5052 scope.go:117] "RemoveContainer" containerID="09b4a0072741f60499d24f40cc4ff3406d5d22671ced431363ab865172786300" Dec 03 12:21:48 crc kubenswrapper[5052]: E1203 12:21:48.674471 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 12:21:48 crc kubenswrapper[5052]: I1203 12:21:48.728951 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/66954cf4-1c7b-4ca6-a6cf-b73bde8d670f-operator-scripts\") pod \"66954cf4-1c7b-4ca6-a6cf-b73bde8d670f\" (UID: \"66954cf4-1c7b-4ca6-a6cf-b73bde8d670f\") " Dec 03 12:21:48 crc kubenswrapper[5052]: I1203 12:21:48.729024 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hdxhc\" (UniqueName: \"kubernetes.io/projected/66954cf4-1c7b-4ca6-a6cf-b73bde8d670f-kube-api-access-hdxhc\") pod \"66954cf4-1c7b-4ca6-a6cf-b73bde8d670f\" (UID: \"66954cf4-1c7b-4ca6-a6cf-b73bde8d670f\") " Dec 03 12:21:48 crc kubenswrapper[5052]: E1203 12:21:48.729498 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/rabbitmq-config-data: configmap "rabbitmq-config-data" not found Dec 03 12:21:48 crc kubenswrapper[5052]: E1203 12:21:48.729646 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c14a906c-f19d-4806-9110-8c47b38c5ae0-config-data podName:c14a906c-f19d-4806-9110-8c47b38c5ae0 nodeName:}" failed. No retries permitted until 2025-12-03 12:21:56.729623177 +0000 UTC m=+3707.317864941 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "config-data" (UniqueName: "kubernetes.io/configmap/c14a906c-f19d-4806-9110-8c47b38c5ae0-config-data") pod "rabbitmq-server-0" (UID: "c14a906c-f19d-4806-9110-8c47b38c5ae0") : configmap "rabbitmq-config-data" not found Dec 03 12:21:48 crc kubenswrapper[5052]: I1203 12:21:48.729779 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/66954cf4-1c7b-4ca6-a6cf-b73bde8d670f-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "66954cf4-1c7b-4ca6-a6cf-b73bde8d670f" (UID: "66954cf4-1c7b-4ca6-a6cf-b73bde8d670f"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:21:48 crc kubenswrapper[5052]: I1203 12:21:48.731307 5052 scope.go:117] "RemoveContainer" containerID="e124f853c8070429efcc9a6778d70b3a8c1c60fccd3f62bb22a54362a2148f4e" Dec 03 12:21:48 crc kubenswrapper[5052]: I1203 12:21:48.739043 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/66954cf4-1c7b-4ca6-a6cf-b73bde8d670f-kube-api-access-hdxhc" (OuterVolumeSpecName: "kube-api-access-hdxhc") pod "66954cf4-1c7b-4ca6-a6cf-b73bde8d670f" (UID: "66954cf4-1c7b-4ca6-a6cf-b73bde8d670f"). InnerVolumeSpecName "kube-api-access-hdxhc". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:21:48 crc kubenswrapper[5052]: I1203 12:21:48.752382 5052 scope.go:117] "RemoveContainer" containerID="0b9c5ec46decf17edaf9cf870fd66992b0a9f87a5bcb4a208561e9aa1a0d41be" Dec 03 12:21:48 crc kubenswrapper[5052]: E1203 12:21:48.789959 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="75bc26794aa58f8bf478527bc4377138449cad0841fd60348ba9d8b17243a1e4" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:21:48 crc kubenswrapper[5052]: E1203 12:21:48.791530 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="75bc26794aa58f8bf478527bc4377138449cad0841fd60348ba9d8b17243a1e4" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:21:48 crc kubenswrapper[5052]: E1203 12:21:48.792706 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="75bc26794aa58f8bf478527bc4377138449cad0841fd60348ba9d8b17243a1e4" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:21:48 crc kubenswrapper[5052]: E1203 12:21:48.792742 5052 prober.go:104] "Probe errored" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" probeType="Readiness" pod="openstack-kuttl-tests/nova-cell0-conductor-0" podUID="efe32106-ca1b-4d84-80a2-c0cddd3541b9" containerName="nova-cell0-conductor-conductor" Dec 03 12:21:48 crc kubenswrapper[5052]: I1203 12:21:48.797444 5052 scope.go:117] "RemoveContainer" containerID="e699b862f9953c2e25b19ab9d64356e9c8a7220759bf33ca05a10010b2639fc3" Dec 03 12:21:48 crc kubenswrapper[5052]: I1203 12:21:48.814574 5052 scope.go:117] "RemoveContainer" containerID="2a7af42a9086e0868a44334065634bf0f3e6cbd7e859d0ffd6a5df7b8b1b5555" Dec 03 12:21:48 crc kubenswrapper[5052]: I1203 12:21:48.831495 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/66954cf4-1c7b-4ca6-a6cf-b73bde8d670f-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:48 crc kubenswrapper[5052]: I1203 12:21:48.831544 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-hdxhc\" (UniqueName: \"kubernetes.io/projected/66954cf4-1c7b-4ca6-a6cf-b73bde8d670f-kube-api-access-hdxhc\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:48 crc kubenswrapper[5052]: I1203 12:21:48.838436 5052 scope.go:117] "RemoveContainer" containerID="19ad1a127fb47796fec06e903f60d68c562fce37f707be6c98c93f3e0d2d25cd" Dec 03 12:21:48 crc kubenswrapper[5052]: I1203 12:21:48.858582 5052 scope.go:117] "RemoveContainer" containerID="ace907d0e00f36ffb29d30e119e8630da96914e43d3d15c7fdb73b39b73ccfd8" Dec 03 12:21:48 crc kubenswrapper[5052]: I1203 12:21:48.884265 5052 scope.go:117] "RemoveContainer" containerID="dc3b9cb9559bad706ad6fd805545ec2569385c158301c0e04856678a08e6dfe0" Dec 03 12:21:48 crc kubenswrapper[5052]: I1203 12:21:48.903620 5052 scope.go:117] "RemoveContainer" containerID="3d06201ab438e62b35c336b3ff97f18ab1544e471098deb210f49f6607c474fe" Dec 03 12:21:48 crc kubenswrapper[5052]: I1203 12:21:48.921867 5052 scope.go:117] "RemoveContainer" containerID="7a966caf1545e10cd03293bfda99f2c9475e9dcbe187dcc16f09580784759143" Dec 03 12:21:48 crc kubenswrapper[5052]: I1203 12:21:48.943664 5052 scope.go:117] "RemoveContainer" containerID="e0441d5665d194615504d1f41084afce1caa468996e04b063d890632f490acca" Dec 03 12:21:48 crc kubenswrapper[5052]: I1203 12:21:48.969148 5052 scope.go:117] "RemoveContainer" containerID="cf05124bcf6f5c1b9dc38517547ce3e4342c12cea5c84ee53adeabf6d90e42cf" Dec 03 12:21:48 crc kubenswrapper[5052]: I1203 12:21:48.995877 5052 scope.go:117] "RemoveContainer" containerID="938a22611459fa789b24b03d2a21853b69071aad6d79f25384679595984908d8" Dec 03 12:21:49 crc kubenswrapper[5052]: I1203 12:21:49.318868 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-8f5b-account-create-update-m9q79" event={"ID":"66954cf4-1c7b-4ca6-a6cf-b73bde8d670f","Type":"ContainerDied","Data":"3a174f5ac8b79234788da4bd83a7272aa338a418da3aaf77385b4d9a22cec8a9"} Dec 03 12:21:49 crc kubenswrapper[5052]: I1203 12:21:49.319899 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-8f5b-account-create-update-m9q79" Dec 03 12:21:49 crc kubenswrapper[5052]: I1203 12:21:49.342658 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/novacell00e20-account-delete-ktxhg" event={"ID":"4edc0513-31af-4aa4-b507-f1181c549be9","Type":"ContainerStarted","Data":"f1f04f3f93da8795ec20a0de70a6a96be5287ed55eaae5bf34cb620a7aa557f3"} Dec 03 12:21:49 crc kubenswrapper[5052]: I1203 12:21:49.342667 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/novacell00e20-account-delete-ktxhg" podUID="4edc0513-31af-4aa4-b507-f1181c549be9" containerName="mariadb-account-delete" containerID="cri-o://f1f04f3f93da8795ec20a0de70a6a96be5287ed55eaae5bf34cb620a7aa557f3" gracePeriod=30 Dec 03 12:21:49 crc kubenswrapper[5052]: I1203 12:21:49.356959 5052 generic.go:334] "Generic (PLEG): container finished" podID="cdf67382-3145-4280-a5a5-41b0185cb07f" containerID="ecdb88788429aa4445200013cd41a7db181ff4aef99f48234f3114a461f621bd" exitCode=0 Dec 03 12:21:49 crc kubenswrapper[5052]: I1203 12:21:49.357277 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-796c4bc57b-zsjsh" Dec 03 12:21:49 crc kubenswrapper[5052]: I1203 12:21:49.357092 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-7mq7s" event={"ID":"cdf67382-3145-4280-a5a5-41b0185cb07f","Type":"ContainerDied","Data":"ecdb88788429aa4445200013cd41a7db181ff4aef99f48234f3114a461f621bd"} Dec 03 12:21:49 crc kubenswrapper[5052]: I1203 12:21:49.389940 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/novacell00e20-account-delete-ktxhg" podStartSLOduration=8.389907853 podStartE2EDuration="8.389907853s" podCreationTimestamp="2025-12-03 12:21:41 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:21:49.369907672 +0000 UTC m=+3699.958149436" watchObservedRunningTime="2025-12-03 12:21:49.389907853 +0000 UTC m=+3699.978149617" Dec 03 12:21:49 crc kubenswrapper[5052]: E1203 12:21:49.394215 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="97a957b8c33e1b7a5cef18f4383a4faea413910406298074a6052c04396cec18" cmd=["/usr/local/bin/container-scripts/status_check.sh"] Dec 03 12:21:49 crc kubenswrapper[5052]: E1203 12:21:49.403023 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="97a957b8c33e1b7a5cef18f4383a4faea413910406298074a6052c04396cec18" cmd=["/usr/local/bin/container-scripts/status_check.sh"] Dec 03 12:21:49 crc kubenswrapper[5052]: E1203 12:21:49.405328 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="97a957b8c33e1b7a5cef18f4383a4faea413910406298074a6052c04396cec18" cmd=["/usr/local/bin/container-scripts/status_check.sh"] Dec 03 12:21:49 crc kubenswrapper[5052]: E1203 12:21:49.405382 5052 prober.go:104] "Probe errored" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" probeType="Readiness" pod="openstack-kuttl-tests/ovn-northd-0" podUID="a77df94c-3e83-47a5-9b5f-eb0aeb41bc3d" containerName="ovn-northd" Dec 03 12:21:49 crc kubenswrapper[5052]: I1203 12:21:49.427637 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-api-8f5b-account-create-update-m9q79"] Dec 03 12:21:49 crc kubenswrapper[5052]: I1203 12:21:49.442600 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-api-8f5b-account-create-update-m9q79"] Dec 03 12:21:49 crc kubenswrapper[5052]: I1203 12:21:49.457046 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/keystone-796c4bc57b-zsjsh"] Dec 03 12:21:49 crc kubenswrapper[5052]: I1203 12:21:49.464251 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/keystone-796c4bc57b-zsjsh"] Dec 03 12:21:49 crc kubenswrapper[5052]: I1203 12:21:49.683751 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2d217a2f-3478-41e0-936a-6b1d01530007" path="/var/lib/kubelet/pods/2d217a2f-3478-41e0-936a-6b1d01530007/volumes" Dec 03 12:21:49 crc kubenswrapper[5052]: I1203 12:21:49.685146 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2d94bf00-7a64-4e90-a48c-b206ec54da6d" path="/var/lib/kubelet/pods/2d94bf00-7a64-4e90-a48c-b206ec54da6d/volumes" Dec 03 12:21:49 crc kubenswrapper[5052]: I1203 12:21:49.686419 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="540ee320-be53-46e0-90a3-079a5804124d" path="/var/lib/kubelet/pods/540ee320-be53-46e0-90a3-079a5804124d/volumes" Dec 03 12:21:49 crc kubenswrapper[5052]: I1203 12:21:49.686920 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="66954cf4-1c7b-4ca6-a6cf-b73bde8d670f" path="/var/lib/kubelet/pods/66954cf4-1c7b-4ca6-a6cf-b73bde8d670f/volumes" Dec 03 12:21:49 crc kubenswrapper[5052]: I1203 12:21:49.687315 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="72b87ae1-6854-475b-a31d-976acc913623" path="/var/lib/kubelet/pods/72b87ae1-6854-475b-a31d-976acc913623/volumes" Dec 03 12:21:49 crc kubenswrapper[5052]: I1203 12:21:49.688067 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="90814ea7-1abb-4760-bbfa-1190f23ab38a" path="/var/lib/kubelet/pods/90814ea7-1abb-4760-bbfa-1190f23ab38a/volumes" Dec 03 12:21:49 crc kubenswrapper[5052]: I1203 12:21:49.689050 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a7907bc2-8e73-4272-b779-fd71d4acb1f4" path="/var/lib/kubelet/pods/a7907bc2-8e73-4272-b779-fd71d4acb1f4/volumes" Dec 03 12:21:49 crc kubenswrapper[5052]: I1203 12:21:49.689568 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="af38eb0d-3924-4c4b-a22e-7e934bdcf172" path="/var/lib/kubelet/pods/af38eb0d-3924-4c4b-a22e-7e934bdcf172/volumes" Dec 03 12:21:49 crc kubenswrapper[5052]: I1203 12:21:49.690100 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bb41ed7e-c96b-44dd-8c4e-ffb702de3f87" path="/var/lib/kubelet/pods/bb41ed7e-c96b-44dd-8c4e-ffb702de3f87/volumes" Dec 03 12:21:49 crc kubenswrapper[5052]: I1203 12:21:49.924434 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:21:50 crc kubenswrapper[5052]: I1203 12:21:50.056079 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/c279eaf1-2ee2-46fe-88be-9a5a65d4897e-config-data\") pod \"c279eaf1-2ee2-46fe-88be-9a5a65d4897e\" (UID: \"c279eaf1-2ee2-46fe-88be-9a5a65d4897e\") " Dec 03 12:21:50 crc kubenswrapper[5052]: I1203 12:21:50.056137 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"rabbitmq-plugins\" (UniqueName: \"kubernetes.io/empty-dir/c279eaf1-2ee2-46fe-88be-9a5a65d4897e-rabbitmq-plugins\") pod \"c279eaf1-2ee2-46fe-88be-9a5a65d4897e\" (UID: \"c279eaf1-2ee2-46fe-88be-9a5a65d4897e\") " Dec 03 12:21:50 crc kubenswrapper[5052]: I1203 12:21:50.056160 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"rabbitmq-confd\" (UniqueName: \"kubernetes.io/projected/c279eaf1-2ee2-46fe-88be-9a5a65d4897e-rabbitmq-confd\") pod \"c279eaf1-2ee2-46fe-88be-9a5a65d4897e\" (UID: \"c279eaf1-2ee2-46fe-88be-9a5a65d4897e\") " Dec 03 12:21:50 crc kubenswrapper[5052]: I1203 12:21:50.056205 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"erlang-cookie-secret\" (UniqueName: \"kubernetes.io/secret/c279eaf1-2ee2-46fe-88be-9a5a65d4897e-erlang-cookie-secret\") pod \"c279eaf1-2ee2-46fe-88be-9a5a65d4897e\" (UID: \"c279eaf1-2ee2-46fe-88be-9a5a65d4897e\") " Dec 03 12:21:50 crc kubenswrapper[5052]: I1203 12:21:50.056228 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"persistence\" (UniqueName: \"kubernetes.io/local-volume/local-storage15-crc\") pod \"c279eaf1-2ee2-46fe-88be-9a5a65d4897e\" (UID: \"c279eaf1-2ee2-46fe-88be-9a5a65d4897e\") " Dec 03 12:21:50 crc kubenswrapper[5052]: I1203 12:21:50.056278 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"rabbitmq-erlang-cookie\" (UniqueName: \"kubernetes.io/empty-dir/c279eaf1-2ee2-46fe-88be-9a5a65d4897e-rabbitmq-erlang-cookie\") pod \"c279eaf1-2ee2-46fe-88be-9a5a65d4897e\" (UID: \"c279eaf1-2ee2-46fe-88be-9a5a65d4897e\") " Dec 03 12:21:50 crc kubenswrapper[5052]: I1203 12:21:50.056301 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q2n96\" (UniqueName: \"kubernetes.io/projected/c279eaf1-2ee2-46fe-88be-9a5a65d4897e-kube-api-access-q2n96\") pod \"c279eaf1-2ee2-46fe-88be-9a5a65d4897e\" (UID: \"c279eaf1-2ee2-46fe-88be-9a5a65d4897e\") " Dec 03 12:21:50 crc kubenswrapper[5052]: I1203 12:21:50.056340 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"rabbitmq-tls\" (UniqueName: \"kubernetes.io/projected/c279eaf1-2ee2-46fe-88be-9a5a65d4897e-rabbitmq-tls\") pod \"c279eaf1-2ee2-46fe-88be-9a5a65d4897e\" (UID: \"c279eaf1-2ee2-46fe-88be-9a5a65d4897e\") " Dec 03 12:21:50 crc kubenswrapper[5052]: I1203 12:21:50.056367 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"server-conf\" (UniqueName: \"kubernetes.io/configmap/c279eaf1-2ee2-46fe-88be-9a5a65d4897e-server-conf\") pod \"c279eaf1-2ee2-46fe-88be-9a5a65d4897e\" (UID: \"c279eaf1-2ee2-46fe-88be-9a5a65d4897e\") " Dec 03 12:21:50 crc kubenswrapper[5052]: I1203 12:21:50.056396 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"pod-info\" (UniqueName: \"kubernetes.io/downward-api/c279eaf1-2ee2-46fe-88be-9a5a65d4897e-pod-info\") pod \"c279eaf1-2ee2-46fe-88be-9a5a65d4897e\" (UID: \"c279eaf1-2ee2-46fe-88be-9a5a65d4897e\") " Dec 03 12:21:50 crc kubenswrapper[5052]: I1203 12:21:50.056421 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"plugins-conf\" (UniqueName: \"kubernetes.io/configmap/c279eaf1-2ee2-46fe-88be-9a5a65d4897e-plugins-conf\") pod \"c279eaf1-2ee2-46fe-88be-9a5a65d4897e\" (UID: \"c279eaf1-2ee2-46fe-88be-9a5a65d4897e\") " Dec 03 12:21:50 crc kubenswrapper[5052]: I1203 12:21:50.057090 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c279eaf1-2ee2-46fe-88be-9a5a65d4897e-rabbitmq-plugins" (OuterVolumeSpecName: "rabbitmq-plugins") pod "c279eaf1-2ee2-46fe-88be-9a5a65d4897e" (UID: "c279eaf1-2ee2-46fe-88be-9a5a65d4897e"). InnerVolumeSpecName "rabbitmq-plugins". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:21:50 crc kubenswrapper[5052]: I1203 12:21:50.057218 5052 reconciler_common.go:293] "Volume detached for volume \"rabbitmq-plugins\" (UniqueName: \"kubernetes.io/empty-dir/c279eaf1-2ee2-46fe-88be-9a5a65d4897e-rabbitmq-plugins\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:50 crc kubenswrapper[5052]: I1203 12:21:50.057229 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c279eaf1-2ee2-46fe-88be-9a5a65d4897e-plugins-conf" (OuterVolumeSpecName: "plugins-conf") pod "c279eaf1-2ee2-46fe-88be-9a5a65d4897e" (UID: "c279eaf1-2ee2-46fe-88be-9a5a65d4897e"). InnerVolumeSpecName "plugins-conf". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:21:50 crc kubenswrapper[5052]: I1203 12:21:50.061475 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c279eaf1-2ee2-46fe-88be-9a5a65d4897e-rabbitmq-erlang-cookie" (OuterVolumeSpecName: "rabbitmq-erlang-cookie") pod "c279eaf1-2ee2-46fe-88be-9a5a65d4897e" (UID: "c279eaf1-2ee2-46fe-88be-9a5a65d4897e"). InnerVolumeSpecName "rabbitmq-erlang-cookie". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:21:50 crc kubenswrapper[5052]: I1203 12:21:50.063194 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c279eaf1-2ee2-46fe-88be-9a5a65d4897e-erlang-cookie-secret" (OuterVolumeSpecName: "erlang-cookie-secret") pod "c279eaf1-2ee2-46fe-88be-9a5a65d4897e" (UID: "c279eaf1-2ee2-46fe-88be-9a5a65d4897e"). InnerVolumeSpecName "erlang-cookie-secret". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:50 crc kubenswrapper[5052]: I1203 12:21:50.064152 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c279eaf1-2ee2-46fe-88be-9a5a65d4897e-kube-api-access-q2n96" (OuterVolumeSpecName: "kube-api-access-q2n96") pod "c279eaf1-2ee2-46fe-88be-9a5a65d4897e" (UID: "c279eaf1-2ee2-46fe-88be-9a5a65d4897e"). InnerVolumeSpecName "kube-api-access-q2n96". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:21:50 crc kubenswrapper[5052]: I1203 12:21:50.067853 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c279eaf1-2ee2-46fe-88be-9a5a65d4897e-rabbitmq-tls" (OuterVolumeSpecName: "rabbitmq-tls") pod "c279eaf1-2ee2-46fe-88be-9a5a65d4897e" (UID: "c279eaf1-2ee2-46fe-88be-9a5a65d4897e"). InnerVolumeSpecName "rabbitmq-tls". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:21:50 crc kubenswrapper[5052]: I1203 12:21:50.067993 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c279eaf1-2ee2-46fe-88be-9a5a65d4897e-pod-info" (OuterVolumeSpecName: "pod-info") pod "c279eaf1-2ee2-46fe-88be-9a5a65d4897e" (UID: "c279eaf1-2ee2-46fe-88be-9a5a65d4897e"). InnerVolumeSpecName "pod-info". PluginName "kubernetes.io/downward-api", VolumeGidValue "" Dec 03 12:21:50 crc kubenswrapper[5052]: I1203 12:21:50.076275 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage15-crc" (OuterVolumeSpecName: "persistence") pod "c279eaf1-2ee2-46fe-88be-9a5a65d4897e" (UID: "c279eaf1-2ee2-46fe-88be-9a5a65d4897e"). InnerVolumeSpecName "local-storage15-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 12:21:50 crc kubenswrapper[5052]: I1203 12:21:50.082313 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c279eaf1-2ee2-46fe-88be-9a5a65d4897e-config-data" (OuterVolumeSpecName: "config-data") pod "c279eaf1-2ee2-46fe-88be-9a5a65d4897e" (UID: "c279eaf1-2ee2-46fe-88be-9a5a65d4897e"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:21:50 crc kubenswrapper[5052]: I1203 12:21:50.099057 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c279eaf1-2ee2-46fe-88be-9a5a65d4897e-server-conf" (OuterVolumeSpecName: "server-conf") pod "c279eaf1-2ee2-46fe-88be-9a5a65d4897e" (UID: "c279eaf1-2ee2-46fe-88be-9a5a65d4897e"). InnerVolumeSpecName "server-conf". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:21:50 crc kubenswrapper[5052]: I1203 12:21:50.143688 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_ovn-northd-0_a77df94c-3e83-47a5-9b5f-eb0aeb41bc3d/ovn-northd/0.log" Dec 03 12:21:50 crc kubenswrapper[5052]: I1203 12:21:50.143766 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:21:50 crc kubenswrapper[5052]: I1203 12:21:50.145651 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c279eaf1-2ee2-46fe-88be-9a5a65d4897e-rabbitmq-confd" (OuterVolumeSpecName: "rabbitmq-confd") pod "c279eaf1-2ee2-46fe-88be-9a5a65d4897e" (UID: "c279eaf1-2ee2-46fe-88be-9a5a65d4897e"). InnerVolumeSpecName "rabbitmq-confd". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:21:50 crc kubenswrapper[5052]: I1203 12:21:50.152292 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:21:50 crc kubenswrapper[5052]: I1203 12:21:50.165183 5052 reconciler_common.go:293] "Volume detached for volume \"rabbitmq-confd\" (UniqueName: \"kubernetes.io/projected/c279eaf1-2ee2-46fe-88be-9a5a65d4897e-rabbitmq-confd\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:50 crc kubenswrapper[5052]: I1203 12:21:50.165227 5052 reconciler_common.go:293] "Volume detached for volume \"erlang-cookie-secret\" (UniqueName: \"kubernetes.io/secret/c279eaf1-2ee2-46fe-88be-9a5a65d4897e-erlang-cookie-secret\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:50 crc kubenswrapper[5052]: I1203 12:21:50.165269 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage15-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage15-crc\") on node \"crc\" " Dec 03 12:21:50 crc kubenswrapper[5052]: I1203 12:21:50.165287 5052 reconciler_common.go:293] "Volume detached for volume \"rabbitmq-erlang-cookie\" (UniqueName: \"kubernetes.io/empty-dir/c279eaf1-2ee2-46fe-88be-9a5a65d4897e-rabbitmq-erlang-cookie\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:50 crc kubenswrapper[5052]: I1203 12:21:50.165302 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-q2n96\" (UniqueName: \"kubernetes.io/projected/c279eaf1-2ee2-46fe-88be-9a5a65d4897e-kube-api-access-q2n96\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:50 crc kubenswrapper[5052]: I1203 12:21:50.165314 5052 reconciler_common.go:293] "Volume detached for volume \"rabbitmq-tls\" (UniqueName: \"kubernetes.io/projected/c279eaf1-2ee2-46fe-88be-9a5a65d4897e-rabbitmq-tls\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:50 crc kubenswrapper[5052]: I1203 12:21:50.165326 5052 reconciler_common.go:293] "Volume detached for volume \"server-conf\" (UniqueName: \"kubernetes.io/configmap/c279eaf1-2ee2-46fe-88be-9a5a65d4897e-server-conf\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:50 crc kubenswrapper[5052]: I1203 12:21:50.165337 5052 reconciler_common.go:293] "Volume detached for volume \"pod-info\" (UniqueName: \"kubernetes.io/downward-api/c279eaf1-2ee2-46fe-88be-9a5a65d4897e-pod-info\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:50 crc kubenswrapper[5052]: I1203 12:21:50.165351 5052 reconciler_common.go:293] "Volume detached for volume \"plugins-conf\" (UniqueName: \"kubernetes.io/configmap/c279eaf1-2ee2-46fe-88be-9a5a65d4897e-plugins-conf\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:50 crc kubenswrapper[5052]: I1203 12:21:50.165363 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/c279eaf1-2ee2-46fe-88be-9a5a65d4897e-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:50 crc kubenswrapper[5052]: I1203 12:21:50.214435 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage15-crc" (UniqueName: "kubernetes.io/local-volume/local-storage15-crc") on node "crc" Dec 03 12:21:50 crc kubenswrapper[5052]: I1203 12:21:50.254967 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" podUID="9e3a0650-c101-4bfa-8137-9909bf8ed4c7" containerName="nova-cell1-novncproxy-novncproxy" probeResult="failure" output="Get \"https://10.217.1.165:6080/vnc_lite.html\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Dec 03 12:21:50 crc kubenswrapper[5052]: I1203 12:21:50.266633 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovn-rundir\" (UniqueName: \"kubernetes.io/empty-dir/a77df94c-3e83-47a5-9b5f-eb0aeb41bc3d-ovn-rundir\") pod \"a77df94c-3e83-47a5-9b5f-eb0aeb41bc3d\" (UID: \"a77df94c-3e83-47a5-9b5f-eb0aeb41bc3d\") " Dec 03 12:21:50 crc kubenswrapper[5052]: I1203 12:21:50.266771 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"persistence\" (UniqueName: \"kubernetes.io/local-volume/local-storage19-crc\") pod \"c14a906c-f19d-4806-9110-8c47b38c5ae0\" (UID: \"c14a906c-f19d-4806-9110-8c47b38c5ae0\") " Dec 03 12:21:50 crc kubenswrapper[5052]: I1203 12:21:50.266814 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"server-conf\" (UniqueName: \"kubernetes.io/configmap/c14a906c-f19d-4806-9110-8c47b38c5ae0-server-conf\") pod \"c14a906c-f19d-4806-9110-8c47b38c5ae0\" (UID: \"c14a906c-f19d-4806-9110-8c47b38c5ae0\") " Dec 03 12:21:50 crc kubenswrapper[5052]: I1203 12:21:50.266936 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/c14a906c-f19d-4806-9110-8c47b38c5ae0-config-data\") pod \"c14a906c-f19d-4806-9110-8c47b38c5ae0\" (UID: \"c14a906c-f19d-4806-9110-8c47b38c5ae0\") " Dec 03 12:21:50 crc kubenswrapper[5052]: I1203 12:21:50.266973 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"plugins-conf\" (UniqueName: \"kubernetes.io/configmap/c14a906c-f19d-4806-9110-8c47b38c5ae0-plugins-conf\") pod \"c14a906c-f19d-4806-9110-8c47b38c5ae0\" (UID: \"c14a906c-f19d-4806-9110-8c47b38c5ae0\") " Dec 03 12:21:50 crc kubenswrapper[5052]: I1203 12:21:50.267015 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"erlang-cookie-secret\" (UniqueName: \"kubernetes.io/secret/c14a906c-f19d-4806-9110-8c47b38c5ae0-erlang-cookie-secret\") pod \"c14a906c-f19d-4806-9110-8c47b38c5ae0\" (UID: \"c14a906c-f19d-4806-9110-8c47b38c5ae0\") " Dec 03 12:21:50 crc kubenswrapper[5052]: I1203 12:21:50.267056 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/a77df94c-3e83-47a5-9b5f-eb0aeb41bc3d-metrics-certs-tls-certs\") pod \"a77df94c-3e83-47a5-9b5f-eb0aeb41bc3d\" (UID: \"a77df94c-3e83-47a5-9b5f-eb0aeb41bc3d\") " Dec 03 12:21:50 crc kubenswrapper[5052]: I1203 12:21:50.267074 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a77df94c-3e83-47a5-9b5f-eb0aeb41bc3d-ovn-rundir" (OuterVolumeSpecName: "ovn-rundir") pod "a77df94c-3e83-47a5-9b5f-eb0aeb41bc3d" (UID: "a77df94c-3e83-47a5-9b5f-eb0aeb41bc3d"). InnerVolumeSpecName "ovn-rundir". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:21:50 crc kubenswrapper[5052]: I1203 12:21:50.267134 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a77df94c-3e83-47a5-9b5f-eb0aeb41bc3d-combined-ca-bundle\") pod \"a77df94c-3e83-47a5-9b5f-eb0aeb41bc3d\" (UID: \"a77df94c-3e83-47a5-9b5f-eb0aeb41bc3d\") " Dec 03 12:21:50 crc kubenswrapper[5052]: I1203 12:21:50.267166 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"rabbitmq-plugins\" (UniqueName: \"kubernetes.io/empty-dir/c14a906c-f19d-4806-9110-8c47b38c5ae0-rabbitmq-plugins\") pod \"c14a906c-f19d-4806-9110-8c47b38c5ae0\" (UID: \"c14a906c-f19d-4806-9110-8c47b38c5ae0\") " Dec 03 12:21:50 crc kubenswrapper[5052]: I1203 12:21:50.267190 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"rabbitmq-tls\" (UniqueName: \"kubernetes.io/projected/c14a906c-f19d-4806-9110-8c47b38c5ae0-rabbitmq-tls\") pod \"c14a906c-f19d-4806-9110-8c47b38c5ae0\" (UID: \"c14a906c-f19d-4806-9110-8c47b38c5ae0\") " Dec 03 12:21:50 crc kubenswrapper[5052]: I1203 12:21:50.267240 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qljvf\" (UniqueName: \"kubernetes.io/projected/c14a906c-f19d-4806-9110-8c47b38c5ae0-kube-api-access-qljvf\") pod \"c14a906c-f19d-4806-9110-8c47b38c5ae0\" (UID: \"c14a906c-f19d-4806-9110-8c47b38c5ae0\") " Dec 03 12:21:50 crc kubenswrapper[5052]: I1203 12:21:50.267266 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovn-northd-tls-certs\" (UniqueName: \"kubernetes.io/secret/a77df94c-3e83-47a5-9b5f-eb0aeb41bc3d-ovn-northd-tls-certs\") pod \"a77df94c-3e83-47a5-9b5f-eb0aeb41bc3d\" (UID: \"a77df94c-3e83-47a5-9b5f-eb0aeb41bc3d\") " Dec 03 12:21:50 crc kubenswrapper[5052]: I1203 12:21:50.267298 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a77df94c-3e83-47a5-9b5f-eb0aeb41bc3d-config\") pod \"a77df94c-3e83-47a5-9b5f-eb0aeb41bc3d\" (UID: \"a77df94c-3e83-47a5-9b5f-eb0aeb41bc3d\") " Dec 03 12:21:50 crc kubenswrapper[5052]: I1203 12:21:50.267345 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"rabbitmq-confd\" (UniqueName: \"kubernetes.io/projected/c14a906c-f19d-4806-9110-8c47b38c5ae0-rabbitmq-confd\") pod \"c14a906c-f19d-4806-9110-8c47b38c5ae0\" (UID: \"c14a906c-f19d-4806-9110-8c47b38c5ae0\") " Dec 03 12:21:50 crc kubenswrapper[5052]: I1203 12:21:50.267387 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tgrmq\" (UniqueName: \"kubernetes.io/projected/a77df94c-3e83-47a5-9b5f-eb0aeb41bc3d-kube-api-access-tgrmq\") pod \"a77df94c-3e83-47a5-9b5f-eb0aeb41bc3d\" (UID: \"a77df94c-3e83-47a5-9b5f-eb0aeb41bc3d\") " Dec 03 12:21:50 crc kubenswrapper[5052]: I1203 12:21:50.267409 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/a77df94c-3e83-47a5-9b5f-eb0aeb41bc3d-scripts\") pod \"a77df94c-3e83-47a5-9b5f-eb0aeb41bc3d\" (UID: \"a77df94c-3e83-47a5-9b5f-eb0aeb41bc3d\") " Dec 03 12:21:50 crc kubenswrapper[5052]: I1203 12:21:50.267442 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"pod-info\" (UniqueName: \"kubernetes.io/downward-api/c14a906c-f19d-4806-9110-8c47b38c5ae0-pod-info\") pod \"c14a906c-f19d-4806-9110-8c47b38c5ae0\" (UID: \"c14a906c-f19d-4806-9110-8c47b38c5ae0\") " Dec 03 12:21:50 crc kubenswrapper[5052]: I1203 12:21:50.267487 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"rabbitmq-erlang-cookie\" (UniqueName: \"kubernetes.io/empty-dir/c14a906c-f19d-4806-9110-8c47b38c5ae0-rabbitmq-erlang-cookie\") pod \"c14a906c-f19d-4806-9110-8c47b38c5ae0\" (UID: \"c14a906c-f19d-4806-9110-8c47b38c5ae0\") " Dec 03 12:21:50 crc kubenswrapper[5052]: I1203 12:21:50.267963 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage15-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage15-crc\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:50 crc kubenswrapper[5052]: I1203 12:21:50.267989 5052 reconciler_common.go:293] "Volume detached for volume \"ovn-rundir\" (UniqueName: \"kubernetes.io/empty-dir/a77df94c-3e83-47a5-9b5f-eb0aeb41bc3d-ovn-rundir\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:50 crc kubenswrapper[5052]: I1203 12:21:50.268329 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a77df94c-3e83-47a5-9b5f-eb0aeb41bc3d-config" (OuterVolumeSpecName: "config") pod "a77df94c-3e83-47a5-9b5f-eb0aeb41bc3d" (UID: "a77df94c-3e83-47a5-9b5f-eb0aeb41bc3d"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:21:50 crc kubenswrapper[5052]: I1203 12:21:50.268665 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c14a906c-f19d-4806-9110-8c47b38c5ae0-rabbitmq-erlang-cookie" (OuterVolumeSpecName: "rabbitmq-erlang-cookie") pod "c14a906c-f19d-4806-9110-8c47b38c5ae0" (UID: "c14a906c-f19d-4806-9110-8c47b38c5ae0"). InnerVolumeSpecName "rabbitmq-erlang-cookie". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:21:50 crc kubenswrapper[5052]: I1203 12:21:50.269005 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c14a906c-f19d-4806-9110-8c47b38c5ae0-rabbitmq-plugins" (OuterVolumeSpecName: "rabbitmq-plugins") pod "c14a906c-f19d-4806-9110-8c47b38c5ae0" (UID: "c14a906c-f19d-4806-9110-8c47b38c5ae0"). InnerVolumeSpecName "rabbitmq-plugins". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:21:50 crc kubenswrapper[5052]: I1203 12:21:50.269108 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c14a906c-f19d-4806-9110-8c47b38c5ae0-plugins-conf" (OuterVolumeSpecName: "plugins-conf") pod "c14a906c-f19d-4806-9110-8c47b38c5ae0" (UID: "c14a906c-f19d-4806-9110-8c47b38c5ae0"). InnerVolumeSpecName "plugins-conf". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:21:50 crc kubenswrapper[5052]: I1203 12:21:50.269423 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a77df94c-3e83-47a5-9b5f-eb0aeb41bc3d-scripts" (OuterVolumeSpecName: "scripts") pod "a77df94c-3e83-47a5-9b5f-eb0aeb41bc3d" (UID: "a77df94c-3e83-47a5-9b5f-eb0aeb41bc3d"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:21:50 crc kubenswrapper[5052]: I1203 12:21:50.269885 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c14a906c-f19d-4806-9110-8c47b38c5ae0-erlang-cookie-secret" (OuterVolumeSpecName: "erlang-cookie-secret") pod "c14a906c-f19d-4806-9110-8c47b38c5ae0" (UID: "c14a906c-f19d-4806-9110-8c47b38c5ae0"). InnerVolumeSpecName "erlang-cookie-secret". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:50 crc kubenswrapper[5052]: I1203 12:21:50.270969 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c14a906c-f19d-4806-9110-8c47b38c5ae0-rabbitmq-tls" (OuterVolumeSpecName: "rabbitmq-tls") pod "c14a906c-f19d-4806-9110-8c47b38c5ae0" (UID: "c14a906c-f19d-4806-9110-8c47b38c5ae0"). InnerVolumeSpecName "rabbitmq-tls". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:21:50 crc kubenswrapper[5052]: I1203 12:21:50.272366 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c14a906c-f19d-4806-9110-8c47b38c5ae0-kube-api-access-qljvf" (OuterVolumeSpecName: "kube-api-access-qljvf") pod "c14a906c-f19d-4806-9110-8c47b38c5ae0" (UID: "c14a906c-f19d-4806-9110-8c47b38c5ae0"). InnerVolumeSpecName "kube-api-access-qljvf". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:21:50 crc kubenswrapper[5052]: I1203 12:21:50.272478 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage19-crc" (OuterVolumeSpecName: "persistence") pod "c14a906c-f19d-4806-9110-8c47b38c5ae0" (UID: "c14a906c-f19d-4806-9110-8c47b38c5ae0"). InnerVolumeSpecName "local-storage19-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 12:21:50 crc kubenswrapper[5052]: I1203 12:21:50.272595 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a77df94c-3e83-47a5-9b5f-eb0aeb41bc3d-kube-api-access-tgrmq" (OuterVolumeSpecName: "kube-api-access-tgrmq") pod "a77df94c-3e83-47a5-9b5f-eb0aeb41bc3d" (UID: "a77df94c-3e83-47a5-9b5f-eb0aeb41bc3d"). InnerVolumeSpecName "kube-api-access-tgrmq". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:21:50 crc kubenswrapper[5052]: I1203 12:21:50.274386 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c14a906c-f19d-4806-9110-8c47b38c5ae0-pod-info" (OuterVolumeSpecName: "pod-info") pod "c14a906c-f19d-4806-9110-8c47b38c5ae0" (UID: "c14a906c-f19d-4806-9110-8c47b38c5ae0"). InnerVolumeSpecName "pod-info". PluginName "kubernetes.io/downward-api", VolumeGidValue "" Dec 03 12:21:50 crc kubenswrapper[5052]: I1203 12:21:50.287528 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c14a906c-f19d-4806-9110-8c47b38c5ae0-config-data" (OuterVolumeSpecName: "config-data") pod "c14a906c-f19d-4806-9110-8c47b38c5ae0" (UID: "c14a906c-f19d-4806-9110-8c47b38c5ae0"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:21:50 crc kubenswrapper[5052]: I1203 12:21:50.291198 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a77df94c-3e83-47a5-9b5f-eb0aeb41bc3d-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "a77df94c-3e83-47a5-9b5f-eb0aeb41bc3d" (UID: "a77df94c-3e83-47a5-9b5f-eb0aeb41bc3d"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:50 crc kubenswrapper[5052]: I1203 12:21:50.319425 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c14a906c-f19d-4806-9110-8c47b38c5ae0-server-conf" (OuterVolumeSpecName: "server-conf") pod "c14a906c-f19d-4806-9110-8c47b38c5ae0" (UID: "c14a906c-f19d-4806-9110-8c47b38c5ae0"). InnerVolumeSpecName "server-conf". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:21:50 crc kubenswrapper[5052]: I1203 12:21:50.340968 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a77df94c-3e83-47a5-9b5f-eb0aeb41bc3d-ovn-northd-tls-certs" (OuterVolumeSpecName: "ovn-northd-tls-certs") pod "a77df94c-3e83-47a5-9b5f-eb0aeb41bc3d" (UID: "a77df94c-3e83-47a5-9b5f-eb0aeb41bc3d"). InnerVolumeSpecName "ovn-northd-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:50 crc kubenswrapper[5052]: I1203 12:21:50.342438 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a77df94c-3e83-47a5-9b5f-eb0aeb41bc3d-metrics-certs-tls-certs" (OuterVolumeSpecName: "metrics-certs-tls-certs") pod "a77df94c-3e83-47a5-9b5f-eb0aeb41bc3d" (UID: "a77df94c-3e83-47a5-9b5f-eb0aeb41bc3d"). InnerVolumeSpecName "metrics-certs-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:50 crc kubenswrapper[5052]: I1203 12:21:50.354290 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c14a906c-f19d-4806-9110-8c47b38c5ae0-rabbitmq-confd" (OuterVolumeSpecName: "rabbitmq-confd") pod "c14a906c-f19d-4806-9110-8c47b38c5ae0" (UID: "c14a906c-f19d-4806-9110-8c47b38c5ae0"). InnerVolumeSpecName "rabbitmq-confd". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:21:50 crc kubenswrapper[5052]: I1203 12:21:50.370143 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a77df94c-3e83-47a5-9b5f-eb0aeb41bc3d-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:50 crc kubenswrapper[5052]: I1203 12:21:50.370186 5052 reconciler_common.go:293] "Volume detached for volume \"rabbitmq-plugins\" (UniqueName: \"kubernetes.io/empty-dir/c14a906c-f19d-4806-9110-8c47b38c5ae0-rabbitmq-plugins\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:50 crc kubenswrapper[5052]: I1203 12:21:50.370199 5052 reconciler_common.go:293] "Volume detached for volume \"rabbitmq-tls\" (UniqueName: \"kubernetes.io/projected/c14a906c-f19d-4806-9110-8c47b38c5ae0-rabbitmq-tls\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:50 crc kubenswrapper[5052]: I1203 12:21:50.370211 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-qljvf\" (UniqueName: \"kubernetes.io/projected/c14a906c-f19d-4806-9110-8c47b38c5ae0-kube-api-access-qljvf\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:50 crc kubenswrapper[5052]: I1203 12:21:50.370226 5052 reconciler_common.go:293] "Volume detached for volume \"ovn-northd-tls-certs\" (UniqueName: \"kubernetes.io/secret/a77df94c-3e83-47a5-9b5f-eb0aeb41bc3d-ovn-northd-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:50 crc kubenswrapper[5052]: I1203 12:21:50.370237 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a77df94c-3e83-47a5-9b5f-eb0aeb41bc3d-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:50 crc kubenswrapper[5052]: I1203 12:21:50.370248 5052 reconciler_common.go:293] "Volume detached for volume \"rabbitmq-confd\" (UniqueName: \"kubernetes.io/projected/c14a906c-f19d-4806-9110-8c47b38c5ae0-rabbitmq-confd\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:50 crc kubenswrapper[5052]: I1203 12:21:50.370260 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-tgrmq\" (UniqueName: \"kubernetes.io/projected/a77df94c-3e83-47a5-9b5f-eb0aeb41bc3d-kube-api-access-tgrmq\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:50 crc kubenswrapper[5052]: I1203 12:21:50.370271 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/a77df94c-3e83-47a5-9b5f-eb0aeb41bc3d-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:50 crc kubenswrapper[5052]: I1203 12:21:50.370282 5052 reconciler_common.go:293] "Volume detached for volume \"pod-info\" (UniqueName: \"kubernetes.io/downward-api/c14a906c-f19d-4806-9110-8c47b38c5ae0-pod-info\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:50 crc kubenswrapper[5052]: I1203 12:21:50.370295 5052 reconciler_common.go:293] "Volume detached for volume \"rabbitmq-erlang-cookie\" (UniqueName: \"kubernetes.io/empty-dir/c14a906c-f19d-4806-9110-8c47b38c5ae0-rabbitmq-erlang-cookie\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:50 crc kubenswrapper[5052]: I1203 12:21:50.370332 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage19-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage19-crc\") on node \"crc\" " Dec 03 12:21:50 crc kubenswrapper[5052]: I1203 12:21:50.370343 5052 reconciler_common.go:293] "Volume detached for volume \"server-conf\" (UniqueName: \"kubernetes.io/configmap/c14a906c-f19d-4806-9110-8c47b38c5ae0-server-conf\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:50 crc kubenswrapper[5052]: I1203 12:21:50.370355 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/c14a906c-f19d-4806-9110-8c47b38c5ae0-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:50 crc kubenswrapper[5052]: I1203 12:21:50.370365 5052 reconciler_common.go:293] "Volume detached for volume \"plugins-conf\" (UniqueName: \"kubernetes.io/configmap/c14a906c-f19d-4806-9110-8c47b38c5ae0-plugins-conf\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:50 crc kubenswrapper[5052]: I1203 12:21:50.370375 5052 reconciler_common.go:293] "Volume detached for volume \"erlang-cookie-secret\" (UniqueName: \"kubernetes.io/secret/c14a906c-f19d-4806-9110-8c47b38c5ae0-erlang-cookie-secret\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:50 crc kubenswrapper[5052]: I1203 12:21:50.370386 5052 reconciler_common.go:293] "Volume detached for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/a77df94c-3e83-47a5-9b5f-eb0aeb41bc3d-metrics-certs-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:50 crc kubenswrapper[5052]: I1203 12:21:50.379054 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-7mq7s" event={"ID":"cdf67382-3145-4280-a5a5-41b0185cb07f","Type":"ContainerStarted","Data":"302fc92c792a1400570841f42f510118b4e77899b5e6948147bf0cee46337588"} Dec 03 12:21:50 crc kubenswrapper[5052]: I1203 12:21:50.379489 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-7mq7s" Dec 03 12:21:50 crc kubenswrapper[5052]: I1203 12:21:50.383391 5052 generic.go:334] "Generic (PLEG): container finished" podID="c279eaf1-2ee2-46fe-88be-9a5a65d4897e" containerID="9e557ebf23292119d2650d0f7e9abf49644bff9d991b5bf5077147a45b27b74b" exitCode=0 Dec 03 12:21:50 crc kubenswrapper[5052]: I1203 12:21:50.383439 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" event={"ID":"c279eaf1-2ee2-46fe-88be-9a5a65d4897e","Type":"ContainerDied","Data":"9e557ebf23292119d2650d0f7e9abf49644bff9d991b5bf5077147a45b27b74b"} Dec 03 12:21:50 crc kubenswrapper[5052]: I1203 12:21:50.383456 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" event={"ID":"c279eaf1-2ee2-46fe-88be-9a5a65d4897e","Type":"ContainerDied","Data":"aa080ded14dd4e75e83d60cd422e78ffb3eaef58b0f54feabd6bb11cbe1dde7a"} Dec 03 12:21:50 crc kubenswrapper[5052]: I1203 12:21:50.383470 5052 scope.go:117] "RemoveContainer" containerID="9e557ebf23292119d2650d0f7e9abf49644bff9d991b5bf5077147a45b27b74b" Dec 03 12:21:50 crc kubenswrapper[5052]: I1203 12:21:50.383602 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:21:50 crc kubenswrapper[5052]: I1203 12:21:50.388946 5052 generic.go:334] "Generic (PLEG): container finished" podID="c14a906c-f19d-4806-9110-8c47b38c5ae0" containerID="9217efd68f2e5602eadef89a260d023bd85316f53943b906f69ea6a06b762ecd" exitCode=0 Dec 03 12:21:50 crc kubenswrapper[5052]: I1203 12:21:50.389010 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-server-0" event={"ID":"c14a906c-f19d-4806-9110-8c47b38c5ae0","Type":"ContainerDied","Data":"9217efd68f2e5602eadef89a260d023bd85316f53943b906f69ea6a06b762ecd"} Dec 03 12:21:50 crc kubenswrapper[5052]: I1203 12:21:50.389057 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-server-0" event={"ID":"c14a906c-f19d-4806-9110-8c47b38c5ae0","Type":"ContainerDied","Data":"b21e77f593b12d889c25fb56626d0100aa2f2aaed9d36690637f8aa4e39de474"} Dec 03 12:21:50 crc kubenswrapper[5052]: I1203 12:21:50.389143 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:21:50 crc kubenswrapper[5052]: I1203 12:21:50.393458 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_ovn-northd-0_a77df94c-3e83-47a5-9b5f-eb0aeb41bc3d/ovn-northd/0.log" Dec 03 12:21:50 crc kubenswrapper[5052]: I1203 12:21:50.393526 5052 generic.go:334] "Generic (PLEG): container finished" podID="a77df94c-3e83-47a5-9b5f-eb0aeb41bc3d" containerID="97a957b8c33e1b7a5cef18f4383a4faea413910406298074a6052c04396cec18" exitCode=139 Dec 03 12:21:50 crc kubenswrapper[5052]: I1203 12:21:50.393556 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovn-northd-0" event={"ID":"a77df94c-3e83-47a5-9b5f-eb0aeb41bc3d","Type":"ContainerDied","Data":"97a957b8c33e1b7a5cef18f4383a4faea413910406298074a6052c04396cec18"} Dec 03 12:21:50 crc kubenswrapper[5052]: I1203 12:21:50.393582 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovn-northd-0" event={"ID":"a77df94c-3e83-47a5-9b5f-eb0aeb41bc3d","Type":"ContainerDied","Data":"c5318b2d6cbe69bd088c706a2833c45418c4ceb09d5906c6f5dc5ce8834b0109"} Dec 03 12:21:50 crc kubenswrapper[5052]: I1203 12:21:50.393655 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:21:50 crc kubenswrapper[5052]: I1203 12:21:50.409074 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-7mq7s" podStartSLOduration=9.409055601 podStartE2EDuration="9.409055601s" podCreationTimestamp="2025-12-03 12:21:41 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:21:50.398323395 +0000 UTC m=+3700.986565159" watchObservedRunningTime="2025-12-03 12:21:50.409055601 +0000 UTC m=+3700.997297365" Dec 03 12:21:50 crc kubenswrapper[5052]: I1203 12:21:50.409768 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage19-crc" (UniqueName: "kubernetes.io/local-volume/local-storage19-crc") on node "crc" Dec 03 12:21:50 crc kubenswrapper[5052]: I1203 12:21:50.432393 5052 scope.go:117] "RemoveContainer" containerID="461a75cc1c9b99044785a22ff9dae21bbe4db12db4d1af47322cef18e6d1565c" Dec 03 12:21:50 crc kubenswrapper[5052]: I1203 12:21:50.442462 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/rabbitmq-cell1-server-0"] Dec 03 12:21:50 crc kubenswrapper[5052]: I1203 12:21:50.452107 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/rabbitmq-cell1-server-0"] Dec 03 12:21:50 crc kubenswrapper[5052]: I1203 12:21:50.461158 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/rabbitmq-server-0"] Dec 03 12:21:50 crc kubenswrapper[5052]: I1203 12:21:50.467063 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/rabbitmq-server-0"] Dec 03 12:21:50 crc kubenswrapper[5052]: I1203 12:21:50.472459 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage19-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage19-crc\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:50 crc kubenswrapper[5052]: I1203 12:21:50.473131 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ovn-northd-0"] Dec 03 12:21:50 crc kubenswrapper[5052]: I1203 12:21:50.475712 5052 scope.go:117] "RemoveContainer" containerID="9e557ebf23292119d2650d0f7e9abf49644bff9d991b5bf5077147a45b27b74b" Dec 03 12:21:50 crc kubenswrapper[5052]: E1203 12:21:50.476160 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9e557ebf23292119d2650d0f7e9abf49644bff9d991b5bf5077147a45b27b74b\": container with ID starting with 9e557ebf23292119d2650d0f7e9abf49644bff9d991b5bf5077147a45b27b74b not found: ID does not exist" containerID="9e557ebf23292119d2650d0f7e9abf49644bff9d991b5bf5077147a45b27b74b" Dec 03 12:21:50 crc kubenswrapper[5052]: I1203 12:21:50.476194 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9e557ebf23292119d2650d0f7e9abf49644bff9d991b5bf5077147a45b27b74b"} err="failed to get container status \"9e557ebf23292119d2650d0f7e9abf49644bff9d991b5bf5077147a45b27b74b\": rpc error: code = NotFound desc = could not find container \"9e557ebf23292119d2650d0f7e9abf49644bff9d991b5bf5077147a45b27b74b\": container with ID starting with 9e557ebf23292119d2650d0f7e9abf49644bff9d991b5bf5077147a45b27b74b not found: ID does not exist" Dec 03 12:21:50 crc kubenswrapper[5052]: I1203 12:21:50.476218 5052 scope.go:117] "RemoveContainer" containerID="461a75cc1c9b99044785a22ff9dae21bbe4db12db4d1af47322cef18e6d1565c" Dec 03 12:21:50 crc kubenswrapper[5052]: E1203 12:21:50.476544 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"461a75cc1c9b99044785a22ff9dae21bbe4db12db4d1af47322cef18e6d1565c\": container with ID starting with 461a75cc1c9b99044785a22ff9dae21bbe4db12db4d1af47322cef18e6d1565c not found: ID does not exist" containerID="461a75cc1c9b99044785a22ff9dae21bbe4db12db4d1af47322cef18e6d1565c" Dec 03 12:21:50 crc kubenswrapper[5052]: I1203 12:21:50.476566 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"461a75cc1c9b99044785a22ff9dae21bbe4db12db4d1af47322cef18e6d1565c"} err="failed to get container status \"461a75cc1c9b99044785a22ff9dae21bbe4db12db4d1af47322cef18e6d1565c\": rpc error: code = NotFound desc = could not find container \"461a75cc1c9b99044785a22ff9dae21bbe4db12db4d1af47322cef18e6d1565c\": container with ID starting with 461a75cc1c9b99044785a22ff9dae21bbe4db12db4d1af47322cef18e6d1565c not found: ID does not exist" Dec 03 12:21:50 crc kubenswrapper[5052]: I1203 12:21:50.476579 5052 scope.go:117] "RemoveContainer" containerID="9217efd68f2e5602eadef89a260d023bd85316f53943b906f69ea6a06b762ecd" Dec 03 12:21:50 crc kubenswrapper[5052]: I1203 12:21:50.478679 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ovn-northd-0"] Dec 03 12:21:50 crc kubenswrapper[5052]: I1203 12:21:50.497684 5052 scope.go:117] "RemoveContainer" containerID="4eb8d3dd99172fe5ebfc87ffeaf97722130ac6901117c3bdc435b6f6a054e23f" Dec 03 12:21:50 crc kubenswrapper[5052]: I1203 12:21:50.518629 5052 scope.go:117] "RemoveContainer" containerID="9217efd68f2e5602eadef89a260d023bd85316f53943b906f69ea6a06b762ecd" Dec 03 12:21:50 crc kubenswrapper[5052]: E1203 12:21:50.519071 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9217efd68f2e5602eadef89a260d023bd85316f53943b906f69ea6a06b762ecd\": container with ID starting with 9217efd68f2e5602eadef89a260d023bd85316f53943b906f69ea6a06b762ecd not found: ID does not exist" containerID="9217efd68f2e5602eadef89a260d023bd85316f53943b906f69ea6a06b762ecd" Dec 03 12:21:50 crc kubenswrapper[5052]: I1203 12:21:50.519118 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9217efd68f2e5602eadef89a260d023bd85316f53943b906f69ea6a06b762ecd"} err="failed to get container status \"9217efd68f2e5602eadef89a260d023bd85316f53943b906f69ea6a06b762ecd\": rpc error: code = NotFound desc = could not find container \"9217efd68f2e5602eadef89a260d023bd85316f53943b906f69ea6a06b762ecd\": container with ID starting with 9217efd68f2e5602eadef89a260d023bd85316f53943b906f69ea6a06b762ecd not found: ID does not exist" Dec 03 12:21:50 crc kubenswrapper[5052]: I1203 12:21:50.519150 5052 scope.go:117] "RemoveContainer" containerID="4eb8d3dd99172fe5ebfc87ffeaf97722130ac6901117c3bdc435b6f6a054e23f" Dec 03 12:21:50 crc kubenswrapper[5052]: E1203 12:21:50.519489 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4eb8d3dd99172fe5ebfc87ffeaf97722130ac6901117c3bdc435b6f6a054e23f\": container with ID starting with 4eb8d3dd99172fe5ebfc87ffeaf97722130ac6901117c3bdc435b6f6a054e23f not found: ID does not exist" containerID="4eb8d3dd99172fe5ebfc87ffeaf97722130ac6901117c3bdc435b6f6a054e23f" Dec 03 12:21:50 crc kubenswrapper[5052]: I1203 12:21:50.519532 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4eb8d3dd99172fe5ebfc87ffeaf97722130ac6901117c3bdc435b6f6a054e23f"} err="failed to get container status \"4eb8d3dd99172fe5ebfc87ffeaf97722130ac6901117c3bdc435b6f6a054e23f\": rpc error: code = NotFound desc = could not find container \"4eb8d3dd99172fe5ebfc87ffeaf97722130ac6901117c3bdc435b6f6a054e23f\": container with ID starting with 4eb8d3dd99172fe5ebfc87ffeaf97722130ac6901117c3bdc435b6f6a054e23f not found: ID does not exist" Dec 03 12:21:50 crc kubenswrapper[5052]: I1203 12:21:50.519552 5052 scope.go:117] "RemoveContainer" containerID="cf3832a4b1ebcd0697f163ca295f373357a42dc4f1fd311347b02c08c7239df4" Dec 03 12:21:50 crc kubenswrapper[5052]: I1203 12:21:50.543647 5052 scope.go:117] "RemoveContainer" containerID="97a957b8c33e1b7a5cef18f4383a4faea413910406298074a6052c04396cec18" Dec 03 12:21:50 crc kubenswrapper[5052]: I1203 12:21:50.593025 5052 scope.go:117] "RemoveContainer" containerID="cf3832a4b1ebcd0697f163ca295f373357a42dc4f1fd311347b02c08c7239df4" Dec 03 12:21:50 crc kubenswrapper[5052]: E1203 12:21:50.593434 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cf3832a4b1ebcd0697f163ca295f373357a42dc4f1fd311347b02c08c7239df4\": container with ID starting with cf3832a4b1ebcd0697f163ca295f373357a42dc4f1fd311347b02c08c7239df4 not found: ID does not exist" containerID="cf3832a4b1ebcd0697f163ca295f373357a42dc4f1fd311347b02c08c7239df4" Dec 03 12:21:50 crc kubenswrapper[5052]: I1203 12:21:50.593471 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cf3832a4b1ebcd0697f163ca295f373357a42dc4f1fd311347b02c08c7239df4"} err="failed to get container status \"cf3832a4b1ebcd0697f163ca295f373357a42dc4f1fd311347b02c08c7239df4\": rpc error: code = NotFound desc = could not find container \"cf3832a4b1ebcd0697f163ca295f373357a42dc4f1fd311347b02c08c7239df4\": container with ID starting with cf3832a4b1ebcd0697f163ca295f373357a42dc4f1fd311347b02c08c7239df4 not found: ID does not exist" Dec 03 12:21:50 crc kubenswrapper[5052]: I1203 12:21:50.593499 5052 scope.go:117] "RemoveContainer" containerID="97a957b8c33e1b7a5cef18f4383a4faea413910406298074a6052c04396cec18" Dec 03 12:21:50 crc kubenswrapper[5052]: E1203 12:21:50.593823 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"97a957b8c33e1b7a5cef18f4383a4faea413910406298074a6052c04396cec18\": container with ID starting with 97a957b8c33e1b7a5cef18f4383a4faea413910406298074a6052c04396cec18 not found: ID does not exist" containerID="97a957b8c33e1b7a5cef18f4383a4faea413910406298074a6052c04396cec18" Dec 03 12:21:50 crc kubenswrapper[5052]: I1203 12:21:50.593867 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"97a957b8c33e1b7a5cef18f4383a4faea413910406298074a6052c04396cec18"} err="failed to get container status \"97a957b8c33e1b7a5cef18f4383a4faea413910406298074a6052c04396cec18\": rpc error: code = NotFound desc = could not find container \"97a957b8c33e1b7a5cef18f4383a4faea413910406298074a6052c04396cec18\": container with ID starting with 97a957b8c33e1b7a5cef18f4383a4faea413910406298074a6052c04396cec18 not found: ID does not exist" Dec 03 12:21:51 crc kubenswrapper[5052]: I1203 12:21:51.291108 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-worker-6b4674c647-56v84" Dec 03 12:21:51 crc kubenswrapper[5052]: I1203 12:21:51.331598 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-keystone-listener-6fb665b666-ftpjr" Dec 03 12:21:51 crc kubenswrapper[5052]: I1203 12:21:51.370270 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:21:51 crc kubenswrapper[5052]: I1203 12:21:51.390532 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/484e6cd1-fb57-4416-9a1a-cf24b4693764-logs\") pod \"484e6cd1-fb57-4416-9a1a-cf24b4693764\" (UID: \"484e6cd1-fb57-4416-9a1a-cf24b4693764\") " Dec 03 12:21:51 crc kubenswrapper[5052]: I1203 12:21:51.390595 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/484e6cd1-fb57-4416-9a1a-cf24b4693764-combined-ca-bundle\") pod \"484e6cd1-fb57-4416-9a1a-cf24b4693764\" (UID: \"484e6cd1-fb57-4416-9a1a-cf24b4693764\") " Dec 03 12:21:51 crc kubenswrapper[5052]: I1203 12:21:51.390660 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/484e6cd1-fb57-4416-9a1a-cf24b4693764-config-data\") pod \"484e6cd1-fb57-4416-9a1a-cf24b4693764\" (UID: \"484e6cd1-fb57-4416-9a1a-cf24b4693764\") " Dec 03 12:21:51 crc kubenswrapper[5052]: I1203 12:21:51.390774 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s5rx8\" (UniqueName: \"kubernetes.io/projected/484e6cd1-fb57-4416-9a1a-cf24b4693764-kube-api-access-s5rx8\") pod \"484e6cd1-fb57-4416-9a1a-cf24b4693764\" (UID: \"484e6cd1-fb57-4416-9a1a-cf24b4693764\") " Dec 03 12:21:51 crc kubenswrapper[5052]: I1203 12:21:51.390877 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/484e6cd1-fb57-4416-9a1a-cf24b4693764-config-data-custom\") pod \"484e6cd1-fb57-4416-9a1a-cf24b4693764\" (UID: \"484e6cd1-fb57-4416-9a1a-cf24b4693764\") " Dec 03 12:21:51 crc kubenswrapper[5052]: I1203 12:21:51.391254 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/484e6cd1-fb57-4416-9a1a-cf24b4693764-logs" (OuterVolumeSpecName: "logs") pod "484e6cd1-fb57-4416-9a1a-cf24b4693764" (UID: "484e6cd1-fb57-4416-9a1a-cf24b4693764"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:21:51 crc kubenswrapper[5052]: I1203 12:21:51.392280 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/484e6cd1-fb57-4416-9a1a-cf24b4693764-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:51 crc kubenswrapper[5052]: I1203 12:21:51.397850 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/484e6cd1-fb57-4416-9a1a-cf24b4693764-kube-api-access-s5rx8" (OuterVolumeSpecName: "kube-api-access-s5rx8") pod "484e6cd1-fb57-4416-9a1a-cf24b4693764" (UID: "484e6cd1-fb57-4416-9a1a-cf24b4693764"). InnerVolumeSpecName "kube-api-access-s5rx8". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:21:51 crc kubenswrapper[5052]: I1203 12:21:51.399040 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/484e6cd1-fb57-4416-9a1a-cf24b4693764-config-data-custom" (OuterVolumeSpecName: "config-data-custom") pod "484e6cd1-fb57-4416-9a1a-cf24b4693764" (UID: "484e6cd1-fb57-4416-9a1a-cf24b4693764"). InnerVolumeSpecName "config-data-custom". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:51 crc kubenswrapper[5052]: I1203 12:21:51.415098 5052 generic.go:334] "Generic (PLEG): container finished" podID="484e6cd1-fb57-4416-9a1a-cf24b4693764" containerID="e00bcd0d914f7fbd5f88db32fed6a7e8a0000e44d444198b8f7f1ed7f6ecdb89" exitCode=0 Dec 03 12:21:51 crc kubenswrapper[5052]: I1203 12:21:51.415121 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-worker-6b4674c647-56v84" Dec 03 12:21:51 crc kubenswrapper[5052]: I1203 12:21:51.415168 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-worker-6b4674c647-56v84" event={"ID":"484e6cd1-fb57-4416-9a1a-cf24b4693764","Type":"ContainerDied","Data":"e00bcd0d914f7fbd5f88db32fed6a7e8a0000e44d444198b8f7f1ed7f6ecdb89"} Dec 03 12:21:51 crc kubenswrapper[5052]: I1203 12:21:51.415275 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-worker-6b4674c647-56v84" event={"ID":"484e6cd1-fb57-4416-9a1a-cf24b4693764","Type":"ContainerDied","Data":"192121d8e29554479b65c83a0f69371b4815e1374a9f01070f02296033b3c0de"} Dec 03 12:21:51 crc kubenswrapper[5052]: I1203 12:21:51.415300 5052 scope.go:117] "RemoveContainer" containerID="e00bcd0d914f7fbd5f88db32fed6a7e8a0000e44d444198b8f7f1ed7f6ecdb89" Dec 03 12:21:51 crc kubenswrapper[5052]: I1203 12:21:51.417658 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/484e6cd1-fb57-4416-9a1a-cf24b4693764-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "484e6cd1-fb57-4416-9a1a-cf24b4693764" (UID: "484e6cd1-fb57-4416-9a1a-cf24b4693764"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:51 crc kubenswrapper[5052]: I1203 12:21:51.421799 5052 generic.go:334] "Generic (PLEG): container finished" podID="efe32106-ca1b-4d84-80a2-c0cddd3541b9" containerID="75bc26794aa58f8bf478527bc4377138449cad0841fd60348ba9d8b17243a1e4" exitCode=0 Dec 03 12:21:51 crc kubenswrapper[5052]: I1203 12:21:51.421853 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:21:51 crc kubenswrapper[5052]: I1203 12:21:51.421896 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-conductor-0" event={"ID":"efe32106-ca1b-4d84-80a2-c0cddd3541b9","Type":"ContainerDied","Data":"75bc26794aa58f8bf478527bc4377138449cad0841fd60348ba9d8b17243a1e4"} Dec 03 12:21:51 crc kubenswrapper[5052]: I1203 12:21:51.421946 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-conductor-0" event={"ID":"efe32106-ca1b-4d84-80a2-c0cddd3541b9","Type":"ContainerDied","Data":"13bb5b7012087ab429759406030370af76979b95cc58559676341cb6a1ce0b66"} Dec 03 12:21:51 crc kubenswrapper[5052]: I1203 12:21:51.425846 5052 generic.go:334] "Generic (PLEG): container finished" podID="9119635e-cb55-4428-9c61-b93ef2afd799" containerID="eaa2b6fb75e11891ca6e0ce7eb90ce65bb570e469fdab944929138063436ea33" exitCode=0 Dec 03 12:21:51 crc kubenswrapper[5052]: I1203 12:21:51.425916 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-keystone-listener-6fb665b666-ftpjr" event={"ID":"9119635e-cb55-4428-9c61-b93ef2afd799","Type":"ContainerDied","Data":"eaa2b6fb75e11891ca6e0ce7eb90ce65bb570e469fdab944929138063436ea33"} Dec 03 12:21:51 crc kubenswrapper[5052]: I1203 12:21:51.425945 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-keystone-listener-6fb665b666-ftpjr" event={"ID":"9119635e-cb55-4428-9c61-b93ef2afd799","Type":"ContainerDied","Data":"4e1a1d86a4b1a8e7ad13fe35c40a03629e8393c18875d941b6004f1f25ed058e"} Dec 03 12:21:51 crc kubenswrapper[5052]: I1203 12:21:51.426009 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-keystone-listener-6fb665b666-ftpjr" Dec 03 12:21:51 crc kubenswrapper[5052]: I1203 12:21:51.445397 5052 scope.go:117] "RemoveContainer" containerID="01a0969c60e54d821f713b7a30fadf577164bb47e2f3fc851bc640b779f911d6" Dec 03 12:21:51 crc kubenswrapper[5052]: I1203 12:21:51.452290 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/484e6cd1-fb57-4416-9a1a-cf24b4693764-config-data" (OuterVolumeSpecName: "config-data") pod "484e6cd1-fb57-4416-9a1a-cf24b4693764" (UID: "484e6cd1-fb57-4416-9a1a-cf24b4693764"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:51 crc kubenswrapper[5052]: I1203 12:21:51.470963 5052 scope.go:117] "RemoveContainer" containerID="e00bcd0d914f7fbd5f88db32fed6a7e8a0000e44d444198b8f7f1ed7f6ecdb89" Dec 03 12:21:51 crc kubenswrapper[5052]: E1203 12:21:51.471799 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e00bcd0d914f7fbd5f88db32fed6a7e8a0000e44d444198b8f7f1ed7f6ecdb89\": container with ID starting with e00bcd0d914f7fbd5f88db32fed6a7e8a0000e44d444198b8f7f1ed7f6ecdb89 not found: ID does not exist" containerID="e00bcd0d914f7fbd5f88db32fed6a7e8a0000e44d444198b8f7f1ed7f6ecdb89" Dec 03 12:21:51 crc kubenswrapper[5052]: I1203 12:21:51.471848 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e00bcd0d914f7fbd5f88db32fed6a7e8a0000e44d444198b8f7f1ed7f6ecdb89"} err="failed to get container status \"e00bcd0d914f7fbd5f88db32fed6a7e8a0000e44d444198b8f7f1ed7f6ecdb89\": rpc error: code = NotFound desc = could not find container \"e00bcd0d914f7fbd5f88db32fed6a7e8a0000e44d444198b8f7f1ed7f6ecdb89\": container with ID starting with e00bcd0d914f7fbd5f88db32fed6a7e8a0000e44d444198b8f7f1ed7f6ecdb89 not found: ID does not exist" Dec 03 12:21:51 crc kubenswrapper[5052]: I1203 12:21:51.471884 5052 scope.go:117] "RemoveContainer" containerID="01a0969c60e54d821f713b7a30fadf577164bb47e2f3fc851bc640b779f911d6" Dec 03 12:21:51 crc kubenswrapper[5052]: E1203 12:21:51.472421 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"01a0969c60e54d821f713b7a30fadf577164bb47e2f3fc851bc640b779f911d6\": container with ID starting with 01a0969c60e54d821f713b7a30fadf577164bb47e2f3fc851bc640b779f911d6 not found: ID does not exist" containerID="01a0969c60e54d821f713b7a30fadf577164bb47e2f3fc851bc640b779f911d6" Dec 03 12:21:51 crc kubenswrapper[5052]: I1203 12:21:51.472485 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"01a0969c60e54d821f713b7a30fadf577164bb47e2f3fc851bc640b779f911d6"} err="failed to get container status \"01a0969c60e54d821f713b7a30fadf577164bb47e2f3fc851bc640b779f911d6\": rpc error: code = NotFound desc = could not find container \"01a0969c60e54d821f713b7a30fadf577164bb47e2f3fc851bc640b779f911d6\": container with ID starting with 01a0969c60e54d821f713b7a30fadf577164bb47e2f3fc851bc640b779f911d6 not found: ID does not exist" Dec 03 12:21:51 crc kubenswrapper[5052]: I1203 12:21:51.472535 5052 scope.go:117] "RemoveContainer" containerID="75bc26794aa58f8bf478527bc4377138449cad0841fd60348ba9d8b17243a1e4" Dec 03 12:21:51 crc kubenswrapper[5052]: I1203 12:21:51.493900 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f25j9\" (UniqueName: \"kubernetes.io/projected/9119635e-cb55-4428-9c61-b93ef2afd799-kube-api-access-f25j9\") pod \"9119635e-cb55-4428-9c61-b93ef2afd799\" (UID: \"9119635e-cb55-4428-9c61-b93ef2afd799\") " Dec 03 12:21:51 crc kubenswrapper[5052]: I1203 12:21:51.493977 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/9119635e-cb55-4428-9c61-b93ef2afd799-logs\") pod \"9119635e-cb55-4428-9c61-b93ef2afd799\" (UID: \"9119635e-cb55-4428-9c61-b93ef2afd799\") " Dec 03 12:21:51 crc kubenswrapper[5052]: I1203 12:21:51.494089 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9119635e-cb55-4428-9c61-b93ef2afd799-combined-ca-bundle\") pod \"9119635e-cb55-4428-9c61-b93ef2afd799\" (UID: \"9119635e-cb55-4428-9c61-b93ef2afd799\") " Dec 03 12:21:51 crc kubenswrapper[5052]: I1203 12:21:51.494151 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/efe32106-ca1b-4d84-80a2-c0cddd3541b9-combined-ca-bundle\") pod \"efe32106-ca1b-4d84-80a2-c0cddd3541b9\" (UID: \"efe32106-ca1b-4d84-80a2-c0cddd3541b9\") " Dec 03 12:21:51 crc kubenswrapper[5052]: I1203 12:21:51.494174 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/9119635e-cb55-4428-9c61-b93ef2afd799-config-data-custom\") pod \"9119635e-cb55-4428-9c61-b93ef2afd799\" (UID: \"9119635e-cb55-4428-9c61-b93ef2afd799\") " Dec 03 12:21:51 crc kubenswrapper[5052]: I1203 12:21:51.494249 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bbz9b\" (UniqueName: \"kubernetes.io/projected/efe32106-ca1b-4d84-80a2-c0cddd3541b9-kube-api-access-bbz9b\") pod \"efe32106-ca1b-4d84-80a2-c0cddd3541b9\" (UID: \"efe32106-ca1b-4d84-80a2-c0cddd3541b9\") " Dec 03 12:21:51 crc kubenswrapper[5052]: I1203 12:21:51.494372 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/efe32106-ca1b-4d84-80a2-c0cddd3541b9-config-data\") pod \"efe32106-ca1b-4d84-80a2-c0cddd3541b9\" (UID: \"efe32106-ca1b-4d84-80a2-c0cddd3541b9\") " Dec 03 12:21:51 crc kubenswrapper[5052]: I1203 12:21:51.494438 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/9119635e-cb55-4428-9c61-b93ef2afd799-config-data\") pod \"9119635e-cb55-4428-9c61-b93ef2afd799\" (UID: \"9119635e-cb55-4428-9c61-b93ef2afd799\") " Dec 03 12:21:51 crc kubenswrapper[5052]: I1203 12:21:51.495131 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/484e6cd1-fb57-4416-9a1a-cf24b4693764-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:51 crc kubenswrapper[5052]: I1203 12:21:51.495162 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-s5rx8\" (UniqueName: \"kubernetes.io/projected/484e6cd1-fb57-4416-9a1a-cf24b4693764-kube-api-access-s5rx8\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:51 crc kubenswrapper[5052]: I1203 12:21:51.495178 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/484e6cd1-fb57-4416-9a1a-cf24b4693764-config-data-custom\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:51 crc kubenswrapper[5052]: I1203 12:21:51.495192 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/484e6cd1-fb57-4416-9a1a-cf24b4693764-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:51 crc kubenswrapper[5052]: I1203 12:21:51.495827 5052 scope.go:117] "RemoveContainer" containerID="75bc26794aa58f8bf478527bc4377138449cad0841fd60348ba9d8b17243a1e4" Dec 03 12:21:51 crc kubenswrapper[5052]: I1203 12:21:51.497222 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9119635e-cb55-4428-9c61-b93ef2afd799-logs" (OuterVolumeSpecName: "logs") pod "9119635e-cb55-4428-9c61-b93ef2afd799" (UID: "9119635e-cb55-4428-9c61-b93ef2afd799"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:21:51 crc kubenswrapper[5052]: E1203 12:21:51.498340 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"75bc26794aa58f8bf478527bc4377138449cad0841fd60348ba9d8b17243a1e4\": container with ID starting with 75bc26794aa58f8bf478527bc4377138449cad0841fd60348ba9d8b17243a1e4 not found: ID does not exist" containerID="75bc26794aa58f8bf478527bc4377138449cad0841fd60348ba9d8b17243a1e4" Dec 03 12:21:51 crc kubenswrapper[5052]: I1203 12:21:51.498392 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"75bc26794aa58f8bf478527bc4377138449cad0841fd60348ba9d8b17243a1e4"} err="failed to get container status \"75bc26794aa58f8bf478527bc4377138449cad0841fd60348ba9d8b17243a1e4\": rpc error: code = NotFound desc = could not find container \"75bc26794aa58f8bf478527bc4377138449cad0841fd60348ba9d8b17243a1e4\": container with ID starting with 75bc26794aa58f8bf478527bc4377138449cad0841fd60348ba9d8b17243a1e4 not found: ID does not exist" Dec 03 12:21:51 crc kubenswrapper[5052]: I1203 12:21:51.498433 5052 scope.go:117] "RemoveContainer" containerID="eaa2b6fb75e11891ca6e0ce7eb90ce65bb570e469fdab944929138063436ea33" Dec 03 12:21:51 crc kubenswrapper[5052]: I1203 12:21:51.498919 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9119635e-cb55-4428-9c61-b93ef2afd799-kube-api-access-f25j9" (OuterVolumeSpecName: "kube-api-access-f25j9") pod "9119635e-cb55-4428-9c61-b93ef2afd799" (UID: "9119635e-cb55-4428-9c61-b93ef2afd799"). InnerVolumeSpecName "kube-api-access-f25j9". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:21:51 crc kubenswrapper[5052]: I1203 12:21:51.499773 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/efe32106-ca1b-4d84-80a2-c0cddd3541b9-kube-api-access-bbz9b" (OuterVolumeSpecName: "kube-api-access-bbz9b") pod "efe32106-ca1b-4d84-80a2-c0cddd3541b9" (UID: "efe32106-ca1b-4d84-80a2-c0cddd3541b9"). InnerVolumeSpecName "kube-api-access-bbz9b". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:21:51 crc kubenswrapper[5052]: I1203 12:21:51.500687 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9119635e-cb55-4428-9c61-b93ef2afd799-config-data-custom" (OuterVolumeSpecName: "config-data-custom") pod "9119635e-cb55-4428-9c61-b93ef2afd799" (UID: "9119635e-cb55-4428-9c61-b93ef2afd799"). InnerVolumeSpecName "config-data-custom". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:51 crc kubenswrapper[5052]: I1203 12:21:51.520182 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9119635e-cb55-4428-9c61-b93ef2afd799-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "9119635e-cb55-4428-9c61-b93ef2afd799" (UID: "9119635e-cb55-4428-9c61-b93ef2afd799"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:51 crc kubenswrapper[5052]: I1203 12:21:51.521213 5052 scope.go:117] "RemoveContainer" containerID="d006b275534cfdfd7a634f938440db7eeb0d006446de410abf688af6391508cb" Dec 03 12:21:51 crc kubenswrapper[5052]: I1203 12:21:51.532238 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/efe32106-ca1b-4d84-80a2-c0cddd3541b9-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "efe32106-ca1b-4d84-80a2-c0cddd3541b9" (UID: "efe32106-ca1b-4d84-80a2-c0cddd3541b9"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:51 crc kubenswrapper[5052]: I1203 12:21:51.536777 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/efe32106-ca1b-4d84-80a2-c0cddd3541b9-config-data" (OuterVolumeSpecName: "config-data") pod "efe32106-ca1b-4d84-80a2-c0cddd3541b9" (UID: "efe32106-ca1b-4d84-80a2-c0cddd3541b9"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:51 crc kubenswrapper[5052]: I1203 12:21:51.544283 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9119635e-cb55-4428-9c61-b93ef2afd799-config-data" (OuterVolumeSpecName: "config-data") pod "9119635e-cb55-4428-9c61-b93ef2afd799" (UID: "9119635e-cb55-4428-9c61-b93ef2afd799"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:21:51 crc kubenswrapper[5052]: I1203 12:21:51.546354 5052 scope.go:117] "RemoveContainer" containerID="eaa2b6fb75e11891ca6e0ce7eb90ce65bb570e469fdab944929138063436ea33" Dec 03 12:21:51 crc kubenswrapper[5052]: E1203 12:21:51.546951 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eaa2b6fb75e11891ca6e0ce7eb90ce65bb570e469fdab944929138063436ea33\": container with ID starting with eaa2b6fb75e11891ca6e0ce7eb90ce65bb570e469fdab944929138063436ea33 not found: ID does not exist" containerID="eaa2b6fb75e11891ca6e0ce7eb90ce65bb570e469fdab944929138063436ea33" Dec 03 12:21:51 crc kubenswrapper[5052]: I1203 12:21:51.546990 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eaa2b6fb75e11891ca6e0ce7eb90ce65bb570e469fdab944929138063436ea33"} err="failed to get container status \"eaa2b6fb75e11891ca6e0ce7eb90ce65bb570e469fdab944929138063436ea33\": rpc error: code = NotFound desc = could not find container \"eaa2b6fb75e11891ca6e0ce7eb90ce65bb570e469fdab944929138063436ea33\": container with ID starting with eaa2b6fb75e11891ca6e0ce7eb90ce65bb570e469fdab944929138063436ea33 not found: ID does not exist" Dec 03 12:21:51 crc kubenswrapper[5052]: I1203 12:21:51.547021 5052 scope.go:117] "RemoveContainer" containerID="d006b275534cfdfd7a634f938440db7eeb0d006446de410abf688af6391508cb" Dec 03 12:21:51 crc kubenswrapper[5052]: E1203 12:21:51.547363 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d006b275534cfdfd7a634f938440db7eeb0d006446de410abf688af6391508cb\": container with ID starting with d006b275534cfdfd7a634f938440db7eeb0d006446de410abf688af6391508cb not found: ID does not exist" containerID="d006b275534cfdfd7a634f938440db7eeb0d006446de410abf688af6391508cb" Dec 03 12:21:51 crc kubenswrapper[5052]: I1203 12:21:51.547387 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d006b275534cfdfd7a634f938440db7eeb0d006446de410abf688af6391508cb"} err="failed to get container status \"d006b275534cfdfd7a634f938440db7eeb0d006446de410abf688af6391508cb\": rpc error: code = NotFound desc = could not find container \"d006b275534cfdfd7a634f938440db7eeb0d006446de410abf688af6391508cb\": container with ID starting with d006b275534cfdfd7a634f938440db7eeb0d006446de410abf688af6391508cb not found: ID does not exist" Dec 03 12:21:51 crc kubenswrapper[5052]: I1203 12:21:51.596981 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/efe32106-ca1b-4d84-80a2-c0cddd3541b9-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:51 crc kubenswrapper[5052]: I1203 12:21:51.597056 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/9119635e-cb55-4428-9c61-b93ef2afd799-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:51 crc kubenswrapper[5052]: I1203 12:21:51.597074 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-f25j9\" (UniqueName: \"kubernetes.io/projected/9119635e-cb55-4428-9c61-b93ef2afd799-kube-api-access-f25j9\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:51 crc kubenswrapper[5052]: I1203 12:21:51.597090 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/9119635e-cb55-4428-9c61-b93ef2afd799-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:51 crc kubenswrapper[5052]: I1203 12:21:51.597103 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9119635e-cb55-4428-9c61-b93ef2afd799-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:51 crc kubenswrapper[5052]: I1203 12:21:51.597117 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/efe32106-ca1b-4d84-80a2-c0cddd3541b9-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:51 crc kubenswrapper[5052]: I1203 12:21:51.597129 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/9119635e-cb55-4428-9c61-b93ef2afd799-config-data-custom\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:51 crc kubenswrapper[5052]: I1203 12:21:51.597141 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-bbz9b\" (UniqueName: \"kubernetes.io/projected/efe32106-ca1b-4d84-80a2-c0cddd3541b9-kube-api-access-bbz9b\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:51 crc kubenswrapper[5052]: I1203 12:21:51.682337 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a77df94c-3e83-47a5-9b5f-eb0aeb41bc3d" path="/var/lib/kubelet/pods/a77df94c-3e83-47a5-9b5f-eb0aeb41bc3d/volumes" Dec 03 12:21:51 crc kubenswrapper[5052]: I1203 12:21:51.683349 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c14a906c-f19d-4806-9110-8c47b38c5ae0" path="/var/lib/kubelet/pods/c14a906c-f19d-4806-9110-8c47b38c5ae0/volumes" Dec 03 12:21:51 crc kubenswrapper[5052]: I1203 12:21:51.684801 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c279eaf1-2ee2-46fe-88be-9a5a65d4897e" path="/var/lib/kubelet/pods/c279eaf1-2ee2-46fe-88be-9a5a65d4897e/volumes" Dec 03 12:21:51 crc kubenswrapper[5052]: I1203 12:21:51.742386 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-worker-6b4674c647-56v84"] Dec 03 12:21:51 crc kubenswrapper[5052]: I1203 12:21:51.752231 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/barbican-worker-6b4674c647-56v84"] Dec 03 12:21:51 crc kubenswrapper[5052]: I1203 12:21:51.765420 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-keystone-listener-6fb665b666-ftpjr"] Dec 03 12:21:51 crc kubenswrapper[5052]: I1203 12:21:51.774637 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/barbican-keystone-listener-6fb665b666-ftpjr"] Dec 03 12:21:51 crc kubenswrapper[5052]: I1203 12:21:51.781008 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-0"] Dec 03 12:21:51 crc kubenswrapper[5052]: I1203 12:21:51.786155 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-0"] Dec 03 12:21:52 crc kubenswrapper[5052]: E1203 12:21:52.307093 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:21:52 crc kubenswrapper[5052]: E1203 12:21:52.307214 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/817345c8-3a42-4dea-8158-75973d9c41d9-operator-scripts podName:817345c8-3a42-4dea-8158-75973d9c41d9 nodeName:}" failed. No retries permitted until 2025-12-03 12:22:00.307190088 +0000 UTC m=+3710.895431882 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/817345c8-3a42-4dea-8158-75973d9c41d9-operator-scripts") pod "neutron-734a-account-create-update-qcgft" (UID: "817345c8-3a42-4dea-8158-75973d9c41d9") : configmap "openstack-scripts" not found Dec 03 12:21:52 crc kubenswrapper[5052]: E1203 12:21:52.408925 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:21:52 crc kubenswrapper[5052]: E1203 12:21:52.409038 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/db31774e-8f50-4862-86f3-93cb93f645a0-operator-scripts podName:db31774e-8f50-4862-86f3-93cb93f645a0 nodeName:}" failed. No retries permitted until 2025-12-03 12:22:00.409011675 +0000 UTC m=+3710.997253459 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/db31774e-8f50-4862-86f3-93cb93f645a0-operator-scripts") pod "glance73ca-account-delete-vpqhp" (UID: "db31774e-8f50-4862-86f3-93cb93f645a0") : configmap "openstack-scripts" not found Dec 03 12:21:53 crc kubenswrapper[5052]: I1203 12:21:53.682675 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="484e6cd1-fb57-4416-9a1a-cf24b4693764" path="/var/lib/kubelet/pods/484e6cd1-fb57-4416-9a1a-cf24b4693764/volumes" Dec 03 12:21:53 crc kubenswrapper[5052]: I1203 12:21:53.683767 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9119635e-cb55-4428-9c61-b93ef2afd799" path="/var/lib/kubelet/pods/9119635e-cb55-4428-9c61-b93ef2afd799/volumes" Dec 03 12:21:53 crc kubenswrapper[5052]: I1203 12:21:53.684308 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="efe32106-ca1b-4d84-80a2-c0cddd3541b9" path="/var/lib/kubelet/pods/efe32106-ca1b-4d84-80a2-c0cddd3541b9/volumes" Dec 03 12:21:58 crc kubenswrapper[5052]: I1203 12:21:58.467689 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-7mq7s" Dec 03 12:21:58 crc kubenswrapper[5052]: I1203 12:21:58.527049 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-5bbbb9b4b9-m4ttn"] Dec 03 12:21:58 crc kubenswrapper[5052]: I1203 12:21:58.527306 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-5bbbb9b4b9-m4ttn" podUID="f12eef70-f626-42ce-b4cf-2a07adec9657" containerName="dnsmasq-dns" containerID="cri-o://1e7096c07ede142e1bcdfe92a44f991a0ced70d36647bcc1f6c50501e03a2b5d" gracePeriod=10 Dec 03 12:21:59 crc kubenswrapper[5052]: I1203 12:21:59.531992 5052 generic.go:334] "Generic (PLEG): container finished" podID="f12eef70-f626-42ce-b4cf-2a07adec9657" containerID="1e7096c07ede142e1bcdfe92a44f991a0ced70d36647bcc1f6c50501e03a2b5d" exitCode=0 Dec 03 12:21:59 crc kubenswrapper[5052]: I1203 12:21:59.532052 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-5bbbb9b4b9-m4ttn" event={"ID":"f12eef70-f626-42ce-b4cf-2a07adec9657","Type":"ContainerDied","Data":"1e7096c07ede142e1bcdfe92a44f991a0ced70d36647bcc1f6c50501e03a2b5d"} Dec 03 12:21:59 crc kubenswrapper[5052]: I1203 12:21:59.648744 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-5bbbb9b4b9-m4ttn" Dec 03 12:21:59 crc kubenswrapper[5052]: I1203 12:21:59.753931 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f12eef70-f626-42ce-b4cf-2a07adec9657-config\") pod \"f12eef70-f626-42ce-b4cf-2a07adec9657\" (UID: \"f12eef70-f626-42ce-b4cf-2a07adec9657\") " Dec 03 12:21:59 crc kubenswrapper[5052]: I1203 12:21:59.754004 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"dns-swift-storage-0\" (UniqueName: \"kubernetes.io/configmap/f12eef70-f626-42ce-b4cf-2a07adec9657-dns-swift-storage-0\") pod \"f12eef70-f626-42ce-b4cf-2a07adec9657\" (UID: \"f12eef70-f626-42ce-b4cf-2a07adec9657\") " Dec 03 12:21:59 crc kubenswrapper[5052]: I1203 12:21:59.754067 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-llb4x\" (UniqueName: \"kubernetes.io/projected/f12eef70-f626-42ce-b4cf-2a07adec9657-kube-api-access-llb4x\") pod \"f12eef70-f626-42ce-b4cf-2a07adec9657\" (UID: \"f12eef70-f626-42ce-b4cf-2a07adec9657\") " Dec 03 12:21:59 crc kubenswrapper[5052]: I1203 12:21:59.754125 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/f12eef70-f626-42ce-b4cf-2a07adec9657-dnsmasq-svc\") pod \"f12eef70-f626-42ce-b4cf-2a07adec9657\" (UID: \"f12eef70-f626-42ce-b4cf-2a07adec9657\") " Dec 03 12:21:59 crc kubenswrapper[5052]: I1203 12:21:59.758561 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f12eef70-f626-42ce-b4cf-2a07adec9657-kube-api-access-llb4x" (OuterVolumeSpecName: "kube-api-access-llb4x") pod "f12eef70-f626-42ce-b4cf-2a07adec9657" (UID: "f12eef70-f626-42ce-b4cf-2a07adec9657"). InnerVolumeSpecName "kube-api-access-llb4x". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:21:59 crc kubenswrapper[5052]: I1203 12:21:59.792749 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f12eef70-f626-42ce-b4cf-2a07adec9657-config" (OuterVolumeSpecName: "config") pod "f12eef70-f626-42ce-b4cf-2a07adec9657" (UID: "f12eef70-f626-42ce-b4cf-2a07adec9657"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:21:59 crc kubenswrapper[5052]: I1203 12:21:59.792799 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f12eef70-f626-42ce-b4cf-2a07adec9657-dns-swift-storage-0" (OuterVolumeSpecName: "dns-swift-storage-0") pod "f12eef70-f626-42ce-b4cf-2a07adec9657" (UID: "f12eef70-f626-42ce-b4cf-2a07adec9657"). InnerVolumeSpecName "dns-swift-storage-0". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:21:59 crc kubenswrapper[5052]: I1203 12:21:59.794758 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f12eef70-f626-42ce-b4cf-2a07adec9657-dnsmasq-svc" (OuterVolumeSpecName: "dnsmasq-svc") pod "f12eef70-f626-42ce-b4cf-2a07adec9657" (UID: "f12eef70-f626-42ce-b4cf-2a07adec9657"). InnerVolumeSpecName "dnsmasq-svc". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:21:59 crc kubenswrapper[5052]: I1203 12:21:59.856121 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f12eef70-f626-42ce-b4cf-2a07adec9657-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:59 crc kubenswrapper[5052]: I1203 12:21:59.856154 5052 reconciler_common.go:293] "Volume detached for volume \"dns-swift-storage-0\" (UniqueName: \"kubernetes.io/configmap/f12eef70-f626-42ce-b4cf-2a07adec9657-dns-swift-storage-0\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:59 crc kubenswrapper[5052]: I1203 12:21:59.856168 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-llb4x\" (UniqueName: \"kubernetes.io/projected/f12eef70-f626-42ce-b4cf-2a07adec9657-kube-api-access-llb4x\") on node \"crc\" DevicePath \"\"" Dec 03 12:21:59 crc kubenswrapper[5052]: I1203 12:21:59.856178 5052 reconciler_common.go:293] "Volume detached for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/f12eef70-f626-42ce-b4cf-2a07adec9657-dnsmasq-svc\") on node \"crc\" DevicePath \"\"" Dec 03 12:22:00 crc kubenswrapper[5052]: E1203 12:22:00.363002 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:22:00 crc kubenswrapper[5052]: E1203 12:22:00.363073 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/817345c8-3a42-4dea-8158-75973d9c41d9-operator-scripts podName:817345c8-3a42-4dea-8158-75973d9c41d9 nodeName:}" failed. No retries permitted until 2025-12-03 12:22:16.36305961 +0000 UTC m=+3726.951301374 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/817345c8-3a42-4dea-8158-75973d9c41d9-operator-scripts") pod "neutron-734a-account-create-update-qcgft" (UID: "817345c8-3a42-4dea-8158-75973d9c41d9") : configmap "openstack-scripts" not found Dec 03 12:22:00 crc kubenswrapper[5052]: E1203 12:22:00.464827 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:22:00 crc kubenswrapper[5052]: E1203 12:22:00.465127 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/db31774e-8f50-4862-86f3-93cb93f645a0-operator-scripts podName:db31774e-8f50-4862-86f3-93cb93f645a0 nodeName:}" failed. No retries permitted until 2025-12-03 12:22:16.465104452 +0000 UTC m=+3727.053346276 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/db31774e-8f50-4862-86f3-93cb93f645a0-operator-scripts") pod "glance73ca-account-delete-vpqhp" (UID: "db31774e-8f50-4862-86f3-93cb93f645a0") : configmap "openstack-scripts" not found Dec 03 12:22:00 crc kubenswrapper[5052]: I1203 12:22:00.546094 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-5bbbb9b4b9-m4ttn" event={"ID":"f12eef70-f626-42ce-b4cf-2a07adec9657","Type":"ContainerDied","Data":"81629af8a7959b1b67cf8533bb2a47a92e3d1553cbe762f77235d9042375cb75"} Dec 03 12:22:00 crc kubenswrapper[5052]: I1203 12:22:00.546633 5052 scope.go:117] "RemoveContainer" containerID="1e7096c07ede142e1bcdfe92a44f991a0ced70d36647bcc1f6c50501e03a2b5d" Dec 03 12:22:00 crc kubenswrapper[5052]: I1203 12:22:00.546141 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-5bbbb9b4b9-m4ttn" Dec 03 12:22:00 crc kubenswrapper[5052]: I1203 12:22:00.576753 5052 scope.go:117] "RemoveContainer" containerID="e7e7dd0b02628c62826b4852f91d8f2be59cbd148aede2a98c85fb101457d775" Dec 03 12:22:00 crc kubenswrapper[5052]: I1203 12:22:00.580381 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-5bbbb9b4b9-m4ttn"] Dec 03 12:22:00 crc kubenswrapper[5052]: I1203 12:22:00.588451 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-5bbbb9b4b9-m4ttn"] Dec 03 12:22:01 crc kubenswrapper[5052]: I1203 12:22:01.673268 5052 scope.go:117] "RemoveContainer" containerID="09b4a0072741f60499d24f40cc4ff3406d5d22671ced431363ab865172786300" Dec 03 12:22:01 crc kubenswrapper[5052]: E1203 12:22:01.673819 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 12:22:01 crc kubenswrapper[5052]: I1203 12:22:01.682233 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f12eef70-f626-42ce-b4cf-2a07adec9657" path="/var/lib/kubelet/pods/f12eef70-f626-42ce-b4cf-2a07adec9657/volumes" Dec 03 12:22:04 crc kubenswrapper[5052]: I1203 12:22:04.522003 5052 scope.go:117] "RemoveContainer" containerID="fab60ab776551b13a2d2abeee894ccd3d25015754ba7d7788bde10100e3537d9" Dec 03 12:22:04 crc kubenswrapper[5052]: I1203 12:22:04.564019 5052 scope.go:117] "RemoveContainer" containerID="679522668d46d6b55c6186c11c5c0d53cb5cd3280e0e0dec04e125121ba4047a" Dec 03 12:22:04 crc kubenswrapper[5052]: I1203 12:22:04.592381 5052 scope.go:117] "RemoveContainer" containerID="67c22604bca84b1f1e31bb59eb85a540591562d9df9164cbcf210f964d1d5beb" Dec 03 12:22:04 crc kubenswrapper[5052]: I1203 12:22:04.692794 5052 scope.go:117] "RemoveContainer" containerID="998bba76a5f7f109bda9a23387d0dfd8298ff434e5f70a2d60d2a1801c384af4" Dec 03 12:22:04 crc kubenswrapper[5052]: I1203 12:22:04.720191 5052 scope.go:117] "RemoveContainer" containerID="e012c7962e8153da90fe1b2a14dc53f9e2c75d7dcb586b6279ae4612ffd6977b" Dec 03 12:22:04 crc kubenswrapper[5052]: I1203 12:22:04.797002 5052 scope.go:117] "RemoveContainer" containerID="ad600490ee3191e301f7e30a3ba1fcff1867ac26a31d4e56b6e4a17fc40b417b" Dec 03 12:22:05 crc kubenswrapper[5052]: I1203 12:22:05.508803 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-69c9545844-xf6zm" Dec 03 12:22:05 crc kubenswrapper[5052]: I1203 12:22:05.604722 5052 generic.go:334] "Generic (PLEG): container finished" podID="03bb65ef-e9a8-4cfb-9536-001e316b1f72" containerID="48fa168d5d7691a3b2d8508087753330d79877c968cc2b4e0f021b0441800e44" exitCode=0 Dec 03 12:22:05 crc kubenswrapper[5052]: I1203 12:22:05.604826 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-69c9545844-xf6zm" event={"ID":"03bb65ef-e9a8-4cfb-9536-001e316b1f72","Type":"ContainerDied","Data":"48fa168d5d7691a3b2d8508087753330d79877c968cc2b4e0f021b0441800e44"} Dec 03 12:22:05 crc kubenswrapper[5052]: I1203 12:22:05.604857 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-69c9545844-xf6zm" event={"ID":"03bb65ef-e9a8-4cfb-9536-001e316b1f72","Type":"ContainerDied","Data":"9f16a2482848753a3055f65a88879c86ee646460485035f648588f1468241420"} Dec 03 12:22:05 crc kubenswrapper[5052]: I1203 12:22:05.604921 5052 scope.go:117] "RemoveContainer" containerID="ece24b1db9c0709fd516d1a80d474c2937fdeb49dd573043245a6e0d4d27d04a" Dec 03 12:22:05 crc kubenswrapper[5052]: I1203 12:22:05.605058 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-69c9545844-xf6zm" Dec 03 12:22:05 crc kubenswrapper[5052]: I1203 12:22:05.627654 5052 scope.go:117] "RemoveContainer" containerID="48fa168d5d7691a3b2d8508087753330d79877c968cc2b4e0f021b0441800e44" Dec 03 12:22:05 crc kubenswrapper[5052]: I1203 12:22:05.643145 5052 scope.go:117] "RemoveContainer" containerID="ece24b1db9c0709fd516d1a80d474c2937fdeb49dd573043245a6e0d4d27d04a" Dec 03 12:22:05 crc kubenswrapper[5052]: I1203 12:22:05.643600 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/03bb65ef-e9a8-4cfb-9536-001e316b1f72-ovndb-tls-certs\") pod \"03bb65ef-e9a8-4cfb-9536-001e316b1f72\" (UID: \"03bb65ef-e9a8-4cfb-9536-001e316b1f72\") " Dec 03 12:22:05 crc kubenswrapper[5052]: E1203 12:22:05.643645 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ece24b1db9c0709fd516d1a80d474c2937fdeb49dd573043245a6e0d4d27d04a\": container with ID starting with ece24b1db9c0709fd516d1a80d474c2937fdeb49dd573043245a6e0d4d27d04a not found: ID does not exist" containerID="ece24b1db9c0709fd516d1a80d474c2937fdeb49dd573043245a6e0d4d27d04a" Dec 03 12:22:05 crc kubenswrapper[5052]: I1203 12:22:05.643681 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ece24b1db9c0709fd516d1a80d474c2937fdeb49dd573043245a6e0d4d27d04a"} err="failed to get container status \"ece24b1db9c0709fd516d1a80d474c2937fdeb49dd573043245a6e0d4d27d04a\": rpc error: code = NotFound desc = could not find container \"ece24b1db9c0709fd516d1a80d474c2937fdeb49dd573043245a6e0d4d27d04a\": container with ID starting with ece24b1db9c0709fd516d1a80d474c2937fdeb49dd573043245a6e0d4d27d04a not found: ID does not exist" Dec 03 12:22:05 crc kubenswrapper[5052]: I1203 12:22:05.643706 5052 scope.go:117] "RemoveContainer" containerID="48fa168d5d7691a3b2d8508087753330d79877c968cc2b4e0f021b0441800e44" Dec 03 12:22:05 crc kubenswrapper[5052]: I1203 12:22:05.643657 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/03bb65ef-e9a8-4cfb-9536-001e316b1f72-httpd-config\") pod \"03bb65ef-e9a8-4cfb-9536-001e316b1f72\" (UID: \"03bb65ef-e9a8-4cfb-9536-001e316b1f72\") " Dec 03 12:22:05 crc kubenswrapper[5052]: I1203 12:22:05.643789 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/03bb65ef-e9a8-4cfb-9536-001e316b1f72-combined-ca-bundle\") pod \"03bb65ef-e9a8-4cfb-9536-001e316b1f72\" (UID: \"03bb65ef-e9a8-4cfb-9536-001e316b1f72\") " Dec 03 12:22:05 crc kubenswrapper[5052]: I1203 12:22:05.643829 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/03bb65ef-e9a8-4cfb-9536-001e316b1f72-public-tls-certs\") pod \"03bb65ef-e9a8-4cfb-9536-001e316b1f72\" (UID: \"03bb65ef-e9a8-4cfb-9536-001e316b1f72\") " Dec 03 12:22:05 crc kubenswrapper[5052]: I1203 12:22:05.643878 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ck8dz\" (UniqueName: \"kubernetes.io/projected/03bb65ef-e9a8-4cfb-9536-001e316b1f72-kube-api-access-ck8dz\") pod \"03bb65ef-e9a8-4cfb-9536-001e316b1f72\" (UID: \"03bb65ef-e9a8-4cfb-9536-001e316b1f72\") " Dec 03 12:22:05 crc kubenswrapper[5052]: I1203 12:22:05.643919 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/03bb65ef-e9a8-4cfb-9536-001e316b1f72-internal-tls-certs\") pod \"03bb65ef-e9a8-4cfb-9536-001e316b1f72\" (UID: \"03bb65ef-e9a8-4cfb-9536-001e316b1f72\") " Dec 03 12:22:05 crc kubenswrapper[5052]: I1203 12:22:05.644013 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/03bb65ef-e9a8-4cfb-9536-001e316b1f72-config\") pod \"03bb65ef-e9a8-4cfb-9536-001e316b1f72\" (UID: \"03bb65ef-e9a8-4cfb-9536-001e316b1f72\") " Dec 03 12:22:05 crc kubenswrapper[5052]: E1203 12:22:05.644876 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"48fa168d5d7691a3b2d8508087753330d79877c968cc2b4e0f021b0441800e44\": container with ID starting with 48fa168d5d7691a3b2d8508087753330d79877c968cc2b4e0f021b0441800e44 not found: ID does not exist" containerID="48fa168d5d7691a3b2d8508087753330d79877c968cc2b4e0f021b0441800e44" Dec 03 12:22:05 crc kubenswrapper[5052]: I1203 12:22:05.644912 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"48fa168d5d7691a3b2d8508087753330d79877c968cc2b4e0f021b0441800e44"} err="failed to get container status \"48fa168d5d7691a3b2d8508087753330d79877c968cc2b4e0f021b0441800e44\": rpc error: code = NotFound desc = could not find container \"48fa168d5d7691a3b2d8508087753330d79877c968cc2b4e0f021b0441800e44\": container with ID starting with 48fa168d5d7691a3b2d8508087753330d79877c968cc2b4e0f021b0441800e44 not found: ID does not exist" Dec 03 12:22:05 crc kubenswrapper[5052]: I1203 12:22:05.648705 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/03bb65ef-e9a8-4cfb-9536-001e316b1f72-kube-api-access-ck8dz" (OuterVolumeSpecName: "kube-api-access-ck8dz") pod "03bb65ef-e9a8-4cfb-9536-001e316b1f72" (UID: "03bb65ef-e9a8-4cfb-9536-001e316b1f72"). InnerVolumeSpecName "kube-api-access-ck8dz". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:22:05 crc kubenswrapper[5052]: I1203 12:22:05.649665 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/03bb65ef-e9a8-4cfb-9536-001e316b1f72-httpd-config" (OuterVolumeSpecName: "httpd-config") pod "03bb65ef-e9a8-4cfb-9536-001e316b1f72" (UID: "03bb65ef-e9a8-4cfb-9536-001e316b1f72"). InnerVolumeSpecName "httpd-config". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:22:05 crc kubenswrapper[5052]: I1203 12:22:05.686645 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/03bb65ef-e9a8-4cfb-9536-001e316b1f72-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "03bb65ef-e9a8-4cfb-9536-001e316b1f72" (UID: "03bb65ef-e9a8-4cfb-9536-001e316b1f72"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:22:05 crc kubenswrapper[5052]: I1203 12:22:05.690299 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/03bb65ef-e9a8-4cfb-9536-001e316b1f72-config" (OuterVolumeSpecName: "config") pod "03bb65ef-e9a8-4cfb-9536-001e316b1f72" (UID: "03bb65ef-e9a8-4cfb-9536-001e316b1f72"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:22:05 crc kubenswrapper[5052]: I1203 12:22:05.691627 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/03bb65ef-e9a8-4cfb-9536-001e316b1f72-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "03bb65ef-e9a8-4cfb-9536-001e316b1f72" (UID: "03bb65ef-e9a8-4cfb-9536-001e316b1f72"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:22:05 crc kubenswrapper[5052]: I1203 12:22:05.695762 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/03bb65ef-e9a8-4cfb-9536-001e316b1f72-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "03bb65ef-e9a8-4cfb-9536-001e316b1f72" (UID: "03bb65ef-e9a8-4cfb-9536-001e316b1f72"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:22:05 crc kubenswrapper[5052]: I1203 12:22:05.705639 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/03bb65ef-e9a8-4cfb-9536-001e316b1f72-ovndb-tls-certs" (OuterVolumeSpecName: "ovndb-tls-certs") pod "03bb65ef-e9a8-4cfb-9536-001e316b1f72" (UID: "03bb65ef-e9a8-4cfb-9536-001e316b1f72"). InnerVolumeSpecName "ovndb-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:22:05 crc kubenswrapper[5052]: I1203 12:22:05.747352 5052 reconciler_common.go:293] "Volume detached for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/03bb65ef-e9a8-4cfb-9536-001e316b1f72-ovndb-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:22:05 crc kubenswrapper[5052]: I1203 12:22:05.747389 5052 reconciler_common.go:293] "Volume detached for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/03bb65ef-e9a8-4cfb-9536-001e316b1f72-httpd-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:22:05 crc kubenswrapper[5052]: I1203 12:22:05.747403 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/03bb65ef-e9a8-4cfb-9536-001e316b1f72-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:22:05 crc kubenswrapper[5052]: I1203 12:22:05.747413 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/03bb65ef-e9a8-4cfb-9536-001e316b1f72-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:22:05 crc kubenswrapper[5052]: I1203 12:22:05.747425 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-ck8dz\" (UniqueName: \"kubernetes.io/projected/03bb65ef-e9a8-4cfb-9536-001e316b1f72-kube-api-access-ck8dz\") on node \"crc\" DevicePath \"\"" Dec 03 12:22:05 crc kubenswrapper[5052]: I1203 12:22:05.747462 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/03bb65ef-e9a8-4cfb-9536-001e316b1f72-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:22:05 crc kubenswrapper[5052]: I1203 12:22:05.747474 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/03bb65ef-e9a8-4cfb-9536-001e316b1f72-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:22:05 crc kubenswrapper[5052]: I1203 12:22:05.967468 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/neutron-69c9545844-xf6zm"] Dec 03 12:22:05 crc kubenswrapper[5052]: I1203 12:22:05.976110 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/neutron-69c9545844-xf6zm"] Dec 03 12:22:07 crc kubenswrapper[5052]: I1203 12:22:07.682836 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="03bb65ef-e9a8-4cfb-9536-001e316b1f72" path="/var/lib/kubelet/pods/03bb65ef-e9a8-4cfb-9536-001e316b1f72/volumes" Dec 03 12:22:11 crc kubenswrapper[5052]: I1203 12:22:11.459283 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:22:11 crc kubenswrapper[5052]: I1203 12:22:11.641828 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/4dd73a47-cf2a-43b8-ba1c-0d47332edaa4-config-data\") pod \"4dd73a47-cf2a-43b8-ba1c-0d47332edaa4\" (UID: \"4dd73a47-cf2a-43b8-ba1c-0d47332edaa4\") " Dec 03 12:22:11 crc kubenswrapper[5052]: I1203 12:22:11.641916 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/4dd73a47-cf2a-43b8-ba1c-0d47332edaa4-logs\") pod \"4dd73a47-cf2a-43b8-ba1c-0d47332edaa4\" (UID: \"4dd73a47-cf2a-43b8-ba1c-0d47332edaa4\") " Dec 03 12:22:11 crc kubenswrapper[5052]: I1203 12:22:11.641949 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/4dd73a47-cf2a-43b8-ba1c-0d47332edaa4-nova-metadata-tls-certs\") pod \"4dd73a47-cf2a-43b8-ba1c-0d47332edaa4\" (UID: \"4dd73a47-cf2a-43b8-ba1c-0d47332edaa4\") " Dec 03 12:22:11 crc kubenswrapper[5052]: I1203 12:22:11.642100 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mdj9b\" (UniqueName: \"kubernetes.io/projected/4dd73a47-cf2a-43b8-ba1c-0d47332edaa4-kube-api-access-mdj9b\") pod \"4dd73a47-cf2a-43b8-ba1c-0d47332edaa4\" (UID: \"4dd73a47-cf2a-43b8-ba1c-0d47332edaa4\") " Dec 03 12:22:11 crc kubenswrapper[5052]: I1203 12:22:11.642154 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4dd73a47-cf2a-43b8-ba1c-0d47332edaa4-combined-ca-bundle\") pod \"4dd73a47-cf2a-43b8-ba1c-0d47332edaa4\" (UID: \"4dd73a47-cf2a-43b8-ba1c-0d47332edaa4\") " Dec 03 12:22:11 crc kubenswrapper[5052]: I1203 12:22:11.643272 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4dd73a47-cf2a-43b8-ba1c-0d47332edaa4-logs" (OuterVolumeSpecName: "logs") pod "4dd73a47-cf2a-43b8-ba1c-0d47332edaa4" (UID: "4dd73a47-cf2a-43b8-ba1c-0d47332edaa4"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:22:11 crc kubenswrapper[5052]: I1203 12:22:11.648211 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4dd73a47-cf2a-43b8-ba1c-0d47332edaa4-kube-api-access-mdj9b" (OuterVolumeSpecName: "kube-api-access-mdj9b") pod "4dd73a47-cf2a-43b8-ba1c-0d47332edaa4" (UID: "4dd73a47-cf2a-43b8-ba1c-0d47332edaa4"). InnerVolumeSpecName "kube-api-access-mdj9b". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:22:11 crc kubenswrapper[5052]: I1203 12:22:11.669365 5052 generic.go:334] "Generic (PLEG): container finished" podID="4dd73a47-cf2a-43b8-ba1c-0d47332edaa4" containerID="c9ed4b9b88de467e9c8e8554fc978077cfb8bd825b23cfae311340cd81ff7333" exitCode=137 Dec 03 12:22:11 crc kubenswrapper[5052]: I1203 12:22:11.669404 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"4dd73a47-cf2a-43b8-ba1c-0d47332edaa4","Type":"ContainerDied","Data":"c9ed4b9b88de467e9c8e8554fc978077cfb8bd825b23cfae311340cd81ff7333"} Dec 03 12:22:11 crc kubenswrapper[5052]: I1203 12:22:11.669430 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"4dd73a47-cf2a-43b8-ba1c-0d47332edaa4","Type":"ContainerDied","Data":"76b4d90822b2cea4dc663e9fb413320e7954c906bd78785f2f60c7e955624e00"} Dec 03 12:22:11 crc kubenswrapper[5052]: I1203 12:22:11.669446 5052 scope.go:117] "RemoveContainer" containerID="c9ed4b9b88de467e9c8e8554fc978077cfb8bd825b23cfae311340cd81ff7333" Dec 03 12:22:11 crc kubenswrapper[5052]: I1203 12:22:11.669577 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:22:11 crc kubenswrapper[5052]: I1203 12:22:11.671690 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4dd73a47-cf2a-43b8-ba1c-0d47332edaa4-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "4dd73a47-cf2a-43b8-ba1c-0d47332edaa4" (UID: "4dd73a47-cf2a-43b8-ba1c-0d47332edaa4"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:22:11 crc kubenswrapper[5052]: I1203 12:22:11.678611 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4dd73a47-cf2a-43b8-ba1c-0d47332edaa4-config-data" (OuterVolumeSpecName: "config-data") pod "4dd73a47-cf2a-43b8-ba1c-0d47332edaa4" (UID: "4dd73a47-cf2a-43b8-ba1c-0d47332edaa4"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:22:11 crc kubenswrapper[5052]: I1203 12:22:11.689481 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4dd73a47-cf2a-43b8-ba1c-0d47332edaa4-nova-metadata-tls-certs" (OuterVolumeSpecName: "nova-metadata-tls-certs") pod "4dd73a47-cf2a-43b8-ba1c-0d47332edaa4" (UID: "4dd73a47-cf2a-43b8-ba1c-0d47332edaa4"). InnerVolumeSpecName "nova-metadata-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:22:11 crc kubenswrapper[5052]: I1203 12:22:11.744062 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-mdj9b\" (UniqueName: \"kubernetes.io/projected/4dd73a47-cf2a-43b8-ba1c-0d47332edaa4-kube-api-access-mdj9b\") on node \"crc\" DevicePath \"\"" Dec 03 12:22:11 crc kubenswrapper[5052]: I1203 12:22:11.744096 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4dd73a47-cf2a-43b8-ba1c-0d47332edaa4-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:22:11 crc kubenswrapper[5052]: I1203 12:22:11.744106 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/4dd73a47-cf2a-43b8-ba1c-0d47332edaa4-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:22:11 crc kubenswrapper[5052]: I1203 12:22:11.744117 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/4dd73a47-cf2a-43b8-ba1c-0d47332edaa4-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:22:11 crc kubenswrapper[5052]: I1203 12:22:11.744128 5052 reconciler_common.go:293] "Volume detached for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/4dd73a47-cf2a-43b8-ba1c-0d47332edaa4-nova-metadata-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:22:11 crc kubenswrapper[5052]: I1203 12:22:11.779648 5052 scope.go:117] "RemoveContainer" containerID="dc90f1007187fa83ea0bb677d36177c1605dca45111209c9532edc07fddefddd" Dec 03 12:22:11 crc kubenswrapper[5052]: I1203 12:22:11.798692 5052 scope.go:117] "RemoveContainer" containerID="c9ed4b9b88de467e9c8e8554fc978077cfb8bd825b23cfae311340cd81ff7333" Dec 03 12:22:11 crc kubenswrapper[5052]: E1203 12:22:11.799205 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c9ed4b9b88de467e9c8e8554fc978077cfb8bd825b23cfae311340cd81ff7333\": container with ID starting with c9ed4b9b88de467e9c8e8554fc978077cfb8bd825b23cfae311340cd81ff7333 not found: ID does not exist" containerID="c9ed4b9b88de467e9c8e8554fc978077cfb8bd825b23cfae311340cd81ff7333" Dec 03 12:22:11 crc kubenswrapper[5052]: I1203 12:22:11.799239 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c9ed4b9b88de467e9c8e8554fc978077cfb8bd825b23cfae311340cd81ff7333"} err="failed to get container status \"c9ed4b9b88de467e9c8e8554fc978077cfb8bd825b23cfae311340cd81ff7333\": rpc error: code = NotFound desc = could not find container \"c9ed4b9b88de467e9c8e8554fc978077cfb8bd825b23cfae311340cd81ff7333\": container with ID starting with c9ed4b9b88de467e9c8e8554fc978077cfb8bd825b23cfae311340cd81ff7333 not found: ID does not exist" Dec 03 12:22:11 crc kubenswrapper[5052]: I1203 12:22:11.799266 5052 scope.go:117] "RemoveContainer" containerID="dc90f1007187fa83ea0bb677d36177c1605dca45111209c9532edc07fddefddd" Dec 03 12:22:11 crc kubenswrapper[5052]: E1203 12:22:11.799550 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dc90f1007187fa83ea0bb677d36177c1605dca45111209c9532edc07fddefddd\": container with ID starting with dc90f1007187fa83ea0bb677d36177c1605dca45111209c9532edc07fddefddd not found: ID does not exist" containerID="dc90f1007187fa83ea0bb677d36177c1605dca45111209c9532edc07fddefddd" Dec 03 12:22:11 crc kubenswrapper[5052]: I1203 12:22:11.799579 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dc90f1007187fa83ea0bb677d36177c1605dca45111209c9532edc07fddefddd"} err="failed to get container status \"dc90f1007187fa83ea0bb677d36177c1605dca45111209c9532edc07fddefddd\": rpc error: code = NotFound desc = could not find container \"dc90f1007187fa83ea0bb677d36177c1605dca45111209c9532edc07fddefddd\": container with ID starting with dc90f1007187fa83ea0bb677d36177c1605dca45111209c9532edc07fddefddd not found: ID does not exist" Dec 03 12:22:11 crc kubenswrapper[5052]: I1203 12:22:11.995018 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:22:12 crc kubenswrapper[5052]: I1203 12:22:12.000409 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:22:13 crc kubenswrapper[5052]: I1203 12:22:13.413809 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:22:13 crc kubenswrapper[5052]: I1203 12:22:13.478359 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/0b121234-d24f-4307-9409-59a7dad3df4c-etc-swift\") pod \"0b121234-d24f-4307-9409-59a7dad3df4c\" (UID: \"0b121234-d24f-4307-9409-59a7dad3df4c\") " Dec 03 12:22:13 crc kubenswrapper[5052]: I1203 12:22:13.478420 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"swift\" (UniqueName: \"kubernetes.io/local-volume/local-storage05-crc\") pod \"0b121234-d24f-4307-9409-59a7dad3df4c\" (UID: \"0b121234-d24f-4307-9409-59a7dad3df4c\") " Dec 03 12:22:13 crc kubenswrapper[5052]: I1203 12:22:13.478481 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"cache\" (UniqueName: \"kubernetes.io/empty-dir/0b121234-d24f-4307-9409-59a7dad3df4c-cache\") pod \"0b121234-d24f-4307-9409-59a7dad3df4c\" (UID: \"0b121234-d24f-4307-9409-59a7dad3df4c\") " Dec 03 12:22:13 crc kubenswrapper[5052]: I1203 12:22:13.478573 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-88lpb\" (UniqueName: \"kubernetes.io/projected/0b121234-d24f-4307-9409-59a7dad3df4c-kube-api-access-88lpb\") pod \"0b121234-d24f-4307-9409-59a7dad3df4c\" (UID: \"0b121234-d24f-4307-9409-59a7dad3df4c\") " Dec 03 12:22:13 crc kubenswrapper[5052]: I1203 12:22:13.478604 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"lock\" (UniqueName: \"kubernetes.io/empty-dir/0b121234-d24f-4307-9409-59a7dad3df4c-lock\") pod \"0b121234-d24f-4307-9409-59a7dad3df4c\" (UID: \"0b121234-d24f-4307-9409-59a7dad3df4c\") " Dec 03 12:22:13 crc kubenswrapper[5052]: I1203 12:22:13.478952 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0b121234-d24f-4307-9409-59a7dad3df4c-cache" (OuterVolumeSpecName: "cache") pod "0b121234-d24f-4307-9409-59a7dad3df4c" (UID: "0b121234-d24f-4307-9409-59a7dad3df4c"). InnerVolumeSpecName "cache". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:22:13 crc kubenswrapper[5052]: I1203 12:22:13.479198 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0b121234-d24f-4307-9409-59a7dad3df4c-lock" (OuterVolumeSpecName: "lock") pod "0b121234-d24f-4307-9409-59a7dad3df4c" (UID: "0b121234-d24f-4307-9409-59a7dad3df4c"). InnerVolumeSpecName "lock". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:22:13 crc kubenswrapper[5052]: I1203 12:22:13.483432 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0b121234-d24f-4307-9409-59a7dad3df4c-etc-swift" (OuterVolumeSpecName: "etc-swift") pod "0b121234-d24f-4307-9409-59a7dad3df4c" (UID: "0b121234-d24f-4307-9409-59a7dad3df4c"). InnerVolumeSpecName "etc-swift". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:22:13 crc kubenswrapper[5052]: I1203 12:22:13.483547 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0b121234-d24f-4307-9409-59a7dad3df4c-kube-api-access-88lpb" (OuterVolumeSpecName: "kube-api-access-88lpb") pod "0b121234-d24f-4307-9409-59a7dad3df4c" (UID: "0b121234-d24f-4307-9409-59a7dad3df4c"). InnerVolumeSpecName "kube-api-access-88lpb". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:22:13 crc kubenswrapper[5052]: I1203 12:22:13.483596 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage05-crc" (OuterVolumeSpecName: "swift") pod "0b121234-d24f-4307-9409-59a7dad3df4c" (UID: "0b121234-d24f-4307-9409-59a7dad3df4c"). InnerVolumeSpecName "local-storage05-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 12:22:13 crc kubenswrapper[5052]: I1203 12:22:13.580271 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-88lpb\" (UniqueName: \"kubernetes.io/projected/0b121234-d24f-4307-9409-59a7dad3df4c-kube-api-access-88lpb\") on node \"crc\" DevicePath \"\"" Dec 03 12:22:13 crc kubenswrapper[5052]: I1203 12:22:13.580303 5052 reconciler_common.go:293] "Volume detached for volume \"lock\" (UniqueName: \"kubernetes.io/empty-dir/0b121234-d24f-4307-9409-59a7dad3df4c-lock\") on node \"crc\" DevicePath \"\"" Dec 03 12:22:13 crc kubenswrapper[5052]: I1203 12:22:13.580312 5052 reconciler_common.go:293] "Volume detached for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/0b121234-d24f-4307-9409-59a7dad3df4c-etc-swift\") on node \"crc\" DevicePath \"\"" Dec 03 12:22:13 crc kubenswrapper[5052]: I1203 12:22:13.580344 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage05-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage05-crc\") on node \"crc\" " Dec 03 12:22:13 crc kubenswrapper[5052]: I1203 12:22:13.580354 5052 reconciler_common.go:293] "Volume detached for volume \"cache\" (UniqueName: \"kubernetes.io/empty-dir/0b121234-d24f-4307-9409-59a7dad3df4c-cache\") on node \"crc\" DevicePath \"\"" Dec 03 12:22:13 crc kubenswrapper[5052]: I1203 12:22:13.599424 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage05-crc" (UniqueName: "kubernetes.io/local-volume/local-storage05-crc") on node "crc" Dec 03 12:22:13 crc kubenswrapper[5052]: I1203 12:22:13.682050 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage05-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage05-crc\") on node \"crc\" DevicePath \"\"" Dec 03 12:22:13 crc kubenswrapper[5052]: I1203 12:22:13.689994 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4dd73a47-cf2a-43b8-ba1c-0d47332edaa4" path="/var/lib/kubelet/pods/4dd73a47-cf2a-43b8-ba1c-0d47332edaa4/volumes" Dec 03 12:22:13 crc kubenswrapper[5052]: I1203 12:22:13.701135 5052 generic.go:334] "Generic (PLEG): container finished" podID="0b121234-d24f-4307-9409-59a7dad3df4c" containerID="e63a7b1f0f328636f34f8466d7114142477116b8ba39ff4c63590f6d3d2ecb61" exitCode=137 Dec 03 12:22:13 crc kubenswrapper[5052]: I1203 12:22:13.701180 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"0b121234-d24f-4307-9409-59a7dad3df4c","Type":"ContainerDied","Data":"e63a7b1f0f328636f34f8466d7114142477116b8ba39ff4c63590f6d3d2ecb61"} Dec 03 12:22:13 crc kubenswrapper[5052]: I1203 12:22:13.701213 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"0b121234-d24f-4307-9409-59a7dad3df4c","Type":"ContainerDied","Data":"0406d9b3dde0ec3ac2702085cbbb703bed2f65a26230fed9c8d699555d0cadb8"} Dec 03 12:22:13 crc kubenswrapper[5052]: I1203 12:22:13.701230 5052 scope.go:117] "RemoveContainer" containerID="e63a7b1f0f328636f34f8466d7114142477116b8ba39ff4c63590f6d3d2ecb61" Dec 03 12:22:13 crc kubenswrapper[5052]: I1203 12:22:13.701415 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:22:13 crc kubenswrapper[5052]: I1203 12:22:13.731190 5052 scope.go:117] "RemoveContainer" containerID="eac0ccbf31e29898c07277ff8b3f25e720d684db3945841e058ea63b8f2fc50e" Dec 03 12:22:13 crc kubenswrapper[5052]: I1203 12:22:13.753501 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/swift-storage-0"] Dec 03 12:22:13 crc kubenswrapper[5052]: I1203 12:22:13.759341 5052 scope.go:117] "RemoveContainer" containerID="9826c564d51675cc0d19bfb280f54c6cdb8da70e11a2735f3456a7c16c5b0331" Dec 03 12:22:13 crc kubenswrapper[5052]: I1203 12:22:13.760653 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/swift-storage-0"] Dec 03 12:22:13 crc kubenswrapper[5052]: I1203 12:22:13.779287 5052 scope.go:117] "RemoveContainer" containerID="38ac8f54d7f675400878ab50b896bde2d4d40ee63497af23ebdca0f75d215cd7" Dec 03 12:22:13 crc kubenswrapper[5052]: I1203 12:22:13.801789 5052 scope.go:117] "RemoveContainer" containerID="a108f5ccd3086ba9eada83058d904fc18a5b2e825c993e540cb30d0f9ced71e9" Dec 03 12:22:13 crc kubenswrapper[5052]: I1203 12:22:13.820225 5052 scope.go:117] "RemoveContainer" containerID="a7be0eed6906ff248e67daeb0cb8433a1ab73cf3e47159ff1cbfe4029900aa36" Dec 03 12:22:13 crc kubenswrapper[5052]: I1203 12:22:13.839793 5052 scope.go:117] "RemoveContainer" containerID="93d4d6d903155179da2cc815475bcc4c3e646bd25173aec67f526949d47734f0" Dec 03 12:22:13 crc kubenswrapper[5052]: I1203 12:22:13.871145 5052 scope.go:117] "RemoveContainer" containerID="280a6f9eeba9635ba75d3f89698a54d959866a87a84e1c34632aae63f6162f88" Dec 03 12:22:13 crc kubenswrapper[5052]: I1203 12:22:13.892934 5052 scope.go:117] "RemoveContainer" containerID="76aa0661476cea8be8d11604d15492437f317021e6ae641e33f6ca912d72fe2e" Dec 03 12:22:13 crc kubenswrapper[5052]: I1203 12:22:13.909428 5052 scope.go:117] "RemoveContainer" containerID="1fb344079bdd4d88652a84cde52ac882ef46f1e1b483946b91fa06958b9781b1" Dec 03 12:22:13 crc kubenswrapper[5052]: I1203 12:22:13.928284 5052 scope.go:117] "RemoveContainer" containerID="7448a246780facb87faed946ab976ad223f836a90fd0ac6cbf876c7f11c6c2d8" Dec 03 12:22:13 crc kubenswrapper[5052]: I1203 12:22:13.954935 5052 scope.go:117] "RemoveContainer" containerID="70f7ea9905786e034fff8dcf46682040188eb7ca7903f795173a38888c5b11d9" Dec 03 12:22:13 crc kubenswrapper[5052]: I1203 12:22:13.979107 5052 scope.go:117] "RemoveContainer" containerID="1de28481ebd24dff85a8213026dad78dd91bf4d0b6c4d77c2b5231b95244a4cb" Dec 03 12:22:13 crc kubenswrapper[5052]: I1203 12:22:13.999557 5052 scope.go:117] "RemoveContainer" containerID="914125fe1587ea7c446a5334d9e2828cf4834d6ad4f2053f842f07ad24a0e67d" Dec 03 12:22:14 crc kubenswrapper[5052]: I1203 12:22:14.024308 5052 scope.go:117] "RemoveContainer" containerID="6b7c8f662acc40a3137f223890e54394fd6013bb9203453b7285ac28a63f277d" Dec 03 12:22:14 crc kubenswrapper[5052]: I1203 12:22:14.059248 5052 scope.go:117] "RemoveContainer" containerID="e63a7b1f0f328636f34f8466d7114142477116b8ba39ff4c63590f6d3d2ecb61" Dec 03 12:22:14 crc kubenswrapper[5052]: E1203 12:22:14.060142 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e63a7b1f0f328636f34f8466d7114142477116b8ba39ff4c63590f6d3d2ecb61\": container with ID starting with e63a7b1f0f328636f34f8466d7114142477116b8ba39ff4c63590f6d3d2ecb61 not found: ID does not exist" containerID="e63a7b1f0f328636f34f8466d7114142477116b8ba39ff4c63590f6d3d2ecb61" Dec 03 12:22:14 crc kubenswrapper[5052]: I1203 12:22:14.060175 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e63a7b1f0f328636f34f8466d7114142477116b8ba39ff4c63590f6d3d2ecb61"} err="failed to get container status \"e63a7b1f0f328636f34f8466d7114142477116b8ba39ff4c63590f6d3d2ecb61\": rpc error: code = NotFound desc = could not find container \"e63a7b1f0f328636f34f8466d7114142477116b8ba39ff4c63590f6d3d2ecb61\": container with ID starting with e63a7b1f0f328636f34f8466d7114142477116b8ba39ff4c63590f6d3d2ecb61 not found: ID does not exist" Dec 03 12:22:14 crc kubenswrapper[5052]: I1203 12:22:14.060198 5052 scope.go:117] "RemoveContainer" containerID="eac0ccbf31e29898c07277ff8b3f25e720d684db3945841e058ea63b8f2fc50e" Dec 03 12:22:14 crc kubenswrapper[5052]: E1203 12:22:14.060613 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eac0ccbf31e29898c07277ff8b3f25e720d684db3945841e058ea63b8f2fc50e\": container with ID starting with eac0ccbf31e29898c07277ff8b3f25e720d684db3945841e058ea63b8f2fc50e not found: ID does not exist" containerID="eac0ccbf31e29898c07277ff8b3f25e720d684db3945841e058ea63b8f2fc50e" Dec 03 12:22:14 crc kubenswrapper[5052]: I1203 12:22:14.060655 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eac0ccbf31e29898c07277ff8b3f25e720d684db3945841e058ea63b8f2fc50e"} err="failed to get container status \"eac0ccbf31e29898c07277ff8b3f25e720d684db3945841e058ea63b8f2fc50e\": rpc error: code = NotFound desc = could not find container \"eac0ccbf31e29898c07277ff8b3f25e720d684db3945841e058ea63b8f2fc50e\": container with ID starting with eac0ccbf31e29898c07277ff8b3f25e720d684db3945841e058ea63b8f2fc50e not found: ID does not exist" Dec 03 12:22:14 crc kubenswrapper[5052]: I1203 12:22:14.060685 5052 scope.go:117] "RemoveContainer" containerID="9826c564d51675cc0d19bfb280f54c6cdb8da70e11a2735f3456a7c16c5b0331" Dec 03 12:22:14 crc kubenswrapper[5052]: E1203 12:22:14.061087 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9826c564d51675cc0d19bfb280f54c6cdb8da70e11a2735f3456a7c16c5b0331\": container with ID starting with 9826c564d51675cc0d19bfb280f54c6cdb8da70e11a2735f3456a7c16c5b0331 not found: ID does not exist" containerID="9826c564d51675cc0d19bfb280f54c6cdb8da70e11a2735f3456a7c16c5b0331" Dec 03 12:22:14 crc kubenswrapper[5052]: I1203 12:22:14.061115 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9826c564d51675cc0d19bfb280f54c6cdb8da70e11a2735f3456a7c16c5b0331"} err="failed to get container status \"9826c564d51675cc0d19bfb280f54c6cdb8da70e11a2735f3456a7c16c5b0331\": rpc error: code = NotFound desc = could not find container \"9826c564d51675cc0d19bfb280f54c6cdb8da70e11a2735f3456a7c16c5b0331\": container with ID starting with 9826c564d51675cc0d19bfb280f54c6cdb8da70e11a2735f3456a7c16c5b0331 not found: ID does not exist" Dec 03 12:22:14 crc kubenswrapper[5052]: I1203 12:22:14.061132 5052 scope.go:117] "RemoveContainer" containerID="38ac8f54d7f675400878ab50b896bde2d4d40ee63497af23ebdca0f75d215cd7" Dec 03 12:22:14 crc kubenswrapper[5052]: E1203 12:22:14.061716 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"38ac8f54d7f675400878ab50b896bde2d4d40ee63497af23ebdca0f75d215cd7\": container with ID starting with 38ac8f54d7f675400878ab50b896bde2d4d40ee63497af23ebdca0f75d215cd7 not found: ID does not exist" containerID="38ac8f54d7f675400878ab50b896bde2d4d40ee63497af23ebdca0f75d215cd7" Dec 03 12:22:14 crc kubenswrapper[5052]: I1203 12:22:14.061740 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"38ac8f54d7f675400878ab50b896bde2d4d40ee63497af23ebdca0f75d215cd7"} err="failed to get container status \"38ac8f54d7f675400878ab50b896bde2d4d40ee63497af23ebdca0f75d215cd7\": rpc error: code = NotFound desc = could not find container \"38ac8f54d7f675400878ab50b896bde2d4d40ee63497af23ebdca0f75d215cd7\": container with ID starting with 38ac8f54d7f675400878ab50b896bde2d4d40ee63497af23ebdca0f75d215cd7 not found: ID does not exist" Dec 03 12:22:14 crc kubenswrapper[5052]: I1203 12:22:14.061754 5052 scope.go:117] "RemoveContainer" containerID="a108f5ccd3086ba9eada83058d904fc18a5b2e825c993e540cb30d0f9ced71e9" Dec 03 12:22:14 crc kubenswrapper[5052]: E1203 12:22:14.062313 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a108f5ccd3086ba9eada83058d904fc18a5b2e825c993e540cb30d0f9ced71e9\": container with ID starting with a108f5ccd3086ba9eada83058d904fc18a5b2e825c993e540cb30d0f9ced71e9 not found: ID does not exist" containerID="a108f5ccd3086ba9eada83058d904fc18a5b2e825c993e540cb30d0f9ced71e9" Dec 03 12:22:14 crc kubenswrapper[5052]: I1203 12:22:14.062336 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a108f5ccd3086ba9eada83058d904fc18a5b2e825c993e540cb30d0f9ced71e9"} err="failed to get container status \"a108f5ccd3086ba9eada83058d904fc18a5b2e825c993e540cb30d0f9ced71e9\": rpc error: code = NotFound desc = could not find container \"a108f5ccd3086ba9eada83058d904fc18a5b2e825c993e540cb30d0f9ced71e9\": container with ID starting with a108f5ccd3086ba9eada83058d904fc18a5b2e825c993e540cb30d0f9ced71e9 not found: ID does not exist" Dec 03 12:22:14 crc kubenswrapper[5052]: I1203 12:22:14.062352 5052 scope.go:117] "RemoveContainer" containerID="a7be0eed6906ff248e67daeb0cb8433a1ab73cf3e47159ff1cbfe4029900aa36" Dec 03 12:22:14 crc kubenswrapper[5052]: E1203 12:22:14.062640 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a7be0eed6906ff248e67daeb0cb8433a1ab73cf3e47159ff1cbfe4029900aa36\": container with ID starting with a7be0eed6906ff248e67daeb0cb8433a1ab73cf3e47159ff1cbfe4029900aa36 not found: ID does not exist" containerID="a7be0eed6906ff248e67daeb0cb8433a1ab73cf3e47159ff1cbfe4029900aa36" Dec 03 12:22:14 crc kubenswrapper[5052]: I1203 12:22:14.062662 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a7be0eed6906ff248e67daeb0cb8433a1ab73cf3e47159ff1cbfe4029900aa36"} err="failed to get container status \"a7be0eed6906ff248e67daeb0cb8433a1ab73cf3e47159ff1cbfe4029900aa36\": rpc error: code = NotFound desc = could not find container \"a7be0eed6906ff248e67daeb0cb8433a1ab73cf3e47159ff1cbfe4029900aa36\": container with ID starting with a7be0eed6906ff248e67daeb0cb8433a1ab73cf3e47159ff1cbfe4029900aa36 not found: ID does not exist" Dec 03 12:22:14 crc kubenswrapper[5052]: I1203 12:22:14.062677 5052 scope.go:117] "RemoveContainer" containerID="93d4d6d903155179da2cc815475bcc4c3e646bd25173aec67f526949d47734f0" Dec 03 12:22:14 crc kubenswrapper[5052]: E1203 12:22:14.063242 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"93d4d6d903155179da2cc815475bcc4c3e646bd25173aec67f526949d47734f0\": container with ID starting with 93d4d6d903155179da2cc815475bcc4c3e646bd25173aec67f526949d47734f0 not found: ID does not exist" containerID="93d4d6d903155179da2cc815475bcc4c3e646bd25173aec67f526949d47734f0" Dec 03 12:22:14 crc kubenswrapper[5052]: I1203 12:22:14.063272 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"93d4d6d903155179da2cc815475bcc4c3e646bd25173aec67f526949d47734f0"} err="failed to get container status \"93d4d6d903155179da2cc815475bcc4c3e646bd25173aec67f526949d47734f0\": rpc error: code = NotFound desc = could not find container \"93d4d6d903155179da2cc815475bcc4c3e646bd25173aec67f526949d47734f0\": container with ID starting with 93d4d6d903155179da2cc815475bcc4c3e646bd25173aec67f526949d47734f0 not found: ID does not exist" Dec 03 12:22:14 crc kubenswrapper[5052]: I1203 12:22:14.063288 5052 scope.go:117] "RemoveContainer" containerID="280a6f9eeba9635ba75d3f89698a54d959866a87a84e1c34632aae63f6162f88" Dec 03 12:22:14 crc kubenswrapper[5052]: E1203 12:22:14.064237 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"280a6f9eeba9635ba75d3f89698a54d959866a87a84e1c34632aae63f6162f88\": container with ID starting with 280a6f9eeba9635ba75d3f89698a54d959866a87a84e1c34632aae63f6162f88 not found: ID does not exist" containerID="280a6f9eeba9635ba75d3f89698a54d959866a87a84e1c34632aae63f6162f88" Dec 03 12:22:14 crc kubenswrapper[5052]: I1203 12:22:14.064288 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"280a6f9eeba9635ba75d3f89698a54d959866a87a84e1c34632aae63f6162f88"} err="failed to get container status \"280a6f9eeba9635ba75d3f89698a54d959866a87a84e1c34632aae63f6162f88\": rpc error: code = NotFound desc = could not find container \"280a6f9eeba9635ba75d3f89698a54d959866a87a84e1c34632aae63f6162f88\": container with ID starting with 280a6f9eeba9635ba75d3f89698a54d959866a87a84e1c34632aae63f6162f88 not found: ID does not exist" Dec 03 12:22:14 crc kubenswrapper[5052]: I1203 12:22:14.064319 5052 scope.go:117] "RemoveContainer" containerID="76aa0661476cea8be8d11604d15492437f317021e6ae641e33f6ca912d72fe2e" Dec 03 12:22:14 crc kubenswrapper[5052]: E1203 12:22:14.065194 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"76aa0661476cea8be8d11604d15492437f317021e6ae641e33f6ca912d72fe2e\": container with ID starting with 76aa0661476cea8be8d11604d15492437f317021e6ae641e33f6ca912d72fe2e not found: ID does not exist" containerID="76aa0661476cea8be8d11604d15492437f317021e6ae641e33f6ca912d72fe2e" Dec 03 12:22:14 crc kubenswrapper[5052]: I1203 12:22:14.065248 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"76aa0661476cea8be8d11604d15492437f317021e6ae641e33f6ca912d72fe2e"} err="failed to get container status \"76aa0661476cea8be8d11604d15492437f317021e6ae641e33f6ca912d72fe2e\": rpc error: code = NotFound desc = could not find container \"76aa0661476cea8be8d11604d15492437f317021e6ae641e33f6ca912d72fe2e\": container with ID starting with 76aa0661476cea8be8d11604d15492437f317021e6ae641e33f6ca912d72fe2e not found: ID does not exist" Dec 03 12:22:14 crc kubenswrapper[5052]: I1203 12:22:14.065271 5052 scope.go:117] "RemoveContainer" containerID="1fb344079bdd4d88652a84cde52ac882ef46f1e1b483946b91fa06958b9781b1" Dec 03 12:22:14 crc kubenswrapper[5052]: E1203 12:22:14.066062 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1fb344079bdd4d88652a84cde52ac882ef46f1e1b483946b91fa06958b9781b1\": container with ID starting with 1fb344079bdd4d88652a84cde52ac882ef46f1e1b483946b91fa06958b9781b1 not found: ID does not exist" containerID="1fb344079bdd4d88652a84cde52ac882ef46f1e1b483946b91fa06958b9781b1" Dec 03 12:22:14 crc kubenswrapper[5052]: I1203 12:22:14.066088 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1fb344079bdd4d88652a84cde52ac882ef46f1e1b483946b91fa06958b9781b1"} err="failed to get container status \"1fb344079bdd4d88652a84cde52ac882ef46f1e1b483946b91fa06958b9781b1\": rpc error: code = NotFound desc = could not find container \"1fb344079bdd4d88652a84cde52ac882ef46f1e1b483946b91fa06958b9781b1\": container with ID starting with 1fb344079bdd4d88652a84cde52ac882ef46f1e1b483946b91fa06958b9781b1 not found: ID does not exist" Dec 03 12:22:14 crc kubenswrapper[5052]: I1203 12:22:14.066104 5052 scope.go:117] "RemoveContainer" containerID="7448a246780facb87faed946ab976ad223f836a90fd0ac6cbf876c7f11c6c2d8" Dec 03 12:22:14 crc kubenswrapper[5052]: E1203 12:22:14.066424 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7448a246780facb87faed946ab976ad223f836a90fd0ac6cbf876c7f11c6c2d8\": container with ID starting with 7448a246780facb87faed946ab976ad223f836a90fd0ac6cbf876c7f11c6c2d8 not found: ID does not exist" containerID="7448a246780facb87faed946ab976ad223f836a90fd0ac6cbf876c7f11c6c2d8" Dec 03 12:22:14 crc kubenswrapper[5052]: I1203 12:22:14.066451 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7448a246780facb87faed946ab976ad223f836a90fd0ac6cbf876c7f11c6c2d8"} err="failed to get container status \"7448a246780facb87faed946ab976ad223f836a90fd0ac6cbf876c7f11c6c2d8\": rpc error: code = NotFound desc = could not find container \"7448a246780facb87faed946ab976ad223f836a90fd0ac6cbf876c7f11c6c2d8\": container with ID starting with 7448a246780facb87faed946ab976ad223f836a90fd0ac6cbf876c7f11c6c2d8 not found: ID does not exist" Dec 03 12:22:14 crc kubenswrapper[5052]: I1203 12:22:14.066683 5052 scope.go:117] "RemoveContainer" containerID="70f7ea9905786e034fff8dcf46682040188eb7ca7903f795173a38888c5b11d9" Dec 03 12:22:14 crc kubenswrapper[5052]: E1203 12:22:14.068098 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"70f7ea9905786e034fff8dcf46682040188eb7ca7903f795173a38888c5b11d9\": container with ID starting with 70f7ea9905786e034fff8dcf46682040188eb7ca7903f795173a38888c5b11d9 not found: ID does not exist" containerID="70f7ea9905786e034fff8dcf46682040188eb7ca7903f795173a38888c5b11d9" Dec 03 12:22:14 crc kubenswrapper[5052]: I1203 12:22:14.068125 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"70f7ea9905786e034fff8dcf46682040188eb7ca7903f795173a38888c5b11d9"} err="failed to get container status \"70f7ea9905786e034fff8dcf46682040188eb7ca7903f795173a38888c5b11d9\": rpc error: code = NotFound desc = could not find container \"70f7ea9905786e034fff8dcf46682040188eb7ca7903f795173a38888c5b11d9\": container with ID starting with 70f7ea9905786e034fff8dcf46682040188eb7ca7903f795173a38888c5b11d9 not found: ID does not exist" Dec 03 12:22:14 crc kubenswrapper[5052]: I1203 12:22:14.068142 5052 scope.go:117] "RemoveContainer" containerID="1de28481ebd24dff85a8213026dad78dd91bf4d0b6c4d77c2b5231b95244a4cb" Dec 03 12:22:14 crc kubenswrapper[5052]: E1203 12:22:14.068881 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1de28481ebd24dff85a8213026dad78dd91bf4d0b6c4d77c2b5231b95244a4cb\": container with ID starting with 1de28481ebd24dff85a8213026dad78dd91bf4d0b6c4d77c2b5231b95244a4cb not found: ID does not exist" containerID="1de28481ebd24dff85a8213026dad78dd91bf4d0b6c4d77c2b5231b95244a4cb" Dec 03 12:22:14 crc kubenswrapper[5052]: I1203 12:22:14.068932 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1de28481ebd24dff85a8213026dad78dd91bf4d0b6c4d77c2b5231b95244a4cb"} err="failed to get container status \"1de28481ebd24dff85a8213026dad78dd91bf4d0b6c4d77c2b5231b95244a4cb\": rpc error: code = NotFound desc = could not find container \"1de28481ebd24dff85a8213026dad78dd91bf4d0b6c4d77c2b5231b95244a4cb\": container with ID starting with 1de28481ebd24dff85a8213026dad78dd91bf4d0b6c4d77c2b5231b95244a4cb not found: ID does not exist" Dec 03 12:22:14 crc kubenswrapper[5052]: I1203 12:22:14.068950 5052 scope.go:117] "RemoveContainer" containerID="914125fe1587ea7c446a5334d9e2828cf4834d6ad4f2053f842f07ad24a0e67d" Dec 03 12:22:14 crc kubenswrapper[5052]: E1203 12:22:14.069723 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"914125fe1587ea7c446a5334d9e2828cf4834d6ad4f2053f842f07ad24a0e67d\": container with ID starting with 914125fe1587ea7c446a5334d9e2828cf4834d6ad4f2053f842f07ad24a0e67d not found: ID does not exist" containerID="914125fe1587ea7c446a5334d9e2828cf4834d6ad4f2053f842f07ad24a0e67d" Dec 03 12:22:14 crc kubenswrapper[5052]: I1203 12:22:14.069769 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"914125fe1587ea7c446a5334d9e2828cf4834d6ad4f2053f842f07ad24a0e67d"} err="failed to get container status \"914125fe1587ea7c446a5334d9e2828cf4834d6ad4f2053f842f07ad24a0e67d\": rpc error: code = NotFound desc = could not find container \"914125fe1587ea7c446a5334d9e2828cf4834d6ad4f2053f842f07ad24a0e67d\": container with ID starting with 914125fe1587ea7c446a5334d9e2828cf4834d6ad4f2053f842f07ad24a0e67d not found: ID does not exist" Dec 03 12:22:14 crc kubenswrapper[5052]: I1203 12:22:14.069803 5052 scope.go:117] "RemoveContainer" containerID="6b7c8f662acc40a3137f223890e54394fd6013bb9203453b7285ac28a63f277d" Dec 03 12:22:14 crc kubenswrapper[5052]: E1203 12:22:14.070130 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6b7c8f662acc40a3137f223890e54394fd6013bb9203453b7285ac28a63f277d\": container with ID starting with 6b7c8f662acc40a3137f223890e54394fd6013bb9203453b7285ac28a63f277d not found: ID does not exist" containerID="6b7c8f662acc40a3137f223890e54394fd6013bb9203453b7285ac28a63f277d" Dec 03 12:22:14 crc kubenswrapper[5052]: I1203 12:22:14.070154 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6b7c8f662acc40a3137f223890e54394fd6013bb9203453b7285ac28a63f277d"} err="failed to get container status \"6b7c8f662acc40a3137f223890e54394fd6013bb9203453b7285ac28a63f277d\": rpc error: code = NotFound desc = could not find container \"6b7c8f662acc40a3137f223890e54394fd6013bb9203453b7285ac28a63f277d\": container with ID starting with 6b7c8f662acc40a3137f223890e54394fd6013bb9203453b7285ac28a63f277d not found: ID does not exist" Dec 03 12:22:14 crc kubenswrapper[5052]: I1203 12:22:14.673912 5052 scope.go:117] "RemoveContainer" containerID="09b4a0072741f60499d24f40cc4ff3406d5d22671ced431363ab865172786300" Dec 03 12:22:14 crc kubenswrapper[5052]: E1203 12:22:14.675375 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 12:22:15 crc kubenswrapper[5052]: I1203 12:22:15.702188 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0b121234-d24f-4307-9409-59a7dad3df4c" path="/var/lib/kubelet/pods/0b121234-d24f-4307-9409-59a7dad3df4c/volumes" Dec 03 12:22:16 crc kubenswrapper[5052]: E1203 12:22:16.427452 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:22:16 crc kubenswrapper[5052]: E1203 12:22:16.428407 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/817345c8-3a42-4dea-8158-75973d9c41d9-operator-scripts podName:817345c8-3a42-4dea-8158-75973d9c41d9 nodeName:}" failed. No retries permitted until 2025-12-03 12:22:48.428384346 +0000 UTC m=+3759.016626110 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/817345c8-3a42-4dea-8158-75973d9c41d9-operator-scripts") pod "neutron-734a-account-create-update-qcgft" (UID: "817345c8-3a42-4dea-8158-75973d9c41d9") : configmap "openstack-scripts" not found Dec 03 12:22:16 crc kubenswrapper[5052]: E1203 12:22:16.528842 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:22:16 crc kubenswrapper[5052]: E1203 12:22:16.528920 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/db31774e-8f50-4862-86f3-93cb93f645a0-operator-scripts podName:db31774e-8f50-4862-86f3-93cb93f645a0 nodeName:}" failed. No retries permitted until 2025-12-03 12:22:48.528906435 +0000 UTC m=+3759.117148199 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/db31774e-8f50-4862-86f3-93cb93f645a0-operator-scripts") pod "glance73ca-account-delete-vpqhp" (UID: "db31774e-8f50-4862-86f3-93cb93f645a0") : configmap "openstack-scripts" not found Dec 03 12:22:16 crc kubenswrapper[5052]: I1203 12:22:16.581160 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement9152-account-delete-x7q46" Dec 03 12:22:16 crc kubenswrapper[5052]: I1203 12:22:16.730975 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qrtcw\" (UniqueName: \"kubernetes.io/projected/4f7ab0ec-52a7-4be2-9886-04bf98989b63-kube-api-access-qrtcw\") pod \"4f7ab0ec-52a7-4be2-9886-04bf98989b63\" (UID: \"4f7ab0ec-52a7-4be2-9886-04bf98989b63\") " Dec 03 12:22:16 crc kubenswrapper[5052]: I1203 12:22:16.731071 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/4f7ab0ec-52a7-4be2-9886-04bf98989b63-operator-scripts\") pod \"4f7ab0ec-52a7-4be2-9886-04bf98989b63\" (UID: \"4f7ab0ec-52a7-4be2-9886-04bf98989b63\") " Dec 03 12:22:16 crc kubenswrapper[5052]: I1203 12:22:16.732171 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4f7ab0ec-52a7-4be2-9886-04bf98989b63-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "4f7ab0ec-52a7-4be2-9886-04bf98989b63" (UID: "4f7ab0ec-52a7-4be2-9886-04bf98989b63"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:22:16 crc kubenswrapper[5052]: I1203 12:22:16.733865 5052 generic.go:334] "Generic (PLEG): container finished" podID="4f7ab0ec-52a7-4be2-9886-04bf98989b63" containerID="1eb47db985e554c7f897dc0656ec6f7756b165606954fa8a078d98e25f051bc1" exitCode=137 Dec 03 12:22:16 crc kubenswrapper[5052]: I1203 12:22:16.733915 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement9152-account-delete-x7q46" event={"ID":"4f7ab0ec-52a7-4be2-9886-04bf98989b63","Type":"ContainerDied","Data":"1eb47db985e554c7f897dc0656ec6f7756b165606954fa8a078d98e25f051bc1"} Dec 03 12:22:16 crc kubenswrapper[5052]: I1203 12:22:16.733948 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement9152-account-delete-x7q46" Dec 03 12:22:16 crc kubenswrapper[5052]: I1203 12:22:16.733972 5052 scope.go:117] "RemoveContainer" containerID="1eb47db985e554c7f897dc0656ec6f7756b165606954fa8a078d98e25f051bc1" Dec 03 12:22:16 crc kubenswrapper[5052]: I1203 12:22:16.733958 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement9152-account-delete-x7q46" event={"ID":"4f7ab0ec-52a7-4be2-9886-04bf98989b63","Type":"ContainerDied","Data":"0c28dc5ab7c2ba469dfbfce79e80be331f7d22e6d6ea521fdd0d4304a576ca02"} Dec 03 12:22:16 crc kubenswrapper[5052]: I1203 12:22:16.738297 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4f7ab0ec-52a7-4be2-9886-04bf98989b63-kube-api-access-qrtcw" (OuterVolumeSpecName: "kube-api-access-qrtcw") pod "4f7ab0ec-52a7-4be2-9886-04bf98989b63" (UID: "4f7ab0ec-52a7-4be2-9886-04bf98989b63"). InnerVolumeSpecName "kube-api-access-qrtcw". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:22:16 crc kubenswrapper[5052]: I1203 12:22:16.779710 5052 scope.go:117] "RemoveContainer" containerID="1eb47db985e554c7f897dc0656ec6f7756b165606954fa8a078d98e25f051bc1" Dec 03 12:22:16 crc kubenswrapper[5052]: E1203 12:22:16.780586 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1eb47db985e554c7f897dc0656ec6f7756b165606954fa8a078d98e25f051bc1\": container with ID starting with 1eb47db985e554c7f897dc0656ec6f7756b165606954fa8a078d98e25f051bc1 not found: ID does not exist" containerID="1eb47db985e554c7f897dc0656ec6f7756b165606954fa8a078d98e25f051bc1" Dec 03 12:22:16 crc kubenswrapper[5052]: I1203 12:22:16.780640 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1eb47db985e554c7f897dc0656ec6f7756b165606954fa8a078d98e25f051bc1"} err="failed to get container status \"1eb47db985e554c7f897dc0656ec6f7756b165606954fa8a078d98e25f051bc1\": rpc error: code = NotFound desc = could not find container \"1eb47db985e554c7f897dc0656ec6f7756b165606954fa8a078d98e25f051bc1\": container with ID starting with 1eb47db985e554c7f897dc0656ec6f7756b165606954fa8a078d98e25f051bc1 not found: ID does not exist" Dec 03 12:22:16 crc kubenswrapper[5052]: I1203 12:22:16.834497 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-qrtcw\" (UniqueName: \"kubernetes.io/projected/4f7ab0ec-52a7-4be2-9886-04bf98989b63-kube-api-access-qrtcw\") on node \"crc\" DevicePath \"\"" Dec 03 12:22:16 crc kubenswrapper[5052]: I1203 12:22:16.834538 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/4f7ab0ec-52a7-4be2-9886-04bf98989b63-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:22:17 crc kubenswrapper[5052]: I1203 12:22:17.068730 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/placement9152-account-delete-x7q46"] Dec 03 12:22:17 crc kubenswrapper[5052]: I1203 12:22:17.074702 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/placement9152-account-delete-x7q46"] Dec 03 12:22:17 crc kubenswrapper[5052]: E1203 12:22:17.454570 5052 cadvisor_stats_provider.go:516] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4dd73a47_cf2a_43b8_ba1c_0d47332edaa4.slice/crio-76b4d90822b2cea4dc663e9fb413320e7954c906bd78785f2f60c7e955624e00\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddb31774e_8f50_4862_86f3_93cb93f645a0.slice/crio-9661574f87aebd235d130b4c19903ffb15d5afc4025f54b00c2da496a34e0aad.scope\": RecentStats: unable to find data in memory cache]" Dec 03 12:22:17 crc kubenswrapper[5052]: I1203 12:22:17.623250 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-734a-account-create-update-qcgft" Dec 03 12:22:17 crc kubenswrapper[5052]: I1203 12:22:17.632178 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance73ca-account-delete-vpqhp" Dec 03 12:22:17 crc kubenswrapper[5052]: I1203 12:22:17.684062 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4f7ab0ec-52a7-4be2-9886-04bf98989b63" path="/var/lib/kubelet/pods/4f7ab0ec-52a7-4be2-9886-04bf98989b63/volumes" Dec 03 12:22:17 crc kubenswrapper[5052]: I1203 12:22:17.746141 5052 generic.go:334] "Generic (PLEG): container finished" podID="db31774e-8f50-4862-86f3-93cb93f645a0" containerID="9661574f87aebd235d130b4c19903ffb15d5afc4025f54b00c2da496a34e0aad" exitCode=137 Dec 03 12:22:17 crc kubenswrapper[5052]: I1203 12:22:17.746199 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance73ca-account-delete-vpqhp" event={"ID":"db31774e-8f50-4862-86f3-93cb93f645a0","Type":"ContainerDied","Data":"9661574f87aebd235d130b4c19903ffb15d5afc4025f54b00c2da496a34e0aad"} Dec 03 12:22:17 crc kubenswrapper[5052]: I1203 12:22:17.746233 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance73ca-account-delete-vpqhp" event={"ID":"db31774e-8f50-4862-86f3-93cb93f645a0","Type":"ContainerDied","Data":"d52fa6a791c4db9bacf26c89f59fbe21fefb28a88af961341f4ea8108f037da9"} Dec 03 12:22:17 crc kubenswrapper[5052]: I1203 12:22:17.746249 5052 scope.go:117] "RemoveContainer" containerID="9661574f87aebd235d130b4c19903ffb15d5afc4025f54b00c2da496a34e0aad" Dec 03 12:22:17 crc kubenswrapper[5052]: I1203 12:22:17.746331 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance73ca-account-delete-vpqhp" Dec 03 12:22:17 crc kubenswrapper[5052]: I1203 12:22:17.751092 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gg8jj\" (UniqueName: \"kubernetes.io/projected/db31774e-8f50-4862-86f3-93cb93f645a0-kube-api-access-gg8jj\") pod \"db31774e-8f50-4862-86f3-93cb93f645a0\" (UID: \"db31774e-8f50-4862-86f3-93cb93f645a0\") " Dec 03 12:22:17 crc kubenswrapper[5052]: I1203 12:22:17.751144 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7p76h\" (UniqueName: \"kubernetes.io/projected/817345c8-3a42-4dea-8158-75973d9c41d9-kube-api-access-7p76h\") pod \"817345c8-3a42-4dea-8158-75973d9c41d9\" (UID: \"817345c8-3a42-4dea-8158-75973d9c41d9\") " Dec 03 12:22:17 crc kubenswrapper[5052]: I1203 12:22:17.751258 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/db31774e-8f50-4862-86f3-93cb93f645a0-operator-scripts\") pod \"db31774e-8f50-4862-86f3-93cb93f645a0\" (UID: \"db31774e-8f50-4862-86f3-93cb93f645a0\") " Dec 03 12:22:17 crc kubenswrapper[5052]: I1203 12:22:17.751310 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/817345c8-3a42-4dea-8158-75973d9c41d9-operator-scripts\") pod \"817345c8-3a42-4dea-8158-75973d9c41d9\" (UID: \"817345c8-3a42-4dea-8158-75973d9c41d9\") " Dec 03 12:22:17 crc kubenswrapper[5052]: I1203 12:22:17.752594 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/817345c8-3a42-4dea-8158-75973d9c41d9-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "817345c8-3a42-4dea-8158-75973d9c41d9" (UID: "817345c8-3a42-4dea-8158-75973d9c41d9"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:22:17 crc kubenswrapper[5052]: I1203 12:22:17.752727 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/db31774e-8f50-4862-86f3-93cb93f645a0-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "db31774e-8f50-4862-86f3-93cb93f645a0" (UID: "db31774e-8f50-4862-86f3-93cb93f645a0"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:22:17 crc kubenswrapper[5052]: I1203 12:22:17.757388 5052 generic.go:334] "Generic (PLEG): container finished" podID="817345c8-3a42-4dea-8158-75973d9c41d9" containerID="609a1a60c18e09d802dbabfa7e561327519116592dcd204cb6b24fec223b9b18" exitCode=137 Dec 03 12:22:17 crc kubenswrapper[5052]: I1203 12:22:17.757451 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-734a-account-create-update-qcgft" event={"ID":"817345c8-3a42-4dea-8158-75973d9c41d9","Type":"ContainerDied","Data":"609a1a60c18e09d802dbabfa7e561327519116592dcd204cb6b24fec223b9b18"} Dec 03 12:22:17 crc kubenswrapper[5052]: I1203 12:22:17.757489 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-734a-account-create-update-qcgft" event={"ID":"817345c8-3a42-4dea-8158-75973d9c41d9","Type":"ContainerDied","Data":"1820e85036ca53b44f2a3156a713a2d04b85ce5d904896d2651e6d8304d19baa"} Dec 03 12:22:17 crc kubenswrapper[5052]: I1203 12:22:17.757569 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-734a-account-create-update-qcgft" Dec 03 12:22:17 crc kubenswrapper[5052]: I1203 12:22:17.758051 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/db31774e-8f50-4862-86f3-93cb93f645a0-kube-api-access-gg8jj" (OuterVolumeSpecName: "kube-api-access-gg8jj") pod "db31774e-8f50-4862-86f3-93cb93f645a0" (UID: "db31774e-8f50-4862-86f3-93cb93f645a0"). InnerVolumeSpecName "kube-api-access-gg8jj". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:22:17 crc kubenswrapper[5052]: I1203 12:22:17.759249 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/817345c8-3a42-4dea-8158-75973d9c41d9-kube-api-access-7p76h" (OuterVolumeSpecName: "kube-api-access-7p76h") pod "817345c8-3a42-4dea-8158-75973d9c41d9" (UID: "817345c8-3a42-4dea-8158-75973d9c41d9"). InnerVolumeSpecName "kube-api-access-7p76h". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:22:17 crc kubenswrapper[5052]: I1203 12:22:17.769006 5052 scope.go:117] "RemoveContainer" containerID="9661574f87aebd235d130b4c19903ffb15d5afc4025f54b00c2da496a34e0aad" Dec 03 12:22:17 crc kubenswrapper[5052]: E1203 12:22:17.769400 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9661574f87aebd235d130b4c19903ffb15d5afc4025f54b00c2da496a34e0aad\": container with ID starting with 9661574f87aebd235d130b4c19903ffb15d5afc4025f54b00c2da496a34e0aad not found: ID does not exist" containerID="9661574f87aebd235d130b4c19903ffb15d5afc4025f54b00c2da496a34e0aad" Dec 03 12:22:17 crc kubenswrapper[5052]: I1203 12:22:17.769435 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9661574f87aebd235d130b4c19903ffb15d5afc4025f54b00c2da496a34e0aad"} err="failed to get container status \"9661574f87aebd235d130b4c19903ffb15d5afc4025f54b00c2da496a34e0aad\": rpc error: code = NotFound desc = could not find container \"9661574f87aebd235d130b4c19903ffb15d5afc4025f54b00c2da496a34e0aad\": container with ID starting with 9661574f87aebd235d130b4c19903ffb15d5afc4025f54b00c2da496a34e0aad not found: ID does not exist" Dec 03 12:22:17 crc kubenswrapper[5052]: I1203 12:22:17.769459 5052 scope.go:117] "RemoveContainer" containerID="609a1a60c18e09d802dbabfa7e561327519116592dcd204cb6b24fec223b9b18" Dec 03 12:22:17 crc kubenswrapper[5052]: I1203 12:22:17.850900 5052 scope.go:117] "RemoveContainer" containerID="609a1a60c18e09d802dbabfa7e561327519116592dcd204cb6b24fec223b9b18" Dec 03 12:22:17 crc kubenswrapper[5052]: E1203 12:22:17.851448 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"609a1a60c18e09d802dbabfa7e561327519116592dcd204cb6b24fec223b9b18\": container with ID starting with 609a1a60c18e09d802dbabfa7e561327519116592dcd204cb6b24fec223b9b18 not found: ID does not exist" containerID="609a1a60c18e09d802dbabfa7e561327519116592dcd204cb6b24fec223b9b18" Dec 03 12:22:17 crc kubenswrapper[5052]: I1203 12:22:17.851566 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"609a1a60c18e09d802dbabfa7e561327519116592dcd204cb6b24fec223b9b18"} err="failed to get container status \"609a1a60c18e09d802dbabfa7e561327519116592dcd204cb6b24fec223b9b18\": rpc error: code = NotFound desc = could not find container \"609a1a60c18e09d802dbabfa7e561327519116592dcd204cb6b24fec223b9b18\": container with ID starting with 609a1a60c18e09d802dbabfa7e561327519116592dcd204cb6b24fec223b9b18 not found: ID does not exist" Dec 03 12:22:17 crc kubenswrapper[5052]: I1203 12:22:17.852626 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-gg8jj\" (UniqueName: \"kubernetes.io/projected/db31774e-8f50-4862-86f3-93cb93f645a0-kube-api-access-gg8jj\") on node \"crc\" DevicePath \"\"" Dec 03 12:22:17 crc kubenswrapper[5052]: I1203 12:22:17.852651 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-7p76h\" (UniqueName: \"kubernetes.io/projected/817345c8-3a42-4dea-8158-75973d9c41d9-kube-api-access-7p76h\") on node \"crc\" DevicePath \"\"" Dec 03 12:22:17 crc kubenswrapper[5052]: I1203 12:22:17.852662 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/db31774e-8f50-4862-86f3-93cb93f645a0-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:22:17 crc kubenswrapper[5052]: I1203 12:22:17.852674 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/817345c8-3a42-4dea-8158-75973d9c41d9-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:22:18 crc kubenswrapper[5052]: I1203 12:22:18.095999 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance73ca-account-delete-vpqhp"] Dec 03 12:22:18 crc kubenswrapper[5052]: I1203 12:22:18.110465 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/glance73ca-account-delete-vpqhp"] Dec 03 12:22:18 crc kubenswrapper[5052]: I1203 12:22:18.121311 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/neutron-734a-account-create-update-qcgft"] Dec 03 12:22:18 crc kubenswrapper[5052]: I1203 12:22:18.127940 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/neutron-734a-account-create-update-qcgft"] Dec 03 12:22:18 crc kubenswrapper[5052]: I1203 12:22:18.611143 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder2067-account-delete-n2tf7" Dec 03 12:22:18 crc kubenswrapper[5052]: I1203 12:22:18.637233 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican0e0e-account-delete-fvkr2" Dec 03 12:22:18 crc kubenswrapper[5052]: I1203 12:22:18.645794 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/novaapi8f5b-account-delete-86k67" Dec 03 12:22:18 crc kubenswrapper[5052]: I1203 12:22:18.660893 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron734a-account-delete-sxpn9" Dec 03 12:22:18 crc kubenswrapper[5052]: I1203 12:22:18.770401 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/fc2a6873-8741-472a-864d-84651abd4c1c-operator-scripts\") pod \"fc2a6873-8741-472a-864d-84651abd4c1c\" (UID: \"fc2a6873-8741-472a-864d-84651abd4c1c\") " Dec 03 12:22:18 crc kubenswrapper[5052]: I1203 12:22:18.770452 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/38427938-9a58-4f07-9f57-061ee6693a0d-operator-scripts\") pod \"38427938-9a58-4f07-9f57-061ee6693a0d\" (UID: \"38427938-9a58-4f07-9f57-061ee6693a0d\") " Dec 03 12:22:18 crc kubenswrapper[5052]: I1203 12:22:18.770483 5052 generic.go:334] "Generic (PLEG): container finished" podID="38427938-9a58-4f07-9f57-061ee6693a0d" containerID="2e5420fa5b52ae593a6a78576f5b291b1d9e69b7a745d45d21824b617230f090" exitCode=137 Dec 03 12:22:18 crc kubenswrapper[5052]: I1203 12:22:18.770695 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron734a-account-delete-sxpn9" Dec 03 12:22:18 crc kubenswrapper[5052]: I1203 12:22:18.771359 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/38427938-9a58-4f07-9f57-061ee6693a0d-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "38427938-9a58-4f07-9f57-061ee6693a0d" (UID: "38427938-9a58-4f07-9f57-061ee6693a0d"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:22:18 crc kubenswrapper[5052]: I1203 12:22:18.771424 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron734a-account-delete-sxpn9" event={"ID":"38427938-9a58-4f07-9f57-061ee6693a0d","Type":"ContainerDied","Data":"2e5420fa5b52ae593a6a78576f5b291b1d9e69b7a745d45d21824b617230f090"} Dec 03 12:22:18 crc kubenswrapper[5052]: I1203 12:22:18.771439 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fc2a6873-8741-472a-864d-84651abd4c1c-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "fc2a6873-8741-472a-864d-84651abd4c1c" (UID: "fc2a6873-8741-472a-864d-84651abd4c1c"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:22:18 crc kubenswrapper[5052]: I1203 12:22:18.771482 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron734a-account-delete-sxpn9" event={"ID":"38427938-9a58-4f07-9f57-061ee6693a0d","Type":"ContainerDied","Data":"430b11412ac94c2f1d55891a970f2e4f53e8276437ac26967b0689285cd56856"} Dec 03 12:22:18 crc kubenswrapper[5052]: I1203 12:22:18.770529 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pd777\" (UniqueName: \"kubernetes.io/projected/f4b213b4-e7bf-475f-8a2a-8c28437c32e0-kube-api-access-pd777\") pod \"f4b213b4-e7bf-475f-8a2a-8c28437c32e0\" (UID: \"f4b213b4-e7bf-475f-8a2a-8c28437c32e0\") " Dec 03 12:22:18 crc kubenswrapper[5052]: I1203 12:22:18.771593 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4c9xg\" (UniqueName: \"kubernetes.io/projected/fc2a6873-8741-472a-864d-84651abd4c1c-kube-api-access-4c9xg\") pod \"fc2a6873-8741-472a-864d-84651abd4c1c\" (UID: \"fc2a6873-8741-472a-864d-84651abd4c1c\") " Dec 03 12:22:18 crc kubenswrapper[5052]: I1203 12:22:18.771630 5052 scope.go:117] "RemoveContainer" containerID="2e5420fa5b52ae593a6a78576f5b291b1d9e69b7a745d45d21824b617230f090" Dec 03 12:22:18 crc kubenswrapper[5052]: I1203 12:22:18.771696 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bx7p4\" (UniqueName: \"kubernetes.io/projected/38427938-9a58-4f07-9f57-061ee6693a0d-kube-api-access-bx7p4\") pod \"38427938-9a58-4f07-9f57-061ee6693a0d\" (UID: \"38427938-9a58-4f07-9f57-061ee6693a0d\") " Dec 03 12:22:18 crc kubenswrapper[5052]: I1203 12:22:18.771717 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5m76k\" (UniqueName: \"kubernetes.io/projected/bb65017d-6c06-4870-946b-80b9b220df36-kube-api-access-5m76k\") pod \"bb65017d-6c06-4870-946b-80b9b220df36\" (UID: \"bb65017d-6c06-4870-946b-80b9b220df36\") " Dec 03 12:22:18 crc kubenswrapper[5052]: I1203 12:22:18.771802 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/bb65017d-6c06-4870-946b-80b9b220df36-operator-scripts\") pod \"bb65017d-6c06-4870-946b-80b9b220df36\" (UID: \"bb65017d-6c06-4870-946b-80b9b220df36\") " Dec 03 12:22:18 crc kubenswrapper[5052]: I1203 12:22:18.771839 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/f4b213b4-e7bf-475f-8a2a-8c28437c32e0-operator-scripts\") pod \"f4b213b4-e7bf-475f-8a2a-8c28437c32e0\" (UID: \"f4b213b4-e7bf-475f-8a2a-8c28437c32e0\") " Dec 03 12:22:18 crc kubenswrapper[5052]: I1203 12:22:18.772301 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/fc2a6873-8741-472a-864d-84651abd4c1c-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:22:18 crc kubenswrapper[5052]: I1203 12:22:18.772321 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/38427938-9a58-4f07-9f57-061ee6693a0d-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:22:18 crc kubenswrapper[5052]: I1203 12:22:18.773312 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bb65017d-6c06-4870-946b-80b9b220df36-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "bb65017d-6c06-4870-946b-80b9b220df36" (UID: "bb65017d-6c06-4870-946b-80b9b220df36"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:22:18 crc kubenswrapper[5052]: I1203 12:22:18.773564 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f4b213b4-e7bf-475f-8a2a-8c28437c32e0-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "f4b213b4-e7bf-475f-8a2a-8c28437c32e0" (UID: "f4b213b4-e7bf-475f-8a2a-8c28437c32e0"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:22:18 crc kubenswrapper[5052]: I1203 12:22:18.777877 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/38427938-9a58-4f07-9f57-061ee6693a0d-kube-api-access-bx7p4" (OuterVolumeSpecName: "kube-api-access-bx7p4") pod "38427938-9a58-4f07-9f57-061ee6693a0d" (UID: "38427938-9a58-4f07-9f57-061ee6693a0d"). InnerVolumeSpecName "kube-api-access-bx7p4". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:22:18 crc kubenswrapper[5052]: I1203 12:22:18.778087 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fc2a6873-8741-472a-864d-84651abd4c1c-kube-api-access-4c9xg" (OuterVolumeSpecName: "kube-api-access-4c9xg") pod "fc2a6873-8741-472a-864d-84651abd4c1c" (UID: "fc2a6873-8741-472a-864d-84651abd4c1c"). InnerVolumeSpecName "kube-api-access-4c9xg". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:22:18 crc kubenswrapper[5052]: I1203 12:22:18.778180 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bb65017d-6c06-4870-946b-80b9b220df36-kube-api-access-5m76k" (OuterVolumeSpecName: "kube-api-access-5m76k") pod "bb65017d-6c06-4870-946b-80b9b220df36" (UID: "bb65017d-6c06-4870-946b-80b9b220df36"). InnerVolumeSpecName "kube-api-access-5m76k". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:22:18 crc kubenswrapper[5052]: I1203 12:22:18.778234 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f4b213b4-e7bf-475f-8a2a-8c28437c32e0-kube-api-access-pd777" (OuterVolumeSpecName: "kube-api-access-pd777") pod "f4b213b4-e7bf-475f-8a2a-8c28437c32e0" (UID: "f4b213b4-e7bf-475f-8a2a-8c28437c32e0"). InnerVolumeSpecName "kube-api-access-pd777". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:22:18 crc kubenswrapper[5052]: I1203 12:22:18.785951 5052 generic.go:334] "Generic (PLEG): container finished" podID="f4b213b4-e7bf-475f-8a2a-8c28437c32e0" containerID="d897f2aa9cfac4051595e006bc6f7f2f03fa68c4d3c838fcc1749c0905fd2e2c" exitCode=137 Dec 03 12:22:18 crc kubenswrapper[5052]: I1203 12:22:18.786082 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder2067-account-delete-n2tf7" event={"ID":"f4b213b4-e7bf-475f-8a2a-8c28437c32e0","Type":"ContainerDied","Data":"d897f2aa9cfac4051595e006bc6f7f2f03fa68c4d3c838fcc1749c0905fd2e2c"} Dec 03 12:22:18 crc kubenswrapper[5052]: I1203 12:22:18.786114 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder2067-account-delete-n2tf7" event={"ID":"f4b213b4-e7bf-475f-8a2a-8c28437c32e0","Type":"ContainerDied","Data":"e23864fbc5f6730ba921194f728f7723b38a4e96fe4d99763788140f4fa13fd1"} Dec 03 12:22:18 crc kubenswrapper[5052]: I1203 12:22:18.786260 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder2067-account-delete-n2tf7" Dec 03 12:22:18 crc kubenswrapper[5052]: I1203 12:22:18.793304 5052 generic.go:334] "Generic (PLEG): container finished" podID="bb65017d-6c06-4870-946b-80b9b220df36" containerID="e22b78f35edac4a97928b1a8902e875b71b34760d326c1e0d71c57d0735bd8b6" exitCode=137 Dec 03 12:22:18 crc kubenswrapper[5052]: I1203 12:22:18.793458 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/novaapi8f5b-account-delete-86k67" event={"ID":"bb65017d-6c06-4870-946b-80b9b220df36","Type":"ContainerDied","Data":"e22b78f35edac4a97928b1a8902e875b71b34760d326c1e0d71c57d0735bd8b6"} Dec 03 12:22:18 crc kubenswrapper[5052]: I1203 12:22:18.794120 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/novaapi8f5b-account-delete-86k67" Dec 03 12:22:18 crc kubenswrapper[5052]: I1203 12:22:18.793504 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/novaapi8f5b-account-delete-86k67" event={"ID":"bb65017d-6c06-4870-946b-80b9b220df36","Type":"ContainerDied","Data":"1d24e0398e0fad5786bfe85a0daeebc30f86ef9a4fda453eb30c0cf2745d1ebe"} Dec 03 12:22:18 crc kubenswrapper[5052]: I1203 12:22:18.797780 5052 generic.go:334] "Generic (PLEG): container finished" podID="fc2a6873-8741-472a-864d-84651abd4c1c" containerID="742fdf1f4f0dcabc5efaae2c5ad0dc3207f2a2b46274caf8b3413d161d2c0942" exitCode=137 Dec 03 12:22:18 crc kubenswrapper[5052]: I1203 12:22:18.798193 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican0e0e-account-delete-fvkr2" Dec 03 12:22:18 crc kubenswrapper[5052]: I1203 12:22:18.798422 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican0e0e-account-delete-fvkr2" event={"ID":"fc2a6873-8741-472a-864d-84651abd4c1c","Type":"ContainerDied","Data":"742fdf1f4f0dcabc5efaae2c5ad0dc3207f2a2b46274caf8b3413d161d2c0942"} Dec 03 12:22:18 crc kubenswrapper[5052]: I1203 12:22:18.798456 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican0e0e-account-delete-fvkr2" event={"ID":"fc2a6873-8741-472a-864d-84651abd4c1c","Type":"ContainerDied","Data":"7876e19d880dc70ae433b6bd5a052ca9f95be159fc29a72a9c4c95cb85db0c42"} Dec 03 12:22:18 crc kubenswrapper[5052]: I1203 12:22:18.874224 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-bx7p4\" (UniqueName: \"kubernetes.io/projected/38427938-9a58-4f07-9f57-061ee6693a0d-kube-api-access-bx7p4\") on node \"crc\" DevicePath \"\"" Dec 03 12:22:18 crc kubenswrapper[5052]: I1203 12:22:18.874266 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-5m76k\" (UniqueName: \"kubernetes.io/projected/bb65017d-6c06-4870-946b-80b9b220df36-kube-api-access-5m76k\") on node \"crc\" DevicePath \"\"" Dec 03 12:22:18 crc kubenswrapper[5052]: I1203 12:22:18.874281 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/bb65017d-6c06-4870-946b-80b9b220df36-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:22:18 crc kubenswrapper[5052]: I1203 12:22:18.874294 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/f4b213b4-e7bf-475f-8a2a-8c28437c32e0-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:22:18 crc kubenswrapper[5052]: I1203 12:22:18.874306 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-pd777\" (UniqueName: \"kubernetes.io/projected/f4b213b4-e7bf-475f-8a2a-8c28437c32e0-kube-api-access-pd777\") on node \"crc\" DevicePath \"\"" Dec 03 12:22:18 crc kubenswrapper[5052]: I1203 12:22:18.874318 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-4c9xg\" (UniqueName: \"kubernetes.io/projected/fc2a6873-8741-472a-864d-84651abd4c1c-kube-api-access-4c9xg\") on node \"crc\" DevicePath \"\"" Dec 03 12:22:18 crc kubenswrapper[5052]: I1203 12:22:18.876646 5052 scope.go:117] "RemoveContainer" containerID="2e5420fa5b52ae593a6a78576f5b291b1d9e69b7a745d45d21824b617230f090" Dec 03 12:22:18 crc kubenswrapper[5052]: I1203 12:22:18.876702 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder2067-account-delete-n2tf7"] Dec 03 12:22:18 crc kubenswrapper[5052]: E1203 12:22:18.882014 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2e5420fa5b52ae593a6a78576f5b291b1d9e69b7a745d45d21824b617230f090\": container with ID starting with 2e5420fa5b52ae593a6a78576f5b291b1d9e69b7a745d45d21824b617230f090 not found: ID does not exist" containerID="2e5420fa5b52ae593a6a78576f5b291b1d9e69b7a745d45d21824b617230f090" Dec 03 12:22:18 crc kubenswrapper[5052]: I1203 12:22:18.883769 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2e5420fa5b52ae593a6a78576f5b291b1d9e69b7a745d45d21824b617230f090"} err="failed to get container status \"2e5420fa5b52ae593a6a78576f5b291b1d9e69b7a745d45d21824b617230f090\": rpc error: code = NotFound desc = could not find container \"2e5420fa5b52ae593a6a78576f5b291b1d9e69b7a745d45d21824b617230f090\": container with ID starting with 2e5420fa5b52ae593a6a78576f5b291b1d9e69b7a745d45d21824b617230f090 not found: ID does not exist" Dec 03 12:22:18 crc kubenswrapper[5052]: I1203 12:22:18.883882 5052 scope.go:117] "RemoveContainer" containerID="d897f2aa9cfac4051595e006bc6f7f2f03fa68c4d3c838fcc1749c0905fd2e2c" Dec 03 12:22:18 crc kubenswrapper[5052]: I1203 12:22:18.886769 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/cinder2067-account-delete-n2tf7"] Dec 03 12:22:18 crc kubenswrapper[5052]: I1203 12:22:18.896345 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/novaapi8f5b-account-delete-86k67"] Dec 03 12:22:18 crc kubenswrapper[5052]: I1203 12:22:18.902610 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/novaapi8f5b-account-delete-86k67"] Dec 03 12:22:18 crc kubenswrapper[5052]: I1203 12:22:18.908626 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican0e0e-account-delete-fvkr2"] Dec 03 12:22:18 crc kubenswrapper[5052]: I1203 12:22:18.908789 5052 scope.go:117] "RemoveContainer" containerID="d897f2aa9cfac4051595e006bc6f7f2f03fa68c4d3c838fcc1749c0905fd2e2c" Dec 03 12:22:18 crc kubenswrapper[5052]: E1203 12:22:18.909341 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d897f2aa9cfac4051595e006bc6f7f2f03fa68c4d3c838fcc1749c0905fd2e2c\": container with ID starting with d897f2aa9cfac4051595e006bc6f7f2f03fa68c4d3c838fcc1749c0905fd2e2c not found: ID does not exist" containerID="d897f2aa9cfac4051595e006bc6f7f2f03fa68c4d3c838fcc1749c0905fd2e2c" Dec 03 12:22:18 crc kubenswrapper[5052]: I1203 12:22:18.909375 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d897f2aa9cfac4051595e006bc6f7f2f03fa68c4d3c838fcc1749c0905fd2e2c"} err="failed to get container status \"d897f2aa9cfac4051595e006bc6f7f2f03fa68c4d3c838fcc1749c0905fd2e2c\": rpc error: code = NotFound desc = could not find container \"d897f2aa9cfac4051595e006bc6f7f2f03fa68c4d3c838fcc1749c0905fd2e2c\": container with ID starting with d897f2aa9cfac4051595e006bc6f7f2f03fa68c4d3c838fcc1749c0905fd2e2c not found: ID does not exist" Dec 03 12:22:18 crc kubenswrapper[5052]: I1203 12:22:18.909399 5052 scope.go:117] "RemoveContainer" containerID="e22b78f35edac4a97928b1a8902e875b71b34760d326c1e0d71c57d0735bd8b6" Dec 03 12:22:18 crc kubenswrapper[5052]: I1203 12:22:18.915898 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/barbican0e0e-account-delete-fvkr2"] Dec 03 12:22:18 crc kubenswrapper[5052]: I1203 12:22:18.934446 5052 scope.go:117] "RemoveContainer" containerID="e22b78f35edac4a97928b1a8902e875b71b34760d326c1e0d71c57d0735bd8b6" Dec 03 12:22:18 crc kubenswrapper[5052]: E1203 12:22:18.936131 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e22b78f35edac4a97928b1a8902e875b71b34760d326c1e0d71c57d0735bd8b6\": container with ID starting with e22b78f35edac4a97928b1a8902e875b71b34760d326c1e0d71c57d0735bd8b6 not found: ID does not exist" containerID="e22b78f35edac4a97928b1a8902e875b71b34760d326c1e0d71c57d0735bd8b6" Dec 03 12:22:18 crc kubenswrapper[5052]: I1203 12:22:18.936184 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e22b78f35edac4a97928b1a8902e875b71b34760d326c1e0d71c57d0735bd8b6"} err="failed to get container status \"e22b78f35edac4a97928b1a8902e875b71b34760d326c1e0d71c57d0735bd8b6\": rpc error: code = NotFound desc = could not find container \"e22b78f35edac4a97928b1a8902e875b71b34760d326c1e0d71c57d0735bd8b6\": container with ID starting with e22b78f35edac4a97928b1a8902e875b71b34760d326c1e0d71c57d0735bd8b6 not found: ID does not exist" Dec 03 12:22:18 crc kubenswrapper[5052]: I1203 12:22:18.936219 5052 scope.go:117] "RemoveContainer" containerID="742fdf1f4f0dcabc5efaae2c5ad0dc3207f2a2b46274caf8b3413d161d2c0942" Dec 03 12:22:18 crc kubenswrapper[5052]: I1203 12:22:18.957874 5052 scope.go:117] "RemoveContainer" containerID="742fdf1f4f0dcabc5efaae2c5ad0dc3207f2a2b46274caf8b3413d161d2c0942" Dec 03 12:22:18 crc kubenswrapper[5052]: E1203 12:22:18.958476 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"742fdf1f4f0dcabc5efaae2c5ad0dc3207f2a2b46274caf8b3413d161d2c0942\": container with ID starting with 742fdf1f4f0dcabc5efaae2c5ad0dc3207f2a2b46274caf8b3413d161d2c0942 not found: ID does not exist" containerID="742fdf1f4f0dcabc5efaae2c5ad0dc3207f2a2b46274caf8b3413d161d2c0942" Dec 03 12:22:18 crc kubenswrapper[5052]: I1203 12:22:18.958546 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"742fdf1f4f0dcabc5efaae2c5ad0dc3207f2a2b46274caf8b3413d161d2c0942"} err="failed to get container status \"742fdf1f4f0dcabc5efaae2c5ad0dc3207f2a2b46274caf8b3413d161d2c0942\": rpc error: code = NotFound desc = could not find container \"742fdf1f4f0dcabc5efaae2c5ad0dc3207f2a2b46274caf8b3413d161d2c0942\": container with ID starting with 742fdf1f4f0dcabc5efaae2c5ad0dc3207f2a2b46274caf8b3413d161d2c0942 not found: ID does not exist" Dec 03 12:22:19 crc kubenswrapper[5052]: I1203 12:22:19.107767 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/neutron734a-account-delete-sxpn9"] Dec 03 12:22:19 crc kubenswrapper[5052]: I1203 12:22:19.114047 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/neutron734a-account-delete-sxpn9"] Dec 03 12:22:19 crc kubenswrapper[5052]: I1203 12:22:19.684415 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="38427938-9a58-4f07-9f57-061ee6693a0d" path="/var/lib/kubelet/pods/38427938-9a58-4f07-9f57-061ee6693a0d/volumes" Dec 03 12:22:19 crc kubenswrapper[5052]: I1203 12:22:19.685448 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="817345c8-3a42-4dea-8158-75973d9c41d9" path="/var/lib/kubelet/pods/817345c8-3a42-4dea-8158-75973d9c41d9/volumes" Dec 03 12:22:19 crc kubenswrapper[5052]: I1203 12:22:19.685965 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bb65017d-6c06-4870-946b-80b9b220df36" path="/var/lib/kubelet/pods/bb65017d-6c06-4870-946b-80b9b220df36/volumes" Dec 03 12:22:19 crc kubenswrapper[5052]: I1203 12:22:19.686404 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="db31774e-8f50-4862-86f3-93cb93f645a0" path="/var/lib/kubelet/pods/db31774e-8f50-4862-86f3-93cb93f645a0/volumes" Dec 03 12:22:19 crc kubenswrapper[5052]: I1203 12:22:19.687479 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f4b213b4-e7bf-475f-8a2a-8c28437c32e0" path="/var/lib/kubelet/pods/f4b213b4-e7bf-475f-8a2a-8c28437c32e0/volumes" Dec 03 12:22:19 crc kubenswrapper[5052]: I1203 12:22:19.688311 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fc2a6873-8741-472a-864d-84651abd4c1c" path="/var/lib/kubelet/pods/fc2a6873-8741-472a-864d-84651abd4c1c/volumes" Dec 03 12:22:19 crc kubenswrapper[5052]: I1203 12:22:19.811570 5052 generic.go:334] "Generic (PLEG): container finished" podID="4edc0513-31af-4aa4-b507-f1181c549be9" containerID="f1f04f3f93da8795ec20a0de70a6a96be5287ed55eaae5bf34cb620a7aa557f3" exitCode=137 Dec 03 12:22:19 crc kubenswrapper[5052]: I1203 12:22:19.811649 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/novacell00e20-account-delete-ktxhg" event={"ID":"4edc0513-31af-4aa4-b507-f1181c549be9","Type":"ContainerDied","Data":"f1f04f3f93da8795ec20a0de70a6a96be5287ed55eaae5bf34cb620a7aa557f3"} Dec 03 12:22:20 crc kubenswrapper[5052]: I1203 12:22:20.182562 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/novacell00e20-account-delete-ktxhg" Dec 03 12:22:20 crc kubenswrapper[5052]: I1203 12:22:20.300661 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jxpwq\" (UniqueName: \"kubernetes.io/projected/4edc0513-31af-4aa4-b507-f1181c549be9-kube-api-access-jxpwq\") pod \"4edc0513-31af-4aa4-b507-f1181c549be9\" (UID: \"4edc0513-31af-4aa4-b507-f1181c549be9\") " Dec 03 12:22:20 crc kubenswrapper[5052]: I1203 12:22:20.300774 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/4edc0513-31af-4aa4-b507-f1181c549be9-operator-scripts\") pod \"4edc0513-31af-4aa4-b507-f1181c549be9\" (UID: \"4edc0513-31af-4aa4-b507-f1181c549be9\") " Dec 03 12:22:20 crc kubenswrapper[5052]: I1203 12:22:20.301549 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4edc0513-31af-4aa4-b507-f1181c549be9-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "4edc0513-31af-4aa4-b507-f1181c549be9" (UID: "4edc0513-31af-4aa4-b507-f1181c549be9"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:22:20 crc kubenswrapper[5052]: I1203 12:22:20.314912 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4edc0513-31af-4aa4-b507-f1181c549be9-kube-api-access-jxpwq" (OuterVolumeSpecName: "kube-api-access-jxpwq") pod "4edc0513-31af-4aa4-b507-f1181c549be9" (UID: "4edc0513-31af-4aa4-b507-f1181c549be9"). InnerVolumeSpecName "kube-api-access-jxpwq". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:22:20 crc kubenswrapper[5052]: I1203 12:22:20.402460 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-jxpwq\" (UniqueName: \"kubernetes.io/projected/4edc0513-31af-4aa4-b507-f1181c549be9-kube-api-access-jxpwq\") on node \"crc\" DevicePath \"\"" Dec 03 12:22:20 crc kubenswrapper[5052]: I1203 12:22:20.402500 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/4edc0513-31af-4aa4-b507-f1181c549be9-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:22:20 crc kubenswrapper[5052]: I1203 12:22:20.826414 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/novacell00e20-account-delete-ktxhg" event={"ID":"4edc0513-31af-4aa4-b507-f1181c549be9","Type":"ContainerDied","Data":"d1e1ec1a0605156f381e4a7e6d168dd2ec758e912d75200346b9ee5705cab0e0"} Dec 03 12:22:20 crc kubenswrapper[5052]: I1203 12:22:20.826471 5052 scope.go:117] "RemoveContainer" containerID="f1f04f3f93da8795ec20a0de70a6a96be5287ed55eaae5bf34cb620a7aa557f3" Dec 03 12:22:20 crc kubenswrapper[5052]: I1203 12:22:20.826481 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/novacell00e20-account-delete-ktxhg" Dec 03 12:22:20 crc kubenswrapper[5052]: I1203 12:22:20.858082 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/novacell00e20-account-delete-ktxhg"] Dec 03 12:22:20 crc kubenswrapper[5052]: I1203 12:22:20.864337 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/novacell00e20-account-delete-ktxhg"] Dec 03 12:22:21 crc kubenswrapper[5052]: I1203 12:22:21.687165 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4edc0513-31af-4aa4-b507-f1181c549be9" path="/var/lib/kubelet/pods/4edc0513-31af-4aa4-b507-f1181c549be9/volumes" Dec 03 12:22:25 crc kubenswrapper[5052]: I1203 12:22:25.392500 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["crc-storage/crc-storage-crc-xtx4j"] Dec 03 12:22:25 crc kubenswrapper[5052]: I1203 12:22:25.398420 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["crc-storage/crc-storage-crc-xtx4j"] Dec 03 12:22:25 crc kubenswrapper[5052]: I1203 12:22:25.553494 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["crc-storage/crc-storage-crc-rxtjx"] Dec 03 12:22:25 crc kubenswrapper[5052]: E1203 12:22:25.553785 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="4f7ab0ec-52a7-4be2-9886-04bf98989b63" containerName="mariadb-account-delete" Dec 03 12:22:25 crc kubenswrapper[5052]: I1203 12:22:25.553796 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="4f7ab0ec-52a7-4be2-9886-04bf98989b63" containerName="mariadb-account-delete" Dec 03 12:22:25 crc kubenswrapper[5052]: E1203 12:22:25.553814 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="0b121234-d24f-4307-9409-59a7dad3df4c" containerName="account-reaper" Dec 03 12:22:25 crc kubenswrapper[5052]: I1203 12:22:25.553821 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="0b121234-d24f-4307-9409-59a7dad3df4c" containerName="account-reaper" Dec 03 12:22:25 crc kubenswrapper[5052]: E1203 12:22:25.553829 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="0b121234-d24f-4307-9409-59a7dad3df4c" containerName="container-server" Dec 03 12:22:25 crc kubenswrapper[5052]: I1203 12:22:25.553835 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="0b121234-d24f-4307-9409-59a7dad3df4c" containerName="container-server" Dec 03 12:22:25 crc kubenswrapper[5052]: E1203 12:22:25.553847 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="c279eaf1-2ee2-46fe-88be-9a5a65d4897e" containerName="setup-container" Dec 03 12:22:25 crc kubenswrapper[5052]: I1203 12:22:25.553852 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="c279eaf1-2ee2-46fe-88be-9a5a65d4897e" containerName="setup-container" Dec 03 12:22:25 crc kubenswrapper[5052]: E1203 12:22:25.553870 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="2d217a2f-3478-41e0-936a-6b1d01530007" containerName="glance-httpd" Dec 03 12:22:25 crc kubenswrapper[5052]: I1203 12:22:25.553877 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d217a2f-3478-41e0-936a-6b1d01530007" containerName="glance-httpd" Dec 03 12:22:25 crc kubenswrapper[5052]: E1203 12:22:25.553890 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="0b121234-d24f-4307-9409-59a7dad3df4c" containerName="account-replicator" Dec 03 12:22:25 crc kubenswrapper[5052]: I1203 12:22:25.553897 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="0b121234-d24f-4307-9409-59a7dad3df4c" containerName="account-replicator" Dec 03 12:22:25 crc kubenswrapper[5052]: E1203 12:22:25.553908 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="bb65017d-6c06-4870-946b-80b9b220df36" containerName="mariadb-account-delete" Dec 03 12:22:25 crc kubenswrapper[5052]: I1203 12:22:25.553914 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb65017d-6c06-4870-946b-80b9b220df36" containerName="mariadb-account-delete" Dec 03 12:22:25 crc kubenswrapper[5052]: E1203 12:22:25.553920 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="c279eaf1-2ee2-46fe-88be-9a5a65d4897e" containerName="rabbitmq" Dec 03 12:22:25 crc kubenswrapper[5052]: I1203 12:22:25.553926 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="c279eaf1-2ee2-46fe-88be-9a5a65d4897e" containerName="rabbitmq" Dec 03 12:22:25 crc kubenswrapper[5052]: E1203 12:22:25.553933 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="0b121234-d24f-4307-9409-59a7dad3df4c" containerName="rsync" Dec 03 12:22:25 crc kubenswrapper[5052]: I1203 12:22:25.553938 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="0b121234-d24f-4307-9409-59a7dad3df4c" containerName="rsync" Dec 03 12:22:25 crc kubenswrapper[5052]: E1203 12:22:25.553948 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="4dd73a47-cf2a-43b8-ba1c-0d47332edaa4" containerName="nova-metadata-log" Dec 03 12:22:25 crc kubenswrapper[5052]: I1203 12:22:25.553954 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="4dd73a47-cf2a-43b8-ba1c-0d47332edaa4" containerName="nova-metadata-log" Dec 03 12:22:25 crc kubenswrapper[5052]: E1203 12:22:25.553964 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="484e6cd1-fb57-4416-9a1a-cf24b4693764" containerName="barbican-worker-log" Dec 03 12:22:25 crc kubenswrapper[5052]: I1203 12:22:25.553970 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="484e6cd1-fb57-4416-9a1a-cf24b4693764" containerName="barbican-worker-log" Dec 03 12:22:25 crc kubenswrapper[5052]: E1203 12:22:25.553977 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="0b121234-d24f-4307-9409-59a7dad3df4c" containerName="object-expirer" Dec 03 12:22:25 crc kubenswrapper[5052]: I1203 12:22:25.553982 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="0b121234-d24f-4307-9409-59a7dad3df4c" containerName="object-expirer" Dec 03 12:22:25 crc kubenswrapper[5052]: E1203 12:22:25.553991 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="bb41ed7e-c96b-44dd-8c4e-ffb702de3f87" containerName="cinder-api" Dec 03 12:22:25 crc kubenswrapper[5052]: I1203 12:22:25.553996 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb41ed7e-c96b-44dd-8c4e-ffb702de3f87" containerName="cinder-api" Dec 03 12:22:25 crc kubenswrapper[5052]: E1203 12:22:25.554004 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="4edc0513-31af-4aa4-b507-f1181c549be9" containerName="mariadb-account-delete" Dec 03 12:22:25 crc kubenswrapper[5052]: I1203 12:22:25.554010 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="4edc0513-31af-4aa4-b507-f1181c549be9" containerName="mariadb-account-delete" Dec 03 12:22:25 crc kubenswrapper[5052]: E1203 12:22:25.554020 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="9e3a0650-c101-4bfa-8137-9909bf8ed4c7" containerName="nova-cell1-novncproxy-novncproxy" Dec 03 12:22:25 crc kubenswrapper[5052]: I1203 12:22:25.554025 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="9e3a0650-c101-4bfa-8137-9909bf8ed4c7" containerName="nova-cell1-novncproxy-novncproxy" Dec 03 12:22:25 crc kubenswrapper[5052]: E1203 12:22:25.554033 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="2d94bf00-7a64-4e90-a48c-b206ec54da6d" containerName="ceilometer-notification-agent" Dec 03 12:22:25 crc kubenswrapper[5052]: I1203 12:22:25.554038 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d94bf00-7a64-4e90-a48c-b206ec54da6d" containerName="ceilometer-notification-agent" Dec 03 12:22:25 crc kubenswrapper[5052]: E1203 12:22:25.554049 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="38427938-9a58-4f07-9f57-061ee6693a0d" containerName="mariadb-account-delete" Dec 03 12:22:25 crc kubenswrapper[5052]: I1203 12:22:25.554054 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="38427938-9a58-4f07-9f57-061ee6693a0d" containerName="mariadb-account-delete" Dec 03 12:22:25 crc kubenswrapper[5052]: E1203 12:22:25.554064 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="03bb65ef-e9a8-4cfb-9536-001e316b1f72" containerName="neutron-httpd" Dec 03 12:22:25 crc kubenswrapper[5052]: I1203 12:22:25.554069 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="03bb65ef-e9a8-4cfb-9536-001e316b1f72" containerName="neutron-httpd" Dec 03 12:22:25 crc kubenswrapper[5052]: E1203 12:22:25.554079 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="a7907bc2-8e73-4272-b779-fd71d4acb1f4" containerName="nova-cell1-conductor-conductor" Dec 03 12:22:25 crc kubenswrapper[5052]: I1203 12:22:25.554085 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="a7907bc2-8e73-4272-b779-fd71d4acb1f4" containerName="nova-cell1-conductor-conductor" Dec 03 12:22:25 crc kubenswrapper[5052]: E1203 12:22:25.554097 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="0b121234-d24f-4307-9409-59a7dad3df4c" containerName="object-replicator" Dec 03 12:22:25 crc kubenswrapper[5052]: I1203 12:22:25.554102 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="0b121234-d24f-4307-9409-59a7dad3df4c" containerName="object-replicator" Dec 03 12:22:25 crc kubenswrapper[5052]: E1203 12:22:25.554109 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="b657a259-eb7a-48e8-8f41-2a3c077aff2a" containerName="kube-state-metrics" Dec 03 12:22:25 crc kubenswrapper[5052]: I1203 12:22:25.554115 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="b657a259-eb7a-48e8-8f41-2a3c077aff2a" containerName="kube-state-metrics" Dec 03 12:22:25 crc kubenswrapper[5052]: E1203 12:22:25.554121 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="0b121234-d24f-4307-9409-59a7dad3df4c" containerName="object-server" Dec 03 12:22:25 crc kubenswrapper[5052]: I1203 12:22:25.554134 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="0b121234-d24f-4307-9409-59a7dad3df4c" containerName="object-server" Dec 03 12:22:25 crc kubenswrapper[5052]: E1203 12:22:25.554143 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="9dec0a31-cba3-4c38-8b5d-4358f16b3704" containerName="nova-api-api" Dec 03 12:22:25 crc kubenswrapper[5052]: I1203 12:22:25.554148 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="9dec0a31-cba3-4c38-8b5d-4358f16b3704" containerName="nova-api-api" Dec 03 12:22:25 crc kubenswrapper[5052]: E1203 12:22:25.554156 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="47aa63e7-b7b2-4c1d-a73d-348b50cc5d2b" containerName="mysql-bootstrap" Dec 03 12:22:25 crc kubenswrapper[5052]: I1203 12:22:25.554162 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="47aa63e7-b7b2-4c1d-a73d-348b50cc5d2b" containerName="mysql-bootstrap" Dec 03 12:22:25 crc kubenswrapper[5052]: E1203 12:22:25.554173 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="a77df94c-3e83-47a5-9b5f-eb0aeb41bc3d" containerName="openstack-network-exporter" Dec 03 12:22:25 crc kubenswrapper[5052]: I1203 12:22:25.554180 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="a77df94c-3e83-47a5-9b5f-eb0aeb41bc3d" containerName="openstack-network-exporter" Dec 03 12:22:25 crc kubenswrapper[5052]: E1203 12:22:25.554191 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="0b121234-d24f-4307-9409-59a7dad3df4c" containerName="account-server" Dec 03 12:22:25 crc kubenswrapper[5052]: I1203 12:22:25.554196 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="0b121234-d24f-4307-9409-59a7dad3df4c" containerName="account-server" Dec 03 12:22:25 crc kubenswrapper[5052]: E1203 12:22:25.554207 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="f12eef70-f626-42ce-b4cf-2a07adec9657" containerName="dnsmasq-dns" Dec 03 12:22:25 crc kubenswrapper[5052]: I1203 12:22:25.554214 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="f12eef70-f626-42ce-b4cf-2a07adec9657" containerName="dnsmasq-dns" Dec 03 12:22:25 crc kubenswrapper[5052]: E1203 12:22:25.554223 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="0b121234-d24f-4307-9409-59a7dad3df4c" containerName="object-updater" Dec 03 12:22:25 crc kubenswrapper[5052]: I1203 12:22:25.554229 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="0b121234-d24f-4307-9409-59a7dad3df4c" containerName="object-updater" Dec 03 12:22:25 crc kubenswrapper[5052]: E1203 12:22:25.554237 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="2d94bf00-7a64-4e90-a48c-b206ec54da6d" containerName="proxy-httpd" Dec 03 12:22:25 crc kubenswrapper[5052]: I1203 12:22:25.554243 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d94bf00-7a64-4e90-a48c-b206ec54da6d" containerName="proxy-httpd" Dec 03 12:22:25 crc kubenswrapper[5052]: E1203 12:22:25.554251 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="7e109538-a2d2-43bb-8cef-bf99d7d0f1e9" containerName="proxy-server" Dec 03 12:22:25 crc kubenswrapper[5052]: I1203 12:22:25.554257 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="7e109538-a2d2-43bb-8cef-bf99d7d0f1e9" containerName="proxy-server" Dec 03 12:22:25 crc kubenswrapper[5052]: E1203 12:22:25.554264 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="540ee320-be53-46e0-90a3-079a5804124d" containerName="keystone-api" Dec 03 12:22:25 crc kubenswrapper[5052]: I1203 12:22:25.554269 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="540ee320-be53-46e0-90a3-079a5804124d" containerName="keystone-api" Dec 03 12:22:25 crc kubenswrapper[5052]: E1203 12:22:25.554278 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="af38eb0d-3924-4c4b-a22e-7e934bdcf172" containerName="barbican-api" Dec 03 12:22:25 crc kubenswrapper[5052]: I1203 12:22:25.554284 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="af38eb0d-3924-4c4b-a22e-7e934bdcf172" containerName="barbican-api" Dec 03 12:22:25 crc kubenswrapper[5052]: E1203 12:22:25.554291 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="4a55b492-d57f-40dc-b719-1294fb24ef13" containerName="cinder-scheduler" Dec 03 12:22:25 crc kubenswrapper[5052]: I1203 12:22:25.554297 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="4a55b492-d57f-40dc-b719-1294fb24ef13" containerName="cinder-scheduler" Dec 03 12:22:25 crc kubenswrapper[5052]: E1203 12:22:25.554307 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="0988bafc-df97-4a53-8308-9e992be58d9a" containerName="mysql-bootstrap" Dec 03 12:22:25 crc kubenswrapper[5052]: I1203 12:22:25.554313 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="0988bafc-df97-4a53-8308-9e992be58d9a" containerName="mysql-bootstrap" Dec 03 12:22:25 crc kubenswrapper[5052]: E1203 12:22:25.554320 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="0988bafc-df97-4a53-8308-9e992be58d9a" containerName="galera" Dec 03 12:22:25 crc kubenswrapper[5052]: I1203 12:22:25.554325 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="0988bafc-df97-4a53-8308-9e992be58d9a" containerName="galera" Dec 03 12:22:25 crc kubenswrapper[5052]: E1203 12:22:25.554333 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="9119635e-cb55-4428-9c61-b93ef2afd799" containerName="barbican-keystone-listener-log" Dec 03 12:22:25 crc kubenswrapper[5052]: I1203 12:22:25.554339 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="9119635e-cb55-4428-9c61-b93ef2afd799" containerName="barbican-keystone-listener-log" Dec 03 12:22:25 crc kubenswrapper[5052]: E1203 12:22:25.554346 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="c14a906c-f19d-4806-9110-8c47b38c5ae0" containerName="setup-container" Dec 03 12:22:25 crc kubenswrapper[5052]: I1203 12:22:25.554353 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="c14a906c-f19d-4806-9110-8c47b38c5ae0" containerName="setup-container" Dec 03 12:22:25 crc kubenswrapper[5052]: E1203 12:22:25.554362 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="47aa63e7-b7b2-4c1d-a73d-348b50cc5d2b" containerName="galera" Dec 03 12:22:25 crc kubenswrapper[5052]: I1203 12:22:25.554368 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="47aa63e7-b7b2-4c1d-a73d-348b50cc5d2b" containerName="galera" Dec 03 12:22:25 crc kubenswrapper[5052]: E1203 12:22:25.554376 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="af38eb0d-3924-4c4b-a22e-7e934bdcf172" containerName="barbican-api-log" Dec 03 12:22:25 crc kubenswrapper[5052]: I1203 12:22:25.554381 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="af38eb0d-3924-4c4b-a22e-7e934bdcf172" containerName="barbican-api-log" Dec 03 12:22:25 crc kubenswrapper[5052]: E1203 12:22:25.554389 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="0b121234-d24f-4307-9409-59a7dad3df4c" containerName="container-auditor" Dec 03 12:22:25 crc kubenswrapper[5052]: I1203 12:22:25.554394 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="0b121234-d24f-4307-9409-59a7dad3df4c" containerName="container-auditor" Dec 03 12:22:25 crc kubenswrapper[5052]: E1203 12:22:25.554402 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="4dd73a47-cf2a-43b8-ba1c-0d47332edaa4" containerName="nova-metadata-metadata" Dec 03 12:22:25 crc kubenswrapper[5052]: I1203 12:22:25.554408 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="4dd73a47-cf2a-43b8-ba1c-0d47332edaa4" containerName="nova-metadata-metadata" Dec 03 12:22:25 crc kubenswrapper[5052]: E1203 12:22:25.554418 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="efe32106-ca1b-4d84-80a2-c0cddd3541b9" containerName="nova-cell0-conductor-conductor" Dec 03 12:22:25 crc kubenswrapper[5052]: I1203 12:22:25.554423 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="efe32106-ca1b-4d84-80a2-c0cddd3541b9" containerName="nova-cell0-conductor-conductor" Dec 03 12:22:25 crc kubenswrapper[5052]: E1203 12:22:25.554429 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="0b121234-d24f-4307-9409-59a7dad3df4c" containerName="swift-recon-cron" Dec 03 12:22:25 crc kubenswrapper[5052]: I1203 12:22:25.554434 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="0b121234-d24f-4307-9409-59a7dad3df4c" containerName="swift-recon-cron" Dec 03 12:22:25 crc kubenswrapper[5052]: E1203 12:22:25.554442 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="90814ea7-1abb-4760-bbfa-1190f23ab38a" containerName="placement-api" Dec 03 12:22:25 crc kubenswrapper[5052]: I1203 12:22:25.554448 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="90814ea7-1abb-4760-bbfa-1190f23ab38a" containerName="placement-api" Dec 03 12:22:25 crc kubenswrapper[5052]: E1203 12:22:25.554457 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="9dec0a31-cba3-4c38-8b5d-4358f16b3704" containerName="nova-api-log" Dec 03 12:22:25 crc kubenswrapper[5052]: I1203 12:22:25.554462 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="9dec0a31-cba3-4c38-8b5d-4358f16b3704" containerName="nova-api-log" Dec 03 12:22:25 crc kubenswrapper[5052]: E1203 12:22:25.554471 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="2d94bf00-7a64-4e90-a48c-b206ec54da6d" containerName="ceilometer-central-agent" Dec 03 12:22:25 crc kubenswrapper[5052]: I1203 12:22:25.554476 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d94bf00-7a64-4e90-a48c-b206ec54da6d" containerName="ceilometer-central-agent" Dec 03 12:22:25 crc kubenswrapper[5052]: E1203 12:22:25.554484 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="fc2a6873-8741-472a-864d-84651abd4c1c" containerName="mariadb-account-delete" Dec 03 12:22:25 crc kubenswrapper[5052]: I1203 12:22:25.554490 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="fc2a6873-8741-472a-864d-84651abd4c1c" containerName="mariadb-account-delete" Dec 03 12:22:25 crc kubenswrapper[5052]: E1203 12:22:25.554499 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="f4b213b4-e7bf-475f-8a2a-8c28437c32e0" containerName="mariadb-account-delete" Dec 03 12:22:25 crc kubenswrapper[5052]: I1203 12:22:25.554530 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="f4b213b4-e7bf-475f-8a2a-8c28437c32e0" containerName="mariadb-account-delete" Dec 03 12:22:25 crc kubenswrapper[5052]: E1203 12:22:25.554538 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="adcea736-eb15-4768-b12c-edb4c810beb3" containerName="memcached" Dec 03 12:22:25 crc kubenswrapper[5052]: I1203 12:22:25.554543 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="adcea736-eb15-4768-b12c-edb4c810beb3" containerName="memcached" Dec 03 12:22:25 crc kubenswrapper[5052]: E1203 12:22:25.554552 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="bb41ed7e-c96b-44dd-8c4e-ffb702de3f87" containerName="cinder-api-log" Dec 03 12:22:25 crc kubenswrapper[5052]: I1203 12:22:25.554557 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb41ed7e-c96b-44dd-8c4e-ffb702de3f87" containerName="cinder-api-log" Dec 03 12:22:25 crc kubenswrapper[5052]: E1203 12:22:25.554568 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="2d94bf00-7a64-4e90-a48c-b206ec54da6d" containerName="sg-core" Dec 03 12:22:25 crc kubenswrapper[5052]: I1203 12:22:25.554574 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d94bf00-7a64-4e90-a48c-b206ec54da6d" containerName="sg-core" Dec 03 12:22:25 crc kubenswrapper[5052]: E1203 12:22:25.554582 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="0b121234-d24f-4307-9409-59a7dad3df4c" containerName="container-replicator" Dec 03 12:22:25 crc kubenswrapper[5052]: I1203 12:22:25.554588 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="0b121234-d24f-4307-9409-59a7dad3df4c" containerName="container-replicator" Dec 03 12:22:25 crc kubenswrapper[5052]: E1203 12:22:25.554595 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="7e109538-a2d2-43bb-8cef-bf99d7d0f1e9" containerName="proxy-httpd" Dec 03 12:22:25 crc kubenswrapper[5052]: I1203 12:22:25.554600 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="7e109538-a2d2-43bb-8cef-bf99d7d0f1e9" containerName="proxy-httpd" Dec 03 12:22:25 crc kubenswrapper[5052]: E1203 12:22:25.554610 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="0b121234-d24f-4307-9409-59a7dad3df4c" containerName="object-auditor" Dec 03 12:22:25 crc kubenswrapper[5052]: I1203 12:22:25.554616 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="0b121234-d24f-4307-9409-59a7dad3df4c" containerName="object-auditor" Dec 03 12:22:25 crc kubenswrapper[5052]: E1203 12:22:25.554624 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="4a55b492-d57f-40dc-b719-1294fb24ef13" containerName="probe" Dec 03 12:22:25 crc kubenswrapper[5052]: I1203 12:22:25.554629 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="4a55b492-d57f-40dc-b719-1294fb24ef13" containerName="probe" Dec 03 12:22:25 crc kubenswrapper[5052]: E1203 12:22:25.554636 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="72b87ae1-6854-475b-a31d-976acc913623" containerName="glance-httpd" Dec 03 12:22:25 crc kubenswrapper[5052]: I1203 12:22:25.554641 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="72b87ae1-6854-475b-a31d-976acc913623" containerName="glance-httpd" Dec 03 12:22:25 crc kubenswrapper[5052]: E1203 12:22:25.554651 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="484e6cd1-fb57-4416-9a1a-cf24b4693764" containerName="barbican-worker" Dec 03 12:22:25 crc kubenswrapper[5052]: I1203 12:22:25.554657 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="484e6cd1-fb57-4416-9a1a-cf24b4693764" containerName="barbican-worker" Dec 03 12:22:25 crc kubenswrapper[5052]: E1203 12:22:25.554667 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="fb73cc9a-89cc-4cb8-87ce-2731e5dab0f7" containerName="proxy-server" Dec 03 12:22:25 crc kubenswrapper[5052]: I1203 12:22:25.554672 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb73cc9a-89cc-4cb8-87ce-2731e5dab0f7" containerName="proxy-server" Dec 03 12:22:25 crc kubenswrapper[5052]: E1203 12:22:25.554681 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="817345c8-3a42-4dea-8158-75973d9c41d9" containerName="mariadb-account-create-update" Dec 03 12:22:25 crc kubenswrapper[5052]: I1203 12:22:25.554688 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="817345c8-3a42-4dea-8158-75973d9c41d9" containerName="mariadb-account-create-update" Dec 03 12:22:25 crc kubenswrapper[5052]: E1203 12:22:25.554696 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="fb73cc9a-89cc-4cb8-87ce-2731e5dab0f7" containerName="proxy-httpd" Dec 03 12:22:25 crc kubenswrapper[5052]: I1203 12:22:25.554702 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb73cc9a-89cc-4cb8-87ce-2731e5dab0f7" containerName="proxy-httpd" Dec 03 12:22:25 crc kubenswrapper[5052]: E1203 12:22:25.554707 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="0b121234-d24f-4307-9409-59a7dad3df4c" containerName="account-auditor" Dec 03 12:22:25 crc kubenswrapper[5052]: I1203 12:22:25.554713 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="0b121234-d24f-4307-9409-59a7dad3df4c" containerName="account-auditor" Dec 03 12:22:25 crc kubenswrapper[5052]: E1203 12:22:25.554719 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="0b121234-d24f-4307-9409-59a7dad3df4c" containerName="container-updater" Dec 03 12:22:25 crc kubenswrapper[5052]: I1203 12:22:25.554725 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="0b121234-d24f-4307-9409-59a7dad3df4c" containerName="container-updater" Dec 03 12:22:25 crc kubenswrapper[5052]: E1203 12:22:25.554734 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="f12eef70-f626-42ce-b4cf-2a07adec9657" containerName="init" Dec 03 12:22:25 crc kubenswrapper[5052]: I1203 12:22:25.554740 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="f12eef70-f626-42ce-b4cf-2a07adec9657" containerName="init" Dec 03 12:22:25 crc kubenswrapper[5052]: E1203 12:22:25.554750 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="03bb65ef-e9a8-4cfb-9536-001e316b1f72" containerName="neutron-api" Dec 03 12:22:25 crc kubenswrapper[5052]: I1203 12:22:25.554756 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="03bb65ef-e9a8-4cfb-9536-001e316b1f72" containerName="neutron-api" Dec 03 12:22:25 crc kubenswrapper[5052]: E1203 12:22:25.554762 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="9119635e-cb55-4428-9c61-b93ef2afd799" containerName="barbican-keystone-listener" Dec 03 12:22:25 crc kubenswrapper[5052]: I1203 12:22:25.554768 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="9119635e-cb55-4428-9c61-b93ef2afd799" containerName="barbican-keystone-listener" Dec 03 12:22:25 crc kubenswrapper[5052]: E1203 12:22:25.554775 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="db31774e-8f50-4862-86f3-93cb93f645a0" containerName="mariadb-account-delete" Dec 03 12:22:25 crc kubenswrapper[5052]: I1203 12:22:25.554781 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="db31774e-8f50-4862-86f3-93cb93f645a0" containerName="mariadb-account-delete" Dec 03 12:22:25 crc kubenswrapper[5052]: E1203 12:22:25.554788 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="72b87ae1-6854-475b-a31d-976acc913623" containerName="glance-log" Dec 03 12:22:25 crc kubenswrapper[5052]: I1203 12:22:25.554793 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="72b87ae1-6854-475b-a31d-976acc913623" containerName="glance-log" Dec 03 12:22:25 crc kubenswrapper[5052]: E1203 12:22:25.554802 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="2d217a2f-3478-41e0-936a-6b1d01530007" containerName="glance-log" Dec 03 12:22:25 crc kubenswrapper[5052]: I1203 12:22:25.554808 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d217a2f-3478-41e0-936a-6b1d01530007" containerName="glance-log" Dec 03 12:22:25 crc kubenswrapper[5052]: E1203 12:22:25.554817 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="c14a906c-f19d-4806-9110-8c47b38c5ae0" containerName="rabbitmq" Dec 03 12:22:25 crc kubenswrapper[5052]: I1203 12:22:25.554823 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="c14a906c-f19d-4806-9110-8c47b38c5ae0" containerName="rabbitmq" Dec 03 12:22:25 crc kubenswrapper[5052]: E1203 12:22:25.554830 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="90814ea7-1abb-4760-bbfa-1190f23ab38a" containerName="placement-log" Dec 03 12:22:25 crc kubenswrapper[5052]: I1203 12:22:25.554836 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="90814ea7-1abb-4760-bbfa-1190f23ab38a" containerName="placement-log" Dec 03 12:22:25 crc kubenswrapper[5052]: E1203 12:22:25.554845 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="a77df94c-3e83-47a5-9b5f-eb0aeb41bc3d" containerName="ovn-northd" Dec 03 12:22:25 crc kubenswrapper[5052]: I1203 12:22:25.554868 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="a77df94c-3e83-47a5-9b5f-eb0aeb41bc3d" containerName="ovn-northd" Dec 03 12:22:25 crc kubenswrapper[5052]: I1203 12:22:25.555012 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="0988bafc-df97-4a53-8308-9e992be58d9a" containerName="galera" Dec 03 12:22:25 crc kubenswrapper[5052]: I1203 12:22:25.555029 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="0b121234-d24f-4307-9409-59a7dad3df4c" containerName="object-auditor" Dec 03 12:22:25 crc kubenswrapper[5052]: I1203 12:22:25.555039 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="a7907bc2-8e73-4272-b779-fd71d4acb1f4" containerName="nova-cell1-conductor-conductor" Dec 03 12:22:25 crc kubenswrapper[5052]: I1203 12:22:25.555053 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="fc2a6873-8741-472a-864d-84651abd4c1c" containerName="mariadb-account-delete" Dec 03 12:22:25 crc kubenswrapper[5052]: I1203 12:22:25.555068 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="9e3a0650-c101-4bfa-8137-9909bf8ed4c7" containerName="nova-cell1-novncproxy-novncproxy" Dec 03 12:22:25 crc kubenswrapper[5052]: I1203 12:22:25.555080 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="adcea736-eb15-4768-b12c-edb4c810beb3" containerName="memcached" Dec 03 12:22:25 crc kubenswrapper[5052]: I1203 12:22:25.555089 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="efe32106-ca1b-4d84-80a2-c0cddd3541b9" containerName="nova-cell0-conductor-conductor" Dec 03 12:22:25 crc kubenswrapper[5052]: I1203 12:22:25.555102 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="7e109538-a2d2-43bb-8cef-bf99d7d0f1e9" containerName="proxy-httpd" Dec 03 12:22:25 crc kubenswrapper[5052]: I1203 12:22:25.555108 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="4a55b492-d57f-40dc-b719-1294fb24ef13" containerName="cinder-scheduler" Dec 03 12:22:25 crc kubenswrapper[5052]: I1203 12:22:25.555117 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="2d94bf00-7a64-4e90-a48c-b206ec54da6d" containerName="ceilometer-central-agent" Dec 03 12:22:25 crc kubenswrapper[5052]: I1203 12:22:25.555128 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="2d217a2f-3478-41e0-936a-6b1d01530007" containerName="glance-httpd" Dec 03 12:22:25 crc kubenswrapper[5052]: I1203 12:22:25.555139 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="4dd73a47-cf2a-43b8-ba1c-0d47332edaa4" containerName="nova-metadata-log" Dec 03 12:22:25 crc kubenswrapper[5052]: I1203 12:22:25.555151 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="2d94bf00-7a64-4e90-a48c-b206ec54da6d" containerName="ceilometer-notification-agent" Dec 03 12:22:25 crc kubenswrapper[5052]: I1203 12:22:25.555162 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="bb65017d-6c06-4870-946b-80b9b220df36" containerName="mariadb-account-delete" Dec 03 12:22:25 crc kubenswrapper[5052]: I1203 12:22:25.555171 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="90814ea7-1abb-4760-bbfa-1190f23ab38a" containerName="placement-api" Dec 03 12:22:25 crc kubenswrapper[5052]: I1203 12:22:25.555181 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="b657a259-eb7a-48e8-8f41-2a3c077aff2a" containerName="kube-state-metrics" Dec 03 12:22:25 crc kubenswrapper[5052]: I1203 12:22:25.555192 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="817345c8-3a42-4dea-8158-75973d9c41d9" containerName="mariadb-account-create-update" Dec 03 12:22:25 crc kubenswrapper[5052]: I1203 12:22:25.555205 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="0b121234-d24f-4307-9409-59a7dad3df4c" containerName="account-replicator" Dec 03 12:22:25 crc kubenswrapper[5052]: I1203 12:22:25.555215 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="0b121234-d24f-4307-9409-59a7dad3df4c" containerName="container-updater" Dec 03 12:22:25 crc kubenswrapper[5052]: I1203 12:22:25.555226 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="0b121234-d24f-4307-9409-59a7dad3df4c" containerName="container-auditor" Dec 03 12:22:25 crc kubenswrapper[5052]: I1203 12:22:25.555237 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="bb41ed7e-c96b-44dd-8c4e-ffb702de3f87" containerName="cinder-api" Dec 03 12:22:25 crc kubenswrapper[5052]: I1203 12:22:25.555244 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="0b121234-d24f-4307-9409-59a7dad3df4c" containerName="object-replicator" Dec 03 12:22:25 crc kubenswrapper[5052]: I1203 12:22:25.555253 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="0b121234-d24f-4307-9409-59a7dad3df4c" containerName="account-server" Dec 03 12:22:25 crc kubenswrapper[5052]: I1203 12:22:25.555263 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="0b121234-d24f-4307-9409-59a7dad3df4c" containerName="account-reaper" Dec 03 12:22:25 crc kubenswrapper[5052]: I1203 12:22:25.555275 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="4edc0513-31af-4aa4-b507-f1181c549be9" containerName="mariadb-account-delete" Dec 03 12:22:25 crc kubenswrapper[5052]: I1203 12:22:25.555283 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="a77df94c-3e83-47a5-9b5f-eb0aeb41bc3d" containerName="openstack-network-exporter" Dec 03 12:22:25 crc kubenswrapper[5052]: I1203 12:22:25.555295 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="bb41ed7e-c96b-44dd-8c4e-ffb702de3f87" containerName="cinder-api-log" Dec 03 12:22:25 crc kubenswrapper[5052]: I1203 12:22:25.555303 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="7e109538-a2d2-43bb-8cef-bf99d7d0f1e9" containerName="proxy-server" Dec 03 12:22:25 crc kubenswrapper[5052]: I1203 12:22:25.555311 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="0b121234-d24f-4307-9409-59a7dad3df4c" containerName="object-expirer" Dec 03 12:22:25 crc kubenswrapper[5052]: I1203 12:22:25.555319 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="4a55b492-d57f-40dc-b719-1294fb24ef13" containerName="probe" Dec 03 12:22:25 crc kubenswrapper[5052]: I1203 12:22:25.555330 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="9dec0a31-cba3-4c38-8b5d-4358f16b3704" containerName="nova-api-log" Dec 03 12:22:25 crc kubenswrapper[5052]: I1203 12:22:25.555338 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="72b87ae1-6854-475b-a31d-976acc913623" containerName="glance-httpd" Dec 03 12:22:25 crc kubenswrapper[5052]: I1203 12:22:25.555349 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="c14a906c-f19d-4806-9110-8c47b38c5ae0" containerName="rabbitmq" Dec 03 12:22:25 crc kubenswrapper[5052]: I1203 12:22:25.555356 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="03bb65ef-e9a8-4cfb-9536-001e316b1f72" containerName="neutron-httpd" Dec 03 12:22:25 crc kubenswrapper[5052]: I1203 12:22:25.555362 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="484e6cd1-fb57-4416-9a1a-cf24b4693764" containerName="barbican-worker-log" Dec 03 12:22:25 crc kubenswrapper[5052]: I1203 12:22:25.555369 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="db31774e-8f50-4862-86f3-93cb93f645a0" containerName="mariadb-account-delete" Dec 03 12:22:25 crc kubenswrapper[5052]: I1203 12:22:25.555376 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="9dec0a31-cba3-4c38-8b5d-4358f16b3704" containerName="nova-api-api" Dec 03 12:22:25 crc kubenswrapper[5052]: I1203 12:22:25.555384 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="2d94bf00-7a64-4e90-a48c-b206ec54da6d" containerName="sg-core" Dec 03 12:22:25 crc kubenswrapper[5052]: I1203 12:22:25.555390 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="af38eb0d-3924-4c4b-a22e-7e934bdcf172" containerName="barbican-api-log" Dec 03 12:22:25 crc kubenswrapper[5052]: I1203 12:22:25.555398 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="2d94bf00-7a64-4e90-a48c-b206ec54da6d" containerName="proxy-httpd" Dec 03 12:22:25 crc kubenswrapper[5052]: I1203 12:22:25.555407 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="0b121234-d24f-4307-9409-59a7dad3df4c" containerName="object-updater" Dec 03 12:22:25 crc kubenswrapper[5052]: I1203 12:22:25.555415 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="c279eaf1-2ee2-46fe-88be-9a5a65d4897e" containerName="rabbitmq" Dec 03 12:22:25 crc kubenswrapper[5052]: I1203 12:22:25.555422 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="fb73cc9a-89cc-4cb8-87ce-2731e5dab0f7" containerName="proxy-httpd" Dec 03 12:22:25 crc kubenswrapper[5052]: I1203 12:22:25.555429 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="03bb65ef-e9a8-4cfb-9536-001e316b1f72" containerName="neutron-api" Dec 03 12:22:25 crc kubenswrapper[5052]: I1203 12:22:25.555435 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="9119635e-cb55-4428-9c61-b93ef2afd799" containerName="barbican-keystone-listener" Dec 03 12:22:25 crc kubenswrapper[5052]: I1203 12:22:25.555443 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="f12eef70-f626-42ce-b4cf-2a07adec9657" containerName="dnsmasq-dns" Dec 03 12:22:25 crc kubenswrapper[5052]: I1203 12:22:25.555453 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="af38eb0d-3924-4c4b-a22e-7e934bdcf172" containerName="barbican-api" Dec 03 12:22:25 crc kubenswrapper[5052]: I1203 12:22:25.555461 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="0b121234-d24f-4307-9409-59a7dad3df4c" containerName="container-replicator" Dec 03 12:22:25 crc kubenswrapper[5052]: I1203 12:22:25.555466 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="0b121234-d24f-4307-9409-59a7dad3df4c" containerName="object-server" Dec 03 12:22:25 crc kubenswrapper[5052]: I1203 12:22:25.555476 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="4dd73a47-cf2a-43b8-ba1c-0d47332edaa4" containerName="nova-metadata-metadata" Dec 03 12:22:25 crc kubenswrapper[5052]: I1203 12:22:25.555484 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="90814ea7-1abb-4760-bbfa-1190f23ab38a" containerName="placement-log" Dec 03 12:22:25 crc kubenswrapper[5052]: I1203 12:22:25.555491 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="4f7ab0ec-52a7-4be2-9886-04bf98989b63" containerName="mariadb-account-delete" Dec 03 12:22:25 crc kubenswrapper[5052]: I1203 12:22:25.555499 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="540ee320-be53-46e0-90a3-079a5804124d" containerName="keystone-api" Dec 03 12:22:25 crc kubenswrapper[5052]: I1203 12:22:25.555527 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="2d217a2f-3478-41e0-936a-6b1d01530007" containerName="glance-log" Dec 03 12:22:25 crc kubenswrapper[5052]: I1203 12:22:25.555536 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="0b121234-d24f-4307-9409-59a7dad3df4c" containerName="account-auditor" Dec 03 12:22:25 crc kubenswrapper[5052]: I1203 12:22:25.555542 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="0b121234-d24f-4307-9409-59a7dad3df4c" containerName="container-server" Dec 03 12:22:25 crc kubenswrapper[5052]: I1203 12:22:25.555549 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="484e6cd1-fb57-4416-9a1a-cf24b4693764" containerName="barbican-worker" Dec 03 12:22:25 crc kubenswrapper[5052]: I1203 12:22:25.555554 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="72b87ae1-6854-475b-a31d-976acc913623" containerName="glance-log" Dec 03 12:22:25 crc kubenswrapper[5052]: I1203 12:22:25.555564 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="9119635e-cb55-4428-9c61-b93ef2afd799" containerName="barbican-keystone-listener-log" Dec 03 12:22:25 crc kubenswrapper[5052]: I1203 12:22:25.555569 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="a77df94c-3e83-47a5-9b5f-eb0aeb41bc3d" containerName="ovn-northd" Dec 03 12:22:25 crc kubenswrapper[5052]: I1203 12:22:25.555575 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="0b121234-d24f-4307-9409-59a7dad3df4c" containerName="rsync" Dec 03 12:22:25 crc kubenswrapper[5052]: I1203 12:22:25.555583 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="fb73cc9a-89cc-4cb8-87ce-2731e5dab0f7" containerName="proxy-server" Dec 03 12:22:25 crc kubenswrapper[5052]: I1203 12:22:25.555593 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="38427938-9a58-4f07-9f57-061ee6693a0d" containerName="mariadb-account-delete" Dec 03 12:22:25 crc kubenswrapper[5052]: I1203 12:22:25.555602 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="f4b213b4-e7bf-475f-8a2a-8c28437c32e0" containerName="mariadb-account-delete" Dec 03 12:22:25 crc kubenswrapper[5052]: I1203 12:22:25.555609 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="0b121234-d24f-4307-9409-59a7dad3df4c" containerName="swift-recon-cron" Dec 03 12:22:25 crc kubenswrapper[5052]: I1203 12:22:25.555615 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="47aa63e7-b7b2-4c1d-a73d-348b50cc5d2b" containerName="galera" Dec 03 12:22:25 crc kubenswrapper[5052]: I1203 12:22:25.556096 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="crc-storage/crc-storage-crc-rxtjx" Dec 03 12:22:25 crc kubenswrapper[5052]: I1203 12:22:25.564186 5052 reflector.go:368] Caches populated for *v1.Secret from object-"crc-storage"/"crc-storage-dockercfg-mf76g" Dec 03 12:22:25 crc kubenswrapper[5052]: I1203 12:22:25.564263 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"crc-storage"/"openshift-service-ca.crt" Dec 03 12:22:25 crc kubenswrapper[5052]: I1203 12:22:25.564533 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"crc-storage"/"kube-root-ca.crt" Dec 03 12:22:25 crc kubenswrapper[5052]: I1203 12:22:25.564660 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"crc-storage"/"crc-storage" Dec 03 12:22:25 crc kubenswrapper[5052]: I1203 12:22:25.566207 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["crc-storage/crc-storage-crc-rxtjx"] Dec 03 12:22:25 crc kubenswrapper[5052]: I1203 12:22:25.673668 5052 scope.go:117] "RemoveContainer" containerID="09b4a0072741f60499d24f40cc4ff3406d5d22671ced431363ab865172786300" Dec 03 12:22:25 crc kubenswrapper[5052]: E1203 12:22:25.673895 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 12:22:25 crc kubenswrapper[5052]: I1203 12:22:25.683847 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9ca00ad6-dad7-495c-bfcc-ca4b5739633c" path="/var/lib/kubelet/pods/9ca00ad6-dad7-495c-bfcc-ca4b5739633c/volumes" Dec 03 12:22:25 crc kubenswrapper[5052]: I1203 12:22:25.686707 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g45w2\" (UniqueName: \"kubernetes.io/projected/df42ebf5-634b-42c9-a731-ebe0827eaf37-kube-api-access-g45w2\") pod \"crc-storage-crc-rxtjx\" (UID: \"df42ebf5-634b-42c9-a731-ebe0827eaf37\") " pod="crc-storage/crc-storage-crc-rxtjx" Dec 03 12:22:25 crc kubenswrapper[5052]: I1203 12:22:25.686787 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/df42ebf5-634b-42c9-a731-ebe0827eaf37-crc-storage\") pod \"crc-storage-crc-rxtjx\" (UID: \"df42ebf5-634b-42c9-a731-ebe0827eaf37\") " pod="crc-storage/crc-storage-crc-rxtjx" Dec 03 12:22:25 crc kubenswrapper[5052]: I1203 12:22:25.686818 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/df42ebf5-634b-42c9-a731-ebe0827eaf37-node-mnt\") pod \"crc-storage-crc-rxtjx\" (UID: \"df42ebf5-634b-42c9-a731-ebe0827eaf37\") " pod="crc-storage/crc-storage-crc-rxtjx" Dec 03 12:22:25 crc kubenswrapper[5052]: I1203 12:22:25.788793 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-g45w2\" (UniqueName: \"kubernetes.io/projected/df42ebf5-634b-42c9-a731-ebe0827eaf37-kube-api-access-g45w2\") pod \"crc-storage-crc-rxtjx\" (UID: \"df42ebf5-634b-42c9-a731-ebe0827eaf37\") " pod="crc-storage/crc-storage-crc-rxtjx" Dec 03 12:22:25 crc kubenswrapper[5052]: I1203 12:22:25.788969 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/df42ebf5-634b-42c9-a731-ebe0827eaf37-crc-storage\") pod \"crc-storage-crc-rxtjx\" (UID: \"df42ebf5-634b-42c9-a731-ebe0827eaf37\") " pod="crc-storage/crc-storage-crc-rxtjx" Dec 03 12:22:25 crc kubenswrapper[5052]: I1203 12:22:25.789009 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/df42ebf5-634b-42c9-a731-ebe0827eaf37-node-mnt\") pod \"crc-storage-crc-rxtjx\" (UID: \"df42ebf5-634b-42c9-a731-ebe0827eaf37\") " pod="crc-storage/crc-storage-crc-rxtjx" Dec 03 12:22:25 crc kubenswrapper[5052]: I1203 12:22:25.789609 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/df42ebf5-634b-42c9-a731-ebe0827eaf37-node-mnt\") pod \"crc-storage-crc-rxtjx\" (UID: \"df42ebf5-634b-42c9-a731-ebe0827eaf37\") " pod="crc-storage/crc-storage-crc-rxtjx" Dec 03 12:22:25 crc kubenswrapper[5052]: I1203 12:22:25.789878 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/df42ebf5-634b-42c9-a731-ebe0827eaf37-crc-storage\") pod \"crc-storage-crc-rxtjx\" (UID: \"df42ebf5-634b-42c9-a731-ebe0827eaf37\") " pod="crc-storage/crc-storage-crc-rxtjx" Dec 03 12:22:25 crc kubenswrapper[5052]: I1203 12:22:25.813882 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-g45w2\" (UniqueName: \"kubernetes.io/projected/df42ebf5-634b-42c9-a731-ebe0827eaf37-kube-api-access-g45w2\") pod \"crc-storage-crc-rxtjx\" (UID: \"df42ebf5-634b-42c9-a731-ebe0827eaf37\") " pod="crc-storage/crc-storage-crc-rxtjx" Dec 03 12:22:25 crc kubenswrapper[5052]: I1203 12:22:25.874377 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="crc-storage/crc-storage-crc-rxtjx" Dec 03 12:22:26 crc kubenswrapper[5052]: I1203 12:22:26.082224 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openshift-marketplace/community-operators-knf6w"] Dec 03 12:22:26 crc kubenswrapper[5052]: I1203 12:22:26.089158 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/community-operators-knf6w" Dec 03 12:22:26 crc kubenswrapper[5052]: I1203 12:22:26.095249 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/community-operators-knf6w"] Dec 03 12:22:26 crc kubenswrapper[5052]: I1203 12:22:26.197679 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/26393ed4-8c85-4bbf-8e4d-d7657e1cfb22-utilities\") pod \"community-operators-knf6w\" (UID: \"26393ed4-8c85-4bbf-8e4d-d7657e1cfb22\") " pod="openshift-marketplace/community-operators-knf6w" Dec 03 12:22:26 crc kubenswrapper[5052]: I1203 12:22:26.197783 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s6g5f\" (UniqueName: \"kubernetes.io/projected/26393ed4-8c85-4bbf-8e4d-d7657e1cfb22-kube-api-access-s6g5f\") pod \"community-operators-knf6w\" (UID: \"26393ed4-8c85-4bbf-8e4d-d7657e1cfb22\") " pod="openshift-marketplace/community-operators-knf6w" Dec 03 12:22:26 crc kubenswrapper[5052]: I1203 12:22:26.197807 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/26393ed4-8c85-4bbf-8e4d-d7657e1cfb22-catalog-content\") pod \"community-operators-knf6w\" (UID: \"26393ed4-8c85-4bbf-8e4d-d7657e1cfb22\") " pod="openshift-marketplace/community-operators-knf6w" Dec 03 12:22:26 crc kubenswrapper[5052]: I1203 12:22:26.298740 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-s6g5f\" (UniqueName: \"kubernetes.io/projected/26393ed4-8c85-4bbf-8e4d-d7657e1cfb22-kube-api-access-s6g5f\") pod \"community-operators-knf6w\" (UID: \"26393ed4-8c85-4bbf-8e4d-d7657e1cfb22\") " pod="openshift-marketplace/community-operators-knf6w" Dec 03 12:22:26 crc kubenswrapper[5052]: I1203 12:22:26.298802 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/26393ed4-8c85-4bbf-8e4d-d7657e1cfb22-catalog-content\") pod \"community-operators-knf6w\" (UID: \"26393ed4-8c85-4bbf-8e4d-d7657e1cfb22\") " pod="openshift-marketplace/community-operators-knf6w" Dec 03 12:22:26 crc kubenswrapper[5052]: I1203 12:22:26.298862 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/26393ed4-8c85-4bbf-8e4d-d7657e1cfb22-utilities\") pod \"community-operators-knf6w\" (UID: \"26393ed4-8c85-4bbf-8e4d-d7657e1cfb22\") " pod="openshift-marketplace/community-operators-knf6w" Dec 03 12:22:26 crc kubenswrapper[5052]: I1203 12:22:26.299695 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/26393ed4-8c85-4bbf-8e4d-d7657e1cfb22-utilities\") pod \"community-operators-knf6w\" (UID: \"26393ed4-8c85-4bbf-8e4d-d7657e1cfb22\") " pod="openshift-marketplace/community-operators-knf6w" Dec 03 12:22:26 crc kubenswrapper[5052]: I1203 12:22:26.299705 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/26393ed4-8c85-4bbf-8e4d-d7657e1cfb22-catalog-content\") pod \"community-operators-knf6w\" (UID: \"26393ed4-8c85-4bbf-8e4d-d7657e1cfb22\") " pod="openshift-marketplace/community-operators-knf6w" Dec 03 12:22:26 crc kubenswrapper[5052]: I1203 12:22:26.318657 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-s6g5f\" (UniqueName: \"kubernetes.io/projected/26393ed4-8c85-4bbf-8e4d-d7657e1cfb22-kube-api-access-s6g5f\") pod \"community-operators-knf6w\" (UID: \"26393ed4-8c85-4bbf-8e4d-d7657e1cfb22\") " pod="openshift-marketplace/community-operators-knf6w" Dec 03 12:22:26 crc kubenswrapper[5052]: I1203 12:22:26.355117 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["crc-storage/crc-storage-crc-rxtjx"] Dec 03 12:22:26 crc kubenswrapper[5052]: I1203 12:22:26.422166 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/community-operators-knf6w" Dec 03 12:22:26 crc kubenswrapper[5052]: I1203 12:22:26.688258 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/community-operators-knf6w"] Dec 03 12:22:26 crc kubenswrapper[5052]: I1203 12:22:26.885860 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="crc-storage/crc-storage-crc-rxtjx" event={"ID":"df42ebf5-634b-42c9-a731-ebe0827eaf37","Type":"ContainerStarted","Data":"23c4551ecc269fd17543b06039c3afcd8e45ba78aa9c417d5ba4b8564eb85ea4"} Dec 03 12:22:26 crc kubenswrapper[5052]: I1203 12:22:26.887726 5052 generic.go:334] "Generic (PLEG): container finished" podID="26393ed4-8c85-4bbf-8e4d-d7657e1cfb22" containerID="d5a694b72e4270ffeb0cb46b41b69991193f18ac4340abd4834610bda1dbe60e" exitCode=0 Dec 03 12:22:26 crc kubenswrapper[5052]: I1203 12:22:26.887783 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/community-operators-knf6w" event={"ID":"26393ed4-8c85-4bbf-8e4d-d7657e1cfb22","Type":"ContainerDied","Data":"d5a694b72e4270ffeb0cb46b41b69991193f18ac4340abd4834610bda1dbe60e"} Dec 03 12:22:26 crc kubenswrapper[5052]: I1203 12:22:26.887814 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/community-operators-knf6w" event={"ID":"26393ed4-8c85-4bbf-8e4d-d7657e1cfb22","Type":"ContainerStarted","Data":"de713702eb2f4996bb3ff8c8770d0dd48ef9580193f625b8c80d6c2469f2a2d1"} Dec 03 12:22:27 crc kubenswrapper[5052]: E1203 12:22:27.654960 5052 cadvisor_stats_provider.go:516] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4dd73a47_cf2a_43b8_ba1c_0d47332edaa4.slice/crio-76b4d90822b2cea4dc663e9fb413320e7954c906bd78785f2f60c7e955624e00\": RecentStats: unable to find data in memory cache]" Dec 03 12:22:27 crc kubenswrapper[5052]: I1203 12:22:27.898695 5052 generic.go:334] "Generic (PLEG): container finished" podID="df42ebf5-634b-42c9-a731-ebe0827eaf37" containerID="27793877d8c79ed6ca234a512d927c37ead277dc4796a3e77a80da117b8b751a" exitCode=0 Dec 03 12:22:27 crc kubenswrapper[5052]: I1203 12:22:27.898741 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="crc-storage/crc-storage-crc-rxtjx" event={"ID":"df42ebf5-634b-42c9-a731-ebe0827eaf37","Type":"ContainerDied","Data":"27793877d8c79ed6ca234a512d927c37ead277dc4796a3e77a80da117b8b751a"} Dec 03 12:22:27 crc kubenswrapper[5052]: I1203 12:22:27.903057 5052 generic.go:334] "Generic (PLEG): container finished" podID="26393ed4-8c85-4bbf-8e4d-d7657e1cfb22" containerID="5967cae95721e1e72b8c4db4802d57efa1fd7e5f8809722a100a92f8f9b5437d" exitCode=0 Dec 03 12:22:27 crc kubenswrapper[5052]: I1203 12:22:27.903105 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/community-operators-knf6w" event={"ID":"26393ed4-8c85-4bbf-8e4d-d7657e1cfb22","Type":"ContainerDied","Data":"5967cae95721e1e72b8c4db4802d57efa1fd7e5f8809722a100a92f8f9b5437d"} Dec 03 12:22:28 crc kubenswrapper[5052]: I1203 12:22:28.915989 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/community-operators-knf6w" event={"ID":"26393ed4-8c85-4bbf-8e4d-d7657e1cfb22","Type":"ContainerStarted","Data":"b314f4abcca774a6ec61d736d4470c651dc41b59c12c79dd5637e02b5addb43e"} Dec 03 12:22:28 crc kubenswrapper[5052]: I1203 12:22:28.945980 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-marketplace/community-operators-knf6w" podStartSLOduration=1.524912814 podStartE2EDuration="2.945958734s" podCreationTimestamp="2025-12-03 12:22:26 +0000 UTC" firstStartedPulling="2025-12-03 12:22:26.891763012 +0000 UTC m=+3737.480004776" lastFinishedPulling="2025-12-03 12:22:28.312808932 +0000 UTC m=+3738.901050696" observedRunningTime="2025-12-03 12:22:28.939325654 +0000 UTC m=+3739.527567428" watchObservedRunningTime="2025-12-03 12:22:28.945958734 +0000 UTC m=+3739.534200498" Dec 03 12:22:29 crc kubenswrapper[5052]: I1203 12:22:29.185400 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="crc-storage/crc-storage-crc-rxtjx" Dec 03 12:22:29 crc kubenswrapper[5052]: I1203 12:22:29.344230 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/df42ebf5-634b-42c9-a731-ebe0827eaf37-crc-storage\") pod \"df42ebf5-634b-42c9-a731-ebe0827eaf37\" (UID: \"df42ebf5-634b-42c9-a731-ebe0827eaf37\") " Dec 03 12:22:29 crc kubenswrapper[5052]: I1203 12:22:29.344682 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g45w2\" (UniqueName: \"kubernetes.io/projected/df42ebf5-634b-42c9-a731-ebe0827eaf37-kube-api-access-g45w2\") pod \"df42ebf5-634b-42c9-a731-ebe0827eaf37\" (UID: \"df42ebf5-634b-42c9-a731-ebe0827eaf37\") " Dec 03 12:22:29 crc kubenswrapper[5052]: I1203 12:22:29.344867 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/df42ebf5-634b-42c9-a731-ebe0827eaf37-node-mnt\") pod \"df42ebf5-634b-42c9-a731-ebe0827eaf37\" (UID: \"df42ebf5-634b-42c9-a731-ebe0827eaf37\") " Dec 03 12:22:29 crc kubenswrapper[5052]: I1203 12:22:29.345343 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/df42ebf5-634b-42c9-a731-ebe0827eaf37-node-mnt" (OuterVolumeSpecName: "node-mnt") pod "df42ebf5-634b-42c9-a731-ebe0827eaf37" (UID: "df42ebf5-634b-42c9-a731-ebe0827eaf37"). InnerVolumeSpecName "node-mnt". PluginName "kubernetes.io/host-path", VolumeGidValue "" Dec 03 12:22:29 crc kubenswrapper[5052]: I1203 12:22:29.349727 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/df42ebf5-634b-42c9-a731-ebe0827eaf37-kube-api-access-g45w2" (OuterVolumeSpecName: "kube-api-access-g45w2") pod "df42ebf5-634b-42c9-a731-ebe0827eaf37" (UID: "df42ebf5-634b-42c9-a731-ebe0827eaf37"). InnerVolumeSpecName "kube-api-access-g45w2". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:22:29 crc kubenswrapper[5052]: I1203 12:22:29.368542 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/df42ebf5-634b-42c9-a731-ebe0827eaf37-crc-storage" (OuterVolumeSpecName: "crc-storage") pod "df42ebf5-634b-42c9-a731-ebe0827eaf37" (UID: "df42ebf5-634b-42c9-a731-ebe0827eaf37"). InnerVolumeSpecName "crc-storage". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:22:29 crc kubenswrapper[5052]: I1203 12:22:29.446197 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-g45w2\" (UniqueName: \"kubernetes.io/projected/df42ebf5-634b-42c9-a731-ebe0827eaf37-kube-api-access-g45w2\") on node \"crc\" DevicePath \"\"" Dec 03 12:22:29 crc kubenswrapper[5052]: I1203 12:22:29.446235 5052 reconciler_common.go:293] "Volume detached for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/df42ebf5-634b-42c9-a731-ebe0827eaf37-node-mnt\") on node \"crc\" DevicePath \"\"" Dec 03 12:22:29 crc kubenswrapper[5052]: I1203 12:22:29.446247 5052 reconciler_common.go:293] "Volume detached for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/df42ebf5-634b-42c9-a731-ebe0827eaf37-crc-storage\") on node \"crc\" DevicePath \"\"" Dec 03 12:22:29 crc kubenswrapper[5052]: I1203 12:22:29.925634 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="crc-storage/crc-storage-crc-rxtjx" Dec 03 12:22:29 crc kubenswrapper[5052]: I1203 12:22:29.925669 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="crc-storage/crc-storage-crc-rxtjx" event={"ID":"df42ebf5-634b-42c9-a731-ebe0827eaf37","Type":"ContainerDied","Data":"23c4551ecc269fd17543b06039c3afcd8e45ba78aa9c417d5ba4b8564eb85ea4"} Dec 03 12:22:29 crc kubenswrapper[5052]: I1203 12:22:29.925719 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="23c4551ecc269fd17543b06039c3afcd8e45ba78aa9c417d5ba4b8564eb85ea4" Dec 03 12:22:32 crc kubenswrapper[5052]: I1203 12:22:32.995192 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["crc-storage/crc-storage-crc-rxtjx"] Dec 03 12:22:33 crc kubenswrapper[5052]: I1203 12:22:33.000437 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["crc-storage/crc-storage-crc-rxtjx"] Dec 03 12:22:33 crc kubenswrapper[5052]: I1203 12:22:33.159077 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["crc-storage/crc-storage-crc-b6699"] Dec 03 12:22:33 crc kubenswrapper[5052]: E1203 12:22:33.159422 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="df42ebf5-634b-42c9-a731-ebe0827eaf37" containerName="storage" Dec 03 12:22:33 crc kubenswrapper[5052]: I1203 12:22:33.159440 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="df42ebf5-634b-42c9-a731-ebe0827eaf37" containerName="storage" Dec 03 12:22:33 crc kubenswrapper[5052]: I1203 12:22:33.159593 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="df42ebf5-634b-42c9-a731-ebe0827eaf37" containerName="storage" Dec 03 12:22:33 crc kubenswrapper[5052]: I1203 12:22:33.160104 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="crc-storage/crc-storage-crc-b6699" Dec 03 12:22:33 crc kubenswrapper[5052]: I1203 12:22:33.166842 5052 reflector.go:368] Caches populated for *v1.Secret from object-"crc-storage"/"crc-storage-dockercfg-mf76g" Dec 03 12:22:33 crc kubenswrapper[5052]: I1203 12:22:33.167056 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"crc-storage"/"openshift-service-ca.crt" Dec 03 12:22:33 crc kubenswrapper[5052]: I1203 12:22:33.167827 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"crc-storage"/"kube-root-ca.crt" Dec 03 12:22:33 crc kubenswrapper[5052]: I1203 12:22:33.168024 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"crc-storage"/"crc-storage" Dec 03 12:22:33 crc kubenswrapper[5052]: I1203 12:22:33.179131 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["crc-storage/crc-storage-crc-b6699"] Dec 03 12:22:33 crc kubenswrapper[5052]: I1203 12:22:33.305678 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-24zc6\" (UniqueName: \"kubernetes.io/projected/889bee72-4f6a-4c9c-817e-dfce966f6fd5-kube-api-access-24zc6\") pod \"crc-storage-crc-b6699\" (UID: \"889bee72-4f6a-4c9c-817e-dfce966f6fd5\") " pod="crc-storage/crc-storage-crc-b6699" Dec 03 12:22:33 crc kubenswrapper[5052]: I1203 12:22:33.305772 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/889bee72-4f6a-4c9c-817e-dfce966f6fd5-node-mnt\") pod \"crc-storage-crc-b6699\" (UID: \"889bee72-4f6a-4c9c-817e-dfce966f6fd5\") " pod="crc-storage/crc-storage-crc-b6699" Dec 03 12:22:33 crc kubenswrapper[5052]: I1203 12:22:33.305840 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/889bee72-4f6a-4c9c-817e-dfce966f6fd5-crc-storage\") pod \"crc-storage-crc-b6699\" (UID: \"889bee72-4f6a-4c9c-817e-dfce966f6fd5\") " pod="crc-storage/crc-storage-crc-b6699" Dec 03 12:22:33 crc kubenswrapper[5052]: I1203 12:22:33.407746 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/889bee72-4f6a-4c9c-817e-dfce966f6fd5-node-mnt\") pod \"crc-storage-crc-b6699\" (UID: \"889bee72-4f6a-4c9c-817e-dfce966f6fd5\") " pod="crc-storage/crc-storage-crc-b6699" Dec 03 12:22:33 crc kubenswrapper[5052]: I1203 12:22:33.407917 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/889bee72-4f6a-4c9c-817e-dfce966f6fd5-crc-storage\") pod \"crc-storage-crc-b6699\" (UID: \"889bee72-4f6a-4c9c-817e-dfce966f6fd5\") " pod="crc-storage/crc-storage-crc-b6699" Dec 03 12:22:33 crc kubenswrapper[5052]: I1203 12:22:33.408079 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-24zc6\" (UniqueName: \"kubernetes.io/projected/889bee72-4f6a-4c9c-817e-dfce966f6fd5-kube-api-access-24zc6\") pod \"crc-storage-crc-b6699\" (UID: \"889bee72-4f6a-4c9c-817e-dfce966f6fd5\") " pod="crc-storage/crc-storage-crc-b6699" Dec 03 12:22:33 crc kubenswrapper[5052]: I1203 12:22:33.408252 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/889bee72-4f6a-4c9c-817e-dfce966f6fd5-node-mnt\") pod \"crc-storage-crc-b6699\" (UID: \"889bee72-4f6a-4c9c-817e-dfce966f6fd5\") " pod="crc-storage/crc-storage-crc-b6699" Dec 03 12:22:33 crc kubenswrapper[5052]: I1203 12:22:33.409435 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/889bee72-4f6a-4c9c-817e-dfce966f6fd5-crc-storage\") pod \"crc-storage-crc-b6699\" (UID: \"889bee72-4f6a-4c9c-817e-dfce966f6fd5\") " pod="crc-storage/crc-storage-crc-b6699" Dec 03 12:22:33 crc kubenswrapper[5052]: I1203 12:22:33.425073 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-24zc6\" (UniqueName: \"kubernetes.io/projected/889bee72-4f6a-4c9c-817e-dfce966f6fd5-kube-api-access-24zc6\") pod \"crc-storage-crc-b6699\" (UID: \"889bee72-4f6a-4c9c-817e-dfce966f6fd5\") " pod="crc-storage/crc-storage-crc-b6699" Dec 03 12:22:33 crc kubenswrapper[5052]: I1203 12:22:33.484808 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="crc-storage/crc-storage-crc-b6699" Dec 03 12:22:33 crc kubenswrapper[5052]: I1203 12:22:33.684423 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="df42ebf5-634b-42c9-a731-ebe0827eaf37" path="/var/lib/kubelet/pods/df42ebf5-634b-42c9-a731-ebe0827eaf37/volumes" Dec 03 12:22:33 crc kubenswrapper[5052]: I1203 12:22:33.705310 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["crc-storage/crc-storage-crc-b6699"] Dec 03 12:22:33 crc kubenswrapper[5052]: I1203 12:22:33.968489 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="crc-storage/crc-storage-crc-b6699" event={"ID":"889bee72-4f6a-4c9c-817e-dfce966f6fd5","Type":"ContainerStarted","Data":"e184b0b06a6b43192d571787a58b1c38f7819821dc913c30a56182dc176d67a5"} Dec 03 12:22:34 crc kubenswrapper[5052]: I1203 12:22:34.980484 5052 generic.go:334] "Generic (PLEG): container finished" podID="889bee72-4f6a-4c9c-817e-dfce966f6fd5" containerID="d4a5529f77a7bdcc5d5d76b2cb904eb4403a956000dcfbbc8e4cb0a549f64b5f" exitCode=0 Dec 03 12:22:34 crc kubenswrapper[5052]: I1203 12:22:34.981922 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="crc-storage/crc-storage-crc-b6699" event={"ID":"889bee72-4f6a-4c9c-817e-dfce966f6fd5","Type":"ContainerDied","Data":"d4a5529f77a7bdcc5d5d76b2cb904eb4403a956000dcfbbc8e4cb0a549f64b5f"} Dec 03 12:22:36 crc kubenswrapper[5052]: I1203 12:22:36.254487 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="crc-storage/crc-storage-crc-b6699" Dec 03 12:22:36 crc kubenswrapper[5052]: I1203 12:22:36.346958 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-24zc6\" (UniqueName: \"kubernetes.io/projected/889bee72-4f6a-4c9c-817e-dfce966f6fd5-kube-api-access-24zc6\") pod \"889bee72-4f6a-4c9c-817e-dfce966f6fd5\" (UID: \"889bee72-4f6a-4c9c-817e-dfce966f6fd5\") " Dec 03 12:22:36 crc kubenswrapper[5052]: I1203 12:22:36.347009 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/889bee72-4f6a-4c9c-817e-dfce966f6fd5-node-mnt\") pod \"889bee72-4f6a-4c9c-817e-dfce966f6fd5\" (UID: \"889bee72-4f6a-4c9c-817e-dfce966f6fd5\") " Dec 03 12:22:36 crc kubenswrapper[5052]: I1203 12:22:36.347083 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/889bee72-4f6a-4c9c-817e-dfce966f6fd5-crc-storage\") pod \"889bee72-4f6a-4c9c-817e-dfce966f6fd5\" (UID: \"889bee72-4f6a-4c9c-817e-dfce966f6fd5\") " Dec 03 12:22:36 crc kubenswrapper[5052]: I1203 12:22:36.347116 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/889bee72-4f6a-4c9c-817e-dfce966f6fd5-node-mnt" (OuterVolumeSpecName: "node-mnt") pod "889bee72-4f6a-4c9c-817e-dfce966f6fd5" (UID: "889bee72-4f6a-4c9c-817e-dfce966f6fd5"). InnerVolumeSpecName "node-mnt". PluginName "kubernetes.io/host-path", VolumeGidValue "" Dec 03 12:22:36 crc kubenswrapper[5052]: I1203 12:22:36.352001 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/889bee72-4f6a-4c9c-817e-dfce966f6fd5-kube-api-access-24zc6" (OuterVolumeSpecName: "kube-api-access-24zc6") pod "889bee72-4f6a-4c9c-817e-dfce966f6fd5" (UID: "889bee72-4f6a-4c9c-817e-dfce966f6fd5"). InnerVolumeSpecName "kube-api-access-24zc6". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:22:36 crc kubenswrapper[5052]: I1203 12:22:36.366844 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/889bee72-4f6a-4c9c-817e-dfce966f6fd5-crc-storage" (OuterVolumeSpecName: "crc-storage") pod "889bee72-4f6a-4c9c-817e-dfce966f6fd5" (UID: "889bee72-4f6a-4c9c-817e-dfce966f6fd5"). InnerVolumeSpecName "crc-storage". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:22:36 crc kubenswrapper[5052]: I1203 12:22:36.422954 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-marketplace/community-operators-knf6w" Dec 03 12:22:36 crc kubenswrapper[5052]: I1203 12:22:36.423257 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openshift-marketplace/community-operators-knf6w" Dec 03 12:22:36 crc kubenswrapper[5052]: I1203 12:22:36.448672 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-24zc6\" (UniqueName: \"kubernetes.io/projected/889bee72-4f6a-4c9c-817e-dfce966f6fd5-kube-api-access-24zc6\") on node \"crc\" DevicePath \"\"" Dec 03 12:22:36 crc kubenswrapper[5052]: I1203 12:22:36.449134 5052 reconciler_common.go:293] "Volume detached for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/889bee72-4f6a-4c9c-817e-dfce966f6fd5-node-mnt\") on node \"crc\" DevicePath \"\"" Dec 03 12:22:36 crc kubenswrapper[5052]: I1203 12:22:36.449272 5052 reconciler_common.go:293] "Volume detached for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/889bee72-4f6a-4c9c-817e-dfce966f6fd5-crc-storage\") on node \"crc\" DevicePath \"\"" Dec 03 12:22:36 crc kubenswrapper[5052]: I1203 12:22:36.473844 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-marketplace/community-operators-knf6w" Dec 03 12:22:36 crc kubenswrapper[5052]: I1203 12:22:36.672726 5052 scope.go:117] "RemoveContainer" containerID="09b4a0072741f60499d24f40cc4ff3406d5d22671ced431363ab865172786300" Dec 03 12:22:36 crc kubenswrapper[5052]: E1203 12:22:36.673203 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 12:22:37 crc kubenswrapper[5052]: I1203 12:22:37.003597 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="crc-storage/crc-storage-crc-b6699" Dec 03 12:22:37 crc kubenswrapper[5052]: I1203 12:22:37.003777 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="crc-storage/crc-storage-crc-b6699" event={"ID":"889bee72-4f6a-4c9c-817e-dfce966f6fd5","Type":"ContainerDied","Data":"e184b0b06a6b43192d571787a58b1c38f7819821dc913c30a56182dc176d67a5"} Dec 03 12:22:37 crc kubenswrapper[5052]: I1203 12:22:37.003812 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e184b0b06a6b43192d571787a58b1c38f7819821dc913c30a56182dc176d67a5" Dec 03 12:22:37 crc kubenswrapper[5052]: I1203 12:22:37.045491 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-marketplace/community-operators-knf6w" Dec 03 12:22:37 crc kubenswrapper[5052]: I1203 12:22:37.099789 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openshift-marketplace/community-operators-knf6w"] Dec 03 12:22:37 crc kubenswrapper[5052]: E1203 12:22:37.883677 5052 cadvisor_stats_provider.go:516] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4dd73a47_cf2a_43b8_ba1c_0d47332edaa4.slice/crio-76b4d90822b2cea4dc663e9fb413320e7954c906bd78785f2f60c7e955624e00\": RecentStats: unable to find data in memory cache]" Dec 03 12:22:39 crc kubenswrapper[5052]: I1203 12:22:39.029681 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openshift-marketplace/community-operators-knf6w" podUID="26393ed4-8c85-4bbf-8e4d-d7657e1cfb22" containerName="registry-server" containerID="cri-o://b314f4abcca774a6ec61d736d4470c651dc41b59c12c79dd5637e02b5addb43e" gracePeriod=2 Dec 03 12:22:40 crc kubenswrapper[5052]: I1203 12:22:40.043369 5052 generic.go:334] "Generic (PLEG): container finished" podID="26393ed4-8c85-4bbf-8e4d-d7657e1cfb22" containerID="b314f4abcca774a6ec61d736d4470c651dc41b59c12c79dd5637e02b5addb43e" exitCode=0 Dec 03 12:22:40 crc kubenswrapper[5052]: I1203 12:22:40.043465 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/community-operators-knf6w" event={"ID":"26393ed4-8c85-4bbf-8e4d-d7657e1cfb22","Type":"ContainerDied","Data":"b314f4abcca774a6ec61d736d4470c651dc41b59c12c79dd5637e02b5addb43e"} Dec 03 12:22:40 crc kubenswrapper[5052]: I1203 12:22:40.043968 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/community-operators-knf6w" event={"ID":"26393ed4-8c85-4bbf-8e4d-d7657e1cfb22","Type":"ContainerDied","Data":"de713702eb2f4996bb3ff8c8770d0dd48ef9580193f625b8c80d6c2469f2a2d1"} Dec 03 12:22:40 crc kubenswrapper[5052]: I1203 12:22:40.043987 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="de713702eb2f4996bb3ff8c8770d0dd48ef9580193f625b8c80d6c2469f2a2d1" Dec 03 12:22:40 crc kubenswrapper[5052]: I1203 12:22:40.087496 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/community-operators-knf6w" Dec 03 12:22:40 crc kubenswrapper[5052]: I1203 12:22:40.212859 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/26393ed4-8c85-4bbf-8e4d-d7657e1cfb22-catalog-content\") pod \"26393ed4-8c85-4bbf-8e4d-d7657e1cfb22\" (UID: \"26393ed4-8c85-4bbf-8e4d-d7657e1cfb22\") " Dec 03 12:22:40 crc kubenswrapper[5052]: I1203 12:22:40.213073 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s6g5f\" (UniqueName: \"kubernetes.io/projected/26393ed4-8c85-4bbf-8e4d-d7657e1cfb22-kube-api-access-s6g5f\") pod \"26393ed4-8c85-4bbf-8e4d-d7657e1cfb22\" (UID: \"26393ed4-8c85-4bbf-8e4d-d7657e1cfb22\") " Dec 03 12:22:40 crc kubenswrapper[5052]: I1203 12:22:40.213115 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/26393ed4-8c85-4bbf-8e4d-d7657e1cfb22-utilities\") pod \"26393ed4-8c85-4bbf-8e4d-d7657e1cfb22\" (UID: \"26393ed4-8c85-4bbf-8e4d-d7657e1cfb22\") " Dec 03 12:22:40 crc kubenswrapper[5052]: I1203 12:22:40.214394 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/26393ed4-8c85-4bbf-8e4d-d7657e1cfb22-utilities" (OuterVolumeSpecName: "utilities") pod "26393ed4-8c85-4bbf-8e4d-d7657e1cfb22" (UID: "26393ed4-8c85-4bbf-8e4d-d7657e1cfb22"). InnerVolumeSpecName "utilities". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:22:40 crc kubenswrapper[5052]: I1203 12:22:40.218931 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/26393ed4-8c85-4bbf-8e4d-d7657e1cfb22-kube-api-access-s6g5f" (OuterVolumeSpecName: "kube-api-access-s6g5f") pod "26393ed4-8c85-4bbf-8e4d-d7657e1cfb22" (UID: "26393ed4-8c85-4bbf-8e4d-d7657e1cfb22"). InnerVolumeSpecName "kube-api-access-s6g5f". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:22:40 crc kubenswrapper[5052]: I1203 12:22:40.271075 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/26393ed4-8c85-4bbf-8e4d-d7657e1cfb22-catalog-content" (OuterVolumeSpecName: "catalog-content") pod "26393ed4-8c85-4bbf-8e4d-d7657e1cfb22" (UID: "26393ed4-8c85-4bbf-8e4d-d7657e1cfb22"). InnerVolumeSpecName "catalog-content". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:22:40 crc kubenswrapper[5052]: I1203 12:22:40.314557 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-s6g5f\" (UniqueName: \"kubernetes.io/projected/26393ed4-8c85-4bbf-8e4d-d7657e1cfb22-kube-api-access-s6g5f\") on node \"crc\" DevicePath \"\"" Dec 03 12:22:40 crc kubenswrapper[5052]: I1203 12:22:40.314603 5052 reconciler_common.go:293] "Volume detached for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/26393ed4-8c85-4bbf-8e4d-d7657e1cfb22-utilities\") on node \"crc\" DevicePath \"\"" Dec 03 12:22:40 crc kubenswrapper[5052]: I1203 12:22:40.314620 5052 reconciler_common.go:293] "Volume detached for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/26393ed4-8c85-4bbf-8e4d-d7657e1cfb22-catalog-content\") on node \"crc\" DevicePath \"\"" Dec 03 12:22:41 crc kubenswrapper[5052]: I1203 12:22:41.052130 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/community-operators-knf6w" Dec 03 12:22:41 crc kubenswrapper[5052]: I1203 12:22:41.084582 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openshift-marketplace/community-operators-knf6w"] Dec 03 12:22:41 crc kubenswrapper[5052]: I1203 12:22:41.093205 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openshift-marketplace/community-operators-knf6w"] Dec 03 12:22:41 crc kubenswrapper[5052]: I1203 12:22:41.682613 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="26393ed4-8c85-4bbf-8e4d-d7657e1cfb22" path="/var/lib/kubelet/pods/26393ed4-8c85-4bbf-8e4d-d7657e1cfb22/volumes" Dec 03 12:22:46 crc kubenswrapper[5052]: I1203 12:22:46.498653 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/rabbitmq-server-0"] Dec 03 12:22:46 crc kubenswrapper[5052]: E1203 12:22:46.499574 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="26393ed4-8c85-4bbf-8e4d-d7657e1cfb22" containerName="registry-server" Dec 03 12:22:46 crc kubenswrapper[5052]: I1203 12:22:46.499589 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="26393ed4-8c85-4bbf-8e4d-d7657e1cfb22" containerName="registry-server" Dec 03 12:22:46 crc kubenswrapper[5052]: E1203 12:22:46.499601 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="26393ed4-8c85-4bbf-8e4d-d7657e1cfb22" containerName="extract-utilities" Dec 03 12:22:46 crc kubenswrapper[5052]: I1203 12:22:46.499607 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="26393ed4-8c85-4bbf-8e4d-d7657e1cfb22" containerName="extract-utilities" Dec 03 12:22:46 crc kubenswrapper[5052]: E1203 12:22:46.499625 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="889bee72-4f6a-4c9c-817e-dfce966f6fd5" containerName="storage" Dec 03 12:22:46 crc kubenswrapper[5052]: I1203 12:22:46.499631 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="889bee72-4f6a-4c9c-817e-dfce966f6fd5" containerName="storage" Dec 03 12:22:46 crc kubenswrapper[5052]: E1203 12:22:46.499651 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="26393ed4-8c85-4bbf-8e4d-d7657e1cfb22" containerName="extract-content" Dec 03 12:22:46 crc kubenswrapper[5052]: I1203 12:22:46.499658 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="26393ed4-8c85-4bbf-8e4d-d7657e1cfb22" containerName="extract-content" Dec 03 12:22:46 crc kubenswrapper[5052]: I1203 12:22:46.499822 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="26393ed4-8c85-4bbf-8e4d-d7657e1cfb22" containerName="registry-server" Dec 03 12:22:46 crc kubenswrapper[5052]: I1203 12:22:46.499846 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="889bee72-4f6a-4c9c-817e-dfce966f6fd5" containerName="storage" Dec 03 12:22:46 crc kubenswrapper[5052]: I1203 12:22:46.500873 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:22:46 crc kubenswrapper[5052]: I1203 12:22:46.510467 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"rabbitmq-default-user" Dec 03 12:22:46 crc kubenswrapper[5052]: I1203 12:22:46.510791 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"rabbitmq-erlang-cookie" Dec 03 12:22:46 crc kubenswrapper[5052]: I1203 12:22:46.513996 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"rabbitmq-server-conf" Dec 03 12:22:46 crc kubenswrapper[5052]: I1203 12:22:46.518922 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"rabbitmq-server-dockercfg-56zhj" Dec 03 12:22:46 crc kubenswrapper[5052]: I1203 12:22:46.519297 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/rabbitmq-server-0"] Dec 03 12:22:46 crc kubenswrapper[5052]: I1203 12:22:46.519843 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"rabbitmq-plugins-conf" Dec 03 12:22:46 crc kubenswrapper[5052]: I1203 12:22:46.608010 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"pod-info\" (UniqueName: \"kubernetes.io/downward-api/cd7b6179-ba41-43bd-bc95-490a88aa15fd-pod-info\") pod \"rabbitmq-server-0\" (UID: \"cd7b6179-ba41-43bd-bc95-490a88aa15fd\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:22:46 crc kubenswrapper[5052]: I1203 12:22:46.608222 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z5nsk\" (UniqueName: \"kubernetes.io/projected/cd7b6179-ba41-43bd-bc95-490a88aa15fd-kube-api-access-z5nsk\") pod \"rabbitmq-server-0\" (UID: \"cd7b6179-ba41-43bd-bc95-490a88aa15fd\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:22:46 crc kubenswrapper[5052]: I1203 12:22:46.608311 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"erlang-cookie-secret\" (UniqueName: \"kubernetes.io/secret/cd7b6179-ba41-43bd-bc95-490a88aa15fd-erlang-cookie-secret\") pod \"rabbitmq-server-0\" (UID: \"cd7b6179-ba41-43bd-bc95-490a88aa15fd\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:22:46 crc kubenswrapper[5052]: I1203 12:22:46.608430 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"server-conf\" (UniqueName: \"kubernetes.io/configmap/cd7b6179-ba41-43bd-bc95-490a88aa15fd-server-conf\") pod \"rabbitmq-server-0\" (UID: \"cd7b6179-ba41-43bd-bc95-490a88aa15fd\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:22:46 crc kubenswrapper[5052]: I1203 12:22:46.608577 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage19-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage19-crc\") pod \"rabbitmq-server-0\" (UID: \"cd7b6179-ba41-43bd-bc95-490a88aa15fd\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:22:46 crc kubenswrapper[5052]: I1203 12:22:46.608735 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"rabbitmq-erlang-cookie\" (UniqueName: \"kubernetes.io/empty-dir/cd7b6179-ba41-43bd-bc95-490a88aa15fd-rabbitmq-erlang-cookie\") pod \"rabbitmq-server-0\" (UID: \"cd7b6179-ba41-43bd-bc95-490a88aa15fd\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:22:46 crc kubenswrapper[5052]: I1203 12:22:46.608780 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-conf\" (UniqueName: \"kubernetes.io/configmap/cd7b6179-ba41-43bd-bc95-490a88aa15fd-plugins-conf\") pod \"rabbitmq-server-0\" (UID: \"cd7b6179-ba41-43bd-bc95-490a88aa15fd\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:22:46 crc kubenswrapper[5052]: I1203 12:22:46.608804 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"rabbitmq-confd\" (UniqueName: \"kubernetes.io/projected/cd7b6179-ba41-43bd-bc95-490a88aa15fd-rabbitmq-confd\") pod \"rabbitmq-server-0\" (UID: \"cd7b6179-ba41-43bd-bc95-490a88aa15fd\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:22:46 crc kubenswrapper[5052]: I1203 12:22:46.608857 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"rabbitmq-plugins\" (UniqueName: \"kubernetes.io/empty-dir/cd7b6179-ba41-43bd-bc95-490a88aa15fd-rabbitmq-plugins\") pod \"rabbitmq-server-0\" (UID: \"cd7b6179-ba41-43bd-bc95-490a88aa15fd\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:22:46 crc kubenswrapper[5052]: I1203 12:22:46.709986 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-z5nsk\" (UniqueName: \"kubernetes.io/projected/cd7b6179-ba41-43bd-bc95-490a88aa15fd-kube-api-access-z5nsk\") pod \"rabbitmq-server-0\" (UID: \"cd7b6179-ba41-43bd-bc95-490a88aa15fd\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:22:46 crc kubenswrapper[5052]: I1203 12:22:46.710035 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"erlang-cookie-secret\" (UniqueName: \"kubernetes.io/secret/cd7b6179-ba41-43bd-bc95-490a88aa15fd-erlang-cookie-secret\") pod \"rabbitmq-server-0\" (UID: \"cd7b6179-ba41-43bd-bc95-490a88aa15fd\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:22:46 crc kubenswrapper[5052]: I1203 12:22:46.710056 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"server-conf\" (UniqueName: \"kubernetes.io/configmap/cd7b6179-ba41-43bd-bc95-490a88aa15fd-server-conf\") pod \"rabbitmq-server-0\" (UID: \"cd7b6179-ba41-43bd-bc95-490a88aa15fd\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:22:46 crc kubenswrapper[5052]: I1203 12:22:46.710096 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage19-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage19-crc\") pod \"rabbitmq-server-0\" (UID: \"cd7b6179-ba41-43bd-bc95-490a88aa15fd\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:22:46 crc kubenswrapper[5052]: I1203 12:22:46.710151 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"rabbitmq-erlang-cookie\" (UniqueName: \"kubernetes.io/empty-dir/cd7b6179-ba41-43bd-bc95-490a88aa15fd-rabbitmq-erlang-cookie\") pod \"rabbitmq-server-0\" (UID: \"cd7b6179-ba41-43bd-bc95-490a88aa15fd\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:22:46 crc kubenswrapper[5052]: I1203 12:22:46.710173 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"rabbitmq-confd\" (UniqueName: \"kubernetes.io/projected/cd7b6179-ba41-43bd-bc95-490a88aa15fd-rabbitmq-confd\") pod \"rabbitmq-server-0\" (UID: \"cd7b6179-ba41-43bd-bc95-490a88aa15fd\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:22:46 crc kubenswrapper[5052]: I1203 12:22:46.710187 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"plugins-conf\" (UniqueName: \"kubernetes.io/configmap/cd7b6179-ba41-43bd-bc95-490a88aa15fd-plugins-conf\") pod \"rabbitmq-server-0\" (UID: \"cd7b6179-ba41-43bd-bc95-490a88aa15fd\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:22:46 crc kubenswrapper[5052]: I1203 12:22:46.710215 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"rabbitmq-plugins\" (UniqueName: \"kubernetes.io/empty-dir/cd7b6179-ba41-43bd-bc95-490a88aa15fd-rabbitmq-plugins\") pod \"rabbitmq-server-0\" (UID: \"cd7b6179-ba41-43bd-bc95-490a88aa15fd\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:22:46 crc kubenswrapper[5052]: I1203 12:22:46.710239 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"pod-info\" (UniqueName: \"kubernetes.io/downward-api/cd7b6179-ba41-43bd-bc95-490a88aa15fd-pod-info\") pod \"rabbitmq-server-0\" (UID: \"cd7b6179-ba41-43bd-bc95-490a88aa15fd\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:22:46 crc kubenswrapper[5052]: I1203 12:22:46.711011 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"rabbitmq-erlang-cookie\" (UniqueName: \"kubernetes.io/empty-dir/cd7b6179-ba41-43bd-bc95-490a88aa15fd-rabbitmq-erlang-cookie\") pod \"rabbitmq-server-0\" (UID: \"cd7b6179-ba41-43bd-bc95-490a88aa15fd\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:22:46 crc kubenswrapper[5052]: I1203 12:22:46.711013 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage19-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage19-crc\") pod \"rabbitmq-server-0\" (UID: \"cd7b6179-ba41-43bd-bc95-490a88aa15fd\") device mount path \"/mnt/openstack/pv19\"" pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:22:46 crc kubenswrapper[5052]: I1203 12:22:46.711208 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"rabbitmq-plugins\" (UniqueName: \"kubernetes.io/empty-dir/cd7b6179-ba41-43bd-bc95-490a88aa15fd-rabbitmq-plugins\") pod \"rabbitmq-server-0\" (UID: \"cd7b6179-ba41-43bd-bc95-490a88aa15fd\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:22:46 crc kubenswrapper[5052]: I1203 12:22:46.712331 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"plugins-conf\" (UniqueName: \"kubernetes.io/configmap/cd7b6179-ba41-43bd-bc95-490a88aa15fd-plugins-conf\") pod \"rabbitmq-server-0\" (UID: \"cd7b6179-ba41-43bd-bc95-490a88aa15fd\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:22:46 crc kubenswrapper[5052]: I1203 12:22:46.712531 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"server-conf\" (UniqueName: \"kubernetes.io/configmap/cd7b6179-ba41-43bd-bc95-490a88aa15fd-server-conf\") pod \"rabbitmq-server-0\" (UID: \"cd7b6179-ba41-43bd-bc95-490a88aa15fd\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:22:46 crc kubenswrapper[5052]: I1203 12:22:46.722367 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"erlang-cookie-secret\" (UniqueName: \"kubernetes.io/secret/cd7b6179-ba41-43bd-bc95-490a88aa15fd-erlang-cookie-secret\") pod \"rabbitmq-server-0\" (UID: \"cd7b6179-ba41-43bd-bc95-490a88aa15fd\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:22:46 crc kubenswrapper[5052]: I1203 12:22:46.722448 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"rabbitmq-confd\" (UniqueName: \"kubernetes.io/projected/cd7b6179-ba41-43bd-bc95-490a88aa15fd-rabbitmq-confd\") pod \"rabbitmq-server-0\" (UID: \"cd7b6179-ba41-43bd-bc95-490a88aa15fd\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:22:46 crc kubenswrapper[5052]: I1203 12:22:46.722499 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"pod-info\" (UniqueName: \"kubernetes.io/downward-api/cd7b6179-ba41-43bd-bc95-490a88aa15fd-pod-info\") pod \"rabbitmq-server-0\" (UID: \"cd7b6179-ba41-43bd-bc95-490a88aa15fd\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:22:46 crc kubenswrapper[5052]: I1203 12:22:46.728276 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-z5nsk\" (UniqueName: \"kubernetes.io/projected/cd7b6179-ba41-43bd-bc95-490a88aa15fd-kube-api-access-z5nsk\") pod \"rabbitmq-server-0\" (UID: \"cd7b6179-ba41-43bd-bc95-490a88aa15fd\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:22:46 crc kubenswrapper[5052]: I1203 12:22:46.740036 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage19-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage19-crc\") pod \"rabbitmq-server-0\" (UID: \"cd7b6179-ba41-43bd-bc95-490a88aa15fd\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:22:46 crc kubenswrapper[5052]: I1203 12:22:46.779915 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/rabbitmq-cell1-server-0"] Dec 03 12:22:46 crc kubenswrapper[5052]: I1203 12:22:46.782448 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:22:46 crc kubenswrapper[5052]: I1203 12:22:46.784859 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"rabbitmq-cell1-default-user" Dec 03 12:22:46 crc kubenswrapper[5052]: I1203 12:22:46.784860 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"rabbitmq-cell1-server-dockercfg-qcdwj" Dec 03 12:22:46 crc kubenswrapper[5052]: I1203 12:22:46.785103 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"rabbitmq-cell1-erlang-cookie" Dec 03 12:22:46 crc kubenswrapper[5052]: I1203 12:22:46.785424 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"rabbitmq-cell1-server-conf" Dec 03 12:22:46 crc kubenswrapper[5052]: I1203 12:22:46.786028 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"rabbitmq-cell1-plugins-conf" Dec 03 12:22:46 crc kubenswrapper[5052]: I1203 12:22:46.788764 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/rabbitmq-cell1-server-0"] Dec 03 12:22:46 crc kubenswrapper[5052]: I1203 12:22:46.820224 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:22:46 crc kubenswrapper[5052]: I1203 12:22:46.915324 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"pod-info\" (UniqueName: \"kubernetes.io/downward-api/5a5320b7-0541-483a-8e0b-78c10d0f8efa-pod-info\") pod \"rabbitmq-cell1-server-0\" (UID: \"5a5320b7-0541-483a-8e0b-78c10d0f8efa\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:22:46 crc kubenswrapper[5052]: I1203 12:22:46.915441 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-95l4t\" (UniqueName: \"kubernetes.io/projected/5a5320b7-0541-483a-8e0b-78c10d0f8efa-kube-api-access-95l4t\") pod \"rabbitmq-cell1-server-0\" (UID: \"5a5320b7-0541-483a-8e0b-78c10d0f8efa\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:22:46 crc kubenswrapper[5052]: I1203 12:22:46.915489 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"rabbitmq-erlang-cookie\" (UniqueName: \"kubernetes.io/empty-dir/5a5320b7-0541-483a-8e0b-78c10d0f8efa-rabbitmq-erlang-cookie\") pod \"rabbitmq-cell1-server-0\" (UID: \"5a5320b7-0541-483a-8e0b-78c10d0f8efa\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:22:46 crc kubenswrapper[5052]: I1203 12:22:46.915533 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-conf\" (UniqueName: \"kubernetes.io/configmap/5a5320b7-0541-483a-8e0b-78c10d0f8efa-plugins-conf\") pod \"rabbitmq-cell1-server-0\" (UID: \"5a5320b7-0541-483a-8e0b-78c10d0f8efa\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:22:46 crc kubenswrapper[5052]: I1203 12:22:46.915573 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"server-conf\" (UniqueName: \"kubernetes.io/configmap/5a5320b7-0541-483a-8e0b-78c10d0f8efa-server-conf\") pod \"rabbitmq-cell1-server-0\" (UID: \"5a5320b7-0541-483a-8e0b-78c10d0f8efa\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:22:46 crc kubenswrapper[5052]: I1203 12:22:46.915625 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"rabbitmq-confd\" (UniqueName: \"kubernetes.io/projected/5a5320b7-0541-483a-8e0b-78c10d0f8efa-rabbitmq-confd\") pod \"rabbitmq-cell1-server-0\" (UID: \"5a5320b7-0541-483a-8e0b-78c10d0f8efa\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:22:46 crc kubenswrapper[5052]: I1203 12:22:46.915658 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"erlang-cookie-secret\" (UniqueName: \"kubernetes.io/secret/5a5320b7-0541-483a-8e0b-78c10d0f8efa-erlang-cookie-secret\") pod \"rabbitmq-cell1-server-0\" (UID: \"5a5320b7-0541-483a-8e0b-78c10d0f8efa\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:22:46 crc kubenswrapper[5052]: I1203 12:22:46.915739 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"rabbitmq-plugins\" (UniqueName: \"kubernetes.io/empty-dir/5a5320b7-0541-483a-8e0b-78c10d0f8efa-rabbitmq-plugins\") pod \"rabbitmq-cell1-server-0\" (UID: \"5a5320b7-0541-483a-8e0b-78c10d0f8efa\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:22:46 crc kubenswrapper[5052]: I1203 12:22:46.915775 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage01-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage01-crc\") pod \"rabbitmq-cell1-server-0\" (UID: \"5a5320b7-0541-483a-8e0b-78c10d0f8efa\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:22:47 crc kubenswrapper[5052]: I1203 12:22:47.016914 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-95l4t\" (UniqueName: \"kubernetes.io/projected/5a5320b7-0541-483a-8e0b-78c10d0f8efa-kube-api-access-95l4t\") pod \"rabbitmq-cell1-server-0\" (UID: \"5a5320b7-0541-483a-8e0b-78c10d0f8efa\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:22:47 crc kubenswrapper[5052]: I1203 12:22:47.016968 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"rabbitmq-erlang-cookie\" (UniqueName: \"kubernetes.io/empty-dir/5a5320b7-0541-483a-8e0b-78c10d0f8efa-rabbitmq-erlang-cookie\") pod \"rabbitmq-cell1-server-0\" (UID: \"5a5320b7-0541-483a-8e0b-78c10d0f8efa\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:22:47 crc kubenswrapper[5052]: I1203 12:22:47.016987 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"plugins-conf\" (UniqueName: \"kubernetes.io/configmap/5a5320b7-0541-483a-8e0b-78c10d0f8efa-plugins-conf\") pod \"rabbitmq-cell1-server-0\" (UID: \"5a5320b7-0541-483a-8e0b-78c10d0f8efa\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:22:47 crc kubenswrapper[5052]: I1203 12:22:47.017009 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"server-conf\" (UniqueName: \"kubernetes.io/configmap/5a5320b7-0541-483a-8e0b-78c10d0f8efa-server-conf\") pod \"rabbitmq-cell1-server-0\" (UID: \"5a5320b7-0541-483a-8e0b-78c10d0f8efa\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:22:47 crc kubenswrapper[5052]: I1203 12:22:47.017034 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"rabbitmq-confd\" (UniqueName: \"kubernetes.io/projected/5a5320b7-0541-483a-8e0b-78c10d0f8efa-rabbitmq-confd\") pod \"rabbitmq-cell1-server-0\" (UID: \"5a5320b7-0541-483a-8e0b-78c10d0f8efa\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:22:47 crc kubenswrapper[5052]: I1203 12:22:47.017055 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"erlang-cookie-secret\" (UniqueName: \"kubernetes.io/secret/5a5320b7-0541-483a-8e0b-78c10d0f8efa-erlang-cookie-secret\") pod \"rabbitmq-cell1-server-0\" (UID: \"5a5320b7-0541-483a-8e0b-78c10d0f8efa\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:22:47 crc kubenswrapper[5052]: I1203 12:22:47.017099 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"rabbitmq-plugins\" (UniqueName: \"kubernetes.io/empty-dir/5a5320b7-0541-483a-8e0b-78c10d0f8efa-rabbitmq-plugins\") pod \"rabbitmq-cell1-server-0\" (UID: \"5a5320b7-0541-483a-8e0b-78c10d0f8efa\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:22:47 crc kubenswrapper[5052]: I1203 12:22:47.017317 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage01-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage01-crc\") pod \"rabbitmq-cell1-server-0\" (UID: \"5a5320b7-0541-483a-8e0b-78c10d0f8efa\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:22:47 crc kubenswrapper[5052]: I1203 12:22:47.017348 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"pod-info\" (UniqueName: \"kubernetes.io/downward-api/5a5320b7-0541-483a-8e0b-78c10d0f8efa-pod-info\") pod \"rabbitmq-cell1-server-0\" (UID: \"5a5320b7-0541-483a-8e0b-78c10d0f8efa\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:22:47 crc kubenswrapper[5052]: I1203 12:22:47.017686 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage01-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage01-crc\") pod \"rabbitmq-cell1-server-0\" (UID: \"5a5320b7-0541-483a-8e0b-78c10d0f8efa\") device mount path \"/mnt/openstack/pv01\"" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:22:47 crc kubenswrapper[5052]: I1203 12:22:47.018167 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"rabbitmq-plugins\" (UniqueName: \"kubernetes.io/empty-dir/5a5320b7-0541-483a-8e0b-78c10d0f8efa-rabbitmq-plugins\") pod \"rabbitmq-cell1-server-0\" (UID: \"5a5320b7-0541-483a-8e0b-78c10d0f8efa\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:22:47 crc kubenswrapper[5052]: I1203 12:22:47.018277 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"rabbitmq-erlang-cookie\" (UniqueName: \"kubernetes.io/empty-dir/5a5320b7-0541-483a-8e0b-78c10d0f8efa-rabbitmq-erlang-cookie\") pod \"rabbitmq-cell1-server-0\" (UID: \"5a5320b7-0541-483a-8e0b-78c10d0f8efa\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:22:47 crc kubenswrapper[5052]: I1203 12:22:47.018642 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"plugins-conf\" (UniqueName: \"kubernetes.io/configmap/5a5320b7-0541-483a-8e0b-78c10d0f8efa-plugins-conf\") pod \"rabbitmq-cell1-server-0\" (UID: \"5a5320b7-0541-483a-8e0b-78c10d0f8efa\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:22:47 crc kubenswrapper[5052]: I1203 12:22:47.020348 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"server-conf\" (UniqueName: \"kubernetes.io/configmap/5a5320b7-0541-483a-8e0b-78c10d0f8efa-server-conf\") pod \"rabbitmq-cell1-server-0\" (UID: \"5a5320b7-0541-483a-8e0b-78c10d0f8efa\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:22:47 crc kubenswrapper[5052]: I1203 12:22:47.026594 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"pod-info\" (UniqueName: \"kubernetes.io/downward-api/5a5320b7-0541-483a-8e0b-78c10d0f8efa-pod-info\") pod \"rabbitmq-cell1-server-0\" (UID: \"5a5320b7-0541-483a-8e0b-78c10d0f8efa\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:22:47 crc kubenswrapper[5052]: I1203 12:22:47.040454 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-95l4t\" (UniqueName: \"kubernetes.io/projected/5a5320b7-0541-483a-8e0b-78c10d0f8efa-kube-api-access-95l4t\") pod \"rabbitmq-cell1-server-0\" (UID: \"5a5320b7-0541-483a-8e0b-78c10d0f8efa\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:22:47 crc kubenswrapper[5052]: I1203 12:22:47.041459 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"rabbitmq-confd\" (UniqueName: \"kubernetes.io/projected/5a5320b7-0541-483a-8e0b-78c10d0f8efa-rabbitmq-confd\") pod \"rabbitmq-cell1-server-0\" (UID: \"5a5320b7-0541-483a-8e0b-78c10d0f8efa\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:22:47 crc kubenswrapper[5052]: I1203 12:22:47.041945 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"erlang-cookie-secret\" (UniqueName: \"kubernetes.io/secret/5a5320b7-0541-483a-8e0b-78c10d0f8efa-erlang-cookie-secret\") pod \"rabbitmq-cell1-server-0\" (UID: \"5a5320b7-0541-483a-8e0b-78c10d0f8efa\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:22:47 crc kubenswrapper[5052]: I1203 12:22:47.049320 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage01-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage01-crc\") pod \"rabbitmq-cell1-server-0\" (UID: \"5a5320b7-0541-483a-8e0b-78c10d0f8efa\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:22:47 crc kubenswrapper[5052]: I1203 12:22:47.105333 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:22:47 crc kubenswrapper[5052]: I1203 12:22:47.334709 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/rabbitmq-server-0"] Dec 03 12:22:47 crc kubenswrapper[5052]: I1203 12:22:47.604778 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/rabbitmq-cell1-server-0"] Dec 03 12:22:47 crc kubenswrapper[5052]: W1203 12:22:47.607829 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a5320b7_0541_483a_8e0b_78c10d0f8efa.slice/crio-f7065d25f09c5b8035f424a2ddebcac36137d019bbf23797b8768ad93bdeb0a5 WatchSource:0}: Error finding container f7065d25f09c5b8035f424a2ddebcac36137d019bbf23797b8768ad93bdeb0a5: Status 404 returned error can't find the container with id f7065d25f09c5b8035f424a2ddebcac36137d019bbf23797b8768ad93bdeb0a5 Dec 03 12:22:48 crc kubenswrapper[5052]: E1203 12:22:48.079831 5052 cadvisor_stats_provider.go:516] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4dd73a47_cf2a_43b8_ba1c_0d47332edaa4.slice/crio-76b4d90822b2cea4dc663e9fb413320e7954c906bd78785f2f60c7e955624e00\": RecentStats: unable to find data in memory cache]" Dec 03 12:22:48 crc kubenswrapper[5052]: I1203 12:22:48.122028 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-server-0" event={"ID":"cd7b6179-ba41-43bd-bc95-490a88aa15fd","Type":"ContainerStarted","Data":"262884bbd4520dabfeb61e2479101855a66a0487971cc3453cf14b95370baa1a"} Dec 03 12:22:48 crc kubenswrapper[5052]: I1203 12:22:48.123917 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" event={"ID":"5a5320b7-0541-483a-8e0b-78c10d0f8efa","Type":"ContainerStarted","Data":"f7065d25f09c5b8035f424a2ddebcac36137d019bbf23797b8768ad93bdeb0a5"} Dec 03 12:22:48 crc kubenswrapper[5052]: I1203 12:22:48.183260 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/openstack-galera-0"] Dec 03 12:22:48 crc kubenswrapper[5052]: I1203 12:22:48.185113 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:22:48 crc kubenswrapper[5052]: I1203 12:22:48.193799 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-galera-openstack-svc" Dec 03 12:22:48 crc kubenswrapper[5052]: I1203 12:22:48.195123 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"openstack-scripts" Dec 03 12:22:48 crc kubenswrapper[5052]: I1203 12:22:48.195415 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"galera-openstack-dockercfg-w7gzp" Dec 03 12:22:48 crc kubenswrapper[5052]: I1203 12:22:48.195700 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"openstack-config-data" Dec 03 12:22:48 crc kubenswrapper[5052]: I1203 12:22:48.204733 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"combined-ca-bundle" Dec 03 12:22:48 crc kubenswrapper[5052]: I1203 12:22:48.206383 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/openstack-galera-0"] Dec 03 12:22:48 crc kubenswrapper[5052]: I1203 12:22:48.337483 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/12f39c79-eeb9-4cf4-8553-1a4d9a07ee82-kolla-config\") pod \"openstack-galera-0\" (UID: \"12f39c79-eeb9-4cf4-8553-1a4d9a07ee82\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:22:48 crc kubenswrapper[5052]: I1203 12:22:48.337665 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/12f39c79-eeb9-4cf4-8553-1a4d9a07ee82-combined-ca-bundle\") pod \"openstack-galera-0\" (UID: \"12f39c79-eeb9-4cf4-8553-1a4d9a07ee82\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:22:48 crc kubenswrapper[5052]: I1203 12:22:48.337724 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c689n\" (UniqueName: \"kubernetes.io/projected/12f39c79-eeb9-4cf4-8553-1a4d9a07ee82-kube-api-access-c689n\") pod \"openstack-galera-0\" (UID: \"12f39c79-eeb9-4cf4-8553-1a4d9a07ee82\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:22:48 crc kubenswrapper[5052]: I1203 12:22:48.337765 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/12f39c79-eeb9-4cf4-8553-1a4d9a07ee82-config-data-generated\") pod \"openstack-galera-0\" (UID: \"12f39c79-eeb9-4cf4-8553-1a4d9a07ee82\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:22:48 crc kubenswrapper[5052]: I1203 12:22:48.337793 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/12f39c79-eeb9-4cf4-8553-1a4d9a07ee82-operator-scripts\") pod \"openstack-galera-0\" (UID: \"12f39c79-eeb9-4cf4-8553-1a4d9a07ee82\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:22:48 crc kubenswrapper[5052]: I1203 12:22:48.337841 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage02-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage02-crc\") pod \"openstack-galera-0\" (UID: \"12f39c79-eeb9-4cf4-8553-1a4d9a07ee82\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:22:48 crc kubenswrapper[5052]: I1203 12:22:48.337968 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/12f39c79-eeb9-4cf4-8553-1a4d9a07ee82-galera-tls-certs\") pod \"openstack-galera-0\" (UID: \"12f39c79-eeb9-4cf4-8553-1a4d9a07ee82\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:22:48 crc kubenswrapper[5052]: I1203 12:22:48.338137 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/12f39c79-eeb9-4cf4-8553-1a4d9a07ee82-config-data-default\") pod \"openstack-galera-0\" (UID: \"12f39c79-eeb9-4cf4-8553-1a4d9a07ee82\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:22:48 crc kubenswrapper[5052]: I1203 12:22:48.439084 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/12f39c79-eeb9-4cf4-8553-1a4d9a07ee82-kolla-config\") pod \"openstack-galera-0\" (UID: \"12f39c79-eeb9-4cf4-8553-1a4d9a07ee82\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:22:48 crc kubenswrapper[5052]: I1203 12:22:48.439466 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/12f39c79-eeb9-4cf4-8553-1a4d9a07ee82-combined-ca-bundle\") pod \"openstack-galera-0\" (UID: \"12f39c79-eeb9-4cf4-8553-1a4d9a07ee82\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:22:48 crc kubenswrapper[5052]: I1203 12:22:48.439504 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-c689n\" (UniqueName: \"kubernetes.io/projected/12f39c79-eeb9-4cf4-8553-1a4d9a07ee82-kube-api-access-c689n\") pod \"openstack-galera-0\" (UID: \"12f39c79-eeb9-4cf4-8553-1a4d9a07ee82\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:22:48 crc kubenswrapper[5052]: I1203 12:22:48.439549 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/12f39c79-eeb9-4cf4-8553-1a4d9a07ee82-config-data-generated\") pod \"openstack-galera-0\" (UID: \"12f39c79-eeb9-4cf4-8553-1a4d9a07ee82\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:22:48 crc kubenswrapper[5052]: I1203 12:22:48.439571 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/12f39c79-eeb9-4cf4-8553-1a4d9a07ee82-operator-scripts\") pod \"openstack-galera-0\" (UID: \"12f39c79-eeb9-4cf4-8553-1a4d9a07ee82\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:22:48 crc kubenswrapper[5052]: I1203 12:22:48.439607 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage02-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage02-crc\") pod \"openstack-galera-0\" (UID: \"12f39c79-eeb9-4cf4-8553-1a4d9a07ee82\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:22:48 crc kubenswrapper[5052]: I1203 12:22:48.439629 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/12f39c79-eeb9-4cf4-8553-1a4d9a07ee82-galera-tls-certs\") pod \"openstack-galera-0\" (UID: \"12f39c79-eeb9-4cf4-8553-1a4d9a07ee82\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:22:48 crc kubenswrapper[5052]: I1203 12:22:48.439663 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/12f39c79-eeb9-4cf4-8553-1a4d9a07ee82-config-data-default\") pod \"openstack-galera-0\" (UID: \"12f39c79-eeb9-4cf4-8553-1a4d9a07ee82\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:22:48 crc kubenswrapper[5052]: I1203 12:22:48.439966 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/12f39c79-eeb9-4cf4-8553-1a4d9a07ee82-kolla-config\") pod \"openstack-galera-0\" (UID: \"12f39c79-eeb9-4cf4-8553-1a4d9a07ee82\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:22:48 crc kubenswrapper[5052]: I1203 12:22:48.440306 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage02-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage02-crc\") pod \"openstack-galera-0\" (UID: \"12f39c79-eeb9-4cf4-8553-1a4d9a07ee82\") device mount path \"/mnt/openstack/pv02\"" pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:22:48 crc kubenswrapper[5052]: I1203 12:22:48.440788 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/12f39c79-eeb9-4cf4-8553-1a4d9a07ee82-config-data-generated\") pod \"openstack-galera-0\" (UID: \"12f39c79-eeb9-4cf4-8553-1a4d9a07ee82\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:22:48 crc kubenswrapper[5052]: I1203 12:22:48.441243 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/12f39c79-eeb9-4cf4-8553-1a4d9a07ee82-operator-scripts\") pod \"openstack-galera-0\" (UID: \"12f39c79-eeb9-4cf4-8553-1a4d9a07ee82\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:22:48 crc kubenswrapper[5052]: I1203 12:22:48.452441 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/12f39c79-eeb9-4cf4-8553-1a4d9a07ee82-galera-tls-certs\") pod \"openstack-galera-0\" (UID: \"12f39c79-eeb9-4cf4-8553-1a4d9a07ee82\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:22:48 crc kubenswrapper[5052]: I1203 12:22:48.452788 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/12f39c79-eeb9-4cf4-8553-1a4d9a07ee82-combined-ca-bundle\") pod \"openstack-galera-0\" (UID: \"12f39c79-eeb9-4cf4-8553-1a4d9a07ee82\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:22:48 crc kubenswrapper[5052]: I1203 12:22:48.457860 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/12f39c79-eeb9-4cf4-8553-1a4d9a07ee82-config-data-default\") pod \"openstack-galera-0\" (UID: \"12f39c79-eeb9-4cf4-8553-1a4d9a07ee82\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:22:48 crc kubenswrapper[5052]: I1203 12:22:48.466131 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-c689n\" (UniqueName: \"kubernetes.io/projected/12f39c79-eeb9-4cf4-8553-1a4d9a07ee82-kube-api-access-c689n\") pod \"openstack-galera-0\" (UID: \"12f39c79-eeb9-4cf4-8553-1a4d9a07ee82\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:22:48 crc kubenswrapper[5052]: I1203 12:22:48.467917 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage02-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage02-crc\") pod \"openstack-galera-0\" (UID: \"12f39c79-eeb9-4cf4-8553-1a4d9a07ee82\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:22:48 crc kubenswrapper[5052]: I1203 12:22:48.514952 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:22:48 crc kubenswrapper[5052]: I1203 12:22:48.673477 5052 scope.go:117] "RemoveContainer" containerID="09b4a0072741f60499d24f40cc4ff3406d5d22671ced431363ab865172786300" Dec 03 12:22:48 crc kubenswrapper[5052]: E1203 12:22:48.673838 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 12:22:48 crc kubenswrapper[5052]: I1203 12:22:48.876635 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/memcached-0"] Dec 03 12:22:48 crc kubenswrapper[5052]: I1203 12:22:48.885042 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/memcached-0" Dec 03 12:22:48 crc kubenswrapper[5052]: I1203 12:22:48.897712 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/openstack-galera-0"] Dec 03 12:22:48 crc kubenswrapper[5052]: I1203 12:22:48.897786 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/memcached-0"] Dec 03 12:22:48 crc kubenswrapper[5052]: I1203 12:22:48.917744 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"memcached-config-data" Dec 03 12:22:48 crc kubenswrapper[5052]: I1203 12:22:48.917963 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"memcached-memcached-dockercfg-2sngt" Dec 03 12:22:49 crc kubenswrapper[5052]: I1203 12:22:49.050617 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/65fc3838-f25d-4eda-9643-6b63215c41ff-config-data\") pod \"memcached-0\" (UID: \"65fc3838-f25d-4eda-9643-6b63215c41ff\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:22:49 crc kubenswrapper[5052]: I1203 12:22:49.050681 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mrkc5\" (UniqueName: \"kubernetes.io/projected/65fc3838-f25d-4eda-9643-6b63215c41ff-kube-api-access-mrkc5\") pod \"memcached-0\" (UID: \"65fc3838-f25d-4eda-9643-6b63215c41ff\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:22:49 crc kubenswrapper[5052]: I1203 12:22:49.050874 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/65fc3838-f25d-4eda-9643-6b63215c41ff-kolla-config\") pod \"memcached-0\" (UID: \"65fc3838-f25d-4eda-9643-6b63215c41ff\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:22:49 crc kubenswrapper[5052]: I1203 12:22:49.134929 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" event={"ID":"5a5320b7-0541-483a-8e0b-78c10d0f8efa","Type":"ContainerStarted","Data":"6c115b6a73d88d94535850532bd7a63e133ef983572d338d54582ff166612eba"} Dec 03 12:22:49 crc kubenswrapper[5052]: I1203 12:22:49.152169 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-galera-0" event={"ID":"12f39c79-eeb9-4cf4-8553-1a4d9a07ee82","Type":"ContainerStarted","Data":"066ac9e3b2ce853ae0441d00f7b5eee75bdffe353d6f1fb0ffad1c5f36411d87"} Dec 03 12:22:49 crc kubenswrapper[5052]: I1203 12:22:49.153134 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/65fc3838-f25d-4eda-9643-6b63215c41ff-kolla-config\") pod \"memcached-0\" (UID: \"65fc3838-f25d-4eda-9643-6b63215c41ff\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:22:49 crc kubenswrapper[5052]: I1203 12:22:49.153238 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/65fc3838-f25d-4eda-9643-6b63215c41ff-config-data\") pod \"memcached-0\" (UID: \"65fc3838-f25d-4eda-9643-6b63215c41ff\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:22:49 crc kubenswrapper[5052]: I1203 12:22:49.153279 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-mrkc5\" (UniqueName: \"kubernetes.io/projected/65fc3838-f25d-4eda-9643-6b63215c41ff-kube-api-access-mrkc5\") pod \"memcached-0\" (UID: \"65fc3838-f25d-4eda-9643-6b63215c41ff\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:22:49 crc kubenswrapper[5052]: I1203 12:22:49.154364 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/65fc3838-f25d-4eda-9643-6b63215c41ff-kolla-config\") pod \"memcached-0\" (UID: \"65fc3838-f25d-4eda-9643-6b63215c41ff\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:22:49 crc kubenswrapper[5052]: I1203 12:22:49.154984 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/65fc3838-f25d-4eda-9643-6b63215c41ff-config-data\") pod \"memcached-0\" (UID: \"65fc3838-f25d-4eda-9643-6b63215c41ff\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:22:49 crc kubenswrapper[5052]: I1203 12:22:49.159872 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-server-0" event={"ID":"cd7b6179-ba41-43bd-bc95-490a88aa15fd","Type":"ContainerStarted","Data":"c9884448416314162e49f4564ffad485559e0dcde7274fbe0abad7844eeaf00d"} Dec 03 12:22:49 crc kubenswrapper[5052]: I1203 12:22:49.175791 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-mrkc5\" (UniqueName: \"kubernetes.io/projected/65fc3838-f25d-4eda-9643-6b63215c41ff-kube-api-access-mrkc5\") pod \"memcached-0\" (UID: \"65fc3838-f25d-4eda-9643-6b63215c41ff\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:22:49 crc kubenswrapper[5052]: I1203 12:22:49.370050 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/memcached-0" Dec 03 12:22:49 crc kubenswrapper[5052]: I1203 12:22:49.541862 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/kube-state-metrics-0"] Dec 03 12:22:49 crc kubenswrapper[5052]: I1203 12:22:49.555456 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:22:49 crc kubenswrapper[5052]: I1203 12:22:49.558356 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"telemetry-ceilometer-dockercfg-cnhqw" Dec 03 12:22:49 crc kubenswrapper[5052]: I1203 12:22:49.596718 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/kube-state-metrics-0"] Dec 03 12:22:49 crc kubenswrapper[5052]: I1203 12:22:49.667828 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tj7sm\" (UniqueName: \"kubernetes.io/projected/61588a52-3ab5-45a4-a595-3b13bb4bcc28-kube-api-access-tj7sm\") pod \"kube-state-metrics-0\" (UID: \"61588a52-3ab5-45a4-a595-3b13bb4bcc28\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:22:49 crc kubenswrapper[5052]: I1203 12:22:49.769935 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-tj7sm\" (UniqueName: \"kubernetes.io/projected/61588a52-3ab5-45a4-a595-3b13bb4bcc28-kube-api-access-tj7sm\") pod \"kube-state-metrics-0\" (UID: \"61588a52-3ab5-45a4-a595-3b13bb4bcc28\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:22:49 crc kubenswrapper[5052]: I1203 12:22:49.770943 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/openstack-cell1-galera-0"] Dec 03 12:22:49 crc kubenswrapper[5052]: I1203 12:22:49.772707 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:22:49 crc kubenswrapper[5052]: I1203 12:22:49.774703 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"galera-openstack-cell1-dockercfg-8pg58" Dec 03 12:22:49 crc kubenswrapper[5052]: I1203 12:22:49.774932 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-galera-openstack-cell1-svc" Dec 03 12:22:49 crc kubenswrapper[5052]: I1203 12:22:49.775826 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"openstack-cell1-config-data" Dec 03 12:22:49 crc kubenswrapper[5052]: I1203 12:22:49.776018 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"openstack-cell1-scripts" Dec 03 12:22:49 crc kubenswrapper[5052]: I1203 12:22:49.785717 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/openstack-cell1-galera-0"] Dec 03 12:22:49 crc kubenswrapper[5052]: I1203 12:22:49.789824 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-tj7sm\" (UniqueName: \"kubernetes.io/projected/61588a52-3ab5-45a4-a595-3b13bb4bcc28-kube-api-access-tj7sm\") pod \"kube-state-metrics-0\" (UID: \"61588a52-3ab5-45a4-a595-3b13bb4bcc28\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:22:49 crc kubenswrapper[5052]: I1203 12:22:49.871658 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1de1d25b-cbcc-4d08-a7cd-aa54ac71aecb-combined-ca-bundle\") pod \"openstack-cell1-galera-0\" (UID: \"1de1d25b-cbcc-4d08-a7cd-aa54ac71aecb\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:22:49 crc kubenswrapper[5052]: I1203 12:22:49.871751 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-thvgx\" (UniqueName: \"kubernetes.io/projected/1de1d25b-cbcc-4d08-a7cd-aa54ac71aecb-kube-api-access-thvgx\") pod \"openstack-cell1-galera-0\" (UID: \"1de1d25b-cbcc-4d08-a7cd-aa54ac71aecb\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:22:49 crc kubenswrapper[5052]: I1203 12:22:49.871813 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage05-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage05-crc\") pod \"openstack-cell1-galera-0\" (UID: \"1de1d25b-cbcc-4d08-a7cd-aa54ac71aecb\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:22:49 crc kubenswrapper[5052]: I1203 12:22:49.872033 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/1de1d25b-cbcc-4d08-a7cd-aa54ac71aecb-config-data-generated\") pod \"openstack-cell1-galera-0\" (UID: \"1de1d25b-cbcc-4d08-a7cd-aa54ac71aecb\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:22:49 crc kubenswrapper[5052]: I1203 12:22:49.872096 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/1de1d25b-cbcc-4d08-a7cd-aa54ac71aecb-operator-scripts\") pod \"openstack-cell1-galera-0\" (UID: \"1de1d25b-cbcc-4d08-a7cd-aa54ac71aecb\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:22:49 crc kubenswrapper[5052]: I1203 12:22:49.872219 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/1de1d25b-cbcc-4d08-a7cd-aa54ac71aecb-config-data-default\") pod \"openstack-cell1-galera-0\" (UID: \"1de1d25b-cbcc-4d08-a7cd-aa54ac71aecb\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:22:49 crc kubenswrapper[5052]: I1203 12:22:49.872261 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/1de1d25b-cbcc-4d08-a7cd-aa54ac71aecb-kolla-config\") pod \"openstack-cell1-galera-0\" (UID: \"1de1d25b-cbcc-4d08-a7cd-aa54ac71aecb\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:22:49 crc kubenswrapper[5052]: I1203 12:22:49.872317 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/1de1d25b-cbcc-4d08-a7cd-aa54ac71aecb-galera-tls-certs\") pod \"openstack-cell1-galera-0\" (UID: \"1de1d25b-cbcc-4d08-a7cd-aa54ac71aecb\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:22:49 crc kubenswrapper[5052]: I1203 12:22:49.900455 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:22:49 crc kubenswrapper[5052]: I1203 12:22:49.932494 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/memcached-0"] Dec 03 12:22:49 crc kubenswrapper[5052]: W1203 12:22:49.952733 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod65fc3838_f25d_4eda_9643_6b63215c41ff.slice/crio-f7f54f3df6f8da495e9edeb2c7a00827bacd10261cf84a185e76caadd0efb178 WatchSource:0}: Error finding container f7f54f3df6f8da495e9edeb2c7a00827bacd10261cf84a185e76caadd0efb178: Status 404 returned error can't find the container with id f7f54f3df6f8da495e9edeb2c7a00827bacd10261cf84a185e76caadd0efb178 Dec 03 12:22:49 crc kubenswrapper[5052]: I1203 12:22:49.973418 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/1de1d25b-cbcc-4d08-a7cd-aa54ac71aecb-galera-tls-certs\") pod \"openstack-cell1-galera-0\" (UID: \"1de1d25b-cbcc-4d08-a7cd-aa54ac71aecb\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:22:49 crc kubenswrapper[5052]: I1203 12:22:49.973538 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1de1d25b-cbcc-4d08-a7cd-aa54ac71aecb-combined-ca-bundle\") pod \"openstack-cell1-galera-0\" (UID: \"1de1d25b-cbcc-4d08-a7cd-aa54ac71aecb\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:22:49 crc kubenswrapper[5052]: I1203 12:22:49.973607 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-thvgx\" (UniqueName: \"kubernetes.io/projected/1de1d25b-cbcc-4d08-a7cd-aa54ac71aecb-kube-api-access-thvgx\") pod \"openstack-cell1-galera-0\" (UID: \"1de1d25b-cbcc-4d08-a7cd-aa54ac71aecb\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:22:49 crc kubenswrapper[5052]: I1203 12:22:49.973636 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage05-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage05-crc\") pod \"openstack-cell1-galera-0\" (UID: \"1de1d25b-cbcc-4d08-a7cd-aa54ac71aecb\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:22:49 crc kubenswrapper[5052]: I1203 12:22:49.973702 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/1de1d25b-cbcc-4d08-a7cd-aa54ac71aecb-config-data-generated\") pod \"openstack-cell1-galera-0\" (UID: \"1de1d25b-cbcc-4d08-a7cd-aa54ac71aecb\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:22:49 crc kubenswrapper[5052]: I1203 12:22:49.973730 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/1de1d25b-cbcc-4d08-a7cd-aa54ac71aecb-operator-scripts\") pod \"openstack-cell1-galera-0\" (UID: \"1de1d25b-cbcc-4d08-a7cd-aa54ac71aecb\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:22:49 crc kubenswrapper[5052]: I1203 12:22:49.973818 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/1de1d25b-cbcc-4d08-a7cd-aa54ac71aecb-config-data-default\") pod \"openstack-cell1-galera-0\" (UID: \"1de1d25b-cbcc-4d08-a7cd-aa54ac71aecb\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:22:49 crc kubenswrapper[5052]: I1203 12:22:49.973870 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/1de1d25b-cbcc-4d08-a7cd-aa54ac71aecb-kolla-config\") pod \"openstack-cell1-galera-0\" (UID: \"1de1d25b-cbcc-4d08-a7cd-aa54ac71aecb\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:22:49 crc kubenswrapper[5052]: I1203 12:22:49.974798 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/1de1d25b-cbcc-4d08-a7cd-aa54ac71aecb-kolla-config\") pod \"openstack-cell1-galera-0\" (UID: \"1de1d25b-cbcc-4d08-a7cd-aa54ac71aecb\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:22:49 crc kubenswrapper[5052]: I1203 12:22:49.975149 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage05-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage05-crc\") pod \"openstack-cell1-galera-0\" (UID: \"1de1d25b-cbcc-4d08-a7cd-aa54ac71aecb\") device mount path \"/mnt/openstack/pv05\"" pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:22:49 crc kubenswrapper[5052]: I1203 12:22:49.975578 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/1de1d25b-cbcc-4d08-a7cd-aa54ac71aecb-config-data-generated\") pod \"openstack-cell1-galera-0\" (UID: \"1de1d25b-cbcc-4d08-a7cd-aa54ac71aecb\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:22:49 crc kubenswrapper[5052]: I1203 12:22:49.976606 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/1de1d25b-cbcc-4d08-a7cd-aa54ac71aecb-config-data-default\") pod \"openstack-cell1-galera-0\" (UID: \"1de1d25b-cbcc-4d08-a7cd-aa54ac71aecb\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:22:49 crc kubenswrapper[5052]: I1203 12:22:49.977147 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/1de1d25b-cbcc-4d08-a7cd-aa54ac71aecb-operator-scripts\") pod \"openstack-cell1-galera-0\" (UID: \"1de1d25b-cbcc-4d08-a7cd-aa54ac71aecb\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:22:49 crc kubenswrapper[5052]: I1203 12:22:49.979816 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1de1d25b-cbcc-4d08-a7cd-aa54ac71aecb-combined-ca-bundle\") pod \"openstack-cell1-galera-0\" (UID: \"1de1d25b-cbcc-4d08-a7cd-aa54ac71aecb\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:22:49 crc kubenswrapper[5052]: I1203 12:22:49.979958 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/1de1d25b-cbcc-4d08-a7cd-aa54ac71aecb-galera-tls-certs\") pod \"openstack-cell1-galera-0\" (UID: \"1de1d25b-cbcc-4d08-a7cd-aa54ac71aecb\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:22:49 crc kubenswrapper[5052]: I1203 12:22:49.997996 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage05-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage05-crc\") pod \"openstack-cell1-galera-0\" (UID: \"1de1d25b-cbcc-4d08-a7cd-aa54ac71aecb\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:22:50 crc kubenswrapper[5052]: I1203 12:22:50.000043 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-thvgx\" (UniqueName: \"kubernetes.io/projected/1de1d25b-cbcc-4d08-a7cd-aa54ac71aecb-kube-api-access-thvgx\") pod \"openstack-cell1-galera-0\" (UID: \"1de1d25b-cbcc-4d08-a7cd-aa54ac71aecb\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:22:50 crc kubenswrapper[5052]: I1203 12:22:50.088435 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:22:50 crc kubenswrapper[5052]: I1203 12:22:50.094652 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ovsdbserver-nb-0"] Dec 03 12:22:50 crc kubenswrapper[5052]: I1203 12:22:50.096227 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:22:50 crc kubenswrapper[5052]: I1203 12:22:50.101940 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-nb-0"] Dec 03 12:22:50 crc kubenswrapper[5052]: I1203 12:22:50.104003 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"ovndbcluster-nb-config" Dec 03 12:22:50 crc kubenswrapper[5052]: I1203 12:22:50.104341 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"ovndbcluster-nb-scripts" Dec 03 12:22:50 crc kubenswrapper[5052]: I1203 12:22:50.104546 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ovncluster-ovndbcluster-nb-dockercfg-4462b" Dec 03 12:22:50 crc kubenswrapper[5052]: I1203 12:22:50.182592 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/memcached-0" event={"ID":"65fc3838-f25d-4eda-9643-6b63215c41ff","Type":"ContainerStarted","Data":"f7f54f3df6f8da495e9edeb2c7a00827bacd10261cf84a185e76caadd0efb178"} Dec 03 12:22:50 crc kubenswrapper[5052]: I1203 12:22:50.183030 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v4jtp\" (UniqueName: \"kubernetes.io/projected/3b588591-8d98-427d-979f-72e7c8260b81-kube-api-access-v4jtp\") pod \"ovsdbserver-nb-0\" (UID: \"3b588591-8d98-427d-979f-72e7c8260b81\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:22:50 crc kubenswrapper[5052]: I1203 12:22:50.183100 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/3b588591-8d98-427d-979f-72e7c8260b81-scripts\") pod \"ovsdbserver-nb-0\" (UID: \"3b588591-8d98-427d-979f-72e7c8260b81\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:22:50 crc kubenswrapper[5052]: I1203 12:22:50.183134 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/3b588591-8d98-427d-979f-72e7c8260b81-ovsdb-rundir\") pod \"ovsdbserver-nb-0\" (UID: \"3b588591-8d98-427d-979f-72e7c8260b81\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:22:50 crc kubenswrapper[5052]: I1203 12:22:50.183169 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3b588591-8d98-427d-979f-72e7c8260b81-config\") pod \"ovsdbserver-nb-0\" (UID: \"3b588591-8d98-427d-979f-72e7c8260b81\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:22:50 crc kubenswrapper[5052]: I1203 12:22:50.183275 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3b588591-8d98-427d-979f-72e7c8260b81-combined-ca-bundle\") pod \"ovsdbserver-nb-0\" (UID: \"3b588591-8d98-427d-979f-72e7c8260b81\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:22:50 crc kubenswrapper[5052]: I1203 12:22:50.183358 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage03-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage03-crc\") pod \"ovsdbserver-nb-0\" (UID: \"3b588591-8d98-427d-979f-72e7c8260b81\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:22:50 crc kubenswrapper[5052]: I1203 12:22:50.194610 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-galera-0" event={"ID":"12f39c79-eeb9-4cf4-8553-1a4d9a07ee82","Type":"ContainerStarted","Data":"dcde059b067d1c9d50353c5be63ee2f4f97d8343ecfb247c42768f1aba1f469c"} Dec 03 12:22:50 crc kubenswrapper[5052]: I1203 12:22:50.285096 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/3b588591-8d98-427d-979f-72e7c8260b81-scripts\") pod \"ovsdbserver-nb-0\" (UID: \"3b588591-8d98-427d-979f-72e7c8260b81\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:22:50 crc kubenswrapper[5052]: I1203 12:22:50.285198 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/3b588591-8d98-427d-979f-72e7c8260b81-ovsdb-rundir\") pod \"ovsdbserver-nb-0\" (UID: \"3b588591-8d98-427d-979f-72e7c8260b81\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:22:50 crc kubenswrapper[5052]: I1203 12:22:50.285285 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3b588591-8d98-427d-979f-72e7c8260b81-config\") pod \"ovsdbserver-nb-0\" (UID: \"3b588591-8d98-427d-979f-72e7c8260b81\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:22:50 crc kubenswrapper[5052]: I1203 12:22:50.285388 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3b588591-8d98-427d-979f-72e7c8260b81-combined-ca-bundle\") pod \"ovsdbserver-nb-0\" (UID: \"3b588591-8d98-427d-979f-72e7c8260b81\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:22:50 crc kubenswrapper[5052]: I1203 12:22:50.285632 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage03-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage03-crc\") pod \"ovsdbserver-nb-0\" (UID: \"3b588591-8d98-427d-979f-72e7c8260b81\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:22:50 crc kubenswrapper[5052]: I1203 12:22:50.286400 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/3b588591-8d98-427d-979f-72e7c8260b81-ovsdb-rundir\") pod \"ovsdbserver-nb-0\" (UID: \"3b588591-8d98-427d-979f-72e7c8260b81\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:22:50 crc kubenswrapper[5052]: I1203 12:22:50.287035 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage03-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage03-crc\") pod \"ovsdbserver-nb-0\" (UID: \"3b588591-8d98-427d-979f-72e7c8260b81\") device mount path \"/mnt/openstack/pv03\"" pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:22:50 crc kubenswrapper[5052]: I1203 12:22:50.287348 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-v4jtp\" (UniqueName: \"kubernetes.io/projected/3b588591-8d98-427d-979f-72e7c8260b81-kube-api-access-v4jtp\") pod \"ovsdbserver-nb-0\" (UID: \"3b588591-8d98-427d-979f-72e7c8260b81\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:22:50 crc kubenswrapper[5052]: I1203 12:22:50.287689 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3b588591-8d98-427d-979f-72e7c8260b81-config\") pod \"ovsdbserver-nb-0\" (UID: \"3b588591-8d98-427d-979f-72e7c8260b81\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:22:50 crc kubenswrapper[5052]: I1203 12:22:50.287710 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/3b588591-8d98-427d-979f-72e7c8260b81-scripts\") pod \"ovsdbserver-nb-0\" (UID: \"3b588591-8d98-427d-979f-72e7c8260b81\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:22:50 crc kubenswrapper[5052]: I1203 12:22:50.293343 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3b588591-8d98-427d-979f-72e7c8260b81-combined-ca-bundle\") pod \"ovsdbserver-nb-0\" (UID: \"3b588591-8d98-427d-979f-72e7c8260b81\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:22:50 crc kubenswrapper[5052]: I1203 12:22:50.305412 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-v4jtp\" (UniqueName: \"kubernetes.io/projected/3b588591-8d98-427d-979f-72e7c8260b81-kube-api-access-v4jtp\") pod \"ovsdbserver-nb-0\" (UID: \"3b588591-8d98-427d-979f-72e7c8260b81\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:22:50 crc kubenswrapper[5052]: I1203 12:22:50.317756 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage03-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage03-crc\") pod \"ovsdbserver-nb-0\" (UID: \"3b588591-8d98-427d-979f-72e7c8260b81\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:22:50 crc kubenswrapper[5052]: I1203 12:22:50.392625 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/kube-state-metrics-0"] Dec 03 12:22:50 crc kubenswrapper[5052]: W1203 12:22:50.394940 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod61588a52_3ab5_45a4_a595_3b13bb4bcc28.slice/crio-64d60f730a695a2d3dd995103f5408750656e118d65fab2dcfbd614393d8bd04 WatchSource:0}: Error finding container 64d60f730a695a2d3dd995103f5408750656e118d65fab2dcfbd614393d8bd04: Status 404 returned error can't find the container with id 64d60f730a695a2d3dd995103f5408750656e118d65fab2dcfbd614393d8bd04 Dec 03 12:22:50 crc kubenswrapper[5052]: I1203 12:22:50.440983 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ovsdbserver-sb-0"] Dec 03 12:22:50 crc kubenswrapper[5052]: I1203 12:22:50.442658 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:22:50 crc kubenswrapper[5052]: I1203 12:22:50.446802 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"ovndbcluster-sb-config" Dec 03 12:22:50 crc kubenswrapper[5052]: I1203 12:22:50.447925 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"ovndbcluster-sb-scripts" Dec 03 12:22:50 crc kubenswrapper[5052]: I1203 12:22:50.448112 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ovncluster-ovndbcluster-sb-dockercfg-2ckjk" Dec 03 12:22:50 crc kubenswrapper[5052]: I1203 12:22:50.452480 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:22:50 crc kubenswrapper[5052]: I1203 12:22:50.452894 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-sb-0"] Dec 03 12:22:50 crc kubenswrapper[5052]: I1203 12:22:50.490948 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage08-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage08-crc\") pod \"ovsdbserver-sb-0\" (UID: \"c71ba7fe-fd95-4f72-a585-6e44cafa116e\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:22:50 crc kubenswrapper[5052]: I1203 12:22:50.491016 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/c71ba7fe-fd95-4f72-a585-6e44cafa116e-scripts\") pod \"ovsdbserver-sb-0\" (UID: \"c71ba7fe-fd95-4f72-a585-6e44cafa116e\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:22:50 crc kubenswrapper[5052]: I1203 12:22:50.491052 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c71ba7fe-fd95-4f72-a585-6e44cafa116e-combined-ca-bundle\") pod \"ovsdbserver-sb-0\" (UID: \"c71ba7fe-fd95-4f72-a585-6e44cafa116e\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:22:50 crc kubenswrapper[5052]: I1203 12:22:50.491088 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c71ba7fe-fd95-4f72-a585-6e44cafa116e-config\") pod \"ovsdbserver-sb-0\" (UID: \"c71ba7fe-fd95-4f72-a585-6e44cafa116e\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:22:50 crc kubenswrapper[5052]: I1203 12:22:50.491132 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-78cj2\" (UniqueName: \"kubernetes.io/projected/c71ba7fe-fd95-4f72-a585-6e44cafa116e-kube-api-access-78cj2\") pod \"ovsdbserver-sb-0\" (UID: \"c71ba7fe-fd95-4f72-a585-6e44cafa116e\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:22:50 crc kubenswrapper[5052]: I1203 12:22:50.492356 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/c71ba7fe-fd95-4f72-a585-6e44cafa116e-ovsdb-rundir\") pod \"ovsdbserver-sb-0\" (UID: \"c71ba7fe-fd95-4f72-a585-6e44cafa116e\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:22:50 crc kubenswrapper[5052]: I1203 12:22:50.564414 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/openstack-cell1-galera-0"] Dec 03 12:22:50 crc kubenswrapper[5052]: W1203 12:22:50.581276 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1de1d25b_cbcc_4d08_a7cd_aa54ac71aecb.slice/crio-bcaa05975157bb25a2852a7e2a286e65dd456e820f2697cfde9f55c2ff7275f9 WatchSource:0}: Error finding container bcaa05975157bb25a2852a7e2a286e65dd456e820f2697cfde9f55c2ff7275f9: Status 404 returned error can't find the container with id bcaa05975157bb25a2852a7e2a286e65dd456e820f2697cfde9f55c2ff7275f9 Dec 03 12:22:50 crc kubenswrapper[5052]: I1203 12:22:50.594453 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage08-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage08-crc\") pod \"ovsdbserver-sb-0\" (UID: \"c71ba7fe-fd95-4f72-a585-6e44cafa116e\") device mount path \"/mnt/openstack/pv08\"" pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:22:50 crc kubenswrapper[5052]: I1203 12:22:50.595789 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage08-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage08-crc\") pod \"ovsdbserver-sb-0\" (UID: \"c71ba7fe-fd95-4f72-a585-6e44cafa116e\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:22:50 crc kubenswrapper[5052]: I1203 12:22:50.595880 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/c71ba7fe-fd95-4f72-a585-6e44cafa116e-scripts\") pod \"ovsdbserver-sb-0\" (UID: \"c71ba7fe-fd95-4f72-a585-6e44cafa116e\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:22:50 crc kubenswrapper[5052]: I1203 12:22:50.595915 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c71ba7fe-fd95-4f72-a585-6e44cafa116e-combined-ca-bundle\") pod \"ovsdbserver-sb-0\" (UID: \"c71ba7fe-fd95-4f72-a585-6e44cafa116e\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:22:50 crc kubenswrapper[5052]: I1203 12:22:50.595953 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c71ba7fe-fd95-4f72-a585-6e44cafa116e-config\") pod \"ovsdbserver-sb-0\" (UID: \"c71ba7fe-fd95-4f72-a585-6e44cafa116e\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:22:50 crc kubenswrapper[5052]: I1203 12:22:50.596017 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-78cj2\" (UniqueName: \"kubernetes.io/projected/c71ba7fe-fd95-4f72-a585-6e44cafa116e-kube-api-access-78cj2\") pod \"ovsdbserver-sb-0\" (UID: \"c71ba7fe-fd95-4f72-a585-6e44cafa116e\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:22:50 crc kubenswrapper[5052]: I1203 12:22:50.596080 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/c71ba7fe-fd95-4f72-a585-6e44cafa116e-ovsdb-rundir\") pod \"ovsdbserver-sb-0\" (UID: \"c71ba7fe-fd95-4f72-a585-6e44cafa116e\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:22:50 crc kubenswrapper[5052]: I1203 12:22:50.596816 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/c71ba7fe-fd95-4f72-a585-6e44cafa116e-ovsdb-rundir\") pod \"ovsdbserver-sb-0\" (UID: \"c71ba7fe-fd95-4f72-a585-6e44cafa116e\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:22:50 crc kubenswrapper[5052]: I1203 12:22:50.597308 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/c71ba7fe-fd95-4f72-a585-6e44cafa116e-scripts\") pod \"ovsdbserver-sb-0\" (UID: \"c71ba7fe-fd95-4f72-a585-6e44cafa116e\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:22:50 crc kubenswrapper[5052]: I1203 12:22:50.597694 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c71ba7fe-fd95-4f72-a585-6e44cafa116e-config\") pod \"ovsdbserver-sb-0\" (UID: \"c71ba7fe-fd95-4f72-a585-6e44cafa116e\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:22:50 crc kubenswrapper[5052]: I1203 12:22:50.600593 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c71ba7fe-fd95-4f72-a585-6e44cafa116e-combined-ca-bundle\") pod \"ovsdbserver-sb-0\" (UID: \"c71ba7fe-fd95-4f72-a585-6e44cafa116e\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:22:50 crc kubenswrapper[5052]: I1203 12:22:50.619387 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-78cj2\" (UniqueName: \"kubernetes.io/projected/c71ba7fe-fd95-4f72-a585-6e44cafa116e-kube-api-access-78cj2\") pod \"ovsdbserver-sb-0\" (UID: \"c71ba7fe-fd95-4f72-a585-6e44cafa116e\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:22:50 crc kubenswrapper[5052]: I1203 12:22:50.629788 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage08-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage08-crc\") pod \"ovsdbserver-sb-0\" (UID: \"c71ba7fe-fd95-4f72-a585-6e44cafa116e\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:22:50 crc kubenswrapper[5052]: I1203 12:22:50.769055 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:22:50 crc kubenswrapper[5052]: I1203 12:22:50.944501 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-nb-0"] Dec 03 12:22:50 crc kubenswrapper[5052]: W1203 12:22:50.958702 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3b588591_8d98_427d_979f_72e7c8260b81.slice/crio-0a44cceb24da4f58ea4816394380042c5acd3c9f07ed421a2d59cf9790dc50f1 WatchSource:0}: Error finding container 0a44cceb24da4f58ea4816394380042c5acd3c9f07ed421a2d59cf9790dc50f1: Status 404 returned error can't find the container with id 0a44cceb24da4f58ea4816394380042c5acd3c9f07ed421a2d59cf9790dc50f1 Dec 03 12:22:51 crc kubenswrapper[5052]: I1203 12:22:51.225065 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-sb-0"] Dec 03 12:22:51 crc kubenswrapper[5052]: I1203 12:22:51.225128 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-cell1-galera-0" event={"ID":"1de1d25b-cbcc-4d08-a7cd-aa54ac71aecb","Type":"ContainerStarted","Data":"2ccaa25866b1b39590c7e2470712343f212e8930f1a8cf1f7f1ec6457651bbae"} Dec 03 12:22:51 crc kubenswrapper[5052]: I1203 12:22:51.225154 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-cell1-galera-0" event={"ID":"1de1d25b-cbcc-4d08-a7cd-aa54ac71aecb","Type":"ContainerStarted","Data":"bcaa05975157bb25a2852a7e2a286e65dd456e820f2697cfde9f55c2ff7275f9"} Dec 03 12:22:51 crc kubenswrapper[5052]: W1203 12:22:51.230390 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc71ba7fe_fd95_4f72_a585_6e44cafa116e.slice/crio-d46eff63c6dc723589bc8fdfc72f53a6a78a9b10307c6d3a9dc6fd6dbe3028b5 WatchSource:0}: Error finding container d46eff63c6dc723589bc8fdfc72f53a6a78a9b10307c6d3a9dc6fd6dbe3028b5: Status 404 returned error can't find the container with id d46eff63c6dc723589bc8fdfc72f53a6a78a9b10307c6d3a9dc6fd6dbe3028b5 Dec 03 12:22:51 crc kubenswrapper[5052]: I1203 12:22:51.233596 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/kube-state-metrics-0" event={"ID":"61588a52-3ab5-45a4-a595-3b13bb4bcc28","Type":"ContainerStarted","Data":"7f4e8c9236258e1ee13e282db4d527b630572e15c3e43c16ee3932b425f0f4a6"} Dec 03 12:22:51 crc kubenswrapper[5052]: I1203 12:22:51.233652 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/kube-state-metrics-0" event={"ID":"61588a52-3ab5-45a4-a595-3b13bb4bcc28","Type":"ContainerStarted","Data":"64d60f730a695a2d3dd995103f5408750656e118d65fab2dcfbd614393d8bd04"} Dec 03 12:22:51 crc kubenswrapper[5052]: I1203 12:22:51.234187 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:22:51 crc kubenswrapper[5052]: I1203 12:22:51.243964 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/memcached-0" event={"ID":"65fc3838-f25d-4eda-9643-6b63215c41ff","Type":"ContainerStarted","Data":"0931db22f5fb013cdcb55311764fa795389b7087d437f3d1b8323540414014d9"} Dec 03 12:22:51 crc kubenswrapper[5052]: I1203 12:22:51.244160 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/memcached-0" Dec 03 12:22:51 crc kubenswrapper[5052]: I1203 12:22:51.260321 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-nb-0" event={"ID":"3b588591-8d98-427d-979f-72e7c8260b81","Type":"ContainerStarted","Data":"6b08aea5dde842833792afa4a6cfad11a22c78249915f3be8d0c1416ac2fca1f"} Dec 03 12:22:51 crc kubenswrapper[5052]: I1203 12:22:51.260373 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-nb-0" event={"ID":"3b588591-8d98-427d-979f-72e7c8260b81","Type":"ContainerStarted","Data":"0a44cceb24da4f58ea4816394380042c5acd3c9f07ed421a2d59cf9790dc50f1"} Dec 03 12:22:51 crc kubenswrapper[5052]: I1203 12:22:51.274235 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/memcached-0" podStartSLOduration=3.274208317 podStartE2EDuration="3.274208317s" podCreationTimestamp="2025-12-03 12:22:48 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:22:51.266585309 +0000 UTC m=+3761.854827073" watchObservedRunningTime="2025-12-03 12:22:51.274208317 +0000 UTC m=+3761.862450081" Dec 03 12:22:51 crc kubenswrapper[5052]: I1203 12:22:51.285341 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/kube-state-metrics-0" podStartSLOduration=1.750223581 podStartE2EDuration="2.285324134s" podCreationTimestamp="2025-12-03 12:22:49 +0000 UTC" firstStartedPulling="2025-12-03 12:22:50.398213055 +0000 UTC m=+3760.986454819" lastFinishedPulling="2025-12-03 12:22:50.933313608 +0000 UTC m=+3761.521555372" observedRunningTime="2025-12-03 12:22:51.285079727 +0000 UTC m=+3761.873321491" watchObservedRunningTime="2025-12-03 12:22:51.285324134 +0000 UTC m=+3761.873565898" Dec 03 12:22:52 crc kubenswrapper[5052]: I1203 12:22:52.273078 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-sb-0" event={"ID":"c71ba7fe-fd95-4f72-a585-6e44cafa116e","Type":"ContainerStarted","Data":"da31a3a0863901281ff074219ed7108e2f7b3e54f4ed5c3a2dc8467247c39bcb"} Dec 03 12:22:52 crc kubenswrapper[5052]: I1203 12:22:52.274490 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-sb-0" event={"ID":"c71ba7fe-fd95-4f72-a585-6e44cafa116e","Type":"ContainerStarted","Data":"5e3dd2ec9e951bd3f8596a23352b92ccc95b07f5ba4091471837424610f5aa50"} Dec 03 12:22:52 crc kubenswrapper[5052]: I1203 12:22:52.274635 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-sb-0" event={"ID":"c71ba7fe-fd95-4f72-a585-6e44cafa116e","Type":"ContainerStarted","Data":"d46eff63c6dc723589bc8fdfc72f53a6a78a9b10307c6d3a9dc6fd6dbe3028b5"} Dec 03 12:22:52 crc kubenswrapper[5052]: I1203 12:22:52.277183 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-nb-0" event={"ID":"3b588591-8d98-427d-979f-72e7c8260b81","Type":"ContainerStarted","Data":"1cff9f5854a876c596bb8b12d2477f784cfb487900c01da703c539509eeff1df"} Dec 03 12:22:52 crc kubenswrapper[5052]: I1203 12:22:52.293925 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/ovsdbserver-sb-0" podStartSLOduration=3.293905341 podStartE2EDuration="3.293905341s" podCreationTimestamp="2025-12-03 12:22:49 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:22:52.289772463 +0000 UTC m=+3762.878014227" watchObservedRunningTime="2025-12-03 12:22:52.293905341 +0000 UTC m=+3762.882147105" Dec 03 12:22:52 crc kubenswrapper[5052]: I1203 12:22:52.312991 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/ovsdbserver-nb-0" podStartSLOduration=3.312953055 podStartE2EDuration="3.312953055s" podCreationTimestamp="2025-12-03 12:22:49 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:22:52.307401557 +0000 UTC m=+3762.895643321" watchObservedRunningTime="2025-12-03 12:22:52.312953055 +0000 UTC m=+3762.901194819" Dec 03 12:22:53 crc kubenswrapper[5052]: I1203 12:22:53.453316 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:22:53 crc kubenswrapper[5052]: I1203 12:22:53.769904 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:22:54 crc kubenswrapper[5052]: I1203 12:22:54.296947 5052 generic.go:334] "Generic (PLEG): container finished" podID="12f39c79-eeb9-4cf4-8553-1a4d9a07ee82" containerID="dcde059b067d1c9d50353c5be63ee2f4f97d8343ecfb247c42768f1aba1f469c" exitCode=0 Dec 03 12:22:54 crc kubenswrapper[5052]: I1203 12:22:54.297372 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-galera-0" event={"ID":"12f39c79-eeb9-4cf4-8553-1a4d9a07ee82","Type":"ContainerDied","Data":"dcde059b067d1c9d50353c5be63ee2f4f97d8343ecfb247c42768f1aba1f469c"} Dec 03 12:22:55 crc kubenswrapper[5052]: I1203 12:22:55.309981 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-galera-0" event={"ID":"12f39c79-eeb9-4cf4-8553-1a4d9a07ee82","Type":"ContainerStarted","Data":"165a4ba65f86633615c852ba29ceb303c3bbd0dbd696084cf0c3a26afe0a4424"} Dec 03 12:22:55 crc kubenswrapper[5052]: I1203 12:22:55.312018 5052 generic.go:334] "Generic (PLEG): container finished" podID="1de1d25b-cbcc-4d08-a7cd-aa54ac71aecb" containerID="2ccaa25866b1b39590c7e2470712343f212e8930f1a8cf1f7f1ec6457651bbae" exitCode=0 Dec 03 12:22:55 crc kubenswrapper[5052]: I1203 12:22:55.312294 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-cell1-galera-0" event={"ID":"1de1d25b-cbcc-4d08-a7cd-aa54ac71aecb","Type":"ContainerDied","Data":"2ccaa25866b1b39590c7e2470712343f212e8930f1a8cf1f7f1ec6457651bbae"} Dec 03 12:22:55 crc kubenswrapper[5052]: I1203 12:22:55.337872 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/openstack-galera-0" podStartSLOduration=8.337855492 podStartE2EDuration="8.337855492s" podCreationTimestamp="2025-12-03 12:22:47 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:22:55.334408893 +0000 UTC m=+3765.922650657" watchObservedRunningTime="2025-12-03 12:22:55.337855492 +0000 UTC m=+3765.926097256" Dec 03 12:22:55 crc kubenswrapper[5052]: I1203 12:22:55.453395 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:22:55 crc kubenswrapper[5052]: I1203 12:22:55.769990 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:22:56 crc kubenswrapper[5052]: I1203 12:22:56.324977 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-cell1-galera-0" event={"ID":"1de1d25b-cbcc-4d08-a7cd-aa54ac71aecb","Type":"ContainerStarted","Data":"46246c5deed2c10d0bc81011e4be47e5c70c228f4dda6fb9a1b20810aff514c7"} Dec 03 12:22:56 crc kubenswrapper[5052]: I1203 12:22:56.346433 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/openstack-cell1-galera-0" podStartSLOduration=8.346416418 podStartE2EDuration="8.346416418s" podCreationTimestamp="2025-12-03 12:22:48 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:22:56.343642029 +0000 UTC m=+3766.931883813" watchObservedRunningTime="2025-12-03 12:22:56.346416418 +0000 UTC m=+3766.934658182" Dec 03 12:22:56 crc kubenswrapper[5052]: I1203 12:22:56.487809 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:22:56 crc kubenswrapper[5052]: I1203 12:22:56.531263 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:22:56 crc kubenswrapper[5052]: I1203 12:22:56.628808 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openshift-marketplace/redhat-marketplace-hx9cz"] Dec 03 12:22:56 crc kubenswrapper[5052]: I1203 12:22:56.630783 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/redhat-marketplace-hx9cz" Dec 03 12:22:56 crc kubenswrapper[5052]: I1203 12:22:56.667564 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/redhat-marketplace-hx9cz"] Dec 03 12:22:56 crc kubenswrapper[5052]: I1203 12:22:56.698917 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/fa334119-b1fd-42e8-b64b-a276ef89e4b5-utilities\") pod \"redhat-marketplace-hx9cz\" (UID: \"fa334119-b1fd-42e8-b64b-a276ef89e4b5\") " pod="openshift-marketplace/redhat-marketplace-hx9cz" Dec 03 12:22:56 crc kubenswrapper[5052]: I1203 12:22:56.698993 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/fa334119-b1fd-42e8-b64b-a276ef89e4b5-catalog-content\") pod \"redhat-marketplace-hx9cz\" (UID: \"fa334119-b1fd-42e8-b64b-a276ef89e4b5\") " pod="openshift-marketplace/redhat-marketplace-hx9cz" Dec 03 12:22:56 crc kubenswrapper[5052]: I1203 12:22:56.699062 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9jpdl\" (UniqueName: \"kubernetes.io/projected/fa334119-b1fd-42e8-b64b-a276ef89e4b5-kube-api-access-9jpdl\") pod \"redhat-marketplace-hx9cz\" (UID: \"fa334119-b1fd-42e8-b64b-a276ef89e4b5\") " pod="openshift-marketplace/redhat-marketplace-hx9cz" Dec 03 12:22:56 crc kubenswrapper[5052]: I1203 12:22:56.799950 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/fa334119-b1fd-42e8-b64b-a276ef89e4b5-utilities\") pod \"redhat-marketplace-hx9cz\" (UID: \"fa334119-b1fd-42e8-b64b-a276ef89e4b5\") " pod="openshift-marketplace/redhat-marketplace-hx9cz" Dec 03 12:22:56 crc kubenswrapper[5052]: I1203 12:22:56.800017 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/fa334119-b1fd-42e8-b64b-a276ef89e4b5-catalog-content\") pod \"redhat-marketplace-hx9cz\" (UID: \"fa334119-b1fd-42e8-b64b-a276ef89e4b5\") " pod="openshift-marketplace/redhat-marketplace-hx9cz" Dec 03 12:22:56 crc kubenswrapper[5052]: I1203 12:22:56.800061 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-9jpdl\" (UniqueName: \"kubernetes.io/projected/fa334119-b1fd-42e8-b64b-a276ef89e4b5-kube-api-access-9jpdl\") pod \"redhat-marketplace-hx9cz\" (UID: \"fa334119-b1fd-42e8-b64b-a276ef89e4b5\") " pod="openshift-marketplace/redhat-marketplace-hx9cz" Dec 03 12:22:56 crc kubenswrapper[5052]: I1203 12:22:56.800503 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/fa334119-b1fd-42e8-b64b-a276ef89e4b5-utilities\") pod \"redhat-marketplace-hx9cz\" (UID: \"fa334119-b1fd-42e8-b64b-a276ef89e4b5\") " pod="openshift-marketplace/redhat-marketplace-hx9cz" Dec 03 12:22:56 crc kubenswrapper[5052]: I1203 12:22:56.800660 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/fa334119-b1fd-42e8-b64b-a276ef89e4b5-catalog-content\") pod \"redhat-marketplace-hx9cz\" (UID: \"fa334119-b1fd-42e8-b64b-a276ef89e4b5\") " pod="openshift-marketplace/redhat-marketplace-hx9cz" Dec 03 12:22:56 crc kubenswrapper[5052]: I1203 12:22:56.814503 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:22:56 crc kubenswrapper[5052]: I1203 12:22:56.820369 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-9jpdl\" (UniqueName: \"kubernetes.io/projected/fa334119-b1fd-42e8-b64b-a276ef89e4b5-kube-api-access-9jpdl\") pod \"redhat-marketplace-hx9cz\" (UID: \"fa334119-b1fd-42e8-b64b-a276ef89e4b5\") " pod="openshift-marketplace/redhat-marketplace-hx9cz" Dec 03 12:22:56 crc kubenswrapper[5052]: I1203 12:22:56.963708 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/redhat-marketplace-hx9cz" Dec 03 12:22:57 crc kubenswrapper[5052]: I1203 12:22:57.374047 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:22:57 crc kubenswrapper[5052]: I1203 12:22:57.453220 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/redhat-marketplace-hx9cz"] Dec 03 12:22:57 crc kubenswrapper[5052]: I1203 12:22:57.522651 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ovn-northd-0"] Dec 03 12:22:57 crc kubenswrapper[5052]: I1203 12:22:57.530206 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:22:57 crc kubenswrapper[5052]: I1203 12:22:57.535184 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"ovnnorthd-config" Dec 03 12:22:57 crc kubenswrapper[5052]: I1203 12:22:57.535271 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"ovnnorthd-scripts" Dec 03 12:22:57 crc kubenswrapper[5052]: I1203 12:22:57.535335 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ovnnorthd-ovnnorthd-dockercfg-lp65l" Dec 03 12:22:57 crc kubenswrapper[5052]: I1203 12:22:57.545610 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ovn-northd-0"] Dec 03 12:22:57 crc kubenswrapper[5052]: I1203 12:22:57.614108 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-rundir\" (UniqueName: \"kubernetes.io/empty-dir/18e325b5-7837-4776-ae05-d9a12ccf9746-ovn-rundir\") pod \"ovn-northd-0\" (UID: \"18e325b5-7837-4776-ae05-d9a12ccf9746\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:22:57 crc kubenswrapper[5052]: I1203 12:22:57.614462 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/18e325b5-7837-4776-ae05-d9a12ccf9746-config\") pod \"ovn-northd-0\" (UID: \"18e325b5-7837-4776-ae05-d9a12ccf9746\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:22:57 crc kubenswrapper[5052]: I1203 12:22:57.614498 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/18e325b5-7837-4776-ae05-d9a12ccf9746-scripts\") pod \"ovn-northd-0\" (UID: \"18e325b5-7837-4776-ae05-d9a12ccf9746\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:22:57 crc kubenswrapper[5052]: I1203 12:22:57.614559 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/18e325b5-7837-4776-ae05-d9a12ccf9746-combined-ca-bundle\") pod \"ovn-northd-0\" (UID: \"18e325b5-7837-4776-ae05-d9a12ccf9746\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:22:57 crc kubenswrapper[5052]: I1203 12:22:57.614581 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jpxt6\" (UniqueName: \"kubernetes.io/projected/18e325b5-7837-4776-ae05-d9a12ccf9746-kube-api-access-jpxt6\") pod \"ovn-northd-0\" (UID: \"18e325b5-7837-4776-ae05-d9a12ccf9746\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:22:57 crc kubenswrapper[5052]: I1203 12:22:57.715763 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/18e325b5-7837-4776-ae05-d9a12ccf9746-config\") pod \"ovn-northd-0\" (UID: \"18e325b5-7837-4776-ae05-d9a12ccf9746\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:22:57 crc kubenswrapper[5052]: I1203 12:22:57.715849 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/18e325b5-7837-4776-ae05-d9a12ccf9746-scripts\") pod \"ovn-northd-0\" (UID: \"18e325b5-7837-4776-ae05-d9a12ccf9746\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:22:57 crc kubenswrapper[5052]: I1203 12:22:57.715931 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/18e325b5-7837-4776-ae05-d9a12ccf9746-combined-ca-bundle\") pod \"ovn-northd-0\" (UID: \"18e325b5-7837-4776-ae05-d9a12ccf9746\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:22:57 crc kubenswrapper[5052]: I1203 12:22:57.715951 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-jpxt6\" (UniqueName: \"kubernetes.io/projected/18e325b5-7837-4776-ae05-d9a12ccf9746-kube-api-access-jpxt6\") pod \"ovn-northd-0\" (UID: \"18e325b5-7837-4776-ae05-d9a12ccf9746\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:22:57 crc kubenswrapper[5052]: I1203 12:22:57.716028 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovn-rundir\" (UniqueName: \"kubernetes.io/empty-dir/18e325b5-7837-4776-ae05-d9a12ccf9746-ovn-rundir\") pod \"ovn-northd-0\" (UID: \"18e325b5-7837-4776-ae05-d9a12ccf9746\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:22:57 crc kubenswrapper[5052]: I1203 12:22:57.716474 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovn-rundir\" (UniqueName: \"kubernetes.io/empty-dir/18e325b5-7837-4776-ae05-d9a12ccf9746-ovn-rundir\") pod \"ovn-northd-0\" (UID: \"18e325b5-7837-4776-ae05-d9a12ccf9746\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:22:57 crc kubenswrapper[5052]: I1203 12:22:57.716892 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/18e325b5-7837-4776-ae05-d9a12ccf9746-config\") pod \"ovn-northd-0\" (UID: \"18e325b5-7837-4776-ae05-d9a12ccf9746\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:22:57 crc kubenswrapper[5052]: I1203 12:22:57.716892 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/18e325b5-7837-4776-ae05-d9a12ccf9746-scripts\") pod \"ovn-northd-0\" (UID: \"18e325b5-7837-4776-ae05-d9a12ccf9746\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:22:57 crc kubenswrapper[5052]: I1203 12:22:57.723150 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/18e325b5-7837-4776-ae05-d9a12ccf9746-combined-ca-bundle\") pod \"ovn-northd-0\" (UID: \"18e325b5-7837-4776-ae05-d9a12ccf9746\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:22:57 crc kubenswrapper[5052]: I1203 12:22:57.736672 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-jpxt6\" (UniqueName: \"kubernetes.io/projected/18e325b5-7837-4776-ae05-d9a12ccf9746-kube-api-access-jpxt6\") pod \"ovn-northd-0\" (UID: \"18e325b5-7837-4776-ae05-d9a12ccf9746\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:22:57 crc kubenswrapper[5052]: I1203 12:22:57.879359 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:22:58 crc kubenswrapper[5052]: E1203 12:22:58.285451 5052 cadvisor_stats_provider.go:516] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4dd73a47_cf2a_43b8_ba1c_0d47332edaa4.slice/crio-76b4d90822b2cea4dc663e9fb413320e7954c906bd78785f2f60c7e955624e00\": RecentStats: unable to find data in memory cache]" Dec 03 12:22:58 crc kubenswrapper[5052]: I1203 12:22:58.313853 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ovn-northd-0"] Dec 03 12:22:58 crc kubenswrapper[5052]: W1203 12:22:58.319260 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod18e325b5_7837_4776_ae05_d9a12ccf9746.slice/crio-9af37b3aa4be7e7fd78cfa6920ebe9e54e9461b8ea134915213618fabec3639b WatchSource:0}: Error finding container 9af37b3aa4be7e7fd78cfa6920ebe9e54e9461b8ea134915213618fabec3639b: Status 404 returned error can't find the container with id 9af37b3aa4be7e7fd78cfa6920ebe9e54e9461b8ea134915213618fabec3639b Dec 03 12:22:58 crc kubenswrapper[5052]: I1203 12:22:58.342846 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovn-northd-0" event={"ID":"18e325b5-7837-4776-ae05-d9a12ccf9746","Type":"ContainerStarted","Data":"9af37b3aa4be7e7fd78cfa6920ebe9e54e9461b8ea134915213618fabec3639b"} Dec 03 12:22:58 crc kubenswrapper[5052]: I1203 12:22:58.347301 5052 generic.go:334] "Generic (PLEG): container finished" podID="fa334119-b1fd-42e8-b64b-a276ef89e4b5" containerID="bfd1131e5b34bdb4dfe470823bc362987fe44180ecf47f11c1f7cf9cd07620fa" exitCode=0 Dec 03 12:22:58 crc kubenswrapper[5052]: I1203 12:22:58.347728 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-marketplace-hx9cz" event={"ID":"fa334119-b1fd-42e8-b64b-a276ef89e4b5","Type":"ContainerDied","Data":"bfd1131e5b34bdb4dfe470823bc362987fe44180ecf47f11c1f7cf9cd07620fa"} Dec 03 12:22:58 crc kubenswrapper[5052]: I1203 12:22:58.347804 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-marketplace-hx9cz" event={"ID":"fa334119-b1fd-42e8-b64b-a276ef89e4b5","Type":"ContainerStarted","Data":"e65548704517b7f3191d1758db0ccb4f7fbc04d1cd605cc4bc87eb302a949b07"} Dec 03 12:22:58 crc kubenswrapper[5052]: I1203 12:22:58.516029 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:22:58 crc kubenswrapper[5052]: I1203 12:22:58.516071 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:22:59 crc kubenswrapper[5052]: I1203 12:22:59.371706 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/memcached-0" Dec 03 12:22:59 crc kubenswrapper[5052]: I1203 12:22:59.376083 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovn-northd-0" event={"ID":"18e325b5-7837-4776-ae05-d9a12ccf9746","Type":"ContainerStarted","Data":"cb277aa344aceb42cc1e9fe2afb23fa062591a751722aa2db98541cb69ea85ad"} Dec 03 12:22:59 crc kubenswrapper[5052]: I1203 12:22:59.376121 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovn-northd-0" event={"ID":"18e325b5-7837-4776-ae05-d9a12ccf9746","Type":"ContainerStarted","Data":"dda098dd0ab939491d6cb42aac03c4daa0a17e93eaa82fa8ed7c6dec3c2a4f1b"} Dec 03 12:22:59 crc kubenswrapper[5052]: I1203 12:22:59.376292 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:22:59 crc kubenswrapper[5052]: I1203 12:22:59.438187 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/ovn-northd-0" podStartSLOduration=2.438168273 podStartE2EDuration="2.438168273s" podCreationTimestamp="2025-12-03 12:22:57 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:22:59.435585829 +0000 UTC m=+3770.023827583" watchObservedRunningTime="2025-12-03 12:22:59.438168273 +0000 UTC m=+3770.026410037" Dec 03 12:22:59 crc kubenswrapper[5052]: I1203 12:22:59.904944 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:23:00 crc kubenswrapper[5052]: I1203 12:23:00.088725 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:23:00 crc kubenswrapper[5052]: I1203 12:23:00.088780 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:23:00 crc kubenswrapper[5052]: I1203 12:23:00.166946 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:23:00 crc kubenswrapper[5052]: I1203 12:23:00.395526 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-marketplace-hx9cz" event={"ID":"fa334119-b1fd-42e8-b64b-a276ef89e4b5","Type":"ContainerDied","Data":"ea57972cc90dc839e2691fd58331a5b91823aeac8b1d3c28fef0d50b86ac8c02"} Dec 03 12:23:00 crc kubenswrapper[5052]: I1203 12:23:00.395422 5052 generic.go:334] "Generic (PLEG): container finished" podID="fa334119-b1fd-42e8-b64b-a276ef89e4b5" containerID="ea57972cc90dc839e2691fd58331a5b91823aeac8b1d3c28fef0d50b86ac8c02" exitCode=0 Dec 03 12:23:00 crc kubenswrapper[5052]: I1203 12:23:00.483840 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:23:00 crc kubenswrapper[5052]: I1203 12:23:00.701222 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:23:00 crc kubenswrapper[5052]: I1203 12:23:00.901200 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/swift-storage-0"] Dec 03 12:23:00 crc kubenswrapper[5052]: I1203 12:23:00.907222 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:23:00 crc kubenswrapper[5052]: I1203 12:23:00.910049 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"swift-ring-files" Dec 03 12:23:00 crc kubenswrapper[5052]: I1203 12:23:00.910308 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"swift-storage-config-data" Dec 03 12:23:00 crc kubenswrapper[5052]: I1203 12:23:00.910527 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"swift-conf" Dec 03 12:23:00 crc kubenswrapper[5052]: I1203 12:23:00.910774 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"swift-swift-dockercfg-7j7gp" Dec 03 12:23:00 crc kubenswrapper[5052]: I1203 12:23:00.924627 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/swift-storage-0"] Dec 03 12:23:01 crc kubenswrapper[5052]: I1203 12:23:01.073622 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9vcl8\" (UniqueName: \"kubernetes.io/projected/7f163259-845f-4519-bd77-fa66e760d4db-kube-api-access-9vcl8\") pod \"swift-storage-0\" (UID: \"7f163259-845f-4519-bd77-fa66e760d4db\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:23:01 crc kubenswrapper[5052]: I1203 12:23:01.073686 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lock\" (UniqueName: \"kubernetes.io/empty-dir/7f163259-845f-4519-bd77-fa66e760d4db-lock\") pod \"swift-storage-0\" (UID: \"7f163259-845f-4519-bd77-fa66e760d4db\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:23:01 crc kubenswrapper[5052]: I1203 12:23:01.073717 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cache\" (UniqueName: \"kubernetes.io/empty-dir/7f163259-845f-4519-bd77-fa66e760d4db-cache\") pod \"swift-storage-0\" (UID: \"7f163259-845f-4519-bd77-fa66e760d4db\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:23:01 crc kubenswrapper[5052]: I1203 12:23:01.073922 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/7f163259-845f-4519-bd77-fa66e760d4db-etc-swift\") pod \"swift-storage-0\" (UID: \"7f163259-845f-4519-bd77-fa66e760d4db\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:23:01 crc kubenswrapper[5052]: I1203 12:23:01.074035 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage06-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage06-crc\") pod \"swift-storage-0\" (UID: \"7f163259-845f-4519-bd77-fa66e760d4db\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:23:01 crc kubenswrapper[5052]: I1203 12:23:01.175580 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/7f163259-845f-4519-bd77-fa66e760d4db-etc-swift\") pod \"swift-storage-0\" (UID: \"7f163259-845f-4519-bd77-fa66e760d4db\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:23:01 crc kubenswrapper[5052]: I1203 12:23:01.175663 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage06-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage06-crc\") pod \"swift-storage-0\" (UID: \"7f163259-845f-4519-bd77-fa66e760d4db\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:23:01 crc kubenswrapper[5052]: I1203 12:23:01.175699 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-9vcl8\" (UniqueName: \"kubernetes.io/projected/7f163259-845f-4519-bd77-fa66e760d4db-kube-api-access-9vcl8\") pod \"swift-storage-0\" (UID: \"7f163259-845f-4519-bd77-fa66e760d4db\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:23:01 crc kubenswrapper[5052]: I1203 12:23:01.175731 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"lock\" (UniqueName: \"kubernetes.io/empty-dir/7f163259-845f-4519-bd77-fa66e760d4db-lock\") pod \"swift-storage-0\" (UID: \"7f163259-845f-4519-bd77-fa66e760d4db\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:23:01 crc kubenswrapper[5052]: I1203 12:23:01.175751 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"cache\" (UniqueName: \"kubernetes.io/empty-dir/7f163259-845f-4519-bd77-fa66e760d4db-cache\") pod \"swift-storage-0\" (UID: \"7f163259-845f-4519-bd77-fa66e760d4db\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:23:01 crc kubenswrapper[5052]: I1203 12:23:01.175973 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage06-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage06-crc\") pod \"swift-storage-0\" (UID: \"7f163259-845f-4519-bd77-fa66e760d4db\") device mount path \"/mnt/openstack/pv06\"" pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:23:01 crc kubenswrapper[5052]: I1203 12:23:01.176171 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"cache\" (UniqueName: \"kubernetes.io/empty-dir/7f163259-845f-4519-bd77-fa66e760d4db-cache\") pod \"swift-storage-0\" (UID: \"7f163259-845f-4519-bd77-fa66e760d4db\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:23:01 crc kubenswrapper[5052]: I1203 12:23:01.176253 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"lock\" (UniqueName: \"kubernetes.io/empty-dir/7f163259-845f-4519-bd77-fa66e760d4db-lock\") pod \"swift-storage-0\" (UID: \"7f163259-845f-4519-bd77-fa66e760d4db\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:23:01 crc kubenswrapper[5052]: E1203 12:23:01.186924 5052 projected.go:288] Couldn't get configMap openstack-kuttl-tests/swift-ring-files: configmap "swift-ring-files" not found Dec 03 12:23:01 crc kubenswrapper[5052]: E1203 12:23:01.186958 5052 projected.go:194] Error preparing data for projected volume etc-swift for pod openstack-kuttl-tests/swift-storage-0: configmap "swift-ring-files" not found Dec 03 12:23:01 crc kubenswrapper[5052]: E1203 12:23:01.187037 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7f163259-845f-4519-bd77-fa66e760d4db-etc-swift podName:7f163259-845f-4519-bd77-fa66e760d4db nodeName:}" failed. No retries permitted until 2025-12-03 12:23:01.687017671 +0000 UTC m=+3772.275259435 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "etc-swift" (UniqueName: "kubernetes.io/projected/7f163259-845f-4519-bd77-fa66e760d4db-etc-swift") pod "swift-storage-0" (UID: "7f163259-845f-4519-bd77-fa66e760d4db") : configmap "swift-ring-files" not found Dec 03 12:23:01 crc kubenswrapper[5052]: I1203 12:23:01.211278 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage06-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage06-crc\") pod \"swift-storage-0\" (UID: \"7f163259-845f-4519-bd77-fa66e760d4db\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:23:01 crc kubenswrapper[5052]: I1203 12:23:01.212763 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-9vcl8\" (UniqueName: \"kubernetes.io/projected/7f163259-845f-4519-bd77-fa66e760d4db-kube-api-access-9vcl8\") pod \"swift-storage-0\" (UID: \"7f163259-845f-4519-bd77-fa66e760d4db\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:23:01 crc kubenswrapper[5052]: I1203 12:23:01.274732 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:23:01 crc kubenswrapper[5052]: I1203 12:23:01.407755 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-marketplace-hx9cz" event={"ID":"fa334119-b1fd-42e8-b64b-a276ef89e4b5","Type":"ContainerStarted","Data":"57d4da62ee25a46d86d917668db0e2040b0ddb72f3931305a731ada69c4b95f9"} Dec 03 12:23:01 crc kubenswrapper[5052]: I1203 12:23:01.427402 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-marketplace/redhat-marketplace-hx9cz" podStartSLOduration=2.972692607 podStartE2EDuration="5.427384161s" podCreationTimestamp="2025-12-03 12:22:56 +0000 UTC" firstStartedPulling="2025-12-03 12:22:58.349467409 +0000 UTC m=+3768.937709173" lastFinishedPulling="2025-12-03 12:23:00.804158973 +0000 UTC m=+3771.392400727" observedRunningTime="2025-12-03 12:23:01.423311515 +0000 UTC m=+3772.011553299" watchObservedRunningTime="2025-12-03 12:23:01.427384161 +0000 UTC m=+3772.015625925" Dec 03 12:23:01 crc kubenswrapper[5052]: I1203 12:23:01.672935 5052 scope.go:117] "RemoveContainer" containerID="09b4a0072741f60499d24f40cc4ff3406d5d22671ced431363ab865172786300" Dec 03 12:23:01 crc kubenswrapper[5052]: E1203 12:23:01.673262 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 12:23:01 crc kubenswrapper[5052]: I1203 12:23:01.786926 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/7f163259-845f-4519-bd77-fa66e760d4db-etc-swift\") pod \"swift-storage-0\" (UID: \"7f163259-845f-4519-bd77-fa66e760d4db\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:23:01 crc kubenswrapper[5052]: E1203 12:23:01.787117 5052 projected.go:288] Couldn't get configMap openstack-kuttl-tests/swift-ring-files: configmap "swift-ring-files" not found Dec 03 12:23:01 crc kubenswrapper[5052]: E1203 12:23:01.787151 5052 projected.go:194] Error preparing data for projected volume etc-swift for pod openstack-kuttl-tests/swift-storage-0: configmap "swift-ring-files" not found Dec 03 12:23:01 crc kubenswrapper[5052]: E1203 12:23:01.787221 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7f163259-845f-4519-bd77-fa66e760d4db-etc-swift podName:7f163259-845f-4519-bd77-fa66e760d4db nodeName:}" failed. No retries permitted until 2025-12-03 12:23:02.787199452 +0000 UTC m=+3773.375441276 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "etc-swift" (UniqueName: "kubernetes.io/projected/7f163259-845f-4519-bd77-fa66e760d4db-etc-swift") pod "swift-storage-0" (UID: "7f163259-845f-4519-bd77-fa66e760d4db") : configmap "swift-ring-files" not found Dec 03 12:23:02 crc kubenswrapper[5052]: I1203 12:23:02.803063 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/7f163259-845f-4519-bd77-fa66e760d4db-etc-swift\") pod \"swift-storage-0\" (UID: \"7f163259-845f-4519-bd77-fa66e760d4db\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:23:02 crc kubenswrapper[5052]: E1203 12:23:02.803233 5052 projected.go:288] Couldn't get configMap openstack-kuttl-tests/swift-ring-files: configmap "swift-ring-files" not found Dec 03 12:23:02 crc kubenswrapper[5052]: E1203 12:23:02.803246 5052 projected.go:194] Error preparing data for projected volume etc-swift for pod openstack-kuttl-tests/swift-storage-0: configmap "swift-ring-files" not found Dec 03 12:23:02 crc kubenswrapper[5052]: E1203 12:23:02.803312 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7f163259-845f-4519-bd77-fa66e760d4db-etc-swift podName:7f163259-845f-4519-bd77-fa66e760d4db nodeName:}" failed. No retries permitted until 2025-12-03 12:23:04.803292422 +0000 UTC m=+3775.391534196 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "etc-swift" (UniqueName: "kubernetes.io/projected/7f163259-845f-4519-bd77-fa66e760d4db-etc-swift") pod "swift-storage-0" (UID: "7f163259-845f-4519-bd77-fa66e760d4db") : configmap "swift-ring-files" not found Dec 03 12:23:04 crc kubenswrapper[5052]: I1203 12:23:04.780109 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/swift-ring-rebalance-b9ssr"] Dec 03 12:23:04 crc kubenswrapper[5052]: I1203 12:23:04.781701 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-ring-rebalance-b9ssr" Dec 03 12:23:04 crc kubenswrapper[5052]: I1203 12:23:04.783652 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"swift-proxy-config-data" Dec 03 12:23:04 crc kubenswrapper[5052]: I1203 12:23:04.783903 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"swift-ring-config-data" Dec 03 12:23:04 crc kubenswrapper[5052]: I1203 12:23:04.784263 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"swift-ring-scripts" Dec 03 12:23:04 crc kubenswrapper[5052]: I1203 12:23:04.804730 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/swift-ring-rebalance-b9ssr"] Dec 03 12:23:04 crc kubenswrapper[5052]: I1203 12:23:04.838625 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/7f163259-845f-4519-bd77-fa66e760d4db-etc-swift\") pod \"swift-storage-0\" (UID: \"7f163259-845f-4519-bd77-fa66e760d4db\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:23:04 crc kubenswrapper[5052]: E1203 12:23:04.838764 5052 projected.go:288] Couldn't get configMap openstack-kuttl-tests/swift-ring-files: configmap "swift-ring-files" not found Dec 03 12:23:04 crc kubenswrapper[5052]: E1203 12:23:04.838800 5052 projected.go:194] Error preparing data for projected volume etc-swift for pod openstack-kuttl-tests/swift-storage-0: configmap "swift-ring-files" not found Dec 03 12:23:04 crc kubenswrapper[5052]: E1203 12:23:04.838867 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7f163259-845f-4519-bd77-fa66e760d4db-etc-swift podName:7f163259-845f-4519-bd77-fa66e760d4db nodeName:}" failed. No retries permitted until 2025-12-03 12:23:08.838842792 +0000 UTC m=+3779.427084746 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "etc-swift" (UniqueName: "kubernetes.io/projected/7f163259-845f-4519-bd77-fa66e760d4db-etc-swift") pod "swift-storage-0" (UID: "7f163259-845f-4519-bd77-fa66e760d4db") : configmap "swift-ring-files" not found Dec 03 12:23:04 crc kubenswrapper[5052]: I1203 12:23:04.940652 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"swiftconf\" (UniqueName: \"kubernetes.io/secret/46e92959-0fd2-4e2b-8369-35cf14716af3-swiftconf\") pod \"swift-ring-rebalance-b9ssr\" (UID: \"46e92959-0fd2-4e2b-8369-35cf14716af3\") " pod="openstack-kuttl-tests/swift-ring-rebalance-b9ssr" Dec 03 12:23:04 crc kubenswrapper[5052]: I1203 12:23:04.940760 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/46e92959-0fd2-4e2b-8369-35cf14716af3-scripts\") pod \"swift-ring-rebalance-b9ssr\" (UID: \"46e92959-0fd2-4e2b-8369-35cf14716af3\") " pod="openstack-kuttl-tests/swift-ring-rebalance-b9ssr" Dec 03 12:23:04 crc kubenswrapper[5052]: I1203 12:23:04.940850 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dispersionconf\" (UniqueName: \"kubernetes.io/secret/46e92959-0fd2-4e2b-8369-35cf14716af3-dispersionconf\") pod \"swift-ring-rebalance-b9ssr\" (UID: \"46e92959-0fd2-4e2b-8369-35cf14716af3\") " pod="openstack-kuttl-tests/swift-ring-rebalance-b9ssr" Dec 03 12:23:04 crc kubenswrapper[5052]: I1203 12:23:04.940945 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/46e92959-0fd2-4e2b-8369-35cf14716af3-combined-ca-bundle\") pod \"swift-ring-rebalance-b9ssr\" (UID: \"46e92959-0fd2-4e2b-8369-35cf14716af3\") " pod="openstack-kuttl-tests/swift-ring-rebalance-b9ssr" Dec 03 12:23:04 crc kubenswrapper[5052]: I1203 12:23:04.940998 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/empty-dir/46e92959-0fd2-4e2b-8369-35cf14716af3-etc-swift\") pod \"swift-ring-rebalance-b9ssr\" (UID: \"46e92959-0fd2-4e2b-8369-35cf14716af3\") " pod="openstack-kuttl-tests/swift-ring-rebalance-b9ssr" Dec 03 12:23:04 crc kubenswrapper[5052]: I1203 12:23:04.941037 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j595l\" (UniqueName: \"kubernetes.io/projected/46e92959-0fd2-4e2b-8369-35cf14716af3-kube-api-access-j595l\") pod \"swift-ring-rebalance-b9ssr\" (UID: \"46e92959-0fd2-4e2b-8369-35cf14716af3\") " pod="openstack-kuttl-tests/swift-ring-rebalance-b9ssr" Dec 03 12:23:04 crc kubenswrapper[5052]: I1203 12:23:04.941066 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ring-data-devices\" (UniqueName: \"kubernetes.io/configmap/46e92959-0fd2-4e2b-8369-35cf14716af3-ring-data-devices\") pod \"swift-ring-rebalance-b9ssr\" (UID: \"46e92959-0fd2-4e2b-8369-35cf14716af3\") " pod="openstack-kuttl-tests/swift-ring-rebalance-b9ssr" Dec 03 12:23:05 crc kubenswrapper[5052]: I1203 12:23:05.042810 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/46e92959-0fd2-4e2b-8369-35cf14716af3-combined-ca-bundle\") pod \"swift-ring-rebalance-b9ssr\" (UID: \"46e92959-0fd2-4e2b-8369-35cf14716af3\") " pod="openstack-kuttl-tests/swift-ring-rebalance-b9ssr" Dec 03 12:23:05 crc kubenswrapper[5052]: I1203 12:23:05.042872 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/empty-dir/46e92959-0fd2-4e2b-8369-35cf14716af3-etc-swift\") pod \"swift-ring-rebalance-b9ssr\" (UID: \"46e92959-0fd2-4e2b-8369-35cf14716af3\") " pod="openstack-kuttl-tests/swift-ring-rebalance-b9ssr" Dec 03 12:23:05 crc kubenswrapper[5052]: I1203 12:23:05.042908 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-j595l\" (UniqueName: \"kubernetes.io/projected/46e92959-0fd2-4e2b-8369-35cf14716af3-kube-api-access-j595l\") pod \"swift-ring-rebalance-b9ssr\" (UID: \"46e92959-0fd2-4e2b-8369-35cf14716af3\") " pod="openstack-kuttl-tests/swift-ring-rebalance-b9ssr" Dec 03 12:23:05 crc kubenswrapper[5052]: I1203 12:23:05.042927 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ring-data-devices\" (UniqueName: \"kubernetes.io/configmap/46e92959-0fd2-4e2b-8369-35cf14716af3-ring-data-devices\") pod \"swift-ring-rebalance-b9ssr\" (UID: \"46e92959-0fd2-4e2b-8369-35cf14716af3\") " pod="openstack-kuttl-tests/swift-ring-rebalance-b9ssr" Dec 03 12:23:05 crc kubenswrapper[5052]: I1203 12:23:05.042956 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"swiftconf\" (UniqueName: \"kubernetes.io/secret/46e92959-0fd2-4e2b-8369-35cf14716af3-swiftconf\") pod \"swift-ring-rebalance-b9ssr\" (UID: \"46e92959-0fd2-4e2b-8369-35cf14716af3\") " pod="openstack-kuttl-tests/swift-ring-rebalance-b9ssr" Dec 03 12:23:05 crc kubenswrapper[5052]: I1203 12:23:05.042974 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/46e92959-0fd2-4e2b-8369-35cf14716af3-scripts\") pod \"swift-ring-rebalance-b9ssr\" (UID: \"46e92959-0fd2-4e2b-8369-35cf14716af3\") " pod="openstack-kuttl-tests/swift-ring-rebalance-b9ssr" Dec 03 12:23:05 crc kubenswrapper[5052]: I1203 12:23:05.043015 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"dispersionconf\" (UniqueName: \"kubernetes.io/secret/46e92959-0fd2-4e2b-8369-35cf14716af3-dispersionconf\") pod \"swift-ring-rebalance-b9ssr\" (UID: \"46e92959-0fd2-4e2b-8369-35cf14716af3\") " pod="openstack-kuttl-tests/swift-ring-rebalance-b9ssr" Dec 03 12:23:05 crc kubenswrapper[5052]: I1203 12:23:05.043665 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"etc-swift\" (UniqueName: \"kubernetes.io/empty-dir/46e92959-0fd2-4e2b-8369-35cf14716af3-etc-swift\") pod \"swift-ring-rebalance-b9ssr\" (UID: \"46e92959-0fd2-4e2b-8369-35cf14716af3\") " pod="openstack-kuttl-tests/swift-ring-rebalance-b9ssr" Dec 03 12:23:05 crc kubenswrapper[5052]: I1203 12:23:05.043983 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/46e92959-0fd2-4e2b-8369-35cf14716af3-scripts\") pod \"swift-ring-rebalance-b9ssr\" (UID: \"46e92959-0fd2-4e2b-8369-35cf14716af3\") " pod="openstack-kuttl-tests/swift-ring-rebalance-b9ssr" Dec 03 12:23:05 crc kubenswrapper[5052]: I1203 12:23:05.044437 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ring-data-devices\" (UniqueName: \"kubernetes.io/configmap/46e92959-0fd2-4e2b-8369-35cf14716af3-ring-data-devices\") pod \"swift-ring-rebalance-b9ssr\" (UID: \"46e92959-0fd2-4e2b-8369-35cf14716af3\") " pod="openstack-kuttl-tests/swift-ring-rebalance-b9ssr" Dec 03 12:23:05 crc kubenswrapper[5052]: I1203 12:23:05.049542 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/46e92959-0fd2-4e2b-8369-35cf14716af3-combined-ca-bundle\") pod \"swift-ring-rebalance-b9ssr\" (UID: \"46e92959-0fd2-4e2b-8369-35cf14716af3\") " pod="openstack-kuttl-tests/swift-ring-rebalance-b9ssr" Dec 03 12:23:05 crc kubenswrapper[5052]: I1203 12:23:05.054076 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"dispersionconf\" (UniqueName: \"kubernetes.io/secret/46e92959-0fd2-4e2b-8369-35cf14716af3-dispersionconf\") pod \"swift-ring-rebalance-b9ssr\" (UID: \"46e92959-0fd2-4e2b-8369-35cf14716af3\") " pod="openstack-kuttl-tests/swift-ring-rebalance-b9ssr" Dec 03 12:23:05 crc kubenswrapper[5052]: I1203 12:23:05.058010 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"swiftconf\" (UniqueName: \"kubernetes.io/secret/46e92959-0fd2-4e2b-8369-35cf14716af3-swiftconf\") pod \"swift-ring-rebalance-b9ssr\" (UID: \"46e92959-0fd2-4e2b-8369-35cf14716af3\") " pod="openstack-kuttl-tests/swift-ring-rebalance-b9ssr" Dec 03 12:23:05 crc kubenswrapper[5052]: I1203 12:23:05.069888 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-j595l\" (UniqueName: \"kubernetes.io/projected/46e92959-0fd2-4e2b-8369-35cf14716af3-kube-api-access-j595l\") pod \"swift-ring-rebalance-b9ssr\" (UID: \"46e92959-0fd2-4e2b-8369-35cf14716af3\") " pod="openstack-kuttl-tests/swift-ring-rebalance-b9ssr" Dec 03 12:23:05 crc kubenswrapper[5052]: I1203 12:23:05.115246 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-ring-rebalance-b9ssr" Dec 03 12:23:05 crc kubenswrapper[5052]: I1203 12:23:05.549094 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/swift-ring-rebalance-b9ssr"] Dec 03 12:23:05 crc kubenswrapper[5052]: W1203 12:23:05.553370 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod46e92959_0fd2_4e2b_8369_35cf14716af3.slice/crio-b1e5ab942f0946bc4bd0a7347a43802032dcabcb1b83787d4c5960008f7e5624 WatchSource:0}: Error finding container b1e5ab942f0946bc4bd0a7347a43802032dcabcb1b83787d4c5960008f7e5624: Status 404 returned error can't find the container with id b1e5ab942f0946bc4bd0a7347a43802032dcabcb1b83787d4c5960008f7e5624 Dec 03 12:23:05 crc kubenswrapper[5052]: I1203 12:23:05.602039 5052 scope.go:117] "RemoveContainer" containerID="4cf218ecb3b4f85168c67eef87ddb54d85c03143ceee481f6d32d5b217a05b13" Dec 03 12:23:05 crc kubenswrapper[5052]: I1203 12:23:05.628577 5052 scope.go:117] "RemoveContainer" containerID="4dd5e22ebf83ff299dc4da047909204239d8d079333b89c5c5d39c2221025070" Dec 03 12:23:05 crc kubenswrapper[5052]: I1203 12:23:05.658548 5052 scope.go:117] "RemoveContainer" containerID="7c40f1aa68ee0e0f29ce1ab35070e6d55c530a232bd4bf5a962f9aa5aa656271" Dec 03 12:23:05 crc kubenswrapper[5052]: I1203 12:23:05.687374 5052 scope.go:117] "RemoveContainer" containerID="acabd11f67c578a289ff1b5572b82ec5c14a08efbfe4929ff13289c867543646" Dec 03 12:23:05 crc kubenswrapper[5052]: I1203 12:23:05.711430 5052 scope.go:117] "RemoveContainer" containerID="8dfc8c0fb51ed3361082e5d6a21d3480abe509b00c224f35e46b8ddb4b3e7415" Dec 03 12:23:05 crc kubenswrapper[5052]: I1203 12:23:05.737175 5052 scope.go:117] "RemoveContainer" containerID="17351fd3e1257f4ce4d46a149fe313d7f60e83dec93abd203cf3ef841ce6d2e5" Dec 03 12:23:05 crc kubenswrapper[5052]: I1203 12:23:05.778951 5052 scope.go:117] "RemoveContainer" containerID="f6d888d44496d050d0248b653846925f4cb1991f287b175e7c992815d8fc3e03" Dec 03 12:23:05 crc kubenswrapper[5052]: I1203 12:23:05.807085 5052 scope.go:117] "RemoveContainer" containerID="9d59b98890d62735c1c3c35a6531d9c3479675d06cd1783c7c1c65a8112e60ca" Dec 03 12:23:05 crc kubenswrapper[5052]: I1203 12:23:05.862025 5052 scope.go:117] "RemoveContainer" containerID="096ac7f05d80e97fa0a9f7155bad8e1122f8bb53f0932b4d4a2ff75baa41a0e8" Dec 03 12:23:05 crc kubenswrapper[5052]: I1203 12:23:05.880397 5052 scope.go:117] "RemoveContainer" containerID="231d5f3ea609c0fd0b8f6c9745cde813952d26ceaed8f036d11f3d3a0034c03a" Dec 03 12:23:05 crc kubenswrapper[5052]: I1203 12:23:05.902471 5052 scope.go:117] "RemoveContainer" containerID="ba753a10be8fb7f00ba30d6776de7a5083ef01995b6ab3a6a792f9958569ab15" Dec 03 12:23:06 crc kubenswrapper[5052]: I1203 12:23:06.298466 5052 scope.go:117] "RemoveContainer" containerID="64bb7856e52629460376e6405674735686ad3f33769a348e6d4f9ecbb1bdc284" Dec 03 12:23:06 crc kubenswrapper[5052]: I1203 12:23:06.336429 5052 scope.go:117] "RemoveContainer" containerID="44461d2df3c8193c47365f75ad3473316c51e0f906c01100d8592472d3b02232" Dec 03 12:23:06 crc kubenswrapper[5052]: I1203 12:23:06.371062 5052 scope.go:117] "RemoveContainer" containerID="c96eef4a8dc590f60d0cb12973141986e22e627a28dcd22a83015316fc20efc1" Dec 03 12:23:06 crc kubenswrapper[5052]: I1203 12:23:06.408623 5052 scope.go:117] "RemoveContainer" containerID="a0b610780c737434b4d97d2fd99c01273a88681fdc19b3a41fdf3d032c04b664" Dec 03 12:23:06 crc kubenswrapper[5052]: I1203 12:23:06.463094 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-ring-rebalance-b9ssr" event={"ID":"46e92959-0fd2-4e2b-8369-35cf14716af3","Type":"ContainerStarted","Data":"3cc4322f202b3e31f0e15cf554b9c3df35c358e0a3edc6031c1321e38a0851d3"} Dec 03 12:23:06 crc kubenswrapper[5052]: I1203 12:23:06.463350 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-ring-rebalance-b9ssr" event={"ID":"46e92959-0fd2-4e2b-8369-35cf14716af3","Type":"ContainerStarted","Data":"b1e5ab942f0946bc4bd0a7347a43802032dcabcb1b83787d4c5960008f7e5624"} Dec 03 12:23:06 crc kubenswrapper[5052]: I1203 12:23:06.964328 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openshift-marketplace/redhat-marketplace-hx9cz" Dec 03 12:23:06 crc kubenswrapper[5052]: I1203 12:23:06.964468 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-marketplace/redhat-marketplace-hx9cz" Dec 03 12:23:07 crc kubenswrapper[5052]: I1203 12:23:07.023002 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-marketplace/redhat-marketplace-hx9cz" Dec 03 12:23:07 crc kubenswrapper[5052]: I1203 12:23:07.046907 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/swift-ring-rebalance-b9ssr" podStartSLOduration=3.046883944 podStartE2EDuration="3.046883944s" podCreationTimestamp="2025-12-03 12:23:04 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:23:06.487429896 +0000 UTC m=+3777.075671660" watchObservedRunningTime="2025-12-03 12:23:07.046883944 +0000 UTC m=+3777.635125708" Dec 03 12:23:07 crc kubenswrapper[5052]: I1203 12:23:07.522286 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-marketplace/redhat-marketplace-hx9cz" Dec 03 12:23:07 crc kubenswrapper[5052]: I1203 12:23:07.569249 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openshift-marketplace/redhat-marketplace-hx9cz"] Dec 03 12:23:08 crc kubenswrapper[5052]: E1203 12:23:08.483718 5052 cadvisor_stats_provider.go:516] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4dd73a47_cf2a_43b8_ba1c_0d47332edaa4.slice/crio-76b4d90822b2cea4dc663e9fb413320e7954c906bd78785f2f60c7e955624e00\": RecentStats: unable to find data in memory cache]" Dec 03 12:23:08 crc kubenswrapper[5052]: I1203 12:23:08.551627 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/keystone-db-create-zrpdw"] Dec 03 12:23:08 crc kubenswrapper[5052]: I1203 12:23:08.552965 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-db-create-zrpdw" Dec 03 12:23:08 crc kubenswrapper[5052]: I1203 12:23:08.560195 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-db-create-zrpdw"] Dec 03 12:23:08 crc kubenswrapper[5052]: I1203 12:23:08.656089 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/keystone-39e2-account-create-update-74dh8"] Dec 03 12:23:08 crc kubenswrapper[5052]: I1203 12:23:08.657215 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-39e2-account-create-update-74dh8" Dec 03 12:23:08 crc kubenswrapper[5052]: I1203 12:23:08.659385 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone-db-secret" Dec 03 12:23:08 crc kubenswrapper[5052]: I1203 12:23:08.663583 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-39e2-account-create-update-74dh8"] Dec 03 12:23:08 crc kubenswrapper[5052]: I1203 12:23:08.712981 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/beb9c625-5acb-4d1e-9044-7fb5287e8cfe-operator-scripts\") pod \"keystone-db-create-zrpdw\" (UID: \"beb9c625-5acb-4d1e-9044-7fb5287e8cfe\") " pod="openstack-kuttl-tests/keystone-db-create-zrpdw" Dec 03 12:23:08 crc kubenswrapper[5052]: I1203 12:23:08.713026 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2vb7c\" (UniqueName: \"kubernetes.io/projected/beb9c625-5acb-4d1e-9044-7fb5287e8cfe-kube-api-access-2vb7c\") pod \"keystone-db-create-zrpdw\" (UID: \"beb9c625-5acb-4d1e-9044-7fb5287e8cfe\") " pod="openstack-kuttl-tests/keystone-db-create-zrpdw" Dec 03 12:23:08 crc kubenswrapper[5052]: I1203 12:23:08.814903 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xg5v8\" (UniqueName: \"kubernetes.io/projected/37da25a2-1fa7-4a14-b8d6-1ba47412c6cd-kube-api-access-xg5v8\") pod \"keystone-39e2-account-create-update-74dh8\" (UID: \"37da25a2-1fa7-4a14-b8d6-1ba47412c6cd\") " pod="openstack-kuttl-tests/keystone-39e2-account-create-update-74dh8" Dec 03 12:23:08 crc kubenswrapper[5052]: I1203 12:23:08.815101 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/37da25a2-1fa7-4a14-b8d6-1ba47412c6cd-operator-scripts\") pod \"keystone-39e2-account-create-update-74dh8\" (UID: \"37da25a2-1fa7-4a14-b8d6-1ba47412c6cd\") " pod="openstack-kuttl-tests/keystone-39e2-account-create-update-74dh8" Dec 03 12:23:08 crc kubenswrapper[5052]: I1203 12:23:08.815170 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/beb9c625-5acb-4d1e-9044-7fb5287e8cfe-operator-scripts\") pod \"keystone-db-create-zrpdw\" (UID: \"beb9c625-5acb-4d1e-9044-7fb5287e8cfe\") " pod="openstack-kuttl-tests/keystone-db-create-zrpdw" Dec 03 12:23:08 crc kubenswrapper[5052]: I1203 12:23:08.815195 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-2vb7c\" (UniqueName: \"kubernetes.io/projected/beb9c625-5acb-4d1e-9044-7fb5287e8cfe-kube-api-access-2vb7c\") pod \"keystone-db-create-zrpdw\" (UID: \"beb9c625-5acb-4d1e-9044-7fb5287e8cfe\") " pod="openstack-kuttl-tests/keystone-db-create-zrpdw" Dec 03 12:23:08 crc kubenswrapper[5052]: I1203 12:23:08.816800 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/beb9c625-5acb-4d1e-9044-7fb5287e8cfe-operator-scripts\") pod \"keystone-db-create-zrpdw\" (UID: \"beb9c625-5acb-4d1e-9044-7fb5287e8cfe\") " pod="openstack-kuttl-tests/keystone-db-create-zrpdw" Dec 03 12:23:08 crc kubenswrapper[5052]: I1203 12:23:08.848848 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-2vb7c\" (UniqueName: \"kubernetes.io/projected/beb9c625-5acb-4d1e-9044-7fb5287e8cfe-kube-api-access-2vb7c\") pod \"keystone-db-create-zrpdw\" (UID: \"beb9c625-5acb-4d1e-9044-7fb5287e8cfe\") " pod="openstack-kuttl-tests/keystone-db-create-zrpdw" Dec 03 12:23:08 crc kubenswrapper[5052]: I1203 12:23:08.881781 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-db-create-zrpdw" Dec 03 12:23:08 crc kubenswrapper[5052]: I1203 12:23:08.916639 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-xg5v8\" (UniqueName: \"kubernetes.io/projected/37da25a2-1fa7-4a14-b8d6-1ba47412c6cd-kube-api-access-xg5v8\") pod \"keystone-39e2-account-create-update-74dh8\" (UID: \"37da25a2-1fa7-4a14-b8d6-1ba47412c6cd\") " pod="openstack-kuttl-tests/keystone-39e2-account-create-update-74dh8" Dec 03 12:23:08 crc kubenswrapper[5052]: I1203 12:23:08.916713 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/37da25a2-1fa7-4a14-b8d6-1ba47412c6cd-operator-scripts\") pod \"keystone-39e2-account-create-update-74dh8\" (UID: \"37da25a2-1fa7-4a14-b8d6-1ba47412c6cd\") " pod="openstack-kuttl-tests/keystone-39e2-account-create-update-74dh8" Dec 03 12:23:08 crc kubenswrapper[5052]: I1203 12:23:08.916821 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/7f163259-845f-4519-bd77-fa66e760d4db-etc-swift\") pod \"swift-storage-0\" (UID: \"7f163259-845f-4519-bd77-fa66e760d4db\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:23:08 crc kubenswrapper[5052]: E1203 12:23:08.916957 5052 projected.go:288] Couldn't get configMap openstack-kuttl-tests/swift-ring-files: configmap "swift-ring-files" not found Dec 03 12:23:08 crc kubenswrapper[5052]: E1203 12:23:08.916976 5052 projected.go:194] Error preparing data for projected volume etc-swift for pod openstack-kuttl-tests/swift-storage-0: configmap "swift-ring-files" not found Dec 03 12:23:08 crc kubenswrapper[5052]: E1203 12:23:08.917017 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7f163259-845f-4519-bd77-fa66e760d4db-etc-swift podName:7f163259-845f-4519-bd77-fa66e760d4db nodeName:}" failed. No retries permitted until 2025-12-03 12:23:16.917003412 +0000 UTC m=+3787.505245176 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "etc-swift" (UniqueName: "kubernetes.io/projected/7f163259-845f-4519-bd77-fa66e760d4db-etc-swift") pod "swift-storage-0" (UID: "7f163259-845f-4519-bd77-fa66e760d4db") : configmap "swift-ring-files" not found Dec 03 12:23:08 crc kubenswrapper[5052]: I1203 12:23:08.920213 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/37da25a2-1fa7-4a14-b8d6-1ba47412c6cd-operator-scripts\") pod \"keystone-39e2-account-create-update-74dh8\" (UID: \"37da25a2-1fa7-4a14-b8d6-1ba47412c6cd\") " pod="openstack-kuttl-tests/keystone-39e2-account-create-update-74dh8" Dec 03 12:23:08 crc kubenswrapper[5052]: I1203 12:23:08.936711 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-xg5v8\" (UniqueName: \"kubernetes.io/projected/37da25a2-1fa7-4a14-b8d6-1ba47412c6cd-kube-api-access-xg5v8\") pod \"keystone-39e2-account-create-update-74dh8\" (UID: \"37da25a2-1fa7-4a14-b8d6-1ba47412c6cd\") " pod="openstack-kuttl-tests/keystone-39e2-account-create-update-74dh8" Dec 03 12:23:08 crc kubenswrapper[5052]: I1203 12:23:08.977113 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-39e2-account-create-update-74dh8" Dec 03 12:23:09 crc kubenswrapper[5052]: I1203 12:23:09.060340 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/placement-db-create-s2jk2"] Dec 03 12:23:09 crc kubenswrapper[5052]: I1203 12:23:09.061749 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-db-create-s2jk2" Dec 03 12:23:09 crc kubenswrapper[5052]: I1203 12:23:09.068377 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/placement-db-create-s2jk2"] Dec 03 12:23:09 crc kubenswrapper[5052]: I1203 12:23:09.185488 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/placement-e65a-account-create-update-8vzns"] Dec 03 12:23:09 crc kubenswrapper[5052]: I1203 12:23:09.187008 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-e65a-account-create-update-8vzns" Dec 03 12:23:09 crc kubenswrapper[5052]: I1203 12:23:09.196911 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"placement-db-secret" Dec 03 12:23:09 crc kubenswrapper[5052]: I1203 12:23:09.202549 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/placement-e65a-account-create-update-8vzns"] Dec 03 12:23:09 crc kubenswrapper[5052]: I1203 12:23:09.226342 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-466n5\" (UniqueName: \"kubernetes.io/projected/4ea65f95-6ea7-4362-a361-514ace4fbc44-kube-api-access-466n5\") pod \"placement-db-create-s2jk2\" (UID: \"4ea65f95-6ea7-4362-a361-514ace4fbc44\") " pod="openstack-kuttl-tests/placement-db-create-s2jk2" Dec 03 12:23:09 crc kubenswrapper[5052]: I1203 12:23:09.226781 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/4ea65f95-6ea7-4362-a361-514ace4fbc44-operator-scripts\") pod \"placement-db-create-s2jk2\" (UID: \"4ea65f95-6ea7-4362-a361-514ace4fbc44\") " pod="openstack-kuttl-tests/placement-db-create-s2jk2" Dec 03 12:23:09 crc kubenswrapper[5052]: I1203 12:23:09.253213 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-db-create-zrpdw"] Dec 03 12:23:09 crc kubenswrapper[5052]: I1203 12:23:09.327948 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/4ea65f95-6ea7-4362-a361-514ace4fbc44-operator-scripts\") pod \"placement-db-create-s2jk2\" (UID: \"4ea65f95-6ea7-4362-a361-514ace4fbc44\") " pod="openstack-kuttl-tests/placement-db-create-s2jk2" Dec 03 12:23:09 crc kubenswrapper[5052]: I1203 12:23:09.328051 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-466n5\" (UniqueName: \"kubernetes.io/projected/4ea65f95-6ea7-4362-a361-514ace4fbc44-kube-api-access-466n5\") pod \"placement-db-create-s2jk2\" (UID: \"4ea65f95-6ea7-4362-a361-514ace4fbc44\") " pod="openstack-kuttl-tests/placement-db-create-s2jk2" Dec 03 12:23:09 crc kubenswrapper[5052]: I1203 12:23:09.328083 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/bd34573a-a034-4c22-8df3-bab80ed70f45-operator-scripts\") pod \"placement-e65a-account-create-update-8vzns\" (UID: \"bd34573a-a034-4c22-8df3-bab80ed70f45\") " pod="openstack-kuttl-tests/placement-e65a-account-create-update-8vzns" Dec 03 12:23:09 crc kubenswrapper[5052]: I1203 12:23:09.328110 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mx2pm\" (UniqueName: \"kubernetes.io/projected/bd34573a-a034-4c22-8df3-bab80ed70f45-kube-api-access-mx2pm\") pod \"placement-e65a-account-create-update-8vzns\" (UID: \"bd34573a-a034-4c22-8df3-bab80ed70f45\") " pod="openstack-kuttl-tests/placement-e65a-account-create-update-8vzns" Dec 03 12:23:09 crc kubenswrapper[5052]: I1203 12:23:09.328769 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/4ea65f95-6ea7-4362-a361-514ace4fbc44-operator-scripts\") pod \"placement-db-create-s2jk2\" (UID: \"4ea65f95-6ea7-4362-a361-514ace4fbc44\") " pod="openstack-kuttl-tests/placement-db-create-s2jk2" Dec 03 12:23:09 crc kubenswrapper[5052]: I1203 12:23:09.346914 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-466n5\" (UniqueName: \"kubernetes.io/projected/4ea65f95-6ea7-4362-a361-514ace4fbc44-kube-api-access-466n5\") pod \"placement-db-create-s2jk2\" (UID: \"4ea65f95-6ea7-4362-a361-514ace4fbc44\") " pod="openstack-kuttl-tests/placement-db-create-s2jk2" Dec 03 12:23:09 crc kubenswrapper[5052]: I1203 12:23:09.385893 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-db-create-s2jk2" Dec 03 12:23:09 crc kubenswrapper[5052]: I1203 12:23:09.429345 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/bd34573a-a034-4c22-8df3-bab80ed70f45-operator-scripts\") pod \"placement-e65a-account-create-update-8vzns\" (UID: \"bd34573a-a034-4c22-8df3-bab80ed70f45\") " pod="openstack-kuttl-tests/placement-e65a-account-create-update-8vzns" Dec 03 12:23:09 crc kubenswrapper[5052]: I1203 12:23:09.429398 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-mx2pm\" (UniqueName: \"kubernetes.io/projected/bd34573a-a034-4c22-8df3-bab80ed70f45-kube-api-access-mx2pm\") pod \"placement-e65a-account-create-update-8vzns\" (UID: \"bd34573a-a034-4c22-8df3-bab80ed70f45\") " pod="openstack-kuttl-tests/placement-e65a-account-create-update-8vzns" Dec 03 12:23:09 crc kubenswrapper[5052]: I1203 12:23:09.430043 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/bd34573a-a034-4c22-8df3-bab80ed70f45-operator-scripts\") pod \"placement-e65a-account-create-update-8vzns\" (UID: \"bd34573a-a034-4c22-8df3-bab80ed70f45\") " pod="openstack-kuttl-tests/placement-e65a-account-create-update-8vzns" Dec 03 12:23:09 crc kubenswrapper[5052]: I1203 12:23:09.446578 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-mx2pm\" (UniqueName: \"kubernetes.io/projected/bd34573a-a034-4c22-8df3-bab80ed70f45-kube-api-access-mx2pm\") pod \"placement-e65a-account-create-update-8vzns\" (UID: \"bd34573a-a034-4c22-8df3-bab80ed70f45\") " pod="openstack-kuttl-tests/placement-e65a-account-create-update-8vzns" Dec 03 12:23:09 crc kubenswrapper[5052]: I1203 12:23:09.507352 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openshift-marketplace/redhat-marketplace-hx9cz" podUID="fa334119-b1fd-42e8-b64b-a276ef89e4b5" containerName="registry-server" containerID="cri-o://57d4da62ee25a46d86d917668db0e2040b0ddb72f3931305a731ada69c4b95f9" gracePeriod=2 Dec 03 12:23:09 crc kubenswrapper[5052]: I1203 12:23:09.508847 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-db-create-zrpdw" event={"ID":"beb9c625-5acb-4d1e-9044-7fb5287e8cfe","Type":"ContainerStarted","Data":"dae22d6ddc7e6009c3d2ef035b575fe16294b7d9baf876c20c60db3ce6c60b06"} Dec 03 12:23:09 crc kubenswrapper[5052]: I1203 12:23:09.508873 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-db-create-zrpdw" event={"ID":"beb9c625-5acb-4d1e-9044-7fb5287e8cfe","Type":"ContainerStarted","Data":"59f4b0ba5358d28cbdffa06d4aacce0c656b2024f33ff7e929b4f3a835a41863"} Dec 03 12:23:09 crc kubenswrapper[5052]: I1203 12:23:09.524831 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-e65a-account-create-update-8vzns" Dec 03 12:23:09 crc kubenswrapper[5052]: I1203 12:23:09.542630 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-39e2-account-create-update-74dh8"] Dec 03 12:23:09 crc kubenswrapper[5052]: I1203 12:23:09.558170 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/keystone-db-create-zrpdw" podStartSLOduration=1.55814629 podStartE2EDuration="1.55814629s" podCreationTimestamp="2025-12-03 12:23:08 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:23:09.535922226 +0000 UTC m=+3780.124164000" watchObservedRunningTime="2025-12-03 12:23:09.55814629 +0000 UTC m=+3780.146388074" Dec 03 12:23:09 crc kubenswrapper[5052]: I1203 12:23:09.577623 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/glance-db-create-gssmd"] Dec 03 12:23:09 crc kubenswrapper[5052]: I1203 12:23:09.579001 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-db-create-gssmd" Dec 03 12:23:09 crc kubenswrapper[5052]: I1203 12:23:09.605276 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-db-create-gssmd"] Dec 03 12:23:09 crc kubenswrapper[5052]: I1203 12:23:09.672697 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/glance-5b9b-account-create-update-5qk9p"] Dec 03 12:23:09 crc kubenswrapper[5052]: I1203 12:23:09.674188 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-5b9b-account-create-update-5qk9p" Dec 03 12:23:09 crc kubenswrapper[5052]: I1203 12:23:09.703880 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-db-secret" Dec 03 12:23:09 crc kubenswrapper[5052]: I1203 12:23:09.733932 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/f3c78563-0630-4849-85ae-8b5d5ad6795c-operator-scripts\") pod \"glance-db-create-gssmd\" (UID: \"f3c78563-0630-4849-85ae-8b5d5ad6795c\") " pod="openstack-kuttl-tests/glance-db-create-gssmd" Dec 03 12:23:09 crc kubenswrapper[5052]: I1203 12:23:09.734076 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-945dq\" (UniqueName: \"kubernetes.io/projected/f3c78563-0630-4849-85ae-8b5d5ad6795c-kube-api-access-945dq\") pod \"glance-db-create-gssmd\" (UID: \"f3c78563-0630-4849-85ae-8b5d5ad6795c\") " pod="openstack-kuttl-tests/glance-db-create-gssmd" Dec 03 12:23:09 crc kubenswrapper[5052]: I1203 12:23:09.745470 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-5b9b-account-create-update-5qk9p"] Dec 03 12:23:09 crc kubenswrapper[5052]: I1203 12:23:09.835766 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-psvsw\" (UniqueName: \"kubernetes.io/projected/60ba2a3e-26ac-4265-b206-cb9ea4c88584-kube-api-access-psvsw\") pod \"glance-5b9b-account-create-update-5qk9p\" (UID: \"60ba2a3e-26ac-4265-b206-cb9ea4c88584\") " pod="openstack-kuttl-tests/glance-5b9b-account-create-update-5qk9p" Dec 03 12:23:09 crc kubenswrapper[5052]: I1203 12:23:09.835840 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-945dq\" (UniqueName: \"kubernetes.io/projected/f3c78563-0630-4849-85ae-8b5d5ad6795c-kube-api-access-945dq\") pod \"glance-db-create-gssmd\" (UID: \"f3c78563-0630-4849-85ae-8b5d5ad6795c\") " pod="openstack-kuttl-tests/glance-db-create-gssmd" Dec 03 12:23:09 crc kubenswrapper[5052]: I1203 12:23:09.836654 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/f3c78563-0630-4849-85ae-8b5d5ad6795c-operator-scripts\") pod \"glance-db-create-gssmd\" (UID: \"f3c78563-0630-4849-85ae-8b5d5ad6795c\") " pod="openstack-kuttl-tests/glance-db-create-gssmd" Dec 03 12:23:09 crc kubenswrapper[5052]: I1203 12:23:09.836708 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/60ba2a3e-26ac-4265-b206-cb9ea4c88584-operator-scripts\") pod \"glance-5b9b-account-create-update-5qk9p\" (UID: \"60ba2a3e-26ac-4265-b206-cb9ea4c88584\") " pod="openstack-kuttl-tests/glance-5b9b-account-create-update-5qk9p" Dec 03 12:23:09 crc kubenswrapper[5052]: I1203 12:23:09.840594 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/f3c78563-0630-4849-85ae-8b5d5ad6795c-operator-scripts\") pod \"glance-db-create-gssmd\" (UID: \"f3c78563-0630-4849-85ae-8b5d5ad6795c\") " pod="openstack-kuttl-tests/glance-db-create-gssmd" Dec 03 12:23:09 crc kubenswrapper[5052]: I1203 12:23:09.863538 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-945dq\" (UniqueName: \"kubernetes.io/projected/f3c78563-0630-4849-85ae-8b5d5ad6795c-kube-api-access-945dq\") pod \"glance-db-create-gssmd\" (UID: \"f3c78563-0630-4849-85ae-8b5d5ad6795c\") " pod="openstack-kuttl-tests/glance-db-create-gssmd" Dec 03 12:23:09 crc kubenswrapper[5052]: I1203 12:23:09.928924 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/placement-db-create-s2jk2"] Dec 03 12:23:09 crc kubenswrapper[5052]: I1203 12:23:09.943075 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-psvsw\" (UniqueName: \"kubernetes.io/projected/60ba2a3e-26ac-4265-b206-cb9ea4c88584-kube-api-access-psvsw\") pod \"glance-5b9b-account-create-update-5qk9p\" (UID: \"60ba2a3e-26ac-4265-b206-cb9ea4c88584\") " pod="openstack-kuttl-tests/glance-5b9b-account-create-update-5qk9p" Dec 03 12:23:09 crc kubenswrapper[5052]: I1203 12:23:09.943236 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-db-create-gssmd" Dec 03 12:23:09 crc kubenswrapper[5052]: I1203 12:23:09.943524 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/60ba2a3e-26ac-4265-b206-cb9ea4c88584-operator-scripts\") pod \"glance-5b9b-account-create-update-5qk9p\" (UID: \"60ba2a3e-26ac-4265-b206-cb9ea4c88584\") " pod="openstack-kuttl-tests/glance-5b9b-account-create-update-5qk9p" Dec 03 12:23:09 crc kubenswrapper[5052]: I1203 12:23:09.945912 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/60ba2a3e-26ac-4265-b206-cb9ea4c88584-operator-scripts\") pod \"glance-5b9b-account-create-update-5qk9p\" (UID: \"60ba2a3e-26ac-4265-b206-cb9ea4c88584\") " pod="openstack-kuttl-tests/glance-5b9b-account-create-update-5qk9p" Dec 03 12:23:10 crc kubenswrapper[5052]: I1203 12:23:10.156072 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/placement-e65a-account-create-update-8vzns"] Dec 03 12:23:10 crc kubenswrapper[5052]: I1203 12:23:10.424189 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-psvsw\" (UniqueName: \"kubernetes.io/projected/60ba2a3e-26ac-4265-b206-cb9ea4c88584-kube-api-access-psvsw\") pod \"glance-5b9b-account-create-update-5qk9p\" (UID: \"60ba2a3e-26ac-4265-b206-cb9ea4c88584\") " pod="openstack-kuttl-tests/glance-5b9b-account-create-update-5qk9p" Dec 03 12:23:10 crc kubenswrapper[5052]: W1203 12:23:10.433457 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbd34573a_a034_4c22_8df3_bab80ed70f45.slice/crio-b036c8dc4c2a6b60d7123b9936ec8af074c68a2174b7ec6347aa51717edc075b WatchSource:0}: Error finding container b036c8dc4c2a6b60d7123b9936ec8af074c68a2174b7ec6347aa51717edc075b: Status 404 returned error can't find the container with id b036c8dc4c2a6b60d7123b9936ec8af074c68a2174b7ec6347aa51717edc075b Dec 03 12:23:10 crc kubenswrapper[5052]: I1203 12:23:10.442450 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"placement-db-secret" Dec 03 12:23:10 crc kubenswrapper[5052]: I1203 12:23:10.525256 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-39e2-account-create-update-74dh8" event={"ID":"37da25a2-1fa7-4a14-b8d6-1ba47412c6cd","Type":"ContainerStarted","Data":"614bea6cc4eabe0daab18d763a2caa9f83d72eec5cc9a7443d6c819050ee26b5"} Dec 03 12:23:10 crc kubenswrapper[5052]: I1203 12:23:10.529823 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-e65a-account-create-update-8vzns" event={"ID":"bd34573a-a034-4c22-8df3-bab80ed70f45","Type":"ContainerStarted","Data":"b036c8dc4c2a6b60d7123b9936ec8af074c68a2174b7ec6347aa51717edc075b"} Dec 03 12:23:10 crc kubenswrapper[5052]: I1203 12:23:10.532156 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-db-create-s2jk2" event={"ID":"4ea65f95-6ea7-4362-a361-514ace4fbc44","Type":"ContainerStarted","Data":"8b3314d980807900df2cfd903b4bcebac6e3556f6cec0dc966c28a1fad24e519"} Dec 03 12:23:10 crc kubenswrapper[5052]: I1203 12:23:10.543895 5052 generic.go:334] "Generic (PLEG): container finished" podID="fa334119-b1fd-42e8-b64b-a276ef89e4b5" containerID="57d4da62ee25a46d86d917668db0e2040b0ddb72f3931305a731ada69c4b95f9" exitCode=0 Dec 03 12:23:10 crc kubenswrapper[5052]: I1203 12:23:10.544070 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-5b9b-account-create-update-5qk9p" Dec 03 12:23:10 crc kubenswrapper[5052]: I1203 12:23:10.544094 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-marketplace-hx9cz" event={"ID":"fa334119-b1fd-42e8-b64b-a276ef89e4b5","Type":"ContainerDied","Data":"57d4da62ee25a46d86d917668db0e2040b0ddb72f3931305a731ada69c4b95f9"} Dec 03 12:23:10 crc kubenswrapper[5052]: I1203 12:23:10.544466 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-marketplace-hx9cz" event={"ID":"fa334119-b1fd-42e8-b64b-a276ef89e4b5","Type":"ContainerDied","Data":"e65548704517b7f3191d1758db0ccb4f7fbc04d1cd605cc4bc87eb302a949b07"} Dec 03 12:23:10 crc kubenswrapper[5052]: I1203 12:23:10.544491 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e65548704517b7f3191d1758db0ccb4f7fbc04d1cd605cc4bc87eb302a949b07" Dec 03 12:23:10 crc kubenswrapper[5052]: I1203 12:23:10.546971 5052 generic.go:334] "Generic (PLEG): container finished" podID="beb9c625-5acb-4d1e-9044-7fb5287e8cfe" containerID="dae22d6ddc7e6009c3d2ef035b575fe16294b7d9baf876c20c60db3ce6c60b06" exitCode=0 Dec 03 12:23:10 crc kubenswrapper[5052]: I1203 12:23:10.547002 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-db-create-zrpdw" event={"ID":"beb9c625-5acb-4d1e-9044-7fb5287e8cfe","Type":"ContainerDied","Data":"dae22d6ddc7e6009c3d2ef035b575fe16294b7d9baf876c20c60db3ce6c60b06"} Dec 03 12:23:10 crc kubenswrapper[5052]: I1203 12:23:10.695966 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/redhat-marketplace-hx9cz" Dec 03 12:23:10 crc kubenswrapper[5052]: I1203 12:23:10.860438 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9jpdl\" (UniqueName: \"kubernetes.io/projected/fa334119-b1fd-42e8-b64b-a276ef89e4b5-kube-api-access-9jpdl\") pod \"fa334119-b1fd-42e8-b64b-a276ef89e4b5\" (UID: \"fa334119-b1fd-42e8-b64b-a276ef89e4b5\") " Dec 03 12:23:10 crc kubenswrapper[5052]: I1203 12:23:10.860637 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/fa334119-b1fd-42e8-b64b-a276ef89e4b5-utilities\") pod \"fa334119-b1fd-42e8-b64b-a276ef89e4b5\" (UID: \"fa334119-b1fd-42e8-b64b-a276ef89e4b5\") " Dec 03 12:23:10 crc kubenswrapper[5052]: I1203 12:23:10.860729 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/fa334119-b1fd-42e8-b64b-a276ef89e4b5-catalog-content\") pod \"fa334119-b1fd-42e8-b64b-a276ef89e4b5\" (UID: \"fa334119-b1fd-42e8-b64b-a276ef89e4b5\") " Dec 03 12:23:10 crc kubenswrapper[5052]: I1203 12:23:10.861494 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fa334119-b1fd-42e8-b64b-a276ef89e4b5-utilities" (OuterVolumeSpecName: "utilities") pod "fa334119-b1fd-42e8-b64b-a276ef89e4b5" (UID: "fa334119-b1fd-42e8-b64b-a276ef89e4b5"). InnerVolumeSpecName "utilities". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:23:10 crc kubenswrapper[5052]: I1203 12:23:10.869114 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fa334119-b1fd-42e8-b64b-a276ef89e4b5-kube-api-access-9jpdl" (OuterVolumeSpecName: "kube-api-access-9jpdl") pod "fa334119-b1fd-42e8-b64b-a276ef89e4b5" (UID: "fa334119-b1fd-42e8-b64b-a276ef89e4b5"). InnerVolumeSpecName "kube-api-access-9jpdl". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:23:10 crc kubenswrapper[5052]: I1203 12:23:10.893924 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fa334119-b1fd-42e8-b64b-a276ef89e4b5-catalog-content" (OuterVolumeSpecName: "catalog-content") pod "fa334119-b1fd-42e8-b64b-a276ef89e4b5" (UID: "fa334119-b1fd-42e8-b64b-a276ef89e4b5"). InnerVolumeSpecName "catalog-content". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:23:10 crc kubenswrapper[5052]: I1203 12:23:10.962882 5052 reconciler_common.go:293] "Volume detached for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/fa334119-b1fd-42e8-b64b-a276ef89e4b5-utilities\") on node \"crc\" DevicePath \"\"" Dec 03 12:23:10 crc kubenswrapper[5052]: I1203 12:23:10.962928 5052 reconciler_common.go:293] "Volume detached for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/fa334119-b1fd-42e8-b64b-a276ef89e4b5-catalog-content\") on node \"crc\" DevicePath \"\"" Dec 03 12:23:10 crc kubenswrapper[5052]: I1203 12:23:10.962944 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-9jpdl\" (UniqueName: \"kubernetes.io/projected/fa334119-b1fd-42e8-b64b-a276ef89e4b5-kube-api-access-9jpdl\") on node \"crc\" DevicePath \"\"" Dec 03 12:23:10 crc kubenswrapper[5052]: I1203 12:23:10.966907 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-db-create-gssmd"] Dec 03 12:23:10 crc kubenswrapper[5052]: W1203 12:23:10.968465 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf3c78563_0630_4849_85ae_8b5d5ad6795c.slice/crio-6d139676a0b123281365ee03ca8cff84cf62ea6ef3820baaf1162390fe817f0f WatchSource:0}: Error finding container 6d139676a0b123281365ee03ca8cff84cf62ea6ef3820baaf1162390fe817f0f: Status 404 returned error can't find the container with id 6d139676a0b123281365ee03ca8cff84cf62ea6ef3820baaf1162390fe817f0f Dec 03 12:23:11 crc kubenswrapper[5052]: I1203 12:23:11.138477 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-5b9b-account-create-update-5qk9p"] Dec 03 12:23:11 crc kubenswrapper[5052]: I1203 12:23:11.559321 5052 generic.go:334] "Generic (PLEG): container finished" podID="bd34573a-a034-4c22-8df3-bab80ed70f45" containerID="2586d14ec5172cd8887cf742de44efde3b182ee97f4136dec4ca8ec79d2e0521" exitCode=0 Dec 03 12:23:11 crc kubenswrapper[5052]: I1203 12:23:11.560068 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-e65a-account-create-update-8vzns" event={"ID":"bd34573a-a034-4c22-8df3-bab80ed70f45","Type":"ContainerDied","Data":"2586d14ec5172cd8887cf742de44efde3b182ee97f4136dec4ca8ec79d2e0521"} Dec 03 12:23:11 crc kubenswrapper[5052]: I1203 12:23:11.563363 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-5b9b-account-create-update-5qk9p" event={"ID":"60ba2a3e-26ac-4265-b206-cb9ea4c88584","Type":"ContainerStarted","Data":"0fb21147d3c613c2babe2b6f711cd697be71aba6d40349fe2ecac629ffec60fa"} Dec 03 12:23:11 crc kubenswrapper[5052]: I1203 12:23:11.563407 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-5b9b-account-create-update-5qk9p" event={"ID":"60ba2a3e-26ac-4265-b206-cb9ea4c88584","Type":"ContainerStarted","Data":"263430252da344ec0d502f5e8bdd2a3487d0b0212ce1908734b74745a321ff08"} Dec 03 12:23:11 crc kubenswrapper[5052]: I1203 12:23:11.565756 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-db-create-gssmd" event={"ID":"f3c78563-0630-4849-85ae-8b5d5ad6795c","Type":"ContainerStarted","Data":"16f725234e733e8166ae824e324c0f1ea6f2d967a6fb37526435b3960d3691cc"} Dec 03 12:23:11 crc kubenswrapper[5052]: I1203 12:23:11.565812 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-db-create-gssmd" event={"ID":"f3c78563-0630-4849-85ae-8b5d5ad6795c","Type":"ContainerStarted","Data":"6d139676a0b123281365ee03ca8cff84cf62ea6ef3820baaf1162390fe817f0f"} Dec 03 12:23:11 crc kubenswrapper[5052]: I1203 12:23:11.567327 5052 generic.go:334] "Generic (PLEG): container finished" podID="4ea65f95-6ea7-4362-a361-514ace4fbc44" containerID="76ce33028e7542af6d393c3f4a62e893bf649320e40a27297e557281f0c774d0" exitCode=0 Dec 03 12:23:11 crc kubenswrapper[5052]: I1203 12:23:11.567411 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-db-create-s2jk2" event={"ID":"4ea65f95-6ea7-4362-a361-514ace4fbc44","Type":"ContainerDied","Data":"76ce33028e7542af6d393c3f4a62e893bf649320e40a27297e557281f0c774d0"} Dec 03 12:23:11 crc kubenswrapper[5052]: I1203 12:23:11.569385 5052 generic.go:334] "Generic (PLEG): container finished" podID="37da25a2-1fa7-4a14-b8d6-1ba47412c6cd" containerID="4263fd69a4c9b5f25263e2be807fba706bb60e8af115e188221406a2cb517c61" exitCode=0 Dec 03 12:23:11 crc kubenswrapper[5052]: I1203 12:23:11.569430 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-39e2-account-create-update-74dh8" event={"ID":"37da25a2-1fa7-4a14-b8d6-1ba47412c6cd","Type":"ContainerDied","Data":"4263fd69a4c9b5f25263e2be807fba706bb60e8af115e188221406a2cb517c61"} Dec 03 12:23:11 crc kubenswrapper[5052]: I1203 12:23:11.569554 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/redhat-marketplace-hx9cz" Dec 03 12:23:11 crc kubenswrapper[5052]: I1203 12:23:11.619035 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/glance-db-create-gssmd" podStartSLOduration=2.619012502 podStartE2EDuration="2.619012502s" podCreationTimestamp="2025-12-03 12:23:09 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:23:11.618862958 +0000 UTC m=+3782.207104722" watchObservedRunningTime="2025-12-03 12:23:11.619012502 +0000 UTC m=+3782.207254266" Dec 03 12:23:11 crc kubenswrapper[5052]: I1203 12:23:11.669875 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/glance-5b9b-account-create-update-5qk9p" podStartSLOduration=2.669854123 podStartE2EDuration="2.669854123s" podCreationTimestamp="2025-12-03 12:23:09 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:23:11.656655647 +0000 UTC m=+3782.244897411" watchObservedRunningTime="2025-12-03 12:23:11.669854123 +0000 UTC m=+3782.258095897" Dec 03 12:23:11 crc kubenswrapper[5052]: I1203 12:23:11.697196 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openshift-marketplace/redhat-marketplace-hx9cz"] Dec 03 12:23:11 crc kubenswrapper[5052]: I1203 12:23:11.716370 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openshift-marketplace/redhat-marketplace-hx9cz"] Dec 03 12:23:12 crc kubenswrapper[5052]: I1203 12:23:12.076148 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-db-create-zrpdw" Dec 03 12:23:12 crc kubenswrapper[5052]: I1203 12:23:12.189208 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/beb9c625-5acb-4d1e-9044-7fb5287e8cfe-operator-scripts\") pod \"beb9c625-5acb-4d1e-9044-7fb5287e8cfe\" (UID: \"beb9c625-5acb-4d1e-9044-7fb5287e8cfe\") " Dec 03 12:23:12 crc kubenswrapper[5052]: I1203 12:23:12.189367 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2vb7c\" (UniqueName: \"kubernetes.io/projected/beb9c625-5acb-4d1e-9044-7fb5287e8cfe-kube-api-access-2vb7c\") pod \"beb9c625-5acb-4d1e-9044-7fb5287e8cfe\" (UID: \"beb9c625-5acb-4d1e-9044-7fb5287e8cfe\") " Dec 03 12:23:12 crc kubenswrapper[5052]: I1203 12:23:12.190830 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/beb9c625-5acb-4d1e-9044-7fb5287e8cfe-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "beb9c625-5acb-4d1e-9044-7fb5287e8cfe" (UID: "beb9c625-5acb-4d1e-9044-7fb5287e8cfe"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:23:12 crc kubenswrapper[5052]: I1203 12:23:12.194612 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/beb9c625-5acb-4d1e-9044-7fb5287e8cfe-kube-api-access-2vb7c" (OuterVolumeSpecName: "kube-api-access-2vb7c") pod "beb9c625-5acb-4d1e-9044-7fb5287e8cfe" (UID: "beb9c625-5acb-4d1e-9044-7fb5287e8cfe"). InnerVolumeSpecName "kube-api-access-2vb7c". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:23:12 crc kubenswrapper[5052]: I1203 12:23:12.291132 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/beb9c625-5acb-4d1e-9044-7fb5287e8cfe-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:23:12 crc kubenswrapper[5052]: I1203 12:23:12.291167 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-2vb7c\" (UniqueName: \"kubernetes.io/projected/beb9c625-5acb-4d1e-9044-7fb5287e8cfe-kube-api-access-2vb7c\") on node \"crc\" DevicePath \"\"" Dec 03 12:23:12 crc kubenswrapper[5052]: I1203 12:23:12.579091 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-db-create-zrpdw" event={"ID":"beb9c625-5acb-4d1e-9044-7fb5287e8cfe","Type":"ContainerDied","Data":"59f4b0ba5358d28cbdffa06d4aacce0c656b2024f33ff7e929b4f3a835a41863"} Dec 03 12:23:12 crc kubenswrapper[5052]: I1203 12:23:12.579132 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="59f4b0ba5358d28cbdffa06d4aacce0c656b2024f33ff7e929b4f3a835a41863" Dec 03 12:23:12 crc kubenswrapper[5052]: I1203 12:23:12.579111 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-db-create-zrpdw" Dec 03 12:23:12 crc kubenswrapper[5052]: I1203 12:23:12.581758 5052 generic.go:334] "Generic (PLEG): container finished" podID="60ba2a3e-26ac-4265-b206-cb9ea4c88584" containerID="0fb21147d3c613c2babe2b6f711cd697be71aba6d40349fe2ecac629ffec60fa" exitCode=0 Dec 03 12:23:12 crc kubenswrapper[5052]: I1203 12:23:12.581871 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-5b9b-account-create-update-5qk9p" event={"ID":"60ba2a3e-26ac-4265-b206-cb9ea4c88584","Type":"ContainerDied","Data":"0fb21147d3c613c2babe2b6f711cd697be71aba6d40349fe2ecac629ffec60fa"} Dec 03 12:23:12 crc kubenswrapper[5052]: I1203 12:23:12.583657 5052 generic.go:334] "Generic (PLEG): container finished" podID="f3c78563-0630-4849-85ae-8b5d5ad6795c" containerID="16f725234e733e8166ae824e324c0f1ea6f2d967a6fb37526435b3960d3691cc" exitCode=0 Dec 03 12:23:12 crc kubenswrapper[5052]: I1203 12:23:12.583907 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-db-create-gssmd" event={"ID":"f3c78563-0630-4849-85ae-8b5d5ad6795c","Type":"ContainerDied","Data":"16f725234e733e8166ae824e324c0f1ea6f2d967a6fb37526435b3960d3691cc"} Dec 03 12:23:12 crc kubenswrapper[5052]: I1203 12:23:12.687292 5052 scope.go:117] "RemoveContainer" containerID="09b4a0072741f60499d24f40cc4ff3406d5d22671ced431363ab865172786300" Dec 03 12:23:12 crc kubenswrapper[5052]: E1203 12:23:12.687835 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 12:23:12 crc kubenswrapper[5052]: I1203 12:23:12.941874 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:23:13 crc kubenswrapper[5052]: I1203 12:23:13.018471 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-39e2-account-create-update-74dh8" Dec 03 12:23:13 crc kubenswrapper[5052]: I1203 12:23:13.121536 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xg5v8\" (UniqueName: \"kubernetes.io/projected/37da25a2-1fa7-4a14-b8d6-1ba47412c6cd-kube-api-access-xg5v8\") pod \"37da25a2-1fa7-4a14-b8d6-1ba47412c6cd\" (UID: \"37da25a2-1fa7-4a14-b8d6-1ba47412c6cd\") " Dec 03 12:23:13 crc kubenswrapper[5052]: I1203 12:23:13.121620 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/37da25a2-1fa7-4a14-b8d6-1ba47412c6cd-operator-scripts\") pod \"37da25a2-1fa7-4a14-b8d6-1ba47412c6cd\" (UID: \"37da25a2-1fa7-4a14-b8d6-1ba47412c6cd\") " Dec 03 12:23:13 crc kubenswrapper[5052]: I1203 12:23:13.123023 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/37da25a2-1fa7-4a14-b8d6-1ba47412c6cd-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "37da25a2-1fa7-4a14-b8d6-1ba47412c6cd" (UID: "37da25a2-1fa7-4a14-b8d6-1ba47412c6cd"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:23:13 crc kubenswrapper[5052]: I1203 12:23:13.128325 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/37da25a2-1fa7-4a14-b8d6-1ba47412c6cd-kube-api-access-xg5v8" (OuterVolumeSpecName: "kube-api-access-xg5v8") pod "37da25a2-1fa7-4a14-b8d6-1ba47412c6cd" (UID: "37da25a2-1fa7-4a14-b8d6-1ba47412c6cd"). InnerVolumeSpecName "kube-api-access-xg5v8". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:23:13 crc kubenswrapper[5052]: I1203 12:23:13.187560 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-db-create-s2jk2" Dec 03 12:23:13 crc kubenswrapper[5052]: I1203 12:23:13.193489 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-e65a-account-create-update-8vzns" Dec 03 12:23:13 crc kubenswrapper[5052]: I1203 12:23:13.223557 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/37da25a2-1fa7-4a14-b8d6-1ba47412c6cd-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:23:13 crc kubenswrapper[5052]: I1203 12:23:13.223598 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-xg5v8\" (UniqueName: \"kubernetes.io/projected/37da25a2-1fa7-4a14-b8d6-1ba47412c6cd-kube-api-access-xg5v8\") on node \"crc\" DevicePath \"\"" Dec 03 12:23:13 crc kubenswrapper[5052]: I1203 12:23:13.324177 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-466n5\" (UniqueName: \"kubernetes.io/projected/4ea65f95-6ea7-4362-a361-514ace4fbc44-kube-api-access-466n5\") pod \"4ea65f95-6ea7-4362-a361-514ace4fbc44\" (UID: \"4ea65f95-6ea7-4362-a361-514ace4fbc44\") " Dec 03 12:23:13 crc kubenswrapper[5052]: I1203 12:23:13.324338 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/4ea65f95-6ea7-4362-a361-514ace4fbc44-operator-scripts\") pod \"4ea65f95-6ea7-4362-a361-514ace4fbc44\" (UID: \"4ea65f95-6ea7-4362-a361-514ace4fbc44\") " Dec 03 12:23:13 crc kubenswrapper[5052]: I1203 12:23:13.324405 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mx2pm\" (UniqueName: \"kubernetes.io/projected/bd34573a-a034-4c22-8df3-bab80ed70f45-kube-api-access-mx2pm\") pod \"bd34573a-a034-4c22-8df3-bab80ed70f45\" (UID: \"bd34573a-a034-4c22-8df3-bab80ed70f45\") " Dec 03 12:23:13 crc kubenswrapper[5052]: I1203 12:23:13.324427 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/bd34573a-a034-4c22-8df3-bab80ed70f45-operator-scripts\") pod \"bd34573a-a034-4c22-8df3-bab80ed70f45\" (UID: \"bd34573a-a034-4c22-8df3-bab80ed70f45\") " Dec 03 12:23:13 crc kubenswrapper[5052]: I1203 12:23:13.324864 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4ea65f95-6ea7-4362-a361-514ace4fbc44-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "4ea65f95-6ea7-4362-a361-514ace4fbc44" (UID: "4ea65f95-6ea7-4362-a361-514ace4fbc44"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:23:13 crc kubenswrapper[5052]: I1203 12:23:13.324942 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bd34573a-a034-4c22-8df3-bab80ed70f45-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "bd34573a-a034-4c22-8df3-bab80ed70f45" (UID: "bd34573a-a034-4c22-8df3-bab80ed70f45"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:23:13 crc kubenswrapper[5052]: I1203 12:23:13.325443 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/4ea65f95-6ea7-4362-a361-514ace4fbc44-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:23:13 crc kubenswrapper[5052]: I1203 12:23:13.325464 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/bd34573a-a034-4c22-8df3-bab80ed70f45-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:23:13 crc kubenswrapper[5052]: I1203 12:23:13.327167 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4ea65f95-6ea7-4362-a361-514ace4fbc44-kube-api-access-466n5" (OuterVolumeSpecName: "kube-api-access-466n5") pod "4ea65f95-6ea7-4362-a361-514ace4fbc44" (UID: "4ea65f95-6ea7-4362-a361-514ace4fbc44"). InnerVolumeSpecName "kube-api-access-466n5". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:23:13 crc kubenswrapper[5052]: I1203 12:23:13.327653 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bd34573a-a034-4c22-8df3-bab80ed70f45-kube-api-access-mx2pm" (OuterVolumeSpecName: "kube-api-access-mx2pm") pod "bd34573a-a034-4c22-8df3-bab80ed70f45" (UID: "bd34573a-a034-4c22-8df3-bab80ed70f45"). InnerVolumeSpecName "kube-api-access-mx2pm". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:23:13 crc kubenswrapper[5052]: I1203 12:23:13.426887 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-mx2pm\" (UniqueName: \"kubernetes.io/projected/bd34573a-a034-4c22-8df3-bab80ed70f45-kube-api-access-mx2pm\") on node \"crc\" DevicePath \"\"" Dec 03 12:23:13 crc kubenswrapper[5052]: I1203 12:23:13.426920 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-466n5\" (UniqueName: \"kubernetes.io/projected/4ea65f95-6ea7-4362-a361-514ace4fbc44-kube-api-access-466n5\") on node \"crc\" DevicePath \"\"" Dec 03 12:23:13 crc kubenswrapper[5052]: I1203 12:23:13.594355 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-db-create-s2jk2" Dec 03 12:23:13 crc kubenswrapper[5052]: I1203 12:23:13.594353 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-db-create-s2jk2" event={"ID":"4ea65f95-6ea7-4362-a361-514ace4fbc44","Type":"ContainerDied","Data":"8b3314d980807900df2cfd903b4bcebac6e3556f6cec0dc966c28a1fad24e519"} Dec 03 12:23:13 crc kubenswrapper[5052]: I1203 12:23:13.594499 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8b3314d980807900df2cfd903b4bcebac6e3556f6cec0dc966c28a1fad24e519" Dec 03 12:23:13 crc kubenswrapper[5052]: I1203 12:23:13.595791 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-39e2-account-create-update-74dh8" Dec 03 12:23:13 crc kubenswrapper[5052]: I1203 12:23:13.595847 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-39e2-account-create-update-74dh8" event={"ID":"37da25a2-1fa7-4a14-b8d6-1ba47412c6cd","Type":"ContainerDied","Data":"614bea6cc4eabe0daab18d763a2caa9f83d72eec5cc9a7443d6c819050ee26b5"} Dec 03 12:23:13 crc kubenswrapper[5052]: I1203 12:23:13.595871 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="614bea6cc4eabe0daab18d763a2caa9f83d72eec5cc9a7443d6c819050ee26b5" Dec 03 12:23:13 crc kubenswrapper[5052]: I1203 12:23:13.598808 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-e65a-account-create-update-8vzns" Dec 03 12:23:13 crc kubenswrapper[5052]: I1203 12:23:13.598838 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-e65a-account-create-update-8vzns" event={"ID":"bd34573a-a034-4c22-8df3-bab80ed70f45","Type":"ContainerDied","Data":"b036c8dc4c2a6b60d7123b9936ec8af074c68a2174b7ec6347aa51717edc075b"} Dec 03 12:23:13 crc kubenswrapper[5052]: I1203 12:23:13.598862 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b036c8dc4c2a6b60d7123b9936ec8af074c68a2174b7ec6347aa51717edc075b" Dec 03 12:23:13 crc kubenswrapper[5052]: I1203 12:23:13.703818 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fa334119-b1fd-42e8-b64b-a276ef89e4b5" path="/var/lib/kubelet/pods/fa334119-b1fd-42e8-b64b-a276ef89e4b5/volumes" Dec 03 12:23:13 crc kubenswrapper[5052]: I1203 12:23:13.919846 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-5b9b-account-create-update-5qk9p" Dec 03 12:23:13 crc kubenswrapper[5052]: I1203 12:23:13.954470 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-psvsw\" (UniqueName: \"kubernetes.io/projected/60ba2a3e-26ac-4265-b206-cb9ea4c88584-kube-api-access-psvsw\") pod \"60ba2a3e-26ac-4265-b206-cb9ea4c88584\" (UID: \"60ba2a3e-26ac-4265-b206-cb9ea4c88584\") " Dec 03 12:23:13 crc kubenswrapper[5052]: I1203 12:23:13.954672 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/60ba2a3e-26ac-4265-b206-cb9ea4c88584-operator-scripts\") pod \"60ba2a3e-26ac-4265-b206-cb9ea4c88584\" (UID: \"60ba2a3e-26ac-4265-b206-cb9ea4c88584\") " Dec 03 12:23:13 crc kubenswrapper[5052]: I1203 12:23:13.955295 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/60ba2a3e-26ac-4265-b206-cb9ea4c88584-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "60ba2a3e-26ac-4265-b206-cb9ea4c88584" (UID: "60ba2a3e-26ac-4265-b206-cb9ea4c88584"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:23:13 crc kubenswrapper[5052]: I1203 12:23:13.955773 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/60ba2a3e-26ac-4265-b206-cb9ea4c88584-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:23:13 crc kubenswrapper[5052]: I1203 12:23:13.971802 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/60ba2a3e-26ac-4265-b206-cb9ea4c88584-kube-api-access-psvsw" (OuterVolumeSpecName: "kube-api-access-psvsw") pod "60ba2a3e-26ac-4265-b206-cb9ea4c88584" (UID: "60ba2a3e-26ac-4265-b206-cb9ea4c88584"). InnerVolumeSpecName "kube-api-access-psvsw". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:23:14 crc kubenswrapper[5052]: I1203 12:23:14.016872 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-db-create-gssmd" Dec 03 12:23:14 crc kubenswrapper[5052]: I1203 12:23:14.058839 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/f3c78563-0630-4849-85ae-8b5d5ad6795c-operator-scripts\") pod \"f3c78563-0630-4849-85ae-8b5d5ad6795c\" (UID: \"f3c78563-0630-4849-85ae-8b5d5ad6795c\") " Dec 03 12:23:14 crc kubenswrapper[5052]: I1203 12:23:14.059004 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-945dq\" (UniqueName: \"kubernetes.io/projected/f3c78563-0630-4849-85ae-8b5d5ad6795c-kube-api-access-945dq\") pod \"f3c78563-0630-4849-85ae-8b5d5ad6795c\" (UID: \"f3c78563-0630-4849-85ae-8b5d5ad6795c\") " Dec 03 12:23:14 crc kubenswrapper[5052]: I1203 12:23:14.059576 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-psvsw\" (UniqueName: \"kubernetes.io/projected/60ba2a3e-26ac-4265-b206-cb9ea4c88584-kube-api-access-psvsw\") on node \"crc\" DevicePath \"\"" Dec 03 12:23:14 crc kubenswrapper[5052]: I1203 12:23:14.073230 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f3c78563-0630-4849-85ae-8b5d5ad6795c-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "f3c78563-0630-4849-85ae-8b5d5ad6795c" (UID: "f3c78563-0630-4849-85ae-8b5d5ad6795c"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:23:14 crc kubenswrapper[5052]: I1203 12:23:14.082094 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f3c78563-0630-4849-85ae-8b5d5ad6795c-kube-api-access-945dq" (OuterVolumeSpecName: "kube-api-access-945dq") pod "f3c78563-0630-4849-85ae-8b5d5ad6795c" (UID: "f3c78563-0630-4849-85ae-8b5d5ad6795c"). InnerVolumeSpecName "kube-api-access-945dq". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:23:14 crc kubenswrapper[5052]: I1203 12:23:14.160794 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/f3c78563-0630-4849-85ae-8b5d5ad6795c-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:23:14 crc kubenswrapper[5052]: I1203 12:23:14.160842 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-945dq\" (UniqueName: \"kubernetes.io/projected/f3c78563-0630-4849-85ae-8b5d5ad6795c-kube-api-access-945dq\") on node \"crc\" DevicePath \"\"" Dec 03 12:23:14 crc kubenswrapper[5052]: I1203 12:23:14.610230 5052 generic.go:334] "Generic (PLEG): container finished" podID="46e92959-0fd2-4e2b-8369-35cf14716af3" containerID="3cc4322f202b3e31f0e15cf554b9c3df35c358e0a3edc6031c1321e38a0851d3" exitCode=0 Dec 03 12:23:14 crc kubenswrapper[5052]: I1203 12:23:14.610343 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-ring-rebalance-b9ssr" event={"ID":"46e92959-0fd2-4e2b-8369-35cf14716af3","Type":"ContainerDied","Data":"3cc4322f202b3e31f0e15cf554b9c3df35c358e0a3edc6031c1321e38a0851d3"} Dec 03 12:23:14 crc kubenswrapper[5052]: I1203 12:23:14.611716 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-5b9b-account-create-update-5qk9p" event={"ID":"60ba2a3e-26ac-4265-b206-cb9ea4c88584","Type":"ContainerDied","Data":"263430252da344ec0d502f5e8bdd2a3487d0b0212ce1908734b74745a321ff08"} Dec 03 12:23:14 crc kubenswrapper[5052]: I1203 12:23:14.611746 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-5b9b-account-create-update-5qk9p" Dec 03 12:23:14 crc kubenswrapper[5052]: I1203 12:23:14.611764 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="263430252da344ec0d502f5e8bdd2a3487d0b0212ce1908734b74745a321ff08" Dec 03 12:23:14 crc kubenswrapper[5052]: I1203 12:23:14.613233 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-db-create-gssmd" event={"ID":"f3c78563-0630-4849-85ae-8b5d5ad6795c","Type":"ContainerDied","Data":"6d139676a0b123281365ee03ca8cff84cf62ea6ef3820baaf1162390fe817f0f"} Dec 03 12:23:14 crc kubenswrapper[5052]: I1203 12:23:14.613257 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6d139676a0b123281365ee03ca8cff84cf62ea6ef3820baaf1162390fe817f0f" Dec 03 12:23:14 crc kubenswrapper[5052]: I1203 12:23:14.613277 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-db-create-gssmd" Dec 03 12:23:16 crc kubenswrapper[5052]: I1203 12:23:16.011558 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-ring-rebalance-b9ssr" Dec 03 12:23:16 crc kubenswrapper[5052]: I1203 12:23:16.191043 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"swiftconf\" (UniqueName: \"kubernetes.io/secret/46e92959-0fd2-4e2b-8369-35cf14716af3-swiftconf\") pod \"46e92959-0fd2-4e2b-8369-35cf14716af3\" (UID: \"46e92959-0fd2-4e2b-8369-35cf14716af3\") " Dec 03 12:23:16 crc kubenswrapper[5052]: I1203 12:23:16.191102 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ring-data-devices\" (UniqueName: \"kubernetes.io/configmap/46e92959-0fd2-4e2b-8369-35cf14716af3-ring-data-devices\") pod \"46e92959-0fd2-4e2b-8369-35cf14716af3\" (UID: \"46e92959-0fd2-4e2b-8369-35cf14716af3\") " Dec 03 12:23:16 crc kubenswrapper[5052]: I1203 12:23:16.191194 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j595l\" (UniqueName: \"kubernetes.io/projected/46e92959-0fd2-4e2b-8369-35cf14716af3-kube-api-access-j595l\") pod \"46e92959-0fd2-4e2b-8369-35cf14716af3\" (UID: \"46e92959-0fd2-4e2b-8369-35cf14716af3\") " Dec 03 12:23:16 crc kubenswrapper[5052]: I1203 12:23:16.191287 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/46e92959-0fd2-4e2b-8369-35cf14716af3-scripts\") pod \"46e92959-0fd2-4e2b-8369-35cf14716af3\" (UID: \"46e92959-0fd2-4e2b-8369-35cf14716af3\") " Dec 03 12:23:16 crc kubenswrapper[5052]: I1203 12:23:16.191882 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"dispersionconf\" (UniqueName: \"kubernetes.io/secret/46e92959-0fd2-4e2b-8369-35cf14716af3-dispersionconf\") pod \"46e92959-0fd2-4e2b-8369-35cf14716af3\" (UID: \"46e92959-0fd2-4e2b-8369-35cf14716af3\") " Dec 03 12:23:16 crc kubenswrapper[5052]: I1203 12:23:16.191980 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/46e92959-0fd2-4e2b-8369-35cf14716af3-ring-data-devices" (OuterVolumeSpecName: "ring-data-devices") pod "46e92959-0fd2-4e2b-8369-35cf14716af3" (UID: "46e92959-0fd2-4e2b-8369-35cf14716af3"). InnerVolumeSpecName "ring-data-devices". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:23:16 crc kubenswrapper[5052]: I1203 12:23:16.192130 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/empty-dir/46e92959-0fd2-4e2b-8369-35cf14716af3-etc-swift\") pod \"46e92959-0fd2-4e2b-8369-35cf14716af3\" (UID: \"46e92959-0fd2-4e2b-8369-35cf14716af3\") " Dec 03 12:23:16 crc kubenswrapper[5052]: I1203 12:23:16.192187 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/46e92959-0fd2-4e2b-8369-35cf14716af3-combined-ca-bundle\") pod \"46e92959-0fd2-4e2b-8369-35cf14716af3\" (UID: \"46e92959-0fd2-4e2b-8369-35cf14716af3\") " Dec 03 12:23:16 crc kubenswrapper[5052]: I1203 12:23:16.192612 5052 reconciler_common.go:293] "Volume detached for volume \"ring-data-devices\" (UniqueName: \"kubernetes.io/configmap/46e92959-0fd2-4e2b-8369-35cf14716af3-ring-data-devices\") on node \"crc\" DevicePath \"\"" Dec 03 12:23:16 crc kubenswrapper[5052]: I1203 12:23:16.192904 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/46e92959-0fd2-4e2b-8369-35cf14716af3-etc-swift" (OuterVolumeSpecName: "etc-swift") pod "46e92959-0fd2-4e2b-8369-35cf14716af3" (UID: "46e92959-0fd2-4e2b-8369-35cf14716af3"). InnerVolumeSpecName "etc-swift". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:23:16 crc kubenswrapper[5052]: I1203 12:23:16.196937 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/46e92959-0fd2-4e2b-8369-35cf14716af3-kube-api-access-j595l" (OuterVolumeSpecName: "kube-api-access-j595l") pod "46e92959-0fd2-4e2b-8369-35cf14716af3" (UID: "46e92959-0fd2-4e2b-8369-35cf14716af3"). InnerVolumeSpecName "kube-api-access-j595l". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:23:16 crc kubenswrapper[5052]: I1203 12:23:16.199320 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/46e92959-0fd2-4e2b-8369-35cf14716af3-dispersionconf" (OuterVolumeSpecName: "dispersionconf") pod "46e92959-0fd2-4e2b-8369-35cf14716af3" (UID: "46e92959-0fd2-4e2b-8369-35cf14716af3"). InnerVolumeSpecName "dispersionconf". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:23:16 crc kubenswrapper[5052]: I1203 12:23:16.215679 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/46e92959-0fd2-4e2b-8369-35cf14716af3-scripts" (OuterVolumeSpecName: "scripts") pod "46e92959-0fd2-4e2b-8369-35cf14716af3" (UID: "46e92959-0fd2-4e2b-8369-35cf14716af3"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:23:16 crc kubenswrapper[5052]: I1203 12:23:16.215778 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/46e92959-0fd2-4e2b-8369-35cf14716af3-swiftconf" (OuterVolumeSpecName: "swiftconf") pod "46e92959-0fd2-4e2b-8369-35cf14716af3" (UID: "46e92959-0fd2-4e2b-8369-35cf14716af3"). InnerVolumeSpecName "swiftconf". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:23:16 crc kubenswrapper[5052]: I1203 12:23:16.217241 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/46e92959-0fd2-4e2b-8369-35cf14716af3-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "46e92959-0fd2-4e2b-8369-35cf14716af3" (UID: "46e92959-0fd2-4e2b-8369-35cf14716af3"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:23:16 crc kubenswrapper[5052]: I1203 12:23:16.293887 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-j595l\" (UniqueName: \"kubernetes.io/projected/46e92959-0fd2-4e2b-8369-35cf14716af3-kube-api-access-j595l\") on node \"crc\" DevicePath \"\"" Dec 03 12:23:16 crc kubenswrapper[5052]: I1203 12:23:16.293921 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/46e92959-0fd2-4e2b-8369-35cf14716af3-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:23:16 crc kubenswrapper[5052]: I1203 12:23:16.293930 5052 reconciler_common.go:293] "Volume detached for volume \"dispersionconf\" (UniqueName: \"kubernetes.io/secret/46e92959-0fd2-4e2b-8369-35cf14716af3-dispersionconf\") on node \"crc\" DevicePath \"\"" Dec 03 12:23:16 crc kubenswrapper[5052]: I1203 12:23:16.293939 5052 reconciler_common.go:293] "Volume detached for volume \"etc-swift\" (UniqueName: \"kubernetes.io/empty-dir/46e92959-0fd2-4e2b-8369-35cf14716af3-etc-swift\") on node \"crc\" DevicePath \"\"" Dec 03 12:23:16 crc kubenswrapper[5052]: I1203 12:23:16.293947 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/46e92959-0fd2-4e2b-8369-35cf14716af3-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:23:16 crc kubenswrapper[5052]: I1203 12:23:16.293955 5052 reconciler_common.go:293] "Volume detached for volume \"swiftconf\" (UniqueName: \"kubernetes.io/secret/46e92959-0fd2-4e2b-8369-35cf14716af3-swiftconf\") on node \"crc\" DevicePath \"\"" Dec 03 12:23:16 crc kubenswrapper[5052]: I1203 12:23:16.630473 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-ring-rebalance-b9ssr" event={"ID":"46e92959-0fd2-4e2b-8369-35cf14716af3","Type":"ContainerDied","Data":"b1e5ab942f0946bc4bd0a7347a43802032dcabcb1b83787d4c5960008f7e5624"} Dec 03 12:23:16 crc kubenswrapper[5052]: I1203 12:23:16.630534 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b1e5ab942f0946bc4bd0a7347a43802032dcabcb1b83787d4c5960008f7e5624" Dec 03 12:23:16 crc kubenswrapper[5052]: I1203 12:23:16.630542 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-ring-rebalance-b9ssr" Dec 03 12:23:17 crc kubenswrapper[5052]: I1203 12:23:17.004398 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/7f163259-845f-4519-bd77-fa66e760d4db-etc-swift\") pod \"swift-storage-0\" (UID: \"7f163259-845f-4519-bd77-fa66e760d4db\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:23:17 crc kubenswrapper[5052]: I1203 12:23:17.023433 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/7f163259-845f-4519-bd77-fa66e760d4db-etc-swift\") pod \"swift-storage-0\" (UID: \"7f163259-845f-4519-bd77-fa66e760d4db\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:23:17 crc kubenswrapper[5052]: I1203 12:23:17.124230 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:23:17 crc kubenswrapper[5052]: I1203 12:23:17.570148 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/swift-storage-0"] Dec 03 12:23:17 crc kubenswrapper[5052]: W1203 12:23:17.574927 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7f163259_845f_4519_bd77_fa66e760d4db.slice/crio-259d9a539fb019903f69a9ddb058a882b6fb34d0e95681ae3f632cb3a65a428a WatchSource:0}: Error finding container 259d9a539fb019903f69a9ddb058a882b6fb34d0e95681ae3f632cb3a65a428a: Status 404 returned error can't find the container with id 259d9a539fb019903f69a9ddb058a882b6fb34d0e95681ae3f632cb3a65a428a Dec 03 12:23:17 crc kubenswrapper[5052]: I1203 12:23:17.642746 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"7f163259-845f-4519-bd77-fa66e760d4db","Type":"ContainerStarted","Data":"259d9a539fb019903f69a9ddb058a882b6fb34d0e95681ae3f632cb3a65a428a"} Dec 03 12:23:18 crc kubenswrapper[5052]: I1203 12:23:18.659808 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"7f163259-845f-4519-bd77-fa66e760d4db","Type":"ContainerStarted","Data":"846159aaedd764c9a39adccf404c36e1798796b1dc058a4077567aead1622992"} Dec 03 12:23:18 crc kubenswrapper[5052]: I1203 12:23:18.660184 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"7f163259-845f-4519-bd77-fa66e760d4db","Type":"ContainerStarted","Data":"c546226444d76b3bae4019c590f2db31d0aa8aa1810311506bf0679abe0ac7da"} Dec 03 12:23:18 crc kubenswrapper[5052]: I1203 12:23:18.660205 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"7f163259-845f-4519-bd77-fa66e760d4db","Type":"ContainerStarted","Data":"c021f6fefe53bade62d592029616b253503b6690e220f6de008a4d7e297826e5"} Dec 03 12:23:18 crc kubenswrapper[5052]: I1203 12:23:18.660224 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"7f163259-845f-4519-bd77-fa66e760d4db","Type":"ContainerStarted","Data":"9818644c3d189b7752f2c0e1f85147e5450c62440f231eba257e3c2c1869464e"} Dec 03 12:23:18 crc kubenswrapper[5052]: I1203 12:23:18.660235 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"7f163259-845f-4519-bd77-fa66e760d4db","Type":"ContainerStarted","Data":"7ce328ddacfb675b5ded2a8547af6ba9940a95baba352f048c9c9dce642c7103"} Dec 03 12:23:18 crc kubenswrapper[5052]: I1203 12:23:18.660250 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"7f163259-845f-4519-bd77-fa66e760d4db","Type":"ContainerStarted","Data":"7f52642c73f37ab9180012c54a5c7e03d2db022e02528a07f0120cdace4c30ed"} Dec 03 12:23:19 crc kubenswrapper[5052]: I1203 12:23:19.683311 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"7f163259-845f-4519-bd77-fa66e760d4db","Type":"ContainerStarted","Data":"c31f1fc401b9b5f1678ef68c3dacb95ce5a0a4803fb016c6e372e24171421a1a"} Dec 03 12:23:19 crc kubenswrapper[5052]: I1203 12:23:19.684926 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"7f163259-845f-4519-bd77-fa66e760d4db","Type":"ContainerStarted","Data":"2de4cfddd3456322e2e08527a0b2b4ae7a715efc41cbfbeb5f1a6fa98e9a5409"} Dec 03 12:23:19 crc kubenswrapper[5052]: I1203 12:23:19.685048 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"7f163259-845f-4519-bd77-fa66e760d4db","Type":"ContainerStarted","Data":"17ba623ea2c89f44d8458d92140c07db44a1f67ae586efa6a1261de7ddb83194"} Dec 03 12:23:19 crc kubenswrapper[5052]: I1203 12:23:19.685122 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"7f163259-845f-4519-bd77-fa66e760d4db","Type":"ContainerStarted","Data":"dfccd3db9b2af1521bf2032a601fa4662b4414697ee02def0c5cd28e2c96cb26"} Dec 03 12:23:19 crc kubenswrapper[5052]: I1203 12:23:19.685202 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"7f163259-845f-4519-bd77-fa66e760d4db","Type":"ContainerStarted","Data":"85d94c44803aac08a74f194f52b8757506853bfa45354a4ed0a41234617514b0"} Dec 03 12:23:19 crc kubenswrapper[5052]: I1203 12:23:19.715026 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/glance-db-sync-5d6m8"] Dec 03 12:23:19 crc kubenswrapper[5052]: E1203 12:23:19.715332 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="fa334119-b1fd-42e8-b64b-a276ef89e4b5" containerName="registry-server" Dec 03 12:23:19 crc kubenswrapper[5052]: I1203 12:23:19.715348 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="fa334119-b1fd-42e8-b64b-a276ef89e4b5" containerName="registry-server" Dec 03 12:23:19 crc kubenswrapper[5052]: E1203 12:23:19.715364 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="bd34573a-a034-4c22-8df3-bab80ed70f45" containerName="mariadb-account-create-update" Dec 03 12:23:19 crc kubenswrapper[5052]: I1203 12:23:19.715370 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd34573a-a034-4c22-8df3-bab80ed70f45" containerName="mariadb-account-create-update" Dec 03 12:23:19 crc kubenswrapper[5052]: E1203 12:23:19.715383 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="46e92959-0fd2-4e2b-8369-35cf14716af3" containerName="swift-ring-rebalance" Dec 03 12:23:19 crc kubenswrapper[5052]: I1203 12:23:19.715390 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="46e92959-0fd2-4e2b-8369-35cf14716af3" containerName="swift-ring-rebalance" Dec 03 12:23:19 crc kubenswrapper[5052]: E1203 12:23:19.715401 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="fa334119-b1fd-42e8-b64b-a276ef89e4b5" containerName="extract-utilities" Dec 03 12:23:19 crc kubenswrapper[5052]: I1203 12:23:19.715406 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="fa334119-b1fd-42e8-b64b-a276ef89e4b5" containerName="extract-utilities" Dec 03 12:23:19 crc kubenswrapper[5052]: E1203 12:23:19.715418 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="f3c78563-0630-4849-85ae-8b5d5ad6795c" containerName="mariadb-database-create" Dec 03 12:23:19 crc kubenswrapper[5052]: I1203 12:23:19.715423 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="f3c78563-0630-4849-85ae-8b5d5ad6795c" containerName="mariadb-database-create" Dec 03 12:23:19 crc kubenswrapper[5052]: E1203 12:23:19.715432 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="60ba2a3e-26ac-4265-b206-cb9ea4c88584" containerName="mariadb-account-create-update" Dec 03 12:23:19 crc kubenswrapper[5052]: I1203 12:23:19.715438 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="60ba2a3e-26ac-4265-b206-cb9ea4c88584" containerName="mariadb-account-create-update" Dec 03 12:23:19 crc kubenswrapper[5052]: E1203 12:23:19.715455 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="beb9c625-5acb-4d1e-9044-7fb5287e8cfe" containerName="mariadb-database-create" Dec 03 12:23:19 crc kubenswrapper[5052]: I1203 12:23:19.715461 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="beb9c625-5acb-4d1e-9044-7fb5287e8cfe" containerName="mariadb-database-create" Dec 03 12:23:19 crc kubenswrapper[5052]: E1203 12:23:19.715473 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="4ea65f95-6ea7-4362-a361-514ace4fbc44" containerName="mariadb-database-create" Dec 03 12:23:19 crc kubenswrapper[5052]: I1203 12:23:19.715479 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="4ea65f95-6ea7-4362-a361-514ace4fbc44" containerName="mariadb-database-create" Dec 03 12:23:19 crc kubenswrapper[5052]: E1203 12:23:19.715487 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="37da25a2-1fa7-4a14-b8d6-1ba47412c6cd" containerName="mariadb-account-create-update" Dec 03 12:23:19 crc kubenswrapper[5052]: I1203 12:23:19.715492 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="37da25a2-1fa7-4a14-b8d6-1ba47412c6cd" containerName="mariadb-account-create-update" Dec 03 12:23:19 crc kubenswrapper[5052]: E1203 12:23:19.715502 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="fa334119-b1fd-42e8-b64b-a276ef89e4b5" containerName="extract-content" Dec 03 12:23:19 crc kubenswrapper[5052]: I1203 12:23:19.715520 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="fa334119-b1fd-42e8-b64b-a276ef89e4b5" containerName="extract-content" Dec 03 12:23:19 crc kubenswrapper[5052]: I1203 12:23:19.715666 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="46e92959-0fd2-4e2b-8369-35cf14716af3" containerName="swift-ring-rebalance" Dec 03 12:23:19 crc kubenswrapper[5052]: I1203 12:23:19.715682 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="60ba2a3e-26ac-4265-b206-cb9ea4c88584" containerName="mariadb-account-create-update" Dec 03 12:23:19 crc kubenswrapper[5052]: I1203 12:23:19.715691 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="fa334119-b1fd-42e8-b64b-a276ef89e4b5" containerName="registry-server" Dec 03 12:23:19 crc kubenswrapper[5052]: I1203 12:23:19.715698 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="beb9c625-5acb-4d1e-9044-7fb5287e8cfe" containerName="mariadb-database-create" Dec 03 12:23:19 crc kubenswrapper[5052]: I1203 12:23:19.715713 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="f3c78563-0630-4849-85ae-8b5d5ad6795c" containerName="mariadb-database-create" Dec 03 12:23:19 crc kubenswrapper[5052]: I1203 12:23:19.715725 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="bd34573a-a034-4c22-8df3-bab80ed70f45" containerName="mariadb-account-create-update" Dec 03 12:23:19 crc kubenswrapper[5052]: I1203 12:23:19.715730 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="37da25a2-1fa7-4a14-b8d6-1ba47412c6cd" containerName="mariadb-account-create-update" Dec 03 12:23:19 crc kubenswrapper[5052]: I1203 12:23:19.715740 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="4ea65f95-6ea7-4362-a361-514ace4fbc44" containerName="mariadb-database-create" Dec 03 12:23:19 crc kubenswrapper[5052]: I1203 12:23:19.716259 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-db-sync-5d6m8" Dec 03 12:23:19 crc kubenswrapper[5052]: I1203 12:23:19.725836 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-config-data" Dec 03 12:23:19 crc kubenswrapper[5052]: I1203 12:23:19.725910 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-glance-dockercfg-cd9ns" Dec 03 12:23:19 crc kubenswrapper[5052]: I1203 12:23:19.744692 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-db-sync-5d6m8"] Dec 03 12:23:19 crc kubenswrapper[5052]: I1203 12:23:19.858769 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/b5a18429-60fa-4407-acb1-74e03616c939-config-data\") pod \"glance-db-sync-5d6m8\" (UID: \"b5a18429-60fa-4407-acb1-74e03616c939\") " pod="openstack-kuttl-tests/glance-db-sync-5d6m8" Dec 03 12:23:19 crc kubenswrapper[5052]: I1203 12:23:19.858985 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b5a18429-60fa-4407-acb1-74e03616c939-combined-ca-bundle\") pod \"glance-db-sync-5d6m8\" (UID: \"b5a18429-60fa-4407-acb1-74e03616c939\") " pod="openstack-kuttl-tests/glance-db-sync-5d6m8" Dec 03 12:23:19 crc kubenswrapper[5052]: I1203 12:23:19.859097 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8ct5w\" (UniqueName: \"kubernetes.io/projected/b5a18429-60fa-4407-acb1-74e03616c939-kube-api-access-8ct5w\") pod \"glance-db-sync-5d6m8\" (UID: \"b5a18429-60fa-4407-acb1-74e03616c939\") " pod="openstack-kuttl-tests/glance-db-sync-5d6m8" Dec 03 12:23:19 crc kubenswrapper[5052]: I1203 12:23:19.859147 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/b5a18429-60fa-4407-acb1-74e03616c939-db-sync-config-data\") pod \"glance-db-sync-5d6m8\" (UID: \"b5a18429-60fa-4407-acb1-74e03616c939\") " pod="openstack-kuttl-tests/glance-db-sync-5d6m8" Dec 03 12:23:19 crc kubenswrapper[5052]: I1203 12:23:19.959943 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b5a18429-60fa-4407-acb1-74e03616c939-combined-ca-bundle\") pod \"glance-db-sync-5d6m8\" (UID: \"b5a18429-60fa-4407-acb1-74e03616c939\") " pod="openstack-kuttl-tests/glance-db-sync-5d6m8" Dec 03 12:23:19 crc kubenswrapper[5052]: I1203 12:23:19.960013 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-8ct5w\" (UniqueName: \"kubernetes.io/projected/b5a18429-60fa-4407-acb1-74e03616c939-kube-api-access-8ct5w\") pod \"glance-db-sync-5d6m8\" (UID: \"b5a18429-60fa-4407-acb1-74e03616c939\") " pod="openstack-kuttl-tests/glance-db-sync-5d6m8" Dec 03 12:23:19 crc kubenswrapper[5052]: I1203 12:23:19.960042 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/b5a18429-60fa-4407-acb1-74e03616c939-db-sync-config-data\") pod \"glance-db-sync-5d6m8\" (UID: \"b5a18429-60fa-4407-acb1-74e03616c939\") " pod="openstack-kuttl-tests/glance-db-sync-5d6m8" Dec 03 12:23:19 crc kubenswrapper[5052]: I1203 12:23:19.960097 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/b5a18429-60fa-4407-acb1-74e03616c939-config-data\") pod \"glance-db-sync-5d6m8\" (UID: \"b5a18429-60fa-4407-acb1-74e03616c939\") " pod="openstack-kuttl-tests/glance-db-sync-5d6m8" Dec 03 12:23:20 crc kubenswrapper[5052]: I1203 12:23:20.024552 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/b5a18429-60fa-4407-acb1-74e03616c939-db-sync-config-data\") pod \"glance-db-sync-5d6m8\" (UID: \"b5a18429-60fa-4407-acb1-74e03616c939\") " pod="openstack-kuttl-tests/glance-db-sync-5d6m8" Dec 03 12:23:20 crc kubenswrapper[5052]: I1203 12:23:20.024624 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/b5a18429-60fa-4407-acb1-74e03616c939-config-data\") pod \"glance-db-sync-5d6m8\" (UID: \"b5a18429-60fa-4407-acb1-74e03616c939\") " pod="openstack-kuttl-tests/glance-db-sync-5d6m8" Dec 03 12:23:20 crc kubenswrapper[5052]: I1203 12:23:20.024649 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b5a18429-60fa-4407-acb1-74e03616c939-combined-ca-bundle\") pod \"glance-db-sync-5d6m8\" (UID: \"b5a18429-60fa-4407-acb1-74e03616c939\") " pod="openstack-kuttl-tests/glance-db-sync-5d6m8" Dec 03 12:23:20 crc kubenswrapper[5052]: I1203 12:23:20.036450 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-8ct5w\" (UniqueName: \"kubernetes.io/projected/b5a18429-60fa-4407-acb1-74e03616c939-kube-api-access-8ct5w\") pod \"glance-db-sync-5d6m8\" (UID: \"b5a18429-60fa-4407-acb1-74e03616c939\") " pod="openstack-kuttl-tests/glance-db-sync-5d6m8" Dec 03 12:23:20 crc kubenswrapper[5052]: I1203 12:23:20.087936 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-db-sync-5d6m8" Dec 03 12:23:20 crc kubenswrapper[5052]: I1203 12:23:20.620346 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-db-sync-5d6m8"] Dec 03 12:23:20 crc kubenswrapper[5052]: W1203 12:23:20.629699 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb5a18429_60fa_4407_acb1_74e03616c939.slice/crio-ec44bac9be49cedad7b1d82fa5f7242ab866cdc0ae1431445e8c4b082bcdafc7 WatchSource:0}: Error finding container ec44bac9be49cedad7b1d82fa5f7242ab866cdc0ae1431445e8c4b082bcdafc7: Status 404 returned error can't find the container with id ec44bac9be49cedad7b1d82fa5f7242ab866cdc0ae1431445e8c4b082bcdafc7 Dec 03 12:23:20 crc kubenswrapper[5052]: I1203 12:23:20.696575 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"7f163259-845f-4519-bd77-fa66e760d4db","Type":"ContainerStarted","Data":"1127b9f54fde360052d578fc34e3f1ed816a67a8fd6f335d02508fa144bf6dc7"} Dec 03 12:23:20 crc kubenswrapper[5052]: I1203 12:23:20.696634 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"7f163259-845f-4519-bd77-fa66e760d4db","Type":"ContainerStarted","Data":"ba5a838f8abec837caa1abd8f6365324f2f4067f3e37755f33f8a119410d9c81"} Dec 03 12:23:20 crc kubenswrapper[5052]: I1203 12:23:20.699025 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-db-sync-5d6m8" event={"ID":"b5a18429-60fa-4407-acb1-74e03616c939","Type":"ContainerStarted","Data":"ec44bac9be49cedad7b1d82fa5f7242ab866cdc0ae1431445e8c4b082bcdafc7"} Dec 03 12:23:20 crc kubenswrapper[5052]: I1203 12:23:20.708306 5052 generic.go:334] "Generic (PLEG): container finished" podID="cd7b6179-ba41-43bd-bc95-490a88aa15fd" containerID="c9884448416314162e49f4564ffad485559e0dcde7274fbe0abad7844eeaf00d" exitCode=0 Dec 03 12:23:20 crc kubenswrapper[5052]: I1203 12:23:20.708382 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-server-0" event={"ID":"cd7b6179-ba41-43bd-bc95-490a88aa15fd","Type":"ContainerDied","Data":"c9884448416314162e49f4564ffad485559e0dcde7274fbe0abad7844eeaf00d"} Dec 03 12:23:20 crc kubenswrapper[5052]: I1203 12:23:20.715054 5052 generic.go:334] "Generic (PLEG): container finished" podID="5a5320b7-0541-483a-8e0b-78c10d0f8efa" containerID="6c115b6a73d88d94535850532bd7a63e133ef983572d338d54582ff166612eba" exitCode=0 Dec 03 12:23:20 crc kubenswrapper[5052]: I1203 12:23:20.715107 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" event={"ID":"5a5320b7-0541-483a-8e0b-78c10d0f8efa","Type":"ContainerDied","Data":"6c115b6a73d88d94535850532bd7a63e133ef983572d338d54582ff166612eba"} Dec 03 12:23:21 crc kubenswrapper[5052]: I1203 12:23:21.729404 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-server-0" event={"ID":"cd7b6179-ba41-43bd-bc95-490a88aa15fd","Type":"ContainerStarted","Data":"229c6c7d07d677ea52e66801e341130c260352a54816a6ceeecbd91e2597db8c"} Dec 03 12:23:21 crc kubenswrapper[5052]: I1203 12:23:21.730009 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:23:21 crc kubenswrapper[5052]: I1203 12:23:21.731997 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" event={"ID":"5a5320b7-0541-483a-8e0b-78c10d0f8efa","Type":"ContainerStarted","Data":"032bc0ff1bc0c7adb6371d41b7d1d30531bb08512a12516306ce34a00afcaad9"} Dec 03 12:23:21 crc kubenswrapper[5052]: I1203 12:23:21.732220 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:23:21 crc kubenswrapper[5052]: I1203 12:23:21.738029 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"7f163259-845f-4519-bd77-fa66e760d4db","Type":"ContainerStarted","Data":"b20fcc98d22748a4865b1aff97aab861964ca72f2da839df89a8240768170b98"} Dec 03 12:23:21 crc kubenswrapper[5052]: I1203 12:23:21.738070 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"7f163259-845f-4519-bd77-fa66e760d4db","Type":"ContainerStarted","Data":"1094f3eb2af0052222cc7befb863b42966285f5596b1be360edb3c5e5e097a14"} Dec 03 12:23:21 crc kubenswrapper[5052]: I1203 12:23:21.739464 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-db-sync-5d6m8" event={"ID":"b5a18429-60fa-4407-acb1-74e03616c939","Type":"ContainerStarted","Data":"c1d51e836a79bfee8be38357907cb5f002159100bca556fe09a10fe835ba32b6"} Dec 03 12:23:21 crc kubenswrapper[5052]: I1203 12:23:21.752347 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/rabbitmq-server-0" podStartSLOduration=36.752329628 podStartE2EDuration="36.752329628s" podCreationTimestamp="2025-12-03 12:22:45 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:23:21.749029614 +0000 UTC m=+3792.337271378" watchObservedRunningTime="2025-12-03 12:23:21.752329628 +0000 UTC m=+3792.340571402" Dec 03 12:23:21 crc kubenswrapper[5052]: I1203 12:23:21.797112 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" podStartSLOduration=36.797060384 podStartE2EDuration="36.797060384s" podCreationTimestamp="2025-12-03 12:22:45 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:23:21.774753468 +0000 UTC m=+3792.362995252" watchObservedRunningTime="2025-12-03 12:23:21.797060384 +0000 UTC m=+3792.385302138" Dec 03 12:23:21 crc kubenswrapper[5052]: I1203 12:23:21.817927 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/swift-storage-0" podStartSLOduration=22.817902519 podStartE2EDuration="22.817902519s" podCreationTimestamp="2025-12-03 12:22:59 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:23:21.809889751 +0000 UTC m=+3792.398131515" watchObservedRunningTime="2025-12-03 12:23:21.817902519 +0000 UTC m=+3792.406144283" Dec 03 12:23:21 crc kubenswrapper[5052]: I1203 12:23:21.831812 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/glance-db-sync-5d6m8" podStartSLOduration=2.831788256 podStartE2EDuration="2.831788256s" podCreationTimestamp="2025-12-03 12:23:19 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:23:21.825162977 +0000 UTC m=+3792.413404751" watchObservedRunningTime="2025-12-03 12:23:21.831788256 +0000 UTC m=+3792.420030020" Dec 03 12:23:21 crc kubenswrapper[5052]: I1203 12:23:21.981505 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-7d9f6cd5d5-xcx8v"] Dec 03 12:23:21 crc kubenswrapper[5052]: I1203 12:23:21.984427 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-7d9f6cd5d5-xcx8v" Dec 03 12:23:21 crc kubenswrapper[5052]: I1203 12:23:21.991398 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"dns-swift-storage-0" Dec 03 12:23:22 crc kubenswrapper[5052]: I1203 12:23:22.002967 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-7d9f6cd5d5-xcx8v"] Dec 03 12:23:22 crc kubenswrapper[5052]: I1203 12:23:22.096124 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nb42t\" (UniqueName: \"kubernetes.io/projected/bda2a6a4-1675-4341-963d-d088b142e4da-kube-api-access-nb42t\") pod \"dnsmasq-dnsmasq-7d9f6cd5d5-xcx8v\" (UID: \"bda2a6a4-1675-4341-963d-d088b142e4da\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-7d9f6cd5d5-xcx8v" Dec 03 12:23:22 crc kubenswrapper[5052]: I1203 12:23:22.096283 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/bda2a6a4-1675-4341-963d-d088b142e4da-config\") pod \"dnsmasq-dnsmasq-7d9f6cd5d5-xcx8v\" (UID: \"bda2a6a4-1675-4341-963d-d088b142e4da\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-7d9f6cd5d5-xcx8v" Dec 03 12:23:22 crc kubenswrapper[5052]: I1203 12:23:22.096667 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/bda2a6a4-1675-4341-963d-d088b142e4da-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-7d9f6cd5d5-xcx8v\" (UID: \"bda2a6a4-1675-4341-963d-d088b142e4da\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-7d9f6cd5d5-xcx8v" Dec 03 12:23:22 crc kubenswrapper[5052]: I1203 12:23:22.096706 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dns-swift-storage-0\" (UniqueName: \"kubernetes.io/configmap/bda2a6a4-1675-4341-963d-d088b142e4da-dns-swift-storage-0\") pod \"dnsmasq-dnsmasq-7d9f6cd5d5-xcx8v\" (UID: \"bda2a6a4-1675-4341-963d-d088b142e4da\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-7d9f6cd5d5-xcx8v" Dec 03 12:23:22 crc kubenswrapper[5052]: I1203 12:23:22.198724 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/bda2a6a4-1675-4341-963d-d088b142e4da-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-7d9f6cd5d5-xcx8v\" (UID: \"bda2a6a4-1675-4341-963d-d088b142e4da\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-7d9f6cd5d5-xcx8v" Dec 03 12:23:22 crc kubenswrapper[5052]: I1203 12:23:22.198771 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"dns-swift-storage-0\" (UniqueName: \"kubernetes.io/configmap/bda2a6a4-1675-4341-963d-d088b142e4da-dns-swift-storage-0\") pod \"dnsmasq-dnsmasq-7d9f6cd5d5-xcx8v\" (UID: \"bda2a6a4-1675-4341-963d-d088b142e4da\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-7d9f6cd5d5-xcx8v" Dec 03 12:23:22 crc kubenswrapper[5052]: I1203 12:23:22.198819 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-nb42t\" (UniqueName: \"kubernetes.io/projected/bda2a6a4-1675-4341-963d-d088b142e4da-kube-api-access-nb42t\") pod \"dnsmasq-dnsmasq-7d9f6cd5d5-xcx8v\" (UID: \"bda2a6a4-1675-4341-963d-d088b142e4da\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-7d9f6cd5d5-xcx8v" Dec 03 12:23:22 crc kubenswrapper[5052]: I1203 12:23:22.198851 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/bda2a6a4-1675-4341-963d-d088b142e4da-config\") pod \"dnsmasq-dnsmasq-7d9f6cd5d5-xcx8v\" (UID: \"bda2a6a4-1675-4341-963d-d088b142e4da\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-7d9f6cd5d5-xcx8v" Dec 03 12:23:22 crc kubenswrapper[5052]: I1203 12:23:22.199493 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/bda2a6a4-1675-4341-963d-d088b142e4da-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-7d9f6cd5d5-xcx8v\" (UID: \"bda2a6a4-1675-4341-963d-d088b142e4da\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-7d9f6cd5d5-xcx8v" Dec 03 12:23:22 crc kubenswrapper[5052]: I1203 12:23:22.199670 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/bda2a6a4-1675-4341-963d-d088b142e4da-config\") pod \"dnsmasq-dnsmasq-7d9f6cd5d5-xcx8v\" (UID: \"bda2a6a4-1675-4341-963d-d088b142e4da\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-7d9f6cd5d5-xcx8v" Dec 03 12:23:22 crc kubenswrapper[5052]: I1203 12:23:22.199946 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"dns-swift-storage-0\" (UniqueName: \"kubernetes.io/configmap/bda2a6a4-1675-4341-963d-d088b142e4da-dns-swift-storage-0\") pod \"dnsmasq-dnsmasq-7d9f6cd5d5-xcx8v\" (UID: \"bda2a6a4-1675-4341-963d-d088b142e4da\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-7d9f6cd5d5-xcx8v" Dec 03 12:23:22 crc kubenswrapper[5052]: I1203 12:23:22.223441 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-nb42t\" (UniqueName: \"kubernetes.io/projected/bda2a6a4-1675-4341-963d-d088b142e4da-kube-api-access-nb42t\") pod \"dnsmasq-dnsmasq-7d9f6cd5d5-xcx8v\" (UID: \"bda2a6a4-1675-4341-963d-d088b142e4da\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-7d9f6cd5d5-xcx8v" Dec 03 12:23:22 crc kubenswrapper[5052]: I1203 12:23:22.303636 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-7d9f6cd5d5-xcx8v" Dec 03 12:23:22 crc kubenswrapper[5052]: W1203 12:23:22.858991 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbda2a6a4_1675_4341_963d_d088b142e4da.slice/crio-e96c22e5c7c486e968c8035a9d043ba4a64d6ced72e8734240245ab7f0643506 WatchSource:0}: Error finding container e96c22e5c7c486e968c8035a9d043ba4a64d6ced72e8734240245ab7f0643506: Status 404 returned error can't find the container with id e96c22e5c7c486e968c8035a9d043ba4a64d6ced72e8734240245ab7f0643506 Dec 03 12:23:22 crc kubenswrapper[5052]: I1203 12:23:22.866787 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-7d9f6cd5d5-xcx8v"] Dec 03 12:23:23 crc kubenswrapper[5052]: I1203 12:23:23.778679 5052 generic.go:334] "Generic (PLEG): container finished" podID="bda2a6a4-1675-4341-963d-d088b142e4da" containerID="1b7cad63063d7371ba6f8f987a89264c0b0086dac46b64684e09fcd1ef4338eb" exitCode=0 Dec 03 12:23:23 crc kubenswrapper[5052]: I1203 12:23:23.778988 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-7d9f6cd5d5-xcx8v" event={"ID":"bda2a6a4-1675-4341-963d-d088b142e4da","Type":"ContainerDied","Data":"1b7cad63063d7371ba6f8f987a89264c0b0086dac46b64684e09fcd1ef4338eb"} Dec 03 12:23:23 crc kubenswrapper[5052]: I1203 12:23:23.779012 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-7d9f6cd5d5-xcx8v" event={"ID":"bda2a6a4-1675-4341-963d-d088b142e4da","Type":"ContainerStarted","Data":"e96c22e5c7c486e968c8035a9d043ba4a64d6ced72e8734240245ab7f0643506"} Dec 03 12:23:24 crc kubenswrapper[5052]: I1203 12:23:24.789495 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-7d9f6cd5d5-xcx8v" event={"ID":"bda2a6a4-1675-4341-963d-d088b142e4da","Type":"ContainerStarted","Data":"2727160f7feae9f174c834d6bd19e8c49863bfda2ef0d99a1bff7a42fbad1f97"} Dec 03 12:23:24 crc kubenswrapper[5052]: I1203 12:23:24.789865 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-7d9f6cd5d5-xcx8v" Dec 03 12:23:24 crc kubenswrapper[5052]: I1203 12:23:24.808685 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-7d9f6cd5d5-xcx8v" podStartSLOduration=3.808666112 podStartE2EDuration="3.808666112s" podCreationTimestamp="2025-12-03 12:23:21 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:23:24.804368659 +0000 UTC m=+3795.392610423" watchObservedRunningTime="2025-12-03 12:23:24.808666112 +0000 UTC m=+3795.396907886" Dec 03 12:23:25 crc kubenswrapper[5052]: I1203 12:23:25.673437 5052 scope.go:117] "RemoveContainer" containerID="09b4a0072741f60499d24f40cc4ff3406d5d22671ced431363ab865172786300" Dec 03 12:23:25 crc kubenswrapper[5052]: E1203 12:23:25.673781 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 12:23:29 crc kubenswrapper[5052]: I1203 12:23:29.836664 5052 generic.go:334] "Generic (PLEG): container finished" podID="b5a18429-60fa-4407-acb1-74e03616c939" containerID="c1d51e836a79bfee8be38357907cb5f002159100bca556fe09a10fe835ba32b6" exitCode=0 Dec 03 12:23:29 crc kubenswrapper[5052]: I1203 12:23:29.837160 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-db-sync-5d6m8" event={"ID":"b5a18429-60fa-4407-acb1-74e03616c939","Type":"ContainerDied","Data":"c1d51e836a79bfee8be38357907cb5f002159100bca556fe09a10fe835ba32b6"} Dec 03 12:23:31 crc kubenswrapper[5052]: I1203 12:23:31.504578 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-db-sync-5d6m8" Dec 03 12:23:31 crc kubenswrapper[5052]: I1203 12:23:31.643363 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8ct5w\" (UniqueName: \"kubernetes.io/projected/b5a18429-60fa-4407-acb1-74e03616c939-kube-api-access-8ct5w\") pod \"b5a18429-60fa-4407-acb1-74e03616c939\" (UID: \"b5a18429-60fa-4407-acb1-74e03616c939\") " Dec 03 12:23:31 crc kubenswrapper[5052]: I1203 12:23:31.643770 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/b5a18429-60fa-4407-acb1-74e03616c939-config-data\") pod \"b5a18429-60fa-4407-acb1-74e03616c939\" (UID: \"b5a18429-60fa-4407-acb1-74e03616c939\") " Dec 03 12:23:31 crc kubenswrapper[5052]: I1203 12:23:31.643800 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/b5a18429-60fa-4407-acb1-74e03616c939-db-sync-config-data\") pod \"b5a18429-60fa-4407-acb1-74e03616c939\" (UID: \"b5a18429-60fa-4407-acb1-74e03616c939\") " Dec 03 12:23:31 crc kubenswrapper[5052]: I1203 12:23:31.643852 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b5a18429-60fa-4407-acb1-74e03616c939-combined-ca-bundle\") pod \"b5a18429-60fa-4407-acb1-74e03616c939\" (UID: \"b5a18429-60fa-4407-acb1-74e03616c939\") " Dec 03 12:23:31 crc kubenswrapper[5052]: I1203 12:23:31.649829 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b5a18429-60fa-4407-acb1-74e03616c939-db-sync-config-data" (OuterVolumeSpecName: "db-sync-config-data") pod "b5a18429-60fa-4407-acb1-74e03616c939" (UID: "b5a18429-60fa-4407-acb1-74e03616c939"). InnerVolumeSpecName "db-sync-config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:23:31 crc kubenswrapper[5052]: I1203 12:23:31.651934 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b5a18429-60fa-4407-acb1-74e03616c939-kube-api-access-8ct5w" (OuterVolumeSpecName: "kube-api-access-8ct5w") pod "b5a18429-60fa-4407-acb1-74e03616c939" (UID: "b5a18429-60fa-4407-acb1-74e03616c939"). InnerVolumeSpecName "kube-api-access-8ct5w". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:23:31 crc kubenswrapper[5052]: I1203 12:23:31.669093 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b5a18429-60fa-4407-acb1-74e03616c939-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "b5a18429-60fa-4407-acb1-74e03616c939" (UID: "b5a18429-60fa-4407-acb1-74e03616c939"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:23:31 crc kubenswrapper[5052]: I1203 12:23:31.692103 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b5a18429-60fa-4407-acb1-74e03616c939-config-data" (OuterVolumeSpecName: "config-data") pod "b5a18429-60fa-4407-acb1-74e03616c939" (UID: "b5a18429-60fa-4407-acb1-74e03616c939"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:23:31 crc kubenswrapper[5052]: I1203 12:23:31.745362 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-8ct5w\" (UniqueName: \"kubernetes.io/projected/b5a18429-60fa-4407-acb1-74e03616c939-kube-api-access-8ct5w\") on node \"crc\" DevicePath \"\"" Dec 03 12:23:31 crc kubenswrapper[5052]: I1203 12:23:31.745404 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/b5a18429-60fa-4407-acb1-74e03616c939-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:23:31 crc kubenswrapper[5052]: I1203 12:23:31.745415 5052 reconciler_common.go:293] "Volume detached for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/b5a18429-60fa-4407-acb1-74e03616c939-db-sync-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:23:31 crc kubenswrapper[5052]: I1203 12:23:31.745423 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b5a18429-60fa-4407-acb1-74e03616c939-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:23:31 crc kubenswrapper[5052]: I1203 12:23:31.861837 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-db-sync-5d6m8" event={"ID":"b5a18429-60fa-4407-acb1-74e03616c939","Type":"ContainerDied","Data":"ec44bac9be49cedad7b1d82fa5f7242ab866cdc0ae1431445e8c4b082bcdafc7"} Dec 03 12:23:31 crc kubenswrapper[5052]: I1203 12:23:31.861898 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ec44bac9be49cedad7b1d82fa5f7242ab866cdc0ae1431445e8c4b082bcdafc7" Dec 03 12:23:31 crc kubenswrapper[5052]: I1203 12:23:31.862477 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-db-sync-5d6m8" Dec 03 12:23:32 crc kubenswrapper[5052]: I1203 12:23:32.305742 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-7d9f6cd5d5-xcx8v" Dec 03 12:23:32 crc kubenswrapper[5052]: I1203 12:23:32.384675 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-7mq7s"] Dec 03 12:23:32 crc kubenswrapper[5052]: I1203 12:23:32.384960 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-7mq7s" podUID="cdf67382-3145-4280-a5a5-41b0185cb07f" containerName="dnsmasq-dns" containerID="cri-o://302fc92c792a1400570841f42f510118b4e77899b5e6948147bf0cee46337588" gracePeriod=10 Dec 03 12:23:32 crc kubenswrapper[5052]: I1203 12:23:32.881904 5052 generic.go:334] "Generic (PLEG): container finished" podID="cdf67382-3145-4280-a5a5-41b0185cb07f" containerID="302fc92c792a1400570841f42f510118b4e77899b5e6948147bf0cee46337588" exitCode=0 Dec 03 12:23:32 crc kubenswrapper[5052]: I1203 12:23:32.881953 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-7mq7s" event={"ID":"cdf67382-3145-4280-a5a5-41b0185cb07f","Type":"ContainerDied","Data":"302fc92c792a1400570841f42f510118b4e77899b5e6948147bf0cee46337588"} Dec 03 12:23:33 crc kubenswrapper[5052]: I1203 12:23:33.129920 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-7mq7s" Dec 03 12:23:33 crc kubenswrapper[5052]: I1203 12:23:33.287260 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/cdf67382-3145-4280-a5a5-41b0185cb07f-config\") pod \"cdf67382-3145-4280-a5a5-41b0185cb07f\" (UID: \"cdf67382-3145-4280-a5a5-41b0185cb07f\") " Dec 03 12:23:33 crc kubenswrapper[5052]: I1203 12:23:33.287348 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/cdf67382-3145-4280-a5a5-41b0185cb07f-dnsmasq-svc\") pod \"cdf67382-3145-4280-a5a5-41b0185cb07f\" (UID: \"cdf67382-3145-4280-a5a5-41b0185cb07f\") " Dec 03 12:23:33 crc kubenswrapper[5052]: I1203 12:23:33.287543 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nkzwn\" (UniqueName: \"kubernetes.io/projected/cdf67382-3145-4280-a5a5-41b0185cb07f-kube-api-access-nkzwn\") pod \"cdf67382-3145-4280-a5a5-41b0185cb07f\" (UID: \"cdf67382-3145-4280-a5a5-41b0185cb07f\") " Dec 03 12:23:33 crc kubenswrapper[5052]: I1203 12:23:33.293133 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cdf67382-3145-4280-a5a5-41b0185cb07f-kube-api-access-nkzwn" (OuterVolumeSpecName: "kube-api-access-nkzwn") pod "cdf67382-3145-4280-a5a5-41b0185cb07f" (UID: "cdf67382-3145-4280-a5a5-41b0185cb07f"). InnerVolumeSpecName "kube-api-access-nkzwn". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:23:33 crc kubenswrapper[5052]: I1203 12:23:33.331335 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cdf67382-3145-4280-a5a5-41b0185cb07f-dnsmasq-svc" (OuterVolumeSpecName: "dnsmasq-svc") pod "cdf67382-3145-4280-a5a5-41b0185cb07f" (UID: "cdf67382-3145-4280-a5a5-41b0185cb07f"). InnerVolumeSpecName "dnsmasq-svc". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:23:33 crc kubenswrapper[5052]: I1203 12:23:33.334416 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cdf67382-3145-4280-a5a5-41b0185cb07f-config" (OuterVolumeSpecName: "config") pod "cdf67382-3145-4280-a5a5-41b0185cb07f" (UID: "cdf67382-3145-4280-a5a5-41b0185cb07f"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:23:33 crc kubenswrapper[5052]: I1203 12:23:33.388840 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/cdf67382-3145-4280-a5a5-41b0185cb07f-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:23:33 crc kubenswrapper[5052]: I1203 12:23:33.388868 5052 reconciler_common.go:293] "Volume detached for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/cdf67382-3145-4280-a5a5-41b0185cb07f-dnsmasq-svc\") on node \"crc\" DevicePath \"\"" Dec 03 12:23:33 crc kubenswrapper[5052]: I1203 12:23:33.388880 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-nkzwn\" (UniqueName: \"kubernetes.io/projected/cdf67382-3145-4280-a5a5-41b0185cb07f-kube-api-access-nkzwn\") on node \"crc\" DevicePath \"\"" Dec 03 12:23:33 crc kubenswrapper[5052]: I1203 12:23:33.895459 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-7mq7s" event={"ID":"cdf67382-3145-4280-a5a5-41b0185cb07f","Type":"ContainerDied","Data":"3f984b732b7d3c63737bf25b10c931347b0989361ec2884621e57852bcae454a"} Dec 03 12:23:33 crc kubenswrapper[5052]: I1203 12:23:33.895520 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-7mq7s" Dec 03 12:23:33 crc kubenswrapper[5052]: I1203 12:23:33.895806 5052 scope.go:117] "RemoveContainer" containerID="302fc92c792a1400570841f42f510118b4e77899b5e6948147bf0cee46337588" Dec 03 12:23:33 crc kubenswrapper[5052]: I1203 12:23:33.930343 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-7mq7s"] Dec 03 12:23:33 crc kubenswrapper[5052]: I1203 12:23:33.930562 5052 scope.go:117] "RemoveContainer" containerID="ecdb88788429aa4445200013cd41a7db181ff4aef99f48234f3114a461f621bd" Dec 03 12:23:33 crc kubenswrapper[5052]: I1203 12:23:33.943480 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-7mq7s"] Dec 03 12:23:35 crc kubenswrapper[5052]: I1203 12:23:35.683709 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cdf67382-3145-4280-a5a5-41b0185cb07f" path="/var/lib/kubelet/pods/cdf67382-3145-4280-a5a5-41b0185cb07f/volumes" Dec 03 12:23:36 crc kubenswrapper[5052]: I1203 12:23:36.824422 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:23:37 crc kubenswrapper[5052]: I1203 12:23:37.082912 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/barbican-db-create-8h8nv"] Dec 03 12:23:37 crc kubenswrapper[5052]: E1203 12:23:37.083349 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="cdf67382-3145-4280-a5a5-41b0185cb07f" containerName="dnsmasq-dns" Dec 03 12:23:37 crc kubenswrapper[5052]: I1203 12:23:37.083372 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="cdf67382-3145-4280-a5a5-41b0185cb07f" containerName="dnsmasq-dns" Dec 03 12:23:37 crc kubenswrapper[5052]: E1203 12:23:37.083399 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="b5a18429-60fa-4407-acb1-74e03616c939" containerName="glance-db-sync" Dec 03 12:23:37 crc kubenswrapper[5052]: I1203 12:23:37.083411 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="b5a18429-60fa-4407-acb1-74e03616c939" containerName="glance-db-sync" Dec 03 12:23:37 crc kubenswrapper[5052]: E1203 12:23:37.083430 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="cdf67382-3145-4280-a5a5-41b0185cb07f" containerName="init" Dec 03 12:23:37 crc kubenswrapper[5052]: I1203 12:23:37.083439 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="cdf67382-3145-4280-a5a5-41b0185cb07f" containerName="init" Dec 03 12:23:37 crc kubenswrapper[5052]: I1203 12:23:37.083645 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="b5a18429-60fa-4407-acb1-74e03616c939" containerName="glance-db-sync" Dec 03 12:23:37 crc kubenswrapper[5052]: I1203 12:23:37.083676 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="cdf67382-3145-4280-a5a5-41b0185cb07f" containerName="dnsmasq-dns" Dec 03 12:23:37 crc kubenswrapper[5052]: I1203 12:23:37.084246 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-db-create-8h8nv" Dec 03 12:23:37 crc kubenswrapper[5052]: I1203 12:23:37.107929 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-db-create-8h8nv"] Dec 03 12:23:37 crc kubenswrapper[5052]: I1203 12:23:37.109755 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:23:37 crc kubenswrapper[5052]: I1203 12:23:37.246861 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/e01a4c75-3d41-4784-a957-681fc74f9f7d-operator-scripts\") pod \"barbican-db-create-8h8nv\" (UID: \"e01a4c75-3d41-4784-a957-681fc74f9f7d\") " pod="openstack-kuttl-tests/barbican-db-create-8h8nv" Dec 03 12:23:37 crc kubenswrapper[5052]: I1203 12:23:37.246931 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6bdqs\" (UniqueName: \"kubernetes.io/projected/e01a4c75-3d41-4784-a957-681fc74f9f7d-kube-api-access-6bdqs\") pod \"barbican-db-create-8h8nv\" (UID: \"e01a4c75-3d41-4784-a957-681fc74f9f7d\") " pod="openstack-kuttl-tests/barbican-db-create-8h8nv" Dec 03 12:23:37 crc kubenswrapper[5052]: I1203 12:23:37.288328 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/barbican-3d67-account-create-update-wmkzq"] Dec 03 12:23:37 crc kubenswrapper[5052]: I1203 12:23:37.289860 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-3d67-account-create-update-wmkzq" Dec 03 12:23:37 crc kubenswrapper[5052]: I1203 12:23:37.291747 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"barbican-db-secret" Dec 03 12:23:37 crc kubenswrapper[5052]: I1203 12:23:37.302564 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/cinder-db-create-7rgm6"] Dec 03 12:23:37 crc kubenswrapper[5052]: I1203 12:23:37.303593 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-db-create-7rgm6" Dec 03 12:23:37 crc kubenswrapper[5052]: I1203 12:23:37.310762 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-3d67-account-create-update-wmkzq"] Dec 03 12:23:37 crc kubenswrapper[5052]: I1203 12:23:37.326227 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-db-create-7rgm6"] Dec 03 12:23:37 crc kubenswrapper[5052]: I1203 12:23:37.347925 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/e01a4c75-3d41-4784-a957-681fc74f9f7d-operator-scripts\") pod \"barbican-db-create-8h8nv\" (UID: \"e01a4c75-3d41-4784-a957-681fc74f9f7d\") " pod="openstack-kuttl-tests/barbican-db-create-8h8nv" Dec 03 12:23:37 crc kubenswrapper[5052]: I1203 12:23:37.348002 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-6bdqs\" (UniqueName: \"kubernetes.io/projected/e01a4c75-3d41-4784-a957-681fc74f9f7d-kube-api-access-6bdqs\") pod \"barbican-db-create-8h8nv\" (UID: \"e01a4c75-3d41-4784-a957-681fc74f9f7d\") " pod="openstack-kuttl-tests/barbican-db-create-8h8nv" Dec 03 12:23:37 crc kubenswrapper[5052]: I1203 12:23:37.349086 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/e01a4c75-3d41-4784-a957-681fc74f9f7d-operator-scripts\") pod \"barbican-db-create-8h8nv\" (UID: \"e01a4c75-3d41-4784-a957-681fc74f9f7d\") " pod="openstack-kuttl-tests/barbican-db-create-8h8nv" Dec 03 12:23:37 crc kubenswrapper[5052]: I1203 12:23:37.391739 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/neutron-db-create-bmmmd"] Dec 03 12:23:37 crc kubenswrapper[5052]: I1203 12:23:37.393058 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-db-create-bmmmd" Dec 03 12:23:37 crc kubenswrapper[5052]: I1203 12:23:37.393324 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-6bdqs\" (UniqueName: \"kubernetes.io/projected/e01a4c75-3d41-4784-a957-681fc74f9f7d-kube-api-access-6bdqs\") pod \"barbican-db-create-8h8nv\" (UID: \"e01a4c75-3d41-4784-a957-681fc74f9f7d\") " pod="openstack-kuttl-tests/barbican-db-create-8h8nv" Dec 03 12:23:37 crc kubenswrapper[5052]: I1203 12:23:37.400665 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/cinder-c151-account-create-update-6ltql"] Dec 03 12:23:37 crc kubenswrapper[5052]: I1203 12:23:37.410649 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-db-create-8h8nv" Dec 03 12:23:37 crc kubenswrapper[5052]: I1203 12:23:37.412087 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-c151-account-create-update-6ltql" Dec 03 12:23:37 crc kubenswrapper[5052]: I1203 12:23:37.419083 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-db-create-bmmmd"] Dec 03 12:23:37 crc kubenswrapper[5052]: I1203 12:23:37.419804 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cinder-db-secret" Dec 03 12:23:37 crc kubenswrapper[5052]: I1203 12:23:37.451728 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/35767293-d6c6-4fba-92bb-053cde681af2-operator-scripts\") pod \"barbican-3d67-account-create-update-wmkzq\" (UID: \"35767293-d6c6-4fba-92bb-053cde681af2\") " pod="openstack-kuttl-tests/barbican-3d67-account-create-update-wmkzq" Dec 03 12:23:37 crc kubenswrapper[5052]: I1203 12:23:37.451791 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7xvzq\" (UniqueName: \"kubernetes.io/projected/35767293-d6c6-4fba-92bb-053cde681af2-kube-api-access-7xvzq\") pod \"barbican-3d67-account-create-update-wmkzq\" (UID: \"35767293-d6c6-4fba-92bb-053cde681af2\") " pod="openstack-kuttl-tests/barbican-3d67-account-create-update-wmkzq" Dec 03 12:23:37 crc kubenswrapper[5052]: I1203 12:23:37.451821 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zrj7j\" (UniqueName: \"kubernetes.io/projected/c8f2575e-56ef-4583-93ac-ea09f2004a79-kube-api-access-zrj7j\") pod \"cinder-db-create-7rgm6\" (UID: \"c8f2575e-56ef-4583-93ac-ea09f2004a79\") " pod="openstack-kuttl-tests/cinder-db-create-7rgm6" Dec 03 12:23:37 crc kubenswrapper[5052]: I1203 12:23:37.451849 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/c8f2575e-56ef-4583-93ac-ea09f2004a79-operator-scripts\") pod \"cinder-db-create-7rgm6\" (UID: \"c8f2575e-56ef-4583-93ac-ea09f2004a79\") " pod="openstack-kuttl-tests/cinder-db-create-7rgm6" Dec 03 12:23:37 crc kubenswrapper[5052]: I1203 12:23:37.482330 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-c151-account-create-update-6ltql"] Dec 03 12:23:37 crc kubenswrapper[5052]: I1203 12:23:37.554871 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qh5mx\" (UniqueName: \"kubernetes.io/projected/59f35833-49e4-4b5d-9646-8c481aa4a9a7-kube-api-access-qh5mx\") pod \"cinder-c151-account-create-update-6ltql\" (UID: \"59f35833-49e4-4b5d-9646-8c481aa4a9a7\") " pod="openstack-kuttl-tests/cinder-c151-account-create-update-6ltql" Dec 03 12:23:37 crc kubenswrapper[5052]: I1203 12:23:37.554940 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/35767293-d6c6-4fba-92bb-053cde681af2-operator-scripts\") pod \"barbican-3d67-account-create-update-wmkzq\" (UID: \"35767293-d6c6-4fba-92bb-053cde681af2\") " pod="openstack-kuttl-tests/barbican-3d67-account-create-update-wmkzq" Dec 03 12:23:37 crc kubenswrapper[5052]: I1203 12:23:37.554966 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/59f35833-49e4-4b5d-9646-8c481aa4a9a7-operator-scripts\") pod \"cinder-c151-account-create-update-6ltql\" (UID: \"59f35833-49e4-4b5d-9646-8c481aa4a9a7\") " pod="openstack-kuttl-tests/cinder-c151-account-create-update-6ltql" Dec 03 12:23:37 crc kubenswrapper[5052]: I1203 12:23:37.554999 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-7xvzq\" (UniqueName: \"kubernetes.io/projected/35767293-d6c6-4fba-92bb-053cde681af2-kube-api-access-7xvzq\") pod \"barbican-3d67-account-create-update-wmkzq\" (UID: \"35767293-d6c6-4fba-92bb-053cde681af2\") " pod="openstack-kuttl-tests/barbican-3d67-account-create-update-wmkzq" Dec 03 12:23:37 crc kubenswrapper[5052]: I1203 12:23:37.555024 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-zrj7j\" (UniqueName: \"kubernetes.io/projected/c8f2575e-56ef-4583-93ac-ea09f2004a79-kube-api-access-zrj7j\") pod \"cinder-db-create-7rgm6\" (UID: \"c8f2575e-56ef-4583-93ac-ea09f2004a79\") " pod="openstack-kuttl-tests/cinder-db-create-7rgm6" Dec 03 12:23:37 crc kubenswrapper[5052]: I1203 12:23:37.555053 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/c8f2575e-56ef-4583-93ac-ea09f2004a79-operator-scripts\") pod \"cinder-db-create-7rgm6\" (UID: \"c8f2575e-56ef-4583-93ac-ea09f2004a79\") " pod="openstack-kuttl-tests/cinder-db-create-7rgm6" Dec 03 12:23:37 crc kubenswrapper[5052]: I1203 12:23:37.555070 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/6b47bc28-48e6-4c70-b4a8-de8251b9a06e-operator-scripts\") pod \"neutron-db-create-bmmmd\" (UID: \"6b47bc28-48e6-4c70-b4a8-de8251b9a06e\") " pod="openstack-kuttl-tests/neutron-db-create-bmmmd" Dec 03 12:23:37 crc kubenswrapper[5052]: I1203 12:23:37.555139 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7lf5q\" (UniqueName: \"kubernetes.io/projected/6b47bc28-48e6-4c70-b4a8-de8251b9a06e-kube-api-access-7lf5q\") pod \"neutron-db-create-bmmmd\" (UID: \"6b47bc28-48e6-4c70-b4a8-de8251b9a06e\") " pod="openstack-kuttl-tests/neutron-db-create-bmmmd" Dec 03 12:23:37 crc kubenswrapper[5052]: I1203 12:23:37.555874 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/35767293-d6c6-4fba-92bb-053cde681af2-operator-scripts\") pod \"barbican-3d67-account-create-update-wmkzq\" (UID: \"35767293-d6c6-4fba-92bb-053cde681af2\") " pod="openstack-kuttl-tests/barbican-3d67-account-create-update-wmkzq" Dec 03 12:23:37 crc kubenswrapper[5052]: I1203 12:23:37.556670 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/c8f2575e-56ef-4583-93ac-ea09f2004a79-operator-scripts\") pod \"cinder-db-create-7rgm6\" (UID: \"c8f2575e-56ef-4583-93ac-ea09f2004a79\") " pod="openstack-kuttl-tests/cinder-db-create-7rgm6" Dec 03 12:23:37 crc kubenswrapper[5052]: I1203 12:23:37.568329 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/keystone-db-sync-z2t88"] Dec 03 12:23:37 crc kubenswrapper[5052]: I1203 12:23:37.569396 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-db-sync-z2t88" Dec 03 12:23:37 crc kubenswrapper[5052]: I1203 12:23:37.577101 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone" Dec 03 12:23:37 crc kubenswrapper[5052]: I1203 12:23:37.577247 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone-config-data" Dec 03 12:23:37 crc kubenswrapper[5052]: I1203 12:23:37.577340 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone-keystone-dockercfg-9qn8j" Dec 03 12:23:37 crc kubenswrapper[5052]: I1203 12:23:37.577441 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone-scripts" Dec 03 12:23:37 crc kubenswrapper[5052]: I1203 12:23:37.605371 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-db-sync-z2t88"] Dec 03 12:23:37 crc kubenswrapper[5052]: I1203 12:23:37.631617 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-zrj7j\" (UniqueName: \"kubernetes.io/projected/c8f2575e-56ef-4583-93ac-ea09f2004a79-kube-api-access-zrj7j\") pod \"cinder-db-create-7rgm6\" (UID: \"c8f2575e-56ef-4583-93ac-ea09f2004a79\") " pod="openstack-kuttl-tests/cinder-db-create-7rgm6" Dec 03 12:23:37 crc kubenswrapper[5052]: I1203 12:23:37.648119 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-7xvzq\" (UniqueName: \"kubernetes.io/projected/35767293-d6c6-4fba-92bb-053cde681af2-kube-api-access-7xvzq\") pod \"barbican-3d67-account-create-update-wmkzq\" (UID: \"35767293-d6c6-4fba-92bb-053cde681af2\") " pod="openstack-kuttl-tests/barbican-3d67-account-create-update-wmkzq" Dec 03 12:23:37 crc kubenswrapper[5052]: I1203 12:23:37.653098 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/neutron-2174-account-create-update-m4lrb"] Dec 03 12:23:37 crc kubenswrapper[5052]: I1203 12:23:37.654948 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-2174-account-create-update-m4lrb" Dec 03 12:23:37 crc kubenswrapper[5052]: I1203 12:23:37.656696 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sl5pg\" (UniqueName: \"kubernetes.io/projected/8ab8eb50-664d-4a76-b74c-c043f8891dce-kube-api-access-sl5pg\") pod \"keystone-db-sync-z2t88\" (UID: \"8ab8eb50-664d-4a76-b74c-c043f8891dce\") " pod="openstack-kuttl-tests/keystone-db-sync-z2t88" Dec 03 12:23:37 crc kubenswrapper[5052]: I1203 12:23:37.656794 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-7lf5q\" (UniqueName: \"kubernetes.io/projected/6b47bc28-48e6-4c70-b4a8-de8251b9a06e-kube-api-access-7lf5q\") pod \"neutron-db-create-bmmmd\" (UID: \"6b47bc28-48e6-4c70-b4a8-de8251b9a06e\") " pod="openstack-kuttl-tests/neutron-db-create-bmmmd" Dec 03 12:23:37 crc kubenswrapper[5052]: I1203 12:23:37.656835 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/8ab8eb50-664d-4a76-b74c-c043f8891dce-config-data\") pod \"keystone-db-sync-z2t88\" (UID: \"8ab8eb50-664d-4a76-b74c-c043f8891dce\") " pod="openstack-kuttl-tests/keystone-db-sync-z2t88" Dec 03 12:23:37 crc kubenswrapper[5052]: I1203 12:23:37.656861 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-qh5mx\" (UniqueName: \"kubernetes.io/projected/59f35833-49e4-4b5d-9646-8c481aa4a9a7-kube-api-access-qh5mx\") pod \"cinder-c151-account-create-update-6ltql\" (UID: \"59f35833-49e4-4b5d-9646-8c481aa4a9a7\") " pod="openstack-kuttl-tests/cinder-c151-account-create-update-6ltql" Dec 03 12:23:37 crc kubenswrapper[5052]: I1203 12:23:37.656902 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/59f35833-49e4-4b5d-9646-8c481aa4a9a7-operator-scripts\") pod \"cinder-c151-account-create-update-6ltql\" (UID: \"59f35833-49e4-4b5d-9646-8c481aa4a9a7\") " pod="openstack-kuttl-tests/cinder-c151-account-create-update-6ltql" Dec 03 12:23:37 crc kubenswrapper[5052]: I1203 12:23:37.656929 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8ab8eb50-664d-4a76-b74c-c043f8891dce-combined-ca-bundle\") pod \"keystone-db-sync-z2t88\" (UID: \"8ab8eb50-664d-4a76-b74c-c043f8891dce\") " pod="openstack-kuttl-tests/keystone-db-sync-z2t88" Dec 03 12:23:37 crc kubenswrapper[5052]: I1203 12:23:37.656966 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/6b47bc28-48e6-4c70-b4a8-de8251b9a06e-operator-scripts\") pod \"neutron-db-create-bmmmd\" (UID: \"6b47bc28-48e6-4c70-b4a8-de8251b9a06e\") " pod="openstack-kuttl-tests/neutron-db-create-bmmmd" Dec 03 12:23:37 crc kubenswrapper[5052]: I1203 12:23:37.657638 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"neutron-db-secret" Dec 03 12:23:37 crc kubenswrapper[5052]: I1203 12:23:37.657789 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/6b47bc28-48e6-4c70-b4a8-de8251b9a06e-operator-scripts\") pod \"neutron-db-create-bmmmd\" (UID: \"6b47bc28-48e6-4c70-b4a8-de8251b9a06e\") " pod="openstack-kuttl-tests/neutron-db-create-bmmmd" Dec 03 12:23:37 crc kubenswrapper[5052]: I1203 12:23:37.658390 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/59f35833-49e4-4b5d-9646-8c481aa4a9a7-operator-scripts\") pod \"cinder-c151-account-create-update-6ltql\" (UID: \"59f35833-49e4-4b5d-9646-8c481aa4a9a7\") " pod="openstack-kuttl-tests/cinder-c151-account-create-update-6ltql" Dec 03 12:23:37 crc kubenswrapper[5052]: I1203 12:23:37.671261 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-2174-account-create-update-m4lrb"] Dec 03 12:23:37 crc kubenswrapper[5052]: I1203 12:23:37.677186 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-7lf5q\" (UniqueName: \"kubernetes.io/projected/6b47bc28-48e6-4c70-b4a8-de8251b9a06e-kube-api-access-7lf5q\") pod \"neutron-db-create-bmmmd\" (UID: \"6b47bc28-48e6-4c70-b4a8-de8251b9a06e\") " pod="openstack-kuttl-tests/neutron-db-create-bmmmd" Dec 03 12:23:37 crc kubenswrapper[5052]: I1203 12:23:37.678700 5052 scope.go:117] "RemoveContainer" containerID="09b4a0072741f60499d24f40cc4ff3406d5d22671ced431363ab865172786300" Dec 03 12:23:37 crc kubenswrapper[5052]: E1203 12:23:37.678901 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 12:23:37 crc kubenswrapper[5052]: I1203 12:23:37.691466 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-qh5mx\" (UniqueName: \"kubernetes.io/projected/59f35833-49e4-4b5d-9646-8c481aa4a9a7-kube-api-access-qh5mx\") pod \"cinder-c151-account-create-update-6ltql\" (UID: \"59f35833-49e4-4b5d-9646-8c481aa4a9a7\") " pod="openstack-kuttl-tests/cinder-c151-account-create-update-6ltql" Dec 03 12:23:37 crc kubenswrapper[5052]: I1203 12:23:37.760748 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b6lcz\" (UniqueName: \"kubernetes.io/projected/514089fc-57f6-44d3-a961-e5f3325576fb-kube-api-access-b6lcz\") pod \"neutron-2174-account-create-update-m4lrb\" (UID: \"514089fc-57f6-44d3-a961-e5f3325576fb\") " pod="openstack-kuttl-tests/neutron-2174-account-create-update-m4lrb" Dec 03 12:23:37 crc kubenswrapper[5052]: I1203 12:23:37.761696 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-sl5pg\" (UniqueName: \"kubernetes.io/projected/8ab8eb50-664d-4a76-b74c-c043f8891dce-kube-api-access-sl5pg\") pod \"keystone-db-sync-z2t88\" (UID: \"8ab8eb50-664d-4a76-b74c-c043f8891dce\") " pod="openstack-kuttl-tests/keystone-db-sync-z2t88" Dec 03 12:23:37 crc kubenswrapper[5052]: I1203 12:23:37.761836 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/8ab8eb50-664d-4a76-b74c-c043f8891dce-config-data\") pod \"keystone-db-sync-z2t88\" (UID: \"8ab8eb50-664d-4a76-b74c-c043f8891dce\") " pod="openstack-kuttl-tests/keystone-db-sync-z2t88" Dec 03 12:23:37 crc kubenswrapper[5052]: I1203 12:23:37.762757 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8ab8eb50-664d-4a76-b74c-c043f8891dce-combined-ca-bundle\") pod \"keystone-db-sync-z2t88\" (UID: \"8ab8eb50-664d-4a76-b74c-c043f8891dce\") " pod="openstack-kuttl-tests/keystone-db-sync-z2t88" Dec 03 12:23:37 crc kubenswrapper[5052]: I1203 12:23:37.762805 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/514089fc-57f6-44d3-a961-e5f3325576fb-operator-scripts\") pod \"neutron-2174-account-create-update-m4lrb\" (UID: \"514089fc-57f6-44d3-a961-e5f3325576fb\") " pod="openstack-kuttl-tests/neutron-2174-account-create-update-m4lrb" Dec 03 12:23:37 crc kubenswrapper[5052]: I1203 12:23:37.767294 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8ab8eb50-664d-4a76-b74c-c043f8891dce-combined-ca-bundle\") pod \"keystone-db-sync-z2t88\" (UID: \"8ab8eb50-664d-4a76-b74c-c043f8891dce\") " pod="openstack-kuttl-tests/keystone-db-sync-z2t88" Dec 03 12:23:37 crc kubenswrapper[5052]: I1203 12:23:37.767671 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/8ab8eb50-664d-4a76-b74c-c043f8891dce-config-data\") pod \"keystone-db-sync-z2t88\" (UID: \"8ab8eb50-664d-4a76-b74c-c043f8891dce\") " pod="openstack-kuttl-tests/keystone-db-sync-z2t88" Dec 03 12:23:37 crc kubenswrapper[5052]: I1203 12:23:37.780478 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-db-create-8h8nv"] Dec 03 12:23:37 crc kubenswrapper[5052]: I1203 12:23:37.785636 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-sl5pg\" (UniqueName: \"kubernetes.io/projected/8ab8eb50-664d-4a76-b74c-c043f8891dce-kube-api-access-sl5pg\") pod \"keystone-db-sync-z2t88\" (UID: \"8ab8eb50-664d-4a76-b74c-c043f8891dce\") " pod="openstack-kuttl-tests/keystone-db-sync-z2t88" Dec 03 12:23:37 crc kubenswrapper[5052]: I1203 12:23:37.808484 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-db-create-bmmmd" Dec 03 12:23:37 crc kubenswrapper[5052]: I1203 12:23:37.815967 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-c151-account-create-update-6ltql" Dec 03 12:23:37 crc kubenswrapper[5052]: I1203 12:23:37.864033 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/514089fc-57f6-44d3-a961-e5f3325576fb-operator-scripts\") pod \"neutron-2174-account-create-update-m4lrb\" (UID: \"514089fc-57f6-44d3-a961-e5f3325576fb\") " pod="openstack-kuttl-tests/neutron-2174-account-create-update-m4lrb" Dec 03 12:23:37 crc kubenswrapper[5052]: I1203 12:23:37.864150 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-b6lcz\" (UniqueName: \"kubernetes.io/projected/514089fc-57f6-44d3-a961-e5f3325576fb-kube-api-access-b6lcz\") pod \"neutron-2174-account-create-update-m4lrb\" (UID: \"514089fc-57f6-44d3-a961-e5f3325576fb\") " pod="openstack-kuttl-tests/neutron-2174-account-create-update-m4lrb" Dec 03 12:23:37 crc kubenswrapper[5052]: I1203 12:23:37.865050 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/514089fc-57f6-44d3-a961-e5f3325576fb-operator-scripts\") pod \"neutron-2174-account-create-update-m4lrb\" (UID: \"514089fc-57f6-44d3-a961-e5f3325576fb\") " pod="openstack-kuttl-tests/neutron-2174-account-create-update-m4lrb" Dec 03 12:23:37 crc kubenswrapper[5052]: I1203 12:23:37.881491 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-b6lcz\" (UniqueName: \"kubernetes.io/projected/514089fc-57f6-44d3-a961-e5f3325576fb-kube-api-access-b6lcz\") pod \"neutron-2174-account-create-update-m4lrb\" (UID: \"514089fc-57f6-44d3-a961-e5f3325576fb\") " pod="openstack-kuttl-tests/neutron-2174-account-create-update-m4lrb" Dec 03 12:23:37 crc kubenswrapper[5052]: I1203 12:23:37.907570 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-3d67-account-create-update-wmkzq" Dec 03 12:23:37 crc kubenswrapper[5052]: I1203 12:23:37.916337 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-db-sync-z2t88" Dec 03 12:23:37 crc kubenswrapper[5052]: I1203 12:23:37.920142 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-db-create-7rgm6" Dec 03 12:23:37 crc kubenswrapper[5052]: I1203 12:23:37.942757 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-db-create-8h8nv" event={"ID":"e01a4c75-3d41-4784-a957-681fc74f9f7d","Type":"ContainerStarted","Data":"c82b8d31529ab92ae32994128c3b402c13a7224952855d634128365187d4bd0f"} Dec 03 12:23:38 crc kubenswrapper[5052]: I1203 12:23:38.034324 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-2174-account-create-update-m4lrb" Dec 03 12:23:38 crc kubenswrapper[5052]: I1203 12:23:38.412500 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-db-create-bmmmd"] Dec 03 12:23:38 crc kubenswrapper[5052]: I1203 12:23:38.519436 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-c151-account-create-update-6ltql"] Dec 03 12:23:38 crc kubenswrapper[5052]: W1203 12:23:38.533735 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod59f35833_49e4_4b5d_9646_8c481aa4a9a7.slice/crio-99c0ac6c190aab21b58d05bb520f5a57ca35bb3df924cb369e862446e66d05a4 WatchSource:0}: Error finding container 99c0ac6c190aab21b58d05bb520f5a57ca35bb3df924cb369e862446e66d05a4: Status 404 returned error can't find the container with id 99c0ac6c190aab21b58d05bb520f5a57ca35bb3df924cb369e862446e66d05a4 Dec 03 12:23:38 crc kubenswrapper[5052]: I1203 12:23:38.578157 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-db-create-7rgm6"] Dec 03 12:23:38 crc kubenswrapper[5052]: I1203 12:23:38.598463 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-3d67-account-create-update-wmkzq"] Dec 03 12:23:38 crc kubenswrapper[5052]: I1203 12:23:38.613664 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-db-sync-z2t88"] Dec 03 12:23:38 crc kubenswrapper[5052]: W1203 12:23:38.654196 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod35767293_d6c6_4fba_92bb_053cde681af2.slice/crio-069cc98715239f8d0fac49c1c17ad353a629388e79b530e4e4a405e713f10bd8 WatchSource:0}: Error finding container 069cc98715239f8d0fac49c1c17ad353a629388e79b530e4e4a405e713f10bd8: Status 404 returned error can't find the container with id 069cc98715239f8d0fac49c1c17ad353a629388e79b530e4e4a405e713f10bd8 Dec 03 12:23:38 crc kubenswrapper[5052]: I1203 12:23:38.717417 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-2174-account-create-update-m4lrb"] Dec 03 12:23:38 crc kubenswrapper[5052]: I1203 12:23:38.959133 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-db-create-8h8nv" event={"ID":"e01a4c75-3d41-4784-a957-681fc74f9f7d","Type":"ContainerStarted","Data":"46ad9f6912447d4884f1470a5dcbdc70613a23efc296c65a89788b0461ace5fb"} Dec 03 12:23:38 crc kubenswrapper[5052]: I1203 12:23:38.965680 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-3d67-account-create-update-wmkzq" event={"ID":"35767293-d6c6-4fba-92bb-053cde681af2","Type":"ContainerStarted","Data":"069cc98715239f8d0fac49c1c17ad353a629388e79b530e4e4a405e713f10bd8"} Dec 03 12:23:38 crc kubenswrapper[5052]: I1203 12:23:38.971698 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-c151-account-create-update-6ltql" event={"ID":"59f35833-49e4-4b5d-9646-8c481aa4a9a7","Type":"ContainerStarted","Data":"99c0ac6c190aab21b58d05bb520f5a57ca35bb3df924cb369e862446e66d05a4"} Dec 03 12:23:38 crc kubenswrapper[5052]: I1203 12:23:38.973987 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-db-create-bmmmd" event={"ID":"6b47bc28-48e6-4c70-b4a8-de8251b9a06e","Type":"ContainerStarted","Data":"4391a061ef12d6e77f121f1319fc0c929b8a1a6d2bde016e1873cf846c090113"} Dec 03 12:23:38 crc kubenswrapper[5052]: I1203 12:23:38.975893 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-2174-account-create-update-m4lrb" event={"ID":"514089fc-57f6-44d3-a961-e5f3325576fb","Type":"ContainerStarted","Data":"f6abffc4ed7165697bdec067fb24d7e2ede1689d5d1792d7dcc890777ac5dfad"} Dec 03 12:23:38 crc kubenswrapper[5052]: I1203 12:23:38.977839 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-db-sync-z2t88" event={"ID":"8ab8eb50-664d-4a76-b74c-c043f8891dce","Type":"ContainerStarted","Data":"4f79e32345024f7811cfc3a5a0de6633e2acdb3bb7b1c5b4876c5c6fb559febd"} Dec 03 12:23:38 crc kubenswrapper[5052]: I1203 12:23:38.981026 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-db-create-7rgm6" event={"ID":"c8f2575e-56ef-4583-93ac-ea09f2004a79","Type":"ContainerStarted","Data":"d7f6de79d11821239e5c74b1be121c5cade0b7250f62d1cbc32b1b6301715c25"} Dec 03 12:23:38 crc kubenswrapper[5052]: I1203 12:23:38.994564 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/barbican-db-create-8h8nv" podStartSLOduration=1.994542536 podStartE2EDuration="1.994542536s" podCreationTimestamp="2025-12-03 12:23:37 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:23:38.977538711 +0000 UTC m=+3809.565780475" watchObservedRunningTime="2025-12-03 12:23:38.994542536 +0000 UTC m=+3809.582784300" Dec 03 12:23:39 crc kubenswrapper[5052]: I1203 12:23:39.002704 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/cinder-db-create-7rgm6" podStartSLOduration=2.002681998 podStartE2EDuration="2.002681998s" podCreationTimestamp="2025-12-03 12:23:37 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:23:39.00099413 +0000 UTC m=+3809.589235904" watchObservedRunningTime="2025-12-03 12:23:39.002681998 +0000 UTC m=+3809.590923762" Dec 03 12:23:39 crc kubenswrapper[5052]: E1203 12:23:39.215600 5052 cadvisor_stats_provider.go:516] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode01a4c75_3d41_4784_a957_681fc74f9f7d.slice/crio-46ad9f6912447d4884f1470a5dcbdc70613a23efc296c65a89788b0461ace5fb.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode01a4c75_3d41_4784_a957_681fc74f9f7d.slice/crio-conmon-46ad9f6912447d4884f1470a5dcbdc70613a23efc296c65a89788b0461ace5fb.scope\": RecentStats: unable to find data in memory cache]" Dec 03 12:23:39 crc kubenswrapper[5052]: I1203 12:23:39.991470 5052 generic.go:334] "Generic (PLEG): container finished" podID="35767293-d6c6-4fba-92bb-053cde681af2" containerID="ca583556bd7883d15cb27eea37734da610b1438d1bc770d94b10a6407a260036" exitCode=0 Dec 03 12:23:39 crc kubenswrapper[5052]: I1203 12:23:39.991556 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-3d67-account-create-update-wmkzq" event={"ID":"35767293-d6c6-4fba-92bb-053cde681af2","Type":"ContainerDied","Data":"ca583556bd7883d15cb27eea37734da610b1438d1bc770d94b10a6407a260036"} Dec 03 12:23:39 crc kubenswrapper[5052]: I1203 12:23:39.993768 5052 generic.go:334] "Generic (PLEG): container finished" podID="59f35833-49e4-4b5d-9646-8c481aa4a9a7" containerID="28fe6a9fcdf74a06f01f23fda2bdb34246e51f9604f3cd36102d78d52be9d6f6" exitCode=0 Dec 03 12:23:39 crc kubenswrapper[5052]: I1203 12:23:39.993843 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-c151-account-create-update-6ltql" event={"ID":"59f35833-49e4-4b5d-9646-8c481aa4a9a7","Type":"ContainerDied","Data":"28fe6a9fcdf74a06f01f23fda2bdb34246e51f9604f3cd36102d78d52be9d6f6"} Dec 03 12:23:39 crc kubenswrapper[5052]: I1203 12:23:39.998389 5052 generic.go:334] "Generic (PLEG): container finished" podID="6b47bc28-48e6-4c70-b4a8-de8251b9a06e" containerID="3418fbee6952b48c95c35c9758d124d82b2c04833e3b31186c3190edd275b7cd" exitCode=0 Dec 03 12:23:39 crc kubenswrapper[5052]: I1203 12:23:39.998410 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-db-create-bmmmd" event={"ID":"6b47bc28-48e6-4c70-b4a8-de8251b9a06e","Type":"ContainerDied","Data":"3418fbee6952b48c95c35c9758d124d82b2c04833e3b31186c3190edd275b7cd"} Dec 03 12:23:40 crc kubenswrapper[5052]: I1203 12:23:40.000704 5052 generic.go:334] "Generic (PLEG): container finished" podID="514089fc-57f6-44d3-a961-e5f3325576fb" containerID="8a99003bc5144e66c0c428e9b1815efa135128f34c7b263b18770d41d3b547ec" exitCode=0 Dec 03 12:23:40 crc kubenswrapper[5052]: I1203 12:23:40.000740 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-2174-account-create-update-m4lrb" event={"ID":"514089fc-57f6-44d3-a961-e5f3325576fb","Type":"ContainerDied","Data":"8a99003bc5144e66c0c428e9b1815efa135128f34c7b263b18770d41d3b547ec"} Dec 03 12:23:40 crc kubenswrapper[5052]: I1203 12:23:40.002793 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-db-sync-z2t88" event={"ID":"8ab8eb50-664d-4a76-b74c-c043f8891dce","Type":"ContainerStarted","Data":"de982574a6d170dab76a64098f7410a6d612c5fad55aa7ca04f5090fda64d238"} Dec 03 12:23:40 crc kubenswrapper[5052]: I1203 12:23:40.004339 5052 generic.go:334] "Generic (PLEG): container finished" podID="c8f2575e-56ef-4583-93ac-ea09f2004a79" containerID="e0aacbc34641afb12682d4842ddff06302fd96d916828aad6ac07df6cad52fcf" exitCode=0 Dec 03 12:23:40 crc kubenswrapper[5052]: I1203 12:23:40.004361 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-db-create-7rgm6" event={"ID":"c8f2575e-56ef-4583-93ac-ea09f2004a79","Type":"ContainerDied","Data":"e0aacbc34641afb12682d4842ddff06302fd96d916828aad6ac07df6cad52fcf"} Dec 03 12:23:40 crc kubenswrapper[5052]: I1203 12:23:40.006027 5052 generic.go:334] "Generic (PLEG): container finished" podID="e01a4c75-3d41-4784-a957-681fc74f9f7d" containerID="46ad9f6912447d4884f1470a5dcbdc70613a23efc296c65a89788b0461ace5fb" exitCode=0 Dec 03 12:23:40 crc kubenswrapper[5052]: I1203 12:23:40.006111 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-db-create-8h8nv" event={"ID":"e01a4c75-3d41-4784-a957-681fc74f9f7d","Type":"ContainerDied","Data":"46ad9f6912447d4884f1470a5dcbdc70613a23efc296c65a89788b0461ace5fb"} Dec 03 12:23:40 crc kubenswrapper[5052]: I1203 12:23:40.068187 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/keystone-db-sync-z2t88" podStartSLOduration=3.06816894 podStartE2EDuration="3.06816894s" podCreationTimestamp="2025-12-03 12:23:37 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:23:40.061054807 +0000 UTC m=+3810.649296581" watchObservedRunningTime="2025-12-03 12:23:40.06816894 +0000 UTC m=+3810.656410704" Dec 03 12:23:41 crc kubenswrapper[5052]: I1203 12:23:41.902290 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-c151-account-create-update-6ltql" Dec 03 12:23:41 crc kubenswrapper[5052]: I1203 12:23:41.921956 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-db-create-8h8nv" Dec 03 12:23:41 crc kubenswrapper[5052]: I1203 12:23:41.930299 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-3d67-account-create-update-wmkzq" Dec 03 12:23:41 crc kubenswrapper[5052]: I1203 12:23:41.939443 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-db-create-bmmmd" Dec 03 12:23:41 crc kubenswrapper[5052]: I1203 12:23:41.944995 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-2174-account-create-update-m4lrb" Dec 03 12:23:41 crc kubenswrapper[5052]: I1203 12:23:41.960660 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-db-create-7rgm6" Dec 03 12:23:42 crc kubenswrapper[5052]: I1203 12:23:42.033532 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-db-create-bmmmd" event={"ID":"6b47bc28-48e6-4c70-b4a8-de8251b9a06e","Type":"ContainerDied","Data":"4391a061ef12d6e77f121f1319fc0c929b8a1a6d2bde016e1873cf846c090113"} Dec 03 12:23:42 crc kubenswrapper[5052]: I1203 12:23:42.033582 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4391a061ef12d6e77f121f1319fc0c929b8a1a6d2bde016e1873cf846c090113" Dec 03 12:23:42 crc kubenswrapper[5052]: I1203 12:23:42.033563 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-db-create-bmmmd" Dec 03 12:23:42 crc kubenswrapper[5052]: I1203 12:23:42.035567 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-2174-account-create-update-m4lrb" event={"ID":"514089fc-57f6-44d3-a961-e5f3325576fb","Type":"ContainerDied","Data":"f6abffc4ed7165697bdec067fb24d7e2ede1689d5d1792d7dcc890777ac5dfad"} Dec 03 12:23:42 crc kubenswrapper[5052]: I1203 12:23:42.035595 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f6abffc4ed7165697bdec067fb24d7e2ede1689d5d1792d7dcc890777ac5dfad" Dec 03 12:23:42 crc kubenswrapper[5052]: I1203 12:23:42.035573 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-2174-account-create-update-m4lrb" Dec 03 12:23:42 crc kubenswrapper[5052]: I1203 12:23:42.037021 5052 generic.go:334] "Generic (PLEG): container finished" podID="8ab8eb50-664d-4a76-b74c-c043f8891dce" containerID="de982574a6d170dab76a64098f7410a6d612c5fad55aa7ca04f5090fda64d238" exitCode=0 Dec 03 12:23:42 crc kubenswrapper[5052]: I1203 12:23:42.037082 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-db-sync-z2t88" event={"ID":"8ab8eb50-664d-4a76-b74c-c043f8891dce","Type":"ContainerDied","Data":"de982574a6d170dab76a64098f7410a6d612c5fad55aa7ca04f5090fda64d238"} Dec 03 12:23:42 crc kubenswrapper[5052]: I1203 12:23:42.040002 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-db-create-7rgm6" Dec 03 12:23:42 crc kubenswrapper[5052]: I1203 12:23:42.040181 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-db-create-7rgm6" event={"ID":"c8f2575e-56ef-4583-93ac-ea09f2004a79","Type":"ContainerDied","Data":"d7f6de79d11821239e5c74b1be121c5cade0b7250f62d1cbc32b1b6301715c25"} Dec 03 12:23:42 crc kubenswrapper[5052]: I1203 12:23:42.040201 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d7f6de79d11821239e5c74b1be121c5cade0b7250f62d1cbc32b1b6301715c25" Dec 03 12:23:42 crc kubenswrapper[5052]: I1203 12:23:42.040452 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/c8f2575e-56ef-4583-93ac-ea09f2004a79-operator-scripts\") pod \"c8f2575e-56ef-4583-93ac-ea09f2004a79\" (UID: \"c8f2575e-56ef-4583-93ac-ea09f2004a79\") " Dec 03 12:23:42 crc kubenswrapper[5052]: I1203 12:23:42.040600 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7lf5q\" (UniqueName: \"kubernetes.io/projected/6b47bc28-48e6-4c70-b4a8-de8251b9a06e-kube-api-access-7lf5q\") pod \"6b47bc28-48e6-4c70-b4a8-de8251b9a06e\" (UID: \"6b47bc28-48e6-4c70-b4a8-de8251b9a06e\") " Dec 03 12:23:42 crc kubenswrapper[5052]: I1203 12:23:42.040682 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b6lcz\" (UniqueName: \"kubernetes.io/projected/514089fc-57f6-44d3-a961-e5f3325576fb-kube-api-access-b6lcz\") pod \"514089fc-57f6-44d3-a961-e5f3325576fb\" (UID: \"514089fc-57f6-44d3-a961-e5f3325576fb\") " Dec 03 12:23:42 crc kubenswrapper[5052]: I1203 12:23:42.040740 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6bdqs\" (UniqueName: \"kubernetes.io/projected/e01a4c75-3d41-4784-a957-681fc74f9f7d-kube-api-access-6bdqs\") pod \"e01a4c75-3d41-4784-a957-681fc74f9f7d\" (UID: \"e01a4c75-3d41-4784-a957-681fc74f9f7d\") " Dec 03 12:23:42 crc kubenswrapper[5052]: I1203 12:23:42.040776 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zrj7j\" (UniqueName: \"kubernetes.io/projected/c8f2575e-56ef-4583-93ac-ea09f2004a79-kube-api-access-zrj7j\") pod \"c8f2575e-56ef-4583-93ac-ea09f2004a79\" (UID: \"c8f2575e-56ef-4583-93ac-ea09f2004a79\") " Dec 03 12:23:42 crc kubenswrapper[5052]: I1203 12:23:42.040848 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/514089fc-57f6-44d3-a961-e5f3325576fb-operator-scripts\") pod \"514089fc-57f6-44d3-a961-e5f3325576fb\" (UID: \"514089fc-57f6-44d3-a961-e5f3325576fb\") " Dec 03 12:23:42 crc kubenswrapper[5052]: I1203 12:23:42.040926 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/6b47bc28-48e6-4c70-b4a8-de8251b9a06e-operator-scripts\") pod \"6b47bc28-48e6-4c70-b4a8-de8251b9a06e\" (UID: \"6b47bc28-48e6-4c70-b4a8-de8251b9a06e\") " Dec 03 12:23:42 crc kubenswrapper[5052]: I1203 12:23:42.041004 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/35767293-d6c6-4fba-92bb-053cde681af2-operator-scripts\") pod \"35767293-d6c6-4fba-92bb-053cde681af2\" (UID: \"35767293-d6c6-4fba-92bb-053cde681af2\") " Dec 03 12:23:42 crc kubenswrapper[5052]: I1203 12:23:42.041011 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c8f2575e-56ef-4583-93ac-ea09f2004a79-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "c8f2575e-56ef-4583-93ac-ea09f2004a79" (UID: "c8f2575e-56ef-4583-93ac-ea09f2004a79"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:23:42 crc kubenswrapper[5052]: I1203 12:23:42.041048 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/59f35833-49e4-4b5d-9646-8c481aa4a9a7-operator-scripts\") pod \"59f35833-49e4-4b5d-9646-8c481aa4a9a7\" (UID: \"59f35833-49e4-4b5d-9646-8c481aa4a9a7\") " Dec 03 12:23:42 crc kubenswrapper[5052]: I1203 12:23:42.041105 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qh5mx\" (UniqueName: \"kubernetes.io/projected/59f35833-49e4-4b5d-9646-8c481aa4a9a7-kube-api-access-qh5mx\") pod \"59f35833-49e4-4b5d-9646-8c481aa4a9a7\" (UID: \"59f35833-49e4-4b5d-9646-8c481aa4a9a7\") " Dec 03 12:23:42 crc kubenswrapper[5052]: I1203 12:23:42.041173 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/e01a4c75-3d41-4784-a957-681fc74f9f7d-operator-scripts\") pod \"e01a4c75-3d41-4784-a957-681fc74f9f7d\" (UID: \"e01a4c75-3d41-4784-a957-681fc74f9f7d\") " Dec 03 12:23:42 crc kubenswrapper[5052]: I1203 12:23:42.041255 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7xvzq\" (UniqueName: \"kubernetes.io/projected/35767293-d6c6-4fba-92bb-053cde681af2-kube-api-access-7xvzq\") pod \"35767293-d6c6-4fba-92bb-053cde681af2\" (UID: \"35767293-d6c6-4fba-92bb-053cde681af2\") " Dec 03 12:23:42 crc kubenswrapper[5052]: I1203 12:23:42.041637 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6b47bc28-48e6-4c70-b4a8-de8251b9a06e-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "6b47bc28-48e6-4c70-b4a8-de8251b9a06e" (UID: "6b47bc28-48e6-4c70-b4a8-de8251b9a06e"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:23:42 crc kubenswrapper[5052]: I1203 12:23:42.041831 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/c8f2575e-56ef-4583-93ac-ea09f2004a79-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:23:42 crc kubenswrapper[5052]: I1203 12:23:42.043118 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e01a4c75-3d41-4784-a957-681fc74f9f7d-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "e01a4c75-3d41-4784-a957-681fc74f9f7d" (UID: "e01a4c75-3d41-4784-a957-681fc74f9f7d"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:23:42 crc kubenswrapper[5052]: I1203 12:23:42.043158 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/59f35833-49e4-4b5d-9646-8c481aa4a9a7-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "59f35833-49e4-4b5d-9646-8c481aa4a9a7" (UID: "59f35833-49e4-4b5d-9646-8c481aa4a9a7"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:23:42 crc kubenswrapper[5052]: I1203 12:23:42.043734 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-db-create-8h8nv" Dec 03 12:23:42 crc kubenswrapper[5052]: I1203 12:23:42.043729 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-db-create-8h8nv" event={"ID":"e01a4c75-3d41-4784-a957-681fc74f9f7d","Type":"ContainerDied","Data":"c82b8d31529ab92ae32994128c3b402c13a7224952855d634128365187d4bd0f"} Dec 03 12:23:42 crc kubenswrapper[5052]: I1203 12:23:42.044031 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c82b8d31529ab92ae32994128c3b402c13a7224952855d634128365187d4bd0f" Dec 03 12:23:42 crc kubenswrapper[5052]: I1203 12:23:42.044301 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/514089fc-57f6-44d3-a961-e5f3325576fb-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "514089fc-57f6-44d3-a961-e5f3325576fb" (UID: "514089fc-57f6-44d3-a961-e5f3325576fb"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:23:42 crc kubenswrapper[5052]: I1203 12:23:42.044438 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/35767293-d6c6-4fba-92bb-053cde681af2-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "35767293-d6c6-4fba-92bb-053cde681af2" (UID: "35767293-d6c6-4fba-92bb-053cde681af2"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:23:42 crc kubenswrapper[5052]: I1203 12:23:42.046038 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-3d67-account-create-update-wmkzq" Dec 03 12:23:42 crc kubenswrapper[5052]: I1203 12:23:42.046373 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-3d67-account-create-update-wmkzq" event={"ID":"35767293-d6c6-4fba-92bb-053cde681af2","Type":"ContainerDied","Data":"069cc98715239f8d0fac49c1c17ad353a629388e79b530e4e4a405e713f10bd8"} Dec 03 12:23:42 crc kubenswrapper[5052]: I1203 12:23:42.046398 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="069cc98715239f8d0fac49c1c17ad353a629388e79b530e4e4a405e713f10bd8" Dec 03 12:23:42 crc kubenswrapper[5052]: I1203 12:23:42.046403 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/35767293-d6c6-4fba-92bb-053cde681af2-kube-api-access-7xvzq" (OuterVolumeSpecName: "kube-api-access-7xvzq") pod "35767293-d6c6-4fba-92bb-053cde681af2" (UID: "35767293-d6c6-4fba-92bb-053cde681af2"). InnerVolumeSpecName "kube-api-access-7xvzq". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:23:42 crc kubenswrapper[5052]: I1203 12:23:42.046850 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e01a4c75-3d41-4784-a957-681fc74f9f7d-kube-api-access-6bdqs" (OuterVolumeSpecName: "kube-api-access-6bdqs") pod "e01a4c75-3d41-4784-a957-681fc74f9f7d" (UID: "e01a4c75-3d41-4784-a957-681fc74f9f7d"). InnerVolumeSpecName "kube-api-access-6bdqs". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:23:42 crc kubenswrapper[5052]: I1203 12:23:42.047227 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/514089fc-57f6-44d3-a961-e5f3325576fb-kube-api-access-b6lcz" (OuterVolumeSpecName: "kube-api-access-b6lcz") pod "514089fc-57f6-44d3-a961-e5f3325576fb" (UID: "514089fc-57f6-44d3-a961-e5f3325576fb"). InnerVolumeSpecName "kube-api-access-b6lcz". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:23:42 crc kubenswrapper[5052]: I1203 12:23:42.049321 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-c151-account-create-update-6ltql" event={"ID":"59f35833-49e4-4b5d-9646-8c481aa4a9a7","Type":"ContainerDied","Data":"99c0ac6c190aab21b58d05bb520f5a57ca35bb3df924cb369e862446e66d05a4"} Dec 03 12:23:42 crc kubenswrapper[5052]: I1203 12:23:42.049376 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="99c0ac6c190aab21b58d05bb520f5a57ca35bb3df924cb369e862446e66d05a4" Dec 03 12:23:42 crc kubenswrapper[5052]: I1203 12:23:42.049374 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-c151-account-create-update-6ltql" Dec 03 12:23:42 crc kubenswrapper[5052]: I1203 12:23:42.049744 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c8f2575e-56ef-4583-93ac-ea09f2004a79-kube-api-access-zrj7j" (OuterVolumeSpecName: "kube-api-access-zrj7j") pod "c8f2575e-56ef-4583-93ac-ea09f2004a79" (UID: "c8f2575e-56ef-4583-93ac-ea09f2004a79"). InnerVolumeSpecName "kube-api-access-zrj7j". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:23:42 crc kubenswrapper[5052]: I1203 12:23:42.058217 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6b47bc28-48e6-4c70-b4a8-de8251b9a06e-kube-api-access-7lf5q" (OuterVolumeSpecName: "kube-api-access-7lf5q") pod "6b47bc28-48e6-4c70-b4a8-de8251b9a06e" (UID: "6b47bc28-48e6-4c70-b4a8-de8251b9a06e"). InnerVolumeSpecName "kube-api-access-7lf5q". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:23:42 crc kubenswrapper[5052]: I1203 12:23:42.060682 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/59f35833-49e4-4b5d-9646-8c481aa4a9a7-kube-api-access-qh5mx" (OuterVolumeSpecName: "kube-api-access-qh5mx") pod "59f35833-49e4-4b5d-9646-8c481aa4a9a7" (UID: "59f35833-49e4-4b5d-9646-8c481aa4a9a7"). InnerVolumeSpecName "kube-api-access-qh5mx". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:23:42 crc kubenswrapper[5052]: I1203 12:23:42.144478 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/35767293-d6c6-4fba-92bb-053cde681af2-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:23:42 crc kubenswrapper[5052]: I1203 12:23:42.144555 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/59f35833-49e4-4b5d-9646-8c481aa4a9a7-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:23:42 crc kubenswrapper[5052]: I1203 12:23:42.144569 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-qh5mx\" (UniqueName: \"kubernetes.io/projected/59f35833-49e4-4b5d-9646-8c481aa4a9a7-kube-api-access-qh5mx\") on node \"crc\" DevicePath \"\"" Dec 03 12:23:42 crc kubenswrapper[5052]: I1203 12:23:42.144584 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/e01a4c75-3d41-4784-a957-681fc74f9f7d-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:23:42 crc kubenswrapper[5052]: I1203 12:23:42.144596 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-7xvzq\" (UniqueName: \"kubernetes.io/projected/35767293-d6c6-4fba-92bb-053cde681af2-kube-api-access-7xvzq\") on node \"crc\" DevicePath \"\"" Dec 03 12:23:42 crc kubenswrapper[5052]: I1203 12:23:42.144630 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-7lf5q\" (UniqueName: \"kubernetes.io/projected/6b47bc28-48e6-4c70-b4a8-de8251b9a06e-kube-api-access-7lf5q\") on node \"crc\" DevicePath \"\"" Dec 03 12:23:42 crc kubenswrapper[5052]: I1203 12:23:42.144644 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-b6lcz\" (UniqueName: \"kubernetes.io/projected/514089fc-57f6-44d3-a961-e5f3325576fb-kube-api-access-b6lcz\") on node \"crc\" DevicePath \"\"" Dec 03 12:23:42 crc kubenswrapper[5052]: I1203 12:23:42.144657 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-6bdqs\" (UniqueName: \"kubernetes.io/projected/e01a4c75-3d41-4784-a957-681fc74f9f7d-kube-api-access-6bdqs\") on node \"crc\" DevicePath \"\"" Dec 03 12:23:42 crc kubenswrapper[5052]: I1203 12:23:42.144668 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-zrj7j\" (UniqueName: \"kubernetes.io/projected/c8f2575e-56ef-4583-93ac-ea09f2004a79-kube-api-access-zrj7j\") on node \"crc\" DevicePath \"\"" Dec 03 12:23:42 crc kubenswrapper[5052]: I1203 12:23:42.144680 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/514089fc-57f6-44d3-a961-e5f3325576fb-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:23:42 crc kubenswrapper[5052]: I1203 12:23:42.144714 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/6b47bc28-48e6-4c70-b4a8-de8251b9a06e-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:23:43 crc kubenswrapper[5052]: I1203 12:23:43.398234 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-db-sync-z2t88" Dec 03 12:23:43 crc kubenswrapper[5052]: I1203 12:23:43.564946 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/8ab8eb50-664d-4a76-b74c-c043f8891dce-config-data\") pod \"8ab8eb50-664d-4a76-b74c-c043f8891dce\" (UID: \"8ab8eb50-664d-4a76-b74c-c043f8891dce\") " Dec 03 12:23:43 crc kubenswrapper[5052]: I1203 12:23:43.565056 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sl5pg\" (UniqueName: \"kubernetes.io/projected/8ab8eb50-664d-4a76-b74c-c043f8891dce-kube-api-access-sl5pg\") pod \"8ab8eb50-664d-4a76-b74c-c043f8891dce\" (UID: \"8ab8eb50-664d-4a76-b74c-c043f8891dce\") " Dec 03 12:23:43 crc kubenswrapper[5052]: I1203 12:23:43.565153 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8ab8eb50-664d-4a76-b74c-c043f8891dce-combined-ca-bundle\") pod \"8ab8eb50-664d-4a76-b74c-c043f8891dce\" (UID: \"8ab8eb50-664d-4a76-b74c-c043f8891dce\") " Dec 03 12:23:43 crc kubenswrapper[5052]: I1203 12:23:43.622993 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8ab8eb50-664d-4a76-b74c-c043f8891dce-kube-api-access-sl5pg" (OuterVolumeSpecName: "kube-api-access-sl5pg") pod "8ab8eb50-664d-4a76-b74c-c043f8891dce" (UID: "8ab8eb50-664d-4a76-b74c-c043f8891dce"). InnerVolumeSpecName "kube-api-access-sl5pg". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:23:43 crc kubenswrapper[5052]: I1203 12:23:43.626653 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8ab8eb50-664d-4a76-b74c-c043f8891dce-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "8ab8eb50-664d-4a76-b74c-c043f8891dce" (UID: "8ab8eb50-664d-4a76-b74c-c043f8891dce"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:23:43 crc kubenswrapper[5052]: I1203 12:23:43.649074 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8ab8eb50-664d-4a76-b74c-c043f8891dce-config-data" (OuterVolumeSpecName: "config-data") pod "8ab8eb50-664d-4a76-b74c-c043f8891dce" (UID: "8ab8eb50-664d-4a76-b74c-c043f8891dce"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:23:43 crc kubenswrapper[5052]: I1203 12:23:43.667532 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8ab8eb50-664d-4a76-b74c-c043f8891dce-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:23:43 crc kubenswrapper[5052]: I1203 12:23:43.667586 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/8ab8eb50-664d-4a76-b74c-c043f8891dce-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:23:43 crc kubenswrapper[5052]: I1203 12:23:43.667600 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-sl5pg\" (UniqueName: \"kubernetes.io/projected/8ab8eb50-664d-4a76-b74c-c043f8891dce-kube-api-access-sl5pg\") on node \"crc\" DevicePath \"\"" Dec 03 12:23:44 crc kubenswrapper[5052]: I1203 12:23:44.079383 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-db-sync-z2t88" event={"ID":"8ab8eb50-664d-4a76-b74c-c043f8891dce","Type":"ContainerDied","Data":"4f79e32345024f7811cfc3a5a0de6633e2acdb3bb7b1c5b4876c5c6fb559febd"} Dec 03 12:23:44 crc kubenswrapper[5052]: I1203 12:23:44.079471 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-db-sync-z2t88" Dec 03 12:23:44 crc kubenswrapper[5052]: I1203 12:23:44.079520 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4f79e32345024f7811cfc3a5a0de6633e2acdb3bb7b1c5b4876c5c6fb559febd" Dec 03 12:23:44 crc kubenswrapper[5052]: I1203 12:23:44.230571 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/keystone-bootstrap-mh8wf"] Dec 03 12:23:44 crc kubenswrapper[5052]: E1203 12:23:44.230939 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="6b47bc28-48e6-4c70-b4a8-de8251b9a06e" containerName="mariadb-database-create" Dec 03 12:23:44 crc kubenswrapper[5052]: I1203 12:23:44.230952 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="6b47bc28-48e6-4c70-b4a8-de8251b9a06e" containerName="mariadb-database-create" Dec 03 12:23:44 crc kubenswrapper[5052]: E1203 12:23:44.230973 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="35767293-d6c6-4fba-92bb-053cde681af2" containerName="mariadb-account-create-update" Dec 03 12:23:44 crc kubenswrapper[5052]: I1203 12:23:44.230980 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="35767293-d6c6-4fba-92bb-053cde681af2" containerName="mariadb-account-create-update" Dec 03 12:23:44 crc kubenswrapper[5052]: E1203 12:23:44.230996 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="c8f2575e-56ef-4583-93ac-ea09f2004a79" containerName="mariadb-database-create" Dec 03 12:23:44 crc kubenswrapper[5052]: I1203 12:23:44.231004 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="c8f2575e-56ef-4583-93ac-ea09f2004a79" containerName="mariadb-database-create" Dec 03 12:23:44 crc kubenswrapper[5052]: E1203 12:23:44.231015 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="e01a4c75-3d41-4784-a957-681fc74f9f7d" containerName="mariadb-database-create" Dec 03 12:23:44 crc kubenswrapper[5052]: I1203 12:23:44.231021 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="e01a4c75-3d41-4784-a957-681fc74f9f7d" containerName="mariadb-database-create" Dec 03 12:23:44 crc kubenswrapper[5052]: E1203 12:23:44.231031 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="514089fc-57f6-44d3-a961-e5f3325576fb" containerName="mariadb-account-create-update" Dec 03 12:23:44 crc kubenswrapper[5052]: I1203 12:23:44.231037 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="514089fc-57f6-44d3-a961-e5f3325576fb" containerName="mariadb-account-create-update" Dec 03 12:23:44 crc kubenswrapper[5052]: E1203 12:23:44.231058 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="8ab8eb50-664d-4a76-b74c-c043f8891dce" containerName="keystone-db-sync" Dec 03 12:23:44 crc kubenswrapper[5052]: I1203 12:23:44.231064 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="8ab8eb50-664d-4a76-b74c-c043f8891dce" containerName="keystone-db-sync" Dec 03 12:23:44 crc kubenswrapper[5052]: E1203 12:23:44.231072 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="59f35833-49e4-4b5d-9646-8c481aa4a9a7" containerName="mariadb-account-create-update" Dec 03 12:23:44 crc kubenswrapper[5052]: I1203 12:23:44.231079 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="59f35833-49e4-4b5d-9646-8c481aa4a9a7" containerName="mariadb-account-create-update" Dec 03 12:23:44 crc kubenswrapper[5052]: I1203 12:23:44.231259 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="514089fc-57f6-44d3-a961-e5f3325576fb" containerName="mariadb-account-create-update" Dec 03 12:23:44 crc kubenswrapper[5052]: I1203 12:23:44.231282 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="c8f2575e-56ef-4583-93ac-ea09f2004a79" containerName="mariadb-database-create" Dec 03 12:23:44 crc kubenswrapper[5052]: I1203 12:23:44.231296 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="8ab8eb50-664d-4a76-b74c-c043f8891dce" containerName="keystone-db-sync" Dec 03 12:23:44 crc kubenswrapper[5052]: I1203 12:23:44.231310 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="59f35833-49e4-4b5d-9646-8c481aa4a9a7" containerName="mariadb-account-create-update" Dec 03 12:23:44 crc kubenswrapper[5052]: I1203 12:23:44.231326 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="6b47bc28-48e6-4c70-b4a8-de8251b9a06e" containerName="mariadb-database-create" Dec 03 12:23:44 crc kubenswrapper[5052]: I1203 12:23:44.231336 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="e01a4c75-3d41-4784-a957-681fc74f9f7d" containerName="mariadb-database-create" Dec 03 12:23:44 crc kubenswrapper[5052]: I1203 12:23:44.231346 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="35767293-d6c6-4fba-92bb-053cde681af2" containerName="mariadb-account-create-update" Dec 03 12:23:44 crc kubenswrapper[5052]: I1203 12:23:44.231954 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-bootstrap-mh8wf" Dec 03 12:23:44 crc kubenswrapper[5052]: I1203 12:23:44.234838 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone-scripts" Dec 03 12:23:44 crc kubenswrapper[5052]: I1203 12:23:44.235077 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"osp-secret" Dec 03 12:23:44 crc kubenswrapper[5052]: I1203 12:23:44.236977 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone-config-data" Dec 03 12:23:44 crc kubenswrapper[5052]: I1203 12:23:44.237163 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone" Dec 03 12:23:44 crc kubenswrapper[5052]: I1203 12:23:44.237327 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone-keystone-dockercfg-9qn8j" Dec 03 12:23:44 crc kubenswrapper[5052]: I1203 12:23:44.249715 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-bootstrap-mh8wf"] Dec 03 12:23:44 crc kubenswrapper[5052]: I1203 12:23:44.379320 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/f21141cf-d708-4da1-8bb7-5900993b9d26-fernet-keys\") pod \"keystone-bootstrap-mh8wf\" (UID: \"f21141cf-d708-4da1-8bb7-5900993b9d26\") " pod="openstack-kuttl-tests/keystone-bootstrap-mh8wf" Dec 03 12:23:44 crc kubenswrapper[5052]: I1203 12:23:44.379375 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/f21141cf-d708-4da1-8bb7-5900993b9d26-config-data\") pod \"keystone-bootstrap-mh8wf\" (UID: \"f21141cf-d708-4da1-8bb7-5900993b9d26\") " pod="openstack-kuttl-tests/keystone-bootstrap-mh8wf" Dec 03 12:23:44 crc kubenswrapper[5052]: I1203 12:23:44.379445 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/f21141cf-d708-4da1-8bb7-5900993b9d26-credential-keys\") pod \"keystone-bootstrap-mh8wf\" (UID: \"f21141cf-d708-4da1-8bb7-5900993b9d26\") " pod="openstack-kuttl-tests/keystone-bootstrap-mh8wf" Dec 03 12:23:44 crc kubenswrapper[5052]: I1203 12:23:44.379474 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f21141cf-d708-4da1-8bb7-5900993b9d26-combined-ca-bundle\") pod \"keystone-bootstrap-mh8wf\" (UID: \"f21141cf-d708-4da1-8bb7-5900993b9d26\") " pod="openstack-kuttl-tests/keystone-bootstrap-mh8wf" Dec 03 12:23:44 crc kubenswrapper[5052]: I1203 12:23:44.379588 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/f21141cf-d708-4da1-8bb7-5900993b9d26-scripts\") pod \"keystone-bootstrap-mh8wf\" (UID: \"f21141cf-d708-4da1-8bb7-5900993b9d26\") " pod="openstack-kuttl-tests/keystone-bootstrap-mh8wf" Dec 03 12:23:44 crc kubenswrapper[5052]: I1203 12:23:44.379617 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f5srj\" (UniqueName: \"kubernetes.io/projected/f21141cf-d708-4da1-8bb7-5900993b9d26-kube-api-access-f5srj\") pod \"keystone-bootstrap-mh8wf\" (UID: \"f21141cf-d708-4da1-8bb7-5900993b9d26\") " pod="openstack-kuttl-tests/keystone-bootstrap-mh8wf" Dec 03 12:23:44 crc kubenswrapper[5052]: I1203 12:23:44.425362 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:23:44 crc kubenswrapper[5052]: I1203 12:23:44.427650 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:23:44 crc kubenswrapper[5052]: I1203 12:23:44.430852 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-scripts" Dec 03 12:23:44 crc kubenswrapper[5052]: I1203 12:23:44.431065 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-config-data" Dec 03 12:23:44 crc kubenswrapper[5052]: I1203 12:23:44.454576 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:23:44 crc kubenswrapper[5052]: I1203 12:23:44.472354 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/placement-db-sync-fspvj"] Dec 03 12:23:44 crc kubenswrapper[5052]: I1203 12:23:44.473431 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-db-sync-fspvj" Dec 03 12:23:44 crc kubenswrapper[5052]: I1203 12:23:44.478275 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"placement-scripts" Dec 03 12:23:44 crc kubenswrapper[5052]: I1203 12:23:44.478369 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"placement-config-data" Dec 03 12:23:44 crc kubenswrapper[5052]: I1203 12:23:44.478546 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"placement-placement-dockercfg-5v8pv" Dec 03 12:23:44 crc kubenswrapper[5052]: I1203 12:23:44.480826 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/f21141cf-d708-4da1-8bb7-5900993b9d26-scripts\") pod \"keystone-bootstrap-mh8wf\" (UID: \"f21141cf-d708-4da1-8bb7-5900993b9d26\") " pod="openstack-kuttl-tests/keystone-bootstrap-mh8wf" Dec 03 12:23:44 crc kubenswrapper[5052]: I1203 12:23:44.480860 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-f5srj\" (UniqueName: \"kubernetes.io/projected/f21141cf-d708-4da1-8bb7-5900993b9d26-kube-api-access-f5srj\") pod \"keystone-bootstrap-mh8wf\" (UID: \"f21141cf-d708-4da1-8bb7-5900993b9d26\") " pod="openstack-kuttl-tests/keystone-bootstrap-mh8wf" Dec 03 12:23:44 crc kubenswrapper[5052]: I1203 12:23:44.480910 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/f21141cf-d708-4da1-8bb7-5900993b9d26-fernet-keys\") pod \"keystone-bootstrap-mh8wf\" (UID: \"f21141cf-d708-4da1-8bb7-5900993b9d26\") " pod="openstack-kuttl-tests/keystone-bootstrap-mh8wf" Dec 03 12:23:44 crc kubenswrapper[5052]: I1203 12:23:44.480926 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/f21141cf-d708-4da1-8bb7-5900993b9d26-config-data\") pod \"keystone-bootstrap-mh8wf\" (UID: \"f21141cf-d708-4da1-8bb7-5900993b9d26\") " pod="openstack-kuttl-tests/keystone-bootstrap-mh8wf" Dec 03 12:23:44 crc kubenswrapper[5052]: I1203 12:23:44.480964 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/f21141cf-d708-4da1-8bb7-5900993b9d26-credential-keys\") pod \"keystone-bootstrap-mh8wf\" (UID: \"f21141cf-d708-4da1-8bb7-5900993b9d26\") " pod="openstack-kuttl-tests/keystone-bootstrap-mh8wf" Dec 03 12:23:44 crc kubenswrapper[5052]: I1203 12:23:44.480979 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f21141cf-d708-4da1-8bb7-5900993b9d26-combined-ca-bundle\") pod \"keystone-bootstrap-mh8wf\" (UID: \"f21141cf-d708-4da1-8bb7-5900993b9d26\") " pod="openstack-kuttl-tests/keystone-bootstrap-mh8wf" Dec 03 12:23:44 crc kubenswrapper[5052]: I1203 12:23:44.486404 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/f21141cf-d708-4da1-8bb7-5900993b9d26-fernet-keys\") pod \"keystone-bootstrap-mh8wf\" (UID: \"f21141cf-d708-4da1-8bb7-5900993b9d26\") " pod="openstack-kuttl-tests/keystone-bootstrap-mh8wf" Dec 03 12:23:44 crc kubenswrapper[5052]: I1203 12:23:44.487017 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/f21141cf-d708-4da1-8bb7-5900993b9d26-scripts\") pod \"keystone-bootstrap-mh8wf\" (UID: \"f21141cf-d708-4da1-8bb7-5900993b9d26\") " pod="openstack-kuttl-tests/keystone-bootstrap-mh8wf" Dec 03 12:23:44 crc kubenswrapper[5052]: I1203 12:23:44.487094 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/placement-db-sync-fspvj"] Dec 03 12:23:44 crc kubenswrapper[5052]: I1203 12:23:44.487640 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/f21141cf-d708-4da1-8bb7-5900993b9d26-config-data\") pod \"keystone-bootstrap-mh8wf\" (UID: \"f21141cf-d708-4da1-8bb7-5900993b9d26\") " pod="openstack-kuttl-tests/keystone-bootstrap-mh8wf" Dec 03 12:23:44 crc kubenswrapper[5052]: I1203 12:23:44.512598 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f21141cf-d708-4da1-8bb7-5900993b9d26-combined-ca-bundle\") pod \"keystone-bootstrap-mh8wf\" (UID: \"f21141cf-d708-4da1-8bb7-5900993b9d26\") " pod="openstack-kuttl-tests/keystone-bootstrap-mh8wf" Dec 03 12:23:44 crc kubenswrapper[5052]: I1203 12:23:44.516886 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/f21141cf-d708-4da1-8bb7-5900993b9d26-credential-keys\") pod \"keystone-bootstrap-mh8wf\" (UID: \"f21141cf-d708-4da1-8bb7-5900993b9d26\") " pod="openstack-kuttl-tests/keystone-bootstrap-mh8wf" Dec 03 12:23:44 crc kubenswrapper[5052]: I1203 12:23:44.518651 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-f5srj\" (UniqueName: \"kubernetes.io/projected/f21141cf-d708-4da1-8bb7-5900993b9d26-kube-api-access-f5srj\") pod \"keystone-bootstrap-mh8wf\" (UID: \"f21141cf-d708-4da1-8bb7-5900993b9d26\") " pod="openstack-kuttl-tests/keystone-bootstrap-mh8wf" Dec 03 12:23:44 crc kubenswrapper[5052]: I1203 12:23:44.561818 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-bootstrap-mh8wf" Dec 03 12:23:44 crc kubenswrapper[5052]: I1203 12:23:44.590862 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6af52f56-d687-43d2-a8da-1780e540ec92-config-data\") pod \"placement-db-sync-fspvj\" (UID: \"6af52f56-d687-43d2-a8da-1780e540ec92\") " pod="openstack-kuttl-tests/placement-db-sync-fspvj" Dec 03 12:23:44 crc kubenswrapper[5052]: I1203 12:23:44.590909 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/2660f107-7c9f-4d25-be86-3ba68a71be57-run-httpd\") pod \"ceilometer-0\" (UID: \"2660f107-7c9f-4d25-be86-3ba68a71be57\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:23:44 crc kubenswrapper[5052]: I1203 12:23:44.590948 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c2s4p\" (UniqueName: \"kubernetes.io/projected/2660f107-7c9f-4d25-be86-3ba68a71be57-kube-api-access-c2s4p\") pod \"ceilometer-0\" (UID: \"2660f107-7c9f-4d25-be86-3ba68a71be57\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:23:44 crc kubenswrapper[5052]: I1203 12:23:44.591036 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8698b\" (UniqueName: \"kubernetes.io/projected/6af52f56-d687-43d2-a8da-1780e540ec92-kube-api-access-8698b\") pod \"placement-db-sync-fspvj\" (UID: \"6af52f56-d687-43d2-a8da-1780e540ec92\") " pod="openstack-kuttl-tests/placement-db-sync-fspvj" Dec 03 12:23:44 crc kubenswrapper[5052]: I1203 12:23:44.591132 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/2660f107-7c9f-4d25-be86-3ba68a71be57-log-httpd\") pod \"ceilometer-0\" (UID: \"2660f107-7c9f-4d25-be86-3ba68a71be57\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:23:44 crc kubenswrapper[5052]: I1203 12:23:44.591154 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/6af52f56-d687-43d2-a8da-1780e540ec92-scripts\") pod \"placement-db-sync-fspvj\" (UID: \"6af52f56-d687-43d2-a8da-1780e540ec92\") " pod="openstack-kuttl-tests/placement-db-sync-fspvj" Dec 03 12:23:44 crc kubenswrapper[5052]: I1203 12:23:44.591185 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/2660f107-7c9f-4d25-be86-3ba68a71be57-scripts\") pod \"ceilometer-0\" (UID: \"2660f107-7c9f-4d25-be86-3ba68a71be57\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:23:44 crc kubenswrapper[5052]: I1203 12:23:44.591214 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/2660f107-7c9f-4d25-be86-3ba68a71be57-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"2660f107-7c9f-4d25-be86-3ba68a71be57\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:23:44 crc kubenswrapper[5052]: I1203 12:23:44.591286 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2660f107-7c9f-4d25-be86-3ba68a71be57-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"2660f107-7c9f-4d25-be86-3ba68a71be57\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:23:44 crc kubenswrapper[5052]: I1203 12:23:44.591308 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6af52f56-d687-43d2-a8da-1780e540ec92-combined-ca-bundle\") pod \"placement-db-sync-fspvj\" (UID: \"6af52f56-d687-43d2-a8da-1780e540ec92\") " pod="openstack-kuttl-tests/placement-db-sync-fspvj" Dec 03 12:23:44 crc kubenswrapper[5052]: I1203 12:23:44.591332 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/6af52f56-d687-43d2-a8da-1780e540ec92-logs\") pod \"placement-db-sync-fspvj\" (UID: \"6af52f56-d687-43d2-a8da-1780e540ec92\") " pod="openstack-kuttl-tests/placement-db-sync-fspvj" Dec 03 12:23:44 crc kubenswrapper[5052]: I1203 12:23:44.591355 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/2660f107-7c9f-4d25-be86-3ba68a71be57-config-data\") pod \"ceilometer-0\" (UID: \"2660f107-7c9f-4d25-be86-3ba68a71be57\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:23:44 crc kubenswrapper[5052]: I1203 12:23:44.694659 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/2660f107-7c9f-4d25-be86-3ba68a71be57-log-httpd\") pod \"ceilometer-0\" (UID: \"2660f107-7c9f-4d25-be86-3ba68a71be57\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:23:44 crc kubenswrapper[5052]: I1203 12:23:44.694711 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/6af52f56-d687-43d2-a8da-1780e540ec92-scripts\") pod \"placement-db-sync-fspvj\" (UID: \"6af52f56-d687-43d2-a8da-1780e540ec92\") " pod="openstack-kuttl-tests/placement-db-sync-fspvj" Dec 03 12:23:44 crc kubenswrapper[5052]: I1203 12:23:44.694737 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/2660f107-7c9f-4d25-be86-3ba68a71be57-scripts\") pod \"ceilometer-0\" (UID: \"2660f107-7c9f-4d25-be86-3ba68a71be57\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:23:44 crc kubenswrapper[5052]: I1203 12:23:44.694758 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/2660f107-7c9f-4d25-be86-3ba68a71be57-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"2660f107-7c9f-4d25-be86-3ba68a71be57\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:23:44 crc kubenswrapper[5052]: I1203 12:23:44.694804 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2660f107-7c9f-4d25-be86-3ba68a71be57-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"2660f107-7c9f-4d25-be86-3ba68a71be57\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:23:44 crc kubenswrapper[5052]: I1203 12:23:44.694821 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6af52f56-d687-43d2-a8da-1780e540ec92-combined-ca-bundle\") pod \"placement-db-sync-fspvj\" (UID: \"6af52f56-d687-43d2-a8da-1780e540ec92\") " pod="openstack-kuttl-tests/placement-db-sync-fspvj" Dec 03 12:23:44 crc kubenswrapper[5052]: I1203 12:23:44.694837 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/6af52f56-d687-43d2-a8da-1780e540ec92-logs\") pod \"placement-db-sync-fspvj\" (UID: \"6af52f56-d687-43d2-a8da-1780e540ec92\") " pod="openstack-kuttl-tests/placement-db-sync-fspvj" Dec 03 12:23:44 crc kubenswrapper[5052]: I1203 12:23:44.694855 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/2660f107-7c9f-4d25-be86-3ba68a71be57-config-data\") pod \"ceilometer-0\" (UID: \"2660f107-7c9f-4d25-be86-3ba68a71be57\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:23:44 crc kubenswrapper[5052]: I1203 12:23:44.694875 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6af52f56-d687-43d2-a8da-1780e540ec92-config-data\") pod \"placement-db-sync-fspvj\" (UID: \"6af52f56-d687-43d2-a8da-1780e540ec92\") " pod="openstack-kuttl-tests/placement-db-sync-fspvj" Dec 03 12:23:44 crc kubenswrapper[5052]: I1203 12:23:44.694889 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/2660f107-7c9f-4d25-be86-3ba68a71be57-run-httpd\") pod \"ceilometer-0\" (UID: \"2660f107-7c9f-4d25-be86-3ba68a71be57\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:23:44 crc kubenswrapper[5052]: I1203 12:23:44.694914 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-c2s4p\" (UniqueName: \"kubernetes.io/projected/2660f107-7c9f-4d25-be86-3ba68a71be57-kube-api-access-c2s4p\") pod \"ceilometer-0\" (UID: \"2660f107-7c9f-4d25-be86-3ba68a71be57\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:23:44 crc kubenswrapper[5052]: I1203 12:23:44.694956 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-8698b\" (UniqueName: \"kubernetes.io/projected/6af52f56-d687-43d2-a8da-1780e540ec92-kube-api-access-8698b\") pod \"placement-db-sync-fspvj\" (UID: \"6af52f56-d687-43d2-a8da-1780e540ec92\") " pod="openstack-kuttl-tests/placement-db-sync-fspvj" Dec 03 12:23:44 crc kubenswrapper[5052]: I1203 12:23:44.695832 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/6af52f56-d687-43d2-a8da-1780e540ec92-logs\") pod \"placement-db-sync-fspvj\" (UID: \"6af52f56-d687-43d2-a8da-1780e540ec92\") " pod="openstack-kuttl-tests/placement-db-sync-fspvj" Dec 03 12:23:44 crc kubenswrapper[5052]: I1203 12:23:44.700676 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/2660f107-7c9f-4d25-be86-3ba68a71be57-run-httpd\") pod \"ceilometer-0\" (UID: \"2660f107-7c9f-4d25-be86-3ba68a71be57\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:23:44 crc kubenswrapper[5052]: I1203 12:23:44.700971 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/2660f107-7c9f-4d25-be86-3ba68a71be57-log-httpd\") pod \"ceilometer-0\" (UID: \"2660f107-7c9f-4d25-be86-3ba68a71be57\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:23:44 crc kubenswrapper[5052]: I1203 12:23:44.701960 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6af52f56-d687-43d2-a8da-1780e540ec92-combined-ca-bundle\") pod \"placement-db-sync-fspvj\" (UID: \"6af52f56-d687-43d2-a8da-1780e540ec92\") " pod="openstack-kuttl-tests/placement-db-sync-fspvj" Dec 03 12:23:44 crc kubenswrapper[5052]: I1203 12:23:44.704022 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/2660f107-7c9f-4d25-be86-3ba68a71be57-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"2660f107-7c9f-4d25-be86-3ba68a71be57\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:23:44 crc kubenswrapper[5052]: I1203 12:23:44.704285 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/6af52f56-d687-43d2-a8da-1780e540ec92-scripts\") pod \"placement-db-sync-fspvj\" (UID: \"6af52f56-d687-43d2-a8da-1780e540ec92\") " pod="openstack-kuttl-tests/placement-db-sync-fspvj" Dec 03 12:23:44 crc kubenswrapper[5052]: I1203 12:23:44.705375 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/2660f107-7c9f-4d25-be86-3ba68a71be57-config-data\") pod \"ceilometer-0\" (UID: \"2660f107-7c9f-4d25-be86-3ba68a71be57\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:23:44 crc kubenswrapper[5052]: I1203 12:23:44.712171 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6af52f56-d687-43d2-a8da-1780e540ec92-config-data\") pod \"placement-db-sync-fspvj\" (UID: \"6af52f56-d687-43d2-a8da-1780e540ec92\") " pod="openstack-kuttl-tests/placement-db-sync-fspvj" Dec 03 12:23:44 crc kubenswrapper[5052]: I1203 12:23:44.719207 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2660f107-7c9f-4d25-be86-3ba68a71be57-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"2660f107-7c9f-4d25-be86-3ba68a71be57\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:23:44 crc kubenswrapper[5052]: I1203 12:23:44.719619 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/2660f107-7c9f-4d25-be86-3ba68a71be57-scripts\") pod \"ceilometer-0\" (UID: \"2660f107-7c9f-4d25-be86-3ba68a71be57\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:23:44 crc kubenswrapper[5052]: I1203 12:23:44.719798 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-8698b\" (UniqueName: \"kubernetes.io/projected/6af52f56-d687-43d2-a8da-1780e540ec92-kube-api-access-8698b\") pod \"placement-db-sync-fspvj\" (UID: \"6af52f56-d687-43d2-a8da-1780e540ec92\") " pod="openstack-kuttl-tests/placement-db-sync-fspvj" Dec 03 12:23:44 crc kubenswrapper[5052]: I1203 12:23:44.724148 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-c2s4p\" (UniqueName: \"kubernetes.io/projected/2660f107-7c9f-4d25-be86-3ba68a71be57-kube-api-access-c2s4p\") pod \"ceilometer-0\" (UID: \"2660f107-7c9f-4d25-be86-3ba68a71be57\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:23:44 crc kubenswrapper[5052]: I1203 12:23:44.746084 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:23:45 crc kubenswrapper[5052]: I1203 12:23:44.885345 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-db-sync-fspvj" Dec 03 12:23:45 crc kubenswrapper[5052]: I1203 12:23:45.045052 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-bootstrap-mh8wf"] Dec 03 12:23:45 crc kubenswrapper[5052]: W1203 12:23:45.047346 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf21141cf_d708_4da1_8bb7_5900993b9d26.slice/crio-f1a94ad5cf4554f9993c31092e25b665ec60c965265b5dbd78fee090ebc7482a WatchSource:0}: Error finding container f1a94ad5cf4554f9993c31092e25b665ec60c965265b5dbd78fee090ebc7482a: Status 404 returned error can't find the container with id f1a94ad5cf4554f9993c31092e25b665ec60c965265b5dbd78fee090ebc7482a Dec 03 12:23:45 crc kubenswrapper[5052]: I1203 12:23:45.090475 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-bootstrap-mh8wf" event={"ID":"f21141cf-d708-4da1-8bb7-5900993b9d26","Type":"ContainerStarted","Data":"f1a94ad5cf4554f9993c31092e25b665ec60c965265b5dbd78fee090ebc7482a"} Dec 03 12:23:45 crc kubenswrapper[5052]: I1203 12:23:45.351796 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:23:45 crc kubenswrapper[5052]: I1203 12:23:45.353441 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:23:45 crc kubenswrapper[5052]: I1203 12:23:45.356047 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-default-external-config-data" Dec 03 12:23:45 crc kubenswrapper[5052]: I1203 12:23:45.357646 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-scripts" Dec 03 12:23:45 crc kubenswrapper[5052]: I1203 12:23:45.357763 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-glance-dockercfg-cd9ns" Dec 03 12:23:45 crc kubenswrapper[5052]: I1203 12:23:45.365927 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:23:45 crc kubenswrapper[5052]: I1203 12:23:45.440432 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:23:45 crc kubenswrapper[5052]: I1203 12:23:45.442291 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:23:45 crc kubenswrapper[5052]: I1203 12:23:45.443928 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-default-internal-config-data" Dec 03 12:23:45 crc kubenswrapper[5052]: I1203 12:23:45.459610 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:23:45 crc kubenswrapper[5052]: I1203 12:23:45.508231 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/b5cc20b6-3503-42f6-b497-8ef83e55317d-config-data\") pod \"glance-default-external-api-0\" (UID: \"b5cc20b6-3503-42f6-b497-8ef83e55317d\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:23:45 crc kubenswrapper[5052]: I1203 12:23:45.508299 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage07-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage07-crc\") pod \"glance-default-external-api-0\" (UID: \"b5cc20b6-3503-42f6-b497-8ef83e55317d\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:23:45 crc kubenswrapper[5052]: I1203 12:23:45.508321 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/b5cc20b6-3503-42f6-b497-8ef83e55317d-httpd-run\") pod \"glance-default-external-api-0\" (UID: \"b5cc20b6-3503-42f6-b497-8ef83e55317d\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:23:45 crc kubenswrapper[5052]: I1203 12:23:45.508341 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/b5cc20b6-3503-42f6-b497-8ef83e55317d-scripts\") pod \"glance-default-external-api-0\" (UID: \"b5cc20b6-3503-42f6-b497-8ef83e55317d\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:23:45 crc kubenswrapper[5052]: I1203 12:23:45.508377 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dhsrv\" (UniqueName: \"kubernetes.io/projected/b5cc20b6-3503-42f6-b497-8ef83e55317d-kube-api-access-dhsrv\") pod \"glance-default-external-api-0\" (UID: \"b5cc20b6-3503-42f6-b497-8ef83e55317d\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:23:45 crc kubenswrapper[5052]: I1203 12:23:45.508411 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b5cc20b6-3503-42f6-b497-8ef83e55317d-combined-ca-bundle\") pod \"glance-default-external-api-0\" (UID: \"b5cc20b6-3503-42f6-b497-8ef83e55317d\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:23:45 crc kubenswrapper[5052]: I1203 12:23:45.508442 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/b5cc20b6-3503-42f6-b497-8ef83e55317d-logs\") pod \"glance-default-external-api-0\" (UID: \"b5cc20b6-3503-42f6-b497-8ef83e55317d\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:23:45 crc kubenswrapper[5052]: I1203 12:23:45.610026 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/b5cc20b6-3503-42f6-b497-8ef83e55317d-logs\") pod \"glance-default-external-api-0\" (UID: \"b5cc20b6-3503-42f6-b497-8ef83e55317d\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:23:45 crc kubenswrapper[5052]: I1203 12:23:45.610072 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/47bc7c32-d14b-4137-be0d-469efe7b3c9e-logs\") pod \"glance-default-internal-api-0\" (UID: \"47bc7c32-d14b-4137-be0d-469efe7b3c9e\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:23:45 crc kubenswrapper[5052]: I1203 12:23:45.610127 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage09-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage09-crc\") pod \"glance-default-internal-api-0\" (UID: \"47bc7c32-d14b-4137-be0d-469efe7b3c9e\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:23:45 crc kubenswrapper[5052]: I1203 12:23:45.610150 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7pqgm\" (UniqueName: \"kubernetes.io/projected/47bc7c32-d14b-4137-be0d-469efe7b3c9e-kube-api-access-7pqgm\") pod \"glance-default-internal-api-0\" (UID: \"47bc7c32-d14b-4137-be0d-469efe7b3c9e\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:23:45 crc kubenswrapper[5052]: I1203 12:23:45.610174 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/b5cc20b6-3503-42f6-b497-8ef83e55317d-config-data\") pod \"glance-default-external-api-0\" (UID: \"b5cc20b6-3503-42f6-b497-8ef83e55317d\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:23:45 crc kubenswrapper[5052]: I1203 12:23:45.610195 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/47bc7c32-d14b-4137-be0d-469efe7b3c9e-config-data\") pod \"glance-default-internal-api-0\" (UID: \"47bc7c32-d14b-4137-be0d-469efe7b3c9e\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:23:45 crc kubenswrapper[5052]: I1203 12:23:45.610218 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/47bc7c32-d14b-4137-be0d-469efe7b3c9e-combined-ca-bundle\") pod \"glance-default-internal-api-0\" (UID: \"47bc7c32-d14b-4137-be0d-469efe7b3c9e\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:23:45 crc kubenswrapper[5052]: I1203 12:23:45.610257 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage07-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage07-crc\") pod \"glance-default-external-api-0\" (UID: \"b5cc20b6-3503-42f6-b497-8ef83e55317d\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:23:45 crc kubenswrapper[5052]: I1203 12:23:45.610276 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/b5cc20b6-3503-42f6-b497-8ef83e55317d-httpd-run\") pod \"glance-default-external-api-0\" (UID: \"b5cc20b6-3503-42f6-b497-8ef83e55317d\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:23:45 crc kubenswrapper[5052]: I1203 12:23:45.610298 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/b5cc20b6-3503-42f6-b497-8ef83e55317d-scripts\") pod \"glance-default-external-api-0\" (UID: \"b5cc20b6-3503-42f6-b497-8ef83e55317d\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:23:45 crc kubenswrapper[5052]: I1203 12:23:45.610324 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/47bc7c32-d14b-4137-be0d-469efe7b3c9e-httpd-run\") pod \"glance-default-internal-api-0\" (UID: \"47bc7c32-d14b-4137-be0d-469efe7b3c9e\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:23:45 crc kubenswrapper[5052]: I1203 12:23:45.610347 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-dhsrv\" (UniqueName: \"kubernetes.io/projected/b5cc20b6-3503-42f6-b497-8ef83e55317d-kube-api-access-dhsrv\") pod \"glance-default-external-api-0\" (UID: \"b5cc20b6-3503-42f6-b497-8ef83e55317d\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:23:45 crc kubenswrapper[5052]: I1203 12:23:45.610364 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/47bc7c32-d14b-4137-be0d-469efe7b3c9e-scripts\") pod \"glance-default-internal-api-0\" (UID: \"47bc7c32-d14b-4137-be0d-469efe7b3c9e\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:23:45 crc kubenswrapper[5052]: I1203 12:23:45.610397 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b5cc20b6-3503-42f6-b497-8ef83e55317d-combined-ca-bundle\") pod \"glance-default-external-api-0\" (UID: \"b5cc20b6-3503-42f6-b497-8ef83e55317d\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:23:45 crc kubenswrapper[5052]: I1203 12:23:45.611087 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage07-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage07-crc\") pod \"glance-default-external-api-0\" (UID: \"b5cc20b6-3503-42f6-b497-8ef83e55317d\") device mount path \"/mnt/openstack/pv07\"" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:23:45 crc kubenswrapper[5052]: I1203 12:23:45.611191 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/b5cc20b6-3503-42f6-b497-8ef83e55317d-httpd-run\") pod \"glance-default-external-api-0\" (UID: \"b5cc20b6-3503-42f6-b497-8ef83e55317d\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:23:45 crc kubenswrapper[5052]: I1203 12:23:45.611107 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/b5cc20b6-3503-42f6-b497-8ef83e55317d-logs\") pod \"glance-default-external-api-0\" (UID: \"b5cc20b6-3503-42f6-b497-8ef83e55317d\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:23:45 crc kubenswrapper[5052]: I1203 12:23:45.615120 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b5cc20b6-3503-42f6-b497-8ef83e55317d-combined-ca-bundle\") pod \"glance-default-external-api-0\" (UID: \"b5cc20b6-3503-42f6-b497-8ef83e55317d\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:23:45 crc kubenswrapper[5052]: I1203 12:23:45.615367 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/b5cc20b6-3503-42f6-b497-8ef83e55317d-config-data\") pod \"glance-default-external-api-0\" (UID: \"b5cc20b6-3503-42f6-b497-8ef83e55317d\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:23:45 crc kubenswrapper[5052]: I1203 12:23:45.616124 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/b5cc20b6-3503-42f6-b497-8ef83e55317d-scripts\") pod \"glance-default-external-api-0\" (UID: \"b5cc20b6-3503-42f6-b497-8ef83e55317d\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:23:45 crc kubenswrapper[5052]: I1203 12:23:45.631474 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-dhsrv\" (UniqueName: \"kubernetes.io/projected/b5cc20b6-3503-42f6-b497-8ef83e55317d-kube-api-access-dhsrv\") pod \"glance-default-external-api-0\" (UID: \"b5cc20b6-3503-42f6-b497-8ef83e55317d\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:23:45 crc kubenswrapper[5052]: I1203 12:23:45.664965 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage07-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage07-crc\") pod \"glance-default-external-api-0\" (UID: \"b5cc20b6-3503-42f6-b497-8ef83e55317d\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:23:45 crc kubenswrapper[5052]: I1203 12:23:45.677063 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:23:45 crc kubenswrapper[5052]: I1203 12:23:45.737186 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage09-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage09-crc\") pod \"glance-default-internal-api-0\" (UID: \"47bc7c32-d14b-4137-be0d-469efe7b3c9e\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:23:45 crc kubenswrapper[5052]: I1203 12:23:45.737227 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-7pqgm\" (UniqueName: \"kubernetes.io/projected/47bc7c32-d14b-4137-be0d-469efe7b3c9e-kube-api-access-7pqgm\") pod \"glance-default-internal-api-0\" (UID: \"47bc7c32-d14b-4137-be0d-469efe7b3c9e\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:23:45 crc kubenswrapper[5052]: I1203 12:23:45.737263 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/47bc7c32-d14b-4137-be0d-469efe7b3c9e-config-data\") pod \"glance-default-internal-api-0\" (UID: \"47bc7c32-d14b-4137-be0d-469efe7b3c9e\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:23:45 crc kubenswrapper[5052]: I1203 12:23:45.737278 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/47bc7c32-d14b-4137-be0d-469efe7b3c9e-combined-ca-bundle\") pod \"glance-default-internal-api-0\" (UID: \"47bc7c32-d14b-4137-be0d-469efe7b3c9e\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:23:45 crc kubenswrapper[5052]: I1203 12:23:45.737339 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/47bc7c32-d14b-4137-be0d-469efe7b3c9e-httpd-run\") pod \"glance-default-internal-api-0\" (UID: \"47bc7c32-d14b-4137-be0d-469efe7b3c9e\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:23:45 crc kubenswrapper[5052]: I1203 12:23:45.737361 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/47bc7c32-d14b-4137-be0d-469efe7b3c9e-scripts\") pod \"glance-default-internal-api-0\" (UID: \"47bc7c32-d14b-4137-be0d-469efe7b3c9e\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:23:45 crc kubenswrapper[5052]: I1203 12:23:45.737408 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/47bc7c32-d14b-4137-be0d-469efe7b3c9e-logs\") pod \"glance-default-internal-api-0\" (UID: \"47bc7c32-d14b-4137-be0d-469efe7b3c9e\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:23:45 crc kubenswrapper[5052]: I1203 12:23:45.737871 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/47bc7c32-d14b-4137-be0d-469efe7b3c9e-logs\") pod \"glance-default-internal-api-0\" (UID: \"47bc7c32-d14b-4137-be0d-469efe7b3c9e\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:23:45 crc kubenswrapper[5052]: I1203 12:23:45.738489 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage09-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage09-crc\") pod \"glance-default-internal-api-0\" (UID: \"47bc7c32-d14b-4137-be0d-469efe7b3c9e\") device mount path \"/mnt/openstack/pv09\"" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:23:45 crc kubenswrapper[5052]: I1203 12:23:45.738606 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/47bc7c32-d14b-4137-be0d-469efe7b3c9e-httpd-run\") pod \"glance-default-internal-api-0\" (UID: \"47bc7c32-d14b-4137-be0d-469efe7b3c9e\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:23:45 crc kubenswrapper[5052]: I1203 12:23:45.753761 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/47bc7c32-d14b-4137-be0d-469efe7b3c9e-scripts\") pod \"glance-default-internal-api-0\" (UID: \"47bc7c32-d14b-4137-be0d-469efe7b3c9e\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:23:45 crc kubenswrapper[5052]: I1203 12:23:45.755403 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/47bc7c32-d14b-4137-be0d-469efe7b3c9e-config-data\") pod \"glance-default-internal-api-0\" (UID: \"47bc7c32-d14b-4137-be0d-469efe7b3c9e\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:23:45 crc kubenswrapper[5052]: I1203 12:23:45.762826 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/47bc7c32-d14b-4137-be0d-469efe7b3c9e-combined-ca-bundle\") pod \"glance-default-internal-api-0\" (UID: \"47bc7c32-d14b-4137-be0d-469efe7b3c9e\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:23:45 crc kubenswrapper[5052]: I1203 12:23:45.773237 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-7pqgm\" (UniqueName: \"kubernetes.io/projected/47bc7c32-d14b-4137-be0d-469efe7b3c9e-kube-api-access-7pqgm\") pod \"glance-default-internal-api-0\" (UID: \"47bc7c32-d14b-4137-be0d-469efe7b3c9e\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:23:45 crc kubenswrapper[5052]: I1203 12:23:45.839336 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:23:45 crc kubenswrapper[5052]: I1203 12:23:45.855945 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage09-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage09-crc\") pod \"glance-default-internal-api-0\" (UID: \"47bc7c32-d14b-4137-be0d-469efe7b3c9e\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:23:45 crc kubenswrapper[5052]: I1203 12:23:45.908342 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:23:45 crc kubenswrapper[5052]: I1203 12:23:45.909232 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:23:46 crc kubenswrapper[5052]: I1203 12:23:46.084422 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:23:46 crc kubenswrapper[5052]: I1203 12:23:46.103201 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/placement-db-sync-fspvj"] Dec 03 12:23:46 crc kubenswrapper[5052]: I1203 12:23:46.104142 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-bootstrap-mh8wf" event={"ID":"f21141cf-d708-4da1-8bb7-5900993b9d26","Type":"ContainerStarted","Data":"67ce4d39f2bdc99c3f3be2f5c14c987070c0b90d869d6a4344098656d6746ae9"} Dec 03 12:23:46 crc kubenswrapper[5052]: I1203 12:23:46.135274 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/keystone-bootstrap-mh8wf" podStartSLOduration=2.135254666 podStartE2EDuration="2.135254666s" podCreationTimestamp="2025-12-03 12:23:44 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:23:46.13118003 +0000 UTC m=+3816.719421794" watchObservedRunningTime="2025-12-03 12:23:46.135254666 +0000 UTC m=+3816.723496430" Dec 03 12:23:46 crc kubenswrapper[5052]: I1203 12:23:46.288388 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:23:46 crc kubenswrapper[5052]: W1203 12:23:46.292626 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb5cc20b6_3503_42f6_b497_8ef83e55317d.slice/crio-372f816b2624ffd8bf73fd50a48ed5a5b138359619500f43b64ed9aa5ccad162 WatchSource:0}: Error finding container 372f816b2624ffd8bf73fd50a48ed5a5b138359619500f43b64ed9aa5ccad162: Status 404 returned error can't find the container with id 372f816b2624ffd8bf73fd50a48ed5a5b138359619500f43b64ed9aa5ccad162 Dec 03 12:23:46 crc kubenswrapper[5052]: I1203 12:23:46.469144 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:23:46 crc kubenswrapper[5052]: W1203 12:23:46.478323 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod47bc7c32_d14b_4137_be0d_469efe7b3c9e.slice/crio-5c968e02667e6873c21095a11d2e3485d1f3e327b4a5d21f39fdba401edb7b2b WatchSource:0}: Error finding container 5c968e02667e6873c21095a11d2e3485d1f3e327b4a5d21f39fdba401edb7b2b: Status 404 returned error can't find the container with id 5c968e02667e6873c21095a11d2e3485d1f3e327b4a5d21f39fdba401edb7b2b Dec 03 12:23:47 crc kubenswrapper[5052]: I1203 12:23:47.129655 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"2660f107-7c9f-4d25-be86-3ba68a71be57","Type":"ContainerStarted","Data":"ed818d88a39bfa952249a233cfe4ea201ee58d711ab8360da187109336679ef1"} Dec 03 12:23:47 crc kubenswrapper[5052]: I1203 12:23:47.130319 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"2660f107-7c9f-4d25-be86-3ba68a71be57","Type":"ContainerStarted","Data":"74ee332ddf2ec00702800023a9ca2733054e8ce5d7e675f3432237abdef4b8cd"} Dec 03 12:23:47 crc kubenswrapper[5052]: I1203 12:23:47.133388 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"b5cc20b6-3503-42f6-b497-8ef83e55317d","Type":"ContainerStarted","Data":"082703df4163afc55b34093947ed1259ed87bfd80ee0c0d6455108fc6bffe2c2"} Dec 03 12:23:47 crc kubenswrapper[5052]: I1203 12:23:47.133451 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"b5cc20b6-3503-42f6-b497-8ef83e55317d","Type":"ContainerStarted","Data":"372f816b2624ffd8bf73fd50a48ed5a5b138359619500f43b64ed9aa5ccad162"} Dec 03 12:23:47 crc kubenswrapper[5052]: I1203 12:23:47.139313 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-db-sync-fspvj" event={"ID":"6af52f56-d687-43d2-a8da-1780e540ec92","Type":"ContainerStarted","Data":"1866d9e97345f3e78cc8443ebbf0430328e7ab4a4ed32a162d7d7f17a4786a03"} Dec 03 12:23:47 crc kubenswrapper[5052]: I1203 12:23:47.139372 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-db-sync-fspvj" event={"ID":"6af52f56-d687-43d2-a8da-1780e540ec92","Type":"ContainerStarted","Data":"9cafba3cbef85cf1980e296d4145eb069c9366691a317fb35d486130de72ac26"} Dec 03 12:23:47 crc kubenswrapper[5052]: I1203 12:23:47.152348 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"47bc7c32-d14b-4137-be0d-469efe7b3c9e","Type":"ContainerStarted","Data":"4457e54a6f4b4579b71b4df60084a738365b0be2f091e39ea0590c7628e18571"} Dec 03 12:23:47 crc kubenswrapper[5052]: I1203 12:23:47.152598 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"47bc7c32-d14b-4137-be0d-469efe7b3c9e","Type":"ContainerStarted","Data":"5c968e02667e6873c21095a11d2e3485d1f3e327b4a5d21f39fdba401edb7b2b"} Dec 03 12:23:47 crc kubenswrapper[5052]: I1203 12:23:47.163358 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/placement-db-sync-fspvj" podStartSLOduration=3.16333509 podStartE2EDuration="3.16333509s" podCreationTimestamp="2025-12-03 12:23:44 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:23:47.160643293 +0000 UTC m=+3817.748885057" watchObservedRunningTime="2025-12-03 12:23:47.16333509 +0000 UTC m=+3817.751576854" Dec 03 12:23:47 crc kubenswrapper[5052]: I1203 12:23:47.616590 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/barbican-db-sync-j9cr4"] Dec 03 12:23:47 crc kubenswrapper[5052]: I1203 12:23:47.630999 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-db-sync-j9cr4" Dec 03 12:23:47 crc kubenswrapper[5052]: I1203 12:23:47.633643 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"barbican-config-data" Dec 03 12:23:47 crc kubenswrapper[5052]: I1203 12:23:47.634267 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"barbican-barbican-dockercfg-jfqpw" Dec 03 12:23:47 crc kubenswrapper[5052]: I1203 12:23:47.646776 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-db-sync-j9cr4"] Dec 03 12:23:47 crc kubenswrapper[5052]: I1203 12:23:47.763429 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/cinder-db-sync-j76tm"] Dec 03 12:23:47 crc kubenswrapper[5052]: I1203 12:23:47.764770 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-db-sync-j76tm" Dec 03 12:23:47 crc kubenswrapper[5052]: I1203 12:23:47.767167 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cinder-config-data" Dec 03 12:23:47 crc kubenswrapper[5052]: I1203 12:23:47.768715 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cinder-scripts" Dec 03 12:23:47 crc kubenswrapper[5052]: I1203 12:23:47.772800 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cinder-cinder-dockercfg-zd22l" Dec 03 12:23:47 crc kubenswrapper[5052]: I1203 12:23:47.782616 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-db-sync-j76tm"] Dec 03 12:23:47 crc kubenswrapper[5052]: I1203 12:23:47.804771 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fz48g\" (UniqueName: \"kubernetes.io/projected/fa8494cf-ad3c-4998-a3fd-a39863f72aff-kube-api-access-fz48g\") pod \"barbican-db-sync-j9cr4\" (UID: \"fa8494cf-ad3c-4998-a3fd-a39863f72aff\") " pod="openstack-kuttl-tests/barbican-db-sync-j9cr4" Dec 03 12:23:47 crc kubenswrapper[5052]: I1203 12:23:47.804973 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/fa8494cf-ad3c-4998-a3fd-a39863f72aff-combined-ca-bundle\") pod \"barbican-db-sync-j9cr4\" (UID: \"fa8494cf-ad3c-4998-a3fd-a39863f72aff\") " pod="openstack-kuttl-tests/barbican-db-sync-j9cr4" Dec 03 12:23:47 crc kubenswrapper[5052]: I1203 12:23:47.805091 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/fa8494cf-ad3c-4998-a3fd-a39863f72aff-db-sync-config-data\") pod \"barbican-db-sync-j9cr4\" (UID: \"fa8494cf-ad3c-4998-a3fd-a39863f72aff\") " pod="openstack-kuttl-tests/barbican-db-sync-j9cr4" Dec 03 12:23:47 crc kubenswrapper[5052]: I1203 12:23:47.906689 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/516cc68a-d42b-40c8-8ca7-a093f47f10d5-scripts\") pod \"cinder-db-sync-j76tm\" (UID: \"516cc68a-d42b-40c8-8ca7-a093f47f10d5\") " pod="openstack-kuttl-tests/cinder-db-sync-j76tm" Dec 03 12:23:47 crc kubenswrapper[5052]: I1203 12:23:47.906746 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/516cc68a-d42b-40c8-8ca7-a093f47f10d5-db-sync-config-data\") pod \"cinder-db-sync-j76tm\" (UID: \"516cc68a-d42b-40c8-8ca7-a093f47f10d5\") " pod="openstack-kuttl-tests/cinder-db-sync-j76tm" Dec 03 12:23:47 crc kubenswrapper[5052]: I1203 12:23:47.906793 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/fa8494cf-ad3c-4998-a3fd-a39863f72aff-combined-ca-bundle\") pod \"barbican-db-sync-j9cr4\" (UID: \"fa8494cf-ad3c-4998-a3fd-a39863f72aff\") " pod="openstack-kuttl-tests/barbican-db-sync-j9cr4" Dec 03 12:23:47 crc kubenswrapper[5052]: I1203 12:23:47.907007 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/fa8494cf-ad3c-4998-a3fd-a39863f72aff-db-sync-config-data\") pod \"barbican-db-sync-j9cr4\" (UID: \"fa8494cf-ad3c-4998-a3fd-a39863f72aff\") " pod="openstack-kuttl-tests/barbican-db-sync-j9cr4" Dec 03 12:23:47 crc kubenswrapper[5052]: I1203 12:23:47.907069 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/516cc68a-d42b-40c8-8ca7-a093f47f10d5-combined-ca-bundle\") pod \"cinder-db-sync-j76tm\" (UID: \"516cc68a-d42b-40c8-8ca7-a093f47f10d5\") " pod="openstack-kuttl-tests/cinder-db-sync-j76tm" Dec 03 12:23:47 crc kubenswrapper[5052]: I1203 12:23:47.907129 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/516cc68a-d42b-40c8-8ca7-a093f47f10d5-config-data\") pod \"cinder-db-sync-j76tm\" (UID: \"516cc68a-d42b-40c8-8ca7-a093f47f10d5\") " pod="openstack-kuttl-tests/cinder-db-sync-j76tm" Dec 03 12:23:47 crc kubenswrapper[5052]: I1203 12:23:47.907229 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-fz48g\" (UniqueName: \"kubernetes.io/projected/fa8494cf-ad3c-4998-a3fd-a39863f72aff-kube-api-access-fz48g\") pod \"barbican-db-sync-j9cr4\" (UID: \"fa8494cf-ad3c-4998-a3fd-a39863f72aff\") " pod="openstack-kuttl-tests/barbican-db-sync-j9cr4" Dec 03 12:23:47 crc kubenswrapper[5052]: I1203 12:23:47.907324 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/516cc68a-d42b-40c8-8ca7-a093f47f10d5-etc-machine-id\") pod \"cinder-db-sync-j76tm\" (UID: \"516cc68a-d42b-40c8-8ca7-a093f47f10d5\") " pod="openstack-kuttl-tests/cinder-db-sync-j76tm" Dec 03 12:23:47 crc kubenswrapper[5052]: I1203 12:23:47.907373 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jczdq\" (UniqueName: \"kubernetes.io/projected/516cc68a-d42b-40c8-8ca7-a093f47f10d5-kube-api-access-jczdq\") pod \"cinder-db-sync-j76tm\" (UID: \"516cc68a-d42b-40c8-8ca7-a093f47f10d5\") " pod="openstack-kuttl-tests/cinder-db-sync-j76tm" Dec 03 12:23:47 crc kubenswrapper[5052]: I1203 12:23:47.911164 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/fa8494cf-ad3c-4998-a3fd-a39863f72aff-combined-ca-bundle\") pod \"barbican-db-sync-j9cr4\" (UID: \"fa8494cf-ad3c-4998-a3fd-a39863f72aff\") " pod="openstack-kuttl-tests/barbican-db-sync-j9cr4" Dec 03 12:23:47 crc kubenswrapper[5052]: I1203 12:23:47.914012 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/fa8494cf-ad3c-4998-a3fd-a39863f72aff-db-sync-config-data\") pod \"barbican-db-sync-j9cr4\" (UID: \"fa8494cf-ad3c-4998-a3fd-a39863f72aff\") " pod="openstack-kuttl-tests/barbican-db-sync-j9cr4" Dec 03 12:23:47 crc kubenswrapper[5052]: I1203 12:23:47.931149 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-fz48g\" (UniqueName: \"kubernetes.io/projected/fa8494cf-ad3c-4998-a3fd-a39863f72aff-kube-api-access-fz48g\") pod \"barbican-db-sync-j9cr4\" (UID: \"fa8494cf-ad3c-4998-a3fd-a39863f72aff\") " pod="openstack-kuttl-tests/barbican-db-sync-j9cr4" Dec 03 12:23:47 crc kubenswrapper[5052]: I1203 12:23:47.947973 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/neutron-db-sync-ggltp"] Dec 03 12:23:47 crc kubenswrapper[5052]: I1203 12:23:47.949101 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-db-sync-ggltp" Dec 03 12:23:47 crc kubenswrapper[5052]: I1203 12:23:47.951065 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-db-sync-j9cr4" Dec 03 12:23:47 crc kubenswrapper[5052]: I1203 12:23:47.960186 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"neutron-neutron-dockercfg-bvk99" Dec 03 12:23:47 crc kubenswrapper[5052]: I1203 12:23:47.960275 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"neutron-config" Dec 03 12:23:47 crc kubenswrapper[5052]: I1203 12:23:47.960452 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"neutron-httpd-config" Dec 03 12:23:47 crc kubenswrapper[5052]: I1203 12:23:47.977082 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-db-sync-ggltp"] Dec 03 12:23:48 crc kubenswrapper[5052]: I1203 12:23:48.010405 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/516cc68a-d42b-40c8-8ca7-a093f47f10d5-etc-machine-id\") pod \"cinder-db-sync-j76tm\" (UID: \"516cc68a-d42b-40c8-8ca7-a093f47f10d5\") " pod="openstack-kuttl-tests/cinder-db-sync-j76tm" Dec 03 12:23:48 crc kubenswrapper[5052]: I1203 12:23:48.010743 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-jczdq\" (UniqueName: \"kubernetes.io/projected/516cc68a-d42b-40c8-8ca7-a093f47f10d5-kube-api-access-jczdq\") pod \"cinder-db-sync-j76tm\" (UID: \"516cc68a-d42b-40c8-8ca7-a093f47f10d5\") " pod="openstack-kuttl-tests/cinder-db-sync-j76tm" Dec 03 12:23:48 crc kubenswrapper[5052]: I1203 12:23:48.010775 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/516cc68a-d42b-40c8-8ca7-a093f47f10d5-scripts\") pod \"cinder-db-sync-j76tm\" (UID: \"516cc68a-d42b-40c8-8ca7-a093f47f10d5\") " pod="openstack-kuttl-tests/cinder-db-sync-j76tm" Dec 03 12:23:48 crc kubenswrapper[5052]: I1203 12:23:48.010794 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/516cc68a-d42b-40c8-8ca7-a093f47f10d5-db-sync-config-data\") pod \"cinder-db-sync-j76tm\" (UID: \"516cc68a-d42b-40c8-8ca7-a093f47f10d5\") " pod="openstack-kuttl-tests/cinder-db-sync-j76tm" Dec 03 12:23:48 crc kubenswrapper[5052]: I1203 12:23:48.010859 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/516cc68a-d42b-40c8-8ca7-a093f47f10d5-etc-machine-id\") pod \"cinder-db-sync-j76tm\" (UID: \"516cc68a-d42b-40c8-8ca7-a093f47f10d5\") " pod="openstack-kuttl-tests/cinder-db-sync-j76tm" Dec 03 12:23:48 crc kubenswrapper[5052]: I1203 12:23:48.010882 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/516cc68a-d42b-40c8-8ca7-a093f47f10d5-combined-ca-bundle\") pod \"cinder-db-sync-j76tm\" (UID: \"516cc68a-d42b-40c8-8ca7-a093f47f10d5\") " pod="openstack-kuttl-tests/cinder-db-sync-j76tm" Dec 03 12:23:48 crc kubenswrapper[5052]: I1203 12:23:48.010913 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/516cc68a-d42b-40c8-8ca7-a093f47f10d5-config-data\") pod \"cinder-db-sync-j76tm\" (UID: \"516cc68a-d42b-40c8-8ca7-a093f47f10d5\") " pod="openstack-kuttl-tests/cinder-db-sync-j76tm" Dec 03 12:23:48 crc kubenswrapper[5052]: I1203 12:23:48.016423 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/516cc68a-d42b-40c8-8ca7-a093f47f10d5-scripts\") pod \"cinder-db-sync-j76tm\" (UID: \"516cc68a-d42b-40c8-8ca7-a093f47f10d5\") " pod="openstack-kuttl-tests/cinder-db-sync-j76tm" Dec 03 12:23:48 crc kubenswrapper[5052]: I1203 12:23:48.016846 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/516cc68a-d42b-40c8-8ca7-a093f47f10d5-config-data\") pod \"cinder-db-sync-j76tm\" (UID: \"516cc68a-d42b-40c8-8ca7-a093f47f10d5\") " pod="openstack-kuttl-tests/cinder-db-sync-j76tm" Dec 03 12:23:48 crc kubenswrapper[5052]: I1203 12:23:48.018050 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/516cc68a-d42b-40c8-8ca7-a093f47f10d5-combined-ca-bundle\") pod \"cinder-db-sync-j76tm\" (UID: \"516cc68a-d42b-40c8-8ca7-a093f47f10d5\") " pod="openstack-kuttl-tests/cinder-db-sync-j76tm" Dec 03 12:23:48 crc kubenswrapper[5052]: I1203 12:23:48.018929 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/516cc68a-d42b-40c8-8ca7-a093f47f10d5-db-sync-config-data\") pod \"cinder-db-sync-j76tm\" (UID: \"516cc68a-d42b-40c8-8ca7-a093f47f10d5\") " pod="openstack-kuttl-tests/cinder-db-sync-j76tm" Dec 03 12:23:48 crc kubenswrapper[5052]: I1203 12:23:48.035763 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-jczdq\" (UniqueName: \"kubernetes.io/projected/516cc68a-d42b-40c8-8ca7-a093f47f10d5-kube-api-access-jczdq\") pod \"cinder-db-sync-j76tm\" (UID: \"516cc68a-d42b-40c8-8ca7-a093f47f10d5\") " pod="openstack-kuttl-tests/cinder-db-sync-j76tm" Dec 03 12:23:48 crc kubenswrapper[5052]: I1203 12:23:48.096601 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-db-sync-j76tm" Dec 03 12:23:48 crc kubenswrapper[5052]: I1203 12:23:48.115386 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/778e86ee-3618-41fa-83c3-eb28609f50e2-combined-ca-bundle\") pod \"neutron-db-sync-ggltp\" (UID: \"778e86ee-3618-41fa-83c3-eb28609f50e2\") " pod="openstack-kuttl-tests/neutron-db-sync-ggltp" Dec 03 12:23:48 crc kubenswrapper[5052]: I1203 12:23:48.115538 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/778e86ee-3618-41fa-83c3-eb28609f50e2-config\") pod \"neutron-db-sync-ggltp\" (UID: \"778e86ee-3618-41fa-83c3-eb28609f50e2\") " pod="openstack-kuttl-tests/neutron-db-sync-ggltp" Dec 03 12:23:48 crc kubenswrapper[5052]: I1203 12:23:48.115578 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ltkxl\" (UniqueName: \"kubernetes.io/projected/778e86ee-3618-41fa-83c3-eb28609f50e2-kube-api-access-ltkxl\") pod \"neutron-db-sync-ggltp\" (UID: \"778e86ee-3618-41fa-83c3-eb28609f50e2\") " pod="openstack-kuttl-tests/neutron-db-sync-ggltp" Dec 03 12:23:48 crc kubenswrapper[5052]: I1203 12:23:48.175526 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"47bc7c32-d14b-4137-be0d-469efe7b3c9e","Type":"ContainerStarted","Data":"7d9fca41cf1e6f2b9bc5937f4cc7bfab19f4008856489fac1501d67bd9566bf7"} Dec 03 12:23:48 crc kubenswrapper[5052]: I1203 12:23:48.175999 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-internal-api-0" podUID="47bc7c32-d14b-4137-be0d-469efe7b3c9e" containerName="glance-log" containerID="cri-o://4457e54a6f4b4579b71b4df60084a738365b0be2f091e39ea0590c7628e18571" gracePeriod=30 Dec 03 12:23:48 crc kubenswrapper[5052]: I1203 12:23:48.180804 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-internal-api-0" podUID="47bc7c32-d14b-4137-be0d-469efe7b3c9e" containerName="glance-httpd" containerID="cri-o://7d9fca41cf1e6f2b9bc5937f4cc7bfab19f4008856489fac1501d67bd9566bf7" gracePeriod=30 Dec 03 12:23:48 crc kubenswrapper[5052]: I1203 12:23:48.193707 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"2660f107-7c9f-4d25-be86-3ba68a71be57","Type":"ContainerStarted","Data":"d6d40488bb533fdd7e199ada04a44691a82fa82a2f30cd2732d83e40c8719a42"} Dec 03 12:23:48 crc kubenswrapper[5052]: I1203 12:23:48.196241 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-external-api-0" podUID="b5cc20b6-3503-42f6-b497-8ef83e55317d" containerName="glance-log" containerID="cri-o://082703df4163afc55b34093947ed1259ed87bfd80ee0c0d6455108fc6bffe2c2" gracePeriod=30 Dec 03 12:23:48 crc kubenswrapper[5052]: I1203 12:23:48.196331 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"b5cc20b6-3503-42f6-b497-8ef83e55317d","Type":"ContainerStarted","Data":"c33625a9b180a8f5d9f089c99bc85667fb97d0d0bf94061fc6af485a4eee1d6a"} Dec 03 12:23:48 crc kubenswrapper[5052]: I1203 12:23:48.196392 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-external-api-0" podUID="b5cc20b6-3503-42f6-b497-8ef83e55317d" containerName="glance-httpd" containerID="cri-o://c33625a9b180a8f5d9f089c99bc85667fb97d0d0bf94061fc6af485a4eee1d6a" gracePeriod=30 Dec 03 12:23:48 crc kubenswrapper[5052]: I1203 12:23:48.232971 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/778e86ee-3618-41fa-83c3-eb28609f50e2-config\") pod \"neutron-db-sync-ggltp\" (UID: \"778e86ee-3618-41fa-83c3-eb28609f50e2\") " pod="openstack-kuttl-tests/neutron-db-sync-ggltp" Dec 03 12:23:48 crc kubenswrapper[5052]: I1203 12:23:48.233444 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-ltkxl\" (UniqueName: \"kubernetes.io/projected/778e86ee-3618-41fa-83c3-eb28609f50e2-kube-api-access-ltkxl\") pod \"neutron-db-sync-ggltp\" (UID: \"778e86ee-3618-41fa-83c3-eb28609f50e2\") " pod="openstack-kuttl-tests/neutron-db-sync-ggltp" Dec 03 12:23:48 crc kubenswrapper[5052]: I1203 12:23:48.233531 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/778e86ee-3618-41fa-83c3-eb28609f50e2-combined-ca-bundle\") pod \"neutron-db-sync-ggltp\" (UID: \"778e86ee-3618-41fa-83c3-eb28609f50e2\") " pod="openstack-kuttl-tests/neutron-db-sync-ggltp" Dec 03 12:23:48 crc kubenswrapper[5052]: I1203 12:23:48.242136 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/778e86ee-3618-41fa-83c3-eb28609f50e2-combined-ca-bundle\") pod \"neutron-db-sync-ggltp\" (UID: \"778e86ee-3618-41fa-83c3-eb28609f50e2\") " pod="openstack-kuttl-tests/neutron-db-sync-ggltp" Dec 03 12:23:48 crc kubenswrapper[5052]: I1203 12:23:48.244052 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/778e86ee-3618-41fa-83c3-eb28609f50e2-config\") pod \"neutron-db-sync-ggltp\" (UID: \"778e86ee-3618-41fa-83c3-eb28609f50e2\") " pod="openstack-kuttl-tests/neutron-db-sync-ggltp" Dec 03 12:23:48 crc kubenswrapper[5052]: I1203 12:23:48.249941 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/glance-default-internal-api-0" podStartSLOduration=4.249914143 podStartE2EDuration="4.249914143s" podCreationTimestamp="2025-12-03 12:23:44 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:23:48.215139911 +0000 UTC m=+3818.803381675" watchObservedRunningTime="2025-12-03 12:23:48.249914143 +0000 UTC m=+3818.838155907" Dec 03 12:23:48 crc kubenswrapper[5052]: I1203 12:23:48.259026 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-ltkxl\" (UniqueName: \"kubernetes.io/projected/778e86ee-3618-41fa-83c3-eb28609f50e2-kube-api-access-ltkxl\") pod \"neutron-db-sync-ggltp\" (UID: \"778e86ee-3618-41fa-83c3-eb28609f50e2\") " pod="openstack-kuttl-tests/neutron-db-sync-ggltp" Dec 03 12:23:48 crc kubenswrapper[5052]: I1203 12:23:48.267280 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/glance-default-external-api-0" podStartSLOduration=4.267254318 podStartE2EDuration="4.267254318s" podCreationTimestamp="2025-12-03 12:23:44 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:23:48.257744317 +0000 UTC m=+3818.845986091" watchObservedRunningTime="2025-12-03 12:23:48.267254318 +0000 UTC m=+3818.855496102" Dec 03 12:23:48 crc kubenswrapper[5052]: I1203 12:23:48.316629 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-db-sync-ggltp" Dec 03 12:23:48 crc kubenswrapper[5052]: I1203 12:23:48.556432 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-db-sync-j9cr4"] Dec 03 12:23:48 crc kubenswrapper[5052]: I1203 12:23:48.697594 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-db-sync-j76tm"] Dec 03 12:23:48 crc kubenswrapper[5052]: I1203 12:23:48.897718 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-db-sync-ggltp"] Dec 03 12:23:48 crc kubenswrapper[5052]: I1203 12:23:48.937079 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:23:48 crc kubenswrapper[5052]: I1203 12:23:48.999044 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:23:49 crc kubenswrapper[5052]: I1203 12:23:49.068675 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/b5cc20b6-3503-42f6-b497-8ef83e55317d-logs\") pod \"b5cc20b6-3503-42f6-b497-8ef83e55317d\" (UID: \"b5cc20b6-3503-42f6-b497-8ef83e55317d\") " Dec 03 12:23:49 crc kubenswrapper[5052]: I1203 12:23:49.068763 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/b5cc20b6-3503-42f6-b497-8ef83e55317d-scripts\") pod \"b5cc20b6-3503-42f6-b497-8ef83e55317d\" (UID: \"b5cc20b6-3503-42f6-b497-8ef83e55317d\") " Dec 03 12:23:49 crc kubenswrapper[5052]: I1203 12:23:49.068793 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"glance\" (UniqueName: \"kubernetes.io/local-volume/local-storage07-crc\") pod \"b5cc20b6-3503-42f6-b497-8ef83e55317d\" (UID: \"b5cc20b6-3503-42f6-b497-8ef83e55317d\") " Dec 03 12:23:49 crc kubenswrapper[5052]: I1203 12:23:49.068831 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/b5cc20b6-3503-42f6-b497-8ef83e55317d-config-data\") pod \"b5cc20b6-3503-42f6-b497-8ef83e55317d\" (UID: \"b5cc20b6-3503-42f6-b497-8ef83e55317d\") " Dec 03 12:23:49 crc kubenswrapper[5052]: I1203 12:23:49.068855 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dhsrv\" (UniqueName: \"kubernetes.io/projected/b5cc20b6-3503-42f6-b497-8ef83e55317d-kube-api-access-dhsrv\") pod \"b5cc20b6-3503-42f6-b497-8ef83e55317d\" (UID: \"b5cc20b6-3503-42f6-b497-8ef83e55317d\") " Dec 03 12:23:49 crc kubenswrapper[5052]: I1203 12:23:49.068956 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/b5cc20b6-3503-42f6-b497-8ef83e55317d-httpd-run\") pod \"b5cc20b6-3503-42f6-b497-8ef83e55317d\" (UID: \"b5cc20b6-3503-42f6-b497-8ef83e55317d\") " Dec 03 12:23:49 crc kubenswrapper[5052]: I1203 12:23:49.068980 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b5cc20b6-3503-42f6-b497-8ef83e55317d-combined-ca-bundle\") pod \"b5cc20b6-3503-42f6-b497-8ef83e55317d\" (UID: \"b5cc20b6-3503-42f6-b497-8ef83e55317d\") " Dec 03 12:23:49 crc kubenswrapper[5052]: I1203 12:23:49.069343 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b5cc20b6-3503-42f6-b497-8ef83e55317d-logs" (OuterVolumeSpecName: "logs") pod "b5cc20b6-3503-42f6-b497-8ef83e55317d" (UID: "b5cc20b6-3503-42f6-b497-8ef83e55317d"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:23:49 crc kubenswrapper[5052]: I1203 12:23:49.069705 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b5cc20b6-3503-42f6-b497-8ef83e55317d-httpd-run" (OuterVolumeSpecName: "httpd-run") pod "b5cc20b6-3503-42f6-b497-8ef83e55317d" (UID: "b5cc20b6-3503-42f6-b497-8ef83e55317d"). InnerVolumeSpecName "httpd-run". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:23:49 crc kubenswrapper[5052]: I1203 12:23:49.072894 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b5cc20b6-3503-42f6-b497-8ef83e55317d-scripts" (OuterVolumeSpecName: "scripts") pod "b5cc20b6-3503-42f6-b497-8ef83e55317d" (UID: "b5cc20b6-3503-42f6-b497-8ef83e55317d"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:23:49 crc kubenswrapper[5052]: I1203 12:23:49.074595 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage07-crc" (OuterVolumeSpecName: "glance") pod "b5cc20b6-3503-42f6-b497-8ef83e55317d" (UID: "b5cc20b6-3503-42f6-b497-8ef83e55317d"). InnerVolumeSpecName "local-storage07-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 12:23:49 crc kubenswrapper[5052]: I1203 12:23:49.074905 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b5cc20b6-3503-42f6-b497-8ef83e55317d-kube-api-access-dhsrv" (OuterVolumeSpecName: "kube-api-access-dhsrv") pod "b5cc20b6-3503-42f6-b497-8ef83e55317d" (UID: "b5cc20b6-3503-42f6-b497-8ef83e55317d"). InnerVolumeSpecName "kube-api-access-dhsrv". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:23:49 crc kubenswrapper[5052]: I1203 12:23:49.105485 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b5cc20b6-3503-42f6-b497-8ef83e55317d-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "b5cc20b6-3503-42f6-b497-8ef83e55317d" (UID: "b5cc20b6-3503-42f6-b497-8ef83e55317d"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:23:49 crc kubenswrapper[5052]: I1203 12:23:49.135673 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b5cc20b6-3503-42f6-b497-8ef83e55317d-config-data" (OuterVolumeSpecName: "config-data") pod "b5cc20b6-3503-42f6-b497-8ef83e55317d" (UID: "b5cc20b6-3503-42f6-b497-8ef83e55317d"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:23:49 crc kubenswrapper[5052]: I1203 12:23:49.170284 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/47bc7c32-d14b-4137-be0d-469efe7b3c9e-config-data\") pod \"47bc7c32-d14b-4137-be0d-469efe7b3c9e\" (UID: \"47bc7c32-d14b-4137-be0d-469efe7b3c9e\") " Dec 03 12:23:49 crc kubenswrapper[5052]: I1203 12:23:49.170343 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"glance\" (UniqueName: \"kubernetes.io/local-volume/local-storage09-crc\") pod \"47bc7c32-d14b-4137-be0d-469efe7b3c9e\" (UID: \"47bc7c32-d14b-4137-be0d-469efe7b3c9e\") " Dec 03 12:23:49 crc kubenswrapper[5052]: I1203 12:23:49.170395 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/47bc7c32-d14b-4137-be0d-469efe7b3c9e-scripts\") pod \"47bc7c32-d14b-4137-be0d-469efe7b3c9e\" (UID: \"47bc7c32-d14b-4137-be0d-469efe7b3c9e\") " Dec 03 12:23:49 crc kubenswrapper[5052]: I1203 12:23:49.170451 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/47bc7c32-d14b-4137-be0d-469efe7b3c9e-combined-ca-bundle\") pod \"47bc7c32-d14b-4137-be0d-469efe7b3c9e\" (UID: \"47bc7c32-d14b-4137-be0d-469efe7b3c9e\") " Dec 03 12:23:49 crc kubenswrapper[5052]: I1203 12:23:49.170503 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7pqgm\" (UniqueName: \"kubernetes.io/projected/47bc7c32-d14b-4137-be0d-469efe7b3c9e-kube-api-access-7pqgm\") pod \"47bc7c32-d14b-4137-be0d-469efe7b3c9e\" (UID: \"47bc7c32-d14b-4137-be0d-469efe7b3c9e\") " Dec 03 12:23:49 crc kubenswrapper[5052]: I1203 12:23:49.170578 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/47bc7c32-d14b-4137-be0d-469efe7b3c9e-logs\") pod \"47bc7c32-d14b-4137-be0d-469efe7b3c9e\" (UID: \"47bc7c32-d14b-4137-be0d-469efe7b3c9e\") " Dec 03 12:23:49 crc kubenswrapper[5052]: I1203 12:23:49.170679 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/47bc7c32-d14b-4137-be0d-469efe7b3c9e-httpd-run\") pod \"47bc7c32-d14b-4137-be0d-469efe7b3c9e\" (UID: \"47bc7c32-d14b-4137-be0d-469efe7b3c9e\") " Dec 03 12:23:49 crc kubenswrapper[5052]: I1203 12:23:49.170976 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/b5cc20b6-3503-42f6-b497-8ef83e55317d-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:23:49 crc kubenswrapper[5052]: I1203 12:23:49.171003 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage07-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage07-crc\") on node \"crc\" " Dec 03 12:23:49 crc kubenswrapper[5052]: I1203 12:23:49.171013 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/b5cc20b6-3503-42f6-b497-8ef83e55317d-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:23:49 crc kubenswrapper[5052]: I1203 12:23:49.171022 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-dhsrv\" (UniqueName: \"kubernetes.io/projected/b5cc20b6-3503-42f6-b497-8ef83e55317d-kube-api-access-dhsrv\") on node \"crc\" DevicePath \"\"" Dec 03 12:23:49 crc kubenswrapper[5052]: I1203 12:23:49.171033 5052 reconciler_common.go:293] "Volume detached for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/b5cc20b6-3503-42f6-b497-8ef83e55317d-httpd-run\") on node \"crc\" DevicePath \"\"" Dec 03 12:23:49 crc kubenswrapper[5052]: I1203 12:23:49.171042 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b5cc20b6-3503-42f6-b497-8ef83e55317d-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:23:49 crc kubenswrapper[5052]: I1203 12:23:49.171052 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/b5cc20b6-3503-42f6-b497-8ef83e55317d-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:23:49 crc kubenswrapper[5052]: I1203 12:23:49.175071 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/47bc7c32-d14b-4137-be0d-469efe7b3c9e-logs" (OuterVolumeSpecName: "logs") pod "47bc7c32-d14b-4137-be0d-469efe7b3c9e" (UID: "47bc7c32-d14b-4137-be0d-469efe7b3c9e"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:23:49 crc kubenswrapper[5052]: I1203 12:23:49.175766 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/47bc7c32-d14b-4137-be0d-469efe7b3c9e-httpd-run" (OuterVolumeSpecName: "httpd-run") pod "47bc7c32-d14b-4137-be0d-469efe7b3c9e" (UID: "47bc7c32-d14b-4137-be0d-469efe7b3c9e"). InnerVolumeSpecName "httpd-run". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:23:49 crc kubenswrapper[5052]: I1203 12:23:49.177701 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage09-crc" (OuterVolumeSpecName: "glance") pod "47bc7c32-d14b-4137-be0d-469efe7b3c9e" (UID: "47bc7c32-d14b-4137-be0d-469efe7b3c9e"). InnerVolumeSpecName "local-storage09-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 12:23:49 crc kubenswrapper[5052]: I1203 12:23:49.178469 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/47bc7c32-d14b-4137-be0d-469efe7b3c9e-kube-api-access-7pqgm" (OuterVolumeSpecName: "kube-api-access-7pqgm") pod "47bc7c32-d14b-4137-be0d-469efe7b3c9e" (UID: "47bc7c32-d14b-4137-be0d-469efe7b3c9e"). InnerVolumeSpecName "kube-api-access-7pqgm". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:23:49 crc kubenswrapper[5052]: I1203 12:23:49.181710 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/47bc7c32-d14b-4137-be0d-469efe7b3c9e-scripts" (OuterVolumeSpecName: "scripts") pod "47bc7c32-d14b-4137-be0d-469efe7b3c9e" (UID: "47bc7c32-d14b-4137-be0d-469efe7b3c9e"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:23:49 crc kubenswrapper[5052]: I1203 12:23:49.192040 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage07-crc" (UniqueName: "kubernetes.io/local-volume/local-storage07-crc") on node "crc" Dec 03 12:23:49 crc kubenswrapper[5052]: I1203 12:23:49.199727 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/47bc7c32-d14b-4137-be0d-469efe7b3c9e-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "47bc7c32-d14b-4137-be0d-469efe7b3c9e" (UID: "47bc7c32-d14b-4137-be0d-469efe7b3c9e"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:23:49 crc kubenswrapper[5052]: I1203 12:23:49.212326 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-db-sync-j76tm" event={"ID":"516cc68a-d42b-40c8-8ca7-a093f47f10d5","Type":"ContainerStarted","Data":"c8ce4daa70d1f8a77bfe3bf6bc7585b0434d0f06a846b28cd4bba56cb2f765b9"} Dec 03 12:23:49 crc kubenswrapper[5052]: I1203 12:23:49.217725 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-db-sync-j9cr4" event={"ID":"fa8494cf-ad3c-4998-a3fd-a39863f72aff","Type":"ContainerStarted","Data":"7e368d1baaf0ba061b0efb437531205a49da54db72f9bde12964c76141ba7147"} Dec 03 12:23:49 crc kubenswrapper[5052]: I1203 12:23:49.217776 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-db-sync-j9cr4" event={"ID":"fa8494cf-ad3c-4998-a3fd-a39863f72aff","Type":"ContainerStarted","Data":"764bfb12d2ec6d7821eea3a915713b90f45046130874a308c822eb14f2da4a17"} Dec 03 12:23:49 crc kubenswrapper[5052]: I1203 12:23:49.223743 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-db-sync-ggltp" event={"ID":"778e86ee-3618-41fa-83c3-eb28609f50e2","Type":"ContainerStarted","Data":"ef3ec7b7f63c95d670ec6dce26e58fcf598dfd84857be75804ebf18715f6be7d"} Dec 03 12:23:49 crc kubenswrapper[5052]: I1203 12:23:49.223798 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-db-sync-ggltp" event={"ID":"778e86ee-3618-41fa-83c3-eb28609f50e2","Type":"ContainerStarted","Data":"87728977e888f686d67bc05c24bd18d9f7c5e06ec6aee6775e0049b269205dab"} Dec 03 12:23:49 crc kubenswrapper[5052]: I1203 12:23:49.226909 5052 generic.go:334] "Generic (PLEG): container finished" podID="6af52f56-d687-43d2-a8da-1780e540ec92" containerID="1866d9e97345f3e78cc8443ebbf0430328e7ab4a4ed32a162d7d7f17a4786a03" exitCode=0 Dec 03 12:23:49 crc kubenswrapper[5052]: I1203 12:23:49.227011 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-db-sync-fspvj" event={"ID":"6af52f56-d687-43d2-a8da-1780e540ec92","Type":"ContainerDied","Data":"1866d9e97345f3e78cc8443ebbf0430328e7ab4a4ed32a162d7d7f17a4786a03"} Dec 03 12:23:49 crc kubenswrapper[5052]: I1203 12:23:49.233846 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/47bc7c32-d14b-4137-be0d-469efe7b3c9e-config-data" (OuterVolumeSpecName: "config-data") pod "47bc7c32-d14b-4137-be0d-469efe7b3c9e" (UID: "47bc7c32-d14b-4137-be0d-469efe7b3c9e"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:23:49 crc kubenswrapper[5052]: I1203 12:23:49.239100 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/barbican-db-sync-j9cr4" podStartSLOduration=2.239081456 podStartE2EDuration="2.239081456s" podCreationTimestamp="2025-12-03 12:23:47 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:23:49.230542602 +0000 UTC m=+3819.818784366" watchObservedRunningTime="2025-12-03 12:23:49.239081456 +0000 UTC m=+3819.827323220" Dec 03 12:23:49 crc kubenswrapper[5052]: I1203 12:23:49.242147 5052 generic.go:334] "Generic (PLEG): container finished" podID="47bc7c32-d14b-4137-be0d-469efe7b3c9e" containerID="7d9fca41cf1e6f2b9bc5937f4cc7bfab19f4008856489fac1501d67bd9566bf7" exitCode=0 Dec 03 12:23:49 crc kubenswrapper[5052]: I1203 12:23:49.242187 5052 generic.go:334] "Generic (PLEG): container finished" podID="47bc7c32-d14b-4137-be0d-469efe7b3c9e" containerID="4457e54a6f4b4579b71b4df60084a738365b0be2f091e39ea0590c7628e18571" exitCode=143 Dec 03 12:23:49 crc kubenswrapper[5052]: I1203 12:23:49.242235 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"47bc7c32-d14b-4137-be0d-469efe7b3c9e","Type":"ContainerDied","Data":"7d9fca41cf1e6f2b9bc5937f4cc7bfab19f4008856489fac1501d67bd9566bf7"} Dec 03 12:23:49 crc kubenswrapper[5052]: I1203 12:23:49.242273 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"47bc7c32-d14b-4137-be0d-469efe7b3c9e","Type":"ContainerDied","Data":"4457e54a6f4b4579b71b4df60084a738365b0be2f091e39ea0590c7628e18571"} Dec 03 12:23:49 crc kubenswrapper[5052]: I1203 12:23:49.242287 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"47bc7c32-d14b-4137-be0d-469efe7b3c9e","Type":"ContainerDied","Data":"5c968e02667e6873c21095a11d2e3485d1f3e327b4a5d21f39fdba401edb7b2b"} Dec 03 12:23:49 crc kubenswrapper[5052]: I1203 12:23:49.242304 5052 scope.go:117] "RemoveContainer" containerID="7d9fca41cf1e6f2b9bc5937f4cc7bfab19f4008856489fac1501d67bd9566bf7" Dec 03 12:23:49 crc kubenswrapper[5052]: I1203 12:23:49.242433 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:23:49 crc kubenswrapper[5052]: I1203 12:23:49.247797 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/neutron-db-sync-ggltp" podStartSLOduration=2.247782194 podStartE2EDuration="2.247782194s" podCreationTimestamp="2025-12-03 12:23:47 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:23:49.246910699 +0000 UTC m=+3819.835152463" watchObservedRunningTime="2025-12-03 12:23:49.247782194 +0000 UTC m=+3819.836023948" Dec 03 12:23:49 crc kubenswrapper[5052]: I1203 12:23:49.262656 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"2660f107-7c9f-4d25-be86-3ba68a71be57","Type":"ContainerStarted","Data":"ad2ecf738e50628058ca414a90615ccf59d2a01114dd7d51ff8e6f11e9458294"} Dec 03 12:23:49 crc kubenswrapper[5052]: I1203 12:23:49.271454 5052 generic.go:334] "Generic (PLEG): container finished" podID="f21141cf-d708-4da1-8bb7-5900993b9d26" containerID="67ce4d39f2bdc99c3f3be2f5c14c987070c0b90d869d6a4344098656d6746ae9" exitCode=0 Dec 03 12:23:49 crc kubenswrapper[5052]: I1203 12:23:49.271774 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-bootstrap-mh8wf" event={"ID":"f21141cf-d708-4da1-8bb7-5900993b9d26","Type":"ContainerDied","Data":"67ce4d39f2bdc99c3f3be2f5c14c987070c0b90d869d6a4344098656d6746ae9"} Dec 03 12:23:49 crc kubenswrapper[5052]: I1203 12:23:49.272449 5052 reconciler_common.go:293] "Volume detached for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/47bc7c32-d14b-4137-be0d-469efe7b3c9e-httpd-run\") on node \"crc\" DevicePath \"\"" Dec 03 12:23:49 crc kubenswrapper[5052]: I1203 12:23:49.272475 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/47bc7c32-d14b-4137-be0d-469efe7b3c9e-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:23:49 crc kubenswrapper[5052]: I1203 12:23:49.272501 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage09-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage09-crc\") on node \"crc\" " Dec 03 12:23:49 crc kubenswrapper[5052]: I1203 12:23:49.272526 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/47bc7c32-d14b-4137-be0d-469efe7b3c9e-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:23:49 crc kubenswrapper[5052]: I1203 12:23:49.272536 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/47bc7c32-d14b-4137-be0d-469efe7b3c9e-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:23:49 crc kubenswrapper[5052]: I1203 12:23:49.272546 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-7pqgm\" (UniqueName: \"kubernetes.io/projected/47bc7c32-d14b-4137-be0d-469efe7b3c9e-kube-api-access-7pqgm\") on node \"crc\" DevicePath \"\"" Dec 03 12:23:49 crc kubenswrapper[5052]: I1203 12:23:49.272554 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage07-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage07-crc\") on node \"crc\" DevicePath \"\"" Dec 03 12:23:49 crc kubenswrapper[5052]: I1203 12:23:49.272563 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/47bc7c32-d14b-4137-be0d-469efe7b3c9e-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:23:49 crc kubenswrapper[5052]: I1203 12:23:49.277774 5052 generic.go:334] "Generic (PLEG): container finished" podID="b5cc20b6-3503-42f6-b497-8ef83e55317d" containerID="c33625a9b180a8f5d9f089c99bc85667fb97d0d0bf94061fc6af485a4eee1d6a" exitCode=0 Dec 03 12:23:49 crc kubenswrapper[5052]: I1203 12:23:49.277814 5052 generic.go:334] "Generic (PLEG): container finished" podID="b5cc20b6-3503-42f6-b497-8ef83e55317d" containerID="082703df4163afc55b34093947ed1259ed87bfd80ee0c0d6455108fc6bffe2c2" exitCode=143 Dec 03 12:23:49 crc kubenswrapper[5052]: I1203 12:23:49.277846 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"b5cc20b6-3503-42f6-b497-8ef83e55317d","Type":"ContainerDied","Data":"c33625a9b180a8f5d9f089c99bc85667fb97d0d0bf94061fc6af485a4eee1d6a"} Dec 03 12:23:49 crc kubenswrapper[5052]: I1203 12:23:49.277877 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"b5cc20b6-3503-42f6-b497-8ef83e55317d","Type":"ContainerDied","Data":"082703df4163afc55b34093947ed1259ed87bfd80ee0c0d6455108fc6bffe2c2"} Dec 03 12:23:49 crc kubenswrapper[5052]: I1203 12:23:49.277889 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"b5cc20b6-3503-42f6-b497-8ef83e55317d","Type":"ContainerDied","Data":"372f816b2624ffd8bf73fd50a48ed5a5b138359619500f43b64ed9aa5ccad162"} Dec 03 12:23:49 crc kubenswrapper[5052]: I1203 12:23:49.277972 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:23:49 crc kubenswrapper[5052]: I1203 12:23:49.316821 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:23:49 crc kubenswrapper[5052]: I1203 12:23:49.324776 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:23:49 crc kubenswrapper[5052]: I1203 12:23:49.326545 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage09-crc" (UniqueName: "kubernetes.io/local-volume/local-storage09-crc") on node "crc" Dec 03 12:23:49 crc kubenswrapper[5052]: I1203 12:23:49.338526 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:23:49 crc kubenswrapper[5052]: E1203 12:23:49.338934 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="47bc7c32-d14b-4137-be0d-469efe7b3c9e" containerName="glance-log" Dec 03 12:23:49 crc kubenswrapper[5052]: I1203 12:23:49.338950 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="47bc7c32-d14b-4137-be0d-469efe7b3c9e" containerName="glance-log" Dec 03 12:23:49 crc kubenswrapper[5052]: E1203 12:23:49.338977 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="b5cc20b6-3503-42f6-b497-8ef83e55317d" containerName="glance-httpd" Dec 03 12:23:49 crc kubenswrapper[5052]: I1203 12:23:49.338984 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="b5cc20b6-3503-42f6-b497-8ef83e55317d" containerName="glance-httpd" Dec 03 12:23:49 crc kubenswrapper[5052]: E1203 12:23:49.338996 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="47bc7c32-d14b-4137-be0d-469efe7b3c9e" containerName="glance-httpd" Dec 03 12:23:49 crc kubenswrapper[5052]: I1203 12:23:49.339003 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="47bc7c32-d14b-4137-be0d-469efe7b3c9e" containerName="glance-httpd" Dec 03 12:23:49 crc kubenswrapper[5052]: E1203 12:23:49.339015 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="b5cc20b6-3503-42f6-b497-8ef83e55317d" containerName="glance-log" Dec 03 12:23:49 crc kubenswrapper[5052]: I1203 12:23:49.339022 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="b5cc20b6-3503-42f6-b497-8ef83e55317d" containerName="glance-log" Dec 03 12:23:49 crc kubenswrapper[5052]: I1203 12:23:49.339228 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="b5cc20b6-3503-42f6-b497-8ef83e55317d" containerName="glance-log" Dec 03 12:23:49 crc kubenswrapper[5052]: I1203 12:23:49.339257 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="47bc7c32-d14b-4137-be0d-469efe7b3c9e" containerName="glance-httpd" Dec 03 12:23:49 crc kubenswrapper[5052]: I1203 12:23:49.339273 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="b5cc20b6-3503-42f6-b497-8ef83e55317d" containerName="glance-httpd" Dec 03 12:23:49 crc kubenswrapper[5052]: I1203 12:23:49.339289 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="47bc7c32-d14b-4137-be0d-469efe7b3c9e" containerName="glance-log" Dec 03 12:23:49 crc kubenswrapper[5052]: I1203 12:23:49.344986 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:23:49 crc kubenswrapper[5052]: I1203 12:23:49.352501 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:23:49 crc kubenswrapper[5052]: I1203 12:23:49.359722 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-glance-dockercfg-cd9ns" Dec 03 12:23:49 crc kubenswrapper[5052]: I1203 12:23:49.360021 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-default-internal-config-data" Dec 03 12:23:49 crc kubenswrapper[5052]: I1203 12:23:49.361787 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-scripts" Dec 03 12:23:49 crc kubenswrapper[5052]: I1203 12:23:49.364915 5052 scope.go:117] "RemoveContainer" containerID="4457e54a6f4b4579b71b4df60084a738365b0be2f091e39ea0590c7628e18571" Dec 03 12:23:49 crc kubenswrapper[5052]: I1203 12:23:49.382554 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage09-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage09-crc\") on node \"crc\" DevicePath \"\"" Dec 03 12:23:49 crc kubenswrapper[5052]: I1203 12:23:49.427193 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:23:49 crc kubenswrapper[5052]: I1203 12:23:49.427794 5052 scope.go:117] "RemoveContainer" containerID="7d9fca41cf1e6f2b9bc5937f4cc7bfab19f4008856489fac1501d67bd9566bf7" Dec 03 12:23:49 crc kubenswrapper[5052]: E1203 12:23:49.428236 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7d9fca41cf1e6f2b9bc5937f4cc7bfab19f4008856489fac1501d67bd9566bf7\": container with ID starting with 7d9fca41cf1e6f2b9bc5937f4cc7bfab19f4008856489fac1501d67bd9566bf7 not found: ID does not exist" containerID="7d9fca41cf1e6f2b9bc5937f4cc7bfab19f4008856489fac1501d67bd9566bf7" Dec 03 12:23:49 crc kubenswrapper[5052]: I1203 12:23:49.428269 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7d9fca41cf1e6f2b9bc5937f4cc7bfab19f4008856489fac1501d67bd9566bf7"} err="failed to get container status \"7d9fca41cf1e6f2b9bc5937f4cc7bfab19f4008856489fac1501d67bd9566bf7\": rpc error: code = NotFound desc = could not find container \"7d9fca41cf1e6f2b9bc5937f4cc7bfab19f4008856489fac1501d67bd9566bf7\": container with ID starting with 7d9fca41cf1e6f2b9bc5937f4cc7bfab19f4008856489fac1501d67bd9566bf7 not found: ID does not exist" Dec 03 12:23:49 crc kubenswrapper[5052]: I1203 12:23:49.428293 5052 scope.go:117] "RemoveContainer" containerID="4457e54a6f4b4579b71b4df60084a738365b0be2f091e39ea0590c7628e18571" Dec 03 12:23:49 crc kubenswrapper[5052]: E1203 12:23:49.428762 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4457e54a6f4b4579b71b4df60084a738365b0be2f091e39ea0590c7628e18571\": container with ID starting with 4457e54a6f4b4579b71b4df60084a738365b0be2f091e39ea0590c7628e18571 not found: ID does not exist" containerID="4457e54a6f4b4579b71b4df60084a738365b0be2f091e39ea0590c7628e18571" Dec 03 12:23:49 crc kubenswrapper[5052]: I1203 12:23:49.428789 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4457e54a6f4b4579b71b4df60084a738365b0be2f091e39ea0590c7628e18571"} err="failed to get container status \"4457e54a6f4b4579b71b4df60084a738365b0be2f091e39ea0590c7628e18571\": rpc error: code = NotFound desc = could not find container \"4457e54a6f4b4579b71b4df60084a738365b0be2f091e39ea0590c7628e18571\": container with ID starting with 4457e54a6f4b4579b71b4df60084a738365b0be2f091e39ea0590c7628e18571 not found: ID does not exist" Dec 03 12:23:49 crc kubenswrapper[5052]: I1203 12:23:49.428805 5052 scope.go:117] "RemoveContainer" containerID="7d9fca41cf1e6f2b9bc5937f4cc7bfab19f4008856489fac1501d67bd9566bf7" Dec 03 12:23:49 crc kubenswrapper[5052]: I1203 12:23:49.429117 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7d9fca41cf1e6f2b9bc5937f4cc7bfab19f4008856489fac1501d67bd9566bf7"} err="failed to get container status \"7d9fca41cf1e6f2b9bc5937f4cc7bfab19f4008856489fac1501d67bd9566bf7\": rpc error: code = NotFound desc = could not find container \"7d9fca41cf1e6f2b9bc5937f4cc7bfab19f4008856489fac1501d67bd9566bf7\": container with ID starting with 7d9fca41cf1e6f2b9bc5937f4cc7bfab19f4008856489fac1501d67bd9566bf7 not found: ID does not exist" Dec 03 12:23:49 crc kubenswrapper[5052]: I1203 12:23:49.429161 5052 scope.go:117] "RemoveContainer" containerID="4457e54a6f4b4579b71b4df60084a738365b0be2f091e39ea0590c7628e18571" Dec 03 12:23:49 crc kubenswrapper[5052]: I1203 12:23:49.429563 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4457e54a6f4b4579b71b4df60084a738365b0be2f091e39ea0590c7628e18571"} err="failed to get container status \"4457e54a6f4b4579b71b4df60084a738365b0be2f091e39ea0590c7628e18571\": rpc error: code = NotFound desc = could not find container \"4457e54a6f4b4579b71b4df60084a738365b0be2f091e39ea0590c7628e18571\": container with ID starting with 4457e54a6f4b4579b71b4df60084a738365b0be2f091e39ea0590c7628e18571 not found: ID does not exist" Dec 03 12:23:49 crc kubenswrapper[5052]: I1203 12:23:49.429591 5052 scope.go:117] "RemoveContainer" containerID="c33625a9b180a8f5d9f089c99bc85667fb97d0d0bf94061fc6af485a4eee1d6a" Dec 03 12:23:49 crc kubenswrapper[5052]: I1203 12:23:49.438410 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:23:49 crc kubenswrapper[5052]: I1203 12:23:49.470022 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:23:49 crc kubenswrapper[5052]: I1203 12:23:49.471464 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:23:49 crc kubenswrapper[5052]: I1203 12:23:49.473619 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-default-external-config-data" Dec 03 12:23:49 crc kubenswrapper[5052]: I1203 12:23:49.483550 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/9514eb24-503a-40b8-a339-481f81fd49de-logs\") pod \"glance-default-internal-api-0\" (UID: \"9514eb24-503a-40b8-a339-481f81fd49de\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:23:49 crc kubenswrapper[5052]: I1203 12:23:49.483585 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/9514eb24-503a-40b8-a339-481f81fd49de-httpd-run\") pod \"glance-default-internal-api-0\" (UID: \"9514eb24-503a-40b8-a339-481f81fd49de\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:23:49 crc kubenswrapper[5052]: I1203 12:23:49.483613 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage09-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage09-crc\") pod \"glance-default-internal-api-0\" (UID: \"9514eb24-503a-40b8-a339-481f81fd49de\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:23:49 crc kubenswrapper[5052]: I1203 12:23:49.483637 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/9514eb24-503a-40b8-a339-481f81fd49de-config-data\") pod \"glance-default-internal-api-0\" (UID: \"9514eb24-503a-40b8-a339-481f81fd49de\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:23:49 crc kubenswrapper[5052]: I1203 12:23:49.483683 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/9514eb24-503a-40b8-a339-481f81fd49de-scripts\") pod \"glance-default-internal-api-0\" (UID: \"9514eb24-503a-40b8-a339-481f81fd49de\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:23:49 crc kubenswrapper[5052]: I1203 12:23:49.483701 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ccpnm\" (UniqueName: \"kubernetes.io/projected/9514eb24-503a-40b8-a339-481f81fd49de-kube-api-access-ccpnm\") pod \"glance-default-internal-api-0\" (UID: \"9514eb24-503a-40b8-a339-481f81fd49de\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:23:49 crc kubenswrapper[5052]: I1203 12:23:49.483733 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9514eb24-503a-40b8-a339-481f81fd49de-combined-ca-bundle\") pod \"glance-default-internal-api-0\" (UID: \"9514eb24-503a-40b8-a339-481f81fd49de\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:23:49 crc kubenswrapper[5052]: I1203 12:23:49.502725 5052 scope.go:117] "RemoveContainer" containerID="082703df4163afc55b34093947ed1259ed87bfd80ee0c0d6455108fc6bffe2c2" Dec 03 12:23:49 crc kubenswrapper[5052]: I1203 12:23:49.521468 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:23:49 crc kubenswrapper[5052]: I1203 12:23:49.559316 5052 scope.go:117] "RemoveContainer" containerID="c33625a9b180a8f5d9f089c99bc85667fb97d0d0bf94061fc6af485a4eee1d6a" Dec 03 12:23:49 crc kubenswrapper[5052]: E1203 12:23:49.560410 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c33625a9b180a8f5d9f089c99bc85667fb97d0d0bf94061fc6af485a4eee1d6a\": container with ID starting with c33625a9b180a8f5d9f089c99bc85667fb97d0d0bf94061fc6af485a4eee1d6a not found: ID does not exist" containerID="c33625a9b180a8f5d9f089c99bc85667fb97d0d0bf94061fc6af485a4eee1d6a" Dec 03 12:23:49 crc kubenswrapper[5052]: I1203 12:23:49.560479 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c33625a9b180a8f5d9f089c99bc85667fb97d0d0bf94061fc6af485a4eee1d6a"} err="failed to get container status \"c33625a9b180a8f5d9f089c99bc85667fb97d0d0bf94061fc6af485a4eee1d6a\": rpc error: code = NotFound desc = could not find container \"c33625a9b180a8f5d9f089c99bc85667fb97d0d0bf94061fc6af485a4eee1d6a\": container with ID starting with c33625a9b180a8f5d9f089c99bc85667fb97d0d0bf94061fc6af485a4eee1d6a not found: ID does not exist" Dec 03 12:23:49 crc kubenswrapper[5052]: I1203 12:23:49.560588 5052 scope.go:117] "RemoveContainer" containerID="082703df4163afc55b34093947ed1259ed87bfd80ee0c0d6455108fc6bffe2c2" Dec 03 12:23:49 crc kubenswrapper[5052]: E1203 12:23:49.560894 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"082703df4163afc55b34093947ed1259ed87bfd80ee0c0d6455108fc6bffe2c2\": container with ID starting with 082703df4163afc55b34093947ed1259ed87bfd80ee0c0d6455108fc6bffe2c2 not found: ID does not exist" containerID="082703df4163afc55b34093947ed1259ed87bfd80ee0c0d6455108fc6bffe2c2" Dec 03 12:23:49 crc kubenswrapper[5052]: I1203 12:23:49.560914 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"082703df4163afc55b34093947ed1259ed87bfd80ee0c0d6455108fc6bffe2c2"} err="failed to get container status \"082703df4163afc55b34093947ed1259ed87bfd80ee0c0d6455108fc6bffe2c2\": rpc error: code = NotFound desc = could not find container \"082703df4163afc55b34093947ed1259ed87bfd80ee0c0d6455108fc6bffe2c2\": container with ID starting with 082703df4163afc55b34093947ed1259ed87bfd80ee0c0d6455108fc6bffe2c2 not found: ID does not exist" Dec 03 12:23:49 crc kubenswrapper[5052]: I1203 12:23:49.560927 5052 scope.go:117] "RemoveContainer" containerID="c33625a9b180a8f5d9f089c99bc85667fb97d0d0bf94061fc6af485a4eee1d6a" Dec 03 12:23:49 crc kubenswrapper[5052]: I1203 12:23:49.561281 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c33625a9b180a8f5d9f089c99bc85667fb97d0d0bf94061fc6af485a4eee1d6a"} err="failed to get container status \"c33625a9b180a8f5d9f089c99bc85667fb97d0d0bf94061fc6af485a4eee1d6a\": rpc error: code = NotFound desc = could not find container \"c33625a9b180a8f5d9f089c99bc85667fb97d0d0bf94061fc6af485a4eee1d6a\": container with ID starting with c33625a9b180a8f5d9f089c99bc85667fb97d0d0bf94061fc6af485a4eee1d6a not found: ID does not exist" Dec 03 12:23:49 crc kubenswrapper[5052]: I1203 12:23:49.561350 5052 scope.go:117] "RemoveContainer" containerID="082703df4163afc55b34093947ed1259ed87bfd80ee0c0d6455108fc6bffe2c2" Dec 03 12:23:49 crc kubenswrapper[5052]: I1203 12:23:49.563948 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"082703df4163afc55b34093947ed1259ed87bfd80ee0c0d6455108fc6bffe2c2"} err="failed to get container status \"082703df4163afc55b34093947ed1259ed87bfd80ee0c0d6455108fc6bffe2c2\": rpc error: code = NotFound desc = could not find container \"082703df4163afc55b34093947ed1259ed87bfd80ee0c0d6455108fc6bffe2c2\": container with ID starting with 082703df4163afc55b34093947ed1259ed87bfd80ee0c0d6455108fc6bffe2c2 not found: ID does not exist" Dec 03 12:23:49 crc kubenswrapper[5052]: I1203 12:23:49.585669 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/9514eb24-503a-40b8-a339-481f81fd49de-logs\") pod \"glance-default-internal-api-0\" (UID: \"9514eb24-503a-40b8-a339-481f81fd49de\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:23:49 crc kubenswrapper[5052]: I1203 12:23:49.586043 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/9514eb24-503a-40b8-a339-481f81fd49de-httpd-run\") pod \"glance-default-internal-api-0\" (UID: \"9514eb24-503a-40b8-a339-481f81fd49de\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:23:49 crc kubenswrapper[5052]: I1203 12:23:49.586088 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage09-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage09-crc\") pod \"glance-default-internal-api-0\" (UID: \"9514eb24-503a-40b8-a339-481f81fd49de\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:23:49 crc kubenswrapper[5052]: I1203 12:23:49.586125 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/9514eb24-503a-40b8-a339-481f81fd49de-config-data\") pod \"glance-default-internal-api-0\" (UID: \"9514eb24-503a-40b8-a339-481f81fd49de\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:23:49 crc kubenswrapper[5052]: I1203 12:23:49.586224 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/9514eb24-503a-40b8-a339-481f81fd49de-logs\") pod \"glance-default-internal-api-0\" (UID: \"9514eb24-503a-40b8-a339-481f81fd49de\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:23:49 crc kubenswrapper[5052]: I1203 12:23:49.586330 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage09-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage09-crc\") pod \"glance-default-internal-api-0\" (UID: \"9514eb24-503a-40b8-a339-481f81fd49de\") device mount path \"/mnt/openstack/pv09\"" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:23:49 crc kubenswrapper[5052]: I1203 12:23:49.586587 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/9514eb24-503a-40b8-a339-481f81fd49de-httpd-run\") pod \"glance-default-internal-api-0\" (UID: \"9514eb24-503a-40b8-a339-481f81fd49de\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:23:49 crc kubenswrapper[5052]: I1203 12:23:49.586162 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2bada3d1-568e-484f-b848-532b9d805f17-combined-ca-bundle\") pod \"glance-default-external-api-0\" (UID: \"2bada3d1-568e-484f-b848-532b9d805f17\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:23:49 crc kubenswrapper[5052]: I1203 12:23:49.587175 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/2bada3d1-568e-484f-b848-532b9d805f17-scripts\") pod \"glance-default-external-api-0\" (UID: \"2bada3d1-568e-484f-b848-532b9d805f17\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:23:49 crc kubenswrapper[5052]: I1203 12:23:49.587253 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/9514eb24-503a-40b8-a339-481f81fd49de-scripts\") pod \"glance-default-internal-api-0\" (UID: \"9514eb24-503a-40b8-a339-481f81fd49de\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:23:49 crc kubenswrapper[5052]: I1203 12:23:49.587299 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-ccpnm\" (UniqueName: \"kubernetes.io/projected/9514eb24-503a-40b8-a339-481f81fd49de-kube-api-access-ccpnm\") pod \"glance-default-internal-api-0\" (UID: \"9514eb24-503a-40b8-a339-481f81fd49de\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:23:49 crc kubenswrapper[5052]: I1203 12:23:49.587331 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage07-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage07-crc\") pod \"glance-default-external-api-0\" (UID: \"2bada3d1-568e-484f-b848-532b9d805f17\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:23:49 crc kubenswrapper[5052]: I1203 12:23:49.587379 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/2bada3d1-568e-484f-b848-532b9d805f17-httpd-run\") pod \"glance-default-external-api-0\" (UID: \"2bada3d1-568e-484f-b848-532b9d805f17\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:23:49 crc kubenswrapper[5052]: I1203 12:23:49.587412 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/2bada3d1-568e-484f-b848-532b9d805f17-logs\") pod \"glance-default-external-api-0\" (UID: \"2bada3d1-568e-484f-b848-532b9d805f17\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:23:49 crc kubenswrapper[5052]: I1203 12:23:49.587462 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9514eb24-503a-40b8-a339-481f81fd49de-combined-ca-bundle\") pod \"glance-default-internal-api-0\" (UID: \"9514eb24-503a-40b8-a339-481f81fd49de\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:23:49 crc kubenswrapper[5052]: I1203 12:23:49.587567 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q8fdx\" (UniqueName: \"kubernetes.io/projected/2bada3d1-568e-484f-b848-532b9d805f17-kube-api-access-q8fdx\") pod \"glance-default-external-api-0\" (UID: \"2bada3d1-568e-484f-b848-532b9d805f17\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:23:49 crc kubenswrapper[5052]: I1203 12:23:49.587685 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/2bada3d1-568e-484f-b848-532b9d805f17-config-data\") pod \"glance-default-external-api-0\" (UID: \"2bada3d1-568e-484f-b848-532b9d805f17\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:23:49 crc kubenswrapper[5052]: I1203 12:23:49.616981 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-ccpnm\" (UniqueName: \"kubernetes.io/projected/9514eb24-503a-40b8-a339-481f81fd49de-kube-api-access-ccpnm\") pod \"glance-default-internal-api-0\" (UID: \"9514eb24-503a-40b8-a339-481f81fd49de\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:23:49 crc kubenswrapper[5052]: I1203 12:23:49.627399 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9514eb24-503a-40b8-a339-481f81fd49de-combined-ca-bundle\") pod \"glance-default-internal-api-0\" (UID: \"9514eb24-503a-40b8-a339-481f81fd49de\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:23:49 crc kubenswrapper[5052]: I1203 12:23:49.628240 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/9514eb24-503a-40b8-a339-481f81fd49de-scripts\") pod \"glance-default-internal-api-0\" (UID: \"9514eb24-503a-40b8-a339-481f81fd49de\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:23:49 crc kubenswrapper[5052]: I1203 12:23:49.630651 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/9514eb24-503a-40b8-a339-481f81fd49de-config-data\") pod \"glance-default-internal-api-0\" (UID: \"9514eb24-503a-40b8-a339-481f81fd49de\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:23:49 crc kubenswrapper[5052]: I1203 12:23:49.681004 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage09-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage09-crc\") pod \"glance-default-internal-api-0\" (UID: \"9514eb24-503a-40b8-a339-481f81fd49de\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:23:49 crc kubenswrapper[5052]: I1203 12:23:49.686835 5052 scope.go:117] "RemoveContainer" containerID="09b4a0072741f60499d24f40cc4ff3406d5d22671ced431363ab865172786300" Dec 03 12:23:49 crc kubenswrapper[5052]: E1203 12:23:49.687898 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 12:23:49 crc kubenswrapper[5052]: I1203 12:23:49.689009 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="47bc7c32-d14b-4137-be0d-469efe7b3c9e" path="/var/lib/kubelet/pods/47bc7c32-d14b-4137-be0d-469efe7b3c9e/volumes" Dec 03 12:23:49 crc kubenswrapper[5052]: I1203 12:23:49.689256 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/2bada3d1-568e-484f-b848-532b9d805f17-logs\") pod \"glance-default-external-api-0\" (UID: \"2bada3d1-568e-484f-b848-532b9d805f17\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:23:49 crc kubenswrapper[5052]: I1203 12:23:49.689298 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-q8fdx\" (UniqueName: \"kubernetes.io/projected/2bada3d1-568e-484f-b848-532b9d805f17-kube-api-access-q8fdx\") pod \"glance-default-external-api-0\" (UID: \"2bada3d1-568e-484f-b848-532b9d805f17\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:23:49 crc kubenswrapper[5052]: I1203 12:23:49.689323 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/2bada3d1-568e-484f-b848-532b9d805f17-config-data\") pod \"glance-default-external-api-0\" (UID: \"2bada3d1-568e-484f-b848-532b9d805f17\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:23:49 crc kubenswrapper[5052]: I1203 12:23:49.689398 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2bada3d1-568e-484f-b848-532b9d805f17-combined-ca-bundle\") pod \"glance-default-external-api-0\" (UID: \"2bada3d1-568e-484f-b848-532b9d805f17\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:23:49 crc kubenswrapper[5052]: I1203 12:23:49.689422 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/2bada3d1-568e-484f-b848-532b9d805f17-scripts\") pod \"glance-default-external-api-0\" (UID: \"2bada3d1-568e-484f-b848-532b9d805f17\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:23:49 crc kubenswrapper[5052]: I1203 12:23:49.689465 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage07-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage07-crc\") pod \"glance-default-external-api-0\" (UID: \"2bada3d1-568e-484f-b848-532b9d805f17\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:23:49 crc kubenswrapper[5052]: I1203 12:23:49.689490 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/2bada3d1-568e-484f-b848-532b9d805f17-httpd-run\") pod \"glance-default-external-api-0\" (UID: \"2bada3d1-568e-484f-b848-532b9d805f17\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:23:49 crc kubenswrapper[5052]: I1203 12:23:49.689924 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/2bada3d1-568e-484f-b848-532b9d805f17-httpd-run\") pod \"glance-default-external-api-0\" (UID: \"2bada3d1-568e-484f-b848-532b9d805f17\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:23:49 crc kubenswrapper[5052]: I1203 12:23:49.690096 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b5cc20b6-3503-42f6-b497-8ef83e55317d" path="/var/lib/kubelet/pods/b5cc20b6-3503-42f6-b497-8ef83e55317d/volumes" Dec 03 12:23:49 crc kubenswrapper[5052]: I1203 12:23:49.690139 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/2bada3d1-568e-484f-b848-532b9d805f17-logs\") pod \"glance-default-external-api-0\" (UID: \"2bada3d1-568e-484f-b848-532b9d805f17\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:23:49 crc kubenswrapper[5052]: I1203 12:23:49.692390 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage07-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage07-crc\") pod \"glance-default-external-api-0\" (UID: \"2bada3d1-568e-484f-b848-532b9d805f17\") device mount path \"/mnt/openstack/pv07\"" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:23:49 crc kubenswrapper[5052]: I1203 12:23:49.700119 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2bada3d1-568e-484f-b848-532b9d805f17-combined-ca-bundle\") pod \"glance-default-external-api-0\" (UID: \"2bada3d1-568e-484f-b848-532b9d805f17\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:23:49 crc kubenswrapper[5052]: I1203 12:23:49.702641 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/2bada3d1-568e-484f-b848-532b9d805f17-config-data\") pod \"glance-default-external-api-0\" (UID: \"2bada3d1-568e-484f-b848-532b9d805f17\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:23:49 crc kubenswrapper[5052]: I1203 12:23:49.706162 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/2bada3d1-568e-484f-b848-532b9d805f17-scripts\") pod \"glance-default-external-api-0\" (UID: \"2bada3d1-568e-484f-b848-532b9d805f17\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:23:49 crc kubenswrapper[5052]: I1203 12:23:49.712929 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-q8fdx\" (UniqueName: \"kubernetes.io/projected/2bada3d1-568e-484f-b848-532b9d805f17-kube-api-access-q8fdx\") pod \"glance-default-external-api-0\" (UID: \"2bada3d1-568e-484f-b848-532b9d805f17\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:23:49 crc kubenswrapper[5052]: I1203 12:23:49.724038 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage07-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage07-crc\") pod \"glance-default-external-api-0\" (UID: \"2bada3d1-568e-484f-b848-532b9d805f17\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:23:49 crc kubenswrapper[5052]: I1203 12:23:49.730047 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:23:49 crc kubenswrapper[5052]: I1203 12:23:49.798495 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:23:50 crc kubenswrapper[5052]: I1203 12:23:50.224607 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:23:50 crc kubenswrapper[5052]: I1203 12:23:50.289499 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"2660f107-7c9f-4d25-be86-3ba68a71be57","Type":"ContainerStarted","Data":"72fd85d4d1caba1fa3643029488ed2257e344e275be11a73bcca540b29eb5ce7"} Dec 03 12:23:50 crc kubenswrapper[5052]: I1203 12:23:50.289857 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:23:50 crc kubenswrapper[5052]: I1203 12:23:50.298335 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-db-sync-j76tm" event={"ID":"516cc68a-d42b-40c8-8ca7-a093f47f10d5","Type":"ContainerStarted","Data":"2cf3827a312792b3066340c95e19c6fcb87688eafedbc9bb41c1a04fa7aebf0c"} Dec 03 12:23:50 crc kubenswrapper[5052]: I1203 12:23:50.316555 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/ceilometer-0" podStartSLOduration=2.644540642 podStartE2EDuration="6.316540149s" podCreationTimestamp="2025-12-03 12:23:44 +0000 UTC" firstStartedPulling="2025-12-03 12:23:46.122752979 +0000 UTC m=+3816.710994743" lastFinishedPulling="2025-12-03 12:23:49.794752486 +0000 UTC m=+3820.382994250" observedRunningTime="2025-12-03 12:23:50.313853182 +0000 UTC m=+3820.902094946" watchObservedRunningTime="2025-12-03 12:23:50.316540149 +0000 UTC m=+3820.904781913" Dec 03 12:23:50 crc kubenswrapper[5052]: I1203 12:23:50.333432 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/cinder-db-sync-j76tm" podStartSLOduration=3.3334148 podStartE2EDuration="3.3334148s" podCreationTimestamp="2025-12-03 12:23:47 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:23:50.32955882 +0000 UTC m=+3820.917800584" watchObservedRunningTime="2025-12-03 12:23:50.3334148 +0000 UTC m=+3820.921656564" Dec 03 12:23:50 crc kubenswrapper[5052]: I1203 12:23:50.388681 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:23:50 crc kubenswrapper[5052]: W1203 12:23:50.733704 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2bada3d1_568e_484f_b848_532b9d805f17.slice/crio-7148488c01085c6ad2fccd3a752106d9662fc13f9c598213aad88e6240ebc2e0 WatchSource:0}: Error finding container 7148488c01085c6ad2fccd3a752106d9662fc13f9c598213aad88e6240ebc2e0: Status 404 returned error can't find the container with id 7148488c01085c6ad2fccd3a752106d9662fc13f9c598213aad88e6240ebc2e0 Dec 03 12:23:50 crc kubenswrapper[5052]: I1203 12:23:50.892036 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-bootstrap-mh8wf" Dec 03 12:23:50 crc kubenswrapper[5052]: I1203 12:23:50.896713 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-db-sync-fspvj" Dec 03 12:23:51 crc kubenswrapper[5052]: I1203 12:23:51.013225 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/f21141cf-d708-4da1-8bb7-5900993b9d26-credential-keys\") pod \"f21141cf-d708-4da1-8bb7-5900993b9d26\" (UID: \"f21141cf-d708-4da1-8bb7-5900993b9d26\") " Dec 03 12:23:51 crc kubenswrapper[5052]: I1203 12:23:51.013301 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f5srj\" (UniqueName: \"kubernetes.io/projected/f21141cf-d708-4da1-8bb7-5900993b9d26-kube-api-access-f5srj\") pod \"f21141cf-d708-4da1-8bb7-5900993b9d26\" (UID: \"f21141cf-d708-4da1-8bb7-5900993b9d26\") " Dec 03 12:23:51 crc kubenswrapper[5052]: I1203 12:23:51.013382 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/f21141cf-d708-4da1-8bb7-5900993b9d26-scripts\") pod \"f21141cf-d708-4da1-8bb7-5900993b9d26\" (UID: \"f21141cf-d708-4da1-8bb7-5900993b9d26\") " Dec 03 12:23:51 crc kubenswrapper[5052]: I1203 12:23:51.013414 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/f21141cf-d708-4da1-8bb7-5900993b9d26-config-data\") pod \"f21141cf-d708-4da1-8bb7-5900993b9d26\" (UID: \"f21141cf-d708-4da1-8bb7-5900993b9d26\") " Dec 03 12:23:51 crc kubenswrapper[5052]: I1203 12:23:51.013434 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8698b\" (UniqueName: \"kubernetes.io/projected/6af52f56-d687-43d2-a8da-1780e540ec92-kube-api-access-8698b\") pod \"6af52f56-d687-43d2-a8da-1780e540ec92\" (UID: \"6af52f56-d687-43d2-a8da-1780e540ec92\") " Dec 03 12:23:51 crc kubenswrapper[5052]: I1203 12:23:51.013467 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6af52f56-d687-43d2-a8da-1780e540ec92-combined-ca-bundle\") pod \"6af52f56-d687-43d2-a8da-1780e540ec92\" (UID: \"6af52f56-d687-43d2-a8da-1780e540ec92\") " Dec 03 12:23:51 crc kubenswrapper[5052]: I1203 12:23:51.013590 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/6af52f56-d687-43d2-a8da-1780e540ec92-logs\") pod \"6af52f56-d687-43d2-a8da-1780e540ec92\" (UID: \"6af52f56-d687-43d2-a8da-1780e540ec92\") " Dec 03 12:23:51 crc kubenswrapper[5052]: I1203 12:23:51.013619 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/f21141cf-d708-4da1-8bb7-5900993b9d26-fernet-keys\") pod \"f21141cf-d708-4da1-8bb7-5900993b9d26\" (UID: \"f21141cf-d708-4da1-8bb7-5900993b9d26\") " Dec 03 12:23:51 crc kubenswrapper[5052]: I1203 12:23:51.013637 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f21141cf-d708-4da1-8bb7-5900993b9d26-combined-ca-bundle\") pod \"f21141cf-d708-4da1-8bb7-5900993b9d26\" (UID: \"f21141cf-d708-4da1-8bb7-5900993b9d26\") " Dec 03 12:23:51 crc kubenswrapper[5052]: I1203 12:23:51.013667 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/6af52f56-d687-43d2-a8da-1780e540ec92-scripts\") pod \"6af52f56-d687-43d2-a8da-1780e540ec92\" (UID: \"6af52f56-d687-43d2-a8da-1780e540ec92\") " Dec 03 12:23:51 crc kubenswrapper[5052]: I1203 12:23:51.013691 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6af52f56-d687-43d2-a8da-1780e540ec92-config-data\") pod \"6af52f56-d687-43d2-a8da-1780e540ec92\" (UID: \"6af52f56-d687-43d2-a8da-1780e540ec92\") " Dec 03 12:23:51 crc kubenswrapper[5052]: I1203 12:23:51.015486 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6af52f56-d687-43d2-a8da-1780e540ec92-logs" (OuterVolumeSpecName: "logs") pod "6af52f56-d687-43d2-a8da-1780e540ec92" (UID: "6af52f56-d687-43d2-a8da-1780e540ec92"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:23:51 crc kubenswrapper[5052]: I1203 12:23:51.020930 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f21141cf-d708-4da1-8bb7-5900993b9d26-credential-keys" (OuterVolumeSpecName: "credential-keys") pod "f21141cf-d708-4da1-8bb7-5900993b9d26" (UID: "f21141cf-d708-4da1-8bb7-5900993b9d26"). InnerVolumeSpecName "credential-keys". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:23:51 crc kubenswrapper[5052]: I1203 12:23:51.024016 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f21141cf-d708-4da1-8bb7-5900993b9d26-scripts" (OuterVolumeSpecName: "scripts") pod "f21141cf-d708-4da1-8bb7-5900993b9d26" (UID: "f21141cf-d708-4da1-8bb7-5900993b9d26"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:23:51 crc kubenswrapper[5052]: I1203 12:23:51.024788 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f21141cf-d708-4da1-8bb7-5900993b9d26-fernet-keys" (OuterVolumeSpecName: "fernet-keys") pod "f21141cf-d708-4da1-8bb7-5900993b9d26" (UID: "f21141cf-d708-4da1-8bb7-5900993b9d26"). InnerVolumeSpecName "fernet-keys". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:23:51 crc kubenswrapper[5052]: I1203 12:23:51.024841 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6af52f56-d687-43d2-a8da-1780e540ec92-scripts" (OuterVolumeSpecName: "scripts") pod "6af52f56-d687-43d2-a8da-1780e540ec92" (UID: "6af52f56-d687-43d2-a8da-1780e540ec92"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:23:51 crc kubenswrapper[5052]: I1203 12:23:51.026481 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6af52f56-d687-43d2-a8da-1780e540ec92-kube-api-access-8698b" (OuterVolumeSpecName: "kube-api-access-8698b") pod "6af52f56-d687-43d2-a8da-1780e540ec92" (UID: "6af52f56-d687-43d2-a8da-1780e540ec92"). InnerVolumeSpecName "kube-api-access-8698b". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:23:51 crc kubenswrapper[5052]: I1203 12:23:51.027267 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f21141cf-d708-4da1-8bb7-5900993b9d26-kube-api-access-f5srj" (OuterVolumeSpecName: "kube-api-access-f5srj") pod "f21141cf-d708-4da1-8bb7-5900993b9d26" (UID: "f21141cf-d708-4da1-8bb7-5900993b9d26"). InnerVolumeSpecName "kube-api-access-f5srj". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:23:51 crc kubenswrapper[5052]: I1203 12:23:51.116314 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/6af52f56-d687-43d2-a8da-1780e540ec92-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:23:51 crc kubenswrapper[5052]: I1203 12:23:51.116479 5052 reconciler_common.go:293] "Volume detached for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/f21141cf-d708-4da1-8bb7-5900993b9d26-fernet-keys\") on node \"crc\" DevicePath \"\"" Dec 03 12:23:51 crc kubenswrapper[5052]: I1203 12:23:51.116496 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/6af52f56-d687-43d2-a8da-1780e540ec92-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:23:51 crc kubenswrapper[5052]: I1203 12:23:51.116531 5052 reconciler_common.go:293] "Volume detached for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/f21141cf-d708-4da1-8bb7-5900993b9d26-credential-keys\") on node \"crc\" DevicePath \"\"" Dec 03 12:23:51 crc kubenswrapper[5052]: I1203 12:23:51.116550 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-f5srj\" (UniqueName: \"kubernetes.io/projected/f21141cf-d708-4da1-8bb7-5900993b9d26-kube-api-access-f5srj\") on node \"crc\" DevicePath \"\"" Dec 03 12:23:51 crc kubenswrapper[5052]: I1203 12:23:51.116561 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/f21141cf-d708-4da1-8bb7-5900993b9d26-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:23:51 crc kubenswrapper[5052]: I1203 12:23:51.116572 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-8698b\" (UniqueName: \"kubernetes.io/projected/6af52f56-d687-43d2-a8da-1780e540ec92-kube-api-access-8698b\") on node \"crc\" DevicePath \"\"" Dec 03 12:23:51 crc kubenswrapper[5052]: I1203 12:23:51.149817 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f21141cf-d708-4da1-8bb7-5900993b9d26-config-data" (OuterVolumeSpecName: "config-data") pod "f21141cf-d708-4da1-8bb7-5900993b9d26" (UID: "f21141cf-d708-4da1-8bb7-5900993b9d26"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:23:51 crc kubenswrapper[5052]: I1203 12:23:51.152491 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6af52f56-d687-43d2-a8da-1780e540ec92-config-data" (OuterVolumeSpecName: "config-data") pod "6af52f56-d687-43d2-a8da-1780e540ec92" (UID: "6af52f56-d687-43d2-a8da-1780e540ec92"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:23:51 crc kubenswrapper[5052]: I1203 12:23:51.153776 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6af52f56-d687-43d2-a8da-1780e540ec92-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "6af52f56-d687-43d2-a8da-1780e540ec92" (UID: "6af52f56-d687-43d2-a8da-1780e540ec92"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:23:51 crc kubenswrapper[5052]: I1203 12:23:51.177171 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f21141cf-d708-4da1-8bb7-5900993b9d26-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "f21141cf-d708-4da1-8bb7-5900993b9d26" (UID: "f21141cf-d708-4da1-8bb7-5900993b9d26"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:23:51 crc kubenswrapper[5052]: I1203 12:23:51.218058 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f21141cf-d708-4da1-8bb7-5900993b9d26-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:23:51 crc kubenswrapper[5052]: I1203 12:23:51.218105 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6af52f56-d687-43d2-a8da-1780e540ec92-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:23:51 crc kubenswrapper[5052]: I1203 12:23:51.218117 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/f21141cf-d708-4da1-8bb7-5900993b9d26-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:23:51 crc kubenswrapper[5052]: I1203 12:23:51.218130 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6af52f56-d687-43d2-a8da-1780e540ec92-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:23:51 crc kubenswrapper[5052]: I1203 12:23:51.336120 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-db-sync-fspvj" event={"ID":"6af52f56-d687-43d2-a8da-1780e540ec92","Type":"ContainerDied","Data":"9cafba3cbef85cf1980e296d4145eb069c9366691a317fb35d486130de72ac26"} Dec 03 12:23:51 crc kubenswrapper[5052]: I1203 12:23:51.336163 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9cafba3cbef85cf1980e296d4145eb069c9366691a317fb35d486130de72ac26" Dec 03 12:23:51 crc kubenswrapper[5052]: I1203 12:23:51.336104 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-db-sync-fspvj" Dec 03 12:23:51 crc kubenswrapper[5052]: I1203 12:23:51.366455 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"9514eb24-503a-40b8-a339-481f81fd49de","Type":"ContainerStarted","Data":"4cb0e3dd05921531ebc08c66cb79294572f5d4e8e2b5f2717b30f817250583e2"} Dec 03 12:23:51 crc kubenswrapper[5052]: I1203 12:23:51.374202 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"2bada3d1-568e-484f-b848-532b9d805f17","Type":"ContainerStarted","Data":"7148488c01085c6ad2fccd3a752106d9662fc13f9c598213aad88e6240ebc2e0"} Dec 03 12:23:51 crc kubenswrapper[5052]: I1203 12:23:51.385664 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/placement-84d4fb48db-blkmf"] Dec 03 12:23:51 crc kubenswrapper[5052]: E1203 12:23:51.392141 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="6af52f56-d687-43d2-a8da-1780e540ec92" containerName="placement-db-sync" Dec 03 12:23:51 crc kubenswrapper[5052]: I1203 12:23:51.392186 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="6af52f56-d687-43d2-a8da-1780e540ec92" containerName="placement-db-sync" Dec 03 12:23:51 crc kubenswrapper[5052]: E1203 12:23:51.392239 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="f21141cf-d708-4da1-8bb7-5900993b9d26" containerName="keystone-bootstrap" Dec 03 12:23:51 crc kubenswrapper[5052]: I1203 12:23:51.392248 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="f21141cf-d708-4da1-8bb7-5900993b9d26" containerName="keystone-bootstrap" Dec 03 12:23:51 crc kubenswrapper[5052]: I1203 12:23:51.392639 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="6af52f56-d687-43d2-a8da-1780e540ec92" containerName="placement-db-sync" Dec 03 12:23:51 crc kubenswrapper[5052]: I1203 12:23:51.392669 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="f21141cf-d708-4da1-8bb7-5900993b9d26" containerName="keystone-bootstrap" Dec 03 12:23:51 crc kubenswrapper[5052]: I1203 12:23:51.395095 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-bootstrap-mh8wf" Dec 03 12:23:51 crc kubenswrapper[5052]: I1203 12:23:51.397168 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-bootstrap-mh8wf" event={"ID":"f21141cf-d708-4da1-8bb7-5900993b9d26","Type":"ContainerDied","Data":"f1a94ad5cf4554f9993c31092e25b665ec60c965265b5dbd78fee090ebc7482a"} Dec 03 12:23:51 crc kubenswrapper[5052]: I1203 12:23:51.397225 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f1a94ad5cf4554f9993c31092e25b665ec60c965265b5dbd78fee090ebc7482a" Dec 03 12:23:51 crc kubenswrapper[5052]: I1203 12:23:51.398300 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-84d4fb48db-blkmf" Dec 03 12:23:51 crc kubenswrapper[5052]: I1203 12:23:51.402005 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"placement-scripts" Dec 03 12:23:51 crc kubenswrapper[5052]: I1203 12:23:51.402198 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"placement-config-data" Dec 03 12:23:51 crc kubenswrapper[5052]: I1203 12:23:51.402256 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"placement-placement-dockercfg-5v8pv" Dec 03 12:23:51 crc kubenswrapper[5052]: I1203 12:23:51.410013 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/placement-84d4fb48db-blkmf"] Dec 03 12:23:51 crc kubenswrapper[5052]: I1203 12:23:51.468093 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/keystone-bootstrap-mh8wf"] Dec 03 12:23:51 crc kubenswrapper[5052]: I1203 12:23:51.475502 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/keystone-bootstrap-mh8wf"] Dec 03 12:23:51 crc kubenswrapper[5052]: I1203 12:23:51.533300 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/3b116fe9-1f40-4614-9381-7f7964fbdcb4-config-data\") pod \"placement-84d4fb48db-blkmf\" (UID: \"3b116fe9-1f40-4614-9381-7f7964fbdcb4\") " pod="openstack-kuttl-tests/placement-84d4fb48db-blkmf" Dec 03 12:23:51 crc kubenswrapper[5052]: I1203 12:23:51.533337 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/3b116fe9-1f40-4614-9381-7f7964fbdcb4-scripts\") pod \"placement-84d4fb48db-blkmf\" (UID: \"3b116fe9-1f40-4614-9381-7f7964fbdcb4\") " pod="openstack-kuttl-tests/placement-84d4fb48db-blkmf" Dec 03 12:23:51 crc kubenswrapper[5052]: I1203 12:23:51.533378 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5v9n6\" (UniqueName: \"kubernetes.io/projected/3b116fe9-1f40-4614-9381-7f7964fbdcb4-kube-api-access-5v9n6\") pod \"placement-84d4fb48db-blkmf\" (UID: \"3b116fe9-1f40-4614-9381-7f7964fbdcb4\") " pod="openstack-kuttl-tests/placement-84d4fb48db-blkmf" Dec 03 12:23:51 crc kubenswrapper[5052]: I1203 12:23:51.533411 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3b116fe9-1f40-4614-9381-7f7964fbdcb4-combined-ca-bundle\") pod \"placement-84d4fb48db-blkmf\" (UID: \"3b116fe9-1f40-4614-9381-7f7964fbdcb4\") " pod="openstack-kuttl-tests/placement-84d4fb48db-blkmf" Dec 03 12:23:51 crc kubenswrapper[5052]: I1203 12:23:51.533431 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/3b116fe9-1f40-4614-9381-7f7964fbdcb4-logs\") pod \"placement-84d4fb48db-blkmf\" (UID: \"3b116fe9-1f40-4614-9381-7f7964fbdcb4\") " pod="openstack-kuttl-tests/placement-84d4fb48db-blkmf" Dec 03 12:23:51 crc kubenswrapper[5052]: I1203 12:23:51.544501 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/keystone-bootstrap-vs67d"] Dec 03 12:23:51 crc kubenswrapper[5052]: I1203 12:23:51.546676 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-bootstrap-vs67d" Dec 03 12:23:51 crc kubenswrapper[5052]: I1203 12:23:51.551269 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone-scripts" Dec 03 12:23:51 crc kubenswrapper[5052]: I1203 12:23:51.551612 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone-keystone-dockercfg-9qn8j" Dec 03 12:23:51 crc kubenswrapper[5052]: I1203 12:23:51.551787 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"osp-secret" Dec 03 12:23:51 crc kubenswrapper[5052]: I1203 12:23:51.554670 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone-config-data" Dec 03 12:23:51 crc kubenswrapper[5052]: I1203 12:23:51.554978 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone" Dec 03 12:23:51 crc kubenswrapper[5052]: I1203 12:23:51.558972 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-bootstrap-vs67d"] Dec 03 12:23:51 crc kubenswrapper[5052]: I1203 12:23:51.638846 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/3b116fe9-1f40-4614-9381-7f7964fbdcb4-scripts\") pod \"placement-84d4fb48db-blkmf\" (UID: \"3b116fe9-1f40-4614-9381-7f7964fbdcb4\") " pod="openstack-kuttl-tests/placement-84d4fb48db-blkmf" Dec 03 12:23:51 crc kubenswrapper[5052]: I1203 12:23:51.638924 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/fcc35227-2ba8-4fc9-83fb-b9e40ccd8b32-scripts\") pod \"keystone-bootstrap-vs67d\" (UID: \"fcc35227-2ba8-4fc9-83fb-b9e40ccd8b32\") " pod="openstack-kuttl-tests/keystone-bootstrap-vs67d" Dec 03 12:23:51 crc kubenswrapper[5052]: I1203 12:23:51.638999 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-5v9n6\" (UniqueName: \"kubernetes.io/projected/3b116fe9-1f40-4614-9381-7f7964fbdcb4-kube-api-access-5v9n6\") pod \"placement-84d4fb48db-blkmf\" (UID: \"3b116fe9-1f40-4614-9381-7f7964fbdcb4\") " pod="openstack-kuttl-tests/placement-84d4fb48db-blkmf" Dec 03 12:23:51 crc kubenswrapper[5052]: I1203 12:23:51.639048 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zvm49\" (UniqueName: \"kubernetes.io/projected/fcc35227-2ba8-4fc9-83fb-b9e40ccd8b32-kube-api-access-zvm49\") pod \"keystone-bootstrap-vs67d\" (UID: \"fcc35227-2ba8-4fc9-83fb-b9e40ccd8b32\") " pod="openstack-kuttl-tests/keystone-bootstrap-vs67d" Dec 03 12:23:51 crc kubenswrapper[5052]: I1203 12:23:51.639081 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/fcc35227-2ba8-4fc9-83fb-b9e40ccd8b32-combined-ca-bundle\") pod \"keystone-bootstrap-vs67d\" (UID: \"fcc35227-2ba8-4fc9-83fb-b9e40ccd8b32\") " pod="openstack-kuttl-tests/keystone-bootstrap-vs67d" Dec 03 12:23:51 crc kubenswrapper[5052]: I1203 12:23:51.639154 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3b116fe9-1f40-4614-9381-7f7964fbdcb4-combined-ca-bundle\") pod \"placement-84d4fb48db-blkmf\" (UID: \"3b116fe9-1f40-4614-9381-7f7964fbdcb4\") " pod="openstack-kuttl-tests/placement-84d4fb48db-blkmf" Dec 03 12:23:51 crc kubenswrapper[5052]: I1203 12:23:51.639187 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/3b116fe9-1f40-4614-9381-7f7964fbdcb4-logs\") pod \"placement-84d4fb48db-blkmf\" (UID: \"3b116fe9-1f40-4614-9381-7f7964fbdcb4\") " pod="openstack-kuttl-tests/placement-84d4fb48db-blkmf" Dec 03 12:23:51 crc kubenswrapper[5052]: I1203 12:23:51.639292 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/fcc35227-2ba8-4fc9-83fb-b9e40ccd8b32-config-data\") pod \"keystone-bootstrap-vs67d\" (UID: \"fcc35227-2ba8-4fc9-83fb-b9e40ccd8b32\") " pod="openstack-kuttl-tests/keystone-bootstrap-vs67d" Dec 03 12:23:51 crc kubenswrapper[5052]: I1203 12:23:51.639326 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/fcc35227-2ba8-4fc9-83fb-b9e40ccd8b32-credential-keys\") pod \"keystone-bootstrap-vs67d\" (UID: \"fcc35227-2ba8-4fc9-83fb-b9e40ccd8b32\") " pod="openstack-kuttl-tests/keystone-bootstrap-vs67d" Dec 03 12:23:51 crc kubenswrapper[5052]: I1203 12:23:51.639427 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/fcc35227-2ba8-4fc9-83fb-b9e40ccd8b32-fernet-keys\") pod \"keystone-bootstrap-vs67d\" (UID: \"fcc35227-2ba8-4fc9-83fb-b9e40ccd8b32\") " pod="openstack-kuttl-tests/keystone-bootstrap-vs67d" Dec 03 12:23:51 crc kubenswrapper[5052]: I1203 12:23:51.639580 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/3b116fe9-1f40-4614-9381-7f7964fbdcb4-config-data\") pod \"placement-84d4fb48db-blkmf\" (UID: \"3b116fe9-1f40-4614-9381-7f7964fbdcb4\") " pod="openstack-kuttl-tests/placement-84d4fb48db-blkmf" Dec 03 12:23:51 crc kubenswrapper[5052]: I1203 12:23:51.641297 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/3b116fe9-1f40-4614-9381-7f7964fbdcb4-logs\") pod \"placement-84d4fb48db-blkmf\" (UID: \"3b116fe9-1f40-4614-9381-7f7964fbdcb4\") " pod="openstack-kuttl-tests/placement-84d4fb48db-blkmf" Dec 03 12:23:51 crc kubenswrapper[5052]: I1203 12:23:51.644392 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3b116fe9-1f40-4614-9381-7f7964fbdcb4-combined-ca-bundle\") pod \"placement-84d4fb48db-blkmf\" (UID: \"3b116fe9-1f40-4614-9381-7f7964fbdcb4\") " pod="openstack-kuttl-tests/placement-84d4fb48db-blkmf" Dec 03 12:23:51 crc kubenswrapper[5052]: I1203 12:23:51.645702 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/3b116fe9-1f40-4614-9381-7f7964fbdcb4-config-data\") pod \"placement-84d4fb48db-blkmf\" (UID: \"3b116fe9-1f40-4614-9381-7f7964fbdcb4\") " pod="openstack-kuttl-tests/placement-84d4fb48db-blkmf" Dec 03 12:23:51 crc kubenswrapper[5052]: I1203 12:23:51.646717 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/3b116fe9-1f40-4614-9381-7f7964fbdcb4-scripts\") pod \"placement-84d4fb48db-blkmf\" (UID: \"3b116fe9-1f40-4614-9381-7f7964fbdcb4\") " pod="openstack-kuttl-tests/placement-84d4fb48db-blkmf" Dec 03 12:23:51 crc kubenswrapper[5052]: I1203 12:23:51.672916 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-5v9n6\" (UniqueName: \"kubernetes.io/projected/3b116fe9-1f40-4614-9381-7f7964fbdcb4-kube-api-access-5v9n6\") pod \"placement-84d4fb48db-blkmf\" (UID: \"3b116fe9-1f40-4614-9381-7f7964fbdcb4\") " pod="openstack-kuttl-tests/placement-84d4fb48db-blkmf" Dec 03 12:23:51 crc kubenswrapper[5052]: I1203 12:23:51.689462 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f21141cf-d708-4da1-8bb7-5900993b9d26" path="/var/lib/kubelet/pods/f21141cf-d708-4da1-8bb7-5900993b9d26/volumes" Dec 03 12:23:51 crc kubenswrapper[5052]: I1203 12:23:51.742497 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/fcc35227-2ba8-4fc9-83fb-b9e40ccd8b32-config-data\") pod \"keystone-bootstrap-vs67d\" (UID: \"fcc35227-2ba8-4fc9-83fb-b9e40ccd8b32\") " pod="openstack-kuttl-tests/keystone-bootstrap-vs67d" Dec 03 12:23:51 crc kubenswrapper[5052]: I1203 12:23:51.742601 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/fcc35227-2ba8-4fc9-83fb-b9e40ccd8b32-credential-keys\") pod \"keystone-bootstrap-vs67d\" (UID: \"fcc35227-2ba8-4fc9-83fb-b9e40ccd8b32\") " pod="openstack-kuttl-tests/keystone-bootstrap-vs67d" Dec 03 12:23:51 crc kubenswrapper[5052]: I1203 12:23:51.742719 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/fcc35227-2ba8-4fc9-83fb-b9e40ccd8b32-fernet-keys\") pod \"keystone-bootstrap-vs67d\" (UID: \"fcc35227-2ba8-4fc9-83fb-b9e40ccd8b32\") " pod="openstack-kuttl-tests/keystone-bootstrap-vs67d" Dec 03 12:23:51 crc kubenswrapper[5052]: I1203 12:23:51.743501 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/fcc35227-2ba8-4fc9-83fb-b9e40ccd8b32-scripts\") pod \"keystone-bootstrap-vs67d\" (UID: \"fcc35227-2ba8-4fc9-83fb-b9e40ccd8b32\") " pod="openstack-kuttl-tests/keystone-bootstrap-vs67d" Dec 03 12:23:51 crc kubenswrapper[5052]: I1203 12:23:51.744029 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/fcc35227-2ba8-4fc9-83fb-b9e40ccd8b32-combined-ca-bundle\") pod \"keystone-bootstrap-vs67d\" (UID: \"fcc35227-2ba8-4fc9-83fb-b9e40ccd8b32\") " pod="openstack-kuttl-tests/keystone-bootstrap-vs67d" Dec 03 12:23:51 crc kubenswrapper[5052]: I1203 12:23:51.744053 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-zvm49\" (UniqueName: \"kubernetes.io/projected/fcc35227-2ba8-4fc9-83fb-b9e40ccd8b32-kube-api-access-zvm49\") pod \"keystone-bootstrap-vs67d\" (UID: \"fcc35227-2ba8-4fc9-83fb-b9e40ccd8b32\") " pod="openstack-kuttl-tests/keystone-bootstrap-vs67d" Dec 03 12:23:51 crc kubenswrapper[5052]: I1203 12:23:51.749603 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/fcc35227-2ba8-4fc9-83fb-b9e40ccd8b32-credential-keys\") pod \"keystone-bootstrap-vs67d\" (UID: \"fcc35227-2ba8-4fc9-83fb-b9e40ccd8b32\") " pod="openstack-kuttl-tests/keystone-bootstrap-vs67d" Dec 03 12:23:51 crc kubenswrapper[5052]: I1203 12:23:51.749613 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/fcc35227-2ba8-4fc9-83fb-b9e40ccd8b32-config-data\") pod \"keystone-bootstrap-vs67d\" (UID: \"fcc35227-2ba8-4fc9-83fb-b9e40ccd8b32\") " pod="openstack-kuttl-tests/keystone-bootstrap-vs67d" Dec 03 12:23:51 crc kubenswrapper[5052]: I1203 12:23:51.751169 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/fcc35227-2ba8-4fc9-83fb-b9e40ccd8b32-combined-ca-bundle\") pod \"keystone-bootstrap-vs67d\" (UID: \"fcc35227-2ba8-4fc9-83fb-b9e40ccd8b32\") " pod="openstack-kuttl-tests/keystone-bootstrap-vs67d" Dec 03 12:23:51 crc kubenswrapper[5052]: I1203 12:23:51.753933 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/fcc35227-2ba8-4fc9-83fb-b9e40ccd8b32-scripts\") pod \"keystone-bootstrap-vs67d\" (UID: \"fcc35227-2ba8-4fc9-83fb-b9e40ccd8b32\") " pod="openstack-kuttl-tests/keystone-bootstrap-vs67d" Dec 03 12:23:51 crc kubenswrapper[5052]: I1203 12:23:51.754632 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/fcc35227-2ba8-4fc9-83fb-b9e40ccd8b32-fernet-keys\") pod \"keystone-bootstrap-vs67d\" (UID: \"fcc35227-2ba8-4fc9-83fb-b9e40ccd8b32\") " pod="openstack-kuttl-tests/keystone-bootstrap-vs67d" Dec 03 12:23:51 crc kubenswrapper[5052]: I1203 12:23:51.761639 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-84d4fb48db-blkmf" Dec 03 12:23:51 crc kubenswrapper[5052]: I1203 12:23:51.763234 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-zvm49\" (UniqueName: \"kubernetes.io/projected/fcc35227-2ba8-4fc9-83fb-b9e40ccd8b32-kube-api-access-zvm49\") pod \"keystone-bootstrap-vs67d\" (UID: \"fcc35227-2ba8-4fc9-83fb-b9e40ccd8b32\") " pod="openstack-kuttl-tests/keystone-bootstrap-vs67d" Dec 03 12:23:52 crc kubenswrapper[5052]: I1203 12:23:52.010778 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-bootstrap-vs67d" Dec 03 12:23:52 crc kubenswrapper[5052]: I1203 12:23:52.219888 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/placement-84d4fb48db-blkmf"] Dec 03 12:23:52 crc kubenswrapper[5052]: I1203 12:23:52.405807 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"2bada3d1-568e-484f-b848-532b9d805f17","Type":"ContainerStarted","Data":"65548fd95f21e38842666bf62b603e65a09c2d7f7fef257d2803d9be9f1bcd15"} Dec 03 12:23:52 crc kubenswrapper[5052]: I1203 12:23:52.409368 5052 generic.go:334] "Generic (PLEG): container finished" podID="fa8494cf-ad3c-4998-a3fd-a39863f72aff" containerID="7e368d1baaf0ba061b0efb437531205a49da54db72f9bde12964c76141ba7147" exitCode=0 Dec 03 12:23:52 crc kubenswrapper[5052]: I1203 12:23:52.409493 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-db-sync-j9cr4" event={"ID":"fa8494cf-ad3c-4998-a3fd-a39863f72aff","Type":"ContainerDied","Data":"7e368d1baaf0ba061b0efb437531205a49da54db72f9bde12964c76141ba7147"} Dec 03 12:23:52 crc kubenswrapper[5052]: I1203 12:23:52.411908 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"9514eb24-503a-40b8-a339-481f81fd49de","Type":"ContainerStarted","Data":"6879b4a0fe254ab4b3401ca94ab6f5e16f1a55dd48a45c455affc3cd5f8d6685"} Dec 03 12:23:52 crc kubenswrapper[5052]: I1203 12:23:52.926395 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-bootstrap-vs67d"] Dec 03 12:23:53 crc kubenswrapper[5052]: I1203 12:23:53.422741 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-bootstrap-vs67d" event={"ID":"fcc35227-2ba8-4fc9-83fb-b9e40ccd8b32","Type":"ContainerStarted","Data":"30065432eb4043c50988772e61b0bac64b14ec13363f4e9b1287f2bf7a2de63f"} Dec 03 12:23:53 crc kubenswrapper[5052]: I1203 12:23:53.422787 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-bootstrap-vs67d" event={"ID":"fcc35227-2ba8-4fc9-83fb-b9e40ccd8b32","Type":"ContainerStarted","Data":"6d1557302bd448348cd17583c8d67bf507046da3892d925a865c1417dbce9f3f"} Dec 03 12:23:53 crc kubenswrapper[5052]: I1203 12:23:53.426787 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-84d4fb48db-blkmf" event={"ID":"3b116fe9-1f40-4614-9381-7f7964fbdcb4","Type":"ContainerStarted","Data":"35babc941d9382068ca3592fefa32271466b9c548b0f0eeab1a1fc57b7318180"} Dec 03 12:23:53 crc kubenswrapper[5052]: I1203 12:23:53.426823 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-84d4fb48db-blkmf" event={"ID":"3b116fe9-1f40-4614-9381-7f7964fbdcb4","Type":"ContainerStarted","Data":"c855f9d1a9983bf4428f3f423fd3f1abd57a52f10787a431f76087b5f92f6bd5"} Dec 03 12:23:53 crc kubenswrapper[5052]: I1203 12:23:53.426835 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-84d4fb48db-blkmf" event={"ID":"3b116fe9-1f40-4614-9381-7f7964fbdcb4","Type":"ContainerStarted","Data":"f571af532a2719a0d8c5301009b0f9c62c4a005df1b26f86307be43d5f5f5645"} Dec 03 12:23:53 crc kubenswrapper[5052]: I1203 12:23:53.427542 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/placement-84d4fb48db-blkmf" Dec 03 12:23:53 crc kubenswrapper[5052]: I1203 12:23:53.427592 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/placement-84d4fb48db-blkmf" Dec 03 12:23:53 crc kubenswrapper[5052]: I1203 12:23:53.432502 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"9514eb24-503a-40b8-a339-481f81fd49de","Type":"ContainerStarted","Data":"f6dbc6deaf8843ea5432f48c4bfb01f18f24c61b85c39d693d7ae58e5df45acf"} Dec 03 12:23:53 crc kubenswrapper[5052]: I1203 12:23:53.437378 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"2bada3d1-568e-484f-b848-532b9d805f17","Type":"ContainerStarted","Data":"3bd600fb62ffa6746a30412a469e6249947d48035142d1cdf479776822f1807b"} Dec 03 12:23:53 crc kubenswrapper[5052]: I1203 12:23:53.455756 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/keystone-bootstrap-vs67d" podStartSLOduration=2.455736178 podStartE2EDuration="2.455736178s" podCreationTimestamp="2025-12-03 12:23:51 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:23:53.449043557 +0000 UTC m=+3824.037285321" watchObservedRunningTime="2025-12-03 12:23:53.455736178 +0000 UTC m=+3824.043977942" Dec 03 12:23:53 crc kubenswrapper[5052]: I1203 12:23:53.470261 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/placement-84d4fb48db-blkmf" podStartSLOduration=2.470242392 podStartE2EDuration="2.470242392s" podCreationTimestamp="2025-12-03 12:23:51 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:23:53.468007238 +0000 UTC m=+3824.056249012" watchObservedRunningTime="2025-12-03 12:23:53.470242392 +0000 UTC m=+3824.058484156" Dec 03 12:23:53 crc kubenswrapper[5052]: I1203 12:23:53.496605 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/glance-default-external-api-0" podStartSLOduration=4.496587584 podStartE2EDuration="4.496587584s" podCreationTimestamp="2025-12-03 12:23:49 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:23:53.490201191 +0000 UTC m=+3824.078442975" watchObservedRunningTime="2025-12-03 12:23:53.496587584 +0000 UTC m=+3824.084829348" Dec 03 12:23:53 crc kubenswrapper[5052]: I1203 12:23:53.516481 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/glance-default-internal-api-0" podStartSLOduration=4.516462081 podStartE2EDuration="4.516462081s" podCreationTimestamp="2025-12-03 12:23:49 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:23:53.509357898 +0000 UTC m=+3824.097599672" watchObservedRunningTime="2025-12-03 12:23:53.516462081 +0000 UTC m=+3824.104703845" Dec 03 12:23:53 crc kubenswrapper[5052]: I1203 12:23:53.881343 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-db-sync-j9cr4" Dec 03 12:23:53 crc kubenswrapper[5052]: I1203 12:23:53.989693 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/fa8494cf-ad3c-4998-a3fd-a39863f72aff-db-sync-config-data\") pod \"fa8494cf-ad3c-4998-a3fd-a39863f72aff\" (UID: \"fa8494cf-ad3c-4998-a3fd-a39863f72aff\") " Dec 03 12:23:53 crc kubenswrapper[5052]: I1203 12:23:53.989862 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fz48g\" (UniqueName: \"kubernetes.io/projected/fa8494cf-ad3c-4998-a3fd-a39863f72aff-kube-api-access-fz48g\") pod \"fa8494cf-ad3c-4998-a3fd-a39863f72aff\" (UID: \"fa8494cf-ad3c-4998-a3fd-a39863f72aff\") " Dec 03 12:23:53 crc kubenswrapper[5052]: I1203 12:23:53.990060 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/fa8494cf-ad3c-4998-a3fd-a39863f72aff-combined-ca-bundle\") pod \"fa8494cf-ad3c-4998-a3fd-a39863f72aff\" (UID: \"fa8494cf-ad3c-4998-a3fd-a39863f72aff\") " Dec 03 12:23:53 crc kubenswrapper[5052]: I1203 12:23:53.995731 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fa8494cf-ad3c-4998-a3fd-a39863f72aff-kube-api-access-fz48g" (OuterVolumeSpecName: "kube-api-access-fz48g") pod "fa8494cf-ad3c-4998-a3fd-a39863f72aff" (UID: "fa8494cf-ad3c-4998-a3fd-a39863f72aff"). InnerVolumeSpecName "kube-api-access-fz48g". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:23:53 crc kubenswrapper[5052]: I1203 12:23:53.995951 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fa8494cf-ad3c-4998-a3fd-a39863f72aff-db-sync-config-data" (OuterVolumeSpecName: "db-sync-config-data") pod "fa8494cf-ad3c-4998-a3fd-a39863f72aff" (UID: "fa8494cf-ad3c-4998-a3fd-a39863f72aff"). InnerVolumeSpecName "db-sync-config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:23:54 crc kubenswrapper[5052]: I1203 12:23:54.022198 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fa8494cf-ad3c-4998-a3fd-a39863f72aff-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "fa8494cf-ad3c-4998-a3fd-a39863f72aff" (UID: "fa8494cf-ad3c-4998-a3fd-a39863f72aff"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:23:54 crc kubenswrapper[5052]: I1203 12:23:54.093218 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-fz48g\" (UniqueName: \"kubernetes.io/projected/fa8494cf-ad3c-4998-a3fd-a39863f72aff-kube-api-access-fz48g\") on node \"crc\" DevicePath \"\"" Dec 03 12:23:54 crc kubenswrapper[5052]: I1203 12:23:54.093248 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/fa8494cf-ad3c-4998-a3fd-a39863f72aff-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:23:54 crc kubenswrapper[5052]: I1203 12:23:54.093262 5052 reconciler_common.go:293] "Volume detached for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/fa8494cf-ad3c-4998-a3fd-a39863f72aff-db-sync-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:23:54 crc kubenswrapper[5052]: I1203 12:23:54.458273 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-db-sync-j9cr4" Dec 03 12:23:54 crc kubenswrapper[5052]: I1203 12:23:54.460843 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-db-sync-j9cr4" event={"ID":"fa8494cf-ad3c-4998-a3fd-a39863f72aff","Type":"ContainerDied","Data":"764bfb12d2ec6d7821eea3a915713b90f45046130874a308c822eb14f2da4a17"} Dec 03 12:23:54 crc kubenswrapper[5052]: I1203 12:23:54.460888 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="764bfb12d2ec6d7821eea3a915713b90f45046130874a308c822eb14f2da4a17" Dec 03 12:23:54 crc kubenswrapper[5052]: I1203 12:23:54.638123 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/barbican-worker-75f6bbd957-c444v"] Dec 03 12:23:54 crc kubenswrapper[5052]: E1203 12:23:54.638701 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="fa8494cf-ad3c-4998-a3fd-a39863f72aff" containerName="barbican-db-sync" Dec 03 12:23:54 crc kubenswrapper[5052]: I1203 12:23:54.638725 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="fa8494cf-ad3c-4998-a3fd-a39863f72aff" containerName="barbican-db-sync" Dec 03 12:23:54 crc kubenswrapper[5052]: I1203 12:23:54.638935 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="fa8494cf-ad3c-4998-a3fd-a39863f72aff" containerName="barbican-db-sync" Dec 03 12:23:54 crc kubenswrapper[5052]: I1203 12:23:54.640086 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-worker-75f6bbd957-c444v" Dec 03 12:23:54 crc kubenswrapper[5052]: I1203 12:23:54.644301 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"barbican-worker-config-data" Dec 03 12:23:54 crc kubenswrapper[5052]: I1203 12:23:54.649334 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"barbican-barbican-dockercfg-jfqpw" Dec 03 12:23:54 crc kubenswrapper[5052]: I1203 12:23:54.649653 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"barbican-config-data" Dec 03 12:23:54 crc kubenswrapper[5052]: I1203 12:23:54.732832 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-worker-75f6bbd957-c444v"] Dec 03 12:23:54 crc kubenswrapper[5052]: I1203 12:23:54.785608 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/barbican-keystone-listener-7c58f6698b-95l8v"] Dec 03 12:23:54 crc kubenswrapper[5052]: I1203 12:23:54.787958 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-keystone-listener-7c58f6698b-95l8v" Dec 03 12:23:54 crc kubenswrapper[5052]: I1203 12:23:54.795888 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"barbican-keystone-listener-config-data" Dec 03 12:23:54 crc kubenswrapper[5052]: I1203 12:23:54.807683 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-keystone-listener-7c58f6698b-95l8v"] Dec 03 12:23:54 crc kubenswrapper[5052]: I1203 12:23:54.819064 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/1cc56b2c-23cc-452f-8de3-d7e8bd2e1e68-config-data\") pod \"barbican-worker-75f6bbd957-c444v\" (UID: \"1cc56b2c-23cc-452f-8de3-d7e8bd2e1e68\") " pod="openstack-kuttl-tests/barbican-worker-75f6bbd957-c444v" Dec 03 12:23:54 crc kubenswrapper[5052]: I1203 12:23:54.819134 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/1cc56b2c-23cc-452f-8de3-d7e8bd2e1e68-config-data-custom\") pod \"barbican-worker-75f6bbd957-c444v\" (UID: \"1cc56b2c-23cc-452f-8de3-d7e8bd2e1e68\") " pod="openstack-kuttl-tests/barbican-worker-75f6bbd957-c444v" Dec 03 12:23:54 crc kubenswrapper[5052]: I1203 12:23:54.819211 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vcl4m\" (UniqueName: \"kubernetes.io/projected/1cc56b2c-23cc-452f-8de3-d7e8bd2e1e68-kube-api-access-vcl4m\") pod \"barbican-worker-75f6bbd957-c444v\" (UID: \"1cc56b2c-23cc-452f-8de3-d7e8bd2e1e68\") " pod="openstack-kuttl-tests/barbican-worker-75f6bbd957-c444v" Dec 03 12:23:54 crc kubenswrapper[5052]: I1203 12:23:54.819268 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1cc56b2c-23cc-452f-8de3-d7e8bd2e1e68-combined-ca-bundle\") pod \"barbican-worker-75f6bbd957-c444v\" (UID: \"1cc56b2c-23cc-452f-8de3-d7e8bd2e1e68\") " pod="openstack-kuttl-tests/barbican-worker-75f6bbd957-c444v" Dec 03 12:23:54 crc kubenswrapper[5052]: I1203 12:23:54.819306 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/1cc56b2c-23cc-452f-8de3-d7e8bd2e1e68-logs\") pod \"barbican-worker-75f6bbd957-c444v\" (UID: \"1cc56b2c-23cc-452f-8de3-d7e8bd2e1e68\") " pod="openstack-kuttl-tests/barbican-worker-75f6bbd957-c444v" Dec 03 12:23:54 crc kubenswrapper[5052]: I1203 12:23:54.821046 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/barbican-api-9c7764976-6d6sh"] Dec 03 12:23:54 crc kubenswrapper[5052]: I1203 12:23:54.822694 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-api-9c7764976-6d6sh" Dec 03 12:23:54 crc kubenswrapper[5052]: I1203 12:23:54.825922 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"barbican-api-config-data" Dec 03 12:23:54 crc kubenswrapper[5052]: I1203 12:23:54.845103 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-api-9c7764976-6d6sh"] Dec 03 12:23:54 crc kubenswrapper[5052]: I1203 12:23:54.921531 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/139928b7-e862-4916-8ca2-37917fa153d7-config-data-custom\") pod \"barbican-api-9c7764976-6d6sh\" (UID: \"139928b7-e862-4916-8ca2-37917fa153d7\") " pod="openstack-kuttl-tests/barbican-api-9c7764976-6d6sh" Dec 03 12:23:54 crc kubenswrapper[5052]: I1203 12:23:54.921592 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/1cc56b2c-23cc-452f-8de3-d7e8bd2e1e68-config-data-custom\") pod \"barbican-worker-75f6bbd957-c444v\" (UID: \"1cc56b2c-23cc-452f-8de3-d7e8bd2e1e68\") " pod="openstack-kuttl-tests/barbican-worker-75f6bbd957-c444v" Dec 03 12:23:54 crc kubenswrapper[5052]: I1203 12:23:54.921619 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/139928b7-e862-4916-8ca2-37917fa153d7-logs\") pod \"barbican-api-9c7764976-6d6sh\" (UID: \"139928b7-e862-4916-8ca2-37917fa153d7\") " pod="openstack-kuttl-tests/barbican-api-9c7764976-6d6sh" Dec 03 12:23:54 crc kubenswrapper[5052]: I1203 12:23:54.921687 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kkvbj\" (UniqueName: \"kubernetes.io/projected/139928b7-e862-4916-8ca2-37917fa153d7-kube-api-access-kkvbj\") pod \"barbican-api-9c7764976-6d6sh\" (UID: \"139928b7-e862-4916-8ca2-37917fa153d7\") " pod="openstack-kuttl-tests/barbican-api-9c7764976-6d6sh" Dec 03 12:23:54 crc kubenswrapper[5052]: I1203 12:23:54.921723 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/98d0abd5-a20b-45e3-aa30-34f038b2fb3d-config-data-custom\") pod \"barbican-keystone-listener-7c58f6698b-95l8v\" (UID: \"98d0abd5-a20b-45e3-aa30-34f038b2fb3d\") " pod="openstack-kuttl-tests/barbican-keystone-listener-7c58f6698b-95l8v" Dec 03 12:23:54 crc kubenswrapper[5052]: I1203 12:23:54.921749 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-vcl4m\" (UniqueName: \"kubernetes.io/projected/1cc56b2c-23cc-452f-8de3-d7e8bd2e1e68-kube-api-access-vcl4m\") pod \"barbican-worker-75f6bbd957-c444v\" (UID: \"1cc56b2c-23cc-452f-8de3-d7e8bd2e1e68\") " pod="openstack-kuttl-tests/barbican-worker-75f6bbd957-c444v" Dec 03 12:23:54 crc kubenswrapper[5052]: I1203 12:23:54.921869 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1cc56b2c-23cc-452f-8de3-d7e8bd2e1e68-combined-ca-bundle\") pod \"barbican-worker-75f6bbd957-c444v\" (UID: \"1cc56b2c-23cc-452f-8de3-d7e8bd2e1e68\") " pod="openstack-kuttl-tests/barbican-worker-75f6bbd957-c444v" Dec 03 12:23:54 crc kubenswrapper[5052]: I1203 12:23:54.921922 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-74wdj\" (UniqueName: \"kubernetes.io/projected/98d0abd5-a20b-45e3-aa30-34f038b2fb3d-kube-api-access-74wdj\") pod \"barbican-keystone-listener-7c58f6698b-95l8v\" (UID: \"98d0abd5-a20b-45e3-aa30-34f038b2fb3d\") " pod="openstack-kuttl-tests/barbican-keystone-listener-7c58f6698b-95l8v" Dec 03 12:23:54 crc kubenswrapper[5052]: I1203 12:23:54.921961 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/1cc56b2c-23cc-452f-8de3-d7e8bd2e1e68-logs\") pod \"barbican-worker-75f6bbd957-c444v\" (UID: \"1cc56b2c-23cc-452f-8de3-d7e8bd2e1e68\") " pod="openstack-kuttl-tests/barbican-worker-75f6bbd957-c444v" Dec 03 12:23:54 crc kubenswrapper[5052]: I1203 12:23:54.921989 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/98d0abd5-a20b-45e3-aa30-34f038b2fb3d-config-data\") pod \"barbican-keystone-listener-7c58f6698b-95l8v\" (UID: \"98d0abd5-a20b-45e3-aa30-34f038b2fb3d\") " pod="openstack-kuttl-tests/barbican-keystone-listener-7c58f6698b-95l8v" Dec 03 12:23:54 crc kubenswrapper[5052]: I1203 12:23:54.922025 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/139928b7-e862-4916-8ca2-37917fa153d7-combined-ca-bundle\") pod \"barbican-api-9c7764976-6d6sh\" (UID: \"139928b7-e862-4916-8ca2-37917fa153d7\") " pod="openstack-kuttl-tests/barbican-api-9c7764976-6d6sh" Dec 03 12:23:54 crc kubenswrapper[5052]: I1203 12:23:54.922051 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/98d0abd5-a20b-45e3-aa30-34f038b2fb3d-logs\") pod \"barbican-keystone-listener-7c58f6698b-95l8v\" (UID: \"98d0abd5-a20b-45e3-aa30-34f038b2fb3d\") " pod="openstack-kuttl-tests/barbican-keystone-listener-7c58f6698b-95l8v" Dec 03 12:23:54 crc kubenswrapper[5052]: I1203 12:23:54.922253 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/98d0abd5-a20b-45e3-aa30-34f038b2fb3d-combined-ca-bundle\") pod \"barbican-keystone-listener-7c58f6698b-95l8v\" (UID: \"98d0abd5-a20b-45e3-aa30-34f038b2fb3d\") " pod="openstack-kuttl-tests/barbican-keystone-listener-7c58f6698b-95l8v" Dec 03 12:23:54 crc kubenswrapper[5052]: I1203 12:23:54.922385 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/139928b7-e862-4916-8ca2-37917fa153d7-config-data\") pod \"barbican-api-9c7764976-6d6sh\" (UID: \"139928b7-e862-4916-8ca2-37917fa153d7\") " pod="openstack-kuttl-tests/barbican-api-9c7764976-6d6sh" Dec 03 12:23:54 crc kubenswrapper[5052]: I1203 12:23:54.922423 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/1cc56b2c-23cc-452f-8de3-d7e8bd2e1e68-config-data\") pod \"barbican-worker-75f6bbd957-c444v\" (UID: \"1cc56b2c-23cc-452f-8de3-d7e8bd2e1e68\") " pod="openstack-kuttl-tests/barbican-worker-75f6bbd957-c444v" Dec 03 12:23:54 crc kubenswrapper[5052]: I1203 12:23:54.922496 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/1cc56b2c-23cc-452f-8de3-d7e8bd2e1e68-logs\") pod \"barbican-worker-75f6bbd957-c444v\" (UID: \"1cc56b2c-23cc-452f-8de3-d7e8bd2e1e68\") " pod="openstack-kuttl-tests/barbican-worker-75f6bbd957-c444v" Dec 03 12:23:54 crc kubenswrapper[5052]: I1203 12:23:54.927056 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/1cc56b2c-23cc-452f-8de3-d7e8bd2e1e68-config-data-custom\") pod \"barbican-worker-75f6bbd957-c444v\" (UID: \"1cc56b2c-23cc-452f-8de3-d7e8bd2e1e68\") " pod="openstack-kuttl-tests/barbican-worker-75f6bbd957-c444v" Dec 03 12:23:54 crc kubenswrapper[5052]: I1203 12:23:54.929450 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/1cc56b2c-23cc-452f-8de3-d7e8bd2e1e68-config-data\") pod \"barbican-worker-75f6bbd957-c444v\" (UID: \"1cc56b2c-23cc-452f-8de3-d7e8bd2e1e68\") " pod="openstack-kuttl-tests/barbican-worker-75f6bbd957-c444v" Dec 03 12:23:54 crc kubenswrapper[5052]: I1203 12:23:54.930365 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1cc56b2c-23cc-452f-8de3-d7e8bd2e1e68-combined-ca-bundle\") pod \"barbican-worker-75f6bbd957-c444v\" (UID: \"1cc56b2c-23cc-452f-8de3-d7e8bd2e1e68\") " pod="openstack-kuttl-tests/barbican-worker-75f6bbd957-c444v" Dec 03 12:23:54 crc kubenswrapper[5052]: I1203 12:23:54.944922 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-vcl4m\" (UniqueName: \"kubernetes.io/projected/1cc56b2c-23cc-452f-8de3-d7e8bd2e1e68-kube-api-access-vcl4m\") pod \"barbican-worker-75f6bbd957-c444v\" (UID: \"1cc56b2c-23cc-452f-8de3-d7e8bd2e1e68\") " pod="openstack-kuttl-tests/barbican-worker-75f6bbd957-c444v" Dec 03 12:23:54 crc kubenswrapper[5052]: I1203 12:23:54.964830 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-worker-75f6bbd957-c444v" Dec 03 12:23:55 crc kubenswrapper[5052]: I1203 12:23:55.024057 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-74wdj\" (UniqueName: \"kubernetes.io/projected/98d0abd5-a20b-45e3-aa30-34f038b2fb3d-kube-api-access-74wdj\") pod \"barbican-keystone-listener-7c58f6698b-95l8v\" (UID: \"98d0abd5-a20b-45e3-aa30-34f038b2fb3d\") " pod="openstack-kuttl-tests/barbican-keystone-listener-7c58f6698b-95l8v" Dec 03 12:23:55 crc kubenswrapper[5052]: I1203 12:23:55.024136 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/98d0abd5-a20b-45e3-aa30-34f038b2fb3d-config-data\") pod \"barbican-keystone-listener-7c58f6698b-95l8v\" (UID: \"98d0abd5-a20b-45e3-aa30-34f038b2fb3d\") " pod="openstack-kuttl-tests/barbican-keystone-listener-7c58f6698b-95l8v" Dec 03 12:23:55 crc kubenswrapper[5052]: I1203 12:23:55.024178 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/139928b7-e862-4916-8ca2-37917fa153d7-combined-ca-bundle\") pod \"barbican-api-9c7764976-6d6sh\" (UID: \"139928b7-e862-4916-8ca2-37917fa153d7\") " pod="openstack-kuttl-tests/barbican-api-9c7764976-6d6sh" Dec 03 12:23:55 crc kubenswrapper[5052]: I1203 12:23:55.024203 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/98d0abd5-a20b-45e3-aa30-34f038b2fb3d-logs\") pod \"barbican-keystone-listener-7c58f6698b-95l8v\" (UID: \"98d0abd5-a20b-45e3-aa30-34f038b2fb3d\") " pod="openstack-kuttl-tests/barbican-keystone-listener-7c58f6698b-95l8v" Dec 03 12:23:55 crc kubenswrapper[5052]: I1203 12:23:55.024250 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/98d0abd5-a20b-45e3-aa30-34f038b2fb3d-combined-ca-bundle\") pod \"barbican-keystone-listener-7c58f6698b-95l8v\" (UID: \"98d0abd5-a20b-45e3-aa30-34f038b2fb3d\") " pod="openstack-kuttl-tests/barbican-keystone-listener-7c58f6698b-95l8v" Dec 03 12:23:55 crc kubenswrapper[5052]: I1203 12:23:55.024299 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/139928b7-e862-4916-8ca2-37917fa153d7-config-data\") pod \"barbican-api-9c7764976-6d6sh\" (UID: \"139928b7-e862-4916-8ca2-37917fa153d7\") " pod="openstack-kuttl-tests/barbican-api-9c7764976-6d6sh" Dec 03 12:23:55 crc kubenswrapper[5052]: I1203 12:23:55.024335 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/139928b7-e862-4916-8ca2-37917fa153d7-config-data-custom\") pod \"barbican-api-9c7764976-6d6sh\" (UID: \"139928b7-e862-4916-8ca2-37917fa153d7\") " pod="openstack-kuttl-tests/barbican-api-9c7764976-6d6sh" Dec 03 12:23:55 crc kubenswrapper[5052]: I1203 12:23:55.024362 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/139928b7-e862-4916-8ca2-37917fa153d7-logs\") pod \"barbican-api-9c7764976-6d6sh\" (UID: \"139928b7-e862-4916-8ca2-37917fa153d7\") " pod="openstack-kuttl-tests/barbican-api-9c7764976-6d6sh" Dec 03 12:23:55 crc kubenswrapper[5052]: I1203 12:23:55.024415 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-kkvbj\" (UniqueName: \"kubernetes.io/projected/139928b7-e862-4916-8ca2-37917fa153d7-kube-api-access-kkvbj\") pod \"barbican-api-9c7764976-6d6sh\" (UID: \"139928b7-e862-4916-8ca2-37917fa153d7\") " pod="openstack-kuttl-tests/barbican-api-9c7764976-6d6sh" Dec 03 12:23:55 crc kubenswrapper[5052]: I1203 12:23:55.024446 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/98d0abd5-a20b-45e3-aa30-34f038b2fb3d-config-data-custom\") pod \"barbican-keystone-listener-7c58f6698b-95l8v\" (UID: \"98d0abd5-a20b-45e3-aa30-34f038b2fb3d\") " pod="openstack-kuttl-tests/barbican-keystone-listener-7c58f6698b-95l8v" Dec 03 12:23:55 crc kubenswrapper[5052]: I1203 12:23:55.025782 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/139928b7-e862-4916-8ca2-37917fa153d7-logs\") pod \"barbican-api-9c7764976-6d6sh\" (UID: \"139928b7-e862-4916-8ca2-37917fa153d7\") " pod="openstack-kuttl-tests/barbican-api-9c7764976-6d6sh" Dec 03 12:23:55 crc kubenswrapper[5052]: I1203 12:23:55.026661 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/98d0abd5-a20b-45e3-aa30-34f038b2fb3d-logs\") pod \"barbican-keystone-listener-7c58f6698b-95l8v\" (UID: \"98d0abd5-a20b-45e3-aa30-34f038b2fb3d\") " pod="openstack-kuttl-tests/barbican-keystone-listener-7c58f6698b-95l8v" Dec 03 12:23:55 crc kubenswrapper[5052]: I1203 12:23:55.028818 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/139928b7-e862-4916-8ca2-37917fa153d7-config-data-custom\") pod \"barbican-api-9c7764976-6d6sh\" (UID: \"139928b7-e862-4916-8ca2-37917fa153d7\") " pod="openstack-kuttl-tests/barbican-api-9c7764976-6d6sh" Dec 03 12:23:55 crc kubenswrapper[5052]: I1203 12:23:55.044161 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/139928b7-e862-4916-8ca2-37917fa153d7-config-data\") pod \"barbican-api-9c7764976-6d6sh\" (UID: \"139928b7-e862-4916-8ca2-37917fa153d7\") " pod="openstack-kuttl-tests/barbican-api-9c7764976-6d6sh" Dec 03 12:23:55 crc kubenswrapper[5052]: I1203 12:23:55.045640 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/98d0abd5-a20b-45e3-aa30-34f038b2fb3d-combined-ca-bundle\") pod \"barbican-keystone-listener-7c58f6698b-95l8v\" (UID: \"98d0abd5-a20b-45e3-aa30-34f038b2fb3d\") " pod="openstack-kuttl-tests/barbican-keystone-listener-7c58f6698b-95l8v" Dec 03 12:23:55 crc kubenswrapper[5052]: I1203 12:23:55.046025 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/98d0abd5-a20b-45e3-aa30-34f038b2fb3d-config-data\") pod \"barbican-keystone-listener-7c58f6698b-95l8v\" (UID: \"98d0abd5-a20b-45e3-aa30-34f038b2fb3d\") " pod="openstack-kuttl-tests/barbican-keystone-listener-7c58f6698b-95l8v" Dec 03 12:23:55 crc kubenswrapper[5052]: I1203 12:23:55.046558 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/98d0abd5-a20b-45e3-aa30-34f038b2fb3d-config-data-custom\") pod \"barbican-keystone-listener-7c58f6698b-95l8v\" (UID: \"98d0abd5-a20b-45e3-aa30-34f038b2fb3d\") " pod="openstack-kuttl-tests/barbican-keystone-listener-7c58f6698b-95l8v" Dec 03 12:23:55 crc kubenswrapper[5052]: I1203 12:23:55.049209 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-kkvbj\" (UniqueName: \"kubernetes.io/projected/139928b7-e862-4916-8ca2-37917fa153d7-kube-api-access-kkvbj\") pod \"barbican-api-9c7764976-6d6sh\" (UID: \"139928b7-e862-4916-8ca2-37917fa153d7\") " pod="openstack-kuttl-tests/barbican-api-9c7764976-6d6sh" Dec 03 12:23:55 crc kubenswrapper[5052]: I1203 12:23:55.049237 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/139928b7-e862-4916-8ca2-37917fa153d7-combined-ca-bundle\") pod \"barbican-api-9c7764976-6d6sh\" (UID: \"139928b7-e862-4916-8ca2-37917fa153d7\") " pod="openstack-kuttl-tests/barbican-api-9c7764976-6d6sh" Dec 03 12:23:55 crc kubenswrapper[5052]: I1203 12:23:55.050342 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-74wdj\" (UniqueName: \"kubernetes.io/projected/98d0abd5-a20b-45e3-aa30-34f038b2fb3d-kube-api-access-74wdj\") pod \"barbican-keystone-listener-7c58f6698b-95l8v\" (UID: \"98d0abd5-a20b-45e3-aa30-34f038b2fb3d\") " pod="openstack-kuttl-tests/barbican-keystone-listener-7c58f6698b-95l8v" Dec 03 12:23:55 crc kubenswrapper[5052]: I1203 12:23:55.117121 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-keystone-listener-7c58f6698b-95l8v" Dec 03 12:23:55 crc kubenswrapper[5052]: I1203 12:23:55.155414 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-api-9c7764976-6d6sh" Dec 03 12:23:55 crc kubenswrapper[5052]: I1203 12:23:55.443862 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-worker-75f6bbd957-c444v"] Dec 03 12:23:55 crc kubenswrapper[5052]: W1203 12:23:55.447932 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1cc56b2c_23cc_452f_8de3_d7e8bd2e1e68.slice/crio-1b9e8ca97381e26e986252ff2a28872074667e0e4d1dc2c14a9ef61486656a82 WatchSource:0}: Error finding container 1b9e8ca97381e26e986252ff2a28872074667e0e4d1dc2c14a9ef61486656a82: Status 404 returned error can't find the container with id 1b9e8ca97381e26e986252ff2a28872074667e0e4d1dc2c14a9ef61486656a82 Dec 03 12:23:55 crc kubenswrapper[5052]: I1203 12:23:55.479484 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-worker-75f6bbd957-c444v" event={"ID":"1cc56b2c-23cc-452f-8de3-d7e8bd2e1e68","Type":"ContainerStarted","Data":"1b9e8ca97381e26e986252ff2a28872074667e0e4d1dc2c14a9ef61486656a82"} Dec 03 12:23:55 crc kubenswrapper[5052]: W1203 12:23:55.678861 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod98d0abd5_a20b_45e3_aa30_34f038b2fb3d.slice/crio-8bf3eaccebc33096e14c55c4d10d9463222d82d8e173cb6c456372d194859d9f WatchSource:0}: Error finding container 8bf3eaccebc33096e14c55c4d10d9463222d82d8e173cb6c456372d194859d9f: Status 404 returned error can't find the container with id 8bf3eaccebc33096e14c55c4d10d9463222d82d8e173cb6c456372d194859d9f Dec 03 12:23:55 crc kubenswrapper[5052]: I1203 12:23:55.694033 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-keystone-listener-7c58f6698b-95l8v"] Dec 03 12:23:55 crc kubenswrapper[5052]: I1203 12:23:55.778345 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-api-9c7764976-6d6sh"] Dec 03 12:23:56 crc kubenswrapper[5052]: I1203 12:23:56.492963 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-worker-75f6bbd957-c444v" event={"ID":"1cc56b2c-23cc-452f-8de3-d7e8bd2e1e68","Type":"ContainerStarted","Data":"a5ca1b03848c3e7731d3e96fd0f8faa937c1665505435b783c4113d6afa25f56"} Dec 03 12:23:56 crc kubenswrapper[5052]: I1203 12:23:56.494130 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-worker-75f6bbd957-c444v" event={"ID":"1cc56b2c-23cc-452f-8de3-d7e8bd2e1e68","Type":"ContainerStarted","Data":"cf30dbd3717dce8af354c25d39edb466422569ee896210a1fcd9f69012eca27c"} Dec 03 12:23:56 crc kubenswrapper[5052]: I1203 12:23:56.495623 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-keystone-listener-7c58f6698b-95l8v" event={"ID":"98d0abd5-a20b-45e3-aa30-34f038b2fb3d","Type":"ContainerStarted","Data":"67002cc386bd7047ddbb9535a8f1433365620854531590af15f527d196de051a"} Dec 03 12:23:56 crc kubenswrapper[5052]: I1203 12:23:56.495728 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-keystone-listener-7c58f6698b-95l8v" event={"ID":"98d0abd5-a20b-45e3-aa30-34f038b2fb3d","Type":"ContainerStarted","Data":"06e057704675bafd6f98189c6b86c43903be3fc1372671a1851fba5082cffb21"} Dec 03 12:23:56 crc kubenswrapper[5052]: I1203 12:23:56.495795 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-keystone-listener-7c58f6698b-95l8v" event={"ID":"98d0abd5-a20b-45e3-aa30-34f038b2fb3d","Type":"ContainerStarted","Data":"8bf3eaccebc33096e14c55c4d10d9463222d82d8e173cb6c456372d194859d9f"} Dec 03 12:23:56 crc kubenswrapper[5052]: I1203 12:23:56.497358 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-9c7764976-6d6sh" event={"ID":"139928b7-e862-4916-8ca2-37917fa153d7","Type":"ContainerStarted","Data":"133d1aba158f4f11359e5209c49b976cf1345d171a018b7bb2ac3533efc2b0ca"} Dec 03 12:23:56 crc kubenswrapper[5052]: I1203 12:23:56.497396 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-9c7764976-6d6sh" event={"ID":"139928b7-e862-4916-8ca2-37917fa153d7","Type":"ContainerStarted","Data":"d1675f69e208967311cc8f60ca7ef2dbea31a0db8566c33dc551f51568901da7"} Dec 03 12:23:56 crc kubenswrapper[5052]: I1203 12:23:56.497405 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-9c7764976-6d6sh" event={"ID":"139928b7-e862-4916-8ca2-37917fa153d7","Type":"ContainerStarted","Data":"2fe046756bc72fd306fc5a353f9fe97050a69bd1405fbba116fb78717175cb14"} Dec 03 12:23:56 crc kubenswrapper[5052]: I1203 12:23:56.497689 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/barbican-api-9c7764976-6d6sh" Dec 03 12:23:56 crc kubenswrapper[5052]: I1203 12:23:56.497865 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/barbican-api-9c7764976-6d6sh" Dec 03 12:23:56 crc kubenswrapper[5052]: I1203 12:23:56.500597 5052 generic.go:334] "Generic (PLEG): container finished" podID="516cc68a-d42b-40c8-8ca7-a093f47f10d5" containerID="2cf3827a312792b3066340c95e19c6fcb87688eafedbc9bb41c1a04fa7aebf0c" exitCode=0 Dec 03 12:23:56 crc kubenswrapper[5052]: I1203 12:23:56.500640 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-db-sync-j76tm" event={"ID":"516cc68a-d42b-40c8-8ca7-a093f47f10d5","Type":"ContainerDied","Data":"2cf3827a312792b3066340c95e19c6fcb87688eafedbc9bb41c1a04fa7aebf0c"} Dec 03 12:23:56 crc kubenswrapper[5052]: I1203 12:23:56.519996 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/barbican-worker-75f6bbd957-c444v" podStartSLOduration=2.519977547 podStartE2EDuration="2.519977547s" podCreationTimestamp="2025-12-03 12:23:54 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:23:56.510202558 +0000 UTC m=+3827.098444322" watchObservedRunningTime="2025-12-03 12:23:56.519977547 +0000 UTC m=+3827.108219311" Dec 03 12:23:56 crc kubenswrapper[5052]: I1203 12:23:56.586078 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/barbican-api-9c7764976-6d6sh" podStartSLOduration=2.586060793 podStartE2EDuration="2.586060793s" podCreationTimestamp="2025-12-03 12:23:54 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:23:56.579039453 +0000 UTC m=+3827.167281227" watchObservedRunningTime="2025-12-03 12:23:56.586060793 +0000 UTC m=+3827.174302557" Dec 03 12:23:56 crc kubenswrapper[5052]: I1203 12:23:56.601406 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/barbican-keystone-listener-7c58f6698b-95l8v" podStartSLOduration=2.601388571 podStartE2EDuration="2.601388571s" podCreationTimestamp="2025-12-03 12:23:54 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:23:56.600504726 +0000 UTC m=+3827.188746490" watchObservedRunningTime="2025-12-03 12:23:56.601388571 +0000 UTC m=+3827.189630335" Dec 03 12:23:57 crc kubenswrapper[5052]: I1203 12:23:57.512648 5052 generic.go:334] "Generic (PLEG): container finished" podID="fcc35227-2ba8-4fc9-83fb-b9e40ccd8b32" containerID="30065432eb4043c50988772e61b0bac64b14ec13363f4e9b1287f2bf7a2de63f" exitCode=0 Dec 03 12:23:57 crc kubenswrapper[5052]: I1203 12:23:57.512735 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-bootstrap-vs67d" event={"ID":"fcc35227-2ba8-4fc9-83fb-b9e40ccd8b32","Type":"ContainerDied","Data":"30065432eb4043c50988772e61b0bac64b14ec13363f4e9b1287f2bf7a2de63f"} Dec 03 12:23:57 crc kubenswrapper[5052]: I1203 12:23:57.885773 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-db-sync-j76tm" Dec 03 12:23:57 crc kubenswrapper[5052]: I1203 12:23:57.985230 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/516cc68a-d42b-40c8-8ca7-a093f47f10d5-etc-machine-id\") pod \"516cc68a-d42b-40c8-8ca7-a093f47f10d5\" (UID: \"516cc68a-d42b-40c8-8ca7-a093f47f10d5\") " Dec 03 12:23:57 crc kubenswrapper[5052]: I1203 12:23:57.985306 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/516cc68a-d42b-40c8-8ca7-a093f47f10d5-config-data\") pod \"516cc68a-d42b-40c8-8ca7-a093f47f10d5\" (UID: \"516cc68a-d42b-40c8-8ca7-a093f47f10d5\") " Dec 03 12:23:57 crc kubenswrapper[5052]: I1203 12:23:57.985351 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/516cc68a-d42b-40c8-8ca7-a093f47f10d5-db-sync-config-data\") pod \"516cc68a-d42b-40c8-8ca7-a093f47f10d5\" (UID: \"516cc68a-d42b-40c8-8ca7-a093f47f10d5\") " Dec 03 12:23:57 crc kubenswrapper[5052]: I1203 12:23:57.985430 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jczdq\" (UniqueName: \"kubernetes.io/projected/516cc68a-d42b-40c8-8ca7-a093f47f10d5-kube-api-access-jczdq\") pod \"516cc68a-d42b-40c8-8ca7-a093f47f10d5\" (UID: \"516cc68a-d42b-40c8-8ca7-a093f47f10d5\") " Dec 03 12:23:57 crc kubenswrapper[5052]: I1203 12:23:57.985465 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/516cc68a-d42b-40c8-8ca7-a093f47f10d5-combined-ca-bundle\") pod \"516cc68a-d42b-40c8-8ca7-a093f47f10d5\" (UID: \"516cc68a-d42b-40c8-8ca7-a093f47f10d5\") " Dec 03 12:23:57 crc kubenswrapper[5052]: I1203 12:23:57.985525 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/516cc68a-d42b-40c8-8ca7-a093f47f10d5-scripts\") pod \"516cc68a-d42b-40c8-8ca7-a093f47f10d5\" (UID: \"516cc68a-d42b-40c8-8ca7-a093f47f10d5\") " Dec 03 12:23:57 crc kubenswrapper[5052]: I1203 12:23:57.986571 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/516cc68a-d42b-40c8-8ca7-a093f47f10d5-etc-machine-id" (OuterVolumeSpecName: "etc-machine-id") pod "516cc68a-d42b-40c8-8ca7-a093f47f10d5" (UID: "516cc68a-d42b-40c8-8ca7-a093f47f10d5"). InnerVolumeSpecName "etc-machine-id". PluginName "kubernetes.io/host-path", VolumeGidValue "" Dec 03 12:23:57 crc kubenswrapper[5052]: I1203 12:23:57.993734 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/516cc68a-d42b-40c8-8ca7-a093f47f10d5-kube-api-access-jczdq" (OuterVolumeSpecName: "kube-api-access-jczdq") pod "516cc68a-d42b-40c8-8ca7-a093f47f10d5" (UID: "516cc68a-d42b-40c8-8ca7-a093f47f10d5"). InnerVolumeSpecName "kube-api-access-jczdq". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:23:57 crc kubenswrapper[5052]: I1203 12:23:57.996131 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/516cc68a-d42b-40c8-8ca7-a093f47f10d5-db-sync-config-data" (OuterVolumeSpecName: "db-sync-config-data") pod "516cc68a-d42b-40c8-8ca7-a093f47f10d5" (UID: "516cc68a-d42b-40c8-8ca7-a093f47f10d5"). InnerVolumeSpecName "db-sync-config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:23:57 crc kubenswrapper[5052]: I1203 12:23:57.999240 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/516cc68a-d42b-40c8-8ca7-a093f47f10d5-scripts" (OuterVolumeSpecName: "scripts") pod "516cc68a-d42b-40c8-8ca7-a093f47f10d5" (UID: "516cc68a-d42b-40c8-8ca7-a093f47f10d5"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:23:58 crc kubenswrapper[5052]: I1203 12:23:58.018256 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/516cc68a-d42b-40c8-8ca7-a093f47f10d5-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "516cc68a-d42b-40c8-8ca7-a093f47f10d5" (UID: "516cc68a-d42b-40c8-8ca7-a093f47f10d5"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:23:58 crc kubenswrapper[5052]: I1203 12:23:58.044468 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/516cc68a-d42b-40c8-8ca7-a093f47f10d5-config-data" (OuterVolumeSpecName: "config-data") pod "516cc68a-d42b-40c8-8ca7-a093f47f10d5" (UID: "516cc68a-d42b-40c8-8ca7-a093f47f10d5"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:23:58 crc kubenswrapper[5052]: I1203 12:23:58.087967 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-jczdq\" (UniqueName: \"kubernetes.io/projected/516cc68a-d42b-40c8-8ca7-a093f47f10d5-kube-api-access-jczdq\") on node \"crc\" DevicePath \"\"" Dec 03 12:23:58 crc kubenswrapper[5052]: I1203 12:23:58.088174 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/516cc68a-d42b-40c8-8ca7-a093f47f10d5-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:23:58 crc kubenswrapper[5052]: I1203 12:23:58.088234 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/516cc68a-d42b-40c8-8ca7-a093f47f10d5-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:23:58 crc kubenswrapper[5052]: I1203 12:23:58.088285 5052 reconciler_common.go:293] "Volume detached for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/516cc68a-d42b-40c8-8ca7-a093f47f10d5-etc-machine-id\") on node \"crc\" DevicePath \"\"" Dec 03 12:23:58 crc kubenswrapper[5052]: I1203 12:23:58.088381 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/516cc68a-d42b-40c8-8ca7-a093f47f10d5-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:23:58 crc kubenswrapper[5052]: I1203 12:23:58.088527 5052 reconciler_common.go:293] "Volume detached for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/516cc68a-d42b-40c8-8ca7-a093f47f10d5-db-sync-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:23:58 crc kubenswrapper[5052]: I1203 12:23:58.525734 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-db-sync-j76tm" event={"ID":"516cc68a-d42b-40c8-8ca7-a093f47f10d5","Type":"ContainerDied","Data":"c8ce4daa70d1f8a77bfe3bf6bc7585b0434d0f06a846b28cd4bba56cb2f765b9"} Dec 03 12:23:58 crc kubenswrapper[5052]: I1203 12:23:58.525797 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c8ce4daa70d1f8a77bfe3bf6bc7585b0434d0f06a846b28cd4bba56cb2f765b9" Dec 03 12:23:58 crc kubenswrapper[5052]: I1203 12:23:58.526665 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-db-sync-j76tm" Dec 03 12:23:58 crc kubenswrapper[5052]: I1203 12:23:58.784209 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 12:23:58 crc kubenswrapper[5052]: E1203 12:23:58.785164 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="516cc68a-d42b-40c8-8ca7-a093f47f10d5" containerName="cinder-db-sync" Dec 03 12:23:58 crc kubenswrapper[5052]: I1203 12:23:58.785185 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="516cc68a-d42b-40c8-8ca7-a093f47f10d5" containerName="cinder-db-sync" Dec 03 12:23:58 crc kubenswrapper[5052]: I1203 12:23:58.785357 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="516cc68a-d42b-40c8-8ca7-a093f47f10d5" containerName="cinder-db-sync" Dec 03 12:23:58 crc kubenswrapper[5052]: I1203 12:23:58.788626 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:23:58 crc kubenswrapper[5052]: I1203 12:23:58.794581 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cinder-scheduler-config-data" Dec 03 12:23:58 crc kubenswrapper[5052]: I1203 12:23:58.794772 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cinder-scripts" Dec 03 12:23:58 crc kubenswrapper[5052]: I1203 12:23:58.794863 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cinder-config-data" Dec 03 12:23:58 crc kubenswrapper[5052]: I1203 12:23:58.801465 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cinder-cinder-dockercfg-zd22l" Dec 03 12:23:58 crc kubenswrapper[5052]: I1203 12:23:58.811347 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 12:23:58 crc kubenswrapper[5052]: I1203 12:23:58.853988 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 12:23:58 crc kubenswrapper[5052]: I1203 12:23:58.855387 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:23:58 crc kubenswrapper[5052]: I1203 12:23:58.858860 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cinder-api-config-data" Dec 03 12:23:58 crc kubenswrapper[5052]: I1203 12:23:58.872312 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 12:23:58 crc kubenswrapper[5052]: I1203 12:23:58.903886 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a823b6ba-b2af-4deb-88d2-0902fd6e3780-config-data\") pod \"cinder-scheduler-0\" (UID: \"a823b6ba-b2af-4deb-88d2-0902fd6e3780\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:23:58 crc kubenswrapper[5052]: I1203 12:23:58.903974 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8tjg8\" (UniqueName: \"kubernetes.io/projected/a823b6ba-b2af-4deb-88d2-0902fd6e3780-kube-api-access-8tjg8\") pod \"cinder-scheduler-0\" (UID: \"a823b6ba-b2af-4deb-88d2-0902fd6e3780\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:23:58 crc kubenswrapper[5052]: I1203 12:23:58.904083 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/a823b6ba-b2af-4deb-88d2-0902fd6e3780-etc-machine-id\") pod \"cinder-scheduler-0\" (UID: \"a823b6ba-b2af-4deb-88d2-0902fd6e3780\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:23:58 crc kubenswrapper[5052]: I1203 12:23:58.904166 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/a823b6ba-b2af-4deb-88d2-0902fd6e3780-scripts\") pod \"cinder-scheduler-0\" (UID: \"a823b6ba-b2af-4deb-88d2-0902fd6e3780\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:23:58 crc kubenswrapper[5052]: I1203 12:23:58.904218 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a823b6ba-b2af-4deb-88d2-0902fd6e3780-combined-ca-bundle\") pod \"cinder-scheduler-0\" (UID: \"a823b6ba-b2af-4deb-88d2-0902fd6e3780\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:23:58 crc kubenswrapper[5052]: I1203 12:23:58.904244 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/a823b6ba-b2af-4deb-88d2-0902fd6e3780-config-data-custom\") pod \"cinder-scheduler-0\" (UID: \"a823b6ba-b2af-4deb-88d2-0902fd6e3780\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:23:58 crc kubenswrapper[5052]: I1203 12:23:58.979058 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-bootstrap-vs67d" Dec 03 12:23:59 crc kubenswrapper[5052]: I1203 12:23:59.006634 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/a823b6ba-b2af-4deb-88d2-0902fd6e3780-scripts\") pod \"cinder-scheduler-0\" (UID: \"a823b6ba-b2af-4deb-88d2-0902fd6e3780\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:23:59 crc kubenswrapper[5052]: I1203 12:23:59.006690 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jgmv4\" (UniqueName: \"kubernetes.io/projected/fc4f1efa-a22b-4c43-b479-22562c0a7e65-kube-api-access-jgmv4\") pod \"cinder-api-0\" (UID: \"fc4f1efa-a22b-4c43-b479-22562c0a7e65\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:23:59 crc kubenswrapper[5052]: I1203 12:23:59.006726 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/fc4f1efa-a22b-4c43-b479-22562c0a7e65-logs\") pod \"cinder-api-0\" (UID: \"fc4f1efa-a22b-4c43-b479-22562c0a7e65\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:23:59 crc kubenswrapper[5052]: I1203 12:23:59.006791 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a823b6ba-b2af-4deb-88d2-0902fd6e3780-combined-ca-bundle\") pod \"cinder-scheduler-0\" (UID: \"a823b6ba-b2af-4deb-88d2-0902fd6e3780\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:23:59 crc kubenswrapper[5052]: I1203 12:23:59.006819 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/fc4f1efa-a22b-4c43-b479-22562c0a7e65-config-data\") pod \"cinder-api-0\" (UID: \"fc4f1efa-a22b-4c43-b479-22562c0a7e65\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:23:59 crc kubenswrapper[5052]: I1203 12:23:59.006842 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/a823b6ba-b2af-4deb-88d2-0902fd6e3780-config-data-custom\") pod \"cinder-scheduler-0\" (UID: \"a823b6ba-b2af-4deb-88d2-0902fd6e3780\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:23:59 crc kubenswrapper[5052]: I1203 12:23:59.006873 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/fc4f1efa-a22b-4c43-b479-22562c0a7e65-combined-ca-bundle\") pod \"cinder-api-0\" (UID: \"fc4f1efa-a22b-4c43-b479-22562c0a7e65\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:23:59 crc kubenswrapper[5052]: I1203 12:23:59.006909 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a823b6ba-b2af-4deb-88d2-0902fd6e3780-config-data\") pod \"cinder-scheduler-0\" (UID: \"a823b6ba-b2af-4deb-88d2-0902fd6e3780\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:23:59 crc kubenswrapper[5052]: I1203 12:23:59.007016 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-8tjg8\" (UniqueName: \"kubernetes.io/projected/a823b6ba-b2af-4deb-88d2-0902fd6e3780-kube-api-access-8tjg8\") pod \"cinder-scheduler-0\" (UID: \"a823b6ba-b2af-4deb-88d2-0902fd6e3780\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:23:59 crc kubenswrapper[5052]: I1203 12:23:59.007072 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/fc4f1efa-a22b-4c43-b479-22562c0a7e65-etc-machine-id\") pod \"cinder-api-0\" (UID: \"fc4f1efa-a22b-4c43-b479-22562c0a7e65\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:23:59 crc kubenswrapper[5052]: I1203 12:23:59.007250 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/fc4f1efa-a22b-4c43-b479-22562c0a7e65-config-data-custom\") pod \"cinder-api-0\" (UID: \"fc4f1efa-a22b-4c43-b479-22562c0a7e65\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:23:59 crc kubenswrapper[5052]: I1203 12:23:59.007371 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/fc4f1efa-a22b-4c43-b479-22562c0a7e65-scripts\") pod \"cinder-api-0\" (UID: \"fc4f1efa-a22b-4c43-b479-22562c0a7e65\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:23:59 crc kubenswrapper[5052]: I1203 12:23:59.007420 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/a823b6ba-b2af-4deb-88d2-0902fd6e3780-etc-machine-id\") pod \"cinder-scheduler-0\" (UID: \"a823b6ba-b2af-4deb-88d2-0902fd6e3780\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:23:59 crc kubenswrapper[5052]: I1203 12:23:59.007480 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/a823b6ba-b2af-4deb-88d2-0902fd6e3780-etc-machine-id\") pod \"cinder-scheduler-0\" (UID: \"a823b6ba-b2af-4deb-88d2-0902fd6e3780\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:23:59 crc kubenswrapper[5052]: I1203 12:23:59.011682 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/a823b6ba-b2af-4deb-88d2-0902fd6e3780-scripts\") pod \"cinder-scheduler-0\" (UID: \"a823b6ba-b2af-4deb-88d2-0902fd6e3780\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:23:59 crc kubenswrapper[5052]: I1203 12:23:59.011734 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/a823b6ba-b2af-4deb-88d2-0902fd6e3780-config-data-custom\") pod \"cinder-scheduler-0\" (UID: \"a823b6ba-b2af-4deb-88d2-0902fd6e3780\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:23:59 crc kubenswrapper[5052]: I1203 12:23:59.016325 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a823b6ba-b2af-4deb-88d2-0902fd6e3780-combined-ca-bundle\") pod \"cinder-scheduler-0\" (UID: \"a823b6ba-b2af-4deb-88d2-0902fd6e3780\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:23:59 crc kubenswrapper[5052]: I1203 12:23:59.023988 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a823b6ba-b2af-4deb-88d2-0902fd6e3780-config-data\") pod \"cinder-scheduler-0\" (UID: \"a823b6ba-b2af-4deb-88d2-0902fd6e3780\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:23:59 crc kubenswrapper[5052]: I1203 12:23:59.027072 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-8tjg8\" (UniqueName: \"kubernetes.io/projected/a823b6ba-b2af-4deb-88d2-0902fd6e3780-kube-api-access-8tjg8\") pod \"cinder-scheduler-0\" (UID: \"a823b6ba-b2af-4deb-88d2-0902fd6e3780\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:23:59 crc kubenswrapper[5052]: I1203 12:23:59.108746 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/fcc35227-2ba8-4fc9-83fb-b9e40ccd8b32-config-data\") pod \"fcc35227-2ba8-4fc9-83fb-b9e40ccd8b32\" (UID: \"fcc35227-2ba8-4fc9-83fb-b9e40ccd8b32\") " Dec 03 12:23:59 crc kubenswrapper[5052]: I1203 12:23:59.108803 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/fcc35227-2ba8-4fc9-83fb-b9e40ccd8b32-scripts\") pod \"fcc35227-2ba8-4fc9-83fb-b9e40ccd8b32\" (UID: \"fcc35227-2ba8-4fc9-83fb-b9e40ccd8b32\") " Dec 03 12:23:59 crc kubenswrapper[5052]: I1203 12:23:59.108883 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zvm49\" (UniqueName: \"kubernetes.io/projected/fcc35227-2ba8-4fc9-83fb-b9e40ccd8b32-kube-api-access-zvm49\") pod \"fcc35227-2ba8-4fc9-83fb-b9e40ccd8b32\" (UID: \"fcc35227-2ba8-4fc9-83fb-b9e40ccd8b32\") " Dec 03 12:23:59 crc kubenswrapper[5052]: I1203 12:23:59.108995 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/fcc35227-2ba8-4fc9-83fb-b9e40ccd8b32-combined-ca-bundle\") pod \"fcc35227-2ba8-4fc9-83fb-b9e40ccd8b32\" (UID: \"fcc35227-2ba8-4fc9-83fb-b9e40ccd8b32\") " Dec 03 12:23:59 crc kubenswrapper[5052]: I1203 12:23:59.109034 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/fcc35227-2ba8-4fc9-83fb-b9e40ccd8b32-fernet-keys\") pod \"fcc35227-2ba8-4fc9-83fb-b9e40ccd8b32\" (UID: \"fcc35227-2ba8-4fc9-83fb-b9e40ccd8b32\") " Dec 03 12:23:59 crc kubenswrapper[5052]: I1203 12:23:59.109094 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/fcc35227-2ba8-4fc9-83fb-b9e40ccd8b32-credential-keys\") pod \"fcc35227-2ba8-4fc9-83fb-b9e40ccd8b32\" (UID: \"fcc35227-2ba8-4fc9-83fb-b9e40ccd8b32\") " Dec 03 12:23:59 crc kubenswrapper[5052]: I1203 12:23:59.109330 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-jgmv4\" (UniqueName: \"kubernetes.io/projected/fc4f1efa-a22b-4c43-b479-22562c0a7e65-kube-api-access-jgmv4\") pod \"cinder-api-0\" (UID: \"fc4f1efa-a22b-4c43-b479-22562c0a7e65\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:23:59 crc kubenswrapper[5052]: I1203 12:23:59.109372 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/fc4f1efa-a22b-4c43-b479-22562c0a7e65-logs\") pod \"cinder-api-0\" (UID: \"fc4f1efa-a22b-4c43-b479-22562c0a7e65\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:23:59 crc kubenswrapper[5052]: I1203 12:23:59.109416 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/fc4f1efa-a22b-4c43-b479-22562c0a7e65-config-data\") pod \"cinder-api-0\" (UID: \"fc4f1efa-a22b-4c43-b479-22562c0a7e65\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:23:59 crc kubenswrapper[5052]: I1203 12:23:59.109443 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/fc4f1efa-a22b-4c43-b479-22562c0a7e65-combined-ca-bundle\") pod \"cinder-api-0\" (UID: \"fc4f1efa-a22b-4c43-b479-22562c0a7e65\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:23:59 crc kubenswrapper[5052]: I1203 12:23:59.109493 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/fc4f1efa-a22b-4c43-b479-22562c0a7e65-etc-machine-id\") pod \"cinder-api-0\" (UID: \"fc4f1efa-a22b-4c43-b479-22562c0a7e65\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:23:59 crc kubenswrapper[5052]: I1203 12:23:59.109561 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/fc4f1efa-a22b-4c43-b479-22562c0a7e65-config-data-custom\") pod \"cinder-api-0\" (UID: \"fc4f1efa-a22b-4c43-b479-22562c0a7e65\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:23:59 crc kubenswrapper[5052]: I1203 12:23:59.109603 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/fc4f1efa-a22b-4c43-b479-22562c0a7e65-scripts\") pod \"cinder-api-0\" (UID: \"fc4f1efa-a22b-4c43-b479-22562c0a7e65\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:23:59 crc kubenswrapper[5052]: I1203 12:23:59.109936 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/fc4f1efa-a22b-4c43-b479-22562c0a7e65-logs\") pod \"cinder-api-0\" (UID: \"fc4f1efa-a22b-4c43-b479-22562c0a7e65\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:23:59 crc kubenswrapper[5052]: I1203 12:23:59.111775 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fcc35227-2ba8-4fc9-83fb-b9e40ccd8b32-scripts" (OuterVolumeSpecName: "scripts") pod "fcc35227-2ba8-4fc9-83fb-b9e40ccd8b32" (UID: "fcc35227-2ba8-4fc9-83fb-b9e40ccd8b32"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:23:59 crc kubenswrapper[5052]: I1203 12:23:59.111897 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/fc4f1efa-a22b-4c43-b479-22562c0a7e65-etc-machine-id\") pod \"cinder-api-0\" (UID: \"fc4f1efa-a22b-4c43-b479-22562c0a7e65\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:23:59 crc kubenswrapper[5052]: I1203 12:23:59.113155 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/fc4f1efa-a22b-4c43-b479-22562c0a7e65-scripts\") pod \"cinder-api-0\" (UID: \"fc4f1efa-a22b-4c43-b479-22562c0a7e65\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:23:59 crc kubenswrapper[5052]: I1203 12:23:59.113568 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:23:59 crc kubenswrapper[5052]: I1203 12:23:59.117118 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/fc4f1efa-a22b-4c43-b479-22562c0a7e65-config-data\") pod \"cinder-api-0\" (UID: \"fc4f1efa-a22b-4c43-b479-22562c0a7e65\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:23:59 crc kubenswrapper[5052]: I1203 12:23:59.121370 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fcc35227-2ba8-4fc9-83fb-b9e40ccd8b32-credential-keys" (OuterVolumeSpecName: "credential-keys") pod "fcc35227-2ba8-4fc9-83fb-b9e40ccd8b32" (UID: "fcc35227-2ba8-4fc9-83fb-b9e40ccd8b32"). InnerVolumeSpecName "credential-keys". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:23:59 crc kubenswrapper[5052]: I1203 12:23:59.122056 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fcc35227-2ba8-4fc9-83fb-b9e40ccd8b32-fernet-keys" (OuterVolumeSpecName: "fernet-keys") pod "fcc35227-2ba8-4fc9-83fb-b9e40ccd8b32" (UID: "fcc35227-2ba8-4fc9-83fb-b9e40ccd8b32"). InnerVolumeSpecName "fernet-keys". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:23:59 crc kubenswrapper[5052]: I1203 12:23:59.122201 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fcc35227-2ba8-4fc9-83fb-b9e40ccd8b32-kube-api-access-zvm49" (OuterVolumeSpecName: "kube-api-access-zvm49") pod "fcc35227-2ba8-4fc9-83fb-b9e40ccd8b32" (UID: "fcc35227-2ba8-4fc9-83fb-b9e40ccd8b32"). InnerVolumeSpecName "kube-api-access-zvm49". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:23:59 crc kubenswrapper[5052]: I1203 12:23:59.122587 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/fc4f1efa-a22b-4c43-b479-22562c0a7e65-combined-ca-bundle\") pod \"cinder-api-0\" (UID: \"fc4f1efa-a22b-4c43-b479-22562c0a7e65\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:23:59 crc kubenswrapper[5052]: I1203 12:23:59.125116 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/fc4f1efa-a22b-4c43-b479-22562c0a7e65-config-data-custom\") pod \"cinder-api-0\" (UID: \"fc4f1efa-a22b-4c43-b479-22562c0a7e65\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:23:59 crc kubenswrapper[5052]: I1203 12:23:59.130840 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-jgmv4\" (UniqueName: \"kubernetes.io/projected/fc4f1efa-a22b-4c43-b479-22562c0a7e65-kube-api-access-jgmv4\") pod \"cinder-api-0\" (UID: \"fc4f1efa-a22b-4c43-b479-22562c0a7e65\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:23:59 crc kubenswrapper[5052]: I1203 12:23:59.143655 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fcc35227-2ba8-4fc9-83fb-b9e40ccd8b32-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "fcc35227-2ba8-4fc9-83fb-b9e40ccd8b32" (UID: "fcc35227-2ba8-4fc9-83fb-b9e40ccd8b32"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:23:59 crc kubenswrapper[5052]: I1203 12:23:59.158339 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fcc35227-2ba8-4fc9-83fb-b9e40ccd8b32-config-data" (OuterVolumeSpecName: "config-data") pod "fcc35227-2ba8-4fc9-83fb-b9e40ccd8b32" (UID: "fcc35227-2ba8-4fc9-83fb-b9e40ccd8b32"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:23:59 crc kubenswrapper[5052]: I1203 12:23:59.174742 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:23:59 crc kubenswrapper[5052]: I1203 12:23:59.211301 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/fcc35227-2ba8-4fc9-83fb-b9e40ccd8b32-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:23:59 crc kubenswrapper[5052]: I1203 12:23:59.211350 5052 reconciler_common.go:293] "Volume detached for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/fcc35227-2ba8-4fc9-83fb-b9e40ccd8b32-fernet-keys\") on node \"crc\" DevicePath \"\"" Dec 03 12:23:59 crc kubenswrapper[5052]: I1203 12:23:59.211362 5052 reconciler_common.go:293] "Volume detached for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/fcc35227-2ba8-4fc9-83fb-b9e40ccd8b32-credential-keys\") on node \"crc\" DevicePath \"\"" Dec 03 12:23:59 crc kubenswrapper[5052]: I1203 12:23:59.211373 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/fcc35227-2ba8-4fc9-83fb-b9e40ccd8b32-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:23:59 crc kubenswrapper[5052]: I1203 12:23:59.211385 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/fcc35227-2ba8-4fc9-83fb-b9e40ccd8b32-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:23:59 crc kubenswrapper[5052]: I1203 12:23:59.211397 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-zvm49\" (UniqueName: \"kubernetes.io/projected/fcc35227-2ba8-4fc9-83fb-b9e40ccd8b32-kube-api-access-zvm49\") on node \"crc\" DevicePath \"\"" Dec 03 12:23:59 crc kubenswrapper[5052]: I1203 12:23:59.538885 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-bootstrap-vs67d" event={"ID":"fcc35227-2ba8-4fc9-83fb-b9e40ccd8b32","Type":"ContainerDied","Data":"6d1557302bd448348cd17583c8d67bf507046da3892d925a865c1417dbce9f3f"} Dec 03 12:23:59 crc kubenswrapper[5052]: I1203 12:23:59.539200 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6d1557302bd448348cd17583c8d67bf507046da3892d925a865c1417dbce9f3f" Dec 03 12:23:59 crc kubenswrapper[5052]: I1203 12:23:59.538933 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-bootstrap-vs67d" Dec 03 12:23:59 crc kubenswrapper[5052]: I1203 12:23:59.627257 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 12:23:59 crc kubenswrapper[5052]: W1203 12:23:59.633250 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda823b6ba_b2af_4deb_88d2_0902fd6e3780.slice/crio-675dcfa8f588697349c3715d475dceca2167384ac3d0acfe92c067220c149936 WatchSource:0}: Error finding container 675dcfa8f588697349c3715d475dceca2167384ac3d0acfe92c067220c149936: Status 404 returned error can't find the container with id 675dcfa8f588697349c3715d475dceca2167384ac3d0acfe92c067220c149936 Dec 03 12:23:59 crc kubenswrapper[5052]: I1203 12:23:59.649749 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/keystone-7995cfc4b8-v5sr4"] Dec 03 12:23:59 crc kubenswrapper[5052]: E1203 12:23:59.650193 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="fcc35227-2ba8-4fc9-83fb-b9e40ccd8b32" containerName="keystone-bootstrap" Dec 03 12:23:59 crc kubenswrapper[5052]: I1203 12:23:59.650208 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="fcc35227-2ba8-4fc9-83fb-b9e40ccd8b32" containerName="keystone-bootstrap" Dec 03 12:23:59 crc kubenswrapper[5052]: I1203 12:23:59.650422 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="fcc35227-2ba8-4fc9-83fb-b9e40ccd8b32" containerName="keystone-bootstrap" Dec 03 12:23:59 crc kubenswrapper[5052]: I1203 12:23:59.651060 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-7995cfc4b8-v5sr4" Dec 03 12:23:59 crc kubenswrapper[5052]: I1203 12:23:59.653933 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone" Dec 03 12:23:59 crc kubenswrapper[5052]: I1203 12:23:59.654056 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone-scripts" Dec 03 12:23:59 crc kubenswrapper[5052]: I1203 12:23:59.654216 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone-config-data" Dec 03 12:23:59 crc kubenswrapper[5052]: I1203 12:23:59.654288 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone-keystone-dockercfg-9qn8j" Dec 03 12:23:59 crc kubenswrapper[5052]: I1203 12:23:59.664489 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-7995cfc4b8-v5sr4"] Dec 03 12:23:59 crc kubenswrapper[5052]: I1203 12:23:59.714055 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 12:23:59 crc kubenswrapper[5052]: I1203 12:23:59.729566 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:23:59 crc kubenswrapper[5052]: I1203 12:23:59.730773 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:23:59 crc kubenswrapper[5052]: I1203 12:23:59.787138 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:23:59 crc kubenswrapper[5052]: I1203 12:23:59.802737 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:23:59 crc kubenswrapper[5052]: I1203 12:23:59.802784 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:23:59 crc kubenswrapper[5052]: I1203 12:23:59.803306 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:23:59 crc kubenswrapper[5052]: I1203 12:23:59.819746 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c2463445-211e-4724-ba2c-4cc1661a6452-combined-ca-bundle\") pod \"keystone-7995cfc4b8-v5sr4\" (UID: \"c2463445-211e-4724-ba2c-4cc1661a6452\") " pod="openstack-kuttl-tests/keystone-7995cfc4b8-v5sr4" Dec 03 12:23:59 crc kubenswrapper[5052]: I1203 12:23:59.819817 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/c2463445-211e-4724-ba2c-4cc1661a6452-fernet-keys\") pod \"keystone-7995cfc4b8-v5sr4\" (UID: \"c2463445-211e-4724-ba2c-4cc1661a6452\") " pod="openstack-kuttl-tests/keystone-7995cfc4b8-v5sr4" Dec 03 12:23:59 crc kubenswrapper[5052]: I1203 12:23:59.819959 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/c2463445-211e-4724-ba2c-4cc1661a6452-scripts\") pod \"keystone-7995cfc4b8-v5sr4\" (UID: \"c2463445-211e-4724-ba2c-4cc1661a6452\") " pod="openstack-kuttl-tests/keystone-7995cfc4b8-v5sr4" Dec 03 12:23:59 crc kubenswrapper[5052]: I1203 12:23:59.819993 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4c8ms\" (UniqueName: \"kubernetes.io/projected/c2463445-211e-4724-ba2c-4cc1661a6452-kube-api-access-4c8ms\") pod \"keystone-7995cfc4b8-v5sr4\" (UID: \"c2463445-211e-4724-ba2c-4cc1661a6452\") " pod="openstack-kuttl-tests/keystone-7995cfc4b8-v5sr4" Dec 03 12:23:59 crc kubenswrapper[5052]: I1203 12:23:59.820052 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/c2463445-211e-4724-ba2c-4cc1661a6452-credential-keys\") pod \"keystone-7995cfc4b8-v5sr4\" (UID: \"c2463445-211e-4724-ba2c-4cc1661a6452\") " pod="openstack-kuttl-tests/keystone-7995cfc4b8-v5sr4" Dec 03 12:23:59 crc kubenswrapper[5052]: I1203 12:23:59.820096 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/c2463445-211e-4724-ba2c-4cc1661a6452-config-data\") pod \"keystone-7995cfc4b8-v5sr4\" (UID: \"c2463445-211e-4724-ba2c-4cc1661a6452\") " pod="openstack-kuttl-tests/keystone-7995cfc4b8-v5sr4" Dec 03 12:23:59 crc kubenswrapper[5052]: I1203 12:23:59.862285 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:23:59 crc kubenswrapper[5052]: I1203 12:23:59.924588 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c2463445-211e-4724-ba2c-4cc1661a6452-combined-ca-bundle\") pod \"keystone-7995cfc4b8-v5sr4\" (UID: \"c2463445-211e-4724-ba2c-4cc1661a6452\") " pod="openstack-kuttl-tests/keystone-7995cfc4b8-v5sr4" Dec 03 12:23:59 crc kubenswrapper[5052]: I1203 12:23:59.924848 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/c2463445-211e-4724-ba2c-4cc1661a6452-fernet-keys\") pod \"keystone-7995cfc4b8-v5sr4\" (UID: \"c2463445-211e-4724-ba2c-4cc1661a6452\") " pod="openstack-kuttl-tests/keystone-7995cfc4b8-v5sr4" Dec 03 12:23:59 crc kubenswrapper[5052]: I1203 12:23:59.925122 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/c2463445-211e-4724-ba2c-4cc1661a6452-scripts\") pod \"keystone-7995cfc4b8-v5sr4\" (UID: \"c2463445-211e-4724-ba2c-4cc1661a6452\") " pod="openstack-kuttl-tests/keystone-7995cfc4b8-v5sr4" Dec 03 12:23:59 crc kubenswrapper[5052]: I1203 12:23:59.925145 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-4c8ms\" (UniqueName: \"kubernetes.io/projected/c2463445-211e-4724-ba2c-4cc1661a6452-kube-api-access-4c8ms\") pod \"keystone-7995cfc4b8-v5sr4\" (UID: \"c2463445-211e-4724-ba2c-4cc1661a6452\") " pod="openstack-kuttl-tests/keystone-7995cfc4b8-v5sr4" Dec 03 12:23:59 crc kubenswrapper[5052]: I1203 12:23:59.925189 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/c2463445-211e-4724-ba2c-4cc1661a6452-credential-keys\") pod \"keystone-7995cfc4b8-v5sr4\" (UID: \"c2463445-211e-4724-ba2c-4cc1661a6452\") " pod="openstack-kuttl-tests/keystone-7995cfc4b8-v5sr4" Dec 03 12:23:59 crc kubenswrapper[5052]: I1203 12:23:59.925214 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/c2463445-211e-4724-ba2c-4cc1661a6452-config-data\") pod \"keystone-7995cfc4b8-v5sr4\" (UID: \"c2463445-211e-4724-ba2c-4cc1661a6452\") " pod="openstack-kuttl-tests/keystone-7995cfc4b8-v5sr4" Dec 03 12:23:59 crc kubenswrapper[5052]: I1203 12:23:59.931090 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c2463445-211e-4724-ba2c-4cc1661a6452-combined-ca-bundle\") pod \"keystone-7995cfc4b8-v5sr4\" (UID: \"c2463445-211e-4724-ba2c-4cc1661a6452\") " pod="openstack-kuttl-tests/keystone-7995cfc4b8-v5sr4" Dec 03 12:23:59 crc kubenswrapper[5052]: I1203 12:23:59.931897 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/c2463445-211e-4724-ba2c-4cc1661a6452-scripts\") pod \"keystone-7995cfc4b8-v5sr4\" (UID: \"c2463445-211e-4724-ba2c-4cc1661a6452\") " pod="openstack-kuttl-tests/keystone-7995cfc4b8-v5sr4" Dec 03 12:23:59 crc kubenswrapper[5052]: I1203 12:23:59.932026 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/c2463445-211e-4724-ba2c-4cc1661a6452-config-data\") pod \"keystone-7995cfc4b8-v5sr4\" (UID: \"c2463445-211e-4724-ba2c-4cc1661a6452\") " pod="openstack-kuttl-tests/keystone-7995cfc4b8-v5sr4" Dec 03 12:23:59 crc kubenswrapper[5052]: I1203 12:23:59.932366 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/c2463445-211e-4724-ba2c-4cc1661a6452-fernet-keys\") pod \"keystone-7995cfc4b8-v5sr4\" (UID: \"c2463445-211e-4724-ba2c-4cc1661a6452\") " pod="openstack-kuttl-tests/keystone-7995cfc4b8-v5sr4" Dec 03 12:23:59 crc kubenswrapper[5052]: I1203 12:23:59.933966 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/c2463445-211e-4724-ba2c-4cc1661a6452-credential-keys\") pod \"keystone-7995cfc4b8-v5sr4\" (UID: \"c2463445-211e-4724-ba2c-4cc1661a6452\") " pod="openstack-kuttl-tests/keystone-7995cfc4b8-v5sr4" Dec 03 12:23:59 crc kubenswrapper[5052]: I1203 12:23:59.944255 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-4c8ms\" (UniqueName: \"kubernetes.io/projected/c2463445-211e-4724-ba2c-4cc1661a6452-kube-api-access-4c8ms\") pod \"keystone-7995cfc4b8-v5sr4\" (UID: \"c2463445-211e-4724-ba2c-4cc1661a6452\") " pod="openstack-kuttl-tests/keystone-7995cfc4b8-v5sr4" Dec 03 12:23:59 crc kubenswrapper[5052]: I1203 12:23:59.989036 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-7995cfc4b8-v5sr4" Dec 03 12:24:00 crc kubenswrapper[5052]: I1203 12:24:00.434435 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:24:00 crc kubenswrapper[5052]: I1203 12:24:00.466643 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-7995cfc4b8-v5sr4"] Dec 03 12:24:00 crc kubenswrapper[5052]: I1203 12:24:00.559069 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-7995cfc4b8-v5sr4" event={"ID":"c2463445-211e-4724-ba2c-4cc1661a6452","Type":"ContainerStarted","Data":"c4af836f28c749948b99e079ba0c0fc678410b9514c9f531a96c216a8d8cfade"} Dec 03 12:24:00 crc kubenswrapper[5052]: I1203 12:24:00.560901 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"a823b6ba-b2af-4deb-88d2-0902fd6e3780","Type":"ContainerStarted","Data":"675dcfa8f588697349c3715d475dceca2167384ac3d0acfe92c067220c149936"} Dec 03 12:24:00 crc kubenswrapper[5052]: I1203 12:24:00.563648 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"fc4f1efa-a22b-4c43-b479-22562c0a7e65","Type":"ContainerStarted","Data":"761c802e32e772d570305634f6352b2399bac37933208705f9277668225fb2d3"} Dec 03 12:24:00 crc kubenswrapper[5052]: I1203 12:24:00.563739 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:24:00 crc kubenswrapper[5052]: I1203 12:24:00.564565 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:24:00 crc kubenswrapper[5052]: I1203 12:24:00.564588 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:24:00 crc kubenswrapper[5052]: I1203 12:24:00.564600 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:24:00 crc kubenswrapper[5052]: I1203 12:24:00.674141 5052 scope.go:117] "RemoveContainer" containerID="09b4a0072741f60499d24f40cc4ff3406d5d22671ced431363ab865172786300" Dec 03 12:24:00 crc kubenswrapper[5052]: E1203 12:24:00.674665 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 12:24:01 crc kubenswrapper[5052]: I1203 12:24:01.581675 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-7995cfc4b8-v5sr4" event={"ID":"c2463445-211e-4724-ba2c-4cc1661a6452","Type":"ContainerStarted","Data":"314b48c25f8f96b0a4711b6bf034891f8f2066552bba485c2589fd24ede9c77d"} Dec 03 12:24:01 crc kubenswrapper[5052]: I1203 12:24:01.582574 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/keystone-7995cfc4b8-v5sr4" Dec 03 12:24:01 crc kubenswrapper[5052]: I1203 12:24:01.597373 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"a823b6ba-b2af-4deb-88d2-0902fd6e3780","Type":"ContainerStarted","Data":"6cd2d71e42ed2933a0efd0de7d0e300b0da269f757f4f18802db5fd7de523192"} Dec 03 12:24:01 crc kubenswrapper[5052]: I1203 12:24:01.597417 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"a823b6ba-b2af-4deb-88d2-0902fd6e3780","Type":"ContainerStarted","Data":"53d7dc20a3dd7792e212463451921763c0652bd6c02e21fa9b1e85765b8915ca"} Dec 03 12:24:01 crc kubenswrapper[5052]: I1203 12:24:01.614245 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/keystone-7995cfc4b8-v5sr4" podStartSLOduration=2.614221428 podStartE2EDuration="2.614221428s" podCreationTimestamp="2025-12-03 12:23:59 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:24:01.607829095 +0000 UTC m=+3832.196070859" watchObservedRunningTime="2025-12-03 12:24:01.614221428 +0000 UTC m=+3832.202463192" Dec 03 12:24:01 crc kubenswrapper[5052]: I1203 12:24:01.619606 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"fc4f1efa-a22b-4c43-b479-22562c0a7e65","Type":"ContainerStarted","Data":"b9e28cc89fab64aa07a40886930111804a359cb4003f7a994ed468c2084e2abf"} Dec 03 12:24:01 crc kubenswrapper[5052]: I1203 12:24:01.652045 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/cinder-scheduler-0" podStartSLOduration=3.6520235359999997 podStartE2EDuration="3.652023536s" podCreationTimestamp="2025-12-03 12:23:58 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:24:01.637688877 +0000 UTC m=+3832.225930661" watchObservedRunningTime="2025-12-03 12:24:01.652023536 +0000 UTC m=+3832.240265300" Dec 03 12:24:02 crc kubenswrapper[5052]: I1203 12:24:02.631644 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"fc4f1efa-a22b-4c43-b479-22562c0a7e65","Type":"ContainerStarted","Data":"00bb5044fc1f5280459a013a740283ad38025f50730707308a452a1f487b74f0"} Dec 03 12:24:02 crc kubenswrapper[5052]: I1203 12:24:02.631729 5052 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Dec 03 12:24:02 crc kubenswrapper[5052]: I1203 12:24:02.631741 5052 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Dec 03 12:24:02 crc kubenswrapper[5052]: I1203 12:24:02.631954 5052 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Dec 03 12:24:02 crc kubenswrapper[5052]: I1203 12:24:02.631966 5052 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Dec 03 12:24:02 crc kubenswrapper[5052]: I1203 12:24:02.667683 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/cinder-api-0" podStartSLOduration=4.667664714 podStartE2EDuration="4.667664714s" podCreationTimestamp="2025-12-03 12:23:58 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:24:02.653309215 +0000 UTC m=+3833.241550989" watchObservedRunningTime="2025-12-03 12:24:02.667664714 +0000 UTC m=+3833.255906478" Dec 03 12:24:02 crc kubenswrapper[5052]: I1203 12:24:02.947669 5052 prober.go:107] "Probe failed" probeType="Liveness" pod="openstack-kuttl-tests/barbican-api-9c7764976-6d6sh" podUID="139928b7-e862-4916-8ca2-37917fa153d7" containerName="barbican-api" probeResult="failure" output="HTTP probe failed with statuscode: 500" Dec 03 12:24:02 crc kubenswrapper[5052]: I1203 12:24:02.948195 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:24:02 crc kubenswrapper[5052]: I1203 12:24:02.961251 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:24:02 crc kubenswrapper[5052]: I1203 12:24:02.961566 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:24:03 crc kubenswrapper[5052]: I1203 12:24:03.049503 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:24:03 crc kubenswrapper[5052]: I1203 12:24:03.645202 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:24:04 crc kubenswrapper[5052]: I1203 12:24:04.113645 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:24:06 crc kubenswrapper[5052]: I1203 12:24:06.607223 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/barbican-api-9c7764976-6d6sh" Dec 03 12:24:06 crc kubenswrapper[5052]: I1203 12:24:06.609231 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/barbican-api-9c7764976-6d6sh" Dec 03 12:24:06 crc kubenswrapper[5052]: I1203 12:24:06.867999 5052 scope.go:117] "RemoveContainer" containerID="e152d21042dfbb59504e298835f095942b27ff95b7e38c26f308eec742685448" Dec 03 12:24:06 crc kubenswrapper[5052]: I1203 12:24:06.906065 5052 scope.go:117] "RemoveContainer" containerID="4432cd8497f4f957b73472527d488a26bff3ce7f4676d54c429f0ab40edb3759" Dec 03 12:24:06 crc kubenswrapper[5052]: I1203 12:24:06.966873 5052 scope.go:117] "RemoveContainer" containerID="4d3f21da798b8c70e54adbb15ba838c217865bc978d044ed0c58d262e9c8b5e3" Dec 03 12:24:07 crc kubenswrapper[5052]: I1203 12:24:07.012499 5052 scope.go:117] "RemoveContainer" containerID="b646684ca873f921091ec3cfe94a27d3c5b4b13b7ebda3dd17e9b77bd7ef3b12" Dec 03 12:24:07 crc kubenswrapper[5052]: I1203 12:24:07.052084 5052 scope.go:117] "RemoveContainer" containerID="815f97f7cb31b7b78a79807aa95002614e45122852e5f05422072d06574a839a" Dec 03 12:24:07 crc kubenswrapper[5052]: I1203 12:24:07.094299 5052 scope.go:117] "RemoveContainer" containerID="518459180ad34e42e3d4890b691c685fdb215176ae41fec275f3e19b1b7907ae" Dec 03 12:24:07 crc kubenswrapper[5052]: I1203 12:24:07.131184 5052 scope.go:117] "RemoveContainer" containerID="7d8e7144eee1a253836215f98a2a034d3756074819c1433459b21b25e1261483" Dec 03 12:24:07 crc kubenswrapper[5052]: I1203 12:24:07.196054 5052 scope.go:117] "RemoveContainer" containerID="1cfce437c4e384a34c57bced804c64f40f25e45c8f65ee39ad0277b7e37ec7f0" Dec 03 12:24:07 crc kubenswrapper[5052]: I1203 12:24:07.236668 5052 scope.go:117] "RemoveContainer" containerID="ecb8cd2f428a35b04fce42b144a6f3606e98b3dcaa31eb3623b92627a69a0813" Dec 03 12:24:07 crc kubenswrapper[5052]: I1203 12:24:07.257764 5052 scope.go:117] "RemoveContainer" containerID="c8beb7f8c222fb4c94ee6aadf2ded6eb09f46f942e334eb694d5a1683fcf80f9" Dec 03 12:24:07 crc kubenswrapper[5052]: I1203 12:24:07.279215 5052 scope.go:117] "RemoveContainer" containerID="d26b41a42104863894f831fd9b945032f3ccd5ce0630c9e4673313dfb9e41580" Dec 03 12:24:07 crc kubenswrapper[5052]: I1203 12:24:07.304460 5052 scope.go:117] "RemoveContainer" containerID="e4692207d1b05926e8b6ed27de18dc9d4ea3c35cfccd32439b2b9da23da7ddaf" Dec 03 12:24:07 crc kubenswrapper[5052]: I1203 12:24:07.333319 5052 scope.go:117] "RemoveContainer" containerID="05e702674cf68c3b05c75966c10349cb828cdd967a55c92bca4816aad25891d7" Dec 03 12:24:07 crc kubenswrapper[5052]: I1203 12:24:07.365117 5052 scope.go:117] "RemoveContainer" containerID="c181c3bdf14ee9b20cbdd2f9d53262cf57598b2c6bebd77c669f32d823697e1b" Dec 03 12:24:07 crc kubenswrapper[5052]: I1203 12:24:07.405560 5052 scope.go:117] "RemoveContainer" containerID="0e69c8cda10283dde61afaf75a7b595710a8bdadd8fdf004e7e5e8d3fa479050" Dec 03 12:24:07 crc kubenswrapper[5052]: I1203 12:24:07.446855 5052 scope.go:117] "RemoveContainer" containerID="7ee346888c18afd8eeba9670f320334edd272704bcbcc941c871d902c641c5ec" Dec 03 12:24:07 crc kubenswrapper[5052]: I1203 12:24:07.468489 5052 scope.go:117] "RemoveContainer" containerID="0152fe83dd8c9d0afd4f66464b63a8a659d1d2dddf64d378d7b1c0444939feca" Dec 03 12:24:09 crc kubenswrapper[5052]: I1203 12:24:09.400277 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:24:11 crc kubenswrapper[5052]: I1203 12:24:11.088326 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:24:13 crc kubenswrapper[5052]: I1203 12:24:13.772967 5052 generic.go:334] "Generic (PLEG): container finished" podID="778e86ee-3618-41fa-83c3-eb28609f50e2" containerID="ef3ec7b7f63c95d670ec6dce26e58fcf598dfd84857be75804ebf18715f6be7d" exitCode=0 Dec 03 12:24:13 crc kubenswrapper[5052]: I1203 12:24:13.773014 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-db-sync-ggltp" event={"ID":"778e86ee-3618-41fa-83c3-eb28609f50e2","Type":"ContainerDied","Data":"ef3ec7b7f63c95d670ec6dce26e58fcf598dfd84857be75804ebf18715f6be7d"} Dec 03 12:24:14 crc kubenswrapper[5052]: I1203 12:24:14.672849 5052 scope.go:117] "RemoveContainer" containerID="09b4a0072741f60499d24f40cc4ff3406d5d22671ced431363ab865172786300" Dec 03 12:24:14 crc kubenswrapper[5052]: E1203 12:24:14.673208 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 12:24:14 crc kubenswrapper[5052]: I1203 12:24:14.750902 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:24:15 crc kubenswrapper[5052]: I1203 12:24:15.179061 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-db-sync-ggltp" Dec 03 12:24:15 crc kubenswrapper[5052]: I1203 12:24:15.215656 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/778e86ee-3618-41fa-83c3-eb28609f50e2-config\") pod \"778e86ee-3618-41fa-83c3-eb28609f50e2\" (UID: \"778e86ee-3618-41fa-83c3-eb28609f50e2\") " Dec 03 12:24:15 crc kubenswrapper[5052]: I1203 12:24:15.215717 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/778e86ee-3618-41fa-83c3-eb28609f50e2-combined-ca-bundle\") pod \"778e86ee-3618-41fa-83c3-eb28609f50e2\" (UID: \"778e86ee-3618-41fa-83c3-eb28609f50e2\") " Dec 03 12:24:15 crc kubenswrapper[5052]: I1203 12:24:15.215813 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ltkxl\" (UniqueName: \"kubernetes.io/projected/778e86ee-3618-41fa-83c3-eb28609f50e2-kube-api-access-ltkxl\") pod \"778e86ee-3618-41fa-83c3-eb28609f50e2\" (UID: \"778e86ee-3618-41fa-83c3-eb28609f50e2\") " Dec 03 12:24:15 crc kubenswrapper[5052]: I1203 12:24:15.223879 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/778e86ee-3618-41fa-83c3-eb28609f50e2-kube-api-access-ltkxl" (OuterVolumeSpecName: "kube-api-access-ltkxl") pod "778e86ee-3618-41fa-83c3-eb28609f50e2" (UID: "778e86ee-3618-41fa-83c3-eb28609f50e2"). InnerVolumeSpecName "kube-api-access-ltkxl". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:24:15 crc kubenswrapper[5052]: I1203 12:24:15.246285 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/778e86ee-3618-41fa-83c3-eb28609f50e2-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "778e86ee-3618-41fa-83c3-eb28609f50e2" (UID: "778e86ee-3618-41fa-83c3-eb28609f50e2"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:24:15 crc kubenswrapper[5052]: I1203 12:24:15.246486 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/778e86ee-3618-41fa-83c3-eb28609f50e2-config" (OuterVolumeSpecName: "config") pod "778e86ee-3618-41fa-83c3-eb28609f50e2" (UID: "778e86ee-3618-41fa-83c3-eb28609f50e2"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:24:15 crc kubenswrapper[5052]: I1203 12:24:15.318786 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/778e86ee-3618-41fa-83c3-eb28609f50e2-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:24:15 crc kubenswrapper[5052]: I1203 12:24:15.318828 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/778e86ee-3618-41fa-83c3-eb28609f50e2-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:24:15 crc kubenswrapper[5052]: I1203 12:24:15.318840 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-ltkxl\" (UniqueName: \"kubernetes.io/projected/778e86ee-3618-41fa-83c3-eb28609f50e2-kube-api-access-ltkxl\") on node \"crc\" DevicePath \"\"" Dec 03 12:24:15 crc kubenswrapper[5052]: I1203 12:24:15.795831 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-db-sync-ggltp" event={"ID":"778e86ee-3618-41fa-83c3-eb28609f50e2","Type":"ContainerDied","Data":"87728977e888f686d67bc05c24bd18d9f7c5e06ec6aee6775e0049b269205dab"} Dec 03 12:24:15 crc kubenswrapper[5052]: I1203 12:24:15.795866 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="87728977e888f686d67bc05c24bd18d9f7c5e06ec6aee6775e0049b269205dab" Dec 03 12:24:15 crc kubenswrapper[5052]: I1203 12:24:15.795889 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-db-sync-ggltp" Dec 03 12:24:16 crc kubenswrapper[5052]: I1203 12:24:16.118265 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/neutron-f699c657d-qh2qk"] Dec 03 12:24:16 crc kubenswrapper[5052]: E1203 12:24:16.119034 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="778e86ee-3618-41fa-83c3-eb28609f50e2" containerName="neutron-db-sync" Dec 03 12:24:16 crc kubenswrapper[5052]: I1203 12:24:16.119052 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="778e86ee-3618-41fa-83c3-eb28609f50e2" containerName="neutron-db-sync" Dec 03 12:24:16 crc kubenswrapper[5052]: I1203 12:24:16.119437 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="778e86ee-3618-41fa-83c3-eb28609f50e2" containerName="neutron-db-sync" Dec 03 12:24:16 crc kubenswrapper[5052]: I1203 12:24:16.123496 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-f699c657d-qh2qk" Dec 03 12:24:16 crc kubenswrapper[5052]: I1203 12:24:16.138556 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-f699c657d-qh2qk"] Dec 03 12:24:16 crc kubenswrapper[5052]: I1203 12:24:16.146902 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"neutron-config" Dec 03 12:24:16 crc kubenswrapper[5052]: I1203 12:24:16.147195 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"neutron-httpd-config" Dec 03 12:24:16 crc kubenswrapper[5052]: I1203 12:24:16.148373 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"neutron-neutron-dockercfg-bvk99" Dec 03 12:24:16 crc kubenswrapper[5052]: I1203 12:24:16.240503 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/a28346c0-bd60-4dac-ac86-20b56296e7d3-httpd-config\") pod \"neutron-f699c657d-qh2qk\" (UID: \"a28346c0-bd60-4dac-ac86-20b56296e7d3\") " pod="openstack-kuttl-tests/neutron-f699c657d-qh2qk" Dec 03 12:24:16 crc kubenswrapper[5052]: I1203 12:24:16.240578 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/a28346c0-bd60-4dac-ac86-20b56296e7d3-config\") pod \"neutron-f699c657d-qh2qk\" (UID: \"a28346c0-bd60-4dac-ac86-20b56296e7d3\") " pod="openstack-kuttl-tests/neutron-f699c657d-qh2qk" Dec 03 12:24:16 crc kubenswrapper[5052]: I1203 12:24:16.240605 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a28346c0-bd60-4dac-ac86-20b56296e7d3-combined-ca-bundle\") pod \"neutron-f699c657d-qh2qk\" (UID: \"a28346c0-bd60-4dac-ac86-20b56296e7d3\") " pod="openstack-kuttl-tests/neutron-f699c657d-qh2qk" Dec 03 12:24:16 crc kubenswrapper[5052]: I1203 12:24:16.240629 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tpmbq\" (UniqueName: \"kubernetes.io/projected/a28346c0-bd60-4dac-ac86-20b56296e7d3-kube-api-access-tpmbq\") pod \"neutron-f699c657d-qh2qk\" (UID: \"a28346c0-bd60-4dac-ac86-20b56296e7d3\") " pod="openstack-kuttl-tests/neutron-f699c657d-qh2qk" Dec 03 12:24:16 crc kubenswrapper[5052]: I1203 12:24:16.342482 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/a28346c0-bd60-4dac-ac86-20b56296e7d3-httpd-config\") pod \"neutron-f699c657d-qh2qk\" (UID: \"a28346c0-bd60-4dac-ac86-20b56296e7d3\") " pod="openstack-kuttl-tests/neutron-f699c657d-qh2qk" Dec 03 12:24:16 crc kubenswrapper[5052]: I1203 12:24:16.342581 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/a28346c0-bd60-4dac-ac86-20b56296e7d3-config\") pod \"neutron-f699c657d-qh2qk\" (UID: \"a28346c0-bd60-4dac-ac86-20b56296e7d3\") " pod="openstack-kuttl-tests/neutron-f699c657d-qh2qk" Dec 03 12:24:16 crc kubenswrapper[5052]: I1203 12:24:16.342624 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a28346c0-bd60-4dac-ac86-20b56296e7d3-combined-ca-bundle\") pod \"neutron-f699c657d-qh2qk\" (UID: \"a28346c0-bd60-4dac-ac86-20b56296e7d3\") " pod="openstack-kuttl-tests/neutron-f699c657d-qh2qk" Dec 03 12:24:16 crc kubenswrapper[5052]: I1203 12:24:16.342654 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-tpmbq\" (UniqueName: \"kubernetes.io/projected/a28346c0-bd60-4dac-ac86-20b56296e7d3-kube-api-access-tpmbq\") pod \"neutron-f699c657d-qh2qk\" (UID: \"a28346c0-bd60-4dac-ac86-20b56296e7d3\") " pod="openstack-kuttl-tests/neutron-f699c657d-qh2qk" Dec 03 12:24:16 crc kubenswrapper[5052]: I1203 12:24:16.350573 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/a28346c0-bd60-4dac-ac86-20b56296e7d3-httpd-config\") pod \"neutron-f699c657d-qh2qk\" (UID: \"a28346c0-bd60-4dac-ac86-20b56296e7d3\") " pod="openstack-kuttl-tests/neutron-f699c657d-qh2qk" Dec 03 12:24:16 crc kubenswrapper[5052]: I1203 12:24:16.350683 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/a28346c0-bd60-4dac-ac86-20b56296e7d3-config\") pod \"neutron-f699c657d-qh2qk\" (UID: \"a28346c0-bd60-4dac-ac86-20b56296e7d3\") " pod="openstack-kuttl-tests/neutron-f699c657d-qh2qk" Dec 03 12:24:16 crc kubenswrapper[5052]: I1203 12:24:16.365333 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a28346c0-bd60-4dac-ac86-20b56296e7d3-combined-ca-bundle\") pod \"neutron-f699c657d-qh2qk\" (UID: \"a28346c0-bd60-4dac-ac86-20b56296e7d3\") " pod="openstack-kuttl-tests/neutron-f699c657d-qh2qk" Dec 03 12:24:16 crc kubenswrapper[5052]: I1203 12:24:16.370327 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-tpmbq\" (UniqueName: \"kubernetes.io/projected/a28346c0-bd60-4dac-ac86-20b56296e7d3-kube-api-access-tpmbq\") pod \"neutron-f699c657d-qh2qk\" (UID: \"a28346c0-bd60-4dac-ac86-20b56296e7d3\") " pod="openstack-kuttl-tests/neutron-f699c657d-qh2qk" Dec 03 12:24:16 crc kubenswrapper[5052]: I1203 12:24:16.466854 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-f699c657d-qh2qk" Dec 03 12:24:16 crc kubenswrapper[5052]: I1203 12:24:16.939845 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-f699c657d-qh2qk"] Dec 03 12:24:16 crc kubenswrapper[5052]: W1203 12:24:16.943809 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda28346c0_bd60_4dac_ac86_20b56296e7d3.slice/crio-164683c6068bdd4d22a79ff8eadc4cb11d9a95ca3e9f04852bf04d50eb2c5e81 WatchSource:0}: Error finding container 164683c6068bdd4d22a79ff8eadc4cb11d9a95ca3e9f04852bf04d50eb2c5e81: Status 404 returned error can't find the container with id 164683c6068bdd4d22a79ff8eadc4cb11d9a95ca3e9f04852bf04d50eb2c5e81 Dec 03 12:24:17 crc kubenswrapper[5052]: I1203 12:24:17.818255 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-f699c657d-qh2qk" event={"ID":"a28346c0-bd60-4dac-ac86-20b56296e7d3","Type":"ContainerStarted","Data":"d10711abf49e5a87c80edf05ade50ea33f8c777fb550c91e299195151230bed0"} Dec 03 12:24:17 crc kubenswrapper[5052]: I1203 12:24:17.819081 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-f699c657d-qh2qk" event={"ID":"a28346c0-bd60-4dac-ac86-20b56296e7d3","Type":"ContainerStarted","Data":"c55b9fbfdc46a4cb2612d4910e298c39d926f0f567206d0d4eb4cba1b2cf66ed"} Dec 03 12:24:17 crc kubenswrapper[5052]: I1203 12:24:17.819098 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-f699c657d-qh2qk" event={"ID":"a28346c0-bd60-4dac-ac86-20b56296e7d3","Type":"ContainerStarted","Data":"164683c6068bdd4d22a79ff8eadc4cb11d9a95ca3e9f04852bf04d50eb2c5e81"} Dec 03 12:24:17 crc kubenswrapper[5052]: I1203 12:24:17.819139 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/neutron-f699c657d-qh2qk" Dec 03 12:24:17 crc kubenswrapper[5052]: I1203 12:24:17.842185 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/neutron-f699c657d-qh2qk" podStartSLOduration=1.8421338760000001 podStartE2EDuration="1.842133876s" podCreationTimestamp="2025-12-03 12:24:16 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:24:17.839369657 +0000 UTC m=+3848.427611441" watchObservedRunningTime="2025-12-03 12:24:17.842133876 +0000 UTC m=+3848.430375640" Dec 03 12:24:22 crc kubenswrapper[5052]: I1203 12:24:22.870740 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/placement-84d4fb48db-blkmf" Dec 03 12:24:22 crc kubenswrapper[5052]: I1203 12:24:22.916952 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/placement-84d4fb48db-blkmf" Dec 03 12:24:27 crc kubenswrapper[5052]: I1203 12:24:27.672933 5052 scope.go:117] "RemoveContainer" containerID="09b4a0072741f60499d24f40cc4ff3406d5d22671ced431363ab865172786300" Dec 03 12:24:27 crc kubenswrapper[5052]: E1203 12:24:27.673796 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 12:24:31 crc kubenswrapper[5052]: I1203 12:24:31.937525 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/keystone-7995cfc4b8-v5sr4" Dec 03 12:24:34 crc kubenswrapper[5052]: I1203 12:24:34.087121 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/openstackclient"] Dec 03 12:24:34 crc kubenswrapper[5052]: I1203 12:24:34.089724 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstackclient" Dec 03 12:24:34 crc kubenswrapper[5052]: I1203 12:24:34.092422 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"openstack-config-secret" Dec 03 12:24:34 crc kubenswrapper[5052]: I1203 12:24:34.092605 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"openstackclient-openstackclient-dockercfg-p42s9" Dec 03 12:24:34 crc kubenswrapper[5052]: I1203 12:24:34.092910 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"openstack-config" Dec 03 12:24:34 crc kubenswrapper[5052]: I1203 12:24:34.114754 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/openstackclient"] Dec 03 12:24:34 crc kubenswrapper[5052]: I1203 12:24:34.170584 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openstack-config-secret\" (UniqueName: \"kubernetes.io/secret/78630d10-b9ba-438d-907d-2be706fc51e7-openstack-config-secret\") pod \"openstackclient\" (UID: \"78630d10-b9ba-438d-907d-2be706fc51e7\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 12:24:34 crc kubenswrapper[5052]: I1203 12:24:34.170755 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/78630d10-b9ba-438d-907d-2be706fc51e7-combined-ca-bundle\") pod \"openstackclient\" (UID: \"78630d10-b9ba-438d-907d-2be706fc51e7\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 12:24:34 crc kubenswrapper[5052]: I1203 12:24:34.170792 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-85595\" (UniqueName: \"kubernetes.io/projected/78630d10-b9ba-438d-907d-2be706fc51e7-kube-api-access-85595\") pod \"openstackclient\" (UID: \"78630d10-b9ba-438d-907d-2be706fc51e7\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 12:24:34 crc kubenswrapper[5052]: I1203 12:24:34.171065 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openstack-config\" (UniqueName: \"kubernetes.io/configmap/78630d10-b9ba-438d-907d-2be706fc51e7-openstack-config\") pod \"openstackclient\" (UID: \"78630d10-b9ba-438d-907d-2be706fc51e7\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 12:24:34 crc kubenswrapper[5052]: I1203 12:24:34.272946 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"openstack-config\" (UniqueName: \"kubernetes.io/configmap/78630d10-b9ba-438d-907d-2be706fc51e7-openstack-config\") pod \"openstackclient\" (UID: \"78630d10-b9ba-438d-907d-2be706fc51e7\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 12:24:34 crc kubenswrapper[5052]: I1203 12:24:34.273025 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"openstack-config-secret\" (UniqueName: \"kubernetes.io/secret/78630d10-b9ba-438d-907d-2be706fc51e7-openstack-config-secret\") pod \"openstackclient\" (UID: \"78630d10-b9ba-438d-907d-2be706fc51e7\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 12:24:34 crc kubenswrapper[5052]: I1203 12:24:34.273066 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/78630d10-b9ba-438d-907d-2be706fc51e7-combined-ca-bundle\") pod \"openstackclient\" (UID: \"78630d10-b9ba-438d-907d-2be706fc51e7\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 12:24:34 crc kubenswrapper[5052]: I1203 12:24:34.273093 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-85595\" (UniqueName: \"kubernetes.io/projected/78630d10-b9ba-438d-907d-2be706fc51e7-kube-api-access-85595\") pod \"openstackclient\" (UID: \"78630d10-b9ba-438d-907d-2be706fc51e7\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 12:24:34 crc kubenswrapper[5052]: I1203 12:24:34.274794 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"openstack-config\" (UniqueName: \"kubernetes.io/configmap/78630d10-b9ba-438d-907d-2be706fc51e7-openstack-config\") pod \"openstackclient\" (UID: \"78630d10-b9ba-438d-907d-2be706fc51e7\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 12:24:34 crc kubenswrapper[5052]: I1203 12:24:34.283264 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"openstack-config-secret\" (UniqueName: \"kubernetes.io/secret/78630d10-b9ba-438d-907d-2be706fc51e7-openstack-config-secret\") pod \"openstackclient\" (UID: \"78630d10-b9ba-438d-907d-2be706fc51e7\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 12:24:34 crc kubenswrapper[5052]: I1203 12:24:34.291413 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/78630d10-b9ba-438d-907d-2be706fc51e7-combined-ca-bundle\") pod \"openstackclient\" (UID: \"78630d10-b9ba-438d-907d-2be706fc51e7\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 12:24:34 crc kubenswrapper[5052]: I1203 12:24:34.293345 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-85595\" (UniqueName: \"kubernetes.io/projected/78630d10-b9ba-438d-907d-2be706fc51e7-kube-api-access-85595\") pod \"openstackclient\" (UID: \"78630d10-b9ba-438d-907d-2be706fc51e7\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 12:24:34 crc kubenswrapper[5052]: I1203 12:24:34.427366 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstackclient" Dec 03 12:24:34 crc kubenswrapper[5052]: I1203 12:24:34.946311 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/openstackclient"] Dec 03 12:24:34 crc kubenswrapper[5052]: I1203 12:24:34.991497 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstackclient" event={"ID":"78630d10-b9ba-438d-907d-2be706fc51e7","Type":"ContainerStarted","Data":"a6625eb4e4104c12bf96e17f255230f202a8405b5ee033ed40e7ec40fa0efb88"} Dec 03 12:24:36 crc kubenswrapper[5052]: I1203 12:24:36.003160 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstackclient" event={"ID":"78630d10-b9ba-438d-907d-2be706fc51e7","Type":"ContainerStarted","Data":"7c24093b1dde24c14b80a87bb357568dbd53207a52044ef5dcbeb86466b30f11"} Dec 03 12:24:36 crc kubenswrapper[5052]: I1203 12:24:36.045964 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/openstackclient" podStartSLOduration=2.045938099 podStartE2EDuration="2.045938099s" podCreationTimestamp="2025-12-03 12:24:34 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:24:36.017546609 +0000 UTC m=+3866.605788373" watchObservedRunningTime="2025-12-03 12:24:36.045938099 +0000 UTC m=+3866.634179863" Dec 03 12:24:41 crc kubenswrapper[5052]: I1203 12:24:41.569324 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/swift-proxy-84c4f5d658-m9l78"] Dec 03 12:24:41 crc kubenswrapper[5052]: I1203 12:24:41.572907 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-proxy-84c4f5d658-m9l78" Dec 03 12:24:41 crc kubenswrapper[5052]: I1203 12:24:41.576156 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"swift-proxy-config-data" Dec 03 12:24:41 crc kubenswrapper[5052]: I1203 12:24:41.613392 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9f515435-a645-4d2b-9bd5-4bb9496f5405-combined-ca-bundle\") pod \"swift-proxy-84c4f5d658-m9l78\" (UID: \"9f515435-a645-4d2b-9bd5-4bb9496f5405\") " pod="openstack-kuttl-tests/swift-proxy-84c4f5d658-m9l78" Dec 03 12:24:41 crc kubenswrapper[5052]: I1203 12:24:41.613481 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/9f515435-a645-4d2b-9bd5-4bb9496f5405-etc-swift\") pod \"swift-proxy-84c4f5d658-m9l78\" (UID: \"9f515435-a645-4d2b-9bd5-4bb9496f5405\") " pod="openstack-kuttl-tests/swift-proxy-84c4f5d658-m9l78" Dec 03 12:24:41 crc kubenswrapper[5052]: I1203 12:24:41.613573 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/9f515435-a645-4d2b-9bd5-4bb9496f5405-run-httpd\") pod \"swift-proxy-84c4f5d658-m9l78\" (UID: \"9f515435-a645-4d2b-9bd5-4bb9496f5405\") " pod="openstack-kuttl-tests/swift-proxy-84c4f5d658-m9l78" Dec 03 12:24:41 crc kubenswrapper[5052]: I1203 12:24:41.613646 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/9f515435-a645-4d2b-9bd5-4bb9496f5405-log-httpd\") pod \"swift-proxy-84c4f5d658-m9l78\" (UID: \"9f515435-a645-4d2b-9bd5-4bb9496f5405\") " pod="openstack-kuttl-tests/swift-proxy-84c4f5d658-m9l78" Dec 03 12:24:41 crc kubenswrapper[5052]: I1203 12:24:41.613677 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mp59r\" (UniqueName: \"kubernetes.io/projected/9f515435-a645-4d2b-9bd5-4bb9496f5405-kube-api-access-mp59r\") pod \"swift-proxy-84c4f5d658-m9l78\" (UID: \"9f515435-a645-4d2b-9bd5-4bb9496f5405\") " pod="openstack-kuttl-tests/swift-proxy-84c4f5d658-m9l78" Dec 03 12:24:41 crc kubenswrapper[5052]: I1203 12:24:41.613719 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/9f515435-a645-4d2b-9bd5-4bb9496f5405-config-data\") pod \"swift-proxy-84c4f5d658-m9l78\" (UID: \"9f515435-a645-4d2b-9bd5-4bb9496f5405\") " pod="openstack-kuttl-tests/swift-proxy-84c4f5d658-m9l78" Dec 03 12:24:41 crc kubenswrapper[5052]: I1203 12:24:41.619810 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/swift-proxy-84c4f5d658-m9l78"] Dec 03 12:24:41 crc kubenswrapper[5052]: I1203 12:24:41.715110 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/9f515435-a645-4d2b-9bd5-4bb9496f5405-log-httpd\") pod \"swift-proxy-84c4f5d658-m9l78\" (UID: \"9f515435-a645-4d2b-9bd5-4bb9496f5405\") " pod="openstack-kuttl-tests/swift-proxy-84c4f5d658-m9l78" Dec 03 12:24:41 crc kubenswrapper[5052]: I1203 12:24:41.715182 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-mp59r\" (UniqueName: \"kubernetes.io/projected/9f515435-a645-4d2b-9bd5-4bb9496f5405-kube-api-access-mp59r\") pod \"swift-proxy-84c4f5d658-m9l78\" (UID: \"9f515435-a645-4d2b-9bd5-4bb9496f5405\") " pod="openstack-kuttl-tests/swift-proxy-84c4f5d658-m9l78" Dec 03 12:24:41 crc kubenswrapper[5052]: I1203 12:24:41.715233 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/9f515435-a645-4d2b-9bd5-4bb9496f5405-config-data\") pod \"swift-proxy-84c4f5d658-m9l78\" (UID: \"9f515435-a645-4d2b-9bd5-4bb9496f5405\") " pod="openstack-kuttl-tests/swift-proxy-84c4f5d658-m9l78" Dec 03 12:24:41 crc kubenswrapper[5052]: I1203 12:24:41.715267 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9f515435-a645-4d2b-9bd5-4bb9496f5405-combined-ca-bundle\") pod \"swift-proxy-84c4f5d658-m9l78\" (UID: \"9f515435-a645-4d2b-9bd5-4bb9496f5405\") " pod="openstack-kuttl-tests/swift-proxy-84c4f5d658-m9l78" Dec 03 12:24:41 crc kubenswrapper[5052]: I1203 12:24:41.715332 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/9f515435-a645-4d2b-9bd5-4bb9496f5405-etc-swift\") pod \"swift-proxy-84c4f5d658-m9l78\" (UID: \"9f515435-a645-4d2b-9bd5-4bb9496f5405\") " pod="openstack-kuttl-tests/swift-proxy-84c4f5d658-m9l78" Dec 03 12:24:41 crc kubenswrapper[5052]: I1203 12:24:41.715430 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/9f515435-a645-4d2b-9bd5-4bb9496f5405-run-httpd\") pod \"swift-proxy-84c4f5d658-m9l78\" (UID: \"9f515435-a645-4d2b-9bd5-4bb9496f5405\") " pod="openstack-kuttl-tests/swift-proxy-84c4f5d658-m9l78" Dec 03 12:24:41 crc kubenswrapper[5052]: I1203 12:24:41.716078 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/9f515435-a645-4d2b-9bd5-4bb9496f5405-run-httpd\") pod \"swift-proxy-84c4f5d658-m9l78\" (UID: \"9f515435-a645-4d2b-9bd5-4bb9496f5405\") " pod="openstack-kuttl-tests/swift-proxy-84c4f5d658-m9l78" Dec 03 12:24:41 crc kubenswrapper[5052]: I1203 12:24:41.716726 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/9f515435-a645-4d2b-9bd5-4bb9496f5405-log-httpd\") pod \"swift-proxy-84c4f5d658-m9l78\" (UID: \"9f515435-a645-4d2b-9bd5-4bb9496f5405\") " pod="openstack-kuttl-tests/swift-proxy-84c4f5d658-m9l78" Dec 03 12:24:41 crc kubenswrapper[5052]: I1203 12:24:41.722953 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9f515435-a645-4d2b-9bd5-4bb9496f5405-combined-ca-bundle\") pod \"swift-proxy-84c4f5d658-m9l78\" (UID: \"9f515435-a645-4d2b-9bd5-4bb9496f5405\") " pod="openstack-kuttl-tests/swift-proxy-84c4f5d658-m9l78" Dec 03 12:24:41 crc kubenswrapper[5052]: I1203 12:24:41.722986 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/9f515435-a645-4d2b-9bd5-4bb9496f5405-config-data\") pod \"swift-proxy-84c4f5d658-m9l78\" (UID: \"9f515435-a645-4d2b-9bd5-4bb9496f5405\") " pod="openstack-kuttl-tests/swift-proxy-84c4f5d658-m9l78" Dec 03 12:24:41 crc kubenswrapper[5052]: I1203 12:24:41.731171 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/9f515435-a645-4d2b-9bd5-4bb9496f5405-etc-swift\") pod \"swift-proxy-84c4f5d658-m9l78\" (UID: \"9f515435-a645-4d2b-9bd5-4bb9496f5405\") " pod="openstack-kuttl-tests/swift-proxy-84c4f5d658-m9l78" Dec 03 12:24:41 crc kubenswrapper[5052]: I1203 12:24:41.733591 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-mp59r\" (UniqueName: \"kubernetes.io/projected/9f515435-a645-4d2b-9bd5-4bb9496f5405-kube-api-access-mp59r\") pod \"swift-proxy-84c4f5d658-m9l78\" (UID: \"9f515435-a645-4d2b-9bd5-4bb9496f5405\") " pod="openstack-kuttl-tests/swift-proxy-84c4f5d658-m9l78" Dec 03 12:24:41 crc kubenswrapper[5052]: I1203 12:24:41.912701 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-proxy-84c4f5d658-m9l78" Dec 03 12:24:42 crc kubenswrapper[5052]: I1203 12:24:42.368038 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:24:42 crc kubenswrapper[5052]: I1203 12:24:42.368688 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="2660f107-7c9f-4d25-be86-3ba68a71be57" containerName="ceilometer-central-agent" containerID="cri-o://ed818d88a39bfa952249a233cfe4ea201ee58d711ab8360da187109336679ef1" gracePeriod=30 Dec 03 12:24:42 crc kubenswrapper[5052]: I1203 12:24:42.368755 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="2660f107-7c9f-4d25-be86-3ba68a71be57" containerName="proxy-httpd" containerID="cri-o://72fd85d4d1caba1fa3643029488ed2257e344e275be11a73bcca540b29eb5ce7" gracePeriod=30 Dec 03 12:24:42 crc kubenswrapper[5052]: I1203 12:24:42.368773 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="2660f107-7c9f-4d25-be86-3ba68a71be57" containerName="sg-core" containerID="cri-o://ad2ecf738e50628058ca414a90615ccf59d2a01114dd7d51ff8e6f11e9458294" gracePeriod=30 Dec 03 12:24:42 crc kubenswrapper[5052]: I1203 12:24:42.368868 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="2660f107-7c9f-4d25-be86-3ba68a71be57" containerName="ceilometer-notification-agent" containerID="cri-o://d6d40488bb533fdd7e199ada04a44691a82fa82a2f30cd2732d83e40c8719a42" gracePeriod=30 Dec 03 12:24:42 crc kubenswrapper[5052]: I1203 12:24:42.427594 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/swift-proxy-84c4f5d658-m9l78"] Dec 03 12:24:42 crc kubenswrapper[5052]: I1203 12:24:42.673567 5052 scope.go:117] "RemoveContainer" containerID="09b4a0072741f60499d24f40cc4ff3406d5d22671ced431363ab865172786300" Dec 03 12:24:42 crc kubenswrapper[5052]: E1203 12:24:42.674157 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 12:24:43 crc kubenswrapper[5052]: I1203 12:24:43.079652 5052 generic.go:334] "Generic (PLEG): container finished" podID="2660f107-7c9f-4d25-be86-3ba68a71be57" containerID="72fd85d4d1caba1fa3643029488ed2257e344e275be11a73bcca540b29eb5ce7" exitCode=0 Dec 03 12:24:43 crc kubenswrapper[5052]: I1203 12:24:43.079696 5052 generic.go:334] "Generic (PLEG): container finished" podID="2660f107-7c9f-4d25-be86-3ba68a71be57" containerID="ad2ecf738e50628058ca414a90615ccf59d2a01114dd7d51ff8e6f11e9458294" exitCode=2 Dec 03 12:24:43 crc kubenswrapper[5052]: I1203 12:24:43.079711 5052 generic.go:334] "Generic (PLEG): container finished" podID="2660f107-7c9f-4d25-be86-3ba68a71be57" containerID="ed818d88a39bfa952249a233cfe4ea201ee58d711ab8360da187109336679ef1" exitCode=0 Dec 03 12:24:43 crc kubenswrapper[5052]: I1203 12:24:43.079725 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"2660f107-7c9f-4d25-be86-3ba68a71be57","Type":"ContainerDied","Data":"72fd85d4d1caba1fa3643029488ed2257e344e275be11a73bcca540b29eb5ce7"} Dec 03 12:24:43 crc kubenswrapper[5052]: I1203 12:24:43.079776 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"2660f107-7c9f-4d25-be86-3ba68a71be57","Type":"ContainerDied","Data":"ad2ecf738e50628058ca414a90615ccf59d2a01114dd7d51ff8e6f11e9458294"} Dec 03 12:24:43 crc kubenswrapper[5052]: I1203 12:24:43.079793 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"2660f107-7c9f-4d25-be86-3ba68a71be57","Type":"ContainerDied","Data":"ed818d88a39bfa952249a233cfe4ea201ee58d711ab8360da187109336679ef1"} Dec 03 12:24:43 crc kubenswrapper[5052]: I1203 12:24:43.082128 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-proxy-84c4f5d658-m9l78" event={"ID":"9f515435-a645-4d2b-9bd5-4bb9496f5405","Type":"ContainerStarted","Data":"41ad3ff3a7101ea8b3778d199fc0aa3d823057eb0199c3f55a79043a27526808"} Dec 03 12:24:43 crc kubenswrapper[5052]: I1203 12:24:43.082194 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-proxy-84c4f5d658-m9l78" event={"ID":"9f515435-a645-4d2b-9bd5-4bb9496f5405","Type":"ContainerStarted","Data":"7401b98806090e0288c0ece96883b73c2a54d14bd85bbf4ba5729b51cadf8ea2"} Dec 03 12:24:43 crc kubenswrapper[5052]: I1203 12:24:43.082210 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-proxy-84c4f5d658-m9l78" event={"ID":"9f515435-a645-4d2b-9bd5-4bb9496f5405","Type":"ContainerStarted","Data":"09c43885facd81df8e08a1ad42cb243e3a23953e3a9571a5e2670caeea4cbf8f"} Dec 03 12:24:43 crc kubenswrapper[5052]: I1203 12:24:43.082303 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/swift-proxy-84c4f5d658-m9l78" Dec 03 12:24:43 crc kubenswrapper[5052]: I1203 12:24:43.106264 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/swift-proxy-84c4f5d658-m9l78" podStartSLOduration=2.106244815 podStartE2EDuration="2.106244815s" podCreationTimestamp="2025-12-03 12:24:41 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:24:43.103675122 +0000 UTC m=+3873.691916886" watchObservedRunningTime="2025-12-03 12:24:43.106244815 +0000 UTC m=+3873.694486579" Dec 03 12:24:44 crc kubenswrapper[5052]: I1203 12:24:44.091376 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/swift-proxy-84c4f5d658-m9l78" Dec 03 12:24:44 crc kubenswrapper[5052]: I1203 12:24:44.749945 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/ceilometer-0" podUID="2660f107-7c9f-4d25-be86-3ba68a71be57" containerName="proxy-httpd" probeResult="failure" output="Get \"http://10.217.1.242:3000/\": dial tcp 10.217.1.242:3000: connect: connection refused" Dec 03 12:24:46 crc kubenswrapper[5052]: I1203 12:24:46.483688 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/neutron-f699c657d-qh2qk" Dec 03 12:24:48 crc kubenswrapper[5052]: I1203 12:24:48.063054 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-api-db-create-846mj"] Dec 03 12:24:48 crc kubenswrapper[5052]: I1203 12:24:48.065592 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-db-create-846mj" Dec 03 12:24:48 crc kubenswrapper[5052]: I1203 12:24:48.084093 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-db-create-846mj"] Dec 03 12:24:48 crc kubenswrapper[5052]: I1203 12:24:48.137098 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:24:48 crc kubenswrapper[5052]: I1203 12:24:48.140857 5052 generic.go:334] "Generic (PLEG): container finished" podID="2660f107-7c9f-4d25-be86-3ba68a71be57" containerID="d6d40488bb533fdd7e199ada04a44691a82fa82a2f30cd2732d83e40c8719a42" exitCode=0 Dec 03 12:24:48 crc kubenswrapper[5052]: I1203 12:24:48.140898 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"2660f107-7c9f-4d25-be86-3ba68a71be57","Type":"ContainerDied","Data":"d6d40488bb533fdd7e199ada04a44691a82fa82a2f30cd2732d83e40c8719a42"} Dec 03 12:24:48 crc kubenswrapper[5052]: I1203 12:24:48.140924 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"2660f107-7c9f-4d25-be86-3ba68a71be57","Type":"ContainerDied","Data":"74ee332ddf2ec00702800023a9ca2733054e8ce5d7e675f3432237abdef4b8cd"} Dec 03 12:24:48 crc kubenswrapper[5052]: I1203 12:24:48.140940 5052 scope.go:117] "RemoveContainer" containerID="72fd85d4d1caba1fa3643029488ed2257e344e275be11a73bcca540b29eb5ce7" Dec 03 12:24:48 crc kubenswrapper[5052]: I1203 12:24:48.159974 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/0160de5f-930d-4766-9302-28c89c2fcf14-operator-scripts\") pod \"nova-api-db-create-846mj\" (UID: \"0160de5f-930d-4766-9302-28c89c2fcf14\") " pod="openstack-kuttl-tests/nova-api-db-create-846mj" Dec 03 12:24:48 crc kubenswrapper[5052]: I1203 12:24:48.160165 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6t94d\" (UniqueName: \"kubernetes.io/projected/0160de5f-930d-4766-9302-28c89c2fcf14-kube-api-access-6t94d\") pod \"nova-api-db-create-846mj\" (UID: \"0160de5f-930d-4766-9302-28c89c2fcf14\") " pod="openstack-kuttl-tests/nova-api-db-create-846mj" Dec 03 12:24:48 crc kubenswrapper[5052]: I1203 12:24:48.179242 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell0-db-create-5fgfv"] Dec 03 12:24:48 crc kubenswrapper[5052]: E1203 12:24:48.179759 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="2660f107-7c9f-4d25-be86-3ba68a71be57" containerName="ceilometer-notification-agent" Dec 03 12:24:48 crc kubenswrapper[5052]: I1203 12:24:48.179779 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="2660f107-7c9f-4d25-be86-3ba68a71be57" containerName="ceilometer-notification-agent" Dec 03 12:24:48 crc kubenswrapper[5052]: E1203 12:24:48.179801 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="2660f107-7c9f-4d25-be86-3ba68a71be57" containerName="proxy-httpd" Dec 03 12:24:48 crc kubenswrapper[5052]: I1203 12:24:48.179809 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="2660f107-7c9f-4d25-be86-3ba68a71be57" containerName="proxy-httpd" Dec 03 12:24:48 crc kubenswrapper[5052]: E1203 12:24:48.179841 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="2660f107-7c9f-4d25-be86-3ba68a71be57" containerName="ceilometer-central-agent" Dec 03 12:24:48 crc kubenswrapper[5052]: I1203 12:24:48.179848 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="2660f107-7c9f-4d25-be86-3ba68a71be57" containerName="ceilometer-central-agent" Dec 03 12:24:48 crc kubenswrapper[5052]: E1203 12:24:48.179878 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="2660f107-7c9f-4d25-be86-3ba68a71be57" containerName="sg-core" Dec 03 12:24:48 crc kubenswrapper[5052]: I1203 12:24:48.179885 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="2660f107-7c9f-4d25-be86-3ba68a71be57" containerName="sg-core" Dec 03 12:24:48 crc kubenswrapper[5052]: I1203 12:24:48.180110 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="2660f107-7c9f-4d25-be86-3ba68a71be57" containerName="ceilometer-notification-agent" Dec 03 12:24:48 crc kubenswrapper[5052]: I1203 12:24:48.180132 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="2660f107-7c9f-4d25-be86-3ba68a71be57" containerName="proxy-httpd" Dec 03 12:24:48 crc kubenswrapper[5052]: I1203 12:24:48.180153 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="2660f107-7c9f-4d25-be86-3ba68a71be57" containerName="sg-core" Dec 03 12:24:48 crc kubenswrapper[5052]: I1203 12:24:48.180170 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="2660f107-7c9f-4d25-be86-3ba68a71be57" containerName="ceilometer-central-agent" Dec 03 12:24:48 crc kubenswrapper[5052]: I1203 12:24:48.180991 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-db-create-5fgfv" Dec 03 12:24:48 crc kubenswrapper[5052]: I1203 12:24:48.204336 5052 scope.go:117] "RemoveContainer" containerID="ad2ecf738e50628058ca414a90615ccf59d2a01114dd7d51ff8e6f11e9458294" Dec 03 12:24:48 crc kubenswrapper[5052]: I1203 12:24:48.212977 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell0-db-create-5fgfv"] Dec 03 12:24:48 crc kubenswrapper[5052]: I1203 12:24:48.261908 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/2660f107-7c9f-4d25-be86-3ba68a71be57-sg-core-conf-yaml\") pod \"2660f107-7c9f-4d25-be86-3ba68a71be57\" (UID: \"2660f107-7c9f-4d25-be86-3ba68a71be57\") " Dec 03 12:24:48 crc kubenswrapper[5052]: I1203 12:24:48.262040 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/2660f107-7c9f-4d25-be86-3ba68a71be57-log-httpd\") pod \"2660f107-7c9f-4d25-be86-3ba68a71be57\" (UID: \"2660f107-7c9f-4d25-be86-3ba68a71be57\") " Dec 03 12:24:48 crc kubenswrapper[5052]: I1203 12:24:48.262067 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/2660f107-7c9f-4d25-be86-3ba68a71be57-run-httpd\") pod \"2660f107-7c9f-4d25-be86-3ba68a71be57\" (UID: \"2660f107-7c9f-4d25-be86-3ba68a71be57\") " Dec 03 12:24:48 crc kubenswrapper[5052]: I1203 12:24:48.262172 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/2660f107-7c9f-4d25-be86-3ba68a71be57-scripts\") pod \"2660f107-7c9f-4d25-be86-3ba68a71be57\" (UID: \"2660f107-7c9f-4d25-be86-3ba68a71be57\") " Dec 03 12:24:48 crc kubenswrapper[5052]: I1203 12:24:48.264368 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/2660f107-7c9f-4d25-be86-3ba68a71be57-config-data\") pod \"2660f107-7c9f-4d25-be86-3ba68a71be57\" (UID: \"2660f107-7c9f-4d25-be86-3ba68a71be57\") " Dec 03 12:24:48 crc kubenswrapper[5052]: I1203 12:24:48.264411 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2660f107-7c9f-4d25-be86-3ba68a71be57-combined-ca-bundle\") pod \"2660f107-7c9f-4d25-be86-3ba68a71be57\" (UID: \"2660f107-7c9f-4d25-be86-3ba68a71be57\") " Dec 03 12:24:48 crc kubenswrapper[5052]: I1203 12:24:48.264442 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c2s4p\" (UniqueName: \"kubernetes.io/projected/2660f107-7c9f-4d25-be86-3ba68a71be57-kube-api-access-c2s4p\") pod \"2660f107-7c9f-4d25-be86-3ba68a71be57\" (UID: \"2660f107-7c9f-4d25-be86-3ba68a71be57\") " Dec 03 12:24:48 crc kubenswrapper[5052]: I1203 12:24:48.265221 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/0160de5f-930d-4766-9302-28c89c2fcf14-operator-scripts\") pod \"nova-api-db-create-846mj\" (UID: \"0160de5f-930d-4766-9302-28c89c2fcf14\") " pod="openstack-kuttl-tests/nova-api-db-create-846mj" Dec 03 12:24:48 crc kubenswrapper[5052]: I1203 12:24:48.265308 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bbxsd\" (UniqueName: \"kubernetes.io/projected/027c0ef8-c154-4de5-961e-d822ab32a761-kube-api-access-bbxsd\") pod \"nova-cell0-db-create-5fgfv\" (UID: \"027c0ef8-c154-4de5-961e-d822ab32a761\") " pod="openstack-kuttl-tests/nova-cell0-db-create-5fgfv" Dec 03 12:24:48 crc kubenswrapper[5052]: I1203 12:24:48.265450 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/027c0ef8-c154-4de5-961e-d822ab32a761-operator-scripts\") pod \"nova-cell0-db-create-5fgfv\" (UID: \"027c0ef8-c154-4de5-961e-d822ab32a761\") " pod="openstack-kuttl-tests/nova-cell0-db-create-5fgfv" Dec 03 12:24:48 crc kubenswrapper[5052]: I1203 12:24:48.265501 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-6t94d\" (UniqueName: \"kubernetes.io/projected/0160de5f-930d-4766-9302-28c89c2fcf14-kube-api-access-6t94d\") pod \"nova-api-db-create-846mj\" (UID: \"0160de5f-930d-4766-9302-28c89c2fcf14\") " pod="openstack-kuttl-tests/nova-api-db-create-846mj" Dec 03 12:24:48 crc kubenswrapper[5052]: I1203 12:24:48.266314 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2660f107-7c9f-4d25-be86-3ba68a71be57-log-httpd" (OuterVolumeSpecName: "log-httpd") pod "2660f107-7c9f-4d25-be86-3ba68a71be57" (UID: "2660f107-7c9f-4d25-be86-3ba68a71be57"). InnerVolumeSpecName "log-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:24:48 crc kubenswrapper[5052]: I1203 12:24:48.267658 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2660f107-7c9f-4d25-be86-3ba68a71be57-run-httpd" (OuterVolumeSpecName: "run-httpd") pod "2660f107-7c9f-4d25-be86-3ba68a71be57" (UID: "2660f107-7c9f-4d25-be86-3ba68a71be57"). InnerVolumeSpecName "run-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:24:48 crc kubenswrapper[5052]: I1203 12:24:48.277854 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2660f107-7c9f-4d25-be86-3ba68a71be57-scripts" (OuterVolumeSpecName: "scripts") pod "2660f107-7c9f-4d25-be86-3ba68a71be57" (UID: "2660f107-7c9f-4d25-be86-3ba68a71be57"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:24:48 crc kubenswrapper[5052]: I1203 12:24:48.278576 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/0160de5f-930d-4766-9302-28c89c2fcf14-operator-scripts\") pod \"nova-api-db-create-846mj\" (UID: \"0160de5f-930d-4766-9302-28c89c2fcf14\") " pod="openstack-kuttl-tests/nova-api-db-create-846mj" Dec 03 12:24:48 crc kubenswrapper[5052]: I1203 12:24:48.290351 5052 scope.go:117] "RemoveContainer" containerID="d6d40488bb533fdd7e199ada04a44691a82fa82a2f30cd2732d83e40c8719a42" Dec 03 12:24:48 crc kubenswrapper[5052]: I1203 12:24:48.293737 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2660f107-7c9f-4d25-be86-3ba68a71be57-kube-api-access-c2s4p" (OuterVolumeSpecName: "kube-api-access-c2s4p") pod "2660f107-7c9f-4d25-be86-3ba68a71be57" (UID: "2660f107-7c9f-4d25-be86-3ba68a71be57"). InnerVolumeSpecName "kube-api-access-c2s4p". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:24:48 crc kubenswrapper[5052]: I1203 12:24:48.298823 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-6t94d\" (UniqueName: \"kubernetes.io/projected/0160de5f-930d-4766-9302-28c89c2fcf14-kube-api-access-6t94d\") pod \"nova-api-db-create-846mj\" (UID: \"0160de5f-930d-4766-9302-28c89c2fcf14\") " pod="openstack-kuttl-tests/nova-api-db-create-846mj" Dec 03 12:24:48 crc kubenswrapper[5052]: I1203 12:24:48.336305 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-api-a918-account-create-update-s6fjd"] Dec 03 12:24:48 crc kubenswrapper[5052]: I1203 12:24:48.343578 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-a918-account-create-update-s6fjd" Dec 03 12:24:48 crc kubenswrapper[5052]: I1203 12:24:48.352672 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-api-db-secret" Dec 03 12:24:48 crc kubenswrapper[5052]: I1203 12:24:48.353483 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2660f107-7c9f-4d25-be86-3ba68a71be57-sg-core-conf-yaml" (OuterVolumeSpecName: "sg-core-conf-yaml") pod "2660f107-7c9f-4d25-be86-3ba68a71be57" (UID: "2660f107-7c9f-4d25-be86-3ba68a71be57"). InnerVolumeSpecName "sg-core-conf-yaml". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:24:48 crc kubenswrapper[5052]: I1203 12:24:48.365674 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-a918-account-create-update-s6fjd"] Dec 03 12:24:48 crc kubenswrapper[5052]: I1203 12:24:48.367014 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-bbxsd\" (UniqueName: \"kubernetes.io/projected/027c0ef8-c154-4de5-961e-d822ab32a761-kube-api-access-bbxsd\") pod \"nova-cell0-db-create-5fgfv\" (UID: \"027c0ef8-c154-4de5-961e-d822ab32a761\") " pod="openstack-kuttl-tests/nova-cell0-db-create-5fgfv" Dec 03 12:24:48 crc kubenswrapper[5052]: I1203 12:24:48.367105 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/027c0ef8-c154-4de5-961e-d822ab32a761-operator-scripts\") pod \"nova-cell0-db-create-5fgfv\" (UID: \"027c0ef8-c154-4de5-961e-d822ab32a761\") " pod="openstack-kuttl-tests/nova-cell0-db-create-5fgfv" Dec 03 12:24:48 crc kubenswrapper[5052]: I1203 12:24:48.369319 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/027c0ef8-c154-4de5-961e-d822ab32a761-operator-scripts\") pod \"nova-cell0-db-create-5fgfv\" (UID: \"027c0ef8-c154-4de5-961e-d822ab32a761\") " pod="openstack-kuttl-tests/nova-cell0-db-create-5fgfv" Dec 03 12:24:48 crc kubenswrapper[5052]: I1203 12:24:48.375855 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k954n\" (UniqueName: \"kubernetes.io/projected/a1ac8f66-94c3-4ac5-b039-fe29314ed99a-kube-api-access-k954n\") pod \"nova-api-a918-account-create-update-s6fjd\" (UID: \"a1ac8f66-94c3-4ac5-b039-fe29314ed99a\") " pod="openstack-kuttl-tests/nova-api-a918-account-create-update-s6fjd" Dec 03 12:24:48 crc kubenswrapper[5052]: I1203 12:24:48.376042 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/a1ac8f66-94c3-4ac5-b039-fe29314ed99a-operator-scripts\") pod \"nova-api-a918-account-create-update-s6fjd\" (UID: \"a1ac8f66-94c3-4ac5-b039-fe29314ed99a\") " pod="openstack-kuttl-tests/nova-api-a918-account-create-update-s6fjd" Dec 03 12:24:48 crc kubenswrapper[5052]: I1203 12:24:48.376195 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-c2s4p\" (UniqueName: \"kubernetes.io/projected/2660f107-7c9f-4d25-be86-3ba68a71be57-kube-api-access-c2s4p\") on node \"crc\" DevicePath \"\"" Dec 03 12:24:48 crc kubenswrapper[5052]: I1203 12:24:48.376484 5052 reconciler_common.go:293] "Volume detached for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/2660f107-7c9f-4d25-be86-3ba68a71be57-sg-core-conf-yaml\") on node \"crc\" DevicePath \"\"" Dec 03 12:24:48 crc kubenswrapper[5052]: I1203 12:24:48.376500 5052 reconciler_common.go:293] "Volume detached for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/2660f107-7c9f-4d25-be86-3ba68a71be57-log-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:24:48 crc kubenswrapper[5052]: I1203 12:24:48.376527 5052 reconciler_common.go:293] "Volume detached for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/2660f107-7c9f-4d25-be86-3ba68a71be57-run-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:24:48 crc kubenswrapper[5052]: I1203 12:24:48.376536 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/2660f107-7c9f-4d25-be86-3ba68a71be57-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:24:48 crc kubenswrapper[5052]: I1203 12:24:48.384088 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell1-db-create-9vd8k"] Dec 03 12:24:48 crc kubenswrapper[5052]: I1203 12:24:48.386325 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-db-create-9vd8k" Dec 03 12:24:48 crc kubenswrapper[5052]: I1203 12:24:48.397922 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-bbxsd\" (UniqueName: \"kubernetes.io/projected/027c0ef8-c154-4de5-961e-d822ab32a761-kube-api-access-bbxsd\") pod \"nova-cell0-db-create-5fgfv\" (UID: \"027c0ef8-c154-4de5-961e-d822ab32a761\") " pod="openstack-kuttl-tests/nova-cell0-db-create-5fgfv" Dec 03 12:24:48 crc kubenswrapper[5052]: I1203 12:24:48.407333 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-db-create-9vd8k"] Dec 03 12:24:48 crc kubenswrapper[5052]: I1203 12:24:48.409774 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2660f107-7c9f-4d25-be86-3ba68a71be57-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "2660f107-7c9f-4d25-be86-3ba68a71be57" (UID: "2660f107-7c9f-4d25-be86-3ba68a71be57"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:24:48 crc kubenswrapper[5052]: I1203 12:24:48.419784 5052 scope.go:117] "RemoveContainer" containerID="ed818d88a39bfa952249a233cfe4ea201ee58d711ab8360da187109336679ef1" Dec 03 12:24:48 crc kubenswrapper[5052]: I1203 12:24:48.470302 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-db-create-846mj" Dec 03 12:24:48 crc kubenswrapper[5052]: I1203 12:24:48.475191 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell0-46b8-account-create-update-g9c2m"] Dec 03 12:24:48 crc kubenswrapper[5052]: I1203 12:24:48.476779 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-46b8-account-create-update-g9c2m" Dec 03 12:24:48 crc kubenswrapper[5052]: I1203 12:24:48.481715 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-k954n\" (UniqueName: \"kubernetes.io/projected/a1ac8f66-94c3-4ac5-b039-fe29314ed99a-kube-api-access-k954n\") pod \"nova-api-a918-account-create-update-s6fjd\" (UID: \"a1ac8f66-94c3-4ac5-b039-fe29314ed99a\") " pod="openstack-kuttl-tests/nova-api-a918-account-create-update-s6fjd" Dec 03 12:24:48 crc kubenswrapper[5052]: I1203 12:24:48.481823 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/a1ac8f66-94c3-4ac5-b039-fe29314ed99a-operator-scripts\") pod \"nova-api-a918-account-create-update-s6fjd\" (UID: \"a1ac8f66-94c3-4ac5-b039-fe29314ed99a\") " pod="openstack-kuttl-tests/nova-api-a918-account-create-update-s6fjd" Dec 03 12:24:48 crc kubenswrapper[5052]: I1203 12:24:48.482006 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/47087a5b-7fd2-498e-b9d0-0bc3c6939637-operator-scripts\") pod \"nova-cell1-db-create-9vd8k\" (UID: \"47087a5b-7fd2-498e-b9d0-0bc3c6939637\") " pod="openstack-kuttl-tests/nova-cell1-db-create-9vd8k" Dec 03 12:24:48 crc kubenswrapper[5052]: I1203 12:24:48.482107 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-72928\" (UniqueName: \"kubernetes.io/projected/47087a5b-7fd2-498e-b9d0-0bc3c6939637-kube-api-access-72928\") pod \"nova-cell1-db-create-9vd8k\" (UID: \"47087a5b-7fd2-498e-b9d0-0bc3c6939637\") " pod="openstack-kuttl-tests/nova-cell1-db-create-9vd8k" Dec 03 12:24:48 crc kubenswrapper[5052]: I1203 12:24:48.482161 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2660f107-7c9f-4d25-be86-3ba68a71be57-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:24:48 crc kubenswrapper[5052]: I1203 12:24:48.482855 5052 scope.go:117] "RemoveContainer" containerID="72fd85d4d1caba1fa3643029488ed2257e344e275be11a73bcca540b29eb5ce7" Dec 03 12:24:48 crc kubenswrapper[5052]: I1203 12:24:48.483198 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/a1ac8f66-94c3-4ac5-b039-fe29314ed99a-operator-scripts\") pod \"nova-api-a918-account-create-update-s6fjd\" (UID: \"a1ac8f66-94c3-4ac5-b039-fe29314ed99a\") " pod="openstack-kuttl-tests/nova-api-a918-account-create-update-s6fjd" Dec 03 12:24:48 crc kubenswrapper[5052]: E1203 12:24:48.483390 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"72fd85d4d1caba1fa3643029488ed2257e344e275be11a73bcca540b29eb5ce7\": container with ID starting with 72fd85d4d1caba1fa3643029488ed2257e344e275be11a73bcca540b29eb5ce7 not found: ID does not exist" containerID="72fd85d4d1caba1fa3643029488ed2257e344e275be11a73bcca540b29eb5ce7" Dec 03 12:24:48 crc kubenswrapper[5052]: I1203 12:24:48.483427 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"72fd85d4d1caba1fa3643029488ed2257e344e275be11a73bcca540b29eb5ce7"} err="failed to get container status \"72fd85d4d1caba1fa3643029488ed2257e344e275be11a73bcca540b29eb5ce7\": rpc error: code = NotFound desc = could not find container \"72fd85d4d1caba1fa3643029488ed2257e344e275be11a73bcca540b29eb5ce7\": container with ID starting with 72fd85d4d1caba1fa3643029488ed2257e344e275be11a73bcca540b29eb5ce7 not found: ID does not exist" Dec 03 12:24:48 crc kubenswrapper[5052]: I1203 12:24:48.483453 5052 scope.go:117] "RemoveContainer" containerID="ad2ecf738e50628058ca414a90615ccf59d2a01114dd7d51ff8e6f11e9458294" Dec 03 12:24:48 crc kubenswrapper[5052]: E1203 12:24:48.484431 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ad2ecf738e50628058ca414a90615ccf59d2a01114dd7d51ff8e6f11e9458294\": container with ID starting with ad2ecf738e50628058ca414a90615ccf59d2a01114dd7d51ff8e6f11e9458294 not found: ID does not exist" containerID="ad2ecf738e50628058ca414a90615ccf59d2a01114dd7d51ff8e6f11e9458294" Dec 03 12:24:48 crc kubenswrapper[5052]: I1203 12:24:48.484464 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ad2ecf738e50628058ca414a90615ccf59d2a01114dd7d51ff8e6f11e9458294"} err="failed to get container status \"ad2ecf738e50628058ca414a90615ccf59d2a01114dd7d51ff8e6f11e9458294\": rpc error: code = NotFound desc = could not find container \"ad2ecf738e50628058ca414a90615ccf59d2a01114dd7d51ff8e6f11e9458294\": container with ID starting with ad2ecf738e50628058ca414a90615ccf59d2a01114dd7d51ff8e6f11e9458294 not found: ID does not exist" Dec 03 12:24:48 crc kubenswrapper[5052]: I1203 12:24:48.484482 5052 scope.go:117] "RemoveContainer" containerID="d6d40488bb533fdd7e199ada04a44691a82fa82a2f30cd2732d83e40c8719a42" Dec 03 12:24:48 crc kubenswrapper[5052]: E1203 12:24:48.484754 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d6d40488bb533fdd7e199ada04a44691a82fa82a2f30cd2732d83e40c8719a42\": container with ID starting with d6d40488bb533fdd7e199ada04a44691a82fa82a2f30cd2732d83e40c8719a42 not found: ID does not exist" containerID="d6d40488bb533fdd7e199ada04a44691a82fa82a2f30cd2732d83e40c8719a42" Dec 03 12:24:48 crc kubenswrapper[5052]: I1203 12:24:48.484774 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d6d40488bb533fdd7e199ada04a44691a82fa82a2f30cd2732d83e40c8719a42"} err="failed to get container status \"d6d40488bb533fdd7e199ada04a44691a82fa82a2f30cd2732d83e40c8719a42\": rpc error: code = NotFound desc = could not find container \"d6d40488bb533fdd7e199ada04a44691a82fa82a2f30cd2732d83e40c8719a42\": container with ID starting with d6d40488bb533fdd7e199ada04a44691a82fa82a2f30cd2732d83e40c8719a42 not found: ID does not exist" Dec 03 12:24:48 crc kubenswrapper[5052]: I1203 12:24:48.484792 5052 scope.go:117] "RemoveContainer" containerID="ed818d88a39bfa952249a233cfe4ea201ee58d711ab8360da187109336679ef1" Dec 03 12:24:48 crc kubenswrapper[5052]: E1203 12:24:48.485014 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ed818d88a39bfa952249a233cfe4ea201ee58d711ab8360da187109336679ef1\": container with ID starting with ed818d88a39bfa952249a233cfe4ea201ee58d711ab8360da187109336679ef1 not found: ID does not exist" containerID="ed818d88a39bfa952249a233cfe4ea201ee58d711ab8360da187109336679ef1" Dec 03 12:24:48 crc kubenswrapper[5052]: I1203 12:24:48.485034 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ed818d88a39bfa952249a233cfe4ea201ee58d711ab8360da187109336679ef1"} err="failed to get container status \"ed818d88a39bfa952249a233cfe4ea201ee58d711ab8360da187109336679ef1\": rpc error: code = NotFound desc = could not find container \"ed818d88a39bfa952249a233cfe4ea201ee58d711ab8360da187109336679ef1\": container with ID starting with ed818d88a39bfa952249a233cfe4ea201ee58d711ab8360da187109336679ef1 not found: ID does not exist" Dec 03 12:24:48 crc kubenswrapper[5052]: I1203 12:24:48.489177 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell0-db-secret" Dec 03 12:24:48 crc kubenswrapper[5052]: I1203 12:24:48.491552 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell0-46b8-account-create-update-g9c2m"] Dec 03 12:24:48 crc kubenswrapper[5052]: I1203 12:24:48.510476 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2660f107-7c9f-4d25-be86-3ba68a71be57-config-data" (OuterVolumeSpecName: "config-data") pod "2660f107-7c9f-4d25-be86-3ba68a71be57" (UID: "2660f107-7c9f-4d25-be86-3ba68a71be57"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:24:48 crc kubenswrapper[5052]: I1203 12:24:48.513796 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-k954n\" (UniqueName: \"kubernetes.io/projected/a1ac8f66-94c3-4ac5-b039-fe29314ed99a-kube-api-access-k954n\") pod \"nova-api-a918-account-create-update-s6fjd\" (UID: \"a1ac8f66-94c3-4ac5-b039-fe29314ed99a\") " pod="openstack-kuttl-tests/nova-api-a918-account-create-update-s6fjd" Dec 03 12:24:48 crc kubenswrapper[5052]: I1203 12:24:48.549202 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-db-create-5fgfv" Dec 03 12:24:48 crc kubenswrapper[5052]: I1203 12:24:48.583681 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-72928\" (UniqueName: \"kubernetes.io/projected/47087a5b-7fd2-498e-b9d0-0bc3c6939637-kube-api-access-72928\") pod \"nova-cell1-db-create-9vd8k\" (UID: \"47087a5b-7fd2-498e-b9d0-0bc3c6939637\") " pod="openstack-kuttl-tests/nova-cell1-db-create-9vd8k" Dec 03 12:24:48 crc kubenswrapper[5052]: I1203 12:24:48.583874 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/47087a5b-7fd2-498e-b9d0-0bc3c6939637-operator-scripts\") pod \"nova-cell1-db-create-9vd8k\" (UID: \"47087a5b-7fd2-498e-b9d0-0bc3c6939637\") " pod="openstack-kuttl-tests/nova-cell1-db-create-9vd8k" Dec 03 12:24:48 crc kubenswrapper[5052]: I1203 12:24:48.583924 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/d04ffd07-eb13-41b2-a0ed-058afc5c8d73-operator-scripts\") pod \"nova-cell0-46b8-account-create-update-g9c2m\" (UID: \"d04ffd07-eb13-41b2-a0ed-058afc5c8d73\") " pod="openstack-kuttl-tests/nova-cell0-46b8-account-create-update-g9c2m" Dec 03 12:24:48 crc kubenswrapper[5052]: I1203 12:24:48.583963 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wg9qt\" (UniqueName: \"kubernetes.io/projected/d04ffd07-eb13-41b2-a0ed-058afc5c8d73-kube-api-access-wg9qt\") pod \"nova-cell0-46b8-account-create-update-g9c2m\" (UID: \"d04ffd07-eb13-41b2-a0ed-058afc5c8d73\") " pod="openstack-kuttl-tests/nova-cell0-46b8-account-create-update-g9c2m" Dec 03 12:24:48 crc kubenswrapper[5052]: I1203 12:24:48.584040 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/2660f107-7c9f-4d25-be86-3ba68a71be57-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:24:48 crc kubenswrapper[5052]: I1203 12:24:48.585059 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/47087a5b-7fd2-498e-b9d0-0bc3c6939637-operator-scripts\") pod \"nova-cell1-db-create-9vd8k\" (UID: \"47087a5b-7fd2-498e-b9d0-0bc3c6939637\") " pod="openstack-kuttl-tests/nova-cell1-db-create-9vd8k" Dec 03 12:24:48 crc kubenswrapper[5052]: I1203 12:24:48.604268 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-72928\" (UniqueName: \"kubernetes.io/projected/47087a5b-7fd2-498e-b9d0-0bc3c6939637-kube-api-access-72928\") pod \"nova-cell1-db-create-9vd8k\" (UID: \"47087a5b-7fd2-498e-b9d0-0bc3c6939637\") " pod="openstack-kuttl-tests/nova-cell1-db-create-9vd8k" Dec 03 12:24:48 crc kubenswrapper[5052]: I1203 12:24:48.662905 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell1-15a6-account-create-update-z2tdn"] Dec 03 12:24:48 crc kubenswrapper[5052]: I1203 12:24:48.664468 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-15a6-account-create-update-z2tdn" Dec 03 12:24:48 crc kubenswrapper[5052]: I1203 12:24:48.667165 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell1-db-secret" Dec 03 12:24:48 crc kubenswrapper[5052]: I1203 12:24:48.679714 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-15a6-account-create-update-z2tdn"] Dec 03 12:24:48 crc kubenswrapper[5052]: I1203 12:24:48.697617 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/d04ffd07-eb13-41b2-a0ed-058afc5c8d73-operator-scripts\") pod \"nova-cell0-46b8-account-create-update-g9c2m\" (UID: \"d04ffd07-eb13-41b2-a0ed-058afc5c8d73\") " pod="openstack-kuttl-tests/nova-cell0-46b8-account-create-update-g9c2m" Dec 03 12:24:48 crc kubenswrapper[5052]: I1203 12:24:48.697675 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-wg9qt\" (UniqueName: \"kubernetes.io/projected/d04ffd07-eb13-41b2-a0ed-058afc5c8d73-kube-api-access-wg9qt\") pod \"nova-cell0-46b8-account-create-update-g9c2m\" (UID: \"d04ffd07-eb13-41b2-a0ed-058afc5c8d73\") " pod="openstack-kuttl-tests/nova-cell0-46b8-account-create-update-g9c2m" Dec 03 12:24:48 crc kubenswrapper[5052]: I1203 12:24:48.699381 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/d04ffd07-eb13-41b2-a0ed-058afc5c8d73-operator-scripts\") pod \"nova-cell0-46b8-account-create-update-g9c2m\" (UID: \"d04ffd07-eb13-41b2-a0ed-058afc5c8d73\") " pod="openstack-kuttl-tests/nova-cell0-46b8-account-create-update-g9c2m" Dec 03 12:24:48 crc kubenswrapper[5052]: I1203 12:24:48.709159 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-a918-account-create-update-s6fjd" Dec 03 12:24:48 crc kubenswrapper[5052]: I1203 12:24:48.724710 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-db-create-9vd8k" Dec 03 12:24:48 crc kubenswrapper[5052]: I1203 12:24:48.730817 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-wg9qt\" (UniqueName: \"kubernetes.io/projected/d04ffd07-eb13-41b2-a0ed-058afc5c8d73-kube-api-access-wg9qt\") pod \"nova-cell0-46b8-account-create-update-g9c2m\" (UID: \"d04ffd07-eb13-41b2-a0ed-058afc5c8d73\") " pod="openstack-kuttl-tests/nova-cell0-46b8-account-create-update-g9c2m" Dec 03 12:24:48 crc kubenswrapper[5052]: I1203 12:24:48.777813 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-46b8-account-create-update-g9c2m" Dec 03 12:24:48 crc kubenswrapper[5052]: I1203 12:24:48.799253 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cvwkq\" (UniqueName: \"kubernetes.io/projected/5bbad3a2-f21b-4ce1-8e37-5cf432d806ef-kube-api-access-cvwkq\") pod \"nova-cell1-15a6-account-create-update-z2tdn\" (UID: \"5bbad3a2-f21b-4ce1-8e37-5cf432d806ef\") " pod="openstack-kuttl-tests/nova-cell1-15a6-account-create-update-z2tdn" Dec 03 12:24:48 crc kubenswrapper[5052]: I1203 12:24:48.799384 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/5bbad3a2-f21b-4ce1-8e37-5cf432d806ef-operator-scripts\") pod \"nova-cell1-15a6-account-create-update-z2tdn\" (UID: \"5bbad3a2-f21b-4ce1-8e37-5cf432d806ef\") " pod="openstack-kuttl-tests/nova-cell1-15a6-account-create-update-z2tdn" Dec 03 12:24:48 crc kubenswrapper[5052]: I1203 12:24:48.901314 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/5bbad3a2-f21b-4ce1-8e37-5cf432d806ef-operator-scripts\") pod \"nova-cell1-15a6-account-create-update-z2tdn\" (UID: \"5bbad3a2-f21b-4ce1-8e37-5cf432d806ef\") " pod="openstack-kuttl-tests/nova-cell1-15a6-account-create-update-z2tdn" Dec 03 12:24:48 crc kubenswrapper[5052]: I1203 12:24:48.901997 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/5bbad3a2-f21b-4ce1-8e37-5cf432d806ef-operator-scripts\") pod \"nova-cell1-15a6-account-create-update-z2tdn\" (UID: \"5bbad3a2-f21b-4ce1-8e37-5cf432d806ef\") " pod="openstack-kuttl-tests/nova-cell1-15a6-account-create-update-z2tdn" Dec 03 12:24:48 crc kubenswrapper[5052]: I1203 12:24:48.902731 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-cvwkq\" (UniqueName: \"kubernetes.io/projected/5bbad3a2-f21b-4ce1-8e37-5cf432d806ef-kube-api-access-cvwkq\") pod \"nova-cell1-15a6-account-create-update-z2tdn\" (UID: \"5bbad3a2-f21b-4ce1-8e37-5cf432d806ef\") " pod="openstack-kuttl-tests/nova-cell1-15a6-account-create-update-z2tdn" Dec 03 12:24:48 crc kubenswrapper[5052]: I1203 12:24:48.924315 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-cvwkq\" (UniqueName: \"kubernetes.io/projected/5bbad3a2-f21b-4ce1-8e37-5cf432d806ef-kube-api-access-cvwkq\") pod \"nova-cell1-15a6-account-create-update-z2tdn\" (UID: \"5bbad3a2-f21b-4ce1-8e37-5cf432d806ef\") " pod="openstack-kuttl-tests/nova-cell1-15a6-account-create-update-z2tdn" Dec 03 12:24:49 crc kubenswrapper[5052]: I1203 12:24:49.097962 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-15a6-account-create-update-z2tdn" Dec 03 12:24:49 crc kubenswrapper[5052]: I1203 12:24:49.113189 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-db-create-846mj"] Dec 03 12:24:49 crc kubenswrapper[5052]: W1203 12:24:49.125609 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0160de5f_930d_4766_9302_28c89c2fcf14.slice/crio-4bceb95dff8b27a56cf78e34d0a92fc5e01546ff874cab7712808cf505f1b742 WatchSource:0}: Error finding container 4bceb95dff8b27a56cf78e34d0a92fc5e01546ff874cab7712808cf505f1b742: Status 404 returned error can't find the container with id 4bceb95dff8b27a56cf78e34d0a92fc5e01546ff874cab7712808cf505f1b742 Dec 03 12:24:49 crc kubenswrapper[5052]: I1203 12:24:49.161618 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-db-create-846mj" event={"ID":"0160de5f-930d-4766-9302-28c89c2fcf14","Type":"ContainerStarted","Data":"4bceb95dff8b27a56cf78e34d0a92fc5e01546ff874cab7712808cf505f1b742"} Dec 03 12:24:49 crc kubenswrapper[5052]: I1203 12:24:49.163147 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:24:49 crc kubenswrapper[5052]: I1203 12:24:49.220576 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:24:49 crc kubenswrapper[5052]: I1203 12:24:49.241402 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:24:49 crc kubenswrapper[5052]: I1203 12:24:49.276749 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:24:49 crc kubenswrapper[5052]: I1203 12:24:49.285687 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:24:49 crc kubenswrapper[5052]: I1203 12:24:49.294431 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-scripts" Dec 03 12:24:49 crc kubenswrapper[5052]: I1203 12:24:49.294731 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-config-data" Dec 03 12:24:49 crc kubenswrapper[5052]: I1203 12:24:49.296194 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:24:49 crc kubenswrapper[5052]: I1203 12:24:49.310270 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/877cfc19-61d2-48a8-ba30-2eb8a39a7c0d-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"877cfc19-61d2-48a8-ba30-2eb8a39a7c0d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:24:49 crc kubenswrapper[5052]: I1203 12:24:49.310354 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/877cfc19-61d2-48a8-ba30-2eb8a39a7c0d-scripts\") pod \"ceilometer-0\" (UID: \"877cfc19-61d2-48a8-ba30-2eb8a39a7c0d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:24:49 crc kubenswrapper[5052]: I1203 12:24:49.310383 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/877cfc19-61d2-48a8-ba30-2eb8a39a7c0d-log-httpd\") pod \"ceilometer-0\" (UID: \"877cfc19-61d2-48a8-ba30-2eb8a39a7c0d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:24:49 crc kubenswrapper[5052]: I1203 12:24:49.310627 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/877cfc19-61d2-48a8-ba30-2eb8a39a7c0d-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"877cfc19-61d2-48a8-ba30-2eb8a39a7c0d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:24:49 crc kubenswrapper[5052]: I1203 12:24:49.310908 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/877cfc19-61d2-48a8-ba30-2eb8a39a7c0d-run-httpd\") pod \"ceilometer-0\" (UID: \"877cfc19-61d2-48a8-ba30-2eb8a39a7c0d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:24:49 crc kubenswrapper[5052]: I1203 12:24:49.311090 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v5m4k\" (UniqueName: \"kubernetes.io/projected/877cfc19-61d2-48a8-ba30-2eb8a39a7c0d-kube-api-access-v5m4k\") pod \"ceilometer-0\" (UID: \"877cfc19-61d2-48a8-ba30-2eb8a39a7c0d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:24:49 crc kubenswrapper[5052]: I1203 12:24:49.311174 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/877cfc19-61d2-48a8-ba30-2eb8a39a7c0d-config-data\") pod \"ceilometer-0\" (UID: \"877cfc19-61d2-48a8-ba30-2eb8a39a7c0d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:24:49 crc kubenswrapper[5052]: I1203 12:24:49.328539 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-a918-account-create-update-s6fjd"] Dec 03 12:24:49 crc kubenswrapper[5052]: I1203 12:24:49.340171 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell0-db-create-5fgfv"] Dec 03 12:24:49 crc kubenswrapper[5052]: I1203 12:24:49.412458 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/877cfc19-61d2-48a8-ba30-2eb8a39a7c0d-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"877cfc19-61d2-48a8-ba30-2eb8a39a7c0d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:24:49 crc kubenswrapper[5052]: I1203 12:24:49.412652 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/877cfc19-61d2-48a8-ba30-2eb8a39a7c0d-scripts\") pod \"ceilometer-0\" (UID: \"877cfc19-61d2-48a8-ba30-2eb8a39a7c0d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:24:49 crc kubenswrapper[5052]: I1203 12:24:49.412690 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/877cfc19-61d2-48a8-ba30-2eb8a39a7c0d-log-httpd\") pod \"ceilometer-0\" (UID: \"877cfc19-61d2-48a8-ba30-2eb8a39a7c0d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:24:49 crc kubenswrapper[5052]: I1203 12:24:49.412762 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/877cfc19-61d2-48a8-ba30-2eb8a39a7c0d-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"877cfc19-61d2-48a8-ba30-2eb8a39a7c0d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:24:49 crc kubenswrapper[5052]: I1203 12:24:49.412815 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/877cfc19-61d2-48a8-ba30-2eb8a39a7c0d-run-httpd\") pod \"ceilometer-0\" (UID: \"877cfc19-61d2-48a8-ba30-2eb8a39a7c0d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:24:49 crc kubenswrapper[5052]: I1203 12:24:49.414474 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/877cfc19-61d2-48a8-ba30-2eb8a39a7c0d-log-httpd\") pod \"ceilometer-0\" (UID: \"877cfc19-61d2-48a8-ba30-2eb8a39a7c0d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:24:49 crc kubenswrapper[5052]: I1203 12:24:49.422188 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/877cfc19-61d2-48a8-ba30-2eb8a39a7c0d-scripts\") pod \"ceilometer-0\" (UID: \"877cfc19-61d2-48a8-ba30-2eb8a39a7c0d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:24:49 crc kubenswrapper[5052]: I1203 12:24:49.423527 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/877cfc19-61d2-48a8-ba30-2eb8a39a7c0d-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"877cfc19-61d2-48a8-ba30-2eb8a39a7c0d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:24:49 crc kubenswrapper[5052]: I1203 12:24:49.423714 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/877cfc19-61d2-48a8-ba30-2eb8a39a7c0d-run-httpd\") pod \"ceilometer-0\" (UID: \"877cfc19-61d2-48a8-ba30-2eb8a39a7c0d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:24:49 crc kubenswrapper[5052]: I1203 12:24:49.423748 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-v5m4k\" (UniqueName: \"kubernetes.io/projected/877cfc19-61d2-48a8-ba30-2eb8a39a7c0d-kube-api-access-v5m4k\") pod \"ceilometer-0\" (UID: \"877cfc19-61d2-48a8-ba30-2eb8a39a7c0d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:24:49 crc kubenswrapper[5052]: I1203 12:24:49.423836 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/877cfc19-61d2-48a8-ba30-2eb8a39a7c0d-config-data\") pod \"ceilometer-0\" (UID: \"877cfc19-61d2-48a8-ba30-2eb8a39a7c0d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:24:49 crc kubenswrapper[5052]: I1203 12:24:49.426149 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/877cfc19-61d2-48a8-ba30-2eb8a39a7c0d-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"877cfc19-61d2-48a8-ba30-2eb8a39a7c0d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:24:49 crc kubenswrapper[5052]: I1203 12:24:49.429214 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/877cfc19-61d2-48a8-ba30-2eb8a39a7c0d-config-data\") pod \"ceilometer-0\" (UID: \"877cfc19-61d2-48a8-ba30-2eb8a39a7c0d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:24:49 crc kubenswrapper[5052]: I1203 12:24:49.450732 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-v5m4k\" (UniqueName: \"kubernetes.io/projected/877cfc19-61d2-48a8-ba30-2eb8a39a7c0d-kube-api-access-v5m4k\") pod \"ceilometer-0\" (UID: \"877cfc19-61d2-48a8-ba30-2eb8a39a7c0d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:24:49 crc kubenswrapper[5052]: I1203 12:24:49.556159 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell0-46b8-account-create-update-g9c2m"] Dec 03 12:24:49 crc kubenswrapper[5052]: I1203 12:24:49.629331 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:24:49 crc kubenswrapper[5052]: I1203 12:24:49.646430 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-db-create-9vd8k"] Dec 03 12:24:49 crc kubenswrapper[5052]: W1203 12:24:49.686613 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod47087a5b_7fd2_498e_b9d0_0bc3c6939637.slice/crio-5a80b3393f0d280254570d49ad67a8803daf7415f928eec0c0f848f240e997c6 WatchSource:0}: Error finding container 5a80b3393f0d280254570d49ad67a8803daf7415f928eec0c0f848f240e997c6: Status 404 returned error can't find the container with id 5a80b3393f0d280254570d49ad67a8803daf7415f928eec0c0f848f240e997c6 Dec 03 12:24:49 crc kubenswrapper[5052]: I1203 12:24:49.719380 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2660f107-7c9f-4d25-be86-3ba68a71be57" path="/var/lib/kubelet/pods/2660f107-7c9f-4d25-be86-3ba68a71be57/volumes" Dec 03 12:24:49 crc kubenswrapper[5052]: W1203 12:24:49.891225 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5bbad3a2_f21b_4ce1_8e37_5cf432d806ef.slice/crio-38df588c5b2296b9da1af544fa019599ea1d9661319b4bc24119aafa96c398f3 WatchSource:0}: Error finding container 38df588c5b2296b9da1af544fa019599ea1d9661319b4bc24119aafa96c398f3: Status 404 returned error can't find the container with id 38df588c5b2296b9da1af544fa019599ea1d9661319b4bc24119aafa96c398f3 Dec 03 12:24:49 crc kubenswrapper[5052]: I1203 12:24:49.909634 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-15a6-account-create-update-z2tdn"] Dec 03 12:24:50 crc kubenswrapper[5052]: I1203 12:24:50.228834 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-a918-account-create-update-s6fjd" event={"ID":"a1ac8f66-94c3-4ac5-b039-fe29314ed99a","Type":"ContainerStarted","Data":"6dd77861dd24def3a954ac96db60a8061925af3b1fbe5c7d29936dd201e6dea4"} Dec 03 12:24:50 crc kubenswrapper[5052]: I1203 12:24:50.228901 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-a918-account-create-update-s6fjd" event={"ID":"a1ac8f66-94c3-4ac5-b039-fe29314ed99a","Type":"ContainerStarted","Data":"fd260aa70b4b2507e48661a0168d739652ade71cb76ae138f0f30645a6bbc7ec"} Dec 03 12:24:50 crc kubenswrapper[5052]: I1203 12:24:50.250915 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-46b8-account-create-update-g9c2m" event={"ID":"d04ffd07-eb13-41b2-a0ed-058afc5c8d73","Type":"ContainerStarted","Data":"92379b4ee05501138180215d0ddb2588eed029321c921b317ba693c25f8551e2"} Dec 03 12:24:50 crc kubenswrapper[5052]: I1203 12:24:50.250992 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-46b8-account-create-update-g9c2m" event={"ID":"d04ffd07-eb13-41b2-a0ed-058afc5c8d73","Type":"ContainerStarted","Data":"1c59c970de1059367a5ac6010acc0dccc89062b3888d591f2c449b1f1942d2d8"} Dec 03 12:24:50 crc kubenswrapper[5052]: I1203 12:24:50.263710 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-api-a918-account-create-update-s6fjd" podStartSLOduration=2.263692173 podStartE2EDuration="2.263692173s" podCreationTimestamp="2025-12-03 12:24:48 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:24:50.263147897 +0000 UTC m=+3880.851389661" watchObservedRunningTime="2025-12-03 12:24:50.263692173 +0000 UTC m=+3880.851933937" Dec 03 12:24:50 crc kubenswrapper[5052]: I1203 12:24:50.267761 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-15a6-account-create-update-z2tdn" event={"ID":"5bbad3a2-f21b-4ce1-8e37-5cf432d806ef","Type":"ContainerStarted","Data":"38df588c5b2296b9da1af544fa019599ea1d9661319b4bc24119aafa96c398f3"} Dec 03 12:24:50 crc kubenswrapper[5052]: I1203 12:24:50.305026 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-db-create-5fgfv" event={"ID":"027c0ef8-c154-4de5-961e-d822ab32a761","Type":"ContainerStarted","Data":"2e86dc76cd3c6e8f0ef1db1ef4ca376f07e13b1e654e9d9b8f0bae1e0f3a5a4f"} Dec 03 12:24:50 crc kubenswrapper[5052]: I1203 12:24:50.305103 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-db-create-5fgfv" event={"ID":"027c0ef8-c154-4de5-961e-d822ab32a761","Type":"ContainerStarted","Data":"ea4946ebd3108e6e8196d65868de6d6543d923fa3c09db2cc07dfbae07c01c74"} Dec 03 12:24:50 crc kubenswrapper[5052]: I1203 12:24:50.309966 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-db-create-9vd8k" event={"ID":"47087a5b-7fd2-498e-b9d0-0bc3c6939637","Type":"ContainerStarted","Data":"245c1a42883f9d4981abbc2944dd195440c9014036cc110f8c69aeadb6a35ea1"} Dec 03 12:24:50 crc kubenswrapper[5052]: I1203 12:24:50.310036 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-db-create-9vd8k" event={"ID":"47087a5b-7fd2-498e-b9d0-0bc3c6939637","Type":"ContainerStarted","Data":"5a80b3393f0d280254570d49ad67a8803daf7415f928eec0c0f848f240e997c6"} Dec 03 12:24:50 crc kubenswrapper[5052]: I1203 12:24:50.340710 5052 generic.go:334] "Generic (PLEG): container finished" podID="0160de5f-930d-4766-9302-28c89c2fcf14" containerID="387c7b8ee696cb426191ea26fa0317bb28fa52a78581eb4197de94eee8d0a4a6" exitCode=0 Dec 03 12:24:50 crc kubenswrapper[5052]: I1203 12:24:50.340772 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-db-create-846mj" event={"ID":"0160de5f-930d-4766-9302-28c89c2fcf14","Type":"ContainerDied","Data":"387c7b8ee696cb426191ea26fa0317bb28fa52a78581eb4197de94eee8d0a4a6"} Dec 03 12:24:50 crc kubenswrapper[5052]: I1203 12:24:50.342479 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-cell0-46b8-account-create-update-g9c2m" podStartSLOduration=2.34240877 podStartE2EDuration="2.34240877s" podCreationTimestamp="2025-12-03 12:24:48 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:24:50.295381887 +0000 UTC m=+3880.883623651" watchObservedRunningTime="2025-12-03 12:24:50.34240877 +0000 UTC m=+3880.930650544" Dec 03 12:24:50 crc kubenswrapper[5052]: I1203 12:24:50.354080 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-cell0-db-create-5fgfv" podStartSLOduration=2.354053622 podStartE2EDuration="2.354053622s" podCreationTimestamp="2025-12-03 12:24:48 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:24:50.339868547 +0000 UTC m=+3880.928110311" watchObservedRunningTime="2025-12-03 12:24:50.354053622 +0000 UTC m=+3880.942295386" Dec 03 12:24:50 crc kubenswrapper[5052]: I1203 12:24:50.387052 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-cell1-db-create-9vd8k" podStartSLOduration=2.387031393 podStartE2EDuration="2.387031393s" podCreationTimestamp="2025-12-03 12:24:48 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:24:50.381219557 +0000 UTC m=+3880.969461331" watchObservedRunningTime="2025-12-03 12:24:50.387031393 +0000 UTC m=+3880.975273157" Dec 03 12:24:50 crc kubenswrapper[5052]: I1203 12:24:50.591945 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:24:51 crc kubenswrapper[5052]: I1203 12:24:51.139353 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:24:51 crc kubenswrapper[5052]: I1203 12:24:51.354414 5052 generic.go:334] "Generic (PLEG): container finished" podID="d04ffd07-eb13-41b2-a0ed-058afc5c8d73" containerID="92379b4ee05501138180215d0ddb2588eed029321c921b317ba693c25f8551e2" exitCode=0 Dec 03 12:24:51 crc kubenswrapper[5052]: I1203 12:24:51.354465 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-46b8-account-create-update-g9c2m" event={"ID":"d04ffd07-eb13-41b2-a0ed-058afc5c8d73","Type":"ContainerDied","Data":"92379b4ee05501138180215d0ddb2588eed029321c921b317ba693c25f8551e2"} Dec 03 12:24:51 crc kubenswrapper[5052]: I1203 12:24:51.356916 5052 generic.go:334] "Generic (PLEG): container finished" podID="5bbad3a2-f21b-4ce1-8e37-5cf432d806ef" containerID="fdfab1b488effd7f6ef251f831aebda5da17360079975291ad8ed38f57cc9bd2" exitCode=0 Dec 03 12:24:51 crc kubenswrapper[5052]: I1203 12:24:51.356985 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-15a6-account-create-update-z2tdn" event={"ID":"5bbad3a2-f21b-4ce1-8e37-5cf432d806ef","Type":"ContainerDied","Data":"fdfab1b488effd7f6ef251f831aebda5da17360079975291ad8ed38f57cc9bd2"} Dec 03 12:24:51 crc kubenswrapper[5052]: I1203 12:24:51.359253 5052 generic.go:334] "Generic (PLEG): container finished" podID="027c0ef8-c154-4de5-961e-d822ab32a761" containerID="2e86dc76cd3c6e8f0ef1db1ef4ca376f07e13b1e654e9d9b8f0bae1e0f3a5a4f" exitCode=0 Dec 03 12:24:51 crc kubenswrapper[5052]: I1203 12:24:51.359316 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-db-create-5fgfv" event={"ID":"027c0ef8-c154-4de5-961e-d822ab32a761","Type":"ContainerDied","Data":"2e86dc76cd3c6e8f0ef1db1ef4ca376f07e13b1e654e9d9b8f0bae1e0f3a5a4f"} Dec 03 12:24:51 crc kubenswrapper[5052]: I1203 12:24:51.363481 5052 generic.go:334] "Generic (PLEG): container finished" podID="47087a5b-7fd2-498e-b9d0-0bc3c6939637" containerID="245c1a42883f9d4981abbc2944dd195440c9014036cc110f8c69aeadb6a35ea1" exitCode=0 Dec 03 12:24:51 crc kubenswrapper[5052]: I1203 12:24:51.363568 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-db-create-9vd8k" event={"ID":"47087a5b-7fd2-498e-b9d0-0bc3c6939637","Type":"ContainerDied","Data":"245c1a42883f9d4981abbc2944dd195440c9014036cc110f8c69aeadb6a35ea1"} Dec 03 12:24:51 crc kubenswrapper[5052]: I1203 12:24:51.365832 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"877cfc19-61d2-48a8-ba30-2eb8a39a7c0d","Type":"ContainerStarted","Data":"aa433cb8f25271a7777220766a0b0418aa7ae5bc02abce6c6a2edf32dd7abacd"} Dec 03 12:24:51 crc kubenswrapper[5052]: I1203 12:24:51.368292 5052 generic.go:334] "Generic (PLEG): container finished" podID="a1ac8f66-94c3-4ac5-b039-fe29314ed99a" containerID="6dd77861dd24def3a954ac96db60a8061925af3b1fbe5c7d29936dd201e6dea4" exitCode=0 Dec 03 12:24:51 crc kubenswrapper[5052]: I1203 12:24:51.368554 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-a918-account-create-update-s6fjd" event={"ID":"a1ac8f66-94c3-4ac5-b039-fe29314ed99a","Type":"ContainerDied","Data":"6dd77861dd24def3a954ac96db60a8061925af3b1fbe5c7d29936dd201e6dea4"} Dec 03 12:24:51 crc kubenswrapper[5052]: I1203 12:24:51.905754 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-db-create-846mj" Dec 03 12:24:51 crc kubenswrapper[5052]: I1203 12:24:51.917529 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/swift-proxy-84c4f5d658-m9l78" Dec 03 12:24:51 crc kubenswrapper[5052]: I1203 12:24:51.927398 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/swift-proxy-84c4f5d658-m9l78" Dec 03 12:24:52 crc kubenswrapper[5052]: I1203 12:24:52.029367 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/0160de5f-930d-4766-9302-28c89c2fcf14-operator-scripts\") pod \"0160de5f-930d-4766-9302-28c89c2fcf14\" (UID: \"0160de5f-930d-4766-9302-28c89c2fcf14\") " Dec 03 12:24:52 crc kubenswrapper[5052]: I1203 12:24:52.029658 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6t94d\" (UniqueName: \"kubernetes.io/projected/0160de5f-930d-4766-9302-28c89c2fcf14-kube-api-access-6t94d\") pod \"0160de5f-930d-4766-9302-28c89c2fcf14\" (UID: \"0160de5f-930d-4766-9302-28c89c2fcf14\") " Dec 03 12:24:52 crc kubenswrapper[5052]: I1203 12:24:52.030702 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0160de5f-930d-4766-9302-28c89c2fcf14-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "0160de5f-930d-4766-9302-28c89c2fcf14" (UID: "0160de5f-930d-4766-9302-28c89c2fcf14"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:24:52 crc kubenswrapper[5052]: I1203 12:24:52.031294 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/0160de5f-930d-4766-9302-28c89c2fcf14-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:24:52 crc kubenswrapper[5052]: I1203 12:24:52.033910 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0160de5f-930d-4766-9302-28c89c2fcf14-kube-api-access-6t94d" (OuterVolumeSpecName: "kube-api-access-6t94d") pod "0160de5f-930d-4766-9302-28c89c2fcf14" (UID: "0160de5f-930d-4766-9302-28c89c2fcf14"). InnerVolumeSpecName "kube-api-access-6t94d". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:24:52 crc kubenswrapper[5052]: I1203 12:24:52.132604 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-6t94d\" (UniqueName: \"kubernetes.io/projected/0160de5f-930d-4766-9302-28c89c2fcf14-kube-api-access-6t94d\") on node \"crc\" DevicePath \"\"" Dec 03 12:24:52 crc kubenswrapper[5052]: I1203 12:24:52.387842 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-db-create-846mj" event={"ID":"0160de5f-930d-4766-9302-28c89c2fcf14","Type":"ContainerDied","Data":"4bceb95dff8b27a56cf78e34d0a92fc5e01546ff874cab7712808cf505f1b742"} Dec 03 12:24:52 crc kubenswrapper[5052]: I1203 12:24:52.387892 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4bceb95dff8b27a56cf78e34d0a92fc5e01546ff874cab7712808cf505f1b742" Dec 03 12:24:52 crc kubenswrapper[5052]: I1203 12:24:52.388268 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-db-create-846mj" Dec 03 12:24:52 crc kubenswrapper[5052]: I1203 12:24:52.393299 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"877cfc19-61d2-48a8-ba30-2eb8a39a7c0d","Type":"ContainerStarted","Data":"f0af04bda2d702b68508554603573208064e8a15663ea9bba8a0ae02fde8add6"} Dec 03 12:24:52 crc kubenswrapper[5052]: I1203 12:24:52.393356 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"877cfc19-61d2-48a8-ba30-2eb8a39a7c0d","Type":"ContainerStarted","Data":"333d5988d6ce193cc6cbf5c47b0c3154146fadf562614a1dddecfa687325ba71"} Dec 03 12:24:53 crc kubenswrapper[5052]: I1203 12:24:53.002595 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-15a6-account-create-update-z2tdn" Dec 03 12:24:53 crc kubenswrapper[5052]: I1203 12:24:53.157193 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/5bbad3a2-f21b-4ce1-8e37-5cf432d806ef-operator-scripts\") pod \"5bbad3a2-f21b-4ce1-8e37-5cf432d806ef\" (UID: \"5bbad3a2-f21b-4ce1-8e37-5cf432d806ef\") " Dec 03 12:24:53 crc kubenswrapper[5052]: I1203 12:24:53.157839 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cvwkq\" (UniqueName: \"kubernetes.io/projected/5bbad3a2-f21b-4ce1-8e37-5cf432d806ef-kube-api-access-cvwkq\") pod \"5bbad3a2-f21b-4ce1-8e37-5cf432d806ef\" (UID: \"5bbad3a2-f21b-4ce1-8e37-5cf432d806ef\") " Dec 03 12:24:53 crc kubenswrapper[5052]: I1203 12:24:53.159130 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5bbad3a2-f21b-4ce1-8e37-5cf432d806ef-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "5bbad3a2-f21b-4ce1-8e37-5cf432d806ef" (UID: "5bbad3a2-f21b-4ce1-8e37-5cf432d806ef"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:24:53 crc kubenswrapper[5052]: I1203 12:24:53.167732 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5bbad3a2-f21b-4ce1-8e37-5cf432d806ef-kube-api-access-cvwkq" (OuterVolumeSpecName: "kube-api-access-cvwkq") pod "5bbad3a2-f21b-4ce1-8e37-5cf432d806ef" (UID: "5bbad3a2-f21b-4ce1-8e37-5cf432d806ef"). InnerVolumeSpecName "kube-api-access-cvwkq". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:24:53 crc kubenswrapper[5052]: I1203 12:24:53.260448 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-cvwkq\" (UniqueName: \"kubernetes.io/projected/5bbad3a2-f21b-4ce1-8e37-5cf432d806ef-kube-api-access-cvwkq\") on node \"crc\" DevicePath \"\"" Dec 03 12:24:53 crc kubenswrapper[5052]: I1203 12:24:53.260484 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/5bbad3a2-f21b-4ce1-8e37-5cf432d806ef-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:24:53 crc kubenswrapper[5052]: I1203 12:24:53.345581 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-db-create-9vd8k" Dec 03 12:24:53 crc kubenswrapper[5052]: I1203 12:24:53.355147 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-a918-account-create-update-s6fjd" Dec 03 12:24:53 crc kubenswrapper[5052]: I1203 12:24:53.364111 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-db-create-5fgfv" Dec 03 12:24:53 crc kubenswrapper[5052]: I1203 12:24:53.389736 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-46b8-account-create-update-g9c2m" Dec 03 12:24:53 crc kubenswrapper[5052]: I1203 12:24:53.414685 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-15a6-account-create-update-z2tdn" event={"ID":"5bbad3a2-f21b-4ce1-8e37-5cf432d806ef","Type":"ContainerDied","Data":"38df588c5b2296b9da1af544fa019599ea1d9661319b4bc24119aafa96c398f3"} Dec 03 12:24:53 crc kubenswrapper[5052]: I1203 12:24:53.414979 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="38df588c5b2296b9da1af544fa019599ea1d9661319b4bc24119aafa96c398f3" Dec 03 12:24:53 crc kubenswrapper[5052]: I1203 12:24:53.415034 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-15a6-account-create-update-z2tdn" Dec 03 12:24:53 crc kubenswrapper[5052]: I1203 12:24:53.440873 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-db-create-5fgfv" event={"ID":"027c0ef8-c154-4de5-961e-d822ab32a761","Type":"ContainerDied","Data":"ea4946ebd3108e6e8196d65868de6d6543d923fa3c09db2cc07dfbae07c01c74"} Dec 03 12:24:53 crc kubenswrapper[5052]: I1203 12:24:53.440984 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ea4946ebd3108e6e8196d65868de6d6543d923fa3c09db2cc07dfbae07c01c74" Dec 03 12:24:53 crc kubenswrapper[5052]: I1203 12:24:53.441082 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-db-create-5fgfv" Dec 03 12:24:53 crc kubenswrapper[5052]: I1203 12:24:53.456988 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-db-create-9vd8k" event={"ID":"47087a5b-7fd2-498e-b9d0-0bc3c6939637","Type":"ContainerDied","Data":"5a80b3393f0d280254570d49ad67a8803daf7415f928eec0c0f848f240e997c6"} Dec 03 12:24:53 crc kubenswrapper[5052]: I1203 12:24:53.457058 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5a80b3393f0d280254570d49ad67a8803daf7415f928eec0c0f848f240e997c6" Dec 03 12:24:53 crc kubenswrapper[5052]: I1203 12:24:53.457179 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-db-create-9vd8k" Dec 03 12:24:53 crc kubenswrapper[5052]: I1203 12:24:53.462426 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/47087a5b-7fd2-498e-b9d0-0bc3c6939637-operator-scripts\") pod \"47087a5b-7fd2-498e-b9d0-0bc3c6939637\" (UID: \"47087a5b-7fd2-498e-b9d0-0bc3c6939637\") " Dec 03 12:24:53 crc kubenswrapper[5052]: I1203 12:24:53.462470 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bbxsd\" (UniqueName: \"kubernetes.io/projected/027c0ef8-c154-4de5-961e-d822ab32a761-kube-api-access-bbxsd\") pod \"027c0ef8-c154-4de5-961e-d822ab32a761\" (UID: \"027c0ef8-c154-4de5-961e-d822ab32a761\") " Dec 03 12:24:53 crc kubenswrapper[5052]: I1203 12:24:53.462617 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/027c0ef8-c154-4de5-961e-d822ab32a761-operator-scripts\") pod \"027c0ef8-c154-4de5-961e-d822ab32a761\" (UID: \"027c0ef8-c154-4de5-961e-d822ab32a761\") " Dec 03 12:24:53 crc kubenswrapper[5052]: I1203 12:24:53.462655 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k954n\" (UniqueName: \"kubernetes.io/projected/a1ac8f66-94c3-4ac5-b039-fe29314ed99a-kube-api-access-k954n\") pod \"a1ac8f66-94c3-4ac5-b039-fe29314ed99a\" (UID: \"a1ac8f66-94c3-4ac5-b039-fe29314ed99a\") " Dec 03 12:24:53 crc kubenswrapper[5052]: I1203 12:24:53.462722 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-72928\" (UniqueName: \"kubernetes.io/projected/47087a5b-7fd2-498e-b9d0-0bc3c6939637-kube-api-access-72928\") pod \"47087a5b-7fd2-498e-b9d0-0bc3c6939637\" (UID: \"47087a5b-7fd2-498e-b9d0-0bc3c6939637\") " Dec 03 12:24:53 crc kubenswrapper[5052]: I1203 12:24:53.462741 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/a1ac8f66-94c3-4ac5-b039-fe29314ed99a-operator-scripts\") pod \"a1ac8f66-94c3-4ac5-b039-fe29314ed99a\" (UID: \"a1ac8f66-94c3-4ac5-b039-fe29314ed99a\") " Dec 03 12:24:53 crc kubenswrapper[5052]: I1203 12:24:53.463763 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a1ac8f66-94c3-4ac5-b039-fe29314ed99a-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "a1ac8f66-94c3-4ac5-b039-fe29314ed99a" (UID: "a1ac8f66-94c3-4ac5-b039-fe29314ed99a"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:24:53 crc kubenswrapper[5052]: I1203 12:24:53.464286 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/027c0ef8-c154-4de5-961e-d822ab32a761-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "027c0ef8-c154-4de5-961e-d822ab32a761" (UID: "027c0ef8-c154-4de5-961e-d822ab32a761"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:24:53 crc kubenswrapper[5052]: I1203 12:24:53.464710 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/47087a5b-7fd2-498e-b9d0-0bc3c6939637-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "47087a5b-7fd2-498e-b9d0-0bc3c6939637" (UID: "47087a5b-7fd2-498e-b9d0-0bc3c6939637"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:24:53 crc kubenswrapper[5052]: I1203 12:24:53.468440 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-a918-account-create-update-s6fjd" Dec 03 12:24:53 crc kubenswrapper[5052]: I1203 12:24:53.468491 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-a918-account-create-update-s6fjd" event={"ID":"a1ac8f66-94c3-4ac5-b039-fe29314ed99a","Type":"ContainerDied","Data":"fd260aa70b4b2507e48661a0168d739652ade71cb76ae138f0f30645a6bbc7ec"} Dec 03 12:24:53 crc kubenswrapper[5052]: I1203 12:24:53.468543 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fd260aa70b4b2507e48661a0168d739652ade71cb76ae138f0f30645a6bbc7ec" Dec 03 12:24:53 crc kubenswrapper[5052]: I1203 12:24:53.470672 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a1ac8f66-94c3-4ac5-b039-fe29314ed99a-kube-api-access-k954n" (OuterVolumeSpecName: "kube-api-access-k954n") pod "a1ac8f66-94c3-4ac5-b039-fe29314ed99a" (UID: "a1ac8f66-94c3-4ac5-b039-fe29314ed99a"). InnerVolumeSpecName "kube-api-access-k954n". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:24:53 crc kubenswrapper[5052]: I1203 12:24:53.470814 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-46b8-account-create-update-g9c2m" event={"ID":"d04ffd07-eb13-41b2-a0ed-058afc5c8d73","Type":"ContainerDied","Data":"1c59c970de1059367a5ac6010acc0dccc89062b3888d591f2c449b1f1942d2d8"} Dec 03 12:24:53 crc kubenswrapper[5052]: I1203 12:24:53.470857 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1c59c970de1059367a5ac6010acc0dccc89062b3888d591f2c449b1f1942d2d8" Dec 03 12:24:53 crc kubenswrapper[5052]: I1203 12:24:53.470925 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-46b8-account-create-update-g9c2m" Dec 03 12:24:53 crc kubenswrapper[5052]: I1203 12:24:53.471680 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/027c0ef8-c154-4de5-961e-d822ab32a761-kube-api-access-bbxsd" (OuterVolumeSpecName: "kube-api-access-bbxsd") pod "027c0ef8-c154-4de5-961e-d822ab32a761" (UID: "027c0ef8-c154-4de5-961e-d822ab32a761"). InnerVolumeSpecName "kube-api-access-bbxsd". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:24:53 crc kubenswrapper[5052]: I1203 12:24:53.485319 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/47087a5b-7fd2-498e-b9d0-0bc3c6939637-kube-api-access-72928" (OuterVolumeSpecName: "kube-api-access-72928") pod "47087a5b-7fd2-498e-b9d0-0bc3c6939637" (UID: "47087a5b-7fd2-498e-b9d0-0bc3c6939637"). InnerVolumeSpecName "kube-api-access-72928". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:24:53 crc kubenswrapper[5052]: I1203 12:24:53.563898 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/d04ffd07-eb13-41b2-a0ed-058afc5c8d73-operator-scripts\") pod \"d04ffd07-eb13-41b2-a0ed-058afc5c8d73\" (UID: \"d04ffd07-eb13-41b2-a0ed-058afc5c8d73\") " Dec 03 12:24:53 crc kubenswrapper[5052]: I1203 12:24:53.564313 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wg9qt\" (UniqueName: \"kubernetes.io/projected/d04ffd07-eb13-41b2-a0ed-058afc5c8d73-kube-api-access-wg9qt\") pod \"d04ffd07-eb13-41b2-a0ed-058afc5c8d73\" (UID: \"d04ffd07-eb13-41b2-a0ed-058afc5c8d73\") " Dec 03 12:24:53 crc kubenswrapper[5052]: I1203 12:24:53.564982 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d04ffd07-eb13-41b2-a0ed-058afc5c8d73-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "d04ffd07-eb13-41b2-a0ed-058afc5c8d73" (UID: "d04ffd07-eb13-41b2-a0ed-058afc5c8d73"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:24:53 crc kubenswrapper[5052]: I1203 12:24:53.565114 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/027c0ef8-c154-4de5-961e-d822ab32a761-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:24:53 crc kubenswrapper[5052]: I1203 12:24:53.565217 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-k954n\" (UniqueName: \"kubernetes.io/projected/a1ac8f66-94c3-4ac5-b039-fe29314ed99a-kube-api-access-k954n\") on node \"crc\" DevicePath \"\"" Dec 03 12:24:53 crc kubenswrapper[5052]: I1203 12:24:53.565307 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/a1ac8f66-94c3-4ac5-b039-fe29314ed99a-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:24:53 crc kubenswrapper[5052]: I1203 12:24:53.565390 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-72928\" (UniqueName: \"kubernetes.io/projected/47087a5b-7fd2-498e-b9d0-0bc3c6939637-kube-api-access-72928\") on node \"crc\" DevicePath \"\"" Dec 03 12:24:53 crc kubenswrapper[5052]: I1203 12:24:53.565659 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/47087a5b-7fd2-498e-b9d0-0bc3c6939637-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:24:53 crc kubenswrapper[5052]: I1203 12:24:53.565766 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-bbxsd\" (UniqueName: \"kubernetes.io/projected/027c0ef8-c154-4de5-961e-d822ab32a761-kube-api-access-bbxsd\") on node \"crc\" DevicePath \"\"" Dec 03 12:24:53 crc kubenswrapper[5052]: I1203 12:24:53.569311 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d04ffd07-eb13-41b2-a0ed-058afc5c8d73-kube-api-access-wg9qt" (OuterVolumeSpecName: "kube-api-access-wg9qt") pod "d04ffd07-eb13-41b2-a0ed-058afc5c8d73" (UID: "d04ffd07-eb13-41b2-a0ed-058afc5c8d73"). InnerVolumeSpecName "kube-api-access-wg9qt". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:24:53 crc kubenswrapper[5052]: I1203 12:24:53.667949 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-wg9qt\" (UniqueName: \"kubernetes.io/projected/d04ffd07-eb13-41b2-a0ed-058afc5c8d73-kube-api-access-wg9qt\") on node \"crc\" DevicePath \"\"" Dec 03 12:24:53 crc kubenswrapper[5052]: I1203 12:24:53.668214 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/d04ffd07-eb13-41b2-a0ed-058afc5c8d73-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:24:54 crc kubenswrapper[5052]: I1203 12:24:54.481749 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"877cfc19-61d2-48a8-ba30-2eb8a39a7c0d","Type":"ContainerStarted","Data":"00870692d2b50338582621d68d6c27be034aaac53091d9351d0a669d8366e448"} Dec 03 12:24:55 crc kubenswrapper[5052]: I1203 12:24:55.504271 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"877cfc19-61d2-48a8-ba30-2eb8a39a7c0d","Type":"ContainerStarted","Data":"b1610ebac2aa0f8aa4b81967dc3ae3b5979590368ec9deafce7d6a3c16ed7f4b"} Dec 03 12:24:55 crc kubenswrapper[5052]: I1203 12:24:55.504553 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="877cfc19-61d2-48a8-ba30-2eb8a39a7c0d" containerName="proxy-httpd" containerID="cri-o://b1610ebac2aa0f8aa4b81967dc3ae3b5979590368ec9deafce7d6a3c16ed7f4b" gracePeriod=30 Dec 03 12:24:55 crc kubenswrapper[5052]: I1203 12:24:55.504581 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="877cfc19-61d2-48a8-ba30-2eb8a39a7c0d" containerName="sg-core" containerID="cri-o://00870692d2b50338582621d68d6c27be034aaac53091d9351d0a669d8366e448" gracePeriod=30 Dec 03 12:24:55 crc kubenswrapper[5052]: I1203 12:24:55.504805 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:24:55 crc kubenswrapper[5052]: I1203 12:24:55.504599 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="877cfc19-61d2-48a8-ba30-2eb8a39a7c0d" containerName="ceilometer-notification-agent" containerID="cri-o://f0af04bda2d702b68508554603573208064e8a15663ea9bba8a0ae02fde8add6" gracePeriod=30 Dec 03 12:24:55 crc kubenswrapper[5052]: I1203 12:24:55.505021 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="877cfc19-61d2-48a8-ba30-2eb8a39a7c0d" containerName="ceilometer-central-agent" containerID="cri-o://333d5988d6ce193cc6cbf5c47b0c3154146fadf562614a1dddecfa687325ba71" gracePeriod=30 Dec 03 12:24:55 crc kubenswrapper[5052]: I1203 12:24:55.531551 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/ceilometer-0" podStartSLOduration=2.464033773 podStartE2EDuration="6.531533178s" podCreationTimestamp="2025-12-03 12:24:49 +0000 UTC" firstStartedPulling="2025-12-03 12:24:50.577294784 +0000 UTC m=+3881.165536548" lastFinishedPulling="2025-12-03 12:24:54.644794179 +0000 UTC m=+3885.233035953" observedRunningTime="2025-12-03 12:24:55.528067979 +0000 UTC m=+3886.116309763" watchObservedRunningTime="2025-12-03 12:24:55.531533178 +0000 UTC m=+3886.119774942" Dec 03 12:24:56 crc kubenswrapper[5052]: I1203 12:24:56.407363 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:24:56 crc kubenswrapper[5052]: I1203 12:24:56.408030 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-internal-api-0" podUID="9514eb24-503a-40b8-a339-481f81fd49de" containerName="glance-log" containerID="cri-o://6879b4a0fe254ab4b3401ca94ab6f5e16f1a55dd48a45c455affc3cd5f8d6685" gracePeriod=30 Dec 03 12:24:56 crc kubenswrapper[5052]: I1203 12:24:56.408079 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-internal-api-0" podUID="9514eb24-503a-40b8-a339-481f81fd49de" containerName="glance-httpd" containerID="cri-o://f6dbc6deaf8843ea5432f48c4bfb01f18f24c61b85c39d693d7ae58e5df45acf" gracePeriod=30 Dec 03 12:24:56 crc kubenswrapper[5052]: I1203 12:24:56.515974 5052 generic.go:334] "Generic (PLEG): container finished" podID="877cfc19-61d2-48a8-ba30-2eb8a39a7c0d" containerID="b1610ebac2aa0f8aa4b81967dc3ae3b5979590368ec9deafce7d6a3c16ed7f4b" exitCode=0 Dec 03 12:24:56 crc kubenswrapper[5052]: I1203 12:24:56.516005 5052 generic.go:334] "Generic (PLEG): container finished" podID="877cfc19-61d2-48a8-ba30-2eb8a39a7c0d" containerID="00870692d2b50338582621d68d6c27be034aaac53091d9351d0a669d8366e448" exitCode=2 Dec 03 12:24:56 crc kubenswrapper[5052]: I1203 12:24:56.516014 5052 generic.go:334] "Generic (PLEG): container finished" podID="877cfc19-61d2-48a8-ba30-2eb8a39a7c0d" containerID="f0af04bda2d702b68508554603573208064e8a15663ea9bba8a0ae02fde8add6" exitCode=0 Dec 03 12:24:56 crc kubenswrapper[5052]: I1203 12:24:56.516044 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"877cfc19-61d2-48a8-ba30-2eb8a39a7c0d","Type":"ContainerDied","Data":"b1610ebac2aa0f8aa4b81967dc3ae3b5979590368ec9deafce7d6a3c16ed7f4b"} Dec 03 12:24:56 crc kubenswrapper[5052]: I1203 12:24:56.516071 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"877cfc19-61d2-48a8-ba30-2eb8a39a7c0d","Type":"ContainerDied","Data":"00870692d2b50338582621d68d6c27be034aaac53091d9351d0a669d8366e448"} Dec 03 12:24:56 crc kubenswrapper[5052]: I1203 12:24:56.516081 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"877cfc19-61d2-48a8-ba30-2eb8a39a7c0d","Type":"ContainerDied","Data":"f0af04bda2d702b68508554603573208064e8a15663ea9bba8a0ae02fde8add6"} Dec 03 12:24:56 crc kubenswrapper[5052]: I1203 12:24:56.673814 5052 scope.go:117] "RemoveContainer" containerID="09b4a0072741f60499d24f40cc4ff3406d5d22671ced431363ab865172786300" Dec 03 12:24:56 crc kubenswrapper[5052]: E1203 12:24:56.674072 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 12:24:57 crc kubenswrapper[5052]: I1203 12:24:57.534449 5052 generic.go:334] "Generic (PLEG): container finished" podID="9514eb24-503a-40b8-a339-481f81fd49de" containerID="6879b4a0fe254ab4b3401ca94ab6f5e16f1a55dd48a45c455affc3cd5f8d6685" exitCode=143 Dec 03 12:24:57 crc kubenswrapper[5052]: I1203 12:24:57.534777 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"9514eb24-503a-40b8-a339-481f81fd49de","Type":"ContainerDied","Data":"6879b4a0fe254ab4b3401ca94ab6f5e16f1a55dd48a45c455affc3cd5f8d6685"} Dec 03 12:24:58 crc kubenswrapper[5052]: I1203 12:24:58.890795 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-db-sync-gpqnk"] Dec 03 12:24:58 crc kubenswrapper[5052]: E1203 12:24:58.891176 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="a1ac8f66-94c3-4ac5-b039-fe29314ed99a" containerName="mariadb-account-create-update" Dec 03 12:24:58 crc kubenswrapper[5052]: I1203 12:24:58.891189 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="a1ac8f66-94c3-4ac5-b039-fe29314ed99a" containerName="mariadb-account-create-update" Dec 03 12:24:58 crc kubenswrapper[5052]: E1203 12:24:58.891207 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="027c0ef8-c154-4de5-961e-d822ab32a761" containerName="mariadb-database-create" Dec 03 12:24:58 crc kubenswrapper[5052]: I1203 12:24:58.891213 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="027c0ef8-c154-4de5-961e-d822ab32a761" containerName="mariadb-database-create" Dec 03 12:24:58 crc kubenswrapper[5052]: E1203 12:24:58.891231 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="5bbad3a2-f21b-4ce1-8e37-5cf432d806ef" containerName="mariadb-account-create-update" Dec 03 12:24:58 crc kubenswrapper[5052]: I1203 12:24:58.891238 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="5bbad3a2-f21b-4ce1-8e37-5cf432d806ef" containerName="mariadb-account-create-update" Dec 03 12:24:58 crc kubenswrapper[5052]: E1203 12:24:58.891251 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="47087a5b-7fd2-498e-b9d0-0bc3c6939637" containerName="mariadb-database-create" Dec 03 12:24:58 crc kubenswrapper[5052]: I1203 12:24:58.891257 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="47087a5b-7fd2-498e-b9d0-0bc3c6939637" containerName="mariadb-database-create" Dec 03 12:24:58 crc kubenswrapper[5052]: E1203 12:24:58.891271 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="0160de5f-930d-4766-9302-28c89c2fcf14" containerName="mariadb-database-create" Dec 03 12:24:58 crc kubenswrapper[5052]: I1203 12:24:58.891278 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="0160de5f-930d-4766-9302-28c89c2fcf14" containerName="mariadb-database-create" Dec 03 12:24:58 crc kubenswrapper[5052]: E1203 12:24:58.891290 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="d04ffd07-eb13-41b2-a0ed-058afc5c8d73" containerName="mariadb-account-create-update" Dec 03 12:24:58 crc kubenswrapper[5052]: I1203 12:24:58.891297 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="d04ffd07-eb13-41b2-a0ed-058afc5c8d73" containerName="mariadb-account-create-update" Dec 03 12:24:58 crc kubenswrapper[5052]: I1203 12:24:58.891477 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="a1ac8f66-94c3-4ac5-b039-fe29314ed99a" containerName="mariadb-account-create-update" Dec 03 12:24:58 crc kubenswrapper[5052]: I1203 12:24:58.891489 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="47087a5b-7fd2-498e-b9d0-0bc3c6939637" containerName="mariadb-database-create" Dec 03 12:24:58 crc kubenswrapper[5052]: I1203 12:24:58.891501 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="5bbad3a2-f21b-4ce1-8e37-5cf432d806ef" containerName="mariadb-account-create-update" Dec 03 12:24:58 crc kubenswrapper[5052]: I1203 12:24:58.891551 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="d04ffd07-eb13-41b2-a0ed-058afc5c8d73" containerName="mariadb-account-create-update" Dec 03 12:24:58 crc kubenswrapper[5052]: I1203 12:24:58.891566 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="0160de5f-930d-4766-9302-28c89c2fcf14" containerName="mariadb-database-create" Dec 03 12:24:58 crc kubenswrapper[5052]: I1203 12:24:58.891576 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="027c0ef8-c154-4de5-961e-d822ab32a761" containerName="mariadb-database-create" Dec 03 12:24:58 crc kubenswrapper[5052]: I1203 12:24:58.892165 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-gpqnk" Dec 03 12:24:58 crc kubenswrapper[5052]: I1203 12:24:58.894607 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell0-conductor-scripts" Dec 03 12:24:58 crc kubenswrapper[5052]: I1203 12:24:58.894766 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell0-conductor-config-data" Dec 03 12:24:58 crc kubenswrapper[5052]: I1203 12:24:58.897397 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-nova-dockercfg-scw5l" Dec 03 12:24:58 crc kubenswrapper[5052]: I1203 12:24:58.958725 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-db-sync-gpqnk"] Dec 03 12:24:59 crc kubenswrapper[5052]: I1203 12:24:59.079605 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/3fd1fe87-a439-480b-a74e-854e7003b7a8-scripts\") pod \"nova-cell0-conductor-db-sync-gpqnk\" (UID: \"3fd1fe87-a439-480b-a74e-854e7003b7a8\") " pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-gpqnk" Dec 03 12:24:59 crc kubenswrapper[5052]: I1203 12:24:59.080109 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/3fd1fe87-a439-480b-a74e-854e7003b7a8-config-data\") pod \"nova-cell0-conductor-db-sync-gpqnk\" (UID: \"3fd1fe87-a439-480b-a74e-854e7003b7a8\") " pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-gpqnk" Dec 03 12:24:59 crc kubenswrapper[5052]: I1203 12:24:59.080172 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3fd1fe87-a439-480b-a74e-854e7003b7a8-combined-ca-bundle\") pod \"nova-cell0-conductor-db-sync-gpqnk\" (UID: \"3fd1fe87-a439-480b-a74e-854e7003b7a8\") " pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-gpqnk" Dec 03 12:24:59 crc kubenswrapper[5052]: I1203 12:24:59.080209 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-97fsm\" (UniqueName: \"kubernetes.io/projected/3fd1fe87-a439-480b-a74e-854e7003b7a8-kube-api-access-97fsm\") pod \"nova-cell0-conductor-db-sync-gpqnk\" (UID: \"3fd1fe87-a439-480b-a74e-854e7003b7a8\") " pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-gpqnk" Dec 03 12:24:59 crc kubenswrapper[5052]: I1203 12:24:59.182355 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/3fd1fe87-a439-480b-a74e-854e7003b7a8-scripts\") pod \"nova-cell0-conductor-db-sync-gpqnk\" (UID: \"3fd1fe87-a439-480b-a74e-854e7003b7a8\") " pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-gpqnk" Dec 03 12:24:59 crc kubenswrapper[5052]: I1203 12:24:59.182525 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/3fd1fe87-a439-480b-a74e-854e7003b7a8-config-data\") pod \"nova-cell0-conductor-db-sync-gpqnk\" (UID: \"3fd1fe87-a439-480b-a74e-854e7003b7a8\") " pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-gpqnk" Dec 03 12:24:59 crc kubenswrapper[5052]: I1203 12:24:59.182575 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3fd1fe87-a439-480b-a74e-854e7003b7a8-combined-ca-bundle\") pod \"nova-cell0-conductor-db-sync-gpqnk\" (UID: \"3fd1fe87-a439-480b-a74e-854e7003b7a8\") " pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-gpqnk" Dec 03 12:24:59 crc kubenswrapper[5052]: I1203 12:24:59.182598 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-97fsm\" (UniqueName: \"kubernetes.io/projected/3fd1fe87-a439-480b-a74e-854e7003b7a8-kube-api-access-97fsm\") pod \"nova-cell0-conductor-db-sync-gpqnk\" (UID: \"3fd1fe87-a439-480b-a74e-854e7003b7a8\") " pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-gpqnk" Dec 03 12:24:59 crc kubenswrapper[5052]: I1203 12:24:59.191305 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/3fd1fe87-a439-480b-a74e-854e7003b7a8-scripts\") pod \"nova-cell0-conductor-db-sync-gpqnk\" (UID: \"3fd1fe87-a439-480b-a74e-854e7003b7a8\") " pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-gpqnk" Dec 03 12:24:59 crc kubenswrapper[5052]: I1203 12:24:59.191920 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3fd1fe87-a439-480b-a74e-854e7003b7a8-combined-ca-bundle\") pod \"nova-cell0-conductor-db-sync-gpqnk\" (UID: \"3fd1fe87-a439-480b-a74e-854e7003b7a8\") " pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-gpqnk" Dec 03 12:24:59 crc kubenswrapper[5052]: I1203 12:24:59.201228 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/3fd1fe87-a439-480b-a74e-854e7003b7a8-config-data\") pod \"nova-cell0-conductor-db-sync-gpqnk\" (UID: \"3fd1fe87-a439-480b-a74e-854e7003b7a8\") " pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-gpqnk" Dec 03 12:24:59 crc kubenswrapper[5052]: I1203 12:24:59.208442 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-97fsm\" (UniqueName: \"kubernetes.io/projected/3fd1fe87-a439-480b-a74e-854e7003b7a8-kube-api-access-97fsm\") pod \"nova-cell0-conductor-db-sync-gpqnk\" (UID: \"3fd1fe87-a439-480b-a74e-854e7003b7a8\") " pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-gpqnk" Dec 03 12:24:59 crc kubenswrapper[5052]: I1203 12:24:59.212134 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-gpqnk" Dec 03 12:24:59 crc kubenswrapper[5052]: I1203 12:24:59.788712 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-db-sync-gpqnk"] Dec 03 12:24:59 crc kubenswrapper[5052]: W1203 12:24:59.844802 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3fd1fe87_a439_480b_a74e_854e7003b7a8.slice/crio-92e51273fbb2edfb5b9f691c10ca8f08a006f9c619b73a63bc0642b7d915ed51 WatchSource:0}: Error finding container 92e51273fbb2edfb5b9f691c10ca8f08a006f9c619b73a63bc0642b7d915ed51: Status 404 returned error can't find the container with id 92e51273fbb2edfb5b9f691c10ca8f08a006f9c619b73a63bc0642b7d915ed51 Dec 03 12:25:00 crc kubenswrapper[5052]: I1203 12:25:00.496443 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:25:00 crc kubenswrapper[5052]: I1203 12:25:00.524282 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9514eb24-503a-40b8-a339-481f81fd49de-combined-ca-bundle\") pod \"9514eb24-503a-40b8-a339-481f81fd49de\" (UID: \"9514eb24-503a-40b8-a339-481f81fd49de\") " Dec 03 12:25:00 crc kubenswrapper[5052]: I1203 12:25:00.524379 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/9514eb24-503a-40b8-a339-481f81fd49de-httpd-run\") pod \"9514eb24-503a-40b8-a339-481f81fd49de\" (UID: \"9514eb24-503a-40b8-a339-481f81fd49de\") " Dec 03 12:25:00 crc kubenswrapper[5052]: I1203 12:25:00.524420 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ccpnm\" (UniqueName: \"kubernetes.io/projected/9514eb24-503a-40b8-a339-481f81fd49de-kube-api-access-ccpnm\") pod \"9514eb24-503a-40b8-a339-481f81fd49de\" (UID: \"9514eb24-503a-40b8-a339-481f81fd49de\") " Dec 03 12:25:00 crc kubenswrapper[5052]: I1203 12:25:00.524459 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/9514eb24-503a-40b8-a339-481f81fd49de-scripts\") pod \"9514eb24-503a-40b8-a339-481f81fd49de\" (UID: \"9514eb24-503a-40b8-a339-481f81fd49de\") " Dec 03 12:25:00 crc kubenswrapper[5052]: I1203 12:25:00.524483 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/9514eb24-503a-40b8-a339-481f81fd49de-config-data\") pod \"9514eb24-503a-40b8-a339-481f81fd49de\" (UID: \"9514eb24-503a-40b8-a339-481f81fd49de\") " Dec 03 12:25:00 crc kubenswrapper[5052]: I1203 12:25:00.524503 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/9514eb24-503a-40b8-a339-481f81fd49de-logs\") pod \"9514eb24-503a-40b8-a339-481f81fd49de\" (UID: \"9514eb24-503a-40b8-a339-481f81fd49de\") " Dec 03 12:25:00 crc kubenswrapper[5052]: I1203 12:25:00.524553 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"glance\" (UniqueName: \"kubernetes.io/local-volume/local-storage09-crc\") pod \"9514eb24-503a-40b8-a339-481f81fd49de\" (UID: \"9514eb24-503a-40b8-a339-481f81fd49de\") " Dec 03 12:25:00 crc kubenswrapper[5052]: I1203 12:25:00.525404 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9514eb24-503a-40b8-a339-481f81fd49de-logs" (OuterVolumeSpecName: "logs") pod "9514eb24-503a-40b8-a339-481f81fd49de" (UID: "9514eb24-503a-40b8-a339-481f81fd49de"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:25:00 crc kubenswrapper[5052]: I1203 12:25:00.525398 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9514eb24-503a-40b8-a339-481f81fd49de-httpd-run" (OuterVolumeSpecName: "httpd-run") pod "9514eb24-503a-40b8-a339-481f81fd49de" (UID: "9514eb24-503a-40b8-a339-481f81fd49de"). InnerVolumeSpecName "httpd-run". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:25:00 crc kubenswrapper[5052]: I1203 12:25:00.529123 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9514eb24-503a-40b8-a339-481f81fd49de-kube-api-access-ccpnm" (OuterVolumeSpecName: "kube-api-access-ccpnm") pod "9514eb24-503a-40b8-a339-481f81fd49de" (UID: "9514eb24-503a-40b8-a339-481f81fd49de"). InnerVolumeSpecName "kube-api-access-ccpnm". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:25:00 crc kubenswrapper[5052]: I1203 12:25:00.533843 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9514eb24-503a-40b8-a339-481f81fd49de-scripts" (OuterVolumeSpecName: "scripts") pod "9514eb24-503a-40b8-a339-481f81fd49de" (UID: "9514eb24-503a-40b8-a339-481f81fd49de"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:25:00 crc kubenswrapper[5052]: I1203 12:25:00.542469 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage09-crc" (OuterVolumeSpecName: "glance") pod "9514eb24-503a-40b8-a339-481f81fd49de" (UID: "9514eb24-503a-40b8-a339-481f81fd49de"). InnerVolumeSpecName "local-storage09-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 12:25:00 crc kubenswrapper[5052]: I1203 12:25:00.576594 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9514eb24-503a-40b8-a339-481f81fd49de-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "9514eb24-503a-40b8-a339-481f81fd49de" (UID: "9514eb24-503a-40b8-a339-481f81fd49de"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:25:00 crc kubenswrapper[5052]: I1203 12:25:00.617068 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-gpqnk" event={"ID":"3fd1fe87-a439-480b-a74e-854e7003b7a8","Type":"ContainerStarted","Data":"4da5e593296a628772a4119fc20ad8f06ce5e05f848f19bd5f817b1a9b9fb997"} Dec 03 12:25:00 crc kubenswrapper[5052]: I1203 12:25:00.617381 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-gpqnk" event={"ID":"3fd1fe87-a439-480b-a74e-854e7003b7a8","Type":"ContainerStarted","Data":"92e51273fbb2edfb5b9f691c10ca8f08a006f9c619b73a63bc0642b7d915ed51"} Dec 03 12:25:00 crc kubenswrapper[5052]: I1203 12:25:00.627351 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9514eb24-503a-40b8-a339-481f81fd49de-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:25:00 crc kubenswrapper[5052]: I1203 12:25:00.627387 5052 reconciler_common.go:293] "Volume detached for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/9514eb24-503a-40b8-a339-481f81fd49de-httpd-run\") on node \"crc\" DevicePath \"\"" Dec 03 12:25:00 crc kubenswrapper[5052]: I1203 12:25:00.627398 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-ccpnm\" (UniqueName: \"kubernetes.io/projected/9514eb24-503a-40b8-a339-481f81fd49de-kube-api-access-ccpnm\") on node \"crc\" DevicePath \"\"" Dec 03 12:25:00 crc kubenswrapper[5052]: I1203 12:25:00.627408 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/9514eb24-503a-40b8-a339-481f81fd49de-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:25:00 crc kubenswrapper[5052]: I1203 12:25:00.627419 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/9514eb24-503a-40b8-a339-481f81fd49de-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:25:00 crc kubenswrapper[5052]: I1203 12:25:00.627444 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage09-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage09-crc\") on node \"crc\" " Dec 03 12:25:00 crc kubenswrapper[5052]: I1203 12:25:00.631704 5052 generic.go:334] "Generic (PLEG): container finished" podID="9514eb24-503a-40b8-a339-481f81fd49de" containerID="f6dbc6deaf8843ea5432f48c4bfb01f18f24c61b85c39d693d7ae58e5df45acf" exitCode=0 Dec 03 12:25:00 crc kubenswrapper[5052]: I1203 12:25:00.631760 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"9514eb24-503a-40b8-a339-481f81fd49de","Type":"ContainerDied","Data":"f6dbc6deaf8843ea5432f48c4bfb01f18f24c61b85c39d693d7ae58e5df45acf"} Dec 03 12:25:00 crc kubenswrapper[5052]: I1203 12:25:00.631792 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"9514eb24-503a-40b8-a339-481f81fd49de","Type":"ContainerDied","Data":"4cb0e3dd05921531ebc08c66cb79294572f5d4e8e2b5f2717b30f817250583e2"} Dec 03 12:25:00 crc kubenswrapper[5052]: I1203 12:25:00.631810 5052 scope.go:117] "RemoveContainer" containerID="f6dbc6deaf8843ea5432f48c4bfb01f18f24c61b85c39d693d7ae58e5df45acf" Dec 03 12:25:00 crc kubenswrapper[5052]: I1203 12:25:00.632007 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:25:00 crc kubenswrapper[5052]: I1203 12:25:00.639773 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9514eb24-503a-40b8-a339-481f81fd49de-config-data" (OuterVolumeSpecName: "config-data") pod "9514eb24-503a-40b8-a339-481f81fd49de" (UID: "9514eb24-503a-40b8-a339-481f81fd49de"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:25:00 crc kubenswrapper[5052]: I1203 12:25:00.661477 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-gpqnk" podStartSLOduration=2.661453406 podStartE2EDuration="2.661453406s" podCreationTimestamp="2025-12-03 12:24:58 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:25:00.651876523 +0000 UTC m=+3891.240118287" watchObservedRunningTime="2025-12-03 12:25:00.661453406 +0000 UTC m=+3891.249695170" Dec 03 12:25:00 crc kubenswrapper[5052]: I1203 12:25:00.671149 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage09-crc" (UniqueName: "kubernetes.io/local-volume/local-storage09-crc") on node "crc" Dec 03 12:25:00 crc kubenswrapper[5052]: I1203 12:25:00.682656 5052 scope.go:117] "RemoveContainer" containerID="6879b4a0fe254ab4b3401ca94ab6f5e16f1a55dd48a45c455affc3cd5f8d6685" Dec 03 12:25:00 crc kubenswrapper[5052]: I1203 12:25:00.705579 5052 scope.go:117] "RemoveContainer" containerID="f6dbc6deaf8843ea5432f48c4bfb01f18f24c61b85c39d693d7ae58e5df45acf" Dec 03 12:25:00 crc kubenswrapper[5052]: E1203 12:25:00.706048 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f6dbc6deaf8843ea5432f48c4bfb01f18f24c61b85c39d693d7ae58e5df45acf\": container with ID starting with f6dbc6deaf8843ea5432f48c4bfb01f18f24c61b85c39d693d7ae58e5df45acf not found: ID does not exist" containerID="f6dbc6deaf8843ea5432f48c4bfb01f18f24c61b85c39d693d7ae58e5df45acf" Dec 03 12:25:00 crc kubenswrapper[5052]: I1203 12:25:00.706085 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f6dbc6deaf8843ea5432f48c4bfb01f18f24c61b85c39d693d7ae58e5df45acf"} err="failed to get container status \"f6dbc6deaf8843ea5432f48c4bfb01f18f24c61b85c39d693d7ae58e5df45acf\": rpc error: code = NotFound desc = could not find container \"f6dbc6deaf8843ea5432f48c4bfb01f18f24c61b85c39d693d7ae58e5df45acf\": container with ID starting with f6dbc6deaf8843ea5432f48c4bfb01f18f24c61b85c39d693d7ae58e5df45acf not found: ID does not exist" Dec 03 12:25:00 crc kubenswrapper[5052]: I1203 12:25:00.706113 5052 scope.go:117] "RemoveContainer" containerID="6879b4a0fe254ab4b3401ca94ab6f5e16f1a55dd48a45c455affc3cd5f8d6685" Dec 03 12:25:00 crc kubenswrapper[5052]: E1203 12:25:00.706327 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6879b4a0fe254ab4b3401ca94ab6f5e16f1a55dd48a45c455affc3cd5f8d6685\": container with ID starting with 6879b4a0fe254ab4b3401ca94ab6f5e16f1a55dd48a45c455affc3cd5f8d6685 not found: ID does not exist" containerID="6879b4a0fe254ab4b3401ca94ab6f5e16f1a55dd48a45c455affc3cd5f8d6685" Dec 03 12:25:00 crc kubenswrapper[5052]: I1203 12:25:00.706351 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6879b4a0fe254ab4b3401ca94ab6f5e16f1a55dd48a45c455affc3cd5f8d6685"} err="failed to get container status \"6879b4a0fe254ab4b3401ca94ab6f5e16f1a55dd48a45c455affc3cd5f8d6685\": rpc error: code = NotFound desc = could not find container \"6879b4a0fe254ab4b3401ca94ab6f5e16f1a55dd48a45c455affc3cd5f8d6685\": container with ID starting with 6879b4a0fe254ab4b3401ca94ab6f5e16f1a55dd48a45c455affc3cd5f8d6685 not found: ID does not exist" Dec 03 12:25:00 crc kubenswrapper[5052]: I1203 12:25:00.729919 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/9514eb24-503a-40b8-a339-481f81fd49de-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:25:00 crc kubenswrapper[5052]: I1203 12:25:00.729950 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage09-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage09-crc\") on node \"crc\" DevicePath \"\"" Dec 03 12:25:00 crc kubenswrapper[5052]: I1203 12:25:00.977681 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:25:00 crc kubenswrapper[5052]: I1203 12:25:00.999818 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:25:01 crc kubenswrapper[5052]: I1203 12:25:01.010901 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:25:01 crc kubenswrapper[5052]: E1203 12:25:01.011438 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="9514eb24-503a-40b8-a339-481f81fd49de" containerName="glance-log" Dec 03 12:25:01 crc kubenswrapper[5052]: I1203 12:25:01.011466 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="9514eb24-503a-40b8-a339-481f81fd49de" containerName="glance-log" Dec 03 12:25:01 crc kubenswrapper[5052]: E1203 12:25:01.011497 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="9514eb24-503a-40b8-a339-481f81fd49de" containerName="glance-httpd" Dec 03 12:25:01 crc kubenswrapper[5052]: I1203 12:25:01.011504 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="9514eb24-503a-40b8-a339-481f81fd49de" containerName="glance-httpd" Dec 03 12:25:01 crc kubenswrapper[5052]: I1203 12:25:01.011729 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="9514eb24-503a-40b8-a339-481f81fd49de" containerName="glance-httpd" Dec 03 12:25:01 crc kubenswrapper[5052]: I1203 12:25:01.011755 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="9514eb24-503a-40b8-a339-481f81fd49de" containerName="glance-log" Dec 03 12:25:01 crc kubenswrapper[5052]: I1203 12:25:01.012907 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:25:01 crc kubenswrapper[5052]: I1203 12:25:01.017398 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-default-internal-config-data" Dec 03 12:25:01 crc kubenswrapper[5052]: I1203 12:25:01.019986 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:25:01 crc kubenswrapper[5052]: I1203 12:25:01.138435 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vnz6l\" (UniqueName: \"kubernetes.io/projected/7fa6aac1-6ef3-434a-945e-3ced02ca5361-kube-api-access-vnz6l\") pod \"glance-default-internal-api-0\" (UID: \"7fa6aac1-6ef3-434a-945e-3ced02ca5361\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:25:01 crc kubenswrapper[5052]: I1203 12:25:01.139057 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/7fa6aac1-6ef3-434a-945e-3ced02ca5361-httpd-run\") pod \"glance-default-internal-api-0\" (UID: \"7fa6aac1-6ef3-434a-945e-3ced02ca5361\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:25:01 crc kubenswrapper[5052]: I1203 12:25:01.139189 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/7fa6aac1-6ef3-434a-945e-3ced02ca5361-combined-ca-bundle\") pod \"glance-default-internal-api-0\" (UID: \"7fa6aac1-6ef3-434a-945e-3ced02ca5361\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:25:01 crc kubenswrapper[5052]: I1203 12:25:01.139345 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/7fa6aac1-6ef3-434a-945e-3ced02ca5361-logs\") pod \"glance-default-internal-api-0\" (UID: \"7fa6aac1-6ef3-434a-945e-3ced02ca5361\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:25:01 crc kubenswrapper[5052]: I1203 12:25:01.139498 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage09-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage09-crc\") pod \"glance-default-internal-api-0\" (UID: \"7fa6aac1-6ef3-434a-945e-3ced02ca5361\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:25:01 crc kubenswrapper[5052]: I1203 12:25:01.139678 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/7fa6aac1-6ef3-434a-945e-3ced02ca5361-config-data\") pod \"glance-default-internal-api-0\" (UID: \"7fa6aac1-6ef3-434a-945e-3ced02ca5361\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:25:01 crc kubenswrapper[5052]: I1203 12:25:01.139792 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/7fa6aac1-6ef3-434a-945e-3ced02ca5361-scripts\") pod \"glance-default-internal-api-0\" (UID: \"7fa6aac1-6ef3-434a-945e-3ced02ca5361\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:25:01 crc kubenswrapper[5052]: I1203 12:25:01.241027 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/7fa6aac1-6ef3-434a-945e-3ced02ca5361-logs\") pod \"glance-default-internal-api-0\" (UID: \"7fa6aac1-6ef3-434a-945e-3ced02ca5361\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:25:01 crc kubenswrapper[5052]: I1203 12:25:01.241358 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage09-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage09-crc\") pod \"glance-default-internal-api-0\" (UID: \"7fa6aac1-6ef3-434a-945e-3ced02ca5361\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:25:01 crc kubenswrapper[5052]: I1203 12:25:01.241593 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/7fa6aac1-6ef3-434a-945e-3ced02ca5361-config-data\") pod \"glance-default-internal-api-0\" (UID: \"7fa6aac1-6ef3-434a-945e-3ced02ca5361\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:25:01 crc kubenswrapper[5052]: I1203 12:25:01.241706 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage09-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage09-crc\") pod \"glance-default-internal-api-0\" (UID: \"7fa6aac1-6ef3-434a-945e-3ced02ca5361\") device mount path \"/mnt/openstack/pv09\"" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:25:01 crc kubenswrapper[5052]: I1203 12:25:01.241617 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/7fa6aac1-6ef3-434a-945e-3ced02ca5361-logs\") pod \"glance-default-internal-api-0\" (UID: \"7fa6aac1-6ef3-434a-945e-3ced02ca5361\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:25:01 crc kubenswrapper[5052]: I1203 12:25:01.241721 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/7fa6aac1-6ef3-434a-945e-3ced02ca5361-scripts\") pod \"glance-default-internal-api-0\" (UID: \"7fa6aac1-6ef3-434a-945e-3ced02ca5361\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:25:01 crc kubenswrapper[5052]: I1203 12:25:01.242051 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-vnz6l\" (UniqueName: \"kubernetes.io/projected/7fa6aac1-6ef3-434a-945e-3ced02ca5361-kube-api-access-vnz6l\") pod \"glance-default-internal-api-0\" (UID: \"7fa6aac1-6ef3-434a-945e-3ced02ca5361\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:25:01 crc kubenswrapper[5052]: I1203 12:25:01.242122 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/7fa6aac1-6ef3-434a-945e-3ced02ca5361-httpd-run\") pod \"glance-default-internal-api-0\" (UID: \"7fa6aac1-6ef3-434a-945e-3ced02ca5361\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:25:01 crc kubenswrapper[5052]: I1203 12:25:01.242164 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/7fa6aac1-6ef3-434a-945e-3ced02ca5361-combined-ca-bundle\") pod \"glance-default-internal-api-0\" (UID: \"7fa6aac1-6ef3-434a-945e-3ced02ca5361\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:25:01 crc kubenswrapper[5052]: I1203 12:25:01.242648 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/7fa6aac1-6ef3-434a-945e-3ced02ca5361-httpd-run\") pod \"glance-default-internal-api-0\" (UID: \"7fa6aac1-6ef3-434a-945e-3ced02ca5361\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:25:01 crc kubenswrapper[5052]: I1203 12:25:01.247191 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/7fa6aac1-6ef3-434a-945e-3ced02ca5361-scripts\") pod \"glance-default-internal-api-0\" (UID: \"7fa6aac1-6ef3-434a-945e-3ced02ca5361\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:25:01 crc kubenswrapper[5052]: I1203 12:25:01.247425 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/7fa6aac1-6ef3-434a-945e-3ced02ca5361-combined-ca-bundle\") pod \"glance-default-internal-api-0\" (UID: \"7fa6aac1-6ef3-434a-945e-3ced02ca5361\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:25:01 crc kubenswrapper[5052]: I1203 12:25:01.248184 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/7fa6aac1-6ef3-434a-945e-3ced02ca5361-config-data\") pod \"glance-default-internal-api-0\" (UID: \"7fa6aac1-6ef3-434a-945e-3ced02ca5361\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:25:01 crc kubenswrapper[5052]: I1203 12:25:01.268389 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-vnz6l\" (UniqueName: \"kubernetes.io/projected/7fa6aac1-6ef3-434a-945e-3ced02ca5361-kube-api-access-vnz6l\") pod \"glance-default-internal-api-0\" (UID: \"7fa6aac1-6ef3-434a-945e-3ced02ca5361\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:25:01 crc kubenswrapper[5052]: I1203 12:25:01.270949 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage09-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage09-crc\") pod \"glance-default-internal-api-0\" (UID: \"7fa6aac1-6ef3-434a-945e-3ced02ca5361\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:25:01 crc kubenswrapper[5052]: I1203 12:25:01.336987 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:25:01 crc kubenswrapper[5052]: I1203 12:25:01.686503 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9514eb24-503a-40b8-a339-481f81fd49de" path="/var/lib/kubelet/pods/9514eb24-503a-40b8-a339-481f81fd49de/volumes" Dec 03 12:25:01 crc kubenswrapper[5052]: I1203 12:25:01.906334 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:25:01 crc kubenswrapper[5052]: W1203 12:25:01.909058 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7fa6aac1_6ef3_434a_945e_3ced02ca5361.slice/crio-8dce4ce3a11724b3a7decc671b33ed571677104363149336cd40027e0c72fe64 WatchSource:0}: Error finding container 8dce4ce3a11724b3a7decc671b33ed571677104363149336cd40027e0c72fe64: Status 404 returned error can't find the container with id 8dce4ce3a11724b3a7decc671b33ed571677104363149336cd40027e0c72fe64 Dec 03 12:25:02 crc kubenswrapper[5052]: I1203 12:25:02.680698 5052 generic.go:334] "Generic (PLEG): container finished" podID="877cfc19-61d2-48a8-ba30-2eb8a39a7c0d" containerID="333d5988d6ce193cc6cbf5c47b0c3154146fadf562614a1dddecfa687325ba71" exitCode=0 Dec 03 12:25:02 crc kubenswrapper[5052]: I1203 12:25:02.680795 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"877cfc19-61d2-48a8-ba30-2eb8a39a7c0d","Type":"ContainerDied","Data":"333d5988d6ce193cc6cbf5c47b0c3154146fadf562614a1dddecfa687325ba71"} Dec 03 12:25:02 crc kubenswrapper[5052]: I1203 12:25:02.680872 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"877cfc19-61d2-48a8-ba30-2eb8a39a7c0d","Type":"ContainerDied","Data":"aa433cb8f25271a7777220766a0b0418aa7ae5bc02abce6c6a2edf32dd7abacd"} Dec 03 12:25:02 crc kubenswrapper[5052]: I1203 12:25:02.680887 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="aa433cb8f25271a7777220766a0b0418aa7ae5bc02abce6c6a2edf32dd7abacd" Dec 03 12:25:02 crc kubenswrapper[5052]: I1203 12:25:02.682552 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"7fa6aac1-6ef3-434a-945e-3ced02ca5361","Type":"ContainerStarted","Data":"8dce4ce3a11724b3a7decc671b33ed571677104363149336cd40027e0c72fe64"} Dec 03 12:25:02 crc kubenswrapper[5052]: I1203 12:25:02.695279 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:25:02 crc kubenswrapper[5052]: I1203 12:25:02.878312 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/877cfc19-61d2-48a8-ba30-2eb8a39a7c0d-config-data\") pod \"877cfc19-61d2-48a8-ba30-2eb8a39a7c0d\" (UID: \"877cfc19-61d2-48a8-ba30-2eb8a39a7c0d\") " Dec 03 12:25:02 crc kubenswrapper[5052]: I1203 12:25:02.878742 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/877cfc19-61d2-48a8-ba30-2eb8a39a7c0d-sg-core-conf-yaml\") pod \"877cfc19-61d2-48a8-ba30-2eb8a39a7c0d\" (UID: \"877cfc19-61d2-48a8-ba30-2eb8a39a7c0d\") " Dec 03 12:25:02 crc kubenswrapper[5052]: I1203 12:25:02.878800 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v5m4k\" (UniqueName: \"kubernetes.io/projected/877cfc19-61d2-48a8-ba30-2eb8a39a7c0d-kube-api-access-v5m4k\") pod \"877cfc19-61d2-48a8-ba30-2eb8a39a7c0d\" (UID: \"877cfc19-61d2-48a8-ba30-2eb8a39a7c0d\") " Dec 03 12:25:02 crc kubenswrapper[5052]: I1203 12:25:02.878826 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/877cfc19-61d2-48a8-ba30-2eb8a39a7c0d-combined-ca-bundle\") pod \"877cfc19-61d2-48a8-ba30-2eb8a39a7c0d\" (UID: \"877cfc19-61d2-48a8-ba30-2eb8a39a7c0d\") " Dec 03 12:25:02 crc kubenswrapper[5052]: I1203 12:25:02.878916 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/877cfc19-61d2-48a8-ba30-2eb8a39a7c0d-run-httpd\") pod \"877cfc19-61d2-48a8-ba30-2eb8a39a7c0d\" (UID: \"877cfc19-61d2-48a8-ba30-2eb8a39a7c0d\") " Dec 03 12:25:02 crc kubenswrapper[5052]: I1203 12:25:02.878978 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/877cfc19-61d2-48a8-ba30-2eb8a39a7c0d-log-httpd\") pod \"877cfc19-61d2-48a8-ba30-2eb8a39a7c0d\" (UID: \"877cfc19-61d2-48a8-ba30-2eb8a39a7c0d\") " Dec 03 12:25:02 crc kubenswrapper[5052]: I1203 12:25:02.879023 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/877cfc19-61d2-48a8-ba30-2eb8a39a7c0d-scripts\") pod \"877cfc19-61d2-48a8-ba30-2eb8a39a7c0d\" (UID: \"877cfc19-61d2-48a8-ba30-2eb8a39a7c0d\") " Dec 03 12:25:02 crc kubenswrapper[5052]: I1203 12:25:02.879402 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/877cfc19-61d2-48a8-ba30-2eb8a39a7c0d-run-httpd" (OuterVolumeSpecName: "run-httpd") pod "877cfc19-61d2-48a8-ba30-2eb8a39a7c0d" (UID: "877cfc19-61d2-48a8-ba30-2eb8a39a7c0d"). InnerVolumeSpecName "run-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:25:02 crc kubenswrapper[5052]: I1203 12:25:02.880121 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/877cfc19-61d2-48a8-ba30-2eb8a39a7c0d-log-httpd" (OuterVolumeSpecName: "log-httpd") pod "877cfc19-61d2-48a8-ba30-2eb8a39a7c0d" (UID: "877cfc19-61d2-48a8-ba30-2eb8a39a7c0d"). InnerVolumeSpecName "log-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:25:02 crc kubenswrapper[5052]: I1203 12:25:02.885829 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/877cfc19-61d2-48a8-ba30-2eb8a39a7c0d-kube-api-access-v5m4k" (OuterVolumeSpecName: "kube-api-access-v5m4k") pod "877cfc19-61d2-48a8-ba30-2eb8a39a7c0d" (UID: "877cfc19-61d2-48a8-ba30-2eb8a39a7c0d"). InnerVolumeSpecName "kube-api-access-v5m4k". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:25:02 crc kubenswrapper[5052]: I1203 12:25:02.898222 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/877cfc19-61d2-48a8-ba30-2eb8a39a7c0d-scripts" (OuterVolumeSpecName: "scripts") pod "877cfc19-61d2-48a8-ba30-2eb8a39a7c0d" (UID: "877cfc19-61d2-48a8-ba30-2eb8a39a7c0d"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:25:02 crc kubenswrapper[5052]: I1203 12:25:02.908574 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/877cfc19-61d2-48a8-ba30-2eb8a39a7c0d-sg-core-conf-yaml" (OuterVolumeSpecName: "sg-core-conf-yaml") pod "877cfc19-61d2-48a8-ba30-2eb8a39a7c0d" (UID: "877cfc19-61d2-48a8-ba30-2eb8a39a7c0d"). InnerVolumeSpecName "sg-core-conf-yaml". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:25:02 crc kubenswrapper[5052]: I1203 12:25:02.973297 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/877cfc19-61d2-48a8-ba30-2eb8a39a7c0d-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "877cfc19-61d2-48a8-ba30-2eb8a39a7c0d" (UID: "877cfc19-61d2-48a8-ba30-2eb8a39a7c0d"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:25:02 crc kubenswrapper[5052]: I1203 12:25:02.982825 5052 reconciler_common.go:293] "Volume detached for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/877cfc19-61d2-48a8-ba30-2eb8a39a7c0d-run-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:25:02 crc kubenswrapper[5052]: I1203 12:25:02.982900 5052 reconciler_common.go:293] "Volume detached for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/877cfc19-61d2-48a8-ba30-2eb8a39a7c0d-log-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:25:02 crc kubenswrapper[5052]: I1203 12:25:02.982911 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/877cfc19-61d2-48a8-ba30-2eb8a39a7c0d-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:25:02 crc kubenswrapper[5052]: I1203 12:25:02.982921 5052 reconciler_common.go:293] "Volume detached for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/877cfc19-61d2-48a8-ba30-2eb8a39a7c0d-sg-core-conf-yaml\") on node \"crc\" DevicePath \"\"" Dec 03 12:25:02 crc kubenswrapper[5052]: I1203 12:25:02.982938 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-v5m4k\" (UniqueName: \"kubernetes.io/projected/877cfc19-61d2-48a8-ba30-2eb8a39a7c0d-kube-api-access-v5m4k\") on node \"crc\" DevicePath \"\"" Dec 03 12:25:02 crc kubenswrapper[5052]: I1203 12:25:02.982952 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/877cfc19-61d2-48a8-ba30-2eb8a39a7c0d-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:25:03 crc kubenswrapper[5052]: I1203 12:25:03.032351 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/877cfc19-61d2-48a8-ba30-2eb8a39a7c0d-config-data" (OuterVolumeSpecName: "config-data") pod "877cfc19-61d2-48a8-ba30-2eb8a39a7c0d" (UID: "877cfc19-61d2-48a8-ba30-2eb8a39a7c0d"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:25:03 crc kubenswrapper[5052]: I1203 12:25:03.084489 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/877cfc19-61d2-48a8-ba30-2eb8a39a7c0d-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:25:03 crc kubenswrapper[5052]: I1203 12:25:03.693658 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"7fa6aac1-6ef3-434a-945e-3ced02ca5361","Type":"ContainerStarted","Data":"d09526b701062188ac0714b7d2d4c1986577ed904704b893ec88c6d251aa69cb"} Dec 03 12:25:03 crc kubenswrapper[5052]: I1203 12:25:03.693993 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"7fa6aac1-6ef3-434a-945e-3ced02ca5361","Type":"ContainerStarted","Data":"f4fbadcae236c9bdee4f357a1121e0e675398d9aeeaeec64e67d0177ab9e352c"} Dec 03 12:25:03 crc kubenswrapper[5052]: I1203 12:25:03.693702 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:25:03 crc kubenswrapper[5052]: I1203 12:25:03.721680 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:25:03 crc kubenswrapper[5052]: I1203 12:25:03.729476 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:25:03 crc kubenswrapper[5052]: I1203 12:25:03.749910 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:25:03 crc kubenswrapper[5052]: E1203 12:25:03.750285 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="877cfc19-61d2-48a8-ba30-2eb8a39a7c0d" containerName="ceilometer-central-agent" Dec 03 12:25:03 crc kubenswrapper[5052]: I1203 12:25:03.750300 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="877cfc19-61d2-48a8-ba30-2eb8a39a7c0d" containerName="ceilometer-central-agent" Dec 03 12:25:03 crc kubenswrapper[5052]: E1203 12:25:03.750315 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="877cfc19-61d2-48a8-ba30-2eb8a39a7c0d" containerName="proxy-httpd" Dec 03 12:25:03 crc kubenswrapper[5052]: I1203 12:25:03.750323 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="877cfc19-61d2-48a8-ba30-2eb8a39a7c0d" containerName="proxy-httpd" Dec 03 12:25:03 crc kubenswrapper[5052]: E1203 12:25:03.750341 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="877cfc19-61d2-48a8-ba30-2eb8a39a7c0d" containerName="ceilometer-notification-agent" Dec 03 12:25:03 crc kubenswrapper[5052]: I1203 12:25:03.750346 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="877cfc19-61d2-48a8-ba30-2eb8a39a7c0d" containerName="ceilometer-notification-agent" Dec 03 12:25:03 crc kubenswrapper[5052]: E1203 12:25:03.750358 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="877cfc19-61d2-48a8-ba30-2eb8a39a7c0d" containerName="sg-core" Dec 03 12:25:03 crc kubenswrapper[5052]: I1203 12:25:03.750364 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="877cfc19-61d2-48a8-ba30-2eb8a39a7c0d" containerName="sg-core" Dec 03 12:25:03 crc kubenswrapper[5052]: I1203 12:25:03.750550 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="877cfc19-61d2-48a8-ba30-2eb8a39a7c0d" containerName="ceilometer-central-agent" Dec 03 12:25:03 crc kubenswrapper[5052]: I1203 12:25:03.750569 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="877cfc19-61d2-48a8-ba30-2eb8a39a7c0d" containerName="proxy-httpd" Dec 03 12:25:03 crc kubenswrapper[5052]: I1203 12:25:03.750579 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="877cfc19-61d2-48a8-ba30-2eb8a39a7c0d" containerName="ceilometer-notification-agent" Dec 03 12:25:03 crc kubenswrapper[5052]: I1203 12:25:03.750591 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="877cfc19-61d2-48a8-ba30-2eb8a39a7c0d" containerName="sg-core" Dec 03 12:25:03 crc kubenswrapper[5052]: I1203 12:25:03.754706 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:25:03 crc kubenswrapper[5052]: I1203 12:25:03.756995 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-scripts" Dec 03 12:25:03 crc kubenswrapper[5052]: I1203 12:25:03.757445 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-config-data" Dec 03 12:25:03 crc kubenswrapper[5052]: I1203 12:25:03.763624 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:25:03 crc kubenswrapper[5052]: I1203 12:25:03.898118 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/3c6a6536-2df8-4c31-9460-66467ae47afd-scripts\") pod \"ceilometer-0\" (UID: \"3c6a6536-2df8-4c31-9460-66467ae47afd\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:25:03 crc kubenswrapper[5052]: I1203 12:25:03.898312 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7xf85\" (UniqueName: \"kubernetes.io/projected/3c6a6536-2df8-4c31-9460-66467ae47afd-kube-api-access-7xf85\") pod \"ceilometer-0\" (UID: \"3c6a6536-2df8-4c31-9460-66467ae47afd\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:25:03 crc kubenswrapper[5052]: I1203 12:25:03.898396 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/3c6a6536-2df8-4c31-9460-66467ae47afd-log-httpd\") pod \"ceilometer-0\" (UID: \"3c6a6536-2df8-4c31-9460-66467ae47afd\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:25:03 crc kubenswrapper[5052]: I1203 12:25:03.898461 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/3c6a6536-2df8-4c31-9460-66467ae47afd-config-data\") pod \"ceilometer-0\" (UID: \"3c6a6536-2df8-4c31-9460-66467ae47afd\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:25:03 crc kubenswrapper[5052]: I1203 12:25:03.898484 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3c6a6536-2df8-4c31-9460-66467ae47afd-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"3c6a6536-2df8-4c31-9460-66467ae47afd\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:25:03 crc kubenswrapper[5052]: I1203 12:25:03.898573 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/3c6a6536-2df8-4c31-9460-66467ae47afd-run-httpd\") pod \"ceilometer-0\" (UID: \"3c6a6536-2df8-4c31-9460-66467ae47afd\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:25:03 crc kubenswrapper[5052]: I1203 12:25:03.898624 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/3c6a6536-2df8-4c31-9460-66467ae47afd-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"3c6a6536-2df8-4c31-9460-66467ae47afd\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:25:03 crc kubenswrapper[5052]: I1203 12:25:03.999860 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/3c6a6536-2df8-4c31-9460-66467ae47afd-config-data\") pod \"ceilometer-0\" (UID: \"3c6a6536-2df8-4c31-9460-66467ae47afd\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:25:03 crc kubenswrapper[5052]: I1203 12:25:03.999910 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3c6a6536-2df8-4c31-9460-66467ae47afd-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"3c6a6536-2df8-4c31-9460-66467ae47afd\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:25:03 crc kubenswrapper[5052]: I1203 12:25:03.999932 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/3c6a6536-2df8-4c31-9460-66467ae47afd-run-httpd\") pod \"ceilometer-0\" (UID: \"3c6a6536-2df8-4c31-9460-66467ae47afd\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:25:04 crc kubenswrapper[5052]: I1203 12:25:03.999956 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/3c6a6536-2df8-4c31-9460-66467ae47afd-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"3c6a6536-2df8-4c31-9460-66467ae47afd\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:25:04 crc kubenswrapper[5052]: I1203 12:25:03.999980 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/3c6a6536-2df8-4c31-9460-66467ae47afd-scripts\") pod \"ceilometer-0\" (UID: \"3c6a6536-2df8-4c31-9460-66467ae47afd\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:25:04 crc kubenswrapper[5052]: I1203 12:25:04.000422 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/3c6a6536-2df8-4c31-9460-66467ae47afd-run-httpd\") pod \"ceilometer-0\" (UID: \"3c6a6536-2df8-4c31-9460-66467ae47afd\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:25:04 crc kubenswrapper[5052]: I1203 12:25:04.000979 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-7xf85\" (UniqueName: \"kubernetes.io/projected/3c6a6536-2df8-4c31-9460-66467ae47afd-kube-api-access-7xf85\") pod \"ceilometer-0\" (UID: \"3c6a6536-2df8-4c31-9460-66467ae47afd\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:25:04 crc kubenswrapper[5052]: I1203 12:25:04.001060 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/3c6a6536-2df8-4c31-9460-66467ae47afd-log-httpd\") pod \"ceilometer-0\" (UID: \"3c6a6536-2df8-4c31-9460-66467ae47afd\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:25:04 crc kubenswrapper[5052]: I1203 12:25:04.001322 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/3c6a6536-2df8-4c31-9460-66467ae47afd-log-httpd\") pod \"ceilometer-0\" (UID: \"3c6a6536-2df8-4c31-9460-66467ae47afd\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:25:04 crc kubenswrapper[5052]: I1203 12:25:04.005189 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/3c6a6536-2df8-4c31-9460-66467ae47afd-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"3c6a6536-2df8-4c31-9460-66467ae47afd\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:25:04 crc kubenswrapper[5052]: I1203 12:25:04.005914 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/3c6a6536-2df8-4c31-9460-66467ae47afd-config-data\") pod \"ceilometer-0\" (UID: \"3c6a6536-2df8-4c31-9460-66467ae47afd\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:25:04 crc kubenswrapper[5052]: I1203 12:25:04.005974 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/3c6a6536-2df8-4c31-9460-66467ae47afd-scripts\") pod \"ceilometer-0\" (UID: \"3c6a6536-2df8-4c31-9460-66467ae47afd\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:25:04 crc kubenswrapper[5052]: I1203 12:25:04.012503 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3c6a6536-2df8-4c31-9460-66467ae47afd-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"3c6a6536-2df8-4c31-9460-66467ae47afd\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:25:04 crc kubenswrapper[5052]: I1203 12:25:04.016231 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-7xf85\" (UniqueName: \"kubernetes.io/projected/3c6a6536-2df8-4c31-9460-66467ae47afd-kube-api-access-7xf85\") pod \"ceilometer-0\" (UID: \"3c6a6536-2df8-4c31-9460-66467ae47afd\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:25:04 crc kubenswrapper[5052]: I1203 12:25:04.073772 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:25:04 crc kubenswrapper[5052]: I1203 12:25:04.513552 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:25:04 crc kubenswrapper[5052]: I1203 12:25:04.705487 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"3c6a6536-2df8-4c31-9460-66467ae47afd","Type":"ContainerStarted","Data":"75dbc76a6a0669f96a333b47d5d898e1ddfe8602f97374071c03fb878c4d75dd"} Dec 03 12:25:05 crc kubenswrapper[5052]: I1203 12:25:05.685562 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="877cfc19-61d2-48a8-ba30-2eb8a39a7c0d" path="/var/lib/kubelet/pods/877cfc19-61d2-48a8-ba30-2eb8a39a7c0d/volumes" Dec 03 12:25:05 crc kubenswrapper[5052]: I1203 12:25:05.740255 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/glance-default-internal-api-0" podStartSLOduration=5.740230325 podStartE2EDuration="5.740230325s" podCreationTimestamp="2025-12-03 12:25:00 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:25:05.73690976 +0000 UTC m=+3896.325151534" watchObservedRunningTime="2025-12-03 12:25:05.740230325 +0000 UTC m=+3896.328472109" Dec 03 12:25:06 crc kubenswrapper[5052]: I1203 12:25:06.216908 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:25:06 crc kubenswrapper[5052]: I1203 12:25:06.735389 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"3c6a6536-2df8-4c31-9460-66467ae47afd","Type":"ContainerStarted","Data":"c3fbc5ff98507543ea52685a6fc326e382fb78b82083f662da673f4a6c84fe96"} Dec 03 12:25:07 crc kubenswrapper[5052]: I1203 12:25:07.748996 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:25:07 crc kubenswrapper[5052]: I1203 12:25:07.750337 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-external-api-0" podUID="2bada3d1-568e-484f-b848-532b9d805f17" containerName="glance-log" containerID="cri-o://65548fd95f21e38842666bf62b603e65a09c2d7f7fef257d2803d9be9f1bcd15" gracePeriod=30 Dec 03 12:25:07 crc kubenswrapper[5052]: I1203 12:25:07.751013 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-external-api-0" podUID="2bada3d1-568e-484f-b848-532b9d805f17" containerName="glance-httpd" containerID="cri-o://3bd600fb62ffa6746a30412a469e6249947d48035142d1cdf479776822f1807b" gracePeriod=30 Dec 03 12:25:07 crc kubenswrapper[5052]: I1203 12:25:07.758835 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"3c6a6536-2df8-4c31-9460-66467ae47afd","Type":"ContainerStarted","Data":"0d65a1099ad7c395baf34b9e1d6eb93e85150cfbdccd9fbdb9a5168272a8b78e"} Dec 03 12:25:07 crc kubenswrapper[5052]: I1203 12:25:07.777316 5052 scope.go:117] "RemoveContainer" containerID="6c1ba05ae9fa6187bf00e4c104305fd60275127701d6a185a2709d3cdff50ab1" Dec 03 12:25:07 crc kubenswrapper[5052]: I1203 12:25:07.882681 5052 scope.go:117] "RemoveContainer" containerID="3a2ca9d08031e2ebddb639a9dacfa6094d35ce57e118c4a4e882deff84684149" Dec 03 12:25:07 crc kubenswrapper[5052]: I1203 12:25:07.940268 5052 scope.go:117] "RemoveContainer" containerID="3ef54de746a038fe4206f2d3e720519abdadde19963a1e8b1d6c80e85e395799" Dec 03 12:25:07 crc kubenswrapper[5052]: I1203 12:25:07.973194 5052 scope.go:117] "RemoveContainer" containerID="a41a6c3524bcee68d48ead6fe403ee2cd332f3a948dfc6434e4cfddf5ded599e" Dec 03 12:25:07 crc kubenswrapper[5052]: I1203 12:25:07.995343 5052 scope.go:117] "RemoveContainer" containerID="9b34bcef1b760b4f3f7417ac9222e8595707f6e3124b14e2613276d75db6ce45" Dec 03 12:25:08 crc kubenswrapper[5052]: I1203 12:25:08.015869 5052 scope.go:117] "RemoveContainer" containerID="b9ffc180c65833f54ac4d801441e9df4db9322afb9d77cb2f4775dc868d1a03d" Dec 03 12:25:08 crc kubenswrapper[5052]: I1203 12:25:08.039883 5052 scope.go:117] "RemoveContainer" containerID="6b10b360bd6b52778b72608626846a9f746ffaa71ad3eb4acd15aceb0aa6bf5d" Dec 03 12:25:08 crc kubenswrapper[5052]: I1203 12:25:08.771479 5052 generic.go:334] "Generic (PLEG): container finished" podID="2bada3d1-568e-484f-b848-532b9d805f17" containerID="65548fd95f21e38842666bf62b603e65a09c2d7f7fef257d2803d9be9f1bcd15" exitCode=143 Dec 03 12:25:08 crc kubenswrapper[5052]: I1203 12:25:08.771583 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"2bada3d1-568e-484f-b848-532b9d805f17","Type":"ContainerDied","Data":"65548fd95f21e38842666bf62b603e65a09c2d7f7fef257d2803d9be9f1bcd15"} Dec 03 12:25:08 crc kubenswrapper[5052]: I1203 12:25:08.774201 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"3c6a6536-2df8-4c31-9460-66467ae47afd","Type":"ContainerStarted","Data":"14145212884fa7c7a20116a85586d48bac7e8df3947e53f9385d4bb6f03e240b"} Dec 03 12:25:09 crc kubenswrapper[5052]: I1203 12:25:09.682581 5052 scope.go:117] "RemoveContainer" containerID="09b4a0072741f60499d24f40cc4ff3406d5d22671ced431363ab865172786300" Dec 03 12:25:09 crc kubenswrapper[5052]: E1203 12:25:09.683196 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 12:25:09 crc kubenswrapper[5052]: I1203 12:25:09.789265 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"3c6a6536-2df8-4c31-9460-66467ae47afd","Type":"ContainerStarted","Data":"85d21be3aeb068fdc6f8a1346eb659b41a4ea6be471cd4681325b5dd005a55e8"} Dec 03 12:25:09 crc kubenswrapper[5052]: I1203 12:25:09.789713 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:25:09 crc kubenswrapper[5052]: I1203 12:25:09.789447 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="3c6a6536-2df8-4c31-9460-66467ae47afd" containerName="ceilometer-central-agent" containerID="cri-o://c3fbc5ff98507543ea52685a6fc326e382fb78b82083f662da673f4a6c84fe96" gracePeriod=30 Dec 03 12:25:09 crc kubenswrapper[5052]: I1203 12:25:09.789593 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="3c6a6536-2df8-4c31-9460-66467ae47afd" containerName="sg-core" containerID="cri-o://14145212884fa7c7a20116a85586d48bac7e8df3947e53f9385d4bb6f03e240b" gracePeriod=30 Dec 03 12:25:09 crc kubenswrapper[5052]: I1203 12:25:09.789596 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="3c6a6536-2df8-4c31-9460-66467ae47afd" containerName="ceilometer-notification-agent" containerID="cri-o://0d65a1099ad7c395baf34b9e1d6eb93e85150cfbdccd9fbdb9a5168272a8b78e" gracePeriod=30 Dec 03 12:25:09 crc kubenswrapper[5052]: I1203 12:25:09.789492 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="3c6a6536-2df8-4c31-9460-66467ae47afd" containerName="proxy-httpd" containerID="cri-o://85d21be3aeb068fdc6f8a1346eb659b41a4ea6be471cd4681325b5dd005a55e8" gracePeriod=30 Dec 03 12:25:09 crc kubenswrapper[5052]: I1203 12:25:09.825084 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/ceilometer-0" podStartSLOduration=1.911775879 podStartE2EDuration="6.825058175s" podCreationTimestamp="2025-12-03 12:25:03 +0000 UTC" firstStartedPulling="2025-12-03 12:25:04.520216283 +0000 UTC m=+3895.108458047" lastFinishedPulling="2025-12-03 12:25:09.433498569 +0000 UTC m=+3900.021740343" observedRunningTime="2025-12-03 12:25:09.817611332 +0000 UTC m=+3900.405853096" watchObservedRunningTime="2025-12-03 12:25:09.825058175 +0000 UTC m=+3900.413299939" Dec 03 12:25:10 crc kubenswrapper[5052]: I1203 12:25:10.804877 5052 generic.go:334] "Generic (PLEG): container finished" podID="3c6a6536-2df8-4c31-9460-66467ae47afd" containerID="85d21be3aeb068fdc6f8a1346eb659b41a4ea6be471cd4681325b5dd005a55e8" exitCode=0 Dec 03 12:25:10 crc kubenswrapper[5052]: I1203 12:25:10.805179 5052 generic.go:334] "Generic (PLEG): container finished" podID="3c6a6536-2df8-4c31-9460-66467ae47afd" containerID="14145212884fa7c7a20116a85586d48bac7e8df3947e53f9385d4bb6f03e240b" exitCode=2 Dec 03 12:25:10 crc kubenswrapper[5052]: I1203 12:25:10.805187 5052 generic.go:334] "Generic (PLEG): container finished" podID="3c6a6536-2df8-4c31-9460-66467ae47afd" containerID="0d65a1099ad7c395baf34b9e1d6eb93e85150cfbdccd9fbdb9a5168272a8b78e" exitCode=0 Dec 03 12:25:10 crc kubenswrapper[5052]: I1203 12:25:10.804954 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"3c6a6536-2df8-4c31-9460-66467ae47afd","Type":"ContainerDied","Data":"85d21be3aeb068fdc6f8a1346eb659b41a4ea6be471cd4681325b5dd005a55e8"} Dec 03 12:25:10 crc kubenswrapper[5052]: I1203 12:25:10.805262 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"3c6a6536-2df8-4c31-9460-66467ae47afd","Type":"ContainerDied","Data":"14145212884fa7c7a20116a85586d48bac7e8df3947e53f9385d4bb6f03e240b"} Dec 03 12:25:10 crc kubenswrapper[5052]: I1203 12:25:10.805299 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"3c6a6536-2df8-4c31-9460-66467ae47afd","Type":"ContainerDied","Data":"0d65a1099ad7c395baf34b9e1d6eb93e85150cfbdccd9fbdb9a5168272a8b78e"} Dec 03 12:25:11 crc kubenswrapper[5052]: I1203 12:25:11.337856 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:25:11 crc kubenswrapper[5052]: I1203 12:25:11.337920 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:25:11 crc kubenswrapper[5052]: I1203 12:25:11.378705 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:25:11 crc kubenswrapper[5052]: I1203 12:25:11.419205 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:25:11 crc kubenswrapper[5052]: I1203 12:25:11.432702 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:25:11 crc kubenswrapper[5052]: I1203 12:25:11.550446 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/2bada3d1-568e-484f-b848-532b9d805f17-httpd-run\") pod \"2bada3d1-568e-484f-b848-532b9d805f17\" (UID: \"2bada3d1-568e-484f-b848-532b9d805f17\") " Dec 03 12:25:11 crc kubenswrapper[5052]: I1203 12:25:11.550486 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/2bada3d1-568e-484f-b848-532b9d805f17-logs\") pod \"2bada3d1-568e-484f-b848-532b9d805f17\" (UID: \"2bada3d1-568e-484f-b848-532b9d805f17\") " Dec 03 12:25:11 crc kubenswrapper[5052]: I1203 12:25:11.550610 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2bada3d1-568e-484f-b848-532b9d805f17-combined-ca-bundle\") pod \"2bada3d1-568e-484f-b848-532b9d805f17\" (UID: \"2bada3d1-568e-484f-b848-532b9d805f17\") " Dec 03 12:25:11 crc kubenswrapper[5052]: I1203 12:25:11.550667 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/2bada3d1-568e-484f-b848-532b9d805f17-scripts\") pod \"2bada3d1-568e-484f-b848-532b9d805f17\" (UID: \"2bada3d1-568e-484f-b848-532b9d805f17\") " Dec 03 12:25:11 crc kubenswrapper[5052]: I1203 12:25:11.550787 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"glance\" (UniqueName: \"kubernetes.io/local-volume/local-storage07-crc\") pod \"2bada3d1-568e-484f-b848-532b9d805f17\" (UID: \"2bada3d1-568e-484f-b848-532b9d805f17\") " Dec 03 12:25:11 crc kubenswrapper[5052]: I1203 12:25:11.550814 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/2bada3d1-568e-484f-b848-532b9d805f17-config-data\") pod \"2bada3d1-568e-484f-b848-532b9d805f17\" (UID: \"2bada3d1-568e-484f-b848-532b9d805f17\") " Dec 03 12:25:11 crc kubenswrapper[5052]: I1203 12:25:11.550837 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q8fdx\" (UniqueName: \"kubernetes.io/projected/2bada3d1-568e-484f-b848-532b9d805f17-kube-api-access-q8fdx\") pod \"2bada3d1-568e-484f-b848-532b9d805f17\" (UID: \"2bada3d1-568e-484f-b848-532b9d805f17\") " Dec 03 12:25:11 crc kubenswrapper[5052]: I1203 12:25:11.550931 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2bada3d1-568e-484f-b848-532b9d805f17-httpd-run" (OuterVolumeSpecName: "httpd-run") pod "2bada3d1-568e-484f-b848-532b9d805f17" (UID: "2bada3d1-568e-484f-b848-532b9d805f17"). InnerVolumeSpecName "httpd-run". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:25:11 crc kubenswrapper[5052]: I1203 12:25:11.551272 5052 reconciler_common.go:293] "Volume detached for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/2bada3d1-568e-484f-b848-532b9d805f17-httpd-run\") on node \"crc\" DevicePath \"\"" Dec 03 12:25:11 crc kubenswrapper[5052]: I1203 12:25:11.551279 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2bada3d1-568e-484f-b848-532b9d805f17-logs" (OuterVolumeSpecName: "logs") pod "2bada3d1-568e-484f-b848-532b9d805f17" (UID: "2bada3d1-568e-484f-b848-532b9d805f17"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:25:11 crc kubenswrapper[5052]: I1203 12:25:11.557032 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2bada3d1-568e-484f-b848-532b9d805f17-kube-api-access-q8fdx" (OuterVolumeSpecName: "kube-api-access-q8fdx") pod "2bada3d1-568e-484f-b848-532b9d805f17" (UID: "2bada3d1-568e-484f-b848-532b9d805f17"). InnerVolumeSpecName "kube-api-access-q8fdx". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:25:11 crc kubenswrapper[5052]: I1203 12:25:11.559662 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2bada3d1-568e-484f-b848-532b9d805f17-scripts" (OuterVolumeSpecName: "scripts") pod "2bada3d1-568e-484f-b848-532b9d805f17" (UID: "2bada3d1-568e-484f-b848-532b9d805f17"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:25:11 crc kubenswrapper[5052]: I1203 12:25:11.559865 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage07-crc" (OuterVolumeSpecName: "glance") pod "2bada3d1-568e-484f-b848-532b9d805f17" (UID: "2bada3d1-568e-484f-b848-532b9d805f17"). InnerVolumeSpecName "local-storage07-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 12:25:11 crc kubenswrapper[5052]: I1203 12:25:11.595917 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2bada3d1-568e-484f-b848-532b9d805f17-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "2bada3d1-568e-484f-b848-532b9d805f17" (UID: "2bada3d1-568e-484f-b848-532b9d805f17"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:25:11 crc kubenswrapper[5052]: I1203 12:25:11.631197 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2bada3d1-568e-484f-b848-532b9d805f17-config-data" (OuterVolumeSpecName: "config-data") pod "2bada3d1-568e-484f-b848-532b9d805f17" (UID: "2bada3d1-568e-484f-b848-532b9d805f17"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:25:11 crc kubenswrapper[5052]: I1203 12:25:11.652307 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/2bada3d1-568e-484f-b848-532b9d805f17-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:25:11 crc kubenswrapper[5052]: I1203 12:25:11.652338 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2bada3d1-568e-484f-b848-532b9d805f17-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:25:11 crc kubenswrapper[5052]: I1203 12:25:11.652348 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/2bada3d1-568e-484f-b848-532b9d805f17-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:25:11 crc kubenswrapper[5052]: I1203 12:25:11.652378 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage07-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage07-crc\") on node \"crc\" " Dec 03 12:25:11 crc kubenswrapper[5052]: I1203 12:25:11.652389 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/2bada3d1-568e-484f-b848-532b9d805f17-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:25:11 crc kubenswrapper[5052]: I1203 12:25:11.652402 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-q8fdx\" (UniqueName: \"kubernetes.io/projected/2bada3d1-568e-484f-b848-532b9d805f17-kube-api-access-q8fdx\") on node \"crc\" DevicePath \"\"" Dec 03 12:25:11 crc kubenswrapper[5052]: I1203 12:25:11.673103 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage07-crc" (UniqueName: "kubernetes.io/local-volume/local-storage07-crc") on node "crc" Dec 03 12:25:11 crc kubenswrapper[5052]: I1203 12:25:11.754356 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage07-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage07-crc\") on node \"crc\" DevicePath \"\"" Dec 03 12:25:11 crc kubenswrapper[5052]: I1203 12:25:11.820082 5052 generic.go:334] "Generic (PLEG): container finished" podID="2bada3d1-568e-484f-b848-532b9d805f17" containerID="3bd600fb62ffa6746a30412a469e6249947d48035142d1cdf479776822f1807b" exitCode=0 Dec 03 12:25:11 crc kubenswrapper[5052]: I1203 12:25:11.820122 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:25:11 crc kubenswrapper[5052]: I1203 12:25:11.820157 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"2bada3d1-568e-484f-b848-532b9d805f17","Type":"ContainerDied","Data":"3bd600fb62ffa6746a30412a469e6249947d48035142d1cdf479776822f1807b"} Dec 03 12:25:11 crc kubenswrapper[5052]: I1203 12:25:11.820214 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"2bada3d1-568e-484f-b848-532b9d805f17","Type":"ContainerDied","Data":"7148488c01085c6ad2fccd3a752106d9662fc13f9c598213aad88e6240ebc2e0"} Dec 03 12:25:11 crc kubenswrapper[5052]: I1203 12:25:11.820239 5052 scope.go:117] "RemoveContainer" containerID="3bd600fb62ffa6746a30412a469e6249947d48035142d1cdf479776822f1807b" Dec 03 12:25:11 crc kubenswrapper[5052]: I1203 12:25:11.820737 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:25:11 crc kubenswrapper[5052]: I1203 12:25:11.820777 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:25:11 crc kubenswrapper[5052]: E1203 12:25:11.837385 5052 cadvisor_stats_provider.go:516] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2bada3d1_568e_484f_b848_532b9d805f17.slice/crio-7148488c01085c6ad2fccd3a752106d9662fc13f9c598213aad88e6240ebc2e0\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2bada3d1_568e_484f_b848_532b9d805f17.slice\": RecentStats: unable to find data in memory cache]" Dec 03 12:25:11 crc kubenswrapper[5052]: I1203 12:25:11.857148 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:25:11 crc kubenswrapper[5052]: I1203 12:25:11.862441 5052 scope.go:117] "RemoveContainer" containerID="65548fd95f21e38842666bf62b603e65a09c2d7f7fef257d2803d9be9f1bcd15" Dec 03 12:25:11 crc kubenswrapper[5052]: I1203 12:25:11.881202 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:25:11 crc kubenswrapper[5052]: I1203 12:25:11.907060 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:25:11 crc kubenswrapper[5052]: E1203 12:25:11.907658 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="2bada3d1-568e-484f-b848-532b9d805f17" containerName="glance-httpd" Dec 03 12:25:11 crc kubenswrapper[5052]: I1203 12:25:11.907679 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="2bada3d1-568e-484f-b848-532b9d805f17" containerName="glance-httpd" Dec 03 12:25:11 crc kubenswrapper[5052]: E1203 12:25:11.907729 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="2bada3d1-568e-484f-b848-532b9d805f17" containerName="glance-log" Dec 03 12:25:11 crc kubenswrapper[5052]: I1203 12:25:11.907739 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="2bada3d1-568e-484f-b848-532b9d805f17" containerName="glance-log" Dec 03 12:25:11 crc kubenswrapper[5052]: I1203 12:25:11.907957 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="2bada3d1-568e-484f-b848-532b9d805f17" containerName="glance-httpd" Dec 03 12:25:11 crc kubenswrapper[5052]: I1203 12:25:11.908001 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="2bada3d1-568e-484f-b848-532b9d805f17" containerName="glance-log" Dec 03 12:25:11 crc kubenswrapper[5052]: I1203 12:25:11.909179 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:25:11 crc kubenswrapper[5052]: I1203 12:25:11.911165 5052 scope.go:117] "RemoveContainer" containerID="3bd600fb62ffa6746a30412a469e6249947d48035142d1cdf479776822f1807b" Dec 03 12:25:11 crc kubenswrapper[5052]: I1203 12:25:11.915405 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:25:11 crc kubenswrapper[5052]: E1203 12:25:11.916702 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3bd600fb62ffa6746a30412a469e6249947d48035142d1cdf479776822f1807b\": container with ID starting with 3bd600fb62ffa6746a30412a469e6249947d48035142d1cdf479776822f1807b not found: ID does not exist" containerID="3bd600fb62ffa6746a30412a469e6249947d48035142d1cdf479776822f1807b" Dec 03 12:25:11 crc kubenswrapper[5052]: I1203 12:25:11.916738 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3bd600fb62ffa6746a30412a469e6249947d48035142d1cdf479776822f1807b"} err="failed to get container status \"3bd600fb62ffa6746a30412a469e6249947d48035142d1cdf479776822f1807b\": rpc error: code = NotFound desc = could not find container \"3bd600fb62ffa6746a30412a469e6249947d48035142d1cdf479776822f1807b\": container with ID starting with 3bd600fb62ffa6746a30412a469e6249947d48035142d1cdf479776822f1807b not found: ID does not exist" Dec 03 12:25:11 crc kubenswrapper[5052]: I1203 12:25:11.916761 5052 scope.go:117] "RemoveContainer" containerID="65548fd95f21e38842666bf62b603e65a09c2d7f7fef257d2803d9be9f1bcd15" Dec 03 12:25:11 crc kubenswrapper[5052]: E1203 12:25:11.917287 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"65548fd95f21e38842666bf62b603e65a09c2d7f7fef257d2803d9be9f1bcd15\": container with ID starting with 65548fd95f21e38842666bf62b603e65a09c2d7f7fef257d2803d9be9f1bcd15 not found: ID does not exist" containerID="65548fd95f21e38842666bf62b603e65a09c2d7f7fef257d2803d9be9f1bcd15" Dec 03 12:25:11 crc kubenswrapper[5052]: I1203 12:25:11.917340 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"65548fd95f21e38842666bf62b603e65a09c2d7f7fef257d2803d9be9f1bcd15"} err="failed to get container status \"65548fd95f21e38842666bf62b603e65a09c2d7f7fef257d2803d9be9f1bcd15\": rpc error: code = NotFound desc = could not find container \"65548fd95f21e38842666bf62b603e65a09c2d7f7fef257d2803d9be9f1bcd15\": container with ID starting with 65548fd95f21e38842666bf62b603e65a09c2d7f7fef257d2803d9be9f1bcd15 not found: ID does not exist" Dec 03 12:25:11 crc kubenswrapper[5052]: I1203 12:25:11.917480 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-default-external-config-data" Dec 03 12:25:12 crc kubenswrapper[5052]: I1203 12:25:12.061245 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/6b96dc78-b389-4764-8ba7-6a4c5e41e345-logs\") pod \"glance-default-external-api-0\" (UID: \"6b96dc78-b389-4764-8ba7-6a4c5e41e345\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:25:12 crc kubenswrapper[5052]: I1203 12:25:12.061367 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6b96dc78-b389-4764-8ba7-6a4c5e41e345-config-data\") pod \"glance-default-external-api-0\" (UID: \"6b96dc78-b389-4764-8ba7-6a4c5e41e345\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:25:12 crc kubenswrapper[5052]: I1203 12:25:12.061397 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/6b96dc78-b389-4764-8ba7-6a4c5e41e345-httpd-run\") pod \"glance-default-external-api-0\" (UID: \"6b96dc78-b389-4764-8ba7-6a4c5e41e345\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:25:12 crc kubenswrapper[5052]: I1203 12:25:12.061418 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6b96dc78-b389-4764-8ba7-6a4c5e41e345-combined-ca-bundle\") pod \"glance-default-external-api-0\" (UID: \"6b96dc78-b389-4764-8ba7-6a4c5e41e345\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:25:12 crc kubenswrapper[5052]: I1203 12:25:12.061442 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage07-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage07-crc\") pod \"glance-default-external-api-0\" (UID: \"6b96dc78-b389-4764-8ba7-6a4c5e41e345\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:25:12 crc kubenswrapper[5052]: I1203 12:25:12.061683 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/6b96dc78-b389-4764-8ba7-6a4c5e41e345-scripts\") pod \"glance-default-external-api-0\" (UID: \"6b96dc78-b389-4764-8ba7-6a4c5e41e345\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:25:12 crc kubenswrapper[5052]: I1203 12:25:12.061742 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hq6bh\" (UniqueName: \"kubernetes.io/projected/6b96dc78-b389-4764-8ba7-6a4c5e41e345-kube-api-access-hq6bh\") pod \"glance-default-external-api-0\" (UID: \"6b96dc78-b389-4764-8ba7-6a4c5e41e345\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:25:12 crc kubenswrapper[5052]: I1203 12:25:12.163993 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6b96dc78-b389-4764-8ba7-6a4c5e41e345-config-data\") pod \"glance-default-external-api-0\" (UID: \"6b96dc78-b389-4764-8ba7-6a4c5e41e345\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:25:12 crc kubenswrapper[5052]: I1203 12:25:12.164066 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/6b96dc78-b389-4764-8ba7-6a4c5e41e345-httpd-run\") pod \"glance-default-external-api-0\" (UID: \"6b96dc78-b389-4764-8ba7-6a4c5e41e345\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:25:12 crc kubenswrapper[5052]: I1203 12:25:12.164096 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6b96dc78-b389-4764-8ba7-6a4c5e41e345-combined-ca-bundle\") pod \"glance-default-external-api-0\" (UID: \"6b96dc78-b389-4764-8ba7-6a4c5e41e345\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:25:12 crc kubenswrapper[5052]: I1203 12:25:12.164128 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage07-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage07-crc\") pod \"glance-default-external-api-0\" (UID: \"6b96dc78-b389-4764-8ba7-6a4c5e41e345\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:25:12 crc kubenswrapper[5052]: I1203 12:25:12.164202 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-hq6bh\" (UniqueName: \"kubernetes.io/projected/6b96dc78-b389-4764-8ba7-6a4c5e41e345-kube-api-access-hq6bh\") pod \"glance-default-external-api-0\" (UID: \"6b96dc78-b389-4764-8ba7-6a4c5e41e345\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:25:12 crc kubenswrapper[5052]: I1203 12:25:12.164224 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/6b96dc78-b389-4764-8ba7-6a4c5e41e345-scripts\") pod \"glance-default-external-api-0\" (UID: \"6b96dc78-b389-4764-8ba7-6a4c5e41e345\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:25:12 crc kubenswrapper[5052]: I1203 12:25:12.164289 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/6b96dc78-b389-4764-8ba7-6a4c5e41e345-logs\") pod \"glance-default-external-api-0\" (UID: \"6b96dc78-b389-4764-8ba7-6a4c5e41e345\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:25:12 crc kubenswrapper[5052]: I1203 12:25:12.164662 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/6b96dc78-b389-4764-8ba7-6a4c5e41e345-httpd-run\") pod \"glance-default-external-api-0\" (UID: \"6b96dc78-b389-4764-8ba7-6a4c5e41e345\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:25:12 crc kubenswrapper[5052]: I1203 12:25:12.164993 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage07-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage07-crc\") pod \"glance-default-external-api-0\" (UID: \"6b96dc78-b389-4764-8ba7-6a4c5e41e345\") device mount path \"/mnt/openstack/pv07\"" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:25:12 crc kubenswrapper[5052]: I1203 12:25:12.165574 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/6b96dc78-b389-4764-8ba7-6a4c5e41e345-logs\") pod \"glance-default-external-api-0\" (UID: \"6b96dc78-b389-4764-8ba7-6a4c5e41e345\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:25:12 crc kubenswrapper[5052]: I1203 12:25:12.171356 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/6b96dc78-b389-4764-8ba7-6a4c5e41e345-scripts\") pod \"glance-default-external-api-0\" (UID: \"6b96dc78-b389-4764-8ba7-6a4c5e41e345\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:25:12 crc kubenswrapper[5052]: I1203 12:25:12.183585 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6b96dc78-b389-4764-8ba7-6a4c5e41e345-combined-ca-bundle\") pod \"glance-default-external-api-0\" (UID: \"6b96dc78-b389-4764-8ba7-6a4c5e41e345\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:25:12 crc kubenswrapper[5052]: I1203 12:25:12.183597 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-hq6bh\" (UniqueName: \"kubernetes.io/projected/6b96dc78-b389-4764-8ba7-6a4c5e41e345-kube-api-access-hq6bh\") pod \"glance-default-external-api-0\" (UID: \"6b96dc78-b389-4764-8ba7-6a4c5e41e345\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:25:12 crc kubenswrapper[5052]: I1203 12:25:12.187551 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6b96dc78-b389-4764-8ba7-6a4c5e41e345-config-data\") pod \"glance-default-external-api-0\" (UID: \"6b96dc78-b389-4764-8ba7-6a4c5e41e345\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:25:12 crc kubenswrapper[5052]: I1203 12:25:12.208842 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage07-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage07-crc\") pod \"glance-default-external-api-0\" (UID: \"6b96dc78-b389-4764-8ba7-6a4c5e41e345\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:25:12 crc kubenswrapper[5052]: I1203 12:25:12.232623 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:25:12 crc kubenswrapper[5052]: I1203 12:25:12.685927 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:25:12 crc kubenswrapper[5052]: W1203 12:25:12.687251 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6b96dc78_b389_4764_8ba7_6a4c5e41e345.slice/crio-4c1b137865c217b5640ab49ca182ffb12a56fdd4218b0de68f6869a2f5827df4 WatchSource:0}: Error finding container 4c1b137865c217b5640ab49ca182ffb12a56fdd4218b0de68f6869a2f5827df4: Status 404 returned error can't find the container with id 4c1b137865c217b5640ab49ca182ffb12a56fdd4218b0de68f6869a2f5827df4 Dec 03 12:25:12 crc kubenswrapper[5052]: I1203 12:25:12.858286 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"6b96dc78-b389-4764-8ba7-6a4c5e41e345","Type":"ContainerStarted","Data":"4c1b137865c217b5640ab49ca182ffb12a56fdd4218b0de68f6869a2f5827df4"} Dec 03 12:25:12 crc kubenswrapper[5052]: I1203 12:25:12.867047 5052 generic.go:334] "Generic (PLEG): container finished" podID="3c6a6536-2df8-4c31-9460-66467ae47afd" containerID="c3fbc5ff98507543ea52685a6fc326e382fb78b82083f662da673f4a6c84fe96" exitCode=0 Dec 03 12:25:12 crc kubenswrapper[5052]: I1203 12:25:12.867125 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"3c6a6536-2df8-4c31-9460-66467ae47afd","Type":"ContainerDied","Data":"c3fbc5ff98507543ea52685a6fc326e382fb78b82083f662da673f4a6c84fe96"} Dec 03 12:25:13 crc kubenswrapper[5052]: I1203 12:25:13.125749 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:25:13 crc kubenswrapper[5052]: I1203 12:25:13.191684 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3c6a6536-2df8-4c31-9460-66467ae47afd-combined-ca-bundle\") pod \"3c6a6536-2df8-4c31-9460-66467ae47afd\" (UID: \"3c6a6536-2df8-4c31-9460-66467ae47afd\") " Dec 03 12:25:13 crc kubenswrapper[5052]: I1203 12:25:13.191824 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/3c6a6536-2df8-4c31-9460-66467ae47afd-config-data\") pod \"3c6a6536-2df8-4c31-9460-66467ae47afd\" (UID: \"3c6a6536-2df8-4c31-9460-66467ae47afd\") " Dec 03 12:25:13 crc kubenswrapper[5052]: I1203 12:25:13.191874 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7xf85\" (UniqueName: \"kubernetes.io/projected/3c6a6536-2df8-4c31-9460-66467ae47afd-kube-api-access-7xf85\") pod \"3c6a6536-2df8-4c31-9460-66467ae47afd\" (UID: \"3c6a6536-2df8-4c31-9460-66467ae47afd\") " Dec 03 12:25:13 crc kubenswrapper[5052]: I1203 12:25:13.191923 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/3c6a6536-2df8-4c31-9460-66467ae47afd-sg-core-conf-yaml\") pod \"3c6a6536-2df8-4c31-9460-66467ae47afd\" (UID: \"3c6a6536-2df8-4c31-9460-66467ae47afd\") " Dec 03 12:25:13 crc kubenswrapper[5052]: I1203 12:25:13.191963 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/3c6a6536-2df8-4c31-9460-66467ae47afd-run-httpd\") pod \"3c6a6536-2df8-4c31-9460-66467ae47afd\" (UID: \"3c6a6536-2df8-4c31-9460-66467ae47afd\") " Dec 03 12:25:13 crc kubenswrapper[5052]: I1203 12:25:13.192041 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/3c6a6536-2df8-4c31-9460-66467ae47afd-scripts\") pod \"3c6a6536-2df8-4c31-9460-66467ae47afd\" (UID: \"3c6a6536-2df8-4c31-9460-66467ae47afd\") " Dec 03 12:25:13 crc kubenswrapper[5052]: I1203 12:25:13.192097 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/3c6a6536-2df8-4c31-9460-66467ae47afd-log-httpd\") pod \"3c6a6536-2df8-4c31-9460-66467ae47afd\" (UID: \"3c6a6536-2df8-4c31-9460-66467ae47afd\") " Dec 03 12:25:13 crc kubenswrapper[5052]: I1203 12:25:13.196124 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c6a6536-2df8-4c31-9460-66467ae47afd-run-httpd" (OuterVolumeSpecName: "run-httpd") pod "3c6a6536-2df8-4c31-9460-66467ae47afd" (UID: "3c6a6536-2df8-4c31-9460-66467ae47afd"). InnerVolumeSpecName "run-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:25:13 crc kubenswrapper[5052]: I1203 12:25:13.196489 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c6a6536-2df8-4c31-9460-66467ae47afd-log-httpd" (OuterVolumeSpecName: "log-httpd") pod "3c6a6536-2df8-4c31-9460-66467ae47afd" (UID: "3c6a6536-2df8-4c31-9460-66467ae47afd"). InnerVolumeSpecName "log-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:25:13 crc kubenswrapper[5052]: I1203 12:25:13.203050 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3c6a6536-2df8-4c31-9460-66467ae47afd-scripts" (OuterVolumeSpecName: "scripts") pod "3c6a6536-2df8-4c31-9460-66467ae47afd" (UID: "3c6a6536-2df8-4c31-9460-66467ae47afd"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:25:13 crc kubenswrapper[5052]: I1203 12:25:13.205778 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3c6a6536-2df8-4c31-9460-66467ae47afd-kube-api-access-7xf85" (OuterVolumeSpecName: "kube-api-access-7xf85") pod "3c6a6536-2df8-4c31-9460-66467ae47afd" (UID: "3c6a6536-2df8-4c31-9460-66467ae47afd"). InnerVolumeSpecName "kube-api-access-7xf85". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:25:13 crc kubenswrapper[5052]: I1203 12:25:13.236657 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3c6a6536-2df8-4c31-9460-66467ae47afd-sg-core-conf-yaml" (OuterVolumeSpecName: "sg-core-conf-yaml") pod "3c6a6536-2df8-4c31-9460-66467ae47afd" (UID: "3c6a6536-2df8-4c31-9460-66467ae47afd"). InnerVolumeSpecName "sg-core-conf-yaml". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:25:13 crc kubenswrapper[5052]: I1203 12:25:13.294711 5052 reconciler_common.go:293] "Volume detached for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/3c6a6536-2df8-4c31-9460-66467ae47afd-log-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:25:13 crc kubenswrapper[5052]: I1203 12:25:13.294784 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-7xf85\" (UniqueName: \"kubernetes.io/projected/3c6a6536-2df8-4c31-9460-66467ae47afd-kube-api-access-7xf85\") on node \"crc\" DevicePath \"\"" Dec 03 12:25:13 crc kubenswrapper[5052]: I1203 12:25:13.294800 5052 reconciler_common.go:293] "Volume detached for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/3c6a6536-2df8-4c31-9460-66467ae47afd-sg-core-conf-yaml\") on node \"crc\" DevicePath \"\"" Dec 03 12:25:13 crc kubenswrapper[5052]: I1203 12:25:13.294811 5052 reconciler_common.go:293] "Volume detached for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/3c6a6536-2df8-4c31-9460-66467ae47afd-run-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:25:13 crc kubenswrapper[5052]: I1203 12:25:13.294845 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/3c6a6536-2df8-4c31-9460-66467ae47afd-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:25:13 crc kubenswrapper[5052]: I1203 12:25:13.313186 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3c6a6536-2df8-4c31-9460-66467ae47afd-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "3c6a6536-2df8-4c31-9460-66467ae47afd" (UID: "3c6a6536-2df8-4c31-9460-66467ae47afd"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:25:13 crc kubenswrapper[5052]: I1203 12:25:13.346936 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3c6a6536-2df8-4c31-9460-66467ae47afd-config-data" (OuterVolumeSpecName: "config-data") pod "3c6a6536-2df8-4c31-9460-66467ae47afd" (UID: "3c6a6536-2df8-4c31-9460-66467ae47afd"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:25:13 crc kubenswrapper[5052]: I1203 12:25:13.397142 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3c6a6536-2df8-4c31-9460-66467ae47afd-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:25:13 crc kubenswrapper[5052]: I1203 12:25:13.397491 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/3c6a6536-2df8-4c31-9460-66467ae47afd-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:25:13 crc kubenswrapper[5052]: I1203 12:25:13.696151 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2bada3d1-568e-484f-b848-532b9d805f17" path="/var/lib/kubelet/pods/2bada3d1-568e-484f-b848-532b9d805f17/volumes" Dec 03 12:25:13 crc kubenswrapper[5052]: I1203 12:25:13.884932 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"6b96dc78-b389-4764-8ba7-6a4c5e41e345","Type":"ContainerStarted","Data":"be65e03ff7ec9d6c0e6aa36bae00bb290c5b0d2746c6c491262833760b551397"} Dec 03 12:25:13 crc kubenswrapper[5052]: I1203 12:25:13.884995 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"6b96dc78-b389-4764-8ba7-6a4c5e41e345","Type":"ContainerStarted","Data":"481bbc2057a57984fb7f4ba0b572e86b7319fb1c2d5b5c1d8d86cafb6e88f80f"} Dec 03 12:25:13 crc kubenswrapper[5052]: I1203 12:25:13.890938 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"3c6a6536-2df8-4c31-9460-66467ae47afd","Type":"ContainerDied","Data":"75dbc76a6a0669f96a333b47d5d898e1ddfe8602f97374071c03fb878c4d75dd"} Dec 03 12:25:13 crc kubenswrapper[5052]: I1203 12:25:13.890988 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:25:13 crc kubenswrapper[5052]: I1203 12:25:13.891000 5052 scope.go:117] "RemoveContainer" containerID="85d21be3aeb068fdc6f8a1346eb659b41a4ea6be471cd4681325b5dd005a55e8" Dec 03 12:25:13 crc kubenswrapper[5052]: I1203 12:25:13.916231 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/glance-default-external-api-0" podStartSLOduration=2.916209355 podStartE2EDuration="2.916209355s" podCreationTimestamp="2025-12-03 12:25:11 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:25:13.908125084 +0000 UTC m=+3904.496366858" watchObservedRunningTime="2025-12-03 12:25:13.916209355 +0000 UTC m=+3904.504451119" Dec 03 12:25:13 crc kubenswrapper[5052]: I1203 12:25:13.935500 5052 scope.go:117] "RemoveContainer" containerID="14145212884fa7c7a20116a85586d48bac7e8df3947e53f9385d4bb6f03e240b" Dec 03 12:25:13 crc kubenswrapper[5052]: I1203 12:25:13.935587 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:25:13 crc kubenswrapper[5052]: I1203 12:25:13.950773 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:25:13 crc kubenswrapper[5052]: I1203 12:25:13.978713 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:25:13 crc kubenswrapper[5052]: E1203 12:25:13.979234 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="3c6a6536-2df8-4c31-9460-66467ae47afd" containerName="ceilometer-notification-agent" Dec 03 12:25:13 crc kubenswrapper[5052]: I1203 12:25:13.979254 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="3c6a6536-2df8-4c31-9460-66467ae47afd" containerName="ceilometer-notification-agent" Dec 03 12:25:13 crc kubenswrapper[5052]: E1203 12:25:13.979281 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="3c6a6536-2df8-4c31-9460-66467ae47afd" containerName="proxy-httpd" Dec 03 12:25:13 crc kubenswrapper[5052]: I1203 12:25:13.979292 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="3c6a6536-2df8-4c31-9460-66467ae47afd" containerName="proxy-httpd" Dec 03 12:25:13 crc kubenswrapper[5052]: E1203 12:25:13.979333 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="3c6a6536-2df8-4c31-9460-66467ae47afd" containerName="sg-core" Dec 03 12:25:13 crc kubenswrapper[5052]: I1203 12:25:13.979381 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="3c6a6536-2df8-4c31-9460-66467ae47afd" containerName="sg-core" Dec 03 12:25:13 crc kubenswrapper[5052]: E1203 12:25:13.979397 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="3c6a6536-2df8-4c31-9460-66467ae47afd" containerName="ceilometer-central-agent" Dec 03 12:25:13 crc kubenswrapper[5052]: I1203 12:25:13.979405 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="3c6a6536-2df8-4c31-9460-66467ae47afd" containerName="ceilometer-central-agent" Dec 03 12:25:13 crc kubenswrapper[5052]: I1203 12:25:13.979664 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="3c6a6536-2df8-4c31-9460-66467ae47afd" containerName="ceilometer-central-agent" Dec 03 12:25:13 crc kubenswrapper[5052]: I1203 12:25:13.979686 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="3c6a6536-2df8-4c31-9460-66467ae47afd" containerName="proxy-httpd" Dec 03 12:25:13 crc kubenswrapper[5052]: I1203 12:25:13.979706 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="3c6a6536-2df8-4c31-9460-66467ae47afd" containerName="sg-core" Dec 03 12:25:13 crc kubenswrapper[5052]: I1203 12:25:13.979717 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="3c6a6536-2df8-4c31-9460-66467ae47afd" containerName="ceilometer-notification-agent" Dec 03 12:25:13 crc kubenswrapper[5052]: I1203 12:25:13.979803 5052 scope.go:117] "RemoveContainer" containerID="0d65a1099ad7c395baf34b9e1d6eb93e85150cfbdccd9fbdb9a5168272a8b78e" Dec 03 12:25:13 crc kubenswrapper[5052]: I1203 12:25:13.982049 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:25:13 crc kubenswrapper[5052]: I1203 12:25:13.985861 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-scripts" Dec 03 12:25:13 crc kubenswrapper[5052]: I1203 12:25:13.985957 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-config-data" Dec 03 12:25:13 crc kubenswrapper[5052]: I1203 12:25:13.991504 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:25:14 crc kubenswrapper[5052]: I1203 12:25:14.048006 5052 scope.go:117] "RemoveContainer" containerID="c3fbc5ff98507543ea52685a6fc326e382fb78b82083f662da673f4a6c84fe96" Dec 03 12:25:14 crc kubenswrapper[5052]: I1203 12:25:14.075153 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:25:14 crc kubenswrapper[5052]: I1203 12:25:14.075705 5052 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Dec 03 12:25:14 crc kubenswrapper[5052]: I1203 12:25:14.090916 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:25:14 crc kubenswrapper[5052]: I1203 12:25:14.108862 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zxkpr\" (UniqueName: \"kubernetes.io/projected/60855363-0138-4524-af60-e509fb113c0b-kube-api-access-zxkpr\") pod \"ceilometer-0\" (UID: \"60855363-0138-4524-af60-e509fb113c0b\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:25:14 crc kubenswrapper[5052]: I1203 12:25:14.108904 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/60855363-0138-4524-af60-e509fb113c0b-run-httpd\") pod \"ceilometer-0\" (UID: \"60855363-0138-4524-af60-e509fb113c0b\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:25:14 crc kubenswrapper[5052]: I1203 12:25:14.108928 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/60855363-0138-4524-af60-e509fb113c0b-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"60855363-0138-4524-af60-e509fb113c0b\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:25:14 crc kubenswrapper[5052]: I1203 12:25:14.108959 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/60855363-0138-4524-af60-e509fb113c0b-log-httpd\") pod \"ceilometer-0\" (UID: \"60855363-0138-4524-af60-e509fb113c0b\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:25:14 crc kubenswrapper[5052]: I1203 12:25:14.108996 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/60855363-0138-4524-af60-e509fb113c0b-config-data\") pod \"ceilometer-0\" (UID: \"60855363-0138-4524-af60-e509fb113c0b\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:25:14 crc kubenswrapper[5052]: I1203 12:25:14.109017 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/60855363-0138-4524-af60-e509fb113c0b-scripts\") pod \"ceilometer-0\" (UID: \"60855363-0138-4524-af60-e509fb113c0b\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:25:14 crc kubenswrapper[5052]: I1203 12:25:14.109130 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/60855363-0138-4524-af60-e509fb113c0b-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"60855363-0138-4524-af60-e509fb113c0b\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:25:14 crc kubenswrapper[5052]: I1203 12:25:14.210309 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/60855363-0138-4524-af60-e509fb113c0b-config-data\") pod \"ceilometer-0\" (UID: \"60855363-0138-4524-af60-e509fb113c0b\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:25:14 crc kubenswrapper[5052]: I1203 12:25:14.210365 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/60855363-0138-4524-af60-e509fb113c0b-scripts\") pod \"ceilometer-0\" (UID: \"60855363-0138-4524-af60-e509fb113c0b\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:25:14 crc kubenswrapper[5052]: I1203 12:25:14.210448 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/60855363-0138-4524-af60-e509fb113c0b-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"60855363-0138-4524-af60-e509fb113c0b\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:25:14 crc kubenswrapper[5052]: I1203 12:25:14.210483 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-zxkpr\" (UniqueName: \"kubernetes.io/projected/60855363-0138-4524-af60-e509fb113c0b-kube-api-access-zxkpr\") pod \"ceilometer-0\" (UID: \"60855363-0138-4524-af60-e509fb113c0b\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:25:14 crc kubenswrapper[5052]: I1203 12:25:14.210523 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/60855363-0138-4524-af60-e509fb113c0b-run-httpd\") pod \"ceilometer-0\" (UID: \"60855363-0138-4524-af60-e509fb113c0b\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:25:14 crc kubenswrapper[5052]: I1203 12:25:14.210543 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/60855363-0138-4524-af60-e509fb113c0b-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"60855363-0138-4524-af60-e509fb113c0b\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:25:14 crc kubenswrapper[5052]: I1203 12:25:14.210571 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/60855363-0138-4524-af60-e509fb113c0b-log-httpd\") pod \"ceilometer-0\" (UID: \"60855363-0138-4524-af60-e509fb113c0b\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:25:14 crc kubenswrapper[5052]: I1203 12:25:14.211079 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/60855363-0138-4524-af60-e509fb113c0b-log-httpd\") pod \"ceilometer-0\" (UID: \"60855363-0138-4524-af60-e509fb113c0b\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:25:14 crc kubenswrapper[5052]: I1203 12:25:14.211632 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/60855363-0138-4524-af60-e509fb113c0b-run-httpd\") pod \"ceilometer-0\" (UID: \"60855363-0138-4524-af60-e509fb113c0b\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:25:14 crc kubenswrapper[5052]: I1203 12:25:14.217076 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/60855363-0138-4524-af60-e509fb113c0b-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"60855363-0138-4524-af60-e509fb113c0b\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:25:14 crc kubenswrapper[5052]: I1203 12:25:14.235329 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/60855363-0138-4524-af60-e509fb113c0b-scripts\") pod \"ceilometer-0\" (UID: \"60855363-0138-4524-af60-e509fb113c0b\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:25:14 crc kubenswrapper[5052]: I1203 12:25:14.236635 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/60855363-0138-4524-af60-e509fb113c0b-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"60855363-0138-4524-af60-e509fb113c0b\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:25:14 crc kubenswrapper[5052]: I1203 12:25:14.244923 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-zxkpr\" (UniqueName: \"kubernetes.io/projected/60855363-0138-4524-af60-e509fb113c0b-kube-api-access-zxkpr\") pod \"ceilometer-0\" (UID: \"60855363-0138-4524-af60-e509fb113c0b\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:25:14 crc kubenswrapper[5052]: I1203 12:25:14.248223 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/60855363-0138-4524-af60-e509fb113c0b-config-data\") pod \"ceilometer-0\" (UID: \"60855363-0138-4524-af60-e509fb113c0b\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:25:14 crc kubenswrapper[5052]: I1203 12:25:14.317715 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:25:14 crc kubenswrapper[5052]: I1203 12:25:14.919052 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:25:14 crc kubenswrapper[5052]: W1203 12:25:14.924623 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod60855363_0138_4524_af60_e509fb113c0b.slice/crio-c9da42ff54f5ae390caaa3ae0583ae8d84a8c1f486893d4c6c2c9a8601bc9431 WatchSource:0}: Error finding container c9da42ff54f5ae390caaa3ae0583ae8d84a8c1f486893d4c6c2c9a8601bc9431: Status 404 returned error can't find the container with id c9da42ff54f5ae390caaa3ae0583ae8d84a8c1f486893d4c6c2c9a8601bc9431 Dec 03 12:25:15 crc kubenswrapper[5052]: I1203 12:25:15.685559 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3c6a6536-2df8-4c31-9460-66467ae47afd" path="/var/lib/kubelet/pods/3c6a6536-2df8-4c31-9460-66467ae47afd/volumes" Dec 03 12:25:15 crc kubenswrapper[5052]: I1203 12:25:15.914982 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"60855363-0138-4524-af60-e509fb113c0b","Type":"ContainerStarted","Data":"7a4d323cb6d6466b95c1a95a6d37b454a8e65321dde7e9773b5b6607401badac"} Dec 03 12:25:15 crc kubenswrapper[5052]: I1203 12:25:15.915333 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"60855363-0138-4524-af60-e509fb113c0b","Type":"ContainerStarted","Data":"c9da42ff54f5ae390caaa3ae0583ae8d84a8c1f486893d4c6c2c9a8601bc9431"} Dec 03 12:25:16 crc kubenswrapper[5052]: I1203 12:25:16.945062 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"60855363-0138-4524-af60-e509fb113c0b","Type":"ContainerStarted","Data":"bc7ed62ebecdd8949db875ccee04947a38e495a8effb4f2799788690d8f5353d"} Dec 03 12:25:18 crc kubenswrapper[5052]: I1203 12:25:18.966459 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"60855363-0138-4524-af60-e509fb113c0b","Type":"ContainerStarted","Data":"875ac2341ca0230e518265fea1fa835454614e579ba0e283663e6e5e5484220a"} Dec 03 12:25:20 crc kubenswrapper[5052]: I1203 12:25:20.673312 5052 scope.go:117] "RemoveContainer" containerID="09b4a0072741f60499d24f40cc4ff3406d5d22671ced431363ab865172786300" Dec 03 12:25:20 crc kubenswrapper[5052]: E1203 12:25:20.673911 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 12:25:22 crc kubenswrapper[5052]: I1203 12:25:22.008238 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"60855363-0138-4524-af60-e509fb113c0b","Type":"ContainerStarted","Data":"db9649b19c8444660353379457a673b760daed7d6142434b80ddd84a4d2f0f10"} Dec 03 12:25:22 crc kubenswrapper[5052]: I1203 12:25:22.008879 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:25:22 crc kubenswrapper[5052]: I1203 12:25:22.041171 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/ceilometer-0" podStartSLOduration=3.30242673 podStartE2EDuration="9.041146136s" podCreationTimestamp="2025-12-03 12:25:13 +0000 UTC" firstStartedPulling="2025-12-03 12:25:14.928347652 +0000 UTC m=+3905.516589416" lastFinishedPulling="2025-12-03 12:25:20.667067058 +0000 UTC m=+3911.255308822" observedRunningTime="2025-12-03 12:25:22.029930846 +0000 UTC m=+3912.618172610" watchObservedRunningTime="2025-12-03 12:25:22.041146136 +0000 UTC m=+3912.629387900" Dec 03 12:25:22 crc kubenswrapper[5052]: I1203 12:25:22.233593 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:25:22 crc kubenswrapper[5052]: I1203 12:25:22.233658 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:25:22 crc kubenswrapper[5052]: I1203 12:25:22.267671 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:25:22 crc kubenswrapper[5052]: I1203 12:25:22.281306 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:25:23 crc kubenswrapper[5052]: I1203 12:25:23.018799 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:25:23 crc kubenswrapper[5052]: I1203 12:25:23.018857 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:25:26 crc kubenswrapper[5052]: I1203 12:25:26.291164 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:25:26 crc kubenswrapper[5052]: I1203 12:25:26.291621 5052 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Dec 03 12:25:26 crc kubenswrapper[5052]: I1203 12:25:26.292608 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:25:27 crc kubenswrapper[5052]: I1203 12:25:27.986607 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:25:27 crc kubenswrapper[5052]: I1203 12:25:27.987148 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="60855363-0138-4524-af60-e509fb113c0b" containerName="ceilometer-central-agent" containerID="cri-o://7a4d323cb6d6466b95c1a95a6d37b454a8e65321dde7e9773b5b6607401badac" gracePeriod=30 Dec 03 12:25:27 crc kubenswrapper[5052]: I1203 12:25:27.987209 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="60855363-0138-4524-af60-e509fb113c0b" containerName="sg-core" containerID="cri-o://875ac2341ca0230e518265fea1fa835454614e579ba0e283663e6e5e5484220a" gracePeriod=30 Dec 03 12:25:27 crc kubenswrapper[5052]: I1203 12:25:27.987228 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="60855363-0138-4524-af60-e509fb113c0b" containerName="proxy-httpd" containerID="cri-o://db9649b19c8444660353379457a673b760daed7d6142434b80ddd84a4d2f0f10" gracePeriod=30 Dec 03 12:25:27 crc kubenswrapper[5052]: I1203 12:25:27.987234 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="60855363-0138-4524-af60-e509fb113c0b" containerName="ceilometer-notification-agent" containerID="cri-o://bc7ed62ebecdd8949db875ccee04947a38e495a8effb4f2799788690d8f5353d" gracePeriod=30 Dec 03 12:25:28 crc kubenswrapper[5052]: I1203 12:25:28.260996 5052 generic.go:334] "Generic (PLEG): container finished" podID="60855363-0138-4524-af60-e509fb113c0b" containerID="db9649b19c8444660353379457a673b760daed7d6142434b80ddd84a4d2f0f10" exitCode=0 Dec 03 12:25:28 crc kubenswrapper[5052]: I1203 12:25:28.261027 5052 generic.go:334] "Generic (PLEG): container finished" podID="60855363-0138-4524-af60-e509fb113c0b" containerID="875ac2341ca0230e518265fea1fa835454614e579ba0e283663e6e5e5484220a" exitCode=2 Dec 03 12:25:28 crc kubenswrapper[5052]: I1203 12:25:28.261047 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"60855363-0138-4524-af60-e509fb113c0b","Type":"ContainerDied","Data":"db9649b19c8444660353379457a673b760daed7d6142434b80ddd84a4d2f0f10"} Dec 03 12:25:28 crc kubenswrapper[5052]: I1203 12:25:28.261072 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"60855363-0138-4524-af60-e509fb113c0b","Type":"ContainerDied","Data":"875ac2341ca0230e518265fea1fa835454614e579ba0e283663e6e5e5484220a"} Dec 03 12:25:29 crc kubenswrapper[5052]: I1203 12:25:29.275508 5052 generic.go:334] "Generic (PLEG): container finished" podID="60855363-0138-4524-af60-e509fb113c0b" containerID="7a4d323cb6d6466b95c1a95a6d37b454a8e65321dde7e9773b5b6607401badac" exitCode=0 Dec 03 12:25:29 crc kubenswrapper[5052]: I1203 12:25:29.275534 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"60855363-0138-4524-af60-e509fb113c0b","Type":"ContainerDied","Data":"7a4d323cb6d6466b95c1a95a6d37b454a8e65321dde7e9773b5b6607401badac"} Dec 03 12:25:29 crc kubenswrapper[5052]: I1203 12:25:29.278578 5052 generic.go:334] "Generic (PLEG): container finished" podID="3fd1fe87-a439-480b-a74e-854e7003b7a8" containerID="4da5e593296a628772a4119fc20ad8f06ce5e05f848f19bd5f817b1a9b9fb997" exitCode=0 Dec 03 12:25:29 crc kubenswrapper[5052]: I1203 12:25:29.278612 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-gpqnk" event={"ID":"3fd1fe87-a439-480b-a74e-854e7003b7a8","Type":"ContainerDied","Data":"4da5e593296a628772a4119fc20ad8f06ce5e05f848f19bd5f817b1a9b9fb997"} Dec 03 12:25:30 crc kubenswrapper[5052]: I1203 12:25:30.290136 5052 generic.go:334] "Generic (PLEG): container finished" podID="60855363-0138-4524-af60-e509fb113c0b" containerID="bc7ed62ebecdd8949db875ccee04947a38e495a8effb4f2799788690d8f5353d" exitCode=0 Dec 03 12:25:30 crc kubenswrapper[5052]: I1203 12:25:30.290167 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"60855363-0138-4524-af60-e509fb113c0b","Type":"ContainerDied","Data":"bc7ed62ebecdd8949db875ccee04947a38e495a8effb4f2799788690d8f5353d"} Dec 03 12:25:30 crc kubenswrapper[5052]: I1203 12:25:30.290397 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"60855363-0138-4524-af60-e509fb113c0b","Type":"ContainerDied","Data":"c9da42ff54f5ae390caaa3ae0583ae8d84a8c1f486893d4c6c2c9a8601bc9431"} Dec 03 12:25:30 crc kubenswrapper[5052]: I1203 12:25:30.290415 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c9da42ff54f5ae390caaa3ae0583ae8d84a8c1f486893d4c6c2c9a8601bc9431" Dec 03 12:25:30 crc kubenswrapper[5052]: I1203 12:25:30.351090 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:25:30 crc kubenswrapper[5052]: I1203 12:25:30.511327 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zxkpr\" (UniqueName: \"kubernetes.io/projected/60855363-0138-4524-af60-e509fb113c0b-kube-api-access-zxkpr\") pod \"60855363-0138-4524-af60-e509fb113c0b\" (UID: \"60855363-0138-4524-af60-e509fb113c0b\") " Dec 03 12:25:30 crc kubenswrapper[5052]: I1203 12:25:30.511393 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/60855363-0138-4524-af60-e509fb113c0b-sg-core-conf-yaml\") pod \"60855363-0138-4524-af60-e509fb113c0b\" (UID: \"60855363-0138-4524-af60-e509fb113c0b\") " Dec 03 12:25:30 crc kubenswrapper[5052]: I1203 12:25:30.511445 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/60855363-0138-4524-af60-e509fb113c0b-config-data\") pod \"60855363-0138-4524-af60-e509fb113c0b\" (UID: \"60855363-0138-4524-af60-e509fb113c0b\") " Dec 03 12:25:30 crc kubenswrapper[5052]: I1203 12:25:30.511477 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/60855363-0138-4524-af60-e509fb113c0b-log-httpd\") pod \"60855363-0138-4524-af60-e509fb113c0b\" (UID: \"60855363-0138-4524-af60-e509fb113c0b\") " Dec 03 12:25:30 crc kubenswrapper[5052]: I1203 12:25:30.511613 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/60855363-0138-4524-af60-e509fb113c0b-combined-ca-bundle\") pod \"60855363-0138-4524-af60-e509fb113c0b\" (UID: \"60855363-0138-4524-af60-e509fb113c0b\") " Dec 03 12:25:30 crc kubenswrapper[5052]: I1203 12:25:30.511646 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/60855363-0138-4524-af60-e509fb113c0b-scripts\") pod \"60855363-0138-4524-af60-e509fb113c0b\" (UID: \"60855363-0138-4524-af60-e509fb113c0b\") " Dec 03 12:25:30 crc kubenswrapper[5052]: I1203 12:25:30.511701 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/60855363-0138-4524-af60-e509fb113c0b-run-httpd\") pod \"60855363-0138-4524-af60-e509fb113c0b\" (UID: \"60855363-0138-4524-af60-e509fb113c0b\") " Dec 03 12:25:30 crc kubenswrapper[5052]: I1203 12:25:30.512437 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60855363-0138-4524-af60-e509fb113c0b-run-httpd" (OuterVolumeSpecName: "run-httpd") pod "60855363-0138-4524-af60-e509fb113c0b" (UID: "60855363-0138-4524-af60-e509fb113c0b"). InnerVolumeSpecName "run-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:25:30 crc kubenswrapper[5052]: I1203 12:25:30.513059 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60855363-0138-4524-af60-e509fb113c0b-log-httpd" (OuterVolumeSpecName: "log-httpd") pod "60855363-0138-4524-af60-e509fb113c0b" (UID: "60855363-0138-4524-af60-e509fb113c0b"). InnerVolumeSpecName "log-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:25:30 crc kubenswrapper[5052]: I1203 12:25:30.533058 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/60855363-0138-4524-af60-e509fb113c0b-kube-api-access-zxkpr" (OuterVolumeSpecName: "kube-api-access-zxkpr") pod "60855363-0138-4524-af60-e509fb113c0b" (UID: "60855363-0138-4524-af60-e509fb113c0b"). InnerVolumeSpecName "kube-api-access-zxkpr". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:25:30 crc kubenswrapper[5052]: I1203 12:25:30.552372 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/60855363-0138-4524-af60-e509fb113c0b-sg-core-conf-yaml" (OuterVolumeSpecName: "sg-core-conf-yaml") pod "60855363-0138-4524-af60-e509fb113c0b" (UID: "60855363-0138-4524-af60-e509fb113c0b"). InnerVolumeSpecName "sg-core-conf-yaml". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:25:30 crc kubenswrapper[5052]: I1203 12:25:30.552763 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/60855363-0138-4524-af60-e509fb113c0b-scripts" (OuterVolumeSpecName: "scripts") pod "60855363-0138-4524-af60-e509fb113c0b" (UID: "60855363-0138-4524-af60-e509fb113c0b"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:25:30 crc kubenswrapper[5052]: I1203 12:25:30.614453 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/60855363-0138-4524-af60-e509fb113c0b-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:25:30 crc kubenswrapper[5052]: I1203 12:25:30.614775 5052 reconciler_common.go:293] "Volume detached for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/60855363-0138-4524-af60-e509fb113c0b-run-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:25:30 crc kubenswrapper[5052]: I1203 12:25:30.614787 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-zxkpr\" (UniqueName: \"kubernetes.io/projected/60855363-0138-4524-af60-e509fb113c0b-kube-api-access-zxkpr\") on node \"crc\" DevicePath \"\"" Dec 03 12:25:30 crc kubenswrapper[5052]: I1203 12:25:30.614797 5052 reconciler_common.go:293] "Volume detached for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/60855363-0138-4524-af60-e509fb113c0b-sg-core-conf-yaml\") on node \"crc\" DevicePath \"\"" Dec 03 12:25:30 crc kubenswrapper[5052]: I1203 12:25:30.614806 5052 reconciler_common.go:293] "Volume detached for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/60855363-0138-4524-af60-e509fb113c0b-log-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:25:30 crc kubenswrapper[5052]: I1203 12:25:30.633849 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/60855363-0138-4524-af60-e509fb113c0b-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "60855363-0138-4524-af60-e509fb113c0b" (UID: "60855363-0138-4524-af60-e509fb113c0b"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:25:30 crc kubenswrapper[5052]: I1203 12:25:30.647025 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-gpqnk" Dec 03 12:25:30 crc kubenswrapper[5052]: I1203 12:25:30.655040 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/60855363-0138-4524-af60-e509fb113c0b-config-data" (OuterVolumeSpecName: "config-data") pod "60855363-0138-4524-af60-e509fb113c0b" (UID: "60855363-0138-4524-af60-e509fb113c0b"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:25:30 crc kubenswrapper[5052]: I1203 12:25:30.715661 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-97fsm\" (UniqueName: \"kubernetes.io/projected/3fd1fe87-a439-480b-a74e-854e7003b7a8-kube-api-access-97fsm\") pod \"3fd1fe87-a439-480b-a74e-854e7003b7a8\" (UID: \"3fd1fe87-a439-480b-a74e-854e7003b7a8\") " Dec 03 12:25:30 crc kubenswrapper[5052]: I1203 12:25:30.715746 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/3fd1fe87-a439-480b-a74e-854e7003b7a8-config-data\") pod \"3fd1fe87-a439-480b-a74e-854e7003b7a8\" (UID: \"3fd1fe87-a439-480b-a74e-854e7003b7a8\") " Dec 03 12:25:30 crc kubenswrapper[5052]: I1203 12:25:30.715835 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3fd1fe87-a439-480b-a74e-854e7003b7a8-combined-ca-bundle\") pod \"3fd1fe87-a439-480b-a74e-854e7003b7a8\" (UID: \"3fd1fe87-a439-480b-a74e-854e7003b7a8\") " Dec 03 12:25:30 crc kubenswrapper[5052]: I1203 12:25:30.715971 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/3fd1fe87-a439-480b-a74e-854e7003b7a8-scripts\") pod \"3fd1fe87-a439-480b-a74e-854e7003b7a8\" (UID: \"3fd1fe87-a439-480b-a74e-854e7003b7a8\") " Dec 03 12:25:30 crc kubenswrapper[5052]: I1203 12:25:30.716474 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/60855363-0138-4524-af60-e509fb113c0b-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:25:30 crc kubenswrapper[5052]: I1203 12:25:30.716497 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/60855363-0138-4524-af60-e509fb113c0b-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:25:30 crc kubenswrapper[5052]: I1203 12:25:30.719718 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3fd1fe87-a439-480b-a74e-854e7003b7a8-scripts" (OuterVolumeSpecName: "scripts") pod "3fd1fe87-a439-480b-a74e-854e7003b7a8" (UID: "3fd1fe87-a439-480b-a74e-854e7003b7a8"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:25:30 crc kubenswrapper[5052]: I1203 12:25:30.721065 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3fd1fe87-a439-480b-a74e-854e7003b7a8-kube-api-access-97fsm" (OuterVolumeSpecName: "kube-api-access-97fsm") pod "3fd1fe87-a439-480b-a74e-854e7003b7a8" (UID: "3fd1fe87-a439-480b-a74e-854e7003b7a8"). InnerVolumeSpecName "kube-api-access-97fsm". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:25:30 crc kubenswrapper[5052]: I1203 12:25:30.742726 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3fd1fe87-a439-480b-a74e-854e7003b7a8-config-data" (OuterVolumeSpecName: "config-data") pod "3fd1fe87-a439-480b-a74e-854e7003b7a8" (UID: "3fd1fe87-a439-480b-a74e-854e7003b7a8"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:25:30 crc kubenswrapper[5052]: I1203 12:25:30.750918 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3fd1fe87-a439-480b-a74e-854e7003b7a8-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "3fd1fe87-a439-480b-a74e-854e7003b7a8" (UID: "3fd1fe87-a439-480b-a74e-854e7003b7a8"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:25:30 crc kubenswrapper[5052]: I1203 12:25:30.819474 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/3fd1fe87-a439-480b-a74e-854e7003b7a8-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:25:30 crc kubenswrapper[5052]: I1203 12:25:30.819531 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-97fsm\" (UniqueName: \"kubernetes.io/projected/3fd1fe87-a439-480b-a74e-854e7003b7a8-kube-api-access-97fsm\") on node \"crc\" DevicePath \"\"" Dec 03 12:25:30 crc kubenswrapper[5052]: I1203 12:25:30.819543 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/3fd1fe87-a439-480b-a74e-854e7003b7a8-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:25:30 crc kubenswrapper[5052]: I1203 12:25:30.819552 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3fd1fe87-a439-480b-a74e-854e7003b7a8-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:25:31 crc kubenswrapper[5052]: I1203 12:25:31.330717 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:25:31 crc kubenswrapper[5052]: I1203 12:25:31.331786 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-gpqnk" Dec 03 12:25:31 crc kubenswrapper[5052]: I1203 12:25:31.331856 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-gpqnk" event={"ID":"3fd1fe87-a439-480b-a74e-854e7003b7a8","Type":"ContainerDied","Data":"92e51273fbb2edfb5b9f691c10ca8f08a006f9c619b73a63bc0642b7d915ed51"} Dec 03 12:25:31 crc kubenswrapper[5052]: I1203 12:25:31.332103 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="92e51273fbb2edfb5b9f691c10ca8f08a006f9c619b73a63bc0642b7d915ed51" Dec 03 12:25:31 crc kubenswrapper[5052]: I1203 12:25:31.433265 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:25:31 crc kubenswrapper[5052]: I1203 12:25:31.449000 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:25:31 crc kubenswrapper[5052]: I1203 12:25:31.463226 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:25:31 crc kubenswrapper[5052]: E1203 12:25:31.464002 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="60855363-0138-4524-af60-e509fb113c0b" containerName="ceilometer-central-agent" Dec 03 12:25:31 crc kubenswrapper[5052]: I1203 12:25:31.464029 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="60855363-0138-4524-af60-e509fb113c0b" containerName="ceilometer-central-agent" Dec 03 12:25:31 crc kubenswrapper[5052]: E1203 12:25:31.464071 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="60855363-0138-4524-af60-e509fb113c0b" containerName="ceilometer-notification-agent" Dec 03 12:25:31 crc kubenswrapper[5052]: I1203 12:25:31.464080 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="60855363-0138-4524-af60-e509fb113c0b" containerName="ceilometer-notification-agent" Dec 03 12:25:31 crc kubenswrapper[5052]: E1203 12:25:31.464097 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="3fd1fe87-a439-480b-a74e-854e7003b7a8" containerName="nova-cell0-conductor-db-sync" Dec 03 12:25:31 crc kubenswrapper[5052]: I1203 12:25:31.464106 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="3fd1fe87-a439-480b-a74e-854e7003b7a8" containerName="nova-cell0-conductor-db-sync" Dec 03 12:25:31 crc kubenswrapper[5052]: E1203 12:25:31.464123 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="60855363-0138-4524-af60-e509fb113c0b" containerName="sg-core" Dec 03 12:25:31 crc kubenswrapper[5052]: I1203 12:25:31.464131 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="60855363-0138-4524-af60-e509fb113c0b" containerName="sg-core" Dec 03 12:25:31 crc kubenswrapper[5052]: E1203 12:25:31.464152 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="60855363-0138-4524-af60-e509fb113c0b" containerName="proxy-httpd" Dec 03 12:25:31 crc kubenswrapper[5052]: I1203 12:25:31.464160 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="60855363-0138-4524-af60-e509fb113c0b" containerName="proxy-httpd" Dec 03 12:25:31 crc kubenswrapper[5052]: I1203 12:25:31.464432 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="60855363-0138-4524-af60-e509fb113c0b" containerName="ceilometer-central-agent" Dec 03 12:25:31 crc kubenswrapper[5052]: I1203 12:25:31.464453 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="60855363-0138-4524-af60-e509fb113c0b" containerName="proxy-httpd" Dec 03 12:25:31 crc kubenswrapper[5052]: I1203 12:25:31.464462 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="3fd1fe87-a439-480b-a74e-854e7003b7a8" containerName="nova-cell0-conductor-db-sync" Dec 03 12:25:31 crc kubenswrapper[5052]: I1203 12:25:31.464476 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="60855363-0138-4524-af60-e509fb113c0b" containerName="ceilometer-notification-agent" Dec 03 12:25:31 crc kubenswrapper[5052]: I1203 12:25:31.464497 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="60855363-0138-4524-af60-e509fb113c0b" containerName="sg-core" Dec 03 12:25:31 crc kubenswrapper[5052]: I1203 12:25:31.466874 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:25:31 crc kubenswrapper[5052]: I1203 12:25:31.469995 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-config-data" Dec 03 12:25:31 crc kubenswrapper[5052]: I1203 12:25:31.470613 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-scripts" Dec 03 12:25:31 crc kubenswrapper[5052]: I1203 12:25:31.485915 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-0"] Dec 03 12:25:31 crc kubenswrapper[5052]: I1203 12:25:31.487897 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:25:31 crc kubenswrapper[5052]: I1203 12:25:31.490675 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-nova-dockercfg-scw5l" Dec 03 12:25:31 crc kubenswrapper[5052]: I1203 12:25:31.491387 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell0-conductor-config-data" Dec 03 12:25:31 crc kubenswrapper[5052]: I1203 12:25:31.503702 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:25:31 crc kubenswrapper[5052]: I1203 12:25:31.520765 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-0"] Dec 03 12:25:31 crc kubenswrapper[5052]: I1203 12:25:31.535885 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/2040c0be-2fbd-4b01-9852-82d0210f1429-run-httpd\") pod \"ceilometer-0\" (UID: \"2040c0be-2fbd-4b01-9852-82d0210f1429\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:25:31 crc kubenswrapper[5052]: I1203 12:25:31.535958 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/2040c0be-2fbd-4b01-9852-82d0210f1429-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"2040c0be-2fbd-4b01-9852-82d0210f1429\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:25:31 crc kubenswrapper[5052]: I1203 12:25:31.535996 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/2040c0be-2fbd-4b01-9852-82d0210f1429-config-data\") pod \"ceilometer-0\" (UID: \"2040c0be-2fbd-4b01-9852-82d0210f1429\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:25:31 crc kubenswrapper[5052]: I1203 12:25:31.536042 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q9m2s\" (UniqueName: \"kubernetes.io/projected/2040c0be-2fbd-4b01-9852-82d0210f1429-kube-api-access-q9m2s\") pod \"ceilometer-0\" (UID: \"2040c0be-2fbd-4b01-9852-82d0210f1429\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:25:31 crc kubenswrapper[5052]: I1203 12:25:31.536077 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/2040c0be-2fbd-4b01-9852-82d0210f1429-scripts\") pod \"ceilometer-0\" (UID: \"2040c0be-2fbd-4b01-9852-82d0210f1429\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:25:31 crc kubenswrapper[5052]: I1203 12:25:31.536101 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2040c0be-2fbd-4b01-9852-82d0210f1429-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"2040c0be-2fbd-4b01-9852-82d0210f1429\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:25:31 crc kubenswrapper[5052]: I1203 12:25:31.536118 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/2040c0be-2fbd-4b01-9852-82d0210f1429-log-httpd\") pod \"ceilometer-0\" (UID: \"2040c0be-2fbd-4b01-9852-82d0210f1429\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:25:31 crc kubenswrapper[5052]: I1203 12:25:31.638169 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8r2b9\" (UniqueName: \"kubernetes.io/projected/1f6f8fbd-5810-4853-89c0-4be22abe85a3-kube-api-access-8r2b9\") pod \"nova-cell0-conductor-0\" (UID: \"1f6f8fbd-5810-4853-89c0-4be22abe85a3\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:25:31 crc kubenswrapper[5052]: I1203 12:25:31.638240 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-q9m2s\" (UniqueName: \"kubernetes.io/projected/2040c0be-2fbd-4b01-9852-82d0210f1429-kube-api-access-q9m2s\") pod \"ceilometer-0\" (UID: \"2040c0be-2fbd-4b01-9852-82d0210f1429\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:25:31 crc kubenswrapper[5052]: I1203 12:25:31.638348 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/2040c0be-2fbd-4b01-9852-82d0210f1429-scripts\") pod \"ceilometer-0\" (UID: \"2040c0be-2fbd-4b01-9852-82d0210f1429\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:25:31 crc kubenswrapper[5052]: I1203 12:25:31.638401 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2040c0be-2fbd-4b01-9852-82d0210f1429-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"2040c0be-2fbd-4b01-9852-82d0210f1429\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:25:31 crc kubenswrapper[5052]: I1203 12:25:31.638424 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/2040c0be-2fbd-4b01-9852-82d0210f1429-log-httpd\") pod \"ceilometer-0\" (UID: \"2040c0be-2fbd-4b01-9852-82d0210f1429\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:25:31 crc kubenswrapper[5052]: I1203 12:25:31.638493 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1f6f8fbd-5810-4853-89c0-4be22abe85a3-combined-ca-bundle\") pod \"nova-cell0-conductor-0\" (UID: \"1f6f8fbd-5810-4853-89c0-4be22abe85a3\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:25:31 crc kubenswrapper[5052]: I1203 12:25:31.638644 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/2040c0be-2fbd-4b01-9852-82d0210f1429-run-httpd\") pod \"ceilometer-0\" (UID: \"2040c0be-2fbd-4b01-9852-82d0210f1429\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:25:31 crc kubenswrapper[5052]: I1203 12:25:31.638665 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/2040c0be-2fbd-4b01-9852-82d0210f1429-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"2040c0be-2fbd-4b01-9852-82d0210f1429\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:25:31 crc kubenswrapper[5052]: I1203 12:25:31.638713 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/2040c0be-2fbd-4b01-9852-82d0210f1429-config-data\") pod \"ceilometer-0\" (UID: \"2040c0be-2fbd-4b01-9852-82d0210f1429\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:25:31 crc kubenswrapper[5052]: I1203 12:25:31.638747 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/1f6f8fbd-5810-4853-89c0-4be22abe85a3-config-data\") pod \"nova-cell0-conductor-0\" (UID: \"1f6f8fbd-5810-4853-89c0-4be22abe85a3\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:25:31 crc kubenswrapper[5052]: I1203 12:25:31.639566 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/2040c0be-2fbd-4b01-9852-82d0210f1429-log-httpd\") pod \"ceilometer-0\" (UID: \"2040c0be-2fbd-4b01-9852-82d0210f1429\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:25:31 crc kubenswrapper[5052]: I1203 12:25:31.639698 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/2040c0be-2fbd-4b01-9852-82d0210f1429-run-httpd\") pod \"ceilometer-0\" (UID: \"2040c0be-2fbd-4b01-9852-82d0210f1429\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:25:31 crc kubenswrapper[5052]: I1203 12:25:31.643494 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/2040c0be-2fbd-4b01-9852-82d0210f1429-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"2040c0be-2fbd-4b01-9852-82d0210f1429\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:25:31 crc kubenswrapper[5052]: I1203 12:25:31.644400 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2040c0be-2fbd-4b01-9852-82d0210f1429-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"2040c0be-2fbd-4b01-9852-82d0210f1429\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:25:31 crc kubenswrapper[5052]: I1203 12:25:31.644464 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/2040c0be-2fbd-4b01-9852-82d0210f1429-config-data\") pod \"ceilometer-0\" (UID: \"2040c0be-2fbd-4b01-9852-82d0210f1429\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:25:31 crc kubenswrapper[5052]: I1203 12:25:31.653539 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/2040c0be-2fbd-4b01-9852-82d0210f1429-scripts\") pod \"ceilometer-0\" (UID: \"2040c0be-2fbd-4b01-9852-82d0210f1429\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:25:31 crc kubenswrapper[5052]: I1203 12:25:31.656786 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-q9m2s\" (UniqueName: \"kubernetes.io/projected/2040c0be-2fbd-4b01-9852-82d0210f1429-kube-api-access-q9m2s\") pod \"ceilometer-0\" (UID: \"2040c0be-2fbd-4b01-9852-82d0210f1429\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:25:31 crc kubenswrapper[5052]: I1203 12:25:31.673541 5052 scope.go:117] "RemoveContainer" containerID="09b4a0072741f60499d24f40cc4ff3406d5d22671ced431363ab865172786300" Dec 03 12:25:31 crc kubenswrapper[5052]: I1203 12:25:31.685399 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="60855363-0138-4524-af60-e509fb113c0b" path="/var/lib/kubelet/pods/60855363-0138-4524-af60-e509fb113c0b/volumes" Dec 03 12:25:31 crc kubenswrapper[5052]: I1203 12:25:31.740843 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-8r2b9\" (UniqueName: \"kubernetes.io/projected/1f6f8fbd-5810-4853-89c0-4be22abe85a3-kube-api-access-8r2b9\") pod \"nova-cell0-conductor-0\" (UID: \"1f6f8fbd-5810-4853-89c0-4be22abe85a3\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:25:31 crc kubenswrapper[5052]: I1203 12:25:31.741649 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1f6f8fbd-5810-4853-89c0-4be22abe85a3-combined-ca-bundle\") pod \"nova-cell0-conductor-0\" (UID: \"1f6f8fbd-5810-4853-89c0-4be22abe85a3\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:25:31 crc kubenswrapper[5052]: I1203 12:25:31.741962 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/1f6f8fbd-5810-4853-89c0-4be22abe85a3-config-data\") pod \"nova-cell0-conductor-0\" (UID: \"1f6f8fbd-5810-4853-89c0-4be22abe85a3\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:25:31 crc kubenswrapper[5052]: I1203 12:25:31.747988 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/1f6f8fbd-5810-4853-89c0-4be22abe85a3-config-data\") pod \"nova-cell0-conductor-0\" (UID: \"1f6f8fbd-5810-4853-89c0-4be22abe85a3\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:25:31 crc kubenswrapper[5052]: I1203 12:25:31.755254 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1f6f8fbd-5810-4853-89c0-4be22abe85a3-combined-ca-bundle\") pod \"nova-cell0-conductor-0\" (UID: \"1f6f8fbd-5810-4853-89c0-4be22abe85a3\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:25:31 crc kubenswrapper[5052]: I1203 12:25:31.761734 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-8r2b9\" (UniqueName: \"kubernetes.io/projected/1f6f8fbd-5810-4853-89c0-4be22abe85a3-kube-api-access-8r2b9\") pod \"nova-cell0-conductor-0\" (UID: \"1f6f8fbd-5810-4853-89c0-4be22abe85a3\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:25:31 crc kubenswrapper[5052]: I1203 12:25:31.794901 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:25:31 crc kubenswrapper[5052]: I1203 12:25:31.813479 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:25:32 crc kubenswrapper[5052]: I1203 12:25:32.316884 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:25:32 crc kubenswrapper[5052]: I1203 12:25:32.324960 5052 provider.go:102] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Dec 03 12:25:32 crc kubenswrapper[5052]: I1203 12:25:32.343742 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"2040c0be-2fbd-4b01-9852-82d0210f1429","Type":"ContainerStarted","Data":"2696fdd099db7e5dda8a7c778a456bbb5d9650d8684765ce13acc4712aaa456d"} Dec 03 12:25:32 crc kubenswrapper[5052]: I1203 12:25:32.347740 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" event={"ID":"10c237bc-72e7-4da3-9f2c-b1688395bf6e","Type":"ContainerStarted","Data":"658f4d3362e1d46f122d256822917d243f01c3aff5e32c450b855cf70ebcd2f9"} Dec 03 12:25:32 crc kubenswrapper[5052]: I1203 12:25:32.431475 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-0"] Dec 03 12:25:33 crc kubenswrapper[5052]: I1203 12:25:33.358833 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"2040c0be-2fbd-4b01-9852-82d0210f1429","Type":"ContainerStarted","Data":"fda1434e3be27361a253ac86d38068672205b80750ce3eb51a12b74e073b1d2b"} Dec 03 12:25:33 crc kubenswrapper[5052]: I1203 12:25:33.361569 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-conductor-0" event={"ID":"1f6f8fbd-5810-4853-89c0-4be22abe85a3","Type":"ContainerStarted","Data":"e9967011ef9d1750324e927a4b9e79d05b9a0a1c3f9835799be84de44b11270d"} Dec 03 12:25:33 crc kubenswrapper[5052]: I1203 12:25:33.361628 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-conductor-0" event={"ID":"1f6f8fbd-5810-4853-89c0-4be22abe85a3","Type":"ContainerStarted","Data":"17266566b416c0315caf9c02a6305bd7d5912d119139bff52a6a18129c6144aa"} Dec 03 12:25:33 crc kubenswrapper[5052]: I1203 12:25:33.362795 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:25:33 crc kubenswrapper[5052]: I1203 12:25:33.386820 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-cell0-conductor-0" podStartSLOduration=2.386799744 podStartE2EDuration="2.386799744s" podCreationTimestamp="2025-12-03 12:25:31 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:25:33.379600969 +0000 UTC m=+3923.967842733" watchObservedRunningTime="2025-12-03 12:25:33.386799744 +0000 UTC m=+3923.975041508" Dec 03 12:25:34 crc kubenswrapper[5052]: I1203 12:25:34.378596 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"2040c0be-2fbd-4b01-9852-82d0210f1429","Type":"ContainerStarted","Data":"86638ee6f1d993932caa91451744753653fa3507238feac44e741c4134134430"} Dec 03 12:25:35 crc kubenswrapper[5052]: I1203 12:25:35.391710 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"2040c0be-2fbd-4b01-9852-82d0210f1429","Type":"ContainerStarted","Data":"e9c38bbe15d84deb1873bc02f167d157b471f71259a406cefaac732efb600e4f"} Dec 03 12:25:36 crc kubenswrapper[5052]: I1203 12:25:36.404854 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"2040c0be-2fbd-4b01-9852-82d0210f1429","Type":"ContainerStarted","Data":"10a32d43eecd0862569a8cc67a498c585b50d021f89d849caa39ca046cce5b68"} Dec 03 12:25:36 crc kubenswrapper[5052]: I1203 12:25:36.406644 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:25:36 crc kubenswrapper[5052]: I1203 12:25:36.440472 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/ceilometer-0" podStartSLOduration=2.071813022 podStartE2EDuration="5.440453791s" podCreationTimestamp="2025-12-03 12:25:31 +0000 UTC" firstStartedPulling="2025-12-03 12:25:32.324695019 +0000 UTC m=+3922.912936783" lastFinishedPulling="2025-12-03 12:25:35.693335788 +0000 UTC m=+3926.281577552" observedRunningTime="2025-12-03 12:25:36.433571575 +0000 UTC m=+3927.021813339" watchObservedRunningTime="2025-12-03 12:25:36.440453791 +0000 UTC m=+3927.028695555" Dec 03 12:25:41 crc kubenswrapper[5052]: I1203 12:25:41.843315 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:25:42 crc kubenswrapper[5052]: I1203 12:25:42.273155 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell0-cell-mapping-d2r55"] Dec 03 12:25:42 crc kubenswrapper[5052]: I1203 12:25:42.274630 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-cell-mapping-d2r55" Dec 03 12:25:42 crc kubenswrapper[5052]: I1203 12:25:42.282796 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell0-manage-config-data" Dec 03 12:25:42 crc kubenswrapper[5052]: I1203 12:25:42.282921 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell0-manage-scripts" Dec 03 12:25:42 crc kubenswrapper[5052]: I1203 12:25:42.287279 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell0-cell-mapping-d2r55"] Dec 03 12:25:42 crc kubenswrapper[5052]: I1203 12:25:42.360551 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/f5999941-d60a-412e-a5d8-8f4e657e0014-config-data\") pod \"nova-cell0-cell-mapping-d2r55\" (UID: \"f5999941-d60a-412e-a5d8-8f4e657e0014\") " pod="openstack-kuttl-tests/nova-cell0-cell-mapping-d2r55" Dec 03 12:25:42 crc kubenswrapper[5052]: I1203 12:25:42.360676 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8ttbs\" (UniqueName: \"kubernetes.io/projected/f5999941-d60a-412e-a5d8-8f4e657e0014-kube-api-access-8ttbs\") pod \"nova-cell0-cell-mapping-d2r55\" (UID: \"f5999941-d60a-412e-a5d8-8f4e657e0014\") " pod="openstack-kuttl-tests/nova-cell0-cell-mapping-d2r55" Dec 03 12:25:42 crc kubenswrapper[5052]: I1203 12:25:42.360713 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/f5999941-d60a-412e-a5d8-8f4e657e0014-scripts\") pod \"nova-cell0-cell-mapping-d2r55\" (UID: \"f5999941-d60a-412e-a5d8-8f4e657e0014\") " pod="openstack-kuttl-tests/nova-cell0-cell-mapping-d2r55" Dec 03 12:25:42 crc kubenswrapper[5052]: I1203 12:25:42.360743 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f5999941-d60a-412e-a5d8-8f4e657e0014-combined-ca-bundle\") pod \"nova-cell0-cell-mapping-d2r55\" (UID: \"f5999941-d60a-412e-a5d8-8f4e657e0014\") " pod="openstack-kuttl-tests/nova-cell0-cell-mapping-d2r55" Dec 03 12:25:42 crc kubenswrapper[5052]: I1203 12:25:42.414280 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:25:42 crc kubenswrapper[5052]: I1203 12:25:42.416166 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:25:42 crc kubenswrapper[5052]: I1203 12:25:42.417862 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-api-config-data" Dec 03 12:25:42 crc kubenswrapper[5052]: I1203 12:25:42.433222 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:25:42 crc kubenswrapper[5052]: I1203 12:25:42.462168 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-8ttbs\" (UniqueName: \"kubernetes.io/projected/f5999941-d60a-412e-a5d8-8f4e657e0014-kube-api-access-8ttbs\") pod \"nova-cell0-cell-mapping-d2r55\" (UID: \"f5999941-d60a-412e-a5d8-8f4e657e0014\") " pod="openstack-kuttl-tests/nova-cell0-cell-mapping-d2r55" Dec 03 12:25:42 crc kubenswrapper[5052]: I1203 12:25:42.462250 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/f5999941-d60a-412e-a5d8-8f4e657e0014-scripts\") pod \"nova-cell0-cell-mapping-d2r55\" (UID: \"f5999941-d60a-412e-a5d8-8f4e657e0014\") " pod="openstack-kuttl-tests/nova-cell0-cell-mapping-d2r55" Dec 03 12:25:42 crc kubenswrapper[5052]: I1203 12:25:42.462297 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f5999941-d60a-412e-a5d8-8f4e657e0014-combined-ca-bundle\") pod \"nova-cell0-cell-mapping-d2r55\" (UID: \"f5999941-d60a-412e-a5d8-8f4e657e0014\") " pod="openstack-kuttl-tests/nova-cell0-cell-mapping-d2r55" Dec 03 12:25:42 crc kubenswrapper[5052]: I1203 12:25:42.462429 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/f5999941-d60a-412e-a5d8-8f4e657e0014-config-data\") pod \"nova-cell0-cell-mapping-d2r55\" (UID: \"f5999941-d60a-412e-a5d8-8f4e657e0014\") " pod="openstack-kuttl-tests/nova-cell0-cell-mapping-d2r55" Dec 03 12:25:42 crc kubenswrapper[5052]: I1203 12:25:42.468973 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/f5999941-d60a-412e-a5d8-8f4e657e0014-scripts\") pod \"nova-cell0-cell-mapping-d2r55\" (UID: \"f5999941-d60a-412e-a5d8-8f4e657e0014\") " pod="openstack-kuttl-tests/nova-cell0-cell-mapping-d2r55" Dec 03 12:25:42 crc kubenswrapper[5052]: I1203 12:25:42.483304 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/f5999941-d60a-412e-a5d8-8f4e657e0014-config-data\") pod \"nova-cell0-cell-mapping-d2r55\" (UID: \"f5999941-d60a-412e-a5d8-8f4e657e0014\") " pod="openstack-kuttl-tests/nova-cell0-cell-mapping-d2r55" Dec 03 12:25:42 crc kubenswrapper[5052]: I1203 12:25:42.487218 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f5999941-d60a-412e-a5d8-8f4e657e0014-combined-ca-bundle\") pod \"nova-cell0-cell-mapping-d2r55\" (UID: \"f5999941-d60a-412e-a5d8-8f4e657e0014\") " pod="openstack-kuttl-tests/nova-cell0-cell-mapping-d2r55" Dec 03 12:25:42 crc kubenswrapper[5052]: I1203 12:25:42.514835 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 12:25:42 crc kubenswrapper[5052]: I1203 12:25:42.517013 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:25:42 crc kubenswrapper[5052]: I1203 12:25:42.517216 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-8ttbs\" (UniqueName: \"kubernetes.io/projected/f5999941-d60a-412e-a5d8-8f4e657e0014-kube-api-access-8ttbs\") pod \"nova-cell0-cell-mapping-d2r55\" (UID: \"f5999941-d60a-412e-a5d8-8f4e657e0014\") " pod="openstack-kuttl-tests/nova-cell0-cell-mapping-d2r55" Dec 03 12:25:42 crc kubenswrapper[5052]: I1203 12:25:42.523551 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell1-novncproxy-config-data" Dec 03 12:25:42 crc kubenswrapper[5052]: I1203 12:25:42.532583 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:25:42 crc kubenswrapper[5052]: I1203 12:25:42.534066 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:25:42 crc kubenswrapper[5052]: I1203 12:25:42.540356 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-metadata-config-data" Dec 03 12:25:42 crc kubenswrapper[5052]: I1203 12:25:42.562142 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 12:25:42 crc kubenswrapper[5052]: I1203 12:25:42.564157 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/04aa1d5d-e683-417e-a96d-ce9af5a96136-config-data\") pod \"nova-cell1-novncproxy-0\" (UID: \"04aa1d5d-e683-417e-a96d-ce9af5a96136\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:25:42 crc kubenswrapper[5052]: I1203 12:25:42.564209 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/abc55d2f-ccfa-495d-a8a6-a6f77d7c6359-logs\") pod \"nova-api-0\" (UID: \"abc55d2f-ccfa-495d-a8a6-a6f77d7c6359\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:25:42 crc kubenswrapper[5052]: I1203 12:25:42.564254 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/abc55d2f-ccfa-495d-a8a6-a6f77d7c6359-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"abc55d2f-ccfa-495d-a8a6-a6f77d7c6359\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:25:42 crc kubenswrapper[5052]: I1203 12:25:42.564275 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/abc55d2f-ccfa-495d-a8a6-a6f77d7c6359-config-data\") pod \"nova-api-0\" (UID: \"abc55d2f-ccfa-495d-a8a6-a6f77d7c6359\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:25:42 crc kubenswrapper[5052]: I1203 12:25:42.564301 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kwz4d\" (UniqueName: \"kubernetes.io/projected/abc55d2f-ccfa-495d-a8a6-a6f77d7c6359-kube-api-access-kwz4d\") pod \"nova-api-0\" (UID: \"abc55d2f-ccfa-495d-a8a6-a6f77d7c6359\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:25:42 crc kubenswrapper[5052]: I1203 12:25:42.564401 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/04aa1d5d-e683-417e-a96d-ce9af5a96136-combined-ca-bundle\") pod \"nova-cell1-novncproxy-0\" (UID: \"04aa1d5d-e683-417e-a96d-ce9af5a96136\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:25:42 crc kubenswrapper[5052]: I1203 12:25:42.564435 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-prnlf\" (UniqueName: \"kubernetes.io/projected/04aa1d5d-e683-417e-a96d-ce9af5a96136-kube-api-access-prnlf\") pod \"nova-cell1-novncproxy-0\" (UID: \"04aa1d5d-e683-417e-a96d-ce9af5a96136\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:25:42 crc kubenswrapper[5052]: I1203 12:25:42.588993 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:25:42 crc kubenswrapper[5052]: I1203 12:25:42.610316 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-cell-mapping-d2r55" Dec 03 12:25:42 crc kubenswrapper[5052]: I1203 12:25:42.666762 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mc2k6\" (UniqueName: \"kubernetes.io/projected/5f909f2c-2366-4056-9cb5-c03a66b8b1ba-kube-api-access-mc2k6\") pod \"nova-metadata-0\" (UID: \"5f909f2c-2366-4056-9cb5-c03a66b8b1ba\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:25:42 crc kubenswrapper[5052]: I1203 12:25:42.666842 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/abc55d2f-ccfa-495d-a8a6-a6f77d7c6359-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"abc55d2f-ccfa-495d-a8a6-a6f77d7c6359\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:25:42 crc kubenswrapper[5052]: I1203 12:25:42.666871 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/abc55d2f-ccfa-495d-a8a6-a6f77d7c6359-config-data\") pod \"nova-api-0\" (UID: \"abc55d2f-ccfa-495d-a8a6-a6f77d7c6359\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:25:42 crc kubenswrapper[5052]: I1203 12:25:42.666888 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/5f909f2c-2366-4056-9cb5-c03a66b8b1ba-config-data\") pod \"nova-metadata-0\" (UID: \"5f909f2c-2366-4056-9cb5-c03a66b8b1ba\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:25:42 crc kubenswrapper[5052]: I1203 12:25:42.666918 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-kwz4d\" (UniqueName: \"kubernetes.io/projected/abc55d2f-ccfa-495d-a8a6-a6f77d7c6359-kube-api-access-kwz4d\") pod \"nova-api-0\" (UID: \"abc55d2f-ccfa-495d-a8a6-a6f77d7c6359\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:25:42 crc kubenswrapper[5052]: I1203 12:25:42.667006 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5f909f2c-2366-4056-9cb5-c03a66b8b1ba-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"5f909f2c-2366-4056-9cb5-c03a66b8b1ba\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:25:42 crc kubenswrapper[5052]: I1203 12:25:42.667066 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/04aa1d5d-e683-417e-a96d-ce9af5a96136-combined-ca-bundle\") pod \"nova-cell1-novncproxy-0\" (UID: \"04aa1d5d-e683-417e-a96d-ce9af5a96136\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:25:42 crc kubenswrapper[5052]: I1203 12:25:42.667101 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-prnlf\" (UniqueName: \"kubernetes.io/projected/04aa1d5d-e683-417e-a96d-ce9af5a96136-kube-api-access-prnlf\") pod \"nova-cell1-novncproxy-0\" (UID: \"04aa1d5d-e683-417e-a96d-ce9af5a96136\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:25:42 crc kubenswrapper[5052]: I1203 12:25:42.667142 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/04aa1d5d-e683-417e-a96d-ce9af5a96136-config-data\") pod \"nova-cell1-novncproxy-0\" (UID: \"04aa1d5d-e683-417e-a96d-ce9af5a96136\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:25:42 crc kubenswrapper[5052]: I1203 12:25:42.667179 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/5f909f2c-2366-4056-9cb5-c03a66b8b1ba-logs\") pod \"nova-metadata-0\" (UID: \"5f909f2c-2366-4056-9cb5-c03a66b8b1ba\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:25:42 crc kubenswrapper[5052]: I1203 12:25:42.667222 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/abc55d2f-ccfa-495d-a8a6-a6f77d7c6359-logs\") pod \"nova-api-0\" (UID: \"abc55d2f-ccfa-495d-a8a6-a6f77d7c6359\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:25:42 crc kubenswrapper[5052]: I1203 12:25:42.670298 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/abc55d2f-ccfa-495d-a8a6-a6f77d7c6359-logs\") pod \"nova-api-0\" (UID: \"abc55d2f-ccfa-495d-a8a6-a6f77d7c6359\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:25:42 crc kubenswrapper[5052]: I1203 12:25:42.673055 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/04aa1d5d-e683-417e-a96d-ce9af5a96136-combined-ca-bundle\") pod \"nova-cell1-novncproxy-0\" (UID: \"04aa1d5d-e683-417e-a96d-ce9af5a96136\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:25:42 crc kubenswrapper[5052]: I1203 12:25:42.677827 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/04aa1d5d-e683-417e-a96d-ce9af5a96136-config-data\") pod \"nova-cell1-novncproxy-0\" (UID: \"04aa1d5d-e683-417e-a96d-ce9af5a96136\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:25:42 crc kubenswrapper[5052]: I1203 12:25:42.679119 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/abc55d2f-ccfa-495d-a8a6-a6f77d7c6359-config-data\") pod \"nova-api-0\" (UID: \"abc55d2f-ccfa-495d-a8a6-a6f77d7c6359\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:25:42 crc kubenswrapper[5052]: I1203 12:25:42.689052 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-prnlf\" (UniqueName: \"kubernetes.io/projected/04aa1d5d-e683-417e-a96d-ce9af5a96136-kube-api-access-prnlf\") pod \"nova-cell1-novncproxy-0\" (UID: \"04aa1d5d-e683-417e-a96d-ce9af5a96136\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:25:42 crc kubenswrapper[5052]: I1203 12:25:42.690490 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/abc55d2f-ccfa-495d-a8a6-a6f77d7c6359-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"abc55d2f-ccfa-495d-a8a6-a6f77d7c6359\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:25:42 crc kubenswrapper[5052]: I1203 12:25:42.691127 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-kwz4d\" (UniqueName: \"kubernetes.io/projected/abc55d2f-ccfa-495d-a8a6-a6f77d7c6359-kube-api-access-kwz4d\") pod \"nova-api-0\" (UID: \"abc55d2f-ccfa-495d-a8a6-a6f77d7c6359\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:25:42 crc kubenswrapper[5052]: I1203 12:25:42.717323 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:25:42 crc kubenswrapper[5052]: I1203 12:25:42.719264 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:25:42 crc kubenswrapper[5052]: I1203 12:25:42.722855 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-scheduler-config-data" Dec 03 12:25:42 crc kubenswrapper[5052]: I1203 12:25:42.734156 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:25:42 crc kubenswrapper[5052]: I1203 12:25:42.742869 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:25:42 crc kubenswrapper[5052]: I1203 12:25:42.770013 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/5f909f2c-2366-4056-9cb5-c03a66b8b1ba-logs\") pod \"nova-metadata-0\" (UID: \"5f909f2c-2366-4056-9cb5-c03a66b8b1ba\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:25:42 crc kubenswrapper[5052]: I1203 12:25:42.770074 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f3f2d6a8-014d-493a-ad53-c68acf01ce61-combined-ca-bundle\") pod \"nova-scheduler-0\" (UID: \"f3f2d6a8-014d-493a-ad53-c68acf01ce61\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:25:42 crc kubenswrapper[5052]: I1203 12:25:42.770124 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-mc2k6\" (UniqueName: \"kubernetes.io/projected/5f909f2c-2366-4056-9cb5-c03a66b8b1ba-kube-api-access-mc2k6\") pod \"nova-metadata-0\" (UID: \"5f909f2c-2366-4056-9cb5-c03a66b8b1ba\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:25:42 crc kubenswrapper[5052]: I1203 12:25:42.770167 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/5f909f2c-2366-4056-9cb5-c03a66b8b1ba-config-data\") pod \"nova-metadata-0\" (UID: \"5f909f2c-2366-4056-9cb5-c03a66b8b1ba\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:25:42 crc kubenswrapper[5052]: I1203 12:25:42.770287 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5f909f2c-2366-4056-9cb5-c03a66b8b1ba-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"5f909f2c-2366-4056-9cb5-c03a66b8b1ba\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:25:42 crc kubenswrapper[5052]: I1203 12:25:42.770320 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/f3f2d6a8-014d-493a-ad53-c68acf01ce61-config-data\") pod \"nova-scheduler-0\" (UID: \"f3f2d6a8-014d-493a-ad53-c68acf01ce61\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:25:42 crc kubenswrapper[5052]: I1203 12:25:42.770367 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pxtw2\" (UniqueName: \"kubernetes.io/projected/f3f2d6a8-014d-493a-ad53-c68acf01ce61-kube-api-access-pxtw2\") pod \"nova-scheduler-0\" (UID: \"f3f2d6a8-014d-493a-ad53-c68acf01ce61\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:25:42 crc kubenswrapper[5052]: I1203 12:25:42.771288 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/5f909f2c-2366-4056-9cb5-c03a66b8b1ba-logs\") pod \"nova-metadata-0\" (UID: \"5f909f2c-2366-4056-9cb5-c03a66b8b1ba\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:25:42 crc kubenswrapper[5052]: I1203 12:25:42.790524 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5f909f2c-2366-4056-9cb5-c03a66b8b1ba-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"5f909f2c-2366-4056-9cb5-c03a66b8b1ba\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:25:42 crc kubenswrapper[5052]: I1203 12:25:42.795154 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/5f909f2c-2366-4056-9cb5-c03a66b8b1ba-config-data\") pod \"nova-metadata-0\" (UID: \"5f909f2c-2366-4056-9cb5-c03a66b8b1ba\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:25:42 crc kubenswrapper[5052]: I1203 12:25:42.829291 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-mc2k6\" (UniqueName: \"kubernetes.io/projected/5f909f2c-2366-4056-9cb5-c03a66b8b1ba-kube-api-access-mc2k6\") pod \"nova-metadata-0\" (UID: \"5f909f2c-2366-4056-9cb5-c03a66b8b1ba\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:25:42 crc kubenswrapper[5052]: I1203 12:25:42.880657 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f3f2d6a8-014d-493a-ad53-c68acf01ce61-combined-ca-bundle\") pod \"nova-scheduler-0\" (UID: \"f3f2d6a8-014d-493a-ad53-c68acf01ce61\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:25:42 crc kubenswrapper[5052]: I1203 12:25:42.881170 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/f3f2d6a8-014d-493a-ad53-c68acf01ce61-config-data\") pod \"nova-scheduler-0\" (UID: \"f3f2d6a8-014d-493a-ad53-c68acf01ce61\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:25:42 crc kubenswrapper[5052]: I1203 12:25:42.881224 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-pxtw2\" (UniqueName: \"kubernetes.io/projected/f3f2d6a8-014d-493a-ad53-c68acf01ce61-kube-api-access-pxtw2\") pod \"nova-scheduler-0\" (UID: \"f3f2d6a8-014d-493a-ad53-c68acf01ce61\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:25:42 crc kubenswrapper[5052]: I1203 12:25:42.905582 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/f3f2d6a8-014d-493a-ad53-c68acf01ce61-config-data\") pod \"nova-scheduler-0\" (UID: \"f3f2d6a8-014d-493a-ad53-c68acf01ce61\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:25:42 crc kubenswrapper[5052]: I1203 12:25:42.906137 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f3f2d6a8-014d-493a-ad53-c68acf01ce61-combined-ca-bundle\") pod \"nova-scheduler-0\" (UID: \"f3f2d6a8-014d-493a-ad53-c68acf01ce61\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:25:42 crc kubenswrapper[5052]: I1203 12:25:42.958129 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:25:42 crc kubenswrapper[5052]: I1203 12:25:42.970377 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:25:42 crc kubenswrapper[5052]: I1203 12:25:42.972154 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-pxtw2\" (UniqueName: \"kubernetes.io/projected/f3f2d6a8-014d-493a-ad53-c68acf01ce61-kube-api-access-pxtw2\") pod \"nova-scheduler-0\" (UID: \"f3f2d6a8-014d-493a-ad53-c68acf01ce61\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:25:43 crc kubenswrapper[5052]: I1203 12:25:43.130695 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:25:43 crc kubenswrapper[5052]: I1203 12:25:43.192157 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell0-cell-mapping-d2r55"] Dec 03 12:25:43 crc kubenswrapper[5052]: I1203 12:25:43.490501 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-cell-mapping-d2r55" event={"ID":"f5999941-d60a-412e-a5d8-8f4e657e0014","Type":"ContainerStarted","Data":"40a22548a28c69710420db1ebd5a411abcc276bbd6884ba1ab5538c145ccfae4"} Dec 03 12:25:43 crc kubenswrapper[5052]: I1203 12:25:43.652259 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:25:43 crc kubenswrapper[5052]: I1203 12:25:43.953493 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 12:25:43 crc kubenswrapper[5052]: I1203 12:25:43.975391 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:25:43 crc kubenswrapper[5052]: I1203 12:25:43.998133 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-db-sync-f6zrf"] Dec 03 12:25:43 crc kubenswrapper[5052]: I1203 12:25:43.999431 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-f6zrf" Dec 03 12:25:44 crc kubenswrapper[5052]: I1203 12:25:44.004172 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell1-conductor-scripts" Dec 03 12:25:44 crc kubenswrapper[5052]: I1203 12:25:44.004395 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell1-conductor-config-data" Dec 03 12:25:44 crc kubenswrapper[5052]: I1203 12:25:44.031440 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-db-sync-f6zrf"] Dec 03 12:25:44 crc kubenswrapper[5052]: I1203 12:25:44.103592 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:25:44 crc kubenswrapper[5052]: I1203 12:25:44.109443 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f18600d3-8cdb-48e0-9b40-4773286fdd82-combined-ca-bundle\") pod \"nova-cell1-conductor-db-sync-f6zrf\" (UID: \"f18600d3-8cdb-48e0-9b40-4773286fdd82\") " pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-f6zrf" Dec 03 12:25:44 crc kubenswrapper[5052]: I1203 12:25:44.109531 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lrq6h\" (UniqueName: \"kubernetes.io/projected/f18600d3-8cdb-48e0-9b40-4773286fdd82-kube-api-access-lrq6h\") pod \"nova-cell1-conductor-db-sync-f6zrf\" (UID: \"f18600d3-8cdb-48e0-9b40-4773286fdd82\") " pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-f6zrf" Dec 03 12:25:44 crc kubenswrapper[5052]: I1203 12:25:44.109559 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/f18600d3-8cdb-48e0-9b40-4773286fdd82-scripts\") pod \"nova-cell1-conductor-db-sync-f6zrf\" (UID: \"f18600d3-8cdb-48e0-9b40-4773286fdd82\") " pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-f6zrf" Dec 03 12:25:44 crc kubenswrapper[5052]: I1203 12:25:44.109602 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/f18600d3-8cdb-48e0-9b40-4773286fdd82-config-data\") pod \"nova-cell1-conductor-db-sync-f6zrf\" (UID: \"f18600d3-8cdb-48e0-9b40-4773286fdd82\") " pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-f6zrf" Dec 03 12:25:44 crc kubenswrapper[5052]: W1203 12:25:44.113272 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf3f2d6a8_014d_493a_ad53_c68acf01ce61.slice/crio-4af2921b9ae039bf9cf7543218dde1530b3da62319d5eac483c65c456f293b04 WatchSource:0}: Error finding container 4af2921b9ae039bf9cf7543218dde1530b3da62319d5eac483c65c456f293b04: Status 404 returned error can't find the container with id 4af2921b9ae039bf9cf7543218dde1530b3da62319d5eac483c65c456f293b04 Dec 03 12:25:44 crc kubenswrapper[5052]: I1203 12:25:44.213215 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f18600d3-8cdb-48e0-9b40-4773286fdd82-combined-ca-bundle\") pod \"nova-cell1-conductor-db-sync-f6zrf\" (UID: \"f18600d3-8cdb-48e0-9b40-4773286fdd82\") " pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-f6zrf" Dec 03 12:25:44 crc kubenswrapper[5052]: I1203 12:25:44.213574 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-lrq6h\" (UniqueName: \"kubernetes.io/projected/f18600d3-8cdb-48e0-9b40-4773286fdd82-kube-api-access-lrq6h\") pod \"nova-cell1-conductor-db-sync-f6zrf\" (UID: \"f18600d3-8cdb-48e0-9b40-4773286fdd82\") " pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-f6zrf" Dec 03 12:25:44 crc kubenswrapper[5052]: I1203 12:25:44.213619 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/f18600d3-8cdb-48e0-9b40-4773286fdd82-scripts\") pod \"nova-cell1-conductor-db-sync-f6zrf\" (UID: \"f18600d3-8cdb-48e0-9b40-4773286fdd82\") " pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-f6zrf" Dec 03 12:25:44 crc kubenswrapper[5052]: I1203 12:25:44.213662 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/f18600d3-8cdb-48e0-9b40-4773286fdd82-config-data\") pod \"nova-cell1-conductor-db-sync-f6zrf\" (UID: \"f18600d3-8cdb-48e0-9b40-4773286fdd82\") " pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-f6zrf" Dec 03 12:25:44 crc kubenswrapper[5052]: I1203 12:25:44.218815 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/f18600d3-8cdb-48e0-9b40-4773286fdd82-scripts\") pod \"nova-cell1-conductor-db-sync-f6zrf\" (UID: \"f18600d3-8cdb-48e0-9b40-4773286fdd82\") " pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-f6zrf" Dec 03 12:25:44 crc kubenswrapper[5052]: I1203 12:25:44.218987 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f18600d3-8cdb-48e0-9b40-4773286fdd82-combined-ca-bundle\") pod \"nova-cell1-conductor-db-sync-f6zrf\" (UID: \"f18600d3-8cdb-48e0-9b40-4773286fdd82\") " pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-f6zrf" Dec 03 12:25:44 crc kubenswrapper[5052]: I1203 12:25:44.221317 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/f18600d3-8cdb-48e0-9b40-4773286fdd82-config-data\") pod \"nova-cell1-conductor-db-sync-f6zrf\" (UID: \"f18600d3-8cdb-48e0-9b40-4773286fdd82\") " pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-f6zrf" Dec 03 12:25:44 crc kubenswrapper[5052]: I1203 12:25:44.231949 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-lrq6h\" (UniqueName: \"kubernetes.io/projected/f18600d3-8cdb-48e0-9b40-4773286fdd82-kube-api-access-lrq6h\") pod \"nova-cell1-conductor-db-sync-f6zrf\" (UID: \"f18600d3-8cdb-48e0-9b40-4773286fdd82\") " pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-f6zrf" Dec 03 12:25:44 crc kubenswrapper[5052]: I1203 12:25:44.359854 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-f6zrf" Dec 03 12:25:44 crc kubenswrapper[5052]: I1203 12:25:44.505932 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"f3f2d6a8-014d-493a-ad53-c68acf01ce61","Type":"ContainerStarted","Data":"1219127b0f28f232cebab108b969dfccbae824421c1a95a3735263eb1e8796a0"} Dec 03 12:25:44 crc kubenswrapper[5052]: I1203 12:25:44.505984 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"f3f2d6a8-014d-493a-ad53-c68acf01ce61","Type":"ContainerStarted","Data":"4af2921b9ae039bf9cf7543218dde1530b3da62319d5eac483c65c456f293b04"} Dec 03 12:25:44 crc kubenswrapper[5052]: I1203 12:25:44.512929 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-cell-mapping-d2r55" event={"ID":"f5999941-d60a-412e-a5d8-8f4e657e0014","Type":"ContainerStarted","Data":"3627f5e2b19aa5df937c1c6ec05a72cfd712e75b011381fe1b0ec8c35bc881bf"} Dec 03 12:25:44 crc kubenswrapper[5052]: I1203 12:25:44.519433 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"abc55d2f-ccfa-495d-a8a6-a6f77d7c6359","Type":"ContainerStarted","Data":"6e694094de1b5f6f8bfbd62dac1a726406c6c627116d18c9c852c2c84f407c7b"} Dec 03 12:25:44 crc kubenswrapper[5052]: I1203 12:25:44.519491 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"abc55d2f-ccfa-495d-a8a6-a6f77d7c6359","Type":"ContainerStarted","Data":"d37d1d57c20d5070c9980e946f4ab10fe33b7925b8fab9cd3ca9d1f75f226131"} Dec 03 12:25:44 crc kubenswrapper[5052]: I1203 12:25:44.519518 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"abc55d2f-ccfa-495d-a8a6-a6f77d7c6359","Type":"ContainerStarted","Data":"20aa87d8eac378f8d314811d60a1b63e602cbf8517927db7a5212c4cc5196336"} Dec 03 12:25:44 crc kubenswrapper[5052]: I1203 12:25:44.521987 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"5f909f2c-2366-4056-9cb5-c03a66b8b1ba","Type":"ContainerStarted","Data":"69fd0c98744de04271306fab61a50f33147641fa4e4c2ab8eec766382b02abd5"} Dec 03 12:25:44 crc kubenswrapper[5052]: I1203 12:25:44.522018 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"5f909f2c-2366-4056-9cb5-c03a66b8b1ba","Type":"ContainerStarted","Data":"28b063d8d55d18da9d42985295c475e98897f1eee57fedd6d01cc380cb473a83"} Dec 03 12:25:44 crc kubenswrapper[5052]: I1203 12:25:44.522030 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"5f909f2c-2366-4056-9cb5-c03a66b8b1ba","Type":"ContainerStarted","Data":"fbc1cb63e9f75a77327225b4dca8b5ebbfa70a678518f07852bd3eef5d78d271"} Dec 03 12:25:44 crc kubenswrapper[5052]: I1203 12:25:44.535075 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" event={"ID":"04aa1d5d-e683-417e-a96d-ce9af5a96136","Type":"ContainerStarted","Data":"10cf3584ca3062418591f9016c0b57483870a9df1de7e1394ee2ccefe62d4933"} Dec 03 12:25:44 crc kubenswrapper[5052]: I1203 12:25:44.535136 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" event={"ID":"04aa1d5d-e683-417e-a96d-ce9af5a96136","Type":"ContainerStarted","Data":"8f49d49c5caaa0aebf5069c64c021effb694efcfd0a78abcac5d12ca7872efa1"} Dec 03 12:25:44 crc kubenswrapper[5052]: I1203 12:25:44.539638 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-scheduler-0" podStartSLOduration=2.539616569 podStartE2EDuration="2.539616569s" podCreationTimestamp="2025-12-03 12:25:42 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:25:44.536605413 +0000 UTC m=+3935.124847197" watchObservedRunningTime="2025-12-03 12:25:44.539616569 +0000 UTC m=+3935.127858333" Dec 03 12:25:44 crc kubenswrapper[5052]: I1203 12:25:44.608232 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-metadata-0" podStartSLOduration=2.608207166 podStartE2EDuration="2.608207166s" podCreationTimestamp="2025-12-03 12:25:42 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:25:44.567628848 +0000 UTC m=+3935.155870622" watchObservedRunningTime="2025-12-03 12:25:44.608207166 +0000 UTC m=+3935.196448930" Dec 03 12:25:44 crc kubenswrapper[5052]: I1203 12:25:44.615442 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" podStartSLOduration=2.615422562 podStartE2EDuration="2.615422562s" podCreationTimestamp="2025-12-03 12:25:42 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:25:44.595399851 +0000 UTC m=+3935.183641615" watchObservedRunningTime="2025-12-03 12:25:44.615422562 +0000 UTC m=+3935.203664326" Dec 03 12:25:44 crc kubenswrapper[5052]: I1203 12:25:44.620821 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-cell0-cell-mapping-d2r55" podStartSLOduration=2.620808096 podStartE2EDuration="2.620808096s" podCreationTimestamp="2025-12-03 12:25:42 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:25:44.618420698 +0000 UTC m=+3935.206662462" watchObservedRunningTime="2025-12-03 12:25:44.620808096 +0000 UTC m=+3935.209049860" Dec 03 12:25:44 crc kubenswrapper[5052]: I1203 12:25:44.665852 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-api-0" podStartSLOduration=2.66581141 podStartE2EDuration="2.66581141s" podCreationTimestamp="2025-12-03 12:25:42 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:25:44.648436974 +0000 UTC m=+3935.236678748" watchObservedRunningTime="2025-12-03 12:25:44.66581141 +0000 UTC m=+3935.254053174" Dec 03 12:25:45 crc kubenswrapper[5052]: I1203 12:25:45.022999 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-db-sync-f6zrf"] Dec 03 12:25:45 crc kubenswrapper[5052]: I1203 12:25:45.548862 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-f6zrf" event={"ID":"f18600d3-8cdb-48e0-9b40-4773286fdd82","Type":"ContainerStarted","Data":"0eb00ee36fea8f5a8d36eaf31a8098b39f40b6cf42acfc1c7a923e4f7ae3d122"} Dec 03 12:25:45 crc kubenswrapper[5052]: I1203 12:25:45.549232 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-f6zrf" event={"ID":"f18600d3-8cdb-48e0-9b40-4773286fdd82","Type":"ContainerStarted","Data":"f4c2ea14bf8994a7a027d2992f8f33b55849cd56ba806eabf1631d2e0072e479"} Dec 03 12:25:45 crc kubenswrapper[5052]: I1203 12:25:45.582531 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-f6zrf" podStartSLOduration=2.582475534 podStartE2EDuration="2.582475534s" podCreationTimestamp="2025-12-03 12:25:43 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:25:45.567071064 +0000 UTC m=+3936.155312838" watchObservedRunningTime="2025-12-03 12:25:45.582475534 +0000 UTC m=+3936.170717298" Dec 03 12:25:47 crc kubenswrapper[5052]: I1203 12:25:47.959178 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:25:47 crc kubenswrapper[5052]: I1203 12:25:47.971891 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:25:47 crc kubenswrapper[5052]: I1203 12:25:47.972131 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:25:48 crc kubenswrapper[5052]: I1203 12:25:48.131996 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:25:52 crc kubenswrapper[5052]: I1203 12:25:52.625586 5052 generic.go:334] "Generic (PLEG): container finished" podID="f5999941-d60a-412e-a5d8-8f4e657e0014" containerID="3627f5e2b19aa5df937c1c6ec05a72cfd712e75b011381fe1b0ec8c35bc881bf" exitCode=0 Dec 03 12:25:52 crc kubenswrapper[5052]: I1203 12:25:52.625690 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-cell-mapping-d2r55" event={"ID":"f5999941-d60a-412e-a5d8-8f4e657e0014","Type":"ContainerDied","Data":"3627f5e2b19aa5df937c1c6ec05a72cfd712e75b011381fe1b0ec8c35bc881bf"} Dec 03 12:25:52 crc kubenswrapper[5052]: I1203 12:25:52.744637 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:25:52 crc kubenswrapper[5052]: I1203 12:25:52.744712 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:25:52 crc kubenswrapper[5052]: I1203 12:25:52.962110 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:25:52 crc kubenswrapper[5052]: I1203 12:25:52.971213 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:25:52 crc kubenswrapper[5052]: I1203 12:25:52.971263 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:25:52 crc kubenswrapper[5052]: I1203 12:25:52.977673 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:25:53 crc kubenswrapper[5052]: I1203 12:25:53.132559 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:25:53 crc kubenswrapper[5052]: I1203 12:25:53.305812 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:25:53 crc kubenswrapper[5052]: I1203 12:25:53.649023 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:25:53 crc kubenswrapper[5052]: I1203 12:25:53.690266 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:25:53 crc kubenswrapper[5052]: I1203 12:25:53.841774 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-api-0" podUID="abc55d2f-ccfa-495d-a8a6-a6f77d7c6359" containerName="nova-api-api" probeResult="failure" output="Get \"http://10.217.0.107:8774/\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Dec 03 12:25:53 crc kubenswrapper[5052]: I1203 12:25:53.841891 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-api-0" podUID="abc55d2f-ccfa-495d-a8a6-a6f77d7c6359" containerName="nova-api-log" probeResult="failure" output="Get \"http://10.217.0.107:8774/\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Dec 03 12:25:54 crc kubenswrapper[5052]: I1203 12:25:54.056321 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-metadata-0" podUID="5f909f2c-2366-4056-9cb5-c03a66b8b1ba" containerName="nova-metadata-log" probeResult="failure" output="Get \"http://10.217.0.109:8775/\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Dec 03 12:25:54 crc kubenswrapper[5052]: I1203 12:25:54.056321 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-metadata-0" podUID="5f909f2c-2366-4056-9cb5-c03a66b8b1ba" containerName="nova-metadata-metadata" probeResult="failure" output="Get \"http://10.217.0.109:8775/\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Dec 03 12:25:54 crc kubenswrapper[5052]: I1203 12:25:54.189707 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-cell-mapping-d2r55" Dec 03 12:25:54 crc kubenswrapper[5052]: I1203 12:25:54.231415 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/f5999941-d60a-412e-a5d8-8f4e657e0014-scripts\") pod \"f5999941-d60a-412e-a5d8-8f4e657e0014\" (UID: \"f5999941-d60a-412e-a5d8-8f4e657e0014\") " Dec 03 12:25:54 crc kubenswrapper[5052]: I1203 12:25:54.231592 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8ttbs\" (UniqueName: \"kubernetes.io/projected/f5999941-d60a-412e-a5d8-8f4e657e0014-kube-api-access-8ttbs\") pod \"f5999941-d60a-412e-a5d8-8f4e657e0014\" (UID: \"f5999941-d60a-412e-a5d8-8f4e657e0014\") " Dec 03 12:25:54 crc kubenswrapper[5052]: I1203 12:25:54.231700 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/f5999941-d60a-412e-a5d8-8f4e657e0014-config-data\") pod \"f5999941-d60a-412e-a5d8-8f4e657e0014\" (UID: \"f5999941-d60a-412e-a5d8-8f4e657e0014\") " Dec 03 12:25:54 crc kubenswrapper[5052]: I1203 12:25:54.231741 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f5999941-d60a-412e-a5d8-8f4e657e0014-combined-ca-bundle\") pod \"f5999941-d60a-412e-a5d8-8f4e657e0014\" (UID: \"f5999941-d60a-412e-a5d8-8f4e657e0014\") " Dec 03 12:25:54 crc kubenswrapper[5052]: I1203 12:25:54.238168 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f5999941-d60a-412e-a5d8-8f4e657e0014-scripts" (OuterVolumeSpecName: "scripts") pod "f5999941-d60a-412e-a5d8-8f4e657e0014" (UID: "f5999941-d60a-412e-a5d8-8f4e657e0014"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:25:54 crc kubenswrapper[5052]: I1203 12:25:54.256053 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f5999941-d60a-412e-a5d8-8f4e657e0014-kube-api-access-8ttbs" (OuterVolumeSpecName: "kube-api-access-8ttbs") pod "f5999941-d60a-412e-a5d8-8f4e657e0014" (UID: "f5999941-d60a-412e-a5d8-8f4e657e0014"). InnerVolumeSpecName "kube-api-access-8ttbs". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:25:54 crc kubenswrapper[5052]: I1203 12:25:54.274468 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f5999941-d60a-412e-a5d8-8f4e657e0014-config-data" (OuterVolumeSpecName: "config-data") pod "f5999941-d60a-412e-a5d8-8f4e657e0014" (UID: "f5999941-d60a-412e-a5d8-8f4e657e0014"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:25:54 crc kubenswrapper[5052]: I1203 12:25:54.282673 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f5999941-d60a-412e-a5d8-8f4e657e0014-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "f5999941-d60a-412e-a5d8-8f4e657e0014" (UID: "f5999941-d60a-412e-a5d8-8f4e657e0014"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:25:54 crc kubenswrapper[5052]: I1203 12:25:54.333726 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/f5999941-d60a-412e-a5d8-8f4e657e0014-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:25:54 crc kubenswrapper[5052]: I1203 12:25:54.333756 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f5999941-d60a-412e-a5d8-8f4e657e0014-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:25:54 crc kubenswrapper[5052]: I1203 12:25:54.333765 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/f5999941-d60a-412e-a5d8-8f4e657e0014-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:25:54 crc kubenswrapper[5052]: I1203 12:25:54.333775 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-8ttbs\" (UniqueName: \"kubernetes.io/projected/f5999941-d60a-412e-a5d8-8f4e657e0014-kube-api-access-8ttbs\") on node \"crc\" DevicePath \"\"" Dec 03 12:25:54 crc kubenswrapper[5052]: I1203 12:25:54.649536 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-cell-mapping-d2r55" Dec 03 12:25:54 crc kubenswrapper[5052]: I1203 12:25:54.656544 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-cell-mapping-d2r55" event={"ID":"f5999941-d60a-412e-a5d8-8f4e657e0014","Type":"ContainerDied","Data":"40a22548a28c69710420db1ebd5a411abcc276bbd6884ba1ab5538c145ccfae4"} Dec 03 12:25:54 crc kubenswrapper[5052]: I1203 12:25:54.656602 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="40a22548a28c69710420db1ebd5a411abcc276bbd6884ba1ab5538c145ccfae4" Dec 03 12:25:54 crc kubenswrapper[5052]: I1203 12:25:54.855464 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:25:54 crc kubenswrapper[5052]: I1203 12:25:54.855757 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-api-0" podUID="abc55d2f-ccfa-495d-a8a6-a6f77d7c6359" containerName="nova-api-log" containerID="cri-o://d37d1d57c20d5070c9980e946f4ab10fe33b7925b8fab9cd3ca9d1f75f226131" gracePeriod=30 Dec 03 12:25:54 crc kubenswrapper[5052]: I1203 12:25:54.855849 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-api-0" podUID="abc55d2f-ccfa-495d-a8a6-a6f77d7c6359" containerName="nova-api-api" containerID="cri-o://6e694094de1b5f6f8bfbd62dac1a726406c6c627116d18c9c852c2c84f407c7b" gracePeriod=30 Dec 03 12:25:54 crc kubenswrapper[5052]: I1203 12:25:54.880463 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:25:54 crc kubenswrapper[5052]: I1203 12:25:54.890808 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:25:54 crc kubenswrapper[5052]: I1203 12:25:54.891069 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-metadata-0" podUID="5f909f2c-2366-4056-9cb5-c03a66b8b1ba" containerName="nova-metadata-log" containerID="cri-o://28b063d8d55d18da9d42985295c475e98897f1eee57fedd6d01cc380cb473a83" gracePeriod=30 Dec 03 12:25:54 crc kubenswrapper[5052]: I1203 12:25:54.891166 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-metadata-0" podUID="5f909f2c-2366-4056-9cb5-c03a66b8b1ba" containerName="nova-metadata-metadata" containerID="cri-o://69fd0c98744de04271306fab61a50f33147641fa4e4c2ab8eec766382b02abd5" gracePeriod=30 Dec 03 12:25:55 crc kubenswrapper[5052]: I1203 12:25:55.663888 5052 generic.go:334] "Generic (PLEG): container finished" podID="abc55d2f-ccfa-495d-a8a6-a6f77d7c6359" containerID="d37d1d57c20d5070c9980e946f4ab10fe33b7925b8fab9cd3ca9d1f75f226131" exitCode=143 Dec 03 12:25:55 crc kubenswrapper[5052]: I1203 12:25:55.664034 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"abc55d2f-ccfa-495d-a8a6-a6f77d7c6359","Type":"ContainerDied","Data":"d37d1d57c20d5070c9980e946f4ab10fe33b7925b8fab9cd3ca9d1f75f226131"} Dec 03 12:25:55 crc kubenswrapper[5052]: I1203 12:25:55.666374 5052 generic.go:334] "Generic (PLEG): container finished" podID="5f909f2c-2366-4056-9cb5-c03a66b8b1ba" containerID="28b063d8d55d18da9d42985295c475e98897f1eee57fedd6d01cc380cb473a83" exitCode=143 Dec 03 12:25:55 crc kubenswrapper[5052]: I1203 12:25:55.666541 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-scheduler-0" podUID="f3f2d6a8-014d-493a-ad53-c68acf01ce61" containerName="nova-scheduler-scheduler" containerID="cri-o://1219127b0f28f232cebab108b969dfccbae824421c1a95a3735263eb1e8796a0" gracePeriod=30 Dec 03 12:25:55 crc kubenswrapper[5052]: I1203 12:25:55.666776 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"5f909f2c-2366-4056-9cb5-c03a66b8b1ba","Type":"ContainerDied","Data":"28b063d8d55d18da9d42985295c475e98897f1eee57fedd6d01cc380cb473a83"} Dec 03 12:25:57 crc kubenswrapper[5052]: I1203 12:25:57.688261 5052 generic.go:334] "Generic (PLEG): container finished" podID="f3f2d6a8-014d-493a-ad53-c68acf01ce61" containerID="1219127b0f28f232cebab108b969dfccbae824421c1a95a3735263eb1e8796a0" exitCode=0 Dec 03 12:25:57 crc kubenswrapper[5052]: I1203 12:25:57.689100 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"f3f2d6a8-014d-493a-ad53-c68acf01ce61","Type":"ContainerDied","Data":"1219127b0f28f232cebab108b969dfccbae824421c1a95a3735263eb1e8796a0"} Dec 03 12:25:57 crc kubenswrapper[5052]: I1203 12:25:57.689142 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"f3f2d6a8-014d-493a-ad53-c68acf01ce61","Type":"ContainerDied","Data":"4af2921b9ae039bf9cf7543218dde1530b3da62319d5eac483c65c456f293b04"} Dec 03 12:25:57 crc kubenswrapper[5052]: I1203 12:25:57.689157 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4af2921b9ae039bf9cf7543218dde1530b3da62319d5eac483c65c456f293b04" Dec 03 12:25:57 crc kubenswrapper[5052]: I1203 12:25:57.704931 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:25:57 crc kubenswrapper[5052]: I1203 12:25:57.798155 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pxtw2\" (UniqueName: \"kubernetes.io/projected/f3f2d6a8-014d-493a-ad53-c68acf01ce61-kube-api-access-pxtw2\") pod \"f3f2d6a8-014d-493a-ad53-c68acf01ce61\" (UID: \"f3f2d6a8-014d-493a-ad53-c68acf01ce61\") " Dec 03 12:25:57 crc kubenswrapper[5052]: I1203 12:25:57.798240 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/f3f2d6a8-014d-493a-ad53-c68acf01ce61-config-data\") pod \"f3f2d6a8-014d-493a-ad53-c68acf01ce61\" (UID: \"f3f2d6a8-014d-493a-ad53-c68acf01ce61\") " Dec 03 12:25:57 crc kubenswrapper[5052]: I1203 12:25:57.798305 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f3f2d6a8-014d-493a-ad53-c68acf01ce61-combined-ca-bundle\") pod \"f3f2d6a8-014d-493a-ad53-c68acf01ce61\" (UID: \"f3f2d6a8-014d-493a-ad53-c68acf01ce61\") " Dec 03 12:25:57 crc kubenswrapper[5052]: I1203 12:25:57.815611 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f3f2d6a8-014d-493a-ad53-c68acf01ce61-kube-api-access-pxtw2" (OuterVolumeSpecName: "kube-api-access-pxtw2") pod "f3f2d6a8-014d-493a-ad53-c68acf01ce61" (UID: "f3f2d6a8-014d-493a-ad53-c68acf01ce61"). InnerVolumeSpecName "kube-api-access-pxtw2". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:25:57 crc kubenswrapper[5052]: I1203 12:25:57.861258 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f3f2d6a8-014d-493a-ad53-c68acf01ce61-config-data" (OuterVolumeSpecName: "config-data") pod "f3f2d6a8-014d-493a-ad53-c68acf01ce61" (UID: "f3f2d6a8-014d-493a-ad53-c68acf01ce61"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:25:57 crc kubenswrapper[5052]: I1203 12:25:57.901787 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-pxtw2\" (UniqueName: \"kubernetes.io/projected/f3f2d6a8-014d-493a-ad53-c68acf01ce61-kube-api-access-pxtw2\") on node \"crc\" DevicePath \"\"" Dec 03 12:25:57 crc kubenswrapper[5052]: I1203 12:25:57.901817 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/f3f2d6a8-014d-493a-ad53-c68acf01ce61-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:25:57 crc kubenswrapper[5052]: I1203 12:25:57.907330 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f3f2d6a8-014d-493a-ad53-c68acf01ce61-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "f3f2d6a8-014d-493a-ad53-c68acf01ce61" (UID: "f3f2d6a8-014d-493a-ad53-c68acf01ce61"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:25:58 crc kubenswrapper[5052]: I1203 12:25:58.003534 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f3f2d6a8-014d-493a-ad53-c68acf01ce61-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:25:58 crc kubenswrapper[5052]: I1203 12:25:58.701858 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:25:58 crc kubenswrapper[5052]: I1203 12:25:58.738517 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:25:58 crc kubenswrapper[5052]: I1203 12:25:58.748900 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:25:58 crc kubenswrapper[5052]: I1203 12:25:58.760921 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:25:58 crc kubenswrapper[5052]: E1203 12:25:58.761419 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="f5999941-d60a-412e-a5d8-8f4e657e0014" containerName="nova-manage" Dec 03 12:25:58 crc kubenswrapper[5052]: I1203 12:25:58.761439 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="f5999941-d60a-412e-a5d8-8f4e657e0014" containerName="nova-manage" Dec 03 12:25:58 crc kubenswrapper[5052]: E1203 12:25:58.761461 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="f3f2d6a8-014d-493a-ad53-c68acf01ce61" containerName="nova-scheduler-scheduler" Dec 03 12:25:58 crc kubenswrapper[5052]: I1203 12:25:58.761468 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="f3f2d6a8-014d-493a-ad53-c68acf01ce61" containerName="nova-scheduler-scheduler" Dec 03 12:25:58 crc kubenswrapper[5052]: I1203 12:25:58.761729 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="f5999941-d60a-412e-a5d8-8f4e657e0014" containerName="nova-manage" Dec 03 12:25:58 crc kubenswrapper[5052]: I1203 12:25:58.761745 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="f3f2d6a8-014d-493a-ad53-c68acf01ce61" containerName="nova-scheduler-scheduler" Dec 03 12:25:58 crc kubenswrapper[5052]: I1203 12:25:58.762455 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:25:58 crc kubenswrapper[5052]: I1203 12:25:58.764705 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-scheduler-config-data" Dec 03 12:25:58 crc kubenswrapper[5052]: I1203 12:25:58.770145 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:25:58 crc kubenswrapper[5052]: I1203 12:25:58.816960 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5ee46523-66cc-414a-b720-542a1abc4021-combined-ca-bundle\") pod \"nova-scheduler-0\" (UID: \"5ee46523-66cc-414a-b720-542a1abc4021\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:25:58 crc kubenswrapper[5052]: I1203 12:25:58.817161 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cq4b2\" (UniqueName: \"kubernetes.io/projected/5ee46523-66cc-414a-b720-542a1abc4021-kube-api-access-cq4b2\") pod \"nova-scheduler-0\" (UID: \"5ee46523-66cc-414a-b720-542a1abc4021\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:25:58 crc kubenswrapper[5052]: I1203 12:25:58.817320 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/5ee46523-66cc-414a-b720-542a1abc4021-config-data\") pod \"nova-scheduler-0\" (UID: \"5ee46523-66cc-414a-b720-542a1abc4021\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:25:58 crc kubenswrapper[5052]: I1203 12:25:58.919083 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5ee46523-66cc-414a-b720-542a1abc4021-combined-ca-bundle\") pod \"nova-scheduler-0\" (UID: \"5ee46523-66cc-414a-b720-542a1abc4021\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:25:58 crc kubenswrapper[5052]: I1203 12:25:58.919183 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-cq4b2\" (UniqueName: \"kubernetes.io/projected/5ee46523-66cc-414a-b720-542a1abc4021-kube-api-access-cq4b2\") pod \"nova-scheduler-0\" (UID: \"5ee46523-66cc-414a-b720-542a1abc4021\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:25:58 crc kubenswrapper[5052]: I1203 12:25:58.919234 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/5ee46523-66cc-414a-b720-542a1abc4021-config-data\") pod \"nova-scheduler-0\" (UID: \"5ee46523-66cc-414a-b720-542a1abc4021\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:25:58 crc kubenswrapper[5052]: I1203 12:25:58.926301 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5ee46523-66cc-414a-b720-542a1abc4021-combined-ca-bundle\") pod \"nova-scheduler-0\" (UID: \"5ee46523-66cc-414a-b720-542a1abc4021\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:25:58 crc kubenswrapper[5052]: I1203 12:25:58.927280 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/5ee46523-66cc-414a-b720-542a1abc4021-config-data\") pod \"nova-scheduler-0\" (UID: \"5ee46523-66cc-414a-b720-542a1abc4021\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:25:58 crc kubenswrapper[5052]: I1203 12:25:58.938020 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-cq4b2\" (UniqueName: \"kubernetes.io/projected/5ee46523-66cc-414a-b720-542a1abc4021-kube-api-access-cq4b2\") pod \"nova-scheduler-0\" (UID: \"5ee46523-66cc-414a-b720-542a1abc4021\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:25:59 crc kubenswrapper[5052]: I1203 12:25:59.095350 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:25:59 crc kubenswrapper[5052]: I1203 12:25:59.582702 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:25:59 crc kubenswrapper[5052]: I1203 12:25:59.712840 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f3f2d6a8-014d-493a-ad53-c68acf01ce61" path="/var/lib/kubelet/pods/f3f2d6a8-014d-493a-ad53-c68acf01ce61/volumes" Dec 03 12:25:59 crc kubenswrapper[5052]: I1203 12:25:59.717853 5052 generic.go:334] "Generic (PLEG): container finished" podID="5f909f2c-2366-4056-9cb5-c03a66b8b1ba" containerID="69fd0c98744de04271306fab61a50f33147641fa4e4c2ab8eec766382b02abd5" exitCode=0 Dec 03 12:25:59 crc kubenswrapper[5052]: I1203 12:25:59.718087 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"5f909f2c-2366-4056-9cb5-c03a66b8b1ba","Type":"ContainerDied","Data":"69fd0c98744de04271306fab61a50f33147641fa4e4c2ab8eec766382b02abd5"} Dec 03 12:25:59 crc kubenswrapper[5052]: I1203 12:25:59.726214 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"5ee46523-66cc-414a-b720-542a1abc4021","Type":"ContainerStarted","Data":"4a4b2bff6ebdad7eb07a577a7edcf9cb015dce95f70c2b8107c1656c94d0a869"} Dec 03 12:25:59 crc kubenswrapper[5052]: I1203 12:25:59.747535 5052 generic.go:334] "Generic (PLEG): container finished" podID="abc55d2f-ccfa-495d-a8a6-a6f77d7c6359" containerID="6e694094de1b5f6f8bfbd62dac1a726406c6c627116d18c9c852c2c84f407c7b" exitCode=0 Dec 03 12:25:59 crc kubenswrapper[5052]: I1203 12:25:59.747582 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"abc55d2f-ccfa-495d-a8a6-a6f77d7c6359","Type":"ContainerDied","Data":"6e694094de1b5f6f8bfbd62dac1a726406c6c627116d18c9c852c2c84f407c7b"} Dec 03 12:25:59 crc kubenswrapper[5052]: I1203 12:25:59.850081 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:25:59 crc kubenswrapper[5052]: I1203 12:25:59.882419 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:25:59 crc kubenswrapper[5052]: I1203 12:25:59.941337 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/abc55d2f-ccfa-495d-a8a6-a6f77d7c6359-combined-ca-bundle\") pod \"abc55d2f-ccfa-495d-a8a6-a6f77d7c6359\" (UID: \"abc55d2f-ccfa-495d-a8a6-a6f77d7c6359\") " Dec 03 12:25:59 crc kubenswrapper[5052]: I1203 12:25:59.941409 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/abc55d2f-ccfa-495d-a8a6-a6f77d7c6359-logs\") pod \"abc55d2f-ccfa-495d-a8a6-a6f77d7c6359\" (UID: \"abc55d2f-ccfa-495d-a8a6-a6f77d7c6359\") " Dec 03 12:25:59 crc kubenswrapper[5052]: I1203 12:25:59.941537 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/5f909f2c-2366-4056-9cb5-c03a66b8b1ba-logs\") pod \"5f909f2c-2366-4056-9cb5-c03a66b8b1ba\" (UID: \"5f909f2c-2366-4056-9cb5-c03a66b8b1ba\") " Dec 03 12:25:59 crc kubenswrapper[5052]: I1203 12:25:59.941635 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kwz4d\" (UniqueName: \"kubernetes.io/projected/abc55d2f-ccfa-495d-a8a6-a6f77d7c6359-kube-api-access-kwz4d\") pod \"abc55d2f-ccfa-495d-a8a6-a6f77d7c6359\" (UID: \"abc55d2f-ccfa-495d-a8a6-a6f77d7c6359\") " Dec 03 12:25:59 crc kubenswrapper[5052]: I1203 12:25:59.941691 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mc2k6\" (UniqueName: \"kubernetes.io/projected/5f909f2c-2366-4056-9cb5-c03a66b8b1ba-kube-api-access-mc2k6\") pod \"5f909f2c-2366-4056-9cb5-c03a66b8b1ba\" (UID: \"5f909f2c-2366-4056-9cb5-c03a66b8b1ba\") " Dec 03 12:25:59 crc kubenswrapper[5052]: I1203 12:25:59.941710 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/5f909f2c-2366-4056-9cb5-c03a66b8b1ba-config-data\") pod \"5f909f2c-2366-4056-9cb5-c03a66b8b1ba\" (UID: \"5f909f2c-2366-4056-9cb5-c03a66b8b1ba\") " Dec 03 12:25:59 crc kubenswrapper[5052]: I1203 12:25:59.941737 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5f909f2c-2366-4056-9cb5-c03a66b8b1ba-combined-ca-bundle\") pod \"5f909f2c-2366-4056-9cb5-c03a66b8b1ba\" (UID: \"5f909f2c-2366-4056-9cb5-c03a66b8b1ba\") " Dec 03 12:25:59 crc kubenswrapper[5052]: I1203 12:25:59.941762 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/abc55d2f-ccfa-495d-a8a6-a6f77d7c6359-config-data\") pod \"abc55d2f-ccfa-495d-a8a6-a6f77d7c6359\" (UID: \"abc55d2f-ccfa-495d-a8a6-a6f77d7c6359\") " Dec 03 12:25:59 crc kubenswrapper[5052]: I1203 12:25:59.942123 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f909f2c-2366-4056-9cb5-c03a66b8b1ba-logs" (OuterVolumeSpecName: "logs") pod "5f909f2c-2366-4056-9cb5-c03a66b8b1ba" (UID: "5f909f2c-2366-4056-9cb5-c03a66b8b1ba"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:25:59 crc kubenswrapper[5052]: I1203 12:25:59.942150 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/abc55d2f-ccfa-495d-a8a6-a6f77d7c6359-logs" (OuterVolumeSpecName: "logs") pod "abc55d2f-ccfa-495d-a8a6-a6f77d7c6359" (UID: "abc55d2f-ccfa-495d-a8a6-a6f77d7c6359"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:25:59 crc kubenswrapper[5052]: I1203 12:25:59.942590 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/abc55d2f-ccfa-495d-a8a6-a6f77d7c6359-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:25:59 crc kubenswrapper[5052]: I1203 12:25:59.942606 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/5f909f2c-2366-4056-9cb5-c03a66b8b1ba-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:25:59 crc kubenswrapper[5052]: I1203 12:25:59.945564 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5f909f2c-2366-4056-9cb5-c03a66b8b1ba-kube-api-access-mc2k6" (OuterVolumeSpecName: "kube-api-access-mc2k6") pod "5f909f2c-2366-4056-9cb5-c03a66b8b1ba" (UID: "5f909f2c-2366-4056-9cb5-c03a66b8b1ba"). InnerVolumeSpecName "kube-api-access-mc2k6". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:25:59 crc kubenswrapper[5052]: I1203 12:25:59.946753 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/abc55d2f-ccfa-495d-a8a6-a6f77d7c6359-kube-api-access-kwz4d" (OuterVolumeSpecName: "kube-api-access-kwz4d") pod "abc55d2f-ccfa-495d-a8a6-a6f77d7c6359" (UID: "abc55d2f-ccfa-495d-a8a6-a6f77d7c6359"). InnerVolumeSpecName "kube-api-access-kwz4d". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:25:59 crc kubenswrapper[5052]: I1203 12:25:59.969761 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/abc55d2f-ccfa-495d-a8a6-a6f77d7c6359-config-data" (OuterVolumeSpecName: "config-data") pod "abc55d2f-ccfa-495d-a8a6-a6f77d7c6359" (UID: "abc55d2f-ccfa-495d-a8a6-a6f77d7c6359"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:25:59 crc kubenswrapper[5052]: I1203 12:25:59.977889 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/abc55d2f-ccfa-495d-a8a6-a6f77d7c6359-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "abc55d2f-ccfa-495d-a8a6-a6f77d7c6359" (UID: "abc55d2f-ccfa-495d-a8a6-a6f77d7c6359"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:25:59 crc kubenswrapper[5052]: I1203 12:25:59.982211 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5f909f2c-2366-4056-9cb5-c03a66b8b1ba-config-data" (OuterVolumeSpecName: "config-data") pod "5f909f2c-2366-4056-9cb5-c03a66b8b1ba" (UID: "5f909f2c-2366-4056-9cb5-c03a66b8b1ba"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:25:59 crc kubenswrapper[5052]: I1203 12:25:59.984529 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5f909f2c-2366-4056-9cb5-c03a66b8b1ba-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "5f909f2c-2366-4056-9cb5-c03a66b8b1ba" (UID: "5f909f2c-2366-4056-9cb5-c03a66b8b1ba"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:26:00 crc kubenswrapper[5052]: I1203 12:26:00.044045 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-kwz4d\" (UniqueName: \"kubernetes.io/projected/abc55d2f-ccfa-495d-a8a6-a6f77d7c6359-kube-api-access-kwz4d\") on node \"crc\" DevicePath \"\"" Dec 03 12:26:00 crc kubenswrapper[5052]: I1203 12:26:00.044081 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-mc2k6\" (UniqueName: \"kubernetes.io/projected/5f909f2c-2366-4056-9cb5-c03a66b8b1ba-kube-api-access-mc2k6\") on node \"crc\" DevicePath \"\"" Dec 03 12:26:00 crc kubenswrapper[5052]: I1203 12:26:00.044092 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/5f909f2c-2366-4056-9cb5-c03a66b8b1ba-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:26:00 crc kubenswrapper[5052]: I1203 12:26:00.044102 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5f909f2c-2366-4056-9cb5-c03a66b8b1ba-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:26:00 crc kubenswrapper[5052]: I1203 12:26:00.044110 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/abc55d2f-ccfa-495d-a8a6-a6f77d7c6359-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:26:00 crc kubenswrapper[5052]: I1203 12:26:00.044117 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/abc55d2f-ccfa-495d-a8a6-a6f77d7c6359-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:26:00 crc kubenswrapper[5052]: I1203 12:26:00.761678 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"5ee46523-66cc-414a-b720-542a1abc4021","Type":"ContainerStarted","Data":"72aac272f84d0737e78fd219a9666e2be0d21ec7dd8418b6d1d7be63eac32c80"} Dec 03 12:26:00 crc kubenswrapper[5052]: I1203 12:26:00.764121 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"abc55d2f-ccfa-495d-a8a6-a6f77d7c6359","Type":"ContainerDied","Data":"20aa87d8eac378f8d314811d60a1b63e602cbf8517927db7a5212c4cc5196336"} Dec 03 12:26:00 crc kubenswrapper[5052]: I1203 12:26:00.764188 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:26:00 crc kubenswrapper[5052]: I1203 12:26:00.764232 5052 scope.go:117] "RemoveContainer" containerID="6e694094de1b5f6f8bfbd62dac1a726406c6c627116d18c9c852c2c84f407c7b" Dec 03 12:26:00 crc kubenswrapper[5052]: I1203 12:26:00.768913 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"5f909f2c-2366-4056-9cb5-c03a66b8b1ba","Type":"ContainerDied","Data":"fbc1cb63e9f75a77327225b4dca8b5ebbfa70a678518f07852bd3eef5d78d271"} Dec 03 12:26:00 crc kubenswrapper[5052]: I1203 12:26:00.768991 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:26:00 crc kubenswrapper[5052]: I1203 12:26:00.795879 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-scheduler-0" podStartSLOduration=2.795854475 podStartE2EDuration="2.795854475s" podCreationTimestamp="2025-12-03 12:25:58 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:26:00.779947381 +0000 UTC m=+3951.368189155" watchObservedRunningTime="2025-12-03 12:26:00.795854475 +0000 UTC m=+3951.384096239" Dec 03 12:26:00 crc kubenswrapper[5052]: I1203 12:26:00.798872 5052 scope.go:117] "RemoveContainer" containerID="d37d1d57c20d5070c9980e946f4ab10fe33b7925b8fab9cd3ca9d1f75f226131" Dec 03 12:26:00 crc kubenswrapper[5052]: I1203 12:26:00.820961 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:26:00 crc kubenswrapper[5052]: I1203 12:26:00.840354 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:26:00 crc kubenswrapper[5052]: I1203 12:26:00.847576 5052 scope.go:117] "RemoveContainer" containerID="69fd0c98744de04271306fab61a50f33147641fa4e4c2ab8eec766382b02abd5" Dec 03 12:26:00 crc kubenswrapper[5052]: I1203 12:26:00.853191 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:26:00 crc kubenswrapper[5052]: I1203 12:26:00.866608 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:26:00 crc kubenswrapper[5052]: I1203 12:26:00.882372 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:26:00 crc kubenswrapper[5052]: E1203 12:26:00.883017 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="abc55d2f-ccfa-495d-a8a6-a6f77d7c6359" containerName="nova-api-api" Dec 03 12:26:00 crc kubenswrapper[5052]: I1203 12:26:00.883033 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="abc55d2f-ccfa-495d-a8a6-a6f77d7c6359" containerName="nova-api-api" Dec 03 12:26:00 crc kubenswrapper[5052]: E1203 12:26:00.883049 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="5f909f2c-2366-4056-9cb5-c03a66b8b1ba" containerName="nova-metadata-log" Dec 03 12:26:00 crc kubenswrapper[5052]: I1203 12:26:00.883057 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f909f2c-2366-4056-9cb5-c03a66b8b1ba" containerName="nova-metadata-log" Dec 03 12:26:00 crc kubenswrapper[5052]: E1203 12:26:00.883098 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="abc55d2f-ccfa-495d-a8a6-a6f77d7c6359" containerName="nova-api-log" Dec 03 12:26:00 crc kubenswrapper[5052]: I1203 12:26:00.883105 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="abc55d2f-ccfa-495d-a8a6-a6f77d7c6359" containerName="nova-api-log" Dec 03 12:26:00 crc kubenswrapper[5052]: E1203 12:26:00.883121 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="5f909f2c-2366-4056-9cb5-c03a66b8b1ba" containerName="nova-metadata-metadata" Dec 03 12:26:00 crc kubenswrapper[5052]: I1203 12:26:00.883128 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f909f2c-2366-4056-9cb5-c03a66b8b1ba" containerName="nova-metadata-metadata" Dec 03 12:26:00 crc kubenswrapper[5052]: I1203 12:26:00.883355 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="abc55d2f-ccfa-495d-a8a6-a6f77d7c6359" containerName="nova-api-log" Dec 03 12:26:00 crc kubenswrapper[5052]: I1203 12:26:00.883368 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="5f909f2c-2366-4056-9cb5-c03a66b8b1ba" containerName="nova-metadata-metadata" Dec 03 12:26:00 crc kubenswrapper[5052]: I1203 12:26:00.883382 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="5f909f2c-2366-4056-9cb5-c03a66b8b1ba" containerName="nova-metadata-log" Dec 03 12:26:00 crc kubenswrapper[5052]: I1203 12:26:00.883400 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="abc55d2f-ccfa-495d-a8a6-a6f77d7c6359" containerName="nova-api-api" Dec 03 12:26:00 crc kubenswrapper[5052]: I1203 12:26:00.884918 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:26:00 crc kubenswrapper[5052]: I1203 12:26:00.890682 5052 scope.go:117] "RemoveContainer" containerID="28b063d8d55d18da9d42985295c475e98897f1eee57fedd6d01cc380cb473a83" Dec 03 12:26:00 crc kubenswrapper[5052]: I1203 12:26:00.891187 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-metadata-config-data" Dec 03 12:26:00 crc kubenswrapper[5052]: I1203 12:26:00.893541 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:26:00 crc kubenswrapper[5052]: I1203 12:26:00.895727 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:26:00 crc kubenswrapper[5052]: I1203 12:26:00.901608 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-api-config-data" Dec 03 12:26:00 crc kubenswrapper[5052]: I1203 12:26:00.903037 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:26:00 crc kubenswrapper[5052]: I1203 12:26:00.917308 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:26:00 crc kubenswrapper[5052]: I1203 12:26:00.960199 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/0a52121e-4974-4d01-977b-71b17710e9bd-config-data\") pod \"nova-metadata-0\" (UID: \"0a52121e-4974-4d01-977b-71b17710e9bd\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:26:00 crc kubenswrapper[5052]: I1203 12:26:00.960263 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/0a52121e-4974-4d01-977b-71b17710e9bd-logs\") pod \"nova-metadata-0\" (UID: \"0a52121e-4974-4d01-977b-71b17710e9bd\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:26:00 crc kubenswrapper[5052]: I1203 12:26:00.960302 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0a52121e-4974-4d01-977b-71b17710e9bd-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"0a52121e-4974-4d01-977b-71b17710e9bd\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:26:00 crc kubenswrapper[5052]: I1203 12:26:00.960361 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/4406519b-7670-42e0-9fd2-b81b12401e90-logs\") pod \"nova-api-0\" (UID: \"4406519b-7670-42e0-9fd2-b81b12401e90\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:26:00 crc kubenswrapper[5052]: I1203 12:26:00.960391 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/4406519b-7670-42e0-9fd2-b81b12401e90-config-data\") pod \"nova-api-0\" (UID: \"4406519b-7670-42e0-9fd2-b81b12401e90\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:26:00 crc kubenswrapper[5052]: I1203 12:26:00.960416 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r8z4v\" (UniqueName: \"kubernetes.io/projected/4406519b-7670-42e0-9fd2-b81b12401e90-kube-api-access-r8z4v\") pod \"nova-api-0\" (UID: \"4406519b-7670-42e0-9fd2-b81b12401e90\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:26:00 crc kubenswrapper[5052]: I1203 12:26:00.960484 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kdgw7\" (UniqueName: \"kubernetes.io/projected/0a52121e-4974-4d01-977b-71b17710e9bd-kube-api-access-kdgw7\") pod \"nova-metadata-0\" (UID: \"0a52121e-4974-4d01-977b-71b17710e9bd\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:26:00 crc kubenswrapper[5052]: I1203 12:26:00.960570 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4406519b-7670-42e0-9fd2-b81b12401e90-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"4406519b-7670-42e0-9fd2-b81b12401e90\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:26:01 crc kubenswrapper[5052]: I1203 12:26:01.062093 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/0a52121e-4974-4d01-977b-71b17710e9bd-config-data\") pod \"nova-metadata-0\" (UID: \"0a52121e-4974-4d01-977b-71b17710e9bd\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:26:01 crc kubenswrapper[5052]: I1203 12:26:01.062145 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/0a52121e-4974-4d01-977b-71b17710e9bd-logs\") pod \"nova-metadata-0\" (UID: \"0a52121e-4974-4d01-977b-71b17710e9bd\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:26:01 crc kubenswrapper[5052]: I1203 12:26:01.062172 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0a52121e-4974-4d01-977b-71b17710e9bd-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"0a52121e-4974-4d01-977b-71b17710e9bd\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:26:01 crc kubenswrapper[5052]: I1203 12:26:01.062222 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/4406519b-7670-42e0-9fd2-b81b12401e90-logs\") pod \"nova-api-0\" (UID: \"4406519b-7670-42e0-9fd2-b81b12401e90\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:26:01 crc kubenswrapper[5052]: I1203 12:26:01.062248 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/4406519b-7670-42e0-9fd2-b81b12401e90-config-data\") pod \"nova-api-0\" (UID: \"4406519b-7670-42e0-9fd2-b81b12401e90\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:26:01 crc kubenswrapper[5052]: I1203 12:26:01.062272 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-r8z4v\" (UniqueName: \"kubernetes.io/projected/4406519b-7670-42e0-9fd2-b81b12401e90-kube-api-access-r8z4v\") pod \"nova-api-0\" (UID: \"4406519b-7670-42e0-9fd2-b81b12401e90\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:26:01 crc kubenswrapper[5052]: I1203 12:26:01.062299 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-kdgw7\" (UniqueName: \"kubernetes.io/projected/0a52121e-4974-4d01-977b-71b17710e9bd-kube-api-access-kdgw7\") pod \"nova-metadata-0\" (UID: \"0a52121e-4974-4d01-977b-71b17710e9bd\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:26:01 crc kubenswrapper[5052]: I1203 12:26:01.062337 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4406519b-7670-42e0-9fd2-b81b12401e90-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"4406519b-7670-42e0-9fd2-b81b12401e90\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:26:01 crc kubenswrapper[5052]: I1203 12:26:01.063548 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/4406519b-7670-42e0-9fd2-b81b12401e90-logs\") pod \"nova-api-0\" (UID: \"4406519b-7670-42e0-9fd2-b81b12401e90\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:26:01 crc kubenswrapper[5052]: I1203 12:26:01.064575 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/0a52121e-4974-4d01-977b-71b17710e9bd-logs\") pod \"nova-metadata-0\" (UID: \"0a52121e-4974-4d01-977b-71b17710e9bd\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:26:01 crc kubenswrapper[5052]: I1203 12:26:01.067916 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4406519b-7670-42e0-9fd2-b81b12401e90-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"4406519b-7670-42e0-9fd2-b81b12401e90\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:26:01 crc kubenswrapper[5052]: I1203 12:26:01.068120 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0a52121e-4974-4d01-977b-71b17710e9bd-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"0a52121e-4974-4d01-977b-71b17710e9bd\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:26:01 crc kubenswrapper[5052]: I1203 12:26:01.072384 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/0a52121e-4974-4d01-977b-71b17710e9bd-config-data\") pod \"nova-metadata-0\" (UID: \"0a52121e-4974-4d01-977b-71b17710e9bd\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:26:01 crc kubenswrapper[5052]: I1203 12:26:01.072448 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/4406519b-7670-42e0-9fd2-b81b12401e90-config-data\") pod \"nova-api-0\" (UID: \"4406519b-7670-42e0-9fd2-b81b12401e90\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:26:01 crc kubenswrapper[5052]: I1203 12:26:01.082831 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-r8z4v\" (UniqueName: \"kubernetes.io/projected/4406519b-7670-42e0-9fd2-b81b12401e90-kube-api-access-r8z4v\") pod \"nova-api-0\" (UID: \"4406519b-7670-42e0-9fd2-b81b12401e90\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:26:01 crc kubenswrapper[5052]: I1203 12:26:01.083647 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-kdgw7\" (UniqueName: \"kubernetes.io/projected/0a52121e-4974-4d01-977b-71b17710e9bd-kube-api-access-kdgw7\") pod \"nova-metadata-0\" (UID: \"0a52121e-4974-4d01-977b-71b17710e9bd\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:26:01 crc kubenswrapper[5052]: I1203 12:26:01.215614 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:26:01 crc kubenswrapper[5052]: I1203 12:26:01.231362 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:26:01 crc kubenswrapper[5052]: I1203 12:26:01.567240 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:26:01 crc kubenswrapper[5052]: I1203 12:26:01.686284 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5f909f2c-2366-4056-9cb5-c03a66b8b1ba" path="/var/lib/kubelet/pods/5f909f2c-2366-4056-9cb5-c03a66b8b1ba/volumes" Dec 03 12:26:01 crc kubenswrapper[5052]: I1203 12:26:01.687892 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="abc55d2f-ccfa-495d-a8a6-a6f77d7c6359" path="/var/lib/kubelet/pods/abc55d2f-ccfa-495d-a8a6-a6f77d7c6359/volumes" Dec 03 12:26:01 crc kubenswrapper[5052]: I1203 12:26:01.783645 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"0a52121e-4974-4d01-977b-71b17710e9bd","Type":"ContainerStarted","Data":"d210d37650a68534c9f6cfb73cbc169bce24cd3a86fb2b62f730b0245234dd17"} Dec 03 12:26:01 crc kubenswrapper[5052]: I1203 12:26:01.801244 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:26:01 crc kubenswrapper[5052]: I1203 12:26:01.811233 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:26:01 crc kubenswrapper[5052]: W1203 12:26:01.820867 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4406519b_7670_42e0_9fd2_b81b12401e90.slice/crio-fc91c2fb3a4da3931904532e155fb104636a8874eef93e70df5a6aa688151cba WatchSource:0}: Error finding container fc91c2fb3a4da3931904532e155fb104636a8874eef93e70df5a6aa688151cba: Status 404 returned error can't find the container with id fc91c2fb3a4da3931904532e155fb104636a8874eef93e70df5a6aa688151cba Dec 03 12:26:02 crc kubenswrapper[5052]: I1203 12:26:02.794583 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"4406519b-7670-42e0-9fd2-b81b12401e90","Type":"ContainerStarted","Data":"fe6fcc4b5c35e5e89c8e77b13bee17e8b3be48617d27501fff73da3736a73b2b"} Dec 03 12:26:02 crc kubenswrapper[5052]: I1203 12:26:02.794945 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"4406519b-7670-42e0-9fd2-b81b12401e90","Type":"ContainerStarted","Data":"ec32da3cb6b86d6c55cfcef58df37fb8c2efb88d4c84fb0f866050839df7c555"} Dec 03 12:26:02 crc kubenswrapper[5052]: I1203 12:26:02.794963 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"4406519b-7670-42e0-9fd2-b81b12401e90","Type":"ContainerStarted","Data":"fc91c2fb3a4da3931904532e155fb104636a8874eef93e70df5a6aa688151cba"} Dec 03 12:26:02 crc kubenswrapper[5052]: I1203 12:26:02.798693 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"0a52121e-4974-4d01-977b-71b17710e9bd","Type":"ContainerStarted","Data":"715304a54c72ee68073a8cfbe3a1ff4db360e104855d120a09bfe027ecf57d61"} Dec 03 12:26:02 crc kubenswrapper[5052]: I1203 12:26:02.798729 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"0a52121e-4974-4d01-977b-71b17710e9bd","Type":"ContainerStarted","Data":"e4a259915a2a2baf4e7137b0f920ba69c1764cfb2c5ddb3399fac3f7de720cd2"} Dec 03 12:26:02 crc kubenswrapper[5052]: I1203 12:26:02.835923 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-api-0" podStartSLOduration=2.835899472 podStartE2EDuration="2.835899472s" podCreationTimestamp="2025-12-03 12:26:00 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:26:02.817380944 +0000 UTC m=+3953.405622708" watchObservedRunningTime="2025-12-03 12:26:02.835899472 +0000 UTC m=+3953.424141236" Dec 03 12:26:02 crc kubenswrapper[5052]: I1203 12:26:02.845905 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-metadata-0" podStartSLOduration=2.845885447 podStartE2EDuration="2.845885447s" podCreationTimestamp="2025-12-03 12:26:00 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:26:02.844635302 +0000 UTC m=+3953.432877086" watchObservedRunningTime="2025-12-03 12:26:02.845885447 +0000 UTC m=+3953.434127211" Dec 03 12:26:04 crc kubenswrapper[5052]: I1203 12:26:04.095873 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:26:04 crc kubenswrapper[5052]: I1203 12:26:04.820311 5052 generic.go:334] "Generic (PLEG): container finished" podID="f18600d3-8cdb-48e0-9b40-4773286fdd82" containerID="0eb00ee36fea8f5a8d36eaf31a8098b39f40b6cf42acfc1c7a923e4f7ae3d122" exitCode=0 Dec 03 12:26:04 crc kubenswrapper[5052]: I1203 12:26:04.820395 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-f6zrf" event={"ID":"f18600d3-8cdb-48e0-9b40-4773286fdd82","Type":"ContainerDied","Data":"0eb00ee36fea8f5a8d36eaf31a8098b39f40b6cf42acfc1c7a923e4f7ae3d122"} Dec 03 12:26:06 crc kubenswrapper[5052]: I1203 12:26:06.216336 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:26:06 crc kubenswrapper[5052]: I1203 12:26:06.216853 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:26:06 crc kubenswrapper[5052]: I1203 12:26:06.226667 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-f6zrf" Dec 03 12:26:06 crc kubenswrapper[5052]: I1203 12:26:06.275012 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/f18600d3-8cdb-48e0-9b40-4773286fdd82-scripts\") pod \"f18600d3-8cdb-48e0-9b40-4773286fdd82\" (UID: \"f18600d3-8cdb-48e0-9b40-4773286fdd82\") " Dec 03 12:26:06 crc kubenswrapper[5052]: I1203 12:26:06.275090 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lrq6h\" (UniqueName: \"kubernetes.io/projected/f18600d3-8cdb-48e0-9b40-4773286fdd82-kube-api-access-lrq6h\") pod \"f18600d3-8cdb-48e0-9b40-4773286fdd82\" (UID: \"f18600d3-8cdb-48e0-9b40-4773286fdd82\") " Dec 03 12:26:06 crc kubenswrapper[5052]: I1203 12:26:06.275124 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f18600d3-8cdb-48e0-9b40-4773286fdd82-combined-ca-bundle\") pod \"f18600d3-8cdb-48e0-9b40-4773286fdd82\" (UID: \"f18600d3-8cdb-48e0-9b40-4773286fdd82\") " Dec 03 12:26:06 crc kubenswrapper[5052]: I1203 12:26:06.275159 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/f18600d3-8cdb-48e0-9b40-4773286fdd82-config-data\") pod \"f18600d3-8cdb-48e0-9b40-4773286fdd82\" (UID: \"f18600d3-8cdb-48e0-9b40-4773286fdd82\") " Dec 03 12:26:06 crc kubenswrapper[5052]: I1203 12:26:06.286029 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f18600d3-8cdb-48e0-9b40-4773286fdd82-scripts" (OuterVolumeSpecName: "scripts") pod "f18600d3-8cdb-48e0-9b40-4773286fdd82" (UID: "f18600d3-8cdb-48e0-9b40-4773286fdd82"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:26:06 crc kubenswrapper[5052]: I1203 12:26:06.288203 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f18600d3-8cdb-48e0-9b40-4773286fdd82-kube-api-access-lrq6h" (OuterVolumeSpecName: "kube-api-access-lrq6h") pod "f18600d3-8cdb-48e0-9b40-4773286fdd82" (UID: "f18600d3-8cdb-48e0-9b40-4773286fdd82"). InnerVolumeSpecName "kube-api-access-lrq6h". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:26:06 crc kubenswrapper[5052]: I1203 12:26:06.303539 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f18600d3-8cdb-48e0-9b40-4773286fdd82-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "f18600d3-8cdb-48e0-9b40-4773286fdd82" (UID: "f18600d3-8cdb-48e0-9b40-4773286fdd82"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:26:06 crc kubenswrapper[5052]: I1203 12:26:06.304971 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f18600d3-8cdb-48e0-9b40-4773286fdd82-config-data" (OuterVolumeSpecName: "config-data") pod "f18600d3-8cdb-48e0-9b40-4773286fdd82" (UID: "f18600d3-8cdb-48e0-9b40-4773286fdd82"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:26:06 crc kubenswrapper[5052]: I1203 12:26:06.377759 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/f18600d3-8cdb-48e0-9b40-4773286fdd82-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:26:06 crc kubenswrapper[5052]: I1203 12:26:06.377802 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-lrq6h\" (UniqueName: \"kubernetes.io/projected/f18600d3-8cdb-48e0-9b40-4773286fdd82-kube-api-access-lrq6h\") on node \"crc\" DevicePath \"\"" Dec 03 12:26:06 crc kubenswrapper[5052]: I1203 12:26:06.377813 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f18600d3-8cdb-48e0-9b40-4773286fdd82-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:26:06 crc kubenswrapper[5052]: I1203 12:26:06.377822 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/f18600d3-8cdb-48e0-9b40-4773286fdd82-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:26:06 crc kubenswrapper[5052]: I1203 12:26:06.841789 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-f6zrf" event={"ID":"f18600d3-8cdb-48e0-9b40-4773286fdd82","Type":"ContainerDied","Data":"f4c2ea14bf8994a7a027d2992f8f33b55849cd56ba806eabf1631d2e0072e479"} Dec 03 12:26:06 crc kubenswrapper[5052]: I1203 12:26:06.841833 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f4c2ea14bf8994a7a027d2992f8f33b55849cd56ba806eabf1631d2e0072e479" Dec 03 12:26:06 crc kubenswrapper[5052]: I1203 12:26:06.841861 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-f6zrf" Dec 03 12:26:06 crc kubenswrapper[5052]: I1203 12:26:06.949308 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-0"] Dec 03 12:26:06 crc kubenswrapper[5052]: E1203 12:26:06.949784 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="f18600d3-8cdb-48e0-9b40-4773286fdd82" containerName="nova-cell1-conductor-db-sync" Dec 03 12:26:06 crc kubenswrapper[5052]: I1203 12:26:06.949809 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="f18600d3-8cdb-48e0-9b40-4773286fdd82" containerName="nova-cell1-conductor-db-sync" Dec 03 12:26:06 crc kubenswrapper[5052]: I1203 12:26:06.950058 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="f18600d3-8cdb-48e0-9b40-4773286fdd82" containerName="nova-cell1-conductor-db-sync" Dec 03 12:26:06 crc kubenswrapper[5052]: I1203 12:26:06.950834 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:26:06 crc kubenswrapper[5052]: I1203 12:26:06.956989 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell1-conductor-config-data" Dec 03 12:26:06 crc kubenswrapper[5052]: I1203 12:26:06.965236 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-0"] Dec 03 12:26:06 crc kubenswrapper[5052]: I1203 12:26:06.989544 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tdj8s\" (UniqueName: \"kubernetes.io/projected/51912caa-ca45-4e7c-a0ee-a40ae65a1896-kube-api-access-tdj8s\") pod \"nova-cell1-conductor-0\" (UID: \"51912caa-ca45-4e7c-a0ee-a40ae65a1896\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:26:06 crc kubenswrapper[5052]: I1203 12:26:06.989625 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/51912caa-ca45-4e7c-a0ee-a40ae65a1896-combined-ca-bundle\") pod \"nova-cell1-conductor-0\" (UID: \"51912caa-ca45-4e7c-a0ee-a40ae65a1896\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:26:06 crc kubenswrapper[5052]: I1203 12:26:06.989670 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/51912caa-ca45-4e7c-a0ee-a40ae65a1896-config-data\") pod \"nova-cell1-conductor-0\" (UID: \"51912caa-ca45-4e7c-a0ee-a40ae65a1896\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:26:07 crc kubenswrapper[5052]: I1203 12:26:07.091789 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-tdj8s\" (UniqueName: \"kubernetes.io/projected/51912caa-ca45-4e7c-a0ee-a40ae65a1896-kube-api-access-tdj8s\") pod \"nova-cell1-conductor-0\" (UID: \"51912caa-ca45-4e7c-a0ee-a40ae65a1896\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:26:07 crc kubenswrapper[5052]: I1203 12:26:07.091874 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/51912caa-ca45-4e7c-a0ee-a40ae65a1896-combined-ca-bundle\") pod \"nova-cell1-conductor-0\" (UID: \"51912caa-ca45-4e7c-a0ee-a40ae65a1896\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:26:07 crc kubenswrapper[5052]: I1203 12:26:07.091920 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/51912caa-ca45-4e7c-a0ee-a40ae65a1896-config-data\") pod \"nova-cell1-conductor-0\" (UID: \"51912caa-ca45-4e7c-a0ee-a40ae65a1896\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:26:07 crc kubenswrapper[5052]: I1203 12:26:07.097159 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/51912caa-ca45-4e7c-a0ee-a40ae65a1896-config-data\") pod \"nova-cell1-conductor-0\" (UID: \"51912caa-ca45-4e7c-a0ee-a40ae65a1896\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:26:07 crc kubenswrapper[5052]: I1203 12:26:07.099336 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/51912caa-ca45-4e7c-a0ee-a40ae65a1896-combined-ca-bundle\") pod \"nova-cell1-conductor-0\" (UID: \"51912caa-ca45-4e7c-a0ee-a40ae65a1896\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:26:07 crc kubenswrapper[5052]: I1203 12:26:07.110387 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-tdj8s\" (UniqueName: \"kubernetes.io/projected/51912caa-ca45-4e7c-a0ee-a40ae65a1896-kube-api-access-tdj8s\") pod \"nova-cell1-conductor-0\" (UID: \"51912caa-ca45-4e7c-a0ee-a40ae65a1896\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:26:07 crc kubenswrapper[5052]: I1203 12:26:07.271530 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:26:07 crc kubenswrapper[5052]: I1203 12:26:07.876597 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-0"] Dec 03 12:26:08 crc kubenswrapper[5052]: I1203 12:26:08.325202 5052 scope.go:117] "RemoveContainer" containerID="f47c386e9ed42113922ee7779b6fcd345651a75d010d2ed9f2e0908c03e6b7e3" Dec 03 12:26:08 crc kubenswrapper[5052]: I1203 12:26:08.353755 5052 scope.go:117] "RemoveContainer" containerID="d99cc4f4b6ecb6e1a9c7ee843b98d6486c6ecec28b0e9d8a368da66230b5fbcc" Dec 03 12:26:08 crc kubenswrapper[5052]: I1203 12:26:08.400952 5052 scope.go:117] "RemoveContainer" containerID="d91c19496f184f5f6be8562c5b5ecd26605542c98135cd177562baf0dce4ffb1" Dec 03 12:26:08 crc kubenswrapper[5052]: I1203 12:26:08.421935 5052 scope.go:117] "RemoveContainer" containerID="9ef2ea9519f50d4f2765e00b600cfe8f29f447a34e20aa72760861bd5ab6b0b0" Dec 03 12:26:08 crc kubenswrapper[5052]: I1203 12:26:08.482174 5052 scope.go:117] "RemoveContainer" containerID="d2dbcc7e2fdee4a46c5830b20391717220b63cfcd058c8f7699961d14ed09d4e" Dec 03 12:26:08 crc kubenswrapper[5052]: I1203 12:26:08.868347 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-conductor-0" event={"ID":"51912caa-ca45-4e7c-a0ee-a40ae65a1896","Type":"ContainerStarted","Data":"7c7ee713ff0737c8c0810e900709d6bd56bb4cc9128cbcf656be4631e51882ae"} Dec 03 12:26:08 crc kubenswrapper[5052]: I1203 12:26:08.868434 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-conductor-0" event={"ID":"51912caa-ca45-4e7c-a0ee-a40ae65a1896","Type":"ContainerStarted","Data":"a467fc950fff63daca6d13f96e45304e11902dc49e2e775a81131f721029ded1"} Dec 03 12:26:08 crc kubenswrapper[5052]: I1203 12:26:08.868773 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:26:08 crc kubenswrapper[5052]: I1203 12:26:08.897053 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-cell1-conductor-0" podStartSLOduration=2.8970226390000002 podStartE2EDuration="2.897022639s" podCreationTimestamp="2025-12-03 12:26:06 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:26:08.894277211 +0000 UTC m=+3959.482518975" watchObservedRunningTime="2025-12-03 12:26:08.897022639 +0000 UTC m=+3959.485264403" Dec 03 12:26:09 crc kubenswrapper[5052]: I1203 12:26:09.096398 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:26:09 crc kubenswrapper[5052]: I1203 12:26:09.129437 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:26:09 crc kubenswrapper[5052]: I1203 12:26:09.926200 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:26:11 crc kubenswrapper[5052]: I1203 12:26:11.217367 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:26:11 crc kubenswrapper[5052]: I1203 12:26:11.217419 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:26:11 crc kubenswrapper[5052]: I1203 12:26:11.232736 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:26:11 crc kubenswrapper[5052]: I1203 12:26:11.232788 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:26:12 crc kubenswrapper[5052]: I1203 12:26:12.342773 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-api-0" podUID="4406519b-7670-42e0-9fd2-b81b12401e90" containerName="nova-api-log" probeResult="failure" output="Get \"http://10.217.0.114:8774/\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Dec 03 12:26:12 crc kubenswrapper[5052]: I1203 12:26:12.343498 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-metadata-0" podUID="0a52121e-4974-4d01-977b-71b17710e9bd" containerName="nova-metadata-metadata" probeResult="failure" output="Get \"http://10.217.0.113:8775/\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Dec 03 12:26:12 crc kubenswrapper[5052]: I1203 12:26:12.343566 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-metadata-0" podUID="0a52121e-4974-4d01-977b-71b17710e9bd" containerName="nova-metadata-log" probeResult="failure" output="Get \"http://10.217.0.113:8775/\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Dec 03 12:26:12 crc kubenswrapper[5052]: I1203 12:26:12.343606 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-api-0" podUID="4406519b-7670-42e0-9fd2-b81b12401e90" containerName="nova-api-api" probeResult="failure" output="Get \"http://10.217.0.114:8774/\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Dec 03 12:26:17 crc kubenswrapper[5052]: I1203 12:26:17.301211 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:26:17 crc kubenswrapper[5052]: I1203 12:26:17.780743 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell1-cell-mapping-hhtnd"] Dec 03 12:26:17 crc kubenswrapper[5052]: I1203 12:26:17.782023 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-cell-mapping-hhtnd" Dec 03 12:26:17 crc kubenswrapper[5052]: I1203 12:26:17.786100 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell1-manage-config-data" Dec 03 12:26:17 crc kubenswrapper[5052]: I1203 12:26:17.786258 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell1-manage-scripts" Dec 03 12:26:17 crc kubenswrapper[5052]: I1203 12:26:17.806156 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-cell-mapping-hhtnd"] Dec 03 12:26:17 crc kubenswrapper[5052]: I1203 12:26:17.899015 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/eb8c41b3-9282-4a42-8122-d648450a757d-combined-ca-bundle\") pod \"nova-cell1-cell-mapping-hhtnd\" (UID: \"eb8c41b3-9282-4a42-8122-d648450a757d\") " pod="openstack-kuttl-tests/nova-cell1-cell-mapping-hhtnd" Dec 03 12:26:17 crc kubenswrapper[5052]: I1203 12:26:17.899174 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hmvts\" (UniqueName: \"kubernetes.io/projected/eb8c41b3-9282-4a42-8122-d648450a757d-kube-api-access-hmvts\") pod \"nova-cell1-cell-mapping-hhtnd\" (UID: \"eb8c41b3-9282-4a42-8122-d648450a757d\") " pod="openstack-kuttl-tests/nova-cell1-cell-mapping-hhtnd" Dec 03 12:26:17 crc kubenswrapper[5052]: I1203 12:26:17.899205 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/eb8c41b3-9282-4a42-8122-d648450a757d-config-data\") pod \"nova-cell1-cell-mapping-hhtnd\" (UID: \"eb8c41b3-9282-4a42-8122-d648450a757d\") " pod="openstack-kuttl-tests/nova-cell1-cell-mapping-hhtnd" Dec 03 12:26:17 crc kubenswrapper[5052]: I1203 12:26:17.899334 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/eb8c41b3-9282-4a42-8122-d648450a757d-scripts\") pod \"nova-cell1-cell-mapping-hhtnd\" (UID: \"eb8c41b3-9282-4a42-8122-d648450a757d\") " pod="openstack-kuttl-tests/nova-cell1-cell-mapping-hhtnd" Dec 03 12:26:18 crc kubenswrapper[5052]: I1203 12:26:18.002181 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/eb8c41b3-9282-4a42-8122-d648450a757d-scripts\") pod \"nova-cell1-cell-mapping-hhtnd\" (UID: \"eb8c41b3-9282-4a42-8122-d648450a757d\") " pod="openstack-kuttl-tests/nova-cell1-cell-mapping-hhtnd" Dec 03 12:26:18 crc kubenswrapper[5052]: I1203 12:26:18.002256 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/eb8c41b3-9282-4a42-8122-d648450a757d-combined-ca-bundle\") pod \"nova-cell1-cell-mapping-hhtnd\" (UID: \"eb8c41b3-9282-4a42-8122-d648450a757d\") " pod="openstack-kuttl-tests/nova-cell1-cell-mapping-hhtnd" Dec 03 12:26:18 crc kubenswrapper[5052]: I1203 12:26:18.002328 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-hmvts\" (UniqueName: \"kubernetes.io/projected/eb8c41b3-9282-4a42-8122-d648450a757d-kube-api-access-hmvts\") pod \"nova-cell1-cell-mapping-hhtnd\" (UID: \"eb8c41b3-9282-4a42-8122-d648450a757d\") " pod="openstack-kuttl-tests/nova-cell1-cell-mapping-hhtnd" Dec 03 12:26:18 crc kubenswrapper[5052]: I1203 12:26:18.002348 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/eb8c41b3-9282-4a42-8122-d648450a757d-config-data\") pod \"nova-cell1-cell-mapping-hhtnd\" (UID: \"eb8c41b3-9282-4a42-8122-d648450a757d\") " pod="openstack-kuttl-tests/nova-cell1-cell-mapping-hhtnd" Dec 03 12:26:18 crc kubenswrapper[5052]: I1203 12:26:18.010479 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/eb8c41b3-9282-4a42-8122-d648450a757d-combined-ca-bundle\") pod \"nova-cell1-cell-mapping-hhtnd\" (UID: \"eb8c41b3-9282-4a42-8122-d648450a757d\") " pod="openstack-kuttl-tests/nova-cell1-cell-mapping-hhtnd" Dec 03 12:26:18 crc kubenswrapper[5052]: I1203 12:26:18.011114 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/eb8c41b3-9282-4a42-8122-d648450a757d-config-data\") pod \"nova-cell1-cell-mapping-hhtnd\" (UID: \"eb8c41b3-9282-4a42-8122-d648450a757d\") " pod="openstack-kuttl-tests/nova-cell1-cell-mapping-hhtnd" Dec 03 12:26:18 crc kubenswrapper[5052]: I1203 12:26:18.011386 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/eb8c41b3-9282-4a42-8122-d648450a757d-scripts\") pod \"nova-cell1-cell-mapping-hhtnd\" (UID: \"eb8c41b3-9282-4a42-8122-d648450a757d\") " pod="openstack-kuttl-tests/nova-cell1-cell-mapping-hhtnd" Dec 03 12:26:18 crc kubenswrapper[5052]: I1203 12:26:18.034393 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-hmvts\" (UniqueName: \"kubernetes.io/projected/eb8c41b3-9282-4a42-8122-d648450a757d-kube-api-access-hmvts\") pod \"nova-cell1-cell-mapping-hhtnd\" (UID: \"eb8c41b3-9282-4a42-8122-d648450a757d\") " pod="openstack-kuttl-tests/nova-cell1-cell-mapping-hhtnd" Dec 03 12:26:18 crc kubenswrapper[5052]: I1203 12:26:18.114872 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-cell-mapping-hhtnd" Dec 03 12:26:18 crc kubenswrapper[5052]: I1203 12:26:18.569172 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-cell-mapping-hhtnd"] Dec 03 12:26:18 crc kubenswrapper[5052]: W1203 12:26:18.577291 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podeb8c41b3_9282_4a42_8122_d648450a757d.slice/crio-1d9079c964e4dcdfef1627bf8ff9cab96c380172dcd2d1a1061ae072c6aa5103 WatchSource:0}: Error finding container 1d9079c964e4dcdfef1627bf8ff9cab96c380172dcd2d1a1061ae072c6aa5103: Status 404 returned error can't find the container with id 1d9079c964e4dcdfef1627bf8ff9cab96c380172dcd2d1a1061ae072c6aa5103 Dec 03 12:26:18 crc kubenswrapper[5052]: I1203 12:26:18.980366 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-cell-mapping-hhtnd" event={"ID":"eb8c41b3-9282-4a42-8122-d648450a757d","Type":"ContainerStarted","Data":"a46cb3d56ad689ac1d90543412ee1d3884621a7da5a8f143e20e1ff107ef6902"} Dec 03 12:26:18 crc kubenswrapper[5052]: I1203 12:26:18.980682 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-cell-mapping-hhtnd" event={"ID":"eb8c41b3-9282-4a42-8122-d648450a757d","Type":"ContainerStarted","Data":"1d9079c964e4dcdfef1627bf8ff9cab96c380172dcd2d1a1061ae072c6aa5103"} Dec 03 12:26:20 crc kubenswrapper[5052]: I1203 12:26:20.010224 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-cell1-cell-mapping-hhtnd" podStartSLOduration=3.010206642 podStartE2EDuration="3.010206642s" podCreationTimestamp="2025-12-03 12:26:17 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:26:20.006038263 +0000 UTC m=+3970.594280057" watchObservedRunningTime="2025-12-03 12:26:20.010206642 +0000 UTC m=+3970.598448406" Dec 03 12:26:21 crc kubenswrapper[5052]: I1203 12:26:21.219204 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:26:21 crc kubenswrapper[5052]: I1203 12:26:21.219551 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:26:21 crc kubenswrapper[5052]: I1203 12:26:21.220955 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:26:21 crc kubenswrapper[5052]: I1203 12:26:21.221520 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:26:21 crc kubenswrapper[5052]: I1203 12:26:21.239499 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:26:21 crc kubenswrapper[5052]: I1203 12:26:21.240063 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:26:21 crc kubenswrapper[5052]: I1203 12:26:21.243678 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:26:21 crc kubenswrapper[5052]: I1203 12:26:21.257237 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:26:22 crc kubenswrapper[5052]: I1203 12:26:22.017549 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:26:22 crc kubenswrapper[5052]: I1203 12:26:22.023164 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:26:23 crc kubenswrapper[5052]: I1203 12:26:23.989777 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:26:23 crc kubenswrapper[5052]: I1203 12:26:23.990308 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="2040c0be-2fbd-4b01-9852-82d0210f1429" containerName="ceilometer-central-agent" containerID="cri-o://fda1434e3be27361a253ac86d38068672205b80750ce3eb51a12b74e073b1d2b" gracePeriod=30 Dec 03 12:26:23 crc kubenswrapper[5052]: I1203 12:26:23.990815 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="2040c0be-2fbd-4b01-9852-82d0210f1429" containerName="proxy-httpd" containerID="cri-o://10a32d43eecd0862569a8cc67a498c585b50d021f89d849caa39ca046cce5b68" gracePeriod=30 Dec 03 12:26:23 crc kubenswrapper[5052]: I1203 12:26:23.990885 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="2040c0be-2fbd-4b01-9852-82d0210f1429" containerName="sg-core" containerID="cri-o://e9c38bbe15d84deb1873bc02f167d157b471f71259a406cefaac732efb600e4f" gracePeriod=30 Dec 03 12:26:23 crc kubenswrapper[5052]: I1203 12:26:23.991029 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="2040c0be-2fbd-4b01-9852-82d0210f1429" containerName="ceilometer-notification-agent" containerID="cri-o://86638ee6f1d993932caa91451744753653fa3507238feac44e741c4134134430" gracePeriod=30 Dec 03 12:26:24 crc kubenswrapper[5052]: I1203 12:26:24.161142 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openshift-marketplace/redhat-operators-qhzn9"] Dec 03 12:26:24 crc kubenswrapper[5052]: I1203 12:26:24.165166 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/redhat-operators-qhzn9" Dec 03 12:26:24 crc kubenswrapper[5052]: I1203 12:26:24.184331 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/redhat-operators-qhzn9"] Dec 03 12:26:24 crc kubenswrapper[5052]: I1203 12:26:24.243471 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/07a34aba-b4ee-4b5e-92ea-d458af9399de-catalog-content\") pod \"redhat-operators-qhzn9\" (UID: \"07a34aba-b4ee-4b5e-92ea-d458af9399de\") " pod="openshift-marketplace/redhat-operators-qhzn9" Dec 03 12:26:24 crc kubenswrapper[5052]: I1203 12:26:24.244025 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/07a34aba-b4ee-4b5e-92ea-d458af9399de-utilities\") pod \"redhat-operators-qhzn9\" (UID: \"07a34aba-b4ee-4b5e-92ea-d458af9399de\") " pod="openshift-marketplace/redhat-operators-qhzn9" Dec 03 12:26:24 crc kubenswrapper[5052]: I1203 12:26:24.244312 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2xprp\" (UniqueName: \"kubernetes.io/projected/07a34aba-b4ee-4b5e-92ea-d458af9399de-kube-api-access-2xprp\") pod \"redhat-operators-qhzn9\" (UID: \"07a34aba-b4ee-4b5e-92ea-d458af9399de\") " pod="openshift-marketplace/redhat-operators-qhzn9" Dec 03 12:26:24 crc kubenswrapper[5052]: I1203 12:26:24.345506 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/07a34aba-b4ee-4b5e-92ea-d458af9399de-utilities\") pod \"redhat-operators-qhzn9\" (UID: \"07a34aba-b4ee-4b5e-92ea-d458af9399de\") " pod="openshift-marketplace/redhat-operators-qhzn9" Dec 03 12:26:24 crc kubenswrapper[5052]: I1203 12:26:24.345849 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-2xprp\" (UniqueName: \"kubernetes.io/projected/07a34aba-b4ee-4b5e-92ea-d458af9399de-kube-api-access-2xprp\") pod \"redhat-operators-qhzn9\" (UID: \"07a34aba-b4ee-4b5e-92ea-d458af9399de\") " pod="openshift-marketplace/redhat-operators-qhzn9" Dec 03 12:26:24 crc kubenswrapper[5052]: I1203 12:26:24.345895 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/07a34aba-b4ee-4b5e-92ea-d458af9399de-catalog-content\") pod \"redhat-operators-qhzn9\" (UID: \"07a34aba-b4ee-4b5e-92ea-d458af9399de\") " pod="openshift-marketplace/redhat-operators-qhzn9" Dec 03 12:26:24 crc kubenswrapper[5052]: I1203 12:26:24.346308 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/07a34aba-b4ee-4b5e-92ea-d458af9399de-utilities\") pod \"redhat-operators-qhzn9\" (UID: \"07a34aba-b4ee-4b5e-92ea-d458af9399de\") " pod="openshift-marketplace/redhat-operators-qhzn9" Dec 03 12:26:24 crc kubenswrapper[5052]: I1203 12:26:24.346342 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/07a34aba-b4ee-4b5e-92ea-d458af9399de-catalog-content\") pod \"redhat-operators-qhzn9\" (UID: \"07a34aba-b4ee-4b5e-92ea-d458af9399de\") " pod="openshift-marketplace/redhat-operators-qhzn9" Dec 03 12:26:24 crc kubenswrapper[5052]: I1203 12:26:24.376066 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-2xprp\" (UniqueName: \"kubernetes.io/projected/07a34aba-b4ee-4b5e-92ea-d458af9399de-kube-api-access-2xprp\") pod \"redhat-operators-qhzn9\" (UID: \"07a34aba-b4ee-4b5e-92ea-d458af9399de\") " pod="openshift-marketplace/redhat-operators-qhzn9" Dec 03 12:26:24 crc kubenswrapper[5052]: I1203 12:26:24.547477 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/redhat-operators-qhzn9" Dec 03 12:26:25 crc kubenswrapper[5052]: I1203 12:26:25.052291 5052 generic.go:334] "Generic (PLEG): container finished" podID="2040c0be-2fbd-4b01-9852-82d0210f1429" containerID="10a32d43eecd0862569a8cc67a498c585b50d021f89d849caa39ca046cce5b68" exitCode=0 Dec 03 12:26:25 crc kubenswrapper[5052]: I1203 12:26:25.052324 5052 generic.go:334] "Generic (PLEG): container finished" podID="2040c0be-2fbd-4b01-9852-82d0210f1429" containerID="e9c38bbe15d84deb1873bc02f167d157b471f71259a406cefaac732efb600e4f" exitCode=2 Dec 03 12:26:25 crc kubenswrapper[5052]: I1203 12:26:25.052331 5052 generic.go:334] "Generic (PLEG): container finished" podID="2040c0be-2fbd-4b01-9852-82d0210f1429" containerID="fda1434e3be27361a253ac86d38068672205b80750ce3eb51a12b74e073b1d2b" exitCode=0 Dec 03 12:26:25 crc kubenswrapper[5052]: I1203 12:26:25.052354 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"2040c0be-2fbd-4b01-9852-82d0210f1429","Type":"ContainerDied","Data":"10a32d43eecd0862569a8cc67a498c585b50d021f89d849caa39ca046cce5b68"} Dec 03 12:26:25 crc kubenswrapper[5052]: I1203 12:26:25.052380 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"2040c0be-2fbd-4b01-9852-82d0210f1429","Type":"ContainerDied","Data":"e9c38bbe15d84deb1873bc02f167d157b471f71259a406cefaac732efb600e4f"} Dec 03 12:26:25 crc kubenswrapper[5052]: I1203 12:26:25.052391 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"2040c0be-2fbd-4b01-9852-82d0210f1429","Type":"ContainerDied","Data":"fda1434e3be27361a253ac86d38068672205b80750ce3eb51a12b74e073b1d2b"} Dec 03 12:26:25 crc kubenswrapper[5052]: I1203 12:26:25.182690 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/redhat-operators-qhzn9"] Dec 03 12:26:26 crc kubenswrapper[5052]: I1203 12:26:26.098691 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-operators-qhzn9" event={"ID":"07a34aba-b4ee-4b5e-92ea-d458af9399de","Type":"ContainerStarted","Data":"28d86f93f18114010b14ff497128eb4b009dde86bf925db45369fc17d7f0ce3b"} Dec 03 12:26:27 crc kubenswrapper[5052]: I1203 12:26:27.108649 5052 generic.go:334] "Generic (PLEG): container finished" podID="eb8c41b3-9282-4a42-8122-d648450a757d" containerID="a46cb3d56ad689ac1d90543412ee1d3884621a7da5a8f143e20e1ff107ef6902" exitCode=0 Dec 03 12:26:27 crc kubenswrapper[5052]: I1203 12:26:27.108768 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-cell-mapping-hhtnd" event={"ID":"eb8c41b3-9282-4a42-8122-d648450a757d","Type":"ContainerDied","Data":"a46cb3d56ad689ac1d90543412ee1d3884621a7da5a8f143e20e1ff107ef6902"} Dec 03 12:26:27 crc kubenswrapper[5052]: I1203 12:26:27.110868 5052 generic.go:334] "Generic (PLEG): container finished" podID="07a34aba-b4ee-4b5e-92ea-d458af9399de" containerID="ede4c3adf898870cfa7b399a0ddcefb756ac3e870f4f18b8f0483222573548e7" exitCode=0 Dec 03 12:26:27 crc kubenswrapper[5052]: I1203 12:26:27.110903 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-operators-qhzn9" event={"ID":"07a34aba-b4ee-4b5e-92ea-d458af9399de","Type":"ContainerDied","Data":"ede4c3adf898870cfa7b399a0ddcefb756ac3e870f4f18b8f0483222573548e7"} Dec 03 12:26:27 crc kubenswrapper[5052]: I1203 12:26:27.848608 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:26:27 crc kubenswrapper[5052]: I1203 12:26:27.978025 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/2040c0be-2fbd-4b01-9852-82d0210f1429-sg-core-conf-yaml\") pod \"2040c0be-2fbd-4b01-9852-82d0210f1429\" (UID: \"2040c0be-2fbd-4b01-9852-82d0210f1429\") " Dec 03 12:26:27 crc kubenswrapper[5052]: I1203 12:26:27.978263 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/2040c0be-2fbd-4b01-9852-82d0210f1429-config-data\") pod \"2040c0be-2fbd-4b01-9852-82d0210f1429\" (UID: \"2040c0be-2fbd-4b01-9852-82d0210f1429\") " Dec 03 12:26:27 crc kubenswrapper[5052]: I1203 12:26:27.978316 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/2040c0be-2fbd-4b01-9852-82d0210f1429-log-httpd\") pod \"2040c0be-2fbd-4b01-9852-82d0210f1429\" (UID: \"2040c0be-2fbd-4b01-9852-82d0210f1429\") " Dec 03 12:26:27 crc kubenswrapper[5052]: I1203 12:26:27.978361 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/2040c0be-2fbd-4b01-9852-82d0210f1429-scripts\") pod \"2040c0be-2fbd-4b01-9852-82d0210f1429\" (UID: \"2040c0be-2fbd-4b01-9852-82d0210f1429\") " Dec 03 12:26:27 crc kubenswrapper[5052]: I1203 12:26:27.978437 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/2040c0be-2fbd-4b01-9852-82d0210f1429-run-httpd\") pod \"2040c0be-2fbd-4b01-9852-82d0210f1429\" (UID: \"2040c0be-2fbd-4b01-9852-82d0210f1429\") " Dec 03 12:26:27 crc kubenswrapper[5052]: I1203 12:26:27.978526 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q9m2s\" (UniqueName: \"kubernetes.io/projected/2040c0be-2fbd-4b01-9852-82d0210f1429-kube-api-access-q9m2s\") pod \"2040c0be-2fbd-4b01-9852-82d0210f1429\" (UID: \"2040c0be-2fbd-4b01-9852-82d0210f1429\") " Dec 03 12:26:27 crc kubenswrapper[5052]: I1203 12:26:27.978706 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2040c0be-2fbd-4b01-9852-82d0210f1429-combined-ca-bundle\") pod \"2040c0be-2fbd-4b01-9852-82d0210f1429\" (UID: \"2040c0be-2fbd-4b01-9852-82d0210f1429\") " Dec 03 12:26:28 crc kubenswrapper[5052]: I1203 12:26:27.981125 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2040c0be-2fbd-4b01-9852-82d0210f1429-log-httpd" (OuterVolumeSpecName: "log-httpd") pod "2040c0be-2fbd-4b01-9852-82d0210f1429" (UID: "2040c0be-2fbd-4b01-9852-82d0210f1429"). InnerVolumeSpecName "log-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:26:28 crc kubenswrapper[5052]: I1203 12:26:28.039591 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2040c0be-2fbd-4b01-9852-82d0210f1429-run-httpd" (OuterVolumeSpecName: "run-httpd") pod "2040c0be-2fbd-4b01-9852-82d0210f1429" (UID: "2040c0be-2fbd-4b01-9852-82d0210f1429"). InnerVolumeSpecName "run-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:26:28 crc kubenswrapper[5052]: I1203 12:26:28.042710 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2040c0be-2fbd-4b01-9852-82d0210f1429-kube-api-access-q9m2s" (OuterVolumeSpecName: "kube-api-access-q9m2s") pod "2040c0be-2fbd-4b01-9852-82d0210f1429" (UID: "2040c0be-2fbd-4b01-9852-82d0210f1429"). InnerVolumeSpecName "kube-api-access-q9m2s". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:26:28 crc kubenswrapper[5052]: I1203 12:26:28.060698 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2040c0be-2fbd-4b01-9852-82d0210f1429-scripts" (OuterVolumeSpecName: "scripts") pod "2040c0be-2fbd-4b01-9852-82d0210f1429" (UID: "2040c0be-2fbd-4b01-9852-82d0210f1429"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:26:28 crc kubenswrapper[5052]: I1203 12:26:28.080931 5052 reconciler_common.go:293] "Volume detached for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/2040c0be-2fbd-4b01-9852-82d0210f1429-log-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:26:28 crc kubenswrapper[5052]: I1203 12:26:28.081181 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/2040c0be-2fbd-4b01-9852-82d0210f1429-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:26:28 crc kubenswrapper[5052]: I1203 12:26:28.081245 5052 reconciler_common.go:293] "Volume detached for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/2040c0be-2fbd-4b01-9852-82d0210f1429-run-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:26:28 crc kubenswrapper[5052]: I1203 12:26:28.081316 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-q9m2s\" (UniqueName: \"kubernetes.io/projected/2040c0be-2fbd-4b01-9852-82d0210f1429-kube-api-access-q9m2s\") on node \"crc\" DevicePath \"\"" Dec 03 12:26:28 crc kubenswrapper[5052]: I1203 12:26:28.101098 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2040c0be-2fbd-4b01-9852-82d0210f1429-sg-core-conf-yaml" (OuterVolumeSpecName: "sg-core-conf-yaml") pod "2040c0be-2fbd-4b01-9852-82d0210f1429" (UID: "2040c0be-2fbd-4b01-9852-82d0210f1429"). InnerVolumeSpecName "sg-core-conf-yaml". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:26:28 crc kubenswrapper[5052]: I1203 12:26:28.130746 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-operators-qhzn9" event={"ID":"07a34aba-b4ee-4b5e-92ea-d458af9399de","Type":"ContainerStarted","Data":"104040fbb1ee554dc74340bedf98f59d521b44f2da342566a71b536c53c4bc01"} Dec 03 12:26:28 crc kubenswrapper[5052]: I1203 12:26:28.138722 5052 generic.go:334] "Generic (PLEG): container finished" podID="2040c0be-2fbd-4b01-9852-82d0210f1429" containerID="86638ee6f1d993932caa91451744753653fa3507238feac44e741c4134134430" exitCode=0 Dec 03 12:26:28 crc kubenswrapper[5052]: I1203 12:26:28.139175 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:26:28 crc kubenswrapper[5052]: I1203 12:26:28.140146 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"2040c0be-2fbd-4b01-9852-82d0210f1429","Type":"ContainerDied","Data":"86638ee6f1d993932caa91451744753653fa3507238feac44e741c4134134430"} Dec 03 12:26:28 crc kubenswrapper[5052]: I1203 12:26:28.140303 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"2040c0be-2fbd-4b01-9852-82d0210f1429","Type":"ContainerDied","Data":"2696fdd099db7e5dda8a7c778a456bbb5d9650d8684765ce13acc4712aaa456d"} Dec 03 12:26:28 crc kubenswrapper[5052]: I1203 12:26:28.140462 5052 scope.go:117] "RemoveContainer" containerID="10a32d43eecd0862569a8cc67a498c585b50d021f89d849caa39ca046cce5b68" Dec 03 12:26:28 crc kubenswrapper[5052]: I1203 12:26:28.156376 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2040c0be-2fbd-4b01-9852-82d0210f1429-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "2040c0be-2fbd-4b01-9852-82d0210f1429" (UID: "2040c0be-2fbd-4b01-9852-82d0210f1429"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:26:28 crc kubenswrapper[5052]: I1203 12:26:28.164629 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2040c0be-2fbd-4b01-9852-82d0210f1429-config-data" (OuterVolumeSpecName: "config-data") pod "2040c0be-2fbd-4b01-9852-82d0210f1429" (UID: "2040c0be-2fbd-4b01-9852-82d0210f1429"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:26:28 crc kubenswrapper[5052]: I1203 12:26:28.217362 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2040c0be-2fbd-4b01-9852-82d0210f1429-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:26:28 crc kubenswrapper[5052]: I1203 12:26:28.217644 5052 reconciler_common.go:293] "Volume detached for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/2040c0be-2fbd-4b01-9852-82d0210f1429-sg-core-conf-yaml\") on node \"crc\" DevicePath \"\"" Dec 03 12:26:28 crc kubenswrapper[5052]: I1203 12:26:28.217656 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/2040c0be-2fbd-4b01-9852-82d0210f1429-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:26:28 crc kubenswrapper[5052]: I1203 12:26:28.244473 5052 scope.go:117] "RemoveContainer" containerID="e9c38bbe15d84deb1873bc02f167d157b471f71259a406cefaac732efb600e4f" Dec 03 12:26:28 crc kubenswrapper[5052]: I1203 12:26:28.276022 5052 scope.go:117] "RemoveContainer" containerID="86638ee6f1d993932caa91451744753653fa3507238feac44e741c4134134430" Dec 03 12:26:28 crc kubenswrapper[5052]: I1203 12:26:28.302744 5052 scope.go:117] "RemoveContainer" containerID="fda1434e3be27361a253ac86d38068672205b80750ce3eb51a12b74e073b1d2b" Dec 03 12:26:28 crc kubenswrapper[5052]: I1203 12:26:28.329005 5052 scope.go:117] "RemoveContainer" containerID="10a32d43eecd0862569a8cc67a498c585b50d021f89d849caa39ca046cce5b68" Dec 03 12:26:28 crc kubenswrapper[5052]: E1203 12:26:28.329760 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"10a32d43eecd0862569a8cc67a498c585b50d021f89d849caa39ca046cce5b68\": container with ID starting with 10a32d43eecd0862569a8cc67a498c585b50d021f89d849caa39ca046cce5b68 not found: ID does not exist" containerID="10a32d43eecd0862569a8cc67a498c585b50d021f89d849caa39ca046cce5b68" Dec 03 12:26:28 crc kubenswrapper[5052]: I1203 12:26:28.329844 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"10a32d43eecd0862569a8cc67a498c585b50d021f89d849caa39ca046cce5b68"} err="failed to get container status \"10a32d43eecd0862569a8cc67a498c585b50d021f89d849caa39ca046cce5b68\": rpc error: code = NotFound desc = could not find container \"10a32d43eecd0862569a8cc67a498c585b50d021f89d849caa39ca046cce5b68\": container with ID starting with 10a32d43eecd0862569a8cc67a498c585b50d021f89d849caa39ca046cce5b68 not found: ID does not exist" Dec 03 12:26:28 crc kubenswrapper[5052]: I1203 12:26:28.329894 5052 scope.go:117] "RemoveContainer" containerID="e9c38bbe15d84deb1873bc02f167d157b471f71259a406cefaac732efb600e4f" Dec 03 12:26:28 crc kubenswrapper[5052]: E1203 12:26:28.330466 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e9c38bbe15d84deb1873bc02f167d157b471f71259a406cefaac732efb600e4f\": container with ID starting with e9c38bbe15d84deb1873bc02f167d157b471f71259a406cefaac732efb600e4f not found: ID does not exist" containerID="e9c38bbe15d84deb1873bc02f167d157b471f71259a406cefaac732efb600e4f" Dec 03 12:26:28 crc kubenswrapper[5052]: I1203 12:26:28.330554 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e9c38bbe15d84deb1873bc02f167d157b471f71259a406cefaac732efb600e4f"} err="failed to get container status \"e9c38bbe15d84deb1873bc02f167d157b471f71259a406cefaac732efb600e4f\": rpc error: code = NotFound desc = could not find container \"e9c38bbe15d84deb1873bc02f167d157b471f71259a406cefaac732efb600e4f\": container with ID starting with e9c38bbe15d84deb1873bc02f167d157b471f71259a406cefaac732efb600e4f not found: ID does not exist" Dec 03 12:26:28 crc kubenswrapper[5052]: I1203 12:26:28.330583 5052 scope.go:117] "RemoveContainer" containerID="86638ee6f1d993932caa91451744753653fa3507238feac44e741c4134134430" Dec 03 12:26:28 crc kubenswrapper[5052]: E1203 12:26:28.330898 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"86638ee6f1d993932caa91451744753653fa3507238feac44e741c4134134430\": container with ID starting with 86638ee6f1d993932caa91451744753653fa3507238feac44e741c4134134430 not found: ID does not exist" containerID="86638ee6f1d993932caa91451744753653fa3507238feac44e741c4134134430" Dec 03 12:26:28 crc kubenswrapper[5052]: I1203 12:26:28.330935 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"86638ee6f1d993932caa91451744753653fa3507238feac44e741c4134134430"} err="failed to get container status \"86638ee6f1d993932caa91451744753653fa3507238feac44e741c4134134430\": rpc error: code = NotFound desc = could not find container \"86638ee6f1d993932caa91451744753653fa3507238feac44e741c4134134430\": container with ID starting with 86638ee6f1d993932caa91451744753653fa3507238feac44e741c4134134430 not found: ID does not exist" Dec 03 12:26:28 crc kubenswrapper[5052]: I1203 12:26:28.330959 5052 scope.go:117] "RemoveContainer" containerID="fda1434e3be27361a253ac86d38068672205b80750ce3eb51a12b74e073b1d2b" Dec 03 12:26:28 crc kubenswrapper[5052]: E1203 12:26:28.331293 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fda1434e3be27361a253ac86d38068672205b80750ce3eb51a12b74e073b1d2b\": container with ID starting with fda1434e3be27361a253ac86d38068672205b80750ce3eb51a12b74e073b1d2b not found: ID does not exist" containerID="fda1434e3be27361a253ac86d38068672205b80750ce3eb51a12b74e073b1d2b" Dec 03 12:26:28 crc kubenswrapper[5052]: I1203 12:26:28.331329 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fda1434e3be27361a253ac86d38068672205b80750ce3eb51a12b74e073b1d2b"} err="failed to get container status \"fda1434e3be27361a253ac86d38068672205b80750ce3eb51a12b74e073b1d2b\": rpc error: code = NotFound desc = could not find container \"fda1434e3be27361a253ac86d38068672205b80750ce3eb51a12b74e073b1d2b\": container with ID starting with fda1434e3be27361a253ac86d38068672205b80750ce3eb51a12b74e073b1d2b not found: ID does not exist" Dec 03 12:26:28 crc kubenswrapper[5052]: I1203 12:26:28.491972 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:26:28 crc kubenswrapper[5052]: I1203 12:26:28.506585 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:26:28 crc kubenswrapper[5052]: I1203 12:26:28.522887 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:26:28 crc kubenswrapper[5052]: E1203 12:26:28.523402 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="2040c0be-2fbd-4b01-9852-82d0210f1429" containerName="ceilometer-notification-agent" Dec 03 12:26:28 crc kubenswrapper[5052]: I1203 12:26:28.523429 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="2040c0be-2fbd-4b01-9852-82d0210f1429" containerName="ceilometer-notification-agent" Dec 03 12:26:28 crc kubenswrapper[5052]: E1203 12:26:28.523474 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="2040c0be-2fbd-4b01-9852-82d0210f1429" containerName="sg-core" Dec 03 12:26:28 crc kubenswrapper[5052]: I1203 12:26:28.523482 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="2040c0be-2fbd-4b01-9852-82d0210f1429" containerName="sg-core" Dec 03 12:26:28 crc kubenswrapper[5052]: E1203 12:26:28.523490 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="2040c0be-2fbd-4b01-9852-82d0210f1429" containerName="ceilometer-central-agent" Dec 03 12:26:28 crc kubenswrapper[5052]: I1203 12:26:28.523496 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="2040c0be-2fbd-4b01-9852-82d0210f1429" containerName="ceilometer-central-agent" Dec 03 12:26:28 crc kubenswrapper[5052]: E1203 12:26:28.523549 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="2040c0be-2fbd-4b01-9852-82d0210f1429" containerName="proxy-httpd" Dec 03 12:26:28 crc kubenswrapper[5052]: I1203 12:26:28.523559 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="2040c0be-2fbd-4b01-9852-82d0210f1429" containerName="proxy-httpd" Dec 03 12:26:28 crc kubenswrapper[5052]: I1203 12:26:28.523752 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="2040c0be-2fbd-4b01-9852-82d0210f1429" containerName="ceilometer-notification-agent" Dec 03 12:26:28 crc kubenswrapper[5052]: I1203 12:26:28.523779 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="2040c0be-2fbd-4b01-9852-82d0210f1429" containerName="proxy-httpd" Dec 03 12:26:28 crc kubenswrapper[5052]: I1203 12:26:28.523797 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="2040c0be-2fbd-4b01-9852-82d0210f1429" containerName="sg-core" Dec 03 12:26:28 crc kubenswrapper[5052]: I1203 12:26:28.523806 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="2040c0be-2fbd-4b01-9852-82d0210f1429" containerName="ceilometer-central-agent" Dec 03 12:26:28 crc kubenswrapper[5052]: I1203 12:26:28.525996 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:26:28 crc kubenswrapper[5052]: I1203 12:26:28.529095 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-config-data" Dec 03 12:26:28 crc kubenswrapper[5052]: I1203 12:26:28.529375 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-scripts" Dec 03 12:26:28 crc kubenswrapper[5052]: I1203 12:26:28.533384 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:26:28 crc kubenswrapper[5052]: I1203 12:26:28.571283 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-cell-mapping-hhtnd" Dec 03 12:26:28 crc kubenswrapper[5052]: I1203 12:26:28.624117 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/eb8c41b3-9282-4a42-8122-d648450a757d-scripts\") pod \"eb8c41b3-9282-4a42-8122-d648450a757d\" (UID: \"eb8c41b3-9282-4a42-8122-d648450a757d\") " Dec 03 12:26:28 crc kubenswrapper[5052]: I1203 12:26:28.624255 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/eb8c41b3-9282-4a42-8122-d648450a757d-config-data\") pod \"eb8c41b3-9282-4a42-8122-d648450a757d\" (UID: \"eb8c41b3-9282-4a42-8122-d648450a757d\") " Dec 03 12:26:28 crc kubenswrapper[5052]: I1203 12:26:28.624293 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hmvts\" (UniqueName: \"kubernetes.io/projected/eb8c41b3-9282-4a42-8122-d648450a757d-kube-api-access-hmvts\") pod \"eb8c41b3-9282-4a42-8122-d648450a757d\" (UID: \"eb8c41b3-9282-4a42-8122-d648450a757d\") " Dec 03 12:26:28 crc kubenswrapper[5052]: I1203 12:26:28.624395 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/eb8c41b3-9282-4a42-8122-d648450a757d-combined-ca-bundle\") pod \"eb8c41b3-9282-4a42-8122-d648450a757d\" (UID: \"eb8c41b3-9282-4a42-8122-d648450a757d\") " Dec 03 12:26:28 crc kubenswrapper[5052]: I1203 12:26:28.624617 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/6ab5f8d6-e515-4f7b-b14f-383b4180f4e7-log-httpd\") pod \"ceilometer-0\" (UID: \"6ab5f8d6-e515-4f7b-b14f-383b4180f4e7\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:26:28 crc kubenswrapper[5052]: I1203 12:26:28.624662 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/6ab5f8d6-e515-4f7b-b14f-383b4180f4e7-scripts\") pod \"ceilometer-0\" (UID: \"6ab5f8d6-e515-4f7b-b14f-383b4180f4e7\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:26:28 crc kubenswrapper[5052]: I1203 12:26:28.624754 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9mhgt\" (UniqueName: \"kubernetes.io/projected/6ab5f8d6-e515-4f7b-b14f-383b4180f4e7-kube-api-access-9mhgt\") pod \"ceilometer-0\" (UID: \"6ab5f8d6-e515-4f7b-b14f-383b4180f4e7\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:26:28 crc kubenswrapper[5052]: I1203 12:26:28.624804 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/6ab5f8d6-e515-4f7b-b14f-383b4180f4e7-run-httpd\") pod \"ceilometer-0\" (UID: \"6ab5f8d6-e515-4f7b-b14f-383b4180f4e7\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:26:28 crc kubenswrapper[5052]: I1203 12:26:28.624888 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6ab5f8d6-e515-4f7b-b14f-383b4180f4e7-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"6ab5f8d6-e515-4f7b-b14f-383b4180f4e7\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:26:28 crc kubenswrapper[5052]: I1203 12:26:28.624949 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/6ab5f8d6-e515-4f7b-b14f-383b4180f4e7-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"6ab5f8d6-e515-4f7b-b14f-383b4180f4e7\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:26:28 crc kubenswrapper[5052]: I1203 12:26:28.624976 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6ab5f8d6-e515-4f7b-b14f-383b4180f4e7-config-data\") pod \"ceilometer-0\" (UID: \"6ab5f8d6-e515-4f7b-b14f-383b4180f4e7\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:26:28 crc kubenswrapper[5052]: I1203 12:26:28.628142 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eb8c41b3-9282-4a42-8122-d648450a757d-kube-api-access-hmvts" (OuterVolumeSpecName: "kube-api-access-hmvts") pod "eb8c41b3-9282-4a42-8122-d648450a757d" (UID: "eb8c41b3-9282-4a42-8122-d648450a757d"). InnerVolumeSpecName "kube-api-access-hmvts". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:26:28 crc kubenswrapper[5052]: I1203 12:26:28.629165 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eb8c41b3-9282-4a42-8122-d648450a757d-scripts" (OuterVolumeSpecName: "scripts") pod "eb8c41b3-9282-4a42-8122-d648450a757d" (UID: "eb8c41b3-9282-4a42-8122-d648450a757d"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:26:28 crc kubenswrapper[5052]: I1203 12:26:28.656887 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eb8c41b3-9282-4a42-8122-d648450a757d-config-data" (OuterVolumeSpecName: "config-data") pod "eb8c41b3-9282-4a42-8122-d648450a757d" (UID: "eb8c41b3-9282-4a42-8122-d648450a757d"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:26:28 crc kubenswrapper[5052]: I1203 12:26:28.659181 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eb8c41b3-9282-4a42-8122-d648450a757d-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "eb8c41b3-9282-4a42-8122-d648450a757d" (UID: "eb8c41b3-9282-4a42-8122-d648450a757d"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:26:28 crc kubenswrapper[5052]: I1203 12:26:28.727087 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-9mhgt\" (UniqueName: \"kubernetes.io/projected/6ab5f8d6-e515-4f7b-b14f-383b4180f4e7-kube-api-access-9mhgt\") pod \"ceilometer-0\" (UID: \"6ab5f8d6-e515-4f7b-b14f-383b4180f4e7\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:26:28 crc kubenswrapper[5052]: I1203 12:26:28.727152 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/6ab5f8d6-e515-4f7b-b14f-383b4180f4e7-run-httpd\") pod \"ceilometer-0\" (UID: \"6ab5f8d6-e515-4f7b-b14f-383b4180f4e7\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:26:28 crc kubenswrapper[5052]: I1203 12:26:28.727214 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6ab5f8d6-e515-4f7b-b14f-383b4180f4e7-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"6ab5f8d6-e515-4f7b-b14f-383b4180f4e7\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:26:28 crc kubenswrapper[5052]: I1203 12:26:28.727246 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/6ab5f8d6-e515-4f7b-b14f-383b4180f4e7-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"6ab5f8d6-e515-4f7b-b14f-383b4180f4e7\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:26:28 crc kubenswrapper[5052]: I1203 12:26:28.727262 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6ab5f8d6-e515-4f7b-b14f-383b4180f4e7-config-data\") pod \"ceilometer-0\" (UID: \"6ab5f8d6-e515-4f7b-b14f-383b4180f4e7\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:26:28 crc kubenswrapper[5052]: I1203 12:26:28.727326 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/6ab5f8d6-e515-4f7b-b14f-383b4180f4e7-log-httpd\") pod \"ceilometer-0\" (UID: \"6ab5f8d6-e515-4f7b-b14f-383b4180f4e7\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:26:28 crc kubenswrapper[5052]: I1203 12:26:28.727359 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/6ab5f8d6-e515-4f7b-b14f-383b4180f4e7-scripts\") pod \"ceilometer-0\" (UID: \"6ab5f8d6-e515-4f7b-b14f-383b4180f4e7\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:26:28 crc kubenswrapper[5052]: I1203 12:26:28.727432 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/eb8c41b3-9282-4a42-8122-d648450a757d-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:26:28 crc kubenswrapper[5052]: I1203 12:26:28.727447 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/eb8c41b3-9282-4a42-8122-d648450a757d-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:26:28 crc kubenswrapper[5052]: I1203 12:26:28.727455 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/eb8c41b3-9282-4a42-8122-d648450a757d-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:26:28 crc kubenswrapper[5052]: I1203 12:26:28.727464 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-hmvts\" (UniqueName: \"kubernetes.io/projected/eb8c41b3-9282-4a42-8122-d648450a757d-kube-api-access-hmvts\") on node \"crc\" DevicePath \"\"" Dec 03 12:26:28 crc kubenswrapper[5052]: I1203 12:26:28.728237 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/6ab5f8d6-e515-4f7b-b14f-383b4180f4e7-run-httpd\") pod \"ceilometer-0\" (UID: \"6ab5f8d6-e515-4f7b-b14f-383b4180f4e7\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:26:28 crc kubenswrapper[5052]: I1203 12:26:28.731200 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/6ab5f8d6-e515-4f7b-b14f-383b4180f4e7-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"6ab5f8d6-e515-4f7b-b14f-383b4180f4e7\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:26:28 crc kubenswrapper[5052]: I1203 12:26:28.731726 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6ab5f8d6-e515-4f7b-b14f-383b4180f4e7-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"6ab5f8d6-e515-4f7b-b14f-383b4180f4e7\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:26:28 crc kubenswrapper[5052]: I1203 12:26:28.732002 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/6ab5f8d6-e515-4f7b-b14f-383b4180f4e7-log-httpd\") pod \"ceilometer-0\" (UID: \"6ab5f8d6-e515-4f7b-b14f-383b4180f4e7\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:26:28 crc kubenswrapper[5052]: I1203 12:26:28.736840 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6ab5f8d6-e515-4f7b-b14f-383b4180f4e7-config-data\") pod \"ceilometer-0\" (UID: \"6ab5f8d6-e515-4f7b-b14f-383b4180f4e7\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:26:28 crc kubenswrapper[5052]: I1203 12:26:28.737991 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/6ab5f8d6-e515-4f7b-b14f-383b4180f4e7-scripts\") pod \"ceilometer-0\" (UID: \"6ab5f8d6-e515-4f7b-b14f-383b4180f4e7\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:26:28 crc kubenswrapper[5052]: I1203 12:26:28.746939 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-9mhgt\" (UniqueName: \"kubernetes.io/projected/6ab5f8d6-e515-4f7b-b14f-383b4180f4e7-kube-api-access-9mhgt\") pod \"ceilometer-0\" (UID: \"6ab5f8d6-e515-4f7b-b14f-383b4180f4e7\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:26:28 crc kubenswrapper[5052]: I1203 12:26:28.873450 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:26:29 crc kubenswrapper[5052]: I1203 12:26:29.174492 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-cell-mapping-hhtnd" Dec 03 12:26:29 crc kubenswrapper[5052]: I1203 12:26:29.175541 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-cell-mapping-hhtnd" event={"ID":"eb8c41b3-9282-4a42-8122-d648450a757d","Type":"ContainerDied","Data":"1d9079c964e4dcdfef1627bf8ff9cab96c380172dcd2d1a1061ae072c6aa5103"} Dec 03 12:26:29 crc kubenswrapper[5052]: I1203 12:26:29.175573 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1d9079c964e4dcdfef1627bf8ff9cab96c380172dcd2d1a1061ae072c6aa5103" Dec 03 12:26:29 crc kubenswrapper[5052]: I1203 12:26:29.722491 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2040c0be-2fbd-4b01-9852-82d0210f1429" path="/var/lib/kubelet/pods/2040c0be-2fbd-4b01-9852-82d0210f1429/volumes" Dec 03 12:26:30 crc kubenswrapper[5052]: I1203 12:26:30.045572 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:26:30 crc kubenswrapper[5052]: I1203 12:26:30.045824 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-scheduler-0" podUID="5ee46523-66cc-414a-b720-542a1abc4021" containerName="nova-scheduler-scheduler" containerID="cri-o://72aac272f84d0737e78fd219a9666e2be0d21ec7dd8418b6d1d7be63eac32c80" gracePeriod=30 Dec 03 12:26:30 crc kubenswrapper[5052]: I1203 12:26:30.054396 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:26:30 crc kubenswrapper[5052]: I1203 12:26:30.054670 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-api-0" podUID="4406519b-7670-42e0-9fd2-b81b12401e90" containerName="nova-api-log" containerID="cri-o://ec32da3cb6b86d6c55cfcef58df37fb8c2efb88d4c84fb0f866050839df7c555" gracePeriod=30 Dec 03 12:26:30 crc kubenswrapper[5052]: I1203 12:26:30.054809 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-api-0" podUID="4406519b-7670-42e0-9fd2-b81b12401e90" containerName="nova-api-api" containerID="cri-o://fe6fcc4b5c35e5e89c8e77b13bee17e8b3be48617d27501fff73da3736a73b2b" gracePeriod=30 Dec 03 12:26:30 crc kubenswrapper[5052]: I1203 12:26:30.060418 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:26:30 crc kubenswrapper[5052]: I1203 12:26:30.061035 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-metadata-0" podUID="0a52121e-4974-4d01-977b-71b17710e9bd" containerName="nova-metadata-log" containerID="cri-o://e4a259915a2a2baf4e7137b0f920ba69c1764cfb2c5ddb3399fac3f7de720cd2" gracePeriod=30 Dec 03 12:26:30 crc kubenswrapper[5052]: I1203 12:26:30.061222 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-metadata-0" podUID="0a52121e-4974-4d01-977b-71b17710e9bd" containerName="nova-metadata-metadata" containerID="cri-o://715304a54c72ee68073a8cfbe3a1ff4db360e104855d120a09bfe027ecf57d61" gracePeriod=30 Dec 03 12:26:30 crc kubenswrapper[5052]: I1203 12:26:30.139709 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:26:30 crc kubenswrapper[5052]: I1203 12:26:30.224000 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"6ab5f8d6-e515-4f7b-b14f-383b4180f4e7","Type":"ContainerStarted","Data":"c8890686d3840eb2ccf4444693da2e2e55acbf4b6e5ded2e5bd7bee34eed05de"} Dec 03 12:26:31 crc kubenswrapper[5052]: I1203 12:26:31.235870 5052 generic.go:334] "Generic (PLEG): container finished" podID="0a52121e-4974-4d01-977b-71b17710e9bd" containerID="e4a259915a2a2baf4e7137b0f920ba69c1764cfb2c5ddb3399fac3f7de720cd2" exitCode=143 Dec 03 12:26:31 crc kubenswrapper[5052]: I1203 12:26:31.235922 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"0a52121e-4974-4d01-977b-71b17710e9bd","Type":"ContainerDied","Data":"e4a259915a2a2baf4e7137b0f920ba69c1764cfb2c5ddb3399fac3f7de720cd2"} Dec 03 12:26:31 crc kubenswrapper[5052]: I1203 12:26:31.325346 5052 generic.go:334] "Generic (PLEG): container finished" podID="4406519b-7670-42e0-9fd2-b81b12401e90" containerID="ec32da3cb6b86d6c55cfcef58df37fb8c2efb88d4c84fb0f866050839df7c555" exitCode=143 Dec 03 12:26:31 crc kubenswrapper[5052]: I1203 12:26:31.325439 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"4406519b-7670-42e0-9fd2-b81b12401e90","Type":"ContainerDied","Data":"ec32da3cb6b86d6c55cfcef58df37fb8c2efb88d4c84fb0f866050839df7c555"} Dec 03 12:26:31 crc kubenswrapper[5052]: I1203 12:26:31.327418 5052 generic.go:334] "Generic (PLEG): container finished" podID="07a34aba-b4ee-4b5e-92ea-d458af9399de" containerID="104040fbb1ee554dc74340bedf98f59d521b44f2da342566a71b536c53c4bc01" exitCode=0 Dec 03 12:26:31 crc kubenswrapper[5052]: I1203 12:26:31.327453 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-operators-qhzn9" event={"ID":"07a34aba-b4ee-4b5e-92ea-d458af9399de","Type":"ContainerDied","Data":"104040fbb1ee554dc74340bedf98f59d521b44f2da342566a71b536c53c4bc01"} Dec 03 12:26:33 crc kubenswrapper[5052]: I1203 12:26:33.468313 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/nova-api-0" podUID="4406519b-7670-42e0-9fd2-b81b12401e90" containerName="nova-api-log" probeResult="failure" output="Get \"http://10.217.0.114:8774/\": read tcp 10.217.0.2:33926->10.217.0.114:8774: read: connection reset by peer" Dec 03 12:26:33 crc kubenswrapper[5052]: I1203 12:26:33.468403 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/nova-api-0" podUID="4406519b-7670-42e0-9fd2-b81b12401e90" containerName="nova-api-api" probeResult="failure" output="Get \"http://10.217.0.114:8774/\": read tcp 10.217.0.2:33938->10.217.0.114:8774: read: connection reset by peer" Dec 03 12:26:33 crc kubenswrapper[5052]: I1203 12:26:33.629268 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/nova-metadata-0" podUID="0a52121e-4974-4d01-977b-71b17710e9bd" containerName="nova-metadata-metadata" probeResult="failure" output="Get \"http://10.217.0.113:8775/\": read tcp 10.217.0.2:58282->10.217.0.113:8775: read: connection reset by peer" Dec 03 12:26:33 crc kubenswrapper[5052]: I1203 12:26:33.629295 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/nova-metadata-0" podUID="0a52121e-4974-4d01-977b-71b17710e9bd" containerName="nova-metadata-log" probeResult="failure" output="Get \"http://10.217.0.113:8775/\": read tcp 10.217.0.2:58284->10.217.0.113:8775: read: connection reset by peer" Dec 03 12:26:34 crc kubenswrapper[5052]: E1203 12:26:34.095869 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of 72aac272f84d0737e78fd219a9666e2be0d21ec7dd8418b6d1d7be63eac32c80 is running failed: container process not found" containerID="72aac272f84d0737e78fd219a9666e2be0d21ec7dd8418b6d1d7be63eac32c80" cmd=["/usr/bin/pgrep","-r","DRST","nova-scheduler"] Dec 03 12:26:34 crc kubenswrapper[5052]: E1203 12:26:34.096402 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of 72aac272f84d0737e78fd219a9666e2be0d21ec7dd8418b6d1d7be63eac32c80 is running failed: container process not found" containerID="72aac272f84d0737e78fd219a9666e2be0d21ec7dd8418b6d1d7be63eac32c80" cmd=["/usr/bin/pgrep","-r","DRST","nova-scheduler"] Dec 03 12:26:34 crc kubenswrapper[5052]: E1203 12:26:34.096684 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of 72aac272f84d0737e78fd219a9666e2be0d21ec7dd8418b6d1d7be63eac32c80 is running failed: container process not found" containerID="72aac272f84d0737e78fd219a9666e2be0d21ec7dd8418b6d1d7be63eac32c80" cmd=["/usr/bin/pgrep","-r","DRST","nova-scheduler"] Dec 03 12:26:34 crc kubenswrapper[5052]: E1203 12:26:34.096709 5052 prober.go:104] "Probe errored" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of 72aac272f84d0737e78fd219a9666e2be0d21ec7dd8418b6d1d7be63eac32c80 is running failed: container process not found" probeType="Readiness" pod="openstack-kuttl-tests/nova-scheduler-0" podUID="5ee46523-66cc-414a-b720-542a1abc4021" containerName="nova-scheduler-scheduler" Dec 03 12:26:34 crc kubenswrapper[5052]: E1203 12:26:34.164822 5052 cadvisor_stats_provider.go:516] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5ee46523_66cc_414a_b720_542a1abc4021.slice/crio-conmon-72aac272f84d0737e78fd219a9666e2be0d21ec7dd8418b6d1d7be63eac32c80.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4406519b_7670_42e0_9fd2_b81b12401e90.slice/crio-conmon-fe6fcc4b5c35e5e89c8e77b13bee17e8b3be48617d27501fff73da3736a73b2b.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0a52121e_4974_4d01_977b_71b17710e9bd.slice/crio-conmon-715304a54c72ee68073a8cfbe3a1ff4db360e104855d120a09bfe027ecf57d61.scope\": RecentStats: unable to find data in memory cache]" Dec 03 12:26:34 crc kubenswrapper[5052]: I1203 12:26:34.367020 5052 generic.go:334] "Generic (PLEG): container finished" podID="5ee46523-66cc-414a-b720-542a1abc4021" containerID="72aac272f84d0737e78fd219a9666e2be0d21ec7dd8418b6d1d7be63eac32c80" exitCode=0 Dec 03 12:26:34 crc kubenswrapper[5052]: I1203 12:26:34.367109 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"5ee46523-66cc-414a-b720-542a1abc4021","Type":"ContainerDied","Data":"72aac272f84d0737e78fd219a9666e2be0d21ec7dd8418b6d1d7be63eac32c80"} Dec 03 12:26:34 crc kubenswrapper[5052]: I1203 12:26:34.370919 5052 generic.go:334] "Generic (PLEG): container finished" podID="0a52121e-4974-4d01-977b-71b17710e9bd" containerID="715304a54c72ee68073a8cfbe3a1ff4db360e104855d120a09bfe027ecf57d61" exitCode=0 Dec 03 12:26:34 crc kubenswrapper[5052]: I1203 12:26:34.371053 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"0a52121e-4974-4d01-977b-71b17710e9bd","Type":"ContainerDied","Data":"715304a54c72ee68073a8cfbe3a1ff4db360e104855d120a09bfe027ecf57d61"} Dec 03 12:26:34 crc kubenswrapper[5052]: I1203 12:26:34.381785 5052 generic.go:334] "Generic (PLEG): container finished" podID="4406519b-7670-42e0-9fd2-b81b12401e90" containerID="fe6fcc4b5c35e5e89c8e77b13bee17e8b3be48617d27501fff73da3736a73b2b" exitCode=0 Dec 03 12:26:34 crc kubenswrapper[5052]: I1203 12:26:34.381823 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"4406519b-7670-42e0-9fd2-b81b12401e90","Type":"ContainerDied","Data":"fe6fcc4b5c35e5e89c8e77b13bee17e8b3be48617d27501fff73da3736a73b2b"} Dec 03 12:26:34 crc kubenswrapper[5052]: I1203 12:26:34.506869 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:26:34 crc kubenswrapper[5052]: I1203 12:26:34.628338 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:26:34 crc kubenswrapper[5052]: I1203 12:26:34.690530 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r8z4v\" (UniqueName: \"kubernetes.io/projected/4406519b-7670-42e0-9fd2-b81b12401e90-kube-api-access-r8z4v\") pod \"4406519b-7670-42e0-9fd2-b81b12401e90\" (UID: \"4406519b-7670-42e0-9fd2-b81b12401e90\") " Dec 03 12:26:34 crc kubenswrapper[5052]: I1203 12:26:34.690920 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/4406519b-7670-42e0-9fd2-b81b12401e90-logs\") pod \"4406519b-7670-42e0-9fd2-b81b12401e90\" (UID: \"4406519b-7670-42e0-9fd2-b81b12401e90\") " Dec 03 12:26:34 crc kubenswrapper[5052]: I1203 12:26:34.690956 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/4406519b-7670-42e0-9fd2-b81b12401e90-config-data\") pod \"4406519b-7670-42e0-9fd2-b81b12401e90\" (UID: \"4406519b-7670-42e0-9fd2-b81b12401e90\") " Dec 03 12:26:34 crc kubenswrapper[5052]: I1203 12:26:34.691080 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4406519b-7670-42e0-9fd2-b81b12401e90-combined-ca-bundle\") pod \"4406519b-7670-42e0-9fd2-b81b12401e90\" (UID: \"4406519b-7670-42e0-9fd2-b81b12401e90\") " Dec 03 12:26:34 crc kubenswrapper[5052]: I1203 12:26:34.694196 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4406519b-7670-42e0-9fd2-b81b12401e90-logs" (OuterVolumeSpecName: "logs") pod "4406519b-7670-42e0-9fd2-b81b12401e90" (UID: "4406519b-7670-42e0-9fd2-b81b12401e90"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:26:34 crc kubenswrapper[5052]: I1203 12:26:34.705092 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4406519b-7670-42e0-9fd2-b81b12401e90-kube-api-access-r8z4v" (OuterVolumeSpecName: "kube-api-access-r8z4v") pod "4406519b-7670-42e0-9fd2-b81b12401e90" (UID: "4406519b-7670-42e0-9fd2-b81b12401e90"). InnerVolumeSpecName "kube-api-access-r8z4v". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:26:34 crc kubenswrapper[5052]: I1203 12:26:34.726657 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4406519b-7670-42e0-9fd2-b81b12401e90-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "4406519b-7670-42e0-9fd2-b81b12401e90" (UID: "4406519b-7670-42e0-9fd2-b81b12401e90"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:26:34 crc kubenswrapper[5052]: I1203 12:26:34.727001 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4406519b-7670-42e0-9fd2-b81b12401e90-config-data" (OuterVolumeSpecName: "config-data") pod "4406519b-7670-42e0-9fd2-b81b12401e90" (UID: "4406519b-7670-42e0-9fd2-b81b12401e90"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:26:34 crc kubenswrapper[5052]: I1203 12:26:34.793133 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kdgw7\" (UniqueName: \"kubernetes.io/projected/0a52121e-4974-4d01-977b-71b17710e9bd-kube-api-access-kdgw7\") pod \"0a52121e-4974-4d01-977b-71b17710e9bd\" (UID: \"0a52121e-4974-4d01-977b-71b17710e9bd\") " Dec 03 12:26:34 crc kubenswrapper[5052]: I1203 12:26:34.793217 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0a52121e-4974-4d01-977b-71b17710e9bd-combined-ca-bundle\") pod \"0a52121e-4974-4d01-977b-71b17710e9bd\" (UID: \"0a52121e-4974-4d01-977b-71b17710e9bd\") " Dec 03 12:26:34 crc kubenswrapper[5052]: I1203 12:26:34.793280 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/0a52121e-4974-4d01-977b-71b17710e9bd-logs\") pod \"0a52121e-4974-4d01-977b-71b17710e9bd\" (UID: \"0a52121e-4974-4d01-977b-71b17710e9bd\") " Dec 03 12:26:34 crc kubenswrapper[5052]: I1203 12:26:34.793306 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/0a52121e-4974-4d01-977b-71b17710e9bd-config-data\") pod \"0a52121e-4974-4d01-977b-71b17710e9bd\" (UID: \"0a52121e-4974-4d01-977b-71b17710e9bd\") " Dec 03 12:26:34 crc kubenswrapper[5052]: I1203 12:26:34.793838 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-r8z4v\" (UniqueName: \"kubernetes.io/projected/4406519b-7670-42e0-9fd2-b81b12401e90-kube-api-access-r8z4v\") on node \"crc\" DevicePath \"\"" Dec 03 12:26:34 crc kubenswrapper[5052]: I1203 12:26:34.793855 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/4406519b-7670-42e0-9fd2-b81b12401e90-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:26:34 crc kubenswrapper[5052]: I1203 12:26:34.793863 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/4406519b-7670-42e0-9fd2-b81b12401e90-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:26:34 crc kubenswrapper[5052]: I1203 12:26:34.793871 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4406519b-7670-42e0-9fd2-b81b12401e90-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:26:34 crc kubenswrapper[5052]: I1203 12:26:34.796628 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a52121e-4974-4d01-977b-71b17710e9bd-logs" (OuterVolumeSpecName: "logs") pod "0a52121e-4974-4d01-977b-71b17710e9bd" (UID: "0a52121e-4974-4d01-977b-71b17710e9bd"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:26:34 crc kubenswrapper[5052]: I1203 12:26:34.810791 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0a52121e-4974-4d01-977b-71b17710e9bd-kube-api-access-kdgw7" (OuterVolumeSpecName: "kube-api-access-kdgw7") pod "0a52121e-4974-4d01-977b-71b17710e9bd" (UID: "0a52121e-4974-4d01-977b-71b17710e9bd"). InnerVolumeSpecName "kube-api-access-kdgw7". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:26:34 crc kubenswrapper[5052]: I1203 12:26:34.848227 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0a52121e-4974-4d01-977b-71b17710e9bd-config-data" (OuterVolumeSpecName: "config-data") pod "0a52121e-4974-4d01-977b-71b17710e9bd" (UID: "0a52121e-4974-4d01-977b-71b17710e9bd"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:26:34 crc kubenswrapper[5052]: I1203 12:26:34.896945 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/0a52121e-4974-4d01-977b-71b17710e9bd-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:26:34 crc kubenswrapper[5052]: I1203 12:26:34.896977 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/0a52121e-4974-4d01-977b-71b17710e9bd-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:26:34 crc kubenswrapper[5052]: I1203 12:26:34.896993 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-kdgw7\" (UniqueName: \"kubernetes.io/projected/0a52121e-4974-4d01-977b-71b17710e9bd-kube-api-access-kdgw7\") on node \"crc\" DevicePath \"\"" Dec 03 12:26:34 crc kubenswrapper[5052]: I1203 12:26:34.912496 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0a52121e-4974-4d01-977b-71b17710e9bd-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "0a52121e-4974-4d01-977b-71b17710e9bd" (UID: "0a52121e-4974-4d01-977b-71b17710e9bd"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:26:34 crc kubenswrapper[5052]: I1203 12:26:34.968734 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:26:34 crc kubenswrapper[5052]: I1203 12:26:34.998765 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0a52121e-4974-4d01-977b-71b17710e9bd-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:26:35 crc kubenswrapper[5052]: I1203 12:26:35.100602 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5ee46523-66cc-414a-b720-542a1abc4021-combined-ca-bundle\") pod \"5ee46523-66cc-414a-b720-542a1abc4021\" (UID: \"5ee46523-66cc-414a-b720-542a1abc4021\") " Dec 03 12:26:35 crc kubenswrapper[5052]: I1203 12:26:35.100915 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cq4b2\" (UniqueName: \"kubernetes.io/projected/5ee46523-66cc-414a-b720-542a1abc4021-kube-api-access-cq4b2\") pod \"5ee46523-66cc-414a-b720-542a1abc4021\" (UID: \"5ee46523-66cc-414a-b720-542a1abc4021\") " Dec 03 12:26:35 crc kubenswrapper[5052]: I1203 12:26:35.101065 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/5ee46523-66cc-414a-b720-542a1abc4021-config-data\") pod \"5ee46523-66cc-414a-b720-542a1abc4021\" (UID: \"5ee46523-66cc-414a-b720-542a1abc4021\") " Dec 03 12:26:35 crc kubenswrapper[5052]: I1203 12:26:35.108881 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5ee46523-66cc-414a-b720-542a1abc4021-kube-api-access-cq4b2" (OuterVolumeSpecName: "kube-api-access-cq4b2") pod "5ee46523-66cc-414a-b720-542a1abc4021" (UID: "5ee46523-66cc-414a-b720-542a1abc4021"). InnerVolumeSpecName "kube-api-access-cq4b2". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:26:35 crc kubenswrapper[5052]: I1203 12:26:35.126555 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5ee46523-66cc-414a-b720-542a1abc4021-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "5ee46523-66cc-414a-b720-542a1abc4021" (UID: "5ee46523-66cc-414a-b720-542a1abc4021"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:26:35 crc kubenswrapper[5052]: I1203 12:26:35.128880 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5ee46523-66cc-414a-b720-542a1abc4021-config-data" (OuterVolumeSpecName: "config-data") pod "5ee46523-66cc-414a-b720-542a1abc4021" (UID: "5ee46523-66cc-414a-b720-542a1abc4021"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:26:35 crc kubenswrapper[5052]: I1203 12:26:35.202859 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-cq4b2\" (UniqueName: \"kubernetes.io/projected/5ee46523-66cc-414a-b720-542a1abc4021-kube-api-access-cq4b2\") on node \"crc\" DevicePath \"\"" Dec 03 12:26:35 crc kubenswrapper[5052]: I1203 12:26:35.202894 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/5ee46523-66cc-414a-b720-542a1abc4021-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:26:35 crc kubenswrapper[5052]: I1203 12:26:35.202903 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5ee46523-66cc-414a-b720-542a1abc4021-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:26:35 crc kubenswrapper[5052]: I1203 12:26:35.395614 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"0a52121e-4974-4d01-977b-71b17710e9bd","Type":"ContainerDied","Data":"d210d37650a68534c9f6cfb73cbc169bce24cd3a86fb2b62f730b0245234dd17"} Dec 03 12:26:35 crc kubenswrapper[5052]: I1203 12:26:35.395856 5052 scope.go:117] "RemoveContainer" containerID="715304a54c72ee68073a8cfbe3a1ff4db360e104855d120a09bfe027ecf57d61" Dec 03 12:26:35 crc kubenswrapper[5052]: I1203 12:26:35.396109 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:26:35 crc kubenswrapper[5052]: I1203 12:26:35.398234 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"6ab5f8d6-e515-4f7b-b14f-383b4180f4e7","Type":"ContainerStarted","Data":"63734bc615818a6cadecc9f3d7235c2f0339297c97215ad371cef23fc29afd3c"} Dec 03 12:26:35 crc kubenswrapper[5052]: I1203 12:26:35.398283 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"6ab5f8d6-e515-4f7b-b14f-383b4180f4e7","Type":"ContainerStarted","Data":"17e6a6fdbd72ea6cd16402a2b231dc4e88079b9588859838d5611cfd4bb232c5"} Dec 03 12:26:35 crc kubenswrapper[5052]: I1203 12:26:35.400664 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"4406519b-7670-42e0-9fd2-b81b12401e90","Type":"ContainerDied","Data":"fc91c2fb3a4da3931904532e155fb104636a8874eef93e70df5a6aa688151cba"} Dec 03 12:26:35 crc kubenswrapper[5052]: I1203 12:26:35.400755 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:26:35 crc kubenswrapper[5052]: I1203 12:26:35.412785 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-operators-qhzn9" event={"ID":"07a34aba-b4ee-4b5e-92ea-d458af9399de","Type":"ContainerStarted","Data":"4085fb5a47809a180ce42109621ec03799dfe9738d29b728f7f0b91a814d42a2"} Dec 03 12:26:35 crc kubenswrapper[5052]: I1203 12:26:35.428237 5052 scope.go:117] "RemoveContainer" containerID="e4a259915a2a2baf4e7137b0f920ba69c1764cfb2c5ddb3399fac3f7de720cd2" Dec 03 12:26:35 crc kubenswrapper[5052]: I1203 12:26:35.428555 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"5ee46523-66cc-414a-b720-542a1abc4021","Type":"ContainerDied","Data":"4a4b2bff6ebdad7eb07a577a7edcf9cb015dce95f70c2b8107c1656c94d0a869"} Dec 03 12:26:35 crc kubenswrapper[5052]: I1203 12:26:35.428690 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:26:35 crc kubenswrapper[5052]: I1203 12:26:35.444116 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-marketplace/redhat-operators-qhzn9" podStartSLOduration=4.379516918 podStartE2EDuration="11.444099097s" podCreationTimestamp="2025-12-03 12:26:24 +0000 UTC" firstStartedPulling="2025-12-03 12:26:27.113147714 +0000 UTC m=+3977.701389478" lastFinishedPulling="2025-12-03 12:26:34.177729893 +0000 UTC m=+3984.765971657" observedRunningTime="2025-12-03 12:26:35.438382064 +0000 UTC m=+3986.026623838" watchObservedRunningTime="2025-12-03 12:26:35.444099097 +0000 UTC m=+3986.032340861" Dec 03 12:26:35 crc kubenswrapper[5052]: I1203 12:26:35.482222 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:26:35 crc kubenswrapper[5052]: I1203 12:26:35.484832 5052 scope.go:117] "RemoveContainer" containerID="fe6fcc4b5c35e5e89c8e77b13bee17e8b3be48617d27501fff73da3736a73b2b" Dec 03 12:26:35 crc kubenswrapper[5052]: I1203 12:26:35.495573 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:26:35 crc kubenswrapper[5052]: I1203 12:26:35.560184 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:26:35 crc kubenswrapper[5052]: I1203 12:26:35.567383 5052 scope.go:117] "RemoveContainer" containerID="ec32da3cb6b86d6c55cfcef58df37fb8c2efb88d4c84fb0f866050839df7c555" Dec 03 12:26:35 crc kubenswrapper[5052]: I1203 12:26:35.599458 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:26:35 crc kubenswrapper[5052]: I1203 12:26:35.616953 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:26:35 crc kubenswrapper[5052]: I1203 12:26:35.618399 5052 scope.go:117] "RemoveContainer" containerID="72aac272f84d0737e78fd219a9666e2be0d21ec7dd8418b6d1d7be63eac32c80" Dec 03 12:26:35 crc kubenswrapper[5052]: E1203 12:26:35.618677 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="0a52121e-4974-4d01-977b-71b17710e9bd" containerName="nova-metadata-metadata" Dec 03 12:26:35 crc kubenswrapper[5052]: I1203 12:26:35.618761 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a52121e-4974-4d01-977b-71b17710e9bd" containerName="nova-metadata-metadata" Dec 03 12:26:35 crc kubenswrapper[5052]: E1203 12:26:35.618776 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="4406519b-7670-42e0-9fd2-b81b12401e90" containerName="nova-api-log" Dec 03 12:26:35 crc kubenswrapper[5052]: I1203 12:26:35.618783 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="4406519b-7670-42e0-9fd2-b81b12401e90" containerName="nova-api-log" Dec 03 12:26:35 crc kubenswrapper[5052]: E1203 12:26:35.618795 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="5ee46523-66cc-414a-b720-542a1abc4021" containerName="nova-scheduler-scheduler" Dec 03 12:26:35 crc kubenswrapper[5052]: I1203 12:26:35.618805 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ee46523-66cc-414a-b720-542a1abc4021" containerName="nova-scheduler-scheduler" Dec 03 12:26:35 crc kubenswrapper[5052]: E1203 12:26:35.619149 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="4406519b-7670-42e0-9fd2-b81b12401e90" containerName="nova-api-api" Dec 03 12:26:35 crc kubenswrapper[5052]: I1203 12:26:35.619173 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="4406519b-7670-42e0-9fd2-b81b12401e90" containerName="nova-api-api" Dec 03 12:26:35 crc kubenswrapper[5052]: E1203 12:26:35.619195 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="0a52121e-4974-4d01-977b-71b17710e9bd" containerName="nova-metadata-log" Dec 03 12:26:35 crc kubenswrapper[5052]: I1203 12:26:35.619201 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a52121e-4974-4d01-977b-71b17710e9bd" containerName="nova-metadata-log" Dec 03 12:26:35 crc kubenswrapper[5052]: E1203 12:26:35.619217 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="eb8c41b3-9282-4a42-8122-d648450a757d" containerName="nova-manage" Dec 03 12:26:35 crc kubenswrapper[5052]: I1203 12:26:35.619223 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="eb8c41b3-9282-4a42-8122-d648450a757d" containerName="nova-manage" Dec 03 12:26:35 crc kubenswrapper[5052]: I1203 12:26:35.619787 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="eb8c41b3-9282-4a42-8122-d648450a757d" containerName="nova-manage" Dec 03 12:26:35 crc kubenswrapper[5052]: I1203 12:26:35.619807 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="0a52121e-4974-4d01-977b-71b17710e9bd" containerName="nova-metadata-log" Dec 03 12:26:35 crc kubenswrapper[5052]: I1203 12:26:35.619817 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="4406519b-7670-42e0-9fd2-b81b12401e90" containerName="nova-api-log" Dec 03 12:26:35 crc kubenswrapper[5052]: I1203 12:26:35.619829 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="4406519b-7670-42e0-9fd2-b81b12401e90" containerName="nova-api-api" Dec 03 12:26:35 crc kubenswrapper[5052]: I1203 12:26:35.619838 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="5ee46523-66cc-414a-b720-542a1abc4021" containerName="nova-scheduler-scheduler" Dec 03 12:26:35 crc kubenswrapper[5052]: I1203 12:26:35.619847 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="0a52121e-4974-4d01-977b-71b17710e9bd" containerName="nova-metadata-metadata" Dec 03 12:26:35 crc kubenswrapper[5052]: I1203 12:26:35.626235 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:26:35 crc kubenswrapper[5052]: I1203 12:26:35.629587 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:26:35 crc kubenswrapper[5052]: I1203 12:26:35.634283 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-metadata-config-data" Dec 03 12:26:35 crc kubenswrapper[5052]: I1203 12:26:35.637567 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:26:35 crc kubenswrapper[5052]: I1203 12:26:35.639537 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:26:35 crc kubenswrapper[5052]: I1203 12:26:35.642296 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-api-config-data" Dec 03 12:26:35 crc kubenswrapper[5052]: I1203 12:26:35.662399 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:26:35 crc kubenswrapper[5052]: I1203 12:26:35.671274 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:26:35 crc kubenswrapper[5052]: I1203 12:26:35.701603 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0a52121e-4974-4d01-977b-71b17710e9bd" path="/var/lib/kubelet/pods/0a52121e-4974-4d01-977b-71b17710e9bd/volumes" Dec 03 12:26:35 crc kubenswrapper[5052]: I1203 12:26:35.702546 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4406519b-7670-42e0-9fd2-b81b12401e90" path="/var/lib/kubelet/pods/4406519b-7670-42e0-9fd2-b81b12401e90/volumes" Dec 03 12:26:35 crc kubenswrapper[5052]: I1203 12:26:35.703244 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5ee46523-66cc-414a-b720-542a1abc4021" path="/var/lib/kubelet/pods/5ee46523-66cc-414a-b720-542a1abc4021/volumes" Dec 03 12:26:35 crc kubenswrapper[5052]: I1203 12:26:35.704633 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:26:35 crc kubenswrapper[5052]: I1203 12:26:35.704672 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:26:35 crc kubenswrapper[5052]: I1203 12:26:35.706299 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:26:35 crc kubenswrapper[5052]: I1203 12:26:35.709892 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-scheduler-config-data" Dec 03 12:26:35 crc kubenswrapper[5052]: I1203 12:26:35.713450 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:26:35 crc kubenswrapper[5052]: I1203 12:26:35.719641 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8e052136-a855-4a20-a8c3-38a8bef97dfc-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"8e052136-a855-4a20-a8c3-38a8bef97dfc\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:26:35 crc kubenswrapper[5052]: I1203 12:26:35.720039 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v84g4\" (UniqueName: \"kubernetes.io/projected/8e052136-a855-4a20-a8c3-38a8bef97dfc-kube-api-access-v84g4\") pod \"nova-metadata-0\" (UID: \"8e052136-a855-4a20-a8c3-38a8bef97dfc\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:26:35 crc kubenswrapper[5052]: I1203 12:26:35.726065 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/8e052136-a855-4a20-a8c3-38a8bef97dfc-config-data\") pod \"nova-metadata-0\" (UID: \"8e052136-a855-4a20-a8c3-38a8bef97dfc\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:26:35 crc kubenswrapper[5052]: I1203 12:26:35.726295 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/8e052136-a855-4a20-a8c3-38a8bef97dfc-logs\") pod \"nova-metadata-0\" (UID: \"8e052136-a855-4a20-a8c3-38a8bef97dfc\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:26:35 crc kubenswrapper[5052]: I1203 12:26:35.828477 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6f4qx\" (UniqueName: \"kubernetes.io/projected/7ef14abf-76c0-4db5-9291-e0f9675425d1-kube-api-access-6f4qx\") pod \"nova-scheduler-0\" (UID: \"7ef14abf-76c0-4db5-9291-e0f9675425d1\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:26:35 crc kubenswrapper[5052]: I1203 12:26:35.828566 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/8e052136-a855-4a20-a8c3-38a8bef97dfc-logs\") pod \"nova-metadata-0\" (UID: \"8e052136-a855-4a20-a8c3-38a8bef97dfc\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:26:35 crc kubenswrapper[5052]: I1203 12:26:35.828610 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/7ef14abf-76c0-4db5-9291-e0f9675425d1-config-data\") pod \"nova-scheduler-0\" (UID: \"7ef14abf-76c0-4db5-9291-e0f9675425d1\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:26:35 crc kubenswrapper[5052]: I1203 12:26:35.828641 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8e052136-a855-4a20-a8c3-38a8bef97dfc-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"8e052136-a855-4a20-a8c3-38a8bef97dfc\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:26:35 crc kubenswrapper[5052]: I1203 12:26:35.828662 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-v84g4\" (UniqueName: \"kubernetes.io/projected/8e052136-a855-4a20-a8c3-38a8bef97dfc-kube-api-access-v84g4\") pod \"nova-metadata-0\" (UID: \"8e052136-a855-4a20-a8c3-38a8bef97dfc\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:26:35 crc kubenswrapper[5052]: I1203 12:26:35.828732 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/7ef14abf-76c0-4db5-9291-e0f9675425d1-combined-ca-bundle\") pod \"nova-scheduler-0\" (UID: \"7ef14abf-76c0-4db5-9291-e0f9675425d1\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:26:35 crc kubenswrapper[5052]: I1203 12:26:35.828749 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/7e415947-bb79-4d7b-80fb-f6d290343e19-logs\") pod \"nova-api-0\" (UID: \"7e415947-bb79-4d7b-80fb-f6d290343e19\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:26:35 crc kubenswrapper[5052]: I1203 12:26:35.828769 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/7e415947-bb79-4d7b-80fb-f6d290343e19-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"7e415947-bb79-4d7b-80fb-f6d290343e19\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:26:35 crc kubenswrapper[5052]: I1203 12:26:35.828810 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/7e415947-bb79-4d7b-80fb-f6d290343e19-config-data\") pod \"nova-api-0\" (UID: \"7e415947-bb79-4d7b-80fb-f6d290343e19\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:26:35 crc kubenswrapper[5052]: I1203 12:26:35.828845 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/8e052136-a855-4a20-a8c3-38a8bef97dfc-config-data\") pod \"nova-metadata-0\" (UID: \"8e052136-a855-4a20-a8c3-38a8bef97dfc\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:26:35 crc kubenswrapper[5052]: I1203 12:26:35.828875 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nbst7\" (UniqueName: \"kubernetes.io/projected/7e415947-bb79-4d7b-80fb-f6d290343e19-kube-api-access-nbst7\") pod \"nova-api-0\" (UID: \"7e415947-bb79-4d7b-80fb-f6d290343e19\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:26:35 crc kubenswrapper[5052]: I1203 12:26:35.829777 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/8e052136-a855-4a20-a8c3-38a8bef97dfc-logs\") pod \"nova-metadata-0\" (UID: \"8e052136-a855-4a20-a8c3-38a8bef97dfc\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:26:35 crc kubenswrapper[5052]: I1203 12:26:35.835032 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8e052136-a855-4a20-a8c3-38a8bef97dfc-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"8e052136-a855-4a20-a8c3-38a8bef97dfc\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:26:35 crc kubenswrapper[5052]: I1203 12:26:35.850828 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/8e052136-a855-4a20-a8c3-38a8bef97dfc-config-data\") pod \"nova-metadata-0\" (UID: \"8e052136-a855-4a20-a8c3-38a8bef97dfc\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:26:35 crc kubenswrapper[5052]: I1203 12:26:35.853427 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-v84g4\" (UniqueName: \"kubernetes.io/projected/8e052136-a855-4a20-a8c3-38a8bef97dfc-kube-api-access-v84g4\") pod \"nova-metadata-0\" (UID: \"8e052136-a855-4a20-a8c3-38a8bef97dfc\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:26:35 crc kubenswrapper[5052]: I1203 12:26:35.930562 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/7ef14abf-76c0-4db5-9291-e0f9675425d1-combined-ca-bundle\") pod \"nova-scheduler-0\" (UID: \"7ef14abf-76c0-4db5-9291-e0f9675425d1\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:26:35 crc kubenswrapper[5052]: I1203 12:26:35.930609 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/7e415947-bb79-4d7b-80fb-f6d290343e19-logs\") pod \"nova-api-0\" (UID: \"7e415947-bb79-4d7b-80fb-f6d290343e19\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:26:35 crc kubenswrapper[5052]: I1203 12:26:35.930635 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/7e415947-bb79-4d7b-80fb-f6d290343e19-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"7e415947-bb79-4d7b-80fb-f6d290343e19\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:26:35 crc kubenswrapper[5052]: I1203 12:26:35.930669 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/7e415947-bb79-4d7b-80fb-f6d290343e19-config-data\") pod \"nova-api-0\" (UID: \"7e415947-bb79-4d7b-80fb-f6d290343e19\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:26:35 crc kubenswrapper[5052]: I1203 12:26:35.930709 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-nbst7\" (UniqueName: \"kubernetes.io/projected/7e415947-bb79-4d7b-80fb-f6d290343e19-kube-api-access-nbst7\") pod \"nova-api-0\" (UID: \"7e415947-bb79-4d7b-80fb-f6d290343e19\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:26:35 crc kubenswrapper[5052]: I1203 12:26:35.930741 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-6f4qx\" (UniqueName: \"kubernetes.io/projected/7ef14abf-76c0-4db5-9291-e0f9675425d1-kube-api-access-6f4qx\") pod \"nova-scheduler-0\" (UID: \"7ef14abf-76c0-4db5-9291-e0f9675425d1\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:26:35 crc kubenswrapper[5052]: I1203 12:26:35.930780 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/7ef14abf-76c0-4db5-9291-e0f9675425d1-config-data\") pod \"nova-scheduler-0\" (UID: \"7ef14abf-76c0-4db5-9291-e0f9675425d1\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:26:35 crc kubenswrapper[5052]: I1203 12:26:35.932093 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/7e415947-bb79-4d7b-80fb-f6d290343e19-logs\") pod \"nova-api-0\" (UID: \"7e415947-bb79-4d7b-80fb-f6d290343e19\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:26:35 crc kubenswrapper[5052]: I1203 12:26:35.936407 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/7e415947-bb79-4d7b-80fb-f6d290343e19-config-data\") pod \"nova-api-0\" (UID: \"7e415947-bb79-4d7b-80fb-f6d290343e19\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:26:35 crc kubenswrapper[5052]: I1203 12:26:35.937277 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/7ef14abf-76c0-4db5-9291-e0f9675425d1-combined-ca-bundle\") pod \"nova-scheduler-0\" (UID: \"7ef14abf-76c0-4db5-9291-e0f9675425d1\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:26:35 crc kubenswrapper[5052]: I1203 12:26:35.947000 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/7ef14abf-76c0-4db5-9291-e0f9675425d1-config-data\") pod \"nova-scheduler-0\" (UID: \"7ef14abf-76c0-4db5-9291-e0f9675425d1\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:26:35 crc kubenswrapper[5052]: I1203 12:26:35.949153 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:26:35 crc kubenswrapper[5052]: I1203 12:26:35.950273 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-6f4qx\" (UniqueName: \"kubernetes.io/projected/7ef14abf-76c0-4db5-9291-e0f9675425d1-kube-api-access-6f4qx\") pod \"nova-scheduler-0\" (UID: \"7ef14abf-76c0-4db5-9291-e0f9675425d1\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:26:35 crc kubenswrapper[5052]: I1203 12:26:35.950536 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/7e415947-bb79-4d7b-80fb-f6d290343e19-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"7e415947-bb79-4d7b-80fb-f6d290343e19\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:26:35 crc kubenswrapper[5052]: I1203 12:26:35.950752 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-nbst7\" (UniqueName: \"kubernetes.io/projected/7e415947-bb79-4d7b-80fb-f6d290343e19-kube-api-access-nbst7\") pod \"nova-api-0\" (UID: \"7e415947-bb79-4d7b-80fb-f6d290343e19\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:26:36 crc kubenswrapper[5052]: I1203 12:26:36.084253 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:26:36 crc kubenswrapper[5052]: I1203 12:26:36.124485 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:26:36 crc kubenswrapper[5052]: I1203 12:26:36.440048 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:26:36 crc kubenswrapper[5052]: I1203 12:26:36.470585 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"6ab5f8d6-e515-4f7b-b14f-383b4180f4e7","Type":"ContainerStarted","Data":"94b056c86cf3f39157a4c778b82ad7f6fc7c02b88c1e228893f8382b18fe8241"} Dec 03 12:26:36 crc kubenswrapper[5052]: I1203 12:26:36.609644 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:26:36 crc kubenswrapper[5052]: W1203 12:26:36.612353 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7ef14abf_76c0_4db5_9291_e0f9675425d1.slice/crio-fe8d431aa6353950723b0a0923ea34d7e989c31fbfadfdca362c4bf3a984602c WatchSource:0}: Error finding container fe8d431aa6353950723b0a0923ea34d7e989c31fbfadfdca362c4bf3a984602c: Status 404 returned error can't find the container with id fe8d431aa6353950723b0a0923ea34d7e989c31fbfadfdca362c4bf3a984602c Dec 03 12:26:36 crc kubenswrapper[5052]: I1203 12:26:36.695899 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:26:37 crc kubenswrapper[5052]: I1203 12:26:37.485960 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"8e052136-a855-4a20-a8c3-38a8bef97dfc","Type":"ContainerStarted","Data":"0e8b8edad48c623ae81447848a940a98d6827814b85cda0ff7b96abb10a64815"} Dec 03 12:26:37 crc kubenswrapper[5052]: I1203 12:26:37.486301 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"8e052136-a855-4a20-a8c3-38a8bef97dfc","Type":"ContainerStarted","Data":"f679be3b6603c919c59273998d120bbb192e07c332db88e94aabe9e26ca2af36"} Dec 03 12:26:37 crc kubenswrapper[5052]: I1203 12:26:37.486317 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"8e052136-a855-4a20-a8c3-38a8bef97dfc","Type":"ContainerStarted","Data":"e5cdd00f5d6f4cc4c468c717371122ada5030689733c650b8b9165f773948b1f"} Dec 03 12:26:37 crc kubenswrapper[5052]: I1203 12:26:37.488220 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"7e415947-bb79-4d7b-80fb-f6d290343e19","Type":"ContainerStarted","Data":"5acdaf1de9637eb9a2dcd430e89e4aa32e1830b27b6a59fec9de95f2f8f0c88f"} Dec 03 12:26:37 crc kubenswrapper[5052]: I1203 12:26:37.488267 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"7e415947-bb79-4d7b-80fb-f6d290343e19","Type":"ContainerStarted","Data":"281db729b55e9d4d2851c2134872d5a117b0767da33f8a3837143567bbe3f595"} Dec 03 12:26:37 crc kubenswrapper[5052]: I1203 12:26:37.488276 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"7e415947-bb79-4d7b-80fb-f6d290343e19","Type":"ContainerStarted","Data":"50a72c7042df3d2414f13ba1037b71f17e5e6a078091df60b1f2fd7de458fc50"} Dec 03 12:26:37 crc kubenswrapper[5052]: I1203 12:26:37.490752 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"7ef14abf-76c0-4db5-9291-e0f9675425d1","Type":"ContainerStarted","Data":"74ef6e1fe61b1c5ce0d3bf482bd7a2e9edd6da7a23654c82cd5a4d8527f480c3"} Dec 03 12:26:37 crc kubenswrapper[5052]: I1203 12:26:37.490828 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"7ef14abf-76c0-4db5-9291-e0f9675425d1","Type":"ContainerStarted","Data":"fe8d431aa6353950723b0a0923ea34d7e989c31fbfadfdca362c4bf3a984602c"} Dec 03 12:26:37 crc kubenswrapper[5052]: I1203 12:26:37.526666 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-metadata-0" podStartSLOduration=2.5266454769999998 podStartE2EDuration="2.526645477s" podCreationTimestamp="2025-12-03 12:26:35 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:26:37.523353723 +0000 UTC m=+3988.111595487" watchObservedRunningTime="2025-12-03 12:26:37.526645477 +0000 UTC m=+3988.114887241" Dec 03 12:26:37 crc kubenswrapper[5052]: I1203 12:26:37.559636 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-scheduler-0" podStartSLOduration=2.559604578 podStartE2EDuration="2.559604578s" podCreationTimestamp="2025-12-03 12:26:35 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:26:37.544128896 +0000 UTC m=+3988.132370660" watchObservedRunningTime="2025-12-03 12:26:37.559604578 +0000 UTC m=+3988.147846342" Dec 03 12:26:37 crc kubenswrapper[5052]: I1203 12:26:37.591909 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-api-0" podStartSLOduration=2.591884679 podStartE2EDuration="2.591884679s" podCreationTimestamp="2025-12-03 12:26:35 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:26:37.562311775 +0000 UTC m=+3988.150553549" watchObservedRunningTime="2025-12-03 12:26:37.591884679 +0000 UTC m=+3988.180126443" Dec 03 12:26:38 crc kubenswrapper[5052]: I1203 12:26:38.501753 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"6ab5f8d6-e515-4f7b-b14f-383b4180f4e7","Type":"ContainerStarted","Data":"059f15379c22e9cbd4b778847ac6fc1ba70f4d80d669d8256de7257a96a80dbb"} Dec 03 12:26:38 crc kubenswrapper[5052]: I1203 12:26:38.531287 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/ceilometer-0" podStartSLOduration=3.379837694 podStartE2EDuration="10.531268261s" podCreationTimestamp="2025-12-03 12:26:28 +0000 UTC" firstStartedPulling="2025-12-03 12:26:30.198765714 +0000 UTC m=+3980.787007468" lastFinishedPulling="2025-12-03 12:26:37.350196271 +0000 UTC m=+3987.938438035" observedRunningTime="2025-12-03 12:26:38.527127323 +0000 UTC m=+3989.115369087" watchObservedRunningTime="2025-12-03 12:26:38.531268261 +0000 UTC m=+3989.119510025" Dec 03 12:26:39 crc kubenswrapper[5052]: I1203 12:26:39.512285 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:26:40 crc kubenswrapper[5052]: I1203 12:26:40.949933 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:26:40 crc kubenswrapper[5052]: I1203 12:26:40.950264 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:26:41 crc kubenswrapper[5052]: I1203 12:26:41.124774 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:26:44 crc kubenswrapper[5052]: I1203 12:26:44.548010 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openshift-marketplace/redhat-operators-qhzn9" Dec 03 12:26:44 crc kubenswrapper[5052]: I1203 12:26:44.548693 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-marketplace/redhat-operators-qhzn9" Dec 03 12:26:44 crc kubenswrapper[5052]: I1203 12:26:44.594038 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-marketplace/redhat-operators-qhzn9" Dec 03 12:26:44 crc kubenswrapper[5052]: I1203 12:26:44.641403 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-marketplace/redhat-operators-qhzn9" Dec 03 12:26:44 crc kubenswrapper[5052]: I1203 12:26:44.835666 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openshift-marketplace/redhat-operators-qhzn9"] Dec 03 12:26:45 crc kubenswrapper[5052]: I1203 12:26:45.950755 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:26:45 crc kubenswrapper[5052]: I1203 12:26:45.951131 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:26:46 crc kubenswrapper[5052]: I1203 12:26:46.084847 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:26:46 crc kubenswrapper[5052]: I1203 12:26:46.084945 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:26:46 crc kubenswrapper[5052]: I1203 12:26:46.125921 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:26:46 crc kubenswrapper[5052]: I1203 12:26:46.155831 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:26:46 crc kubenswrapper[5052]: I1203 12:26:46.579045 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openshift-marketplace/redhat-operators-qhzn9" podUID="07a34aba-b4ee-4b5e-92ea-d458af9399de" containerName="registry-server" containerID="cri-o://4085fb5a47809a180ce42109621ec03799dfe9738d29b728f7f0b91a814d42a2" gracePeriod=2 Dec 03 12:26:46 crc kubenswrapper[5052]: I1203 12:26:46.607680 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:26:47 crc kubenswrapper[5052]: I1203 12:26:47.043707 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-metadata-0" podUID="8e052136-a855-4a20-a8c3-38a8bef97dfc" containerName="nova-metadata-metadata" probeResult="failure" output="Get \"http://10.217.0.119:8775/\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Dec 03 12:26:47 crc kubenswrapper[5052]: I1203 12:26:47.043762 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-metadata-0" podUID="8e052136-a855-4a20-a8c3-38a8bef97dfc" containerName="nova-metadata-log" probeResult="failure" output="Get \"http://10.217.0.119:8775/\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Dec 03 12:26:47 crc kubenswrapper[5052]: I1203 12:26:47.077123 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/redhat-operators-qhzn9" Dec 03 12:26:47 crc kubenswrapper[5052]: I1203 12:26:47.157994 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/07a34aba-b4ee-4b5e-92ea-d458af9399de-utilities\") pod \"07a34aba-b4ee-4b5e-92ea-d458af9399de\" (UID: \"07a34aba-b4ee-4b5e-92ea-d458af9399de\") " Dec 03 12:26:47 crc kubenswrapper[5052]: I1203 12:26:47.158247 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2xprp\" (UniqueName: \"kubernetes.io/projected/07a34aba-b4ee-4b5e-92ea-d458af9399de-kube-api-access-2xprp\") pod \"07a34aba-b4ee-4b5e-92ea-d458af9399de\" (UID: \"07a34aba-b4ee-4b5e-92ea-d458af9399de\") " Dec 03 12:26:47 crc kubenswrapper[5052]: I1203 12:26:47.158379 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/07a34aba-b4ee-4b5e-92ea-d458af9399de-catalog-content\") pod \"07a34aba-b4ee-4b5e-92ea-d458af9399de\" (UID: \"07a34aba-b4ee-4b5e-92ea-d458af9399de\") " Dec 03 12:26:47 crc kubenswrapper[5052]: I1203 12:26:47.159452 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07a34aba-b4ee-4b5e-92ea-d458af9399de-utilities" (OuterVolumeSpecName: "utilities") pod "07a34aba-b4ee-4b5e-92ea-d458af9399de" (UID: "07a34aba-b4ee-4b5e-92ea-d458af9399de"). InnerVolumeSpecName "utilities". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:26:47 crc kubenswrapper[5052]: I1203 12:26:47.164414 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/07a34aba-b4ee-4b5e-92ea-d458af9399de-kube-api-access-2xprp" (OuterVolumeSpecName: "kube-api-access-2xprp") pod "07a34aba-b4ee-4b5e-92ea-d458af9399de" (UID: "07a34aba-b4ee-4b5e-92ea-d458af9399de"). InnerVolumeSpecName "kube-api-access-2xprp". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:26:47 crc kubenswrapper[5052]: I1203 12:26:47.177129 5052 reconciler_common.go:293] "Volume detached for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/07a34aba-b4ee-4b5e-92ea-d458af9399de-utilities\") on node \"crc\" DevicePath \"\"" Dec 03 12:26:47 crc kubenswrapper[5052]: I1203 12:26:47.177234 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-2xprp\" (UniqueName: \"kubernetes.io/projected/07a34aba-b4ee-4b5e-92ea-d458af9399de-kube-api-access-2xprp\") on node \"crc\" DevicePath \"\"" Dec 03 12:26:47 crc kubenswrapper[5052]: I1203 12:26:47.178266 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-api-0" podUID="7e415947-bb79-4d7b-80fb-f6d290343e19" containerName="nova-api-log" probeResult="failure" output="Get \"http://10.217.0.120:8774/\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Dec 03 12:26:47 crc kubenswrapper[5052]: I1203 12:26:47.178273 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-api-0" podUID="7e415947-bb79-4d7b-80fb-f6d290343e19" containerName="nova-api-api" probeResult="failure" output="Get \"http://10.217.0.120:8774/\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Dec 03 12:26:47 crc kubenswrapper[5052]: I1203 12:26:47.290535 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07a34aba-b4ee-4b5e-92ea-d458af9399de-catalog-content" (OuterVolumeSpecName: "catalog-content") pod "07a34aba-b4ee-4b5e-92ea-d458af9399de" (UID: "07a34aba-b4ee-4b5e-92ea-d458af9399de"). InnerVolumeSpecName "catalog-content". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:26:47 crc kubenswrapper[5052]: I1203 12:26:47.379981 5052 reconciler_common.go:293] "Volume detached for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/07a34aba-b4ee-4b5e-92ea-d458af9399de-catalog-content\") on node \"crc\" DevicePath \"\"" Dec 03 12:26:47 crc kubenswrapper[5052]: I1203 12:26:47.593186 5052 generic.go:334] "Generic (PLEG): container finished" podID="07a34aba-b4ee-4b5e-92ea-d458af9399de" containerID="4085fb5a47809a180ce42109621ec03799dfe9738d29b728f7f0b91a814d42a2" exitCode=0 Dec 03 12:26:47 crc kubenswrapper[5052]: I1203 12:26:47.593234 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-operators-qhzn9" event={"ID":"07a34aba-b4ee-4b5e-92ea-d458af9399de","Type":"ContainerDied","Data":"4085fb5a47809a180ce42109621ec03799dfe9738d29b728f7f0b91a814d42a2"} Dec 03 12:26:47 crc kubenswrapper[5052]: I1203 12:26:47.593291 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/redhat-operators-qhzn9" Dec 03 12:26:47 crc kubenswrapper[5052]: I1203 12:26:47.593314 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-operators-qhzn9" event={"ID":"07a34aba-b4ee-4b5e-92ea-d458af9399de","Type":"ContainerDied","Data":"28d86f93f18114010b14ff497128eb4b009dde86bf925db45369fc17d7f0ce3b"} Dec 03 12:26:47 crc kubenswrapper[5052]: I1203 12:26:47.593343 5052 scope.go:117] "RemoveContainer" containerID="4085fb5a47809a180ce42109621ec03799dfe9738d29b728f7f0b91a814d42a2" Dec 03 12:26:47 crc kubenswrapper[5052]: I1203 12:26:47.638907 5052 scope.go:117] "RemoveContainer" containerID="104040fbb1ee554dc74340bedf98f59d521b44f2da342566a71b536c53c4bc01" Dec 03 12:26:47 crc kubenswrapper[5052]: I1203 12:26:47.641226 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openshift-marketplace/redhat-operators-qhzn9"] Dec 03 12:26:47 crc kubenswrapper[5052]: I1203 12:26:47.671329 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openshift-marketplace/redhat-operators-qhzn9"] Dec 03 12:26:47 crc kubenswrapper[5052]: I1203 12:26:47.679813 5052 scope.go:117] "RemoveContainer" containerID="ede4c3adf898870cfa7b399a0ddcefb756ac3e870f4f18b8f0483222573548e7" Dec 03 12:26:47 crc kubenswrapper[5052]: I1203 12:26:47.691504 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="07a34aba-b4ee-4b5e-92ea-d458af9399de" path="/var/lib/kubelet/pods/07a34aba-b4ee-4b5e-92ea-d458af9399de/volumes" Dec 03 12:26:47 crc kubenswrapper[5052]: I1203 12:26:47.711813 5052 scope.go:117] "RemoveContainer" containerID="4085fb5a47809a180ce42109621ec03799dfe9738d29b728f7f0b91a814d42a2" Dec 03 12:26:47 crc kubenswrapper[5052]: E1203 12:26:47.712287 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4085fb5a47809a180ce42109621ec03799dfe9738d29b728f7f0b91a814d42a2\": container with ID starting with 4085fb5a47809a180ce42109621ec03799dfe9738d29b728f7f0b91a814d42a2 not found: ID does not exist" containerID="4085fb5a47809a180ce42109621ec03799dfe9738d29b728f7f0b91a814d42a2" Dec 03 12:26:47 crc kubenswrapper[5052]: I1203 12:26:47.712416 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4085fb5a47809a180ce42109621ec03799dfe9738d29b728f7f0b91a814d42a2"} err="failed to get container status \"4085fb5a47809a180ce42109621ec03799dfe9738d29b728f7f0b91a814d42a2\": rpc error: code = NotFound desc = could not find container \"4085fb5a47809a180ce42109621ec03799dfe9738d29b728f7f0b91a814d42a2\": container with ID starting with 4085fb5a47809a180ce42109621ec03799dfe9738d29b728f7f0b91a814d42a2 not found: ID does not exist" Dec 03 12:26:47 crc kubenswrapper[5052]: I1203 12:26:47.712541 5052 scope.go:117] "RemoveContainer" containerID="104040fbb1ee554dc74340bedf98f59d521b44f2da342566a71b536c53c4bc01" Dec 03 12:26:47 crc kubenswrapper[5052]: E1203 12:26:47.712840 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"104040fbb1ee554dc74340bedf98f59d521b44f2da342566a71b536c53c4bc01\": container with ID starting with 104040fbb1ee554dc74340bedf98f59d521b44f2da342566a71b536c53c4bc01 not found: ID does not exist" containerID="104040fbb1ee554dc74340bedf98f59d521b44f2da342566a71b536c53c4bc01" Dec 03 12:26:47 crc kubenswrapper[5052]: I1203 12:26:47.712957 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"104040fbb1ee554dc74340bedf98f59d521b44f2da342566a71b536c53c4bc01"} err="failed to get container status \"104040fbb1ee554dc74340bedf98f59d521b44f2da342566a71b536c53c4bc01\": rpc error: code = NotFound desc = could not find container \"104040fbb1ee554dc74340bedf98f59d521b44f2da342566a71b536c53c4bc01\": container with ID starting with 104040fbb1ee554dc74340bedf98f59d521b44f2da342566a71b536c53c4bc01 not found: ID does not exist" Dec 03 12:26:47 crc kubenswrapper[5052]: I1203 12:26:47.713034 5052 scope.go:117] "RemoveContainer" containerID="ede4c3adf898870cfa7b399a0ddcefb756ac3e870f4f18b8f0483222573548e7" Dec 03 12:26:47 crc kubenswrapper[5052]: E1203 12:26:47.713370 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ede4c3adf898870cfa7b399a0ddcefb756ac3e870f4f18b8f0483222573548e7\": container with ID starting with ede4c3adf898870cfa7b399a0ddcefb756ac3e870f4f18b8f0483222573548e7 not found: ID does not exist" containerID="ede4c3adf898870cfa7b399a0ddcefb756ac3e870f4f18b8f0483222573548e7" Dec 03 12:26:47 crc kubenswrapper[5052]: I1203 12:26:47.713412 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ede4c3adf898870cfa7b399a0ddcefb756ac3e870f4f18b8f0483222573548e7"} err="failed to get container status \"ede4c3adf898870cfa7b399a0ddcefb756ac3e870f4f18b8f0483222573548e7\": rpc error: code = NotFound desc = could not find container \"ede4c3adf898870cfa7b399a0ddcefb756ac3e870f4f18b8f0483222573548e7\": container with ID starting with ede4c3adf898870cfa7b399a0ddcefb756ac3e870f4f18b8f0483222573548e7 not found: ID does not exist" Dec 03 12:26:55 crc kubenswrapper[5052]: I1203 12:26:55.952200 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:26:55 crc kubenswrapper[5052]: I1203 12:26:55.954311 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:26:55 crc kubenswrapper[5052]: I1203 12:26:55.955670 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:26:55 crc kubenswrapper[5052]: I1203 12:26:55.955723 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:26:56 crc kubenswrapper[5052]: I1203 12:26:56.090638 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:26:56 crc kubenswrapper[5052]: I1203 12:26:56.091154 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:26:56 crc kubenswrapper[5052]: I1203 12:26:56.091241 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:26:56 crc kubenswrapper[5052]: I1203 12:26:56.093984 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:26:56 crc kubenswrapper[5052]: I1203 12:26:56.685667 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:26:56 crc kubenswrapper[5052]: I1203 12:26:56.691098 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:26:58 crc kubenswrapper[5052]: I1203 12:26:58.877049 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:27:08 crc kubenswrapper[5052]: I1203 12:27:08.673613 5052 scope.go:117] "RemoveContainer" containerID="b80b6e8c4ad454ae227b95729ae24f6a9c106e69b758ed3c06b5fe2924584a0c" Dec 03 12:27:08 crc kubenswrapper[5052]: I1203 12:27:08.703640 5052 scope.go:117] "RemoveContainer" containerID="d8891df0b213dbaf1fa63a8450de0cdab0ef7ec799eb7a4b97714ab88cffe2ab" Dec 03 12:27:08 crc kubenswrapper[5052]: I1203 12:27:08.736041 5052 scope.go:117] "RemoveContainer" containerID="2f07afc7faedc91462a894d76d86fe113f7f5b41d2ce70d3b0e31378e31a2ea3" Dec 03 12:27:08 crc kubenswrapper[5052]: I1203 12:27:08.800666 5052 scope.go:117] "RemoveContainer" containerID="fcd7a92adc3ca1e7e96f3a9c20674d72f6b8c11f7f6f67a8af2cf51365e8fcb1" Dec 03 12:27:08 crc kubenswrapper[5052]: I1203 12:27:08.860473 5052 scope.go:117] "RemoveContainer" containerID="065aebd7d9dcbe855a40abb9f1fcec166d09f5a018bf0567b9d6ef02a6577350" Dec 03 12:27:08 crc kubenswrapper[5052]: I1203 12:27:08.907359 5052 scope.go:117] "RemoveContainer" containerID="d21edbe94c25709bf834fc9a057d971ffaeed3e9a181d6a8bd2dc198291496be" Dec 03 12:27:08 crc kubenswrapper[5052]: I1203 12:27:08.958948 5052 scope.go:117] "RemoveContainer" containerID="e47bb47cae9cbe2f35ecd613fb6be4fc2447cc9124af569f2f18e065e8af2171" Dec 03 12:27:08 crc kubenswrapper[5052]: I1203 12:27:08.994045 5052 scope.go:117] "RemoveContainer" containerID="e3b8749f931ef525570e1e11172cba2d27c70a94440e0ea98f51fe1e60b90b6d" Dec 03 12:27:09 crc kubenswrapper[5052]: I1203 12:27:09.018209 5052 scope.go:117] "RemoveContainer" containerID="f82b984cb5939c510f6703add68625e1e534cdf59488ddc53cb91e355ac65330" Dec 03 12:27:09 crc kubenswrapper[5052]: I1203 12:27:09.041123 5052 scope.go:117] "RemoveContainer" containerID="0c61528c202083e822a6d60926f5531661737d72572cc3938c0bb3db2f2fcd35" Dec 03 12:27:09 crc kubenswrapper[5052]: I1203 12:27:09.062757 5052 scope.go:117] "RemoveContainer" containerID="4d363d9441eef4d9d65d1bcd8bc7f04f2fe0d45b88e46676a88a9cb0748fb83e" Dec 03 12:27:09 crc kubenswrapper[5052]: I1203 12:27:09.092781 5052 scope.go:117] "RemoveContainer" containerID="4acfccfaa6f110a7544ac7a681df22aceab18e0f00a4e818bfb5f8c9536032f3" Dec 03 12:27:09 crc kubenswrapper[5052]: I1203 12:27:09.135766 5052 scope.go:117] "RemoveContainer" containerID="739b5f17c284ab8823d07f252de61cfc9a034a60b12ad3eafd6d9d840e4cf76d" Dec 03 12:27:10 crc kubenswrapper[5052]: I1203 12:27:10.704051 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/rabbitmq-server-0"] Dec 03 12:27:10 crc kubenswrapper[5052]: I1203 12:27:10.829772 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/barbican-worker-64d9f4d555-4v5kn"] Dec 03 12:27:10 crc kubenswrapper[5052]: E1203 12:27:10.830252 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="07a34aba-b4ee-4b5e-92ea-d458af9399de" containerName="registry-server" Dec 03 12:27:10 crc kubenswrapper[5052]: I1203 12:27:10.830275 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="07a34aba-b4ee-4b5e-92ea-d458af9399de" containerName="registry-server" Dec 03 12:27:10 crc kubenswrapper[5052]: E1203 12:27:10.830310 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="07a34aba-b4ee-4b5e-92ea-d458af9399de" containerName="extract-content" Dec 03 12:27:10 crc kubenswrapper[5052]: I1203 12:27:10.830319 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="07a34aba-b4ee-4b5e-92ea-d458af9399de" containerName="extract-content" Dec 03 12:27:10 crc kubenswrapper[5052]: E1203 12:27:10.830347 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="07a34aba-b4ee-4b5e-92ea-d458af9399de" containerName="extract-utilities" Dec 03 12:27:10 crc kubenswrapper[5052]: I1203 12:27:10.830357 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="07a34aba-b4ee-4b5e-92ea-d458af9399de" containerName="extract-utilities" Dec 03 12:27:10 crc kubenswrapper[5052]: I1203 12:27:10.830605 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="07a34aba-b4ee-4b5e-92ea-d458af9399de" containerName="registry-server" Dec 03 12:27:10 crc kubenswrapper[5052]: I1203 12:27:10.833277 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-worker-64d9f4d555-4v5kn" Dec 03 12:27:10 crc kubenswrapper[5052]: I1203 12:27:10.844785 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/6228bbc6-4c33-4ad8-ad26-440a8e6b441c-logs\") pod \"barbican-worker-64d9f4d555-4v5kn\" (UID: \"6228bbc6-4c33-4ad8-ad26-440a8e6b441c\") " pod="openstack-kuttl-tests/barbican-worker-64d9f4d555-4v5kn" Dec 03 12:27:10 crc kubenswrapper[5052]: I1203 12:27:10.844847 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/6228bbc6-4c33-4ad8-ad26-440a8e6b441c-config-data-custom\") pod \"barbican-worker-64d9f4d555-4v5kn\" (UID: \"6228bbc6-4c33-4ad8-ad26-440a8e6b441c\") " pod="openstack-kuttl-tests/barbican-worker-64d9f4d555-4v5kn" Dec 03 12:27:10 crc kubenswrapper[5052]: I1203 12:27:10.844905 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6228bbc6-4c33-4ad8-ad26-440a8e6b441c-config-data\") pod \"barbican-worker-64d9f4d555-4v5kn\" (UID: \"6228bbc6-4c33-4ad8-ad26-440a8e6b441c\") " pod="openstack-kuttl-tests/barbican-worker-64d9f4d555-4v5kn" Dec 03 12:27:10 crc kubenswrapper[5052]: I1203 12:27:10.844974 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z9bxb\" (UniqueName: \"kubernetes.io/projected/6228bbc6-4c33-4ad8-ad26-440a8e6b441c-kube-api-access-z9bxb\") pod \"barbican-worker-64d9f4d555-4v5kn\" (UID: \"6228bbc6-4c33-4ad8-ad26-440a8e6b441c\") " pod="openstack-kuttl-tests/barbican-worker-64d9f4d555-4v5kn" Dec 03 12:27:10 crc kubenswrapper[5052]: I1203 12:27:10.845006 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6228bbc6-4c33-4ad8-ad26-440a8e6b441c-combined-ca-bundle\") pod \"barbican-worker-64d9f4d555-4v5kn\" (UID: \"6228bbc6-4c33-4ad8-ad26-440a8e6b441c\") " pod="openstack-kuttl-tests/barbican-worker-64d9f4d555-4v5kn" Dec 03 12:27:10 crc kubenswrapper[5052]: I1203 12:27:10.876267 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/barbican-keystone-listener-6df6465ddd-km2kt"] Dec 03 12:27:10 crc kubenswrapper[5052]: I1203 12:27:10.886580 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-keystone-listener-6df6465ddd-km2kt" Dec 03 12:27:10 crc kubenswrapper[5052]: I1203 12:27:10.920886 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-worker-64d9f4d555-4v5kn"] Dec 03 12:27:10 crc kubenswrapper[5052]: I1203 12:27:10.951447 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-keystone-listener-6df6465ddd-km2kt"] Dec 03 12:27:10 crc kubenswrapper[5052]: I1203 12:27:10.953197 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/7f4dfec9-0aa8-437c-9b8d-771b30a93af9-config-data-custom\") pod \"barbican-keystone-listener-6df6465ddd-km2kt\" (UID: \"7f4dfec9-0aa8-437c-9b8d-771b30a93af9\") " pod="openstack-kuttl-tests/barbican-keystone-listener-6df6465ddd-km2kt" Dec 03 12:27:10 crc kubenswrapper[5052]: I1203 12:27:10.953329 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/6228bbc6-4c33-4ad8-ad26-440a8e6b441c-logs\") pod \"barbican-worker-64d9f4d555-4v5kn\" (UID: \"6228bbc6-4c33-4ad8-ad26-440a8e6b441c\") " pod="openstack-kuttl-tests/barbican-worker-64d9f4d555-4v5kn" Dec 03 12:27:10 crc kubenswrapper[5052]: I1203 12:27:10.953369 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/7f4dfec9-0aa8-437c-9b8d-771b30a93af9-config-data\") pod \"barbican-keystone-listener-6df6465ddd-km2kt\" (UID: \"7f4dfec9-0aa8-437c-9b8d-771b30a93af9\") " pod="openstack-kuttl-tests/barbican-keystone-listener-6df6465ddd-km2kt" Dec 03 12:27:10 crc kubenswrapper[5052]: I1203 12:27:10.953402 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/6228bbc6-4c33-4ad8-ad26-440a8e6b441c-config-data-custom\") pod \"barbican-worker-64d9f4d555-4v5kn\" (UID: \"6228bbc6-4c33-4ad8-ad26-440a8e6b441c\") " pod="openstack-kuttl-tests/barbican-worker-64d9f4d555-4v5kn" Dec 03 12:27:10 crc kubenswrapper[5052]: I1203 12:27:10.953430 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/7f4dfec9-0aa8-437c-9b8d-771b30a93af9-combined-ca-bundle\") pod \"barbican-keystone-listener-6df6465ddd-km2kt\" (UID: \"7f4dfec9-0aa8-437c-9b8d-771b30a93af9\") " pod="openstack-kuttl-tests/barbican-keystone-listener-6df6465ddd-km2kt" Dec 03 12:27:10 crc kubenswrapper[5052]: I1203 12:27:10.953462 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/7f4dfec9-0aa8-437c-9b8d-771b30a93af9-logs\") pod \"barbican-keystone-listener-6df6465ddd-km2kt\" (UID: \"7f4dfec9-0aa8-437c-9b8d-771b30a93af9\") " pod="openstack-kuttl-tests/barbican-keystone-listener-6df6465ddd-km2kt" Dec 03 12:27:10 crc kubenswrapper[5052]: I1203 12:27:10.953496 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6228bbc6-4c33-4ad8-ad26-440a8e6b441c-config-data\") pod \"barbican-worker-64d9f4d555-4v5kn\" (UID: \"6228bbc6-4c33-4ad8-ad26-440a8e6b441c\") " pod="openstack-kuttl-tests/barbican-worker-64d9f4d555-4v5kn" Dec 03 12:27:10 crc kubenswrapper[5052]: I1203 12:27:10.953608 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v76qr\" (UniqueName: \"kubernetes.io/projected/7f4dfec9-0aa8-437c-9b8d-771b30a93af9-kube-api-access-v76qr\") pod \"barbican-keystone-listener-6df6465ddd-km2kt\" (UID: \"7f4dfec9-0aa8-437c-9b8d-771b30a93af9\") " pod="openstack-kuttl-tests/barbican-keystone-listener-6df6465ddd-km2kt" Dec 03 12:27:10 crc kubenswrapper[5052]: I1203 12:27:10.953657 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-z9bxb\" (UniqueName: \"kubernetes.io/projected/6228bbc6-4c33-4ad8-ad26-440a8e6b441c-kube-api-access-z9bxb\") pod \"barbican-worker-64d9f4d555-4v5kn\" (UID: \"6228bbc6-4c33-4ad8-ad26-440a8e6b441c\") " pod="openstack-kuttl-tests/barbican-worker-64d9f4d555-4v5kn" Dec 03 12:27:10 crc kubenswrapper[5052]: I1203 12:27:10.953693 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6228bbc6-4c33-4ad8-ad26-440a8e6b441c-combined-ca-bundle\") pod \"barbican-worker-64d9f4d555-4v5kn\" (UID: \"6228bbc6-4c33-4ad8-ad26-440a8e6b441c\") " pod="openstack-kuttl-tests/barbican-worker-64d9f4d555-4v5kn" Dec 03 12:27:10 crc kubenswrapper[5052]: I1203 12:27:10.955811 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/6228bbc6-4c33-4ad8-ad26-440a8e6b441c-logs\") pod \"barbican-worker-64d9f4d555-4v5kn\" (UID: \"6228bbc6-4c33-4ad8-ad26-440a8e6b441c\") " pod="openstack-kuttl-tests/barbican-worker-64d9f4d555-4v5kn" Dec 03 12:27:10 crc kubenswrapper[5052]: I1203 12:27:10.967778 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/6228bbc6-4c33-4ad8-ad26-440a8e6b441c-config-data-custom\") pod \"barbican-worker-64d9f4d555-4v5kn\" (UID: \"6228bbc6-4c33-4ad8-ad26-440a8e6b441c\") " pod="openstack-kuttl-tests/barbican-worker-64d9f4d555-4v5kn" Dec 03 12:27:10 crc kubenswrapper[5052]: I1203 12:27:10.974121 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6228bbc6-4c33-4ad8-ad26-440a8e6b441c-config-data\") pod \"barbican-worker-64d9f4d555-4v5kn\" (UID: \"6228bbc6-4c33-4ad8-ad26-440a8e6b441c\") " pod="openstack-kuttl-tests/barbican-worker-64d9f4d555-4v5kn" Dec 03 12:27:10 crc kubenswrapper[5052]: I1203 12:27:10.986303 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6228bbc6-4c33-4ad8-ad26-440a8e6b441c-combined-ca-bundle\") pod \"barbican-worker-64d9f4d555-4v5kn\" (UID: \"6228bbc6-4c33-4ad8-ad26-440a8e6b441c\") " pod="openstack-kuttl-tests/barbican-worker-64d9f4d555-4v5kn" Dec 03 12:27:10 crc kubenswrapper[5052]: I1203 12:27:10.997122 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-z9bxb\" (UniqueName: \"kubernetes.io/projected/6228bbc6-4c33-4ad8-ad26-440a8e6b441c-kube-api-access-z9bxb\") pod \"barbican-worker-64d9f4d555-4v5kn\" (UID: \"6228bbc6-4c33-4ad8-ad26-440a8e6b441c\") " pod="openstack-kuttl-tests/barbican-worker-64d9f4d555-4v5kn" Dec 03 12:27:11 crc kubenswrapper[5052]: I1203 12:27:11.040658 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 12:27:11 crc kubenswrapper[5052]: I1203 12:27:11.040973 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/cinder-scheduler-0" podUID="a823b6ba-b2af-4deb-88d2-0902fd6e3780" containerName="cinder-scheduler" containerID="cri-o://53d7dc20a3dd7792e212463451921763c0652bd6c02e21fa9b1e85765b8915ca" gracePeriod=30 Dec 03 12:27:11 crc kubenswrapper[5052]: I1203 12:27:11.041493 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/cinder-scheduler-0" podUID="a823b6ba-b2af-4deb-88d2-0902fd6e3780" containerName="probe" containerID="cri-o://6cd2d71e42ed2933a0efd0de7d0e300b0da269f757f4f18802db5fd7de523192" gracePeriod=30 Dec 03 12:27:11 crc kubenswrapper[5052]: I1203 12:27:11.056674 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/7f4dfec9-0aa8-437c-9b8d-771b30a93af9-config-data\") pod \"barbican-keystone-listener-6df6465ddd-km2kt\" (UID: \"7f4dfec9-0aa8-437c-9b8d-771b30a93af9\") " pod="openstack-kuttl-tests/barbican-keystone-listener-6df6465ddd-km2kt" Dec 03 12:27:11 crc kubenswrapper[5052]: I1203 12:27:11.056748 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/7f4dfec9-0aa8-437c-9b8d-771b30a93af9-combined-ca-bundle\") pod \"barbican-keystone-listener-6df6465ddd-km2kt\" (UID: \"7f4dfec9-0aa8-437c-9b8d-771b30a93af9\") " pod="openstack-kuttl-tests/barbican-keystone-listener-6df6465ddd-km2kt" Dec 03 12:27:11 crc kubenswrapper[5052]: I1203 12:27:11.056781 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/7f4dfec9-0aa8-437c-9b8d-771b30a93af9-logs\") pod \"barbican-keystone-listener-6df6465ddd-km2kt\" (UID: \"7f4dfec9-0aa8-437c-9b8d-771b30a93af9\") " pod="openstack-kuttl-tests/barbican-keystone-listener-6df6465ddd-km2kt" Dec 03 12:27:11 crc kubenswrapper[5052]: I1203 12:27:11.056831 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-v76qr\" (UniqueName: \"kubernetes.io/projected/7f4dfec9-0aa8-437c-9b8d-771b30a93af9-kube-api-access-v76qr\") pod \"barbican-keystone-listener-6df6465ddd-km2kt\" (UID: \"7f4dfec9-0aa8-437c-9b8d-771b30a93af9\") " pod="openstack-kuttl-tests/barbican-keystone-listener-6df6465ddd-km2kt" Dec 03 12:27:11 crc kubenswrapper[5052]: I1203 12:27:11.056908 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/7f4dfec9-0aa8-437c-9b8d-771b30a93af9-config-data-custom\") pod \"barbican-keystone-listener-6df6465ddd-km2kt\" (UID: \"7f4dfec9-0aa8-437c-9b8d-771b30a93af9\") " pod="openstack-kuttl-tests/barbican-keystone-listener-6df6465ddd-km2kt" Dec 03 12:27:11 crc kubenswrapper[5052]: I1203 12:27:11.059722 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/7f4dfec9-0aa8-437c-9b8d-771b30a93af9-logs\") pod \"barbican-keystone-listener-6df6465ddd-km2kt\" (UID: \"7f4dfec9-0aa8-437c-9b8d-771b30a93af9\") " pod="openstack-kuttl-tests/barbican-keystone-listener-6df6465ddd-km2kt" Dec 03 12:27:11 crc kubenswrapper[5052]: I1203 12:27:11.070342 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/7f4dfec9-0aa8-437c-9b8d-771b30a93af9-combined-ca-bundle\") pod \"barbican-keystone-listener-6df6465ddd-km2kt\" (UID: \"7f4dfec9-0aa8-437c-9b8d-771b30a93af9\") " pod="openstack-kuttl-tests/barbican-keystone-listener-6df6465ddd-km2kt" Dec 03 12:27:11 crc kubenswrapper[5052]: I1203 12:27:11.073259 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/7f4dfec9-0aa8-437c-9b8d-771b30a93af9-config-data-custom\") pod \"barbican-keystone-listener-6df6465ddd-km2kt\" (UID: \"7f4dfec9-0aa8-437c-9b8d-771b30a93af9\") " pod="openstack-kuttl-tests/barbican-keystone-listener-6df6465ddd-km2kt" Dec 03 12:27:11 crc kubenswrapper[5052]: I1203 12:27:11.080735 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/7f4dfec9-0aa8-437c-9b8d-771b30a93af9-config-data\") pod \"barbican-keystone-listener-6df6465ddd-km2kt\" (UID: \"7f4dfec9-0aa8-437c-9b8d-771b30a93af9\") " pod="openstack-kuttl-tests/barbican-keystone-listener-6df6465ddd-km2kt" Dec 03 12:27:11 crc kubenswrapper[5052]: I1203 12:27:11.085595 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/barbican-api-78665465bd-9q8ff"] Dec 03 12:27:11 crc kubenswrapper[5052]: I1203 12:27:11.087616 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-api-78665465bd-9q8ff" Dec 03 12:27:11 crc kubenswrapper[5052]: I1203 12:27:11.116095 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-v76qr\" (UniqueName: \"kubernetes.io/projected/7f4dfec9-0aa8-437c-9b8d-771b30a93af9-kube-api-access-v76qr\") pod \"barbican-keystone-listener-6df6465ddd-km2kt\" (UID: \"7f4dfec9-0aa8-437c-9b8d-771b30a93af9\") " pod="openstack-kuttl-tests/barbican-keystone-listener-6df6465ddd-km2kt" Dec 03 12:27:11 crc kubenswrapper[5052]: I1203 12:27:11.148799 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-api-78665465bd-9q8ff"] Dec 03 12:27:11 crc kubenswrapper[5052]: I1203 12:27:11.177386 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-worker-64d9f4d555-4v5kn" Dec 03 12:27:11 crc kubenswrapper[5052]: I1203 12:27:11.218498 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-keystone-listener-6df6465ddd-km2kt" Dec 03 12:27:11 crc kubenswrapper[5052]: I1203 12:27:11.257237 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/rabbitmq-cell1-server-0"] Dec 03 12:27:11 crc kubenswrapper[5052]: I1203 12:27:11.276209 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2825bd7d-50f6-4d2d-827a-e0d307d1b6de-combined-ca-bundle\") pod \"barbican-api-78665465bd-9q8ff\" (UID: \"2825bd7d-50f6-4d2d-827a-e0d307d1b6de\") " pod="openstack-kuttl-tests/barbican-api-78665465bd-9q8ff" Dec 03 12:27:11 crc kubenswrapper[5052]: I1203 12:27:11.278198 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8gplk\" (UniqueName: \"kubernetes.io/projected/2825bd7d-50f6-4d2d-827a-e0d307d1b6de-kube-api-access-8gplk\") pod \"barbican-api-78665465bd-9q8ff\" (UID: \"2825bd7d-50f6-4d2d-827a-e0d307d1b6de\") " pod="openstack-kuttl-tests/barbican-api-78665465bd-9q8ff" Dec 03 12:27:11 crc kubenswrapper[5052]: I1203 12:27:11.278411 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/2825bd7d-50f6-4d2d-827a-e0d307d1b6de-config-data\") pod \"barbican-api-78665465bd-9q8ff\" (UID: \"2825bd7d-50f6-4d2d-827a-e0d307d1b6de\") " pod="openstack-kuttl-tests/barbican-api-78665465bd-9q8ff" Dec 03 12:27:11 crc kubenswrapper[5052]: I1203 12:27:11.278433 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/2825bd7d-50f6-4d2d-827a-e0d307d1b6de-logs\") pod \"barbican-api-78665465bd-9q8ff\" (UID: \"2825bd7d-50f6-4d2d-827a-e0d307d1b6de\") " pod="openstack-kuttl-tests/barbican-api-78665465bd-9q8ff" Dec 03 12:27:11 crc kubenswrapper[5052]: I1203 12:27:11.278460 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/2825bd7d-50f6-4d2d-827a-e0d307d1b6de-config-data-custom\") pod \"barbican-api-78665465bd-9q8ff\" (UID: \"2825bd7d-50f6-4d2d-827a-e0d307d1b6de\") " pod="openstack-kuttl-tests/barbican-api-78665465bd-9q8ff" Dec 03 12:27:11 crc kubenswrapper[5052]: I1203 12:27:11.319575 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/keystone-66778fc6-56rjc"] Dec 03 12:27:11 crc kubenswrapper[5052]: I1203 12:27:11.320809 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-66778fc6-56rjc" Dec 03 12:27:11 crc kubenswrapper[5052]: I1203 12:27:11.383392 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/aa94b30e-0028-45a0-88aa-b5db5fc2f8fd-config-data\") pod \"keystone-66778fc6-56rjc\" (UID: \"aa94b30e-0028-45a0-88aa-b5db5fc2f8fd\") " pod="openstack-kuttl-tests/keystone-66778fc6-56rjc" Dec 03 12:27:11 crc kubenswrapper[5052]: I1203 12:27:11.383439 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/aa94b30e-0028-45a0-88aa-b5db5fc2f8fd-scripts\") pod \"keystone-66778fc6-56rjc\" (UID: \"aa94b30e-0028-45a0-88aa-b5db5fc2f8fd\") " pod="openstack-kuttl-tests/keystone-66778fc6-56rjc" Dec 03 12:27:11 crc kubenswrapper[5052]: I1203 12:27:11.383475 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/aa94b30e-0028-45a0-88aa-b5db5fc2f8fd-fernet-keys\") pod \"keystone-66778fc6-56rjc\" (UID: \"aa94b30e-0028-45a0-88aa-b5db5fc2f8fd\") " pod="openstack-kuttl-tests/keystone-66778fc6-56rjc" Dec 03 12:27:11 crc kubenswrapper[5052]: I1203 12:27:11.383575 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/2825bd7d-50f6-4d2d-827a-e0d307d1b6de-config-data\") pod \"barbican-api-78665465bd-9q8ff\" (UID: \"2825bd7d-50f6-4d2d-827a-e0d307d1b6de\") " pod="openstack-kuttl-tests/barbican-api-78665465bd-9q8ff" Dec 03 12:27:11 crc kubenswrapper[5052]: I1203 12:27:11.383599 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/2825bd7d-50f6-4d2d-827a-e0d307d1b6de-logs\") pod \"barbican-api-78665465bd-9q8ff\" (UID: \"2825bd7d-50f6-4d2d-827a-e0d307d1b6de\") " pod="openstack-kuttl-tests/barbican-api-78665465bd-9q8ff" Dec 03 12:27:11 crc kubenswrapper[5052]: I1203 12:27:11.383640 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/aa94b30e-0028-45a0-88aa-b5db5fc2f8fd-combined-ca-bundle\") pod \"keystone-66778fc6-56rjc\" (UID: \"aa94b30e-0028-45a0-88aa-b5db5fc2f8fd\") " pod="openstack-kuttl-tests/keystone-66778fc6-56rjc" Dec 03 12:27:11 crc kubenswrapper[5052]: I1203 12:27:11.383665 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/2825bd7d-50f6-4d2d-827a-e0d307d1b6de-config-data-custom\") pod \"barbican-api-78665465bd-9q8ff\" (UID: \"2825bd7d-50f6-4d2d-827a-e0d307d1b6de\") " pod="openstack-kuttl-tests/barbican-api-78665465bd-9q8ff" Dec 03 12:27:11 crc kubenswrapper[5052]: I1203 12:27:11.383727 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2825bd7d-50f6-4d2d-827a-e0d307d1b6de-combined-ca-bundle\") pod \"barbican-api-78665465bd-9q8ff\" (UID: \"2825bd7d-50f6-4d2d-827a-e0d307d1b6de\") " pod="openstack-kuttl-tests/barbican-api-78665465bd-9q8ff" Dec 03 12:27:11 crc kubenswrapper[5052]: I1203 12:27:11.383752 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-8gplk\" (UniqueName: \"kubernetes.io/projected/2825bd7d-50f6-4d2d-827a-e0d307d1b6de-kube-api-access-8gplk\") pod \"barbican-api-78665465bd-9q8ff\" (UID: \"2825bd7d-50f6-4d2d-827a-e0d307d1b6de\") " pod="openstack-kuttl-tests/barbican-api-78665465bd-9q8ff" Dec 03 12:27:11 crc kubenswrapper[5052]: I1203 12:27:11.383779 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/aa94b30e-0028-45a0-88aa-b5db5fc2f8fd-credential-keys\") pod \"keystone-66778fc6-56rjc\" (UID: \"aa94b30e-0028-45a0-88aa-b5db5fc2f8fd\") " pod="openstack-kuttl-tests/keystone-66778fc6-56rjc" Dec 03 12:27:11 crc kubenswrapper[5052]: I1203 12:27:11.383805 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fmmp2\" (UniqueName: \"kubernetes.io/projected/aa94b30e-0028-45a0-88aa-b5db5fc2f8fd-kube-api-access-fmmp2\") pod \"keystone-66778fc6-56rjc\" (UID: \"aa94b30e-0028-45a0-88aa-b5db5fc2f8fd\") " pod="openstack-kuttl-tests/keystone-66778fc6-56rjc" Dec 03 12:27:11 crc kubenswrapper[5052]: I1203 12:27:11.390011 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/2825bd7d-50f6-4d2d-827a-e0d307d1b6de-config-data\") pod \"barbican-api-78665465bd-9q8ff\" (UID: \"2825bd7d-50f6-4d2d-827a-e0d307d1b6de\") " pod="openstack-kuttl-tests/barbican-api-78665465bd-9q8ff" Dec 03 12:27:11 crc kubenswrapper[5052]: I1203 12:27:11.390312 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/2825bd7d-50f6-4d2d-827a-e0d307d1b6de-logs\") pod \"barbican-api-78665465bd-9q8ff\" (UID: \"2825bd7d-50f6-4d2d-827a-e0d307d1b6de\") " pod="openstack-kuttl-tests/barbican-api-78665465bd-9q8ff" Dec 03 12:27:11 crc kubenswrapper[5052]: I1203 12:27:11.390809 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2825bd7d-50f6-4d2d-827a-e0d307d1b6de-combined-ca-bundle\") pod \"barbican-api-78665465bd-9q8ff\" (UID: \"2825bd7d-50f6-4d2d-827a-e0d307d1b6de\") " pod="openstack-kuttl-tests/barbican-api-78665465bd-9q8ff" Dec 03 12:27:11 crc kubenswrapper[5052]: I1203 12:27:11.391294 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/2825bd7d-50f6-4d2d-827a-e0d307d1b6de-config-data-custom\") pod \"barbican-api-78665465bd-9q8ff\" (UID: \"2825bd7d-50f6-4d2d-827a-e0d307d1b6de\") " pod="openstack-kuttl-tests/barbican-api-78665465bd-9q8ff" Dec 03 12:27:11 crc kubenswrapper[5052]: I1203 12:27:11.433407 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-66778fc6-56rjc"] Dec 03 12:27:11 crc kubenswrapper[5052]: I1203 12:27:11.452580 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-8gplk\" (UniqueName: \"kubernetes.io/projected/2825bd7d-50f6-4d2d-827a-e0d307d1b6de-kube-api-access-8gplk\") pod \"barbican-api-78665465bd-9q8ff\" (UID: \"2825bd7d-50f6-4d2d-827a-e0d307d1b6de\") " pod="openstack-kuttl-tests/barbican-api-78665465bd-9q8ff" Dec 03 12:27:11 crc kubenswrapper[5052]: I1203 12:27:11.511399 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/aa94b30e-0028-45a0-88aa-b5db5fc2f8fd-combined-ca-bundle\") pod \"keystone-66778fc6-56rjc\" (UID: \"aa94b30e-0028-45a0-88aa-b5db5fc2f8fd\") " pod="openstack-kuttl-tests/keystone-66778fc6-56rjc" Dec 03 12:27:11 crc kubenswrapper[5052]: I1203 12:27:11.511546 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/aa94b30e-0028-45a0-88aa-b5db5fc2f8fd-credential-keys\") pod \"keystone-66778fc6-56rjc\" (UID: \"aa94b30e-0028-45a0-88aa-b5db5fc2f8fd\") " pod="openstack-kuttl-tests/keystone-66778fc6-56rjc" Dec 03 12:27:11 crc kubenswrapper[5052]: I1203 12:27:11.511577 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-fmmp2\" (UniqueName: \"kubernetes.io/projected/aa94b30e-0028-45a0-88aa-b5db5fc2f8fd-kube-api-access-fmmp2\") pod \"keystone-66778fc6-56rjc\" (UID: \"aa94b30e-0028-45a0-88aa-b5db5fc2f8fd\") " pod="openstack-kuttl-tests/keystone-66778fc6-56rjc" Dec 03 12:27:11 crc kubenswrapper[5052]: I1203 12:27:11.511714 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/aa94b30e-0028-45a0-88aa-b5db5fc2f8fd-config-data\") pod \"keystone-66778fc6-56rjc\" (UID: \"aa94b30e-0028-45a0-88aa-b5db5fc2f8fd\") " pod="openstack-kuttl-tests/keystone-66778fc6-56rjc" Dec 03 12:27:11 crc kubenswrapper[5052]: I1203 12:27:11.511743 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/aa94b30e-0028-45a0-88aa-b5db5fc2f8fd-scripts\") pod \"keystone-66778fc6-56rjc\" (UID: \"aa94b30e-0028-45a0-88aa-b5db5fc2f8fd\") " pod="openstack-kuttl-tests/keystone-66778fc6-56rjc" Dec 03 12:27:11 crc kubenswrapper[5052]: I1203 12:27:11.511795 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/aa94b30e-0028-45a0-88aa-b5db5fc2f8fd-fernet-keys\") pod \"keystone-66778fc6-56rjc\" (UID: \"aa94b30e-0028-45a0-88aa-b5db5fc2f8fd\") " pod="openstack-kuttl-tests/keystone-66778fc6-56rjc" Dec 03 12:27:11 crc kubenswrapper[5052]: I1203 12:27:11.517887 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/aa94b30e-0028-45a0-88aa-b5db5fc2f8fd-fernet-keys\") pod \"keystone-66778fc6-56rjc\" (UID: \"aa94b30e-0028-45a0-88aa-b5db5fc2f8fd\") " pod="openstack-kuttl-tests/keystone-66778fc6-56rjc" Dec 03 12:27:11 crc kubenswrapper[5052]: I1203 12:27:11.521442 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/aa94b30e-0028-45a0-88aa-b5db5fc2f8fd-combined-ca-bundle\") pod \"keystone-66778fc6-56rjc\" (UID: \"aa94b30e-0028-45a0-88aa-b5db5fc2f8fd\") " pod="openstack-kuttl-tests/keystone-66778fc6-56rjc" Dec 03 12:27:11 crc kubenswrapper[5052]: I1203 12:27:11.527986 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/aa94b30e-0028-45a0-88aa-b5db5fc2f8fd-credential-keys\") pod \"keystone-66778fc6-56rjc\" (UID: \"aa94b30e-0028-45a0-88aa-b5db5fc2f8fd\") " pod="openstack-kuttl-tests/keystone-66778fc6-56rjc" Dec 03 12:27:11 crc kubenswrapper[5052]: I1203 12:27:11.569019 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/aa94b30e-0028-45a0-88aa-b5db5fc2f8fd-scripts\") pod \"keystone-66778fc6-56rjc\" (UID: \"aa94b30e-0028-45a0-88aa-b5db5fc2f8fd\") " pod="openstack-kuttl-tests/keystone-66778fc6-56rjc" Dec 03 12:27:11 crc kubenswrapper[5052]: I1203 12:27:11.572985 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/aa94b30e-0028-45a0-88aa-b5db5fc2f8fd-config-data\") pod \"keystone-66778fc6-56rjc\" (UID: \"aa94b30e-0028-45a0-88aa-b5db5fc2f8fd\") " pod="openstack-kuttl-tests/keystone-66778fc6-56rjc" Dec 03 12:27:11 crc kubenswrapper[5052]: I1203 12:27:11.573484 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-api-78665465bd-9q8ff" Dec 03 12:27:11 crc kubenswrapper[5052]: I1203 12:27:11.619223 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-fmmp2\" (UniqueName: \"kubernetes.io/projected/aa94b30e-0028-45a0-88aa-b5db5fc2f8fd-kube-api-access-fmmp2\") pod \"keystone-66778fc6-56rjc\" (UID: \"aa94b30e-0028-45a0-88aa-b5db5fc2f8fd\") " pod="openstack-kuttl-tests/keystone-66778fc6-56rjc" Dec 03 12:27:11 crc kubenswrapper[5052]: I1203 12:27:11.659527 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/neutron-544d7dcfcd-7xmk7"] Dec 03 12:27:11 crc kubenswrapper[5052]: I1203 12:27:11.704907 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-544d7dcfcd-7xmk7" Dec 03 12:27:11 crc kubenswrapper[5052]: I1203 12:27:11.705447 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-66778fc6-56rjc" Dec 03 12:27:11 crc kubenswrapper[5052]: I1203 12:27:11.766380 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 12:27:11 crc kubenswrapper[5052]: I1203 12:27:11.766416 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-0"] Dec 03 12:27:11 crc kubenswrapper[5052]: I1203 12:27:11.766590 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-cell0-conductor-0" podUID="1f6f8fbd-5810-4853-89c0-4be22abe85a3" containerName="nova-cell0-conductor-conductor" containerID="cri-o://e9967011ef9d1750324e927a4b9e79d05b9a0a1c3f9835799be84de44b11270d" gracePeriod=30 Dec 03 12:27:11 crc kubenswrapper[5052]: I1203 12:27:11.766779 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/cinder-api-0" podUID="fc4f1efa-a22b-4c43-b479-22562c0a7e65" containerName="cinder-api-log" containerID="cri-o://b9e28cc89fab64aa07a40886930111804a359cb4003f7a994ed468c2084e2abf" gracePeriod=30 Dec 03 12:27:11 crc kubenswrapper[5052]: I1203 12:27:11.767048 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/cinder-api-0" podUID="fc4f1efa-a22b-4c43-b479-22562c0a7e65" containerName="cinder-api" containerID="cri-o://00bb5044fc1f5280459a013a740283ad38025f50730707308a452a1f487b74f0" gracePeriod=30 Dec 03 12:27:11 crc kubenswrapper[5052]: I1203 12:27:11.810678 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-544d7dcfcd-7xmk7"] Dec 03 12:27:11 crc kubenswrapper[5052]: E1203 12:27:11.821455 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="e9967011ef9d1750324e927a4b9e79d05b9a0a1c3f9835799be84de44b11270d" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:27:11 crc kubenswrapper[5052]: E1203 12:27:11.826855 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="e9967011ef9d1750324e927a4b9e79d05b9a0a1c3f9835799be84de44b11270d" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:27:11 crc kubenswrapper[5052]: I1203 12:27:11.828156 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/135a97dd-af49-4eb8-8224-c3090757ed2a-httpd-config\") pod \"neutron-544d7dcfcd-7xmk7\" (UID: \"135a97dd-af49-4eb8-8224-c3090757ed2a\") " pod="openstack-kuttl-tests/neutron-544d7dcfcd-7xmk7" Dec 03 12:27:11 crc kubenswrapper[5052]: I1203 12:27:11.828194 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/135a97dd-af49-4eb8-8224-c3090757ed2a-combined-ca-bundle\") pod \"neutron-544d7dcfcd-7xmk7\" (UID: \"135a97dd-af49-4eb8-8224-c3090757ed2a\") " pod="openstack-kuttl-tests/neutron-544d7dcfcd-7xmk7" Dec 03 12:27:11 crc kubenswrapper[5052]: I1203 12:27:11.828247 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jdmsc\" (UniqueName: \"kubernetes.io/projected/135a97dd-af49-4eb8-8224-c3090757ed2a-kube-api-access-jdmsc\") pod \"neutron-544d7dcfcd-7xmk7\" (UID: \"135a97dd-af49-4eb8-8224-c3090757ed2a\") " pod="openstack-kuttl-tests/neutron-544d7dcfcd-7xmk7" Dec 03 12:27:11 crc kubenswrapper[5052]: I1203 12:27:11.828288 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/135a97dd-af49-4eb8-8224-c3090757ed2a-config\") pod \"neutron-544d7dcfcd-7xmk7\" (UID: \"135a97dd-af49-4eb8-8224-c3090757ed2a\") " pod="openstack-kuttl-tests/neutron-544d7dcfcd-7xmk7" Dec 03 12:27:11 crc kubenswrapper[5052]: I1203 12:27:11.844653 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell1-cell-mapping-hhtnd"] Dec 03 12:27:11 crc kubenswrapper[5052]: I1203 12:27:11.863857 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell1-cell-mapping-hhtnd"] Dec 03 12:27:11 crc kubenswrapper[5052]: E1203 12:27:11.865566 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="e9967011ef9d1750324e927a4b9e79d05b9a0a1c3f9835799be84de44b11270d" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:27:11 crc kubenswrapper[5052]: E1203 12:27:11.865616 5052 prober.go:104] "Probe errored" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" probeType="Readiness" pod="openstack-kuttl-tests/nova-cell0-conductor-0" podUID="1f6f8fbd-5810-4853-89c0-4be22abe85a3" containerName="nova-cell0-conductor-conductor" Dec 03 12:27:11 crc kubenswrapper[5052]: I1203 12:27:11.929647 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/135a97dd-af49-4eb8-8224-c3090757ed2a-httpd-config\") pod \"neutron-544d7dcfcd-7xmk7\" (UID: \"135a97dd-af49-4eb8-8224-c3090757ed2a\") " pod="openstack-kuttl-tests/neutron-544d7dcfcd-7xmk7" Dec 03 12:27:11 crc kubenswrapper[5052]: I1203 12:27:11.929709 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/135a97dd-af49-4eb8-8224-c3090757ed2a-combined-ca-bundle\") pod \"neutron-544d7dcfcd-7xmk7\" (UID: \"135a97dd-af49-4eb8-8224-c3090757ed2a\") " pod="openstack-kuttl-tests/neutron-544d7dcfcd-7xmk7" Dec 03 12:27:11 crc kubenswrapper[5052]: I1203 12:27:11.929785 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-jdmsc\" (UniqueName: \"kubernetes.io/projected/135a97dd-af49-4eb8-8224-c3090757ed2a-kube-api-access-jdmsc\") pod \"neutron-544d7dcfcd-7xmk7\" (UID: \"135a97dd-af49-4eb8-8224-c3090757ed2a\") " pod="openstack-kuttl-tests/neutron-544d7dcfcd-7xmk7" Dec 03 12:27:11 crc kubenswrapper[5052]: I1203 12:27:11.929838 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/135a97dd-af49-4eb8-8224-c3090757ed2a-config\") pod \"neutron-544d7dcfcd-7xmk7\" (UID: \"135a97dd-af49-4eb8-8224-c3090757ed2a\") " pod="openstack-kuttl-tests/neutron-544d7dcfcd-7xmk7" Dec 03 12:27:11 crc kubenswrapper[5052]: I1203 12:27:11.938727 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:27:11 crc kubenswrapper[5052]: I1203 12:27:11.939028 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-metadata-0" podUID="8e052136-a855-4a20-a8c3-38a8bef97dfc" containerName="nova-metadata-log" containerID="cri-o://f679be3b6603c919c59273998d120bbb192e07c332db88e94aabe9e26ca2af36" gracePeriod=30 Dec 03 12:27:11 crc kubenswrapper[5052]: I1203 12:27:11.939610 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-metadata-0" podUID="8e052136-a855-4a20-a8c3-38a8bef97dfc" containerName="nova-metadata-metadata" containerID="cri-o://0e8b8edad48c623ae81447848a940a98d6827814b85cda0ff7b96abb10a64815" gracePeriod=30 Dec 03 12:27:11 crc kubenswrapper[5052]: I1203 12:27:11.980689 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/135a97dd-af49-4eb8-8224-c3090757ed2a-httpd-config\") pod \"neutron-544d7dcfcd-7xmk7\" (UID: \"135a97dd-af49-4eb8-8224-c3090757ed2a\") " pod="openstack-kuttl-tests/neutron-544d7dcfcd-7xmk7" Dec 03 12:27:11 crc kubenswrapper[5052]: I1203 12:27:11.984459 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/135a97dd-af49-4eb8-8224-c3090757ed2a-config\") pod \"neutron-544d7dcfcd-7xmk7\" (UID: \"135a97dd-af49-4eb8-8224-c3090757ed2a\") " pod="openstack-kuttl-tests/neutron-544d7dcfcd-7xmk7" Dec 03 12:27:11 crc kubenswrapper[5052]: I1203 12:27:11.989639 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/135a97dd-af49-4eb8-8224-c3090757ed2a-combined-ca-bundle\") pod \"neutron-544d7dcfcd-7xmk7\" (UID: \"135a97dd-af49-4eb8-8224-c3090757ed2a\") " pod="openstack-kuttl-tests/neutron-544d7dcfcd-7xmk7" Dec 03 12:27:12 crc kubenswrapper[5052]: I1203 12:27:12.005541 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-jdmsc\" (UniqueName: \"kubernetes.io/projected/135a97dd-af49-4eb8-8224-c3090757ed2a-kube-api-access-jdmsc\") pod \"neutron-544d7dcfcd-7xmk7\" (UID: \"135a97dd-af49-4eb8-8224-c3090757ed2a\") " pod="openstack-kuttl-tests/neutron-544d7dcfcd-7xmk7" Dec 03 12:27:12 crc kubenswrapper[5052]: I1203 12:27:12.048583 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 12:27:12 crc kubenswrapper[5052]: I1203 12:27:12.048902 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" podUID="04aa1d5d-e683-417e-a96d-ce9af5a96136" containerName="nova-cell1-novncproxy-novncproxy" containerID="cri-o://10cf3584ca3062418591f9016c0b57483870a9df1de7e1394ee2ccefe62d4933" gracePeriod=30 Dec 03 12:27:12 crc kubenswrapper[5052]: I1203 12:27:12.106831 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-544d7dcfcd-7xmk7" Dec 03 12:27:12 crc kubenswrapper[5052]: I1203 12:27:12.132098 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:27:12 crc kubenswrapper[5052]: I1203 12:27:12.132347 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-scheduler-0" podUID="7ef14abf-76c0-4db5-9291-e0f9675425d1" containerName="nova-scheduler-scheduler" containerID="cri-o://74ef6e1fe61b1c5ce0d3bf482bd7a2e9edd6da7a23654c82cd5a4d8527f480c3" gracePeriod=30 Dec 03 12:27:12 crc kubenswrapper[5052]: I1203 12:27:12.205552 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:27:12 crc kubenswrapper[5052]: I1203 12:27:12.205784 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-api-0" podUID="7e415947-bb79-4d7b-80fb-f6d290343e19" containerName="nova-api-log" containerID="cri-o://281db729b55e9d4d2851c2134872d5a117b0767da33f8a3837143567bbe3f595" gracePeriod=30 Dec 03 12:27:12 crc kubenswrapper[5052]: I1203 12:27:12.209080 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-api-0" podUID="7e415947-bb79-4d7b-80fb-f6d290343e19" containerName="nova-api-api" containerID="cri-o://5acdaf1de9637eb9a2dcd430e89e4aa32e1830b27b6a59fec9de95f2f8f0c88f" gracePeriod=30 Dec 03 12:27:12 crc kubenswrapper[5052]: I1203 12:27:12.244168 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-0"] Dec 03 12:27:12 crc kubenswrapper[5052]: I1203 12:27:12.244375 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-cell1-conductor-0" podUID="51912caa-ca45-4e7c-a0ee-a40ae65a1896" containerName="nova-cell1-conductor-conductor" containerID="cri-o://7c7ee713ff0737c8c0810e900709d6bd56bb4cc9128cbcf656be4631e51882ae" gracePeriod=30 Dec 03 12:27:12 crc kubenswrapper[5052]: E1203 12:27:12.287191 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="7c7ee713ff0737c8c0810e900709d6bd56bb4cc9128cbcf656be4631e51882ae" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:27:12 crc kubenswrapper[5052]: E1203 12:27:12.295491 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="7c7ee713ff0737c8c0810e900709d6bd56bb4cc9128cbcf656be4631e51882ae" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:27:12 crc kubenswrapper[5052]: E1203 12:27:12.311139 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="7c7ee713ff0737c8c0810e900709d6bd56bb4cc9128cbcf656be4631e51882ae" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:27:12 crc kubenswrapper[5052]: E1203 12:27:12.311212 5052 prober.go:104] "Probe errored" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" probeType="Readiness" pod="openstack-kuttl-tests/nova-cell1-conductor-0" podUID="51912caa-ca45-4e7c-a0ee-a40ae65a1896" containerName="nova-cell1-conductor-conductor" Dec 03 12:27:12 crc kubenswrapper[5052]: I1203 12:27:12.329315 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-worker-64d9f4d555-4v5kn"] Dec 03 12:27:12 crc kubenswrapper[5052]: I1203 12:27:12.574192 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-keystone-listener-6df6465ddd-km2kt"] Dec 03 12:27:12 crc kubenswrapper[5052]: I1203 12:27:12.831461 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-66778fc6-56rjc"] Dec 03 12:27:12 crc kubenswrapper[5052]: I1203 12:27:12.886093 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-544d7dcfcd-7xmk7"] Dec 03 12:27:12 crc kubenswrapper[5052]: I1203 12:27:12.923118 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-api-78665465bd-9q8ff"] Dec 03 12:27:12 crc kubenswrapper[5052]: I1203 12:27:12.939903 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-66778fc6-56rjc" event={"ID":"aa94b30e-0028-45a0-88aa-b5db5fc2f8fd","Type":"ContainerStarted","Data":"b36bb6e4d776a944073051ec87e592c7e4df8a5403df5a5943eaa23ba3ffe5e3"} Dec 03 12:27:12 crc kubenswrapper[5052]: I1203 12:27:12.955797 5052 generic.go:334] "Generic (PLEG): container finished" podID="7e415947-bb79-4d7b-80fb-f6d290343e19" containerID="281db729b55e9d4d2851c2134872d5a117b0767da33f8a3837143567bbe3f595" exitCode=143 Dec 03 12:27:12 crc kubenswrapper[5052]: I1203 12:27:12.955872 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"7e415947-bb79-4d7b-80fb-f6d290343e19","Type":"ContainerDied","Data":"281db729b55e9d4d2851c2134872d5a117b0767da33f8a3837143567bbe3f595"} Dec 03 12:27:12 crc kubenswrapper[5052]: I1203 12:27:12.960063 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" podUID="04aa1d5d-e683-417e-a96d-ce9af5a96136" containerName="nova-cell1-novncproxy-novncproxy" probeResult="failure" output="Get \"http://10.217.0.108:6080/vnc_lite.html\": dial tcp 10.217.0.108:6080: connect: connection refused" Dec 03 12:27:12 crc kubenswrapper[5052]: I1203 12:27:12.967329 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-544d7dcfcd-7xmk7" event={"ID":"135a97dd-af49-4eb8-8224-c3090757ed2a","Type":"ContainerStarted","Data":"e4b8d4e8b2dce2be1e5e689f7cc49293bcb1ef7c5271e2b0df6422ab8b2e5ef7"} Dec 03 12:27:12 crc kubenswrapper[5052]: I1203 12:27:12.983865 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-worker-64d9f4d555-4v5kn" event={"ID":"6228bbc6-4c33-4ad8-ad26-440a8e6b441c","Type":"ContainerStarted","Data":"cad85f08979cf86c1f2f2aa7c3637ac5ab17fe2a4099a131b06311d2a7a911ff"} Dec 03 12:27:12 crc kubenswrapper[5052]: I1203 12:27:12.983978 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-worker-64d9f4d555-4v5kn" event={"ID":"6228bbc6-4c33-4ad8-ad26-440a8e6b441c","Type":"ContainerStarted","Data":"44946a83cf1f5a6cd01a57945285db24d218d9d49064b66a61eaf725fc91384e"} Dec 03 12:27:12 crc kubenswrapper[5052]: I1203 12:27:12.992145 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-keystone-listener-6df6465ddd-km2kt" event={"ID":"7f4dfec9-0aa8-437c-9b8d-771b30a93af9","Type":"ContainerStarted","Data":"54f3fa89ac27b88281c422bf43fdb03928a159ddffa85177a0997bcb486d4eee"} Dec 03 12:27:12 crc kubenswrapper[5052]: I1203 12:27:12.992189 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-keystone-listener-6df6465ddd-km2kt" event={"ID":"7f4dfec9-0aa8-437c-9b8d-771b30a93af9","Type":"ContainerStarted","Data":"8991d70aa079bf5ccd5fd999ab882977681310442bca6d1449b9233386d7876a"} Dec 03 12:27:12 crc kubenswrapper[5052]: I1203 12:27:12.995610 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/memcached-0"] Dec 03 12:27:12 crc kubenswrapper[5052]: I1203 12:27:12.995853 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/memcached-0" podUID="65fc3838-f25d-4eda-9643-6b63215c41ff" containerName="memcached" containerID="cri-o://0931db22f5fb013cdcb55311764fa795389b7087d437f3d1b8323540414014d9" gracePeriod=30 Dec 03 12:27:13 crc kubenswrapper[5052]: I1203 12:27:13.053818 5052 generic.go:334] "Generic (PLEG): container finished" podID="fc4f1efa-a22b-4c43-b479-22562c0a7e65" containerID="b9e28cc89fab64aa07a40886930111804a359cb4003f7a994ed468c2084e2abf" exitCode=143 Dec 03 12:27:13 crc kubenswrapper[5052]: I1203 12:27:13.053917 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"fc4f1efa-a22b-4c43-b479-22562c0a7e65","Type":"ContainerDied","Data":"b9e28cc89fab64aa07a40886930111804a359cb4003f7a994ed468c2084e2abf"} Dec 03 12:27:13 crc kubenswrapper[5052]: I1203 12:27:13.056870 5052 generic.go:334] "Generic (PLEG): container finished" podID="8e052136-a855-4a20-a8c3-38a8bef97dfc" containerID="f679be3b6603c919c59273998d120bbb192e07c332db88e94aabe9e26ca2af36" exitCode=143 Dec 03 12:27:13 crc kubenswrapper[5052]: I1203 12:27:13.056916 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"8e052136-a855-4a20-a8c3-38a8bef97dfc","Type":"ContainerDied","Data":"f679be3b6603c919c59273998d120bbb192e07c332db88e94aabe9e26ca2af36"} Dec 03 12:27:13 crc kubenswrapper[5052]: I1203 12:27:13.067217 5052 generic.go:334] "Generic (PLEG): container finished" podID="a823b6ba-b2af-4deb-88d2-0902fd6e3780" containerID="6cd2d71e42ed2933a0efd0de7d0e300b0da269f757f4f18802db5fd7de523192" exitCode=0 Dec 03 12:27:13 crc kubenswrapper[5052]: I1203 12:27:13.067573 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"a823b6ba-b2af-4deb-88d2-0902fd6e3780","Type":"ContainerDied","Data":"6cd2d71e42ed2933a0efd0de7d0e300b0da269f757f4f18802db5fd7de523192"} Dec 03 12:27:13 crc kubenswrapper[5052]: I1203 12:27:13.165160 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:27:13 crc kubenswrapper[5052]: I1203 12:27:13.165416 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-external-api-0" podUID="6b96dc78-b389-4764-8ba7-6a4c5e41e345" containerName="glance-log" containerID="cri-o://481bbc2057a57984fb7f4ba0b572e86b7319fb1c2d5b5c1d8d86cafb6e88f80f" gracePeriod=30 Dec 03 12:27:13 crc kubenswrapper[5052]: I1203 12:27:13.165671 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-external-api-0" podUID="6b96dc78-b389-4764-8ba7-6a4c5e41e345" containerName="glance-httpd" containerID="cri-o://be65e03ff7ec9d6c0e6aa36bae00bb290c5b0d2746c6c491262833760b551397" gracePeriod=30 Dec 03 12:27:13 crc kubenswrapper[5052]: I1203 12:27:13.347691 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/neutron-544d7dcfcd-7xmk7"] Dec 03 12:27:13 crc kubenswrapper[5052]: I1203 12:27:13.355987 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/swift-storage-0"] Dec 03 12:27:13 crc kubenswrapper[5052]: I1203 12:27:13.356457 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="7f163259-845f-4519-bd77-fa66e760d4db" containerName="account-server" containerID="cri-o://7f52642c73f37ab9180012c54a5c7e03d2db022e02528a07f0120cdace4c30ed" gracePeriod=30 Dec 03 12:27:13 crc kubenswrapper[5052]: I1203 12:27:13.356933 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="7f163259-845f-4519-bd77-fa66e760d4db" containerName="swift-recon-cron" containerID="cri-o://b20fcc98d22748a4865b1aff97aab861964ca72f2da839df89a8240768170b98" gracePeriod=30 Dec 03 12:27:13 crc kubenswrapper[5052]: I1203 12:27:13.357003 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="7f163259-845f-4519-bd77-fa66e760d4db" containerName="rsync" containerID="cri-o://1094f3eb2af0052222cc7befb863b42966285f5596b1be360edb3c5e5e097a14" gracePeriod=30 Dec 03 12:27:13 crc kubenswrapper[5052]: I1203 12:27:13.357038 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="7f163259-845f-4519-bd77-fa66e760d4db" containerName="object-expirer" containerID="cri-o://1127b9f54fde360052d578fc34e3f1ed816a67a8fd6f335d02508fa144bf6dc7" gracePeriod=30 Dec 03 12:27:13 crc kubenswrapper[5052]: I1203 12:27:13.357089 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="7f163259-845f-4519-bd77-fa66e760d4db" containerName="object-updater" containerID="cri-o://ba5a838f8abec837caa1abd8f6365324f2f4067f3e37755f33f8a119410d9c81" gracePeriod=30 Dec 03 12:27:13 crc kubenswrapper[5052]: I1203 12:27:13.357130 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="7f163259-845f-4519-bd77-fa66e760d4db" containerName="object-auditor" containerID="cri-o://c31f1fc401b9b5f1678ef68c3dacb95ce5a0a4803fb016c6e372e24171421a1a" gracePeriod=30 Dec 03 12:27:13 crc kubenswrapper[5052]: I1203 12:27:13.357177 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="7f163259-845f-4519-bd77-fa66e760d4db" containerName="object-replicator" containerID="cri-o://2de4cfddd3456322e2e08527a0b2b4ae7a715efc41cbfbeb5f1a6fa98e9a5409" gracePeriod=30 Dec 03 12:27:13 crc kubenswrapper[5052]: I1203 12:27:13.357214 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="7f163259-845f-4519-bd77-fa66e760d4db" containerName="object-server" containerID="cri-o://17ba623ea2c89f44d8458d92140c07db44a1f67ae586efa6a1261de7ddb83194" gracePeriod=30 Dec 03 12:27:13 crc kubenswrapper[5052]: I1203 12:27:13.357252 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="7f163259-845f-4519-bd77-fa66e760d4db" containerName="container-updater" containerID="cri-o://dfccd3db9b2af1521bf2032a601fa4662b4414697ee02def0c5cd28e2c96cb26" gracePeriod=30 Dec 03 12:27:13 crc kubenswrapper[5052]: I1203 12:27:13.357285 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="7f163259-845f-4519-bd77-fa66e760d4db" containerName="container-auditor" containerID="cri-o://85d94c44803aac08a74f194f52b8757506853bfa45354a4ed0a41234617514b0" gracePeriod=30 Dec 03 12:27:13 crc kubenswrapper[5052]: I1203 12:27:13.357317 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="7f163259-845f-4519-bd77-fa66e760d4db" containerName="container-replicator" containerID="cri-o://846159aaedd764c9a39adccf404c36e1798796b1dc058a4077567aead1622992" gracePeriod=30 Dec 03 12:27:13 crc kubenswrapper[5052]: I1203 12:27:13.357354 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="7f163259-845f-4519-bd77-fa66e760d4db" containerName="container-server" containerID="cri-o://c546226444d76b3bae4019c590f2db31d0aa8aa1810311506bf0679abe0ac7da" gracePeriod=30 Dec 03 12:27:13 crc kubenswrapper[5052]: I1203 12:27:13.357385 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="7f163259-845f-4519-bd77-fa66e760d4db" containerName="account-reaper" containerID="cri-o://c021f6fefe53bade62d592029616b253503b6690e220f6de008a4d7e297826e5" gracePeriod=30 Dec 03 12:27:13 crc kubenswrapper[5052]: I1203 12:27:13.357416 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="7f163259-845f-4519-bd77-fa66e760d4db" containerName="account-auditor" containerID="cri-o://9818644c3d189b7752f2c0e1f85147e5450c62440f231eba257e3c2c1869464e" gracePeriod=30 Dec 03 12:27:13 crc kubenswrapper[5052]: I1203 12:27:13.357445 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="7f163259-845f-4519-bd77-fa66e760d4db" containerName="account-replicator" containerID="cri-o://7ce328ddacfb675b5ded2a8547af6ba9940a95baba352f048c9c9dce642c7103" gracePeriod=30 Dec 03 12:27:13 crc kubenswrapper[5052]: I1203 12:27:13.416460 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:27:13 crc kubenswrapper[5052]: I1203 12:27:13.419839 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-internal-api-0" podUID="7fa6aac1-6ef3-434a-945e-3ced02ca5361" containerName="glance-log" containerID="cri-o://f4fbadcae236c9bdee4f357a1121e0e675398d9aeeaeec64e67d0177ab9e352c" gracePeriod=30 Dec 03 12:27:13 crc kubenswrapper[5052]: I1203 12:27:13.420027 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-internal-api-0" podUID="7fa6aac1-6ef3-434a-945e-3ced02ca5361" containerName="glance-httpd" containerID="cri-o://d09526b701062188ac0714b7d2d4c1986577ed904704b893ec88c6d251aa69cb" gracePeriod=30 Dec 03 12:27:13 crc kubenswrapper[5052]: I1203 12:27:13.499057 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/neutron-797ccd5f76-v8cmg"] Dec 03 12:27:13 crc kubenswrapper[5052]: I1203 12:27:13.500975 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-797ccd5f76-v8cmg" Dec 03 12:27:13 crc kubenswrapper[5052]: I1203 12:27:13.593851 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-797ccd5f76-v8cmg"] Dec 03 12:27:13 crc kubenswrapper[5052]: I1203 12:27:13.602131 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/5314ef6e-9ca6-4c9b-98b0-907b5a6e423c-httpd-config\") pod \"neutron-797ccd5f76-v8cmg\" (UID: \"5314ef6e-9ca6-4c9b-98b0-907b5a6e423c\") " pod="openstack-kuttl-tests/neutron-797ccd5f76-v8cmg" Dec 03 12:27:13 crc kubenswrapper[5052]: I1203 12:27:13.602277 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/5314ef6e-9ca6-4c9b-98b0-907b5a6e423c-config\") pod \"neutron-797ccd5f76-v8cmg\" (UID: \"5314ef6e-9ca6-4c9b-98b0-907b5a6e423c\") " pod="openstack-kuttl-tests/neutron-797ccd5f76-v8cmg" Dec 03 12:27:13 crc kubenswrapper[5052]: I1203 12:27:13.602434 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5314ef6e-9ca6-4c9b-98b0-907b5a6e423c-combined-ca-bundle\") pod \"neutron-797ccd5f76-v8cmg\" (UID: \"5314ef6e-9ca6-4c9b-98b0-907b5a6e423c\") " pod="openstack-kuttl-tests/neutron-797ccd5f76-v8cmg" Dec 03 12:27:13 crc kubenswrapper[5052]: I1203 12:27:13.602534 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4h5fs\" (UniqueName: \"kubernetes.io/projected/5314ef6e-9ca6-4c9b-98b0-907b5a6e423c-kube-api-access-4h5fs\") pod \"neutron-797ccd5f76-v8cmg\" (UID: \"5314ef6e-9ca6-4c9b-98b0-907b5a6e423c\") " pod="openstack-kuttl-tests/neutron-797ccd5f76-v8cmg" Dec 03 12:27:13 crc kubenswrapper[5052]: I1203 12:27:13.733073 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/5314ef6e-9ca6-4c9b-98b0-907b5a6e423c-config\") pod \"neutron-797ccd5f76-v8cmg\" (UID: \"5314ef6e-9ca6-4c9b-98b0-907b5a6e423c\") " pod="openstack-kuttl-tests/neutron-797ccd5f76-v8cmg" Dec 03 12:27:13 crc kubenswrapper[5052]: I1203 12:27:13.733316 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5314ef6e-9ca6-4c9b-98b0-907b5a6e423c-combined-ca-bundle\") pod \"neutron-797ccd5f76-v8cmg\" (UID: \"5314ef6e-9ca6-4c9b-98b0-907b5a6e423c\") " pod="openstack-kuttl-tests/neutron-797ccd5f76-v8cmg" Dec 03 12:27:13 crc kubenswrapper[5052]: I1203 12:27:13.733441 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-4h5fs\" (UniqueName: \"kubernetes.io/projected/5314ef6e-9ca6-4c9b-98b0-907b5a6e423c-kube-api-access-4h5fs\") pod \"neutron-797ccd5f76-v8cmg\" (UID: \"5314ef6e-9ca6-4c9b-98b0-907b5a6e423c\") " pod="openstack-kuttl-tests/neutron-797ccd5f76-v8cmg" Dec 03 12:27:13 crc kubenswrapper[5052]: I1203 12:27:13.738418 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/5314ef6e-9ca6-4c9b-98b0-907b5a6e423c-httpd-config\") pod \"neutron-797ccd5f76-v8cmg\" (UID: \"5314ef6e-9ca6-4c9b-98b0-907b5a6e423c\") " pod="openstack-kuttl-tests/neutron-797ccd5f76-v8cmg" Dec 03 12:27:13 crc kubenswrapper[5052]: I1203 12:27:13.747412 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/5314ef6e-9ca6-4c9b-98b0-907b5a6e423c-config\") pod \"neutron-797ccd5f76-v8cmg\" (UID: \"5314ef6e-9ca6-4c9b-98b0-907b5a6e423c\") " pod="openstack-kuttl-tests/neutron-797ccd5f76-v8cmg" Dec 03 12:27:13 crc kubenswrapper[5052]: I1203 12:27:13.780418 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-4h5fs\" (UniqueName: \"kubernetes.io/projected/5314ef6e-9ca6-4c9b-98b0-907b5a6e423c-kube-api-access-4h5fs\") pod \"neutron-797ccd5f76-v8cmg\" (UID: \"5314ef6e-9ca6-4c9b-98b0-907b5a6e423c\") " pod="openstack-kuttl-tests/neutron-797ccd5f76-v8cmg" Dec 03 12:27:13 crc kubenswrapper[5052]: I1203 12:27:13.790705 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eb8c41b3-9282-4a42-8122-d648450a757d" path="/var/lib/kubelet/pods/eb8c41b3-9282-4a42-8122-d648450a757d/volumes" Dec 03 12:27:13 crc kubenswrapper[5052]: I1203 12:27:13.795784 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/keystone-66778fc6-56rjc"] Dec 03 12:27:13 crc kubenswrapper[5052]: I1203 12:27:13.806340 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/5314ef6e-9ca6-4c9b-98b0-907b5a6e423c-httpd-config\") pod \"neutron-797ccd5f76-v8cmg\" (UID: \"5314ef6e-9ca6-4c9b-98b0-907b5a6e423c\") " pod="openstack-kuttl-tests/neutron-797ccd5f76-v8cmg" Dec 03 12:27:13 crc kubenswrapper[5052]: I1203 12:27:13.812877 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5314ef6e-9ca6-4c9b-98b0-907b5a6e423c-combined-ca-bundle\") pod \"neutron-797ccd5f76-v8cmg\" (UID: \"5314ef6e-9ca6-4c9b-98b0-907b5a6e423c\") " pod="openstack-kuttl-tests/neutron-797ccd5f76-v8cmg" Dec 03 12:27:13 crc kubenswrapper[5052]: I1203 12:27:13.876626 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/keystone-64d6d88f66-7z998"] Dec 03 12:27:13 crc kubenswrapper[5052]: I1203 12:27:13.878030 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-64d6d88f66-7z998" Dec 03 12:27:13 crc kubenswrapper[5052]: I1203 12:27:13.886918 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-797ccd5f76-v8cmg" Dec 03 12:27:13 crc kubenswrapper[5052]: I1203 12:27:13.900133 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:27:13 crc kubenswrapper[5052]: I1203 12:27:13.909135 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-64d6d88f66-7z998"] Dec 03 12:27:14 crc kubenswrapper[5052]: I1203 12:27:13.995954 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/rabbitmq-server-0" podUID="cd7b6179-ba41-43bd-bc95-490a88aa15fd" containerName="rabbitmq" containerID="cri-o://229c6c7d07d677ea52e66801e341130c260352a54816a6ceeecbd91e2597db8c" gracePeriod=604797 Dec 03 12:27:14 crc kubenswrapper[5052]: I1203 12:27:14.060541 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/04aa1d5d-e683-417e-a96d-ce9af5a96136-config-data\") pod \"04aa1d5d-e683-417e-a96d-ce9af5a96136\" (UID: \"04aa1d5d-e683-417e-a96d-ce9af5a96136\") " Dec 03 12:27:14 crc kubenswrapper[5052]: I1203 12:27:14.060804 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/04aa1d5d-e683-417e-a96d-ce9af5a96136-combined-ca-bundle\") pod \"04aa1d5d-e683-417e-a96d-ce9af5a96136\" (UID: \"04aa1d5d-e683-417e-a96d-ce9af5a96136\") " Dec 03 12:27:14 crc kubenswrapper[5052]: I1203 12:27:14.060869 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-prnlf\" (UniqueName: \"kubernetes.io/projected/04aa1d5d-e683-417e-a96d-ce9af5a96136-kube-api-access-prnlf\") pod \"04aa1d5d-e683-417e-a96d-ce9af5a96136\" (UID: \"04aa1d5d-e683-417e-a96d-ce9af5a96136\") " Dec 03 12:27:14 crc kubenswrapper[5052]: I1203 12:27:14.061092 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/ef332add-3219-4025-be5b-9d7048cd4a3d-credential-keys\") pod \"keystone-64d6d88f66-7z998\" (UID: \"ef332add-3219-4025-be5b-9d7048cd4a3d\") " pod="openstack-kuttl-tests/keystone-64d6d88f66-7z998" Dec 03 12:27:14 crc kubenswrapper[5052]: I1203 12:27:14.061223 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/ef332add-3219-4025-be5b-9d7048cd4a3d-config-data\") pod \"keystone-64d6d88f66-7z998\" (UID: \"ef332add-3219-4025-be5b-9d7048cd4a3d\") " pod="openstack-kuttl-tests/keystone-64d6d88f66-7z998" Dec 03 12:27:14 crc kubenswrapper[5052]: I1203 12:27:14.061272 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/ef332add-3219-4025-be5b-9d7048cd4a3d-fernet-keys\") pod \"keystone-64d6d88f66-7z998\" (UID: \"ef332add-3219-4025-be5b-9d7048cd4a3d\") " pod="openstack-kuttl-tests/keystone-64d6d88f66-7z998" Dec 03 12:27:14 crc kubenswrapper[5052]: I1203 12:27:14.061289 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ef332add-3219-4025-be5b-9d7048cd4a3d-combined-ca-bundle\") pod \"keystone-64d6d88f66-7z998\" (UID: \"ef332add-3219-4025-be5b-9d7048cd4a3d\") " pod="openstack-kuttl-tests/keystone-64d6d88f66-7z998" Dec 03 12:27:14 crc kubenswrapper[5052]: I1203 12:27:14.061314 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/ef332add-3219-4025-be5b-9d7048cd4a3d-scripts\") pod \"keystone-64d6d88f66-7z998\" (UID: \"ef332add-3219-4025-be5b-9d7048cd4a3d\") " pod="openstack-kuttl-tests/keystone-64d6d88f66-7z998" Dec 03 12:27:14 crc kubenswrapper[5052]: I1203 12:27:14.061387 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x4krj\" (UniqueName: \"kubernetes.io/projected/ef332add-3219-4025-be5b-9d7048cd4a3d-kube-api-access-x4krj\") pod \"keystone-64d6d88f66-7z998\" (UID: \"ef332add-3219-4025-be5b-9d7048cd4a3d\") " pod="openstack-kuttl-tests/keystone-64d6d88f66-7z998" Dec 03 12:27:14 crc kubenswrapper[5052]: I1203 12:27:14.084501 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/04aa1d5d-e683-417e-a96d-ce9af5a96136-kube-api-access-prnlf" (OuterVolumeSpecName: "kube-api-access-prnlf") pod "04aa1d5d-e683-417e-a96d-ce9af5a96136" (UID: "04aa1d5d-e683-417e-a96d-ce9af5a96136"). InnerVolumeSpecName "kube-api-access-prnlf". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:27:14 crc kubenswrapper[5052]: I1203 12:27:14.158797 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/04aa1d5d-e683-417e-a96d-ce9af5a96136-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "04aa1d5d-e683-417e-a96d-ce9af5a96136" (UID: "04aa1d5d-e683-417e-a96d-ce9af5a96136"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:27:14 crc kubenswrapper[5052]: I1203 12:27:14.159071 5052 generic.go:334] "Generic (PLEG): container finished" podID="04aa1d5d-e683-417e-a96d-ce9af5a96136" containerID="10cf3584ca3062418591f9016c0b57483870a9df1de7e1394ee2ccefe62d4933" exitCode=0 Dec 03 12:27:14 crc kubenswrapper[5052]: I1203 12:27:14.159158 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" event={"ID":"04aa1d5d-e683-417e-a96d-ce9af5a96136","Type":"ContainerDied","Data":"10cf3584ca3062418591f9016c0b57483870a9df1de7e1394ee2ccefe62d4933"} Dec 03 12:27:14 crc kubenswrapper[5052]: I1203 12:27:14.159188 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" event={"ID":"04aa1d5d-e683-417e-a96d-ce9af5a96136","Type":"ContainerDied","Data":"8f49d49c5caaa0aebf5069c64c021effb694efcfd0a78abcac5d12ca7872efa1"} Dec 03 12:27:14 crc kubenswrapper[5052]: I1203 12:27:14.159217 5052 scope.go:117] "RemoveContainer" containerID="10cf3584ca3062418591f9016c0b57483870a9df1de7e1394ee2ccefe62d4933" Dec 03 12:27:14 crc kubenswrapper[5052]: I1203 12:27:14.159389 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:27:14 crc kubenswrapper[5052]: I1203 12:27:14.164178 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/ef332add-3219-4025-be5b-9d7048cd4a3d-credential-keys\") pod \"keystone-64d6d88f66-7z998\" (UID: \"ef332add-3219-4025-be5b-9d7048cd4a3d\") " pod="openstack-kuttl-tests/keystone-64d6d88f66-7z998" Dec 03 12:27:14 crc kubenswrapper[5052]: I1203 12:27:14.164352 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/ef332add-3219-4025-be5b-9d7048cd4a3d-config-data\") pod \"keystone-64d6d88f66-7z998\" (UID: \"ef332add-3219-4025-be5b-9d7048cd4a3d\") " pod="openstack-kuttl-tests/keystone-64d6d88f66-7z998" Dec 03 12:27:14 crc kubenswrapper[5052]: I1203 12:27:14.164414 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/ef332add-3219-4025-be5b-9d7048cd4a3d-fernet-keys\") pod \"keystone-64d6d88f66-7z998\" (UID: \"ef332add-3219-4025-be5b-9d7048cd4a3d\") " pod="openstack-kuttl-tests/keystone-64d6d88f66-7z998" Dec 03 12:27:14 crc kubenswrapper[5052]: I1203 12:27:14.164437 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ef332add-3219-4025-be5b-9d7048cd4a3d-combined-ca-bundle\") pod \"keystone-64d6d88f66-7z998\" (UID: \"ef332add-3219-4025-be5b-9d7048cd4a3d\") " pod="openstack-kuttl-tests/keystone-64d6d88f66-7z998" Dec 03 12:27:14 crc kubenswrapper[5052]: I1203 12:27:14.164476 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/ef332add-3219-4025-be5b-9d7048cd4a3d-scripts\") pod \"keystone-64d6d88f66-7z998\" (UID: \"ef332add-3219-4025-be5b-9d7048cd4a3d\") " pod="openstack-kuttl-tests/keystone-64d6d88f66-7z998" Dec 03 12:27:14 crc kubenswrapper[5052]: I1203 12:27:14.164561 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-x4krj\" (UniqueName: \"kubernetes.io/projected/ef332add-3219-4025-be5b-9d7048cd4a3d-kube-api-access-x4krj\") pod \"keystone-64d6d88f66-7z998\" (UID: \"ef332add-3219-4025-be5b-9d7048cd4a3d\") " pod="openstack-kuttl-tests/keystone-64d6d88f66-7z998" Dec 03 12:27:14 crc kubenswrapper[5052]: I1203 12:27:14.164691 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/04aa1d5d-e683-417e-a96d-ce9af5a96136-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:14 crc kubenswrapper[5052]: I1203 12:27:14.164707 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-prnlf\" (UniqueName: \"kubernetes.io/projected/04aa1d5d-e683-417e-a96d-ce9af5a96136-kube-api-access-prnlf\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:14 crc kubenswrapper[5052]: I1203 12:27:14.192584 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-x4krj\" (UniqueName: \"kubernetes.io/projected/ef332add-3219-4025-be5b-9d7048cd4a3d-kube-api-access-x4krj\") pod \"keystone-64d6d88f66-7z998\" (UID: \"ef332add-3219-4025-be5b-9d7048cd4a3d\") " pod="openstack-kuttl-tests/keystone-64d6d88f66-7z998" Dec 03 12:27:14 crc kubenswrapper[5052]: I1203 12:27:14.201558 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/ef332add-3219-4025-be5b-9d7048cd4a3d-credential-keys\") pod \"keystone-64d6d88f66-7z998\" (UID: \"ef332add-3219-4025-be5b-9d7048cd4a3d\") " pod="openstack-kuttl-tests/keystone-64d6d88f66-7z998" Dec 03 12:27:14 crc kubenswrapper[5052]: I1203 12:27:14.202248 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-keystone-listener-6df6465ddd-km2kt" event={"ID":"7f4dfec9-0aa8-437c-9b8d-771b30a93af9","Type":"ContainerStarted","Data":"bc39c9a553475072254f96ed656dff329e085eb9d86080fd0b82411e0f0780a9"} Dec 03 12:27:14 crc kubenswrapper[5052]: I1203 12:27:14.212534 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/ef332add-3219-4025-be5b-9d7048cd4a3d-config-data\") pod \"keystone-64d6d88f66-7z998\" (UID: \"ef332add-3219-4025-be5b-9d7048cd4a3d\") " pod="openstack-kuttl-tests/keystone-64d6d88f66-7z998" Dec 03 12:27:14 crc kubenswrapper[5052]: I1203 12:27:14.213115 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/ef332add-3219-4025-be5b-9d7048cd4a3d-fernet-keys\") pod \"keystone-64d6d88f66-7z998\" (UID: \"ef332add-3219-4025-be5b-9d7048cd4a3d\") " pod="openstack-kuttl-tests/keystone-64d6d88f66-7z998" Dec 03 12:27:14 crc kubenswrapper[5052]: I1203 12:27:14.213347 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/ef332add-3219-4025-be5b-9d7048cd4a3d-scripts\") pod \"keystone-64d6d88f66-7z998\" (UID: \"ef332add-3219-4025-be5b-9d7048cd4a3d\") " pod="openstack-kuttl-tests/keystone-64d6d88f66-7z998" Dec 03 12:27:14 crc kubenswrapper[5052]: I1203 12:27:14.214631 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ef332add-3219-4025-be5b-9d7048cd4a3d-combined-ca-bundle\") pod \"keystone-64d6d88f66-7z998\" (UID: \"ef332add-3219-4025-be5b-9d7048cd4a3d\") " pod="openstack-kuttl-tests/keystone-64d6d88f66-7z998" Dec 03 12:27:14 crc kubenswrapper[5052]: I1203 12:27:14.224090 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-64d6d88f66-7z998" Dec 03 12:27:14 crc kubenswrapper[5052]: I1203 12:27:14.238140 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/04aa1d5d-e683-417e-a96d-ce9af5a96136-config-data" (OuterVolumeSpecName: "config-data") pod "04aa1d5d-e683-417e-a96d-ce9af5a96136" (UID: "04aa1d5d-e683-417e-a96d-ce9af5a96136"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:27:14 crc kubenswrapper[5052]: I1203 12:27:14.271551 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/barbican-keystone-listener-6df6465ddd-km2kt" podStartSLOduration=4.271531543 podStartE2EDuration="4.271531543s" podCreationTimestamp="2025-12-03 12:27:10 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:27:14.253806637 +0000 UTC m=+4024.842048401" watchObservedRunningTime="2025-12-03 12:27:14.271531543 +0000 UTC m=+4024.859773327" Dec 03 12:27:14 crc kubenswrapper[5052]: I1203 12:27:14.274036 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/04aa1d5d-e683-417e-a96d-ce9af5a96136-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:14 crc kubenswrapper[5052]: I1203 12:27:14.302643 5052 generic.go:334] "Generic (PLEG): container finished" podID="7f163259-845f-4519-bd77-fa66e760d4db" containerID="1094f3eb2af0052222cc7befb863b42966285f5596b1be360edb3c5e5e097a14" exitCode=0 Dec 03 12:27:14 crc kubenswrapper[5052]: I1203 12:27:14.302672 5052 generic.go:334] "Generic (PLEG): container finished" podID="7f163259-845f-4519-bd77-fa66e760d4db" containerID="1127b9f54fde360052d578fc34e3f1ed816a67a8fd6f335d02508fa144bf6dc7" exitCode=0 Dec 03 12:27:14 crc kubenswrapper[5052]: I1203 12:27:14.302682 5052 generic.go:334] "Generic (PLEG): container finished" podID="7f163259-845f-4519-bd77-fa66e760d4db" containerID="ba5a838f8abec837caa1abd8f6365324f2f4067f3e37755f33f8a119410d9c81" exitCode=0 Dec 03 12:27:14 crc kubenswrapper[5052]: I1203 12:27:14.302692 5052 generic.go:334] "Generic (PLEG): container finished" podID="7f163259-845f-4519-bd77-fa66e760d4db" containerID="c31f1fc401b9b5f1678ef68c3dacb95ce5a0a4803fb016c6e372e24171421a1a" exitCode=0 Dec 03 12:27:14 crc kubenswrapper[5052]: I1203 12:27:14.302700 5052 generic.go:334] "Generic (PLEG): container finished" podID="7f163259-845f-4519-bd77-fa66e760d4db" containerID="2de4cfddd3456322e2e08527a0b2b4ae7a715efc41cbfbeb5f1a6fa98e9a5409" exitCode=0 Dec 03 12:27:14 crc kubenswrapper[5052]: I1203 12:27:14.302707 5052 generic.go:334] "Generic (PLEG): container finished" podID="7f163259-845f-4519-bd77-fa66e760d4db" containerID="17ba623ea2c89f44d8458d92140c07db44a1f67ae586efa6a1261de7ddb83194" exitCode=0 Dec 03 12:27:14 crc kubenswrapper[5052]: I1203 12:27:14.302714 5052 generic.go:334] "Generic (PLEG): container finished" podID="7f163259-845f-4519-bd77-fa66e760d4db" containerID="dfccd3db9b2af1521bf2032a601fa4662b4414697ee02def0c5cd28e2c96cb26" exitCode=0 Dec 03 12:27:14 crc kubenswrapper[5052]: I1203 12:27:14.302722 5052 generic.go:334] "Generic (PLEG): container finished" podID="7f163259-845f-4519-bd77-fa66e760d4db" containerID="85d94c44803aac08a74f194f52b8757506853bfa45354a4ed0a41234617514b0" exitCode=0 Dec 03 12:27:14 crc kubenswrapper[5052]: I1203 12:27:14.302730 5052 generic.go:334] "Generic (PLEG): container finished" podID="7f163259-845f-4519-bd77-fa66e760d4db" containerID="846159aaedd764c9a39adccf404c36e1798796b1dc058a4077567aead1622992" exitCode=0 Dec 03 12:27:14 crc kubenswrapper[5052]: I1203 12:27:14.302739 5052 generic.go:334] "Generic (PLEG): container finished" podID="7f163259-845f-4519-bd77-fa66e760d4db" containerID="c546226444d76b3bae4019c590f2db31d0aa8aa1810311506bf0679abe0ac7da" exitCode=0 Dec 03 12:27:14 crc kubenswrapper[5052]: I1203 12:27:14.302748 5052 generic.go:334] "Generic (PLEG): container finished" podID="7f163259-845f-4519-bd77-fa66e760d4db" containerID="c021f6fefe53bade62d592029616b253503b6690e220f6de008a4d7e297826e5" exitCode=0 Dec 03 12:27:14 crc kubenswrapper[5052]: I1203 12:27:14.302757 5052 generic.go:334] "Generic (PLEG): container finished" podID="7f163259-845f-4519-bd77-fa66e760d4db" containerID="9818644c3d189b7752f2c0e1f85147e5450c62440f231eba257e3c2c1869464e" exitCode=0 Dec 03 12:27:14 crc kubenswrapper[5052]: I1203 12:27:14.302765 5052 generic.go:334] "Generic (PLEG): container finished" podID="7f163259-845f-4519-bd77-fa66e760d4db" containerID="7ce328ddacfb675b5ded2a8547af6ba9940a95baba352f048c9c9dce642c7103" exitCode=0 Dec 03 12:27:14 crc kubenswrapper[5052]: I1203 12:27:14.302825 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"7f163259-845f-4519-bd77-fa66e760d4db","Type":"ContainerDied","Data":"1094f3eb2af0052222cc7befb863b42966285f5596b1be360edb3c5e5e097a14"} Dec 03 12:27:14 crc kubenswrapper[5052]: I1203 12:27:14.302894 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"7f163259-845f-4519-bd77-fa66e760d4db","Type":"ContainerDied","Data":"1127b9f54fde360052d578fc34e3f1ed816a67a8fd6f335d02508fa144bf6dc7"} Dec 03 12:27:14 crc kubenswrapper[5052]: I1203 12:27:14.302910 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"7f163259-845f-4519-bd77-fa66e760d4db","Type":"ContainerDied","Data":"ba5a838f8abec837caa1abd8f6365324f2f4067f3e37755f33f8a119410d9c81"} Dec 03 12:27:14 crc kubenswrapper[5052]: I1203 12:27:14.302923 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"7f163259-845f-4519-bd77-fa66e760d4db","Type":"ContainerDied","Data":"c31f1fc401b9b5f1678ef68c3dacb95ce5a0a4803fb016c6e372e24171421a1a"} Dec 03 12:27:14 crc kubenswrapper[5052]: I1203 12:27:14.302932 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"7f163259-845f-4519-bd77-fa66e760d4db","Type":"ContainerDied","Data":"2de4cfddd3456322e2e08527a0b2b4ae7a715efc41cbfbeb5f1a6fa98e9a5409"} Dec 03 12:27:14 crc kubenswrapper[5052]: I1203 12:27:14.302943 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"7f163259-845f-4519-bd77-fa66e760d4db","Type":"ContainerDied","Data":"17ba623ea2c89f44d8458d92140c07db44a1f67ae586efa6a1261de7ddb83194"} Dec 03 12:27:14 crc kubenswrapper[5052]: I1203 12:27:14.302952 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"7f163259-845f-4519-bd77-fa66e760d4db","Type":"ContainerDied","Data":"dfccd3db9b2af1521bf2032a601fa4662b4414697ee02def0c5cd28e2c96cb26"} Dec 03 12:27:14 crc kubenswrapper[5052]: I1203 12:27:14.302963 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"7f163259-845f-4519-bd77-fa66e760d4db","Type":"ContainerDied","Data":"85d94c44803aac08a74f194f52b8757506853bfa45354a4ed0a41234617514b0"} Dec 03 12:27:14 crc kubenswrapper[5052]: I1203 12:27:14.302971 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"7f163259-845f-4519-bd77-fa66e760d4db","Type":"ContainerDied","Data":"846159aaedd764c9a39adccf404c36e1798796b1dc058a4077567aead1622992"} Dec 03 12:27:14 crc kubenswrapper[5052]: I1203 12:27:14.302980 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"7f163259-845f-4519-bd77-fa66e760d4db","Type":"ContainerDied","Data":"c546226444d76b3bae4019c590f2db31d0aa8aa1810311506bf0679abe0ac7da"} Dec 03 12:27:14 crc kubenswrapper[5052]: I1203 12:27:14.302988 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"7f163259-845f-4519-bd77-fa66e760d4db","Type":"ContainerDied","Data":"c021f6fefe53bade62d592029616b253503b6690e220f6de008a4d7e297826e5"} Dec 03 12:27:14 crc kubenswrapper[5052]: I1203 12:27:14.302997 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"7f163259-845f-4519-bd77-fa66e760d4db","Type":"ContainerDied","Data":"9818644c3d189b7752f2c0e1f85147e5450c62440f231eba257e3c2c1869464e"} Dec 03 12:27:14 crc kubenswrapper[5052]: I1203 12:27:14.303133 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"7f163259-845f-4519-bd77-fa66e760d4db","Type":"ContainerDied","Data":"7ce328ddacfb675b5ded2a8547af6ba9940a95baba352f048c9c9dce642c7103"} Dec 03 12:27:14 crc kubenswrapper[5052]: I1203 12:27:14.309330 5052 generic.go:334] "Generic (PLEG): container finished" podID="7fa6aac1-6ef3-434a-945e-3ced02ca5361" containerID="f4fbadcae236c9bdee4f357a1121e0e675398d9aeeaeec64e67d0177ab9e352c" exitCode=143 Dec 03 12:27:14 crc kubenswrapper[5052]: I1203 12:27:14.309407 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"7fa6aac1-6ef3-434a-945e-3ced02ca5361","Type":"ContainerDied","Data":"f4fbadcae236c9bdee4f357a1121e0e675398d9aeeaeec64e67d0177ab9e352c"} Dec 03 12:27:14 crc kubenswrapper[5052]: I1203 12:27:14.311329 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-66778fc6-56rjc" event={"ID":"aa94b30e-0028-45a0-88aa-b5db5fc2f8fd","Type":"ContainerStarted","Data":"22275b3672b8b4add52b127bbe276c42c2a5e8d7f524d0cf5e1961f6aee9186c"} Dec 03 12:27:14 crc kubenswrapper[5052]: I1203 12:27:14.311870 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/keystone-66778fc6-56rjc" Dec 03 12:27:14 crc kubenswrapper[5052]: I1203 12:27:14.356040 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-keystone-listener-7c58f6698b-95l8v"] Dec 03 12:27:14 crc kubenswrapper[5052]: I1203 12:27:14.356300 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/barbican-keystone-listener-7c58f6698b-95l8v" podUID="98d0abd5-a20b-45e3-aa30-34f038b2fb3d" containerName="barbican-keystone-listener-log" containerID="cri-o://06e057704675bafd6f98189c6b86c43903be3fc1372671a1851fba5082cffb21" gracePeriod=30 Dec 03 12:27:14 crc kubenswrapper[5052]: I1203 12:27:14.356774 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/barbican-keystone-listener-7c58f6698b-95l8v" podUID="98d0abd5-a20b-45e3-aa30-34f038b2fb3d" containerName="barbican-keystone-listener" containerID="cri-o://67002cc386bd7047ddbb9535a8f1433365620854531590af15f527d196de051a" gracePeriod=30 Dec 03 12:27:14 crc kubenswrapper[5052]: I1203 12:27:14.357964 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/keystone-66778fc6-56rjc" podStartSLOduration=3.357943419 podStartE2EDuration="3.357943419s" podCreationTimestamp="2025-12-03 12:27:11 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:27:14.339114632 +0000 UTC m=+4024.927356406" watchObservedRunningTime="2025-12-03 12:27:14.357943419 +0000 UTC m=+4024.946185183" Dec 03 12:27:14 crc kubenswrapper[5052]: I1203 12:27:14.370311 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-worker-64d9f4d555-4v5kn" event={"ID":"6228bbc6-4c33-4ad8-ad26-440a8e6b441c","Type":"ContainerStarted","Data":"595a1417c6646739c93678d8740cd28a9eadb61887c7e437c16c9c0ee312a81e"} Dec 03 12:27:14 crc kubenswrapper[5052]: I1203 12:27:14.378066 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/memcached-0" podUID="65fc3838-f25d-4eda-9643-6b63215c41ff" containerName="memcached" probeResult="failure" output="dial tcp 10.217.1.217:11211: connect: connection refused" Dec 03 12:27:14 crc kubenswrapper[5052]: I1203 12:27:14.412339 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/barbican-worker-64d9f4d555-4v5kn" podStartSLOduration=4.412325361 podStartE2EDuration="4.412325361s" podCreationTimestamp="2025-12-03 12:27:10 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:27:14.408944045 +0000 UTC m=+4024.997185809" watchObservedRunningTime="2025-12-03 12:27:14.412325361 +0000 UTC m=+4025.000567115" Dec 03 12:27:14 crc kubenswrapper[5052]: I1203 12:27:14.462807 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-worker-75f6bbd957-c444v"] Dec 03 12:27:14 crc kubenswrapper[5052]: I1203 12:27:14.463087 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/barbican-worker-75f6bbd957-c444v" podUID="1cc56b2c-23cc-452f-8de3-d7e8bd2e1e68" containerName="barbican-worker-log" containerID="cri-o://cf30dbd3717dce8af354c25d39edb466422569ee896210a1fcd9f69012eca27c" gracePeriod=30 Dec 03 12:27:14 crc kubenswrapper[5052]: I1203 12:27:14.463578 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/barbican-worker-75f6bbd957-c444v" podUID="1cc56b2c-23cc-452f-8de3-d7e8bd2e1e68" containerName="barbican-worker" containerID="cri-o://a5ca1b03848c3e7731d3e96fd0f8faa937c1665505435b783c4113d6afa25f56" gracePeriod=30 Dec 03 12:27:14 crc kubenswrapper[5052]: I1203 12:27:14.464176 5052 generic.go:334] "Generic (PLEG): container finished" podID="6b96dc78-b389-4764-8ba7-6a4c5e41e345" containerID="481bbc2057a57984fb7f4ba0b572e86b7319fb1c2d5b5c1d8d86cafb6e88f80f" exitCode=143 Dec 03 12:27:14 crc kubenswrapper[5052]: I1203 12:27:14.464232 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"6b96dc78-b389-4764-8ba7-6a4c5e41e345","Type":"ContainerDied","Data":"481bbc2057a57984fb7f4ba0b572e86b7319fb1c2d5b5c1d8d86cafb6e88f80f"} Dec 03 12:27:14 crc kubenswrapper[5052]: I1203 12:27:14.472167 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-78665465bd-9q8ff" event={"ID":"2825bd7d-50f6-4d2d-827a-e0d307d1b6de","Type":"ContainerStarted","Data":"a389a539341125b103495868581a628b481d42547aa3cdd10558662eabdfce98"} Dec 03 12:27:14 crc kubenswrapper[5052]: I1203 12:27:14.472583 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-78665465bd-9q8ff" event={"ID":"2825bd7d-50f6-4d2d-827a-e0d307d1b6de","Type":"ContainerStarted","Data":"e95f312b10938bf3db0b2eb1850f8a9d9b0a2d38be69c7c0dbd457adee9cce11"} Dec 03 12:27:14 crc kubenswrapper[5052]: I1203 12:27:14.472668 5052 scope.go:117] "RemoveContainer" containerID="10cf3584ca3062418591f9016c0b57483870a9df1de7e1394ee2ccefe62d4933" Dec 03 12:27:14 crc kubenswrapper[5052]: E1203 12:27:14.477027 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"10cf3584ca3062418591f9016c0b57483870a9df1de7e1394ee2ccefe62d4933\": container with ID starting with 10cf3584ca3062418591f9016c0b57483870a9df1de7e1394ee2ccefe62d4933 not found: ID does not exist" containerID="10cf3584ca3062418591f9016c0b57483870a9df1de7e1394ee2ccefe62d4933" Dec 03 12:27:14 crc kubenswrapper[5052]: I1203 12:27:14.477079 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"10cf3584ca3062418591f9016c0b57483870a9df1de7e1394ee2ccefe62d4933"} err="failed to get container status \"10cf3584ca3062418591f9016c0b57483870a9df1de7e1394ee2ccefe62d4933\": rpc error: code = NotFound desc = could not find container \"10cf3584ca3062418591f9016c0b57483870a9df1de7e1394ee2ccefe62d4933\": container with ID starting with 10cf3584ca3062418591f9016c0b57483870a9df1de7e1394ee2ccefe62d4933 not found: ID does not exist" Dec 03 12:27:14 crc kubenswrapper[5052]: I1203 12:27:14.497294 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-544d7dcfcd-7xmk7" event={"ID":"135a97dd-af49-4eb8-8224-c3090757ed2a","Type":"ContainerStarted","Data":"47023cf3d975406d17e04a8c3b72230adbcd30f7306a30b2f6867cc8b0f83bdc"} Dec 03 12:27:14 crc kubenswrapper[5052]: I1203 12:27:14.556764 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" podUID="5a5320b7-0541-483a-8e0b-78c10d0f8efa" containerName="rabbitmq" containerID="cri-o://032bc0ff1bc0c7adb6371d41b7d1d30531bb08512a12516306ce34a00afcaad9" gracePeriod=604797 Dec 03 12:27:14 crc kubenswrapper[5052]: I1203 12:27:14.579294 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 12:27:14 crc kubenswrapper[5052]: I1203 12:27:14.612320 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 12:27:14 crc kubenswrapper[5052]: I1203 12:27:14.648000 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 12:27:14 crc kubenswrapper[5052]: E1203 12:27:14.648531 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="04aa1d5d-e683-417e-a96d-ce9af5a96136" containerName="nova-cell1-novncproxy-novncproxy" Dec 03 12:27:14 crc kubenswrapper[5052]: I1203 12:27:14.648555 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="04aa1d5d-e683-417e-a96d-ce9af5a96136" containerName="nova-cell1-novncproxy-novncproxy" Dec 03 12:27:14 crc kubenswrapper[5052]: I1203 12:27:14.648855 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="04aa1d5d-e683-417e-a96d-ce9af5a96136" containerName="nova-cell1-novncproxy-novncproxy" Dec 03 12:27:14 crc kubenswrapper[5052]: I1203 12:27:14.649806 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:27:14 crc kubenswrapper[5052]: I1203 12:27:14.654934 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell1-novncproxy-config-data" Dec 03 12:27:14 crc kubenswrapper[5052]: I1203 12:27:14.674497 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 12:27:14 crc kubenswrapper[5052]: I1203 12:27:14.761757 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-797ccd5f76-v8cmg"] Dec 03 12:27:14 crc kubenswrapper[5052]: I1203 12:27:14.791429 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/460b2796-2ea1-4ddd-a4e7-eae181ddfc81-config-data\") pod \"nova-cell1-novncproxy-0\" (UID: \"460b2796-2ea1-4ddd-a4e7-eae181ddfc81\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:27:14 crc kubenswrapper[5052]: I1203 12:27:14.791597 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-899hm\" (UniqueName: \"kubernetes.io/projected/460b2796-2ea1-4ddd-a4e7-eae181ddfc81-kube-api-access-899hm\") pod \"nova-cell1-novncproxy-0\" (UID: \"460b2796-2ea1-4ddd-a4e7-eae181ddfc81\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:27:14 crc kubenswrapper[5052]: I1203 12:27:14.791644 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/460b2796-2ea1-4ddd-a4e7-eae181ddfc81-combined-ca-bundle\") pod \"nova-cell1-novncproxy-0\" (UID: \"460b2796-2ea1-4ddd-a4e7-eae181ddfc81\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:27:14 crc kubenswrapper[5052]: I1203 12:27:14.896144 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-899hm\" (UniqueName: \"kubernetes.io/projected/460b2796-2ea1-4ddd-a4e7-eae181ddfc81-kube-api-access-899hm\") pod \"nova-cell1-novncproxy-0\" (UID: \"460b2796-2ea1-4ddd-a4e7-eae181ddfc81\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:27:14 crc kubenswrapper[5052]: I1203 12:27:14.896462 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/460b2796-2ea1-4ddd-a4e7-eae181ddfc81-combined-ca-bundle\") pod \"nova-cell1-novncproxy-0\" (UID: \"460b2796-2ea1-4ddd-a4e7-eae181ddfc81\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:27:14 crc kubenswrapper[5052]: I1203 12:27:14.896555 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/460b2796-2ea1-4ddd-a4e7-eae181ddfc81-config-data\") pod \"nova-cell1-novncproxy-0\" (UID: \"460b2796-2ea1-4ddd-a4e7-eae181ddfc81\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:27:14 crc kubenswrapper[5052]: I1203 12:27:14.909355 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/460b2796-2ea1-4ddd-a4e7-eae181ddfc81-config-data\") pod \"nova-cell1-novncproxy-0\" (UID: \"460b2796-2ea1-4ddd-a4e7-eae181ddfc81\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:27:14 crc kubenswrapper[5052]: I1203 12:27:14.923962 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-899hm\" (UniqueName: \"kubernetes.io/projected/460b2796-2ea1-4ddd-a4e7-eae181ddfc81-kube-api-access-899hm\") pod \"nova-cell1-novncproxy-0\" (UID: \"460b2796-2ea1-4ddd-a4e7-eae181ddfc81\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:27:14 crc kubenswrapper[5052]: I1203 12:27:14.923968 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/460b2796-2ea1-4ddd-a4e7-eae181ddfc81-combined-ca-bundle\") pod \"nova-cell1-novncproxy-0\" (UID: \"460b2796-2ea1-4ddd-a4e7-eae181ddfc81\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:27:15 crc kubenswrapper[5052]: I1203 12:27:15.010186 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:27:15 crc kubenswrapper[5052]: I1203 12:27:15.255244 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/keystone-bootstrap-vs67d"] Dec 03 12:27:15 crc kubenswrapper[5052]: I1203 12:27:15.259074 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/memcached-0" Dec 03 12:27:15 crc kubenswrapper[5052]: I1203 12:27:15.305329 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/keystone-bootstrap-vs67d"] Dec 03 12:27:16 crc kubenswrapper[5052]: I1203 12:27:15.411055 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mrkc5\" (UniqueName: \"kubernetes.io/projected/65fc3838-f25d-4eda-9643-6b63215c41ff-kube-api-access-mrkc5\") pod \"65fc3838-f25d-4eda-9643-6b63215c41ff\" (UID: \"65fc3838-f25d-4eda-9643-6b63215c41ff\") " Dec 03 12:27:16 crc kubenswrapper[5052]: I1203 12:27:15.411198 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/65fc3838-f25d-4eda-9643-6b63215c41ff-config-data\") pod \"65fc3838-f25d-4eda-9643-6b63215c41ff\" (UID: \"65fc3838-f25d-4eda-9643-6b63215c41ff\") " Dec 03 12:27:16 crc kubenswrapper[5052]: I1203 12:27:15.411242 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/65fc3838-f25d-4eda-9643-6b63215c41ff-kolla-config\") pod \"65fc3838-f25d-4eda-9643-6b63215c41ff\" (UID: \"65fc3838-f25d-4eda-9643-6b63215c41ff\") " Dec 03 12:27:16 crc kubenswrapper[5052]: I1203 12:27:15.424585 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-64d6d88f66-7z998"] Dec 03 12:27:16 crc kubenswrapper[5052]: I1203 12:27:15.432707 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/65fc3838-f25d-4eda-9643-6b63215c41ff-kolla-config" (OuterVolumeSpecName: "kolla-config") pod "65fc3838-f25d-4eda-9643-6b63215c41ff" (UID: "65fc3838-f25d-4eda-9643-6b63215c41ff"). InnerVolumeSpecName "kolla-config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:27:16 crc kubenswrapper[5052]: I1203 12:27:15.440350 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/barbican-keystone-listener-58b6cdf6bb-5xxd8"] Dec 03 12:27:16 crc kubenswrapper[5052]: E1203 12:27:15.440775 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="65fc3838-f25d-4eda-9643-6b63215c41ff" containerName="memcached" Dec 03 12:27:16 crc kubenswrapper[5052]: I1203 12:27:15.440786 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="65fc3838-f25d-4eda-9643-6b63215c41ff" containerName="memcached" Dec 03 12:27:16 crc kubenswrapper[5052]: I1203 12:27:15.441003 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="65fc3838-f25d-4eda-9643-6b63215c41ff" containerName="memcached" Dec 03 12:27:16 crc kubenswrapper[5052]: I1203 12:27:15.442547 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/65fc3838-f25d-4eda-9643-6b63215c41ff-config-data" (OuterVolumeSpecName: "config-data") pod "65fc3838-f25d-4eda-9643-6b63215c41ff" (UID: "65fc3838-f25d-4eda-9643-6b63215c41ff"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:27:16 crc kubenswrapper[5052]: I1203 12:27:15.444787 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-keystone-listener-58b6cdf6bb-5xxd8" Dec 03 12:27:16 crc kubenswrapper[5052]: I1203 12:27:15.446229 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/65fc3838-f25d-4eda-9643-6b63215c41ff-kube-api-access-mrkc5" (OuterVolumeSpecName: "kube-api-access-mrkc5") pod "65fc3838-f25d-4eda-9643-6b63215c41ff" (UID: "65fc3838-f25d-4eda-9643-6b63215c41ff"). InnerVolumeSpecName "kube-api-access-mrkc5". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:27:16 crc kubenswrapper[5052]: I1203 12:27:15.503569 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/barbican-worker-679bfdf7b7-tg6h7"] Dec 03 12:27:16 crc kubenswrapper[5052]: I1203 12:27:15.505527 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-worker-679bfdf7b7-tg6h7" Dec 03 12:27:16 crc kubenswrapper[5052]: I1203 12:27:15.515767 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/71df5bf1-7fd8-4a4d-b0f5-85e3911ae8a9-config-data-custom\") pod \"barbican-keystone-listener-58b6cdf6bb-5xxd8\" (UID: \"71df5bf1-7fd8-4a4d-b0f5-85e3911ae8a9\") " pod="openstack-kuttl-tests/barbican-keystone-listener-58b6cdf6bb-5xxd8" Dec 03 12:27:16 crc kubenswrapper[5052]: I1203 12:27:15.515913 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/71df5bf1-7fd8-4a4d-b0f5-85e3911ae8a9-logs\") pod \"barbican-keystone-listener-58b6cdf6bb-5xxd8\" (UID: \"71df5bf1-7fd8-4a4d-b0f5-85e3911ae8a9\") " pod="openstack-kuttl-tests/barbican-keystone-listener-58b6cdf6bb-5xxd8" Dec 03 12:27:16 crc kubenswrapper[5052]: I1203 12:27:15.515954 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vr9rf\" (UniqueName: \"kubernetes.io/projected/71df5bf1-7fd8-4a4d-b0f5-85e3911ae8a9-kube-api-access-vr9rf\") pod \"barbican-keystone-listener-58b6cdf6bb-5xxd8\" (UID: \"71df5bf1-7fd8-4a4d-b0f5-85e3911ae8a9\") " pod="openstack-kuttl-tests/barbican-keystone-listener-58b6cdf6bb-5xxd8" Dec 03 12:27:16 crc kubenswrapper[5052]: I1203 12:27:15.516035 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/71df5bf1-7fd8-4a4d-b0f5-85e3911ae8a9-config-data\") pod \"barbican-keystone-listener-58b6cdf6bb-5xxd8\" (UID: \"71df5bf1-7fd8-4a4d-b0f5-85e3911ae8a9\") " pod="openstack-kuttl-tests/barbican-keystone-listener-58b6cdf6bb-5xxd8" Dec 03 12:27:16 crc kubenswrapper[5052]: I1203 12:27:15.516063 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/71df5bf1-7fd8-4a4d-b0f5-85e3911ae8a9-combined-ca-bundle\") pod \"barbican-keystone-listener-58b6cdf6bb-5xxd8\" (UID: \"71df5bf1-7fd8-4a4d-b0f5-85e3911ae8a9\") " pod="openstack-kuttl-tests/barbican-keystone-listener-58b6cdf6bb-5xxd8" Dec 03 12:27:16 crc kubenswrapper[5052]: I1203 12:27:15.516132 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-mrkc5\" (UniqueName: \"kubernetes.io/projected/65fc3838-f25d-4eda-9643-6b63215c41ff-kube-api-access-mrkc5\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:16 crc kubenswrapper[5052]: I1203 12:27:15.516147 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/65fc3838-f25d-4eda-9643-6b63215c41ff-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:16 crc kubenswrapper[5052]: I1203 12:27:15.516160 5052 reconciler_common.go:293] "Volume detached for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/65fc3838-f25d-4eda-9643-6b63215c41ff-kolla-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:16 crc kubenswrapper[5052]: I1203 12:27:15.623546 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-keystone-listener-58b6cdf6bb-5xxd8"] Dec 03 12:27:16 crc kubenswrapper[5052]: I1203 12:27:15.625939 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/72a32b4d-d61d-43a5-a45d-c1fd4bad672d-combined-ca-bundle\") pod \"barbican-worker-679bfdf7b7-tg6h7\" (UID: \"72a32b4d-d61d-43a5-a45d-c1fd4bad672d\") " pod="openstack-kuttl-tests/barbican-worker-679bfdf7b7-tg6h7" Dec 03 12:27:16 crc kubenswrapper[5052]: I1203 12:27:15.626046 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/71df5bf1-7fd8-4a4d-b0f5-85e3911ae8a9-config-data-custom\") pod \"barbican-keystone-listener-58b6cdf6bb-5xxd8\" (UID: \"71df5bf1-7fd8-4a4d-b0f5-85e3911ae8a9\") " pod="openstack-kuttl-tests/barbican-keystone-listener-58b6cdf6bb-5xxd8" Dec 03 12:27:16 crc kubenswrapper[5052]: I1203 12:27:15.626078 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/72a32b4d-d61d-43a5-a45d-c1fd4bad672d-config-data\") pod \"barbican-worker-679bfdf7b7-tg6h7\" (UID: \"72a32b4d-d61d-43a5-a45d-c1fd4bad672d\") " pod="openstack-kuttl-tests/barbican-worker-679bfdf7b7-tg6h7" Dec 03 12:27:16 crc kubenswrapper[5052]: I1203 12:27:15.626218 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/71df5bf1-7fd8-4a4d-b0f5-85e3911ae8a9-logs\") pod \"barbican-keystone-listener-58b6cdf6bb-5xxd8\" (UID: \"71df5bf1-7fd8-4a4d-b0f5-85e3911ae8a9\") " pod="openstack-kuttl-tests/barbican-keystone-listener-58b6cdf6bb-5xxd8" Dec 03 12:27:16 crc kubenswrapper[5052]: I1203 12:27:15.626267 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-vr9rf\" (UniqueName: \"kubernetes.io/projected/71df5bf1-7fd8-4a4d-b0f5-85e3911ae8a9-kube-api-access-vr9rf\") pod \"barbican-keystone-listener-58b6cdf6bb-5xxd8\" (UID: \"71df5bf1-7fd8-4a4d-b0f5-85e3911ae8a9\") " pod="openstack-kuttl-tests/barbican-keystone-listener-58b6cdf6bb-5xxd8" Dec 03 12:27:16 crc kubenswrapper[5052]: I1203 12:27:15.626321 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/72a32b4d-d61d-43a5-a45d-c1fd4bad672d-config-data-custom\") pod \"barbican-worker-679bfdf7b7-tg6h7\" (UID: \"72a32b4d-d61d-43a5-a45d-c1fd4bad672d\") " pod="openstack-kuttl-tests/barbican-worker-679bfdf7b7-tg6h7" Dec 03 12:27:16 crc kubenswrapper[5052]: I1203 12:27:15.626401 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/72a32b4d-d61d-43a5-a45d-c1fd4bad672d-logs\") pod \"barbican-worker-679bfdf7b7-tg6h7\" (UID: \"72a32b4d-d61d-43a5-a45d-c1fd4bad672d\") " pod="openstack-kuttl-tests/barbican-worker-679bfdf7b7-tg6h7" Dec 03 12:27:16 crc kubenswrapper[5052]: I1203 12:27:15.626483 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/71df5bf1-7fd8-4a4d-b0f5-85e3911ae8a9-config-data\") pod \"barbican-keystone-listener-58b6cdf6bb-5xxd8\" (UID: \"71df5bf1-7fd8-4a4d-b0f5-85e3911ae8a9\") " pod="openstack-kuttl-tests/barbican-keystone-listener-58b6cdf6bb-5xxd8" Dec 03 12:27:16 crc kubenswrapper[5052]: I1203 12:27:15.626545 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/71df5bf1-7fd8-4a4d-b0f5-85e3911ae8a9-combined-ca-bundle\") pod \"barbican-keystone-listener-58b6cdf6bb-5xxd8\" (UID: \"71df5bf1-7fd8-4a4d-b0f5-85e3911ae8a9\") " pod="openstack-kuttl-tests/barbican-keystone-listener-58b6cdf6bb-5xxd8" Dec 03 12:27:16 crc kubenswrapper[5052]: I1203 12:27:15.626565 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dg6kv\" (UniqueName: \"kubernetes.io/projected/72a32b4d-d61d-43a5-a45d-c1fd4bad672d-kube-api-access-dg6kv\") pod \"barbican-worker-679bfdf7b7-tg6h7\" (UID: \"72a32b4d-d61d-43a5-a45d-c1fd4bad672d\") " pod="openstack-kuttl-tests/barbican-worker-679bfdf7b7-tg6h7" Dec 03 12:27:16 crc kubenswrapper[5052]: I1203 12:27:15.627177 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/71df5bf1-7fd8-4a4d-b0f5-85e3911ae8a9-logs\") pod \"barbican-keystone-listener-58b6cdf6bb-5xxd8\" (UID: \"71df5bf1-7fd8-4a4d-b0f5-85e3911ae8a9\") " pod="openstack-kuttl-tests/barbican-keystone-listener-58b6cdf6bb-5xxd8" Dec 03 12:27:16 crc kubenswrapper[5052]: I1203 12:27:15.643809 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/71df5bf1-7fd8-4a4d-b0f5-85e3911ae8a9-config-data-custom\") pod \"barbican-keystone-listener-58b6cdf6bb-5xxd8\" (UID: \"71df5bf1-7fd8-4a4d-b0f5-85e3911ae8a9\") " pod="openstack-kuttl-tests/barbican-keystone-listener-58b6cdf6bb-5xxd8" Dec 03 12:27:16 crc kubenswrapper[5052]: I1203 12:27:15.644208 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-797ccd5f76-v8cmg" event={"ID":"5314ef6e-9ca6-4c9b-98b0-907b5a6e423c","Type":"ContainerStarted","Data":"8e4750b9efd2c05e5ba67059b1ff61da310da2de12da9f735d469317ccd1840f"} Dec 03 12:27:16 crc kubenswrapper[5052]: I1203 12:27:15.655632 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/71df5bf1-7fd8-4a4d-b0f5-85e3911ae8a9-config-data\") pod \"barbican-keystone-listener-58b6cdf6bb-5xxd8\" (UID: \"71df5bf1-7fd8-4a4d-b0f5-85e3911ae8a9\") " pod="openstack-kuttl-tests/barbican-keystone-listener-58b6cdf6bb-5xxd8" Dec 03 12:27:16 crc kubenswrapper[5052]: I1203 12:27:15.666294 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/71df5bf1-7fd8-4a4d-b0f5-85e3911ae8a9-combined-ca-bundle\") pod \"barbican-keystone-listener-58b6cdf6bb-5xxd8\" (UID: \"71df5bf1-7fd8-4a4d-b0f5-85e3911ae8a9\") " pod="openstack-kuttl-tests/barbican-keystone-listener-58b6cdf6bb-5xxd8" Dec 03 12:27:16 crc kubenswrapper[5052]: I1203 12:27:15.678948 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/neutron-544d7dcfcd-7xmk7" podUID="135a97dd-af49-4eb8-8224-c3090757ed2a" containerName="neutron-api" containerID="cri-o://47023cf3d975406d17e04a8c3b72230adbcd30f7306a30b2f6867cc8b0f83bdc" gracePeriod=30 Dec 03 12:27:16 crc kubenswrapper[5052]: I1203 12:27:15.679236 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/neutron-544d7dcfcd-7xmk7" podUID="135a97dd-af49-4eb8-8224-c3090757ed2a" containerName="neutron-httpd" containerID="cri-o://1ee73b25fb176fb0f7a5ba787284992b6e0c4d72b574d2a31c1df2ab8746440f" gracePeriod=30 Dec 03 12:27:16 crc kubenswrapper[5052]: I1203 12:27:15.722809 5052 generic.go:334] "Generic (PLEG): container finished" podID="1cc56b2c-23cc-452f-8de3-d7e8bd2e1e68" containerID="cf30dbd3717dce8af354c25d39edb466422569ee896210a1fcd9f69012eca27c" exitCode=143 Dec 03 12:27:16 crc kubenswrapper[5052]: I1203 12:27:15.726656 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-vr9rf\" (UniqueName: \"kubernetes.io/projected/71df5bf1-7fd8-4a4d-b0f5-85e3911ae8a9-kube-api-access-vr9rf\") pod \"barbican-keystone-listener-58b6cdf6bb-5xxd8\" (UID: \"71df5bf1-7fd8-4a4d-b0f5-85e3911ae8a9\") " pod="openstack-kuttl-tests/barbican-keystone-listener-58b6cdf6bb-5xxd8" Dec 03 12:27:16 crc kubenswrapper[5052]: I1203 12:27:15.728663 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/72a32b4d-d61d-43a5-a45d-c1fd4bad672d-config-data-custom\") pod \"barbican-worker-679bfdf7b7-tg6h7\" (UID: \"72a32b4d-d61d-43a5-a45d-c1fd4bad672d\") " pod="openstack-kuttl-tests/barbican-worker-679bfdf7b7-tg6h7" Dec 03 12:27:16 crc kubenswrapper[5052]: I1203 12:27:15.728714 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/72a32b4d-d61d-43a5-a45d-c1fd4bad672d-logs\") pod \"barbican-worker-679bfdf7b7-tg6h7\" (UID: \"72a32b4d-d61d-43a5-a45d-c1fd4bad672d\") " pod="openstack-kuttl-tests/barbican-worker-679bfdf7b7-tg6h7" Dec 03 12:27:16 crc kubenswrapper[5052]: I1203 12:27:15.728753 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-dg6kv\" (UniqueName: \"kubernetes.io/projected/72a32b4d-d61d-43a5-a45d-c1fd4bad672d-kube-api-access-dg6kv\") pod \"barbican-worker-679bfdf7b7-tg6h7\" (UID: \"72a32b4d-d61d-43a5-a45d-c1fd4bad672d\") " pod="openstack-kuttl-tests/barbican-worker-679bfdf7b7-tg6h7" Dec 03 12:27:16 crc kubenswrapper[5052]: I1203 12:27:15.728811 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/72a32b4d-d61d-43a5-a45d-c1fd4bad672d-combined-ca-bundle\") pod \"barbican-worker-679bfdf7b7-tg6h7\" (UID: \"72a32b4d-d61d-43a5-a45d-c1fd4bad672d\") " pod="openstack-kuttl-tests/barbican-worker-679bfdf7b7-tg6h7" Dec 03 12:27:16 crc kubenswrapper[5052]: I1203 12:27:15.728857 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/72a32b4d-d61d-43a5-a45d-c1fd4bad672d-config-data\") pod \"barbican-worker-679bfdf7b7-tg6h7\" (UID: \"72a32b4d-d61d-43a5-a45d-c1fd4bad672d\") " pod="openstack-kuttl-tests/barbican-worker-679bfdf7b7-tg6h7" Dec 03 12:27:16 crc kubenswrapper[5052]: I1203 12:27:15.732019 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/72a32b4d-d61d-43a5-a45d-c1fd4bad672d-logs\") pod \"barbican-worker-679bfdf7b7-tg6h7\" (UID: \"72a32b4d-d61d-43a5-a45d-c1fd4bad672d\") " pod="openstack-kuttl-tests/barbican-worker-679bfdf7b7-tg6h7" Dec 03 12:27:16 crc kubenswrapper[5052]: I1203 12:27:15.742360 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/72a32b4d-d61d-43a5-a45d-c1fd4bad672d-config-data\") pod \"barbican-worker-679bfdf7b7-tg6h7\" (UID: \"72a32b4d-d61d-43a5-a45d-c1fd4bad672d\") " pod="openstack-kuttl-tests/barbican-worker-679bfdf7b7-tg6h7" Dec 03 12:27:16 crc kubenswrapper[5052]: I1203 12:27:15.742944 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/72a32b4d-d61d-43a5-a45d-c1fd4bad672d-config-data-custom\") pod \"barbican-worker-679bfdf7b7-tg6h7\" (UID: \"72a32b4d-d61d-43a5-a45d-c1fd4bad672d\") " pod="openstack-kuttl-tests/barbican-worker-679bfdf7b7-tg6h7" Dec 03 12:27:16 crc kubenswrapper[5052]: I1203 12:27:15.750143 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/72a32b4d-d61d-43a5-a45d-c1fd4bad672d-combined-ca-bundle\") pod \"barbican-worker-679bfdf7b7-tg6h7\" (UID: \"72a32b4d-d61d-43a5-a45d-c1fd4bad672d\") " pod="openstack-kuttl-tests/barbican-worker-679bfdf7b7-tg6h7" Dec 03 12:27:16 crc kubenswrapper[5052]: I1203 12:27:15.755081 5052 generic.go:334] "Generic (PLEG): container finished" podID="98d0abd5-a20b-45e3-aa30-34f038b2fb3d" containerID="06e057704675bafd6f98189c6b86c43903be3fc1372671a1851fba5082cffb21" exitCode=143 Dec 03 12:27:16 crc kubenswrapper[5052]: I1203 12:27:15.768922 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-dg6kv\" (UniqueName: \"kubernetes.io/projected/72a32b4d-d61d-43a5-a45d-c1fd4bad672d-kube-api-access-dg6kv\") pod \"barbican-worker-679bfdf7b7-tg6h7\" (UID: \"72a32b4d-d61d-43a5-a45d-c1fd4bad672d\") " pod="openstack-kuttl-tests/barbican-worker-679bfdf7b7-tg6h7" Dec 03 12:27:16 crc kubenswrapper[5052]: I1203 12:27:15.850362 5052 generic.go:334] "Generic (PLEG): container finished" podID="7f163259-845f-4519-bd77-fa66e760d4db" containerID="7f52642c73f37ab9180012c54a5c7e03d2db022e02528a07f0120cdace4c30ed" exitCode=0 Dec 03 12:27:16 crc kubenswrapper[5052]: I1203 12:27:15.872127 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/neutron-544d7dcfcd-7xmk7" podStartSLOduration=4.872105667 podStartE2EDuration="4.872105667s" podCreationTimestamp="2025-12-03 12:27:11 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:27:15.713104759 +0000 UTC m=+4026.301346523" watchObservedRunningTime="2025-12-03 12:27:15.872105667 +0000 UTC m=+4026.460347431" Dec 03 12:27:16 crc kubenswrapper[5052]: I1203 12:27:15.885102 5052 generic.go:334] "Generic (PLEG): container finished" podID="65fc3838-f25d-4eda-9643-6b63215c41ff" containerID="0931db22f5fb013cdcb55311764fa795389b7087d437f3d1b8323540414014d9" exitCode=0 Dec 03 12:27:16 crc kubenswrapper[5052]: I1203 12:27:15.885270 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/memcached-0" Dec 03 12:27:16 crc kubenswrapper[5052]: I1203 12:27:15.898607 5052 generic.go:334] "Generic (PLEG): container finished" podID="a823b6ba-b2af-4deb-88d2-0902fd6e3780" containerID="53d7dc20a3dd7792e212463451921763c0652bd6c02e21fa9b1e85765b8915ca" exitCode=0 Dec 03 12:27:16 crc kubenswrapper[5052]: I1203 12:27:15.900102 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/keystone-66778fc6-56rjc" podUID="aa94b30e-0028-45a0-88aa-b5db5fc2f8fd" containerName="keystone-api" containerID="cri-o://22275b3672b8b4add52b127bbe276c42c2a5e8d7f524d0cf5e1961f6aee9186c" gracePeriod=30 Dec 03 12:27:16 crc kubenswrapper[5052]: I1203 12:27:15.916109 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/barbican-api-78665465bd-9q8ff" podStartSLOduration=5.916080292 podStartE2EDuration="5.916080292s" podCreationTimestamp="2025-12-03 12:27:10 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:27:15.893660662 +0000 UTC m=+4026.481902436" watchObservedRunningTime="2025-12-03 12:27:15.916080292 +0000 UTC m=+4026.504322056" Dec 03 12:27:16 crc kubenswrapper[5052]: E1203 12:27:16.127595 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of 74ef6e1fe61b1c5ce0d3bf482bd7a2e9edd6da7a23654c82cd5a4d8527f480c3 is running failed: container process not found" containerID="74ef6e1fe61b1c5ce0d3bf482bd7a2e9edd6da7a23654c82cd5a4d8527f480c3" cmd=["/usr/bin/pgrep","-r","DRST","nova-scheduler"] Dec 03 12:27:16 crc kubenswrapper[5052]: E1203 12:27:16.130737 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of 74ef6e1fe61b1c5ce0d3bf482bd7a2e9edd6da7a23654c82cd5a4d8527f480c3 is running failed: container process not found" containerID="74ef6e1fe61b1c5ce0d3bf482bd7a2e9edd6da7a23654c82cd5a4d8527f480c3" cmd=["/usr/bin/pgrep","-r","DRST","nova-scheduler"] Dec 03 12:27:16 crc kubenswrapper[5052]: E1203 12:27:16.142542 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of 74ef6e1fe61b1c5ce0d3bf482bd7a2e9edd6da7a23654c82cd5a4d8527f480c3 is running failed: container process not found" containerID="74ef6e1fe61b1c5ce0d3bf482bd7a2e9edd6da7a23654c82cd5a4d8527f480c3" cmd=["/usr/bin/pgrep","-r","DRST","nova-scheduler"] Dec 03 12:27:16 crc kubenswrapper[5052]: E1203 12:27:16.142613 5052 prober.go:104] "Probe errored" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of 74ef6e1fe61b1c5ce0d3bf482bd7a2e9edd6da7a23654c82cd5a4d8527f480c3 is running failed: container process not found" probeType="Readiness" pod="openstack-kuttl-tests/nova-scheduler-0" podUID="7ef14abf-76c0-4db5-9291-e0f9675425d1" containerName="nova-scheduler-scheduler" Dec 03 12:27:16 crc kubenswrapper[5052]: I1203 12:27:16.183210 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="04aa1d5d-e683-417e-a96d-ce9af5a96136" path="/var/lib/kubelet/pods/04aa1d5d-e683-417e-a96d-ce9af5a96136/volumes" Dec 03 12:27:16 crc kubenswrapper[5052]: I1203 12:27:16.183942 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fcc35227-2ba8-4fc9-83fb-b9e40ccd8b32" path="/var/lib/kubelet/pods/fcc35227-2ba8-4fc9-83fb-b9e40ccd8b32/volumes" Dec 03 12:27:16 crc kubenswrapper[5052]: I1203 12:27:16.184926 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-544d7dcfcd-7xmk7" event={"ID":"135a97dd-af49-4eb8-8224-c3090757ed2a","Type":"ContainerStarted","Data":"1ee73b25fb176fb0f7a5ba787284992b6e0c4d72b574d2a31c1df2ab8746440f"} Dec 03 12:27:16 crc kubenswrapper[5052]: I1203 12:27:16.184954 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-worker-75f6bbd957-c444v" event={"ID":"1cc56b2c-23cc-452f-8de3-d7e8bd2e1e68","Type":"ContainerDied","Data":"cf30dbd3717dce8af354c25d39edb466422569ee896210a1fcd9f69012eca27c"} Dec 03 12:27:16 crc kubenswrapper[5052]: I1203 12:27:16.184976 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-worker-679bfdf7b7-tg6h7"] Dec 03 12:27:16 crc kubenswrapper[5052]: I1203 12:27:16.184994 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-keystone-listener-7c58f6698b-95l8v" event={"ID":"98d0abd5-a20b-45e3-aa30-34f038b2fb3d","Type":"ContainerDied","Data":"06e057704675bafd6f98189c6b86c43903be3fc1372671a1851fba5082cffb21"} Dec 03 12:27:16 crc kubenswrapper[5052]: I1203 12:27:16.185014 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/neutron-544d7dcfcd-7xmk7" Dec 03 12:27:16 crc kubenswrapper[5052]: I1203 12:27:16.185028 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/barbican-api-78665465bd-9q8ff" Dec 03 12:27:16 crc kubenswrapper[5052]: I1203 12:27:16.185040 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/barbican-api-78665465bd-9q8ff" Dec 03 12:27:16 crc kubenswrapper[5052]: I1203 12:27:16.185050 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"7f163259-845f-4519-bd77-fa66e760d4db","Type":"ContainerDied","Data":"7f52642c73f37ab9180012c54a5c7e03d2db022e02528a07f0120cdace4c30ed"} Dec 03 12:27:16 crc kubenswrapper[5052]: I1203 12:27:16.185067 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/neutron-797ccd5f76-v8cmg"] Dec 03 12:27:16 crc kubenswrapper[5052]: I1203 12:27:16.185084 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-78665465bd-9q8ff" event={"ID":"2825bd7d-50f6-4d2d-827a-e0d307d1b6de","Type":"ContainerStarted","Data":"d5dba2f0d9dfdc433b2210944bb2049a3f79b8b7f7b0c6ac0516d79962651461"} Dec 03 12:27:16 crc kubenswrapper[5052]: I1203 12:27:16.185099 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-api-78665465bd-9q8ff"] Dec 03 12:27:16 crc kubenswrapper[5052]: I1203 12:27:16.185115 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/neutron-6bf55bd9d6-mrgs9"] Dec 03 12:27:16 crc kubenswrapper[5052]: I1203 12:27:16.187433 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/memcached-0" event={"ID":"65fc3838-f25d-4eda-9643-6b63215c41ff","Type":"ContainerDied","Data":"0931db22f5fb013cdcb55311764fa795389b7087d437f3d1b8323540414014d9"} Dec 03 12:27:16 crc kubenswrapper[5052]: I1203 12:27:16.187460 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/memcached-0" event={"ID":"65fc3838-f25d-4eda-9643-6b63215c41ff","Type":"ContainerDied","Data":"f7f54f3df6f8da495e9edeb2c7a00827bacd10261cf84a185e76caadd0efb178"} Dec 03 12:27:16 crc kubenswrapper[5052]: I1203 12:27:16.187481 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-6bf55bd9d6-mrgs9"] Dec 03 12:27:16 crc kubenswrapper[5052]: I1203 12:27:16.187496 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/placement-6cdf7689bb-fscpc"] Dec 03 12:27:16 crc kubenswrapper[5052]: I1203 12:27:16.187640 5052 scope.go:117] "RemoveContainer" containerID="0931db22f5fb013cdcb55311764fa795389b7087d437f3d1b8323540414014d9" Dec 03 12:27:16 crc kubenswrapper[5052]: I1203 12:27:16.187697 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-6bf55bd9d6-mrgs9" Dec 03 12:27:16 crc kubenswrapper[5052]: I1203 12:27:16.191207 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/placement-6cdf7689bb-fscpc"] Dec 03 12:27:16 crc kubenswrapper[5052]: I1203 12:27:16.191240 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/barbican-api-5d855df9cb-xdg6p"] Dec 03 12:27:16 crc kubenswrapper[5052]: I1203 12:27:16.191578 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-6cdf7689bb-fscpc" Dec 03 12:27:16 crc kubenswrapper[5052]: I1203 12:27:16.193963 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"a823b6ba-b2af-4deb-88d2-0902fd6e3780","Type":"ContainerDied","Data":"53d7dc20a3dd7792e212463451921763c0652bd6c02e21fa9b1e85765b8915ca"} Dec 03 12:27:16 crc kubenswrapper[5052]: I1203 12:27:16.194062 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-api-5d855df9cb-xdg6p"] Dec 03 12:27:16 crc kubenswrapper[5052]: I1203 12:27:16.194150 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-api-5d855df9cb-xdg6p" Dec 03 12:27:16 crc kubenswrapper[5052]: I1203 12:27:16.245862 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/51bb178b-072f-4a7a-928b-5a8845d536b7-logs\") pod \"placement-6cdf7689bb-fscpc\" (UID: \"51bb178b-072f-4a7a-928b-5a8845d536b7\") " pod="openstack-kuttl-tests/placement-6cdf7689bb-fscpc" Dec 03 12:27:16 crc kubenswrapper[5052]: I1203 12:27:16.246023 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/15cf6575-b6be-4f67-861a-61ebf02d1495-config\") pod \"neutron-6bf55bd9d6-mrgs9\" (UID: \"15cf6575-b6be-4f67-861a-61ebf02d1495\") " pod="openstack-kuttl-tests/neutron-6bf55bd9d6-mrgs9" Dec 03 12:27:16 crc kubenswrapper[5052]: I1203 12:27:16.246186 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/15cf6575-b6be-4f67-861a-61ebf02d1495-httpd-config\") pod \"neutron-6bf55bd9d6-mrgs9\" (UID: \"15cf6575-b6be-4f67-861a-61ebf02d1495\") " pod="openstack-kuttl-tests/neutron-6bf55bd9d6-mrgs9" Dec 03 12:27:16 crc kubenswrapper[5052]: I1203 12:27:16.246368 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xjczt\" (UniqueName: \"kubernetes.io/projected/15cf6575-b6be-4f67-861a-61ebf02d1495-kube-api-access-xjczt\") pod \"neutron-6bf55bd9d6-mrgs9\" (UID: \"15cf6575-b6be-4f67-861a-61ebf02d1495\") " pod="openstack-kuttl-tests/neutron-6bf55bd9d6-mrgs9" Dec 03 12:27:16 crc kubenswrapper[5052]: I1203 12:27:16.246430 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/3a27c483-00fc-4b06-af6b-3c4b0ff8d0d1-config-data-custom\") pod \"barbican-api-5d855df9cb-xdg6p\" (UID: \"3a27c483-00fc-4b06-af6b-3c4b0ff8d0d1\") " pod="openstack-kuttl-tests/barbican-api-5d855df9cb-xdg6p" Dec 03 12:27:16 crc kubenswrapper[5052]: I1203 12:27:16.246490 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/3a27c483-00fc-4b06-af6b-3c4b0ff8d0d1-logs\") pod \"barbican-api-5d855df9cb-xdg6p\" (UID: \"3a27c483-00fc-4b06-af6b-3c4b0ff8d0d1\") " pod="openstack-kuttl-tests/barbican-api-5d855df9cb-xdg6p" Dec 03 12:27:16 crc kubenswrapper[5052]: I1203 12:27:16.246762 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kkm8f\" (UniqueName: \"kubernetes.io/projected/51bb178b-072f-4a7a-928b-5a8845d536b7-kube-api-access-kkm8f\") pod \"placement-6cdf7689bb-fscpc\" (UID: \"51bb178b-072f-4a7a-928b-5a8845d536b7\") " pod="openstack-kuttl-tests/placement-6cdf7689bb-fscpc" Dec 03 12:27:16 crc kubenswrapper[5052]: I1203 12:27:16.246806 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3a27c483-00fc-4b06-af6b-3c4b0ff8d0d1-combined-ca-bundle\") pod \"barbican-api-5d855df9cb-xdg6p\" (UID: \"3a27c483-00fc-4b06-af6b-3c4b0ff8d0d1\") " pod="openstack-kuttl-tests/barbican-api-5d855df9cb-xdg6p" Dec 03 12:27:16 crc kubenswrapper[5052]: I1203 12:27:16.246856 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/15cf6575-b6be-4f67-861a-61ebf02d1495-combined-ca-bundle\") pod \"neutron-6bf55bd9d6-mrgs9\" (UID: \"15cf6575-b6be-4f67-861a-61ebf02d1495\") " pod="openstack-kuttl-tests/neutron-6bf55bd9d6-mrgs9" Dec 03 12:27:16 crc kubenswrapper[5052]: I1203 12:27:16.246894 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/51bb178b-072f-4a7a-928b-5a8845d536b7-config-data\") pod \"placement-6cdf7689bb-fscpc\" (UID: \"51bb178b-072f-4a7a-928b-5a8845d536b7\") " pod="openstack-kuttl-tests/placement-6cdf7689bb-fscpc" Dec 03 12:27:16 crc kubenswrapper[5052]: I1203 12:27:16.246941 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/3a27c483-00fc-4b06-af6b-3c4b0ff8d0d1-config-data\") pod \"barbican-api-5d855df9cb-xdg6p\" (UID: \"3a27c483-00fc-4b06-af6b-3c4b0ff8d0d1\") " pod="openstack-kuttl-tests/barbican-api-5d855df9cb-xdg6p" Dec 03 12:27:16 crc kubenswrapper[5052]: I1203 12:27:16.246982 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/51bb178b-072f-4a7a-928b-5a8845d536b7-scripts\") pod \"placement-6cdf7689bb-fscpc\" (UID: \"51bb178b-072f-4a7a-928b-5a8845d536b7\") " pod="openstack-kuttl-tests/placement-6cdf7689bb-fscpc" Dec 03 12:27:16 crc kubenswrapper[5052]: I1203 12:27:16.247043 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h68bg\" (UniqueName: \"kubernetes.io/projected/3a27c483-00fc-4b06-af6b-3c4b0ff8d0d1-kube-api-access-h68bg\") pod \"barbican-api-5d855df9cb-xdg6p\" (UID: \"3a27c483-00fc-4b06-af6b-3c4b0ff8d0d1\") " pod="openstack-kuttl-tests/barbican-api-5d855df9cb-xdg6p" Dec 03 12:27:16 crc kubenswrapper[5052]: I1203 12:27:16.247085 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/51bb178b-072f-4a7a-928b-5a8845d536b7-combined-ca-bundle\") pod \"placement-6cdf7689bb-fscpc\" (UID: \"51bb178b-072f-4a7a-928b-5a8845d536b7\") " pod="openstack-kuttl-tests/placement-6cdf7689bb-fscpc" Dec 03 12:27:16 crc kubenswrapper[5052]: I1203 12:27:16.350693 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/3a27c483-00fc-4b06-af6b-3c4b0ff8d0d1-logs\") pod \"barbican-api-5d855df9cb-xdg6p\" (UID: \"3a27c483-00fc-4b06-af6b-3c4b0ff8d0d1\") " pod="openstack-kuttl-tests/barbican-api-5d855df9cb-xdg6p" Dec 03 12:27:16 crc kubenswrapper[5052]: I1203 12:27:16.351062 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-kkm8f\" (UniqueName: \"kubernetes.io/projected/51bb178b-072f-4a7a-928b-5a8845d536b7-kube-api-access-kkm8f\") pod \"placement-6cdf7689bb-fscpc\" (UID: \"51bb178b-072f-4a7a-928b-5a8845d536b7\") " pod="openstack-kuttl-tests/placement-6cdf7689bb-fscpc" Dec 03 12:27:16 crc kubenswrapper[5052]: I1203 12:27:16.351089 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3a27c483-00fc-4b06-af6b-3c4b0ff8d0d1-combined-ca-bundle\") pod \"barbican-api-5d855df9cb-xdg6p\" (UID: \"3a27c483-00fc-4b06-af6b-3c4b0ff8d0d1\") " pod="openstack-kuttl-tests/barbican-api-5d855df9cb-xdg6p" Dec 03 12:27:16 crc kubenswrapper[5052]: I1203 12:27:16.351116 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/15cf6575-b6be-4f67-861a-61ebf02d1495-combined-ca-bundle\") pod \"neutron-6bf55bd9d6-mrgs9\" (UID: \"15cf6575-b6be-4f67-861a-61ebf02d1495\") " pod="openstack-kuttl-tests/neutron-6bf55bd9d6-mrgs9" Dec 03 12:27:16 crc kubenswrapper[5052]: I1203 12:27:16.351136 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/51bb178b-072f-4a7a-928b-5a8845d536b7-config-data\") pod \"placement-6cdf7689bb-fscpc\" (UID: \"51bb178b-072f-4a7a-928b-5a8845d536b7\") " pod="openstack-kuttl-tests/placement-6cdf7689bb-fscpc" Dec 03 12:27:16 crc kubenswrapper[5052]: I1203 12:27:16.351160 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/3a27c483-00fc-4b06-af6b-3c4b0ff8d0d1-config-data\") pod \"barbican-api-5d855df9cb-xdg6p\" (UID: \"3a27c483-00fc-4b06-af6b-3c4b0ff8d0d1\") " pod="openstack-kuttl-tests/barbican-api-5d855df9cb-xdg6p" Dec 03 12:27:16 crc kubenswrapper[5052]: I1203 12:27:16.351187 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/51bb178b-072f-4a7a-928b-5a8845d536b7-scripts\") pod \"placement-6cdf7689bb-fscpc\" (UID: \"51bb178b-072f-4a7a-928b-5a8845d536b7\") " pod="openstack-kuttl-tests/placement-6cdf7689bb-fscpc" Dec 03 12:27:16 crc kubenswrapper[5052]: I1203 12:27:16.351207 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-h68bg\" (UniqueName: \"kubernetes.io/projected/3a27c483-00fc-4b06-af6b-3c4b0ff8d0d1-kube-api-access-h68bg\") pod \"barbican-api-5d855df9cb-xdg6p\" (UID: \"3a27c483-00fc-4b06-af6b-3c4b0ff8d0d1\") " pod="openstack-kuttl-tests/barbican-api-5d855df9cb-xdg6p" Dec 03 12:27:16 crc kubenswrapper[5052]: I1203 12:27:16.351228 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/51bb178b-072f-4a7a-928b-5a8845d536b7-combined-ca-bundle\") pod \"placement-6cdf7689bb-fscpc\" (UID: \"51bb178b-072f-4a7a-928b-5a8845d536b7\") " pod="openstack-kuttl-tests/placement-6cdf7689bb-fscpc" Dec 03 12:27:16 crc kubenswrapper[5052]: I1203 12:27:16.351287 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/51bb178b-072f-4a7a-928b-5a8845d536b7-logs\") pod \"placement-6cdf7689bb-fscpc\" (UID: \"51bb178b-072f-4a7a-928b-5a8845d536b7\") " pod="openstack-kuttl-tests/placement-6cdf7689bb-fscpc" Dec 03 12:27:16 crc kubenswrapper[5052]: I1203 12:27:16.351323 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/15cf6575-b6be-4f67-861a-61ebf02d1495-config\") pod \"neutron-6bf55bd9d6-mrgs9\" (UID: \"15cf6575-b6be-4f67-861a-61ebf02d1495\") " pod="openstack-kuttl-tests/neutron-6bf55bd9d6-mrgs9" Dec 03 12:27:16 crc kubenswrapper[5052]: I1203 12:27:16.351359 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/15cf6575-b6be-4f67-861a-61ebf02d1495-httpd-config\") pod \"neutron-6bf55bd9d6-mrgs9\" (UID: \"15cf6575-b6be-4f67-861a-61ebf02d1495\") " pod="openstack-kuttl-tests/neutron-6bf55bd9d6-mrgs9" Dec 03 12:27:16 crc kubenswrapper[5052]: I1203 12:27:16.351396 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-xjczt\" (UniqueName: \"kubernetes.io/projected/15cf6575-b6be-4f67-861a-61ebf02d1495-kube-api-access-xjczt\") pod \"neutron-6bf55bd9d6-mrgs9\" (UID: \"15cf6575-b6be-4f67-861a-61ebf02d1495\") " pod="openstack-kuttl-tests/neutron-6bf55bd9d6-mrgs9" Dec 03 12:27:16 crc kubenswrapper[5052]: I1203 12:27:16.351418 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/3a27c483-00fc-4b06-af6b-3c4b0ff8d0d1-config-data-custom\") pod \"barbican-api-5d855df9cb-xdg6p\" (UID: \"3a27c483-00fc-4b06-af6b-3c4b0ff8d0d1\") " pod="openstack-kuttl-tests/barbican-api-5d855df9cb-xdg6p" Dec 03 12:27:16 crc kubenswrapper[5052]: I1203 12:27:16.357412 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/3a27c483-00fc-4b06-af6b-3c4b0ff8d0d1-logs\") pod \"barbican-api-5d855df9cb-xdg6p\" (UID: \"3a27c483-00fc-4b06-af6b-3c4b0ff8d0d1\") " pod="openstack-kuttl-tests/barbican-api-5d855df9cb-xdg6p" Dec 03 12:27:16 crc kubenswrapper[5052]: I1203 12:27:16.358321 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/51bb178b-072f-4a7a-928b-5a8845d536b7-logs\") pod \"placement-6cdf7689bb-fscpc\" (UID: \"51bb178b-072f-4a7a-928b-5a8845d536b7\") " pod="openstack-kuttl-tests/placement-6cdf7689bb-fscpc" Dec 03 12:27:16 crc kubenswrapper[5052]: I1203 12:27:16.358826 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/51bb178b-072f-4a7a-928b-5a8845d536b7-scripts\") pod \"placement-6cdf7689bb-fscpc\" (UID: \"51bb178b-072f-4a7a-928b-5a8845d536b7\") " pod="openstack-kuttl-tests/placement-6cdf7689bb-fscpc" Dec 03 12:27:16 crc kubenswrapper[5052]: I1203 12:27:16.366500 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/15cf6575-b6be-4f67-861a-61ebf02d1495-config\") pod \"neutron-6bf55bd9d6-mrgs9\" (UID: \"15cf6575-b6be-4f67-861a-61ebf02d1495\") " pod="openstack-kuttl-tests/neutron-6bf55bd9d6-mrgs9" Dec 03 12:27:16 crc kubenswrapper[5052]: I1203 12:27:16.367380 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3a27c483-00fc-4b06-af6b-3c4b0ff8d0d1-combined-ca-bundle\") pod \"barbican-api-5d855df9cb-xdg6p\" (UID: \"3a27c483-00fc-4b06-af6b-3c4b0ff8d0d1\") " pod="openstack-kuttl-tests/barbican-api-5d855df9cb-xdg6p" Dec 03 12:27:16 crc kubenswrapper[5052]: I1203 12:27:16.371243 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/15cf6575-b6be-4f67-861a-61ebf02d1495-combined-ca-bundle\") pod \"neutron-6bf55bd9d6-mrgs9\" (UID: \"15cf6575-b6be-4f67-861a-61ebf02d1495\") " pod="openstack-kuttl-tests/neutron-6bf55bd9d6-mrgs9" Dec 03 12:27:16 crc kubenswrapper[5052]: I1203 12:27:16.371481 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/3a27c483-00fc-4b06-af6b-3c4b0ff8d0d1-config-data-custom\") pod \"barbican-api-5d855df9cb-xdg6p\" (UID: \"3a27c483-00fc-4b06-af6b-3c4b0ff8d0d1\") " pod="openstack-kuttl-tests/barbican-api-5d855df9cb-xdg6p" Dec 03 12:27:16 crc kubenswrapper[5052]: I1203 12:27:16.373131 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/51bb178b-072f-4a7a-928b-5a8845d536b7-combined-ca-bundle\") pod \"placement-6cdf7689bb-fscpc\" (UID: \"51bb178b-072f-4a7a-928b-5a8845d536b7\") " pod="openstack-kuttl-tests/placement-6cdf7689bb-fscpc" Dec 03 12:27:16 crc kubenswrapper[5052]: I1203 12:27:16.395722 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-h68bg\" (UniqueName: \"kubernetes.io/projected/3a27c483-00fc-4b06-af6b-3c4b0ff8d0d1-kube-api-access-h68bg\") pod \"barbican-api-5d855df9cb-xdg6p\" (UID: \"3a27c483-00fc-4b06-af6b-3c4b0ff8d0d1\") " pod="openstack-kuttl-tests/barbican-api-5d855df9cb-xdg6p" Dec 03 12:27:16 crc kubenswrapper[5052]: I1203 12:27:16.400264 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-kkm8f\" (UniqueName: \"kubernetes.io/projected/51bb178b-072f-4a7a-928b-5a8845d536b7-kube-api-access-kkm8f\") pod \"placement-6cdf7689bb-fscpc\" (UID: \"51bb178b-072f-4a7a-928b-5a8845d536b7\") " pod="openstack-kuttl-tests/placement-6cdf7689bb-fscpc" Dec 03 12:27:16 crc kubenswrapper[5052]: I1203 12:27:16.401302 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/15cf6575-b6be-4f67-861a-61ebf02d1495-httpd-config\") pod \"neutron-6bf55bd9d6-mrgs9\" (UID: \"15cf6575-b6be-4f67-861a-61ebf02d1495\") " pod="openstack-kuttl-tests/neutron-6bf55bd9d6-mrgs9" Dec 03 12:27:16 crc kubenswrapper[5052]: I1203 12:27:16.404093 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/51bb178b-072f-4a7a-928b-5a8845d536b7-config-data\") pod \"placement-6cdf7689bb-fscpc\" (UID: \"51bb178b-072f-4a7a-928b-5a8845d536b7\") " pod="openstack-kuttl-tests/placement-6cdf7689bb-fscpc" Dec 03 12:27:16 crc kubenswrapper[5052]: I1203 12:27:16.408877 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/3a27c483-00fc-4b06-af6b-3c4b0ff8d0d1-config-data\") pod \"barbican-api-5d855df9cb-xdg6p\" (UID: \"3a27c483-00fc-4b06-af6b-3c4b0ff8d0d1\") " pod="openstack-kuttl-tests/barbican-api-5d855df9cb-xdg6p" Dec 03 12:27:16 crc kubenswrapper[5052]: I1203 12:27:16.411047 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-xjczt\" (UniqueName: \"kubernetes.io/projected/15cf6575-b6be-4f67-861a-61ebf02d1495-kube-api-access-xjczt\") pod \"neutron-6bf55bd9d6-mrgs9\" (UID: \"15cf6575-b6be-4f67-861a-61ebf02d1495\") " pod="openstack-kuttl-tests/neutron-6bf55bd9d6-mrgs9" Dec 03 12:27:16 crc kubenswrapper[5052]: I1203 12:27:16.450094 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/cinder-api-0" podUID="fc4f1efa-a22b-4c43-b479-22562c0a7e65" containerName="cinder-api" probeResult="failure" output="Get \"http://10.217.0.22:8776/healthcheck\": read tcp 10.217.0.2:49778->10.217.0.22:8776: read: connection reset by peer" Dec 03 12:27:16 crc kubenswrapper[5052]: I1203 12:27:16.519430 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/nova-metadata-0" podUID="8e052136-a855-4a20-a8c3-38a8bef97dfc" containerName="nova-metadata-metadata" probeResult="failure" output="Get \"http://10.217.0.119:8775/\": read tcp 10.217.0.2:45610->10.217.0.119:8775: read: connection reset by peer" Dec 03 12:27:16 crc kubenswrapper[5052]: I1203 12:27:16.519588 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/nova-metadata-0" podUID="8e052136-a855-4a20-a8c3-38a8bef97dfc" containerName="nova-metadata-log" probeResult="failure" output="Get \"http://10.217.0.119:8775/\": read tcp 10.217.0.2:45604->10.217.0.119:8775: read: connection reset by peer" Dec 03 12:27:16 crc kubenswrapper[5052]: I1203 12:27:16.683613 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 12:27:16 crc kubenswrapper[5052]: I1203 12:27:16.743588 5052 scope.go:117] "RemoveContainer" containerID="0931db22f5fb013cdcb55311764fa795389b7087d437f3d1b8323540414014d9" Dec 03 12:27:16 crc kubenswrapper[5052]: E1203 12:27:16.744620 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0931db22f5fb013cdcb55311764fa795389b7087d437f3d1b8323540414014d9\": container with ID starting with 0931db22f5fb013cdcb55311764fa795389b7087d437f3d1b8323540414014d9 not found: ID does not exist" containerID="0931db22f5fb013cdcb55311764fa795389b7087d437f3d1b8323540414014d9" Dec 03 12:27:16 crc kubenswrapper[5052]: I1203 12:27:16.744736 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0931db22f5fb013cdcb55311764fa795389b7087d437f3d1b8323540414014d9"} err="failed to get container status \"0931db22f5fb013cdcb55311764fa795389b7087d437f3d1b8323540414014d9\": rpc error: code = NotFound desc = could not find container \"0931db22f5fb013cdcb55311764fa795389b7087d437f3d1b8323540414014d9\": container with ID starting with 0931db22f5fb013cdcb55311764fa795389b7087d437f3d1b8323540414014d9 not found: ID does not exist" Dec 03 12:27:16 crc kubenswrapper[5052]: I1203 12:27:16.814342 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-keystone-listener-58b6cdf6bb-5xxd8" Dec 03 12:27:16 crc kubenswrapper[5052]: I1203 12:27:16.821398 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/rabbitmq-server-0" podUID="cd7b6179-ba41-43bd-bc95-490a88aa15fd" containerName="rabbitmq" probeResult="failure" output="dial tcp 10.217.1.214:5672: connect: connection refused" Dec 03 12:27:16 crc kubenswrapper[5052]: E1203 12:27:16.822039 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="e9967011ef9d1750324e927a4b9e79d05b9a0a1c3f9835799be84de44b11270d" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:27:16 crc kubenswrapper[5052]: E1203 12:27:16.825997 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="e9967011ef9d1750324e927a4b9e79d05b9a0a1c3f9835799be84de44b11270d" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:27:16 crc kubenswrapper[5052]: E1203 12:27:16.832433 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="e9967011ef9d1750324e927a4b9e79d05b9a0a1c3f9835799be84de44b11270d" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:27:16 crc kubenswrapper[5052]: E1203 12:27:16.832499 5052 prober.go:104] "Probe errored" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" probeType="Readiness" pod="openstack-kuttl-tests/nova-cell0-conductor-0" podUID="1f6f8fbd-5810-4853-89c0-4be22abe85a3" containerName="nova-cell0-conductor-conductor" Dec 03 12:27:16 crc kubenswrapper[5052]: I1203 12:27:16.855682 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-worker-679bfdf7b7-tg6h7" Dec 03 12:27:16 crc kubenswrapper[5052]: I1203 12:27:16.862115 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:27:16 crc kubenswrapper[5052]: I1203 12:27:16.880029 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-6bf55bd9d6-mrgs9" Dec 03 12:27:16 crc kubenswrapper[5052]: I1203 12:27:16.908709 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-6cdf7689bb-fscpc" Dec 03 12:27:16 crc kubenswrapper[5052]: I1203 12:27:16.922889 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-api-5d855df9cb-xdg6p" Dec 03 12:27:16 crc kubenswrapper[5052]: I1203 12:27:16.963742 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:27:16 crc kubenswrapper[5052]: I1203 12:27:16.964312 5052 generic.go:334] "Generic (PLEG): container finished" podID="aa94b30e-0028-45a0-88aa-b5db5fc2f8fd" containerID="22275b3672b8b4add52b127bbe276c42c2a5e8d7f524d0cf5e1961f6aee9186c" exitCode=0 Dec 03 12:27:16 crc kubenswrapper[5052]: I1203 12:27:16.964393 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-66778fc6-56rjc" event={"ID":"aa94b30e-0028-45a0-88aa-b5db5fc2f8fd","Type":"ContainerDied","Data":"22275b3672b8b4add52b127bbe276c42c2a5e8d7f524d0cf5e1961f6aee9186c"} Dec 03 12:27:16 crc kubenswrapper[5052]: I1203 12:27:16.967562 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/a823b6ba-b2af-4deb-88d2-0902fd6e3780-config-data-custom\") pod \"a823b6ba-b2af-4deb-88d2-0902fd6e3780\" (UID: \"a823b6ba-b2af-4deb-88d2-0902fd6e3780\") " Dec 03 12:27:16 crc kubenswrapper[5052]: I1203 12:27:16.967604 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8tjg8\" (UniqueName: \"kubernetes.io/projected/a823b6ba-b2af-4deb-88d2-0902fd6e3780-kube-api-access-8tjg8\") pod \"a823b6ba-b2af-4deb-88d2-0902fd6e3780\" (UID: \"a823b6ba-b2af-4deb-88d2-0902fd6e3780\") " Dec 03 12:27:16 crc kubenswrapper[5052]: I1203 12:27:16.967720 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a823b6ba-b2af-4deb-88d2-0902fd6e3780-combined-ca-bundle\") pod \"a823b6ba-b2af-4deb-88d2-0902fd6e3780\" (UID: \"a823b6ba-b2af-4deb-88d2-0902fd6e3780\") " Dec 03 12:27:16 crc kubenswrapper[5052]: I1203 12:27:16.967827 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/a823b6ba-b2af-4deb-88d2-0902fd6e3780-scripts\") pod \"a823b6ba-b2af-4deb-88d2-0902fd6e3780\" (UID: \"a823b6ba-b2af-4deb-88d2-0902fd6e3780\") " Dec 03 12:27:16 crc kubenswrapper[5052]: I1203 12:27:16.967878 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a823b6ba-b2af-4deb-88d2-0902fd6e3780-config-data\") pod \"a823b6ba-b2af-4deb-88d2-0902fd6e3780\" (UID: \"a823b6ba-b2af-4deb-88d2-0902fd6e3780\") " Dec 03 12:27:16 crc kubenswrapper[5052]: I1203 12:27:16.967992 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/a823b6ba-b2af-4deb-88d2-0902fd6e3780-etc-machine-id\") pod \"a823b6ba-b2af-4deb-88d2-0902fd6e3780\" (UID: \"a823b6ba-b2af-4deb-88d2-0902fd6e3780\") " Dec 03 12:27:16 crc kubenswrapper[5052]: I1203 12:27:16.972951 5052 generic.go:334] "Generic (PLEG): container finished" podID="7e415947-bb79-4d7b-80fb-f6d290343e19" containerID="5acdaf1de9637eb9a2dcd430e89e4aa32e1830b27b6a59fec9de95f2f8f0c88f" exitCode=0 Dec 03 12:27:16 crc kubenswrapper[5052]: I1203 12:27:16.973018 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"7e415947-bb79-4d7b-80fb-f6d290343e19","Type":"ContainerDied","Data":"5acdaf1de9637eb9a2dcd430e89e4aa32e1830b27b6a59fec9de95f2f8f0c88f"} Dec 03 12:27:16 crc kubenswrapper[5052]: I1203 12:27:16.973049 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"7e415947-bb79-4d7b-80fb-f6d290343e19","Type":"ContainerDied","Data":"50a72c7042df3d2414f13ba1037b71f17e5e6a078091df60b1f2fd7de458fc50"} Dec 03 12:27:16 crc kubenswrapper[5052]: I1203 12:27:16.973061 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="50a72c7042df3d2414f13ba1037b71f17e5e6a078091df60b1f2fd7de458fc50" Dec 03 12:27:16 crc kubenswrapper[5052]: I1203 12:27:16.973927 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:27:16 crc kubenswrapper[5052]: I1203 12:27:16.974130 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/a823b6ba-b2af-4deb-88d2-0902fd6e3780-etc-machine-id" (OuterVolumeSpecName: "etc-machine-id") pod "a823b6ba-b2af-4deb-88d2-0902fd6e3780" (UID: "a823b6ba-b2af-4deb-88d2-0902fd6e3780"). InnerVolumeSpecName "etc-machine-id". PluginName "kubernetes.io/host-path", VolumeGidValue "" Dec 03 12:27:16 crc kubenswrapper[5052]: I1203 12:27:16.979623 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a823b6ba-b2af-4deb-88d2-0902fd6e3780-config-data-custom" (OuterVolumeSpecName: "config-data-custom") pod "a823b6ba-b2af-4deb-88d2-0902fd6e3780" (UID: "a823b6ba-b2af-4deb-88d2-0902fd6e3780"). InnerVolumeSpecName "config-data-custom". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:27:16 crc kubenswrapper[5052]: I1203 12:27:16.987162 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a823b6ba-b2af-4deb-88d2-0902fd6e3780-kube-api-access-8tjg8" (OuterVolumeSpecName: "kube-api-access-8tjg8") pod "a823b6ba-b2af-4deb-88d2-0902fd6e3780" (UID: "a823b6ba-b2af-4deb-88d2-0902fd6e3780"). InnerVolumeSpecName "kube-api-access-8tjg8". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:27:16 crc kubenswrapper[5052]: I1203 12:27:16.998940 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-797ccd5f76-v8cmg" event={"ID":"5314ef6e-9ca6-4c9b-98b0-907b5a6e423c","Type":"ContainerStarted","Data":"922671b6af1b9388525c76529a11dfbf987f5c33b66d696ad739f90656999386"} Dec 03 12:27:17 crc kubenswrapper[5052]: I1203 12:27:17.005739 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a823b6ba-b2af-4deb-88d2-0902fd6e3780-scripts" (OuterVolumeSpecName: "scripts") pod "a823b6ba-b2af-4deb-88d2-0902fd6e3780" (UID: "a823b6ba-b2af-4deb-88d2-0902fd6e3780"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:27:17 crc kubenswrapper[5052]: I1203 12:27:17.037477 5052 generic.go:334] "Generic (PLEG): container finished" podID="fc4f1efa-a22b-4c43-b479-22562c0a7e65" containerID="00bb5044fc1f5280459a013a740283ad38025f50730707308a452a1f487b74f0" exitCode=0 Dec 03 12:27:17 crc kubenswrapper[5052]: I1203 12:27:17.037910 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"fc4f1efa-a22b-4c43-b479-22562c0a7e65","Type":"ContainerDied","Data":"00bb5044fc1f5280459a013a740283ad38025f50730707308a452a1f487b74f0"} Dec 03 12:27:17 crc kubenswrapper[5052]: I1203 12:27:17.072570 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/7e415947-bb79-4d7b-80fb-f6d290343e19-logs\") pod \"7e415947-bb79-4d7b-80fb-f6d290343e19\" (UID: \"7e415947-bb79-4d7b-80fb-f6d290343e19\") " Dec 03 12:27:17 crc kubenswrapper[5052]: I1203 12:27:17.072680 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6f4qx\" (UniqueName: \"kubernetes.io/projected/7ef14abf-76c0-4db5-9291-e0f9675425d1-kube-api-access-6f4qx\") pod \"7ef14abf-76c0-4db5-9291-e0f9675425d1\" (UID: \"7ef14abf-76c0-4db5-9291-e0f9675425d1\") " Dec 03 12:27:17 crc kubenswrapper[5052]: I1203 12:27:17.072805 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/7ef14abf-76c0-4db5-9291-e0f9675425d1-config-data\") pod \"7ef14abf-76c0-4db5-9291-e0f9675425d1\" (UID: \"7ef14abf-76c0-4db5-9291-e0f9675425d1\") " Dec 03 12:27:17 crc kubenswrapper[5052]: I1203 12:27:17.072840 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nbst7\" (UniqueName: \"kubernetes.io/projected/7e415947-bb79-4d7b-80fb-f6d290343e19-kube-api-access-nbst7\") pod \"7e415947-bb79-4d7b-80fb-f6d290343e19\" (UID: \"7e415947-bb79-4d7b-80fb-f6d290343e19\") " Dec 03 12:27:17 crc kubenswrapper[5052]: I1203 12:27:17.072941 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/7ef14abf-76c0-4db5-9291-e0f9675425d1-combined-ca-bundle\") pod \"7ef14abf-76c0-4db5-9291-e0f9675425d1\" (UID: \"7ef14abf-76c0-4db5-9291-e0f9675425d1\") " Dec 03 12:27:17 crc kubenswrapper[5052]: I1203 12:27:17.072974 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/7e415947-bb79-4d7b-80fb-f6d290343e19-combined-ca-bundle\") pod \"7e415947-bb79-4d7b-80fb-f6d290343e19\" (UID: \"7e415947-bb79-4d7b-80fb-f6d290343e19\") " Dec 03 12:27:17 crc kubenswrapper[5052]: I1203 12:27:17.073226 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/7e415947-bb79-4d7b-80fb-f6d290343e19-config-data\") pod \"7e415947-bb79-4d7b-80fb-f6d290343e19\" (UID: \"7e415947-bb79-4d7b-80fb-f6d290343e19\") " Dec 03 12:27:17 crc kubenswrapper[5052]: I1203 12:27:17.074156 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/a823b6ba-b2af-4deb-88d2-0902fd6e3780-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:17 crc kubenswrapper[5052]: I1203 12:27:17.074181 5052 reconciler_common.go:293] "Volume detached for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/a823b6ba-b2af-4deb-88d2-0902fd6e3780-etc-machine-id\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:17 crc kubenswrapper[5052]: I1203 12:27:17.074194 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/a823b6ba-b2af-4deb-88d2-0902fd6e3780-config-data-custom\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:17 crc kubenswrapper[5052]: I1203 12:27:17.074205 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-8tjg8\" (UniqueName: \"kubernetes.io/projected/a823b6ba-b2af-4deb-88d2-0902fd6e3780-kube-api-access-8tjg8\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:17 crc kubenswrapper[5052]: I1203 12:27:17.075990 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e415947-bb79-4d7b-80fb-f6d290343e19-logs" (OuterVolumeSpecName: "logs") pod "7e415947-bb79-4d7b-80fb-f6d290343e19" (UID: "7e415947-bb79-4d7b-80fb-f6d290343e19"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:27:17 crc kubenswrapper[5052]: I1203 12:27:17.076270 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:27:17 crc kubenswrapper[5052]: I1203 12:27:17.077028 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"a823b6ba-b2af-4deb-88d2-0902fd6e3780","Type":"ContainerDied","Data":"675dcfa8f588697349c3715d475dceca2167384ac3d0acfe92c067220c149936"} Dec 03 12:27:17 crc kubenswrapper[5052]: I1203 12:27:17.077075 5052 scope.go:117] "RemoveContainer" containerID="6cd2d71e42ed2933a0efd0de7d0e300b0da269f757f4f18802db5fd7de523192" Dec 03 12:27:17 crc kubenswrapper[5052]: I1203 12:27:17.091316 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-64d6d88f66-7z998" event={"ID":"ef332add-3219-4025-be5b-9d7048cd4a3d","Type":"ContainerStarted","Data":"abaf7b1a3b67fb796df97ce3ddcdd16ea257efe8f07d67446028267864b9c9d5"} Dec 03 12:27:17 crc kubenswrapper[5052]: I1203 12:27:17.098598 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7e415947-bb79-4d7b-80fb-f6d290343e19-kube-api-access-nbst7" (OuterVolumeSpecName: "kube-api-access-nbst7") pod "7e415947-bb79-4d7b-80fb-f6d290343e19" (UID: "7e415947-bb79-4d7b-80fb-f6d290343e19"). InnerVolumeSpecName "kube-api-access-nbst7". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:27:17 crc kubenswrapper[5052]: I1203 12:27:17.098742 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7ef14abf-76c0-4db5-9291-e0f9675425d1-kube-api-access-6f4qx" (OuterVolumeSpecName: "kube-api-access-6f4qx") pod "7ef14abf-76c0-4db5-9291-e0f9675425d1" (UID: "7ef14abf-76c0-4db5-9291-e0f9675425d1"). InnerVolumeSpecName "kube-api-access-6f4qx". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:27:17 crc kubenswrapper[5052]: I1203 12:27:17.119663 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" podUID="5a5320b7-0541-483a-8e0b-78c10d0f8efa" containerName="rabbitmq" probeResult="failure" output="dial tcp 10.217.1.215:5672: connect: connection refused" Dec 03 12:27:17 crc kubenswrapper[5052]: I1203 12:27:17.125364 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7e415947-bb79-4d7b-80fb-f6d290343e19-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "7e415947-bb79-4d7b-80fb-f6d290343e19" (UID: "7e415947-bb79-4d7b-80fb-f6d290343e19"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:27:17 crc kubenswrapper[5052]: I1203 12:27:17.132333 5052 generic.go:334] "Generic (PLEG): container finished" podID="7ef14abf-76c0-4db5-9291-e0f9675425d1" containerID="74ef6e1fe61b1c5ce0d3bf482bd7a2e9edd6da7a23654c82cd5a4d8527f480c3" exitCode=0 Dec 03 12:27:17 crc kubenswrapper[5052]: I1203 12:27:17.132418 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"7ef14abf-76c0-4db5-9291-e0f9675425d1","Type":"ContainerDied","Data":"74ef6e1fe61b1c5ce0d3bf482bd7a2e9edd6da7a23654c82cd5a4d8527f480c3"} Dec 03 12:27:17 crc kubenswrapper[5052]: I1203 12:27:17.132446 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"7ef14abf-76c0-4db5-9291-e0f9675425d1","Type":"ContainerDied","Data":"fe8d431aa6353950723b0a0923ea34d7e989c31fbfadfdca362c4bf3a984602c"} Dec 03 12:27:17 crc kubenswrapper[5052]: I1203 12:27:17.132506 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:27:17 crc kubenswrapper[5052]: I1203 12:27:17.224935 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7e415947-bb79-4d7b-80fb-f6d290343e19-config-data" (OuterVolumeSpecName: "config-data") pod "7e415947-bb79-4d7b-80fb-f6d290343e19" (UID: "7e415947-bb79-4d7b-80fb-f6d290343e19"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:27:17 crc kubenswrapper[5052]: I1203 12:27:17.225222 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a823b6ba-b2af-4deb-88d2-0902fd6e3780-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "a823b6ba-b2af-4deb-88d2-0902fd6e3780" (UID: "a823b6ba-b2af-4deb-88d2-0902fd6e3780"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:27:17 crc kubenswrapper[5052]: I1203 12:27:17.228033 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-6f4qx\" (UniqueName: \"kubernetes.io/projected/7ef14abf-76c0-4db5-9291-e0f9675425d1-kube-api-access-6f4qx\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:17 crc kubenswrapper[5052]: I1203 12:27:17.228074 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-nbst7\" (UniqueName: \"kubernetes.io/projected/7e415947-bb79-4d7b-80fb-f6d290343e19-kube-api-access-nbst7\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:17 crc kubenswrapper[5052]: I1203 12:27:17.228084 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/7e415947-bb79-4d7b-80fb-f6d290343e19-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:17 crc kubenswrapper[5052]: I1203 12:27:17.228094 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a823b6ba-b2af-4deb-88d2-0902fd6e3780-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:17 crc kubenswrapper[5052]: I1203 12:27:17.228103 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/7e415947-bb79-4d7b-80fb-f6d290343e19-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:17 crc kubenswrapper[5052]: I1203 12:27:17.228114 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/7e415947-bb79-4d7b-80fb-f6d290343e19-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:17 crc kubenswrapper[5052]: I1203 12:27:17.231481 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:27:17 crc kubenswrapper[5052]: I1203 12:27:17.231994 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="6ab5f8d6-e515-4f7b-b14f-383b4180f4e7" containerName="ceilometer-central-agent" containerID="cri-o://63734bc615818a6cadecc9f3d7235c2f0339297c97215ad371cef23fc29afd3c" gracePeriod=30 Dec 03 12:27:17 crc kubenswrapper[5052]: I1203 12:27:17.232303 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="6ab5f8d6-e515-4f7b-b14f-383b4180f4e7" containerName="proxy-httpd" containerID="cri-o://059f15379c22e9cbd4b778847ac6fc1ba70f4d80d669d8256de7257a96a80dbb" gracePeriod=30 Dec 03 12:27:17 crc kubenswrapper[5052]: I1203 12:27:17.232589 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="6ab5f8d6-e515-4f7b-b14f-383b4180f4e7" containerName="ceilometer-notification-agent" containerID="cri-o://17e6a6fdbd72ea6cd16402a2b231dc4e88079b9588859838d5611cfd4bb232c5" gracePeriod=30 Dec 03 12:27:17 crc kubenswrapper[5052]: I1203 12:27:17.232686 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="6ab5f8d6-e515-4f7b-b14f-383b4180f4e7" containerName="sg-core" containerID="cri-o://94b056c86cf3f39157a4c778b82ad7f6fc7c02b88c1e228893f8382b18fe8241" gracePeriod=30 Dec 03 12:27:17 crc kubenswrapper[5052]: I1203 12:27:17.238737 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7ef14abf-76c0-4db5-9291-e0f9675425d1-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "7ef14abf-76c0-4db5-9291-e0f9675425d1" (UID: "7ef14abf-76c0-4db5-9291-e0f9675425d1"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:27:17 crc kubenswrapper[5052]: I1203 12:27:17.254217 5052 generic.go:334] "Generic (PLEG): container finished" podID="51912caa-ca45-4e7c-a0ee-a40ae65a1896" containerID="7c7ee713ff0737c8c0810e900709d6bd56bb4cc9128cbcf656be4631e51882ae" exitCode=0 Dec 03 12:27:17 crc kubenswrapper[5052]: I1203 12:27:17.254343 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-conductor-0" event={"ID":"51912caa-ca45-4e7c-a0ee-a40ae65a1896","Type":"ContainerDied","Data":"7c7ee713ff0737c8c0810e900709d6bd56bb4cc9128cbcf656be4631e51882ae"} Dec 03 12:27:17 crc kubenswrapper[5052]: I1203 12:27:17.273053 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7ef14abf-76c0-4db5-9291-e0f9675425d1-config-data" (OuterVolumeSpecName: "config-data") pod "7ef14abf-76c0-4db5-9291-e0f9675425d1" (UID: "7ef14abf-76c0-4db5-9291-e0f9675425d1"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:27:17 crc kubenswrapper[5052]: E1203 12:27:17.273291 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of 7c7ee713ff0737c8c0810e900709d6bd56bb4cc9128cbcf656be4631e51882ae is running failed: container process not found" containerID="7c7ee713ff0737c8c0810e900709d6bd56bb4cc9128cbcf656be4631e51882ae" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:27:17 crc kubenswrapper[5052]: I1203 12:27:17.274269 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" event={"ID":"460b2796-2ea1-4ddd-a4e7-eae181ddfc81","Type":"ContainerStarted","Data":"32b328087df01322dd99751590da4943c3be251fca301d63ddc59ce90100dda3"} Dec 03 12:27:17 crc kubenswrapper[5052]: I1203 12:27:17.283802 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/barbican-api-78665465bd-9q8ff" podUID="2825bd7d-50f6-4d2d-827a-e0d307d1b6de" containerName="barbican-api-log" containerID="cri-o://a389a539341125b103495868581a628b481d42547aa3cdd10558662eabdfce98" gracePeriod=30 Dec 03 12:27:17 crc kubenswrapper[5052]: E1203 12:27:17.284061 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of 7c7ee713ff0737c8c0810e900709d6bd56bb4cc9128cbcf656be4631e51882ae is running failed: container process not found" containerID="7c7ee713ff0737c8c0810e900709d6bd56bb4cc9128cbcf656be4631e51882ae" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:27:17 crc kubenswrapper[5052]: I1203 12:27:17.284732 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/barbican-api-78665465bd-9q8ff" podUID="2825bd7d-50f6-4d2d-827a-e0d307d1b6de" containerName="barbican-api" containerID="cri-o://d5dba2f0d9dfdc433b2210944bb2049a3f79b8b7f7b0c6ac0516d79962651461" gracePeriod=30 Dec 03 12:27:17 crc kubenswrapper[5052]: E1203 12:27:17.297408 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of 7c7ee713ff0737c8c0810e900709d6bd56bb4cc9128cbcf656be4631e51882ae is running failed: container process not found" containerID="7c7ee713ff0737c8c0810e900709d6bd56bb4cc9128cbcf656be4631e51882ae" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:27:17 crc kubenswrapper[5052]: E1203 12:27:17.297498 5052 prober.go:104] "Probe errored" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of 7c7ee713ff0737c8c0810e900709d6bd56bb4cc9128cbcf656be4631e51882ae is running failed: container process not found" probeType="Readiness" pod="openstack-kuttl-tests/nova-cell1-conductor-0" podUID="51912caa-ca45-4e7c-a0ee-a40ae65a1896" containerName="nova-cell1-conductor-conductor" Dec 03 12:27:17 crc kubenswrapper[5052]: I1203 12:27:17.330121 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/7ef14abf-76c0-4db5-9291-e0f9675425d1-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:17 crc kubenswrapper[5052]: I1203 12:27:17.330150 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/7ef14abf-76c0-4db5-9291-e0f9675425d1-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:17 crc kubenswrapper[5052]: I1203 12:27:17.336918 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a823b6ba-b2af-4deb-88d2-0902fd6e3780-config-data" (OuterVolumeSpecName: "config-data") pod "a823b6ba-b2af-4deb-88d2-0902fd6e3780" (UID: "a823b6ba-b2af-4deb-88d2-0902fd6e3780"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:27:17 crc kubenswrapper[5052]: I1203 12:27:17.438839 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a823b6ba-b2af-4deb-88d2-0902fd6e3780-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:17 crc kubenswrapper[5052]: I1203 12:27:17.551033 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/placement-6cdf7689bb-fscpc"] Dec 03 12:27:17 crc kubenswrapper[5052]: I1203 12:27:17.580772 5052 scope.go:117] "RemoveContainer" containerID="53d7dc20a3dd7792e212463451921763c0652bd6c02e21fa9b1e85765b8915ca" Dec 03 12:27:17 crc kubenswrapper[5052]: I1203 12:27:17.745311 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 12:27:17 crc kubenswrapper[5052]: I1203 12:27:17.759908 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 12:27:17 crc kubenswrapper[5052]: I1203 12:27:17.781185 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:27:17 crc kubenswrapper[5052]: I1203 12:27:17.797696 5052 scope.go:117] "RemoveContainer" containerID="74ef6e1fe61b1c5ce0d3bf482bd7a2e9edd6da7a23654c82cd5a4d8527f480c3" Dec 03 12:27:17 crc kubenswrapper[5052]: I1203 12:27:17.804617 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/placement-7897d64b6b-j8w2r"] Dec 03 12:27:17 crc kubenswrapper[5052]: E1203 12:27:17.805612 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="a823b6ba-b2af-4deb-88d2-0902fd6e3780" containerName="cinder-scheduler" Dec 03 12:27:17 crc kubenswrapper[5052]: I1203 12:27:17.805632 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="a823b6ba-b2af-4deb-88d2-0902fd6e3780" containerName="cinder-scheduler" Dec 03 12:27:17 crc kubenswrapper[5052]: E1203 12:27:17.805644 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="7ef14abf-76c0-4db5-9291-e0f9675425d1" containerName="nova-scheduler-scheduler" Dec 03 12:27:17 crc kubenswrapper[5052]: I1203 12:27:17.805650 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="7ef14abf-76c0-4db5-9291-e0f9675425d1" containerName="nova-scheduler-scheduler" Dec 03 12:27:17 crc kubenswrapper[5052]: E1203 12:27:17.805679 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="7e415947-bb79-4d7b-80fb-f6d290343e19" containerName="nova-api-log" Dec 03 12:27:17 crc kubenswrapper[5052]: I1203 12:27:17.805685 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="7e415947-bb79-4d7b-80fb-f6d290343e19" containerName="nova-api-log" Dec 03 12:27:17 crc kubenswrapper[5052]: E1203 12:27:17.805707 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="7e415947-bb79-4d7b-80fb-f6d290343e19" containerName="nova-api-api" Dec 03 12:27:17 crc kubenswrapper[5052]: I1203 12:27:17.805712 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="7e415947-bb79-4d7b-80fb-f6d290343e19" containerName="nova-api-api" Dec 03 12:27:17 crc kubenswrapper[5052]: E1203 12:27:17.805723 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="a823b6ba-b2af-4deb-88d2-0902fd6e3780" containerName="probe" Dec 03 12:27:17 crc kubenswrapper[5052]: I1203 12:27:17.805728 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="a823b6ba-b2af-4deb-88d2-0902fd6e3780" containerName="probe" Dec 03 12:27:17 crc kubenswrapper[5052]: I1203 12:27:17.809606 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="7e415947-bb79-4d7b-80fb-f6d290343e19" containerName="nova-api-api" Dec 03 12:27:17 crc kubenswrapper[5052]: I1203 12:27:17.809660 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="a823b6ba-b2af-4deb-88d2-0902fd6e3780" containerName="probe" Dec 03 12:27:17 crc kubenswrapper[5052]: I1203 12:27:17.809682 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="7ef14abf-76c0-4db5-9291-e0f9675425d1" containerName="nova-scheduler-scheduler" Dec 03 12:27:17 crc kubenswrapper[5052]: I1203 12:27:17.809693 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="7e415947-bb79-4d7b-80fb-f6d290343e19" containerName="nova-api-log" Dec 03 12:27:17 crc kubenswrapper[5052]: I1203 12:27:17.809709 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="a823b6ba-b2af-4deb-88d2-0902fd6e3780" containerName="cinder-scheduler" Dec 03 12:27:17 crc kubenswrapper[5052]: I1203 12:27:17.811051 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-7897d64b6b-j8w2r" Dec 03 12:27:17 crc kubenswrapper[5052]: I1203 12:27:17.814976 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-placement-public-svc" Dec 03 12:27:17 crc kubenswrapper[5052]: I1203 12:27:17.815164 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-placement-internal-svc" Dec 03 12:27:17 crc kubenswrapper[5052]: I1203 12:27:17.829812 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 12:27:17 crc kubenswrapper[5052]: I1203 12:27:17.831465 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:27:17 crc kubenswrapper[5052]: I1203 12:27:17.839495 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cinder-scheduler-config-data" Dec 03 12:27:17 crc kubenswrapper[5052]: I1203 12:27:17.859911 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:27:17 crc kubenswrapper[5052]: I1203 12:27:17.886915 5052 scope.go:117] "RemoveContainer" containerID="74ef6e1fe61b1c5ce0d3bf482bd7a2e9edd6da7a23654c82cd5a4d8527f480c3" Dec 03 12:27:17 crc kubenswrapper[5052]: E1203 12:27:17.887957 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"74ef6e1fe61b1c5ce0d3bf482bd7a2e9edd6da7a23654c82cd5a4d8527f480c3\": container with ID starting with 74ef6e1fe61b1c5ce0d3bf482bd7a2e9edd6da7a23654c82cd5a4d8527f480c3 not found: ID does not exist" containerID="74ef6e1fe61b1c5ce0d3bf482bd7a2e9edd6da7a23654c82cd5a4d8527f480c3" Dec 03 12:27:17 crc kubenswrapper[5052]: I1203 12:27:17.888009 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"74ef6e1fe61b1c5ce0d3bf482bd7a2e9edd6da7a23654c82cd5a4d8527f480c3"} err="failed to get container status \"74ef6e1fe61b1c5ce0d3bf482bd7a2e9edd6da7a23654c82cd5a4d8527f480c3\": rpc error: code = NotFound desc = could not find container \"74ef6e1fe61b1c5ce0d3bf482bd7a2e9edd6da7a23654c82cd5a4d8527f480c3\": container with ID starting with 74ef6e1fe61b1c5ce0d3bf482bd7a2e9edd6da7a23654c82cd5a4d8527f480c3 not found: ID does not exist" Dec 03 12:27:17 crc kubenswrapper[5052]: I1203 12:27:17.889379 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 12:27:17 crc kubenswrapper[5052]: I1203 12:27:17.890255 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:27:17 crc kubenswrapper[5052]: I1203 12:27:17.917994 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:27:17 crc kubenswrapper[5052]: E1203 12:27:17.918487 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="51912caa-ca45-4e7c-a0ee-a40ae65a1896" containerName="nova-cell1-conductor-conductor" Dec 03 12:27:17 crc kubenswrapper[5052]: I1203 12:27:17.918500 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="51912caa-ca45-4e7c-a0ee-a40ae65a1896" containerName="nova-cell1-conductor-conductor" Dec 03 12:27:17 crc kubenswrapper[5052]: I1203 12:27:17.919728 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="51912caa-ca45-4e7c-a0ee-a40ae65a1896" containerName="nova-cell1-conductor-conductor" Dec 03 12:27:17 crc kubenswrapper[5052]: I1203 12:27:17.920762 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:27:17 crc kubenswrapper[5052]: I1203 12:27:17.930855 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/placement-7897d64b6b-j8w2r"] Dec 03 12:27:17 crc kubenswrapper[5052]: I1203 12:27:17.942011 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:27:17 crc kubenswrapper[5052]: I1203 12:27:17.942354 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-scheduler-config-data" Dec 03 12:27:17 crc kubenswrapper[5052]: I1203 12:27:17.957131 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/51912caa-ca45-4e7c-a0ee-a40ae65a1896-config-data\") pod \"51912caa-ca45-4e7c-a0ee-a40ae65a1896\" (UID: \"51912caa-ca45-4e7c-a0ee-a40ae65a1896\") " Dec 03 12:27:17 crc kubenswrapper[5052]: I1203 12:27:17.961624 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tdj8s\" (UniqueName: \"kubernetes.io/projected/51912caa-ca45-4e7c-a0ee-a40ae65a1896-kube-api-access-tdj8s\") pod \"51912caa-ca45-4e7c-a0ee-a40ae65a1896\" (UID: \"51912caa-ca45-4e7c-a0ee-a40ae65a1896\") " Dec 03 12:27:17 crc kubenswrapper[5052]: I1203 12:27:17.962224 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/51912caa-ca45-4e7c-a0ee-a40ae65a1896-combined-ca-bundle\") pod \"51912caa-ca45-4e7c-a0ee-a40ae65a1896\" (UID: \"51912caa-ca45-4e7c-a0ee-a40ae65a1896\") " Dec 03 12:27:17 crc kubenswrapper[5052]: I1203 12:27:17.963270 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/6436bca9-c7a1-4540-bee3-b6dbabb53c89-scripts\") pod \"cinder-scheduler-0\" (UID: \"6436bca9-c7a1-4540-bee3-b6dbabb53c89\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:27:17 crc kubenswrapper[5052]: I1203 12:27:17.964144 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ncjrh\" (UniqueName: \"kubernetes.io/projected/6436bca9-c7a1-4540-bee3-b6dbabb53c89-kube-api-access-ncjrh\") pod \"cinder-scheduler-0\" (UID: \"6436bca9-c7a1-4540-bee3-b6dbabb53c89\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:27:17 crc kubenswrapper[5052]: I1203 12:27:17.966703 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/82331185-51eb-4ef5-8cf6-61dc74b73740-logs\") pod \"placement-7897d64b6b-j8w2r\" (UID: \"82331185-51eb-4ef5-8cf6-61dc74b73740\") " pod="openstack-kuttl-tests/placement-7897d64b6b-j8w2r" Dec 03 12:27:17 crc kubenswrapper[5052]: I1203 12:27:17.966728 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6436bca9-c7a1-4540-bee3-b6dbabb53c89-combined-ca-bundle\") pod \"cinder-scheduler-0\" (UID: \"6436bca9-c7a1-4540-bee3-b6dbabb53c89\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:27:17 crc kubenswrapper[5052]: I1203 12:27:17.966753 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/6436bca9-c7a1-4540-bee3-b6dbabb53c89-etc-machine-id\") pod \"cinder-scheduler-0\" (UID: \"6436bca9-c7a1-4540-bee3-b6dbabb53c89\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:27:17 crc kubenswrapper[5052]: I1203 12:27:17.966860 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/6436bca9-c7a1-4540-bee3-b6dbabb53c89-config-data-custom\") pod \"cinder-scheduler-0\" (UID: \"6436bca9-c7a1-4540-bee3-b6dbabb53c89\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:27:17 crc kubenswrapper[5052]: I1203 12:27:17.966890 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/82331185-51eb-4ef5-8cf6-61dc74b73740-public-tls-certs\") pod \"placement-7897d64b6b-j8w2r\" (UID: \"82331185-51eb-4ef5-8cf6-61dc74b73740\") " pod="openstack-kuttl-tests/placement-7897d64b6b-j8w2r" Dec 03 12:27:17 crc kubenswrapper[5052]: I1203 12:27:17.967014 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/82331185-51eb-4ef5-8cf6-61dc74b73740-config-data\") pod \"placement-7897d64b6b-j8w2r\" (UID: \"82331185-51eb-4ef5-8cf6-61dc74b73740\") " pod="openstack-kuttl-tests/placement-7897d64b6b-j8w2r" Dec 03 12:27:17 crc kubenswrapper[5052]: I1203 12:27:17.967110 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/82331185-51eb-4ef5-8cf6-61dc74b73740-combined-ca-bundle\") pod \"placement-7897d64b6b-j8w2r\" (UID: \"82331185-51eb-4ef5-8cf6-61dc74b73740\") " pod="openstack-kuttl-tests/placement-7897d64b6b-j8w2r" Dec 03 12:27:17 crc kubenswrapper[5052]: I1203 12:27:17.967146 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-znzsm\" (UniqueName: \"kubernetes.io/projected/82331185-51eb-4ef5-8cf6-61dc74b73740-kube-api-access-znzsm\") pod \"placement-7897d64b6b-j8w2r\" (UID: \"82331185-51eb-4ef5-8cf6-61dc74b73740\") " pod="openstack-kuttl-tests/placement-7897d64b6b-j8w2r" Dec 03 12:27:17 crc kubenswrapper[5052]: I1203 12:27:17.967204 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/82331185-51eb-4ef5-8cf6-61dc74b73740-scripts\") pod \"placement-7897d64b6b-j8w2r\" (UID: \"82331185-51eb-4ef5-8cf6-61dc74b73740\") " pod="openstack-kuttl-tests/placement-7897d64b6b-j8w2r" Dec 03 12:27:17 crc kubenswrapper[5052]: I1203 12:27:17.967230 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/82331185-51eb-4ef5-8cf6-61dc74b73740-internal-tls-certs\") pod \"placement-7897d64b6b-j8w2r\" (UID: \"82331185-51eb-4ef5-8cf6-61dc74b73740\") " pod="openstack-kuttl-tests/placement-7897d64b6b-j8w2r" Dec 03 12:27:17 crc kubenswrapper[5052]: I1203 12:27:17.967285 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6436bca9-c7a1-4540-bee3-b6dbabb53c89-config-data\") pod \"cinder-scheduler-0\" (UID: \"6436bca9-c7a1-4540-bee3-b6dbabb53c89\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:27:17 crc kubenswrapper[5052]: I1203 12:27:17.975484 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/51912caa-ca45-4e7c-a0ee-a40ae65a1896-kube-api-access-tdj8s" (OuterVolumeSpecName: "kube-api-access-tdj8s") pod "51912caa-ca45-4e7c-a0ee-a40ae65a1896" (UID: "51912caa-ca45-4e7c-a0ee-a40ae65a1896"). InnerVolumeSpecName "kube-api-access-tdj8s". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.004526 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/51912caa-ca45-4e7c-a0ee-a40ae65a1896-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "51912caa-ca45-4e7c-a0ee-a40ae65a1896" (UID: "51912caa-ca45-4e7c-a0ee-a40ae65a1896"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.010728 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/51912caa-ca45-4e7c-a0ee-a40ae65a1896-config-data" (OuterVolumeSpecName: "config-data") pod "51912caa-ca45-4e7c-a0ee-a40ae65a1896" (UID: "51912caa-ca45-4e7c-a0ee-a40ae65a1896"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.069360 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8a0b169d-6971-4530-936e-8e5f0f57816f-combined-ca-bundle\") pod \"nova-scheduler-0\" (UID: \"8a0b169d-6971-4530-936e-8e5f0f57816f\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.069412 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/82331185-51eb-4ef5-8cf6-61dc74b73740-config-data\") pod \"placement-7897d64b6b-j8w2r\" (UID: \"82331185-51eb-4ef5-8cf6-61dc74b73740\") " pod="openstack-kuttl-tests/placement-7897d64b6b-j8w2r" Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.069446 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-25mdm\" (UniqueName: \"kubernetes.io/projected/8a0b169d-6971-4530-936e-8e5f0f57816f-kube-api-access-25mdm\") pod \"nova-scheduler-0\" (UID: \"8a0b169d-6971-4530-936e-8e5f0f57816f\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.069525 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/82331185-51eb-4ef5-8cf6-61dc74b73740-combined-ca-bundle\") pod \"placement-7897d64b6b-j8w2r\" (UID: \"82331185-51eb-4ef5-8cf6-61dc74b73740\") " pod="openstack-kuttl-tests/placement-7897d64b6b-j8w2r" Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.069555 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-znzsm\" (UniqueName: \"kubernetes.io/projected/82331185-51eb-4ef5-8cf6-61dc74b73740-kube-api-access-znzsm\") pod \"placement-7897d64b6b-j8w2r\" (UID: \"82331185-51eb-4ef5-8cf6-61dc74b73740\") " pod="openstack-kuttl-tests/placement-7897d64b6b-j8w2r" Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.069579 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/8a0b169d-6971-4530-936e-8e5f0f57816f-config-data\") pod \"nova-scheduler-0\" (UID: \"8a0b169d-6971-4530-936e-8e5f0f57816f\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.069616 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/82331185-51eb-4ef5-8cf6-61dc74b73740-scripts\") pod \"placement-7897d64b6b-j8w2r\" (UID: \"82331185-51eb-4ef5-8cf6-61dc74b73740\") " pod="openstack-kuttl-tests/placement-7897d64b6b-j8w2r" Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.069637 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/82331185-51eb-4ef5-8cf6-61dc74b73740-internal-tls-certs\") pod \"placement-7897d64b6b-j8w2r\" (UID: \"82331185-51eb-4ef5-8cf6-61dc74b73740\") " pod="openstack-kuttl-tests/placement-7897d64b6b-j8w2r" Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.069686 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6436bca9-c7a1-4540-bee3-b6dbabb53c89-config-data\") pod \"cinder-scheduler-0\" (UID: \"6436bca9-c7a1-4540-bee3-b6dbabb53c89\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.069742 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/6436bca9-c7a1-4540-bee3-b6dbabb53c89-scripts\") pod \"cinder-scheduler-0\" (UID: \"6436bca9-c7a1-4540-bee3-b6dbabb53c89\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.069785 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-ncjrh\" (UniqueName: \"kubernetes.io/projected/6436bca9-c7a1-4540-bee3-b6dbabb53c89-kube-api-access-ncjrh\") pod \"cinder-scheduler-0\" (UID: \"6436bca9-c7a1-4540-bee3-b6dbabb53c89\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.069813 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/82331185-51eb-4ef5-8cf6-61dc74b73740-logs\") pod \"placement-7897d64b6b-j8w2r\" (UID: \"82331185-51eb-4ef5-8cf6-61dc74b73740\") " pod="openstack-kuttl-tests/placement-7897d64b6b-j8w2r" Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.069835 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6436bca9-c7a1-4540-bee3-b6dbabb53c89-combined-ca-bundle\") pod \"cinder-scheduler-0\" (UID: \"6436bca9-c7a1-4540-bee3-b6dbabb53c89\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.069861 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/6436bca9-c7a1-4540-bee3-b6dbabb53c89-etc-machine-id\") pod \"cinder-scheduler-0\" (UID: \"6436bca9-c7a1-4540-bee3-b6dbabb53c89\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.069949 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/6436bca9-c7a1-4540-bee3-b6dbabb53c89-config-data-custom\") pod \"cinder-scheduler-0\" (UID: \"6436bca9-c7a1-4540-bee3-b6dbabb53c89\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.069975 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/82331185-51eb-4ef5-8cf6-61dc74b73740-public-tls-certs\") pod \"placement-7897d64b6b-j8w2r\" (UID: \"82331185-51eb-4ef5-8cf6-61dc74b73740\") " pod="openstack-kuttl-tests/placement-7897d64b6b-j8w2r" Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.070055 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/51912caa-ca45-4e7c-a0ee-a40ae65a1896-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.070070 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/51912caa-ca45-4e7c-a0ee-a40ae65a1896-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.070084 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-tdj8s\" (UniqueName: \"kubernetes.io/projected/51912caa-ca45-4e7c-a0ee-a40ae65a1896-kube-api-access-tdj8s\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.075231 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/82331185-51eb-4ef5-8cf6-61dc74b73740-combined-ca-bundle\") pod \"placement-7897d64b6b-j8w2r\" (UID: \"82331185-51eb-4ef5-8cf6-61dc74b73740\") " pod="openstack-kuttl-tests/placement-7897d64b6b-j8w2r" Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.075255 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6436bca9-c7a1-4540-bee3-b6dbabb53c89-combined-ca-bundle\") pod \"cinder-scheduler-0\" (UID: \"6436bca9-c7a1-4540-bee3-b6dbabb53c89\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.075763 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/82331185-51eb-4ef5-8cf6-61dc74b73740-logs\") pod \"placement-7897d64b6b-j8w2r\" (UID: \"82331185-51eb-4ef5-8cf6-61dc74b73740\") " pod="openstack-kuttl-tests/placement-7897d64b6b-j8w2r" Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.076066 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/6436bca9-c7a1-4540-bee3-b6dbabb53c89-etc-machine-id\") pod \"cinder-scheduler-0\" (UID: \"6436bca9-c7a1-4540-bee3-b6dbabb53c89\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.079668 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/6436bca9-c7a1-4540-bee3-b6dbabb53c89-config-data-custom\") pod \"cinder-scheduler-0\" (UID: \"6436bca9-c7a1-4540-bee3-b6dbabb53c89\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.080271 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/6436bca9-c7a1-4540-bee3-b6dbabb53c89-scripts\") pod \"cinder-scheduler-0\" (UID: \"6436bca9-c7a1-4540-bee3-b6dbabb53c89\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.080547 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/82331185-51eb-4ef5-8cf6-61dc74b73740-config-data\") pod \"placement-7897d64b6b-j8w2r\" (UID: \"82331185-51eb-4ef5-8cf6-61dc74b73740\") " pod="openstack-kuttl-tests/placement-7897d64b6b-j8w2r" Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.080655 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/82331185-51eb-4ef5-8cf6-61dc74b73740-scripts\") pod \"placement-7897d64b6b-j8w2r\" (UID: \"82331185-51eb-4ef5-8cf6-61dc74b73740\") " pod="openstack-kuttl-tests/placement-7897d64b6b-j8w2r" Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.083142 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/82331185-51eb-4ef5-8cf6-61dc74b73740-public-tls-certs\") pod \"placement-7897d64b6b-j8w2r\" (UID: \"82331185-51eb-4ef5-8cf6-61dc74b73740\") " pod="openstack-kuttl-tests/placement-7897d64b6b-j8w2r" Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.083597 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6436bca9-c7a1-4540-bee3-b6dbabb53c89-config-data\") pod \"cinder-scheduler-0\" (UID: \"6436bca9-c7a1-4540-bee3-b6dbabb53c89\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.083856 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/82331185-51eb-4ef5-8cf6-61dc74b73740-internal-tls-certs\") pod \"placement-7897d64b6b-j8w2r\" (UID: \"82331185-51eb-4ef5-8cf6-61dc74b73740\") " pod="openstack-kuttl-tests/placement-7897d64b6b-j8w2r" Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.092370 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-znzsm\" (UniqueName: \"kubernetes.io/projected/82331185-51eb-4ef5-8cf6-61dc74b73740-kube-api-access-znzsm\") pod \"placement-7897d64b6b-j8w2r\" (UID: \"82331185-51eb-4ef5-8cf6-61dc74b73740\") " pod="openstack-kuttl-tests/placement-7897d64b6b-j8w2r" Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.101230 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-ncjrh\" (UniqueName: \"kubernetes.io/projected/6436bca9-c7a1-4540-bee3-b6dbabb53c89-kube-api-access-ncjrh\") pod \"cinder-scheduler-0\" (UID: \"6436bca9-c7a1-4540-bee3-b6dbabb53c89\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.158030 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-7897d64b6b-j8w2r" Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.179830 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8a0b169d-6971-4530-936e-8e5f0f57816f-combined-ca-bundle\") pod \"nova-scheduler-0\" (UID: \"8a0b169d-6971-4530-936e-8e5f0f57816f\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.179894 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-25mdm\" (UniqueName: \"kubernetes.io/projected/8a0b169d-6971-4530-936e-8e5f0f57816f-kube-api-access-25mdm\") pod \"nova-scheduler-0\" (UID: \"8a0b169d-6971-4530-936e-8e5f0f57816f\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.179952 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/8a0b169d-6971-4530-936e-8e5f0f57816f-config-data\") pod \"nova-scheduler-0\" (UID: \"8a0b169d-6971-4530-936e-8e5f0f57816f\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.184452 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.187366 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/8a0b169d-6971-4530-936e-8e5f0f57816f-config-data\") pod \"nova-scheduler-0\" (UID: \"8a0b169d-6971-4530-936e-8e5f0f57816f\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.189578 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8a0b169d-6971-4530-936e-8e5f0f57816f-combined-ca-bundle\") pod \"nova-scheduler-0\" (UID: \"8a0b169d-6971-4530-936e-8e5f0f57816f\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.233192 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-25mdm\" (UniqueName: \"kubernetes.io/projected/8a0b169d-6971-4530-936e-8e5f0f57816f-kube-api-access-25mdm\") pod \"nova-scheduler-0\" (UID: \"8a0b169d-6971-4530-936e-8e5f0f57816f\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.302612 5052 generic.go:334] "Generic (PLEG): container finished" podID="7fa6aac1-6ef3-434a-945e-3ced02ca5361" containerID="d09526b701062188ac0714b7d2d4c1986577ed904704b893ec88c6d251aa69cb" exitCode=0 Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.302712 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"7fa6aac1-6ef3-434a-945e-3ced02ca5361","Type":"ContainerDied","Data":"d09526b701062188ac0714b7d2d4c1986577ed904704b893ec88c6d251aa69cb"} Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.305322 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-64d6d88f66-7z998" event={"ID":"ef332add-3219-4025-be5b-9d7048cd4a3d","Type":"ContainerStarted","Data":"bc067ea70660c1847f572c9b0681b01351a86de5eb2ae47c967575bf575054ad"} Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.305397 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/keystone-64d6d88f66-7z998" Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.309124 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-66778fc6-56rjc" event={"ID":"aa94b30e-0028-45a0-88aa-b5db5fc2f8fd","Type":"ContainerDied","Data":"b36bb6e4d776a944073051ec87e592c7e4df8a5403df5a5943eaa23ba3ffe5e3"} Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.309151 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b36bb6e4d776a944073051ec87e592c7e4df8a5403df5a5943eaa23ba3ffe5e3" Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.339084 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/keystone-64d6d88f66-7z998" podStartSLOduration=5.339067539 podStartE2EDuration="5.339067539s" podCreationTimestamp="2025-12-03 12:27:13 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:27:18.33385681 +0000 UTC m=+4028.922098574" watchObservedRunningTime="2025-12-03 12:27:18.339067539 +0000 UTC m=+4028.927309303" Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.339748 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.340356 5052 generic.go:334] "Generic (PLEG): container finished" podID="6ab5f8d6-e515-4f7b-b14f-383b4180f4e7" containerID="059f15379c22e9cbd4b778847ac6fc1ba70f4d80d669d8256de7257a96a80dbb" exitCode=0 Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.340381 5052 generic.go:334] "Generic (PLEG): container finished" podID="6ab5f8d6-e515-4f7b-b14f-383b4180f4e7" containerID="94b056c86cf3f39157a4c778b82ad7f6fc7c02b88c1e228893f8382b18fe8241" exitCode=2 Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.340391 5052 generic.go:334] "Generic (PLEG): container finished" podID="6ab5f8d6-e515-4f7b-b14f-383b4180f4e7" containerID="63734bc615818a6cadecc9f3d7235c2f0339297c97215ad371cef23fc29afd3c" exitCode=0 Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.340430 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"6ab5f8d6-e515-4f7b-b14f-383b4180f4e7","Type":"ContainerDied","Data":"059f15379c22e9cbd4b778847ac6fc1ba70f4d80d669d8256de7257a96a80dbb"} Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.340455 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"6ab5f8d6-e515-4f7b-b14f-383b4180f4e7","Type":"ContainerDied","Data":"94b056c86cf3f39157a4c778b82ad7f6fc7c02b88c1e228893f8382b18fe8241"} Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.340469 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"6ab5f8d6-e515-4f7b-b14f-383b4180f4e7","Type":"ContainerDied","Data":"63734bc615818a6cadecc9f3d7235c2f0339297c97215ad371cef23fc29afd3c"} Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.344673 5052 generic.go:334] "Generic (PLEG): container finished" podID="6b96dc78-b389-4764-8ba7-6a4c5e41e345" containerID="be65e03ff7ec9d6c0e6aa36bae00bb290c5b0d2746c6c491262833760b551397" exitCode=0 Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.344733 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"6b96dc78-b389-4764-8ba7-6a4c5e41e345","Type":"ContainerDied","Data":"be65e03ff7ec9d6c0e6aa36bae00bb290c5b0d2746c6c491262833760b551397"} Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.348288 5052 generic.go:334] "Generic (PLEG): container finished" podID="8e052136-a855-4a20-a8c3-38a8bef97dfc" containerID="0e8b8edad48c623ae81447848a940a98d6827814b85cda0ff7b96abb10a64815" exitCode=0 Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.348337 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"8e052136-a855-4a20-a8c3-38a8bef97dfc","Type":"ContainerDied","Data":"0e8b8edad48c623ae81447848a940a98d6827814b85cda0ff7b96abb10a64815"} Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.357026 5052 generic.go:334] "Generic (PLEG): container finished" podID="2825bd7d-50f6-4d2d-827a-e0d307d1b6de" containerID="d5dba2f0d9dfdc433b2210944bb2049a3f79b8b7f7b0c6ac0516d79962651461" exitCode=0 Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.357064 5052 generic.go:334] "Generic (PLEG): container finished" podID="2825bd7d-50f6-4d2d-827a-e0d307d1b6de" containerID="a389a539341125b103495868581a628b481d42547aa3cdd10558662eabdfce98" exitCode=143 Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.357116 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-78665465bd-9q8ff" event={"ID":"2825bd7d-50f6-4d2d-827a-e0d307d1b6de","Type":"ContainerDied","Data":"d5dba2f0d9dfdc433b2210944bb2049a3f79b8b7f7b0c6ac0516d79962651461"} Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.357147 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-78665465bd-9q8ff" event={"ID":"2825bd7d-50f6-4d2d-827a-e0d307d1b6de","Type":"ContainerDied","Data":"a389a539341125b103495868581a628b481d42547aa3cdd10558662eabdfce98"} Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.361473 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-797ccd5f76-v8cmg" event={"ID":"5314ef6e-9ca6-4c9b-98b0-907b5a6e423c","Type":"ContainerStarted","Data":"c989c0a379e7a6f19d77f112ad039489181d227f7ffa0c338a6048fa3a8be370"} Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.361628 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/neutron-797ccd5f76-v8cmg" podUID="5314ef6e-9ca6-4c9b-98b0-907b5a6e423c" containerName="neutron-api" containerID="cri-o://922671b6af1b9388525c76529a11dfbf987f5c33b66d696ad739f90656999386" gracePeriod=30 Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.361968 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/neutron-797ccd5f76-v8cmg" Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.362145 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/neutron-797ccd5f76-v8cmg" podUID="5314ef6e-9ca6-4c9b-98b0-907b5a6e423c" containerName="neutron-httpd" containerID="cri-o://c989c0a379e7a6f19d77f112ad039489181d227f7ffa0c338a6048fa3a8be370" gracePeriod=30 Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.380009 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"fc4f1efa-a22b-4c43-b479-22562c0a7e65","Type":"ContainerDied","Data":"761c802e32e772d570305634f6352b2399bac37933208705f9277668225fb2d3"} Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.380050 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="761c802e32e772d570305634f6352b2399bac37933208705f9277668225fb2d3" Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.385858 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-conductor-0" event={"ID":"51912caa-ca45-4e7c-a0ee-a40ae65a1896","Type":"ContainerDied","Data":"a467fc950fff63daca6d13f96e45304e11902dc49e2e775a81131f721029ded1"} Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.385936 5052 scope.go:117] "RemoveContainer" containerID="7c7ee713ff0737c8c0810e900709d6bd56bb4cc9128cbcf656be4631e51882ae" Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.386088 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.388640 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/neutron-797ccd5f76-v8cmg" podStartSLOduration=5.388620443 podStartE2EDuration="5.388620443s" podCreationTimestamp="2025-12-03 12:27:13 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:27:18.387243064 +0000 UTC m=+4028.975484828" watchObservedRunningTime="2025-12-03 12:27:18.388620443 +0000 UTC m=+4028.976862207" Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.412319 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" event={"ID":"460b2796-2ea1-4ddd-a4e7-eae181ddfc81","Type":"ContainerStarted","Data":"c99aecbfc0f318ebcfe8e5c07b7758a65a70339a148d9b58bf3923d596c20d7b"} Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.417269 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.441074 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.468307 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" podStartSLOduration=4.468283287 podStartE2EDuration="4.468283287s" podCreationTimestamp="2025-12-03 12:27:14 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:27:18.444134478 +0000 UTC m=+4029.032376262" watchObservedRunningTime="2025-12-03 12:27:18.468283287 +0000 UTC m=+4029.056525131" Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.482804 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.490482 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-66778fc6-56rjc" Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.508357 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.523565 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:27:18 crc kubenswrapper[5052]: E1203 12:27:18.524081 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="fc4f1efa-a22b-4c43-b479-22562c0a7e65" containerName="cinder-api" Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.524105 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="fc4f1efa-a22b-4c43-b479-22562c0a7e65" containerName="cinder-api" Dec 03 12:27:18 crc kubenswrapper[5052]: E1203 12:27:18.524120 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="fc4f1efa-a22b-4c43-b479-22562c0a7e65" containerName="cinder-api-log" Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.524127 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="fc4f1efa-a22b-4c43-b479-22562c0a7e65" containerName="cinder-api-log" Dec 03 12:27:18 crc kubenswrapper[5052]: E1203 12:27:18.524154 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="aa94b30e-0028-45a0-88aa-b5db5fc2f8fd" containerName="keystone-api" Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.524163 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa94b30e-0028-45a0-88aa-b5db5fc2f8fd" containerName="keystone-api" Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.524384 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="fc4f1efa-a22b-4c43-b479-22562c0a7e65" containerName="cinder-api" Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.524400 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="aa94b30e-0028-45a0-88aa-b5db5fc2f8fd" containerName="keystone-api" Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.524427 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="fc4f1efa-a22b-4c43-b479-22562c0a7e65" containerName="cinder-api-log" Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.525695 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.535378 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-api-config-data" Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.542273 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.597933 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/fc4f1efa-a22b-4c43-b479-22562c0a7e65-config-data-custom\") pod \"fc4f1efa-a22b-4c43-b479-22562c0a7e65\" (UID: \"fc4f1efa-a22b-4c43-b479-22562c0a7e65\") " Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.598079 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/fc4f1efa-a22b-4c43-b479-22562c0a7e65-logs\") pod \"fc4f1efa-a22b-4c43-b479-22562c0a7e65\" (UID: \"fc4f1efa-a22b-4c43-b479-22562c0a7e65\") " Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.598212 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/aa94b30e-0028-45a0-88aa-b5db5fc2f8fd-combined-ca-bundle\") pod \"aa94b30e-0028-45a0-88aa-b5db5fc2f8fd\" (UID: \"aa94b30e-0028-45a0-88aa-b5db5fc2f8fd\") " Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.598280 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jgmv4\" (UniqueName: \"kubernetes.io/projected/fc4f1efa-a22b-4c43-b479-22562c0a7e65-kube-api-access-jgmv4\") pod \"fc4f1efa-a22b-4c43-b479-22562c0a7e65\" (UID: \"fc4f1efa-a22b-4c43-b479-22562c0a7e65\") " Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.598354 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/fc4f1efa-a22b-4c43-b479-22562c0a7e65-scripts\") pod \"fc4f1efa-a22b-4c43-b479-22562c0a7e65\" (UID: \"fc4f1efa-a22b-4c43-b479-22562c0a7e65\") " Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.598387 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/aa94b30e-0028-45a0-88aa-b5db5fc2f8fd-config-data\") pod \"aa94b30e-0028-45a0-88aa-b5db5fc2f8fd\" (UID: \"aa94b30e-0028-45a0-88aa-b5db5fc2f8fd\") " Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.598424 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/fc4f1efa-a22b-4c43-b479-22562c0a7e65-combined-ca-bundle\") pod \"fc4f1efa-a22b-4c43-b479-22562c0a7e65\" (UID: \"fc4f1efa-a22b-4c43-b479-22562c0a7e65\") " Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.598479 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/fc4f1efa-a22b-4c43-b479-22562c0a7e65-config-data\") pod \"fc4f1efa-a22b-4c43-b479-22562c0a7e65\" (UID: \"fc4f1efa-a22b-4c43-b479-22562c0a7e65\") " Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.598558 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fmmp2\" (UniqueName: \"kubernetes.io/projected/aa94b30e-0028-45a0-88aa-b5db5fc2f8fd-kube-api-access-fmmp2\") pod \"aa94b30e-0028-45a0-88aa-b5db5fc2f8fd\" (UID: \"aa94b30e-0028-45a0-88aa-b5db5fc2f8fd\") " Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.598596 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/aa94b30e-0028-45a0-88aa-b5db5fc2f8fd-scripts\") pod \"aa94b30e-0028-45a0-88aa-b5db5fc2f8fd\" (UID: \"aa94b30e-0028-45a0-88aa-b5db5fc2f8fd\") " Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.598629 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/fc4f1efa-a22b-4c43-b479-22562c0a7e65-etc-machine-id\") pod \"fc4f1efa-a22b-4c43-b479-22562c0a7e65\" (UID: \"fc4f1efa-a22b-4c43-b479-22562c0a7e65\") " Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.598681 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/aa94b30e-0028-45a0-88aa-b5db5fc2f8fd-credential-keys\") pod \"aa94b30e-0028-45a0-88aa-b5db5fc2f8fd\" (UID: \"aa94b30e-0028-45a0-88aa-b5db5fc2f8fd\") " Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.598742 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/aa94b30e-0028-45a0-88aa-b5db5fc2f8fd-fernet-keys\") pod \"aa94b30e-0028-45a0-88aa-b5db5fc2f8fd\" (UID: \"aa94b30e-0028-45a0-88aa-b5db5fc2f8fd\") " Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.601395 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.606634 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/fc4f1efa-a22b-4c43-b479-22562c0a7e65-etc-machine-id" (OuterVolumeSpecName: "etc-machine-id") pod "fc4f1efa-a22b-4c43-b479-22562c0a7e65" (UID: "fc4f1efa-a22b-4c43-b479-22562c0a7e65"). InnerVolumeSpecName "etc-machine-id". PluginName "kubernetes.io/host-path", VolumeGidValue "" Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.609628 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fc4f1efa-a22b-4c43-b479-22562c0a7e65-logs" (OuterVolumeSpecName: "logs") pod "fc4f1efa-a22b-4c43-b479-22562c0a7e65" (UID: "fc4f1efa-a22b-4c43-b479-22562c0a7e65"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.613357 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.640251 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fc4f1efa-a22b-4c43-b479-22562c0a7e65-kube-api-access-jgmv4" (OuterVolumeSpecName: "kube-api-access-jgmv4") pod "fc4f1efa-a22b-4c43-b479-22562c0a7e65" (UID: "fc4f1efa-a22b-4c43-b479-22562c0a7e65"). InnerVolumeSpecName "kube-api-access-jgmv4". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.652931 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fc4f1efa-a22b-4c43-b479-22562c0a7e65-config-data-custom" (OuterVolumeSpecName: "config-data-custom") pod "fc4f1efa-a22b-4c43-b479-22562c0a7e65" (UID: "fc4f1efa-a22b-4c43-b479-22562c0a7e65"). InnerVolumeSpecName "config-data-custom". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.660542 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aa94b30e-0028-45a0-88aa-b5db5fc2f8fd-credential-keys" (OuterVolumeSpecName: "credential-keys") pod "aa94b30e-0028-45a0-88aa-b5db5fc2f8fd" (UID: "aa94b30e-0028-45a0-88aa-b5db5fc2f8fd"). InnerVolumeSpecName "credential-keys". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.664952 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aa94b30e-0028-45a0-88aa-b5db5fc2f8fd-scripts" (OuterVolumeSpecName: "scripts") pod "aa94b30e-0028-45a0-88aa-b5db5fc2f8fd" (UID: "aa94b30e-0028-45a0-88aa-b5db5fc2f8fd"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.669041 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fc4f1efa-a22b-4c43-b479-22562c0a7e65-scripts" (OuterVolumeSpecName: "scripts") pod "fc4f1efa-a22b-4c43-b479-22562c0a7e65" (UID: "fc4f1efa-a22b-4c43-b479-22562c0a7e65"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.669966 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aa94b30e-0028-45a0-88aa-b5db5fc2f8fd-fernet-keys" (OuterVolumeSpecName: "fernet-keys") pod "aa94b30e-0028-45a0-88aa-b5db5fc2f8fd" (UID: "aa94b30e-0028-45a0-88aa-b5db5fc2f8fd"). InnerVolumeSpecName "fernet-keys". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.696240 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-0"] Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.701730 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aa94b30e-0028-45a0-88aa-b5db5fc2f8fd-kube-api-access-fmmp2" (OuterVolumeSpecName: "kube-api-access-fmmp2") pod "aa94b30e-0028-45a0-88aa-b5db5fc2f8fd" (UID: "aa94b30e-0028-45a0-88aa-b5db5fc2f8fd"). InnerVolumeSpecName "kube-api-access-fmmp2". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.704937 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/7fa6aac1-6ef3-434a-945e-3ced02ca5361-config-data\") pod \"7fa6aac1-6ef3-434a-945e-3ced02ca5361\" (UID: \"7fa6aac1-6ef3-434a-945e-3ced02ca5361\") " Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.705001 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/7fa6aac1-6ef3-434a-945e-3ced02ca5361-scripts\") pod \"7fa6aac1-6ef3-434a-945e-3ced02ca5361\" (UID: \"7fa6aac1-6ef3-434a-945e-3ced02ca5361\") " Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.705067 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8e052136-a855-4a20-a8c3-38a8bef97dfc-combined-ca-bundle\") pod \"8e052136-a855-4a20-a8c3-38a8bef97dfc\" (UID: \"8e052136-a855-4a20-a8c3-38a8bef97dfc\") " Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.705159 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/7fa6aac1-6ef3-434a-945e-3ced02ca5361-combined-ca-bundle\") pod \"7fa6aac1-6ef3-434a-945e-3ced02ca5361\" (UID: \"7fa6aac1-6ef3-434a-945e-3ced02ca5361\") " Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.705181 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/8e052136-a855-4a20-a8c3-38a8bef97dfc-config-data\") pod \"8e052136-a855-4a20-a8c3-38a8bef97dfc\" (UID: \"8e052136-a855-4a20-a8c3-38a8bef97dfc\") " Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.705239 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/8e052136-a855-4a20-a8c3-38a8bef97dfc-logs\") pod \"8e052136-a855-4a20-a8c3-38a8bef97dfc\" (UID: \"8e052136-a855-4a20-a8c3-38a8bef97dfc\") " Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.705283 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"glance\" (UniqueName: \"kubernetes.io/local-volume/local-storage09-crc\") pod \"7fa6aac1-6ef3-434a-945e-3ced02ca5361\" (UID: \"7fa6aac1-6ef3-434a-945e-3ced02ca5361\") " Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.705344 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/7fa6aac1-6ef3-434a-945e-3ced02ca5361-httpd-run\") pod \"7fa6aac1-6ef3-434a-945e-3ced02ca5361\" (UID: \"7fa6aac1-6ef3-434a-945e-3ced02ca5361\") " Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.705384 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/7fa6aac1-6ef3-434a-945e-3ced02ca5361-logs\") pod \"7fa6aac1-6ef3-434a-945e-3ced02ca5361\" (UID: \"7fa6aac1-6ef3-434a-945e-3ced02ca5361\") " Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.705414 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v84g4\" (UniqueName: \"kubernetes.io/projected/8e052136-a855-4a20-a8c3-38a8bef97dfc-kube-api-access-v84g4\") pod \"8e052136-a855-4a20-a8c3-38a8bef97dfc\" (UID: \"8e052136-a855-4a20-a8c3-38a8bef97dfc\") " Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.705544 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vnz6l\" (UniqueName: \"kubernetes.io/projected/7fa6aac1-6ef3-434a-945e-3ced02ca5361-kube-api-access-vnz6l\") pod \"7fa6aac1-6ef3-434a-945e-3ced02ca5361\" (UID: \"7fa6aac1-6ef3-434a-945e-3ced02ca5361\") " Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.707206 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7fa6aac1-6ef3-434a-945e-3ced02ca5361-logs" (OuterVolumeSpecName: "logs") pod "7fa6aac1-6ef3-434a-945e-3ced02ca5361" (UID: "7fa6aac1-6ef3-434a-945e-3ced02ca5361"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.710447 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7fa6aac1-6ef3-434a-945e-3ced02ca5361-httpd-run" (OuterVolumeSpecName: "httpd-run") pod "7fa6aac1-6ef3-434a-945e-3ced02ca5361" (UID: "7fa6aac1-6ef3-434a-945e-3ced02ca5361"). InnerVolumeSpecName "httpd-run". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.710864 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8e052136-a855-4a20-a8c3-38a8bef97dfc-logs" (OuterVolumeSpecName: "logs") pod "8e052136-a855-4a20-a8c3-38a8bef97dfc" (UID: "8e052136-a855-4a20-a8c3-38a8bef97dfc"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.711463 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/810e47c2-430b-424d-92ea-66c20ec8501f-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"810e47c2-430b-424d-92ea-66c20ec8501f\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.711714 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/810e47c2-430b-424d-92ea-66c20ec8501f-logs\") pod \"nova-api-0\" (UID: \"810e47c2-430b-424d-92ea-66c20ec8501f\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.711768 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/810e47c2-430b-424d-92ea-66c20ec8501f-config-data\") pod \"nova-api-0\" (UID: \"810e47c2-430b-424d-92ea-66c20ec8501f\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.711951 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4fbl2\" (UniqueName: \"kubernetes.io/projected/810e47c2-430b-424d-92ea-66c20ec8501f-kube-api-access-4fbl2\") pod \"nova-api-0\" (UID: \"810e47c2-430b-424d-92ea-66c20ec8501f\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.712214 5052 reconciler_common.go:293] "Volume detached for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/aa94b30e-0028-45a0-88aa-b5db5fc2f8fd-credential-keys\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.715665 5052 reconciler_common.go:293] "Volume detached for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/aa94b30e-0028-45a0-88aa-b5db5fc2f8fd-fernet-keys\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.715705 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/8e052136-a855-4a20-a8c3-38a8bef97dfc-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.715736 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/fc4f1efa-a22b-4c43-b479-22562c0a7e65-config-data-custom\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.715754 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/fc4f1efa-a22b-4c43-b479-22562c0a7e65-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.715765 5052 reconciler_common.go:293] "Volume detached for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/7fa6aac1-6ef3-434a-945e-3ced02ca5361-httpd-run\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.715775 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/7fa6aac1-6ef3-434a-945e-3ced02ca5361-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.715796 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-jgmv4\" (UniqueName: \"kubernetes.io/projected/fc4f1efa-a22b-4c43-b479-22562c0a7e65-kube-api-access-jgmv4\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.715805 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/fc4f1efa-a22b-4c43-b479-22562c0a7e65-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.715815 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-fmmp2\" (UniqueName: \"kubernetes.io/projected/aa94b30e-0028-45a0-88aa-b5db5fc2f8fd-kube-api-access-fmmp2\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.715829 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/aa94b30e-0028-45a0-88aa-b5db5fc2f8fd-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.715840 5052 reconciler_common.go:293] "Volume detached for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/fc4f1efa-a22b-4c43-b479-22562c0a7e65-etc-machine-id\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.764959 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage09-crc" (OuterVolumeSpecName: "glance") pod "7fa6aac1-6ef3-434a-945e-3ced02ca5361" (UID: "7fa6aac1-6ef3-434a-945e-3ced02ca5361"). InnerVolumeSpecName "local-storage09-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.771864 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8e052136-a855-4a20-a8c3-38a8bef97dfc-kube-api-access-v84g4" (OuterVolumeSpecName: "kube-api-access-v84g4") pod "8e052136-a855-4a20-a8c3-38a8bef97dfc" (UID: "8e052136-a855-4a20-a8c3-38a8bef97dfc"). InnerVolumeSpecName "kube-api-access-v84g4". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.771995 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7fa6aac1-6ef3-434a-945e-3ced02ca5361-scripts" (OuterVolumeSpecName: "scripts") pod "7fa6aac1-6ef3-434a-945e-3ced02ca5361" (UID: "7fa6aac1-6ef3-434a-945e-3ced02ca5361"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.772659 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7fa6aac1-6ef3-434a-945e-3ced02ca5361-kube-api-access-vnz6l" (OuterVolumeSpecName: "kube-api-access-vnz6l") pod "7fa6aac1-6ef3-434a-945e-3ced02ca5361" (UID: "7fa6aac1-6ef3-434a-945e-3ced02ca5361"). InnerVolumeSpecName "kube-api-access-vnz6l". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.776706 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-api-78665465bd-9q8ff" Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.779368 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-0"] Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.798700 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-0"] Dec 03 12:27:18 crc kubenswrapper[5052]: E1203 12:27:18.799244 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="8e052136-a855-4a20-a8c3-38a8bef97dfc" containerName="nova-metadata-log" Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.799264 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="8e052136-a855-4a20-a8c3-38a8bef97dfc" containerName="nova-metadata-log" Dec 03 12:27:18 crc kubenswrapper[5052]: E1203 12:27:18.799293 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="7fa6aac1-6ef3-434a-945e-3ced02ca5361" containerName="glance-httpd" Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.799301 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="7fa6aac1-6ef3-434a-945e-3ced02ca5361" containerName="glance-httpd" Dec 03 12:27:18 crc kubenswrapper[5052]: E1203 12:27:18.799320 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="2825bd7d-50f6-4d2d-827a-e0d307d1b6de" containerName="barbican-api" Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.799329 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="2825bd7d-50f6-4d2d-827a-e0d307d1b6de" containerName="barbican-api" Dec 03 12:27:18 crc kubenswrapper[5052]: E1203 12:27:18.799351 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="7fa6aac1-6ef3-434a-945e-3ced02ca5361" containerName="glance-log" Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.799360 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="7fa6aac1-6ef3-434a-945e-3ced02ca5361" containerName="glance-log" Dec 03 12:27:18 crc kubenswrapper[5052]: E1203 12:27:18.799380 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="8e052136-a855-4a20-a8c3-38a8bef97dfc" containerName="nova-metadata-metadata" Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.799389 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="8e052136-a855-4a20-a8c3-38a8bef97dfc" containerName="nova-metadata-metadata" Dec 03 12:27:18 crc kubenswrapper[5052]: E1203 12:27:18.799406 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="2825bd7d-50f6-4d2d-827a-e0d307d1b6de" containerName="barbican-api-log" Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.799416 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="2825bd7d-50f6-4d2d-827a-e0d307d1b6de" containerName="barbican-api-log" Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.799688 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="8e052136-a855-4a20-a8c3-38a8bef97dfc" containerName="nova-metadata-metadata" Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.799710 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="2825bd7d-50f6-4d2d-827a-e0d307d1b6de" containerName="barbican-api-log" Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.799738 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="8e052136-a855-4a20-a8c3-38a8bef97dfc" containerName="nova-metadata-log" Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.799752 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="2825bd7d-50f6-4d2d-827a-e0d307d1b6de" containerName="barbican-api" Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.799769 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="7fa6aac1-6ef3-434a-945e-3ced02ca5361" containerName="glance-log" Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.799783 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="7fa6aac1-6ef3-434a-945e-3ced02ca5361" containerName="glance-httpd" Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.800658 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.837875 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/810e47c2-430b-424d-92ea-66c20ec8501f-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"810e47c2-430b-424d-92ea-66c20ec8501f\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.838003 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/810e47c2-430b-424d-92ea-66c20ec8501f-logs\") pod \"nova-api-0\" (UID: \"810e47c2-430b-424d-92ea-66c20ec8501f\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.838038 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/810e47c2-430b-424d-92ea-66c20ec8501f-config-data\") pod \"nova-api-0\" (UID: \"810e47c2-430b-424d-92ea-66c20ec8501f\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.838153 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-4fbl2\" (UniqueName: \"kubernetes.io/projected/810e47c2-430b-424d-92ea-66c20ec8501f-kube-api-access-4fbl2\") pod \"nova-api-0\" (UID: \"810e47c2-430b-424d-92ea-66c20ec8501f\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.838267 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage09-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage09-crc\") on node \"crc\" " Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.838285 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-v84g4\" (UniqueName: \"kubernetes.io/projected/8e052136-a855-4a20-a8c3-38a8bef97dfc-kube-api-access-v84g4\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.838299 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-vnz6l\" (UniqueName: \"kubernetes.io/projected/7fa6aac1-6ef3-434a-945e-3ced02ca5361-kube-api-access-vnz6l\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.838311 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/7fa6aac1-6ef3-434a-945e-3ced02ca5361-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.842962 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/810e47c2-430b-424d-92ea-66c20ec8501f-logs\") pod \"nova-api-0\" (UID: \"810e47c2-430b-424d-92ea-66c20ec8501f\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.849976 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fc4f1efa-a22b-4c43-b479-22562c0a7e65-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "fc4f1efa-a22b-4c43-b479-22562c0a7e65" (UID: "fc4f1efa-a22b-4c43-b479-22562c0a7e65"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.851084 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell1-conductor-config-data" Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.856723 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aa94b30e-0028-45a0-88aa-b5db5fc2f8fd-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "aa94b30e-0028-45a0-88aa-b5db5fc2f8fd" (UID: "aa94b30e-0028-45a0-88aa-b5db5fc2f8fd"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.864108 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-0"] Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.884544 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/810e47c2-430b-424d-92ea-66c20ec8501f-config-data\") pod \"nova-api-0\" (UID: \"810e47c2-430b-424d-92ea-66c20ec8501f\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.898730 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aa94b30e-0028-45a0-88aa-b5db5fc2f8fd-config-data" (OuterVolumeSpecName: "config-data") pod "aa94b30e-0028-45a0-88aa-b5db5fc2f8fd" (UID: "aa94b30e-0028-45a0-88aa-b5db5fc2f8fd"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.912538 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8e052136-a855-4a20-a8c3-38a8bef97dfc-config-data" (OuterVolumeSpecName: "config-data") pod "8e052136-a855-4a20-a8c3-38a8bef97dfc" (UID: "8e052136-a855-4a20-a8c3-38a8bef97dfc"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.919384 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-4fbl2\" (UniqueName: \"kubernetes.io/projected/810e47c2-430b-424d-92ea-66c20ec8501f-kube-api-access-4fbl2\") pod \"nova-api-0\" (UID: \"810e47c2-430b-424d-92ea-66c20ec8501f\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.936605 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-keystone-listener-58b6cdf6bb-5xxd8"] Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.941628 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/810e47c2-430b-424d-92ea-66c20ec8501f-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"810e47c2-430b-424d-92ea-66c20ec8501f\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.942201 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/2825bd7d-50f6-4d2d-827a-e0d307d1b6de-config-data-custom\") pod \"2825bd7d-50f6-4d2d-827a-e0d307d1b6de\" (UID: \"2825bd7d-50f6-4d2d-827a-e0d307d1b6de\") " Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.942453 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8gplk\" (UniqueName: \"kubernetes.io/projected/2825bd7d-50f6-4d2d-827a-e0d307d1b6de-kube-api-access-8gplk\") pod \"2825bd7d-50f6-4d2d-827a-e0d307d1b6de\" (UID: \"2825bd7d-50f6-4d2d-827a-e0d307d1b6de\") " Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.942491 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/2825bd7d-50f6-4d2d-827a-e0d307d1b6de-logs\") pod \"2825bd7d-50f6-4d2d-827a-e0d307d1b6de\" (UID: \"2825bd7d-50f6-4d2d-827a-e0d307d1b6de\") " Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.942611 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2825bd7d-50f6-4d2d-827a-e0d307d1b6de-combined-ca-bundle\") pod \"2825bd7d-50f6-4d2d-827a-e0d307d1b6de\" (UID: \"2825bd7d-50f6-4d2d-827a-e0d307d1b6de\") " Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.942680 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/2825bd7d-50f6-4d2d-827a-e0d307d1b6de-config-data\") pod \"2825bd7d-50f6-4d2d-827a-e0d307d1b6de\" (UID: \"2825bd7d-50f6-4d2d-827a-e0d307d1b6de\") " Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.943299 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/fcee2113-2b2f-47f9-b118-4f7488ad604e-combined-ca-bundle\") pod \"nova-cell1-conductor-0\" (UID: \"fcee2113-2b2f-47f9-b118-4f7488ad604e\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.943454 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/fcee2113-2b2f-47f9-b118-4f7488ad604e-config-data\") pod \"nova-cell1-conductor-0\" (UID: \"fcee2113-2b2f-47f9-b118-4f7488ad604e\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.943607 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xttsw\" (UniqueName: \"kubernetes.io/projected/fcee2113-2b2f-47f9-b118-4f7488ad604e-kube-api-access-xttsw\") pod \"nova-cell1-conductor-0\" (UID: \"fcee2113-2b2f-47f9-b118-4f7488ad604e\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.943704 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/aa94b30e-0028-45a0-88aa-b5db5fc2f8fd-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.943726 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/aa94b30e-0028-45a0-88aa-b5db5fc2f8fd-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.943740 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/fc4f1efa-a22b-4c43-b479-22562c0a7e65-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.943754 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/8e052136-a855-4a20-a8c3-38a8bef97dfc-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.947731 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2825bd7d-50f6-4d2d-827a-e0d307d1b6de-logs" (OuterVolumeSpecName: "logs") pod "2825bd7d-50f6-4d2d-827a-e0d307d1b6de" (UID: "2825bd7d-50f6-4d2d-827a-e0d307d1b6de"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.964090 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2825bd7d-50f6-4d2d-827a-e0d307d1b6de-kube-api-access-8gplk" (OuterVolumeSpecName: "kube-api-access-8gplk") pod "2825bd7d-50f6-4d2d-827a-e0d307d1b6de" (UID: "2825bd7d-50f6-4d2d-827a-e0d307d1b6de"). InnerVolumeSpecName "kube-api-access-8gplk". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:27:18 crc kubenswrapper[5052]: I1203 12:27:18.966918 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7fa6aac1-6ef3-434a-945e-3ced02ca5361-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "7fa6aac1-6ef3-434a-945e-3ced02ca5361" (UID: "7fa6aac1-6ef3-434a-945e-3ced02ca5361"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:27:19 crc kubenswrapper[5052]: I1203 12:27:19.034684 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2825bd7d-50f6-4d2d-827a-e0d307d1b6de-config-data-custom" (OuterVolumeSpecName: "config-data-custom") pod "2825bd7d-50f6-4d2d-827a-e0d307d1b6de" (UID: "2825bd7d-50f6-4d2d-827a-e0d307d1b6de"). InnerVolumeSpecName "config-data-custom". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:27:19 crc kubenswrapper[5052]: I1203 12:27:19.037531 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fc4f1efa-a22b-4c43-b479-22562c0a7e65-config-data" (OuterVolumeSpecName: "config-data") pod "fc4f1efa-a22b-4c43-b479-22562c0a7e65" (UID: "fc4f1efa-a22b-4c43-b479-22562c0a7e65"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:27:19 crc kubenswrapper[5052]: I1203 12:27:19.044995 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-xttsw\" (UniqueName: \"kubernetes.io/projected/fcee2113-2b2f-47f9-b118-4f7488ad604e-kube-api-access-xttsw\") pod \"nova-cell1-conductor-0\" (UID: \"fcee2113-2b2f-47f9-b118-4f7488ad604e\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:27:19 crc kubenswrapper[5052]: I1203 12:27:19.045113 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/fcee2113-2b2f-47f9-b118-4f7488ad604e-combined-ca-bundle\") pod \"nova-cell1-conductor-0\" (UID: \"fcee2113-2b2f-47f9-b118-4f7488ad604e\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:27:19 crc kubenswrapper[5052]: I1203 12:27:19.045204 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/fcee2113-2b2f-47f9-b118-4f7488ad604e-config-data\") pod \"nova-cell1-conductor-0\" (UID: \"fcee2113-2b2f-47f9-b118-4f7488ad604e\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:27:19 crc kubenswrapper[5052]: I1203 12:27:19.045327 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-8gplk\" (UniqueName: \"kubernetes.io/projected/2825bd7d-50f6-4d2d-827a-e0d307d1b6de-kube-api-access-8gplk\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:19 crc kubenswrapper[5052]: I1203 12:27:19.045348 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/2825bd7d-50f6-4d2d-827a-e0d307d1b6de-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:19 crc kubenswrapper[5052]: I1203 12:27:19.045363 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/fc4f1efa-a22b-4c43-b479-22562c0a7e65-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:19 crc kubenswrapper[5052]: I1203 12:27:19.045375 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/7fa6aac1-6ef3-434a-945e-3ced02ca5361-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:19 crc kubenswrapper[5052]: I1203 12:27:19.045387 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/2825bd7d-50f6-4d2d-827a-e0d307d1b6de-config-data-custom\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:19 crc kubenswrapper[5052]: I1203 12:27:19.053116 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage09-crc" (UniqueName: "kubernetes.io/local-volume/local-storage09-crc") on node "crc" Dec 03 12:27:19 crc kubenswrapper[5052]: I1203 12:27:19.080116 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/fcee2113-2b2f-47f9-b118-4f7488ad604e-config-data\") pod \"nova-cell1-conductor-0\" (UID: \"fcee2113-2b2f-47f9-b118-4f7488ad604e\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:27:19 crc kubenswrapper[5052]: I1203 12:27:19.083069 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/fcee2113-2b2f-47f9-b118-4f7488ad604e-combined-ca-bundle\") pod \"nova-cell1-conductor-0\" (UID: \"fcee2113-2b2f-47f9-b118-4f7488ad604e\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:27:19 crc kubenswrapper[5052]: I1203 12:27:19.148460 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage09-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage09-crc\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:19 crc kubenswrapper[5052]: I1203 12:27:19.155605 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/placement-6cdf7689bb-fscpc"] Dec 03 12:27:19 crc kubenswrapper[5052]: I1203 12:27:19.157461 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-xttsw\" (UniqueName: \"kubernetes.io/projected/fcee2113-2b2f-47f9-b118-4f7488ad604e-kube-api-access-xttsw\") pod \"nova-cell1-conductor-0\" (UID: \"fcee2113-2b2f-47f9-b118-4f7488ad604e\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:27:19 crc kubenswrapper[5052]: I1203 12:27:19.191117 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8e052136-a855-4a20-a8c3-38a8bef97dfc-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "8e052136-a855-4a20-a8c3-38a8bef97dfc" (UID: "8e052136-a855-4a20-a8c3-38a8bef97dfc"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:27:19 crc kubenswrapper[5052]: I1203 12:27:19.201626 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:27:19 crc kubenswrapper[5052]: I1203 12:27:19.218572 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:27:19 crc kubenswrapper[5052]: I1203 12:27:19.219570 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7fa6aac1-6ef3-434a-945e-3ced02ca5361-config-data" (OuterVolumeSpecName: "config-data") pod "7fa6aac1-6ef3-434a-945e-3ced02ca5361" (UID: "7fa6aac1-6ef3-434a-945e-3ced02ca5361"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:27:19 crc kubenswrapper[5052]: I1203 12:27:19.229552 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2825bd7d-50f6-4d2d-827a-e0d307d1b6de-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "2825bd7d-50f6-4d2d-827a-e0d307d1b6de" (UID: "2825bd7d-50f6-4d2d-827a-e0d307d1b6de"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:27:19 crc kubenswrapper[5052]: I1203 12:27:19.251984 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8e052136-a855-4a20-a8c3-38a8bef97dfc-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:19 crc kubenswrapper[5052]: I1203 12:27:19.252022 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2825bd7d-50f6-4d2d-827a-e0d307d1b6de-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:19 crc kubenswrapper[5052]: I1203 12:27:19.252036 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/7fa6aac1-6ef3-434a-945e-3ced02ca5361-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:19 crc kubenswrapper[5052]: I1203 12:27:19.256027 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-worker-679bfdf7b7-tg6h7"] Dec 03 12:27:19 crc kubenswrapper[5052]: I1203 12:27:19.330654 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2825bd7d-50f6-4d2d-827a-e0d307d1b6de-config-data" (OuterVolumeSpecName: "config-data") pod "2825bd7d-50f6-4d2d-827a-e0d307d1b6de" (UID: "2825bd7d-50f6-4d2d-827a-e0d307d1b6de"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:27:19 crc kubenswrapper[5052]: I1203 12:27:19.344179 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-api-5d855df9cb-xdg6p"] Dec 03 12:27:19 crc kubenswrapper[5052]: I1203 12:27:19.354206 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/2825bd7d-50f6-4d2d-827a-e0d307d1b6de-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:19 crc kubenswrapper[5052]: I1203 12:27:19.431825 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-6bf55bd9d6-mrgs9"] Dec 03 12:27:19 crc kubenswrapper[5052]: I1203 12:27:19.442720 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-keystone-listener-58b6cdf6bb-5xxd8" event={"ID":"71df5bf1-7fd8-4a4d-b0f5-85e3911ae8a9","Type":"ContainerStarted","Data":"9bfb97776da1f99ca1bf1dce1ae364ac91f105b29678722d7112b93147540221"} Dec 03 12:27:19 crc kubenswrapper[5052]: I1203 12:27:19.474823 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-6cdf7689bb-fscpc" event={"ID":"51bb178b-072f-4a7a-928b-5a8845d536b7","Type":"ContainerStarted","Data":"60a86a4c230f97868c413bde12086e18296cdc745808a7ffcb0c69014f0f5bd7"} Dec 03 12:27:19 crc kubenswrapper[5052]: I1203 12:27:19.501468 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-78665465bd-9q8ff" event={"ID":"2825bd7d-50f6-4d2d-827a-e0d307d1b6de","Type":"ContainerDied","Data":"e95f312b10938bf3db0b2eb1850f8a9d9b0a2d38be69c7c0dbd457adee9cce11"} Dec 03 12:27:19 crc kubenswrapper[5052]: I1203 12:27:19.501546 5052 scope.go:117] "RemoveContainer" containerID="d5dba2f0d9dfdc433b2210944bb2049a3f79b8b7f7b0c6ac0516d79962651461" Dec 03 12:27:19 crc kubenswrapper[5052]: I1203 12:27:19.501691 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-api-78665465bd-9q8ff" Dec 03 12:27:19 crc kubenswrapper[5052]: I1203 12:27:19.515816 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"7fa6aac1-6ef3-434a-945e-3ced02ca5361","Type":"ContainerDied","Data":"8dce4ce3a11724b3a7decc671b33ed571677104363149336cd40027e0c72fe64"} Dec 03 12:27:19 crc kubenswrapper[5052]: I1203 12:27:19.515938 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:27:19 crc kubenswrapper[5052]: I1203 12:27:19.547135 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:27:19 crc kubenswrapper[5052]: I1203 12:27:19.565254 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-5d855df9cb-xdg6p" event={"ID":"3a27c483-00fc-4b06-af6b-3c4b0ff8d0d1","Type":"ContainerStarted","Data":"cc1a0008b39944b5b62eec529e54a52c2cfcdd0258dd27e2496043f63eb6d80d"} Dec 03 12:27:19 crc kubenswrapper[5052]: I1203 12:27:19.588965 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"6b96dc78-b389-4764-8ba7-6a4c5e41e345","Type":"ContainerDied","Data":"4c1b137865c217b5640ab49ca182ffb12a56fdd4218b0de68f6869a2f5827df4"} Dec 03 12:27:19 crc kubenswrapper[5052]: I1203 12:27:19.589149 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:27:19 crc kubenswrapper[5052]: I1203 12:27:19.603889 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:27:19 crc kubenswrapper[5052]: I1203 12:27:19.603899 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"8e052136-a855-4a20-a8c3-38a8bef97dfc","Type":"ContainerDied","Data":"e5cdd00f5d6f4cc4c468c717371122ada5030689733c650b8b9165f773948b1f"} Dec 03 12:27:19 crc kubenswrapper[5052]: I1203 12:27:19.636051 5052 generic.go:334] "Generic (PLEG): container finished" podID="5314ef6e-9ca6-4c9b-98b0-907b5a6e423c" containerID="c989c0a379e7a6f19d77f112ad039489181d227f7ffa0c338a6048fa3a8be370" exitCode=0 Dec 03 12:27:19 crc kubenswrapper[5052]: I1203 12:27:19.636126 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-797ccd5f76-v8cmg" event={"ID":"5314ef6e-9ca6-4c9b-98b0-907b5a6e423c","Type":"ContainerDied","Data":"c989c0a379e7a6f19d77f112ad039489181d227f7ffa0c338a6048fa3a8be370"} Dec 03 12:27:19 crc kubenswrapper[5052]: I1203 12:27:19.640358 5052 scope.go:117] "RemoveContainer" containerID="a389a539341125b103495868581a628b481d42547aa3cdd10558662eabdfce98" Dec 03 12:27:19 crc kubenswrapper[5052]: I1203 12:27:19.661735 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-api-78665465bd-9q8ff"] Dec 03 12:27:19 crc kubenswrapper[5052]: I1203 12:27:19.663632 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:27:19 crc kubenswrapper[5052]: I1203 12:27:19.665444 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-66778fc6-56rjc" Dec 03 12:27:19 crc kubenswrapper[5052]: I1203 12:27:19.663625 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-worker-679bfdf7b7-tg6h7" event={"ID":"72a32b4d-d61d-43a5-a45d-c1fd4bad672d","Type":"ContainerStarted","Data":"822106e884f3485cc561862c636fdd5bf103caa3793d11830c8ecdaa4d4430cc"} Dec 03 12:27:19 crc kubenswrapper[5052]: I1203 12:27:19.669538 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hq6bh\" (UniqueName: \"kubernetes.io/projected/6b96dc78-b389-4764-8ba7-6a4c5e41e345-kube-api-access-hq6bh\") pod \"6b96dc78-b389-4764-8ba7-6a4c5e41e345\" (UID: \"6b96dc78-b389-4764-8ba7-6a4c5e41e345\") " Dec 03 12:27:19 crc kubenswrapper[5052]: I1203 12:27:19.669681 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/6b96dc78-b389-4764-8ba7-6a4c5e41e345-scripts\") pod \"6b96dc78-b389-4764-8ba7-6a4c5e41e345\" (UID: \"6b96dc78-b389-4764-8ba7-6a4c5e41e345\") " Dec 03 12:27:19 crc kubenswrapper[5052]: I1203 12:27:19.669801 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/6b96dc78-b389-4764-8ba7-6a4c5e41e345-httpd-run\") pod \"6b96dc78-b389-4764-8ba7-6a4c5e41e345\" (UID: \"6b96dc78-b389-4764-8ba7-6a4c5e41e345\") " Dec 03 12:27:19 crc kubenswrapper[5052]: I1203 12:27:19.669829 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6b96dc78-b389-4764-8ba7-6a4c5e41e345-config-data\") pod \"6b96dc78-b389-4764-8ba7-6a4c5e41e345\" (UID: \"6b96dc78-b389-4764-8ba7-6a4c5e41e345\") " Dec 03 12:27:19 crc kubenswrapper[5052]: I1203 12:27:19.669888 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"glance\" (UniqueName: \"kubernetes.io/local-volume/local-storage07-crc\") pod \"6b96dc78-b389-4764-8ba7-6a4c5e41e345\" (UID: \"6b96dc78-b389-4764-8ba7-6a4c5e41e345\") " Dec 03 12:27:19 crc kubenswrapper[5052]: I1203 12:27:19.669981 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/6b96dc78-b389-4764-8ba7-6a4c5e41e345-logs\") pod \"6b96dc78-b389-4764-8ba7-6a4c5e41e345\" (UID: \"6b96dc78-b389-4764-8ba7-6a4c5e41e345\") " Dec 03 12:27:19 crc kubenswrapper[5052]: I1203 12:27:19.670027 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6b96dc78-b389-4764-8ba7-6a4c5e41e345-combined-ca-bundle\") pod \"6b96dc78-b389-4764-8ba7-6a4c5e41e345\" (UID: \"6b96dc78-b389-4764-8ba7-6a4c5e41e345\") " Dec 03 12:27:19 crc kubenswrapper[5052]: I1203 12:27:19.678940 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b96dc78-b389-4764-8ba7-6a4c5e41e345-httpd-run" (OuterVolumeSpecName: "httpd-run") pod "6b96dc78-b389-4764-8ba7-6a4c5e41e345" (UID: "6b96dc78-b389-4764-8ba7-6a4c5e41e345"). InnerVolumeSpecName "httpd-run". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:27:19 crc kubenswrapper[5052]: I1203 12:27:19.679596 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b96dc78-b389-4764-8ba7-6a4c5e41e345-logs" (OuterVolumeSpecName: "logs") pod "6b96dc78-b389-4764-8ba7-6a4c5e41e345" (UID: "6b96dc78-b389-4764-8ba7-6a4c5e41e345"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:27:19 crc kubenswrapper[5052]: I1203 12:27:19.683316 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6b96dc78-b389-4764-8ba7-6a4c5e41e345-kube-api-access-hq6bh" (OuterVolumeSpecName: "kube-api-access-hq6bh") pod "6b96dc78-b389-4764-8ba7-6a4c5e41e345" (UID: "6b96dc78-b389-4764-8ba7-6a4c5e41e345"). InnerVolumeSpecName "kube-api-access-hq6bh". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:27:19 crc kubenswrapper[5052]: I1203 12:27:19.719809 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage07-crc" (OuterVolumeSpecName: "glance") pod "6b96dc78-b389-4764-8ba7-6a4c5e41e345" (UID: "6b96dc78-b389-4764-8ba7-6a4c5e41e345"). InnerVolumeSpecName "local-storage07-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 12:27:19 crc kubenswrapper[5052]: I1203 12:27:19.738880 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6b96dc78-b389-4764-8ba7-6a4c5e41e345-scripts" (OuterVolumeSpecName: "scripts") pod "6b96dc78-b389-4764-8ba7-6a4c5e41e345" (UID: "6b96dc78-b389-4764-8ba7-6a4c5e41e345"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:27:19 crc kubenswrapper[5052]: I1203 12:27:19.776838 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage07-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage07-crc\") on node \"crc\" " Dec 03 12:27:19 crc kubenswrapper[5052]: I1203 12:27:19.779377 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/6b96dc78-b389-4764-8ba7-6a4c5e41e345-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:19 crc kubenswrapper[5052]: I1203 12:27:19.780541 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-hq6bh\" (UniqueName: \"kubernetes.io/projected/6b96dc78-b389-4764-8ba7-6a4c5e41e345-kube-api-access-hq6bh\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:19 crc kubenswrapper[5052]: I1203 12:27:19.780566 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/6b96dc78-b389-4764-8ba7-6a4c5e41e345-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:19 crc kubenswrapper[5052]: I1203 12:27:19.780576 5052 reconciler_common.go:293] "Volume detached for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/6b96dc78-b389-4764-8ba7-6a4c5e41e345-httpd-run\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:19 crc kubenswrapper[5052]: I1203 12:27:19.803624 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6b96dc78-b389-4764-8ba7-6a4c5e41e345-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "6b96dc78-b389-4764-8ba7-6a4c5e41e345" (UID: "6b96dc78-b389-4764-8ba7-6a4c5e41e345"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:27:19 crc kubenswrapper[5052]: I1203 12:27:19.826799 5052 scope.go:117] "RemoveContainer" containerID="d09526b701062188ac0714b7d2d4c1986577ed904704b893ec88c6d251aa69cb" Dec 03 12:27:19 crc kubenswrapper[5052]: I1203 12:27:19.851895 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage07-crc" (UniqueName: "kubernetes.io/local-volume/local-storage07-crc") on node "crc" Dec 03 12:27:19 crc kubenswrapper[5052]: I1203 12:27:19.893067 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage07-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage07-crc\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:19 crc kubenswrapper[5052]: I1203 12:27:19.893124 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6b96dc78-b389-4764-8ba7-6a4c5e41e345-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:19 crc kubenswrapper[5052]: I1203 12:27:19.894782 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6b96dc78-b389-4764-8ba7-6a4c5e41e345-config-data" (OuterVolumeSpecName: "config-data") pod "6b96dc78-b389-4764-8ba7-6a4c5e41e345" (UID: "6b96dc78-b389-4764-8ba7-6a4c5e41e345"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:27:20 crc kubenswrapper[5052]: I1203 12:27:20.012489 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6b96dc78-b389-4764-8ba7-6a4c5e41e345-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:20 crc kubenswrapper[5052]: I1203 12:27:20.126778 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="51912caa-ca45-4e7c-a0ee-a40ae65a1896" path="/var/lib/kubelet/pods/51912caa-ca45-4e7c-a0ee-a40ae65a1896/volumes" Dec 03 12:27:20 crc kubenswrapper[5052]: I1203 12:27:20.128732 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7e415947-bb79-4d7b-80fb-f6d290343e19" path="/var/lib/kubelet/pods/7e415947-bb79-4d7b-80fb-f6d290343e19/volumes" Dec 03 12:27:20 crc kubenswrapper[5052]: I1203 12:27:20.131945 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7ef14abf-76c0-4db5-9291-e0f9675425d1" path="/var/lib/kubelet/pods/7ef14abf-76c0-4db5-9291-e0f9675425d1/volumes" Dec 03 12:27:20 crc kubenswrapper[5052]: I1203 12:27:20.141734 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a823b6ba-b2af-4deb-88d2-0902fd6e3780" path="/var/lib/kubelet/pods/a823b6ba-b2af-4deb-88d2-0902fd6e3780/volumes" Dec 03 12:27:20 crc kubenswrapper[5052]: I1203 12:27:20.144240 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/barbican-api-78665465bd-9q8ff"] Dec 03 12:27:20 crc kubenswrapper[5052]: I1203 12:27:20.144280 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:27:20 crc kubenswrapper[5052]: I1203 12:27:20.144294 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:27:20 crc kubenswrapper[5052]: I1203 12:27:20.144308 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:27:20 crc kubenswrapper[5052]: I1203 12:27:20.144323 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:27:20 crc kubenswrapper[5052]: E1203 12:27:20.144784 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="6b96dc78-b389-4764-8ba7-6a4c5e41e345" containerName="glance-httpd" Dec 03 12:27:20 crc kubenswrapper[5052]: I1203 12:27:20.144820 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="6b96dc78-b389-4764-8ba7-6a4c5e41e345" containerName="glance-httpd" Dec 03 12:27:20 crc kubenswrapper[5052]: E1203 12:27:20.144859 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="6b96dc78-b389-4764-8ba7-6a4c5e41e345" containerName="glance-log" Dec 03 12:27:20 crc kubenswrapper[5052]: I1203 12:27:20.144867 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="6b96dc78-b389-4764-8ba7-6a4c5e41e345" containerName="glance-log" Dec 03 12:27:20 crc kubenswrapper[5052]: I1203 12:27:20.145110 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="6b96dc78-b389-4764-8ba7-6a4c5e41e345" containerName="glance-httpd" Dec 03 12:27:20 crc kubenswrapper[5052]: I1203 12:27:20.145155 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="6b96dc78-b389-4764-8ba7-6a4c5e41e345" containerName="glance-log" Dec 03 12:27:20 crc kubenswrapper[5052]: I1203 12:27:20.146341 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:27:20 crc kubenswrapper[5052]: I1203 12:27:20.146375 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:27:20 crc kubenswrapper[5052]: I1203 12:27:20.146391 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:27:20 crc kubenswrapper[5052]: I1203 12:27:20.146440 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:27:20 crc kubenswrapper[5052]: I1203 12:27:20.147595 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:27:20 crc kubenswrapper[5052]: I1203 12:27:20.157668 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-default-internal-config-data" Dec 03 12:27:20 crc kubenswrapper[5052]: I1203 12:27:20.160902 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 12:27:20 crc kubenswrapper[5052]: I1203 12:27:20.160948 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:27:20 crc kubenswrapper[5052]: I1203 12:27:20.160962 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/placement-7897d64b6b-j8w2r"] Dec 03 12:27:20 crc kubenswrapper[5052]: I1203 12:27:20.161002 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:27:20 crc kubenswrapper[5052]: I1203 12:27:20.161114 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:27:20 crc kubenswrapper[5052]: I1203 12:27:20.165892 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-metadata-config-data" Dec 03 12:27:20 crc kubenswrapper[5052]: I1203 12:27:20.166623 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-0"] Dec 03 12:27:20 crc kubenswrapper[5052]: W1203 12:27:20.193146 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6436bca9_c7a1_4540_bee3_b6dbabb53c89.slice/crio-3f9c5d49b50567e223e4f8d3fb3e1de27e0f73957b9617b5db32845c00be043e WatchSource:0}: Error finding container 3f9c5d49b50567e223e4f8d3fb3e1de27e0f73957b9617b5db32845c00be043e: Status 404 returned error can't find the container with id 3f9c5d49b50567e223e4f8d3fb3e1de27e0f73957b9617b5db32845c00be043e Dec 03 12:27:20 crc kubenswrapper[5052]: I1203 12:27:20.287419 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:27:20 crc kubenswrapper[5052]: I1203 12:27:20.320131 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7qnb6\" (UniqueName: \"kubernetes.io/projected/8a2c61ae-6521-424f-aeac-52e1ec1c319a-kube-api-access-7qnb6\") pod \"nova-metadata-0\" (UID: \"8a2c61ae-6521-424f-aeac-52e1ec1c319a\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:27:20 crc kubenswrapper[5052]: I1203 12:27:20.320228 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/8a2c61ae-6521-424f-aeac-52e1ec1c319a-logs\") pod \"nova-metadata-0\" (UID: \"8a2c61ae-6521-424f-aeac-52e1ec1c319a\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:27:20 crc kubenswrapper[5052]: I1203 12:27:20.320310 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0b3a2637-ed0d-4348-ac7e-c92db6503191-combined-ca-bundle\") pod \"glance-default-internal-api-0\" (UID: \"0b3a2637-ed0d-4348-ac7e-c92db6503191\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:27:20 crc kubenswrapper[5052]: I1203 12:27:20.320390 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/8a2c61ae-6521-424f-aeac-52e1ec1c319a-config-data\") pod \"nova-metadata-0\" (UID: \"8a2c61ae-6521-424f-aeac-52e1ec1c319a\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:27:20 crc kubenswrapper[5052]: I1203 12:27:20.320446 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zv9nb\" (UniqueName: \"kubernetes.io/projected/0b3a2637-ed0d-4348-ac7e-c92db6503191-kube-api-access-zv9nb\") pod \"glance-default-internal-api-0\" (UID: \"0b3a2637-ed0d-4348-ac7e-c92db6503191\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:27:20 crc kubenswrapper[5052]: I1203 12:27:20.320474 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/0b3a2637-ed0d-4348-ac7e-c92db6503191-httpd-run\") pod \"glance-default-internal-api-0\" (UID: \"0b3a2637-ed0d-4348-ac7e-c92db6503191\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:27:20 crc kubenswrapper[5052]: I1203 12:27:20.320772 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage09-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage09-crc\") pod \"glance-default-internal-api-0\" (UID: \"0b3a2637-ed0d-4348-ac7e-c92db6503191\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:27:20 crc kubenswrapper[5052]: I1203 12:27:20.320841 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/0b3a2637-ed0d-4348-ac7e-c92db6503191-logs\") pod \"glance-default-internal-api-0\" (UID: \"0b3a2637-ed0d-4348-ac7e-c92db6503191\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:27:20 crc kubenswrapper[5052]: I1203 12:27:20.320900 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8a2c61ae-6521-424f-aeac-52e1ec1c319a-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"8a2c61ae-6521-424f-aeac-52e1ec1c319a\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:27:20 crc kubenswrapper[5052]: I1203 12:27:20.320976 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/0b3a2637-ed0d-4348-ac7e-c92db6503191-config-data\") pod \"glance-default-internal-api-0\" (UID: \"0b3a2637-ed0d-4348-ac7e-c92db6503191\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:27:20 crc kubenswrapper[5052]: I1203 12:27:20.320998 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/0b3a2637-ed0d-4348-ac7e-c92db6503191-scripts\") pod \"glance-default-internal-api-0\" (UID: \"0b3a2637-ed0d-4348-ac7e-c92db6503191\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:27:20 crc kubenswrapper[5052]: I1203 12:27:20.423732 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0b3a2637-ed0d-4348-ac7e-c92db6503191-combined-ca-bundle\") pod \"glance-default-internal-api-0\" (UID: \"0b3a2637-ed0d-4348-ac7e-c92db6503191\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:27:20 crc kubenswrapper[5052]: I1203 12:27:20.423795 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/8a2c61ae-6521-424f-aeac-52e1ec1c319a-config-data\") pod \"nova-metadata-0\" (UID: \"8a2c61ae-6521-424f-aeac-52e1ec1c319a\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:27:20 crc kubenswrapper[5052]: I1203 12:27:20.423822 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/0b3a2637-ed0d-4348-ac7e-c92db6503191-httpd-run\") pod \"glance-default-internal-api-0\" (UID: \"0b3a2637-ed0d-4348-ac7e-c92db6503191\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:27:20 crc kubenswrapper[5052]: I1203 12:27:20.423843 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-zv9nb\" (UniqueName: \"kubernetes.io/projected/0b3a2637-ed0d-4348-ac7e-c92db6503191-kube-api-access-zv9nb\") pod \"glance-default-internal-api-0\" (UID: \"0b3a2637-ed0d-4348-ac7e-c92db6503191\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:27:20 crc kubenswrapper[5052]: I1203 12:27:20.423898 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage09-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage09-crc\") pod \"glance-default-internal-api-0\" (UID: \"0b3a2637-ed0d-4348-ac7e-c92db6503191\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:27:20 crc kubenswrapper[5052]: I1203 12:27:20.423928 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/0b3a2637-ed0d-4348-ac7e-c92db6503191-logs\") pod \"glance-default-internal-api-0\" (UID: \"0b3a2637-ed0d-4348-ac7e-c92db6503191\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:27:20 crc kubenswrapper[5052]: I1203 12:27:20.423962 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8a2c61ae-6521-424f-aeac-52e1ec1c319a-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"8a2c61ae-6521-424f-aeac-52e1ec1c319a\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:27:20 crc kubenswrapper[5052]: I1203 12:27:20.424009 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/0b3a2637-ed0d-4348-ac7e-c92db6503191-config-data\") pod \"glance-default-internal-api-0\" (UID: \"0b3a2637-ed0d-4348-ac7e-c92db6503191\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:27:20 crc kubenswrapper[5052]: I1203 12:27:20.424030 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/0b3a2637-ed0d-4348-ac7e-c92db6503191-scripts\") pod \"glance-default-internal-api-0\" (UID: \"0b3a2637-ed0d-4348-ac7e-c92db6503191\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:27:20 crc kubenswrapper[5052]: I1203 12:27:20.424111 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-7qnb6\" (UniqueName: \"kubernetes.io/projected/8a2c61ae-6521-424f-aeac-52e1ec1c319a-kube-api-access-7qnb6\") pod \"nova-metadata-0\" (UID: \"8a2c61ae-6521-424f-aeac-52e1ec1c319a\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:27:20 crc kubenswrapper[5052]: I1203 12:27:20.424162 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/8a2c61ae-6521-424f-aeac-52e1ec1c319a-logs\") pod \"nova-metadata-0\" (UID: \"8a2c61ae-6521-424f-aeac-52e1ec1c319a\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:27:20 crc kubenswrapper[5052]: I1203 12:27:20.424790 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/8a2c61ae-6521-424f-aeac-52e1ec1c319a-logs\") pod \"nova-metadata-0\" (UID: \"8a2c61ae-6521-424f-aeac-52e1ec1c319a\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:27:20 crc kubenswrapper[5052]: I1203 12:27:20.427056 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/0b3a2637-ed0d-4348-ac7e-c92db6503191-logs\") pod \"glance-default-internal-api-0\" (UID: \"0b3a2637-ed0d-4348-ac7e-c92db6503191\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:27:20 crc kubenswrapper[5052]: I1203 12:27:20.430289 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/0b3a2637-ed0d-4348-ac7e-c92db6503191-httpd-run\") pod \"glance-default-internal-api-0\" (UID: \"0b3a2637-ed0d-4348-ac7e-c92db6503191\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:27:20 crc kubenswrapper[5052]: I1203 12:27:20.430289 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage09-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage09-crc\") pod \"glance-default-internal-api-0\" (UID: \"0b3a2637-ed0d-4348-ac7e-c92db6503191\") device mount path \"/mnt/openstack/pv09\"" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:27:20 crc kubenswrapper[5052]: I1203 12:27:20.431436 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0b3a2637-ed0d-4348-ac7e-c92db6503191-combined-ca-bundle\") pod \"glance-default-internal-api-0\" (UID: \"0b3a2637-ed0d-4348-ac7e-c92db6503191\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:27:20 crc kubenswrapper[5052]: I1203 12:27:20.450211 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/8a2c61ae-6521-424f-aeac-52e1ec1c319a-config-data\") pod \"nova-metadata-0\" (UID: \"8a2c61ae-6521-424f-aeac-52e1ec1c319a\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:27:20 crc kubenswrapper[5052]: I1203 12:27:20.457375 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-zv9nb\" (UniqueName: \"kubernetes.io/projected/0b3a2637-ed0d-4348-ac7e-c92db6503191-kube-api-access-zv9nb\") pod \"glance-default-internal-api-0\" (UID: \"0b3a2637-ed0d-4348-ac7e-c92db6503191\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:27:20 crc kubenswrapper[5052]: I1203 12:27:20.458393 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8a2c61ae-6521-424f-aeac-52e1ec1c319a-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"8a2c61ae-6521-424f-aeac-52e1ec1c319a\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:27:20 crc kubenswrapper[5052]: I1203 12:27:20.458600 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/0b3a2637-ed0d-4348-ac7e-c92db6503191-config-data\") pod \"glance-default-internal-api-0\" (UID: \"0b3a2637-ed0d-4348-ac7e-c92db6503191\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:27:20 crc kubenswrapper[5052]: I1203 12:27:20.458798 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/0b3a2637-ed0d-4348-ac7e-c92db6503191-scripts\") pod \"glance-default-internal-api-0\" (UID: \"0b3a2637-ed0d-4348-ac7e-c92db6503191\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:27:20 crc kubenswrapper[5052]: I1203 12:27:20.465702 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-7qnb6\" (UniqueName: \"kubernetes.io/projected/8a2c61ae-6521-424f-aeac-52e1ec1c319a-kube-api-access-7qnb6\") pod \"nova-metadata-0\" (UID: \"8a2c61ae-6521-424f-aeac-52e1ec1c319a\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:27:20 crc kubenswrapper[5052]: I1203 12:27:20.523982 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage09-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage09-crc\") pod \"glance-default-internal-api-0\" (UID: \"0b3a2637-ed0d-4348-ac7e-c92db6503191\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:27:20 crc kubenswrapper[5052]: I1203 12:27:20.716156 5052 generic.go:334] "Generic (PLEG): container finished" podID="6ab5f8d6-e515-4f7b-b14f-383b4180f4e7" containerID="17e6a6fdbd72ea6cd16402a2b231dc4e88079b9588859838d5611cfd4bb232c5" exitCode=0 Dec 03 12:27:20 crc kubenswrapper[5052]: I1203 12:27:20.716243 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"6ab5f8d6-e515-4f7b-b14f-383b4180f4e7","Type":"ContainerDied","Data":"17e6a6fdbd72ea6cd16402a2b231dc4e88079b9588859838d5611cfd4bb232c5"} Dec 03 12:27:20 crc kubenswrapper[5052]: I1203 12:27:20.721479 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-conductor-0" event={"ID":"fcee2113-2b2f-47f9-b118-4f7488ad604e","Type":"ContainerStarted","Data":"21f1f1e7abed6f68ee10cf152087e6c1c6027b4d29dd63cadc265e93aaf59778"} Dec 03 12:27:20 crc kubenswrapper[5052]: I1203 12:27:20.730394 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-6bf55bd9d6-mrgs9" event={"ID":"15cf6575-b6be-4f67-861a-61ebf02d1495","Type":"ContainerStarted","Data":"d5ae8e4de2be17ff4124fe42924a487acef730e0b664ce364b3a370ad4fccabb"} Dec 03 12:27:20 crc kubenswrapper[5052]: I1203 12:27:20.743818 5052 generic.go:334] "Generic (PLEG): container finished" podID="98d0abd5-a20b-45e3-aa30-34f038b2fb3d" containerID="67002cc386bd7047ddbb9535a8f1433365620854531590af15f527d196de051a" exitCode=0 Dec 03 12:27:20 crc kubenswrapper[5052]: I1203 12:27:20.743903 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-keystone-listener-7c58f6698b-95l8v" event={"ID":"98d0abd5-a20b-45e3-aa30-34f038b2fb3d","Type":"ContainerDied","Data":"67002cc386bd7047ddbb9535a8f1433365620854531590af15f527d196de051a"} Dec 03 12:27:20 crc kubenswrapper[5052]: I1203 12:27:20.751203 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-7897d64b6b-j8w2r" event={"ID":"82331185-51eb-4ef5-8cf6-61dc74b73740","Type":"ContainerStarted","Data":"0cb65f56986c143c9460824320cb63c61956b2e8af1869d12d750f07524ab709"} Dec 03 12:27:20 crc kubenswrapper[5052]: I1203 12:27:20.767835 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-5d855df9cb-xdg6p" event={"ID":"3a27c483-00fc-4b06-af6b-3c4b0ff8d0d1","Type":"ContainerStarted","Data":"03ad0a39928a5dbe15709e4ca192498a258450f53a516b11f0805a2e1f446f33"} Dec 03 12:27:20 crc kubenswrapper[5052]: I1203 12:27:20.775752 5052 generic.go:334] "Generic (PLEG): container finished" podID="1cc56b2c-23cc-452f-8de3-d7e8bd2e1e68" containerID="a5ca1b03848c3e7731d3e96fd0f8faa937c1665505435b783c4113d6afa25f56" exitCode=0 Dec 03 12:27:20 crc kubenswrapper[5052]: I1203 12:27:20.776090 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-worker-75f6bbd957-c444v" event={"ID":"1cc56b2c-23cc-452f-8de3-d7e8bd2e1e68","Type":"ContainerDied","Data":"a5ca1b03848c3e7731d3e96fd0f8faa937c1665505435b783c4113d6afa25f56"} Dec 03 12:27:20 crc kubenswrapper[5052]: I1203 12:27:20.778610 5052 generic.go:334] "Generic (PLEG): container finished" podID="cd7b6179-ba41-43bd-bc95-490a88aa15fd" containerID="229c6c7d07d677ea52e66801e341130c260352a54816a6ceeecbd91e2597db8c" exitCode=0 Dec 03 12:27:20 crc kubenswrapper[5052]: I1203 12:27:20.778652 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-server-0" event={"ID":"cd7b6179-ba41-43bd-bc95-490a88aa15fd","Type":"ContainerDied","Data":"229c6c7d07d677ea52e66801e341130c260352a54816a6ceeecbd91e2597db8c"} Dec 03 12:27:20 crc kubenswrapper[5052]: I1203 12:27:20.824489 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-6cdf7689bb-fscpc" event={"ID":"51bb178b-072f-4a7a-928b-5a8845d536b7","Type":"ContainerStarted","Data":"3426f46bd5e0b4a541ed02f94900e4a755d8e523a368d5a2ed4870a257b7116f"} Dec 03 12:27:20 crc kubenswrapper[5052]: I1203 12:27:20.835880 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-keystone-listener-58b6cdf6bb-5xxd8" event={"ID":"71df5bf1-7fd8-4a4d-b0f5-85e3911ae8a9","Type":"ContainerStarted","Data":"f902f915da399e4d8dce5c8b225bbaf37cabe6b67b4273e6cf9af96b3dcda39a"} Dec 03 12:27:20 crc kubenswrapper[5052]: I1203 12:27:20.840386 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"8a0b169d-6971-4530-936e-8e5f0f57816f","Type":"ContainerStarted","Data":"f6f54f7b48fc592006074b7eddf9b7017f72f6b712a3311d0bd99462642c84d1"} Dec 03 12:27:20 crc kubenswrapper[5052]: I1203 12:27:20.843320 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"810e47c2-430b-424d-92ea-66c20ec8501f","Type":"ContainerStarted","Data":"62324b7258f7853ee5e387fa3e5e7fff9881fa6e1b186d82369a9b7c2b057c48"} Dec 03 12:27:20 crc kubenswrapper[5052]: I1203 12:27:20.847196 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"6436bca9-c7a1-4540-bee3-b6dbabb53c89","Type":"ContainerStarted","Data":"3f9c5d49b50567e223e4f8d3fb3e1de27e0f73957b9617b5db32845c00be043e"} Dec 03 12:27:20 crc kubenswrapper[5052]: I1203 12:27:20.932092 5052 scope.go:117] "RemoveContainer" containerID="f4fbadcae236c9bdee4f357a1121e0e675398d9aeeaeec64e67d0177ab9e352c" Dec 03 12:27:21 crc kubenswrapper[5052]: I1203 12:27:21.052312 5052 scope.go:117] "RemoveContainer" containerID="be65e03ff7ec9d6c0e6aa36bae00bb290c5b0d2746c6c491262833760b551397" Dec 03 12:27:21 crc kubenswrapper[5052]: I1203 12:27:21.335124 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:27:21 crc kubenswrapper[5052]: I1203 12:27:21.352306 5052 scope.go:117] "RemoveContainer" containerID="481bbc2057a57984fb7f4ba0b572e86b7319fb1c2d5b5c1d8d86cafb6e88f80f" Dec 03 12:27:21 crc kubenswrapper[5052]: I1203 12:27:21.454894 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"persistence\" (UniqueName: \"kubernetes.io/local-volume/local-storage19-crc\") pod \"cd7b6179-ba41-43bd-bc95-490a88aa15fd\" (UID: \"cd7b6179-ba41-43bd-bc95-490a88aa15fd\") " Dec 03 12:27:21 crc kubenswrapper[5052]: I1203 12:27:21.455215 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"pod-info\" (UniqueName: \"kubernetes.io/downward-api/cd7b6179-ba41-43bd-bc95-490a88aa15fd-pod-info\") pod \"cd7b6179-ba41-43bd-bc95-490a88aa15fd\" (UID: \"cd7b6179-ba41-43bd-bc95-490a88aa15fd\") " Dec 03 12:27:21 crc kubenswrapper[5052]: I1203 12:27:21.455273 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"rabbitmq-erlang-cookie\" (UniqueName: \"kubernetes.io/empty-dir/cd7b6179-ba41-43bd-bc95-490a88aa15fd-rabbitmq-erlang-cookie\") pod \"cd7b6179-ba41-43bd-bc95-490a88aa15fd\" (UID: \"cd7b6179-ba41-43bd-bc95-490a88aa15fd\") " Dec 03 12:27:21 crc kubenswrapper[5052]: I1203 12:27:21.455344 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z5nsk\" (UniqueName: \"kubernetes.io/projected/cd7b6179-ba41-43bd-bc95-490a88aa15fd-kube-api-access-z5nsk\") pod \"cd7b6179-ba41-43bd-bc95-490a88aa15fd\" (UID: \"cd7b6179-ba41-43bd-bc95-490a88aa15fd\") " Dec 03 12:27:21 crc kubenswrapper[5052]: I1203 12:27:21.455397 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"rabbitmq-plugins\" (UniqueName: \"kubernetes.io/empty-dir/cd7b6179-ba41-43bd-bc95-490a88aa15fd-rabbitmq-plugins\") pod \"cd7b6179-ba41-43bd-bc95-490a88aa15fd\" (UID: \"cd7b6179-ba41-43bd-bc95-490a88aa15fd\") " Dec 03 12:27:21 crc kubenswrapper[5052]: I1203 12:27:21.455492 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"rabbitmq-confd\" (UniqueName: \"kubernetes.io/projected/cd7b6179-ba41-43bd-bc95-490a88aa15fd-rabbitmq-confd\") pod \"cd7b6179-ba41-43bd-bc95-490a88aa15fd\" (UID: \"cd7b6179-ba41-43bd-bc95-490a88aa15fd\") " Dec 03 12:27:21 crc kubenswrapper[5052]: I1203 12:27:21.455641 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"erlang-cookie-secret\" (UniqueName: \"kubernetes.io/secret/cd7b6179-ba41-43bd-bc95-490a88aa15fd-erlang-cookie-secret\") pod \"cd7b6179-ba41-43bd-bc95-490a88aa15fd\" (UID: \"cd7b6179-ba41-43bd-bc95-490a88aa15fd\") " Dec 03 12:27:21 crc kubenswrapper[5052]: I1203 12:27:21.455725 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"server-conf\" (UniqueName: \"kubernetes.io/configmap/cd7b6179-ba41-43bd-bc95-490a88aa15fd-server-conf\") pod \"cd7b6179-ba41-43bd-bc95-490a88aa15fd\" (UID: \"cd7b6179-ba41-43bd-bc95-490a88aa15fd\") " Dec 03 12:27:21 crc kubenswrapper[5052]: I1203 12:27:21.455770 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"plugins-conf\" (UniqueName: \"kubernetes.io/configmap/cd7b6179-ba41-43bd-bc95-490a88aa15fd-plugins-conf\") pod \"cd7b6179-ba41-43bd-bc95-490a88aa15fd\" (UID: \"cd7b6179-ba41-43bd-bc95-490a88aa15fd\") " Dec 03 12:27:21 crc kubenswrapper[5052]: I1203 12:27:21.457063 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd7b6179-ba41-43bd-bc95-490a88aa15fd-rabbitmq-plugins" (OuterVolumeSpecName: "rabbitmq-plugins") pod "cd7b6179-ba41-43bd-bc95-490a88aa15fd" (UID: "cd7b6179-ba41-43bd-bc95-490a88aa15fd"). InnerVolumeSpecName "rabbitmq-plugins". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:27:21 crc kubenswrapper[5052]: I1203 12:27:21.457142 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd7b6179-ba41-43bd-bc95-490a88aa15fd-rabbitmq-erlang-cookie" (OuterVolumeSpecName: "rabbitmq-erlang-cookie") pod "cd7b6179-ba41-43bd-bc95-490a88aa15fd" (UID: "cd7b6179-ba41-43bd-bc95-490a88aa15fd"). InnerVolumeSpecName "rabbitmq-erlang-cookie". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:27:21 crc kubenswrapper[5052]: I1203 12:27:21.462445 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cd7b6179-ba41-43bd-bc95-490a88aa15fd-plugins-conf" (OuterVolumeSpecName: "plugins-conf") pod "cd7b6179-ba41-43bd-bc95-490a88aa15fd" (UID: "cd7b6179-ba41-43bd-bc95-490a88aa15fd"). InnerVolumeSpecName "plugins-conf". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:27:21 crc kubenswrapper[5052]: I1203 12:27:21.471046 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cd7b6179-ba41-43bd-bc95-490a88aa15fd-erlang-cookie-secret" (OuterVolumeSpecName: "erlang-cookie-secret") pod "cd7b6179-ba41-43bd-bc95-490a88aa15fd" (UID: "cd7b6179-ba41-43bd-bc95-490a88aa15fd"). InnerVolumeSpecName "erlang-cookie-secret". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:27:21 crc kubenswrapper[5052]: I1203 12:27:21.477468 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage19-crc" (OuterVolumeSpecName: "persistence") pod "cd7b6179-ba41-43bd-bc95-490a88aa15fd" (UID: "cd7b6179-ba41-43bd-bc95-490a88aa15fd"). InnerVolumeSpecName "local-storage19-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 12:27:21 crc kubenswrapper[5052]: I1203 12:27:21.478621 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cd7b6179-ba41-43bd-bc95-490a88aa15fd-kube-api-access-z5nsk" (OuterVolumeSpecName: "kube-api-access-z5nsk") pod "cd7b6179-ba41-43bd-bc95-490a88aa15fd" (UID: "cd7b6179-ba41-43bd-bc95-490a88aa15fd"). InnerVolumeSpecName "kube-api-access-z5nsk". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:27:21 crc kubenswrapper[5052]: I1203 12:27:21.486194 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/cd7b6179-ba41-43bd-bc95-490a88aa15fd-pod-info" (OuterVolumeSpecName: "pod-info") pod "cd7b6179-ba41-43bd-bc95-490a88aa15fd" (UID: "cd7b6179-ba41-43bd-bc95-490a88aa15fd"). InnerVolumeSpecName "pod-info". PluginName "kubernetes.io/downward-api", VolumeGidValue "" Dec 03 12:27:21 crc kubenswrapper[5052]: I1203 12:27:21.567033 5052 reconciler_common.go:293] "Volume detached for volume \"erlang-cookie-secret\" (UniqueName: \"kubernetes.io/secret/cd7b6179-ba41-43bd-bc95-490a88aa15fd-erlang-cookie-secret\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:21 crc kubenswrapper[5052]: I1203 12:27:21.567072 5052 reconciler_common.go:293] "Volume detached for volume \"plugins-conf\" (UniqueName: \"kubernetes.io/configmap/cd7b6179-ba41-43bd-bc95-490a88aa15fd-plugins-conf\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:21 crc kubenswrapper[5052]: I1203 12:27:21.567119 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage19-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage19-crc\") on node \"crc\" " Dec 03 12:27:21 crc kubenswrapper[5052]: I1203 12:27:21.567165 5052 reconciler_common.go:293] "Volume detached for volume \"pod-info\" (UniqueName: \"kubernetes.io/downward-api/cd7b6179-ba41-43bd-bc95-490a88aa15fd-pod-info\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:21 crc kubenswrapper[5052]: I1203 12:27:21.567189 5052 reconciler_common.go:293] "Volume detached for volume \"rabbitmq-erlang-cookie\" (UniqueName: \"kubernetes.io/empty-dir/cd7b6179-ba41-43bd-bc95-490a88aa15fd-rabbitmq-erlang-cookie\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:21 crc kubenswrapper[5052]: I1203 12:27:21.567205 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-z5nsk\" (UniqueName: \"kubernetes.io/projected/cd7b6179-ba41-43bd-bc95-490a88aa15fd-kube-api-access-z5nsk\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:21 crc kubenswrapper[5052]: I1203 12:27:21.567218 5052 reconciler_common.go:293] "Volume detached for volume \"rabbitmq-plugins\" (UniqueName: \"kubernetes.io/empty-dir/cd7b6179-ba41-43bd-bc95-490a88aa15fd-rabbitmq-plugins\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:21 crc kubenswrapper[5052]: I1203 12:27:21.695136 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2825bd7d-50f6-4d2d-827a-e0d307d1b6de" path="/var/lib/kubelet/pods/2825bd7d-50f6-4d2d-827a-e0d307d1b6de/volumes" Dec 03 12:27:21 crc kubenswrapper[5052]: I1203 12:27:21.696280 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7fa6aac1-6ef3-434a-945e-3ced02ca5361" path="/var/lib/kubelet/pods/7fa6aac1-6ef3-434a-945e-3ced02ca5361/volumes" Dec 03 12:27:21 crc kubenswrapper[5052]: I1203 12:27:21.697404 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8e052136-a855-4a20-a8c3-38a8bef97dfc" path="/var/lib/kubelet/pods/8e052136-a855-4a20-a8c3-38a8bef97dfc/volumes" Dec 03 12:27:21 crc kubenswrapper[5052]: I1203 12:27:21.766603 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage19-crc" (UniqueName: "kubernetes.io/local-volume/local-storage19-crc") on node "crc" Dec 03 12:27:21 crc kubenswrapper[5052]: I1203 12:27:21.781978 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage19-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage19-crc\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:21 crc kubenswrapper[5052]: E1203 12:27:21.841524 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of e9967011ef9d1750324e927a4b9e79d05b9a0a1c3f9835799be84de44b11270d is running failed: container process not found" containerID="e9967011ef9d1750324e927a4b9e79d05b9a0a1c3f9835799be84de44b11270d" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:27:21 crc kubenswrapper[5052]: E1203 12:27:21.842059 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of e9967011ef9d1750324e927a4b9e79d05b9a0a1c3f9835799be84de44b11270d is running failed: container process not found" containerID="e9967011ef9d1750324e927a4b9e79d05b9a0a1c3f9835799be84de44b11270d" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:27:21 crc kubenswrapper[5052]: E1203 12:27:21.842840 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of e9967011ef9d1750324e927a4b9e79d05b9a0a1c3f9835799be84de44b11270d is running failed: container process not found" containerID="e9967011ef9d1750324e927a4b9e79d05b9a0a1c3f9835799be84de44b11270d" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:27:21 crc kubenswrapper[5052]: E1203 12:27:21.842891 5052 prober.go:104] "Probe errored" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of e9967011ef9d1750324e927a4b9e79d05b9a0a1c3f9835799be84de44b11270d is running failed: container process not found" probeType="Readiness" pod="openstack-kuttl-tests/nova-cell0-conductor-0" podUID="1f6f8fbd-5810-4853-89c0-4be22abe85a3" containerName="nova-cell0-conductor-conductor" Dec 03 12:27:21 crc kubenswrapper[5052]: I1203 12:27:21.887703 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cd7b6179-ba41-43bd-bc95-490a88aa15fd-server-conf" (OuterVolumeSpecName: "server-conf") pod "cd7b6179-ba41-43bd-bc95-490a88aa15fd" (UID: "cd7b6179-ba41-43bd-bc95-490a88aa15fd"). InnerVolumeSpecName "server-conf". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:27:21 crc kubenswrapper[5052]: I1203 12:27:21.911285 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-cell1-conductor-0" podStartSLOduration=3.911259846 podStartE2EDuration="3.911259846s" podCreationTimestamp="2025-12-03 12:27:18 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:27:21.907812328 +0000 UTC m=+4032.496054122" watchObservedRunningTime="2025-12-03 12:27:21.911259846 +0000 UTC m=+4032.499501610" Dec 03 12:27:21 crc kubenswrapper[5052]: I1203 12:27:21.941366 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:27:21 crc kubenswrapper[5052]: I1203 12:27:21.944277 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:27:21 crc kubenswrapper[5052]: I1203 12:27:21.946289 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:27:21 crc kubenswrapper[5052]: I1203 12:27:21.946318 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-conductor-0" event={"ID":"fcee2113-2b2f-47f9-b118-4f7488ad604e","Type":"ContainerStarted","Data":"df1192fd6167f45271208db821d1f0909fc60a2b10aada89f5e4d0015c104317"} Dec 03 12:27:21 crc kubenswrapper[5052]: I1203 12:27:21.946338 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-6bf55bd9d6-mrgs9" event={"ID":"15cf6575-b6be-4f67-861a-61ebf02d1495","Type":"ContainerStarted","Data":"6b5f5843b9647706bd37b6d16a4de3c4357a058f24ee264c620b57797b2edcdc"} Dec 03 12:27:21 crc kubenswrapper[5052]: I1203 12:27:21.946351 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-keystone-listener-7c58f6698b-95l8v" event={"ID":"98d0abd5-a20b-45e3-aa30-34f038b2fb3d","Type":"ContainerDied","Data":"8bf3eaccebc33096e14c55c4d10d9463222d82d8e173cb6c456372d194859d9f"} Dec 03 12:27:21 crc kubenswrapper[5052]: I1203 12:27:21.946367 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8bf3eaccebc33096e14c55c4d10d9463222d82d8e173cb6c456372d194859d9f" Dec 03 12:27:21 crc kubenswrapper[5052]: I1203 12:27:21.946379 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"6ab5f8d6-e515-4f7b-b14f-383b4180f4e7","Type":"ContainerDied","Data":"c8890686d3840eb2ccf4444693da2e2e55acbf4b6e5ded2e5bd7bee34eed05de"} Dec 03 12:27:21 crc kubenswrapper[5052]: I1203 12:27:21.946391 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c8890686d3840eb2ccf4444693da2e2e55acbf4b6e5ded2e5bd7bee34eed05de" Dec 03 12:27:21 crc kubenswrapper[5052]: I1203 12:27:21.965769 5052 generic.go:334] "Generic (PLEG): container finished" podID="1f6f8fbd-5810-4853-89c0-4be22abe85a3" containerID="e9967011ef9d1750324e927a4b9e79d05b9a0a1c3f9835799be84de44b11270d" exitCode=0 Dec 03 12:27:21 crc kubenswrapper[5052]: I1203 12:27:21.965843 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-conductor-0" event={"ID":"1f6f8fbd-5810-4853-89c0-4be22abe85a3","Type":"ContainerDied","Data":"e9967011ef9d1750324e927a4b9e79d05b9a0a1c3f9835799be84de44b11270d"} Dec 03 12:27:21 crc kubenswrapper[5052]: I1203 12:27:21.965876 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-conductor-0" event={"ID":"1f6f8fbd-5810-4853-89c0-4be22abe85a3","Type":"ContainerDied","Data":"17266566b416c0315caf9c02a6305bd7d5912d119139bff52a6a18129c6144aa"} Dec 03 12:27:21 crc kubenswrapper[5052]: I1203 12:27:21.965889 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="17266566b416c0315caf9c02a6305bd7d5912d119139bff52a6a18129c6144aa" Dec 03 12:27:21 crc kubenswrapper[5052]: I1203 12:27:21.969718 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cd7b6179-ba41-43bd-bc95-490a88aa15fd-rabbitmq-confd" (OuterVolumeSpecName: "rabbitmq-confd") pod "cd7b6179-ba41-43bd-bc95-490a88aa15fd" (UID: "cd7b6179-ba41-43bd-bc95-490a88aa15fd"). InnerVolumeSpecName "rabbitmq-confd". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:27:21 crc kubenswrapper[5052]: I1203 12:27:21.981943 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-keystone-listener-7c58f6698b-95l8v" Dec 03 12:27:21 crc kubenswrapper[5052]: I1203 12:27:21.984824 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-worker-75f6bbd957-c444v" event={"ID":"1cc56b2c-23cc-452f-8de3-d7e8bd2e1e68","Type":"ContainerDied","Data":"1b9e8ca97381e26e986252ff2a28872074667e0e4d1dc2c14a9ef61486656a82"} Dec 03 12:27:21 crc kubenswrapper[5052]: I1203 12:27:21.984864 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1b9e8ca97381e26e986252ff2a28872074667e0e4d1dc2c14a9ef61486656a82" Dec 03 12:27:21 crc kubenswrapper[5052]: I1203 12:27:21.985979 5052 reconciler_common.go:293] "Volume detached for volume \"rabbitmq-confd\" (UniqueName: \"kubernetes.io/projected/cd7b6179-ba41-43bd-bc95-490a88aa15fd-rabbitmq-confd\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:21 crc kubenswrapper[5052]: I1203 12:27:21.986011 5052 reconciler_common.go:293] "Volume detached for volume \"server-conf\" (UniqueName: \"kubernetes.io/configmap/cd7b6179-ba41-43bd-bc95-490a88aa15fd-server-conf\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:22 crc kubenswrapper[5052]: I1203 12:27:22.004902 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:27:22 crc kubenswrapper[5052]: I1203 12:27:22.004999 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-server-0" event={"ID":"cd7b6179-ba41-43bd-bc95-490a88aa15fd","Type":"ContainerDied","Data":"262884bbd4520dabfeb61e2479101855a66a0487971cc3453cf14b95370baa1a"} Dec 03 12:27:22 crc kubenswrapper[5052]: I1203 12:27:22.064551 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"8a0b169d-6971-4530-936e-8e5f0f57816f","Type":"ContainerStarted","Data":"47d8a341afac3d19e0311930b0aa745891784bd644a5b9f1e35346d168347db9"} Dec 03 12:27:22 crc kubenswrapper[5052]: I1203 12:27:22.088192 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/98d0abd5-a20b-45e3-aa30-34f038b2fb3d-logs\") pod \"98d0abd5-a20b-45e3-aa30-34f038b2fb3d\" (UID: \"98d0abd5-a20b-45e3-aa30-34f038b2fb3d\") " Dec 03 12:27:22 crc kubenswrapper[5052]: I1203 12:27:22.088401 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-74wdj\" (UniqueName: \"kubernetes.io/projected/98d0abd5-a20b-45e3-aa30-34f038b2fb3d-kube-api-access-74wdj\") pod \"98d0abd5-a20b-45e3-aa30-34f038b2fb3d\" (UID: \"98d0abd5-a20b-45e3-aa30-34f038b2fb3d\") " Dec 03 12:27:22 crc kubenswrapper[5052]: I1203 12:27:22.088727 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/98d0abd5-a20b-45e3-aa30-34f038b2fb3d-config-data\") pod \"98d0abd5-a20b-45e3-aa30-34f038b2fb3d\" (UID: \"98d0abd5-a20b-45e3-aa30-34f038b2fb3d\") " Dec 03 12:27:22 crc kubenswrapper[5052]: I1203 12:27:22.089076 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/98d0abd5-a20b-45e3-aa30-34f038b2fb3d-config-data-custom\") pod \"98d0abd5-a20b-45e3-aa30-34f038b2fb3d\" (UID: \"98d0abd5-a20b-45e3-aa30-34f038b2fb3d\") " Dec 03 12:27:22 crc kubenswrapper[5052]: I1203 12:27:22.089144 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/98d0abd5-a20b-45e3-aa30-34f038b2fb3d-combined-ca-bundle\") pod \"98d0abd5-a20b-45e3-aa30-34f038b2fb3d\" (UID: \"98d0abd5-a20b-45e3-aa30-34f038b2fb3d\") " Dec 03 12:27:22 crc kubenswrapper[5052]: I1203 12:27:22.089767 5052 scope.go:117] "RemoveContainer" containerID="0e8b8edad48c623ae81447848a940a98d6827814b85cda0ff7b96abb10a64815" Dec 03 12:27:22 crc kubenswrapper[5052]: I1203 12:27:22.090912 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/98d0abd5-a20b-45e3-aa30-34f038b2fb3d-logs" (OuterVolumeSpecName: "logs") pod "98d0abd5-a20b-45e3-aa30-34f038b2fb3d" (UID: "98d0abd5-a20b-45e3-aa30-34f038b2fb3d"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:27:22 crc kubenswrapper[5052]: I1203 12:27:22.092465 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:27:22 crc kubenswrapper[5052]: I1203 12:27:22.092606 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-worker-75f6bbd957-c444v" Dec 03 12:27:22 crc kubenswrapper[5052]: I1203 12:27:22.094046 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-worker-679bfdf7b7-tg6h7" event={"ID":"72a32b4d-d61d-43a5-a45d-c1fd4bad672d","Type":"ContainerStarted","Data":"902a5085d0a5d4b6d6bd379e7d79548b8cf29e5be3d818b61c582ef847c19f19"} Dec 03 12:27:22 crc kubenswrapper[5052]: I1203 12:27:22.094199 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:27:22 crc kubenswrapper[5052]: I1203 12:27:22.094934 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/98d0abd5-a20b-45e3-aa30-34f038b2fb3d-config-data-custom" (OuterVolumeSpecName: "config-data-custom") pod "98d0abd5-a20b-45e3-aa30-34f038b2fb3d" (UID: "98d0abd5-a20b-45e3-aa30-34f038b2fb3d"). InnerVolumeSpecName "config-data-custom". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:27:22 crc kubenswrapper[5052]: I1203 12:27:22.111952 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:27:22 crc kubenswrapper[5052]: I1203 12:27:22.120550 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/rabbitmq-server-0"] Dec 03 12:27:22 crc kubenswrapper[5052]: I1203 12:27:22.129878 5052 generic.go:334] "Generic (PLEG): container finished" podID="5a5320b7-0541-483a-8e0b-78c10d0f8efa" containerID="032bc0ff1bc0c7adb6371d41b7d1d30531bb08512a12516306ce34a00afcaad9" exitCode=0 Dec 03 12:27:22 crc kubenswrapper[5052]: I1203 12:27:22.129935 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" event={"ID":"5a5320b7-0541-483a-8e0b-78c10d0f8efa","Type":"ContainerDied","Data":"032bc0ff1bc0c7adb6371d41b7d1d30531bb08512a12516306ce34a00afcaad9"} Dec 03 12:27:22 crc kubenswrapper[5052]: I1203 12:27:22.129969 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" event={"ID":"5a5320b7-0541-483a-8e0b-78c10d0f8efa","Type":"ContainerDied","Data":"f7065d25f09c5b8035f424a2ddebcac36137d019bbf23797b8768ad93bdeb0a5"} Dec 03 12:27:22 crc kubenswrapper[5052]: I1203 12:27:22.141846 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/98d0abd5-a20b-45e3-aa30-34f038b2fb3d-kube-api-access-74wdj" (OuterVolumeSpecName: "kube-api-access-74wdj") pod "98d0abd5-a20b-45e3-aa30-34f038b2fb3d" (UID: "98d0abd5-a20b-45e3-aa30-34f038b2fb3d"). InnerVolumeSpecName "kube-api-access-74wdj". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:27:22 crc kubenswrapper[5052]: I1203 12:27:22.148218 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/rabbitmq-server-0"] Dec 03 12:27:22 crc kubenswrapper[5052]: I1203 12:27:22.187006 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/rabbitmq-server-0"] Dec 03 12:27:22 crc kubenswrapper[5052]: E1203 12:27:22.187380 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="6ab5f8d6-e515-4f7b-b14f-383b4180f4e7" containerName="ceilometer-notification-agent" Dec 03 12:27:22 crc kubenswrapper[5052]: I1203 12:27:22.187397 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="6ab5f8d6-e515-4f7b-b14f-383b4180f4e7" containerName="ceilometer-notification-agent" Dec 03 12:27:22 crc kubenswrapper[5052]: E1203 12:27:22.187410 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="6ab5f8d6-e515-4f7b-b14f-383b4180f4e7" containerName="proxy-httpd" Dec 03 12:27:22 crc kubenswrapper[5052]: I1203 12:27:22.187419 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="6ab5f8d6-e515-4f7b-b14f-383b4180f4e7" containerName="proxy-httpd" Dec 03 12:27:22 crc kubenswrapper[5052]: E1203 12:27:22.187432 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="98d0abd5-a20b-45e3-aa30-34f038b2fb3d" containerName="barbican-keystone-listener-log" Dec 03 12:27:22 crc kubenswrapper[5052]: I1203 12:27:22.187439 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="98d0abd5-a20b-45e3-aa30-34f038b2fb3d" containerName="barbican-keystone-listener-log" Dec 03 12:27:22 crc kubenswrapper[5052]: E1203 12:27:22.187450 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="6ab5f8d6-e515-4f7b-b14f-383b4180f4e7" containerName="ceilometer-central-agent" Dec 03 12:27:22 crc kubenswrapper[5052]: I1203 12:27:22.187456 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="6ab5f8d6-e515-4f7b-b14f-383b4180f4e7" containerName="ceilometer-central-agent" Dec 03 12:27:22 crc kubenswrapper[5052]: E1203 12:27:22.187469 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="5a5320b7-0541-483a-8e0b-78c10d0f8efa" containerName="rabbitmq" Dec 03 12:27:22 crc kubenswrapper[5052]: I1203 12:27:22.187475 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="5a5320b7-0541-483a-8e0b-78c10d0f8efa" containerName="rabbitmq" Dec 03 12:27:22 crc kubenswrapper[5052]: E1203 12:27:22.187486 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="cd7b6179-ba41-43bd-bc95-490a88aa15fd" containerName="rabbitmq" Dec 03 12:27:22 crc kubenswrapper[5052]: I1203 12:27:22.187492 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="cd7b6179-ba41-43bd-bc95-490a88aa15fd" containerName="rabbitmq" Dec 03 12:27:22 crc kubenswrapper[5052]: E1203 12:27:22.187520 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="1f6f8fbd-5810-4853-89c0-4be22abe85a3" containerName="nova-cell0-conductor-conductor" Dec 03 12:27:22 crc kubenswrapper[5052]: I1203 12:27:22.187526 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f6f8fbd-5810-4853-89c0-4be22abe85a3" containerName="nova-cell0-conductor-conductor" Dec 03 12:27:22 crc kubenswrapper[5052]: E1203 12:27:22.187535 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="1cc56b2c-23cc-452f-8de3-d7e8bd2e1e68" containerName="barbican-worker" Dec 03 12:27:22 crc kubenswrapper[5052]: I1203 12:27:22.187540 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="1cc56b2c-23cc-452f-8de3-d7e8bd2e1e68" containerName="barbican-worker" Dec 03 12:27:22 crc kubenswrapper[5052]: E1203 12:27:22.187551 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="5a5320b7-0541-483a-8e0b-78c10d0f8efa" containerName="setup-container" Dec 03 12:27:22 crc kubenswrapper[5052]: I1203 12:27:22.187558 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="5a5320b7-0541-483a-8e0b-78c10d0f8efa" containerName="setup-container" Dec 03 12:27:22 crc kubenswrapper[5052]: E1203 12:27:22.187572 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="6ab5f8d6-e515-4f7b-b14f-383b4180f4e7" containerName="sg-core" Dec 03 12:27:22 crc kubenswrapper[5052]: I1203 12:27:22.187578 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="6ab5f8d6-e515-4f7b-b14f-383b4180f4e7" containerName="sg-core" Dec 03 12:27:22 crc kubenswrapper[5052]: E1203 12:27:22.187585 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="cd7b6179-ba41-43bd-bc95-490a88aa15fd" containerName="setup-container" Dec 03 12:27:22 crc kubenswrapper[5052]: I1203 12:27:22.187591 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="cd7b6179-ba41-43bd-bc95-490a88aa15fd" containerName="setup-container" Dec 03 12:27:22 crc kubenswrapper[5052]: E1203 12:27:22.187601 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="1cc56b2c-23cc-452f-8de3-d7e8bd2e1e68" containerName="barbican-worker-log" Dec 03 12:27:22 crc kubenswrapper[5052]: I1203 12:27:22.187607 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="1cc56b2c-23cc-452f-8de3-d7e8bd2e1e68" containerName="barbican-worker-log" Dec 03 12:27:22 crc kubenswrapper[5052]: E1203 12:27:22.187615 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="98d0abd5-a20b-45e3-aa30-34f038b2fb3d" containerName="barbican-keystone-listener" Dec 03 12:27:22 crc kubenswrapper[5052]: I1203 12:27:22.187621 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="98d0abd5-a20b-45e3-aa30-34f038b2fb3d" containerName="barbican-keystone-listener" Dec 03 12:27:22 crc kubenswrapper[5052]: I1203 12:27:22.187962 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="cd7b6179-ba41-43bd-bc95-490a88aa15fd" containerName="rabbitmq" Dec 03 12:27:22 crc kubenswrapper[5052]: I1203 12:27:22.187982 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="1cc56b2c-23cc-452f-8de3-d7e8bd2e1e68" containerName="barbican-worker-log" Dec 03 12:27:22 crc kubenswrapper[5052]: I1203 12:27:22.187995 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="6ab5f8d6-e515-4f7b-b14f-383b4180f4e7" containerName="proxy-httpd" Dec 03 12:27:22 crc kubenswrapper[5052]: I1203 12:27:22.188008 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="5a5320b7-0541-483a-8e0b-78c10d0f8efa" containerName="rabbitmq" Dec 03 12:27:22 crc kubenswrapper[5052]: I1203 12:27:22.188015 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="1cc56b2c-23cc-452f-8de3-d7e8bd2e1e68" containerName="barbican-worker" Dec 03 12:27:22 crc kubenswrapper[5052]: I1203 12:27:22.188022 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="6ab5f8d6-e515-4f7b-b14f-383b4180f4e7" containerName="ceilometer-central-agent" Dec 03 12:27:22 crc kubenswrapper[5052]: I1203 12:27:22.188034 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="1f6f8fbd-5810-4853-89c0-4be22abe85a3" containerName="nova-cell0-conductor-conductor" Dec 03 12:27:22 crc kubenswrapper[5052]: I1203 12:27:22.188042 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="6ab5f8d6-e515-4f7b-b14f-383b4180f4e7" containerName="ceilometer-notification-agent" Dec 03 12:27:22 crc kubenswrapper[5052]: I1203 12:27:22.188052 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="98d0abd5-a20b-45e3-aa30-34f038b2fb3d" containerName="barbican-keystone-listener" Dec 03 12:27:22 crc kubenswrapper[5052]: I1203 12:27:22.188063 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="98d0abd5-a20b-45e3-aa30-34f038b2fb3d" containerName="barbican-keystone-listener-log" Dec 03 12:27:22 crc kubenswrapper[5052]: I1203 12:27:22.188073 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="6ab5f8d6-e515-4f7b-b14f-383b4180f4e7" containerName="sg-core" Dec 03 12:27:22 crc kubenswrapper[5052]: I1203 12:27:22.189033 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:27:22 crc kubenswrapper[5052]: I1203 12:27:22.195620 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"rabbitmq-plugins-conf" Dec 03 12:27:22 crc kubenswrapper[5052]: I1203 12:27:22.195855 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"rabbitmq-server-conf" Dec 03 12:27:22 crc kubenswrapper[5052]: I1203 12:27:22.196041 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"rabbitmq-erlang-cookie" Dec 03 12:27:22 crc kubenswrapper[5052]: I1203 12:27:22.196210 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"rabbitmq-config-data" Dec 03 12:27:22 crc kubenswrapper[5052]: I1203 12:27:22.196436 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-rabbitmq-svc" Dec 03 12:27:22 crc kubenswrapper[5052]: I1203 12:27:22.196655 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"rabbitmq-default-user" Dec 03 12:27:22 crc kubenswrapper[5052]: I1203 12:27:22.196868 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"rabbitmq-server-dockercfg-56zhj" Dec 03 12:27:22 crc kubenswrapper[5052]: I1203 12:27:22.197382 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-scheduler-0" podStartSLOduration=5.197361232 podStartE2EDuration="5.197361232s" podCreationTimestamp="2025-12-03 12:27:17 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:27:22.111941594 +0000 UTC m=+4032.700183358" watchObservedRunningTime="2025-12-03 12:27:22.197361232 +0000 UTC m=+4032.785602996" Dec 03 12:27:22 crc kubenswrapper[5052]: I1203 12:27:22.201637 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/1f6f8fbd-5810-4853-89c0-4be22abe85a3-config-data\") pod \"1f6f8fbd-5810-4853-89c0-4be22abe85a3\" (UID: \"1f6f8fbd-5810-4853-89c0-4be22abe85a3\") " Dec 03 12:27:22 crc kubenswrapper[5052]: I1203 12:27:22.201821 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6ab5f8d6-e515-4f7b-b14f-383b4180f4e7-combined-ca-bundle\") pod \"6ab5f8d6-e515-4f7b-b14f-383b4180f4e7\" (UID: \"6ab5f8d6-e515-4f7b-b14f-383b4180f4e7\") " Dec 03 12:27:22 crc kubenswrapper[5052]: I1203 12:27:22.201929 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/1cc56b2c-23cc-452f-8de3-d7e8bd2e1e68-config-data\") pod \"1cc56b2c-23cc-452f-8de3-d7e8bd2e1e68\" (UID: \"1cc56b2c-23cc-452f-8de3-d7e8bd2e1e68\") " Dec 03 12:27:22 crc kubenswrapper[5052]: I1203 12:27:22.202033 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6ab5f8d6-e515-4f7b-b14f-383b4180f4e7-config-data\") pod \"6ab5f8d6-e515-4f7b-b14f-383b4180f4e7\" (UID: \"6ab5f8d6-e515-4f7b-b14f-383b4180f4e7\") " Dec 03 12:27:22 crc kubenswrapper[5052]: I1203 12:27:22.202152 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8r2b9\" (UniqueName: \"kubernetes.io/projected/1f6f8fbd-5810-4853-89c0-4be22abe85a3-kube-api-access-8r2b9\") pod \"1f6f8fbd-5810-4853-89c0-4be22abe85a3\" (UID: \"1f6f8fbd-5810-4853-89c0-4be22abe85a3\") " Dec 03 12:27:22 crc kubenswrapper[5052]: I1203 12:27:22.202315 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"rabbitmq-confd\" (UniqueName: \"kubernetes.io/projected/5a5320b7-0541-483a-8e0b-78c10d0f8efa-rabbitmq-confd\") pod \"5a5320b7-0541-483a-8e0b-78c10d0f8efa\" (UID: \"5a5320b7-0541-483a-8e0b-78c10d0f8efa\") " Dec 03 12:27:22 crc kubenswrapper[5052]: I1203 12:27:22.202425 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"pod-info\" (UniqueName: \"kubernetes.io/downward-api/5a5320b7-0541-483a-8e0b-78c10d0f8efa-pod-info\") pod \"5a5320b7-0541-483a-8e0b-78c10d0f8efa\" (UID: \"5a5320b7-0541-483a-8e0b-78c10d0f8efa\") " Dec 03 12:27:22 crc kubenswrapper[5052]: I1203 12:27:22.202619 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"rabbitmq-plugins\" (UniqueName: \"kubernetes.io/empty-dir/5a5320b7-0541-483a-8e0b-78c10d0f8efa-rabbitmq-plugins\") pod \"5a5320b7-0541-483a-8e0b-78c10d0f8efa\" (UID: \"5a5320b7-0541-483a-8e0b-78c10d0f8efa\") " Dec 03 12:27:22 crc kubenswrapper[5052]: I1203 12:27:22.202786 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"server-conf\" (UniqueName: \"kubernetes.io/configmap/5a5320b7-0541-483a-8e0b-78c10d0f8efa-server-conf\") pod \"5a5320b7-0541-483a-8e0b-78c10d0f8efa\" (UID: \"5a5320b7-0541-483a-8e0b-78c10d0f8efa\") " Dec 03 12:27:22 crc kubenswrapper[5052]: I1203 12:27:22.202921 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"erlang-cookie-secret\" (UniqueName: \"kubernetes.io/secret/5a5320b7-0541-483a-8e0b-78c10d0f8efa-erlang-cookie-secret\") pod \"5a5320b7-0541-483a-8e0b-78c10d0f8efa\" (UID: \"5a5320b7-0541-483a-8e0b-78c10d0f8efa\") " Dec 03 12:27:22 crc kubenswrapper[5052]: I1203 12:27:22.203034 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/1cc56b2c-23cc-452f-8de3-d7e8bd2e1e68-logs\") pod \"1cc56b2c-23cc-452f-8de3-d7e8bd2e1e68\" (UID: \"1cc56b2c-23cc-452f-8de3-d7e8bd2e1e68\") " Dec 03 12:27:22 crc kubenswrapper[5052]: I1203 12:27:22.203133 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/6ab5f8d6-e515-4f7b-b14f-383b4180f4e7-log-httpd\") pod \"6ab5f8d6-e515-4f7b-b14f-383b4180f4e7\" (UID: \"6ab5f8d6-e515-4f7b-b14f-383b4180f4e7\") " Dec 03 12:27:22 crc kubenswrapper[5052]: I1203 12:27:22.203266 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1f6f8fbd-5810-4853-89c0-4be22abe85a3-combined-ca-bundle\") pod \"1f6f8fbd-5810-4853-89c0-4be22abe85a3\" (UID: \"1f6f8fbd-5810-4853-89c0-4be22abe85a3\") " Dec 03 12:27:22 crc kubenswrapper[5052]: I1203 12:27:22.208896 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"rabbitmq-erlang-cookie\" (UniqueName: \"kubernetes.io/empty-dir/5a5320b7-0541-483a-8e0b-78c10d0f8efa-rabbitmq-erlang-cookie\") pod \"5a5320b7-0541-483a-8e0b-78c10d0f8efa\" (UID: \"5a5320b7-0541-483a-8e0b-78c10d0f8efa\") " Dec 03 12:27:22 crc kubenswrapper[5052]: I1203 12:27:22.209049 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/6ab5f8d6-e515-4f7b-b14f-383b4180f4e7-scripts\") pod \"6ab5f8d6-e515-4f7b-b14f-383b4180f4e7\" (UID: \"6ab5f8d6-e515-4f7b-b14f-383b4180f4e7\") " Dec 03 12:27:22 crc kubenswrapper[5052]: I1203 12:27:22.209151 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"plugins-conf\" (UniqueName: \"kubernetes.io/configmap/5a5320b7-0541-483a-8e0b-78c10d0f8efa-plugins-conf\") pod \"5a5320b7-0541-483a-8e0b-78c10d0f8efa\" (UID: \"5a5320b7-0541-483a-8e0b-78c10d0f8efa\") " Dec 03 12:27:22 crc kubenswrapper[5052]: I1203 12:27:22.209357 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vcl4m\" (UniqueName: \"kubernetes.io/projected/1cc56b2c-23cc-452f-8de3-d7e8bd2e1e68-kube-api-access-vcl4m\") pod \"1cc56b2c-23cc-452f-8de3-d7e8bd2e1e68\" (UID: \"1cc56b2c-23cc-452f-8de3-d7e8bd2e1e68\") " Dec 03 12:27:22 crc kubenswrapper[5052]: I1203 12:27:22.209487 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9mhgt\" (UniqueName: \"kubernetes.io/projected/6ab5f8d6-e515-4f7b-b14f-383b4180f4e7-kube-api-access-9mhgt\") pod \"6ab5f8d6-e515-4f7b-b14f-383b4180f4e7\" (UID: \"6ab5f8d6-e515-4f7b-b14f-383b4180f4e7\") " Dec 03 12:27:22 crc kubenswrapper[5052]: I1203 12:27:22.209787 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1cc56b2c-23cc-452f-8de3-d7e8bd2e1e68-combined-ca-bundle\") pod \"1cc56b2c-23cc-452f-8de3-d7e8bd2e1e68\" (UID: \"1cc56b2c-23cc-452f-8de3-d7e8bd2e1e68\") " Dec 03 12:27:22 crc kubenswrapper[5052]: I1203 12:27:22.209910 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"persistence\" (UniqueName: \"kubernetes.io/local-volume/local-storage01-crc\") pod \"5a5320b7-0541-483a-8e0b-78c10d0f8efa\" (UID: \"5a5320b7-0541-483a-8e0b-78c10d0f8efa\") " Dec 03 12:27:22 crc kubenswrapper[5052]: I1203 12:27:22.210019 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/6ab5f8d6-e515-4f7b-b14f-383b4180f4e7-run-httpd\") pod \"6ab5f8d6-e515-4f7b-b14f-383b4180f4e7\" (UID: \"6ab5f8d6-e515-4f7b-b14f-383b4180f4e7\") " Dec 03 12:27:22 crc kubenswrapper[5052]: I1203 12:27:22.210120 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/1cc56b2c-23cc-452f-8de3-d7e8bd2e1e68-config-data-custom\") pod \"1cc56b2c-23cc-452f-8de3-d7e8bd2e1e68\" (UID: \"1cc56b2c-23cc-452f-8de3-d7e8bd2e1e68\") " Dec 03 12:27:22 crc kubenswrapper[5052]: I1203 12:27:22.210236 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-95l4t\" (UniqueName: \"kubernetes.io/projected/5a5320b7-0541-483a-8e0b-78c10d0f8efa-kube-api-access-95l4t\") pod \"5a5320b7-0541-483a-8e0b-78c10d0f8efa\" (UID: \"5a5320b7-0541-483a-8e0b-78c10d0f8efa\") " Dec 03 12:27:22 crc kubenswrapper[5052]: I1203 12:27:22.210353 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/6ab5f8d6-e515-4f7b-b14f-383b4180f4e7-sg-core-conf-yaml\") pod \"6ab5f8d6-e515-4f7b-b14f-383b4180f4e7\" (UID: \"6ab5f8d6-e515-4f7b-b14f-383b4180f4e7\") " Dec 03 12:27:22 crc kubenswrapper[5052]: I1203 12:27:22.211259 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/98d0abd5-a20b-45e3-aa30-34f038b2fb3d-config-data-custom\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:22 crc kubenswrapper[5052]: I1203 12:27:22.220263 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/98d0abd5-a20b-45e3-aa30-34f038b2fb3d-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:22 crc kubenswrapper[5052]: I1203 12:27:22.220530 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-74wdj\" (UniqueName: \"kubernetes.io/projected/98d0abd5-a20b-45e3-aa30-34f038b2fb3d-kube-api-access-74wdj\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:22 crc kubenswrapper[5052]: I1203 12:27:22.211896 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5a5320b7-0541-483a-8e0b-78c10d0f8efa-rabbitmq-plugins" (OuterVolumeSpecName: "rabbitmq-plugins") pod "5a5320b7-0541-483a-8e0b-78c10d0f8efa" (UID: "5a5320b7-0541-483a-8e0b-78c10d0f8efa"). InnerVolumeSpecName "rabbitmq-plugins". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:27:22 crc kubenswrapper[5052]: I1203 12:27:22.216545 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5a5320b7-0541-483a-8e0b-78c10d0f8efa-rabbitmq-erlang-cookie" (OuterVolumeSpecName: "rabbitmq-erlang-cookie") pod "5a5320b7-0541-483a-8e0b-78c10d0f8efa" (UID: "5a5320b7-0541-483a-8e0b-78c10d0f8efa"). InnerVolumeSpecName "rabbitmq-erlang-cookie". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:27:22 crc kubenswrapper[5052]: I1203 12:27:22.218204 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1cc56b2c-23cc-452f-8de3-d7e8bd2e1e68-logs" (OuterVolumeSpecName: "logs") pod "1cc56b2c-23cc-452f-8de3-d7e8bd2e1e68" (UID: "1cc56b2c-23cc-452f-8de3-d7e8bd2e1e68"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:27:22 crc kubenswrapper[5052]: I1203 12:27:22.225140 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6ab5f8d6-e515-4f7b-b14f-383b4180f4e7-log-httpd" (OuterVolumeSpecName: "log-httpd") pod "6ab5f8d6-e515-4f7b-b14f-383b4180f4e7" (UID: "6ab5f8d6-e515-4f7b-b14f-383b4180f4e7"). InnerVolumeSpecName "log-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:27:22 crc kubenswrapper[5052]: I1203 12:27:22.227596 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6ab5f8d6-e515-4f7b-b14f-383b4180f4e7-run-httpd" (OuterVolumeSpecName: "run-httpd") pod "6ab5f8d6-e515-4f7b-b14f-383b4180f4e7" (UID: "6ab5f8d6-e515-4f7b-b14f-383b4180f4e7"). InnerVolumeSpecName "run-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:27:22 crc kubenswrapper[5052]: I1203 12:27:22.227891 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5a5320b7-0541-483a-8e0b-78c10d0f8efa-plugins-conf" (OuterVolumeSpecName: "plugins-conf") pod "5a5320b7-0541-483a-8e0b-78c10d0f8efa" (UID: "5a5320b7-0541-483a-8e0b-78c10d0f8efa"). InnerVolumeSpecName "plugins-conf". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:27:22 crc kubenswrapper[5052]: I1203 12:27:22.230576 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/rabbitmq-server-0"] Dec 03 12:27:22 crc kubenswrapper[5052]: I1203 12:27:22.257453 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5a5320b7-0541-483a-8e0b-78c10d0f8efa-pod-info" (OuterVolumeSpecName: "pod-info") pod "5a5320b7-0541-483a-8e0b-78c10d0f8efa" (UID: "5a5320b7-0541-483a-8e0b-78c10d0f8efa"). InnerVolumeSpecName "pod-info". PluginName "kubernetes.io/downward-api", VolumeGidValue "" Dec 03 12:27:22 crc kubenswrapper[5052]: I1203 12:27:22.282240 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6ab5f8d6-e515-4f7b-b14f-383b4180f4e7-kube-api-access-9mhgt" (OuterVolumeSpecName: "kube-api-access-9mhgt") pod "6ab5f8d6-e515-4f7b-b14f-383b4180f4e7" (UID: "6ab5f8d6-e515-4f7b-b14f-383b4180f4e7"). InnerVolumeSpecName "kube-api-access-9mhgt". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:27:22 crc kubenswrapper[5052]: I1203 12:27:22.295180 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1f6f8fbd-5810-4853-89c0-4be22abe85a3-kube-api-access-8r2b9" (OuterVolumeSpecName: "kube-api-access-8r2b9") pod "1f6f8fbd-5810-4853-89c0-4be22abe85a3" (UID: "1f6f8fbd-5810-4853-89c0-4be22abe85a3"). InnerVolumeSpecName "kube-api-access-8r2b9". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:27:22 crc kubenswrapper[5052]: I1203 12:27:22.295239 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5a5320b7-0541-483a-8e0b-78c10d0f8efa-erlang-cookie-secret" (OuterVolumeSpecName: "erlang-cookie-secret") pod "5a5320b7-0541-483a-8e0b-78c10d0f8efa" (UID: "5a5320b7-0541-483a-8e0b-78c10d0f8efa"). InnerVolumeSpecName "erlang-cookie-secret". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:27:22 crc kubenswrapper[5052]: I1203 12:27:22.297800 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5a5320b7-0541-483a-8e0b-78c10d0f8efa-kube-api-access-95l4t" (OuterVolumeSpecName: "kube-api-access-95l4t") pod "5a5320b7-0541-483a-8e0b-78c10d0f8efa" (UID: "5a5320b7-0541-483a-8e0b-78c10d0f8efa"). InnerVolumeSpecName "kube-api-access-95l4t". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:27:22 crc kubenswrapper[5052]: I1203 12:27:22.303131 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6ab5f8d6-e515-4f7b-b14f-383b4180f4e7-scripts" (OuterVolumeSpecName: "scripts") pod "6ab5f8d6-e515-4f7b-b14f-383b4180f4e7" (UID: "6ab5f8d6-e515-4f7b-b14f-383b4180f4e7"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:27:22 crc kubenswrapper[5052]: I1203 12:27:22.303292 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1cc56b2c-23cc-452f-8de3-d7e8bd2e1e68-kube-api-access-vcl4m" (OuterVolumeSpecName: "kube-api-access-vcl4m") pod "1cc56b2c-23cc-452f-8de3-d7e8bd2e1e68" (UID: "1cc56b2c-23cc-452f-8de3-d7e8bd2e1e68"). InnerVolumeSpecName "kube-api-access-vcl4m". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:27:22 crc kubenswrapper[5052]: I1203 12:27:22.309081 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1cc56b2c-23cc-452f-8de3-d7e8bd2e1e68-config-data-custom" (OuterVolumeSpecName: "config-data-custom") pod "1cc56b2c-23cc-452f-8de3-d7e8bd2e1e68" (UID: "1cc56b2c-23cc-452f-8de3-d7e8bd2e1e68"). InnerVolumeSpecName "config-data-custom". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:27:22 crc kubenswrapper[5052]: I1203 12:27:22.310025 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage01-crc" (OuterVolumeSpecName: "persistence") pod "5a5320b7-0541-483a-8e0b-78c10d0f8efa" (UID: "5a5320b7-0541-483a-8e0b-78c10d0f8efa"). InnerVolumeSpecName "local-storage01-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 12:27:22 crc kubenswrapper[5052]: I1203 12:27:22.322566 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"rabbitmq-erlang-cookie\" (UniqueName: \"kubernetes.io/empty-dir/2c03f99a-98fd-4e86-ad29-fb69e94bd436-rabbitmq-erlang-cookie\") pod \"rabbitmq-server-0\" (UID: \"2c03f99a-98fd-4e86-ad29-fb69e94bd436\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:27:22 crc kubenswrapper[5052]: I1203 12:27:22.322626 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"server-conf\" (UniqueName: \"kubernetes.io/configmap/2c03f99a-98fd-4e86-ad29-fb69e94bd436-server-conf\") pod \"rabbitmq-server-0\" (UID: \"2c03f99a-98fd-4e86-ad29-fb69e94bd436\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:27:22 crc kubenswrapper[5052]: I1203 12:27:22.322686 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage19-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage19-crc\") pod \"rabbitmq-server-0\" (UID: \"2c03f99a-98fd-4e86-ad29-fb69e94bd436\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:27:22 crc kubenswrapper[5052]: I1203 12:27:22.322738 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-conf\" (UniqueName: \"kubernetes.io/configmap/2c03f99a-98fd-4e86-ad29-fb69e94bd436-plugins-conf\") pod \"rabbitmq-server-0\" (UID: \"2c03f99a-98fd-4e86-ad29-fb69e94bd436\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:27:22 crc kubenswrapper[5052]: I1203 12:27:22.322756 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/2c03f99a-98fd-4e86-ad29-fb69e94bd436-config-data\") pod \"rabbitmq-server-0\" (UID: \"2c03f99a-98fd-4e86-ad29-fb69e94bd436\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:27:22 crc kubenswrapper[5052]: I1203 12:27:22.322780 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"rabbitmq-plugins\" (UniqueName: \"kubernetes.io/empty-dir/2c03f99a-98fd-4e86-ad29-fb69e94bd436-rabbitmq-plugins\") pod \"rabbitmq-server-0\" (UID: \"2c03f99a-98fd-4e86-ad29-fb69e94bd436\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:27:22 crc kubenswrapper[5052]: I1203 12:27:22.322803 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"rabbitmq-confd\" (UniqueName: \"kubernetes.io/projected/2c03f99a-98fd-4e86-ad29-fb69e94bd436-rabbitmq-confd\") pod \"rabbitmq-server-0\" (UID: \"2c03f99a-98fd-4e86-ad29-fb69e94bd436\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:27:22 crc kubenswrapper[5052]: I1203 12:27:22.322828 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"erlang-cookie-secret\" (UniqueName: \"kubernetes.io/secret/2c03f99a-98fd-4e86-ad29-fb69e94bd436-erlang-cookie-secret\") pod \"rabbitmq-server-0\" (UID: \"2c03f99a-98fd-4e86-ad29-fb69e94bd436\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:27:22 crc kubenswrapper[5052]: I1203 12:27:22.322853 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cgp57\" (UniqueName: \"kubernetes.io/projected/2c03f99a-98fd-4e86-ad29-fb69e94bd436-kube-api-access-cgp57\") pod \"rabbitmq-server-0\" (UID: \"2c03f99a-98fd-4e86-ad29-fb69e94bd436\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:27:22 crc kubenswrapper[5052]: I1203 12:27:22.322880 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"pod-info\" (UniqueName: \"kubernetes.io/downward-api/2c03f99a-98fd-4e86-ad29-fb69e94bd436-pod-info\") pod \"rabbitmq-server-0\" (UID: \"2c03f99a-98fd-4e86-ad29-fb69e94bd436\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:27:22 crc kubenswrapper[5052]: I1203 12:27:22.322905 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"rabbitmq-tls\" (UniqueName: \"kubernetes.io/projected/2c03f99a-98fd-4e86-ad29-fb69e94bd436-rabbitmq-tls\") pod \"rabbitmq-server-0\" (UID: \"2c03f99a-98fd-4e86-ad29-fb69e94bd436\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:27:22 crc kubenswrapper[5052]: I1203 12:27:22.322956 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-8r2b9\" (UniqueName: \"kubernetes.io/projected/1f6f8fbd-5810-4853-89c0-4be22abe85a3-kube-api-access-8r2b9\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:22 crc kubenswrapper[5052]: I1203 12:27:22.322966 5052 reconciler_common.go:293] "Volume detached for volume \"pod-info\" (UniqueName: \"kubernetes.io/downward-api/5a5320b7-0541-483a-8e0b-78c10d0f8efa-pod-info\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:22 crc kubenswrapper[5052]: I1203 12:27:22.322976 5052 reconciler_common.go:293] "Volume detached for volume \"rabbitmq-plugins\" (UniqueName: \"kubernetes.io/empty-dir/5a5320b7-0541-483a-8e0b-78c10d0f8efa-rabbitmq-plugins\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:22 crc kubenswrapper[5052]: I1203 12:27:22.322985 5052 reconciler_common.go:293] "Volume detached for volume \"erlang-cookie-secret\" (UniqueName: \"kubernetes.io/secret/5a5320b7-0541-483a-8e0b-78c10d0f8efa-erlang-cookie-secret\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:22 crc kubenswrapper[5052]: I1203 12:27:22.322993 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/1cc56b2c-23cc-452f-8de3-d7e8bd2e1e68-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:22 crc kubenswrapper[5052]: I1203 12:27:22.323001 5052 reconciler_common.go:293] "Volume detached for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/6ab5f8d6-e515-4f7b-b14f-383b4180f4e7-log-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:22 crc kubenswrapper[5052]: I1203 12:27:22.323010 5052 reconciler_common.go:293] "Volume detached for volume \"rabbitmq-erlang-cookie\" (UniqueName: \"kubernetes.io/empty-dir/5a5320b7-0541-483a-8e0b-78c10d0f8efa-rabbitmq-erlang-cookie\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:22 crc kubenswrapper[5052]: I1203 12:27:22.323018 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/6ab5f8d6-e515-4f7b-b14f-383b4180f4e7-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:22 crc kubenswrapper[5052]: I1203 12:27:22.323026 5052 reconciler_common.go:293] "Volume detached for volume \"plugins-conf\" (UniqueName: \"kubernetes.io/configmap/5a5320b7-0541-483a-8e0b-78c10d0f8efa-plugins-conf\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:22 crc kubenswrapper[5052]: I1203 12:27:22.323037 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-vcl4m\" (UniqueName: \"kubernetes.io/projected/1cc56b2c-23cc-452f-8de3-d7e8bd2e1e68-kube-api-access-vcl4m\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:22 crc kubenswrapper[5052]: I1203 12:27:22.323046 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-9mhgt\" (UniqueName: \"kubernetes.io/projected/6ab5f8d6-e515-4f7b-b14f-383b4180f4e7-kube-api-access-9mhgt\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:22 crc kubenswrapper[5052]: I1203 12:27:22.323064 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage01-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage01-crc\") on node \"crc\" " Dec 03 12:27:22 crc kubenswrapper[5052]: I1203 12:27:22.323074 5052 reconciler_common.go:293] "Volume detached for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/6ab5f8d6-e515-4f7b-b14f-383b4180f4e7-run-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:22 crc kubenswrapper[5052]: I1203 12:27:22.323083 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/1cc56b2c-23cc-452f-8de3-d7e8bd2e1e68-config-data-custom\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:22 crc kubenswrapper[5052]: I1203 12:27:22.323091 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-95l4t\" (UniqueName: \"kubernetes.io/projected/5a5320b7-0541-483a-8e0b-78c10d0f8efa-kube-api-access-95l4t\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:22 crc kubenswrapper[5052]: I1203 12:27:22.424882 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage19-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage19-crc\") pod \"rabbitmq-server-0\" (UID: \"2c03f99a-98fd-4e86-ad29-fb69e94bd436\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:27:22 crc kubenswrapper[5052]: I1203 12:27:22.424975 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"plugins-conf\" (UniqueName: \"kubernetes.io/configmap/2c03f99a-98fd-4e86-ad29-fb69e94bd436-plugins-conf\") pod \"rabbitmq-server-0\" (UID: \"2c03f99a-98fd-4e86-ad29-fb69e94bd436\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:27:22 crc kubenswrapper[5052]: I1203 12:27:22.425007 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/2c03f99a-98fd-4e86-ad29-fb69e94bd436-config-data\") pod \"rabbitmq-server-0\" (UID: \"2c03f99a-98fd-4e86-ad29-fb69e94bd436\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:27:22 crc kubenswrapper[5052]: I1203 12:27:22.425029 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"rabbitmq-plugins\" (UniqueName: \"kubernetes.io/empty-dir/2c03f99a-98fd-4e86-ad29-fb69e94bd436-rabbitmq-plugins\") pod \"rabbitmq-server-0\" (UID: \"2c03f99a-98fd-4e86-ad29-fb69e94bd436\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:27:22 crc kubenswrapper[5052]: I1203 12:27:22.425060 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"rabbitmq-confd\" (UniqueName: \"kubernetes.io/projected/2c03f99a-98fd-4e86-ad29-fb69e94bd436-rabbitmq-confd\") pod \"rabbitmq-server-0\" (UID: \"2c03f99a-98fd-4e86-ad29-fb69e94bd436\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:27:22 crc kubenswrapper[5052]: I1203 12:27:22.425094 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"erlang-cookie-secret\" (UniqueName: \"kubernetes.io/secret/2c03f99a-98fd-4e86-ad29-fb69e94bd436-erlang-cookie-secret\") pod \"rabbitmq-server-0\" (UID: \"2c03f99a-98fd-4e86-ad29-fb69e94bd436\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:27:22 crc kubenswrapper[5052]: I1203 12:27:22.425133 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-cgp57\" (UniqueName: \"kubernetes.io/projected/2c03f99a-98fd-4e86-ad29-fb69e94bd436-kube-api-access-cgp57\") pod \"rabbitmq-server-0\" (UID: \"2c03f99a-98fd-4e86-ad29-fb69e94bd436\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:27:22 crc kubenswrapper[5052]: I1203 12:27:22.425173 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"pod-info\" (UniqueName: \"kubernetes.io/downward-api/2c03f99a-98fd-4e86-ad29-fb69e94bd436-pod-info\") pod \"rabbitmq-server-0\" (UID: \"2c03f99a-98fd-4e86-ad29-fb69e94bd436\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:27:22 crc kubenswrapper[5052]: I1203 12:27:22.425259 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"rabbitmq-tls\" (UniqueName: \"kubernetes.io/projected/2c03f99a-98fd-4e86-ad29-fb69e94bd436-rabbitmq-tls\") pod \"rabbitmq-server-0\" (UID: \"2c03f99a-98fd-4e86-ad29-fb69e94bd436\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:27:22 crc kubenswrapper[5052]: I1203 12:27:22.425260 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage19-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage19-crc\") pod \"rabbitmq-server-0\" (UID: \"2c03f99a-98fd-4e86-ad29-fb69e94bd436\") device mount path \"/mnt/openstack/pv19\"" pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:27:22 crc kubenswrapper[5052]: I1203 12:27:22.425293 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"rabbitmq-erlang-cookie\" (UniqueName: \"kubernetes.io/empty-dir/2c03f99a-98fd-4e86-ad29-fb69e94bd436-rabbitmq-erlang-cookie\") pod \"rabbitmq-server-0\" (UID: \"2c03f99a-98fd-4e86-ad29-fb69e94bd436\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:27:22 crc kubenswrapper[5052]: I1203 12:27:22.425325 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"server-conf\" (UniqueName: \"kubernetes.io/configmap/2c03f99a-98fd-4e86-ad29-fb69e94bd436-server-conf\") pod \"rabbitmq-server-0\" (UID: \"2c03f99a-98fd-4e86-ad29-fb69e94bd436\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:27:22 crc kubenswrapper[5052]: I1203 12:27:22.426906 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"rabbitmq-plugins\" (UniqueName: \"kubernetes.io/empty-dir/2c03f99a-98fd-4e86-ad29-fb69e94bd436-rabbitmq-plugins\") pod \"rabbitmq-server-0\" (UID: \"2c03f99a-98fd-4e86-ad29-fb69e94bd436\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:27:22 crc kubenswrapper[5052]: I1203 12:27:22.427035 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/2c03f99a-98fd-4e86-ad29-fb69e94bd436-config-data\") pod \"rabbitmq-server-0\" (UID: \"2c03f99a-98fd-4e86-ad29-fb69e94bd436\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:27:22 crc kubenswrapper[5052]: I1203 12:27:22.429275 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"plugins-conf\" (UniqueName: \"kubernetes.io/configmap/2c03f99a-98fd-4e86-ad29-fb69e94bd436-plugins-conf\") pod \"rabbitmq-server-0\" (UID: \"2c03f99a-98fd-4e86-ad29-fb69e94bd436\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:27:22 crc kubenswrapper[5052]: I1203 12:27:22.429444 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"rabbitmq-erlang-cookie\" (UniqueName: \"kubernetes.io/empty-dir/2c03f99a-98fd-4e86-ad29-fb69e94bd436-rabbitmq-erlang-cookie\") pod \"rabbitmq-server-0\" (UID: \"2c03f99a-98fd-4e86-ad29-fb69e94bd436\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:27:22 crc kubenswrapper[5052]: I1203 12:27:22.430129 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"server-conf\" (UniqueName: \"kubernetes.io/configmap/2c03f99a-98fd-4e86-ad29-fb69e94bd436-server-conf\") pod \"rabbitmq-server-0\" (UID: \"2c03f99a-98fd-4e86-ad29-fb69e94bd436\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:27:22 crc kubenswrapper[5052]: I1203 12:27:22.467336 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"erlang-cookie-secret\" (UniqueName: \"kubernetes.io/secret/2c03f99a-98fd-4e86-ad29-fb69e94bd436-erlang-cookie-secret\") pod \"rabbitmq-server-0\" (UID: \"2c03f99a-98fd-4e86-ad29-fb69e94bd436\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:27:22 crc kubenswrapper[5052]: I1203 12:27:22.468994 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"rabbitmq-tls\" (UniqueName: \"kubernetes.io/projected/2c03f99a-98fd-4e86-ad29-fb69e94bd436-rabbitmq-tls\") pod \"rabbitmq-server-0\" (UID: \"2c03f99a-98fd-4e86-ad29-fb69e94bd436\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:27:22 crc kubenswrapper[5052]: I1203 12:27:22.470499 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"rabbitmq-confd\" (UniqueName: \"kubernetes.io/projected/2c03f99a-98fd-4e86-ad29-fb69e94bd436-rabbitmq-confd\") pod \"rabbitmq-server-0\" (UID: \"2c03f99a-98fd-4e86-ad29-fb69e94bd436\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:27:22 crc kubenswrapper[5052]: I1203 12:27:22.473991 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"pod-info\" (UniqueName: \"kubernetes.io/downward-api/2c03f99a-98fd-4e86-ad29-fb69e94bd436-pod-info\") pod \"rabbitmq-server-0\" (UID: \"2c03f99a-98fd-4e86-ad29-fb69e94bd436\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:27:22 crc kubenswrapper[5052]: I1203 12:27:22.483669 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-cgp57\" (UniqueName: \"kubernetes.io/projected/2c03f99a-98fd-4e86-ad29-fb69e94bd436-kube-api-access-cgp57\") pod \"rabbitmq-server-0\" (UID: \"2c03f99a-98fd-4e86-ad29-fb69e94bd436\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:27:22 crc kubenswrapper[5052]: I1203 12:27:22.747257 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage01-crc" (UniqueName: "kubernetes.io/local-volume/local-storage01-crc") on node "crc" Dec 03 12:27:22 crc kubenswrapper[5052]: I1203 12:27:22.835378 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage01-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage01-crc\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:22 crc kubenswrapper[5052]: I1203 12:27:22.846053 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:27:22 crc kubenswrapper[5052]: I1203 12:27:22.973475 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:27:23 crc kubenswrapper[5052]: I1203 12:27:22.998099 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/98d0abd5-a20b-45e3-aa30-34f038b2fb3d-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "98d0abd5-a20b-45e3-aa30-34f038b2fb3d" (UID: "98d0abd5-a20b-45e3-aa30-34f038b2fb3d"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:27:23 crc kubenswrapper[5052]: I1203 12:27:23.012026 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage19-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage19-crc\") pod \"rabbitmq-server-0\" (UID: \"2c03f99a-98fd-4e86-ad29-fb69e94bd436\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:27:23 crc kubenswrapper[5052]: I1203 12:27:23.029683 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1cc56b2c-23cc-452f-8de3-d7e8bd2e1e68-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "1cc56b2c-23cc-452f-8de3-d7e8bd2e1e68" (UID: "1cc56b2c-23cc-452f-8de3-d7e8bd2e1e68"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:27:23 crc kubenswrapper[5052]: I1203 12:27:23.073774 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5a5320b7-0541-483a-8e0b-78c10d0f8efa-server-conf" (OuterVolumeSpecName: "server-conf") pod "5a5320b7-0541-483a-8e0b-78c10d0f8efa" (UID: "5a5320b7-0541-483a-8e0b-78c10d0f8efa"). InnerVolumeSpecName "server-conf". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:27:23 crc kubenswrapper[5052]: I1203 12:27:23.078405 5052 reconciler_common.go:293] "Volume detached for volume \"server-conf\" (UniqueName: \"kubernetes.io/configmap/5a5320b7-0541-483a-8e0b-78c10d0f8efa-server-conf\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:23 crc kubenswrapper[5052]: I1203 12:27:23.079960 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1cc56b2c-23cc-452f-8de3-d7e8bd2e1e68-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:23 crc kubenswrapper[5052]: I1203 12:27:23.079974 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/98d0abd5-a20b-45e3-aa30-34f038b2fb3d-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:23 crc kubenswrapper[5052]: I1203 12:27:23.101548 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6ab5f8d6-e515-4f7b-b14f-383b4180f4e7-sg-core-conf-yaml" (OuterVolumeSpecName: "sg-core-conf-yaml") pod "6ab5f8d6-e515-4f7b-b14f-383b4180f4e7" (UID: "6ab5f8d6-e515-4f7b-b14f-383b4180f4e7"). InnerVolumeSpecName "sg-core-conf-yaml". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:27:23 crc kubenswrapper[5052]: I1203 12:27:23.112687 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1f6f8fbd-5810-4853-89c0-4be22abe85a3-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "1f6f8fbd-5810-4853-89c0-4be22abe85a3" (UID: "1f6f8fbd-5810-4853-89c0-4be22abe85a3"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:27:23 crc kubenswrapper[5052]: I1203 12:27:23.112813 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1f6f8fbd-5810-4853-89c0-4be22abe85a3-config-data" (OuterVolumeSpecName: "config-data") pod "1f6f8fbd-5810-4853-89c0-4be22abe85a3" (UID: "1f6f8fbd-5810-4853-89c0-4be22abe85a3"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:27:23 crc kubenswrapper[5052]: I1203 12:27:23.169062 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-5d855df9cb-xdg6p" event={"ID":"3a27c483-00fc-4b06-af6b-3c4b0ff8d0d1","Type":"ContainerStarted","Data":"16e0da8b4d5026dda5b45a3d5fe087062b021f9377ca4b9ad0a08e9f37208416"} Dec 03 12:27:23 crc kubenswrapper[5052]: I1203 12:27:23.170802 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/barbican-api-5d855df9cb-xdg6p" Dec 03 12:27:23 crc kubenswrapper[5052]: I1203 12:27:23.170870 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/barbican-api-5d855df9cb-xdg6p" Dec 03 12:27:23 crc kubenswrapper[5052]: I1203 12:27:23.180097 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"6436bca9-c7a1-4540-bee3-b6dbabb53c89","Type":"ContainerStarted","Data":"4f7e4bab55b18e1135d280cc95a601724a2d3fc12cda71f14ad7f590adb9df61"} Dec 03 12:27:23 crc kubenswrapper[5052]: I1203 12:27:23.190032 5052 reconciler_common.go:293] "Volume detached for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/6ab5f8d6-e515-4f7b-b14f-383b4180f4e7-sg-core-conf-yaml\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:23 crc kubenswrapper[5052]: I1203 12:27:23.190073 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/1f6f8fbd-5810-4853-89c0-4be22abe85a3-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:23 crc kubenswrapper[5052]: I1203 12:27:23.190091 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1f6f8fbd-5810-4853-89c0-4be22abe85a3-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:23 crc kubenswrapper[5052]: I1203 12:27:23.192968 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-6cdf7689bb-fscpc" event={"ID":"51bb178b-072f-4a7a-928b-5a8845d536b7","Type":"ContainerStarted","Data":"849a9a922c59a61b5c664acc3061a8b4ea9ea6688c539bbfce5e6562aedbfcf0"} Dec 03 12:27:23 crc kubenswrapper[5052]: I1203 12:27:23.193243 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/placement-6cdf7689bb-fscpc" podUID="51bb178b-072f-4a7a-928b-5a8845d536b7" containerName="placement-log" containerID="cri-o://3426f46bd5e0b4a541ed02f94900e4a755d8e523a368d5a2ed4870a257b7116f" gracePeriod=30 Dec 03 12:27:23 crc kubenswrapper[5052]: I1203 12:27:23.193564 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/placement-6cdf7689bb-fscpc" Dec 03 12:27:23 crc kubenswrapper[5052]: I1203 12:27:23.193653 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/placement-6cdf7689bb-fscpc" Dec 03 12:27:23 crc kubenswrapper[5052]: I1203 12:27:23.193741 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/placement-6cdf7689bb-fscpc" podUID="51bb178b-072f-4a7a-928b-5a8845d536b7" containerName="placement-api" containerID="cri-o://849a9a922c59a61b5c664acc3061a8b4ea9ea6688c539bbfce5e6562aedbfcf0" gracePeriod=30 Dec 03 12:27:23 crc kubenswrapper[5052]: I1203 12:27:23.208795 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/barbican-api-5d855df9cb-xdg6p" podStartSLOduration=8.208768479 podStartE2EDuration="8.208768479s" podCreationTimestamp="2025-12-03 12:27:15 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:27:23.191660571 +0000 UTC m=+4033.779902355" watchObservedRunningTime="2025-12-03 12:27:23.208768479 +0000 UTC m=+4033.797010243" Dec 03 12:27:23 crc kubenswrapper[5052]: I1203 12:27:23.253804 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1cc56b2c-23cc-452f-8de3-d7e8bd2e1e68-config-data" (OuterVolumeSpecName: "config-data") pod "1cc56b2c-23cc-452f-8de3-d7e8bd2e1e68" (UID: "1cc56b2c-23cc-452f-8de3-d7e8bd2e1e68"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:27:23 crc kubenswrapper[5052]: I1203 12:27:23.254071 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"810e47c2-430b-424d-92ea-66c20ec8501f","Type":"ContainerStarted","Data":"85ec55fcdf485c866d81bbe48593514b623381629978696113751cab29e59861"} Dec 03 12:27:23 crc kubenswrapper[5052]: I1203 12:27:23.266723 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/placement-6cdf7689bb-fscpc" podStartSLOduration=8.266701102 podStartE2EDuration="8.266701102s" podCreationTimestamp="2025-12-03 12:27:15 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:27:23.229234583 +0000 UTC m=+4033.817476347" watchObservedRunningTime="2025-12-03 12:27:23.266701102 +0000 UTC m=+4033.854942866" Dec 03 12:27:23 crc kubenswrapper[5052]: I1203 12:27:23.315109 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"8a2c61ae-6521-424f-aeac-52e1ec1c319a","Type":"ContainerStarted","Data":"012ec83742c8d05cd9b1b280425de3addfd86df96f18dcaafb3102f1adcfbf48"} Dec 03 12:27:23 crc kubenswrapper[5052]: I1203 12:27:23.319352 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/98d0abd5-a20b-45e3-aa30-34f038b2fb3d-config-data" (OuterVolumeSpecName: "config-data") pod "98d0abd5-a20b-45e3-aa30-34f038b2fb3d" (UID: "98d0abd5-a20b-45e3-aa30-34f038b2fb3d"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:27:23 crc kubenswrapper[5052]: I1203 12:27:23.326164 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/1cc56b2c-23cc-452f-8de3-d7e8bd2e1e68-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:23 crc kubenswrapper[5052]: I1203 12:27:23.326194 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/98d0abd5-a20b-45e3-aa30-34f038b2fb3d-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:23 crc kubenswrapper[5052]: I1203 12:27:23.333675 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-worker-679bfdf7b7-tg6h7" event={"ID":"72a32b4d-d61d-43a5-a45d-c1fd4bad672d","Type":"ContainerStarted","Data":"6d2a1c1187cb71ec6764a5792108b52b3d8a71f435dceee72f6a125558106e90"} Dec 03 12:27:23 crc kubenswrapper[5052]: I1203 12:27:23.340606 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:27:23 crc kubenswrapper[5052]: I1203 12:27:23.357844 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6ab5f8d6-e515-4f7b-b14f-383b4180f4e7-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "6ab5f8d6-e515-4f7b-b14f-383b4180f4e7" (UID: "6ab5f8d6-e515-4f7b-b14f-383b4180f4e7"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:27:23 crc kubenswrapper[5052]: I1203 12:27:23.372995 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-keystone-listener-58b6cdf6bb-5xxd8" event={"ID":"71df5bf1-7fd8-4a4d-b0f5-85e3911ae8a9","Type":"ContainerStarted","Data":"994203d634c00abfeeeb2db379076b53d6f741dba1b79d202d758f959e9439c6"} Dec 03 12:27:23 crc kubenswrapper[5052]: I1203 12:27:23.389815 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"0b3a2637-ed0d-4348-ac7e-c92db6503191","Type":"ContainerStarted","Data":"7b40deaa77ac2e192f6c398118bda4081a63324a94ddcda3a887e339639b15c1"} Dec 03 12:27:23 crc kubenswrapper[5052]: I1203 12:27:23.389928 5052 scope.go:117] "RemoveContainer" containerID="f679be3b6603c919c59273998d120bbb192e07c332db88e94aabe9e26ca2af36" Dec 03 12:27:23 crc kubenswrapper[5052]: I1203 12:27:23.432620 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6ab5f8d6-e515-4f7b-b14f-383b4180f4e7-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:23 crc kubenswrapper[5052]: I1203 12:27:23.442916 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/barbican-worker-679bfdf7b7-tg6h7" podStartSLOduration=8.442899242 podStartE2EDuration="8.442899242s" podCreationTimestamp="2025-12-03 12:27:15 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:27:23.357884985 +0000 UTC m=+4033.946126769" watchObservedRunningTime="2025-12-03 12:27:23.442899242 +0000 UTC m=+4034.031141006" Dec 03 12:27:23 crc kubenswrapper[5052]: I1203 12:27:23.473725 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:27:23 crc kubenswrapper[5052]: I1203 12:27:23.474634 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-7897d64b6b-j8w2r" event={"ID":"82331185-51eb-4ef5-8cf6-61dc74b73740","Type":"ContainerStarted","Data":"bdd07debdd83a2d2d982d8b066d0fefba55b8fee5901eb6d6479f72b63d3ae51"} Dec 03 12:27:23 crc kubenswrapper[5052]: I1203 12:27:23.475460 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:27:23 crc kubenswrapper[5052]: I1203 12:27:23.475560 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:27:23 crc kubenswrapper[5052]: I1203 12:27:23.475606 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-keystone-listener-7c58f6698b-95l8v" Dec 03 12:27:23 crc kubenswrapper[5052]: I1203 12:27:23.476262 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-worker-75f6bbd957-c444v" Dec 03 12:27:23 crc kubenswrapper[5052]: I1203 12:27:23.502046 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-worker-64d9f4d555-4v5kn"] Dec 03 12:27:23 crc kubenswrapper[5052]: I1203 12:27:23.502341 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/barbican-worker-64d9f4d555-4v5kn" podUID="6228bbc6-4c33-4ad8-ad26-440a8e6b441c" containerName="barbican-worker-log" containerID="cri-o://cad85f08979cf86c1f2f2aa7c3637ac5ab17fe2a4099a131b06311d2a7a911ff" gracePeriod=30 Dec 03 12:27:23 crc kubenswrapper[5052]: I1203 12:27:23.502474 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/barbican-worker-64d9f4d555-4v5kn" podUID="6228bbc6-4c33-4ad8-ad26-440a8e6b441c" containerName="barbican-worker" containerID="cri-o://595a1417c6646739c93678d8740cd28a9eadb61887c7e437c16c9c0ee312a81e" gracePeriod=30 Dec 03 12:27:23 crc kubenswrapper[5052]: I1203 12:27:23.526886 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/barbican-keystone-listener-58b6cdf6bb-5xxd8" podStartSLOduration=8.526867018 podStartE2EDuration="8.526867018s" podCreationTimestamp="2025-12-03 12:27:15 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:27:23.402092197 +0000 UTC m=+4033.990333961" watchObservedRunningTime="2025-12-03 12:27:23.526867018 +0000 UTC m=+4034.115108782" Dec 03 12:27:24 crc kubenswrapper[5052]: I1203 12:27:23.544361 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-keystone-listener-6df6465ddd-km2kt"] Dec 03 12:27:24 crc kubenswrapper[5052]: I1203 12:27:23.544643 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/barbican-keystone-listener-6df6465ddd-km2kt" podUID="7f4dfec9-0aa8-437c-9b8d-771b30a93af9" containerName="barbican-keystone-listener-log" containerID="cri-o://54f3fa89ac27b88281c422bf43fdb03928a159ddffa85177a0997bcb486d4eee" gracePeriod=30 Dec 03 12:27:24 crc kubenswrapper[5052]: I1203 12:27:23.545230 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/barbican-keystone-listener-6df6465ddd-km2kt" podUID="7f4dfec9-0aa8-437c-9b8d-771b30a93af9" containerName="barbican-keystone-listener" containerID="cri-o://bc39c9a553475072254f96ed656dff329e085eb9d86080fd0b82411e0f0780a9" gracePeriod=30 Dec 03 12:27:24 crc kubenswrapper[5052]: I1203 12:27:23.551730 5052 scope.go:117] "RemoveContainer" containerID="229c6c7d07d677ea52e66801e341130c260352a54816a6ceeecbd91e2597db8c" Dec 03 12:27:24 crc kubenswrapper[5052]: I1203 12:27:23.580705 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5a5320b7-0541-483a-8e0b-78c10d0f8efa-rabbitmq-confd" (OuterVolumeSpecName: "rabbitmq-confd") pod "5a5320b7-0541-483a-8e0b-78c10d0f8efa" (UID: "5a5320b7-0541-483a-8e0b-78c10d0f8efa"). InnerVolumeSpecName "rabbitmq-confd". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:27:24 crc kubenswrapper[5052]: I1203 12:27:23.580794 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6ab5f8d6-e515-4f7b-b14f-383b4180f4e7-config-data" (OuterVolumeSpecName: "config-data") pod "6ab5f8d6-e515-4f7b-b14f-383b4180f4e7" (UID: "6ab5f8d6-e515-4f7b-b14f-383b4180f4e7"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:27:24 crc kubenswrapper[5052]: I1203 12:27:23.640874 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6ab5f8d6-e515-4f7b-b14f-383b4180f4e7-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:24 crc kubenswrapper[5052]: I1203 12:27:23.640905 5052 reconciler_common.go:293] "Volume detached for volume \"rabbitmq-confd\" (UniqueName: \"kubernetes.io/projected/5a5320b7-0541-483a-8e0b-78c10d0f8efa-rabbitmq-confd\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:24 crc kubenswrapper[5052]: I1203 12:27:24.102919 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cd7b6179-ba41-43bd-bc95-490a88aa15fd" path="/var/lib/kubelet/pods/cd7b6179-ba41-43bd-bc95-490a88aa15fd/volumes" Dec 03 12:27:24 crc kubenswrapper[5052]: I1203 12:27:24.248101 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:27:24 crc kubenswrapper[5052]: I1203 12:27:24.304399 5052 scope.go:117] "RemoveContainer" containerID="c9884448416314162e49f4564ffad485559e0dcde7274fbe0abad7844eeaf00d" Dec 03 12:27:24 crc kubenswrapper[5052]: I1203 12:27:24.388761 5052 scope.go:117] "RemoveContainer" containerID="032bc0ff1bc0c7adb6371d41b7d1d30531bb08512a12516306ce34a00afcaad9" Dec 03 12:27:24 crc kubenswrapper[5052]: I1203 12:27:24.502266 5052 generic.go:334] "Generic (PLEG): container finished" podID="51bb178b-072f-4a7a-928b-5a8845d536b7" containerID="3426f46bd5e0b4a541ed02f94900e4a755d8e523a368d5a2ed4870a257b7116f" exitCode=143 Dec 03 12:27:24 crc kubenswrapper[5052]: I1203 12:27:24.502329 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-6cdf7689bb-fscpc" event={"ID":"51bb178b-072f-4a7a-928b-5a8845d536b7","Type":"ContainerDied","Data":"3426f46bd5e0b4a541ed02f94900e4a755d8e523a368d5a2ed4870a257b7116f"} Dec 03 12:27:24 crc kubenswrapper[5052]: I1203 12:27:24.504253 5052 generic.go:334] "Generic (PLEG): container finished" podID="6228bbc6-4c33-4ad8-ad26-440a8e6b441c" containerID="cad85f08979cf86c1f2f2aa7c3637ac5ab17fe2a4099a131b06311d2a7a911ff" exitCode=143 Dec 03 12:27:24 crc kubenswrapper[5052]: I1203 12:27:24.504292 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-worker-64d9f4d555-4v5kn" event={"ID":"6228bbc6-4c33-4ad8-ad26-440a8e6b441c","Type":"ContainerDied","Data":"cad85f08979cf86c1f2f2aa7c3637ac5ab17fe2a4099a131b06311d2a7a911ff"} Dec 03 12:27:24 crc kubenswrapper[5052]: I1203 12:27:24.507927 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-keystone-listener-6df6465ddd-km2kt" event={"ID":"7f4dfec9-0aa8-437c-9b8d-771b30a93af9","Type":"ContainerDied","Data":"54f3fa89ac27b88281c422bf43fdb03928a159ddffa85177a0997bcb486d4eee"} Dec 03 12:27:24 crc kubenswrapper[5052]: I1203 12:27:24.507983 5052 generic.go:334] "Generic (PLEG): container finished" podID="7f4dfec9-0aa8-437c-9b8d-771b30a93af9" containerID="54f3fa89ac27b88281c422bf43fdb03928a159ddffa85177a0997bcb486d4eee" exitCode=143 Dec 03 12:27:24 crc kubenswrapper[5052]: I1203 12:27:24.643360 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-worker-75f6bbd957-c444v"] Dec 03 12:27:24 crc kubenswrapper[5052]: I1203 12:27:24.658477 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/barbican-worker-75f6bbd957-c444v"] Dec 03 12:27:24 crc kubenswrapper[5052]: I1203 12:27:24.679464 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:27:24 crc kubenswrapper[5052]: I1203 12:27:24.709118 5052 scope.go:117] "RemoveContainer" containerID="6c115b6a73d88d94535850532bd7a63e133ef983572d338d54582ff166612eba" Dec 03 12:27:24 crc kubenswrapper[5052]: I1203 12:27:24.717035 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:27:24 crc kubenswrapper[5052]: I1203 12:27:24.740754 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-0"] Dec 03 12:27:24 crc kubenswrapper[5052]: I1203 12:27:24.779091 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-0"] Dec 03 12:27:24 crc kubenswrapper[5052]: I1203 12:27:24.810958 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:27:24 crc kubenswrapper[5052]: I1203 12:27:24.816900 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:27:24 crc kubenswrapper[5052]: I1203 12:27:24.820102 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-config-data" Dec 03 12:27:24 crc kubenswrapper[5052]: I1203 12:27:24.821104 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-scripts" Dec 03 12:27:24 crc kubenswrapper[5052]: I1203 12:27:24.833173 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/rabbitmq-cell1-server-0"] Dec 03 12:27:24 crc kubenswrapper[5052]: I1203 12:27:24.835718 5052 scope.go:117] "RemoveContainer" containerID="032bc0ff1bc0c7adb6371d41b7d1d30531bb08512a12516306ce34a00afcaad9" Dec 03 12:27:24 crc kubenswrapper[5052]: E1203 12:27:24.844972 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"032bc0ff1bc0c7adb6371d41b7d1d30531bb08512a12516306ce34a00afcaad9\": container with ID starting with 032bc0ff1bc0c7adb6371d41b7d1d30531bb08512a12516306ce34a00afcaad9 not found: ID does not exist" containerID="032bc0ff1bc0c7adb6371d41b7d1d30531bb08512a12516306ce34a00afcaad9" Dec 03 12:27:24 crc kubenswrapper[5052]: I1203 12:27:24.845309 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"032bc0ff1bc0c7adb6371d41b7d1d30531bb08512a12516306ce34a00afcaad9"} err="failed to get container status \"032bc0ff1bc0c7adb6371d41b7d1d30531bb08512a12516306ce34a00afcaad9\": rpc error: code = NotFound desc = could not find container \"032bc0ff1bc0c7adb6371d41b7d1d30531bb08512a12516306ce34a00afcaad9\": container with ID starting with 032bc0ff1bc0c7adb6371d41b7d1d30531bb08512a12516306ce34a00afcaad9 not found: ID does not exist" Dec 03 12:27:24 crc kubenswrapper[5052]: I1203 12:27:24.845442 5052 scope.go:117] "RemoveContainer" containerID="6c115b6a73d88d94535850532bd7a63e133ef983572d338d54582ff166612eba" Dec 03 12:27:24 crc kubenswrapper[5052]: E1203 12:27:24.851764 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6c115b6a73d88d94535850532bd7a63e133ef983572d338d54582ff166612eba\": container with ID starting with 6c115b6a73d88d94535850532bd7a63e133ef983572d338d54582ff166612eba not found: ID does not exist" containerID="6c115b6a73d88d94535850532bd7a63e133ef983572d338d54582ff166612eba" Dec 03 12:27:24 crc kubenswrapper[5052]: I1203 12:27:24.852248 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6c115b6a73d88d94535850532bd7a63e133ef983572d338d54582ff166612eba"} err="failed to get container status \"6c115b6a73d88d94535850532bd7a63e133ef983572d338d54582ff166612eba\": rpc error: code = NotFound desc = could not find container \"6c115b6a73d88d94535850532bd7a63e133ef983572d338d54582ff166612eba\": container with ID starting with 6c115b6a73d88d94535850532bd7a63e133ef983572d338d54582ff166612eba not found: ID does not exist" Dec 03 12:27:24 crc kubenswrapper[5052]: I1203 12:27:24.866063 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-0"] Dec 03 12:27:24 crc kubenswrapper[5052]: I1203 12:27:24.874471 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:27:24 crc kubenswrapper[5052]: I1203 12:27:24.879840 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell0-conductor-config-data" Dec 03 12:27:24 crc kubenswrapper[5052]: I1203 12:27:24.881263 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sgtpx\" (UniqueName: \"kubernetes.io/projected/5d3294b3-d25c-4d8a-ba28-aaad7385693b-kube-api-access-sgtpx\") pod \"ceilometer-0\" (UID: \"5d3294b3-d25c-4d8a-ba28-aaad7385693b\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:27:24 crc kubenswrapper[5052]: I1203 12:27:24.881337 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/5d3294b3-d25c-4d8a-ba28-aaad7385693b-config-data\") pod \"ceilometer-0\" (UID: \"5d3294b3-d25c-4d8a-ba28-aaad7385693b\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:27:24 crc kubenswrapper[5052]: I1203 12:27:24.881407 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/5d3294b3-d25c-4d8a-ba28-aaad7385693b-run-httpd\") pod \"ceilometer-0\" (UID: \"5d3294b3-d25c-4d8a-ba28-aaad7385693b\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:27:24 crc kubenswrapper[5052]: I1203 12:27:24.881436 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/5d3294b3-d25c-4d8a-ba28-aaad7385693b-scripts\") pod \"ceilometer-0\" (UID: \"5d3294b3-d25c-4d8a-ba28-aaad7385693b\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:27:24 crc kubenswrapper[5052]: I1203 12:27:24.881466 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5d3294b3-d25c-4d8a-ba28-aaad7385693b-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"5d3294b3-d25c-4d8a-ba28-aaad7385693b\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:27:24 crc kubenswrapper[5052]: I1203 12:27:24.881492 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/5d3294b3-d25c-4d8a-ba28-aaad7385693b-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"5d3294b3-d25c-4d8a-ba28-aaad7385693b\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:27:24 crc kubenswrapper[5052]: I1203 12:27:24.881537 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/5d3294b3-d25c-4d8a-ba28-aaad7385693b-log-httpd\") pod \"ceilometer-0\" (UID: \"5d3294b3-d25c-4d8a-ba28-aaad7385693b\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:27:24 crc kubenswrapper[5052]: I1203 12:27:24.887584 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/rabbitmq-cell1-server-0"] Dec 03 12:27:24 crc kubenswrapper[5052]: I1203 12:27:24.911149 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:27:24 crc kubenswrapper[5052]: I1203 12:27:24.926897 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-0"] Dec 03 12:27:24 crc kubenswrapper[5052]: I1203 12:27:24.942650 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/rabbitmq-cell1-server-0"] Dec 03 12:27:24 crc kubenswrapper[5052]: I1203 12:27:24.945008 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:27:24 crc kubenswrapper[5052]: I1203 12:27:24.951210 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"rabbitmq-cell1-server-dockercfg-qcdwj" Dec 03 12:27:24 crc kubenswrapper[5052]: I1203 12:27:24.951574 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"rabbitmq-cell1-plugins-conf" Dec 03 12:27:24 crc kubenswrapper[5052]: I1203 12:27:24.951755 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"rabbitmq-cell1-config-data" Dec 03 12:27:24 crc kubenswrapper[5052]: I1203 12:27:24.952037 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"rabbitmq-cell1-server-conf" Dec 03 12:27:24 crc kubenswrapper[5052]: I1203 12:27:24.952295 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"rabbitmq-cell1-erlang-cookie" Dec 03 12:27:24 crc kubenswrapper[5052]: I1203 12:27:24.952444 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"rabbitmq-cell1-default-user" Dec 03 12:27:24 crc kubenswrapper[5052]: I1203 12:27:24.952646 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-rabbitmq-cell1-svc" Dec 03 12:27:24 crc kubenswrapper[5052]: I1203 12:27:24.963550 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/rabbitmq-cell1-server-0"] Dec 03 12:27:24 crc kubenswrapper[5052]: I1203 12:27:24.992557 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/5d3294b3-d25c-4d8a-ba28-aaad7385693b-config-data\") pod \"ceilometer-0\" (UID: \"5d3294b3-d25c-4d8a-ba28-aaad7385693b\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:27:24 crc kubenswrapper[5052]: I1203 12:27:24.994899 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/5d3294b3-d25c-4d8a-ba28-aaad7385693b-run-httpd\") pod \"ceilometer-0\" (UID: \"5d3294b3-d25c-4d8a-ba28-aaad7385693b\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:27:24 crc kubenswrapper[5052]: I1203 12:27:24.995556 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/5d3294b3-d25c-4d8a-ba28-aaad7385693b-scripts\") pod \"ceilometer-0\" (UID: \"5d3294b3-d25c-4d8a-ba28-aaad7385693b\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:27:24 crc kubenswrapper[5052]: I1203 12:27:24.995709 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5d3294b3-d25c-4d8a-ba28-aaad7385693b-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"5d3294b3-d25c-4d8a-ba28-aaad7385693b\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:27:24 crc kubenswrapper[5052]: I1203 12:27:24.995833 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/5d3294b3-d25c-4d8a-ba28-aaad7385693b-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"5d3294b3-d25c-4d8a-ba28-aaad7385693b\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:27:24 crc kubenswrapper[5052]: I1203 12:27:24.995949 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/5d3294b3-d25c-4d8a-ba28-aaad7385693b-log-httpd\") pod \"ceilometer-0\" (UID: \"5d3294b3-d25c-4d8a-ba28-aaad7385693b\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:27:24 crc kubenswrapper[5052]: I1203 12:27:24.996088 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c8nwg\" (UniqueName: \"kubernetes.io/projected/c080dc7f-df72-487f-b2c1-83b143d34784-kube-api-access-c8nwg\") pod \"nova-cell0-conductor-0\" (UID: \"c080dc7f-df72-487f-b2c1-83b143d34784\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:27:24 crc kubenswrapper[5052]: I1203 12:27:24.996222 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/c080dc7f-df72-487f-b2c1-83b143d34784-config-data\") pod \"nova-cell0-conductor-0\" (UID: \"c080dc7f-df72-487f-b2c1-83b143d34784\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:27:24 crc kubenswrapper[5052]: I1203 12:27:24.996420 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c080dc7f-df72-487f-b2c1-83b143d34784-combined-ca-bundle\") pod \"nova-cell0-conductor-0\" (UID: \"c080dc7f-df72-487f-b2c1-83b143d34784\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:27:24 crc kubenswrapper[5052]: I1203 12:27:24.996602 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-sgtpx\" (UniqueName: \"kubernetes.io/projected/5d3294b3-d25c-4d8a-ba28-aaad7385693b-kube-api-access-sgtpx\") pod \"ceilometer-0\" (UID: \"5d3294b3-d25c-4d8a-ba28-aaad7385693b\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:27:24 crc kubenswrapper[5052]: I1203 12:27:24.998062 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/5d3294b3-d25c-4d8a-ba28-aaad7385693b-log-httpd\") pod \"ceilometer-0\" (UID: \"5d3294b3-d25c-4d8a-ba28-aaad7385693b\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:27:25 crc kubenswrapper[5052]: I1203 12:27:25.000077 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/5d3294b3-d25c-4d8a-ba28-aaad7385693b-run-httpd\") pod \"ceilometer-0\" (UID: \"5d3294b3-d25c-4d8a-ba28-aaad7385693b\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:27:25 crc kubenswrapper[5052]: I1203 12:27:25.005277 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/5d3294b3-d25c-4d8a-ba28-aaad7385693b-scripts\") pod \"ceilometer-0\" (UID: \"5d3294b3-d25c-4d8a-ba28-aaad7385693b\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:27:25 crc kubenswrapper[5052]: I1203 12:27:25.014930 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5d3294b3-d25c-4d8a-ba28-aaad7385693b-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"5d3294b3-d25c-4d8a-ba28-aaad7385693b\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:27:25 crc kubenswrapper[5052]: I1203 12:27:25.015767 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:27:25 crc kubenswrapper[5052]: I1203 12:27:25.036858 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-sgtpx\" (UniqueName: \"kubernetes.io/projected/5d3294b3-d25c-4d8a-ba28-aaad7385693b-kube-api-access-sgtpx\") pod \"ceilometer-0\" (UID: \"5d3294b3-d25c-4d8a-ba28-aaad7385693b\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:27:25 crc kubenswrapper[5052]: I1203 12:27:25.037377 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/5d3294b3-d25c-4d8a-ba28-aaad7385693b-config-data\") pod \"ceilometer-0\" (UID: \"5d3294b3-d25c-4d8a-ba28-aaad7385693b\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:27:25 crc kubenswrapper[5052]: I1203 12:27:25.052813 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/5d3294b3-d25c-4d8a-ba28-aaad7385693b-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"5d3294b3-d25c-4d8a-ba28-aaad7385693b\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:27:25 crc kubenswrapper[5052]: I1203 12:27:25.092817 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:27:25 crc kubenswrapper[5052]: I1203 12:27:25.102927 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c080dc7f-df72-487f-b2c1-83b143d34784-combined-ca-bundle\") pod \"nova-cell0-conductor-0\" (UID: \"c080dc7f-df72-487f-b2c1-83b143d34784\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:27:25 crc kubenswrapper[5052]: I1203 12:27:25.103203 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-c8nwg\" (UniqueName: \"kubernetes.io/projected/c080dc7f-df72-487f-b2c1-83b143d34784-kube-api-access-c8nwg\") pod \"nova-cell0-conductor-0\" (UID: \"c080dc7f-df72-487f-b2c1-83b143d34784\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:27:25 crc kubenswrapper[5052]: I1203 12:27:25.103254 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/c080dc7f-df72-487f-b2c1-83b143d34784-config-data\") pod \"nova-cell0-conductor-0\" (UID: \"c080dc7f-df72-487f-b2c1-83b143d34784\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:27:25 crc kubenswrapper[5052]: I1203 12:27:25.124550 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c080dc7f-df72-487f-b2c1-83b143d34784-combined-ca-bundle\") pod \"nova-cell0-conductor-0\" (UID: \"c080dc7f-df72-487f-b2c1-83b143d34784\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:27:25 crc kubenswrapper[5052]: I1203 12:27:25.137326 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/c080dc7f-df72-487f-b2c1-83b143d34784-config-data\") pod \"nova-cell0-conductor-0\" (UID: \"c080dc7f-df72-487f-b2c1-83b143d34784\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:27:25 crc kubenswrapper[5052]: I1203 12:27:25.185968 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:27:25 crc kubenswrapper[5052]: I1203 12:27:25.197829 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-c8nwg\" (UniqueName: \"kubernetes.io/projected/c080dc7f-df72-487f-b2c1-83b143d34784-kube-api-access-c8nwg\") pod \"nova-cell0-conductor-0\" (UID: \"c080dc7f-df72-487f-b2c1-83b143d34784\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:27:25 crc kubenswrapper[5052]: I1203 12:27:25.205227 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"rabbitmq-erlang-cookie\" (UniqueName: \"kubernetes.io/empty-dir/b8613a94-1e2e-477f-8d52-713c70f5282c-rabbitmq-erlang-cookie\") pod \"rabbitmq-cell1-server-0\" (UID: \"b8613a94-1e2e-477f-8d52-713c70f5282c\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:27:25 crc kubenswrapper[5052]: I1203 12:27:25.205283 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"rabbitmq-confd\" (UniqueName: \"kubernetes.io/projected/b8613a94-1e2e-477f-8d52-713c70f5282c-rabbitmq-confd\") pod \"rabbitmq-cell1-server-0\" (UID: \"b8613a94-1e2e-477f-8d52-713c70f5282c\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:27:25 crc kubenswrapper[5052]: I1203 12:27:25.205315 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"pod-info\" (UniqueName: \"kubernetes.io/downward-api/b8613a94-1e2e-477f-8d52-713c70f5282c-pod-info\") pod \"rabbitmq-cell1-server-0\" (UID: \"b8613a94-1e2e-477f-8d52-713c70f5282c\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:27:25 crc kubenswrapper[5052]: I1203 12:27:25.205443 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-conf\" (UniqueName: \"kubernetes.io/configmap/b8613a94-1e2e-477f-8d52-713c70f5282c-plugins-conf\") pod \"rabbitmq-cell1-server-0\" (UID: \"b8613a94-1e2e-477f-8d52-713c70f5282c\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:27:25 crc kubenswrapper[5052]: I1203 12:27:25.205475 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"rabbitmq-tls\" (UniqueName: \"kubernetes.io/projected/b8613a94-1e2e-477f-8d52-713c70f5282c-rabbitmq-tls\") pod \"rabbitmq-cell1-server-0\" (UID: \"b8613a94-1e2e-477f-8d52-713c70f5282c\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:27:25 crc kubenswrapper[5052]: I1203 12:27:25.205500 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/b8613a94-1e2e-477f-8d52-713c70f5282c-config-data\") pod \"rabbitmq-cell1-server-0\" (UID: \"b8613a94-1e2e-477f-8d52-713c70f5282c\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:27:25 crc kubenswrapper[5052]: I1203 12:27:25.205569 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"rabbitmq-plugins\" (UniqueName: \"kubernetes.io/empty-dir/b8613a94-1e2e-477f-8d52-713c70f5282c-rabbitmq-plugins\") pod \"rabbitmq-cell1-server-0\" (UID: \"b8613a94-1e2e-477f-8d52-713c70f5282c\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:27:25 crc kubenswrapper[5052]: I1203 12:27:25.205606 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage01-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage01-crc\") pod \"rabbitmq-cell1-server-0\" (UID: \"b8613a94-1e2e-477f-8d52-713c70f5282c\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:27:25 crc kubenswrapper[5052]: I1203 12:27:25.205667 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"server-conf\" (UniqueName: \"kubernetes.io/configmap/b8613a94-1e2e-477f-8d52-713c70f5282c-server-conf\") pod \"rabbitmq-cell1-server-0\" (UID: \"b8613a94-1e2e-477f-8d52-713c70f5282c\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:27:25 crc kubenswrapper[5052]: I1203 12:27:25.205719 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"erlang-cookie-secret\" (UniqueName: \"kubernetes.io/secret/b8613a94-1e2e-477f-8d52-713c70f5282c-erlang-cookie-secret\") pod \"rabbitmq-cell1-server-0\" (UID: \"b8613a94-1e2e-477f-8d52-713c70f5282c\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:27:25 crc kubenswrapper[5052]: I1203 12:27:25.205787 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6sgkj\" (UniqueName: \"kubernetes.io/projected/b8613a94-1e2e-477f-8d52-713c70f5282c-kube-api-access-6sgkj\") pod \"rabbitmq-cell1-server-0\" (UID: \"b8613a94-1e2e-477f-8d52-713c70f5282c\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:27:25 crc kubenswrapper[5052]: I1203 12:27:25.215590 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/rabbitmq-server-0"] Dec 03 12:27:25 crc kubenswrapper[5052]: I1203 12:27:25.239785 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:27:25 crc kubenswrapper[5052]: I1203 12:27:25.311708 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"plugins-conf\" (UniqueName: \"kubernetes.io/configmap/b8613a94-1e2e-477f-8d52-713c70f5282c-plugins-conf\") pod \"rabbitmq-cell1-server-0\" (UID: \"b8613a94-1e2e-477f-8d52-713c70f5282c\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:27:25 crc kubenswrapper[5052]: I1203 12:27:25.312011 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"rabbitmq-tls\" (UniqueName: \"kubernetes.io/projected/b8613a94-1e2e-477f-8d52-713c70f5282c-rabbitmq-tls\") pod \"rabbitmq-cell1-server-0\" (UID: \"b8613a94-1e2e-477f-8d52-713c70f5282c\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:27:25 crc kubenswrapper[5052]: I1203 12:27:25.312121 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/b8613a94-1e2e-477f-8d52-713c70f5282c-config-data\") pod \"rabbitmq-cell1-server-0\" (UID: \"b8613a94-1e2e-477f-8d52-713c70f5282c\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:27:25 crc kubenswrapper[5052]: I1203 12:27:25.312246 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"rabbitmq-plugins\" (UniqueName: \"kubernetes.io/empty-dir/b8613a94-1e2e-477f-8d52-713c70f5282c-rabbitmq-plugins\") pod \"rabbitmq-cell1-server-0\" (UID: \"b8613a94-1e2e-477f-8d52-713c70f5282c\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:27:25 crc kubenswrapper[5052]: I1203 12:27:25.312361 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage01-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage01-crc\") pod \"rabbitmq-cell1-server-0\" (UID: \"b8613a94-1e2e-477f-8d52-713c70f5282c\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:27:25 crc kubenswrapper[5052]: I1203 12:27:25.312500 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"server-conf\" (UniqueName: \"kubernetes.io/configmap/b8613a94-1e2e-477f-8d52-713c70f5282c-server-conf\") pod \"rabbitmq-cell1-server-0\" (UID: \"b8613a94-1e2e-477f-8d52-713c70f5282c\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:27:25 crc kubenswrapper[5052]: I1203 12:27:25.312699 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"erlang-cookie-secret\" (UniqueName: \"kubernetes.io/secret/b8613a94-1e2e-477f-8d52-713c70f5282c-erlang-cookie-secret\") pod \"rabbitmq-cell1-server-0\" (UID: \"b8613a94-1e2e-477f-8d52-713c70f5282c\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:27:25 crc kubenswrapper[5052]: I1203 12:27:25.312835 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-6sgkj\" (UniqueName: \"kubernetes.io/projected/b8613a94-1e2e-477f-8d52-713c70f5282c-kube-api-access-6sgkj\") pod \"rabbitmq-cell1-server-0\" (UID: \"b8613a94-1e2e-477f-8d52-713c70f5282c\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:27:25 crc kubenswrapper[5052]: I1203 12:27:25.312958 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"rabbitmq-erlang-cookie\" (UniqueName: \"kubernetes.io/empty-dir/b8613a94-1e2e-477f-8d52-713c70f5282c-rabbitmq-erlang-cookie\") pod \"rabbitmq-cell1-server-0\" (UID: \"b8613a94-1e2e-477f-8d52-713c70f5282c\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:27:25 crc kubenswrapper[5052]: I1203 12:27:25.313051 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"rabbitmq-confd\" (UniqueName: \"kubernetes.io/projected/b8613a94-1e2e-477f-8d52-713c70f5282c-rabbitmq-confd\") pod \"rabbitmq-cell1-server-0\" (UID: \"b8613a94-1e2e-477f-8d52-713c70f5282c\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:27:25 crc kubenswrapper[5052]: I1203 12:27:25.313166 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"pod-info\" (UniqueName: \"kubernetes.io/downward-api/b8613a94-1e2e-477f-8d52-713c70f5282c-pod-info\") pod \"rabbitmq-cell1-server-0\" (UID: \"b8613a94-1e2e-477f-8d52-713c70f5282c\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:27:25 crc kubenswrapper[5052]: I1203 12:27:25.315322 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"plugins-conf\" (UniqueName: \"kubernetes.io/configmap/b8613a94-1e2e-477f-8d52-713c70f5282c-plugins-conf\") pod \"rabbitmq-cell1-server-0\" (UID: \"b8613a94-1e2e-477f-8d52-713c70f5282c\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:27:25 crc kubenswrapper[5052]: I1203 12:27:25.316386 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/b8613a94-1e2e-477f-8d52-713c70f5282c-config-data\") pod \"rabbitmq-cell1-server-0\" (UID: \"b8613a94-1e2e-477f-8d52-713c70f5282c\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:27:25 crc kubenswrapper[5052]: I1203 12:27:25.317865 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"server-conf\" (UniqueName: \"kubernetes.io/configmap/b8613a94-1e2e-477f-8d52-713c70f5282c-server-conf\") pod \"rabbitmq-cell1-server-0\" (UID: \"b8613a94-1e2e-477f-8d52-713c70f5282c\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:27:25 crc kubenswrapper[5052]: I1203 12:27:25.318610 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"pod-info\" (UniqueName: \"kubernetes.io/downward-api/b8613a94-1e2e-477f-8d52-713c70f5282c-pod-info\") pod \"rabbitmq-cell1-server-0\" (UID: \"b8613a94-1e2e-477f-8d52-713c70f5282c\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:27:25 crc kubenswrapper[5052]: I1203 12:27:25.320175 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"rabbitmq-erlang-cookie\" (UniqueName: \"kubernetes.io/empty-dir/b8613a94-1e2e-477f-8d52-713c70f5282c-rabbitmq-erlang-cookie\") pod \"rabbitmq-cell1-server-0\" (UID: \"b8613a94-1e2e-477f-8d52-713c70f5282c\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:27:25 crc kubenswrapper[5052]: I1203 12:27:25.335344 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"erlang-cookie-secret\" (UniqueName: \"kubernetes.io/secret/b8613a94-1e2e-477f-8d52-713c70f5282c-erlang-cookie-secret\") pod \"rabbitmq-cell1-server-0\" (UID: \"b8613a94-1e2e-477f-8d52-713c70f5282c\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:27:25 crc kubenswrapper[5052]: I1203 12:27:25.343070 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"rabbitmq-tls\" (UniqueName: \"kubernetes.io/projected/b8613a94-1e2e-477f-8d52-713c70f5282c-rabbitmq-tls\") pod \"rabbitmq-cell1-server-0\" (UID: \"b8613a94-1e2e-477f-8d52-713c70f5282c\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:27:25 crc kubenswrapper[5052]: I1203 12:27:25.343773 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"rabbitmq-confd\" (UniqueName: \"kubernetes.io/projected/b8613a94-1e2e-477f-8d52-713c70f5282c-rabbitmq-confd\") pod \"rabbitmq-cell1-server-0\" (UID: \"b8613a94-1e2e-477f-8d52-713c70f5282c\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:27:25 crc kubenswrapper[5052]: I1203 12:27:25.344419 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"rabbitmq-plugins\" (UniqueName: \"kubernetes.io/empty-dir/b8613a94-1e2e-477f-8d52-713c70f5282c-rabbitmq-plugins\") pod \"rabbitmq-cell1-server-0\" (UID: \"b8613a94-1e2e-477f-8d52-713c70f5282c\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:27:25 crc kubenswrapper[5052]: I1203 12:27:25.346629 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage01-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage01-crc\") pod \"rabbitmq-cell1-server-0\" (UID: \"b8613a94-1e2e-477f-8d52-713c70f5282c\") device mount path \"/mnt/openstack/pv01\"" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:27:25 crc kubenswrapper[5052]: I1203 12:27:25.357442 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-6sgkj\" (UniqueName: \"kubernetes.io/projected/b8613a94-1e2e-477f-8d52-713c70f5282c-kube-api-access-6sgkj\") pod \"rabbitmq-cell1-server-0\" (UID: \"b8613a94-1e2e-477f-8d52-713c70f5282c\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:27:25 crc kubenswrapper[5052]: I1203 12:27:25.469972 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage01-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage01-crc\") pod \"rabbitmq-cell1-server-0\" (UID: \"b8613a94-1e2e-477f-8d52-713c70f5282c\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:27:25 crc kubenswrapper[5052]: I1203 12:27:25.567470 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"0b3a2637-ed0d-4348-ac7e-c92db6503191","Type":"ContainerStarted","Data":"35e2dee61a61ea5e56329f496973eb6a0f2fe0619c6f0df2e55caadaea334f93"} Dec 03 12:27:25 crc kubenswrapper[5052]: I1203 12:27:25.576157 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-6bf55bd9d6-mrgs9" event={"ID":"15cf6575-b6be-4f67-861a-61ebf02d1495","Type":"ContainerStarted","Data":"b4a9d01e3dc994f108359d2cce819e8f260555e99eb3dff86f96ba5dc6edd4b0"} Dec 03 12:27:25 crc kubenswrapper[5052]: I1203 12:27:25.576354 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/neutron-6bf55bd9d6-mrgs9" Dec 03 12:27:25 crc kubenswrapper[5052]: I1203 12:27:25.585133 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-7897d64b6b-j8w2r" event={"ID":"82331185-51eb-4ef5-8cf6-61dc74b73740","Type":"ContainerStarted","Data":"8435a02270401c91707d9204caf291f3365f9acf4871f8d7542a8e53a028b8f7"} Dec 03 12:27:25 crc kubenswrapper[5052]: I1203 12:27:25.585688 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/placement-7897d64b6b-j8w2r" Dec 03 12:27:25 crc kubenswrapper[5052]: I1203 12:27:25.586091 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/placement-7897d64b6b-j8w2r" Dec 03 12:27:25 crc kubenswrapper[5052]: I1203 12:27:25.595554 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"810e47c2-430b-424d-92ea-66c20ec8501f","Type":"ContainerStarted","Data":"22904b310fc3f26f046b8a7eda3ddcc73a97bb5641a37e30766bd99ffba27bde"} Dec 03 12:27:25 crc kubenswrapper[5052]: I1203 12:27:25.603466 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"8a2c61ae-6521-424f-aeac-52e1ec1c319a","Type":"ContainerStarted","Data":"90ccfa6fea4d46147f2030f8a0f1ffee182ce45ef8183a711e81ed3a0eaa5b15"} Dec 03 12:27:25 crc kubenswrapper[5052]: I1203 12:27:25.605241 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-server-0" event={"ID":"2c03f99a-98fd-4e86-ad29-fb69e94bd436","Type":"ContainerStarted","Data":"8d68154bede68b4efe4328c546c86de3b50b2f79ee843aa36a7c95ffbde2f980"} Dec 03 12:27:25 crc kubenswrapper[5052]: I1203 12:27:25.637245 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"6436bca9-c7a1-4540-bee3-b6dbabb53c89","Type":"ContainerStarted","Data":"584f69d542c77d9dd7acd735af74068b5eca3f64ae82019e8afdda51237d1117"} Dec 03 12:27:25 crc kubenswrapper[5052]: I1203 12:27:25.647339 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/placement-7897d64b6b-j8w2r" podStartSLOduration=8.647321461 podStartE2EDuration="8.647321461s" podCreationTimestamp="2025-12-03 12:27:17 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:27:25.622714478 +0000 UTC m=+4036.210956252" watchObservedRunningTime="2025-12-03 12:27:25.647321461 +0000 UTC m=+4036.235563225" Dec 03 12:27:25 crc kubenswrapper[5052]: I1203 12:27:25.648018 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/neutron-6bf55bd9d6-mrgs9" podStartSLOduration=10.648010941 podStartE2EDuration="10.648010941s" podCreationTimestamp="2025-12-03 12:27:15 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:27:25.594634227 +0000 UTC m=+4036.182875991" watchObservedRunningTime="2025-12-03 12:27:25.648010941 +0000 UTC m=+4036.236252705" Dec 03 12:27:25 crc kubenswrapper[5052]: I1203 12:27:25.670497 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:27:25 crc kubenswrapper[5052]: I1203 12:27:25.670592 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:27:25 crc kubenswrapper[5052]: I1203 12:27:25.691892 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-api-0" podStartSLOduration=7.691875132 podStartE2EDuration="7.691875132s" podCreationTimestamp="2025-12-03 12:27:18 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:27:25.667127256 +0000 UTC m=+4036.255369020" watchObservedRunningTime="2025-12-03 12:27:25.691875132 +0000 UTC m=+4036.280116896" Dec 03 12:27:25 crc kubenswrapper[5052]: I1203 12:27:25.693561 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/cinder-scheduler-0" podStartSLOduration=8.69355172 podStartE2EDuration="8.69355172s" podCreationTimestamp="2025-12-03 12:27:17 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:27:25.688091835 +0000 UTC m=+4036.276333599" watchObservedRunningTime="2025-12-03 12:27:25.69355172 +0000 UTC m=+4036.281793484" Dec 03 12:27:25 crc kubenswrapper[5052]: I1203 12:27:25.721153 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1cc56b2c-23cc-452f-8de3-d7e8bd2e1e68" path="/var/lib/kubelet/pods/1cc56b2c-23cc-452f-8de3-d7e8bd2e1e68/volumes" Dec 03 12:27:25 crc kubenswrapper[5052]: I1203 12:27:25.722457 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1f6f8fbd-5810-4853-89c0-4be22abe85a3" path="/var/lib/kubelet/pods/1f6f8fbd-5810-4853-89c0-4be22abe85a3/volumes" Dec 03 12:27:25 crc kubenswrapper[5052]: I1203 12:27:25.723708 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5a5320b7-0541-483a-8e0b-78c10d0f8efa" path="/var/lib/kubelet/pods/5a5320b7-0541-483a-8e0b-78c10d0f8efa/volumes" Dec 03 12:27:25 crc kubenswrapper[5052]: I1203 12:27:25.725190 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6ab5f8d6-e515-4f7b-b14f-383b4180f4e7" path="/var/lib/kubelet/pods/6ab5f8d6-e515-4f7b-b14f-383b4180f4e7/volumes" Dec 03 12:27:25 crc kubenswrapper[5052]: I1203 12:27:25.972477 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-0"] Dec 03 12:27:26 crc kubenswrapper[5052]: I1203 12:27:26.106859 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:27:26 crc kubenswrapper[5052]: I1203 12:27:26.366535 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/rabbitmq-cell1-server-0"] Dec 03 12:27:26 crc kubenswrapper[5052]: I1203 12:27:26.670143 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"5d3294b3-d25c-4d8a-ba28-aaad7385693b","Type":"ContainerStarted","Data":"64f15a43acc40c1b2d84b1728c906a2170144700373881238dc5e3bbe90f757b"} Dec 03 12:27:26 crc kubenswrapper[5052]: I1203 12:27:26.686224 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" event={"ID":"b8613a94-1e2e-477f-8d52-713c70f5282c","Type":"ContainerStarted","Data":"655770c6b6b108506058af5b98eb7807e27572e48bb338cd2a061b9f4c2ce826"} Dec 03 12:27:26 crc kubenswrapper[5052]: I1203 12:27:26.693141 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-conductor-0" event={"ID":"c080dc7f-df72-487f-b2c1-83b143d34784","Type":"ContainerStarted","Data":"790991ac480a9e358efc31b18c1e765ec3b8f7c9c3386cf9c51c6dc7f800750c"} Dec 03 12:27:26 crc kubenswrapper[5052]: I1203 12:27:26.693193 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-conductor-0" event={"ID":"c080dc7f-df72-487f-b2c1-83b143d34784","Type":"ContainerStarted","Data":"9b964a0bf45ecb54f67b88f63c0e3bbdac82ac6ab7ec4719bcdf02c46f1d08fc"} Dec 03 12:27:26 crc kubenswrapper[5052]: I1203 12:27:26.693490 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:27:26 crc kubenswrapper[5052]: I1203 12:27:26.716124 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"8a2c61ae-6521-424f-aeac-52e1ec1c319a","Type":"ContainerStarted","Data":"70af550381ae175843af90fae4ec5c50a17835a3a8274e54adab63fa88f5a895"} Dec 03 12:27:26 crc kubenswrapper[5052]: I1203 12:27:26.717821 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-cell0-conductor-0" podStartSLOduration=2.717802194 podStartE2EDuration="2.717802194s" podCreationTimestamp="2025-12-03 12:27:24 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:27:26.713609554 +0000 UTC m=+4037.301851328" watchObservedRunningTime="2025-12-03 12:27:26.717802194 +0000 UTC m=+4037.306043958" Dec 03 12:27:26 crc kubenswrapper[5052]: I1203 12:27:26.760119 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-metadata-0" podStartSLOduration=7.760102221 podStartE2EDuration="7.760102221s" podCreationTimestamp="2025-12-03 12:27:19 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:27:26.749901 +0000 UTC m=+4037.338142764" watchObservedRunningTime="2025-12-03 12:27:26.760102221 +0000 UTC m=+4037.348343985" Dec 03 12:27:26 crc kubenswrapper[5052]: I1203 12:27:26.891930 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ovn-northd-0"] Dec 03 12:27:26 crc kubenswrapper[5052]: I1203 12:27:26.892525 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ovn-northd-0" podUID="18e325b5-7837-4776-ae05-d9a12ccf9746" containerName="ovn-northd" containerID="cri-o://dda098dd0ab939491d6cb42aac03c4daa0a17e93eaa82fa8ed7c6dec3c2a4f1b" gracePeriod=30 Dec 03 12:27:26 crc kubenswrapper[5052]: I1203 12:27:26.892728 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ovn-northd-0" podUID="18e325b5-7837-4776-ae05-d9a12ccf9746" containerName="openstack-network-exporter" containerID="cri-o://cb277aa344aceb42cc1e9fe2afb23fa062591a751722aa2db98541cb69ea85ad" gracePeriod=30 Dec 03 12:27:26 crc kubenswrapper[5052]: I1203 12:27:26.945114 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:27:26 crc kubenswrapper[5052]: I1203 12:27:26.945213 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:27:27 crc kubenswrapper[5052]: I1203 12:27:27.738024 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-server-0" event={"ID":"2c03f99a-98fd-4e86-ad29-fb69e94bd436","Type":"ContainerStarted","Data":"e1e47b47265fb722fd8606f3ed96e8cbc15ed3bc36658d9fdd76dc06cbe471c8"} Dec 03 12:27:27 crc kubenswrapper[5052]: I1203 12:27:27.742610 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-nb-0"] Dec 03 12:27:27 crc kubenswrapper[5052]: I1203 12:27:27.743005 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ovsdbserver-nb-0" podUID="3b588591-8d98-427d-979f-72e7c8260b81" containerName="openstack-network-exporter" containerID="cri-o://1cff9f5854a876c596bb8b12d2477f784cfb487900c01da703c539509eeff1df" gracePeriod=300 Dec 03 12:27:27 crc kubenswrapper[5052]: I1203 12:27:27.752412 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"0b3a2637-ed0d-4348-ac7e-c92db6503191","Type":"ContainerStarted","Data":"8c8df8bba4000ac70f93809a90d4fade952afb4ce410d7dcf985178e32099131"} Dec 03 12:27:27 crc kubenswrapper[5052]: I1203 12:27:27.773019 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"5d3294b3-d25c-4d8a-ba28-aaad7385693b","Type":"ContainerStarted","Data":"518e554556294e8e06ab1671e9f2b277371917fe0bafb7f6898e763ce48bb5e1"} Dec 03 12:27:27 crc kubenswrapper[5052]: I1203 12:27:27.803981 5052 generic.go:334] "Generic (PLEG): container finished" podID="18e325b5-7837-4776-ae05-d9a12ccf9746" containerID="cb277aa344aceb42cc1e9fe2afb23fa062591a751722aa2db98541cb69ea85ad" exitCode=2 Dec 03 12:27:27 crc kubenswrapper[5052]: I1203 12:27:27.804098 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovn-northd-0" event={"ID":"18e325b5-7837-4776-ae05-d9a12ccf9746","Type":"ContainerDied","Data":"cb277aa344aceb42cc1e9fe2afb23fa062591a751722aa2db98541cb69ea85ad"} Dec 03 12:27:27 crc kubenswrapper[5052]: E1203 12:27:27.882581 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="dda098dd0ab939491d6cb42aac03c4daa0a17e93eaa82fa8ed7c6dec3c2a4f1b" cmd=["/usr/local/bin/container-scripts/status_check.sh"] Dec 03 12:27:27 crc kubenswrapper[5052]: E1203 12:27:27.884604 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="dda098dd0ab939491d6cb42aac03c4daa0a17e93eaa82fa8ed7c6dec3c2a4f1b" cmd=["/usr/local/bin/container-scripts/status_check.sh"] Dec 03 12:27:27 crc kubenswrapper[5052]: E1203 12:27:27.903314 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="dda098dd0ab939491d6cb42aac03c4daa0a17e93eaa82fa8ed7c6dec3c2a4f1b" cmd=["/usr/local/bin/container-scripts/status_check.sh"] Dec 03 12:27:27 crc kubenswrapper[5052]: E1203 12:27:27.903611 5052 prober.go:104] "Probe errored" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" probeType="Readiness" pod="openstack-kuttl-tests/ovn-northd-0" podUID="18e325b5-7837-4776-ae05-d9a12ccf9746" containerName="ovn-northd" Dec 03 12:27:27 crc kubenswrapper[5052]: I1203 12:27:27.951252 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ovsdbserver-nb-0" podUID="3b588591-8d98-427d-979f-72e7c8260b81" containerName="ovsdbserver-nb" containerID="cri-o://6b08aea5dde842833792afa4a6cfad11a22c78249915f3be8d0c1416ac2fca1f" gracePeriod=300 Dec 03 12:27:28 crc kubenswrapper[5052]: I1203 12:27:28.186022 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:27:28 crc kubenswrapper[5052]: I1203 12:27:28.299007 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/placement-7897d64b6b-j8w2r" Dec 03 12:27:28 crc kubenswrapper[5052]: I1203 12:27:28.332045 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/glance-default-internal-api-0" podStartSLOduration=9.332021347 podStartE2EDuration="9.332021347s" podCreationTimestamp="2025-12-03 12:27:19 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:27:27.834365183 +0000 UTC m=+4038.422606957" watchObservedRunningTime="2025-12-03 12:27:28.332021347 +0000 UTC m=+4038.920263111" Dec 03 12:27:28 crc kubenswrapper[5052]: I1203 12:27:28.342823 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:27:28 crc kubenswrapper[5052]: I1203 12:27:28.388822 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:27:28 crc kubenswrapper[5052]: I1203 12:27:28.506202 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:27:28 crc kubenswrapper[5052]: I1203 12:27:28.785207 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_ovsdbserver-nb-0_3b588591-8d98-427d-979f-72e7c8260b81/ovsdbserver-nb/0.log" Dec 03 12:27:28 crc kubenswrapper[5052]: I1203 12:27:28.785640 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:27:28 crc kubenswrapper[5052]: I1203 12:27:28.858948 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/3b588591-8d98-427d-979f-72e7c8260b81-scripts\") pod \"3b588591-8d98-427d-979f-72e7c8260b81\" (UID: \"3b588591-8d98-427d-979f-72e7c8260b81\") " Dec 03 12:27:28 crc kubenswrapper[5052]: I1203 12:27:28.859405 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3b588591-8d98-427d-979f-72e7c8260b81-combined-ca-bundle\") pod \"3b588591-8d98-427d-979f-72e7c8260b81\" (UID: \"3b588591-8d98-427d-979f-72e7c8260b81\") " Dec 03 12:27:28 crc kubenswrapper[5052]: I1203 12:27:28.859469 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v4jtp\" (UniqueName: \"kubernetes.io/projected/3b588591-8d98-427d-979f-72e7c8260b81-kube-api-access-v4jtp\") pod \"3b588591-8d98-427d-979f-72e7c8260b81\" (UID: \"3b588591-8d98-427d-979f-72e7c8260b81\") " Dec 03 12:27:28 crc kubenswrapper[5052]: I1203 12:27:28.859644 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3b588591-8d98-427d-979f-72e7c8260b81-config\") pod \"3b588591-8d98-427d-979f-72e7c8260b81\" (UID: \"3b588591-8d98-427d-979f-72e7c8260b81\") " Dec 03 12:27:28 crc kubenswrapper[5052]: I1203 12:27:28.859666 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3b588591-8d98-427d-979f-72e7c8260b81-scripts" (OuterVolumeSpecName: "scripts") pod "3b588591-8d98-427d-979f-72e7c8260b81" (UID: "3b588591-8d98-427d-979f-72e7c8260b81"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:27:28 crc kubenswrapper[5052]: I1203 12:27:28.859673 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovndbcluster-nb-etc-ovn\" (UniqueName: \"kubernetes.io/local-volume/local-storage03-crc\") pod \"3b588591-8d98-427d-979f-72e7c8260b81\" (UID: \"3b588591-8d98-427d-979f-72e7c8260b81\") " Dec 03 12:27:28 crc kubenswrapper[5052]: I1203 12:27:28.859945 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/3b588591-8d98-427d-979f-72e7c8260b81-ovsdb-rundir\") pod \"3b588591-8d98-427d-979f-72e7c8260b81\" (UID: \"3b588591-8d98-427d-979f-72e7c8260b81\") " Dec 03 12:27:28 crc kubenswrapper[5052]: I1203 12:27:28.860688 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/3b588591-8d98-427d-979f-72e7c8260b81-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:28 crc kubenswrapper[5052]: I1203 12:27:28.861036 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b588591-8d98-427d-979f-72e7c8260b81-ovsdb-rundir" (OuterVolumeSpecName: "ovsdb-rundir") pod "3b588591-8d98-427d-979f-72e7c8260b81" (UID: "3b588591-8d98-427d-979f-72e7c8260b81"). InnerVolumeSpecName "ovsdb-rundir". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:27:28 crc kubenswrapper[5052]: I1203 12:27:28.861418 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3b588591-8d98-427d-979f-72e7c8260b81-config" (OuterVolumeSpecName: "config") pod "3b588591-8d98-427d-979f-72e7c8260b81" (UID: "3b588591-8d98-427d-979f-72e7c8260b81"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:27:28 crc kubenswrapper[5052]: I1203 12:27:28.866474 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" event={"ID":"b8613a94-1e2e-477f-8d52-713c70f5282c","Type":"ContainerStarted","Data":"fdf73317a2bb37c60cc14493b2e7c924d803a06e05cbba2abbe853ddbf7c4fb6"} Dec 03 12:27:28 crc kubenswrapper[5052]: I1203 12:27:28.869807 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3b588591-8d98-427d-979f-72e7c8260b81-kube-api-access-v4jtp" (OuterVolumeSpecName: "kube-api-access-v4jtp") pod "3b588591-8d98-427d-979f-72e7c8260b81" (UID: "3b588591-8d98-427d-979f-72e7c8260b81"). InnerVolumeSpecName "kube-api-access-v4jtp". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:27:28 crc kubenswrapper[5052]: I1203 12:27:28.875033 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_neutron-797ccd5f76-v8cmg_5314ef6e-9ca6-4c9b-98b0-907b5a6e423c/neutron-api/0.log" Dec 03 12:27:28 crc kubenswrapper[5052]: I1203 12:27:28.875095 5052 generic.go:334] "Generic (PLEG): container finished" podID="5314ef6e-9ca6-4c9b-98b0-907b5a6e423c" containerID="922671b6af1b9388525c76529a11dfbf987f5c33b66d696ad739f90656999386" exitCode=1 Dec 03 12:27:28 crc kubenswrapper[5052]: I1203 12:27:28.875198 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-797ccd5f76-v8cmg" event={"ID":"5314ef6e-9ca6-4c9b-98b0-907b5a6e423c","Type":"ContainerDied","Data":"922671b6af1b9388525c76529a11dfbf987f5c33b66d696ad739f90656999386"} Dec 03 12:27:28 crc kubenswrapper[5052]: I1203 12:27:28.878466 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage03-crc" (OuterVolumeSpecName: "ovndbcluster-nb-etc-ovn") pod "3b588591-8d98-427d-979f-72e7c8260b81" (UID: "3b588591-8d98-427d-979f-72e7c8260b81"). InnerVolumeSpecName "local-storage03-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 12:27:28 crc kubenswrapper[5052]: I1203 12:27:28.895223 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"5d3294b3-d25c-4d8a-ba28-aaad7385693b","Type":"ContainerStarted","Data":"98b12c00821c1fa8f0a555940a026478afc3a124403bdbe42ea9907f1c09c593"} Dec 03 12:27:28 crc kubenswrapper[5052]: I1203 12:27:28.914464 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3b588591-8d98-427d-979f-72e7c8260b81-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "3b588591-8d98-427d-979f-72e7c8260b81" (UID: "3b588591-8d98-427d-979f-72e7c8260b81"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:27:28 crc kubenswrapper[5052]: I1203 12:27:28.927192 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_ovsdbserver-nb-0_3b588591-8d98-427d-979f-72e7c8260b81/ovsdbserver-nb/0.log" Dec 03 12:27:28 crc kubenswrapper[5052]: I1203 12:27:28.927236 5052 generic.go:334] "Generic (PLEG): container finished" podID="3b588591-8d98-427d-979f-72e7c8260b81" containerID="1cff9f5854a876c596bb8b12d2477f784cfb487900c01da703c539509eeff1df" exitCode=2 Dec 03 12:27:28 crc kubenswrapper[5052]: I1203 12:27:28.927250 5052 generic.go:334] "Generic (PLEG): container finished" podID="3b588591-8d98-427d-979f-72e7c8260b81" containerID="6b08aea5dde842833792afa4a6cfad11a22c78249915f3be8d0c1416ac2fca1f" exitCode=143 Dec 03 12:27:28 crc kubenswrapper[5052]: I1203 12:27:28.928059 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:27:28 crc kubenswrapper[5052]: I1203 12:27:28.929996 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-nb-0" event={"ID":"3b588591-8d98-427d-979f-72e7c8260b81","Type":"ContainerDied","Data":"1cff9f5854a876c596bb8b12d2477f784cfb487900c01da703c539509eeff1df"} Dec 03 12:27:28 crc kubenswrapper[5052]: I1203 12:27:28.930049 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-nb-0" event={"ID":"3b588591-8d98-427d-979f-72e7c8260b81","Type":"ContainerDied","Data":"6b08aea5dde842833792afa4a6cfad11a22c78249915f3be8d0c1416ac2fca1f"} Dec 03 12:27:28 crc kubenswrapper[5052]: I1203 12:27:28.930063 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-nb-0" event={"ID":"3b588591-8d98-427d-979f-72e7c8260b81","Type":"ContainerDied","Data":"0a44cceb24da4f58ea4816394380042c5acd3c9f07ed421a2d59cf9790dc50f1"} Dec 03 12:27:28 crc kubenswrapper[5052]: I1203 12:27:28.930086 5052 scope.go:117] "RemoveContainer" containerID="1cff9f5854a876c596bb8b12d2477f784cfb487900c01da703c539509eeff1df" Dec 03 12:27:28 crc kubenswrapper[5052]: I1203 12:27:28.967490 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3b588591-8d98-427d-979f-72e7c8260b81-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:28 crc kubenswrapper[5052]: I1203 12:27:28.967538 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-v4jtp\" (UniqueName: \"kubernetes.io/projected/3b588591-8d98-427d-979f-72e7c8260b81-kube-api-access-v4jtp\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:28 crc kubenswrapper[5052]: I1203 12:27:28.967565 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3b588591-8d98-427d-979f-72e7c8260b81-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:28 crc kubenswrapper[5052]: I1203 12:27:28.967584 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage03-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage03-crc\") on node \"crc\" " Dec 03 12:27:28 crc kubenswrapper[5052]: I1203 12:27:28.967593 5052 reconciler_common.go:293] "Volume detached for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/3b588591-8d98-427d-979f-72e7c8260b81-ovsdb-rundir\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:29 crc kubenswrapper[5052]: I1203 12:27:29.040714 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:27:29 crc kubenswrapper[5052]: I1203 12:27:29.053112 5052 scope.go:117] "RemoveContainer" containerID="6b08aea5dde842833792afa4a6cfad11a22c78249915f3be8d0c1416ac2fca1f" Dec 03 12:27:29 crc kubenswrapper[5052]: I1203 12:27:29.076955 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-nb-0"] Dec 03 12:27:29 crc kubenswrapper[5052]: I1203 12:27:29.082811 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-nb-0"] Dec 03 12:27:29 crc kubenswrapper[5052]: I1203 12:27:29.093599 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ovsdbserver-nb-0"] Dec 03 12:27:29 crc kubenswrapper[5052]: E1203 12:27:29.094184 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="3b588591-8d98-427d-979f-72e7c8260b81" containerName="ovsdbserver-nb" Dec 03 12:27:29 crc kubenswrapper[5052]: I1203 12:27:29.094205 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b588591-8d98-427d-979f-72e7c8260b81" containerName="ovsdbserver-nb" Dec 03 12:27:29 crc kubenswrapper[5052]: E1203 12:27:29.094242 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="3b588591-8d98-427d-979f-72e7c8260b81" containerName="openstack-network-exporter" Dec 03 12:27:29 crc kubenswrapper[5052]: I1203 12:27:29.094252 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b588591-8d98-427d-979f-72e7c8260b81" containerName="openstack-network-exporter" Dec 03 12:27:29 crc kubenswrapper[5052]: I1203 12:27:29.094483 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="3b588591-8d98-427d-979f-72e7c8260b81" containerName="openstack-network-exporter" Dec 03 12:27:29 crc kubenswrapper[5052]: I1203 12:27:29.094507 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="3b588591-8d98-427d-979f-72e7c8260b81" containerName="ovsdbserver-nb" Dec 03 12:27:29 crc kubenswrapper[5052]: I1203 12:27:29.095904 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:27:29 crc kubenswrapper[5052]: I1203 12:27:29.098822 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage03-crc" (UniqueName: "kubernetes.io/local-volume/local-storage03-crc") on node "crc" Dec 03 12:27:29 crc kubenswrapper[5052]: I1203 12:27:29.101347 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"ovndbcluster-nb-scripts" Dec 03 12:27:29 crc kubenswrapper[5052]: I1203 12:27:29.101663 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ovncluster-ovndbcluster-nb-dockercfg-4462b" Dec 03 12:27:29 crc kubenswrapper[5052]: I1203 12:27:29.101816 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-ovndbcluster-nb-ovndbs" Dec 03 12:27:29 crc kubenswrapper[5052]: I1203 12:27:29.101988 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"ovndbcluster-nb-config" Dec 03 12:27:29 crc kubenswrapper[5052]: I1203 12:27:29.102126 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-ovn-metrics" Dec 03 12:27:29 crc kubenswrapper[5052]: I1203 12:27:29.109328 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-nb-0"] Dec 03 12:27:29 crc kubenswrapper[5052]: I1203 12:27:29.165497 5052 scope.go:117] "RemoveContainer" containerID="1cff9f5854a876c596bb8b12d2477f784cfb487900c01da703c539509eeff1df" Dec 03 12:27:29 crc kubenswrapper[5052]: E1203 12:27:29.175987 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1cff9f5854a876c596bb8b12d2477f784cfb487900c01da703c539509eeff1df\": container with ID starting with 1cff9f5854a876c596bb8b12d2477f784cfb487900c01da703c539509eeff1df not found: ID does not exist" containerID="1cff9f5854a876c596bb8b12d2477f784cfb487900c01da703c539509eeff1df" Dec 03 12:27:29 crc kubenswrapper[5052]: I1203 12:27:29.176035 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1cff9f5854a876c596bb8b12d2477f784cfb487900c01da703c539509eeff1df"} err="failed to get container status \"1cff9f5854a876c596bb8b12d2477f784cfb487900c01da703c539509eeff1df\": rpc error: code = NotFound desc = could not find container \"1cff9f5854a876c596bb8b12d2477f784cfb487900c01da703c539509eeff1df\": container with ID starting with 1cff9f5854a876c596bb8b12d2477f784cfb487900c01da703c539509eeff1df not found: ID does not exist" Dec 03 12:27:29 crc kubenswrapper[5052]: I1203 12:27:29.176072 5052 scope.go:117] "RemoveContainer" containerID="6b08aea5dde842833792afa4a6cfad11a22c78249915f3be8d0c1416ac2fca1f" Dec 03 12:27:29 crc kubenswrapper[5052]: I1203 12:27:29.178439 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8n685\" (UniqueName: \"kubernetes.io/projected/f45e9eda-e62f-4307-ae18-2a8da7d6fe7c-kube-api-access-8n685\") pod \"ovsdbserver-nb-0\" (UID: \"f45e9eda-e62f-4307-ae18-2a8da7d6fe7c\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:27:29 crc kubenswrapper[5052]: I1203 12:27:29.178515 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f45e9eda-e62f-4307-ae18-2a8da7d6fe7c-config\") pod \"ovsdbserver-nb-0\" (UID: \"f45e9eda-e62f-4307-ae18-2a8da7d6fe7c\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:27:29 crc kubenswrapper[5052]: I1203 12:27:29.178592 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f45e9eda-e62f-4307-ae18-2a8da7d6fe7c-combined-ca-bundle\") pod \"ovsdbserver-nb-0\" (UID: \"f45e9eda-e62f-4307-ae18-2a8da7d6fe7c\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:27:29 crc kubenswrapper[5052]: I1203 12:27:29.178652 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovsdbserver-nb-tls-certs\" (UniqueName: \"kubernetes.io/secret/f45e9eda-e62f-4307-ae18-2a8da7d6fe7c-ovsdbserver-nb-tls-certs\") pod \"ovsdbserver-nb-0\" (UID: \"f45e9eda-e62f-4307-ae18-2a8da7d6fe7c\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:27:29 crc kubenswrapper[5052]: I1203 12:27:29.178676 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage03-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage03-crc\") pod \"ovsdbserver-nb-0\" (UID: \"f45e9eda-e62f-4307-ae18-2a8da7d6fe7c\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:27:29 crc kubenswrapper[5052]: I1203 12:27:29.178717 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/f45e9eda-e62f-4307-ae18-2a8da7d6fe7c-metrics-certs-tls-certs\") pod \"ovsdbserver-nb-0\" (UID: \"f45e9eda-e62f-4307-ae18-2a8da7d6fe7c\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:27:29 crc kubenswrapper[5052]: I1203 12:27:29.178740 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/f45e9eda-e62f-4307-ae18-2a8da7d6fe7c-scripts\") pod \"ovsdbserver-nb-0\" (UID: \"f45e9eda-e62f-4307-ae18-2a8da7d6fe7c\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:27:29 crc kubenswrapper[5052]: I1203 12:27:29.178870 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/f45e9eda-e62f-4307-ae18-2a8da7d6fe7c-ovsdb-rundir\") pod \"ovsdbserver-nb-0\" (UID: \"f45e9eda-e62f-4307-ae18-2a8da7d6fe7c\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:27:29 crc kubenswrapper[5052]: I1203 12:27:29.180374 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage03-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage03-crc\") pod \"ovsdbserver-nb-0\" (UID: \"f45e9eda-e62f-4307-ae18-2a8da7d6fe7c\") device mount path \"/mnt/openstack/pv03\"" pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:27:29 crc kubenswrapper[5052]: E1203 12:27:29.188751 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6b08aea5dde842833792afa4a6cfad11a22c78249915f3be8d0c1416ac2fca1f\": container with ID starting with 6b08aea5dde842833792afa4a6cfad11a22c78249915f3be8d0c1416ac2fca1f not found: ID does not exist" containerID="6b08aea5dde842833792afa4a6cfad11a22c78249915f3be8d0c1416ac2fca1f" Dec 03 12:27:29 crc kubenswrapper[5052]: I1203 12:27:29.188821 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6b08aea5dde842833792afa4a6cfad11a22c78249915f3be8d0c1416ac2fca1f"} err="failed to get container status \"6b08aea5dde842833792afa4a6cfad11a22c78249915f3be8d0c1416ac2fca1f\": rpc error: code = NotFound desc = could not find container \"6b08aea5dde842833792afa4a6cfad11a22c78249915f3be8d0c1416ac2fca1f\": container with ID starting with 6b08aea5dde842833792afa4a6cfad11a22c78249915f3be8d0c1416ac2fca1f not found: ID does not exist" Dec 03 12:27:29 crc kubenswrapper[5052]: I1203 12:27:29.188853 5052 scope.go:117] "RemoveContainer" containerID="1cff9f5854a876c596bb8b12d2477f784cfb487900c01da703c539509eeff1df" Dec 03 12:27:29 crc kubenswrapper[5052]: I1203 12:27:29.190357 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1cff9f5854a876c596bb8b12d2477f784cfb487900c01da703c539509eeff1df"} err="failed to get container status \"1cff9f5854a876c596bb8b12d2477f784cfb487900c01da703c539509eeff1df\": rpc error: code = NotFound desc = could not find container \"1cff9f5854a876c596bb8b12d2477f784cfb487900c01da703c539509eeff1df\": container with ID starting with 1cff9f5854a876c596bb8b12d2477f784cfb487900c01da703c539509eeff1df not found: ID does not exist" Dec 03 12:27:29 crc kubenswrapper[5052]: I1203 12:27:29.190401 5052 scope.go:117] "RemoveContainer" containerID="6b08aea5dde842833792afa4a6cfad11a22c78249915f3be8d0c1416ac2fca1f" Dec 03 12:27:29 crc kubenswrapper[5052]: I1203 12:27:29.191359 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6b08aea5dde842833792afa4a6cfad11a22c78249915f3be8d0c1416ac2fca1f"} err="failed to get container status \"6b08aea5dde842833792afa4a6cfad11a22c78249915f3be8d0c1416ac2fca1f\": rpc error: code = NotFound desc = could not find container \"6b08aea5dde842833792afa4a6cfad11a22c78249915f3be8d0c1416ac2fca1f\": container with ID starting with 6b08aea5dde842833792afa4a6cfad11a22c78249915f3be8d0c1416ac2fca1f not found: ID does not exist" Dec 03 12:27:29 crc kubenswrapper[5052]: I1203 12:27:29.230656 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:27:29 crc kubenswrapper[5052]: I1203 12:27:29.233160 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:27:29 crc kubenswrapper[5052]: I1203 12:27:29.277606 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:27:29 crc kubenswrapper[5052]: I1203 12:27:29.302608 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_neutron-797ccd5f76-v8cmg_5314ef6e-9ca6-4c9b-98b0-907b5a6e423c/neutron-api/0.log" Dec 03 12:27:29 crc kubenswrapper[5052]: I1203 12:27:29.302707 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-797ccd5f76-v8cmg" Dec 03 12:27:29 crc kubenswrapper[5052]: I1203 12:27:29.281615 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovsdbserver-nb-tls-certs\" (UniqueName: \"kubernetes.io/secret/f45e9eda-e62f-4307-ae18-2a8da7d6fe7c-ovsdbserver-nb-tls-certs\") pod \"ovsdbserver-nb-0\" (UID: \"f45e9eda-e62f-4307-ae18-2a8da7d6fe7c\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:27:29 crc kubenswrapper[5052]: I1203 12:27:29.303166 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/f45e9eda-e62f-4307-ae18-2a8da7d6fe7c-metrics-certs-tls-certs\") pod \"ovsdbserver-nb-0\" (UID: \"f45e9eda-e62f-4307-ae18-2a8da7d6fe7c\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:27:29 crc kubenswrapper[5052]: I1203 12:27:29.303200 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/f45e9eda-e62f-4307-ae18-2a8da7d6fe7c-scripts\") pod \"ovsdbserver-nb-0\" (UID: \"f45e9eda-e62f-4307-ae18-2a8da7d6fe7c\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:27:29 crc kubenswrapper[5052]: I1203 12:27:29.303479 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/f45e9eda-e62f-4307-ae18-2a8da7d6fe7c-ovsdb-rundir\") pod \"ovsdbserver-nb-0\" (UID: \"f45e9eda-e62f-4307-ae18-2a8da7d6fe7c\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:27:29 crc kubenswrapper[5052]: I1203 12:27:29.303521 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-8n685\" (UniqueName: \"kubernetes.io/projected/f45e9eda-e62f-4307-ae18-2a8da7d6fe7c-kube-api-access-8n685\") pod \"ovsdbserver-nb-0\" (UID: \"f45e9eda-e62f-4307-ae18-2a8da7d6fe7c\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:27:29 crc kubenswrapper[5052]: I1203 12:27:29.304564 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/f45e9eda-e62f-4307-ae18-2a8da7d6fe7c-ovsdb-rundir\") pod \"ovsdbserver-nb-0\" (UID: \"f45e9eda-e62f-4307-ae18-2a8da7d6fe7c\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:27:29 crc kubenswrapper[5052]: I1203 12:27:29.305699 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/f45e9eda-e62f-4307-ae18-2a8da7d6fe7c-scripts\") pod \"ovsdbserver-nb-0\" (UID: \"f45e9eda-e62f-4307-ae18-2a8da7d6fe7c\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:27:29 crc kubenswrapper[5052]: I1203 12:27:29.310706 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f45e9eda-e62f-4307-ae18-2a8da7d6fe7c-config\") pod \"ovsdbserver-nb-0\" (UID: \"f45e9eda-e62f-4307-ae18-2a8da7d6fe7c\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:27:29 crc kubenswrapper[5052]: I1203 12:27:29.321342 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovsdbserver-nb-tls-certs\" (UniqueName: \"kubernetes.io/secret/f45e9eda-e62f-4307-ae18-2a8da7d6fe7c-ovsdbserver-nb-tls-certs\") pod \"ovsdbserver-nb-0\" (UID: \"f45e9eda-e62f-4307-ae18-2a8da7d6fe7c\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:27:29 crc kubenswrapper[5052]: I1203 12:27:29.328870 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage03-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage03-crc\") pod \"ovsdbserver-nb-0\" (UID: \"f45e9eda-e62f-4307-ae18-2a8da7d6fe7c\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:27:29 crc kubenswrapper[5052]: I1203 12:27:29.330955 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f45e9eda-e62f-4307-ae18-2a8da7d6fe7c-config\") pod \"ovsdbserver-nb-0\" (UID: \"f45e9eda-e62f-4307-ae18-2a8da7d6fe7c\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:27:29 crc kubenswrapper[5052]: I1203 12:27:29.331794 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f45e9eda-e62f-4307-ae18-2a8da7d6fe7c-combined-ca-bundle\") pod \"ovsdbserver-nb-0\" (UID: \"f45e9eda-e62f-4307-ae18-2a8da7d6fe7c\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:27:29 crc kubenswrapper[5052]: I1203 12:27:29.334273 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-8n685\" (UniqueName: \"kubernetes.io/projected/f45e9eda-e62f-4307-ae18-2a8da7d6fe7c-kube-api-access-8n685\") pod \"ovsdbserver-nb-0\" (UID: \"f45e9eda-e62f-4307-ae18-2a8da7d6fe7c\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:27:29 crc kubenswrapper[5052]: I1203 12:27:29.335563 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f45e9eda-e62f-4307-ae18-2a8da7d6fe7c-combined-ca-bundle\") pod \"ovsdbserver-nb-0\" (UID: \"f45e9eda-e62f-4307-ae18-2a8da7d6fe7c\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:27:29 crc kubenswrapper[5052]: I1203 12:27:29.364079 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/f45e9eda-e62f-4307-ae18-2a8da7d6fe7c-metrics-certs-tls-certs\") pod \"ovsdbserver-nb-0\" (UID: \"f45e9eda-e62f-4307-ae18-2a8da7d6fe7c\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:27:29 crc kubenswrapper[5052]: I1203 12:27:29.433089 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/5314ef6e-9ca6-4c9b-98b0-907b5a6e423c-config\") pod \"5314ef6e-9ca6-4c9b-98b0-907b5a6e423c\" (UID: \"5314ef6e-9ca6-4c9b-98b0-907b5a6e423c\") " Dec 03 12:27:29 crc kubenswrapper[5052]: I1203 12:27:29.433169 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/5314ef6e-9ca6-4c9b-98b0-907b5a6e423c-httpd-config\") pod \"5314ef6e-9ca6-4c9b-98b0-907b5a6e423c\" (UID: \"5314ef6e-9ca6-4c9b-98b0-907b5a6e423c\") " Dec 03 12:27:29 crc kubenswrapper[5052]: I1203 12:27:29.433233 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4h5fs\" (UniqueName: \"kubernetes.io/projected/5314ef6e-9ca6-4c9b-98b0-907b5a6e423c-kube-api-access-4h5fs\") pod \"5314ef6e-9ca6-4c9b-98b0-907b5a6e423c\" (UID: \"5314ef6e-9ca6-4c9b-98b0-907b5a6e423c\") " Dec 03 12:27:29 crc kubenswrapper[5052]: I1203 12:27:29.433353 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5314ef6e-9ca6-4c9b-98b0-907b5a6e423c-combined-ca-bundle\") pod \"5314ef6e-9ca6-4c9b-98b0-907b5a6e423c\" (UID: \"5314ef6e-9ca6-4c9b-98b0-907b5a6e423c\") " Dec 03 12:27:29 crc kubenswrapper[5052]: I1203 12:27:29.440747 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5314ef6e-9ca6-4c9b-98b0-907b5a6e423c-httpd-config" (OuterVolumeSpecName: "httpd-config") pod "5314ef6e-9ca6-4c9b-98b0-907b5a6e423c" (UID: "5314ef6e-9ca6-4c9b-98b0-907b5a6e423c"). InnerVolumeSpecName "httpd-config". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:27:29 crc kubenswrapper[5052]: I1203 12:27:29.444860 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5314ef6e-9ca6-4c9b-98b0-907b5a6e423c-kube-api-access-4h5fs" (OuterVolumeSpecName: "kube-api-access-4h5fs") pod "5314ef6e-9ca6-4c9b-98b0-907b5a6e423c" (UID: "5314ef6e-9ca6-4c9b-98b0-907b5a6e423c"). InnerVolumeSpecName "kube-api-access-4h5fs". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:27:29 crc kubenswrapper[5052]: I1203 12:27:29.484066 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:27:29 crc kubenswrapper[5052]: I1203 12:27:29.518733 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5314ef6e-9ca6-4c9b-98b0-907b5a6e423c-config" (OuterVolumeSpecName: "config") pod "5314ef6e-9ca6-4c9b-98b0-907b5a6e423c" (UID: "5314ef6e-9ca6-4c9b-98b0-907b5a6e423c"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:27:29 crc kubenswrapper[5052]: I1203 12:27:29.532691 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5314ef6e-9ca6-4c9b-98b0-907b5a6e423c-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "5314ef6e-9ca6-4c9b-98b0-907b5a6e423c" (UID: "5314ef6e-9ca6-4c9b-98b0-907b5a6e423c"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:27:29 crc kubenswrapper[5052]: I1203 12:27:29.535143 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5314ef6e-9ca6-4c9b-98b0-907b5a6e423c-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:29 crc kubenswrapper[5052]: I1203 12:27:29.535180 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/5314ef6e-9ca6-4c9b-98b0-907b5a6e423c-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:29 crc kubenswrapper[5052]: I1203 12:27:29.535191 5052 reconciler_common.go:293] "Volume detached for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/5314ef6e-9ca6-4c9b-98b0-907b5a6e423c-httpd-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:29 crc kubenswrapper[5052]: I1203 12:27:29.535202 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-4h5fs\" (UniqueName: \"kubernetes.io/projected/5314ef6e-9ca6-4c9b-98b0-907b5a6e423c-kube-api-access-4h5fs\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:29 crc kubenswrapper[5052]: I1203 12:27:29.701711 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3b588591-8d98-427d-979f-72e7c8260b81" path="/var/lib/kubelet/pods/3b588591-8d98-427d-979f-72e7c8260b81/volumes" Dec 03 12:27:29 crc kubenswrapper[5052]: I1203 12:27:29.758931 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/barbican-api-5d855df9cb-xdg6p" Dec 03 12:27:29 crc kubenswrapper[5052]: I1203 12:27:29.982094 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"5d3294b3-d25c-4d8a-ba28-aaad7385693b","Type":"ContainerStarted","Data":"f1eb88cceb1d1670bec09abbba8e4706b62c84aa3f20a8a8d0f5e15a0d9bf509"} Dec 03 12:27:30 crc kubenswrapper[5052]: I1203 12:27:30.006761 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_neutron-797ccd5f76-v8cmg_5314ef6e-9ca6-4c9b-98b0-907b5a6e423c/neutron-api/0.log" Dec 03 12:27:30 crc kubenswrapper[5052]: I1203 12:27:30.008045 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-797ccd5f76-v8cmg" Dec 03 12:27:30 crc kubenswrapper[5052]: I1203 12:27:30.008184 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-797ccd5f76-v8cmg" event={"ID":"5314ef6e-9ca6-4c9b-98b0-907b5a6e423c","Type":"ContainerDied","Data":"8e4750b9efd2c05e5ba67059b1ff61da310da2de12da9f735d469317ccd1840f"} Dec 03 12:27:30 crc kubenswrapper[5052]: I1203 12:27:30.008250 5052 scope.go:117] "RemoveContainer" containerID="c989c0a379e7a6f19d77f112ad039489181d227f7ffa0c338a6048fa3a8be370" Dec 03 12:27:30 crc kubenswrapper[5052]: I1203 12:27:30.074411 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/neutron-797ccd5f76-v8cmg"] Dec 03 12:27:30 crc kubenswrapper[5052]: I1203 12:27:30.114867 5052 scope.go:117] "RemoveContainer" containerID="922671b6af1b9388525c76529a11dfbf987f5c33b66d696ad739f90656999386" Dec 03 12:27:30 crc kubenswrapper[5052]: I1203 12:27:30.114961 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/neutron-797ccd5f76-v8cmg"] Dec 03 12:27:30 crc kubenswrapper[5052]: I1203 12:27:30.315964 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-api-0" podUID="810e47c2-430b-424d-92ea-66c20ec8501f" containerName="nova-api-api" probeResult="failure" output="Get \"http://10.217.0.138:8774/\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Dec 03 12:27:30 crc kubenswrapper[5052]: I1203 12:27:30.316301 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-api-0" podUID="810e47c2-430b-424d-92ea-66c20ec8501f" containerName="nova-api-log" probeResult="failure" output="Get \"http://10.217.0.138:8774/\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Dec 03 12:27:30 crc kubenswrapper[5052]: I1203 12:27:30.324099 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-nb-0"] Dec 03 12:27:31 crc kubenswrapper[5052]: I1203 12:27:31.052115 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-nb-0" event={"ID":"f45e9eda-e62f-4307-ae18-2a8da7d6fe7c","Type":"ContainerStarted","Data":"09275335090deb24b922ed4694cb037b39bf0e6b72303e640ea22dfe86fa1716"} Dec 03 12:27:31 crc kubenswrapper[5052]: I1203 12:27:31.052416 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-nb-0" event={"ID":"f45e9eda-e62f-4307-ae18-2a8da7d6fe7c","Type":"ContainerStarted","Data":"51139b01af3d8c144c68f7acd6378312fcf37bab84f5b5451556bdcd8b8d38af"} Dec 03 12:27:31 crc kubenswrapper[5052]: I1203 12:27:31.130566 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-sb-0"] Dec 03 12:27:31 crc kubenswrapper[5052]: I1203 12:27:31.131779 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ovsdbserver-sb-0" podUID="c71ba7fe-fd95-4f72-a585-6e44cafa116e" containerName="openstack-network-exporter" containerID="cri-o://da31a3a0863901281ff074219ed7108e2f7b3e54f4ed5c3a2dc8467247c39bcb" gracePeriod=300 Dec 03 12:27:31 crc kubenswrapper[5052]: I1203 12:27:31.251756 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ovsdbserver-sb-0" podUID="c71ba7fe-fd95-4f72-a585-6e44cafa116e" containerName="ovsdbserver-sb" containerID="cri-o://5e3dd2ec9e951bd3f8596a23352b92ccc95b07f5ba4091471837424610f5aa50" gracePeriod=300 Dec 03 12:27:31 crc kubenswrapper[5052]: I1203 12:27:31.697888 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5314ef6e-9ca6-4c9b-98b0-907b5a6e423c" path="/var/lib/kubelet/pods/5314ef6e-9ca6-4c9b-98b0-907b5a6e423c/volumes" Dec 03 12:27:31 crc kubenswrapper[5052]: I1203 12:27:31.945691 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:27:31 crc kubenswrapper[5052]: I1203 12:27:31.945767 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:27:31 crc kubenswrapper[5052]: I1203 12:27:31.945784 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:27:31 crc kubenswrapper[5052]: I1203 12:27:31.945804 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:27:31 crc kubenswrapper[5052]: I1203 12:27:31.954871 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_ovsdbserver-sb-0_c71ba7fe-fd95-4f72-a585-6e44cafa116e/ovsdbserver-sb/0.log" Dec 03 12:27:31 crc kubenswrapper[5052]: I1203 12:27:31.954963 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:27:32 crc kubenswrapper[5052]: I1203 12:27:32.020906 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:27:32 crc kubenswrapper[5052]: I1203 12:27:32.021067 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:27:32 crc kubenswrapper[5052]: I1203 12:27:32.071963 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_ovsdbserver-sb-0_c71ba7fe-fd95-4f72-a585-6e44cafa116e/ovsdbserver-sb/0.log" Dec 03 12:27:32 crc kubenswrapper[5052]: I1203 12:27:32.072003 5052 generic.go:334] "Generic (PLEG): container finished" podID="c71ba7fe-fd95-4f72-a585-6e44cafa116e" containerID="da31a3a0863901281ff074219ed7108e2f7b3e54f4ed5c3a2dc8467247c39bcb" exitCode=2 Dec 03 12:27:32 crc kubenswrapper[5052]: I1203 12:27:32.072017 5052 generic.go:334] "Generic (PLEG): container finished" podID="c71ba7fe-fd95-4f72-a585-6e44cafa116e" containerID="5e3dd2ec9e951bd3f8596a23352b92ccc95b07f5ba4091471837424610f5aa50" exitCode=143 Dec 03 12:27:32 crc kubenswrapper[5052]: I1203 12:27:32.072059 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-sb-0" event={"ID":"c71ba7fe-fd95-4f72-a585-6e44cafa116e","Type":"ContainerDied","Data":"da31a3a0863901281ff074219ed7108e2f7b3e54f4ed5c3a2dc8467247c39bcb"} Dec 03 12:27:32 crc kubenswrapper[5052]: I1203 12:27:32.072087 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-sb-0" event={"ID":"c71ba7fe-fd95-4f72-a585-6e44cafa116e","Type":"ContainerDied","Data":"5e3dd2ec9e951bd3f8596a23352b92ccc95b07f5ba4091471837424610f5aa50"} Dec 03 12:27:32 crc kubenswrapper[5052]: I1203 12:27:32.072095 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-sb-0" event={"ID":"c71ba7fe-fd95-4f72-a585-6e44cafa116e","Type":"ContainerDied","Data":"d46eff63c6dc723589bc8fdfc72f53a6a78a9b10307c6d3a9dc6fd6dbe3028b5"} Dec 03 12:27:32 crc kubenswrapper[5052]: I1203 12:27:32.072111 5052 scope.go:117] "RemoveContainer" containerID="da31a3a0863901281ff074219ed7108e2f7b3e54f4ed5c3a2dc8467247c39bcb" Dec 03 12:27:32 crc kubenswrapper[5052]: I1203 12:27:32.072204 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:27:32 crc kubenswrapper[5052]: I1203 12:27:32.100442 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-nb-0" event={"ID":"f45e9eda-e62f-4307-ae18-2a8da7d6fe7c","Type":"ContainerStarted","Data":"901636e0be8460bcc74b0c6a41492e349cbdbd334bbc41042f9dfdf0a187e9d1"} Dec 03 12:27:32 crc kubenswrapper[5052]: I1203 12:27:32.113870 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"5d3294b3-d25c-4d8a-ba28-aaad7385693b","Type":"ContainerStarted","Data":"7e8b797c3e63fa66aad0206d3411e34757b19c124c459e9c5d9dc7e3133dad59"} Dec 03 12:27:32 crc kubenswrapper[5052]: I1203 12:27:32.113940 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:27:32 crc kubenswrapper[5052]: I1203 12:27:32.114259 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:27:32 crc kubenswrapper[5052]: I1203 12:27:32.114588 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:27:32 crc kubenswrapper[5052]: I1203 12:27:32.141213 5052 scope.go:117] "RemoveContainer" containerID="5e3dd2ec9e951bd3f8596a23352b92ccc95b07f5ba4091471837424610f5aa50" Dec 03 12:27:32 crc kubenswrapper[5052]: I1203 12:27:32.147507 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c71ba7fe-fd95-4f72-a585-6e44cafa116e-config\") pod \"c71ba7fe-fd95-4f72-a585-6e44cafa116e\" (UID: \"c71ba7fe-fd95-4f72-a585-6e44cafa116e\") " Dec 03 12:27:32 crc kubenswrapper[5052]: I1203 12:27:32.147638 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/c71ba7fe-fd95-4f72-a585-6e44cafa116e-scripts\") pod \"c71ba7fe-fd95-4f72-a585-6e44cafa116e\" (UID: \"c71ba7fe-fd95-4f72-a585-6e44cafa116e\") " Dec 03 12:27:32 crc kubenswrapper[5052]: I1203 12:27:32.147706 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/c71ba7fe-fd95-4f72-a585-6e44cafa116e-ovsdb-rundir\") pod \"c71ba7fe-fd95-4f72-a585-6e44cafa116e\" (UID: \"c71ba7fe-fd95-4f72-a585-6e44cafa116e\") " Dec 03 12:27:32 crc kubenswrapper[5052]: I1203 12:27:32.147758 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c71ba7fe-fd95-4f72-a585-6e44cafa116e-combined-ca-bundle\") pod \"c71ba7fe-fd95-4f72-a585-6e44cafa116e\" (UID: \"c71ba7fe-fd95-4f72-a585-6e44cafa116e\") " Dec 03 12:27:32 crc kubenswrapper[5052]: I1203 12:27:32.147788 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-78cj2\" (UniqueName: \"kubernetes.io/projected/c71ba7fe-fd95-4f72-a585-6e44cafa116e-kube-api-access-78cj2\") pod \"c71ba7fe-fd95-4f72-a585-6e44cafa116e\" (UID: \"c71ba7fe-fd95-4f72-a585-6e44cafa116e\") " Dec 03 12:27:32 crc kubenswrapper[5052]: I1203 12:27:32.147855 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovndbcluster-sb-etc-ovn\" (UniqueName: \"kubernetes.io/local-volume/local-storage08-crc\") pod \"c71ba7fe-fd95-4f72-a585-6e44cafa116e\" (UID: \"c71ba7fe-fd95-4f72-a585-6e44cafa116e\") " Dec 03 12:27:32 crc kubenswrapper[5052]: I1203 12:27:32.148475 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c71ba7fe-fd95-4f72-a585-6e44cafa116e-ovsdb-rundir" (OuterVolumeSpecName: "ovsdb-rundir") pod "c71ba7fe-fd95-4f72-a585-6e44cafa116e" (UID: "c71ba7fe-fd95-4f72-a585-6e44cafa116e"). InnerVolumeSpecName "ovsdb-rundir". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:27:32 crc kubenswrapper[5052]: I1203 12:27:32.149172 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c71ba7fe-fd95-4f72-a585-6e44cafa116e-config" (OuterVolumeSpecName: "config") pod "c71ba7fe-fd95-4f72-a585-6e44cafa116e" (UID: "c71ba7fe-fd95-4f72-a585-6e44cafa116e"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:27:32 crc kubenswrapper[5052]: I1203 12:27:32.149739 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c71ba7fe-fd95-4f72-a585-6e44cafa116e-scripts" (OuterVolumeSpecName: "scripts") pod "c71ba7fe-fd95-4f72-a585-6e44cafa116e" (UID: "c71ba7fe-fd95-4f72-a585-6e44cafa116e"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:27:32 crc kubenswrapper[5052]: I1203 12:27:32.163249 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c71ba7fe-fd95-4f72-a585-6e44cafa116e-kube-api-access-78cj2" (OuterVolumeSpecName: "kube-api-access-78cj2") pod "c71ba7fe-fd95-4f72-a585-6e44cafa116e" (UID: "c71ba7fe-fd95-4f72-a585-6e44cafa116e"). InnerVolumeSpecName "kube-api-access-78cj2". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:27:32 crc kubenswrapper[5052]: I1203 12:27:32.190987 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage08-crc" (OuterVolumeSpecName: "ovndbcluster-sb-etc-ovn") pod "c71ba7fe-fd95-4f72-a585-6e44cafa116e" (UID: "c71ba7fe-fd95-4f72-a585-6e44cafa116e"). InnerVolumeSpecName "local-storage08-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 12:27:32 crc kubenswrapper[5052]: I1203 12:27:32.195617 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/ceilometer-0" podStartSLOduration=3.299260371 podStartE2EDuration="8.195595482s" podCreationTimestamp="2025-12-03 12:27:24 +0000 UTC" firstStartedPulling="2025-12-03 12:27:26.12447041 +0000 UTC m=+4036.712712174" lastFinishedPulling="2025-12-03 12:27:31.020805521 +0000 UTC m=+4041.609047285" observedRunningTime="2025-12-03 12:27:32.191402922 +0000 UTC m=+4042.779644716" watchObservedRunningTime="2025-12-03 12:27:32.195595482 +0000 UTC m=+4042.783837246" Dec 03 12:27:32 crc kubenswrapper[5052]: I1203 12:27:32.208636 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/ovsdbserver-nb-0" podStartSLOduration=3.208614703 podStartE2EDuration="3.208614703s" podCreationTimestamp="2025-12-03 12:27:29 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:27:32.1328159 +0000 UTC m=+4042.721057664" watchObservedRunningTime="2025-12-03 12:27:32.208614703 +0000 UTC m=+4042.796856467" Dec 03 12:27:32 crc kubenswrapper[5052]: I1203 12:27:32.217758 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c71ba7fe-fd95-4f72-a585-6e44cafa116e-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "c71ba7fe-fd95-4f72-a585-6e44cafa116e" (UID: "c71ba7fe-fd95-4f72-a585-6e44cafa116e"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:27:32 crc kubenswrapper[5052]: I1203 12:27:32.249946 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage08-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage08-crc\") on node \"crc\" " Dec 03 12:27:32 crc kubenswrapper[5052]: I1203 12:27:32.249983 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c71ba7fe-fd95-4f72-a585-6e44cafa116e-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:32 crc kubenswrapper[5052]: I1203 12:27:32.249993 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/c71ba7fe-fd95-4f72-a585-6e44cafa116e-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:32 crc kubenswrapper[5052]: I1203 12:27:32.250002 5052 reconciler_common.go:293] "Volume detached for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/c71ba7fe-fd95-4f72-a585-6e44cafa116e-ovsdb-rundir\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:32 crc kubenswrapper[5052]: I1203 12:27:32.250012 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c71ba7fe-fd95-4f72-a585-6e44cafa116e-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:32 crc kubenswrapper[5052]: I1203 12:27:32.250021 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-78cj2\" (UniqueName: \"kubernetes.io/projected/c71ba7fe-fd95-4f72-a585-6e44cafa116e-kube-api-access-78cj2\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:32 crc kubenswrapper[5052]: I1203 12:27:32.294727 5052 scope.go:117] "RemoveContainer" containerID="da31a3a0863901281ff074219ed7108e2f7b3e54f4ed5c3a2dc8467247c39bcb" Dec 03 12:27:32 crc kubenswrapper[5052]: E1203 12:27:32.329939 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"da31a3a0863901281ff074219ed7108e2f7b3e54f4ed5c3a2dc8467247c39bcb\": container with ID starting with da31a3a0863901281ff074219ed7108e2f7b3e54f4ed5c3a2dc8467247c39bcb not found: ID does not exist" containerID="da31a3a0863901281ff074219ed7108e2f7b3e54f4ed5c3a2dc8467247c39bcb" Dec 03 12:27:32 crc kubenswrapper[5052]: I1203 12:27:32.329983 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"da31a3a0863901281ff074219ed7108e2f7b3e54f4ed5c3a2dc8467247c39bcb"} err="failed to get container status \"da31a3a0863901281ff074219ed7108e2f7b3e54f4ed5c3a2dc8467247c39bcb\": rpc error: code = NotFound desc = could not find container \"da31a3a0863901281ff074219ed7108e2f7b3e54f4ed5c3a2dc8467247c39bcb\": container with ID starting with da31a3a0863901281ff074219ed7108e2f7b3e54f4ed5c3a2dc8467247c39bcb not found: ID does not exist" Dec 03 12:27:32 crc kubenswrapper[5052]: I1203 12:27:32.330008 5052 scope.go:117] "RemoveContainer" containerID="5e3dd2ec9e951bd3f8596a23352b92ccc95b07f5ba4091471837424610f5aa50" Dec 03 12:27:32 crc kubenswrapper[5052]: E1203 12:27:32.332639 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5e3dd2ec9e951bd3f8596a23352b92ccc95b07f5ba4091471837424610f5aa50\": container with ID starting with 5e3dd2ec9e951bd3f8596a23352b92ccc95b07f5ba4091471837424610f5aa50 not found: ID does not exist" containerID="5e3dd2ec9e951bd3f8596a23352b92ccc95b07f5ba4091471837424610f5aa50" Dec 03 12:27:32 crc kubenswrapper[5052]: I1203 12:27:32.332675 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5e3dd2ec9e951bd3f8596a23352b92ccc95b07f5ba4091471837424610f5aa50"} err="failed to get container status \"5e3dd2ec9e951bd3f8596a23352b92ccc95b07f5ba4091471837424610f5aa50\": rpc error: code = NotFound desc = could not find container \"5e3dd2ec9e951bd3f8596a23352b92ccc95b07f5ba4091471837424610f5aa50\": container with ID starting with 5e3dd2ec9e951bd3f8596a23352b92ccc95b07f5ba4091471837424610f5aa50 not found: ID does not exist" Dec 03 12:27:32 crc kubenswrapper[5052]: I1203 12:27:32.332694 5052 scope.go:117] "RemoveContainer" containerID="da31a3a0863901281ff074219ed7108e2f7b3e54f4ed5c3a2dc8467247c39bcb" Dec 03 12:27:32 crc kubenswrapper[5052]: I1203 12:27:32.333941 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"da31a3a0863901281ff074219ed7108e2f7b3e54f4ed5c3a2dc8467247c39bcb"} err="failed to get container status \"da31a3a0863901281ff074219ed7108e2f7b3e54f4ed5c3a2dc8467247c39bcb\": rpc error: code = NotFound desc = could not find container \"da31a3a0863901281ff074219ed7108e2f7b3e54f4ed5c3a2dc8467247c39bcb\": container with ID starting with da31a3a0863901281ff074219ed7108e2f7b3e54f4ed5c3a2dc8467247c39bcb not found: ID does not exist" Dec 03 12:27:32 crc kubenswrapper[5052]: I1203 12:27:32.333976 5052 scope.go:117] "RemoveContainer" containerID="5e3dd2ec9e951bd3f8596a23352b92ccc95b07f5ba4091471837424610f5aa50" Dec 03 12:27:32 crc kubenswrapper[5052]: I1203 12:27:32.334301 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5e3dd2ec9e951bd3f8596a23352b92ccc95b07f5ba4091471837424610f5aa50"} err="failed to get container status \"5e3dd2ec9e951bd3f8596a23352b92ccc95b07f5ba4091471837424610f5aa50\": rpc error: code = NotFound desc = could not find container \"5e3dd2ec9e951bd3f8596a23352b92ccc95b07f5ba4091471837424610f5aa50\": container with ID starting with 5e3dd2ec9e951bd3f8596a23352b92ccc95b07f5ba4091471837424610f5aa50 not found: ID does not exist" Dec 03 12:27:32 crc kubenswrapper[5052]: I1203 12:27:32.421125 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage08-crc" (UniqueName: "kubernetes.io/local-volume/local-storage08-crc") on node "crc" Dec 03 12:27:32 crc kubenswrapper[5052]: I1203 12:27:32.462891 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage08-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage08-crc\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:32 crc kubenswrapper[5052]: I1203 12:27:32.485338 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:27:32 crc kubenswrapper[5052]: I1203 12:27:32.487620 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-sb-0"] Dec 03 12:27:32 crc kubenswrapper[5052]: I1203 12:27:32.538153 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-sb-0"] Dec 03 12:27:32 crc kubenswrapper[5052]: I1203 12:27:32.588624 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ovsdbserver-sb-0"] Dec 03 12:27:32 crc kubenswrapper[5052]: E1203 12:27:32.589083 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="5314ef6e-9ca6-4c9b-98b0-907b5a6e423c" containerName="neutron-httpd" Dec 03 12:27:32 crc kubenswrapper[5052]: I1203 12:27:32.589096 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="5314ef6e-9ca6-4c9b-98b0-907b5a6e423c" containerName="neutron-httpd" Dec 03 12:27:32 crc kubenswrapper[5052]: E1203 12:27:32.589152 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="5314ef6e-9ca6-4c9b-98b0-907b5a6e423c" containerName="neutron-api" Dec 03 12:27:32 crc kubenswrapper[5052]: I1203 12:27:32.589159 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="5314ef6e-9ca6-4c9b-98b0-907b5a6e423c" containerName="neutron-api" Dec 03 12:27:32 crc kubenswrapper[5052]: E1203 12:27:32.589176 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="c71ba7fe-fd95-4f72-a585-6e44cafa116e" containerName="openstack-network-exporter" Dec 03 12:27:32 crc kubenswrapper[5052]: I1203 12:27:32.589188 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="c71ba7fe-fd95-4f72-a585-6e44cafa116e" containerName="openstack-network-exporter" Dec 03 12:27:32 crc kubenswrapper[5052]: E1203 12:27:32.589226 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="c71ba7fe-fd95-4f72-a585-6e44cafa116e" containerName="ovsdbserver-sb" Dec 03 12:27:32 crc kubenswrapper[5052]: I1203 12:27:32.589234 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="c71ba7fe-fd95-4f72-a585-6e44cafa116e" containerName="ovsdbserver-sb" Dec 03 12:27:32 crc kubenswrapper[5052]: I1203 12:27:32.589470 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="c71ba7fe-fd95-4f72-a585-6e44cafa116e" containerName="ovsdbserver-sb" Dec 03 12:27:32 crc kubenswrapper[5052]: I1203 12:27:32.589490 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="c71ba7fe-fd95-4f72-a585-6e44cafa116e" containerName="openstack-network-exporter" Dec 03 12:27:32 crc kubenswrapper[5052]: I1203 12:27:32.589516 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="5314ef6e-9ca6-4c9b-98b0-907b5a6e423c" containerName="neutron-api" Dec 03 12:27:32 crc kubenswrapper[5052]: I1203 12:27:32.589550 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="5314ef6e-9ca6-4c9b-98b0-907b5a6e423c" containerName="neutron-httpd" Dec 03 12:27:32 crc kubenswrapper[5052]: I1203 12:27:32.593188 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:27:32 crc kubenswrapper[5052]: I1203 12:27:32.598935 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ovncluster-ovndbcluster-sb-dockercfg-2ckjk" Dec 03 12:27:32 crc kubenswrapper[5052]: I1203 12:27:32.599167 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-ovndbcluster-sb-ovndbs" Dec 03 12:27:32 crc kubenswrapper[5052]: I1203 12:27:32.599288 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"ovndbcluster-sb-scripts" Dec 03 12:27:32 crc kubenswrapper[5052]: I1203 12:27:32.606022 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"ovndbcluster-sb-config" Dec 03 12:27:32 crc kubenswrapper[5052]: I1203 12:27:32.609968 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-sb-0"] Dec 03 12:27:32 crc kubenswrapper[5052]: I1203 12:27:32.669013 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/5c7fe95d-f637-4021-a5e2-69668046d2ff-metrics-certs-tls-certs\") pod \"ovsdbserver-sb-0\" (UID: \"5c7fe95d-f637-4021-a5e2-69668046d2ff\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:27:32 crc kubenswrapper[5052]: I1203 12:27:32.669244 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5c7fe95d-f637-4021-a5e2-69668046d2ff-combined-ca-bundle\") pod \"ovsdbserver-sb-0\" (UID: \"5c7fe95d-f637-4021-a5e2-69668046d2ff\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:27:32 crc kubenswrapper[5052]: I1203 12:27:32.669434 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage08-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage08-crc\") pod \"ovsdbserver-sb-0\" (UID: \"5c7fe95d-f637-4021-a5e2-69668046d2ff\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:27:32 crc kubenswrapper[5052]: I1203 12:27:32.669549 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/5c7fe95d-f637-4021-a5e2-69668046d2ff-ovsdb-rundir\") pod \"ovsdbserver-sb-0\" (UID: \"5c7fe95d-f637-4021-a5e2-69668046d2ff\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:27:32 crc kubenswrapper[5052]: I1203 12:27:32.669661 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5c7fe95d-f637-4021-a5e2-69668046d2ff-config\") pod \"ovsdbserver-sb-0\" (UID: \"5c7fe95d-f637-4021-a5e2-69668046d2ff\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:27:32 crc kubenswrapper[5052]: I1203 12:27:32.669762 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovsdbserver-sb-tls-certs\" (UniqueName: \"kubernetes.io/secret/5c7fe95d-f637-4021-a5e2-69668046d2ff-ovsdbserver-sb-tls-certs\") pod \"ovsdbserver-sb-0\" (UID: \"5c7fe95d-f637-4021-a5e2-69668046d2ff\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:27:32 crc kubenswrapper[5052]: I1203 12:27:32.669880 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/5c7fe95d-f637-4021-a5e2-69668046d2ff-scripts\") pod \"ovsdbserver-sb-0\" (UID: \"5c7fe95d-f637-4021-a5e2-69668046d2ff\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:27:32 crc kubenswrapper[5052]: I1203 12:27:32.669992 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-67l4w\" (UniqueName: \"kubernetes.io/projected/5c7fe95d-f637-4021-a5e2-69668046d2ff-kube-api-access-67l4w\") pod \"ovsdbserver-sb-0\" (UID: \"5c7fe95d-f637-4021-a5e2-69668046d2ff\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:27:32 crc kubenswrapper[5052]: I1203 12:27:32.697370 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/barbican-api-5d855df9cb-xdg6p" Dec 03 12:27:32 crc kubenswrapper[5052]: I1203 12:27:32.770733 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage08-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage08-crc\") pod \"ovsdbserver-sb-0\" (UID: \"5c7fe95d-f637-4021-a5e2-69668046d2ff\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:27:32 crc kubenswrapper[5052]: I1203 12:27:32.770785 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/5c7fe95d-f637-4021-a5e2-69668046d2ff-ovsdb-rundir\") pod \"ovsdbserver-sb-0\" (UID: \"5c7fe95d-f637-4021-a5e2-69668046d2ff\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:27:32 crc kubenswrapper[5052]: I1203 12:27:32.770818 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5c7fe95d-f637-4021-a5e2-69668046d2ff-config\") pod \"ovsdbserver-sb-0\" (UID: \"5c7fe95d-f637-4021-a5e2-69668046d2ff\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:27:32 crc kubenswrapper[5052]: I1203 12:27:32.770855 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovsdbserver-sb-tls-certs\" (UniqueName: \"kubernetes.io/secret/5c7fe95d-f637-4021-a5e2-69668046d2ff-ovsdbserver-sb-tls-certs\") pod \"ovsdbserver-sb-0\" (UID: \"5c7fe95d-f637-4021-a5e2-69668046d2ff\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:27:32 crc kubenswrapper[5052]: I1203 12:27:32.770899 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/5c7fe95d-f637-4021-a5e2-69668046d2ff-scripts\") pod \"ovsdbserver-sb-0\" (UID: \"5c7fe95d-f637-4021-a5e2-69668046d2ff\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:27:32 crc kubenswrapper[5052]: I1203 12:27:32.770941 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-67l4w\" (UniqueName: \"kubernetes.io/projected/5c7fe95d-f637-4021-a5e2-69668046d2ff-kube-api-access-67l4w\") pod \"ovsdbserver-sb-0\" (UID: \"5c7fe95d-f637-4021-a5e2-69668046d2ff\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:27:32 crc kubenswrapper[5052]: I1203 12:27:32.770970 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/5c7fe95d-f637-4021-a5e2-69668046d2ff-metrics-certs-tls-certs\") pod \"ovsdbserver-sb-0\" (UID: \"5c7fe95d-f637-4021-a5e2-69668046d2ff\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:27:32 crc kubenswrapper[5052]: I1203 12:27:32.770998 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5c7fe95d-f637-4021-a5e2-69668046d2ff-combined-ca-bundle\") pod \"ovsdbserver-sb-0\" (UID: \"5c7fe95d-f637-4021-a5e2-69668046d2ff\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:27:32 crc kubenswrapper[5052]: I1203 12:27:32.773081 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage08-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage08-crc\") pod \"ovsdbserver-sb-0\" (UID: \"5c7fe95d-f637-4021-a5e2-69668046d2ff\") device mount path \"/mnt/openstack/pv08\"" pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:27:32 crc kubenswrapper[5052]: I1203 12:27:32.774263 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/5c7fe95d-f637-4021-a5e2-69668046d2ff-ovsdb-rundir\") pod \"ovsdbserver-sb-0\" (UID: \"5c7fe95d-f637-4021-a5e2-69668046d2ff\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:27:32 crc kubenswrapper[5052]: I1203 12:27:32.775164 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5c7fe95d-f637-4021-a5e2-69668046d2ff-config\") pod \"ovsdbserver-sb-0\" (UID: \"5c7fe95d-f637-4021-a5e2-69668046d2ff\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:27:32 crc kubenswrapper[5052]: I1203 12:27:32.780206 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/5c7fe95d-f637-4021-a5e2-69668046d2ff-scripts\") pod \"ovsdbserver-sb-0\" (UID: \"5c7fe95d-f637-4021-a5e2-69668046d2ff\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:27:32 crc kubenswrapper[5052]: I1203 12:27:32.788459 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5c7fe95d-f637-4021-a5e2-69668046d2ff-combined-ca-bundle\") pod \"ovsdbserver-sb-0\" (UID: \"5c7fe95d-f637-4021-a5e2-69668046d2ff\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:27:32 crc kubenswrapper[5052]: I1203 12:27:32.789404 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/5c7fe95d-f637-4021-a5e2-69668046d2ff-metrics-certs-tls-certs\") pod \"ovsdbserver-sb-0\" (UID: \"5c7fe95d-f637-4021-a5e2-69668046d2ff\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:27:32 crc kubenswrapper[5052]: I1203 12:27:32.797043 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-api-9c7764976-6d6sh"] Dec 03 12:27:32 crc kubenswrapper[5052]: I1203 12:27:32.797323 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/barbican-api-9c7764976-6d6sh" podUID="139928b7-e862-4916-8ca2-37917fa153d7" containerName="barbican-api-log" containerID="cri-o://d1675f69e208967311cc8f60ca7ef2dbea31a0db8566c33dc551f51568901da7" gracePeriod=30 Dec 03 12:27:32 crc kubenswrapper[5052]: I1203 12:27:32.797944 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/barbican-api-9c7764976-6d6sh" podUID="139928b7-e862-4916-8ca2-37917fa153d7" containerName="barbican-api" containerID="cri-o://133d1aba158f4f11359e5209c49b976cf1345d171a018b7bb2ac3533efc2b0ca" gracePeriod=30 Dec 03 12:27:32 crc kubenswrapper[5052]: I1203 12:27:32.812790 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-67l4w\" (UniqueName: \"kubernetes.io/projected/5c7fe95d-f637-4021-a5e2-69668046d2ff-kube-api-access-67l4w\") pod \"ovsdbserver-sb-0\" (UID: \"5c7fe95d-f637-4021-a5e2-69668046d2ff\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:27:32 crc kubenswrapper[5052]: I1203 12:27:32.831307 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovsdbserver-sb-tls-certs\" (UniqueName: \"kubernetes.io/secret/5c7fe95d-f637-4021-a5e2-69668046d2ff-ovsdbserver-sb-tls-certs\") pod \"ovsdbserver-sb-0\" (UID: \"5c7fe95d-f637-4021-a5e2-69668046d2ff\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:27:32 crc kubenswrapper[5052]: I1203 12:27:32.875881 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage08-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage08-crc\") pod \"ovsdbserver-sb-0\" (UID: \"5c7fe95d-f637-4021-a5e2-69668046d2ff\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:27:32 crc kubenswrapper[5052]: E1203 12:27:32.885126 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="dda098dd0ab939491d6cb42aac03c4daa0a17e93eaa82fa8ed7c6dec3c2a4f1b" cmd=["/usr/local/bin/container-scripts/status_check.sh"] Dec 03 12:27:32 crc kubenswrapper[5052]: E1203 12:27:32.886872 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="dda098dd0ab939491d6cb42aac03c4daa0a17e93eaa82fa8ed7c6dec3c2a4f1b" cmd=["/usr/local/bin/container-scripts/status_check.sh"] Dec 03 12:27:32 crc kubenswrapper[5052]: E1203 12:27:32.889677 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="dda098dd0ab939491d6cb42aac03c4daa0a17e93eaa82fa8ed7c6dec3c2a4f1b" cmd=["/usr/local/bin/container-scripts/status_check.sh"] Dec 03 12:27:32 crc kubenswrapper[5052]: E1203 12:27:32.889853 5052 prober.go:104] "Probe errored" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" probeType="Readiness" pod="openstack-kuttl-tests/ovn-northd-0" podUID="18e325b5-7837-4776-ae05-d9a12ccf9746" containerName="ovn-northd" Dec 03 12:27:32 crc kubenswrapper[5052]: I1203 12:27:32.943986 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:27:33 crc kubenswrapper[5052]: I1203 12:27:33.033917 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-metadata-0" podUID="8a2c61ae-6521-424f-aeac-52e1ec1c319a" containerName="nova-metadata-log" probeResult="failure" output="Get \"http://10.217.0.141:8775/\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Dec 03 12:27:33 crc kubenswrapper[5052]: I1203 12:27:33.034335 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-metadata-0" podUID="8a2c61ae-6521-424f-aeac-52e1ec1c319a" containerName="nova-metadata-metadata" probeResult="failure" output="Get \"http://10.217.0.141:8775/\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Dec 03 12:27:33 crc kubenswrapper[5052]: I1203 12:27:33.157964 5052 generic.go:334] "Generic (PLEG): container finished" podID="139928b7-e862-4916-8ca2-37917fa153d7" containerID="d1675f69e208967311cc8f60ca7ef2dbea31a0db8566c33dc551f51568901da7" exitCode=143 Dec 03 12:27:33 crc kubenswrapper[5052]: I1203 12:27:33.158738 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-9c7764976-6d6sh" event={"ID":"139928b7-e862-4916-8ca2-37917fa153d7","Type":"ContainerDied","Data":"d1675f69e208967311cc8f60ca7ef2dbea31a0db8566c33dc551f51568901da7"} Dec 03 12:27:33 crc kubenswrapper[5052]: I1203 12:27:33.174029 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_neutron-6bf55bd9d6-mrgs9_15cf6575-b6be-4f67-861a-61ebf02d1495/neutron-api/0.log" Dec 03 12:27:33 crc kubenswrapper[5052]: I1203 12:27:33.174080 5052 generic.go:334] "Generic (PLEG): container finished" podID="15cf6575-b6be-4f67-861a-61ebf02d1495" containerID="6b5f5843b9647706bd37b6d16a4de3c4357a058f24ee264c620b57797b2edcdc" exitCode=1 Dec 03 12:27:33 crc kubenswrapper[5052]: I1203 12:27:33.174160 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-6bf55bd9d6-mrgs9" event={"ID":"15cf6575-b6be-4f67-861a-61ebf02d1495","Type":"ContainerDied","Data":"6b5f5843b9647706bd37b6d16a4de3c4357a058f24ee264c620b57797b2edcdc"} Dec 03 12:27:33 crc kubenswrapper[5052]: I1203 12:27:33.178717 5052 scope.go:117] "RemoveContainer" containerID="6b5f5843b9647706bd37b6d16a4de3c4357a058f24ee264c620b57797b2edcdc" Dec 03 12:27:33 crc kubenswrapper[5052]: I1203 12:27:33.187725 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/neutron-6bf55bd9d6-mrgs9" podUID="15cf6575-b6be-4f67-861a-61ebf02d1495" containerName="neutron-httpd" probeResult="failure" output="HTTP probe failed with statuscode: 503" Dec 03 12:27:33 crc kubenswrapper[5052]: I1203 12:27:33.568297 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-sb-0"] Dec 03 12:27:33 crc kubenswrapper[5052]: I1203 12:27:33.741651 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c71ba7fe-fd95-4f72-a585-6e44cafa116e" path="/var/lib/kubelet/pods/c71ba7fe-fd95-4f72-a585-6e44cafa116e/volumes" Dec 03 12:27:33 crc kubenswrapper[5052]: I1203 12:27:33.966349 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_ovn-northd-0_18e325b5-7837-4776-ae05-d9a12ccf9746/ovn-northd/0.log" Dec 03 12:27:33 crc kubenswrapper[5052]: I1203 12:27:33.966714 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:27:34 crc kubenswrapper[5052]: I1203 12:27:34.119565 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/18e325b5-7837-4776-ae05-d9a12ccf9746-config\") pod \"18e325b5-7837-4776-ae05-d9a12ccf9746\" (UID: \"18e325b5-7837-4776-ae05-d9a12ccf9746\") " Dec 03 12:27:34 crc kubenswrapper[5052]: I1203 12:27:34.120233 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jpxt6\" (UniqueName: \"kubernetes.io/projected/18e325b5-7837-4776-ae05-d9a12ccf9746-kube-api-access-jpxt6\") pod \"18e325b5-7837-4776-ae05-d9a12ccf9746\" (UID: \"18e325b5-7837-4776-ae05-d9a12ccf9746\") " Dec 03 12:27:34 crc kubenswrapper[5052]: I1203 12:27:34.120359 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/18e325b5-7837-4776-ae05-d9a12ccf9746-combined-ca-bundle\") pod \"18e325b5-7837-4776-ae05-d9a12ccf9746\" (UID: \"18e325b5-7837-4776-ae05-d9a12ccf9746\") " Dec 03 12:27:34 crc kubenswrapper[5052]: I1203 12:27:34.120471 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/18e325b5-7837-4776-ae05-d9a12ccf9746-scripts\") pod \"18e325b5-7837-4776-ae05-d9a12ccf9746\" (UID: \"18e325b5-7837-4776-ae05-d9a12ccf9746\") " Dec 03 12:27:34 crc kubenswrapper[5052]: I1203 12:27:34.120840 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovn-rundir\" (UniqueName: \"kubernetes.io/empty-dir/18e325b5-7837-4776-ae05-d9a12ccf9746-ovn-rundir\") pod \"18e325b5-7837-4776-ae05-d9a12ccf9746\" (UID: \"18e325b5-7837-4776-ae05-d9a12ccf9746\") " Dec 03 12:27:34 crc kubenswrapper[5052]: I1203 12:27:34.120196 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/18e325b5-7837-4776-ae05-d9a12ccf9746-config" (OuterVolumeSpecName: "config") pod "18e325b5-7837-4776-ae05-d9a12ccf9746" (UID: "18e325b5-7837-4776-ae05-d9a12ccf9746"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:27:34 crc kubenswrapper[5052]: I1203 12:27:34.121241 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/18e325b5-7837-4776-ae05-d9a12ccf9746-scripts" (OuterVolumeSpecName: "scripts") pod "18e325b5-7837-4776-ae05-d9a12ccf9746" (UID: "18e325b5-7837-4776-ae05-d9a12ccf9746"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:27:34 crc kubenswrapper[5052]: I1203 12:27:34.121812 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/18e325b5-7837-4776-ae05-d9a12ccf9746-ovn-rundir" (OuterVolumeSpecName: "ovn-rundir") pod "18e325b5-7837-4776-ae05-d9a12ccf9746" (UID: "18e325b5-7837-4776-ae05-d9a12ccf9746"). InnerVolumeSpecName "ovn-rundir". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:27:34 crc kubenswrapper[5052]: I1203 12:27:34.122175 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/18e325b5-7837-4776-ae05-d9a12ccf9746-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:34 crc kubenswrapper[5052]: I1203 12:27:34.122273 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/18e325b5-7837-4776-ae05-d9a12ccf9746-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:34 crc kubenswrapper[5052]: I1203 12:27:34.122359 5052 reconciler_common.go:293] "Volume detached for volume \"ovn-rundir\" (UniqueName: \"kubernetes.io/empty-dir/18e325b5-7837-4776-ae05-d9a12ccf9746-ovn-rundir\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:34 crc kubenswrapper[5052]: I1203 12:27:34.134972 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/18e325b5-7837-4776-ae05-d9a12ccf9746-kube-api-access-jpxt6" (OuterVolumeSpecName: "kube-api-access-jpxt6") pod "18e325b5-7837-4776-ae05-d9a12ccf9746" (UID: "18e325b5-7837-4776-ae05-d9a12ccf9746"). InnerVolumeSpecName "kube-api-access-jpxt6". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:27:34 crc kubenswrapper[5052]: I1203 12:27:34.161989 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/18e325b5-7837-4776-ae05-d9a12ccf9746-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "18e325b5-7837-4776-ae05-d9a12ccf9746" (UID: "18e325b5-7837-4776-ae05-d9a12ccf9746"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:27:34 crc kubenswrapper[5052]: I1203 12:27:34.221105 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-sb-0" event={"ID":"5c7fe95d-f637-4021-a5e2-69668046d2ff","Type":"ContainerStarted","Data":"63b5496e0ce2e84297ac272518addd75c23a27efda2a42a79c0e221732c3c3c6"} Dec 03 12:27:34 crc kubenswrapper[5052]: I1203 12:27:34.221182 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-sb-0" event={"ID":"5c7fe95d-f637-4021-a5e2-69668046d2ff","Type":"ContainerStarted","Data":"31511075d0844903f8bc40b464ebcdc51e2d3f8cd1b35d091216e2e8554c72d3"} Dec 03 12:27:34 crc kubenswrapper[5052]: I1203 12:27:34.226765 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-jpxt6\" (UniqueName: \"kubernetes.io/projected/18e325b5-7837-4776-ae05-d9a12ccf9746-kube-api-access-jpxt6\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:34 crc kubenswrapper[5052]: I1203 12:27:34.226815 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/18e325b5-7837-4776-ae05-d9a12ccf9746-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:34 crc kubenswrapper[5052]: I1203 12:27:34.243232 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_ovn-northd-0_18e325b5-7837-4776-ae05-d9a12ccf9746/ovn-northd/0.log" Dec 03 12:27:34 crc kubenswrapper[5052]: I1203 12:27:34.243317 5052 generic.go:334] "Generic (PLEG): container finished" podID="18e325b5-7837-4776-ae05-d9a12ccf9746" containerID="dda098dd0ab939491d6cb42aac03c4daa0a17e93eaa82fa8ed7c6dec3c2a4f1b" exitCode=139 Dec 03 12:27:34 crc kubenswrapper[5052]: I1203 12:27:34.243407 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovn-northd-0" event={"ID":"18e325b5-7837-4776-ae05-d9a12ccf9746","Type":"ContainerDied","Data":"dda098dd0ab939491d6cb42aac03c4daa0a17e93eaa82fa8ed7c6dec3c2a4f1b"} Dec 03 12:27:34 crc kubenswrapper[5052]: I1203 12:27:34.243433 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovn-northd-0" event={"ID":"18e325b5-7837-4776-ae05-d9a12ccf9746","Type":"ContainerDied","Data":"9af37b3aa4be7e7fd78cfa6920ebe9e54e9461b8ea134915213618fabec3639b"} Dec 03 12:27:34 crc kubenswrapper[5052]: I1203 12:27:34.243450 5052 scope.go:117] "RemoveContainer" containerID="cb277aa344aceb42cc1e9fe2afb23fa062591a751722aa2db98541cb69ea85ad" Dec 03 12:27:34 crc kubenswrapper[5052]: I1203 12:27:34.243603 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:27:34 crc kubenswrapper[5052]: I1203 12:27:34.267600 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_neutron-6bf55bd9d6-mrgs9_15cf6575-b6be-4f67-861a-61ebf02d1495/neutron-api/0.log" Dec 03 12:27:34 crc kubenswrapper[5052]: I1203 12:27:34.268919 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-6bf55bd9d6-mrgs9" event={"ID":"15cf6575-b6be-4f67-861a-61ebf02d1495","Type":"ContainerStarted","Data":"608a10c81b5afaba04aa84f9efa1c5579de39b1dc465fbf89ad2188610955b3b"} Dec 03 12:27:34 crc kubenswrapper[5052]: I1203 12:27:34.270033 5052 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Dec 03 12:27:34 crc kubenswrapper[5052]: I1203 12:27:34.270061 5052 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Dec 03 12:27:34 crc kubenswrapper[5052]: I1203 12:27:34.276119 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/neutron-6bf55bd9d6-mrgs9" podUID="15cf6575-b6be-4f67-861a-61ebf02d1495" containerName="neutron-httpd" probeResult="failure" output="HTTP probe failed with statuscode: 503" Dec 03 12:27:34 crc kubenswrapper[5052]: I1203 12:27:34.317057 5052 scope.go:117] "RemoveContainer" containerID="dda098dd0ab939491d6cb42aac03c4daa0a17e93eaa82fa8ed7c6dec3c2a4f1b" Dec 03 12:27:34 crc kubenswrapper[5052]: I1203 12:27:34.368601 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ovn-northd-0"] Dec 03 12:27:34 crc kubenswrapper[5052]: I1203 12:27:34.387743 5052 scope.go:117] "RemoveContainer" containerID="cb277aa344aceb42cc1e9fe2afb23fa062591a751722aa2db98541cb69ea85ad" Dec 03 12:27:34 crc kubenswrapper[5052]: E1203 12:27:34.389487 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cb277aa344aceb42cc1e9fe2afb23fa062591a751722aa2db98541cb69ea85ad\": container with ID starting with cb277aa344aceb42cc1e9fe2afb23fa062591a751722aa2db98541cb69ea85ad not found: ID does not exist" containerID="cb277aa344aceb42cc1e9fe2afb23fa062591a751722aa2db98541cb69ea85ad" Dec 03 12:27:34 crc kubenswrapper[5052]: I1203 12:27:34.389613 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ovn-northd-0"] Dec 03 12:27:34 crc kubenswrapper[5052]: I1203 12:27:34.393670 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cb277aa344aceb42cc1e9fe2afb23fa062591a751722aa2db98541cb69ea85ad"} err="failed to get container status \"cb277aa344aceb42cc1e9fe2afb23fa062591a751722aa2db98541cb69ea85ad\": rpc error: code = NotFound desc = could not find container \"cb277aa344aceb42cc1e9fe2afb23fa062591a751722aa2db98541cb69ea85ad\": container with ID starting with cb277aa344aceb42cc1e9fe2afb23fa062591a751722aa2db98541cb69ea85ad not found: ID does not exist" Dec 03 12:27:34 crc kubenswrapper[5052]: I1203 12:27:34.393717 5052 scope.go:117] "RemoveContainer" containerID="dda098dd0ab939491d6cb42aac03c4daa0a17e93eaa82fa8ed7c6dec3c2a4f1b" Dec 03 12:27:34 crc kubenswrapper[5052]: E1203 12:27:34.399666 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dda098dd0ab939491d6cb42aac03c4daa0a17e93eaa82fa8ed7c6dec3c2a4f1b\": container with ID starting with dda098dd0ab939491d6cb42aac03c4daa0a17e93eaa82fa8ed7c6dec3c2a4f1b not found: ID does not exist" containerID="dda098dd0ab939491d6cb42aac03c4daa0a17e93eaa82fa8ed7c6dec3c2a4f1b" Dec 03 12:27:34 crc kubenswrapper[5052]: I1203 12:27:34.399720 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dda098dd0ab939491d6cb42aac03c4daa0a17e93eaa82fa8ed7c6dec3c2a4f1b"} err="failed to get container status \"dda098dd0ab939491d6cb42aac03c4daa0a17e93eaa82fa8ed7c6dec3c2a4f1b\": rpc error: code = NotFound desc = could not find container \"dda098dd0ab939491d6cb42aac03c4daa0a17e93eaa82fa8ed7c6dec3c2a4f1b\": container with ID starting with dda098dd0ab939491d6cb42aac03c4daa0a17e93eaa82fa8ed7c6dec3c2a4f1b not found: ID does not exist" Dec 03 12:27:34 crc kubenswrapper[5052]: I1203 12:27:34.411653 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ovn-northd-0"] Dec 03 12:27:34 crc kubenswrapper[5052]: E1203 12:27:34.412491 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="18e325b5-7837-4776-ae05-d9a12ccf9746" containerName="openstack-network-exporter" Dec 03 12:27:34 crc kubenswrapper[5052]: I1203 12:27:34.412526 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="18e325b5-7837-4776-ae05-d9a12ccf9746" containerName="openstack-network-exporter" Dec 03 12:27:34 crc kubenswrapper[5052]: E1203 12:27:34.412547 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="18e325b5-7837-4776-ae05-d9a12ccf9746" containerName="ovn-northd" Dec 03 12:27:34 crc kubenswrapper[5052]: I1203 12:27:34.412554 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="18e325b5-7837-4776-ae05-d9a12ccf9746" containerName="ovn-northd" Dec 03 12:27:34 crc kubenswrapper[5052]: I1203 12:27:34.412751 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="18e325b5-7837-4776-ae05-d9a12ccf9746" containerName="ovn-northd" Dec 03 12:27:34 crc kubenswrapper[5052]: I1203 12:27:34.412772 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="18e325b5-7837-4776-ae05-d9a12ccf9746" containerName="openstack-network-exporter" Dec 03 12:27:34 crc kubenswrapper[5052]: I1203 12:27:34.414152 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:27:34 crc kubenswrapper[5052]: I1203 12:27:34.421209 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"ovnnorthd-scripts" Dec 03 12:27:34 crc kubenswrapper[5052]: I1203 12:27:34.421345 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ovnnorthd-ovnnorthd-dockercfg-lp65l" Dec 03 12:27:34 crc kubenswrapper[5052]: I1203 12:27:34.421416 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"ovnnorthd-config" Dec 03 12:27:34 crc kubenswrapper[5052]: I1203 12:27:34.422394 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-ovnnorthd-ovndbs" Dec 03 12:27:34 crc kubenswrapper[5052]: I1203 12:27:34.423387 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ovn-northd-0"] Dec 03 12:27:34 crc kubenswrapper[5052]: I1203 12:27:34.484866 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:27:34 crc kubenswrapper[5052]: I1203 12:27:34.545678 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/a26ed42f-71e7-4c92-8b7e-ad194358d3e5-metrics-certs-tls-certs\") pod \"ovn-northd-0\" (UID: \"a26ed42f-71e7-4c92-8b7e-ad194358d3e5\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:27:34 crc kubenswrapper[5052]: I1203 12:27:34.546238 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/a26ed42f-71e7-4c92-8b7e-ad194358d3e5-scripts\") pod \"ovn-northd-0\" (UID: \"a26ed42f-71e7-4c92-8b7e-ad194358d3e5\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:27:34 crc kubenswrapper[5052]: I1203 12:27:34.546495 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fmjns\" (UniqueName: \"kubernetes.io/projected/a26ed42f-71e7-4c92-8b7e-ad194358d3e5-kube-api-access-fmjns\") pod \"ovn-northd-0\" (UID: \"a26ed42f-71e7-4c92-8b7e-ad194358d3e5\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:27:34 crc kubenswrapper[5052]: I1203 12:27:34.546660 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a26ed42f-71e7-4c92-8b7e-ad194358d3e5-config\") pod \"ovn-northd-0\" (UID: \"a26ed42f-71e7-4c92-8b7e-ad194358d3e5\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:27:34 crc kubenswrapper[5052]: I1203 12:27:34.547073 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a26ed42f-71e7-4c92-8b7e-ad194358d3e5-combined-ca-bundle\") pod \"ovn-northd-0\" (UID: \"a26ed42f-71e7-4c92-8b7e-ad194358d3e5\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:27:34 crc kubenswrapper[5052]: I1203 12:27:34.547250 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-rundir\" (UniqueName: \"kubernetes.io/empty-dir/a26ed42f-71e7-4c92-8b7e-ad194358d3e5-ovn-rundir\") pod \"ovn-northd-0\" (UID: \"a26ed42f-71e7-4c92-8b7e-ad194358d3e5\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:27:34 crc kubenswrapper[5052]: I1203 12:27:34.547436 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-northd-tls-certs\" (UniqueName: \"kubernetes.io/secret/a26ed42f-71e7-4c92-8b7e-ad194358d3e5-ovn-northd-tls-certs\") pod \"ovn-northd-0\" (UID: \"a26ed42f-71e7-4c92-8b7e-ad194358d3e5\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:27:34 crc kubenswrapper[5052]: I1203 12:27:34.649757 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovn-rundir\" (UniqueName: \"kubernetes.io/empty-dir/a26ed42f-71e7-4c92-8b7e-ad194358d3e5-ovn-rundir\") pod \"ovn-northd-0\" (UID: \"a26ed42f-71e7-4c92-8b7e-ad194358d3e5\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:27:34 crc kubenswrapper[5052]: I1203 12:27:34.649863 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovn-northd-tls-certs\" (UniqueName: \"kubernetes.io/secret/a26ed42f-71e7-4c92-8b7e-ad194358d3e5-ovn-northd-tls-certs\") pod \"ovn-northd-0\" (UID: \"a26ed42f-71e7-4c92-8b7e-ad194358d3e5\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:27:34 crc kubenswrapper[5052]: I1203 12:27:34.649901 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/a26ed42f-71e7-4c92-8b7e-ad194358d3e5-metrics-certs-tls-certs\") pod \"ovn-northd-0\" (UID: \"a26ed42f-71e7-4c92-8b7e-ad194358d3e5\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:27:34 crc kubenswrapper[5052]: I1203 12:27:34.649931 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/a26ed42f-71e7-4c92-8b7e-ad194358d3e5-scripts\") pod \"ovn-northd-0\" (UID: \"a26ed42f-71e7-4c92-8b7e-ad194358d3e5\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:27:34 crc kubenswrapper[5052]: I1203 12:27:34.650011 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-fmjns\" (UniqueName: \"kubernetes.io/projected/a26ed42f-71e7-4c92-8b7e-ad194358d3e5-kube-api-access-fmjns\") pod \"ovn-northd-0\" (UID: \"a26ed42f-71e7-4c92-8b7e-ad194358d3e5\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:27:34 crc kubenswrapper[5052]: I1203 12:27:34.650177 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a26ed42f-71e7-4c92-8b7e-ad194358d3e5-config\") pod \"ovn-northd-0\" (UID: \"a26ed42f-71e7-4c92-8b7e-ad194358d3e5\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:27:34 crc kubenswrapper[5052]: I1203 12:27:34.651221 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a26ed42f-71e7-4c92-8b7e-ad194358d3e5-combined-ca-bundle\") pod \"ovn-northd-0\" (UID: \"a26ed42f-71e7-4c92-8b7e-ad194358d3e5\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:27:34 crc kubenswrapper[5052]: I1203 12:27:34.651326 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovn-rundir\" (UniqueName: \"kubernetes.io/empty-dir/a26ed42f-71e7-4c92-8b7e-ad194358d3e5-ovn-rundir\") pod \"ovn-northd-0\" (UID: \"a26ed42f-71e7-4c92-8b7e-ad194358d3e5\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:27:34 crc kubenswrapper[5052]: I1203 12:27:34.651077 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/a26ed42f-71e7-4c92-8b7e-ad194358d3e5-scripts\") pod \"ovn-northd-0\" (UID: \"a26ed42f-71e7-4c92-8b7e-ad194358d3e5\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:27:34 crc kubenswrapper[5052]: I1203 12:27:34.652045 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a26ed42f-71e7-4c92-8b7e-ad194358d3e5-config\") pod \"ovn-northd-0\" (UID: \"a26ed42f-71e7-4c92-8b7e-ad194358d3e5\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:27:34 crc kubenswrapper[5052]: I1203 12:27:34.658560 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/a26ed42f-71e7-4c92-8b7e-ad194358d3e5-metrics-certs-tls-certs\") pod \"ovn-northd-0\" (UID: \"a26ed42f-71e7-4c92-8b7e-ad194358d3e5\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:27:34 crc kubenswrapper[5052]: I1203 12:27:34.658753 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovn-northd-tls-certs\" (UniqueName: \"kubernetes.io/secret/a26ed42f-71e7-4c92-8b7e-ad194358d3e5-ovn-northd-tls-certs\") pod \"ovn-northd-0\" (UID: \"a26ed42f-71e7-4c92-8b7e-ad194358d3e5\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:27:34 crc kubenswrapper[5052]: I1203 12:27:34.662481 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a26ed42f-71e7-4c92-8b7e-ad194358d3e5-combined-ca-bundle\") pod \"ovn-northd-0\" (UID: \"a26ed42f-71e7-4c92-8b7e-ad194358d3e5\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:27:34 crc kubenswrapper[5052]: I1203 12:27:34.670972 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-fmjns\" (UniqueName: \"kubernetes.io/projected/a26ed42f-71e7-4c92-8b7e-ad194358d3e5-kube-api-access-fmjns\") pod \"ovn-northd-0\" (UID: \"a26ed42f-71e7-4c92-8b7e-ad194358d3e5\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:27:34 crc kubenswrapper[5052]: I1203 12:27:34.751093 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:27:35 crc kubenswrapper[5052]: I1203 12:27:35.242321 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:27:35 crc kubenswrapper[5052]: I1203 12:27:35.278517 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:27:35 crc kubenswrapper[5052]: I1203 12:27:35.287872 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-sb-0" event={"ID":"5c7fe95d-f637-4021-a5e2-69668046d2ff","Type":"ContainerStarted","Data":"b01f8b34b21ba5bc171276b7695dd41167a5b350ce509d80a35ce85a684f8937"} Dec 03 12:27:35 crc kubenswrapper[5052]: I1203 12:27:35.289620 5052 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Dec 03 12:27:35 crc kubenswrapper[5052]: W1203 12:27:35.327625 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda26ed42f_71e7_4c92_8b7e_ad194358d3e5.slice/crio-f49b4773f97825cb5a1553388c57cb6ce5ee61bcf03f1f22983c2a6b3594e891 WatchSource:0}: Error finding container f49b4773f97825cb5a1553388c57cb6ce5ee61bcf03f1f22983c2a6b3594e891: Status 404 returned error can't find the container with id f49b4773f97825cb5a1553388c57cb6ce5ee61bcf03f1f22983c2a6b3594e891 Dec 03 12:27:35 crc kubenswrapper[5052]: I1203 12:27:35.329311 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ovn-northd-0"] Dec 03 12:27:35 crc kubenswrapper[5052]: I1203 12:27:35.331635 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/ovsdbserver-sb-0" podStartSLOduration=3.33160744 podStartE2EDuration="3.33160744s" podCreationTimestamp="2025-12-03 12:27:32 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:27:35.327025049 +0000 UTC m=+4045.915266813" watchObservedRunningTime="2025-12-03 12:27:35.33160744 +0000 UTC m=+4045.919849204" Dec 03 12:27:35 crc kubenswrapper[5052]: I1203 12:27:35.558350 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:27:35 crc kubenswrapper[5052]: I1203 12:27:35.688299 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="18e325b5-7837-4776-ae05-d9a12ccf9746" path="/var/lib/kubelet/pods/18e325b5-7837-4776-ae05-d9a12ccf9746/volumes" Dec 03 12:27:35 crc kubenswrapper[5052]: I1203 12:27:35.753348 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:27:35 crc kubenswrapper[5052]: I1203 12:27:35.944303 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:27:35 crc kubenswrapper[5052]: I1203 12:27:35.999318 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/barbican-api-9c7764976-6d6sh" podUID="139928b7-e862-4916-8ca2-37917fa153d7" containerName="barbican-api-log" probeResult="failure" output="Get \"http://10.217.0.12:9311/healthcheck\": read tcp 10.217.0.2:42018->10.217.0.12:9311: read: connection reset by peer" Dec 03 12:27:35 crc kubenswrapper[5052]: I1203 12:27:35.999400 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/barbican-api-9c7764976-6d6sh" podUID="139928b7-e862-4916-8ca2-37917fa153d7" containerName="barbican-api" probeResult="failure" output="Get \"http://10.217.0.12:9311/healthcheck\": read tcp 10.217.0.2:42026->10.217.0.12:9311: read: connection reset by peer" Dec 03 12:27:36 crc kubenswrapper[5052]: I1203 12:27:36.307840 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovn-northd-0" event={"ID":"a26ed42f-71e7-4c92-8b7e-ad194358d3e5","Type":"ContainerStarted","Data":"911da6cb170ba67cad5584869214b5ef5f33b8f7400194bc4bc1ce4ca1a692f4"} Dec 03 12:27:36 crc kubenswrapper[5052]: I1203 12:27:36.308434 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovn-northd-0" event={"ID":"a26ed42f-71e7-4c92-8b7e-ad194358d3e5","Type":"ContainerStarted","Data":"93474533ee16033979340cc7914412b7661d56c4b5e3b0058db42692da5f5166"} Dec 03 12:27:36 crc kubenswrapper[5052]: I1203 12:27:36.308448 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovn-northd-0" event={"ID":"a26ed42f-71e7-4c92-8b7e-ad194358d3e5","Type":"ContainerStarted","Data":"f49b4773f97825cb5a1553388c57cb6ce5ee61bcf03f1f22983c2a6b3594e891"} Dec 03 12:27:36 crc kubenswrapper[5052]: I1203 12:27:36.308470 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:27:36 crc kubenswrapper[5052]: I1203 12:27:36.313830 5052 generic.go:334] "Generic (PLEG): container finished" podID="139928b7-e862-4916-8ca2-37917fa153d7" containerID="133d1aba158f4f11359e5209c49b976cf1345d171a018b7bb2ac3533efc2b0ca" exitCode=0 Dec 03 12:27:36 crc kubenswrapper[5052]: I1203 12:27:36.314093 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-9c7764976-6d6sh" event={"ID":"139928b7-e862-4916-8ca2-37917fa153d7","Type":"ContainerDied","Data":"133d1aba158f4f11359e5209c49b976cf1345d171a018b7bb2ac3533efc2b0ca"} Dec 03 12:27:36 crc kubenswrapper[5052]: I1203 12:27:36.344837 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/ovn-northd-0" podStartSLOduration=2.344806789 podStartE2EDuration="2.344806789s" podCreationTimestamp="2025-12-03 12:27:34 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:27:36.332018474 +0000 UTC m=+4046.920260268" watchObservedRunningTime="2025-12-03 12:27:36.344806789 +0000 UTC m=+4046.933048573" Dec 03 12:27:36 crc kubenswrapper[5052]: I1203 12:27:36.383570 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:27:36 crc kubenswrapper[5052]: I1203 12:27:36.995791 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-api-9c7764976-6d6sh" Dec 03 12:27:37 crc kubenswrapper[5052]: I1203 12:27:37.102442 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/139928b7-e862-4916-8ca2-37917fa153d7-config-data-custom\") pod \"139928b7-e862-4916-8ca2-37917fa153d7\" (UID: \"139928b7-e862-4916-8ca2-37917fa153d7\") " Dec 03 12:27:37 crc kubenswrapper[5052]: I1203 12:27:37.102950 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/139928b7-e862-4916-8ca2-37917fa153d7-combined-ca-bundle\") pod \"139928b7-e862-4916-8ca2-37917fa153d7\" (UID: \"139928b7-e862-4916-8ca2-37917fa153d7\") " Dec 03 12:27:37 crc kubenswrapper[5052]: I1203 12:27:37.103025 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/139928b7-e862-4916-8ca2-37917fa153d7-config-data\") pod \"139928b7-e862-4916-8ca2-37917fa153d7\" (UID: \"139928b7-e862-4916-8ca2-37917fa153d7\") " Dec 03 12:27:37 crc kubenswrapper[5052]: I1203 12:27:37.103056 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/139928b7-e862-4916-8ca2-37917fa153d7-logs\") pod \"139928b7-e862-4916-8ca2-37917fa153d7\" (UID: \"139928b7-e862-4916-8ca2-37917fa153d7\") " Dec 03 12:27:37 crc kubenswrapper[5052]: I1203 12:27:37.103286 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kkvbj\" (UniqueName: \"kubernetes.io/projected/139928b7-e862-4916-8ca2-37917fa153d7-kube-api-access-kkvbj\") pod \"139928b7-e862-4916-8ca2-37917fa153d7\" (UID: \"139928b7-e862-4916-8ca2-37917fa153d7\") " Dec 03 12:27:37 crc kubenswrapper[5052]: I1203 12:27:37.103767 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/139928b7-e862-4916-8ca2-37917fa153d7-logs" (OuterVolumeSpecName: "logs") pod "139928b7-e862-4916-8ca2-37917fa153d7" (UID: "139928b7-e862-4916-8ca2-37917fa153d7"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:27:37 crc kubenswrapper[5052]: I1203 12:27:37.104899 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/139928b7-e862-4916-8ca2-37917fa153d7-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:37 crc kubenswrapper[5052]: I1203 12:27:37.109449 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/139928b7-e862-4916-8ca2-37917fa153d7-kube-api-access-kkvbj" (OuterVolumeSpecName: "kube-api-access-kkvbj") pod "139928b7-e862-4916-8ca2-37917fa153d7" (UID: "139928b7-e862-4916-8ca2-37917fa153d7"). InnerVolumeSpecName "kube-api-access-kkvbj". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:27:37 crc kubenswrapper[5052]: I1203 12:27:37.124666 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/139928b7-e862-4916-8ca2-37917fa153d7-config-data-custom" (OuterVolumeSpecName: "config-data-custom") pod "139928b7-e862-4916-8ca2-37917fa153d7" (UID: "139928b7-e862-4916-8ca2-37917fa153d7"). InnerVolumeSpecName "config-data-custom". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:27:37 crc kubenswrapper[5052]: I1203 12:27:37.138421 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/139928b7-e862-4916-8ca2-37917fa153d7-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "139928b7-e862-4916-8ca2-37917fa153d7" (UID: "139928b7-e862-4916-8ca2-37917fa153d7"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:27:37 crc kubenswrapper[5052]: I1203 12:27:37.173748 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/139928b7-e862-4916-8ca2-37917fa153d7-config-data" (OuterVolumeSpecName: "config-data") pod "139928b7-e862-4916-8ca2-37917fa153d7" (UID: "139928b7-e862-4916-8ca2-37917fa153d7"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:27:37 crc kubenswrapper[5052]: I1203 12:27:37.208059 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-kkvbj\" (UniqueName: \"kubernetes.io/projected/139928b7-e862-4916-8ca2-37917fa153d7-kube-api-access-kkvbj\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:37 crc kubenswrapper[5052]: I1203 12:27:37.208095 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/139928b7-e862-4916-8ca2-37917fa153d7-config-data-custom\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:37 crc kubenswrapper[5052]: I1203 12:27:37.208106 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/139928b7-e862-4916-8ca2-37917fa153d7-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:37 crc kubenswrapper[5052]: I1203 12:27:37.208119 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/139928b7-e862-4916-8ca2-37917fa153d7-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:37 crc kubenswrapper[5052]: I1203 12:27:37.327288 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-9c7764976-6d6sh" event={"ID":"139928b7-e862-4916-8ca2-37917fa153d7","Type":"ContainerDied","Data":"2fe046756bc72fd306fc5a353f9fe97050a69bd1405fbba116fb78717175cb14"} Dec 03 12:27:37 crc kubenswrapper[5052]: I1203 12:27:37.327347 5052 scope.go:117] "RemoveContainer" containerID="133d1aba158f4f11359e5209c49b976cf1345d171a018b7bb2ac3533efc2b0ca" Dec 03 12:27:37 crc kubenswrapper[5052]: I1203 12:27:37.327302 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-api-9c7764976-6d6sh" Dec 03 12:27:37 crc kubenswrapper[5052]: I1203 12:27:37.364674 5052 scope.go:117] "RemoveContainer" containerID="d1675f69e208967311cc8f60ca7ef2dbea31a0db8566c33dc551f51568901da7" Dec 03 12:27:37 crc kubenswrapper[5052]: I1203 12:27:37.392562 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-api-9c7764976-6d6sh"] Dec 03 12:27:37 crc kubenswrapper[5052]: I1203 12:27:37.426142 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/barbican-api-9c7764976-6d6sh"] Dec 03 12:27:37 crc kubenswrapper[5052]: I1203 12:27:37.683887 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="139928b7-e862-4916-8ca2-37917fa153d7" path="/var/lib/kubelet/pods/139928b7-e862-4916-8ca2-37917fa153d7/volumes" Dec 03 12:27:37 crc kubenswrapper[5052]: I1203 12:27:37.945293 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:27:38 crc kubenswrapper[5052]: I1203 12:27:38.991099 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:27:39 crc kubenswrapper[5052]: I1203 12:27:39.225847 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:27:39 crc kubenswrapper[5052]: I1203 12:27:39.227592 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:27:39 crc kubenswrapper[5052]: I1203 12:27:39.229448 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:27:39 crc kubenswrapper[5052]: I1203 12:27:39.230858 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:27:39 crc kubenswrapper[5052]: I1203 12:27:39.349868 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:27:39 crc kubenswrapper[5052]: I1203 12:27:39.355671 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:27:39 crc kubenswrapper[5052]: I1203 12:27:39.412226 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:27:41 crc kubenswrapper[5052]: I1203 12:27:41.190880 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/barbican-api-6d8b99877d-pf4c5"] Dec 03 12:27:41 crc kubenswrapper[5052]: E1203 12:27:41.191667 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="139928b7-e862-4916-8ca2-37917fa153d7" containerName="barbican-api-log" Dec 03 12:27:41 crc kubenswrapper[5052]: I1203 12:27:41.191683 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="139928b7-e862-4916-8ca2-37917fa153d7" containerName="barbican-api-log" Dec 03 12:27:41 crc kubenswrapper[5052]: E1203 12:27:41.191714 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="139928b7-e862-4916-8ca2-37917fa153d7" containerName="barbican-api" Dec 03 12:27:41 crc kubenswrapper[5052]: I1203 12:27:41.191721 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="139928b7-e862-4916-8ca2-37917fa153d7" containerName="barbican-api" Dec 03 12:27:41 crc kubenswrapper[5052]: I1203 12:27:41.191933 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="139928b7-e862-4916-8ca2-37917fa153d7" containerName="barbican-api" Dec 03 12:27:41 crc kubenswrapper[5052]: I1203 12:27:41.191956 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="139928b7-e862-4916-8ca2-37917fa153d7" containerName="barbican-api-log" Dec 03 12:27:41 crc kubenswrapper[5052]: I1203 12:27:41.193213 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-api-6d8b99877d-pf4c5" Dec 03 12:27:41 crc kubenswrapper[5052]: I1203 12:27:41.196061 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-barbican-public-svc" Dec 03 12:27:41 crc kubenswrapper[5052]: I1203 12:27:41.196260 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-barbican-internal-svc" Dec 03 12:27:41 crc kubenswrapper[5052]: I1203 12:27:41.205550 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-api-6d8b99877d-pf4c5"] Dec 03 12:27:41 crc kubenswrapper[5052]: I1203 12:27:41.296341 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/2db0a4b0-75b7-4495-be94-761466b6f3b4-config-data\") pod \"barbican-api-6d8b99877d-pf4c5\" (UID: \"2db0a4b0-75b7-4495-be94-761466b6f3b4\") " pod="openstack-kuttl-tests/barbican-api-6d8b99877d-pf4c5" Dec 03 12:27:41 crc kubenswrapper[5052]: I1203 12:27:41.296386 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/2db0a4b0-75b7-4495-be94-761466b6f3b4-config-data-custom\") pod \"barbican-api-6d8b99877d-pf4c5\" (UID: \"2db0a4b0-75b7-4495-be94-761466b6f3b4\") " pod="openstack-kuttl-tests/barbican-api-6d8b99877d-pf4c5" Dec 03 12:27:41 crc kubenswrapper[5052]: I1203 12:27:41.296411 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-69jk5\" (UniqueName: \"kubernetes.io/projected/2db0a4b0-75b7-4495-be94-761466b6f3b4-kube-api-access-69jk5\") pod \"barbican-api-6d8b99877d-pf4c5\" (UID: \"2db0a4b0-75b7-4495-be94-761466b6f3b4\") " pod="openstack-kuttl-tests/barbican-api-6d8b99877d-pf4c5" Dec 03 12:27:41 crc kubenswrapper[5052]: I1203 12:27:41.296503 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2db0a4b0-75b7-4495-be94-761466b6f3b4-combined-ca-bundle\") pod \"barbican-api-6d8b99877d-pf4c5\" (UID: \"2db0a4b0-75b7-4495-be94-761466b6f3b4\") " pod="openstack-kuttl-tests/barbican-api-6d8b99877d-pf4c5" Dec 03 12:27:41 crc kubenswrapper[5052]: I1203 12:27:41.296741 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/2db0a4b0-75b7-4495-be94-761466b6f3b4-public-tls-certs\") pod \"barbican-api-6d8b99877d-pf4c5\" (UID: \"2db0a4b0-75b7-4495-be94-761466b6f3b4\") " pod="openstack-kuttl-tests/barbican-api-6d8b99877d-pf4c5" Dec 03 12:27:41 crc kubenswrapper[5052]: I1203 12:27:41.296940 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/2db0a4b0-75b7-4495-be94-761466b6f3b4-internal-tls-certs\") pod \"barbican-api-6d8b99877d-pf4c5\" (UID: \"2db0a4b0-75b7-4495-be94-761466b6f3b4\") " pod="openstack-kuttl-tests/barbican-api-6d8b99877d-pf4c5" Dec 03 12:27:41 crc kubenswrapper[5052]: I1203 12:27:41.297130 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/2db0a4b0-75b7-4495-be94-761466b6f3b4-logs\") pod \"barbican-api-6d8b99877d-pf4c5\" (UID: \"2db0a4b0-75b7-4495-be94-761466b6f3b4\") " pod="openstack-kuttl-tests/barbican-api-6d8b99877d-pf4c5" Dec 03 12:27:41 crc kubenswrapper[5052]: I1203 12:27:41.399221 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/2db0a4b0-75b7-4495-be94-761466b6f3b4-public-tls-certs\") pod \"barbican-api-6d8b99877d-pf4c5\" (UID: \"2db0a4b0-75b7-4495-be94-761466b6f3b4\") " pod="openstack-kuttl-tests/barbican-api-6d8b99877d-pf4c5" Dec 03 12:27:41 crc kubenswrapper[5052]: I1203 12:27:41.399396 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/2db0a4b0-75b7-4495-be94-761466b6f3b4-internal-tls-certs\") pod \"barbican-api-6d8b99877d-pf4c5\" (UID: \"2db0a4b0-75b7-4495-be94-761466b6f3b4\") " pod="openstack-kuttl-tests/barbican-api-6d8b99877d-pf4c5" Dec 03 12:27:41 crc kubenswrapper[5052]: I1203 12:27:41.399481 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/2db0a4b0-75b7-4495-be94-761466b6f3b4-logs\") pod \"barbican-api-6d8b99877d-pf4c5\" (UID: \"2db0a4b0-75b7-4495-be94-761466b6f3b4\") " pod="openstack-kuttl-tests/barbican-api-6d8b99877d-pf4c5" Dec 03 12:27:41 crc kubenswrapper[5052]: I1203 12:27:41.399566 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/2db0a4b0-75b7-4495-be94-761466b6f3b4-config-data\") pod \"barbican-api-6d8b99877d-pf4c5\" (UID: \"2db0a4b0-75b7-4495-be94-761466b6f3b4\") " pod="openstack-kuttl-tests/barbican-api-6d8b99877d-pf4c5" Dec 03 12:27:41 crc kubenswrapper[5052]: I1203 12:27:41.399595 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/2db0a4b0-75b7-4495-be94-761466b6f3b4-config-data-custom\") pod \"barbican-api-6d8b99877d-pf4c5\" (UID: \"2db0a4b0-75b7-4495-be94-761466b6f3b4\") " pod="openstack-kuttl-tests/barbican-api-6d8b99877d-pf4c5" Dec 03 12:27:41 crc kubenswrapper[5052]: I1203 12:27:41.399623 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-69jk5\" (UniqueName: \"kubernetes.io/projected/2db0a4b0-75b7-4495-be94-761466b6f3b4-kube-api-access-69jk5\") pod \"barbican-api-6d8b99877d-pf4c5\" (UID: \"2db0a4b0-75b7-4495-be94-761466b6f3b4\") " pod="openstack-kuttl-tests/barbican-api-6d8b99877d-pf4c5" Dec 03 12:27:41 crc kubenswrapper[5052]: I1203 12:27:41.399651 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2db0a4b0-75b7-4495-be94-761466b6f3b4-combined-ca-bundle\") pod \"barbican-api-6d8b99877d-pf4c5\" (UID: \"2db0a4b0-75b7-4495-be94-761466b6f3b4\") " pod="openstack-kuttl-tests/barbican-api-6d8b99877d-pf4c5" Dec 03 12:27:41 crc kubenswrapper[5052]: I1203 12:27:41.403797 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/2db0a4b0-75b7-4495-be94-761466b6f3b4-logs\") pod \"barbican-api-6d8b99877d-pf4c5\" (UID: \"2db0a4b0-75b7-4495-be94-761466b6f3b4\") " pod="openstack-kuttl-tests/barbican-api-6d8b99877d-pf4c5" Dec 03 12:27:41 crc kubenswrapper[5052]: I1203 12:27:41.409712 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2db0a4b0-75b7-4495-be94-761466b6f3b4-combined-ca-bundle\") pod \"barbican-api-6d8b99877d-pf4c5\" (UID: \"2db0a4b0-75b7-4495-be94-761466b6f3b4\") " pod="openstack-kuttl-tests/barbican-api-6d8b99877d-pf4c5" Dec 03 12:27:41 crc kubenswrapper[5052]: I1203 12:27:41.409767 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/2db0a4b0-75b7-4495-be94-761466b6f3b4-config-data-custom\") pod \"barbican-api-6d8b99877d-pf4c5\" (UID: \"2db0a4b0-75b7-4495-be94-761466b6f3b4\") " pod="openstack-kuttl-tests/barbican-api-6d8b99877d-pf4c5" Dec 03 12:27:41 crc kubenswrapper[5052]: I1203 12:27:41.411315 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/2db0a4b0-75b7-4495-be94-761466b6f3b4-config-data\") pod \"barbican-api-6d8b99877d-pf4c5\" (UID: \"2db0a4b0-75b7-4495-be94-761466b6f3b4\") " pod="openstack-kuttl-tests/barbican-api-6d8b99877d-pf4c5" Dec 03 12:27:41 crc kubenswrapper[5052]: I1203 12:27:41.414567 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/2db0a4b0-75b7-4495-be94-761466b6f3b4-internal-tls-certs\") pod \"barbican-api-6d8b99877d-pf4c5\" (UID: \"2db0a4b0-75b7-4495-be94-761466b6f3b4\") " pod="openstack-kuttl-tests/barbican-api-6d8b99877d-pf4c5" Dec 03 12:27:41 crc kubenswrapper[5052]: I1203 12:27:41.416096 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/2db0a4b0-75b7-4495-be94-761466b6f3b4-public-tls-certs\") pod \"barbican-api-6d8b99877d-pf4c5\" (UID: \"2db0a4b0-75b7-4495-be94-761466b6f3b4\") " pod="openstack-kuttl-tests/barbican-api-6d8b99877d-pf4c5" Dec 03 12:27:41 crc kubenswrapper[5052]: I1203 12:27:41.425220 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-69jk5\" (UniqueName: \"kubernetes.io/projected/2db0a4b0-75b7-4495-be94-761466b6f3b4-kube-api-access-69jk5\") pod \"barbican-api-6d8b99877d-pf4c5\" (UID: \"2db0a4b0-75b7-4495-be94-761466b6f3b4\") " pod="openstack-kuttl-tests/barbican-api-6d8b99877d-pf4c5" Dec 03 12:27:41 crc kubenswrapper[5052]: I1203 12:27:41.514688 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-api-6d8b99877d-pf4c5" Dec 03 12:27:41 crc kubenswrapper[5052]: I1203 12:27:41.949086 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:27:41 crc kubenswrapper[5052]: I1203 12:27:41.954424 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:27:41 crc kubenswrapper[5052]: I1203 12:27:41.954595 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:27:41 crc kubenswrapper[5052]: I1203 12:27:41.965853 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-api-6d8b99877d-pf4c5"] Dec 03 12:27:42 crc kubenswrapper[5052]: I1203 12:27:42.121299 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/neutron-544d7dcfcd-7xmk7" Dec 03 12:27:42 crc kubenswrapper[5052]: I1203 12:27:42.379581 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_neutron-6bf55bd9d6-mrgs9_15cf6575-b6be-4f67-861a-61ebf02d1495/neutron-api/1.log" Dec 03 12:27:42 crc kubenswrapper[5052]: I1203 12:27:42.380367 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_neutron-6bf55bd9d6-mrgs9_15cf6575-b6be-4f67-861a-61ebf02d1495/neutron-api/0.log" Dec 03 12:27:42 crc kubenswrapper[5052]: I1203 12:27:42.380405 5052 generic.go:334] "Generic (PLEG): container finished" podID="15cf6575-b6be-4f67-861a-61ebf02d1495" containerID="608a10c81b5afaba04aa84f9efa1c5579de39b1dc465fbf89ad2188610955b3b" exitCode=1 Dec 03 12:27:42 crc kubenswrapper[5052]: I1203 12:27:42.380435 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-6bf55bd9d6-mrgs9" event={"ID":"15cf6575-b6be-4f67-861a-61ebf02d1495","Type":"ContainerDied","Data":"608a10c81b5afaba04aa84f9efa1c5579de39b1dc465fbf89ad2188610955b3b"} Dec 03 12:27:42 crc kubenswrapper[5052]: I1203 12:27:42.380478 5052 scope.go:117] "RemoveContainer" containerID="6b5f5843b9647706bd37b6d16a4de3c4357a058f24ee264c620b57797b2edcdc" Dec 03 12:27:42 crc kubenswrapper[5052]: I1203 12:27:42.381216 5052 scope.go:117] "RemoveContainer" containerID="608a10c81b5afaba04aa84f9efa1c5579de39b1dc465fbf89ad2188610955b3b" Dec 03 12:27:42 crc kubenswrapper[5052]: E1203 12:27:42.381529 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"neutron-api\" with CrashLoopBackOff: \"back-off 10s restarting failed container=neutron-api pod=neutron-6bf55bd9d6-mrgs9_openstack-kuttl-tests(15cf6575-b6be-4f67-861a-61ebf02d1495)\"" pod="openstack-kuttl-tests/neutron-6bf55bd9d6-mrgs9" podUID="15cf6575-b6be-4f67-861a-61ebf02d1495" Dec 03 12:27:42 crc kubenswrapper[5052]: I1203 12:27:42.384149 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/neutron-6bf55bd9d6-mrgs9" podUID="15cf6575-b6be-4f67-861a-61ebf02d1495" containerName="neutron-httpd" probeResult="failure" output="HTTP probe failed with statuscode: 503" Dec 03 12:27:42 crc kubenswrapper[5052]: I1203 12:27:42.387222 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-6d8b99877d-pf4c5" event={"ID":"2db0a4b0-75b7-4495-be94-761466b6f3b4","Type":"ContainerStarted","Data":"a827f835542939adf9f746a906121547f47a7f5c3fb4c3ffc2ca5f8c6a877335"} Dec 03 12:27:42 crc kubenswrapper[5052]: I1203 12:27:42.387324 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-6d8b99877d-pf4c5" event={"ID":"2db0a4b0-75b7-4495-be94-761466b6f3b4","Type":"ContainerStarted","Data":"6f52b869f59ba37ec0a917f4a7b67a1de39d21455470c71dd1d01458c3be2575"} Dec 03 12:27:42 crc kubenswrapper[5052]: I1203 12:27:42.391297 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:27:43 crc kubenswrapper[5052]: I1203 12:27:43.399448 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-6d8b99877d-pf4c5" event={"ID":"2db0a4b0-75b7-4495-be94-761466b6f3b4","Type":"ContainerStarted","Data":"c56d4357a3132e1b0f7677a361c19f24c409f537f898fcb3c7fc4ac9bdcac64c"} Dec 03 12:27:43 crc kubenswrapper[5052]: I1203 12:27:43.399842 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/barbican-api-6d8b99877d-pf4c5" Dec 03 12:27:43 crc kubenswrapper[5052]: I1203 12:27:43.399886 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/barbican-api-6d8b99877d-pf4c5" Dec 03 12:27:43 crc kubenswrapper[5052]: I1203 12:27:43.402842 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_neutron-6bf55bd9d6-mrgs9_15cf6575-b6be-4f67-861a-61ebf02d1495/neutron-api/1.log" Dec 03 12:27:43 crc kubenswrapper[5052]: I1203 12:27:43.430682 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/barbican-api-6d8b99877d-pf4c5" podStartSLOduration=2.430662663 podStartE2EDuration="2.430662663s" podCreationTimestamp="2025-12-03 12:27:41 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:27:43.428952005 +0000 UTC m=+4054.017193769" watchObservedRunningTime="2025-12-03 12:27:43.430662663 +0000 UTC m=+4054.018904427" Dec 03 12:27:44 crc kubenswrapper[5052]: I1203 12:27:44.358412 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:27:44 crc kubenswrapper[5052]: I1203 12:27:44.432344 5052 generic.go:334] "Generic (PLEG): container finished" podID="7f163259-845f-4519-bd77-fa66e760d4db" containerID="b20fcc98d22748a4865b1aff97aab861964ca72f2da839df89a8240768170b98" exitCode=137 Dec 03 12:27:44 crc kubenswrapper[5052]: I1203 12:27:44.433652 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"7f163259-845f-4519-bd77-fa66e760d4db","Type":"ContainerDied","Data":"b20fcc98d22748a4865b1aff97aab861964ca72f2da839df89a8240768170b98"} Dec 03 12:27:44 crc kubenswrapper[5052]: I1203 12:27:44.433731 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"7f163259-845f-4519-bd77-fa66e760d4db","Type":"ContainerDied","Data":"259d9a539fb019903f69a9ddb058a882b6fb34d0e95681ae3f632cb3a65a428a"} Dec 03 12:27:44 crc kubenswrapper[5052]: I1203 12:27:44.433756 5052 scope.go:117] "RemoveContainer" containerID="b20fcc98d22748a4865b1aff97aab861964ca72f2da839df89a8240768170b98" Dec 03 12:27:44 crc kubenswrapper[5052]: I1203 12:27:44.433985 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:27:44 crc kubenswrapper[5052]: I1203 12:27:44.466857 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"lock\" (UniqueName: \"kubernetes.io/empty-dir/7f163259-845f-4519-bd77-fa66e760d4db-lock\") pod \"7f163259-845f-4519-bd77-fa66e760d4db\" (UID: \"7f163259-845f-4519-bd77-fa66e760d4db\") " Dec 03 12:27:44 crc kubenswrapper[5052]: I1203 12:27:44.467108 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"cache\" (UniqueName: \"kubernetes.io/empty-dir/7f163259-845f-4519-bd77-fa66e760d4db-cache\") pod \"7f163259-845f-4519-bd77-fa66e760d4db\" (UID: \"7f163259-845f-4519-bd77-fa66e760d4db\") " Dec 03 12:27:44 crc kubenswrapper[5052]: I1203 12:27:44.467151 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/7f163259-845f-4519-bd77-fa66e760d4db-etc-swift\") pod \"7f163259-845f-4519-bd77-fa66e760d4db\" (UID: \"7f163259-845f-4519-bd77-fa66e760d4db\") " Dec 03 12:27:44 crc kubenswrapper[5052]: I1203 12:27:44.467176 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"swift\" (UniqueName: \"kubernetes.io/local-volume/local-storage06-crc\") pod \"7f163259-845f-4519-bd77-fa66e760d4db\" (UID: \"7f163259-845f-4519-bd77-fa66e760d4db\") " Dec 03 12:27:44 crc kubenswrapper[5052]: I1203 12:27:44.467205 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9vcl8\" (UniqueName: \"kubernetes.io/projected/7f163259-845f-4519-bd77-fa66e760d4db-kube-api-access-9vcl8\") pod \"7f163259-845f-4519-bd77-fa66e760d4db\" (UID: \"7f163259-845f-4519-bd77-fa66e760d4db\") " Dec 03 12:27:44 crc kubenswrapper[5052]: I1203 12:27:44.468111 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f163259-845f-4519-bd77-fa66e760d4db-cache" (OuterVolumeSpecName: "cache") pod "7f163259-845f-4519-bd77-fa66e760d4db" (UID: "7f163259-845f-4519-bd77-fa66e760d4db"). InnerVolumeSpecName "cache". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:27:44 crc kubenswrapper[5052]: I1203 12:27:44.468285 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f163259-845f-4519-bd77-fa66e760d4db-lock" (OuterVolumeSpecName: "lock") pod "7f163259-845f-4519-bd77-fa66e760d4db" (UID: "7f163259-845f-4519-bd77-fa66e760d4db"). InnerVolumeSpecName "lock". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:27:44 crc kubenswrapper[5052]: I1203 12:27:44.476557 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage06-crc" (OuterVolumeSpecName: "swift") pod "7f163259-845f-4519-bd77-fa66e760d4db" (UID: "7f163259-845f-4519-bd77-fa66e760d4db"). InnerVolumeSpecName "local-storage06-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 12:27:44 crc kubenswrapper[5052]: I1203 12:27:44.487892 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7f163259-845f-4519-bd77-fa66e760d4db-etc-swift" (OuterVolumeSpecName: "etc-swift") pod "7f163259-845f-4519-bd77-fa66e760d4db" (UID: "7f163259-845f-4519-bd77-fa66e760d4db"). InnerVolumeSpecName "etc-swift". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:27:44 crc kubenswrapper[5052]: I1203 12:27:44.488063 5052 scope.go:117] "RemoveContainer" containerID="1094f3eb2af0052222cc7befb863b42966285f5596b1be360edb3c5e5e097a14" Dec 03 12:27:44 crc kubenswrapper[5052]: I1203 12:27:44.489941 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7f163259-845f-4519-bd77-fa66e760d4db-kube-api-access-9vcl8" (OuterVolumeSpecName: "kube-api-access-9vcl8") pod "7f163259-845f-4519-bd77-fa66e760d4db" (UID: "7f163259-845f-4519-bd77-fa66e760d4db"). InnerVolumeSpecName "kube-api-access-9vcl8". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:27:44 crc kubenswrapper[5052]: I1203 12:27:44.570133 5052 reconciler_common.go:293] "Volume detached for volume \"lock\" (UniqueName: \"kubernetes.io/empty-dir/7f163259-845f-4519-bd77-fa66e760d4db-lock\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:44 crc kubenswrapper[5052]: I1203 12:27:44.570457 5052 reconciler_common.go:293] "Volume detached for volume \"cache\" (UniqueName: \"kubernetes.io/empty-dir/7f163259-845f-4519-bd77-fa66e760d4db-cache\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:44 crc kubenswrapper[5052]: I1203 12:27:44.570474 5052 reconciler_common.go:293] "Volume detached for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/7f163259-845f-4519-bd77-fa66e760d4db-etc-swift\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:44 crc kubenswrapper[5052]: I1203 12:27:44.570544 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage06-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage06-crc\") on node \"crc\" " Dec 03 12:27:44 crc kubenswrapper[5052]: I1203 12:27:44.570557 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-9vcl8\" (UniqueName: \"kubernetes.io/projected/7f163259-845f-4519-bd77-fa66e760d4db-kube-api-access-9vcl8\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:44 crc kubenswrapper[5052]: I1203 12:27:44.601105 5052 scope.go:117] "RemoveContainer" containerID="1127b9f54fde360052d578fc34e3f1ed816a67a8fd6f335d02508fa144bf6dc7" Dec 03 12:27:44 crc kubenswrapper[5052]: I1203 12:27:44.601630 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage06-crc" (UniqueName: "kubernetes.io/local-volume/local-storage06-crc") on node "crc" Dec 03 12:27:44 crc kubenswrapper[5052]: I1203 12:27:44.636441 5052 scope.go:117] "RemoveContainer" containerID="ba5a838f8abec837caa1abd8f6365324f2f4067f3e37755f33f8a119410d9c81" Dec 03 12:27:44 crc kubenswrapper[5052]: I1203 12:27:44.660707 5052 scope.go:117] "RemoveContainer" containerID="c31f1fc401b9b5f1678ef68c3dacb95ce5a0a4803fb016c6e372e24171421a1a" Dec 03 12:27:44 crc kubenswrapper[5052]: I1203 12:27:44.681095 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage06-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage06-crc\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:44 crc kubenswrapper[5052]: I1203 12:27:44.701728 5052 scope.go:117] "RemoveContainer" containerID="2de4cfddd3456322e2e08527a0b2b4ae7a715efc41cbfbeb5f1a6fa98e9a5409" Dec 03 12:27:44 crc kubenswrapper[5052]: I1203 12:27:44.726793 5052 scope.go:117] "RemoveContainer" containerID="17ba623ea2c89f44d8458d92140c07db44a1f67ae586efa6a1261de7ddb83194" Dec 03 12:27:44 crc kubenswrapper[5052]: I1203 12:27:44.757711 5052 scope.go:117] "RemoveContainer" containerID="dfccd3db9b2af1521bf2032a601fa4662b4414697ee02def0c5cd28e2c96cb26" Dec 03 12:27:44 crc kubenswrapper[5052]: I1203 12:27:44.788952 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/swift-storage-0"] Dec 03 12:27:44 crc kubenswrapper[5052]: I1203 12:27:44.807715 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/swift-storage-0"] Dec 03 12:27:44 crc kubenswrapper[5052]: I1203 12:27:44.807875 5052 scope.go:117] "RemoveContainer" containerID="85d94c44803aac08a74f194f52b8757506853bfa45354a4ed0a41234617514b0" Dec 03 12:27:44 crc kubenswrapper[5052]: I1203 12:27:44.841802 5052 scope.go:117] "RemoveContainer" containerID="846159aaedd764c9a39adccf404c36e1798796b1dc058a4077567aead1622992" Dec 03 12:27:44 crc kubenswrapper[5052]: I1203 12:27:44.856981 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/swift-storage-0"] Dec 03 12:27:44 crc kubenswrapper[5052]: E1203 12:27:44.857625 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="7f163259-845f-4519-bd77-fa66e760d4db" containerName="account-replicator" Dec 03 12:27:44 crc kubenswrapper[5052]: I1203 12:27:44.857647 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f163259-845f-4519-bd77-fa66e760d4db" containerName="account-replicator" Dec 03 12:27:44 crc kubenswrapper[5052]: E1203 12:27:44.857661 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="7f163259-845f-4519-bd77-fa66e760d4db" containerName="container-server" Dec 03 12:27:44 crc kubenswrapper[5052]: I1203 12:27:44.857670 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f163259-845f-4519-bd77-fa66e760d4db" containerName="container-server" Dec 03 12:27:44 crc kubenswrapper[5052]: E1203 12:27:44.857682 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="7f163259-845f-4519-bd77-fa66e760d4db" containerName="object-updater" Dec 03 12:27:44 crc kubenswrapper[5052]: I1203 12:27:44.857689 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f163259-845f-4519-bd77-fa66e760d4db" containerName="object-updater" Dec 03 12:27:44 crc kubenswrapper[5052]: E1203 12:27:44.857705 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="7f163259-845f-4519-bd77-fa66e760d4db" containerName="account-reaper" Dec 03 12:27:44 crc kubenswrapper[5052]: I1203 12:27:44.857710 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f163259-845f-4519-bd77-fa66e760d4db" containerName="account-reaper" Dec 03 12:27:44 crc kubenswrapper[5052]: E1203 12:27:44.857728 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="7f163259-845f-4519-bd77-fa66e760d4db" containerName="account-server" Dec 03 12:27:44 crc kubenswrapper[5052]: I1203 12:27:44.857734 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f163259-845f-4519-bd77-fa66e760d4db" containerName="account-server" Dec 03 12:27:44 crc kubenswrapper[5052]: E1203 12:27:44.857757 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="7f163259-845f-4519-bd77-fa66e760d4db" containerName="container-replicator" Dec 03 12:27:44 crc kubenswrapper[5052]: I1203 12:27:44.857767 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f163259-845f-4519-bd77-fa66e760d4db" containerName="container-replicator" Dec 03 12:27:44 crc kubenswrapper[5052]: E1203 12:27:44.857782 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="7f163259-845f-4519-bd77-fa66e760d4db" containerName="object-server" Dec 03 12:27:44 crc kubenswrapper[5052]: I1203 12:27:44.857789 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f163259-845f-4519-bd77-fa66e760d4db" containerName="object-server" Dec 03 12:27:44 crc kubenswrapper[5052]: E1203 12:27:44.857803 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="7f163259-845f-4519-bd77-fa66e760d4db" containerName="container-updater" Dec 03 12:27:44 crc kubenswrapper[5052]: I1203 12:27:44.857813 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f163259-845f-4519-bd77-fa66e760d4db" containerName="container-updater" Dec 03 12:27:44 crc kubenswrapper[5052]: E1203 12:27:44.857831 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="7f163259-845f-4519-bd77-fa66e760d4db" containerName="object-replicator" Dec 03 12:27:44 crc kubenswrapper[5052]: I1203 12:27:44.857837 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f163259-845f-4519-bd77-fa66e760d4db" containerName="object-replicator" Dec 03 12:27:44 crc kubenswrapper[5052]: E1203 12:27:44.857846 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="7f163259-845f-4519-bd77-fa66e760d4db" containerName="rsync" Dec 03 12:27:44 crc kubenswrapper[5052]: I1203 12:27:44.857857 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f163259-845f-4519-bd77-fa66e760d4db" containerName="rsync" Dec 03 12:27:44 crc kubenswrapper[5052]: E1203 12:27:44.857881 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="7f163259-845f-4519-bd77-fa66e760d4db" containerName="account-auditor" Dec 03 12:27:44 crc kubenswrapper[5052]: I1203 12:27:44.857892 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f163259-845f-4519-bd77-fa66e760d4db" containerName="account-auditor" Dec 03 12:27:44 crc kubenswrapper[5052]: E1203 12:27:44.857904 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="7f163259-845f-4519-bd77-fa66e760d4db" containerName="container-auditor" Dec 03 12:27:44 crc kubenswrapper[5052]: I1203 12:27:44.857914 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f163259-845f-4519-bd77-fa66e760d4db" containerName="container-auditor" Dec 03 12:27:44 crc kubenswrapper[5052]: E1203 12:27:44.857926 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="7f163259-845f-4519-bd77-fa66e760d4db" containerName="swift-recon-cron" Dec 03 12:27:44 crc kubenswrapper[5052]: I1203 12:27:44.857934 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f163259-845f-4519-bd77-fa66e760d4db" containerName="swift-recon-cron" Dec 03 12:27:44 crc kubenswrapper[5052]: E1203 12:27:44.857951 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="7f163259-845f-4519-bd77-fa66e760d4db" containerName="object-auditor" Dec 03 12:27:44 crc kubenswrapper[5052]: I1203 12:27:44.857959 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f163259-845f-4519-bd77-fa66e760d4db" containerName="object-auditor" Dec 03 12:27:44 crc kubenswrapper[5052]: E1203 12:27:44.857982 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="7f163259-845f-4519-bd77-fa66e760d4db" containerName="object-expirer" Dec 03 12:27:44 crc kubenswrapper[5052]: I1203 12:27:44.857988 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f163259-845f-4519-bd77-fa66e760d4db" containerName="object-expirer" Dec 03 12:27:44 crc kubenswrapper[5052]: I1203 12:27:44.864670 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="7f163259-845f-4519-bd77-fa66e760d4db" containerName="container-updater" Dec 03 12:27:44 crc kubenswrapper[5052]: I1203 12:27:44.864727 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="7f163259-845f-4519-bd77-fa66e760d4db" containerName="object-updater" Dec 03 12:27:44 crc kubenswrapper[5052]: I1203 12:27:44.864763 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="7f163259-845f-4519-bd77-fa66e760d4db" containerName="container-server" Dec 03 12:27:44 crc kubenswrapper[5052]: I1203 12:27:44.864785 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="7f163259-845f-4519-bd77-fa66e760d4db" containerName="object-server" Dec 03 12:27:44 crc kubenswrapper[5052]: I1203 12:27:44.864820 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="7f163259-845f-4519-bd77-fa66e760d4db" containerName="object-auditor" Dec 03 12:27:44 crc kubenswrapper[5052]: I1203 12:27:44.864837 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="7f163259-845f-4519-bd77-fa66e760d4db" containerName="account-reaper" Dec 03 12:27:44 crc kubenswrapper[5052]: I1203 12:27:44.864854 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="7f163259-845f-4519-bd77-fa66e760d4db" containerName="object-expirer" Dec 03 12:27:44 crc kubenswrapper[5052]: I1203 12:27:44.864887 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="7f163259-845f-4519-bd77-fa66e760d4db" containerName="swift-recon-cron" Dec 03 12:27:44 crc kubenswrapper[5052]: I1203 12:27:44.864916 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="7f163259-845f-4519-bd77-fa66e760d4db" containerName="object-replicator" Dec 03 12:27:44 crc kubenswrapper[5052]: I1203 12:27:44.864933 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="7f163259-845f-4519-bd77-fa66e760d4db" containerName="account-replicator" Dec 03 12:27:44 crc kubenswrapper[5052]: I1203 12:27:44.864947 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="7f163259-845f-4519-bd77-fa66e760d4db" containerName="account-server" Dec 03 12:27:44 crc kubenswrapper[5052]: I1203 12:27:44.864961 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="7f163259-845f-4519-bd77-fa66e760d4db" containerName="account-auditor" Dec 03 12:27:44 crc kubenswrapper[5052]: I1203 12:27:44.864969 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="7f163259-845f-4519-bd77-fa66e760d4db" containerName="container-auditor" Dec 03 12:27:44 crc kubenswrapper[5052]: I1203 12:27:44.864985 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="7f163259-845f-4519-bd77-fa66e760d4db" containerName="rsync" Dec 03 12:27:44 crc kubenswrapper[5052]: I1203 12:27:44.865002 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="7f163259-845f-4519-bd77-fa66e760d4db" containerName="container-replicator" Dec 03 12:27:44 crc kubenswrapper[5052]: I1203 12:27:44.867818 5052 scope.go:117] "RemoveContainer" containerID="c546226444d76b3bae4019c590f2db31d0aa8aa1810311506bf0679abe0ac7da" Dec 03 12:27:44 crc kubenswrapper[5052]: I1203 12:27:44.875932 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/swift-storage-0"] Dec 03 12:27:44 crc kubenswrapper[5052]: I1203 12:27:44.876104 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:27:44 crc kubenswrapper[5052]: I1203 12:27:44.879701 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"swift-storage-config-data" Dec 03 12:27:44 crc kubenswrapper[5052]: I1203 12:27:44.897807 5052 scope.go:117] "RemoveContainer" containerID="c021f6fefe53bade62d592029616b253503b6690e220f6de008a4d7e297826e5" Dec 03 12:27:44 crc kubenswrapper[5052]: I1203 12:27:44.927376 5052 scope.go:117] "RemoveContainer" containerID="9818644c3d189b7752f2c0e1f85147e5450c62440f231eba257e3c2c1869464e" Dec 03 12:27:44 crc kubenswrapper[5052]: I1203 12:27:44.953114 5052 scope.go:117] "RemoveContainer" containerID="7ce328ddacfb675b5ded2a8547af6ba9940a95baba352f048c9c9dce642c7103" Dec 03 12:27:44 crc kubenswrapper[5052]: I1203 12:27:44.997550 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d5r28\" (UniqueName: \"kubernetes.io/projected/9a71429c-9d59-4863-8949-dcb4f79a35e2-kube-api-access-d5r28\") pod \"swift-storage-0\" (UID: \"9a71429c-9d59-4863-8949-dcb4f79a35e2\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:27:44 crc kubenswrapper[5052]: I1203 12:27:44.997927 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage06-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage06-crc\") pod \"swift-storage-0\" (UID: \"9a71429c-9d59-4863-8949-dcb4f79a35e2\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:27:44 crc kubenswrapper[5052]: I1203 12:27:44.998163 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cache\" (UniqueName: \"kubernetes.io/empty-dir/9a71429c-9d59-4863-8949-dcb4f79a35e2-cache\") pod \"swift-storage-0\" (UID: \"9a71429c-9d59-4863-8949-dcb4f79a35e2\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:27:44 crc kubenswrapper[5052]: I1203 12:27:44.998377 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lock\" (UniqueName: \"kubernetes.io/empty-dir/9a71429c-9d59-4863-8949-dcb4f79a35e2-lock\") pod \"swift-storage-0\" (UID: \"9a71429c-9d59-4863-8949-dcb4f79a35e2\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:27:44 crc kubenswrapper[5052]: I1203 12:27:44.998614 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/9a71429c-9d59-4863-8949-dcb4f79a35e2-etc-swift\") pod \"swift-storage-0\" (UID: \"9a71429c-9d59-4863-8949-dcb4f79a35e2\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:27:45 crc kubenswrapper[5052]: I1203 12:27:45.000262 5052 scope.go:117] "RemoveContainer" containerID="7f52642c73f37ab9180012c54a5c7e03d2db022e02528a07f0120cdace4c30ed" Dec 03 12:27:45 crc kubenswrapper[5052]: I1203 12:27:45.025967 5052 scope.go:117] "RemoveContainer" containerID="b20fcc98d22748a4865b1aff97aab861964ca72f2da839df89a8240768170b98" Dec 03 12:27:45 crc kubenswrapper[5052]: E1203 12:27:45.027008 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b20fcc98d22748a4865b1aff97aab861964ca72f2da839df89a8240768170b98\": container with ID starting with b20fcc98d22748a4865b1aff97aab861964ca72f2da839df89a8240768170b98 not found: ID does not exist" containerID="b20fcc98d22748a4865b1aff97aab861964ca72f2da839df89a8240768170b98" Dec 03 12:27:45 crc kubenswrapper[5052]: I1203 12:27:45.027050 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b20fcc98d22748a4865b1aff97aab861964ca72f2da839df89a8240768170b98"} err="failed to get container status \"b20fcc98d22748a4865b1aff97aab861964ca72f2da839df89a8240768170b98\": rpc error: code = NotFound desc = could not find container \"b20fcc98d22748a4865b1aff97aab861964ca72f2da839df89a8240768170b98\": container with ID starting with b20fcc98d22748a4865b1aff97aab861964ca72f2da839df89a8240768170b98 not found: ID does not exist" Dec 03 12:27:45 crc kubenswrapper[5052]: I1203 12:27:45.027076 5052 scope.go:117] "RemoveContainer" containerID="1094f3eb2af0052222cc7befb863b42966285f5596b1be360edb3c5e5e097a14" Dec 03 12:27:45 crc kubenswrapper[5052]: E1203 12:27:45.027472 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1094f3eb2af0052222cc7befb863b42966285f5596b1be360edb3c5e5e097a14\": container with ID starting with 1094f3eb2af0052222cc7befb863b42966285f5596b1be360edb3c5e5e097a14 not found: ID does not exist" containerID="1094f3eb2af0052222cc7befb863b42966285f5596b1be360edb3c5e5e097a14" Dec 03 12:27:45 crc kubenswrapper[5052]: I1203 12:27:45.027503 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1094f3eb2af0052222cc7befb863b42966285f5596b1be360edb3c5e5e097a14"} err="failed to get container status \"1094f3eb2af0052222cc7befb863b42966285f5596b1be360edb3c5e5e097a14\": rpc error: code = NotFound desc = could not find container \"1094f3eb2af0052222cc7befb863b42966285f5596b1be360edb3c5e5e097a14\": container with ID starting with 1094f3eb2af0052222cc7befb863b42966285f5596b1be360edb3c5e5e097a14 not found: ID does not exist" Dec 03 12:27:45 crc kubenswrapper[5052]: I1203 12:27:45.027553 5052 scope.go:117] "RemoveContainer" containerID="1127b9f54fde360052d578fc34e3f1ed816a67a8fd6f335d02508fa144bf6dc7" Dec 03 12:27:45 crc kubenswrapper[5052]: E1203 12:27:45.027939 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1127b9f54fde360052d578fc34e3f1ed816a67a8fd6f335d02508fa144bf6dc7\": container with ID starting with 1127b9f54fde360052d578fc34e3f1ed816a67a8fd6f335d02508fa144bf6dc7 not found: ID does not exist" containerID="1127b9f54fde360052d578fc34e3f1ed816a67a8fd6f335d02508fa144bf6dc7" Dec 03 12:27:45 crc kubenswrapper[5052]: I1203 12:27:45.028047 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1127b9f54fde360052d578fc34e3f1ed816a67a8fd6f335d02508fa144bf6dc7"} err="failed to get container status \"1127b9f54fde360052d578fc34e3f1ed816a67a8fd6f335d02508fa144bf6dc7\": rpc error: code = NotFound desc = could not find container \"1127b9f54fde360052d578fc34e3f1ed816a67a8fd6f335d02508fa144bf6dc7\": container with ID starting with 1127b9f54fde360052d578fc34e3f1ed816a67a8fd6f335d02508fa144bf6dc7 not found: ID does not exist" Dec 03 12:27:45 crc kubenswrapper[5052]: I1203 12:27:45.028181 5052 scope.go:117] "RemoveContainer" containerID="ba5a838f8abec837caa1abd8f6365324f2f4067f3e37755f33f8a119410d9c81" Dec 03 12:27:45 crc kubenswrapper[5052]: E1203 12:27:45.028587 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ba5a838f8abec837caa1abd8f6365324f2f4067f3e37755f33f8a119410d9c81\": container with ID starting with ba5a838f8abec837caa1abd8f6365324f2f4067f3e37755f33f8a119410d9c81 not found: ID does not exist" containerID="ba5a838f8abec837caa1abd8f6365324f2f4067f3e37755f33f8a119410d9c81" Dec 03 12:27:45 crc kubenswrapper[5052]: I1203 12:27:45.028624 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ba5a838f8abec837caa1abd8f6365324f2f4067f3e37755f33f8a119410d9c81"} err="failed to get container status \"ba5a838f8abec837caa1abd8f6365324f2f4067f3e37755f33f8a119410d9c81\": rpc error: code = NotFound desc = could not find container \"ba5a838f8abec837caa1abd8f6365324f2f4067f3e37755f33f8a119410d9c81\": container with ID starting with ba5a838f8abec837caa1abd8f6365324f2f4067f3e37755f33f8a119410d9c81 not found: ID does not exist" Dec 03 12:27:45 crc kubenswrapper[5052]: I1203 12:27:45.028650 5052 scope.go:117] "RemoveContainer" containerID="c31f1fc401b9b5f1678ef68c3dacb95ce5a0a4803fb016c6e372e24171421a1a" Dec 03 12:27:45 crc kubenswrapper[5052]: E1203 12:27:45.028955 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c31f1fc401b9b5f1678ef68c3dacb95ce5a0a4803fb016c6e372e24171421a1a\": container with ID starting with c31f1fc401b9b5f1678ef68c3dacb95ce5a0a4803fb016c6e372e24171421a1a not found: ID does not exist" containerID="c31f1fc401b9b5f1678ef68c3dacb95ce5a0a4803fb016c6e372e24171421a1a" Dec 03 12:27:45 crc kubenswrapper[5052]: I1203 12:27:45.028976 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c31f1fc401b9b5f1678ef68c3dacb95ce5a0a4803fb016c6e372e24171421a1a"} err="failed to get container status \"c31f1fc401b9b5f1678ef68c3dacb95ce5a0a4803fb016c6e372e24171421a1a\": rpc error: code = NotFound desc = could not find container \"c31f1fc401b9b5f1678ef68c3dacb95ce5a0a4803fb016c6e372e24171421a1a\": container with ID starting with c31f1fc401b9b5f1678ef68c3dacb95ce5a0a4803fb016c6e372e24171421a1a not found: ID does not exist" Dec 03 12:27:45 crc kubenswrapper[5052]: I1203 12:27:45.028988 5052 scope.go:117] "RemoveContainer" containerID="2de4cfddd3456322e2e08527a0b2b4ae7a715efc41cbfbeb5f1a6fa98e9a5409" Dec 03 12:27:45 crc kubenswrapper[5052]: E1203 12:27:45.029292 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2de4cfddd3456322e2e08527a0b2b4ae7a715efc41cbfbeb5f1a6fa98e9a5409\": container with ID starting with 2de4cfddd3456322e2e08527a0b2b4ae7a715efc41cbfbeb5f1a6fa98e9a5409 not found: ID does not exist" containerID="2de4cfddd3456322e2e08527a0b2b4ae7a715efc41cbfbeb5f1a6fa98e9a5409" Dec 03 12:27:45 crc kubenswrapper[5052]: I1203 12:27:45.029396 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2de4cfddd3456322e2e08527a0b2b4ae7a715efc41cbfbeb5f1a6fa98e9a5409"} err="failed to get container status \"2de4cfddd3456322e2e08527a0b2b4ae7a715efc41cbfbeb5f1a6fa98e9a5409\": rpc error: code = NotFound desc = could not find container \"2de4cfddd3456322e2e08527a0b2b4ae7a715efc41cbfbeb5f1a6fa98e9a5409\": container with ID starting with 2de4cfddd3456322e2e08527a0b2b4ae7a715efc41cbfbeb5f1a6fa98e9a5409 not found: ID does not exist" Dec 03 12:27:45 crc kubenswrapper[5052]: I1203 12:27:45.029870 5052 scope.go:117] "RemoveContainer" containerID="17ba623ea2c89f44d8458d92140c07db44a1f67ae586efa6a1261de7ddb83194" Dec 03 12:27:45 crc kubenswrapper[5052]: E1203 12:27:45.030209 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"17ba623ea2c89f44d8458d92140c07db44a1f67ae586efa6a1261de7ddb83194\": container with ID starting with 17ba623ea2c89f44d8458d92140c07db44a1f67ae586efa6a1261de7ddb83194 not found: ID does not exist" containerID="17ba623ea2c89f44d8458d92140c07db44a1f67ae586efa6a1261de7ddb83194" Dec 03 12:27:45 crc kubenswrapper[5052]: I1203 12:27:45.030232 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"17ba623ea2c89f44d8458d92140c07db44a1f67ae586efa6a1261de7ddb83194"} err="failed to get container status \"17ba623ea2c89f44d8458d92140c07db44a1f67ae586efa6a1261de7ddb83194\": rpc error: code = NotFound desc = could not find container \"17ba623ea2c89f44d8458d92140c07db44a1f67ae586efa6a1261de7ddb83194\": container with ID starting with 17ba623ea2c89f44d8458d92140c07db44a1f67ae586efa6a1261de7ddb83194 not found: ID does not exist" Dec 03 12:27:45 crc kubenswrapper[5052]: I1203 12:27:45.030246 5052 scope.go:117] "RemoveContainer" containerID="dfccd3db9b2af1521bf2032a601fa4662b4414697ee02def0c5cd28e2c96cb26" Dec 03 12:27:45 crc kubenswrapper[5052]: E1203 12:27:45.030475 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dfccd3db9b2af1521bf2032a601fa4662b4414697ee02def0c5cd28e2c96cb26\": container with ID starting with dfccd3db9b2af1521bf2032a601fa4662b4414697ee02def0c5cd28e2c96cb26 not found: ID does not exist" containerID="dfccd3db9b2af1521bf2032a601fa4662b4414697ee02def0c5cd28e2c96cb26" Dec 03 12:27:45 crc kubenswrapper[5052]: I1203 12:27:45.030594 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dfccd3db9b2af1521bf2032a601fa4662b4414697ee02def0c5cd28e2c96cb26"} err="failed to get container status \"dfccd3db9b2af1521bf2032a601fa4662b4414697ee02def0c5cd28e2c96cb26\": rpc error: code = NotFound desc = could not find container \"dfccd3db9b2af1521bf2032a601fa4662b4414697ee02def0c5cd28e2c96cb26\": container with ID starting with dfccd3db9b2af1521bf2032a601fa4662b4414697ee02def0c5cd28e2c96cb26 not found: ID does not exist" Dec 03 12:27:45 crc kubenswrapper[5052]: I1203 12:27:45.030703 5052 scope.go:117] "RemoveContainer" containerID="85d94c44803aac08a74f194f52b8757506853bfa45354a4ed0a41234617514b0" Dec 03 12:27:45 crc kubenswrapper[5052]: E1203 12:27:45.031063 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"85d94c44803aac08a74f194f52b8757506853bfa45354a4ed0a41234617514b0\": container with ID starting with 85d94c44803aac08a74f194f52b8757506853bfa45354a4ed0a41234617514b0 not found: ID does not exist" containerID="85d94c44803aac08a74f194f52b8757506853bfa45354a4ed0a41234617514b0" Dec 03 12:27:45 crc kubenswrapper[5052]: I1203 12:27:45.031176 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"85d94c44803aac08a74f194f52b8757506853bfa45354a4ed0a41234617514b0"} err="failed to get container status \"85d94c44803aac08a74f194f52b8757506853bfa45354a4ed0a41234617514b0\": rpc error: code = NotFound desc = could not find container \"85d94c44803aac08a74f194f52b8757506853bfa45354a4ed0a41234617514b0\": container with ID starting with 85d94c44803aac08a74f194f52b8757506853bfa45354a4ed0a41234617514b0 not found: ID does not exist" Dec 03 12:27:45 crc kubenswrapper[5052]: I1203 12:27:45.031271 5052 scope.go:117] "RemoveContainer" containerID="846159aaedd764c9a39adccf404c36e1798796b1dc058a4077567aead1622992" Dec 03 12:27:45 crc kubenswrapper[5052]: E1203 12:27:45.031591 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"846159aaedd764c9a39adccf404c36e1798796b1dc058a4077567aead1622992\": container with ID starting with 846159aaedd764c9a39adccf404c36e1798796b1dc058a4077567aead1622992 not found: ID does not exist" containerID="846159aaedd764c9a39adccf404c36e1798796b1dc058a4077567aead1622992" Dec 03 12:27:45 crc kubenswrapper[5052]: I1203 12:27:45.031621 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"846159aaedd764c9a39adccf404c36e1798796b1dc058a4077567aead1622992"} err="failed to get container status \"846159aaedd764c9a39adccf404c36e1798796b1dc058a4077567aead1622992\": rpc error: code = NotFound desc = could not find container \"846159aaedd764c9a39adccf404c36e1798796b1dc058a4077567aead1622992\": container with ID starting with 846159aaedd764c9a39adccf404c36e1798796b1dc058a4077567aead1622992 not found: ID does not exist" Dec 03 12:27:45 crc kubenswrapper[5052]: I1203 12:27:45.031642 5052 scope.go:117] "RemoveContainer" containerID="c546226444d76b3bae4019c590f2db31d0aa8aa1810311506bf0679abe0ac7da" Dec 03 12:27:45 crc kubenswrapper[5052]: E1203 12:27:45.031982 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c546226444d76b3bae4019c590f2db31d0aa8aa1810311506bf0679abe0ac7da\": container with ID starting with c546226444d76b3bae4019c590f2db31d0aa8aa1810311506bf0679abe0ac7da not found: ID does not exist" containerID="c546226444d76b3bae4019c590f2db31d0aa8aa1810311506bf0679abe0ac7da" Dec 03 12:27:45 crc kubenswrapper[5052]: I1203 12:27:45.032094 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c546226444d76b3bae4019c590f2db31d0aa8aa1810311506bf0679abe0ac7da"} err="failed to get container status \"c546226444d76b3bae4019c590f2db31d0aa8aa1810311506bf0679abe0ac7da\": rpc error: code = NotFound desc = could not find container \"c546226444d76b3bae4019c590f2db31d0aa8aa1810311506bf0679abe0ac7da\": container with ID starting with c546226444d76b3bae4019c590f2db31d0aa8aa1810311506bf0679abe0ac7da not found: ID does not exist" Dec 03 12:27:45 crc kubenswrapper[5052]: I1203 12:27:45.032214 5052 scope.go:117] "RemoveContainer" containerID="c021f6fefe53bade62d592029616b253503b6690e220f6de008a4d7e297826e5" Dec 03 12:27:45 crc kubenswrapper[5052]: E1203 12:27:45.032595 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c021f6fefe53bade62d592029616b253503b6690e220f6de008a4d7e297826e5\": container with ID starting with c021f6fefe53bade62d592029616b253503b6690e220f6de008a4d7e297826e5 not found: ID does not exist" containerID="c021f6fefe53bade62d592029616b253503b6690e220f6de008a4d7e297826e5" Dec 03 12:27:45 crc kubenswrapper[5052]: I1203 12:27:45.032623 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c021f6fefe53bade62d592029616b253503b6690e220f6de008a4d7e297826e5"} err="failed to get container status \"c021f6fefe53bade62d592029616b253503b6690e220f6de008a4d7e297826e5\": rpc error: code = NotFound desc = could not find container \"c021f6fefe53bade62d592029616b253503b6690e220f6de008a4d7e297826e5\": container with ID starting with c021f6fefe53bade62d592029616b253503b6690e220f6de008a4d7e297826e5 not found: ID does not exist" Dec 03 12:27:45 crc kubenswrapper[5052]: I1203 12:27:45.032639 5052 scope.go:117] "RemoveContainer" containerID="9818644c3d189b7752f2c0e1f85147e5450c62440f231eba257e3c2c1869464e" Dec 03 12:27:45 crc kubenswrapper[5052]: E1203 12:27:45.032916 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9818644c3d189b7752f2c0e1f85147e5450c62440f231eba257e3c2c1869464e\": container with ID starting with 9818644c3d189b7752f2c0e1f85147e5450c62440f231eba257e3c2c1869464e not found: ID does not exist" containerID="9818644c3d189b7752f2c0e1f85147e5450c62440f231eba257e3c2c1869464e" Dec 03 12:27:45 crc kubenswrapper[5052]: I1203 12:27:45.032946 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9818644c3d189b7752f2c0e1f85147e5450c62440f231eba257e3c2c1869464e"} err="failed to get container status \"9818644c3d189b7752f2c0e1f85147e5450c62440f231eba257e3c2c1869464e\": rpc error: code = NotFound desc = could not find container \"9818644c3d189b7752f2c0e1f85147e5450c62440f231eba257e3c2c1869464e\": container with ID starting with 9818644c3d189b7752f2c0e1f85147e5450c62440f231eba257e3c2c1869464e not found: ID does not exist" Dec 03 12:27:45 crc kubenswrapper[5052]: I1203 12:27:45.032967 5052 scope.go:117] "RemoveContainer" containerID="7ce328ddacfb675b5ded2a8547af6ba9940a95baba352f048c9c9dce642c7103" Dec 03 12:27:45 crc kubenswrapper[5052]: E1203 12:27:45.033228 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7ce328ddacfb675b5ded2a8547af6ba9940a95baba352f048c9c9dce642c7103\": container with ID starting with 7ce328ddacfb675b5ded2a8547af6ba9940a95baba352f048c9c9dce642c7103 not found: ID does not exist" containerID="7ce328ddacfb675b5ded2a8547af6ba9940a95baba352f048c9c9dce642c7103" Dec 03 12:27:45 crc kubenswrapper[5052]: I1203 12:27:45.033253 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7ce328ddacfb675b5ded2a8547af6ba9940a95baba352f048c9c9dce642c7103"} err="failed to get container status \"7ce328ddacfb675b5ded2a8547af6ba9940a95baba352f048c9c9dce642c7103\": rpc error: code = NotFound desc = could not find container \"7ce328ddacfb675b5ded2a8547af6ba9940a95baba352f048c9c9dce642c7103\": container with ID starting with 7ce328ddacfb675b5ded2a8547af6ba9940a95baba352f048c9c9dce642c7103 not found: ID does not exist" Dec 03 12:27:45 crc kubenswrapper[5052]: I1203 12:27:45.033266 5052 scope.go:117] "RemoveContainer" containerID="7f52642c73f37ab9180012c54a5c7e03d2db022e02528a07f0120cdace4c30ed" Dec 03 12:27:45 crc kubenswrapper[5052]: E1203 12:27:45.033501 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7f52642c73f37ab9180012c54a5c7e03d2db022e02528a07f0120cdace4c30ed\": container with ID starting with 7f52642c73f37ab9180012c54a5c7e03d2db022e02528a07f0120cdace4c30ed not found: ID does not exist" containerID="7f52642c73f37ab9180012c54a5c7e03d2db022e02528a07f0120cdace4c30ed" Dec 03 12:27:45 crc kubenswrapper[5052]: I1203 12:27:45.033548 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7f52642c73f37ab9180012c54a5c7e03d2db022e02528a07f0120cdace4c30ed"} err="failed to get container status \"7f52642c73f37ab9180012c54a5c7e03d2db022e02528a07f0120cdace4c30ed\": rpc error: code = NotFound desc = could not find container \"7f52642c73f37ab9180012c54a5c7e03d2db022e02528a07f0120cdace4c30ed\": container with ID starting with 7f52642c73f37ab9180012c54a5c7e03d2db022e02528a07f0120cdace4c30ed not found: ID does not exist" Dec 03 12:27:45 crc kubenswrapper[5052]: I1203 12:27:45.101060 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-d5r28\" (UniqueName: \"kubernetes.io/projected/9a71429c-9d59-4863-8949-dcb4f79a35e2-kube-api-access-d5r28\") pod \"swift-storage-0\" (UID: \"9a71429c-9d59-4863-8949-dcb4f79a35e2\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:27:45 crc kubenswrapper[5052]: I1203 12:27:45.101175 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage06-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage06-crc\") pod \"swift-storage-0\" (UID: \"9a71429c-9d59-4863-8949-dcb4f79a35e2\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:27:45 crc kubenswrapper[5052]: I1203 12:27:45.101224 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"cache\" (UniqueName: \"kubernetes.io/empty-dir/9a71429c-9d59-4863-8949-dcb4f79a35e2-cache\") pod \"swift-storage-0\" (UID: \"9a71429c-9d59-4863-8949-dcb4f79a35e2\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:27:45 crc kubenswrapper[5052]: I1203 12:27:45.101316 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"lock\" (UniqueName: \"kubernetes.io/empty-dir/9a71429c-9d59-4863-8949-dcb4f79a35e2-lock\") pod \"swift-storage-0\" (UID: \"9a71429c-9d59-4863-8949-dcb4f79a35e2\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:27:45 crc kubenswrapper[5052]: I1203 12:27:45.101339 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/9a71429c-9d59-4863-8949-dcb4f79a35e2-etc-swift\") pod \"swift-storage-0\" (UID: \"9a71429c-9d59-4863-8949-dcb4f79a35e2\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:27:45 crc kubenswrapper[5052]: I1203 12:27:45.102370 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage06-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage06-crc\") pod \"swift-storage-0\" (UID: \"9a71429c-9d59-4863-8949-dcb4f79a35e2\") device mount path \"/mnt/openstack/pv06\"" pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:27:45 crc kubenswrapper[5052]: I1203 12:27:45.105015 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"lock\" (UniqueName: \"kubernetes.io/empty-dir/9a71429c-9d59-4863-8949-dcb4f79a35e2-lock\") pod \"swift-storage-0\" (UID: \"9a71429c-9d59-4863-8949-dcb4f79a35e2\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:27:45 crc kubenswrapper[5052]: I1203 12:27:45.105297 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"cache\" (UniqueName: \"kubernetes.io/empty-dir/9a71429c-9d59-4863-8949-dcb4f79a35e2-cache\") pod \"swift-storage-0\" (UID: \"9a71429c-9d59-4863-8949-dcb4f79a35e2\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:27:45 crc kubenswrapper[5052]: I1203 12:27:45.121223 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-d5r28\" (UniqueName: \"kubernetes.io/projected/9a71429c-9d59-4863-8949-dcb4f79a35e2-kube-api-access-d5r28\") pod \"swift-storage-0\" (UID: \"9a71429c-9d59-4863-8949-dcb4f79a35e2\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:27:45 crc kubenswrapper[5052]: I1203 12:27:45.122240 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/9a71429c-9d59-4863-8949-dcb4f79a35e2-etc-swift\") pod \"swift-storage-0\" (UID: \"9a71429c-9d59-4863-8949-dcb4f79a35e2\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:27:45 crc kubenswrapper[5052]: I1203 12:27:45.130424 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage06-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage06-crc\") pod \"swift-storage-0\" (UID: \"9a71429c-9d59-4863-8949-dcb4f79a35e2\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:27:45 crc kubenswrapper[5052]: I1203 12:27:45.194720 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:27:45 crc kubenswrapper[5052]: I1203 12:27:45.686482 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7f163259-845f-4519-bd77-fa66e760d4db" path="/var/lib/kubelet/pods/7f163259-845f-4519-bd77-fa66e760d4db/volumes" Dec 03 12:27:45 crc kubenswrapper[5052]: I1203 12:27:45.782329 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/swift-storage-0"] Dec 03 12:27:46 crc kubenswrapper[5052]: I1203 12:27:46.116763 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/keystone-64d6d88f66-7z998" Dec 03 12:27:46 crc kubenswrapper[5052]: I1203 12:27:46.219876 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/keystone-7995cfc4b8-v5sr4"] Dec 03 12:27:46 crc kubenswrapper[5052]: I1203 12:27:46.220103 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/keystone-7995cfc4b8-v5sr4" podUID="c2463445-211e-4724-ba2c-4cc1661a6452" containerName="keystone-api" containerID="cri-o://314b48c25f8f96b0a4711b6bf034891f8f2066552bba485c2589fd24ede9c77d" gracePeriod=30 Dec 03 12:27:46 crc kubenswrapper[5052]: I1203 12:27:46.463228 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"9a71429c-9d59-4863-8949-dcb4f79a35e2","Type":"ContainerStarted","Data":"1637629a53175a4befbb5dad0d15dd5909adc30a1253d4051e99d888b51f7ccb"} Dec 03 12:27:46 crc kubenswrapper[5052]: I1203 12:27:46.463272 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"9a71429c-9d59-4863-8949-dcb4f79a35e2","Type":"ContainerStarted","Data":"30effbbdf9935462e10a9a1d9efeda543d42e320e8af45e4e06802d2c0e922c2"} Dec 03 12:27:46 crc kubenswrapper[5052]: I1203 12:27:46.465169 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_neutron-544d7dcfcd-7xmk7_135a97dd-af49-4eb8-8224-c3090757ed2a/neutron-httpd/0.log" Dec 03 12:27:46 crc kubenswrapper[5052]: I1203 12:27:46.465691 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_neutron-544d7dcfcd-7xmk7_135a97dd-af49-4eb8-8224-c3090757ed2a/neutron-api/0.log" Dec 03 12:27:46 crc kubenswrapper[5052]: I1203 12:27:46.465732 5052 generic.go:334] "Generic (PLEG): container finished" podID="135a97dd-af49-4eb8-8224-c3090757ed2a" containerID="1ee73b25fb176fb0f7a5ba787284992b6e0c4d72b574d2a31c1df2ab8746440f" exitCode=137 Dec 03 12:27:46 crc kubenswrapper[5052]: I1203 12:27:46.465746 5052 generic.go:334] "Generic (PLEG): container finished" podID="135a97dd-af49-4eb8-8224-c3090757ed2a" containerID="47023cf3d975406d17e04a8c3b72230adbcd30f7306a30b2f6867cc8b0f83bdc" exitCode=137 Dec 03 12:27:46 crc kubenswrapper[5052]: I1203 12:27:46.465763 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-544d7dcfcd-7xmk7" event={"ID":"135a97dd-af49-4eb8-8224-c3090757ed2a","Type":"ContainerDied","Data":"1ee73b25fb176fb0f7a5ba787284992b6e0c4d72b574d2a31c1df2ab8746440f"} Dec 03 12:27:46 crc kubenswrapper[5052]: I1203 12:27:46.465784 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-544d7dcfcd-7xmk7" event={"ID":"135a97dd-af49-4eb8-8224-c3090757ed2a","Type":"ContainerDied","Data":"47023cf3d975406d17e04a8c3b72230adbcd30f7306a30b2f6867cc8b0f83bdc"} Dec 03 12:27:46 crc kubenswrapper[5052]: I1203 12:27:46.717667 5052 pod_container_manager_linux.go:210] "Failed to delete cgroup paths" cgroupName=["kubepods","besteffort","pod65fc3838-f25d-4eda-9643-6b63215c41ff"] err="unable to destroy cgroup paths for cgroup [kubepods besteffort pod65fc3838-f25d-4eda-9643-6b63215c41ff] : Timed out while waiting for systemd to remove kubepods-besteffort-pod65fc3838_f25d_4eda_9643_6b63215c41ff.slice" Dec 03 12:27:46 crc kubenswrapper[5052]: E1203 12:27:46.717736 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to delete cgroup paths for [kubepods besteffort pod65fc3838-f25d-4eda-9643-6b63215c41ff] : unable to destroy cgroup paths for cgroup [kubepods besteffort pod65fc3838-f25d-4eda-9643-6b63215c41ff] : Timed out while waiting for systemd to remove kubepods-besteffort-pod65fc3838_f25d_4eda_9643_6b63215c41ff.slice" pod="openstack-kuttl-tests/memcached-0" podUID="65fc3838-f25d-4eda-9643-6b63215c41ff" Dec 03 12:27:46 crc kubenswrapper[5052]: I1203 12:27:46.880460 5052 kubelet.go:2542] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openstack-kuttl-tests/neutron-6bf55bd9d6-mrgs9" Dec 03 12:27:46 crc kubenswrapper[5052]: I1203 12:27:46.882585 5052 scope.go:117] "RemoveContainer" containerID="608a10c81b5afaba04aa84f9efa1c5579de39b1dc465fbf89ad2188610955b3b" Dec 03 12:27:46 crc kubenswrapper[5052]: E1203 12:27:46.882842 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"neutron-api\" with CrashLoopBackOff: \"back-off 10s restarting failed container=neutron-api pod=neutron-6bf55bd9d6-mrgs9_openstack-kuttl-tests(15cf6575-b6be-4f67-861a-61ebf02d1495)\"" pod="openstack-kuttl-tests/neutron-6bf55bd9d6-mrgs9" podUID="15cf6575-b6be-4f67-861a-61ebf02d1495" Dec 03 12:27:46 crc kubenswrapper[5052]: I1203 12:27:46.889052 5052 prober.go:107] "Probe failed" probeType="Liveness" pod="openstack-kuttl-tests/neutron-6bf55bd9d6-mrgs9" podUID="15cf6575-b6be-4f67-861a-61ebf02d1495" containerName="neutron-httpd" probeResult="failure" output="HTTP probe failed with statuscode: 503" Dec 03 12:27:46 crc kubenswrapper[5052]: I1203 12:27:46.889324 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/neutron-6bf55bd9d6-mrgs9" podUID="15cf6575-b6be-4f67-861a-61ebf02d1495" containerName="neutron-httpd" probeResult="failure" output="HTTP probe failed with statuscode: 503" Dec 03 12:27:46 crc kubenswrapper[5052]: I1203 12:27:46.901359 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/neutron-6bf55bd9d6-mrgs9" podUID="15cf6575-b6be-4f67-861a-61ebf02d1495" containerName="neutron-httpd" probeResult="failure" output="HTTP probe failed with statuscode: 503" Dec 03 12:27:46 crc kubenswrapper[5052]: I1203 12:27:46.963735 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_neutron-544d7dcfcd-7xmk7_135a97dd-af49-4eb8-8224-c3090757ed2a/neutron-httpd/0.log" Dec 03 12:27:46 crc kubenswrapper[5052]: I1203 12:27:46.964212 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_neutron-544d7dcfcd-7xmk7_135a97dd-af49-4eb8-8224-c3090757ed2a/neutron-api/0.log" Dec 03 12:27:46 crc kubenswrapper[5052]: I1203 12:27:46.964273 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-544d7dcfcd-7xmk7" Dec 03 12:27:47 crc kubenswrapper[5052]: I1203 12:27:47.069002 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jdmsc\" (UniqueName: \"kubernetes.io/projected/135a97dd-af49-4eb8-8224-c3090757ed2a-kube-api-access-jdmsc\") pod \"135a97dd-af49-4eb8-8224-c3090757ed2a\" (UID: \"135a97dd-af49-4eb8-8224-c3090757ed2a\") " Dec 03 12:27:47 crc kubenswrapper[5052]: I1203 12:27:47.069051 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/135a97dd-af49-4eb8-8224-c3090757ed2a-config\") pod \"135a97dd-af49-4eb8-8224-c3090757ed2a\" (UID: \"135a97dd-af49-4eb8-8224-c3090757ed2a\") " Dec 03 12:27:47 crc kubenswrapper[5052]: I1203 12:27:47.069161 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/135a97dd-af49-4eb8-8224-c3090757ed2a-combined-ca-bundle\") pod \"135a97dd-af49-4eb8-8224-c3090757ed2a\" (UID: \"135a97dd-af49-4eb8-8224-c3090757ed2a\") " Dec 03 12:27:47 crc kubenswrapper[5052]: I1203 12:27:47.069260 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/135a97dd-af49-4eb8-8224-c3090757ed2a-httpd-config\") pod \"135a97dd-af49-4eb8-8224-c3090757ed2a\" (UID: \"135a97dd-af49-4eb8-8224-c3090757ed2a\") " Dec 03 12:27:47 crc kubenswrapper[5052]: I1203 12:27:47.077749 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/135a97dd-af49-4eb8-8224-c3090757ed2a-kube-api-access-jdmsc" (OuterVolumeSpecName: "kube-api-access-jdmsc") pod "135a97dd-af49-4eb8-8224-c3090757ed2a" (UID: "135a97dd-af49-4eb8-8224-c3090757ed2a"). InnerVolumeSpecName "kube-api-access-jdmsc". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:27:47 crc kubenswrapper[5052]: I1203 12:27:47.106733 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/135a97dd-af49-4eb8-8224-c3090757ed2a-httpd-config" (OuterVolumeSpecName: "httpd-config") pod "135a97dd-af49-4eb8-8224-c3090757ed2a" (UID: "135a97dd-af49-4eb8-8224-c3090757ed2a"). InnerVolumeSpecName "httpd-config". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:27:47 crc kubenswrapper[5052]: I1203 12:27:47.130935 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/135a97dd-af49-4eb8-8224-c3090757ed2a-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "135a97dd-af49-4eb8-8224-c3090757ed2a" (UID: "135a97dd-af49-4eb8-8224-c3090757ed2a"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:27:47 crc kubenswrapper[5052]: I1203 12:27:47.134460 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/135a97dd-af49-4eb8-8224-c3090757ed2a-config" (OuterVolumeSpecName: "config") pod "135a97dd-af49-4eb8-8224-c3090757ed2a" (UID: "135a97dd-af49-4eb8-8224-c3090757ed2a"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:27:47 crc kubenswrapper[5052]: I1203 12:27:47.172261 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/135a97dd-af49-4eb8-8224-c3090757ed2a-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:47 crc kubenswrapper[5052]: I1203 12:27:47.172306 5052 reconciler_common.go:293] "Volume detached for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/135a97dd-af49-4eb8-8224-c3090757ed2a-httpd-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:47 crc kubenswrapper[5052]: I1203 12:27:47.172321 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-jdmsc\" (UniqueName: \"kubernetes.io/projected/135a97dd-af49-4eb8-8224-c3090757ed2a-kube-api-access-jdmsc\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:47 crc kubenswrapper[5052]: I1203 12:27:47.172338 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/135a97dd-af49-4eb8-8224-c3090757ed2a-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:47 crc kubenswrapper[5052]: I1203 12:27:47.479093 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_neutron-544d7dcfcd-7xmk7_135a97dd-af49-4eb8-8224-c3090757ed2a/neutron-httpd/0.log" Dec 03 12:27:47 crc kubenswrapper[5052]: I1203 12:27:47.479878 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_neutron-544d7dcfcd-7xmk7_135a97dd-af49-4eb8-8224-c3090757ed2a/neutron-api/0.log" Dec 03 12:27:47 crc kubenswrapper[5052]: I1203 12:27:47.479944 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-544d7dcfcd-7xmk7" event={"ID":"135a97dd-af49-4eb8-8224-c3090757ed2a","Type":"ContainerDied","Data":"e4b8d4e8b2dce2be1e5e689f7cc49293bcb1ef7c5271e2b0df6422ab8b2e5ef7"} Dec 03 12:27:47 crc kubenswrapper[5052]: I1203 12:27:47.479981 5052 scope.go:117] "RemoveContainer" containerID="1ee73b25fb176fb0f7a5ba787284992b6e0c4d72b574d2a31c1df2ab8746440f" Dec 03 12:27:47 crc kubenswrapper[5052]: I1203 12:27:47.480113 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-544d7dcfcd-7xmk7" Dec 03 12:27:47 crc kubenswrapper[5052]: I1203 12:27:47.496968 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"9a71429c-9d59-4863-8949-dcb4f79a35e2","Type":"ContainerStarted","Data":"7fe073f06854f186159ef18897942608baedecbba0d07d0c8013355c37619643"} Dec 03 12:27:47 crc kubenswrapper[5052]: I1203 12:27:47.497006 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/memcached-0" Dec 03 12:27:47 crc kubenswrapper[5052]: I1203 12:27:47.497020 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"9a71429c-9d59-4863-8949-dcb4f79a35e2","Type":"ContainerStarted","Data":"44b04da4e7f0a025325c3b6375369b7ac347c2ad50331290bfb481efa7ef0736"} Dec 03 12:27:47 crc kubenswrapper[5052]: I1203 12:27:47.497034 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"9a71429c-9d59-4863-8949-dcb4f79a35e2","Type":"ContainerStarted","Data":"9d3d20ffcb7e4c042c65cb9789046c8a52471258ecf145006245287d12709e44"} Dec 03 12:27:47 crc kubenswrapper[5052]: I1203 12:27:47.497046 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"9a71429c-9d59-4863-8949-dcb4f79a35e2","Type":"ContainerStarted","Data":"be42b27cc3c5b3773e9ff8c15fd7487a0c826144d8760ad2e8dbee3a1515ac77"} Dec 03 12:27:47 crc kubenswrapper[5052]: I1203 12:27:47.497058 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"9a71429c-9d59-4863-8949-dcb4f79a35e2","Type":"ContainerStarted","Data":"9920899d34f69573106a85bde57a4bdf8c52b176a8195efa2c785d4039c638ea"} Dec 03 12:27:47 crc kubenswrapper[5052]: I1203 12:27:47.524586 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/neutron-544d7dcfcd-7xmk7"] Dec 03 12:27:47 crc kubenswrapper[5052]: I1203 12:27:47.531583 5052 scope.go:117] "RemoveContainer" containerID="47023cf3d975406d17e04a8c3b72230adbcd30f7306a30b2f6867cc8b0f83bdc" Dec 03 12:27:47 crc kubenswrapper[5052]: I1203 12:27:47.536402 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/neutron-544d7dcfcd-7xmk7"] Dec 03 12:27:47 crc kubenswrapper[5052]: I1203 12:27:47.551290 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/memcached-0"] Dec 03 12:27:47 crc kubenswrapper[5052]: I1203 12:27:47.567847 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/memcached-0"] Dec 03 12:27:47 crc kubenswrapper[5052]: I1203 12:27:47.582913 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/memcached-0"] Dec 03 12:27:47 crc kubenswrapper[5052]: E1203 12:27:47.583361 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="135a97dd-af49-4eb8-8224-c3090757ed2a" containerName="neutron-api" Dec 03 12:27:47 crc kubenswrapper[5052]: I1203 12:27:47.583383 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="135a97dd-af49-4eb8-8224-c3090757ed2a" containerName="neutron-api" Dec 03 12:27:47 crc kubenswrapper[5052]: E1203 12:27:47.583444 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="135a97dd-af49-4eb8-8224-c3090757ed2a" containerName="neutron-httpd" Dec 03 12:27:47 crc kubenswrapper[5052]: I1203 12:27:47.583453 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="135a97dd-af49-4eb8-8224-c3090757ed2a" containerName="neutron-httpd" Dec 03 12:27:47 crc kubenswrapper[5052]: I1203 12:27:47.583895 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="135a97dd-af49-4eb8-8224-c3090757ed2a" containerName="neutron-httpd" Dec 03 12:27:47 crc kubenswrapper[5052]: I1203 12:27:47.583923 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="135a97dd-af49-4eb8-8224-c3090757ed2a" containerName="neutron-api" Dec 03 12:27:47 crc kubenswrapper[5052]: I1203 12:27:47.584770 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/memcached-0" Dec 03 12:27:47 crc kubenswrapper[5052]: I1203 12:27:47.587820 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-memcached-svc" Dec 03 12:27:47 crc kubenswrapper[5052]: I1203 12:27:47.587820 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"memcached-memcached-dockercfg-2sngt" Dec 03 12:27:47 crc kubenswrapper[5052]: I1203 12:27:47.589585 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"memcached-config-data" Dec 03 12:27:47 crc kubenswrapper[5052]: I1203 12:27:47.599385 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/memcached-0"] Dec 03 12:27:47 crc kubenswrapper[5052]: I1203 12:27:47.680083 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/b35e6939-8f23-496c-ac55-9534be4ea8be-config-data\") pod \"memcached-0\" (UID: \"b35e6939-8f23-496c-ac55-9534be4ea8be\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:27:47 crc kubenswrapper[5052]: I1203 12:27:47.680190 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j2cn6\" (UniqueName: \"kubernetes.io/projected/b35e6939-8f23-496c-ac55-9534be4ea8be-kube-api-access-j2cn6\") pod \"memcached-0\" (UID: \"b35e6939-8f23-496c-ac55-9534be4ea8be\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:27:47 crc kubenswrapper[5052]: I1203 12:27:47.680233 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b35e6939-8f23-496c-ac55-9534be4ea8be-combined-ca-bundle\") pod \"memcached-0\" (UID: \"b35e6939-8f23-496c-ac55-9534be4ea8be\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:27:47 crc kubenswrapper[5052]: I1203 12:27:47.680289 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/b35e6939-8f23-496c-ac55-9534be4ea8be-kolla-config\") pod \"memcached-0\" (UID: \"b35e6939-8f23-496c-ac55-9534be4ea8be\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:27:47 crc kubenswrapper[5052]: I1203 12:27:47.680315 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"memcached-tls-certs\" (UniqueName: \"kubernetes.io/secret/b35e6939-8f23-496c-ac55-9534be4ea8be-memcached-tls-certs\") pod \"memcached-0\" (UID: \"b35e6939-8f23-496c-ac55-9534be4ea8be\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:27:47 crc kubenswrapper[5052]: I1203 12:27:47.688394 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="135a97dd-af49-4eb8-8224-c3090757ed2a" path="/var/lib/kubelet/pods/135a97dd-af49-4eb8-8224-c3090757ed2a/volumes" Dec 03 12:27:47 crc kubenswrapper[5052]: I1203 12:27:47.689190 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="65fc3838-f25d-4eda-9643-6b63215c41ff" path="/var/lib/kubelet/pods/65fc3838-f25d-4eda-9643-6b63215c41ff/volumes" Dec 03 12:27:47 crc kubenswrapper[5052]: I1203 12:27:47.781899 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"memcached-tls-certs\" (UniqueName: \"kubernetes.io/secret/b35e6939-8f23-496c-ac55-9534be4ea8be-memcached-tls-certs\") pod \"memcached-0\" (UID: \"b35e6939-8f23-496c-ac55-9534be4ea8be\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:27:47 crc kubenswrapper[5052]: I1203 12:27:47.782058 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/b35e6939-8f23-496c-ac55-9534be4ea8be-config-data\") pod \"memcached-0\" (UID: \"b35e6939-8f23-496c-ac55-9534be4ea8be\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:27:47 crc kubenswrapper[5052]: I1203 12:27:47.782192 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-j2cn6\" (UniqueName: \"kubernetes.io/projected/b35e6939-8f23-496c-ac55-9534be4ea8be-kube-api-access-j2cn6\") pod \"memcached-0\" (UID: \"b35e6939-8f23-496c-ac55-9534be4ea8be\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:27:47 crc kubenswrapper[5052]: I1203 12:27:47.782277 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b35e6939-8f23-496c-ac55-9534be4ea8be-combined-ca-bundle\") pod \"memcached-0\" (UID: \"b35e6939-8f23-496c-ac55-9534be4ea8be\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:27:47 crc kubenswrapper[5052]: I1203 12:27:47.782373 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/b35e6939-8f23-496c-ac55-9534be4ea8be-kolla-config\") pod \"memcached-0\" (UID: \"b35e6939-8f23-496c-ac55-9534be4ea8be\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:27:47 crc kubenswrapper[5052]: I1203 12:27:47.784188 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/b35e6939-8f23-496c-ac55-9534be4ea8be-config-data\") pod \"memcached-0\" (UID: \"b35e6939-8f23-496c-ac55-9534be4ea8be\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:27:47 crc kubenswrapper[5052]: I1203 12:27:47.784635 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/b35e6939-8f23-496c-ac55-9534be4ea8be-kolla-config\") pod \"memcached-0\" (UID: \"b35e6939-8f23-496c-ac55-9534be4ea8be\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:27:47 crc kubenswrapper[5052]: I1203 12:27:47.789339 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b35e6939-8f23-496c-ac55-9534be4ea8be-combined-ca-bundle\") pod \"memcached-0\" (UID: \"b35e6939-8f23-496c-ac55-9534be4ea8be\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:27:47 crc kubenswrapper[5052]: I1203 12:27:47.798233 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"memcached-tls-certs\" (UniqueName: \"kubernetes.io/secret/b35e6939-8f23-496c-ac55-9534be4ea8be-memcached-tls-certs\") pod \"memcached-0\" (UID: \"b35e6939-8f23-496c-ac55-9534be4ea8be\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:27:47 crc kubenswrapper[5052]: I1203 12:27:47.809350 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-j2cn6\" (UniqueName: \"kubernetes.io/projected/b35e6939-8f23-496c-ac55-9534be4ea8be-kube-api-access-j2cn6\") pod \"memcached-0\" (UID: \"b35e6939-8f23-496c-ac55-9534be4ea8be\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:27:47 crc kubenswrapper[5052]: I1203 12:27:47.911496 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/memcached-0" Dec 03 12:27:48 crc kubenswrapper[5052]: I1203 12:27:48.144122 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/placement-6cdf7689bb-fscpc" Dec 03 12:27:48 crc kubenswrapper[5052]: I1203 12:27:48.233537 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/placement-6cdf7689bb-fscpc" Dec 03 12:27:48 crc kubenswrapper[5052]: W1203 12:27:48.397718 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb35e6939_8f23_496c_ac55_9534be4ea8be.slice/crio-ef859d2fc4d61f028b9227837d828fbe4c68fa551371ec131c1b1149ab6d5d3f WatchSource:0}: Error finding container ef859d2fc4d61f028b9227837d828fbe4c68fa551371ec131c1b1149ab6d5d3f: Status 404 returned error can't find the container with id ef859d2fc4d61f028b9227837d828fbe4c68fa551371ec131c1b1149ab6d5d3f Dec 03 12:27:48 crc kubenswrapper[5052]: I1203 12:27:48.399923 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/memcached-0"] Dec 03 12:27:48 crc kubenswrapper[5052]: I1203 12:27:48.509882 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"9a71429c-9d59-4863-8949-dcb4f79a35e2","Type":"ContainerStarted","Data":"844dc1daec944c795b946deb25e04e380651676c9d2e4f9f20f0b937c87cd282"} Dec 03 12:27:48 crc kubenswrapper[5052]: I1203 12:27:48.512207 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/memcached-0" event={"ID":"b35e6939-8f23-496c-ac55-9534be4ea8be","Type":"ContainerStarted","Data":"ef859d2fc4d61f028b9227837d828fbe4c68fa551371ec131c1b1149ab6d5d3f"} Dec 03 12:27:49 crc kubenswrapper[5052]: I1203 12:27:49.104319 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/placement-7897d64b6b-j8w2r" Dec 03 12:27:49 crc kubenswrapper[5052]: I1203 12:27:49.177346 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/placement-84d4fb48db-blkmf"] Dec 03 12:27:49 crc kubenswrapper[5052]: I1203 12:27:49.188687 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/placement-84d4fb48db-blkmf" podUID="3b116fe9-1f40-4614-9381-7f7964fbdcb4" containerName="placement-log" containerID="cri-o://c855f9d1a9983bf4428f3f423fd3f1abd57a52f10787a431f76087b5f92f6bd5" gracePeriod=30 Dec 03 12:27:49 crc kubenswrapper[5052]: I1203 12:27:49.188749 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/placement-84d4fb48db-blkmf" podUID="3b116fe9-1f40-4614-9381-7f7964fbdcb4" containerName="placement-api" containerID="cri-o://35babc941d9382068ca3592fefa32271466b9c548b0f0eeab1a1fc57b7318180" gracePeriod=30 Dec 03 12:27:49 crc kubenswrapper[5052]: I1203 12:27:49.527942 5052 generic.go:334] "Generic (PLEG): container finished" podID="3b116fe9-1f40-4614-9381-7f7964fbdcb4" containerID="c855f9d1a9983bf4428f3f423fd3f1abd57a52f10787a431f76087b5f92f6bd5" exitCode=143 Dec 03 12:27:49 crc kubenswrapper[5052]: I1203 12:27:49.528199 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-84d4fb48db-blkmf" event={"ID":"3b116fe9-1f40-4614-9381-7f7964fbdcb4","Type":"ContainerDied","Data":"c855f9d1a9983bf4428f3f423fd3f1abd57a52f10787a431f76087b5f92f6bd5"} Dec 03 12:27:49 crc kubenswrapper[5052]: I1203 12:27:49.531272 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"9a71429c-9d59-4863-8949-dcb4f79a35e2","Type":"ContainerStarted","Data":"e56722fdb2d4fc10478af03b1d3b9ea29d7757b10975bec562e15e2949b85364"} Dec 03 12:27:49 crc kubenswrapper[5052]: I1203 12:27:49.849213 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:27:50 crc kubenswrapper[5052]: I1203 12:27:50.216256 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-7995cfc4b8-v5sr4" Dec 03 12:27:50 crc kubenswrapper[5052]: I1203 12:27:50.345992 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/c2463445-211e-4724-ba2c-4cc1661a6452-config-data\") pod \"c2463445-211e-4724-ba2c-4cc1661a6452\" (UID: \"c2463445-211e-4724-ba2c-4cc1661a6452\") " Dec 03 12:27:50 crc kubenswrapper[5052]: I1203 12:27:50.346564 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/c2463445-211e-4724-ba2c-4cc1661a6452-credential-keys\") pod \"c2463445-211e-4724-ba2c-4cc1661a6452\" (UID: \"c2463445-211e-4724-ba2c-4cc1661a6452\") " Dec 03 12:27:50 crc kubenswrapper[5052]: I1203 12:27:50.346638 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c2463445-211e-4724-ba2c-4cc1661a6452-combined-ca-bundle\") pod \"c2463445-211e-4724-ba2c-4cc1661a6452\" (UID: \"c2463445-211e-4724-ba2c-4cc1661a6452\") " Dec 03 12:27:50 crc kubenswrapper[5052]: I1203 12:27:50.346665 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4c8ms\" (UniqueName: \"kubernetes.io/projected/c2463445-211e-4724-ba2c-4cc1661a6452-kube-api-access-4c8ms\") pod \"c2463445-211e-4724-ba2c-4cc1661a6452\" (UID: \"c2463445-211e-4724-ba2c-4cc1661a6452\") " Dec 03 12:27:50 crc kubenswrapper[5052]: I1203 12:27:50.346729 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/c2463445-211e-4724-ba2c-4cc1661a6452-scripts\") pod \"c2463445-211e-4724-ba2c-4cc1661a6452\" (UID: \"c2463445-211e-4724-ba2c-4cc1661a6452\") " Dec 03 12:27:50 crc kubenswrapper[5052]: I1203 12:27:50.346753 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/c2463445-211e-4724-ba2c-4cc1661a6452-fernet-keys\") pod \"c2463445-211e-4724-ba2c-4cc1661a6452\" (UID: \"c2463445-211e-4724-ba2c-4cc1661a6452\") " Dec 03 12:27:50 crc kubenswrapper[5052]: I1203 12:27:50.355522 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c2463445-211e-4724-ba2c-4cc1661a6452-fernet-keys" (OuterVolumeSpecName: "fernet-keys") pod "c2463445-211e-4724-ba2c-4cc1661a6452" (UID: "c2463445-211e-4724-ba2c-4cc1661a6452"). InnerVolumeSpecName "fernet-keys". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:27:50 crc kubenswrapper[5052]: I1203 12:27:50.355793 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c2463445-211e-4724-ba2c-4cc1661a6452-kube-api-access-4c8ms" (OuterVolumeSpecName: "kube-api-access-4c8ms") pod "c2463445-211e-4724-ba2c-4cc1661a6452" (UID: "c2463445-211e-4724-ba2c-4cc1661a6452"). InnerVolumeSpecName "kube-api-access-4c8ms". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:27:50 crc kubenswrapper[5052]: I1203 12:27:50.356534 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c2463445-211e-4724-ba2c-4cc1661a6452-credential-keys" (OuterVolumeSpecName: "credential-keys") pod "c2463445-211e-4724-ba2c-4cc1661a6452" (UID: "c2463445-211e-4724-ba2c-4cc1661a6452"). InnerVolumeSpecName "credential-keys". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:27:50 crc kubenswrapper[5052]: I1203 12:27:50.357001 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c2463445-211e-4724-ba2c-4cc1661a6452-scripts" (OuterVolumeSpecName: "scripts") pod "c2463445-211e-4724-ba2c-4cc1661a6452" (UID: "c2463445-211e-4724-ba2c-4cc1661a6452"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:27:50 crc kubenswrapper[5052]: I1203 12:27:50.385952 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c2463445-211e-4724-ba2c-4cc1661a6452-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "c2463445-211e-4724-ba2c-4cc1661a6452" (UID: "c2463445-211e-4724-ba2c-4cc1661a6452"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:27:50 crc kubenswrapper[5052]: I1203 12:27:50.388231 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c2463445-211e-4724-ba2c-4cc1661a6452-config-data" (OuterVolumeSpecName: "config-data") pod "c2463445-211e-4724-ba2c-4cc1661a6452" (UID: "c2463445-211e-4724-ba2c-4cc1661a6452"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:27:50 crc kubenswrapper[5052]: I1203 12:27:50.449154 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/c2463445-211e-4724-ba2c-4cc1661a6452-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:50 crc kubenswrapper[5052]: I1203 12:27:50.449950 5052 reconciler_common.go:293] "Volume detached for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/c2463445-211e-4724-ba2c-4cc1661a6452-credential-keys\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:50 crc kubenswrapper[5052]: I1203 12:27:50.450020 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c2463445-211e-4724-ba2c-4cc1661a6452-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:50 crc kubenswrapper[5052]: I1203 12:27:50.450084 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-4c8ms\" (UniqueName: \"kubernetes.io/projected/c2463445-211e-4724-ba2c-4cc1661a6452-kube-api-access-4c8ms\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:50 crc kubenswrapper[5052]: I1203 12:27:50.450388 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/c2463445-211e-4724-ba2c-4cc1661a6452-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:50 crc kubenswrapper[5052]: I1203 12:27:50.450445 5052 reconciler_common.go:293] "Volume detached for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/c2463445-211e-4724-ba2c-4cc1661a6452-fernet-keys\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:50 crc kubenswrapper[5052]: I1203 12:27:50.541711 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/memcached-0" event={"ID":"b35e6939-8f23-496c-ac55-9534be4ea8be","Type":"ContainerStarted","Data":"8d5e1c2473a3c27b159232ab3f630f8e1ca663780d8fc9b7b1eaac3611c97961"} Dec 03 12:27:50 crc kubenswrapper[5052]: I1203 12:27:50.542270 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/memcached-0" Dec 03 12:27:50 crc kubenswrapper[5052]: I1203 12:27:50.543743 5052 generic.go:334] "Generic (PLEG): container finished" podID="c2463445-211e-4724-ba2c-4cc1661a6452" containerID="314b48c25f8f96b0a4711b6bf034891f8f2066552bba485c2589fd24ede9c77d" exitCode=0 Dec 03 12:27:50 crc kubenswrapper[5052]: I1203 12:27:50.543813 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-7995cfc4b8-v5sr4" event={"ID":"c2463445-211e-4724-ba2c-4cc1661a6452","Type":"ContainerDied","Data":"314b48c25f8f96b0a4711b6bf034891f8f2066552bba485c2589fd24ede9c77d"} Dec 03 12:27:50 crc kubenswrapper[5052]: I1203 12:27:50.543894 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-7995cfc4b8-v5sr4" event={"ID":"c2463445-211e-4724-ba2c-4cc1661a6452","Type":"ContainerDied","Data":"c4af836f28c749948b99e079ba0c0fc678410b9514c9f531a96c216a8d8cfade"} Dec 03 12:27:50 crc kubenswrapper[5052]: I1203 12:27:50.543919 5052 scope.go:117] "RemoveContainer" containerID="314b48c25f8f96b0a4711b6bf034891f8f2066552bba485c2589fd24ede9c77d" Dec 03 12:27:50 crc kubenswrapper[5052]: I1203 12:27:50.544115 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-7995cfc4b8-v5sr4" Dec 03 12:27:50 crc kubenswrapper[5052]: I1203 12:27:50.553313 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"9a71429c-9d59-4863-8949-dcb4f79a35e2","Type":"ContainerStarted","Data":"1313894d0c0976392a7edf9b30f67f35fdafa36110b15f07dc7f5ffd68038ba8"} Dec 03 12:27:50 crc kubenswrapper[5052]: I1203 12:27:50.574748 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/memcached-0" podStartSLOduration=3.574726939 podStartE2EDuration="3.574726939s" podCreationTimestamp="2025-12-03 12:27:47 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:27:50.567141993 +0000 UTC m=+4061.155383767" watchObservedRunningTime="2025-12-03 12:27:50.574726939 +0000 UTC m=+4061.162968703" Dec 03 12:27:50 crc kubenswrapper[5052]: I1203 12:27:50.578545 5052 scope.go:117] "RemoveContainer" containerID="314b48c25f8f96b0a4711b6bf034891f8f2066552bba485c2589fd24ede9c77d" Dec 03 12:27:50 crc kubenswrapper[5052]: E1203 12:27:50.578990 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"314b48c25f8f96b0a4711b6bf034891f8f2066552bba485c2589fd24ede9c77d\": container with ID starting with 314b48c25f8f96b0a4711b6bf034891f8f2066552bba485c2589fd24ede9c77d not found: ID does not exist" containerID="314b48c25f8f96b0a4711b6bf034891f8f2066552bba485c2589fd24ede9c77d" Dec 03 12:27:50 crc kubenswrapper[5052]: I1203 12:27:50.579071 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"314b48c25f8f96b0a4711b6bf034891f8f2066552bba485c2589fd24ede9c77d"} err="failed to get container status \"314b48c25f8f96b0a4711b6bf034891f8f2066552bba485c2589fd24ede9c77d\": rpc error: code = NotFound desc = could not find container \"314b48c25f8f96b0a4711b6bf034891f8f2066552bba485c2589fd24ede9c77d\": container with ID starting with 314b48c25f8f96b0a4711b6bf034891f8f2066552bba485c2589fd24ede9c77d not found: ID does not exist" Dec 03 12:27:50 crc kubenswrapper[5052]: I1203 12:27:50.594607 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/keystone-7995cfc4b8-v5sr4"] Dec 03 12:27:50 crc kubenswrapper[5052]: I1203 12:27:50.603544 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/keystone-7995cfc4b8-v5sr4"] Dec 03 12:27:50 crc kubenswrapper[5052]: I1203 12:27:50.859058 5052 pod_container_manager_linux.go:210] "Failed to delete cgroup paths" cgroupName=["kubepods","besteffort","podaa94b30e-0028-45a0-88aa-b5db5fc2f8fd"] err="unable to destroy cgroup paths for cgroup [kubepods besteffort podaa94b30e-0028-45a0-88aa-b5db5fc2f8fd] : Timed out while waiting for systemd to remove kubepods-besteffort-podaa94b30e_0028_45a0_88aa_b5db5fc2f8fd.slice" Dec 03 12:27:50 crc kubenswrapper[5052]: E1203 12:27:50.859111 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to delete cgroup paths for [kubepods besteffort podaa94b30e-0028-45a0-88aa-b5db5fc2f8fd] : unable to destroy cgroup paths for cgroup [kubepods besteffort podaa94b30e-0028-45a0-88aa-b5db5fc2f8fd] : Timed out while waiting for systemd to remove kubepods-besteffort-podaa94b30e_0028_45a0_88aa_b5db5fc2f8fd.slice" pod="openstack-kuttl-tests/keystone-66778fc6-56rjc" podUID="aa94b30e-0028-45a0-88aa-b5db5fc2f8fd" Dec 03 12:27:50 crc kubenswrapper[5052]: I1203 12:27:50.865832 5052 pod_container_manager_linux.go:210] "Failed to delete cgroup paths" cgroupName=["kubepods","besteffort","podfc4f1efa-a22b-4c43-b479-22562c0a7e65"] err="unable to destroy cgroup paths for cgroup [kubepods besteffort podfc4f1efa-a22b-4c43-b479-22562c0a7e65] : Timed out while waiting for systemd to remove kubepods-besteffort-podfc4f1efa_a22b_4c43_b479_22562c0a7e65.slice" Dec 03 12:27:50 crc kubenswrapper[5052]: E1203 12:27:50.865893 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to delete cgroup paths for [kubepods besteffort podfc4f1efa-a22b-4c43-b479-22562c0a7e65] : unable to destroy cgroup paths for cgroup [kubepods besteffort podfc4f1efa-a22b-4c43-b479-22562c0a7e65] : Timed out while waiting for systemd to remove kubepods-besteffort-podfc4f1efa_a22b_4c43_b479_22562c0a7e65.slice" pod="openstack-kuttl-tests/cinder-api-0" podUID="fc4f1efa-a22b-4c43-b479-22562c0a7e65" Dec 03 12:27:51 crc kubenswrapper[5052]: I1203 12:27:51.027501 5052 pod_container_manager_linux.go:210] "Failed to delete cgroup paths" cgroupName=["kubepods","besteffort","pod6b96dc78-b389-4764-8ba7-6a4c5e41e345"] err="unable to destroy cgroup paths for cgroup [kubepods besteffort pod6b96dc78-b389-4764-8ba7-6a4c5e41e345] : Timed out while waiting for systemd to remove kubepods-besteffort-pod6b96dc78_b389_4764_8ba7_6a4c5e41e345.slice" Dec 03 12:27:51 crc kubenswrapper[5052]: E1203 12:27:51.027564 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to delete cgroup paths for [kubepods besteffort pod6b96dc78-b389-4764-8ba7-6a4c5e41e345] : unable to destroy cgroup paths for cgroup [kubepods besteffort pod6b96dc78-b389-4764-8ba7-6a4c5e41e345] : Timed out while waiting for systemd to remove kubepods-besteffort-pod6b96dc78_b389_4764_8ba7_6a4c5e41e345.slice" pod="openstack-kuttl-tests/glance-default-external-api-0" podUID="6b96dc78-b389-4764-8ba7-6a4c5e41e345" Dec 03 12:27:51 crc kubenswrapper[5052]: I1203 12:27:51.570259 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:27:51 crc kubenswrapper[5052]: I1203 12:27:51.571325 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"9a71429c-9d59-4863-8949-dcb4f79a35e2","Type":"ContainerStarted","Data":"6d6d0a9e0c44982301515f6c27041d26e825a4e904631b906720c66939d7d046"} Dec 03 12:27:51 crc kubenswrapper[5052]: I1203 12:27:51.571364 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"9a71429c-9d59-4863-8949-dcb4f79a35e2","Type":"ContainerStarted","Data":"2a0e35f878dfc50c4b4c6c92dc0efc336af0f313d37084f949296ada2fc8a9a9"} Dec 03 12:27:51 crc kubenswrapper[5052]: I1203 12:27:51.571370 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:27:51 crc kubenswrapper[5052]: I1203 12:27:51.571412 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-66778fc6-56rjc" Dec 03 12:27:51 crc kubenswrapper[5052]: I1203 12:27:51.631969 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 12:27:51 crc kubenswrapper[5052]: I1203 12:27:51.656558 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 12:27:51 crc kubenswrapper[5052]: I1203 12:27:51.714283 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c2463445-211e-4724-ba2c-4cc1661a6452" path="/var/lib/kubelet/pods/c2463445-211e-4724-ba2c-4cc1661a6452/volumes" Dec 03 12:27:51 crc kubenswrapper[5052]: I1203 12:27:51.715005 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fc4f1efa-a22b-4c43-b479-22562c0a7e65" path="/var/lib/kubelet/pods/fc4f1efa-a22b-4c43-b479-22562c0a7e65/volumes" Dec 03 12:27:51 crc kubenswrapper[5052]: I1203 12:27:51.721072 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/keystone-66778fc6-56rjc"] Dec 03 12:27:51 crc kubenswrapper[5052]: I1203 12:27:51.721127 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 12:27:51 crc kubenswrapper[5052]: E1203 12:27:51.721604 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="c2463445-211e-4724-ba2c-4cc1661a6452" containerName="keystone-api" Dec 03 12:27:51 crc kubenswrapper[5052]: I1203 12:27:51.721622 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="c2463445-211e-4724-ba2c-4cc1661a6452" containerName="keystone-api" Dec 03 12:27:51 crc kubenswrapper[5052]: I1203 12:27:51.721882 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="c2463445-211e-4724-ba2c-4cc1661a6452" containerName="keystone-api" Dec 03 12:27:51 crc kubenswrapper[5052]: I1203 12:27:51.723076 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:27:51 crc kubenswrapper[5052]: I1203 12:27:51.728255 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cinder-api-config-data" Dec 03 12:27:51 crc kubenswrapper[5052]: I1203 12:27:51.741033 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/keystone-66778fc6-56rjc"] Dec 03 12:27:51 crc kubenswrapper[5052]: I1203 12:27:51.766061 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 12:27:51 crc kubenswrapper[5052]: I1203 12:27:51.792976 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:27:51 crc kubenswrapper[5052]: I1203 12:27:51.819978 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:27:51 crc kubenswrapper[5052]: I1203 12:27:51.834533 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:27:51 crc kubenswrapper[5052]: I1203 12:27:51.836785 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:27:51 crc kubenswrapper[5052]: I1203 12:27:51.842686 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-default-external-config-data" Dec 03 12:27:51 crc kubenswrapper[5052]: I1203 12:27:51.849392 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:27:51 crc kubenswrapper[5052]: I1203 12:27:51.883799 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/2b5843e3-84ff-4174-bd4a-f67ba2a3a238-logs\") pod \"cinder-api-0\" (UID: \"2b5843e3-84ff-4174-bd4a-f67ba2a3a238\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:27:51 crc kubenswrapper[5052]: I1203 12:27:51.883880 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2b5843e3-84ff-4174-bd4a-f67ba2a3a238-combined-ca-bundle\") pod \"cinder-api-0\" (UID: \"2b5843e3-84ff-4174-bd4a-f67ba2a3a238\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:27:51 crc kubenswrapper[5052]: I1203 12:27:51.883921 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/2b5843e3-84ff-4174-bd4a-f67ba2a3a238-etc-machine-id\") pod \"cinder-api-0\" (UID: \"2b5843e3-84ff-4174-bd4a-f67ba2a3a238\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:27:51 crc kubenswrapper[5052]: I1203 12:27:51.883942 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/2b5843e3-84ff-4174-bd4a-f67ba2a3a238-config-data\") pod \"cinder-api-0\" (UID: \"2b5843e3-84ff-4174-bd4a-f67ba2a3a238\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:27:51 crc kubenswrapper[5052]: I1203 12:27:51.884027 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mjwfk\" (UniqueName: \"kubernetes.io/projected/2b5843e3-84ff-4174-bd4a-f67ba2a3a238-kube-api-access-mjwfk\") pod \"cinder-api-0\" (UID: \"2b5843e3-84ff-4174-bd4a-f67ba2a3a238\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:27:51 crc kubenswrapper[5052]: I1203 12:27:51.884048 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/2b5843e3-84ff-4174-bd4a-f67ba2a3a238-scripts\") pod \"cinder-api-0\" (UID: \"2b5843e3-84ff-4174-bd4a-f67ba2a3a238\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:27:51 crc kubenswrapper[5052]: I1203 12:27:51.884090 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/2b5843e3-84ff-4174-bd4a-f67ba2a3a238-config-data-custom\") pod \"cinder-api-0\" (UID: \"2b5843e3-84ff-4174-bd4a-f67ba2a3a238\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:27:51 crc kubenswrapper[5052]: I1203 12:27:51.985994 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/2b5843e3-84ff-4174-bd4a-f67ba2a3a238-logs\") pod \"cinder-api-0\" (UID: \"2b5843e3-84ff-4174-bd4a-f67ba2a3a238\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:27:51 crc kubenswrapper[5052]: I1203 12:27:51.986658 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/71e0e74b-5d01-467f-afb2-7e55d9d58297-scripts\") pod \"glance-default-external-api-0\" (UID: \"71e0e74b-5d01-467f-afb2-7e55d9d58297\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:27:51 crc kubenswrapper[5052]: I1203 12:27:51.986836 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/71e0e74b-5d01-467f-afb2-7e55d9d58297-logs\") pod \"glance-default-external-api-0\" (UID: \"71e0e74b-5d01-467f-afb2-7e55d9d58297\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:27:51 crc kubenswrapper[5052]: I1203 12:27:51.986500 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/2b5843e3-84ff-4174-bd4a-f67ba2a3a238-logs\") pod \"cinder-api-0\" (UID: \"2b5843e3-84ff-4174-bd4a-f67ba2a3a238\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:27:51 crc kubenswrapper[5052]: I1203 12:27:51.987079 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2b5843e3-84ff-4174-bd4a-f67ba2a3a238-combined-ca-bundle\") pod \"cinder-api-0\" (UID: \"2b5843e3-84ff-4174-bd4a-f67ba2a3a238\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:27:51 crc kubenswrapper[5052]: I1203 12:27:51.987160 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/2b5843e3-84ff-4174-bd4a-f67ba2a3a238-etc-machine-id\") pod \"cinder-api-0\" (UID: \"2b5843e3-84ff-4174-bd4a-f67ba2a3a238\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:27:51 crc kubenswrapper[5052]: I1203 12:27:51.987186 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/2b5843e3-84ff-4174-bd4a-f67ba2a3a238-config-data\") pod \"cinder-api-0\" (UID: \"2b5843e3-84ff-4174-bd4a-f67ba2a3a238\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:27:51 crc kubenswrapper[5052]: I1203 12:27:51.987218 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage07-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage07-crc\") pod \"glance-default-external-api-0\" (UID: \"71e0e74b-5d01-467f-afb2-7e55d9d58297\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:27:51 crc kubenswrapper[5052]: I1203 12:27:51.987297 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/71e0e74b-5d01-467f-afb2-7e55d9d58297-config-data\") pod \"glance-default-external-api-0\" (UID: \"71e0e74b-5d01-467f-afb2-7e55d9d58297\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:27:51 crc kubenswrapper[5052]: I1203 12:27:51.987293 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/2b5843e3-84ff-4174-bd4a-f67ba2a3a238-etc-machine-id\") pod \"cinder-api-0\" (UID: \"2b5843e3-84ff-4174-bd4a-f67ba2a3a238\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:27:51 crc kubenswrapper[5052]: I1203 12:27:51.987421 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/71e0e74b-5d01-467f-afb2-7e55d9d58297-httpd-run\") pod \"glance-default-external-api-0\" (UID: \"71e0e74b-5d01-467f-afb2-7e55d9d58297\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:27:51 crc kubenswrapper[5052]: I1203 12:27:51.987496 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-mjwfk\" (UniqueName: \"kubernetes.io/projected/2b5843e3-84ff-4174-bd4a-f67ba2a3a238-kube-api-access-mjwfk\") pod \"cinder-api-0\" (UID: \"2b5843e3-84ff-4174-bd4a-f67ba2a3a238\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:27:51 crc kubenswrapper[5052]: I1203 12:27:51.987540 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/2b5843e3-84ff-4174-bd4a-f67ba2a3a238-scripts\") pod \"cinder-api-0\" (UID: \"2b5843e3-84ff-4174-bd4a-f67ba2a3a238\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:27:51 crc kubenswrapper[5052]: I1203 12:27:51.987639 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/2b5843e3-84ff-4174-bd4a-f67ba2a3a238-config-data-custom\") pod \"cinder-api-0\" (UID: \"2b5843e3-84ff-4174-bd4a-f67ba2a3a238\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:27:51 crc kubenswrapper[5052]: I1203 12:27:51.987715 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/71e0e74b-5d01-467f-afb2-7e55d9d58297-combined-ca-bundle\") pod \"glance-default-external-api-0\" (UID: \"71e0e74b-5d01-467f-afb2-7e55d9d58297\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:27:51 crc kubenswrapper[5052]: I1203 12:27:51.987901 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z74ch\" (UniqueName: \"kubernetes.io/projected/71e0e74b-5d01-467f-afb2-7e55d9d58297-kube-api-access-z74ch\") pod \"glance-default-external-api-0\" (UID: \"71e0e74b-5d01-467f-afb2-7e55d9d58297\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:27:51 crc kubenswrapper[5052]: I1203 12:27:51.993763 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/2b5843e3-84ff-4174-bd4a-f67ba2a3a238-config-data-custom\") pod \"cinder-api-0\" (UID: \"2b5843e3-84ff-4174-bd4a-f67ba2a3a238\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:27:51 crc kubenswrapper[5052]: I1203 12:27:51.993850 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/2b5843e3-84ff-4174-bd4a-f67ba2a3a238-config-data\") pod \"cinder-api-0\" (UID: \"2b5843e3-84ff-4174-bd4a-f67ba2a3a238\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:27:51 crc kubenswrapper[5052]: I1203 12:27:51.996786 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/2b5843e3-84ff-4174-bd4a-f67ba2a3a238-scripts\") pod \"cinder-api-0\" (UID: \"2b5843e3-84ff-4174-bd4a-f67ba2a3a238\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:27:52 crc kubenswrapper[5052]: I1203 12:27:52.004132 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2b5843e3-84ff-4174-bd4a-f67ba2a3a238-combined-ca-bundle\") pod \"cinder-api-0\" (UID: \"2b5843e3-84ff-4174-bd4a-f67ba2a3a238\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:27:52 crc kubenswrapper[5052]: I1203 12:27:52.007074 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-mjwfk\" (UniqueName: \"kubernetes.io/projected/2b5843e3-84ff-4174-bd4a-f67ba2a3a238-kube-api-access-mjwfk\") pod \"cinder-api-0\" (UID: \"2b5843e3-84ff-4174-bd4a-f67ba2a3a238\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:27:52 crc kubenswrapper[5052]: I1203 12:27:52.089999 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/71e0e74b-5d01-467f-afb2-7e55d9d58297-httpd-run\") pod \"glance-default-external-api-0\" (UID: \"71e0e74b-5d01-467f-afb2-7e55d9d58297\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:27:52 crc kubenswrapper[5052]: I1203 12:27:52.090104 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/71e0e74b-5d01-467f-afb2-7e55d9d58297-combined-ca-bundle\") pod \"glance-default-external-api-0\" (UID: \"71e0e74b-5d01-467f-afb2-7e55d9d58297\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:27:52 crc kubenswrapper[5052]: I1203 12:27:52.090183 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-z74ch\" (UniqueName: \"kubernetes.io/projected/71e0e74b-5d01-467f-afb2-7e55d9d58297-kube-api-access-z74ch\") pod \"glance-default-external-api-0\" (UID: \"71e0e74b-5d01-467f-afb2-7e55d9d58297\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:27:52 crc kubenswrapper[5052]: I1203 12:27:52.090208 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/71e0e74b-5d01-467f-afb2-7e55d9d58297-scripts\") pod \"glance-default-external-api-0\" (UID: \"71e0e74b-5d01-467f-afb2-7e55d9d58297\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:27:52 crc kubenswrapper[5052]: I1203 12:27:52.090231 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/71e0e74b-5d01-467f-afb2-7e55d9d58297-logs\") pod \"glance-default-external-api-0\" (UID: \"71e0e74b-5d01-467f-afb2-7e55d9d58297\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:27:52 crc kubenswrapper[5052]: I1203 12:27:52.090283 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage07-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage07-crc\") pod \"glance-default-external-api-0\" (UID: \"71e0e74b-5d01-467f-afb2-7e55d9d58297\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:27:52 crc kubenswrapper[5052]: I1203 12:27:52.090320 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/71e0e74b-5d01-467f-afb2-7e55d9d58297-config-data\") pod \"glance-default-external-api-0\" (UID: \"71e0e74b-5d01-467f-afb2-7e55d9d58297\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:27:52 crc kubenswrapper[5052]: I1203 12:27:52.090632 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/71e0e74b-5d01-467f-afb2-7e55d9d58297-httpd-run\") pod \"glance-default-external-api-0\" (UID: \"71e0e74b-5d01-467f-afb2-7e55d9d58297\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:27:52 crc kubenswrapper[5052]: I1203 12:27:52.090972 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/71e0e74b-5d01-467f-afb2-7e55d9d58297-logs\") pod \"glance-default-external-api-0\" (UID: \"71e0e74b-5d01-467f-afb2-7e55d9d58297\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:27:52 crc kubenswrapper[5052]: I1203 12:27:52.091291 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:27:52 crc kubenswrapper[5052]: I1203 12:27:52.091315 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage07-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage07-crc\") pod \"glance-default-external-api-0\" (UID: \"71e0e74b-5d01-467f-afb2-7e55d9d58297\") device mount path \"/mnt/openstack/pv07\"" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:27:52 crc kubenswrapper[5052]: I1203 12:27:52.094162 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/71e0e74b-5d01-467f-afb2-7e55d9d58297-scripts\") pod \"glance-default-external-api-0\" (UID: \"71e0e74b-5d01-467f-afb2-7e55d9d58297\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:27:52 crc kubenswrapper[5052]: I1203 12:27:52.097325 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/71e0e74b-5d01-467f-afb2-7e55d9d58297-combined-ca-bundle\") pod \"glance-default-external-api-0\" (UID: \"71e0e74b-5d01-467f-afb2-7e55d9d58297\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:27:52 crc kubenswrapper[5052]: I1203 12:27:52.102115 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/71e0e74b-5d01-467f-afb2-7e55d9d58297-config-data\") pod \"glance-default-external-api-0\" (UID: \"71e0e74b-5d01-467f-afb2-7e55d9d58297\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:27:52 crc kubenswrapper[5052]: I1203 12:27:52.112638 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-z74ch\" (UniqueName: \"kubernetes.io/projected/71e0e74b-5d01-467f-afb2-7e55d9d58297-kube-api-access-z74ch\") pod \"glance-default-external-api-0\" (UID: \"71e0e74b-5d01-467f-afb2-7e55d9d58297\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:27:52 crc kubenswrapper[5052]: I1203 12:27:52.138683 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage07-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage07-crc\") pod \"glance-default-external-api-0\" (UID: \"71e0e74b-5d01-467f-afb2-7e55d9d58297\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:27:52 crc kubenswrapper[5052]: I1203 12:27:52.171988 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:27:53 crc kubenswrapper[5052]: I1203 12:27:52.329252 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/placement-84d4fb48db-blkmf" podUID="3b116fe9-1f40-4614-9381-7f7964fbdcb4" containerName="placement-log" probeResult="failure" output="Get \"http://10.217.1.251:8778/\": read tcp 10.217.0.2:37656->10.217.1.251:8778: read: connection reset by peer" Dec 03 12:27:53 crc kubenswrapper[5052]: I1203 12:27:52.329253 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/placement-84d4fb48db-blkmf" podUID="3b116fe9-1f40-4614-9381-7f7964fbdcb4" containerName="placement-api" probeResult="failure" output="Get \"http://10.217.1.251:8778/\": read tcp 10.217.0.2:37652->10.217.1.251:8778: read: connection reset by peer" Dec 03 12:27:53 crc kubenswrapper[5052]: I1203 12:27:52.587084 5052 generic.go:334] "Generic (PLEG): container finished" podID="3b116fe9-1f40-4614-9381-7f7964fbdcb4" containerID="35babc941d9382068ca3592fefa32271466b9c548b0f0eeab1a1fc57b7318180" exitCode=0 Dec 03 12:27:53 crc kubenswrapper[5052]: I1203 12:27:52.587148 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-84d4fb48db-blkmf" event={"ID":"3b116fe9-1f40-4614-9381-7f7964fbdcb4","Type":"ContainerDied","Data":"35babc941d9382068ca3592fefa32271466b9c548b0f0eeab1a1fc57b7318180"} Dec 03 12:27:53 crc kubenswrapper[5052]: I1203 12:27:52.597040 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"9a71429c-9d59-4863-8949-dcb4f79a35e2","Type":"ContainerStarted","Data":"59b11d0a23a20a97f6d34709bb5f894ff76b50c46bb80978918201b3b72f674c"} Dec 03 12:27:53 crc kubenswrapper[5052]: I1203 12:27:52.597083 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"9a71429c-9d59-4863-8949-dcb4f79a35e2","Type":"ContainerStarted","Data":"707deedabc8432d6991a028d80075701f8f2bd14b0998f6a88d8ce1c2703f42b"} Dec 03 12:27:53 crc kubenswrapper[5052]: I1203 12:27:52.597095 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"9a71429c-9d59-4863-8949-dcb4f79a35e2","Type":"ContainerStarted","Data":"bdab1d1a8f6c5c7a6c89c49bf44da7d62eab1cd075defa615ab51c3d50e3986e"} Dec 03 12:27:53 crc kubenswrapper[5052]: I1203 12:27:52.597105 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"9a71429c-9d59-4863-8949-dcb4f79a35e2","Type":"ContainerStarted","Data":"a6e47b8819de3aa2c92fe2ccff5f26450ecf089c945a22ec1bb902b9689f2e90"} Dec 03 12:27:53 crc kubenswrapper[5052]: I1203 12:27:52.621220 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 12:27:53 crc kubenswrapper[5052]: I1203 12:27:52.640946 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/swift-storage-0" podStartSLOduration=8.640927102 podStartE2EDuration="8.640927102s" podCreationTimestamp="2025-12-03 12:27:44 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:27:52.636175627 +0000 UTC m=+4063.224417411" watchObservedRunningTime="2025-12-03 12:27:52.640927102 +0000 UTC m=+4063.229168866" Dec 03 12:27:53 crc kubenswrapper[5052]: I1203 12:27:52.834859 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-7f9fd59985-48h54"] Dec 03 12:27:53 crc kubenswrapper[5052]: I1203 12:27:52.837015 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-7f9fd59985-48h54" Dec 03 12:27:53 crc kubenswrapper[5052]: I1203 12:27:52.862690 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-7f9fd59985-48h54"] Dec 03 12:27:53 crc kubenswrapper[5052]: I1203 12:27:52.913783 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qrf7l\" (UniqueName: \"kubernetes.io/projected/0f665fa3-6863-43d7-a43c-7e9cf3975345-kube-api-access-qrf7l\") pod \"dnsmasq-dnsmasq-7f9fd59985-48h54\" (UID: \"0f665fa3-6863-43d7-a43c-7e9cf3975345\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-7f9fd59985-48h54" Dec 03 12:27:53 crc kubenswrapper[5052]: I1203 12:27:52.913850 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0f665fa3-6863-43d7-a43c-7e9cf3975345-config\") pod \"dnsmasq-dnsmasq-7f9fd59985-48h54\" (UID: \"0f665fa3-6863-43d7-a43c-7e9cf3975345\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-7f9fd59985-48h54" Dec 03 12:27:53 crc kubenswrapper[5052]: I1203 12:27:52.913934 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dns-swift-storage-0\" (UniqueName: \"kubernetes.io/configmap/0f665fa3-6863-43d7-a43c-7e9cf3975345-dns-swift-storage-0\") pod \"dnsmasq-dnsmasq-7f9fd59985-48h54\" (UID: \"0f665fa3-6863-43d7-a43c-7e9cf3975345\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-7f9fd59985-48h54" Dec 03 12:27:53 crc kubenswrapper[5052]: I1203 12:27:52.914040 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/0f665fa3-6863-43d7-a43c-7e9cf3975345-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-7f9fd59985-48h54\" (UID: \"0f665fa3-6863-43d7-a43c-7e9cf3975345\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-7f9fd59985-48h54" Dec 03 12:27:53 crc kubenswrapper[5052]: I1203 12:27:53.015415 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-qrf7l\" (UniqueName: \"kubernetes.io/projected/0f665fa3-6863-43d7-a43c-7e9cf3975345-kube-api-access-qrf7l\") pod \"dnsmasq-dnsmasq-7f9fd59985-48h54\" (UID: \"0f665fa3-6863-43d7-a43c-7e9cf3975345\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-7f9fd59985-48h54" Dec 03 12:27:53 crc kubenswrapper[5052]: I1203 12:27:53.015482 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0f665fa3-6863-43d7-a43c-7e9cf3975345-config\") pod \"dnsmasq-dnsmasq-7f9fd59985-48h54\" (UID: \"0f665fa3-6863-43d7-a43c-7e9cf3975345\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-7f9fd59985-48h54" Dec 03 12:27:53 crc kubenswrapper[5052]: I1203 12:27:53.015609 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"dns-swift-storage-0\" (UniqueName: \"kubernetes.io/configmap/0f665fa3-6863-43d7-a43c-7e9cf3975345-dns-swift-storage-0\") pod \"dnsmasq-dnsmasq-7f9fd59985-48h54\" (UID: \"0f665fa3-6863-43d7-a43c-7e9cf3975345\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-7f9fd59985-48h54" Dec 03 12:27:53 crc kubenswrapper[5052]: I1203 12:27:53.015650 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/0f665fa3-6863-43d7-a43c-7e9cf3975345-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-7f9fd59985-48h54\" (UID: \"0f665fa3-6863-43d7-a43c-7e9cf3975345\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-7f9fd59985-48h54" Dec 03 12:27:53 crc kubenswrapper[5052]: I1203 12:27:53.016764 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/0f665fa3-6863-43d7-a43c-7e9cf3975345-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-7f9fd59985-48h54\" (UID: \"0f665fa3-6863-43d7-a43c-7e9cf3975345\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-7f9fd59985-48h54" Dec 03 12:27:53 crc kubenswrapper[5052]: I1203 12:27:53.019539 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0f665fa3-6863-43d7-a43c-7e9cf3975345-config\") pod \"dnsmasq-dnsmasq-7f9fd59985-48h54\" (UID: \"0f665fa3-6863-43d7-a43c-7e9cf3975345\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-7f9fd59985-48h54" Dec 03 12:27:53 crc kubenswrapper[5052]: I1203 12:27:53.020294 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"dns-swift-storage-0\" (UniqueName: \"kubernetes.io/configmap/0f665fa3-6863-43d7-a43c-7e9cf3975345-dns-swift-storage-0\") pod \"dnsmasq-dnsmasq-7f9fd59985-48h54\" (UID: \"0f665fa3-6863-43d7-a43c-7e9cf3975345\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-7f9fd59985-48h54" Dec 03 12:27:53 crc kubenswrapper[5052]: I1203 12:27:53.050752 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-qrf7l\" (UniqueName: \"kubernetes.io/projected/0f665fa3-6863-43d7-a43c-7e9cf3975345-kube-api-access-qrf7l\") pod \"dnsmasq-dnsmasq-7f9fd59985-48h54\" (UID: \"0f665fa3-6863-43d7-a43c-7e9cf3975345\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-7f9fd59985-48h54" Dec 03 12:27:53 crc kubenswrapper[5052]: I1203 12:27:53.163588 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-7f9fd59985-48h54" Dec 03 12:27:53 crc kubenswrapper[5052]: I1203 12:27:53.205573 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/barbican-api-6d8b99877d-pf4c5" Dec 03 12:27:53 crc kubenswrapper[5052]: I1203 12:27:53.447728 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-84d4fb48db-blkmf" Dec 03 12:27:53 crc kubenswrapper[5052]: I1203 12:27:53.520682 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-7f9fd59985-48h54"] Dec 03 12:27:53 crc kubenswrapper[5052]: I1203 12:27:53.565150 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:27:53 crc kubenswrapper[5052]: I1203 12:27:53.566397 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/barbican-api-6d8b99877d-pf4c5" Dec 03 12:27:53 crc kubenswrapper[5052]: I1203 12:27:53.629863 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-7f9fd59985-48h54" event={"ID":"0f665fa3-6863-43d7-a43c-7e9cf3975345","Type":"ContainerStarted","Data":"365f3653c2568d37b362668e23549a4e914687211c12879b235adf576a5b6e58"} Dec 03 12:27:53 crc kubenswrapper[5052]: I1203 12:27:53.630521 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5v9n6\" (UniqueName: \"kubernetes.io/projected/3b116fe9-1f40-4614-9381-7f7964fbdcb4-kube-api-access-5v9n6\") pod \"3b116fe9-1f40-4614-9381-7f7964fbdcb4\" (UID: \"3b116fe9-1f40-4614-9381-7f7964fbdcb4\") " Dec 03 12:27:53 crc kubenswrapper[5052]: I1203 12:27:53.630583 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/3b116fe9-1f40-4614-9381-7f7964fbdcb4-logs\") pod \"3b116fe9-1f40-4614-9381-7f7964fbdcb4\" (UID: \"3b116fe9-1f40-4614-9381-7f7964fbdcb4\") " Dec 03 12:27:53 crc kubenswrapper[5052]: I1203 12:27:53.630615 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3b116fe9-1f40-4614-9381-7f7964fbdcb4-combined-ca-bundle\") pod \"3b116fe9-1f40-4614-9381-7f7964fbdcb4\" (UID: \"3b116fe9-1f40-4614-9381-7f7964fbdcb4\") " Dec 03 12:27:53 crc kubenswrapper[5052]: I1203 12:27:53.630654 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/3b116fe9-1f40-4614-9381-7f7964fbdcb4-scripts\") pod \"3b116fe9-1f40-4614-9381-7f7964fbdcb4\" (UID: \"3b116fe9-1f40-4614-9381-7f7964fbdcb4\") " Dec 03 12:27:53 crc kubenswrapper[5052]: I1203 12:27:53.630779 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/3b116fe9-1f40-4614-9381-7f7964fbdcb4-config-data\") pod \"3b116fe9-1f40-4614-9381-7f7964fbdcb4\" (UID: \"3b116fe9-1f40-4614-9381-7f7964fbdcb4\") " Dec 03 12:27:53 crc kubenswrapper[5052]: I1203 12:27:53.632788 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b116fe9-1f40-4614-9381-7f7964fbdcb4-logs" (OuterVolumeSpecName: "logs") pod "3b116fe9-1f40-4614-9381-7f7964fbdcb4" (UID: "3b116fe9-1f40-4614-9381-7f7964fbdcb4"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:27:53 crc kubenswrapper[5052]: I1203 12:27:53.634447 5052 generic.go:334] "Generic (PLEG): container finished" podID="6228bbc6-4c33-4ad8-ad26-440a8e6b441c" containerID="595a1417c6646739c93678d8740cd28a9eadb61887c7e437c16c9c0ee312a81e" exitCode=137 Dec 03 12:27:53 crc kubenswrapper[5052]: I1203 12:27:53.634670 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-worker-64d9f4d555-4v5kn" event={"ID":"6228bbc6-4c33-4ad8-ad26-440a8e6b441c","Type":"ContainerDied","Data":"595a1417c6646739c93678d8740cd28a9eadb61887c7e437c16c9c0ee312a81e"} Dec 03 12:27:53 crc kubenswrapper[5052]: I1203 12:27:53.636190 5052 generic.go:334] "Generic (PLEG): container finished" podID="51bb178b-072f-4a7a-928b-5a8845d536b7" containerID="849a9a922c59a61b5c664acc3061a8b4ea9ea6688c539bbfce5e6562aedbfcf0" exitCode=137 Dec 03 12:27:53 crc kubenswrapper[5052]: I1203 12:27:53.636245 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-6cdf7689bb-fscpc" event={"ID":"51bb178b-072f-4a7a-928b-5a8845d536b7","Type":"ContainerDied","Data":"849a9a922c59a61b5c664acc3061a8b4ea9ea6688c539bbfce5e6562aedbfcf0"} Dec 03 12:27:53 crc kubenswrapper[5052]: I1203 12:27:53.639098 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3b116fe9-1f40-4614-9381-7f7964fbdcb4-kube-api-access-5v9n6" (OuterVolumeSpecName: "kube-api-access-5v9n6") pod "3b116fe9-1f40-4614-9381-7f7964fbdcb4" (UID: "3b116fe9-1f40-4614-9381-7f7964fbdcb4"). InnerVolumeSpecName "kube-api-access-5v9n6". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:27:53 crc kubenswrapper[5052]: I1203 12:27:53.639413 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-84d4fb48db-blkmf" event={"ID":"3b116fe9-1f40-4614-9381-7f7964fbdcb4","Type":"ContainerDied","Data":"f571af532a2719a0d8c5301009b0f9c62c4a005df1b26f86307be43d5f5f5645"} Dec 03 12:27:53 crc kubenswrapper[5052]: I1203 12:27:53.639461 5052 scope.go:117] "RemoveContainer" containerID="35babc941d9382068ca3592fefa32271466b9c548b0f0eeab1a1fc57b7318180" Dec 03 12:27:53 crc kubenswrapper[5052]: I1203 12:27:53.639751 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-84d4fb48db-blkmf" Dec 03 12:27:53 crc kubenswrapper[5052]: I1203 12:27:53.649054 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"2b5843e3-84ff-4174-bd4a-f67ba2a3a238","Type":"ContainerStarted","Data":"667a1283a5ee29a96d6e7cde347d08e4fba330b96f0a08eb1f51d024fa8e92ed"} Dec 03 12:27:53 crc kubenswrapper[5052]: I1203 12:27:53.649102 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"2b5843e3-84ff-4174-bd4a-f67ba2a3a238","Type":"ContainerStarted","Data":"07dca5ea36e03e44babf3998118304eca164a251d9ef6d346c1411df5f02ad8c"} Dec 03 12:27:53 crc kubenswrapper[5052]: W1203 12:27:53.651828 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod71e0e74b_5d01_467f_afb2_7e55d9d58297.slice/crio-6c875eb0f775ff19c7b6e41484c54ec06ca19355ef5f39d9e1228da177fb721c WatchSource:0}: Error finding container 6c875eb0f775ff19c7b6e41484c54ec06ca19355ef5f39d9e1228da177fb721c: Status 404 returned error can't find the container with id 6c875eb0f775ff19c7b6e41484c54ec06ca19355ef5f39d9e1228da177fb721c Dec 03 12:27:53 crc kubenswrapper[5052]: I1203 12:27:53.654214 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-api-5d855df9cb-xdg6p"] Dec 03 12:27:53 crc kubenswrapper[5052]: I1203 12:27:53.654464 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/barbican-api-5d855df9cb-xdg6p" podUID="3a27c483-00fc-4b06-af6b-3c4b0ff8d0d1" containerName="barbican-api-log" containerID="cri-o://03ad0a39928a5dbe15709e4ca192498a258450f53a516b11f0805a2e1f446f33" gracePeriod=30 Dec 03 12:27:53 crc kubenswrapper[5052]: I1203 12:27:53.654611 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/barbican-api-5d855df9cb-xdg6p" podUID="3a27c483-00fc-4b06-af6b-3c4b0ff8d0d1" containerName="barbican-api" containerID="cri-o://16e0da8b4d5026dda5b45a3d5fe087062b021f9377ca4b9ad0a08e9f37208416" gracePeriod=30 Dec 03 12:27:53 crc kubenswrapper[5052]: I1203 12:27:53.662016 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3b116fe9-1f40-4614-9381-7f7964fbdcb4-scripts" (OuterVolumeSpecName: "scripts") pod "3b116fe9-1f40-4614-9381-7f7964fbdcb4" (UID: "3b116fe9-1f40-4614-9381-7f7964fbdcb4"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:27:53 crc kubenswrapper[5052]: I1203 12:27:53.696147 5052 scope.go:117] "RemoveContainer" containerID="c855f9d1a9983bf4428f3f423fd3f1abd57a52f10787a431f76087b5f92f6bd5" Dec 03 12:27:53 crc kubenswrapper[5052]: I1203 12:27:53.723489 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3b116fe9-1f40-4614-9381-7f7964fbdcb4-config-data" (OuterVolumeSpecName: "config-data") pod "3b116fe9-1f40-4614-9381-7f7964fbdcb4" (UID: "3b116fe9-1f40-4614-9381-7f7964fbdcb4"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:27:53 crc kubenswrapper[5052]: I1203 12:27:53.734713 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/3b116fe9-1f40-4614-9381-7f7964fbdcb4-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:53 crc kubenswrapper[5052]: I1203 12:27:53.734744 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-5v9n6\" (UniqueName: \"kubernetes.io/projected/3b116fe9-1f40-4614-9381-7f7964fbdcb4-kube-api-access-5v9n6\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:53 crc kubenswrapper[5052]: I1203 12:27:53.734757 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/3b116fe9-1f40-4614-9381-7f7964fbdcb4-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:53 crc kubenswrapper[5052]: I1203 12:27:53.734769 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/3b116fe9-1f40-4614-9381-7f7964fbdcb4-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:53 crc kubenswrapper[5052]: I1203 12:27:53.743280 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6b96dc78-b389-4764-8ba7-6a4c5e41e345" path="/var/lib/kubelet/pods/6b96dc78-b389-4764-8ba7-6a4c5e41e345/volumes" Dec 03 12:27:53 crc kubenswrapper[5052]: I1203 12:27:53.745624 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3b116fe9-1f40-4614-9381-7f7964fbdcb4-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "3b116fe9-1f40-4614-9381-7f7964fbdcb4" (UID: "3b116fe9-1f40-4614-9381-7f7964fbdcb4"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:27:53 crc kubenswrapper[5052]: I1203 12:27:53.749479 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aa94b30e-0028-45a0-88aa-b5db5fc2f8fd" path="/var/lib/kubelet/pods/aa94b30e-0028-45a0-88aa-b5db5fc2f8fd/volumes" Dec 03 12:27:53 crc kubenswrapper[5052]: I1203 12:27:53.824226 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-6cdf7689bb-fscpc" Dec 03 12:27:53 crc kubenswrapper[5052]: I1203 12:27:53.836714 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3b116fe9-1f40-4614-9381-7f7964fbdcb4-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:53 crc kubenswrapper[5052]: I1203 12:27:53.939219 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/51bb178b-072f-4a7a-928b-5a8845d536b7-scripts\") pod \"51bb178b-072f-4a7a-928b-5a8845d536b7\" (UID: \"51bb178b-072f-4a7a-928b-5a8845d536b7\") " Dec 03 12:27:53 crc kubenswrapper[5052]: I1203 12:27:53.939335 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/51bb178b-072f-4a7a-928b-5a8845d536b7-combined-ca-bundle\") pod \"51bb178b-072f-4a7a-928b-5a8845d536b7\" (UID: \"51bb178b-072f-4a7a-928b-5a8845d536b7\") " Dec 03 12:27:53 crc kubenswrapper[5052]: I1203 12:27:53.939375 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/51bb178b-072f-4a7a-928b-5a8845d536b7-config-data\") pod \"51bb178b-072f-4a7a-928b-5a8845d536b7\" (UID: \"51bb178b-072f-4a7a-928b-5a8845d536b7\") " Dec 03 12:27:53 crc kubenswrapper[5052]: I1203 12:27:53.939440 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/51bb178b-072f-4a7a-928b-5a8845d536b7-logs\") pod \"51bb178b-072f-4a7a-928b-5a8845d536b7\" (UID: \"51bb178b-072f-4a7a-928b-5a8845d536b7\") " Dec 03 12:27:53 crc kubenswrapper[5052]: I1203 12:27:53.939584 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kkm8f\" (UniqueName: \"kubernetes.io/projected/51bb178b-072f-4a7a-928b-5a8845d536b7-kube-api-access-kkm8f\") pod \"51bb178b-072f-4a7a-928b-5a8845d536b7\" (UID: \"51bb178b-072f-4a7a-928b-5a8845d536b7\") " Dec 03 12:27:53 crc kubenswrapper[5052]: I1203 12:27:53.945722 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/51bb178b-072f-4a7a-928b-5a8845d536b7-kube-api-access-kkm8f" (OuterVolumeSpecName: "kube-api-access-kkm8f") pod "51bb178b-072f-4a7a-928b-5a8845d536b7" (UID: "51bb178b-072f-4a7a-928b-5a8845d536b7"). InnerVolumeSpecName "kube-api-access-kkm8f". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:27:53 crc kubenswrapper[5052]: I1203 12:27:53.946650 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/51bb178b-072f-4a7a-928b-5a8845d536b7-logs" (OuterVolumeSpecName: "logs") pod "51bb178b-072f-4a7a-928b-5a8845d536b7" (UID: "51bb178b-072f-4a7a-928b-5a8845d536b7"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:27:53 crc kubenswrapper[5052]: I1203 12:27:53.967832 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/51bb178b-072f-4a7a-928b-5a8845d536b7-scripts" (OuterVolumeSpecName: "scripts") pod "51bb178b-072f-4a7a-928b-5a8845d536b7" (UID: "51bb178b-072f-4a7a-928b-5a8845d536b7"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:27:54 crc kubenswrapper[5052]: I1203 12:27:54.042980 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/51bb178b-072f-4a7a-928b-5a8845d536b7-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:54 crc kubenswrapper[5052]: I1203 12:27:54.043200 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/51bb178b-072f-4a7a-928b-5a8845d536b7-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:54 crc kubenswrapper[5052]: I1203 12:27:54.043210 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-kkm8f\" (UniqueName: \"kubernetes.io/projected/51bb178b-072f-4a7a-928b-5a8845d536b7-kube-api-access-kkm8f\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:54 crc kubenswrapper[5052]: I1203 12:27:54.045087 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/51bb178b-072f-4a7a-928b-5a8845d536b7-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "51bb178b-072f-4a7a-928b-5a8845d536b7" (UID: "51bb178b-072f-4a7a-928b-5a8845d536b7"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:27:54 crc kubenswrapper[5052]: I1203 12:27:54.125024 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/51bb178b-072f-4a7a-928b-5a8845d536b7-config-data" (OuterVolumeSpecName: "config-data") pod "51bb178b-072f-4a7a-928b-5a8845d536b7" (UID: "51bb178b-072f-4a7a-928b-5a8845d536b7"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:27:54 crc kubenswrapper[5052]: I1203 12:27:54.145472 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/51bb178b-072f-4a7a-928b-5a8845d536b7-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:54 crc kubenswrapper[5052]: I1203 12:27:54.145498 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/51bb178b-072f-4a7a-928b-5a8845d536b7-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:54 crc kubenswrapper[5052]: I1203 12:27:54.226005 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/placement-84d4fb48db-blkmf"] Dec 03 12:27:54 crc kubenswrapper[5052]: I1203 12:27:54.227437 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-worker-64d9f4d555-4v5kn" Dec 03 12:27:54 crc kubenswrapper[5052]: I1203 12:27:54.272108 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/placement-84d4fb48db-blkmf"] Dec 03 12:27:54 crc kubenswrapper[5052]: I1203 12:27:54.322859 5052 pod_container_manager_linux.go:210] "Failed to delete cgroup paths" cgroupName=["kubepods","besteffort","pod98d0abd5-a20b-45e3-aa30-34f038b2fb3d"] err="unable to destroy cgroup paths for cgroup [kubepods besteffort pod98d0abd5-a20b-45e3-aa30-34f038b2fb3d] : Timed out while waiting for systemd to remove kubepods-besteffort-pod98d0abd5_a20b_45e3_aa30_34f038b2fb3d.slice" Dec 03 12:27:54 crc kubenswrapper[5052]: E1203 12:27:54.322919 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to delete cgroup paths for [kubepods besteffort pod98d0abd5-a20b-45e3-aa30-34f038b2fb3d] : unable to destroy cgroup paths for cgroup [kubepods besteffort pod98d0abd5-a20b-45e3-aa30-34f038b2fb3d] : Timed out while waiting for systemd to remove kubepods-besteffort-pod98d0abd5_a20b_45e3_aa30_34f038b2fb3d.slice" pod="openstack-kuttl-tests/barbican-keystone-listener-7c58f6698b-95l8v" podUID="98d0abd5-a20b-45e3-aa30-34f038b2fb3d" Dec 03 12:27:54 crc kubenswrapper[5052]: I1203 12:27:54.347326 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-keystone-listener-6df6465ddd-km2kt" Dec 03 12:27:54 crc kubenswrapper[5052]: I1203 12:27:54.361480 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/6228bbc6-4c33-4ad8-ad26-440a8e6b441c-logs\") pod \"6228bbc6-4c33-4ad8-ad26-440a8e6b441c\" (UID: \"6228bbc6-4c33-4ad8-ad26-440a8e6b441c\") " Dec 03 12:27:54 crc kubenswrapper[5052]: I1203 12:27:54.361693 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6228bbc6-4c33-4ad8-ad26-440a8e6b441c-combined-ca-bundle\") pod \"6228bbc6-4c33-4ad8-ad26-440a8e6b441c\" (UID: \"6228bbc6-4c33-4ad8-ad26-440a8e6b441c\") " Dec 03 12:27:54 crc kubenswrapper[5052]: I1203 12:27:54.361747 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z9bxb\" (UniqueName: \"kubernetes.io/projected/6228bbc6-4c33-4ad8-ad26-440a8e6b441c-kube-api-access-z9bxb\") pod \"6228bbc6-4c33-4ad8-ad26-440a8e6b441c\" (UID: \"6228bbc6-4c33-4ad8-ad26-440a8e6b441c\") " Dec 03 12:27:54 crc kubenswrapper[5052]: I1203 12:27:54.361837 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/6228bbc6-4c33-4ad8-ad26-440a8e6b441c-config-data-custom\") pod \"6228bbc6-4c33-4ad8-ad26-440a8e6b441c\" (UID: \"6228bbc6-4c33-4ad8-ad26-440a8e6b441c\") " Dec 03 12:27:54 crc kubenswrapper[5052]: I1203 12:27:54.361857 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6228bbc6-4c33-4ad8-ad26-440a8e6b441c-config-data\") pod \"6228bbc6-4c33-4ad8-ad26-440a8e6b441c\" (UID: \"6228bbc6-4c33-4ad8-ad26-440a8e6b441c\") " Dec 03 12:27:54 crc kubenswrapper[5052]: I1203 12:27:54.362134 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6228bbc6-4c33-4ad8-ad26-440a8e6b441c-logs" (OuterVolumeSpecName: "logs") pod "6228bbc6-4c33-4ad8-ad26-440a8e6b441c" (UID: "6228bbc6-4c33-4ad8-ad26-440a8e6b441c"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:27:54 crc kubenswrapper[5052]: I1203 12:27:54.363310 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/6228bbc6-4c33-4ad8-ad26-440a8e6b441c-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:54 crc kubenswrapper[5052]: I1203 12:27:54.372572 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6228bbc6-4c33-4ad8-ad26-440a8e6b441c-kube-api-access-z9bxb" (OuterVolumeSpecName: "kube-api-access-z9bxb") pod "6228bbc6-4c33-4ad8-ad26-440a8e6b441c" (UID: "6228bbc6-4c33-4ad8-ad26-440a8e6b441c"). InnerVolumeSpecName "kube-api-access-z9bxb". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:27:54 crc kubenswrapper[5052]: I1203 12:27:54.383386 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6228bbc6-4c33-4ad8-ad26-440a8e6b441c-config-data-custom" (OuterVolumeSpecName: "config-data-custom") pod "6228bbc6-4c33-4ad8-ad26-440a8e6b441c" (UID: "6228bbc6-4c33-4ad8-ad26-440a8e6b441c"). InnerVolumeSpecName "config-data-custom". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:27:54 crc kubenswrapper[5052]: I1203 12:27:54.403909 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6228bbc6-4c33-4ad8-ad26-440a8e6b441c-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "6228bbc6-4c33-4ad8-ad26-440a8e6b441c" (UID: "6228bbc6-4c33-4ad8-ad26-440a8e6b441c"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:27:54 crc kubenswrapper[5052]: I1203 12:27:54.418960 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6228bbc6-4c33-4ad8-ad26-440a8e6b441c-config-data" (OuterVolumeSpecName: "config-data") pod "6228bbc6-4c33-4ad8-ad26-440a8e6b441c" (UID: "6228bbc6-4c33-4ad8-ad26-440a8e6b441c"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:27:54 crc kubenswrapper[5052]: I1203 12:27:54.463977 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/7f4dfec9-0aa8-437c-9b8d-771b30a93af9-combined-ca-bundle\") pod \"7f4dfec9-0aa8-437c-9b8d-771b30a93af9\" (UID: \"7f4dfec9-0aa8-437c-9b8d-771b30a93af9\") " Dec 03 12:27:54 crc kubenswrapper[5052]: I1203 12:27:54.464066 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v76qr\" (UniqueName: \"kubernetes.io/projected/7f4dfec9-0aa8-437c-9b8d-771b30a93af9-kube-api-access-v76qr\") pod \"7f4dfec9-0aa8-437c-9b8d-771b30a93af9\" (UID: \"7f4dfec9-0aa8-437c-9b8d-771b30a93af9\") " Dec 03 12:27:54 crc kubenswrapper[5052]: I1203 12:27:54.464143 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/7f4dfec9-0aa8-437c-9b8d-771b30a93af9-config-data-custom\") pod \"7f4dfec9-0aa8-437c-9b8d-771b30a93af9\" (UID: \"7f4dfec9-0aa8-437c-9b8d-771b30a93af9\") " Dec 03 12:27:54 crc kubenswrapper[5052]: I1203 12:27:54.464245 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/7f4dfec9-0aa8-437c-9b8d-771b30a93af9-config-data\") pod \"7f4dfec9-0aa8-437c-9b8d-771b30a93af9\" (UID: \"7f4dfec9-0aa8-437c-9b8d-771b30a93af9\") " Dec 03 12:27:54 crc kubenswrapper[5052]: I1203 12:27:54.464305 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/7f4dfec9-0aa8-437c-9b8d-771b30a93af9-logs\") pod \"7f4dfec9-0aa8-437c-9b8d-771b30a93af9\" (UID: \"7f4dfec9-0aa8-437c-9b8d-771b30a93af9\") " Dec 03 12:27:54 crc kubenswrapper[5052]: I1203 12:27:54.464788 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/6228bbc6-4c33-4ad8-ad26-440a8e6b441c-config-data-custom\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:54 crc kubenswrapper[5052]: I1203 12:27:54.464808 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6228bbc6-4c33-4ad8-ad26-440a8e6b441c-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:54 crc kubenswrapper[5052]: I1203 12:27:54.464817 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6228bbc6-4c33-4ad8-ad26-440a8e6b441c-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:54 crc kubenswrapper[5052]: I1203 12:27:54.464845 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-z9bxb\" (UniqueName: \"kubernetes.io/projected/6228bbc6-4c33-4ad8-ad26-440a8e6b441c-kube-api-access-z9bxb\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:54 crc kubenswrapper[5052]: I1203 12:27:54.465227 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f4dfec9-0aa8-437c-9b8d-771b30a93af9-logs" (OuterVolumeSpecName: "logs") pod "7f4dfec9-0aa8-437c-9b8d-771b30a93af9" (UID: "7f4dfec9-0aa8-437c-9b8d-771b30a93af9"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:27:54 crc kubenswrapper[5052]: I1203 12:27:54.468759 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7f4dfec9-0aa8-437c-9b8d-771b30a93af9-config-data-custom" (OuterVolumeSpecName: "config-data-custom") pod "7f4dfec9-0aa8-437c-9b8d-771b30a93af9" (UID: "7f4dfec9-0aa8-437c-9b8d-771b30a93af9"). InnerVolumeSpecName "config-data-custom". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:27:54 crc kubenswrapper[5052]: I1203 12:27:54.469607 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7f4dfec9-0aa8-437c-9b8d-771b30a93af9-kube-api-access-v76qr" (OuterVolumeSpecName: "kube-api-access-v76qr") pod "7f4dfec9-0aa8-437c-9b8d-771b30a93af9" (UID: "7f4dfec9-0aa8-437c-9b8d-771b30a93af9"). InnerVolumeSpecName "kube-api-access-v76qr". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:27:54 crc kubenswrapper[5052]: I1203 12:27:54.504032 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7f4dfec9-0aa8-437c-9b8d-771b30a93af9-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "7f4dfec9-0aa8-437c-9b8d-771b30a93af9" (UID: "7f4dfec9-0aa8-437c-9b8d-771b30a93af9"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:27:54 crc kubenswrapper[5052]: I1203 12:27:54.533826 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7f4dfec9-0aa8-437c-9b8d-771b30a93af9-config-data" (OuterVolumeSpecName: "config-data") pod "7f4dfec9-0aa8-437c-9b8d-771b30a93af9" (UID: "7f4dfec9-0aa8-437c-9b8d-771b30a93af9"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:27:54 crc kubenswrapper[5052]: I1203 12:27:54.566297 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/7f4dfec9-0aa8-437c-9b8d-771b30a93af9-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:54 crc kubenswrapper[5052]: I1203 12:27:54.566432 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/7f4dfec9-0aa8-437c-9b8d-771b30a93af9-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:54 crc kubenswrapper[5052]: I1203 12:27:54.566505 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/7f4dfec9-0aa8-437c-9b8d-771b30a93af9-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:54 crc kubenswrapper[5052]: I1203 12:27:54.566620 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-v76qr\" (UniqueName: \"kubernetes.io/projected/7f4dfec9-0aa8-437c-9b8d-771b30a93af9-kube-api-access-v76qr\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:54 crc kubenswrapper[5052]: I1203 12:27:54.566689 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/7f4dfec9-0aa8-437c-9b8d-771b30a93af9-config-data-custom\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:54 crc kubenswrapper[5052]: I1203 12:27:54.684479 5052 generic.go:334] "Generic (PLEG): container finished" podID="3a27c483-00fc-4b06-af6b-3c4b0ff8d0d1" containerID="03ad0a39928a5dbe15709e4ca192498a258450f53a516b11f0805a2e1f446f33" exitCode=143 Dec 03 12:27:54 crc kubenswrapper[5052]: I1203 12:27:54.684607 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-5d855df9cb-xdg6p" event={"ID":"3a27c483-00fc-4b06-af6b-3c4b0ff8d0d1","Type":"ContainerDied","Data":"03ad0a39928a5dbe15709e4ca192498a258450f53a516b11f0805a2e1f446f33"} Dec 03 12:27:54 crc kubenswrapper[5052]: I1203 12:27:54.688197 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-worker-64d9f4d555-4v5kn" event={"ID":"6228bbc6-4c33-4ad8-ad26-440a8e6b441c","Type":"ContainerDied","Data":"44946a83cf1f5a6cd01a57945285db24d218d9d49064b66a61eaf725fc91384e"} Dec 03 12:27:54 crc kubenswrapper[5052]: I1203 12:27:54.688258 5052 scope.go:117] "RemoveContainer" containerID="595a1417c6646739c93678d8740cd28a9eadb61887c7e437c16c9c0ee312a81e" Dec 03 12:27:54 crc kubenswrapper[5052]: I1203 12:27:54.688501 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-worker-64d9f4d555-4v5kn" Dec 03 12:27:54 crc kubenswrapper[5052]: I1203 12:27:54.697067 5052 generic.go:334] "Generic (PLEG): container finished" podID="7f4dfec9-0aa8-437c-9b8d-771b30a93af9" containerID="bc39c9a553475072254f96ed656dff329e085eb9d86080fd0b82411e0f0780a9" exitCode=137 Dec 03 12:27:54 crc kubenswrapper[5052]: I1203 12:27:54.697186 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-keystone-listener-6df6465ddd-km2kt" event={"ID":"7f4dfec9-0aa8-437c-9b8d-771b30a93af9","Type":"ContainerDied","Data":"bc39c9a553475072254f96ed656dff329e085eb9d86080fd0b82411e0f0780a9"} Dec 03 12:27:54 crc kubenswrapper[5052]: I1203 12:27:54.697224 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-keystone-listener-6df6465ddd-km2kt" event={"ID":"7f4dfec9-0aa8-437c-9b8d-771b30a93af9","Type":"ContainerDied","Data":"8991d70aa079bf5ccd5fd999ab882977681310442bca6d1449b9233386d7876a"} Dec 03 12:27:54 crc kubenswrapper[5052]: I1203 12:27:54.697353 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-keystone-listener-6df6465ddd-km2kt" Dec 03 12:27:54 crc kubenswrapper[5052]: I1203 12:27:54.708995 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-6cdf7689bb-fscpc" Dec 03 12:27:54 crc kubenswrapper[5052]: I1203 12:27:54.709221 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-6cdf7689bb-fscpc" event={"ID":"51bb178b-072f-4a7a-928b-5a8845d536b7","Type":"ContainerDied","Data":"60a86a4c230f97868c413bde12086e18296cdc745808a7ffcb0c69014f0f5bd7"} Dec 03 12:27:54 crc kubenswrapper[5052]: I1203 12:27:54.738914 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"2b5843e3-84ff-4174-bd4a-f67ba2a3a238","Type":"ContainerStarted","Data":"e07ed2652a16aa998baf13477cacedfad92c750d43fb52c99642ffc63d38a6cf"} Dec 03 12:27:54 crc kubenswrapper[5052]: I1203 12:27:54.738970 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:27:54 crc kubenswrapper[5052]: I1203 12:27:54.740856 5052 scope.go:117] "RemoveContainer" containerID="cad85f08979cf86c1f2f2aa7c3637ac5ab17fe2a4099a131b06311d2a7a911ff" Dec 03 12:27:54 crc kubenswrapper[5052]: I1203 12:27:54.742139 5052 generic.go:334] "Generic (PLEG): container finished" podID="0f665fa3-6863-43d7-a43c-7e9cf3975345" containerID="b8429722e957a292c9a5cae7e6de3f9027900fdd6e4da3df96e4d483cfac94ec" exitCode=0 Dec 03 12:27:54 crc kubenswrapper[5052]: I1203 12:27:54.742228 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-7f9fd59985-48h54" event={"ID":"0f665fa3-6863-43d7-a43c-7e9cf3975345","Type":"ContainerDied","Data":"b8429722e957a292c9a5cae7e6de3f9027900fdd6e4da3df96e4d483cfac94ec"} Dec 03 12:27:54 crc kubenswrapper[5052]: I1203 12:27:54.757709 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-keystone-listener-7c58f6698b-95l8v" Dec 03 12:27:54 crc kubenswrapper[5052]: I1203 12:27:54.757716 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"71e0e74b-5d01-467f-afb2-7e55d9d58297","Type":"ContainerStarted","Data":"8bcc87cf614b2c81facd280057f18832930b2317f445a10147336b258adbe781"} Dec 03 12:27:54 crc kubenswrapper[5052]: I1203 12:27:54.757779 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"71e0e74b-5d01-467f-afb2-7e55d9d58297","Type":"ContainerStarted","Data":"6c875eb0f775ff19c7b6e41484c54ec06ca19355ef5f39d9e1228da177fb721c"} Dec 03 12:27:54 crc kubenswrapper[5052]: I1203 12:27:54.763584 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-worker-64d9f4d555-4v5kn"] Dec 03 12:27:54 crc kubenswrapper[5052]: I1203 12:27:54.790327 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/barbican-worker-64d9f4d555-4v5kn"] Dec 03 12:27:54 crc kubenswrapper[5052]: I1203 12:27:54.807650 5052 scope.go:117] "RemoveContainer" containerID="bc39c9a553475072254f96ed656dff329e085eb9d86080fd0b82411e0f0780a9" Dec 03 12:27:54 crc kubenswrapper[5052]: I1203 12:27:54.812443 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-keystone-listener-6df6465ddd-km2kt"] Dec 03 12:27:54 crc kubenswrapper[5052]: I1203 12:27:54.835953 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/barbican-keystone-listener-6df6465ddd-km2kt"] Dec 03 12:27:54 crc kubenswrapper[5052]: I1203 12:27:54.846093 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/cinder-api-0" podStartSLOduration=3.846075672 podStartE2EDuration="3.846075672s" podCreationTimestamp="2025-12-03 12:27:51 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:27:54.798566886 +0000 UTC m=+4065.386808650" watchObservedRunningTime="2025-12-03 12:27:54.846075672 +0000 UTC m=+4065.434317436" Dec 03 12:27:54 crc kubenswrapper[5052]: I1203 12:27:54.861904 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/placement-6cdf7689bb-fscpc"] Dec 03 12:27:54 crc kubenswrapper[5052]: I1203 12:27:54.862078 5052 scope.go:117] "RemoveContainer" containerID="54f3fa89ac27b88281c422bf43fdb03928a159ddffa85177a0997bcb486d4eee" Dec 03 12:27:54 crc kubenswrapper[5052]: I1203 12:27:54.872252 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/placement-6cdf7689bb-fscpc"] Dec 03 12:27:54 crc kubenswrapper[5052]: I1203 12:27:54.884167 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-keystone-listener-7c58f6698b-95l8v"] Dec 03 12:27:54 crc kubenswrapper[5052]: I1203 12:27:54.891701 5052 scope.go:117] "RemoveContainer" containerID="bc39c9a553475072254f96ed656dff329e085eb9d86080fd0b82411e0f0780a9" Dec 03 12:27:54 crc kubenswrapper[5052]: E1203 12:27:54.892214 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bc39c9a553475072254f96ed656dff329e085eb9d86080fd0b82411e0f0780a9\": container with ID starting with bc39c9a553475072254f96ed656dff329e085eb9d86080fd0b82411e0f0780a9 not found: ID does not exist" containerID="bc39c9a553475072254f96ed656dff329e085eb9d86080fd0b82411e0f0780a9" Dec 03 12:27:54 crc kubenswrapper[5052]: I1203 12:27:54.892252 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bc39c9a553475072254f96ed656dff329e085eb9d86080fd0b82411e0f0780a9"} err="failed to get container status \"bc39c9a553475072254f96ed656dff329e085eb9d86080fd0b82411e0f0780a9\": rpc error: code = NotFound desc = could not find container \"bc39c9a553475072254f96ed656dff329e085eb9d86080fd0b82411e0f0780a9\": container with ID starting with bc39c9a553475072254f96ed656dff329e085eb9d86080fd0b82411e0f0780a9 not found: ID does not exist" Dec 03 12:27:54 crc kubenswrapper[5052]: I1203 12:27:54.892277 5052 scope.go:117] "RemoveContainer" containerID="54f3fa89ac27b88281c422bf43fdb03928a159ddffa85177a0997bcb486d4eee" Dec 03 12:27:54 crc kubenswrapper[5052]: I1203 12:27:54.897250 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/barbican-keystone-listener-7c58f6698b-95l8v"] Dec 03 12:27:54 crc kubenswrapper[5052]: E1203 12:27:54.908719 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"54f3fa89ac27b88281c422bf43fdb03928a159ddffa85177a0997bcb486d4eee\": container with ID starting with 54f3fa89ac27b88281c422bf43fdb03928a159ddffa85177a0997bcb486d4eee not found: ID does not exist" containerID="54f3fa89ac27b88281c422bf43fdb03928a159ddffa85177a0997bcb486d4eee" Dec 03 12:27:54 crc kubenswrapper[5052]: I1203 12:27:54.908773 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"54f3fa89ac27b88281c422bf43fdb03928a159ddffa85177a0997bcb486d4eee"} err="failed to get container status \"54f3fa89ac27b88281c422bf43fdb03928a159ddffa85177a0997bcb486d4eee\": rpc error: code = NotFound desc = could not find container \"54f3fa89ac27b88281c422bf43fdb03928a159ddffa85177a0997bcb486d4eee\": container with ID starting with 54f3fa89ac27b88281c422bf43fdb03928a159ddffa85177a0997bcb486d4eee not found: ID does not exist" Dec 03 12:27:54 crc kubenswrapper[5052]: I1203 12:27:54.908808 5052 scope.go:117] "RemoveContainer" containerID="849a9a922c59a61b5c664acc3061a8b4ea9ea6688c539bbfce5e6562aedbfcf0" Dec 03 12:27:54 crc kubenswrapper[5052]: I1203 12:27:54.934604 5052 scope.go:117] "RemoveContainer" containerID="3426f46bd5e0b4a541ed02f94900e4a755d8e523a368d5a2ed4870a257b7116f" Dec 03 12:27:55 crc kubenswrapper[5052]: I1203 12:27:55.193356 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:27:55 crc kubenswrapper[5052]: I1203 12:27:55.696554 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3b116fe9-1f40-4614-9381-7f7964fbdcb4" path="/var/lib/kubelet/pods/3b116fe9-1f40-4614-9381-7f7964fbdcb4/volumes" Dec 03 12:27:55 crc kubenswrapper[5052]: I1203 12:27:55.697270 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="51bb178b-072f-4a7a-928b-5a8845d536b7" path="/var/lib/kubelet/pods/51bb178b-072f-4a7a-928b-5a8845d536b7/volumes" Dec 03 12:27:55 crc kubenswrapper[5052]: I1203 12:27:55.698001 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6228bbc6-4c33-4ad8-ad26-440a8e6b441c" path="/var/lib/kubelet/pods/6228bbc6-4c33-4ad8-ad26-440a8e6b441c/volumes" Dec 03 12:27:55 crc kubenswrapper[5052]: I1203 12:27:55.699119 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7f4dfec9-0aa8-437c-9b8d-771b30a93af9" path="/var/lib/kubelet/pods/7f4dfec9-0aa8-437c-9b8d-771b30a93af9/volumes" Dec 03 12:27:55 crc kubenswrapper[5052]: I1203 12:27:55.699846 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="98d0abd5-a20b-45e3-aa30-34f038b2fb3d" path="/var/lib/kubelet/pods/98d0abd5-a20b-45e3-aa30-34f038b2fb3d/volumes" Dec 03 12:27:55 crc kubenswrapper[5052]: I1203 12:27:55.770806 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"71e0e74b-5d01-467f-afb2-7e55d9d58297","Type":"ContainerStarted","Data":"6e64026bbfc0cc36204875e424068283a0de98c7caf69823427bc926fa18e20d"} Dec 03 12:27:55 crc kubenswrapper[5052]: I1203 12:27:55.780237 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-7f9fd59985-48h54" event={"ID":"0f665fa3-6863-43d7-a43c-7e9cf3975345","Type":"ContainerStarted","Data":"93b52a7aa013e6c6f3027aaba01303c05198f39c6fc6938bc82fb3dc438f99a6"} Dec 03 12:27:55 crc kubenswrapper[5052]: I1203 12:27:55.780554 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-7f9fd59985-48h54" Dec 03 12:27:55 crc kubenswrapper[5052]: I1203 12:27:55.792463 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/glance-default-external-api-0" podStartSLOduration=4.792440363 podStartE2EDuration="4.792440363s" podCreationTimestamp="2025-12-03 12:27:51 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:27:55.788284364 +0000 UTC m=+4066.376526128" watchObservedRunningTime="2025-12-03 12:27:55.792440363 +0000 UTC m=+4066.380682127" Dec 03 12:27:55 crc kubenswrapper[5052]: I1203 12:27:55.816498 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-7f9fd59985-48h54" podStartSLOduration=3.816480149 podStartE2EDuration="3.816480149s" podCreationTimestamp="2025-12-03 12:27:52 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:27:55.812832525 +0000 UTC m=+4066.401074309" watchObservedRunningTime="2025-12-03 12:27:55.816480149 +0000 UTC m=+4066.404721913" Dec 03 12:27:56 crc kubenswrapper[5052]: I1203 12:27:56.930236 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/barbican-api-5d855df9cb-xdg6p" podUID="3a27c483-00fc-4b06-af6b-3c4b0ff8d0d1" containerName="barbican-api-log" probeResult="failure" output="Get \"http://10.217.0.134:9311/healthcheck\": dial tcp 10.217.0.134:9311: connect: connection refused" Dec 03 12:27:56 crc kubenswrapper[5052]: I1203 12:27:56.930253 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/barbican-api-5d855df9cb-xdg6p" podUID="3a27c483-00fc-4b06-af6b-3c4b0ff8d0d1" containerName="barbican-api" probeResult="failure" output="Get \"http://10.217.0.134:9311/healthcheck\": dial tcp 10.217.0.134:9311: connect: connection refused" Dec 03 12:27:57 crc kubenswrapper[5052]: I1203 12:27:57.040892 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/kube-state-metrics-0"] Dec 03 12:27:57 crc kubenswrapper[5052]: I1203 12:27:57.041135 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/kube-state-metrics-0" podUID="61588a52-3ab5-45a4-a595-3b13bb4bcc28" containerName="kube-state-metrics" containerID="cri-o://7f4e8c9236258e1ee13e282db4d527b630572e15c3e43c16ee3932b425f0f4a6" gracePeriod=30 Dec 03 12:27:57 crc kubenswrapper[5052]: I1203 12:27:57.101099 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:27:57 crc kubenswrapper[5052]: I1203 12:27:57.101326 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="5d3294b3-d25c-4d8a-ba28-aaad7385693b" containerName="ceilometer-central-agent" containerID="cri-o://518e554556294e8e06ab1671e9f2b277371917fe0bafb7f6898e763ce48bb5e1" gracePeriod=30 Dec 03 12:27:57 crc kubenswrapper[5052]: I1203 12:27:57.101388 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="5d3294b3-d25c-4d8a-ba28-aaad7385693b" containerName="proxy-httpd" containerID="cri-o://7e8b797c3e63fa66aad0206d3411e34757b19c124c459e9c5d9dc7e3133dad59" gracePeriod=30 Dec 03 12:27:57 crc kubenswrapper[5052]: I1203 12:27:57.101435 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="5d3294b3-d25c-4d8a-ba28-aaad7385693b" containerName="sg-core" containerID="cri-o://f1eb88cceb1d1670bec09abbba8e4706b62c84aa3f20a8a8d0f5e15a0d9bf509" gracePeriod=30 Dec 03 12:27:57 crc kubenswrapper[5052]: I1203 12:27:57.101466 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="5d3294b3-d25c-4d8a-ba28-aaad7385693b" containerName="ceilometer-notification-agent" containerID="cri-o://98b12c00821c1fa8f0a555940a026478afc3a124403bdbe42ea9907f1c09c593" gracePeriod=30 Dec 03 12:27:57 crc kubenswrapper[5052]: E1203 12:27:57.719043 5052 cadvisor_stats_provider.go:516] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5d3294b3_d25c_4d8a_ba28_aaad7385693b.slice/crio-98b12c00821c1fa8f0a555940a026478afc3a124403bdbe42ea9907f1c09c593.scope\": RecentStats: unable to find data in memory cache]" Dec 03 12:27:57 crc kubenswrapper[5052]: I1203 12:27:57.825189 5052 generic.go:334] "Generic (PLEG): container finished" podID="3a27c483-00fc-4b06-af6b-3c4b0ff8d0d1" containerID="16e0da8b4d5026dda5b45a3d5fe087062b021f9377ca4b9ad0a08e9f37208416" exitCode=0 Dec 03 12:27:57 crc kubenswrapper[5052]: I1203 12:27:57.825253 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-5d855df9cb-xdg6p" event={"ID":"3a27c483-00fc-4b06-af6b-3c4b0ff8d0d1","Type":"ContainerDied","Data":"16e0da8b4d5026dda5b45a3d5fe087062b021f9377ca4b9ad0a08e9f37208416"} Dec 03 12:27:57 crc kubenswrapper[5052]: I1203 12:27:57.829217 5052 generic.go:334] "Generic (PLEG): container finished" podID="5d3294b3-d25c-4d8a-ba28-aaad7385693b" containerID="7e8b797c3e63fa66aad0206d3411e34757b19c124c459e9c5d9dc7e3133dad59" exitCode=0 Dec 03 12:27:57 crc kubenswrapper[5052]: I1203 12:27:57.829263 5052 generic.go:334] "Generic (PLEG): container finished" podID="5d3294b3-d25c-4d8a-ba28-aaad7385693b" containerID="f1eb88cceb1d1670bec09abbba8e4706b62c84aa3f20a8a8d0f5e15a0d9bf509" exitCode=2 Dec 03 12:27:57 crc kubenswrapper[5052]: I1203 12:27:57.829272 5052 generic.go:334] "Generic (PLEG): container finished" podID="5d3294b3-d25c-4d8a-ba28-aaad7385693b" containerID="98b12c00821c1fa8f0a555940a026478afc3a124403bdbe42ea9907f1c09c593" exitCode=0 Dec 03 12:27:57 crc kubenswrapper[5052]: I1203 12:27:57.829281 5052 generic.go:334] "Generic (PLEG): container finished" podID="5d3294b3-d25c-4d8a-ba28-aaad7385693b" containerID="518e554556294e8e06ab1671e9f2b277371917fe0bafb7f6898e763ce48bb5e1" exitCode=0 Dec 03 12:27:57 crc kubenswrapper[5052]: I1203 12:27:57.829318 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"5d3294b3-d25c-4d8a-ba28-aaad7385693b","Type":"ContainerDied","Data":"7e8b797c3e63fa66aad0206d3411e34757b19c124c459e9c5d9dc7e3133dad59"} Dec 03 12:27:57 crc kubenswrapper[5052]: I1203 12:27:57.829362 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"5d3294b3-d25c-4d8a-ba28-aaad7385693b","Type":"ContainerDied","Data":"f1eb88cceb1d1670bec09abbba8e4706b62c84aa3f20a8a8d0f5e15a0d9bf509"} Dec 03 12:27:57 crc kubenswrapper[5052]: I1203 12:27:57.829372 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"5d3294b3-d25c-4d8a-ba28-aaad7385693b","Type":"ContainerDied","Data":"98b12c00821c1fa8f0a555940a026478afc3a124403bdbe42ea9907f1c09c593"} Dec 03 12:27:57 crc kubenswrapper[5052]: I1203 12:27:57.829381 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"5d3294b3-d25c-4d8a-ba28-aaad7385693b","Type":"ContainerDied","Data":"518e554556294e8e06ab1671e9f2b277371917fe0bafb7f6898e763ce48bb5e1"} Dec 03 12:27:57 crc kubenswrapper[5052]: I1203 12:27:57.830563 5052 generic.go:334] "Generic (PLEG): container finished" podID="61588a52-3ab5-45a4-a595-3b13bb4bcc28" containerID="7f4e8c9236258e1ee13e282db4d527b630572e15c3e43c16ee3932b425f0f4a6" exitCode=2 Dec 03 12:27:57 crc kubenswrapper[5052]: I1203 12:27:57.830607 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/kube-state-metrics-0" event={"ID":"61588a52-3ab5-45a4-a595-3b13bb4bcc28","Type":"ContainerDied","Data":"7f4e8c9236258e1ee13e282db4d527b630572e15c3e43c16ee3932b425f0f4a6"} Dec 03 12:27:57 crc kubenswrapper[5052]: I1203 12:27:57.913593 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/memcached-0" Dec 03 12:27:58 crc kubenswrapper[5052]: I1203 12:27:58.009371 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-api-5d855df9cb-xdg6p" Dec 03 12:27:58 crc kubenswrapper[5052]: I1203 12:27:58.077370 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/keystone-bootstrap-8nbqq"] Dec 03 12:27:58 crc kubenswrapper[5052]: E1203 12:27:58.077886 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="3a27c483-00fc-4b06-af6b-3c4b0ff8d0d1" containerName="barbican-api-log" Dec 03 12:27:58 crc kubenswrapper[5052]: I1203 12:27:58.077914 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="3a27c483-00fc-4b06-af6b-3c4b0ff8d0d1" containerName="barbican-api-log" Dec 03 12:27:58 crc kubenswrapper[5052]: E1203 12:27:58.077935 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="3b116fe9-1f40-4614-9381-7f7964fbdcb4" containerName="placement-log" Dec 03 12:27:58 crc kubenswrapper[5052]: I1203 12:27:58.077944 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b116fe9-1f40-4614-9381-7f7964fbdcb4" containerName="placement-log" Dec 03 12:27:58 crc kubenswrapper[5052]: E1203 12:27:58.077957 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="6228bbc6-4c33-4ad8-ad26-440a8e6b441c" containerName="barbican-worker" Dec 03 12:27:58 crc kubenswrapper[5052]: I1203 12:27:58.077965 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="6228bbc6-4c33-4ad8-ad26-440a8e6b441c" containerName="barbican-worker" Dec 03 12:27:58 crc kubenswrapper[5052]: E1203 12:27:58.077983 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="3a27c483-00fc-4b06-af6b-3c4b0ff8d0d1" containerName="barbican-api" Dec 03 12:27:58 crc kubenswrapper[5052]: I1203 12:27:58.077992 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="3a27c483-00fc-4b06-af6b-3c4b0ff8d0d1" containerName="barbican-api" Dec 03 12:27:58 crc kubenswrapper[5052]: E1203 12:27:58.078005 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="51bb178b-072f-4a7a-928b-5a8845d536b7" containerName="placement-log" Dec 03 12:27:58 crc kubenswrapper[5052]: I1203 12:27:58.078013 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="51bb178b-072f-4a7a-928b-5a8845d536b7" containerName="placement-log" Dec 03 12:27:58 crc kubenswrapper[5052]: E1203 12:27:58.078039 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="3b116fe9-1f40-4614-9381-7f7964fbdcb4" containerName="placement-api" Dec 03 12:27:58 crc kubenswrapper[5052]: I1203 12:27:58.078047 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b116fe9-1f40-4614-9381-7f7964fbdcb4" containerName="placement-api" Dec 03 12:27:58 crc kubenswrapper[5052]: E1203 12:27:58.078069 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="51bb178b-072f-4a7a-928b-5a8845d536b7" containerName="placement-api" Dec 03 12:27:58 crc kubenswrapper[5052]: I1203 12:27:58.078076 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="51bb178b-072f-4a7a-928b-5a8845d536b7" containerName="placement-api" Dec 03 12:27:58 crc kubenswrapper[5052]: E1203 12:27:58.078108 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="7f4dfec9-0aa8-437c-9b8d-771b30a93af9" containerName="barbican-keystone-listener" Dec 03 12:27:58 crc kubenswrapper[5052]: I1203 12:27:58.078117 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f4dfec9-0aa8-437c-9b8d-771b30a93af9" containerName="barbican-keystone-listener" Dec 03 12:27:58 crc kubenswrapper[5052]: E1203 12:27:58.078127 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="7f4dfec9-0aa8-437c-9b8d-771b30a93af9" containerName="barbican-keystone-listener-log" Dec 03 12:27:58 crc kubenswrapper[5052]: I1203 12:27:58.078135 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f4dfec9-0aa8-437c-9b8d-771b30a93af9" containerName="barbican-keystone-listener-log" Dec 03 12:27:58 crc kubenswrapper[5052]: E1203 12:27:58.078144 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="6228bbc6-4c33-4ad8-ad26-440a8e6b441c" containerName="barbican-worker-log" Dec 03 12:27:58 crc kubenswrapper[5052]: I1203 12:27:58.078152 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="6228bbc6-4c33-4ad8-ad26-440a8e6b441c" containerName="barbican-worker-log" Dec 03 12:27:58 crc kubenswrapper[5052]: I1203 12:27:58.078377 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="7f4dfec9-0aa8-437c-9b8d-771b30a93af9" containerName="barbican-keystone-listener-log" Dec 03 12:27:58 crc kubenswrapper[5052]: I1203 12:27:58.078398 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="7f4dfec9-0aa8-437c-9b8d-771b30a93af9" containerName="barbican-keystone-listener" Dec 03 12:27:58 crc kubenswrapper[5052]: I1203 12:27:58.078409 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="3a27c483-00fc-4b06-af6b-3c4b0ff8d0d1" containerName="barbican-api" Dec 03 12:27:58 crc kubenswrapper[5052]: I1203 12:27:58.078420 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="3a27c483-00fc-4b06-af6b-3c4b0ff8d0d1" containerName="barbican-api-log" Dec 03 12:27:58 crc kubenswrapper[5052]: I1203 12:27:58.078435 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="51bb178b-072f-4a7a-928b-5a8845d536b7" containerName="placement-api" Dec 03 12:27:58 crc kubenswrapper[5052]: I1203 12:27:58.078447 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="51bb178b-072f-4a7a-928b-5a8845d536b7" containerName="placement-log" Dec 03 12:27:58 crc kubenswrapper[5052]: I1203 12:27:58.078458 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="3b116fe9-1f40-4614-9381-7f7964fbdcb4" containerName="placement-log" Dec 03 12:27:58 crc kubenswrapper[5052]: I1203 12:27:58.078473 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="6228bbc6-4c33-4ad8-ad26-440a8e6b441c" containerName="barbican-worker" Dec 03 12:27:58 crc kubenswrapper[5052]: I1203 12:27:58.078485 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="6228bbc6-4c33-4ad8-ad26-440a8e6b441c" containerName="barbican-worker-log" Dec 03 12:27:58 crc kubenswrapper[5052]: I1203 12:27:58.078502 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="3b116fe9-1f40-4614-9381-7f7964fbdcb4" containerName="placement-api" Dec 03 12:27:58 crc kubenswrapper[5052]: I1203 12:27:58.079423 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-bootstrap-8nbqq" Dec 03 12:27:58 crc kubenswrapper[5052]: I1203 12:27:58.085217 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"osp-secret" Dec 03 12:27:58 crc kubenswrapper[5052]: I1203 12:27:58.101681 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-bootstrap-8nbqq"] Dec 03 12:27:58 crc kubenswrapper[5052]: I1203 12:27:58.143641 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3a27c483-00fc-4b06-af6b-3c4b0ff8d0d1-combined-ca-bundle\") pod \"3a27c483-00fc-4b06-af6b-3c4b0ff8d0d1\" (UID: \"3a27c483-00fc-4b06-af6b-3c4b0ff8d0d1\") " Dec 03 12:27:58 crc kubenswrapper[5052]: I1203 12:27:58.143784 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/3a27c483-00fc-4b06-af6b-3c4b0ff8d0d1-logs\") pod \"3a27c483-00fc-4b06-af6b-3c4b0ff8d0d1\" (UID: \"3a27c483-00fc-4b06-af6b-3c4b0ff8d0d1\") " Dec 03 12:27:58 crc kubenswrapper[5052]: I1203 12:27:58.143896 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/3a27c483-00fc-4b06-af6b-3c4b0ff8d0d1-config-data-custom\") pod \"3a27c483-00fc-4b06-af6b-3c4b0ff8d0d1\" (UID: \"3a27c483-00fc-4b06-af6b-3c4b0ff8d0d1\") " Dec 03 12:27:58 crc kubenswrapper[5052]: I1203 12:27:58.143954 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/3a27c483-00fc-4b06-af6b-3c4b0ff8d0d1-config-data\") pod \"3a27c483-00fc-4b06-af6b-3c4b0ff8d0d1\" (UID: \"3a27c483-00fc-4b06-af6b-3c4b0ff8d0d1\") " Dec 03 12:27:58 crc kubenswrapper[5052]: I1203 12:27:58.143977 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h68bg\" (UniqueName: \"kubernetes.io/projected/3a27c483-00fc-4b06-af6b-3c4b0ff8d0d1-kube-api-access-h68bg\") pod \"3a27c483-00fc-4b06-af6b-3c4b0ff8d0d1\" (UID: \"3a27c483-00fc-4b06-af6b-3c4b0ff8d0d1\") " Dec 03 12:27:58 crc kubenswrapper[5052]: I1203 12:27:58.144487 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3a27c483-00fc-4b06-af6b-3c4b0ff8d0d1-logs" (OuterVolumeSpecName: "logs") pod "3a27c483-00fc-4b06-af6b-3c4b0ff8d0d1" (UID: "3a27c483-00fc-4b06-af6b-3c4b0ff8d0d1"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:27:58 crc kubenswrapper[5052]: I1203 12:27:58.158552 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3a27c483-00fc-4b06-af6b-3c4b0ff8d0d1-kube-api-access-h68bg" (OuterVolumeSpecName: "kube-api-access-h68bg") pod "3a27c483-00fc-4b06-af6b-3c4b0ff8d0d1" (UID: "3a27c483-00fc-4b06-af6b-3c4b0ff8d0d1"). InnerVolumeSpecName "kube-api-access-h68bg". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:27:58 crc kubenswrapper[5052]: I1203 12:27:58.184333 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:27:58 crc kubenswrapper[5052]: I1203 12:27:58.184615 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-scheduler-0" podUID="8a0b169d-6971-4530-936e-8e5f0f57816f" containerName="nova-scheduler-scheduler" containerID="cri-o://47d8a341afac3d19e0311930b0aa745891784bd644a5b9f1e35346d168347db9" gracePeriod=30 Dec 03 12:27:58 crc kubenswrapper[5052]: I1203 12:27:58.184669 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3a27c483-00fc-4b06-af6b-3c4b0ff8d0d1-config-data-custom" (OuterVolumeSpecName: "config-data-custom") pod "3a27c483-00fc-4b06-af6b-3c4b0ff8d0d1" (UID: "3a27c483-00fc-4b06-af6b-3c4b0ff8d0d1"). InnerVolumeSpecName "config-data-custom". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:27:58 crc kubenswrapper[5052]: I1203 12:27:58.232358 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-0"] Dec 03 12:27:58 crc kubenswrapper[5052]: I1203 12:27:58.232844 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-cell0-conductor-0" podUID="c080dc7f-df72-487f-b2c1-83b143d34784" containerName="nova-cell0-conductor-conductor" containerID="cri-o://790991ac480a9e358efc31b18c1e765ec3b8f7c9c3386cf9c51c6dc7f800750c" gracePeriod=30 Dec 03 12:27:58 crc kubenswrapper[5052]: I1203 12:27:58.253612 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/ee394270-24f0-4687-9129-a5937efe9826-scripts\") pod \"keystone-bootstrap-8nbqq\" (UID: \"ee394270-24f0-4687-9129-a5937efe9826\") " pod="openstack-kuttl-tests/keystone-bootstrap-8nbqq" Dec 03 12:27:58 crc kubenswrapper[5052]: I1203 12:27:58.253701 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ee394270-24f0-4687-9129-a5937efe9826-combined-ca-bundle\") pod \"keystone-bootstrap-8nbqq\" (UID: \"ee394270-24f0-4687-9129-a5937efe9826\") " pod="openstack-kuttl-tests/keystone-bootstrap-8nbqq" Dec 03 12:27:58 crc kubenswrapper[5052]: I1203 12:27:58.253838 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mfzvn\" (UniqueName: \"kubernetes.io/projected/ee394270-24f0-4687-9129-a5937efe9826-kube-api-access-mfzvn\") pod \"keystone-bootstrap-8nbqq\" (UID: \"ee394270-24f0-4687-9129-a5937efe9826\") " pod="openstack-kuttl-tests/keystone-bootstrap-8nbqq" Dec 03 12:27:58 crc kubenswrapper[5052]: I1203 12:27:58.253868 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/ee394270-24f0-4687-9129-a5937efe9826-credential-keys\") pod \"keystone-bootstrap-8nbqq\" (UID: \"ee394270-24f0-4687-9129-a5937efe9826\") " pod="openstack-kuttl-tests/keystone-bootstrap-8nbqq" Dec 03 12:27:58 crc kubenswrapper[5052]: I1203 12:27:58.254060 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/ee394270-24f0-4687-9129-a5937efe9826-fernet-keys\") pod \"keystone-bootstrap-8nbqq\" (UID: \"ee394270-24f0-4687-9129-a5937efe9826\") " pod="openstack-kuttl-tests/keystone-bootstrap-8nbqq" Dec 03 12:27:58 crc kubenswrapper[5052]: I1203 12:27:58.254197 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/ee394270-24f0-4687-9129-a5937efe9826-config-data\") pod \"keystone-bootstrap-8nbqq\" (UID: \"ee394270-24f0-4687-9129-a5937efe9826\") " pod="openstack-kuttl-tests/keystone-bootstrap-8nbqq" Dec 03 12:27:58 crc kubenswrapper[5052]: I1203 12:27:58.254289 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/3a27c483-00fc-4b06-af6b-3c4b0ff8d0d1-config-data-custom\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:58 crc kubenswrapper[5052]: I1203 12:27:58.254308 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-h68bg\" (UniqueName: \"kubernetes.io/projected/3a27c483-00fc-4b06-af6b-3c4b0ff8d0d1-kube-api-access-h68bg\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:58 crc kubenswrapper[5052]: I1203 12:27:58.254320 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/3a27c483-00fc-4b06-af6b-3c4b0ff8d0d1-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:58 crc kubenswrapper[5052]: I1203 12:27:58.262603 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 12:27:58 crc kubenswrapper[5052]: I1203 12:27:58.262908 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/cinder-scheduler-0" podUID="6436bca9-c7a1-4540-bee3-b6dbabb53c89" containerName="cinder-scheduler" containerID="cri-o://4f7e4bab55b18e1135d280cc95a601724a2d3fc12cda71f14ad7f590adb9df61" gracePeriod=30 Dec 03 12:27:58 crc kubenswrapper[5052]: I1203 12:27:58.263417 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/cinder-scheduler-0" podUID="6436bca9-c7a1-4540-bee3-b6dbabb53c89" containerName="probe" containerID="cri-o://584f69d542c77d9dd7acd735af74068b5eca3f64ae82019e8afdda51237d1117" gracePeriod=30 Dec 03 12:27:58 crc kubenswrapper[5052]: I1203 12:27:58.270702 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3a27c483-00fc-4b06-af6b-3c4b0ff8d0d1-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "3a27c483-00fc-4b06-af6b-3c4b0ff8d0d1" (UID: "3a27c483-00fc-4b06-af6b-3c4b0ff8d0d1"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:27:58 crc kubenswrapper[5052]: I1203 12:27:58.280420 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:27:58 crc kubenswrapper[5052]: I1203 12:27:58.280902 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-external-api-0" podUID="71e0e74b-5d01-467f-afb2-7e55d9d58297" containerName="glance-log" containerID="cri-o://8bcc87cf614b2c81facd280057f18832930b2317f445a10147336b258adbe781" gracePeriod=30 Dec 03 12:27:58 crc kubenswrapper[5052]: I1203 12:27:58.281490 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-external-api-0" podUID="71e0e74b-5d01-467f-afb2-7e55d9d58297" containerName="glance-httpd" containerID="cri-o://6e64026bbfc0cc36204875e424068283a0de98c7caf69823427bc926fa18e20d" gracePeriod=30 Dec 03 12:27:58 crc kubenswrapper[5052]: I1203 12:27:58.355662 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-mfzvn\" (UniqueName: \"kubernetes.io/projected/ee394270-24f0-4687-9129-a5937efe9826-kube-api-access-mfzvn\") pod \"keystone-bootstrap-8nbqq\" (UID: \"ee394270-24f0-4687-9129-a5937efe9826\") " pod="openstack-kuttl-tests/keystone-bootstrap-8nbqq" Dec 03 12:27:58 crc kubenswrapper[5052]: I1203 12:27:58.356036 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/ee394270-24f0-4687-9129-a5937efe9826-credential-keys\") pod \"keystone-bootstrap-8nbqq\" (UID: \"ee394270-24f0-4687-9129-a5937efe9826\") " pod="openstack-kuttl-tests/keystone-bootstrap-8nbqq" Dec 03 12:27:58 crc kubenswrapper[5052]: I1203 12:27:58.356130 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/ee394270-24f0-4687-9129-a5937efe9826-fernet-keys\") pod \"keystone-bootstrap-8nbqq\" (UID: \"ee394270-24f0-4687-9129-a5937efe9826\") " pod="openstack-kuttl-tests/keystone-bootstrap-8nbqq" Dec 03 12:27:58 crc kubenswrapper[5052]: I1203 12:27:58.356181 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/ee394270-24f0-4687-9129-a5937efe9826-config-data\") pod \"keystone-bootstrap-8nbqq\" (UID: \"ee394270-24f0-4687-9129-a5937efe9826\") " pod="openstack-kuttl-tests/keystone-bootstrap-8nbqq" Dec 03 12:27:58 crc kubenswrapper[5052]: I1203 12:27:58.356243 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/ee394270-24f0-4687-9129-a5937efe9826-scripts\") pod \"keystone-bootstrap-8nbqq\" (UID: \"ee394270-24f0-4687-9129-a5937efe9826\") " pod="openstack-kuttl-tests/keystone-bootstrap-8nbqq" Dec 03 12:27:58 crc kubenswrapper[5052]: I1203 12:27:58.356279 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ee394270-24f0-4687-9129-a5937efe9826-combined-ca-bundle\") pod \"keystone-bootstrap-8nbqq\" (UID: \"ee394270-24f0-4687-9129-a5937efe9826\") " pod="openstack-kuttl-tests/keystone-bootstrap-8nbqq" Dec 03 12:27:58 crc kubenswrapper[5052]: I1203 12:27:58.356355 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3a27c483-00fc-4b06-af6b-3c4b0ff8d0d1-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:58 crc kubenswrapper[5052]: I1203 12:27:58.362584 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ee394270-24f0-4687-9129-a5937efe9826-combined-ca-bundle\") pod \"keystone-bootstrap-8nbqq\" (UID: \"ee394270-24f0-4687-9129-a5937efe9826\") " pod="openstack-kuttl-tests/keystone-bootstrap-8nbqq" Dec 03 12:27:58 crc kubenswrapper[5052]: E1203 12:27:58.368702 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="47d8a341afac3d19e0311930b0aa745891784bd644a5b9f1e35346d168347db9" cmd=["/usr/bin/pgrep","-r","DRST","nova-scheduler"] Dec 03 12:27:58 crc kubenswrapper[5052]: I1203 12:27:58.372158 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/ee394270-24f0-4687-9129-a5937efe9826-fernet-keys\") pod \"keystone-bootstrap-8nbqq\" (UID: \"ee394270-24f0-4687-9129-a5937efe9826\") " pod="openstack-kuttl-tests/keystone-bootstrap-8nbqq" Dec 03 12:27:58 crc kubenswrapper[5052]: I1203 12:27:58.374281 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/ee394270-24f0-4687-9129-a5937efe9826-scripts\") pod \"keystone-bootstrap-8nbqq\" (UID: \"ee394270-24f0-4687-9129-a5937efe9826\") " pod="openstack-kuttl-tests/keystone-bootstrap-8nbqq" Dec 03 12:27:58 crc kubenswrapper[5052]: E1203 12:27:58.378756 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="47d8a341afac3d19e0311930b0aa745891784bd644a5b9f1e35346d168347db9" cmd=["/usr/bin/pgrep","-r","DRST","nova-scheduler"] Dec 03 12:27:58 crc kubenswrapper[5052]: I1203 12:27:58.387372 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/ee394270-24f0-4687-9129-a5937efe9826-config-data\") pod \"keystone-bootstrap-8nbqq\" (UID: \"ee394270-24f0-4687-9129-a5937efe9826\") " pod="openstack-kuttl-tests/keystone-bootstrap-8nbqq" Dec 03 12:27:58 crc kubenswrapper[5052]: I1203 12:27:58.389901 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/ee394270-24f0-4687-9129-a5937efe9826-credential-keys\") pod \"keystone-bootstrap-8nbqq\" (UID: \"ee394270-24f0-4687-9129-a5937efe9826\") " pod="openstack-kuttl-tests/keystone-bootstrap-8nbqq" Dec 03 12:27:58 crc kubenswrapper[5052]: I1203 12:27:58.401714 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3a27c483-00fc-4b06-af6b-3c4b0ff8d0d1-config-data" (OuterVolumeSpecName: "config-data") pod "3a27c483-00fc-4b06-af6b-3c4b0ff8d0d1" (UID: "3a27c483-00fc-4b06-af6b-3c4b0ff8d0d1"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:27:58 crc kubenswrapper[5052]: E1203 12:27:58.401864 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="47d8a341afac3d19e0311930b0aa745891784bd644a5b9f1e35346d168347db9" cmd=["/usr/bin/pgrep","-r","DRST","nova-scheduler"] Dec 03 12:27:58 crc kubenswrapper[5052]: E1203 12:27:58.401928 5052 prober.go:104] "Probe errored" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" probeType="Readiness" pod="openstack-kuttl-tests/nova-scheduler-0" podUID="8a0b169d-6971-4530-936e-8e5f0f57816f" containerName="nova-scheduler-scheduler" Dec 03 12:27:58 crc kubenswrapper[5052]: I1203 12:27:58.412568 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 12:27:58 crc kubenswrapper[5052]: I1203 12:27:58.412860 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/cinder-api-0" podUID="2b5843e3-84ff-4174-bd4a-f67ba2a3a238" containerName="cinder-api-log" containerID="cri-o://667a1283a5ee29a96d6e7cde347d08e4fba330b96f0a08eb1f51d024fa8e92ed" gracePeriod=30 Dec 03 12:27:58 crc kubenswrapper[5052]: I1203 12:27:58.413003 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/cinder-api-0" podUID="2b5843e3-84ff-4174-bd4a-f67ba2a3a238" containerName="cinder-api" containerID="cri-o://e07ed2652a16aa998baf13477cacedfad92c750d43fb52c99642ffc63d38a6cf" gracePeriod=30 Dec 03 12:27:58 crc kubenswrapper[5052]: I1203 12:27:58.414549 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-mfzvn\" (UniqueName: \"kubernetes.io/projected/ee394270-24f0-4687-9129-a5937efe9826-kube-api-access-mfzvn\") pod \"keystone-bootstrap-8nbqq\" (UID: \"ee394270-24f0-4687-9129-a5937efe9826\") " pod="openstack-kuttl-tests/keystone-bootstrap-8nbqq" Dec 03 12:27:58 crc kubenswrapper[5052]: I1203 12:27:58.423978 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-bootstrap-8nbqq" Dec 03 12:27:58 crc kubenswrapper[5052]: I1203 12:27:58.427302 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/neutron-6bf55bd9d6-mrgs9"] Dec 03 12:27:58 crc kubenswrapper[5052]: I1203 12:27:58.427543 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/neutron-6bf55bd9d6-mrgs9" podUID="15cf6575-b6be-4f67-861a-61ebf02d1495" containerName="neutron-httpd" containerID="cri-o://b4a9d01e3dc994f108359d2cce819e8f260555e99eb3dff86f96ba5dc6edd4b0" gracePeriod=30 Dec 03 12:27:58 crc kubenswrapper[5052]: I1203 12:27:58.439183 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/neutron-6bf55bd9d6-mrgs9" podUID="15cf6575-b6be-4f67-861a-61ebf02d1495" containerName="neutron-httpd" probeResult="failure" output="HTTP probe failed with statuscode: 503" Dec 03 12:27:58 crc kubenswrapper[5052]: I1203 12:27:58.460947 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/3a27c483-00fc-4b06-af6b-3c4b0ff8d0d1-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:58 crc kubenswrapper[5052]: I1203 12:27:58.535954 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/neutron-56ff9fdb74-cft5n"] Dec 03 12:27:58 crc kubenswrapper[5052]: I1203 12:27:58.538261 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-56ff9fdb74-cft5n" Dec 03 12:27:58 crc kubenswrapper[5052]: I1203 12:27:58.545683 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-neutron-public-svc" Dec 03 12:27:58 crc kubenswrapper[5052]: I1203 12:27:58.545875 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-neutron-internal-svc" Dec 03 12:27:58 crc kubenswrapper[5052]: I1203 12:27:58.545984 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-neutron-ovndbs" Dec 03 12:27:58 crc kubenswrapper[5052]: I1203 12:27:58.577417 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-56ff9fdb74-cft5n"] Dec 03 12:27:58 crc kubenswrapper[5052]: I1203 12:27:58.656771 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:27:58 crc kubenswrapper[5052]: I1203 12:27:58.657028 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-internal-api-0" podUID="0b3a2637-ed0d-4348-ac7e-c92db6503191" containerName="glance-log" containerID="cri-o://35e2dee61a61ea5e56329f496973eb6a0f2fe0619c6f0df2e55caadaea334f93" gracePeriod=30 Dec 03 12:27:58 crc kubenswrapper[5052]: I1203 12:27:58.657601 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-internal-api-0" podUID="0b3a2637-ed0d-4348-ac7e-c92db6503191" containerName="glance-httpd" containerID="cri-o://8c8df8bba4000ac70f93809a90d4fade952afb4ce410d7dcf985178e32099131" gracePeriod=30 Dec 03 12:27:58 crc kubenswrapper[5052]: I1203 12:27:58.675453 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/5c13362a-3cdf-4ff3-b57d-eb8c6cdcdc98-ovndb-tls-certs\") pod \"neutron-56ff9fdb74-cft5n\" (UID: \"5c13362a-3cdf-4ff3-b57d-eb8c6cdcdc98\") " pod="openstack-kuttl-tests/neutron-56ff9fdb74-cft5n" Dec 03 12:27:58 crc kubenswrapper[5052]: I1203 12:27:58.675502 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/5c13362a-3cdf-4ff3-b57d-eb8c6cdcdc98-config\") pod \"neutron-56ff9fdb74-cft5n\" (UID: \"5c13362a-3cdf-4ff3-b57d-eb8c6cdcdc98\") " pod="openstack-kuttl-tests/neutron-56ff9fdb74-cft5n" Dec 03 12:27:58 crc kubenswrapper[5052]: I1203 12:27:58.675546 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/5c13362a-3cdf-4ff3-b57d-eb8c6cdcdc98-httpd-config\") pod \"neutron-56ff9fdb74-cft5n\" (UID: \"5c13362a-3cdf-4ff3-b57d-eb8c6cdcdc98\") " pod="openstack-kuttl-tests/neutron-56ff9fdb74-cft5n" Dec 03 12:27:58 crc kubenswrapper[5052]: I1203 12:27:58.675800 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/5c13362a-3cdf-4ff3-b57d-eb8c6cdcdc98-internal-tls-certs\") pod \"neutron-56ff9fdb74-cft5n\" (UID: \"5c13362a-3cdf-4ff3-b57d-eb8c6cdcdc98\") " pod="openstack-kuttl-tests/neutron-56ff9fdb74-cft5n" Dec 03 12:27:58 crc kubenswrapper[5052]: I1203 12:27:58.676030 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5c13362a-3cdf-4ff3-b57d-eb8c6cdcdc98-combined-ca-bundle\") pod \"neutron-56ff9fdb74-cft5n\" (UID: \"5c13362a-3cdf-4ff3-b57d-eb8c6cdcdc98\") " pod="openstack-kuttl-tests/neutron-56ff9fdb74-cft5n" Dec 03 12:27:58 crc kubenswrapper[5052]: I1203 12:27:58.676204 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pvvfv\" (UniqueName: \"kubernetes.io/projected/5c13362a-3cdf-4ff3-b57d-eb8c6cdcdc98-kube-api-access-pvvfv\") pod \"neutron-56ff9fdb74-cft5n\" (UID: \"5c13362a-3cdf-4ff3-b57d-eb8c6cdcdc98\") " pod="openstack-kuttl-tests/neutron-56ff9fdb74-cft5n" Dec 03 12:27:58 crc kubenswrapper[5052]: I1203 12:27:58.676229 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/5c13362a-3cdf-4ff3-b57d-eb8c6cdcdc98-public-tls-certs\") pod \"neutron-56ff9fdb74-cft5n\" (UID: \"5c13362a-3cdf-4ff3-b57d-eb8c6cdcdc98\") " pod="openstack-kuttl-tests/neutron-56ff9fdb74-cft5n" Dec 03 12:27:58 crc kubenswrapper[5052]: I1203 12:27:58.779461 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/5c13362a-3cdf-4ff3-b57d-eb8c6cdcdc98-ovndb-tls-certs\") pod \"neutron-56ff9fdb74-cft5n\" (UID: \"5c13362a-3cdf-4ff3-b57d-eb8c6cdcdc98\") " pod="openstack-kuttl-tests/neutron-56ff9fdb74-cft5n" Dec 03 12:27:58 crc kubenswrapper[5052]: I1203 12:27:58.779721 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/5c13362a-3cdf-4ff3-b57d-eb8c6cdcdc98-config\") pod \"neutron-56ff9fdb74-cft5n\" (UID: \"5c13362a-3cdf-4ff3-b57d-eb8c6cdcdc98\") " pod="openstack-kuttl-tests/neutron-56ff9fdb74-cft5n" Dec 03 12:27:58 crc kubenswrapper[5052]: I1203 12:27:58.779738 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/5c13362a-3cdf-4ff3-b57d-eb8c6cdcdc98-httpd-config\") pod \"neutron-56ff9fdb74-cft5n\" (UID: \"5c13362a-3cdf-4ff3-b57d-eb8c6cdcdc98\") " pod="openstack-kuttl-tests/neutron-56ff9fdb74-cft5n" Dec 03 12:27:58 crc kubenswrapper[5052]: I1203 12:27:58.779813 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/5c13362a-3cdf-4ff3-b57d-eb8c6cdcdc98-internal-tls-certs\") pod \"neutron-56ff9fdb74-cft5n\" (UID: \"5c13362a-3cdf-4ff3-b57d-eb8c6cdcdc98\") " pod="openstack-kuttl-tests/neutron-56ff9fdb74-cft5n" Dec 03 12:27:58 crc kubenswrapper[5052]: I1203 12:27:58.779887 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5c13362a-3cdf-4ff3-b57d-eb8c6cdcdc98-combined-ca-bundle\") pod \"neutron-56ff9fdb74-cft5n\" (UID: \"5c13362a-3cdf-4ff3-b57d-eb8c6cdcdc98\") " pod="openstack-kuttl-tests/neutron-56ff9fdb74-cft5n" Dec 03 12:27:58 crc kubenswrapper[5052]: I1203 12:27:58.779972 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-pvvfv\" (UniqueName: \"kubernetes.io/projected/5c13362a-3cdf-4ff3-b57d-eb8c6cdcdc98-kube-api-access-pvvfv\") pod \"neutron-56ff9fdb74-cft5n\" (UID: \"5c13362a-3cdf-4ff3-b57d-eb8c6cdcdc98\") " pod="openstack-kuttl-tests/neutron-56ff9fdb74-cft5n" Dec 03 12:27:58 crc kubenswrapper[5052]: I1203 12:27:58.779990 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/5c13362a-3cdf-4ff3-b57d-eb8c6cdcdc98-public-tls-certs\") pod \"neutron-56ff9fdb74-cft5n\" (UID: \"5c13362a-3cdf-4ff3-b57d-eb8c6cdcdc98\") " pod="openstack-kuttl-tests/neutron-56ff9fdb74-cft5n" Dec 03 12:27:58 crc kubenswrapper[5052]: I1203 12:27:58.788791 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/5c13362a-3cdf-4ff3-b57d-eb8c6cdcdc98-config\") pod \"neutron-56ff9fdb74-cft5n\" (UID: \"5c13362a-3cdf-4ff3-b57d-eb8c6cdcdc98\") " pod="openstack-kuttl-tests/neutron-56ff9fdb74-cft5n" Dec 03 12:27:58 crc kubenswrapper[5052]: I1203 12:27:58.793652 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/5c13362a-3cdf-4ff3-b57d-eb8c6cdcdc98-internal-tls-certs\") pod \"neutron-56ff9fdb74-cft5n\" (UID: \"5c13362a-3cdf-4ff3-b57d-eb8c6cdcdc98\") " pod="openstack-kuttl-tests/neutron-56ff9fdb74-cft5n" Dec 03 12:27:58 crc kubenswrapper[5052]: I1203 12:27:58.800925 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/5c13362a-3cdf-4ff3-b57d-eb8c6cdcdc98-public-tls-certs\") pod \"neutron-56ff9fdb74-cft5n\" (UID: \"5c13362a-3cdf-4ff3-b57d-eb8c6cdcdc98\") " pod="openstack-kuttl-tests/neutron-56ff9fdb74-cft5n" Dec 03 12:27:58 crc kubenswrapper[5052]: I1203 12:27:58.803566 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5c13362a-3cdf-4ff3-b57d-eb8c6cdcdc98-combined-ca-bundle\") pod \"neutron-56ff9fdb74-cft5n\" (UID: \"5c13362a-3cdf-4ff3-b57d-eb8c6cdcdc98\") " pod="openstack-kuttl-tests/neutron-56ff9fdb74-cft5n" Dec 03 12:27:58 crc kubenswrapper[5052]: I1203 12:27:58.807808 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/5c13362a-3cdf-4ff3-b57d-eb8c6cdcdc98-httpd-config\") pod \"neutron-56ff9fdb74-cft5n\" (UID: \"5c13362a-3cdf-4ff3-b57d-eb8c6cdcdc98\") " pod="openstack-kuttl-tests/neutron-56ff9fdb74-cft5n" Dec 03 12:27:58 crc kubenswrapper[5052]: I1203 12:27:58.820358 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:27:58 crc kubenswrapper[5052]: I1203 12:27:58.825343 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/5c13362a-3cdf-4ff3-b57d-eb8c6cdcdc98-ovndb-tls-certs\") pod \"neutron-56ff9fdb74-cft5n\" (UID: \"5c13362a-3cdf-4ff3-b57d-eb8c6cdcdc98\") " pod="openstack-kuttl-tests/neutron-56ff9fdb74-cft5n" Dec 03 12:27:58 crc kubenswrapper[5052]: I1203 12:27:58.844973 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-pvvfv\" (UniqueName: \"kubernetes.io/projected/5c13362a-3cdf-4ff3-b57d-eb8c6cdcdc98-kube-api-access-pvvfv\") pod \"neutron-56ff9fdb74-cft5n\" (UID: \"5c13362a-3cdf-4ff3-b57d-eb8c6cdcdc98\") " pod="openstack-kuttl-tests/neutron-56ff9fdb74-cft5n" Dec 03 12:27:58 crc kubenswrapper[5052]: I1203 12:27:58.883203 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-56ff9fdb74-cft5n" Dec 03 12:27:58 crc kubenswrapper[5052]: I1203 12:27:58.952879 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/kube-state-metrics-0" event={"ID":"61588a52-3ab5-45a4-a595-3b13bb4bcc28","Type":"ContainerDied","Data":"64d60f730a695a2d3dd995103f5408750656e118d65fab2dcfbd614393d8bd04"} Dec 03 12:27:58 crc kubenswrapper[5052]: I1203 12:27:58.952944 5052 scope.go:117] "RemoveContainer" containerID="7f4e8c9236258e1ee13e282db4d527b630572e15c3e43c16ee3932b425f0f4a6" Dec 03 12:27:58 crc kubenswrapper[5052]: I1203 12:27:58.953114 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:27:58 crc kubenswrapper[5052]: I1203 12:27:58.988722 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tj7sm\" (UniqueName: \"kubernetes.io/projected/61588a52-3ab5-45a4-a595-3b13bb4bcc28-kube-api-access-tj7sm\") pod \"61588a52-3ab5-45a4-a595-3b13bb4bcc28\" (UID: \"61588a52-3ab5-45a4-a595-3b13bb4bcc28\") " Dec 03 12:27:59 crc kubenswrapper[5052]: I1203 12:27:59.002658 5052 generic.go:334] "Generic (PLEG): container finished" podID="71e0e74b-5d01-467f-afb2-7e55d9d58297" containerID="8bcc87cf614b2c81facd280057f18832930b2317f445a10147336b258adbe781" exitCode=143 Dec 03 12:27:59 crc kubenswrapper[5052]: I1203 12:27:59.003001 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"71e0e74b-5d01-467f-afb2-7e55d9d58297","Type":"ContainerDied","Data":"8bcc87cf614b2c81facd280057f18832930b2317f445a10147336b258adbe781"} Dec 03 12:27:59 crc kubenswrapper[5052]: I1203 12:27:59.003234 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/61588a52-3ab5-45a4-a595-3b13bb4bcc28-kube-api-access-tj7sm" (OuterVolumeSpecName: "kube-api-access-tj7sm") pod "61588a52-3ab5-45a4-a595-3b13bb4bcc28" (UID: "61588a52-3ab5-45a4-a595-3b13bb4bcc28"). InnerVolumeSpecName "kube-api-access-tj7sm". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:27:59 crc kubenswrapper[5052]: I1203 12:27:59.044098 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-5d855df9cb-xdg6p" event={"ID":"3a27c483-00fc-4b06-af6b-3c4b0ff8d0d1","Type":"ContainerDied","Data":"cc1a0008b39944b5b62eec529e54a52c2cfcdd0258dd27e2496043f63eb6d80d"} Dec 03 12:27:59 crc kubenswrapper[5052]: I1203 12:27:59.044215 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-api-5d855df9cb-xdg6p" Dec 03 12:27:59 crc kubenswrapper[5052]: I1203 12:27:59.091252 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-tj7sm\" (UniqueName: \"kubernetes.io/projected/61588a52-3ab5-45a4-a595-3b13bb4bcc28-kube-api-access-tj7sm\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:59 crc kubenswrapper[5052]: I1203 12:27:59.198926 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-api-5d855df9cb-xdg6p"] Dec 03 12:27:59 crc kubenswrapper[5052]: I1203 12:27:59.200286 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:27:59 crc kubenswrapper[5052]: I1203 12:27:59.208792 5052 scope.go:117] "RemoveContainer" containerID="16e0da8b4d5026dda5b45a3d5fe087062b021f9377ca4b9ad0a08e9f37208416" Dec 03 12:27:59 crc kubenswrapper[5052]: I1203 12:27:59.217470 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/barbican-api-5d855df9cb-xdg6p"] Dec 03 12:27:59 crc kubenswrapper[5052]: I1203 12:27:59.269707 5052 scope.go:117] "RemoveContainer" containerID="03ad0a39928a5dbe15709e4ca192498a258450f53a516b11f0805a2e1f446f33" Dec 03 12:27:59 crc kubenswrapper[5052]: I1203 12:27:59.298170 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/5d3294b3-d25c-4d8a-ba28-aaad7385693b-run-httpd\") pod \"5d3294b3-d25c-4d8a-ba28-aaad7385693b\" (UID: \"5d3294b3-d25c-4d8a-ba28-aaad7385693b\") " Dec 03 12:27:59 crc kubenswrapper[5052]: I1203 12:27:59.298613 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/5d3294b3-d25c-4d8a-ba28-aaad7385693b-sg-core-conf-yaml\") pod \"5d3294b3-d25c-4d8a-ba28-aaad7385693b\" (UID: \"5d3294b3-d25c-4d8a-ba28-aaad7385693b\") " Dec 03 12:27:59 crc kubenswrapper[5052]: I1203 12:27:59.298984 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5d3294b3-d25c-4d8a-ba28-aaad7385693b-run-httpd" (OuterVolumeSpecName: "run-httpd") pod "5d3294b3-d25c-4d8a-ba28-aaad7385693b" (UID: "5d3294b3-d25c-4d8a-ba28-aaad7385693b"). InnerVolumeSpecName "run-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:27:59 crc kubenswrapper[5052]: I1203 12:27:59.299385 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/5d3294b3-d25c-4d8a-ba28-aaad7385693b-scripts\") pod \"5d3294b3-d25c-4d8a-ba28-aaad7385693b\" (UID: \"5d3294b3-d25c-4d8a-ba28-aaad7385693b\") " Dec 03 12:27:59 crc kubenswrapper[5052]: I1203 12:27:59.299487 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/5d3294b3-d25c-4d8a-ba28-aaad7385693b-config-data\") pod \"5d3294b3-d25c-4d8a-ba28-aaad7385693b\" (UID: \"5d3294b3-d25c-4d8a-ba28-aaad7385693b\") " Dec 03 12:27:59 crc kubenswrapper[5052]: I1203 12:27:59.299546 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/5d3294b3-d25c-4d8a-ba28-aaad7385693b-log-httpd\") pod \"5d3294b3-d25c-4d8a-ba28-aaad7385693b\" (UID: \"5d3294b3-d25c-4d8a-ba28-aaad7385693b\") " Dec 03 12:27:59 crc kubenswrapper[5052]: I1203 12:27:59.299680 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5d3294b3-d25c-4d8a-ba28-aaad7385693b-combined-ca-bundle\") pod \"5d3294b3-d25c-4d8a-ba28-aaad7385693b\" (UID: \"5d3294b3-d25c-4d8a-ba28-aaad7385693b\") " Dec 03 12:27:59 crc kubenswrapper[5052]: I1203 12:27:59.299729 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sgtpx\" (UniqueName: \"kubernetes.io/projected/5d3294b3-d25c-4d8a-ba28-aaad7385693b-kube-api-access-sgtpx\") pod \"5d3294b3-d25c-4d8a-ba28-aaad7385693b\" (UID: \"5d3294b3-d25c-4d8a-ba28-aaad7385693b\") " Dec 03 12:27:59 crc kubenswrapper[5052]: I1203 12:27:59.300778 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5d3294b3-d25c-4d8a-ba28-aaad7385693b-log-httpd" (OuterVolumeSpecName: "log-httpd") pod "5d3294b3-d25c-4d8a-ba28-aaad7385693b" (UID: "5d3294b3-d25c-4d8a-ba28-aaad7385693b"). InnerVolumeSpecName "log-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:27:59 crc kubenswrapper[5052]: I1203 12:27:59.300837 5052 reconciler_common.go:293] "Volume detached for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/5d3294b3-d25c-4d8a-ba28-aaad7385693b-run-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:59 crc kubenswrapper[5052]: I1203 12:27:59.306219 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5d3294b3-d25c-4d8a-ba28-aaad7385693b-scripts" (OuterVolumeSpecName: "scripts") pod "5d3294b3-d25c-4d8a-ba28-aaad7385693b" (UID: "5d3294b3-d25c-4d8a-ba28-aaad7385693b"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:27:59 crc kubenswrapper[5052]: I1203 12:27:59.306373 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5d3294b3-d25c-4d8a-ba28-aaad7385693b-kube-api-access-sgtpx" (OuterVolumeSpecName: "kube-api-access-sgtpx") pod "5d3294b3-d25c-4d8a-ba28-aaad7385693b" (UID: "5d3294b3-d25c-4d8a-ba28-aaad7385693b"). InnerVolumeSpecName "kube-api-access-sgtpx". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:27:59 crc kubenswrapper[5052]: I1203 12:27:59.339790 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5d3294b3-d25c-4d8a-ba28-aaad7385693b-sg-core-conf-yaml" (OuterVolumeSpecName: "sg-core-conf-yaml") pod "5d3294b3-d25c-4d8a-ba28-aaad7385693b" (UID: "5d3294b3-d25c-4d8a-ba28-aaad7385693b"). InnerVolumeSpecName "sg-core-conf-yaml". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:27:59 crc kubenswrapper[5052]: I1203 12:27:59.380927 5052 patch_prober.go:28] interesting pod/machine-config-daemon-74bj6 container/machine-config-daemon namespace/openshift-machine-config-operator: Liveness probe status=failure output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" start-of-body= Dec 03 12:27:59 crc kubenswrapper[5052]: I1203 12:27:59.380991 5052 prober.go:107] "Probe failed" probeType="Liveness" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" containerName="machine-config-daemon" probeResult="failure" output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" Dec 03 12:27:59 crc kubenswrapper[5052]: I1203 12:27:59.396744 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/kube-state-metrics-0"] Dec 03 12:27:59 crc kubenswrapper[5052]: I1203 12:27:59.403275 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/5d3294b3-d25c-4d8a-ba28-aaad7385693b-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:59 crc kubenswrapper[5052]: I1203 12:27:59.403324 5052 reconciler_common.go:293] "Volume detached for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/5d3294b3-d25c-4d8a-ba28-aaad7385693b-log-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:59 crc kubenswrapper[5052]: I1203 12:27:59.403342 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-sgtpx\" (UniqueName: \"kubernetes.io/projected/5d3294b3-d25c-4d8a-ba28-aaad7385693b-kube-api-access-sgtpx\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:59 crc kubenswrapper[5052]: I1203 12:27:59.403354 5052 reconciler_common.go:293] "Volume detached for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/5d3294b3-d25c-4d8a-ba28-aaad7385693b-sg-core-conf-yaml\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:59 crc kubenswrapper[5052]: I1203 12:27:59.411712 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/kube-state-metrics-0"] Dec 03 12:27:59 crc kubenswrapper[5052]: I1203 12:27:59.450207 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/kube-state-metrics-0"] Dec 03 12:27:59 crc kubenswrapper[5052]: E1203 12:27:59.451129 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="5d3294b3-d25c-4d8a-ba28-aaad7385693b" containerName="ceilometer-notification-agent" Dec 03 12:27:59 crc kubenswrapper[5052]: I1203 12:27:59.451145 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="5d3294b3-d25c-4d8a-ba28-aaad7385693b" containerName="ceilometer-notification-agent" Dec 03 12:27:59 crc kubenswrapper[5052]: E1203 12:27:59.451165 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="5d3294b3-d25c-4d8a-ba28-aaad7385693b" containerName="ceilometer-central-agent" Dec 03 12:27:59 crc kubenswrapper[5052]: I1203 12:27:59.451170 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="5d3294b3-d25c-4d8a-ba28-aaad7385693b" containerName="ceilometer-central-agent" Dec 03 12:27:59 crc kubenswrapper[5052]: E1203 12:27:59.451191 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="5d3294b3-d25c-4d8a-ba28-aaad7385693b" containerName="proxy-httpd" Dec 03 12:27:59 crc kubenswrapper[5052]: I1203 12:27:59.451198 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="5d3294b3-d25c-4d8a-ba28-aaad7385693b" containerName="proxy-httpd" Dec 03 12:27:59 crc kubenswrapper[5052]: E1203 12:27:59.451222 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="61588a52-3ab5-45a4-a595-3b13bb4bcc28" containerName="kube-state-metrics" Dec 03 12:27:59 crc kubenswrapper[5052]: I1203 12:27:59.451229 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="61588a52-3ab5-45a4-a595-3b13bb4bcc28" containerName="kube-state-metrics" Dec 03 12:27:59 crc kubenswrapper[5052]: E1203 12:27:59.451247 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="5d3294b3-d25c-4d8a-ba28-aaad7385693b" containerName="sg-core" Dec 03 12:27:59 crc kubenswrapper[5052]: I1203 12:27:59.451253 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="5d3294b3-d25c-4d8a-ba28-aaad7385693b" containerName="sg-core" Dec 03 12:27:59 crc kubenswrapper[5052]: I1203 12:27:59.451524 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="5d3294b3-d25c-4d8a-ba28-aaad7385693b" containerName="proxy-httpd" Dec 03 12:27:59 crc kubenswrapper[5052]: I1203 12:27:59.451542 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="5d3294b3-d25c-4d8a-ba28-aaad7385693b" containerName="ceilometer-notification-agent" Dec 03 12:27:59 crc kubenswrapper[5052]: I1203 12:27:59.451556 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="5d3294b3-d25c-4d8a-ba28-aaad7385693b" containerName="ceilometer-central-agent" Dec 03 12:27:59 crc kubenswrapper[5052]: I1203 12:27:59.451579 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="61588a52-3ab5-45a4-a595-3b13bb4bcc28" containerName="kube-state-metrics" Dec 03 12:27:59 crc kubenswrapper[5052]: I1203 12:27:59.451588 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="5d3294b3-d25c-4d8a-ba28-aaad7385693b" containerName="sg-core" Dec 03 12:27:59 crc kubenswrapper[5052]: I1203 12:27:59.452544 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:27:59 crc kubenswrapper[5052]: I1203 12:27:59.455978 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"kube-state-metrics-tls-config" Dec 03 12:27:59 crc kubenswrapper[5052]: I1203 12:27:59.459955 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-kube-state-metrics-svc" Dec 03 12:27:59 crc kubenswrapper[5052]: I1203 12:27:59.475743 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-bootstrap-8nbqq"] Dec 03 12:27:59 crc kubenswrapper[5052]: I1203 12:27:59.481860 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5d3294b3-d25c-4d8a-ba28-aaad7385693b-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "5d3294b3-d25c-4d8a-ba28-aaad7385693b" (UID: "5d3294b3-d25c-4d8a-ba28-aaad7385693b"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:27:59 crc kubenswrapper[5052]: I1203 12:27:59.506668 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5d3294b3-d25c-4d8a-ba28-aaad7385693b-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:59 crc kubenswrapper[5052]: I1203 12:27:59.527189 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/kube-state-metrics-0"] Dec 03 12:27:59 crc kubenswrapper[5052]: I1203 12:27:59.582135 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5d3294b3-d25c-4d8a-ba28-aaad7385693b-config-data" (OuterVolumeSpecName: "config-data") pod "5d3294b3-d25c-4d8a-ba28-aaad7385693b" (UID: "5d3294b3-d25c-4d8a-ba28-aaad7385693b"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:27:59 crc kubenswrapper[5052]: I1203 12:27:59.610817 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/44520845-0277-47ab-acc6-812aba4656ff-combined-ca-bundle\") pod \"kube-state-metrics-0\" (UID: \"44520845-0277-47ab-acc6-812aba4656ff\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:27:59 crc kubenswrapper[5052]: I1203 12:27:59.611213 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ht8f4\" (UniqueName: \"kubernetes.io/projected/44520845-0277-47ab-acc6-812aba4656ff-kube-api-access-ht8f4\") pod \"kube-state-metrics-0\" (UID: \"44520845-0277-47ab-acc6-812aba4656ff\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:27:59 crc kubenswrapper[5052]: I1203 12:27:59.611254 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls-certs\" (UniqueName: \"kubernetes.io/secret/44520845-0277-47ab-acc6-812aba4656ff-kube-state-metrics-tls-certs\") pod \"kube-state-metrics-0\" (UID: \"44520845-0277-47ab-acc6-812aba4656ff\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:27:59 crc kubenswrapper[5052]: I1203 12:27:59.611280 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls-config\" (UniqueName: \"kubernetes.io/secret/44520845-0277-47ab-acc6-812aba4656ff-kube-state-metrics-tls-config\") pod \"kube-state-metrics-0\" (UID: \"44520845-0277-47ab-acc6-812aba4656ff\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:27:59 crc kubenswrapper[5052]: I1203 12:27:59.611392 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/5d3294b3-d25c-4d8a-ba28-aaad7385693b-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:27:59 crc kubenswrapper[5052]: I1203 12:27:59.660190 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-56ff9fdb74-cft5n"] Dec 03 12:27:59 crc kubenswrapper[5052]: I1203 12:27:59.691526 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3a27c483-00fc-4b06-af6b-3c4b0ff8d0d1" path="/var/lib/kubelet/pods/3a27c483-00fc-4b06-af6b-3c4b0ff8d0d1/volumes" Dec 03 12:27:59 crc kubenswrapper[5052]: I1203 12:27:59.692195 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="61588a52-3ab5-45a4-a595-3b13bb4bcc28" path="/var/lib/kubelet/pods/61588a52-3ab5-45a4-a595-3b13bb4bcc28/volumes" Dec 03 12:27:59 crc kubenswrapper[5052]: I1203 12:27:59.712954 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/44520845-0277-47ab-acc6-812aba4656ff-combined-ca-bundle\") pod \"kube-state-metrics-0\" (UID: \"44520845-0277-47ab-acc6-812aba4656ff\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:27:59 crc kubenswrapper[5052]: I1203 12:27:59.713073 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-ht8f4\" (UniqueName: \"kubernetes.io/projected/44520845-0277-47ab-acc6-812aba4656ff-kube-api-access-ht8f4\") pod \"kube-state-metrics-0\" (UID: \"44520845-0277-47ab-acc6-812aba4656ff\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:27:59 crc kubenswrapper[5052]: I1203 12:27:59.713131 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls-certs\" (UniqueName: \"kubernetes.io/secret/44520845-0277-47ab-acc6-812aba4656ff-kube-state-metrics-tls-certs\") pod \"kube-state-metrics-0\" (UID: \"44520845-0277-47ab-acc6-812aba4656ff\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:27:59 crc kubenswrapper[5052]: I1203 12:27:59.713162 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls-config\" (UniqueName: \"kubernetes.io/secret/44520845-0277-47ab-acc6-812aba4656ff-kube-state-metrics-tls-config\") pod \"kube-state-metrics-0\" (UID: \"44520845-0277-47ab-acc6-812aba4656ff\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:27:59 crc kubenswrapper[5052]: I1203 12:27:59.718119 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls-config\" (UniqueName: \"kubernetes.io/secret/44520845-0277-47ab-acc6-812aba4656ff-kube-state-metrics-tls-config\") pod \"kube-state-metrics-0\" (UID: \"44520845-0277-47ab-acc6-812aba4656ff\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:27:59 crc kubenswrapper[5052]: I1203 12:27:59.725224 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls-certs\" (UniqueName: \"kubernetes.io/secret/44520845-0277-47ab-acc6-812aba4656ff-kube-state-metrics-tls-certs\") pod \"kube-state-metrics-0\" (UID: \"44520845-0277-47ab-acc6-812aba4656ff\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:27:59 crc kubenswrapper[5052]: I1203 12:27:59.725442 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/44520845-0277-47ab-acc6-812aba4656ff-combined-ca-bundle\") pod \"kube-state-metrics-0\" (UID: \"44520845-0277-47ab-acc6-812aba4656ff\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:27:59 crc kubenswrapper[5052]: I1203 12:27:59.734906 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-ht8f4\" (UniqueName: \"kubernetes.io/projected/44520845-0277-47ab-acc6-812aba4656ff-kube-api-access-ht8f4\") pod \"kube-state-metrics-0\" (UID: \"44520845-0277-47ab-acc6-812aba4656ff\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:27:59 crc kubenswrapper[5052]: I1203 12:27:59.820104 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:28:00 crc kubenswrapper[5052]: I1203 12:28:00.094893 5052 generic.go:334] "Generic (PLEG): container finished" podID="2c03f99a-98fd-4e86-ad29-fb69e94bd436" containerID="e1e47b47265fb722fd8606f3ed96e8cbc15ed3bc36658d9fdd76dc06cbe471c8" exitCode=0 Dec 03 12:28:00 crc kubenswrapper[5052]: I1203 12:28:00.095171 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-server-0" event={"ID":"2c03f99a-98fd-4e86-ad29-fb69e94bd436","Type":"ContainerDied","Data":"e1e47b47265fb722fd8606f3ed96e8cbc15ed3bc36658d9fdd76dc06cbe471c8"} Dec 03 12:28:00 crc kubenswrapper[5052]: I1203 12:28:00.125496 5052 generic.go:334] "Generic (PLEG): container finished" podID="0b3a2637-ed0d-4348-ac7e-c92db6503191" containerID="35e2dee61a61ea5e56329f496973eb6a0f2fe0619c6f0df2e55caadaea334f93" exitCode=143 Dec 03 12:28:00 crc kubenswrapper[5052]: I1203 12:28:00.125618 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"0b3a2637-ed0d-4348-ac7e-c92db6503191","Type":"ContainerDied","Data":"35e2dee61a61ea5e56329f496973eb6a0f2fe0619c6f0df2e55caadaea334f93"} Dec 03 12:28:00 crc kubenswrapper[5052]: I1203 12:28:00.130210 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"5d3294b3-d25c-4d8a-ba28-aaad7385693b","Type":"ContainerDied","Data":"64f15a43acc40c1b2d84b1728c906a2170144700373881238dc5e3bbe90f757b"} Dec 03 12:28:00 crc kubenswrapper[5052]: I1203 12:28:00.130263 5052 scope.go:117] "RemoveContainer" containerID="7e8b797c3e63fa66aad0206d3411e34757b19c124c459e9c5d9dc7e3133dad59" Dec 03 12:28:00 crc kubenswrapper[5052]: I1203 12:28:00.130429 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:28:00 crc kubenswrapper[5052]: I1203 12:28:00.135473 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-56ff9fdb74-cft5n" event={"ID":"5c13362a-3cdf-4ff3-b57d-eb8c6cdcdc98","Type":"ContainerStarted","Data":"7eb375afddb95b73f21f6b1d166ca117940d6174ccbd794be68d71b6d2fd2a74"} Dec 03 12:28:00 crc kubenswrapper[5052]: I1203 12:28:00.135632 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-56ff9fdb74-cft5n" event={"ID":"5c13362a-3cdf-4ff3-b57d-eb8c6cdcdc98","Type":"ContainerStarted","Data":"519ff8f87376a328beb7fed38c294edc36f7f254b41c60c34eefe2f7a8ebde99"} Dec 03 12:28:00 crc kubenswrapper[5052]: I1203 12:28:00.143576 5052 generic.go:334] "Generic (PLEG): container finished" podID="2b5843e3-84ff-4174-bd4a-f67ba2a3a238" containerID="667a1283a5ee29a96d6e7cde347d08e4fba330b96f0a08eb1f51d024fa8e92ed" exitCode=143 Dec 03 12:28:00 crc kubenswrapper[5052]: I1203 12:28:00.143649 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"2b5843e3-84ff-4174-bd4a-f67ba2a3a238","Type":"ContainerDied","Data":"667a1283a5ee29a96d6e7cde347d08e4fba330b96f0a08eb1f51d024fa8e92ed"} Dec 03 12:28:00 crc kubenswrapper[5052]: I1203 12:28:00.146430 5052 generic.go:334] "Generic (PLEG): container finished" podID="71e0e74b-5d01-467f-afb2-7e55d9d58297" containerID="6e64026bbfc0cc36204875e424068283a0de98c7caf69823427bc926fa18e20d" exitCode=0 Dec 03 12:28:00 crc kubenswrapper[5052]: I1203 12:28:00.146490 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"71e0e74b-5d01-467f-afb2-7e55d9d58297","Type":"ContainerDied","Data":"6e64026bbfc0cc36204875e424068283a0de98c7caf69823427bc926fa18e20d"} Dec 03 12:28:00 crc kubenswrapper[5052]: I1203 12:28:00.182420 5052 generic.go:334] "Generic (PLEG): container finished" podID="6436bca9-c7a1-4540-bee3-b6dbabb53c89" containerID="584f69d542c77d9dd7acd735af74068b5eca3f64ae82019e8afdda51237d1117" exitCode=0 Dec 03 12:28:00 crc kubenswrapper[5052]: I1203 12:28:00.182571 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"6436bca9-c7a1-4540-bee3-b6dbabb53c89","Type":"ContainerDied","Data":"584f69d542c77d9dd7acd735af74068b5eca3f64ae82019e8afdda51237d1117"} Dec 03 12:28:00 crc kubenswrapper[5052]: I1203 12:28:00.185122 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_neutron-6bf55bd9d6-mrgs9_15cf6575-b6be-4f67-861a-61ebf02d1495/neutron-api/1.log" Dec 03 12:28:00 crc kubenswrapper[5052]: I1203 12:28:00.185492 5052 generic.go:334] "Generic (PLEG): container finished" podID="15cf6575-b6be-4f67-861a-61ebf02d1495" containerID="b4a9d01e3dc994f108359d2cce819e8f260555e99eb3dff86f96ba5dc6edd4b0" exitCode=0 Dec 03 12:28:00 crc kubenswrapper[5052]: I1203 12:28:00.185576 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-6bf55bd9d6-mrgs9" event={"ID":"15cf6575-b6be-4f67-861a-61ebf02d1495","Type":"ContainerDied","Data":"b4a9d01e3dc994f108359d2cce819e8f260555e99eb3dff86f96ba5dc6edd4b0"} Dec 03 12:28:00 crc kubenswrapper[5052]: I1203 12:28:00.242816 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-bootstrap-8nbqq" event={"ID":"ee394270-24f0-4687-9129-a5937efe9826","Type":"ContainerStarted","Data":"0680111a7796404e507144aefcc56ae790dfd5e497642c8b08e59f43944a5d46"} Dec 03 12:28:00 crc kubenswrapper[5052]: I1203 12:28:00.242876 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-bootstrap-8nbqq" event={"ID":"ee394270-24f0-4687-9129-a5937efe9826","Type":"ContainerStarted","Data":"cb0d233f8370b05426b89cc1b9907e38d01ff16d1f15c8569343bc9b8aa39f5e"} Dec 03 12:28:00 crc kubenswrapper[5052]: E1203 12:28:00.249236 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="790991ac480a9e358efc31b18c1e765ec3b8f7c9c3386cf9c51c6dc7f800750c" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:28:00 crc kubenswrapper[5052]: I1203 12:28:00.273194 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/keystone-bootstrap-8nbqq" podStartSLOduration=2.273176152 podStartE2EDuration="2.273176152s" podCreationTimestamp="2025-12-03 12:27:58 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:28:00.27133807 +0000 UTC m=+4070.859579834" watchObservedRunningTime="2025-12-03 12:28:00.273176152 +0000 UTC m=+4070.861417916" Dec 03 12:28:00 crc kubenswrapper[5052]: E1203 12:28:00.301253 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="790991ac480a9e358efc31b18c1e765ec3b8f7c9c3386cf9c51c6dc7f800750c" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:28:00 crc kubenswrapper[5052]: E1203 12:28:00.316932 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="790991ac480a9e358efc31b18c1e765ec3b8f7c9c3386cf9c51c6dc7f800750c" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:28:00 crc kubenswrapper[5052]: E1203 12:28:00.317061 5052 prober.go:104] "Probe errored" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" probeType="Readiness" pod="openstack-kuttl-tests/nova-cell0-conductor-0" podUID="c080dc7f-df72-487f-b2c1-83b143d34784" containerName="nova-cell0-conductor-conductor" Dec 03 12:28:00 crc kubenswrapper[5052]: I1203 12:28:00.403947 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:28:00 crc kubenswrapper[5052]: I1203 12:28:00.434444 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:28:00 crc kubenswrapper[5052]: I1203 12:28:00.440270 5052 scope.go:117] "RemoveContainer" containerID="f1eb88cceb1d1670bec09abbba8e4706b62c84aa3f20a8a8d0f5e15a0d9bf509" Dec 03 12:28:00 crc kubenswrapper[5052]: I1203 12:28:00.477325 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:28:00 crc kubenswrapper[5052]: I1203 12:28:00.480628 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:28:00 crc kubenswrapper[5052]: I1203 12:28:00.485170 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-scripts" Dec 03 12:28:00 crc kubenswrapper[5052]: I1203 12:28:00.487070 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-config-data" Dec 03 12:28:00 crc kubenswrapper[5052]: I1203 12:28:00.490649 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-ceilometer-internal-svc" Dec 03 12:28:00 crc kubenswrapper[5052]: I1203 12:28:00.501980 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:28:00 crc kubenswrapper[5052]: I1203 12:28:00.505136 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:28:00 crc kubenswrapper[5052]: I1203 12:28:00.583115 5052 scope.go:117] "RemoveContainer" containerID="98b12c00821c1fa8f0a555940a026478afc3a124403bdbe42ea9907f1c09c593" Dec 03 12:28:00 crc kubenswrapper[5052]: I1203 12:28:00.610759 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_neutron-6bf55bd9d6-mrgs9_15cf6575-b6be-4f67-861a-61ebf02d1495/neutron-api/1.log" Dec 03 12:28:00 crc kubenswrapper[5052]: I1203 12:28:00.613660 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-6bf55bd9d6-mrgs9" Dec 03 12:28:00 crc kubenswrapper[5052]: I1203 12:28:00.645824 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/71e0e74b-5d01-467f-afb2-7e55d9d58297-config-data\") pod \"71e0e74b-5d01-467f-afb2-7e55d9d58297\" (UID: \"71e0e74b-5d01-467f-afb2-7e55d9d58297\") " Dec 03 12:28:00 crc kubenswrapper[5052]: I1203 12:28:00.645934 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/71e0e74b-5d01-467f-afb2-7e55d9d58297-combined-ca-bundle\") pod \"71e0e74b-5d01-467f-afb2-7e55d9d58297\" (UID: \"71e0e74b-5d01-467f-afb2-7e55d9d58297\") " Dec 03 12:28:00 crc kubenswrapper[5052]: I1203 12:28:00.645959 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/71e0e74b-5d01-467f-afb2-7e55d9d58297-logs\") pod \"71e0e74b-5d01-467f-afb2-7e55d9d58297\" (UID: \"71e0e74b-5d01-467f-afb2-7e55d9d58297\") " Dec 03 12:28:00 crc kubenswrapper[5052]: I1203 12:28:00.645987 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/71e0e74b-5d01-467f-afb2-7e55d9d58297-scripts\") pod \"71e0e74b-5d01-467f-afb2-7e55d9d58297\" (UID: \"71e0e74b-5d01-467f-afb2-7e55d9d58297\") " Dec 03 12:28:00 crc kubenswrapper[5052]: I1203 12:28:00.646015 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"glance\" (UniqueName: \"kubernetes.io/local-volume/local-storage07-crc\") pod \"71e0e74b-5d01-467f-afb2-7e55d9d58297\" (UID: \"71e0e74b-5d01-467f-afb2-7e55d9d58297\") " Dec 03 12:28:00 crc kubenswrapper[5052]: I1203 12:28:00.646038 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/71e0e74b-5d01-467f-afb2-7e55d9d58297-httpd-run\") pod \"71e0e74b-5d01-467f-afb2-7e55d9d58297\" (UID: \"71e0e74b-5d01-467f-afb2-7e55d9d58297\") " Dec 03 12:28:00 crc kubenswrapper[5052]: I1203 12:28:00.646224 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z74ch\" (UniqueName: \"kubernetes.io/projected/71e0e74b-5d01-467f-afb2-7e55d9d58297-kube-api-access-z74ch\") pod \"71e0e74b-5d01-467f-afb2-7e55d9d58297\" (UID: \"71e0e74b-5d01-467f-afb2-7e55d9d58297\") " Dec 03 12:28:00 crc kubenswrapper[5052]: I1203 12:28:00.646563 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/e7e769bc-cba8-4955-9d3d-14d1acc7fb5b-config-data\") pod \"ceilometer-0\" (UID: \"e7e769bc-cba8-4955-9d3d-14d1acc7fb5b\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:28:00 crc kubenswrapper[5052]: I1203 12:28:00.646597 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z6lcc\" (UniqueName: \"kubernetes.io/projected/e7e769bc-cba8-4955-9d3d-14d1acc7fb5b-kube-api-access-z6lcc\") pod \"ceilometer-0\" (UID: \"e7e769bc-cba8-4955-9d3d-14d1acc7fb5b\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:28:00 crc kubenswrapper[5052]: I1203 12:28:00.646674 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/e7e769bc-cba8-4955-9d3d-14d1acc7fb5b-ceilometer-tls-certs\") pod \"ceilometer-0\" (UID: \"e7e769bc-cba8-4955-9d3d-14d1acc7fb5b\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:28:00 crc kubenswrapper[5052]: I1203 12:28:00.646709 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/e7e769bc-cba8-4955-9d3d-14d1acc7fb5b-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"e7e769bc-cba8-4955-9d3d-14d1acc7fb5b\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:28:00 crc kubenswrapper[5052]: I1203 12:28:00.646756 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/e7e769bc-cba8-4955-9d3d-14d1acc7fb5b-scripts\") pod \"ceilometer-0\" (UID: \"e7e769bc-cba8-4955-9d3d-14d1acc7fb5b\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:28:00 crc kubenswrapper[5052]: I1203 12:28:00.646785 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/e7e769bc-cba8-4955-9d3d-14d1acc7fb5b-log-httpd\") pod \"ceilometer-0\" (UID: \"e7e769bc-cba8-4955-9d3d-14d1acc7fb5b\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:28:00 crc kubenswrapper[5052]: I1203 12:28:00.646815 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/e7e769bc-cba8-4955-9d3d-14d1acc7fb5b-run-httpd\") pod \"ceilometer-0\" (UID: \"e7e769bc-cba8-4955-9d3d-14d1acc7fb5b\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:28:00 crc kubenswrapper[5052]: I1203 12:28:00.646830 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e7e769bc-cba8-4955-9d3d-14d1acc7fb5b-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"e7e769bc-cba8-4955-9d3d-14d1acc7fb5b\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:28:00 crc kubenswrapper[5052]: I1203 12:28:00.675817 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71e0e74b-5d01-467f-afb2-7e55d9d58297-httpd-run" (OuterVolumeSpecName: "httpd-run") pod "71e0e74b-5d01-467f-afb2-7e55d9d58297" (UID: "71e0e74b-5d01-467f-afb2-7e55d9d58297"). InnerVolumeSpecName "httpd-run". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:28:00 crc kubenswrapper[5052]: I1203 12:28:00.676838 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71e0e74b-5d01-467f-afb2-7e55d9d58297-logs" (OuterVolumeSpecName: "logs") pod "71e0e74b-5d01-467f-afb2-7e55d9d58297" (UID: "71e0e74b-5d01-467f-afb2-7e55d9d58297"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:28:00 crc kubenswrapper[5052]: I1203 12:28:00.677779 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage07-crc" (OuterVolumeSpecName: "glance") pod "71e0e74b-5d01-467f-afb2-7e55d9d58297" (UID: "71e0e74b-5d01-467f-afb2-7e55d9d58297"). InnerVolumeSpecName "local-storage07-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 12:28:00 crc kubenswrapper[5052]: I1203 12:28:00.690156 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/kube-state-metrics-0"] Dec 03 12:28:00 crc kubenswrapper[5052]: I1203 12:28:00.705752 5052 scope.go:117] "RemoveContainer" containerID="518e554556294e8e06ab1671e9f2b277371917fe0bafb7f6898e763ce48bb5e1" Dec 03 12:28:00 crc kubenswrapper[5052]: I1203 12:28:00.707255 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/71e0e74b-5d01-467f-afb2-7e55d9d58297-kube-api-access-z74ch" (OuterVolumeSpecName: "kube-api-access-z74ch") pod "71e0e74b-5d01-467f-afb2-7e55d9d58297" (UID: "71e0e74b-5d01-467f-afb2-7e55d9d58297"). InnerVolumeSpecName "kube-api-access-z74ch". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:28:00 crc kubenswrapper[5052]: I1203 12:28:00.710750 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/71e0e74b-5d01-467f-afb2-7e55d9d58297-scripts" (OuterVolumeSpecName: "scripts") pod "71e0e74b-5d01-467f-afb2-7e55d9d58297" (UID: "71e0e74b-5d01-467f-afb2-7e55d9d58297"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:28:00 crc kubenswrapper[5052]: I1203 12:28:00.750555 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xjczt\" (UniqueName: \"kubernetes.io/projected/15cf6575-b6be-4f67-861a-61ebf02d1495-kube-api-access-xjczt\") pod \"15cf6575-b6be-4f67-861a-61ebf02d1495\" (UID: \"15cf6575-b6be-4f67-861a-61ebf02d1495\") " Dec 03 12:28:00 crc kubenswrapper[5052]: I1203 12:28:00.750632 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/15cf6575-b6be-4f67-861a-61ebf02d1495-config\") pod \"15cf6575-b6be-4f67-861a-61ebf02d1495\" (UID: \"15cf6575-b6be-4f67-861a-61ebf02d1495\") " Dec 03 12:28:00 crc kubenswrapper[5052]: I1203 12:28:00.750723 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/15cf6575-b6be-4f67-861a-61ebf02d1495-combined-ca-bundle\") pod \"15cf6575-b6be-4f67-861a-61ebf02d1495\" (UID: \"15cf6575-b6be-4f67-861a-61ebf02d1495\") " Dec 03 12:28:00 crc kubenswrapper[5052]: I1203 12:28:00.750776 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/15cf6575-b6be-4f67-861a-61ebf02d1495-httpd-config\") pod \"15cf6575-b6be-4f67-861a-61ebf02d1495\" (UID: \"15cf6575-b6be-4f67-861a-61ebf02d1495\") " Dec 03 12:28:00 crc kubenswrapper[5052]: I1203 12:28:00.751093 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/e7e769bc-cba8-4955-9d3d-14d1acc7fb5b-ceilometer-tls-certs\") pod \"ceilometer-0\" (UID: \"e7e769bc-cba8-4955-9d3d-14d1acc7fb5b\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:28:00 crc kubenswrapper[5052]: I1203 12:28:00.751183 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/e7e769bc-cba8-4955-9d3d-14d1acc7fb5b-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"e7e769bc-cba8-4955-9d3d-14d1acc7fb5b\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:28:00 crc kubenswrapper[5052]: I1203 12:28:00.751275 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/e7e769bc-cba8-4955-9d3d-14d1acc7fb5b-scripts\") pod \"ceilometer-0\" (UID: \"e7e769bc-cba8-4955-9d3d-14d1acc7fb5b\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:28:00 crc kubenswrapper[5052]: I1203 12:28:00.751331 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/e7e769bc-cba8-4955-9d3d-14d1acc7fb5b-log-httpd\") pod \"ceilometer-0\" (UID: \"e7e769bc-cba8-4955-9d3d-14d1acc7fb5b\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:28:00 crc kubenswrapper[5052]: I1203 12:28:00.751372 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/e7e769bc-cba8-4955-9d3d-14d1acc7fb5b-run-httpd\") pod \"ceilometer-0\" (UID: \"e7e769bc-cba8-4955-9d3d-14d1acc7fb5b\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:28:00 crc kubenswrapper[5052]: I1203 12:28:00.751412 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e7e769bc-cba8-4955-9d3d-14d1acc7fb5b-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"e7e769bc-cba8-4955-9d3d-14d1acc7fb5b\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:28:00 crc kubenswrapper[5052]: I1203 12:28:00.751488 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/e7e769bc-cba8-4955-9d3d-14d1acc7fb5b-config-data\") pod \"ceilometer-0\" (UID: \"e7e769bc-cba8-4955-9d3d-14d1acc7fb5b\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:28:00 crc kubenswrapper[5052]: I1203 12:28:00.751550 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-z6lcc\" (UniqueName: \"kubernetes.io/projected/e7e769bc-cba8-4955-9d3d-14d1acc7fb5b-kube-api-access-z6lcc\") pod \"ceilometer-0\" (UID: \"e7e769bc-cba8-4955-9d3d-14d1acc7fb5b\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:28:00 crc kubenswrapper[5052]: I1203 12:28:00.751729 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/71e0e74b-5d01-467f-afb2-7e55d9d58297-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:00 crc kubenswrapper[5052]: I1203 12:28:00.751743 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/71e0e74b-5d01-467f-afb2-7e55d9d58297-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:00 crc kubenswrapper[5052]: I1203 12:28:00.751786 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage07-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage07-crc\") on node \"crc\" " Dec 03 12:28:00 crc kubenswrapper[5052]: I1203 12:28:00.751801 5052 reconciler_common.go:293] "Volume detached for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/71e0e74b-5d01-467f-afb2-7e55d9d58297-httpd-run\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:00 crc kubenswrapper[5052]: I1203 12:28:00.751813 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-z74ch\" (UniqueName: \"kubernetes.io/projected/71e0e74b-5d01-467f-afb2-7e55d9d58297-kube-api-access-z74ch\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:00 crc kubenswrapper[5052]: I1203 12:28:00.758926 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/e7e769bc-cba8-4955-9d3d-14d1acc7fb5b-log-httpd\") pod \"ceilometer-0\" (UID: \"e7e769bc-cba8-4955-9d3d-14d1acc7fb5b\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:28:00 crc kubenswrapper[5052]: I1203 12:28:00.764299 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/e7e769bc-cba8-4955-9d3d-14d1acc7fb5b-run-httpd\") pod \"ceilometer-0\" (UID: \"e7e769bc-cba8-4955-9d3d-14d1acc7fb5b\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:28:00 crc kubenswrapper[5052]: I1203 12:28:00.791700 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/15cf6575-b6be-4f67-861a-61ebf02d1495-kube-api-access-xjczt" (OuterVolumeSpecName: "kube-api-access-xjczt") pod "15cf6575-b6be-4f67-861a-61ebf02d1495" (UID: "15cf6575-b6be-4f67-861a-61ebf02d1495"). InnerVolumeSpecName "kube-api-access-xjczt". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:28:00 crc kubenswrapper[5052]: I1203 12:28:00.802703 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/e7e769bc-cba8-4955-9d3d-14d1acc7fb5b-scripts\") pod \"ceilometer-0\" (UID: \"e7e769bc-cba8-4955-9d3d-14d1acc7fb5b\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:28:00 crc kubenswrapper[5052]: I1203 12:28:00.802839 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/15cf6575-b6be-4f67-861a-61ebf02d1495-httpd-config" (OuterVolumeSpecName: "httpd-config") pod "15cf6575-b6be-4f67-861a-61ebf02d1495" (UID: "15cf6575-b6be-4f67-861a-61ebf02d1495"). InnerVolumeSpecName "httpd-config". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:28:00 crc kubenswrapper[5052]: I1203 12:28:00.803324 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/e7e769bc-cba8-4955-9d3d-14d1acc7fb5b-ceilometer-tls-certs\") pod \"ceilometer-0\" (UID: \"e7e769bc-cba8-4955-9d3d-14d1acc7fb5b\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:28:00 crc kubenswrapper[5052]: I1203 12:28:00.803565 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/e7e769bc-cba8-4955-9d3d-14d1acc7fb5b-config-data\") pod \"ceilometer-0\" (UID: \"e7e769bc-cba8-4955-9d3d-14d1acc7fb5b\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:28:00 crc kubenswrapper[5052]: I1203 12:28:00.803996 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e7e769bc-cba8-4955-9d3d-14d1acc7fb5b-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"e7e769bc-cba8-4955-9d3d-14d1acc7fb5b\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:28:00 crc kubenswrapper[5052]: I1203 12:28:00.806502 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/e7e769bc-cba8-4955-9d3d-14d1acc7fb5b-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"e7e769bc-cba8-4955-9d3d-14d1acc7fb5b\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:28:00 crc kubenswrapper[5052]: I1203 12:28:00.821611 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-z6lcc\" (UniqueName: \"kubernetes.io/projected/e7e769bc-cba8-4955-9d3d-14d1acc7fb5b-kube-api-access-z6lcc\") pod \"ceilometer-0\" (UID: \"e7e769bc-cba8-4955-9d3d-14d1acc7fb5b\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:28:00 crc kubenswrapper[5052]: I1203 12:28:00.843648 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:28:00 crc kubenswrapper[5052]: I1203 12:28:00.849004 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/71e0e74b-5d01-467f-afb2-7e55d9d58297-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "71e0e74b-5d01-467f-afb2-7e55d9d58297" (UID: "71e0e74b-5d01-467f-afb2-7e55d9d58297"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:28:00 crc kubenswrapper[5052]: I1203 12:28:00.853423 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:28:00 crc kubenswrapper[5052]: I1203 12:28:00.861136 5052 reconciler_common.go:293] "Volume detached for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/15cf6575-b6be-4f67-861a-61ebf02d1495-httpd-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:00 crc kubenswrapper[5052]: I1203 12:28:00.861408 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/71e0e74b-5d01-467f-afb2-7e55d9d58297-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:00 crc kubenswrapper[5052]: I1203 12:28:00.861468 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-xjczt\" (UniqueName: \"kubernetes.io/projected/15cf6575-b6be-4f67-861a-61ebf02d1495-kube-api-access-xjczt\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:00 crc kubenswrapper[5052]: I1203 12:28:00.864501 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage07-crc" (UniqueName: "kubernetes.io/local-volume/local-storage07-crc") on node "crc" Dec 03 12:28:00 crc kubenswrapper[5052]: I1203 12:28:00.876075 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/71e0e74b-5d01-467f-afb2-7e55d9d58297-config-data" (OuterVolumeSpecName: "config-data") pod "71e0e74b-5d01-467f-afb2-7e55d9d58297" (UID: "71e0e74b-5d01-467f-afb2-7e55d9d58297"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:28:00 crc kubenswrapper[5052]: I1203 12:28:00.956848 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/15cf6575-b6be-4f67-861a-61ebf02d1495-config" (OuterVolumeSpecName: "config") pod "15cf6575-b6be-4f67-861a-61ebf02d1495" (UID: "15cf6575-b6be-4f67-861a-61ebf02d1495"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:28:00 crc kubenswrapper[5052]: I1203 12:28:00.963084 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/2b5843e3-84ff-4174-bd4a-f67ba2a3a238-logs\") pod \"2b5843e3-84ff-4174-bd4a-f67ba2a3a238\" (UID: \"2b5843e3-84ff-4174-bd4a-f67ba2a3a238\") " Dec 03 12:28:00 crc kubenswrapper[5052]: I1203 12:28:00.963691 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/2b5843e3-84ff-4174-bd4a-f67ba2a3a238-etc-machine-id\") pod \"2b5843e3-84ff-4174-bd4a-f67ba2a3a238\" (UID: \"2b5843e3-84ff-4174-bd4a-f67ba2a3a238\") " Dec 03 12:28:00 crc kubenswrapper[5052]: I1203 12:28:00.963892 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/2b5843e3-84ff-4174-bd4a-f67ba2a3a238-scripts\") pod \"2b5843e3-84ff-4174-bd4a-f67ba2a3a238\" (UID: \"2b5843e3-84ff-4174-bd4a-f67ba2a3a238\") " Dec 03 12:28:00 crc kubenswrapper[5052]: I1203 12:28:00.964057 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mjwfk\" (UniqueName: \"kubernetes.io/projected/2b5843e3-84ff-4174-bd4a-f67ba2a3a238-kube-api-access-mjwfk\") pod \"2b5843e3-84ff-4174-bd4a-f67ba2a3a238\" (UID: \"2b5843e3-84ff-4174-bd4a-f67ba2a3a238\") " Dec 03 12:28:00 crc kubenswrapper[5052]: I1203 12:28:00.964149 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/2b5843e3-84ff-4174-bd4a-f67ba2a3a238-config-data\") pod \"2b5843e3-84ff-4174-bd4a-f67ba2a3a238\" (UID: \"2b5843e3-84ff-4174-bd4a-f67ba2a3a238\") " Dec 03 12:28:00 crc kubenswrapper[5052]: I1203 12:28:00.964307 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/2b5843e3-84ff-4174-bd4a-f67ba2a3a238-config-data-custom\") pod \"2b5843e3-84ff-4174-bd4a-f67ba2a3a238\" (UID: \"2b5843e3-84ff-4174-bd4a-f67ba2a3a238\") " Dec 03 12:28:00 crc kubenswrapper[5052]: I1203 12:28:00.964477 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2b5843e3-84ff-4174-bd4a-f67ba2a3a238-combined-ca-bundle\") pod \"2b5843e3-84ff-4174-bd4a-f67ba2a3a238\" (UID: \"2b5843e3-84ff-4174-bd4a-f67ba2a3a238\") " Dec 03 12:28:00 crc kubenswrapper[5052]: I1203 12:28:00.965372 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/71e0e74b-5d01-467f-afb2-7e55d9d58297-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:00 crc kubenswrapper[5052]: I1203 12:28:00.965468 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage07-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage07-crc\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:00 crc kubenswrapper[5052]: I1203 12:28:00.965616 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/15cf6575-b6be-4f67-861a-61ebf02d1495-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:00 crc kubenswrapper[5052]: I1203 12:28:00.966748 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/2b5843e3-84ff-4174-bd4a-f67ba2a3a238-etc-machine-id" (OuterVolumeSpecName: "etc-machine-id") pod "2b5843e3-84ff-4174-bd4a-f67ba2a3a238" (UID: "2b5843e3-84ff-4174-bd4a-f67ba2a3a238"). InnerVolumeSpecName "etc-machine-id". PluginName "kubernetes.io/host-path", VolumeGidValue "" Dec 03 12:28:00 crc kubenswrapper[5052]: I1203 12:28:00.967987 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2b5843e3-84ff-4174-bd4a-f67ba2a3a238-logs" (OuterVolumeSpecName: "logs") pod "2b5843e3-84ff-4174-bd4a-f67ba2a3a238" (UID: "2b5843e3-84ff-4174-bd4a-f67ba2a3a238"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:28:00 crc kubenswrapper[5052]: I1203 12:28:00.976553 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/15cf6575-b6be-4f67-861a-61ebf02d1495-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "15cf6575-b6be-4f67-861a-61ebf02d1495" (UID: "15cf6575-b6be-4f67-861a-61ebf02d1495"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:28:00 crc kubenswrapper[5052]: I1203 12:28:00.984731 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2b5843e3-84ff-4174-bd4a-f67ba2a3a238-scripts" (OuterVolumeSpecName: "scripts") pod "2b5843e3-84ff-4174-bd4a-f67ba2a3a238" (UID: "2b5843e3-84ff-4174-bd4a-f67ba2a3a238"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:28:01 crc kubenswrapper[5052]: I1203 12:28:01.005407 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2b5843e3-84ff-4174-bd4a-f67ba2a3a238-config-data-custom" (OuterVolumeSpecName: "config-data-custom") pod "2b5843e3-84ff-4174-bd4a-f67ba2a3a238" (UID: "2b5843e3-84ff-4174-bd4a-f67ba2a3a238"). InnerVolumeSpecName "config-data-custom". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:28:01 crc kubenswrapper[5052]: I1203 12:28:01.017408 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2b5843e3-84ff-4174-bd4a-f67ba2a3a238-kube-api-access-mjwfk" (OuterVolumeSpecName: "kube-api-access-mjwfk") pod "2b5843e3-84ff-4174-bd4a-f67ba2a3a238" (UID: "2b5843e3-84ff-4174-bd4a-f67ba2a3a238"). InnerVolumeSpecName "kube-api-access-mjwfk". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:28:01 crc kubenswrapper[5052]: I1203 12:28:01.072262 5052 reconciler_common.go:293] "Volume detached for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/2b5843e3-84ff-4174-bd4a-f67ba2a3a238-etc-machine-id\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:01 crc kubenswrapper[5052]: I1203 12:28:01.072293 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/2b5843e3-84ff-4174-bd4a-f67ba2a3a238-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:01 crc kubenswrapper[5052]: I1203 12:28:01.072307 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-mjwfk\" (UniqueName: \"kubernetes.io/projected/2b5843e3-84ff-4174-bd4a-f67ba2a3a238-kube-api-access-mjwfk\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:01 crc kubenswrapper[5052]: I1203 12:28:01.072320 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/2b5843e3-84ff-4174-bd4a-f67ba2a3a238-config-data-custom\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:01 crc kubenswrapper[5052]: I1203 12:28:01.072332 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/15cf6575-b6be-4f67-861a-61ebf02d1495-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:01 crc kubenswrapper[5052]: I1203 12:28:01.072344 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/2b5843e3-84ff-4174-bd4a-f67ba2a3a238-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:01 crc kubenswrapper[5052]: I1203 12:28:01.113069 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2b5843e3-84ff-4174-bd4a-f67ba2a3a238-config-data" (OuterVolumeSpecName: "config-data") pod "2b5843e3-84ff-4174-bd4a-f67ba2a3a238" (UID: "2b5843e3-84ff-4174-bd4a-f67ba2a3a238"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:28:01 crc kubenswrapper[5052]: I1203 12:28:01.124852 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2b5843e3-84ff-4174-bd4a-f67ba2a3a238-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "2b5843e3-84ff-4174-bd4a-f67ba2a3a238" (UID: "2b5843e3-84ff-4174-bd4a-f67ba2a3a238"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:28:01 crc kubenswrapper[5052]: I1203 12:28:01.174342 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2b5843e3-84ff-4174-bd4a-f67ba2a3a238-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:01 crc kubenswrapper[5052]: I1203 12:28:01.174389 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/2b5843e3-84ff-4174-bd4a-f67ba2a3a238-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:01 crc kubenswrapper[5052]: I1203 12:28:01.304167 5052 generic.go:334] "Generic (PLEG): container finished" podID="b8613a94-1e2e-477f-8d52-713c70f5282c" containerID="fdf73317a2bb37c60cc14493b2e7c924d803a06e05cbba2abbe853ddbf7c4fb6" exitCode=0 Dec 03 12:28:01 crc kubenswrapper[5052]: I1203 12:28:01.304522 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" event={"ID":"b8613a94-1e2e-477f-8d52-713c70f5282c","Type":"ContainerDied","Data":"fdf73317a2bb37c60cc14493b2e7c924d803a06e05cbba2abbe853ddbf7c4fb6"} Dec 03 12:28:01 crc kubenswrapper[5052]: I1203 12:28:01.323760 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/kube-state-metrics-0" event={"ID":"44520845-0277-47ab-acc6-812aba4656ff","Type":"ContainerStarted","Data":"fa0dca2ac03e31aa8e3e1bbce341f29f20144a5c8b32c9f9ac776cb15fc046ba"} Dec 03 12:28:01 crc kubenswrapper[5052]: I1203 12:28:01.357068 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-56ff9fdb74-cft5n" event={"ID":"5c13362a-3cdf-4ff3-b57d-eb8c6cdcdc98","Type":"ContainerStarted","Data":"136d74558e9fead2cd7cf270320c4f5f5b870699d1758c2cc84506b5c369b70e"} Dec 03 12:28:01 crc kubenswrapper[5052]: I1203 12:28:01.357568 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/neutron-56ff9fdb74-cft5n" Dec 03 12:28:01 crc kubenswrapper[5052]: I1203 12:28:01.369271 5052 generic.go:334] "Generic (PLEG): container finished" podID="2b5843e3-84ff-4174-bd4a-f67ba2a3a238" containerID="e07ed2652a16aa998baf13477cacedfad92c750d43fb52c99642ffc63d38a6cf" exitCode=0 Dec 03 12:28:01 crc kubenswrapper[5052]: I1203 12:28:01.369467 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"2b5843e3-84ff-4174-bd4a-f67ba2a3a238","Type":"ContainerDied","Data":"e07ed2652a16aa998baf13477cacedfad92c750d43fb52c99642ffc63d38a6cf"} Dec 03 12:28:01 crc kubenswrapper[5052]: I1203 12:28:01.369716 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"2b5843e3-84ff-4174-bd4a-f67ba2a3a238","Type":"ContainerDied","Data":"07dca5ea36e03e44babf3998118304eca164a251d9ef6d346c1411df5f02ad8c"} Dec 03 12:28:01 crc kubenswrapper[5052]: I1203 12:28:01.369808 5052 scope.go:117] "RemoveContainer" containerID="e07ed2652a16aa998baf13477cacedfad92c750d43fb52c99642ffc63d38a6cf" Dec 03 12:28:01 crc kubenswrapper[5052]: I1203 12:28:01.370046 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:28:01 crc kubenswrapper[5052]: I1203 12:28:01.386949 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/neutron-56ff9fdb74-cft5n" podStartSLOduration=3.386913011 podStartE2EDuration="3.386913011s" podCreationTimestamp="2025-12-03 12:27:58 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:28:01.383543635 +0000 UTC m=+4071.971785399" watchObservedRunningTime="2025-12-03 12:28:01.386913011 +0000 UTC m=+4071.975154785" Dec 03 12:28:01 crc kubenswrapper[5052]: I1203 12:28:01.396792 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-server-0" event={"ID":"2c03f99a-98fd-4e86-ad29-fb69e94bd436","Type":"ContainerStarted","Data":"33c00a517cac73b55f22b39dd9800951f3eddc4eb90411d1bc3e8a5477c02571"} Dec 03 12:28:01 crc kubenswrapper[5052]: I1203 12:28:01.399563 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:28:01 crc kubenswrapper[5052]: I1203 12:28:01.418406 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:28:01 crc kubenswrapper[5052]: I1203 12:28:01.419068 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"71e0e74b-5d01-467f-afb2-7e55d9d58297","Type":"ContainerDied","Data":"6c875eb0f775ff19c7b6e41484c54ec06ca19355ef5f39d9e1228da177fb721c"} Dec 03 12:28:01 crc kubenswrapper[5052]: I1203 12:28:01.430899 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_neutron-6bf55bd9d6-mrgs9_15cf6575-b6be-4f67-861a-61ebf02d1495/neutron-api/1.log" Dec 03 12:28:01 crc kubenswrapper[5052]: I1203 12:28:01.438901 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-6bf55bd9d6-mrgs9" event={"ID":"15cf6575-b6be-4f67-861a-61ebf02d1495","Type":"ContainerDied","Data":"d5ae8e4de2be17ff4124fe42924a487acef730e0b664ce364b3a370ad4fccabb"} Dec 03 12:28:01 crc kubenswrapper[5052]: I1203 12:28:01.439040 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-6bf55bd9d6-mrgs9" Dec 03 12:28:01 crc kubenswrapper[5052]: I1203 12:28:01.444627 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/rabbitmq-server-0" podStartSLOduration=39.444601627 podStartE2EDuration="39.444601627s" podCreationTimestamp="2025-12-03 12:27:22 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:28:01.42858415 +0000 UTC m=+4072.016825934" watchObservedRunningTime="2025-12-03 12:28:01.444601627 +0000 UTC m=+4072.032843391" Dec 03 12:28:01 crc kubenswrapper[5052]: I1203 12:28:01.478702 5052 scope.go:117] "RemoveContainer" containerID="667a1283a5ee29a96d6e7cde347d08e4fba330b96f0a08eb1f51d024fa8e92ed" Dec 03 12:28:01 crc kubenswrapper[5052]: I1203 12:28:01.545272 5052 scope.go:117] "RemoveContainer" containerID="e07ed2652a16aa998baf13477cacedfad92c750d43fb52c99642ffc63d38a6cf" Dec 03 12:28:01 crc kubenswrapper[5052]: E1203 12:28:01.560636 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e07ed2652a16aa998baf13477cacedfad92c750d43fb52c99642ffc63d38a6cf\": container with ID starting with e07ed2652a16aa998baf13477cacedfad92c750d43fb52c99642ffc63d38a6cf not found: ID does not exist" containerID="e07ed2652a16aa998baf13477cacedfad92c750d43fb52c99642ffc63d38a6cf" Dec 03 12:28:01 crc kubenswrapper[5052]: I1203 12:28:01.560706 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e07ed2652a16aa998baf13477cacedfad92c750d43fb52c99642ffc63d38a6cf"} err="failed to get container status \"e07ed2652a16aa998baf13477cacedfad92c750d43fb52c99642ffc63d38a6cf\": rpc error: code = NotFound desc = could not find container \"e07ed2652a16aa998baf13477cacedfad92c750d43fb52c99642ffc63d38a6cf\": container with ID starting with e07ed2652a16aa998baf13477cacedfad92c750d43fb52c99642ffc63d38a6cf not found: ID does not exist" Dec 03 12:28:01 crc kubenswrapper[5052]: I1203 12:28:01.560733 5052 scope.go:117] "RemoveContainer" containerID="667a1283a5ee29a96d6e7cde347d08e4fba330b96f0a08eb1f51d024fa8e92ed" Dec 03 12:28:01 crc kubenswrapper[5052]: E1203 12:28:01.561296 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"667a1283a5ee29a96d6e7cde347d08e4fba330b96f0a08eb1f51d024fa8e92ed\": container with ID starting with 667a1283a5ee29a96d6e7cde347d08e4fba330b96f0a08eb1f51d024fa8e92ed not found: ID does not exist" containerID="667a1283a5ee29a96d6e7cde347d08e4fba330b96f0a08eb1f51d024fa8e92ed" Dec 03 12:28:01 crc kubenswrapper[5052]: I1203 12:28:01.561338 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"667a1283a5ee29a96d6e7cde347d08e4fba330b96f0a08eb1f51d024fa8e92ed"} err="failed to get container status \"667a1283a5ee29a96d6e7cde347d08e4fba330b96f0a08eb1f51d024fa8e92ed\": rpc error: code = NotFound desc = could not find container \"667a1283a5ee29a96d6e7cde347d08e4fba330b96f0a08eb1f51d024fa8e92ed\": container with ID starting with 667a1283a5ee29a96d6e7cde347d08e4fba330b96f0a08eb1f51d024fa8e92ed not found: ID does not exist" Dec 03 12:28:01 crc kubenswrapper[5052]: I1203 12:28:01.561389 5052 scope.go:117] "RemoveContainer" containerID="6e64026bbfc0cc36204875e424068283a0de98c7caf69823427bc926fa18e20d" Dec 03 12:28:01 crc kubenswrapper[5052]: I1203 12:28:01.570978 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 12:28:01 crc kubenswrapper[5052]: I1203 12:28:01.591678 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 12:28:01 crc kubenswrapper[5052]: I1203 12:28:01.616720 5052 scope.go:117] "RemoveContainer" containerID="8bcc87cf614b2c81facd280057f18832930b2317f445a10147336b258adbe781" Dec 03 12:28:01 crc kubenswrapper[5052]: I1203 12:28:01.628620 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 12:28:01 crc kubenswrapper[5052]: E1203 12:28:01.629106 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="2b5843e3-84ff-4174-bd4a-f67ba2a3a238" containerName="cinder-api" Dec 03 12:28:01 crc kubenswrapper[5052]: I1203 12:28:01.629124 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="2b5843e3-84ff-4174-bd4a-f67ba2a3a238" containerName="cinder-api" Dec 03 12:28:01 crc kubenswrapper[5052]: E1203 12:28:01.629143 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="71e0e74b-5d01-467f-afb2-7e55d9d58297" containerName="glance-httpd" Dec 03 12:28:01 crc kubenswrapper[5052]: I1203 12:28:01.629150 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="71e0e74b-5d01-467f-afb2-7e55d9d58297" containerName="glance-httpd" Dec 03 12:28:01 crc kubenswrapper[5052]: E1203 12:28:01.629163 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="15cf6575-b6be-4f67-861a-61ebf02d1495" containerName="neutron-api" Dec 03 12:28:01 crc kubenswrapper[5052]: I1203 12:28:01.629169 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="15cf6575-b6be-4f67-861a-61ebf02d1495" containerName="neutron-api" Dec 03 12:28:01 crc kubenswrapper[5052]: E1203 12:28:01.629183 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="15cf6575-b6be-4f67-861a-61ebf02d1495" containerName="neutron-api" Dec 03 12:28:01 crc kubenswrapper[5052]: I1203 12:28:01.629189 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="15cf6575-b6be-4f67-861a-61ebf02d1495" containerName="neutron-api" Dec 03 12:28:01 crc kubenswrapper[5052]: E1203 12:28:01.629208 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="71e0e74b-5d01-467f-afb2-7e55d9d58297" containerName="glance-log" Dec 03 12:28:01 crc kubenswrapper[5052]: I1203 12:28:01.629214 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="71e0e74b-5d01-467f-afb2-7e55d9d58297" containerName="glance-log" Dec 03 12:28:01 crc kubenswrapper[5052]: E1203 12:28:01.629221 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="2b5843e3-84ff-4174-bd4a-f67ba2a3a238" containerName="cinder-api-log" Dec 03 12:28:01 crc kubenswrapper[5052]: I1203 12:28:01.629227 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="2b5843e3-84ff-4174-bd4a-f67ba2a3a238" containerName="cinder-api-log" Dec 03 12:28:01 crc kubenswrapper[5052]: E1203 12:28:01.629234 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="15cf6575-b6be-4f67-861a-61ebf02d1495" containerName="neutron-httpd" Dec 03 12:28:01 crc kubenswrapper[5052]: I1203 12:28:01.629239 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="15cf6575-b6be-4f67-861a-61ebf02d1495" containerName="neutron-httpd" Dec 03 12:28:01 crc kubenswrapper[5052]: I1203 12:28:01.629407 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="71e0e74b-5d01-467f-afb2-7e55d9d58297" containerName="glance-httpd" Dec 03 12:28:01 crc kubenswrapper[5052]: I1203 12:28:01.629417 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="15cf6575-b6be-4f67-861a-61ebf02d1495" containerName="neutron-api" Dec 03 12:28:01 crc kubenswrapper[5052]: I1203 12:28:01.629429 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="71e0e74b-5d01-467f-afb2-7e55d9d58297" containerName="glance-log" Dec 03 12:28:01 crc kubenswrapper[5052]: I1203 12:28:01.629442 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="2b5843e3-84ff-4174-bd4a-f67ba2a3a238" containerName="cinder-api-log" Dec 03 12:28:01 crc kubenswrapper[5052]: I1203 12:28:01.629455 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="15cf6575-b6be-4f67-861a-61ebf02d1495" containerName="neutron-api" Dec 03 12:28:01 crc kubenswrapper[5052]: I1203 12:28:01.629464 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="2b5843e3-84ff-4174-bd4a-f67ba2a3a238" containerName="cinder-api" Dec 03 12:28:01 crc kubenswrapper[5052]: I1203 12:28:01.629472 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="15cf6575-b6be-4f67-861a-61ebf02d1495" containerName="neutron-httpd" Dec 03 12:28:01 crc kubenswrapper[5052]: I1203 12:28:01.630569 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:28:01 crc kubenswrapper[5052]: I1203 12:28:01.642500 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cinder-api-config-data" Dec 03 12:28:01 crc kubenswrapper[5052]: I1203 12:28:01.642769 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-cinder-internal-svc" Dec 03 12:28:01 crc kubenswrapper[5052]: I1203 12:28:01.642910 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-cinder-public-svc" Dec 03 12:28:01 crc kubenswrapper[5052]: I1203 12:28:01.652937 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:28:01 crc kubenswrapper[5052]: W1203 12:28:01.661233 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode7e769bc_cba8_4955_9d3d_14d1acc7fb5b.slice/crio-19faf52ea0db2c5e7ec4c8c64930e9e227c325f2bbe38efaefc6c6c5aa244a80 WatchSource:0}: Error finding container 19faf52ea0db2c5e7ec4c8c64930e9e227c325f2bbe38efaefc6c6c5aa244a80: Status 404 returned error can't find the container with id 19faf52ea0db2c5e7ec4c8c64930e9e227c325f2bbe38efaefc6c6c5aa244a80 Dec 03 12:28:01 crc kubenswrapper[5052]: I1203 12:28:01.687352 5052 scope.go:117] "RemoveContainer" containerID="608a10c81b5afaba04aa84f9efa1c5579de39b1dc465fbf89ad2188610955b3b" Dec 03 12:28:01 crc kubenswrapper[5052]: I1203 12:28:01.687619 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/61cad274-57bd-4993-aaa6-3e6013900d88-etc-machine-id\") pod \"cinder-api-0\" (UID: \"61cad274-57bd-4993-aaa6-3e6013900d88\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:28:01 crc kubenswrapper[5052]: I1203 12:28:01.687682 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/61cad274-57bd-4993-aaa6-3e6013900d88-combined-ca-bundle\") pod \"cinder-api-0\" (UID: \"61cad274-57bd-4993-aaa6-3e6013900d88\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:28:01 crc kubenswrapper[5052]: I1203 12:28:01.687722 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/61cad274-57bd-4993-aaa6-3e6013900d88-logs\") pod \"cinder-api-0\" (UID: \"61cad274-57bd-4993-aaa6-3e6013900d88\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:28:01 crc kubenswrapper[5052]: I1203 12:28:01.687757 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vq44t\" (UniqueName: \"kubernetes.io/projected/61cad274-57bd-4993-aaa6-3e6013900d88-kube-api-access-vq44t\") pod \"cinder-api-0\" (UID: \"61cad274-57bd-4993-aaa6-3e6013900d88\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:28:01 crc kubenswrapper[5052]: I1203 12:28:01.687784 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/61cad274-57bd-4993-aaa6-3e6013900d88-public-tls-certs\") pod \"cinder-api-0\" (UID: \"61cad274-57bd-4993-aaa6-3e6013900d88\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:28:01 crc kubenswrapper[5052]: I1203 12:28:01.687870 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/61cad274-57bd-4993-aaa6-3e6013900d88-internal-tls-certs\") pod \"cinder-api-0\" (UID: \"61cad274-57bd-4993-aaa6-3e6013900d88\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:28:01 crc kubenswrapper[5052]: I1203 12:28:01.687887 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/61cad274-57bd-4993-aaa6-3e6013900d88-scripts\") pod \"cinder-api-0\" (UID: \"61cad274-57bd-4993-aaa6-3e6013900d88\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:28:01 crc kubenswrapper[5052]: I1203 12:28:01.687929 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/61cad274-57bd-4993-aaa6-3e6013900d88-config-data\") pod \"cinder-api-0\" (UID: \"61cad274-57bd-4993-aaa6-3e6013900d88\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:28:01 crc kubenswrapper[5052]: I1203 12:28:01.687966 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/61cad274-57bd-4993-aaa6-3e6013900d88-config-data-custom\") pod \"cinder-api-0\" (UID: \"61cad274-57bd-4993-aaa6-3e6013900d88\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:28:01 crc kubenswrapper[5052]: I1203 12:28:01.689975 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2b5843e3-84ff-4174-bd4a-f67ba2a3a238" path="/var/lib/kubelet/pods/2b5843e3-84ff-4174-bd4a-f67ba2a3a238/volumes" Dec 03 12:28:01 crc kubenswrapper[5052]: I1203 12:28:01.690810 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5d3294b3-d25c-4d8a-ba28-aaad7385693b" path="/var/lib/kubelet/pods/5d3294b3-d25c-4d8a-ba28-aaad7385693b/volumes" Dec 03 12:28:01 crc kubenswrapper[5052]: I1203 12:28:01.691813 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:28:01 crc kubenswrapper[5052]: I1203 12:28:01.693638 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 12:28:01 crc kubenswrapper[5052]: I1203 12:28:01.705745 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/neutron-6bf55bd9d6-mrgs9"] Dec 03 12:28:01 crc kubenswrapper[5052]: I1203 12:28:01.720205 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/neutron-6bf55bd9d6-mrgs9"] Dec 03 12:28:01 crc kubenswrapper[5052]: I1203 12:28:01.731665 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:28:01 crc kubenswrapper[5052]: I1203 12:28:01.733378 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:28:01 crc kubenswrapper[5052]: I1203 12:28:01.735838 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-glance-default-public-svc" Dec 03 12:28:01 crc kubenswrapper[5052]: I1203 12:28:01.735998 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-default-external-config-data" Dec 03 12:28:01 crc kubenswrapper[5052]: I1203 12:28:01.744366 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:28:01 crc kubenswrapper[5052]: I1203 12:28:01.771605 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:28:01 crc kubenswrapper[5052]: I1203 12:28:01.790060 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/61cad274-57bd-4993-aaa6-3e6013900d88-internal-tls-certs\") pod \"cinder-api-0\" (UID: \"61cad274-57bd-4993-aaa6-3e6013900d88\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:28:01 crc kubenswrapper[5052]: I1203 12:28:01.790141 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/61cad274-57bd-4993-aaa6-3e6013900d88-scripts\") pod \"cinder-api-0\" (UID: \"61cad274-57bd-4993-aaa6-3e6013900d88\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:28:01 crc kubenswrapper[5052]: I1203 12:28:01.790254 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/61cad274-57bd-4993-aaa6-3e6013900d88-config-data\") pod \"cinder-api-0\" (UID: \"61cad274-57bd-4993-aaa6-3e6013900d88\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:28:01 crc kubenswrapper[5052]: I1203 12:28:01.790322 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/61cad274-57bd-4993-aaa6-3e6013900d88-config-data-custom\") pod \"cinder-api-0\" (UID: \"61cad274-57bd-4993-aaa6-3e6013900d88\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:28:01 crc kubenswrapper[5052]: I1203 12:28:01.790373 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/61cad274-57bd-4993-aaa6-3e6013900d88-etc-machine-id\") pod \"cinder-api-0\" (UID: \"61cad274-57bd-4993-aaa6-3e6013900d88\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:28:01 crc kubenswrapper[5052]: I1203 12:28:01.790432 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/61cad274-57bd-4993-aaa6-3e6013900d88-combined-ca-bundle\") pod \"cinder-api-0\" (UID: \"61cad274-57bd-4993-aaa6-3e6013900d88\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:28:01 crc kubenswrapper[5052]: I1203 12:28:01.790465 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/61cad274-57bd-4993-aaa6-3e6013900d88-logs\") pod \"cinder-api-0\" (UID: \"61cad274-57bd-4993-aaa6-3e6013900d88\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:28:01 crc kubenswrapper[5052]: I1203 12:28:01.790532 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-vq44t\" (UniqueName: \"kubernetes.io/projected/61cad274-57bd-4993-aaa6-3e6013900d88-kube-api-access-vq44t\") pod \"cinder-api-0\" (UID: \"61cad274-57bd-4993-aaa6-3e6013900d88\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:28:01 crc kubenswrapper[5052]: I1203 12:28:01.790560 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/61cad274-57bd-4993-aaa6-3e6013900d88-public-tls-certs\") pod \"cinder-api-0\" (UID: \"61cad274-57bd-4993-aaa6-3e6013900d88\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:28:01 crc kubenswrapper[5052]: I1203 12:28:01.790971 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/61cad274-57bd-4993-aaa6-3e6013900d88-etc-machine-id\") pod \"cinder-api-0\" (UID: \"61cad274-57bd-4993-aaa6-3e6013900d88\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:28:01 crc kubenswrapper[5052]: I1203 12:28:01.793362 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/61cad274-57bd-4993-aaa6-3e6013900d88-logs\") pod \"cinder-api-0\" (UID: \"61cad274-57bd-4993-aaa6-3e6013900d88\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:28:01 crc kubenswrapper[5052]: I1203 12:28:01.794504 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/61cad274-57bd-4993-aaa6-3e6013900d88-internal-tls-certs\") pod \"cinder-api-0\" (UID: \"61cad274-57bd-4993-aaa6-3e6013900d88\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:28:01 crc kubenswrapper[5052]: I1203 12:28:01.797586 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/61cad274-57bd-4993-aaa6-3e6013900d88-public-tls-certs\") pod \"cinder-api-0\" (UID: \"61cad274-57bd-4993-aaa6-3e6013900d88\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:28:01 crc kubenswrapper[5052]: I1203 12:28:01.797926 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/61cad274-57bd-4993-aaa6-3e6013900d88-config-data\") pod \"cinder-api-0\" (UID: \"61cad274-57bd-4993-aaa6-3e6013900d88\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:28:01 crc kubenswrapper[5052]: I1203 12:28:01.798405 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/61cad274-57bd-4993-aaa6-3e6013900d88-scripts\") pod \"cinder-api-0\" (UID: \"61cad274-57bd-4993-aaa6-3e6013900d88\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:28:01 crc kubenswrapper[5052]: I1203 12:28:01.807952 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-vq44t\" (UniqueName: \"kubernetes.io/projected/61cad274-57bd-4993-aaa6-3e6013900d88-kube-api-access-vq44t\") pod \"cinder-api-0\" (UID: \"61cad274-57bd-4993-aaa6-3e6013900d88\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:28:01 crc kubenswrapper[5052]: I1203 12:28:01.808680 5052 scope.go:117] "RemoveContainer" containerID="b4a9d01e3dc994f108359d2cce819e8f260555e99eb3dff86f96ba5dc6edd4b0" Dec 03 12:28:01 crc kubenswrapper[5052]: I1203 12:28:01.808916 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/61cad274-57bd-4993-aaa6-3e6013900d88-config-data-custom\") pod \"cinder-api-0\" (UID: \"61cad274-57bd-4993-aaa6-3e6013900d88\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:28:01 crc kubenswrapper[5052]: I1203 12:28:01.812112 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/61cad274-57bd-4993-aaa6-3e6013900d88-combined-ca-bundle\") pod \"cinder-api-0\" (UID: \"61cad274-57bd-4993-aaa6-3e6013900d88\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:28:01 crc kubenswrapper[5052]: I1203 12:28:01.892467 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/05b9d3cc-93de-45a6-8849-027a4aad3715-combined-ca-bundle\") pod \"glance-default-external-api-0\" (UID: \"05b9d3cc-93de-45a6-8849-027a4aad3715\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:28:01 crc kubenswrapper[5052]: I1203 12:28:01.892650 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/05b9d3cc-93de-45a6-8849-027a4aad3715-config-data\") pod \"glance-default-external-api-0\" (UID: \"05b9d3cc-93de-45a6-8849-027a4aad3715\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:28:01 crc kubenswrapper[5052]: I1203 12:28:01.892681 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xvh8g\" (UniqueName: \"kubernetes.io/projected/05b9d3cc-93de-45a6-8849-027a4aad3715-kube-api-access-xvh8g\") pod \"glance-default-external-api-0\" (UID: \"05b9d3cc-93de-45a6-8849-027a4aad3715\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:28:01 crc kubenswrapper[5052]: I1203 12:28:01.892712 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/05b9d3cc-93de-45a6-8849-027a4aad3715-public-tls-certs\") pod \"glance-default-external-api-0\" (UID: \"05b9d3cc-93de-45a6-8849-027a4aad3715\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:28:01 crc kubenswrapper[5052]: I1203 12:28:01.892808 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/05b9d3cc-93de-45a6-8849-027a4aad3715-httpd-run\") pod \"glance-default-external-api-0\" (UID: \"05b9d3cc-93de-45a6-8849-027a4aad3715\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:28:01 crc kubenswrapper[5052]: I1203 12:28:01.893552 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/05b9d3cc-93de-45a6-8849-027a4aad3715-logs\") pod \"glance-default-external-api-0\" (UID: \"05b9d3cc-93de-45a6-8849-027a4aad3715\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:28:01 crc kubenswrapper[5052]: I1203 12:28:01.893579 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/05b9d3cc-93de-45a6-8849-027a4aad3715-scripts\") pod \"glance-default-external-api-0\" (UID: \"05b9d3cc-93de-45a6-8849-027a4aad3715\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:28:01 crc kubenswrapper[5052]: I1203 12:28:01.893629 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage07-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage07-crc\") pod \"glance-default-external-api-0\" (UID: \"05b9d3cc-93de-45a6-8849-027a4aad3715\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:28:01 crc kubenswrapper[5052]: I1203 12:28:01.965675 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:28:02 crc kubenswrapper[5052]: I1203 12:28:01.997818 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/05b9d3cc-93de-45a6-8849-027a4aad3715-combined-ca-bundle\") pod \"glance-default-external-api-0\" (UID: \"05b9d3cc-93de-45a6-8849-027a4aad3715\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:28:02 crc kubenswrapper[5052]: I1203 12:28:01.998010 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/05b9d3cc-93de-45a6-8849-027a4aad3715-config-data\") pod \"glance-default-external-api-0\" (UID: \"05b9d3cc-93de-45a6-8849-027a4aad3715\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:28:02 crc kubenswrapper[5052]: I1203 12:28:01.998049 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-xvh8g\" (UniqueName: \"kubernetes.io/projected/05b9d3cc-93de-45a6-8849-027a4aad3715-kube-api-access-xvh8g\") pod \"glance-default-external-api-0\" (UID: \"05b9d3cc-93de-45a6-8849-027a4aad3715\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:28:02 crc kubenswrapper[5052]: I1203 12:28:01.998104 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/05b9d3cc-93de-45a6-8849-027a4aad3715-public-tls-certs\") pod \"glance-default-external-api-0\" (UID: \"05b9d3cc-93de-45a6-8849-027a4aad3715\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:28:02 crc kubenswrapper[5052]: I1203 12:28:01.998219 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/05b9d3cc-93de-45a6-8849-027a4aad3715-httpd-run\") pod \"glance-default-external-api-0\" (UID: \"05b9d3cc-93de-45a6-8849-027a4aad3715\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:28:02 crc kubenswrapper[5052]: I1203 12:28:01.998252 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/05b9d3cc-93de-45a6-8849-027a4aad3715-logs\") pod \"glance-default-external-api-0\" (UID: \"05b9d3cc-93de-45a6-8849-027a4aad3715\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:28:02 crc kubenswrapper[5052]: I1203 12:28:01.998296 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/05b9d3cc-93de-45a6-8849-027a4aad3715-scripts\") pod \"glance-default-external-api-0\" (UID: \"05b9d3cc-93de-45a6-8849-027a4aad3715\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:28:02 crc kubenswrapper[5052]: I1203 12:28:01.998334 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage07-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage07-crc\") pod \"glance-default-external-api-0\" (UID: \"05b9d3cc-93de-45a6-8849-027a4aad3715\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:28:02 crc kubenswrapper[5052]: I1203 12:28:01.999122 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage07-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage07-crc\") pod \"glance-default-external-api-0\" (UID: \"05b9d3cc-93de-45a6-8849-027a4aad3715\") device mount path \"/mnt/openstack/pv07\"" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:28:02 crc kubenswrapper[5052]: I1203 12:28:01.999560 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/05b9d3cc-93de-45a6-8849-027a4aad3715-logs\") pod \"glance-default-external-api-0\" (UID: \"05b9d3cc-93de-45a6-8849-027a4aad3715\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:28:02 crc kubenswrapper[5052]: I1203 12:28:01.999869 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/05b9d3cc-93de-45a6-8849-027a4aad3715-httpd-run\") pod \"glance-default-external-api-0\" (UID: \"05b9d3cc-93de-45a6-8849-027a4aad3715\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:28:02 crc kubenswrapper[5052]: I1203 12:28:02.011462 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/05b9d3cc-93de-45a6-8849-027a4aad3715-combined-ca-bundle\") pod \"glance-default-external-api-0\" (UID: \"05b9d3cc-93de-45a6-8849-027a4aad3715\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:28:02 crc kubenswrapper[5052]: I1203 12:28:02.012663 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/05b9d3cc-93de-45a6-8849-027a4aad3715-scripts\") pod \"glance-default-external-api-0\" (UID: \"05b9d3cc-93de-45a6-8849-027a4aad3715\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:28:02 crc kubenswrapper[5052]: I1203 12:28:02.031905 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/05b9d3cc-93de-45a6-8849-027a4aad3715-config-data\") pod \"glance-default-external-api-0\" (UID: \"05b9d3cc-93de-45a6-8849-027a4aad3715\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:28:02 crc kubenswrapper[5052]: I1203 12:28:02.032717 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/05b9d3cc-93de-45a6-8849-027a4aad3715-public-tls-certs\") pod \"glance-default-external-api-0\" (UID: \"05b9d3cc-93de-45a6-8849-027a4aad3715\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:28:02 crc kubenswrapper[5052]: I1203 12:28:02.033683 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-xvh8g\" (UniqueName: \"kubernetes.io/projected/05b9d3cc-93de-45a6-8849-027a4aad3715-kube-api-access-xvh8g\") pod \"glance-default-external-api-0\" (UID: \"05b9d3cc-93de-45a6-8849-027a4aad3715\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:28:02 crc kubenswrapper[5052]: I1203 12:28:02.057054 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage07-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage07-crc\") pod \"glance-default-external-api-0\" (UID: \"05b9d3cc-93de-45a6-8849-027a4aad3715\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:28:02 crc kubenswrapper[5052]: I1203 12:28:02.355339 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:28:02 crc kubenswrapper[5052]: I1203 12:28:02.463382 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"e7e769bc-cba8-4955-9d3d-14d1acc7fb5b","Type":"ContainerStarted","Data":"19faf52ea0db2c5e7ec4c8c64930e9e227c325f2bbe38efaefc6c6c5aa244a80"} Dec 03 12:28:02 crc kubenswrapper[5052]: I1203 12:28:02.482999 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" event={"ID":"b8613a94-1e2e-477f-8d52-713c70f5282c","Type":"ContainerStarted","Data":"ecd92b1e5f2c4bb6955463da6c223b6fc821bf3dfbf16ddd65252cba7d35cea5"} Dec 03 12:28:02 crc kubenswrapper[5052]: I1203 12:28:02.484382 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:28:02 crc kubenswrapper[5052]: I1203 12:28:02.487580 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/kube-state-metrics-0" event={"ID":"44520845-0277-47ab-acc6-812aba4656ff","Type":"ContainerStarted","Data":"8048ed0425b57a9a994078140f513eae158b441a12e3e7142612e180f27c9cd3"} Dec 03 12:28:02 crc kubenswrapper[5052]: I1203 12:28:02.488261 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:28:02 crc kubenswrapper[5052]: I1203 12:28:02.552069 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" podStartSLOduration=38.552042765 podStartE2EDuration="38.552042765s" podCreationTimestamp="2025-12-03 12:27:24 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:28:02.542491623 +0000 UTC m=+4073.130733387" watchObservedRunningTime="2025-12-03 12:28:02.552042765 +0000 UTC m=+4073.140284529" Dec 03 12:28:02 crc kubenswrapper[5052]: I1203 12:28:02.578175 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/kube-state-metrics-0" podStartSLOduration=3.099502809 podStartE2EDuration="3.57812372s" podCreationTimestamp="2025-12-03 12:27:59 +0000 UTC" firstStartedPulling="2025-12-03 12:28:00.719982565 +0000 UTC m=+4071.308224329" lastFinishedPulling="2025-12-03 12:28:01.198603476 +0000 UTC m=+4071.786845240" observedRunningTime="2025-12-03 12:28:02.572561641 +0000 UTC m=+4073.160803405" watchObservedRunningTime="2025-12-03 12:28:02.57812372 +0000 UTC m=+4073.166365484" Dec 03 12:28:02 crc kubenswrapper[5052]: I1203 12:28:02.629838 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 12:28:02 crc kubenswrapper[5052]: I1203 12:28:02.953386 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:28:02 crc kubenswrapper[5052]: W1203 12:28:02.957972 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod05b9d3cc_93de_45a6_8849_027a4aad3715.slice/crio-171b29e4c8764e7231cf3eb42f1982e93714f46c393ac0a499a2ee8c10aad697 WatchSource:0}: Error finding container 171b29e4c8764e7231cf3eb42f1982e93714f46c393ac0a499a2ee8c10aad697: Status 404 returned error can't find the container with id 171b29e4c8764e7231cf3eb42f1982e93714f46c393ac0a499a2ee8c10aad697 Dec 03 12:28:03 crc kubenswrapper[5052]: I1203 12:28:03.167915 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-7f9fd59985-48h54" Dec 03 12:28:03 crc kubenswrapper[5052]: I1203 12:28:03.271868 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-7d9f6cd5d5-xcx8v"] Dec 03 12:28:03 crc kubenswrapper[5052]: I1203 12:28:03.272097 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-7d9f6cd5d5-xcx8v" podUID="bda2a6a4-1675-4341-963d-d088b142e4da" containerName="dnsmasq-dns" containerID="cri-o://2727160f7feae9f174c834d6bd19e8c49863bfda2ef0d99a1bff7a42fbad1f97" gracePeriod=10 Dec 03 12:28:03 crc kubenswrapper[5052]: E1203 12:28:03.374033 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="47d8a341afac3d19e0311930b0aa745891784bd644a5b9f1e35346d168347db9" cmd=["/usr/bin/pgrep","-r","DRST","nova-scheduler"] Dec 03 12:28:03 crc kubenswrapper[5052]: E1203 12:28:03.385666 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="47d8a341afac3d19e0311930b0aa745891784bd644a5b9f1e35346d168347db9" cmd=["/usr/bin/pgrep","-r","DRST","nova-scheduler"] Dec 03 12:28:03 crc kubenswrapper[5052]: E1203 12:28:03.395661 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="47d8a341afac3d19e0311930b0aa745891784bd644a5b9f1e35346d168347db9" cmd=["/usr/bin/pgrep","-r","DRST","nova-scheduler"] Dec 03 12:28:03 crc kubenswrapper[5052]: E1203 12:28:03.395736 5052 prober.go:104] "Probe errored" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" probeType="Readiness" pod="openstack-kuttl-tests/nova-scheduler-0" podUID="8a0b169d-6971-4530-936e-8e5f0f57816f" containerName="nova-scheduler-scheduler" Dec 03 12:28:03 crc kubenswrapper[5052]: I1203 12:28:03.535278 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"e7e769bc-cba8-4955-9d3d-14d1acc7fb5b","Type":"ContainerStarted","Data":"ce04bf8dc26d500c7eeaadff490cd8b971ffdcceacc5c68dc3033bfb0a4f5e56"} Dec 03 12:28:03 crc kubenswrapper[5052]: I1203 12:28:03.563462 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"05b9d3cc-93de-45a6-8849-027a4aad3715","Type":"ContainerStarted","Data":"171b29e4c8764e7231cf3eb42f1982e93714f46c393ac0a499a2ee8c10aad697"} Dec 03 12:28:03 crc kubenswrapper[5052]: I1203 12:28:03.570395 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"61cad274-57bd-4993-aaa6-3e6013900d88","Type":"ContainerStarted","Data":"4a8e66b1f4c01429203f6a3b2e8e1ac5e1270c3ab51113fb2692b0d064c61351"} Dec 03 12:28:03 crc kubenswrapper[5052]: I1203 12:28:03.711236 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="15cf6575-b6be-4f67-861a-61ebf02d1495" path="/var/lib/kubelet/pods/15cf6575-b6be-4f67-861a-61ebf02d1495/volumes" Dec 03 12:28:03 crc kubenswrapper[5052]: I1203 12:28:03.713124 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="71e0e74b-5d01-467f-afb2-7e55d9d58297" path="/var/lib/kubelet/pods/71e0e74b-5d01-467f-afb2-7e55d9d58297/volumes" Dec 03 12:28:04 crc kubenswrapper[5052]: I1203 12:28:04.341200 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-0"] Dec 03 12:28:04 crc kubenswrapper[5052]: I1203 12:28:04.341698 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-cell1-conductor-0" podUID="fcee2113-2b2f-47f9-b118-4f7488ad604e" containerName="nova-cell1-conductor-conductor" containerID="cri-o://df1192fd6167f45271208db821d1f0909fc60a2b10aada89f5e4d0015c104317" gracePeriod=30 Dec 03 12:28:04 crc kubenswrapper[5052]: I1203 12:28:04.470936 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:28:04 crc kubenswrapper[5052]: I1203 12:28:04.580087 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0b3a2637-ed0d-4348-ac7e-c92db6503191-combined-ca-bundle\") pod \"0b3a2637-ed0d-4348-ac7e-c92db6503191\" (UID: \"0b3a2637-ed0d-4348-ac7e-c92db6503191\") " Dec 03 12:28:04 crc kubenswrapper[5052]: I1203 12:28:04.580243 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/0b3a2637-ed0d-4348-ac7e-c92db6503191-config-data\") pod \"0b3a2637-ed0d-4348-ac7e-c92db6503191\" (UID: \"0b3a2637-ed0d-4348-ac7e-c92db6503191\") " Dec 03 12:28:04 crc kubenswrapper[5052]: I1203 12:28:04.580298 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"glance\" (UniqueName: \"kubernetes.io/local-volume/local-storage09-crc\") pod \"0b3a2637-ed0d-4348-ac7e-c92db6503191\" (UID: \"0b3a2637-ed0d-4348-ac7e-c92db6503191\") " Dec 03 12:28:04 crc kubenswrapper[5052]: I1203 12:28:04.580337 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/0b3a2637-ed0d-4348-ac7e-c92db6503191-httpd-run\") pod \"0b3a2637-ed0d-4348-ac7e-c92db6503191\" (UID: \"0b3a2637-ed0d-4348-ac7e-c92db6503191\") " Dec 03 12:28:04 crc kubenswrapper[5052]: I1203 12:28:04.580375 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/0b3a2637-ed0d-4348-ac7e-c92db6503191-logs\") pod \"0b3a2637-ed0d-4348-ac7e-c92db6503191\" (UID: \"0b3a2637-ed0d-4348-ac7e-c92db6503191\") " Dec 03 12:28:04 crc kubenswrapper[5052]: I1203 12:28:04.580454 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/0b3a2637-ed0d-4348-ac7e-c92db6503191-scripts\") pod \"0b3a2637-ed0d-4348-ac7e-c92db6503191\" (UID: \"0b3a2637-ed0d-4348-ac7e-c92db6503191\") " Dec 03 12:28:04 crc kubenswrapper[5052]: I1203 12:28:04.580496 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zv9nb\" (UniqueName: \"kubernetes.io/projected/0b3a2637-ed0d-4348-ac7e-c92db6503191-kube-api-access-zv9nb\") pod \"0b3a2637-ed0d-4348-ac7e-c92db6503191\" (UID: \"0b3a2637-ed0d-4348-ac7e-c92db6503191\") " Dec 03 12:28:04 crc kubenswrapper[5052]: I1203 12:28:04.587061 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0b3a2637-ed0d-4348-ac7e-c92db6503191-kube-api-access-zv9nb" (OuterVolumeSpecName: "kube-api-access-zv9nb") pod "0b3a2637-ed0d-4348-ac7e-c92db6503191" (UID: "0b3a2637-ed0d-4348-ac7e-c92db6503191"). InnerVolumeSpecName "kube-api-access-zv9nb". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:28:04 crc kubenswrapper[5052]: I1203 12:28:04.587875 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0b3a2637-ed0d-4348-ac7e-c92db6503191-httpd-run" (OuterVolumeSpecName: "httpd-run") pod "0b3a2637-ed0d-4348-ac7e-c92db6503191" (UID: "0b3a2637-ed0d-4348-ac7e-c92db6503191"). InnerVolumeSpecName "httpd-run". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:28:04 crc kubenswrapper[5052]: I1203 12:28:04.587974 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0b3a2637-ed0d-4348-ac7e-c92db6503191-logs" (OuterVolumeSpecName: "logs") pod "0b3a2637-ed0d-4348-ac7e-c92db6503191" (UID: "0b3a2637-ed0d-4348-ac7e-c92db6503191"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:28:04 crc kubenswrapper[5052]: I1203 12:28:04.594369 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage09-crc" (OuterVolumeSpecName: "glance") pod "0b3a2637-ed0d-4348-ac7e-c92db6503191" (UID: "0b3a2637-ed0d-4348-ac7e-c92db6503191"). InnerVolumeSpecName "local-storage09-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 12:28:04 crc kubenswrapper[5052]: I1203 12:28:04.594704 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0b3a2637-ed0d-4348-ac7e-c92db6503191-scripts" (OuterVolumeSpecName: "scripts") pod "0b3a2637-ed0d-4348-ac7e-c92db6503191" (UID: "0b3a2637-ed0d-4348-ac7e-c92db6503191"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:28:04 crc kubenswrapper[5052]: I1203 12:28:04.668473 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0b3a2637-ed0d-4348-ac7e-c92db6503191-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "0b3a2637-ed0d-4348-ac7e-c92db6503191" (UID: "0b3a2637-ed0d-4348-ac7e-c92db6503191"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:28:04 crc kubenswrapper[5052]: I1203 12:28:04.682622 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0b3a2637-ed0d-4348-ac7e-c92db6503191-config-data" (OuterVolumeSpecName: "config-data") pod "0b3a2637-ed0d-4348-ac7e-c92db6503191" (UID: "0b3a2637-ed0d-4348-ac7e-c92db6503191"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:28:04 crc kubenswrapper[5052]: I1203 12:28:04.683724 5052 reconciler_common.go:293] "Volume detached for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/0b3a2637-ed0d-4348-ac7e-c92db6503191-httpd-run\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:04 crc kubenswrapper[5052]: I1203 12:28:04.683754 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/0b3a2637-ed0d-4348-ac7e-c92db6503191-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:04 crc kubenswrapper[5052]: I1203 12:28:04.683767 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/0b3a2637-ed0d-4348-ac7e-c92db6503191-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:04 crc kubenswrapper[5052]: I1203 12:28:04.683779 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-zv9nb\" (UniqueName: \"kubernetes.io/projected/0b3a2637-ed0d-4348-ac7e-c92db6503191-kube-api-access-zv9nb\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:04 crc kubenswrapper[5052]: I1203 12:28:04.683793 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0b3a2637-ed0d-4348-ac7e-c92db6503191-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:04 crc kubenswrapper[5052]: I1203 12:28:04.683805 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/0b3a2637-ed0d-4348-ac7e-c92db6503191-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:04 crc kubenswrapper[5052]: I1203 12:28:04.683830 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage09-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage09-crc\") on node \"crc\" " Dec 03 12:28:04 crc kubenswrapper[5052]: I1203 12:28:04.693150 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"05b9d3cc-93de-45a6-8849-027a4aad3715","Type":"ContainerStarted","Data":"07f1af7d6a787079787ad912bf9e1ca56fb4cb7d8e231d7b665fb887db97be81"} Dec 03 12:28:04 crc kubenswrapper[5052]: I1203 12:28:04.700557 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"61cad274-57bd-4993-aaa6-3e6013900d88","Type":"ContainerStarted","Data":"7a324498f8dc1fb392c84dac59b70dd32b53652f4626455f3411fdb91b9fb201"} Dec 03 12:28:04 crc kubenswrapper[5052]: I1203 12:28:04.710812 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage09-crc" (UniqueName: "kubernetes.io/local-volume/local-storage09-crc") on node "crc" Dec 03 12:28:04 crc kubenswrapper[5052]: I1203 12:28:04.735812 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"e7e769bc-cba8-4955-9d3d-14d1acc7fb5b","Type":"ContainerStarted","Data":"14273699743c5c98bca470132190fea0be82840c80649fe7f4e02e858e70da72"} Dec 03 12:28:04 crc kubenswrapper[5052]: I1203 12:28:04.744632 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-7d9f6cd5d5-xcx8v" Dec 03 12:28:04 crc kubenswrapper[5052]: I1203 12:28:04.758607 5052 generic.go:334] "Generic (PLEG): container finished" podID="bda2a6a4-1675-4341-963d-d088b142e4da" containerID="2727160f7feae9f174c834d6bd19e8c49863bfda2ef0d99a1bff7a42fbad1f97" exitCode=0 Dec 03 12:28:04 crc kubenswrapper[5052]: I1203 12:28:04.758684 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-7d9f6cd5d5-xcx8v" event={"ID":"bda2a6a4-1675-4341-963d-d088b142e4da","Type":"ContainerDied","Data":"2727160f7feae9f174c834d6bd19e8c49863bfda2ef0d99a1bff7a42fbad1f97"} Dec 03 12:28:04 crc kubenswrapper[5052]: I1203 12:28:04.758736 5052 scope.go:117] "RemoveContainer" containerID="2727160f7feae9f174c834d6bd19e8c49863bfda2ef0d99a1bff7a42fbad1f97" Dec 03 12:28:04 crc kubenswrapper[5052]: I1203 12:28:04.786393 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage09-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage09-crc\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:04 crc kubenswrapper[5052]: I1203 12:28:04.787573 5052 generic.go:334] "Generic (PLEG): container finished" podID="0b3a2637-ed0d-4348-ac7e-c92db6503191" containerID="8c8df8bba4000ac70f93809a90d4fade952afb4ce410d7dcf985178e32099131" exitCode=0 Dec 03 12:28:04 crc kubenswrapper[5052]: I1203 12:28:04.787675 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:28:04 crc kubenswrapper[5052]: I1203 12:28:04.787691 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"0b3a2637-ed0d-4348-ac7e-c92db6503191","Type":"ContainerDied","Data":"8c8df8bba4000ac70f93809a90d4fade952afb4ce410d7dcf985178e32099131"} Dec 03 12:28:04 crc kubenswrapper[5052]: I1203 12:28:04.790113 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"0b3a2637-ed0d-4348-ac7e-c92db6503191","Type":"ContainerDied","Data":"7b40deaa77ac2e192f6c398118bda4081a63324a94ddcda3a887e339639b15c1"} Dec 03 12:28:04 crc kubenswrapper[5052]: I1203 12:28:04.887491 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nb42t\" (UniqueName: \"kubernetes.io/projected/bda2a6a4-1675-4341-963d-d088b142e4da-kube-api-access-nb42t\") pod \"bda2a6a4-1675-4341-963d-d088b142e4da\" (UID: \"bda2a6a4-1675-4341-963d-d088b142e4da\") " Dec 03 12:28:04 crc kubenswrapper[5052]: I1203 12:28:04.887652 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/bda2a6a4-1675-4341-963d-d088b142e4da-config\") pod \"bda2a6a4-1675-4341-963d-d088b142e4da\" (UID: \"bda2a6a4-1675-4341-963d-d088b142e4da\") " Dec 03 12:28:04 crc kubenswrapper[5052]: I1203 12:28:04.887776 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"dns-swift-storage-0\" (UniqueName: \"kubernetes.io/configmap/bda2a6a4-1675-4341-963d-d088b142e4da-dns-swift-storage-0\") pod \"bda2a6a4-1675-4341-963d-d088b142e4da\" (UID: \"bda2a6a4-1675-4341-963d-d088b142e4da\") " Dec 03 12:28:04 crc kubenswrapper[5052]: I1203 12:28:04.887806 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/bda2a6a4-1675-4341-963d-d088b142e4da-dnsmasq-svc\") pod \"bda2a6a4-1675-4341-963d-d088b142e4da\" (UID: \"bda2a6a4-1675-4341-963d-d088b142e4da\") " Dec 03 12:28:04 crc kubenswrapper[5052]: I1203 12:28:04.899245 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bda2a6a4-1675-4341-963d-d088b142e4da-kube-api-access-nb42t" (OuterVolumeSpecName: "kube-api-access-nb42t") pod "bda2a6a4-1675-4341-963d-d088b142e4da" (UID: "bda2a6a4-1675-4341-963d-d088b142e4da"). InnerVolumeSpecName "kube-api-access-nb42t". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:28:04 crc kubenswrapper[5052]: I1203 12:28:04.907770 5052 scope.go:117] "RemoveContainer" containerID="1b7cad63063d7371ba6f8f987a89264c0b0086dac46b64684e09fcd1ef4338eb" Dec 03 12:28:04 crc kubenswrapper[5052]: I1203 12:28:04.922881 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:28:04 crc kubenswrapper[5052]: I1203 12:28:04.958346 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:28:04 crc kubenswrapper[5052]: I1203 12:28:04.979821 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:28:04 crc kubenswrapper[5052]: E1203 12:28:04.980319 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="0b3a2637-ed0d-4348-ac7e-c92db6503191" containerName="glance-log" Dec 03 12:28:04 crc kubenswrapper[5052]: I1203 12:28:04.980342 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="0b3a2637-ed0d-4348-ac7e-c92db6503191" containerName="glance-log" Dec 03 12:28:04 crc kubenswrapper[5052]: E1203 12:28:04.980368 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="bda2a6a4-1675-4341-963d-d088b142e4da" containerName="init" Dec 03 12:28:04 crc kubenswrapper[5052]: I1203 12:28:04.980376 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="bda2a6a4-1675-4341-963d-d088b142e4da" containerName="init" Dec 03 12:28:04 crc kubenswrapper[5052]: E1203 12:28:04.980388 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="bda2a6a4-1675-4341-963d-d088b142e4da" containerName="dnsmasq-dns" Dec 03 12:28:04 crc kubenswrapper[5052]: I1203 12:28:04.980397 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="bda2a6a4-1675-4341-963d-d088b142e4da" containerName="dnsmasq-dns" Dec 03 12:28:04 crc kubenswrapper[5052]: E1203 12:28:04.980413 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="0b3a2637-ed0d-4348-ac7e-c92db6503191" containerName="glance-httpd" Dec 03 12:28:04 crc kubenswrapper[5052]: I1203 12:28:04.980420 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="0b3a2637-ed0d-4348-ac7e-c92db6503191" containerName="glance-httpd" Dec 03 12:28:04 crc kubenswrapper[5052]: I1203 12:28:04.980636 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="0b3a2637-ed0d-4348-ac7e-c92db6503191" containerName="glance-log" Dec 03 12:28:04 crc kubenswrapper[5052]: I1203 12:28:04.980655 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="bda2a6a4-1675-4341-963d-d088b142e4da" containerName="dnsmasq-dns" Dec 03 12:28:04 crc kubenswrapper[5052]: I1203 12:28:04.980665 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="0b3a2637-ed0d-4348-ac7e-c92db6503191" containerName="glance-httpd" Dec 03 12:28:04 crc kubenswrapper[5052]: I1203 12:28:04.981765 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:28:04 crc kubenswrapper[5052]: I1203 12:28:04.981874 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:28:04 crc kubenswrapper[5052]: I1203 12:28:04.990793 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-nb42t\" (UniqueName: \"kubernetes.io/projected/bda2a6a4-1675-4341-963d-d088b142e4da-kube-api-access-nb42t\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:04 crc kubenswrapper[5052]: I1203 12:28:04.992427 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-default-internal-config-data" Dec 03 12:28:04 crc kubenswrapper[5052]: I1203 12:28:04.992733 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-glance-default-internal-svc" Dec 03 12:28:05 crc kubenswrapper[5052]: I1203 12:28:05.093054 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/82bcca5b-e2c3-43ce-aba3-eb879796dc3e-scripts\") pod \"glance-default-internal-api-0\" (UID: \"82bcca5b-e2c3-43ce-aba3-eb879796dc3e\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:28:05 crc kubenswrapper[5052]: I1203 12:28:05.093159 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/82bcca5b-e2c3-43ce-aba3-eb879796dc3e-httpd-run\") pod \"glance-default-internal-api-0\" (UID: \"82bcca5b-e2c3-43ce-aba3-eb879796dc3e\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:28:05 crc kubenswrapper[5052]: I1203 12:28:05.093240 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/82bcca5b-e2c3-43ce-aba3-eb879796dc3e-config-data\") pod \"glance-default-internal-api-0\" (UID: \"82bcca5b-e2c3-43ce-aba3-eb879796dc3e\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:28:05 crc kubenswrapper[5052]: I1203 12:28:05.093263 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/82bcca5b-e2c3-43ce-aba3-eb879796dc3e-logs\") pod \"glance-default-internal-api-0\" (UID: \"82bcca5b-e2c3-43ce-aba3-eb879796dc3e\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:28:05 crc kubenswrapper[5052]: I1203 12:28:05.093295 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage09-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage09-crc\") pod \"glance-default-internal-api-0\" (UID: \"82bcca5b-e2c3-43ce-aba3-eb879796dc3e\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:28:05 crc kubenswrapper[5052]: I1203 12:28:05.093329 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5xg6n\" (UniqueName: \"kubernetes.io/projected/82bcca5b-e2c3-43ce-aba3-eb879796dc3e-kube-api-access-5xg6n\") pod \"glance-default-internal-api-0\" (UID: \"82bcca5b-e2c3-43ce-aba3-eb879796dc3e\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:28:05 crc kubenswrapper[5052]: I1203 12:28:05.093360 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/82bcca5b-e2c3-43ce-aba3-eb879796dc3e-combined-ca-bundle\") pod \"glance-default-internal-api-0\" (UID: \"82bcca5b-e2c3-43ce-aba3-eb879796dc3e\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:28:05 crc kubenswrapper[5052]: I1203 12:28:05.093397 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/82bcca5b-e2c3-43ce-aba3-eb879796dc3e-internal-tls-certs\") pod \"glance-default-internal-api-0\" (UID: \"82bcca5b-e2c3-43ce-aba3-eb879796dc3e\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:28:05 crc kubenswrapper[5052]: I1203 12:28:05.117479 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bda2a6a4-1675-4341-963d-d088b142e4da-config" (OuterVolumeSpecName: "config") pod "bda2a6a4-1675-4341-963d-d088b142e4da" (UID: "bda2a6a4-1675-4341-963d-d088b142e4da"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:28:05 crc kubenswrapper[5052]: I1203 12:28:05.117709 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bda2a6a4-1675-4341-963d-d088b142e4da-dns-swift-storage-0" (OuterVolumeSpecName: "dns-swift-storage-0") pod "bda2a6a4-1675-4341-963d-d088b142e4da" (UID: "bda2a6a4-1675-4341-963d-d088b142e4da"). InnerVolumeSpecName "dns-swift-storage-0". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:28:05 crc kubenswrapper[5052]: I1203 12:28:05.127260 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bda2a6a4-1675-4341-963d-d088b142e4da-dnsmasq-svc" (OuterVolumeSpecName: "dnsmasq-svc") pod "bda2a6a4-1675-4341-963d-d088b142e4da" (UID: "bda2a6a4-1675-4341-963d-d088b142e4da"). InnerVolumeSpecName "dnsmasq-svc". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:28:05 crc kubenswrapper[5052]: I1203 12:28:05.195669 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-5xg6n\" (UniqueName: \"kubernetes.io/projected/82bcca5b-e2c3-43ce-aba3-eb879796dc3e-kube-api-access-5xg6n\") pod \"glance-default-internal-api-0\" (UID: \"82bcca5b-e2c3-43ce-aba3-eb879796dc3e\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:28:05 crc kubenswrapper[5052]: I1203 12:28:05.195742 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/82bcca5b-e2c3-43ce-aba3-eb879796dc3e-combined-ca-bundle\") pod \"glance-default-internal-api-0\" (UID: \"82bcca5b-e2c3-43ce-aba3-eb879796dc3e\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:28:05 crc kubenswrapper[5052]: I1203 12:28:05.195765 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/82bcca5b-e2c3-43ce-aba3-eb879796dc3e-internal-tls-certs\") pod \"glance-default-internal-api-0\" (UID: \"82bcca5b-e2c3-43ce-aba3-eb879796dc3e\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:28:05 crc kubenswrapper[5052]: I1203 12:28:05.195828 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/82bcca5b-e2c3-43ce-aba3-eb879796dc3e-scripts\") pod \"glance-default-internal-api-0\" (UID: \"82bcca5b-e2c3-43ce-aba3-eb879796dc3e\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:28:05 crc kubenswrapper[5052]: I1203 12:28:05.195903 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/82bcca5b-e2c3-43ce-aba3-eb879796dc3e-httpd-run\") pod \"glance-default-internal-api-0\" (UID: \"82bcca5b-e2c3-43ce-aba3-eb879796dc3e\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:28:05 crc kubenswrapper[5052]: I1203 12:28:05.195965 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/82bcca5b-e2c3-43ce-aba3-eb879796dc3e-config-data\") pod \"glance-default-internal-api-0\" (UID: \"82bcca5b-e2c3-43ce-aba3-eb879796dc3e\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:28:05 crc kubenswrapper[5052]: I1203 12:28:05.196003 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/82bcca5b-e2c3-43ce-aba3-eb879796dc3e-logs\") pod \"glance-default-internal-api-0\" (UID: \"82bcca5b-e2c3-43ce-aba3-eb879796dc3e\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:28:05 crc kubenswrapper[5052]: I1203 12:28:05.196031 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage09-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage09-crc\") pod \"glance-default-internal-api-0\" (UID: \"82bcca5b-e2c3-43ce-aba3-eb879796dc3e\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:28:05 crc kubenswrapper[5052]: I1203 12:28:05.196095 5052 reconciler_common.go:293] "Volume detached for volume \"dns-swift-storage-0\" (UniqueName: \"kubernetes.io/configmap/bda2a6a4-1675-4341-963d-d088b142e4da-dns-swift-storage-0\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:05 crc kubenswrapper[5052]: I1203 12:28:05.196107 5052 reconciler_common.go:293] "Volume detached for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/bda2a6a4-1675-4341-963d-d088b142e4da-dnsmasq-svc\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:05 crc kubenswrapper[5052]: I1203 12:28:05.196117 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/bda2a6a4-1675-4341-963d-d088b142e4da-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:05 crc kubenswrapper[5052]: I1203 12:28:05.196330 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage09-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage09-crc\") pod \"glance-default-internal-api-0\" (UID: \"82bcca5b-e2c3-43ce-aba3-eb879796dc3e\") device mount path \"/mnt/openstack/pv09\"" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:28:05 crc kubenswrapper[5052]: I1203 12:28:05.201281 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/82bcca5b-e2c3-43ce-aba3-eb879796dc3e-logs\") pod \"glance-default-internal-api-0\" (UID: \"82bcca5b-e2c3-43ce-aba3-eb879796dc3e\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:28:05 crc kubenswrapper[5052]: I1203 12:28:05.201666 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/82bcca5b-e2c3-43ce-aba3-eb879796dc3e-httpd-run\") pod \"glance-default-internal-api-0\" (UID: \"82bcca5b-e2c3-43ce-aba3-eb879796dc3e\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:28:05 crc kubenswrapper[5052]: I1203 12:28:05.205551 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/82bcca5b-e2c3-43ce-aba3-eb879796dc3e-scripts\") pod \"glance-default-internal-api-0\" (UID: \"82bcca5b-e2c3-43ce-aba3-eb879796dc3e\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:28:05 crc kubenswrapper[5052]: I1203 12:28:05.219496 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-5xg6n\" (UniqueName: \"kubernetes.io/projected/82bcca5b-e2c3-43ce-aba3-eb879796dc3e-kube-api-access-5xg6n\") pod \"glance-default-internal-api-0\" (UID: \"82bcca5b-e2c3-43ce-aba3-eb879796dc3e\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:28:05 crc kubenswrapper[5052]: I1203 12:28:05.221257 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/82bcca5b-e2c3-43ce-aba3-eb879796dc3e-combined-ca-bundle\") pod \"glance-default-internal-api-0\" (UID: \"82bcca5b-e2c3-43ce-aba3-eb879796dc3e\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:28:05 crc kubenswrapper[5052]: I1203 12:28:05.221838 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/82bcca5b-e2c3-43ce-aba3-eb879796dc3e-config-data\") pod \"glance-default-internal-api-0\" (UID: \"82bcca5b-e2c3-43ce-aba3-eb879796dc3e\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:28:05 crc kubenswrapper[5052]: I1203 12:28:05.222092 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/82bcca5b-e2c3-43ce-aba3-eb879796dc3e-internal-tls-certs\") pod \"glance-default-internal-api-0\" (UID: \"82bcca5b-e2c3-43ce-aba3-eb879796dc3e\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:28:05 crc kubenswrapper[5052]: I1203 12:28:05.239910 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage09-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage09-crc\") pod \"glance-default-internal-api-0\" (UID: \"82bcca5b-e2c3-43ce-aba3-eb879796dc3e\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:28:05 crc kubenswrapper[5052]: E1203 12:28:05.243685 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="790991ac480a9e358efc31b18c1e765ec3b8f7c9c3386cf9c51c6dc7f800750c" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:28:05 crc kubenswrapper[5052]: E1203 12:28:05.252262 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="790991ac480a9e358efc31b18c1e765ec3b8f7c9c3386cf9c51c6dc7f800750c" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:28:05 crc kubenswrapper[5052]: E1203 12:28:05.260578 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="790991ac480a9e358efc31b18c1e765ec3b8f7c9c3386cf9c51c6dc7f800750c" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:28:05 crc kubenswrapper[5052]: E1203 12:28:05.260664 5052 prober.go:104] "Probe errored" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" probeType="Readiness" pod="openstack-kuttl-tests/nova-cell0-conductor-0" podUID="c080dc7f-df72-487f-b2c1-83b143d34784" containerName="nova-cell0-conductor-conductor" Dec 03 12:28:05 crc kubenswrapper[5052]: I1203 12:28:05.346268 5052 scope.go:117] "RemoveContainer" containerID="8c8df8bba4000ac70f93809a90d4fade952afb4ce410d7dcf985178e32099131" Dec 03 12:28:05 crc kubenswrapper[5052]: I1203 12:28:05.369880 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:28:05 crc kubenswrapper[5052]: I1203 12:28:05.396730 5052 scope.go:117] "RemoveContainer" containerID="35e2dee61a61ea5e56329f496973eb6a0f2fe0619c6f0df2e55caadaea334f93" Dec 03 12:28:05 crc kubenswrapper[5052]: I1203 12:28:05.443154 5052 scope.go:117] "RemoveContainer" containerID="8c8df8bba4000ac70f93809a90d4fade952afb4ce410d7dcf985178e32099131" Dec 03 12:28:05 crc kubenswrapper[5052]: E1203 12:28:05.443649 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8c8df8bba4000ac70f93809a90d4fade952afb4ce410d7dcf985178e32099131\": container with ID starting with 8c8df8bba4000ac70f93809a90d4fade952afb4ce410d7dcf985178e32099131 not found: ID does not exist" containerID="8c8df8bba4000ac70f93809a90d4fade952afb4ce410d7dcf985178e32099131" Dec 03 12:28:05 crc kubenswrapper[5052]: I1203 12:28:05.443725 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8c8df8bba4000ac70f93809a90d4fade952afb4ce410d7dcf985178e32099131"} err="failed to get container status \"8c8df8bba4000ac70f93809a90d4fade952afb4ce410d7dcf985178e32099131\": rpc error: code = NotFound desc = could not find container \"8c8df8bba4000ac70f93809a90d4fade952afb4ce410d7dcf985178e32099131\": container with ID starting with 8c8df8bba4000ac70f93809a90d4fade952afb4ce410d7dcf985178e32099131 not found: ID does not exist" Dec 03 12:28:05 crc kubenswrapper[5052]: I1203 12:28:05.443759 5052 scope.go:117] "RemoveContainer" containerID="35e2dee61a61ea5e56329f496973eb6a0f2fe0619c6f0df2e55caadaea334f93" Dec 03 12:28:05 crc kubenswrapper[5052]: E1203 12:28:05.444163 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"35e2dee61a61ea5e56329f496973eb6a0f2fe0619c6f0df2e55caadaea334f93\": container with ID starting with 35e2dee61a61ea5e56329f496973eb6a0f2fe0619c6f0df2e55caadaea334f93 not found: ID does not exist" containerID="35e2dee61a61ea5e56329f496973eb6a0f2fe0619c6f0df2e55caadaea334f93" Dec 03 12:28:05 crc kubenswrapper[5052]: I1203 12:28:05.444216 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"35e2dee61a61ea5e56329f496973eb6a0f2fe0619c6f0df2e55caadaea334f93"} err="failed to get container status \"35e2dee61a61ea5e56329f496973eb6a0f2fe0619c6f0df2e55caadaea334f93\": rpc error: code = NotFound desc = could not find container \"35e2dee61a61ea5e56329f496973eb6a0f2fe0619c6f0df2e55caadaea334f93\": container with ID starting with 35e2dee61a61ea5e56329f496973eb6a0f2fe0619c6f0df2e55caadaea334f93 not found: ID does not exist" Dec 03 12:28:05 crc kubenswrapper[5052]: I1203 12:28:05.709386 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0b3a2637-ed0d-4348-ac7e-c92db6503191" path="/var/lib/kubelet/pods/0b3a2637-ed0d-4348-ac7e-c92db6503191/volumes" Dec 03 12:28:05 crc kubenswrapper[5052]: I1203 12:28:05.815537 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"e7e769bc-cba8-4955-9d3d-14d1acc7fb5b","Type":"ContainerStarted","Data":"94e6dc8d77f6943b437cb5870adcaa6c29d491e693bc37360bb9fd740b7610f1"} Dec 03 12:28:05 crc kubenswrapper[5052]: I1203 12:28:05.817532 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-7d9f6cd5d5-xcx8v" event={"ID":"bda2a6a4-1675-4341-963d-d088b142e4da","Type":"ContainerDied","Data":"e96c22e5c7c486e968c8035a9d043ba4a64d6ced72e8734240245ab7f0643506"} Dec 03 12:28:05 crc kubenswrapper[5052]: I1203 12:28:05.817742 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-7d9f6cd5d5-xcx8v" Dec 03 12:28:05 crc kubenswrapper[5052]: I1203 12:28:05.832624 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"05b9d3cc-93de-45a6-8849-027a4aad3715","Type":"ContainerStarted","Data":"2f2f681ab4e070f1a45adfa2327a6b72d7c87fcccf7bb015c88103e25b6472d9"} Dec 03 12:28:05 crc kubenswrapper[5052]: I1203 12:28:05.849807 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"61cad274-57bd-4993-aaa6-3e6013900d88","Type":"ContainerStarted","Data":"d1ad86af122d9acbc9e7b5bb95477effe197a4939198ba933eb8fb9e83c0f569"} Dec 03 12:28:05 crc kubenswrapper[5052]: I1203 12:28:05.850426 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:28:05 crc kubenswrapper[5052]: I1203 12:28:05.880582 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-7d9f6cd5d5-xcx8v"] Dec 03 12:28:05 crc kubenswrapper[5052]: I1203 12:28:05.891676 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-7d9f6cd5d5-xcx8v"] Dec 03 12:28:05 crc kubenswrapper[5052]: I1203 12:28:05.899295 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/glance-default-external-api-0" podStartSLOduration=4.899272203 podStartE2EDuration="4.899272203s" podCreationTimestamp="2025-12-03 12:28:01 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:28:05.897011808 +0000 UTC m=+4076.485253582" watchObservedRunningTime="2025-12-03 12:28:05.899272203 +0000 UTC m=+4076.487513977" Dec 03 12:28:06 crc kubenswrapper[5052]: I1203 12:28:06.054120 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/cinder-api-0" podStartSLOduration=5.054100921 podStartE2EDuration="5.054100921s" podCreationTimestamp="2025-12-03 12:28:01 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:28:05.955958421 +0000 UTC m=+4076.544200195" watchObservedRunningTime="2025-12-03 12:28:06.054100921 +0000 UTC m=+4076.642342685" Dec 03 12:28:06 crc kubenswrapper[5052]: I1203 12:28:06.102883 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:28:06 crc kubenswrapper[5052]: I1203 12:28:06.862874 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"82bcca5b-e2c3-43ce-aba3-eb879796dc3e","Type":"ContainerStarted","Data":"75b9c75f04326f0c84c32e2979b31896339503f5d49a49fbaac6f84c100d77b0"} Dec 03 12:28:06 crc kubenswrapper[5052]: I1203 12:28:06.862951 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"82bcca5b-e2c3-43ce-aba3-eb879796dc3e","Type":"ContainerStarted","Data":"9048c3f157c9a9bacdb7bf1e88d5723f208a649d589a35d9f378cacdb9dbea22"} Dec 03 12:28:06 crc kubenswrapper[5052]: I1203 12:28:06.872818 5052 generic.go:334] "Generic (PLEG): container finished" podID="ee394270-24f0-4687-9129-a5937efe9826" containerID="0680111a7796404e507144aefcc56ae790dfd5e497642c8b08e59f43944a5d46" exitCode=0 Dec 03 12:28:06 crc kubenswrapper[5052]: I1203 12:28:06.872898 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-bootstrap-8nbqq" event={"ID":"ee394270-24f0-4687-9129-a5937efe9826","Type":"ContainerDied","Data":"0680111a7796404e507144aefcc56ae790dfd5e497642c8b08e59f43944a5d46"} Dec 03 12:28:07 crc kubenswrapper[5052]: I1203 12:28:07.685964 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bda2a6a4-1675-4341-963d-d088b142e4da" path="/var/lib/kubelet/pods/bda2a6a4-1675-4341-963d-d088b142e4da/volumes" Dec 03 12:28:07 crc kubenswrapper[5052]: I1203 12:28:07.887727 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"82bcca5b-e2c3-43ce-aba3-eb879796dc3e","Type":"ContainerStarted","Data":"fa45d03e94f8aa43afec4ca3932bb3d4299d6fe6674927e304720e875b1c73d2"} Dec 03 12:28:07 crc kubenswrapper[5052]: I1203 12:28:07.943347 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/glance-default-internal-api-0" podStartSLOduration=3.943315954 podStartE2EDuration="3.943315954s" podCreationTimestamp="2025-12-03 12:28:04 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:28:07.910799606 +0000 UTC m=+4078.499041370" watchObservedRunningTime="2025-12-03 12:28:07.943315954 +0000 UTC m=+4078.531557718" Dec 03 12:28:08 crc kubenswrapper[5052]: I1203 12:28:08.300246 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-bootstrap-8nbqq" Dec 03 12:28:08 crc kubenswrapper[5052]: E1203 12:28:08.345672 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="47d8a341afac3d19e0311930b0aa745891784bd644a5b9f1e35346d168347db9" cmd=["/usr/bin/pgrep","-r","DRST","nova-scheduler"] Dec 03 12:28:08 crc kubenswrapper[5052]: E1203 12:28:08.362298 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="47d8a341afac3d19e0311930b0aa745891784bd644a5b9f1e35346d168347db9" cmd=["/usr/bin/pgrep","-r","DRST","nova-scheduler"] Dec 03 12:28:08 crc kubenswrapper[5052]: I1203 12:28:08.364681 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/ee394270-24f0-4687-9129-a5937efe9826-fernet-keys\") pod \"ee394270-24f0-4687-9129-a5937efe9826\" (UID: \"ee394270-24f0-4687-9129-a5937efe9826\") " Dec 03 12:28:08 crc kubenswrapper[5052]: I1203 12:28:08.364735 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/ee394270-24f0-4687-9129-a5937efe9826-credential-keys\") pod \"ee394270-24f0-4687-9129-a5937efe9826\" (UID: \"ee394270-24f0-4687-9129-a5937efe9826\") " Dec 03 12:28:08 crc kubenswrapper[5052]: I1203 12:28:08.364763 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/ee394270-24f0-4687-9129-a5937efe9826-scripts\") pod \"ee394270-24f0-4687-9129-a5937efe9826\" (UID: \"ee394270-24f0-4687-9129-a5937efe9826\") " Dec 03 12:28:08 crc kubenswrapper[5052]: I1203 12:28:08.364812 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mfzvn\" (UniqueName: \"kubernetes.io/projected/ee394270-24f0-4687-9129-a5937efe9826-kube-api-access-mfzvn\") pod \"ee394270-24f0-4687-9129-a5937efe9826\" (UID: \"ee394270-24f0-4687-9129-a5937efe9826\") " Dec 03 12:28:08 crc kubenswrapper[5052]: I1203 12:28:08.364833 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/ee394270-24f0-4687-9129-a5937efe9826-config-data\") pod \"ee394270-24f0-4687-9129-a5937efe9826\" (UID: \"ee394270-24f0-4687-9129-a5937efe9826\") " Dec 03 12:28:08 crc kubenswrapper[5052]: I1203 12:28:08.364942 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ee394270-24f0-4687-9129-a5937efe9826-combined-ca-bundle\") pod \"ee394270-24f0-4687-9129-a5937efe9826\" (UID: \"ee394270-24f0-4687-9129-a5937efe9826\") " Dec 03 12:28:08 crc kubenswrapper[5052]: I1203 12:28:08.369940 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ee394270-24f0-4687-9129-a5937efe9826-fernet-keys" (OuterVolumeSpecName: "fernet-keys") pod "ee394270-24f0-4687-9129-a5937efe9826" (UID: "ee394270-24f0-4687-9129-a5937efe9826"). InnerVolumeSpecName "fernet-keys". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:28:08 crc kubenswrapper[5052]: I1203 12:28:08.371713 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ee394270-24f0-4687-9129-a5937efe9826-scripts" (OuterVolumeSpecName: "scripts") pod "ee394270-24f0-4687-9129-a5937efe9826" (UID: "ee394270-24f0-4687-9129-a5937efe9826"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:28:08 crc kubenswrapper[5052]: E1203 12:28:08.375541 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="47d8a341afac3d19e0311930b0aa745891784bd644a5b9f1e35346d168347db9" cmd=["/usr/bin/pgrep","-r","DRST","nova-scheduler"] Dec 03 12:28:08 crc kubenswrapper[5052]: E1203 12:28:08.375603 5052 prober.go:104] "Probe errored" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" probeType="Readiness" pod="openstack-kuttl-tests/nova-scheduler-0" podUID="8a0b169d-6971-4530-936e-8e5f0f57816f" containerName="nova-scheduler-scheduler" Dec 03 12:28:08 crc kubenswrapper[5052]: I1203 12:28:08.377854 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ee394270-24f0-4687-9129-a5937efe9826-kube-api-access-mfzvn" (OuterVolumeSpecName: "kube-api-access-mfzvn") pod "ee394270-24f0-4687-9129-a5937efe9826" (UID: "ee394270-24f0-4687-9129-a5937efe9826"). InnerVolumeSpecName "kube-api-access-mfzvn". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:28:08 crc kubenswrapper[5052]: I1203 12:28:08.425023 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ee394270-24f0-4687-9129-a5937efe9826-credential-keys" (OuterVolumeSpecName: "credential-keys") pod "ee394270-24f0-4687-9129-a5937efe9826" (UID: "ee394270-24f0-4687-9129-a5937efe9826"). InnerVolumeSpecName "credential-keys". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:28:08 crc kubenswrapper[5052]: I1203 12:28:08.427804 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ee394270-24f0-4687-9129-a5937efe9826-config-data" (OuterVolumeSpecName: "config-data") pod "ee394270-24f0-4687-9129-a5937efe9826" (UID: "ee394270-24f0-4687-9129-a5937efe9826"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:28:08 crc kubenswrapper[5052]: I1203 12:28:08.448148 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ee394270-24f0-4687-9129-a5937efe9826-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "ee394270-24f0-4687-9129-a5937efe9826" (UID: "ee394270-24f0-4687-9129-a5937efe9826"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:28:08 crc kubenswrapper[5052]: I1203 12:28:08.475039 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ee394270-24f0-4687-9129-a5937efe9826-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:08 crc kubenswrapper[5052]: I1203 12:28:08.475087 5052 reconciler_common.go:293] "Volume detached for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/ee394270-24f0-4687-9129-a5937efe9826-fernet-keys\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:08 crc kubenswrapper[5052]: I1203 12:28:08.475100 5052 reconciler_common.go:293] "Volume detached for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/ee394270-24f0-4687-9129-a5937efe9826-credential-keys\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:08 crc kubenswrapper[5052]: I1203 12:28:08.475111 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/ee394270-24f0-4687-9129-a5937efe9826-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:08 crc kubenswrapper[5052]: I1203 12:28:08.475122 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-mfzvn\" (UniqueName: \"kubernetes.io/projected/ee394270-24f0-4687-9129-a5937efe9826-kube-api-access-mfzvn\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:08 crc kubenswrapper[5052]: I1203 12:28:08.475135 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/ee394270-24f0-4687-9129-a5937efe9826-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:08 crc kubenswrapper[5052]: I1203 12:28:08.727723 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openshift-marketplace/certified-operators-75j45"] Dec 03 12:28:08 crc kubenswrapper[5052]: E1203 12:28:08.728539 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="ee394270-24f0-4687-9129-a5937efe9826" containerName="keystone-bootstrap" Dec 03 12:28:08 crc kubenswrapper[5052]: I1203 12:28:08.728558 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee394270-24f0-4687-9129-a5937efe9826" containerName="keystone-bootstrap" Dec 03 12:28:08 crc kubenswrapper[5052]: I1203 12:28:08.728784 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="ee394270-24f0-4687-9129-a5937efe9826" containerName="keystone-bootstrap" Dec 03 12:28:08 crc kubenswrapper[5052]: I1203 12:28:08.730408 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/certified-operators-75j45" Dec 03 12:28:08 crc kubenswrapper[5052]: I1203 12:28:08.763968 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/certified-operators-75j45"] Dec 03 12:28:08 crc kubenswrapper[5052]: I1203 12:28:08.779718 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bgqmr\" (UniqueName: \"kubernetes.io/projected/c03ce333-5bdc-4cdb-9a81-53db1322b7bf-kube-api-access-bgqmr\") pod \"certified-operators-75j45\" (UID: \"c03ce333-5bdc-4cdb-9a81-53db1322b7bf\") " pod="openshift-marketplace/certified-operators-75j45" Dec 03 12:28:08 crc kubenswrapper[5052]: I1203 12:28:08.779757 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/c03ce333-5bdc-4cdb-9a81-53db1322b7bf-utilities\") pod \"certified-operators-75j45\" (UID: \"c03ce333-5bdc-4cdb-9a81-53db1322b7bf\") " pod="openshift-marketplace/certified-operators-75j45" Dec 03 12:28:08 crc kubenswrapper[5052]: I1203 12:28:08.780051 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/c03ce333-5bdc-4cdb-9a81-53db1322b7bf-catalog-content\") pod \"certified-operators-75j45\" (UID: \"c03ce333-5bdc-4cdb-9a81-53db1322b7bf\") " pod="openshift-marketplace/certified-operators-75j45" Dec 03 12:28:08 crc kubenswrapper[5052]: I1203 12:28:08.889314 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-bgqmr\" (UniqueName: \"kubernetes.io/projected/c03ce333-5bdc-4cdb-9a81-53db1322b7bf-kube-api-access-bgqmr\") pod \"certified-operators-75j45\" (UID: \"c03ce333-5bdc-4cdb-9a81-53db1322b7bf\") " pod="openshift-marketplace/certified-operators-75j45" Dec 03 12:28:08 crc kubenswrapper[5052]: I1203 12:28:08.893240 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/c03ce333-5bdc-4cdb-9a81-53db1322b7bf-utilities\") pod \"certified-operators-75j45\" (UID: \"c03ce333-5bdc-4cdb-9a81-53db1322b7bf\") " pod="openshift-marketplace/certified-operators-75j45" Dec 03 12:28:08 crc kubenswrapper[5052]: I1203 12:28:08.893731 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/c03ce333-5bdc-4cdb-9a81-53db1322b7bf-catalog-content\") pod \"certified-operators-75j45\" (UID: \"c03ce333-5bdc-4cdb-9a81-53db1322b7bf\") " pod="openshift-marketplace/certified-operators-75j45" Dec 03 12:28:08 crc kubenswrapper[5052]: I1203 12:28:08.894592 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/c03ce333-5bdc-4cdb-9a81-53db1322b7bf-catalog-content\") pod \"certified-operators-75j45\" (UID: \"c03ce333-5bdc-4cdb-9a81-53db1322b7bf\") " pod="openshift-marketplace/certified-operators-75j45" Dec 03 12:28:08 crc kubenswrapper[5052]: I1203 12:28:08.894629 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/c03ce333-5bdc-4cdb-9a81-53db1322b7bf-utilities\") pod \"certified-operators-75j45\" (UID: \"c03ce333-5bdc-4cdb-9a81-53db1322b7bf\") " pod="openshift-marketplace/certified-operators-75j45" Dec 03 12:28:08 crc kubenswrapper[5052]: I1203 12:28:08.922522 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"e7e769bc-cba8-4955-9d3d-14d1acc7fb5b","Type":"ContainerStarted","Data":"ed335dc5276810aababf7754b74b1511a5d7d1fa065ad7710e9fe074646e5384"} Dec 03 12:28:08 crc kubenswrapper[5052]: I1203 12:28:08.923641 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-bgqmr\" (UniqueName: \"kubernetes.io/projected/c03ce333-5bdc-4cdb-9a81-53db1322b7bf-kube-api-access-bgqmr\") pod \"certified-operators-75j45\" (UID: \"c03ce333-5bdc-4cdb-9a81-53db1322b7bf\") " pod="openshift-marketplace/certified-operators-75j45" Dec 03 12:28:08 crc kubenswrapper[5052]: I1203 12:28:08.923896 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:28:08 crc kubenswrapper[5052]: I1203 12:28:08.945262 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-bootstrap-8nbqq" Dec 03 12:28:08 crc kubenswrapper[5052]: I1203 12:28:08.946655 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-bootstrap-8nbqq" event={"ID":"ee394270-24f0-4687-9129-a5937efe9826","Type":"ContainerDied","Data":"cb0d233f8370b05426b89cc1b9907e38d01ff16d1f15c8569343bc9b8aa39f5e"} Dec 03 12:28:08 crc kubenswrapper[5052]: I1203 12:28:08.946714 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cb0d233f8370b05426b89cc1b9907e38d01ff16d1f15c8569343bc9b8aa39f5e" Dec 03 12:28:09 crc kubenswrapper[5052]: I1203 12:28:09.008864 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/ceilometer-0" podStartSLOduration=2.383808814 podStartE2EDuration="9.008831446s" podCreationTimestamp="2025-12-03 12:28:00 +0000 UTC" firstStartedPulling="2025-12-03 12:28:01.686409619 +0000 UTC m=+4072.274651383" lastFinishedPulling="2025-12-03 12:28:08.311432251 +0000 UTC m=+4078.899674015" observedRunningTime="2025-12-03 12:28:08.974877417 +0000 UTC m=+4079.563119181" watchObservedRunningTime="2025-12-03 12:28:09.008831446 +0000 UTC m=+4079.597073210" Dec 03 12:28:09 crc kubenswrapper[5052]: I1203 12:28:09.051120 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/certified-operators-75j45" Dec 03 12:28:09 crc kubenswrapper[5052]: I1203 12:28:09.095690 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/keystone-68cf4f75d7-6pwrw"] Dec 03 12:28:09 crc kubenswrapper[5052]: I1203 12:28:09.100416 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-68cf4f75d7-6pwrw" Dec 03 12:28:09 crc kubenswrapper[5052]: I1203 12:28:09.108946 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-keystone-public-svc" Dec 03 12:28:09 crc kubenswrapper[5052]: I1203 12:28:09.112363 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-68cf4f75d7-6pwrw"] Dec 03 12:28:09 crc kubenswrapper[5052]: I1203 12:28:09.128863 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-keystone-internal-svc" Dec 03 12:28:09 crc kubenswrapper[5052]: I1203 12:28:09.205427 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e381ca57-c1a2-4eb5-91db-760bc2c4f8d6-combined-ca-bundle\") pod \"keystone-68cf4f75d7-6pwrw\" (UID: \"e381ca57-c1a2-4eb5-91db-760bc2c4f8d6\") " pod="openstack-kuttl-tests/keystone-68cf4f75d7-6pwrw" Dec 03 12:28:09 crc kubenswrapper[5052]: I1203 12:28:09.205529 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/e381ca57-c1a2-4eb5-91db-760bc2c4f8d6-fernet-keys\") pod \"keystone-68cf4f75d7-6pwrw\" (UID: \"e381ca57-c1a2-4eb5-91db-760bc2c4f8d6\") " pod="openstack-kuttl-tests/keystone-68cf4f75d7-6pwrw" Dec 03 12:28:09 crc kubenswrapper[5052]: I1203 12:28:09.205576 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/e381ca57-c1a2-4eb5-91db-760bc2c4f8d6-config-data\") pod \"keystone-68cf4f75d7-6pwrw\" (UID: \"e381ca57-c1a2-4eb5-91db-760bc2c4f8d6\") " pod="openstack-kuttl-tests/keystone-68cf4f75d7-6pwrw" Dec 03 12:28:09 crc kubenswrapper[5052]: I1203 12:28:09.205608 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/e381ca57-c1a2-4eb5-91db-760bc2c4f8d6-internal-tls-certs\") pod \"keystone-68cf4f75d7-6pwrw\" (UID: \"e381ca57-c1a2-4eb5-91db-760bc2c4f8d6\") " pod="openstack-kuttl-tests/keystone-68cf4f75d7-6pwrw" Dec 03 12:28:09 crc kubenswrapper[5052]: I1203 12:28:09.205712 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/e381ca57-c1a2-4eb5-91db-760bc2c4f8d6-public-tls-certs\") pod \"keystone-68cf4f75d7-6pwrw\" (UID: \"e381ca57-c1a2-4eb5-91db-760bc2c4f8d6\") " pod="openstack-kuttl-tests/keystone-68cf4f75d7-6pwrw" Dec 03 12:28:09 crc kubenswrapper[5052]: I1203 12:28:09.205778 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/e381ca57-c1a2-4eb5-91db-760bc2c4f8d6-scripts\") pod \"keystone-68cf4f75d7-6pwrw\" (UID: \"e381ca57-c1a2-4eb5-91db-760bc2c4f8d6\") " pod="openstack-kuttl-tests/keystone-68cf4f75d7-6pwrw" Dec 03 12:28:09 crc kubenswrapper[5052]: I1203 12:28:09.205817 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/e381ca57-c1a2-4eb5-91db-760bc2c4f8d6-credential-keys\") pod \"keystone-68cf4f75d7-6pwrw\" (UID: \"e381ca57-c1a2-4eb5-91db-760bc2c4f8d6\") " pod="openstack-kuttl-tests/keystone-68cf4f75d7-6pwrw" Dec 03 12:28:09 crc kubenswrapper[5052]: I1203 12:28:09.205896 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-55hzr\" (UniqueName: \"kubernetes.io/projected/e381ca57-c1a2-4eb5-91db-760bc2c4f8d6-kube-api-access-55hzr\") pod \"keystone-68cf4f75d7-6pwrw\" (UID: \"e381ca57-c1a2-4eb5-91db-760bc2c4f8d6\") " pod="openstack-kuttl-tests/keystone-68cf4f75d7-6pwrw" Dec 03 12:28:09 crc kubenswrapper[5052]: E1203 12:28:09.210678 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="df1192fd6167f45271208db821d1f0909fc60a2b10aada89f5e4d0015c104317" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:28:09 crc kubenswrapper[5052]: E1203 12:28:09.222642 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="df1192fd6167f45271208db821d1f0909fc60a2b10aada89f5e4d0015c104317" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:28:09 crc kubenswrapper[5052]: E1203 12:28:09.231378 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="df1192fd6167f45271208db821d1f0909fc60a2b10aada89f5e4d0015c104317" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:28:09 crc kubenswrapper[5052]: E1203 12:28:09.231455 5052 prober.go:104] "Probe errored" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" probeType="Readiness" pod="openstack-kuttl-tests/nova-cell1-conductor-0" podUID="fcee2113-2b2f-47f9-b118-4f7488ad604e" containerName="nova-cell1-conductor-conductor" Dec 03 12:28:09 crc kubenswrapper[5052]: I1203 12:28:09.307125 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e381ca57-c1a2-4eb5-91db-760bc2c4f8d6-combined-ca-bundle\") pod \"keystone-68cf4f75d7-6pwrw\" (UID: \"e381ca57-c1a2-4eb5-91db-760bc2c4f8d6\") " pod="openstack-kuttl-tests/keystone-68cf4f75d7-6pwrw" Dec 03 12:28:09 crc kubenswrapper[5052]: I1203 12:28:09.307493 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/e381ca57-c1a2-4eb5-91db-760bc2c4f8d6-fernet-keys\") pod \"keystone-68cf4f75d7-6pwrw\" (UID: \"e381ca57-c1a2-4eb5-91db-760bc2c4f8d6\") " pod="openstack-kuttl-tests/keystone-68cf4f75d7-6pwrw" Dec 03 12:28:09 crc kubenswrapper[5052]: I1203 12:28:09.310841 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/e381ca57-c1a2-4eb5-91db-760bc2c4f8d6-config-data\") pod \"keystone-68cf4f75d7-6pwrw\" (UID: \"e381ca57-c1a2-4eb5-91db-760bc2c4f8d6\") " pod="openstack-kuttl-tests/keystone-68cf4f75d7-6pwrw" Dec 03 12:28:09 crc kubenswrapper[5052]: I1203 12:28:09.310896 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/e381ca57-c1a2-4eb5-91db-760bc2c4f8d6-internal-tls-certs\") pod \"keystone-68cf4f75d7-6pwrw\" (UID: \"e381ca57-c1a2-4eb5-91db-760bc2c4f8d6\") " pod="openstack-kuttl-tests/keystone-68cf4f75d7-6pwrw" Dec 03 12:28:09 crc kubenswrapper[5052]: I1203 12:28:09.311024 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/e381ca57-c1a2-4eb5-91db-760bc2c4f8d6-public-tls-certs\") pod \"keystone-68cf4f75d7-6pwrw\" (UID: \"e381ca57-c1a2-4eb5-91db-760bc2c4f8d6\") " pod="openstack-kuttl-tests/keystone-68cf4f75d7-6pwrw" Dec 03 12:28:09 crc kubenswrapper[5052]: I1203 12:28:09.311109 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/e381ca57-c1a2-4eb5-91db-760bc2c4f8d6-scripts\") pod \"keystone-68cf4f75d7-6pwrw\" (UID: \"e381ca57-c1a2-4eb5-91db-760bc2c4f8d6\") " pod="openstack-kuttl-tests/keystone-68cf4f75d7-6pwrw" Dec 03 12:28:09 crc kubenswrapper[5052]: I1203 12:28:09.311150 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/e381ca57-c1a2-4eb5-91db-760bc2c4f8d6-credential-keys\") pod \"keystone-68cf4f75d7-6pwrw\" (UID: \"e381ca57-c1a2-4eb5-91db-760bc2c4f8d6\") " pod="openstack-kuttl-tests/keystone-68cf4f75d7-6pwrw" Dec 03 12:28:09 crc kubenswrapper[5052]: I1203 12:28:09.311222 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-55hzr\" (UniqueName: \"kubernetes.io/projected/e381ca57-c1a2-4eb5-91db-760bc2c4f8d6-kube-api-access-55hzr\") pod \"keystone-68cf4f75d7-6pwrw\" (UID: \"e381ca57-c1a2-4eb5-91db-760bc2c4f8d6\") " pod="openstack-kuttl-tests/keystone-68cf4f75d7-6pwrw" Dec 03 12:28:09 crc kubenswrapper[5052]: I1203 12:28:09.319769 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/e381ca57-c1a2-4eb5-91db-760bc2c4f8d6-public-tls-certs\") pod \"keystone-68cf4f75d7-6pwrw\" (UID: \"e381ca57-c1a2-4eb5-91db-760bc2c4f8d6\") " pod="openstack-kuttl-tests/keystone-68cf4f75d7-6pwrw" Dec 03 12:28:09 crc kubenswrapper[5052]: I1203 12:28:09.321474 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/e381ca57-c1a2-4eb5-91db-760bc2c4f8d6-config-data\") pod \"keystone-68cf4f75d7-6pwrw\" (UID: \"e381ca57-c1a2-4eb5-91db-760bc2c4f8d6\") " pod="openstack-kuttl-tests/keystone-68cf4f75d7-6pwrw" Dec 03 12:28:09 crc kubenswrapper[5052]: I1203 12:28:09.321807 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/e381ca57-c1a2-4eb5-91db-760bc2c4f8d6-scripts\") pod \"keystone-68cf4f75d7-6pwrw\" (UID: \"e381ca57-c1a2-4eb5-91db-760bc2c4f8d6\") " pod="openstack-kuttl-tests/keystone-68cf4f75d7-6pwrw" Dec 03 12:28:09 crc kubenswrapper[5052]: I1203 12:28:09.322287 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e381ca57-c1a2-4eb5-91db-760bc2c4f8d6-combined-ca-bundle\") pod \"keystone-68cf4f75d7-6pwrw\" (UID: \"e381ca57-c1a2-4eb5-91db-760bc2c4f8d6\") " pod="openstack-kuttl-tests/keystone-68cf4f75d7-6pwrw" Dec 03 12:28:09 crc kubenswrapper[5052]: I1203 12:28:09.323004 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/e381ca57-c1a2-4eb5-91db-760bc2c4f8d6-internal-tls-certs\") pod \"keystone-68cf4f75d7-6pwrw\" (UID: \"e381ca57-c1a2-4eb5-91db-760bc2c4f8d6\") " pod="openstack-kuttl-tests/keystone-68cf4f75d7-6pwrw" Dec 03 12:28:09 crc kubenswrapper[5052]: I1203 12:28:09.323393 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/e381ca57-c1a2-4eb5-91db-760bc2c4f8d6-credential-keys\") pod \"keystone-68cf4f75d7-6pwrw\" (UID: \"e381ca57-c1a2-4eb5-91db-760bc2c4f8d6\") " pod="openstack-kuttl-tests/keystone-68cf4f75d7-6pwrw" Dec 03 12:28:09 crc kubenswrapper[5052]: I1203 12:28:09.328939 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/e381ca57-c1a2-4eb5-91db-760bc2c4f8d6-fernet-keys\") pod \"keystone-68cf4f75d7-6pwrw\" (UID: \"e381ca57-c1a2-4eb5-91db-760bc2c4f8d6\") " pod="openstack-kuttl-tests/keystone-68cf4f75d7-6pwrw" Dec 03 12:28:09 crc kubenswrapper[5052]: I1203 12:28:09.337523 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-55hzr\" (UniqueName: \"kubernetes.io/projected/e381ca57-c1a2-4eb5-91db-760bc2c4f8d6-kube-api-access-55hzr\") pod \"keystone-68cf4f75d7-6pwrw\" (UID: \"e381ca57-c1a2-4eb5-91db-760bc2c4f8d6\") " pod="openstack-kuttl-tests/keystone-68cf4f75d7-6pwrw" Dec 03 12:28:09 crc kubenswrapper[5052]: I1203 12:28:09.429183 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-68cf4f75d7-6pwrw" Dec 03 12:28:09 crc kubenswrapper[5052]: I1203 12:28:09.429529 5052 scope.go:117] "RemoveContainer" containerID="d090e68312aedebe296e1c2b129632e2019e3685d8daf2a12e3a6ceb18d11578" Dec 03 12:28:09 crc kubenswrapper[5052]: I1203 12:28:09.490103 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/certified-operators-75j45"] Dec 03 12:28:09 crc kubenswrapper[5052]: I1203 12:28:09.542859 5052 scope.go:117] "RemoveContainer" containerID="45053dc66310b77c52cb144a03e6e50f334cdd69a31d2716d30553971203e459" Dec 03 12:28:09 crc kubenswrapper[5052]: I1203 12:28:09.869598 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:28:10 crc kubenswrapper[5052]: I1203 12:28:10.012648 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/certified-operators-75j45" event={"ID":"c03ce333-5bdc-4cdb-9a81-53db1322b7bf","Type":"ContainerStarted","Data":"20d0d6b095d35bba4010e4db05af2ee2d91770c7e3da2b155855b734fe480d55"} Dec 03 12:28:10 crc kubenswrapper[5052]: E1203 12:28:10.245626 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="790991ac480a9e358efc31b18c1e765ec3b8f7c9c3386cf9c51c6dc7f800750c" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:28:10 crc kubenswrapper[5052]: E1203 12:28:10.247761 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="790991ac480a9e358efc31b18c1e765ec3b8f7c9c3386cf9c51c6dc7f800750c" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:28:10 crc kubenswrapper[5052]: E1203 12:28:10.248937 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="790991ac480a9e358efc31b18c1e765ec3b8f7c9c3386cf9c51c6dc7f800750c" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:28:10 crc kubenswrapper[5052]: E1203 12:28:10.248969 5052 prober.go:104] "Probe errored" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" probeType="Readiness" pod="openstack-kuttl-tests/nova-cell0-conductor-0" podUID="c080dc7f-df72-487f-b2c1-83b143d34784" containerName="nova-cell0-conductor-conductor" Dec 03 12:28:10 crc kubenswrapper[5052]: I1203 12:28:10.263036 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-68cf4f75d7-6pwrw"] Dec 03 12:28:11 crc kubenswrapper[5052]: I1203 12:28:11.030347 5052 generic.go:334] "Generic (PLEG): container finished" podID="c03ce333-5bdc-4cdb-9a81-53db1322b7bf" containerID="969d3991d620214517bbadf0f483ee93450cc288b469877f94b38cc69da68c7c" exitCode=0 Dec 03 12:28:11 crc kubenswrapper[5052]: I1203 12:28:11.030996 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/certified-operators-75j45" event={"ID":"c03ce333-5bdc-4cdb-9a81-53db1322b7bf","Type":"ContainerDied","Data":"969d3991d620214517bbadf0f483ee93450cc288b469877f94b38cc69da68c7c"} Dec 03 12:28:11 crc kubenswrapper[5052]: I1203 12:28:11.079935 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-68cf4f75d7-6pwrw" event={"ID":"e381ca57-c1a2-4eb5-91db-760bc2c4f8d6","Type":"ContainerStarted","Data":"d19da63420f13dff7e86e98ffcfd9d0d5b7be226d32bac80e02cb52bd5e47040"} Dec 03 12:28:12 crc kubenswrapper[5052]: I1203 12:28:12.092970 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-68cf4f75d7-6pwrw" event={"ID":"e381ca57-c1a2-4eb5-91db-760bc2c4f8d6","Type":"ContainerStarted","Data":"80156d90682246d7cf6bcb1366f8a297c8108fbbbd9601898bba95b488041705"} Dec 03 12:28:12 crc kubenswrapper[5052]: I1203 12:28:12.093637 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/keystone-68cf4f75d7-6pwrw" Dec 03 12:28:12 crc kubenswrapper[5052]: I1203 12:28:12.145369 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/keystone-68cf4f75d7-6pwrw" podStartSLOduration=3.145343368 podStartE2EDuration="3.145343368s" podCreationTimestamp="2025-12-03 12:28:09 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:28:12.136678721 +0000 UTC m=+4082.724920495" watchObservedRunningTime="2025-12-03 12:28:12.145343368 +0000 UTC m=+4082.733585132" Dec 03 12:28:12 crc kubenswrapper[5052]: I1203 12:28:12.356771 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:28:12 crc kubenswrapper[5052]: I1203 12:28:12.357112 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:28:12 crc kubenswrapper[5052]: I1203 12:28:12.395410 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:28:12 crc kubenswrapper[5052]: I1203 12:28:12.408195 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:28:13 crc kubenswrapper[5052]: I1203 12:28:13.126326 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/certified-operators-75j45" event={"ID":"c03ce333-5bdc-4cdb-9a81-53db1322b7bf","Type":"ContainerStarted","Data":"2d6610eeb64b5a8303646130c31ca4e821ae8775d47940490419f30250450ca1"} Dec 03 12:28:13 crc kubenswrapper[5052]: I1203 12:28:13.127591 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:28:13 crc kubenswrapper[5052]: I1203 12:28:13.128110 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:28:13 crc kubenswrapper[5052]: E1203 12:28:13.345672 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="47d8a341afac3d19e0311930b0aa745891784bd644a5b9f1e35346d168347db9" cmd=["/usr/bin/pgrep","-r","DRST","nova-scheduler"] Dec 03 12:28:13 crc kubenswrapper[5052]: E1203 12:28:13.348265 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="47d8a341afac3d19e0311930b0aa745891784bd644a5b9f1e35346d168347db9" cmd=["/usr/bin/pgrep","-r","DRST","nova-scheduler"] Dec 03 12:28:13 crc kubenswrapper[5052]: E1203 12:28:13.350000 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="47d8a341afac3d19e0311930b0aa745891784bd644a5b9f1e35346d168347db9" cmd=["/usr/bin/pgrep","-r","DRST","nova-scheduler"] Dec 03 12:28:13 crc kubenswrapper[5052]: E1203 12:28:13.350078 5052 prober.go:104] "Probe errored" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" probeType="Readiness" pod="openstack-kuttl-tests/nova-scheduler-0" podUID="8a0b169d-6971-4530-936e-8e5f0f57816f" containerName="nova-scheduler-scheduler" Dec 03 12:28:14 crc kubenswrapper[5052]: E1203 12:28:14.204501 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="df1192fd6167f45271208db821d1f0909fc60a2b10aada89f5e4d0015c104317" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:28:14 crc kubenswrapper[5052]: E1203 12:28:14.209964 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="df1192fd6167f45271208db821d1f0909fc60a2b10aada89f5e4d0015c104317" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:28:14 crc kubenswrapper[5052]: E1203 12:28:14.211990 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="df1192fd6167f45271208db821d1f0909fc60a2b10aada89f5e4d0015c104317" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:28:14 crc kubenswrapper[5052]: E1203 12:28:14.212044 5052 prober.go:104] "Probe errored" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" probeType="Readiness" pod="openstack-kuttl-tests/nova-cell1-conductor-0" podUID="fcee2113-2b2f-47f9-b118-4f7488ad604e" containerName="nova-cell1-conductor-conductor" Dec 03 12:28:14 crc kubenswrapper[5052]: I1203 12:28:14.256154 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:28:14 crc kubenswrapper[5052]: I1203 12:28:14.362748 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:28:15 crc kubenswrapper[5052]: I1203 12:28:15.163429 5052 generic.go:334] "Generic (PLEG): container finished" podID="c03ce333-5bdc-4cdb-9a81-53db1322b7bf" containerID="2d6610eeb64b5a8303646130c31ca4e821ae8775d47940490419f30250450ca1" exitCode=0 Dec 03 12:28:15 crc kubenswrapper[5052]: I1203 12:28:15.164112 5052 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Dec 03 12:28:15 crc kubenswrapper[5052]: I1203 12:28:15.164127 5052 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Dec 03 12:28:15 crc kubenswrapper[5052]: I1203 12:28:15.164113 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/certified-operators-75j45" event={"ID":"c03ce333-5bdc-4cdb-9a81-53db1322b7bf","Type":"ContainerDied","Data":"2d6610eeb64b5a8303646130c31ca4e821ae8775d47940490419f30250450ca1"} Dec 03 12:28:15 crc kubenswrapper[5052]: I1203 12:28:15.182924 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell0-cell-mapping-d2r55"] Dec 03 12:28:15 crc kubenswrapper[5052]: I1203 12:28:15.207657 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell0-cell-mapping-d2r55"] Dec 03 12:28:15 crc kubenswrapper[5052]: E1203 12:28:15.243694 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="790991ac480a9e358efc31b18c1e765ec3b8f7c9c3386cf9c51c6dc7f800750c" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:28:15 crc kubenswrapper[5052]: E1203 12:28:15.252640 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="790991ac480a9e358efc31b18c1e765ec3b8f7c9c3386cf9c51c6dc7f800750c" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:28:15 crc kubenswrapper[5052]: I1203 12:28:15.259048 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:28:15 crc kubenswrapper[5052]: I1203 12:28:15.259321 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-api-0" podUID="810e47c2-430b-424d-92ea-66c20ec8501f" containerName="nova-api-log" containerID="cri-o://85ec55fcdf485c866d81bbe48593514b623381629978696113751cab29e59861" gracePeriod=30 Dec 03 12:28:15 crc kubenswrapper[5052]: E1203 12:28:15.259007 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="790991ac480a9e358efc31b18c1e765ec3b8f7c9c3386cf9c51c6dc7f800750c" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:28:15 crc kubenswrapper[5052]: E1203 12:28:15.259484 5052 prober.go:104] "Probe errored" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" probeType="Readiness" pod="openstack-kuttl-tests/nova-cell0-conductor-0" podUID="c080dc7f-df72-487f-b2c1-83b143d34784" containerName="nova-cell0-conductor-conductor" Dec 03 12:28:15 crc kubenswrapper[5052]: I1203 12:28:15.259951 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-api-0" podUID="810e47c2-430b-424d-92ea-66c20ec8501f" containerName="nova-api-api" containerID="cri-o://22904b310fc3f26f046b8a7eda3ddcc73a97bb5641a37e30766bd99ffba27bde" gracePeriod=30 Dec 03 12:28:15 crc kubenswrapper[5052]: I1203 12:28:15.371164 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:28:15 crc kubenswrapper[5052]: I1203 12:28:15.371835 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:28:15 crc kubenswrapper[5052]: I1203 12:28:15.379662 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:28:15 crc kubenswrapper[5052]: I1203 12:28:15.380066 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-metadata-0" podUID="8a2c61ae-6521-424f-aeac-52e1ec1c319a" containerName="nova-metadata-log" containerID="cri-o://90ccfa6fea4d46147f2030f8a0f1ffee182ce45ef8183a711e81ed3a0eaa5b15" gracePeriod=30 Dec 03 12:28:15 crc kubenswrapper[5052]: I1203 12:28:15.380226 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-metadata-0" podUID="8a2c61ae-6521-424f-aeac-52e1ec1c319a" containerName="nova-metadata-metadata" containerID="cri-o://70af550381ae175843af90fae4ec5c50a17835a3a8274e54adab63fa88f5a895" gracePeriod=30 Dec 03 12:28:15 crc kubenswrapper[5052]: I1203 12:28:15.428005 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:28:15 crc kubenswrapper[5052]: I1203 12:28:15.428622 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:28:15 crc kubenswrapper[5052]: I1203 12:28:15.631981 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell0-cell-mapping-6nbfg"] Dec 03 12:28:15 crc kubenswrapper[5052]: I1203 12:28:15.636375 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-cell-mapping-6nbfg" Dec 03 12:28:15 crc kubenswrapper[5052]: I1203 12:28:15.641969 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell0-manage-scripts" Dec 03 12:28:15 crc kubenswrapper[5052]: I1203 12:28:15.645234 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell0-manage-config-data" Dec 03 12:28:15 crc kubenswrapper[5052]: I1203 12:28:15.652537 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell0-cell-mapping-6nbfg"] Dec 03 12:28:15 crc kubenswrapper[5052]: I1203 12:28:15.677213 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:28:15 crc kubenswrapper[5052]: I1203 12:28:15.717928 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f5999941-d60a-412e-a5d8-8f4e657e0014" path="/var/lib/kubelet/pods/f5999941-d60a-412e-a5d8-8f4e657e0014/volumes" Dec 03 12:28:15 crc kubenswrapper[5052]: I1203 12:28:15.783968 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xl5r4\" (UniqueName: \"kubernetes.io/projected/e2d1b100-5918-4166-9fa1-5a094612727f-kube-api-access-xl5r4\") pod \"nova-cell0-cell-mapping-6nbfg\" (UID: \"e2d1b100-5918-4166-9fa1-5a094612727f\") " pod="openstack-kuttl-tests/nova-cell0-cell-mapping-6nbfg" Dec 03 12:28:15 crc kubenswrapper[5052]: I1203 12:28:15.784069 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e2d1b100-5918-4166-9fa1-5a094612727f-combined-ca-bundle\") pod \"nova-cell0-cell-mapping-6nbfg\" (UID: \"e2d1b100-5918-4166-9fa1-5a094612727f\") " pod="openstack-kuttl-tests/nova-cell0-cell-mapping-6nbfg" Dec 03 12:28:15 crc kubenswrapper[5052]: I1203 12:28:15.784123 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/e2d1b100-5918-4166-9fa1-5a094612727f-config-data\") pod \"nova-cell0-cell-mapping-6nbfg\" (UID: \"e2d1b100-5918-4166-9fa1-5a094612727f\") " pod="openstack-kuttl-tests/nova-cell0-cell-mapping-6nbfg" Dec 03 12:28:15 crc kubenswrapper[5052]: I1203 12:28:15.784174 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/e2d1b100-5918-4166-9fa1-5a094612727f-scripts\") pod \"nova-cell0-cell-mapping-6nbfg\" (UID: \"e2d1b100-5918-4166-9fa1-5a094612727f\") " pod="openstack-kuttl-tests/nova-cell0-cell-mapping-6nbfg" Dec 03 12:28:15 crc kubenswrapper[5052]: I1203 12:28:15.885775 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-xl5r4\" (UniqueName: \"kubernetes.io/projected/e2d1b100-5918-4166-9fa1-5a094612727f-kube-api-access-xl5r4\") pod \"nova-cell0-cell-mapping-6nbfg\" (UID: \"e2d1b100-5918-4166-9fa1-5a094612727f\") " pod="openstack-kuttl-tests/nova-cell0-cell-mapping-6nbfg" Dec 03 12:28:15 crc kubenswrapper[5052]: I1203 12:28:15.885871 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e2d1b100-5918-4166-9fa1-5a094612727f-combined-ca-bundle\") pod \"nova-cell0-cell-mapping-6nbfg\" (UID: \"e2d1b100-5918-4166-9fa1-5a094612727f\") " pod="openstack-kuttl-tests/nova-cell0-cell-mapping-6nbfg" Dec 03 12:28:15 crc kubenswrapper[5052]: I1203 12:28:15.885937 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/e2d1b100-5918-4166-9fa1-5a094612727f-config-data\") pod \"nova-cell0-cell-mapping-6nbfg\" (UID: \"e2d1b100-5918-4166-9fa1-5a094612727f\") " pod="openstack-kuttl-tests/nova-cell0-cell-mapping-6nbfg" Dec 03 12:28:15 crc kubenswrapper[5052]: I1203 12:28:15.885993 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/e2d1b100-5918-4166-9fa1-5a094612727f-scripts\") pod \"nova-cell0-cell-mapping-6nbfg\" (UID: \"e2d1b100-5918-4166-9fa1-5a094612727f\") " pod="openstack-kuttl-tests/nova-cell0-cell-mapping-6nbfg" Dec 03 12:28:15 crc kubenswrapper[5052]: I1203 12:28:15.906112 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-xl5r4\" (UniqueName: \"kubernetes.io/projected/e2d1b100-5918-4166-9fa1-5a094612727f-kube-api-access-xl5r4\") pod \"nova-cell0-cell-mapping-6nbfg\" (UID: \"e2d1b100-5918-4166-9fa1-5a094612727f\") " pod="openstack-kuttl-tests/nova-cell0-cell-mapping-6nbfg" Dec 03 12:28:15 crc kubenswrapper[5052]: I1203 12:28:15.907338 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e2d1b100-5918-4166-9fa1-5a094612727f-combined-ca-bundle\") pod \"nova-cell0-cell-mapping-6nbfg\" (UID: \"e2d1b100-5918-4166-9fa1-5a094612727f\") " pod="openstack-kuttl-tests/nova-cell0-cell-mapping-6nbfg" Dec 03 12:28:15 crc kubenswrapper[5052]: I1203 12:28:15.908024 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/e2d1b100-5918-4166-9fa1-5a094612727f-scripts\") pod \"nova-cell0-cell-mapping-6nbfg\" (UID: \"e2d1b100-5918-4166-9fa1-5a094612727f\") " pod="openstack-kuttl-tests/nova-cell0-cell-mapping-6nbfg" Dec 03 12:28:15 crc kubenswrapper[5052]: I1203 12:28:15.910020 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 12:28:15 crc kubenswrapper[5052]: I1203 12:28:15.910247 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" podUID="460b2796-2ea1-4ddd-a4e7-eae181ddfc81" containerName="nova-cell1-novncproxy-novncproxy" containerID="cri-o://c99aecbfc0f318ebcfe8e5c07b7758a65a70339a148d9b58bf3923d596c20d7b" gracePeriod=30 Dec 03 12:28:15 crc kubenswrapper[5052]: I1203 12:28:15.912182 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/e2d1b100-5918-4166-9fa1-5a094612727f-config-data\") pod \"nova-cell0-cell-mapping-6nbfg\" (UID: \"e2d1b100-5918-4166-9fa1-5a094612727f\") " pod="openstack-kuttl-tests/nova-cell0-cell-mapping-6nbfg" Dec 03 12:28:15 crc kubenswrapper[5052]: I1203 12:28:15.956467 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-cell-mapping-6nbfg" Dec 03 12:28:16 crc kubenswrapper[5052]: I1203 12:28:16.210138 5052 generic.go:334] "Generic (PLEG): container finished" podID="810e47c2-430b-424d-92ea-66c20ec8501f" containerID="85ec55fcdf485c866d81bbe48593514b623381629978696113751cab29e59861" exitCode=143 Dec 03 12:28:16 crc kubenswrapper[5052]: I1203 12:28:16.210226 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"810e47c2-430b-424d-92ea-66c20ec8501f","Type":"ContainerDied","Data":"85ec55fcdf485c866d81bbe48593514b623381629978696113751cab29e59861"} Dec 03 12:28:16 crc kubenswrapper[5052]: I1203 12:28:16.223928 5052 generic.go:334] "Generic (PLEG): container finished" podID="8a2c61ae-6521-424f-aeac-52e1ec1c319a" containerID="90ccfa6fea4d46147f2030f8a0f1ffee182ce45ef8183a711e81ed3a0eaa5b15" exitCode=143 Dec 03 12:28:16 crc kubenswrapper[5052]: I1203 12:28:16.224005 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"8a2c61ae-6521-424f-aeac-52e1ec1c319a","Type":"ContainerDied","Data":"90ccfa6fea4d46147f2030f8a0f1ffee182ce45ef8183a711e81ed3a0eaa5b15"} Dec 03 12:28:16 crc kubenswrapper[5052]: I1203 12:28:16.233581 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:28:16 crc kubenswrapper[5052]: I1203 12:28:16.240427 5052 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Dec 03 12:28:16 crc kubenswrapper[5052]: I1203 12:28:16.242069 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/certified-operators-75j45" event={"ID":"c03ce333-5bdc-4cdb-9a81-53db1322b7bf","Type":"ContainerStarted","Data":"aaf64268b97b9c5579d623002cd16894fe6902dd1ab89073573d9f967bab79ed"} Dec 03 12:28:16 crc kubenswrapper[5052]: I1203 12:28:16.243888 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:28:16 crc kubenswrapper[5052]: I1203 12:28:16.244060 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:28:16 crc kubenswrapper[5052]: I1203 12:28:16.244952 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell1-cell-mapping-7v2kh"] Dec 03 12:28:16 crc kubenswrapper[5052]: I1203 12:28:16.246389 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-cell-mapping-7v2kh" Dec 03 12:28:16 crc kubenswrapper[5052]: I1203 12:28:16.248678 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell1-manage-config-data" Dec 03 12:28:16 crc kubenswrapper[5052]: I1203 12:28:16.248858 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell1-manage-scripts" Dec 03 12:28:16 crc kubenswrapper[5052]: I1203 12:28:16.257119 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-cell-mapping-7v2kh"] Dec 03 12:28:16 crc kubenswrapper[5052]: I1203 12:28:16.290310 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-marketplace/certified-operators-75j45" podStartSLOduration=3.708129839 podStartE2EDuration="8.290288014s" podCreationTimestamp="2025-12-03 12:28:08 +0000 UTC" firstStartedPulling="2025-12-03 12:28:11.043687994 +0000 UTC m=+4081.631929758" lastFinishedPulling="2025-12-03 12:28:15.625846169 +0000 UTC m=+4086.214087933" observedRunningTime="2025-12-03 12:28:16.287153144 +0000 UTC m=+4086.875394918" watchObservedRunningTime="2025-12-03 12:28:16.290288014 +0000 UTC m=+4086.878529778" Dec 03 12:28:16 crc kubenswrapper[5052]: I1203 12:28:16.365462 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rq2rx\" (UniqueName: \"kubernetes.io/projected/51d1c5f3-0622-4c29-a19a-f1ecd293654e-kube-api-access-rq2rx\") pod \"nova-cell1-cell-mapping-7v2kh\" (UID: \"51d1c5f3-0622-4c29-a19a-f1ecd293654e\") " pod="openstack-kuttl-tests/nova-cell1-cell-mapping-7v2kh" Dec 03 12:28:16 crc kubenswrapper[5052]: I1203 12:28:16.365828 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/51d1c5f3-0622-4c29-a19a-f1ecd293654e-scripts\") pod \"nova-cell1-cell-mapping-7v2kh\" (UID: \"51d1c5f3-0622-4c29-a19a-f1ecd293654e\") " pod="openstack-kuttl-tests/nova-cell1-cell-mapping-7v2kh" Dec 03 12:28:16 crc kubenswrapper[5052]: I1203 12:28:16.365948 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/51d1c5f3-0622-4c29-a19a-f1ecd293654e-combined-ca-bundle\") pod \"nova-cell1-cell-mapping-7v2kh\" (UID: \"51d1c5f3-0622-4c29-a19a-f1ecd293654e\") " pod="openstack-kuttl-tests/nova-cell1-cell-mapping-7v2kh" Dec 03 12:28:16 crc kubenswrapper[5052]: I1203 12:28:16.366238 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/51d1c5f3-0622-4c29-a19a-f1ecd293654e-config-data\") pod \"nova-cell1-cell-mapping-7v2kh\" (UID: \"51d1c5f3-0622-4c29-a19a-f1ecd293654e\") " pod="openstack-kuttl-tests/nova-cell1-cell-mapping-7v2kh" Dec 03 12:28:16 crc kubenswrapper[5052]: I1203 12:28:16.468244 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/51d1c5f3-0622-4c29-a19a-f1ecd293654e-config-data\") pod \"nova-cell1-cell-mapping-7v2kh\" (UID: \"51d1c5f3-0622-4c29-a19a-f1ecd293654e\") " pod="openstack-kuttl-tests/nova-cell1-cell-mapping-7v2kh" Dec 03 12:28:16 crc kubenswrapper[5052]: I1203 12:28:16.468680 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-rq2rx\" (UniqueName: \"kubernetes.io/projected/51d1c5f3-0622-4c29-a19a-f1ecd293654e-kube-api-access-rq2rx\") pod \"nova-cell1-cell-mapping-7v2kh\" (UID: \"51d1c5f3-0622-4c29-a19a-f1ecd293654e\") " pod="openstack-kuttl-tests/nova-cell1-cell-mapping-7v2kh" Dec 03 12:28:16 crc kubenswrapper[5052]: I1203 12:28:16.468782 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/51d1c5f3-0622-4c29-a19a-f1ecd293654e-scripts\") pod \"nova-cell1-cell-mapping-7v2kh\" (UID: \"51d1c5f3-0622-4c29-a19a-f1ecd293654e\") " pod="openstack-kuttl-tests/nova-cell1-cell-mapping-7v2kh" Dec 03 12:28:16 crc kubenswrapper[5052]: I1203 12:28:16.468821 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/51d1c5f3-0622-4c29-a19a-f1ecd293654e-combined-ca-bundle\") pod \"nova-cell1-cell-mapping-7v2kh\" (UID: \"51d1c5f3-0622-4c29-a19a-f1ecd293654e\") " pod="openstack-kuttl-tests/nova-cell1-cell-mapping-7v2kh" Dec 03 12:28:16 crc kubenswrapper[5052]: I1203 12:28:16.477089 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/51d1c5f3-0622-4c29-a19a-f1ecd293654e-scripts\") pod \"nova-cell1-cell-mapping-7v2kh\" (UID: \"51d1c5f3-0622-4c29-a19a-f1ecd293654e\") " pod="openstack-kuttl-tests/nova-cell1-cell-mapping-7v2kh" Dec 03 12:28:16 crc kubenswrapper[5052]: I1203 12:28:16.480242 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/51d1c5f3-0622-4c29-a19a-f1ecd293654e-combined-ca-bundle\") pod \"nova-cell1-cell-mapping-7v2kh\" (UID: \"51d1c5f3-0622-4c29-a19a-f1ecd293654e\") " pod="openstack-kuttl-tests/nova-cell1-cell-mapping-7v2kh" Dec 03 12:28:16 crc kubenswrapper[5052]: I1203 12:28:16.486427 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/51d1c5f3-0622-4c29-a19a-f1ecd293654e-config-data\") pod \"nova-cell1-cell-mapping-7v2kh\" (UID: \"51d1c5f3-0622-4c29-a19a-f1ecd293654e\") " pod="openstack-kuttl-tests/nova-cell1-cell-mapping-7v2kh" Dec 03 12:28:16 crc kubenswrapper[5052]: I1203 12:28:16.489329 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-rq2rx\" (UniqueName: \"kubernetes.io/projected/51d1c5f3-0622-4c29-a19a-f1ecd293654e-kube-api-access-rq2rx\") pod \"nova-cell1-cell-mapping-7v2kh\" (UID: \"51d1c5f3-0622-4c29-a19a-f1ecd293654e\") " pod="openstack-kuttl-tests/nova-cell1-cell-mapping-7v2kh" Dec 03 12:28:16 crc kubenswrapper[5052]: I1203 12:28:16.581025 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-cell-mapping-7v2kh" Dec 03 12:28:16 crc kubenswrapper[5052]: I1203 12:28:16.607621 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell0-cell-mapping-6nbfg"] Dec 03 12:28:16 crc kubenswrapper[5052]: I1203 12:28:16.856589 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:28:17 crc kubenswrapper[5052]: I1203 12:28:17.251723 5052 generic.go:334] "Generic (PLEG): container finished" podID="460b2796-2ea1-4ddd-a4e7-eae181ddfc81" containerID="c99aecbfc0f318ebcfe8e5c07b7758a65a70339a148d9b58bf3923d596c20d7b" exitCode=0 Dec 03 12:28:17 crc kubenswrapper[5052]: I1203 12:28:17.251804 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" event={"ID":"460b2796-2ea1-4ddd-a4e7-eae181ddfc81","Type":"ContainerDied","Data":"c99aecbfc0f318ebcfe8e5c07b7758a65a70339a148d9b58bf3923d596c20d7b"} Dec 03 12:28:17 crc kubenswrapper[5052]: I1203 12:28:17.255126 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-cell-mapping-6nbfg" event={"ID":"e2d1b100-5918-4166-9fa1-5a094612727f","Type":"ContainerStarted","Data":"82b7c2702af47b346cff159435051eca59fb061ec03e5c7b1b1b84e2fffe24d8"} Dec 03 12:28:17 crc kubenswrapper[5052]: I1203 12:28:17.255171 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-cell-mapping-7v2kh"] Dec 03 12:28:17 crc kubenswrapper[5052]: I1203 12:28:17.996851 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:28:18 crc kubenswrapper[5052]: I1203 12:28:18.157248 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/460b2796-2ea1-4ddd-a4e7-eae181ddfc81-config-data\") pod \"460b2796-2ea1-4ddd-a4e7-eae181ddfc81\" (UID: \"460b2796-2ea1-4ddd-a4e7-eae181ddfc81\") " Dec 03 12:28:18 crc kubenswrapper[5052]: I1203 12:28:18.157667 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-899hm\" (UniqueName: \"kubernetes.io/projected/460b2796-2ea1-4ddd-a4e7-eae181ddfc81-kube-api-access-899hm\") pod \"460b2796-2ea1-4ddd-a4e7-eae181ddfc81\" (UID: \"460b2796-2ea1-4ddd-a4e7-eae181ddfc81\") " Dec 03 12:28:18 crc kubenswrapper[5052]: I1203 12:28:18.157896 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/460b2796-2ea1-4ddd-a4e7-eae181ddfc81-combined-ca-bundle\") pod \"460b2796-2ea1-4ddd-a4e7-eae181ddfc81\" (UID: \"460b2796-2ea1-4ddd-a4e7-eae181ddfc81\") " Dec 03 12:28:18 crc kubenswrapper[5052]: I1203 12:28:18.167687 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/460b2796-2ea1-4ddd-a4e7-eae181ddfc81-kube-api-access-899hm" (OuterVolumeSpecName: "kube-api-access-899hm") pod "460b2796-2ea1-4ddd-a4e7-eae181ddfc81" (UID: "460b2796-2ea1-4ddd-a4e7-eae181ddfc81"). InnerVolumeSpecName "kube-api-access-899hm". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:28:18 crc kubenswrapper[5052]: I1203 12:28:18.208640 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/460b2796-2ea1-4ddd-a4e7-eae181ddfc81-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "460b2796-2ea1-4ddd-a4e7-eae181ddfc81" (UID: "460b2796-2ea1-4ddd-a4e7-eae181ddfc81"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:28:18 crc kubenswrapper[5052]: I1203 12:28:18.211702 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/460b2796-2ea1-4ddd-a4e7-eae181ddfc81-config-data" (OuterVolumeSpecName: "config-data") pod "460b2796-2ea1-4ddd-a4e7-eae181ddfc81" (UID: "460b2796-2ea1-4ddd-a4e7-eae181ddfc81"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:28:18 crc kubenswrapper[5052]: I1203 12:28:18.259964 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/460b2796-2ea1-4ddd-a4e7-eae181ddfc81-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:18 crc kubenswrapper[5052]: I1203 12:28:18.260012 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-899hm\" (UniqueName: \"kubernetes.io/projected/460b2796-2ea1-4ddd-a4e7-eae181ddfc81-kube-api-access-899hm\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:18 crc kubenswrapper[5052]: I1203 12:28:18.260029 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/460b2796-2ea1-4ddd-a4e7-eae181ddfc81-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:18 crc kubenswrapper[5052]: I1203 12:28:18.267947 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-cell-mapping-6nbfg" event={"ID":"e2d1b100-5918-4166-9fa1-5a094612727f","Type":"ContainerStarted","Data":"8a517664093b91e9ef8c95cddf9f11b7aba074cb44d5aee9b5c50411254e47b7"} Dec 03 12:28:18 crc kubenswrapper[5052]: I1203 12:28:18.270768 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-cell-mapping-7v2kh" event={"ID":"51d1c5f3-0622-4c29-a19a-f1ecd293654e","Type":"ContainerStarted","Data":"fa20bc0e792f1502682a7b88dcf29415b5895d6f5c2357ab8bb5c18cd557e3f5"} Dec 03 12:28:18 crc kubenswrapper[5052]: I1203 12:28:18.272089 5052 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Dec 03 12:28:18 crc kubenswrapper[5052]: I1203 12:28:18.272104 5052 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Dec 03 12:28:18 crc kubenswrapper[5052]: I1203 12:28:18.272695 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:28:18 crc kubenswrapper[5052]: I1203 12:28:18.278726 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" event={"ID":"460b2796-2ea1-4ddd-a4e7-eae181ddfc81","Type":"ContainerDied","Data":"32b328087df01322dd99751590da4943c3be251fca301d63ddc59ce90100dda3"} Dec 03 12:28:18 crc kubenswrapper[5052]: I1203 12:28:18.278795 5052 scope.go:117] "RemoveContainer" containerID="c99aecbfc0f318ebcfe8e5c07b7758a65a70339a148d9b58bf3923d596c20d7b" Dec 03 12:28:18 crc kubenswrapper[5052]: E1203 12:28:18.343530 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="47d8a341afac3d19e0311930b0aa745891784bd644a5b9f1e35346d168347db9" cmd=["/usr/bin/pgrep","-r","DRST","nova-scheduler"] Dec 03 12:28:18 crc kubenswrapper[5052]: E1203 12:28:18.345017 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="47d8a341afac3d19e0311930b0aa745891784bd644a5b9f1e35346d168347db9" cmd=["/usr/bin/pgrep","-r","DRST","nova-scheduler"] Dec 03 12:28:18 crc kubenswrapper[5052]: E1203 12:28:18.346468 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="47d8a341afac3d19e0311930b0aa745891784bd644a5b9f1e35346d168347db9" cmd=["/usr/bin/pgrep","-r","DRST","nova-scheduler"] Dec 03 12:28:18 crc kubenswrapper[5052]: E1203 12:28:18.346606 5052 prober.go:104] "Probe errored" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" probeType="Readiness" pod="openstack-kuttl-tests/nova-scheduler-0" podUID="8a0b169d-6971-4530-936e-8e5f0f57816f" containerName="nova-scheduler-scheduler" Dec 03 12:28:18 crc kubenswrapper[5052]: I1203 12:28:18.802350 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/nova-metadata-0" podUID="8a2c61ae-6521-424f-aeac-52e1ec1c319a" containerName="nova-metadata-log" probeResult="failure" output="Get \"http://10.217.0.141:8775/\": read tcp 10.217.0.2:45244->10.217.0.141:8775: read: connection reset by peer" Dec 03 12:28:18 crc kubenswrapper[5052]: I1203 12:28:18.802350 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/nova-metadata-0" podUID="8a2c61ae-6521-424f-aeac-52e1ec1c319a" containerName="nova-metadata-metadata" probeResult="failure" output="Get \"http://10.217.0.141:8775/\": read tcp 10.217.0.2:45230->10.217.0.141:8775: read: connection reset by peer" Dec 03 12:28:19 crc kubenswrapper[5052]: I1203 12:28:19.055846 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-marketplace/certified-operators-75j45" Dec 03 12:28:19 crc kubenswrapper[5052]: I1203 12:28:19.055965 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openshift-marketplace/certified-operators-75j45" Dec 03 12:28:19 crc kubenswrapper[5052]: I1203 12:28:19.102136 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-marketplace/certified-operators-75j45" Dec 03 12:28:19 crc kubenswrapper[5052]: E1203 12:28:19.205413 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="df1192fd6167f45271208db821d1f0909fc60a2b10aada89f5e4d0015c104317" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:28:19 crc kubenswrapper[5052]: E1203 12:28:19.207235 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="df1192fd6167f45271208db821d1f0909fc60a2b10aada89f5e4d0015c104317" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:28:19 crc kubenswrapper[5052]: E1203 12:28:19.208244 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="df1192fd6167f45271208db821d1f0909fc60a2b10aada89f5e4d0015c104317" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:28:19 crc kubenswrapper[5052]: E1203 12:28:19.208287 5052 prober.go:104] "Probe errored" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" probeType="Readiness" pod="openstack-kuttl-tests/nova-cell1-conductor-0" podUID="fcee2113-2b2f-47f9-b118-4f7488ad604e" containerName="nova-cell1-conductor-conductor" Dec 03 12:28:19 crc kubenswrapper[5052]: I1203 12:28:19.362852 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/nova-api-0" podUID="810e47c2-430b-424d-92ea-66c20ec8501f" containerName="nova-api-log" probeResult="failure" output="Get \"http://10.217.0.138:8774/\": dial tcp 10.217.0.138:8774: connect: connection refused" Dec 03 12:28:19 crc kubenswrapper[5052]: I1203 12:28:19.363257 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/nova-api-0" podUID="810e47c2-430b-424d-92ea-66c20ec8501f" containerName="nova-api-api" probeResult="failure" output="Get \"http://10.217.0.138:8774/\": dial tcp 10.217.0.138:8774: connect: connection refused" Dec 03 12:28:19 crc kubenswrapper[5052]: I1203 12:28:19.376498 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-cell0-cell-mapping-6nbfg" podStartSLOduration=4.37647923 podStartE2EDuration="4.37647923s" podCreationTimestamp="2025-12-03 12:28:15 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:28:19.368988376 +0000 UTC m=+4089.957230140" watchObservedRunningTime="2025-12-03 12:28:19.37647923 +0000 UTC m=+4089.964720994" Dec 03 12:28:19 crc kubenswrapper[5052]: I1203 12:28:19.454591 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 12:28:19 crc kubenswrapper[5052]: I1203 12:28:19.482051 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 12:28:19 crc kubenswrapper[5052]: I1203 12:28:19.492806 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 12:28:19 crc kubenswrapper[5052]: E1203 12:28:19.493376 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="460b2796-2ea1-4ddd-a4e7-eae181ddfc81" containerName="nova-cell1-novncproxy-novncproxy" Dec 03 12:28:19 crc kubenswrapper[5052]: I1203 12:28:19.493398 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="460b2796-2ea1-4ddd-a4e7-eae181ddfc81" containerName="nova-cell1-novncproxy-novncproxy" Dec 03 12:28:19 crc kubenswrapper[5052]: I1203 12:28:19.493838 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="460b2796-2ea1-4ddd-a4e7-eae181ddfc81" containerName="nova-cell1-novncproxy-novncproxy" Dec 03 12:28:19 crc kubenswrapper[5052]: I1203 12:28:19.495373 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:28:19 crc kubenswrapper[5052]: I1203 12:28:19.500266 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-nova-novncproxy-cell1-public-svc" Dec 03 12:28:19 crc kubenswrapper[5052]: I1203 12:28:19.501635 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell1-novncproxy-config-data" Dec 03 12:28:19 crc kubenswrapper[5052]: I1203 12:28:19.501950 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-nova-novncproxy-cell1-vencrypt" Dec 03 12:28:19 crc kubenswrapper[5052]: I1203 12:28:19.509648 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 12:28:19 crc kubenswrapper[5052]: I1203 12:28:19.667988 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/bea08f45-6aef-4ccc-8252-d840baad2bf2-combined-ca-bundle\") pod \"nova-cell1-novncproxy-0\" (UID: \"bea08f45-6aef-4ccc-8252-d840baad2bf2\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:28:19 crc kubenswrapper[5052]: I1203 12:28:19.668276 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/bea08f45-6aef-4ccc-8252-d840baad2bf2-config-data\") pod \"nova-cell1-novncproxy-0\" (UID: \"bea08f45-6aef-4ccc-8252-d840baad2bf2\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:28:19 crc kubenswrapper[5052]: I1203 12:28:19.668355 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nova-novncproxy-tls-certs\" (UniqueName: \"kubernetes.io/secret/bea08f45-6aef-4ccc-8252-d840baad2bf2-nova-novncproxy-tls-certs\") pod \"nova-cell1-novncproxy-0\" (UID: \"bea08f45-6aef-4ccc-8252-d840baad2bf2\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:28:19 crc kubenswrapper[5052]: I1203 12:28:19.668544 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"vencrypt-tls-certs\" (UniqueName: \"kubernetes.io/secret/bea08f45-6aef-4ccc-8252-d840baad2bf2-vencrypt-tls-certs\") pod \"nova-cell1-novncproxy-0\" (UID: \"bea08f45-6aef-4ccc-8252-d840baad2bf2\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:28:19 crc kubenswrapper[5052]: I1203 12:28:19.668720 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cdbdz\" (UniqueName: \"kubernetes.io/projected/bea08f45-6aef-4ccc-8252-d840baad2bf2-kube-api-access-cdbdz\") pod \"nova-cell1-novncproxy-0\" (UID: \"bea08f45-6aef-4ccc-8252-d840baad2bf2\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:28:19 crc kubenswrapper[5052]: I1203 12:28:19.700248 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="460b2796-2ea1-4ddd-a4e7-eae181ddfc81" path="/var/lib/kubelet/pods/460b2796-2ea1-4ddd-a4e7-eae181ddfc81/volumes" Dec 03 12:28:19 crc kubenswrapper[5052]: I1203 12:28:19.770715 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/bea08f45-6aef-4ccc-8252-d840baad2bf2-config-data\") pod \"nova-cell1-novncproxy-0\" (UID: \"bea08f45-6aef-4ccc-8252-d840baad2bf2\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:28:19 crc kubenswrapper[5052]: I1203 12:28:19.770770 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"nova-novncproxy-tls-certs\" (UniqueName: \"kubernetes.io/secret/bea08f45-6aef-4ccc-8252-d840baad2bf2-nova-novncproxy-tls-certs\") pod \"nova-cell1-novncproxy-0\" (UID: \"bea08f45-6aef-4ccc-8252-d840baad2bf2\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:28:19 crc kubenswrapper[5052]: I1203 12:28:19.771083 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"vencrypt-tls-certs\" (UniqueName: \"kubernetes.io/secret/bea08f45-6aef-4ccc-8252-d840baad2bf2-vencrypt-tls-certs\") pod \"nova-cell1-novncproxy-0\" (UID: \"bea08f45-6aef-4ccc-8252-d840baad2bf2\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:28:19 crc kubenswrapper[5052]: I1203 12:28:19.771161 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-cdbdz\" (UniqueName: \"kubernetes.io/projected/bea08f45-6aef-4ccc-8252-d840baad2bf2-kube-api-access-cdbdz\") pod \"nova-cell1-novncproxy-0\" (UID: \"bea08f45-6aef-4ccc-8252-d840baad2bf2\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:28:19 crc kubenswrapper[5052]: I1203 12:28:19.771219 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/bea08f45-6aef-4ccc-8252-d840baad2bf2-combined-ca-bundle\") pod \"nova-cell1-novncproxy-0\" (UID: \"bea08f45-6aef-4ccc-8252-d840baad2bf2\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:28:19 crc kubenswrapper[5052]: I1203 12:28:19.777961 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"vencrypt-tls-certs\" (UniqueName: \"kubernetes.io/secret/bea08f45-6aef-4ccc-8252-d840baad2bf2-vencrypt-tls-certs\") pod \"nova-cell1-novncproxy-0\" (UID: \"bea08f45-6aef-4ccc-8252-d840baad2bf2\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:28:19 crc kubenswrapper[5052]: I1203 12:28:19.785242 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/bea08f45-6aef-4ccc-8252-d840baad2bf2-config-data\") pod \"nova-cell1-novncproxy-0\" (UID: \"bea08f45-6aef-4ccc-8252-d840baad2bf2\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:28:19 crc kubenswrapper[5052]: I1203 12:28:19.785397 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/bea08f45-6aef-4ccc-8252-d840baad2bf2-combined-ca-bundle\") pod \"nova-cell1-novncproxy-0\" (UID: \"bea08f45-6aef-4ccc-8252-d840baad2bf2\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:28:19 crc kubenswrapper[5052]: I1203 12:28:19.787206 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"nova-novncproxy-tls-certs\" (UniqueName: \"kubernetes.io/secret/bea08f45-6aef-4ccc-8252-d840baad2bf2-nova-novncproxy-tls-certs\") pod \"nova-cell1-novncproxy-0\" (UID: \"bea08f45-6aef-4ccc-8252-d840baad2bf2\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:28:19 crc kubenswrapper[5052]: I1203 12:28:19.796318 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-cdbdz\" (UniqueName: \"kubernetes.io/projected/bea08f45-6aef-4ccc-8252-d840baad2bf2-kube-api-access-cdbdz\") pod \"nova-cell1-novncproxy-0\" (UID: \"bea08f45-6aef-4ccc-8252-d840baad2bf2\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:28:19 crc kubenswrapper[5052]: I1203 12:28:19.823994 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:28:20 crc kubenswrapper[5052]: E1203 12:28:20.243713 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="790991ac480a9e358efc31b18c1e765ec3b8f7c9c3386cf9c51c6dc7f800750c" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:28:20 crc kubenswrapper[5052]: E1203 12:28:20.245615 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="790991ac480a9e358efc31b18c1e765ec3b8f7c9c3386cf9c51c6dc7f800750c" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:28:20 crc kubenswrapper[5052]: E1203 12:28:20.246665 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="790991ac480a9e358efc31b18c1e765ec3b8f7c9c3386cf9c51c6dc7f800750c" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:28:20 crc kubenswrapper[5052]: E1203 12:28:20.246699 5052 prober.go:104] "Probe errored" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" probeType="Readiness" pod="openstack-kuttl-tests/nova-cell0-conductor-0" podUID="c080dc7f-df72-487f-b2c1-83b143d34784" containerName="nova-cell0-conductor-conductor" Dec 03 12:28:20 crc kubenswrapper[5052]: I1203 12:28:20.349271 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 12:28:20 crc kubenswrapper[5052]: I1203 12:28:20.409490 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" event={"ID":"bea08f45-6aef-4ccc-8252-d840baad2bf2","Type":"ContainerStarted","Data":"7989f6bd58d24fdb13a69c51dbace06625effb27d2be03b8530cb269630b489e"} Dec 03 12:28:20 crc kubenswrapper[5052]: I1203 12:28:20.413428 5052 generic.go:334] "Generic (PLEG): container finished" podID="8a2c61ae-6521-424f-aeac-52e1ec1c319a" containerID="70af550381ae175843af90fae4ec5c50a17835a3a8274e54adab63fa88f5a895" exitCode=0 Dec 03 12:28:20 crc kubenswrapper[5052]: I1203 12:28:20.413503 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"8a2c61ae-6521-424f-aeac-52e1ec1c319a","Type":"ContainerDied","Data":"70af550381ae175843af90fae4ec5c50a17835a3a8274e54adab63fa88f5a895"} Dec 03 12:28:20 crc kubenswrapper[5052]: I1203 12:28:20.419156 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-cell-mapping-7v2kh" event={"ID":"51d1c5f3-0622-4c29-a19a-f1ecd293654e","Type":"ContainerStarted","Data":"c1a01685a9df4a046418f219f4791232079759a5dfaf1dcd3a5230170ee077ab"} Dec 03 12:28:20 crc kubenswrapper[5052]: I1203 12:28:20.647864 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:28:20 crc kubenswrapper[5052]: I1203 12:28:20.647994 5052 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Dec 03 12:28:20 crc kubenswrapper[5052]: I1203 12:28:20.655177 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:28:21 crc kubenswrapper[5052]: I1203 12:28:21.443849 5052 generic.go:334] "Generic (PLEG): container finished" podID="8a0b169d-6971-4530-936e-8e5f0f57816f" containerID="47d8a341afac3d19e0311930b0aa745891784bd644a5b9f1e35346d168347db9" exitCode=0 Dec 03 12:28:21 crc kubenswrapper[5052]: I1203 12:28:21.443941 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"8a0b169d-6971-4530-936e-8e5f0f57816f","Type":"ContainerDied","Data":"47d8a341afac3d19e0311930b0aa745891784bd644a5b9f1e35346d168347db9"} Dec 03 12:28:21 crc kubenswrapper[5052]: I1203 12:28:21.460843 5052 generic.go:334] "Generic (PLEG): container finished" podID="810e47c2-430b-424d-92ea-66c20ec8501f" containerID="22904b310fc3f26f046b8a7eda3ddcc73a97bb5641a37e30766bd99ffba27bde" exitCode=0 Dec 03 12:28:21 crc kubenswrapper[5052]: I1203 12:28:21.460978 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"810e47c2-430b-424d-92ea-66c20ec8501f","Type":"ContainerDied","Data":"22904b310fc3f26f046b8a7eda3ddcc73a97bb5641a37e30766bd99ffba27bde"} Dec 03 12:28:21 crc kubenswrapper[5052]: I1203 12:28:21.467668 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" event={"ID":"bea08f45-6aef-4ccc-8252-d840baad2bf2","Type":"ContainerStarted","Data":"47da0b9399486978354254c5d335b3c5c60cbdaa11b8b2dfb0593a15c8eb3a11"} Dec 03 12:28:21 crc kubenswrapper[5052]: I1203 12:28:21.495610 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-cell1-cell-mapping-7v2kh" podStartSLOduration=5.495589943 podStartE2EDuration="5.495589943s" podCreationTimestamp="2025-12-03 12:28:16 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:28:21.495425128 +0000 UTC m=+4092.083666892" watchObservedRunningTime="2025-12-03 12:28:21.495589943 +0000 UTC m=+4092.083831727" Dec 03 12:28:21 crc kubenswrapper[5052]: I1203 12:28:21.522349 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" podStartSLOduration=2.522331676 podStartE2EDuration="2.522331676s" podCreationTimestamp="2025-12-03 12:28:19 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:28:21.517056816 +0000 UTC m=+4092.105298570" watchObservedRunningTime="2025-12-03 12:28:21.522331676 +0000 UTC m=+4092.110573440" Dec 03 12:28:21 crc kubenswrapper[5052]: I1203 12:28:21.744163 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:28:21 crc kubenswrapper[5052]: I1203 12:28:21.935906 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8a0b169d-6971-4530-936e-8e5f0f57816f-combined-ca-bundle\") pod \"8a0b169d-6971-4530-936e-8e5f0f57816f\" (UID: \"8a0b169d-6971-4530-936e-8e5f0f57816f\") " Dec 03 12:28:21 crc kubenswrapper[5052]: I1203 12:28:21.936148 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/8a0b169d-6971-4530-936e-8e5f0f57816f-config-data\") pod \"8a0b169d-6971-4530-936e-8e5f0f57816f\" (UID: \"8a0b169d-6971-4530-936e-8e5f0f57816f\") " Dec 03 12:28:21 crc kubenswrapper[5052]: I1203 12:28:21.936244 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-25mdm\" (UniqueName: \"kubernetes.io/projected/8a0b169d-6971-4530-936e-8e5f0f57816f-kube-api-access-25mdm\") pod \"8a0b169d-6971-4530-936e-8e5f0f57816f\" (UID: \"8a0b169d-6971-4530-936e-8e5f0f57816f\") " Dec 03 12:28:21 crc kubenswrapper[5052]: I1203 12:28:21.971155 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8a0b169d-6971-4530-936e-8e5f0f57816f-kube-api-access-25mdm" (OuterVolumeSpecName: "kube-api-access-25mdm") pod "8a0b169d-6971-4530-936e-8e5f0f57816f" (UID: "8a0b169d-6971-4530-936e-8e5f0f57816f"). InnerVolumeSpecName "kube-api-access-25mdm". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:28:21 crc kubenswrapper[5052]: I1203 12:28:21.972923 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/cinder-api-0" podUID="61cad274-57bd-4993-aaa6-3e6013900d88" containerName="cinder-api" probeResult="failure" output="Get \"https://10.217.0.159:8776/healthcheck\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Dec 03 12:28:21 crc kubenswrapper[5052]: I1203 12:28:21.982246 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8a0b169d-6971-4530-936e-8e5f0f57816f-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "8a0b169d-6971-4530-936e-8e5f0f57816f" (UID: "8a0b169d-6971-4530-936e-8e5f0f57816f"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:28:21 crc kubenswrapper[5052]: I1203 12:28:21.987588 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8a0b169d-6971-4530-936e-8e5f0f57816f-config-data" (OuterVolumeSpecName: "config-data") pod "8a0b169d-6971-4530-936e-8e5f0f57816f" (UID: "8a0b169d-6971-4530-936e-8e5f0f57816f"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:28:22 crc kubenswrapper[5052]: I1203 12:28:22.040222 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/8a0b169d-6971-4530-936e-8e5f0f57816f-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:22 crc kubenswrapper[5052]: I1203 12:28:22.040352 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-25mdm\" (UniqueName: \"kubernetes.io/projected/8a0b169d-6971-4530-936e-8e5f0f57816f-kube-api-access-25mdm\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:22 crc kubenswrapper[5052]: I1203 12:28:22.040376 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8a0b169d-6971-4530-936e-8e5f0f57816f-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:22 crc kubenswrapper[5052]: I1203 12:28:22.071932 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:28:22 crc kubenswrapper[5052]: I1203 12:28:22.087599 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:28:22 crc kubenswrapper[5052]: I1203 12:28:22.243393 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/8a2c61ae-6521-424f-aeac-52e1ec1c319a-config-data\") pod \"8a2c61ae-6521-424f-aeac-52e1ec1c319a\" (UID: \"8a2c61ae-6521-424f-aeac-52e1ec1c319a\") " Dec 03 12:28:22 crc kubenswrapper[5052]: I1203 12:28:22.243463 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/810e47c2-430b-424d-92ea-66c20ec8501f-combined-ca-bundle\") pod \"810e47c2-430b-424d-92ea-66c20ec8501f\" (UID: \"810e47c2-430b-424d-92ea-66c20ec8501f\") " Dec 03 12:28:22 crc kubenswrapper[5052]: I1203 12:28:22.243600 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/8a2c61ae-6521-424f-aeac-52e1ec1c319a-logs\") pod \"8a2c61ae-6521-424f-aeac-52e1ec1c319a\" (UID: \"8a2c61ae-6521-424f-aeac-52e1ec1c319a\") " Dec 03 12:28:22 crc kubenswrapper[5052]: I1203 12:28:22.243643 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7qnb6\" (UniqueName: \"kubernetes.io/projected/8a2c61ae-6521-424f-aeac-52e1ec1c319a-kube-api-access-7qnb6\") pod \"8a2c61ae-6521-424f-aeac-52e1ec1c319a\" (UID: \"8a2c61ae-6521-424f-aeac-52e1ec1c319a\") " Dec 03 12:28:22 crc kubenswrapper[5052]: I1203 12:28:22.243711 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8a2c61ae-6521-424f-aeac-52e1ec1c319a-combined-ca-bundle\") pod \"8a2c61ae-6521-424f-aeac-52e1ec1c319a\" (UID: \"8a2c61ae-6521-424f-aeac-52e1ec1c319a\") " Dec 03 12:28:22 crc kubenswrapper[5052]: I1203 12:28:22.243768 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/810e47c2-430b-424d-92ea-66c20ec8501f-logs\") pod \"810e47c2-430b-424d-92ea-66c20ec8501f\" (UID: \"810e47c2-430b-424d-92ea-66c20ec8501f\") " Dec 03 12:28:22 crc kubenswrapper[5052]: I1203 12:28:22.243913 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/810e47c2-430b-424d-92ea-66c20ec8501f-config-data\") pod \"810e47c2-430b-424d-92ea-66c20ec8501f\" (UID: \"810e47c2-430b-424d-92ea-66c20ec8501f\") " Dec 03 12:28:22 crc kubenswrapper[5052]: I1203 12:28:22.243932 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4fbl2\" (UniqueName: \"kubernetes.io/projected/810e47c2-430b-424d-92ea-66c20ec8501f-kube-api-access-4fbl2\") pod \"810e47c2-430b-424d-92ea-66c20ec8501f\" (UID: \"810e47c2-430b-424d-92ea-66c20ec8501f\") " Dec 03 12:28:22 crc kubenswrapper[5052]: I1203 12:28:22.246004 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/810e47c2-430b-424d-92ea-66c20ec8501f-logs" (OuterVolumeSpecName: "logs") pod "810e47c2-430b-424d-92ea-66c20ec8501f" (UID: "810e47c2-430b-424d-92ea-66c20ec8501f"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:28:22 crc kubenswrapper[5052]: I1203 12:28:22.247350 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8a2c61ae-6521-424f-aeac-52e1ec1c319a-logs" (OuterVolumeSpecName: "logs") pod "8a2c61ae-6521-424f-aeac-52e1ec1c319a" (UID: "8a2c61ae-6521-424f-aeac-52e1ec1c319a"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:28:22 crc kubenswrapper[5052]: I1203 12:28:22.248754 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8a2c61ae-6521-424f-aeac-52e1ec1c319a-kube-api-access-7qnb6" (OuterVolumeSpecName: "kube-api-access-7qnb6") pod "8a2c61ae-6521-424f-aeac-52e1ec1c319a" (UID: "8a2c61ae-6521-424f-aeac-52e1ec1c319a"). InnerVolumeSpecName "kube-api-access-7qnb6". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:28:22 crc kubenswrapper[5052]: I1203 12:28:22.269702 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/810e47c2-430b-424d-92ea-66c20ec8501f-kube-api-access-4fbl2" (OuterVolumeSpecName: "kube-api-access-4fbl2") pod "810e47c2-430b-424d-92ea-66c20ec8501f" (UID: "810e47c2-430b-424d-92ea-66c20ec8501f"). InnerVolumeSpecName "kube-api-access-4fbl2". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:28:22 crc kubenswrapper[5052]: I1203 12:28:22.303573 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8a2c61ae-6521-424f-aeac-52e1ec1c319a-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "8a2c61ae-6521-424f-aeac-52e1ec1c319a" (UID: "8a2c61ae-6521-424f-aeac-52e1ec1c319a"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:28:22 crc kubenswrapper[5052]: I1203 12:28:22.336718 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/810e47c2-430b-424d-92ea-66c20ec8501f-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "810e47c2-430b-424d-92ea-66c20ec8501f" (UID: "810e47c2-430b-424d-92ea-66c20ec8501f"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:28:22 crc kubenswrapper[5052]: I1203 12:28:22.345805 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8a2c61ae-6521-424f-aeac-52e1ec1c319a-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:22 crc kubenswrapper[5052]: I1203 12:28:22.345843 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/810e47c2-430b-424d-92ea-66c20ec8501f-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:22 crc kubenswrapper[5052]: I1203 12:28:22.345856 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-4fbl2\" (UniqueName: \"kubernetes.io/projected/810e47c2-430b-424d-92ea-66c20ec8501f-kube-api-access-4fbl2\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:22 crc kubenswrapper[5052]: I1203 12:28:22.345867 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/810e47c2-430b-424d-92ea-66c20ec8501f-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:22 crc kubenswrapper[5052]: I1203 12:28:22.345876 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/8a2c61ae-6521-424f-aeac-52e1ec1c319a-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:22 crc kubenswrapper[5052]: I1203 12:28:22.345884 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-7qnb6\" (UniqueName: \"kubernetes.io/projected/8a2c61ae-6521-424f-aeac-52e1ec1c319a-kube-api-access-7qnb6\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:22 crc kubenswrapper[5052]: I1203 12:28:22.380963 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8a2c61ae-6521-424f-aeac-52e1ec1c319a-config-data" (OuterVolumeSpecName: "config-data") pod "8a2c61ae-6521-424f-aeac-52e1ec1c319a" (UID: "8a2c61ae-6521-424f-aeac-52e1ec1c319a"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:28:22 crc kubenswrapper[5052]: I1203 12:28:22.393422 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/810e47c2-430b-424d-92ea-66c20ec8501f-config-data" (OuterVolumeSpecName: "config-data") pod "810e47c2-430b-424d-92ea-66c20ec8501f" (UID: "810e47c2-430b-424d-92ea-66c20ec8501f"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:28:22 crc kubenswrapper[5052]: I1203 12:28:22.453740 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/810e47c2-430b-424d-92ea-66c20ec8501f-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:22 crc kubenswrapper[5052]: I1203 12:28:22.453777 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/8a2c61ae-6521-424f-aeac-52e1ec1c319a-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:22 crc kubenswrapper[5052]: E1203 12:28:22.508475 5052 cadvisor_stats_provider.go:516] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8a2c61ae_6521_424f_aeac_52e1ec1c319a.slice/crio-conmon-70af550381ae175843af90fae4ec5c50a17835a3a8274e54adab63fa88f5a895.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8a0b169d_6971_4530_936e_8e5f0f57816f.slice/crio-47d8a341afac3d19e0311930b0aa745891784bd644a5b9f1e35346d168347db9.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfcee2113_2b2f_47f9_b118_4f7488ad604e.slice/crio-conmon-df1192fd6167f45271208db821d1f0909fc60a2b10aada89f5e4d0015c104317.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8a2c61ae_6521_424f_aeac_52e1ec1c319a.slice/crio-70af550381ae175843af90fae4ec5c50a17835a3a8274e54adab63fa88f5a895.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod460b2796_2ea1_4ddd_a4e7_eae181ddfc81.slice/crio-32b328087df01322dd99751590da4943c3be251fca301d63ddc59ce90100dda3\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod810e47c2_430b_424d_92ea_66c20ec8501f.slice/crio-conmon-22904b310fc3f26f046b8a7eda3ddcc73a97bb5641a37e30766bd99ffba27bde.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6436bca9_c7a1_4540_bee3_b6dbabb53c89.slice/crio-conmon-4f7e4bab55b18e1135d280cc95a601724a2d3fc12cda71f14ad7f590adb9df61.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod460b2796_2ea1_4ddd_a4e7_eae181ddfc81.slice/crio-conmon-c99aecbfc0f318ebcfe8e5c07b7758a65a70339a148d9b58bf3923d596c20d7b.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8a0b169d_6971_4530_936e_8e5f0f57816f.slice/crio-conmon-47d8a341afac3d19e0311930b0aa745891784bd644a5b9f1e35346d168347db9.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod460b2796_2ea1_4ddd_a4e7_eae181ddfc81.slice\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod810e47c2_430b_424d_92ea_66c20ec8501f.slice/crio-22904b310fc3f26f046b8a7eda3ddcc73a97bb5641a37e30766bd99ffba27bde.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc080dc7f_df72_487f_b2c1_83b143d34784.slice/crio-conmon-790991ac480a9e358efc31b18c1e765ec3b8f7c9c3386cf9c51c6dc7f800750c.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfcee2113_2b2f_47f9_b118_4f7488ad604e.slice/crio-df1192fd6167f45271208db821d1f0909fc60a2b10aada89f5e4d0015c104317.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6436bca9_c7a1_4540_bee3_b6dbabb53c89.slice/crio-4f7e4bab55b18e1135d280cc95a601724a2d3fc12cda71f14ad7f590adb9df61.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod460b2796_2ea1_4ddd_a4e7_eae181ddfc81.slice/crio-c99aecbfc0f318ebcfe8e5c07b7758a65a70339a148d9b58bf3923d596c20d7b.scope\": RecentStats: unable to find data in memory cache]" Dec 03 12:28:22 crc kubenswrapper[5052]: I1203 12:28:22.514388 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"8a2c61ae-6521-424f-aeac-52e1ec1c319a","Type":"ContainerDied","Data":"012ec83742c8d05cd9b1b280425de3addfd86df96f18dcaafb3102f1adcfbf48"} Dec 03 12:28:22 crc kubenswrapper[5052]: I1203 12:28:22.514445 5052 scope.go:117] "RemoveContainer" containerID="70af550381ae175843af90fae4ec5c50a17835a3a8274e54adab63fa88f5a895" Dec 03 12:28:22 crc kubenswrapper[5052]: I1203 12:28:22.514647 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:28:22 crc kubenswrapper[5052]: I1203 12:28:22.524549 5052 generic.go:334] "Generic (PLEG): container finished" podID="fcee2113-2b2f-47f9-b118-4f7488ad604e" containerID="df1192fd6167f45271208db821d1f0909fc60a2b10aada89f5e4d0015c104317" exitCode=0 Dec 03 12:28:22 crc kubenswrapper[5052]: I1203 12:28:22.524626 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-conductor-0" event={"ID":"fcee2113-2b2f-47f9-b118-4f7488ad604e","Type":"ContainerDied","Data":"df1192fd6167f45271208db821d1f0909fc60a2b10aada89f5e4d0015c104317"} Dec 03 12:28:22 crc kubenswrapper[5052]: I1203 12:28:22.524656 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-conductor-0" event={"ID":"fcee2113-2b2f-47f9-b118-4f7488ad604e","Type":"ContainerDied","Data":"21f1f1e7abed6f68ee10cf152087e6c1c6027b4d29dd63cadc265e93aaf59778"} Dec 03 12:28:22 crc kubenswrapper[5052]: I1203 12:28:22.524671 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="21f1f1e7abed6f68ee10cf152087e6c1c6027b4d29dd63cadc265e93aaf59778" Dec 03 12:28:22 crc kubenswrapper[5052]: I1203 12:28:22.527735 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:28:22 crc kubenswrapper[5052]: I1203 12:28:22.529857 5052 generic.go:334] "Generic (PLEG): container finished" podID="6436bca9-c7a1-4540-bee3-b6dbabb53c89" containerID="4f7e4bab55b18e1135d280cc95a601724a2d3fc12cda71f14ad7f590adb9df61" exitCode=0 Dec 03 12:28:22 crc kubenswrapper[5052]: I1203 12:28:22.529930 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"6436bca9-c7a1-4540-bee3-b6dbabb53c89","Type":"ContainerDied","Data":"4f7e4bab55b18e1135d280cc95a601724a2d3fc12cda71f14ad7f590adb9df61"} Dec 03 12:28:22 crc kubenswrapper[5052]: I1203 12:28:22.534758 5052 generic.go:334] "Generic (PLEG): container finished" podID="c080dc7f-df72-487f-b2c1-83b143d34784" containerID="790991ac480a9e358efc31b18c1e765ec3b8f7c9c3386cf9c51c6dc7f800750c" exitCode=0 Dec 03 12:28:22 crc kubenswrapper[5052]: I1203 12:28:22.534822 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-conductor-0" event={"ID":"c080dc7f-df72-487f-b2c1-83b143d34784","Type":"ContainerDied","Data":"790991ac480a9e358efc31b18c1e765ec3b8f7c9c3386cf9c51c6dc7f800750c"} Dec 03 12:28:22 crc kubenswrapper[5052]: I1203 12:28:22.545484 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:28:22 crc kubenswrapper[5052]: I1203 12:28:22.554043 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:28:22 crc kubenswrapper[5052]: I1203 12:28:22.563005 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:28:22 crc kubenswrapper[5052]: I1203 12:28:22.563573 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"8a0b169d-6971-4530-936e-8e5f0f57816f","Type":"ContainerDied","Data":"f6f54f7b48fc592006074b7eddf9b7017f72f6b712a3311d0bd99462642c84d1"} Dec 03 12:28:22 crc kubenswrapper[5052]: I1203 12:28:22.582633 5052 scope.go:117] "RemoveContainer" containerID="90ccfa6fea4d46147f2030f8a0f1ffee182ce45ef8183a711e81ed3a0eaa5b15" Dec 03 12:28:22 crc kubenswrapper[5052]: I1203 12:28:22.591723 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:28:22 crc kubenswrapper[5052]: E1203 12:28:22.592320 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="8a0b169d-6971-4530-936e-8e5f0f57816f" containerName="nova-scheduler-scheduler" Dec 03 12:28:22 crc kubenswrapper[5052]: I1203 12:28:22.592346 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="8a0b169d-6971-4530-936e-8e5f0f57816f" containerName="nova-scheduler-scheduler" Dec 03 12:28:22 crc kubenswrapper[5052]: E1203 12:28:22.592437 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="810e47c2-430b-424d-92ea-66c20ec8501f" containerName="nova-api-api" Dec 03 12:28:22 crc kubenswrapper[5052]: I1203 12:28:22.592450 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="810e47c2-430b-424d-92ea-66c20ec8501f" containerName="nova-api-api" Dec 03 12:28:22 crc kubenswrapper[5052]: E1203 12:28:22.592470 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="8a2c61ae-6521-424f-aeac-52e1ec1c319a" containerName="nova-metadata-metadata" Dec 03 12:28:22 crc kubenswrapper[5052]: I1203 12:28:22.592477 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="8a2c61ae-6521-424f-aeac-52e1ec1c319a" containerName="nova-metadata-metadata" Dec 03 12:28:22 crc kubenswrapper[5052]: E1203 12:28:22.592533 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="8a2c61ae-6521-424f-aeac-52e1ec1c319a" containerName="nova-metadata-log" Dec 03 12:28:22 crc kubenswrapper[5052]: I1203 12:28:22.592544 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="8a2c61ae-6521-424f-aeac-52e1ec1c319a" containerName="nova-metadata-log" Dec 03 12:28:22 crc kubenswrapper[5052]: E1203 12:28:22.592574 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="810e47c2-430b-424d-92ea-66c20ec8501f" containerName="nova-api-log" Dec 03 12:28:22 crc kubenswrapper[5052]: I1203 12:28:22.592582 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="810e47c2-430b-424d-92ea-66c20ec8501f" containerName="nova-api-log" Dec 03 12:28:22 crc kubenswrapper[5052]: E1203 12:28:22.592596 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="fcee2113-2b2f-47f9-b118-4f7488ad604e" containerName="nova-cell1-conductor-conductor" Dec 03 12:28:22 crc kubenswrapper[5052]: I1203 12:28:22.592603 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="fcee2113-2b2f-47f9-b118-4f7488ad604e" containerName="nova-cell1-conductor-conductor" Dec 03 12:28:22 crc kubenswrapper[5052]: I1203 12:28:22.592879 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="810e47c2-430b-424d-92ea-66c20ec8501f" containerName="nova-api-log" Dec 03 12:28:22 crc kubenswrapper[5052]: I1203 12:28:22.592902 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="8a2c61ae-6521-424f-aeac-52e1ec1c319a" containerName="nova-metadata-log" Dec 03 12:28:22 crc kubenswrapper[5052]: I1203 12:28:22.592925 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="810e47c2-430b-424d-92ea-66c20ec8501f" containerName="nova-api-api" Dec 03 12:28:22 crc kubenswrapper[5052]: I1203 12:28:22.592941 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="8a0b169d-6971-4530-936e-8e5f0f57816f" containerName="nova-scheduler-scheduler" Dec 03 12:28:22 crc kubenswrapper[5052]: I1203 12:28:22.593056 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="8a2c61ae-6521-424f-aeac-52e1ec1c319a" containerName="nova-metadata-metadata" Dec 03 12:28:22 crc kubenswrapper[5052]: I1203 12:28:22.593075 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="fcee2113-2b2f-47f9-b118-4f7488ad604e" containerName="nova-cell1-conductor-conductor" Dec 03 12:28:22 crc kubenswrapper[5052]: I1203 12:28:22.594605 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:28:22 crc kubenswrapper[5052]: I1203 12:28:22.595158 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"810e47c2-430b-424d-92ea-66c20ec8501f","Type":"ContainerDied","Data":"62324b7258f7853ee5e387fa3e5e7fff9881fa6e1b186d82369a9b7c2b057c48"} Dec 03 12:28:22 crc kubenswrapper[5052]: I1203 12:28:22.595248 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:28:22 crc kubenswrapper[5052]: I1203 12:28:22.597352 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-metadata-config-data" Dec 03 12:28:22 crc kubenswrapper[5052]: I1203 12:28:22.602686 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-nova-metadata-internal-svc" Dec 03 12:28:22 crc kubenswrapper[5052]: I1203 12:28:22.614908 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:28:22 crc kubenswrapper[5052]: I1203 12:28:22.627632 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:28:22 crc kubenswrapper[5052]: I1203 12:28:22.632973 5052 scope.go:117] "RemoveContainer" containerID="47d8a341afac3d19e0311930b0aa745891784bd644a5b9f1e35346d168347db9" Dec 03 12:28:22 crc kubenswrapper[5052]: I1203 12:28:22.637790 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:28:22 crc kubenswrapper[5052]: I1203 12:28:22.656004 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xttsw\" (UniqueName: \"kubernetes.io/projected/fcee2113-2b2f-47f9-b118-4f7488ad604e-kube-api-access-xttsw\") pod \"fcee2113-2b2f-47f9-b118-4f7488ad604e\" (UID: \"fcee2113-2b2f-47f9-b118-4f7488ad604e\") " Dec 03 12:28:22 crc kubenswrapper[5052]: I1203 12:28:22.656059 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/fcee2113-2b2f-47f9-b118-4f7488ad604e-combined-ca-bundle\") pod \"fcee2113-2b2f-47f9-b118-4f7488ad604e\" (UID: \"fcee2113-2b2f-47f9-b118-4f7488ad604e\") " Dec 03 12:28:22 crc kubenswrapper[5052]: I1203 12:28:22.656277 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/fcee2113-2b2f-47f9-b118-4f7488ad604e-config-data\") pod \"fcee2113-2b2f-47f9-b118-4f7488ad604e\" (UID: \"fcee2113-2b2f-47f9-b118-4f7488ad604e\") " Dec 03 12:28:22 crc kubenswrapper[5052]: I1203 12:28:22.677688 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fcee2113-2b2f-47f9-b118-4f7488ad604e-kube-api-access-xttsw" (OuterVolumeSpecName: "kube-api-access-xttsw") pod "fcee2113-2b2f-47f9-b118-4f7488ad604e" (UID: "fcee2113-2b2f-47f9-b118-4f7488ad604e"). InnerVolumeSpecName "kube-api-access-xttsw". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:28:22 crc kubenswrapper[5052]: I1203 12:28:22.679367 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:28:22 crc kubenswrapper[5052]: I1203 12:28:22.681169 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:28:22 crc kubenswrapper[5052]: I1203 12:28:22.683741 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-scheduler-config-data" Dec 03 12:28:22 crc kubenswrapper[5052]: I1203 12:28:22.725396 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:28:22 crc kubenswrapper[5052]: I1203 12:28:22.728402 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fcee2113-2b2f-47f9-b118-4f7488ad604e-config-data" (OuterVolumeSpecName: "config-data") pod "fcee2113-2b2f-47f9-b118-4f7488ad604e" (UID: "fcee2113-2b2f-47f9-b118-4f7488ad604e"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:28:22 crc kubenswrapper[5052]: I1203 12:28:22.759970 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-67g6g\" (UniqueName: \"kubernetes.io/projected/2fe3990a-96b5-4186-a646-ffc72cee7c57-kube-api-access-67g6g\") pod \"nova-scheduler-0\" (UID: \"2fe3990a-96b5-4186-a646-ffc72cee7c57\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:28:22 crc kubenswrapper[5052]: I1203 12:28:22.760046 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2fe3990a-96b5-4186-a646-ffc72cee7c57-combined-ca-bundle\") pod \"nova-scheduler-0\" (UID: \"2fe3990a-96b5-4186-a646-ffc72cee7c57\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:28:22 crc kubenswrapper[5052]: I1203 12:28:22.760221 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/18dae0fe-fcde-4a2e-bfd5-3c4ee6976894-config-data\") pod \"nova-metadata-0\" (UID: \"18dae0fe-fcde-4a2e-bfd5-3c4ee6976894\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:28:22 crc kubenswrapper[5052]: I1203 12:28:22.760308 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bn57m\" (UniqueName: \"kubernetes.io/projected/18dae0fe-fcde-4a2e-bfd5-3c4ee6976894-kube-api-access-bn57m\") pod \"nova-metadata-0\" (UID: \"18dae0fe-fcde-4a2e-bfd5-3c4ee6976894\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:28:22 crc kubenswrapper[5052]: I1203 12:28:22.760359 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/18dae0fe-fcde-4a2e-bfd5-3c4ee6976894-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"18dae0fe-fcde-4a2e-bfd5-3c4ee6976894\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:28:22 crc kubenswrapper[5052]: I1203 12:28:22.760383 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/2fe3990a-96b5-4186-a646-ffc72cee7c57-config-data\") pod \"nova-scheduler-0\" (UID: \"2fe3990a-96b5-4186-a646-ffc72cee7c57\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:28:22 crc kubenswrapper[5052]: I1203 12:28:22.760498 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/18dae0fe-fcde-4a2e-bfd5-3c4ee6976894-logs\") pod \"nova-metadata-0\" (UID: \"18dae0fe-fcde-4a2e-bfd5-3c4ee6976894\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:28:22 crc kubenswrapper[5052]: I1203 12:28:22.760673 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/18dae0fe-fcde-4a2e-bfd5-3c4ee6976894-nova-metadata-tls-certs\") pod \"nova-metadata-0\" (UID: \"18dae0fe-fcde-4a2e-bfd5-3c4ee6976894\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:28:22 crc kubenswrapper[5052]: I1203 12:28:22.760799 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-xttsw\" (UniqueName: \"kubernetes.io/projected/fcee2113-2b2f-47f9-b118-4f7488ad604e-kube-api-access-xttsw\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:22 crc kubenswrapper[5052]: I1203 12:28:22.760830 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/fcee2113-2b2f-47f9-b118-4f7488ad604e-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:22 crc kubenswrapper[5052]: I1203 12:28:22.769991 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fcee2113-2b2f-47f9-b118-4f7488ad604e-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "fcee2113-2b2f-47f9-b118-4f7488ad604e" (UID: "fcee2113-2b2f-47f9-b118-4f7488ad604e"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:28:22 crc kubenswrapper[5052]: I1203 12:28:22.862614 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-bn57m\" (UniqueName: \"kubernetes.io/projected/18dae0fe-fcde-4a2e-bfd5-3c4ee6976894-kube-api-access-bn57m\") pod \"nova-metadata-0\" (UID: \"18dae0fe-fcde-4a2e-bfd5-3c4ee6976894\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:28:22 crc kubenswrapper[5052]: I1203 12:28:22.862988 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/18dae0fe-fcde-4a2e-bfd5-3c4ee6976894-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"18dae0fe-fcde-4a2e-bfd5-3c4ee6976894\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:28:22 crc kubenswrapper[5052]: I1203 12:28:22.863011 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/2fe3990a-96b5-4186-a646-ffc72cee7c57-config-data\") pod \"nova-scheduler-0\" (UID: \"2fe3990a-96b5-4186-a646-ffc72cee7c57\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:28:22 crc kubenswrapper[5052]: I1203 12:28:22.863057 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/18dae0fe-fcde-4a2e-bfd5-3c4ee6976894-logs\") pod \"nova-metadata-0\" (UID: \"18dae0fe-fcde-4a2e-bfd5-3c4ee6976894\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:28:22 crc kubenswrapper[5052]: I1203 12:28:22.863114 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/18dae0fe-fcde-4a2e-bfd5-3c4ee6976894-nova-metadata-tls-certs\") pod \"nova-metadata-0\" (UID: \"18dae0fe-fcde-4a2e-bfd5-3c4ee6976894\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:28:22 crc kubenswrapper[5052]: I1203 12:28:22.863183 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-67g6g\" (UniqueName: \"kubernetes.io/projected/2fe3990a-96b5-4186-a646-ffc72cee7c57-kube-api-access-67g6g\") pod \"nova-scheduler-0\" (UID: \"2fe3990a-96b5-4186-a646-ffc72cee7c57\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:28:22 crc kubenswrapper[5052]: I1203 12:28:22.863212 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2fe3990a-96b5-4186-a646-ffc72cee7c57-combined-ca-bundle\") pod \"nova-scheduler-0\" (UID: \"2fe3990a-96b5-4186-a646-ffc72cee7c57\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:28:22 crc kubenswrapper[5052]: I1203 12:28:22.863417 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/18dae0fe-fcde-4a2e-bfd5-3c4ee6976894-config-data\") pod \"nova-metadata-0\" (UID: \"18dae0fe-fcde-4a2e-bfd5-3c4ee6976894\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:28:22 crc kubenswrapper[5052]: I1203 12:28:22.863676 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/fcee2113-2b2f-47f9-b118-4f7488ad604e-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:22 crc kubenswrapper[5052]: I1203 12:28:22.865832 5052 scope.go:117] "RemoveContainer" containerID="22904b310fc3f26f046b8a7eda3ddcc73a97bb5641a37e30766bd99ffba27bde" Dec 03 12:28:22 crc kubenswrapper[5052]: I1203 12:28:22.872615 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/18dae0fe-fcde-4a2e-bfd5-3c4ee6976894-nova-metadata-tls-certs\") pod \"nova-metadata-0\" (UID: \"18dae0fe-fcde-4a2e-bfd5-3c4ee6976894\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:28:22 crc kubenswrapper[5052]: I1203 12:28:22.873643 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2fe3990a-96b5-4186-a646-ffc72cee7c57-combined-ca-bundle\") pod \"nova-scheduler-0\" (UID: \"2fe3990a-96b5-4186-a646-ffc72cee7c57\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:28:22 crc kubenswrapper[5052]: I1203 12:28:22.875180 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/2fe3990a-96b5-4186-a646-ffc72cee7c57-config-data\") pod \"nova-scheduler-0\" (UID: \"2fe3990a-96b5-4186-a646-ffc72cee7c57\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:28:22 crc kubenswrapper[5052]: I1203 12:28:22.875815 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/18dae0fe-fcde-4a2e-bfd5-3c4ee6976894-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"18dae0fe-fcde-4a2e-bfd5-3c4ee6976894\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:28:22 crc kubenswrapper[5052]: I1203 12:28:22.888810 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:28:22 crc kubenswrapper[5052]: I1203 12:28:22.890574 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/18dae0fe-fcde-4a2e-bfd5-3c4ee6976894-config-data\") pod \"nova-metadata-0\" (UID: \"18dae0fe-fcde-4a2e-bfd5-3c4ee6976894\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:28:22 crc kubenswrapper[5052]: I1203 12:28:22.890913 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/18dae0fe-fcde-4a2e-bfd5-3c4ee6976894-logs\") pod \"nova-metadata-0\" (UID: \"18dae0fe-fcde-4a2e-bfd5-3c4ee6976894\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:28:22 crc kubenswrapper[5052]: I1203 12:28:22.900133 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-67g6g\" (UniqueName: \"kubernetes.io/projected/2fe3990a-96b5-4186-a646-ffc72cee7c57-kube-api-access-67g6g\") pod \"nova-scheduler-0\" (UID: \"2fe3990a-96b5-4186-a646-ffc72cee7c57\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:28:22 crc kubenswrapper[5052]: I1203 12:28:22.904264 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-bn57m\" (UniqueName: \"kubernetes.io/projected/18dae0fe-fcde-4a2e-bfd5-3c4ee6976894-kube-api-access-bn57m\") pod \"nova-metadata-0\" (UID: \"18dae0fe-fcde-4a2e-bfd5-3c4ee6976894\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:28:22 crc kubenswrapper[5052]: I1203 12:28:22.904334 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:28:22 crc kubenswrapper[5052]: I1203 12:28:22.913321 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:28:22 crc kubenswrapper[5052]: I1203 12:28:22.916646 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:28:22 crc kubenswrapper[5052]: I1203 12:28:22.921737 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-api-config-data" Dec 03 12:28:22 crc kubenswrapper[5052]: I1203 12:28:22.923998 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-nova-public-svc" Dec 03 12:28:22 crc kubenswrapper[5052]: I1203 12:28:22.925474 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-nova-internal-svc" Dec 03 12:28:22 crc kubenswrapper[5052]: I1203 12:28:22.933690 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:28:22 crc kubenswrapper[5052]: I1203 12:28:22.935531 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:28:22 crc kubenswrapper[5052]: I1203 12:28:22.954875 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:28:22 crc kubenswrapper[5052]: I1203 12:28:22.966257 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/d808ef0b-751e-4576-befc-e23a06fe2ff0-config-data\") pod \"nova-api-0\" (UID: \"d808ef0b-751e-4576-befc-e23a06fe2ff0\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:28:22 crc kubenswrapper[5052]: I1203 12:28:22.966316 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6hhvl\" (UniqueName: \"kubernetes.io/projected/d808ef0b-751e-4576-befc-e23a06fe2ff0-kube-api-access-6hhvl\") pod \"nova-api-0\" (UID: \"d808ef0b-751e-4576-befc-e23a06fe2ff0\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:28:22 crc kubenswrapper[5052]: I1203 12:28:22.966341 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/d808ef0b-751e-4576-befc-e23a06fe2ff0-public-tls-certs\") pod \"nova-api-0\" (UID: \"d808ef0b-751e-4576-befc-e23a06fe2ff0\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:28:22 crc kubenswrapper[5052]: I1203 12:28:22.966964 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/d808ef0b-751e-4576-befc-e23a06fe2ff0-internal-tls-certs\") pod \"nova-api-0\" (UID: \"d808ef0b-751e-4576-befc-e23a06fe2ff0\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:28:22 crc kubenswrapper[5052]: I1203 12:28:22.967231 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d808ef0b-751e-4576-befc-e23a06fe2ff0-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"d808ef0b-751e-4576-befc-e23a06fe2ff0\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:28:22 crc kubenswrapper[5052]: I1203 12:28:22.967265 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/d808ef0b-751e-4576-befc-e23a06fe2ff0-logs\") pod \"nova-api-0\" (UID: \"d808ef0b-751e-4576-befc-e23a06fe2ff0\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:28:23 crc kubenswrapper[5052]: I1203 12:28:23.011133 5052 scope.go:117] "RemoveContainer" containerID="85ec55fcdf485c866d81bbe48593514b623381629978696113751cab29e59861" Dec 03 12:28:23 crc kubenswrapper[5052]: I1203 12:28:23.068585 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6436bca9-c7a1-4540-bee3-b6dbabb53c89-config-data\") pod \"6436bca9-c7a1-4540-bee3-b6dbabb53c89\" (UID: \"6436bca9-c7a1-4540-bee3-b6dbabb53c89\") " Dec 03 12:28:23 crc kubenswrapper[5052]: I1203 12:28:23.068664 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6436bca9-c7a1-4540-bee3-b6dbabb53c89-combined-ca-bundle\") pod \"6436bca9-c7a1-4540-bee3-b6dbabb53c89\" (UID: \"6436bca9-c7a1-4540-bee3-b6dbabb53c89\") " Dec 03 12:28:23 crc kubenswrapper[5052]: I1203 12:28:23.068748 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/6436bca9-c7a1-4540-bee3-b6dbabb53c89-scripts\") pod \"6436bca9-c7a1-4540-bee3-b6dbabb53c89\" (UID: \"6436bca9-c7a1-4540-bee3-b6dbabb53c89\") " Dec 03 12:28:23 crc kubenswrapper[5052]: I1203 12:28:23.068869 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ncjrh\" (UniqueName: \"kubernetes.io/projected/6436bca9-c7a1-4540-bee3-b6dbabb53c89-kube-api-access-ncjrh\") pod \"6436bca9-c7a1-4540-bee3-b6dbabb53c89\" (UID: \"6436bca9-c7a1-4540-bee3-b6dbabb53c89\") " Dec 03 12:28:23 crc kubenswrapper[5052]: I1203 12:28:23.068932 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/6436bca9-c7a1-4540-bee3-b6dbabb53c89-etc-machine-id\") pod \"6436bca9-c7a1-4540-bee3-b6dbabb53c89\" (UID: \"6436bca9-c7a1-4540-bee3-b6dbabb53c89\") " Dec 03 12:28:23 crc kubenswrapper[5052]: I1203 12:28:23.069117 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/6436bca9-c7a1-4540-bee3-b6dbabb53c89-config-data-custom\") pod \"6436bca9-c7a1-4540-bee3-b6dbabb53c89\" (UID: \"6436bca9-c7a1-4540-bee3-b6dbabb53c89\") " Dec 03 12:28:23 crc kubenswrapper[5052]: I1203 12:28:23.069335 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/d808ef0b-751e-4576-befc-e23a06fe2ff0-config-data\") pod \"nova-api-0\" (UID: \"d808ef0b-751e-4576-befc-e23a06fe2ff0\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:28:23 crc kubenswrapper[5052]: I1203 12:28:23.069370 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-6hhvl\" (UniqueName: \"kubernetes.io/projected/d808ef0b-751e-4576-befc-e23a06fe2ff0-kube-api-access-6hhvl\") pod \"nova-api-0\" (UID: \"d808ef0b-751e-4576-befc-e23a06fe2ff0\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:28:23 crc kubenswrapper[5052]: I1203 12:28:23.069386 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/d808ef0b-751e-4576-befc-e23a06fe2ff0-public-tls-certs\") pod \"nova-api-0\" (UID: \"d808ef0b-751e-4576-befc-e23a06fe2ff0\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:28:23 crc kubenswrapper[5052]: I1203 12:28:23.069444 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/d808ef0b-751e-4576-befc-e23a06fe2ff0-internal-tls-certs\") pod \"nova-api-0\" (UID: \"d808ef0b-751e-4576-befc-e23a06fe2ff0\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:28:23 crc kubenswrapper[5052]: I1203 12:28:23.069534 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d808ef0b-751e-4576-befc-e23a06fe2ff0-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"d808ef0b-751e-4576-befc-e23a06fe2ff0\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:28:23 crc kubenswrapper[5052]: I1203 12:28:23.069553 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/d808ef0b-751e-4576-befc-e23a06fe2ff0-logs\") pod \"nova-api-0\" (UID: \"d808ef0b-751e-4576-befc-e23a06fe2ff0\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:28:23 crc kubenswrapper[5052]: I1203 12:28:23.070390 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/d808ef0b-751e-4576-befc-e23a06fe2ff0-logs\") pod \"nova-api-0\" (UID: \"d808ef0b-751e-4576-befc-e23a06fe2ff0\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:28:23 crc kubenswrapper[5052]: I1203 12:28:23.070980 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/6436bca9-c7a1-4540-bee3-b6dbabb53c89-etc-machine-id" (OuterVolumeSpecName: "etc-machine-id") pod "6436bca9-c7a1-4540-bee3-b6dbabb53c89" (UID: "6436bca9-c7a1-4540-bee3-b6dbabb53c89"). InnerVolumeSpecName "etc-machine-id". PluginName "kubernetes.io/host-path", VolumeGidValue "" Dec 03 12:28:23 crc kubenswrapper[5052]: I1203 12:28:23.074428 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/d808ef0b-751e-4576-befc-e23a06fe2ff0-internal-tls-certs\") pod \"nova-api-0\" (UID: \"d808ef0b-751e-4576-befc-e23a06fe2ff0\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:28:23 crc kubenswrapper[5052]: I1203 12:28:23.079542 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/d808ef0b-751e-4576-befc-e23a06fe2ff0-config-data\") pod \"nova-api-0\" (UID: \"d808ef0b-751e-4576-befc-e23a06fe2ff0\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:28:23 crc kubenswrapper[5052]: I1203 12:28:23.084352 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/d808ef0b-751e-4576-befc-e23a06fe2ff0-public-tls-certs\") pod \"nova-api-0\" (UID: \"d808ef0b-751e-4576-befc-e23a06fe2ff0\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:28:23 crc kubenswrapper[5052]: I1203 12:28:23.086247 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d808ef0b-751e-4576-befc-e23a06fe2ff0-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"d808ef0b-751e-4576-befc-e23a06fe2ff0\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:28:23 crc kubenswrapper[5052]: I1203 12:28:23.101207 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-6hhvl\" (UniqueName: \"kubernetes.io/projected/d808ef0b-751e-4576-befc-e23a06fe2ff0-kube-api-access-6hhvl\") pod \"nova-api-0\" (UID: \"d808ef0b-751e-4576-befc-e23a06fe2ff0\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:28:23 crc kubenswrapper[5052]: I1203 12:28:23.142180 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:28:23 crc kubenswrapper[5052]: I1203 12:28:23.171241 5052 reconciler_common.go:293] "Volume detached for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/6436bca9-c7a1-4540-bee3-b6dbabb53c89-etc-machine-id\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:23 crc kubenswrapper[5052]: I1203 12:28:23.261854 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:28:23 crc kubenswrapper[5052]: I1203 12:28:23.605614 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"6436bca9-c7a1-4540-bee3-b6dbabb53c89","Type":"ContainerDied","Data":"3f9c5d49b50567e223e4f8d3fb3e1de27e0f73957b9617b5db32845c00be043e"} Dec 03 12:28:23 crc kubenswrapper[5052]: I1203 12:28:23.605944 5052 scope.go:117] "RemoveContainer" containerID="584f69d542c77d9dd7acd735af74068b5eca3f64ae82019e8afdda51237d1117" Dec 03 12:28:23 crc kubenswrapper[5052]: I1203 12:28:23.605838 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:28:23 crc kubenswrapper[5052]: I1203 12:28:23.611626 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:28:23 crc kubenswrapper[5052]: I1203 12:28:23.686175 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="810e47c2-430b-424d-92ea-66c20ec8501f" path="/var/lib/kubelet/pods/810e47c2-430b-424d-92ea-66c20ec8501f/volumes" Dec 03 12:28:23 crc kubenswrapper[5052]: I1203 12:28:23.687088 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8a0b169d-6971-4530-936e-8e5f0f57816f" path="/var/lib/kubelet/pods/8a0b169d-6971-4530-936e-8e5f0f57816f/volumes" Dec 03 12:28:23 crc kubenswrapper[5052]: I1203 12:28:23.687635 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8a2c61ae-6521-424f-aeac-52e1ec1c319a" path="/var/lib/kubelet/pods/8a2c61ae-6521-424f-aeac-52e1ec1c319a/volumes" Dec 03 12:28:23 crc kubenswrapper[5052]: I1203 12:28:23.707932 5052 scope.go:117] "RemoveContainer" containerID="4f7e4bab55b18e1135d280cc95a601724a2d3fc12cda71f14ad7f590adb9df61" Dec 03 12:28:23 crc kubenswrapper[5052]: I1203 12:28:23.723889 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-0"] Dec 03 12:28:23 crc kubenswrapper[5052]: I1203 12:28:23.737712 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-0"] Dec 03 12:28:23 crc kubenswrapper[5052]: I1203 12:28:23.754370 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-0"] Dec 03 12:28:23 crc kubenswrapper[5052]: E1203 12:28:23.754909 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="6436bca9-c7a1-4540-bee3-b6dbabb53c89" containerName="probe" Dec 03 12:28:23 crc kubenswrapper[5052]: I1203 12:28:23.754927 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="6436bca9-c7a1-4540-bee3-b6dbabb53c89" containerName="probe" Dec 03 12:28:23 crc kubenswrapper[5052]: E1203 12:28:23.754947 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="6436bca9-c7a1-4540-bee3-b6dbabb53c89" containerName="cinder-scheduler" Dec 03 12:28:23 crc kubenswrapper[5052]: I1203 12:28:23.754954 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="6436bca9-c7a1-4540-bee3-b6dbabb53c89" containerName="cinder-scheduler" Dec 03 12:28:23 crc kubenswrapper[5052]: I1203 12:28:23.755166 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="6436bca9-c7a1-4540-bee3-b6dbabb53c89" containerName="cinder-scheduler" Dec 03 12:28:23 crc kubenswrapper[5052]: I1203 12:28:23.755199 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="6436bca9-c7a1-4540-bee3-b6dbabb53c89" containerName="probe" Dec 03 12:28:23 crc kubenswrapper[5052]: I1203 12:28:23.755892 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:28:23 crc kubenswrapper[5052]: I1203 12:28:23.758612 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell1-conductor-config-data" Dec 03 12:28:23 crc kubenswrapper[5052]: I1203 12:28:23.767251 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-0"] Dec 03 12:28:23 crc kubenswrapper[5052]: I1203 12:28:23.789577 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/5ae7c56e-3507-403b-b58a-1a6da1ed1ab3-config-data\") pod \"nova-cell1-conductor-0\" (UID: \"5ae7c56e-3507-403b-b58a-1a6da1ed1ab3\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:28:23 crc kubenswrapper[5052]: I1203 12:28:23.789629 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wdcfr\" (UniqueName: \"kubernetes.io/projected/5ae7c56e-3507-403b-b58a-1a6da1ed1ab3-kube-api-access-wdcfr\") pod \"nova-cell1-conductor-0\" (UID: \"5ae7c56e-3507-403b-b58a-1a6da1ed1ab3\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:28:23 crc kubenswrapper[5052]: I1203 12:28:23.789657 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5ae7c56e-3507-403b-b58a-1a6da1ed1ab3-combined-ca-bundle\") pod \"nova-cell1-conductor-0\" (UID: \"5ae7c56e-3507-403b-b58a-1a6da1ed1ab3\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:28:23 crc kubenswrapper[5052]: I1203 12:28:23.891633 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/5ae7c56e-3507-403b-b58a-1a6da1ed1ab3-config-data\") pod \"nova-cell1-conductor-0\" (UID: \"5ae7c56e-3507-403b-b58a-1a6da1ed1ab3\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:28:23 crc kubenswrapper[5052]: I1203 12:28:23.891682 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-wdcfr\" (UniqueName: \"kubernetes.io/projected/5ae7c56e-3507-403b-b58a-1a6da1ed1ab3-kube-api-access-wdcfr\") pod \"nova-cell1-conductor-0\" (UID: \"5ae7c56e-3507-403b-b58a-1a6da1ed1ab3\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:28:23 crc kubenswrapper[5052]: I1203 12:28:23.891721 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5ae7c56e-3507-403b-b58a-1a6da1ed1ab3-combined-ca-bundle\") pod \"nova-cell1-conductor-0\" (UID: \"5ae7c56e-3507-403b-b58a-1a6da1ed1ab3\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:28:23 crc kubenswrapper[5052]: I1203 12:28:23.897005 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/5ae7c56e-3507-403b-b58a-1a6da1ed1ab3-config-data\") pod \"nova-cell1-conductor-0\" (UID: \"5ae7c56e-3507-403b-b58a-1a6da1ed1ab3\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:28:23 crc kubenswrapper[5052]: I1203 12:28:23.908425 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5ae7c56e-3507-403b-b58a-1a6da1ed1ab3-combined-ca-bundle\") pod \"nova-cell1-conductor-0\" (UID: \"5ae7c56e-3507-403b-b58a-1a6da1ed1ab3\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:28:23 crc kubenswrapper[5052]: I1203 12:28:23.918206 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-wdcfr\" (UniqueName: \"kubernetes.io/projected/5ae7c56e-3507-403b-b58a-1a6da1ed1ab3-kube-api-access-wdcfr\") pod \"nova-cell1-conductor-0\" (UID: \"5ae7c56e-3507-403b-b58a-1a6da1ed1ab3\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:28:24 crc kubenswrapper[5052]: I1203 12:28:24.072786 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:28:24 crc kubenswrapper[5052]: I1203 12:28:24.357653 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6436bca9-c7a1-4540-bee3-b6dbabb53c89-config-data-custom" (OuterVolumeSpecName: "config-data-custom") pod "6436bca9-c7a1-4540-bee3-b6dbabb53c89" (UID: "6436bca9-c7a1-4540-bee3-b6dbabb53c89"). InnerVolumeSpecName "config-data-custom". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:28:24 crc kubenswrapper[5052]: I1203 12:28:24.357899 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6436bca9-c7a1-4540-bee3-b6dbabb53c89-kube-api-access-ncjrh" (OuterVolumeSpecName: "kube-api-access-ncjrh") pod "6436bca9-c7a1-4540-bee3-b6dbabb53c89" (UID: "6436bca9-c7a1-4540-bee3-b6dbabb53c89"). InnerVolumeSpecName "kube-api-access-ncjrh". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:28:24 crc kubenswrapper[5052]: I1203 12:28:24.359201 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6436bca9-c7a1-4540-bee3-b6dbabb53c89-scripts" (OuterVolumeSpecName: "scripts") pod "6436bca9-c7a1-4540-bee3-b6dbabb53c89" (UID: "6436bca9-c7a1-4540-bee3-b6dbabb53c89"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:28:24 crc kubenswrapper[5052]: I1203 12:28:24.407368 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/6436bca9-c7a1-4540-bee3-b6dbabb53c89-config-data-custom\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:24 crc kubenswrapper[5052]: I1203 12:28:24.408830 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/6436bca9-c7a1-4540-bee3-b6dbabb53c89-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:24 crc kubenswrapper[5052]: I1203 12:28:24.408847 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-ncjrh\" (UniqueName: \"kubernetes.io/projected/6436bca9-c7a1-4540-bee3-b6dbabb53c89-kube-api-access-ncjrh\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:24 crc kubenswrapper[5052]: I1203 12:28:24.435785 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6436bca9-c7a1-4540-bee3-b6dbabb53c89-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "6436bca9-c7a1-4540-bee3-b6dbabb53c89" (UID: "6436bca9-c7a1-4540-bee3-b6dbabb53c89"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:28:24 crc kubenswrapper[5052]: I1203 12:28:24.510066 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6436bca9-c7a1-4540-bee3-b6dbabb53c89-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:24 crc kubenswrapper[5052]: I1203 12:28:24.525047 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6436bca9-c7a1-4540-bee3-b6dbabb53c89-config-data" (OuterVolumeSpecName: "config-data") pod "6436bca9-c7a1-4540-bee3-b6dbabb53c89" (UID: "6436bca9-c7a1-4540-bee3-b6dbabb53c89"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:28:24 crc kubenswrapper[5052]: I1203 12:28:24.611944 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6436bca9-c7a1-4540-bee3-b6dbabb53c89-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:24 crc kubenswrapper[5052]: I1203 12:28:24.727458 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:28:24 crc kubenswrapper[5052]: I1203 12:28:24.816626 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/c080dc7f-df72-487f-b2c1-83b143d34784-config-data\") pod \"c080dc7f-df72-487f-b2c1-83b143d34784\" (UID: \"c080dc7f-df72-487f-b2c1-83b143d34784\") " Dec 03 12:28:24 crc kubenswrapper[5052]: I1203 12:28:24.816713 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c8nwg\" (UniqueName: \"kubernetes.io/projected/c080dc7f-df72-487f-b2c1-83b143d34784-kube-api-access-c8nwg\") pod \"c080dc7f-df72-487f-b2c1-83b143d34784\" (UID: \"c080dc7f-df72-487f-b2c1-83b143d34784\") " Dec 03 12:28:24 crc kubenswrapper[5052]: I1203 12:28:24.816844 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c080dc7f-df72-487f-b2c1-83b143d34784-combined-ca-bundle\") pod \"c080dc7f-df72-487f-b2c1-83b143d34784\" (UID: \"c080dc7f-df72-487f-b2c1-83b143d34784\") " Dec 03 12:28:24 crc kubenswrapper[5052]: I1203 12:28:24.824105 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:28:24 crc kubenswrapper[5052]: I1203 12:28:24.826018 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c080dc7f-df72-487f-b2c1-83b143d34784-kube-api-access-c8nwg" (OuterVolumeSpecName: "kube-api-access-c8nwg") pod "c080dc7f-df72-487f-b2c1-83b143d34784" (UID: "c080dc7f-df72-487f-b2c1-83b143d34784"). InnerVolumeSpecName "kube-api-access-c8nwg". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:28:24 crc kubenswrapper[5052]: I1203 12:28:24.852831 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c080dc7f-df72-487f-b2c1-83b143d34784-config-data" (OuterVolumeSpecName: "config-data") pod "c080dc7f-df72-487f-b2c1-83b143d34784" (UID: "c080dc7f-df72-487f-b2c1-83b143d34784"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:28:24 crc kubenswrapper[5052]: I1203 12:28:24.874628 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c080dc7f-df72-487f-b2c1-83b143d34784-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "c080dc7f-df72-487f-b2c1-83b143d34784" (UID: "c080dc7f-df72-487f-b2c1-83b143d34784"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:28:24 crc kubenswrapper[5052]: W1203 12:28:24.902663 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2fe3990a_96b5_4186_a646_ffc72cee7c57.slice/crio-20bfc436d9c14edbb7f38eb528b458af2431287e7c6f5ae3b8edc2da21eef0f7 WatchSource:0}: Error finding container 20bfc436d9c14edbb7f38eb528b458af2431287e7c6f5ae3b8edc2da21eef0f7: Status 404 returned error can't find the container with id 20bfc436d9c14edbb7f38eb528b458af2431287e7c6f5ae3b8edc2da21eef0f7 Dec 03 12:28:24 crc kubenswrapper[5052]: I1203 12:28:24.914824 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:28:24 crc kubenswrapper[5052]: I1203 12:28:24.919713 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c080dc7f-df72-487f-b2c1-83b143d34784-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:24 crc kubenswrapper[5052]: I1203 12:28:24.919750 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/c080dc7f-df72-487f-b2c1-83b143d34784-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:24 crc kubenswrapper[5052]: I1203 12:28:24.919764 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-c8nwg\" (UniqueName: \"kubernetes.io/projected/c080dc7f-df72-487f-b2c1-83b143d34784-kube-api-access-c8nwg\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:25 crc kubenswrapper[5052]: I1203 12:28:25.022852 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 12:28:25 crc kubenswrapper[5052]: I1203 12:28:25.034717 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 12:28:25 crc kubenswrapper[5052]: I1203 12:28:25.066572 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 12:28:25 crc kubenswrapper[5052]: E1203 12:28:25.067152 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="c080dc7f-df72-487f-b2c1-83b143d34784" containerName="nova-cell0-conductor-conductor" Dec 03 12:28:25 crc kubenswrapper[5052]: I1203 12:28:25.067171 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="c080dc7f-df72-487f-b2c1-83b143d34784" containerName="nova-cell0-conductor-conductor" Dec 03 12:28:25 crc kubenswrapper[5052]: I1203 12:28:25.067450 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="c080dc7f-df72-487f-b2c1-83b143d34784" containerName="nova-cell0-conductor-conductor" Dec 03 12:28:25 crc kubenswrapper[5052]: I1203 12:28:25.073265 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:28:25 crc kubenswrapper[5052]: I1203 12:28:25.078831 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cinder-scheduler-config-data" Dec 03 12:28:25 crc kubenswrapper[5052]: W1203 12:28:25.082918 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd808ef0b_751e_4576_befc_e23a06fe2ff0.slice/crio-cee8f16e8e1679b820f59ea004534fd031076cdb92642915cc2863807a77f993 WatchSource:0}: Error finding container cee8f16e8e1679b820f59ea004534fd031076cdb92642915cc2863807a77f993: Status 404 returned error can't find the container with id cee8f16e8e1679b820f59ea004534fd031076cdb92642915cc2863807a77f993 Dec 03 12:28:25 crc kubenswrapper[5052]: I1203 12:28:25.091173 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 12:28:25 crc kubenswrapper[5052]: W1203 12:28:25.119852 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5ae7c56e_3507_403b_b58a_1a6da1ed1ab3.slice/crio-b5d7aeb00cbc1f82d726e8263d9745fee3d43b3b5dca68f30f648ab2bc48da8c WatchSource:0}: Error finding container b5d7aeb00cbc1f82d726e8263d9745fee3d43b3b5dca68f30f648ab2bc48da8c: Status 404 returned error can't find the container with id b5d7aeb00cbc1f82d726e8263d9745fee3d43b3b5dca68f30f648ab2bc48da8c Dec 03 12:28:25 crc kubenswrapper[5052]: I1203 12:28:25.123326 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:28:25 crc kubenswrapper[5052]: I1203 12:28:25.125477 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/214a26fb-e9b2-4a87-8fa5-ae2ed0d8a828-config-data-custom\") pod \"cinder-scheduler-0\" (UID: \"214a26fb-e9b2-4a87-8fa5-ae2ed0d8a828\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:28:25 crc kubenswrapper[5052]: I1203 12:28:25.125751 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/214a26fb-e9b2-4a87-8fa5-ae2ed0d8a828-scripts\") pod \"cinder-scheduler-0\" (UID: \"214a26fb-e9b2-4a87-8fa5-ae2ed0d8a828\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:28:25 crc kubenswrapper[5052]: I1203 12:28:25.125840 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bkz7n\" (UniqueName: \"kubernetes.io/projected/214a26fb-e9b2-4a87-8fa5-ae2ed0d8a828-kube-api-access-bkz7n\") pod \"cinder-scheduler-0\" (UID: \"214a26fb-e9b2-4a87-8fa5-ae2ed0d8a828\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:28:25 crc kubenswrapper[5052]: I1203 12:28:25.125999 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/214a26fb-e9b2-4a87-8fa5-ae2ed0d8a828-combined-ca-bundle\") pod \"cinder-scheduler-0\" (UID: \"214a26fb-e9b2-4a87-8fa5-ae2ed0d8a828\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:28:25 crc kubenswrapper[5052]: I1203 12:28:25.126104 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/214a26fb-e9b2-4a87-8fa5-ae2ed0d8a828-config-data\") pod \"cinder-scheduler-0\" (UID: \"214a26fb-e9b2-4a87-8fa5-ae2ed0d8a828\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:28:25 crc kubenswrapper[5052]: I1203 12:28:25.126162 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/214a26fb-e9b2-4a87-8fa5-ae2ed0d8a828-etc-machine-id\") pod \"cinder-scheduler-0\" (UID: \"214a26fb-e9b2-4a87-8fa5-ae2ed0d8a828\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:28:25 crc kubenswrapper[5052]: I1203 12:28:25.140011 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-0"] Dec 03 12:28:25 crc kubenswrapper[5052]: I1203 12:28:25.151880 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:28:25 crc kubenswrapper[5052]: I1203 12:28:25.228910 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/214a26fb-e9b2-4a87-8fa5-ae2ed0d8a828-config-data-custom\") pod \"cinder-scheduler-0\" (UID: \"214a26fb-e9b2-4a87-8fa5-ae2ed0d8a828\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:28:25 crc kubenswrapper[5052]: I1203 12:28:25.229002 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/214a26fb-e9b2-4a87-8fa5-ae2ed0d8a828-scripts\") pod \"cinder-scheduler-0\" (UID: \"214a26fb-e9b2-4a87-8fa5-ae2ed0d8a828\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:28:25 crc kubenswrapper[5052]: I1203 12:28:25.229055 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-bkz7n\" (UniqueName: \"kubernetes.io/projected/214a26fb-e9b2-4a87-8fa5-ae2ed0d8a828-kube-api-access-bkz7n\") pod \"cinder-scheduler-0\" (UID: \"214a26fb-e9b2-4a87-8fa5-ae2ed0d8a828\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:28:25 crc kubenswrapper[5052]: I1203 12:28:25.229126 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/214a26fb-e9b2-4a87-8fa5-ae2ed0d8a828-combined-ca-bundle\") pod \"cinder-scheduler-0\" (UID: \"214a26fb-e9b2-4a87-8fa5-ae2ed0d8a828\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:28:25 crc kubenswrapper[5052]: I1203 12:28:25.229150 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/214a26fb-e9b2-4a87-8fa5-ae2ed0d8a828-config-data\") pod \"cinder-scheduler-0\" (UID: \"214a26fb-e9b2-4a87-8fa5-ae2ed0d8a828\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:28:25 crc kubenswrapper[5052]: I1203 12:28:25.229174 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/214a26fb-e9b2-4a87-8fa5-ae2ed0d8a828-etc-machine-id\") pod \"cinder-scheduler-0\" (UID: \"214a26fb-e9b2-4a87-8fa5-ae2ed0d8a828\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:28:25 crc kubenswrapper[5052]: I1203 12:28:25.229269 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/214a26fb-e9b2-4a87-8fa5-ae2ed0d8a828-etc-machine-id\") pod \"cinder-scheduler-0\" (UID: \"214a26fb-e9b2-4a87-8fa5-ae2ed0d8a828\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:28:25 crc kubenswrapper[5052]: I1203 12:28:25.241125 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/214a26fb-e9b2-4a87-8fa5-ae2ed0d8a828-scripts\") pod \"cinder-scheduler-0\" (UID: \"214a26fb-e9b2-4a87-8fa5-ae2ed0d8a828\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:28:25 crc kubenswrapper[5052]: I1203 12:28:25.241500 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/214a26fb-e9b2-4a87-8fa5-ae2ed0d8a828-combined-ca-bundle\") pod \"cinder-scheduler-0\" (UID: \"214a26fb-e9b2-4a87-8fa5-ae2ed0d8a828\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:28:25 crc kubenswrapper[5052]: I1203 12:28:25.244745 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/214a26fb-e9b2-4a87-8fa5-ae2ed0d8a828-config-data-custom\") pod \"cinder-scheduler-0\" (UID: \"214a26fb-e9b2-4a87-8fa5-ae2ed0d8a828\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:28:25 crc kubenswrapper[5052]: I1203 12:28:25.246347 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/214a26fb-e9b2-4a87-8fa5-ae2ed0d8a828-config-data\") pod \"cinder-scheduler-0\" (UID: \"214a26fb-e9b2-4a87-8fa5-ae2ed0d8a828\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:28:25 crc kubenswrapper[5052]: I1203 12:28:25.264812 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-bkz7n\" (UniqueName: \"kubernetes.io/projected/214a26fb-e9b2-4a87-8fa5-ae2ed0d8a828-kube-api-access-bkz7n\") pod \"cinder-scheduler-0\" (UID: \"214a26fb-e9b2-4a87-8fa5-ae2ed0d8a828\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:28:25 crc kubenswrapper[5052]: I1203 12:28:25.417899 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:28:25 crc kubenswrapper[5052]: I1203 12:28:25.714162 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6436bca9-c7a1-4540-bee3-b6dbabb53c89" path="/var/lib/kubelet/pods/6436bca9-c7a1-4540-bee3-b6dbabb53c89/volumes" Dec 03 12:28:25 crc kubenswrapper[5052]: I1203 12:28:25.719301 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fcee2113-2b2f-47f9-b118-4f7488ad604e" path="/var/lib/kubelet/pods/fcee2113-2b2f-47f9-b118-4f7488ad604e/volumes" Dec 03 12:28:25 crc kubenswrapper[5052]: I1203 12:28:25.727886 5052 generic.go:334] "Generic (PLEG): container finished" podID="e2d1b100-5918-4166-9fa1-5a094612727f" containerID="8a517664093b91e9ef8c95cddf9f11b7aba074cb44d5aee9b5c50411254e47b7" exitCode=0 Dec 03 12:28:25 crc kubenswrapper[5052]: I1203 12:28:25.728413 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-cell-mapping-6nbfg" event={"ID":"e2d1b100-5918-4166-9fa1-5a094612727f","Type":"ContainerDied","Data":"8a517664093b91e9ef8c95cddf9f11b7aba074cb44d5aee9b5c50411254e47b7"} Dec 03 12:28:25 crc kubenswrapper[5052]: I1203 12:28:25.756071 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-conductor-0" event={"ID":"c080dc7f-df72-487f-b2c1-83b143d34784","Type":"ContainerDied","Data":"9b964a0bf45ecb54f67b88f63c0e3bbdac82ac6ab7ec4719bcdf02c46f1d08fc"} Dec 03 12:28:25 crc kubenswrapper[5052]: I1203 12:28:25.756130 5052 scope.go:117] "RemoveContainer" containerID="790991ac480a9e358efc31b18c1e765ec3b8f7c9c3386cf9c51c6dc7f800750c" Dec 03 12:28:25 crc kubenswrapper[5052]: I1203 12:28:25.756304 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:28:25 crc kubenswrapper[5052]: I1203 12:28:25.780496 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-conductor-0" event={"ID":"5ae7c56e-3507-403b-b58a-1a6da1ed1ab3","Type":"ContainerStarted","Data":"61a4505e879a4045e7b22afd402d01101f72aedd5245a9a876b0255b885b15ed"} Dec 03 12:28:25 crc kubenswrapper[5052]: I1203 12:28:25.790621 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-conductor-0" event={"ID":"5ae7c56e-3507-403b-b58a-1a6da1ed1ab3","Type":"ContainerStarted","Data":"b5d7aeb00cbc1f82d726e8263d9745fee3d43b3b5dca68f30f648ab2bc48da8c"} Dec 03 12:28:25 crc kubenswrapper[5052]: I1203 12:28:25.791543 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:28:25 crc kubenswrapper[5052]: I1203 12:28:25.792052 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"2fe3990a-96b5-4186-a646-ffc72cee7c57","Type":"ContainerStarted","Data":"c1264e30ab70f1cf44fa7b15b62f3332a4a2e2340f43c8dd6e409046273165fc"} Dec 03 12:28:25 crc kubenswrapper[5052]: I1203 12:28:25.792183 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"2fe3990a-96b5-4186-a646-ffc72cee7c57","Type":"ContainerStarted","Data":"20bfc436d9c14edbb7f38eb528b458af2431287e7c6f5ae3b8edc2da21eef0f7"} Dec 03 12:28:25 crc kubenswrapper[5052]: I1203 12:28:25.795606 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"18dae0fe-fcde-4a2e-bfd5-3c4ee6976894","Type":"ContainerStarted","Data":"bf3135f41d57b4319b491c492f40b5262bdbbb4439ae6f42157eab9d6b44e24e"} Dec 03 12:28:25 crc kubenswrapper[5052]: I1203 12:28:25.795664 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"18dae0fe-fcde-4a2e-bfd5-3c4ee6976894","Type":"ContainerStarted","Data":"99dbf56ce8d246aecdf81aeda0179117dfbc80314e99724ce9f7f2fa4b34dad1"} Dec 03 12:28:25 crc kubenswrapper[5052]: I1203 12:28:25.798693 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"d808ef0b-751e-4576-befc-e23a06fe2ff0","Type":"ContainerStarted","Data":"9b7bab21260f028f58b297b4ef87a5852a2f31508dfedff95c3c79316a694a8f"} Dec 03 12:28:25 crc kubenswrapper[5052]: I1203 12:28:25.798737 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"d808ef0b-751e-4576-befc-e23a06fe2ff0","Type":"ContainerStarted","Data":"cee8f16e8e1679b820f59ea004534fd031076cdb92642915cc2863807a77f993"} Dec 03 12:28:25 crc kubenswrapper[5052]: I1203 12:28:25.818752 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-0"] Dec 03 12:28:25 crc kubenswrapper[5052]: I1203 12:28:25.848164 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-0"] Dec 03 12:28:25 crc kubenswrapper[5052]: I1203 12:28:25.878314 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-0"] Dec 03 12:28:25 crc kubenswrapper[5052]: I1203 12:28:25.879968 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:28:25 crc kubenswrapper[5052]: I1203 12:28:25.880498 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-cell1-conductor-0" podStartSLOduration=2.880485107 podStartE2EDuration="2.880485107s" podCreationTimestamp="2025-12-03 12:28:23 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:28:25.810651414 +0000 UTC m=+4096.398893188" watchObservedRunningTime="2025-12-03 12:28:25.880485107 +0000 UTC m=+4096.468726871" Dec 03 12:28:25 crc kubenswrapper[5052]: I1203 12:28:25.884417 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell0-conductor-config-data" Dec 03 12:28:25 crc kubenswrapper[5052]: I1203 12:28:25.911323 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-0"] Dec 03 12:28:25 crc kubenswrapper[5052]: I1203 12:28:25.916689 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-scheduler-0" podStartSLOduration=3.91666769 podStartE2EDuration="3.91666769s" podCreationTimestamp="2025-12-03 12:28:22 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:28:25.830756728 +0000 UTC m=+4096.418998502" watchObservedRunningTime="2025-12-03 12:28:25.91666769 +0000 UTC m=+4096.504909454" Dec 03 12:28:25 crc kubenswrapper[5052]: I1203 12:28:25.954655 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bgp8v\" (UniqueName: \"kubernetes.io/projected/6efd22cd-98c7-48d8-a346-289b78e8aa1c-kube-api-access-bgp8v\") pod \"nova-cell0-conductor-0\" (UID: \"6efd22cd-98c7-48d8-a346-289b78e8aa1c\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:28:25 crc kubenswrapper[5052]: I1203 12:28:25.954777 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6efd22cd-98c7-48d8-a346-289b78e8aa1c-combined-ca-bundle\") pod \"nova-cell0-conductor-0\" (UID: \"6efd22cd-98c7-48d8-a346-289b78e8aa1c\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:28:25 crc kubenswrapper[5052]: I1203 12:28:25.954852 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6efd22cd-98c7-48d8-a346-289b78e8aa1c-config-data\") pod \"nova-cell0-conductor-0\" (UID: \"6efd22cd-98c7-48d8-a346-289b78e8aa1c\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:28:25 crc kubenswrapper[5052]: I1203 12:28:25.998080 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 12:28:26 crc kubenswrapper[5052]: I1203 12:28:26.056366 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6efd22cd-98c7-48d8-a346-289b78e8aa1c-combined-ca-bundle\") pod \"nova-cell0-conductor-0\" (UID: \"6efd22cd-98c7-48d8-a346-289b78e8aa1c\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:28:26 crc kubenswrapper[5052]: I1203 12:28:26.056490 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6efd22cd-98c7-48d8-a346-289b78e8aa1c-config-data\") pod \"nova-cell0-conductor-0\" (UID: \"6efd22cd-98c7-48d8-a346-289b78e8aa1c\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:28:26 crc kubenswrapper[5052]: I1203 12:28:26.056746 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-bgp8v\" (UniqueName: \"kubernetes.io/projected/6efd22cd-98c7-48d8-a346-289b78e8aa1c-kube-api-access-bgp8v\") pod \"nova-cell0-conductor-0\" (UID: \"6efd22cd-98c7-48d8-a346-289b78e8aa1c\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:28:26 crc kubenswrapper[5052]: I1203 12:28:26.061220 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6efd22cd-98c7-48d8-a346-289b78e8aa1c-config-data\") pod \"nova-cell0-conductor-0\" (UID: \"6efd22cd-98c7-48d8-a346-289b78e8aa1c\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:28:26 crc kubenswrapper[5052]: I1203 12:28:26.065237 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6efd22cd-98c7-48d8-a346-289b78e8aa1c-combined-ca-bundle\") pod \"nova-cell0-conductor-0\" (UID: \"6efd22cd-98c7-48d8-a346-289b78e8aa1c\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:28:26 crc kubenswrapper[5052]: I1203 12:28:26.080187 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-bgp8v\" (UniqueName: \"kubernetes.io/projected/6efd22cd-98c7-48d8-a346-289b78e8aa1c-kube-api-access-bgp8v\") pod \"nova-cell0-conductor-0\" (UID: \"6efd22cd-98c7-48d8-a346-289b78e8aa1c\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:28:26 crc kubenswrapper[5052]: I1203 12:28:26.207994 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:28:26 crc kubenswrapper[5052]: I1203 12:28:26.741102 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-0"] Dec 03 12:28:26 crc kubenswrapper[5052]: I1203 12:28:26.829228 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"18dae0fe-fcde-4a2e-bfd5-3c4ee6976894","Type":"ContainerStarted","Data":"486f511e2e18cfc8ea29b34eeca83447c73622373f7fc22a86772a5bb4c258ab"} Dec 03 12:28:26 crc kubenswrapper[5052]: I1203 12:28:26.832535 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"d808ef0b-751e-4576-befc-e23a06fe2ff0","Type":"ContainerStarted","Data":"bc91adc346315718fd765cd7a7a58ccc06d83103b40be9d46a7d2e14981369da"} Dec 03 12:28:26 crc kubenswrapper[5052]: I1203 12:28:26.841349 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-conductor-0" event={"ID":"6efd22cd-98c7-48d8-a346-289b78e8aa1c","Type":"ContainerStarted","Data":"85c3ce2cbe0bd990ceed107cc324a44256bec481856bf0d9c8cdfe468f9e3a70"} Dec 03 12:28:26 crc kubenswrapper[5052]: I1203 12:28:26.853454 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"214a26fb-e9b2-4a87-8fa5-ae2ed0d8a828","Type":"ContainerStarted","Data":"fbd118daf512469b0a39320f016c6056298c40b87f1e280084246d4ba8f21cf7"} Dec 03 12:28:26 crc kubenswrapper[5052]: I1203 12:28:26.859776 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-metadata-0" podStartSLOduration=4.859750678 podStartE2EDuration="4.859750678s" podCreationTimestamp="2025-12-03 12:28:22 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:28:26.847355784 +0000 UTC m=+4097.435597548" watchObservedRunningTime="2025-12-03 12:28:26.859750678 +0000 UTC m=+4097.447992442" Dec 03 12:28:26 crc kubenswrapper[5052]: I1203 12:28:26.890050 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-api-0" podStartSLOduration=4.890024492 podStartE2EDuration="4.890024492s" podCreationTimestamp="2025-12-03 12:28:22 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:28:26.868395385 +0000 UTC m=+4097.456637159" watchObservedRunningTime="2025-12-03 12:28:26.890024492 +0000 UTC m=+4097.478266266" Dec 03 12:28:26 crc kubenswrapper[5052]: I1203 12:28:26.948042 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/nova-metadata-0" podUID="8a2c61ae-6521-424f-aeac-52e1ec1c319a" containerName="nova-metadata-log" probeResult="failure" output="Get \"http://10.217.0.141:8775/\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Dec 03 12:28:26 crc kubenswrapper[5052]: I1203 12:28:26.948474 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/nova-metadata-0" podUID="8a2c61ae-6521-424f-aeac-52e1ec1c319a" containerName="nova-metadata-metadata" probeResult="failure" output="Get \"http://10.217.0.141:8775/\": dial tcp 10.217.0.141:8775: i/o timeout (Client.Timeout exceeded while awaiting headers)" Dec 03 12:28:27 crc kubenswrapper[5052]: I1203 12:28:27.172241 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-cell-mapping-6nbfg" Dec 03 12:28:27 crc kubenswrapper[5052]: I1203 12:28:27.285845 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/e2d1b100-5918-4166-9fa1-5a094612727f-scripts\") pod \"e2d1b100-5918-4166-9fa1-5a094612727f\" (UID: \"e2d1b100-5918-4166-9fa1-5a094612727f\") " Dec 03 12:28:27 crc kubenswrapper[5052]: I1203 12:28:27.285906 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e2d1b100-5918-4166-9fa1-5a094612727f-combined-ca-bundle\") pod \"e2d1b100-5918-4166-9fa1-5a094612727f\" (UID: \"e2d1b100-5918-4166-9fa1-5a094612727f\") " Dec 03 12:28:27 crc kubenswrapper[5052]: I1203 12:28:27.286028 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/e2d1b100-5918-4166-9fa1-5a094612727f-config-data\") pod \"e2d1b100-5918-4166-9fa1-5a094612727f\" (UID: \"e2d1b100-5918-4166-9fa1-5a094612727f\") " Dec 03 12:28:27 crc kubenswrapper[5052]: I1203 12:28:27.286224 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xl5r4\" (UniqueName: \"kubernetes.io/projected/e2d1b100-5918-4166-9fa1-5a094612727f-kube-api-access-xl5r4\") pod \"e2d1b100-5918-4166-9fa1-5a094612727f\" (UID: \"e2d1b100-5918-4166-9fa1-5a094612727f\") " Dec 03 12:28:27 crc kubenswrapper[5052]: I1203 12:28:27.291705 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e2d1b100-5918-4166-9fa1-5a094612727f-scripts" (OuterVolumeSpecName: "scripts") pod "e2d1b100-5918-4166-9fa1-5a094612727f" (UID: "e2d1b100-5918-4166-9fa1-5a094612727f"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:28:27 crc kubenswrapper[5052]: I1203 12:28:27.292366 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e2d1b100-5918-4166-9fa1-5a094612727f-kube-api-access-xl5r4" (OuterVolumeSpecName: "kube-api-access-xl5r4") pod "e2d1b100-5918-4166-9fa1-5a094612727f" (UID: "e2d1b100-5918-4166-9fa1-5a094612727f"). InnerVolumeSpecName "kube-api-access-xl5r4". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:28:27 crc kubenswrapper[5052]: I1203 12:28:27.319150 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e2d1b100-5918-4166-9fa1-5a094612727f-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "e2d1b100-5918-4166-9fa1-5a094612727f" (UID: "e2d1b100-5918-4166-9fa1-5a094612727f"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:28:27 crc kubenswrapper[5052]: I1203 12:28:27.328872 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e2d1b100-5918-4166-9fa1-5a094612727f-config-data" (OuterVolumeSpecName: "config-data") pod "e2d1b100-5918-4166-9fa1-5a094612727f" (UID: "e2d1b100-5918-4166-9fa1-5a094612727f"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:28:27 crc kubenswrapper[5052]: I1203 12:28:27.388527 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/e2d1b100-5918-4166-9fa1-5a094612727f-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:27 crc kubenswrapper[5052]: I1203 12:28:27.388561 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e2d1b100-5918-4166-9fa1-5a094612727f-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:27 crc kubenswrapper[5052]: I1203 12:28:27.388571 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/e2d1b100-5918-4166-9fa1-5a094612727f-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:27 crc kubenswrapper[5052]: I1203 12:28:27.388580 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-xl5r4\" (UniqueName: \"kubernetes.io/projected/e2d1b100-5918-4166-9fa1-5a094612727f-kube-api-access-xl5r4\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:27 crc kubenswrapper[5052]: I1203 12:28:27.701090 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c080dc7f-df72-487f-b2c1-83b143d34784" path="/var/lib/kubelet/pods/c080dc7f-df72-487f-b2c1-83b143d34784/volumes" Dec 03 12:28:27 crc kubenswrapper[5052]: I1203 12:28:27.868936 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-cell-mapping-6nbfg" event={"ID":"e2d1b100-5918-4166-9fa1-5a094612727f","Type":"ContainerDied","Data":"82b7c2702af47b346cff159435051eca59fb061ec03e5c7b1b1b84e2fffe24d8"} Dec 03 12:28:27 crc kubenswrapper[5052]: I1203 12:28:27.868985 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="82b7c2702af47b346cff159435051eca59fb061ec03e5c7b1b1b84e2fffe24d8" Dec 03 12:28:27 crc kubenswrapper[5052]: I1203 12:28:27.869073 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-cell-mapping-6nbfg" Dec 03 12:28:27 crc kubenswrapper[5052]: I1203 12:28:27.874459 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-conductor-0" event={"ID":"6efd22cd-98c7-48d8-a346-289b78e8aa1c","Type":"ContainerStarted","Data":"ae1f276ec0ccddfcdb84f346890a9d2aa8e21c3960425cd4ca2a6483bc8b8ae7"} Dec 03 12:28:27 crc kubenswrapper[5052]: I1203 12:28:27.874629 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:28:27 crc kubenswrapper[5052]: I1203 12:28:27.876883 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"214a26fb-e9b2-4a87-8fa5-ae2ed0d8a828","Type":"ContainerStarted","Data":"27a238af716222ccb80b51b0ddb93658176d3aa5f8db130deec5a43ab67bb332"} Dec 03 12:28:27 crc kubenswrapper[5052]: I1203 12:28:27.878165 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"214a26fb-e9b2-4a87-8fa5-ae2ed0d8a828","Type":"ContainerStarted","Data":"b6c09457c183d144bb440f045e7a0230c87297b84897525513a2e671062b9c96"} Dec 03 12:28:27 crc kubenswrapper[5052]: I1203 12:28:27.909151 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-cell0-conductor-0" podStartSLOduration=2.9091288889999998 podStartE2EDuration="2.909128889s" podCreationTimestamp="2025-12-03 12:28:25 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:28:27.890864317 +0000 UTC m=+4098.479106081" watchObservedRunningTime="2025-12-03 12:28:27.909128889 +0000 UTC m=+4098.497370663" Dec 03 12:28:27 crc kubenswrapper[5052]: I1203 12:28:27.923972 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/cinder-scheduler-0" podStartSLOduration=2.923950332 podStartE2EDuration="2.923950332s" podCreationTimestamp="2025-12-03 12:28:25 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:28:27.917896709 +0000 UTC m=+4098.506138483" watchObservedRunningTime="2025-12-03 12:28:27.923950332 +0000 UTC m=+4098.512192096" Dec 03 12:28:27 crc kubenswrapper[5052]: I1203 12:28:27.934684 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:28:28 crc kubenswrapper[5052]: I1203 12:28:28.143486 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:28:28 crc kubenswrapper[5052]: I1203 12:28:28.143612 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:28:28 crc kubenswrapper[5052]: I1203 12:28:28.889566 5052 generic.go:334] "Generic (PLEG): container finished" podID="51d1c5f3-0622-4c29-a19a-f1ecd293654e" containerID="c1a01685a9df4a046418f219f4791232079759a5dfaf1dcd3a5230170ee077ab" exitCode=0 Dec 03 12:28:28 crc kubenswrapper[5052]: I1203 12:28:28.889921 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-cell-mapping-7v2kh" event={"ID":"51d1c5f3-0622-4c29-a19a-f1ecd293654e","Type":"ContainerDied","Data":"c1a01685a9df4a046418f219f4791232079759a5dfaf1dcd3a5230170ee077ab"} Dec 03 12:28:28 crc kubenswrapper[5052]: I1203 12:28:28.893981 5052 generic.go:334] "Generic (PLEG): container finished" podID="2fe3990a-96b5-4186-a646-ffc72cee7c57" containerID="c1264e30ab70f1cf44fa7b15b62f3332a4a2e2340f43c8dd6e409046273165fc" exitCode=1 Dec 03 12:28:28 crc kubenswrapper[5052]: I1203 12:28:28.894769 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"2fe3990a-96b5-4186-a646-ffc72cee7c57","Type":"ContainerDied","Data":"c1264e30ab70f1cf44fa7b15b62f3332a4a2e2340f43c8dd6e409046273165fc"} Dec 03 12:28:28 crc kubenswrapper[5052]: I1203 12:28:28.895376 5052 scope.go:117] "RemoveContainer" containerID="c1264e30ab70f1cf44fa7b15b62f3332a4a2e2340f43c8dd6e409046273165fc" Dec 03 12:28:28 crc kubenswrapper[5052]: I1203 12:28:28.898048 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/neutron-56ff9fdb74-cft5n" Dec 03 12:28:29 crc kubenswrapper[5052]: I1203 12:28:29.017454 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/neutron-f699c657d-qh2qk"] Dec 03 12:28:29 crc kubenswrapper[5052]: I1203 12:28:29.017716 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/neutron-f699c657d-qh2qk" podUID="a28346c0-bd60-4dac-ac86-20b56296e7d3" containerName="neutron-httpd" containerID="cri-o://d10711abf49e5a87c80edf05ade50ea33f8c777fb550c91e299195151230bed0" gracePeriod=30 Dec 03 12:28:29 crc kubenswrapper[5052]: I1203 12:28:29.018005 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/neutron-f699c657d-qh2qk" podUID="a28346c0-bd60-4dac-ac86-20b56296e7d3" containerName="neutron-api" containerID="cri-o://c55b9fbfdc46a4cb2612d4910e298c39d926f0f567206d0d4eb4cba1b2cf66ed" gracePeriod=30 Dec 03 12:28:29 crc kubenswrapper[5052]: I1203 12:28:29.119816 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-marketplace/certified-operators-75j45" Dec 03 12:28:29 crc kubenswrapper[5052]: I1203 12:28:29.180387 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openshift-marketplace/certified-operators-75j45"] Dec 03 12:28:29 crc kubenswrapper[5052]: I1203 12:28:29.380012 5052 patch_prober.go:28] interesting pod/machine-config-daemon-74bj6 container/machine-config-daemon namespace/openshift-machine-config-operator: Liveness probe status=failure output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" start-of-body= Dec 03 12:28:29 crc kubenswrapper[5052]: I1203 12:28:29.380070 5052 prober.go:107] "Probe failed" probeType="Liveness" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" containerName="machine-config-daemon" probeResult="failure" output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" Dec 03 12:28:29 crc kubenswrapper[5052]: I1203 12:28:29.824229 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:28:29 crc kubenswrapper[5052]: I1203 12:28:29.845488 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:28:29 crc kubenswrapper[5052]: I1203 12:28:29.905221 5052 generic.go:334] "Generic (PLEG): container finished" podID="a28346c0-bd60-4dac-ac86-20b56296e7d3" containerID="d10711abf49e5a87c80edf05ade50ea33f8c777fb550c91e299195151230bed0" exitCode=0 Dec 03 12:28:29 crc kubenswrapper[5052]: I1203 12:28:29.905288 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-f699c657d-qh2qk" event={"ID":"a28346c0-bd60-4dac-ac86-20b56296e7d3","Type":"ContainerDied","Data":"d10711abf49e5a87c80edf05ade50ea33f8c777fb550c91e299195151230bed0"} Dec 03 12:28:29 crc kubenswrapper[5052]: I1203 12:28:29.907398 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"2fe3990a-96b5-4186-a646-ffc72cee7c57","Type":"ContainerStarted","Data":"98da5decb3746695c92ec1ef5e46ac70fd2430efb58f4f9e676f42d7ef1f7a6c"} Dec 03 12:28:29 crc kubenswrapper[5052]: I1203 12:28:29.908302 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openshift-marketplace/certified-operators-75j45" podUID="c03ce333-5bdc-4cdb-9a81-53db1322b7bf" containerName="registry-server" containerID="cri-o://aaf64268b97b9c5579d623002cd16894fe6902dd1ab89073573d9f967bab79ed" gracePeriod=2 Dec 03 12:28:30 crc kubenswrapper[5052]: I1203 12:28:30.117389 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:28:30 crc kubenswrapper[5052]: I1203 12:28:30.426748 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:28:30 crc kubenswrapper[5052]: I1203 12:28:30.666162 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/certified-operators-75j45" Dec 03 12:28:30 crc kubenswrapper[5052]: I1203 12:28:30.670793 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-cell-mapping-7v2kh" Dec 03 12:28:30 crc kubenswrapper[5052]: I1203 12:28:30.757821 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/c03ce333-5bdc-4cdb-9a81-53db1322b7bf-catalog-content\") pod \"c03ce333-5bdc-4cdb-9a81-53db1322b7bf\" (UID: \"c03ce333-5bdc-4cdb-9a81-53db1322b7bf\") " Dec 03 12:28:30 crc kubenswrapper[5052]: I1203 12:28:30.758038 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rq2rx\" (UniqueName: \"kubernetes.io/projected/51d1c5f3-0622-4c29-a19a-f1ecd293654e-kube-api-access-rq2rx\") pod \"51d1c5f3-0622-4c29-a19a-f1ecd293654e\" (UID: \"51d1c5f3-0622-4c29-a19a-f1ecd293654e\") " Dec 03 12:28:30 crc kubenswrapper[5052]: I1203 12:28:30.758113 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/51d1c5f3-0622-4c29-a19a-f1ecd293654e-scripts\") pod \"51d1c5f3-0622-4c29-a19a-f1ecd293654e\" (UID: \"51d1c5f3-0622-4c29-a19a-f1ecd293654e\") " Dec 03 12:28:30 crc kubenswrapper[5052]: I1203 12:28:30.758139 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/51d1c5f3-0622-4c29-a19a-f1ecd293654e-combined-ca-bundle\") pod \"51d1c5f3-0622-4c29-a19a-f1ecd293654e\" (UID: \"51d1c5f3-0622-4c29-a19a-f1ecd293654e\") " Dec 03 12:28:30 crc kubenswrapper[5052]: I1203 12:28:30.758163 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/51d1c5f3-0622-4c29-a19a-f1ecd293654e-config-data\") pod \"51d1c5f3-0622-4c29-a19a-f1ecd293654e\" (UID: \"51d1c5f3-0622-4c29-a19a-f1ecd293654e\") " Dec 03 12:28:30 crc kubenswrapper[5052]: I1203 12:28:30.758207 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bgqmr\" (UniqueName: \"kubernetes.io/projected/c03ce333-5bdc-4cdb-9a81-53db1322b7bf-kube-api-access-bgqmr\") pod \"c03ce333-5bdc-4cdb-9a81-53db1322b7bf\" (UID: \"c03ce333-5bdc-4cdb-9a81-53db1322b7bf\") " Dec 03 12:28:30 crc kubenswrapper[5052]: I1203 12:28:30.758246 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/c03ce333-5bdc-4cdb-9a81-53db1322b7bf-utilities\") pod \"c03ce333-5bdc-4cdb-9a81-53db1322b7bf\" (UID: \"c03ce333-5bdc-4cdb-9a81-53db1322b7bf\") " Dec 03 12:28:30 crc kubenswrapper[5052]: I1203 12:28:30.760033 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c03ce333-5bdc-4cdb-9a81-53db1322b7bf-utilities" (OuterVolumeSpecName: "utilities") pod "c03ce333-5bdc-4cdb-9a81-53db1322b7bf" (UID: "c03ce333-5bdc-4cdb-9a81-53db1322b7bf"). InnerVolumeSpecName "utilities". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:28:30 crc kubenswrapper[5052]: I1203 12:28:30.764304 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c03ce333-5bdc-4cdb-9a81-53db1322b7bf-kube-api-access-bgqmr" (OuterVolumeSpecName: "kube-api-access-bgqmr") pod "c03ce333-5bdc-4cdb-9a81-53db1322b7bf" (UID: "c03ce333-5bdc-4cdb-9a81-53db1322b7bf"). InnerVolumeSpecName "kube-api-access-bgqmr". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:28:30 crc kubenswrapper[5052]: I1203 12:28:30.775481 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/51d1c5f3-0622-4c29-a19a-f1ecd293654e-kube-api-access-rq2rx" (OuterVolumeSpecName: "kube-api-access-rq2rx") pod "51d1c5f3-0622-4c29-a19a-f1ecd293654e" (UID: "51d1c5f3-0622-4c29-a19a-f1ecd293654e"). InnerVolumeSpecName "kube-api-access-rq2rx". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:28:30 crc kubenswrapper[5052]: I1203 12:28:30.775575 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/51d1c5f3-0622-4c29-a19a-f1ecd293654e-scripts" (OuterVolumeSpecName: "scripts") pod "51d1c5f3-0622-4c29-a19a-f1ecd293654e" (UID: "51d1c5f3-0622-4c29-a19a-f1ecd293654e"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:28:30 crc kubenswrapper[5052]: I1203 12:28:30.785171 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/51d1c5f3-0622-4c29-a19a-f1ecd293654e-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "51d1c5f3-0622-4c29-a19a-f1ecd293654e" (UID: "51d1c5f3-0622-4c29-a19a-f1ecd293654e"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:28:30 crc kubenswrapper[5052]: I1203 12:28:30.787055 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/51d1c5f3-0622-4c29-a19a-f1ecd293654e-config-data" (OuterVolumeSpecName: "config-data") pod "51d1c5f3-0622-4c29-a19a-f1ecd293654e" (UID: "51d1c5f3-0622-4c29-a19a-f1ecd293654e"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:28:30 crc kubenswrapper[5052]: I1203 12:28:30.807733 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c03ce333-5bdc-4cdb-9a81-53db1322b7bf-catalog-content" (OuterVolumeSpecName: "catalog-content") pod "c03ce333-5bdc-4cdb-9a81-53db1322b7bf" (UID: "c03ce333-5bdc-4cdb-9a81-53db1322b7bf"). InnerVolumeSpecName "catalog-content". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:28:30 crc kubenswrapper[5052]: I1203 12:28:30.854537 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:28:30 crc kubenswrapper[5052]: I1203 12:28:30.860370 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-bgqmr\" (UniqueName: \"kubernetes.io/projected/c03ce333-5bdc-4cdb-9a81-53db1322b7bf-kube-api-access-bgqmr\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:30 crc kubenswrapper[5052]: I1203 12:28:30.860409 5052 reconciler_common.go:293] "Volume detached for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/c03ce333-5bdc-4cdb-9a81-53db1322b7bf-utilities\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:30 crc kubenswrapper[5052]: I1203 12:28:30.860421 5052 reconciler_common.go:293] "Volume detached for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/c03ce333-5bdc-4cdb-9a81-53db1322b7bf-catalog-content\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:30 crc kubenswrapper[5052]: I1203 12:28:30.860437 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-rq2rx\" (UniqueName: \"kubernetes.io/projected/51d1c5f3-0622-4c29-a19a-f1ecd293654e-kube-api-access-rq2rx\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:30 crc kubenswrapper[5052]: I1203 12:28:30.860449 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/51d1c5f3-0622-4c29-a19a-f1ecd293654e-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:30 crc kubenswrapper[5052]: I1203 12:28:30.860461 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/51d1c5f3-0622-4c29-a19a-f1ecd293654e-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:30 crc kubenswrapper[5052]: I1203 12:28:30.860474 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/51d1c5f3-0622-4c29-a19a-f1ecd293654e-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:30 crc kubenswrapper[5052]: I1203 12:28:30.921396 5052 generic.go:334] "Generic (PLEG): container finished" podID="c03ce333-5bdc-4cdb-9a81-53db1322b7bf" containerID="aaf64268b97b9c5579d623002cd16894fe6902dd1ab89073573d9f967bab79ed" exitCode=0 Dec 03 12:28:30 crc kubenswrapper[5052]: I1203 12:28:30.921477 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/certified-operators-75j45" Dec 03 12:28:30 crc kubenswrapper[5052]: I1203 12:28:30.921493 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/certified-operators-75j45" event={"ID":"c03ce333-5bdc-4cdb-9a81-53db1322b7bf","Type":"ContainerDied","Data":"aaf64268b97b9c5579d623002cd16894fe6902dd1ab89073573d9f967bab79ed"} Dec 03 12:28:30 crc kubenswrapper[5052]: I1203 12:28:30.922576 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/certified-operators-75j45" event={"ID":"c03ce333-5bdc-4cdb-9a81-53db1322b7bf","Type":"ContainerDied","Data":"20d0d6b095d35bba4010e4db05af2ee2d91770c7e3da2b155855b734fe480d55"} Dec 03 12:28:30 crc kubenswrapper[5052]: I1203 12:28:30.922603 5052 scope.go:117] "RemoveContainer" containerID="aaf64268b97b9c5579d623002cd16894fe6902dd1ab89073573d9f967bab79ed" Dec 03 12:28:30 crc kubenswrapper[5052]: I1203 12:28:30.925340 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-cell-mapping-7v2kh" Dec 03 12:28:30 crc kubenswrapper[5052]: I1203 12:28:30.925651 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-cell-mapping-7v2kh" event={"ID":"51d1c5f3-0622-4c29-a19a-f1ecd293654e","Type":"ContainerDied","Data":"fa20bc0e792f1502682a7b88dcf29415b5895d6f5c2357ab8bb5c18cd557e3f5"} Dec 03 12:28:30 crc kubenswrapper[5052]: I1203 12:28:30.925675 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fa20bc0e792f1502682a7b88dcf29415b5895d6f5c2357ab8bb5c18cd557e3f5" Dec 03 12:28:30 crc kubenswrapper[5052]: I1203 12:28:30.950611 5052 scope.go:117] "RemoveContainer" containerID="2d6610eeb64b5a8303646130c31ca4e821ae8775d47940490419f30250450ca1" Dec 03 12:28:30 crc kubenswrapper[5052]: I1203 12:28:30.986216 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openshift-marketplace/certified-operators-75j45"] Dec 03 12:28:30 crc kubenswrapper[5052]: I1203 12:28:30.996122 5052 scope.go:117] "RemoveContainer" containerID="969d3991d620214517bbadf0f483ee93450cc288b469877f94b38cc69da68c7c" Dec 03 12:28:30 crc kubenswrapper[5052]: I1203 12:28:30.996701 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openshift-marketplace/certified-operators-75j45"] Dec 03 12:28:31 crc kubenswrapper[5052]: I1203 12:28:31.034797 5052 scope.go:117] "RemoveContainer" containerID="aaf64268b97b9c5579d623002cd16894fe6902dd1ab89073573d9f967bab79ed" Dec 03 12:28:31 crc kubenswrapper[5052]: E1203 12:28:31.035414 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aaf64268b97b9c5579d623002cd16894fe6902dd1ab89073573d9f967bab79ed\": container with ID starting with aaf64268b97b9c5579d623002cd16894fe6902dd1ab89073573d9f967bab79ed not found: ID does not exist" containerID="aaf64268b97b9c5579d623002cd16894fe6902dd1ab89073573d9f967bab79ed" Dec 03 12:28:31 crc kubenswrapper[5052]: I1203 12:28:31.035446 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aaf64268b97b9c5579d623002cd16894fe6902dd1ab89073573d9f967bab79ed"} err="failed to get container status \"aaf64268b97b9c5579d623002cd16894fe6902dd1ab89073573d9f967bab79ed\": rpc error: code = NotFound desc = could not find container \"aaf64268b97b9c5579d623002cd16894fe6902dd1ab89073573d9f967bab79ed\": container with ID starting with aaf64268b97b9c5579d623002cd16894fe6902dd1ab89073573d9f967bab79ed not found: ID does not exist" Dec 03 12:28:31 crc kubenswrapper[5052]: I1203 12:28:31.035484 5052 scope.go:117] "RemoveContainer" containerID="2d6610eeb64b5a8303646130c31ca4e821ae8775d47940490419f30250450ca1" Dec 03 12:28:31 crc kubenswrapper[5052]: E1203 12:28:31.036096 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2d6610eeb64b5a8303646130c31ca4e821ae8775d47940490419f30250450ca1\": container with ID starting with 2d6610eeb64b5a8303646130c31ca4e821ae8775d47940490419f30250450ca1 not found: ID does not exist" containerID="2d6610eeb64b5a8303646130c31ca4e821ae8775d47940490419f30250450ca1" Dec 03 12:28:31 crc kubenswrapper[5052]: I1203 12:28:31.036135 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d6610eeb64b5a8303646130c31ca4e821ae8775d47940490419f30250450ca1"} err="failed to get container status \"2d6610eeb64b5a8303646130c31ca4e821ae8775d47940490419f30250450ca1\": rpc error: code = NotFound desc = could not find container \"2d6610eeb64b5a8303646130c31ca4e821ae8775d47940490419f30250450ca1\": container with ID starting with 2d6610eeb64b5a8303646130c31ca4e821ae8775d47940490419f30250450ca1 not found: ID does not exist" Dec 03 12:28:31 crc kubenswrapper[5052]: I1203 12:28:31.036159 5052 scope.go:117] "RemoveContainer" containerID="969d3991d620214517bbadf0f483ee93450cc288b469877f94b38cc69da68c7c" Dec 03 12:28:31 crc kubenswrapper[5052]: E1203 12:28:31.036399 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"969d3991d620214517bbadf0f483ee93450cc288b469877f94b38cc69da68c7c\": container with ID starting with 969d3991d620214517bbadf0f483ee93450cc288b469877f94b38cc69da68c7c not found: ID does not exist" containerID="969d3991d620214517bbadf0f483ee93450cc288b469877f94b38cc69da68c7c" Dec 03 12:28:31 crc kubenswrapper[5052]: I1203 12:28:31.036430 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"969d3991d620214517bbadf0f483ee93450cc288b469877f94b38cc69da68c7c"} err="failed to get container status \"969d3991d620214517bbadf0f483ee93450cc288b469877f94b38cc69da68c7c\": rpc error: code = NotFound desc = could not find container \"969d3991d620214517bbadf0f483ee93450cc288b469877f94b38cc69da68c7c\": container with ID starting with 969d3991d620214517bbadf0f483ee93450cc288b469877f94b38cc69da68c7c not found: ID does not exist" Dec 03 12:28:31 crc kubenswrapper[5052]: I1203 12:28:31.684820 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c03ce333-5bdc-4cdb-9a81-53db1322b7bf" path="/var/lib/kubelet/pods/c03ce333-5bdc-4cdb-9a81-53db1322b7bf/volumes" Dec 03 12:28:31 crc kubenswrapper[5052]: I1203 12:28:31.944194 5052 generic.go:334] "Generic (PLEG): container finished" podID="2fe3990a-96b5-4186-a646-ffc72cee7c57" containerID="98da5decb3746695c92ec1ef5e46ac70fd2430efb58f4f9e676f42d7ef1f7a6c" exitCode=1 Dec 03 12:28:31 crc kubenswrapper[5052]: I1203 12:28:31.944367 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"2fe3990a-96b5-4186-a646-ffc72cee7c57","Type":"ContainerDied","Data":"98da5decb3746695c92ec1ef5e46ac70fd2430efb58f4f9e676f42d7ef1f7a6c"} Dec 03 12:28:31 crc kubenswrapper[5052]: I1203 12:28:31.944572 5052 scope.go:117] "RemoveContainer" containerID="c1264e30ab70f1cf44fa7b15b62f3332a4a2e2340f43c8dd6e409046273165fc" Dec 03 12:28:31 crc kubenswrapper[5052]: I1203 12:28:31.945187 5052 scope.go:117] "RemoveContainer" containerID="98da5decb3746695c92ec1ef5e46ac70fd2430efb58f4f9e676f42d7ef1f7a6c" Dec 03 12:28:31 crc kubenswrapper[5052]: E1203 12:28:31.945417 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"nova-scheduler-scheduler\" with CrashLoopBackOff: \"back-off 10s restarting failed container=nova-scheduler-scheduler pod=nova-scheduler-0_openstack-kuttl-tests(2fe3990a-96b5-4186-a646-ffc72cee7c57)\"" pod="openstack-kuttl-tests/nova-scheduler-0" podUID="2fe3990a-96b5-4186-a646-ffc72cee7c57" Dec 03 12:28:32 crc kubenswrapper[5052]: I1203 12:28:32.932871 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:28:32 crc kubenswrapper[5052]: I1203 12:28:32.932924 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:28:32 crc kubenswrapper[5052]: I1203 12:28:32.935062 5052 kubelet.go:2542] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:28:32 crc kubenswrapper[5052]: I1203 12:28:32.935129 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:28:32 crc kubenswrapper[5052]: I1203 12:28:32.956188 5052 scope.go:117] "RemoveContainer" containerID="98da5decb3746695c92ec1ef5e46ac70fd2430efb58f4f9e676f42d7ef1f7a6c" Dec 03 12:28:32 crc kubenswrapper[5052]: E1203 12:28:32.956580 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"nova-scheduler-scheduler\" with CrashLoopBackOff: \"back-off 10s restarting failed container=nova-scheduler-scheduler pod=nova-scheduler-0_openstack-kuttl-tests(2fe3990a-96b5-4186-a646-ffc72cee7c57)\"" pod="openstack-kuttl-tests/nova-scheduler-0" podUID="2fe3990a-96b5-4186-a646-ffc72cee7c57" Dec 03 12:28:33 crc kubenswrapper[5052]: I1203 12:28:33.143453 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:28:33 crc kubenswrapper[5052]: I1203 12:28:33.143528 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:28:33 crc kubenswrapper[5052]: I1203 12:28:33.262294 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:28:33 crc kubenswrapper[5052]: I1203 12:28:33.262340 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:28:33 crc kubenswrapper[5052]: I1203 12:28:33.966339 5052 scope.go:117] "RemoveContainer" containerID="98da5decb3746695c92ec1ef5e46ac70fd2430efb58f4f9e676f42d7ef1f7a6c" Dec 03 12:28:33 crc kubenswrapper[5052]: E1203 12:28:33.967078 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"nova-scheduler-scheduler\" with CrashLoopBackOff: \"back-off 10s restarting failed container=nova-scheduler-scheduler pod=nova-scheduler-0_openstack-kuttl-tests(2fe3990a-96b5-4186-a646-ffc72cee7c57)\"" pod="openstack-kuttl-tests/nova-scheduler-0" podUID="2fe3990a-96b5-4186-a646-ffc72cee7c57" Dec 03 12:28:34 crc kubenswrapper[5052]: I1203 12:28:34.100028 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:28:34 crc kubenswrapper[5052]: I1203 12:28:34.190791 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-metadata-0" podUID="18dae0fe-fcde-4a2e-bfd5-3c4ee6976894" containerName="nova-metadata-log" probeResult="failure" output="Get \"https://10.217.0.167:8775/\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Dec 03 12:28:34 crc kubenswrapper[5052]: I1203 12:28:34.190795 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-metadata-0" podUID="18dae0fe-fcde-4a2e-bfd5-3c4ee6976894" containerName="nova-metadata-metadata" probeResult="failure" output="Get \"https://10.217.0.167:8775/\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Dec 03 12:28:34 crc kubenswrapper[5052]: I1203 12:28:34.275714 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-api-0" podUID="d808ef0b-751e-4576-befc-e23a06fe2ff0" containerName="nova-api-log" probeResult="failure" output="Get \"https://10.217.0.169:8774/\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Dec 03 12:28:34 crc kubenswrapper[5052]: I1203 12:28:34.276015 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-api-0" podUID="d808ef0b-751e-4576-befc-e23a06fe2ff0" containerName="nova-api-api" probeResult="failure" output="Get \"https://10.217.0.169:8774/\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Dec 03 12:28:35 crc kubenswrapper[5052]: I1203 12:28:35.687033 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:28:36 crc kubenswrapper[5052]: I1203 12:28:36.237900 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:28:36 crc kubenswrapper[5052]: I1203 12:28:36.425442 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:28:36 crc kubenswrapper[5052]: I1203 12:28:36.425828 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-api-0" podUID="d808ef0b-751e-4576-befc-e23a06fe2ff0" containerName="nova-api-log" containerID="cri-o://9b7bab21260f028f58b297b4ef87a5852a2f31508dfedff95c3c79316a694a8f" gracePeriod=30 Dec 03 12:28:36 crc kubenswrapper[5052]: I1203 12:28:36.425864 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-api-0" podUID="d808ef0b-751e-4576-befc-e23a06fe2ff0" containerName="nova-api-api" containerID="cri-o://bc91adc346315718fd765cd7a7a58ccc06d83103b40be9d46a7d2e14981369da" gracePeriod=30 Dec 03 12:28:36 crc kubenswrapper[5052]: I1203 12:28:36.469317 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:28:36 crc kubenswrapper[5052]: I1203 12:28:36.490440 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:28:36 crc kubenswrapper[5052]: I1203 12:28:36.491055 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-metadata-0" podUID="18dae0fe-fcde-4a2e-bfd5-3c4ee6976894" containerName="nova-metadata-log" containerID="cri-o://bf3135f41d57b4319b491c492f40b5262bdbbb4439ae6f42157eab9d6b44e24e" gracePeriod=30 Dec 03 12:28:36 crc kubenswrapper[5052]: I1203 12:28:36.491060 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-metadata-0" podUID="18dae0fe-fcde-4a2e-bfd5-3c4ee6976894" containerName="nova-metadata-metadata" containerID="cri-o://486f511e2e18cfc8ea29b34eeca83447c73622373f7fc22a86772a5bb4c258ab" gracePeriod=30 Dec 03 12:28:36 crc kubenswrapper[5052]: I1203 12:28:36.962267 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:28:36 crc kubenswrapper[5052]: I1203 12:28:36.995681 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/2fe3990a-96b5-4186-a646-ffc72cee7c57-config-data\") pod \"2fe3990a-96b5-4186-a646-ffc72cee7c57\" (UID: \"2fe3990a-96b5-4186-a646-ffc72cee7c57\") " Dec 03 12:28:36 crc kubenswrapper[5052]: I1203 12:28:36.995737 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-67g6g\" (UniqueName: \"kubernetes.io/projected/2fe3990a-96b5-4186-a646-ffc72cee7c57-kube-api-access-67g6g\") pod \"2fe3990a-96b5-4186-a646-ffc72cee7c57\" (UID: \"2fe3990a-96b5-4186-a646-ffc72cee7c57\") " Dec 03 12:28:36 crc kubenswrapper[5052]: I1203 12:28:36.995949 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2fe3990a-96b5-4186-a646-ffc72cee7c57-combined-ca-bundle\") pod \"2fe3990a-96b5-4186-a646-ffc72cee7c57\" (UID: \"2fe3990a-96b5-4186-a646-ffc72cee7c57\") " Dec 03 12:28:37 crc kubenswrapper[5052]: I1203 12:28:37.018097 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"2fe3990a-96b5-4186-a646-ffc72cee7c57","Type":"ContainerDied","Data":"20bfc436d9c14edbb7f38eb528b458af2431287e7c6f5ae3b8edc2da21eef0f7"} Dec 03 12:28:37 crc kubenswrapper[5052]: I1203 12:28:37.018633 5052 scope.go:117] "RemoveContainer" containerID="98da5decb3746695c92ec1ef5e46ac70fd2430efb58f4f9e676f42d7ef1f7a6c" Dec 03 12:28:37 crc kubenswrapper[5052]: I1203 12:28:37.019009 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:28:37 crc kubenswrapper[5052]: I1203 12:28:37.019806 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2fe3990a-96b5-4186-a646-ffc72cee7c57-kube-api-access-67g6g" (OuterVolumeSpecName: "kube-api-access-67g6g") pod "2fe3990a-96b5-4186-a646-ffc72cee7c57" (UID: "2fe3990a-96b5-4186-a646-ffc72cee7c57"). InnerVolumeSpecName "kube-api-access-67g6g". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:28:37 crc kubenswrapper[5052]: I1203 12:28:37.022857 5052 generic.go:334] "Generic (PLEG): container finished" podID="18dae0fe-fcde-4a2e-bfd5-3c4ee6976894" containerID="bf3135f41d57b4319b491c492f40b5262bdbbb4439ae6f42157eab9d6b44e24e" exitCode=143 Dec 03 12:28:37 crc kubenswrapper[5052]: I1203 12:28:37.022919 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"18dae0fe-fcde-4a2e-bfd5-3c4ee6976894","Type":"ContainerDied","Data":"bf3135f41d57b4319b491c492f40b5262bdbbb4439ae6f42157eab9d6b44e24e"} Dec 03 12:28:37 crc kubenswrapper[5052]: I1203 12:28:37.025025 5052 generic.go:334] "Generic (PLEG): container finished" podID="d808ef0b-751e-4576-befc-e23a06fe2ff0" containerID="9b7bab21260f028f58b297b4ef87a5852a2f31508dfedff95c3c79316a694a8f" exitCode=143 Dec 03 12:28:37 crc kubenswrapper[5052]: I1203 12:28:37.025050 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"d808ef0b-751e-4576-befc-e23a06fe2ff0","Type":"ContainerDied","Data":"9b7bab21260f028f58b297b4ef87a5852a2f31508dfedff95c3c79316a694a8f"} Dec 03 12:28:37 crc kubenswrapper[5052]: I1203 12:28:37.043047 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2fe3990a-96b5-4186-a646-ffc72cee7c57-config-data" (OuterVolumeSpecName: "config-data") pod "2fe3990a-96b5-4186-a646-ffc72cee7c57" (UID: "2fe3990a-96b5-4186-a646-ffc72cee7c57"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:28:37 crc kubenswrapper[5052]: I1203 12:28:37.062909 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2fe3990a-96b5-4186-a646-ffc72cee7c57-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "2fe3990a-96b5-4186-a646-ffc72cee7c57" (UID: "2fe3990a-96b5-4186-a646-ffc72cee7c57"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:28:37 crc kubenswrapper[5052]: I1203 12:28:37.097693 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/2fe3990a-96b5-4186-a646-ffc72cee7c57-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:37 crc kubenswrapper[5052]: I1203 12:28:37.097729 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-67g6g\" (UniqueName: \"kubernetes.io/projected/2fe3990a-96b5-4186-a646-ffc72cee7c57-kube-api-access-67g6g\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:37 crc kubenswrapper[5052]: I1203 12:28:37.097740 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2fe3990a-96b5-4186-a646-ffc72cee7c57-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:37 crc kubenswrapper[5052]: I1203 12:28:37.364932 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:28:37 crc kubenswrapper[5052]: I1203 12:28:37.380316 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:28:37 crc kubenswrapper[5052]: I1203 12:28:37.394402 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:28:37 crc kubenswrapper[5052]: E1203 12:28:37.394869 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="51d1c5f3-0622-4c29-a19a-f1ecd293654e" containerName="nova-manage" Dec 03 12:28:37 crc kubenswrapper[5052]: I1203 12:28:37.394886 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="51d1c5f3-0622-4c29-a19a-f1ecd293654e" containerName="nova-manage" Dec 03 12:28:37 crc kubenswrapper[5052]: E1203 12:28:37.394898 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="c03ce333-5bdc-4cdb-9a81-53db1322b7bf" containerName="extract-utilities" Dec 03 12:28:37 crc kubenswrapper[5052]: I1203 12:28:37.394904 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="c03ce333-5bdc-4cdb-9a81-53db1322b7bf" containerName="extract-utilities" Dec 03 12:28:37 crc kubenswrapper[5052]: E1203 12:28:37.394926 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="2fe3990a-96b5-4186-a646-ffc72cee7c57" containerName="nova-scheduler-scheduler" Dec 03 12:28:37 crc kubenswrapper[5052]: I1203 12:28:37.394932 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="2fe3990a-96b5-4186-a646-ffc72cee7c57" containerName="nova-scheduler-scheduler" Dec 03 12:28:37 crc kubenswrapper[5052]: E1203 12:28:37.394948 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="c03ce333-5bdc-4cdb-9a81-53db1322b7bf" containerName="registry-server" Dec 03 12:28:37 crc kubenswrapper[5052]: I1203 12:28:37.394953 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="c03ce333-5bdc-4cdb-9a81-53db1322b7bf" containerName="registry-server" Dec 03 12:28:37 crc kubenswrapper[5052]: E1203 12:28:37.394966 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="2fe3990a-96b5-4186-a646-ffc72cee7c57" containerName="nova-scheduler-scheduler" Dec 03 12:28:37 crc kubenswrapper[5052]: I1203 12:28:37.394972 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="2fe3990a-96b5-4186-a646-ffc72cee7c57" containerName="nova-scheduler-scheduler" Dec 03 12:28:37 crc kubenswrapper[5052]: E1203 12:28:37.394983 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="c03ce333-5bdc-4cdb-9a81-53db1322b7bf" containerName="extract-content" Dec 03 12:28:37 crc kubenswrapper[5052]: I1203 12:28:37.394989 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="c03ce333-5bdc-4cdb-9a81-53db1322b7bf" containerName="extract-content" Dec 03 12:28:37 crc kubenswrapper[5052]: E1203 12:28:37.395004 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="e2d1b100-5918-4166-9fa1-5a094612727f" containerName="nova-manage" Dec 03 12:28:37 crc kubenswrapper[5052]: I1203 12:28:37.395010 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="e2d1b100-5918-4166-9fa1-5a094612727f" containerName="nova-manage" Dec 03 12:28:37 crc kubenswrapper[5052]: I1203 12:28:37.395178 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="51d1c5f3-0622-4c29-a19a-f1ecd293654e" containerName="nova-manage" Dec 03 12:28:37 crc kubenswrapper[5052]: I1203 12:28:37.395188 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="e2d1b100-5918-4166-9fa1-5a094612727f" containerName="nova-manage" Dec 03 12:28:37 crc kubenswrapper[5052]: I1203 12:28:37.395205 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="c03ce333-5bdc-4cdb-9a81-53db1322b7bf" containerName="registry-server" Dec 03 12:28:37 crc kubenswrapper[5052]: I1203 12:28:37.395213 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="2fe3990a-96b5-4186-a646-ffc72cee7c57" containerName="nova-scheduler-scheduler" Dec 03 12:28:37 crc kubenswrapper[5052]: I1203 12:28:37.395225 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="2fe3990a-96b5-4186-a646-ffc72cee7c57" containerName="nova-scheduler-scheduler" Dec 03 12:28:37 crc kubenswrapper[5052]: I1203 12:28:37.395906 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:28:37 crc kubenswrapper[5052]: I1203 12:28:37.398821 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-scheduler-config-data" Dec 03 12:28:37 crc kubenswrapper[5052]: I1203 12:28:37.420574 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:28:37 crc kubenswrapper[5052]: I1203 12:28:37.504198 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b7c664ff-377a-4d83-a65e-6ecce5b9c9c0-combined-ca-bundle\") pod \"nova-scheduler-0\" (UID: \"b7c664ff-377a-4d83-a65e-6ecce5b9c9c0\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:28:37 crc kubenswrapper[5052]: I1203 12:28:37.504320 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c4v8f\" (UniqueName: \"kubernetes.io/projected/b7c664ff-377a-4d83-a65e-6ecce5b9c9c0-kube-api-access-c4v8f\") pod \"nova-scheduler-0\" (UID: \"b7c664ff-377a-4d83-a65e-6ecce5b9c9c0\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:28:37 crc kubenswrapper[5052]: I1203 12:28:37.504765 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/b7c664ff-377a-4d83-a65e-6ecce5b9c9c0-config-data\") pod \"nova-scheduler-0\" (UID: \"b7c664ff-377a-4d83-a65e-6ecce5b9c9c0\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:28:37 crc kubenswrapper[5052]: I1203 12:28:37.606969 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-c4v8f\" (UniqueName: \"kubernetes.io/projected/b7c664ff-377a-4d83-a65e-6ecce5b9c9c0-kube-api-access-c4v8f\") pod \"nova-scheduler-0\" (UID: \"b7c664ff-377a-4d83-a65e-6ecce5b9c9c0\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:28:37 crc kubenswrapper[5052]: I1203 12:28:37.607181 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/b7c664ff-377a-4d83-a65e-6ecce5b9c9c0-config-data\") pod \"nova-scheduler-0\" (UID: \"b7c664ff-377a-4d83-a65e-6ecce5b9c9c0\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:28:37 crc kubenswrapper[5052]: I1203 12:28:37.607276 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b7c664ff-377a-4d83-a65e-6ecce5b9c9c0-combined-ca-bundle\") pod \"nova-scheduler-0\" (UID: \"b7c664ff-377a-4d83-a65e-6ecce5b9c9c0\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:28:37 crc kubenswrapper[5052]: I1203 12:28:37.610910 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/b7c664ff-377a-4d83-a65e-6ecce5b9c9c0-config-data\") pod \"nova-scheduler-0\" (UID: \"b7c664ff-377a-4d83-a65e-6ecce5b9c9c0\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:28:37 crc kubenswrapper[5052]: I1203 12:28:37.611267 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b7c664ff-377a-4d83-a65e-6ecce5b9c9c0-combined-ca-bundle\") pod \"nova-scheduler-0\" (UID: \"b7c664ff-377a-4d83-a65e-6ecce5b9c9c0\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:28:37 crc kubenswrapper[5052]: I1203 12:28:37.623686 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-c4v8f\" (UniqueName: \"kubernetes.io/projected/b7c664ff-377a-4d83-a65e-6ecce5b9c9c0-kube-api-access-c4v8f\") pod \"nova-scheduler-0\" (UID: \"b7c664ff-377a-4d83-a65e-6ecce5b9c9c0\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:28:37 crc kubenswrapper[5052]: I1203 12:28:37.683143 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2fe3990a-96b5-4186-a646-ffc72cee7c57" path="/var/lib/kubelet/pods/2fe3990a-96b5-4186-a646-ffc72cee7c57/volumes" Dec 03 12:28:37 crc kubenswrapper[5052]: I1203 12:28:37.732760 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:28:38 crc kubenswrapper[5052]: I1203 12:28:38.267323 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:28:38 crc kubenswrapper[5052]: W1203 12:28:38.293323 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb7c664ff_377a_4d83_a65e_6ecce5b9c9c0.slice/crio-3de0218af395e15d0f4d62000548093e62171a5b7c0f818e75375b1db56420aa WatchSource:0}: Error finding container 3de0218af395e15d0f4d62000548093e62171a5b7c0f818e75375b1db56420aa: Status 404 returned error can't find the container with id 3de0218af395e15d0f4d62000548093e62171a5b7c0f818e75375b1db56420aa Dec 03 12:28:39 crc kubenswrapper[5052]: I1203 12:28:39.084155 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"b7c664ff-377a-4d83-a65e-6ecce5b9c9c0","Type":"ContainerStarted","Data":"316d2d796b1bcce26d30624eb3760ad0fa3839c99e679788444235cfcf404776"} Dec 03 12:28:39 crc kubenswrapper[5052]: I1203 12:28:39.084497 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"b7c664ff-377a-4d83-a65e-6ecce5b9c9c0","Type":"ContainerStarted","Data":"3de0218af395e15d0f4d62000548093e62171a5b7c0f818e75375b1db56420aa"} Dec 03 12:28:39 crc kubenswrapper[5052]: I1203 12:28:39.107253 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-scheduler-0" podStartSLOduration=2.107231775 podStartE2EDuration="2.107231775s" podCreationTimestamp="2025-12-03 12:28:37 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:28:39.100460072 +0000 UTC m=+4109.688701856" watchObservedRunningTime="2025-12-03 12:28:39.107231775 +0000 UTC m=+4109.695473539" Dec 03 12:28:39 crc kubenswrapper[5052]: I1203 12:28:39.536627 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-f699c657d-qh2qk" Dec 03 12:28:39 crc kubenswrapper[5052]: I1203 12:28:39.651854 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a28346c0-bd60-4dac-ac86-20b56296e7d3-combined-ca-bundle\") pod \"a28346c0-bd60-4dac-ac86-20b56296e7d3\" (UID: \"a28346c0-bd60-4dac-ac86-20b56296e7d3\") " Dec 03 12:28:39 crc kubenswrapper[5052]: I1203 12:28:39.652340 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tpmbq\" (UniqueName: \"kubernetes.io/projected/a28346c0-bd60-4dac-ac86-20b56296e7d3-kube-api-access-tpmbq\") pod \"a28346c0-bd60-4dac-ac86-20b56296e7d3\" (UID: \"a28346c0-bd60-4dac-ac86-20b56296e7d3\") " Dec 03 12:28:39 crc kubenswrapper[5052]: I1203 12:28:39.652425 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/a28346c0-bd60-4dac-ac86-20b56296e7d3-httpd-config\") pod \"a28346c0-bd60-4dac-ac86-20b56296e7d3\" (UID: \"a28346c0-bd60-4dac-ac86-20b56296e7d3\") " Dec 03 12:28:39 crc kubenswrapper[5052]: I1203 12:28:39.652647 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/a28346c0-bd60-4dac-ac86-20b56296e7d3-config\") pod \"a28346c0-bd60-4dac-ac86-20b56296e7d3\" (UID: \"a28346c0-bd60-4dac-ac86-20b56296e7d3\") " Dec 03 12:28:39 crc kubenswrapper[5052]: I1203 12:28:39.660370 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a28346c0-bd60-4dac-ac86-20b56296e7d3-httpd-config" (OuterVolumeSpecName: "httpd-config") pod "a28346c0-bd60-4dac-ac86-20b56296e7d3" (UID: "a28346c0-bd60-4dac-ac86-20b56296e7d3"). InnerVolumeSpecName "httpd-config". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:28:39 crc kubenswrapper[5052]: I1203 12:28:39.660452 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a28346c0-bd60-4dac-ac86-20b56296e7d3-kube-api-access-tpmbq" (OuterVolumeSpecName: "kube-api-access-tpmbq") pod "a28346c0-bd60-4dac-ac86-20b56296e7d3" (UID: "a28346c0-bd60-4dac-ac86-20b56296e7d3"). InnerVolumeSpecName "kube-api-access-tpmbq". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:28:39 crc kubenswrapper[5052]: I1203 12:28:39.727766 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a28346c0-bd60-4dac-ac86-20b56296e7d3-config" (OuterVolumeSpecName: "config") pod "a28346c0-bd60-4dac-ac86-20b56296e7d3" (UID: "a28346c0-bd60-4dac-ac86-20b56296e7d3"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:28:39 crc kubenswrapper[5052]: I1203 12:28:39.736696 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a28346c0-bd60-4dac-ac86-20b56296e7d3-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "a28346c0-bd60-4dac-ac86-20b56296e7d3" (UID: "a28346c0-bd60-4dac-ac86-20b56296e7d3"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:28:39 crc kubenswrapper[5052]: I1203 12:28:39.756917 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a28346c0-bd60-4dac-ac86-20b56296e7d3-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:39 crc kubenswrapper[5052]: I1203 12:28:39.757373 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-tpmbq\" (UniqueName: \"kubernetes.io/projected/a28346c0-bd60-4dac-ac86-20b56296e7d3-kube-api-access-tpmbq\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:39 crc kubenswrapper[5052]: I1203 12:28:39.757781 5052 reconciler_common.go:293] "Volume detached for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/a28346c0-bd60-4dac-ac86-20b56296e7d3-httpd-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:39 crc kubenswrapper[5052]: I1203 12:28:39.757913 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/a28346c0-bd60-4dac-ac86-20b56296e7d3-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:40 crc kubenswrapper[5052]: I1203 12:28:40.094424 5052 generic.go:334] "Generic (PLEG): container finished" podID="18dae0fe-fcde-4a2e-bfd5-3c4ee6976894" containerID="486f511e2e18cfc8ea29b34eeca83447c73622373f7fc22a86772a5bb4c258ab" exitCode=0 Dec 03 12:28:40 crc kubenswrapper[5052]: I1203 12:28:40.094696 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"18dae0fe-fcde-4a2e-bfd5-3c4ee6976894","Type":"ContainerDied","Data":"486f511e2e18cfc8ea29b34eeca83447c73622373f7fc22a86772a5bb4c258ab"} Dec 03 12:28:40 crc kubenswrapper[5052]: I1203 12:28:40.094725 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"18dae0fe-fcde-4a2e-bfd5-3c4ee6976894","Type":"ContainerDied","Data":"99dbf56ce8d246aecdf81aeda0179117dfbc80314e99724ce9f7f2fa4b34dad1"} Dec 03 12:28:40 crc kubenswrapper[5052]: I1203 12:28:40.094735 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="99dbf56ce8d246aecdf81aeda0179117dfbc80314e99724ce9f7f2fa4b34dad1" Dec 03 12:28:40 crc kubenswrapper[5052]: I1203 12:28:40.096495 5052 generic.go:334] "Generic (PLEG): container finished" podID="d808ef0b-751e-4576-befc-e23a06fe2ff0" containerID="bc91adc346315718fd765cd7a7a58ccc06d83103b40be9d46a7d2e14981369da" exitCode=0 Dec 03 12:28:40 crc kubenswrapper[5052]: I1203 12:28:40.096552 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"d808ef0b-751e-4576-befc-e23a06fe2ff0","Type":"ContainerDied","Data":"bc91adc346315718fd765cd7a7a58ccc06d83103b40be9d46a7d2e14981369da"} Dec 03 12:28:40 crc kubenswrapper[5052]: I1203 12:28:40.096569 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"d808ef0b-751e-4576-befc-e23a06fe2ff0","Type":"ContainerDied","Data":"cee8f16e8e1679b820f59ea004534fd031076cdb92642915cc2863807a77f993"} Dec 03 12:28:40 crc kubenswrapper[5052]: I1203 12:28:40.096577 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cee8f16e8e1679b820f59ea004534fd031076cdb92642915cc2863807a77f993" Dec 03 12:28:40 crc kubenswrapper[5052]: I1203 12:28:40.099643 5052 generic.go:334] "Generic (PLEG): container finished" podID="a28346c0-bd60-4dac-ac86-20b56296e7d3" containerID="c55b9fbfdc46a4cb2612d4910e298c39d926f0f567206d0d4eb4cba1b2cf66ed" exitCode=0 Dec 03 12:28:40 crc kubenswrapper[5052]: I1203 12:28:40.099727 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-f699c657d-qh2qk" Dec 03 12:28:40 crc kubenswrapper[5052]: I1203 12:28:40.099788 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-f699c657d-qh2qk" event={"ID":"a28346c0-bd60-4dac-ac86-20b56296e7d3","Type":"ContainerDied","Data":"c55b9fbfdc46a4cb2612d4910e298c39d926f0f567206d0d4eb4cba1b2cf66ed"} Dec 03 12:28:40 crc kubenswrapper[5052]: I1203 12:28:40.099823 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-f699c657d-qh2qk" event={"ID":"a28346c0-bd60-4dac-ac86-20b56296e7d3","Type":"ContainerDied","Data":"164683c6068bdd4d22a79ff8eadc4cb11d9a95ca3e9f04852bf04d50eb2c5e81"} Dec 03 12:28:40 crc kubenswrapper[5052]: I1203 12:28:40.099840 5052 scope.go:117] "RemoveContainer" containerID="d10711abf49e5a87c80edf05ade50ea33f8c777fb550c91e299195151230bed0" Dec 03 12:28:40 crc kubenswrapper[5052]: I1203 12:28:40.125808 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:28:40 crc kubenswrapper[5052]: I1203 12:28:40.144457 5052 scope.go:117] "RemoveContainer" containerID="c55b9fbfdc46a4cb2612d4910e298c39d926f0f567206d0d4eb4cba1b2cf66ed" Dec 03 12:28:40 crc kubenswrapper[5052]: I1203 12:28:40.160566 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:28:40 crc kubenswrapper[5052]: I1203 12:28:40.164649 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/neutron-f699c657d-qh2qk"] Dec 03 12:28:40 crc kubenswrapper[5052]: I1203 12:28:40.195602 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d808ef0b-751e-4576-befc-e23a06fe2ff0-combined-ca-bundle\") pod \"d808ef0b-751e-4576-befc-e23a06fe2ff0\" (UID: \"d808ef0b-751e-4576-befc-e23a06fe2ff0\") " Dec 03 12:28:40 crc kubenswrapper[5052]: I1203 12:28:40.195707 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/d808ef0b-751e-4576-befc-e23a06fe2ff0-config-data\") pod \"d808ef0b-751e-4576-befc-e23a06fe2ff0\" (UID: \"d808ef0b-751e-4576-befc-e23a06fe2ff0\") " Dec 03 12:28:40 crc kubenswrapper[5052]: I1203 12:28:40.195755 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/d808ef0b-751e-4576-befc-e23a06fe2ff0-public-tls-certs\") pod \"d808ef0b-751e-4576-befc-e23a06fe2ff0\" (UID: \"d808ef0b-751e-4576-befc-e23a06fe2ff0\") " Dec 03 12:28:40 crc kubenswrapper[5052]: I1203 12:28:40.195789 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/d808ef0b-751e-4576-befc-e23a06fe2ff0-logs\") pod \"d808ef0b-751e-4576-befc-e23a06fe2ff0\" (UID: \"d808ef0b-751e-4576-befc-e23a06fe2ff0\") " Dec 03 12:28:40 crc kubenswrapper[5052]: I1203 12:28:40.195855 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/d808ef0b-751e-4576-befc-e23a06fe2ff0-internal-tls-certs\") pod \"d808ef0b-751e-4576-befc-e23a06fe2ff0\" (UID: \"d808ef0b-751e-4576-befc-e23a06fe2ff0\") " Dec 03 12:28:40 crc kubenswrapper[5052]: I1203 12:28:40.195879 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6hhvl\" (UniqueName: \"kubernetes.io/projected/d808ef0b-751e-4576-befc-e23a06fe2ff0-kube-api-access-6hhvl\") pod \"d808ef0b-751e-4576-befc-e23a06fe2ff0\" (UID: \"d808ef0b-751e-4576-befc-e23a06fe2ff0\") " Dec 03 12:28:40 crc kubenswrapper[5052]: I1203 12:28:40.197221 5052 scope.go:117] "RemoveContainer" containerID="d10711abf49e5a87c80edf05ade50ea33f8c777fb550c91e299195151230bed0" Dec 03 12:28:40 crc kubenswrapper[5052]: I1203 12:28:40.198008 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d808ef0b-751e-4576-befc-e23a06fe2ff0-logs" (OuterVolumeSpecName: "logs") pod "d808ef0b-751e-4576-befc-e23a06fe2ff0" (UID: "d808ef0b-751e-4576-befc-e23a06fe2ff0"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:28:40 crc kubenswrapper[5052]: E1203 12:28:40.198022 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d10711abf49e5a87c80edf05ade50ea33f8c777fb550c91e299195151230bed0\": container with ID starting with d10711abf49e5a87c80edf05ade50ea33f8c777fb550c91e299195151230bed0 not found: ID does not exist" containerID="d10711abf49e5a87c80edf05ade50ea33f8c777fb550c91e299195151230bed0" Dec 03 12:28:40 crc kubenswrapper[5052]: I1203 12:28:40.198091 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d10711abf49e5a87c80edf05ade50ea33f8c777fb550c91e299195151230bed0"} err="failed to get container status \"d10711abf49e5a87c80edf05ade50ea33f8c777fb550c91e299195151230bed0\": rpc error: code = NotFound desc = could not find container \"d10711abf49e5a87c80edf05ade50ea33f8c777fb550c91e299195151230bed0\": container with ID starting with d10711abf49e5a87c80edf05ade50ea33f8c777fb550c91e299195151230bed0 not found: ID does not exist" Dec 03 12:28:40 crc kubenswrapper[5052]: I1203 12:28:40.198122 5052 scope.go:117] "RemoveContainer" containerID="c55b9fbfdc46a4cb2612d4910e298c39d926f0f567206d0d4eb4cba1b2cf66ed" Dec 03 12:28:40 crc kubenswrapper[5052]: E1203 12:28:40.200292 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c55b9fbfdc46a4cb2612d4910e298c39d926f0f567206d0d4eb4cba1b2cf66ed\": container with ID starting with c55b9fbfdc46a4cb2612d4910e298c39d926f0f567206d0d4eb4cba1b2cf66ed not found: ID does not exist" containerID="c55b9fbfdc46a4cb2612d4910e298c39d926f0f567206d0d4eb4cba1b2cf66ed" Dec 03 12:28:40 crc kubenswrapper[5052]: I1203 12:28:40.200344 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c55b9fbfdc46a4cb2612d4910e298c39d926f0f567206d0d4eb4cba1b2cf66ed"} err="failed to get container status \"c55b9fbfdc46a4cb2612d4910e298c39d926f0f567206d0d4eb4cba1b2cf66ed\": rpc error: code = NotFound desc = could not find container \"c55b9fbfdc46a4cb2612d4910e298c39d926f0f567206d0d4eb4cba1b2cf66ed\": container with ID starting with c55b9fbfdc46a4cb2612d4910e298c39d926f0f567206d0d4eb4cba1b2cf66ed not found: ID does not exist" Dec 03 12:28:40 crc kubenswrapper[5052]: I1203 12:28:40.202365 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/d808ef0b-751e-4576-befc-e23a06fe2ff0-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:40 crc kubenswrapper[5052]: I1203 12:28:40.205487 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/neutron-f699c657d-qh2qk"] Dec 03 12:28:40 crc kubenswrapper[5052]: I1203 12:28:40.215137 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d808ef0b-751e-4576-befc-e23a06fe2ff0-kube-api-access-6hhvl" (OuterVolumeSpecName: "kube-api-access-6hhvl") pod "d808ef0b-751e-4576-befc-e23a06fe2ff0" (UID: "d808ef0b-751e-4576-befc-e23a06fe2ff0"). InnerVolumeSpecName "kube-api-access-6hhvl". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:28:40 crc kubenswrapper[5052]: I1203 12:28:40.233677 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d808ef0b-751e-4576-befc-e23a06fe2ff0-config-data" (OuterVolumeSpecName: "config-data") pod "d808ef0b-751e-4576-befc-e23a06fe2ff0" (UID: "d808ef0b-751e-4576-befc-e23a06fe2ff0"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:28:40 crc kubenswrapper[5052]: I1203 12:28:40.253349 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d808ef0b-751e-4576-befc-e23a06fe2ff0-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "d808ef0b-751e-4576-befc-e23a06fe2ff0" (UID: "d808ef0b-751e-4576-befc-e23a06fe2ff0"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:28:40 crc kubenswrapper[5052]: I1203 12:28:40.255812 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d808ef0b-751e-4576-befc-e23a06fe2ff0-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "d808ef0b-751e-4576-befc-e23a06fe2ff0" (UID: "d808ef0b-751e-4576-befc-e23a06fe2ff0"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:28:40 crc kubenswrapper[5052]: I1203 12:28:40.261951 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d808ef0b-751e-4576-befc-e23a06fe2ff0-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "d808ef0b-751e-4576-befc-e23a06fe2ff0" (UID: "d808ef0b-751e-4576-befc-e23a06fe2ff0"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:28:40 crc kubenswrapper[5052]: I1203 12:28:40.303545 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/18dae0fe-fcde-4a2e-bfd5-3c4ee6976894-nova-metadata-tls-certs\") pod \"18dae0fe-fcde-4a2e-bfd5-3c4ee6976894\" (UID: \"18dae0fe-fcde-4a2e-bfd5-3c4ee6976894\") " Dec 03 12:28:40 crc kubenswrapper[5052]: I1203 12:28:40.303593 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/18dae0fe-fcde-4a2e-bfd5-3c4ee6976894-config-data\") pod \"18dae0fe-fcde-4a2e-bfd5-3c4ee6976894\" (UID: \"18dae0fe-fcde-4a2e-bfd5-3c4ee6976894\") " Dec 03 12:28:40 crc kubenswrapper[5052]: I1203 12:28:40.303869 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/18dae0fe-fcde-4a2e-bfd5-3c4ee6976894-combined-ca-bundle\") pod \"18dae0fe-fcde-4a2e-bfd5-3c4ee6976894\" (UID: \"18dae0fe-fcde-4a2e-bfd5-3c4ee6976894\") " Dec 03 12:28:40 crc kubenswrapper[5052]: I1203 12:28:40.303949 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/18dae0fe-fcde-4a2e-bfd5-3c4ee6976894-logs\") pod \"18dae0fe-fcde-4a2e-bfd5-3c4ee6976894\" (UID: \"18dae0fe-fcde-4a2e-bfd5-3c4ee6976894\") " Dec 03 12:28:40 crc kubenswrapper[5052]: I1203 12:28:40.304035 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bn57m\" (UniqueName: \"kubernetes.io/projected/18dae0fe-fcde-4a2e-bfd5-3c4ee6976894-kube-api-access-bn57m\") pod \"18dae0fe-fcde-4a2e-bfd5-3c4ee6976894\" (UID: \"18dae0fe-fcde-4a2e-bfd5-3c4ee6976894\") " Dec 03 12:28:40 crc kubenswrapper[5052]: I1203 12:28:40.304410 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/18dae0fe-fcde-4a2e-bfd5-3c4ee6976894-logs" (OuterVolumeSpecName: "logs") pod "18dae0fe-fcde-4a2e-bfd5-3c4ee6976894" (UID: "18dae0fe-fcde-4a2e-bfd5-3c4ee6976894"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:28:40 crc kubenswrapper[5052]: I1203 12:28:40.304834 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d808ef0b-751e-4576-befc-e23a06fe2ff0-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:40 crc kubenswrapper[5052]: I1203 12:28:40.304860 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/d808ef0b-751e-4576-befc-e23a06fe2ff0-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:40 crc kubenswrapper[5052]: I1203 12:28:40.304872 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/18dae0fe-fcde-4a2e-bfd5-3c4ee6976894-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:40 crc kubenswrapper[5052]: I1203 12:28:40.304883 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/d808ef0b-751e-4576-befc-e23a06fe2ff0-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:40 crc kubenswrapper[5052]: I1203 12:28:40.304896 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/d808ef0b-751e-4576-befc-e23a06fe2ff0-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:40 crc kubenswrapper[5052]: I1203 12:28:40.304909 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-6hhvl\" (UniqueName: \"kubernetes.io/projected/d808ef0b-751e-4576-befc-e23a06fe2ff0-kube-api-access-6hhvl\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:40 crc kubenswrapper[5052]: I1203 12:28:40.307575 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/18dae0fe-fcde-4a2e-bfd5-3c4ee6976894-kube-api-access-bn57m" (OuterVolumeSpecName: "kube-api-access-bn57m") pod "18dae0fe-fcde-4a2e-bfd5-3c4ee6976894" (UID: "18dae0fe-fcde-4a2e-bfd5-3c4ee6976894"). InnerVolumeSpecName "kube-api-access-bn57m". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:28:40 crc kubenswrapper[5052]: I1203 12:28:40.335038 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/18dae0fe-fcde-4a2e-bfd5-3c4ee6976894-config-data" (OuterVolumeSpecName: "config-data") pod "18dae0fe-fcde-4a2e-bfd5-3c4ee6976894" (UID: "18dae0fe-fcde-4a2e-bfd5-3c4ee6976894"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:28:40 crc kubenswrapper[5052]: I1203 12:28:40.335857 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/18dae0fe-fcde-4a2e-bfd5-3c4ee6976894-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "18dae0fe-fcde-4a2e-bfd5-3c4ee6976894" (UID: "18dae0fe-fcde-4a2e-bfd5-3c4ee6976894"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:28:40 crc kubenswrapper[5052]: I1203 12:28:40.362160 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/18dae0fe-fcde-4a2e-bfd5-3c4ee6976894-nova-metadata-tls-certs" (OuterVolumeSpecName: "nova-metadata-tls-certs") pod "18dae0fe-fcde-4a2e-bfd5-3c4ee6976894" (UID: "18dae0fe-fcde-4a2e-bfd5-3c4ee6976894"). InnerVolumeSpecName "nova-metadata-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:28:40 crc kubenswrapper[5052]: I1203 12:28:40.406758 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-bn57m\" (UniqueName: \"kubernetes.io/projected/18dae0fe-fcde-4a2e-bfd5-3c4ee6976894-kube-api-access-bn57m\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:40 crc kubenswrapper[5052]: I1203 12:28:40.406804 5052 reconciler_common.go:293] "Volume detached for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/18dae0fe-fcde-4a2e-bfd5-3c4ee6976894-nova-metadata-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:40 crc kubenswrapper[5052]: I1203 12:28:40.406817 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/18dae0fe-fcde-4a2e-bfd5-3c4ee6976894-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:40 crc kubenswrapper[5052]: I1203 12:28:40.406830 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/18dae0fe-fcde-4a2e-bfd5-3c4ee6976894-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:41 crc kubenswrapper[5052]: I1203 12:28:41.113402 5052 generic.go:334] "Generic (PLEG): container finished" podID="b7c664ff-377a-4d83-a65e-6ecce5b9c9c0" containerID="316d2d796b1bcce26d30624eb3760ad0fa3839c99e679788444235cfcf404776" exitCode=1 Dec 03 12:28:41 crc kubenswrapper[5052]: I1203 12:28:41.113501 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"b7c664ff-377a-4d83-a65e-6ecce5b9c9c0","Type":"ContainerDied","Data":"316d2d796b1bcce26d30624eb3760ad0fa3839c99e679788444235cfcf404776"} Dec 03 12:28:41 crc kubenswrapper[5052]: I1203 12:28:41.114247 5052 scope.go:117] "RemoveContainer" containerID="316d2d796b1bcce26d30624eb3760ad0fa3839c99e679788444235cfcf404776" Dec 03 12:28:41 crc kubenswrapper[5052]: I1203 12:28:41.119343 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:28:41 crc kubenswrapper[5052]: I1203 12:28:41.125684 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:28:41 crc kubenswrapper[5052]: I1203 12:28:41.173700 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:28:41 crc kubenswrapper[5052]: I1203 12:28:41.196568 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:28:41 crc kubenswrapper[5052]: I1203 12:28:41.227583 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:28:41 crc kubenswrapper[5052]: I1203 12:28:41.251063 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:28:41 crc kubenswrapper[5052]: E1203 12:28:41.251495 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="a28346c0-bd60-4dac-ac86-20b56296e7d3" containerName="neutron-httpd" Dec 03 12:28:41 crc kubenswrapper[5052]: I1203 12:28:41.251531 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="a28346c0-bd60-4dac-ac86-20b56296e7d3" containerName="neutron-httpd" Dec 03 12:28:41 crc kubenswrapper[5052]: E1203 12:28:41.251551 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="18dae0fe-fcde-4a2e-bfd5-3c4ee6976894" containerName="nova-metadata-log" Dec 03 12:28:41 crc kubenswrapper[5052]: I1203 12:28:41.251558 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="18dae0fe-fcde-4a2e-bfd5-3c4ee6976894" containerName="nova-metadata-log" Dec 03 12:28:41 crc kubenswrapper[5052]: E1203 12:28:41.251580 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="18dae0fe-fcde-4a2e-bfd5-3c4ee6976894" containerName="nova-metadata-metadata" Dec 03 12:28:41 crc kubenswrapper[5052]: I1203 12:28:41.251588 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="18dae0fe-fcde-4a2e-bfd5-3c4ee6976894" containerName="nova-metadata-metadata" Dec 03 12:28:41 crc kubenswrapper[5052]: E1203 12:28:41.251616 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="d808ef0b-751e-4576-befc-e23a06fe2ff0" containerName="nova-api-api" Dec 03 12:28:41 crc kubenswrapper[5052]: I1203 12:28:41.251624 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="d808ef0b-751e-4576-befc-e23a06fe2ff0" containerName="nova-api-api" Dec 03 12:28:41 crc kubenswrapper[5052]: E1203 12:28:41.251645 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="a28346c0-bd60-4dac-ac86-20b56296e7d3" containerName="neutron-api" Dec 03 12:28:41 crc kubenswrapper[5052]: I1203 12:28:41.251652 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="a28346c0-bd60-4dac-ac86-20b56296e7d3" containerName="neutron-api" Dec 03 12:28:41 crc kubenswrapper[5052]: E1203 12:28:41.251825 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="d808ef0b-751e-4576-befc-e23a06fe2ff0" containerName="nova-api-log" Dec 03 12:28:41 crc kubenswrapper[5052]: I1203 12:28:41.251832 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="d808ef0b-751e-4576-befc-e23a06fe2ff0" containerName="nova-api-log" Dec 03 12:28:41 crc kubenswrapper[5052]: I1203 12:28:41.252046 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="d808ef0b-751e-4576-befc-e23a06fe2ff0" containerName="nova-api-log" Dec 03 12:28:41 crc kubenswrapper[5052]: I1203 12:28:41.252063 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="18dae0fe-fcde-4a2e-bfd5-3c4ee6976894" containerName="nova-metadata-metadata" Dec 03 12:28:41 crc kubenswrapper[5052]: I1203 12:28:41.252080 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="a28346c0-bd60-4dac-ac86-20b56296e7d3" containerName="neutron-httpd" Dec 03 12:28:41 crc kubenswrapper[5052]: I1203 12:28:41.252092 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="d808ef0b-751e-4576-befc-e23a06fe2ff0" containerName="nova-api-api" Dec 03 12:28:41 crc kubenswrapper[5052]: I1203 12:28:41.252106 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="18dae0fe-fcde-4a2e-bfd5-3c4ee6976894" containerName="nova-metadata-log" Dec 03 12:28:41 crc kubenswrapper[5052]: I1203 12:28:41.252124 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="a28346c0-bd60-4dac-ac86-20b56296e7d3" containerName="neutron-api" Dec 03 12:28:41 crc kubenswrapper[5052]: I1203 12:28:41.254598 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:28:41 crc kubenswrapper[5052]: I1203 12:28:41.261038 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-metadata-config-data" Dec 03 12:28:41 crc kubenswrapper[5052]: I1203 12:28:41.264855 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-nova-metadata-internal-svc" Dec 03 12:28:41 crc kubenswrapper[5052]: I1203 12:28:41.275247 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:28:41 crc kubenswrapper[5052]: I1203 12:28:41.298062 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:28:41 crc kubenswrapper[5052]: I1203 12:28:41.310564 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:28:41 crc kubenswrapper[5052]: I1203 12:28:41.313209 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:28:41 crc kubenswrapper[5052]: I1203 12:28:41.315799 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:28:41 crc kubenswrapper[5052]: I1203 12:28:41.318494 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-nova-public-svc" Dec 03 12:28:41 crc kubenswrapper[5052]: I1203 12:28:41.318706 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-nova-internal-svc" Dec 03 12:28:41 crc kubenswrapper[5052]: I1203 12:28:41.318822 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-api-config-data" Dec 03 12:28:41 crc kubenswrapper[5052]: I1203 12:28:41.327723 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/ada0927b-5399-48af-8a11-ee31292b5d2a-nova-metadata-tls-certs\") pod \"nova-metadata-0\" (UID: \"ada0927b-5399-48af-8a11-ee31292b5d2a\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:28:41 crc kubenswrapper[5052]: I1203 12:28:41.327819 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/ada0927b-5399-48af-8a11-ee31292b5d2a-config-data\") pod \"nova-metadata-0\" (UID: \"ada0927b-5399-48af-8a11-ee31292b5d2a\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:28:41 crc kubenswrapper[5052]: I1203 12:28:41.327839 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ada0927b-5399-48af-8a11-ee31292b5d2a-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"ada0927b-5399-48af-8a11-ee31292b5d2a\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:28:41 crc kubenswrapper[5052]: I1203 12:28:41.327862 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-krrp4\" (UniqueName: \"kubernetes.io/projected/ada0927b-5399-48af-8a11-ee31292b5d2a-kube-api-access-krrp4\") pod \"nova-metadata-0\" (UID: \"ada0927b-5399-48af-8a11-ee31292b5d2a\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:28:41 crc kubenswrapper[5052]: I1203 12:28:41.327899 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/ada0927b-5399-48af-8a11-ee31292b5d2a-logs\") pod \"nova-metadata-0\" (UID: \"ada0927b-5399-48af-8a11-ee31292b5d2a\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:28:41 crc kubenswrapper[5052]: I1203 12:28:41.387118 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/keystone-68cf4f75d7-6pwrw" Dec 03 12:28:41 crc kubenswrapper[5052]: I1203 12:28:41.429558 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b045ac1c-0a9b-47e7-b6d7-39963ed900ca-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"b045ac1c-0a9b-47e7-b6d7-39963ed900ca\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:28:41 crc kubenswrapper[5052]: I1203 12:28:41.429618 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/b045ac1c-0a9b-47e7-b6d7-39963ed900ca-internal-tls-certs\") pod \"nova-api-0\" (UID: \"b045ac1c-0a9b-47e7-b6d7-39963ed900ca\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:28:41 crc kubenswrapper[5052]: I1203 12:28:41.429655 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/ada0927b-5399-48af-8a11-ee31292b5d2a-nova-metadata-tls-certs\") pod \"nova-metadata-0\" (UID: \"ada0927b-5399-48af-8a11-ee31292b5d2a\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:28:41 crc kubenswrapper[5052]: I1203 12:28:41.429685 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/b045ac1c-0a9b-47e7-b6d7-39963ed900ca-config-data\") pod \"nova-api-0\" (UID: \"b045ac1c-0a9b-47e7-b6d7-39963ed900ca\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:28:41 crc kubenswrapper[5052]: I1203 12:28:41.429728 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/b045ac1c-0a9b-47e7-b6d7-39963ed900ca-public-tls-certs\") pod \"nova-api-0\" (UID: \"b045ac1c-0a9b-47e7-b6d7-39963ed900ca\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:28:41 crc kubenswrapper[5052]: I1203 12:28:41.429776 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/ada0927b-5399-48af-8a11-ee31292b5d2a-config-data\") pod \"nova-metadata-0\" (UID: \"ada0927b-5399-48af-8a11-ee31292b5d2a\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:28:41 crc kubenswrapper[5052]: I1203 12:28:41.429797 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ada0927b-5399-48af-8a11-ee31292b5d2a-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"ada0927b-5399-48af-8a11-ee31292b5d2a\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:28:41 crc kubenswrapper[5052]: I1203 12:28:41.429817 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-krrp4\" (UniqueName: \"kubernetes.io/projected/ada0927b-5399-48af-8a11-ee31292b5d2a-kube-api-access-krrp4\") pod \"nova-metadata-0\" (UID: \"ada0927b-5399-48af-8a11-ee31292b5d2a\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:28:41 crc kubenswrapper[5052]: I1203 12:28:41.429853 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/ada0927b-5399-48af-8a11-ee31292b5d2a-logs\") pod \"nova-metadata-0\" (UID: \"ada0927b-5399-48af-8a11-ee31292b5d2a\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:28:41 crc kubenswrapper[5052]: I1203 12:28:41.429889 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/b045ac1c-0a9b-47e7-b6d7-39963ed900ca-logs\") pod \"nova-api-0\" (UID: \"b045ac1c-0a9b-47e7-b6d7-39963ed900ca\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:28:41 crc kubenswrapper[5052]: I1203 12:28:41.429917 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7bpj4\" (UniqueName: \"kubernetes.io/projected/b045ac1c-0a9b-47e7-b6d7-39963ed900ca-kube-api-access-7bpj4\") pod \"nova-api-0\" (UID: \"b045ac1c-0a9b-47e7-b6d7-39963ed900ca\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:28:41 crc kubenswrapper[5052]: I1203 12:28:41.439983 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/ada0927b-5399-48af-8a11-ee31292b5d2a-logs\") pod \"nova-metadata-0\" (UID: \"ada0927b-5399-48af-8a11-ee31292b5d2a\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:28:41 crc kubenswrapper[5052]: I1203 12:28:41.443119 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/keystone-64d6d88f66-7z998"] Dec 03 12:28:41 crc kubenswrapper[5052]: I1203 12:28:41.443341 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/keystone-64d6d88f66-7z998" podUID="ef332add-3219-4025-be5b-9d7048cd4a3d" containerName="keystone-api" containerID="cri-o://bc067ea70660c1847f572c9b0681b01351a86de5eb2ae47c967575bf575054ad" gracePeriod=30 Dec 03 12:28:41 crc kubenswrapper[5052]: I1203 12:28:41.445626 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/ada0927b-5399-48af-8a11-ee31292b5d2a-nova-metadata-tls-certs\") pod \"nova-metadata-0\" (UID: \"ada0927b-5399-48af-8a11-ee31292b5d2a\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:28:41 crc kubenswrapper[5052]: I1203 12:28:41.445671 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ada0927b-5399-48af-8a11-ee31292b5d2a-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"ada0927b-5399-48af-8a11-ee31292b5d2a\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:28:41 crc kubenswrapper[5052]: I1203 12:28:41.446783 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/ada0927b-5399-48af-8a11-ee31292b5d2a-config-data\") pod \"nova-metadata-0\" (UID: \"ada0927b-5399-48af-8a11-ee31292b5d2a\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:28:41 crc kubenswrapper[5052]: I1203 12:28:41.467160 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-krrp4\" (UniqueName: \"kubernetes.io/projected/ada0927b-5399-48af-8a11-ee31292b5d2a-kube-api-access-krrp4\") pod \"nova-metadata-0\" (UID: \"ada0927b-5399-48af-8a11-ee31292b5d2a\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:28:41 crc kubenswrapper[5052]: I1203 12:28:41.535269 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/b045ac1c-0a9b-47e7-b6d7-39963ed900ca-logs\") pod \"nova-api-0\" (UID: \"b045ac1c-0a9b-47e7-b6d7-39963ed900ca\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:28:41 crc kubenswrapper[5052]: I1203 12:28:41.535337 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-7bpj4\" (UniqueName: \"kubernetes.io/projected/b045ac1c-0a9b-47e7-b6d7-39963ed900ca-kube-api-access-7bpj4\") pod \"nova-api-0\" (UID: \"b045ac1c-0a9b-47e7-b6d7-39963ed900ca\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:28:41 crc kubenswrapper[5052]: I1203 12:28:41.535374 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b045ac1c-0a9b-47e7-b6d7-39963ed900ca-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"b045ac1c-0a9b-47e7-b6d7-39963ed900ca\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:28:41 crc kubenswrapper[5052]: I1203 12:28:41.535466 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/b045ac1c-0a9b-47e7-b6d7-39963ed900ca-internal-tls-certs\") pod \"nova-api-0\" (UID: \"b045ac1c-0a9b-47e7-b6d7-39963ed900ca\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:28:41 crc kubenswrapper[5052]: I1203 12:28:41.535593 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/b045ac1c-0a9b-47e7-b6d7-39963ed900ca-config-data\") pod \"nova-api-0\" (UID: \"b045ac1c-0a9b-47e7-b6d7-39963ed900ca\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:28:41 crc kubenswrapper[5052]: I1203 12:28:41.535708 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/b045ac1c-0a9b-47e7-b6d7-39963ed900ca-public-tls-certs\") pod \"nova-api-0\" (UID: \"b045ac1c-0a9b-47e7-b6d7-39963ed900ca\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:28:41 crc kubenswrapper[5052]: I1203 12:28:41.536939 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/b045ac1c-0a9b-47e7-b6d7-39963ed900ca-logs\") pod \"nova-api-0\" (UID: \"b045ac1c-0a9b-47e7-b6d7-39963ed900ca\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:28:41 crc kubenswrapper[5052]: I1203 12:28:41.540308 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/b045ac1c-0a9b-47e7-b6d7-39963ed900ca-config-data\") pod \"nova-api-0\" (UID: \"b045ac1c-0a9b-47e7-b6d7-39963ed900ca\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:28:41 crc kubenswrapper[5052]: I1203 12:28:41.540734 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/b045ac1c-0a9b-47e7-b6d7-39963ed900ca-public-tls-certs\") pod \"nova-api-0\" (UID: \"b045ac1c-0a9b-47e7-b6d7-39963ed900ca\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:28:41 crc kubenswrapper[5052]: I1203 12:28:41.548173 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/b045ac1c-0a9b-47e7-b6d7-39963ed900ca-internal-tls-certs\") pod \"nova-api-0\" (UID: \"b045ac1c-0a9b-47e7-b6d7-39963ed900ca\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:28:41 crc kubenswrapper[5052]: I1203 12:28:41.548199 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b045ac1c-0a9b-47e7-b6d7-39963ed900ca-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"b045ac1c-0a9b-47e7-b6d7-39963ed900ca\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:28:41 crc kubenswrapper[5052]: I1203 12:28:41.553343 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-7bpj4\" (UniqueName: \"kubernetes.io/projected/b045ac1c-0a9b-47e7-b6d7-39963ed900ca-kube-api-access-7bpj4\") pod \"nova-api-0\" (UID: \"b045ac1c-0a9b-47e7-b6d7-39963ed900ca\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:28:41 crc kubenswrapper[5052]: I1203 12:28:41.602603 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:28:41 crc kubenswrapper[5052]: I1203 12:28:41.637043 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:28:41 crc kubenswrapper[5052]: I1203 12:28:41.691544 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="18dae0fe-fcde-4a2e-bfd5-3c4ee6976894" path="/var/lib/kubelet/pods/18dae0fe-fcde-4a2e-bfd5-3c4ee6976894/volumes" Dec 03 12:28:41 crc kubenswrapper[5052]: I1203 12:28:41.692332 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a28346c0-bd60-4dac-ac86-20b56296e7d3" path="/var/lib/kubelet/pods/a28346c0-bd60-4dac-ac86-20b56296e7d3/volumes" Dec 03 12:28:41 crc kubenswrapper[5052]: I1203 12:28:41.693138 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d808ef0b-751e-4576-befc-e23a06fe2ff0" path="/var/lib/kubelet/pods/d808ef0b-751e-4576-befc-e23a06fe2ff0/volumes" Dec 03 12:28:42 crc kubenswrapper[5052]: I1203 12:28:42.123983 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:28:42 crc kubenswrapper[5052]: I1203 12:28:42.132292 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"b7c664ff-377a-4d83-a65e-6ecce5b9c9c0","Type":"ContainerStarted","Data":"90846d8253e71e3173475a87c18312a8dd83d07df9b74f3d070d02db7371154d"} Dec 03 12:28:42 crc kubenswrapper[5052]: I1203 12:28:42.133639 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"ada0927b-5399-48af-8a11-ee31292b5d2a","Type":"ContainerStarted","Data":"97adeabe88ffb13e78adb2eab4bb2275e65d7bc8c09c46dbc96035c3b34c0b20"} Dec 03 12:28:42 crc kubenswrapper[5052]: W1203 12:28:42.192192 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb045ac1c_0a9b_47e7_b6d7_39963ed900ca.slice/crio-1f9f0cde741ad5b01bd6e1ab1e4c58d26f18932b4923c41afcc48cff13448279 WatchSource:0}: Error finding container 1f9f0cde741ad5b01bd6e1ab1e4c58d26f18932b4923c41afcc48cff13448279: Status 404 returned error can't find the container with id 1f9f0cde741ad5b01bd6e1ab1e4c58d26f18932b4923c41afcc48cff13448279 Dec 03 12:28:42 crc kubenswrapper[5052]: I1203 12:28:42.194397 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:28:42 crc kubenswrapper[5052]: I1203 12:28:42.732856 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:28:43 crc kubenswrapper[5052]: I1203 12:28:43.144827 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"ada0927b-5399-48af-8a11-ee31292b5d2a","Type":"ContainerStarted","Data":"08dfbdd50f0698222b0397a521292a1d56421652d4a20cca08cf4aa13cdeb4db"} Dec 03 12:28:43 crc kubenswrapper[5052]: I1203 12:28:43.144872 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"ada0927b-5399-48af-8a11-ee31292b5d2a","Type":"ContainerStarted","Data":"203cfb7bbd64d8e75d9f87a48f8ac62ffc001878165757ef3e41902cf4c16eca"} Dec 03 12:28:43 crc kubenswrapper[5052]: I1203 12:28:43.147170 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"b045ac1c-0a9b-47e7-b6d7-39963ed900ca","Type":"ContainerStarted","Data":"4c61b9762409a322fb33d02189df22d0c55d2e1ff0f67b9641779b9bf985e9df"} Dec 03 12:28:43 crc kubenswrapper[5052]: I1203 12:28:43.147224 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"b045ac1c-0a9b-47e7-b6d7-39963ed900ca","Type":"ContainerStarted","Data":"8e5a6d9e4abfbaf5773d33e6944a2aef3bb8a26e7c727e56a7010bc0bdc6a2c6"} Dec 03 12:28:43 crc kubenswrapper[5052]: I1203 12:28:43.147239 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"b045ac1c-0a9b-47e7-b6d7-39963ed900ca","Type":"ContainerStarted","Data":"1f9f0cde741ad5b01bd6e1ab1e4c58d26f18932b4923c41afcc48cff13448279"} Dec 03 12:28:43 crc kubenswrapper[5052]: I1203 12:28:43.185535 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-metadata-0" podStartSLOduration=2.185502068 podStartE2EDuration="2.185502068s" podCreationTimestamp="2025-12-03 12:28:41 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:28:43.182533113 +0000 UTC m=+4113.770774877" watchObservedRunningTime="2025-12-03 12:28:43.185502068 +0000 UTC m=+4113.773743832" Dec 03 12:28:43 crc kubenswrapper[5052]: I1203 12:28:43.205734 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-api-0" podStartSLOduration=2.205719675 podStartE2EDuration="2.205719675s" podCreationTimestamp="2025-12-03 12:28:41 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:28:43.202869213 +0000 UTC m=+4113.791110977" watchObservedRunningTime="2025-12-03 12:28:43.205719675 +0000 UTC m=+4113.793961439" Dec 03 12:28:44 crc kubenswrapper[5052]: I1203 12:28:44.624273 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/keystone-64d6d88f66-7z998" podUID="ef332add-3219-4025-be5b-9d7048cd4a3d" containerName="keystone-api" probeResult="failure" output="Get \"http://10.217.0.128:5000/v3\": read tcp 10.217.0.2:54582->10.217.0.128:5000: read: connection reset by peer" Dec 03 12:28:45 crc kubenswrapper[5052]: I1203 12:28:45.004139 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-64d6d88f66-7z998" Dec 03 12:28:45 crc kubenswrapper[5052]: I1203 12:28:45.121058 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x4krj\" (UniqueName: \"kubernetes.io/projected/ef332add-3219-4025-be5b-9d7048cd4a3d-kube-api-access-x4krj\") pod \"ef332add-3219-4025-be5b-9d7048cd4a3d\" (UID: \"ef332add-3219-4025-be5b-9d7048cd4a3d\") " Dec 03 12:28:45 crc kubenswrapper[5052]: I1203 12:28:45.121109 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/ef332add-3219-4025-be5b-9d7048cd4a3d-config-data\") pod \"ef332add-3219-4025-be5b-9d7048cd4a3d\" (UID: \"ef332add-3219-4025-be5b-9d7048cd4a3d\") " Dec 03 12:28:45 crc kubenswrapper[5052]: I1203 12:28:45.121150 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/ef332add-3219-4025-be5b-9d7048cd4a3d-scripts\") pod \"ef332add-3219-4025-be5b-9d7048cd4a3d\" (UID: \"ef332add-3219-4025-be5b-9d7048cd4a3d\") " Dec 03 12:28:45 crc kubenswrapper[5052]: I1203 12:28:45.121277 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/ef332add-3219-4025-be5b-9d7048cd4a3d-fernet-keys\") pod \"ef332add-3219-4025-be5b-9d7048cd4a3d\" (UID: \"ef332add-3219-4025-be5b-9d7048cd4a3d\") " Dec 03 12:28:45 crc kubenswrapper[5052]: I1203 12:28:45.121308 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ef332add-3219-4025-be5b-9d7048cd4a3d-combined-ca-bundle\") pod \"ef332add-3219-4025-be5b-9d7048cd4a3d\" (UID: \"ef332add-3219-4025-be5b-9d7048cd4a3d\") " Dec 03 12:28:45 crc kubenswrapper[5052]: I1203 12:28:45.121346 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/ef332add-3219-4025-be5b-9d7048cd4a3d-credential-keys\") pod \"ef332add-3219-4025-be5b-9d7048cd4a3d\" (UID: \"ef332add-3219-4025-be5b-9d7048cd4a3d\") " Dec 03 12:28:45 crc kubenswrapper[5052]: I1203 12:28:45.126564 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/swift-proxy-84cc784c4-9pgx5"] Dec 03 12:28:45 crc kubenswrapper[5052]: E1203 12:28:45.127274 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="ef332add-3219-4025-be5b-9d7048cd4a3d" containerName="keystone-api" Dec 03 12:28:45 crc kubenswrapper[5052]: I1203 12:28:45.127295 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="ef332add-3219-4025-be5b-9d7048cd4a3d" containerName="keystone-api" Dec 03 12:28:45 crc kubenswrapper[5052]: I1203 12:28:45.127480 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="ef332add-3219-4025-be5b-9d7048cd4a3d" containerName="keystone-api" Dec 03 12:28:45 crc kubenswrapper[5052]: I1203 12:28:45.130934 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-proxy-84cc784c4-9pgx5" Dec 03 12:28:45 crc kubenswrapper[5052]: I1203 12:28:45.132599 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ef332add-3219-4025-be5b-9d7048cd4a3d-fernet-keys" (OuterVolumeSpecName: "fernet-keys") pod "ef332add-3219-4025-be5b-9d7048cd4a3d" (UID: "ef332add-3219-4025-be5b-9d7048cd4a3d"). InnerVolumeSpecName "fernet-keys". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:28:45 crc kubenswrapper[5052]: I1203 12:28:45.135469 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ef332add-3219-4025-be5b-9d7048cd4a3d-scripts" (OuterVolumeSpecName: "scripts") pod "ef332add-3219-4025-be5b-9d7048cd4a3d" (UID: "ef332add-3219-4025-be5b-9d7048cd4a3d"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:28:45 crc kubenswrapper[5052]: I1203 12:28:45.135716 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-swift-public-svc" Dec 03 12:28:45 crc kubenswrapper[5052]: I1203 12:28:45.135730 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-swift-internal-svc" Dec 03 12:28:45 crc kubenswrapper[5052]: I1203 12:28:45.137586 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ef332add-3219-4025-be5b-9d7048cd4a3d-kube-api-access-x4krj" (OuterVolumeSpecName: "kube-api-access-x4krj") pod "ef332add-3219-4025-be5b-9d7048cd4a3d" (UID: "ef332add-3219-4025-be5b-9d7048cd4a3d"). InnerVolumeSpecName "kube-api-access-x4krj". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:28:45 crc kubenswrapper[5052]: I1203 12:28:45.137817 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ef332add-3219-4025-be5b-9d7048cd4a3d-credential-keys" (OuterVolumeSpecName: "credential-keys") pod "ef332add-3219-4025-be5b-9d7048cd4a3d" (UID: "ef332add-3219-4025-be5b-9d7048cd4a3d"). InnerVolumeSpecName "credential-keys". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:28:45 crc kubenswrapper[5052]: I1203 12:28:45.144804 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/swift-proxy-84cc784c4-9pgx5"] Dec 03 12:28:45 crc kubenswrapper[5052]: I1203 12:28:45.178013 5052 generic.go:334] "Generic (PLEG): container finished" podID="b7c664ff-377a-4d83-a65e-6ecce5b9c9c0" containerID="90846d8253e71e3173475a87c18312a8dd83d07df9b74f3d070d02db7371154d" exitCode=1 Dec 03 12:28:45 crc kubenswrapper[5052]: I1203 12:28:45.178096 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"b7c664ff-377a-4d83-a65e-6ecce5b9c9c0","Type":"ContainerDied","Data":"90846d8253e71e3173475a87c18312a8dd83d07df9b74f3d070d02db7371154d"} Dec 03 12:28:45 crc kubenswrapper[5052]: I1203 12:28:45.178132 5052 scope.go:117] "RemoveContainer" containerID="316d2d796b1bcce26d30624eb3760ad0fa3839c99e679788444235cfcf404776" Dec 03 12:28:45 crc kubenswrapper[5052]: I1203 12:28:45.178876 5052 scope.go:117] "RemoveContainer" containerID="90846d8253e71e3173475a87c18312a8dd83d07df9b74f3d070d02db7371154d" Dec 03 12:28:45 crc kubenswrapper[5052]: E1203 12:28:45.179264 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"nova-scheduler-scheduler\" with CrashLoopBackOff: \"back-off 10s restarting failed container=nova-scheduler-scheduler pod=nova-scheduler-0_openstack-kuttl-tests(b7c664ff-377a-4d83-a65e-6ecce5b9c9c0)\"" pod="openstack-kuttl-tests/nova-scheduler-0" podUID="b7c664ff-377a-4d83-a65e-6ecce5b9c9c0" Dec 03 12:28:45 crc kubenswrapper[5052]: I1203 12:28:45.181444 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ef332add-3219-4025-be5b-9d7048cd4a3d-config-data" (OuterVolumeSpecName: "config-data") pod "ef332add-3219-4025-be5b-9d7048cd4a3d" (UID: "ef332add-3219-4025-be5b-9d7048cd4a3d"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:28:45 crc kubenswrapper[5052]: I1203 12:28:45.181590 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ef332add-3219-4025-be5b-9d7048cd4a3d-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "ef332add-3219-4025-be5b-9d7048cd4a3d" (UID: "ef332add-3219-4025-be5b-9d7048cd4a3d"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:28:45 crc kubenswrapper[5052]: I1203 12:28:45.188412 5052 generic.go:334] "Generic (PLEG): container finished" podID="ef332add-3219-4025-be5b-9d7048cd4a3d" containerID="bc067ea70660c1847f572c9b0681b01351a86de5eb2ae47c967575bf575054ad" exitCode=0 Dec 03 12:28:45 crc kubenswrapper[5052]: I1203 12:28:45.188462 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-64d6d88f66-7z998" event={"ID":"ef332add-3219-4025-be5b-9d7048cd4a3d","Type":"ContainerDied","Data":"bc067ea70660c1847f572c9b0681b01351a86de5eb2ae47c967575bf575054ad"} Dec 03 12:28:45 crc kubenswrapper[5052]: I1203 12:28:45.188974 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-64d6d88f66-7z998" Dec 03 12:28:45 crc kubenswrapper[5052]: I1203 12:28:45.190435 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-64d6d88f66-7z998" event={"ID":"ef332add-3219-4025-be5b-9d7048cd4a3d","Type":"ContainerDied","Data":"abaf7b1a3b67fb796df97ce3ddcdd16ea257efe8f07d67446028267864b9c9d5"} Dec 03 12:28:45 crc kubenswrapper[5052]: I1203 12:28:45.223634 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/6af456b9-3c1e-4d13-ab42-843b94a6eccc-internal-tls-certs\") pod \"swift-proxy-84cc784c4-9pgx5\" (UID: \"6af456b9-3c1e-4d13-ab42-843b94a6eccc\") " pod="openstack-kuttl-tests/swift-proxy-84cc784c4-9pgx5" Dec 03 12:28:45 crc kubenswrapper[5052]: I1203 12:28:45.223760 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hg5ln\" (UniqueName: \"kubernetes.io/projected/6af456b9-3c1e-4d13-ab42-843b94a6eccc-kube-api-access-hg5ln\") pod \"swift-proxy-84cc784c4-9pgx5\" (UID: \"6af456b9-3c1e-4d13-ab42-843b94a6eccc\") " pod="openstack-kuttl-tests/swift-proxy-84cc784c4-9pgx5" Dec 03 12:28:45 crc kubenswrapper[5052]: I1203 12:28:45.223791 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6af456b9-3c1e-4d13-ab42-843b94a6eccc-combined-ca-bundle\") pod \"swift-proxy-84cc784c4-9pgx5\" (UID: \"6af456b9-3c1e-4d13-ab42-843b94a6eccc\") " pod="openstack-kuttl-tests/swift-proxy-84cc784c4-9pgx5" Dec 03 12:28:45 crc kubenswrapper[5052]: I1203 12:28:45.223817 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6af456b9-3c1e-4d13-ab42-843b94a6eccc-config-data\") pod \"swift-proxy-84cc784c4-9pgx5\" (UID: \"6af456b9-3c1e-4d13-ab42-843b94a6eccc\") " pod="openstack-kuttl-tests/swift-proxy-84cc784c4-9pgx5" Dec 03 12:28:45 crc kubenswrapper[5052]: I1203 12:28:45.223880 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/6af456b9-3c1e-4d13-ab42-843b94a6eccc-log-httpd\") pod \"swift-proxy-84cc784c4-9pgx5\" (UID: \"6af456b9-3c1e-4d13-ab42-843b94a6eccc\") " pod="openstack-kuttl-tests/swift-proxy-84cc784c4-9pgx5" Dec 03 12:28:45 crc kubenswrapper[5052]: I1203 12:28:45.223912 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/6af456b9-3c1e-4d13-ab42-843b94a6eccc-public-tls-certs\") pod \"swift-proxy-84cc784c4-9pgx5\" (UID: \"6af456b9-3c1e-4d13-ab42-843b94a6eccc\") " pod="openstack-kuttl-tests/swift-proxy-84cc784c4-9pgx5" Dec 03 12:28:45 crc kubenswrapper[5052]: I1203 12:28:45.223935 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/6af456b9-3c1e-4d13-ab42-843b94a6eccc-etc-swift\") pod \"swift-proxy-84cc784c4-9pgx5\" (UID: \"6af456b9-3c1e-4d13-ab42-843b94a6eccc\") " pod="openstack-kuttl-tests/swift-proxy-84cc784c4-9pgx5" Dec 03 12:28:45 crc kubenswrapper[5052]: I1203 12:28:45.223994 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/6af456b9-3c1e-4d13-ab42-843b94a6eccc-run-httpd\") pod \"swift-proxy-84cc784c4-9pgx5\" (UID: \"6af456b9-3c1e-4d13-ab42-843b94a6eccc\") " pod="openstack-kuttl-tests/swift-proxy-84cc784c4-9pgx5" Dec 03 12:28:45 crc kubenswrapper[5052]: I1203 12:28:45.224080 5052 reconciler_common.go:293] "Volume detached for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/ef332add-3219-4025-be5b-9d7048cd4a3d-fernet-keys\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:45 crc kubenswrapper[5052]: I1203 12:28:45.224096 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ef332add-3219-4025-be5b-9d7048cd4a3d-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:45 crc kubenswrapper[5052]: I1203 12:28:45.224106 5052 reconciler_common.go:293] "Volume detached for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/ef332add-3219-4025-be5b-9d7048cd4a3d-credential-keys\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:45 crc kubenswrapper[5052]: I1203 12:28:45.224115 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-x4krj\" (UniqueName: \"kubernetes.io/projected/ef332add-3219-4025-be5b-9d7048cd4a3d-kube-api-access-x4krj\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:45 crc kubenswrapper[5052]: I1203 12:28:45.224123 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/ef332add-3219-4025-be5b-9d7048cd4a3d-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:45 crc kubenswrapper[5052]: I1203 12:28:45.224131 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/ef332add-3219-4025-be5b-9d7048cd4a3d-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:45 crc kubenswrapper[5052]: I1203 12:28:45.254630 5052 scope.go:117] "RemoveContainer" containerID="bc067ea70660c1847f572c9b0681b01351a86de5eb2ae47c967575bf575054ad" Dec 03 12:28:45 crc kubenswrapper[5052]: I1203 12:28:45.272998 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/keystone-64d6d88f66-7z998"] Dec 03 12:28:45 crc kubenswrapper[5052]: I1203 12:28:45.284360 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/keystone-64d6d88f66-7z998"] Dec 03 12:28:45 crc kubenswrapper[5052]: I1203 12:28:45.285103 5052 scope.go:117] "RemoveContainer" containerID="bc067ea70660c1847f572c9b0681b01351a86de5eb2ae47c967575bf575054ad" Dec 03 12:28:45 crc kubenswrapper[5052]: E1203 12:28:45.285644 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bc067ea70660c1847f572c9b0681b01351a86de5eb2ae47c967575bf575054ad\": container with ID starting with bc067ea70660c1847f572c9b0681b01351a86de5eb2ae47c967575bf575054ad not found: ID does not exist" containerID="bc067ea70660c1847f572c9b0681b01351a86de5eb2ae47c967575bf575054ad" Dec 03 12:28:45 crc kubenswrapper[5052]: I1203 12:28:45.285700 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bc067ea70660c1847f572c9b0681b01351a86de5eb2ae47c967575bf575054ad"} err="failed to get container status \"bc067ea70660c1847f572c9b0681b01351a86de5eb2ae47c967575bf575054ad\": rpc error: code = NotFound desc = could not find container \"bc067ea70660c1847f572c9b0681b01351a86de5eb2ae47c967575bf575054ad\": container with ID starting with bc067ea70660c1847f572c9b0681b01351a86de5eb2ae47c967575bf575054ad not found: ID does not exist" Dec 03 12:28:45 crc kubenswrapper[5052]: I1203 12:28:45.326363 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/6af456b9-3c1e-4d13-ab42-843b94a6eccc-internal-tls-certs\") pod \"swift-proxy-84cc784c4-9pgx5\" (UID: \"6af456b9-3c1e-4d13-ab42-843b94a6eccc\") " pod="openstack-kuttl-tests/swift-proxy-84cc784c4-9pgx5" Dec 03 12:28:45 crc kubenswrapper[5052]: I1203 12:28:45.326558 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-hg5ln\" (UniqueName: \"kubernetes.io/projected/6af456b9-3c1e-4d13-ab42-843b94a6eccc-kube-api-access-hg5ln\") pod \"swift-proxy-84cc784c4-9pgx5\" (UID: \"6af456b9-3c1e-4d13-ab42-843b94a6eccc\") " pod="openstack-kuttl-tests/swift-proxy-84cc784c4-9pgx5" Dec 03 12:28:45 crc kubenswrapper[5052]: I1203 12:28:45.326588 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6af456b9-3c1e-4d13-ab42-843b94a6eccc-combined-ca-bundle\") pod \"swift-proxy-84cc784c4-9pgx5\" (UID: \"6af456b9-3c1e-4d13-ab42-843b94a6eccc\") " pod="openstack-kuttl-tests/swift-proxy-84cc784c4-9pgx5" Dec 03 12:28:45 crc kubenswrapper[5052]: I1203 12:28:45.326609 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6af456b9-3c1e-4d13-ab42-843b94a6eccc-config-data\") pod \"swift-proxy-84cc784c4-9pgx5\" (UID: \"6af456b9-3c1e-4d13-ab42-843b94a6eccc\") " pod="openstack-kuttl-tests/swift-proxy-84cc784c4-9pgx5" Dec 03 12:28:45 crc kubenswrapper[5052]: I1203 12:28:45.326633 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/6af456b9-3c1e-4d13-ab42-843b94a6eccc-log-httpd\") pod \"swift-proxy-84cc784c4-9pgx5\" (UID: \"6af456b9-3c1e-4d13-ab42-843b94a6eccc\") " pod="openstack-kuttl-tests/swift-proxy-84cc784c4-9pgx5" Dec 03 12:28:45 crc kubenswrapper[5052]: I1203 12:28:45.326649 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/6af456b9-3c1e-4d13-ab42-843b94a6eccc-public-tls-certs\") pod \"swift-proxy-84cc784c4-9pgx5\" (UID: \"6af456b9-3c1e-4d13-ab42-843b94a6eccc\") " pod="openstack-kuttl-tests/swift-proxy-84cc784c4-9pgx5" Dec 03 12:28:45 crc kubenswrapper[5052]: I1203 12:28:45.326665 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/6af456b9-3c1e-4d13-ab42-843b94a6eccc-etc-swift\") pod \"swift-proxy-84cc784c4-9pgx5\" (UID: \"6af456b9-3c1e-4d13-ab42-843b94a6eccc\") " pod="openstack-kuttl-tests/swift-proxy-84cc784c4-9pgx5" Dec 03 12:28:45 crc kubenswrapper[5052]: I1203 12:28:45.326707 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/6af456b9-3c1e-4d13-ab42-843b94a6eccc-run-httpd\") pod \"swift-proxy-84cc784c4-9pgx5\" (UID: \"6af456b9-3c1e-4d13-ab42-843b94a6eccc\") " pod="openstack-kuttl-tests/swift-proxy-84cc784c4-9pgx5" Dec 03 12:28:45 crc kubenswrapper[5052]: I1203 12:28:45.327176 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/6af456b9-3c1e-4d13-ab42-843b94a6eccc-run-httpd\") pod \"swift-proxy-84cc784c4-9pgx5\" (UID: \"6af456b9-3c1e-4d13-ab42-843b94a6eccc\") " pod="openstack-kuttl-tests/swift-proxy-84cc784c4-9pgx5" Dec 03 12:28:45 crc kubenswrapper[5052]: I1203 12:28:45.330478 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6af456b9-3c1e-4d13-ab42-843b94a6eccc-config-data\") pod \"swift-proxy-84cc784c4-9pgx5\" (UID: \"6af456b9-3c1e-4d13-ab42-843b94a6eccc\") " pod="openstack-kuttl-tests/swift-proxy-84cc784c4-9pgx5" Dec 03 12:28:45 crc kubenswrapper[5052]: I1203 12:28:45.330734 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/6af456b9-3c1e-4d13-ab42-843b94a6eccc-log-httpd\") pod \"swift-proxy-84cc784c4-9pgx5\" (UID: \"6af456b9-3c1e-4d13-ab42-843b94a6eccc\") " pod="openstack-kuttl-tests/swift-proxy-84cc784c4-9pgx5" Dec 03 12:28:45 crc kubenswrapper[5052]: I1203 12:28:45.330737 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6af456b9-3c1e-4d13-ab42-843b94a6eccc-combined-ca-bundle\") pod \"swift-proxy-84cc784c4-9pgx5\" (UID: \"6af456b9-3c1e-4d13-ab42-843b94a6eccc\") " pod="openstack-kuttl-tests/swift-proxy-84cc784c4-9pgx5" Dec 03 12:28:45 crc kubenswrapper[5052]: I1203 12:28:45.332198 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/6af456b9-3c1e-4d13-ab42-843b94a6eccc-internal-tls-certs\") pod \"swift-proxy-84cc784c4-9pgx5\" (UID: \"6af456b9-3c1e-4d13-ab42-843b94a6eccc\") " pod="openstack-kuttl-tests/swift-proxy-84cc784c4-9pgx5" Dec 03 12:28:45 crc kubenswrapper[5052]: I1203 12:28:45.333308 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/6af456b9-3c1e-4d13-ab42-843b94a6eccc-public-tls-certs\") pod \"swift-proxy-84cc784c4-9pgx5\" (UID: \"6af456b9-3c1e-4d13-ab42-843b94a6eccc\") " pod="openstack-kuttl-tests/swift-proxy-84cc784c4-9pgx5" Dec 03 12:28:45 crc kubenswrapper[5052]: I1203 12:28:45.334935 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/6af456b9-3c1e-4d13-ab42-843b94a6eccc-etc-swift\") pod \"swift-proxy-84cc784c4-9pgx5\" (UID: \"6af456b9-3c1e-4d13-ab42-843b94a6eccc\") " pod="openstack-kuttl-tests/swift-proxy-84cc784c4-9pgx5" Dec 03 12:28:45 crc kubenswrapper[5052]: I1203 12:28:45.349385 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-hg5ln\" (UniqueName: \"kubernetes.io/projected/6af456b9-3c1e-4d13-ab42-843b94a6eccc-kube-api-access-hg5ln\") pod \"swift-proxy-84cc784c4-9pgx5\" (UID: \"6af456b9-3c1e-4d13-ab42-843b94a6eccc\") " pod="openstack-kuttl-tests/swift-proxy-84cc784c4-9pgx5" Dec 03 12:28:45 crc kubenswrapper[5052]: I1203 12:28:45.555188 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-proxy-84cc784c4-9pgx5" Dec 03 12:28:45 crc kubenswrapper[5052]: I1203 12:28:45.565687 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-0"] Dec 03 12:28:45 crc kubenswrapper[5052]: I1203 12:28:45.565917 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-cell0-conductor-0" podUID="6efd22cd-98c7-48d8-a346-289b78e8aa1c" containerName="nova-cell0-conductor-conductor" containerID="cri-o://ae1f276ec0ccddfcdb84f346890a9d2aa8e21c3960425cd4ca2a6483bc8b8ae7" gracePeriod=30 Dec 03 12:28:45 crc kubenswrapper[5052]: I1203 12:28:45.578409 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:28:45 crc kubenswrapper[5052]: I1203 12:28:45.579204 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-metadata-0" podUID="ada0927b-5399-48af-8a11-ee31292b5d2a" containerName="nova-metadata-metadata" containerID="cri-o://08dfbdd50f0698222b0397a521292a1d56421652d4a20cca08cf4aa13cdeb4db" gracePeriod=30 Dec 03 12:28:45 crc kubenswrapper[5052]: I1203 12:28:45.578836 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-metadata-0" podUID="ada0927b-5399-48af-8a11-ee31292b5d2a" containerName="nova-metadata-log" containerID="cri-o://203cfb7bbd64d8e75d9f87a48f8ac62ffc001878165757ef3e41902cf4c16eca" gracePeriod=30 Dec 03 12:28:45 crc kubenswrapper[5052]: I1203 12:28:45.586264 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:28:45 crc kubenswrapper[5052]: I1203 12:28:45.586448 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-api-0" podUID="b045ac1c-0a9b-47e7-b6d7-39963ed900ca" containerName="nova-api-log" containerID="cri-o://8e5a6d9e4abfbaf5773d33e6944a2aef3bb8a26e7c727e56a7010bc0bdc6a2c6" gracePeriod=30 Dec 03 12:28:45 crc kubenswrapper[5052]: I1203 12:28:45.586900 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-api-0" podUID="b045ac1c-0a9b-47e7-b6d7-39963ed900ca" containerName="nova-api-api" containerID="cri-o://4c61b9762409a322fb33d02189df22d0c55d2e1ff0f67b9641779b9bf985e9df" gracePeriod=30 Dec 03 12:28:45 crc kubenswrapper[5052]: I1203 12:28:45.649453 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:28:45 crc kubenswrapper[5052]: I1203 12:28:45.667922 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 12:28:45 crc kubenswrapper[5052]: I1203 12:28:45.668181 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" podUID="bea08f45-6aef-4ccc-8252-d840baad2bf2" containerName="nova-cell1-novncproxy-novncproxy" containerID="cri-o://47da0b9399486978354254c5d335b3c5c60cbdaa11b8b2dfb0593a15c8eb3a11" gracePeriod=30 Dec 03 12:28:45 crc kubenswrapper[5052]: I1203 12:28:45.698761 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ef332add-3219-4025-be5b-9d7048cd4a3d" path="/var/lib/kubelet/pods/ef332add-3219-4025-be5b-9d7048cd4a3d/volumes" Dec 03 12:28:45 crc kubenswrapper[5052]: I1203 12:28:45.757231 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-0"] Dec 03 12:28:45 crc kubenswrapper[5052]: I1203 12:28:45.757950 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-cell1-conductor-0" podUID="5ae7c56e-3507-403b-b58a-1a6da1ed1ab3" containerName="nova-cell1-conductor-conductor" containerID="cri-o://61a4505e879a4045e7b22afd402d01101f72aedd5245a9a876b0255b885b15ed" gracePeriod=30 Dec 03 12:28:46 crc kubenswrapper[5052]: I1203 12:28:46.117229 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/swift-proxy-84cc784c4-9pgx5"] Dec 03 12:28:46 crc kubenswrapper[5052]: E1203 12:28:46.217294 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="ae1f276ec0ccddfcdb84f346890a9d2aa8e21c3960425cd4ca2a6483bc8b8ae7" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:28:46 crc kubenswrapper[5052]: E1203 12:28:46.219289 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="ae1f276ec0ccddfcdb84f346890a9d2aa8e21c3960425cd4ca2a6483bc8b8ae7" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:28:46 crc kubenswrapper[5052]: I1203 12:28:46.220255 5052 generic.go:334] "Generic (PLEG): container finished" podID="b045ac1c-0a9b-47e7-b6d7-39963ed900ca" containerID="4c61b9762409a322fb33d02189df22d0c55d2e1ff0f67b9641779b9bf985e9df" exitCode=0 Dec 03 12:28:46 crc kubenswrapper[5052]: I1203 12:28:46.220281 5052 generic.go:334] "Generic (PLEG): container finished" podID="b045ac1c-0a9b-47e7-b6d7-39963ed900ca" containerID="8e5a6d9e4abfbaf5773d33e6944a2aef3bb8a26e7c727e56a7010bc0bdc6a2c6" exitCode=143 Dec 03 12:28:46 crc kubenswrapper[5052]: I1203 12:28:46.220327 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"b045ac1c-0a9b-47e7-b6d7-39963ed900ca","Type":"ContainerDied","Data":"4c61b9762409a322fb33d02189df22d0c55d2e1ff0f67b9641779b9bf985e9df"} Dec 03 12:28:46 crc kubenswrapper[5052]: I1203 12:28:46.220362 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"b045ac1c-0a9b-47e7-b6d7-39963ed900ca","Type":"ContainerDied","Data":"8e5a6d9e4abfbaf5773d33e6944a2aef3bb8a26e7c727e56a7010bc0bdc6a2c6"} Dec 03 12:28:46 crc kubenswrapper[5052]: E1203 12:28:46.222453 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="ae1f276ec0ccddfcdb84f346890a9d2aa8e21c3960425cd4ca2a6483bc8b8ae7" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:28:46 crc kubenswrapper[5052]: E1203 12:28:46.222499 5052 prober.go:104] "Probe errored" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" probeType="Readiness" pod="openstack-kuttl-tests/nova-cell0-conductor-0" podUID="6efd22cd-98c7-48d8-a346-289b78e8aa1c" containerName="nova-cell0-conductor-conductor" Dec 03 12:28:46 crc kubenswrapper[5052]: I1203 12:28:46.226080 5052 generic.go:334] "Generic (PLEG): container finished" podID="ada0927b-5399-48af-8a11-ee31292b5d2a" containerID="08dfbdd50f0698222b0397a521292a1d56421652d4a20cca08cf4aa13cdeb4db" exitCode=0 Dec 03 12:28:46 crc kubenswrapper[5052]: I1203 12:28:46.226111 5052 generic.go:334] "Generic (PLEG): container finished" podID="ada0927b-5399-48af-8a11-ee31292b5d2a" containerID="203cfb7bbd64d8e75d9f87a48f8ac62ffc001878165757ef3e41902cf4c16eca" exitCode=143 Dec 03 12:28:46 crc kubenswrapper[5052]: I1203 12:28:46.226172 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"ada0927b-5399-48af-8a11-ee31292b5d2a","Type":"ContainerDied","Data":"08dfbdd50f0698222b0397a521292a1d56421652d4a20cca08cf4aa13cdeb4db"} Dec 03 12:28:46 crc kubenswrapper[5052]: I1203 12:28:46.226206 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"ada0927b-5399-48af-8a11-ee31292b5d2a","Type":"ContainerDied","Data":"203cfb7bbd64d8e75d9f87a48f8ac62ffc001878165757ef3e41902cf4c16eca"} Dec 03 12:28:46 crc kubenswrapper[5052]: I1203 12:28:46.226222 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"ada0927b-5399-48af-8a11-ee31292b5d2a","Type":"ContainerDied","Data":"97adeabe88ffb13e78adb2eab4bb2275e65d7bc8c09c46dbc96035c3b34c0b20"} Dec 03 12:28:46 crc kubenswrapper[5052]: I1203 12:28:46.226235 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="97adeabe88ffb13e78adb2eab4bb2275e65d7bc8c09c46dbc96035c3b34c0b20" Dec 03 12:28:46 crc kubenswrapper[5052]: I1203 12:28:46.232989 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-proxy-84cc784c4-9pgx5" event={"ID":"6af456b9-3c1e-4d13-ab42-843b94a6eccc","Type":"ContainerStarted","Data":"57747c3003398e289e5ac01543fab3e660cc94651b02052539b92d4fe11f2f31"} Dec 03 12:28:46 crc kubenswrapper[5052]: I1203 12:28:46.260365 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:28:46 crc kubenswrapper[5052]: I1203 12:28:46.354930 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-krrp4\" (UniqueName: \"kubernetes.io/projected/ada0927b-5399-48af-8a11-ee31292b5d2a-kube-api-access-krrp4\") pod \"ada0927b-5399-48af-8a11-ee31292b5d2a\" (UID: \"ada0927b-5399-48af-8a11-ee31292b5d2a\") " Dec 03 12:28:46 crc kubenswrapper[5052]: I1203 12:28:46.355058 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ada0927b-5399-48af-8a11-ee31292b5d2a-combined-ca-bundle\") pod \"ada0927b-5399-48af-8a11-ee31292b5d2a\" (UID: \"ada0927b-5399-48af-8a11-ee31292b5d2a\") " Dec 03 12:28:46 crc kubenswrapper[5052]: I1203 12:28:46.355136 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/ada0927b-5399-48af-8a11-ee31292b5d2a-config-data\") pod \"ada0927b-5399-48af-8a11-ee31292b5d2a\" (UID: \"ada0927b-5399-48af-8a11-ee31292b5d2a\") " Dec 03 12:28:46 crc kubenswrapper[5052]: I1203 12:28:46.355171 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/ada0927b-5399-48af-8a11-ee31292b5d2a-logs\") pod \"ada0927b-5399-48af-8a11-ee31292b5d2a\" (UID: \"ada0927b-5399-48af-8a11-ee31292b5d2a\") " Dec 03 12:28:46 crc kubenswrapper[5052]: I1203 12:28:46.355215 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/ada0927b-5399-48af-8a11-ee31292b5d2a-nova-metadata-tls-certs\") pod \"ada0927b-5399-48af-8a11-ee31292b5d2a\" (UID: \"ada0927b-5399-48af-8a11-ee31292b5d2a\") " Dec 03 12:28:46 crc kubenswrapper[5052]: I1203 12:28:46.360962 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ada0927b-5399-48af-8a11-ee31292b5d2a-logs" (OuterVolumeSpecName: "logs") pod "ada0927b-5399-48af-8a11-ee31292b5d2a" (UID: "ada0927b-5399-48af-8a11-ee31292b5d2a"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:28:46 crc kubenswrapper[5052]: I1203 12:28:46.368496 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ada0927b-5399-48af-8a11-ee31292b5d2a-kube-api-access-krrp4" (OuterVolumeSpecName: "kube-api-access-krrp4") pod "ada0927b-5399-48af-8a11-ee31292b5d2a" (UID: "ada0927b-5399-48af-8a11-ee31292b5d2a"). InnerVolumeSpecName "kube-api-access-krrp4". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:28:46 crc kubenswrapper[5052]: I1203 12:28:46.400630 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ada0927b-5399-48af-8a11-ee31292b5d2a-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "ada0927b-5399-48af-8a11-ee31292b5d2a" (UID: "ada0927b-5399-48af-8a11-ee31292b5d2a"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:28:46 crc kubenswrapper[5052]: I1203 12:28:46.435940 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ada0927b-5399-48af-8a11-ee31292b5d2a-nova-metadata-tls-certs" (OuterVolumeSpecName: "nova-metadata-tls-certs") pod "ada0927b-5399-48af-8a11-ee31292b5d2a" (UID: "ada0927b-5399-48af-8a11-ee31292b5d2a"). InnerVolumeSpecName "nova-metadata-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:28:46 crc kubenswrapper[5052]: I1203 12:28:46.457553 5052 reconciler_common.go:293] "Volume detached for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/ada0927b-5399-48af-8a11-ee31292b5d2a-nova-metadata-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:46 crc kubenswrapper[5052]: I1203 12:28:46.457614 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-krrp4\" (UniqueName: \"kubernetes.io/projected/ada0927b-5399-48af-8a11-ee31292b5d2a-kube-api-access-krrp4\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:46 crc kubenswrapper[5052]: I1203 12:28:46.457632 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ada0927b-5399-48af-8a11-ee31292b5d2a-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:46 crc kubenswrapper[5052]: I1203 12:28:46.457646 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/ada0927b-5399-48af-8a11-ee31292b5d2a-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:46 crc kubenswrapper[5052]: I1203 12:28:46.472805 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ada0927b-5399-48af-8a11-ee31292b5d2a-config-data" (OuterVolumeSpecName: "config-data") pod "ada0927b-5399-48af-8a11-ee31292b5d2a" (UID: "ada0927b-5399-48af-8a11-ee31292b5d2a"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:28:46 crc kubenswrapper[5052]: I1203 12:28:46.496906 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:28:46 crc kubenswrapper[5052]: I1203 12:28:46.560059 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/ada0927b-5399-48af-8a11-ee31292b5d2a-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:46 crc kubenswrapper[5052]: I1203 12:28:46.675893 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b045ac1c-0a9b-47e7-b6d7-39963ed900ca-combined-ca-bundle\") pod \"b045ac1c-0a9b-47e7-b6d7-39963ed900ca\" (UID: \"b045ac1c-0a9b-47e7-b6d7-39963ed900ca\") " Dec 03 12:28:46 crc kubenswrapper[5052]: I1203 12:28:46.676273 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/b045ac1c-0a9b-47e7-b6d7-39963ed900ca-public-tls-certs\") pod \"b045ac1c-0a9b-47e7-b6d7-39963ed900ca\" (UID: \"b045ac1c-0a9b-47e7-b6d7-39963ed900ca\") " Dec 03 12:28:46 crc kubenswrapper[5052]: I1203 12:28:46.676333 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/b045ac1c-0a9b-47e7-b6d7-39963ed900ca-logs\") pod \"b045ac1c-0a9b-47e7-b6d7-39963ed900ca\" (UID: \"b045ac1c-0a9b-47e7-b6d7-39963ed900ca\") " Dec 03 12:28:46 crc kubenswrapper[5052]: I1203 12:28:46.676356 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/b045ac1c-0a9b-47e7-b6d7-39963ed900ca-config-data\") pod \"b045ac1c-0a9b-47e7-b6d7-39963ed900ca\" (UID: \"b045ac1c-0a9b-47e7-b6d7-39963ed900ca\") " Dec 03 12:28:46 crc kubenswrapper[5052]: I1203 12:28:46.676503 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7bpj4\" (UniqueName: \"kubernetes.io/projected/b045ac1c-0a9b-47e7-b6d7-39963ed900ca-kube-api-access-7bpj4\") pod \"b045ac1c-0a9b-47e7-b6d7-39963ed900ca\" (UID: \"b045ac1c-0a9b-47e7-b6d7-39963ed900ca\") " Dec 03 12:28:46 crc kubenswrapper[5052]: I1203 12:28:46.676581 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/b045ac1c-0a9b-47e7-b6d7-39963ed900ca-internal-tls-certs\") pod \"b045ac1c-0a9b-47e7-b6d7-39963ed900ca\" (UID: \"b045ac1c-0a9b-47e7-b6d7-39963ed900ca\") " Dec 03 12:28:46 crc kubenswrapper[5052]: I1203 12:28:46.681340 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b045ac1c-0a9b-47e7-b6d7-39963ed900ca-logs" (OuterVolumeSpecName: "logs") pod "b045ac1c-0a9b-47e7-b6d7-39963ed900ca" (UID: "b045ac1c-0a9b-47e7-b6d7-39963ed900ca"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:28:46 crc kubenswrapper[5052]: I1203 12:28:46.688828 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b045ac1c-0a9b-47e7-b6d7-39963ed900ca-kube-api-access-7bpj4" (OuterVolumeSpecName: "kube-api-access-7bpj4") pod "b045ac1c-0a9b-47e7-b6d7-39963ed900ca" (UID: "b045ac1c-0a9b-47e7-b6d7-39963ed900ca"). InnerVolumeSpecName "kube-api-access-7bpj4". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:28:46 crc kubenswrapper[5052]: I1203 12:28:46.747726 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b045ac1c-0a9b-47e7-b6d7-39963ed900ca-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "b045ac1c-0a9b-47e7-b6d7-39963ed900ca" (UID: "b045ac1c-0a9b-47e7-b6d7-39963ed900ca"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:28:46 crc kubenswrapper[5052]: I1203 12:28:46.768282 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b045ac1c-0a9b-47e7-b6d7-39963ed900ca-config-data" (OuterVolumeSpecName: "config-data") pod "b045ac1c-0a9b-47e7-b6d7-39963ed900ca" (UID: "b045ac1c-0a9b-47e7-b6d7-39963ed900ca"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:28:46 crc kubenswrapper[5052]: I1203 12:28:46.782402 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-7bpj4\" (UniqueName: \"kubernetes.io/projected/b045ac1c-0a9b-47e7-b6d7-39963ed900ca-kube-api-access-7bpj4\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:46 crc kubenswrapper[5052]: I1203 12:28:46.782443 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b045ac1c-0a9b-47e7-b6d7-39963ed900ca-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:46 crc kubenswrapper[5052]: I1203 12:28:46.782455 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/b045ac1c-0a9b-47e7-b6d7-39963ed900ca-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:46 crc kubenswrapper[5052]: I1203 12:28:46.782467 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/b045ac1c-0a9b-47e7-b6d7-39963ed900ca-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:46 crc kubenswrapper[5052]: I1203 12:28:46.800982 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b045ac1c-0a9b-47e7-b6d7-39963ed900ca-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "b045ac1c-0a9b-47e7-b6d7-39963ed900ca" (UID: "b045ac1c-0a9b-47e7-b6d7-39963ed900ca"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:28:46 crc kubenswrapper[5052]: I1203 12:28:46.802635 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b045ac1c-0a9b-47e7-b6d7-39963ed900ca-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "b045ac1c-0a9b-47e7-b6d7-39963ed900ca" (UID: "b045ac1c-0a9b-47e7-b6d7-39963ed900ca"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:28:46 crc kubenswrapper[5052]: I1203 12:28:46.883683 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/b045ac1c-0a9b-47e7-b6d7-39963ed900ca-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:46 crc kubenswrapper[5052]: I1203 12:28:46.883722 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/b045ac1c-0a9b-47e7-b6d7-39963ed900ca-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:47 crc kubenswrapper[5052]: I1203 12:28:47.032996 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:28:47 crc kubenswrapper[5052]: I1203 12:28:47.041401 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:28:47 crc kubenswrapper[5052]: I1203 12:28:47.189124 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b7c664ff-377a-4d83-a65e-6ecce5b9c9c0-combined-ca-bundle\") pod \"b7c664ff-377a-4d83-a65e-6ecce5b9c9c0\" (UID: \"b7c664ff-377a-4d83-a65e-6ecce5b9c9c0\") " Dec 03 12:28:47 crc kubenswrapper[5052]: I1203 12:28:47.189423 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c4v8f\" (UniqueName: \"kubernetes.io/projected/b7c664ff-377a-4d83-a65e-6ecce5b9c9c0-kube-api-access-c4v8f\") pod \"b7c664ff-377a-4d83-a65e-6ecce5b9c9c0\" (UID: \"b7c664ff-377a-4d83-a65e-6ecce5b9c9c0\") " Dec 03 12:28:47 crc kubenswrapper[5052]: I1203 12:28:47.189479 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/bea08f45-6aef-4ccc-8252-d840baad2bf2-combined-ca-bundle\") pod \"bea08f45-6aef-4ccc-8252-d840baad2bf2\" (UID: \"bea08f45-6aef-4ccc-8252-d840baad2bf2\") " Dec 03 12:28:47 crc kubenswrapper[5052]: I1203 12:28:47.189528 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"vencrypt-tls-certs\" (UniqueName: \"kubernetes.io/secret/bea08f45-6aef-4ccc-8252-d840baad2bf2-vencrypt-tls-certs\") pod \"bea08f45-6aef-4ccc-8252-d840baad2bf2\" (UID: \"bea08f45-6aef-4ccc-8252-d840baad2bf2\") " Dec 03 12:28:47 crc kubenswrapper[5052]: I1203 12:28:47.189587 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"nova-novncproxy-tls-certs\" (UniqueName: \"kubernetes.io/secret/bea08f45-6aef-4ccc-8252-d840baad2bf2-nova-novncproxy-tls-certs\") pod \"bea08f45-6aef-4ccc-8252-d840baad2bf2\" (UID: \"bea08f45-6aef-4ccc-8252-d840baad2bf2\") " Dec 03 12:28:47 crc kubenswrapper[5052]: I1203 12:28:47.189662 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cdbdz\" (UniqueName: \"kubernetes.io/projected/bea08f45-6aef-4ccc-8252-d840baad2bf2-kube-api-access-cdbdz\") pod \"bea08f45-6aef-4ccc-8252-d840baad2bf2\" (UID: \"bea08f45-6aef-4ccc-8252-d840baad2bf2\") " Dec 03 12:28:47 crc kubenswrapper[5052]: I1203 12:28:47.189728 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/bea08f45-6aef-4ccc-8252-d840baad2bf2-config-data\") pod \"bea08f45-6aef-4ccc-8252-d840baad2bf2\" (UID: \"bea08f45-6aef-4ccc-8252-d840baad2bf2\") " Dec 03 12:28:47 crc kubenswrapper[5052]: I1203 12:28:47.189752 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/b7c664ff-377a-4d83-a65e-6ecce5b9c9c0-config-data\") pod \"b7c664ff-377a-4d83-a65e-6ecce5b9c9c0\" (UID: \"b7c664ff-377a-4d83-a65e-6ecce5b9c9c0\") " Dec 03 12:28:47 crc kubenswrapper[5052]: I1203 12:28:47.194880 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b7c664ff-377a-4d83-a65e-6ecce5b9c9c0-kube-api-access-c4v8f" (OuterVolumeSpecName: "kube-api-access-c4v8f") pod "b7c664ff-377a-4d83-a65e-6ecce5b9c9c0" (UID: "b7c664ff-377a-4d83-a65e-6ecce5b9c9c0"). InnerVolumeSpecName "kube-api-access-c4v8f". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:28:47 crc kubenswrapper[5052]: I1203 12:28:47.197859 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bea08f45-6aef-4ccc-8252-d840baad2bf2-kube-api-access-cdbdz" (OuterVolumeSpecName: "kube-api-access-cdbdz") pod "bea08f45-6aef-4ccc-8252-d840baad2bf2" (UID: "bea08f45-6aef-4ccc-8252-d840baad2bf2"). InnerVolumeSpecName "kube-api-access-cdbdz". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:28:47 crc kubenswrapper[5052]: I1203 12:28:47.259782 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bea08f45-6aef-4ccc-8252-d840baad2bf2-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "bea08f45-6aef-4ccc-8252-d840baad2bf2" (UID: "bea08f45-6aef-4ccc-8252-d840baad2bf2"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:28:47 crc kubenswrapper[5052]: I1203 12:28:47.284010 5052 generic.go:334] "Generic (PLEG): container finished" podID="5ae7c56e-3507-403b-b58a-1a6da1ed1ab3" containerID="61a4505e879a4045e7b22afd402d01101f72aedd5245a9a876b0255b885b15ed" exitCode=0 Dec 03 12:28:47 crc kubenswrapper[5052]: I1203 12:28:47.284104 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-conductor-0" event={"ID":"5ae7c56e-3507-403b-b58a-1a6da1ed1ab3","Type":"ContainerDied","Data":"61a4505e879a4045e7b22afd402d01101f72aedd5245a9a876b0255b885b15ed"} Dec 03 12:28:47 crc kubenswrapper[5052]: I1203 12:28:47.286451 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bea08f45-6aef-4ccc-8252-d840baad2bf2-config-data" (OuterVolumeSpecName: "config-data") pod "bea08f45-6aef-4ccc-8252-d840baad2bf2" (UID: "bea08f45-6aef-4ccc-8252-d840baad2bf2"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:28:47 crc kubenswrapper[5052]: I1203 12:28:47.286639 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-proxy-84cc784c4-9pgx5" event={"ID":"6af456b9-3c1e-4d13-ab42-843b94a6eccc","Type":"ContainerStarted","Data":"dc7840751c070b24059d5c6705a80932dcd86fe7d9711d447555b21056996478"} Dec 03 12:28:47 crc kubenswrapper[5052]: I1203 12:28:47.286673 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-proxy-84cc784c4-9pgx5" event={"ID":"6af456b9-3c1e-4d13-ab42-843b94a6eccc","Type":"ContainerStarted","Data":"e658cf1a323255cebd83f830ba316f88e6f46fb6e418f976a2cc18aafeb2c7b8"} Dec 03 12:28:47 crc kubenswrapper[5052]: I1203 12:28:47.287048 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/swift-proxy-84cc784c4-9pgx5" Dec 03 12:28:47 crc kubenswrapper[5052]: I1203 12:28:47.287081 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/swift-proxy-84cc784c4-9pgx5" Dec 03 12:28:47 crc kubenswrapper[5052]: I1203 12:28:47.289085 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:28:47 crc kubenswrapper[5052]: I1203 12:28:47.289105 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"b7c664ff-377a-4d83-a65e-6ecce5b9c9c0","Type":"ContainerDied","Data":"3de0218af395e15d0f4d62000548093e62171a5b7c0f818e75375b1db56420aa"} Dec 03 12:28:47 crc kubenswrapper[5052]: I1203 12:28:47.289138 5052 scope.go:117] "RemoveContainer" containerID="90846d8253e71e3173475a87c18312a8dd83d07df9b74f3d070d02db7371154d" Dec 03 12:28:47 crc kubenswrapper[5052]: I1203 12:28:47.291494 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-cdbdz\" (UniqueName: \"kubernetes.io/projected/bea08f45-6aef-4ccc-8252-d840baad2bf2-kube-api-access-cdbdz\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:47 crc kubenswrapper[5052]: I1203 12:28:47.291549 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/bea08f45-6aef-4ccc-8252-d840baad2bf2-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:47 crc kubenswrapper[5052]: I1203 12:28:47.291562 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-c4v8f\" (UniqueName: \"kubernetes.io/projected/b7c664ff-377a-4d83-a65e-6ecce5b9c9c0-kube-api-access-c4v8f\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:47 crc kubenswrapper[5052]: I1203 12:28:47.291571 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/bea08f45-6aef-4ccc-8252-d840baad2bf2-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:47 crc kubenswrapper[5052]: I1203 12:28:47.299068 5052 generic.go:334] "Generic (PLEG): container finished" podID="bea08f45-6aef-4ccc-8252-d840baad2bf2" containerID="47da0b9399486978354254c5d335b3c5c60cbdaa11b8b2dfb0593a15c8eb3a11" exitCode=0 Dec 03 12:28:47 crc kubenswrapper[5052]: I1203 12:28:47.299201 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" event={"ID":"bea08f45-6aef-4ccc-8252-d840baad2bf2","Type":"ContainerDied","Data":"47da0b9399486978354254c5d335b3c5c60cbdaa11b8b2dfb0593a15c8eb3a11"} Dec 03 12:28:47 crc kubenswrapper[5052]: I1203 12:28:47.299266 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" event={"ID":"bea08f45-6aef-4ccc-8252-d840baad2bf2","Type":"ContainerDied","Data":"7989f6bd58d24fdb13a69c51dbace06625effb27d2be03b8530cb269630b489e"} Dec 03 12:28:47 crc kubenswrapper[5052]: I1203 12:28:47.299351 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:28:47 crc kubenswrapper[5052]: I1203 12:28:47.312912 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b7c664ff-377a-4d83-a65e-6ecce5b9c9c0-config-data" (OuterVolumeSpecName: "config-data") pod "b7c664ff-377a-4d83-a65e-6ecce5b9c9c0" (UID: "b7c664ff-377a-4d83-a65e-6ecce5b9c9c0"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:28:47 crc kubenswrapper[5052]: I1203 12:28:47.322565 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b7c664ff-377a-4d83-a65e-6ecce5b9c9c0-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "b7c664ff-377a-4d83-a65e-6ecce5b9c9c0" (UID: "b7c664ff-377a-4d83-a65e-6ecce5b9c9c0"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:28:47 crc kubenswrapper[5052]: I1203 12:28:47.328701 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bea08f45-6aef-4ccc-8252-d840baad2bf2-vencrypt-tls-certs" (OuterVolumeSpecName: "vencrypt-tls-certs") pod "bea08f45-6aef-4ccc-8252-d840baad2bf2" (UID: "bea08f45-6aef-4ccc-8252-d840baad2bf2"). InnerVolumeSpecName "vencrypt-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:28:47 crc kubenswrapper[5052]: I1203 12:28:47.331460 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bea08f45-6aef-4ccc-8252-d840baad2bf2-nova-novncproxy-tls-certs" (OuterVolumeSpecName: "nova-novncproxy-tls-certs") pod "bea08f45-6aef-4ccc-8252-d840baad2bf2" (UID: "bea08f45-6aef-4ccc-8252-d840baad2bf2"). InnerVolumeSpecName "nova-novncproxy-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:28:47 crc kubenswrapper[5052]: I1203 12:28:47.333569 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"b045ac1c-0a9b-47e7-b6d7-39963ed900ca","Type":"ContainerDied","Data":"1f9f0cde741ad5b01bd6e1ab1e4c58d26f18932b4923c41afcc48cff13448279"} Dec 03 12:28:47 crc kubenswrapper[5052]: I1203 12:28:47.333584 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:28:47 crc kubenswrapper[5052]: I1203 12:28:47.352112 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:28:47 crc kubenswrapper[5052]: I1203 12:28:47.370806 5052 scope.go:117] "RemoveContainer" containerID="47da0b9399486978354254c5d335b3c5c60cbdaa11b8b2dfb0593a15c8eb3a11" Dec 03 12:28:47 crc kubenswrapper[5052]: I1203 12:28:47.381799 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:28:47 crc kubenswrapper[5052]: I1203 12:28:47.382076 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/swift-proxy-84cc784c4-9pgx5" podStartSLOduration=2.382056836 podStartE2EDuration="2.382056836s" podCreationTimestamp="2025-12-03 12:28:45 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:28:47.313364295 +0000 UTC m=+4117.901606059" watchObservedRunningTime="2025-12-03 12:28:47.382056836 +0000 UTC m=+4117.970298600" Dec 03 12:28:47 crc kubenswrapper[5052]: I1203 12:28:47.388992 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:28:47 crc kubenswrapper[5052]: I1203 12:28:47.395438 5052 reconciler_common.go:293] "Volume detached for volume \"nova-novncproxy-tls-certs\" (UniqueName: \"kubernetes.io/secret/bea08f45-6aef-4ccc-8252-d840baad2bf2-nova-novncproxy-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:47 crc kubenswrapper[5052]: I1203 12:28:47.395490 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/b7c664ff-377a-4d83-a65e-6ecce5b9c9c0-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:47 crc kubenswrapper[5052]: I1203 12:28:47.395499 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b7c664ff-377a-4d83-a65e-6ecce5b9c9c0-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:47 crc kubenswrapper[5052]: I1203 12:28:47.395578 5052 reconciler_common.go:293] "Volume detached for volume \"vencrypt-tls-certs\" (UniqueName: \"kubernetes.io/secret/bea08f45-6aef-4ccc-8252-d840baad2bf2-vencrypt-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:47 crc kubenswrapper[5052]: I1203 12:28:47.402558 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:28:47 crc kubenswrapper[5052]: I1203 12:28:47.416082 5052 scope.go:117] "RemoveContainer" containerID="47da0b9399486978354254c5d335b3c5c60cbdaa11b8b2dfb0593a15c8eb3a11" Dec 03 12:28:47 crc kubenswrapper[5052]: E1203 12:28:47.416985 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"47da0b9399486978354254c5d335b3c5c60cbdaa11b8b2dfb0593a15c8eb3a11\": container with ID starting with 47da0b9399486978354254c5d335b3c5c60cbdaa11b8b2dfb0593a15c8eb3a11 not found: ID does not exist" containerID="47da0b9399486978354254c5d335b3c5c60cbdaa11b8b2dfb0593a15c8eb3a11" Dec 03 12:28:47 crc kubenswrapper[5052]: I1203 12:28:47.417104 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"47da0b9399486978354254c5d335b3c5c60cbdaa11b8b2dfb0593a15c8eb3a11"} err="failed to get container status \"47da0b9399486978354254c5d335b3c5c60cbdaa11b8b2dfb0593a15c8eb3a11\": rpc error: code = NotFound desc = could not find container \"47da0b9399486978354254c5d335b3c5c60cbdaa11b8b2dfb0593a15c8eb3a11\": container with ID starting with 47da0b9399486978354254c5d335b3c5c60cbdaa11b8b2dfb0593a15c8eb3a11 not found: ID does not exist" Dec 03 12:28:47 crc kubenswrapper[5052]: I1203 12:28:47.417215 5052 scope.go:117] "RemoveContainer" containerID="4c61b9762409a322fb33d02189df22d0c55d2e1ff0f67b9641779b9bf985e9df" Dec 03 12:28:47 crc kubenswrapper[5052]: I1203 12:28:47.447559 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:28:47 crc kubenswrapper[5052]: E1203 12:28:47.448372 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="b045ac1c-0a9b-47e7-b6d7-39963ed900ca" containerName="nova-api-log" Dec 03 12:28:47 crc kubenswrapper[5052]: I1203 12:28:47.448556 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="b045ac1c-0a9b-47e7-b6d7-39963ed900ca" containerName="nova-api-log" Dec 03 12:28:47 crc kubenswrapper[5052]: E1203 12:28:47.448658 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="b045ac1c-0a9b-47e7-b6d7-39963ed900ca" containerName="nova-api-api" Dec 03 12:28:47 crc kubenswrapper[5052]: I1203 12:28:47.449221 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="b045ac1c-0a9b-47e7-b6d7-39963ed900ca" containerName="nova-api-api" Dec 03 12:28:47 crc kubenswrapper[5052]: E1203 12:28:47.449314 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="ada0927b-5399-48af-8a11-ee31292b5d2a" containerName="nova-metadata-metadata" Dec 03 12:28:47 crc kubenswrapper[5052]: I1203 12:28:47.449371 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="ada0927b-5399-48af-8a11-ee31292b5d2a" containerName="nova-metadata-metadata" Dec 03 12:28:47 crc kubenswrapper[5052]: E1203 12:28:47.449427 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="5ae7c56e-3507-403b-b58a-1a6da1ed1ab3" containerName="nova-cell1-conductor-conductor" Dec 03 12:28:47 crc kubenswrapper[5052]: I1203 12:28:47.449477 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ae7c56e-3507-403b-b58a-1a6da1ed1ab3" containerName="nova-cell1-conductor-conductor" Dec 03 12:28:47 crc kubenswrapper[5052]: E1203 12:28:47.449556 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="ada0927b-5399-48af-8a11-ee31292b5d2a" containerName="nova-metadata-log" Dec 03 12:28:47 crc kubenswrapper[5052]: I1203 12:28:47.449642 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="ada0927b-5399-48af-8a11-ee31292b5d2a" containerName="nova-metadata-log" Dec 03 12:28:47 crc kubenswrapper[5052]: E1203 12:28:47.449702 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="b7c664ff-377a-4d83-a65e-6ecce5b9c9c0" containerName="nova-scheduler-scheduler" Dec 03 12:28:47 crc kubenswrapper[5052]: I1203 12:28:47.449766 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="b7c664ff-377a-4d83-a65e-6ecce5b9c9c0" containerName="nova-scheduler-scheduler" Dec 03 12:28:47 crc kubenswrapper[5052]: E1203 12:28:47.449826 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="bea08f45-6aef-4ccc-8252-d840baad2bf2" containerName="nova-cell1-novncproxy-novncproxy" Dec 03 12:28:47 crc kubenswrapper[5052]: I1203 12:28:47.449876 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="bea08f45-6aef-4ccc-8252-d840baad2bf2" containerName="nova-cell1-novncproxy-novncproxy" Dec 03 12:28:47 crc kubenswrapper[5052]: E1203 12:28:47.449947 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="b7c664ff-377a-4d83-a65e-6ecce5b9c9c0" containerName="nova-scheduler-scheduler" Dec 03 12:28:47 crc kubenswrapper[5052]: I1203 12:28:47.449999 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="b7c664ff-377a-4d83-a65e-6ecce5b9c9c0" containerName="nova-scheduler-scheduler" Dec 03 12:28:47 crc kubenswrapper[5052]: I1203 12:28:47.450214 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="5ae7c56e-3507-403b-b58a-1a6da1ed1ab3" containerName="nova-cell1-conductor-conductor" Dec 03 12:28:47 crc kubenswrapper[5052]: I1203 12:28:47.450289 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="b7c664ff-377a-4d83-a65e-6ecce5b9c9c0" containerName="nova-scheduler-scheduler" Dec 03 12:28:47 crc kubenswrapper[5052]: I1203 12:28:47.450347 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="ada0927b-5399-48af-8a11-ee31292b5d2a" containerName="nova-metadata-log" Dec 03 12:28:47 crc kubenswrapper[5052]: I1203 12:28:47.450409 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="bea08f45-6aef-4ccc-8252-d840baad2bf2" containerName="nova-cell1-novncproxy-novncproxy" Dec 03 12:28:47 crc kubenswrapper[5052]: I1203 12:28:47.450461 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="ada0927b-5399-48af-8a11-ee31292b5d2a" containerName="nova-metadata-metadata" Dec 03 12:28:47 crc kubenswrapper[5052]: I1203 12:28:47.450540 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="b7c664ff-377a-4d83-a65e-6ecce5b9c9c0" containerName="nova-scheduler-scheduler" Dec 03 12:28:47 crc kubenswrapper[5052]: I1203 12:28:47.450607 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="b045ac1c-0a9b-47e7-b6d7-39963ed900ca" containerName="nova-api-log" Dec 03 12:28:47 crc kubenswrapper[5052]: I1203 12:28:47.450690 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="b045ac1c-0a9b-47e7-b6d7-39963ed900ca" containerName="nova-api-api" Dec 03 12:28:47 crc kubenswrapper[5052]: I1203 12:28:47.451714 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:28:47 crc kubenswrapper[5052]: I1203 12:28:47.458382 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:28:47 crc kubenswrapper[5052]: I1203 12:28:47.468852 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-metadata-config-data" Dec 03 12:28:47 crc kubenswrapper[5052]: I1203 12:28:47.469031 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-nova-metadata-internal-svc" Dec 03 12:28:47 crc kubenswrapper[5052]: I1203 12:28:47.469570 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:28:47 crc kubenswrapper[5052]: I1203 12:28:47.478582 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:28:47 crc kubenswrapper[5052]: I1203 12:28:47.486794 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:28:47 crc kubenswrapper[5052]: I1203 12:28:47.488561 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:28:47 crc kubenswrapper[5052]: I1203 12:28:47.491379 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-nova-internal-svc" Dec 03 12:28:47 crc kubenswrapper[5052]: I1203 12:28:47.492224 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-nova-public-svc" Dec 03 12:28:47 crc kubenswrapper[5052]: I1203 12:28:47.496276 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/5ae7c56e-3507-403b-b58a-1a6da1ed1ab3-config-data\") pod \"5ae7c56e-3507-403b-b58a-1a6da1ed1ab3\" (UID: \"5ae7c56e-3507-403b-b58a-1a6da1ed1ab3\") " Dec 03 12:28:47 crc kubenswrapper[5052]: I1203 12:28:47.496540 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wdcfr\" (UniqueName: \"kubernetes.io/projected/5ae7c56e-3507-403b-b58a-1a6da1ed1ab3-kube-api-access-wdcfr\") pod \"5ae7c56e-3507-403b-b58a-1a6da1ed1ab3\" (UID: \"5ae7c56e-3507-403b-b58a-1a6da1ed1ab3\") " Dec 03 12:28:47 crc kubenswrapper[5052]: I1203 12:28:47.496701 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5ae7c56e-3507-403b-b58a-1a6da1ed1ab3-combined-ca-bundle\") pod \"5ae7c56e-3507-403b-b58a-1a6da1ed1ab3\" (UID: \"5ae7c56e-3507-403b-b58a-1a6da1ed1ab3\") " Dec 03 12:28:47 crc kubenswrapper[5052]: I1203 12:28:47.502126 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-api-config-data" Dec 03 12:28:47 crc kubenswrapper[5052]: I1203 12:28:47.529771 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5ae7c56e-3507-403b-b58a-1a6da1ed1ab3-kube-api-access-wdcfr" (OuterVolumeSpecName: "kube-api-access-wdcfr") pod "5ae7c56e-3507-403b-b58a-1a6da1ed1ab3" (UID: "5ae7c56e-3507-403b-b58a-1a6da1ed1ab3"). InnerVolumeSpecName "kube-api-access-wdcfr". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:28:47 crc kubenswrapper[5052]: I1203 12:28:47.547275 5052 scope.go:117] "RemoveContainer" containerID="8e5a6d9e4abfbaf5773d33e6944a2aef3bb8a26e7c727e56a7010bc0bdc6a2c6" Dec 03 12:28:47 crc kubenswrapper[5052]: I1203 12:28:47.568652 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5ae7c56e-3507-403b-b58a-1a6da1ed1ab3-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "5ae7c56e-3507-403b-b58a-1a6da1ed1ab3" (UID: "5ae7c56e-3507-403b-b58a-1a6da1ed1ab3"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:28:47 crc kubenswrapper[5052]: I1203 12:28:47.600093 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/b72919f4-0bc0-444b-9ddb-859d386a53b5-logs\") pod \"nova-metadata-0\" (UID: \"b72919f4-0bc0-444b-9ddb-859d386a53b5\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:28:47 crc kubenswrapper[5052]: I1203 12:28:47.600635 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l9cmf\" (UniqueName: \"kubernetes.io/projected/a740ecee-4c7e-4e42-bfb8-8119ffbfcca0-kube-api-access-l9cmf\") pod \"nova-api-0\" (UID: \"a740ecee-4c7e-4e42-bfb8-8119ffbfcca0\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:28:47 crc kubenswrapper[5052]: I1203 12:28:47.600732 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/a740ecee-4c7e-4e42-bfb8-8119ffbfcca0-internal-tls-certs\") pod \"nova-api-0\" (UID: \"a740ecee-4c7e-4e42-bfb8-8119ffbfcca0\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:28:47 crc kubenswrapper[5052]: I1203 12:28:47.600866 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/a740ecee-4c7e-4e42-bfb8-8119ffbfcca0-public-tls-certs\") pod \"nova-api-0\" (UID: \"a740ecee-4c7e-4e42-bfb8-8119ffbfcca0\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:28:47 crc kubenswrapper[5052]: I1203 12:28:47.600956 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a740ecee-4c7e-4e42-bfb8-8119ffbfcca0-config-data\") pod \"nova-api-0\" (UID: \"a740ecee-4c7e-4e42-bfb8-8119ffbfcca0\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:28:47 crc kubenswrapper[5052]: I1203 12:28:47.601049 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/a740ecee-4c7e-4e42-bfb8-8119ffbfcca0-logs\") pod \"nova-api-0\" (UID: \"a740ecee-4c7e-4e42-bfb8-8119ffbfcca0\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:28:47 crc kubenswrapper[5052]: I1203 12:28:47.601117 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b72919f4-0bc0-444b-9ddb-859d386a53b5-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"b72919f4-0bc0-444b-9ddb-859d386a53b5\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:28:47 crc kubenswrapper[5052]: I1203 12:28:47.601191 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tc8qm\" (UniqueName: \"kubernetes.io/projected/b72919f4-0bc0-444b-9ddb-859d386a53b5-kube-api-access-tc8qm\") pod \"nova-metadata-0\" (UID: \"b72919f4-0bc0-444b-9ddb-859d386a53b5\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:28:47 crc kubenswrapper[5052]: I1203 12:28:47.601278 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/b72919f4-0bc0-444b-9ddb-859d386a53b5-config-data\") pod \"nova-metadata-0\" (UID: \"b72919f4-0bc0-444b-9ddb-859d386a53b5\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:28:47 crc kubenswrapper[5052]: I1203 12:28:47.601361 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a740ecee-4c7e-4e42-bfb8-8119ffbfcca0-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"a740ecee-4c7e-4e42-bfb8-8119ffbfcca0\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:28:47 crc kubenswrapper[5052]: I1203 12:28:47.601890 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/b72919f4-0bc0-444b-9ddb-859d386a53b5-nova-metadata-tls-certs\") pod \"nova-metadata-0\" (UID: \"b72919f4-0bc0-444b-9ddb-859d386a53b5\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:28:47 crc kubenswrapper[5052]: I1203 12:28:47.602020 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5ae7c56e-3507-403b-b58a-1a6da1ed1ab3-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:47 crc kubenswrapper[5052]: I1203 12:28:47.602107 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-wdcfr\" (UniqueName: \"kubernetes.io/projected/5ae7c56e-3507-403b-b58a-1a6da1ed1ab3-kube-api-access-wdcfr\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:47 crc kubenswrapper[5052]: I1203 12:28:47.618658 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5ae7c56e-3507-403b-b58a-1a6da1ed1ab3-config-data" (OuterVolumeSpecName: "config-data") pod "5ae7c56e-3507-403b-b58a-1a6da1ed1ab3" (UID: "5ae7c56e-3507-403b-b58a-1a6da1ed1ab3"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:28:47 crc kubenswrapper[5052]: I1203 12:28:47.650587 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:28:47 crc kubenswrapper[5052]: I1203 12:28:47.695362 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ada0927b-5399-48af-8a11-ee31292b5d2a" path="/var/lib/kubelet/pods/ada0927b-5399-48af-8a11-ee31292b5d2a/volumes" Dec 03 12:28:47 crc kubenswrapper[5052]: I1203 12:28:47.697756 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b045ac1c-0a9b-47e7-b6d7-39963ed900ca" path="/var/lib/kubelet/pods/b045ac1c-0a9b-47e7-b6d7-39963ed900ca/volumes" Dec 03 12:28:47 crc kubenswrapper[5052]: I1203 12:28:47.698664 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:28:47 crc kubenswrapper[5052]: I1203 12:28:47.706470 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/b72919f4-0bc0-444b-9ddb-859d386a53b5-config-data\") pod \"nova-metadata-0\" (UID: \"b72919f4-0bc0-444b-9ddb-859d386a53b5\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:28:47 crc kubenswrapper[5052]: I1203 12:28:47.706565 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a740ecee-4c7e-4e42-bfb8-8119ffbfcca0-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"a740ecee-4c7e-4e42-bfb8-8119ffbfcca0\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:28:47 crc kubenswrapper[5052]: I1203 12:28:47.706602 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/b72919f4-0bc0-444b-9ddb-859d386a53b5-nova-metadata-tls-certs\") pod \"nova-metadata-0\" (UID: \"b72919f4-0bc0-444b-9ddb-859d386a53b5\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:28:47 crc kubenswrapper[5052]: I1203 12:28:47.706654 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/b72919f4-0bc0-444b-9ddb-859d386a53b5-logs\") pod \"nova-metadata-0\" (UID: \"b72919f4-0bc0-444b-9ddb-859d386a53b5\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:28:47 crc kubenswrapper[5052]: I1203 12:28:47.706712 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-l9cmf\" (UniqueName: \"kubernetes.io/projected/a740ecee-4c7e-4e42-bfb8-8119ffbfcca0-kube-api-access-l9cmf\") pod \"nova-api-0\" (UID: \"a740ecee-4c7e-4e42-bfb8-8119ffbfcca0\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:28:47 crc kubenswrapper[5052]: I1203 12:28:47.706738 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/a740ecee-4c7e-4e42-bfb8-8119ffbfcca0-internal-tls-certs\") pod \"nova-api-0\" (UID: \"a740ecee-4c7e-4e42-bfb8-8119ffbfcca0\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:28:47 crc kubenswrapper[5052]: I1203 12:28:47.706791 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/a740ecee-4c7e-4e42-bfb8-8119ffbfcca0-public-tls-certs\") pod \"nova-api-0\" (UID: \"a740ecee-4c7e-4e42-bfb8-8119ffbfcca0\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:28:47 crc kubenswrapper[5052]: I1203 12:28:47.706831 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a740ecee-4c7e-4e42-bfb8-8119ffbfcca0-config-data\") pod \"nova-api-0\" (UID: \"a740ecee-4c7e-4e42-bfb8-8119ffbfcca0\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:28:47 crc kubenswrapper[5052]: I1203 12:28:47.706859 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/a740ecee-4c7e-4e42-bfb8-8119ffbfcca0-logs\") pod \"nova-api-0\" (UID: \"a740ecee-4c7e-4e42-bfb8-8119ffbfcca0\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:28:47 crc kubenswrapper[5052]: I1203 12:28:47.706876 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b72919f4-0bc0-444b-9ddb-859d386a53b5-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"b72919f4-0bc0-444b-9ddb-859d386a53b5\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:28:47 crc kubenswrapper[5052]: I1203 12:28:47.706907 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-tc8qm\" (UniqueName: \"kubernetes.io/projected/b72919f4-0bc0-444b-9ddb-859d386a53b5-kube-api-access-tc8qm\") pod \"nova-metadata-0\" (UID: \"b72919f4-0bc0-444b-9ddb-859d386a53b5\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:28:47 crc kubenswrapper[5052]: I1203 12:28:47.706980 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/5ae7c56e-3507-403b-b58a-1a6da1ed1ab3-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:47 crc kubenswrapper[5052]: I1203 12:28:47.725059 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/a740ecee-4c7e-4e42-bfb8-8119ffbfcca0-logs\") pod \"nova-api-0\" (UID: \"a740ecee-4c7e-4e42-bfb8-8119ffbfcca0\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:28:47 crc kubenswrapper[5052]: I1203 12:28:47.725391 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/b72919f4-0bc0-444b-9ddb-859d386a53b5-logs\") pod \"nova-metadata-0\" (UID: \"b72919f4-0bc0-444b-9ddb-859d386a53b5\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:28:47 crc kubenswrapper[5052]: I1203 12:28:47.729309 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/b72919f4-0bc0-444b-9ddb-859d386a53b5-config-data\") pod \"nova-metadata-0\" (UID: \"b72919f4-0bc0-444b-9ddb-859d386a53b5\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:28:47 crc kubenswrapper[5052]: I1203 12:28:47.730711 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:28:47 crc kubenswrapper[5052]: I1203 12:28:47.750320 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/a740ecee-4c7e-4e42-bfb8-8119ffbfcca0-public-tls-certs\") pod \"nova-api-0\" (UID: \"a740ecee-4c7e-4e42-bfb8-8119ffbfcca0\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:28:47 crc kubenswrapper[5052]: I1203 12:28:47.750835 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a740ecee-4c7e-4e42-bfb8-8119ffbfcca0-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"a740ecee-4c7e-4e42-bfb8-8119ffbfcca0\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:28:47 crc kubenswrapper[5052]: I1203 12:28:47.751296 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-tc8qm\" (UniqueName: \"kubernetes.io/projected/b72919f4-0bc0-444b-9ddb-859d386a53b5-kube-api-access-tc8qm\") pod \"nova-metadata-0\" (UID: \"b72919f4-0bc0-444b-9ddb-859d386a53b5\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:28:47 crc kubenswrapper[5052]: I1203 12:28:47.751864 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b72919f4-0bc0-444b-9ddb-859d386a53b5-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"b72919f4-0bc0-444b-9ddb-859d386a53b5\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:28:47 crc kubenswrapper[5052]: I1203 12:28:47.752033 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a740ecee-4c7e-4e42-bfb8-8119ffbfcca0-config-data\") pod \"nova-api-0\" (UID: \"a740ecee-4c7e-4e42-bfb8-8119ffbfcca0\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:28:47 crc kubenswrapper[5052]: I1203 12:28:47.754706 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/a740ecee-4c7e-4e42-bfb8-8119ffbfcca0-internal-tls-certs\") pod \"nova-api-0\" (UID: \"a740ecee-4c7e-4e42-bfb8-8119ffbfcca0\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:28:47 crc kubenswrapper[5052]: I1203 12:28:47.754787 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/b72919f4-0bc0-444b-9ddb-859d386a53b5-nova-metadata-tls-certs\") pod \"nova-metadata-0\" (UID: \"b72919f4-0bc0-444b-9ddb-859d386a53b5\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:28:47 crc kubenswrapper[5052]: I1203 12:28:47.782498 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 12:28:47 crc kubenswrapper[5052]: I1203 12:28:47.797485 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-l9cmf\" (UniqueName: \"kubernetes.io/projected/a740ecee-4c7e-4e42-bfb8-8119ffbfcca0-kube-api-access-l9cmf\") pod \"nova-api-0\" (UID: \"a740ecee-4c7e-4e42-bfb8-8119ffbfcca0\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:28:47 crc kubenswrapper[5052]: I1203 12:28:47.836987 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:28:47 crc kubenswrapper[5052]: I1203 12:28:47.861945 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:28:47 crc kubenswrapper[5052]: I1203 12:28:47.870247 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:28:47 crc kubenswrapper[5052]: I1203 12:28:47.885370 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 12:28:47 crc kubenswrapper[5052]: I1203 12:28:47.896297 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:28:47 crc kubenswrapper[5052]: I1203 12:28:47.885950 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:28:47 crc kubenswrapper[5052]: I1203 12:28:47.903472 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-scheduler-config-data" Dec 03 12:28:47 crc kubenswrapper[5052]: I1203 12:28:47.912569 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 12:28:47 crc kubenswrapper[5052]: I1203 12:28:47.913925 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:28:47 crc kubenswrapper[5052]: I1203 12:28:47.917421 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-nova-novncproxy-cell1-public-svc" Dec 03 12:28:47 crc kubenswrapper[5052]: I1203 12:28:47.917676 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-nova-novncproxy-cell1-vencrypt" Dec 03 12:28:47 crc kubenswrapper[5052]: I1203 12:28:47.917803 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell1-novncproxy-config-data" Dec 03 12:28:47 crc kubenswrapper[5052]: I1203 12:28:47.922839 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sd7ql\" (UniqueName: \"kubernetes.io/projected/3a971df6-d01d-4f12-9867-a11a4a44a1ba-kube-api-access-sd7ql\") pod \"nova-scheduler-0\" (UID: \"3a971df6-d01d-4f12-9867-a11a4a44a1ba\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:28:47 crc kubenswrapper[5052]: I1203 12:28:47.923039 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/3a971df6-d01d-4f12-9867-a11a4a44a1ba-config-data\") pod \"nova-scheduler-0\" (UID: \"3a971df6-d01d-4f12-9867-a11a4a44a1ba\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:28:47 crc kubenswrapper[5052]: I1203 12:28:47.923148 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3a971df6-d01d-4f12-9867-a11a4a44a1ba-combined-ca-bundle\") pod \"nova-scheduler-0\" (UID: \"3a971df6-d01d-4f12-9867-a11a4a44a1ba\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:28:47 crc kubenswrapper[5052]: I1203 12:28:47.927729 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 12:28:48 crc kubenswrapper[5052]: I1203 12:28:48.025111 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/3a971df6-d01d-4f12-9867-a11a4a44a1ba-config-data\") pod \"nova-scheduler-0\" (UID: \"3a971df6-d01d-4f12-9867-a11a4a44a1ba\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:28:48 crc kubenswrapper[5052]: I1203 12:28:48.025182 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3a971df6-d01d-4f12-9867-a11a4a44a1ba-combined-ca-bundle\") pod \"nova-scheduler-0\" (UID: \"3a971df6-d01d-4f12-9867-a11a4a44a1ba\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:28:48 crc kubenswrapper[5052]: I1203 12:28:48.025207 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f21db826-d275-4464-937c-113fc7ddab00-combined-ca-bundle\") pod \"nova-cell1-novncproxy-0\" (UID: \"f21db826-d275-4464-937c-113fc7ddab00\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:28:48 crc kubenswrapper[5052]: I1203 12:28:48.025267 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mh9xd\" (UniqueName: \"kubernetes.io/projected/f21db826-d275-4464-937c-113fc7ddab00-kube-api-access-mh9xd\") pod \"nova-cell1-novncproxy-0\" (UID: \"f21db826-d275-4464-937c-113fc7ddab00\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:28:48 crc kubenswrapper[5052]: I1203 12:28:48.025294 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"vencrypt-tls-certs\" (UniqueName: \"kubernetes.io/secret/f21db826-d275-4464-937c-113fc7ddab00-vencrypt-tls-certs\") pod \"nova-cell1-novncproxy-0\" (UID: \"f21db826-d275-4464-937c-113fc7ddab00\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:28:48 crc kubenswrapper[5052]: I1203 12:28:48.025342 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-sd7ql\" (UniqueName: \"kubernetes.io/projected/3a971df6-d01d-4f12-9867-a11a4a44a1ba-kube-api-access-sd7ql\") pod \"nova-scheduler-0\" (UID: \"3a971df6-d01d-4f12-9867-a11a4a44a1ba\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:28:48 crc kubenswrapper[5052]: I1203 12:28:48.025363 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nova-novncproxy-tls-certs\" (UniqueName: \"kubernetes.io/secret/f21db826-d275-4464-937c-113fc7ddab00-nova-novncproxy-tls-certs\") pod \"nova-cell1-novncproxy-0\" (UID: \"f21db826-d275-4464-937c-113fc7ddab00\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:28:48 crc kubenswrapper[5052]: I1203 12:28:48.025420 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/f21db826-d275-4464-937c-113fc7ddab00-config-data\") pod \"nova-cell1-novncproxy-0\" (UID: \"f21db826-d275-4464-937c-113fc7ddab00\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:28:48 crc kubenswrapper[5052]: I1203 12:28:48.034093 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/3a971df6-d01d-4f12-9867-a11a4a44a1ba-config-data\") pod \"nova-scheduler-0\" (UID: \"3a971df6-d01d-4f12-9867-a11a4a44a1ba\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:28:48 crc kubenswrapper[5052]: I1203 12:28:48.034849 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3a971df6-d01d-4f12-9867-a11a4a44a1ba-combined-ca-bundle\") pod \"nova-scheduler-0\" (UID: \"3a971df6-d01d-4f12-9867-a11a4a44a1ba\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:28:48 crc kubenswrapper[5052]: I1203 12:28:48.050564 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-sd7ql\" (UniqueName: \"kubernetes.io/projected/3a971df6-d01d-4f12-9867-a11a4a44a1ba-kube-api-access-sd7ql\") pod \"nova-scheduler-0\" (UID: \"3a971df6-d01d-4f12-9867-a11a4a44a1ba\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:28:48 crc kubenswrapper[5052]: I1203 12:28:48.126884 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"nova-novncproxy-tls-certs\" (UniqueName: \"kubernetes.io/secret/f21db826-d275-4464-937c-113fc7ddab00-nova-novncproxy-tls-certs\") pod \"nova-cell1-novncproxy-0\" (UID: \"f21db826-d275-4464-937c-113fc7ddab00\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:28:48 crc kubenswrapper[5052]: I1203 12:28:48.126957 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/f21db826-d275-4464-937c-113fc7ddab00-config-data\") pod \"nova-cell1-novncproxy-0\" (UID: \"f21db826-d275-4464-937c-113fc7ddab00\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:28:48 crc kubenswrapper[5052]: I1203 12:28:48.127050 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f21db826-d275-4464-937c-113fc7ddab00-combined-ca-bundle\") pod \"nova-cell1-novncproxy-0\" (UID: \"f21db826-d275-4464-937c-113fc7ddab00\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:28:48 crc kubenswrapper[5052]: I1203 12:28:48.127090 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-mh9xd\" (UniqueName: \"kubernetes.io/projected/f21db826-d275-4464-937c-113fc7ddab00-kube-api-access-mh9xd\") pod \"nova-cell1-novncproxy-0\" (UID: \"f21db826-d275-4464-937c-113fc7ddab00\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:28:48 crc kubenswrapper[5052]: I1203 12:28:48.127129 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"vencrypt-tls-certs\" (UniqueName: \"kubernetes.io/secret/f21db826-d275-4464-937c-113fc7ddab00-vencrypt-tls-certs\") pod \"nova-cell1-novncproxy-0\" (UID: \"f21db826-d275-4464-937c-113fc7ddab00\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:28:48 crc kubenswrapper[5052]: I1203 12:28:48.136188 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"vencrypt-tls-certs\" (UniqueName: \"kubernetes.io/secret/f21db826-d275-4464-937c-113fc7ddab00-vencrypt-tls-certs\") pod \"nova-cell1-novncproxy-0\" (UID: \"f21db826-d275-4464-937c-113fc7ddab00\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:28:48 crc kubenswrapper[5052]: I1203 12:28:48.136288 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f21db826-d275-4464-937c-113fc7ddab00-combined-ca-bundle\") pod \"nova-cell1-novncproxy-0\" (UID: \"f21db826-d275-4464-937c-113fc7ddab00\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:28:48 crc kubenswrapper[5052]: I1203 12:28:48.140089 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"nova-novncproxy-tls-certs\" (UniqueName: \"kubernetes.io/secret/f21db826-d275-4464-937c-113fc7ddab00-nova-novncproxy-tls-certs\") pod \"nova-cell1-novncproxy-0\" (UID: \"f21db826-d275-4464-937c-113fc7ddab00\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:28:48 crc kubenswrapper[5052]: I1203 12:28:48.140705 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/f21db826-d275-4464-937c-113fc7ddab00-config-data\") pod \"nova-cell1-novncproxy-0\" (UID: \"f21db826-d275-4464-937c-113fc7ddab00\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:28:48 crc kubenswrapper[5052]: I1203 12:28:48.151805 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-mh9xd\" (UniqueName: \"kubernetes.io/projected/f21db826-d275-4464-937c-113fc7ddab00-kube-api-access-mh9xd\") pod \"nova-cell1-novncproxy-0\" (UID: \"f21db826-d275-4464-937c-113fc7ddab00\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:28:48 crc kubenswrapper[5052]: I1203 12:28:48.334942 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:28:48 crc kubenswrapper[5052]: I1203 12:28:48.346973 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:28:48 crc kubenswrapper[5052]: I1203 12:28:48.377828 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-conductor-0" event={"ID":"5ae7c56e-3507-403b-b58a-1a6da1ed1ab3","Type":"ContainerDied","Data":"b5d7aeb00cbc1f82d726e8263d9745fee3d43b3b5dca68f30f648ab2bc48da8c"} Dec 03 12:28:48 crc kubenswrapper[5052]: I1203 12:28:48.377863 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:28:48 crc kubenswrapper[5052]: I1203 12:28:48.377891 5052 scope.go:117] "RemoveContainer" containerID="61a4505e879a4045e7b22afd402d01101f72aedd5245a9a876b0255b885b15ed" Dec 03 12:28:48 crc kubenswrapper[5052]: I1203 12:28:48.409175 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-0"] Dec 03 12:28:48 crc kubenswrapper[5052]: I1203 12:28:48.438972 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-0"] Dec 03 12:28:48 crc kubenswrapper[5052]: I1203 12:28:48.461189 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:28:48 crc kubenswrapper[5052]: W1203 12:28:48.475907 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb72919f4_0bc0_444b_9ddb_859d386a53b5.slice/crio-80bea364197d6323eeb3e89869ba88bf1c35cf5d017cae5e8b46ea7daead6f7e WatchSource:0}: Error finding container 80bea364197d6323eeb3e89869ba88bf1c35cf5d017cae5e8b46ea7daead6f7e: Status 404 returned error can't find the container with id 80bea364197d6323eeb3e89869ba88bf1c35cf5d017cae5e8b46ea7daead6f7e Dec 03 12:28:48 crc kubenswrapper[5052]: I1203 12:28:48.482982 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-0"] Dec 03 12:28:48 crc kubenswrapper[5052]: I1203 12:28:48.484917 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:28:48 crc kubenswrapper[5052]: I1203 12:28:48.490490 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell1-conductor-config-data" Dec 03 12:28:48 crc kubenswrapper[5052]: I1203 12:28:48.511043 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-0"] Dec 03 12:28:48 crc kubenswrapper[5052]: I1203 12:28:48.535713 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/026d5ba1-5f6f-4f39-bcb6-9c51d37608f2-config-data\") pod \"nova-cell1-conductor-0\" (UID: \"026d5ba1-5f6f-4f39-bcb6-9c51d37608f2\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:28:48 crc kubenswrapper[5052]: I1203 12:28:48.535775 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/026d5ba1-5f6f-4f39-bcb6-9c51d37608f2-combined-ca-bundle\") pod \"nova-cell1-conductor-0\" (UID: \"026d5ba1-5f6f-4f39-bcb6-9c51d37608f2\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:28:48 crc kubenswrapper[5052]: I1203 12:28:48.535868 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cpggs\" (UniqueName: \"kubernetes.io/projected/026d5ba1-5f6f-4f39-bcb6-9c51d37608f2-kube-api-access-cpggs\") pod \"nova-cell1-conductor-0\" (UID: \"026d5ba1-5f6f-4f39-bcb6-9c51d37608f2\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:28:48 crc kubenswrapper[5052]: I1203 12:28:48.549110 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:28:48 crc kubenswrapper[5052]: W1203 12:28:48.582296 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda740ecee_4c7e_4e42_bfb8_8119ffbfcca0.slice/crio-ef4460d4df17c0f7628e13a6d5f896627b209d3d05c91e600735eaaa42f2d703 WatchSource:0}: Error finding container ef4460d4df17c0f7628e13a6d5f896627b209d3d05c91e600735eaaa42f2d703: Status 404 returned error can't find the container with id ef4460d4df17c0f7628e13a6d5f896627b209d3d05c91e600735eaaa42f2d703 Dec 03 12:28:48 crc kubenswrapper[5052]: I1203 12:28:48.637077 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/026d5ba1-5f6f-4f39-bcb6-9c51d37608f2-combined-ca-bundle\") pod \"nova-cell1-conductor-0\" (UID: \"026d5ba1-5f6f-4f39-bcb6-9c51d37608f2\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:28:48 crc kubenswrapper[5052]: I1203 12:28:48.637166 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-cpggs\" (UniqueName: \"kubernetes.io/projected/026d5ba1-5f6f-4f39-bcb6-9c51d37608f2-kube-api-access-cpggs\") pod \"nova-cell1-conductor-0\" (UID: \"026d5ba1-5f6f-4f39-bcb6-9c51d37608f2\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:28:48 crc kubenswrapper[5052]: I1203 12:28:48.637251 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/026d5ba1-5f6f-4f39-bcb6-9c51d37608f2-config-data\") pod \"nova-cell1-conductor-0\" (UID: \"026d5ba1-5f6f-4f39-bcb6-9c51d37608f2\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:28:48 crc kubenswrapper[5052]: I1203 12:28:48.643238 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/026d5ba1-5f6f-4f39-bcb6-9c51d37608f2-combined-ca-bundle\") pod \"nova-cell1-conductor-0\" (UID: \"026d5ba1-5f6f-4f39-bcb6-9c51d37608f2\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:28:48 crc kubenswrapper[5052]: I1203 12:28:48.643942 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/026d5ba1-5f6f-4f39-bcb6-9c51d37608f2-config-data\") pod \"nova-cell1-conductor-0\" (UID: \"026d5ba1-5f6f-4f39-bcb6-9c51d37608f2\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:28:48 crc kubenswrapper[5052]: I1203 12:28:48.658594 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-cpggs\" (UniqueName: \"kubernetes.io/projected/026d5ba1-5f6f-4f39-bcb6-9c51d37608f2-kube-api-access-cpggs\") pod \"nova-cell1-conductor-0\" (UID: \"026d5ba1-5f6f-4f39-bcb6-9c51d37608f2\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:28:48 crc kubenswrapper[5052]: I1203 12:28:48.822039 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:28:48 crc kubenswrapper[5052]: I1203 12:28:48.917245 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:28:49 crc kubenswrapper[5052]: I1203 12:28:49.127028 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 12:28:49 crc kubenswrapper[5052]: W1203 12:28:49.153933 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf21db826_d275_4464_937c_113fc7ddab00.slice/crio-4301a172b5076b2f8d49cc867c7ad17dac5888bd3fd98f2a97eca78fff4e2d33 WatchSource:0}: Error finding container 4301a172b5076b2f8d49cc867c7ad17dac5888bd3fd98f2a97eca78fff4e2d33: Status 404 returned error can't find the container with id 4301a172b5076b2f8d49cc867c7ad17dac5888bd3fd98f2a97eca78fff4e2d33 Dec 03 12:28:49 crc kubenswrapper[5052]: I1203 12:28:49.288353 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:28:49 crc kubenswrapper[5052]: I1203 12:28:49.428649 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:28:49 crc kubenswrapper[5052]: I1203 12:28:49.454589 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:28:49 crc kubenswrapper[5052]: I1203 12:28:49.460486 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6efd22cd-98c7-48d8-a346-289b78e8aa1c-config-data\") pod \"6efd22cd-98c7-48d8-a346-289b78e8aa1c\" (UID: \"6efd22cd-98c7-48d8-a346-289b78e8aa1c\") " Dec 03 12:28:49 crc kubenswrapper[5052]: I1203 12:28:49.460766 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6efd22cd-98c7-48d8-a346-289b78e8aa1c-combined-ca-bundle\") pod \"6efd22cd-98c7-48d8-a346-289b78e8aa1c\" (UID: \"6efd22cd-98c7-48d8-a346-289b78e8aa1c\") " Dec 03 12:28:49 crc kubenswrapper[5052]: I1203 12:28:49.460816 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bgp8v\" (UniqueName: \"kubernetes.io/projected/6efd22cd-98c7-48d8-a346-289b78e8aa1c-kube-api-access-bgp8v\") pod \"6efd22cd-98c7-48d8-a346-289b78e8aa1c\" (UID: \"6efd22cd-98c7-48d8-a346-289b78e8aa1c\") " Dec 03 12:28:49 crc kubenswrapper[5052]: I1203 12:28:49.466901 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"a740ecee-4c7e-4e42-bfb8-8119ffbfcca0","Type":"ContainerStarted","Data":"a4444872bf34dcdac7f5d8135bb76c3cc02268af96f1647ecca10b5d17728ea9"} Dec 03 12:28:49 crc kubenswrapper[5052]: I1203 12:28:49.466952 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"a740ecee-4c7e-4e42-bfb8-8119ffbfcca0","Type":"ContainerStarted","Data":"ef4460d4df17c0f7628e13a6d5f896627b209d3d05c91e600735eaaa42f2d703"} Dec 03 12:28:49 crc kubenswrapper[5052]: I1203 12:28:49.469933 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6efd22cd-98c7-48d8-a346-289b78e8aa1c-kube-api-access-bgp8v" (OuterVolumeSpecName: "kube-api-access-bgp8v") pod "6efd22cd-98c7-48d8-a346-289b78e8aa1c" (UID: "6efd22cd-98c7-48d8-a346-289b78e8aa1c"). InnerVolumeSpecName "kube-api-access-bgp8v". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:28:49 crc kubenswrapper[5052]: I1203 12:28:49.473746 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" event={"ID":"f21db826-d275-4464-937c-113fc7ddab00","Type":"ContainerStarted","Data":"4301a172b5076b2f8d49cc867c7ad17dac5888bd3fd98f2a97eca78fff4e2d33"} Dec 03 12:28:49 crc kubenswrapper[5052]: I1203 12:28:49.473795 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:28:49 crc kubenswrapper[5052]: I1203 12:28:49.486814 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 12:28:49 crc kubenswrapper[5052]: I1203 12:28:49.493266 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"b72919f4-0bc0-444b-9ddb-859d386a53b5","Type":"ContainerStarted","Data":"ba961703896a39a5fa95d99d5f68e5bc3cd82febfad30c6d56817cbc86b81596"} Dec 03 12:28:49 crc kubenswrapper[5052]: I1203 12:28:49.493309 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"b72919f4-0bc0-444b-9ddb-859d386a53b5","Type":"ContainerStarted","Data":"196d809c625b6d7be7a93108322dbb398749088ab5233d8d6400d8ad5534eed2"} Dec 03 12:28:49 crc kubenswrapper[5052]: I1203 12:28:49.493320 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"b72919f4-0bc0-444b-9ddb-859d386a53b5","Type":"ContainerStarted","Data":"80bea364197d6323eeb3e89869ba88bf1c35cf5d017cae5e8b46ea7daead6f7e"} Dec 03 12:28:49 crc kubenswrapper[5052]: I1203 12:28:49.501306 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"3a971df6-d01d-4f12-9867-a11a4a44a1ba","Type":"ContainerStarted","Data":"cbe88589a81e849f6f0daa7ff8ef7a534e510717cf1dea7a764f871758401466"} Dec 03 12:28:49 crc kubenswrapper[5052]: I1203 12:28:49.504555 5052 generic.go:334] "Generic (PLEG): container finished" podID="6efd22cd-98c7-48d8-a346-289b78e8aa1c" containerID="ae1f276ec0ccddfcdb84f346890a9d2aa8e21c3960425cd4ca2a6483bc8b8ae7" exitCode=0 Dec 03 12:28:49 crc kubenswrapper[5052]: I1203 12:28:49.504596 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-conductor-0" event={"ID":"6efd22cd-98c7-48d8-a346-289b78e8aa1c","Type":"ContainerDied","Data":"ae1f276ec0ccddfcdb84f346890a9d2aa8e21c3960425cd4ca2a6483bc8b8ae7"} Dec 03 12:28:49 crc kubenswrapper[5052]: I1203 12:28:49.504617 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-conductor-0" event={"ID":"6efd22cd-98c7-48d8-a346-289b78e8aa1c","Type":"ContainerDied","Data":"85c3ce2cbe0bd990ceed107cc324a44256bec481856bf0d9c8cdfe468f9e3a70"} Dec 03 12:28:49 crc kubenswrapper[5052]: I1203 12:28:49.504632 5052 scope.go:117] "RemoveContainer" containerID="ae1f276ec0ccddfcdb84f346890a9d2aa8e21c3960425cd4ca2a6483bc8b8ae7" Dec 03 12:28:49 crc kubenswrapper[5052]: I1203 12:28:49.504630 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:28:49 crc kubenswrapper[5052]: I1203 12:28:49.531019 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-0"] Dec 03 12:28:49 crc kubenswrapper[5052]: I1203 12:28:49.539830 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-metadata-0" podStartSLOduration=2.539811522 podStartE2EDuration="2.539811522s" podCreationTimestamp="2025-12-03 12:28:47 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:28:49.513822481 +0000 UTC m=+4120.102064245" watchObservedRunningTime="2025-12-03 12:28:49.539811522 +0000 UTC m=+4120.128053286" Dec 03 12:28:49 crc kubenswrapper[5052]: I1203 12:28:49.549290 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-scheduler-0" podStartSLOduration=2.549233351 podStartE2EDuration="2.549233351s" podCreationTimestamp="2025-12-03 12:28:47 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:28:49.530987931 +0000 UTC m=+4120.119229695" watchObservedRunningTime="2025-12-03 12:28:49.549233351 +0000 UTC m=+4120.137475115" Dec 03 12:28:49 crc kubenswrapper[5052]: I1203 12:28:49.566074 5052 scope.go:117] "RemoveContainer" containerID="ae1f276ec0ccddfcdb84f346890a9d2aa8e21c3960425cd4ca2a6483bc8b8ae7" Dec 03 12:28:49 crc kubenswrapper[5052]: I1203 12:28:49.566987 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-bgp8v\" (UniqueName: \"kubernetes.io/projected/6efd22cd-98c7-48d8-a346-289b78e8aa1c-kube-api-access-bgp8v\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:49 crc kubenswrapper[5052]: E1203 12:28:49.567729 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ae1f276ec0ccddfcdb84f346890a9d2aa8e21c3960425cd4ca2a6483bc8b8ae7\": container with ID starting with ae1f276ec0ccddfcdb84f346890a9d2aa8e21c3960425cd4ca2a6483bc8b8ae7 not found: ID does not exist" containerID="ae1f276ec0ccddfcdb84f346890a9d2aa8e21c3960425cd4ca2a6483bc8b8ae7" Dec 03 12:28:49 crc kubenswrapper[5052]: I1203 12:28:49.567782 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ae1f276ec0ccddfcdb84f346890a9d2aa8e21c3960425cd4ca2a6483bc8b8ae7"} err="failed to get container status \"ae1f276ec0ccddfcdb84f346890a9d2aa8e21c3960425cd4ca2a6483bc8b8ae7\": rpc error: code = NotFound desc = could not find container \"ae1f276ec0ccddfcdb84f346890a9d2aa8e21c3960425cd4ca2a6483bc8b8ae7\": container with ID starting with ae1f276ec0ccddfcdb84f346890a9d2aa8e21c3960425cd4ca2a6483bc8b8ae7 not found: ID does not exist" Dec 03 12:28:49 crc kubenswrapper[5052]: I1203 12:28:49.578173 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-0"] Dec 03 12:28:49 crc kubenswrapper[5052]: I1203 12:28:49.612078 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6efd22cd-98c7-48d8-a346-289b78e8aa1c-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "6efd22cd-98c7-48d8-a346-289b78e8aa1c" (UID: "6efd22cd-98c7-48d8-a346-289b78e8aa1c"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:28:49 crc kubenswrapper[5052]: I1203 12:28:49.628897 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6efd22cd-98c7-48d8-a346-289b78e8aa1c-config-data" (OuterVolumeSpecName: "config-data") pod "6efd22cd-98c7-48d8-a346-289b78e8aa1c" (UID: "6efd22cd-98c7-48d8-a346-289b78e8aa1c"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:28:49 crc kubenswrapper[5052]: I1203 12:28:49.669194 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6efd22cd-98c7-48d8-a346-289b78e8aa1c-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:49 crc kubenswrapper[5052]: I1203 12:28:49.669230 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6efd22cd-98c7-48d8-a346-289b78e8aa1c-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:49 crc kubenswrapper[5052]: I1203 12:28:49.796682 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5ae7c56e-3507-403b-b58a-1a6da1ed1ab3" path="/var/lib/kubelet/pods/5ae7c56e-3507-403b-b58a-1a6da1ed1ab3/volumes" Dec 03 12:28:49 crc kubenswrapper[5052]: I1203 12:28:49.797210 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b7c664ff-377a-4d83-a65e-6ecce5b9c9c0" path="/var/lib/kubelet/pods/b7c664ff-377a-4d83-a65e-6ecce5b9c9c0/volumes" Dec 03 12:28:49 crc kubenswrapper[5052]: I1203 12:28:49.797697 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bea08f45-6aef-4ccc-8252-d840baad2bf2" path="/var/lib/kubelet/pods/bea08f45-6aef-4ccc-8252-d840baad2bf2/volumes" Dec 03 12:28:49 crc kubenswrapper[5052]: I1203 12:28:49.897699 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-0"] Dec 03 12:28:49 crc kubenswrapper[5052]: I1203 12:28:49.921208 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-0"] Dec 03 12:28:49 crc kubenswrapper[5052]: I1203 12:28:49.948755 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-0"] Dec 03 12:28:49 crc kubenswrapper[5052]: E1203 12:28:49.949297 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="6efd22cd-98c7-48d8-a346-289b78e8aa1c" containerName="nova-cell0-conductor-conductor" Dec 03 12:28:49 crc kubenswrapper[5052]: I1203 12:28:49.949321 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="6efd22cd-98c7-48d8-a346-289b78e8aa1c" containerName="nova-cell0-conductor-conductor" Dec 03 12:28:49 crc kubenswrapper[5052]: I1203 12:28:49.949632 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="6efd22cd-98c7-48d8-a346-289b78e8aa1c" containerName="nova-cell0-conductor-conductor" Dec 03 12:28:49 crc kubenswrapper[5052]: I1203 12:28:49.950500 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:28:49 crc kubenswrapper[5052]: I1203 12:28:49.953309 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell0-conductor-config-data" Dec 03 12:28:49 crc kubenswrapper[5052]: I1203 12:28:49.969062 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-0"] Dec 03 12:28:50 crc kubenswrapper[5052]: I1203 12:28:49.997926 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:28:50 crc kubenswrapper[5052]: I1203 12:28:49.998663 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="e7e769bc-cba8-4955-9d3d-14d1acc7fb5b" containerName="proxy-httpd" containerID="cri-o://ed335dc5276810aababf7754b74b1511a5d7d1fa065ad7710e9fe074646e5384" gracePeriod=30 Dec 03 12:28:50 crc kubenswrapper[5052]: I1203 12:28:49.998715 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="e7e769bc-cba8-4955-9d3d-14d1acc7fb5b" containerName="sg-core" containerID="cri-o://94e6dc8d77f6943b437cb5870adcaa6c29d491e693bc37360bb9fd740b7610f1" gracePeriod=30 Dec 03 12:28:50 crc kubenswrapper[5052]: I1203 12:28:49.998808 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="e7e769bc-cba8-4955-9d3d-14d1acc7fb5b" containerName="ceilometer-notification-agent" containerID="cri-o://14273699743c5c98bca470132190fea0be82840c80649fe7f4e02e858e70da72" gracePeriod=30 Dec 03 12:28:50 crc kubenswrapper[5052]: I1203 12:28:50.000210 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="e7e769bc-cba8-4955-9d3d-14d1acc7fb5b" containerName="ceilometer-central-agent" containerID="cri-o://ce04bf8dc26d500c7eeaadff490cd8b971ffdcceacc5c68dc3033bfb0a4f5e56" gracePeriod=30 Dec 03 12:28:50 crc kubenswrapper[5052]: I1203 12:28:50.089571 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/211396f3-156a-4ac5-917b-6d41c236d3c2-config-data\") pod \"nova-cell0-conductor-0\" (UID: \"211396f3-156a-4ac5-917b-6d41c236d3c2\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:28:50 crc kubenswrapper[5052]: I1203 12:28:50.089675 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fgwwj\" (UniqueName: \"kubernetes.io/projected/211396f3-156a-4ac5-917b-6d41c236d3c2-kube-api-access-fgwwj\") pod \"nova-cell0-conductor-0\" (UID: \"211396f3-156a-4ac5-917b-6d41c236d3c2\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:28:50 crc kubenswrapper[5052]: I1203 12:28:50.089733 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/211396f3-156a-4ac5-917b-6d41c236d3c2-combined-ca-bundle\") pod \"nova-cell0-conductor-0\" (UID: \"211396f3-156a-4ac5-917b-6d41c236d3c2\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:28:50 crc kubenswrapper[5052]: I1203 12:28:50.191066 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-fgwwj\" (UniqueName: \"kubernetes.io/projected/211396f3-156a-4ac5-917b-6d41c236d3c2-kube-api-access-fgwwj\") pod \"nova-cell0-conductor-0\" (UID: \"211396f3-156a-4ac5-917b-6d41c236d3c2\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:28:50 crc kubenswrapper[5052]: I1203 12:28:50.191177 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/211396f3-156a-4ac5-917b-6d41c236d3c2-combined-ca-bundle\") pod \"nova-cell0-conductor-0\" (UID: \"211396f3-156a-4ac5-917b-6d41c236d3c2\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:28:50 crc kubenswrapper[5052]: I1203 12:28:50.191300 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/211396f3-156a-4ac5-917b-6d41c236d3c2-config-data\") pod \"nova-cell0-conductor-0\" (UID: \"211396f3-156a-4ac5-917b-6d41c236d3c2\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:28:50 crc kubenswrapper[5052]: I1203 12:28:50.206690 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/211396f3-156a-4ac5-917b-6d41c236d3c2-config-data\") pod \"nova-cell0-conductor-0\" (UID: \"211396f3-156a-4ac5-917b-6d41c236d3c2\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:28:50 crc kubenswrapper[5052]: I1203 12:28:50.212083 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/211396f3-156a-4ac5-917b-6d41c236d3c2-combined-ca-bundle\") pod \"nova-cell0-conductor-0\" (UID: \"211396f3-156a-4ac5-917b-6d41c236d3c2\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:28:50 crc kubenswrapper[5052]: I1203 12:28:50.220354 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-fgwwj\" (UniqueName: \"kubernetes.io/projected/211396f3-156a-4ac5-917b-6d41c236d3c2-kube-api-access-fgwwj\") pod \"nova-cell0-conductor-0\" (UID: \"211396f3-156a-4ac5-917b-6d41c236d3c2\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:28:50 crc kubenswrapper[5052]: I1203 12:28:50.292273 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:28:50 crc kubenswrapper[5052]: I1203 12:28:50.522406 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"3a971df6-d01d-4f12-9867-a11a4a44a1ba","Type":"ContainerStarted","Data":"1cbec8b86689f5bf35e48f86fab05f7750b27f7dd4f48552f1d8c25b1c37adb2"} Dec 03 12:28:50 crc kubenswrapper[5052]: I1203 12:28:50.522835 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-scheduler-0" podUID="3a971df6-d01d-4f12-9867-a11a4a44a1ba" containerName="nova-scheduler-scheduler" containerID="cri-o://1cbec8b86689f5bf35e48f86fab05f7750b27f7dd4f48552f1d8c25b1c37adb2" gracePeriod=30 Dec 03 12:28:50 crc kubenswrapper[5052]: I1203 12:28:50.540603 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"a740ecee-4c7e-4e42-bfb8-8119ffbfcca0","Type":"ContainerStarted","Data":"9266cf294e7ac6a94b81960b8647a5807b0b2d54f0c4b9ad133adc1aa1aef8e8"} Dec 03 12:28:50 crc kubenswrapper[5052]: I1203 12:28:50.540774 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-api-0" podUID="a740ecee-4c7e-4e42-bfb8-8119ffbfcca0" containerName="nova-api-log" containerID="cri-o://a4444872bf34dcdac7f5d8135bb76c3cc02268af96f1647ecca10b5d17728ea9" gracePeriod=30 Dec 03 12:28:50 crc kubenswrapper[5052]: I1203 12:28:50.541306 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-api-0" podUID="a740ecee-4c7e-4e42-bfb8-8119ffbfcca0" containerName="nova-api-api" containerID="cri-o://9266cf294e7ac6a94b81960b8647a5807b0b2d54f0c4b9ad133adc1aa1aef8e8" gracePeriod=30 Dec 03 12:28:50 crc kubenswrapper[5052]: I1203 12:28:50.556601 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-conductor-0" event={"ID":"026d5ba1-5f6f-4f39-bcb6-9c51d37608f2","Type":"ContainerStarted","Data":"f0d03487bb9b0001a9b06ddae0262ca9d26eb73013d9bccc682cbc0b7a9e1bbe"} Dec 03 12:28:50 crc kubenswrapper[5052]: I1203 12:28:50.556674 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-conductor-0" event={"ID":"026d5ba1-5f6f-4f39-bcb6-9c51d37608f2","Type":"ContainerStarted","Data":"e00e8647a336fdd69de7a88c71814a98d53ec41e92ad473debb2ba71e491f11c"} Dec 03 12:28:50 crc kubenswrapper[5052]: I1203 12:28:50.556859 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-cell1-conductor-0" podUID="026d5ba1-5f6f-4f39-bcb6-9c51d37608f2" containerName="nova-cell1-conductor-conductor" containerID="cri-o://f0d03487bb9b0001a9b06ddae0262ca9d26eb73013d9bccc682cbc0b7a9e1bbe" gracePeriod=30 Dec 03 12:28:50 crc kubenswrapper[5052]: I1203 12:28:50.557126 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:28:50 crc kubenswrapper[5052]: I1203 12:28:50.575370 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" event={"ID":"f21db826-d275-4464-937c-113fc7ddab00","Type":"ContainerStarted","Data":"458afe5396f5d654ae2179515a4f44886989891f21a1c00342344e6ced981d50"} Dec 03 12:28:50 crc kubenswrapper[5052]: I1203 12:28:50.575577 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" podUID="f21db826-d275-4464-937c-113fc7ddab00" containerName="nova-cell1-novncproxy-novncproxy" containerID="cri-o://458afe5396f5d654ae2179515a4f44886989891f21a1c00342344e6ced981d50" gracePeriod=30 Dec 03 12:28:50 crc kubenswrapper[5052]: I1203 12:28:50.592980 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-api-0" podStartSLOduration=3.592955952 podStartE2EDuration="3.592955952s" podCreationTimestamp="2025-12-03 12:28:47 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:28:50.571630124 +0000 UTC m=+4121.159871918" watchObservedRunningTime="2025-12-03 12:28:50.592955952 +0000 UTC m=+4121.181197716" Dec 03 12:28:50 crc kubenswrapper[5052]: I1203 12:28:50.597199 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-cell1-conductor-0" podStartSLOduration=2.597182993 podStartE2EDuration="2.597182993s" podCreationTimestamp="2025-12-03 12:28:48 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:28:50.595643019 +0000 UTC m=+4121.183884783" watchObservedRunningTime="2025-12-03 12:28:50.597182993 +0000 UTC m=+4121.185424757" Dec 03 12:28:50 crc kubenswrapper[5052]: I1203 12:28:50.611827 5052 generic.go:334] "Generic (PLEG): container finished" podID="e7e769bc-cba8-4955-9d3d-14d1acc7fb5b" containerID="ed335dc5276810aababf7754b74b1511a5d7d1fa065ad7710e9fe074646e5384" exitCode=0 Dec 03 12:28:50 crc kubenswrapper[5052]: I1203 12:28:50.611859 5052 generic.go:334] "Generic (PLEG): container finished" podID="e7e769bc-cba8-4955-9d3d-14d1acc7fb5b" containerID="94e6dc8d77f6943b437cb5870adcaa6c29d491e693bc37360bb9fd740b7610f1" exitCode=2 Dec 03 12:28:50 crc kubenswrapper[5052]: I1203 12:28:50.611865 5052 generic.go:334] "Generic (PLEG): container finished" podID="e7e769bc-cba8-4955-9d3d-14d1acc7fb5b" containerID="ce04bf8dc26d500c7eeaadff490cd8b971ffdcceacc5c68dc3033bfb0a4f5e56" exitCode=0 Dec 03 12:28:50 crc kubenswrapper[5052]: I1203 12:28:50.612040 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-metadata-0" podUID="b72919f4-0bc0-444b-9ddb-859d386a53b5" containerName="nova-metadata-log" containerID="cri-o://196d809c625b6d7be7a93108322dbb398749088ab5233d8d6400d8ad5534eed2" gracePeriod=30 Dec 03 12:28:50 crc kubenswrapper[5052]: I1203 12:28:50.612304 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"e7e769bc-cba8-4955-9d3d-14d1acc7fb5b","Type":"ContainerDied","Data":"ed335dc5276810aababf7754b74b1511a5d7d1fa065ad7710e9fe074646e5384"} Dec 03 12:28:50 crc kubenswrapper[5052]: I1203 12:28:50.612328 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"e7e769bc-cba8-4955-9d3d-14d1acc7fb5b","Type":"ContainerDied","Data":"94e6dc8d77f6943b437cb5870adcaa6c29d491e693bc37360bb9fd740b7610f1"} Dec 03 12:28:50 crc kubenswrapper[5052]: I1203 12:28:50.612337 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"e7e769bc-cba8-4955-9d3d-14d1acc7fb5b","Type":"ContainerDied","Data":"ce04bf8dc26d500c7eeaadff490cd8b971ffdcceacc5c68dc3033bfb0a4f5e56"} Dec 03 12:28:50 crc kubenswrapper[5052]: I1203 12:28:50.612377 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-metadata-0" podUID="b72919f4-0bc0-444b-9ddb-859d386a53b5" containerName="nova-metadata-metadata" containerID="cri-o://ba961703896a39a5fa95d99d5f68e5bc3cd82febfad30c6d56817cbc86b81596" gracePeriod=30 Dec 03 12:28:50 crc kubenswrapper[5052]: I1203 12:28:50.653299 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" podStartSLOduration=3.653266234 podStartE2EDuration="3.653266234s" podCreationTimestamp="2025-12-03 12:28:47 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:28:50.636437433 +0000 UTC m=+4121.224679207" watchObservedRunningTime="2025-12-03 12:28:50.653266234 +0000 UTC m=+4121.241507998" Dec 03 12:28:50 crc kubenswrapper[5052]: I1203 12:28:50.910379 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-0"] Dec 03 12:28:50 crc kubenswrapper[5052]: W1203 12:28:50.929646 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod211396f3_156a_4ac5_917b_6d41c236d3c2.slice/crio-63e1d551272047e4f06252a3ba2f200f584eb0e6682ccbf55490dee16f84d1a9 WatchSource:0}: Error finding container 63e1d551272047e4f06252a3ba2f200f584eb0e6682ccbf55490dee16f84d1a9: Status 404 returned error can't find the container with id 63e1d551272047e4f06252a3ba2f200f584eb0e6682ccbf55490dee16f84d1a9 Dec 03 12:28:50 crc kubenswrapper[5052]: I1203 12:28:50.962359 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-0"] Dec 03 12:28:51 crc kubenswrapper[5052]: I1203 12:28:51.370130 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:28:51 crc kubenswrapper[5052]: I1203 12:28:51.430901 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:28:51 crc kubenswrapper[5052]: I1203 12:28:51.542011 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l9cmf\" (UniqueName: \"kubernetes.io/projected/a740ecee-4c7e-4e42-bfb8-8119ffbfcca0-kube-api-access-l9cmf\") pod \"a740ecee-4c7e-4e42-bfb8-8119ffbfcca0\" (UID: \"a740ecee-4c7e-4e42-bfb8-8119ffbfcca0\") " Dec 03 12:28:51 crc kubenswrapper[5052]: I1203 12:28:51.542090 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/a740ecee-4c7e-4e42-bfb8-8119ffbfcca0-public-tls-certs\") pod \"a740ecee-4c7e-4e42-bfb8-8119ffbfcca0\" (UID: \"a740ecee-4c7e-4e42-bfb8-8119ffbfcca0\") " Dec 03 12:28:51 crc kubenswrapper[5052]: I1203 12:28:51.542128 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/a740ecee-4c7e-4e42-bfb8-8119ffbfcca0-logs\") pod \"a740ecee-4c7e-4e42-bfb8-8119ffbfcca0\" (UID: \"a740ecee-4c7e-4e42-bfb8-8119ffbfcca0\") " Dec 03 12:28:51 crc kubenswrapper[5052]: I1203 12:28:51.542168 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/b72919f4-0bc0-444b-9ddb-859d386a53b5-config-data\") pod \"b72919f4-0bc0-444b-9ddb-859d386a53b5\" (UID: \"b72919f4-0bc0-444b-9ddb-859d386a53b5\") " Dec 03 12:28:51 crc kubenswrapper[5052]: I1203 12:28:51.542200 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/b72919f4-0bc0-444b-9ddb-859d386a53b5-logs\") pod \"b72919f4-0bc0-444b-9ddb-859d386a53b5\" (UID: \"b72919f4-0bc0-444b-9ddb-859d386a53b5\") " Dec 03 12:28:51 crc kubenswrapper[5052]: I1203 12:28:51.542227 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/a740ecee-4c7e-4e42-bfb8-8119ffbfcca0-internal-tls-certs\") pod \"a740ecee-4c7e-4e42-bfb8-8119ffbfcca0\" (UID: \"a740ecee-4c7e-4e42-bfb8-8119ffbfcca0\") " Dec 03 12:28:51 crc kubenswrapper[5052]: I1203 12:28:51.542280 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b72919f4-0bc0-444b-9ddb-859d386a53b5-combined-ca-bundle\") pod \"b72919f4-0bc0-444b-9ddb-859d386a53b5\" (UID: \"b72919f4-0bc0-444b-9ddb-859d386a53b5\") " Dec 03 12:28:51 crc kubenswrapper[5052]: I1203 12:28:51.542323 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tc8qm\" (UniqueName: \"kubernetes.io/projected/b72919f4-0bc0-444b-9ddb-859d386a53b5-kube-api-access-tc8qm\") pod \"b72919f4-0bc0-444b-9ddb-859d386a53b5\" (UID: \"b72919f4-0bc0-444b-9ddb-859d386a53b5\") " Dec 03 12:28:51 crc kubenswrapper[5052]: I1203 12:28:51.542355 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a740ecee-4c7e-4e42-bfb8-8119ffbfcca0-combined-ca-bundle\") pod \"a740ecee-4c7e-4e42-bfb8-8119ffbfcca0\" (UID: \"a740ecee-4c7e-4e42-bfb8-8119ffbfcca0\") " Dec 03 12:28:51 crc kubenswrapper[5052]: I1203 12:28:51.542402 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/b72919f4-0bc0-444b-9ddb-859d386a53b5-nova-metadata-tls-certs\") pod \"b72919f4-0bc0-444b-9ddb-859d386a53b5\" (UID: \"b72919f4-0bc0-444b-9ddb-859d386a53b5\") " Dec 03 12:28:51 crc kubenswrapper[5052]: I1203 12:28:51.542480 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a740ecee-4c7e-4e42-bfb8-8119ffbfcca0-config-data\") pod \"a740ecee-4c7e-4e42-bfb8-8119ffbfcca0\" (UID: \"a740ecee-4c7e-4e42-bfb8-8119ffbfcca0\") " Dec 03 12:28:51 crc kubenswrapper[5052]: I1203 12:28:51.543354 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a740ecee-4c7e-4e42-bfb8-8119ffbfcca0-logs" (OuterVolumeSpecName: "logs") pod "a740ecee-4c7e-4e42-bfb8-8119ffbfcca0" (UID: "a740ecee-4c7e-4e42-bfb8-8119ffbfcca0"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:28:51 crc kubenswrapper[5052]: I1203 12:28:51.547043 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b72919f4-0bc0-444b-9ddb-859d386a53b5-logs" (OuterVolumeSpecName: "logs") pod "b72919f4-0bc0-444b-9ddb-859d386a53b5" (UID: "b72919f4-0bc0-444b-9ddb-859d386a53b5"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:28:51 crc kubenswrapper[5052]: I1203 12:28:51.555977 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b72919f4-0bc0-444b-9ddb-859d386a53b5-kube-api-access-tc8qm" (OuterVolumeSpecName: "kube-api-access-tc8qm") pod "b72919f4-0bc0-444b-9ddb-859d386a53b5" (UID: "b72919f4-0bc0-444b-9ddb-859d386a53b5"). InnerVolumeSpecName "kube-api-access-tc8qm". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:28:51 crc kubenswrapper[5052]: I1203 12:28:51.563782 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a740ecee-4c7e-4e42-bfb8-8119ffbfcca0-kube-api-access-l9cmf" (OuterVolumeSpecName: "kube-api-access-l9cmf") pod "a740ecee-4c7e-4e42-bfb8-8119ffbfcca0" (UID: "a740ecee-4c7e-4e42-bfb8-8119ffbfcca0"). InnerVolumeSpecName "kube-api-access-l9cmf". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:28:51 crc kubenswrapper[5052]: I1203 12:28:51.607956 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:28:51 crc kubenswrapper[5052]: I1203 12:28:51.621328 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b72919f4-0bc0-444b-9ddb-859d386a53b5-config-data" (OuterVolumeSpecName: "config-data") pod "b72919f4-0bc0-444b-9ddb-859d386a53b5" (UID: "b72919f4-0bc0-444b-9ddb-859d386a53b5"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:28:51 crc kubenswrapper[5052]: I1203 12:28:51.635165 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b72919f4-0bc0-444b-9ddb-859d386a53b5-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "b72919f4-0bc0-444b-9ddb-859d386a53b5" (UID: "b72919f4-0bc0-444b-9ddb-859d386a53b5"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:28:51 crc kubenswrapper[5052]: I1203 12:28:51.635995 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-conductor-0" event={"ID":"211396f3-156a-4ac5-917b-6d41c236d3c2","Type":"ContainerStarted","Data":"0dcdd2aec7240ab8991d19f9ba29d4cff2af7594281adff351a9d0b94735f781"} Dec 03 12:28:51 crc kubenswrapper[5052]: I1203 12:28:51.636037 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-conductor-0" event={"ID":"211396f3-156a-4ac5-917b-6d41c236d3c2","Type":"ContainerStarted","Data":"63e1d551272047e4f06252a3ba2f200f584eb0e6682ccbf55490dee16f84d1a9"} Dec 03 12:28:51 crc kubenswrapper[5052]: I1203 12:28:51.636077 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-cell0-conductor-0" podUID="211396f3-156a-4ac5-917b-6d41c236d3c2" containerName="nova-cell0-conductor-conductor" containerID="cri-o://0dcdd2aec7240ab8991d19f9ba29d4cff2af7594281adff351a9d0b94735f781" gracePeriod=30 Dec 03 12:28:51 crc kubenswrapper[5052]: I1203 12:28:51.636321 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:28:51 crc kubenswrapper[5052]: I1203 12:28:51.644384 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a740ecee-4c7e-4e42-bfb8-8119ffbfcca0-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "a740ecee-4c7e-4e42-bfb8-8119ffbfcca0" (UID: "a740ecee-4c7e-4e42-bfb8-8119ffbfcca0"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:28:51 crc kubenswrapper[5052]: I1203 12:28:51.644760 5052 generic.go:334] "Generic (PLEG): container finished" podID="e7e769bc-cba8-4955-9d3d-14d1acc7fb5b" containerID="14273699743c5c98bca470132190fea0be82840c80649fe7f4e02e858e70da72" exitCode=0 Dec 03 12:28:51 crc kubenswrapper[5052]: I1203 12:28:51.644969 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"e7e769bc-cba8-4955-9d3d-14d1acc7fb5b","Type":"ContainerDied","Data":"14273699743c5c98bca470132190fea0be82840c80649fe7f4e02e858e70da72"} Dec 03 12:28:51 crc kubenswrapper[5052]: I1203 12:28:51.645004 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"e7e769bc-cba8-4955-9d3d-14d1acc7fb5b","Type":"ContainerDied","Data":"19faf52ea0db2c5e7ec4c8c64930e9e227c325f2bbe38efaefc6c6c5aa244a80"} Dec 03 12:28:51 crc kubenswrapper[5052]: I1203 12:28:51.645039 5052 scope.go:117] "RemoveContainer" containerID="ed335dc5276810aababf7754b74b1511a5d7d1fa065ad7710e9fe074646e5384" Dec 03 12:28:51 crc kubenswrapper[5052]: I1203 12:28:51.645224 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:28:51 crc kubenswrapper[5052]: I1203 12:28:51.646799 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-tc8qm\" (UniqueName: \"kubernetes.io/projected/b72919f4-0bc0-444b-9ddb-859d386a53b5-kube-api-access-tc8qm\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:51 crc kubenswrapper[5052]: I1203 12:28:51.646819 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a740ecee-4c7e-4e42-bfb8-8119ffbfcca0-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:51 crc kubenswrapper[5052]: I1203 12:28:51.646833 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-l9cmf\" (UniqueName: \"kubernetes.io/projected/a740ecee-4c7e-4e42-bfb8-8119ffbfcca0-kube-api-access-l9cmf\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:51 crc kubenswrapper[5052]: I1203 12:28:51.646844 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/a740ecee-4c7e-4e42-bfb8-8119ffbfcca0-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:51 crc kubenswrapper[5052]: I1203 12:28:51.646854 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/b72919f4-0bc0-444b-9ddb-859d386a53b5-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:51 crc kubenswrapper[5052]: I1203 12:28:51.646863 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/b72919f4-0bc0-444b-9ddb-859d386a53b5-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:51 crc kubenswrapper[5052]: I1203 12:28:51.646872 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b72919f4-0bc0-444b-9ddb-859d386a53b5-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:51 crc kubenswrapper[5052]: I1203 12:28:51.681033 5052 generic.go:334] "Generic (PLEG): container finished" podID="a740ecee-4c7e-4e42-bfb8-8119ffbfcca0" containerID="9266cf294e7ac6a94b81960b8647a5807b0b2d54f0c4b9ad133adc1aa1aef8e8" exitCode=0 Dec 03 12:28:51 crc kubenswrapper[5052]: I1203 12:28:51.681061 5052 generic.go:334] "Generic (PLEG): container finished" podID="a740ecee-4c7e-4e42-bfb8-8119ffbfcca0" containerID="a4444872bf34dcdac7f5d8135bb76c3cc02268af96f1647ecca10b5d17728ea9" exitCode=143 Dec 03 12:28:51 crc kubenswrapper[5052]: I1203 12:28:51.681144 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:28:51 crc kubenswrapper[5052]: I1203 12:28:51.716915 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-cell0-conductor-0" podStartSLOduration=2.716894272 podStartE2EDuration="2.716894272s" podCreationTimestamp="2025-12-03 12:28:49 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:28:51.716882772 +0000 UTC m=+4122.305124536" watchObservedRunningTime="2025-12-03 12:28:51.716894272 +0000 UTC m=+4122.305136036" Dec 03 12:28:51 crc kubenswrapper[5052]: I1203 12:28:51.729676 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a740ecee-4c7e-4e42-bfb8-8119ffbfcca0-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "a740ecee-4c7e-4e42-bfb8-8119ffbfcca0" (UID: "a740ecee-4c7e-4e42-bfb8-8119ffbfcca0"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:28:51 crc kubenswrapper[5052]: I1203 12:28:51.725241 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a740ecee-4c7e-4e42-bfb8-8119ffbfcca0-config-data" (OuterVolumeSpecName: "config-data") pod "a740ecee-4c7e-4e42-bfb8-8119ffbfcca0" (UID: "a740ecee-4c7e-4e42-bfb8-8119ffbfcca0"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:28:51 crc kubenswrapper[5052]: I1203 12:28:51.753607 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/e7e769bc-cba8-4955-9d3d-14d1acc7fb5b-ceilometer-tls-certs\") pod \"e7e769bc-cba8-4955-9d3d-14d1acc7fb5b\" (UID: \"e7e769bc-cba8-4955-9d3d-14d1acc7fb5b\") " Dec 03 12:28:51 crc kubenswrapper[5052]: I1203 12:28:51.753740 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/e7e769bc-cba8-4955-9d3d-14d1acc7fb5b-run-httpd\") pod \"e7e769bc-cba8-4955-9d3d-14d1acc7fb5b\" (UID: \"e7e769bc-cba8-4955-9d3d-14d1acc7fb5b\") " Dec 03 12:28:51 crc kubenswrapper[5052]: I1203 12:28:51.753768 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/e7e769bc-cba8-4955-9d3d-14d1acc7fb5b-sg-core-conf-yaml\") pod \"e7e769bc-cba8-4955-9d3d-14d1acc7fb5b\" (UID: \"e7e769bc-cba8-4955-9d3d-14d1acc7fb5b\") " Dec 03 12:28:51 crc kubenswrapper[5052]: I1203 12:28:51.753797 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z6lcc\" (UniqueName: \"kubernetes.io/projected/e7e769bc-cba8-4955-9d3d-14d1acc7fb5b-kube-api-access-z6lcc\") pod \"e7e769bc-cba8-4955-9d3d-14d1acc7fb5b\" (UID: \"e7e769bc-cba8-4955-9d3d-14d1acc7fb5b\") " Dec 03 12:28:51 crc kubenswrapper[5052]: I1203 12:28:51.754447 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e7e769bc-cba8-4955-9d3d-14d1acc7fb5b-combined-ca-bundle\") pod \"e7e769bc-cba8-4955-9d3d-14d1acc7fb5b\" (UID: \"e7e769bc-cba8-4955-9d3d-14d1acc7fb5b\") " Dec 03 12:28:51 crc kubenswrapper[5052]: I1203 12:28:51.754531 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/e7e769bc-cba8-4955-9d3d-14d1acc7fb5b-config-data\") pod \"e7e769bc-cba8-4955-9d3d-14d1acc7fb5b\" (UID: \"e7e769bc-cba8-4955-9d3d-14d1acc7fb5b\") " Dec 03 12:28:51 crc kubenswrapper[5052]: I1203 12:28:51.754556 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/e7e769bc-cba8-4955-9d3d-14d1acc7fb5b-log-httpd\") pod \"e7e769bc-cba8-4955-9d3d-14d1acc7fb5b\" (UID: \"e7e769bc-cba8-4955-9d3d-14d1acc7fb5b\") " Dec 03 12:28:51 crc kubenswrapper[5052]: I1203 12:28:51.754581 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/e7e769bc-cba8-4955-9d3d-14d1acc7fb5b-scripts\") pod \"e7e769bc-cba8-4955-9d3d-14d1acc7fb5b\" (UID: \"e7e769bc-cba8-4955-9d3d-14d1acc7fb5b\") " Dec 03 12:28:51 crc kubenswrapper[5052]: I1203 12:28:51.755167 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/a740ecee-4c7e-4e42-bfb8-8119ffbfcca0-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:51 crc kubenswrapper[5052]: I1203 12:28:51.755184 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a740ecee-4c7e-4e42-bfb8-8119ffbfcca0-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:51 crc kubenswrapper[5052]: I1203 12:28:51.760130 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6efd22cd-98c7-48d8-a346-289b78e8aa1c" path="/var/lib/kubelet/pods/6efd22cd-98c7-48d8-a346-289b78e8aa1c/volumes" Dec 03 12:28:51 crc kubenswrapper[5052]: I1203 12:28:51.771697 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e7e769bc-cba8-4955-9d3d-14d1acc7fb5b-run-httpd" (OuterVolumeSpecName: "run-httpd") pod "e7e769bc-cba8-4955-9d3d-14d1acc7fb5b" (UID: "e7e769bc-cba8-4955-9d3d-14d1acc7fb5b"). InnerVolumeSpecName "run-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:28:51 crc kubenswrapper[5052]: I1203 12:28:51.772677 5052 generic.go:334] "Generic (PLEG): container finished" podID="b72919f4-0bc0-444b-9ddb-859d386a53b5" containerID="ba961703896a39a5fa95d99d5f68e5bc3cd82febfad30c6d56817cbc86b81596" exitCode=0 Dec 03 12:28:51 crc kubenswrapper[5052]: I1203 12:28:51.772716 5052 generic.go:334] "Generic (PLEG): container finished" podID="b72919f4-0bc0-444b-9ddb-859d386a53b5" containerID="196d809c625b6d7be7a93108322dbb398749088ab5233d8d6400d8ad5534eed2" exitCode=143 Dec 03 12:28:51 crc kubenswrapper[5052]: I1203 12:28:51.772772 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:28:51 crc kubenswrapper[5052]: I1203 12:28:51.787802 5052 scope.go:117] "RemoveContainer" containerID="94e6dc8d77f6943b437cb5870adcaa6c29d491e693bc37360bb9fd740b7610f1" Dec 03 12:28:51 crc kubenswrapper[5052]: I1203 12:28:51.788878 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e7e769bc-cba8-4955-9d3d-14d1acc7fb5b-log-httpd" (OuterVolumeSpecName: "log-httpd") pod "e7e769bc-cba8-4955-9d3d-14d1acc7fb5b" (UID: "e7e769bc-cba8-4955-9d3d-14d1acc7fb5b"). InnerVolumeSpecName "log-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:28:51 crc kubenswrapper[5052]: I1203 12:28:51.798391 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e7e769bc-cba8-4955-9d3d-14d1acc7fb5b-kube-api-access-z6lcc" (OuterVolumeSpecName: "kube-api-access-z6lcc") pod "e7e769bc-cba8-4955-9d3d-14d1acc7fb5b" (UID: "e7e769bc-cba8-4955-9d3d-14d1acc7fb5b"). InnerVolumeSpecName "kube-api-access-z6lcc". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:28:51 crc kubenswrapper[5052]: I1203 12:28:51.823848 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e7e769bc-cba8-4955-9d3d-14d1acc7fb5b-scripts" (OuterVolumeSpecName: "scripts") pod "e7e769bc-cba8-4955-9d3d-14d1acc7fb5b" (UID: "e7e769bc-cba8-4955-9d3d-14d1acc7fb5b"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:28:51 crc kubenswrapper[5052]: I1203 12:28:51.840729 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a740ecee-4c7e-4e42-bfb8-8119ffbfcca0-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "a740ecee-4c7e-4e42-bfb8-8119ffbfcca0" (UID: "a740ecee-4c7e-4e42-bfb8-8119ffbfcca0"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:28:51 crc kubenswrapper[5052]: I1203 12:28:51.881681 5052 reconciler_common.go:293] "Volume detached for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/e7e769bc-cba8-4955-9d3d-14d1acc7fb5b-run-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:51 crc kubenswrapper[5052]: I1203 12:28:51.881719 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-z6lcc\" (UniqueName: \"kubernetes.io/projected/e7e769bc-cba8-4955-9d3d-14d1acc7fb5b-kube-api-access-z6lcc\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:51 crc kubenswrapper[5052]: I1203 12:28:51.881731 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/a740ecee-4c7e-4e42-bfb8-8119ffbfcca0-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:51 crc kubenswrapper[5052]: I1203 12:28:51.881741 5052 reconciler_common.go:293] "Volume detached for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/e7e769bc-cba8-4955-9d3d-14d1acc7fb5b-log-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:51 crc kubenswrapper[5052]: I1203 12:28:51.881750 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/e7e769bc-cba8-4955-9d3d-14d1acc7fb5b-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:51 crc kubenswrapper[5052]: I1203 12:28:51.923711 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b72919f4-0bc0-444b-9ddb-859d386a53b5-nova-metadata-tls-certs" (OuterVolumeSpecName: "nova-metadata-tls-certs") pod "b72919f4-0bc0-444b-9ddb-859d386a53b5" (UID: "b72919f4-0bc0-444b-9ddb-859d386a53b5"). InnerVolumeSpecName "nova-metadata-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:28:51 crc kubenswrapper[5052]: I1203 12:28:51.959669 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e7e769bc-cba8-4955-9d3d-14d1acc7fb5b-sg-core-conf-yaml" (OuterVolumeSpecName: "sg-core-conf-yaml") pod "e7e769bc-cba8-4955-9d3d-14d1acc7fb5b" (UID: "e7e769bc-cba8-4955-9d3d-14d1acc7fb5b"). InnerVolumeSpecName "sg-core-conf-yaml". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:28:51 crc kubenswrapper[5052]: I1203 12:28:51.982663 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e7e769bc-cba8-4955-9d3d-14d1acc7fb5b-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "e7e769bc-cba8-4955-9d3d-14d1acc7fb5b" (UID: "e7e769bc-cba8-4955-9d3d-14d1acc7fb5b"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:28:51 crc kubenswrapper[5052]: I1203 12:28:51.986013 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e7e769bc-cba8-4955-9d3d-14d1acc7fb5b-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:51 crc kubenswrapper[5052]: I1203 12:28:51.986185 5052 reconciler_common.go:293] "Volume detached for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/b72919f4-0bc0-444b-9ddb-859d386a53b5-nova-metadata-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:51 crc kubenswrapper[5052]: I1203 12:28:51.986290 5052 reconciler_common.go:293] "Volume detached for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/e7e769bc-cba8-4955-9d3d-14d1acc7fb5b-sg-core-conf-yaml\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:52 crc kubenswrapper[5052]: I1203 12:28:52.007646 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e7e769bc-cba8-4955-9d3d-14d1acc7fb5b-ceilometer-tls-certs" (OuterVolumeSpecName: "ceilometer-tls-certs") pod "e7e769bc-cba8-4955-9d3d-14d1acc7fb5b" (UID: "e7e769bc-cba8-4955-9d3d-14d1acc7fb5b"). InnerVolumeSpecName "ceilometer-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:28:52 crc kubenswrapper[5052]: I1203 12:28:52.049217 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e7e769bc-cba8-4955-9d3d-14d1acc7fb5b-config-data" (OuterVolumeSpecName: "config-data") pod "e7e769bc-cba8-4955-9d3d-14d1acc7fb5b" (UID: "e7e769bc-cba8-4955-9d3d-14d1acc7fb5b"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:28:52 crc kubenswrapper[5052]: I1203 12:28:52.092142 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/e7e769bc-cba8-4955-9d3d-14d1acc7fb5b-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:52 crc kubenswrapper[5052]: I1203 12:28:52.092171 5052 reconciler_common.go:293] "Volume detached for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/e7e769bc-cba8-4955-9d3d-14d1acc7fb5b-ceilometer-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:52 crc kubenswrapper[5052]: I1203 12:28:52.130061 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"a740ecee-4c7e-4e42-bfb8-8119ffbfcca0","Type":"ContainerDied","Data":"9266cf294e7ac6a94b81960b8647a5807b0b2d54f0c4b9ad133adc1aa1aef8e8"} Dec 03 12:28:52 crc kubenswrapper[5052]: I1203 12:28:52.130116 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"a740ecee-4c7e-4e42-bfb8-8119ffbfcca0","Type":"ContainerDied","Data":"a4444872bf34dcdac7f5d8135bb76c3cc02268af96f1647ecca10b5d17728ea9"} Dec 03 12:28:52 crc kubenswrapper[5052]: I1203 12:28:52.130226 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"a740ecee-4c7e-4e42-bfb8-8119ffbfcca0","Type":"ContainerDied","Data":"ef4460d4df17c0f7628e13a6d5f896627b209d3d05c91e600735eaaa42f2d703"} Dec 03 12:28:52 crc kubenswrapper[5052]: I1203 12:28:52.130238 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"b72919f4-0bc0-444b-9ddb-859d386a53b5","Type":"ContainerDied","Data":"ba961703896a39a5fa95d99d5f68e5bc3cd82febfad30c6d56817cbc86b81596"} Dec 03 12:28:52 crc kubenswrapper[5052]: I1203 12:28:52.130251 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"b72919f4-0bc0-444b-9ddb-859d386a53b5","Type":"ContainerDied","Data":"196d809c625b6d7be7a93108322dbb398749088ab5233d8d6400d8ad5534eed2"} Dec 03 12:28:52 crc kubenswrapper[5052]: I1203 12:28:52.130259 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"b72919f4-0bc0-444b-9ddb-859d386a53b5","Type":"ContainerDied","Data":"80bea364197d6323eeb3e89869ba88bf1c35cf5d017cae5e8b46ea7daead6f7e"} Dec 03 12:28:52 crc kubenswrapper[5052]: I1203 12:28:52.148368 5052 scope.go:117] "RemoveContainer" containerID="14273699743c5c98bca470132190fea0be82840c80649fe7f4e02e858e70da72" Dec 03 12:28:52 crc kubenswrapper[5052]: I1203 12:28:52.177163 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:28:52 crc kubenswrapper[5052]: I1203 12:28:52.190706 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:28:52 crc kubenswrapper[5052]: I1203 12:28:52.196688 5052 scope.go:117] "RemoveContainer" containerID="ce04bf8dc26d500c7eeaadff490cd8b971ffdcceacc5c68dc3033bfb0a4f5e56" Dec 03 12:28:52 crc kubenswrapper[5052]: I1203 12:28:52.203551 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:28:52 crc kubenswrapper[5052]: I1203 12:28:52.238664 5052 scope.go:117] "RemoveContainer" containerID="ed335dc5276810aababf7754b74b1511a5d7d1fa065ad7710e9fe074646e5384" Dec 03 12:28:52 crc kubenswrapper[5052]: E1203 12:28:52.239159 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ed335dc5276810aababf7754b74b1511a5d7d1fa065ad7710e9fe074646e5384\": container with ID starting with ed335dc5276810aababf7754b74b1511a5d7d1fa065ad7710e9fe074646e5384 not found: ID does not exist" containerID="ed335dc5276810aababf7754b74b1511a5d7d1fa065ad7710e9fe074646e5384" Dec 03 12:28:52 crc kubenswrapper[5052]: I1203 12:28:52.239239 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ed335dc5276810aababf7754b74b1511a5d7d1fa065ad7710e9fe074646e5384"} err="failed to get container status \"ed335dc5276810aababf7754b74b1511a5d7d1fa065ad7710e9fe074646e5384\": rpc error: code = NotFound desc = could not find container \"ed335dc5276810aababf7754b74b1511a5d7d1fa065ad7710e9fe074646e5384\": container with ID starting with ed335dc5276810aababf7754b74b1511a5d7d1fa065ad7710e9fe074646e5384 not found: ID does not exist" Dec 03 12:28:52 crc kubenswrapper[5052]: I1203 12:28:52.239275 5052 scope.go:117] "RemoveContainer" containerID="94e6dc8d77f6943b437cb5870adcaa6c29d491e693bc37360bb9fd740b7610f1" Dec 03 12:28:52 crc kubenswrapper[5052]: E1203 12:28:52.240622 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"94e6dc8d77f6943b437cb5870adcaa6c29d491e693bc37360bb9fd740b7610f1\": container with ID starting with 94e6dc8d77f6943b437cb5870adcaa6c29d491e693bc37360bb9fd740b7610f1 not found: ID does not exist" containerID="94e6dc8d77f6943b437cb5870adcaa6c29d491e693bc37360bb9fd740b7610f1" Dec 03 12:28:52 crc kubenswrapper[5052]: I1203 12:28:52.240662 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"94e6dc8d77f6943b437cb5870adcaa6c29d491e693bc37360bb9fd740b7610f1"} err="failed to get container status \"94e6dc8d77f6943b437cb5870adcaa6c29d491e693bc37360bb9fd740b7610f1\": rpc error: code = NotFound desc = could not find container \"94e6dc8d77f6943b437cb5870adcaa6c29d491e693bc37360bb9fd740b7610f1\": container with ID starting with 94e6dc8d77f6943b437cb5870adcaa6c29d491e693bc37360bb9fd740b7610f1 not found: ID does not exist" Dec 03 12:28:52 crc kubenswrapper[5052]: I1203 12:28:52.240687 5052 scope.go:117] "RemoveContainer" containerID="14273699743c5c98bca470132190fea0be82840c80649fe7f4e02e858e70da72" Dec 03 12:28:52 crc kubenswrapper[5052]: E1203 12:28:52.241030 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"14273699743c5c98bca470132190fea0be82840c80649fe7f4e02e858e70da72\": container with ID starting with 14273699743c5c98bca470132190fea0be82840c80649fe7f4e02e858e70da72 not found: ID does not exist" containerID="14273699743c5c98bca470132190fea0be82840c80649fe7f4e02e858e70da72" Dec 03 12:28:52 crc kubenswrapper[5052]: I1203 12:28:52.241073 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"14273699743c5c98bca470132190fea0be82840c80649fe7f4e02e858e70da72"} err="failed to get container status \"14273699743c5c98bca470132190fea0be82840c80649fe7f4e02e858e70da72\": rpc error: code = NotFound desc = could not find container \"14273699743c5c98bca470132190fea0be82840c80649fe7f4e02e858e70da72\": container with ID starting with 14273699743c5c98bca470132190fea0be82840c80649fe7f4e02e858e70da72 not found: ID does not exist" Dec 03 12:28:52 crc kubenswrapper[5052]: I1203 12:28:52.241096 5052 scope.go:117] "RemoveContainer" containerID="ce04bf8dc26d500c7eeaadff490cd8b971ffdcceacc5c68dc3033bfb0a4f5e56" Dec 03 12:28:52 crc kubenswrapper[5052]: E1203 12:28:52.241476 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ce04bf8dc26d500c7eeaadff490cd8b971ffdcceacc5c68dc3033bfb0a4f5e56\": container with ID starting with ce04bf8dc26d500c7eeaadff490cd8b971ffdcceacc5c68dc3033bfb0a4f5e56 not found: ID does not exist" containerID="ce04bf8dc26d500c7eeaadff490cd8b971ffdcceacc5c68dc3033bfb0a4f5e56" Dec 03 12:28:52 crc kubenswrapper[5052]: I1203 12:28:52.241579 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ce04bf8dc26d500c7eeaadff490cd8b971ffdcceacc5c68dc3033bfb0a4f5e56"} err="failed to get container status \"ce04bf8dc26d500c7eeaadff490cd8b971ffdcceacc5c68dc3033bfb0a4f5e56\": rpc error: code = NotFound desc = could not find container \"ce04bf8dc26d500c7eeaadff490cd8b971ffdcceacc5c68dc3033bfb0a4f5e56\": container with ID starting with ce04bf8dc26d500c7eeaadff490cd8b971ffdcceacc5c68dc3033bfb0a4f5e56 not found: ID does not exist" Dec 03 12:28:52 crc kubenswrapper[5052]: I1203 12:28:52.241601 5052 scope.go:117] "RemoveContainer" containerID="9266cf294e7ac6a94b81960b8647a5807b0b2d54f0c4b9ad133adc1aa1aef8e8" Dec 03 12:28:52 crc kubenswrapper[5052]: I1203 12:28:52.281834 5052 scope.go:117] "RemoveContainer" containerID="a4444872bf34dcdac7f5d8135bb76c3cc02268af96f1647ecca10b5d17728ea9" Dec 03 12:28:52 crc kubenswrapper[5052]: I1203 12:28:52.307705 5052 scope.go:117] "RemoveContainer" containerID="9266cf294e7ac6a94b81960b8647a5807b0b2d54f0c4b9ad133adc1aa1aef8e8" Dec 03 12:28:52 crc kubenswrapper[5052]: E1203 12:28:52.320058 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9266cf294e7ac6a94b81960b8647a5807b0b2d54f0c4b9ad133adc1aa1aef8e8\": container with ID starting with 9266cf294e7ac6a94b81960b8647a5807b0b2d54f0c4b9ad133adc1aa1aef8e8 not found: ID does not exist" containerID="9266cf294e7ac6a94b81960b8647a5807b0b2d54f0c4b9ad133adc1aa1aef8e8" Dec 03 12:28:52 crc kubenswrapper[5052]: I1203 12:28:52.320140 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9266cf294e7ac6a94b81960b8647a5807b0b2d54f0c4b9ad133adc1aa1aef8e8"} err="failed to get container status \"9266cf294e7ac6a94b81960b8647a5807b0b2d54f0c4b9ad133adc1aa1aef8e8\": rpc error: code = NotFound desc = could not find container \"9266cf294e7ac6a94b81960b8647a5807b0b2d54f0c4b9ad133adc1aa1aef8e8\": container with ID starting with 9266cf294e7ac6a94b81960b8647a5807b0b2d54f0c4b9ad133adc1aa1aef8e8 not found: ID does not exist" Dec 03 12:28:52 crc kubenswrapper[5052]: I1203 12:28:52.320168 5052 scope.go:117] "RemoveContainer" containerID="a4444872bf34dcdac7f5d8135bb76c3cc02268af96f1647ecca10b5d17728ea9" Dec 03 12:28:52 crc kubenswrapper[5052]: E1203 12:28:52.321184 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a4444872bf34dcdac7f5d8135bb76c3cc02268af96f1647ecca10b5d17728ea9\": container with ID starting with a4444872bf34dcdac7f5d8135bb76c3cc02268af96f1647ecca10b5d17728ea9 not found: ID does not exist" containerID="a4444872bf34dcdac7f5d8135bb76c3cc02268af96f1647ecca10b5d17728ea9" Dec 03 12:28:52 crc kubenswrapper[5052]: I1203 12:28:52.321267 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a4444872bf34dcdac7f5d8135bb76c3cc02268af96f1647ecca10b5d17728ea9"} err="failed to get container status \"a4444872bf34dcdac7f5d8135bb76c3cc02268af96f1647ecca10b5d17728ea9\": rpc error: code = NotFound desc = could not find container \"a4444872bf34dcdac7f5d8135bb76c3cc02268af96f1647ecca10b5d17728ea9\": container with ID starting with a4444872bf34dcdac7f5d8135bb76c3cc02268af96f1647ecca10b5d17728ea9 not found: ID does not exist" Dec 03 12:28:52 crc kubenswrapper[5052]: I1203 12:28:52.321285 5052 scope.go:117] "RemoveContainer" containerID="9266cf294e7ac6a94b81960b8647a5807b0b2d54f0c4b9ad133adc1aa1aef8e8" Dec 03 12:28:52 crc kubenswrapper[5052]: I1203 12:28:52.323651 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9266cf294e7ac6a94b81960b8647a5807b0b2d54f0c4b9ad133adc1aa1aef8e8"} err="failed to get container status \"9266cf294e7ac6a94b81960b8647a5807b0b2d54f0c4b9ad133adc1aa1aef8e8\": rpc error: code = NotFound desc = could not find container \"9266cf294e7ac6a94b81960b8647a5807b0b2d54f0c4b9ad133adc1aa1aef8e8\": container with ID starting with 9266cf294e7ac6a94b81960b8647a5807b0b2d54f0c4b9ad133adc1aa1aef8e8 not found: ID does not exist" Dec 03 12:28:52 crc kubenswrapper[5052]: I1203 12:28:52.323733 5052 scope.go:117] "RemoveContainer" containerID="a4444872bf34dcdac7f5d8135bb76c3cc02268af96f1647ecca10b5d17728ea9" Dec 03 12:28:52 crc kubenswrapper[5052]: I1203 12:28:52.324506 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a4444872bf34dcdac7f5d8135bb76c3cc02268af96f1647ecca10b5d17728ea9"} err="failed to get container status \"a4444872bf34dcdac7f5d8135bb76c3cc02268af96f1647ecca10b5d17728ea9\": rpc error: code = NotFound desc = could not find container \"a4444872bf34dcdac7f5d8135bb76c3cc02268af96f1647ecca10b5d17728ea9\": container with ID starting with a4444872bf34dcdac7f5d8135bb76c3cc02268af96f1647ecca10b5d17728ea9 not found: ID does not exist" Dec 03 12:28:52 crc kubenswrapper[5052]: I1203 12:28:52.324569 5052 scope.go:117] "RemoveContainer" containerID="ba961703896a39a5fa95d99d5f68e5bc3cd82febfad30c6d56817cbc86b81596" Dec 03 12:28:52 crc kubenswrapper[5052]: I1203 12:28:52.330564 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:28:52 crc kubenswrapper[5052]: E1203 12:28:52.338499 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="b72919f4-0bc0-444b-9ddb-859d386a53b5" containerName="nova-metadata-log" Dec 03 12:28:52 crc kubenswrapper[5052]: I1203 12:28:52.339766 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="b72919f4-0bc0-444b-9ddb-859d386a53b5" containerName="nova-metadata-log" Dec 03 12:28:52 crc kubenswrapper[5052]: E1203 12:28:52.339803 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="e7e769bc-cba8-4955-9d3d-14d1acc7fb5b" containerName="ceilometer-central-agent" Dec 03 12:28:52 crc kubenswrapper[5052]: I1203 12:28:52.339810 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="e7e769bc-cba8-4955-9d3d-14d1acc7fb5b" containerName="ceilometer-central-agent" Dec 03 12:28:52 crc kubenswrapper[5052]: E1203 12:28:52.339836 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="e7e769bc-cba8-4955-9d3d-14d1acc7fb5b" containerName="ceilometer-notification-agent" Dec 03 12:28:52 crc kubenswrapper[5052]: I1203 12:28:52.339843 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="e7e769bc-cba8-4955-9d3d-14d1acc7fb5b" containerName="ceilometer-notification-agent" Dec 03 12:28:52 crc kubenswrapper[5052]: E1203 12:28:52.339859 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="a740ecee-4c7e-4e42-bfb8-8119ffbfcca0" containerName="nova-api-api" Dec 03 12:28:52 crc kubenswrapper[5052]: I1203 12:28:52.339866 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="a740ecee-4c7e-4e42-bfb8-8119ffbfcca0" containerName="nova-api-api" Dec 03 12:28:52 crc kubenswrapper[5052]: E1203 12:28:52.339889 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="e7e769bc-cba8-4955-9d3d-14d1acc7fb5b" containerName="proxy-httpd" Dec 03 12:28:52 crc kubenswrapper[5052]: I1203 12:28:52.339895 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="e7e769bc-cba8-4955-9d3d-14d1acc7fb5b" containerName="proxy-httpd" Dec 03 12:28:52 crc kubenswrapper[5052]: E1203 12:28:52.341003 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="a740ecee-4c7e-4e42-bfb8-8119ffbfcca0" containerName="nova-api-log" Dec 03 12:28:52 crc kubenswrapper[5052]: I1203 12:28:52.341377 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="a740ecee-4c7e-4e42-bfb8-8119ffbfcca0" containerName="nova-api-log" Dec 03 12:28:52 crc kubenswrapper[5052]: E1203 12:28:52.341413 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="b72919f4-0bc0-444b-9ddb-859d386a53b5" containerName="nova-metadata-metadata" Dec 03 12:28:52 crc kubenswrapper[5052]: I1203 12:28:52.341425 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="b72919f4-0bc0-444b-9ddb-859d386a53b5" containerName="nova-metadata-metadata" Dec 03 12:28:52 crc kubenswrapper[5052]: E1203 12:28:52.341561 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="e7e769bc-cba8-4955-9d3d-14d1acc7fb5b" containerName="sg-core" Dec 03 12:28:52 crc kubenswrapper[5052]: I1203 12:28:52.341575 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="e7e769bc-cba8-4955-9d3d-14d1acc7fb5b" containerName="sg-core" Dec 03 12:28:52 crc kubenswrapper[5052]: I1203 12:28:52.353093 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="a740ecee-4c7e-4e42-bfb8-8119ffbfcca0" containerName="nova-api-log" Dec 03 12:28:52 crc kubenswrapper[5052]: I1203 12:28:52.353654 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="e7e769bc-cba8-4955-9d3d-14d1acc7fb5b" containerName="ceilometer-central-agent" Dec 03 12:28:52 crc kubenswrapper[5052]: I1203 12:28:52.354001 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="b72919f4-0bc0-444b-9ddb-859d386a53b5" containerName="nova-metadata-metadata" Dec 03 12:28:52 crc kubenswrapper[5052]: I1203 12:28:52.354326 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="b72919f4-0bc0-444b-9ddb-859d386a53b5" containerName="nova-metadata-log" Dec 03 12:28:52 crc kubenswrapper[5052]: I1203 12:28:52.354436 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="a740ecee-4c7e-4e42-bfb8-8119ffbfcca0" containerName="nova-api-api" Dec 03 12:28:52 crc kubenswrapper[5052]: I1203 12:28:52.354490 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="e7e769bc-cba8-4955-9d3d-14d1acc7fb5b" containerName="sg-core" Dec 03 12:28:52 crc kubenswrapper[5052]: I1203 12:28:52.355154 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="e7e769bc-cba8-4955-9d3d-14d1acc7fb5b" containerName="proxy-httpd" Dec 03 12:28:52 crc kubenswrapper[5052]: I1203 12:28:52.355190 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="e7e769bc-cba8-4955-9d3d-14d1acc7fb5b" containerName="ceilometer-notification-agent" Dec 03 12:28:52 crc kubenswrapper[5052]: I1203 12:28:52.370144 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:28:52 crc kubenswrapper[5052]: I1203 12:28:52.373936 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-nova-internal-svc" Dec 03 12:28:52 crc kubenswrapper[5052]: I1203 12:28:52.374098 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-api-config-data" Dec 03 12:28:52 crc kubenswrapper[5052]: I1203 12:28:52.383816 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:28:52 crc kubenswrapper[5052]: I1203 12:28:52.391603 5052 scope.go:117] "RemoveContainer" containerID="196d809c625b6d7be7a93108322dbb398749088ab5233d8d6400d8ad5534eed2" Dec 03 12:28:52 crc kubenswrapper[5052]: I1203 12:28:52.393491 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-nova-public-svc" Dec 03 12:28:52 crc kubenswrapper[5052]: I1203 12:28:52.400092 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:28:52 crc kubenswrapper[5052]: I1203 12:28:52.403199 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/9632f411-9c90-4a2c-9c79-7a2138f0ff43-logs\") pod \"nova-api-0\" (UID: \"9632f411-9c90-4a2c-9c79-7a2138f0ff43\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:28:52 crc kubenswrapper[5052]: I1203 12:28:52.403380 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/9632f411-9c90-4a2c-9c79-7a2138f0ff43-public-tls-certs\") pod \"nova-api-0\" (UID: \"9632f411-9c90-4a2c-9c79-7a2138f0ff43\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:28:52 crc kubenswrapper[5052]: I1203 12:28:52.403535 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fqjs7\" (UniqueName: \"kubernetes.io/projected/9632f411-9c90-4a2c-9c79-7a2138f0ff43-kube-api-access-fqjs7\") pod \"nova-api-0\" (UID: \"9632f411-9c90-4a2c-9c79-7a2138f0ff43\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:28:52 crc kubenswrapper[5052]: I1203 12:28:52.403787 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9632f411-9c90-4a2c-9c79-7a2138f0ff43-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"9632f411-9c90-4a2c-9c79-7a2138f0ff43\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:28:52 crc kubenswrapper[5052]: I1203 12:28:52.403963 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/9632f411-9c90-4a2c-9c79-7a2138f0ff43-internal-tls-certs\") pod \"nova-api-0\" (UID: \"9632f411-9c90-4a2c-9c79-7a2138f0ff43\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:28:52 crc kubenswrapper[5052]: I1203 12:28:52.404335 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/9632f411-9c90-4a2c-9c79-7a2138f0ff43-config-data\") pod \"nova-api-0\" (UID: \"9632f411-9c90-4a2c-9c79-7a2138f0ff43\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:28:52 crc kubenswrapper[5052]: I1203 12:28:52.407771 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:28:52 crc kubenswrapper[5052]: I1203 12:28:52.409568 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:28:52 crc kubenswrapper[5052]: I1203 12:28:52.414867 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-metadata-config-data" Dec 03 12:28:52 crc kubenswrapper[5052]: I1203 12:28:52.419037 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-nova-metadata-internal-svc" Dec 03 12:28:52 crc kubenswrapper[5052]: I1203 12:28:52.423775 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:28:52 crc kubenswrapper[5052]: I1203 12:28:52.438009 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:28:52 crc kubenswrapper[5052]: I1203 12:28:52.446634 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:28:52 crc kubenswrapper[5052]: I1203 12:28:52.461701 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:28:52 crc kubenswrapper[5052]: I1203 12:28:52.464739 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:28:52 crc kubenswrapper[5052]: I1203 12:28:52.470752 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-config-data" Dec 03 12:28:52 crc kubenswrapper[5052]: I1203 12:28:52.471051 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-ceilometer-internal-svc" Dec 03 12:28:52 crc kubenswrapper[5052]: I1203 12:28:52.471157 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-scripts" Dec 03 12:28:52 crc kubenswrapper[5052]: I1203 12:28:52.472802 5052 scope.go:117] "RemoveContainer" containerID="ba961703896a39a5fa95d99d5f68e5bc3cd82febfad30c6d56817cbc86b81596" Dec 03 12:28:52 crc kubenswrapper[5052]: E1203 12:28:52.474720 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ba961703896a39a5fa95d99d5f68e5bc3cd82febfad30c6d56817cbc86b81596\": container with ID starting with ba961703896a39a5fa95d99d5f68e5bc3cd82febfad30c6d56817cbc86b81596 not found: ID does not exist" containerID="ba961703896a39a5fa95d99d5f68e5bc3cd82febfad30c6d56817cbc86b81596" Dec 03 12:28:52 crc kubenswrapper[5052]: I1203 12:28:52.474764 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ba961703896a39a5fa95d99d5f68e5bc3cd82febfad30c6d56817cbc86b81596"} err="failed to get container status \"ba961703896a39a5fa95d99d5f68e5bc3cd82febfad30c6d56817cbc86b81596\": rpc error: code = NotFound desc = could not find container \"ba961703896a39a5fa95d99d5f68e5bc3cd82febfad30c6d56817cbc86b81596\": container with ID starting with ba961703896a39a5fa95d99d5f68e5bc3cd82febfad30c6d56817cbc86b81596 not found: ID does not exist" Dec 03 12:28:52 crc kubenswrapper[5052]: I1203 12:28:52.474790 5052 scope.go:117] "RemoveContainer" containerID="196d809c625b6d7be7a93108322dbb398749088ab5233d8d6400d8ad5534eed2" Dec 03 12:28:52 crc kubenswrapper[5052]: E1203 12:28:52.475232 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"196d809c625b6d7be7a93108322dbb398749088ab5233d8d6400d8ad5534eed2\": container with ID starting with 196d809c625b6d7be7a93108322dbb398749088ab5233d8d6400d8ad5534eed2 not found: ID does not exist" containerID="196d809c625b6d7be7a93108322dbb398749088ab5233d8d6400d8ad5534eed2" Dec 03 12:28:52 crc kubenswrapper[5052]: I1203 12:28:52.475270 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"196d809c625b6d7be7a93108322dbb398749088ab5233d8d6400d8ad5534eed2"} err="failed to get container status \"196d809c625b6d7be7a93108322dbb398749088ab5233d8d6400d8ad5534eed2\": rpc error: code = NotFound desc = could not find container \"196d809c625b6d7be7a93108322dbb398749088ab5233d8d6400d8ad5534eed2\": container with ID starting with 196d809c625b6d7be7a93108322dbb398749088ab5233d8d6400d8ad5534eed2 not found: ID does not exist" Dec 03 12:28:52 crc kubenswrapper[5052]: I1203 12:28:52.475290 5052 scope.go:117] "RemoveContainer" containerID="ba961703896a39a5fa95d99d5f68e5bc3cd82febfad30c6d56817cbc86b81596" Dec 03 12:28:52 crc kubenswrapper[5052]: I1203 12:28:52.475629 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ba961703896a39a5fa95d99d5f68e5bc3cd82febfad30c6d56817cbc86b81596"} err="failed to get container status \"ba961703896a39a5fa95d99d5f68e5bc3cd82febfad30c6d56817cbc86b81596\": rpc error: code = NotFound desc = could not find container \"ba961703896a39a5fa95d99d5f68e5bc3cd82febfad30c6d56817cbc86b81596\": container with ID starting with ba961703896a39a5fa95d99d5f68e5bc3cd82febfad30c6d56817cbc86b81596 not found: ID does not exist" Dec 03 12:28:52 crc kubenswrapper[5052]: I1203 12:28:52.475656 5052 scope.go:117] "RemoveContainer" containerID="196d809c625b6d7be7a93108322dbb398749088ab5233d8d6400d8ad5534eed2" Dec 03 12:28:52 crc kubenswrapper[5052]: I1203 12:28:52.475979 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"196d809c625b6d7be7a93108322dbb398749088ab5233d8d6400d8ad5534eed2"} err="failed to get container status \"196d809c625b6d7be7a93108322dbb398749088ab5233d8d6400d8ad5534eed2\": rpc error: code = NotFound desc = could not find container \"196d809c625b6d7be7a93108322dbb398749088ab5233d8d6400d8ad5534eed2\": container with ID starting with 196d809c625b6d7be7a93108322dbb398749088ab5233d8d6400d8ad5534eed2 not found: ID does not exist" Dec 03 12:28:52 crc kubenswrapper[5052]: I1203 12:28:52.492724 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:28:52 crc kubenswrapper[5052]: I1203 12:28:52.506421 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sn676\" (UniqueName: \"kubernetes.io/projected/a65d6de5-f108-40c9-81f6-ca37534c7710-kube-api-access-sn676\") pod \"ceilometer-0\" (UID: \"a65d6de5-f108-40c9-81f6-ca37534c7710\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:28:52 crc kubenswrapper[5052]: I1203 12:28:52.506472 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/8da1e6bc-5fd9-4886-ae5e-01a1dbd307e7-config-data\") pod \"nova-metadata-0\" (UID: \"8da1e6bc-5fd9-4886-ae5e-01a1dbd307e7\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:28:52 crc kubenswrapper[5052]: I1203 12:28:52.506551 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/9632f411-9c90-4a2c-9c79-7a2138f0ff43-internal-tls-certs\") pod \"nova-api-0\" (UID: \"9632f411-9c90-4a2c-9c79-7a2138f0ff43\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:28:52 crc kubenswrapper[5052]: I1203 12:28:52.506630 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/9632f411-9c90-4a2c-9c79-7a2138f0ff43-config-data\") pod \"nova-api-0\" (UID: \"9632f411-9c90-4a2c-9c79-7a2138f0ff43\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:28:52 crc kubenswrapper[5052]: I1203 12:28:52.506693 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/a65d6de5-f108-40c9-81f6-ca37534c7710-log-httpd\") pod \"ceilometer-0\" (UID: \"a65d6de5-f108-40c9-81f6-ca37534c7710\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:28:52 crc kubenswrapper[5052]: I1203 12:28:52.506724 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a65d6de5-f108-40c9-81f6-ca37534c7710-config-data\") pod \"ceilometer-0\" (UID: \"a65d6de5-f108-40c9-81f6-ca37534c7710\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:28:52 crc kubenswrapper[5052]: I1203 12:28:52.506757 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/a65d6de5-f108-40c9-81f6-ca37534c7710-ceilometer-tls-certs\") pod \"ceilometer-0\" (UID: \"a65d6de5-f108-40c9-81f6-ca37534c7710\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:28:52 crc kubenswrapper[5052]: I1203 12:28:52.506783 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8da1e6bc-5fd9-4886-ae5e-01a1dbd307e7-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"8da1e6bc-5fd9-4886-ae5e-01a1dbd307e7\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:28:52 crc kubenswrapper[5052]: I1203 12:28:52.506811 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/9632f411-9c90-4a2c-9c79-7a2138f0ff43-logs\") pod \"nova-api-0\" (UID: \"9632f411-9c90-4a2c-9c79-7a2138f0ff43\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:28:52 crc kubenswrapper[5052]: I1203 12:28:52.506862 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/8da1e6bc-5fd9-4886-ae5e-01a1dbd307e7-logs\") pod \"nova-metadata-0\" (UID: \"8da1e6bc-5fd9-4886-ae5e-01a1dbd307e7\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:28:52 crc kubenswrapper[5052]: I1203 12:28:52.506896 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/a65d6de5-f108-40c9-81f6-ca37534c7710-run-httpd\") pod \"ceilometer-0\" (UID: \"a65d6de5-f108-40c9-81f6-ca37534c7710\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:28:52 crc kubenswrapper[5052]: I1203 12:28:52.506930 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g4jvq\" (UniqueName: \"kubernetes.io/projected/8da1e6bc-5fd9-4886-ae5e-01a1dbd307e7-kube-api-access-g4jvq\") pod \"nova-metadata-0\" (UID: \"8da1e6bc-5fd9-4886-ae5e-01a1dbd307e7\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:28:52 crc kubenswrapper[5052]: I1203 12:28:52.506967 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/9632f411-9c90-4a2c-9c79-7a2138f0ff43-public-tls-certs\") pod \"nova-api-0\" (UID: \"9632f411-9c90-4a2c-9c79-7a2138f0ff43\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:28:52 crc kubenswrapper[5052]: I1203 12:28:52.507000 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-fqjs7\" (UniqueName: \"kubernetes.io/projected/9632f411-9c90-4a2c-9c79-7a2138f0ff43-kube-api-access-fqjs7\") pod \"nova-api-0\" (UID: \"9632f411-9c90-4a2c-9c79-7a2138f0ff43\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:28:52 crc kubenswrapper[5052]: I1203 12:28:52.507033 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9632f411-9c90-4a2c-9c79-7a2138f0ff43-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"9632f411-9c90-4a2c-9c79-7a2138f0ff43\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:28:52 crc kubenswrapper[5052]: I1203 12:28:52.507063 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/a65d6de5-f108-40c9-81f6-ca37534c7710-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"a65d6de5-f108-40c9-81f6-ca37534c7710\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:28:52 crc kubenswrapper[5052]: I1203 12:28:52.507086 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a65d6de5-f108-40c9-81f6-ca37534c7710-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"a65d6de5-f108-40c9-81f6-ca37534c7710\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:28:52 crc kubenswrapper[5052]: I1203 12:28:52.507121 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/a65d6de5-f108-40c9-81f6-ca37534c7710-scripts\") pod \"ceilometer-0\" (UID: \"a65d6de5-f108-40c9-81f6-ca37534c7710\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:28:52 crc kubenswrapper[5052]: I1203 12:28:52.507147 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/8da1e6bc-5fd9-4886-ae5e-01a1dbd307e7-nova-metadata-tls-certs\") pod \"nova-metadata-0\" (UID: \"8da1e6bc-5fd9-4886-ae5e-01a1dbd307e7\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:28:52 crc kubenswrapper[5052]: I1203 12:28:52.513578 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/9632f411-9c90-4a2c-9c79-7a2138f0ff43-logs\") pod \"nova-api-0\" (UID: \"9632f411-9c90-4a2c-9c79-7a2138f0ff43\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:28:52 crc kubenswrapper[5052]: I1203 12:28:52.514770 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/9632f411-9c90-4a2c-9c79-7a2138f0ff43-internal-tls-certs\") pod \"nova-api-0\" (UID: \"9632f411-9c90-4a2c-9c79-7a2138f0ff43\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:28:52 crc kubenswrapper[5052]: I1203 12:28:52.515034 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/9632f411-9c90-4a2c-9c79-7a2138f0ff43-config-data\") pod \"nova-api-0\" (UID: \"9632f411-9c90-4a2c-9c79-7a2138f0ff43\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:28:52 crc kubenswrapper[5052]: I1203 12:28:52.516000 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9632f411-9c90-4a2c-9c79-7a2138f0ff43-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"9632f411-9c90-4a2c-9c79-7a2138f0ff43\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:28:52 crc kubenswrapper[5052]: I1203 12:28:52.517726 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/9632f411-9c90-4a2c-9c79-7a2138f0ff43-public-tls-certs\") pod \"nova-api-0\" (UID: \"9632f411-9c90-4a2c-9c79-7a2138f0ff43\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:28:52 crc kubenswrapper[5052]: I1203 12:28:52.534749 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-fqjs7\" (UniqueName: \"kubernetes.io/projected/9632f411-9c90-4a2c-9c79-7a2138f0ff43-kube-api-access-fqjs7\") pod \"nova-api-0\" (UID: \"9632f411-9c90-4a2c-9c79-7a2138f0ff43\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:28:52 crc kubenswrapper[5052]: I1203 12:28:52.609482 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/a65d6de5-f108-40c9-81f6-ca37534c7710-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"a65d6de5-f108-40c9-81f6-ca37534c7710\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:28:52 crc kubenswrapper[5052]: I1203 12:28:52.609545 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a65d6de5-f108-40c9-81f6-ca37534c7710-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"a65d6de5-f108-40c9-81f6-ca37534c7710\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:28:52 crc kubenswrapper[5052]: I1203 12:28:52.609612 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/a65d6de5-f108-40c9-81f6-ca37534c7710-scripts\") pod \"ceilometer-0\" (UID: \"a65d6de5-f108-40c9-81f6-ca37534c7710\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:28:52 crc kubenswrapper[5052]: I1203 12:28:52.609649 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/8da1e6bc-5fd9-4886-ae5e-01a1dbd307e7-nova-metadata-tls-certs\") pod \"nova-metadata-0\" (UID: \"8da1e6bc-5fd9-4886-ae5e-01a1dbd307e7\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:28:52 crc kubenswrapper[5052]: I1203 12:28:52.609679 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-sn676\" (UniqueName: \"kubernetes.io/projected/a65d6de5-f108-40c9-81f6-ca37534c7710-kube-api-access-sn676\") pod \"ceilometer-0\" (UID: \"a65d6de5-f108-40c9-81f6-ca37534c7710\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:28:52 crc kubenswrapper[5052]: I1203 12:28:52.609694 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/8da1e6bc-5fd9-4886-ae5e-01a1dbd307e7-config-data\") pod \"nova-metadata-0\" (UID: \"8da1e6bc-5fd9-4886-ae5e-01a1dbd307e7\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:28:52 crc kubenswrapper[5052]: I1203 12:28:52.609907 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/a65d6de5-f108-40c9-81f6-ca37534c7710-log-httpd\") pod \"ceilometer-0\" (UID: \"a65d6de5-f108-40c9-81f6-ca37534c7710\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:28:52 crc kubenswrapper[5052]: I1203 12:28:52.609968 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a65d6de5-f108-40c9-81f6-ca37534c7710-config-data\") pod \"ceilometer-0\" (UID: \"a65d6de5-f108-40c9-81f6-ca37534c7710\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:28:52 crc kubenswrapper[5052]: I1203 12:28:52.609990 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/a65d6de5-f108-40c9-81f6-ca37534c7710-ceilometer-tls-certs\") pod \"ceilometer-0\" (UID: \"a65d6de5-f108-40c9-81f6-ca37534c7710\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:28:52 crc kubenswrapper[5052]: I1203 12:28:52.610009 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8da1e6bc-5fd9-4886-ae5e-01a1dbd307e7-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"8da1e6bc-5fd9-4886-ae5e-01a1dbd307e7\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:28:52 crc kubenswrapper[5052]: I1203 12:28:52.610044 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/8da1e6bc-5fd9-4886-ae5e-01a1dbd307e7-logs\") pod \"nova-metadata-0\" (UID: \"8da1e6bc-5fd9-4886-ae5e-01a1dbd307e7\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:28:52 crc kubenswrapper[5052]: I1203 12:28:52.610063 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/a65d6de5-f108-40c9-81f6-ca37534c7710-run-httpd\") pod \"ceilometer-0\" (UID: \"a65d6de5-f108-40c9-81f6-ca37534c7710\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:28:52 crc kubenswrapper[5052]: I1203 12:28:52.610088 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-g4jvq\" (UniqueName: \"kubernetes.io/projected/8da1e6bc-5fd9-4886-ae5e-01a1dbd307e7-kube-api-access-g4jvq\") pod \"nova-metadata-0\" (UID: \"8da1e6bc-5fd9-4886-ae5e-01a1dbd307e7\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:28:52 crc kubenswrapper[5052]: I1203 12:28:52.611919 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/a65d6de5-f108-40c9-81f6-ca37534c7710-log-httpd\") pod \"ceilometer-0\" (UID: \"a65d6de5-f108-40c9-81f6-ca37534c7710\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:28:52 crc kubenswrapper[5052]: I1203 12:28:52.612497 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/8da1e6bc-5fd9-4886-ae5e-01a1dbd307e7-logs\") pod \"nova-metadata-0\" (UID: \"8da1e6bc-5fd9-4886-ae5e-01a1dbd307e7\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:28:52 crc kubenswrapper[5052]: I1203 12:28:52.612831 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/a65d6de5-f108-40c9-81f6-ca37534c7710-run-httpd\") pod \"ceilometer-0\" (UID: \"a65d6de5-f108-40c9-81f6-ca37534c7710\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:28:52 crc kubenswrapper[5052]: I1203 12:28:52.629005 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/a65d6de5-f108-40c9-81f6-ca37534c7710-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"a65d6de5-f108-40c9-81f6-ca37534c7710\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:28:52 crc kubenswrapper[5052]: I1203 12:28:52.629099 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a65d6de5-f108-40c9-81f6-ca37534c7710-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"a65d6de5-f108-40c9-81f6-ca37534c7710\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:28:52 crc kubenswrapper[5052]: I1203 12:28:52.629223 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a65d6de5-f108-40c9-81f6-ca37534c7710-config-data\") pod \"ceilometer-0\" (UID: \"a65d6de5-f108-40c9-81f6-ca37534c7710\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:28:52 crc kubenswrapper[5052]: I1203 12:28:52.629232 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/a65d6de5-f108-40c9-81f6-ca37534c7710-ceilometer-tls-certs\") pod \"ceilometer-0\" (UID: \"a65d6de5-f108-40c9-81f6-ca37534c7710\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:28:52 crc kubenswrapper[5052]: I1203 12:28:52.629236 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8da1e6bc-5fd9-4886-ae5e-01a1dbd307e7-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"8da1e6bc-5fd9-4886-ae5e-01a1dbd307e7\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:28:52 crc kubenswrapper[5052]: I1203 12:28:52.629382 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/8da1e6bc-5fd9-4886-ae5e-01a1dbd307e7-config-data\") pod \"nova-metadata-0\" (UID: \"8da1e6bc-5fd9-4886-ae5e-01a1dbd307e7\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:28:52 crc kubenswrapper[5052]: I1203 12:28:52.629602 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/8da1e6bc-5fd9-4886-ae5e-01a1dbd307e7-nova-metadata-tls-certs\") pod \"nova-metadata-0\" (UID: \"8da1e6bc-5fd9-4886-ae5e-01a1dbd307e7\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:28:52 crc kubenswrapper[5052]: I1203 12:28:52.633240 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-g4jvq\" (UniqueName: \"kubernetes.io/projected/8da1e6bc-5fd9-4886-ae5e-01a1dbd307e7-kube-api-access-g4jvq\") pod \"nova-metadata-0\" (UID: \"8da1e6bc-5fd9-4886-ae5e-01a1dbd307e7\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:28:52 crc kubenswrapper[5052]: I1203 12:28:52.633255 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-sn676\" (UniqueName: \"kubernetes.io/projected/a65d6de5-f108-40c9-81f6-ca37534c7710-kube-api-access-sn676\") pod \"ceilometer-0\" (UID: \"a65d6de5-f108-40c9-81f6-ca37534c7710\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:28:52 crc kubenswrapper[5052]: I1203 12:28:52.633638 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/a65d6de5-f108-40c9-81f6-ca37534c7710-scripts\") pod \"ceilometer-0\" (UID: \"a65d6de5-f108-40c9-81f6-ca37534c7710\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:28:52 crc kubenswrapper[5052]: I1203 12:28:52.696478 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:28:52 crc kubenswrapper[5052]: I1203 12:28:52.733578 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:28:52 crc kubenswrapper[5052]: I1203 12:28:52.785087 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:28:53 crc kubenswrapper[5052]: W1203 12:28:53.237654 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9632f411_9c90_4a2c_9c79_7a2138f0ff43.slice/crio-46802a9fcd3cca32970ff223109fd1f6599c5af0684d81be0309e7266636cb58 WatchSource:0}: Error finding container 46802a9fcd3cca32970ff223109fd1f6599c5af0684d81be0309e7266636cb58: Status 404 returned error can't find the container with id 46802a9fcd3cca32970ff223109fd1f6599c5af0684d81be0309e7266636cb58 Dec 03 12:28:53 crc kubenswrapper[5052]: I1203 12:28:53.248195 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:28:53 crc kubenswrapper[5052]: I1203 12:28:53.338637 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:28:53 crc kubenswrapper[5052]: I1203 12:28:53.347311 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:28:53 crc kubenswrapper[5052]: W1203 12:28:53.391982 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8da1e6bc_5fd9_4886_ae5e_01a1dbd307e7.slice/crio-badec2b73b71ba2cfcec93c0d16127a834889a9d7cc237301f7abaa9ad32431a WatchSource:0}: Error finding container badec2b73b71ba2cfcec93c0d16127a834889a9d7cc237301f7abaa9ad32431a: Status 404 returned error can't find the container with id badec2b73b71ba2cfcec93c0d16127a834889a9d7cc237301f7abaa9ad32431a Dec 03 12:28:53 crc kubenswrapper[5052]: I1203 12:28:53.393778 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:28:53 crc kubenswrapper[5052]: I1203 12:28:53.506574 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:28:53 crc kubenswrapper[5052]: I1203 12:28:53.531048 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:28:53 crc kubenswrapper[5052]: I1203 12:28:53.581944 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:28:53 crc kubenswrapper[5052]: I1203 12:28:53.669110 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:28:53 crc kubenswrapper[5052]: I1203 12:28:53.739624 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a740ecee-4c7e-4e42-bfb8-8119ffbfcca0" path="/var/lib/kubelet/pods/a740ecee-4c7e-4e42-bfb8-8119ffbfcca0/volumes" Dec 03 12:28:53 crc kubenswrapper[5052]: I1203 12:28:53.753972 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b72919f4-0bc0-444b-9ddb-859d386a53b5" path="/var/lib/kubelet/pods/b72919f4-0bc0-444b-9ddb-859d386a53b5/volumes" Dec 03 12:28:53 crc kubenswrapper[5052]: I1203 12:28:53.756142 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e7e769bc-cba8-4955-9d3d-14d1acc7fb5b" path="/var/lib/kubelet/pods/e7e769bc-cba8-4955-9d3d-14d1acc7fb5b/volumes" Dec 03 12:28:53 crc kubenswrapper[5052]: I1203 12:28:53.847182 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"a65d6de5-f108-40c9-81f6-ca37534c7710","Type":"ContainerStarted","Data":"7994bb2b17aebae05cb47ee8c2ba1bddcc96649268b928224de1f42b453d8f40"} Dec 03 12:28:53 crc kubenswrapper[5052]: I1203 12:28:53.850720 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"8da1e6bc-5fd9-4886-ae5e-01a1dbd307e7","Type":"ContainerStarted","Data":"a2fdb86f428561f36257e1c0427682d91b3c9d697feb5500615c2839dbc040b8"} Dec 03 12:28:53 crc kubenswrapper[5052]: I1203 12:28:53.850789 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"8da1e6bc-5fd9-4886-ae5e-01a1dbd307e7","Type":"ContainerStarted","Data":"badec2b73b71ba2cfcec93c0d16127a834889a9d7cc237301f7abaa9ad32431a"} Dec 03 12:28:53 crc kubenswrapper[5052]: I1203 12:28:53.852768 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"9632f411-9c90-4a2c-9c79-7a2138f0ff43","Type":"ContainerStarted","Data":"46f54610e1f342af288f0ebfbb47c0d93619fe37f5035a7b333c0f28e092259a"} Dec 03 12:28:53 crc kubenswrapper[5052]: I1203 12:28:53.852823 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"9632f411-9c90-4a2c-9c79-7a2138f0ff43","Type":"ContainerStarted","Data":"46802a9fcd3cca32970ff223109fd1f6599c5af0684d81be0309e7266636cb58"} Dec 03 12:28:54 crc kubenswrapper[5052]: I1203 12:28:54.864454 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"a65d6de5-f108-40c9-81f6-ca37534c7710","Type":"ContainerStarted","Data":"e0121d77497086b42ab3b08972dbfa1ef63c206df7997ec8f0f64203225a2322"} Dec 03 12:28:54 crc kubenswrapper[5052]: I1203 12:28:54.867415 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"8da1e6bc-5fd9-4886-ae5e-01a1dbd307e7","Type":"ContainerStarted","Data":"07c66122c20f41a0d54c2503579b4ef239da6f910b95c44525198f30ad0ed3f0"} Dec 03 12:28:54 crc kubenswrapper[5052]: I1203 12:28:54.867583 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-metadata-0" podUID="8da1e6bc-5fd9-4886-ae5e-01a1dbd307e7" containerName="nova-metadata-log" containerID="cri-o://a2fdb86f428561f36257e1c0427682d91b3c9d697feb5500615c2839dbc040b8" gracePeriod=30 Dec 03 12:28:54 crc kubenswrapper[5052]: I1203 12:28:54.867628 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-metadata-0" podUID="8da1e6bc-5fd9-4886-ae5e-01a1dbd307e7" containerName="nova-metadata-metadata" containerID="cri-o://07c66122c20f41a0d54c2503579b4ef239da6f910b95c44525198f30ad0ed3f0" gracePeriod=30 Dec 03 12:28:54 crc kubenswrapper[5052]: I1203 12:28:54.871082 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"9632f411-9c90-4a2c-9c79-7a2138f0ff43","Type":"ContainerStarted","Data":"d598054ff3cc894755197631e8e45c78cce05130d3836690788ea53b61d7d3a5"} Dec 03 12:28:54 crc kubenswrapper[5052]: I1203 12:28:54.871206 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-api-0" podUID="9632f411-9c90-4a2c-9c79-7a2138f0ff43" containerName="nova-api-log" containerID="cri-o://46f54610e1f342af288f0ebfbb47c0d93619fe37f5035a7b333c0f28e092259a" gracePeriod=30 Dec 03 12:28:54 crc kubenswrapper[5052]: I1203 12:28:54.871331 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-api-0" podUID="9632f411-9c90-4a2c-9c79-7a2138f0ff43" containerName="nova-api-api" containerID="cri-o://d598054ff3cc894755197631e8e45c78cce05130d3836690788ea53b61d7d3a5" gracePeriod=30 Dec 03 12:28:54 crc kubenswrapper[5052]: I1203 12:28:54.901948 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-metadata-0" podStartSLOduration=2.90192771 podStartE2EDuration="2.90192771s" podCreationTimestamp="2025-12-03 12:28:52 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:28:54.898492732 +0000 UTC m=+4125.486734496" watchObservedRunningTime="2025-12-03 12:28:54.90192771 +0000 UTC m=+4125.490169474" Dec 03 12:28:54 crc kubenswrapper[5052]: I1203 12:28:54.946335 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-api-0" podStartSLOduration=2.946315667 podStartE2EDuration="2.946315667s" podCreationTimestamp="2025-12-03 12:28:52 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:28:54.933485031 +0000 UTC m=+4125.521726815" watchObservedRunningTime="2025-12-03 12:28:54.946315667 +0000 UTC m=+4125.534557431" Dec 03 12:28:55 crc kubenswrapper[5052]: I1203 12:28:55.421272 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:28:55 crc kubenswrapper[5052]: I1203 12:28:55.447707 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:28:55 crc kubenswrapper[5052]: I1203 12:28:55.498560 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/9632f411-9c90-4a2c-9c79-7a2138f0ff43-public-tls-certs\") pod \"9632f411-9c90-4a2c-9c79-7a2138f0ff43\" (UID: \"9632f411-9c90-4a2c-9c79-7a2138f0ff43\") " Dec 03 12:28:55 crc kubenswrapper[5052]: I1203 12:28:55.498653 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/9632f411-9c90-4a2c-9c79-7a2138f0ff43-internal-tls-certs\") pod \"9632f411-9c90-4a2c-9c79-7a2138f0ff43\" (UID: \"9632f411-9c90-4a2c-9c79-7a2138f0ff43\") " Dec 03 12:28:55 crc kubenswrapper[5052]: I1203 12:28:55.498740 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/8da1e6bc-5fd9-4886-ae5e-01a1dbd307e7-nova-metadata-tls-certs\") pod \"8da1e6bc-5fd9-4886-ae5e-01a1dbd307e7\" (UID: \"8da1e6bc-5fd9-4886-ae5e-01a1dbd307e7\") " Dec 03 12:28:55 crc kubenswrapper[5052]: I1203 12:28:55.498776 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/8da1e6bc-5fd9-4886-ae5e-01a1dbd307e7-logs\") pod \"8da1e6bc-5fd9-4886-ae5e-01a1dbd307e7\" (UID: \"8da1e6bc-5fd9-4886-ae5e-01a1dbd307e7\") " Dec 03 12:28:55 crc kubenswrapper[5052]: I1203 12:28:55.498794 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fqjs7\" (UniqueName: \"kubernetes.io/projected/9632f411-9c90-4a2c-9c79-7a2138f0ff43-kube-api-access-fqjs7\") pod \"9632f411-9c90-4a2c-9c79-7a2138f0ff43\" (UID: \"9632f411-9c90-4a2c-9c79-7a2138f0ff43\") " Dec 03 12:28:55 crc kubenswrapper[5052]: I1203 12:28:55.498823 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/8da1e6bc-5fd9-4886-ae5e-01a1dbd307e7-config-data\") pod \"8da1e6bc-5fd9-4886-ae5e-01a1dbd307e7\" (UID: \"8da1e6bc-5fd9-4886-ae5e-01a1dbd307e7\") " Dec 03 12:28:55 crc kubenswrapper[5052]: I1203 12:28:55.498861 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/9632f411-9c90-4a2c-9c79-7a2138f0ff43-config-data\") pod \"9632f411-9c90-4a2c-9c79-7a2138f0ff43\" (UID: \"9632f411-9c90-4a2c-9c79-7a2138f0ff43\") " Dec 03 12:28:55 crc kubenswrapper[5052]: I1203 12:28:55.498933 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8da1e6bc-5fd9-4886-ae5e-01a1dbd307e7-combined-ca-bundle\") pod \"8da1e6bc-5fd9-4886-ae5e-01a1dbd307e7\" (UID: \"8da1e6bc-5fd9-4886-ae5e-01a1dbd307e7\") " Dec 03 12:28:55 crc kubenswrapper[5052]: I1203 12:28:55.498953 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9632f411-9c90-4a2c-9c79-7a2138f0ff43-combined-ca-bundle\") pod \"9632f411-9c90-4a2c-9c79-7a2138f0ff43\" (UID: \"9632f411-9c90-4a2c-9c79-7a2138f0ff43\") " Dec 03 12:28:55 crc kubenswrapper[5052]: I1203 12:28:55.498973 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g4jvq\" (UniqueName: \"kubernetes.io/projected/8da1e6bc-5fd9-4886-ae5e-01a1dbd307e7-kube-api-access-g4jvq\") pod \"8da1e6bc-5fd9-4886-ae5e-01a1dbd307e7\" (UID: \"8da1e6bc-5fd9-4886-ae5e-01a1dbd307e7\") " Dec 03 12:28:55 crc kubenswrapper[5052]: I1203 12:28:55.498991 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/9632f411-9c90-4a2c-9c79-7a2138f0ff43-logs\") pod \"9632f411-9c90-4a2c-9c79-7a2138f0ff43\" (UID: \"9632f411-9c90-4a2c-9c79-7a2138f0ff43\") " Dec 03 12:28:55 crc kubenswrapper[5052]: I1203 12:28:55.499821 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9632f411-9c90-4a2c-9c79-7a2138f0ff43-logs" (OuterVolumeSpecName: "logs") pod "9632f411-9c90-4a2c-9c79-7a2138f0ff43" (UID: "9632f411-9c90-4a2c-9c79-7a2138f0ff43"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:28:55 crc kubenswrapper[5052]: I1203 12:28:55.501317 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8da1e6bc-5fd9-4886-ae5e-01a1dbd307e7-logs" (OuterVolumeSpecName: "logs") pod "8da1e6bc-5fd9-4886-ae5e-01a1dbd307e7" (UID: "8da1e6bc-5fd9-4886-ae5e-01a1dbd307e7"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:28:55 crc kubenswrapper[5052]: I1203 12:28:55.526809 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8da1e6bc-5fd9-4886-ae5e-01a1dbd307e7-kube-api-access-g4jvq" (OuterVolumeSpecName: "kube-api-access-g4jvq") pod "8da1e6bc-5fd9-4886-ae5e-01a1dbd307e7" (UID: "8da1e6bc-5fd9-4886-ae5e-01a1dbd307e7"). InnerVolumeSpecName "kube-api-access-g4jvq". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:28:55 crc kubenswrapper[5052]: I1203 12:28:55.529237 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9632f411-9c90-4a2c-9c79-7a2138f0ff43-config-data" (OuterVolumeSpecName: "config-data") pod "9632f411-9c90-4a2c-9c79-7a2138f0ff43" (UID: "9632f411-9c90-4a2c-9c79-7a2138f0ff43"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:28:55 crc kubenswrapper[5052]: I1203 12:28:55.531290 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9632f411-9c90-4a2c-9c79-7a2138f0ff43-kube-api-access-fqjs7" (OuterVolumeSpecName: "kube-api-access-fqjs7") pod "9632f411-9c90-4a2c-9c79-7a2138f0ff43" (UID: "9632f411-9c90-4a2c-9c79-7a2138f0ff43"). InnerVolumeSpecName "kube-api-access-fqjs7". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:28:55 crc kubenswrapper[5052]: I1203 12:28:55.540916 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8da1e6bc-5fd9-4886-ae5e-01a1dbd307e7-config-data" (OuterVolumeSpecName: "config-data") pod "8da1e6bc-5fd9-4886-ae5e-01a1dbd307e7" (UID: "8da1e6bc-5fd9-4886-ae5e-01a1dbd307e7"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:28:55 crc kubenswrapper[5052]: I1203 12:28:55.553411 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8da1e6bc-5fd9-4886-ae5e-01a1dbd307e7-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "8da1e6bc-5fd9-4886-ae5e-01a1dbd307e7" (UID: "8da1e6bc-5fd9-4886-ae5e-01a1dbd307e7"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:28:55 crc kubenswrapper[5052]: I1203 12:28:55.554741 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9632f411-9c90-4a2c-9c79-7a2138f0ff43-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "9632f411-9c90-4a2c-9c79-7a2138f0ff43" (UID: "9632f411-9c90-4a2c-9c79-7a2138f0ff43"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:28:55 crc kubenswrapper[5052]: I1203 12:28:55.560949 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9632f411-9c90-4a2c-9c79-7a2138f0ff43-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "9632f411-9c90-4a2c-9c79-7a2138f0ff43" (UID: "9632f411-9c90-4a2c-9c79-7a2138f0ff43"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:28:55 crc kubenswrapper[5052]: I1203 12:28:55.564277 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/swift-proxy-84cc784c4-9pgx5" Dec 03 12:28:55 crc kubenswrapper[5052]: I1203 12:28:55.568532 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/swift-proxy-84cc784c4-9pgx5" Dec 03 12:28:55 crc kubenswrapper[5052]: I1203 12:28:55.584491 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8da1e6bc-5fd9-4886-ae5e-01a1dbd307e7-nova-metadata-tls-certs" (OuterVolumeSpecName: "nova-metadata-tls-certs") pod "8da1e6bc-5fd9-4886-ae5e-01a1dbd307e7" (UID: "8da1e6bc-5fd9-4886-ae5e-01a1dbd307e7"). InnerVolumeSpecName "nova-metadata-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:28:55 crc kubenswrapper[5052]: I1203 12:28:55.584526 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9632f411-9c90-4a2c-9c79-7a2138f0ff43-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "9632f411-9c90-4a2c-9c79-7a2138f0ff43" (UID: "9632f411-9c90-4a2c-9c79-7a2138f0ff43"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:28:55 crc kubenswrapper[5052]: I1203 12:28:55.604079 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8da1e6bc-5fd9-4886-ae5e-01a1dbd307e7-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:55 crc kubenswrapper[5052]: I1203 12:28:55.604115 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9632f411-9c90-4a2c-9c79-7a2138f0ff43-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:55 crc kubenswrapper[5052]: I1203 12:28:55.604128 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-g4jvq\" (UniqueName: \"kubernetes.io/projected/8da1e6bc-5fd9-4886-ae5e-01a1dbd307e7-kube-api-access-g4jvq\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:55 crc kubenswrapper[5052]: I1203 12:28:55.604138 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/9632f411-9c90-4a2c-9c79-7a2138f0ff43-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:55 crc kubenswrapper[5052]: I1203 12:28:55.604148 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/9632f411-9c90-4a2c-9c79-7a2138f0ff43-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:55 crc kubenswrapper[5052]: I1203 12:28:55.604156 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/9632f411-9c90-4a2c-9c79-7a2138f0ff43-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:55 crc kubenswrapper[5052]: I1203 12:28:55.604165 5052 reconciler_common.go:293] "Volume detached for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/8da1e6bc-5fd9-4886-ae5e-01a1dbd307e7-nova-metadata-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:55 crc kubenswrapper[5052]: I1203 12:28:55.604173 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/8da1e6bc-5fd9-4886-ae5e-01a1dbd307e7-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:55 crc kubenswrapper[5052]: I1203 12:28:55.604183 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-fqjs7\" (UniqueName: \"kubernetes.io/projected/9632f411-9c90-4a2c-9c79-7a2138f0ff43-kube-api-access-fqjs7\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:55 crc kubenswrapper[5052]: I1203 12:28:55.604191 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/8da1e6bc-5fd9-4886-ae5e-01a1dbd307e7-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:55 crc kubenswrapper[5052]: I1203 12:28:55.604201 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/9632f411-9c90-4a2c-9c79-7a2138f0ff43-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:55 crc kubenswrapper[5052]: I1203 12:28:55.698411 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/swift-proxy-84c4f5d658-m9l78"] Dec 03 12:28:55 crc kubenswrapper[5052]: I1203 12:28:55.698735 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-proxy-84c4f5d658-m9l78" podUID="9f515435-a645-4d2b-9bd5-4bb9496f5405" containerName="proxy-httpd" containerID="cri-o://7401b98806090e0288c0ece96883b73c2a54d14bd85bbf4ba5729b51cadf8ea2" gracePeriod=30 Dec 03 12:28:55 crc kubenswrapper[5052]: I1203 12:28:55.698841 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-proxy-84c4f5d658-m9l78" podUID="9f515435-a645-4d2b-9bd5-4bb9496f5405" containerName="proxy-server" containerID="cri-o://41ad3ff3a7101ea8b3778d199fc0aa3d823057eb0199c3f55a79043a27526808" gracePeriod=30 Dec 03 12:28:55 crc kubenswrapper[5052]: I1203 12:28:55.889929 5052 generic.go:334] "Generic (PLEG): container finished" podID="9f515435-a645-4d2b-9bd5-4bb9496f5405" containerID="7401b98806090e0288c0ece96883b73c2a54d14bd85bbf4ba5729b51cadf8ea2" exitCode=0 Dec 03 12:28:55 crc kubenswrapper[5052]: I1203 12:28:55.890012 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-proxy-84c4f5d658-m9l78" event={"ID":"9f515435-a645-4d2b-9bd5-4bb9496f5405","Type":"ContainerDied","Data":"7401b98806090e0288c0ece96883b73c2a54d14bd85bbf4ba5729b51cadf8ea2"} Dec 03 12:28:55 crc kubenswrapper[5052]: I1203 12:28:55.892580 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"a65d6de5-f108-40c9-81f6-ca37534c7710","Type":"ContainerStarted","Data":"55f42c22fa007045b7bcac5be13d03999c40148b147e8c2a61d685132dd50766"} Dec 03 12:28:55 crc kubenswrapper[5052]: I1203 12:28:55.894904 5052 generic.go:334] "Generic (PLEG): container finished" podID="8da1e6bc-5fd9-4886-ae5e-01a1dbd307e7" containerID="07c66122c20f41a0d54c2503579b4ef239da6f910b95c44525198f30ad0ed3f0" exitCode=0 Dec 03 12:28:55 crc kubenswrapper[5052]: I1203 12:28:55.894928 5052 generic.go:334] "Generic (PLEG): container finished" podID="8da1e6bc-5fd9-4886-ae5e-01a1dbd307e7" containerID="a2fdb86f428561f36257e1c0427682d91b3c9d697feb5500615c2839dbc040b8" exitCode=143 Dec 03 12:28:55 crc kubenswrapper[5052]: I1203 12:28:55.894978 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"8da1e6bc-5fd9-4886-ae5e-01a1dbd307e7","Type":"ContainerDied","Data":"07c66122c20f41a0d54c2503579b4ef239da6f910b95c44525198f30ad0ed3f0"} Dec 03 12:28:55 crc kubenswrapper[5052]: I1203 12:28:55.895001 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"8da1e6bc-5fd9-4886-ae5e-01a1dbd307e7","Type":"ContainerDied","Data":"a2fdb86f428561f36257e1c0427682d91b3c9d697feb5500615c2839dbc040b8"} Dec 03 12:28:55 crc kubenswrapper[5052]: I1203 12:28:55.895012 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"8da1e6bc-5fd9-4886-ae5e-01a1dbd307e7","Type":"ContainerDied","Data":"badec2b73b71ba2cfcec93c0d16127a834889a9d7cc237301f7abaa9ad32431a"} Dec 03 12:28:55 crc kubenswrapper[5052]: I1203 12:28:55.895027 5052 scope.go:117] "RemoveContainer" containerID="07c66122c20f41a0d54c2503579b4ef239da6f910b95c44525198f30ad0ed3f0" Dec 03 12:28:55 crc kubenswrapper[5052]: I1203 12:28:55.895044 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:28:55 crc kubenswrapper[5052]: I1203 12:28:55.905660 5052 generic.go:334] "Generic (PLEG): container finished" podID="9632f411-9c90-4a2c-9c79-7a2138f0ff43" containerID="d598054ff3cc894755197631e8e45c78cce05130d3836690788ea53b61d7d3a5" exitCode=0 Dec 03 12:28:55 crc kubenswrapper[5052]: I1203 12:28:55.905687 5052 generic.go:334] "Generic (PLEG): container finished" podID="9632f411-9c90-4a2c-9c79-7a2138f0ff43" containerID="46f54610e1f342af288f0ebfbb47c0d93619fe37f5035a7b333c0f28e092259a" exitCode=143 Dec 03 12:28:55 crc kubenswrapper[5052]: I1203 12:28:55.905774 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:28:55 crc kubenswrapper[5052]: I1203 12:28:55.905830 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"9632f411-9c90-4a2c-9c79-7a2138f0ff43","Type":"ContainerDied","Data":"d598054ff3cc894755197631e8e45c78cce05130d3836690788ea53b61d7d3a5"} Dec 03 12:28:55 crc kubenswrapper[5052]: I1203 12:28:55.905861 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"9632f411-9c90-4a2c-9c79-7a2138f0ff43","Type":"ContainerDied","Data":"46f54610e1f342af288f0ebfbb47c0d93619fe37f5035a7b333c0f28e092259a"} Dec 03 12:28:55 crc kubenswrapper[5052]: I1203 12:28:55.905872 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"9632f411-9c90-4a2c-9c79-7a2138f0ff43","Type":"ContainerDied","Data":"46802a9fcd3cca32970ff223109fd1f6599c5af0684d81be0309e7266636cb58"} Dec 03 12:28:55 crc kubenswrapper[5052]: I1203 12:28:55.929497 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:28:55 crc kubenswrapper[5052]: I1203 12:28:55.948777 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:28:55 crc kubenswrapper[5052]: I1203 12:28:55.965043 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:28:55 crc kubenswrapper[5052]: E1203 12:28:55.965638 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="8da1e6bc-5fd9-4886-ae5e-01a1dbd307e7" containerName="nova-metadata-metadata" Dec 03 12:28:55 crc kubenswrapper[5052]: I1203 12:28:55.965662 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="8da1e6bc-5fd9-4886-ae5e-01a1dbd307e7" containerName="nova-metadata-metadata" Dec 03 12:28:55 crc kubenswrapper[5052]: E1203 12:28:55.965684 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="8da1e6bc-5fd9-4886-ae5e-01a1dbd307e7" containerName="nova-metadata-log" Dec 03 12:28:55 crc kubenswrapper[5052]: I1203 12:28:55.965693 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="8da1e6bc-5fd9-4886-ae5e-01a1dbd307e7" containerName="nova-metadata-log" Dec 03 12:28:55 crc kubenswrapper[5052]: E1203 12:28:55.965714 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="9632f411-9c90-4a2c-9c79-7a2138f0ff43" containerName="nova-api-api" Dec 03 12:28:55 crc kubenswrapper[5052]: I1203 12:28:55.965725 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="9632f411-9c90-4a2c-9c79-7a2138f0ff43" containerName="nova-api-api" Dec 03 12:28:55 crc kubenswrapper[5052]: E1203 12:28:55.965737 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="9632f411-9c90-4a2c-9c79-7a2138f0ff43" containerName="nova-api-log" Dec 03 12:28:55 crc kubenswrapper[5052]: I1203 12:28:55.965745 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="9632f411-9c90-4a2c-9c79-7a2138f0ff43" containerName="nova-api-log" Dec 03 12:28:55 crc kubenswrapper[5052]: I1203 12:28:55.965986 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="9632f411-9c90-4a2c-9c79-7a2138f0ff43" containerName="nova-api-api" Dec 03 12:28:55 crc kubenswrapper[5052]: I1203 12:28:55.966016 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="9632f411-9c90-4a2c-9c79-7a2138f0ff43" containerName="nova-api-log" Dec 03 12:28:55 crc kubenswrapper[5052]: I1203 12:28:55.966041 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="8da1e6bc-5fd9-4886-ae5e-01a1dbd307e7" containerName="nova-metadata-metadata" Dec 03 12:28:55 crc kubenswrapper[5052]: I1203 12:28:55.966060 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="8da1e6bc-5fd9-4886-ae5e-01a1dbd307e7" containerName="nova-metadata-log" Dec 03 12:28:55 crc kubenswrapper[5052]: I1203 12:28:55.967327 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:28:55 crc kubenswrapper[5052]: I1203 12:28:55.971863 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-nova-metadata-internal-svc" Dec 03 12:28:55 crc kubenswrapper[5052]: I1203 12:28:55.972177 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-metadata-config-data" Dec 03 12:28:55 crc kubenswrapper[5052]: I1203 12:28:55.976429 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:28:56 crc kubenswrapper[5052]: I1203 12:28:56.017704 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8k2mf\" (UniqueName: \"kubernetes.io/projected/a1219c4c-2ddf-4796-91dc-794412acc782-kube-api-access-8k2mf\") pod \"nova-metadata-0\" (UID: \"a1219c4c-2ddf-4796-91dc-794412acc782\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:28:56 crc kubenswrapper[5052]: I1203 12:28:56.017778 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/a1219c4c-2ddf-4796-91dc-794412acc782-nova-metadata-tls-certs\") pod \"nova-metadata-0\" (UID: \"a1219c4c-2ddf-4796-91dc-794412acc782\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:28:56 crc kubenswrapper[5052]: I1203 12:28:56.017841 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a1219c4c-2ddf-4796-91dc-794412acc782-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"a1219c4c-2ddf-4796-91dc-794412acc782\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:28:56 crc kubenswrapper[5052]: I1203 12:28:56.017873 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a1219c4c-2ddf-4796-91dc-794412acc782-config-data\") pod \"nova-metadata-0\" (UID: \"a1219c4c-2ddf-4796-91dc-794412acc782\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:28:56 crc kubenswrapper[5052]: I1203 12:28:56.018043 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/a1219c4c-2ddf-4796-91dc-794412acc782-logs\") pod \"nova-metadata-0\" (UID: \"a1219c4c-2ddf-4796-91dc-794412acc782\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:28:56 crc kubenswrapper[5052]: I1203 12:28:56.050066 5052 scope.go:117] "RemoveContainer" containerID="a2fdb86f428561f36257e1c0427682d91b3c9d697feb5500615c2839dbc040b8" Dec 03 12:28:56 crc kubenswrapper[5052]: I1203 12:28:56.076686 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:28:56 crc kubenswrapper[5052]: I1203 12:28:56.108822 5052 scope.go:117] "RemoveContainer" containerID="07c66122c20f41a0d54c2503579b4ef239da6f910b95c44525198f30ad0ed3f0" Dec 03 12:28:56 crc kubenswrapper[5052]: E1203 12:28:56.110110 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"07c66122c20f41a0d54c2503579b4ef239da6f910b95c44525198f30ad0ed3f0\": container with ID starting with 07c66122c20f41a0d54c2503579b4ef239da6f910b95c44525198f30ad0ed3f0 not found: ID does not exist" containerID="07c66122c20f41a0d54c2503579b4ef239da6f910b95c44525198f30ad0ed3f0" Dec 03 12:28:56 crc kubenswrapper[5052]: I1203 12:28:56.110161 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"07c66122c20f41a0d54c2503579b4ef239da6f910b95c44525198f30ad0ed3f0"} err="failed to get container status \"07c66122c20f41a0d54c2503579b4ef239da6f910b95c44525198f30ad0ed3f0\": rpc error: code = NotFound desc = could not find container \"07c66122c20f41a0d54c2503579b4ef239da6f910b95c44525198f30ad0ed3f0\": container with ID starting with 07c66122c20f41a0d54c2503579b4ef239da6f910b95c44525198f30ad0ed3f0 not found: ID does not exist" Dec 03 12:28:56 crc kubenswrapper[5052]: I1203 12:28:56.110185 5052 scope.go:117] "RemoveContainer" containerID="a2fdb86f428561f36257e1c0427682d91b3c9d697feb5500615c2839dbc040b8" Dec 03 12:28:56 crc kubenswrapper[5052]: E1203 12:28:56.113161 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a2fdb86f428561f36257e1c0427682d91b3c9d697feb5500615c2839dbc040b8\": container with ID starting with a2fdb86f428561f36257e1c0427682d91b3c9d697feb5500615c2839dbc040b8 not found: ID does not exist" containerID="a2fdb86f428561f36257e1c0427682d91b3c9d697feb5500615c2839dbc040b8" Dec 03 12:28:56 crc kubenswrapper[5052]: I1203 12:28:56.113203 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a2fdb86f428561f36257e1c0427682d91b3c9d697feb5500615c2839dbc040b8"} err="failed to get container status \"a2fdb86f428561f36257e1c0427682d91b3c9d697feb5500615c2839dbc040b8\": rpc error: code = NotFound desc = could not find container \"a2fdb86f428561f36257e1c0427682d91b3c9d697feb5500615c2839dbc040b8\": container with ID starting with a2fdb86f428561f36257e1c0427682d91b3c9d697feb5500615c2839dbc040b8 not found: ID does not exist" Dec 03 12:28:56 crc kubenswrapper[5052]: I1203 12:28:56.113225 5052 scope.go:117] "RemoveContainer" containerID="07c66122c20f41a0d54c2503579b4ef239da6f910b95c44525198f30ad0ed3f0" Dec 03 12:28:56 crc kubenswrapper[5052]: I1203 12:28:56.113467 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"07c66122c20f41a0d54c2503579b4ef239da6f910b95c44525198f30ad0ed3f0"} err="failed to get container status \"07c66122c20f41a0d54c2503579b4ef239da6f910b95c44525198f30ad0ed3f0\": rpc error: code = NotFound desc = could not find container \"07c66122c20f41a0d54c2503579b4ef239da6f910b95c44525198f30ad0ed3f0\": container with ID starting with 07c66122c20f41a0d54c2503579b4ef239da6f910b95c44525198f30ad0ed3f0 not found: ID does not exist" Dec 03 12:28:56 crc kubenswrapper[5052]: I1203 12:28:56.113484 5052 scope.go:117] "RemoveContainer" containerID="a2fdb86f428561f36257e1c0427682d91b3c9d697feb5500615c2839dbc040b8" Dec 03 12:28:56 crc kubenswrapper[5052]: I1203 12:28:56.113691 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a2fdb86f428561f36257e1c0427682d91b3c9d697feb5500615c2839dbc040b8"} err="failed to get container status \"a2fdb86f428561f36257e1c0427682d91b3c9d697feb5500615c2839dbc040b8\": rpc error: code = NotFound desc = could not find container \"a2fdb86f428561f36257e1c0427682d91b3c9d697feb5500615c2839dbc040b8\": container with ID starting with a2fdb86f428561f36257e1c0427682d91b3c9d697feb5500615c2839dbc040b8 not found: ID does not exist" Dec 03 12:28:56 crc kubenswrapper[5052]: I1203 12:28:56.113705 5052 scope.go:117] "RemoveContainer" containerID="d598054ff3cc894755197631e8e45c78cce05130d3836690788ea53b61d7d3a5" Dec 03 12:28:56 crc kubenswrapper[5052]: I1203 12:28:56.123353 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/a1219c4c-2ddf-4796-91dc-794412acc782-logs\") pod \"nova-metadata-0\" (UID: \"a1219c4c-2ddf-4796-91dc-794412acc782\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:28:56 crc kubenswrapper[5052]: I1203 12:28:56.123468 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-8k2mf\" (UniqueName: \"kubernetes.io/projected/a1219c4c-2ddf-4796-91dc-794412acc782-kube-api-access-8k2mf\") pod \"nova-metadata-0\" (UID: \"a1219c4c-2ddf-4796-91dc-794412acc782\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:28:56 crc kubenswrapper[5052]: I1203 12:28:56.123530 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/a1219c4c-2ddf-4796-91dc-794412acc782-nova-metadata-tls-certs\") pod \"nova-metadata-0\" (UID: \"a1219c4c-2ddf-4796-91dc-794412acc782\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:28:56 crc kubenswrapper[5052]: I1203 12:28:56.123602 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a1219c4c-2ddf-4796-91dc-794412acc782-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"a1219c4c-2ddf-4796-91dc-794412acc782\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:28:56 crc kubenswrapper[5052]: I1203 12:28:56.123632 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a1219c4c-2ddf-4796-91dc-794412acc782-config-data\") pod \"nova-metadata-0\" (UID: \"a1219c4c-2ddf-4796-91dc-794412acc782\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:28:56 crc kubenswrapper[5052]: I1203 12:28:56.125173 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/a1219c4c-2ddf-4796-91dc-794412acc782-logs\") pod \"nova-metadata-0\" (UID: \"a1219c4c-2ddf-4796-91dc-794412acc782\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:28:56 crc kubenswrapper[5052]: I1203 12:28:56.138269 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:28:56 crc kubenswrapper[5052]: I1203 12:28:56.147970 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-8k2mf\" (UniqueName: \"kubernetes.io/projected/a1219c4c-2ddf-4796-91dc-794412acc782-kube-api-access-8k2mf\") pod \"nova-metadata-0\" (UID: \"a1219c4c-2ddf-4796-91dc-794412acc782\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:28:56 crc kubenswrapper[5052]: I1203 12:28:56.156592 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/a1219c4c-2ddf-4796-91dc-794412acc782-nova-metadata-tls-certs\") pod \"nova-metadata-0\" (UID: \"a1219c4c-2ddf-4796-91dc-794412acc782\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:28:56 crc kubenswrapper[5052]: I1203 12:28:56.162451 5052 scope.go:117] "RemoveContainer" containerID="46f54610e1f342af288f0ebfbb47c0d93619fe37f5035a7b333c0f28e092259a" Dec 03 12:28:56 crc kubenswrapper[5052]: I1203 12:28:56.162676 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:28:56 crc kubenswrapper[5052]: I1203 12:28:56.166668 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:28:56 crc kubenswrapper[5052]: I1203 12:28:56.168970 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a1219c4c-2ddf-4796-91dc-794412acc782-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"a1219c4c-2ddf-4796-91dc-794412acc782\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:28:56 crc kubenswrapper[5052]: I1203 12:28:56.173267 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-nova-public-svc" Dec 03 12:28:56 crc kubenswrapper[5052]: I1203 12:28:56.173873 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-api-config-data" Dec 03 12:28:56 crc kubenswrapper[5052]: I1203 12:28:56.174634 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-nova-internal-svc" Dec 03 12:28:56 crc kubenswrapper[5052]: I1203 12:28:56.179644 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:28:56 crc kubenswrapper[5052]: I1203 12:28:56.196444 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a1219c4c-2ddf-4796-91dc-794412acc782-config-data\") pod \"nova-metadata-0\" (UID: \"a1219c4c-2ddf-4796-91dc-794412acc782\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:28:56 crc kubenswrapper[5052]: I1203 12:28:56.322749 5052 scope.go:117] "RemoveContainer" containerID="d598054ff3cc894755197631e8e45c78cce05130d3836690788ea53b61d7d3a5" Dec 03 12:28:56 crc kubenswrapper[5052]: E1203 12:28:56.323302 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d598054ff3cc894755197631e8e45c78cce05130d3836690788ea53b61d7d3a5\": container with ID starting with d598054ff3cc894755197631e8e45c78cce05130d3836690788ea53b61d7d3a5 not found: ID does not exist" containerID="d598054ff3cc894755197631e8e45c78cce05130d3836690788ea53b61d7d3a5" Dec 03 12:28:56 crc kubenswrapper[5052]: I1203 12:28:56.323346 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d598054ff3cc894755197631e8e45c78cce05130d3836690788ea53b61d7d3a5"} err="failed to get container status \"d598054ff3cc894755197631e8e45c78cce05130d3836690788ea53b61d7d3a5\": rpc error: code = NotFound desc = could not find container \"d598054ff3cc894755197631e8e45c78cce05130d3836690788ea53b61d7d3a5\": container with ID starting with d598054ff3cc894755197631e8e45c78cce05130d3836690788ea53b61d7d3a5 not found: ID does not exist" Dec 03 12:28:56 crc kubenswrapper[5052]: I1203 12:28:56.323372 5052 scope.go:117] "RemoveContainer" containerID="46f54610e1f342af288f0ebfbb47c0d93619fe37f5035a7b333c0f28e092259a" Dec 03 12:28:56 crc kubenswrapper[5052]: E1203 12:28:56.323798 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"46f54610e1f342af288f0ebfbb47c0d93619fe37f5035a7b333c0f28e092259a\": container with ID starting with 46f54610e1f342af288f0ebfbb47c0d93619fe37f5035a7b333c0f28e092259a not found: ID does not exist" containerID="46f54610e1f342af288f0ebfbb47c0d93619fe37f5035a7b333c0f28e092259a" Dec 03 12:28:56 crc kubenswrapper[5052]: I1203 12:28:56.323850 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"46f54610e1f342af288f0ebfbb47c0d93619fe37f5035a7b333c0f28e092259a"} err="failed to get container status \"46f54610e1f342af288f0ebfbb47c0d93619fe37f5035a7b333c0f28e092259a\": rpc error: code = NotFound desc = could not find container \"46f54610e1f342af288f0ebfbb47c0d93619fe37f5035a7b333c0f28e092259a\": container with ID starting with 46f54610e1f342af288f0ebfbb47c0d93619fe37f5035a7b333c0f28e092259a not found: ID does not exist" Dec 03 12:28:56 crc kubenswrapper[5052]: I1203 12:28:56.323884 5052 scope.go:117] "RemoveContainer" containerID="d598054ff3cc894755197631e8e45c78cce05130d3836690788ea53b61d7d3a5" Dec 03 12:28:56 crc kubenswrapper[5052]: I1203 12:28:56.324248 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d598054ff3cc894755197631e8e45c78cce05130d3836690788ea53b61d7d3a5"} err="failed to get container status \"d598054ff3cc894755197631e8e45c78cce05130d3836690788ea53b61d7d3a5\": rpc error: code = NotFound desc = could not find container \"d598054ff3cc894755197631e8e45c78cce05130d3836690788ea53b61d7d3a5\": container with ID starting with d598054ff3cc894755197631e8e45c78cce05130d3836690788ea53b61d7d3a5 not found: ID does not exist" Dec 03 12:28:56 crc kubenswrapper[5052]: I1203 12:28:56.324291 5052 scope.go:117] "RemoveContainer" containerID="46f54610e1f342af288f0ebfbb47c0d93619fe37f5035a7b333c0f28e092259a" Dec 03 12:28:56 crc kubenswrapper[5052]: I1203 12:28:56.324610 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"46f54610e1f342af288f0ebfbb47c0d93619fe37f5035a7b333c0f28e092259a"} err="failed to get container status \"46f54610e1f342af288f0ebfbb47c0d93619fe37f5035a7b333c0f28e092259a\": rpc error: code = NotFound desc = could not find container \"46f54610e1f342af288f0ebfbb47c0d93619fe37f5035a7b333c0f28e092259a\": container with ID starting with 46f54610e1f342af288f0ebfbb47c0d93619fe37f5035a7b333c0f28e092259a not found: ID does not exist" Dec 03 12:28:56 crc kubenswrapper[5052]: I1203 12:28:56.331069 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/12f98520-40ab-4d0a-b48b-84cdd9005194-internal-tls-certs\") pod \"nova-api-0\" (UID: \"12f98520-40ab-4d0a-b48b-84cdd9005194\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:28:56 crc kubenswrapper[5052]: I1203 12:28:56.331203 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/12f98520-40ab-4d0a-b48b-84cdd9005194-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"12f98520-40ab-4d0a-b48b-84cdd9005194\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:28:56 crc kubenswrapper[5052]: I1203 12:28:56.331257 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/12f98520-40ab-4d0a-b48b-84cdd9005194-logs\") pod \"nova-api-0\" (UID: \"12f98520-40ab-4d0a-b48b-84cdd9005194\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:28:56 crc kubenswrapper[5052]: I1203 12:28:56.331304 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/12f98520-40ab-4d0a-b48b-84cdd9005194-public-tls-certs\") pod \"nova-api-0\" (UID: \"12f98520-40ab-4d0a-b48b-84cdd9005194\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:28:56 crc kubenswrapper[5052]: I1203 12:28:56.331370 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/12f98520-40ab-4d0a-b48b-84cdd9005194-config-data\") pod \"nova-api-0\" (UID: \"12f98520-40ab-4d0a-b48b-84cdd9005194\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:28:56 crc kubenswrapper[5052]: I1203 12:28:56.331450 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bctqz\" (UniqueName: \"kubernetes.io/projected/12f98520-40ab-4d0a-b48b-84cdd9005194-kube-api-access-bctqz\") pod \"nova-api-0\" (UID: \"12f98520-40ab-4d0a-b48b-84cdd9005194\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:28:56 crc kubenswrapper[5052]: I1203 12:28:56.410769 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:28:56 crc kubenswrapper[5052]: I1203 12:28:56.434300 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/12f98520-40ab-4d0a-b48b-84cdd9005194-config-data\") pod \"nova-api-0\" (UID: \"12f98520-40ab-4d0a-b48b-84cdd9005194\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:28:56 crc kubenswrapper[5052]: I1203 12:28:56.434408 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-bctqz\" (UniqueName: \"kubernetes.io/projected/12f98520-40ab-4d0a-b48b-84cdd9005194-kube-api-access-bctqz\") pod \"nova-api-0\" (UID: \"12f98520-40ab-4d0a-b48b-84cdd9005194\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:28:56 crc kubenswrapper[5052]: I1203 12:28:56.434500 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/12f98520-40ab-4d0a-b48b-84cdd9005194-internal-tls-certs\") pod \"nova-api-0\" (UID: \"12f98520-40ab-4d0a-b48b-84cdd9005194\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:28:56 crc kubenswrapper[5052]: I1203 12:28:56.434667 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/12f98520-40ab-4d0a-b48b-84cdd9005194-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"12f98520-40ab-4d0a-b48b-84cdd9005194\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:28:56 crc kubenswrapper[5052]: I1203 12:28:56.434707 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/12f98520-40ab-4d0a-b48b-84cdd9005194-logs\") pod \"nova-api-0\" (UID: \"12f98520-40ab-4d0a-b48b-84cdd9005194\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:28:56 crc kubenswrapper[5052]: I1203 12:28:56.434745 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/12f98520-40ab-4d0a-b48b-84cdd9005194-public-tls-certs\") pod \"nova-api-0\" (UID: \"12f98520-40ab-4d0a-b48b-84cdd9005194\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:28:56 crc kubenswrapper[5052]: I1203 12:28:56.436371 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/12f98520-40ab-4d0a-b48b-84cdd9005194-logs\") pod \"nova-api-0\" (UID: \"12f98520-40ab-4d0a-b48b-84cdd9005194\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:28:56 crc kubenswrapper[5052]: I1203 12:28:56.448228 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/12f98520-40ab-4d0a-b48b-84cdd9005194-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"12f98520-40ab-4d0a-b48b-84cdd9005194\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:28:56 crc kubenswrapper[5052]: I1203 12:28:56.448889 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/12f98520-40ab-4d0a-b48b-84cdd9005194-internal-tls-certs\") pod \"nova-api-0\" (UID: \"12f98520-40ab-4d0a-b48b-84cdd9005194\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:28:56 crc kubenswrapper[5052]: I1203 12:28:56.449499 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/12f98520-40ab-4d0a-b48b-84cdd9005194-public-tls-certs\") pod \"nova-api-0\" (UID: \"12f98520-40ab-4d0a-b48b-84cdd9005194\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:28:56 crc kubenswrapper[5052]: I1203 12:28:56.459264 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/12f98520-40ab-4d0a-b48b-84cdd9005194-config-data\") pod \"nova-api-0\" (UID: \"12f98520-40ab-4d0a-b48b-84cdd9005194\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:28:56 crc kubenswrapper[5052]: I1203 12:28:56.475095 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-bctqz\" (UniqueName: \"kubernetes.io/projected/12f98520-40ab-4d0a-b48b-84cdd9005194-kube-api-access-bctqz\") pod \"nova-api-0\" (UID: \"12f98520-40ab-4d0a-b48b-84cdd9005194\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:28:56 crc kubenswrapper[5052]: I1203 12:28:56.623534 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:28:56 crc kubenswrapper[5052]: I1203 12:28:56.785028 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:28:56 crc kubenswrapper[5052]: I1203 12:28:56.871168 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:28:56 crc kubenswrapper[5052]: I1203 12:28:56.926132 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-proxy-84c4f5d658-m9l78" Dec 03 12:28:56 crc kubenswrapper[5052]: I1203 12:28:56.948651 5052 generic.go:334] "Generic (PLEG): container finished" podID="9f515435-a645-4d2b-9bd5-4bb9496f5405" containerID="41ad3ff3a7101ea8b3778d199fc0aa3d823057eb0199c3f55a79043a27526808" exitCode=0 Dec 03 12:28:56 crc kubenswrapper[5052]: I1203 12:28:56.948770 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-proxy-84c4f5d658-m9l78" Dec 03 12:28:56 crc kubenswrapper[5052]: I1203 12:28:56.948828 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-proxy-84c4f5d658-m9l78" event={"ID":"9f515435-a645-4d2b-9bd5-4bb9496f5405","Type":"ContainerDied","Data":"41ad3ff3a7101ea8b3778d199fc0aa3d823057eb0199c3f55a79043a27526808"} Dec 03 12:28:56 crc kubenswrapper[5052]: I1203 12:28:56.948892 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-proxy-84c4f5d658-m9l78" event={"ID":"9f515435-a645-4d2b-9bd5-4bb9496f5405","Type":"ContainerDied","Data":"09c43885facd81df8e08a1ad42cb243e3a23953e3a9571a5e2670caeea4cbf8f"} Dec 03 12:28:56 crc kubenswrapper[5052]: I1203 12:28:56.948921 5052 scope.go:117] "RemoveContainer" containerID="41ad3ff3a7101ea8b3778d199fc0aa3d823057eb0199c3f55a79043a27526808" Dec 03 12:28:56 crc kubenswrapper[5052]: I1203 12:28:56.974622 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"a65d6de5-f108-40c9-81f6-ca37534c7710","Type":"ContainerStarted","Data":"b39e98c50755a3d0626f639cee9dde7e86c700c52f6ddaade4e3e9fe9b523800"} Dec 03 12:28:57 crc kubenswrapper[5052]: I1203 12:28:57.010331 5052 scope.go:117] "RemoveContainer" containerID="7401b98806090e0288c0ece96883b73c2a54d14bd85bbf4ba5729b51cadf8ea2" Dec 03 12:28:57 crc kubenswrapper[5052]: I1203 12:28:57.052620 5052 scope.go:117] "RemoveContainer" containerID="41ad3ff3a7101ea8b3778d199fc0aa3d823057eb0199c3f55a79043a27526808" Dec 03 12:28:57 crc kubenswrapper[5052]: E1203 12:28:57.053430 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"41ad3ff3a7101ea8b3778d199fc0aa3d823057eb0199c3f55a79043a27526808\": container with ID starting with 41ad3ff3a7101ea8b3778d199fc0aa3d823057eb0199c3f55a79043a27526808 not found: ID does not exist" containerID="41ad3ff3a7101ea8b3778d199fc0aa3d823057eb0199c3f55a79043a27526808" Dec 03 12:28:57 crc kubenswrapper[5052]: I1203 12:28:57.053479 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"41ad3ff3a7101ea8b3778d199fc0aa3d823057eb0199c3f55a79043a27526808"} err="failed to get container status \"41ad3ff3a7101ea8b3778d199fc0aa3d823057eb0199c3f55a79043a27526808\": rpc error: code = NotFound desc = could not find container \"41ad3ff3a7101ea8b3778d199fc0aa3d823057eb0199c3f55a79043a27526808\": container with ID starting with 41ad3ff3a7101ea8b3778d199fc0aa3d823057eb0199c3f55a79043a27526808 not found: ID does not exist" Dec 03 12:28:57 crc kubenswrapper[5052]: I1203 12:28:57.053524 5052 scope.go:117] "RemoveContainer" containerID="7401b98806090e0288c0ece96883b73c2a54d14bd85bbf4ba5729b51cadf8ea2" Dec 03 12:28:57 crc kubenswrapper[5052]: E1203 12:28:57.054118 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7401b98806090e0288c0ece96883b73c2a54d14bd85bbf4ba5729b51cadf8ea2\": container with ID starting with 7401b98806090e0288c0ece96883b73c2a54d14bd85bbf4ba5729b51cadf8ea2 not found: ID does not exist" containerID="7401b98806090e0288c0ece96883b73c2a54d14bd85bbf4ba5729b51cadf8ea2" Dec 03 12:28:57 crc kubenswrapper[5052]: I1203 12:28:57.054186 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7401b98806090e0288c0ece96883b73c2a54d14bd85bbf4ba5729b51cadf8ea2"} err="failed to get container status \"7401b98806090e0288c0ece96883b73c2a54d14bd85bbf4ba5729b51cadf8ea2\": rpc error: code = NotFound desc = could not find container \"7401b98806090e0288c0ece96883b73c2a54d14bd85bbf4ba5729b51cadf8ea2\": container with ID starting with 7401b98806090e0288c0ece96883b73c2a54d14bd85bbf4ba5729b51cadf8ea2 not found: ID does not exist" Dec 03 12:28:57 crc kubenswrapper[5052]: I1203 12:28:57.061996 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/9f515435-a645-4d2b-9bd5-4bb9496f5405-run-httpd\") pod \"9f515435-a645-4d2b-9bd5-4bb9496f5405\" (UID: \"9f515435-a645-4d2b-9bd5-4bb9496f5405\") " Dec 03 12:28:57 crc kubenswrapper[5052]: I1203 12:28:57.062086 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/9f515435-a645-4d2b-9bd5-4bb9496f5405-config-data\") pod \"9f515435-a645-4d2b-9bd5-4bb9496f5405\" (UID: \"9f515435-a645-4d2b-9bd5-4bb9496f5405\") " Dec 03 12:28:57 crc kubenswrapper[5052]: I1203 12:28:57.062184 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mp59r\" (UniqueName: \"kubernetes.io/projected/9f515435-a645-4d2b-9bd5-4bb9496f5405-kube-api-access-mp59r\") pod \"9f515435-a645-4d2b-9bd5-4bb9496f5405\" (UID: \"9f515435-a645-4d2b-9bd5-4bb9496f5405\") " Dec 03 12:28:57 crc kubenswrapper[5052]: I1203 12:28:57.062242 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9f515435-a645-4d2b-9bd5-4bb9496f5405-combined-ca-bundle\") pod \"9f515435-a645-4d2b-9bd5-4bb9496f5405\" (UID: \"9f515435-a645-4d2b-9bd5-4bb9496f5405\") " Dec 03 12:28:57 crc kubenswrapper[5052]: I1203 12:28:57.062385 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/9f515435-a645-4d2b-9bd5-4bb9496f5405-log-httpd\") pod \"9f515435-a645-4d2b-9bd5-4bb9496f5405\" (UID: \"9f515435-a645-4d2b-9bd5-4bb9496f5405\") " Dec 03 12:28:57 crc kubenswrapper[5052]: I1203 12:28:57.062428 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/9f515435-a645-4d2b-9bd5-4bb9496f5405-etc-swift\") pod \"9f515435-a645-4d2b-9bd5-4bb9496f5405\" (UID: \"9f515435-a645-4d2b-9bd5-4bb9496f5405\") " Dec 03 12:28:57 crc kubenswrapper[5052]: I1203 12:28:57.065845 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f515435-a645-4d2b-9bd5-4bb9496f5405-run-httpd" (OuterVolumeSpecName: "run-httpd") pod "9f515435-a645-4d2b-9bd5-4bb9496f5405" (UID: "9f515435-a645-4d2b-9bd5-4bb9496f5405"). InnerVolumeSpecName "run-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:28:57 crc kubenswrapper[5052]: I1203 12:28:57.066222 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f515435-a645-4d2b-9bd5-4bb9496f5405-log-httpd" (OuterVolumeSpecName: "log-httpd") pod "9f515435-a645-4d2b-9bd5-4bb9496f5405" (UID: "9f515435-a645-4d2b-9bd5-4bb9496f5405"). InnerVolumeSpecName "log-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:28:57 crc kubenswrapper[5052]: I1203 12:28:57.092104 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9f515435-a645-4d2b-9bd5-4bb9496f5405-kube-api-access-mp59r" (OuterVolumeSpecName: "kube-api-access-mp59r") pod "9f515435-a645-4d2b-9bd5-4bb9496f5405" (UID: "9f515435-a645-4d2b-9bd5-4bb9496f5405"). InnerVolumeSpecName "kube-api-access-mp59r". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:28:57 crc kubenswrapper[5052]: I1203 12:28:57.092606 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9f515435-a645-4d2b-9bd5-4bb9496f5405-etc-swift" (OuterVolumeSpecName: "etc-swift") pod "9f515435-a645-4d2b-9bd5-4bb9496f5405" (UID: "9f515435-a645-4d2b-9bd5-4bb9496f5405"). InnerVolumeSpecName "etc-swift". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:28:57 crc kubenswrapper[5052]: I1203 12:28:57.148529 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:28:57 crc kubenswrapper[5052]: W1203 12:28:57.155497 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda1219c4c_2ddf_4796_91dc_794412acc782.slice/crio-a7e0a610c5ef7645b9c47f5262deb1f12b88d93827f984b0bdb49fec1b8ef83b WatchSource:0}: Error finding container a7e0a610c5ef7645b9c47f5262deb1f12b88d93827f984b0bdb49fec1b8ef83b: Status 404 returned error can't find the container with id a7e0a610c5ef7645b9c47f5262deb1f12b88d93827f984b0bdb49fec1b8ef83b Dec 03 12:28:57 crc kubenswrapper[5052]: E1203 12:28:57.160612 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9f515435-a645-4d2b-9bd5-4bb9496f5405-config-data podName:9f515435-a645-4d2b-9bd5-4bb9496f5405 nodeName:}" failed. No retries permitted until 2025-12-03 12:28:57.660580906 +0000 UTC m=+4128.248822750 (durationBeforeRetry 500ms). Error: error cleaning subPath mounts for volume "config-data" (UniqueName: "kubernetes.io/secret/9f515435-a645-4d2b-9bd5-4bb9496f5405-config-data") pod "9f515435-a645-4d2b-9bd5-4bb9496f5405" (UID: "9f515435-a645-4d2b-9bd5-4bb9496f5405") : error deleting /var/lib/kubelet/pods/9f515435-a645-4d2b-9bd5-4bb9496f5405/volume-subpaths: remove /var/lib/kubelet/pods/9f515435-a645-4d2b-9bd5-4bb9496f5405/volume-subpaths: no such file or directory Dec 03 12:28:57 crc kubenswrapper[5052]: I1203 12:28:57.163265 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9f515435-a645-4d2b-9bd5-4bb9496f5405-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "9f515435-a645-4d2b-9bd5-4bb9496f5405" (UID: "9f515435-a645-4d2b-9bd5-4bb9496f5405"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:28:57 crc kubenswrapper[5052]: I1203 12:28:57.164486 5052 reconciler_common.go:293] "Volume detached for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/9f515435-a645-4d2b-9bd5-4bb9496f5405-log-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:57 crc kubenswrapper[5052]: I1203 12:28:57.164528 5052 reconciler_common.go:293] "Volume detached for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/9f515435-a645-4d2b-9bd5-4bb9496f5405-etc-swift\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:57 crc kubenswrapper[5052]: I1203 12:28:57.164539 5052 reconciler_common.go:293] "Volume detached for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/9f515435-a645-4d2b-9bd5-4bb9496f5405-run-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:57 crc kubenswrapper[5052]: I1203 12:28:57.164549 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-mp59r\" (UniqueName: \"kubernetes.io/projected/9f515435-a645-4d2b-9bd5-4bb9496f5405-kube-api-access-mp59r\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:57 crc kubenswrapper[5052]: I1203 12:28:57.164561 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9f515435-a645-4d2b-9bd5-4bb9496f5405-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:57 crc kubenswrapper[5052]: I1203 12:28:57.272339 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:28:57 crc kubenswrapper[5052]: W1203 12:28:57.279020 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod12f98520_40ab_4d0a_b48b_84cdd9005194.slice/crio-03fbd692cc206304d6ce92ed6c700a712ed048266c9469a27901df1f34c8baba WatchSource:0}: Error finding container 03fbd692cc206304d6ce92ed6c700a712ed048266c9469a27901df1f34c8baba: Status 404 returned error can't find the container with id 03fbd692cc206304d6ce92ed6c700a712ed048266c9469a27901df1f34c8baba Dec 03 12:28:57 crc kubenswrapper[5052]: I1203 12:28:57.674045 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/9f515435-a645-4d2b-9bd5-4bb9496f5405-config-data\") pod \"9f515435-a645-4d2b-9bd5-4bb9496f5405\" (UID: \"9f515435-a645-4d2b-9bd5-4bb9496f5405\") " Dec 03 12:28:57 crc kubenswrapper[5052]: I1203 12:28:57.678480 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9f515435-a645-4d2b-9bd5-4bb9496f5405-config-data" (OuterVolumeSpecName: "config-data") pod "9f515435-a645-4d2b-9bd5-4bb9496f5405" (UID: "9f515435-a645-4d2b-9bd5-4bb9496f5405"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:28:57 crc kubenswrapper[5052]: I1203 12:28:57.683596 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8da1e6bc-5fd9-4886-ae5e-01a1dbd307e7" path="/var/lib/kubelet/pods/8da1e6bc-5fd9-4886-ae5e-01a1dbd307e7/volumes" Dec 03 12:28:57 crc kubenswrapper[5052]: I1203 12:28:57.684337 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9632f411-9c90-4a2c-9c79-7a2138f0ff43" path="/var/lib/kubelet/pods/9632f411-9c90-4a2c-9c79-7a2138f0ff43/volumes" Dec 03 12:28:57 crc kubenswrapper[5052]: I1203 12:28:57.777774 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/9f515435-a645-4d2b-9bd5-4bb9496f5405-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:57 crc kubenswrapper[5052]: I1203 12:28:57.876010 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/swift-proxy-84c4f5d658-m9l78"] Dec 03 12:28:57 crc kubenswrapper[5052]: I1203 12:28:57.891204 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/swift-proxy-84c4f5d658-m9l78"] Dec 03 12:28:58 crc kubenswrapper[5052]: I1203 12:28:58.004315 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"12f98520-40ab-4d0a-b48b-84cdd9005194","Type":"ContainerStarted","Data":"9d81b16f8190231f8e209ce2911c03014e429b33c5316cafe9152f4b974fec6b"} Dec 03 12:28:58 crc kubenswrapper[5052]: I1203 12:28:58.004581 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"12f98520-40ab-4d0a-b48b-84cdd9005194","Type":"ContainerStarted","Data":"68ac7a9d06036534e0af4f800761162dd7d95c40656710410c5ecdf61fa05368"} Dec 03 12:28:58 crc kubenswrapper[5052]: I1203 12:28:58.004595 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"12f98520-40ab-4d0a-b48b-84cdd9005194","Type":"ContainerStarted","Data":"03fbd692cc206304d6ce92ed6c700a712ed048266c9469a27901df1f34c8baba"} Dec 03 12:28:58 crc kubenswrapper[5052]: I1203 12:28:58.004454 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-api-0" podUID="12f98520-40ab-4d0a-b48b-84cdd9005194" containerName="nova-api-api" containerID="cri-o://9d81b16f8190231f8e209ce2911c03014e429b33c5316cafe9152f4b974fec6b" gracePeriod=30 Dec 03 12:28:58 crc kubenswrapper[5052]: I1203 12:28:58.004397 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-api-0" podUID="12f98520-40ab-4d0a-b48b-84cdd9005194" containerName="nova-api-log" containerID="cri-o://68ac7a9d06036534e0af4f800761162dd7d95c40656710410c5ecdf61fa05368" gracePeriod=30 Dec 03 12:28:58 crc kubenswrapper[5052]: I1203 12:28:58.008846 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"a1219c4c-2ddf-4796-91dc-794412acc782","Type":"ContainerStarted","Data":"bca6a9ef98c07e4231ae7fe76378e651c687dd9be2f44296fae8a750007f8c21"} Dec 03 12:28:58 crc kubenswrapper[5052]: I1203 12:28:58.009013 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"a1219c4c-2ddf-4796-91dc-794412acc782","Type":"ContainerStarted","Data":"dade77d2933a2006677ea6d4ae01efc11be40a32911ed7be983068b4d33189c2"} Dec 03 12:28:58 crc kubenswrapper[5052]: I1203 12:28:58.009110 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"a1219c4c-2ddf-4796-91dc-794412acc782","Type":"ContainerStarted","Data":"a7e0a610c5ef7645b9c47f5262deb1f12b88d93827f984b0bdb49fec1b8ef83b"} Dec 03 12:28:58 crc kubenswrapper[5052]: I1203 12:28:58.009075 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-metadata-0" podUID="a1219c4c-2ddf-4796-91dc-794412acc782" containerName="nova-metadata-metadata" containerID="cri-o://bca6a9ef98c07e4231ae7fe76378e651c687dd9be2f44296fae8a750007f8c21" gracePeriod=30 Dec 03 12:28:58 crc kubenswrapper[5052]: I1203 12:28:58.009012 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-metadata-0" podUID="a1219c4c-2ddf-4796-91dc-794412acc782" containerName="nova-metadata-log" containerID="cri-o://dade77d2933a2006677ea6d4ae01efc11be40a32911ed7be983068b4d33189c2" gracePeriod=30 Dec 03 12:28:58 crc kubenswrapper[5052]: I1203 12:28:58.045392 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-api-0" podStartSLOduration=2.04537021 podStartE2EDuration="2.04537021s" podCreationTimestamp="2025-12-03 12:28:56 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:28:58.042701484 +0000 UTC m=+4128.630943248" watchObservedRunningTime="2025-12-03 12:28:58.04537021 +0000 UTC m=+4128.633611974" Dec 03 12:28:58 crc kubenswrapper[5052]: I1203 12:28:58.078940 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-metadata-0" podStartSLOduration=3.078885857 podStartE2EDuration="3.078885857s" podCreationTimestamp="2025-12-03 12:28:55 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:28:58.073236025 +0000 UTC m=+4128.661477789" watchObservedRunningTime="2025-12-03 12:28:58.078885857 +0000 UTC m=+4128.667127621" Dec 03 12:28:58 crc kubenswrapper[5052]: E1203 12:28:58.826854 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="f0d03487bb9b0001a9b06ddae0262ca9d26eb73013d9bccc682cbc0b7a9e1bbe" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:28:58 crc kubenswrapper[5052]: E1203 12:28:58.829849 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="f0d03487bb9b0001a9b06ddae0262ca9d26eb73013d9bccc682cbc0b7a9e1bbe" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:28:58 crc kubenswrapper[5052]: E1203 12:28:58.831815 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="f0d03487bb9b0001a9b06ddae0262ca9d26eb73013d9bccc682cbc0b7a9e1bbe" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:28:58 crc kubenswrapper[5052]: E1203 12:28:58.831854 5052 prober.go:104] "Probe errored" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" probeType="Readiness" pod="openstack-kuttl-tests/nova-cell1-conductor-0" podUID="026d5ba1-5f6f-4f39-bcb6-9c51d37608f2" containerName="nova-cell1-conductor-conductor" Dec 03 12:28:58 crc kubenswrapper[5052]: I1203 12:28:58.984480 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:28:59 crc kubenswrapper[5052]: I1203 12:28:59.027118 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/12f98520-40ab-4d0a-b48b-84cdd9005194-config-data\") pod \"12f98520-40ab-4d0a-b48b-84cdd9005194\" (UID: \"12f98520-40ab-4d0a-b48b-84cdd9005194\") " Dec 03 12:28:59 crc kubenswrapper[5052]: I1203 12:28:59.027238 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/12f98520-40ab-4d0a-b48b-84cdd9005194-internal-tls-certs\") pod \"12f98520-40ab-4d0a-b48b-84cdd9005194\" (UID: \"12f98520-40ab-4d0a-b48b-84cdd9005194\") " Dec 03 12:28:59 crc kubenswrapper[5052]: I1203 12:28:59.027323 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/12f98520-40ab-4d0a-b48b-84cdd9005194-public-tls-certs\") pod \"12f98520-40ab-4d0a-b48b-84cdd9005194\" (UID: \"12f98520-40ab-4d0a-b48b-84cdd9005194\") " Dec 03 12:28:59 crc kubenswrapper[5052]: I1203 12:28:59.027400 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/12f98520-40ab-4d0a-b48b-84cdd9005194-combined-ca-bundle\") pod \"12f98520-40ab-4d0a-b48b-84cdd9005194\" (UID: \"12f98520-40ab-4d0a-b48b-84cdd9005194\") " Dec 03 12:28:59 crc kubenswrapper[5052]: I1203 12:28:59.027493 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bctqz\" (UniqueName: \"kubernetes.io/projected/12f98520-40ab-4d0a-b48b-84cdd9005194-kube-api-access-bctqz\") pod \"12f98520-40ab-4d0a-b48b-84cdd9005194\" (UID: \"12f98520-40ab-4d0a-b48b-84cdd9005194\") " Dec 03 12:28:59 crc kubenswrapper[5052]: I1203 12:28:59.027534 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/12f98520-40ab-4d0a-b48b-84cdd9005194-logs\") pod \"12f98520-40ab-4d0a-b48b-84cdd9005194\" (UID: \"12f98520-40ab-4d0a-b48b-84cdd9005194\") " Dec 03 12:28:59 crc kubenswrapper[5052]: I1203 12:28:59.028346 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/12f98520-40ab-4d0a-b48b-84cdd9005194-logs" (OuterVolumeSpecName: "logs") pod "12f98520-40ab-4d0a-b48b-84cdd9005194" (UID: "12f98520-40ab-4d0a-b48b-84cdd9005194"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:28:59 crc kubenswrapper[5052]: I1203 12:28:59.031774 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="a65d6de5-f108-40c9-81f6-ca37534c7710" containerName="ceilometer-central-agent" containerID="cri-o://e0121d77497086b42ab3b08972dbfa1ef63c206df7997ec8f0f64203225a2322" gracePeriod=30 Dec 03 12:28:59 crc kubenswrapper[5052]: I1203 12:28:59.031846 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"a65d6de5-f108-40c9-81f6-ca37534c7710","Type":"ContainerStarted","Data":"c18a95292e924f3db5577fc56cbeeb0a4fd9bbc3a34919d2cc0208c6fb0b74b9"} Dec 03 12:28:59 crc kubenswrapper[5052]: I1203 12:28:59.031885 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:28:59 crc kubenswrapper[5052]: I1203 12:28:59.032189 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="a65d6de5-f108-40c9-81f6-ca37534c7710" containerName="proxy-httpd" containerID="cri-o://c18a95292e924f3db5577fc56cbeeb0a4fd9bbc3a34919d2cc0208c6fb0b74b9" gracePeriod=30 Dec 03 12:28:59 crc kubenswrapper[5052]: I1203 12:28:59.032243 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="a65d6de5-f108-40c9-81f6-ca37534c7710" containerName="sg-core" containerID="cri-o://b39e98c50755a3d0626f639cee9dde7e86c700c52f6ddaade4e3e9fe9b523800" gracePeriod=30 Dec 03 12:28:59 crc kubenswrapper[5052]: I1203 12:28:59.033158 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="a65d6de5-f108-40c9-81f6-ca37534c7710" containerName="ceilometer-notification-agent" containerID="cri-o://55f42c22fa007045b7bcac5be13d03999c40148b147e8c2a61d685132dd50766" gracePeriod=30 Dec 03 12:28:59 crc kubenswrapper[5052]: I1203 12:28:59.033726 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/12f98520-40ab-4d0a-b48b-84cdd9005194-kube-api-access-bctqz" (OuterVolumeSpecName: "kube-api-access-bctqz") pod "12f98520-40ab-4d0a-b48b-84cdd9005194" (UID: "12f98520-40ab-4d0a-b48b-84cdd9005194"). InnerVolumeSpecName "kube-api-access-bctqz". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:28:59 crc kubenswrapper[5052]: I1203 12:28:59.050744 5052 generic.go:334] "Generic (PLEG): container finished" podID="12f98520-40ab-4d0a-b48b-84cdd9005194" containerID="9d81b16f8190231f8e209ce2911c03014e429b33c5316cafe9152f4b974fec6b" exitCode=0 Dec 03 12:28:59 crc kubenswrapper[5052]: I1203 12:28:59.050773 5052 generic.go:334] "Generic (PLEG): container finished" podID="12f98520-40ab-4d0a-b48b-84cdd9005194" containerID="68ac7a9d06036534e0af4f800761162dd7d95c40656710410c5ecdf61fa05368" exitCode=143 Dec 03 12:28:59 crc kubenswrapper[5052]: I1203 12:28:59.050837 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"12f98520-40ab-4d0a-b48b-84cdd9005194","Type":"ContainerDied","Data":"9d81b16f8190231f8e209ce2911c03014e429b33c5316cafe9152f4b974fec6b"} Dec 03 12:28:59 crc kubenswrapper[5052]: I1203 12:28:59.050864 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"12f98520-40ab-4d0a-b48b-84cdd9005194","Type":"ContainerDied","Data":"68ac7a9d06036534e0af4f800761162dd7d95c40656710410c5ecdf61fa05368"} Dec 03 12:28:59 crc kubenswrapper[5052]: I1203 12:28:59.050873 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"12f98520-40ab-4d0a-b48b-84cdd9005194","Type":"ContainerDied","Data":"03fbd692cc206304d6ce92ed6c700a712ed048266c9469a27901df1f34c8baba"} Dec 03 12:28:59 crc kubenswrapper[5052]: I1203 12:28:59.050888 5052 scope.go:117] "RemoveContainer" containerID="9d81b16f8190231f8e209ce2911c03014e429b33c5316cafe9152f4b974fec6b" Dec 03 12:28:59 crc kubenswrapper[5052]: I1203 12:28:59.051003 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:28:59 crc kubenswrapper[5052]: I1203 12:28:59.069984 5052 generic.go:334] "Generic (PLEG): container finished" podID="a1219c4c-2ddf-4796-91dc-794412acc782" containerID="bca6a9ef98c07e4231ae7fe76378e651c687dd9be2f44296fae8a750007f8c21" exitCode=0 Dec 03 12:28:59 crc kubenswrapper[5052]: I1203 12:28:59.070022 5052 generic.go:334] "Generic (PLEG): container finished" podID="a1219c4c-2ddf-4796-91dc-794412acc782" containerID="dade77d2933a2006677ea6d4ae01efc11be40a32911ed7be983068b4d33189c2" exitCode=143 Dec 03 12:28:59 crc kubenswrapper[5052]: I1203 12:28:59.070051 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"a1219c4c-2ddf-4796-91dc-794412acc782","Type":"ContainerDied","Data":"bca6a9ef98c07e4231ae7fe76378e651c687dd9be2f44296fae8a750007f8c21"} Dec 03 12:28:59 crc kubenswrapper[5052]: I1203 12:28:59.070084 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"a1219c4c-2ddf-4796-91dc-794412acc782","Type":"ContainerDied","Data":"dade77d2933a2006677ea6d4ae01efc11be40a32911ed7be983068b4d33189c2"} Dec 03 12:28:59 crc kubenswrapper[5052]: I1203 12:28:59.074115 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/12f98520-40ab-4d0a-b48b-84cdd9005194-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "12f98520-40ab-4d0a-b48b-84cdd9005194" (UID: "12f98520-40ab-4d0a-b48b-84cdd9005194"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:28:59 crc kubenswrapper[5052]: I1203 12:28:59.075679 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/ceilometer-0" podStartSLOduration=2.826632371 podStartE2EDuration="7.075659967s" podCreationTimestamp="2025-12-03 12:28:52 +0000 UTC" firstStartedPulling="2025-12-03 12:28:53.524369621 +0000 UTC m=+4124.112611375" lastFinishedPulling="2025-12-03 12:28:57.773397207 +0000 UTC m=+4128.361638971" observedRunningTime="2025-12-03 12:28:59.061842642 +0000 UTC m=+4129.650084396" watchObservedRunningTime="2025-12-03 12:28:59.075659967 +0000 UTC m=+4129.663901731" Dec 03 12:28:59 crc kubenswrapper[5052]: I1203 12:28:59.113007 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/12f98520-40ab-4d0a-b48b-84cdd9005194-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "12f98520-40ab-4d0a-b48b-84cdd9005194" (UID: "12f98520-40ab-4d0a-b48b-84cdd9005194"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:28:59 crc kubenswrapper[5052]: I1203 12:28:59.130055 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/12f98520-40ab-4d0a-b48b-84cdd9005194-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:59 crc kubenswrapper[5052]: I1203 12:28:59.130085 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/12f98520-40ab-4d0a-b48b-84cdd9005194-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:59 crc kubenswrapper[5052]: I1203 12:28:59.130096 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-bctqz\" (UniqueName: \"kubernetes.io/projected/12f98520-40ab-4d0a-b48b-84cdd9005194-kube-api-access-bctqz\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:59 crc kubenswrapper[5052]: I1203 12:28:59.130106 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/12f98520-40ab-4d0a-b48b-84cdd9005194-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:59 crc kubenswrapper[5052]: I1203 12:28:59.132255 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/12f98520-40ab-4d0a-b48b-84cdd9005194-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "12f98520-40ab-4d0a-b48b-84cdd9005194" (UID: "12f98520-40ab-4d0a-b48b-84cdd9005194"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:28:59 crc kubenswrapper[5052]: I1203 12:28:59.134068 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/12f98520-40ab-4d0a-b48b-84cdd9005194-config-data" (OuterVolumeSpecName: "config-data") pod "12f98520-40ab-4d0a-b48b-84cdd9005194" (UID: "12f98520-40ab-4d0a-b48b-84cdd9005194"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:28:59 crc kubenswrapper[5052]: I1203 12:28:59.232048 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/12f98520-40ab-4d0a-b48b-84cdd9005194-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:59 crc kubenswrapper[5052]: I1203 12:28:59.232386 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/12f98520-40ab-4d0a-b48b-84cdd9005194-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:59 crc kubenswrapper[5052]: I1203 12:28:59.262502 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:28:59 crc kubenswrapper[5052]: I1203 12:28:59.268610 5052 scope.go:117] "RemoveContainer" containerID="68ac7a9d06036534e0af4f800761162dd7d95c40656710410c5ecdf61fa05368" Dec 03 12:28:59 crc kubenswrapper[5052]: I1203 12:28:59.335206 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a1219c4c-2ddf-4796-91dc-794412acc782-combined-ca-bundle\") pod \"a1219c4c-2ddf-4796-91dc-794412acc782\" (UID: \"a1219c4c-2ddf-4796-91dc-794412acc782\") " Dec 03 12:28:59 crc kubenswrapper[5052]: I1203 12:28:59.335286 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/a1219c4c-2ddf-4796-91dc-794412acc782-logs\") pod \"a1219c4c-2ddf-4796-91dc-794412acc782\" (UID: \"a1219c4c-2ddf-4796-91dc-794412acc782\") " Dec 03 12:28:59 crc kubenswrapper[5052]: I1203 12:28:59.335308 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a1219c4c-2ddf-4796-91dc-794412acc782-config-data\") pod \"a1219c4c-2ddf-4796-91dc-794412acc782\" (UID: \"a1219c4c-2ddf-4796-91dc-794412acc782\") " Dec 03 12:28:59 crc kubenswrapper[5052]: I1203 12:28:59.335342 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8k2mf\" (UniqueName: \"kubernetes.io/projected/a1219c4c-2ddf-4796-91dc-794412acc782-kube-api-access-8k2mf\") pod \"a1219c4c-2ddf-4796-91dc-794412acc782\" (UID: \"a1219c4c-2ddf-4796-91dc-794412acc782\") " Dec 03 12:28:59 crc kubenswrapper[5052]: I1203 12:28:59.335535 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/a1219c4c-2ddf-4796-91dc-794412acc782-nova-metadata-tls-certs\") pod \"a1219c4c-2ddf-4796-91dc-794412acc782\" (UID: \"a1219c4c-2ddf-4796-91dc-794412acc782\") " Dec 03 12:28:59 crc kubenswrapper[5052]: I1203 12:28:59.336150 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a1219c4c-2ddf-4796-91dc-794412acc782-logs" (OuterVolumeSpecName: "logs") pod "a1219c4c-2ddf-4796-91dc-794412acc782" (UID: "a1219c4c-2ddf-4796-91dc-794412acc782"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:28:59 crc kubenswrapper[5052]: I1203 12:28:59.337820 5052 scope.go:117] "RemoveContainer" containerID="9d81b16f8190231f8e209ce2911c03014e429b33c5316cafe9152f4b974fec6b" Dec 03 12:28:59 crc kubenswrapper[5052]: E1203 12:28:59.342671 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9d81b16f8190231f8e209ce2911c03014e429b33c5316cafe9152f4b974fec6b\": container with ID starting with 9d81b16f8190231f8e209ce2911c03014e429b33c5316cafe9152f4b974fec6b not found: ID does not exist" containerID="9d81b16f8190231f8e209ce2911c03014e429b33c5316cafe9152f4b974fec6b" Dec 03 12:28:59 crc kubenswrapper[5052]: I1203 12:28:59.342720 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9d81b16f8190231f8e209ce2911c03014e429b33c5316cafe9152f4b974fec6b"} err="failed to get container status \"9d81b16f8190231f8e209ce2911c03014e429b33c5316cafe9152f4b974fec6b\": rpc error: code = NotFound desc = could not find container \"9d81b16f8190231f8e209ce2911c03014e429b33c5316cafe9152f4b974fec6b\": container with ID starting with 9d81b16f8190231f8e209ce2911c03014e429b33c5316cafe9152f4b974fec6b not found: ID does not exist" Dec 03 12:28:59 crc kubenswrapper[5052]: I1203 12:28:59.342749 5052 scope.go:117] "RemoveContainer" containerID="68ac7a9d06036534e0af4f800761162dd7d95c40656710410c5ecdf61fa05368" Dec 03 12:28:59 crc kubenswrapper[5052]: I1203 12:28:59.348646 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a1219c4c-2ddf-4796-91dc-794412acc782-kube-api-access-8k2mf" (OuterVolumeSpecName: "kube-api-access-8k2mf") pod "a1219c4c-2ddf-4796-91dc-794412acc782" (UID: "a1219c4c-2ddf-4796-91dc-794412acc782"). InnerVolumeSpecName "kube-api-access-8k2mf". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:28:59 crc kubenswrapper[5052]: E1203 12:28:59.348692 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"68ac7a9d06036534e0af4f800761162dd7d95c40656710410c5ecdf61fa05368\": container with ID starting with 68ac7a9d06036534e0af4f800761162dd7d95c40656710410c5ecdf61fa05368 not found: ID does not exist" containerID="68ac7a9d06036534e0af4f800761162dd7d95c40656710410c5ecdf61fa05368" Dec 03 12:28:59 crc kubenswrapper[5052]: I1203 12:28:59.348733 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"68ac7a9d06036534e0af4f800761162dd7d95c40656710410c5ecdf61fa05368"} err="failed to get container status \"68ac7a9d06036534e0af4f800761162dd7d95c40656710410c5ecdf61fa05368\": rpc error: code = NotFound desc = could not find container \"68ac7a9d06036534e0af4f800761162dd7d95c40656710410c5ecdf61fa05368\": container with ID starting with 68ac7a9d06036534e0af4f800761162dd7d95c40656710410c5ecdf61fa05368 not found: ID does not exist" Dec 03 12:28:59 crc kubenswrapper[5052]: I1203 12:28:59.348759 5052 scope.go:117] "RemoveContainer" containerID="9d81b16f8190231f8e209ce2911c03014e429b33c5316cafe9152f4b974fec6b" Dec 03 12:28:59 crc kubenswrapper[5052]: I1203 12:28:59.350714 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9d81b16f8190231f8e209ce2911c03014e429b33c5316cafe9152f4b974fec6b"} err="failed to get container status \"9d81b16f8190231f8e209ce2911c03014e429b33c5316cafe9152f4b974fec6b\": rpc error: code = NotFound desc = could not find container \"9d81b16f8190231f8e209ce2911c03014e429b33c5316cafe9152f4b974fec6b\": container with ID starting with 9d81b16f8190231f8e209ce2911c03014e429b33c5316cafe9152f4b974fec6b not found: ID does not exist" Dec 03 12:28:59 crc kubenswrapper[5052]: I1203 12:28:59.350769 5052 scope.go:117] "RemoveContainer" containerID="68ac7a9d06036534e0af4f800761162dd7d95c40656710410c5ecdf61fa05368" Dec 03 12:28:59 crc kubenswrapper[5052]: I1203 12:28:59.355777 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"68ac7a9d06036534e0af4f800761162dd7d95c40656710410c5ecdf61fa05368"} err="failed to get container status \"68ac7a9d06036534e0af4f800761162dd7d95c40656710410c5ecdf61fa05368\": rpc error: code = NotFound desc = could not find container \"68ac7a9d06036534e0af4f800761162dd7d95c40656710410c5ecdf61fa05368\": container with ID starting with 68ac7a9d06036534e0af4f800761162dd7d95c40656710410c5ecdf61fa05368 not found: ID does not exist" Dec 03 12:28:59 crc kubenswrapper[5052]: I1203 12:28:59.372637 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a1219c4c-2ddf-4796-91dc-794412acc782-config-data" (OuterVolumeSpecName: "config-data") pod "a1219c4c-2ddf-4796-91dc-794412acc782" (UID: "a1219c4c-2ddf-4796-91dc-794412acc782"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:28:59 crc kubenswrapper[5052]: I1203 12:28:59.372689 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a1219c4c-2ddf-4796-91dc-794412acc782-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "a1219c4c-2ddf-4796-91dc-794412acc782" (UID: "a1219c4c-2ddf-4796-91dc-794412acc782"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:28:59 crc kubenswrapper[5052]: I1203 12:28:59.379186 5052 patch_prober.go:28] interesting pod/machine-config-daemon-74bj6 container/machine-config-daemon namespace/openshift-machine-config-operator: Liveness probe status=failure output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" start-of-body= Dec 03 12:28:59 crc kubenswrapper[5052]: I1203 12:28:59.379236 5052 prober.go:107] "Probe failed" probeType="Liveness" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" containerName="machine-config-daemon" probeResult="failure" output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" Dec 03 12:28:59 crc kubenswrapper[5052]: I1203 12:28:59.379274 5052 kubelet.go:2542] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" Dec 03 12:28:59 crc kubenswrapper[5052]: I1203 12:28:59.380097 5052 kuberuntime_manager.go:1027] "Message for Container of pod" containerName="machine-config-daemon" containerStatusID={"Type":"cri-o","ID":"658f4d3362e1d46f122d256822917d243f01c3aff5e32c450b855cf70ebcd2f9"} pod="openshift-machine-config-operator/machine-config-daemon-74bj6" containerMessage="Container machine-config-daemon failed liveness probe, will be restarted" Dec 03 12:28:59 crc kubenswrapper[5052]: I1203 12:28:59.380152 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" containerName="machine-config-daemon" containerID="cri-o://658f4d3362e1d46f122d256822917d243f01c3aff5e32c450b855cf70ebcd2f9" gracePeriod=600 Dec 03 12:28:59 crc kubenswrapper[5052]: I1203 12:28:59.404323 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a1219c4c-2ddf-4796-91dc-794412acc782-nova-metadata-tls-certs" (OuterVolumeSpecName: "nova-metadata-tls-certs") pod "a1219c4c-2ddf-4796-91dc-794412acc782" (UID: "a1219c4c-2ddf-4796-91dc-794412acc782"). InnerVolumeSpecName "nova-metadata-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:28:59 crc kubenswrapper[5052]: I1203 12:28:59.423715 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:28:59 crc kubenswrapper[5052]: I1203 12:28:59.451711 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:28:59 crc kubenswrapper[5052]: I1203 12:28:59.454351 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a1219c4c-2ddf-4796-91dc-794412acc782-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:59 crc kubenswrapper[5052]: I1203 12:28:59.454416 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/a1219c4c-2ddf-4796-91dc-794412acc782-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:59 crc kubenswrapper[5052]: I1203 12:28:59.454432 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a1219c4c-2ddf-4796-91dc-794412acc782-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:59 crc kubenswrapper[5052]: I1203 12:28:59.454446 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-8k2mf\" (UniqueName: \"kubernetes.io/projected/a1219c4c-2ddf-4796-91dc-794412acc782-kube-api-access-8k2mf\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:59 crc kubenswrapper[5052]: I1203 12:28:59.454462 5052 reconciler_common.go:293] "Volume detached for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/a1219c4c-2ddf-4796-91dc-794412acc782-nova-metadata-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:28:59 crc kubenswrapper[5052]: I1203 12:28:59.468535 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:28:59 crc kubenswrapper[5052]: E1203 12:28:59.468963 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="12f98520-40ab-4d0a-b48b-84cdd9005194" containerName="nova-api-log" Dec 03 12:28:59 crc kubenswrapper[5052]: I1203 12:28:59.468987 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="12f98520-40ab-4d0a-b48b-84cdd9005194" containerName="nova-api-log" Dec 03 12:28:59 crc kubenswrapper[5052]: E1203 12:28:59.469006 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="a1219c4c-2ddf-4796-91dc-794412acc782" containerName="nova-metadata-metadata" Dec 03 12:28:59 crc kubenswrapper[5052]: I1203 12:28:59.469013 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="a1219c4c-2ddf-4796-91dc-794412acc782" containerName="nova-metadata-metadata" Dec 03 12:28:59 crc kubenswrapper[5052]: E1203 12:28:59.469024 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="12f98520-40ab-4d0a-b48b-84cdd9005194" containerName="nova-api-api" Dec 03 12:28:59 crc kubenswrapper[5052]: I1203 12:28:59.469030 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="12f98520-40ab-4d0a-b48b-84cdd9005194" containerName="nova-api-api" Dec 03 12:28:59 crc kubenswrapper[5052]: E1203 12:28:59.469051 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="9f515435-a645-4d2b-9bd5-4bb9496f5405" containerName="proxy-httpd" Dec 03 12:28:59 crc kubenswrapper[5052]: I1203 12:28:59.469057 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="9f515435-a645-4d2b-9bd5-4bb9496f5405" containerName="proxy-httpd" Dec 03 12:28:59 crc kubenswrapper[5052]: E1203 12:28:59.469067 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="9f515435-a645-4d2b-9bd5-4bb9496f5405" containerName="proxy-server" Dec 03 12:28:59 crc kubenswrapper[5052]: I1203 12:28:59.469072 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="9f515435-a645-4d2b-9bd5-4bb9496f5405" containerName="proxy-server" Dec 03 12:28:59 crc kubenswrapper[5052]: E1203 12:28:59.469082 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="a1219c4c-2ddf-4796-91dc-794412acc782" containerName="nova-metadata-log" Dec 03 12:28:59 crc kubenswrapper[5052]: I1203 12:28:59.469088 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="a1219c4c-2ddf-4796-91dc-794412acc782" containerName="nova-metadata-log" Dec 03 12:28:59 crc kubenswrapper[5052]: I1203 12:28:59.469254 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="9f515435-a645-4d2b-9bd5-4bb9496f5405" containerName="proxy-server" Dec 03 12:28:59 crc kubenswrapper[5052]: I1203 12:28:59.469265 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="a1219c4c-2ddf-4796-91dc-794412acc782" containerName="nova-metadata-log" Dec 03 12:28:59 crc kubenswrapper[5052]: I1203 12:28:59.469282 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="12f98520-40ab-4d0a-b48b-84cdd9005194" containerName="nova-api-api" Dec 03 12:28:59 crc kubenswrapper[5052]: I1203 12:28:59.469294 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="9f515435-a645-4d2b-9bd5-4bb9496f5405" containerName="proxy-httpd" Dec 03 12:28:59 crc kubenswrapper[5052]: I1203 12:28:59.469304 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="a1219c4c-2ddf-4796-91dc-794412acc782" containerName="nova-metadata-metadata" Dec 03 12:28:59 crc kubenswrapper[5052]: I1203 12:28:59.469316 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="12f98520-40ab-4d0a-b48b-84cdd9005194" containerName="nova-api-log" Dec 03 12:28:59 crc kubenswrapper[5052]: I1203 12:28:59.470373 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:28:59 crc kubenswrapper[5052]: I1203 12:28:59.476933 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:28:59 crc kubenswrapper[5052]: I1203 12:28:59.477932 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-nova-internal-svc" Dec 03 12:28:59 crc kubenswrapper[5052]: I1203 12:28:59.478327 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-api-config-data" Dec 03 12:28:59 crc kubenswrapper[5052]: I1203 12:28:59.478465 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-nova-public-svc" Dec 03 12:28:59 crc kubenswrapper[5052]: I1203 12:28:59.556031 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p4phv\" (UniqueName: \"kubernetes.io/projected/b5b5ad16-1e45-4761-b5d2-60ffd5934ccb-kube-api-access-p4phv\") pod \"nova-api-0\" (UID: \"b5b5ad16-1e45-4761-b5d2-60ffd5934ccb\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:28:59 crc kubenswrapper[5052]: I1203 12:28:59.556101 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/b5b5ad16-1e45-4761-b5d2-60ffd5934ccb-public-tls-certs\") pod \"nova-api-0\" (UID: \"b5b5ad16-1e45-4761-b5d2-60ffd5934ccb\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:28:59 crc kubenswrapper[5052]: I1203 12:28:59.556155 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/b5b5ad16-1e45-4761-b5d2-60ffd5934ccb-config-data\") pod \"nova-api-0\" (UID: \"b5b5ad16-1e45-4761-b5d2-60ffd5934ccb\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:28:59 crc kubenswrapper[5052]: I1203 12:28:59.556179 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/b5b5ad16-1e45-4761-b5d2-60ffd5934ccb-internal-tls-certs\") pod \"nova-api-0\" (UID: \"b5b5ad16-1e45-4761-b5d2-60ffd5934ccb\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:28:59 crc kubenswrapper[5052]: I1203 12:28:59.556198 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/b5b5ad16-1e45-4761-b5d2-60ffd5934ccb-logs\") pod \"nova-api-0\" (UID: \"b5b5ad16-1e45-4761-b5d2-60ffd5934ccb\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:28:59 crc kubenswrapper[5052]: I1203 12:28:59.556242 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b5b5ad16-1e45-4761-b5d2-60ffd5934ccb-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"b5b5ad16-1e45-4761-b5d2-60ffd5934ccb\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:28:59 crc kubenswrapper[5052]: I1203 12:28:59.658046 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b5b5ad16-1e45-4761-b5d2-60ffd5934ccb-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"b5b5ad16-1e45-4761-b5d2-60ffd5934ccb\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:28:59 crc kubenswrapper[5052]: I1203 12:28:59.658723 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-p4phv\" (UniqueName: \"kubernetes.io/projected/b5b5ad16-1e45-4761-b5d2-60ffd5934ccb-kube-api-access-p4phv\") pod \"nova-api-0\" (UID: \"b5b5ad16-1e45-4761-b5d2-60ffd5934ccb\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:28:59 crc kubenswrapper[5052]: I1203 12:28:59.658907 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/b5b5ad16-1e45-4761-b5d2-60ffd5934ccb-public-tls-certs\") pod \"nova-api-0\" (UID: \"b5b5ad16-1e45-4761-b5d2-60ffd5934ccb\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:28:59 crc kubenswrapper[5052]: I1203 12:28:59.659081 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/b5b5ad16-1e45-4761-b5d2-60ffd5934ccb-config-data\") pod \"nova-api-0\" (UID: \"b5b5ad16-1e45-4761-b5d2-60ffd5934ccb\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:28:59 crc kubenswrapper[5052]: I1203 12:28:59.659207 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/b5b5ad16-1e45-4761-b5d2-60ffd5934ccb-internal-tls-certs\") pod \"nova-api-0\" (UID: \"b5b5ad16-1e45-4761-b5d2-60ffd5934ccb\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:28:59 crc kubenswrapper[5052]: I1203 12:28:59.659319 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/b5b5ad16-1e45-4761-b5d2-60ffd5934ccb-logs\") pod \"nova-api-0\" (UID: \"b5b5ad16-1e45-4761-b5d2-60ffd5934ccb\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:28:59 crc kubenswrapper[5052]: I1203 12:28:59.659962 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/b5b5ad16-1e45-4761-b5d2-60ffd5934ccb-logs\") pod \"nova-api-0\" (UID: \"b5b5ad16-1e45-4761-b5d2-60ffd5934ccb\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:28:59 crc kubenswrapper[5052]: I1203 12:28:59.662750 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b5b5ad16-1e45-4761-b5d2-60ffd5934ccb-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"b5b5ad16-1e45-4761-b5d2-60ffd5934ccb\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:28:59 crc kubenswrapper[5052]: I1203 12:28:59.663016 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/b5b5ad16-1e45-4761-b5d2-60ffd5934ccb-internal-tls-certs\") pod \"nova-api-0\" (UID: \"b5b5ad16-1e45-4761-b5d2-60ffd5934ccb\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:28:59 crc kubenswrapper[5052]: I1203 12:28:59.663779 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/b5b5ad16-1e45-4761-b5d2-60ffd5934ccb-config-data\") pod \"nova-api-0\" (UID: \"b5b5ad16-1e45-4761-b5d2-60ffd5934ccb\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:28:59 crc kubenswrapper[5052]: I1203 12:28:59.664568 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/b5b5ad16-1e45-4761-b5d2-60ffd5934ccb-public-tls-certs\") pod \"nova-api-0\" (UID: \"b5b5ad16-1e45-4761-b5d2-60ffd5934ccb\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:28:59 crc kubenswrapper[5052]: I1203 12:28:59.678876 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-p4phv\" (UniqueName: \"kubernetes.io/projected/b5b5ad16-1e45-4761-b5d2-60ffd5934ccb-kube-api-access-p4phv\") pod \"nova-api-0\" (UID: \"b5b5ad16-1e45-4761-b5d2-60ffd5934ccb\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:28:59 crc kubenswrapper[5052]: I1203 12:28:59.692955 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="12f98520-40ab-4d0a-b48b-84cdd9005194" path="/var/lib/kubelet/pods/12f98520-40ab-4d0a-b48b-84cdd9005194/volumes" Dec 03 12:28:59 crc kubenswrapper[5052]: I1203 12:28:59.693894 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9f515435-a645-4d2b-9bd5-4bb9496f5405" path="/var/lib/kubelet/pods/9f515435-a645-4d2b-9bd5-4bb9496f5405/volumes" Dec 03 12:28:59 crc kubenswrapper[5052]: I1203 12:28:59.797478 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:29:00 crc kubenswrapper[5052]: I1203 12:29:00.086314 5052 generic.go:334] "Generic (PLEG): container finished" podID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" containerID="658f4d3362e1d46f122d256822917d243f01c3aff5e32c450b855cf70ebcd2f9" exitCode=0 Dec 03 12:29:00 crc kubenswrapper[5052]: I1203 12:29:00.086458 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" event={"ID":"10c237bc-72e7-4da3-9f2c-b1688395bf6e","Type":"ContainerDied","Data":"658f4d3362e1d46f122d256822917d243f01c3aff5e32c450b855cf70ebcd2f9"} Dec 03 12:29:00 crc kubenswrapper[5052]: I1203 12:29:00.086604 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" event={"ID":"10c237bc-72e7-4da3-9f2c-b1688395bf6e","Type":"ContainerStarted","Data":"e6edaf188f85ea4f11f487f11cc489c700f25fb42b5ee269f4f496f1e127d772"} Dec 03 12:29:00 crc kubenswrapper[5052]: I1203 12:29:00.086629 5052 scope.go:117] "RemoveContainer" containerID="09b4a0072741f60499d24f40cc4ff3406d5d22671ced431363ab865172786300" Dec 03 12:29:00 crc kubenswrapper[5052]: I1203 12:29:00.103479 5052 generic.go:334] "Generic (PLEG): container finished" podID="a65d6de5-f108-40c9-81f6-ca37534c7710" containerID="c18a95292e924f3db5577fc56cbeeb0a4fd9bbc3a34919d2cc0208c6fb0b74b9" exitCode=0 Dec 03 12:29:00 crc kubenswrapper[5052]: I1203 12:29:00.103552 5052 generic.go:334] "Generic (PLEG): container finished" podID="a65d6de5-f108-40c9-81f6-ca37534c7710" containerID="b39e98c50755a3d0626f639cee9dde7e86c700c52f6ddaade4e3e9fe9b523800" exitCode=2 Dec 03 12:29:00 crc kubenswrapper[5052]: I1203 12:29:00.103568 5052 generic.go:334] "Generic (PLEG): container finished" podID="a65d6de5-f108-40c9-81f6-ca37534c7710" containerID="55f42c22fa007045b7bcac5be13d03999c40148b147e8c2a61d685132dd50766" exitCode=0 Dec 03 12:29:00 crc kubenswrapper[5052]: I1203 12:29:00.103569 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"a65d6de5-f108-40c9-81f6-ca37534c7710","Type":"ContainerDied","Data":"c18a95292e924f3db5577fc56cbeeb0a4fd9bbc3a34919d2cc0208c6fb0b74b9"} Dec 03 12:29:00 crc kubenswrapper[5052]: I1203 12:29:00.103930 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"a65d6de5-f108-40c9-81f6-ca37534c7710","Type":"ContainerDied","Data":"b39e98c50755a3d0626f639cee9dde7e86c700c52f6ddaade4e3e9fe9b523800"} Dec 03 12:29:00 crc kubenswrapper[5052]: I1203 12:29:00.103950 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"a65d6de5-f108-40c9-81f6-ca37534c7710","Type":"ContainerDied","Data":"55f42c22fa007045b7bcac5be13d03999c40148b147e8c2a61d685132dd50766"} Dec 03 12:29:00 crc kubenswrapper[5052]: I1203 12:29:00.110616 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"a1219c4c-2ddf-4796-91dc-794412acc782","Type":"ContainerDied","Data":"a7e0a610c5ef7645b9c47f5262deb1f12b88d93827f984b0bdb49fec1b8ef83b"} Dec 03 12:29:00 crc kubenswrapper[5052]: I1203 12:29:00.110696 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:29:00 crc kubenswrapper[5052]: I1203 12:29:00.112024 5052 scope.go:117] "RemoveContainer" containerID="bca6a9ef98c07e4231ae7fe76378e651c687dd9be2f44296fae8a750007f8c21" Dec 03 12:29:00 crc kubenswrapper[5052]: I1203 12:29:00.139622 5052 scope.go:117] "RemoveContainer" containerID="dade77d2933a2006677ea6d4ae01efc11be40a32911ed7be983068b4d33189c2" Dec 03 12:29:00 crc kubenswrapper[5052]: I1203 12:29:00.148575 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:29:00 crc kubenswrapper[5052]: I1203 12:29:00.166939 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:29:00 crc kubenswrapper[5052]: I1203 12:29:00.175290 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:29:00 crc kubenswrapper[5052]: I1203 12:29:00.177419 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:29:00 crc kubenswrapper[5052]: I1203 12:29:00.181435 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-metadata-config-data" Dec 03 12:29:00 crc kubenswrapper[5052]: I1203 12:29:00.181679 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-nova-metadata-internal-svc" Dec 03 12:29:00 crc kubenswrapper[5052]: I1203 12:29:00.185761 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:29:00 crc kubenswrapper[5052]: I1203 12:29:00.274400 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4ldbv\" (UniqueName: \"kubernetes.io/projected/caaecb05-f5f1-421d-8e91-818774a2c0ec-kube-api-access-4ldbv\") pod \"nova-metadata-0\" (UID: \"caaecb05-f5f1-421d-8e91-818774a2c0ec\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:29:00 crc kubenswrapper[5052]: I1203 12:29:00.274474 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/caaecb05-f5f1-421d-8e91-818774a2c0ec-config-data\") pod \"nova-metadata-0\" (UID: \"caaecb05-f5f1-421d-8e91-818774a2c0ec\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:29:00 crc kubenswrapper[5052]: I1203 12:29:00.274583 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/caaecb05-f5f1-421d-8e91-818774a2c0ec-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"caaecb05-f5f1-421d-8e91-818774a2c0ec\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:29:00 crc kubenswrapper[5052]: I1203 12:29:00.274641 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/caaecb05-f5f1-421d-8e91-818774a2c0ec-logs\") pod \"nova-metadata-0\" (UID: \"caaecb05-f5f1-421d-8e91-818774a2c0ec\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:29:00 crc kubenswrapper[5052]: I1203 12:29:00.274790 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/caaecb05-f5f1-421d-8e91-818774a2c0ec-nova-metadata-tls-certs\") pod \"nova-metadata-0\" (UID: \"caaecb05-f5f1-421d-8e91-818774a2c0ec\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:29:00 crc kubenswrapper[5052]: E1203 12:29:00.295643 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="0dcdd2aec7240ab8991d19f9ba29d4cff2af7594281adff351a9d0b94735f781" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:29:00 crc kubenswrapper[5052]: E1203 12:29:00.297044 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="0dcdd2aec7240ab8991d19f9ba29d4cff2af7594281adff351a9d0b94735f781" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:29:00 crc kubenswrapper[5052]: I1203 12:29:00.308240 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:29:00 crc kubenswrapper[5052]: E1203 12:29:00.313742 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="0dcdd2aec7240ab8991d19f9ba29d4cff2af7594281adff351a9d0b94735f781" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:29:00 crc kubenswrapper[5052]: E1203 12:29:00.313807 5052 prober.go:104] "Probe errored" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" probeType="Readiness" pod="openstack-kuttl-tests/nova-cell0-conductor-0" podUID="211396f3-156a-4ac5-917b-6d41c236d3c2" containerName="nova-cell0-conductor-conductor" Dec 03 12:29:00 crc kubenswrapper[5052]: I1203 12:29:00.377064 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/caaecb05-f5f1-421d-8e91-818774a2c0ec-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"caaecb05-f5f1-421d-8e91-818774a2c0ec\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:29:00 crc kubenswrapper[5052]: I1203 12:29:00.377117 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/caaecb05-f5f1-421d-8e91-818774a2c0ec-logs\") pod \"nova-metadata-0\" (UID: \"caaecb05-f5f1-421d-8e91-818774a2c0ec\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:29:00 crc kubenswrapper[5052]: I1203 12:29:00.377187 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/caaecb05-f5f1-421d-8e91-818774a2c0ec-nova-metadata-tls-certs\") pod \"nova-metadata-0\" (UID: \"caaecb05-f5f1-421d-8e91-818774a2c0ec\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:29:00 crc kubenswrapper[5052]: I1203 12:29:00.377341 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-4ldbv\" (UniqueName: \"kubernetes.io/projected/caaecb05-f5f1-421d-8e91-818774a2c0ec-kube-api-access-4ldbv\") pod \"nova-metadata-0\" (UID: \"caaecb05-f5f1-421d-8e91-818774a2c0ec\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:29:00 crc kubenswrapper[5052]: I1203 12:29:00.377382 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/caaecb05-f5f1-421d-8e91-818774a2c0ec-config-data\") pod \"nova-metadata-0\" (UID: \"caaecb05-f5f1-421d-8e91-818774a2c0ec\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:29:00 crc kubenswrapper[5052]: I1203 12:29:00.377784 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/caaecb05-f5f1-421d-8e91-818774a2c0ec-logs\") pod \"nova-metadata-0\" (UID: \"caaecb05-f5f1-421d-8e91-818774a2c0ec\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:29:00 crc kubenswrapper[5052]: I1203 12:29:00.383339 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/caaecb05-f5f1-421d-8e91-818774a2c0ec-config-data\") pod \"nova-metadata-0\" (UID: \"caaecb05-f5f1-421d-8e91-818774a2c0ec\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:29:00 crc kubenswrapper[5052]: I1203 12:29:00.383939 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/caaecb05-f5f1-421d-8e91-818774a2c0ec-nova-metadata-tls-certs\") pod \"nova-metadata-0\" (UID: \"caaecb05-f5f1-421d-8e91-818774a2c0ec\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:29:00 crc kubenswrapper[5052]: I1203 12:29:00.384359 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/caaecb05-f5f1-421d-8e91-818774a2c0ec-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"caaecb05-f5f1-421d-8e91-818774a2c0ec\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:29:00 crc kubenswrapper[5052]: I1203 12:29:00.394264 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-4ldbv\" (UniqueName: \"kubernetes.io/projected/caaecb05-f5f1-421d-8e91-818774a2c0ec-kube-api-access-4ldbv\") pod \"nova-metadata-0\" (UID: \"caaecb05-f5f1-421d-8e91-818774a2c0ec\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:29:00 crc kubenswrapper[5052]: I1203 12:29:00.499421 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:29:01 crc kubenswrapper[5052]: I1203 12:29:01.093826 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:29:01 crc kubenswrapper[5052]: W1203 12:29:01.110253 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcaaecb05_f5f1_421d_8e91_818774a2c0ec.slice/crio-d5c2864b66fc824342017a2d07a8d11e2ef48d803062db360c43d8cd43d15324 WatchSource:0}: Error finding container d5c2864b66fc824342017a2d07a8d11e2ef48d803062db360c43d8cd43d15324: Status 404 returned error can't find the container with id d5c2864b66fc824342017a2d07a8d11e2ef48d803062db360c43d8cd43d15324 Dec 03 12:29:01 crc kubenswrapper[5052]: I1203 12:29:01.129781 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"caaecb05-f5f1-421d-8e91-818774a2c0ec","Type":"ContainerStarted","Data":"d5c2864b66fc824342017a2d07a8d11e2ef48d803062db360c43d8cd43d15324"} Dec 03 12:29:01 crc kubenswrapper[5052]: I1203 12:29:01.196754 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"b5b5ad16-1e45-4761-b5d2-60ffd5934ccb","Type":"ContainerStarted","Data":"ca424ba734d34bcc97da61c506acc5ac518a281efdcc25fe387f6db662517ec3"} Dec 03 12:29:01 crc kubenswrapper[5052]: I1203 12:29:01.197079 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"b5b5ad16-1e45-4761-b5d2-60ffd5934ccb","Type":"ContainerStarted","Data":"0e4e8a3b2fb3bbe0432215ed54bf039b98ca0a274e59dc10062d81386889219c"} Dec 03 12:29:01 crc kubenswrapper[5052]: I1203 12:29:01.197093 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"b5b5ad16-1e45-4761-b5d2-60ffd5934ccb","Type":"ContainerStarted","Data":"b6f8d9e3b9e593880b90758ff3f0063a1575fa66b5df5219ef9bdd6d868ba33a"} Dec 03 12:29:01 crc kubenswrapper[5052]: I1203 12:29:01.226934 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-api-0" podStartSLOduration=2.226915378 podStartE2EDuration="2.226915378s" podCreationTimestamp="2025-12-03 12:28:59 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:29:01.217952902 +0000 UTC m=+4131.806194666" watchObservedRunningTime="2025-12-03 12:29:01.226915378 +0000 UTC m=+4131.815157142" Dec 03 12:29:01 crc kubenswrapper[5052]: I1203 12:29:01.685909 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a1219c4c-2ddf-4796-91dc-794412acc782" path="/var/lib/kubelet/pods/a1219c4c-2ddf-4796-91dc-794412acc782/volumes" Dec 03 12:29:02 crc kubenswrapper[5052]: I1203 12:29:02.230371 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"caaecb05-f5f1-421d-8e91-818774a2c0ec","Type":"ContainerStarted","Data":"e2e4d22232c587b84e25f9aceb08f0b3c39f38badcaa667d82792f3314b6a1b9"} Dec 03 12:29:02 crc kubenswrapper[5052]: I1203 12:29:02.230417 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"caaecb05-f5f1-421d-8e91-818774a2c0ec","Type":"ContainerStarted","Data":"cda20389ae104a8a76ec7b7e03e816da57afd5a07e90e7f0e1bf79a5704aa60d"} Dec 03 12:29:02 crc kubenswrapper[5052]: I1203 12:29:02.274795 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-metadata-0" podStartSLOduration=2.274768866 podStartE2EDuration="2.274768866s" podCreationTimestamp="2025-12-03 12:29:00 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:29:02.271959246 +0000 UTC m=+4132.860201030" watchObservedRunningTime="2025-12-03 12:29:02.274768866 +0000 UTC m=+4132.863010650" Dec 03 12:29:03 crc kubenswrapper[5052]: E1203 12:29:03.826278 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="f0d03487bb9b0001a9b06ddae0262ca9d26eb73013d9bccc682cbc0b7a9e1bbe" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:29:03 crc kubenswrapper[5052]: E1203 12:29:03.833939 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="f0d03487bb9b0001a9b06ddae0262ca9d26eb73013d9bccc682cbc0b7a9e1bbe" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:29:03 crc kubenswrapper[5052]: E1203 12:29:03.835455 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="f0d03487bb9b0001a9b06ddae0262ca9d26eb73013d9bccc682cbc0b7a9e1bbe" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:29:03 crc kubenswrapper[5052]: E1203 12:29:03.835544 5052 prober.go:104] "Probe errored" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" probeType="Readiness" pod="openstack-kuttl-tests/nova-cell1-conductor-0" podUID="026d5ba1-5f6f-4f39-bcb6-9c51d37608f2" containerName="nova-cell1-conductor-conductor" Dec 03 12:29:05 crc kubenswrapper[5052]: E1203 12:29:05.295646 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="0dcdd2aec7240ab8991d19f9ba29d4cff2af7594281adff351a9d0b94735f781" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:29:05 crc kubenswrapper[5052]: E1203 12:29:05.297482 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="0dcdd2aec7240ab8991d19f9ba29d4cff2af7594281adff351a9d0b94735f781" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:29:05 crc kubenswrapper[5052]: E1203 12:29:05.299444 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="0dcdd2aec7240ab8991d19f9ba29d4cff2af7594281adff351a9d0b94735f781" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:29:05 crc kubenswrapper[5052]: E1203 12:29:05.299495 5052 prober.go:104] "Probe errored" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" probeType="Readiness" pod="openstack-kuttl-tests/nova-cell0-conductor-0" podUID="211396f3-156a-4ac5-917b-6d41c236d3c2" containerName="nova-cell0-conductor-conductor" Dec 03 12:29:05 crc kubenswrapper[5052]: I1203 12:29:05.499909 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:29:05 crc kubenswrapper[5052]: I1203 12:29:05.499971 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:29:08 crc kubenswrapper[5052]: I1203 12:29:08.289900 5052 generic.go:334] "Generic (PLEG): container finished" podID="a65d6de5-f108-40c9-81f6-ca37534c7710" containerID="e0121d77497086b42ab3b08972dbfa1ef63c206df7997ec8f0f64203225a2322" exitCode=0 Dec 03 12:29:08 crc kubenswrapper[5052]: I1203 12:29:08.290228 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"a65d6de5-f108-40c9-81f6-ca37534c7710","Type":"ContainerDied","Data":"e0121d77497086b42ab3b08972dbfa1ef63c206df7997ec8f0f64203225a2322"} Dec 03 12:29:08 crc kubenswrapper[5052]: I1203 12:29:08.706787 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:29:08 crc kubenswrapper[5052]: I1203 12:29:08.735981 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/a65d6de5-f108-40c9-81f6-ca37534c7710-run-httpd\") pod \"a65d6de5-f108-40c9-81f6-ca37534c7710\" (UID: \"a65d6de5-f108-40c9-81f6-ca37534c7710\") " Dec 03 12:29:08 crc kubenswrapper[5052]: I1203 12:29:08.736085 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a65d6de5-f108-40c9-81f6-ca37534c7710-config-data\") pod \"a65d6de5-f108-40c9-81f6-ca37534c7710\" (UID: \"a65d6de5-f108-40c9-81f6-ca37534c7710\") " Dec 03 12:29:08 crc kubenswrapper[5052]: I1203 12:29:08.736160 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sn676\" (UniqueName: \"kubernetes.io/projected/a65d6de5-f108-40c9-81f6-ca37534c7710-kube-api-access-sn676\") pod \"a65d6de5-f108-40c9-81f6-ca37534c7710\" (UID: \"a65d6de5-f108-40c9-81f6-ca37534c7710\") " Dec 03 12:29:08 crc kubenswrapper[5052]: I1203 12:29:08.736212 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/a65d6de5-f108-40c9-81f6-ca37534c7710-ceilometer-tls-certs\") pod \"a65d6de5-f108-40c9-81f6-ca37534c7710\" (UID: \"a65d6de5-f108-40c9-81f6-ca37534c7710\") " Dec 03 12:29:08 crc kubenswrapper[5052]: I1203 12:29:08.736280 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/a65d6de5-f108-40c9-81f6-ca37534c7710-sg-core-conf-yaml\") pod \"a65d6de5-f108-40c9-81f6-ca37534c7710\" (UID: \"a65d6de5-f108-40c9-81f6-ca37534c7710\") " Dec 03 12:29:08 crc kubenswrapper[5052]: I1203 12:29:08.736367 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/a65d6de5-f108-40c9-81f6-ca37534c7710-scripts\") pod \"a65d6de5-f108-40c9-81f6-ca37534c7710\" (UID: \"a65d6de5-f108-40c9-81f6-ca37534c7710\") " Dec 03 12:29:08 crc kubenswrapper[5052]: I1203 12:29:08.736417 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a65d6de5-f108-40c9-81f6-ca37534c7710-combined-ca-bundle\") pod \"a65d6de5-f108-40c9-81f6-ca37534c7710\" (UID: \"a65d6de5-f108-40c9-81f6-ca37534c7710\") " Dec 03 12:29:08 crc kubenswrapper[5052]: I1203 12:29:08.736465 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/a65d6de5-f108-40c9-81f6-ca37534c7710-log-httpd\") pod \"a65d6de5-f108-40c9-81f6-ca37534c7710\" (UID: \"a65d6de5-f108-40c9-81f6-ca37534c7710\") " Dec 03 12:29:08 crc kubenswrapper[5052]: I1203 12:29:08.738348 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a65d6de5-f108-40c9-81f6-ca37534c7710-run-httpd" (OuterVolumeSpecName: "run-httpd") pod "a65d6de5-f108-40c9-81f6-ca37534c7710" (UID: "a65d6de5-f108-40c9-81f6-ca37534c7710"). InnerVolumeSpecName "run-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:29:08 crc kubenswrapper[5052]: I1203 12:29:08.740823 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a65d6de5-f108-40c9-81f6-ca37534c7710-log-httpd" (OuterVolumeSpecName: "log-httpd") pod "a65d6de5-f108-40c9-81f6-ca37534c7710" (UID: "a65d6de5-f108-40c9-81f6-ca37534c7710"). InnerVolumeSpecName "log-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:29:08 crc kubenswrapper[5052]: I1203 12:29:08.745742 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a65d6de5-f108-40c9-81f6-ca37534c7710-scripts" (OuterVolumeSpecName: "scripts") pod "a65d6de5-f108-40c9-81f6-ca37534c7710" (UID: "a65d6de5-f108-40c9-81f6-ca37534c7710"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:29:08 crc kubenswrapper[5052]: I1203 12:29:08.745860 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a65d6de5-f108-40c9-81f6-ca37534c7710-kube-api-access-sn676" (OuterVolumeSpecName: "kube-api-access-sn676") pod "a65d6de5-f108-40c9-81f6-ca37534c7710" (UID: "a65d6de5-f108-40c9-81f6-ca37534c7710"). InnerVolumeSpecName "kube-api-access-sn676". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:29:08 crc kubenswrapper[5052]: I1203 12:29:08.773531 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a65d6de5-f108-40c9-81f6-ca37534c7710-sg-core-conf-yaml" (OuterVolumeSpecName: "sg-core-conf-yaml") pod "a65d6de5-f108-40c9-81f6-ca37534c7710" (UID: "a65d6de5-f108-40c9-81f6-ca37534c7710"). InnerVolumeSpecName "sg-core-conf-yaml". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:29:08 crc kubenswrapper[5052]: I1203 12:29:08.799064 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a65d6de5-f108-40c9-81f6-ca37534c7710-ceilometer-tls-certs" (OuterVolumeSpecName: "ceilometer-tls-certs") pod "a65d6de5-f108-40c9-81f6-ca37534c7710" (UID: "a65d6de5-f108-40c9-81f6-ca37534c7710"). InnerVolumeSpecName "ceilometer-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:29:08 crc kubenswrapper[5052]: E1203 12:29:08.827553 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="f0d03487bb9b0001a9b06ddae0262ca9d26eb73013d9bccc682cbc0b7a9e1bbe" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:29:08 crc kubenswrapper[5052]: E1203 12:29:08.833314 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="f0d03487bb9b0001a9b06ddae0262ca9d26eb73013d9bccc682cbc0b7a9e1bbe" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:29:08 crc kubenswrapper[5052]: E1203 12:29:08.835118 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="f0d03487bb9b0001a9b06ddae0262ca9d26eb73013d9bccc682cbc0b7a9e1bbe" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:29:08 crc kubenswrapper[5052]: E1203 12:29:08.835157 5052 prober.go:104] "Probe errored" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" probeType="Readiness" pod="openstack-kuttl-tests/nova-cell1-conductor-0" podUID="026d5ba1-5f6f-4f39-bcb6-9c51d37608f2" containerName="nova-cell1-conductor-conductor" Dec 03 12:29:08 crc kubenswrapper[5052]: I1203 12:29:08.840276 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/a65d6de5-f108-40c9-81f6-ca37534c7710-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:29:08 crc kubenswrapper[5052]: I1203 12:29:08.840298 5052 reconciler_common.go:293] "Volume detached for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/a65d6de5-f108-40c9-81f6-ca37534c7710-log-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:29:08 crc kubenswrapper[5052]: I1203 12:29:08.840307 5052 reconciler_common.go:293] "Volume detached for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/a65d6de5-f108-40c9-81f6-ca37534c7710-run-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:29:08 crc kubenswrapper[5052]: I1203 12:29:08.840318 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-sn676\" (UniqueName: \"kubernetes.io/projected/a65d6de5-f108-40c9-81f6-ca37534c7710-kube-api-access-sn676\") on node \"crc\" DevicePath \"\"" Dec 03 12:29:08 crc kubenswrapper[5052]: I1203 12:29:08.840328 5052 reconciler_common.go:293] "Volume detached for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/a65d6de5-f108-40c9-81f6-ca37534c7710-ceilometer-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:29:08 crc kubenswrapper[5052]: I1203 12:29:08.840335 5052 reconciler_common.go:293] "Volume detached for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/a65d6de5-f108-40c9-81f6-ca37534c7710-sg-core-conf-yaml\") on node \"crc\" DevicePath \"\"" Dec 03 12:29:08 crc kubenswrapper[5052]: I1203 12:29:08.842770 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a65d6de5-f108-40c9-81f6-ca37534c7710-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "a65d6de5-f108-40c9-81f6-ca37534c7710" (UID: "a65d6de5-f108-40c9-81f6-ca37534c7710"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:29:08 crc kubenswrapper[5052]: I1203 12:29:08.861984 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a65d6de5-f108-40c9-81f6-ca37534c7710-config-data" (OuterVolumeSpecName: "config-data") pod "a65d6de5-f108-40c9-81f6-ca37534c7710" (UID: "a65d6de5-f108-40c9-81f6-ca37534c7710"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:29:08 crc kubenswrapper[5052]: I1203 12:29:08.942748 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a65d6de5-f108-40c9-81f6-ca37534c7710-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:29:08 crc kubenswrapper[5052]: I1203 12:29:08.942781 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a65d6de5-f108-40c9-81f6-ca37534c7710-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:29:09 crc kubenswrapper[5052]: I1203 12:29:09.303862 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"a65d6de5-f108-40c9-81f6-ca37534c7710","Type":"ContainerDied","Data":"7994bb2b17aebae05cb47ee8c2ba1bddcc96649268b928224de1f42b453d8f40"} Dec 03 12:29:09 crc kubenswrapper[5052]: I1203 12:29:09.303925 5052 scope.go:117] "RemoveContainer" containerID="c18a95292e924f3db5577fc56cbeeb0a4fd9bbc3a34919d2cc0208c6fb0b74b9" Dec 03 12:29:09 crc kubenswrapper[5052]: I1203 12:29:09.303940 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:29:09 crc kubenswrapper[5052]: I1203 12:29:09.336775 5052 scope.go:117] "RemoveContainer" containerID="b39e98c50755a3d0626f639cee9dde7e86c700c52f6ddaade4e3e9fe9b523800" Dec 03 12:29:09 crc kubenswrapper[5052]: I1203 12:29:09.348734 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:29:09 crc kubenswrapper[5052]: I1203 12:29:09.372063 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:29:09 crc kubenswrapper[5052]: I1203 12:29:09.372968 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:29:09 crc kubenswrapper[5052]: E1203 12:29:09.373426 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="a65d6de5-f108-40c9-81f6-ca37534c7710" containerName="proxy-httpd" Dec 03 12:29:09 crc kubenswrapper[5052]: I1203 12:29:09.373449 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="a65d6de5-f108-40c9-81f6-ca37534c7710" containerName="proxy-httpd" Dec 03 12:29:09 crc kubenswrapper[5052]: E1203 12:29:09.373491 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="a65d6de5-f108-40c9-81f6-ca37534c7710" containerName="ceilometer-notification-agent" Dec 03 12:29:09 crc kubenswrapper[5052]: I1203 12:29:09.373499 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="a65d6de5-f108-40c9-81f6-ca37534c7710" containerName="ceilometer-notification-agent" Dec 03 12:29:09 crc kubenswrapper[5052]: E1203 12:29:09.373543 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="a65d6de5-f108-40c9-81f6-ca37534c7710" containerName="sg-core" Dec 03 12:29:09 crc kubenswrapper[5052]: I1203 12:29:09.373552 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="a65d6de5-f108-40c9-81f6-ca37534c7710" containerName="sg-core" Dec 03 12:29:09 crc kubenswrapper[5052]: E1203 12:29:09.373567 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="a65d6de5-f108-40c9-81f6-ca37534c7710" containerName="ceilometer-central-agent" Dec 03 12:29:09 crc kubenswrapper[5052]: I1203 12:29:09.373574 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="a65d6de5-f108-40c9-81f6-ca37534c7710" containerName="ceilometer-central-agent" Dec 03 12:29:09 crc kubenswrapper[5052]: I1203 12:29:09.373925 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="a65d6de5-f108-40c9-81f6-ca37534c7710" containerName="ceilometer-notification-agent" Dec 03 12:29:09 crc kubenswrapper[5052]: I1203 12:29:09.373972 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="a65d6de5-f108-40c9-81f6-ca37534c7710" containerName="proxy-httpd" Dec 03 12:29:09 crc kubenswrapper[5052]: I1203 12:29:09.373996 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="a65d6de5-f108-40c9-81f6-ca37534c7710" containerName="ceilometer-central-agent" Dec 03 12:29:09 crc kubenswrapper[5052]: I1203 12:29:09.374008 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="a65d6de5-f108-40c9-81f6-ca37534c7710" containerName="sg-core" Dec 03 12:29:09 crc kubenswrapper[5052]: I1203 12:29:09.377198 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:29:09 crc kubenswrapper[5052]: I1203 12:29:09.381799 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-scripts" Dec 03 12:29:09 crc kubenswrapper[5052]: I1203 12:29:09.385860 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-ceilometer-internal-svc" Dec 03 12:29:09 crc kubenswrapper[5052]: I1203 12:29:09.386804 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-config-data" Dec 03 12:29:09 crc kubenswrapper[5052]: I1203 12:29:09.393943 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:29:09 crc kubenswrapper[5052]: I1203 12:29:09.395370 5052 scope.go:117] "RemoveContainer" containerID="55f42c22fa007045b7bcac5be13d03999c40148b147e8c2a61d685132dd50766" Dec 03 12:29:09 crc kubenswrapper[5052]: I1203 12:29:09.428804 5052 scope.go:117] "RemoveContainer" containerID="e0121d77497086b42ab3b08972dbfa1ef63c206df7997ec8f0f64203225a2322" Dec 03 12:29:09 crc kubenswrapper[5052]: I1203 12:29:09.451605 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/e75eea2c-2eb1-4f0f-97e5-63ce147e68dd-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"e75eea2c-2eb1-4f0f-97e5-63ce147e68dd\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:29:09 crc kubenswrapper[5052]: I1203 12:29:09.451653 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/e75eea2c-2eb1-4f0f-97e5-63ce147e68dd-config-data\") pod \"ceilometer-0\" (UID: \"e75eea2c-2eb1-4f0f-97e5-63ce147e68dd\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:29:09 crc kubenswrapper[5052]: I1203 12:29:09.451706 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/e75eea2c-2eb1-4f0f-97e5-63ce147e68dd-log-httpd\") pod \"ceilometer-0\" (UID: \"e75eea2c-2eb1-4f0f-97e5-63ce147e68dd\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:29:09 crc kubenswrapper[5052]: I1203 12:29:09.451740 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gh476\" (UniqueName: \"kubernetes.io/projected/e75eea2c-2eb1-4f0f-97e5-63ce147e68dd-kube-api-access-gh476\") pod \"ceilometer-0\" (UID: \"e75eea2c-2eb1-4f0f-97e5-63ce147e68dd\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:29:09 crc kubenswrapper[5052]: I1203 12:29:09.451831 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e75eea2c-2eb1-4f0f-97e5-63ce147e68dd-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"e75eea2c-2eb1-4f0f-97e5-63ce147e68dd\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:29:09 crc kubenswrapper[5052]: I1203 12:29:09.451867 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/e75eea2c-2eb1-4f0f-97e5-63ce147e68dd-run-httpd\") pod \"ceilometer-0\" (UID: \"e75eea2c-2eb1-4f0f-97e5-63ce147e68dd\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:29:09 crc kubenswrapper[5052]: I1203 12:29:09.452023 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/e75eea2c-2eb1-4f0f-97e5-63ce147e68dd-scripts\") pod \"ceilometer-0\" (UID: \"e75eea2c-2eb1-4f0f-97e5-63ce147e68dd\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:29:09 crc kubenswrapper[5052]: I1203 12:29:09.452051 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/e75eea2c-2eb1-4f0f-97e5-63ce147e68dd-ceilometer-tls-certs\") pod \"ceilometer-0\" (UID: \"e75eea2c-2eb1-4f0f-97e5-63ce147e68dd\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:29:09 crc kubenswrapper[5052]: I1203 12:29:09.553344 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/e75eea2c-2eb1-4f0f-97e5-63ce147e68dd-log-httpd\") pod \"ceilometer-0\" (UID: \"e75eea2c-2eb1-4f0f-97e5-63ce147e68dd\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:29:09 crc kubenswrapper[5052]: I1203 12:29:09.553400 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-gh476\" (UniqueName: \"kubernetes.io/projected/e75eea2c-2eb1-4f0f-97e5-63ce147e68dd-kube-api-access-gh476\") pod \"ceilometer-0\" (UID: \"e75eea2c-2eb1-4f0f-97e5-63ce147e68dd\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:29:09 crc kubenswrapper[5052]: I1203 12:29:09.553482 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e75eea2c-2eb1-4f0f-97e5-63ce147e68dd-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"e75eea2c-2eb1-4f0f-97e5-63ce147e68dd\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:29:09 crc kubenswrapper[5052]: I1203 12:29:09.553555 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/e75eea2c-2eb1-4f0f-97e5-63ce147e68dd-run-httpd\") pod \"ceilometer-0\" (UID: \"e75eea2c-2eb1-4f0f-97e5-63ce147e68dd\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:29:09 crc kubenswrapper[5052]: I1203 12:29:09.553591 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/e75eea2c-2eb1-4f0f-97e5-63ce147e68dd-scripts\") pod \"ceilometer-0\" (UID: \"e75eea2c-2eb1-4f0f-97e5-63ce147e68dd\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:29:09 crc kubenswrapper[5052]: I1203 12:29:09.553611 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/e75eea2c-2eb1-4f0f-97e5-63ce147e68dd-ceilometer-tls-certs\") pod \"ceilometer-0\" (UID: \"e75eea2c-2eb1-4f0f-97e5-63ce147e68dd\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:29:09 crc kubenswrapper[5052]: I1203 12:29:09.553687 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/e75eea2c-2eb1-4f0f-97e5-63ce147e68dd-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"e75eea2c-2eb1-4f0f-97e5-63ce147e68dd\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:29:09 crc kubenswrapper[5052]: I1203 12:29:09.553709 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/e75eea2c-2eb1-4f0f-97e5-63ce147e68dd-config-data\") pod \"ceilometer-0\" (UID: \"e75eea2c-2eb1-4f0f-97e5-63ce147e68dd\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:29:09 crc kubenswrapper[5052]: I1203 12:29:09.554924 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/e75eea2c-2eb1-4f0f-97e5-63ce147e68dd-run-httpd\") pod \"ceilometer-0\" (UID: \"e75eea2c-2eb1-4f0f-97e5-63ce147e68dd\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:29:09 crc kubenswrapper[5052]: I1203 12:29:09.555059 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/e75eea2c-2eb1-4f0f-97e5-63ce147e68dd-log-httpd\") pod \"ceilometer-0\" (UID: \"e75eea2c-2eb1-4f0f-97e5-63ce147e68dd\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:29:09 crc kubenswrapper[5052]: I1203 12:29:09.559834 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/e75eea2c-2eb1-4f0f-97e5-63ce147e68dd-ceilometer-tls-certs\") pod \"ceilometer-0\" (UID: \"e75eea2c-2eb1-4f0f-97e5-63ce147e68dd\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:29:09 crc kubenswrapper[5052]: I1203 12:29:09.559915 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/e75eea2c-2eb1-4f0f-97e5-63ce147e68dd-scripts\") pod \"ceilometer-0\" (UID: \"e75eea2c-2eb1-4f0f-97e5-63ce147e68dd\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:29:09 crc kubenswrapper[5052]: I1203 12:29:09.560774 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/e75eea2c-2eb1-4f0f-97e5-63ce147e68dd-config-data\") pod \"ceilometer-0\" (UID: \"e75eea2c-2eb1-4f0f-97e5-63ce147e68dd\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:29:09 crc kubenswrapper[5052]: I1203 12:29:09.560875 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e75eea2c-2eb1-4f0f-97e5-63ce147e68dd-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"e75eea2c-2eb1-4f0f-97e5-63ce147e68dd\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:29:09 crc kubenswrapper[5052]: I1203 12:29:09.561568 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/e75eea2c-2eb1-4f0f-97e5-63ce147e68dd-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"e75eea2c-2eb1-4f0f-97e5-63ce147e68dd\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:29:09 crc kubenswrapper[5052]: I1203 12:29:09.573978 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-gh476\" (UniqueName: \"kubernetes.io/projected/e75eea2c-2eb1-4f0f-97e5-63ce147e68dd-kube-api-access-gh476\") pod \"ceilometer-0\" (UID: \"e75eea2c-2eb1-4f0f-97e5-63ce147e68dd\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:29:09 crc kubenswrapper[5052]: I1203 12:29:09.692050 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a65d6de5-f108-40c9-81f6-ca37534c7710" path="/var/lib/kubelet/pods/a65d6de5-f108-40c9-81f6-ca37534c7710/volumes" Dec 03 12:29:09 crc kubenswrapper[5052]: I1203 12:29:09.714976 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:29:09 crc kubenswrapper[5052]: I1203 12:29:09.800005 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:29:09 crc kubenswrapper[5052]: I1203 12:29:09.803384 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:29:10 crc kubenswrapper[5052]: I1203 12:29:10.210063 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:29:10 crc kubenswrapper[5052]: E1203 12:29:10.296392 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="0dcdd2aec7240ab8991d19f9ba29d4cff2af7594281adff351a9d0b94735f781" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:29:10 crc kubenswrapper[5052]: E1203 12:29:10.298171 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="0dcdd2aec7240ab8991d19f9ba29d4cff2af7594281adff351a9d0b94735f781" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:29:10 crc kubenswrapper[5052]: E1203 12:29:10.300653 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="0dcdd2aec7240ab8991d19f9ba29d4cff2af7594281adff351a9d0b94735f781" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:29:10 crc kubenswrapper[5052]: E1203 12:29:10.300690 5052 prober.go:104] "Probe errored" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" probeType="Readiness" pod="openstack-kuttl-tests/nova-cell0-conductor-0" podUID="211396f3-156a-4ac5-917b-6d41c236d3c2" containerName="nova-cell0-conductor-conductor" Dec 03 12:29:10 crc kubenswrapper[5052]: I1203 12:29:10.316907 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"e75eea2c-2eb1-4f0f-97e5-63ce147e68dd","Type":"ContainerStarted","Data":"127ac4faa7dc9b9d18ac8577cb23acb88c76ba657840c8b9ce67ac14e97a34a9"} Dec 03 12:29:10 crc kubenswrapper[5052]: I1203 12:29:10.499931 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:29:10 crc kubenswrapper[5052]: I1203 12:29:10.501079 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:29:10 crc kubenswrapper[5052]: I1203 12:29:10.811661 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-api-0" podUID="b5b5ad16-1e45-4761-b5d2-60ffd5934ccb" containerName="nova-api-api" probeResult="failure" output="Get \"https://10.217.0.188:8774/\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Dec 03 12:29:10 crc kubenswrapper[5052]: I1203 12:29:10.811672 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-api-0" podUID="b5b5ad16-1e45-4761-b5d2-60ffd5934ccb" containerName="nova-api-log" probeResult="failure" output="Get \"https://10.217.0.188:8774/\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Dec 03 12:29:10 crc kubenswrapper[5052]: I1203 12:29:10.931118 5052 scope.go:117] "RemoveContainer" containerID="bfd1131e5b34bdb4dfe470823bc362987fe44180ecf47f11c1f7cf9cd07620fa" Dec 03 12:29:10 crc kubenswrapper[5052]: I1203 12:29:10.959740 5052 scope.go:117] "RemoveContainer" containerID="ea57972cc90dc839e2691fd58331a5b91823aeac8b1d3c28fef0d50b86ac8c02" Dec 03 12:29:11 crc kubenswrapper[5052]: I1203 12:29:11.013731 5052 scope.go:117] "RemoveContainer" containerID="27793877d8c79ed6ca234a512d927c37ead277dc4796a3e77a80da117b8b751a" Dec 03 12:29:11 crc kubenswrapper[5052]: I1203 12:29:11.041302 5052 scope.go:117] "RemoveContainer" containerID="d5a694b72e4270ffeb0cb46b41b69991193f18ac4340abd4834610bda1dbe60e" Dec 03 12:29:11 crc kubenswrapper[5052]: I1203 12:29:11.078756 5052 scope.go:117] "RemoveContainer" containerID="b314f4abcca774a6ec61d736d4470c651dc41b59c12c79dd5637e02b5addb43e" Dec 03 12:29:11 crc kubenswrapper[5052]: I1203 12:29:11.103463 5052 scope.go:117] "RemoveContainer" containerID="57d4da62ee25a46d86d917668db0e2040b0ddb72f3931305a731ada69c4b95f9" Dec 03 12:29:11 crc kubenswrapper[5052]: I1203 12:29:11.151804 5052 scope.go:117] "RemoveContainer" containerID="5967cae95721e1e72b8c4db4802d57efa1fd7e5f8809722a100a92f8f9b5437d" Dec 03 12:29:11 crc kubenswrapper[5052]: I1203 12:29:11.348192 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"e75eea2c-2eb1-4f0f-97e5-63ce147e68dd","Type":"ContainerStarted","Data":"a6f27ec51db1c9719c74272ae88d2ed48068153f9cc4d46a92cce3985e31c2e5"} Dec 03 12:29:11 crc kubenswrapper[5052]: I1203 12:29:11.517729 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-metadata-0" podUID="caaecb05-f5f1-421d-8e91-818774a2c0ec" containerName="nova-metadata-log" probeResult="failure" output="Get \"https://10.217.0.189:8775/\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Dec 03 12:29:11 crc kubenswrapper[5052]: I1203 12:29:11.518295 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-metadata-0" podUID="caaecb05-f5f1-421d-8e91-818774a2c0ec" containerName="nova-metadata-metadata" probeResult="failure" output="Get \"https://10.217.0.189:8775/\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Dec 03 12:29:12 crc kubenswrapper[5052]: I1203 12:29:12.359901 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"e75eea2c-2eb1-4f0f-97e5-63ce147e68dd","Type":"ContainerStarted","Data":"069e9d7b5055dc14bd89a6f68f4d0ccea363e1ff3aad490e83aff00d8a0ad411"} Dec 03 12:29:13 crc kubenswrapper[5052]: I1203 12:29:13.373012 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"e75eea2c-2eb1-4f0f-97e5-63ce147e68dd","Type":"ContainerStarted","Data":"102038f6978c676a1d8853f850be294a8aea8d6b74a6f1949c24231d714a3608"} Dec 03 12:29:13 crc kubenswrapper[5052]: E1203 12:29:13.826930 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="f0d03487bb9b0001a9b06ddae0262ca9d26eb73013d9bccc682cbc0b7a9e1bbe" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:29:13 crc kubenswrapper[5052]: E1203 12:29:13.829339 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="f0d03487bb9b0001a9b06ddae0262ca9d26eb73013d9bccc682cbc0b7a9e1bbe" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:29:13 crc kubenswrapper[5052]: E1203 12:29:13.830968 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="f0d03487bb9b0001a9b06ddae0262ca9d26eb73013d9bccc682cbc0b7a9e1bbe" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:29:13 crc kubenswrapper[5052]: E1203 12:29:13.831028 5052 prober.go:104] "Probe errored" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" probeType="Readiness" pod="openstack-kuttl-tests/nova-cell1-conductor-0" podUID="026d5ba1-5f6f-4f39-bcb6-9c51d37608f2" containerName="nova-cell1-conductor-conductor" Dec 03 12:29:14 crc kubenswrapper[5052]: I1203 12:29:14.386129 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"e75eea2c-2eb1-4f0f-97e5-63ce147e68dd","Type":"ContainerStarted","Data":"da3d10f0fb54aeb0fa5fdc57f022b12303d52fa87da0b230d95a66aac9d2be59"} Dec 03 12:29:14 crc kubenswrapper[5052]: I1203 12:29:14.386531 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:29:14 crc kubenswrapper[5052]: I1203 12:29:14.416591 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/ceilometer-0" podStartSLOduration=1.565231823 podStartE2EDuration="5.416576347s" podCreationTimestamp="2025-12-03 12:29:09 +0000 UTC" firstStartedPulling="2025-12-03 12:29:10.232681221 +0000 UTC m=+4140.820922985" lastFinishedPulling="2025-12-03 12:29:14.084025745 +0000 UTC m=+4144.672267509" observedRunningTime="2025-12-03 12:29:14.41071807 +0000 UTC m=+4144.998959834" watchObservedRunningTime="2025-12-03 12:29:14.416576347 +0000 UTC m=+4145.004818111" Dec 03 12:29:15 crc kubenswrapper[5052]: E1203 12:29:15.322434 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="0dcdd2aec7240ab8991d19f9ba29d4cff2af7594281adff351a9d0b94735f781" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:29:15 crc kubenswrapper[5052]: E1203 12:29:15.327406 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="0dcdd2aec7240ab8991d19f9ba29d4cff2af7594281adff351a9d0b94735f781" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:29:15 crc kubenswrapper[5052]: E1203 12:29:15.329138 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="0dcdd2aec7240ab8991d19f9ba29d4cff2af7594281adff351a9d0b94735f781" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:29:15 crc kubenswrapper[5052]: E1203 12:29:15.329216 5052 prober.go:104] "Probe errored" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" probeType="Readiness" pod="openstack-kuttl-tests/nova-cell0-conductor-0" podUID="211396f3-156a-4ac5-917b-6d41c236d3c2" containerName="nova-cell0-conductor-conductor" Dec 03 12:29:18 crc kubenswrapper[5052]: E1203 12:29:18.826958 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="f0d03487bb9b0001a9b06ddae0262ca9d26eb73013d9bccc682cbc0b7a9e1bbe" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:29:18 crc kubenswrapper[5052]: E1203 12:29:18.831574 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="f0d03487bb9b0001a9b06ddae0262ca9d26eb73013d9bccc682cbc0b7a9e1bbe" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:29:18 crc kubenswrapper[5052]: E1203 12:29:18.832895 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="f0d03487bb9b0001a9b06ddae0262ca9d26eb73013d9bccc682cbc0b7a9e1bbe" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:29:18 crc kubenswrapper[5052]: E1203 12:29:18.832942 5052 prober.go:104] "Probe errored" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" probeType="Readiness" pod="openstack-kuttl-tests/nova-cell1-conductor-0" podUID="026d5ba1-5f6f-4f39-bcb6-9c51d37608f2" containerName="nova-cell1-conductor-conductor" Dec 03 12:29:19 crc kubenswrapper[5052]: I1203 12:29:19.810008 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:29:19 crc kubenswrapper[5052]: I1203 12:29:19.810449 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:29:19 crc kubenswrapper[5052]: I1203 12:29:19.813119 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:29:19 crc kubenswrapper[5052]: I1203 12:29:19.857915 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:29:20 crc kubenswrapper[5052]: E1203 12:29:20.295818 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="0dcdd2aec7240ab8991d19f9ba29d4cff2af7594281adff351a9d0b94735f781" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:29:20 crc kubenswrapper[5052]: E1203 12:29:20.297442 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="0dcdd2aec7240ab8991d19f9ba29d4cff2af7594281adff351a9d0b94735f781" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:29:20 crc kubenswrapper[5052]: E1203 12:29:20.298798 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="0dcdd2aec7240ab8991d19f9ba29d4cff2af7594281adff351a9d0b94735f781" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:29:20 crc kubenswrapper[5052]: E1203 12:29:20.298877 5052 prober.go:104] "Probe errored" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" probeType="Readiness" pod="openstack-kuttl-tests/nova-cell0-conductor-0" podUID="211396f3-156a-4ac5-917b-6d41c236d3c2" containerName="nova-cell0-conductor-conductor" Dec 03 12:29:20 crc kubenswrapper[5052]: I1203 12:29:20.460747 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:29:20 crc kubenswrapper[5052]: I1203 12:29:20.468437 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:29:20 crc kubenswrapper[5052]: I1203 12:29:20.507905 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:29:20 crc kubenswrapper[5052]: I1203 12:29:20.530465 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:29:20 crc kubenswrapper[5052]: I1203 12:29:20.563646 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:29:21 crc kubenswrapper[5052]: I1203 12:29:21.190117 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:29:21 crc kubenswrapper[5052]: I1203 12:29:21.195145 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:29:21 crc kubenswrapper[5052]: I1203 12:29:21.201458 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:29:21 crc kubenswrapper[5052]: I1203 12:29:21.331758 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3a971df6-d01d-4f12-9867-a11a4a44a1ba-combined-ca-bundle\") pod \"3a971df6-d01d-4f12-9867-a11a4a44a1ba\" (UID: \"3a971df6-d01d-4f12-9867-a11a4a44a1ba\") " Dec 03 12:29:21 crc kubenswrapper[5052]: I1203 12:29:21.331896 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sd7ql\" (UniqueName: \"kubernetes.io/projected/3a971df6-d01d-4f12-9867-a11a4a44a1ba-kube-api-access-sd7ql\") pod \"3a971df6-d01d-4f12-9867-a11a4a44a1ba\" (UID: \"3a971df6-d01d-4f12-9867-a11a4a44a1ba\") " Dec 03 12:29:21 crc kubenswrapper[5052]: I1203 12:29:21.331943 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/3a971df6-d01d-4f12-9867-a11a4a44a1ba-config-data\") pod \"3a971df6-d01d-4f12-9867-a11a4a44a1ba\" (UID: \"3a971df6-d01d-4f12-9867-a11a4a44a1ba\") " Dec 03 12:29:21 crc kubenswrapper[5052]: I1203 12:29:21.332005 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"vencrypt-tls-certs\" (UniqueName: \"kubernetes.io/secret/f21db826-d275-4464-937c-113fc7ddab00-vencrypt-tls-certs\") pod \"f21db826-d275-4464-937c-113fc7ddab00\" (UID: \"f21db826-d275-4464-937c-113fc7ddab00\") " Dec 03 12:29:21 crc kubenswrapper[5052]: I1203 12:29:21.332036 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"nova-novncproxy-tls-certs\" (UniqueName: \"kubernetes.io/secret/f21db826-d275-4464-937c-113fc7ddab00-nova-novncproxy-tls-certs\") pod \"f21db826-d275-4464-937c-113fc7ddab00\" (UID: \"f21db826-d275-4464-937c-113fc7ddab00\") " Dec 03 12:29:21 crc kubenswrapper[5052]: I1203 12:29:21.332066 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/026d5ba1-5f6f-4f39-bcb6-9c51d37608f2-combined-ca-bundle\") pod \"026d5ba1-5f6f-4f39-bcb6-9c51d37608f2\" (UID: \"026d5ba1-5f6f-4f39-bcb6-9c51d37608f2\") " Dec 03 12:29:21 crc kubenswrapper[5052]: I1203 12:29:21.332151 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/f21db826-d275-4464-937c-113fc7ddab00-config-data\") pod \"f21db826-d275-4464-937c-113fc7ddab00\" (UID: \"f21db826-d275-4464-937c-113fc7ddab00\") " Dec 03 12:29:21 crc kubenswrapper[5052]: I1203 12:29:21.332173 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mh9xd\" (UniqueName: \"kubernetes.io/projected/f21db826-d275-4464-937c-113fc7ddab00-kube-api-access-mh9xd\") pod \"f21db826-d275-4464-937c-113fc7ddab00\" (UID: \"f21db826-d275-4464-937c-113fc7ddab00\") " Dec 03 12:29:21 crc kubenswrapper[5052]: I1203 12:29:21.332223 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/026d5ba1-5f6f-4f39-bcb6-9c51d37608f2-config-data\") pod \"026d5ba1-5f6f-4f39-bcb6-9c51d37608f2\" (UID: \"026d5ba1-5f6f-4f39-bcb6-9c51d37608f2\") " Dec 03 12:29:21 crc kubenswrapper[5052]: I1203 12:29:21.332825 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f21db826-d275-4464-937c-113fc7ddab00-combined-ca-bundle\") pod \"f21db826-d275-4464-937c-113fc7ddab00\" (UID: \"f21db826-d275-4464-937c-113fc7ddab00\") " Dec 03 12:29:21 crc kubenswrapper[5052]: I1203 12:29:21.332899 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cpggs\" (UniqueName: \"kubernetes.io/projected/026d5ba1-5f6f-4f39-bcb6-9c51d37608f2-kube-api-access-cpggs\") pod \"026d5ba1-5f6f-4f39-bcb6-9c51d37608f2\" (UID: \"026d5ba1-5f6f-4f39-bcb6-9c51d37608f2\") " Dec 03 12:29:21 crc kubenswrapper[5052]: I1203 12:29:21.337759 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3a971df6-d01d-4f12-9867-a11a4a44a1ba-kube-api-access-sd7ql" (OuterVolumeSpecName: "kube-api-access-sd7ql") pod "3a971df6-d01d-4f12-9867-a11a4a44a1ba" (UID: "3a971df6-d01d-4f12-9867-a11a4a44a1ba"). InnerVolumeSpecName "kube-api-access-sd7ql". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:29:21 crc kubenswrapper[5052]: I1203 12:29:21.337833 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/026d5ba1-5f6f-4f39-bcb6-9c51d37608f2-kube-api-access-cpggs" (OuterVolumeSpecName: "kube-api-access-cpggs") pod "026d5ba1-5f6f-4f39-bcb6-9c51d37608f2" (UID: "026d5ba1-5f6f-4f39-bcb6-9c51d37608f2"). InnerVolumeSpecName "kube-api-access-cpggs". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:29:21 crc kubenswrapper[5052]: I1203 12:29:21.338001 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f21db826-d275-4464-937c-113fc7ddab00-kube-api-access-mh9xd" (OuterVolumeSpecName: "kube-api-access-mh9xd") pod "f21db826-d275-4464-937c-113fc7ddab00" (UID: "f21db826-d275-4464-937c-113fc7ddab00"). InnerVolumeSpecName "kube-api-access-mh9xd". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:29:21 crc kubenswrapper[5052]: I1203 12:29:21.359598 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3a971df6-d01d-4f12-9867-a11a4a44a1ba-config-data" (OuterVolumeSpecName: "config-data") pod "3a971df6-d01d-4f12-9867-a11a4a44a1ba" (UID: "3a971df6-d01d-4f12-9867-a11a4a44a1ba"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:29:21 crc kubenswrapper[5052]: I1203 12:29:21.362127 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f21db826-d275-4464-937c-113fc7ddab00-config-data" (OuterVolumeSpecName: "config-data") pod "f21db826-d275-4464-937c-113fc7ddab00" (UID: "f21db826-d275-4464-937c-113fc7ddab00"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:29:21 crc kubenswrapper[5052]: I1203 12:29:21.366188 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f21db826-d275-4464-937c-113fc7ddab00-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "f21db826-d275-4464-937c-113fc7ddab00" (UID: "f21db826-d275-4464-937c-113fc7ddab00"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:29:21 crc kubenswrapper[5052]: I1203 12:29:21.368664 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3a971df6-d01d-4f12-9867-a11a4a44a1ba-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "3a971df6-d01d-4f12-9867-a11a4a44a1ba" (UID: "3a971df6-d01d-4f12-9867-a11a4a44a1ba"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:29:21 crc kubenswrapper[5052]: I1203 12:29:21.368741 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/026d5ba1-5f6f-4f39-bcb6-9c51d37608f2-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "026d5ba1-5f6f-4f39-bcb6-9c51d37608f2" (UID: "026d5ba1-5f6f-4f39-bcb6-9c51d37608f2"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:29:21 crc kubenswrapper[5052]: I1203 12:29:21.378968 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/026d5ba1-5f6f-4f39-bcb6-9c51d37608f2-config-data" (OuterVolumeSpecName: "config-data") pod "026d5ba1-5f6f-4f39-bcb6-9c51d37608f2" (UID: "026d5ba1-5f6f-4f39-bcb6-9c51d37608f2"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:29:21 crc kubenswrapper[5052]: I1203 12:29:21.386844 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f21db826-d275-4464-937c-113fc7ddab00-vencrypt-tls-certs" (OuterVolumeSpecName: "vencrypt-tls-certs") pod "f21db826-d275-4464-937c-113fc7ddab00" (UID: "f21db826-d275-4464-937c-113fc7ddab00"). InnerVolumeSpecName "vencrypt-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:29:21 crc kubenswrapper[5052]: I1203 12:29:21.391442 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f21db826-d275-4464-937c-113fc7ddab00-nova-novncproxy-tls-certs" (OuterVolumeSpecName: "nova-novncproxy-tls-certs") pod "f21db826-d275-4464-937c-113fc7ddab00" (UID: "f21db826-d275-4464-937c-113fc7ddab00"). InnerVolumeSpecName "nova-novncproxy-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:29:21 crc kubenswrapper[5052]: I1203 12:29:21.435710 5052 reconciler_common.go:293] "Volume detached for volume \"vencrypt-tls-certs\" (UniqueName: \"kubernetes.io/secret/f21db826-d275-4464-937c-113fc7ddab00-vencrypt-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:29:21 crc kubenswrapper[5052]: I1203 12:29:21.435749 5052 reconciler_common.go:293] "Volume detached for volume \"nova-novncproxy-tls-certs\" (UniqueName: \"kubernetes.io/secret/f21db826-d275-4464-937c-113fc7ddab00-nova-novncproxy-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:29:21 crc kubenswrapper[5052]: I1203 12:29:21.435764 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/026d5ba1-5f6f-4f39-bcb6-9c51d37608f2-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:29:21 crc kubenswrapper[5052]: I1203 12:29:21.435777 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/f21db826-d275-4464-937c-113fc7ddab00-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:29:21 crc kubenswrapper[5052]: I1203 12:29:21.435790 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-mh9xd\" (UniqueName: \"kubernetes.io/projected/f21db826-d275-4464-937c-113fc7ddab00-kube-api-access-mh9xd\") on node \"crc\" DevicePath \"\"" Dec 03 12:29:21 crc kubenswrapper[5052]: I1203 12:29:21.435801 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/026d5ba1-5f6f-4f39-bcb6-9c51d37608f2-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:29:21 crc kubenswrapper[5052]: I1203 12:29:21.435813 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f21db826-d275-4464-937c-113fc7ddab00-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:29:21 crc kubenswrapper[5052]: I1203 12:29:21.435826 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-cpggs\" (UniqueName: \"kubernetes.io/projected/026d5ba1-5f6f-4f39-bcb6-9c51d37608f2-kube-api-access-cpggs\") on node \"crc\" DevicePath \"\"" Dec 03 12:29:21 crc kubenswrapper[5052]: I1203 12:29:21.435838 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3a971df6-d01d-4f12-9867-a11a4a44a1ba-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:29:21 crc kubenswrapper[5052]: I1203 12:29:21.435849 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-sd7ql\" (UniqueName: \"kubernetes.io/projected/3a971df6-d01d-4f12-9867-a11a4a44a1ba-kube-api-access-sd7ql\") on node \"crc\" DevicePath \"\"" Dec 03 12:29:21 crc kubenswrapper[5052]: I1203 12:29:21.435860 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/3a971df6-d01d-4f12-9867-a11a4a44a1ba-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:29:21 crc kubenswrapper[5052]: I1203 12:29:21.470679 5052 generic.go:334] "Generic (PLEG): container finished" podID="026d5ba1-5f6f-4f39-bcb6-9c51d37608f2" containerID="f0d03487bb9b0001a9b06ddae0262ca9d26eb73013d9bccc682cbc0b7a9e1bbe" exitCode=137 Dec 03 12:29:21 crc kubenswrapper[5052]: I1203 12:29:21.470729 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:29:21 crc kubenswrapper[5052]: I1203 12:29:21.470739 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-conductor-0" event={"ID":"026d5ba1-5f6f-4f39-bcb6-9c51d37608f2","Type":"ContainerDied","Data":"f0d03487bb9b0001a9b06ddae0262ca9d26eb73013d9bccc682cbc0b7a9e1bbe"} Dec 03 12:29:21 crc kubenswrapper[5052]: I1203 12:29:21.470764 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-conductor-0" event={"ID":"026d5ba1-5f6f-4f39-bcb6-9c51d37608f2","Type":"ContainerDied","Data":"e00e8647a336fdd69de7a88c71814a98d53ec41e92ad473debb2ba71e491f11c"} Dec 03 12:29:21 crc kubenswrapper[5052]: I1203 12:29:21.470780 5052 scope.go:117] "RemoveContainer" containerID="f0d03487bb9b0001a9b06ddae0262ca9d26eb73013d9bccc682cbc0b7a9e1bbe" Dec 03 12:29:21 crc kubenswrapper[5052]: I1203 12:29:21.474957 5052 generic.go:334] "Generic (PLEG): container finished" podID="f21db826-d275-4464-937c-113fc7ddab00" containerID="458afe5396f5d654ae2179515a4f44886989891f21a1c00342344e6ced981d50" exitCode=137 Dec 03 12:29:21 crc kubenswrapper[5052]: I1203 12:29:21.476066 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:29:21 crc kubenswrapper[5052]: I1203 12:29:21.477972 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" event={"ID":"f21db826-d275-4464-937c-113fc7ddab00","Type":"ContainerDied","Data":"458afe5396f5d654ae2179515a4f44886989891f21a1c00342344e6ced981d50"} Dec 03 12:29:21 crc kubenswrapper[5052]: I1203 12:29:21.478013 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" event={"ID":"f21db826-d275-4464-937c-113fc7ddab00","Type":"ContainerDied","Data":"4301a172b5076b2f8d49cc867c7ad17dac5888bd3fd98f2a97eca78fff4e2d33"} Dec 03 12:29:21 crc kubenswrapper[5052]: I1203 12:29:21.481107 5052 generic.go:334] "Generic (PLEG): container finished" podID="3a971df6-d01d-4f12-9867-a11a4a44a1ba" containerID="1cbec8b86689f5bf35e48f86fab05f7750b27f7dd4f48552f1d8c25b1c37adb2" exitCode=137 Dec 03 12:29:21 crc kubenswrapper[5052]: I1203 12:29:21.481179 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"3a971df6-d01d-4f12-9867-a11a4a44a1ba","Type":"ContainerDied","Data":"1cbec8b86689f5bf35e48f86fab05f7750b27f7dd4f48552f1d8c25b1c37adb2"} Dec 03 12:29:21 crc kubenswrapper[5052]: I1203 12:29:21.481240 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"3a971df6-d01d-4f12-9867-a11a4a44a1ba","Type":"ContainerDied","Data":"cbe88589a81e849f6f0daa7ff8ef7a534e510717cf1dea7a764f871758401466"} Dec 03 12:29:21 crc kubenswrapper[5052]: I1203 12:29:21.481385 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:29:21 crc kubenswrapper[5052]: I1203 12:29:21.487238 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:29:21 crc kubenswrapper[5052]: I1203 12:29:21.506785 5052 scope.go:117] "RemoveContainer" containerID="f0d03487bb9b0001a9b06ddae0262ca9d26eb73013d9bccc682cbc0b7a9e1bbe" Dec 03 12:29:21 crc kubenswrapper[5052]: E1203 12:29:21.507137 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f0d03487bb9b0001a9b06ddae0262ca9d26eb73013d9bccc682cbc0b7a9e1bbe\": container with ID starting with f0d03487bb9b0001a9b06ddae0262ca9d26eb73013d9bccc682cbc0b7a9e1bbe not found: ID does not exist" containerID="f0d03487bb9b0001a9b06ddae0262ca9d26eb73013d9bccc682cbc0b7a9e1bbe" Dec 03 12:29:21 crc kubenswrapper[5052]: I1203 12:29:21.507171 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f0d03487bb9b0001a9b06ddae0262ca9d26eb73013d9bccc682cbc0b7a9e1bbe"} err="failed to get container status \"f0d03487bb9b0001a9b06ddae0262ca9d26eb73013d9bccc682cbc0b7a9e1bbe\": rpc error: code = NotFound desc = could not find container \"f0d03487bb9b0001a9b06ddae0262ca9d26eb73013d9bccc682cbc0b7a9e1bbe\": container with ID starting with f0d03487bb9b0001a9b06ddae0262ca9d26eb73013d9bccc682cbc0b7a9e1bbe not found: ID does not exist" Dec 03 12:29:21 crc kubenswrapper[5052]: I1203 12:29:21.507241 5052 scope.go:117] "RemoveContainer" containerID="458afe5396f5d654ae2179515a4f44886989891f21a1c00342344e6ced981d50" Dec 03 12:29:21 crc kubenswrapper[5052]: I1203 12:29:21.557594 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 12:29:21 crc kubenswrapper[5052]: I1203 12:29:21.568207 5052 scope.go:117] "RemoveContainer" containerID="458afe5396f5d654ae2179515a4f44886989891f21a1c00342344e6ced981d50" Dec 03 12:29:21 crc kubenswrapper[5052]: E1203 12:29:21.577905 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"458afe5396f5d654ae2179515a4f44886989891f21a1c00342344e6ced981d50\": container with ID starting with 458afe5396f5d654ae2179515a4f44886989891f21a1c00342344e6ced981d50 not found: ID does not exist" containerID="458afe5396f5d654ae2179515a4f44886989891f21a1c00342344e6ced981d50" Dec 03 12:29:21 crc kubenswrapper[5052]: I1203 12:29:21.577956 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"458afe5396f5d654ae2179515a4f44886989891f21a1c00342344e6ced981d50"} err="failed to get container status \"458afe5396f5d654ae2179515a4f44886989891f21a1c00342344e6ced981d50\": rpc error: code = NotFound desc = could not find container \"458afe5396f5d654ae2179515a4f44886989891f21a1c00342344e6ced981d50\": container with ID starting with 458afe5396f5d654ae2179515a4f44886989891f21a1c00342344e6ced981d50 not found: ID does not exist" Dec 03 12:29:21 crc kubenswrapper[5052]: I1203 12:29:21.577986 5052 scope.go:117] "RemoveContainer" containerID="1cbec8b86689f5bf35e48f86fab05f7750b27f7dd4f48552f1d8c25b1c37adb2" Dec 03 12:29:21 crc kubenswrapper[5052]: I1203 12:29:21.591939 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 12:29:21 crc kubenswrapper[5052]: I1203 12:29:21.605611 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:29:21 crc kubenswrapper[5052]: I1203 12:29:21.640355 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:29:21 crc kubenswrapper[5052]: I1203 12:29:21.669748 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 12:29:21 crc kubenswrapper[5052]: E1203 12:29:21.670443 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="3a971df6-d01d-4f12-9867-a11a4a44a1ba" containerName="nova-scheduler-scheduler" Dec 03 12:29:21 crc kubenswrapper[5052]: I1203 12:29:21.670464 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="3a971df6-d01d-4f12-9867-a11a4a44a1ba" containerName="nova-scheduler-scheduler" Dec 03 12:29:21 crc kubenswrapper[5052]: E1203 12:29:21.670532 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="026d5ba1-5f6f-4f39-bcb6-9c51d37608f2" containerName="nova-cell1-conductor-conductor" Dec 03 12:29:21 crc kubenswrapper[5052]: I1203 12:29:21.670543 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="026d5ba1-5f6f-4f39-bcb6-9c51d37608f2" containerName="nova-cell1-conductor-conductor" Dec 03 12:29:21 crc kubenswrapper[5052]: E1203 12:29:21.670568 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="f21db826-d275-4464-937c-113fc7ddab00" containerName="nova-cell1-novncproxy-novncproxy" Dec 03 12:29:21 crc kubenswrapper[5052]: I1203 12:29:21.670602 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="f21db826-d275-4464-937c-113fc7ddab00" containerName="nova-cell1-novncproxy-novncproxy" Dec 03 12:29:21 crc kubenswrapper[5052]: I1203 12:29:21.670864 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="3a971df6-d01d-4f12-9867-a11a4a44a1ba" containerName="nova-scheduler-scheduler" Dec 03 12:29:21 crc kubenswrapper[5052]: I1203 12:29:21.670894 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="f21db826-d275-4464-937c-113fc7ddab00" containerName="nova-cell1-novncproxy-novncproxy" Dec 03 12:29:21 crc kubenswrapper[5052]: I1203 12:29:21.670910 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="026d5ba1-5f6f-4f39-bcb6-9c51d37608f2" containerName="nova-cell1-conductor-conductor" Dec 03 12:29:21 crc kubenswrapper[5052]: I1203 12:29:21.671723 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:29:21 crc kubenswrapper[5052]: I1203 12:29:21.676404 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-nova-novncproxy-cell1-vencrypt" Dec 03 12:29:21 crc kubenswrapper[5052]: I1203 12:29:21.676720 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell1-novncproxy-config-data" Dec 03 12:29:21 crc kubenswrapper[5052]: I1203 12:29:21.676795 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-nova-novncproxy-cell1-public-svc" Dec 03 12:29:21 crc kubenswrapper[5052]: I1203 12:29:21.680089 5052 scope.go:117] "RemoveContainer" containerID="1cbec8b86689f5bf35e48f86fab05f7750b27f7dd4f48552f1d8c25b1c37adb2" Dec 03 12:29:21 crc kubenswrapper[5052]: E1203 12:29:21.680447 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1cbec8b86689f5bf35e48f86fab05f7750b27f7dd4f48552f1d8c25b1c37adb2\": container with ID starting with 1cbec8b86689f5bf35e48f86fab05f7750b27f7dd4f48552f1d8c25b1c37adb2 not found: ID does not exist" containerID="1cbec8b86689f5bf35e48f86fab05f7750b27f7dd4f48552f1d8c25b1c37adb2" Dec 03 12:29:21 crc kubenswrapper[5052]: I1203 12:29:21.680538 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1cbec8b86689f5bf35e48f86fab05f7750b27f7dd4f48552f1d8c25b1c37adb2"} err="failed to get container status \"1cbec8b86689f5bf35e48f86fab05f7750b27f7dd4f48552f1d8c25b1c37adb2\": rpc error: code = NotFound desc = could not find container \"1cbec8b86689f5bf35e48f86fab05f7750b27f7dd4f48552f1d8c25b1c37adb2\": container with ID starting with 1cbec8b86689f5bf35e48f86fab05f7750b27f7dd4f48552f1d8c25b1c37adb2 not found: ID does not exist" Dec 03 12:29:21 crc kubenswrapper[5052]: I1203 12:29:21.697735 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3a971df6-d01d-4f12-9867-a11a4a44a1ba" path="/var/lib/kubelet/pods/3a971df6-d01d-4f12-9867-a11a4a44a1ba/volumes" Dec 03 12:29:21 crc kubenswrapper[5052]: I1203 12:29:21.698391 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f21db826-d275-4464-937c-113fc7ddab00" path="/var/lib/kubelet/pods/f21db826-d275-4464-937c-113fc7ddab00/volumes" Dec 03 12:29:21 crc kubenswrapper[5052]: I1203 12:29:21.699112 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-0"] Dec 03 12:29:21 crc kubenswrapper[5052]: I1203 12:29:21.706063 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:29:21 crc kubenswrapper[5052]: I1203 12:29:21.707729 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:29:21 crc kubenswrapper[5052]: I1203 12:29:21.709891 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-scheduler-config-data" Dec 03 12:29:21 crc kubenswrapper[5052]: I1203 12:29:21.717912 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-0"] Dec 03 12:29:21 crc kubenswrapper[5052]: I1203 12:29:21.732757 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 12:29:21 crc kubenswrapper[5052]: I1203 12:29:21.743063 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:29:21 crc kubenswrapper[5052]: I1203 12:29:21.752453 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-0"] Dec 03 12:29:21 crc kubenswrapper[5052]: I1203 12:29:21.753873 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:29:21 crc kubenswrapper[5052]: I1203 12:29:21.756598 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell1-conductor-config-data" Dec 03 12:29:21 crc kubenswrapper[5052]: I1203 12:29:21.769347 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-0"] Dec 03 12:29:21 crc kubenswrapper[5052]: I1203 12:29:21.848601 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/733f463b-dcf8-4a61-8e4e-4bf1c042d397-config-data\") pod \"nova-cell1-conductor-0\" (UID: \"733f463b-dcf8-4a61-8e4e-4bf1c042d397\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:29:21 crc kubenswrapper[5052]: I1203 12:29:21.849593 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"vencrypt-tls-certs\" (UniqueName: \"kubernetes.io/secret/53193ff3-1a41-4e26-b38f-957088aa7e8b-vencrypt-tls-certs\") pod \"nova-cell1-novncproxy-0\" (UID: \"53193ff3-1a41-4e26-b38f-957088aa7e8b\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:29:21 crc kubenswrapper[5052]: I1203 12:29:21.849771 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nova-novncproxy-tls-certs\" (UniqueName: \"kubernetes.io/secret/53193ff3-1a41-4e26-b38f-957088aa7e8b-nova-novncproxy-tls-certs\") pod \"nova-cell1-novncproxy-0\" (UID: \"53193ff3-1a41-4e26-b38f-957088aa7e8b\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:29:21 crc kubenswrapper[5052]: I1203 12:29:21.849879 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zwjfg\" (UniqueName: \"kubernetes.io/projected/733f463b-dcf8-4a61-8e4e-4bf1c042d397-kube-api-access-zwjfg\") pod \"nova-cell1-conductor-0\" (UID: \"733f463b-dcf8-4a61-8e4e-4bf1c042d397\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:29:21 crc kubenswrapper[5052]: I1203 12:29:21.850071 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/53193ff3-1a41-4e26-b38f-957088aa7e8b-combined-ca-bundle\") pod \"nova-cell1-novncproxy-0\" (UID: \"53193ff3-1a41-4e26-b38f-957088aa7e8b\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:29:21 crc kubenswrapper[5052]: I1203 12:29:21.850200 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/733f463b-dcf8-4a61-8e4e-4bf1c042d397-combined-ca-bundle\") pod \"nova-cell1-conductor-0\" (UID: \"733f463b-dcf8-4a61-8e4e-4bf1c042d397\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:29:21 crc kubenswrapper[5052]: I1203 12:29:21.850329 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bjgf2\" (UniqueName: \"kubernetes.io/projected/b953ffa0-088c-4450-8d60-04044b33cfa0-kube-api-access-bjgf2\") pod \"nova-scheduler-0\" (UID: \"b953ffa0-088c-4450-8d60-04044b33cfa0\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:29:21 crc kubenswrapper[5052]: I1203 12:29:21.850547 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/b953ffa0-088c-4450-8d60-04044b33cfa0-config-data\") pod \"nova-scheduler-0\" (UID: \"b953ffa0-088c-4450-8d60-04044b33cfa0\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:29:21 crc kubenswrapper[5052]: I1203 12:29:21.850712 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rzgf2\" (UniqueName: \"kubernetes.io/projected/53193ff3-1a41-4e26-b38f-957088aa7e8b-kube-api-access-rzgf2\") pod \"nova-cell1-novncproxy-0\" (UID: \"53193ff3-1a41-4e26-b38f-957088aa7e8b\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:29:21 crc kubenswrapper[5052]: I1203 12:29:21.850851 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/53193ff3-1a41-4e26-b38f-957088aa7e8b-config-data\") pod \"nova-cell1-novncproxy-0\" (UID: \"53193ff3-1a41-4e26-b38f-957088aa7e8b\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:29:21 crc kubenswrapper[5052]: I1203 12:29:21.851027 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b953ffa0-088c-4450-8d60-04044b33cfa0-combined-ca-bundle\") pod \"nova-scheduler-0\" (UID: \"b953ffa0-088c-4450-8d60-04044b33cfa0\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:29:21 crc kubenswrapper[5052]: E1203 12:29:21.924494 5052 cadvisor_stats_provider.go:516] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod211396f3_156a_4ac5_917b_6d41c236d3c2.slice/crio-conmon-0dcdd2aec7240ab8991d19f9ba29d4cff2af7594281adff351a9d0b94735f781.scope\": RecentStats: unable to find data in memory cache]" Dec 03 12:29:21 crc kubenswrapper[5052]: I1203 12:29:21.953355 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/53193ff3-1a41-4e26-b38f-957088aa7e8b-combined-ca-bundle\") pod \"nova-cell1-novncproxy-0\" (UID: \"53193ff3-1a41-4e26-b38f-957088aa7e8b\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:29:21 crc kubenswrapper[5052]: I1203 12:29:21.953459 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/733f463b-dcf8-4a61-8e4e-4bf1c042d397-combined-ca-bundle\") pod \"nova-cell1-conductor-0\" (UID: \"733f463b-dcf8-4a61-8e4e-4bf1c042d397\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:29:21 crc kubenswrapper[5052]: I1203 12:29:21.953538 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-bjgf2\" (UniqueName: \"kubernetes.io/projected/b953ffa0-088c-4450-8d60-04044b33cfa0-kube-api-access-bjgf2\") pod \"nova-scheduler-0\" (UID: \"b953ffa0-088c-4450-8d60-04044b33cfa0\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:29:21 crc kubenswrapper[5052]: I1203 12:29:21.953583 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/b953ffa0-088c-4450-8d60-04044b33cfa0-config-data\") pod \"nova-scheduler-0\" (UID: \"b953ffa0-088c-4450-8d60-04044b33cfa0\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:29:21 crc kubenswrapper[5052]: I1203 12:29:21.953622 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-rzgf2\" (UniqueName: \"kubernetes.io/projected/53193ff3-1a41-4e26-b38f-957088aa7e8b-kube-api-access-rzgf2\") pod \"nova-cell1-novncproxy-0\" (UID: \"53193ff3-1a41-4e26-b38f-957088aa7e8b\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:29:21 crc kubenswrapper[5052]: I1203 12:29:21.953662 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/53193ff3-1a41-4e26-b38f-957088aa7e8b-config-data\") pod \"nova-cell1-novncproxy-0\" (UID: \"53193ff3-1a41-4e26-b38f-957088aa7e8b\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:29:21 crc kubenswrapper[5052]: I1203 12:29:21.953707 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b953ffa0-088c-4450-8d60-04044b33cfa0-combined-ca-bundle\") pod \"nova-scheduler-0\" (UID: \"b953ffa0-088c-4450-8d60-04044b33cfa0\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:29:21 crc kubenswrapper[5052]: I1203 12:29:21.953792 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/733f463b-dcf8-4a61-8e4e-4bf1c042d397-config-data\") pod \"nova-cell1-conductor-0\" (UID: \"733f463b-dcf8-4a61-8e4e-4bf1c042d397\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:29:21 crc kubenswrapper[5052]: I1203 12:29:21.953824 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"vencrypt-tls-certs\" (UniqueName: \"kubernetes.io/secret/53193ff3-1a41-4e26-b38f-957088aa7e8b-vencrypt-tls-certs\") pod \"nova-cell1-novncproxy-0\" (UID: \"53193ff3-1a41-4e26-b38f-957088aa7e8b\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:29:21 crc kubenswrapper[5052]: I1203 12:29:21.953867 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"nova-novncproxy-tls-certs\" (UniqueName: \"kubernetes.io/secret/53193ff3-1a41-4e26-b38f-957088aa7e8b-nova-novncproxy-tls-certs\") pod \"nova-cell1-novncproxy-0\" (UID: \"53193ff3-1a41-4e26-b38f-957088aa7e8b\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:29:21 crc kubenswrapper[5052]: I1203 12:29:21.953893 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-zwjfg\" (UniqueName: \"kubernetes.io/projected/733f463b-dcf8-4a61-8e4e-4bf1c042d397-kube-api-access-zwjfg\") pod \"nova-cell1-conductor-0\" (UID: \"733f463b-dcf8-4a61-8e4e-4bf1c042d397\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:29:21 crc kubenswrapper[5052]: I1203 12:29:21.959962 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/53193ff3-1a41-4e26-b38f-957088aa7e8b-combined-ca-bundle\") pod \"nova-cell1-novncproxy-0\" (UID: \"53193ff3-1a41-4e26-b38f-957088aa7e8b\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:29:21 crc kubenswrapper[5052]: I1203 12:29:21.963111 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/733f463b-dcf8-4a61-8e4e-4bf1c042d397-config-data\") pod \"nova-cell1-conductor-0\" (UID: \"733f463b-dcf8-4a61-8e4e-4bf1c042d397\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:29:21 crc kubenswrapper[5052]: I1203 12:29:21.966945 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"vencrypt-tls-certs\" (UniqueName: \"kubernetes.io/secret/53193ff3-1a41-4e26-b38f-957088aa7e8b-vencrypt-tls-certs\") pod \"nova-cell1-novncproxy-0\" (UID: \"53193ff3-1a41-4e26-b38f-957088aa7e8b\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:29:21 crc kubenswrapper[5052]: I1203 12:29:21.967730 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"nova-novncproxy-tls-certs\" (UniqueName: \"kubernetes.io/secret/53193ff3-1a41-4e26-b38f-957088aa7e8b-nova-novncproxy-tls-certs\") pod \"nova-cell1-novncproxy-0\" (UID: \"53193ff3-1a41-4e26-b38f-957088aa7e8b\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:29:21 crc kubenswrapper[5052]: I1203 12:29:21.969395 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/53193ff3-1a41-4e26-b38f-957088aa7e8b-config-data\") pod \"nova-cell1-novncproxy-0\" (UID: \"53193ff3-1a41-4e26-b38f-957088aa7e8b\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:29:21 crc kubenswrapper[5052]: I1203 12:29:21.970701 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/b953ffa0-088c-4450-8d60-04044b33cfa0-config-data\") pod \"nova-scheduler-0\" (UID: \"b953ffa0-088c-4450-8d60-04044b33cfa0\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:29:21 crc kubenswrapper[5052]: I1203 12:29:21.973417 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/733f463b-dcf8-4a61-8e4e-4bf1c042d397-combined-ca-bundle\") pod \"nova-cell1-conductor-0\" (UID: \"733f463b-dcf8-4a61-8e4e-4bf1c042d397\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:29:21 crc kubenswrapper[5052]: I1203 12:29:21.975037 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-rzgf2\" (UniqueName: \"kubernetes.io/projected/53193ff3-1a41-4e26-b38f-957088aa7e8b-kube-api-access-rzgf2\") pod \"nova-cell1-novncproxy-0\" (UID: \"53193ff3-1a41-4e26-b38f-957088aa7e8b\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:29:21 crc kubenswrapper[5052]: I1203 12:29:21.977629 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b953ffa0-088c-4450-8d60-04044b33cfa0-combined-ca-bundle\") pod \"nova-scheduler-0\" (UID: \"b953ffa0-088c-4450-8d60-04044b33cfa0\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:29:21 crc kubenswrapper[5052]: I1203 12:29:21.979404 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-bjgf2\" (UniqueName: \"kubernetes.io/projected/b953ffa0-088c-4450-8d60-04044b33cfa0-kube-api-access-bjgf2\") pod \"nova-scheduler-0\" (UID: \"b953ffa0-088c-4450-8d60-04044b33cfa0\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:29:21 crc kubenswrapper[5052]: I1203 12:29:21.980052 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-zwjfg\" (UniqueName: \"kubernetes.io/projected/733f463b-dcf8-4a61-8e4e-4bf1c042d397-kube-api-access-zwjfg\") pod \"nova-cell1-conductor-0\" (UID: \"733f463b-dcf8-4a61-8e4e-4bf1c042d397\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:29:21 crc kubenswrapper[5052]: I1203 12:29:21.993122 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:29:22 crc kubenswrapper[5052]: I1203 12:29:22.103276 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:29:22 crc kubenswrapper[5052]: I1203 12:29:22.123950 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:29:22 crc kubenswrapper[5052]: I1203 12:29:22.132647 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:29:22 crc kubenswrapper[5052]: I1203 12:29:22.259147 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/211396f3-156a-4ac5-917b-6d41c236d3c2-combined-ca-bundle\") pod \"211396f3-156a-4ac5-917b-6d41c236d3c2\" (UID: \"211396f3-156a-4ac5-917b-6d41c236d3c2\") " Dec 03 12:29:22 crc kubenswrapper[5052]: I1203 12:29:22.260350 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fgwwj\" (UniqueName: \"kubernetes.io/projected/211396f3-156a-4ac5-917b-6d41c236d3c2-kube-api-access-fgwwj\") pod \"211396f3-156a-4ac5-917b-6d41c236d3c2\" (UID: \"211396f3-156a-4ac5-917b-6d41c236d3c2\") " Dec 03 12:29:22 crc kubenswrapper[5052]: I1203 12:29:22.260599 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/211396f3-156a-4ac5-917b-6d41c236d3c2-config-data\") pod \"211396f3-156a-4ac5-917b-6d41c236d3c2\" (UID: \"211396f3-156a-4ac5-917b-6d41c236d3c2\") " Dec 03 12:29:22 crc kubenswrapper[5052]: I1203 12:29:22.265926 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/211396f3-156a-4ac5-917b-6d41c236d3c2-kube-api-access-fgwwj" (OuterVolumeSpecName: "kube-api-access-fgwwj") pod "211396f3-156a-4ac5-917b-6d41c236d3c2" (UID: "211396f3-156a-4ac5-917b-6d41c236d3c2"). InnerVolumeSpecName "kube-api-access-fgwwj". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:29:22 crc kubenswrapper[5052]: I1203 12:29:22.291706 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/211396f3-156a-4ac5-917b-6d41c236d3c2-config-data" (OuterVolumeSpecName: "config-data") pod "211396f3-156a-4ac5-917b-6d41c236d3c2" (UID: "211396f3-156a-4ac5-917b-6d41c236d3c2"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:29:22 crc kubenswrapper[5052]: I1203 12:29:22.292709 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/211396f3-156a-4ac5-917b-6d41c236d3c2-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "211396f3-156a-4ac5-917b-6d41c236d3c2" (UID: "211396f3-156a-4ac5-917b-6d41c236d3c2"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:29:22 crc kubenswrapper[5052]: I1203 12:29:22.362689 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/211396f3-156a-4ac5-917b-6d41c236d3c2-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:29:22 crc kubenswrapper[5052]: I1203 12:29:22.362738 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/211396f3-156a-4ac5-917b-6d41c236d3c2-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:29:22 crc kubenswrapper[5052]: I1203 12:29:22.362755 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-fgwwj\" (UniqueName: \"kubernetes.io/projected/211396f3-156a-4ac5-917b-6d41c236d3c2-kube-api-access-fgwwj\") on node \"crc\" DevicePath \"\"" Dec 03 12:29:22 crc kubenswrapper[5052]: I1203 12:29:22.455877 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 12:29:22 crc kubenswrapper[5052]: W1203 12:29:22.458148 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod53193ff3_1a41_4e26_b38f_957088aa7e8b.slice/crio-7aa40d31e35ffdc4b2176768f89abf61eaa08b7b0c8e93d4f2c0bd090c30d97f WatchSource:0}: Error finding container 7aa40d31e35ffdc4b2176768f89abf61eaa08b7b0c8e93d4f2c0bd090c30d97f: Status 404 returned error can't find the container with id 7aa40d31e35ffdc4b2176768f89abf61eaa08b7b0c8e93d4f2c0bd090c30d97f Dec 03 12:29:22 crc kubenswrapper[5052]: I1203 12:29:22.491896 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" event={"ID":"53193ff3-1a41-4e26-b38f-957088aa7e8b","Type":"ContainerStarted","Data":"7aa40d31e35ffdc4b2176768f89abf61eaa08b7b0c8e93d4f2c0bd090c30d97f"} Dec 03 12:29:22 crc kubenswrapper[5052]: I1203 12:29:22.493541 5052 generic.go:334] "Generic (PLEG): container finished" podID="211396f3-156a-4ac5-917b-6d41c236d3c2" containerID="0dcdd2aec7240ab8991d19f9ba29d4cff2af7594281adff351a9d0b94735f781" exitCode=137 Dec 03 12:29:22 crc kubenswrapper[5052]: I1203 12:29:22.493592 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-conductor-0" event={"ID":"211396f3-156a-4ac5-917b-6d41c236d3c2","Type":"ContainerDied","Data":"0dcdd2aec7240ab8991d19f9ba29d4cff2af7594281adff351a9d0b94735f781"} Dec 03 12:29:22 crc kubenswrapper[5052]: I1203 12:29:22.493613 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-conductor-0" event={"ID":"211396f3-156a-4ac5-917b-6d41c236d3c2","Type":"ContainerDied","Data":"63e1d551272047e4f06252a3ba2f200f584eb0e6682ccbf55490dee16f84d1a9"} Dec 03 12:29:22 crc kubenswrapper[5052]: I1203 12:29:22.493635 5052 scope.go:117] "RemoveContainer" containerID="0dcdd2aec7240ab8991d19f9ba29d4cff2af7594281adff351a9d0b94735f781" Dec 03 12:29:22 crc kubenswrapper[5052]: I1203 12:29:22.493754 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:29:22 crc kubenswrapper[5052]: I1203 12:29:22.528320 5052 scope.go:117] "RemoveContainer" containerID="0dcdd2aec7240ab8991d19f9ba29d4cff2af7594281adff351a9d0b94735f781" Dec 03 12:29:22 crc kubenswrapper[5052]: E1203 12:29:22.529085 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0dcdd2aec7240ab8991d19f9ba29d4cff2af7594281adff351a9d0b94735f781\": container with ID starting with 0dcdd2aec7240ab8991d19f9ba29d4cff2af7594281adff351a9d0b94735f781 not found: ID does not exist" containerID="0dcdd2aec7240ab8991d19f9ba29d4cff2af7594281adff351a9d0b94735f781" Dec 03 12:29:22 crc kubenswrapper[5052]: I1203 12:29:22.529135 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0dcdd2aec7240ab8991d19f9ba29d4cff2af7594281adff351a9d0b94735f781"} err="failed to get container status \"0dcdd2aec7240ab8991d19f9ba29d4cff2af7594281adff351a9d0b94735f781\": rpc error: code = NotFound desc = could not find container \"0dcdd2aec7240ab8991d19f9ba29d4cff2af7594281adff351a9d0b94735f781\": container with ID starting with 0dcdd2aec7240ab8991d19f9ba29d4cff2af7594281adff351a9d0b94735f781 not found: ID does not exist" Dec 03 12:29:22 crc kubenswrapper[5052]: I1203 12:29:22.531557 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-0"] Dec 03 12:29:22 crc kubenswrapper[5052]: I1203 12:29:22.542116 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-0"] Dec 03 12:29:22 crc kubenswrapper[5052]: I1203 12:29:22.554087 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-0"] Dec 03 12:29:22 crc kubenswrapper[5052]: E1203 12:29:22.555228 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="211396f3-156a-4ac5-917b-6d41c236d3c2" containerName="nova-cell0-conductor-conductor" Dec 03 12:29:22 crc kubenswrapper[5052]: I1203 12:29:22.555251 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="211396f3-156a-4ac5-917b-6d41c236d3c2" containerName="nova-cell0-conductor-conductor" Dec 03 12:29:22 crc kubenswrapper[5052]: I1203 12:29:22.555456 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="211396f3-156a-4ac5-917b-6d41c236d3c2" containerName="nova-cell0-conductor-conductor" Dec 03 12:29:22 crc kubenswrapper[5052]: I1203 12:29:22.556236 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:29:22 crc kubenswrapper[5052]: I1203 12:29:22.559890 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell0-conductor-config-data" Dec 03 12:29:22 crc kubenswrapper[5052]: I1203 12:29:22.567418 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-0"] Dec 03 12:29:22 crc kubenswrapper[5052]: I1203 12:29:22.627428 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:29:22 crc kubenswrapper[5052]: I1203 12:29:22.669282 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7x9gt\" (UniqueName: \"kubernetes.io/projected/16a065b5-a603-4cb3-a4c0-b36c0b92bf3f-kube-api-access-7x9gt\") pod \"nova-cell0-conductor-0\" (UID: \"16a065b5-a603-4cb3-a4c0-b36c0b92bf3f\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:29:22 crc kubenswrapper[5052]: I1203 12:29:22.670031 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/16a065b5-a603-4cb3-a4c0-b36c0b92bf3f-config-data\") pod \"nova-cell0-conductor-0\" (UID: \"16a065b5-a603-4cb3-a4c0-b36c0b92bf3f\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:29:22 crc kubenswrapper[5052]: I1203 12:29:22.670133 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/16a065b5-a603-4cb3-a4c0-b36c0b92bf3f-combined-ca-bundle\") pod \"nova-cell0-conductor-0\" (UID: \"16a065b5-a603-4cb3-a4c0-b36c0b92bf3f\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:29:22 crc kubenswrapper[5052]: W1203 12:29:22.696690 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod733f463b_dcf8_4a61_8e4e_4bf1c042d397.slice/crio-9929deb72ae6d5d35726a21123b7bb836c354c04102b48a8e5d3f5f7222a1ff0 WatchSource:0}: Error finding container 9929deb72ae6d5d35726a21123b7bb836c354c04102b48a8e5d3f5f7222a1ff0: Status 404 returned error can't find the container with id 9929deb72ae6d5d35726a21123b7bb836c354c04102b48a8e5d3f5f7222a1ff0 Dec 03 12:29:22 crc kubenswrapper[5052]: I1203 12:29:22.700184 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-0"] Dec 03 12:29:22 crc kubenswrapper[5052]: I1203 12:29:22.772592 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/16a065b5-a603-4cb3-a4c0-b36c0b92bf3f-combined-ca-bundle\") pod \"nova-cell0-conductor-0\" (UID: \"16a065b5-a603-4cb3-a4c0-b36c0b92bf3f\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:29:22 crc kubenswrapper[5052]: I1203 12:29:22.773021 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-7x9gt\" (UniqueName: \"kubernetes.io/projected/16a065b5-a603-4cb3-a4c0-b36c0b92bf3f-kube-api-access-7x9gt\") pod \"nova-cell0-conductor-0\" (UID: \"16a065b5-a603-4cb3-a4c0-b36c0b92bf3f\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:29:22 crc kubenswrapper[5052]: I1203 12:29:22.774341 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/16a065b5-a603-4cb3-a4c0-b36c0b92bf3f-config-data\") pod \"nova-cell0-conductor-0\" (UID: \"16a065b5-a603-4cb3-a4c0-b36c0b92bf3f\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:29:22 crc kubenswrapper[5052]: I1203 12:29:22.777736 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/16a065b5-a603-4cb3-a4c0-b36c0b92bf3f-combined-ca-bundle\") pod \"nova-cell0-conductor-0\" (UID: \"16a065b5-a603-4cb3-a4c0-b36c0b92bf3f\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:29:22 crc kubenswrapper[5052]: I1203 12:29:22.778913 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/16a065b5-a603-4cb3-a4c0-b36c0b92bf3f-config-data\") pod \"nova-cell0-conductor-0\" (UID: \"16a065b5-a603-4cb3-a4c0-b36c0b92bf3f\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:29:22 crc kubenswrapper[5052]: I1203 12:29:22.791828 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-7x9gt\" (UniqueName: \"kubernetes.io/projected/16a065b5-a603-4cb3-a4c0-b36c0b92bf3f-kube-api-access-7x9gt\") pod \"nova-cell0-conductor-0\" (UID: \"16a065b5-a603-4cb3-a4c0-b36c0b92bf3f\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:29:22 crc kubenswrapper[5052]: I1203 12:29:22.885628 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:29:23 crc kubenswrapper[5052]: I1203 12:29:23.355181 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-0"] Dec 03 12:29:23 crc kubenswrapper[5052]: W1203 12:29:23.362172 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod16a065b5_a603_4cb3_a4c0_b36c0b92bf3f.slice/crio-5189538592fa7792bfe6f41aa547a24c4cedd88ab25440d8ee94c398f4e1b743 WatchSource:0}: Error finding container 5189538592fa7792bfe6f41aa547a24c4cedd88ab25440d8ee94c398f4e1b743: Status 404 returned error can't find the container with id 5189538592fa7792bfe6f41aa547a24c4cedd88ab25440d8ee94c398f4e1b743 Dec 03 12:29:23 crc kubenswrapper[5052]: I1203 12:29:23.519931 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-conductor-0" event={"ID":"733f463b-dcf8-4a61-8e4e-4bf1c042d397","Type":"ContainerStarted","Data":"fc5e4fca262b35b92bb410a508d650dc4bdca0fc0848d2a90abb1ede218d5a33"} Dec 03 12:29:23 crc kubenswrapper[5052]: I1203 12:29:23.519976 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-conductor-0" event={"ID":"733f463b-dcf8-4a61-8e4e-4bf1c042d397","Type":"ContainerStarted","Data":"9929deb72ae6d5d35726a21123b7bb836c354c04102b48a8e5d3f5f7222a1ff0"} Dec 03 12:29:23 crc kubenswrapper[5052]: I1203 12:29:23.523390 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:29:23 crc kubenswrapper[5052]: I1203 12:29:23.523491 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"b953ffa0-088c-4450-8d60-04044b33cfa0","Type":"ContainerStarted","Data":"6f35ce14790c9ffb5796ad1467515b2be517ae60eb2573a1571ba5f99f630cad"} Dec 03 12:29:23 crc kubenswrapper[5052]: I1203 12:29:23.523585 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"b953ffa0-088c-4450-8d60-04044b33cfa0","Type":"ContainerStarted","Data":"6a7eedcc94f5e157a4977ec0b9a950f9234fa3da67edb3f87d442887fe9bd7f0"} Dec 03 12:29:23 crc kubenswrapper[5052]: I1203 12:29:23.529169 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-conductor-0" event={"ID":"16a065b5-a603-4cb3-a4c0-b36c0b92bf3f","Type":"ContainerStarted","Data":"5189538592fa7792bfe6f41aa547a24c4cedd88ab25440d8ee94c398f4e1b743"} Dec 03 12:29:23 crc kubenswrapper[5052]: I1203 12:29:23.534346 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" event={"ID":"53193ff3-1a41-4e26-b38f-957088aa7e8b","Type":"ContainerStarted","Data":"894cb495101405b55444078f27862c4a46c3937975fecb53b501499a7dc2a56d"} Dec 03 12:29:23 crc kubenswrapper[5052]: I1203 12:29:23.542318 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-cell1-conductor-0" podStartSLOduration=2.542298474 podStartE2EDuration="2.542298474s" podCreationTimestamp="2025-12-03 12:29:21 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:29:23.538275629 +0000 UTC m=+4154.126517393" watchObservedRunningTime="2025-12-03 12:29:23.542298474 +0000 UTC m=+4154.130540238" Dec 03 12:29:23 crc kubenswrapper[5052]: I1203 12:29:23.558666 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" podStartSLOduration=2.5586440010000002 podStartE2EDuration="2.558644001s" podCreationTimestamp="2025-12-03 12:29:21 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:29:23.555784139 +0000 UTC m=+4154.144025903" watchObservedRunningTime="2025-12-03 12:29:23.558644001 +0000 UTC m=+4154.146885785" Dec 03 12:29:23 crc kubenswrapper[5052]: I1203 12:29:23.685745 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="026d5ba1-5f6f-4f39-bcb6-9c51d37608f2" path="/var/lib/kubelet/pods/026d5ba1-5f6f-4f39-bcb6-9c51d37608f2/volumes" Dec 03 12:29:23 crc kubenswrapper[5052]: I1203 12:29:23.686470 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="211396f3-156a-4ac5-917b-6d41c236d3c2" path="/var/lib/kubelet/pods/211396f3-156a-4ac5-917b-6d41c236d3c2/volumes" Dec 03 12:29:24 crc kubenswrapper[5052]: I1203 12:29:24.547527 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-conductor-0" event={"ID":"16a065b5-a603-4cb3-a4c0-b36c0b92bf3f","Type":"ContainerStarted","Data":"c1a2fa107302843e2249473b85f61687344a151311126ad578b5760b51b1b60e"} Dec 03 12:29:24 crc kubenswrapper[5052]: I1203 12:29:24.564877 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-cell0-conductor-0" podStartSLOduration=2.564855801 podStartE2EDuration="2.564855801s" podCreationTimestamp="2025-12-03 12:29:22 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:29:24.564281834 +0000 UTC m=+4155.152523618" watchObservedRunningTime="2025-12-03 12:29:24.564855801 +0000 UTC m=+4155.153097565" Dec 03 12:29:24 crc kubenswrapper[5052]: I1203 12:29:24.566016 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-scheduler-0" podStartSLOduration=3.566006033 podStartE2EDuration="3.566006033s" podCreationTimestamp="2025-12-03 12:29:21 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:29:23.57963356 +0000 UTC m=+4154.167875334" watchObservedRunningTime="2025-12-03 12:29:24.566006033 +0000 UTC m=+4155.154247797" Dec 03 12:29:25 crc kubenswrapper[5052]: I1203 12:29:25.558932 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:29:26 crc kubenswrapper[5052]: I1203 12:29:26.993707 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:29:27 crc kubenswrapper[5052]: I1203 12:29:27.103893 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:29:32 crc kubenswrapper[5052]: I1203 12:29:32.008999 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:29:32 crc kubenswrapper[5052]: I1203 12:29:32.028586 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:29:32 crc kubenswrapper[5052]: I1203 12:29:32.103747 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:29:32 crc kubenswrapper[5052]: I1203 12:29:32.134175 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:29:32 crc kubenswrapper[5052]: I1203 12:29:32.157203 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:29:32 crc kubenswrapper[5052]: I1203 12:29:32.648203 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:29:32 crc kubenswrapper[5052]: I1203 12:29:32.657230 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:29:32 crc kubenswrapper[5052]: I1203 12:29:32.921134 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:29:39 crc kubenswrapper[5052]: I1203 12:29:39.725018 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:29:54 crc kubenswrapper[5052]: I1203 12:29:54.396547 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/openstackclient"] Dec 03 12:29:54 crc kubenswrapper[5052]: I1203 12:29:54.397161 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/openstackclient" podUID="78630d10-b9ba-438d-907d-2be706fc51e7" containerName="openstackclient" containerID="cri-o://7c24093b1dde24c14b80a87bb357568dbd53207a52044ef5dcbeb86466b30f11" gracePeriod=2 Dec 03 12:29:54 crc kubenswrapper[5052]: I1203 12:29:54.404957 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/memcached-0"] Dec 03 12:29:54 crc kubenswrapper[5052]: I1203 12:29:54.405271 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/memcached-0" podUID="b35e6939-8f23-496c-ac55-9534be4ea8be" containerName="memcached" containerID="cri-o://8d5e1c2473a3c27b159232ab3f630f8e1ca663780d8fc9b7b1eaac3611c97961" gracePeriod=30 Dec 03 12:29:54 crc kubenswrapper[5052]: I1203 12:29:54.420658 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-nb-0"] Dec 03 12:29:54 crc kubenswrapper[5052]: I1203 12:29:54.421020 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ovsdbserver-nb-0" podUID="f45e9eda-e62f-4307-ae18-2a8da7d6fe7c" containerName="openstack-network-exporter" containerID="cri-o://901636e0be8460bcc74b0c6a41492e349cbdbd334bbc41042f9dfdf0a187e9d1" gracePeriod=300 Dec 03 12:29:54 crc kubenswrapper[5052]: I1203 12:29:54.443424 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ovn-northd-0"] Dec 03 12:29:54 crc kubenswrapper[5052]: I1203 12:29:54.444922 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ovn-northd-0" podUID="a26ed42f-71e7-4c92-8b7e-ad194358d3e5" containerName="ovn-northd" containerID="cri-o://93474533ee16033979340cc7914412b7661d56c4b5e3b0058db42692da5f5166" gracePeriod=30 Dec 03 12:29:54 crc kubenswrapper[5052]: I1203 12:29:54.445036 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ovn-northd-0" podUID="a26ed42f-71e7-4c92-8b7e-ad194358d3e5" containerName="openstack-network-exporter" containerID="cri-o://911da6cb170ba67cad5584869214b5ef5f33b8f7400194bc4bc1ce4ca1a692f4" gracePeriod=30 Dec 03 12:29:54 crc kubenswrapper[5052]: I1203 12:29:54.459830 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/openstackclient"] Dec 03 12:29:54 crc kubenswrapper[5052]: I1203 12:29:54.495673 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/barbican-keystone-listener-77b6bbbdcb-4jlsc"] Dec 03 12:29:54 crc kubenswrapper[5052]: E1203 12:29:54.496364 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="78630d10-b9ba-438d-907d-2be706fc51e7" containerName="openstackclient" Dec 03 12:29:54 crc kubenswrapper[5052]: I1203 12:29:54.496375 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="78630d10-b9ba-438d-907d-2be706fc51e7" containerName="openstackclient" Dec 03 12:29:54 crc kubenswrapper[5052]: I1203 12:29:54.496574 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="78630d10-b9ba-438d-907d-2be706fc51e7" containerName="openstackclient" Dec 03 12:29:54 crc kubenswrapper[5052]: I1203 12:29:54.497673 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-keystone-listener-77b6bbbdcb-4jlsc" Dec 03 12:29:54 crc kubenswrapper[5052]: I1203 12:29:54.513864 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/aed41b09-319e-466c-9eb9-38208e1880a3-config-data-custom\") pod \"barbican-keystone-listener-77b6bbbdcb-4jlsc\" (UID: \"aed41b09-319e-466c-9eb9-38208e1880a3\") " pod="openstack-kuttl-tests/barbican-keystone-listener-77b6bbbdcb-4jlsc" Dec 03 12:29:54 crc kubenswrapper[5052]: I1203 12:29:54.513944 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/aed41b09-319e-466c-9eb9-38208e1880a3-logs\") pod \"barbican-keystone-listener-77b6bbbdcb-4jlsc\" (UID: \"aed41b09-319e-466c-9eb9-38208e1880a3\") " pod="openstack-kuttl-tests/barbican-keystone-listener-77b6bbbdcb-4jlsc" Dec 03 12:29:54 crc kubenswrapper[5052]: I1203 12:29:54.513996 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wv5t5\" (UniqueName: \"kubernetes.io/projected/aed41b09-319e-466c-9eb9-38208e1880a3-kube-api-access-wv5t5\") pod \"barbican-keystone-listener-77b6bbbdcb-4jlsc\" (UID: \"aed41b09-319e-466c-9eb9-38208e1880a3\") " pod="openstack-kuttl-tests/barbican-keystone-listener-77b6bbbdcb-4jlsc" Dec 03 12:29:54 crc kubenswrapper[5052]: I1203 12:29:54.514030 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/aed41b09-319e-466c-9eb9-38208e1880a3-combined-ca-bundle\") pod \"barbican-keystone-listener-77b6bbbdcb-4jlsc\" (UID: \"aed41b09-319e-466c-9eb9-38208e1880a3\") " pod="openstack-kuttl-tests/barbican-keystone-listener-77b6bbbdcb-4jlsc" Dec 03 12:29:54 crc kubenswrapper[5052]: I1203 12:29:54.514059 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/aed41b09-319e-466c-9eb9-38208e1880a3-config-data\") pod \"barbican-keystone-listener-77b6bbbdcb-4jlsc\" (UID: \"aed41b09-319e-466c-9eb9-38208e1880a3\") " pod="openstack-kuttl-tests/barbican-keystone-listener-77b6bbbdcb-4jlsc" Dec 03 12:29:54 crc kubenswrapper[5052]: I1203 12:29:54.540692 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-keystone-listener-77b6bbbdcb-4jlsc"] Dec 03 12:29:54 crc kubenswrapper[5052]: I1203 12:29:54.552574 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:29:54 crc kubenswrapper[5052]: I1203 12:29:54.552868 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-external-api-0" podUID="05b9d3cc-93de-45a6-8849-027a4aad3715" containerName="glance-log" containerID="cri-o://07f1af7d6a787079787ad912bf9e1ca56fb4cb7d8e231d7b665fb887db97be81" gracePeriod=30 Dec 03 12:29:54 crc kubenswrapper[5052]: I1203 12:29:54.553028 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-external-api-0" podUID="05b9d3cc-93de-45a6-8849-027a4aad3715" containerName="glance-httpd" containerID="cri-o://2f2f681ab4e070f1a45adfa2327a6b72d7c87fcccf7bb015c88103e25b6472d9" gracePeriod=30 Dec 03 12:29:54 crc kubenswrapper[5052]: I1203 12:29:54.571915 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:29:54 crc kubenswrapper[5052]: I1203 12:29:54.572242 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-metadata-0" podUID="caaecb05-f5f1-421d-8e91-818774a2c0ec" containerName="nova-metadata-log" containerID="cri-o://cda20389ae104a8a76ec7b7e03e816da57afd5a07e90e7f0e1bf79a5704aa60d" gracePeriod=30 Dec 03 12:29:54 crc kubenswrapper[5052]: I1203 12:29:54.572381 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-metadata-0" podUID="caaecb05-f5f1-421d-8e91-818774a2c0ec" containerName="nova-metadata-metadata" containerID="cri-o://e2e4d22232c587b84e25f9aceb08f0b3c39f38badcaa667d82792f3314b6a1b9" gracePeriod=30 Dec 03 12:29:54 crc kubenswrapper[5052]: I1203 12:29:54.616734 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/aed41b09-319e-466c-9eb9-38208e1880a3-logs\") pod \"barbican-keystone-listener-77b6bbbdcb-4jlsc\" (UID: \"aed41b09-319e-466c-9eb9-38208e1880a3\") " pod="openstack-kuttl-tests/barbican-keystone-listener-77b6bbbdcb-4jlsc" Dec 03 12:29:54 crc kubenswrapper[5052]: I1203 12:29:54.616830 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-wv5t5\" (UniqueName: \"kubernetes.io/projected/aed41b09-319e-466c-9eb9-38208e1880a3-kube-api-access-wv5t5\") pod \"barbican-keystone-listener-77b6bbbdcb-4jlsc\" (UID: \"aed41b09-319e-466c-9eb9-38208e1880a3\") " pod="openstack-kuttl-tests/barbican-keystone-listener-77b6bbbdcb-4jlsc" Dec 03 12:29:54 crc kubenswrapper[5052]: I1203 12:29:54.616883 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/aed41b09-319e-466c-9eb9-38208e1880a3-combined-ca-bundle\") pod \"barbican-keystone-listener-77b6bbbdcb-4jlsc\" (UID: \"aed41b09-319e-466c-9eb9-38208e1880a3\") " pod="openstack-kuttl-tests/barbican-keystone-listener-77b6bbbdcb-4jlsc" Dec 03 12:29:54 crc kubenswrapper[5052]: I1203 12:29:54.616929 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/aed41b09-319e-466c-9eb9-38208e1880a3-config-data\") pod \"barbican-keystone-listener-77b6bbbdcb-4jlsc\" (UID: \"aed41b09-319e-466c-9eb9-38208e1880a3\") " pod="openstack-kuttl-tests/barbican-keystone-listener-77b6bbbdcb-4jlsc" Dec 03 12:29:54 crc kubenswrapper[5052]: I1203 12:29:54.617033 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/aed41b09-319e-466c-9eb9-38208e1880a3-config-data-custom\") pod \"barbican-keystone-listener-77b6bbbdcb-4jlsc\" (UID: \"aed41b09-319e-466c-9eb9-38208e1880a3\") " pod="openstack-kuttl-tests/barbican-keystone-listener-77b6bbbdcb-4jlsc" Dec 03 12:29:54 crc kubenswrapper[5052]: I1203 12:29:54.622216 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/aed41b09-319e-466c-9eb9-38208e1880a3-logs\") pod \"barbican-keystone-listener-77b6bbbdcb-4jlsc\" (UID: \"aed41b09-319e-466c-9eb9-38208e1880a3\") " pod="openstack-kuttl-tests/barbican-keystone-listener-77b6bbbdcb-4jlsc" Dec 03 12:29:54 crc kubenswrapper[5052]: I1203 12:29:54.635520 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/aed41b09-319e-466c-9eb9-38208e1880a3-combined-ca-bundle\") pod \"barbican-keystone-listener-77b6bbbdcb-4jlsc\" (UID: \"aed41b09-319e-466c-9eb9-38208e1880a3\") " pod="openstack-kuttl-tests/barbican-keystone-listener-77b6bbbdcb-4jlsc" Dec 03 12:29:54 crc kubenswrapper[5052]: I1203 12:29:54.637676 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/aed41b09-319e-466c-9eb9-38208e1880a3-config-data\") pod \"barbican-keystone-listener-77b6bbbdcb-4jlsc\" (UID: \"aed41b09-319e-466c-9eb9-38208e1880a3\") " pod="openstack-kuttl-tests/barbican-keystone-listener-77b6bbbdcb-4jlsc" Dec 03 12:29:54 crc kubenswrapper[5052]: I1203 12:29:54.661176 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/barbican-worker-6d687d9c57-4gplx"] Dec 03 12:29:54 crc kubenswrapper[5052]: I1203 12:29:54.668045 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/aed41b09-319e-466c-9eb9-38208e1880a3-config-data-custom\") pod \"barbican-keystone-listener-77b6bbbdcb-4jlsc\" (UID: \"aed41b09-319e-466c-9eb9-38208e1880a3\") " pod="openstack-kuttl-tests/barbican-keystone-listener-77b6bbbdcb-4jlsc" Dec 03 12:29:54 crc kubenswrapper[5052]: I1203 12:29:54.686235 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-worker-6d687d9c57-4gplx"] Dec 03 12:29:54 crc kubenswrapper[5052]: I1203 12:29:54.687610 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-worker-6d687d9c57-4gplx" Dec 03 12:29:54 crc kubenswrapper[5052]: I1203 12:29:54.699059 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-wv5t5\" (UniqueName: \"kubernetes.io/projected/aed41b09-319e-466c-9eb9-38208e1880a3-kube-api-access-wv5t5\") pod \"barbican-keystone-listener-77b6bbbdcb-4jlsc\" (UID: \"aed41b09-319e-466c-9eb9-38208e1880a3\") " pod="openstack-kuttl-tests/barbican-keystone-listener-77b6bbbdcb-4jlsc" Dec 03 12:29:54 crc kubenswrapper[5052]: I1203 12:29:54.718146 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s4zxs\" (UniqueName: \"kubernetes.io/projected/a65652a1-62fa-40ff-8b8a-940b2cc795b7-kube-api-access-s4zxs\") pod \"barbican-worker-6d687d9c57-4gplx\" (UID: \"a65652a1-62fa-40ff-8b8a-940b2cc795b7\") " pod="openstack-kuttl-tests/barbican-worker-6d687d9c57-4gplx" Dec 03 12:29:54 crc kubenswrapper[5052]: I1203 12:29:54.718194 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a65652a1-62fa-40ff-8b8a-940b2cc795b7-config-data\") pod \"barbican-worker-6d687d9c57-4gplx\" (UID: \"a65652a1-62fa-40ff-8b8a-940b2cc795b7\") " pod="openstack-kuttl-tests/barbican-worker-6d687d9c57-4gplx" Dec 03 12:29:54 crc kubenswrapper[5052]: I1203 12:29:54.718239 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/a65652a1-62fa-40ff-8b8a-940b2cc795b7-logs\") pod \"barbican-worker-6d687d9c57-4gplx\" (UID: \"a65652a1-62fa-40ff-8b8a-940b2cc795b7\") " pod="openstack-kuttl-tests/barbican-worker-6d687d9c57-4gplx" Dec 03 12:29:54 crc kubenswrapper[5052]: I1203 12:29:54.718287 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a65652a1-62fa-40ff-8b8a-940b2cc795b7-combined-ca-bundle\") pod \"barbican-worker-6d687d9c57-4gplx\" (UID: \"a65652a1-62fa-40ff-8b8a-940b2cc795b7\") " pod="openstack-kuttl-tests/barbican-worker-6d687d9c57-4gplx" Dec 03 12:29:54 crc kubenswrapper[5052]: I1203 12:29:54.718337 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/a65652a1-62fa-40ff-8b8a-940b2cc795b7-config-data-custom\") pod \"barbican-worker-6d687d9c57-4gplx\" (UID: \"a65652a1-62fa-40ff-8b8a-940b2cc795b7\") " pod="openstack-kuttl-tests/barbican-worker-6d687d9c57-4gplx" Dec 03 12:29:54 crc kubenswrapper[5052]: I1203 12:29:54.772769 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 12:29:54 crc kubenswrapper[5052]: I1203 12:29:54.773055 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/cinder-scheduler-0" podUID="214a26fb-e9b2-4a87-8fa5-ae2ed0d8a828" containerName="cinder-scheduler" containerID="cri-o://b6c09457c183d144bb440f045e7a0230c87297b84897525513a2e671062b9c96" gracePeriod=30 Dec 03 12:29:54 crc kubenswrapper[5052]: I1203 12:29:54.773477 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/cinder-scheduler-0" podUID="214a26fb-e9b2-4a87-8fa5-ae2ed0d8a828" containerName="probe" containerID="cri-o://27a238af716222ccb80b51b0ddb93658176d3aa5f8db130deec5a43ab67bb332" gracePeriod=30 Dec 03 12:29:54 crc kubenswrapper[5052]: E1203 12:29:54.779101 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="93474533ee16033979340cc7914412b7661d56c4b5e3b0058db42692da5f5166" cmd=["/usr/local/bin/container-scripts/status_check.sh"] Dec 03 12:29:54 crc kubenswrapper[5052]: I1203 12:29:54.796643 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-keystone-listener-77b6bbbdcb-4jlsc" Dec 03 12:29:54 crc kubenswrapper[5052]: E1203 12:29:54.796715 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="93474533ee16033979340cc7914412b7661d56c4b5e3b0058db42692da5f5166" cmd=["/usr/local/bin/container-scripts/status_check.sh"] Dec 03 12:29:54 crc kubenswrapper[5052]: I1203 12:29:54.807307 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 12:29:54 crc kubenswrapper[5052]: I1203 12:29:54.807558 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" podUID="53193ff3-1a41-4e26-b38f-957088aa7e8b" containerName="nova-cell1-novncproxy-novncproxy" containerID="cri-o://894cb495101405b55444078f27862c4a46c3937975fecb53b501499a7dc2a56d" gracePeriod=30 Dec 03 12:29:54 crc kubenswrapper[5052]: I1203 12:29:54.837690 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ovsdbserver-nb-0" podUID="f45e9eda-e62f-4307-ae18-2a8da7d6fe7c" containerName="ovsdbserver-nb" containerID="cri-o://09275335090deb24b922ed4694cb037b39bf0e6b72303e640ea22dfe86fa1716" gracePeriod=300 Dec 03 12:29:54 crc kubenswrapper[5052]: E1203 12:29:54.869777 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="93474533ee16033979340cc7914412b7661d56c4b5e3b0058db42692da5f5166" cmd=["/usr/local/bin/container-scripts/status_check.sh"] Dec 03 12:29:54 crc kubenswrapper[5052]: E1203 12:29:54.869837 5052 prober.go:104] "Probe errored" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" probeType="Readiness" pod="openstack-kuttl-tests/ovn-northd-0" podUID="a26ed42f-71e7-4c92-8b7e-ad194358d3e5" containerName="ovn-northd" Dec 03 12:29:54 crc kubenswrapper[5052]: I1203 12:29:54.887707 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/a65652a1-62fa-40ff-8b8a-940b2cc795b7-logs\") pod \"barbican-worker-6d687d9c57-4gplx\" (UID: \"a65652a1-62fa-40ff-8b8a-940b2cc795b7\") " pod="openstack-kuttl-tests/barbican-worker-6d687d9c57-4gplx" Dec 03 12:29:54 crc kubenswrapper[5052]: I1203 12:29:54.887844 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a65652a1-62fa-40ff-8b8a-940b2cc795b7-combined-ca-bundle\") pod \"barbican-worker-6d687d9c57-4gplx\" (UID: \"a65652a1-62fa-40ff-8b8a-940b2cc795b7\") " pod="openstack-kuttl-tests/barbican-worker-6d687d9c57-4gplx" Dec 03 12:29:54 crc kubenswrapper[5052]: I1203 12:29:54.887983 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/a65652a1-62fa-40ff-8b8a-940b2cc795b7-config-data-custom\") pod \"barbican-worker-6d687d9c57-4gplx\" (UID: \"a65652a1-62fa-40ff-8b8a-940b2cc795b7\") " pod="openstack-kuttl-tests/barbican-worker-6d687d9c57-4gplx" Dec 03 12:29:54 crc kubenswrapper[5052]: I1203 12:29:54.888377 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-s4zxs\" (UniqueName: \"kubernetes.io/projected/a65652a1-62fa-40ff-8b8a-940b2cc795b7-kube-api-access-s4zxs\") pod \"barbican-worker-6d687d9c57-4gplx\" (UID: \"a65652a1-62fa-40ff-8b8a-940b2cc795b7\") " pod="openstack-kuttl-tests/barbican-worker-6d687d9c57-4gplx" Dec 03 12:29:54 crc kubenswrapper[5052]: I1203 12:29:54.888489 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a65652a1-62fa-40ff-8b8a-940b2cc795b7-config-data\") pod \"barbican-worker-6d687d9c57-4gplx\" (UID: \"a65652a1-62fa-40ff-8b8a-940b2cc795b7\") " pod="openstack-kuttl-tests/barbican-worker-6d687d9c57-4gplx" Dec 03 12:29:54 crc kubenswrapper[5052]: I1203 12:29:54.896129 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/a65652a1-62fa-40ff-8b8a-940b2cc795b7-logs\") pod \"barbican-worker-6d687d9c57-4gplx\" (UID: \"a65652a1-62fa-40ff-8b8a-940b2cc795b7\") " pod="openstack-kuttl-tests/barbican-worker-6d687d9c57-4gplx" Dec 03 12:29:54 crc kubenswrapper[5052]: I1203 12:29:54.905921 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/a65652a1-62fa-40ff-8b8a-940b2cc795b7-config-data-custom\") pod \"barbican-worker-6d687d9c57-4gplx\" (UID: \"a65652a1-62fa-40ff-8b8a-940b2cc795b7\") " pod="openstack-kuttl-tests/barbican-worker-6d687d9c57-4gplx" Dec 03 12:29:54 crc kubenswrapper[5052]: I1203 12:29:54.910577 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/ovsdbserver-nb-0" podUID="f45e9eda-e62f-4307-ae18-2a8da7d6fe7c" containerName="ovsdbserver-nb" probeResult="failure" output="" Dec 03 12:29:54 crc kubenswrapper[5052]: I1203 12:29:54.911926 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a65652a1-62fa-40ff-8b8a-940b2cc795b7-config-data\") pod \"barbican-worker-6d687d9c57-4gplx\" (UID: \"a65652a1-62fa-40ff-8b8a-940b2cc795b7\") " pod="openstack-kuttl-tests/barbican-worker-6d687d9c57-4gplx" Dec 03 12:29:54 crc kubenswrapper[5052]: I1203 12:29:54.914378 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-s4zxs\" (UniqueName: \"kubernetes.io/projected/a65652a1-62fa-40ff-8b8a-940b2cc795b7-kube-api-access-s4zxs\") pod \"barbican-worker-6d687d9c57-4gplx\" (UID: \"a65652a1-62fa-40ff-8b8a-940b2cc795b7\") " pod="openstack-kuttl-tests/barbican-worker-6d687d9c57-4gplx" Dec 03 12:29:54 crc kubenswrapper[5052]: I1203 12:29:54.930885 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:29:54 crc kubenswrapper[5052]: I1203 12:29:54.931539 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-api-0" podUID="b5b5ad16-1e45-4761-b5d2-60ffd5934ccb" containerName="nova-api-log" containerID="cri-o://0e4e8a3b2fb3bbe0432215ed54bf039b98ca0a274e59dc10062d81386889219c" gracePeriod=30 Dec 03 12:29:54 crc kubenswrapper[5052]: I1203 12:29:54.937565 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-api-0" podUID="b5b5ad16-1e45-4761-b5d2-60ffd5934ccb" containerName="nova-api-api" containerID="cri-o://ca424ba734d34bcc97da61c506acc5ac518a281efdcc25fe387f6db662517ec3" gracePeriod=30 Dec 03 12:29:54 crc kubenswrapper[5052]: I1203 12:29:54.967845 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:29:54 crc kubenswrapper[5052]: I1203 12:29:54.968090 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-scheduler-0" podUID="b953ffa0-088c-4450-8d60-04044b33cfa0" containerName="nova-scheduler-scheduler" containerID="cri-o://6f35ce14790c9ffb5796ad1467515b2be517ae60eb2573a1571ba5f99f630cad" gracePeriod=30 Dec 03 12:29:54 crc kubenswrapper[5052]: I1203 12:29:54.973250 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a65652a1-62fa-40ff-8b8a-940b2cc795b7-combined-ca-bundle\") pod \"barbican-worker-6d687d9c57-4gplx\" (UID: \"a65652a1-62fa-40ff-8b8a-940b2cc795b7\") " pod="openstack-kuttl-tests/barbican-worker-6d687d9c57-4gplx" Dec 03 12:29:55 crc kubenswrapper[5052]: I1203 12:29:55.083318 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/placement-5c6478db8b-bt2rw"] Dec 03 12:29:55 crc kubenswrapper[5052]: I1203 12:29:55.085484 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-5c6478db8b-bt2rw" Dec 03 12:29:55 crc kubenswrapper[5052]: I1203 12:29:55.121065 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/barbican-api-5fc7894bfd-g7hl5"] Dec 03 12:29:55 crc kubenswrapper[5052]: I1203 12:29:55.124142 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-api-5fc7894bfd-g7hl5" Dec 03 12:29:55 crc kubenswrapper[5052]: I1203 12:29:55.147569 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/keystone-7cbf547d94-9krth"] Dec 03 12:29:55 crc kubenswrapper[5052]: I1203 12:29:55.149347 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-7cbf547d94-9krth" Dec 03 12:29:55 crc kubenswrapper[5052]: I1203 12:29:55.163587 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-7cbf547d94-9krth"] Dec 03 12:29:55 crc kubenswrapper[5052]: I1203 12:29:55.179296 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-worker-6d687d9c57-4gplx" Dec 03 12:29:55 crc kubenswrapper[5052]: I1203 12:29:55.205834 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/cfb8e98d-3873-43e0-81dc-4450983179c2-config-data\") pod \"barbican-api-5fc7894bfd-g7hl5\" (UID: \"cfb8e98d-3873-43e0-81dc-4450983179c2\") " pod="openstack-kuttl-tests/barbican-api-5fc7894bfd-g7hl5" Dec 03 12:29:55 crc kubenswrapper[5052]: I1203 12:29:55.205886 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/6927052c-212d-42fc-8236-b053ffc9a9d9-scripts\") pod \"keystone-7cbf547d94-9krth\" (UID: \"6927052c-212d-42fc-8236-b053ffc9a9d9\") " pod="openstack-kuttl-tests/keystone-7cbf547d94-9krth" Dec 03 12:29:55 crc kubenswrapper[5052]: I1203 12:29:55.205913 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/8bddf2b1-8116-4b6e-9cd8-be8fe1cd4b64-config-data\") pod \"placement-5c6478db8b-bt2rw\" (UID: \"8bddf2b1-8116-4b6e-9cd8-be8fe1cd4b64\") " pod="openstack-kuttl-tests/placement-5c6478db8b-bt2rw" Dec 03 12:29:55 crc kubenswrapper[5052]: I1203 12:29:55.205943 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nxlhc\" (UniqueName: \"kubernetes.io/projected/8bddf2b1-8116-4b6e-9cd8-be8fe1cd4b64-kube-api-access-nxlhc\") pod \"placement-5c6478db8b-bt2rw\" (UID: \"8bddf2b1-8116-4b6e-9cd8-be8fe1cd4b64\") " pod="openstack-kuttl-tests/placement-5c6478db8b-bt2rw" Dec 03 12:29:55 crc kubenswrapper[5052]: I1203 12:29:55.205963 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6927052c-212d-42fc-8236-b053ffc9a9d9-combined-ca-bundle\") pod \"keystone-7cbf547d94-9krth\" (UID: \"6927052c-212d-42fc-8236-b053ffc9a9d9\") " pod="openstack-kuttl-tests/keystone-7cbf547d94-9krth" Dec 03 12:29:55 crc kubenswrapper[5052]: I1203 12:29:55.205987 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/8bddf2b1-8116-4b6e-9cd8-be8fe1cd4b64-scripts\") pod \"placement-5c6478db8b-bt2rw\" (UID: \"8bddf2b1-8116-4b6e-9cd8-be8fe1cd4b64\") " pod="openstack-kuttl-tests/placement-5c6478db8b-bt2rw" Dec 03 12:29:55 crc kubenswrapper[5052]: I1203 12:29:55.206006 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/6927052c-212d-42fc-8236-b053ffc9a9d9-fernet-keys\") pod \"keystone-7cbf547d94-9krth\" (UID: \"6927052c-212d-42fc-8236-b053ffc9a9d9\") " pod="openstack-kuttl-tests/keystone-7cbf547d94-9krth" Dec 03 12:29:55 crc kubenswrapper[5052]: I1203 12:29:55.206042 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gvz7n\" (UniqueName: \"kubernetes.io/projected/6927052c-212d-42fc-8236-b053ffc9a9d9-kube-api-access-gvz7n\") pod \"keystone-7cbf547d94-9krth\" (UID: \"6927052c-212d-42fc-8236-b053ffc9a9d9\") " pod="openstack-kuttl-tests/keystone-7cbf547d94-9krth" Dec 03 12:29:55 crc kubenswrapper[5052]: I1203 12:29:55.206061 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/6927052c-212d-42fc-8236-b053ffc9a9d9-credential-keys\") pod \"keystone-7cbf547d94-9krth\" (UID: \"6927052c-212d-42fc-8236-b053ffc9a9d9\") " pod="openstack-kuttl-tests/keystone-7cbf547d94-9krth" Dec 03 12:29:55 crc kubenswrapper[5052]: I1203 12:29:55.206091 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/8bddf2b1-8116-4b6e-9cd8-be8fe1cd4b64-public-tls-certs\") pod \"placement-5c6478db8b-bt2rw\" (UID: \"8bddf2b1-8116-4b6e-9cd8-be8fe1cd4b64\") " pod="openstack-kuttl-tests/placement-5c6478db8b-bt2rw" Dec 03 12:29:55 crc kubenswrapper[5052]: I1203 12:29:55.206122 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/cfb8e98d-3873-43e0-81dc-4450983179c2-logs\") pod \"barbican-api-5fc7894bfd-g7hl5\" (UID: \"cfb8e98d-3873-43e0-81dc-4450983179c2\") " pod="openstack-kuttl-tests/barbican-api-5fc7894bfd-g7hl5" Dec 03 12:29:55 crc kubenswrapper[5052]: I1203 12:29:55.206142 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/cfb8e98d-3873-43e0-81dc-4450983179c2-config-data-custom\") pod \"barbican-api-5fc7894bfd-g7hl5\" (UID: \"cfb8e98d-3873-43e0-81dc-4450983179c2\") " pod="openstack-kuttl-tests/barbican-api-5fc7894bfd-g7hl5" Dec 03 12:29:55 crc kubenswrapper[5052]: I1203 12:29:55.206162 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/6927052c-212d-42fc-8236-b053ffc9a9d9-public-tls-certs\") pod \"keystone-7cbf547d94-9krth\" (UID: \"6927052c-212d-42fc-8236-b053ffc9a9d9\") " pod="openstack-kuttl-tests/keystone-7cbf547d94-9krth" Dec 03 12:29:55 crc kubenswrapper[5052]: I1203 12:29:55.206201 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8bddf2b1-8116-4b6e-9cd8-be8fe1cd4b64-combined-ca-bundle\") pod \"placement-5c6478db8b-bt2rw\" (UID: \"8bddf2b1-8116-4b6e-9cd8-be8fe1cd4b64\") " pod="openstack-kuttl-tests/placement-5c6478db8b-bt2rw" Dec 03 12:29:55 crc kubenswrapper[5052]: I1203 12:29:55.206230 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/8bddf2b1-8116-4b6e-9cd8-be8fe1cd4b64-internal-tls-certs\") pod \"placement-5c6478db8b-bt2rw\" (UID: \"8bddf2b1-8116-4b6e-9cd8-be8fe1cd4b64\") " pod="openstack-kuttl-tests/placement-5c6478db8b-bt2rw" Dec 03 12:29:55 crc kubenswrapper[5052]: I1203 12:29:55.206257 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6927052c-212d-42fc-8236-b053ffc9a9d9-config-data\") pod \"keystone-7cbf547d94-9krth\" (UID: \"6927052c-212d-42fc-8236-b053ffc9a9d9\") " pod="openstack-kuttl-tests/keystone-7cbf547d94-9krth" Dec 03 12:29:55 crc kubenswrapper[5052]: I1203 12:29:55.206276 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8sxqm\" (UniqueName: \"kubernetes.io/projected/cfb8e98d-3873-43e0-81dc-4450983179c2-kube-api-access-8sxqm\") pod \"barbican-api-5fc7894bfd-g7hl5\" (UID: \"cfb8e98d-3873-43e0-81dc-4450983179c2\") " pod="openstack-kuttl-tests/barbican-api-5fc7894bfd-g7hl5" Dec 03 12:29:55 crc kubenswrapper[5052]: I1203 12:29:55.206296 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/cfb8e98d-3873-43e0-81dc-4450983179c2-public-tls-certs\") pod \"barbican-api-5fc7894bfd-g7hl5\" (UID: \"cfb8e98d-3873-43e0-81dc-4450983179c2\") " pod="openstack-kuttl-tests/barbican-api-5fc7894bfd-g7hl5" Dec 03 12:29:55 crc kubenswrapper[5052]: I1203 12:29:55.206325 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/8bddf2b1-8116-4b6e-9cd8-be8fe1cd4b64-logs\") pod \"placement-5c6478db8b-bt2rw\" (UID: \"8bddf2b1-8116-4b6e-9cd8-be8fe1cd4b64\") " pod="openstack-kuttl-tests/placement-5c6478db8b-bt2rw" Dec 03 12:29:55 crc kubenswrapper[5052]: I1203 12:29:55.206347 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/cfb8e98d-3873-43e0-81dc-4450983179c2-combined-ca-bundle\") pod \"barbican-api-5fc7894bfd-g7hl5\" (UID: \"cfb8e98d-3873-43e0-81dc-4450983179c2\") " pod="openstack-kuttl-tests/barbican-api-5fc7894bfd-g7hl5" Dec 03 12:29:55 crc kubenswrapper[5052]: I1203 12:29:55.206373 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/cfb8e98d-3873-43e0-81dc-4450983179c2-internal-tls-certs\") pod \"barbican-api-5fc7894bfd-g7hl5\" (UID: \"cfb8e98d-3873-43e0-81dc-4450983179c2\") " pod="openstack-kuttl-tests/barbican-api-5fc7894bfd-g7hl5" Dec 03 12:29:55 crc kubenswrapper[5052]: I1203 12:29:55.206409 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/6927052c-212d-42fc-8236-b053ffc9a9d9-internal-tls-certs\") pod \"keystone-7cbf547d94-9krth\" (UID: \"6927052c-212d-42fc-8236-b053ffc9a9d9\") " pod="openstack-kuttl-tests/keystone-7cbf547d94-9krth" Dec 03 12:29:55 crc kubenswrapper[5052]: I1203 12:29:55.243176 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/placement-5c6478db8b-bt2rw"] Dec 03 12:29:55 crc kubenswrapper[5052]: I1203 12:29:55.310263 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-gvz7n\" (UniqueName: \"kubernetes.io/projected/6927052c-212d-42fc-8236-b053ffc9a9d9-kube-api-access-gvz7n\") pod \"keystone-7cbf547d94-9krth\" (UID: \"6927052c-212d-42fc-8236-b053ffc9a9d9\") " pod="openstack-kuttl-tests/keystone-7cbf547d94-9krth" Dec 03 12:29:55 crc kubenswrapper[5052]: I1203 12:29:55.310298 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/6927052c-212d-42fc-8236-b053ffc9a9d9-credential-keys\") pod \"keystone-7cbf547d94-9krth\" (UID: \"6927052c-212d-42fc-8236-b053ffc9a9d9\") " pod="openstack-kuttl-tests/keystone-7cbf547d94-9krth" Dec 03 12:29:55 crc kubenswrapper[5052]: I1203 12:29:55.310340 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/8bddf2b1-8116-4b6e-9cd8-be8fe1cd4b64-public-tls-certs\") pod \"placement-5c6478db8b-bt2rw\" (UID: \"8bddf2b1-8116-4b6e-9cd8-be8fe1cd4b64\") " pod="openstack-kuttl-tests/placement-5c6478db8b-bt2rw" Dec 03 12:29:55 crc kubenswrapper[5052]: I1203 12:29:55.310385 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/cfb8e98d-3873-43e0-81dc-4450983179c2-logs\") pod \"barbican-api-5fc7894bfd-g7hl5\" (UID: \"cfb8e98d-3873-43e0-81dc-4450983179c2\") " pod="openstack-kuttl-tests/barbican-api-5fc7894bfd-g7hl5" Dec 03 12:29:55 crc kubenswrapper[5052]: I1203 12:29:55.310407 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/cfb8e98d-3873-43e0-81dc-4450983179c2-config-data-custom\") pod \"barbican-api-5fc7894bfd-g7hl5\" (UID: \"cfb8e98d-3873-43e0-81dc-4450983179c2\") " pod="openstack-kuttl-tests/barbican-api-5fc7894bfd-g7hl5" Dec 03 12:29:55 crc kubenswrapper[5052]: I1203 12:29:55.310424 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/6927052c-212d-42fc-8236-b053ffc9a9d9-public-tls-certs\") pod \"keystone-7cbf547d94-9krth\" (UID: \"6927052c-212d-42fc-8236-b053ffc9a9d9\") " pod="openstack-kuttl-tests/keystone-7cbf547d94-9krth" Dec 03 12:29:55 crc kubenswrapper[5052]: I1203 12:29:55.310477 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8bddf2b1-8116-4b6e-9cd8-be8fe1cd4b64-combined-ca-bundle\") pod \"placement-5c6478db8b-bt2rw\" (UID: \"8bddf2b1-8116-4b6e-9cd8-be8fe1cd4b64\") " pod="openstack-kuttl-tests/placement-5c6478db8b-bt2rw" Dec 03 12:29:55 crc kubenswrapper[5052]: I1203 12:29:55.310536 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/8bddf2b1-8116-4b6e-9cd8-be8fe1cd4b64-internal-tls-certs\") pod \"placement-5c6478db8b-bt2rw\" (UID: \"8bddf2b1-8116-4b6e-9cd8-be8fe1cd4b64\") " pod="openstack-kuttl-tests/placement-5c6478db8b-bt2rw" Dec 03 12:29:55 crc kubenswrapper[5052]: I1203 12:29:55.310568 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6927052c-212d-42fc-8236-b053ffc9a9d9-config-data\") pod \"keystone-7cbf547d94-9krth\" (UID: \"6927052c-212d-42fc-8236-b053ffc9a9d9\") " pod="openstack-kuttl-tests/keystone-7cbf547d94-9krth" Dec 03 12:29:55 crc kubenswrapper[5052]: I1203 12:29:55.310592 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-8sxqm\" (UniqueName: \"kubernetes.io/projected/cfb8e98d-3873-43e0-81dc-4450983179c2-kube-api-access-8sxqm\") pod \"barbican-api-5fc7894bfd-g7hl5\" (UID: \"cfb8e98d-3873-43e0-81dc-4450983179c2\") " pod="openstack-kuttl-tests/barbican-api-5fc7894bfd-g7hl5" Dec 03 12:29:55 crc kubenswrapper[5052]: I1203 12:29:55.310611 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/cfb8e98d-3873-43e0-81dc-4450983179c2-public-tls-certs\") pod \"barbican-api-5fc7894bfd-g7hl5\" (UID: \"cfb8e98d-3873-43e0-81dc-4450983179c2\") " pod="openstack-kuttl-tests/barbican-api-5fc7894bfd-g7hl5" Dec 03 12:29:55 crc kubenswrapper[5052]: I1203 12:29:55.310648 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/8bddf2b1-8116-4b6e-9cd8-be8fe1cd4b64-logs\") pod \"placement-5c6478db8b-bt2rw\" (UID: \"8bddf2b1-8116-4b6e-9cd8-be8fe1cd4b64\") " pod="openstack-kuttl-tests/placement-5c6478db8b-bt2rw" Dec 03 12:29:55 crc kubenswrapper[5052]: I1203 12:29:55.310707 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/cfb8e98d-3873-43e0-81dc-4450983179c2-combined-ca-bundle\") pod \"barbican-api-5fc7894bfd-g7hl5\" (UID: \"cfb8e98d-3873-43e0-81dc-4450983179c2\") " pod="openstack-kuttl-tests/barbican-api-5fc7894bfd-g7hl5" Dec 03 12:29:55 crc kubenswrapper[5052]: I1203 12:29:55.310737 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/cfb8e98d-3873-43e0-81dc-4450983179c2-internal-tls-certs\") pod \"barbican-api-5fc7894bfd-g7hl5\" (UID: \"cfb8e98d-3873-43e0-81dc-4450983179c2\") " pod="openstack-kuttl-tests/barbican-api-5fc7894bfd-g7hl5" Dec 03 12:29:55 crc kubenswrapper[5052]: I1203 12:29:55.310778 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/6927052c-212d-42fc-8236-b053ffc9a9d9-internal-tls-certs\") pod \"keystone-7cbf547d94-9krth\" (UID: \"6927052c-212d-42fc-8236-b053ffc9a9d9\") " pod="openstack-kuttl-tests/keystone-7cbf547d94-9krth" Dec 03 12:29:55 crc kubenswrapper[5052]: I1203 12:29:55.310813 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/cfb8e98d-3873-43e0-81dc-4450983179c2-config-data\") pod \"barbican-api-5fc7894bfd-g7hl5\" (UID: \"cfb8e98d-3873-43e0-81dc-4450983179c2\") " pod="openstack-kuttl-tests/barbican-api-5fc7894bfd-g7hl5" Dec 03 12:29:55 crc kubenswrapper[5052]: I1203 12:29:55.310829 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/6927052c-212d-42fc-8236-b053ffc9a9d9-scripts\") pod \"keystone-7cbf547d94-9krth\" (UID: \"6927052c-212d-42fc-8236-b053ffc9a9d9\") " pod="openstack-kuttl-tests/keystone-7cbf547d94-9krth" Dec 03 12:29:55 crc kubenswrapper[5052]: I1203 12:29:55.310858 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/8bddf2b1-8116-4b6e-9cd8-be8fe1cd4b64-config-data\") pod \"placement-5c6478db8b-bt2rw\" (UID: \"8bddf2b1-8116-4b6e-9cd8-be8fe1cd4b64\") " pod="openstack-kuttl-tests/placement-5c6478db8b-bt2rw" Dec 03 12:29:55 crc kubenswrapper[5052]: I1203 12:29:55.310885 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-nxlhc\" (UniqueName: \"kubernetes.io/projected/8bddf2b1-8116-4b6e-9cd8-be8fe1cd4b64-kube-api-access-nxlhc\") pod \"placement-5c6478db8b-bt2rw\" (UID: \"8bddf2b1-8116-4b6e-9cd8-be8fe1cd4b64\") " pod="openstack-kuttl-tests/placement-5c6478db8b-bt2rw" Dec 03 12:29:55 crc kubenswrapper[5052]: I1203 12:29:55.310908 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6927052c-212d-42fc-8236-b053ffc9a9d9-combined-ca-bundle\") pod \"keystone-7cbf547d94-9krth\" (UID: \"6927052c-212d-42fc-8236-b053ffc9a9d9\") " pod="openstack-kuttl-tests/keystone-7cbf547d94-9krth" Dec 03 12:29:55 crc kubenswrapper[5052]: I1203 12:29:55.310933 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/8bddf2b1-8116-4b6e-9cd8-be8fe1cd4b64-scripts\") pod \"placement-5c6478db8b-bt2rw\" (UID: \"8bddf2b1-8116-4b6e-9cd8-be8fe1cd4b64\") " pod="openstack-kuttl-tests/placement-5c6478db8b-bt2rw" Dec 03 12:29:55 crc kubenswrapper[5052]: I1203 12:29:55.310950 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/6927052c-212d-42fc-8236-b053ffc9a9d9-fernet-keys\") pod \"keystone-7cbf547d94-9krth\" (UID: \"6927052c-212d-42fc-8236-b053ffc9a9d9\") " pod="openstack-kuttl-tests/keystone-7cbf547d94-9krth" Dec 03 12:29:55 crc kubenswrapper[5052]: I1203 12:29:55.335800 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/8bddf2b1-8116-4b6e-9cd8-be8fe1cd4b64-logs\") pod \"placement-5c6478db8b-bt2rw\" (UID: \"8bddf2b1-8116-4b6e-9cd8-be8fe1cd4b64\") " pod="openstack-kuttl-tests/placement-5c6478db8b-bt2rw" Dec 03 12:29:55 crc kubenswrapper[5052]: I1203 12:29:55.337323 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-api-5fc7894bfd-g7hl5"] Dec 03 12:29:55 crc kubenswrapper[5052]: I1203 12:29:55.341283 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/cfb8e98d-3873-43e0-81dc-4450983179c2-config-data-custom\") pod \"barbican-api-5fc7894bfd-g7hl5\" (UID: \"cfb8e98d-3873-43e0-81dc-4450983179c2\") " pod="openstack-kuttl-tests/barbican-api-5fc7894bfd-g7hl5" Dec 03 12:29:55 crc kubenswrapper[5052]: I1203 12:29:55.357394 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/8bddf2b1-8116-4b6e-9cd8-be8fe1cd4b64-public-tls-certs\") pod \"placement-5c6478db8b-bt2rw\" (UID: \"8bddf2b1-8116-4b6e-9cd8-be8fe1cd4b64\") " pod="openstack-kuttl-tests/placement-5c6478db8b-bt2rw" Dec 03 12:29:55 crc kubenswrapper[5052]: I1203 12:29:55.358051 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/cfb8e98d-3873-43e0-81dc-4450983179c2-internal-tls-certs\") pod \"barbican-api-5fc7894bfd-g7hl5\" (UID: \"cfb8e98d-3873-43e0-81dc-4450983179c2\") " pod="openstack-kuttl-tests/barbican-api-5fc7894bfd-g7hl5" Dec 03 12:29:55 crc kubenswrapper[5052]: I1203 12:29:55.359030 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8bddf2b1-8116-4b6e-9cd8-be8fe1cd4b64-combined-ca-bundle\") pod \"placement-5c6478db8b-bt2rw\" (UID: \"8bddf2b1-8116-4b6e-9cd8-be8fe1cd4b64\") " pod="openstack-kuttl-tests/placement-5c6478db8b-bt2rw" Dec 03 12:29:55 crc kubenswrapper[5052]: I1203 12:29:55.362095 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-nxlhc\" (UniqueName: \"kubernetes.io/projected/8bddf2b1-8116-4b6e-9cd8-be8fe1cd4b64-kube-api-access-nxlhc\") pod \"placement-5c6478db8b-bt2rw\" (UID: \"8bddf2b1-8116-4b6e-9cd8-be8fe1cd4b64\") " pod="openstack-kuttl-tests/placement-5c6478db8b-bt2rw" Dec 03 12:29:55 crc kubenswrapper[5052]: I1203 12:29:55.363874 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/cfb8e98d-3873-43e0-81dc-4450983179c2-logs\") pod \"barbican-api-5fc7894bfd-g7hl5\" (UID: \"cfb8e98d-3873-43e0-81dc-4450983179c2\") " pod="openstack-kuttl-tests/barbican-api-5fc7894bfd-g7hl5" Dec 03 12:29:55 crc kubenswrapper[5052]: I1203 12:29:55.363930 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-gvz7n\" (UniqueName: \"kubernetes.io/projected/6927052c-212d-42fc-8236-b053ffc9a9d9-kube-api-access-gvz7n\") pod \"keystone-7cbf547d94-9krth\" (UID: \"6927052c-212d-42fc-8236-b053ffc9a9d9\") " pod="openstack-kuttl-tests/keystone-7cbf547d94-9krth" Dec 03 12:29:55 crc kubenswrapper[5052]: I1203 12:29:55.364627 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6927052c-212d-42fc-8236-b053ffc9a9d9-combined-ca-bundle\") pod \"keystone-7cbf547d94-9krth\" (UID: \"6927052c-212d-42fc-8236-b053ffc9a9d9\") " pod="openstack-kuttl-tests/keystone-7cbf547d94-9krth" Dec 03 12:29:55 crc kubenswrapper[5052]: I1203 12:29:55.364866 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/6927052c-212d-42fc-8236-b053ffc9a9d9-internal-tls-certs\") pod \"keystone-7cbf547d94-9krth\" (UID: \"6927052c-212d-42fc-8236-b053ffc9a9d9\") " pod="openstack-kuttl-tests/keystone-7cbf547d94-9krth" Dec 03 12:29:55 crc kubenswrapper[5052]: I1203 12:29:55.365121 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6927052c-212d-42fc-8236-b053ffc9a9d9-config-data\") pod \"keystone-7cbf547d94-9krth\" (UID: \"6927052c-212d-42fc-8236-b053ffc9a9d9\") " pod="openstack-kuttl-tests/keystone-7cbf547d94-9krth" Dec 03 12:29:55 crc kubenswrapper[5052]: I1203 12:29:55.365531 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/6927052c-212d-42fc-8236-b053ffc9a9d9-public-tls-certs\") pod \"keystone-7cbf547d94-9krth\" (UID: \"6927052c-212d-42fc-8236-b053ffc9a9d9\") " pod="openstack-kuttl-tests/keystone-7cbf547d94-9krth" Dec 03 12:29:55 crc kubenswrapper[5052]: I1203 12:29:55.367158 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-8sxqm\" (UniqueName: \"kubernetes.io/projected/cfb8e98d-3873-43e0-81dc-4450983179c2-kube-api-access-8sxqm\") pod \"barbican-api-5fc7894bfd-g7hl5\" (UID: \"cfb8e98d-3873-43e0-81dc-4450983179c2\") " pod="openstack-kuttl-tests/barbican-api-5fc7894bfd-g7hl5" Dec 03 12:29:55 crc kubenswrapper[5052]: I1203 12:29:55.368119 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/cfb8e98d-3873-43e0-81dc-4450983179c2-combined-ca-bundle\") pod \"barbican-api-5fc7894bfd-g7hl5\" (UID: \"cfb8e98d-3873-43e0-81dc-4450983179c2\") " pod="openstack-kuttl-tests/barbican-api-5fc7894bfd-g7hl5" Dec 03 12:29:55 crc kubenswrapper[5052]: I1203 12:29:55.374907 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/8bddf2b1-8116-4b6e-9cd8-be8fe1cd4b64-scripts\") pod \"placement-5c6478db8b-bt2rw\" (UID: \"8bddf2b1-8116-4b6e-9cd8-be8fe1cd4b64\") " pod="openstack-kuttl-tests/placement-5c6478db8b-bt2rw" Dec 03 12:29:55 crc kubenswrapper[5052]: I1203 12:29:55.405999 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/8bddf2b1-8116-4b6e-9cd8-be8fe1cd4b64-config-data\") pod \"placement-5c6478db8b-bt2rw\" (UID: \"8bddf2b1-8116-4b6e-9cd8-be8fe1cd4b64\") " pod="openstack-kuttl-tests/placement-5c6478db8b-bt2rw" Dec 03 12:29:55 crc kubenswrapper[5052]: I1203 12:29:55.437353 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/6927052c-212d-42fc-8236-b053ffc9a9d9-credential-keys\") pod \"keystone-7cbf547d94-9krth\" (UID: \"6927052c-212d-42fc-8236-b053ffc9a9d9\") " pod="openstack-kuttl-tests/keystone-7cbf547d94-9krth" Dec 03 12:29:55 crc kubenswrapper[5052]: I1203 12:29:55.443106 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/8bddf2b1-8116-4b6e-9cd8-be8fe1cd4b64-internal-tls-certs\") pod \"placement-5c6478db8b-bt2rw\" (UID: \"8bddf2b1-8116-4b6e-9cd8-be8fe1cd4b64\") " pod="openstack-kuttl-tests/placement-5c6478db8b-bt2rw" Dec 03 12:29:55 crc kubenswrapper[5052]: I1203 12:29:55.443260 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/6927052c-212d-42fc-8236-b053ffc9a9d9-fernet-keys\") pod \"keystone-7cbf547d94-9krth\" (UID: \"6927052c-212d-42fc-8236-b053ffc9a9d9\") " pod="openstack-kuttl-tests/keystone-7cbf547d94-9krth" Dec 03 12:29:55 crc kubenswrapper[5052]: I1203 12:29:55.444299 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/cfb8e98d-3873-43e0-81dc-4450983179c2-config-data\") pod \"barbican-api-5fc7894bfd-g7hl5\" (UID: \"cfb8e98d-3873-43e0-81dc-4450983179c2\") " pod="openstack-kuttl-tests/barbican-api-5fc7894bfd-g7hl5" Dec 03 12:29:55 crc kubenswrapper[5052]: I1203 12:29:55.445373 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/6927052c-212d-42fc-8236-b053ffc9a9d9-scripts\") pod \"keystone-7cbf547d94-9krth\" (UID: \"6927052c-212d-42fc-8236-b053ffc9a9d9\") " pod="openstack-kuttl-tests/keystone-7cbf547d94-9krth" Dec 03 12:29:55 crc kubenswrapper[5052]: I1203 12:29:55.451382 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/cfb8e98d-3873-43e0-81dc-4450983179c2-public-tls-certs\") pod \"barbican-api-5fc7894bfd-g7hl5\" (UID: \"cfb8e98d-3873-43e0-81dc-4450983179c2\") " pod="openstack-kuttl-tests/barbican-api-5fc7894bfd-g7hl5" Dec 03 12:29:55 crc kubenswrapper[5052]: I1203 12:29:55.471800 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-api-5fc7894bfd-g7hl5" Dec 03 12:29:55 crc kubenswrapper[5052]: I1203 12:29:55.506033 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-0"] Dec 03 12:29:55 crc kubenswrapper[5052]: I1203 12:29:55.506261 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-cell0-conductor-0" podUID="16a065b5-a603-4cb3-a4c0-b36c0b92bf3f" containerName="nova-cell0-conductor-conductor" containerID="cri-o://c1a2fa107302843e2249473b85f61687344a151311126ad578b5760b51b1b60e" gracePeriod=30 Dec 03 12:29:55 crc kubenswrapper[5052]: I1203 12:29:55.517176 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:29:55 crc kubenswrapper[5052]: I1203 12:29:55.518440 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-internal-api-0" podUID="82bcca5b-e2c3-43ce-aba3-eb879796dc3e" containerName="glance-log" containerID="cri-o://75b9c75f04326f0c84c32e2979b31896339503f5d49a49fbaac6f84c100d77b0" gracePeriod=30 Dec 03 12:29:55 crc kubenswrapper[5052]: I1203 12:29:55.518644 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-internal-api-0" podUID="82bcca5b-e2c3-43ce-aba3-eb879796dc3e" containerName="glance-httpd" containerID="cri-o://fa45d03e94f8aa43afec4ca3932bb3d4299d6fe6674927e304720e875b1c73d2" gracePeriod=30 Dec 03 12:29:55 crc kubenswrapper[5052]: I1203 12:29:55.526396 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/neutron-db474c7dd-tg7mq"] Dec 03 12:29:55 crc kubenswrapper[5052]: I1203 12:29:55.528164 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-db474c7dd-tg7mq" Dec 03 12:29:55 crc kubenswrapper[5052]: I1203 12:29:55.556204 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-7cbf547d94-9krth" Dec 03 12:29:55 crc kubenswrapper[5052]: I1203 12:29:55.568430 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/openstack-cell1-galera-0"] Dec 03 12:29:55 crc kubenswrapper[5052]: I1203 12:29:55.587849 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-sb-0"] Dec 03 12:29:55 crc kubenswrapper[5052]: I1203 12:29:55.588463 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ovsdbserver-sb-0" podUID="5c7fe95d-f637-4021-a5e2-69668046d2ff" containerName="openstack-network-exporter" containerID="cri-o://b01f8b34b21ba5bc171276b7695dd41167a5b350ce509d80a35ce85a684f8937" gracePeriod=300 Dec 03 12:29:55 crc kubenswrapper[5052]: I1203 12:29:55.596700 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/openstack-galera-0"] Dec 03 12:29:55 crc kubenswrapper[5052]: I1203 12:29:55.603902 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 12:29:55 crc kubenswrapper[5052]: I1203 12:29:55.604324 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/cinder-api-0" podUID="61cad274-57bd-4993-aaa6-3e6013900d88" containerName="cinder-api-log" containerID="cri-o://7a324498f8dc1fb392c84dac59b70dd32b53652f4626455f3411fdb91b9fb201" gracePeriod=30 Dec 03 12:29:55 crc kubenswrapper[5052]: I1203 12:29:55.604696 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/cinder-api-0" podUID="61cad274-57bd-4993-aaa6-3e6013900d88" containerName="cinder-api" containerID="cri-o://d1ad86af122d9acbc9e7b5bb95477effe197a4939198ba933eb8fb9e83c0f569" gracePeriod=30 Dec 03 12:29:55 crc kubenswrapper[5052]: I1203 12:29:55.614520 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-db474c7dd-tg7mq"] Dec 03 12:29:55 crc kubenswrapper[5052]: I1203 12:29:55.628151 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/886ea0a4-5096-4ea8-a74c-de243d1f4f40-public-tls-certs\") pod \"neutron-db474c7dd-tg7mq\" (UID: \"886ea0a4-5096-4ea8-a74c-de243d1f4f40\") " pod="openstack-kuttl-tests/neutron-db474c7dd-tg7mq" Dec 03 12:29:55 crc kubenswrapper[5052]: I1203 12:29:55.628207 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/886ea0a4-5096-4ea8-a74c-de243d1f4f40-ovndb-tls-certs\") pod \"neutron-db474c7dd-tg7mq\" (UID: \"886ea0a4-5096-4ea8-a74c-de243d1f4f40\") " pod="openstack-kuttl-tests/neutron-db474c7dd-tg7mq" Dec 03 12:29:55 crc kubenswrapper[5052]: I1203 12:29:55.628302 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/886ea0a4-5096-4ea8-a74c-de243d1f4f40-config\") pod \"neutron-db474c7dd-tg7mq\" (UID: \"886ea0a4-5096-4ea8-a74c-de243d1f4f40\") " pod="openstack-kuttl-tests/neutron-db474c7dd-tg7mq" Dec 03 12:29:55 crc kubenswrapper[5052]: I1203 12:29:55.628376 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-czgzb\" (UniqueName: \"kubernetes.io/projected/886ea0a4-5096-4ea8-a74c-de243d1f4f40-kube-api-access-czgzb\") pod \"neutron-db474c7dd-tg7mq\" (UID: \"886ea0a4-5096-4ea8-a74c-de243d1f4f40\") " pod="openstack-kuttl-tests/neutron-db474c7dd-tg7mq" Dec 03 12:29:55 crc kubenswrapper[5052]: I1203 12:29:55.628400 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/886ea0a4-5096-4ea8-a74c-de243d1f4f40-internal-tls-certs\") pod \"neutron-db474c7dd-tg7mq\" (UID: \"886ea0a4-5096-4ea8-a74c-de243d1f4f40\") " pod="openstack-kuttl-tests/neutron-db474c7dd-tg7mq" Dec 03 12:29:55 crc kubenswrapper[5052]: I1203 12:29:55.628450 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/886ea0a4-5096-4ea8-a74c-de243d1f4f40-httpd-config\") pod \"neutron-db474c7dd-tg7mq\" (UID: \"886ea0a4-5096-4ea8-a74c-de243d1f4f40\") " pod="openstack-kuttl-tests/neutron-db474c7dd-tg7mq" Dec 03 12:29:55 crc kubenswrapper[5052]: I1203 12:29:55.628467 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/886ea0a4-5096-4ea8-a74c-de243d1f4f40-combined-ca-bundle\") pod \"neutron-db474c7dd-tg7mq\" (UID: \"886ea0a4-5096-4ea8-a74c-de243d1f4f40\") " pod="openstack-kuttl-tests/neutron-db474c7dd-tg7mq" Dec 03 12:29:55 crc kubenswrapper[5052]: I1203 12:29:55.719262 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-5c6478db8b-bt2rw" Dec 03 12:29:55 crc kubenswrapper[5052]: I1203 12:29:55.730243 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-czgzb\" (UniqueName: \"kubernetes.io/projected/886ea0a4-5096-4ea8-a74c-de243d1f4f40-kube-api-access-czgzb\") pod \"neutron-db474c7dd-tg7mq\" (UID: \"886ea0a4-5096-4ea8-a74c-de243d1f4f40\") " pod="openstack-kuttl-tests/neutron-db474c7dd-tg7mq" Dec 03 12:29:55 crc kubenswrapper[5052]: I1203 12:29:55.730305 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/886ea0a4-5096-4ea8-a74c-de243d1f4f40-internal-tls-certs\") pod \"neutron-db474c7dd-tg7mq\" (UID: \"886ea0a4-5096-4ea8-a74c-de243d1f4f40\") " pod="openstack-kuttl-tests/neutron-db474c7dd-tg7mq" Dec 03 12:29:55 crc kubenswrapper[5052]: I1203 12:29:55.730380 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/886ea0a4-5096-4ea8-a74c-de243d1f4f40-httpd-config\") pod \"neutron-db474c7dd-tg7mq\" (UID: \"886ea0a4-5096-4ea8-a74c-de243d1f4f40\") " pod="openstack-kuttl-tests/neutron-db474c7dd-tg7mq" Dec 03 12:29:55 crc kubenswrapper[5052]: I1203 12:29:55.730398 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/886ea0a4-5096-4ea8-a74c-de243d1f4f40-combined-ca-bundle\") pod \"neutron-db474c7dd-tg7mq\" (UID: \"886ea0a4-5096-4ea8-a74c-de243d1f4f40\") " pod="openstack-kuttl-tests/neutron-db474c7dd-tg7mq" Dec 03 12:29:55 crc kubenswrapper[5052]: I1203 12:29:55.730416 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/886ea0a4-5096-4ea8-a74c-de243d1f4f40-public-tls-certs\") pod \"neutron-db474c7dd-tg7mq\" (UID: \"886ea0a4-5096-4ea8-a74c-de243d1f4f40\") " pod="openstack-kuttl-tests/neutron-db474c7dd-tg7mq" Dec 03 12:29:55 crc kubenswrapper[5052]: I1203 12:29:55.730466 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/886ea0a4-5096-4ea8-a74c-de243d1f4f40-ovndb-tls-certs\") pod \"neutron-db474c7dd-tg7mq\" (UID: \"886ea0a4-5096-4ea8-a74c-de243d1f4f40\") " pod="openstack-kuttl-tests/neutron-db474c7dd-tg7mq" Dec 03 12:29:55 crc kubenswrapper[5052]: I1203 12:29:55.730618 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/886ea0a4-5096-4ea8-a74c-de243d1f4f40-config\") pod \"neutron-db474c7dd-tg7mq\" (UID: \"886ea0a4-5096-4ea8-a74c-de243d1f4f40\") " pod="openstack-kuttl-tests/neutron-db474c7dd-tg7mq" Dec 03 12:29:55 crc kubenswrapper[5052]: I1203 12:29:55.737995 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ovsdbserver-sb-0" podUID="5c7fe95d-f637-4021-a5e2-69668046d2ff" containerName="ovsdbserver-sb" containerID="cri-o://63b5496e0ce2e84297ac272518addd75c23a27efda2a42a79c0e221732c3c3c6" gracePeriod=300 Dec 03 12:29:55 crc kubenswrapper[5052]: I1203 12:29:55.740417 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/886ea0a4-5096-4ea8-a74c-de243d1f4f40-internal-tls-certs\") pod \"neutron-db474c7dd-tg7mq\" (UID: \"886ea0a4-5096-4ea8-a74c-de243d1f4f40\") " pod="openstack-kuttl-tests/neutron-db474c7dd-tg7mq" Dec 03 12:29:55 crc kubenswrapper[5052]: I1203 12:29:55.753641 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/886ea0a4-5096-4ea8-a74c-de243d1f4f40-combined-ca-bundle\") pod \"neutron-db474c7dd-tg7mq\" (UID: \"886ea0a4-5096-4ea8-a74c-de243d1f4f40\") " pod="openstack-kuttl-tests/neutron-db474c7dd-tg7mq" Dec 03 12:29:55 crc kubenswrapper[5052]: I1203 12:29:55.770892 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/886ea0a4-5096-4ea8-a74c-de243d1f4f40-httpd-config\") pod \"neutron-db474c7dd-tg7mq\" (UID: \"886ea0a4-5096-4ea8-a74c-de243d1f4f40\") " pod="openstack-kuttl-tests/neutron-db474c7dd-tg7mq" Dec 03 12:29:55 crc kubenswrapper[5052]: I1203 12:29:55.772635 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/886ea0a4-5096-4ea8-a74c-de243d1f4f40-ovndb-tls-certs\") pod \"neutron-db474c7dd-tg7mq\" (UID: \"886ea0a4-5096-4ea8-a74c-de243d1f4f40\") " pod="openstack-kuttl-tests/neutron-db474c7dd-tg7mq" Dec 03 12:29:55 crc kubenswrapper[5052]: I1203 12:29:55.775036 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/886ea0a4-5096-4ea8-a74c-de243d1f4f40-config\") pod \"neutron-db474c7dd-tg7mq\" (UID: \"886ea0a4-5096-4ea8-a74c-de243d1f4f40\") " pod="openstack-kuttl-tests/neutron-db474c7dd-tg7mq" Dec 03 12:29:55 crc kubenswrapper[5052]: I1203 12:29:55.776718 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-czgzb\" (UniqueName: \"kubernetes.io/projected/886ea0a4-5096-4ea8-a74c-de243d1f4f40-kube-api-access-czgzb\") pod \"neutron-db474c7dd-tg7mq\" (UID: \"886ea0a4-5096-4ea8-a74c-de243d1f4f40\") " pod="openstack-kuttl-tests/neutron-db474c7dd-tg7mq" Dec 03 12:29:55 crc kubenswrapper[5052]: I1203 12:29:55.780723 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/886ea0a4-5096-4ea8-a74c-de243d1f4f40-public-tls-certs\") pod \"neutron-db474c7dd-tg7mq\" (UID: \"886ea0a4-5096-4ea8-a74c-de243d1f4f40\") " pod="openstack-kuttl-tests/neutron-db474c7dd-tg7mq" Dec 03 12:29:55 crc kubenswrapper[5052]: I1203 12:29:55.822758 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-keystone-listener-77b6bbbdcb-4jlsc"] Dec 03 12:29:55 crc kubenswrapper[5052]: I1203 12:29:55.932820 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/openstack-cell1-galera-0" podUID="1de1d25b-cbcc-4d08-a7cd-aa54ac71aecb" containerName="galera" containerID="cri-o://46246c5deed2c10d0bc81011e4be47e5c70c228f4dda6fb9a1b20810aff514c7" gracePeriod=30 Dec 03 12:29:55 crc kubenswrapper[5052]: I1203 12:29:55.942109 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/openstack-galera-0" podUID="12f39c79-eeb9-4cf4-8553-1a4d9a07ee82" containerName="galera" containerID="cri-o://165a4ba65f86633615c852ba29ceb303c3bbd0dbd696084cf0c3a26afe0a4424" gracePeriod=30 Dec 03 12:29:55 crc kubenswrapper[5052]: I1203 12:29:55.998836 5052 generic.go:334] "Generic (PLEG): container finished" podID="214a26fb-e9b2-4a87-8fa5-ae2ed0d8a828" containerID="27a238af716222ccb80b51b0ddb93658176d3aa5f8db130deec5a43ab67bb332" exitCode=0 Dec 03 12:29:55 crc kubenswrapper[5052]: I1203 12:29:55.999167 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"214a26fb-e9b2-4a87-8fa5-ae2ed0d8a828","Type":"ContainerDied","Data":"27a238af716222ccb80b51b0ddb93658176d3aa5f8db130deec5a43ab67bb332"} Dec 03 12:29:56 crc kubenswrapper[5052]: I1203 12:29:56.001561 5052 generic.go:334] "Generic (PLEG): container finished" podID="b35e6939-8f23-496c-ac55-9534be4ea8be" containerID="8d5e1c2473a3c27b159232ab3f630f8e1ca663780d8fc9b7b1eaac3611c97961" exitCode=0 Dec 03 12:29:56 crc kubenswrapper[5052]: I1203 12:29:56.001632 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/memcached-0" event={"ID":"b35e6939-8f23-496c-ac55-9534be4ea8be","Type":"ContainerDied","Data":"8d5e1c2473a3c27b159232ab3f630f8e1ca663780d8fc9b7b1eaac3611c97961"} Dec 03 12:29:56 crc kubenswrapper[5052]: I1203 12:29:56.003777 5052 generic.go:334] "Generic (PLEG): container finished" podID="caaecb05-f5f1-421d-8e91-818774a2c0ec" containerID="cda20389ae104a8a76ec7b7e03e816da57afd5a07e90e7f0e1bf79a5704aa60d" exitCode=143 Dec 03 12:29:56 crc kubenswrapper[5052]: I1203 12:29:56.003832 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"caaecb05-f5f1-421d-8e91-818774a2c0ec","Type":"ContainerDied","Data":"cda20389ae104a8a76ec7b7e03e816da57afd5a07e90e7f0e1bf79a5704aa60d"} Dec 03 12:29:56 crc kubenswrapper[5052]: I1203 12:29:56.006456 5052 generic.go:334] "Generic (PLEG): container finished" podID="b5b5ad16-1e45-4761-b5d2-60ffd5934ccb" containerID="0e4e8a3b2fb3bbe0432215ed54bf039b98ca0a274e59dc10062d81386889219c" exitCode=143 Dec 03 12:29:56 crc kubenswrapper[5052]: I1203 12:29:56.006544 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"b5b5ad16-1e45-4761-b5d2-60ffd5934ccb","Type":"ContainerDied","Data":"0e4e8a3b2fb3bbe0432215ed54bf039b98ca0a274e59dc10062d81386889219c"} Dec 03 12:29:56 crc kubenswrapper[5052]: I1203 12:29:56.008360 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_ovsdbserver-sb-0_5c7fe95d-f637-4021-a5e2-69668046d2ff/ovsdbserver-sb/0.log" Dec 03 12:29:56 crc kubenswrapper[5052]: I1203 12:29:56.008393 5052 generic.go:334] "Generic (PLEG): container finished" podID="5c7fe95d-f637-4021-a5e2-69668046d2ff" containerID="b01f8b34b21ba5bc171276b7695dd41167a5b350ce509d80a35ce85a684f8937" exitCode=2 Dec 03 12:29:56 crc kubenswrapper[5052]: I1203 12:29:56.008415 5052 generic.go:334] "Generic (PLEG): container finished" podID="5c7fe95d-f637-4021-a5e2-69668046d2ff" containerID="63b5496e0ce2e84297ac272518addd75c23a27efda2a42a79c0e221732c3c3c6" exitCode=143 Dec 03 12:29:56 crc kubenswrapper[5052]: I1203 12:29:56.008473 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-sb-0" event={"ID":"5c7fe95d-f637-4021-a5e2-69668046d2ff","Type":"ContainerDied","Data":"b01f8b34b21ba5bc171276b7695dd41167a5b350ce509d80a35ce85a684f8937"} Dec 03 12:29:56 crc kubenswrapper[5052]: I1203 12:29:56.008493 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-sb-0" event={"ID":"5c7fe95d-f637-4021-a5e2-69668046d2ff","Type":"ContainerDied","Data":"63b5496e0ce2e84297ac272518addd75c23a27efda2a42a79c0e221732c3c3c6"} Dec 03 12:29:56 crc kubenswrapper[5052]: I1203 12:29:56.018010 5052 generic.go:334] "Generic (PLEG): container finished" podID="61cad274-57bd-4993-aaa6-3e6013900d88" containerID="7a324498f8dc1fb392c84dac59b70dd32b53652f4626455f3411fdb91b9fb201" exitCode=143 Dec 03 12:29:56 crc kubenswrapper[5052]: I1203 12:29:56.018092 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"61cad274-57bd-4993-aaa6-3e6013900d88","Type":"ContainerDied","Data":"7a324498f8dc1fb392c84dac59b70dd32b53652f4626455f3411fdb91b9fb201"} Dec 03 12:29:56 crc kubenswrapper[5052]: I1203 12:29:56.031590 5052 generic.go:334] "Generic (PLEG): container finished" podID="a26ed42f-71e7-4c92-8b7e-ad194358d3e5" containerID="911da6cb170ba67cad5584869214b5ef5f33b8f7400194bc4bc1ce4ca1a692f4" exitCode=2 Dec 03 12:29:56 crc kubenswrapper[5052]: I1203 12:29:56.031640 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovn-northd-0" event={"ID":"a26ed42f-71e7-4c92-8b7e-ad194358d3e5","Type":"ContainerDied","Data":"911da6cb170ba67cad5584869214b5ef5f33b8f7400194bc4bc1ce4ca1a692f4"} Dec 03 12:29:56 crc kubenswrapper[5052]: I1203 12:29:56.041949 5052 generic.go:334] "Generic (PLEG): container finished" podID="05b9d3cc-93de-45a6-8849-027a4aad3715" containerID="07f1af7d6a787079787ad912bf9e1ca56fb4cb7d8e231d7b665fb887db97be81" exitCode=143 Dec 03 12:29:56 crc kubenswrapper[5052]: I1203 12:29:56.042042 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"05b9d3cc-93de-45a6-8849-027a4aad3715","Type":"ContainerDied","Data":"07f1af7d6a787079787ad912bf9e1ca56fb4cb7d8e231d7b665fb887db97be81"} Dec 03 12:29:56 crc kubenswrapper[5052]: I1203 12:29:56.078311 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-api-5fc7894bfd-g7hl5"] Dec 03 12:29:56 crc kubenswrapper[5052]: I1203 12:29:56.093152 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-worker-6d687d9c57-4gplx"] Dec 03 12:29:56 crc kubenswrapper[5052]: I1203 12:29:56.119098 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-db474c7dd-tg7mq" Dec 03 12:29:56 crc kubenswrapper[5052]: I1203 12:29:56.129048 5052 generic.go:334] "Generic (PLEG): container finished" podID="82bcca5b-e2c3-43ce-aba3-eb879796dc3e" containerID="75b9c75f04326f0c84c32e2979b31896339503f5d49a49fbaac6f84c100d77b0" exitCode=143 Dec 03 12:29:56 crc kubenswrapper[5052]: I1203 12:29:56.129108 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"82bcca5b-e2c3-43ce-aba3-eb879796dc3e","Type":"ContainerDied","Data":"75b9c75f04326f0c84c32e2979b31896339503f5d49a49fbaac6f84c100d77b0"} Dec 03 12:29:56 crc kubenswrapper[5052]: I1203 12:29:56.163357 5052 generic.go:334] "Generic (PLEG): container finished" podID="53193ff3-1a41-4e26-b38f-957088aa7e8b" containerID="894cb495101405b55444078f27862c4a46c3937975fecb53b501499a7dc2a56d" exitCode=0 Dec 03 12:29:56 crc kubenswrapper[5052]: I1203 12:29:56.163413 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" event={"ID":"53193ff3-1a41-4e26-b38f-957088aa7e8b","Type":"ContainerDied","Data":"894cb495101405b55444078f27862c4a46c3937975fecb53b501499a7dc2a56d"} Dec 03 12:29:56 crc kubenswrapper[5052]: I1203 12:29:56.198521 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_ovsdbserver-nb-0_f45e9eda-e62f-4307-ae18-2a8da7d6fe7c/ovsdbserver-nb/0.log" Dec 03 12:29:56 crc kubenswrapper[5052]: I1203 12:29:56.198580 5052 generic.go:334] "Generic (PLEG): container finished" podID="f45e9eda-e62f-4307-ae18-2a8da7d6fe7c" containerID="901636e0be8460bcc74b0c6a41492e349cbdbd334bbc41042f9dfdf0a187e9d1" exitCode=2 Dec 03 12:29:56 crc kubenswrapper[5052]: I1203 12:29:56.198604 5052 generic.go:334] "Generic (PLEG): container finished" podID="f45e9eda-e62f-4307-ae18-2a8da7d6fe7c" containerID="09275335090deb24b922ed4694cb037b39bf0e6b72303e640ea22dfe86fa1716" exitCode=143 Dec 03 12:29:56 crc kubenswrapper[5052]: I1203 12:29:56.198630 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-nb-0" event={"ID":"f45e9eda-e62f-4307-ae18-2a8da7d6fe7c","Type":"ContainerDied","Data":"901636e0be8460bcc74b0c6a41492e349cbdbd334bbc41042f9dfdf0a187e9d1"} Dec 03 12:29:56 crc kubenswrapper[5052]: I1203 12:29:56.198663 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-nb-0" event={"ID":"f45e9eda-e62f-4307-ae18-2a8da7d6fe7c","Type":"ContainerDied","Data":"09275335090deb24b922ed4694cb037b39bf0e6b72303e640ea22dfe86fa1716"} Dec 03 12:29:56 crc kubenswrapper[5052]: I1203 12:29:56.393573 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-worker-6d687d9c57-4gplx"] Dec 03 12:29:56 crc kubenswrapper[5052]: I1203 12:29:56.430166 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-keystone-listener-77b6bbbdcb-4jlsc"] Dec 03 12:29:56 crc kubenswrapper[5052]: I1203 12:29:56.482118 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-0"] Dec 03 12:29:56 crc kubenswrapper[5052]: I1203 12:29:56.482431 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-cell1-conductor-0" podUID="733f463b-dcf8-4a61-8e4e-4bf1c042d397" containerName="nova-cell1-conductor-conductor" containerID="cri-o://fc5e4fca262b35b92bb410a508d650dc4bdca0fc0848d2a90abb1ede218d5a33" gracePeriod=30 Dec 03 12:29:56 crc kubenswrapper[5052]: I1203 12:29:56.684205 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/barbican-keystone-listener-cd5b58cfd-jf8vt"] Dec 03 12:29:56 crc kubenswrapper[5052]: I1203 12:29:56.686659 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-keystone-listener-cd5b58cfd-jf8vt" Dec 03 12:29:56 crc kubenswrapper[5052]: I1203 12:29:56.799983 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-keystone-listener-cd5b58cfd-jf8vt"] Dec 03 12:29:56 crc kubenswrapper[5052]: I1203 12:29:56.923592 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-7cbf547d94-9krth"] Dec 03 12:29:56 crc kubenswrapper[5052]: I1203 12:29:56.925059 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/019689d4-d40b-4e9a-a028-e427f1aa9d9b-config-data\") pod \"barbican-keystone-listener-cd5b58cfd-jf8vt\" (UID: \"019689d4-d40b-4e9a-a028-e427f1aa9d9b\") " pod="openstack-kuttl-tests/barbican-keystone-listener-cd5b58cfd-jf8vt" Dec 03 12:29:56 crc kubenswrapper[5052]: I1203 12:29:56.925109 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/019689d4-d40b-4e9a-a028-e427f1aa9d9b-logs\") pod \"barbican-keystone-listener-cd5b58cfd-jf8vt\" (UID: \"019689d4-d40b-4e9a-a028-e427f1aa9d9b\") " pod="openstack-kuttl-tests/barbican-keystone-listener-cd5b58cfd-jf8vt" Dec 03 12:29:56 crc kubenswrapper[5052]: I1203 12:29:56.925169 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/019689d4-d40b-4e9a-a028-e427f1aa9d9b-config-data-custom\") pod \"barbican-keystone-listener-cd5b58cfd-jf8vt\" (UID: \"019689d4-d40b-4e9a-a028-e427f1aa9d9b\") " pod="openstack-kuttl-tests/barbican-keystone-listener-cd5b58cfd-jf8vt" Dec 03 12:29:56 crc kubenswrapper[5052]: I1203 12:29:56.925212 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6rggl\" (UniqueName: \"kubernetes.io/projected/019689d4-d40b-4e9a-a028-e427f1aa9d9b-kube-api-access-6rggl\") pod \"barbican-keystone-listener-cd5b58cfd-jf8vt\" (UID: \"019689d4-d40b-4e9a-a028-e427f1aa9d9b\") " pod="openstack-kuttl-tests/barbican-keystone-listener-cd5b58cfd-jf8vt" Dec 03 12:29:56 crc kubenswrapper[5052]: I1203 12:29:56.925345 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/019689d4-d40b-4e9a-a028-e427f1aa9d9b-combined-ca-bundle\") pod \"barbican-keystone-listener-cd5b58cfd-jf8vt\" (UID: \"019689d4-d40b-4e9a-a028-e427f1aa9d9b\") " pod="openstack-kuttl-tests/barbican-keystone-listener-cd5b58cfd-jf8vt" Dec 03 12:29:56 crc kubenswrapper[5052]: I1203 12:29:56.966198 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/barbican-worker-644558fbc5-4gtxj"] Dec 03 12:29:56 crc kubenswrapper[5052]: I1203 12:29:56.967965 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-worker-644558fbc5-4gtxj" Dec 03 12:29:56 crc kubenswrapper[5052]: I1203 12:29:56.981123 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-worker-644558fbc5-4gtxj"] Dec 03 12:29:57 crc kubenswrapper[5052]: I1203 12:29:57.026576 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/019689d4-d40b-4e9a-a028-e427f1aa9d9b-combined-ca-bundle\") pod \"barbican-keystone-listener-cd5b58cfd-jf8vt\" (UID: \"019689d4-d40b-4e9a-a028-e427f1aa9d9b\") " pod="openstack-kuttl-tests/barbican-keystone-listener-cd5b58cfd-jf8vt" Dec 03 12:29:57 crc kubenswrapper[5052]: I1203 12:29:57.026766 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/019689d4-d40b-4e9a-a028-e427f1aa9d9b-config-data\") pod \"barbican-keystone-listener-cd5b58cfd-jf8vt\" (UID: \"019689d4-d40b-4e9a-a028-e427f1aa9d9b\") " pod="openstack-kuttl-tests/barbican-keystone-listener-cd5b58cfd-jf8vt" Dec 03 12:29:57 crc kubenswrapper[5052]: I1203 12:29:57.026809 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/019689d4-d40b-4e9a-a028-e427f1aa9d9b-logs\") pod \"barbican-keystone-listener-cd5b58cfd-jf8vt\" (UID: \"019689d4-d40b-4e9a-a028-e427f1aa9d9b\") " pod="openstack-kuttl-tests/barbican-keystone-listener-cd5b58cfd-jf8vt" Dec 03 12:29:57 crc kubenswrapper[5052]: I1203 12:29:57.026863 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/019689d4-d40b-4e9a-a028-e427f1aa9d9b-config-data-custom\") pod \"barbican-keystone-listener-cd5b58cfd-jf8vt\" (UID: \"019689d4-d40b-4e9a-a028-e427f1aa9d9b\") " pod="openstack-kuttl-tests/barbican-keystone-listener-cd5b58cfd-jf8vt" Dec 03 12:29:57 crc kubenswrapper[5052]: I1203 12:29:57.026902 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-6rggl\" (UniqueName: \"kubernetes.io/projected/019689d4-d40b-4e9a-a028-e427f1aa9d9b-kube-api-access-6rggl\") pod \"barbican-keystone-listener-cd5b58cfd-jf8vt\" (UID: \"019689d4-d40b-4e9a-a028-e427f1aa9d9b\") " pod="openstack-kuttl-tests/barbican-keystone-listener-cd5b58cfd-jf8vt" Dec 03 12:29:57 crc kubenswrapper[5052]: I1203 12:29:57.028463 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/019689d4-d40b-4e9a-a028-e427f1aa9d9b-logs\") pod \"barbican-keystone-listener-cd5b58cfd-jf8vt\" (UID: \"019689d4-d40b-4e9a-a028-e427f1aa9d9b\") " pod="openstack-kuttl-tests/barbican-keystone-listener-cd5b58cfd-jf8vt" Dec 03 12:29:57 crc kubenswrapper[5052]: I1203 12:29:57.038836 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/019689d4-d40b-4e9a-a028-e427f1aa9d9b-config-data\") pod \"barbican-keystone-listener-cd5b58cfd-jf8vt\" (UID: \"019689d4-d40b-4e9a-a028-e427f1aa9d9b\") " pod="openstack-kuttl-tests/barbican-keystone-listener-cd5b58cfd-jf8vt" Dec 03 12:29:57 crc kubenswrapper[5052]: I1203 12:29:57.044104 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/019689d4-d40b-4e9a-a028-e427f1aa9d9b-config-data-custom\") pod \"barbican-keystone-listener-cd5b58cfd-jf8vt\" (UID: \"019689d4-d40b-4e9a-a028-e427f1aa9d9b\") " pod="openstack-kuttl-tests/barbican-keystone-listener-cd5b58cfd-jf8vt" Dec 03 12:29:57 crc kubenswrapper[5052]: I1203 12:29:57.099380 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/019689d4-d40b-4e9a-a028-e427f1aa9d9b-combined-ca-bundle\") pod \"barbican-keystone-listener-cd5b58cfd-jf8vt\" (UID: \"019689d4-d40b-4e9a-a028-e427f1aa9d9b\") " pod="openstack-kuttl-tests/barbican-keystone-listener-cd5b58cfd-jf8vt" Dec 03 12:29:57 crc kubenswrapper[5052]: I1203 12:29:57.115164 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:29:57 crc kubenswrapper[5052]: I1203 12:29:57.115530 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="e75eea2c-2eb1-4f0f-97e5-63ce147e68dd" containerName="ceilometer-central-agent" containerID="cri-o://a6f27ec51db1c9719c74272ae88d2ed48068153f9cc4d46a92cce3985e31c2e5" gracePeriod=30 Dec 03 12:29:57 crc kubenswrapper[5052]: I1203 12:29:57.116019 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="e75eea2c-2eb1-4f0f-97e5-63ce147e68dd" containerName="proxy-httpd" containerID="cri-o://da3d10f0fb54aeb0fa5fdc57f022b12303d52fa87da0b230d95a66aac9d2be59" gracePeriod=30 Dec 03 12:29:57 crc kubenswrapper[5052]: I1203 12:29:57.116737 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="e75eea2c-2eb1-4f0f-97e5-63ce147e68dd" containerName="ceilometer-notification-agent" containerID="cri-o://069e9d7b5055dc14bd89a6f68f4d0ccea363e1ff3aad490e83aff00d8a0ad411" gracePeriod=30 Dec 03 12:29:57 crc kubenswrapper[5052]: I1203 12:29:57.116877 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="e75eea2c-2eb1-4f0f-97e5-63ce147e68dd" containerName="sg-core" containerID="cri-o://102038f6978c676a1d8853f850be294a8aea8d6b74a6f1949c24231d714a3608" gracePeriod=30 Dec 03 12:29:57 crc kubenswrapper[5052]: I1203 12:29:57.126049 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-6rggl\" (UniqueName: \"kubernetes.io/projected/019689d4-d40b-4e9a-a028-e427f1aa9d9b-kube-api-access-6rggl\") pod \"barbican-keystone-listener-cd5b58cfd-jf8vt\" (UID: \"019689d4-d40b-4e9a-a028-e427f1aa9d9b\") " pod="openstack-kuttl-tests/barbican-keystone-listener-cd5b58cfd-jf8vt" Dec 03 12:29:57 crc kubenswrapper[5052]: I1203 12:29:57.128721 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/8ac6e8f0-d49c-4a4c-846a-6cec226bd360-logs\") pod \"barbican-worker-644558fbc5-4gtxj\" (UID: \"8ac6e8f0-d49c-4a4c-846a-6cec226bd360\") " pod="openstack-kuttl-tests/barbican-worker-644558fbc5-4gtxj" Dec 03 12:29:57 crc kubenswrapper[5052]: I1203 12:29:57.128778 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8ac6e8f0-d49c-4a4c-846a-6cec226bd360-combined-ca-bundle\") pod \"barbican-worker-644558fbc5-4gtxj\" (UID: \"8ac6e8f0-d49c-4a4c-846a-6cec226bd360\") " pod="openstack-kuttl-tests/barbican-worker-644558fbc5-4gtxj" Dec 03 12:29:57 crc kubenswrapper[5052]: I1203 12:29:57.128803 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/8ac6e8f0-d49c-4a4c-846a-6cec226bd360-config-data-custom\") pod \"barbican-worker-644558fbc5-4gtxj\" (UID: \"8ac6e8f0-d49c-4a4c-846a-6cec226bd360\") " pod="openstack-kuttl-tests/barbican-worker-644558fbc5-4gtxj" Dec 03 12:29:57 crc kubenswrapper[5052]: I1203 12:29:57.128877 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fqrfd\" (UniqueName: \"kubernetes.io/projected/8ac6e8f0-d49c-4a4c-846a-6cec226bd360-kube-api-access-fqrfd\") pod \"barbican-worker-644558fbc5-4gtxj\" (UID: \"8ac6e8f0-d49c-4a4c-846a-6cec226bd360\") " pod="openstack-kuttl-tests/barbican-worker-644558fbc5-4gtxj" Dec 03 12:29:57 crc kubenswrapper[5052]: I1203 12:29:57.128930 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/8ac6e8f0-d49c-4a4c-846a-6cec226bd360-config-data\") pod \"barbican-worker-644558fbc5-4gtxj\" (UID: \"8ac6e8f0-d49c-4a4c-846a-6cec226bd360\") " pod="openstack-kuttl-tests/barbican-worker-644558fbc5-4gtxj" Dec 03 12:29:57 crc kubenswrapper[5052]: E1203 12:29:57.129391 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="6f35ce14790c9ffb5796ad1467515b2be517ae60eb2573a1571ba5f99f630cad" cmd=["/usr/bin/pgrep","-r","DRST","nova-scheduler"] Dec 03 12:29:57 crc kubenswrapper[5052]: I1203 12:29:57.157955 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/neutron-db474c7dd-tg7mq"] Dec 03 12:29:57 crc kubenswrapper[5052]: E1203 12:29:57.183166 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="6f35ce14790c9ffb5796ad1467515b2be517ae60eb2573a1571ba5f99f630cad" cmd=["/usr/bin/pgrep","-r","DRST","nova-scheduler"] Dec 03 12:29:57 crc kubenswrapper[5052]: I1203 12:29:57.190578 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-api-5fc7894bfd-g7hl5"] Dec 03 12:29:57 crc kubenswrapper[5052]: E1203 12:29:57.199963 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="6f35ce14790c9ffb5796ad1467515b2be517ae60eb2573a1571ba5f99f630cad" cmd=["/usr/bin/pgrep","-r","DRST","nova-scheduler"] Dec 03 12:29:57 crc kubenswrapper[5052]: E1203 12:29:57.200039 5052 prober.go:104] "Probe errored" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" probeType="Readiness" pod="openstack-kuttl-tests/nova-scheduler-0" podUID="b953ffa0-088c-4450-8d60-04044b33cfa0" containerName="nova-scheduler-scheduler" Dec 03 12:29:57 crc kubenswrapper[5052]: E1203 12:29:57.200097 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="fc5e4fca262b35b92bb410a508d650dc4bdca0fc0848d2a90abb1ede218d5a33" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:29:57 crc kubenswrapper[5052]: E1203 12:29:57.208957 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="fc5e4fca262b35b92bb410a508d650dc4bdca0fc0848d2a90abb1ede218d5a33" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:29:57 crc kubenswrapper[5052]: I1203 12:29:57.230333 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-fqrfd\" (UniqueName: \"kubernetes.io/projected/8ac6e8f0-d49c-4a4c-846a-6cec226bd360-kube-api-access-fqrfd\") pod \"barbican-worker-644558fbc5-4gtxj\" (UID: \"8ac6e8f0-d49c-4a4c-846a-6cec226bd360\") " pod="openstack-kuttl-tests/barbican-worker-644558fbc5-4gtxj" Dec 03 12:29:57 crc kubenswrapper[5052]: I1203 12:29:57.230415 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/8ac6e8f0-d49c-4a4c-846a-6cec226bd360-config-data\") pod \"barbican-worker-644558fbc5-4gtxj\" (UID: \"8ac6e8f0-d49c-4a4c-846a-6cec226bd360\") " pod="openstack-kuttl-tests/barbican-worker-644558fbc5-4gtxj" Dec 03 12:29:57 crc kubenswrapper[5052]: I1203 12:29:57.230582 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/8ac6e8f0-d49c-4a4c-846a-6cec226bd360-logs\") pod \"barbican-worker-644558fbc5-4gtxj\" (UID: \"8ac6e8f0-d49c-4a4c-846a-6cec226bd360\") " pod="openstack-kuttl-tests/barbican-worker-644558fbc5-4gtxj" Dec 03 12:29:57 crc kubenswrapper[5052]: I1203 12:29:57.230606 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8ac6e8f0-d49c-4a4c-846a-6cec226bd360-combined-ca-bundle\") pod \"barbican-worker-644558fbc5-4gtxj\" (UID: \"8ac6e8f0-d49c-4a4c-846a-6cec226bd360\") " pod="openstack-kuttl-tests/barbican-worker-644558fbc5-4gtxj" Dec 03 12:29:57 crc kubenswrapper[5052]: I1203 12:29:57.230642 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/8ac6e8f0-d49c-4a4c-846a-6cec226bd360-config-data-custom\") pod \"barbican-worker-644558fbc5-4gtxj\" (UID: \"8ac6e8f0-d49c-4a4c-846a-6cec226bd360\") " pod="openstack-kuttl-tests/barbican-worker-644558fbc5-4gtxj" Dec 03 12:29:57 crc kubenswrapper[5052]: I1203 12:29:57.231976 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/8ac6e8f0-d49c-4a4c-846a-6cec226bd360-logs\") pod \"barbican-worker-644558fbc5-4gtxj\" (UID: \"8ac6e8f0-d49c-4a4c-846a-6cec226bd360\") " pod="openstack-kuttl-tests/barbican-worker-644558fbc5-4gtxj" Dec 03 12:29:57 crc kubenswrapper[5052]: I1203 12:29:57.236939 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/8ac6e8f0-d49c-4a4c-846a-6cec226bd360-config-data\") pod \"barbican-worker-644558fbc5-4gtxj\" (UID: \"8ac6e8f0-d49c-4a4c-846a-6cec226bd360\") " pod="openstack-kuttl-tests/barbican-worker-644558fbc5-4gtxj" Dec 03 12:29:57 crc kubenswrapper[5052]: I1203 12:29:57.237429 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/8ac6e8f0-d49c-4a4c-846a-6cec226bd360-config-data-custom\") pod \"barbican-worker-644558fbc5-4gtxj\" (UID: \"8ac6e8f0-d49c-4a4c-846a-6cec226bd360\") " pod="openstack-kuttl-tests/barbican-worker-644558fbc5-4gtxj" Dec 03 12:29:57 crc kubenswrapper[5052]: I1203 12:29:57.240651 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/placement-5c6478db8b-bt2rw"] Dec 03 12:29:57 crc kubenswrapper[5052]: I1203 12:29:57.248569 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8ac6e8f0-d49c-4a4c-846a-6cec226bd360-combined-ca-bundle\") pod \"barbican-worker-644558fbc5-4gtxj\" (UID: \"8ac6e8f0-d49c-4a4c-846a-6cec226bd360\") " pod="openstack-kuttl-tests/barbican-worker-644558fbc5-4gtxj" Dec 03 12:29:57 crc kubenswrapper[5052]: E1203 12:29:57.268608 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="fc5e4fca262b35b92bb410a508d650dc4bdca0fc0848d2a90abb1ede218d5a33" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:29:57 crc kubenswrapper[5052]: E1203 12:29:57.268674 5052 prober.go:104] "Probe errored" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" probeType="Readiness" pod="openstack-kuttl-tests/nova-cell1-conductor-0" podUID="733f463b-dcf8-4a61-8e4e-4bf1c042d397" containerName="nova-cell1-conductor-conductor" Dec 03 12:29:57 crc kubenswrapper[5052]: I1203 12:29:57.270103 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-fqrfd\" (UniqueName: \"kubernetes.io/projected/8ac6e8f0-d49c-4a4c-846a-6cec226bd360-kube-api-access-fqrfd\") pod \"barbican-worker-644558fbc5-4gtxj\" (UID: \"8ac6e8f0-d49c-4a4c-846a-6cec226bd360\") " pod="openstack-kuttl-tests/barbican-worker-644558fbc5-4gtxj" Dec 03 12:29:57 crc kubenswrapper[5052]: I1203 12:29:57.309791 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-keystone-listener-77b6bbbdcb-4jlsc" event={"ID":"aed41b09-319e-466c-9eb9-38208e1880a3","Type":"ContainerStarted","Data":"d94de750247c1241b19be7117917b99bc882f3e7734d81c7bd3e29a29ec26f1c"} Dec 03 12:29:57 crc kubenswrapper[5052]: I1203 12:29:57.327776 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-5fc7894bfd-g7hl5" event={"ID":"cfb8e98d-3873-43e0-81dc-4450983179c2","Type":"ContainerStarted","Data":"2efd148878bd3b821fbdb42496959e8b2ff832e6e52d5d0916ca6f1eb95aa2f5"} Dec 03 12:29:57 crc kubenswrapper[5052]: I1203 12:29:57.335708 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/neutron-56f9c7c6b6-kvxpr"] Dec 03 12:29:57 crc kubenswrapper[5052]: I1203 12:29:57.337414 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-56f9c7c6b6-kvxpr" Dec 03 12:29:57 crc kubenswrapper[5052]: I1203 12:29:57.339816 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-worker-6d687d9c57-4gplx" event={"ID":"a65652a1-62fa-40ff-8b8a-940b2cc795b7","Type":"ContainerStarted","Data":"9e5d3ebacee74cc3f19f073892deec26b2b6d3e604678ea5b944624e06141bd5"} Dec 03 12:29:57 crc kubenswrapper[5052]: I1203 12:29:57.354760 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-7cbf547d94-9krth" event={"ID":"6927052c-212d-42fc-8236-b053ffc9a9d9","Type":"ContainerStarted","Data":"5a2899587be3f4e77f5c08e5d3c326409dcaaa5a13802cb665f621fed0dcb74c"} Dec 03 12:29:57 crc kubenswrapper[5052]: I1203 12:29:57.397588 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/barbican-api-ff6b8c44b-7ndgm"] Dec 03 12:29:57 crc kubenswrapper[5052]: I1203 12:29:57.399646 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-api-ff6b8c44b-7ndgm" Dec 03 12:29:57 crc kubenswrapper[5052]: I1203 12:29:57.402075 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-keystone-listener-cd5b58cfd-jf8vt" Dec 03 12:29:57 crc kubenswrapper[5052]: I1203 12:29:57.418056 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-worker-644558fbc5-4gtxj" Dec 03 12:29:57 crc kubenswrapper[5052]: I1203 12:29:57.454751 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xm6xf\" (UniqueName: \"kubernetes.io/projected/5f522d90-036e-4565-8293-97097912fbb1-kube-api-access-xm6xf\") pod \"neutron-56f9c7c6b6-kvxpr\" (UID: \"5f522d90-036e-4565-8293-97097912fbb1\") " pod="openstack-kuttl-tests/neutron-56f9c7c6b6-kvxpr" Dec 03 12:29:57 crc kubenswrapper[5052]: I1203 12:29:57.454824 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/5f522d90-036e-4565-8293-97097912fbb1-internal-tls-certs\") pod \"neutron-56f9c7c6b6-kvxpr\" (UID: \"5f522d90-036e-4565-8293-97097912fbb1\") " pod="openstack-kuttl-tests/neutron-56f9c7c6b6-kvxpr" Dec 03 12:29:57 crc kubenswrapper[5052]: I1203 12:29:57.454852 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/5f522d90-036e-4565-8293-97097912fbb1-ovndb-tls-certs\") pod \"neutron-56f9c7c6b6-kvxpr\" (UID: \"5f522d90-036e-4565-8293-97097912fbb1\") " pod="openstack-kuttl-tests/neutron-56f9c7c6b6-kvxpr" Dec 03 12:29:57 crc kubenswrapper[5052]: I1203 12:29:57.454885 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5f522d90-036e-4565-8293-97097912fbb1-combined-ca-bundle\") pod \"neutron-56f9c7c6b6-kvxpr\" (UID: \"5f522d90-036e-4565-8293-97097912fbb1\") " pod="openstack-kuttl-tests/neutron-56f9c7c6b6-kvxpr" Dec 03 12:29:57 crc kubenswrapper[5052]: I1203 12:29:57.454918 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/5f522d90-036e-4565-8293-97097912fbb1-config\") pod \"neutron-56f9c7c6b6-kvxpr\" (UID: \"5f522d90-036e-4565-8293-97097912fbb1\") " pod="openstack-kuttl-tests/neutron-56f9c7c6b6-kvxpr" Dec 03 12:29:57 crc kubenswrapper[5052]: I1203 12:29:57.454966 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/5f522d90-036e-4565-8293-97097912fbb1-public-tls-certs\") pod \"neutron-56f9c7c6b6-kvxpr\" (UID: \"5f522d90-036e-4565-8293-97097912fbb1\") " pod="openstack-kuttl-tests/neutron-56f9c7c6b6-kvxpr" Dec 03 12:29:57 crc kubenswrapper[5052]: I1203 12:29:57.454998 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/5f522d90-036e-4565-8293-97097912fbb1-httpd-config\") pod \"neutron-56f9c7c6b6-kvxpr\" (UID: \"5f522d90-036e-4565-8293-97097912fbb1\") " pod="openstack-kuttl-tests/neutron-56f9c7c6b6-kvxpr" Dec 03 12:29:57 crc kubenswrapper[5052]: I1203 12:29:57.457628 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-56f9c7c6b6-kvxpr"] Dec 03 12:29:57 crc kubenswrapper[5052]: I1203 12:29:57.480616 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-api-ff6b8c44b-7ndgm"] Dec 03 12:29:57 crc kubenswrapper[5052]: I1203 12:29:57.506709 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/placement-7dffdbf8-hzmbm"] Dec 03 12:29:57 crc kubenswrapper[5052]: I1203 12:29:57.515708 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-7dffdbf8-hzmbm" Dec 03 12:29:57 crc kubenswrapper[5052]: I1203 12:29:57.521590 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/placement-7dffdbf8-hzmbm"] Dec 03 12:29:57 crc kubenswrapper[5052]: I1203 12:29:57.529595 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/keystone-7cbf547d94-9krth"] Dec 03 12:29:57 crc kubenswrapper[5052]: I1203 12:29:57.538581 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/keystone-564bc88b6-bvll5"] Dec 03 12:29:57 crc kubenswrapper[5052]: I1203 12:29:57.539880 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-564bc88b6-bvll5" Dec 03 12:29:57 crc kubenswrapper[5052]: I1203 12:29:57.558240 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-564bc88b6-bvll5"] Dec 03 12:29:57 crc kubenswrapper[5052]: I1203 12:29:57.560132 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/48831cad-910f-46f1-aefe-7cc4cbfa984f-public-tls-certs\") pod \"barbican-api-ff6b8c44b-7ndgm\" (UID: \"48831cad-910f-46f1-aefe-7cc4cbfa984f\") " pod="openstack-kuttl-tests/barbican-api-ff6b8c44b-7ndgm" Dec 03 12:29:57 crc kubenswrapper[5052]: I1203 12:29:57.560182 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/5f522d90-036e-4565-8293-97097912fbb1-public-tls-certs\") pod \"neutron-56f9c7c6b6-kvxpr\" (UID: \"5f522d90-036e-4565-8293-97097912fbb1\") " pod="openstack-kuttl-tests/neutron-56f9c7c6b6-kvxpr" Dec 03 12:29:57 crc kubenswrapper[5052]: I1203 12:29:57.560222 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/5f522d90-036e-4565-8293-97097912fbb1-httpd-config\") pod \"neutron-56f9c7c6b6-kvxpr\" (UID: \"5f522d90-036e-4565-8293-97097912fbb1\") " pod="openstack-kuttl-tests/neutron-56f9c7c6b6-kvxpr" Dec 03 12:29:57 crc kubenswrapper[5052]: I1203 12:29:57.560270 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/48831cad-910f-46f1-aefe-7cc4cbfa984f-internal-tls-certs\") pod \"barbican-api-ff6b8c44b-7ndgm\" (UID: \"48831cad-910f-46f1-aefe-7cc4cbfa984f\") " pod="openstack-kuttl-tests/barbican-api-ff6b8c44b-7ndgm" Dec 03 12:29:57 crc kubenswrapper[5052]: I1203 12:29:57.560303 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/48831cad-910f-46f1-aefe-7cc4cbfa984f-logs\") pod \"barbican-api-ff6b8c44b-7ndgm\" (UID: \"48831cad-910f-46f1-aefe-7cc4cbfa984f\") " pod="openstack-kuttl-tests/barbican-api-ff6b8c44b-7ndgm" Dec 03 12:29:57 crc kubenswrapper[5052]: I1203 12:29:57.560345 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-xm6xf\" (UniqueName: \"kubernetes.io/projected/5f522d90-036e-4565-8293-97097912fbb1-kube-api-access-xm6xf\") pod \"neutron-56f9c7c6b6-kvxpr\" (UID: \"5f522d90-036e-4565-8293-97097912fbb1\") " pod="openstack-kuttl-tests/neutron-56f9c7c6b6-kvxpr" Dec 03 12:29:57 crc kubenswrapper[5052]: I1203 12:29:57.560362 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k84ln\" (UniqueName: \"kubernetes.io/projected/48831cad-910f-46f1-aefe-7cc4cbfa984f-kube-api-access-k84ln\") pod \"barbican-api-ff6b8c44b-7ndgm\" (UID: \"48831cad-910f-46f1-aefe-7cc4cbfa984f\") " pod="openstack-kuttl-tests/barbican-api-ff6b8c44b-7ndgm" Dec 03 12:29:57 crc kubenswrapper[5052]: I1203 12:29:57.560409 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/5f522d90-036e-4565-8293-97097912fbb1-internal-tls-certs\") pod \"neutron-56f9c7c6b6-kvxpr\" (UID: \"5f522d90-036e-4565-8293-97097912fbb1\") " pod="openstack-kuttl-tests/neutron-56f9c7c6b6-kvxpr" Dec 03 12:29:57 crc kubenswrapper[5052]: I1203 12:29:57.560443 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/5f522d90-036e-4565-8293-97097912fbb1-ovndb-tls-certs\") pod \"neutron-56f9c7c6b6-kvxpr\" (UID: \"5f522d90-036e-4565-8293-97097912fbb1\") " pod="openstack-kuttl-tests/neutron-56f9c7c6b6-kvxpr" Dec 03 12:29:57 crc kubenswrapper[5052]: I1203 12:29:57.560478 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/48831cad-910f-46f1-aefe-7cc4cbfa984f-config-data-custom\") pod \"barbican-api-ff6b8c44b-7ndgm\" (UID: \"48831cad-910f-46f1-aefe-7cc4cbfa984f\") " pod="openstack-kuttl-tests/barbican-api-ff6b8c44b-7ndgm" Dec 03 12:29:57 crc kubenswrapper[5052]: I1203 12:29:57.560497 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/48831cad-910f-46f1-aefe-7cc4cbfa984f-config-data\") pod \"barbican-api-ff6b8c44b-7ndgm\" (UID: \"48831cad-910f-46f1-aefe-7cc4cbfa984f\") " pod="openstack-kuttl-tests/barbican-api-ff6b8c44b-7ndgm" Dec 03 12:29:57 crc kubenswrapper[5052]: I1203 12:29:57.560535 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5f522d90-036e-4565-8293-97097912fbb1-combined-ca-bundle\") pod \"neutron-56f9c7c6b6-kvxpr\" (UID: \"5f522d90-036e-4565-8293-97097912fbb1\") " pod="openstack-kuttl-tests/neutron-56f9c7c6b6-kvxpr" Dec 03 12:29:57 crc kubenswrapper[5052]: I1203 12:29:57.560556 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/48831cad-910f-46f1-aefe-7cc4cbfa984f-combined-ca-bundle\") pod \"barbican-api-ff6b8c44b-7ndgm\" (UID: \"48831cad-910f-46f1-aefe-7cc4cbfa984f\") " pod="openstack-kuttl-tests/barbican-api-ff6b8c44b-7ndgm" Dec 03 12:29:57 crc kubenswrapper[5052]: I1203 12:29:57.560588 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/5f522d90-036e-4565-8293-97097912fbb1-config\") pod \"neutron-56f9c7c6b6-kvxpr\" (UID: \"5f522d90-036e-4565-8293-97097912fbb1\") " pod="openstack-kuttl-tests/neutron-56f9c7c6b6-kvxpr" Dec 03 12:29:57 crc kubenswrapper[5052]: I1203 12:29:57.584251 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/5f522d90-036e-4565-8293-97097912fbb1-httpd-config\") pod \"neutron-56f9c7c6b6-kvxpr\" (UID: \"5f522d90-036e-4565-8293-97097912fbb1\") " pod="openstack-kuttl-tests/neutron-56f9c7c6b6-kvxpr" Dec 03 12:29:57 crc kubenswrapper[5052]: I1203 12:29:57.586176 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/5f522d90-036e-4565-8293-97097912fbb1-config\") pod \"neutron-56f9c7c6b6-kvxpr\" (UID: \"5f522d90-036e-4565-8293-97097912fbb1\") " pod="openstack-kuttl-tests/neutron-56f9c7c6b6-kvxpr" Dec 03 12:29:57 crc kubenswrapper[5052]: I1203 12:29:57.587154 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/5f522d90-036e-4565-8293-97097912fbb1-public-tls-certs\") pod \"neutron-56f9c7c6b6-kvxpr\" (UID: \"5f522d90-036e-4565-8293-97097912fbb1\") " pod="openstack-kuttl-tests/neutron-56f9c7c6b6-kvxpr" Dec 03 12:29:57 crc kubenswrapper[5052]: I1203 12:29:57.593093 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5f522d90-036e-4565-8293-97097912fbb1-combined-ca-bundle\") pod \"neutron-56f9c7c6b6-kvxpr\" (UID: \"5f522d90-036e-4565-8293-97097912fbb1\") " pod="openstack-kuttl-tests/neutron-56f9c7c6b6-kvxpr" Dec 03 12:29:57 crc kubenswrapper[5052]: I1203 12:29:57.597424 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/5f522d90-036e-4565-8293-97097912fbb1-internal-tls-certs\") pod \"neutron-56f9c7c6b6-kvxpr\" (UID: \"5f522d90-036e-4565-8293-97097912fbb1\") " pod="openstack-kuttl-tests/neutron-56f9c7c6b6-kvxpr" Dec 03 12:29:57 crc kubenswrapper[5052]: I1203 12:29:57.607679 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-xm6xf\" (UniqueName: \"kubernetes.io/projected/5f522d90-036e-4565-8293-97097912fbb1-kube-api-access-xm6xf\") pod \"neutron-56f9c7c6b6-kvxpr\" (UID: \"5f522d90-036e-4565-8293-97097912fbb1\") " pod="openstack-kuttl-tests/neutron-56f9c7c6b6-kvxpr" Dec 03 12:29:57 crc kubenswrapper[5052]: I1203 12:29:57.613656 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/5f522d90-036e-4565-8293-97097912fbb1-ovndb-tls-certs\") pod \"neutron-56f9c7c6b6-kvxpr\" (UID: \"5f522d90-036e-4565-8293-97097912fbb1\") " pod="openstack-kuttl-tests/neutron-56f9c7c6b6-kvxpr" Dec 03 12:29:57 crc kubenswrapper[5052]: I1203 12:29:57.665056 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/c0ce9ced-33b0-4879-81a4-b5c066ec0631-credential-keys\") pod \"keystone-564bc88b6-bvll5\" (UID: \"c0ce9ced-33b0-4879-81a4-b5c066ec0631\") " pod="openstack-kuttl-tests/keystone-564bc88b6-bvll5" Dec 03 12:29:57 crc kubenswrapper[5052]: I1203 12:29:57.665309 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/53fe6403-27db-44b7-a9a6-787afbc3bbb8-internal-tls-certs\") pod \"placement-7dffdbf8-hzmbm\" (UID: \"53fe6403-27db-44b7-a9a6-787afbc3bbb8\") " pod="openstack-kuttl-tests/placement-7dffdbf8-hzmbm" Dec 03 12:29:57 crc kubenswrapper[5052]: I1203 12:29:57.665435 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/53fe6403-27db-44b7-a9a6-787afbc3bbb8-logs\") pod \"placement-7dffdbf8-hzmbm\" (UID: \"53fe6403-27db-44b7-a9a6-787afbc3bbb8\") " pod="openstack-kuttl-tests/placement-7dffdbf8-hzmbm" Dec 03 12:29:57 crc kubenswrapper[5052]: I1203 12:29:57.665557 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/c0ce9ced-33b0-4879-81a4-b5c066ec0631-config-data\") pod \"keystone-564bc88b6-bvll5\" (UID: \"c0ce9ced-33b0-4879-81a4-b5c066ec0631\") " pod="openstack-kuttl-tests/keystone-564bc88b6-bvll5" Dec 03 12:29:57 crc kubenswrapper[5052]: I1203 12:29:57.665680 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/48831cad-910f-46f1-aefe-7cc4cbfa984f-internal-tls-certs\") pod \"barbican-api-ff6b8c44b-7ndgm\" (UID: \"48831cad-910f-46f1-aefe-7cc4cbfa984f\") " pod="openstack-kuttl-tests/barbican-api-ff6b8c44b-7ndgm" Dec 03 12:29:57 crc kubenswrapper[5052]: I1203 12:29:57.683100 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/48831cad-910f-46f1-aefe-7cc4cbfa984f-internal-tls-certs\") pod \"barbican-api-ff6b8c44b-7ndgm\" (UID: \"48831cad-910f-46f1-aefe-7cc4cbfa984f\") " pod="openstack-kuttl-tests/barbican-api-ff6b8c44b-7ndgm" Dec 03 12:29:57 crc kubenswrapper[5052]: I1203 12:29:57.699385 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/48831cad-910f-46f1-aefe-7cc4cbfa984f-logs\") pod \"barbican-api-ff6b8c44b-7ndgm\" (UID: \"48831cad-910f-46f1-aefe-7cc4cbfa984f\") " pod="openstack-kuttl-tests/barbican-api-ff6b8c44b-7ndgm" Dec 03 12:29:57 crc kubenswrapper[5052]: I1203 12:29:57.699536 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/c0ce9ced-33b0-4879-81a4-b5c066ec0631-internal-tls-certs\") pod \"keystone-564bc88b6-bvll5\" (UID: \"c0ce9ced-33b0-4879-81a4-b5c066ec0631\") " pod="openstack-kuttl-tests/keystone-564bc88b6-bvll5" Dec 03 12:29:57 crc kubenswrapper[5052]: I1203 12:29:57.699661 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-k84ln\" (UniqueName: \"kubernetes.io/projected/48831cad-910f-46f1-aefe-7cc4cbfa984f-kube-api-access-k84ln\") pod \"barbican-api-ff6b8c44b-7ndgm\" (UID: \"48831cad-910f-46f1-aefe-7cc4cbfa984f\") " pod="openstack-kuttl-tests/barbican-api-ff6b8c44b-7ndgm" Dec 03 12:29:57 crc kubenswrapper[5052]: I1203 12:29:57.699797 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tpxrx\" (UniqueName: \"kubernetes.io/projected/53fe6403-27db-44b7-a9a6-787afbc3bbb8-kube-api-access-tpxrx\") pod \"placement-7dffdbf8-hzmbm\" (UID: \"53fe6403-27db-44b7-a9a6-787afbc3bbb8\") " pod="openstack-kuttl-tests/placement-7dffdbf8-hzmbm" Dec 03 12:29:57 crc kubenswrapper[5052]: I1203 12:29:57.699922 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/48831cad-910f-46f1-aefe-7cc4cbfa984f-logs\") pod \"barbican-api-ff6b8c44b-7ndgm\" (UID: \"48831cad-910f-46f1-aefe-7cc4cbfa984f\") " pod="openstack-kuttl-tests/barbican-api-ff6b8c44b-7ndgm" Dec 03 12:29:57 crc kubenswrapper[5052]: I1203 12:29:57.700136 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/53fe6403-27db-44b7-a9a6-787afbc3bbb8-scripts\") pod \"placement-7dffdbf8-hzmbm\" (UID: \"53fe6403-27db-44b7-a9a6-787afbc3bbb8\") " pod="openstack-kuttl-tests/placement-7dffdbf8-hzmbm" Dec 03 12:29:57 crc kubenswrapper[5052]: I1203 12:29:57.700246 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/c0ce9ced-33b0-4879-81a4-b5c066ec0631-fernet-keys\") pod \"keystone-564bc88b6-bvll5\" (UID: \"c0ce9ced-33b0-4879-81a4-b5c066ec0631\") " pod="openstack-kuttl-tests/keystone-564bc88b6-bvll5" Dec 03 12:29:57 crc kubenswrapper[5052]: I1203 12:29:57.700283 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/53fe6403-27db-44b7-a9a6-787afbc3bbb8-combined-ca-bundle\") pod \"placement-7dffdbf8-hzmbm\" (UID: \"53fe6403-27db-44b7-a9a6-787afbc3bbb8\") " pod="openstack-kuttl-tests/placement-7dffdbf8-hzmbm" Dec 03 12:29:57 crc kubenswrapper[5052]: I1203 12:29:57.700300 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c0ce9ced-33b0-4879-81a4-b5c066ec0631-combined-ca-bundle\") pod \"keystone-564bc88b6-bvll5\" (UID: \"c0ce9ced-33b0-4879-81a4-b5c066ec0631\") " pod="openstack-kuttl-tests/keystone-564bc88b6-bvll5" Dec 03 12:29:57 crc kubenswrapper[5052]: I1203 12:29:57.700403 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/48831cad-910f-46f1-aefe-7cc4cbfa984f-config-data-custom\") pod \"barbican-api-ff6b8c44b-7ndgm\" (UID: \"48831cad-910f-46f1-aefe-7cc4cbfa984f\") " pod="openstack-kuttl-tests/barbican-api-ff6b8c44b-7ndgm" Dec 03 12:29:57 crc kubenswrapper[5052]: I1203 12:29:57.700425 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/48831cad-910f-46f1-aefe-7cc4cbfa984f-config-data\") pod \"barbican-api-ff6b8c44b-7ndgm\" (UID: \"48831cad-910f-46f1-aefe-7cc4cbfa984f\") " pod="openstack-kuttl-tests/barbican-api-ff6b8c44b-7ndgm" Dec 03 12:29:57 crc kubenswrapper[5052]: I1203 12:29:57.700457 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/48831cad-910f-46f1-aefe-7cc4cbfa984f-combined-ca-bundle\") pod \"barbican-api-ff6b8c44b-7ndgm\" (UID: \"48831cad-910f-46f1-aefe-7cc4cbfa984f\") " pod="openstack-kuttl-tests/barbican-api-ff6b8c44b-7ndgm" Dec 03 12:29:57 crc kubenswrapper[5052]: I1203 12:29:57.700504 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/c0ce9ced-33b0-4879-81a4-b5c066ec0631-scripts\") pod \"keystone-564bc88b6-bvll5\" (UID: \"c0ce9ced-33b0-4879-81a4-b5c066ec0631\") " pod="openstack-kuttl-tests/keystone-564bc88b6-bvll5" Dec 03 12:29:57 crc kubenswrapper[5052]: I1203 12:29:57.700600 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/53fe6403-27db-44b7-a9a6-787afbc3bbb8-config-data\") pod \"placement-7dffdbf8-hzmbm\" (UID: \"53fe6403-27db-44b7-a9a6-787afbc3bbb8\") " pod="openstack-kuttl-tests/placement-7dffdbf8-hzmbm" Dec 03 12:29:57 crc kubenswrapper[5052]: I1203 12:29:57.700695 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d57ht\" (UniqueName: \"kubernetes.io/projected/c0ce9ced-33b0-4879-81a4-b5c066ec0631-kube-api-access-d57ht\") pod \"keystone-564bc88b6-bvll5\" (UID: \"c0ce9ced-33b0-4879-81a4-b5c066ec0631\") " pod="openstack-kuttl-tests/keystone-564bc88b6-bvll5" Dec 03 12:29:57 crc kubenswrapper[5052]: I1203 12:29:57.700713 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/48831cad-910f-46f1-aefe-7cc4cbfa984f-public-tls-certs\") pod \"barbican-api-ff6b8c44b-7ndgm\" (UID: \"48831cad-910f-46f1-aefe-7cc4cbfa984f\") " pod="openstack-kuttl-tests/barbican-api-ff6b8c44b-7ndgm" Dec 03 12:29:57 crc kubenswrapper[5052]: I1203 12:29:57.700733 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/c0ce9ced-33b0-4879-81a4-b5c066ec0631-public-tls-certs\") pod \"keystone-564bc88b6-bvll5\" (UID: \"c0ce9ced-33b0-4879-81a4-b5c066ec0631\") " pod="openstack-kuttl-tests/keystone-564bc88b6-bvll5" Dec 03 12:29:57 crc kubenswrapper[5052]: I1203 12:29:57.700778 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/53fe6403-27db-44b7-a9a6-787afbc3bbb8-public-tls-certs\") pod \"placement-7dffdbf8-hzmbm\" (UID: \"53fe6403-27db-44b7-a9a6-787afbc3bbb8\") " pod="openstack-kuttl-tests/placement-7dffdbf8-hzmbm" Dec 03 12:29:57 crc kubenswrapper[5052]: I1203 12:29:57.711379 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/48831cad-910f-46f1-aefe-7cc4cbfa984f-config-data-custom\") pod \"barbican-api-ff6b8c44b-7ndgm\" (UID: \"48831cad-910f-46f1-aefe-7cc4cbfa984f\") " pod="openstack-kuttl-tests/barbican-api-ff6b8c44b-7ndgm" Dec 03 12:29:57 crc kubenswrapper[5052]: I1203 12:29:57.717542 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/48831cad-910f-46f1-aefe-7cc4cbfa984f-config-data\") pod \"barbican-api-ff6b8c44b-7ndgm\" (UID: \"48831cad-910f-46f1-aefe-7cc4cbfa984f\") " pod="openstack-kuttl-tests/barbican-api-ff6b8c44b-7ndgm" Dec 03 12:29:57 crc kubenswrapper[5052]: I1203 12:29:57.728305 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/48831cad-910f-46f1-aefe-7cc4cbfa984f-public-tls-certs\") pod \"barbican-api-ff6b8c44b-7ndgm\" (UID: \"48831cad-910f-46f1-aefe-7cc4cbfa984f\") " pod="openstack-kuttl-tests/barbican-api-ff6b8c44b-7ndgm" Dec 03 12:29:57 crc kubenswrapper[5052]: I1203 12:29:57.749194 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/48831cad-910f-46f1-aefe-7cc4cbfa984f-combined-ca-bundle\") pod \"barbican-api-ff6b8c44b-7ndgm\" (UID: \"48831cad-910f-46f1-aefe-7cc4cbfa984f\") " pod="openstack-kuttl-tests/barbican-api-ff6b8c44b-7ndgm" Dec 03 12:29:57 crc kubenswrapper[5052]: I1203 12:29:57.749293 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-k84ln\" (UniqueName: \"kubernetes.io/projected/48831cad-910f-46f1-aefe-7cc4cbfa984f-kube-api-access-k84ln\") pod \"barbican-api-ff6b8c44b-7ndgm\" (UID: \"48831cad-910f-46f1-aefe-7cc4cbfa984f\") " pod="openstack-kuttl-tests/barbican-api-ff6b8c44b-7ndgm" Dec 03 12:29:57 crc kubenswrapper[5052]: I1203 12:29:57.804440 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/c0ce9ced-33b0-4879-81a4-b5c066ec0631-internal-tls-certs\") pod \"keystone-564bc88b6-bvll5\" (UID: \"c0ce9ced-33b0-4879-81a4-b5c066ec0631\") " pod="openstack-kuttl-tests/keystone-564bc88b6-bvll5" Dec 03 12:29:57 crc kubenswrapper[5052]: I1203 12:29:57.804541 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-tpxrx\" (UniqueName: \"kubernetes.io/projected/53fe6403-27db-44b7-a9a6-787afbc3bbb8-kube-api-access-tpxrx\") pod \"placement-7dffdbf8-hzmbm\" (UID: \"53fe6403-27db-44b7-a9a6-787afbc3bbb8\") " pod="openstack-kuttl-tests/placement-7dffdbf8-hzmbm" Dec 03 12:29:57 crc kubenswrapper[5052]: I1203 12:29:57.804589 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/53fe6403-27db-44b7-a9a6-787afbc3bbb8-scripts\") pod \"placement-7dffdbf8-hzmbm\" (UID: \"53fe6403-27db-44b7-a9a6-787afbc3bbb8\") " pod="openstack-kuttl-tests/placement-7dffdbf8-hzmbm" Dec 03 12:29:57 crc kubenswrapper[5052]: I1203 12:29:57.804617 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/c0ce9ced-33b0-4879-81a4-b5c066ec0631-fernet-keys\") pod \"keystone-564bc88b6-bvll5\" (UID: \"c0ce9ced-33b0-4879-81a4-b5c066ec0631\") " pod="openstack-kuttl-tests/keystone-564bc88b6-bvll5" Dec 03 12:29:57 crc kubenswrapper[5052]: I1203 12:29:57.804640 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c0ce9ced-33b0-4879-81a4-b5c066ec0631-combined-ca-bundle\") pod \"keystone-564bc88b6-bvll5\" (UID: \"c0ce9ced-33b0-4879-81a4-b5c066ec0631\") " pod="openstack-kuttl-tests/keystone-564bc88b6-bvll5" Dec 03 12:29:57 crc kubenswrapper[5052]: I1203 12:29:57.804664 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/53fe6403-27db-44b7-a9a6-787afbc3bbb8-combined-ca-bundle\") pod \"placement-7dffdbf8-hzmbm\" (UID: \"53fe6403-27db-44b7-a9a6-787afbc3bbb8\") " pod="openstack-kuttl-tests/placement-7dffdbf8-hzmbm" Dec 03 12:29:57 crc kubenswrapper[5052]: I1203 12:29:57.804759 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/c0ce9ced-33b0-4879-81a4-b5c066ec0631-scripts\") pod \"keystone-564bc88b6-bvll5\" (UID: \"c0ce9ced-33b0-4879-81a4-b5c066ec0631\") " pod="openstack-kuttl-tests/keystone-564bc88b6-bvll5" Dec 03 12:29:57 crc kubenswrapper[5052]: I1203 12:29:57.804777 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-56f9c7c6b6-kvxpr" Dec 03 12:29:57 crc kubenswrapper[5052]: I1203 12:29:57.804790 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/53fe6403-27db-44b7-a9a6-787afbc3bbb8-config-data\") pod \"placement-7dffdbf8-hzmbm\" (UID: \"53fe6403-27db-44b7-a9a6-787afbc3bbb8\") " pod="openstack-kuttl-tests/placement-7dffdbf8-hzmbm" Dec 03 12:29:57 crc kubenswrapper[5052]: I1203 12:29:57.813283 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-d57ht\" (UniqueName: \"kubernetes.io/projected/c0ce9ced-33b0-4879-81a4-b5c066ec0631-kube-api-access-d57ht\") pod \"keystone-564bc88b6-bvll5\" (UID: \"c0ce9ced-33b0-4879-81a4-b5c066ec0631\") " pod="openstack-kuttl-tests/keystone-564bc88b6-bvll5" Dec 03 12:29:57 crc kubenswrapper[5052]: I1203 12:29:57.815166 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/53fe6403-27db-44b7-a9a6-787afbc3bbb8-scripts\") pod \"placement-7dffdbf8-hzmbm\" (UID: \"53fe6403-27db-44b7-a9a6-787afbc3bbb8\") " pod="openstack-kuttl-tests/placement-7dffdbf8-hzmbm" Dec 03 12:29:57 crc kubenswrapper[5052]: I1203 12:29:57.815566 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/53fe6403-27db-44b7-a9a6-787afbc3bbb8-combined-ca-bundle\") pod \"placement-7dffdbf8-hzmbm\" (UID: \"53fe6403-27db-44b7-a9a6-787afbc3bbb8\") " pod="openstack-kuttl-tests/placement-7dffdbf8-hzmbm" Dec 03 12:29:57 crc kubenswrapper[5052]: I1203 12:29:57.819041 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/c0ce9ced-33b0-4879-81a4-b5c066ec0631-public-tls-certs\") pod \"keystone-564bc88b6-bvll5\" (UID: \"c0ce9ced-33b0-4879-81a4-b5c066ec0631\") " pod="openstack-kuttl-tests/keystone-564bc88b6-bvll5" Dec 03 12:29:57 crc kubenswrapper[5052]: I1203 12:29:57.819105 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/53fe6403-27db-44b7-a9a6-787afbc3bbb8-public-tls-certs\") pod \"placement-7dffdbf8-hzmbm\" (UID: \"53fe6403-27db-44b7-a9a6-787afbc3bbb8\") " pod="openstack-kuttl-tests/placement-7dffdbf8-hzmbm" Dec 03 12:29:57 crc kubenswrapper[5052]: I1203 12:29:57.819923 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/c0ce9ced-33b0-4879-81a4-b5c066ec0631-credential-keys\") pod \"keystone-564bc88b6-bvll5\" (UID: \"c0ce9ced-33b0-4879-81a4-b5c066ec0631\") " pod="openstack-kuttl-tests/keystone-564bc88b6-bvll5" Dec 03 12:29:57 crc kubenswrapper[5052]: I1203 12:29:57.820164 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/53fe6403-27db-44b7-a9a6-787afbc3bbb8-internal-tls-certs\") pod \"placement-7dffdbf8-hzmbm\" (UID: \"53fe6403-27db-44b7-a9a6-787afbc3bbb8\") " pod="openstack-kuttl-tests/placement-7dffdbf8-hzmbm" Dec 03 12:29:57 crc kubenswrapper[5052]: I1203 12:29:57.820207 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/53fe6403-27db-44b7-a9a6-787afbc3bbb8-logs\") pod \"placement-7dffdbf8-hzmbm\" (UID: \"53fe6403-27db-44b7-a9a6-787afbc3bbb8\") " pod="openstack-kuttl-tests/placement-7dffdbf8-hzmbm" Dec 03 12:29:57 crc kubenswrapper[5052]: I1203 12:29:57.820266 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/c0ce9ced-33b0-4879-81a4-b5c066ec0631-config-data\") pod \"keystone-564bc88b6-bvll5\" (UID: \"c0ce9ced-33b0-4879-81a4-b5c066ec0631\") " pod="openstack-kuttl-tests/keystone-564bc88b6-bvll5" Dec 03 12:29:57 crc kubenswrapper[5052]: I1203 12:29:57.823155 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/53fe6403-27db-44b7-a9a6-787afbc3bbb8-config-data\") pod \"placement-7dffdbf8-hzmbm\" (UID: \"53fe6403-27db-44b7-a9a6-787afbc3bbb8\") " pod="openstack-kuttl-tests/placement-7dffdbf8-hzmbm" Dec 03 12:29:57 crc kubenswrapper[5052]: I1203 12:29:57.825834 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/c0ce9ced-33b0-4879-81a4-b5c066ec0631-credential-keys\") pod \"keystone-564bc88b6-bvll5\" (UID: \"c0ce9ced-33b0-4879-81a4-b5c066ec0631\") " pod="openstack-kuttl-tests/keystone-564bc88b6-bvll5" Dec 03 12:29:57 crc kubenswrapper[5052]: I1203 12:29:57.826701 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/53fe6403-27db-44b7-a9a6-787afbc3bbb8-logs\") pod \"placement-7dffdbf8-hzmbm\" (UID: \"53fe6403-27db-44b7-a9a6-787afbc3bbb8\") " pod="openstack-kuttl-tests/placement-7dffdbf8-hzmbm" Dec 03 12:29:57 crc kubenswrapper[5052]: I1203 12:29:57.827605 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/c0ce9ced-33b0-4879-81a4-b5c066ec0631-internal-tls-certs\") pod \"keystone-564bc88b6-bvll5\" (UID: \"c0ce9ced-33b0-4879-81a4-b5c066ec0631\") " pod="openstack-kuttl-tests/keystone-564bc88b6-bvll5" Dec 03 12:29:57 crc kubenswrapper[5052]: I1203 12:29:57.828666 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/c0ce9ced-33b0-4879-81a4-b5c066ec0631-fernet-keys\") pod \"keystone-564bc88b6-bvll5\" (UID: \"c0ce9ced-33b0-4879-81a4-b5c066ec0631\") " pod="openstack-kuttl-tests/keystone-564bc88b6-bvll5" Dec 03 12:29:57 crc kubenswrapper[5052]: I1203 12:29:57.828686 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/c0ce9ced-33b0-4879-81a4-b5c066ec0631-config-data\") pod \"keystone-564bc88b6-bvll5\" (UID: \"c0ce9ced-33b0-4879-81a4-b5c066ec0631\") " pod="openstack-kuttl-tests/keystone-564bc88b6-bvll5" Dec 03 12:29:57 crc kubenswrapper[5052]: I1203 12:29:57.829136 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/c0ce9ced-33b0-4879-81a4-b5c066ec0631-scripts\") pod \"keystone-564bc88b6-bvll5\" (UID: \"c0ce9ced-33b0-4879-81a4-b5c066ec0631\") " pod="openstack-kuttl-tests/keystone-564bc88b6-bvll5" Dec 03 12:29:57 crc kubenswrapper[5052]: I1203 12:29:57.832968 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c0ce9ced-33b0-4879-81a4-b5c066ec0631-combined-ca-bundle\") pod \"keystone-564bc88b6-bvll5\" (UID: \"c0ce9ced-33b0-4879-81a4-b5c066ec0631\") " pod="openstack-kuttl-tests/keystone-564bc88b6-bvll5" Dec 03 12:29:57 crc kubenswrapper[5052]: I1203 12:29:57.836195 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/53fe6403-27db-44b7-a9a6-787afbc3bbb8-internal-tls-certs\") pod \"placement-7dffdbf8-hzmbm\" (UID: \"53fe6403-27db-44b7-a9a6-787afbc3bbb8\") " pod="openstack-kuttl-tests/placement-7dffdbf8-hzmbm" Dec 03 12:29:57 crc kubenswrapper[5052]: I1203 12:29:57.840037 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/53fe6403-27db-44b7-a9a6-787afbc3bbb8-public-tls-certs\") pod \"placement-7dffdbf8-hzmbm\" (UID: \"53fe6403-27db-44b7-a9a6-787afbc3bbb8\") " pod="openstack-kuttl-tests/placement-7dffdbf8-hzmbm" Dec 03 12:29:57 crc kubenswrapper[5052]: I1203 12:29:57.840813 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-d57ht\" (UniqueName: \"kubernetes.io/projected/c0ce9ced-33b0-4879-81a4-b5c066ec0631-kube-api-access-d57ht\") pod \"keystone-564bc88b6-bvll5\" (UID: \"c0ce9ced-33b0-4879-81a4-b5c066ec0631\") " pod="openstack-kuttl-tests/keystone-564bc88b6-bvll5" Dec 03 12:29:57 crc kubenswrapper[5052]: I1203 12:29:57.849050 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-tpxrx\" (UniqueName: \"kubernetes.io/projected/53fe6403-27db-44b7-a9a6-787afbc3bbb8-kube-api-access-tpxrx\") pod \"placement-7dffdbf8-hzmbm\" (UID: \"53fe6403-27db-44b7-a9a6-787afbc3bbb8\") " pod="openstack-kuttl-tests/placement-7dffdbf8-hzmbm" Dec 03 12:29:57 crc kubenswrapper[5052]: I1203 12:29:57.883202 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/c0ce9ced-33b0-4879-81a4-b5c066ec0631-public-tls-certs\") pod \"keystone-564bc88b6-bvll5\" (UID: \"c0ce9ced-33b0-4879-81a4-b5c066ec0631\") " pod="openstack-kuttl-tests/keystone-564bc88b6-bvll5" Dec 03 12:29:57 crc kubenswrapper[5052]: E1203 12:29:57.995724 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of 63b5496e0ce2e84297ac272518addd75c23a27efda2a42a79c0e221732c3c3c6 is running failed: container process not found" containerID="63b5496e0ce2e84297ac272518addd75c23a27efda2a42a79c0e221732c3c3c6" cmd=["/usr/bin/pidof","ovsdb-server"] Dec 03 12:29:57 crc kubenswrapper[5052]: E1203 12:29:57.998689 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of 63b5496e0ce2e84297ac272518addd75c23a27efda2a42a79c0e221732c3c3c6 is running failed: container process not found" containerID="63b5496e0ce2e84297ac272518addd75c23a27efda2a42a79c0e221732c3c3c6" cmd=["/usr/bin/pidof","ovsdb-server"] Dec 03 12:29:57 crc kubenswrapper[5052]: E1203 12:29:57.998958 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="c1a2fa107302843e2249473b85f61687344a151311126ad578b5760b51b1b60e" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:29:58 crc kubenswrapper[5052]: E1203 12:29:58.001799 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of 63b5496e0ce2e84297ac272518addd75c23a27efda2a42a79c0e221732c3c3c6 is running failed: container process not found" containerID="63b5496e0ce2e84297ac272518addd75c23a27efda2a42a79c0e221732c3c3c6" cmd=["/usr/bin/pidof","ovsdb-server"] Dec 03 12:29:58 crc kubenswrapper[5052]: E1203 12:29:58.001846 5052 prober.go:104] "Probe errored" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of 63b5496e0ce2e84297ac272518addd75c23a27efda2a42a79c0e221732c3c3c6 is running failed: container process not found" probeType="Readiness" pod="openstack-kuttl-tests/ovsdbserver-sb-0" podUID="5c7fe95d-f637-4021-a5e2-69668046d2ff" containerName="ovsdbserver-sb" Dec 03 12:29:58 crc kubenswrapper[5052]: E1203 12:29:58.034690 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="c1a2fa107302843e2249473b85f61687344a151311126ad578b5760b51b1b60e" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:29:58 crc kubenswrapper[5052]: I1203 12:29:58.046668 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-api-ff6b8c44b-7ndgm" Dec 03 12:29:58 crc kubenswrapper[5052]: E1203 12:29:58.047614 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="c1a2fa107302843e2249473b85f61687344a151311126ad578b5760b51b1b60e" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:29:58 crc kubenswrapper[5052]: E1203 12:29:58.047667 5052 prober.go:104] "Probe errored" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" probeType="Readiness" pod="openstack-kuttl-tests/nova-cell0-conductor-0" podUID="16a065b5-a603-4cb3-a4c0-b36c0b92bf3f" containerName="nova-cell0-conductor-conductor" Dec 03 12:29:58 crc kubenswrapper[5052]: I1203 12:29:58.079537 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-7dffdbf8-hzmbm" Dec 03 12:29:58 crc kubenswrapper[5052]: I1203 12:29:58.100831 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-564bc88b6-bvll5" Dec 03 12:29:58 crc kubenswrapper[5052]: I1203 12:29:58.118056 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:29:58 crc kubenswrapper[5052]: I1203 12:29:58.209316 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_ovsdbserver-nb-0_f45e9eda-e62f-4307-ae18-2a8da7d6fe7c/ovsdbserver-nb/0.log" Dec 03 12:29:58 crc kubenswrapper[5052]: I1203 12:29:58.209667 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:29:58 crc kubenswrapper[5052]: I1203 12:29:58.212496 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/memcached-0" Dec 03 12:29:58 crc kubenswrapper[5052]: I1203 12:29:58.215758 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_ovsdbserver-sb-0_5c7fe95d-f637-4021-a5e2-69668046d2ff/ovsdbserver-sb/0.log" Dec 03 12:29:58 crc kubenswrapper[5052]: I1203 12:29:58.215831 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:29:58 crc kubenswrapper[5052]: I1203 12:29:58.257085 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"nova-novncproxy-tls-certs\" (UniqueName: \"kubernetes.io/secret/53193ff3-1a41-4e26-b38f-957088aa7e8b-nova-novncproxy-tls-certs\") pod \"53193ff3-1a41-4e26-b38f-957088aa7e8b\" (UID: \"53193ff3-1a41-4e26-b38f-957088aa7e8b\") " Dec 03 12:29:58 crc kubenswrapper[5052]: I1203 12:29:58.257189 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/53193ff3-1a41-4e26-b38f-957088aa7e8b-combined-ca-bundle\") pod \"53193ff3-1a41-4e26-b38f-957088aa7e8b\" (UID: \"53193ff3-1a41-4e26-b38f-957088aa7e8b\") " Dec 03 12:29:58 crc kubenswrapper[5052]: I1203 12:29:58.257280 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/53193ff3-1a41-4e26-b38f-957088aa7e8b-config-data\") pod \"53193ff3-1a41-4e26-b38f-957088aa7e8b\" (UID: \"53193ff3-1a41-4e26-b38f-957088aa7e8b\") " Dec 03 12:29:58 crc kubenswrapper[5052]: I1203 12:29:58.257296 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"vencrypt-tls-certs\" (UniqueName: \"kubernetes.io/secret/53193ff3-1a41-4e26-b38f-957088aa7e8b-vencrypt-tls-certs\") pod \"53193ff3-1a41-4e26-b38f-957088aa7e8b\" (UID: \"53193ff3-1a41-4e26-b38f-957088aa7e8b\") " Dec 03 12:29:58 crc kubenswrapper[5052]: I1203 12:29:58.257445 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rzgf2\" (UniqueName: \"kubernetes.io/projected/53193ff3-1a41-4e26-b38f-957088aa7e8b-kube-api-access-rzgf2\") pod \"53193ff3-1a41-4e26-b38f-957088aa7e8b\" (UID: \"53193ff3-1a41-4e26-b38f-957088aa7e8b\") " Dec 03 12:29:58 crc kubenswrapper[5052]: I1203 12:29:58.267722 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/placement-5c6478db8b-bt2rw"] Dec 03 12:29:58 crc kubenswrapper[5052]: I1203 12:29:58.267892 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/53193ff3-1a41-4e26-b38f-957088aa7e8b-kube-api-access-rzgf2" (OuterVolumeSpecName: "kube-api-access-rzgf2") pod "53193ff3-1a41-4e26-b38f-957088aa7e8b" (UID: "53193ff3-1a41-4e26-b38f-957088aa7e8b"). InnerVolumeSpecName "kube-api-access-rzgf2". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:29:58 crc kubenswrapper[5052]: I1203 12:29:58.303773 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/neutron-db474c7dd-tg7mq"] Dec 03 12:29:58 crc kubenswrapper[5052]: I1203 12:29:58.332604 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/53193ff3-1a41-4e26-b38f-957088aa7e8b-config-data" (OuterVolumeSpecName: "config-data") pod "53193ff3-1a41-4e26-b38f-957088aa7e8b" (UID: "53193ff3-1a41-4e26-b38f-957088aa7e8b"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:29:58 crc kubenswrapper[5052]: I1203 12:29:58.397856 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5c7fe95d-f637-4021-a5e2-69668046d2ff-config\") pod \"5c7fe95d-f637-4021-a5e2-69668046d2ff\" (UID: \"5c7fe95d-f637-4021-a5e2-69668046d2ff\") " Dec 03 12:29:58 crc kubenswrapper[5052]: I1203 12:29:58.422677 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f45e9eda-e62f-4307-ae18-2a8da7d6fe7c-config\") pod \"f45e9eda-e62f-4307-ae18-2a8da7d6fe7c\" (UID: \"f45e9eda-e62f-4307-ae18-2a8da7d6fe7c\") " Dec 03 12:29:58 crc kubenswrapper[5052]: I1203 12:29:58.422776 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8n685\" (UniqueName: \"kubernetes.io/projected/f45e9eda-e62f-4307-ae18-2a8da7d6fe7c-kube-api-access-8n685\") pod \"f45e9eda-e62f-4307-ae18-2a8da7d6fe7c\" (UID: \"f45e9eda-e62f-4307-ae18-2a8da7d6fe7c\") " Dec 03 12:29:58 crc kubenswrapper[5052]: I1203 12:29:58.422826 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/b35e6939-8f23-496c-ac55-9534be4ea8be-config-data\") pod \"b35e6939-8f23-496c-ac55-9534be4ea8be\" (UID: \"b35e6939-8f23-496c-ac55-9534be4ea8be\") " Dec 03 12:29:58 crc kubenswrapper[5052]: I1203 12:29:58.422869 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"memcached-tls-certs\" (UniqueName: \"kubernetes.io/secret/b35e6939-8f23-496c-ac55-9534be4ea8be-memcached-tls-certs\") pod \"b35e6939-8f23-496c-ac55-9534be4ea8be\" (UID: \"b35e6939-8f23-496c-ac55-9534be4ea8be\") " Dec 03 12:29:58 crc kubenswrapper[5052]: I1203 12:29:58.422891 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-67l4w\" (UniqueName: \"kubernetes.io/projected/5c7fe95d-f637-4021-a5e2-69668046d2ff-kube-api-access-67l4w\") pod \"5c7fe95d-f637-4021-a5e2-69668046d2ff\" (UID: \"5c7fe95d-f637-4021-a5e2-69668046d2ff\") " Dec 03 12:29:58 crc kubenswrapper[5052]: I1203 12:29:58.422913 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5c7fe95d-f637-4021-a5e2-69668046d2ff-combined-ca-bundle\") pod \"5c7fe95d-f637-4021-a5e2-69668046d2ff\" (UID: \"5c7fe95d-f637-4021-a5e2-69668046d2ff\") " Dec 03 12:29:58 crc kubenswrapper[5052]: I1203 12:29:58.422941 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f45e9eda-e62f-4307-ae18-2a8da7d6fe7c-combined-ca-bundle\") pod \"f45e9eda-e62f-4307-ae18-2a8da7d6fe7c\" (UID: \"f45e9eda-e62f-4307-ae18-2a8da7d6fe7c\") " Dec 03 12:29:58 crc kubenswrapper[5052]: I1203 12:29:58.422961 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b35e6939-8f23-496c-ac55-9534be4ea8be-combined-ca-bundle\") pod \"b35e6939-8f23-496c-ac55-9534be4ea8be\" (UID: \"b35e6939-8f23-496c-ac55-9534be4ea8be\") " Dec 03 12:29:58 crc kubenswrapper[5052]: I1203 12:29:58.422995 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovndbcluster-sb-etc-ovn\" (UniqueName: \"kubernetes.io/local-volume/local-storage08-crc\") pod \"5c7fe95d-f637-4021-a5e2-69668046d2ff\" (UID: \"5c7fe95d-f637-4021-a5e2-69668046d2ff\") " Dec 03 12:29:58 crc kubenswrapper[5052]: I1203 12:29:58.423021 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovsdbserver-sb-tls-certs\" (UniqueName: \"kubernetes.io/secret/5c7fe95d-f637-4021-a5e2-69668046d2ff-ovsdbserver-sb-tls-certs\") pod \"5c7fe95d-f637-4021-a5e2-69668046d2ff\" (UID: \"5c7fe95d-f637-4021-a5e2-69668046d2ff\") " Dec 03 12:29:58 crc kubenswrapper[5052]: I1203 12:29:58.423082 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/f45e9eda-e62f-4307-ae18-2a8da7d6fe7c-metrics-certs-tls-certs\") pod \"f45e9eda-e62f-4307-ae18-2a8da7d6fe7c\" (UID: \"f45e9eda-e62f-4307-ae18-2a8da7d6fe7c\") " Dec 03 12:29:58 crc kubenswrapper[5052]: I1203 12:29:58.423132 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/5c7fe95d-f637-4021-a5e2-69668046d2ff-scripts\") pod \"5c7fe95d-f637-4021-a5e2-69668046d2ff\" (UID: \"5c7fe95d-f637-4021-a5e2-69668046d2ff\") " Dec 03 12:29:58 crc kubenswrapper[5052]: I1203 12:29:58.423218 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/5c7fe95d-f637-4021-a5e2-69668046d2ff-ovsdb-rundir\") pod \"5c7fe95d-f637-4021-a5e2-69668046d2ff\" (UID: \"5c7fe95d-f637-4021-a5e2-69668046d2ff\") " Dec 03 12:29:58 crc kubenswrapper[5052]: I1203 12:29:58.423247 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovndbcluster-nb-etc-ovn\" (UniqueName: \"kubernetes.io/local-volume/local-storage03-crc\") pod \"f45e9eda-e62f-4307-ae18-2a8da7d6fe7c\" (UID: \"f45e9eda-e62f-4307-ae18-2a8da7d6fe7c\") " Dec 03 12:29:58 crc kubenswrapper[5052]: I1203 12:29:58.423300 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovsdbserver-nb-tls-certs\" (UniqueName: \"kubernetes.io/secret/f45e9eda-e62f-4307-ae18-2a8da7d6fe7c-ovsdbserver-nb-tls-certs\") pod \"f45e9eda-e62f-4307-ae18-2a8da7d6fe7c\" (UID: \"f45e9eda-e62f-4307-ae18-2a8da7d6fe7c\") " Dec 03 12:29:58 crc kubenswrapper[5052]: I1203 12:29:58.423320 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/b35e6939-8f23-496c-ac55-9534be4ea8be-kolla-config\") pod \"b35e6939-8f23-496c-ac55-9534be4ea8be\" (UID: \"b35e6939-8f23-496c-ac55-9534be4ea8be\") " Dec 03 12:29:58 crc kubenswrapper[5052]: I1203 12:29:58.423347 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/f45e9eda-e62f-4307-ae18-2a8da7d6fe7c-scripts\") pod \"f45e9eda-e62f-4307-ae18-2a8da7d6fe7c\" (UID: \"f45e9eda-e62f-4307-ae18-2a8da7d6fe7c\") " Dec 03 12:29:58 crc kubenswrapper[5052]: I1203 12:29:58.423368 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/5c7fe95d-f637-4021-a5e2-69668046d2ff-metrics-certs-tls-certs\") pod \"5c7fe95d-f637-4021-a5e2-69668046d2ff\" (UID: \"5c7fe95d-f637-4021-a5e2-69668046d2ff\") " Dec 03 12:29:58 crc kubenswrapper[5052]: I1203 12:29:58.423398 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j2cn6\" (UniqueName: \"kubernetes.io/projected/b35e6939-8f23-496c-ac55-9534be4ea8be-kube-api-access-j2cn6\") pod \"b35e6939-8f23-496c-ac55-9534be4ea8be\" (UID: \"b35e6939-8f23-496c-ac55-9534be4ea8be\") " Dec 03 12:29:58 crc kubenswrapper[5052]: I1203 12:29:58.423454 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/f45e9eda-e62f-4307-ae18-2a8da7d6fe7c-ovsdb-rundir\") pod \"f45e9eda-e62f-4307-ae18-2a8da7d6fe7c\" (UID: \"f45e9eda-e62f-4307-ae18-2a8da7d6fe7c\") " Dec 03 12:29:58 crc kubenswrapper[5052]: I1203 12:29:58.401662 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5c7fe95d-f637-4021-a5e2-69668046d2ff-config" (OuterVolumeSpecName: "config") pod "5c7fe95d-f637-4021-a5e2-69668046d2ff" (UID: "5c7fe95d-f637-4021-a5e2-69668046d2ff"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:29:58 crc kubenswrapper[5052]: I1203 12:29:58.423769 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b35e6939-8f23-496c-ac55-9534be4ea8be-config-data" (OuterVolumeSpecName: "config-data") pod "b35e6939-8f23-496c-ac55-9534be4ea8be" (UID: "b35e6939-8f23-496c-ac55-9534be4ea8be"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:29:58 crc kubenswrapper[5052]: I1203 12:29:58.434452 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/nova-metadata-0" podUID="caaecb05-f5f1-421d-8e91-818774a2c0ec" containerName="nova-metadata-log" probeResult="failure" output="Get \"https://10.217.0.189:8775/\": read tcp 10.217.0.2:43506->10.217.0.189:8775: read: connection reset by peer" Dec 03 12:29:58 crc kubenswrapper[5052]: I1203 12:29:58.435007 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/nova-metadata-0" podUID="caaecb05-f5f1-421d-8e91-818774a2c0ec" containerName="nova-metadata-metadata" probeResult="failure" output="Get \"https://10.217.0.189:8775/\": read tcp 10.217.0.2:43490->10.217.0.189:8775: read: connection reset by peer" Dec 03 12:29:58 crc kubenswrapper[5052]: I1203 12:29:58.437854 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f45e9eda-e62f-4307-ae18-2a8da7d6fe7c-config" (OuterVolumeSpecName: "config") pod "f45e9eda-e62f-4307-ae18-2a8da7d6fe7c" (UID: "f45e9eda-e62f-4307-ae18-2a8da7d6fe7c"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:29:58 crc kubenswrapper[5052]: I1203 12:29:58.454188 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/53193ff3-1a41-4e26-b38f-957088aa7e8b-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:29:58 crc kubenswrapper[5052]: I1203 12:29:58.454221 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/b35e6939-8f23-496c-ac55-9534be4ea8be-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:29:58 crc kubenswrapper[5052]: I1203 12:29:58.454235 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-rzgf2\" (UniqueName: \"kubernetes.io/projected/53193ff3-1a41-4e26-b38f-957088aa7e8b-kube-api-access-rzgf2\") on node \"crc\" DevicePath \"\"" Dec 03 12:29:58 crc kubenswrapper[5052]: I1203 12:29:58.454248 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5c7fe95d-f637-4021-a5e2-69668046d2ff-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:29:58 crc kubenswrapper[5052]: I1203 12:29:58.454257 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f45e9eda-e62f-4307-ae18-2a8da7d6fe7c-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:29:58 crc kubenswrapper[5052]: I1203 12:29:58.458438 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5c7fe95d-f637-4021-a5e2-69668046d2ff-scripts" (OuterVolumeSpecName: "scripts") pod "5c7fe95d-f637-4021-a5e2-69668046d2ff" (UID: "5c7fe95d-f637-4021-a5e2-69668046d2ff"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:29:58 crc kubenswrapper[5052]: I1203 12:29:58.458645 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b35e6939-8f23-496c-ac55-9534be4ea8be-kolla-config" (OuterVolumeSpecName: "kolla-config") pod "b35e6939-8f23-496c-ac55-9534be4ea8be" (UID: "b35e6939-8f23-496c-ac55-9534be4ea8be"). InnerVolumeSpecName "kolla-config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:29:58 crc kubenswrapper[5052]: I1203 12:29:58.463255 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f45e9eda-e62f-4307-ae18-2a8da7d6fe7c-scripts" (OuterVolumeSpecName: "scripts") pod "f45e9eda-e62f-4307-ae18-2a8da7d6fe7c" (UID: "f45e9eda-e62f-4307-ae18-2a8da7d6fe7c"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:29:58 crc kubenswrapper[5052]: I1203 12:29:58.467653 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c7fe95d-f637-4021-a5e2-69668046d2ff-ovsdb-rundir" (OuterVolumeSpecName: "ovsdb-rundir") pod "5c7fe95d-f637-4021-a5e2-69668046d2ff" (UID: "5c7fe95d-f637-4021-a5e2-69668046d2ff"). InnerVolumeSpecName "ovsdb-rundir". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:29:58 crc kubenswrapper[5052]: I1203 12:29:58.475371 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/53193ff3-1a41-4e26-b38f-957088aa7e8b-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "53193ff3-1a41-4e26-b38f-957088aa7e8b" (UID: "53193ff3-1a41-4e26-b38f-957088aa7e8b"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:29:58 crc kubenswrapper[5052]: I1203 12:29:58.478134 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" event={"ID":"53193ff3-1a41-4e26-b38f-957088aa7e8b","Type":"ContainerDied","Data":"7aa40d31e35ffdc4b2176768f89abf61eaa08b7b0c8e93d4f2c0bd090c30d97f"} Dec 03 12:29:58 crc kubenswrapper[5052]: I1203 12:29:58.478180 5052 scope.go:117] "RemoveContainer" containerID="894cb495101405b55444078f27862c4a46c3937975fecb53b501499a7dc2a56d" Dec 03 12:29:58 crc kubenswrapper[5052]: I1203 12:29:58.478372 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:29:58 crc kubenswrapper[5052]: I1203 12:29:58.479366 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f45e9eda-e62f-4307-ae18-2a8da7d6fe7c-ovsdb-rundir" (OuterVolumeSpecName: "ovsdb-rundir") pod "f45e9eda-e62f-4307-ae18-2a8da7d6fe7c" (UID: "f45e9eda-e62f-4307-ae18-2a8da7d6fe7c"). InnerVolumeSpecName "ovsdb-rundir". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:29:58 crc kubenswrapper[5052]: I1203 12:29:58.491196 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage08-crc" (OuterVolumeSpecName: "ovndbcluster-sb-etc-ovn") pod "5c7fe95d-f637-4021-a5e2-69668046d2ff" (UID: "5c7fe95d-f637-4021-a5e2-69668046d2ff"). InnerVolumeSpecName "local-storage08-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 12:29:58 crc kubenswrapper[5052]: I1203 12:29:58.491286 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage03-crc" (OuterVolumeSpecName: "ovndbcluster-nb-etc-ovn") pod "f45e9eda-e62f-4307-ae18-2a8da7d6fe7c" (UID: "f45e9eda-e62f-4307-ae18-2a8da7d6fe7c"). InnerVolumeSpecName "local-storage03-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 12:29:58 crc kubenswrapper[5052]: I1203 12:29:58.491789 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f45e9eda-e62f-4307-ae18-2a8da7d6fe7c-kube-api-access-8n685" (OuterVolumeSpecName: "kube-api-access-8n685") pod "f45e9eda-e62f-4307-ae18-2a8da7d6fe7c" (UID: "f45e9eda-e62f-4307-ae18-2a8da7d6fe7c"). InnerVolumeSpecName "kube-api-access-8n685". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:29:58 crc kubenswrapper[5052]: I1203 12:29:58.491869 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/53193ff3-1a41-4e26-b38f-957088aa7e8b-vencrypt-tls-certs" (OuterVolumeSpecName: "vencrypt-tls-certs") pod "53193ff3-1a41-4e26-b38f-957088aa7e8b" (UID: "53193ff3-1a41-4e26-b38f-957088aa7e8b"). InnerVolumeSpecName "vencrypt-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:29:58 crc kubenswrapper[5052]: I1203 12:29:58.491962 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstackclient" Dec 03 12:29:58 crc kubenswrapper[5052]: I1203 12:29:58.492033 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-5fc7894bfd-g7hl5" event={"ID":"cfb8e98d-3873-43e0-81dc-4450983179c2","Type":"ContainerStarted","Data":"0a861967403e553bfc0bb4f31723c2962d1eb4b87dfc36853566225a16745bf9"} Dec 03 12:29:58 crc kubenswrapper[5052]: I1203 12:29:58.501284 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b35e6939-8f23-496c-ac55-9534be4ea8be-kube-api-access-j2cn6" (OuterVolumeSpecName: "kube-api-access-j2cn6") pod "b35e6939-8f23-496c-ac55-9534be4ea8be" (UID: "b35e6939-8f23-496c-ac55-9534be4ea8be"). InnerVolumeSpecName "kube-api-access-j2cn6". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:29:58 crc kubenswrapper[5052]: I1203 12:29:58.527297 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5c7fe95d-f637-4021-a5e2-69668046d2ff-kube-api-access-67l4w" (OuterVolumeSpecName: "kube-api-access-67l4w") pod "5c7fe95d-f637-4021-a5e2-69668046d2ff" (UID: "5c7fe95d-f637-4021-a5e2-69668046d2ff"). InnerVolumeSpecName "kube-api-access-67l4w". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:29:58 crc kubenswrapper[5052]: I1203 12:29:58.530020 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_ovsdbserver-nb-0_f45e9eda-e62f-4307-ae18-2a8da7d6fe7c/ovsdbserver-nb/0.log" Dec 03 12:29:58 crc kubenswrapper[5052]: I1203 12:29:58.530110 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-nb-0" event={"ID":"f45e9eda-e62f-4307-ae18-2a8da7d6fe7c","Type":"ContainerDied","Data":"51139b01af3d8c144c68f7acd6378312fcf37bab84f5b5451556bdcd8b8d38af"} Dec 03 12:29:58 crc kubenswrapper[5052]: I1203 12:29:58.530153 5052 scope.go:117] "RemoveContainer" containerID="901636e0be8460bcc74b0c6a41492e349cbdbd334bbc41042f9dfdf0a187e9d1" Dec 03 12:29:58 crc kubenswrapper[5052]: I1203 12:29:58.530312 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:29:58 crc kubenswrapper[5052]: E1203 12:29:58.534182 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="165a4ba65f86633615c852ba29ceb303c3bbd0dbd696084cf0c3a26afe0a4424" cmd=["/bin/bash","/var/lib/operator-scripts/mysql_probe.sh","readiness"] Dec 03 12:29:58 crc kubenswrapper[5052]: E1203 12:29:58.545367 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="165a4ba65f86633615c852ba29ceb303c3bbd0dbd696084cf0c3a26afe0a4424" cmd=["/bin/bash","/var/lib/operator-scripts/mysql_probe.sh","readiness"] Dec 03 12:29:58 crc kubenswrapper[5052]: I1203 12:29:58.556828 5052 reconciler_common.go:293] "Volume detached for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/5c7fe95d-f637-4021-a5e2-69668046d2ff-ovsdb-rundir\") on node \"crc\" DevicePath \"\"" Dec 03 12:29:58 crc kubenswrapper[5052]: I1203 12:29:58.556870 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage03-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage03-crc\") on node \"crc\" " Dec 03 12:29:58 crc kubenswrapper[5052]: I1203 12:29:58.556880 5052 reconciler_common.go:293] "Volume detached for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/b35e6939-8f23-496c-ac55-9534be4ea8be-kolla-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:29:58 crc kubenswrapper[5052]: I1203 12:29:58.556890 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/f45e9eda-e62f-4307-ae18-2a8da7d6fe7c-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:29:58 crc kubenswrapper[5052]: I1203 12:29:58.556901 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-j2cn6\" (UniqueName: \"kubernetes.io/projected/b35e6939-8f23-496c-ac55-9534be4ea8be-kube-api-access-j2cn6\") on node \"crc\" DevicePath \"\"" Dec 03 12:29:58 crc kubenswrapper[5052]: I1203 12:29:58.556913 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/53193ff3-1a41-4e26-b38f-957088aa7e8b-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:29:58 crc kubenswrapper[5052]: I1203 12:29:58.556922 5052 reconciler_common.go:293] "Volume detached for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/f45e9eda-e62f-4307-ae18-2a8da7d6fe7c-ovsdb-rundir\") on node \"crc\" DevicePath \"\"" Dec 03 12:29:58 crc kubenswrapper[5052]: I1203 12:29:58.556930 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-8n685\" (UniqueName: \"kubernetes.io/projected/f45e9eda-e62f-4307-ae18-2a8da7d6fe7c-kube-api-access-8n685\") on node \"crc\" DevicePath \"\"" Dec 03 12:29:58 crc kubenswrapper[5052]: I1203 12:29:58.556938 5052 reconciler_common.go:293] "Volume detached for volume \"vencrypt-tls-certs\" (UniqueName: \"kubernetes.io/secret/53193ff3-1a41-4e26-b38f-957088aa7e8b-vencrypt-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:29:58 crc kubenswrapper[5052]: I1203 12:29:58.556946 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-67l4w\" (UniqueName: \"kubernetes.io/projected/5c7fe95d-f637-4021-a5e2-69668046d2ff-kube-api-access-67l4w\") on node \"crc\" DevicePath \"\"" Dec 03 12:29:58 crc kubenswrapper[5052]: I1203 12:29:58.556959 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage08-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage08-crc\") on node \"crc\" " Dec 03 12:29:58 crc kubenswrapper[5052]: I1203 12:29:58.556969 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/5c7fe95d-f637-4021-a5e2-69668046d2ff-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:29:58 crc kubenswrapper[5052]: I1203 12:29:58.577152 5052 generic.go:334] "Generic (PLEG): container finished" podID="78630d10-b9ba-438d-907d-2be706fc51e7" containerID="7c24093b1dde24c14b80a87bb357568dbd53207a52044ef5dcbeb86466b30f11" exitCode=137 Dec 03 12:29:58 crc kubenswrapper[5052]: I1203 12:29:58.577452 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstackclient" Dec 03 12:29:58 crc kubenswrapper[5052]: E1203 12:29:58.585052 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="165a4ba65f86633615c852ba29ceb303c3bbd0dbd696084cf0c3a26afe0a4424" cmd=["/bin/bash","/var/lib/operator-scripts/mysql_probe.sh","readiness"] Dec 03 12:29:58 crc kubenswrapper[5052]: E1203 12:29:58.586427 5052 prober.go:104] "Probe errored" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" probeType="Readiness" pod="openstack-kuttl-tests/openstack-galera-0" podUID="12f39c79-eeb9-4cf4-8553-1a4d9a07ee82" containerName="galera" Dec 03 12:29:58 crc kubenswrapper[5052]: I1203 12:29:58.595116 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_ovsdbserver-sb-0_5c7fe95d-f637-4021-a5e2-69668046d2ff/ovsdbserver-sb/0.log" Dec 03 12:29:58 crc kubenswrapper[5052]: I1203 12:29:58.595348 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-sb-0" event={"ID":"5c7fe95d-f637-4021-a5e2-69668046d2ff","Type":"ContainerDied","Data":"31511075d0844903f8bc40b464ebcdc51e2d3f8cd1b35d091216e2e8554c72d3"} Dec 03 12:29:58 crc kubenswrapper[5052]: I1203 12:29:58.595506 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:29:58 crc kubenswrapper[5052]: I1203 12:29:58.637367 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5c7fe95d-f637-4021-a5e2-69668046d2ff-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "5c7fe95d-f637-4021-a5e2-69668046d2ff" (UID: "5c7fe95d-f637-4021-a5e2-69668046d2ff"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:29:58 crc kubenswrapper[5052]: I1203 12:29:58.643546 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-db474c7dd-tg7mq" event={"ID":"886ea0a4-5096-4ea8-a74c-de243d1f4f40","Type":"ContainerStarted","Data":"c4ff4fbe3dacec716771465b6ff8ae9204a17ae870e13c0be6a701515ebce0cc"} Dec 03 12:29:58 crc kubenswrapper[5052]: I1203 12:29:58.648122 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-5c6478db8b-bt2rw" event={"ID":"8bddf2b1-8116-4b6e-9cd8-be8fe1cd4b64","Type":"ContainerStarted","Data":"bbf760a832ee67de313c68ecfd209742e1ca2c2298b42e73aea44c6e592aa9d7"} Dec 03 12:29:58 crc kubenswrapper[5052]: I1203 12:29:58.652568 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage08-crc" (UniqueName: "kubernetes.io/local-volume/local-storage08-crc") on node "crc" Dec 03 12:29:58 crc kubenswrapper[5052]: I1203 12:29:58.661637 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b35e6939-8f23-496c-ac55-9534be4ea8be-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "b35e6939-8f23-496c-ac55-9534be4ea8be" (UID: "b35e6939-8f23-496c-ac55-9534be4ea8be"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:29:58 crc kubenswrapper[5052]: I1203 12:29:58.662492 5052 generic.go:334] "Generic (PLEG): container finished" podID="e75eea2c-2eb1-4f0f-97e5-63ce147e68dd" containerID="da3d10f0fb54aeb0fa5fdc57f022b12303d52fa87da0b230d95a66aac9d2be59" exitCode=0 Dec 03 12:29:58 crc kubenswrapper[5052]: I1203 12:29:58.662528 5052 generic.go:334] "Generic (PLEG): container finished" podID="e75eea2c-2eb1-4f0f-97e5-63ce147e68dd" containerID="102038f6978c676a1d8853f850be294a8aea8d6b74a6f1949c24231d714a3608" exitCode=2 Dec 03 12:29:58 crc kubenswrapper[5052]: I1203 12:29:58.662575 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"e75eea2c-2eb1-4f0f-97e5-63ce147e68dd","Type":"ContainerDied","Data":"da3d10f0fb54aeb0fa5fdc57f022b12303d52fa87da0b230d95a66aac9d2be59"} Dec 03 12:29:58 crc kubenswrapper[5052]: I1203 12:29:58.662618 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"e75eea2c-2eb1-4f0f-97e5-63ce147e68dd","Type":"ContainerDied","Data":"102038f6978c676a1d8853f850be294a8aea8d6b74a6f1949c24231d714a3608"} Dec 03 12:29:58 crc kubenswrapper[5052]: I1203 12:29:58.664818 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-85595\" (UniqueName: \"kubernetes.io/projected/78630d10-b9ba-438d-907d-2be706fc51e7-kube-api-access-85595\") pod \"78630d10-b9ba-438d-907d-2be706fc51e7\" (UID: \"78630d10-b9ba-438d-907d-2be706fc51e7\") " Dec 03 12:29:58 crc kubenswrapper[5052]: I1203 12:29:58.668425 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/78630d10-b9ba-438d-907d-2be706fc51e7-combined-ca-bundle\") pod \"78630d10-b9ba-438d-907d-2be706fc51e7\" (UID: \"78630d10-b9ba-438d-907d-2be706fc51e7\") " Dec 03 12:29:58 crc kubenswrapper[5052]: I1203 12:29:58.670152 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"openstack-config\" (UniqueName: \"kubernetes.io/configmap/78630d10-b9ba-438d-907d-2be706fc51e7-openstack-config\") pod \"78630d10-b9ba-438d-907d-2be706fc51e7\" (UID: \"78630d10-b9ba-438d-907d-2be706fc51e7\") " Dec 03 12:29:58 crc kubenswrapper[5052]: I1203 12:29:58.670525 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/78630d10-b9ba-438d-907d-2be706fc51e7-kube-api-access-85595" (OuterVolumeSpecName: "kube-api-access-85595") pod "78630d10-b9ba-438d-907d-2be706fc51e7" (UID: "78630d10-b9ba-438d-907d-2be706fc51e7"). InnerVolumeSpecName "kube-api-access-85595". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:29:58 crc kubenswrapper[5052]: I1203 12:29:58.670710 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"openstack-config-secret\" (UniqueName: \"kubernetes.io/secret/78630d10-b9ba-438d-907d-2be706fc51e7-openstack-config-secret\") pod \"78630d10-b9ba-438d-907d-2be706fc51e7\" (UID: \"78630d10-b9ba-438d-907d-2be706fc51e7\") " Dec 03 12:29:58 crc kubenswrapper[5052]: I1203 12:29:58.670719 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/memcached-0" Dec 03 12:29:58 crc kubenswrapper[5052]: I1203 12:29:58.671215 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/memcached-0" event={"ID":"b35e6939-8f23-496c-ac55-9534be4ea8be","Type":"ContainerDied","Data":"ef859d2fc4d61f028b9227837d828fbe4c68fa551371ec131c1b1149ab6d5d3f"} Dec 03 12:29:58 crc kubenswrapper[5052]: I1203 12:29:58.675011 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5c7fe95d-f637-4021-a5e2-69668046d2ff-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:29:58 crc kubenswrapper[5052]: I1203 12:29:58.675320 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b35e6939-8f23-496c-ac55-9534be4ea8be-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:29:58 crc kubenswrapper[5052]: I1203 12:29:58.675336 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage08-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage08-crc\") on node \"crc\" DevicePath \"\"" Dec 03 12:29:58 crc kubenswrapper[5052]: I1203 12:29:58.675351 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-85595\" (UniqueName: \"kubernetes.io/projected/78630d10-b9ba-438d-907d-2be706fc51e7-kube-api-access-85595\") on node \"crc\" DevicePath \"\"" Dec 03 12:29:58 crc kubenswrapper[5052]: I1203 12:29:58.760006 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/53193ff3-1a41-4e26-b38f-957088aa7e8b-nova-novncproxy-tls-certs" (OuterVolumeSpecName: "nova-novncproxy-tls-certs") pod "53193ff3-1a41-4e26-b38f-957088aa7e8b" (UID: "53193ff3-1a41-4e26-b38f-957088aa7e8b"). InnerVolumeSpecName "nova-novncproxy-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:29:58 crc kubenswrapper[5052]: I1203 12:29:58.761664 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f45e9eda-e62f-4307-ae18-2a8da7d6fe7c-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "f45e9eda-e62f-4307-ae18-2a8da7d6fe7c" (UID: "f45e9eda-e62f-4307-ae18-2a8da7d6fe7c"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:29:58 crc kubenswrapper[5052]: I1203 12:29:58.765367 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-keystone-listener-cd5b58cfd-jf8vt"] Dec 03 12:29:58 crc kubenswrapper[5052]: I1203 12:29:58.772545 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/78630d10-b9ba-438d-907d-2be706fc51e7-openstack-config" (OuterVolumeSpecName: "openstack-config") pod "78630d10-b9ba-438d-907d-2be706fc51e7" (UID: "78630d10-b9ba-438d-907d-2be706fc51e7"). InnerVolumeSpecName "openstack-config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:29:58 crc kubenswrapper[5052]: I1203 12:29:58.777395 5052 reconciler_common.go:293] "Volume detached for volume \"openstack-config\" (UniqueName: \"kubernetes.io/configmap/78630d10-b9ba-438d-907d-2be706fc51e7-openstack-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:29:58 crc kubenswrapper[5052]: I1203 12:29:58.777431 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f45e9eda-e62f-4307-ae18-2a8da7d6fe7c-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:29:58 crc kubenswrapper[5052]: I1203 12:29:58.777444 5052 reconciler_common.go:293] "Volume detached for volume \"nova-novncproxy-tls-certs\" (UniqueName: \"kubernetes.io/secret/53193ff3-1a41-4e26-b38f-957088aa7e8b-nova-novncproxy-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:29:58 crc kubenswrapper[5052]: I1203 12:29:58.785281 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-worker-644558fbc5-4gtxj"] Dec 03 12:29:58 crc kubenswrapper[5052]: W1203 12:29:58.800955 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8ac6e8f0_d49c_4a4c_846a_6cec226bd360.slice/crio-4546b826c99134ff60d23f10a99bb12e6e7977932e01290b96ec7338744adb66 WatchSource:0}: Error finding container 4546b826c99134ff60d23f10a99bb12e6e7977932e01290b96ec7338744adb66: Status 404 returned error can't find the container with id 4546b826c99134ff60d23f10a99bb12e6e7977932e01290b96ec7338744adb66 Dec 03 12:29:58 crc kubenswrapper[5052]: W1203 12:29:58.803795 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod019689d4_d40b_4e9a_a028_e427f1aa9d9b.slice/crio-1388cc58677cb99a408eeb9662787e5c6d0865f5f7b0080c83b290fe93da2278 WatchSource:0}: Error finding container 1388cc58677cb99a408eeb9662787e5c6d0865f5f7b0080c83b290fe93da2278: Status 404 returned error can't find the container with id 1388cc58677cb99a408eeb9662787e5c6d0865f5f7b0080c83b290fe93da2278 Dec 03 12:29:58 crc kubenswrapper[5052]: I1203 12:29:58.846121 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/78630d10-b9ba-438d-907d-2be706fc51e7-openstack-config-secret" (OuterVolumeSpecName: "openstack-config-secret") pod "78630d10-b9ba-438d-907d-2be706fc51e7" (UID: "78630d10-b9ba-438d-907d-2be706fc51e7"). InnerVolumeSpecName "openstack-config-secret". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:29:58 crc kubenswrapper[5052]: I1203 12:29:58.862040 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f45e9eda-e62f-4307-ae18-2a8da7d6fe7c-metrics-certs-tls-certs" (OuterVolumeSpecName: "metrics-certs-tls-certs") pod "f45e9eda-e62f-4307-ae18-2a8da7d6fe7c" (UID: "f45e9eda-e62f-4307-ae18-2a8da7d6fe7c"). InnerVolumeSpecName "metrics-certs-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:29:58 crc kubenswrapper[5052]: I1203 12:29:58.864918 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage03-crc" (UniqueName: "kubernetes.io/local-volume/local-storage03-crc") on node "crc" Dec 03 12:29:58 crc kubenswrapper[5052]: I1203 12:29:58.880011 5052 reconciler_common.go:293] "Volume detached for volume \"openstack-config-secret\" (UniqueName: \"kubernetes.io/secret/78630d10-b9ba-438d-907d-2be706fc51e7-openstack-config-secret\") on node \"crc\" DevicePath \"\"" Dec 03 12:29:58 crc kubenswrapper[5052]: I1203 12:29:58.880046 5052 reconciler_common.go:293] "Volume detached for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/f45e9eda-e62f-4307-ae18-2a8da7d6fe7c-metrics-certs-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:29:58 crc kubenswrapper[5052]: I1203 12:29:58.880057 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage03-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage03-crc\") on node \"crc\" DevicePath \"\"" Dec 03 12:29:58 crc kubenswrapper[5052]: I1203 12:29:58.912320 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/cinder-api-0" podUID="61cad274-57bd-4993-aaa6-3e6013900d88" containerName="cinder-api" probeResult="failure" output="Get \"https://10.217.0.159:8776/healthcheck\": read tcp 10.217.0.2:55384->10.217.0.159:8776: read: connection reset by peer" Dec 03 12:29:58 crc kubenswrapper[5052]: I1203 12:29:58.912779 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-56f9c7c6b6-kvxpr"] Dec 03 12:29:58 crc kubenswrapper[5052]: I1203 12:29:58.942319 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5c7fe95d-f637-4021-a5e2-69668046d2ff-metrics-certs-tls-certs" (OuterVolumeSpecName: "metrics-certs-tls-certs") pod "5c7fe95d-f637-4021-a5e2-69668046d2ff" (UID: "5c7fe95d-f637-4021-a5e2-69668046d2ff"). InnerVolumeSpecName "metrics-certs-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:29:58 crc kubenswrapper[5052]: I1203 12:29:58.975200 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b35e6939-8f23-496c-ac55-9534be4ea8be-memcached-tls-certs" (OuterVolumeSpecName: "memcached-tls-certs") pod "b35e6939-8f23-496c-ac55-9534be4ea8be" (UID: "b35e6939-8f23-496c-ac55-9534be4ea8be"). InnerVolumeSpecName "memcached-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:29:58 crc kubenswrapper[5052]: I1203 12:29:58.982338 5052 reconciler_common.go:293] "Volume detached for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/5c7fe95d-f637-4021-a5e2-69668046d2ff-metrics-certs-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:29:58 crc kubenswrapper[5052]: I1203 12:29:58.982372 5052 reconciler_common.go:293] "Volume detached for volume \"memcached-tls-certs\" (UniqueName: \"kubernetes.io/secret/b35e6939-8f23-496c-ac55-9534be4ea8be-memcached-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:29:58 crc kubenswrapper[5052]: I1203 12:29:58.991361 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5c7fe95d-f637-4021-a5e2-69668046d2ff-ovsdbserver-sb-tls-certs" (OuterVolumeSpecName: "ovsdbserver-sb-tls-certs") pod "5c7fe95d-f637-4021-a5e2-69668046d2ff" (UID: "5c7fe95d-f637-4021-a5e2-69668046d2ff"). InnerVolumeSpecName "ovsdbserver-sb-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:29:58 crc kubenswrapper[5052]: I1203 12:29:58.993692 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/78630d10-b9ba-438d-907d-2be706fc51e7-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "78630d10-b9ba-438d-907d-2be706fc51e7" (UID: "78630d10-b9ba-438d-907d-2be706fc51e7"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.031302 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f45e9eda-e62f-4307-ae18-2a8da7d6fe7c-ovsdbserver-nb-tls-certs" (OuterVolumeSpecName: "ovsdbserver-nb-tls-certs") pod "f45e9eda-e62f-4307-ae18-2a8da7d6fe7c" (UID: "f45e9eda-e62f-4307-ae18-2a8da7d6fe7c"). InnerVolumeSpecName "ovsdbserver-nb-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.086179 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/78630d10-b9ba-438d-907d-2be706fc51e7-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.086205 5052 reconciler_common.go:293] "Volume detached for volume \"ovsdbserver-sb-tls-certs\" (UniqueName: \"kubernetes.io/secret/5c7fe95d-f637-4021-a5e2-69668046d2ff-ovsdbserver-sb-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.086217 5052 reconciler_common.go:293] "Volume detached for volume \"ovsdbserver-nb-tls-certs\" (UniqueName: \"kubernetes.io/secret/f45e9eda-e62f-4307-ae18-2a8da7d6fe7c-ovsdbserver-nb-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.129973 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.151236 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/placement-7dffdbf8-hzmbm"] Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.161505 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.170539 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 12:29:59 crc kubenswrapper[5052]: E1203 12:29:59.171004 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="5c7fe95d-f637-4021-a5e2-69668046d2ff" containerName="openstack-network-exporter" Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.171016 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c7fe95d-f637-4021-a5e2-69668046d2ff" containerName="openstack-network-exporter" Dec 03 12:29:59 crc kubenswrapper[5052]: E1203 12:29:59.171040 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="f45e9eda-e62f-4307-ae18-2a8da7d6fe7c" containerName="openstack-network-exporter" Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.171046 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="f45e9eda-e62f-4307-ae18-2a8da7d6fe7c" containerName="openstack-network-exporter" Dec 03 12:29:59 crc kubenswrapper[5052]: E1203 12:29:59.171060 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="b35e6939-8f23-496c-ac55-9534be4ea8be" containerName="memcached" Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.171068 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="b35e6939-8f23-496c-ac55-9534be4ea8be" containerName="memcached" Dec 03 12:29:59 crc kubenswrapper[5052]: E1203 12:29:59.171086 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="f45e9eda-e62f-4307-ae18-2a8da7d6fe7c" containerName="ovsdbserver-nb" Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.171092 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="f45e9eda-e62f-4307-ae18-2a8da7d6fe7c" containerName="ovsdbserver-nb" Dec 03 12:29:59 crc kubenswrapper[5052]: E1203 12:29:59.171111 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="53193ff3-1a41-4e26-b38f-957088aa7e8b" containerName="nova-cell1-novncproxy-novncproxy" Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.171117 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="53193ff3-1a41-4e26-b38f-957088aa7e8b" containerName="nova-cell1-novncproxy-novncproxy" Dec 03 12:29:59 crc kubenswrapper[5052]: E1203 12:29:59.171126 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="5c7fe95d-f637-4021-a5e2-69668046d2ff" containerName="ovsdbserver-sb" Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.171131 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c7fe95d-f637-4021-a5e2-69668046d2ff" containerName="ovsdbserver-sb" Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.171304 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="53193ff3-1a41-4e26-b38f-957088aa7e8b" containerName="nova-cell1-novncproxy-novncproxy" Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.171318 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="f45e9eda-e62f-4307-ae18-2a8da7d6fe7c" containerName="ovsdbserver-nb" Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.171329 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="5c7fe95d-f637-4021-a5e2-69668046d2ff" containerName="openstack-network-exporter" Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.171338 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="b35e6939-8f23-496c-ac55-9534be4ea8be" containerName="memcached" Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.171351 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="5c7fe95d-f637-4021-a5e2-69668046d2ff" containerName="ovsdbserver-sb" Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.171367 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="f45e9eda-e62f-4307-ae18-2a8da7d6fe7c" containerName="openstack-network-exporter" Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.172069 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.174305 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-nova-novncproxy-cell1-vencrypt" Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.174459 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell1-novncproxy-config-data" Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.179625 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.181605 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-nova-novncproxy-cell1-public-svc" Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.215891 5052 scope.go:117] "RemoveContainer" containerID="09275335090deb24b922ed4694cb037b39bf0e6b72303e640ea22dfe86fa1716" Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.261209 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-nb-0"] Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.283643 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-nb-0"] Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.285973 5052 scope.go:117] "RemoveContainer" containerID="7c24093b1dde24c14b80a87bb357568dbd53207a52044ef5dcbeb86466b30f11" Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.296769 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lr65w\" (UniqueName: \"kubernetes.io/projected/f4e095cd-36d8-4136-b262-f6d52717a05c-kube-api-access-lr65w\") pod \"nova-cell1-novncproxy-0\" (UID: \"f4e095cd-36d8-4136-b262-f6d52717a05c\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.296824 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nova-novncproxy-tls-certs\" (UniqueName: \"kubernetes.io/secret/f4e095cd-36d8-4136-b262-f6d52717a05c-nova-novncproxy-tls-certs\") pod \"nova-cell1-novncproxy-0\" (UID: \"f4e095cd-36d8-4136-b262-f6d52717a05c\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.296854 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/f4e095cd-36d8-4136-b262-f6d52717a05c-config-data\") pod \"nova-cell1-novncproxy-0\" (UID: \"f4e095cd-36d8-4136-b262-f6d52717a05c\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.296908 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f4e095cd-36d8-4136-b262-f6d52717a05c-combined-ca-bundle\") pod \"nova-cell1-novncproxy-0\" (UID: \"f4e095cd-36d8-4136-b262-f6d52717a05c\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.299801 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"vencrypt-tls-certs\" (UniqueName: \"kubernetes.io/secret/f4e095cd-36d8-4136-b262-f6d52717a05c-vencrypt-tls-certs\") pod \"nova-cell1-novncproxy-0\" (UID: \"f4e095cd-36d8-4136-b262-f6d52717a05c\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.301532 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-api-ff6b8c44b-7ndgm"] Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.318226 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ovsdbserver-nb-0"] Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.321671 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.326796 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-ovndbcluster-nb-ovndbs" Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.327450 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ovncluster-ovndbcluster-nb-dockercfg-4462b" Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.330672 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"ovndbcluster-nb-config" Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.330722 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"ovndbcluster-nb-scripts" Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.335804 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-564bc88b6-bvll5"] Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.351766 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-nb-0"] Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.362194 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-sb-0"] Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.373577 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-sb-0"] Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.382476 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ovsdbserver-sb-0"] Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.386774 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.390800 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/memcached-0"] Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.391961 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"ovndbcluster-sb-scripts" Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.392057 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"ovndbcluster-sb-config" Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.392063 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-ovndbcluster-sb-ovndbs" Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.392220 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ovncluster-ovndbcluster-sb-dockercfg-2ckjk" Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.404908 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-lr65w\" (UniqueName: \"kubernetes.io/projected/f4e095cd-36d8-4136-b262-f6d52717a05c-kube-api-access-lr65w\") pod \"nova-cell1-novncproxy-0\" (UID: \"f4e095cd-36d8-4136-b262-f6d52717a05c\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.404979 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"nova-novncproxy-tls-certs\" (UniqueName: \"kubernetes.io/secret/f4e095cd-36d8-4136-b262-f6d52717a05c-nova-novncproxy-tls-certs\") pod \"nova-cell1-novncproxy-0\" (UID: \"f4e095cd-36d8-4136-b262-f6d52717a05c\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.405013 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/f4e095cd-36d8-4136-b262-f6d52717a05c-config-data\") pod \"nova-cell1-novncproxy-0\" (UID: \"f4e095cd-36d8-4136-b262-f6d52717a05c\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.405051 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f4e095cd-36d8-4136-b262-f6d52717a05c-combined-ca-bundle\") pod \"nova-cell1-novncproxy-0\" (UID: \"f4e095cd-36d8-4136-b262-f6d52717a05c\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.405248 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"vencrypt-tls-certs\" (UniqueName: \"kubernetes.io/secret/f4e095cd-36d8-4136-b262-f6d52717a05c-vencrypt-tls-certs\") pod \"nova-cell1-novncproxy-0\" (UID: \"f4e095cd-36d8-4136-b262-f6d52717a05c\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.406062 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/memcached-0"] Dec 03 12:29:59 crc kubenswrapper[5052]: W1203 12:29:59.406312 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc0ce9ced_33b0_4879_81a4_b5c066ec0631.slice/crio-7d60e6875dab88bfe727e3ecc5b73bec4dc81727c206ef38a830168d386edffa WatchSource:0}: Error finding container 7d60e6875dab88bfe727e3ecc5b73bec4dc81727c206ef38a830168d386edffa: Status 404 returned error can't find the container with id 7d60e6875dab88bfe727e3ecc5b73bec4dc81727c206ef38a830168d386edffa Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.422076 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"nova-novncproxy-tls-certs\" (UniqueName: \"kubernetes.io/secret/f4e095cd-36d8-4136-b262-f6d52717a05c-nova-novncproxy-tls-certs\") pod \"nova-cell1-novncproxy-0\" (UID: \"f4e095cd-36d8-4136-b262-f6d52717a05c\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.427030 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"vencrypt-tls-certs\" (UniqueName: \"kubernetes.io/secret/f4e095cd-36d8-4136-b262-f6d52717a05c-vencrypt-tls-certs\") pod \"nova-cell1-novncproxy-0\" (UID: \"f4e095cd-36d8-4136-b262-f6d52717a05c\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.427094 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-sb-0"] Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.436414 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/memcached-0"] Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.436966 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/f4e095cd-36d8-4136-b262-f6d52717a05c-config-data\") pod \"nova-cell1-novncproxy-0\" (UID: \"f4e095cd-36d8-4136-b262-f6d52717a05c\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.438116 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/memcached-0" Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.438905 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f4e095cd-36d8-4136-b262-f6d52717a05c-combined-ca-bundle\") pod \"nova-cell1-novncproxy-0\" (UID: \"f4e095cd-36d8-4136-b262-f6d52717a05c\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.440440 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"memcached-config-data" Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.440683 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-memcached-svc" Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.440897 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"memcached-memcached-dockercfg-2sngt" Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.448454 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-lr65w\" (UniqueName: \"kubernetes.io/projected/f4e095cd-36d8-4136-b262-f6d52717a05c-kube-api-access-lr65w\") pod \"nova-cell1-novncproxy-0\" (UID: \"f4e095cd-36d8-4136-b262-f6d52717a05c\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.477957 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/memcached-0"] Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.512240 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovsdbserver-sb-tls-certs\" (UniqueName: \"kubernetes.io/secret/50f5af82-8ee5-45f4-b158-2166f1be6cb8-ovsdbserver-sb-tls-certs\") pod \"ovsdbserver-sb-0\" (UID: \"50f5af82-8ee5-45f4-b158-2166f1be6cb8\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.512345 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/98f722f9-fa25-40b6-84ba-96127d07293c-config\") pod \"ovsdbserver-nb-0\" (UID: \"98f722f9-fa25-40b6-84ba-96127d07293c\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.512399 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4jrvq\" (UniqueName: \"kubernetes.io/projected/98f722f9-fa25-40b6-84ba-96127d07293c-kube-api-access-4jrvq\") pod \"ovsdbserver-nb-0\" (UID: \"98f722f9-fa25-40b6-84ba-96127d07293c\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.512431 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/98f722f9-fa25-40b6-84ba-96127d07293c-metrics-certs-tls-certs\") pod \"ovsdbserver-nb-0\" (UID: \"98f722f9-fa25-40b6-84ba-96127d07293c\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.512459 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dp9m4\" (UniqueName: \"kubernetes.io/projected/50f5af82-8ee5-45f4-b158-2166f1be6cb8-kube-api-access-dp9m4\") pod \"ovsdbserver-sb-0\" (UID: \"50f5af82-8ee5-45f4-b158-2166f1be6cb8\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.512483 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/98f722f9-fa25-40b6-84ba-96127d07293c-scripts\") pod \"ovsdbserver-nb-0\" (UID: \"98f722f9-fa25-40b6-84ba-96127d07293c\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.512568 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/50f5af82-8ee5-45f4-b158-2166f1be6cb8-config\") pod \"ovsdbserver-sb-0\" (UID: \"50f5af82-8ee5-45f4-b158-2166f1be6cb8\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.512652 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/50f5af82-8ee5-45f4-b158-2166f1be6cb8-combined-ca-bundle\") pod \"ovsdbserver-sb-0\" (UID: \"50f5af82-8ee5-45f4-b158-2166f1be6cb8\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.512858 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovsdbserver-nb-tls-certs\" (UniqueName: \"kubernetes.io/secret/98f722f9-fa25-40b6-84ba-96127d07293c-ovsdbserver-nb-tls-certs\") pod \"ovsdbserver-nb-0\" (UID: \"98f722f9-fa25-40b6-84ba-96127d07293c\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.512983 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/50f5af82-8ee5-45f4-b158-2166f1be6cb8-metrics-certs-tls-certs\") pod \"ovsdbserver-sb-0\" (UID: \"50f5af82-8ee5-45f4-b158-2166f1be6cb8\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.513013 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage03-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage03-crc\") pod \"ovsdbserver-nb-0\" (UID: \"98f722f9-fa25-40b6-84ba-96127d07293c\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.513044 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/50f5af82-8ee5-45f4-b158-2166f1be6cb8-ovsdb-rundir\") pod \"ovsdbserver-sb-0\" (UID: \"50f5af82-8ee5-45f4-b158-2166f1be6cb8\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.513093 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage08-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage08-crc\") pod \"ovsdbserver-sb-0\" (UID: \"50f5af82-8ee5-45f4-b158-2166f1be6cb8\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.513126 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/98f722f9-fa25-40b6-84ba-96127d07293c-ovsdb-rundir\") pod \"ovsdbserver-nb-0\" (UID: \"98f722f9-fa25-40b6-84ba-96127d07293c\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.513209 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/98f722f9-fa25-40b6-84ba-96127d07293c-combined-ca-bundle\") pod \"ovsdbserver-nb-0\" (UID: \"98f722f9-fa25-40b6-84ba-96127d07293c\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.513232 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/50f5af82-8ee5-45f4-b158-2166f1be6cb8-scripts\") pod \"ovsdbserver-sb-0\" (UID: \"50f5af82-8ee5-45f4-b158-2166f1be6cb8\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.520411 5052 scope.go:117] "RemoveContainer" containerID="b01f8b34b21ba5bc171276b7695dd41167a5b350ce509d80a35ce85a684f8937" Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.566276 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.570965 5052 scope.go:117] "RemoveContainer" containerID="63b5496e0ce2e84297ac272518addd75c23a27efda2a42a79c0e221732c3c3c6" Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.614717 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/98f722f9-fa25-40b6-84ba-96127d07293c-ovsdb-rundir\") pod \"ovsdbserver-nb-0\" (UID: \"98f722f9-fa25-40b6-84ba-96127d07293c\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.614785 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/19fe6517-a9bd-4463-b2b1-e006bd5d8145-config-data\") pod \"memcached-0\" (UID: \"19fe6517-a9bd-4463-b2b1-e006bd5d8145\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.614822 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/19fe6517-a9bd-4463-b2b1-e006bd5d8145-kolla-config\") pod \"memcached-0\" (UID: \"19fe6517-a9bd-4463-b2b1-e006bd5d8145\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.614854 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/98f722f9-fa25-40b6-84ba-96127d07293c-combined-ca-bundle\") pod \"ovsdbserver-nb-0\" (UID: \"98f722f9-fa25-40b6-84ba-96127d07293c\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.614930 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/50f5af82-8ee5-45f4-b158-2166f1be6cb8-scripts\") pod \"ovsdbserver-sb-0\" (UID: \"50f5af82-8ee5-45f4-b158-2166f1be6cb8\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.615015 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"memcached-tls-certs\" (UniqueName: \"kubernetes.io/secret/19fe6517-a9bd-4463-b2b1-e006bd5d8145-memcached-tls-certs\") pod \"memcached-0\" (UID: \"19fe6517-a9bd-4463-b2b1-e006bd5d8145\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.615102 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovsdbserver-sb-tls-certs\" (UniqueName: \"kubernetes.io/secret/50f5af82-8ee5-45f4-b158-2166f1be6cb8-ovsdbserver-sb-tls-certs\") pod \"ovsdbserver-sb-0\" (UID: \"50f5af82-8ee5-45f4-b158-2166f1be6cb8\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.615179 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/98f722f9-fa25-40b6-84ba-96127d07293c-config\") pod \"ovsdbserver-nb-0\" (UID: \"98f722f9-fa25-40b6-84ba-96127d07293c\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.615223 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-4jrvq\" (UniqueName: \"kubernetes.io/projected/98f722f9-fa25-40b6-84ba-96127d07293c-kube-api-access-4jrvq\") pod \"ovsdbserver-nb-0\" (UID: \"98f722f9-fa25-40b6-84ba-96127d07293c\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.615283 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/98f722f9-fa25-40b6-84ba-96127d07293c-metrics-certs-tls-certs\") pod \"ovsdbserver-nb-0\" (UID: \"98f722f9-fa25-40b6-84ba-96127d07293c\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.615314 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-dp9m4\" (UniqueName: \"kubernetes.io/projected/50f5af82-8ee5-45f4-b158-2166f1be6cb8-kube-api-access-dp9m4\") pod \"ovsdbserver-sb-0\" (UID: \"50f5af82-8ee5-45f4-b158-2166f1be6cb8\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.615391 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/98f722f9-fa25-40b6-84ba-96127d07293c-scripts\") pod \"ovsdbserver-nb-0\" (UID: \"98f722f9-fa25-40b6-84ba-96127d07293c\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.615415 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/50f5af82-8ee5-45f4-b158-2166f1be6cb8-config\") pod \"ovsdbserver-sb-0\" (UID: \"50f5af82-8ee5-45f4-b158-2166f1be6cb8\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.615492 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gcvg4\" (UniqueName: \"kubernetes.io/projected/19fe6517-a9bd-4463-b2b1-e006bd5d8145-kube-api-access-gcvg4\") pod \"memcached-0\" (UID: \"19fe6517-a9bd-4463-b2b1-e006bd5d8145\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.615622 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/50f5af82-8ee5-45f4-b158-2166f1be6cb8-combined-ca-bundle\") pod \"ovsdbserver-sb-0\" (UID: \"50f5af82-8ee5-45f4-b158-2166f1be6cb8\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.615664 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/19fe6517-a9bd-4463-b2b1-e006bd5d8145-combined-ca-bundle\") pod \"memcached-0\" (UID: \"19fe6517-a9bd-4463-b2b1-e006bd5d8145\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.615693 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovsdbserver-nb-tls-certs\" (UniqueName: \"kubernetes.io/secret/98f722f9-fa25-40b6-84ba-96127d07293c-ovsdbserver-nb-tls-certs\") pod \"ovsdbserver-nb-0\" (UID: \"98f722f9-fa25-40b6-84ba-96127d07293c\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.616099 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/50f5af82-8ee5-45f4-b158-2166f1be6cb8-metrics-certs-tls-certs\") pod \"ovsdbserver-sb-0\" (UID: \"50f5af82-8ee5-45f4-b158-2166f1be6cb8\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.616132 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage03-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage03-crc\") pod \"ovsdbserver-nb-0\" (UID: \"98f722f9-fa25-40b6-84ba-96127d07293c\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.616156 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/50f5af82-8ee5-45f4-b158-2166f1be6cb8-ovsdb-rundir\") pod \"ovsdbserver-sb-0\" (UID: \"50f5af82-8ee5-45f4-b158-2166f1be6cb8\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.616194 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage08-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage08-crc\") pod \"ovsdbserver-sb-0\" (UID: \"50f5af82-8ee5-45f4-b158-2166f1be6cb8\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.616613 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage08-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage08-crc\") pod \"ovsdbserver-sb-0\" (UID: \"50f5af82-8ee5-45f4-b158-2166f1be6cb8\") device mount path \"/mnt/openstack/pv08\"" pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.620193 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage03-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage03-crc\") pod \"ovsdbserver-nb-0\" (UID: \"98f722f9-fa25-40b6-84ba-96127d07293c\") device mount path \"/mnt/openstack/pv03\"" pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.620364 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/98f722f9-fa25-40b6-84ba-96127d07293c-config\") pod \"ovsdbserver-nb-0\" (UID: \"98f722f9-fa25-40b6-84ba-96127d07293c\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.621594 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/98f722f9-fa25-40b6-84ba-96127d07293c-scripts\") pod \"ovsdbserver-nb-0\" (UID: \"98f722f9-fa25-40b6-84ba-96127d07293c\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.622000 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovsdbserver-sb-tls-certs\" (UniqueName: \"kubernetes.io/secret/50f5af82-8ee5-45f4-b158-2166f1be6cb8-ovsdbserver-sb-tls-certs\") pod \"ovsdbserver-sb-0\" (UID: \"50f5af82-8ee5-45f4-b158-2166f1be6cb8\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.622610 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/50f5af82-8ee5-45f4-b158-2166f1be6cb8-config\") pod \"ovsdbserver-sb-0\" (UID: \"50f5af82-8ee5-45f4-b158-2166f1be6cb8\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.622974 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/98f722f9-fa25-40b6-84ba-96127d07293c-ovsdb-rundir\") pod \"ovsdbserver-nb-0\" (UID: \"98f722f9-fa25-40b6-84ba-96127d07293c\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.623075 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/50f5af82-8ee5-45f4-b158-2166f1be6cb8-ovsdb-rundir\") pod \"ovsdbserver-sb-0\" (UID: \"50f5af82-8ee5-45f4-b158-2166f1be6cb8\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.625844 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/50f5af82-8ee5-45f4-b158-2166f1be6cb8-scripts\") pod \"ovsdbserver-sb-0\" (UID: \"50f5af82-8ee5-45f4-b158-2166f1be6cb8\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.628689 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/98f722f9-fa25-40b6-84ba-96127d07293c-metrics-certs-tls-certs\") pod \"ovsdbserver-nb-0\" (UID: \"98f722f9-fa25-40b6-84ba-96127d07293c\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.637929 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovsdbserver-nb-tls-certs\" (UniqueName: \"kubernetes.io/secret/98f722f9-fa25-40b6-84ba-96127d07293c-ovsdbserver-nb-tls-certs\") pod \"ovsdbserver-nb-0\" (UID: \"98f722f9-fa25-40b6-84ba-96127d07293c\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.643531 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/50f5af82-8ee5-45f4-b158-2166f1be6cb8-metrics-certs-tls-certs\") pod \"ovsdbserver-sb-0\" (UID: \"50f5af82-8ee5-45f4-b158-2166f1be6cb8\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.645146 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-dp9m4\" (UniqueName: \"kubernetes.io/projected/50f5af82-8ee5-45f4-b158-2166f1be6cb8-kube-api-access-dp9m4\") pod \"ovsdbserver-sb-0\" (UID: \"50f5af82-8ee5-45f4-b158-2166f1be6cb8\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.645341 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/50f5af82-8ee5-45f4-b158-2166f1be6cb8-combined-ca-bundle\") pod \"ovsdbserver-sb-0\" (UID: \"50f5af82-8ee5-45f4-b158-2166f1be6cb8\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.646063 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/98f722f9-fa25-40b6-84ba-96127d07293c-combined-ca-bundle\") pod \"ovsdbserver-nb-0\" (UID: \"98f722f9-fa25-40b6-84ba-96127d07293c\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.647357 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-4jrvq\" (UniqueName: \"kubernetes.io/projected/98f722f9-fa25-40b6-84ba-96127d07293c-kube-api-access-4jrvq\") pod \"ovsdbserver-nb-0\" (UID: \"98f722f9-fa25-40b6-84ba-96127d07293c\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.669862 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage08-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage08-crc\") pod \"ovsdbserver-sb-0\" (UID: \"50f5af82-8ee5-45f4-b158-2166f1be6cb8\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.711236 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.718731 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/19fe6517-a9bd-4463-b2b1-e006bd5d8145-config-data\") pod \"memcached-0\" (UID: \"19fe6517-a9bd-4463-b2b1-e006bd5d8145\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.718796 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/19fe6517-a9bd-4463-b2b1-e006bd5d8145-kolla-config\") pod \"memcached-0\" (UID: \"19fe6517-a9bd-4463-b2b1-e006bd5d8145\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.718842 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"memcached-tls-certs\" (UniqueName: \"kubernetes.io/secret/19fe6517-a9bd-4463-b2b1-e006bd5d8145-memcached-tls-certs\") pod \"memcached-0\" (UID: \"19fe6517-a9bd-4463-b2b1-e006bd5d8145\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.718982 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-gcvg4\" (UniqueName: \"kubernetes.io/projected/19fe6517-a9bd-4463-b2b1-e006bd5d8145-kube-api-access-gcvg4\") pod \"memcached-0\" (UID: \"19fe6517-a9bd-4463-b2b1-e006bd5d8145\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.719027 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/19fe6517-a9bd-4463-b2b1-e006bd5d8145-combined-ca-bundle\") pod \"memcached-0\" (UID: \"19fe6517-a9bd-4463-b2b1-e006bd5d8145\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.720222 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/19fe6517-a9bd-4463-b2b1-e006bd5d8145-kolla-config\") pod \"memcached-0\" (UID: \"19fe6517-a9bd-4463-b2b1-e006bd5d8145\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.721184 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/19fe6517-a9bd-4463-b2b1-e006bd5d8145-config-data\") pod \"memcached-0\" (UID: \"19fe6517-a9bd-4463-b2b1-e006bd5d8145\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.723979 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"memcached-tls-certs\" (UniqueName: \"kubernetes.io/secret/19fe6517-a9bd-4463-b2b1-e006bd5d8145-memcached-tls-certs\") pod \"memcached-0\" (UID: \"19fe6517-a9bd-4463-b2b1-e006bd5d8145\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.727651 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/19fe6517-a9bd-4463-b2b1-e006bd5d8145-combined-ca-bundle\") pod \"memcached-0\" (UID: \"19fe6517-a9bd-4463-b2b1-e006bd5d8145\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.782769 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-gcvg4\" (UniqueName: \"kubernetes.io/projected/19fe6517-a9bd-4463-b2b1-e006bd5d8145-kube-api-access-gcvg4\") pod \"memcached-0\" (UID: \"19fe6517-a9bd-4463-b2b1-e006bd5d8145\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.786589 5052 generic.go:334] "Generic (PLEG): container finished" podID="b5b5ad16-1e45-4761-b5d2-60ffd5934ccb" containerID="ca424ba734d34bcc97da61c506acc5ac518a281efdcc25fe387f6db662517ec3" exitCode=0 Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.787679 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage03-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage03-crc\") pod \"ovsdbserver-nb-0\" (UID: \"98f722f9-fa25-40b6-84ba-96127d07293c\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.812583 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="53193ff3-1a41-4e26-b38f-957088aa7e8b" path="/var/lib/kubelet/pods/53193ff3-1a41-4e26-b38f-957088aa7e8b/volumes" Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.814784 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5c7fe95d-f637-4021-a5e2-69668046d2ff" path="/var/lib/kubelet/pods/5c7fe95d-f637-4021-a5e2-69668046d2ff/volumes" Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.817474 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/nova-api-0" podUID="b5b5ad16-1e45-4761-b5d2-60ffd5934ccb" containerName="nova-api-api" probeResult="failure" output="Get \"https://10.217.0.188:8774/\": dial tcp 10.217.0.188:8774: connect: connection refused" Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.820724 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="78630d10-b9ba-438d-907d-2be706fc51e7" path="/var/lib/kubelet/pods/78630d10-b9ba-438d-907d-2be706fc51e7/volumes" Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.821980 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b35e6939-8f23-496c-ac55-9534be4ea8be" path="/var/lib/kubelet/pods/b35e6939-8f23-496c-ac55-9534be4ea8be/volumes" Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.823432 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f45e9eda-e62f-4307-ae18-2a8da7d6fe7c" path="/var/lib/kubelet/pods/f45e9eda-e62f-4307-ae18-2a8da7d6fe7c/volumes" Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.831422 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-worker-644558fbc5-4gtxj" event={"ID":"8ac6e8f0-d49c-4a4c-846a-6cec226bd360","Type":"ContainerStarted","Data":"4546b826c99134ff60d23f10a99bb12e6e7977932e01290b96ec7338744adb66"} Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.831465 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-564bc88b6-bvll5" event={"ID":"c0ce9ced-33b0-4879-81a4-b5c066ec0631","Type":"ContainerStarted","Data":"7d60e6875dab88bfe727e3ecc5b73bec4dc81727c206ef38a830168d386edffa"} Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.831479 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"b5b5ad16-1e45-4761-b5d2-60ffd5934ccb","Type":"ContainerDied","Data":"ca424ba734d34bcc97da61c506acc5ac518a281efdcc25fe387f6db662517ec3"} Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.831496 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-ff6b8c44b-7ndgm" event={"ID":"48831cad-910f-46f1-aefe-7cc4cbfa984f","Type":"ContainerStarted","Data":"1f06f16f71b99e617383aa251524a1b2334b34f46ead54b868e09f13e83c19cc"} Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.831635 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-worker-6d687d9c57-4gplx" event={"ID":"a65652a1-62fa-40ff-8b8a-940b2cc795b7","Type":"ContainerStarted","Data":"09f8bee6ba47ae352b63fc6a9643dadd544c69e4d30aec7017b6e2983d7db541"} Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.836725 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/nova-api-0" podUID="b5b5ad16-1e45-4761-b5d2-60ffd5934ccb" containerName="nova-api-log" probeResult="failure" output="Get \"https://10.217.0.188:8774/\": dial tcp 10.217.0.188:8774: connect: connection refused" Dec 03 12:29:59 crc kubenswrapper[5052]: E1203 12:29:59.841980 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="93474533ee16033979340cc7914412b7661d56c4b5e3b0058db42692da5f5166" cmd=["/usr/local/bin/container-scripts/status_check.sh"] Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.866874 5052 scope.go:117] "RemoveContainer" containerID="8d5e1c2473a3c27b159232ab3f630f8e1ca663780d8fc9b7b1eaac3611c97961" Dec 03 12:29:59 crc kubenswrapper[5052]: E1203 12:29:59.894275 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="93474533ee16033979340cc7914412b7661d56c4b5e3b0058db42692da5f5166" cmd=["/usr/local/bin/container-scripts/status_check.sh"] Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.895826 5052 generic.go:334] "Generic (PLEG): container finished" podID="e75eea2c-2eb1-4f0f-97e5-63ce147e68dd" containerID="a6f27ec51db1c9719c74272ae88d2ed48068153f9cc4d46a92cce3985e31c2e5" exitCode=0 Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.895896 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"e75eea2c-2eb1-4f0f-97e5-63ce147e68dd","Type":"ContainerDied","Data":"a6f27ec51db1c9719c74272ae88d2ed48068153f9cc4d46a92cce3985e31c2e5"} Dec 03 12:29:59 crc kubenswrapper[5052]: E1203 12:29:59.897028 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="93474533ee16033979340cc7914412b7661d56c4b5e3b0058db42692da5f5166" cmd=["/usr/local/bin/container-scripts/status_check.sh"] Dec 03 12:29:59 crc kubenswrapper[5052]: E1203 12:29:59.897076 5052 prober.go:104] "Probe errored" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" probeType="Readiness" pod="openstack-kuttl-tests/ovn-northd-0" podUID="a26ed42f-71e7-4c92-8b7e-ad194358d3e5" containerName="ovn-northd" Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.899558 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-7dffdbf8-hzmbm" event={"ID":"53fe6403-27db-44b7-a9a6-787afbc3bbb8","Type":"ContainerStarted","Data":"9f3746f193daa58337fe694f77011f189858dade68192aac0e70fb07c702c05a"} Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.902054 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-keystone-listener-77b6bbbdcb-4jlsc" event={"ID":"aed41b09-319e-466c-9eb9-38208e1880a3","Type":"ContainerStarted","Data":"c4632c9d758da0d71f63ae0b7ef4c62003466443e6cb1e897bae7412a5ae794f"} Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.904647 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-56f9c7c6b6-kvxpr" event={"ID":"5f522d90-036e-4565-8293-97097912fbb1","Type":"ContainerStarted","Data":"1dda49dafe50f2de1258c72c294640965be08d91eee18557be9600402edf867a"} Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.912965 5052 generic.go:334] "Generic (PLEG): container finished" podID="caaecb05-f5f1-421d-8e91-818774a2c0ec" containerID="e2e4d22232c587b84e25f9aceb08f0b3c39f38badcaa667d82792f3314b6a1b9" exitCode=0 Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.913091 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"caaecb05-f5f1-421d-8e91-818774a2c0ec","Type":"ContainerDied","Data":"e2e4d22232c587b84e25f9aceb08f0b3c39f38badcaa667d82792f3314b6a1b9"} Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.916695 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-keystone-listener-cd5b58cfd-jf8vt" event={"ID":"019689d4-d40b-4e9a-a028-e427f1aa9d9b","Type":"ContainerStarted","Data":"1388cc58677cb99a408eeb9662787e5c6d0865f5f7b0080c83b290fe93da2278"} Dec 03 12:29:59 crc kubenswrapper[5052]: I1203 12:29:59.958887 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:30:00 crc kubenswrapper[5052]: I1203 12:30:00.080059 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/memcached-0" Dec 03 12:30:00 crc kubenswrapper[5052]: E1203 12:30:00.090961 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="46246c5deed2c10d0bc81011e4be47e5c70c228f4dda6fb9a1b20810aff514c7" cmd=["/bin/bash","/var/lib/operator-scripts/mysql_probe.sh","readiness"] Dec 03 12:30:00 crc kubenswrapper[5052]: E1203 12:30:00.093917 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="46246c5deed2c10d0bc81011e4be47e5c70c228f4dda6fb9a1b20810aff514c7" cmd=["/bin/bash","/var/lib/operator-scripts/mysql_probe.sh","readiness"] Dec 03 12:30:00 crc kubenswrapper[5052]: E1203 12:30:00.112165 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="46246c5deed2c10d0bc81011e4be47e5c70c228f4dda6fb9a1b20810aff514c7" cmd=["/bin/bash","/var/lib/operator-scripts/mysql_probe.sh","readiness"] Dec 03 12:30:00 crc kubenswrapper[5052]: E1203 12:30:00.112233 5052 prober.go:104] "Probe errored" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" probeType="Readiness" pod="openstack-kuttl-tests/openstack-cell1-galera-0" podUID="1de1d25b-cbcc-4d08-a7cd-aa54ac71aecb" containerName="galera" Dec 03 12:30:00 crc kubenswrapper[5052]: I1203 12:30:00.127354 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 12:30:00 crc kubenswrapper[5052]: W1203 12:30:00.154145 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf4e095cd_36d8_4136_b262_f6d52717a05c.slice/crio-1e8fea595fbe0a6787690b161ee2b73ba9ae2e861acdb4233b860c488db1f389 WatchSource:0}: Error finding container 1e8fea595fbe0a6787690b161ee2b73ba9ae2e861acdb4233b860c488db1f389: Status 404 returned error can't find the container with id 1e8fea595fbe0a6787690b161ee2b73ba9ae2e861acdb4233b860c488db1f389 Dec 03 12:30:00 crc kubenswrapper[5052]: I1203 12:30:00.173823 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openshift-operator-lifecycle-manager/collect-profiles-29412750-42nq6"] Dec 03 12:30:00 crc kubenswrapper[5052]: I1203 12:30:00.178301 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-operator-lifecycle-manager/collect-profiles-29412750-42nq6" Dec 03 12:30:00 crc kubenswrapper[5052]: I1203 12:30:00.180725 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-operator-lifecycle-manager"/"collect-profiles-config" Dec 03 12:30:00 crc kubenswrapper[5052]: I1203 12:30:00.181206 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-operator-lifecycle-manager"/"collect-profiles-dockercfg-kzf4t" Dec 03 12:30:00 crc kubenswrapper[5052]: I1203 12:30:00.194817 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-operator-lifecycle-manager/collect-profiles-29412750-42nq6"] Dec 03 12:30:00 crc kubenswrapper[5052]: I1203 12:30:00.286186 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-sb-0"] Dec 03 12:30:00 crc kubenswrapper[5052]: I1203 12:30:00.338810 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/6a77ebe2-ac70-4620-acef-bda957db5766-config-volume\") pod \"collect-profiles-29412750-42nq6\" (UID: \"6a77ebe2-ac70-4620-acef-bda957db5766\") " pod="openshift-operator-lifecycle-manager/collect-profiles-29412750-42nq6" Dec 03 12:30:00 crc kubenswrapper[5052]: I1203 12:30:00.338868 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jhnj6\" (UniqueName: \"kubernetes.io/projected/6a77ebe2-ac70-4620-acef-bda957db5766-kube-api-access-jhnj6\") pod \"collect-profiles-29412750-42nq6\" (UID: \"6a77ebe2-ac70-4620-acef-bda957db5766\") " pod="openshift-operator-lifecycle-manager/collect-profiles-29412750-42nq6" Dec 03 12:30:00 crc kubenswrapper[5052]: I1203 12:30:00.339115 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-volume\" (UniqueName: \"kubernetes.io/secret/6a77ebe2-ac70-4620-acef-bda957db5766-secret-volume\") pod \"collect-profiles-29412750-42nq6\" (UID: \"6a77ebe2-ac70-4620-acef-bda957db5766\") " pod="openshift-operator-lifecycle-manager/collect-profiles-29412750-42nq6" Dec 03 12:30:00 crc kubenswrapper[5052]: I1203 12:30:00.441724 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/6a77ebe2-ac70-4620-acef-bda957db5766-config-volume\") pod \"collect-profiles-29412750-42nq6\" (UID: \"6a77ebe2-ac70-4620-acef-bda957db5766\") " pod="openshift-operator-lifecycle-manager/collect-profiles-29412750-42nq6" Dec 03 12:30:00 crc kubenswrapper[5052]: I1203 12:30:00.441792 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-jhnj6\" (UniqueName: \"kubernetes.io/projected/6a77ebe2-ac70-4620-acef-bda957db5766-kube-api-access-jhnj6\") pod \"collect-profiles-29412750-42nq6\" (UID: \"6a77ebe2-ac70-4620-acef-bda957db5766\") " pod="openshift-operator-lifecycle-manager/collect-profiles-29412750-42nq6" Dec 03 12:30:00 crc kubenswrapper[5052]: I1203 12:30:00.441861 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"secret-volume\" (UniqueName: \"kubernetes.io/secret/6a77ebe2-ac70-4620-acef-bda957db5766-secret-volume\") pod \"collect-profiles-29412750-42nq6\" (UID: \"6a77ebe2-ac70-4620-acef-bda957db5766\") " pod="openshift-operator-lifecycle-manager/collect-profiles-29412750-42nq6" Dec 03 12:30:00 crc kubenswrapper[5052]: I1203 12:30:00.443228 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/6a77ebe2-ac70-4620-acef-bda957db5766-config-volume\") pod \"collect-profiles-29412750-42nq6\" (UID: \"6a77ebe2-ac70-4620-acef-bda957db5766\") " pod="openshift-operator-lifecycle-manager/collect-profiles-29412750-42nq6" Dec 03 12:30:00 crc kubenswrapper[5052]: I1203 12:30:00.448569 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"secret-volume\" (UniqueName: \"kubernetes.io/secret/6a77ebe2-ac70-4620-acef-bda957db5766-secret-volume\") pod \"collect-profiles-29412750-42nq6\" (UID: \"6a77ebe2-ac70-4620-acef-bda957db5766\") " pod="openshift-operator-lifecycle-manager/collect-profiles-29412750-42nq6" Dec 03 12:30:00 crc kubenswrapper[5052]: I1203 12:30:00.462475 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-jhnj6\" (UniqueName: \"kubernetes.io/projected/6a77ebe2-ac70-4620-acef-bda957db5766-kube-api-access-jhnj6\") pod \"collect-profiles-29412750-42nq6\" (UID: \"6a77ebe2-ac70-4620-acef-bda957db5766\") " pod="openshift-operator-lifecycle-manager/collect-profiles-29412750-42nq6" Dec 03 12:30:00 crc kubenswrapper[5052]: I1203 12:30:00.500385 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/nova-metadata-0" podUID="caaecb05-f5f1-421d-8e91-818774a2c0ec" containerName="nova-metadata-log" probeResult="failure" output="Get \"https://10.217.0.189:8775/\": dial tcp 10.217.0.189:8775: connect: connection refused" Dec 03 12:30:00 crc kubenswrapper[5052]: I1203 12:30:00.500542 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/nova-metadata-0" podUID="caaecb05-f5f1-421d-8e91-818774a2c0ec" containerName="nova-metadata-metadata" probeResult="failure" output="Get \"https://10.217.0.189:8775/\": dial tcp 10.217.0.189:8775: connect: connection refused" Dec 03 12:30:00 crc kubenswrapper[5052]: I1203 12:30:00.516682 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-nb-0"] Dec 03 12:30:00 crc kubenswrapper[5052]: W1203 12:30:00.525341 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod98f722f9_fa25_40b6_84ba_96127d07293c.slice/crio-b6c799fc84b3d867e28e78a12cece87252f80013fd30584cea813d72ca3deda4 WatchSource:0}: Error finding container b6c799fc84b3d867e28e78a12cece87252f80013fd30584cea813d72ca3deda4: Status 404 returned error can't find the container with id b6c799fc84b3d867e28e78a12cece87252f80013fd30584cea813d72ca3deda4 Dec 03 12:30:00 crc kubenswrapper[5052]: I1203 12:30:00.556492 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-operator-lifecycle-manager/collect-profiles-29412750-42nq6" Dec 03 12:30:00 crc kubenswrapper[5052]: I1203 12:30:00.637707 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/memcached-0"] Dec 03 12:30:00 crc kubenswrapper[5052]: I1203 12:30:00.940361 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/memcached-0" event={"ID":"19fe6517-a9bd-4463-b2b1-e006bd5d8145","Type":"ContainerStarted","Data":"c05d4bc250127b98e1df0c953be791f5ea8cd76589505431daa66756603ad4be"} Dec 03 12:30:00 crc kubenswrapper[5052]: I1203 12:30:00.942982 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" event={"ID":"f4e095cd-36d8-4136-b262-f6d52717a05c","Type":"ContainerStarted","Data":"1e8fea595fbe0a6787690b161ee2b73ba9ae2e861acdb4233b860c488db1f389"} Dec 03 12:30:00 crc kubenswrapper[5052]: I1203 12:30:00.944670 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-nb-0" event={"ID":"98f722f9-fa25-40b6-84ba-96127d07293c","Type":"ContainerStarted","Data":"b6c799fc84b3d867e28e78a12cece87252f80013fd30584cea813d72ca3deda4"} Dec 03 12:30:00 crc kubenswrapper[5052]: I1203 12:30:00.946784 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"caaecb05-f5f1-421d-8e91-818774a2c0ec","Type":"ContainerDied","Data":"d5c2864b66fc824342017a2d07a8d11e2ef48d803062db360c43d8cd43d15324"} Dec 03 12:30:00 crc kubenswrapper[5052]: I1203 12:30:00.946823 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d5c2864b66fc824342017a2d07a8d11e2ef48d803062db360c43d8cd43d15324" Dec 03 12:30:00 crc kubenswrapper[5052]: I1203 12:30:00.950213 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-7cbf547d94-9krth" event={"ID":"6927052c-212d-42fc-8236-b053ffc9a9d9","Type":"ContainerStarted","Data":"edaf8cd0d0bbbd5a49f586c7c2c1e7bcdb11f76c906dcb6bcd4c10118fa5950b"} Dec 03 12:30:00 crc kubenswrapper[5052]: I1203 12:30:00.954242 5052 generic.go:334] "Generic (PLEG): container finished" podID="61cad274-57bd-4993-aaa6-3e6013900d88" containerID="d1ad86af122d9acbc9e7b5bb95477effe197a4939198ba933eb8fb9e83c0f569" exitCode=0 Dec 03 12:30:00 crc kubenswrapper[5052]: I1203 12:30:00.954342 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"61cad274-57bd-4993-aaa6-3e6013900d88","Type":"ContainerDied","Data":"d1ad86af122d9acbc9e7b5bb95477effe197a4939198ba933eb8fb9e83c0f569"} Dec 03 12:30:00 crc kubenswrapper[5052]: I1203 12:30:00.956027 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-sb-0" event={"ID":"50f5af82-8ee5-45f4-b158-2166f1be6cb8","Type":"ContainerStarted","Data":"a34cefb9fab15586d1ef541f04657dfba9f32d8dac3286264dba4943680b0e47"} Dec 03 12:30:01 crc kubenswrapper[5052]: I1203 12:30:01.105058 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:30:01 crc kubenswrapper[5052]: I1203 12:30:01.187084 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-operator-lifecycle-manager/collect-profiles-29412750-42nq6"] Dec 03 12:30:01 crc kubenswrapper[5052]: W1203 12:30:01.201713 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a77ebe2_ac70_4620_acef_bda957db5766.slice/crio-eb228dee2e6cdc91772f817a0a1e0dd61bffbdf799bae9cf1b3f9e10d968e07e WatchSource:0}: Error finding container eb228dee2e6cdc91772f817a0a1e0dd61bffbdf799bae9cf1b3f9e10d968e07e: Status 404 returned error can't find the container with id eb228dee2e6cdc91772f817a0a1e0dd61bffbdf799bae9cf1b3f9e10d968e07e Dec 03 12:30:01 crc kubenswrapper[5052]: I1203 12:30:01.257610 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/caaecb05-f5f1-421d-8e91-818774a2c0ec-config-data\") pod \"caaecb05-f5f1-421d-8e91-818774a2c0ec\" (UID: \"caaecb05-f5f1-421d-8e91-818774a2c0ec\") " Dec 03 12:30:01 crc kubenswrapper[5052]: I1203 12:30:01.257853 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/caaecb05-f5f1-421d-8e91-818774a2c0ec-logs\") pod \"caaecb05-f5f1-421d-8e91-818774a2c0ec\" (UID: \"caaecb05-f5f1-421d-8e91-818774a2c0ec\") " Dec 03 12:30:01 crc kubenswrapper[5052]: I1203 12:30:01.257963 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/caaecb05-f5f1-421d-8e91-818774a2c0ec-combined-ca-bundle\") pod \"caaecb05-f5f1-421d-8e91-818774a2c0ec\" (UID: \"caaecb05-f5f1-421d-8e91-818774a2c0ec\") " Dec 03 12:30:01 crc kubenswrapper[5052]: I1203 12:30:01.258026 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/caaecb05-f5f1-421d-8e91-818774a2c0ec-nova-metadata-tls-certs\") pod \"caaecb05-f5f1-421d-8e91-818774a2c0ec\" (UID: \"caaecb05-f5f1-421d-8e91-818774a2c0ec\") " Dec 03 12:30:01 crc kubenswrapper[5052]: I1203 12:30:01.258139 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4ldbv\" (UniqueName: \"kubernetes.io/projected/caaecb05-f5f1-421d-8e91-818774a2c0ec-kube-api-access-4ldbv\") pod \"caaecb05-f5f1-421d-8e91-818774a2c0ec\" (UID: \"caaecb05-f5f1-421d-8e91-818774a2c0ec\") " Dec 03 12:30:01 crc kubenswrapper[5052]: I1203 12:30:01.258916 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/caaecb05-f5f1-421d-8e91-818774a2c0ec-logs" (OuterVolumeSpecName: "logs") pod "caaecb05-f5f1-421d-8e91-818774a2c0ec" (UID: "caaecb05-f5f1-421d-8e91-818774a2c0ec"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:30:01 crc kubenswrapper[5052]: I1203 12:30:01.259248 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/caaecb05-f5f1-421d-8e91-818774a2c0ec-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:01 crc kubenswrapper[5052]: I1203 12:30:01.263561 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/caaecb05-f5f1-421d-8e91-818774a2c0ec-kube-api-access-4ldbv" (OuterVolumeSpecName: "kube-api-access-4ldbv") pod "caaecb05-f5f1-421d-8e91-818774a2c0ec" (UID: "caaecb05-f5f1-421d-8e91-818774a2c0ec"). InnerVolumeSpecName "kube-api-access-4ldbv". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:30:01 crc kubenswrapper[5052]: I1203 12:30:01.298399 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/caaecb05-f5f1-421d-8e91-818774a2c0ec-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "caaecb05-f5f1-421d-8e91-818774a2c0ec" (UID: "caaecb05-f5f1-421d-8e91-818774a2c0ec"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:01 crc kubenswrapper[5052]: I1203 12:30:01.299093 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/caaecb05-f5f1-421d-8e91-818774a2c0ec-config-data" (OuterVolumeSpecName: "config-data") pod "caaecb05-f5f1-421d-8e91-818774a2c0ec" (UID: "caaecb05-f5f1-421d-8e91-818774a2c0ec"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:01 crc kubenswrapper[5052]: I1203 12:30:01.361149 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/caaecb05-f5f1-421d-8e91-818774a2c0ec-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:01 crc kubenswrapper[5052]: I1203 12:30:01.361491 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/caaecb05-f5f1-421d-8e91-818774a2c0ec-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:01 crc kubenswrapper[5052]: I1203 12:30:01.361525 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-4ldbv\" (UniqueName: \"kubernetes.io/projected/caaecb05-f5f1-421d-8e91-818774a2c0ec-kube-api-access-4ldbv\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:01 crc kubenswrapper[5052]: I1203 12:30:01.378146 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/caaecb05-f5f1-421d-8e91-818774a2c0ec-nova-metadata-tls-certs" (OuterVolumeSpecName: "nova-metadata-tls-certs") pod "caaecb05-f5f1-421d-8e91-818774a2c0ec" (UID: "caaecb05-f5f1-421d-8e91-818774a2c0ec"). InnerVolumeSpecName "nova-metadata-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:01 crc kubenswrapper[5052]: I1203 12:30:01.463961 5052 reconciler_common.go:293] "Volume detached for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/caaecb05-f5f1-421d-8e91-818774a2c0ec-nova-metadata-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:01 crc kubenswrapper[5052]: I1203 12:30:01.990943 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/cinder-api-0" podUID="61cad274-57bd-4993-aaa6-3e6013900d88" containerName="cinder-api" probeResult="failure" output="Get \"https://10.217.0.159:8776/healthcheck\": dial tcp 10.217.0.159:8776: connect: connection refused" Dec 03 12:30:01 crc kubenswrapper[5052]: I1203 12:30:01.995616 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" podUID="53193ff3-1a41-4e26-b38f-957088aa7e8b" containerName="nova-cell1-novncproxy-novncproxy" probeResult="failure" output="Get \"https://10.217.0.191:6080/vnc_lite.html\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Dec 03 12:30:01 crc kubenswrapper[5052]: I1203 12:30:01.998943 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-nb-0" event={"ID":"98f722f9-fa25-40b6-84ba-96127d07293c","Type":"ContainerStarted","Data":"b6e565e6b6e8e07c8000bbead3eed4e1cd45529d9dc3aa7fcb642cf668f780aa"} Dec 03 12:30:02 crc kubenswrapper[5052]: I1203 12:30:02.010403 5052 generic.go:334] "Generic (PLEG): container finished" podID="05b9d3cc-93de-45a6-8849-027a4aad3715" containerID="2f2f681ab4e070f1a45adfa2327a6b72d7c87fcccf7bb015c88103e25b6472d9" exitCode=0 Dec 03 12:30:02 crc kubenswrapper[5052]: I1203 12:30:02.010469 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"05b9d3cc-93de-45a6-8849-027a4aad3715","Type":"ContainerDied","Data":"2f2f681ab4e070f1a45adfa2327a6b72d7c87fcccf7bb015c88103e25b6472d9"} Dec 03 12:30:02 crc kubenswrapper[5052]: I1203 12:30:02.046433 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:30:02 crc kubenswrapper[5052]: I1203 12:30:02.047300 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-5fc7894bfd-g7hl5" event={"ID":"cfb8e98d-3873-43e0-81dc-4450983179c2","Type":"ContainerStarted","Data":"27c4bfe7d3a13a820b6065867e281595e076068c529bea4be94b943aa7105f13"} Dec 03 12:30:02 crc kubenswrapper[5052]: I1203 12:30:02.048883 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-db474c7dd-tg7mq" event={"ID":"886ea0a4-5096-4ea8-a74c-de243d1f4f40","Type":"ContainerStarted","Data":"36c7c94da2f3146bb43c39fdf96923c24534dd36b2033a6240c6be99921f9ca4"} Dec 03 12:30:02 crc kubenswrapper[5052]: I1203 12:30:02.051050 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-56f9c7c6b6-kvxpr" event={"ID":"5f522d90-036e-4565-8293-97097912fbb1","Type":"ContainerStarted","Data":"83e0a3b478794ea6778ae35cae6f68d4eef32d5b5841dc00889915eed9b1dfbf"} Dec 03 12:30:02 crc kubenswrapper[5052]: I1203 12:30:02.055529 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/placement-7dffdbf8-hzmbm"] Dec 03 12:30:02 crc kubenswrapper[5052]: I1203 12:30:02.063887 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-worker-644558fbc5-4gtxj" event={"ID":"8ac6e8f0-d49c-4a4c-846a-6cec226bd360","Type":"ContainerStarted","Data":"57de2e7ba65bad4f495dc483cc50efdd66d36654d3d53aadfa56507cc2faada6"} Dec 03 12:30:02 crc kubenswrapper[5052]: I1203 12:30:02.068611 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"b5b5ad16-1e45-4761-b5d2-60ffd5934ccb","Type":"ContainerDied","Data":"b6f8d9e3b9e593880b90758ff3f0063a1575fa66b5df5219ef9bdd6d868ba33a"} Dec 03 12:30:02 crc kubenswrapper[5052]: I1203 12:30:02.068690 5052 scope.go:117] "RemoveContainer" containerID="ca424ba734d34bcc97da61c506acc5ac518a281efdcc25fe387f6db662517ec3" Dec 03 12:30:02 crc kubenswrapper[5052]: I1203 12:30:02.068885 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:30:02 crc kubenswrapper[5052]: I1203 12:30:02.099114 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/placement-688bd8fb5d-pjs4f"] Dec 03 12:30:02 crc kubenswrapper[5052]: E1203 12:30:02.099650 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="b5b5ad16-1e45-4761-b5d2-60ffd5934ccb" containerName="nova-api-log" Dec 03 12:30:02 crc kubenswrapper[5052]: I1203 12:30:02.099668 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="b5b5ad16-1e45-4761-b5d2-60ffd5934ccb" containerName="nova-api-log" Dec 03 12:30:02 crc kubenswrapper[5052]: E1203 12:30:02.099689 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="caaecb05-f5f1-421d-8e91-818774a2c0ec" containerName="nova-metadata-metadata" Dec 03 12:30:02 crc kubenswrapper[5052]: I1203 12:30:02.099695 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="caaecb05-f5f1-421d-8e91-818774a2c0ec" containerName="nova-metadata-metadata" Dec 03 12:30:02 crc kubenswrapper[5052]: E1203 12:30:02.099707 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="b5b5ad16-1e45-4761-b5d2-60ffd5934ccb" containerName="nova-api-api" Dec 03 12:30:02 crc kubenswrapper[5052]: I1203 12:30:02.099714 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="b5b5ad16-1e45-4761-b5d2-60ffd5934ccb" containerName="nova-api-api" Dec 03 12:30:02 crc kubenswrapper[5052]: E1203 12:30:02.099724 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="caaecb05-f5f1-421d-8e91-818774a2c0ec" containerName="nova-metadata-log" Dec 03 12:30:02 crc kubenswrapper[5052]: I1203 12:30:02.099732 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="caaecb05-f5f1-421d-8e91-818774a2c0ec" containerName="nova-metadata-log" Dec 03 12:30:02 crc kubenswrapper[5052]: I1203 12:30:02.099911 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="b5b5ad16-1e45-4761-b5d2-60ffd5934ccb" containerName="nova-api-api" Dec 03 12:30:02 crc kubenswrapper[5052]: I1203 12:30:02.099924 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="b5b5ad16-1e45-4761-b5d2-60ffd5934ccb" containerName="nova-api-log" Dec 03 12:30:02 crc kubenswrapper[5052]: I1203 12:30:02.099951 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="caaecb05-f5f1-421d-8e91-818774a2c0ec" containerName="nova-metadata-log" Dec 03 12:30:02 crc kubenswrapper[5052]: I1203 12:30:02.099962 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="caaecb05-f5f1-421d-8e91-818774a2c0ec" containerName="nova-metadata-metadata" Dec 03 12:30:02 crc kubenswrapper[5052]: I1203 12:30:02.101077 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-688bd8fb5d-pjs4f" Dec 03 12:30:02 crc kubenswrapper[5052]: I1203 12:30:02.108722 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-keystone-listener-cd5b58cfd-jf8vt" event={"ID":"019689d4-d40b-4e9a-a028-e427f1aa9d9b","Type":"ContainerStarted","Data":"30570e4c9e23e90b86c1701d30a06063dcb7b48a31496a770e386fcaa5d3ac42"} Dec 03 12:30:02 crc kubenswrapper[5052]: E1203 12:30:02.115045 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="6f35ce14790c9ffb5796ad1467515b2be517ae60eb2573a1571ba5f99f630cad" cmd=["/usr/bin/pgrep","-r","DRST","nova-scheduler"] Dec 03 12:30:02 crc kubenswrapper[5052]: E1203 12:30:02.143744 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="6f35ce14790c9ffb5796ad1467515b2be517ae60eb2573a1571ba5f99f630cad" cmd=["/usr/bin/pgrep","-r","DRST","nova-scheduler"] Dec 03 12:30:02 crc kubenswrapper[5052]: I1203 12:30:02.143954 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-ff6b8c44b-7ndgm" event={"ID":"48831cad-910f-46f1-aefe-7cc4cbfa984f","Type":"ContainerStarted","Data":"b447fe4d34e0569d637c38fcee6f843a5b3a8e43d2bd2e661971752993efd16e"} Dec 03 12:30:02 crc kubenswrapper[5052]: E1203 12:30:02.143953 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="fc5e4fca262b35b92bb410a508d650dc4bdca0fc0848d2a90abb1ede218d5a33" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:30:02 crc kubenswrapper[5052]: E1203 12:30:02.158602 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="fc5e4fca262b35b92bb410a508d650dc4bdca0fc0848d2a90abb1ede218d5a33" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:30:02 crc kubenswrapper[5052]: I1203 12:30:02.158820 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/placement-688bd8fb5d-pjs4f"] Dec 03 12:30:02 crc kubenswrapper[5052]: I1203 12:30:02.176691 5052 scope.go:117] "RemoveContainer" containerID="0e4e8a3b2fb3bbe0432215ed54bf039b98ca0a274e59dc10062d81386889219c" Dec 03 12:30:02 crc kubenswrapper[5052]: E1203 12:30:02.181683 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="6f35ce14790c9ffb5796ad1467515b2be517ae60eb2573a1571ba5f99f630cad" cmd=["/usr/bin/pgrep","-r","DRST","nova-scheduler"] Dec 03 12:30:02 crc kubenswrapper[5052]: E1203 12:30:02.181752 5052 prober.go:104] "Probe errored" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" probeType="Readiness" pod="openstack-kuttl-tests/nova-scheduler-0" podUID="b953ffa0-088c-4450-8d60-04044b33cfa0" containerName="nova-scheduler-scheduler" Dec 03 12:30:02 crc kubenswrapper[5052]: E1203 12:30:02.183328 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="fc5e4fca262b35b92bb410a508d650dc4bdca0fc0848d2a90abb1ede218d5a33" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:30:02 crc kubenswrapper[5052]: E1203 12:30:02.183393 5052 prober.go:104] "Probe errored" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" probeType="Readiness" pod="openstack-kuttl-tests/nova-cell1-conductor-0" podUID="733f463b-dcf8-4a61-8e4e-4bf1c042d397" containerName="nova-cell1-conductor-conductor" Dec 03 12:30:02 crc kubenswrapper[5052]: I1203 12:30:02.185164 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-sb-0" event={"ID":"50f5af82-8ee5-45f4-b158-2166f1be6cb8","Type":"ContainerStarted","Data":"e56245b2be2e929328f0828e016ea41db39854cc67eb6cfae7d314651f18c3c0"} Dec 03 12:30:02 crc kubenswrapper[5052]: I1203 12:30:02.189980 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/b5b5ad16-1e45-4761-b5d2-60ffd5934ccb-public-tls-certs\") pod \"b5b5ad16-1e45-4761-b5d2-60ffd5934ccb\" (UID: \"b5b5ad16-1e45-4761-b5d2-60ffd5934ccb\") " Dec 03 12:30:02 crc kubenswrapper[5052]: I1203 12:30:02.190472 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p4phv\" (UniqueName: \"kubernetes.io/projected/b5b5ad16-1e45-4761-b5d2-60ffd5934ccb-kube-api-access-p4phv\") pod \"b5b5ad16-1e45-4761-b5d2-60ffd5934ccb\" (UID: \"b5b5ad16-1e45-4761-b5d2-60ffd5934ccb\") " Dec 03 12:30:02 crc kubenswrapper[5052]: I1203 12:30:02.190672 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b5b5ad16-1e45-4761-b5d2-60ffd5934ccb-combined-ca-bundle\") pod \"b5b5ad16-1e45-4761-b5d2-60ffd5934ccb\" (UID: \"b5b5ad16-1e45-4761-b5d2-60ffd5934ccb\") " Dec 03 12:30:02 crc kubenswrapper[5052]: I1203 12:30:02.190737 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/b5b5ad16-1e45-4761-b5d2-60ffd5934ccb-logs\") pod \"b5b5ad16-1e45-4761-b5d2-60ffd5934ccb\" (UID: \"b5b5ad16-1e45-4761-b5d2-60ffd5934ccb\") " Dec 03 12:30:02 crc kubenswrapper[5052]: I1203 12:30:02.190774 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/b5b5ad16-1e45-4761-b5d2-60ffd5934ccb-internal-tls-certs\") pod \"b5b5ad16-1e45-4761-b5d2-60ffd5934ccb\" (UID: \"b5b5ad16-1e45-4761-b5d2-60ffd5934ccb\") " Dec 03 12:30:02 crc kubenswrapper[5052]: I1203 12:30:02.190865 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/b5b5ad16-1e45-4761-b5d2-60ffd5934ccb-config-data\") pod \"b5b5ad16-1e45-4761-b5d2-60ffd5934ccb\" (UID: \"b5b5ad16-1e45-4761-b5d2-60ffd5934ccb\") " Dec 03 12:30:02 crc kubenswrapper[5052]: I1203 12:30:02.191264 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/a0e45e1a-ced9-4958-9a75-24a69280f679-scripts\") pod \"placement-688bd8fb5d-pjs4f\" (UID: \"a0e45e1a-ced9-4958-9a75-24a69280f679\") " pod="openstack-kuttl-tests/placement-688bd8fb5d-pjs4f" Dec 03 12:30:02 crc kubenswrapper[5052]: I1203 12:30:02.191377 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/a0e45e1a-ced9-4958-9a75-24a69280f679-internal-tls-certs\") pod \"placement-688bd8fb5d-pjs4f\" (UID: \"a0e45e1a-ced9-4958-9a75-24a69280f679\") " pod="openstack-kuttl-tests/placement-688bd8fb5d-pjs4f" Dec 03 12:30:02 crc kubenswrapper[5052]: I1203 12:30:02.191674 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4tb7p\" (UniqueName: \"kubernetes.io/projected/a0e45e1a-ced9-4958-9a75-24a69280f679-kube-api-access-4tb7p\") pod \"placement-688bd8fb5d-pjs4f\" (UID: \"a0e45e1a-ced9-4958-9a75-24a69280f679\") " pod="openstack-kuttl-tests/placement-688bd8fb5d-pjs4f" Dec 03 12:30:02 crc kubenswrapper[5052]: I1203 12:30:02.191799 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a0e45e1a-ced9-4958-9a75-24a69280f679-combined-ca-bundle\") pod \"placement-688bd8fb5d-pjs4f\" (UID: \"a0e45e1a-ced9-4958-9a75-24a69280f679\") " pod="openstack-kuttl-tests/placement-688bd8fb5d-pjs4f" Dec 03 12:30:02 crc kubenswrapper[5052]: I1203 12:30:02.192084 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a0e45e1a-ced9-4958-9a75-24a69280f679-config-data\") pod \"placement-688bd8fb5d-pjs4f\" (UID: \"a0e45e1a-ced9-4958-9a75-24a69280f679\") " pod="openstack-kuttl-tests/placement-688bd8fb5d-pjs4f" Dec 03 12:30:02 crc kubenswrapper[5052]: I1203 12:30:02.192118 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/a0e45e1a-ced9-4958-9a75-24a69280f679-logs\") pod \"placement-688bd8fb5d-pjs4f\" (UID: \"a0e45e1a-ced9-4958-9a75-24a69280f679\") " pod="openstack-kuttl-tests/placement-688bd8fb5d-pjs4f" Dec 03 12:30:02 crc kubenswrapper[5052]: I1203 12:30:02.192165 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/a0e45e1a-ced9-4958-9a75-24a69280f679-public-tls-certs\") pod \"placement-688bd8fb5d-pjs4f\" (UID: \"a0e45e1a-ced9-4958-9a75-24a69280f679\") " pod="openstack-kuttl-tests/placement-688bd8fb5d-pjs4f" Dec 03 12:30:02 crc kubenswrapper[5052]: I1203 12:30:02.197801 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-7dffdbf8-hzmbm" event={"ID":"53fe6403-27db-44b7-a9a6-787afbc3bbb8","Type":"ContainerStarted","Data":"c110d80f3a367aca9b254271248cc75eddf2eb31290f7f48d1f7ccab8d5d16fd"} Dec 03 12:30:02 crc kubenswrapper[5052]: I1203 12:30:02.222577 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b5b5ad16-1e45-4761-b5d2-60ffd5934ccb-logs" (OuterVolumeSpecName: "logs") pod "b5b5ad16-1e45-4761-b5d2-60ffd5934ccb" (UID: "b5b5ad16-1e45-4761-b5d2-60ffd5934ccb"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:30:02 crc kubenswrapper[5052]: I1203 12:30:02.229847 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b5b5ad16-1e45-4761-b5d2-60ffd5934ccb-kube-api-access-p4phv" (OuterVolumeSpecName: "kube-api-access-p4phv") pod "b5b5ad16-1e45-4761-b5d2-60ffd5934ccb" (UID: "b5b5ad16-1e45-4761-b5d2-60ffd5934ccb"). InnerVolumeSpecName "kube-api-access-p4phv". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:30:02 crc kubenswrapper[5052]: I1203 12:30:02.249328 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-operator-lifecycle-manager/collect-profiles-29412750-42nq6" event={"ID":"6a77ebe2-ac70-4620-acef-bda957db5766","Type":"ContainerStarted","Data":"a9e5b3a74a64e4b357cabfb3951a5add7c42865e635bf5406c566aff3b0049b6"} Dec 03 12:30:02 crc kubenswrapper[5052]: I1203 12:30:02.249372 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-operator-lifecycle-manager/collect-profiles-29412750-42nq6" event={"ID":"6a77ebe2-ac70-4620-acef-bda957db5766","Type":"ContainerStarted","Data":"eb228dee2e6cdc91772f817a0a1e0dd61bffbdf799bae9cf1b3f9e10d968e07e"} Dec 03 12:30:02 crc kubenswrapper[5052]: I1203 12:30:02.281016 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-564bc88b6-bvll5" event={"ID":"c0ce9ced-33b0-4879-81a4-b5c066ec0631","Type":"ContainerStarted","Data":"589675564f1f842a7ea700f8f790adcdebd983109491fa331d941686f179e466"} Dec 03 12:30:02 crc kubenswrapper[5052]: I1203 12:30:02.296815 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/a0e45e1a-ced9-4958-9a75-24a69280f679-public-tls-certs\") pod \"placement-688bd8fb5d-pjs4f\" (UID: \"a0e45e1a-ced9-4958-9a75-24a69280f679\") " pod="openstack-kuttl-tests/placement-688bd8fb5d-pjs4f" Dec 03 12:30:02 crc kubenswrapper[5052]: I1203 12:30:02.296875 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/a0e45e1a-ced9-4958-9a75-24a69280f679-scripts\") pod \"placement-688bd8fb5d-pjs4f\" (UID: \"a0e45e1a-ced9-4958-9a75-24a69280f679\") " pod="openstack-kuttl-tests/placement-688bd8fb5d-pjs4f" Dec 03 12:30:02 crc kubenswrapper[5052]: I1203 12:30:02.296926 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/a0e45e1a-ced9-4958-9a75-24a69280f679-internal-tls-certs\") pod \"placement-688bd8fb5d-pjs4f\" (UID: \"a0e45e1a-ced9-4958-9a75-24a69280f679\") " pod="openstack-kuttl-tests/placement-688bd8fb5d-pjs4f" Dec 03 12:30:02 crc kubenswrapper[5052]: I1203 12:30:02.297015 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-4tb7p\" (UniqueName: \"kubernetes.io/projected/a0e45e1a-ced9-4958-9a75-24a69280f679-kube-api-access-4tb7p\") pod \"placement-688bd8fb5d-pjs4f\" (UID: \"a0e45e1a-ced9-4958-9a75-24a69280f679\") " pod="openstack-kuttl-tests/placement-688bd8fb5d-pjs4f" Dec 03 12:30:02 crc kubenswrapper[5052]: I1203 12:30:02.297064 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a0e45e1a-ced9-4958-9a75-24a69280f679-combined-ca-bundle\") pod \"placement-688bd8fb5d-pjs4f\" (UID: \"a0e45e1a-ced9-4958-9a75-24a69280f679\") " pod="openstack-kuttl-tests/placement-688bd8fb5d-pjs4f" Dec 03 12:30:02 crc kubenswrapper[5052]: I1203 12:30:02.297105 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a0e45e1a-ced9-4958-9a75-24a69280f679-config-data\") pod \"placement-688bd8fb5d-pjs4f\" (UID: \"a0e45e1a-ced9-4958-9a75-24a69280f679\") " pod="openstack-kuttl-tests/placement-688bd8fb5d-pjs4f" Dec 03 12:30:02 crc kubenswrapper[5052]: I1203 12:30:02.297126 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/a0e45e1a-ced9-4958-9a75-24a69280f679-logs\") pod \"placement-688bd8fb5d-pjs4f\" (UID: \"a0e45e1a-ced9-4958-9a75-24a69280f679\") " pod="openstack-kuttl-tests/placement-688bd8fb5d-pjs4f" Dec 03 12:30:02 crc kubenswrapper[5052]: I1203 12:30:02.297183 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-p4phv\" (UniqueName: \"kubernetes.io/projected/b5b5ad16-1e45-4761-b5d2-60ffd5934ccb-kube-api-access-p4phv\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:02 crc kubenswrapper[5052]: I1203 12:30:02.297194 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/b5b5ad16-1e45-4761-b5d2-60ffd5934ccb-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:02 crc kubenswrapper[5052]: I1203 12:30:02.297972 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/a0e45e1a-ced9-4958-9a75-24a69280f679-logs\") pod \"placement-688bd8fb5d-pjs4f\" (UID: \"a0e45e1a-ced9-4958-9a75-24a69280f679\") " pod="openstack-kuttl-tests/placement-688bd8fb5d-pjs4f" Dec 03 12:30:02 crc kubenswrapper[5052]: I1203 12:30:02.331060 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/a0e45e1a-ced9-4958-9a75-24a69280f679-internal-tls-certs\") pod \"placement-688bd8fb5d-pjs4f\" (UID: \"a0e45e1a-ced9-4958-9a75-24a69280f679\") " pod="openstack-kuttl-tests/placement-688bd8fb5d-pjs4f" Dec 03 12:30:02 crc kubenswrapper[5052]: I1203 12:30:02.335705 5052 generic.go:334] "Generic (PLEG): container finished" podID="82bcca5b-e2c3-43ce-aba3-eb879796dc3e" containerID="fa45d03e94f8aa43afec4ca3932bb3d4299d6fe6674927e304720e875b1c73d2" exitCode=0 Dec 03 12:30:02 crc kubenswrapper[5052]: I1203 12:30:02.336243 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"82bcca5b-e2c3-43ce-aba3-eb879796dc3e","Type":"ContainerDied","Data":"fa45d03e94f8aa43afec4ca3932bb3d4299d6fe6674927e304720e875b1c73d2"} Dec 03 12:30:02 crc kubenswrapper[5052]: I1203 12:30:02.342397 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/a0e45e1a-ced9-4958-9a75-24a69280f679-scripts\") pod \"placement-688bd8fb5d-pjs4f\" (UID: \"a0e45e1a-ced9-4958-9a75-24a69280f679\") " pod="openstack-kuttl-tests/placement-688bd8fb5d-pjs4f" Dec 03 12:30:02 crc kubenswrapper[5052]: I1203 12:30:02.352137 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" event={"ID":"f4e095cd-36d8-4136-b262-f6d52717a05c","Type":"ContainerStarted","Data":"1e7025513d6b144fd4de246b9cff9dd24d587dcce1c3318a8c75d6b9165b00ff"} Dec 03 12:30:02 crc kubenswrapper[5052]: I1203 12:30:02.352917 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a0e45e1a-ced9-4958-9a75-24a69280f679-config-data\") pod \"placement-688bd8fb5d-pjs4f\" (UID: \"a0e45e1a-ced9-4958-9a75-24a69280f679\") " pod="openstack-kuttl-tests/placement-688bd8fb5d-pjs4f" Dec 03 12:30:02 crc kubenswrapper[5052]: I1203 12:30:02.374554 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a0e45e1a-ced9-4958-9a75-24a69280f679-combined-ca-bundle\") pod \"placement-688bd8fb5d-pjs4f\" (UID: \"a0e45e1a-ced9-4958-9a75-24a69280f679\") " pod="openstack-kuttl-tests/placement-688bd8fb5d-pjs4f" Dec 03 12:30:02 crc kubenswrapper[5052]: I1203 12:30:02.387700 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-5c6478db8b-bt2rw" event={"ID":"8bddf2b1-8116-4b6e-9cd8-be8fe1cd4b64","Type":"ContainerStarted","Data":"33f8dbc2efb3ac8c4348e12af03e0b1919430c85f4a7565b4e5fc747718afe50"} Dec 03 12:30:02 crc kubenswrapper[5052]: I1203 12:30:02.389357 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/a0e45e1a-ced9-4958-9a75-24a69280f679-public-tls-certs\") pod \"placement-688bd8fb5d-pjs4f\" (UID: \"a0e45e1a-ced9-4958-9a75-24a69280f679\") " pod="openstack-kuttl-tests/placement-688bd8fb5d-pjs4f" Dec 03 12:30:02 crc kubenswrapper[5052]: I1203 12:30:02.398972 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/glance-default-external-api-0" podUID="05b9d3cc-93de-45a6-8849-027a4aad3715" containerName="glance-log" probeResult="failure" output="Get \"https://10.217.0.160:9292/healthcheck\": dial tcp 10.217.0.160:9292: connect: connection refused" Dec 03 12:30:02 crc kubenswrapper[5052]: I1203 12:30:02.399274 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/glance-default-external-api-0" podUID="05b9d3cc-93de-45a6-8849-027a4aad3715" containerName="glance-httpd" probeResult="failure" output="Get \"https://10.217.0.160:9292/healthcheck\": dial tcp 10.217.0.160:9292: connect: connection refused" Dec 03 12:30:02 crc kubenswrapper[5052]: I1203 12:30:02.400321 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-4tb7p\" (UniqueName: \"kubernetes.io/projected/a0e45e1a-ced9-4958-9a75-24a69280f679-kube-api-access-4tb7p\") pod \"placement-688bd8fb5d-pjs4f\" (UID: \"a0e45e1a-ced9-4958-9a75-24a69280f679\") " pod="openstack-kuttl-tests/placement-688bd8fb5d-pjs4f" Dec 03 12:30:02 crc kubenswrapper[5052]: I1203 12:30:02.456845 5052 generic.go:334] "Generic (PLEG): container finished" podID="e75eea2c-2eb1-4f0f-97e5-63ce147e68dd" containerID="069e9d7b5055dc14bd89a6f68f4d0ccea363e1ff3aad490e83aff00d8a0ad411" exitCode=0 Dec 03 12:30:02 crc kubenswrapper[5052]: I1203 12:30:02.456941 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:30:02 crc kubenswrapper[5052]: I1203 12:30:02.457353 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-688bd8fb5d-pjs4f" Dec 03 12:30:02 crc kubenswrapper[5052]: I1203 12:30:02.457905 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"e75eea2c-2eb1-4f0f-97e5-63ce147e68dd","Type":"ContainerDied","Data":"069e9d7b5055dc14bd89a6f68f4d0ccea363e1ff3aad490e83aff00d8a0ad411"} Dec 03 12:30:02 crc kubenswrapper[5052]: I1203 12:30:02.458124 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/keystone-7cbf547d94-9krth" podUID="6927052c-212d-42fc-8236-b053ffc9a9d9" containerName="keystone-api" containerID="cri-o://edaf8cd0d0bbbd5a49f586c7c2c1e7bcdb11f76c906dcb6bcd4c10118fa5950b" gracePeriod=30 Dec 03 12:30:02 crc kubenswrapper[5052]: I1203 12:30:02.458174 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/keystone-7cbf547d94-9krth" Dec 03 12:30:02 crc kubenswrapper[5052]: I1203 12:30:02.509403 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/keystone-7cbf547d94-9krth" podStartSLOduration=8.509385677000001 podStartE2EDuration="8.509385677s" podCreationTimestamp="2025-12-03 12:29:54 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:30:02.489951802 +0000 UTC m=+4193.078193576" watchObservedRunningTime="2025-12-03 12:30:02.509385677 +0000 UTC m=+4193.097627441" Dec 03 12:30:02 crc kubenswrapper[5052]: I1203 12:30:02.540924 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:30:02 crc kubenswrapper[5052]: I1203 12:30:02.564198 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:30:02 crc kubenswrapper[5052]: I1203 12:30:02.636587 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:30:02 crc kubenswrapper[5052]: I1203 12:30:02.638654 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:30:02 crc kubenswrapper[5052]: I1203 12:30:02.644953 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-nova-metadata-internal-svc" Dec 03 12:30:02 crc kubenswrapper[5052]: I1203 12:30:02.645286 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-metadata-config-data" Dec 03 12:30:02 crc kubenswrapper[5052]: I1203 12:30:02.645757 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b5b5ad16-1e45-4761-b5d2-60ffd5934ccb-config-data" (OuterVolumeSpecName: "config-data") pod "b5b5ad16-1e45-4761-b5d2-60ffd5934ccb" (UID: "b5b5ad16-1e45-4761-b5d2-60ffd5934ccb"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:02 crc kubenswrapper[5052]: I1203 12:30:02.682653 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:30:02 crc kubenswrapper[5052]: I1203 12:30:02.687059 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b5b5ad16-1e45-4761-b5d2-60ffd5934ccb-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "b5b5ad16-1e45-4761-b5d2-60ffd5934ccb" (UID: "b5b5ad16-1e45-4761-b5d2-60ffd5934ccb"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:02 crc kubenswrapper[5052]: I1203 12:30:02.751738 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b5b5ad16-1e45-4761-b5d2-60ffd5934ccb-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "b5b5ad16-1e45-4761-b5d2-60ffd5934ccb" (UID: "b5b5ad16-1e45-4761-b5d2-60ffd5934ccb"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:02 crc kubenswrapper[5052]: I1203 12:30:02.777300 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b5b5ad16-1e45-4761-b5d2-60ffd5934ccb-combined-ca-bundle\") pod \"b5b5ad16-1e45-4761-b5d2-60ffd5934ccb\" (UID: \"b5b5ad16-1e45-4761-b5d2-60ffd5934ccb\") " Dec 03 12:30:02 crc kubenswrapper[5052]: W1203 12:30:02.777433 5052 empty_dir.go:500] Warning: Unmount skipped because path does not exist: /var/lib/kubelet/pods/b5b5ad16-1e45-4761-b5d2-60ffd5934ccb/volumes/kubernetes.io~secret/combined-ca-bundle Dec 03 12:30:02 crc kubenswrapper[5052]: I1203 12:30:02.777457 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b5b5ad16-1e45-4761-b5d2-60ffd5934ccb-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "b5b5ad16-1e45-4761-b5d2-60ffd5934ccb" (UID: "b5b5ad16-1e45-4761-b5d2-60ffd5934ccb"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:02 crc kubenswrapper[5052]: I1203 12:30:02.777703 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/39117149-c15d-4753-9324-8a42abe5b007-config-data\") pod \"nova-metadata-0\" (UID: \"39117149-c15d-4753-9324-8a42abe5b007\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:30:02 crc kubenswrapper[5052]: I1203 12:30:02.777789 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/39117149-c15d-4753-9324-8a42abe5b007-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"39117149-c15d-4753-9324-8a42abe5b007\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:30:02 crc kubenswrapper[5052]: I1203 12:30:02.777899 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tcf7d\" (UniqueName: \"kubernetes.io/projected/39117149-c15d-4753-9324-8a42abe5b007-kube-api-access-tcf7d\") pod \"nova-metadata-0\" (UID: \"39117149-c15d-4753-9324-8a42abe5b007\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:30:02 crc kubenswrapper[5052]: I1203 12:30:02.777966 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/39117149-c15d-4753-9324-8a42abe5b007-logs\") pod \"nova-metadata-0\" (UID: \"39117149-c15d-4753-9324-8a42abe5b007\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:30:02 crc kubenswrapper[5052]: I1203 12:30:02.778113 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/39117149-c15d-4753-9324-8a42abe5b007-nova-metadata-tls-certs\") pod \"nova-metadata-0\" (UID: \"39117149-c15d-4753-9324-8a42abe5b007\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:30:02 crc kubenswrapper[5052]: I1203 12:30:02.778262 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b5b5ad16-1e45-4761-b5d2-60ffd5934ccb-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:02 crc kubenswrapper[5052]: I1203 12:30:02.778274 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/b5b5ad16-1e45-4761-b5d2-60ffd5934ccb-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:02 crc kubenswrapper[5052]: I1203 12:30:02.778282 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/b5b5ad16-1e45-4761-b5d2-60ffd5934ccb-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:02 crc kubenswrapper[5052]: I1203 12:30:02.889521 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-tcf7d\" (UniqueName: \"kubernetes.io/projected/39117149-c15d-4753-9324-8a42abe5b007-kube-api-access-tcf7d\") pod \"nova-metadata-0\" (UID: \"39117149-c15d-4753-9324-8a42abe5b007\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:30:02 crc kubenswrapper[5052]: I1203 12:30:02.889613 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/39117149-c15d-4753-9324-8a42abe5b007-logs\") pod \"nova-metadata-0\" (UID: \"39117149-c15d-4753-9324-8a42abe5b007\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:30:02 crc kubenswrapper[5052]: I1203 12:30:02.889796 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/39117149-c15d-4753-9324-8a42abe5b007-nova-metadata-tls-certs\") pod \"nova-metadata-0\" (UID: \"39117149-c15d-4753-9324-8a42abe5b007\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:30:02 crc kubenswrapper[5052]: I1203 12:30:02.889945 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/39117149-c15d-4753-9324-8a42abe5b007-config-data\") pod \"nova-metadata-0\" (UID: \"39117149-c15d-4753-9324-8a42abe5b007\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:30:02 crc kubenswrapper[5052]: I1203 12:30:02.890334 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/39117149-c15d-4753-9324-8a42abe5b007-logs\") pod \"nova-metadata-0\" (UID: \"39117149-c15d-4753-9324-8a42abe5b007\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:30:02 crc kubenswrapper[5052]: I1203 12:30:02.892305 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/39117149-c15d-4753-9324-8a42abe5b007-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"39117149-c15d-4753-9324-8a42abe5b007\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:30:02 crc kubenswrapper[5052]: E1203 12:30:02.896860 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="c1a2fa107302843e2249473b85f61687344a151311126ad578b5760b51b1b60e" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:30:02 crc kubenswrapper[5052]: I1203 12:30:02.911813 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/memcached-0" podUID="b35e6939-8f23-496c-ac55-9534be4ea8be" containerName="memcached" probeResult="failure" output="dial tcp 10.217.0.151:11211: i/o timeout" Dec 03 12:30:02 crc kubenswrapper[5052]: I1203 12:30:02.915554 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/39117149-c15d-4753-9324-8a42abe5b007-config-data\") pod \"nova-metadata-0\" (UID: \"39117149-c15d-4753-9324-8a42abe5b007\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:30:02 crc kubenswrapper[5052]: I1203 12:30:02.921865 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/39117149-c15d-4753-9324-8a42abe5b007-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"39117149-c15d-4753-9324-8a42abe5b007\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:30:02 crc kubenswrapper[5052]: I1203 12:30:02.929571 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/39117149-c15d-4753-9324-8a42abe5b007-nova-metadata-tls-certs\") pod \"nova-metadata-0\" (UID: \"39117149-c15d-4753-9324-8a42abe5b007\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:30:02 crc kubenswrapper[5052]: I1203 12:30:02.949815 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-tcf7d\" (UniqueName: \"kubernetes.io/projected/39117149-c15d-4753-9324-8a42abe5b007-kube-api-access-tcf7d\") pod \"nova-metadata-0\" (UID: \"39117149-c15d-4753-9324-8a42abe5b007\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:30:02 crc kubenswrapper[5052]: E1203 12:30:02.950001 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="c1a2fa107302843e2249473b85f61687344a151311126ad578b5760b51b1b60e" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:30:02 crc kubenswrapper[5052]: E1203 12:30:02.992354 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="c1a2fa107302843e2249473b85f61687344a151311126ad578b5760b51b1b60e" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:30:02 crc kubenswrapper[5052]: E1203 12:30:02.992423 5052 prober.go:104] "Probe errored" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" probeType="Readiness" pod="openstack-kuttl-tests/nova-cell0-conductor-0" podUID="16a065b5-a603-4cb3-a4c0-b36c0b92bf3f" containerName="nova-cell0-conductor-conductor" Dec 03 12:30:03 crc kubenswrapper[5052]: I1203 12:30:03.032276 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b5b5ad16-1e45-4761-b5d2-60ffd5934ccb-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "b5b5ad16-1e45-4761-b5d2-60ffd5934ccb" (UID: "b5b5ad16-1e45-4761-b5d2-60ffd5934ccb"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:03 crc kubenswrapper[5052]: I1203 12:30:03.109576 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/b5b5ad16-1e45-4761-b5d2-60ffd5934ccb-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:03 crc kubenswrapper[5052]: I1203 12:30:03.164929 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:30:03 crc kubenswrapper[5052]: I1203 12:30:03.225779 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:30:03 crc kubenswrapper[5052]: I1203 12:30:03.257302 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:30:03 crc kubenswrapper[5052]: I1203 12:30:03.395635 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:30:03 crc kubenswrapper[5052]: I1203 12:30:03.423597 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:30:03 crc kubenswrapper[5052]: E1203 12:30:03.424416 5052 cadvisor_stats_provider.go:516] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a77ebe2_ac70_4620_acef_bda957db5766.slice/crio-conmon-a9e5b3a74a64e4b357cabfb3951a5add7c42865e635bf5406c566aff3b0049b6.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb953ffa0_088c_4450_8d60_04044b33cfa0.slice/crio-6f35ce14790c9ffb5796ad1467515b2be517ae60eb2573a1571ba5f99f630cad.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6927052c_212d_42fc_8236_b053ffc9a9d9.slice/crio-edaf8cd0d0bbbd5a49f586c7c2c1e7bcdb11f76c906dcb6bcd4c10118fa5950b.scope\": RecentStats: unable to find data in memory cache]" Dec 03 12:30:03 crc kubenswrapper[5052]: I1203 12:30:03.442605 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vq44t\" (UniqueName: \"kubernetes.io/projected/61cad274-57bd-4993-aaa6-3e6013900d88-kube-api-access-vq44t\") pod \"61cad274-57bd-4993-aaa6-3e6013900d88\" (UID: \"61cad274-57bd-4993-aaa6-3e6013900d88\") " Dec 03 12:30:03 crc kubenswrapper[5052]: I1203 12:30:03.442930 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/82bcca5b-e2c3-43ce-aba3-eb879796dc3e-config-data\") pod \"82bcca5b-e2c3-43ce-aba3-eb879796dc3e\" (UID: \"82bcca5b-e2c3-43ce-aba3-eb879796dc3e\") " Dec 03 12:30:03 crc kubenswrapper[5052]: I1203 12:30:03.442958 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/82bcca5b-e2c3-43ce-aba3-eb879796dc3e-logs\") pod \"82bcca5b-e2c3-43ce-aba3-eb879796dc3e\" (UID: \"82bcca5b-e2c3-43ce-aba3-eb879796dc3e\") " Dec 03 12:30:03 crc kubenswrapper[5052]: I1203 12:30:03.443011 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/61cad274-57bd-4993-aaa6-3e6013900d88-public-tls-certs\") pod \"61cad274-57bd-4993-aaa6-3e6013900d88\" (UID: \"61cad274-57bd-4993-aaa6-3e6013900d88\") " Dec 03 12:30:03 crc kubenswrapper[5052]: I1203 12:30:03.443086 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"glance\" (UniqueName: \"kubernetes.io/local-volume/local-storage09-crc\") pod \"82bcca5b-e2c3-43ce-aba3-eb879796dc3e\" (UID: \"82bcca5b-e2c3-43ce-aba3-eb879796dc3e\") " Dec 03 12:30:03 crc kubenswrapper[5052]: I1203 12:30:03.443119 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/82bcca5b-e2c3-43ce-aba3-eb879796dc3e-combined-ca-bundle\") pod \"82bcca5b-e2c3-43ce-aba3-eb879796dc3e\" (UID: \"82bcca5b-e2c3-43ce-aba3-eb879796dc3e\") " Dec 03 12:30:03 crc kubenswrapper[5052]: I1203 12:30:03.443172 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/61cad274-57bd-4993-aaa6-3e6013900d88-scripts\") pod \"61cad274-57bd-4993-aaa6-3e6013900d88\" (UID: \"61cad274-57bd-4993-aaa6-3e6013900d88\") " Dec 03 12:30:03 crc kubenswrapper[5052]: I1203 12:30:03.443203 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/61cad274-57bd-4993-aaa6-3e6013900d88-etc-machine-id\") pod \"61cad274-57bd-4993-aaa6-3e6013900d88\" (UID: \"61cad274-57bd-4993-aaa6-3e6013900d88\") " Dec 03 12:30:03 crc kubenswrapper[5052]: I1203 12:30:03.443221 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/61cad274-57bd-4993-aaa6-3e6013900d88-combined-ca-bundle\") pod \"61cad274-57bd-4993-aaa6-3e6013900d88\" (UID: \"61cad274-57bd-4993-aaa6-3e6013900d88\") " Dec 03 12:30:03 crc kubenswrapper[5052]: I1203 12:30:03.443287 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/82bcca5b-e2c3-43ce-aba3-eb879796dc3e-httpd-run\") pod \"82bcca5b-e2c3-43ce-aba3-eb879796dc3e\" (UID: \"82bcca5b-e2c3-43ce-aba3-eb879796dc3e\") " Dec 03 12:30:03 crc kubenswrapper[5052]: I1203 12:30:03.443307 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/82bcca5b-e2c3-43ce-aba3-eb879796dc3e-scripts\") pod \"82bcca5b-e2c3-43ce-aba3-eb879796dc3e\" (UID: \"82bcca5b-e2c3-43ce-aba3-eb879796dc3e\") " Dec 03 12:30:03 crc kubenswrapper[5052]: I1203 12:30:03.443322 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/61cad274-57bd-4993-aaa6-3e6013900d88-logs\") pod \"61cad274-57bd-4993-aaa6-3e6013900d88\" (UID: \"61cad274-57bd-4993-aaa6-3e6013900d88\") " Dec 03 12:30:03 crc kubenswrapper[5052]: I1203 12:30:03.443338 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/82bcca5b-e2c3-43ce-aba3-eb879796dc3e-internal-tls-certs\") pod \"82bcca5b-e2c3-43ce-aba3-eb879796dc3e\" (UID: \"82bcca5b-e2c3-43ce-aba3-eb879796dc3e\") " Dec 03 12:30:03 crc kubenswrapper[5052]: I1203 12:30:03.443356 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/61cad274-57bd-4993-aaa6-3e6013900d88-config-data\") pod \"61cad274-57bd-4993-aaa6-3e6013900d88\" (UID: \"61cad274-57bd-4993-aaa6-3e6013900d88\") " Dec 03 12:30:03 crc kubenswrapper[5052]: I1203 12:30:03.443372 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/61cad274-57bd-4993-aaa6-3e6013900d88-config-data-custom\") pod \"61cad274-57bd-4993-aaa6-3e6013900d88\" (UID: \"61cad274-57bd-4993-aaa6-3e6013900d88\") " Dec 03 12:30:03 crc kubenswrapper[5052]: I1203 12:30:03.443390 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/61cad274-57bd-4993-aaa6-3e6013900d88-internal-tls-certs\") pod \"61cad274-57bd-4993-aaa6-3e6013900d88\" (UID: \"61cad274-57bd-4993-aaa6-3e6013900d88\") " Dec 03 12:30:03 crc kubenswrapper[5052]: I1203 12:30:03.443407 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5xg6n\" (UniqueName: \"kubernetes.io/projected/82bcca5b-e2c3-43ce-aba3-eb879796dc3e-kube-api-access-5xg6n\") pod \"82bcca5b-e2c3-43ce-aba3-eb879796dc3e\" (UID: \"82bcca5b-e2c3-43ce-aba3-eb879796dc3e\") " Dec 03 12:30:03 crc kubenswrapper[5052]: I1203 12:30:03.451157 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/61cad274-57bd-4993-aaa6-3e6013900d88-logs" (OuterVolumeSpecName: "logs") pod "61cad274-57bd-4993-aaa6-3e6013900d88" (UID: "61cad274-57bd-4993-aaa6-3e6013900d88"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:30:03 crc kubenswrapper[5052]: I1203 12:30:03.465461 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:30:03 crc kubenswrapper[5052]: E1203 12:30:03.466203 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="61cad274-57bd-4993-aaa6-3e6013900d88" containerName="cinder-api-log" Dec 03 12:30:03 crc kubenswrapper[5052]: I1203 12:30:03.466241 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="61cad274-57bd-4993-aaa6-3e6013900d88" containerName="cinder-api-log" Dec 03 12:30:03 crc kubenswrapper[5052]: E1203 12:30:03.466255 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="61cad274-57bd-4993-aaa6-3e6013900d88" containerName="cinder-api" Dec 03 12:30:03 crc kubenswrapper[5052]: I1203 12:30:03.466262 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="61cad274-57bd-4993-aaa6-3e6013900d88" containerName="cinder-api" Dec 03 12:30:03 crc kubenswrapper[5052]: E1203 12:30:03.466284 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="82bcca5b-e2c3-43ce-aba3-eb879796dc3e" containerName="glance-httpd" Dec 03 12:30:03 crc kubenswrapper[5052]: I1203 12:30:03.466292 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="82bcca5b-e2c3-43ce-aba3-eb879796dc3e" containerName="glance-httpd" Dec 03 12:30:03 crc kubenswrapper[5052]: E1203 12:30:03.466314 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="82bcca5b-e2c3-43ce-aba3-eb879796dc3e" containerName="glance-log" Dec 03 12:30:03 crc kubenswrapper[5052]: I1203 12:30:03.466321 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="82bcca5b-e2c3-43ce-aba3-eb879796dc3e" containerName="glance-log" Dec 03 12:30:03 crc kubenswrapper[5052]: I1203 12:30:03.466617 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="61cad274-57bd-4993-aaa6-3e6013900d88" containerName="cinder-api-log" Dec 03 12:30:03 crc kubenswrapper[5052]: I1203 12:30:03.466643 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="82bcca5b-e2c3-43ce-aba3-eb879796dc3e" containerName="glance-log" Dec 03 12:30:03 crc kubenswrapper[5052]: I1203 12:30:03.466664 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="61cad274-57bd-4993-aaa6-3e6013900d88" containerName="cinder-api" Dec 03 12:30:03 crc kubenswrapper[5052]: I1203 12:30:03.466678 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="82bcca5b-e2c3-43ce-aba3-eb879796dc3e" containerName="glance-httpd" Dec 03 12:30:03 crc kubenswrapper[5052]: I1203 12:30:03.468690 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/61cad274-57bd-4993-aaa6-3e6013900d88-etc-machine-id" (OuterVolumeSpecName: "etc-machine-id") pod "61cad274-57bd-4993-aaa6-3e6013900d88" (UID: "61cad274-57bd-4993-aaa6-3e6013900d88"). InnerVolumeSpecName "etc-machine-id". PluginName "kubernetes.io/host-path", VolumeGidValue "" Dec 03 12:30:03 crc kubenswrapper[5052]: I1203 12:30:03.470155 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/82bcca5b-e2c3-43ce-aba3-eb879796dc3e-httpd-run" (OuterVolumeSpecName: "httpd-run") pod "82bcca5b-e2c3-43ce-aba3-eb879796dc3e" (UID: "82bcca5b-e2c3-43ce-aba3-eb879796dc3e"). InnerVolumeSpecName "httpd-run". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:30:03 crc kubenswrapper[5052]: I1203 12:30:03.470299 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/82bcca5b-e2c3-43ce-aba3-eb879796dc3e-logs" (OuterVolumeSpecName: "logs") pod "82bcca5b-e2c3-43ce-aba3-eb879796dc3e" (UID: "82bcca5b-e2c3-43ce-aba3-eb879796dc3e"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:30:03 crc kubenswrapper[5052]: I1203 12:30:03.471414 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:30:03 crc kubenswrapper[5052]: I1203 12:30:03.490228 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-nova-internal-svc" Dec 03 12:30:03 crc kubenswrapper[5052]: I1203 12:30:03.490391 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-nova-public-svc" Dec 03 12:30:03 crc kubenswrapper[5052]: I1203 12:30:03.490563 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-api-config-data" Dec 03 12:30:03 crc kubenswrapper[5052]: I1203 12:30:03.499152 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:30:03 crc kubenswrapper[5052]: I1203 12:30:03.509738 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/placement-688bd8fb5d-pjs4f"] Dec 03 12:30:03 crc kubenswrapper[5052]: I1203 12:30:03.511938 5052 generic.go:334] "Generic (PLEG): container finished" podID="6a77ebe2-ac70-4620-acef-bda957db5766" containerID="a9e5b3a74a64e4b357cabfb3951a5add7c42865e635bf5406c566aff3b0049b6" exitCode=0 Dec 03 12:30:03 crc kubenswrapper[5052]: I1203 12:30:03.511993 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-operator-lifecycle-manager/collect-profiles-29412750-42nq6" event={"ID":"6a77ebe2-ac70-4620-acef-bda957db5766","Type":"ContainerDied","Data":"a9e5b3a74a64e4b357cabfb3951a5add7c42865e635bf5406c566aff3b0049b6"} Dec 03 12:30:03 crc kubenswrapper[5052]: I1203 12:30:03.521058 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/61cad274-57bd-4993-aaa6-3e6013900d88-kube-api-access-vq44t" (OuterVolumeSpecName: "kube-api-access-vq44t") pod "61cad274-57bd-4993-aaa6-3e6013900d88" (UID: "61cad274-57bd-4993-aaa6-3e6013900d88"). InnerVolumeSpecName "kube-api-access-vq44t". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:30:03 crc kubenswrapper[5052]: I1203 12:30:03.523900 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/61cad274-57bd-4993-aaa6-3e6013900d88-config-data-custom" (OuterVolumeSpecName: "config-data-custom") pod "61cad274-57bd-4993-aaa6-3e6013900d88" (UID: "61cad274-57bd-4993-aaa6-3e6013900d88"). InnerVolumeSpecName "config-data-custom". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:03 crc kubenswrapper[5052]: I1203 12:30:03.526567 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/placement-65444bfc86-th5pw"] Dec 03 12:30:03 crc kubenswrapper[5052]: I1203 12:30:03.528312 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-65444bfc86-th5pw" Dec 03 12:30:03 crc kubenswrapper[5052]: I1203 12:30:03.536203 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-keystone-listener-77b6bbbdcb-4jlsc" event={"ID":"aed41b09-319e-466c-9eb9-38208e1880a3","Type":"ContainerStarted","Data":"0a78aa778c6e6e1f2c1d7cc5f57fa0dc6dd213f54da357a449dda9efb5e3bc63"} Dec 03 12:30:03 crc kubenswrapper[5052]: I1203 12:30:03.536346 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/barbican-keystone-listener-77b6bbbdcb-4jlsc" podUID="aed41b09-319e-466c-9eb9-38208e1880a3" containerName="barbican-keystone-listener-log" containerID="cri-o://c4632c9d758da0d71f63ae0b7ef4c62003466443e6cb1e897bae7412a5ae794f" gracePeriod=30 Dec 03 12:30:03 crc kubenswrapper[5052]: I1203 12:30:03.536573 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/barbican-keystone-listener-77b6bbbdcb-4jlsc" podUID="aed41b09-319e-466c-9eb9-38208e1880a3" containerName="barbican-keystone-listener" containerID="cri-o://0a78aa778c6e6e1f2c1d7cc5f57fa0dc6dd213f54da357a449dda9efb5e3bc63" gracePeriod=30 Dec 03 12:30:03 crc kubenswrapper[5052]: I1203 12:30:03.542707 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage09-crc" (OuterVolumeSpecName: "glance") pod "82bcca5b-e2c3-43ce-aba3-eb879796dc3e" (UID: "82bcca5b-e2c3-43ce-aba3-eb879796dc3e"). InnerVolumeSpecName "local-storage09-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 12:30:03 crc kubenswrapper[5052]: I1203 12:30:03.545304 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/4314928d-a43f-4b38-9c72-63977e59345b-config-data\") pod \"nova-api-0\" (UID: \"4314928d-a43f-4b38-9c72-63977e59345b\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:30:03 crc kubenswrapper[5052]: I1203 12:30:03.545375 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/4314928d-a43f-4b38-9c72-63977e59345b-internal-tls-certs\") pod \"nova-api-0\" (UID: \"4314928d-a43f-4b38-9c72-63977e59345b\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:30:03 crc kubenswrapper[5052]: I1203 12:30:03.545412 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/4314928d-a43f-4b38-9c72-63977e59345b-public-tls-certs\") pod \"nova-api-0\" (UID: \"4314928d-a43f-4b38-9c72-63977e59345b\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:30:03 crc kubenswrapper[5052]: I1203 12:30:03.545474 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pfvmv\" (UniqueName: \"kubernetes.io/projected/4314928d-a43f-4b38-9c72-63977e59345b-kube-api-access-pfvmv\") pod \"nova-api-0\" (UID: \"4314928d-a43f-4b38-9c72-63977e59345b\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:30:03 crc kubenswrapper[5052]: I1203 12:30:03.545516 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/4314928d-a43f-4b38-9c72-63977e59345b-logs\") pod \"nova-api-0\" (UID: \"4314928d-a43f-4b38-9c72-63977e59345b\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:30:03 crc kubenswrapper[5052]: I1203 12:30:03.545567 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4314928d-a43f-4b38-9c72-63977e59345b-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"4314928d-a43f-4b38-9c72-63977e59345b\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:30:03 crc kubenswrapper[5052]: I1203 12:30:03.545629 5052 reconciler_common.go:293] "Volume detached for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/61cad274-57bd-4993-aaa6-3e6013900d88-etc-machine-id\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:03 crc kubenswrapper[5052]: I1203 12:30:03.545642 5052 reconciler_common.go:293] "Volume detached for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/82bcca5b-e2c3-43ce-aba3-eb879796dc3e-httpd-run\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:03 crc kubenswrapper[5052]: I1203 12:30:03.545650 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/61cad274-57bd-4993-aaa6-3e6013900d88-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:03 crc kubenswrapper[5052]: I1203 12:30:03.545658 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/61cad274-57bd-4993-aaa6-3e6013900d88-config-data-custom\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:03 crc kubenswrapper[5052]: I1203 12:30:03.545667 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-vq44t\" (UniqueName: \"kubernetes.io/projected/61cad274-57bd-4993-aaa6-3e6013900d88-kube-api-access-vq44t\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:03 crc kubenswrapper[5052]: I1203 12:30:03.545676 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/82bcca5b-e2c3-43ce-aba3-eb879796dc3e-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:03 crc kubenswrapper[5052]: I1203 12:30:03.545696 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage09-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage09-crc\") on node \"crc\" " Dec 03 12:30:03 crc kubenswrapper[5052]: I1203 12:30:03.548744 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/82bcca5b-e2c3-43ce-aba3-eb879796dc3e-kube-api-access-5xg6n" (OuterVolumeSpecName: "kube-api-access-5xg6n") pod "82bcca5b-e2c3-43ce-aba3-eb879796dc3e" (UID: "82bcca5b-e2c3-43ce-aba3-eb879796dc3e"). InnerVolumeSpecName "kube-api-access-5xg6n". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:30:03 crc kubenswrapper[5052]: I1203 12:30:03.548923 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/placement-65444bfc86-th5pw"] Dec 03 12:30:03 crc kubenswrapper[5052]: I1203 12:30:03.552559 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/82bcca5b-e2c3-43ce-aba3-eb879796dc3e-scripts" (OuterVolumeSpecName: "scripts") pod "82bcca5b-e2c3-43ce-aba3-eb879796dc3e" (UID: "82bcca5b-e2c3-43ce-aba3-eb879796dc3e"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:03 crc kubenswrapper[5052]: I1203 12:30:03.554926 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/61cad274-57bd-4993-aaa6-3e6013900d88-scripts" (OuterVolumeSpecName: "scripts") pod "61cad274-57bd-4993-aaa6-3e6013900d88" (UID: "61cad274-57bd-4993-aaa6-3e6013900d88"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:03 crc kubenswrapper[5052]: I1203 12:30:03.559719 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/placement-688bd8fb5d-pjs4f"] Dec 03 12:30:03 crc kubenswrapper[5052]: I1203 12:30:03.599743 5052 generic.go:334] "Generic (PLEG): container finished" podID="b953ffa0-088c-4450-8d60-04044b33cfa0" containerID="6f35ce14790c9ffb5796ad1467515b2be517ae60eb2573a1571ba5f99f630cad" exitCode=0 Dec 03 12:30:03 crc kubenswrapper[5052]: I1203 12:30:03.599846 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"b953ffa0-088c-4450-8d60-04044b33cfa0","Type":"ContainerDied","Data":"6f35ce14790c9ffb5796ad1467515b2be517ae60eb2573a1571ba5f99f630cad"} Dec 03 12:30:03 crc kubenswrapper[5052]: I1203 12:30:03.625739 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/barbican-keystone-listener-77b6bbbdcb-4jlsc" podStartSLOduration=9.625717989 podStartE2EDuration="9.625717989s" podCreationTimestamp="2025-12-03 12:29:54 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:30:03.61697836 +0000 UTC m=+4194.205220124" watchObservedRunningTime="2025-12-03 12:30:03.625717989 +0000 UTC m=+4194.213959753" Dec 03 12:30:03 crc kubenswrapper[5052]: I1203 12:30:03.638329 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"61cad274-57bd-4993-aaa6-3e6013900d88","Type":"ContainerDied","Data":"4a8e66b1f4c01429203f6a3b2e8e1ac5e1270c3ab51113fb2692b0d064c61351"} Dec 03 12:30:03 crc kubenswrapper[5052]: I1203 12:30:03.638368 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:30:03 crc kubenswrapper[5052]: I1203 12:30:03.638401 5052 scope.go:117] "RemoveContainer" containerID="d1ad86af122d9acbc9e7b5bb95477effe197a4939198ba933eb8fb9e83c0f569" Dec 03 12:30:03 crc kubenswrapper[5052]: I1203 12:30:03.648263 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4314928d-a43f-4b38-9c72-63977e59345b-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"4314928d-a43f-4b38-9c72-63977e59345b\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:30:03 crc kubenswrapper[5052]: I1203 12:30:03.648347 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/cfd8af50-a644-4b24-8483-dd344f2218e4-public-tls-certs\") pod \"placement-65444bfc86-th5pw\" (UID: \"cfd8af50-a644-4b24-8483-dd344f2218e4\") " pod="openstack-kuttl-tests/placement-65444bfc86-th5pw" Dec 03 12:30:03 crc kubenswrapper[5052]: I1203 12:30:03.648381 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/4314928d-a43f-4b38-9c72-63977e59345b-config-data\") pod \"nova-api-0\" (UID: \"4314928d-a43f-4b38-9c72-63977e59345b\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:30:03 crc kubenswrapper[5052]: I1203 12:30:03.648436 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/cfd8af50-a644-4b24-8483-dd344f2218e4-internal-tls-certs\") pod \"placement-65444bfc86-th5pw\" (UID: \"cfd8af50-a644-4b24-8483-dd344f2218e4\") " pod="openstack-kuttl-tests/placement-65444bfc86-th5pw" Dec 03 12:30:03 crc kubenswrapper[5052]: I1203 12:30:03.648538 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/cfd8af50-a644-4b24-8483-dd344f2218e4-combined-ca-bundle\") pod \"placement-65444bfc86-th5pw\" (UID: \"cfd8af50-a644-4b24-8483-dd344f2218e4\") " pod="openstack-kuttl-tests/placement-65444bfc86-th5pw" Dec 03 12:30:03 crc kubenswrapper[5052]: I1203 12:30:03.648608 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jkbfb\" (UniqueName: \"kubernetes.io/projected/cfd8af50-a644-4b24-8483-dd344f2218e4-kube-api-access-jkbfb\") pod \"placement-65444bfc86-th5pw\" (UID: \"cfd8af50-a644-4b24-8483-dd344f2218e4\") " pod="openstack-kuttl-tests/placement-65444bfc86-th5pw" Dec 03 12:30:03 crc kubenswrapper[5052]: I1203 12:30:03.648659 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/4314928d-a43f-4b38-9c72-63977e59345b-internal-tls-certs\") pod \"nova-api-0\" (UID: \"4314928d-a43f-4b38-9c72-63977e59345b\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:30:03 crc kubenswrapper[5052]: I1203 12:30:03.648686 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/4314928d-a43f-4b38-9c72-63977e59345b-public-tls-certs\") pod \"nova-api-0\" (UID: \"4314928d-a43f-4b38-9c72-63977e59345b\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:30:03 crc kubenswrapper[5052]: I1203 12:30:03.648768 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/cfd8af50-a644-4b24-8483-dd344f2218e4-scripts\") pod \"placement-65444bfc86-th5pw\" (UID: \"cfd8af50-a644-4b24-8483-dd344f2218e4\") " pod="openstack-kuttl-tests/placement-65444bfc86-th5pw" Dec 03 12:30:03 crc kubenswrapper[5052]: I1203 12:30:03.648793 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-pfvmv\" (UniqueName: \"kubernetes.io/projected/4314928d-a43f-4b38-9c72-63977e59345b-kube-api-access-pfvmv\") pod \"nova-api-0\" (UID: \"4314928d-a43f-4b38-9c72-63977e59345b\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:30:03 crc kubenswrapper[5052]: I1203 12:30:03.648847 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/4314928d-a43f-4b38-9c72-63977e59345b-logs\") pod \"nova-api-0\" (UID: \"4314928d-a43f-4b38-9c72-63977e59345b\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:30:03 crc kubenswrapper[5052]: I1203 12:30:03.648911 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/cfd8af50-a644-4b24-8483-dd344f2218e4-logs\") pod \"placement-65444bfc86-th5pw\" (UID: \"cfd8af50-a644-4b24-8483-dd344f2218e4\") " pod="openstack-kuttl-tests/placement-65444bfc86-th5pw" Dec 03 12:30:03 crc kubenswrapper[5052]: I1203 12:30:03.648940 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/cfd8af50-a644-4b24-8483-dd344f2218e4-config-data\") pod \"placement-65444bfc86-th5pw\" (UID: \"cfd8af50-a644-4b24-8483-dd344f2218e4\") " pod="openstack-kuttl-tests/placement-65444bfc86-th5pw" Dec 03 12:30:03 crc kubenswrapper[5052]: I1203 12:30:03.649479 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-5xg6n\" (UniqueName: \"kubernetes.io/projected/82bcca5b-e2c3-43ce-aba3-eb879796dc3e-kube-api-access-5xg6n\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:03 crc kubenswrapper[5052]: I1203 12:30:03.649492 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/61cad274-57bd-4993-aaa6-3e6013900d88-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:03 crc kubenswrapper[5052]: I1203 12:30:03.649501 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/82bcca5b-e2c3-43ce-aba3-eb879796dc3e-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:03 crc kubenswrapper[5052]: I1203 12:30:03.655975 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/4314928d-a43f-4b38-9c72-63977e59345b-logs\") pod \"nova-api-0\" (UID: \"4314928d-a43f-4b38-9c72-63977e59345b\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:30:03 crc kubenswrapper[5052]: I1203 12:30:03.658400 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:30:03 crc kubenswrapper[5052]: I1203 12:30:03.658484 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"82bcca5b-e2c3-43ce-aba3-eb879796dc3e","Type":"ContainerDied","Data":"9048c3f157c9a9bacdb7bf1e88d5723f208a649d589a35d9f378cacdb9dbea22"} Dec 03 12:30:03 crc kubenswrapper[5052]: I1203 12:30:03.660964 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/memcached-0" event={"ID":"19fe6517-a9bd-4463-b2b1-e006bd5d8145","Type":"ContainerStarted","Data":"486a415678ff9d664168e49847f09106017f5e5fe69b855bfe47c5bb2d5e7dcd"} Dec 03 12:30:03 crc kubenswrapper[5052]: I1203 12:30:03.661104 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/memcached-0" Dec 03 12:30:03 crc kubenswrapper[5052]: I1203 12:30:03.662664 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-worker-6d687d9c57-4gplx" event={"ID":"a65652a1-62fa-40ff-8b8a-940b2cc795b7","Type":"ContainerStarted","Data":"f18b0f1af72b5634a7b8a197502ec544d35bc31178131d0cc5f858ca0d990b27"} Dec 03 12:30:03 crc kubenswrapper[5052]: I1203 12:30:03.663024 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/barbican-api-5fc7894bfd-g7hl5" Dec 03 12:30:03 crc kubenswrapper[5052]: I1203 12:30:03.663052 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/barbican-api-5fc7894bfd-g7hl5" Dec 03 12:30:03 crc kubenswrapper[5052]: I1203 12:30:03.663108 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/barbican-api-5fc7894bfd-g7hl5" podUID="cfb8e98d-3873-43e0-81dc-4450983179c2" containerName="barbican-api" containerID="cri-o://27c4bfe7d3a13a820b6065867e281595e076068c529bea4be94b943aa7105f13" gracePeriod=30 Dec 03 12:30:03 crc kubenswrapper[5052]: I1203 12:30:03.663319 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/barbican-worker-6d687d9c57-4gplx" podUID="a65652a1-62fa-40ff-8b8a-940b2cc795b7" containerName="barbican-worker-log" containerID="cri-o://09f8bee6ba47ae352b63fc6a9643dadd544c69e4d30aec7017b6e2983d7db541" gracePeriod=30 Dec 03 12:30:03 crc kubenswrapper[5052]: I1203 12:30:03.663390 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/barbican-worker-6d687d9c57-4gplx" podUID="a65652a1-62fa-40ff-8b8a-940b2cc795b7" containerName="barbican-worker" containerID="cri-o://f18b0f1af72b5634a7b8a197502ec544d35bc31178131d0cc5f858ca0d990b27" gracePeriod=30 Dec 03 12:30:03 crc kubenswrapper[5052]: I1203 12:30:03.663789 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/keystone-564bc88b6-bvll5" Dec 03 12:30:03 crc kubenswrapper[5052]: I1203 12:30:03.662735 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/barbican-api-5fc7894bfd-g7hl5" podUID="cfb8e98d-3873-43e0-81dc-4450983179c2" containerName="barbican-api-log" containerID="cri-o://0a861967403e553bfc0bb4f31723c2962d1eb4b87dfc36853566225a16745bf9" gracePeriod=30 Dec 03 12:30:03 crc kubenswrapper[5052]: I1203 12:30:03.671674 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4314928d-a43f-4b38-9c72-63977e59345b-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"4314928d-a43f-4b38-9c72-63977e59345b\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:30:03 crc kubenswrapper[5052]: I1203 12:30:03.672345 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/4314928d-a43f-4b38-9c72-63977e59345b-config-data\") pod \"nova-api-0\" (UID: \"4314928d-a43f-4b38-9c72-63977e59345b\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:30:03 crc kubenswrapper[5052]: I1203 12:30:03.674410 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/4314928d-a43f-4b38-9c72-63977e59345b-internal-tls-certs\") pod \"nova-api-0\" (UID: \"4314928d-a43f-4b38-9c72-63977e59345b\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:30:03 crc kubenswrapper[5052]: I1203 12:30:03.676272 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/barbican-api-5fc7894bfd-g7hl5" podUID="cfb8e98d-3873-43e0-81dc-4450983179c2" containerName="barbican-api-log" probeResult="failure" output="Get \"https://10.217.0.198:9311/healthcheck\": EOF" Dec 03 12:30:03 crc kubenswrapper[5052]: I1203 12:30:03.677081 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/4314928d-a43f-4b38-9c72-63977e59345b-public-tls-certs\") pod \"nova-api-0\" (UID: \"4314928d-a43f-4b38-9c72-63977e59345b\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:30:03 crc kubenswrapper[5052]: I1203 12:30:03.692393 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/keystone-564bc88b6-bvll5" podStartSLOduration=6.692376722 podStartE2EDuration="6.692376722s" podCreationTimestamp="2025-12-03 12:29:57 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:30:03.689906211 +0000 UTC m=+4194.278147995" watchObservedRunningTime="2025-12-03 12:30:03.692376722 +0000 UTC m=+4194.280618486" Dec 03 12:30:03 crc kubenswrapper[5052]: I1203 12:30:03.716202 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-pfvmv\" (UniqueName: \"kubernetes.io/projected/4314928d-a43f-4b38-9c72-63977e59345b-kube-api-access-pfvmv\") pod \"nova-api-0\" (UID: \"4314928d-a43f-4b38-9c72-63977e59345b\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:30:03 crc kubenswrapper[5052]: I1203 12:30:03.723530 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b5b5ad16-1e45-4761-b5d2-60ffd5934ccb" path="/var/lib/kubelet/pods/b5b5ad16-1e45-4761-b5d2-60ffd5934ccb/volumes" Dec 03 12:30:03 crc kubenswrapper[5052]: I1203 12:30:03.724861 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="caaecb05-f5f1-421d-8e91-818774a2c0ec" path="/var/lib/kubelet/pods/caaecb05-f5f1-421d-8e91-818774a2c0ec/volumes" Dec 03 12:30:03 crc kubenswrapper[5052]: I1203 12:30:03.741250 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/barbican-worker-6d687d9c57-4gplx" podStartSLOduration=9.741231636 podStartE2EDuration="9.741231636s" podCreationTimestamp="2025-12-03 12:29:54 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:30:03.724285283 +0000 UTC m=+4194.312527057" watchObservedRunningTime="2025-12-03 12:30:03.741231636 +0000 UTC m=+4194.329473400" Dec 03 12:30:03 crc kubenswrapper[5052]: I1203 12:30:03.758360 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" podStartSLOduration=4.758341045 podStartE2EDuration="4.758341045s" podCreationTimestamp="2025-12-03 12:29:59 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:30:03.74449755 +0000 UTC m=+4194.332739344" watchObservedRunningTime="2025-12-03 12:30:03.758341045 +0000 UTC m=+4194.346582799" Dec 03 12:30:03 crc kubenswrapper[5052]: I1203 12:30:03.762641 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/cfd8af50-a644-4b24-8483-dd344f2218e4-logs\") pod \"placement-65444bfc86-th5pw\" (UID: \"cfd8af50-a644-4b24-8483-dd344f2218e4\") " pod="openstack-kuttl-tests/placement-65444bfc86-th5pw" Dec 03 12:30:03 crc kubenswrapper[5052]: I1203 12:30:03.762920 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/cfd8af50-a644-4b24-8483-dd344f2218e4-logs\") pod \"placement-65444bfc86-th5pw\" (UID: \"cfd8af50-a644-4b24-8483-dd344f2218e4\") " pod="openstack-kuttl-tests/placement-65444bfc86-th5pw" Dec 03 12:30:03 crc kubenswrapper[5052]: I1203 12:30:03.763087 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/cfd8af50-a644-4b24-8483-dd344f2218e4-config-data\") pod \"placement-65444bfc86-th5pw\" (UID: \"cfd8af50-a644-4b24-8483-dd344f2218e4\") " pod="openstack-kuttl-tests/placement-65444bfc86-th5pw" Dec 03 12:30:03 crc kubenswrapper[5052]: I1203 12:30:03.769807 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/cfd8af50-a644-4b24-8483-dd344f2218e4-public-tls-certs\") pod \"placement-65444bfc86-th5pw\" (UID: \"cfd8af50-a644-4b24-8483-dd344f2218e4\") " pod="openstack-kuttl-tests/placement-65444bfc86-th5pw" Dec 03 12:30:03 crc kubenswrapper[5052]: I1203 12:30:03.773918 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/memcached-0" podStartSLOduration=4.772345595 podStartE2EDuration="4.772345595s" podCreationTimestamp="2025-12-03 12:29:59 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:30:03.767064894 +0000 UTC m=+4194.355306688" watchObservedRunningTime="2025-12-03 12:30:03.772345595 +0000 UTC m=+4194.360587369" Dec 03 12:30:03 crc kubenswrapper[5052]: I1203 12:30:03.782917 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/cfd8af50-a644-4b24-8483-dd344f2218e4-internal-tls-certs\") pod \"placement-65444bfc86-th5pw\" (UID: \"cfd8af50-a644-4b24-8483-dd344f2218e4\") " pod="openstack-kuttl-tests/placement-65444bfc86-th5pw" Dec 03 12:30:03 crc kubenswrapper[5052]: I1203 12:30:03.782997 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/cfd8af50-a644-4b24-8483-dd344f2218e4-combined-ca-bundle\") pod \"placement-65444bfc86-th5pw\" (UID: \"cfd8af50-a644-4b24-8483-dd344f2218e4\") " pod="openstack-kuttl-tests/placement-65444bfc86-th5pw" Dec 03 12:30:03 crc kubenswrapper[5052]: I1203 12:30:03.783073 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-jkbfb\" (UniqueName: \"kubernetes.io/projected/cfd8af50-a644-4b24-8483-dd344f2218e4-kube-api-access-jkbfb\") pod \"placement-65444bfc86-th5pw\" (UID: \"cfd8af50-a644-4b24-8483-dd344f2218e4\") " pod="openstack-kuttl-tests/placement-65444bfc86-th5pw" Dec 03 12:30:03 crc kubenswrapper[5052]: I1203 12:30:03.784207 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/cfd8af50-a644-4b24-8483-dd344f2218e4-scripts\") pod \"placement-65444bfc86-th5pw\" (UID: \"cfd8af50-a644-4b24-8483-dd344f2218e4\") " pod="openstack-kuttl-tests/placement-65444bfc86-th5pw" Dec 03 12:30:03 crc kubenswrapper[5052]: I1203 12:30:03.786658 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/cfd8af50-a644-4b24-8483-dd344f2218e4-public-tls-certs\") pod \"placement-65444bfc86-th5pw\" (UID: \"cfd8af50-a644-4b24-8483-dd344f2218e4\") " pod="openstack-kuttl-tests/placement-65444bfc86-th5pw" Dec 03 12:30:03 crc kubenswrapper[5052]: I1203 12:30:03.792495 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/cfd8af50-a644-4b24-8483-dd344f2218e4-combined-ca-bundle\") pod \"placement-65444bfc86-th5pw\" (UID: \"cfd8af50-a644-4b24-8483-dd344f2218e4\") " pod="openstack-kuttl-tests/placement-65444bfc86-th5pw" Dec 03 12:30:03 crc kubenswrapper[5052]: I1203 12:30:03.801908 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/cfd8af50-a644-4b24-8483-dd344f2218e4-internal-tls-certs\") pod \"placement-65444bfc86-th5pw\" (UID: \"cfd8af50-a644-4b24-8483-dd344f2218e4\") " pod="openstack-kuttl-tests/placement-65444bfc86-th5pw" Dec 03 12:30:03 crc kubenswrapper[5052]: I1203 12:30:03.802523 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/cfd8af50-a644-4b24-8483-dd344f2218e4-config-data\") pod \"placement-65444bfc86-th5pw\" (UID: \"cfd8af50-a644-4b24-8483-dd344f2218e4\") " pod="openstack-kuttl-tests/placement-65444bfc86-th5pw" Dec 03 12:30:03 crc kubenswrapper[5052]: I1203 12:30:03.802532 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/cfd8af50-a644-4b24-8483-dd344f2218e4-scripts\") pod \"placement-65444bfc86-th5pw\" (UID: \"cfd8af50-a644-4b24-8483-dd344f2218e4\") " pod="openstack-kuttl-tests/placement-65444bfc86-th5pw" Dec 03 12:30:03 crc kubenswrapper[5052]: I1203 12:30:03.805288 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:30:03 crc kubenswrapper[5052]: I1203 12:30:03.820919 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-jkbfb\" (UniqueName: \"kubernetes.io/projected/cfd8af50-a644-4b24-8483-dd344f2218e4-kube-api-access-jkbfb\") pod \"placement-65444bfc86-th5pw\" (UID: \"cfd8af50-a644-4b24-8483-dd344f2218e4\") " pod="openstack-kuttl-tests/placement-65444bfc86-th5pw" Dec 03 12:30:03 crc kubenswrapper[5052]: I1203 12:30:03.822503 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/barbican-api-5fc7894bfd-g7hl5" podStartSLOduration=9.822483136 podStartE2EDuration="9.822483136s" podCreationTimestamp="2025-12-03 12:29:54 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:30:03.798024667 +0000 UTC m=+4194.386266431" watchObservedRunningTime="2025-12-03 12:30:03.822483136 +0000 UTC m=+4194.410724900" Dec 03 12:30:04 crc kubenswrapper[5052]: I1203 12:30:04.561935 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage09-crc" (UniqueName: "kubernetes.io/local-volume/local-storage09-crc") on node "crc" Dec 03 12:30:04 crc kubenswrapper[5052]: I1203 12:30:04.611920 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/82bcca5b-e2c3-43ce-aba3-eb879796dc3e-config-data" (OuterVolumeSpecName: "config-data") pod "82bcca5b-e2c3-43ce-aba3-eb879796dc3e" (UID: "82bcca5b-e2c3-43ce-aba3-eb879796dc3e"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:04 crc kubenswrapper[5052]: I1203 12:30:04.620694 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/82bcca5b-e2c3-43ce-aba3-eb879796dc3e-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:04 crc kubenswrapper[5052]: I1203 12:30:04.620729 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage09-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage09-crc\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:04 crc kubenswrapper[5052]: I1203 12:30:04.639904 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/82bcca5b-e2c3-43ce-aba3-eb879796dc3e-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "82bcca5b-e2c3-43ce-aba3-eb879796dc3e" (UID: "82bcca5b-e2c3-43ce-aba3-eb879796dc3e"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:04 crc kubenswrapper[5052]: W1203 12:30:04.668735 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod39117149_c15d_4753_9324_8a42abe5b007.slice/crio-4e54eebec6cc33b44308d0d2910a2dc282c24d4bd3abab322cc14f37bec326e1 WatchSource:0}: Error finding container 4e54eebec6cc33b44308d0d2910a2dc282c24d4bd3abab322cc14f37bec326e1: Status 404 returned error can't find the container with id 4e54eebec6cc33b44308d0d2910a2dc282c24d4bd3abab322cc14f37bec326e1 Dec 03 12:30:04 crc kubenswrapper[5052]: I1203 12:30:04.679573 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/61cad274-57bd-4993-aaa6-3e6013900d88-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "61cad274-57bd-4993-aaa6-3e6013900d88" (UID: "61cad274-57bd-4993-aaa6-3e6013900d88"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:04 crc kubenswrapper[5052]: I1203 12:30:04.692576 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/82bcca5b-e2c3-43ce-aba3-eb879796dc3e-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "82bcca5b-e2c3-43ce-aba3-eb879796dc3e" (UID: "82bcca5b-e2c3-43ce-aba3-eb879796dc3e"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:04 crc kubenswrapper[5052]: I1203 12:30:04.726610 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/61cad274-57bd-4993-aaa6-3e6013900d88-config-data" (OuterVolumeSpecName: "config-data") pod "61cad274-57bd-4993-aaa6-3e6013900d88" (UID: "61cad274-57bd-4993-aaa6-3e6013900d88"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:04 crc kubenswrapper[5052]: I1203 12:30:04.731336 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/61cad274-57bd-4993-aaa6-3e6013900d88-config-data\") pod \"61cad274-57bd-4993-aaa6-3e6013900d88\" (UID: \"61cad274-57bd-4993-aaa6-3e6013900d88\") " Dec 03 12:30:04 crc kubenswrapper[5052]: I1203 12:30:04.736279 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/61cad274-57bd-4993-aaa6-3e6013900d88-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:04 crc kubenswrapper[5052]: I1203 12:30:04.736299 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/82bcca5b-e2c3-43ce-aba3-eb879796dc3e-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:04 crc kubenswrapper[5052]: I1203 12:30:04.736308 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/82bcca5b-e2c3-43ce-aba3-eb879796dc3e-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:04 crc kubenswrapper[5052]: W1203 12:30:04.736998 5052 empty_dir.go:500] Warning: Unmount skipped because path does not exist: /var/lib/kubelet/pods/61cad274-57bd-4993-aaa6-3e6013900d88/volumes/kubernetes.io~secret/config-data Dec 03 12:30:04 crc kubenswrapper[5052]: I1203 12:30:04.737025 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/61cad274-57bd-4993-aaa6-3e6013900d88-config-data" (OuterVolumeSpecName: "config-data") pod "61cad274-57bd-4993-aaa6-3e6013900d88" (UID: "61cad274-57bd-4993-aaa6-3e6013900d88"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:04 crc kubenswrapper[5052]: I1203 12:30:04.748368 5052 generic.go:334] "Generic (PLEG): container finished" podID="16a065b5-a603-4cb3-a4c0-b36c0b92bf3f" containerID="c1a2fa107302843e2249473b85f61687344a151311126ad578b5760b51b1b60e" exitCode=0 Dec 03 12:30:04 crc kubenswrapper[5052]: I1203 12:30:04.768327 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/61cad274-57bd-4993-aaa6-3e6013900d88-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "61cad274-57bd-4993-aaa6-3e6013900d88" (UID: "61cad274-57bd-4993-aaa6-3e6013900d88"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:04 crc kubenswrapper[5052]: E1203 12:30:04.779849 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="93474533ee16033979340cc7914412b7661d56c4b5e3b0058db42692da5f5166" cmd=["/usr/local/bin/container-scripts/status_check.sh"] Dec 03 12:30:04 crc kubenswrapper[5052]: I1203 12:30:04.800726 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/61cad274-57bd-4993-aaa6-3e6013900d88-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "61cad274-57bd-4993-aaa6-3e6013900d88" (UID: "61cad274-57bd-4993-aaa6-3e6013900d88"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:04 crc kubenswrapper[5052]: E1203 12:30:04.813675 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="93474533ee16033979340cc7914412b7661d56c4b5e3b0058db42692da5f5166" cmd=["/usr/local/bin/container-scripts/status_check.sh"] Dec 03 12:30:04 crc kubenswrapper[5052]: E1203 12:30:04.821979 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="93474533ee16033979340cc7914412b7661d56c4b5e3b0058db42692da5f5166" cmd=["/usr/local/bin/container-scripts/status_check.sh"] Dec 03 12:30:04 crc kubenswrapper[5052]: E1203 12:30:04.822041 5052 prober.go:104] "Probe errored" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" probeType="Readiness" pod="openstack-kuttl-tests/ovn-northd-0" podUID="a26ed42f-71e7-4c92-8b7e-ad194358d3e5" containerName="ovn-northd" Dec 03 12:30:04 crc kubenswrapper[5052]: I1203 12:30:04.842557 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/61cad274-57bd-4993-aaa6-3e6013900d88-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:04 crc kubenswrapper[5052]: I1203 12:30:04.842592 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/61cad274-57bd-4993-aaa6-3e6013900d88-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:04 crc kubenswrapper[5052]: I1203 12:30:04.842604 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/61cad274-57bd-4993-aaa6-3e6013900d88-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:04 crc kubenswrapper[5052]: I1203 12:30:04.843912 5052 generic.go:334] "Generic (PLEG): container finished" podID="6927052c-212d-42fc-8236-b053ffc9a9d9" containerID="edaf8cd0d0bbbd5a49f586c7c2c1e7bcdb11f76c906dcb6bcd4c10118fa5950b" exitCode=0 Dec 03 12:30:04 crc kubenswrapper[5052]: I1203 12:30:04.855502 5052 generic.go:334] "Generic (PLEG): container finished" podID="733f463b-dcf8-4a61-8e4e-4bf1c042d397" containerID="fc5e4fca262b35b92bb410a508d650dc4bdca0fc0848d2a90abb1ede218d5a33" exitCode=0 Dec 03 12:30:04 crc kubenswrapper[5052]: I1203 12:30:04.896985 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/barbican-keystone-listener-cd5b58cfd-jf8vt" podStartSLOduration=8.896964313 podStartE2EDuration="8.896964313s" podCreationTimestamp="2025-12-03 12:29:56 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:30:04.846700898 +0000 UTC m=+4195.434942662" watchObservedRunningTime="2025-12-03 12:30:04.896964313 +0000 UTC m=+4195.485206077" Dec 03 12:30:04 crc kubenswrapper[5052]: I1203 12:30:04.914990 5052 generic.go:334] "Generic (PLEG): container finished" podID="aed41b09-319e-466c-9eb9-38208e1880a3" containerID="c4632c9d758da0d71f63ae0b7ef4c62003466443e6cb1e897bae7412a5ae794f" exitCode=143 Dec 03 12:30:04 crc kubenswrapper[5052]: I1203 12:30:04.962404 5052 generic.go:334] "Generic (PLEG): container finished" podID="cfb8e98d-3873-43e0-81dc-4450983179c2" containerID="27c4bfe7d3a13a820b6065867e281595e076068c529bea4be94b943aa7105f13" exitCode=0 Dec 03 12:30:04 crc kubenswrapper[5052]: I1203 12:30:04.962444 5052 generic.go:334] "Generic (PLEG): container finished" podID="cfb8e98d-3873-43e0-81dc-4450983179c2" containerID="0a861967403e553bfc0bb4f31723c2962d1eb4b87dfc36853566225a16745bf9" exitCode=143 Dec 03 12:30:05 crc kubenswrapper[5052]: I1203 12:30:05.029161 5052 generic.go:334] "Generic (PLEG): container finished" podID="a65652a1-62fa-40ff-8b8a-940b2cc795b7" containerID="09f8bee6ba47ae352b63fc6a9643dadd544c69e4d30aec7017b6e2983d7db541" exitCode=143 Dec 03 12:30:05 crc kubenswrapper[5052]: I1203 12:30:05.117313 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/neutron-56f9c7c6b6-kvxpr" podStartSLOduration=8.117294981 podStartE2EDuration="8.117294981s" podCreationTimestamp="2025-12-03 12:29:57 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:30:05.051452002 +0000 UTC m=+4195.639693766" watchObservedRunningTime="2025-12-03 12:30:05.117294981 +0000 UTC m=+4195.705536745" Dec 03 12:30:05 crc kubenswrapper[5052]: I1203 12:30:05.477636 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/barbican-api-5fc7894bfd-g7hl5" podUID="cfb8e98d-3873-43e0-81dc-4450983179c2" containerName="barbican-api-log" probeResult="failure" output="Get \"https://10.217.0.198:9311/healthcheck\": dial tcp 10.217.0.198:9311: connect: connection refused" Dec 03 12:30:06 crc kubenswrapper[5052]: E1203 12:30:06.180024 5052 kubelet.go:2526] "Housekeeping took longer than expected" err="housekeeping took too long" expected="1s" actual="2.508s" Dec 03 12:30:06 crc kubenswrapper[5052]: I1203 12:30:06.180380 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:30:06 crc kubenswrapper[5052]: I1203 12:30:06.180399 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/neutron-56f9c7c6b6-kvxpr" Dec 03 12:30:06 crc kubenswrapper[5052]: I1203 12:30:06.180409 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-conductor-0" event={"ID":"16a065b5-a603-4cb3-a4c0-b36c0b92bf3f","Type":"ContainerDied","Data":"c1a2fa107302843e2249473b85f61687344a151311126ad578b5760b51b1b60e"} Dec 03 12:30:06 crc kubenswrapper[5052]: I1203 12:30:06.197795 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:30:06 crc kubenswrapper[5052]: I1203 12:30:06.202720 5052 scope.go:117] "RemoveContainer" containerID="7a324498f8dc1fb392c84dac59b70dd32b53652f4626455f3411fdb91b9fb201" Dec 03 12:30:06 crc kubenswrapper[5052]: I1203 12:30:06.203104 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-688bd8fb5d-pjs4f" event={"ID":"a0e45e1a-ced9-4958-9a75-24a69280f679","Type":"ContainerStarted","Data":"54d422bce70ace0e85e1e00918a25a268dd0d21415c0068651ee5074885c6030"} Dec 03 12:30:06 crc kubenswrapper[5052]: I1203 12:30:06.203142 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"39117149-c15d-4753-9324-8a42abe5b007","Type":"ContainerStarted","Data":"4e54eebec6cc33b44308d0d2910a2dc282c24d4bd3abab322cc14f37bec326e1"} Dec 03 12:30:06 crc kubenswrapper[5052]: I1203 12:30:06.203158 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"e75eea2c-2eb1-4f0f-97e5-63ce147e68dd","Type":"ContainerDied","Data":"127ac4faa7dc9b9d18ac8577cb23acb88c76ba657840c8b9ce67ac14e97a34a9"} Dec 03 12:30:06 crc kubenswrapper[5052]: I1203 12:30:06.203175 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="127ac4faa7dc9b9d18ac8577cb23acb88c76ba657840c8b9ce67ac14e97a34a9" Dec 03 12:30:06 crc kubenswrapper[5052]: I1203 12:30:06.203192 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-keystone-listener-cd5b58cfd-jf8vt" event={"ID":"019689d4-d40b-4e9a-a028-e427f1aa9d9b","Type":"ContainerStarted","Data":"a988e23bbce42676cbdcb4abb64e1c8911c6df2c4d18faf34f5c6450d0879857"} Dec 03 12:30:06 crc kubenswrapper[5052]: I1203 12:30:06.203213 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:30:06 crc kubenswrapper[5052]: I1203 12:30:06.203228 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-7cbf547d94-9krth" event={"ID":"6927052c-212d-42fc-8236-b053ffc9a9d9","Type":"ContainerDied","Data":"edaf8cd0d0bbbd5a49f586c7c2c1e7bcdb11f76c906dcb6bcd4c10118fa5950b"} Dec 03 12:30:06 crc kubenswrapper[5052]: I1203 12:30:06.203246 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:30:06 crc kubenswrapper[5052]: I1203 12:30:06.203260 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-conductor-0" event={"ID":"733f463b-dcf8-4a61-8e4e-4bf1c042d397","Type":"ContainerDied","Data":"fc5e4fca262b35b92bb410a508d650dc4bdca0fc0848d2a90abb1ede218d5a33"} Dec 03 12:30:06 crc kubenswrapper[5052]: I1203 12:30:06.203282 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"05b9d3cc-93de-45a6-8849-027a4aad3715","Type":"ContainerDied","Data":"171b29e4c8764e7231cf3eb42f1982e93714f46c393ac0a499a2ee8c10aad697"} Dec 03 12:30:06 crc kubenswrapper[5052]: I1203 12:30:06.203298 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-keystone-listener-77b6bbbdcb-4jlsc" event={"ID":"aed41b09-319e-466c-9eb9-38208e1880a3","Type":"ContainerDied","Data":"c4632c9d758da0d71f63ae0b7ef4c62003466443e6cb1e897bae7412a5ae794f"} Dec 03 12:30:06 crc kubenswrapper[5052]: I1203 12:30:06.203314 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"b953ffa0-088c-4450-8d60-04044b33cfa0","Type":"ContainerDied","Data":"6a7eedcc94f5e157a4977ec0b9a950f9234fa3da67edb3f87d442887fe9bd7f0"} Dec 03 12:30:06 crc kubenswrapper[5052]: I1203 12:30:06.203328 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6a7eedcc94f5e157a4977ec0b9a950f9234fa3da67edb3f87d442887fe9bd7f0" Dec 03 12:30:06 crc kubenswrapper[5052]: I1203 12:30:06.203338 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-5fc7894bfd-g7hl5" event={"ID":"cfb8e98d-3873-43e0-81dc-4450983179c2","Type":"ContainerDied","Data":"27c4bfe7d3a13a820b6065867e281595e076068c529bea4be94b943aa7105f13"} Dec 03 12:30:06 crc kubenswrapper[5052]: I1203 12:30:06.203353 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-5fc7894bfd-g7hl5" event={"ID":"cfb8e98d-3873-43e0-81dc-4450983179c2","Type":"ContainerDied","Data":"0a861967403e553bfc0bb4f31723c2962d1eb4b87dfc36853566225a16745bf9"} Dec 03 12:30:06 crc kubenswrapper[5052]: I1203 12:30:06.203368 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-keystone-listener-58b6cdf6bb-5xxd8"] Dec 03 12:30:06 crc kubenswrapper[5052]: I1203 12:30:06.203386 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-56f9c7c6b6-kvxpr" event={"ID":"5f522d90-036e-4565-8293-97097912fbb1","Type":"ContainerStarted","Data":"091f1e3cedc375760a2fcdc8a6147bfa5ecc673f83f421f7c9f903fa52ed8e8d"} Dec 03 12:30:06 crc kubenswrapper[5052]: I1203 12:30:06.203401 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-worker-6d687d9c57-4gplx" event={"ID":"a65652a1-62fa-40ff-8b8a-940b2cc795b7","Type":"ContainerDied","Data":"09f8bee6ba47ae352b63fc6a9643dadd544c69e4d30aec7017b6e2983d7db541"} Dec 03 12:30:06 crc kubenswrapper[5052]: I1203 12:30:06.203416 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-conductor-0" event={"ID":"733f463b-dcf8-4a61-8e4e-4bf1c042d397","Type":"ContainerDied","Data":"9929deb72ae6d5d35726a21123b7bb836c354c04102b48a8e5d3f5f7222a1ff0"} Dec 03 12:30:06 crc kubenswrapper[5052]: I1203 12:30:06.203427 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9929deb72ae6d5d35726a21123b7bb836c354c04102b48a8e5d3f5f7222a1ff0" Dec 03 12:30:06 crc kubenswrapper[5052]: I1203 12:30:06.203438 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-conductor-0" event={"ID":"16a065b5-a603-4cb3-a4c0-b36c0b92bf3f","Type":"ContainerDied","Data":"5189538592fa7792bfe6f41aa547a24c4cedd88ab25440d8ee94c398f4e1b743"} Dec 03 12:30:06 crc kubenswrapper[5052]: I1203 12:30:06.203450 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5189538592fa7792bfe6f41aa547a24c4cedd88ab25440d8ee94c398f4e1b743" Dec 03 12:30:06 crc kubenswrapper[5052]: I1203 12:30:06.203462 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-ff6b8c44b-7ndgm" event={"ID":"48831cad-910f-46f1-aefe-7cc4cbfa984f","Type":"ContainerStarted","Data":"f6c28d3863e0d0d300bbffab66b3c5240cbeaa832b51d4164b9f0b3c092d3831"} Dec 03 12:30:06 crc kubenswrapper[5052]: I1203 12:30:06.203480 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-db474c7dd-tg7mq" event={"ID":"886ea0a4-5096-4ea8-a74c-de243d1f4f40","Type":"ContainerStarted","Data":"71e595450c5a81e73076dfa57785d77ddaafc47e00972e087f567c2fa539835b"} Dec 03 12:30:06 crc kubenswrapper[5052]: I1203 12:30:06.203494 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"4314928d-a43f-4b38-9c72-63977e59345b","Type":"ContainerStarted","Data":"296ee69d2ec970208a8577c0e6c144ec5511a61fc7b242cf5d81b2653243ce31"} Dec 03 12:30:06 crc kubenswrapper[5052]: I1203 12:30:06.203506 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-sb-0" event={"ID":"50f5af82-8ee5-45f4-b158-2166f1be6cb8","Type":"ContainerStarted","Data":"e9076e00de7ae30a9d54e2e79b337633b061bc7f8a01e8aa825185ef328777b0"} Dec 03 12:30:06 crc kubenswrapper[5052]: I1203 12:30:06.269468 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:30:06 crc kubenswrapper[5052]: I1203 12:30:06.281970 5052 scope.go:117] "RemoveContainer" containerID="fa45d03e94f8aa43afec4ca3932bb3d4299d6fe6674927e304720e875b1c73d2" Dec 03 12:30:06 crc kubenswrapper[5052]: I1203 12:30:06.308849 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/05b9d3cc-93de-45a6-8849-027a4aad3715-logs\") pod \"05b9d3cc-93de-45a6-8849-027a4aad3715\" (UID: \"05b9d3cc-93de-45a6-8849-027a4aad3715\") " Dec 03 12:30:06 crc kubenswrapper[5052]: I1203 12:30:06.309006 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/05b9d3cc-93de-45a6-8849-027a4aad3715-httpd-run\") pod \"05b9d3cc-93de-45a6-8849-027a4aad3715\" (UID: \"05b9d3cc-93de-45a6-8849-027a4aad3715\") " Dec 03 12:30:06 crc kubenswrapper[5052]: I1203 12:30:06.309030 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/e75eea2c-2eb1-4f0f-97e5-63ce147e68dd-log-httpd\") pod \"e75eea2c-2eb1-4f0f-97e5-63ce147e68dd\" (UID: \"e75eea2c-2eb1-4f0f-97e5-63ce147e68dd\") " Dec 03 12:30:06 crc kubenswrapper[5052]: I1203 12:30:06.309051 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/05b9d3cc-93de-45a6-8849-027a4aad3715-scripts\") pod \"05b9d3cc-93de-45a6-8849-027a4aad3715\" (UID: \"05b9d3cc-93de-45a6-8849-027a4aad3715\") " Dec 03 12:30:06 crc kubenswrapper[5052]: I1203 12:30:06.309081 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xvh8g\" (UniqueName: \"kubernetes.io/projected/05b9d3cc-93de-45a6-8849-027a4aad3715-kube-api-access-xvh8g\") pod \"05b9d3cc-93de-45a6-8849-027a4aad3715\" (UID: \"05b9d3cc-93de-45a6-8849-027a4aad3715\") " Dec 03 12:30:06 crc kubenswrapper[5052]: I1203 12:30:06.309105 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/05b9d3cc-93de-45a6-8849-027a4aad3715-combined-ca-bundle\") pod \"05b9d3cc-93de-45a6-8849-027a4aad3715\" (UID: \"05b9d3cc-93de-45a6-8849-027a4aad3715\") " Dec 03 12:30:06 crc kubenswrapper[5052]: I1203 12:30:06.309126 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/05b9d3cc-93de-45a6-8849-027a4aad3715-public-tls-certs\") pod \"05b9d3cc-93de-45a6-8849-027a4aad3715\" (UID: \"05b9d3cc-93de-45a6-8849-027a4aad3715\") " Dec 03 12:30:06 crc kubenswrapper[5052]: I1203 12:30:06.309375 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/05b9d3cc-93de-45a6-8849-027a4aad3715-logs" (OuterVolumeSpecName: "logs") pod "05b9d3cc-93de-45a6-8849-027a4aad3715" (UID: "05b9d3cc-93de-45a6-8849-027a4aad3715"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:30:06 crc kubenswrapper[5052]: I1203 12:30:06.309521 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e75eea2c-2eb1-4f0f-97e5-63ce147e68dd-combined-ca-bundle\") pod \"e75eea2c-2eb1-4f0f-97e5-63ce147e68dd\" (UID: \"e75eea2c-2eb1-4f0f-97e5-63ce147e68dd\") " Dec 03 12:30:06 crc kubenswrapper[5052]: I1203 12:30:06.309768 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/e75eea2c-2eb1-4f0f-97e5-63ce147e68dd-run-httpd\") pod \"e75eea2c-2eb1-4f0f-97e5-63ce147e68dd\" (UID: \"e75eea2c-2eb1-4f0f-97e5-63ce147e68dd\") " Dec 03 12:30:06 crc kubenswrapper[5052]: I1203 12:30:06.310025 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/e75eea2c-2eb1-4f0f-97e5-63ce147e68dd-config-data\") pod \"e75eea2c-2eb1-4f0f-97e5-63ce147e68dd\" (UID: \"e75eea2c-2eb1-4f0f-97e5-63ce147e68dd\") " Dec 03 12:30:06 crc kubenswrapper[5052]: I1203 12:30:06.310145 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/e75eea2c-2eb1-4f0f-97e5-63ce147e68dd-scripts\") pod \"e75eea2c-2eb1-4f0f-97e5-63ce147e68dd\" (UID: \"e75eea2c-2eb1-4f0f-97e5-63ce147e68dd\") " Dec 03 12:30:06 crc kubenswrapper[5052]: I1203 12:30:06.310253 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/05b9d3cc-93de-45a6-8849-027a4aad3715-config-data\") pod \"05b9d3cc-93de-45a6-8849-027a4aad3715\" (UID: \"05b9d3cc-93de-45a6-8849-027a4aad3715\") " Dec 03 12:30:06 crc kubenswrapper[5052]: I1203 12:30:06.310374 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gh476\" (UniqueName: \"kubernetes.io/projected/e75eea2c-2eb1-4f0f-97e5-63ce147e68dd-kube-api-access-gh476\") pod \"e75eea2c-2eb1-4f0f-97e5-63ce147e68dd\" (UID: \"e75eea2c-2eb1-4f0f-97e5-63ce147e68dd\") " Dec 03 12:30:06 crc kubenswrapper[5052]: I1203 12:30:06.310494 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"glance\" (UniqueName: \"kubernetes.io/local-volume/local-storage07-crc\") pod \"05b9d3cc-93de-45a6-8849-027a4aad3715\" (UID: \"05b9d3cc-93de-45a6-8849-027a4aad3715\") " Dec 03 12:30:06 crc kubenswrapper[5052]: I1203 12:30:06.310623 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/e75eea2c-2eb1-4f0f-97e5-63ce147e68dd-sg-core-conf-yaml\") pod \"e75eea2c-2eb1-4f0f-97e5-63ce147e68dd\" (UID: \"e75eea2c-2eb1-4f0f-97e5-63ce147e68dd\") " Dec 03 12:30:06 crc kubenswrapper[5052]: I1203 12:30:06.310735 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/e75eea2c-2eb1-4f0f-97e5-63ce147e68dd-ceilometer-tls-certs\") pod \"e75eea2c-2eb1-4f0f-97e5-63ce147e68dd\" (UID: \"e75eea2c-2eb1-4f0f-97e5-63ce147e68dd\") " Dec 03 12:30:06 crc kubenswrapper[5052]: I1203 12:30:06.311895 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/05b9d3cc-93de-45a6-8849-027a4aad3715-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:06 crc kubenswrapper[5052]: I1203 12:30:06.309951 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/05b9d3cc-93de-45a6-8849-027a4aad3715-httpd-run" (OuterVolumeSpecName: "httpd-run") pod "05b9d3cc-93de-45a6-8849-027a4aad3715" (UID: "05b9d3cc-93de-45a6-8849-027a4aad3715"). InnerVolumeSpecName "httpd-run". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:30:06 crc kubenswrapper[5052]: I1203 12:30:06.310382 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e75eea2c-2eb1-4f0f-97e5-63ce147e68dd-run-httpd" (OuterVolumeSpecName: "run-httpd") pod "e75eea2c-2eb1-4f0f-97e5-63ce147e68dd" (UID: "e75eea2c-2eb1-4f0f-97e5-63ce147e68dd"). InnerVolumeSpecName "run-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:30:06 crc kubenswrapper[5052]: I1203 12:30:06.312787 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e75eea2c-2eb1-4f0f-97e5-63ce147e68dd-log-httpd" (OuterVolumeSpecName: "log-httpd") pod "e75eea2c-2eb1-4f0f-97e5-63ce147e68dd" (UID: "e75eea2c-2eb1-4f0f-97e5-63ce147e68dd"). InnerVolumeSpecName "log-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:30:06 crc kubenswrapper[5052]: I1203 12:30:06.333635 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e75eea2c-2eb1-4f0f-97e5-63ce147e68dd-scripts" (OuterVolumeSpecName: "scripts") pod "e75eea2c-2eb1-4f0f-97e5-63ce147e68dd" (UID: "e75eea2c-2eb1-4f0f-97e5-63ce147e68dd"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:06 crc kubenswrapper[5052]: I1203 12:30:06.344892 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e75eea2c-2eb1-4f0f-97e5-63ce147e68dd-kube-api-access-gh476" (OuterVolumeSpecName: "kube-api-access-gh476") pod "e75eea2c-2eb1-4f0f-97e5-63ce147e68dd" (UID: "e75eea2c-2eb1-4f0f-97e5-63ce147e68dd"). InnerVolumeSpecName "kube-api-access-gh476". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:30:06 crc kubenswrapper[5052]: I1203 12:30:06.346084 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/05b9d3cc-93de-45a6-8849-027a4aad3715-scripts" (OuterVolumeSpecName: "scripts") pod "05b9d3cc-93de-45a6-8849-027a4aad3715" (UID: "05b9d3cc-93de-45a6-8849-027a4aad3715"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:06 crc kubenswrapper[5052]: I1203 12:30:06.355894 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage07-crc" (OuterVolumeSpecName: "glance") pod "05b9d3cc-93de-45a6-8849-027a4aad3715" (UID: "05b9d3cc-93de-45a6-8849-027a4aad3715"). InnerVolumeSpecName "local-storage07-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 12:30:06 crc kubenswrapper[5052]: I1203 12:30:06.360290 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/05b9d3cc-93de-45a6-8849-027a4aad3715-kube-api-access-xvh8g" (OuterVolumeSpecName: "kube-api-access-xvh8g") pod "05b9d3cc-93de-45a6-8849-027a4aad3715" (UID: "05b9d3cc-93de-45a6-8849-027a4aad3715"). InnerVolumeSpecName "kube-api-access-xvh8g". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:30:06 crc kubenswrapper[5052]: I1203 12:30:06.416944 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-gh476\" (UniqueName: \"kubernetes.io/projected/e75eea2c-2eb1-4f0f-97e5-63ce147e68dd-kube-api-access-gh476\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:06 crc kubenswrapper[5052]: I1203 12:30:06.416983 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage07-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage07-crc\") on node \"crc\" " Dec 03 12:30:06 crc kubenswrapper[5052]: I1203 12:30:06.416994 5052 reconciler_common.go:293] "Volume detached for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/05b9d3cc-93de-45a6-8849-027a4aad3715-httpd-run\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:06 crc kubenswrapper[5052]: I1203 12:30:06.417002 5052 reconciler_common.go:293] "Volume detached for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/e75eea2c-2eb1-4f0f-97e5-63ce147e68dd-log-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:06 crc kubenswrapper[5052]: I1203 12:30:06.417011 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/05b9d3cc-93de-45a6-8849-027a4aad3715-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:06 crc kubenswrapper[5052]: I1203 12:30:06.417022 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-xvh8g\" (UniqueName: \"kubernetes.io/projected/05b9d3cc-93de-45a6-8849-027a4aad3715-kube-api-access-xvh8g\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:06 crc kubenswrapper[5052]: I1203 12:30:06.417031 5052 reconciler_common.go:293] "Volume detached for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/e75eea2c-2eb1-4f0f-97e5-63ce147e68dd-run-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:06 crc kubenswrapper[5052]: I1203 12:30:06.417039 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/e75eea2c-2eb1-4f0f-97e5-63ce147e68dd-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:06 crc kubenswrapper[5052]: I1203 12:30:06.425855 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e75eea2c-2eb1-4f0f-97e5-63ce147e68dd-sg-core-conf-yaml" (OuterVolumeSpecName: "sg-core-conf-yaml") pod "e75eea2c-2eb1-4f0f-97e5-63ce147e68dd" (UID: "e75eea2c-2eb1-4f0f-97e5-63ce147e68dd"). InnerVolumeSpecName "sg-core-conf-yaml". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:06 crc kubenswrapper[5052]: I1203 12:30:06.442685 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/05b9d3cc-93de-45a6-8849-027a4aad3715-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "05b9d3cc-93de-45a6-8849-027a4aad3715" (UID: "05b9d3cc-93de-45a6-8849-027a4aad3715"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:06 crc kubenswrapper[5052]: I1203 12:30:06.465462 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/05b9d3cc-93de-45a6-8849-027a4aad3715-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "05b9d3cc-93de-45a6-8849-027a4aad3715" (UID: "05b9d3cc-93de-45a6-8849-027a4aad3715"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:06 crc kubenswrapper[5052]: I1203 12:30:06.467055 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage07-crc" (UniqueName: "kubernetes.io/local-volume/local-storage07-crc") on node "crc" Dec 03 12:30:06 crc kubenswrapper[5052]: I1203 12:30:06.502045 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e75eea2c-2eb1-4f0f-97e5-63ce147e68dd-ceilometer-tls-certs" (OuterVolumeSpecName: "ceilometer-tls-certs") pod "e75eea2c-2eb1-4f0f-97e5-63ce147e68dd" (UID: "e75eea2c-2eb1-4f0f-97e5-63ce147e68dd"). InnerVolumeSpecName "ceilometer-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:06 crc kubenswrapper[5052]: I1203 12:30:06.503269 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/05b9d3cc-93de-45a6-8849-027a4aad3715-config-data" (OuterVolumeSpecName: "config-data") pod "05b9d3cc-93de-45a6-8849-027a4aad3715" (UID: "05b9d3cc-93de-45a6-8849-027a4aad3715"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:06 crc kubenswrapper[5052]: I1203 12:30:06.504084 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e75eea2c-2eb1-4f0f-97e5-63ce147e68dd-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "e75eea2c-2eb1-4f0f-97e5-63ce147e68dd" (UID: "e75eea2c-2eb1-4f0f-97e5-63ce147e68dd"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:06 crc kubenswrapper[5052]: I1203 12:30:06.518875 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/05b9d3cc-93de-45a6-8849-027a4aad3715-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:06 crc kubenswrapper[5052]: I1203 12:30:06.518912 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage07-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage07-crc\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:06 crc kubenswrapper[5052]: I1203 12:30:06.518924 5052 reconciler_common.go:293] "Volume detached for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/e75eea2c-2eb1-4f0f-97e5-63ce147e68dd-sg-core-conf-yaml\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:06 crc kubenswrapper[5052]: I1203 12:30:06.518936 5052 reconciler_common.go:293] "Volume detached for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/e75eea2c-2eb1-4f0f-97e5-63ce147e68dd-ceilometer-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:06 crc kubenswrapper[5052]: I1203 12:30:06.518980 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/05b9d3cc-93de-45a6-8849-027a4aad3715-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:06 crc kubenswrapper[5052]: I1203 12:30:06.519011 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/05b9d3cc-93de-45a6-8849-027a4aad3715-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:06 crc kubenswrapper[5052]: I1203 12:30:06.519022 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e75eea2c-2eb1-4f0f-97e5-63ce147e68dd-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:06 crc kubenswrapper[5052]: I1203 12:30:06.538276 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e75eea2c-2eb1-4f0f-97e5-63ce147e68dd-config-data" (OuterVolumeSpecName: "config-data") pod "e75eea2c-2eb1-4f0f-97e5-63ce147e68dd" (UID: "e75eea2c-2eb1-4f0f-97e5-63ce147e68dd"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:06 crc kubenswrapper[5052]: I1203 12:30:06.552733 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:30:06 crc kubenswrapper[5052]: I1203 12:30:06.556139 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-65444bfc86-th5pw" Dec 03 12:30:06 crc kubenswrapper[5052]: I1203 12:30:06.568530 5052 scope.go:117] "RemoveContainer" containerID="75b9c75f04326f0c84c32e2979b31896339503f5d49a49fbaac6f84c100d77b0" Dec 03 12:30:06 crc kubenswrapper[5052]: I1203 12:30:06.620638 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b953ffa0-088c-4450-8d60-04044b33cfa0-combined-ca-bundle\") pod \"b953ffa0-088c-4450-8d60-04044b33cfa0\" (UID: \"b953ffa0-088c-4450-8d60-04044b33cfa0\") " Dec 03 12:30:06 crc kubenswrapper[5052]: I1203 12:30:06.621402 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bjgf2\" (UniqueName: \"kubernetes.io/projected/b953ffa0-088c-4450-8d60-04044b33cfa0-kube-api-access-bjgf2\") pod \"b953ffa0-088c-4450-8d60-04044b33cfa0\" (UID: \"b953ffa0-088c-4450-8d60-04044b33cfa0\") " Dec 03 12:30:06 crc kubenswrapper[5052]: I1203 12:30:06.621475 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/b953ffa0-088c-4450-8d60-04044b33cfa0-config-data\") pod \"b953ffa0-088c-4450-8d60-04044b33cfa0\" (UID: \"b953ffa0-088c-4450-8d60-04044b33cfa0\") " Dec 03 12:30:06 crc kubenswrapper[5052]: I1203 12:30:06.622266 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/e75eea2c-2eb1-4f0f-97e5-63ce147e68dd-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:06 crc kubenswrapper[5052]: I1203 12:30:06.628546 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b953ffa0-088c-4450-8d60-04044b33cfa0-kube-api-access-bjgf2" (OuterVolumeSpecName: "kube-api-access-bjgf2") pod "b953ffa0-088c-4450-8d60-04044b33cfa0" (UID: "b953ffa0-088c-4450-8d60-04044b33cfa0"). InnerVolumeSpecName "kube-api-access-bjgf2". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:30:06 crc kubenswrapper[5052]: I1203 12:30:06.633109 5052 scope.go:117] "RemoveContainer" containerID="2f2f681ab4e070f1a45adfa2327a6b72d7c87fcccf7bb015c88103e25b6472d9" Dec 03 12:30:06 crc kubenswrapper[5052]: I1203 12:30:06.665784 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:30:06 crc kubenswrapper[5052]: I1203 12:30:06.674179 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:30:06 crc kubenswrapper[5052]: I1203 12:30:06.710128 5052 scope.go:117] "RemoveContainer" containerID="07f1af7d6a787079787ad912bf9e1ca56fb4cb7d8e231d7b665fb887db97be81" Dec 03 12:30:06 crc kubenswrapper[5052]: I1203 12:30:06.725688 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/16a065b5-a603-4cb3-a4c0-b36c0b92bf3f-config-data\") pod \"16a065b5-a603-4cb3-a4c0-b36c0b92bf3f\" (UID: \"16a065b5-a603-4cb3-a4c0-b36c0b92bf3f\") " Dec 03 12:30:06 crc kubenswrapper[5052]: I1203 12:30:06.725759 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/733f463b-dcf8-4a61-8e4e-4bf1c042d397-config-data\") pod \"733f463b-dcf8-4a61-8e4e-4bf1c042d397\" (UID: \"733f463b-dcf8-4a61-8e4e-4bf1c042d397\") " Dec 03 12:30:06 crc kubenswrapper[5052]: I1203 12:30:06.726003 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/733f463b-dcf8-4a61-8e4e-4bf1c042d397-combined-ca-bundle\") pod \"733f463b-dcf8-4a61-8e4e-4bf1c042d397\" (UID: \"733f463b-dcf8-4a61-8e4e-4bf1c042d397\") " Dec 03 12:30:06 crc kubenswrapper[5052]: I1203 12:30:06.726048 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7x9gt\" (UniqueName: \"kubernetes.io/projected/16a065b5-a603-4cb3-a4c0-b36c0b92bf3f-kube-api-access-7x9gt\") pod \"16a065b5-a603-4cb3-a4c0-b36c0b92bf3f\" (UID: \"16a065b5-a603-4cb3-a4c0-b36c0b92bf3f\") " Dec 03 12:30:06 crc kubenswrapper[5052]: I1203 12:30:06.726081 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/16a065b5-a603-4cb3-a4c0-b36c0b92bf3f-combined-ca-bundle\") pod \"16a065b5-a603-4cb3-a4c0-b36c0b92bf3f\" (UID: \"16a065b5-a603-4cb3-a4c0-b36c0b92bf3f\") " Dec 03 12:30:06 crc kubenswrapper[5052]: I1203 12:30:06.726121 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zwjfg\" (UniqueName: \"kubernetes.io/projected/733f463b-dcf8-4a61-8e4e-4bf1c042d397-kube-api-access-zwjfg\") pod \"733f463b-dcf8-4a61-8e4e-4bf1c042d397\" (UID: \"733f463b-dcf8-4a61-8e4e-4bf1c042d397\") " Dec 03 12:30:06 crc kubenswrapper[5052]: I1203 12:30:06.727140 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-bjgf2\" (UniqueName: \"kubernetes.io/projected/b953ffa0-088c-4450-8d60-04044b33cfa0-kube-api-access-bjgf2\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:06 crc kubenswrapper[5052]: I1203 12:30:06.732762 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/16a065b5-a603-4cb3-a4c0-b36c0b92bf3f-kube-api-access-7x9gt" (OuterVolumeSpecName: "kube-api-access-7x9gt") pod "16a065b5-a603-4cb3-a4c0-b36c0b92bf3f" (UID: "16a065b5-a603-4cb3-a4c0-b36c0b92bf3f"). InnerVolumeSpecName "kube-api-access-7x9gt". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:30:06 crc kubenswrapper[5052]: I1203 12:30:06.735720 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-7cbf547d94-9krth" Dec 03 12:30:06 crc kubenswrapper[5052]: I1203 12:30:06.741152 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-api-5fc7894bfd-g7hl5" Dec 03 12:30:06 crc kubenswrapper[5052]: I1203 12:30:06.744487 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-operator-lifecycle-manager/collect-profiles-29412750-42nq6" Dec 03 12:30:06 crc kubenswrapper[5052]: I1203 12:30:06.815494 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/733f463b-dcf8-4a61-8e4e-4bf1c042d397-kube-api-access-zwjfg" (OuterVolumeSpecName: "kube-api-access-zwjfg") pod "733f463b-dcf8-4a61-8e4e-4bf1c042d397" (UID: "733f463b-dcf8-4a61-8e4e-4bf1c042d397"). InnerVolumeSpecName "kube-api-access-zwjfg". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:30:06 crc kubenswrapper[5052]: I1203 12:30:06.829733 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/cfb8e98d-3873-43e0-81dc-4450983179c2-combined-ca-bundle\") pod \"cfb8e98d-3873-43e0-81dc-4450983179c2\" (UID: \"cfb8e98d-3873-43e0-81dc-4450983179c2\") " Dec 03 12:30:06 crc kubenswrapper[5052]: I1203 12:30:06.829811 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/cfb8e98d-3873-43e0-81dc-4450983179c2-internal-tls-certs\") pod \"cfb8e98d-3873-43e0-81dc-4450983179c2\" (UID: \"cfb8e98d-3873-43e0-81dc-4450983179c2\") " Dec 03 12:30:06 crc kubenswrapper[5052]: I1203 12:30:06.829851 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gvz7n\" (UniqueName: \"kubernetes.io/projected/6927052c-212d-42fc-8236-b053ffc9a9d9-kube-api-access-gvz7n\") pod \"6927052c-212d-42fc-8236-b053ffc9a9d9\" (UID: \"6927052c-212d-42fc-8236-b053ffc9a9d9\") " Dec 03 12:30:06 crc kubenswrapper[5052]: I1203 12:30:06.829896 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6927052c-212d-42fc-8236-b053ffc9a9d9-combined-ca-bundle\") pod \"6927052c-212d-42fc-8236-b053ffc9a9d9\" (UID: \"6927052c-212d-42fc-8236-b053ffc9a9d9\") " Dec 03 12:30:06 crc kubenswrapper[5052]: I1203 12:30:06.829928 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8sxqm\" (UniqueName: \"kubernetes.io/projected/cfb8e98d-3873-43e0-81dc-4450983179c2-kube-api-access-8sxqm\") pod \"cfb8e98d-3873-43e0-81dc-4450983179c2\" (UID: \"cfb8e98d-3873-43e0-81dc-4450983179c2\") " Dec 03 12:30:06 crc kubenswrapper[5052]: I1203 12:30:06.829967 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/6927052c-212d-42fc-8236-b053ffc9a9d9-internal-tls-certs\") pod \"6927052c-212d-42fc-8236-b053ffc9a9d9\" (UID: \"6927052c-212d-42fc-8236-b053ffc9a9d9\") " Dec 03 12:30:06 crc kubenswrapper[5052]: I1203 12:30:06.830021 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/cfb8e98d-3873-43e0-81dc-4450983179c2-config-data-custom\") pod \"cfb8e98d-3873-43e0-81dc-4450983179c2\" (UID: \"cfb8e98d-3873-43e0-81dc-4450983179c2\") " Dec 03 12:30:06 crc kubenswrapper[5052]: I1203 12:30:06.830058 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/6927052c-212d-42fc-8236-b053ffc9a9d9-scripts\") pod \"6927052c-212d-42fc-8236-b053ffc9a9d9\" (UID: \"6927052c-212d-42fc-8236-b053ffc9a9d9\") " Dec 03 12:30:06 crc kubenswrapper[5052]: I1203 12:30:06.830090 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"secret-volume\" (UniqueName: \"kubernetes.io/secret/6a77ebe2-ac70-4620-acef-bda957db5766-secret-volume\") pod \"6a77ebe2-ac70-4620-acef-bda957db5766\" (UID: \"6a77ebe2-ac70-4620-acef-bda957db5766\") " Dec 03 12:30:06 crc kubenswrapper[5052]: I1203 12:30:06.830131 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/6927052c-212d-42fc-8236-b053ffc9a9d9-fernet-keys\") pod \"6927052c-212d-42fc-8236-b053ffc9a9d9\" (UID: \"6927052c-212d-42fc-8236-b053ffc9a9d9\") " Dec 03 12:30:06 crc kubenswrapper[5052]: I1203 12:30:06.830153 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/cfb8e98d-3873-43e0-81dc-4450983179c2-public-tls-certs\") pod \"cfb8e98d-3873-43e0-81dc-4450983179c2\" (UID: \"cfb8e98d-3873-43e0-81dc-4450983179c2\") " Dec 03 12:30:06 crc kubenswrapper[5052]: I1203 12:30:06.830188 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/cfb8e98d-3873-43e0-81dc-4450983179c2-logs\") pod \"cfb8e98d-3873-43e0-81dc-4450983179c2\" (UID: \"cfb8e98d-3873-43e0-81dc-4450983179c2\") " Dec 03 12:30:06 crc kubenswrapper[5052]: I1203 12:30:06.830217 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6927052c-212d-42fc-8236-b053ffc9a9d9-config-data\") pod \"6927052c-212d-42fc-8236-b053ffc9a9d9\" (UID: \"6927052c-212d-42fc-8236-b053ffc9a9d9\") " Dec 03 12:30:06 crc kubenswrapper[5052]: I1203 12:30:06.830232 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jhnj6\" (UniqueName: \"kubernetes.io/projected/6a77ebe2-ac70-4620-acef-bda957db5766-kube-api-access-jhnj6\") pod \"6a77ebe2-ac70-4620-acef-bda957db5766\" (UID: \"6a77ebe2-ac70-4620-acef-bda957db5766\") " Dec 03 12:30:06 crc kubenswrapper[5052]: I1203 12:30:06.830278 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/6927052c-212d-42fc-8236-b053ffc9a9d9-credential-keys\") pod \"6927052c-212d-42fc-8236-b053ffc9a9d9\" (UID: \"6927052c-212d-42fc-8236-b053ffc9a9d9\") " Dec 03 12:30:06 crc kubenswrapper[5052]: I1203 12:30:06.830293 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/6927052c-212d-42fc-8236-b053ffc9a9d9-public-tls-certs\") pod \"6927052c-212d-42fc-8236-b053ffc9a9d9\" (UID: \"6927052c-212d-42fc-8236-b053ffc9a9d9\") " Dec 03 12:30:06 crc kubenswrapper[5052]: I1203 12:30:06.830312 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/6a77ebe2-ac70-4620-acef-bda957db5766-config-volume\") pod \"6a77ebe2-ac70-4620-acef-bda957db5766\" (UID: \"6a77ebe2-ac70-4620-acef-bda957db5766\") " Dec 03 12:30:06 crc kubenswrapper[5052]: I1203 12:30:06.830339 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/cfb8e98d-3873-43e0-81dc-4450983179c2-config-data\") pod \"cfb8e98d-3873-43e0-81dc-4450983179c2\" (UID: \"cfb8e98d-3873-43e0-81dc-4450983179c2\") " Dec 03 12:30:06 crc kubenswrapper[5052]: I1203 12:30:06.830750 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-7x9gt\" (UniqueName: \"kubernetes.io/projected/16a065b5-a603-4cb3-a4c0-b36c0b92bf3f-kube-api-access-7x9gt\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:06 crc kubenswrapper[5052]: I1203 12:30:06.830767 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-zwjfg\" (UniqueName: \"kubernetes.io/projected/733f463b-dcf8-4a61-8e4e-4bf1c042d397-kube-api-access-zwjfg\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:06 crc kubenswrapper[5052]: I1203 12:30:06.841320 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cfb8e98d-3873-43e0-81dc-4450983179c2-logs" (OuterVolumeSpecName: "logs") pod "cfb8e98d-3873-43e0-81dc-4450983179c2" (UID: "cfb8e98d-3873-43e0-81dc-4450983179c2"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:30:06 crc kubenswrapper[5052]: I1203 12:30:06.846965 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6a77ebe2-ac70-4620-acef-bda957db5766-config-volume" (OuterVolumeSpecName: "config-volume") pod "6a77ebe2-ac70-4620-acef-bda957db5766" (UID: "6a77ebe2-ac70-4620-acef-bda957db5766"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:30:06 crc kubenswrapper[5052]: I1203 12:30:06.860686 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cfb8e98d-3873-43e0-81dc-4450983179c2-config-data-custom" (OuterVolumeSpecName: "config-data-custom") pod "cfb8e98d-3873-43e0-81dc-4450983179c2" (UID: "cfb8e98d-3873-43e0-81dc-4450983179c2"). InnerVolumeSpecName "config-data-custom". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:06 crc kubenswrapper[5052]: I1203 12:30:06.860895 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6927052c-212d-42fc-8236-b053ffc9a9d9-credential-keys" (OuterVolumeSpecName: "credential-keys") pod "6927052c-212d-42fc-8236-b053ffc9a9d9" (UID: "6927052c-212d-42fc-8236-b053ffc9a9d9"). InnerVolumeSpecName "credential-keys". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:06 crc kubenswrapper[5052]: I1203 12:30:06.875804 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6a77ebe2-ac70-4620-acef-bda957db5766-secret-volume" (OuterVolumeSpecName: "secret-volume") pod "6a77ebe2-ac70-4620-acef-bda957db5766" (UID: "6a77ebe2-ac70-4620-acef-bda957db5766"). InnerVolumeSpecName "secret-volume". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:06 crc kubenswrapper[5052]: I1203 12:30:06.877998 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6a77ebe2-ac70-4620-acef-bda957db5766-kube-api-access-jhnj6" (OuterVolumeSpecName: "kube-api-access-jhnj6") pod "6a77ebe2-ac70-4620-acef-bda957db5766" (UID: "6a77ebe2-ac70-4620-acef-bda957db5766"). InnerVolumeSpecName "kube-api-access-jhnj6". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:30:06 crc kubenswrapper[5052]: I1203 12:30:06.882763 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6927052c-212d-42fc-8236-b053ffc9a9d9-kube-api-access-gvz7n" (OuterVolumeSpecName: "kube-api-access-gvz7n") pod "6927052c-212d-42fc-8236-b053ffc9a9d9" (UID: "6927052c-212d-42fc-8236-b053ffc9a9d9"). InnerVolumeSpecName "kube-api-access-gvz7n". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:30:06 crc kubenswrapper[5052]: I1203 12:30:06.883222 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cfb8e98d-3873-43e0-81dc-4450983179c2-kube-api-access-8sxqm" (OuterVolumeSpecName: "kube-api-access-8sxqm") pod "cfb8e98d-3873-43e0-81dc-4450983179c2" (UID: "cfb8e98d-3873-43e0-81dc-4450983179c2"). InnerVolumeSpecName "kube-api-access-8sxqm". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:30:06 crc kubenswrapper[5052]: I1203 12:30:06.884315 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6927052c-212d-42fc-8236-b053ffc9a9d9-fernet-keys" (OuterVolumeSpecName: "fernet-keys") pod "6927052c-212d-42fc-8236-b053ffc9a9d9" (UID: "6927052c-212d-42fc-8236-b053ffc9a9d9"). InnerVolumeSpecName "fernet-keys". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:06 crc kubenswrapper[5052]: I1203 12:30:06.889025 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6927052c-212d-42fc-8236-b053ffc9a9d9-scripts" (OuterVolumeSpecName: "scripts") pod "6927052c-212d-42fc-8236-b053ffc9a9d9" (UID: "6927052c-212d-42fc-8236-b053ffc9a9d9"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:06 crc kubenswrapper[5052]: I1203 12:30:06.936015 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-8sxqm\" (UniqueName: \"kubernetes.io/projected/cfb8e98d-3873-43e0-81dc-4450983179c2-kube-api-access-8sxqm\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:06 crc kubenswrapper[5052]: I1203 12:30:06.936308 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/cfb8e98d-3873-43e0-81dc-4450983179c2-config-data-custom\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:06 crc kubenswrapper[5052]: I1203 12:30:06.936319 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/6927052c-212d-42fc-8236-b053ffc9a9d9-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:06 crc kubenswrapper[5052]: I1203 12:30:06.936328 5052 reconciler_common.go:293] "Volume detached for volume \"secret-volume\" (UniqueName: \"kubernetes.io/secret/6a77ebe2-ac70-4620-acef-bda957db5766-secret-volume\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:06 crc kubenswrapper[5052]: I1203 12:30:06.936338 5052 reconciler_common.go:293] "Volume detached for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/6927052c-212d-42fc-8236-b053ffc9a9d9-fernet-keys\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:06 crc kubenswrapper[5052]: I1203 12:30:06.936348 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/cfb8e98d-3873-43e0-81dc-4450983179c2-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:06 crc kubenswrapper[5052]: I1203 12:30:06.936356 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-jhnj6\" (UniqueName: \"kubernetes.io/projected/6a77ebe2-ac70-4620-acef-bda957db5766-kube-api-access-jhnj6\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:06 crc kubenswrapper[5052]: I1203 12:30:06.936364 5052 reconciler_common.go:293] "Volume detached for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/6927052c-212d-42fc-8236-b053ffc9a9d9-credential-keys\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:06 crc kubenswrapper[5052]: I1203 12:30:06.936372 5052 reconciler_common.go:293] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/6a77ebe2-ac70-4620-acef-bda957db5766-config-volume\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:06 crc kubenswrapper[5052]: I1203 12:30:06.936380 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-gvz7n\" (UniqueName: \"kubernetes.io/projected/6927052c-212d-42fc-8236-b053ffc9a9d9-kube-api-access-gvz7n\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:07.136974 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-operator-lifecycle-manager/collect-profiles-29412750-42nq6" event={"ID":"6a77ebe2-ac70-4620-acef-bda957db5766","Type":"ContainerDied","Data":"eb228dee2e6cdc91772f817a0a1e0dd61bffbdf799bae9cf1b3f9e10d968e07e"} Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:07.137009 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="eb228dee2e6cdc91772f817a0a1e0dd61bffbdf799bae9cf1b3f9e10d968e07e" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:07.137030 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-operator-lifecycle-manager/collect-profiles-29412750-42nq6" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:07.139220 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:07.147567 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-5fc7894bfd-g7hl5" event={"ID":"cfb8e98d-3873-43e0-81dc-4450983179c2","Type":"ContainerDied","Data":"2efd148878bd3b821fbdb42496959e8b2ff832e6e52d5d0916ca6f1eb95aa2f5"} Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:07.147612 5052 scope.go:117] "RemoveContainer" containerID="27c4bfe7d3a13a820b6065867e281595e076068c529bea4be94b943aa7105f13" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:07.147631 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-api-5fc7894bfd-g7hl5" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:07.151777 5052 generic.go:334] "Generic (PLEG): container finished" podID="214a26fb-e9b2-4a87-8fa5-ae2ed0d8a828" containerID="b6c09457c183d144bb440f045e7a0230c87297b84897525513a2e671062b9c96" exitCode=0 Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:07.151927 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"214a26fb-e9b2-4a87-8fa5-ae2ed0d8a828","Type":"ContainerDied","Data":"b6c09457c183d144bb440f045e7a0230c87297b84897525513a2e671062b9c96"} Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:07.153977 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-7cbf547d94-9krth" event={"ID":"6927052c-212d-42fc-8236-b053ffc9a9d9","Type":"ContainerDied","Data":"5a2899587be3f4e77f5c08e5d3c326409dcaaa5a13802cb665f621fed0dcb74c"} Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:07.154152 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-7cbf547d94-9krth" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:07.154252 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:07.154312 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:07.154399 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:07.155722 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:07.155872 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/neutron-db474c7dd-tg7mq" podUID="886ea0a4-5096-4ea8-a74c-de243d1f4f40" containerName="neutron-api" containerID="cri-o://36c7c94da2f3146bb43c39fdf96923c24534dd36b2033a6240c6be99921f9ca4" gracePeriod=30 Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:07.156113 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/neutron-db474c7dd-tg7mq" podUID="886ea0a4-5096-4ea8-a74c-de243d1f4f40" containerName="neutron-httpd" containerID="cri-o://71e595450c5a81e73076dfa57785d77ddaafc47e00972e087f567c2fa539835b" gracePeriod=30 Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:07.156345 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/barbican-keystone-listener-58b6cdf6bb-5xxd8" podUID="71df5bf1-7fd8-4a4d-b0f5-85e3911ae8a9" containerName="barbican-keystone-listener-log" containerID="cri-o://f902f915da399e4d8dce5c8b225bbaf37cabe6b67b4273e6cf9af96b3dcda39a" gracePeriod=30 Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:07.156426 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/barbican-keystone-listener-58b6cdf6bb-5xxd8" podUID="71df5bf1-7fd8-4a4d-b0f5-85e3911ae8a9" containerName="barbican-keystone-listener" containerID="cri-o://994203d634c00abfeeeb2db379076b53d6f741dba1b79d202d758f959e9439c6" gracePeriod=30 Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:07.181396 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b953ffa0-088c-4450-8d60-04044b33cfa0-config-data" (OuterVolumeSpecName: "config-data") pod "b953ffa0-088c-4450-8d60-04044b33cfa0" (UID: "b953ffa0-088c-4450-8d60-04044b33cfa0"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:07.193787 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/placement-65444bfc86-th5pw"] Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:07.204960 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/barbican-api-ff6b8c44b-7ndgm" podStartSLOduration=10.204941318 podStartE2EDuration="10.204941318s" podCreationTimestamp="2025-12-03 12:29:57 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:30:07.200304206 +0000 UTC m=+4197.788545970" watchObservedRunningTime="2025-12-03 12:30:07.204941318 +0000 UTC m=+4197.793183082" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:07.225373 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/neutron-db474c7dd-tg7mq" podStartSLOduration=12.22532466 podStartE2EDuration="12.22532466s" podCreationTimestamp="2025-12-03 12:29:55 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:30:07.217047854 +0000 UTC m=+4197.805289608" watchObservedRunningTime="2025-12-03 12:30:07.22532466 +0000 UTC m=+4197.813566424" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:07.235751 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b953ffa0-088c-4450-8d60-04044b33cfa0-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "b953ffa0-088c-4450-8d60-04044b33cfa0" (UID: "b953ffa0-088c-4450-8d60-04044b33cfa0"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:07.244924 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b953ffa0-088c-4450-8d60-04044b33cfa0-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:07.244949 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/b953ffa0-088c-4450-8d60-04044b33cfa0-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:07.264596 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/733f463b-dcf8-4a61-8e4e-4bf1c042d397-config-data" (OuterVolumeSpecName: "config-data") pod "733f463b-dcf8-4a61-8e4e-4bf1c042d397" (UID: "733f463b-dcf8-4a61-8e4e-4bf1c042d397"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:07.266349 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6927052c-212d-42fc-8236-b053ffc9a9d9-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "6927052c-212d-42fc-8236-b053ffc9a9d9" (UID: "6927052c-212d-42fc-8236-b053ffc9a9d9"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:07.267487 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/ovsdbserver-sb-0" podStartSLOduration=8.267468923 podStartE2EDuration="8.267468923s" podCreationTimestamp="2025-12-03 12:29:59 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:30:07.255196372 +0000 UTC m=+4197.843438136" watchObservedRunningTime="2025-12-03 12:30:07.267468923 +0000 UTC m=+4197.855710687" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:07.268061 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/16a065b5-a603-4cb3-a4c0-b36c0b92bf3f-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "16a065b5-a603-4cb3-a4c0-b36c0b92bf3f" (UID: "16a065b5-a603-4cb3-a4c0-b36c0b92bf3f"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:07.268534 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/16a065b5-a603-4cb3-a4c0-b36c0b92bf3f-config-data" (OuterVolumeSpecName: "config-data") pod "16a065b5-a603-4cb3-a4c0-b36c0b92bf3f" (UID: "16a065b5-a603-4cb3-a4c0-b36c0b92bf3f"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:07.270698 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6927052c-212d-42fc-8236-b053ffc9a9d9-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "6927052c-212d-42fc-8236-b053ffc9a9d9" (UID: "6927052c-212d-42fc-8236-b053ffc9a9d9"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:07.281683 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6927052c-212d-42fc-8236-b053ffc9a9d9-config-data" (OuterVolumeSpecName: "config-data") pod "6927052c-212d-42fc-8236-b053ffc9a9d9" (UID: "6927052c-212d-42fc-8236-b053ffc9a9d9"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:07.293413 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/733f463b-dcf8-4a61-8e4e-4bf1c042d397-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "733f463b-dcf8-4a61-8e4e-4bf1c042d397" (UID: "733f463b-dcf8-4a61-8e4e-4bf1c042d397"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:07.295081 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cfb8e98d-3873-43e0-81dc-4450983179c2-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "cfb8e98d-3873-43e0-81dc-4450983179c2" (UID: "cfb8e98d-3873-43e0-81dc-4450983179c2"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:07.307332 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6927052c-212d-42fc-8236-b053ffc9a9d9-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "6927052c-212d-42fc-8236-b053ffc9a9d9" (UID: "6927052c-212d-42fc-8236-b053ffc9a9d9"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:07.308210 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cfb8e98d-3873-43e0-81dc-4450983179c2-config-data" (OuterVolumeSpecName: "config-data") pod "cfb8e98d-3873-43e0-81dc-4450983179c2" (UID: "cfb8e98d-3873-43e0-81dc-4450983179c2"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:07.314671 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cfb8e98d-3873-43e0-81dc-4450983179c2-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "cfb8e98d-3873-43e0-81dc-4450983179c2" (UID: "cfb8e98d-3873-43e0-81dc-4450983179c2"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:07.324043 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cfb8e98d-3873-43e0-81dc-4450983179c2-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "cfb8e98d-3873-43e0-81dc-4450983179c2" (UID: "cfb8e98d-3873-43e0-81dc-4450983179c2"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:07.347868 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/733f463b-dcf8-4a61-8e4e-4bf1c042d397-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:07.349553 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/cfb8e98d-3873-43e0-81dc-4450983179c2-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:07.349575 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/16a065b5-a603-4cb3-a4c0-b36c0b92bf3f-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:07.349588 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/cfb8e98d-3873-43e0-81dc-4450983179c2-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:07.349601 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6927052c-212d-42fc-8236-b053ffc9a9d9-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:07.349613 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/16a065b5-a603-4cb3-a4c0-b36c0b92bf3f-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:07.349625 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/6927052c-212d-42fc-8236-b053ffc9a9d9-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:07.349636 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/733f463b-dcf8-4a61-8e4e-4bf1c042d397-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:07.349648 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/cfb8e98d-3873-43e0-81dc-4450983179c2-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:07.349661 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6927052c-212d-42fc-8236-b053ffc9a9d9-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:07.349672 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/6927052c-212d-42fc-8236-b053ffc9a9d9-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:07.349684 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/cfb8e98d-3873-43e0-81dc-4450983179c2-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:07.640197 5052 scope.go:117] "RemoveContainer" containerID="0a861967403e553bfc0bb4f31723c2962d1eb4b87dfc36853566225a16745bf9" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:07.687604 5052 scope.go:117] "RemoveContainer" containerID="edaf8cd0d0bbbd5a49f586c7c2c1e7bcdb11f76c906dcb6bcd4c10118fa5950b" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:07.703500 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:07.716828 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:07.732373 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-0"] Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:07.752894 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:30:09 crc kubenswrapper[5052]: E1203 12:30:07.753401 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="e75eea2c-2eb1-4f0f-97e5-63ce147e68dd" containerName="ceilometer-notification-agent" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:07.753414 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="e75eea2c-2eb1-4f0f-97e5-63ce147e68dd" containerName="ceilometer-notification-agent" Dec 03 12:30:09 crc kubenswrapper[5052]: E1203 12:30:07.753422 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="05b9d3cc-93de-45a6-8849-027a4aad3715" containerName="glance-log" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:07.753429 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="05b9d3cc-93de-45a6-8849-027a4aad3715" containerName="glance-log" Dec 03 12:30:09 crc kubenswrapper[5052]: E1203 12:30:07.753447 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="05b9d3cc-93de-45a6-8849-027a4aad3715" containerName="glance-httpd" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:07.753455 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="05b9d3cc-93de-45a6-8849-027a4aad3715" containerName="glance-httpd" Dec 03 12:30:09 crc kubenswrapper[5052]: E1203 12:30:07.753469 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="cfb8e98d-3873-43e0-81dc-4450983179c2" containerName="barbican-api" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:07.753475 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="cfb8e98d-3873-43e0-81dc-4450983179c2" containerName="barbican-api" Dec 03 12:30:09 crc kubenswrapper[5052]: E1203 12:30:07.753489 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="b953ffa0-088c-4450-8d60-04044b33cfa0" containerName="nova-scheduler-scheduler" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:07.753495 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="b953ffa0-088c-4450-8d60-04044b33cfa0" containerName="nova-scheduler-scheduler" Dec 03 12:30:09 crc kubenswrapper[5052]: E1203 12:30:07.753535 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="6927052c-212d-42fc-8236-b053ffc9a9d9" containerName="keystone-api" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:07.753541 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="6927052c-212d-42fc-8236-b053ffc9a9d9" containerName="keystone-api" Dec 03 12:30:09 crc kubenswrapper[5052]: E1203 12:30:07.753553 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="e75eea2c-2eb1-4f0f-97e5-63ce147e68dd" containerName="ceilometer-central-agent" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:07.753560 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="e75eea2c-2eb1-4f0f-97e5-63ce147e68dd" containerName="ceilometer-central-agent" Dec 03 12:30:09 crc kubenswrapper[5052]: E1203 12:30:07.753576 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="6a77ebe2-ac70-4620-acef-bda957db5766" containerName="collect-profiles" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:07.753583 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="6a77ebe2-ac70-4620-acef-bda957db5766" containerName="collect-profiles" Dec 03 12:30:09 crc kubenswrapper[5052]: E1203 12:30:07.753604 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="e75eea2c-2eb1-4f0f-97e5-63ce147e68dd" containerName="proxy-httpd" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:07.753611 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="e75eea2c-2eb1-4f0f-97e5-63ce147e68dd" containerName="proxy-httpd" Dec 03 12:30:09 crc kubenswrapper[5052]: E1203 12:30:07.753622 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="733f463b-dcf8-4a61-8e4e-4bf1c042d397" containerName="nova-cell1-conductor-conductor" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:07.753629 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="733f463b-dcf8-4a61-8e4e-4bf1c042d397" containerName="nova-cell1-conductor-conductor" Dec 03 12:30:09 crc kubenswrapper[5052]: E1203 12:30:07.753655 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="cfb8e98d-3873-43e0-81dc-4450983179c2" containerName="barbican-api-log" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:07.753660 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="cfb8e98d-3873-43e0-81dc-4450983179c2" containerName="barbican-api-log" Dec 03 12:30:09 crc kubenswrapper[5052]: E1203 12:30:07.753675 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="16a065b5-a603-4cb3-a4c0-b36c0b92bf3f" containerName="nova-cell0-conductor-conductor" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:07.753681 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="16a065b5-a603-4cb3-a4c0-b36c0b92bf3f" containerName="nova-cell0-conductor-conductor" Dec 03 12:30:09 crc kubenswrapper[5052]: E1203 12:30:07.753693 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="e75eea2c-2eb1-4f0f-97e5-63ce147e68dd" containerName="sg-core" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:07.753699 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="e75eea2c-2eb1-4f0f-97e5-63ce147e68dd" containerName="sg-core" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:07.753878 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="e75eea2c-2eb1-4f0f-97e5-63ce147e68dd" containerName="sg-core" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:07.753888 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="e75eea2c-2eb1-4f0f-97e5-63ce147e68dd" containerName="proxy-httpd" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:07.753903 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="e75eea2c-2eb1-4f0f-97e5-63ce147e68dd" containerName="ceilometer-central-agent" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:07.753911 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="6a77ebe2-ac70-4620-acef-bda957db5766" containerName="collect-profiles" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:07.753923 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="6927052c-212d-42fc-8236-b053ffc9a9d9" containerName="keystone-api" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:07.753933 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="05b9d3cc-93de-45a6-8849-027a4aad3715" containerName="glance-httpd" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:07.753945 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="cfb8e98d-3873-43e0-81dc-4450983179c2" containerName="barbican-api" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:07.753954 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="e75eea2c-2eb1-4f0f-97e5-63ce147e68dd" containerName="ceilometer-notification-agent" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:07.753961 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="cfb8e98d-3873-43e0-81dc-4450983179c2" containerName="barbican-api-log" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:07.753971 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="b953ffa0-088c-4450-8d60-04044b33cfa0" containerName="nova-scheduler-scheduler" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:07.753980 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="16a065b5-a603-4cb3-a4c0-b36c0b92bf3f" containerName="nova-cell0-conductor-conductor" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:07.753990 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="05b9d3cc-93de-45a6-8849-027a4aad3715" containerName="glance-log" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:07.754000 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="733f463b-dcf8-4a61-8e4e-4bf1c042d397" containerName="nova-cell1-conductor-conductor" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:07.755043 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:07.759979 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-scripts" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:07.760231 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-default-external-config-data" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:07.760601 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-glance-dockercfg-cd9ns" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:07.762830 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-glance-default-public-svc" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:07.784274 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-0"] Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:07.810863 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-0"] Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:07.854982 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:07.859753 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/e8f63775-6260-4abb-abd7-45a412bf0ca5-public-tls-certs\") pod \"glance-default-external-api-0\" (UID: \"e8f63775-6260-4abb-abd7-45a412bf0ca5\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:07.859793 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/e8f63775-6260-4abb-abd7-45a412bf0ca5-config-data\") pod \"glance-default-external-api-0\" (UID: \"e8f63775-6260-4abb-abd7-45a412bf0ca5\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:07.859823 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/e8f63775-6260-4abb-abd7-45a412bf0ca5-scripts\") pod \"glance-default-external-api-0\" (UID: \"e8f63775-6260-4abb-abd7-45a412bf0ca5\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:07.859866 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e8f63775-6260-4abb-abd7-45a412bf0ca5-combined-ca-bundle\") pod \"glance-default-external-api-0\" (UID: \"e8f63775-6260-4abb-abd7-45a412bf0ca5\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:07.859887 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/e8f63775-6260-4abb-abd7-45a412bf0ca5-httpd-run\") pod \"glance-default-external-api-0\" (UID: \"e8f63775-6260-4abb-abd7-45a412bf0ca5\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:07.860196 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage07-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage07-crc\") pod \"glance-default-external-api-0\" (UID: \"e8f63775-6260-4abb-abd7-45a412bf0ca5\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:07.860220 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/e8f63775-6260-4abb-abd7-45a412bf0ca5-logs\") pod \"glance-default-external-api-0\" (UID: \"e8f63775-6260-4abb-abd7-45a412bf0ca5\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:07.860246 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4cnf9\" (UniqueName: \"kubernetes.io/projected/e8f63775-6260-4abb-abd7-45a412bf0ca5-kube-api-access-4cnf9\") pod \"glance-default-external-api-0\" (UID: \"e8f63775-6260-4abb-abd7-45a412bf0ca5\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:07.871840 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-0"] Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:07.893577 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-0"] Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:07.894814 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:07.898249 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell1-conductor-config-data" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:07.961920 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:07.962729 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2dfbec8a-3a04-407b-8145-e12a3afee2e0-combined-ca-bundle\") pod \"nova-cell1-conductor-0\" (UID: \"2dfbec8a-3a04-407b-8145-e12a3afee2e0\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:07.962788 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/e8f63775-6260-4abb-abd7-45a412bf0ca5-public-tls-certs\") pod \"glance-default-external-api-0\" (UID: \"e8f63775-6260-4abb-abd7-45a412bf0ca5\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:07.962813 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/e8f63775-6260-4abb-abd7-45a412bf0ca5-config-data\") pod \"glance-default-external-api-0\" (UID: \"e8f63775-6260-4abb-abd7-45a412bf0ca5\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:07.962846 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/e8f63775-6260-4abb-abd7-45a412bf0ca5-scripts\") pod \"glance-default-external-api-0\" (UID: \"e8f63775-6260-4abb-abd7-45a412bf0ca5\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:07.962891 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e8f63775-6260-4abb-abd7-45a412bf0ca5-combined-ca-bundle\") pod \"glance-default-external-api-0\" (UID: \"e8f63775-6260-4abb-abd7-45a412bf0ca5\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:07.962911 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/e8f63775-6260-4abb-abd7-45a412bf0ca5-httpd-run\") pod \"glance-default-external-api-0\" (UID: \"e8f63775-6260-4abb-abd7-45a412bf0ca5\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:07.962943 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage07-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage07-crc\") pod \"glance-default-external-api-0\" (UID: \"e8f63775-6260-4abb-abd7-45a412bf0ca5\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:07.962968 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/e8f63775-6260-4abb-abd7-45a412bf0ca5-logs\") pod \"glance-default-external-api-0\" (UID: \"e8f63775-6260-4abb-abd7-45a412bf0ca5\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:07.962989 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-4cnf9\" (UniqueName: \"kubernetes.io/projected/e8f63775-6260-4abb-abd7-45a412bf0ca5-kube-api-access-4cnf9\") pod \"glance-default-external-api-0\" (UID: \"e8f63775-6260-4abb-abd7-45a412bf0ca5\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:07.963045 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/2dfbec8a-3a04-407b-8145-e12a3afee2e0-config-data\") pod \"nova-cell1-conductor-0\" (UID: \"2dfbec8a-3a04-407b-8145-e12a3afee2e0\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:07.963089 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sd9m4\" (UniqueName: \"kubernetes.io/projected/2dfbec8a-3a04-407b-8145-e12a3afee2e0-kube-api-access-sd9m4\") pod \"nova-cell1-conductor-0\" (UID: \"2dfbec8a-3a04-407b-8145-e12a3afee2e0\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:07.964130 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage07-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage07-crc\") pod \"glance-default-external-api-0\" (UID: \"e8f63775-6260-4abb-abd7-45a412bf0ca5\") device mount path \"/mnt/openstack/pv07\"" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:07.969110 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/e8f63775-6260-4abb-abd7-45a412bf0ca5-config-data\") pod \"glance-default-external-api-0\" (UID: \"e8f63775-6260-4abb-abd7-45a412bf0ca5\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:07.969376 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/e8f63775-6260-4abb-abd7-45a412bf0ca5-logs\") pod \"glance-default-external-api-0\" (UID: \"e8f63775-6260-4abb-abd7-45a412bf0ca5\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:07.971856 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/e8f63775-6260-4abb-abd7-45a412bf0ca5-httpd-run\") pod \"glance-default-external-api-0\" (UID: \"e8f63775-6260-4abb-abd7-45a412bf0ca5\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:07.974674 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e8f63775-6260-4abb-abd7-45a412bf0ca5-combined-ca-bundle\") pod \"glance-default-external-api-0\" (UID: \"e8f63775-6260-4abb-abd7-45a412bf0ca5\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:07.975231 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/e8f63775-6260-4abb-abd7-45a412bf0ca5-scripts\") pod \"glance-default-external-api-0\" (UID: \"e8f63775-6260-4abb-abd7-45a412bf0ca5\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:07.976274 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:07.978677 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/e8f63775-6260-4abb-abd7-45a412bf0ca5-public-tls-certs\") pod \"glance-default-external-api-0\" (UID: \"e8f63775-6260-4abb-abd7-45a412bf0ca5\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:07.988499 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-4cnf9\" (UniqueName: \"kubernetes.io/projected/e8f63775-6260-4abb-abd7-45a412bf0ca5-kube-api-access-4cnf9\") pod \"glance-default-external-api-0\" (UID: \"e8f63775-6260-4abb-abd7-45a412bf0ca5\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:07.991094 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-0"] Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:07.998963 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:08.003433 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage07-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage07-crc\") pod \"glance-default-external-api-0\" (UID: \"e8f63775-6260-4abb-abd7-45a412bf0ca5\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:08.015748 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:08.026565 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-0"] Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:08.028043 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:08.030107 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell0-conductor-config-data" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:08.042573 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:08.050760 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/barbican-api-ff6b8c44b-7ndgm" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:08.050789 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/barbican-api-ff6b8c44b-7ndgm" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:08.050873 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:08.051335 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-0"] Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:08.053999 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-ceilometer-internal-svc" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:08.054300 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-scripts" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:08.057640 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-config-data" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:08.077296 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:08.079753 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4jn4f\" (UniqueName: \"kubernetes.io/projected/e2718906-e52d-42fa-ad42-35626e99d461-kube-api-access-4jn4f\") pod \"nova-cell0-conductor-0\" (UID: \"e2718906-e52d-42fa-ad42-35626e99d461\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:08.079823 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-sd9m4\" (UniqueName: \"kubernetes.io/projected/2dfbec8a-3a04-407b-8145-e12a3afee2e0-kube-api-access-sd9m4\") pod \"nova-cell1-conductor-0\" (UID: \"2dfbec8a-3a04-407b-8145-e12a3afee2e0\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:08.079866 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2dfbec8a-3a04-407b-8145-e12a3afee2e0-combined-ca-bundle\") pod \"nova-cell1-conductor-0\" (UID: \"2dfbec8a-3a04-407b-8145-e12a3afee2e0\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:08.079946 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a24b0824-26d0-4b81-b6cb-d617c36d45ab-config-data\") pod \"ceilometer-0\" (UID: \"a24b0824-26d0-4b81-b6cb-d617c36d45ab\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:08.079987 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/a24b0824-26d0-4b81-b6cb-d617c36d45ab-run-httpd\") pod \"ceilometer-0\" (UID: \"a24b0824-26d0-4b81-b6cb-d617c36d45ab\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:08.080044 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/a24b0824-26d0-4b81-b6cb-d617c36d45ab-ceilometer-tls-certs\") pod \"ceilometer-0\" (UID: \"a24b0824-26d0-4b81-b6cb-d617c36d45ab\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:08.080189 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qr6nj\" (UniqueName: \"kubernetes.io/projected/a24b0824-26d0-4b81-b6cb-d617c36d45ab-kube-api-access-qr6nj\") pod \"ceilometer-0\" (UID: \"a24b0824-26d0-4b81-b6cb-d617c36d45ab\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:08.080379 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/a24b0824-26d0-4b81-b6cb-d617c36d45ab-log-httpd\") pod \"ceilometer-0\" (UID: \"a24b0824-26d0-4b81-b6cb-d617c36d45ab\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:08.080412 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e2718906-e52d-42fa-ad42-35626e99d461-combined-ca-bundle\") pod \"nova-cell0-conductor-0\" (UID: \"e2718906-e52d-42fa-ad42-35626e99d461\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:08.080668 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/a24b0824-26d0-4b81-b6cb-d617c36d45ab-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"a24b0824-26d0-4b81-b6cb-d617c36d45ab\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:08.080760 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a24b0824-26d0-4b81-b6cb-d617c36d45ab-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"a24b0824-26d0-4b81-b6cb-d617c36d45ab\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:08.080781 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/e2718906-e52d-42fa-ad42-35626e99d461-config-data\") pod \"nova-cell0-conductor-0\" (UID: \"e2718906-e52d-42fa-ad42-35626e99d461\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:08.080902 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/2dfbec8a-3a04-407b-8145-e12a3afee2e0-config-data\") pod \"nova-cell1-conductor-0\" (UID: \"2dfbec8a-3a04-407b-8145-e12a3afee2e0\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:08.080941 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/a24b0824-26d0-4b81-b6cb-d617c36d45ab-scripts\") pod \"ceilometer-0\" (UID: \"a24b0824-26d0-4b81-b6cb-d617c36d45ab\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:08.088650 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:08.093943 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2dfbec8a-3a04-407b-8145-e12a3afee2e0-combined-ca-bundle\") pod \"nova-cell1-conductor-0\" (UID: \"2dfbec8a-3a04-407b-8145-e12a3afee2e0\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:08.103785 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/2dfbec8a-3a04-407b-8145-e12a3afee2e0-config-data\") pod \"nova-cell1-conductor-0\" (UID: \"2dfbec8a-3a04-407b-8145-e12a3afee2e0\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:08.117399 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-sd9m4\" (UniqueName: \"kubernetes.io/projected/2dfbec8a-3a04-407b-8145-e12a3afee2e0-kube-api-access-sd9m4\") pod \"nova-cell1-conductor-0\" (UID: \"2dfbec8a-3a04-407b-8145-e12a3afee2e0\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:08.118997 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:08.119240 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:08.122691 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-scheduler-config-data" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:08.127387 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/keystone-7cbf547d94-9krth"] Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:08.147081 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/keystone-7cbf547d94-9krth"] Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:08.159989 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-api-5fc7894bfd-g7hl5"] Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:08.171574 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:08.175577 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-65444bfc86-th5pw" event={"ID":"cfd8af50-a644-4b24-8483-dd344f2218e4","Type":"ContainerStarted","Data":"3d7601570722d86f8224664e7bb57ae02c41b3b00b91c1982ff975c685f871f9"} Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:08.179248 5052 generic.go:334] "Generic (PLEG): container finished" podID="71df5bf1-7fd8-4a4d-b0f5-85e3911ae8a9" containerID="f902f915da399e4d8dce5c8b225bbaf37cabe6b67b4273e6cf9af96b3dcda39a" exitCode=143 Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:08.179308 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-keystone-listener-58b6cdf6bb-5xxd8" event={"ID":"71df5bf1-7fd8-4a4d-b0f5-85e3911ae8a9","Type":"ContainerDied","Data":"f902f915da399e4d8dce5c8b225bbaf37cabe6b67b4273e6cf9af96b3dcda39a"} Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:08.181686 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/barbican-api-5fc7894bfd-g7hl5"] Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:08.182190 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_ovn-northd-0_a26ed42f-71e7-4c92-8b7e-ad194358d3e5/ovn-northd/0.log" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:08.182217 5052 generic.go:334] "Generic (PLEG): container finished" podID="a26ed42f-71e7-4c92-8b7e-ad194358d3e5" containerID="93474533ee16033979340cc7914412b7661d56c4b5e3b0058db42692da5f5166" exitCode=139 Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:08.182344 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovn-northd-0" event={"ID":"a26ed42f-71e7-4c92-8b7e-ad194358d3e5","Type":"ContainerDied","Data":"93474533ee16033979340cc7914412b7661d56c4b5e3b0058db42692da5f5166"} Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:08.184934 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/a24b0824-26d0-4b81-b6cb-d617c36d45ab-run-httpd\") pod \"ceilometer-0\" (UID: \"a24b0824-26d0-4b81-b6cb-d617c36d45ab\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:08.185220 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/a24b0824-26d0-4b81-b6cb-d617c36d45ab-ceilometer-tls-certs\") pod \"ceilometer-0\" (UID: \"a24b0824-26d0-4b81-b6cb-d617c36d45ab\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:08.185270 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/f23969c4-775b-49b1-a88f-6fe74fd1db75-config-data\") pod \"nova-scheduler-0\" (UID: \"f23969c4-775b-49b1-a88f-6fe74fd1db75\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:08.185336 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-qr6nj\" (UniqueName: \"kubernetes.io/projected/a24b0824-26d0-4b81-b6cb-d617c36d45ab-kube-api-access-qr6nj\") pod \"ceilometer-0\" (UID: \"a24b0824-26d0-4b81-b6cb-d617c36d45ab\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:08.185418 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/a24b0824-26d0-4b81-b6cb-d617c36d45ab-log-httpd\") pod \"ceilometer-0\" (UID: \"a24b0824-26d0-4b81-b6cb-d617c36d45ab\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:08.185444 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/a24b0824-26d0-4b81-b6cb-d617c36d45ab-run-httpd\") pod \"ceilometer-0\" (UID: \"a24b0824-26d0-4b81-b6cb-d617c36d45ab\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:08.186661 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e2718906-e52d-42fa-ad42-35626e99d461-combined-ca-bundle\") pod \"nova-cell0-conductor-0\" (UID: \"e2718906-e52d-42fa-ad42-35626e99d461\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:08.186700 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/a24b0824-26d0-4b81-b6cb-d617c36d45ab-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"a24b0824-26d0-4b81-b6cb-d617c36d45ab\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:08.186766 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a24b0824-26d0-4b81-b6cb-d617c36d45ab-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"a24b0824-26d0-4b81-b6cb-d617c36d45ab\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:08.186789 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/e2718906-e52d-42fa-ad42-35626e99d461-config-data\") pod \"nova-cell0-conductor-0\" (UID: \"e2718906-e52d-42fa-ad42-35626e99d461\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:08.186816 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f23969c4-775b-49b1-a88f-6fe74fd1db75-combined-ca-bundle\") pod \"nova-scheduler-0\" (UID: \"f23969c4-775b-49b1-a88f-6fe74fd1db75\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:08.186932 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4b2md\" (UniqueName: \"kubernetes.io/projected/f23969c4-775b-49b1-a88f-6fe74fd1db75-kube-api-access-4b2md\") pod \"nova-scheduler-0\" (UID: \"f23969c4-775b-49b1-a88f-6fe74fd1db75\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:08.186970 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/a24b0824-26d0-4b81-b6cb-d617c36d45ab-scripts\") pod \"ceilometer-0\" (UID: \"a24b0824-26d0-4b81-b6cb-d617c36d45ab\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:08.187043 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-4jn4f\" (UniqueName: \"kubernetes.io/projected/e2718906-e52d-42fa-ad42-35626e99d461-kube-api-access-4jn4f\") pod \"nova-cell0-conductor-0\" (UID: \"e2718906-e52d-42fa-ad42-35626e99d461\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:08.187122 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a24b0824-26d0-4b81-b6cb-d617c36d45ab-config-data\") pod \"ceilometer-0\" (UID: \"a24b0824-26d0-4b81-b6cb-d617c36d45ab\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:08.187193 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/a24b0824-26d0-4b81-b6cb-d617c36d45ab-log-httpd\") pod \"ceilometer-0\" (UID: \"a24b0824-26d0-4b81-b6cb-d617c36d45ab\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:08.192249 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e2718906-e52d-42fa-ad42-35626e99d461-combined-ca-bundle\") pod \"nova-cell0-conductor-0\" (UID: \"e2718906-e52d-42fa-ad42-35626e99d461\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:08.192303 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/a24b0824-26d0-4b81-b6cb-d617c36d45ab-ceilometer-tls-certs\") pod \"ceilometer-0\" (UID: \"a24b0824-26d0-4b81-b6cb-d617c36d45ab\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:08.193266 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/a24b0824-26d0-4b81-b6cb-d617c36d45ab-scripts\") pod \"ceilometer-0\" (UID: \"a24b0824-26d0-4b81-b6cb-d617c36d45ab\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:08.198129 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/a24b0824-26d0-4b81-b6cb-d617c36d45ab-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"a24b0824-26d0-4b81-b6cb-d617c36d45ab\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:08.204905 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/e2718906-e52d-42fa-ad42-35626e99d461-config-data\") pod \"nova-cell0-conductor-0\" (UID: \"e2718906-e52d-42fa-ad42-35626e99d461\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:08.205011 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a24b0824-26d0-4b81-b6cb-d617c36d45ab-config-data\") pod \"ceilometer-0\" (UID: \"a24b0824-26d0-4b81-b6cb-d617c36d45ab\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:08.206614 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a24b0824-26d0-4b81-b6cb-d617c36d45ab-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"a24b0824-26d0-4b81-b6cb-d617c36d45ab\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:08.210981 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-4jn4f\" (UniqueName: \"kubernetes.io/projected/e2718906-e52d-42fa-ad42-35626e99d461-kube-api-access-4jn4f\") pod \"nova-cell0-conductor-0\" (UID: \"e2718906-e52d-42fa-ad42-35626e99d461\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:08.216022 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-qr6nj\" (UniqueName: \"kubernetes.io/projected/a24b0824-26d0-4b81-b6cb-d617c36d45ab-kube-api-access-qr6nj\") pod \"ceilometer-0\" (UID: \"a24b0824-26d0-4b81-b6cb-d617c36d45ab\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:08.241193 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openshift-operator-lifecycle-manager/collect-profiles-29412705-7rvq9"] Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:08.259316 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openshift-operator-lifecycle-manager/collect-profiles-29412705-7rvq9"] Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:08.278416 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:08.291114 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-4b2md\" (UniqueName: \"kubernetes.io/projected/f23969c4-775b-49b1-a88f-6fe74fd1db75-kube-api-access-4b2md\") pod \"nova-scheduler-0\" (UID: \"f23969c4-775b-49b1-a88f-6fe74fd1db75\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:08.291479 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/f23969c4-775b-49b1-a88f-6fe74fd1db75-config-data\") pod \"nova-scheduler-0\" (UID: \"f23969c4-775b-49b1-a88f-6fe74fd1db75\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:08.291894 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f23969c4-775b-49b1-a88f-6fe74fd1db75-combined-ca-bundle\") pod \"nova-scheduler-0\" (UID: \"f23969c4-775b-49b1-a88f-6fe74fd1db75\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:08.296650 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f23969c4-775b-49b1-a88f-6fe74fd1db75-combined-ca-bundle\") pod \"nova-scheduler-0\" (UID: \"f23969c4-775b-49b1-a88f-6fe74fd1db75\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:08.298117 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/f23969c4-775b-49b1-a88f-6fe74fd1db75-config-data\") pod \"nova-scheduler-0\" (UID: \"f23969c4-775b-49b1-a88f-6fe74fd1db75\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:08.313615 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-4b2md\" (UniqueName: \"kubernetes.io/projected/f23969c4-775b-49b1-a88f-6fe74fd1db75-kube-api-access-4b2md\") pod \"nova-scheduler-0\" (UID: \"f23969c4-775b-49b1-a88f-6fe74fd1db75\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:08.366215 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:08.396994 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:30:09 crc kubenswrapper[5052]: E1203 12:30:08.537887 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="165a4ba65f86633615c852ba29ceb303c3bbd0dbd696084cf0c3a26afe0a4424" cmd=["/bin/bash","/var/lib/operator-scripts/mysql_probe.sh","readiness"] Dec 03 12:30:09 crc kubenswrapper[5052]: E1203 12:30:08.546952 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="165a4ba65f86633615c852ba29ceb303c3bbd0dbd696084cf0c3a26afe0a4424" cmd=["/bin/bash","/var/lib/operator-scripts/mysql_probe.sh","readiness"] Dec 03 12:30:09 crc kubenswrapper[5052]: E1203 12:30:08.555042 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="165a4ba65f86633615c852ba29ceb303c3bbd0dbd696084cf0c3a26afe0a4424" cmd=["/bin/bash","/var/lib/operator-scripts/mysql_probe.sh","readiness"] Dec 03 12:30:09 crc kubenswrapper[5052]: E1203 12:30:08.555108 5052 prober.go:104] "Probe errored" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" probeType="Readiness" pod="openstack-kuttl-tests/openstack-galera-0" podUID="12f39c79-eeb9-4cf4-8553-1a4d9a07ee82" containerName="galera" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:08.614714 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:08.713331 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:08.765841 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:08.830165 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-nb-0"] Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:09.193389 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:09.244015 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:09.570000 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:09.603967 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:09.728944 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="05b9d3cc-93de-45a6-8849-027a4aad3715" path="/var/lib/kubelet/pods/05b9d3cc-93de-45a6-8849-027a4aad3715/volumes" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:09.735493 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="16a065b5-a603-4cb3-a4c0-b36c0b92bf3f" path="/var/lib/kubelet/pods/16a065b5-a603-4cb3-a4c0-b36c0b92bf3f/volumes" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:09.737041 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="323fd029-104c-4804-8829-3ed690c6c15b" path="/var/lib/kubelet/pods/323fd029-104c-4804-8829-3ed690c6c15b/volumes" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:09.741826 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6927052c-212d-42fc-8236-b053ffc9a9d9" path="/var/lib/kubelet/pods/6927052c-212d-42fc-8236-b053ffc9a9d9/volumes" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:09.744265 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="733f463b-dcf8-4a61-8e4e-4bf1c042d397" path="/var/lib/kubelet/pods/733f463b-dcf8-4a61-8e4e-4bf1c042d397/volumes" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:09.745971 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b953ffa0-088c-4450-8d60-04044b33cfa0" path="/var/lib/kubelet/pods/b953ffa0-088c-4450-8d60-04044b33cfa0/volumes" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:09.748985 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cfb8e98d-3873-43e0-81dc-4450983179c2" path="/var/lib/kubelet/pods/cfb8e98d-3873-43e0-81dc-4450983179c2/volumes" Dec 03 12:30:09 crc kubenswrapper[5052]: I1203 12:30:09.749880 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e75eea2c-2eb1-4f0f-97e5-63ce147e68dd" path="/var/lib/kubelet/pods/e75eea2c-2eb1-4f0f-97e5-63ce147e68dd/volumes" Dec 03 12:30:09 crc kubenswrapper[5052]: E1203 12:30:09.755838 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of 93474533ee16033979340cc7914412b7661d56c4b5e3b0058db42692da5f5166 is running failed: container process not found" containerID="93474533ee16033979340cc7914412b7661d56c4b5e3b0058db42692da5f5166" cmd=["/usr/local/bin/container-scripts/status_check.sh"] Dec 03 12:30:09 crc kubenswrapper[5052]: E1203 12:30:09.761737 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of 93474533ee16033979340cc7914412b7661d56c4b5e3b0058db42692da5f5166 is running failed: container process not found" containerID="93474533ee16033979340cc7914412b7661d56c4b5e3b0058db42692da5f5166" cmd=["/usr/local/bin/container-scripts/status_check.sh"] Dec 03 12:30:09 crc kubenswrapper[5052]: E1203 12:30:09.767660 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of 93474533ee16033979340cc7914412b7661d56c4b5e3b0058db42692da5f5166 is running failed: container process not found" containerID="93474533ee16033979340cc7914412b7661d56c4b5e3b0058db42692da5f5166" cmd=["/usr/local/bin/container-scripts/status_check.sh"] Dec 03 12:30:09 crc kubenswrapper[5052]: E1203 12:30:09.767736 5052 prober.go:104] "Probe errored" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of 93474533ee16033979340cc7914412b7661d56c4b5e3b0058db42692da5f5166 is running failed: container process not found" probeType="Readiness" pod="openstack-kuttl-tests/ovn-northd-0" podUID="a26ed42f-71e7-4c92-8b7e-ad194358d3e5" containerName="ovn-northd" Dec 03 12:30:10 crc kubenswrapper[5052]: I1203 12:30:10.087745 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/memcached-0" Dec 03 12:30:10 crc kubenswrapper[5052]: E1203 12:30:10.094710 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="46246c5deed2c10d0bc81011e4be47e5c70c228f4dda6fb9a1b20810aff514c7" cmd=["/bin/bash","/var/lib/operator-scripts/mysql_probe.sh","readiness"] Dec 03 12:30:10 crc kubenswrapper[5052]: E1203 12:30:10.096328 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="46246c5deed2c10d0bc81011e4be47e5c70c228f4dda6fb9a1b20810aff514c7" cmd=["/bin/bash","/var/lib/operator-scripts/mysql_probe.sh","readiness"] Dec 03 12:30:10 crc kubenswrapper[5052]: E1203 12:30:10.097690 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="46246c5deed2c10d0bc81011e4be47e5c70c228f4dda6fb9a1b20810aff514c7" cmd=["/bin/bash","/var/lib/operator-scripts/mysql_probe.sh","readiness"] Dec 03 12:30:10 crc kubenswrapper[5052]: E1203 12:30:10.097727 5052 prober.go:104] "Probe errored" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" probeType="Readiness" pod="openstack-kuttl-tests/openstack-cell1-galera-0" podUID="1de1d25b-cbcc-4d08-a7cd-aa54ac71aecb" containerName="galera" Dec 03 12:30:10 crc kubenswrapper[5052]: I1203 12:30:10.162517 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_ovn-northd-0_a26ed42f-71e7-4c92-8b7e-ad194358d3e5/ovn-northd/0.log" Dec 03 12:30:10 crc kubenswrapper[5052]: I1203 12:30:10.162594 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:30:10 crc kubenswrapper[5052]: I1203 12:30:10.250822 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:30:10 crc kubenswrapper[5052]: I1203 12:30:10.290251 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/214a26fb-e9b2-4a87-8fa5-ae2ed0d8a828-scripts\") pod \"214a26fb-e9b2-4a87-8fa5-ae2ed0d8a828\" (UID: \"214a26fb-e9b2-4a87-8fa5-ae2ed0d8a828\") " Dec 03 12:30:10 crc kubenswrapper[5052]: I1203 12:30:10.290312 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/214a26fb-e9b2-4a87-8fa5-ae2ed0d8a828-etc-machine-id\") pod \"214a26fb-e9b2-4a87-8fa5-ae2ed0d8a828\" (UID: \"214a26fb-e9b2-4a87-8fa5-ae2ed0d8a828\") " Dec 03 12:30:10 crc kubenswrapper[5052]: I1203 12:30:10.290375 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/214a26fb-e9b2-4a87-8fa5-ae2ed0d8a828-combined-ca-bundle\") pod \"214a26fb-e9b2-4a87-8fa5-ae2ed0d8a828\" (UID: \"214a26fb-e9b2-4a87-8fa5-ae2ed0d8a828\") " Dec 03 12:30:10 crc kubenswrapper[5052]: I1203 12:30:10.290401 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/214a26fb-e9b2-4a87-8fa5-ae2ed0d8a828-config-data\") pod \"214a26fb-e9b2-4a87-8fa5-ae2ed0d8a828\" (UID: \"214a26fb-e9b2-4a87-8fa5-ae2ed0d8a828\") " Dec 03 12:30:10 crc kubenswrapper[5052]: I1203 12:30:10.290454 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/214a26fb-e9b2-4a87-8fa5-ae2ed0d8a828-config-data-custom\") pod \"214a26fb-e9b2-4a87-8fa5-ae2ed0d8a828\" (UID: \"214a26fb-e9b2-4a87-8fa5-ae2ed0d8a828\") " Dec 03 12:30:10 crc kubenswrapper[5052]: I1203 12:30:10.290495 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bkz7n\" (UniqueName: \"kubernetes.io/projected/214a26fb-e9b2-4a87-8fa5-ae2ed0d8a828-kube-api-access-bkz7n\") pod \"214a26fb-e9b2-4a87-8fa5-ae2ed0d8a828\" (UID: \"214a26fb-e9b2-4a87-8fa5-ae2ed0d8a828\") " Dec 03 12:30:10 crc kubenswrapper[5052]: I1203 12:30:10.290552 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovn-rundir\" (UniqueName: \"kubernetes.io/empty-dir/a26ed42f-71e7-4c92-8b7e-ad194358d3e5-ovn-rundir\") pod \"a26ed42f-71e7-4c92-8b7e-ad194358d3e5\" (UID: \"a26ed42f-71e7-4c92-8b7e-ad194358d3e5\") " Dec 03 12:30:10 crc kubenswrapper[5052]: I1203 12:30:10.290590 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a26ed42f-71e7-4c92-8b7e-ad194358d3e5-combined-ca-bundle\") pod \"a26ed42f-71e7-4c92-8b7e-ad194358d3e5\" (UID: \"a26ed42f-71e7-4c92-8b7e-ad194358d3e5\") " Dec 03 12:30:10 crc kubenswrapper[5052]: I1203 12:30:10.290638 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a26ed42f-71e7-4c92-8b7e-ad194358d3e5-config\") pod \"a26ed42f-71e7-4c92-8b7e-ad194358d3e5\" (UID: \"a26ed42f-71e7-4c92-8b7e-ad194358d3e5\") " Dec 03 12:30:10 crc kubenswrapper[5052]: I1203 12:30:10.290688 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovn-northd-tls-certs\" (UniqueName: \"kubernetes.io/secret/a26ed42f-71e7-4c92-8b7e-ad194358d3e5-ovn-northd-tls-certs\") pod \"a26ed42f-71e7-4c92-8b7e-ad194358d3e5\" (UID: \"a26ed42f-71e7-4c92-8b7e-ad194358d3e5\") " Dec 03 12:30:10 crc kubenswrapper[5052]: I1203 12:30:10.290785 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fmjns\" (UniqueName: \"kubernetes.io/projected/a26ed42f-71e7-4c92-8b7e-ad194358d3e5-kube-api-access-fmjns\") pod \"a26ed42f-71e7-4c92-8b7e-ad194358d3e5\" (UID: \"a26ed42f-71e7-4c92-8b7e-ad194358d3e5\") " Dec 03 12:30:10 crc kubenswrapper[5052]: I1203 12:30:10.290823 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/a26ed42f-71e7-4c92-8b7e-ad194358d3e5-metrics-certs-tls-certs\") pod \"a26ed42f-71e7-4c92-8b7e-ad194358d3e5\" (UID: \"a26ed42f-71e7-4c92-8b7e-ad194358d3e5\") " Dec 03 12:30:10 crc kubenswrapper[5052]: I1203 12:30:10.290863 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/a26ed42f-71e7-4c92-8b7e-ad194358d3e5-scripts\") pod \"a26ed42f-71e7-4c92-8b7e-ad194358d3e5\" (UID: \"a26ed42f-71e7-4c92-8b7e-ad194358d3e5\") " Dec 03 12:30:10 crc kubenswrapper[5052]: I1203 12:30:10.292104 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a26ed42f-71e7-4c92-8b7e-ad194358d3e5-scripts" (OuterVolumeSpecName: "scripts") pod "a26ed42f-71e7-4c92-8b7e-ad194358d3e5" (UID: "a26ed42f-71e7-4c92-8b7e-ad194358d3e5"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:30:10 crc kubenswrapper[5052]: I1203 12:30:10.302314 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/214a26fb-e9b2-4a87-8fa5-ae2ed0d8a828-etc-machine-id" (OuterVolumeSpecName: "etc-machine-id") pod "214a26fb-e9b2-4a87-8fa5-ae2ed0d8a828" (UID: "214a26fb-e9b2-4a87-8fa5-ae2ed0d8a828"). InnerVolumeSpecName "etc-machine-id". PluginName "kubernetes.io/host-path", VolumeGidValue "" Dec 03 12:30:10 crc kubenswrapper[5052]: I1203 12:30:10.302655 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a26ed42f-71e7-4c92-8b7e-ad194358d3e5-config" (OuterVolumeSpecName: "config") pod "a26ed42f-71e7-4c92-8b7e-ad194358d3e5" (UID: "a26ed42f-71e7-4c92-8b7e-ad194358d3e5"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:30:10 crc kubenswrapper[5052]: I1203 12:30:10.314184 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a26ed42f-71e7-4c92-8b7e-ad194358d3e5-ovn-rundir" (OuterVolumeSpecName: "ovn-rundir") pod "a26ed42f-71e7-4c92-8b7e-ad194358d3e5" (UID: "a26ed42f-71e7-4c92-8b7e-ad194358d3e5"). InnerVolumeSpecName "ovn-rundir". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:30:10 crc kubenswrapper[5052]: I1203 12:30:10.316994 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/214a26fb-e9b2-4a87-8fa5-ae2ed0d8a828-scripts" (OuterVolumeSpecName: "scripts") pod "214a26fb-e9b2-4a87-8fa5-ae2ed0d8a828" (UID: "214a26fb-e9b2-4a87-8fa5-ae2ed0d8a828"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:10 crc kubenswrapper[5052]: I1203 12:30:10.346973 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-5c6478db8b-bt2rw" event={"ID":"8bddf2b1-8116-4b6e-9cd8-be8fe1cd4b64","Type":"ContainerStarted","Data":"5f22191c711f94909400c3cf19b845a8d0a3f693f6879266161b330e5d69cafc"} Dec 03 12:30:10 crc kubenswrapper[5052]: I1203 12:30:10.349264 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/placement-5c6478db8b-bt2rw" podUID="8bddf2b1-8116-4b6e-9cd8-be8fe1cd4b64" containerName="placement-log" containerID="cri-o://33f8dbc2efb3ac8c4348e12af03e0b1919430c85f4a7565b4e5fc747718afe50" gracePeriod=30 Dec 03 12:30:10 crc kubenswrapper[5052]: I1203 12:30:10.349425 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/placement-5c6478db8b-bt2rw" Dec 03 12:30:10 crc kubenswrapper[5052]: I1203 12:30:10.350077 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/placement-5c6478db8b-bt2rw" podUID="8bddf2b1-8116-4b6e-9cd8-be8fe1cd4b64" containerName="placement-api" containerID="cri-o://5f22191c711f94909400c3cf19b845a8d0a3f693f6879266161b330e5d69cafc" gracePeriod=30 Dec 03 12:30:10 crc kubenswrapper[5052]: I1203 12:30:10.350270 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/placement-5c6478db8b-bt2rw" Dec 03 12:30:10 crc kubenswrapper[5052]: I1203 12:30:10.394789 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a26ed42f-71e7-4c92-8b7e-ad194358d3e5-kube-api-access-fmjns" (OuterVolumeSpecName: "kube-api-access-fmjns") pod "a26ed42f-71e7-4c92-8b7e-ad194358d3e5" (UID: "a26ed42f-71e7-4c92-8b7e-ad194358d3e5"). InnerVolumeSpecName "kube-api-access-fmjns". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:30:10 crc kubenswrapper[5052]: I1203 12:30:10.397565 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-fmjns\" (UniqueName: \"kubernetes.io/projected/a26ed42f-71e7-4c92-8b7e-ad194358d3e5-kube-api-access-fmjns\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:10 crc kubenswrapper[5052]: I1203 12:30:10.397597 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/a26ed42f-71e7-4c92-8b7e-ad194358d3e5-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:10 crc kubenswrapper[5052]: I1203 12:30:10.397608 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/214a26fb-e9b2-4a87-8fa5-ae2ed0d8a828-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:10 crc kubenswrapper[5052]: I1203 12:30:10.397619 5052 reconciler_common.go:293] "Volume detached for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/214a26fb-e9b2-4a87-8fa5-ae2ed0d8a828-etc-machine-id\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:10 crc kubenswrapper[5052]: I1203 12:30:10.397632 5052 reconciler_common.go:293] "Volume detached for volume \"ovn-rundir\" (UniqueName: \"kubernetes.io/empty-dir/a26ed42f-71e7-4c92-8b7e-ad194358d3e5-ovn-rundir\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:10 crc kubenswrapper[5052]: I1203 12:30:10.397643 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a26ed42f-71e7-4c92-8b7e-ad194358d3e5-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:10 crc kubenswrapper[5052]: I1203 12:30:10.397641 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/214a26fb-e9b2-4a87-8fa5-ae2ed0d8a828-config-data-custom" (OuterVolumeSpecName: "config-data-custom") pod "214a26fb-e9b2-4a87-8fa5-ae2ed0d8a828" (UID: "214a26fb-e9b2-4a87-8fa5-ae2ed0d8a828"). InnerVolumeSpecName "config-data-custom". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:10 crc kubenswrapper[5052]: I1203 12:30:10.404804 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/214a26fb-e9b2-4a87-8fa5-ae2ed0d8a828-kube-api-access-bkz7n" (OuterVolumeSpecName: "kube-api-access-bkz7n") pod "214a26fb-e9b2-4a87-8fa5-ae2ed0d8a828" (UID: "214a26fb-e9b2-4a87-8fa5-ae2ed0d8a828"). InnerVolumeSpecName "kube-api-access-bkz7n". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:30:10 crc kubenswrapper[5052]: I1203 12:30:10.458960 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/placement-5c6478db8b-bt2rw" podUID="8bddf2b1-8116-4b6e-9cd8-be8fe1cd4b64" containerName="placement-log" probeResult="failure" output="Get \"https://10.217.0.197:8778/\": read tcp 10.217.0.2:49140->10.217.0.197:8778: read: connection reset by peer" Dec 03 12:30:10 crc kubenswrapper[5052]: I1203 12:30:10.488991 5052 generic.go:334] "Generic (PLEG): container finished" podID="71df5bf1-7fd8-4a4d-b0f5-85e3911ae8a9" containerID="994203d634c00abfeeeb2db379076b53d6f741dba1b79d202d758f959e9439c6" exitCode=0 Dec 03 12:30:10 crc kubenswrapper[5052]: I1203 12:30:10.489076 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-keystone-listener-58b6cdf6bb-5xxd8" event={"ID":"71df5bf1-7fd8-4a4d-b0f5-85e3911ae8a9","Type":"ContainerDied","Data":"994203d634c00abfeeeb2db379076b53d6f741dba1b79d202d758f959e9439c6"} Dec 03 12:30:10 crc kubenswrapper[5052]: I1203 12:30:10.497854 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:30:10 crc kubenswrapper[5052]: I1203 12:30:10.525153 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/214a26fb-e9b2-4a87-8fa5-ae2ed0d8a828-config-data-custom\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:10 crc kubenswrapper[5052]: I1203 12:30:10.525182 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-bkz7n\" (UniqueName: \"kubernetes.io/projected/214a26fb-e9b2-4a87-8fa5-ae2ed0d8a828-kube-api-access-bkz7n\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:10 crc kubenswrapper[5052]: I1203 12:30:10.582964 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-7dffdbf8-hzmbm" event={"ID":"53fe6403-27db-44b7-a9a6-787afbc3bbb8","Type":"ContainerStarted","Data":"d3907f3c074c217040963cd0054e9403720f2049ff0f6f8d3ee8174b0d8a46a0"} Dec 03 12:30:10 crc kubenswrapper[5052]: I1203 12:30:10.583128 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/placement-7dffdbf8-hzmbm" podUID="53fe6403-27db-44b7-a9a6-787afbc3bbb8" containerName="placement-log" containerID="cri-o://c110d80f3a367aca9b254271248cc75eddf2eb31290f7f48d1f7ccab8d5d16fd" gracePeriod=30 Dec 03 12:30:10 crc kubenswrapper[5052]: I1203 12:30:10.583353 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/placement-7dffdbf8-hzmbm" Dec 03 12:30:10 crc kubenswrapper[5052]: I1203 12:30:10.584722 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/placement-7dffdbf8-hzmbm" podUID="53fe6403-27db-44b7-a9a6-787afbc3bbb8" containerName="placement-api" containerID="cri-o://d3907f3c074c217040963cd0054e9403720f2049ff0f6f8d3ee8174b0d8a46a0" gracePeriod=30 Dec 03 12:30:10 crc kubenswrapper[5052]: I1203 12:30:10.585075 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/placement-7dffdbf8-hzmbm" Dec 03 12:30:10 crc kubenswrapper[5052]: I1203 12:30:10.617442 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-0"] Dec 03 12:30:10 crc kubenswrapper[5052]: I1203 12:30:10.624380 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/placement-7dffdbf8-hzmbm" podUID="53fe6403-27db-44b7-a9a6-787afbc3bbb8" containerName="placement-log" probeResult="failure" output="Get \"https://10.217.0.205:8778/\": read tcp 10.217.0.2:33328->10.217.0.205:8778: read: connection reset by peer" Dec 03 12:30:10 crc kubenswrapper[5052]: I1203 12:30:10.637309 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"214a26fb-e9b2-4a87-8fa5-ae2ed0d8a828","Type":"ContainerDied","Data":"fbd118daf512469b0a39320f016c6056298c40b87f1e280084246d4ba8f21cf7"} Dec 03 12:30:10 crc kubenswrapper[5052]: I1203 12:30:10.637365 5052 scope.go:117] "RemoveContainer" containerID="27a238af716222ccb80b51b0ddb93658176d3aa5f8db130deec5a43ab67bb332" Dec 03 12:30:10 crc kubenswrapper[5052]: I1203 12:30:10.637486 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:30:10 crc kubenswrapper[5052]: I1203 12:30:10.661629 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:30:10 crc kubenswrapper[5052]: I1203 12:30:10.678219 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-worker-644558fbc5-4gtxj" event={"ID":"8ac6e8f0-d49c-4a4c-846a-6cec226bd360","Type":"ContainerStarted","Data":"4893c6e7800c80d6b9f1c1b9d2a79b5fd298a43e5935cb9d7b3f6072575ccfc6"} Dec 03 12:30:10 crc kubenswrapper[5052]: I1203 12:30:10.708377 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:30:10 crc kubenswrapper[5052]: I1203 12:30:10.710878 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"39117149-c15d-4753-9324-8a42abe5b007","Type":"ContainerStarted","Data":"53093380d15a12407af002e8218c329f263122e535b3f7a98513361ca8b1c783"} Dec 03 12:30:10 crc kubenswrapper[5052]: I1203 12:30:10.732920 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a26ed42f-71e7-4c92-8b7e-ad194358d3e5-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "a26ed42f-71e7-4c92-8b7e-ad194358d3e5" (UID: "a26ed42f-71e7-4c92-8b7e-ad194358d3e5"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:10 crc kubenswrapper[5052]: I1203 12:30:10.742673 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:30:10 crc kubenswrapper[5052]: I1203 12:30:10.762977 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-65444bfc86-th5pw" event={"ID":"cfd8af50-a644-4b24-8483-dd344f2218e4","Type":"ContainerStarted","Data":"dc83501080d7c2386bc4543b9befadc35e0e843ede852c450da98e997424e43e"} Dec 03 12:30:10 crc kubenswrapper[5052]: I1203 12:30:10.765433 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/keystone-564bc88b6-bvll5"] Dec 03 12:30:10 crc kubenswrapper[5052]: I1203 12:30:10.765909 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/keystone-564bc88b6-bvll5" podUID="c0ce9ced-33b0-4879-81a4-b5c066ec0631" containerName="keystone-api" containerID="cri-o://589675564f1f842a7ea700f8f790adcdebd983109491fa331d941686f179e466" gracePeriod=30 Dec 03 12:30:10 crc kubenswrapper[5052]: I1203 12:30:10.772754 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/placement-5c6478db8b-bt2rw" podStartSLOduration=16.772738301 podStartE2EDuration="16.772738301s" podCreationTimestamp="2025-12-03 12:29:54 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:30:10.457877674 +0000 UTC m=+4201.046119438" watchObservedRunningTime="2025-12-03 12:30:10.772738301 +0000 UTC m=+4201.360980065" Dec 03 12:30:10 crc kubenswrapper[5052]: I1203 12:30:10.793538 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_ovn-northd-0_a26ed42f-71e7-4c92-8b7e-ad194358d3e5/ovn-northd/0.log" Dec 03 12:30:10 crc kubenswrapper[5052]: I1203 12:30:10.793629 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovn-northd-0" event={"ID":"a26ed42f-71e7-4c92-8b7e-ad194358d3e5","Type":"ContainerDied","Data":"f49b4773f97825cb5a1553388c57cb6ce5ee61bcf03f1f22983c2a6b3594e891"} Dec 03 12:30:10 crc kubenswrapper[5052]: I1203 12:30:10.793722 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:30:10 crc kubenswrapper[5052]: I1203 12:30:10.814676 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/keystone-564bc88b6-bvll5" podUID="c0ce9ced-33b0-4879-81a4-b5c066ec0631" containerName="keystone-api" probeResult="failure" output="Get \"https://10.217.0.206:5000/v3\": EOF" Dec 03 12:30:10 crc kubenswrapper[5052]: I1203 12:30:10.819023 5052 generic.go:334] "Generic (PLEG): container finished" podID="886ea0a4-5096-4ea8-a74c-de243d1f4f40" containerID="71e595450c5a81e73076dfa57785d77ddaafc47e00972e087f567c2fa539835b" exitCode=0 Dec 03 12:30:10 crc kubenswrapper[5052]: I1203 12:30:10.819166 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-db474c7dd-tg7mq" event={"ID":"886ea0a4-5096-4ea8-a74c-de243d1f4f40","Type":"ContainerDied","Data":"71e595450c5a81e73076dfa57785d77ddaafc47e00972e087f567c2fa539835b"} Dec 03 12:30:10 crc kubenswrapper[5052]: I1203 12:30:10.819816 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/214a26fb-e9b2-4a87-8fa5-ae2ed0d8a828-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "214a26fb-e9b2-4a87-8fa5-ae2ed0d8a828" (UID: "214a26fb-e9b2-4a87-8fa5-ae2ed0d8a828"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:10 crc kubenswrapper[5052]: I1203 12:30:10.835764 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-0"] Dec 03 12:30:10 crc kubenswrapper[5052]: I1203 12:30:10.839643 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/214a26fb-e9b2-4a87-8fa5-ae2ed0d8a828-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:10 crc kubenswrapper[5052]: I1203 12:30:10.839680 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a26ed42f-71e7-4c92-8b7e-ad194358d3e5-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:10 crc kubenswrapper[5052]: I1203 12:30:10.851732 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"4314928d-a43f-4b38-9c72-63977e59345b","Type":"ContainerStarted","Data":"50954953a47c0484d7617bce920a6b97d031a9d9d0a476b236d1e1bdff5f0cfc"} Dec 03 12:30:10 crc kubenswrapper[5052]: I1203 12:30:10.854388 5052 scope.go:117] "RemoveContainer" containerID="b6c09457c183d144bb440f045e7a0230c87297b84897525513a2e671062b9c96" Dec 03 12:30:10 crc kubenswrapper[5052]: I1203 12:30:10.859737 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-nb-0" event={"ID":"98f722f9-fa25-40b6-84ba-96127d07293c","Type":"ContainerStarted","Data":"c30cd5ee45f2cebbe8afbf744e6817d788d7a3b824716ed969e21464a285f555"} Dec 03 12:30:10 crc kubenswrapper[5052]: I1203 12:30:10.860327 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ovsdbserver-nb-0" podUID="98f722f9-fa25-40b6-84ba-96127d07293c" containerName="openstack-network-exporter" containerID="cri-o://c30cd5ee45f2cebbe8afbf744e6817d788d7a3b824716ed969e21464a285f555" gracePeriod=300 Dec 03 12:30:10 crc kubenswrapper[5052]: I1203 12:30:10.867587 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-688bd8fb5d-pjs4f" event={"ID":"a0e45e1a-ced9-4958-9a75-24a69280f679","Type":"ContainerStarted","Data":"009d530d399c78fb2e9155e989a69050f595b2f0a1be3bf13247c7f1d91a158a"} Dec 03 12:30:10 crc kubenswrapper[5052]: I1203 12:30:10.870092 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/keystone-6b8b855786-bltgj"] Dec 03 12:30:10 crc kubenswrapper[5052]: E1203 12:30:10.871665 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="a26ed42f-71e7-4c92-8b7e-ad194358d3e5" containerName="ovn-northd" Dec 03 12:30:10 crc kubenswrapper[5052]: I1203 12:30:10.871762 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="a26ed42f-71e7-4c92-8b7e-ad194358d3e5" containerName="ovn-northd" Dec 03 12:30:10 crc kubenswrapper[5052]: E1203 12:30:10.871804 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="a26ed42f-71e7-4c92-8b7e-ad194358d3e5" containerName="openstack-network-exporter" Dec 03 12:30:10 crc kubenswrapper[5052]: I1203 12:30:10.871814 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="a26ed42f-71e7-4c92-8b7e-ad194358d3e5" containerName="openstack-network-exporter" Dec 03 12:30:10 crc kubenswrapper[5052]: E1203 12:30:10.871829 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="214a26fb-e9b2-4a87-8fa5-ae2ed0d8a828" containerName="probe" Dec 03 12:30:10 crc kubenswrapper[5052]: I1203 12:30:10.871836 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="214a26fb-e9b2-4a87-8fa5-ae2ed0d8a828" containerName="probe" Dec 03 12:30:10 crc kubenswrapper[5052]: E1203 12:30:10.871881 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="214a26fb-e9b2-4a87-8fa5-ae2ed0d8a828" containerName="cinder-scheduler" Dec 03 12:30:10 crc kubenswrapper[5052]: I1203 12:30:10.871891 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="214a26fb-e9b2-4a87-8fa5-ae2ed0d8a828" containerName="cinder-scheduler" Dec 03 12:30:10 crc kubenswrapper[5052]: I1203 12:30:10.872150 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="a26ed42f-71e7-4c92-8b7e-ad194358d3e5" containerName="openstack-network-exporter" Dec 03 12:30:10 crc kubenswrapper[5052]: I1203 12:30:10.872184 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="214a26fb-e9b2-4a87-8fa5-ae2ed0d8a828" containerName="probe" Dec 03 12:30:10 crc kubenswrapper[5052]: I1203 12:30:10.872205 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="a26ed42f-71e7-4c92-8b7e-ad194358d3e5" containerName="ovn-northd" Dec 03 12:30:10 crc kubenswrapper[5052]: I1203 12:30:10.872227 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="214a26fb-e9b2-4a87-8fa5-ae2ed0d8a828" containerName="cinder-scheduler" Dec 03 12:30:10 crc kubenswrapper[5052]: I1203 12:30:10.876002 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-6b8b855786-bltgj" Dec 03 12:30:10 crc kubenswrapper[5052]: I1203 12:30:10.898656 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-6b8b855786-bltgj"] Dec 03 12:30:10 crc kubenswrapper[5052]: I1203 12:30:10.905941 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:30:10 crc kubenswrapper[5052]: I1203 12:30:10.929424 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-sb-0"] Dec 03 12:30:10 crc kubenswrapper[5052]: I1203 12:30:10.933184 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/placement-7dffdbf8-hzmbm" podStartSLOduration=13.93316421 podStartE2EDuration="13.93316421s" podCreationTimestamp="2025-12-03 12:29:57 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:30:10.661368832 +0000 UTC m=+4201.249610596" watchObservedRunningTime="2025-12-03 12:30:10.93316421 +0000 UTC m=+4201.521405974" Dec 03 12:30:10 crc kubenswrapper[5052]: I1203 12:30:10.941426 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/43933abc-db67-4ce5-9f30-6f8dbf3f59c8-config-data\") pod \"keystone-6b8b855786-bltgj\" (UID: \"43933abc-db67-4ce5-9f30-6f8dbf3f59c8\") " pod="openstack-kuttl-tests/keystone-6b8b855786-bltgj" Dec 03 12:30:10 crc kubenswrapper[5052]: I1203 12:30:10.941511 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/43933abc-db67-4ce5-9f30-6f8dbf3f59c8-credential-keys\") pod \"keystone-6b8b855786-bltgj\" (UID: \"43933abc-db67-4ce5-9f30-6f8dbf3f59c8\") " pod="openstack-kuttl-tests/keystone-6b8b855786-bltgj" Dec 03 12:30:10 crc kubenswrapper[5052]: I1203 12:30:10.947339 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/43933abc-db67-4ce5-9f30-6f8dbf3f59c8-scripts\") pod \"keystone-6b8b855786-bltgj\" (UID: \"43933abc-db67-4ce5-9f30-6f8dbf3f59c8\") " pod="openstack-kuttl-tests/keystone-6b8b855786-bltgj" Dec 03 12:30:10 crc kubenswrapper[5052]: I1203 12:30:10.947383 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/43933abc-db67-4ce5-9f30-6f8dbf3f59c8-public-tls-certs\") pod \"keystone-6b8b855786-bltgj\" (UID: \"43933abc-db67-4ce5-9f30-6f8dbf3f59c8\") " pod="openstack-kuttl-tests/keystone-6b8b855786-bltgj" Dec 03 12:30:10 crc kubenswrapper[5052]: I1203 12:30:10.947421 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/43933abc-db67-4ce5-9f30-6f8dbf3f59c8-internal-tls-certs\") pod \"keystone-6b8b855786-bltgj\" (UID: \"43933abc-db67-4ce5-9f30-6f8dbf3f59c8\") " pod="openstack-kuttl-tests/keystone-6b8b855786-bltgj" Dec 03 12:30:10 crc kubenswrapper[5052]: I1203 12:30:10.947529 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xq8cj\" (UniqueName: \"kubernetes.io/projected/43933abc-db67-4ce5-9f30-6f8dbf3f59c8-kube-api-access-xq8cj\") pod \"keystone-6b8b855786-bltgj\" (UID: \"43933abc-db67-4ce5-9f30-6f8dbf3f59c8\") " pod="openstack-kuttl-tests/keystone-6b8b855786-bltgj" Dec 03 12:30:10 crc kubenswrapper[5052]: I1203 12:30:10.947741 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/43933abc-db67-4ce5-9f30-6f8dbf3f59c8-fernet-keys\") pod \"keystone-6b8b855786-bltgj\" (UID: \"43933abc-db67-4ce5-9f30-6f8dbf3f59c8\") " pod="openstack-kuttl-tests/keystone-6b8b855786-bltgj" Dec 03 12:30:10 crc kubenswrapper[5052]: I1203 12:30:10.947895 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/43933abc-db67-4ce5-9f30-6f8dbf3f59c8-combined-ca-bundle\") pod \"keystone-6b8b855786-bltgj\" (UID: \"43933abc-db67-4ce5-9f30-6f8dbf3f59c8\") " pod="openstack-kuttl-tests/keystone-6b8b855786-bltgj" Dec 03 12:30:10 crc kubenswrapper[5052]: I1203 12:30:10.955949 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/barbican-worker-644558fbc5-4gtxj" podStartSLOduration=14.955932059 podStartE2EDuration="14.955932059s" podCreationTimestamp="2025-12-03 12:29:56 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:30:10.724074262 +0000 UTC m=+4201.312316036" watchObservedRunningTime="2025-12-03 12:30:10.955932059 +0000 UTC m=+4201.544173823" Dec 03 12:30:10 crc kubenswrapper[5052]: I1203 12:30:10.994441 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-worker-679bfdf7b7-tg6h7"] Dec 03 12:30:10 crc kubenswrapper[5052]: I1203 12:30:10.995036 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/barbican-worker-679bfdf7b7-tg6h7" podUID="72a32b4d-d61d-43a5-a45d-c1fd4bad672d" containerName="barbican-worker-log" containerID="cri-o://902a5085d0a5d4b6d6bd379e7d79548b8cf29e5be3d818b61c582ef847c19f19" gracePeriod=30 Dec 03 12:30:10 crc kubenswrapper[5052]: I1203 12:30:10.995542 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/barbican-worker-679bfdf7b7-tg6h7" podUID="72a32b4d-d61d-43a5-a45d-c1fd4bad672d" containerName="barbican-worker" containerID="cri-o://6d2a1c1187cb71ec6764a5792108b52b3d8a71f435dceee72f6a125558106e90" gracePeriod=30 Dec 03 12:30:11 crc kubenswrapper[5052]: I1203 12:30:11.016819 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/ovsdbserver-nb-0" podStartSLOduration=12.016799367 podStartE2EDuration="12.016799367s" podCreationTimestamp="2025-12-03 12:29:59 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:30:10.895537046 +0000 UTC m=+4201.483778840" watchObservedRunningTime="2025-12-03 12:30:11.016799367 +0000 UTC m=+4201.605041131" Dec 03 12:30:11 crc kubenswrapper[5052]: I1203 12:30:11.051180 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/43933abc-db67-4ce5-9f30-6f8dbf3f59c8-scripts\") pod \"keystone-6b8b855786-bltgj\" (UID: \"43933abc-db67-4ce5-9f30-6f8dbf3f59c8\") " pod="openstack-kuttl-tests/keystone-6b8b855786-bltgj" Dec 03 12:30:11 crc kubenswrapper[5052]: I1203 12:30:11.051233 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/43933abc-db67-4ce5-9f30-6f8dbf3f59c8-public-tls-certs\") pod \"keystone-6b8b855786-bltgj\" (UID: \"43933abc-db67-4ce5-9f30-6f8dbf3f59c8\") " pod="openstack-kuttl-tests/keystone-6b8b855786-bltgj" Dec 03 12:30:11 crc kubenswrapper[5052]: I1203 12:30:11.051264 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/43933abc-db67-4ce5-9f30-6f8dbf3f59c8-internal-tls-certs\") pod \"keystone-6b8b855786-bltgj\" (UID: \"43933abc-db67-4ce5-9f30-6f8dbf3f59c8\") " pod="openstack-kuttl-tests/keystone-6b8b855786-bltgj" Dec 03 12:30:11 crc kubenswrapper[5052]: I1203 12:30:11.051315 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-xq8cj\" (UniqueName: \"kubernetes.io/projected/43933abc-db67-4ce5-9f30-6f8dbf3f59c8-kube-api-access-xq8cj\") pod \"keystone-6b8b855786-bltgj\" (UID: \"43933abc-db67-4ce5-9f30-6f8dbf3f59c8\") " pod="openstack-kuttl-tests/keystone-6b8b855786-bltgj" Dec 03 12:30:11 crc kubenswrapper[5052]: I1203 12:30:11.051396 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/43933abc-db67-4ce5-9f30-6f8dbf3f59c8-fernet-keys\") pod \"keystone-6b8b855786-bltgj\" (UID: \"43933abc-db67-4ce5-9f30-6f8dbf3f59c8\") " pod="openstack-kuttl-tests/keystone-6b8b855786-bltgj" Dec 03 12:30:11 crc kubenswrapper[5052]: I1203 12:30:11.051470 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/43933abc-db67-4ce5-9f30-6f8dbf3f59c8-combined-ca-bundle\") pod \"keystone-6b8b855786-bltgj\" (UID: \"43933abc-db67-4ce5-9f30-6f8dbf3f59c8\") " pod="openstack-kuttl-tests/keystone-6b8b855786-bltgj" Dec 03 12:30:11 crc kubenswrapper[5052]: I1203 12:30:11.051503 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/43933abc-db67-4ce5-9f30-6f8dbf3f59c8-config-data\") pod \"keystone-6b8b855786-bltgj\" (UID: \"43933abc-db67-4ce5-9f30-6f8dbf3f59c8\") " pod="openstack-kuttl-tests/keystone-6b8b855786-bltgj" Dec 03 12:30:11 crc kubenswrapper[5052]: I1203 12:30:11.051594 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/43933abc-db67-4ce5-9f30-6f8dbf3f59c8-credential-keys\") pod \"keystone-6b8b855786-bltgj\" (UID: \"43933abc-db67-4ce5-9f30-6f8dbf3f59c8\") " pod="openstack-kuttl-tests/keystone-6b8b855786-bltgj" Dec 03 12:30:11 crc kubenswrapper[5052]: I1203 12:30:11.059998 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/43933abc-db67-4ce5-9f30-6f8dbf3f59c8-scripts\") pod \"keystone-6b8b855786-bltgj\" (UID: \"43933abc-db67-4ce5-9f30-6f8dbf3f59c8\") " pod="openstack-kuttl-tests/keystone-6b8b855786-bltgj" Dec 03 12:30:11 crc kubenswrapper[5052]: I1203 12:30:11.060225 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/43933abc-db67-4ce5-9f30-6f8dbf3f59c8-fernet-keys\") pod \"keystone-6b8b855786-bltgj\" (UID: \"43933abc-db67-4ce5-9f30-6f8dbf3f59c8\") " pod="openstack-kuttl-tests/keystone-6b8b855786-bltgj" Dec 03 12:30:11 crc kubenswrapper[5052]: I1203 12:30:11.060302 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/43933abc-db67-4ce5-9f30-6f8dbf3f59c8-config-data\") pod \"keystone-6b8b855786-bltgj\" (UID: \"43933abc-db67-4ce5-9f30-6f8dbf3f59c8\") " pod="openstack-kuttl-tests/keystone-6b8b855786-bltgj" Dec 03 12:30:11 crc kubenswrapper[5052]: I1203 12:30:11.062021 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/43933abc-db67-4ce5-9f30-6f8dbf3f59c8-credential-keys\") pod \"keystone-6b8b855786-bltgj\" (UID: \"43933abc-db67-4ce5-9f30-6f8dbf3f59c8\") " pod="openstack-kuttl-tests/keystone-6b8b855786-bltgj" Dec 03 12:30:11 crc kubenswrapper[5052]: I1203 12:30:11.066883 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/43933abc-db67-4ce5-9f30-6f8dbf3f59c8-public-tls-certs\") pod \"keystone-6b8b855786-bltgj\" (UID: \"43933abc-db67-4ce5-9f30-6f8dbf3f59c8\") " pod="openstack-kuttl-tests/keystone-6b8b855786-bltgj" Dec 03 12:30:11 crc kubenswrapper[5052]: I1203 12:30:11.068233 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ovsdbserver-nb-0" podUID="98f722f9-fa25-40b6-84ba-96127d07293c" containerName="ovsdbserver-nb" containerID="cri-o://b6e565e6b6e8e07c8000bbead3eed4e1cd45529d9dc3aa7fcb642cf668f780aa" gracePeriod=300 Dec 03 12:30:11 crc kubenswrapper[5052]: I1203 12:30:11.071080 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-xq8cj\" (UniqueName: \"kubernetes.io/projected/43933abc-db67-4ce5-9f30-6f8dbf3f59c8-kube-api-access-xq8cj\") pod \"keystone-6b8b855786-bltgj\" (UID: \"43933abc-db67-4ce5-9f30-6f8dbf3f59c8\") " pod="openstack-kuttl-tests/keystone-6b8b855786-bltgj" Dec 03 12:30:11 crc kubenswrapper[5052]: I1203 12:30:11.085576 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/43933abc-db67-4ce5-9f30-6f8dbf3f59c8-combined-ca-bundle\") pod \"keystone-6b8b855786-bltgj\" (UID: \"43933abc-db67-4ce5-9f30-6f8dbf3f59c8\") " pod="openstack-kuttl-tests/keystone-6b8b855786-bltgj" Dec 03 12:30:11 crc kubenswrapper[5052]: I1203 12:30:11.086067 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/43933abc-db67-4ce5-9f30-6f8dbf3f59c8-internal-tls-certs\") pod \"keystone-6b8b855786-bltgj\" (UID: \"43933abc-db67-4ce5-9f30-6f8dbf3f59c8\") " pod="openstack-kuttl-tests/keystone-6b8b855786-bltgj" Dec 03 12:30:11 crc kubenswrapper[5052]: I1203 12:30:11.092648 5052 scope.go:117] "RemoveContainer" containerID="911da6cb170ba67cad5584869214b5ef5f33b8f7400194bc4bc1ce4ca1a692f4" Dec 03 12:30:11 crc kubenswrapper[5052]: I1203 12:30:11.210747 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a26ed42f-71e7-4c92-8b7e-ad194358d3e5-ovn-northd-tls-certs" (OuterVolumeSpecName: "ovn-northd-tls-certs") pod "a26ed42f-71e7-4c92-8b7e-ad194358d3e5" (UID: "a26ed42f-71e7-4c92-8b7e-ad194358d3e5"). InnerVolumeSpecName "ovn-northd-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:11 crc kubenswrapper[5052]: I1203 12:30:11.218928 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a26ed42f-71e7-4c92-8b7e-ad194358d3e5-metrics-certs-tls-certs" (OuterVolumeSpecName: "metrics-certs-tls-certs") pod "a26ed42f-71e7-4c92-8b7e-ad194358d3e5" (UID: "a26ed42f-71e7-4c92-8b7e-ad194358d3e5"). InnerVolumeSpecName "metrics-certs-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:11 crc kubenswrapper[5052]: I1203 12:30:11.219417 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/214a26fb-e9b2-4a87-8fa5-ae2ed0d8a828-config-data" (OuterVolumeSpecName: "config-data") pod "214a26fb-e9b2-4a87-8fa5-ae2ed0d8a828" (UID: "214a26fb-e9b2-4a87-8fa5-ae2ed0d8a828"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:11 crc kubenswrapper[5052]: I1203 12:30:11.228042 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-6b8b855786-bltgj" Dec 03 12:30:11 crc kubenswrapper[5052]: I1203 12:30:11.260381 5052 reconciler_common.go:293] "Volume detached for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/a26ed42f-71e7-4c92-8b7e-ad194358d3e5-metrics-certs-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:11 crc kubenswrapper[5052]: I1203 12:30:11.260434 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/214a26fb-e9b2-4a87-8fa5-ae2ed0d8a828-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:11 crc kubenswrapper[5052]: I1203 12:30:11.260445 5052 reconciler_common.go:293] "Volume detached for volume \"ovn-northd-tls-certs\" (UniqueName: \"kubernetes.io/secret/a26ed42f-71e7-4c92-8b7e-ad194358d3e5-ovn-northd-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:11 crc kubenswrapper[5052]: I1203 12:30:11.434897 5052 scope.go:117] "RemoveContainer" containerID="93474533ee16033979340cc7914412b7661d56c4b5e3b0058db42692da5f5166" Dec 03 12:30:11 crc kubenswrapper[5052]: I1203 12:30:11.507804 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-keystone-listener-58b6cdf6bb-5xxd8" Dec 03 12:30:11 crc kubenswrapper[5052]: I1203 12:30:11.613565 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 12:30:11 crc kubenswrapper[5052]: I1203 12:30:11.704836 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/71df5bf1-7fd8-4a4d-b0f5-85e3911ae8a9-config-data-custom\") pod \"71df5bf1-7fd8-4a4d-b0f5-85e3911ae8a9\" (UID: \"71df5bf1-7fd8-4a4d-b0f5-85e3911ae8a9\") " Dec 03 12:30:11 crc kubenswrapper[5052]: I1203 12:30:11.704943 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/71df5bf1-7fd8-4a4d-b0f5-85e3911ae8a9-combined-ca-bundle\") pod \"71df5bf1-7fd8-4a4d-b0f5-85e3911ae8a9\" (UID: \"71df5bf1-7fd8-4a4d-b0f5-85e3911ae8a9\") " Dec 03 12:30:11 crc kubenswrapper[5052]: I1203 12:30:11.704988 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/71df5bf1-7fd8-4a4d-b0f5-85e3911ae8a9-logs\") pod \"71df5bf1-7fd8-4a4d-b0f5-85e3911ae8a9\" (UID: \"71df5bf1-7fd8-4a4d-b0f5-85e3911ae8a9\") " Dec 03 12:30:11 crc kubenswrapper[5052]: I1203 12:30:11.705035 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/71df5bf1-7fd8-4a4d-b0f5-85e3911ae8a9-config-data\") pod \"71df5bf1-7fd8-4a4d-b0f5-85e3911ae8a9\" (UID: \"71df5bf1-7fd8-4a4d-b0f5-85e3911ae8a9\") " Dec 03 12:30:11 crc kubenswrapper[5052]: I1203 12:30:11.705062 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vr9rf\" (UniqueName: \"kubernetes.io/projected/71df5bf1-7fd8-4a4d-b0f5-85e3911ae8a9-kube-api-access-vr9rf\") pod \"71df5bf1-7fd8-4a4d-b0f5-85e3911ae8a9\" (UID: \"71df5bf1-7fd8-4a4d-b0f5-85e3911ae8a9\") " Dec 03 12:30:11 crc kubenswrapper[5052]: I1203 12:30:11.717273 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71df5bf1-7fd8-4a4d-b0f5-85e3911ae8a9-logs" (OuterVolumeSpecName: "logs") pod "71df5bf1-7fd8-4a4d-b0f5-85e3911ae8a9" (UID: "71df5bf1-7fd8-4a4d-b0f5-85e3911ae8a9"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:30:11 crc kubenswrapper[5052]: I1203 12:30:11.776233 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/71df5bf1-7fd8-4a4d-b0f5-85e3911ae8a9-kube-api-access-vr9rf" (OuterVolumeSpecName: "kube-api-access-vr9rf") pod "71df5bf1-7fd8-4a4d-b0f5-85e3911ae8a9" (UID: "71df5bf1-7fd8-4a4d-b0f5-85e3911ae8a9"). InnerVolumeSpecName "kube-api-access-vr9rf". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:30:11 crc kubenswrapper[5052]: I1203 12:30:11.788801 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 12:30:11 crc kubenswrapper[5052]: I1203 12:30:11.788840 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 12:30:11 crc kubenswrapper[5052]: E1203 12:30:11.794080 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="71df5bf1-7fd8-4a4d-b0f5-85e3911ae8a9" containerName="barbican-keystone-listener-log" Dec 03 12:30:11 crc kubenswrapper[5052]: I1203 12:30:11.794119 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="71df5bf1-7fd8-4a4d-b0f5-85e3911ae8a9" containerName="barbican-keystone-listener-log" Dec 03 12:30:11 crc kubenswrapper[5052]: E1203 12:30:11.794177 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="71df5bf1-7fd8-4a4d-b0f5-85e3911ae8a9" containerName="barbican-keystone-listener" Dec 03 12:30:11 crc kubenswrapper[5052]: I1203 12:30:11.794186 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="71df5bf1-7fd8-4a4d-b0f5-85e3911ae8a9" containerName="barbican-keystone-listener" Dec 03 12:30:11 crc kubenswrapper[5052]: I1203 12:30:11.794451 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="71df5bf1-7fd8-4a4d-b0f5-85e3911ae8a9" containerName="barbican-keystone-listener" Dec 03 12:30:11 crc kubenswrapper[5052]: I1203 12:30:11.794471 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="71df5bf1-7fd8-4a4d-b0f5-85e3911ae8a9" containerName="barbican-keystone-listener-log" Dec 03 12:30:11 crc kubenswrapper[5052]: I1203 12:30:11.795686 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ovn-northd-0"] Dec 03 12:30:11 crc kubenswrapper[5052]: I1203 12:30:11.795814 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:30:11 crc kubenswrapper[5052]: I1203 12:30:11.798073 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/71df5bf1-7fd8-4a4d-b0f5-85e3911ae8a9-config-data-custom" (OuterVolumeSpecName: "config-data-custom") pod "71df5bf1-7fd8-4a4d-b0f5-85e3911ae8a9" (UID: "71df5bf1-7fd8-4a4d-b0f5-85e3911ae8a9"). InnerVolumeSpecName "config-data-custom". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:11 crc kubenswrapper[5052]: I1203 12:30:11.801338 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cinder-cinder-dockercfg-zd22l" Dec 03 12:30:11 crc kubenswrapper[5052]: I1203 12:30:11.801697 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cinder-scheduler-config-data" Dec 03 12:30:11 crc kubenswrapper[5052]: I1203 12:30:11.801944 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cinder-config-data" Dec 03 12:30:11 crc kubenswrapper[5052]: I1203 12:30:11.802170 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cinder-scripts" Dec 03 12:30:11 crc kubenswrapper[5052]: I1203 12:30:11.806727 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ovn-northd-0"] Dec 03 12:30:11 crc kubenswrapper[5052]: I1203 12:30:11.813360 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bz7zz\" (UniqueName: \"kubernetes.io/projected/bb1723c7-4e41-4b0e-b6b8-f8024dafdaee-kube-api-access-bz7zz\") pod \"cinder-scheduler-0\" (UID: \"bb1723c7-4e41-4b0e-b6b8-f8024dafdaee\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:30:11 crc kubenswrapper[5052]: I1203 12:30:11.813430 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/bb1723c7-4e41-4b0e-b6b8-f8024dafdaee-config-data-custom\") pod \"cinder-scheduler-0\" (UID: \"bb1723c7-4e41-4b0e-b6b8-f8024dafdaee\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:30:11 crc kubenswrapper[5052]: I1203 12:30:11.813475 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/bb1723c7-4e41-4b0e-b6b8-f8024dafdaee-config-data\") pod \"cinder-scheduler-0\" (UID: \"bb1723c7-4e41-4b0e-b6b8-f8024dafdaee\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:30:11 crc kubenswrapper[5052]: I1203 12:30:11.813734 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/bb1723c7-4e41-4b0e-b6b8-f8024dafdaee-scripts\") pod \"cinder-scheduler-0\" (UID: \"bb1723c7-4e41-4b0e-b6b8-f8024dafdaee\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:30:11 crc kubenswrapper[5052]: I1203 12:30:11.813776 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/bb1723c7-4e41-4b0e-b6b8-f8024dafdaee-etc-machine-id\") pod \"cinder-scheduler-0\" (UID: \"bb1723c7-4e41-4b0e-b6b8-f8024dafdaee\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:30:11 crc kubenswrapper[5052]: I1203 12:30:11.813821 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/bb1723c7-4e41-4b0e-b6b8-f8024dafdaee-combined-ca-bundle\") pod \"cinder-scheduler-0\" (UID: \"bb1723c7-4e41-4b0e-b6b8-f8024dafdaee\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:30:11 crc kubenswrapper[5052]: I1203 12:30:11.813934 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/71df5bf1-7fd8-4a4d-b0f5-85e3911ae8a9-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:11 crc kubenswrapper[5052]: I1203 12:30:11.813950 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-vr9rf\" (UniqueName: \"kubernetes.io/projected/71df5bf1-7fd8-4a4d-b0f5-85e3911ae8a9-kube-api-access-vr9rf\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:11 crc kubenswrapper[5052]: I1203 12:30:11.813961 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/71df5bf1-7fd8-4a4d-b0f5-85e3911ae8a9-config-data-custom\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:11 crc kubenswrapper[5052]: I1203 12:30:11.848866 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 12:30:11 crc kubenswrapper[5052]: I1203 12:30:11.864563 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ovn-northd-0"] Dec 03 12:30:11 crc kubenswrapper[5052]: I1203 12:30:11.866299 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:30:11 crc kubenswrapper[5052]: I1203 12:30:11.886300 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ovn-northd-0"] Dec 03 12:30:11 crc kubenswrapper[5052]: I1203 12:30:11.886437 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"ovnnorthd-config" Dec 03 12:30:11 crc kubenswrapper[5052]: I1203 12:30:11.886674 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ovnnorthd-ovnnorthd-dockercfg-lp65l" Dec 03 12:30:11 crc kubenswrapper[5052]: I1203 12:30:11.888674 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-ovnnorthd-ovndbs" Dec 03 12:30:11 crc kubenswrapper[5052]: I1203 12:30:11.888935 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"ovnnorthd-scripts" Dec 03 12:30:11 crc kubenswrapper[5052]: I1203 12:30:11.898145 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/neutron-56f9c7c6b6-kvxpr"] Dec 03 12:30:11 crc kubenswrapper[5052]: I1203 12:30:11.898404 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/neutron-56f9c7c6b6-kvxpr" podUID="5f522d90-036e-4565-8293-97097912fbb1" containerName="neutron-api" containerID="cri-o://83e0a3b478794ea6778ae35cae6f68d4eef32d5b5841dc00889915eed9b1dfbf" gracePeriod=30 Dec 03 12:30:11 crc kubenswrapper[5052]: I1203 12:30:11.898855 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/neutron-56f9c7c6b6-kvxpr" podUID="5f522d90-036e-4565-8293-97097912fbb1" containerName="neutron-httpd" containerID="cri-o://091f1e3cedc375760a2fcdc8a6147bfa5ecc673f83f421f7c9f903fa52ed8e8d" gracePeriod=30 Dec 03 12:30:11 crc kubenswrapper[5052]: I1203 12:30:11.907638 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-conductor-0" event={"ID":"e2718906-e52d-42fa-ad42-35626e99d461","Type":"ContainerStarted","Data":"b62a7c39d4bca0804816b2a547791ef7327cceb21e0af84713fe3c446866867d"} Dec 03 12:30:11 crc kubenswrapper[5052]: I1203 12:30:11.910566 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-conductor-0" event={"ID":"2dfbec8a-3a04-407b-8145-e12a3afee2e0","Type":"ContainerStarted","Data":"1cdb30cee2fbfa2eeec820d62852f60c9458350e3fa13e3998acbbd4a219ff59"} Dec 03 12:30:11 crc kubenswrapper[5052]: I1203 12:30:11.915755 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-northd-tls-certs\" (UniqueName: \"kubernetes.io/secret/1d48b723-3f75-4d1d-83d5-8315ea5464c6-ovn-northd-tls-certs\") pod \"ovn-northd-0\" (UID: \"1d48b723-3f75-4d1d-83d5-8315ea5464c6\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:30:11 crc kubenswrapper[5052]: I1203 12:30:11.915815 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1d48b723-3f75-4d1d-83d5-8315ea5464c6-combined-ca-bundle\") pod \"ovn-northd-0\" (UID: \"1d48b723-3f75-4d1d-83d5-8315ea5464c6\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:30:11 crc kubenswrapper[5052]: I1203 12:30:11.915894 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/1d48b723-3f75-4d1d-83d5-8315ea5464c6-metrics-certs-tls-certs\") pod \"ovn-northd-0\" (UID: \"1d48b723-3f75-4d1d-83d5-8315ea5464c6\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:30:11 crc kubenswrapper[5052]: I1203 12:30:11.915926 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/bb1723c7-4e41-4b0e-b6b8-f8024dafdaee-scripts\") pod \"cinder-scheduler-0\" (UID: \"bb1723c7-4e41-4b0e-b6b8-f8024dafdaee\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:30:11 crc kubenswrapper[5052]: I1203 12:30:11.915948 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/1d48b723-3f75-4d1d-83d5-8315ea5464c6-scripts\") pod \"ovn-northd-0\" (UID: \"1d48b723-3f75-4d1d-83d5-8315ea5464c6\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:30:11 crc kubenswrapper[5052]: I1203 12:30:11.915973 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/bb1723c7-4e41-4b0e-b6b8-f8024dafdaee-etc-machine-id\") pod \"cinder-scheduler-0\" (UID: \"bb1723c7-4e41-4b0e-b6b8-f8024dafdaee\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:30:11 crc kubenswrapper[5052]: I1203 12:30:11.916002 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/bb1723c7-4e41-4b0e-b6b8-f8024dafdaee-combined-ca-bundle\") pod \"cinder-scheduler-0\" (UID: \"bb1723c7-4e41-4b0e-b6b8-f8024dafdaee\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:30:11 crc kubenswrapper[5052]: I1203 12:30:11.916047 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1d48b723-3f75-4d1d-83d5-8315ea5464c6-config\") pod \"ovn-northd-0\" (UID: \"1d48b723-3f75-4d1d-83d5-8315ea5464c6\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:30:11 crc kubenswrapper[5052]: I1203 12:30:11.916124 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-rundir\" (UniqueName: \"kubernetes.io/empty-dir/1d48b723-3f75-4d1d-83d5-8315ea5464c6-ovn-rundir\") pod \"ovn-northd-0\" (UID: \"1d48b723-3f75-4d1d-83d5-8315ea5464c6\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:30:11 crc kubenswrapper[5052]: I1203 12:30:11.916200 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-bz7zz\" (UniqueName: \"kubernetes.io/projected/bb1723c7-4e41-4b0e-b6b8-f8024dafdaee-kube-api-access-bz7zz\") pod \"cinder-scheduler-0\" (UID: \"bb1723c7-4e41-4b0e-b6b8-f8024dafdaee\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:30:11 crc kubenswrapper[5052]: I1203 12:30:11.916245 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/bb1723c7-4e41-4b0e-b6b8-f8024dafdaee-config-data-custom\") pod \"cinder-scheduler-0\" (UID: \"bb1723c7-4e41-4b0e-b6b8-f8024dafdaee\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:30:11 crc kubenswrapper[5052]: I1203 12:30:11.916272 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5qphm\" (UniqueName: \"kubernetes.io/projected/1d48b723-3f75-4d1d-83d5-8315ea5464c6-kube-api-access-5qphm\") pod \"ovn-northd-0\" (UID: \"1d48b723-3f75-4d1d-83d5-8315ea5464c6\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:30:11 crc kubenswrapper[5052]: I1203 12:30:11.916296 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/bb1723c7-4e41-4b0e-b6b8-f8024dafdaee-config-data\") pod \"cinder-scheduler-0\" (UID: \"bb1723c7-4e41-4b0e-b6b8-f8024dafdaee\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:30:11 crc kubenswrapper[5052]: I1203 12:30:11.916419 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/bb1723c7-4e41-4b0e-b6b8-f8024dafdaee-etc-machine-id\") pod \"cinder-scheduler-0\" (UID: \"bb1723c7-4e41-4b0e-b6b8-f8024dafdaee\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:30:11 crc kubenswrapper[5052]: I1203 12:30:11.918796 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/neutron-769cb78446-m6hzp"] Dec 03 12:30:11 crc kubenswrapper[5052]: I1203 12:30:11.920969 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-769cb78446-m6hzp" Dec 03 12:30:11 crc kubenswrapper[5052]: I1203 12:30:11.938699 5052 scope.go:117] "RemoveContainer" containerID="698051bad98a7216c5799f152665bdd1626301fe4b20746cd84d12b4443a0156" Dec 03 12:30:11 crc kubenswrapper[5052]: I1203 12:30:11.940787 5052 generic.go:334] "Generic (PLEG): container finished" podID="8bddf2b1-8116-4b6e-9cd8-be8fe1cd4b64" containerID="33f8dbc2efb3ac8c4348e12af03e0b1919430c85f4a7565b4e5fc747718afe50" exitCode=143 Dec 03 12:30:11 crc kubenswrapper[5052]: I1203 12:30:11.940882 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-5c6478db8b-bt2rw" event={"ID":"8bddf2b1-8116-4b6e-9cd8-be8fe1cd4b64","Type":"ContainerDied","Data":"33f8dbc2efb3ac8c4348e12af03e0b1919430c85f4a7565b4e5fc747718afe50"} Dec 03 12:30:11 crc kubenswrapper[5052]: I1203 12:30:11.941070 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/bb1723c7-4e41-4b0e-b6b8-f8024dafdaee-scripts\") pod \"cinder-scheduler-0\" (UID: \"bb1723c7-4e41-4b0e-b6b8-f8024dafdaee\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:30:11 crc kubenswrapper[5052]: I1203 12:30:11.941693 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/bb1723c7-4e41-4b0e-b6b8-f8024dafdaee-config-data-custom\") pod \"cinder-scheduler-0\" (UID: \"bb1723c7-4e41-4b0e-b6b8-f8024dafdaee\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:30:11 crc kubenswrapper[5052]: I1203 12:30:11.942913 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/bb1723c7-4e41-4b0e-b6b8-f8024dafdaee-combined-ca-bundle\") pod \"cinder-scheduler-0\" (UID: \"bb1723c7-4e41-4b0e-b6b8-f8024dafdaee\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:30:11 crc kubenswrapper[5052]: I1203 12:30:11.944484 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/bb1723c7-4e41-4b0e-b6b8-f8024dafdaee-config-data\") pod \"cinder-scheduler-0\" (UID: \"bb1723c7-4e41-4b0e-b6b8-f8024dafdaee\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:30:11 crc kubenswrapper[5052]: I1203 12:30:11.958982 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-keystone-listener-58b6cdf6bb-5xxd8" Dec 03 12:30:11 crc kubenswrapper[5052]: I1203 12:30:11.959646 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-keystone-listener-58b6cdf6bb-5xxd8" event={"ID":"71df5bf1-7fd8-4a4d-b0f5-85e3911ae8a9","Type":"ContainerDied","Data":"9bfb97776da1f99ca1bf1dce1ae364ac91f105b29678722d7112b93147540221"} Dec 03 12:30:11 crc kubenswrapper[5052]: I1203 12:30:11.959698 5052 scope.go:117] "RemoveContainer" containerID="994203d634c00abfeeeb2db379076b53d6f741dba1b79d202d758f959e9439c6" Dec 03 12:30:11 crc kubenswrapper[5052]: I1203 12:30:11.960151 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-bz7zz\" (UniqueName: \"kubernetes.io/projected/bb1723c7-4e41-4b0e-b6b8-f8024dafdaee-kube-api-access-bz7zz\") pod \"cinder-scheduler-0\" (UID: \"bb1723c7-4e41-4b0e-b6b8-f8024dafdaee\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:30:11 crc kubenswrapper[5052]: W1203 12:30:11.963642 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod43933abc_db67_4ce5_9f30_6f8dbf3f59c8.slice/crio-d2387624a050b94b6cfa3ba971af84ee298e7c7fed409795b6b82257adc33a22 WatchSource:0}: Error finding container d2387624a050b94b6cfa3ba971af84ee298e7c7fed409795b6b82257adc33a22: Status 404 returned error can't find the container with id d2387624a050b94b6cfa3ba971af84ee298e7c7fed409795b6b82257adc33a22 Dec 03 12:30:11 crc kubenswrapper[5052]: I1203 12:30:11.969815 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-769cb78446-m6hzp"] Dec 03 12:30:11 crc kubenswrapper[5052]: I1203 12:30:11.971154 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"e8f63775-6260-4abb-abd7-45a412bf0ca5","Type":"ContainerStarted","Data":"b5bcab5248e26c3b89471f10eb9f60539f01ea5100b242c03a87ef7e7f42057a"} Dec 03 12:30:11 crc kubenswrapper[5052]: I1203 12:30:11.987707 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"f23969c4-775b-49b1-a88f-6fe74fd1db75","Type":"ContainerStarted","Data":"fa68fb091076369591b072c2f0542f7384d3920d776d3e6a905fb2a53ebd501d"} Dec 03 12:30:11 crc kubenswrapper[5052]: I1203 12:30:11.999551 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"a24b0824-26d0-4b81-b6cb-d617c36d45ab","Type":"ContainerStarted","Data":"a7a1fed3037d074c3c5826d734310aca486f80ded95efe62964a8c98bb827927"} Dec 03 12:30:12 crc kubenswrapper[5052]: I1203 12:30:12.001273 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-6b8b855786-bltgj"] Dec 03 12:30:12 crc kubenswrapper[5052]: I1203 12:30:12.009580 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ovn-northd-0"] Dec 03 12:30:12 crc kubenswrapper[5052]: E1203 12:30:12.010347 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[combined-ca-bundle config kube-api-access-5qphm metrics-certs-tls-certs ovn-northd-tls-certs ovn-rundir scripts], unattached volumes=[], failed to process volumes=[]: context canceled" pod="openstack-kuttl-tests/ovn-northd-0" podUID="1d48b723-3f75-4d1d-83d5-8315ea5464c6" Dec 03 12:30:12 crc kubenswrapper[5052]: I1203 12:30:12.011836 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/neutron-56f9c7c6b6-kvxpr" podUID="5f522d90-036e-4565-8293-97097912fbb1" containerName="neutron-httpd" probeResult="failure" output="Get \"https://10.217.0.203:9696/\": read tcp 10.217.0.2:49660->10.217.0.203:9696: read: connection reset by peer" Dec 03 12:30:12 crc kubenswrapper[5052]: I1203 12:30:12.012202 5052 generic.go:334] "Generic (PLEG): container finished" podID="53fe6403-27db-44b7-a9a6-787afbc3bbb8" containerID="c110d80f3a367aca9b254271248cc75eddf2eb31290f7f48d1f7ccab8d5d16fd" exitCode=143 Dec 03 12:30:12 crc kubenswrapper[5052]: I1203 12:30:12.012307 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-7dffdbf8-hzmbm" event={"ID":"53fe6403-27db-44b7-a9a6-787afbc3bbb8","Type":"ContainerDied","Data":"c110d80f3a367aca9b254271248cc75eddf2eb31290f7f48d1f7ccab8d5d16fd"} Dec 03 12:30:12 crc kubenswrapper[5052]: I1203 12:30:12.018193 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-5qphm\" (UniqueName: \"kubernetes.io/projected/1d48b723-3f75-4d1d-83d5-8315ea5464c6-kube-api-access-5qphm\") pod \"ovn-northd-0\" (UID: \"1d48b723-3f75-4d1d-83d5-8315ea5464c6\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:30:12 crc kubenswrapper[5052]: I1203 12:30:12.022745 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovn-northd-tls-certs\" (UniqueName: \"kubernetes.io/secret/1d48b723-3f75-4d1d-83d5-8315ea5464c6-ovn-northd-tls-certs\") pod \"ovn-northd-0\" (UID: \"1d48b723-3f75-4d1d-83d5-8315ea5464c6\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:30:12 crc kubenswrapper[5052]: I1203 12:30:12.023039 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1d48b723-3f75-4d1d-83d5-8315ea5464c6-combined-ca-bundle\") pod \"ovn-northd-0\" (UID: \"1d48b723-3f75-4d1d-83d5-8315ea5464c6\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:30:12 crc kubenswrapper[5052]: I1203 12:30:12.023284 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/1d48b723-3f75-4d1d-83d5-8315ea5464c6-metrics-certs-tls-certs\") pod \"ovn-northd-0\" (UID: \"1d48b723-3f75-4d1d-83d5-8315ea5464c6\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:30:12 crc kubenswrapper[5052]: I1203 12:30:12.023406 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/1d48b723-3f75-4d1d-83d5-8315ea5464c6-scripts\") pod \"ovn-northd-0\" (UID: \"1d48b723-3f75-4d1d-83d5-8315ea5464c6\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:30:12 crc kubenswrapper[5052]: I1203 12:30:12.023599 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1d48b723-3f75-4d1d-83d5-8315ea5464c6-config\") pod \"ovn-northd-0\" (UID: \"1d48b723-3f75-4d1d-83d5-8315ea5464c6\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:30:12 crc kubenswrapper[5052]: I1203 12:30:12.023795 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovn-rundir\" (UniqueName: \"kubernetes.io/empty-dir/1d48b723-3f75-4d1d-83d5-8315ea5464c6-ovn-rundir\") pod \"ovn-northd-0\" (UID: \"1d48b723-3f75-4d1d-83d5-8315ea5464c6\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:30:12 crc kubenswrapper[5052]: I1203 12:30:12.024378 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovn-rundir\" (UniqueName: \"kubernetes.io/empty-dir/1d48b723-3f75-4d1d-83d5-8315ea5464c6-ovn-rundir\") pod \"ovn-northd-0\" (UID: \"1d48b723-3f75-4d1d-83d5-8315ea5464c6\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:30:12 crc kubenswrapper[5052]: I1203 12:30:12.025118 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/1d48b723-3f75-4d1d-83d5-8315ea5464c6-scripts\") pod \"ovn-northd-0\" (UID: \"1d48b723-3f75-4d1d-83d5-8315ea5464c6\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:30:12 crc kubenswrapper[5052]: I1203 12:30:12.025696 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1d48b723-3f75-4d1d-83d5-8315ea5464c6-config\") pod \"ovn-northd-0\" (UID: \"1d48b723-3f75-4d1d-83d5-8315ea5464c6\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:30:12 crc kubenswrapper[5052]: I1203 12:30:12.027978 5052 scope.go:117] "RemoveContainer" containerID="b9e28cc89fab64aa07a40886930111804a359cb4003f7a994ed468c2084e2abf" Dec 03 12:30:12 crc kubenswrapper[5052]: I1203 12:30:12.041687 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-5qphm\" (UniqueName: \"kubernetes.io/projected/1d48b723-3f75-4d1d-83d5-8315ea5464c6-kube-api-access-5qphm\") pod \"ovn-northd-0\" (UID: \"1d48b723-3f75-4d1d-83d5-8315ea5464c6\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:30:12 crc kubenswrapper[5052]: I1203 12:30:12.058338 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/1d48b723-3f75-4d1d-83d5-8315ea5464c6-metrics-certs-tls-certs\") pod \"ovn-northd-0\" (UID: \"1d48b723-3f75-4d1d-83d5-8315ea5464c6\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:30:12 crc kubenswrapper[5052]: I1203 12:30:12.058351 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1d48b723-3f75-4d1d-83d5-8315ea5464c6-combined-ca-bundle\") pod \"ovn-northd-0\" (UID: \"1d48b723-3f75-4d1d-83d5-8315ea5464c6\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:30:12 crc kubenswrapper[5052]: I1203 12:30:12.059129 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovn-northd-tls-certs\" (UniqueName: \"kubernetes.io/secret/1d48b723-3f75-4d1d-83d5-8315ea5464c6-ovn-northd-tls-certs\") pod \"ovn-northd-0\" (UID: \"1d48b723-3f75-4d1d-83d5-8315ea5464c6\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:30:12 crc kubenswrapper[5052]: I1203 12:30:12.105817 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/71df5bf1-7fd8-4a4d-b0f5-85e3911ae8a9-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "71df5bf1-7fd8-4a4d-b0f5-85e3911ae8a9" (UID: "71df5bf1-7fd8-4a4d-b0f5-85e3911ae8a9"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:12 crc kubenswrapper[5052]: I1203 12:30:12.127242 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/760aa987-6355-40cb-97c0-16c7eeddd135-combined-ca-bundle\") pod \"neutron-769cb78446-m6hzp\" (UID: \"760aa987-6355-40cb-97c0-16c7eeddd135\") " pod="openstack-kuttl-tests/neutron-769cb78446-m6hzp" Dec 03 12:30:12 crc kubenswrapper[5052]: I1203 12:30:12.127322 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7rmck\" (UniqueName: \"kubernetes.io/projected/760aa987-6355-40cb-97c0-16c7eeddd135-kube-api-access-7rmck\") pod \"neutron-769cb78446-m6hzp\" (UID: \"760aa987-6355-40cb-97c0-16c7eeddd135\") " pod="openstack-kuttl-tests/neutron-769cb78446-m6hzp" Dec 03 12:30:12 crc kubenswrapper[5052]: I1203 12:30:12.127386 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/760aa987-6355-40cb-97c0-16c7eeddd135-ovndb-tls-certs\") pod \"neutron-769cb78446-m6hzp\" (UID: \"760aa987-6355-40cb-97c0-16c7eeddd135\") " pod="openstack-kuttl-tests/neutron-769cb78446-m6hzp" Dec 03 12:30:12 crc kubenswrapper[5052]: I1203 12:30:12.127412 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/760aa987-6355-40cb-97c0-16c7eeddd135-public-tls-certs\") pod \"neutron-769cb78446-m6hzp\" (UID: \"760aa987-6355-40cb-97c0-16c7eeddd135\") " pod="openstack-kuttl-tests/neutron-769cb78446-m6hzp" Dec 03 12:30:12 crc kubenswrapper[5052]: I1203 12:30:12.127488 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/760aa987-6355-40cb-97c0-16c7eeddd135-httpd-config\") pod \"neutron-769cb78446-m6hzp\" (UID: \"760aa987-6355-40cb-97c0-16c7eeddd135\") " pod="openstack-kuttl-tests/neutron-769cb78446-m6hzp" Dec 03 12:30:12 crc kubenswrapper[5052]: I1203 12:30:12.127551 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/760aa987-6355-40cb-97c0-16c7eeddd135-internal-tls-certs\") pod \"neutron-769cb78446-m6hzp\" (UID: \"760aa987-6355-40cb-97c0-16c7eeddd135\") " pod="openstack-kuttl-tests/neutron-769cb78446-m6hzp" Dec 03 12:30:12 crc kubenswrapper[5052]: I1203 12:30:12.127637 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/760aa987-6355-40cb-97c0-16c7eeddd135-config\") pod \"neutron-769cb78446-m6hzp\" (UID: \"760aa987-6355-40cb-97c0-16c7eeddd135\") " pod="openstack-kuttl-tests/neutron-769cb78446-m6hzp" Dec 03 12:30:12 crc kubenswrapper[5052]: I1203 12:30:12.127719 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/71df5bf1-7fd8-4a4d-b0f5-85e3911ae8a9-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:12 crc kubenswrapper[5052]: I1203 12:30:12.131020 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:30:12 crc kubenswrapper[5052]: I1203 12:30:12.173689 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/71df5bf1-7fd8-4a4d-b0f5-85e3911ae8a9-config-data" (OuterVolumeSpecName: "config-data") pod "71df5bf1-7fd8-4a4d-b0f5-85e3911ae8a9" (UID: "71df5bf1-7fd8-4a4d-b0f5-85e3911ae8a9"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:12 crc kubenswrapper[5052]: I1203 12:30:12.229926 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/760aa987-6355-40cb-97c0-16c7eeddd135-public-tls-certs\") pod \"neutron-769cb78446-m6hzp\" (UID: \"760aa987-6355-40cb-97c0-16c7eeddd135\") " pod="openstack-kuttl-tests/neutron-769cb78446-m6hzp" Dec 03 12:30:12 crc kubenswrapper[5052]: I1203 12:30:12.230051 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/760aa987-6355-40cb-97c0-16c7eeddd135-httpd-config\") pod \"neutron-769cb78446-m6hzp\" (UID: \"760aa987-6355-40cb-97c0-16c7eeddd135\") " pod="openstack-kuttl-tests/neutron-769cb78446-m6hzp" Dec 03 12:30:12 crc kubenswrapper[5052]: I1203 12:30:12.230109 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/760aa987-6355-40cb-97c0-16c7eeddd135-internal-tls-certs\") pod \"neutron-769cb78446-m6hzp\" (UID: \"760aa987-6355-40cb-97c0-16c7eeddd135\") " pod="openstack-kuttl-tests/neutron-769cb78446-m6hzp" Dec 03 12:30:12 crc kubenswrapper[5052]: I1203 12:30:12.230152 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/760aa987-6355-40cb-97c0-16c7eeddd135-config\") pod \"neutron-769cb78446-m6hzp\" (UID: \"760aa987-6355-40cb-97c0-16c7eeddd135\") " pod="openstack-kuttl-tests/neutron-769cb78446-m6hzp" Dec 03 12:30:12 crc kubenswrapper[5052]: I1203 12:30:12.230276 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/760aa987-6355-40cb-97c0-16c7eeddd135-combined-ca-bundle\") pod \"neutron-769cb78446-m6hzp\" (UID: \"760aa987-6355-40cb-97c0-16c7eeddd135\") " pod="openstack-kuttl-tests/neutron-769cb78446-m6hzp" Dec 03 12:30:12 crc kubenswrapper[5052]: I1203 12:30:12.230345 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-7rmck\" (UniqueName: \"kubernetes.io/projected/760aa987-6355-40cb-97c0-16c7eeddd135-kube-api-access-7rmck\") pod \"neutron-769cb78446-m6hzp\" (UID: \"760aa987-6355-40cb-97c0-16c7eeddd135\") " pod="openstack-kuttl-tests/neutron-769cb78446-m6hzp" Dec 03 12:30:12 crc kubenswrapper[5052]: I1203 12:30:12.230398 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/760aa987-6355-40cb-97c0-16c7eeddd135-ovndb-tls-certs\") pod \"neutron-769cb78446-m6hzp\" (UID: \"760aa987-6355-40cb-97c0-16c7eeddd135\") " pod="openstack-kuttl-tests/neutron-769cb78446-m6hzp" Dec 03 12:30:12 crc kubenswrapper[5052]: I1203 12:30:12.230472 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/71df5bf1-7fd8-4a4d-b0f5-85e3911ae8a9-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:12 crc kubenswrapper[5052]: I1203 12:30:12.236613 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/760aa987-6355-40cb-97c0-16c7eeddd135-public-tls-certs\") pod \"neutron-769cb78446-m6hzp\" (UID: \"760aa987-6355-40cb-97c0-16c7eeddd135\") " pod="openstack-kuttl-tests/neutron-769cb78446-m6hzp" Dec 03 12:30:12 crc kubenswrapper[5052]: I1203 12:30:12.237003 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/760aa987-6355-40cb-97c0-16c7eeddd135-internal-tls-certs\") pod \"neutron-769cb78446-m6hzp\" (UID: \"760aa987-6355-40cb-97c0-16c7eeddd135\") " pod="openstack-kuttl-tests/neutron-769cb78446-m6hzp" Dec 03 12:30:12 crc kubenswrapper[5052]: I1203 12:30:12.241901 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/760aa987-6355-40cb-97c0-16c7eeddd135-config\") pod \"neutron-769cb78446-m6hzp\" (UID: \"760aa987-6355-40cb-97c0-16c7eeddd135\") " pod="openstack-kuttl-tests/neutron-769cb78446-m6hzp" Dec 03 12:30:12 crc kubenswrapper[5052]: I1203 12:30:12.242691 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/760aa987-6355-40cb-97c0-16c7eeddd135-ovndb-tls-certs\") pod \"neutron-769cb78446-m6hzp\" (UID: \"760aa987-6355-40cb-97c0-16c7eeddd135\") " pod="openstack-kuttl-tests/neutron-769cb78446-m6hzp" Dec 03 12:30:12 crc kubenswrapper[5052]: I1203 12:30:12.243618 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/760aa987-6355-40cb-97c0-16c7eeddd135-combined-ca-bundle\") pod \"neutron-769cb78446-m6hzp\" (UID: \"760aa987-6355-40cb-97c0-16c7eeddd135\") " pod="openstack-kuttl-tests/neutron-769cb78446-m6hzp" Dec 03 12:30:12 crc kubenswrapper[5052]: I1203 12:30:12.268291 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/760aa987-6355-40cb-97c0-16c7eeddd135-httpd-config\") pod \"neutron-769cb78446-m6hzp\" (UID: \"760aa987-6355-40cb-97c0-16c7eeddd135\") " pod="openstack-kuttl-tests/neutron-769cb78446-m6hzp" Dec 03 12:30:12 crc kubenswrapper[5052]: I1203 12:30:12.269615 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-7rmck\" (UniqueName: \"kubernetes.io/projected/760aa987-6355-40cb-97c0-16c7eeddd135-kube-api-access-7rmck\") pod \"neutron-769cb78446-m6hzp\" (UID: \"760aa987-6355-40cb-97c0-16c7eeddd135\") " pod="openstack-kuttl-tests/neutron-769cb78446-m6hzp" Dec 03 12:30:12 crc kubenswrapper[5052]: I1203 12:30:12.368222 5052 scope.go:117] "RemoveContainer" containerID="f902f915da399e4d8dce5c8b225bbaf37cabe6b67b4273e6cf9af96b3dcda39a" Dec 03 12:30:12 crc kubenswrapper[5052]: I1203 12:30:12.378886 5052 scope.go:117] "RemoveContainer" containerID="67002cc386bd7047ddbb9535a8f1433365620854531590af15f527d196de051a" Dec 03 12:30:12 crc kubenswrapper[5052]: I1203 12:30:12.514519 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-769cb78446-m6hzp" Dec 03 12:30:12 crc kubenswrapper[5052]: I1203 12:30:12.547644 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-keystone-listener-58b6cdf6bb-5xxd8"] Dec 03 12:30:12 crc kubenswrapper[5052]: I1203 12:30:12.555150 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/barbican-keystone-listener-58b6cdf6bb-5xxd8"] Dec 03 12:30:12 crc kubenswrapper[5052]: I1203 12:30:12.622890 5052 scope.go:117] "RemoveContainer" containerID="a5ca1b03848c3e7731d3e96fd0f8faa937c1665505435b783c4113d6afa25f56" Dec 03 12:30:12 crc kubenswrapper[5052]: I1203 12:30:12.674070 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 12:30:12 crc kubenswrapper[5052]: I1203 12:30:12.831335 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/neutron-769cb78446-m6hzp"] Dec 03 12:30:12 crc kubenswrapper[5052]: I1203 12:30:12.841358 5052 scope.go:117] "RemoveContainer" containerID="30065432eb4043c50988772e61b0bac64b14ec13363f4e9b1287f2bf7a2de63f" Dec 03 12:30:12 crc kubenswrapper[5052]: I1203 12:30:12.859552 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/neutron-76bcd49fb6-54vvs"] Dec 03 12:30:12 crc kubenswrapper[5052]: I1203 12:30:12.861812 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-76bcd49fb6-54vvs" Dec 03 12:30:12 crc kubenswrapper[5052]: I1203 12:30:12.881463 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-76bcd49fb6-54vvs"] Dec 03 12:30:12 crc kubenswrapper[5052]: I1203 12:30:12.950343 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/6bfd5f51-8b37-4fea-8134-a2cdcd4f6049-ovndb-tls-certs\") pod \"neutron-76bcd49fb6-54vvs\" (UID: \"6bfd5f51-8b37-4fea-8134-a2cdcd4f6049\") " pod="openstack-kuttl-tests/neutron-76bcd49fb6-54vvs" Dec 03 12:30:12 crc kubenswrapper[5052]: I1203 12:30:12.950422 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/6bfd5f51-8b37-4fea-8134-a2cdcd4f6049-config\") pod \"neutron-76bcd49fb6-54vvs\" (UID: \"6bfd5f51-8b37-4fea-8134-a2cdcd4f6049\") " pod="openstack-kuttl-tests/neutron-76bcd49fb6-54vvs" Dec 03 12:30:12 crc kubenswrapper[5052]: I1203 12:30:12.950480 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/6bfd5f51-8b37-4fea-8134-a2cdcd4f6049-public-tls-certs\") pod \"neutron-76bcd49fb6-54vvs\" (UID: \"6bfd5f51-8b37-4fea-8134-a2cdcd4f6049\") " pod="openstack-kuttl-tests/neutron-76bcd49fb6-54vvs" Dec 03 12:30:12 crc kubenswrapper[5052]: I1203 12:30:12.950500 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xt6hn\" (UniqueName: \"kubernetes.io/projected/6bfd5f51-8b37-4fea-8134-a2cdcd4f6049-kube-api-access-xt6hn\") pod \"neutron-76bcd49fb6-54vvs\" (UID: \"6bfd5f51-8b37-4fea-8134-a2cdcd4f6049\") " pod="openstack-kuttl-tests/neutron-76bcd49fb6-54vvs" Dec 03 12:30:12 crc kubenswrapper[5052]: I1203 12:30:12.950543 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/6bfd5f51-8b37-4fea-8134-a2cdcd4f6049-httpd-config\") pod \"neutron-76bcd49fb6-54vvs\" (UID: \"6bfd5f51-8b37-4fea-8134-a2cdcd4f6049\") " pod="openstack-kuttl-tests/neutron-76bcd49fb6-54vvs" Dec 03 12:30:12 crc kubenswrapper[5052]: I1203 12:30:12.950567 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/6bfd5f51-8b37-4fea-8134-a2cdcd4f6049-internal-tls-certs\") pod \"neutron-76bcd49fb6-54vvs\" (UID: \"6bfd5f51-8b37-4fea-8134-a2cdcd4f6049\") " pod="openstack-kuttl-tests/neutron-76bcd49fb6-54vvs" Dec 03 12:30:12 crc kubenswrapper[5052]: I1203 12:30:12.950586 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6bfd5f51-8b37-4fea-8134-a2cdcd4f6049-combined-ca-bundle\") pod \"neutron-76bcd49fb6-54vvs\" (UID: \"6bfd5f51-8b37-4fea-8134-a2cdcd4f6049\") " pod="openstack-kuttl-tests/neutron-76bcd49fb6-54vvs" Dec 03 12:30:13 crc kubenswrapper[5052]: I1203 12:30:13.002823 5052 scope.go:117] "RemoveContainer" containerID="06e057704675bafd6f98189c6b86c43903be3fc1372671a1851fba5082cffb21" Dec 03 12:30:13 crc kubenswrapper[5052]: I1203 12:30:13.028486 5052 generic.go:334] "Generic (PLEG): container finished" podID="72a32b4d-d61d-43a5-a45d-c1fd4bad672d" containerID="902a5085d0a5d4b6d6bd379e7d79548b8cf29e5be3d818b61c582ef847c19f19" exitCode=143 Dec 03 12:30:13 crc kubenswrapper[5052]: I1203 12:30:13.028593 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-worker-679bfdf7b7-tg6h7" event={"ID":"72a32b4d-d61d-43a5-a45d-c1fd4bad672d","Type":"ContainerDied","Data":"902a5085d0a5d4b6d6bd379e7d79548b8cf29e5be3d818b61c582ef847c19f19"} Dec 03 12:30:13 crc kubenswrapper[5052]: I1203 12:30:13.052275 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/6bfd5f51-8b37-4fea-8134-a2cdcd4f6049-httpd-config\") pod \"neutron-76bcd49fb6-54vvs\" (UID: \"6bfd5f51-8b37-4fea-8134-a2cdcd4f6049\") " pod="openstack-kuttl-tests/neutron-76bcd49fb6-54vvs" Dec 03 12:30:13 crc kubenswrapper[5052]: I1203 12:30:13.052326 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/6bfd5f51-8b37-4fea-8134-a2cdcd4f6049-internal-tls-certs\") pod \"neutron-76bcd49fb6-54vvs\" (UID: \"6bfd5f51-8b37-4fea-8134-a2cdcd4f6049\") " pod="openstack-kuttl-tests/neutron-76bcd49fb6-54vvs" Dec 03 12:30:13 crc kubenswrapper[5052]: I1203 12:30:13.052353 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6bfd5f51-8b37-4fea-8134-a2cdcd4f6049-combined-ca-bundle\") pod \"neutron-76bcd49fb6-54vvs\" (UID: \"6bfd5f51-8b37-4fea-8134-a2cdcd4f6049\") " pod="openstack-kuttl-tests/neutron-76bcd49fb6-54vvs" Dec 03 12:30:13 crc kubenswrapper[5052]: I1203 12:30:13.052492 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/6bfd5f51-8b37-4fea-8134-a2cdcd4f6049-ovndb-tls-certs\") pod \"neutron-76bcd49fb6-54vvs\" (UID: \"6bfd5f51-8b37-4fea-8134-a2cdcd4f6049\") " pod="openstack-kuttl-tests/neutron-76bcd49fb6-54vvs" Dec 03 12:30:13 crc kubenswrapper[5052]: I1203 12:30:13.052568 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/6bfd5f51-8b37-4fea-8134-a2cdcd4f6049-config\") pod \"neutron-76bcd49fb6-54vvs\" (UID: \"6bfd5f51-8b37-4fea-8134-a2cdcd4f6049\") " pod="openstack-kuttl-tests/neutron-76bcd49fb6-54vvs" Dec 03 12:30:13 crc kubenswrapper[5052]: I1203 12:30:13.052610 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/6bfd5f51-8b37-4fea-8134-a2cdcd4f6049-public-tls-certs\") pod \"neutron-76bcd49fb6-54vvs\" (UID: \"6bfd5f51-8b37-4fea-8134-a2cdcd4f6049\") " pod="openstack-kuttl-tests/neutron-76bcd49fb6-54vvs" Dec 03 12:30:13 crc kubenswrapper[5052]: I1203 12:30:13.052629 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-xt6hn\" (UniqueName: \"kubernetes.io/projected/6bfd5f51-8b37-4fea-8134-a2cdcd4f6049-kube-api-access-xt6hn\") pod \"neutron-76bcd49fb6-54vvs\" (UID: \"6bfd5f51-8b37-4fea-8134-a2cdcd4f6049\") " pod="openstack-kuttl-tests/neutron-76bcd49fb6-54vvs" Dec 03 12:30:13 crc kubenswrapper[5052]: I1203 12:30:13.060405 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/6bfd5f51-8b37-4fea-8134-a2cdcd4f6049-internal-tls-certs\") pod \"neutron-76bcd49fb6-54vvs\" (UID: \"6bfd5f51-8b37-4fea-8134-a2cdcd4f6049\") " pod="openstack-kuttl-tests/neutron-76bcd49fb6-54vvs" Dec 03 12:30:13 crc kubenswrapper[5052]: I1203 12:30:13.060662 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/6bfd5f51-8b37-4fea-8134-a2cdcd4f6049-httpd-config\") pod \"neutron-76bcd49fb6-54vvs\" (UID: \"6bfd5f51-8b37-4fea-8134-a2cdcd4f6049\") " pod="openstack-kuttl-tests/neutron-76bcd49fb6-54vvs" Dec 03 12:30:13 crc kubenswrapper[5052]: I1203 12:30:13.061453 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/6bfd5f51-8b37-4fea-8134-a2cdcd4f6049-public-tls-certs\") pod \"neutron-76bcd49fb6-54vvs\" (UID: \"6bfd5f51-8b37-4fea-8134-a2cdcd4f6049\") " pod="openstack-kuttl-tests/neutron-76bcd49fb6-54vvs" Dec 03 12:30:13 crc kubenswrapper[5052]: I1203 12:30:13.062681 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/barbican-api-ff6b8c44b-7ndgm" podUID="48831cad-910f-46f1-aefe-7cc4cbfa984f" containerName="barbican-api-log" probeResult="failure" output="Get \"https://10.217.0.204:9311/healthcheck\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Dec 03 12:30:13 crc kubenswrapper[5052]: I1203 12:30:13.066679 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/6bfd5f51-8b37-4fea-8134-a2cdcd4f6049-config\") pod \"neutron-76bcd49fb6-54vvs\" (UID: \"6bfd5f51-8b37-4fea-8134-a2cdcd4f6049\") " pod="openstack-kuttl-tests/neutron-76bcd49fb6-54vvs" Dec 03 12:30:13 crc kubenswrapper[5052]: I1203 12:30:13.067567 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6bfd5f51-8b37-4fea-8134-a2cdcd4f6049-combined-ca-bundle\") pod \"neutron-76bcd49fb6-54vvs\" (UID: \"6bfd5f51-8b37-4fea-8134-a2cdcd4f6049\") " pod="openstack-kuttl-tests/neutron-76bcd49fb6-54vvs" Dec 03 12:30:13 crc kubenswrapper[5052]: I1203 12:30:13.070385 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/6bfd5f51-8b37-4fea-8134-a2cdcd4f6049-ovndb-tls-certs\") pod \"neutron-76bcd49fb6-54vvs\" (UID: \"6bfd5f51-8b37-4fea-8134-a2cdcd4f6049\") " pod="openstack-kuttl-tests/neutron-76bcd49fb6-54vvs" Dec 03 12:30:13 crc kubenswrapper[5052]: I1203 12:30:13.070702 5052 generic.go:334] "Generic (PLEG): container finished" podID="53fe6403-27db-44b7-a9a6-787afbc3bbb8" containerID="d3907f3c074c217040963cd0054e9403720f2049ff0f6f8d3ee8174b0d8a46a0" exitCode=0 Dec 03 12:30:13 crc kubenswrapper[5052]: I1203 12:30:13.070761 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-7dffdbf8-hzmbm" event={"ID":"53fe6403-27db-44b7-a9a6-787afbc3bbb8","Type":"ContainerDied","Data":"d3907f3c074c217040963cd0054e9403720f2049ff0f6f8d3ee8174b0d8a46a0"} Dec 03 12:30:13 crc kubenswrapper[5052]: I1203 12:30:13.070907 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-xt6hn\" (UniqueName: \"kubernetes.io/projected/6bfd5f51-8b37-4fea-8134-a2cdcd4f6049-kube-api-access-xt6hn\") pod \"neutron-76bcd49fb6-54vvs\" (UID: \"6bfd5f51-8b37-4fea-8134-a2cdcd4f6049\") " pod="openstack-kuttl-tests/neutron-76bcd49fb6-54vvs" Dec 03 12:30:13 crc kubenswrapper[5052]: I1203 12:30:13.075184 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"f23969c4-775b-49b1-a88f-6fe74fd1db75","Type":"ContainerStarted","Data":"9f8a3f17d990e268d789edd61d07026ea31ff500db1611d7cec6ad35c4fc914c"} Dec 03 12:30:13 crc kubenswrapper[5052]: I1203 12:30:13.094020 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-688bd8fb5d-pjs4f" event={"ID":"a0e45e1a-ced9-4958-9a75-24a69280f679","Type":"ContainerStarted","Data":"b5aa97990bcd4bfc80db39c69ed703786808679eda611dc607c177d51eeb3707"} Dec 03 12:30:13 crc kubenswrapper[5052]: I1203 12:30:13.094620 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/placement-688bd8fb5d-pjs4f" podUID="a0e45e1a-ced9-4958-9a75-24a69280f679" containerName="placement-log" containerID="cri-o://009d530d399c78fb2e9155e989a69050f595b2f0a1be3bf13247c7f1d91a158a" gracePeriod=30 Dec 03 12:30:13 crc kubenswrapper[5052]: I1203 12:30:13.094912 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/placement-688bd8fb5d-pjs4f" Dec 03 12:30:13 crc kubenswrapper[5052]: I1203 12:30:13.094950 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/placement-688bd8fb5d-pjs4f" Dec 03 12:30:13 crc kubenswrapper[5052]: I1203 12:30:13.094968 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/placement-688bd8fb5d-pjs4f" podUID="a0e45e1a-ced9-4958-9a75-24a69280f679" containerName="placement-api" containerID="cri-o://b5aa97990bcd4bfc80db39c69ed703786808679eda611dc607c177d51eeb3707" gracePeriod=30 Dec 03 12:30:13 crc kubenswrapper[5052]: I1203 12:30:13.110175 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-scheduler-0" podStartSLOduration=6.110160555 podStartE2EDuration="6.110160555s" podCreationTimestamp="2025-12-03 12:30:07 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:30:13.103652859 +0000 UTC m=+4203.691894643" watchObservedRunningTime="2025-12-03 12:30:13.110160555 +0000 UTC m=+4203.698402319" Dec 03 12:30:13 crc kubenswrapper[5052]: I1203 12:30:13.115288 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-conductor-0" event={"ID":"2dfbec8a-3a04-407b-8145-e12a3afee2e0","Type":"ContainerStarted","Data":"f709bafefdc660d91c439ef2df4b5fa8dafa8380b2dee355f1b04956932b91e7"} Dec 03 12:30:13 crc kubenswrapper[5052]: I1203 12:30:13.116283 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:30:13 crc kubenswrapper[5052]: I1203 12:30:13.122084 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"39117149-c15d-4753-9324-8a42abe5b007","Type":"ContainerStarted","Data":"5b99120371dd1ea96363322c8c525108a18d3c48f5844ec07a58fedeb98a76df"} Dec 03 12:30:13 crc kubenswrapper[5052]: I1203 12:30:13.127809 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/placement-688bd8fb5d-pjs4f" podStartSLOduration=11.127793488 podStartE2EDuration="11.127793488s" podCreationTimestamp="2025-12-03 12:30:02 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:30:13.123871966 +0000 UTC m=+4203.712113760" watchObservedRunningTime="2025-12-03 12:30:13.127793488 +0000 UTC m=+4203.716035252" Dec 03 12:30:13 crc kubenswrapper[5052]: I1203 12:30:13.147150 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-cell1-conductor-0" podStartSLOduration=6.14713532 podStartE2EDuration="6.14713532s" podCreationTimestamp="2025-12-03 12:30:07 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:30:13.14500932 +0000 UTC m=+4203.733251084" watchObservedRunningTime="2025-12-03 12:30:13.14713532 +0000 UTC m=+4203.735377084" Dec 03 12:30:13 crc kubenswrapper[5052]: I1203 12:30:13.191476 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-metadata-0" podStartSLOduration=11.191457285 podStartE2EDuration="11.191457285s" podCreationTimestamp="2025-12-03 12:30:02 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:30:13.167131991 +0000 UTC m=+4203.755373755" watchObservedRunningTime="2025-12-03 12:30:13.191457285 +0000 UTC m=+4203.779699049" Dec 03 12:30:13 crc kubenswrapper[5052]: I1203 12:30:13.191702 5052 generic.go:334] "Generic (PLEG): container finished" podID="8bddf2b1-8116-4b6e-9cd8-be8fe1cd4b64" containerID="5f22191c711f94909400c3cf19b845a8d0a3f693f6879266161b330e5d69cafc" exitCode=0 Dec 03 12:30:13 crc kubenswrapper[5052]: I1203 12:30:13.191747 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-5c6478db8b-bt2rw" event={"ID":"8bddf2b1-8116-4b6e-9cd8-be8fe1cd4b64","Type":"ContainerDied","Data":"5f22191c711f94909400c3cf19b845a8d0a3f693f6879266161b330e5d69cafc"} Dec 03 12:30:13 crc kubenswrapper[5052]: I1203 12:30:13.220799 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_ovsdbserver-nb-0_98f722f9-fa25-40b6-84ba-96127d07293c/ovsdbserver-nb/0.log" Dec 03 12:30:13 crc kubenswrapper[5052]: I1203 12:30:13.220860 5052 generic.go:334] "Generic (PLEG): container finished" podID="98f722f9-fa25-40b6-84ba-96127d07293c" containerID="c30cd5ee45f2cebbe8afbf744e6817d788d7a3b824716ed969e21464a285f555" exitCode=2 Dec 03 12:30:13 crc kubenswrapper[5052]: I1203 12:30:13.220877 5052 generic.go:334] "Generic (PLEG): container finished" podID="98f722f9-fa25-40b6-84ba-96127d07293c" containerID="b6e565e6b6e8e07c8000bbead3eed4e1cd45529d9dc3aa7fcb642cf668f780aa" exitCode=143 Dec 03 12:30:13 crc kubenswrapper[5052]: I1203 12:30:13.221010 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-nb-0" event={"ID":"98f722f9-fa25-40b6-84ba-96127d07293c","Type":"ContainerDied","Data":"c30cd5ee45f2cebbe8afbf744e6817d788d7a3b824716ed969e21464a285f555"} Dec 03 12:30:13 crc kubenswrapper[5052]: I1203 12:30:13.221045 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-nb-0" event={"ID":"98f722f9-fa25-40b6-84ba-96127d07293c","Type":"ContainerDied","Data":"b6e565e6b6e8e07c8000bbead3eed4e1cd45529d9dc3aa7fcb642cf668f780aa"} Dec 03 12:30:13 crc kubenswrapper[5052]: I1203 12:30:13.221276 5052 scope.go:117] "RemoveContainer" containerID="cf30dbd3717dce8af354c25d39edb466422569ee896210a1fcd9f69012eca27c" Dec 03 12:30:13 crc kubenswrapper[5052]: I1203 12:30:13.226100 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:30:13 crc kubenswrapper[5052]: I1203 12:30:13.226377 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:30:13 crc kubenswrapper[5052]: I1203 12:30:13.226695 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:30:13 crc kubenswrapper[5052]: I1203 12:30:13.226749 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:30:13 crc kubenswrapper[5052]: I1203 12:30:13.233440 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"bb1723c7-4e41-4b0e-b6b8-f8024dafdaee","Type":"ContainerStarted","Data":"0416fe74f81dcb0b84e39cd58ebd5f9449a6bd5290a02aed47a52edaf01b8dda"} Dec 03 12:30:13 crc kubenswrapper[5052]: I1203 12:30:13.259005 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-76bcd49fb6-54vvs" Dec 03 12:30:13 crc kubenswrapper[5052]: I1203 12:30:13.296967 5052 generic.go:334] "Generic (PLEG): container finished" podID="1de1d25b-cbcc-4d08-a7cd-aa54ac71aecb" containerID="46246c5deed2c10d0bc81011e4be47e5c70c228f4dda6fb9a1b20810aff514c7" exitCode=0 Dec 03 12:30:13 crc kubenswrapper[5052]: I1203 12:30:13.297076 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-cell1-galera-0" event={"ID":"1de1d25b-cbcc-4d08-a7cd-aa54ac71aecb","Type":"ContainerDied","Data":"46246c5deed2c10d0bc81011e4be47e5c70c228f4dda6fb9a1b20810aff514c7"} Dec 03 12:30:13 crc kubenswrapper[5052]: I1203 12:30:13.312593 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"4314928d-a43f-4b38-9c72-63977e59345b","Type":"ContainerStarted","Data":"55d873486c41fcc800af3aac0c5bdbf6786c715209a5758840bdce2e1c3fba02"} Dec 03 12:30:13 crc kubenswrapper[5052]: I1203 12:30:13.327097 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-65444bfc86-th5pw" event={"ID":"cfd8af50-a644-4b24-8483-dd344f2218e4","Type":"ContainerStarted","Data":"fefeb53cf7ddfecbec19b54d8ffa06426f1efa88fb8452a1c522b836c48d8085"} Dec 03 12:30:13 crc kubenswrapper[5052]: I1203 12:30:13.327137 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/placement-65444bfc86-th5pw" Dec 03 12:30:13 crc kubenswrapper[5052]: I1203 12:30:13.327159 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/placement-65444bfc86-th5pw" Dec 03 12:30:13 crc kubenswrapper[5052]: I1203 12:30:13.346898 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/neutron-769cb78446-m6hzp"] Dec 03 12:30:13 crc kubenswrapper[5052]: I1203 12:30:13.380441 5052 generic.go:334] "Generic (PLEG): container finished" podID="5f522d90-036e-4565-8293-97097912fbb1" containerID="091f1e3cedc375760a2fcdc8a6147bfa5ecc673f83f421f7c9f903fa52ed8e8d" exitCode=0 Dec 03 12:30:13 crc kubenswrapper[5052]: I1203 12:30:13.380644 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-56f9c7c6b6-kvxpr" event={"ID":"5f522d90-036e-4565-8293-97097912fbb1","Type":"ContainerDied","Data":"091f1e3cedc375760a2fcdc8a6147bfa5ecc673f83f421f7c9f903fa52ed8e8d"} Dec 03 12:30:13 crc kubenswrapper[5052]: I1203 12:30:13.415299 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"e8f63775-6260-4abb-abd7-45a412bf0ca5","Type":"ContainerStarted","Data":"b4d3741161a3d95e91cfc6ceb2e9626438d026ce0c88039796b48eae93f58046"} Dec 03 12:30:13 crc kubenswrapper[5052]: I1203 12:30:13.437823 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-6b8b855786-bltgj" event={"ID":"43933abc-db67-4ce5-9f30-6f8dbf3f59c8","Type":"ContainerStarted","Data":"aec2d2ca3d9979ac47cbc62e58c9a1c7b06d1e13c5205fc585d92567ca0e4ec3"} Dec 03 12:30:13 crc kubenswrapper[5052]: I1203 12:30:13.437873 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-6b8b855786-bltgj" event={"ID":"43933abc-db67-4ce5-9f30-6f8dbf3f59c8","Type":"ContainerStarted","Data":"d2387624a050b94b6cfa3ba971af84ee298e7c7fed409795b6b82257adc33a22"} Dec 03 12:30:13 crc kubenswrapper[5052]: I1203 12:30:13.441573 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-api-0" podStartSLOduration=10.441551964 podStartE2EDuration="10.441551964s" podCreationTimestamp="2025-12-03 12:30:03 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:30:13.330135074 +0000 UTC m=+4203.918376828" watchObservedRunningTime="2025-12-03 12:30:13.441551964 +0000 UTC m=+4204.029793728" Dec 03 12:30:13 crc kubenswrapper[5052]: I1203 12:30:13.451147 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-conductor-0" event={"ID":"e2718906-e52d-42fa-ad42-35626e99d461","Type":"ContainerStarted","Data":"003ad3a32b6c302f684066c22cf6e0ad91b95336410c2156d1794d7ec8417301"} Dec 03 12:30:13 crc kubenswrapper[5052]: I1203 12:30:13.452150 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:30:13 crc kubenswrapper[5052]: I1203 12:30:13.465318 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:30:13 crc kubenswrapper[5052]: I1203 12:30:13.472774 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ovsdbserver-sb-0" podUID="50f5af82-8ee5-45f4-b158-2166f1be6cb8" containerName="openstack-network-exporter" containerID="cri-o://e9076e00de7ae30a9d54e2e79b337633b061bc7f8a01e8aa825185ef328777b0" gracePeriod=300 Dec 03 12:30:13 crc kubenswrapper[5052]: I1203 12:30:13.481382 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/placement-65444bfc86-th5pw" podStartSLOduration=10.48136407 podStartE2EDuration="10.48136407s" podCreationTimestamp="2025-12-03 12:30:03 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:30:13.387052188 +0000 UTC m=+4203.975293952" watchObservedRunningTime="2025-12-03 12:30:13.48136407 +0000 UTC m=+4204.069605834" Dec 03 12:30:13 crc kubenswrapper[5052]: I1203 12:30:13.507536 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-cell0-conductor-0" podStartSLOduration=6.5075013760000004 podStartE2EDuration="6.507501376s" podCreationTimestamp="2025-12-03 12:30:07 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:30:13.476309326 +0000 UTC m=+4204.064551090" watchObservedRunningTime="2025-12-03 12:30:13.507501376 +0000 UTC m=+4204.095743140" Dec 03 12:30:13 crc kubenswrapper[5052]: I1203 12:30:13.517742 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:30:13 crc kubenswrapper[5052]: I1203 12:30:13.615256 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:30:13 crc kubenswrapper[5052]: I1203 12:30:13.673618 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/neutron-76bcd49fb6-54vvs"] Dec 03 12:30:13 crc kubenswrapper[5052]: I1203 12:30:13.680607 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovn-northd-tls-certs\" (UniqueName: \"kubernetes.io/secret/1d48b723-3f75-4d1d-83d5-8315ea5464c6-ovn-northd-tls-certs\") pod \"1d48b723-3f75-4d1d-83d5-8315ea5464c6\" (UID: \"1d48b723-3f75-4d1d-83d5-8315ea5464c6\") " Dec 03 12:30:13 crc kubenswrapper[5052]: I1203 12:30:13.680682 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1d48b723-3f75-4d1d-83d5-8315ea5464c6-config\") pod \"1d48b723-3f75-4d1d-83d5-8315ea5464c6\" (UID: \"1d48b723-3f75-4d1d-83d5-8315ea5464c6\") " Dec 03 12:30:13 crc kubenswrapper[5052]: I1203 12:30:13.680762 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovn-rundir\" (UniqueName: \"kubernetes.io/empty-dir/1d48b723-3f75-4d1d-83d5-8315ea5464c6-ovn-rundir\") pod \"1d48b723-3f75-4d1d-83d5-8315ea5464c6\" (UID: \"1d48b723-3f75-4d1d-83d5-8315ea5464c6\") " Dec 03 12:30:13 crc kubenswrapper[5052]: I1203 12:30:13.680900 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1d48b723-3f75-4d1d-83d5-8315ea5464c6-combined-ca-bundle\") pod \"1d48b723-3f75-4d1d-83d5-8315ea5464c6\" (UID: \"1d48b723-3f75-4d1d-83d5-8315ea5464c6\") " Dec 03 12:30:13 crc kubenswrapper[5052]: I1203 12:30:13.680964 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5qphm\" (UniqueName: \"kubernetes.io/projected/1d48b723-3f75-4d1d-83d5-8315ea5464c6-kube-api-access-5qphm\") pod \"1d48b723-3f75-4d1d-83d5-8315ea5464c6\" (UID: \"1d48b723-3f75-4d1d-83d5-8315ea5464c6\") " Dec 03 12:30:13 crc kubenswrapper[5052]: I1203 12:30:13.681045 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/1d48b723-3f75-4d1d-83d5-8315ea5464c6-scripts\") pod \"1d48b723-3f75-4d1d-83d5-8315ea5464c6\" (UID: \"1d48b723-3f75-4d1d-83d5-8315ea5464c6\") " Dec 03 12:30:13 crc kubenswrapper[5052]: I1203 12:30:13.681077 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/1d48b723-3f75-4d1d-83d5-8315ea5464c6-metrics-certs-tls-certs\") pod \"1d48b723-3f75-4d1d-83d5-8315ea5464c6\" (UID: \"1d48b723-3f75-4d1d-83d5-8315ea5464c6\") " Dec 03 12:30:13 crc kubenswrapper[5052]: I1203 12:30:13.690431 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1d48b723-3f75-4d1d-83d5-8315ea5464c6-scripts" (OuterVolumeSpecName: "scripts") pod "1d48b723-3f75-4d1d-83d5-8315ea5464c6" (UID: "1d48b723-3f75-4d1d-83d5-8315ea5464c6"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:30:13 crc kubenswrapper[5052]: I1203 12:30:13.690686 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1d48b723-3f75-4d1d-83d5-8315ea5464c6-ovn-rundir" (OuterVolumeSpecName: "ovn-rundir") pod "1d48b723-3f75-4d1d-83d5-8315ea5464c6" (UID: "1d48b723-3f75-4d1d-83d5-8315ea5464c6"). InnerVolumeSpecName "ovn-rundir". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:30:13 crc kubenswrapper[5052]: I1203 12:30:13.690895 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1d48b723-3f75-4d1d-83d5-8315ea5464c6-config" (OuterVolumeSpecName: "config") pod "1d48b723-3f75-4d1d-83d5-8315ea5464c6" (UID: "1d48b723-3f75-4d1d-83d5-8315ea5464c6"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:30:13 crc kubenswrapper[5052]: I1203 12:30:13.693741 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1d48b723-3f75-4d1d-83d5-8315ea5464c6-kube-api-access-5qphm" (OuterVolumeSpecName: "kube-api-access-5qphm") pod "1d48b723-3f75-4d1d-83d5-8315ea5464c6" (UID: "1d48b723-3f75-4d1d-83d5-8315ea5464c6"). InnerVolumeSpecName "kube-api-access-5qphm". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:30:13 crc kubenswrapper[5052]: I1203 12:30:13.695607 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1d48b723-3f75-4d1d-83d5-8315ea5464c6-ovn-northd-tls-certs" (OuterVolumeSpecName: "ovn-northd-tls-certs") pod "1d48b723-3f75-4d1d-83d5-8315ea5464c6" (UID: "1d48b723-3f75-4d1d-83d5-8315ea5464c6"). InnerVolumeSpecName "ovn-northd-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:13 crc kubenswrapper[5052]: I1203 12:30:13.704167 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1d48b723-3f75-4d1d-83d5-8315ea5464c6-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "1d48b723-3f75-4d1d-83d5-8315ea5464c6" (UID: "1d48b723-3f75-4d1d-83d5-8315ea5464c6"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:13 crc kubenswrapper[5052]: I1203 12:30:13.707776 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1d48b723-3f75-4d1d-83d5-8315ea5464c6-metrics-certs-tls-certs" (OuterVolumeSpecName: "metrics-certs-tls-certs") pod "1d48b723-3f75-4d1d-83d5-8315ea5464c6" (UID: "1d48b723-3f75-4d1d-83d5-8315ea5464c6"). InnerVolumeSpecName "metrics-certs-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:13 crc kubenswrapper[5052]: I1203 12:30:13.747750 5052 scope.go:117] "RemoveContainer" containerID="00bb5044fc1f5280459a013a740283ad38025f50730707308a452a1f487b74f0" Dec 03 12:30:13 crc kubenswrapper[5052]: I1203 12:30:13.783587 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1d48b723-3f75-4d1d-83d5-8315ea5464c6-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:13 crc kubenswrapper[5052]: I1203 12:30:13.783634 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-5qphm\" (UniqueName: \"kubernetes.io/projected/1d48b723-3f75-4d1d-83d5-8315ea5464c6-kube-api-access-5qphm\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:13 crc kubenswrapper[5052]: I1203 12:30:13.783652 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/1d48b723-3f75-4d1d-83d5-8315ea5464c6-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:13 crc kubenswrapper[5052]: I1203 12:30:13.783664 5052 reconciler_common.go:293] "Volume detached for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/1d48b723-3f75-4d1d-83d5-8315ea5464c6-metrics-certs-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:13 crc kubenswrapper[5052]: I1203 12:30:13.783676 5052 reconciler_common.go:293] "Volume detached for volume \"ovn-northd-tls-certs\" (UniqueName: \"kubernetes.io/secret/1d48b723-3f75-4d1d-83d5-8315ea5464c6-ovn-northd-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:13 crc kubenswrapper[5052]: I1203 12:30:13.783688 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1d48b723-3f75-4d1d-83d5-8315ea5464c6-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:13 crc kubenswrapper[5052]: I1203 12:30:13.783701 5052 reconciler_common.go:293] "Volume detached for volume \"ovn-rundir\" (UniqueName: \"kubernetes.io/empty-dir/1d48b723-3f75-4d1d-83d5-8315ea5464c6-ovn-rundir\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:14 crc kubenswrapper[5052]: I1203 12:30:14.076026 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="214a26fb-e9b2-4a87-8fa5-ae2ed0d8a828" path="/var/lib/kubelet/pods/214a26fb-e9b2-4a87-8fa5-ae2ed0d8a828/volumes" Dec 03 12:30:14 crc kubenswrapper[5052]: I1203 12:30:14.078043 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="71df5bf1-7fd8-4a4d-b0f5-85e3911ae8a9" path="/var/lib/kubelet/pods/71df5bf1-7fd8-4a4d-b0f5-85e3911ae8a9/volumes" Dec 03 12:30:14 crc kubenswrapper[5052]: I1203 12:30:14.079702 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a26ed42f-71e7-4c92-8b7e-ad194358d3e5" path="/var/lib/kubelet/pods/a26ed42f-71e7-4c92-8b7e-ad194358d3e5/volumes" Dec 03 12:30:14 crc kubenswrapper[5052]: I1203 12:30:14.081578 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/neutron-58d7cc6794-9z49k"] Dec 03 12:30:14 crc kubenswrapper[5052]: I1203 12:30:14.085815 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-58d7cc6794-9z49k"] Dec 03 12:30:14 crc kubenswrapper[5052]: I1203 12:30:14.085852 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:30:14 crc kubenswrapper[5052]: I1203 12:30:14.085866 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:30:14 crc kubenswrapper[5052]: I1203 12:30:14.085952 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-58d7cc6794-9z49k" Dec 03 12:30:14 crc kubenswrapper[5052]: I1203 12:30:14.198005 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4b3e73a0-1320-427f-bcd7-d8062e315e39-combined-ca-bundle\") pod \"neutron-58d7cc6794-9z49k\" (UID: \"4b3e73a0-1320-427f-bcd7-d8062e315e39\") " pod="openstack-kuttl-tests/neutron-58d7cc6794-9z49k" Dec 03 12:30:14 crc kubenswrapper[5052]: I1203 12:30:14.198065 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wmkts\" (UniqueName: \"kubernetes.io/projected/4b3e73a0-1320-427f-bcd7-d8062e315e39-kube-api-access-wmkts\") pod \"neutron-58d7cc6794-9z49k\" (UID: \"4b3e73a0-1320-427f-bcd7-d8062e315e39\") " pod="openstack-kuttl-tests/neutron-58d7cc6794-9z49k" Dec 03 12:30:14 crc kubenswrapper[5052]: I1203 12:30:14.198101 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/4b3e73a0-1320-427f-bcd7-d8062e315e39-ovndb-tls-certs\") pod \"neutron-58d7cc6794-9z49k\" (UID: \"4b3e73a0-1320-427f-bcd7-d8062e315e39\") " pod="openstack-kuttl-tests/neutron-58d7cc6794-9z49k" Dec 03 12:30:14 crc kubenswrapper[5052]: I1203 12:30:14.198160 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/4b3e73a0-1320-427f-bcd7-d8062e315e39-internal-tls-certs\") pod \"neutron-58d7cc6794-9z49k\" (UID: \"4b3e73a0-1320-427f-bcd7-d8062e315e39\") " pod="openstack-kuttl-tests/neutron-58d7cc6794-9z49k" Dec 03 12:30:14 crc kubenswrapper[5052]: I1203 12:30:14.198202 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/4b3e73a0-1320-427f-bcd7-d8062e315e39-httpd-config\") pod \"neutron-58d7cc6794-9z49k\" (UID: \"4b3e73a0-1320-427f-bcd7-d8062e315e39\") " pod="openstack-kuttl-tests/neutron-58d7cc6794-9z49k" Dec 03 12:30:14 crc kubenswrapper[5052]: I1203 12:30:14.198273 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/4b3e73a0-1320-427f-bcd7-d8062e315e39-public-tls-certs\") pod \"neutron-58d7cc6794-9z49k\" (UID: \"4b3e73a0-1320-427f-bcd7-d8062e315e39\") " pod="openstack-kuttl-tests/neutron-58d7cc6794-9z49k" Dec 03 12:30:14 crc kubenswrapper[5052]: I1203 12:30:14.198360 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/4b3e73a0-1320-427f-bcd7-d8062e315e39-config\") pod \"neutron-58d7cc6794-9z49k\" (UID: \"4b3e73a0-1320-427f-bcd7-d8062e315e39\") " pod="openstack-kuttl-tests/neutron-58d7cc6794-9z49k" Dec 03 12:30:14 crc kubenswrapper[5052]: I1203 12:30:14.281655 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-metadata-0" podUID="39117149-c15d-4753-9324-8a42abe5b007" containerName="nova-metadata-log" probeResult="failure" output="Get \"https://10.217.0.213:8775/\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Dec 03 12:30:14 crc kubenswrapper[5052]: I1203 12:30:14.282205 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-metadata-0" podUID="39117149-c15d-4753-9324-8a42abe5b007" containerName="nova-metadata-metadata" probeResult="failure" output="Get \"https://10.217.0.213:8775/\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Dec 03 12:30:14 crc kubenswrapper[5052]: I1203 12:30:14.299697 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4b3e73a0-1320-427f-bcd7-d8062e315e39-combined-ca-bundle\") pod \"neutron-58d7cc6794-9z49k\" (UID: \"4b3e73a0-1320-427f-bcd7-d8062e315e39\") " pod="openstack-kuttl-tests/neutron-58d7cc6794-9z49k" Dec 03 12:30:14 crc kubenswrapper[5052]: I1203 12:30:14.299737 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-wmkts\" (UniqueName: \"kubernetes.io/projected/4b3e73a0-1320-427f-bcd7-d8062e315e39-kube-api-access-wmkts\") pod \"neutron-58d7cc6794-9z49k\" (UID: \"4b3e73a0-1320-427f-bcd7-d8062e315e39\") " pod="openstack-kuttl-tests/neutron-58d7cc6794-9z49k" Dec 03 12:30:14 crc kubenswrapper[5052]: I1203 12:30:14.299766 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/4b3e73a0-1320-427f-bcd7-d8062e315e39-ovndb-tls-certs\") pod \"neutron-58d7cc6794-9z49k\" (UID: \"4b3e73a0-1320-427f-bcd7-d8062e315e39\") " pod="openstack-kuttl-tests/neutron-58d7cc6794-9z49k" Dec 03 12:30:14 crc kubenswrapper[5052]: I1203 12:30:14.299798 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/4b3e73a0-1320-427f-bcd7-d8062e315e39-internal-tls-certs\") pod \"neutron-58d7cc6794-9z49k\" (UID: \"4b3e73a0-1320-427f-bcd7-d8062e315e39\") " pod="openstack-kuttl-tests/neutron-58d7cc6794-9z49k" Dec 03 12:30:14 crc kubenswrapper[5052]: I1203 12:30:14.299822 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/4b3e73a0-1320-427f-bcd7-d8062e315e39-httpd-config\") pod \"neutron-58d7cc6794-9z49k\" (UID: \"4b3e73a0-1320-427f-bcd7-d8062e315e39\") " pod="openstack-kuttl-tests/neutron-58d7cc6794-9z49k" Dec 03 12:30:14 crc kubenswrapper[5052]: I1203 12:30:14.299869 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/4b3e73a0-1320-427f-bcd7-d8062e315e39-public-tls-certs\") pod \"neutron-58d7cc6794-9z49k\" (UID: \"4b3e73a0-1320-427f-bcd7-d8062e315e39\") " pod="openstack-kuttl-tests/neutron-58d7cc6794-9z49k" Dec 03 12:30:14 crc kubenswrapper[5052]: I1203 12:30:14.299922 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/4b3e73a0-1320-427f-bcd7-d8062e315e39-config\") pod \"neutron-58d7cc6794-9z49k\" (UID: \"4b3e73a0-1320-427f-bcd7-d8062e315e39\") " pod="openstack-kuttl-tests/neutron-58d7cc6794-9z49k" Dec 03 12:30:14 crc kubenswrapper[5052]: I1203 12:30:14.304874 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/4b3e73a0-1320-427f-bcd7-d8062e315e39-config\") pod \"neutron-58d7cc6794-9z49k\" (UID: \"4b3e73a0-1320-427f-bcd7-d8062e315e39\") " pod="openstack-kuttl-tests/neutron-58d7cc6794-9z49k" Dec 03 12:30:14 crc kubenswrapper[5052]: I1203 12:30:14.312206 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/4b3e73a0-1320-427f-bcd7-d8062e315e39-public-tls-certs\") pod \"neutron-58d7cc6794-9z49k\" (UID: \"4b3e73a0-1320-427f-bcd7-d8062e315e39\") " pod="openstack-kuttl-tests/neutron-58d7cc6794-9z49k" Dec 03 12:30:14 crc kubenswrapper[5052]: I1203 12:30:14.312419 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/4b3e73a0-1320-427f-bcd7-d8062e315e39-ovndb-tls-certs\") pod \"neutron-58d7cc6794-9z49k\" (UID: \"4b3e73a0-1320-427f-bcd7-d8062e315e39\") " pod="openstack-kuttl-tests/neutron-58d7cc6794-9z49k" Dec 03 12:30:14 crc kubenswrapper[5052]: I1203 12:30:14.312750 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/4b3e73a0-1320-427f-bcd7-d8062e315e39-internal-tls-certs\") pod \"neutron-58d7cc6794-9z49k\" (UID: \"4b3e73a0-1320-427f-bcd7-d8062e315e39\") " pod="openstack-kuttl-tests/neutron-58d7cc6794-9z49k" Dec 03 12:30:14 crc kubenswrapper[5052]: I1203 12:30:14.313349 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4b3e73a0-1320-427f-bcd7-d8062e315e39-combined-ca-bundle\") pod \"neutron-58d7cc6794-9z49k\" (UID: \"4b3e73a0-1320-427f-bcd7-d8062e315e39\") " pod="openstack-kuttl-tests/neutron-58d7cc6794-9z49k" Dec 03 12:30:14 crc kubenswrapper[5052]: I1203 12:30:14.314407 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/4b3e73a0-1320-427f-bcd7-d8062e315e39-httpd-config\") pod \"neutron-58d7cc6794-9z49k\" (UID: \"4b3e73a0-1320-427f-bcd7-d8062e315e39\") " pod="openstack-kuttl-tests/neutron-58d7cc6794-9z49k" Dec 03 12:30:14 crc kubenswrapper[5052]: I1203 12:30:14.342542 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-wmkts\" (UniqueName: \"kubernetes.io/projected/4b3e73a0-1320-427f-bcd7-d8062e315e39-kube-api-access-wmkts\") pod \"neutron-58d7cc6794-9z49k\" (UID: \"4b3e73a0-1320-427f-bcd7-d8062e315e39\") " pod="openstack-kuttl-tests/neutron-58d7cc6794-9z49k" Dec 03 12:30:14 crc kubenswrapper[5052]: I1203 12:30:14.511707 5052 generic.go:334] "Generic (PLEG): container finished" podID="50f5af82-8ee5-45f4-b158-2166f1be6cb8" containerID="e9076e00de7ae30a9d54e2e79b337633b061bc7f8a01e8aa825185ef328777b0" exitCode=2 Dec 03 12:30:14 crc kubenswrapper[5052]: I1203 12:30:14.511779 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-sb-0" event={"ID":"50f5af82-8ee5-45f4-b158-2166f1be6cb8","Type":"ContainerDied","Data":"e9076e00de7ae30a9d54e2e79b337633b061bc7f8a01e8aa825185ef328777b0"} Dec 03 12:30:14 crc kubenswrapper[5052]: I1203 12:30:14.539839 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-769cb78446-m6hzp" event={"ID":"760aa987-6355-40cb-97c0-16c7eeddd135","Type":"ContainerStarted","Data":"b2c4cb5fe70990b0b235abbc3aedfcf98912b1818086712902f3b4d14422a734"} Dec 03 12:30:14 crc kubenswrapper[5052]: I1203 12:30:14.543972 5052 generic.go:334] "Generic (PLEG): container finished" podID="a0e45e1a-ced9-4958-9a75-24a69280f679" containerID="009d530d399c78fb2e9155e989a69050f595b2f0a1be3bf13247c7f1d91a158a" exitCode=143 Dec 03 12:30:14 crc kubenswrapper[5052]: I1203 12:30:14.544264 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-688bd8fb5d-pjs4f" event={"ID":"a0e45e1a-ced9-4958-9a75-24a69280f679","Type":"ContainerDied","Data":"009d530d399c78fb2e9155e989a69050f595b2f0a1be3bf13247c7f1d91a158a"} Dec 03 12:30:14 crc kubenswrapper[5052]: I1203 12:30:14.544303 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:30:14 crc kubenswrapper[5052]: I1203 12:30:14.550954 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/keystone-6b8b855786-bltgj" Dec 03 12:30:14 crc kubenswrapper[5052]: I1203 12:30:14.590444 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/keystone-6b8b855786-bltgj" podStartSLOduration=4.590424435 podStartE2EDuration="4.590424435s" podCreationTimestamp="2025-12-03 12:30:10 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:30:14.575071227 +0000 UTC m=+4205.163312991" watchObservedRunningTime="2025-12-03 12:30:14.590424435 +0000 UTC m=+4205.178666189" Dec 03 12:30:14 crc kubenswrapper[5052]: I1203 12:30:14.741773 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ovsdbserver-sb-0" podUID="50f5af82-8ee5-45f4-b158-2166f1be6cb8" containerName="ovsdbserver-sb" containerID="cri-o://e56245b2be2e929328f0828e016ea41db39854cc67eb6cfae7d314651f18c3c0" gracePeriod=299 Dec 03 12:30:14 crc kubenswrapper[5052]: E1203 12:30:14.756551 5052 log.go:32] "ExecSync cmd from runtime service failed" err=< Dec 03 12:30:14 crc kubenswrapper[5052]: rpc error: code = Unknown desc = command error: the container `e56245b2be2e929328f0828e016ea41db39854cc67eb6cfae7d314651f18c3c0` is not running Dec 03 12:30:14 crc kubenswrapper[5052]: , stdout: , stderr: , exit code -1 Dec 03 12:30:14 crc kubenswrapper[5052]: > containerID="e56245b2be2e929328f0828e016ea41db39854cc67eb6cfae7d314651f18c3c0" cmd=["/usr/bin/pidof","ovsdb-server"] Dec 03 12:30:14 crc kubenswrapper[5052]: E1203 12:30:14.772664 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of e56245b2be2e929328f0828e016ea41db39854cc67eb6cfae7d314651f18c3c0 is running failed: container process not found" containerID="e56245b2be2e929328f0828e016ea41db39854cc67eb6cfae7d314651f18c3c0" cmd=["/usr/bin/pidof","ovsdb-server"] Dec 03 12:30:14 crc kubenswrapper[5052]: E1203 12:30:14.786828 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of e56245b2be2e929328f0828e016ea41db39854cc67eb6cfae7d314651f18c3c0 is running failed: container process not found" containerID="e56245b2be2e929328f0828e016ea41db39854cc67eb6cfae7d314651f18c3c0" cmd=["/usr/bin/pidof","ovsdb-server"] Dec 03 12:30:14 crc kubenswrapper[5052]: E1203 12:30:14.786900 5052 prober.go:104] "Probe errored" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of e56245b2be2e929328f0828e016ea41db39854cc67eb6cfae7d314651f18c3c0 is running failed: container process not found" probeType="Readiness" pod="openstack-kuttl-tests/ovsdbserver-sb-0" podUID="50f5af82-8ee5-45f4-b158-2166f1be6cb8" containerName="ovsdbserver-sb" Dec 03 12:30:14 crc kubenswrapper[5052]: I1203 12:30:14.833346 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-api-0" podUID="4314928d-a43f-4b38-9c72-63977e59345b" containerName="nova-api-api" probeResult="failure" output="Get \"https://10.217.0.214:8774/\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Dec 03 12:30:14 crc kubenswrapper[5052]: I1203 12:30:14.833669 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-api-0" podUID="4314928d-a43f-4b38-9c72-63977e59345b" containerName="nova-api-log" probeResult="failure" output="Get \"https://10.217.0.214:8774/\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Dec 03 12:30:14 crc kubenswrapper[5052]: I1203 12:30:14.959667 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:30:15 crc kubenswrapper[5052]: I1203 12:30:15.061664 5052 prober.go:107] "Probe failed" probeType="Liveness" pod="openstack-kuttl-tests/barbican-api-ff6b8c44b-7ndgm" podUID="48831cad-910f-46f1-aefe-7cc4cbfa984f" containerName="barbican-api" probeResult="failure" output="Get \"https://10.217.0.204:9311/healthcheck\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Dec 03 12:30:15 crc kubenswrapper[5052]: I1203 12:30:15.063885 5052 prober.go:107] "Probe failed" probeType="Liveness" pod="openstack-kuttl-tests/barbican-api-ff6b8c44b-7ndgm" podUID="48831cad-910f-46f1-aefe-7cc4cbfa984f" containerName="barbican-api-log" probeResult="failure" output="Get \"https://10.217.0.204:9311/healthcheck\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Dec 03 12:30:15 crc kubenswrapper[5052]: I1203 12:30:15.357731 5052 scope.go:117] "RemoveContainer" containerID="67ce4d39f2bdc99c3f3be2f5c14c987070c0b90d869d6a4344098656d6746ae9" Dec 03 12:30:15 crc kubenswrapper[5052]: I1203 12:30:15.557694 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-58d7cc6794-9z49k" Dec 03 12:30:15 crc kubenswrapper[5052]: I1203 12:30:15.682090 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_ovsdbserver-nb-0_98f722f9-fa25-40b6-84ba-96127d07293c/ovsdbserver-nb/0.log" Dec 03 12:30:15 crc kubenswrapper[5052]: I1203 12:30:15.682222 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:30:15 crc kubenswrapper[5052]: I1203 12:30:15.719126 5052 generic.go:334] "Generic (PLEG): container finished" podID="50f5af82-8ee5-45f4-b158-2166f1be6cb8" containerID="e56245b2be2e929328f0828e016ea41db39854cc67eb6cfae7d314651f18c3c0" exitCode=0 Dec 03 12:30:15 crc kubenswrapper[5052]: I1203 12:30:15.719494 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:30:15 crc kubenswrapper[5052]: I1203 12:30:15.721278 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-5c6478db8b-bt2rw" Dec 03 12:30:15 crc kubenswrapper[5052]: I1203 12:30:15.768640 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/98f722f9-fa25-40b6-84ba-96127d07293c-ovsdb-rundir\") pod \"98f722f9-fa25-40b6-84ba-96127d07293c\" (UID: \"98f722f9-fa25-40b6-84ba-96127d07293c\") " Dec 03 12:30:15 crc kubenswrapper[5052]: I1203 12:30:15.768694 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/98f722f9-fa25-40b6-84ba-96127d07293c-config\") pod \"98f722f9-fa25-40b6-84ba-96127d07293c\" (UID: \"98f722f9-fa25-40b6-84ba-96127d07293c\") " Dec 03 12:30:15 crc kubenswrapper[5052]: I1203 12:30:15.768730 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/98f722f9-fa25-40b6-84ba-96127d07293c-metrics-certs-tls-certs\") pod \"98f722f9-fa25-40b6-84ba-96127d07293c\" (UID: \"98f722f9-fa25-40b6-84ba-96127d07293c\") " Dec 03 12:30:15 crc kubenswrapper[5052]: I1203 12:30:15.768763 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/8bddf2b1-8116-4b6e-9cd8-be8fe1cd4b64-scripts\") pod \"8bddf2b1-8116-4b6e-9cd8-be8fe1cd4b64\" (UID: \"8bddf2b1-8116-4b6e-9cd8-be8fe1cd4b64\") " Dec 03 12:30:15 crc kubenswrapper[5052]: I1203 12:30:15.768785 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovsdbserver-nb-tls-certs\" (UniqueName: \"kubernetes.io/secret/98f722f9-fa25-40b6-84ba-96127d07293c-ovsdbserver-nb-tls-certs\") pod \"98f722f9-fa25-40b6-84ba-96127d07293c\" (UID: \"98f722f9-fa25-40b6-84ba-96127d07293c\") " Dec 03 12:30:15 crc kubenswrapper[5052]: I1203 12:30:15.768810 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/1de1d25b-cbcc-4d08-a7cd-aa54ac71aecb-operator-scripts\") pod \"1de1d25b-cbcc-4d08-a7cd-aa54ac71aecb\" (UID: \"1de1d25b-cbcc-4d08-a7cd-aa54ac71aecb\") " Dec 03 12:30:15 crc kubenswrapper[5052]: I1203 12:30:15.768858 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/8bddf2b1-8116-4b6e-9cd8-be8fe1cd4b64-config-data\") pod \"8bddf2b1-8116-4b6e-9cd8-be8fe1cd4b64\" (UID: \"8bddf2b1-8116-4b6e-9cd8-be8fe1cd4b64\") " Dec 03 12:30:15 crc kubenswrapper[5052]: I1203 12:30:15.768876 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8bddf2b1-8116-4b6e-9cd8-be8fe1cd4b64-combined-ca-bundle\") pod \"8bddf2b1-8116-4b6e-9cd8-be8fe1cd4b64\" (UID: \"8bddf2b1-8116-4b6e-9cd8-be8fe1cd4b64\") " Dec 03 12:30:15 crc kubenswrapper[5052]: I1203 12:30:15.768924 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/1de1d25b-cbcc-4d08-a7cd-aa54ac71aecb-kolla-config\") pod \"1de1d25b-cbcc-4d08-a7cd-aa54ac71aecb\" (UID: \"1de1d25b-cbcc-4d08-a7cd-aa54ac71aecb\") " Dec 03 12:30:15 crc kubenswrapper[5052]: I1203 12:30:15.768949 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/1de1d25b-cbcc-4d08-a7cd-aa54ac71aecb-galera-tls-certs\") pod \"1de1d25b-cbcc-4d08-a7cd-aa54ac71aecb\" (UID: \"1de1d25b-cbcc-4d08-a7cd-aa54ac71aecb\") " Dec 03 12:30:15 crc kubenswrapper[5052]: I1203 12:30:15.768970 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1de1d25b-cbcc-4d08-a7cd-aa54ac71aecb-combined-ca-bundle\") pod \"1de1d25b-cbcc-4d08-a7cd-aa54ac71aecb\" (UID: \"1de1d25b-cbcc-4d08-a7cd-aa54ac71aecb\") " Dec 03 12:30:15 crc kubenswrapper[5052]: I1203 12:30:15.769025 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4jrvq\" (UniqueName: \"kubernetes.io/projected/98f722f9-fa25-40b6-84ba-96127d07293c-kube-api-access-4jrvq\") pod \"98f722f9-fa25-40b6-84ba-96127d07293c\" (UID: \"98f722f9-fa25-40b6-84ba-96127d07293c\") " Dec 03 12:30:15 crc kubenswrapper[5052]: I1203 12:30:15.769061 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/1de1d25b-cbcc-4d08-a7cd-aa54ac71aecb-config-data-generated\") pod \"1de1d25b-cbcc-4d08-a7cd-aa54ac71aecb\" (UID: \"1de1d25b-cbcc-4d08-a7cd-aa54ac71aecb\") " Dec 03 12:30:15 crc kubenswrapper[5052]: I1203 12:30:15.769085 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-thvgx\" (UniqueName: \"kubernetes.io/projected/1de1d25b-cbcc-4d08-a7cd-aa54ac71aecb-kube-api-access-thvgx\") pod \"1de1d25b-cbcc-4d08-a7cd-aa54ac71aecb\" (UID: \"1de1d25b-cbcc-4d08-a7cd-aa54ac71aecb\") " Dec 03 12:30:15 crc kubenswrapper[5052]: I1203 12:30:15.769125 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/98f722f9-fa25-40b6-84ba-96127d07293c-combined-ca-bundle\") pod \"98f722f9-fa25-40b6-84ba-96127d07293c\" (UID: \"98f722f9-fa25-40b6-84ba-96127d07293c\") " Dec 03 12:30:15 crc kubenswrapper[5052]: I1203 12:30:15.769148 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nxlhc\" (UniqueName: \"kubernetes.io/projected/8bddf2b1-8116-4b6e-9cd8-be8fe1cd4b64-kube-api-access-nxlhc\") pod \"8bddf2b1-8116-4b6e-9cd8-be8fe1cd4b64\" (UID: \"8bddf2b1-8116-4b6e-9cd8-be8fe1cd4b64\") " Dec 03 12:30:15 crc kubenswrapper[5052]: I1203 12:30:15.769184 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/1de1d25b-cbcc-4d08-a7cd-aa54ac71aecb-config-data-default\") pod \"1de1d25b-cbcc-4d08-a7cd-aa54ac71aecb\" (UID: \"1de1d25b-cbcc-4d08-a7cd-aa54ac71aecb\") " Dec 03 12:30:15 crc kubenswrapper[5052]: I1203 12:30:15.769218 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovndbcluster-nb-etc-ovn\" (UniqueName: \"kubernetes.io/local-volume/local-storage03-crc\") pod \"98f722f9-fa25-40b6-84ba-96127d07293c\" (UID: \"98f722f9-fa25-40b6-84ba-96127d07293c\") " Dec 03 12:30:15 crc kubenswrapper[5052]: I1203 12:30:15.769249 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/8bddf2b1-8116-4b6e-9cd8-be8fe1cd4b64-internal-tls-certs\") pod \"8bddf2b1-8116-4b6e-9cd8-be8fe1cd4b64\" (UID: \"8bddf2b1-8116-4b6e-9cd8-be8fe1cd4b64\") " Dec 03 12:30:15 crc kubenswrapper[5052]: I1203 12:30:15.769308 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/8bddf2b1-8116-4b6e-9cd8-be8fe1cd4b64-public-tls-certs\") pod \"8bddf2b1-8116-4b6e-9cd8-be8fe1cd4b64\" (UID: \"8bddf2b1-8116-4b6e-9cd8-be8fe1cd4b64\") " Dec 03 12:30:15 crc kubenswrapper[5052]: I1203 12:30:15.769333 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/98f722f9-fa25-40b6-84ba-96127d07293c-scripts\") pod \"98f722f9-fa25-40b6-84ba-96127d07293c\" (UID: \"98f722f9-fa25-40b6-84ba-96127d07293c\") " Dec 03 12:30:15 crc kubenswrapper[5052]: I1203 12:30:15.769351 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"mysql-db\" (UniqueName: \"kubernetes.io/local-volume/local-storage05-crc\") pod \"1de1d25b-cbcc-4d08-a7cd-aa54ac71aecb\" (UID: \"1de1d25b-cbcc-4d08-a7cd-aa54ac71aecb\") " Dec 03 12:30:15 crc kubenswrapper[5052]: I1203 12:30:15.769377 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/8bddf2b1-8116-4b6e-9cd8-be8fe1cd4b64-logs\") pod \"8bddf2b1-8116-4b6e-9cd8-be8fe1cd4b64\" (UID: \"8bddf2b1-8116-4b6e-9cd8-be8fe1cd4b64\") " Dec 03 12:30:15 crc kubenswrapper[5052]: I1203 12:30:15.770289 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8bddf2b1-8116-4b6e-9cd8-be8fe1cd4b64-logs" (OuterVolumeSpecName: "logs") pod "8bddf2b1-8116-4b6e-9cd8-be8fe1cd4b64" (UID: "8bddf2b1-8116-4b6e-9cd8-be8fe1cd4b64"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:30:15 crc kubenswrapper[5052]: I1203 12:30:15.770346 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1de1d25b-cbcc-4d08-a7cd-aa54ac71aecb-config-data-default" (OuterVolumeSpecName: "config-data-default") pod "1de1d25b-cbcc-4d08-a7cd-aa54ac71aecb" (UID: "1de1d25b-cbcc-4d08-a7cd-aa54ac71aecb"). InnerVolumeSpecName "config-data-default". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:30:15 crc kubenswrapper[5052]: I1203 12:30:15.775394 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_ovsdbserver-nb-0_98f722f9-fa25-40b6-84ba-96127d07293c/ovsdbserver-nb/0.log" Dec 03 12:30:15 crc kubenswrapper[5052]: I1203 12:30:15.776941 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/98f722f9-fa25-40b6-84ba-96127d07293c-scripts" (OuterVolumeSpecName: "scripts") pod "98f722f9-fa25-40b6-84ba-96127d07293c" (UID: "98f722f9-fa25-40b6-84ba-96127d07293c"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:30:15 crc kubenswrapper[5052]: I1203 12:30:15.777335 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/98f722f9-fa25-40b6-84ba-96127d07293c-kube-api-access-4jrvq" (OuterVolumeSpecName: "kube-api-access-4jrvq") pod "98f722f9-fa25-40b6-84ba-96127d07293c" (UID: "98f722f9-fa25-40b6-84ba-96127d07293c"). InnerVolumeSpecName "kube-api-access-4jrvq". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:30:15 crc kubenswrapper[5052]: I1203 12:30:15.781767 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1de1d25b-cbcc-4d08-a7cd-aa54ac71aecb-config-data-generated" (OuterVolumeSpecName: "config-data-generated") pod "1de1d25b-cbcc-4d08-a7cd-aa54ac71aecb" (UID: "1de1d25b-cbcc-4d08-a7cd-aa54ac71aecb"). InnerVolumeSpecName "config-data-generated". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:30:15 crc kubenswrapper[5052]: I1203 12:30:15.781937 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:30:15 crc kubenswrapper[5052]: I1203 12:30:15.814412 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1de1d25b-cbcc-4d08-a7cd-aa54ac71aecb-kolla-config" (OuterVolumeSpecName: "kolla-config") pod "1de1d25b-cbcc-4d08-a7cd-aa54ac71aecb" (UID: "1de1d25b-cbcc-4d08-a7cd-aa54ac71aecb"). InnerVolumeSpecName "kolla-config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:30:15 crc kubenswrapper[5052]: I1203 12:30:15.815846 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/98f722f9-fa25-40b6-84ba-96127d07293c-config" (OuterVolumeSpecName: "config") pod "98f722f9-fa25-40b6-84ba-96127d07293c" (UID: "98f722f9-fa25-40b6-84ba-96127d07293c"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:30:15 crc kubenswrapper[5052]: I1203 12:30:15.820891 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/98f722f9-fa25-40b6-84ba-96127d07293c-ovsdb-rundir" (OuterVolumeSpecName: "ovsdb-rundir") pod "98f722f9-fa25-40b6-84ba-96127d07293c" (UID: "98f722f9-fa25-40b6-84ba-96127d07293c"). InnerVolumeSpecName "ovsdb-rundir". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:30:15 crc kubenswrapper[5052]: I1203 12:30:15.846472 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8bddf2b1-8116-4b6e-9cd8-be8fe1cd4b64-kube-api-access-nxlhc" (OuterVolumeSpecName: "kube-api-access-nxlhc") pod "8bddf2b1-8116-4b6e-9cd8-be8fe1cd4b64" (UID: "8bddf2b1-8116-4b6e-9cd8-be8fe1cd4b64"). InnerVolumeSpecName "kube-api-access-nxlhc". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:30:15 crc kubenswrapper[5052]: I1203 12:30:15.847059 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1de1d25b-cbcc-4d08-a7cd-aa54ac71aecb-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "1de1d25b-cbcc-4d08-a7cd-aa54ac71aecb" (UID: "1de1d25b-cbcc-4d08-a7cd-aa54ac71aecb"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:30:15 crc kubenswrapper[5052]: I1203 12:30:15.859459 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8bddf2b1-8116-4b6e-9cd8-be8fe1cd4b64-scripts" (OuterVolumeSpecName: "scripts") pod "8bddf2b1-8116-4b6e-9cd8-be8fe1cd4b64" (UID: "8bddf2b1-8116-4b6e-9cd8-be8fe1cd4b64"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:15 crc kubenswrapper[5052]: I1203 12:30:15.873151 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/98f722f9-fa25-40b6-84ba-96127d07293c-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:15 crc kubenswrapper[5052]: I1203 12:30:15.873185 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/8bddf2b1-8116-4b6e-9cd8-be8fe1cd4b64-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:15 crc kubenswrapper[5052]: I1203 12:30:15.873194 5052 reconciler_common.go:293] "Volume detached for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/98f722f9-fa25-40b6-84ba-96127d07293c-ovsdb-rundir\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:15 crc kubenswrapper[5052]: I1203 12:30:15.873204 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/98f722f9-fa25-40b6-84ba-96127d07293c-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:15 crc kubenswrapper[5052]: I1203 12:30:15.873215 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/8bddf2b1-8116-4b6e-9cd8-be8fe1cd4b64-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:15 crc kubenswrapper[5052]: I1203 12:30:15.873224 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/1de1d25b-cbcc-4d08-a7cd-aa54ac71aecb-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:15 crc kubenswrapper[5052]: I1203 12:30:15.873231 5052 reconciler_common.go:293] "Volume detached for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/1de1d25b-cbcc-4d08-a7cd-aa54ac71aecb-kolla-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:15 crc kubenswrapper[5052]: I1203 12:30:15.873239 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-4jrvq\" (UniqueName: \"kubernetes.io/projected/98f722f9-fa25-40b6-84ba-96127d07293c-kube-api-access-4jrvq\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:15 crc kubenswrapper[5052]: I1203 12:30:15.873249 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/1de1d25b-cbcc-4d08-a7cd-aa54ac71aecb-config-data-generated\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:15 crc kubenswrapper[5052]: I1203 12:30:15.873257 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-nxlhc\" (UniqueName: \"kubernetes.io/projected/8bddf2b1-8116-4b6e-9cd8-be8fe1cd4b64-kube-api-access-nxlhc\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:15 crc kubenswrapper[5052]: I1203 12:30:15.873265 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/1de1d25b-cbcc-4d08-a7cd-aa54ac71aecb-config-data-default\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:15 crc kubenswrapper[5052]: I1203 12:30:15.876910 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage03-crc" (OuterVolumeSpecName: "ovndbcluster-nb-etc-ovn") pod "98f722f9-fa25-40b6-84ba-96127d07293c" (UID: "98f722f9-fa25-40b6-84ba-96127d07293c"). InnerVolumeSpecName "local-storage03-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 12:30:15 crc kubenswrapper[5052]: I1203 12:30:15.944487 5052 generic.go:334] "Generic (PLEG): container finished" podID="a0e45e1a-ced9-4958-9a75-24a69280f679" containerID="b5aa97990bcd4bfc80db39c69ed703786808679eda611dc607c177d51eeb3707" exitCode=0 Dec 03 12:30:15 crc kubenswrapper[5052]: I1203 12:30:15.958154 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1de1d25b-cbcc-4d08-a7cd-aa54ac71aecb-kube-api-access-thvgx" (OuterVolumeSpecName: "kube-api-access-thvgx") pod "1de1d25b-cbcc-4d08-a7cd-aa54ac71aecb" (UID: "1de1d25b-cbcc-4d08-a7cd-aa54ac71aecb"). InnerVolumeSpecName "kube-api-access-thvgx". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:30:15 crc kubenswrapper[5052]: I1203 12:30:15.963010 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:30:15 crc kubenswrapper[5052]: I1203 12:30:15.983143 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-thvgx\" (UniqueName: \"kubernetes.io/projected/1de1d25b-cbcc-4d08-a7cd-aa54ac71aecb-kube-api-access-thvgx\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:15 crc kubenswrapper[5052]: I1203 12:30:15.990016 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage03-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage03-crc\") on node \"crc\" " Dec 03 12:30:16 crc kubenswrapper[5052]: I1203 12:30:16.065971 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage05-crc" (OuterVolumeSpecName: "mysql-db") pod "1de1d25b-cbcc-4d08-a7cd-aa54ac71aecb" (UID: "1de1d25b-cbcc-4d08-a7cd-aa54ac71aecb"). InnerVolumeSpecName "local-storage05-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 12:30:16 crc kubenswrapper[5052]: I1203 12:30:16.097944 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage05-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage05-crc\") on node \"crc\" " Dec 03 12:30:16 crc kubenswrapper[5052]: I1203 12:30:16.267773 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage03-crc" (UniqueName: "kubernetes.io/local-volume/local-storage03-crc") on node "crc" Dec 03 12:30:16 crc kubenswrapper[5052]: I1203 12:30:16.301916 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage03-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage03-crc\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:16 crc kubenswrapper[5052]: I1203 12:30:16.346391 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage05-crc" (UniqueName: "kubernetes.io/local-volume/local-storage05-crc") on node "crc" Dec 03 12:30:16 crc kubenswrapper[5052]: I1203 12:30:16.405932 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage05-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage05-crc\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:16 crc kubenswrapper[5052]: I1203 12:30:16.705192 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-7dffdbf8-hzmbm" Dec 03 12:30:16 crc kubenswrapper[5052]: I1203 12:30:16.712094 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1de1d25b-cbcc-4d08-a7cd-aa54ac71aecb-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "1de1d25b-cbcc-4d08-a7cd-aa54ac71aecb" (UID: "1de1d25b-cbcc-4d08-a7cd-aa54ac71aecb"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:16 crc kubenswrapper[5052]: E1203 12:30:16.728487 5052 kubelet.go:2526] "Housekeeping took longer than expected" err="housekeeping took too long" expected="1s" actual="1.057s" Dec 03 12:30:16 crc kubenswrapper[5052]: I1203 12:30:16.728697 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:30:16 crc kubenswrapper[5052]: I1203 12:30:16.728723 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-5c6478db8b-bt2rw" event={"ID":"8bddf2b1-8116-4b6e-9cd8-be8fe1cd4b64","Type":"ContainerDied","Data":"bbf760a832ee67de313c68ecfd209742e1ca2c2298b42e73aea44c6e592aa9d7"} Dec 03 12:30:16 crc kubenswrapper[5052]: I1203 12:30:16.728750 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-sb-0" event={"ID":"50f5af82-8ee5-45f4-b158-2166f1be6cb8","Type":"ContainerDied","Data":"e56245b2be2e929328f0828e016ea41db39854cc67eb6cfae7d314651f18c3c0"} Dec 03 12:30:16 crc kubenswrapper[5052]: I1203 12:30:16.728764 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-nb-0" event={"ID":"98f722f9-fa25-40b6-84ba-96127d07293c","Type":"ContainerDied","Data":"b6c799fc84b3d867e28e78a12cece87252f80013fd30584cea813d72ca3deda4"} Dec 03 12:30:16 crc kubenswrapper[5052]: I1203 12:30:16.728777 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-7dffdbf8-hzmbm" event={"ID":"53fe6403-27db-44b7-a9a6-787afbc3bbb8","Type":"ContainerDied","Data":"9f3746f193daa58337fe694f77011f189858dade68192aac0e70fb07c702c05a"} Dec 03 12:30:16 crc kubenswrapper[5052]: I1203 12:30:16.728791 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ovn-northd-0"] Dec 03 12:30:16 crc kubenswrapper[5052]: I1203 12:30:16.728807 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ovn-northd-0"] Dec 03 12:30:16 crc kubenswrapper[5052]: I1203 12:30:16.728823 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-688bd8fb5d-pjs4f" event={"ID":"a0e45e1a-ced9-4958-9a75-24a69280f679","Type":"ContainerDied","Data":"b5aa97990bcd4bfc80db39c69ed703786808679eda611dc607c177d51eeb3707"} Dec 03 12:30:16 crc kubenswrapper[5052]: I1203 12:30:16.728840 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ovn-northd-0"] Dec 03 12:30:16 crc kubenswrapper[5052]: E1203 12:30:16.729228 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="53fe6403-27db-44b7-a9a6-787afbc3bbb8" containerName="placement-api" Dec 03 12:30:16 crc kubenswrapper[5052]: I1203 12:30:16.729249 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="53fe6403-27db-44b7-a9a6-787afbc3bbb8" containerName="placement-api" Dec 03 12:30:16 crc kubenswrapper[5052]: E1203 12:30:16.729297 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="1de1d25b-cbcc-4d08-a7cd-aa54ac71aecb" containerName="galera" Dec 03 12:30:16 crc kubenswrapper[5052]: I1203 12:30:16.729306 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="1de1d25b-cbcc-4d08-a7cd-aa54ac71aecb" containerName="galera" Dec 03 12:30:16 crc kubenswrapper[5052]: E1203 12:30:16.729319 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="1de1d25b-cbcc-4d08-a7cd-aa54ac71aecb" containerName="mysql-bootstrap" Dec 03 12:30:16 crc kubenswrapper[5052]: I1203 12:30:16.729327 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="1de1d25b-cbcc-4d08-a7cd-aa54ac71aecb" containerName="mysql-bootstrap" Dec 03 12:30:16 crc kubenswrapper[5052]: E1203 12:30:16.729337 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="8bddf2b1-8116-4b6e-9cd8-be8fe1cd4b64" containerName="placement-api" Dec 03 12:30:16 crc kubenswrapper[5052]: I1203 12:30:16.729346 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="8bddf2b1-8116-4b6e-9cd8-be8fe1cd4b64" containerName="placement-api" Dec 03 12:30:16 crc kubenswrapper[5052]: E1203 12:30:16.729363 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="53fe6403-27db-44b7-a9a6-787afbc3bbb8" containerName="placement-log" Dec 03 12:30:16 crc kubenswrapper[5052]: I1203 12:30:16.729372 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="53fe6403-27db-44b7-a9a6-787afbc3bbb8" containerName="placement-log" Dec 03 12:30:16 crc kubenswrapper[5052]: E1203 12:30:16.729385 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="98f722f9-fa25-40b6-84ba-96127d07293c" containerName="openstack-network-exporter" Dec 03 12:30:16 crc kubenswrapper[5052]: I1203 12:30:16.729393 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="98f722f9-fa25-40b6-84ba-96127d07293c" containerName="openstack-network-exporter" Dec 03 12:30:16 crc kubenswrapper[5052]: E1203 12:30:16.729412 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="8bddf2b1-8116-4b6e-9cd8-be8fe1cd4b64" containerName="placement-log" Dec 03 12:30:16 crc kubenswrapper[5052]: I1203 12:30:16.729419 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="8bddf2b1-8116-4b6e-9cd8-be8fe1cd4b64" containerName="placement-log" Dec 03 12:30:16 crc kubenswrapper[5052]: E1203 12:30:16.729435 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="98f722f9-fa25-40b6-84ba-96127d07293c" containerName="ovsdbserver-nb" Dec 03 12:30:16 crc kubenswrapper[5052]: I1203 12:30:16.729443 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="98f722f9-fa25-40b6-84ba-96127d07293c" containerName="ovsdbserver-nb" Dec 03 12:30:16 crc kubenswrapper[5052]: I1203 12:30:16.732011 5052 scope.go:117] "RemoveContainer" containerID="5f22191c711f94909400c3cf19b845a8d0a3f693f6879266161b330e5d69cafc" Dec 03 12:30:16 crc kubenswrapper[5052]: I1203 12:30:16.738147 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="98f722f9-fa25-40b6-84ba-96127d07293c" containerName="openstack-network-exporter" Dec 03 12:30:16 crc kubenswrapper[5052]: I1203 12:30:16.738219 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="8bddf2b1-8116-4b6e-9cd8-be8fe1cd4b64" containerName="placement-log" Dec 03 12:30:16 crc kubenswrapper[5052]: I1203 12:30:16.738245 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="53fe6403-27db-44b7-a9a6-787afbc3bbb8" containerName="placement-api" Dec 03 12:30:16 crc kubenswrapper[5052]: I1203 12:30:16.738261 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="98f722f9-fa25-40b6-84ba-96127d07293c" containerName="ovsdbserver-nb" Dec 03 12:30:16 crc kubenswrapper[5052]: I1203 12:30:16.738277 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="1de1d25b-cbcc-4d08-a7cd-aa54ac71aecb" containerName="galera" Dec 03 12:30:16 crc kubenswrapper[5052]: I1203 12:30:16.738291 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="53fe6403-27db-44b7-a9a6-787afbc3bbb8" containerName="placement-log" Dec 03 12:30:16 crc kubenswrapper[5052]: I1203 12:30:16.738304 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="8bddf2b1-8116-4b6e-9cd8-be8fe1cd4b64" containerName="placement-api" Dec 03 12:30:16 crc kubenswrapper[5052]: I1203 12:30:16.739771 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ovn-northd-0"] Dec 03 12:30:16 crc kubenswrapper[5052]: I1203 12:30:16.739808 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-cell1-galera-0" event={"ID":"1de1d25b-cbcc-4d08-a7cd-aa54ac71aecb","Type":"ContainerDied","Data":"bcaa05975157bb25a2852a7e2a286e65dd456e820f2697cfde9f55c2ff7275f9"} Dec 03 12:30:16 crc kubenswrapper[5052]: I1203 12:30:16.739850 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/neutron-76bcd49fb6-54vvs"] Dec 03 12:30:16 crc kubenswrapper[5052]: I1203 12:30:16.739876 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-58d7cc6794-9z49k"] Dec 03 12:30:16 crc kubenswrapper[5052]: I1203 12:30:16.739976 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:30:16 crc kubenswrapper[5052]: I1203 12:30:16.742352 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1de1d25b-cbcc-4d08-a7cd-aa54ac71aecb-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:16 crc kubenswrapper[5052]: I1203 12:30:16.743052 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"ovnnorthd-scripts" Dec 03 12:30:16 crc kubenswrapper[5052]: I1203 12:30:16.743299 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"ovnnorthd-config" Dec 03 12:30:16 crc kubenswrapper[5052]: I1203 12:30:16.743912 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ovnnorthd-ovnnorthd-dockercfg-lp65l" Dec 03 12:30:16 crc kubenswrapper[5052]: I1203 12:30:16.744717 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-ovnnorthd-ovndbs" Dec 03 12:30:16 crc kubenswrapper[5052]: I1203 12:30:16.786935 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/98f722f9-fa25-40b6-84ba-96127d07293c-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "98f722f9-fa25-40b6-84ba-96127d07293c" (UID: "98f722f9-fa25-40b6-84ba-96127d07293c"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:16 crc kubenswrapper[5052]: I1203 12:30:16.845469 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/53fe6403-27db-44b7-a9a6-787afbc3bbb8-logs\") pod \"53fe6403-27db-44b7-a9a6-787afbc3bbb8\" (UID: \"53fe6403-27db-44b7-a9a6-787afbc3bbb8\") " Dec 03 12:30:16 crc kubenswrapper[5052]: I1203 12:30:16.845598 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tpxrx\" (UniqueName: \"kubernetes.io/projected/53fe6403-27db-44b7-a9a6-787afbc3bbb8-kube-api-access-tpxrx\") pod \"53fe6403-27db-44b7-a9a6-787afbc3bbb8\" (UID: \"53fe6403-27db-44b7-a9a6-787afbc3bbb8\") " Dec 03 12:30:16 crc kubenswrapper[5052]: I1203 12:30:16.845643 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/53fe6403-27db-44b7-a9a6-787afbc3bbb8-scripts\") pod \"53fe6403-27db-44b7-a9a6-787afbc3bbb8\" (UID: \"53fe6403-27db-44b7-a9a6-787afbc3bbb8\") " Dec 03 12:30:16 crc kubenswrapper[5052]: I1203 12:30:16.845664 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/53fe6403-27db-44b7-a9a6-787afbc3bbb8-internal-tls-certs\") pod \"53fe6403-27db-44b7-a9a6-787afbc3bbb8\" (UID: \"53fe6403-27db-44b7-a9a6-787afbc3bbb8\") " Dec 03 12:30:16 crc kubenswrapper[5052]: I1203 12:30:16.845794 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/53fe6403-27db-44b7-a9a6-787afbc3bbb8-combined-ca-bundle\") pod \"53fe6403-27db-44b7-a9a6-787afbc3bbb8\" (UID: \"53fe6403-27db-44b7-a9a6-787afbc3bbb8\") " Dec 03 12:30:16 crc kubenswrapper[5052]: I1203 12:30:16.845823 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/53fe6403-27db-44b7-a9a6-787afbc3bbb8-public-tls-certs\") pod \"53fe6403-27db-44b7-a9a6-787afbc3bbb8\" (UID: \"53fe6403-27db-44b7-a9a6-787afbc3bbb8\") " Dec 03 12:30:16 crc kubenswrapper[5052]: I1203 12:30:16.845941 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/53fe6403-27db-44b7-a9a6-787afbc3bbb8-config-data\") pod \"53fe6403-27db-44b7-a9a6-787afbc3bbb8\" (UID: \"53fe6403-27db-44b7-a9a6-787afbc3bbb8\") " Dec 03 12:30:16 crc kubenswrapper[5052]: I1203 12:30:16.851324 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x5d8s\" (UniqueName: \"kubernetes.io/projected/a9f65397-a08a-4798-8532-646209c5d090-kube-api-access-x5d8s\") pod \"ovn-northd-0\" (UID: \"a9f65397-a08a-4798-8532-646209c5d090\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:30:16 crc kubenswrapper[5052]: I1203 12:30:16.851505 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-rundir\" (UniqueName: \"kubernetes.io/empty-dir/a9f65397-a08a-4798-8532-646209c5d090-ovn-rundir\") pod \"ovn-northd-0\" (UID: \"a9f65397-a08a-4798-8532-646209c5d090\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:30:16 crc kubenswrapper[5052]: I1203 12:30:16.851737 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/a9f65397-a08a-4798-8532-646209c5d090-scripts\") pod \"ovn-northd-0\" (UID: \"a9f65397-a08a-4798-8532-646209c5d090\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:30:16 crc kubenswrapper[5052]: I1203 12:30:16.855715 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/53fe6403-27db-44b7-a9a6-787afbc3bbb8-logs" (OuterVolumeSpecName: "logs") pod "53fe6403-27db-44b7-a9a6-787afbc3bbb8" (UID: "53fe6403-27db-44b7-a9a6-787afbc3bbb8"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:30:16 crc kubenswrapper[5052]: I1203 12:30:16.859900 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a9f65397-a08a-4798-8532-646209c5d090-combined-ca-bundle\") pod \"ovn-northd-0\" (UID: \"a9f65397-a08a-4798-8532-646209c5d090\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:30:16 crc kubenswrapper[5052]: I1203 12:30:16.859958 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-northd-tls-certs\" (UniqueName: \"kubernetes.io/secret/a9f65397-a08a-4798-8532-646209c5d090-ovn-northd-tls-certs\") pod \"ovn-northd-0\" (UID: \"a9f65397-a08a-4798-8532-646209c5d090\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:30:16 crc kubenswrapper[5052]: I1203 12:30:16.860027 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a9f65397-a08a-4798-8532-646209c5d090-config\") pod \"ovn-northd-0\" (UID: \"a9f65397-a08a-4798-8532-646209c5d090\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:30:16 crc kubenswrapper[5052]: I1203 12:30:16.860117 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/a9f65397-a08a-4798-8532-646209c5d090-metrics-certs-tls-certs\") pod \"ovn-northd-0\" (UID: \"a9f65397-a08a-4798-8532-646209c5d090\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:30:16 crc kubenswrapper[5052]: I1203 12:30:16.860289 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/98f722f9-fa25-40b6-84ba-96127d07293c-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:16 crc kubenswrapper[5052]: I1203 12:30:16.860312 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/53fe6403-27db-44b7-a9a6-787afbc3bbb8-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:16 crc kubenswrapper[5052]: I1203 12:30:16.861686 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/98f722f9-fa25-40b6-84ba-96127d07293c-ovsdbserver-nb-tls-certs" (OuterVolumeSpecName: "ovsdbserver-nb-tls-certs") pod "98f722f9-fa25-40b6-84ba-96127d07293c" (UID: "98f722f9-fa25-40b6-84ba-96127d07293c"). InnerVolumeSpecName "ovsdbserver-nb-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:16 crc kubenswrapper[5052]: I1203 12:30:16.907804 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/53fe6403-27db-44b7-a9a6-787afbc3bbb8-kube-api-access-tpxrx" (OuterVolumeSpecName: "kube-api-access-tpxrx") pod "53fe6403-27db-44b7-a9a6-787afbc3bbb8" (UID: "53fe6403-27db-44b7-a9a6-787afbc3bbb8"). InnerVolumeSpecName "kube-api-access-tpxrx". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:30:16 crc kubenswrapper[5052]: I1203 12:30:16.915689 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/53fe6403-27db-44b7-a9a6-787afbc3bbb8-scripts" (OuterVolumeSpecName: "scripts") pod "53fe6403-27db-44b7-a9a6-787afbc3bbb8" (UID: "53fe6403-27db-44b7-a9a6-787afbc3bbb8"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:16 crc kubenswrapper[5052]: I1203 12:30:16.955872 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8bddf2b1-8116-4b6e-9cd8-be8fe1cd4b64-config-data" (OuterVolumeSpecName: "config-data") pod "8bddf2b1-8116-4b6e-9cd8-be8fe1cd4b64" (UID: "8bddf2b1-8116-4b6e-9cd8-be8fe1cd4b64"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:16 crc kubenswrapper[5052]: I1203 12:30:16.961902 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-x5d8s\" (UniqueName: \"kubernetes.io/projected/a9f65397-a08a-4798-8532-646209c5d090-kube-api-access-x5d8s\") pod \"ovn-northd-0\" (UID: \"a9f65397-a08a-4798-8532-646209c5d090\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:30:16 crc kubenswrapper[5052]: I1203 12:30:16.961973 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovn-rundir\" (UniqueName: \"kubernetes.io/empty-dir/a9f65397-a08a-4798-8532-646209c5d090-ovn-rundir\") pod \"ovn-northd-0\" (UID: \"a9f65397-a08a-4798-8532-646209c5d090\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:30:16 crc kubenswrapper[5052]: I1203 12:30:16.961997 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/a9f65397-a08a-4798-8532-646209c5d090-scripts\") pod \"ovn-northd-0\" (UID: \"a9f65397-a08a-4798-8532-646209c5d090\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:30:16 crc kubenswrapper[5052]: I1203 12:30:16.962063 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a9f65397-a08a-4798-8532-646209c5d090-combined-ca-bundle\") pod \"ovn-northd-0\" (UID: \"a9f65397-a08a-4798-8532-646209c5d090\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:30:16 crc kubenswrapper[5052]: I1203 12:30:16.962087 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovn-northd-tls-certs\" (UniqueName: \"kubernetes.io/secret/a9f65397-a08a-4798-8532-646209c5d090-ovn-northd-tls-certs\") pod \"ovn-northd-0\" (UID: \"a9f65397-a08a-4798-8532-646209c5d090\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:30:16 crc kubenswrapper[5052]: I1203 12:30:16.962120 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a9f65397-a08a-4798-8532-646209c5d090-config\") pod \"ovn-northd-0\" (UID: \"a9f65397-a08a-4798-8532-646209c5d090\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:30:16 crc kubenswrapper[5052]: I1203 12:30:16.962153 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/a9f65397-a08a-4798-8532-646209c5d090-metrics-certs-tls-certs\") pod \"ovn-northd-0\" (UID: \"a9f65397-a08a-4798-8532-646209c5d090\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:30:16 crc kubenswrapper[5052]: I1203 12:30:16.962231 5052 reconciler_common.go:293] "Volume detached for volume \"ovsdbserver-nb-tls-certs\" (UniqueName: \"kubernetes.io/secret/98f722f9-fa25-40b6-84ba-96127d07293c-ovsdbserver-nb-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:16 crc kubenswrapper[5052]: I1203 12:30:16.962250 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/8bddf2b1-8116-4b6e-9cd8-be8fe1cd4b64-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:16 crc kubenswrapper[5052]: I1203 12:30:16.962261 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-tpxrx\" (UniqueName: \"kubernetes.io/projected/53fe6403-27db-44b7-a9a6-787afbc3bbb8-kube-api-access-tpxrx\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:16 crc kubenswrapper[5052]: I1203 12:30:16.962271 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/53fe6403-27db-44b7-a9a6-787afbc3bbb8-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:16 crc kubenswrapper[5052]: I1203 12:30:16.963215 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/a9f65397-a08a-4798-8532-646209c5d090-scripts\") pod \"ovn-northd-0\" (UID: \"a9f65397-a08a-4798-8532-646209c5d090\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:30:16 crc kubenswrapper[5052]: I1203 12:30:16.964223 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovn-rundir\" (UniqueName: \"kubernetes.io/empty-dir/a9f65397-a08a-4798-8532-646209c5d090-ovn-rundir\") pod \"ovn-northd-0\" (UID: \"a9f65397-a08a-4798-8532-646209c5d090\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:30:16 crc kubenswrapper[5052]: I1203 12:30:16.965451 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a9f65397-a08a-4798-8532-646209c5d090-config\") pod \"ovn-northd-0\" (UID: \"a9f65397-a08a-4798-8532-646209c5d090\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:30:16 crc kubenswrapper[5052]: I1203 12:30:16.987107 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovn-northd-tls-certs\" (UniqueName: \"kubernetes.io/secret/a9f65397-a08a-4798-8532-646209c5d090-ovn-northd-tls-certs\") pod \"ovn-northd-0\" (UID: \"a9f65397-a08a-4798-8532-646209c5d090\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:30:16 crc kubenswrapper[5052]: I1203 12:30:16.988110 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/a9f65397-a08a-4798-8532-646209c5d090-metrics-certs-tls-certs\") pod \"ovn-northd-0\" (UID: \"a9f65397-a08a-4798-8532-646209c5d090\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:30:16 crc kubenswrapper[5052]: I1203 12:30:16.990373 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a9f65397-a08a-4798-8532-646209c5d090-combined-ca-bundle\") pod \"ovn-northd-0\" (UID: \"a9f65397-a08a-4798-8532-646209c5d090\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:30:16 crc kubenswrapper[5052]: I1203 12:30:16.996578 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-x5d8s\" (UniqueName: \"kubernetes.io/projected/a9f65397-a08a-4798-8532-646209c5d090-kube-api-access-x5d8s\") pod \"ovn-northd-0\" (UID: \"a9f65397-a08a-4798-8532-646209c5d090\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:30:17 crc kubenswrapper[5052]: I1203 12:30:17.048746 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1de1d25b-cbcc-4d08-a7cd-aa54ac71aecb-galera-tls-certs" (OuterVolumeSpecName: "galera-tls-certs") pod "1de1d25b-cbcc-4d08-a7cd-aa54ac71aecb" (UID: "1de1d25b-cbcc-4d08-a7cd-aa54ac71aecb"). InnerVolumeSpecName "galera-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:17 crc kubenswrapper[5052]: I1203 12:30:17.048888 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8bddf2b1-8116-4b6e-9cd8-be8fe1cd4b64-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "8bddf2b1-8116-4b6e-9cd8-be8fe1cd4b64" (UID: "8bddf2b1-8116-4b6e-9cd8-be8fe1cd4b64"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:17 crc kubenswrapper[5052]: I1203 12:30:17.056181 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"a24b0824-26d0-4b81-b6cb-d617c36d45ab","Type":"ContainerStarted","Data":"4b3af605b0d8c2dcd2ab4507ac26ec3ce43fc470db20730adce6dbd29442e123"} Dec 03 12:30:17 crc kubenswrapper[5052]: I1203 12:30:17.061642 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-58d7cc6794-9z49k" event={"ID":"4b3e73a0-1320-427f-bcd7-d8062e315e39","Type":"ContainerStarted","Data":"409382895fc1e13c5f55bec79f9858c93d095ab609a5e7c29543e7085095770b"} Dec 03 12:30:17 crc kubenswrapper[5052]: I1203 12:30:17.063911 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8bddf2b1-8116-4b6e-9cd8-be8fe1cd4b64-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:17 crc kubenswrapper[5052]: I1203 12:30:17.063945 5052 reconciler_common.go:293] "Volume detached for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/1de1d25b-cbcc-4d08-a7cd-aa54ac71aecb-galera-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:17 crc kubenswrapper[5052]: I1203 12:30:17.084072 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-sb-0" event={"ID":"50f5af82-8ee5-45f4-b158-2166f1be6cb8","Type":"ContainerDied","Data":"a34cefb9fab15586d1ef541f04657dfba9f32d8dac3286264dba4943680b0e47"} Dec 03 12:30:17 crc kubenswrapper[5052]: I1203 12:30:17.084117 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a34cefb9fab15586d1ef541f04657dfba9f32d8dac3286264dba4943680b0e47" Dec 03 12:30:17 crc kubenswrapper[5052]: I1203 12:30:17.091980 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-688bd8fb5d-pjs4f" event={"ID":"a0e45e1a-ced9-4958-9a75-24a69280f679","Type":"ContainerDied","Data":"54d422bce70ace0e85e1e00918a25a268dd0d21415c0068651ee5074885c6030"} Dec 03 12:30:17 crc kubenswrapper[5052]: I1203 12:30:17.092028 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="54d422bce70ace0e85e1e00918a25a268dd0d21415c0068651ee5074885c6030" Dec 03 12:30:17 crc kubenswrapper[5052]: I1203 12:30:17.093253 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-76bcd49fb6-54vvs" event={"ID":"6bfd5f51-8b37-4fea-8134-a2cdcd4f6049","Type":"ContainerStarted","Data":"1a42dc3b980f218afed77ce3e44ebc5314da8b49e81ad613c50769fa0114595f"} Dec 03 12:30:17 crc kubenswrapper[5052]: I1203 12:30:17.112346 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-769cb78446-m6hzp" event={"ID":"760aa987-6355-40cb-97c0-16c7eeddd135","Type":"ContainerStarted","Data":"1cdac21bc509913963cca59d86d5d3a547a3bc1e21fc0a5a1c2c9f6e47211913"} Dec 03 12:30:17 crc kubenswrapper[5052]: I1203 12:30:17.134588 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"e8f63775-6260-4abb-abd7-45a412bf0ca5","Type":"ContainerStarted","Data":"b4461aa30a7576cad9c21f4e4005920d51c63b774d33068ff773dbfc856ca202"} Dec 03 12:30:17 crc kubenswrapper[5052]: I1203 12:30:17.138842 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-external-api-0" podUID="e8f63775-6260-4abb-abd7-45a412bf0ca5" containerName="glance-log" containerID="cri-o://b4d3741161a3d95e91cfc6ceb2e9626438d026ce0c88039796b48eae93f58046" gracePeriod=30 Dec 03 12:30:17 crc kubenswrapper[5052]: I1203 12:30:17.139977 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-external-api-0" podUID="e8f63775-6260-4abb-abd7-45a412bf0ca5" containerName="glance-httpd" containerID="cri-o://b4461aa30a7576cad9c21f4e4005920d51c63b774d33068ff773dbfc856ca202" gracePeriod=30 Dec 03 12:30:17 crc kubenswrapper[5052]: I1203 12:30:17.147938 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-api-0" podUID="4314928d-a43f-4b38-9c72-63977e59345b" containerName="nova-api-log" containerID="cri-o://50954953a47c0484d7617bce920a6b97d031a9d9d0a476b236d1e1bdff5f0cfc" gracePeriod=30 Dec 03 12:30:17 crc kubenswrapper[5052]: I1203 12:30:17.148053 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"bb1723c7-4e41-4b0e-b6b8-f8024dafdaee","Type":"ContainerStarted","Data":"94df1fd3c0977441ba1cf70763e96ac140d6a232f2c0d6cfe09fb6aa6ae75dc3"} Dec 03 12:30:17 crc kubenswrapper[5052]: I1203 12:30:17.148108 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-5c6478db8b-bt2rw" Dec 03 12:30:17 crc kubenswrapper[5052]: I1203 12:30:17.148640 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-7dffdbf8-hzmbm" Dec 03 12:30:17 crc kubenswrapper[5052]: I1203 12:30:17.148891 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-api-0" podUID="4314928d-a43f-4b38-9c72-63977e59345b" containerName="nova-api-api" containerID="cri-o://55d873486c41fcc800af3aac0c5bdbf6786c715209a5758840bdce2e1c3fba02" gracePeriod=30 Dec 03 12:30:17 crc kubenswrapper[5052]: I1203 12:30:17.196234 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/glance-default-external-api-0" podStartSLOduration=10.19621273 podStartE2EDuration="10.19621273s" podCreationTimestamp="2025-12-03 12:30:07 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:30:17.186940925 +0000 UTC m=+4207.775182689" watchObservedRunningTime="2025-12-03 12:30:17.19621273 +0000 UTC m=+4207.784454504" Dec 03 12:30:17 crc kubenswrapper[5052]: I1203 12:30:17.332898 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/53fe6403-27db-44b7-a9a6-787afbc3bbb8-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "53fe6403-27db-44b7-a9a6-787afbc3bbb8" (UID: "53fe6403-27db-44b7-a9a6-787afbc3bbb8"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:17 crc kubenswrapper[5052]: I1203 12:30:17.403902 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/53fe6403-27db-44b7-a9a6-787afbc3bbb8-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:17 crc kubenswrapper[5052]: I1203 12:30:17.428346 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/98f722f9-fa25-40b6-84ba-96127d07293c-metrics-certs-tls-certs" (OuterVolumeSpecName: "metrics-certs-tls-certs") pod "98f722f9-fa25-40b6-84ba-96127d07293c" (UID: "98f722f9-fa25-40b6-84ba-96127d07293c"). InnerVolumeSpecName "metrics-certs-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:17 crc kubenswrapper[5052]: E1203 12:30:17.447500 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of 003ad3a32b6c302f684066c22cf6e0ad91b95336410c2156d1794d7ec8417301 is running failed: container process not found" containerID="003ad3a32b6c302f684066c22cf6e0ad91b95336410c2156d1794d7ec8417301" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:30:17 crc kubenswrapper[5052]: E1203 12:30:17.447701 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of f709bafefdc660d91c439ef2df4b5fa8dafa8380b2dee355f1b04956932b91e7 is running failed: container process not found" containerID="f709bafefdc660d91c439ef2df4b5fa8dafa8380b2dee355f1b04956932b91e7" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:30:17 crc kubenswrapper[5052]: E1203 12:30:17.448321 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of 003ad3a32b6c302f684066c22cf6e0ad91b95336410c2156d1794d7ec8417301 is running failed: container process not found" containerID="003ad3a32b6c302f684066c22cf6e0ad91b95336410c2156d1794d7ec8417301" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:30:17 crc kubenswrapper[5052]: E1203 12:30:17.448458 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of f709bafefdc660d91c439ef2df4b5fa8dafa8380b2dee355f1b04956932b91e7 is running failed: container process not found" containerID="f709bafefdc660d91c439ef2df4b5fa8dafa8380b2dee355f1b04956932b91e7" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:30:17 crc kubenswrapper[5052]: E1203 12:30:17.477068 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of f709bafefdc660d91c439ef2df4b5fa8dafa8380b2dee355f1b04956932b91e7 is running failed: container process not found" containerID="f709bafefdc660d91c439ef2df4b5fa8dafa8380b2dee355f1b04956932b91e7" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:30:17 crc kubenswrapper[5052]: E1203 12:30:17.477135 5052 prober.go:104] "Probe errored" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of f709bafefdc660d91c439ef2df4b5fa8dafa8380b2dee355f1b04956932b91e7 is running failed: container process not found" probeType="Liveness" pod="openstack-kuttl-tests/nova-cell1-conductor-0" podUID="2dfbec8a-3a04-407b-8145-e12a3afee2e0" containerName="nova-cell1-conductor-conductor" Dec 03 12:30:17 crc kubenswrapper[5052]: E1203 12:30:17.478131 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of 003ad3a32b6c302f684066c22cf6e0ad91b95336410c2156d1794d7ec8417301 is running failed: container process not found" containerID="003ad3a32b6c302f684066c22cf6e0ad91b95336410c2156d1794d7ec8417301" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:30:17 crc kubenswrapper[5052]: E1203 12:30:17.478161 5052 prober.go:104] "Probe errored" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of 003ad3a32b6c302f684066c22cf6e0ad91b95336410c2156d1794d7ec8417301 is running failed: container process not found" probeType="Liveness" pod="openstack-kuttl-tests/nova-cell0-conductor-0" podUID="e2718906-e52d-42fa-ad42-35626e99d461" containerName="nova-cell0-conductor-conductor" Dec 03 12:30:17 crc kubenswrapper[5052]: I1203 12:30:17.507240 5052 reconciler_common.go:293] "Volume detached for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/98f722f9-fa25-40b6-84ba-96127d07293c-metrics-certs-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:17 crc kubenswrapper[5052]: I1203 12:30:17.517405 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/53fe6403-27db-44b7-a9a6-787afbc3bbb8-config-data" (OuterVolumeSpecName: "config-data") pod "53fe6403-27db-44b7-a9a6-787afbc3bbb8" (UID: "53fe6403-27db-44b7-a9a6-787afbc3bbb8"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:17 crc kubenswrapper[5052]: I1203 12:30:17.519720 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8bddf2b1-8116-4b6e-9cd8-be8fe1cd4b64-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "8bddf2b1-8116-4b6e-9cd8-be8fe1cd4b64" (UID: "8bddf2b1-8116-4b6e-9cd8-be8fe1cd4b64"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:17 crc kubenswrapper[5052]: I1203 12:30:17.565631 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/53fe6403-27db-44b7-a9a6-787afbc3bbb8-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "53fe6403-27db-44b7-a9a6-787afbc3bbb8" (UID: "53fe6403-27db-44b7-a9a6-787afbc3bbb8"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:17 crc kubenswrapper[5052]: I1203 12:30:17.565713 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8bddf2b1-8116-4b6e-9cd8-be8fe1cd4b64-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "8bddf2b1-8116-4b6e-9cd8-be8fe1cd4b64" (UID: "8bddf2b1-8116-4b6e-9cd8-be8fe1cd4b64"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:17 crc kubenswrapper[5052]: I1203 12:30:17.565761 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/53fe6403-27db-44b7-a9a6-787afbc3bbb8-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "53fe6403-27db-44b7-a9a6-787afbc3bbb8" (UID: "53fe6403-27db-44b7-a9a6-787afbc3bbb8"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:17 crc kubenswrapper[5052]: I1203 12:30:17.608842 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/53fe6403-27db-44b7-a9a6-787afbc3bbb8-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:17 crc kubenswrapper[5052]: I1203 12:30:17.608873 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/8bddf2b1-8116-4b6e-9cd8-be8fe1cd4b64-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:17 crc kubenswrapper[5052]: I1203 12:30:17.608884 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/8bddf2b1-8116-4b6e-9cd8-be8fe1cd4b64-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:17 crc kubenswrapper[5052]: I1203 12:30:17.608895 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/53fe6403-27db-44b7-a9a6-787afbc3bbb8-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:17 crc kubenswrapper[5052]: I1203 12:30:17.608902 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/53fe6403-27db-44b7-a9a6-787afbc3bbb8-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:17 crc kubenswrapper[5052]: I1203 12:30:17.645896 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-688bd8fb5d-pjs4f" Dec 03 12:30:17 crc kubenswrapper[5052]: I1203 12:30:17.661834 5052 scope.go:117] "RemoveContainer" containerID="33f8dbc2efb3ac8c4348e12af03e0b1919430c85f4a7565b4e5fc747718afe50" Dec 03 12:30:17 crc kubenswrapper[5052]: I1203 12:30:17.700620 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1d48b723-3f75-4d1d-83d5-8315ea5464c6" path="/var/lib/kubelet/pods/1d48b723-3f75-4d1d-83d5-8315ea5464c6/volumes" Dec 03 12:30:17 crc kubenswrapper[5052]: I1203 12:30:17.753155 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/openstack-cell1-galera-0"] Dec 03 12:30:17 crc kubenswrapper[5052]: I1203 12:30:17.792607 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/openstack-cell1-galera-0"] Dec 03 12:30:17 crc kubenswrapper[5052]: I1203 12:30:17.818818 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4tb7p\" (UniqueName: \"kubernetes.io/projected/a0e45e1a-ced9-4958-9a75-24a69280f679-kube-api-access-4tb7p\") pod \"a0e45e1a-ced9-4958-9a75-24a69280f679\" (UID: \"a0e45e1a-ced9-4958-9a75-24a69280f679\") " Dec 03 12:30:17 crc kubenswrapper[5052]: I1203 12:30:17.818878 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/a0e45e1a-ced9-4958-9a75-24a69280f679-public-tls-certs\") pod \"a0e45e1a-ced9-4958-9a75-24a69280f679\" (UID: \"a0e45e1a-ced9-4958-9a75-24a69280f679\") " Dec 03 12:30:17 crc kubenswrapper[5052]: I1203 12:30:17.818927 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/a0e45e1a-ced9-4958-9a75-24a69280f679-logs\") pod \"a0e45e1a-ced9-4958-9a75-24a69280f679\" (UID: \"a0e45e1a-ced9-4958-9a75-24a69280f679\") " Dec 03 12:30:17 crc kubenswrapper[5052]: I1203 12:30:17.818953 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/a0e45e1a-ced9-4958-9a75-24a69280f679-internal-tls-certs\") pod \"a0e45e1a-ced9-4958-9a75-24a69280f679\" (UID: \"a0e45e1a-ced9-4958-9a75-24a69280f679\") " Dec 03 12:30:17 crc kubenswrapper[5052]: I1203 12:30:17.819006 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a0e45e1a-ced9-4958-9a75-24a69280f679-config-data\") pod \"a0e45e1a-ced9-4958-9a75-24a69280f679\" (UID: \"a0e45e1a-ced9-4958-9a75-24a69280f679\") " Dec 03 12:30:17 crc kubenswrapper[5052]: I1203 12:30:17.819148 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/a0e45e1a-ced9-4958-9a75-24a69280f679-scripts\") pod \"a0e45e1a-ced9-4958-9a75-24a69280f679\" (UID: \"a0e45e1a-ced9-4958-9a75-24a69280f679\") " Dec 03 12:30:17 crc kubenswrapper[5052]: I1203 12:30:17.819204 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a0e45e1a-ced9-4958-9a75-24a69280f679-combined-ca-bundle\") pod \"a0e45e1a-ced9-4958-9a75-24a69280f679\" (UID: \"a0e45e1a-ced9-4958-9a75-24a69280f679\") " Dec 03 12:30:17 crc kubenswrapper[5052]: I1203 12:30:17.822218 5052 scope.go:117] "RemoveContainer" containerID="c30cd5ee45f2cebbe8afbf744e6817d788d7a3b824716ed969e21464a285f555" Dec 03 12:30:17 crc kubenswrapper[5052]: I1203 12:30:17.835866 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a0e45e1a-ced9-4958-9a75-24a69280f679-logs" (OuterVolumeSpecName: "logs") pod "a0e45e1a-ced9-4958-9a75-24a69280f679" (UID: "a0e45e1a-ced9-4958-9a75-24a69280f679"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:30:17 crc kubenswrapper[5052]: I1203 12:30:17.836172 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a0e45e1a-ced9-4958-9a75-24a69280f679-kube-api-access-4tb7p" (OuterVolumeSpecName: "kube-api-access-4tb7p") pod "a0e45e1a-ced9-4958-9a75-24a69280f679" (UID: "a0e45e1a-ced9-4958-9a75-24a69280f679"). InnerVolumeSpecName "kube-api-access-4tb7p". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:30:17 crc kubenswrapper[5052]: I1203 12:30:17.855750 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a0e45e1a-ced9-4958-9a75-24a69280f679-scripts" (OuterVolumeSpecName: "scripts") pod "a0e45e1a-ced9-4958-9a75-24a69280f679" (UID: "a0e45e1a-ced9-4958-9a75-24a69280f679"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:17 crc kubenswrapper[5052]: I1203 12:30:17.889353 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/openstack-cell1-galera-0"] Dec 03 12:30:17 crc kubenswrapper[5052]: E1203 12:30:17.890888 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="a0e45e1a-ced9-4958-9a75-24a69280f679" containerName="placement-log" Dec 03 12:30:17 crc kubenswrapper[5052]: I1203 12:30:17.890912 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="a0e45e1a-ced9-4958-9a75-24a69280f679" containerName="placement-log" Dec 03 12:30:17 crc kubenswrapper[5052]: E1203 12:30:17.890953 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="a0e45e1a-ced9-4958-9a75-24a69280f679" containerName="placement-api" Dec 03 12:30:17 crc kubenswrapper[5052]: I1203 12:30:17.890959 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="a0e45e1a-ced9-4958-9a75-24a69280f679" containerName="placement-api" Dec 03 12:30:17 crc kubenswrapper[5052]: I1203 12:30:17.891269 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="a0e45e1a-ced9-4958-9a75-24a69280f679" containerName="placement-api" Dec 03 12:30:17 crc kubenswrapper[5052]: I1203 12:30:17.891308 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="a0e45e1a-ced9-4958-9a75-24a69280f679" containerName="placement-log" Dec 03 12:30:17 crc kubenswrapper[5052]: I1203 12:30:17.895857 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:30:17 crc kubenswrapper[5052]: I1203 12:30:17.901714 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-galera-openstack-cell1-svc" Dec 03 12:30:17 crc kubenswrapper[5052]: I1203 12:30:17.901994 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"openstack-cell1-scripts" Dec 03 12:30:17 crc kubenswrapper[5052]: I1203 12:30:17.902115 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"openstack-cell1-config-data" Dec 03 12:30:17 crc kubenswrapper[5052]: I1203 12:30:17.902280 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"galera-openstack-cell1-dockercfg-8pg58" Dec 03 12:30:17 crc kubenswrapper[5052]: I1203 12:30:17.920721 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/openstack-cell1-galera-0"] Dec 03 12:30:17 crc kubenswrapper[5052]: I1203 12:30:17.930243 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/aa55d0c8-56de-4edd-92b3-9252f169fed8-config-data-default\") pod \"openstack-cell1-galera-0\" (UID: \"aa55d0c8-56de-4edd-92b3-9252f169fed8\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:30:17 crc kubenswrapper[5052]: I1203 12:30:17.930353 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/aa55d0c8-56de-4edd-92b3-9252f169fed8-galera-tls-certs\") pod \"openstack-cell1-galera-0\" (UID: \"aa55d0c8-56de-4edd-92b3-9252f169fed8\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:30:17 crc kubenswrapper[5052]: I1203 12:30:17.930413 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/aa55d0c8-56de-4edd-92b3-9252f169fed8-kolla-config\") pod \"openstack-cell1-galera-0\" (UID: \"aa55d0c8-56de-4edd-92b3-9252f169fed8\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:30:17 crc kubenswrapper[5052]: I1203 12:30:17.930445 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/aa55d0c8-56de-4edd-92b3-9252f169fed8-combined-ca-bundle\") pod \"openstack-cell1-galera-0\" (UID: \"aa55d0c8-56de-4edd-92b3-9252f169fed8\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:30:17 crc kubenswrapper[5052]: I1203 12:30:17.930470 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/aa55d0c8-56de-4edd-92b3-9252f169fed8-config-data-generated\") pod \"openstack-cell1-galera-0\" (UID: \"aa55d0c8-56de-4edd-92b3-9252f169fed8\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:30:17 crc kubenswrapper[5052]: I1203 12:30:17.930534 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-94jwv\" (UniqueName: \"kubernetes.io/projected/aa55d0c8-56de-4edd-92b3-9252f169fed8-kube-api-access-94jwv\") pod \"openstack-cell1-galera-0\" (UID: \"aa55d0c8-56de-4edd-92b3-9252f169fed8\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:30:17 crc kubenswrapper[5052]: I1203 12:30:17.930602 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage05-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage05-crc\") pod \"openstack-cell1-galera-0\" (UID: \"aa55d0c8-56de-4edd-92b3-9252f169fed8\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:30:17 crc kubenswrapper[5052]: I1203 12:30:17.930634 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/aa55d0c8-56de-4edd-92b3-9252f169fed8-operator-scripts\") pod \"openstack-cell1-galera-0\" (UID: \"aa55d0c8-56de-4edd-92b3-9252f169fed8\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:30:17 crc kubenswrapper[5052]: I1203 12:30:17.930699 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/a0e45e1a-ced9-4958-9a75-24a69280f679-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:17 crc kubenswrapper[5052]: I1203 12:30:17.930714 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-4tb7p\" (UniqueName: \"kubernetes.io/projected/a0e45e1a-ced9-4958-9a75-24a69280f679-kube-api-access-4tb7p\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:17 crc kubenswrapper[5052]: I1203 12:30:17.930727 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/a0e45e1a-ced9-4958-9a75-24a69280f679-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:17 crc kubenswrapper[5052]: I1203 12:30:17.937061 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:17.949451 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a0e45e1a-ced9-4958-9a75-24a69280f679-config-data" (OuterVolumeSpecName: "config-data") pod "a0e45e1a-ced9-4958-9a75-24a69280f679" (UID: "a0e45e1a-ced9-4958-9a75-24a69280f679"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:18.003712 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a0e45e1a-ced9-4958-9a75-24a69280f679-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "a0e45e1a-ced9-4958-9a75-24a69280f679" (UID: "a0e45e1a-ced9-4958-9a75-24a69280f679"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:18.038038 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/50f5af82-8ee5-45f4-b158-2166f1be6cb8-ovsdb-rundir\") pod \"50f5af82-8ee5-45f4-b158-2166f1be6cb8\" (UID: \"50f5af82-8ee5-45f4-b158-2166f1be6cb8\") " Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:18.038094 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovsdbserver-sb-tls-certs\" (UniqueName: \"kubernetes.io/secret/50f5af82-8ee5-45f4-b158-2166f1be6cb8-ovsdbserver-sb-tls-certs\") pod \"50f5af82-8ee5-45f4-b158-2166f1be6cb8\" (UID: \"50f5af82-8ee5-45f4-b158-2166f1be6cb8\") " Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:18.038184 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/50f5af82-8ee5-45f4-b158-2166f1be6cb8-metrics-certs-tls-certs\") pod \"50f5af82-8ee5-45f4-b158-2166f1be6cb8\" (UID: \"50f5af82-8ee5-45f4-b158-2166f1be6cb8\") " Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:18.038285 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dp9m4\" (UniqueName: \"kubernetes.io/projected/50f5af82-8ee5-45f4-b158-2166f1be6cb8-kube-api-access-dp9m4\") pod \"50f5af82-8ee5-45f4-b158-2166f1be6cb8\" (UID: \"50f5af82-8ee5-45f4-b158-2166f1be6cb8\") " Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:18.038312 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovndbcluster-sb-etc-ovn\" (UniqueName: \"kubernetes.io/local-volume/local-storage08-crc\") pod \"50f5af82-8ee5-45f4-b158-2166f1be6cb8\" (UID: \"50f5af82-8ee5-45f4-b158-2166f1be6cb8\") " Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:18.038419 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/50f5af82-8ee5-45f4-b158-2166f1be6cb8-combined-ca-bundle\") pod \"50f5af82-8ee5-45f4-b158-2166f1be6cb8\" (UID: \"50f5af82-8ee5-45f4-b158-2166f1be6cb8\") " Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:18.038537 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/50f5af82-8ee5-45f4-b158-2166f1be6cb8-config\") pod \"50f5af82-8ee5-45f4-b158-2166f1be6cb8\" (UID: \"50f5af82-8ee5-45f4-b158-2166f1be6cb8\") " Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:18.038567 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/50f5af82-8ee5-45f4-b158-2166f1be6cb8-scripts\") pod \"50f5af82-8ee5-45f4-b158-2166f1be6cb8\" (UID: \"50f5af82-8ee5-45f4-b158-2166f1be6cb8\") " Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:18.038802 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/aa55d0c8-56de-4edd-92b3-9252f169fed8-kolla-config\") pod \"openstack-cell1-galera-0\" (UID: \"aa55d0c8-56de-4edd-92b3-9252f169fed8\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:18.038833 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/aa55d0c8-56de-4edd-92b3-9252f169fed8-combined-ca-bundle\") pod \"openstack-cell1-galera-0\" (UID: \"aa55d0c8-56de-4edd-92b3-9252f169fed8\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:18.038855 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/aa55d0c8-56de-4edd-92b3-9252f169fed8-config-data-generated\") pod \"openstack-cell1-galera-0\" (UID: \"aa55d0c8-56de-4edd-92b3-9252f169fed8\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:18.038909 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-94jwv\" (UniqueName: \"kubernetes.io/projected/aa55d0c8-56de-4edd-92b3-9252f169fed8-kube-api-access-94jwv\") pod \"openstack-cell1-galera-0\" (UID: \"aa55d0c8-56de-4edd-92b3-9252f169fed8\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:18.038969 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage05-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage05-crc\") pod \"openstack-cell1-galera-0\" (UID: \"aa55d0c8-56de-4edd-92b3-9252f169fed8\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:18.039035 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/aa55d0c8-56de-4edd-92b3-9252f169fed8-operator-scripts\") pod \"openstack-cell1-galera-0\" (UID: \"aa55d0c8-56de-4edd-92b3-9252f169fed8\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:18.039078 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/aa55d0c8-56de-4edd-92b3-9252f169fed8-config-data-default\") pod \"openstack-cell1-galera-0\" (UID: \"aa55d0c8-56de-4edd-92b3-9252f169fed8\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:18.039130 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/aa55d0c8-56de-4edd-92b3-9252f169fed8-galera-tls-certs\") pod \"openstack-cell1-galera-0\" (UID: \"aa55d0c8-56de-4edd-92b3-9252f169fed8\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:18.039196 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a0e45e1a-ced9-4958-9a75-24a69280f679-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:18.039208 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a0e45e1a-ced9-4958-9a75-24a69280f679-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:18.040130 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/50f5af82-8ee5-45f4-b158-2166f1be6cb8-scripts" (OuterVolumeSpecName: "scripts") pod "50f5af82-8ee5-45f4-b158-2166f1be6cb8" (UID: "50f5af82-8ee5-45f4-b158-2166f1be6cb8"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:18.040369 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/50f5af82-8ee5-45f4-b158-2166f1be6cb8-ovsdb-rundir" (OuterVolumeSpecName: "ovsdb-rundir") pod "50f5af82-8ee5-45f4-b158-2166f1be6cb8" (UID: "50f5af82-8ee5-45f4-b158-2166f1be6cb8"). InnerVolumeSpecName "ovsdb-rundir". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:18.049098 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/aa55d0c8-56de-4edd-92b3-9252f169fed8-galera-tls-certs\") pod \"openstack-cell1-galera-0\" (UID: \"aa55d0c8-56de-4edd-92b3-9252f169fed8\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:18.049379 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage05-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage05-crc\") pod \"openstack-cell1-galera-0\" (UID: \"aa55d0c8-56de-4edd-92b3-9252f169fed8\") device mount path \"/mnt/openstack/pv05\"" pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:18.049962 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/50f5af82-8ee5-45f4-b158-2166f1be6cb8-config" (OuterVolumeSpecName: "config") pod "50f5af82-8ee5-45f4-b158-2166f1be6cb8" (UID: "50f5af82-8ee5-45f4-b158-2166f1be6cb8"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:18.050032 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/barbican-api-ff6b8c44b-7ndgm" podUID="48831cad-910f-46f1-aefe-7cc4cbfa984f" containerName="barbican-api" probeResult="failure" output="Get \"https://10.217.0.204:9311/healthcheck\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:18.050970 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/50f5af82-8ee5-45f4-b158-2166f1be6cb8-kube-api-access-dp9m4" (OuterVolumeSpecName: "kube-api-access-dp9m4") pod "50f5af82-8ee5-45f4-b158-2166f1be6cb8" (UID: "50f5af82-8ee5-45f4-b158-2166f1be6cb8"). InnerVolumeSpecName "kube-api-access-dp9m4". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:18.063373 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/aa55d0c8-56de-4edd-92b3-9252f169fed8-kolla-config\") pod \"openstack-cell1-galera-0\" (UID: \"aa55d0c8-56de-4edd-92b3-9252f169fed8\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:18.065141 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/aa55d0c8-56de-4edd-92b3-9252f169fed8-config-data-generated\") pod \"openstack-cell1-galera-0\" (UID: \"aa55d0c8-56de-4edd-92b3-9252f169fed8\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:18.065211 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage08-crc" (OuterVolumeSpecName: "ovndbcluster-sb-etc-ovn") pod "50f5af82-8ee5-45f4-b158-2166f1be6cb8" (UID: "50f5af82-8ee5-45f4-b158-2166f1be6cb8"). InnerVolumeSpecName "local-storage08-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:18.065687 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/aa55d0c8-56de-4edd-92b3-9252f169fed8-config-data-default\") pod \"openstack-cell1-galera-0\" (UID: \"aa55d0c8-56de-4edd-92b3-9252f169fed8\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:18.066304 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/aa55d0c8-56de-4edd-92b3-9252f169fed8-operator-scripts\") pod \"openstack-cell1-galera-0\" (UID: \"aa55d0c8-56de-4edd-92b3-9252f169fed8\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:18.067805 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/barbican-api-ff6b8c44b-7ndgm" podUID="48831cad-910f-46f1-aefe-7cc4cbfa984f" containerName="barbican-api-log" probeResult="failure" output="Get \"https://10.217.0.204:9311/healthcheck\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:18.080231 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/aa55d0c8-56de-4edd-92b3-9252f169fed8-combined-ca-bundle\") pod \"openstack-cell1-galera-0\" (UID: \"aa55d0c8-56de-4edd-92b3-9252f169fed8\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:18.093615 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-94jwv\" (UniqueName: \"kubernetes.io/projected/aa55d0c8-56de-4edd-92b3-9252f169fed8-kube-api-access-94jwv\") pod \"openstack-cell1-galera-0\" (UID: \"aa55d0c8-56de-4edd-92b3-9252f169fed8\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:18.133831 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a0e45e1a-ced9-4958-9a75-24a69280f679-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "a0e45e1a-ced9-4958-9a75-24a69280f679" (UID: "a0e45e1a-ced9-4958-9a75-24a69280f679"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:18.134595 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage05-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage05-crc\") pod \"openstack-cell1-galera-0\" (UID: \"aa55d0c8-56de-4edd-92b3-9252f169fed8\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:18.142480 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/50f5af82-8ee5-45f4-b158-2166f1be6cb8-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:18.142611 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/50f5af82-8ee5-45f4-b158-2166f1be6cb8-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:18.142651 5052 reconciler_common.go:293] "Volume detached for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/50f5af82-8ee5-45f4-b158-2166f1be6cb8-ovsdb-rundir\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:18.142679 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-dp9m4\" (UniqueName: \"kubernetes.io/projected/50f5af82-8ee5-45f4-b158-2166f1be6cb8-kube-api-access-dp9m4\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:18.142708 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage08-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage08-crc\") on node \"crc\" " Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:18.142721 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/a0e45e1a-ced9-4958-9a75-24a69280f679-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:18.148562 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/50f5af82-8ee5-45f4-b158-2166f1be6cb8-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "50f5af82-8ee5-45f4-b158-2166f1be6cb8" (UID: "50f5af82-8ee5-45f4-b158-2166f1be6cb8"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:18.163953 5052 scope.go:117] "RemoveContainer" containerID="b6e565e6b6e8e07c8000bbead3eed4e1cd45529d9dc3aa7fcb642cf668f780aa" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:18.165797 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:18.165995 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" podUID="f4e095cd-36d8-4136-b262-f6d52717a05c" containerName="nova-cell1-novncproxy-novncproxy" containerID="cri-o://1e7025513d6b144fd4de246b9cff9dd24d587dcce1c3318a8c75d6b9165b00ff" gracePeriod=30 Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:18.226537 5052 generic.go:334] "Generic (PLEG): container finished" podID="e8f63775-6260-4abb-abd7-45a412bf0ca5" containerID="b4461aa30a7576cad9c21f4e4005920d51c63b774d33068ff773dbfc856ca202" exitCode=0 Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:18.226566 5052 generic.go:334] "Generic (PLEG): container finished" podID="e8f63775-6260-4abb-abd7-45a412bf0ca5" containerID="b4d3741161a3d95e91cfc6ceb2e9626438d026ce0c88039796b48eae93f58046" exitCode=143 Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:18.226637 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"e8f63775-6260-4abb-abd7-45a412bf0ca5","Type":"ContainerDied","Data":"b4461aa30a7576cad9c21f4e4005920d51c63b774d33068ff773dbfc856ca202"} Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:18.226706 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"e8f63775-6260-4abb-abd7-45a412bf0ca5","Type":"ContainerDied","Data":"b4d3741161a3d95e91cfc6ceb2e9626438d026ce0c88039796b48eae93f58046"} Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:18.259384 5052 generic.go:334] "Generic (PLEG): container finished" podID="e2718906-e52d-42fa-ad42-35626e99d461" containerID="003ad3a32b6c302f684066c22cf6e0ad91b95336410c2156d1794d7ec8417301" exitCode=1 Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:18.259484 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-conductor-0" event={"ID":"e2718906-e52d-42fa-ad42-35626e99d461","Type":"ContainerDied","Data":"003ad3a32b6c302f684066c22cf6e0ad91b95336410c2156d1794d7ec8417301"} Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:18.266013 5052 scope.go:117] "RemoveContainer" containerID="003ad3a32b6c302f684066c22cf6e0ad91b95336410c2156d1794d7ec8417301" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:18.267605 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-nb-0"] Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:18.271034 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/50f5af82-8ee5-45f4-b158-2166f1be6cb8-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:18.279796 5052 scope.go:117] "RemoveContainer" containerID="d3907f3c074c217040963cd0054e9403720f2049ff0f6f8d3ee8174b0d8a46a0" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:18.287429 5052 generic.go:334] "Generic (PLEG): container finished" podID="4314928d-a43f-4b38-9c72-63977e59345b" containerID="50954953a47c0484d7617bce920a6b97d031a9d9d0a476b236d1e1bdff5f0cfc" exitCode=143 Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:18.287508 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"4314928d-a43f-4b38-9c72-63977e59345b","Type":"ContainerDied","Data":"50954953a47c0484d7617bce920a6b97d031a9d9d0a476b236d1e1bdff5f0cfc"} Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:18.294321 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-nb-0"] Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:18.329105 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage08-crc" (UniqueName: "kubernetes.io/local-volume/local-storage08-crc") on node "crc" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:18.329158 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ovsdbserver-nb-0"] Dec 03 12:30:27 crc kubenswrapper[5052]: E1203 12:30:18.335747 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="50f5af82-8ee5-45f4-b158-2166f1be6cb8" containerName="openstack-network-exporter" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:18.335777 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="50f5af82-8ee5-45f4-b158-2166f1be6cb8" containerName="openstack-network-exporter" Dec 03 12:30:27 crc kubenswrapper[5052]: E1203 12:30:18.335791 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="50f5af82-8ee5-45f4-b158-2166f1be6cb8" containerName="ovsdbserver-sb" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:18.335798 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="50f5af82-8ee5-45f4-b158-2166f1be6cb8" containerName="ovsdbserver-sb" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:18.336134 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="50f5af82-8ee5-45f4-b158-2166f1be6cb8" containerName="ovsdbserver-sb" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:18.336148 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="50f5af82-8ee5-45f4-b158-2166f1be6cb8" containerName="openstack-network-exporter" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:18.337237 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-nb-0"] Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:18.337319 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:18.351248 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-ovndbcluster-nb-ovndbs" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:18.351679 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-76bcd49fb6-54vvs" event={"ID":"6bfd5f51-8b37-4fea-8134-a2cdcd4f6049","Type":"ContainerStarted","Data":"01711fff696becc6dfb1b8494228bffbaa942ad3546265fb3e7434412310303c"} Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:18.351766 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"ovndbcluster-nb-config" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:18.351867 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"ovndbcluster-nb-scripts" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:18.352795 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ovncluster-ovndbcluster-nb-dockercfg-4462b" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:18.362730 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a0e45e1a-ced9-4958-9a75-24a69280f679-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "a0e45e1a-ced9-4958-9a75-24a69280f679" (UID: "a0e45e1a-ced9-4958-9a75-24a69280f679"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:18.366685 5052 generic.go:334] "Generic (PLEG): container finished" podID="2dfbec8a-3a04-407b-8145-e12a3afee2e0" containerID="f709bafefdc660d91c439ef2df4b5fa8dafa8380b2dee355f1b04956932b91e7" exitCode=1 Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:18.366788 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:18.366779 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-conductor-0" event={"ID":"2dfbec8a-3a04-407b-8145-e12a3afee2e0","Type":"ContainerDied","Data":"f709bafefdc660d91c439ef2df4b5fa8dafa8380b2dee355f1b04956932b91e7"} Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:18.366880 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-688bd8fb5d-pjs4f" Dec 03 12:30:27 crc kubenswrapper[5052]: E1203 12:30:18.369357 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of f709bafefdc660d91c439ef2df4b5fa8dafa8380b2dee355f1b04956932b91e7 is running failed: container process not found" containerID="f709bafefdc660d91c439ef2df4b5fa8dafa8380b2dee355f1b04956932b91e7" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:18.372619 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/dccae7c1-009e-4b3b-b66a-ecbf6e53489a-combined-ca-bundle\") pod \"ovsdbserver-nb-0\" (UID: \"dccae7c1-009e-4b3b-b66a-ecbf6e53489a\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:18.372695 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage03-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage03-crc\") pod \"ovsdbserver-nb-0\" (UID: \"dccae7c1-009e-4b3b-b66a-ecbf6e53489a\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:30:27 crc kubenswrapper[5052]: E1203 12:30:18.372766 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of f709bafefdc660d91c439ef2df4b5fa8dafa8380b2dee355f1b04956932b91e7 is running failed: container process not found" containerID="f709bafefdc660d91c439ef2df4b5fa8dafa8380b2dee355f1b04956932b91e7" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:18.372788 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/dccae7c1-009e-4b3b-b66a-ecbf6e53489a-ovsdb-rundir\") pod \"ovsdbserver-nb-0\" (UID: \"dccae7c1-009e-4b3b-b66a-ecbf6e53489a\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:18.372828 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/dccae7c1-009e-4b3b-b66a-ecbf6e53489a-scripts\") pod \"ovsdbserver-nb-0\" (UID: \"dccae7c1-009e-4b3b-b66a-ecbf6e53489a\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:18.372854 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovsdbserver-nb-tls-certs\" (UniqueName: \"kubernetes.io/secret/dccae7c1-009e-4b3b-b66a-ecbf6e53489a-ovsdbserver-nb-tls-certs\") pod \"ovsdbserver-nb-0\" (UID: \"dccae7c1-009e-4b3b-b66a-ecbf6e53489a\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:18.372872 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/dccae7c1-009e-4b3b-b66a-ecbf6e53489a-metrics-certs-tls-certs\") pod \"ovsdbserver-nb-0\" (UID: \"dccae7c1-009e-4b3b-b66a-ecbf6e53489a\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:18.372890 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/dccae7c1-009e-4b3b-b66a-ecbf6e53489a-config\") pod \"ovsdbserver-nb-0\" (UID: \"dccae7c1-009e-4b3b-b66a-ecbf6e53489a\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:18.372939 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b9v9t\" (UniqueName: \"kubernetes.io/projected/dccae7c1-009e-4b3b-b66a-ecbf6e53489a-kube-api-access-b9v9t\") pod \"ovsdbserver-nb-0\" (UID: \"dccae7c1-009e-4b3b-b66a-ecbf6e53489a\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:18.372995 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage08-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage08-crc\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:18.373006 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/a0e45e1a-ced9-4958-9a75-24a69280f679-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:27 crc kubenswrapper[5052]: E1203 12:30:18.377801 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of f709bafefdc660d91c439ef2df4b5fa8dafa8380b2dee355f1b04956932b91e7 is running failed: container process not found" containerID="f709bafefdc660d91c439ef2df4b5fa8dafa8380b2dee355f1b04956932b91e7" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:30:27 crc kubenswrapper[5052]: E1203 12:30:18.377842 5052 prober.go:104] "Probe errored" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of f709bafefdc660d91c439ef2df4b5fa8dafa8380b2dee355f1b04956932b91e7 is running failed: container process not found" probeType="Readiness" pod="openstack-kuttl-tests/nova-cell1-conductor-0" podUID="2dfbec8a-3a04-407b-8145-e12a3afee2e0" containerName="nova-cell1-conductor-conductor" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:18.386792 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/50f5af82-8ee5-45f4-b158-2166f1be6cb8-metrics-certs-tls-certs" (OuterVolumeSpecName: "metrics-certs-tls-certs") pod "50f5af82-8ee5-45f4-b158-2166f1be6cb8" (UID: "50f5af82-8ee5-45f4-b158-2166f1be6cb8"). InnerVolumeSpecName "metrics-certs-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:18.412152 5052 scope.go:117] "RemoveContainer" containerID="f709bafefdc660d91c439ef2df4b5fa8dafa8380b2dee355f1b04956932b91e7" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:18.418412 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/placement-5c6478db8b-bt2rw"] Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:18.424431 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/50f5af82-8ee5-45f4-b158-2166f1be6cb8-ovsdbserver-sb-tls-certs" (OuterVolumeSpecName: "ovsdbserver-sb-tls-certs") pod "50f5af82-8ee5-45f4-b158-2166f1be6cb8" (UID: "50f5af82-8ee5-45f4-b158-2166f1be6cb8"). InnerVolumeSpecName "ovsdbserver-sb-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:18.427974 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/placement-5c6478db8b-bt2rw"] Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:18.436618 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/placement-7dffdbf8-hzmbm"] Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:18.445853 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/placement-7dffdbf8-hzmbm"] Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:18.474029 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/dccae7c1-009e-4b3b-b66a-ecbf6e53489a-ovsdb-rundir\") pod \"ovsdbserver-nb-0\" (UID: \"dccae7c1-009e-4b3b-b66a-ecbf6e53489a\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:18.474117 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/dccae7c1-009e-4b3b-b66a-ecbf6e53489a-scripts\") pod \"ovsdbserver-nb-0\" (UID: \"dccae7c1-009e-4b3b-b66a-ecbf6e53489a\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:18.474175 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovsdbserver-nb-tls-certs\" (UniqueName: \"kubernetes.io/secret/dccae7c1-009e-4b3b-b66a-ecbf6e53489a-ovsdbserver-nb-tls-certs\") pod \"ovsdbserver-nb-0\" (UID: \"dccae7c1-009e-4b3b-b66a-ecbf6e53489a\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:18.474202 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/dccae7c1-009e-4b3b-b66a-ecbf6e53489a-metrics-certs-tls-certs\") pod \"ovsdbserver-nb-0\" (UID: \"dccae7c1-009e-4b3b-b66a-ecbf6e53489a\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:18.474228 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/dccae7c1-009e-4b3b-b66a-ecbf6e53489a-config\") pod \"ovsdbserver-nb-0\" (UID: \"dccae7c1-009e-4b3b-b66a-ecbf6e53489a\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:18.474293 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-b9v9t\" (UniqueName: \"kubernetes.io/projected/dccae7c1-009e-4b3b-b66a-ecbf6e53489a-kube-api-access-b9v9t\") pod \"ovsdbserver-nb-0\" (UID: \"dccae7c1-009e-4b3b-b66a-ecbf6e53489a\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:18.474337 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/dccae7c1-009e-4b3b-b66a-ecbf6e53489a-combined-ca-bundle\") pod \"ovsdbserver-nb-0\" (UID: \"dccae7c1-009e-4b3b-b66a-ecbf6e53489a\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:18.475219 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage03-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage03-crc\") pod \"ovsdbserver-nb-0\" (UID: \"dccae7c1-009e-4b3b-b66a-ecbf6e53489a\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:18.475343 5052 reconciler_common.go:293] "Volume detached for volume \"ovsdbserver-sb-tls-certs\" (UniqueName: \"kubernetes.io/secret/50f5af82-8ee5-45f4-b158-2166f1be6cb8-ovsdbserver-sb-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:18.475359 5052 reconciler_common.go:293] "Volume detached for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/50f5af82-8ee5-45f4-b158-2166f1be6cb8-metrics-certs-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:18.475656 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage03-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage03-crc\") pod \"ovsdbserver-nb-0\" (UID: \"dccae7c1-009e-4b3b-b66a-ecbf6e53489a\") device mount path \"/mnt/openstack/pv03\"" pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:18.478276 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/dccae7c1-009e-4b3b-b66a-ecbf6e53489a-scripts\") pod \"ovsdbserver-nb-0\" (UID: \"dccae7c1-009e-4b3b-b66a-ecbf6e53489a\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:18.478937 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/dccae7c1-009e-4b3b-b66a-ecbf6e53489a-ovsdb-rundir\") pod \"ovsdbserver-nb-0\" (UID: \"dccae7c1-009e-4b3b-b66a-ecbf6e53489a\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:18.480569 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/dccae7c1-009e-4b3b-b66a-ecbf6e53489a-config\") pod \"ovsdbserver-nb-0\" (UID: \"dccae7c1-009e-4b3b-b66a-ecbf6e53489a\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:18.483078 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/dccae7c1-009e-4b3b-b66a-ecbf6e53489a-metrics-certs-tls-certs\") pod \"ovsdbserver-nb-0\" (UID: \"dccae7c1-009e-4b3b-b66a-ecbf6e53489a\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:18.483330 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovsdbserver-nb-tls-certs\" (UniqueName: \"kubernetes.io/secret/dccae7c1-009e-4b3b-b66a-ecbf6e53489a-ovsdbserver-nb-tls-certs\") pod \"ovsdbserver-nb-0\" (UID: \"dccae7c1-009e-4b3b-b66a-ecbf6e53489a\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:18.487045 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/dccae7c1-009e-4b3b-b66a-ecbf6e53489a-combined-ca-bundle\") pod \"ovsdbserver-nb-0\" (UID: \"dccae7c1-009e-4b3b-b66a-ecbf6e53489a\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:18.503036 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-b9v9t\" (UniqueName: \"kubernetes.io/projected/dccae7c1-009e-4b3b-b66a-ecbf6e53489a-kube-api-access-b9v9t\") pod \"ovsdbserver-nb-0\" (UID: \"dccae7c1-009e-4b3b-b66a-ecbf6e53489a\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:30:27 crc kubenswrapper[5052]: E1203 12:30:18.518531 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="165a4ba65f86633615c852ba29ceb303c3bbd0dbd696084cf0c3a26afe0a4424" cmd=["/bin/bash","/var/lib/operator-scripts/mysql_probe.sh","readiness"] Dec 03 12:30:27 crc kubenswrapper[5052]: E1203 12:30:18.520037 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="165a4ba65f86633615c852ba29ceb303c3bbd0dbd696084cf0c3a26afe0a4424" cmd=["/bin/bash","/var/lib/operator-scripts/mysql_probe.sh","readiness"] Dec 03 12:30:27 crc kubenswrapper[5052]: E1203 12:30:18.523051 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="165a4ba65f86633615c852ba29ceb303c3bbd0dbd696084cf0c3a26afe0a4424" cmd=["/bin/bash","/var/lib/operator-scripts/mysql_probe.sh","readiness"] Dec 03 12:30:27 crc kubenswrapper[5052]: E1203 12:30:18.523121 5052 prober.go:104] "Probe errored" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" probeType="Readiness" pod="openstack-kuttl-tests/openstack-galera-0" podUID="12f39c79-eeb9-4cf4-8553-1a4d9a07ee82" containerName="galera" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:18.538476 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:18.546803 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage03-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage03-crc\") pod \"ovsdbserver-nb-0\" (UID: \"dccae7c1-009e-4b3b-b66a-ecbf6e53489a\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:18.555424 5052 scope.go:117] "RemoveContainer" containerID="c110d80f3a367aca9b254271248cc75eddf2eb31290f7f48d1f7ccab8d5d16fd" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:18.616590 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:18.618980 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:18.622004 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:18.649238 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/placement-688bd8fb5d-pjs4f"] Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:18.682792 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/placement-688bd8fb5d-pjs4f"] Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:18.769004 5052 scope.go:117] "RemoveContainer" containerID="46246c5deed2c10d0bc81011e4be47e5c70c228f4dda6fb9a1b20810aff514c7" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:18.777121 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-sb-0"] Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:18.876564 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-sb-0"] Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:18.918559 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ovsdbserver-sb-0"] Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:18.921300 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:18.945399 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-sb-0"] Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:18.961284 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ovncluster-ovndbcluster-sb-dockercfg-2ckjk" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:18.961501 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-ovndbcluster-sb-ovndbs" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:18.961683 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"ovndbcluster-sb-scripts" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:18.961837 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"ovndbcluster-sb-config" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:18.995555 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:19.009132 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovsdbserver-sb-tls-certs\" (UniqueName: \"kubernetes.io/secret/3fec3449-9615-4377-a600-69d816ec5416-ovsdbserver-sb-tls-certs\") pod \"ovsdbserver-sb-0\" (UID: \"3fec3449-9615-4377-a600-69d816ec5416\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:19.009217 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/3fec3449-9615-4377-a600-69d816ec5416-metrics-certs-tls-certs\") pod \"ovsdbserver-sb-0\" (UID: \"3fec3449-9615-4377-a600-69d816ec5416\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:19.009334 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/3fec3449-9615-4377-a600-69d816ec5416-ovsdb-rundir\") pod \"ovsdbserver-sb-0\" (UID: \"3fec3449-9615-4377-a600-69d816ec5416\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:19.009372 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3fec3449-9615-4377-a600-69d816ec5416-config\") pod \"ovsdbserver-sb-0\" (UID: \"3fec3449-9615-4377-a600-69d816ec5416\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:19.009464 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage08-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage08-crc\") pod \"ovsdbserver-sb-0\" (UID: \"3fec3449-9615-4377-a600-69d816ec5416\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:19.009558 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8dr2d\" (UniqueName: \"kubernetes.io/projected/3fec3449-9615-4377-a600-69d816ec5416-kube-api-access-8dr2d\") pod \"ovsdbserver-sb-0\" (UID: \"3fec3449-9615-4377-a600-69d816ec5416\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:19.009636 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/3fec3449-9615-4377-a600-69d816ec5416-scripts\") pod \"ovsdbserver-sb-0\" (UID: \"3fec3449-9615-4377-a600-69d816ec5416\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:19.009672 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3fec3449-9615-4377-a600-69d816ec5416-combined-ca-bundle\") pod \"ovsdbserver-sb-0\" (UID: \"3fec3449-9615-4377-a600-69d816ec5416\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:19.172156 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovsdbserver-sb-tls-certs\" (UniqueName: \"kubernetes.io/secret/3fec3449-9615-4377-a600-69d816ec5416-ovsdbserver-sb-tls-certs\") pod \"ovsdbserver-sb-0\" (UID: \"3fec3449-9615-4377-a600-69d816ec5416\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:19.172685 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/3fec3449-9615-4377-a600-69d816ec5416-metrics-certs-tls-certs\") pod \"ovsdbserver-sb-0\" (UID: \"3fec3449-9615-4377-a600-69d816ec5416\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:19.172786 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/3fec3449-9615-4377-a600-69d816ec5416-ovsdb-rundir\") pod \"ovsdbserver-sb-0\" (UID: \"3fec3449-9615-4377-a600-69d816ec5416\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:19.172822 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3fec3449-9615-4377-a600-69d816ec5416-config\") pod \"ovsdbserver-sb-0\" (UID: \"3fec3449-9615-4377-a600-69d816ec5416\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:19.172907 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage08-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage08-crc\") pod \"ovsdbserver-sb-0\" (UID: \"3fec3449-9615-4377-a600-69d816ec5416\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:19.172950 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-8dr2d\" (UniqueName: \"kubernetes.io/projected/3fec3449-9615-4377-a600-69d816ec5416-kube-api-access-8dr2d\") pod \"ovsdbserver-sb-0\" (UID: \"3fec3449-9615-4377-a600-69d816ec5416\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:19.173008 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/3fec3449-9615-4377-a600-69d816ec5416-scripts\") pod \"ovsdbserver-sb-0\" (UID: \"3fec3449-9615-4377-a600-69d816ec5416\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:19.173027 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3fec3449-9615-4377-a600-69d816ec5416-combined-ca-bundle\") pod \"ovsdbserver-sb-0\" (UID: \"3fec3449-9615-4377-a600-69d816ec5416\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:19.174367 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3fec3449-9615-4377-a600-69d816ec5416-config\") pod \"ovsdbserver-sb-0\" (UID: \"3fec3449-9615-4377-a600-69d816ec5416\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:19.177899 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/3fec3449-9615-4377-a600-69d816ec5416-ovsdb-rundir\") pod \"ovsdbserver-sb-0\" (UID: \"3fec3449-9615-4377-a600-69d816ec5416\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:19.178708 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/3fec3449-9615-4377-a600-69d816ec5416-metrics-certs-tls-certs\") pod \"ovsdbserver-sb-0\" (UID: \"3fec3449-9615-4377-a600-69d816ec5416\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:19.179296 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/3fec3449-9615-4377-a600-69d816ec5416-scripts\") pod \"ovsdbserver-sb-0\" (UID: \"3fec3449-9615-4377-a600-69d816ec5416\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:19.179848 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage08-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage08-crc\") pod \"ovsdbserver-sb-0\" (UID: \"3fec3449-9615-4377-a600-69d816ec5416\") device mount path \"/mnt/openstack/pv08\"" pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:19.183190 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3fec3449-9615-4377-a600-69d816ec5416-combined-ca-bundle\") pod \"ovsdbserver-sb-0\" (UID: \"3fec3449-9615-4377-a600-69d816ec5416\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:19.196878 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovsdbserver-sb-tls-certs\" (UniqueName: \"kubernetes.io/secret/3fec3449-9615-4377-a600-69d816ec5416-ovsdbserver-sb-tls-certs\") pod \"ovsdbserver-sb-0\" (UID: \"3fec3449-9615-4377-a600-69d816ec5416\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:19.206274 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-8dr2d\" (UniqueName: \"kubernetes.io/projected/3fec3449-9615-4377-a600-69d816ec5416-kube-api-access-8dr2d\") pod \"ovsdbserver-sb-0\" (UID: \"3fec3449-9615-4377-a600-69d816ec5416\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:19.230249 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage08-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage08-crc\") pod \"ovsdbserver-sb-0\" (UID: \"3fec3449-9615-4377-a600-69d816ec5416\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:19.323794 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:19.397852 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"e8f63775-6260-4abb-abd7-45a412bf0ca5","Type":"ContainerDied","Data":"b5bcab5248e26c3b89471f10eb9f60539f01ea5100b242c03a87ef7e7f42057a"} Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:19.397936 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b5bcab5248e26c3b89471f10eb9f60539f01ea5100b242c03a87ef7e7f42057a" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:19.399990 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-58d7cc6794-9z49k" event={"ID":"4b3e73a0-1320-427f-bcd7-d8062e315e39","Type":"ContainerStarted","Data":"e1fe0ce8db29a1d1e0ea67a8eaf8d8a7753223ea2fe9336a061719ff4a0463c3"} Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:19.402143 5052 generic.go:334] "Generic (PLEG): container finished" podID="c0ce9ced-33b0-4879-81a4-b5c066ec0631" containerID="589675564f1f842a7ea700f8f790adcdebd983109491fa331d941686f179e466" exitCode=0 Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:19.402171 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-564bc88b6-bvll5" event={"ID":"c0ce9ced-33b0-4879-81a4-b5c066ec0631","Type":"ContainerDied","Data":"589675564f1f842a7ea700f8f790adcdebd983109491fa331d941686f179e466"} Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:19.408306 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-769cb78446-m6hzp" event={"ID":"760aa987-6355-40cb-97c0-16c7eeddd135","Type":"ContainerStarted","Data":"bd93058f7e9a54e5b0b1a6eecb1d99267f8e0625ea9ee3a2cc7eea950b1986a3"} Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:19.414170 5052 generic.go:334] "Generic (PLEG): container finished" podID="f4e095cd-36d8-4136-b262-f6d52717a05c" containerID="1e7025513d6b144fd4de246b9cff9dd24d587dcce1c3318a8c75d6b9165b00ff" exitCode=0 Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:19.414241 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" event={"ID":"f4e095cd-36d8-4136-b262-f6d52717a05c","Type":"ContainerDied","Data":"1e7025513d6b144fd4de246b9cff9dd24d587dcce1c3318a8c75d6b9165b00ff"} Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:19.441778 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:19.442131 5052 scope.go:117] "RemoveContainer" containerID="2ccaa25866b1b39590c7e2470712343f212e8930f1a8cf1f7f1ec6457651bbae" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:19.474409 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:19.490692 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:19.575982 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" podUID="f4e095cd-36d8-4136-b262-f6d52717a05c" containerName="nova-cell1-novncproxy-novncproxy" probeResult="failure" output="Get \"https://10.217.0.207:6080/vnc_lite.html\": dial tcp 10.217.0.207:6080: connect: connection refused" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:19.594873 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/e8f63775-6260-4abb-abd7-45a412bf0ca5-logs\") pod \"e8f63775-6260-4abb-abd7-45a412bf0ca5\" (UID: \"e8f63775-6260-4abb-abd7-45a412bf0ca5\") " Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:19.595023 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e8f63775-6260-4abb-abd7-45a412bf0ca5-combined-ca-bundle\") pod \"e8f63775-6260-4abb-abd7-45a412bf0ca5\" (UID: \"e8f63775-6260-4abb-abd7-45a412bf0ca5\") " Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:19.595048 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/e8f63775-6260-4abb-abd7-45a412bf0ca5-public-tls-certs\") pod \"e8f63775-6260-4abb-abd7-45a412bf0ca5\" (UID: \"e8f63775-6260-4abb-abd7-45a412bf0ca5\") " Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:19.595165 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/e8f63775-6260-4abb-abd7-45a412bf0ca5-httpd-run\") pod \"e8f63775-6260-4abb-abd7-45a412bf0ca5\" (UID: \"e8f63775-6260-4abb-abd7-45a412bf0ca5\") " Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:19.595192 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/e8f63775-6260-4abb-abd7-45a412bf0ca5-scripts\") pod \"e8f63775-6260-4abb-abd7-45a412bf0ca5\" (UID: \"e8f63775-6260-4abb-abd7-45a412bf0ca5\") " Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:19.595218 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"glance\" (UniqueName: \"kubernetes.io/local-volume/local-storage07-crc\") pod \"e8f63775-6260-4abb-abd7-45a412bf0ca5\" (UID: \"e8f63775-6260-4abb-abd7-45a412bf0ca5\") " Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:19.595237 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/e8f63775-6260-4abb-abd7-45a412bf0ca5-config-data\") pod \"e8f63775-6260-4abb-abd7-45a412bf0ca5\" (UID: \"e8f63775-6260-4abb-abd7-45a412bf0ca5\") " Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:19.595335 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4cnf9\" (UniqueName: \"kubernetes.io/projected/e8f63775-6260-4abb-abd7-45a412bf0ca5-kube-api-access-4cnf9\") pod \"e8f63775-6260-4abb-abd7-45a412bf0ca5\" (UID: \"e8f63775-6260-4abb-abd7-45a412bf0ca5\") " Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:19.595421 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e8f63775-6260-4abb-abd7-45a412bf0ca5-logs" (OuterVolumeSpecName: "logs") pod "e8f63775-6260-4abb-abd7-45a412bf0ca5" (UID: "e8f63775-6260-4abb-abd7-45a412bf0ca5"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:19.595822 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/e8f63775-6260-4abb-abd7-45a412bf0ca5-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:19.596841 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e8f63775-6260-4abb-abd7-45a412bf0ca5-httpd-run" (OuterVolumeSpecName: "httpd-run") pod "e8f63775-6260-4abb-abd7-45a412bf0ca5" (UID: "e8f63775-6260-4abb-abd7-45a412bf0ca5"). InnerVolumeSpecName "httpd-run". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:19.605981 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage07-crc" (OuterVolumeSpecName: "glance") pod "e8f63775-6260-4abb-abd7-45a412bf0ca5" (UID: "e8f63775-6260-4abb-abd7-45a412bf0ca5"). InnerVolumeSpecName "local-storage07-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:19.610584 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e8f63775-6260-4abb-abd7-45a412bf0ca5-kube-api-access-4cnf9" (OuterVolumeSpecName: "kube-api-access-4cnf9") pod "e8f63775-6260-4abb-abd7-45a412bf0ca5" (UID: "e8f63775-6260-4abb-abd7-45a412bf0ca5"). InnerVolumeSpecName "kube-api-access-4cnf9". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:19.625676 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e8f63775-6260-4abb-abd7-45a412bf0ca5-scripts" (OuterVolumeSpecName: "scripts") pod "e8f63775-6260-4abb-abd7-45a412bf0ca5" (UID: "e8f63775-6260-4abb-abd7-45a412bf0ca5"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:19.652988 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e8f63775-6260-4abb-abd7-45a412bf0ca5-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "e8f63775-6260-4abb-abd7-45a412bf0ca5" (UID: "e8f63775-6260-4abb-abd7-45a412bf0ca5"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:19.692001 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e8f63775-6260-4abb-abd7-45a412bf0ca5-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "e8f63775-6260-4abb-abd7-45a412bf0ca5" (UID: "e8f63775-6260-4abb-abd7-45a412bf0ca5"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:19.698210 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-4cnf9\" (UniqueName: \"kubernetes.io/projected/e8f63775-6260-4abb-abd7-45a412bf0ca5-kube-api-access-4cnf9\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:19.698237 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e8f63775-6260-4abb-abd7-45a412bf0ca5-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:19.698246 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/e8f63775-6260-4abb-abd7-45a412bf0ca5-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:19.698254 5052 reconciler_common.go:293] "Volume detached for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/e8f63775-6260-4abb-abd7-45a412bf0ca5-httpd-run\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:19.698265 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/e8f63775-6260-4abb-abd7-45a412bf0ca5-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:19.698307 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage07-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage07-crc\") on node \"crc\" " Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:19.716701 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e8f63775-6260-4abb-abd7-45a412bf0ca5-config-data" (OuterVolumeSpecName: "config-data") pod "e8f63775-6260-4abb-abd7-45a412bf0ca5" (UID: "e8f63775-6260-4abb-abd7-45a412bf0ca5"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:19.725930 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage07-crc" (UniqueName: "kubernetes.io/local-volume/local-storage07-crc") on node "crc" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:19.729125 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1de1d25b-cbcc-4d08-a7cd-aa54ac71aecb" path="/var/lib/kubelet/pods/1de1d25b-cbcc-4d08-a7cd-aa54ac71aecb/volumes" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:19.729833 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="50f5af82-8ee5-45f4-b158-2166f1be6cb8" path="/var/lib/kubelet/pods/50f5af82-8ee5-45f4-b158-2166f1be6cb8/volumes" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:19.731088 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="53fe6403-27db-44b7-a9a6-787afbc3bbb8" path="/var/lib/kubelet/pods/53fe6403-27db-44b7-a9a6-787afbc3bbb8/volumes" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:19.731779 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8bddf2b1-8116-4b6e-9cd8-be8fe1cd4b64" path="/var/lib/kubelet/pods/8bddf2b1-8116-4b6e-9cd8-be8fe1cd4b64/volumes" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:19.732487 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="98f722f9-fa25-40b6-84ba-96127d07293c" path="/var/lib/kubelet/pods/98f722f9-fa25-40b6-84ba-96127d07293c/volumes" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:19.733558 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a0e45e1a-ced9-4958-9a75-24a69280f679" path="/var/lib/kubelet/pods/a0e45e1a-ced9-4958-9a75-24a69280f679/volumes" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:19.849417 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage07-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage07-crc\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:19.849441 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/e8f63775-6260-4abb-abd7-45a412bf0ca5-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:20.069730 5052 prober.go:107] "Probe failed" probeType="Liveness" pod="openstack-kuttl-tests/barbican-api-ff6b8c44b-7ndgm" podUID="48831cad-910f-46f1-aefe-7cc4cbfa984f" containerName="barbican-api" probeResult="failure" output="Get \"https://10.217.0.204:9311/healthcheck\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:20.069734 5052 prober.go:107] "Probe failed" probeType="Liveness" pod="openstack-kuttl-tests/barbican-api-ff6b8c44b-7ndgm" podUID="48831cad-910f-46f1-aefe-7cc4cbfa984f" containerName="barbican-api-log" probeResult="failure" output="Get \"https://10.217.0.204:9311/healthcheck\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:20.367973 5052 kubelet.go:2542] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:20.399331 5052 kubelet.go:2542] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:20.430911 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:20.455913 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:20.468697 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:20.479937 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:30:27 crc kubenswrapper[5052]: E1203 12:30:20.480431 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="e8f63775-6260-4abb-abd7-45a412bf0ca5" containerName="glance-log" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:20.480448 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="e8f63775-6260-4abb-abd7-45a412bf0ca5" containerName="glance-log" Dec 03 12:30:27 crc kubenswrapper[5052]: E1203 12:30:20.480492 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="e8f63775-6260-4abb-abd7-45a412bf0ca5" containerName="glance-httpd" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:20.480501 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="e8f63775-6260-4abb-abd7-45a412bf0ca5" containerName="glance-httpd" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:20.480764 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="e8f63775-6260-4abb-abd7-45a412bf0ca5" containerName="glance-httpd" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:20.480790 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="e8f63775-6260-4abb-abd7-45a412bf0ca5" containerName="glance-log" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:20.482093 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:20.491364 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-scripts" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:20.491757 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-glance-default-public-svc" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:20.492475 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-glance-dockercfg-cd9ns" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:20.492758 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-default-external-config-data" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:20.512946 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:20.762508 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/a65428bc-9326-4ace-a985-9f796b951304-scripts\") pod \"glance-default-external-api-0\" (UID: \"a65428bc-9326-4ace-a985-9f796b951304\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:20.762955 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage07-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage07-crc\") pod \"glance-default-external-api-0\" (UID: \"a65428bc-9326-4ace-a985-9f796b951304\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:20.763034 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a65428bc-9326-4ace-a985-9f796b951304-combined-ca-bundle\") pod \"glance-default-external-api-0\" (UID: \"a65428bc-9326-4ace-a985-9f796b951304\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:20.763065 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-npmcg\" (UniqueName: \"kubernetes.io/projected/a65428bc-9326-4ace-a985-9f796b951304-kube-api-access-npmcg\") pod \"glance-default-external-api-0\" (UID: \"a65428bc-9326-4ace-a985-9f796b951304\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:20.763100 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/a65428bc-9326-4ace-a985-9f796b951304-httpd-run\") pod \"glance-default-external-api-0\" (UID: \"a65428bc-9326-4ace-a985-9f796b951304\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:20.763131 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/a65428bc-9326-4ace-a985-9f796b951304-logs\") pod \"glance-default-external-api-0\" (UID: \"a65428bc-9326-4ace-a985-9f796b951304\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:20.763175 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/a65428bc-9326-4ace-a985-9f796b951304-public-tls-certs\") pod \"glance-default-external-api-0\" (UID: \"a65428bc-9326-4ace-a985-9f796b951304\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:20.763471 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a65428bc-9326-4ace-a985-9f796b951304-config-data\") pod \"glance-default-external-api-0\" (UID: \"a65428bc-9326-4ace-a985-9f796b951304\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:20.865058 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/a65428bc-9326-4ace-a985-9f796b951304-scripts\") pod \"glance-default-external-api-0\" (UID: \"a65428bc-9326-4ace-a985-9f796b951304\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:20.865135 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage07-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage07-crc\") pod \"glance-default-external-api-0\" (UID: \"a65428bc-9326-4ace-a985-9f796b951304\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:20.865189 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a65428bc-9326-4ace-a985-9f796b951304-combined-ca-bundle\") pod \"glance-default-external-api-0\" (UID: \"a65428bc-9326-4ace-a985-9f796b951304\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:20.865209 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-npmcg\" (UniqueName: \"kubernetes.io/projected/a65428bc-9326-4ace-a985-9f796b951304-kube-api-access-npmcg\") pod \"glance-default-external-api-0\" (UID: \"a65428bc-9326-4ace-a985-9f796b951304\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:20.865235 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/a65428bc-9326-4ace-a985-9f796b951304-httpd-run\") pod \"glance-default-external-api-0\" (UID: \"a65428bc-9326-4ace-a985-9f796b951304\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:20.865257 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/a65428bc-9326-4ace-a985-9f796b951304-logs\") pod \"glance-default-external-api-0\" (UID: \"a65428bc-9326-4ace-a985-9f796b951304\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:20.865290 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/a65428bc-9326-4ace-a985-9f796b951304-public-tls-certs\") pod \"glance-default-external-api-0\" (UID: \"a65428bc-9326-4ace-a985-9f796b951304\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:20.865360 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a65428bc-9326-4ace-a985-9f796b951304-config-data\") pod \"glance-default-external-api-0\" (UID: \"a65428bc-9326-4ace-a985-9f796b951304\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:20.866634 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/a65428bc-9326-4ace-a985-9f796b951304-logs\") pod \"glance-default-external-api-0\" (UID: \"a65428bc-9326-4ace-a985-9f796b951304\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:20.867603 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage07-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage07-crc\") pod \"glance-default-external-api-0\" (UID: \"a65428bc-9326-4ace-a985-9f796b951304\") device mount path \"/mnt/openstack/pv07\"" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:20.870012 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/a65428bc-9326-4ace-a985-9f796b951304-httpd-run\") pod \"glance-default-external-api-0\" (UID: \"a65428bc-9326-4ace-a985-9f796b951304\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:20.880908 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a65428bc-9326-4ace-a985-9f796b951304-config-data\") pod \"glance-default-external-api-0\" (UID: \"a65428bc-9326-4ace-a985-9f796b951304\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:20.882175 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/a65428bc-9326-4ace-a985-9f796b951304-scripts\") pod \"glance-default-external-api-0\" (UID: \"a65428bc-9326-4ace-a985-9f796b951304\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:20.884145 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/a65428bc-9326-4ace-a985-9f796b951304-public-tls-certs\") pod \"glance-default-external-api-0\" (UID: \"a65428bc-9326-4ace-a985-9f796b951304\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:20.885077 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-npmcg\" (UniqueName: \"kubernetes.io/projected/a65428bc-9326-4ace-a985-9f796b951304-kube-api-access-npmcg\") pod \"glance-default-external-api-0\" (UID: \"a65428bc-9326-4ace-a985-9f796b951304\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:20.885837 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a65428bc-9326-4ace-a985-9f796b951304-combined-ca-bundle\") pod \"glance-default-external-api-0\" (UID: \"a65428bc-9326-4ace-a985-9f796b951304\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:20.908900 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage07-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage07-crc\") pod \"glance-default-external-api-0\" (UID: \"a65428bc-9326-4ace-a985-9f796b951304\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:21.104420 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:21.427126 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-api-ff6b8c44b-7ndgm"] Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:21.427601 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/barbican-api-ff6b8c44b-7ndgm" podUID="48831cad-910f-46f1-aefe-7cc4cbfa984f" containerName="barbican-api-log" containerID="cri-o://b447fe4d34e0569d637c38fcee6f843a5b3a8e43d2bd2e661971752993efd16e" gracePeriod=30 Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:21.427740 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/barbican-api-ff6b8c44b-7ndgm" podUID="48831cad-910f-46f1-aefe-7cc4cbfa984f" containerName="barbican-api" containerID="cri-o://f6c28d3863e0d0d300bbffab66b3c5240cbeaa832b51d4164b9f0b3c092d3831" gracePeriod=30 Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:21.433950 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/barbican-api-ff6b8c44b-7ndgm" podUID="48831cad-910f-46f1-aefe-7cc4cbfa984f" containerName="barbican-api" probeResult="failure" output="Get \"https://10.217.0.204:9311/healthcheck\": EOF" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:21.434605 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/barbican-api-ff6b8c44b-7ndgm" podUID="48831cad-910f-46f1-aefe-7cc4cbfa984f" containerName="barbican-api-log" probeResult="failure" output="Get \"https://10.217.0.204:9311/healthcheck\": EOF" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:21.436473 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/barbican-api-ff6b8c44b-7ndgm" podUID="48831cad-910f-46f1-aefe-7cc4cbfa984f" containerName="barbican-api" probeResult="failure" output="Get \"https://10.217.0.204:9311/healthcheck\": read tcp 10.217.0.2:59334->10.217.0.204:9311: read: connection reset by peer" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:21.436564 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/barbican-api-ff6b8c44b-7ndgm" podUID="48831cad-910f-46f1-aefe-7cc4cbfa984f" containerName="barbican-api-log" probeResult="failure" output="Get \"https://10.217.0.204:9311/healthcheck\": read tcp 10.217.0.2:59344->10.217.0.204:9311: read: connection reset by peer" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:21.439140 5052 prober.go:107] "Probe failed" probeType="Liveness" pod="openstack-kuttl-tests/barbican-api-ff6b8c44b-7ndgm" podUID="48831cad-910f-46f1-aefe-7cc4cbfa984f" containerName="barbican-api" probeResult="failure" output="Get \"https://10.217.0.204:9311/healthcheck\": EOF" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:21.439198 5052 kubelet.go:2542] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openstack-kuttl-tests/barbican-api-ff6b8c44b-7ndgm" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:21.439283 5052 prober.go:107] "Probe failed" probeType="Liveness" pod="openstack-kuttl-tests/barbican-api-ff6b8c44b-7ndgm" podUID="48831cad-910f-46f1-aefe-7cc4cbfa984f" containerName="barbican-api-log" probeResult="failure" output="Get \"https://10.217.0.204:9311/healthcheck\": EOF" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:21.439369 5052 kubelet.go:2542] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openstack-kuttl-tests/barbican-api-ff6b8c44b-7ndgm" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:21.465598 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/barbican-api-64d69c4db4-hkscs"] Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:21.469031 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-api-64d69c4db4-hkscs" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:21.538013 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-api-64d69c4db4-hkscs"] Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:21.577441 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/ff58ae3c-d75e-4038-be70-00f88178a18f-public-tls-certs\") pod \"barbican-api-64d69c4db4-hkscs\" (UID: \"ff58ae3c-d75e-4038-be70-00f88178a18f\") " pod="openstack-kuttl-tests/barbican-api-64d69c4db4-hkscs" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:21.577492 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/ff58ae3c-d75e-4038-be70-00f88178a18f-internal-tls-certs\") pod \"barbican-api-64d69c4db4-hkscs\" (UID: \"ff58ae3c-d75e-4038-be70-00f88178a18f\") " pod="openstack-kuttl-tests/barbican-api-64d69c4db4-hkscs" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:21.577549 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/ff58ae3c-d75e-4038-be70-00f88178a18f-config-data\") pod \"barbican-api-64d69c4db4-hkscs\" (UID: \"ff58ae3c-d75e-4038-be70-00f88178a18f\") " pod="openstack-kuttl-tests/barbican-api-64d69c4db4-hkscs" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:21.578383 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jm72m\" (UniqueName: \"kubernetes.io/projected/ff58ae3c-d75e-4038-be70-00f88178a18f-kube-api-access-jm72m\") pod \"barbican-api-64d69c4db4-hkscs\" (UID: \"ff58ae3c-d75e-4038-be70-00f88178a18f\") " pod="openstack-kuttl-tests/barbican-api-64d69c4db4-hkscs" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:21.578499 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/ff58ae3c-d75e-4038-be70-00f88178a18f-config-data-custom\") pod \"barbican-api-64d69c4db4-hkscs\" (UID: \"ff58ae3c-d75e-4038-be70-00f88178a18f\") " pod="openstack-kuttl-tests/barbican-api-64d69c4db4-hkscs" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:21.578574 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/ff58ae3c-d75e-4038-be70-00f88178a18f-logs\") pod \"barbican-api-64d69c4db4-hkscs\" (UID: \"ff58ae3c-d75e-4038-be70-00f88178a18f\") " pod="openstack-kuttl-tests/barbican-api-64d69c4db4-hkscs" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:21.578691 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ff58ae3c-d75e-4038-be70-00f88178a18f-combined-ca-bundle\") pod \"barbican-api-64d69c4db4-hkscs\" (UID: \"ff58ae3c-d75e-4038-be70-00f88178a18f\") " pod="openstack-kuttl-tests/barbican-api-64d69c4db4-hkscs" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:21.680340 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/ff58ae3c-d75e-4038-be70-00f88178a18f-config-data-custom\") pod \"barbican-api-64d69c4db4-hkscs\" (UID: \"ff58ae3c-d75e-4038-be70-00f88178a18f\") " pod="openstack-kuttl-tests/barbican-api-64d69c4db4-hkscs" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:21.680430 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/ff58ae3c-d75e-4038-be70-00f88178a18f-logs\") pod \"barbican-api-64d69c4db4-hkscs\" (UID: \"ff58ae3c-d75e-4038-be70-00f88178a18f\") " pod="openstack-kuttl-tests/barbican-api-64d69c4db4-hkscs" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:21.680877 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/ff58ae3c-d75e-4038-be70-00f88178a18f-logs\") pod \"barbican-api-64d69c4db4-hkscs\" (UID: \"ff58ae3c-d75e-4038-be70-00f88178a18f\") " pod="openstack-kuttl-tests/barbican-api-64d69c4db4-hkscs" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:21.680935 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ff58ae3c-d75e-4038-be70-00f88178a18f-combined-ca-bundle\") pod \"barbican-api-64d69c4db4-hkscs\" (UID: \"ff58ae3c-d75e-4038-be70-00f88178a18f\") " pod="openstack-kuttl-tests/barbican-api-64d69c4db4-hkscs" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:21.681378 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/ff58ae3c-d75e-4038-be70-00f88178a18f-public-tls-certs\") pod \"barbican-api-64d69c4db4-hkscs\" (UID: \"ff58ae3c-d75e-4038-be70-00f88178a18f\") " pod="openstack-kuttl-tests/barbican-api-64d69c4db4-hkscs" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:21.681426 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/ff58ae3c-d75e-4038-be70-00f88178a18f-internal-tls-certs\") pod \"barbican-api-64d69c4db4-hkscs\" (UID: \"ff58ae3c-d75e-4038-be70-00f88178a18f\") " pod="openstack-kuttl-tests/barbican-api-64d69c4db4-hkscs" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:21.681487 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/ff58ae3c-d75e-4038-be70-00f88178a18f-config-data\") pod \"barbican-api-64d69c4db4-hkscs\" (UID: \"ff58ae3c-d75e-4038-be70-00f88178a18f\") " pod="openstack-kuttl-tests/barbican-api-64d69c4db4-hkscs" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:21.681685 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-jm72m\" (UniqueName: \"kubernetes.io/projected/ff58ae3c-d75e-4038-be70-00f88178a18f-kube-api-access-jm72m\") pod \"barbican-api-64d69c4db4-hkscs\" (UID: \"ff58ae3c-d75e-4038-be70-00f88178a18f\") " pod="openstack-kuttl-tests/barbican-api-64d69c4db4-hkscs" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:21.685737 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e8f63775-6260-4abb-abd7-45a412bf0ca5" path="/var/lib/kubelet/pods/e8f63775-6260-4abb-abd7-45a412bf0ca5/volumes" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:21.685963 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ff58ae3c-d75e-4038-be70-00f88178a18f-combined-ca-bundle\") pod \"barbican-api-64d69c4db4-hkscs\" (UID: \"ff58ae3c-d75e-4038-be70-00f88178a18f\") " pod="openstack-kuttl-tests/barbican-api-64d69c4db4-hkscs" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:21.686344 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/ff58ae3c-d75e-4038-be70-00f88178a18f-public-tls-certs\") pod \"barbican-api-64d69c4db4-hkscs\" (UID: \"ff58ae3c-d75e-4038-be70-00f88178a18f\") " pod="openstack-kuttl-tests/barbican-api-64d69c4db4-hkscs" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:21.688725 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/ff58ae3c-d75e-4038-be70-00f88178a18f-config-data\") pod \"barbican-api-64d69c4db4-hkscs\" (UID: \"ff58ae3c-d75e-4038-be70-00f88178a18f\") " pod="openstack-kuttl-tests/barbican-api-64d69c4db4-hkscs" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:21.691208 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/ff58ae3c-d75e-4038-be70-00f88178a18f-internal-tls-certs\") pod \"barbican-api-64d69c4db4-hkscs\" (UID: \"ff58ae3c-d75e-4038-be70-00f88178a18f\") " pod="openstack-kuttl-tests/barbican-api-64d69c4db4-hkscs" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:21.700147 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/ff58ae3c-d75e-4038-be70-00f88178a18f-config-data-custom\") pod \"barbican-api-64d69c4db4-hkscs\" (UID: \"ff58ae3c-d75e-4038-be70-00f88178a18f\") " pod="openstack-kuttl-tests/barbican-api-64d69c4db4-hkscs" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:21.703177 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-jm72m\" (UniqueName: \"kubernetes.io/projected/ff58ae3c-d75e-4038-be70-00f88178a18f-kube-api-access-jm72m\") pod \"barbican-api-64d69c4db4-hkscs\" (UID: \"ff58ae3c-d75e-4038-be70-00f88178a18f\") " pod="openstack-kuttl-tests/barbican-api-64d69c4db4-hkscs" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:21.849785 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-api-64d69c4db4-hkscs" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:21.929870 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-api-64d69c4db4-hkscs"] Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:21.960395 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/barbican-api-68fc8f9776-n9gml"] Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:21.962235 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-api-68fc8f9776-n9gml" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:21.979261 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-api-68fc8f9776-n9gml"] Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:22.090911 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/82f142ab-5be9-4727-b3c5-102d6f2401fb-internal-tls-certs\") pod \"barbican-api-68fc8f9776-n9gml\" (UID: \"82f142ab-5be9-4727-b3c5-102d6f2401fb\") " pod="openstack-kuttl-tests/barbican-api-68fc8f9776-n9gml" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:22.091102 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-md4n8\" (UniqueName: \"kubernetes.io/projected/82f142ab-5be9-4727-b3c5-102d6f2401fb-kube-api-access-md4n8\") pod \"barbican-api-68fc8f9776-n9gml\" (UID: \"82f142ab-5be9-4727-b3c5-102d6f2401fb\") " pod="openstack-kuttl-tests/barbican-api-68fc8f9776-n9gml" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:22.091173 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/82f142ab-5be9-4727-b3c5-102d6f2401fb-config-data-custom\") pod \"barbican-api-68fc8f9776-n9gml\" (UID: \"82f142ab-5be9-4727-b3c5-102d6f2401fb\") " pod="openstack-kuttl-tests/barbican-api-68fc8f9776-n9gml" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:22.091201 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/82f142ab-5be9-4727-b3c5-102d6f2401fb-logs\") pod \"barbican-api-68fc8f9776-n9gml\" (UID: \"82f142ab-5be9-4727-b3c5-102d6f2401fb\") " pod="openstack-kuttl-tests/barbican-api-68fc8f9776-n9gml" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:22.091261 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/82f142ab-5be9-4727-b3c5-102d6f2401fb-config-data\") pod \"barbican-api-68fc8f9776-n9gml\" (UID: \"82f142ab-5be9-4727-b3c5-102d6f2401fb\") " pod="openstack-kuttl-tests/barbican-api-68fc8f9776-n9gml" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:22.091307 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/82f142ab-5be9-4727-b3c5-102d6f2401fb-public-tls-certs\") pod \"barbican-api-68fc8f9776-n9gml\" (UID: \"82f142ab-5be9-4727-b3c5-102d6f2401fb\") " pod="openstack-kuttl-tests/barbican-api-68fc8f9776-n9gml" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:22.091467 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/82f142ab-5be9-4727-b3c5-102d6f2401fb-combined-ca-bundle\") pod \"barbican-api-68fc8f9776-n9gml\" (UID: \"82f142ab-5be9-4727-b3c5-102d6f2401fb\") " pod="openstack-kuttl-tests/barbican-api-68fc8f9776-n9gml" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:22.193201 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/82f142ab-5be9-4727-b3c5-102d6f2401fb-config-data\") pod \"barbican-api-68fc8f9776-n9gml\" (UID: \"82f142ab-5be9-4727-b3c5-102d6f2401fb\") " pod="openstack-kuttl-tests/barbican-api-68fc8f9776-n9gml" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:22.193246 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/82f142ab-5be9-4727-b3c5-102d6f2401fb-public-tls-certs\") pod \"barbican-api-68fc8f9776-n9gml\" (UID: \"82f142ab-5be9-4727-b3c5-102d6f2401fb\") " pod="openstack-kuttl-tests/barbican-api-68fc8f9776-n9gml" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:22.193301 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/82f142ab-5be9-4727-b3c5-102d6f2401fb-combined-ca-bundle\") pod \"barbican-api-68fc8f9776-n9gml\" (UID: \"82f142ab-5be9-4727-b3c5-102d6f2401fb\") " pod="openstack-kuttl-tests/barbican-api-68fc8f9776-n9gml" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:22.193391 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/82f142ab-5be9-4727-b3c5-102d6f2401fb-internal-tls-certs\") pod \"barbican-api-68fc8f9776-n9gml\" (UID: \"82f142ab-5be9-4727-b3c5-102d6f2401fb\") " pod="openstack-kuttl-tests/barbican-api-68fc8f9776-n9gml" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:22.193434 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-md4n8\" (UniqueName: \"kubernetes.io/projected/82f142ab-5be9-4727-b3c5-102d6f2401fb-kube-api-access-md4n8\") pod \"barbican-api-68fc8f9776-n9gml\" (UID: \"82f142ab-5be9-4727-b3c5-102d6f2401fb\") " pod="openstack-kuttl-tests/barbican-api-68fc8f9776-n9gml" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:22.193459 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/82f142ab-5be9-4727-b3c5-102d6f2401fb-config-data-custom\") pod \"barbican-api-68fc8f9776-n9gml\" (UID: \"82f142ab-5be9-4727-b3c5-102d6f2401fb\") " pod="openstack-kuttl-tests/barbican-api-68fc8f9776-n9gml" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:22.193476 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/82f142ab-5be9-4727-b3c5-102d6f2401fb-logs\") pod \"barbican-api-68fc8f9776-n9gml\" (UID: \"82f142ab-5be9-4727-b3c5-102d6f2401fb\") " pod="openstack-kuttl-tests/barbican-api-68fc8f9776-n9gml" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:22.194010 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/82f142ab-5be9-4727-b3c5-102d6f2401fb-logs\") pod \"barbican-api-68fc8f9776-n9gml\" (UID: \"82f142ab-5be9-4727-b3c5-102d6f2401fb\") " pod="openstack-kuttl-tests/barbican-api-68fc8f9776-n9gml" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:22.198099 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/82f142ab-5be9-4727-b3c5-102d6f2401fb-config-data-custom\") pod \"barbican-api-68fc8f9776-n9gml\" (UID: \"82f142ab-5be9-4727-b3c5-102d6f2401fb\") " pod="openstack-kuttl-tests/barbican-api-68fc8f9776-n9gml" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:22.198622 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/82f142ab-5be9-4727-b3c5-102d6f2401fb-combined-ca-bundle\") pod \"barbican-api-68fc8f9776-n9gml\" (UID: \"82f142ab-5be9-4727-b3c5-102d6f2401fb\") " pod="openstack-kuttl-tests/barbican-api-68fc8f9776-n9gml" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:22.200958 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/82f142ab-5be9-4727-b3c5-102d6f2401fb-internal-tls-certs\") pod \"barbican-api-68fc8f9776-n9gml\" (UID: \"82f142ab-5be9-4727-b3c5-102d6f2401fb\") " pod="openstack-kuttl-tests/barbican-api-68fc8f9776-n9gml" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:22.214849 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/82f142ab-5be9-4727-b3c5-102d6f2401fb-public-tls-certs\") pod \"barbican-api-68fc8f9776-n9gml\" (UID: \"82f142ab-5be9-4727-b3c5-102d6f2401fb\") " pod="openstack-kuttl-tests/barbican-api-68fc8f9776-n9gml" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:22.215796 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/82f142ab-5be9-4727-b3c5-102d6f2401fb-config-data\") pod \"barbican-api-68fc8f9776-n9gml\" (UID: \"82f142ab-5be9-4727-b3c5-102d6f2401fb\") " pod="openstack-kuttl-tests/barbican-api-68fc8f9776-n9gml" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:22.217417 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-md4n8\" (UniqueName: \"kubernetes.io/projected/82f142ab-5be9-4727-b3c5-102d6f2401fb-kube-api-access-md4n8\") pod \"barbican-api-68fc8f9776-n9gml\" (UID: \"82f142ab-5be9-4727-b3c5-102d6f2401fb\") " pod="openstack-kuttl-tests/barbican-api-68fc8f9776-n9gml" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:22.283114 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-api-68fc8f9776-n9gml" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:22.456209 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-conductor-0" event={"ID":"2dfbec8a-3a04-407b-8145-e12a3afee2e0","Type":"ContainerStarted","Data":"421784ccacc82cea351bc14db49b9cace93a8a873e7c71fae104a1ba2d00d40c"} Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:22.458448 5052 generic.go:334] "Generic (PLEG): container finished" podID="72a32b4d-d61d-43a5-a45d-c1fd4bad672d" containerID="6d2a1c1187cb71ec6764a5792108b52b3d8a71f435dceee72f6a125558106e90" exitCode=0 Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:22.458491 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-worker-679bfdf7b7-tg6h7" event={"ID":"72a32b4d-d61d-43a5-a45d-c1fd4bad672d","Type":"ContainerDied","Data":"6d2a1c1187cb71ec6764a5792108b52b3d8a71f435dceee72f6a125558106e90"} Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:23.472582 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-conductor-0" event={"ID":"e2718906-e52d-42fa-ad42-35626e99d461","Type":"ContainerStarted","Data":"96578ef7df608942efd7e88b79f20d9254a37ea8b6ab07704fddbcfab40445f1"} Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:24.234753 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-metadata-0" podUID="39117149-c15d-4753-9324-8a42abe5b007" containerName="nova-metadata-metadata" probeResult="failure" output="Get \"https://10.217.0.213:8775/\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:24.235578 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-metadata-0" podUID="39117149-c15d-4753-9324-8a42abe5b007" containerName="nova-metadata-log" probeResult="failure" output="Get \"https://10.217.0.213:8775/\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:24.489173 5052 generic.go:334] "Generic (PLEG): container finished" podID="48831cad-910f-46f1-aefe-7cc4cbfa984f" containerID="b447fe4d34e0569d637c38fcee6f843a5b3a8e43d2bd2e661971752993efd16e" exitCode=143 Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:24.489232 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-ff6b8c44b-7ndgm" event={"ID":"48831cad-910f-46f1-aefe-7cc4cbfa984f","Type":"ContainerDied","Data":"b447fe4d34e0569d637c38fcee6f843a5b3a8e43d2bd2e661971752993efd16e"} Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:24.493486 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"bb1723c7-4e41-4b0e-b6b8-f8024dafdaee","Type":"ContainerStarted","Data":"ab269ac11d99aed29d8c53741cb5bdd97ed1fe052bc19e5e9c3e0f4152eaeed4"} Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:24.498722 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-76bcd49fb6-54vvs" event={"ID":"6bfd5f51-8b37-4fea-8134-a2cdcd4f6049","Type":"ContainerStarted","Data":"0a0d6f3815eb929a36dbe3844272f6d6abfd6692349a5d9de8c42e6c899327d3"} Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:24.498769 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:24.498795 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/neutron-769cb78446-m6hzp" podUID="760aa987-6355-40cb-97c0-16c7eeddd135" containerName="neutron-api" containerID="cri-o://1cdac21bc509913963cca59d86d5d3a547a3bc1e21fc0a5a1c2c9f6e47211913" gracePeriod=30 Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:24.498942 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/neutron-769cb78446-m6hzp" podUID="760aa987-6355-40cb-97c0-16c7eeddd135" containerName="neutron-httpd" containerID="cri-o://bd93058f7e9a54e5b0b1a6eecb1d99267f8e0625ea9ee3a2cc7eea950b1986a3" gracePeriod=30 Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:24.499234 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/neutron-769cb78446-m6hzp" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:24.505096 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/neutron-769cb78446-m6hzp" podUID="760aa987-6355-40cb-97c0-16c7eeddd135" containerName="neutron-httpd" probeResult="failure" output="HTTP probe failed with statuscode: 503" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:24.532957 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/neutron-769cb78446-m6hzp" podStartSLOduration=13.532933445 podStartE2EDuration="13.532933445s" podCreationTimestamp="2025-12-03 12:30:11 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:30:24.519867482 +0000 UTC m=+4215.108109246" watchObservedRunningTime="2025-12-03 12:30:24.532933445 +0000 UTC m=+4215.121175209" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:24.567336 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" podUID="f4e095cd-36d8-4136-b262-f6d52717a05c" containerName="nova-cell1-novncproxy-novncproxy" probeResult="failure" output="Get \"https://10.217.0.207:6080/vnc_lite.html\": dial tcp 10.217.0.207:6080: connect: connection refused" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:25.521009 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-58d7cc6794-9z49k" event={"ID":"4b3e73a0-1320-427f-bcd7-d8062e315e39","Type":"ContainerStarted","Data":"82c2ce15a7d7e6c87d6ed20e3eaafec5eb24b16b4abc6508d88b41511f4560a0"} Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:25.522405 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/neutron-58d7cc6794-9z49k" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:25.526206 5052 generic.go:334] "Generic (PLEG): container finished" podID="760aa987-6355-40cb-97c0-16c7eeddd135" containerID="bd93058f7e9a54e5b0b1a6eecb1d99267f8e0625ea9ee3a2cc7eea950b1986a3" exitCode=0 Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:25.526242 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-769cb78446-m6hzp" event={"ID":"760aa987-6355-40cb-97c0-16c7eeddd135","Type":"ContainerDied","Data":"bd93058f7e9a54e5b0b1a6eecb1d99267f8e0625ea9ee3a2cc7eea950b1986a3"} Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:25.528812 5052 generic.go:334] "Generic (PLEG): container finished" podID="2dfbec8a-3a04-407b-8145-e12a3afee2e0" containerID="421784ccacc82cea351bc14db49b9cace93a8a873e7c71fae104a1ba2d00d40c" exitCode=1 Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:25.528943 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-conductor-0" event={"ID":"2dfbec8a-3a04-407b-8145-e12a3afee2e0","Type":"ContainerDied","Data":"421784ccacc82cea351bc14db49b9cace93a8a873e7c71fae104a1ba2d00d40c"} Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:25.528980 5052 scope.go:117] "RemoveContainer" containerID="f709bafefdc660d91c439ef2df4b5fa8dafa8380b2dee355f1b04956932b91e7" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:25.529384 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:25.529551 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/neutron-76bcd49fb6-54vvs" podUID="6bfd5f51-8b37-4fea-8134-a2cdcd4f6049" containerName="neutron-api" containerID="cri-o://01711fff696becc6dfb1b8494228bffbaa942ad3546265fb3e7434412310303c" gracePeriod=30 Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:25.529635 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/neutron-76bcd49fb6-54vvs" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:25.529680 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/neutron-76bcd49fb6-54vvs" podUID="6bfd5f51-8b37-4fea-8134-a2cdcd4f6049" containerName="neutron-httpd" containerID="cri-o://0a0d6f3815eb929a36dbe3844272f6d6abfd6692349a5d9de8c42e6c899327d3" gracePeriod=30 Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:25.530054 5052 scope.go:117] "RemoveContainer" containerID="421784ccacc82cea351bc14db49b9cace93a8a873e7c71fae104a1ba2d00d40c" Dec 03 12:30:27 crc kubenswrapper[5052]: E1203 12:30:25.530253 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"nova-cell1-conductor-conductor\" with CrashLoopBackOff: \"back-off 10s restarting failed container=nova-cell1-conductor-conductor pod=nova-cell1-conductor-0_openstack-kuttl-tests(2dfbec8a-3a04-407b-8145-e12a3afee2e0)\"" pod="openstack-kuttl-tests/nova-cell1-conductor-0" podUID="2dfbec8a-3a04-407b-8145-e12a3afee2e0" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:25.554733 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/neutron-58d7cc6794-9z49k" podStartSLOduration=12.554709779 podStartE2EDuration="12.554709779s" podCreationTimestamp="2025-12-03 12:30:13 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:30:25.549978444 +0000 UTC m=+4216.138220208" watchObservedRunningTime="2025-12-03 12:30:25.554709779 +0000 UTC m=+4216.142951543" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:25.626616 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/cinder-scheduler-0" podStartSLOduration=14.62658622 podStartE2EDuration="14.62658622s" podCreationTimestamp="2025-12-03 12:30:11 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:30:25.62622569 +0000 UTC m=+4216.214467464" watchObservedRunningTime="2025-12-03 12:30:25.62658622 +0000 UTC m=+4216.214827984" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:25.652786 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/neutron-76bcd49fb6-54vvs" podStartSLOduration=13.652767846 podStartE2EDuration="13.652767846s" podCreationTimestamp="2025-12-03 12:30:12 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:30:25.648461714 +0000 UTC m=+4216.236703478" watchObservedRunningTime="2025-12-03 12:30:25.652767846 +0000 UTC m=+4216.241009610" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:26.122940 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/neutron-db474c7dd-tg7mq" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:26.124598 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/neutron-db474c7dd-tg7mq" podUID="886ea0a4-5096-4ea8-a74c-de243d1f4f40" containerName="neutron-httpd" probeResult="failure" output="Get \"https://10.217.0.200:9696/\": dial tcp 10.217.0.200:9696: connect: connection refused" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:26.367173 5052 kubelet.go:2542] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:30:27 crc kubenswrapper[5052]: E1203 12:30:26.400430 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of 96578ef7df608942efd7e88b79f20d9254a37ea8b6ab07704fddbcfab40445f1 is running failed: container process not found" containerID="96578ef7df608942efd7e88b79f20d9254a37ea8b6ab07704fddbcfab40445f1" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:30:27 crc kubenswrapper[5052]: E1203 12:30:26.400879 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of 96578ef7df608942efd7e88b79f20d9254a37ea8b6ab07704fddbcfab40445f1 is running failed: container process not found" containerID="96578ef7df608942efd7e88b79f20d9254a37ea8b6ab07704fddbcfab40445f1" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:30:27 crc kubenswrapper[5052]: E1203 12:30:26.401162 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of 96578ef7df608942efd7e88b79f20d9254a37ea8b6ab07704fddbcfab40445f1 is running failed: container process not found" containerID="96578ef7df608942efd7e88b79f20d9254a37ea8b6ab07704fddbcfab40445f1" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:30:27 crc kubenswrapper[5052]: E1203 12:30:26.401191 5052 prober.go:104] "Probe errored" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of 96578ef7df608942efd7e88b79f20d9254a37ea8b6ab07704fddbcfab40445f1 is running failed: container process not found" probeType="Liveness" pod="openstack-kuttl-tests/nova-cell0-conductor-0" podUID="e2718906-e52d-42fa-ad42-35626e99d461" containerName="nova-cell0-conductor-conductor" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:26.540790 5052 scope.go:117] "RemoveContainer" containerID="421784ccacc82cea351bc14db49b9cace93a8a873e7c71fae104a1ba2d00d40c" Dec 03 12:30:27 crc kubenswrapper[5052]: E1203 12:30:26.541183 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"nova-cell1-conductor-conductor\" with CrashLoopBackOff: \"back-off 10s restarting failed container=nova-cell1-conductor-conductor pod=nova-cell1-conductor-0_openstack-kuttl-tests(2dfbec8a-3a04-407b-8145-e12a3afee2e0)\"" pod="openstack-kuttl-tests/nova-cell1-conductor-0" podUID="2dfbec8a-3a04-407b-8145-e12a3afee2e0" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:27.131379 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:27.553415 5052 scope.go:117] "RemoveContainer" containerID="421784ccacc82cea351bc14db49b9cace93a8a873e7c71fae104a1ba2d00d40c" Dec 03 12:30:27 crc kubenswrapper[5052]: E1203 12:30:27.554114 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"nova-cell1-conductor-conductor\" with CrashLoopBackOff: \"back-off 10s restarting failed container=nova-cell1-conductor-conductor pod=nova-cell1-conductor-0_openstack-kuttl-tests(2dfbec8a-3a04-407b-8145-e12a3afee2e0)\"" pod="openstack-kuttl-tests/nova-cell1-conductor-0" podUID="2dfbec8a-3a04-407b-8145-e12a3afee2e0" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:27.556000 5052 generic.go:334] "Generic (PLEG): container finished" podID="e2718906-e52d-42fa-ad42-35626e99d461" containerID="96578ef7df608942efd7e88b79f20d9254a37ea8b6ab07704fddbcfab40445f1" exitCode=1 Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:27.556038 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-conductor-0" event={"ID":"e2718906-e52d-42fa-ad42-35626e99d461","Type":"ContainerDied","Data":"96578ef7df608942efd7e88b79f20d9254a37ea8b6ab07704fddbcfab40445f1"} Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:27.556062 5052 scope.go:117] "RemoveContainer" containerID="003ad3a32b6c302f684066c22cf6e0ad91b95336410c2156d1794d7ec8417301" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:27.556398 5052 scope.go:117] "RemoveContainer" containerID="96578ef7df608942efd7e88b79f20d9254a37ea8b6ab07704fddbcfab40445f1" Dec 03 12:30:27 crc kubenswrapper[5052]: E1203 12:30:27.556621 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"nova-cell0-conductor-conductor\" with CrashLoopBackOff: \"back-off 10s restarting failed container=nova-cell0-conductor-conductor pod=nova-cell0-conductor-0_openstack-kuttl-tests(e2718906-e52d-42fa-ad42-35626e99d461)\"" pod="openstack-kuttl-tests/nova-cell0-conductor-0" podUID="e2718906-e52d-42fa-ad42-35626e99d461" Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:27.559800 5052 generic.go:334] "Generic (PLEG): container finished" podID="12f39c79-eeb9-4cf4-8553-1a4d9a07ee82" containerID="165a4ba65f86633615c852ba29ceb303c3bbd0dbd696084cf0c3a26afe0a4424" exitCode=137 Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:27.559881 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-galera-0" event={"ID":"12f39c79-eeb9-4cf4-8553-1a4d9a07ee82","Type":"ContainerDied","Data":"165a4ba65f86633615c852ba29ceb303c3bbd0dbd696084cf0c3a26afe0a4424"} Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:27.562570 5052 generic.go:334] "Generic (PLEG): container finished" podID="6bfd5f51-8b37-4fea-8134-a2cdcd4f6049" containerID="0a0d6f3815eb929a36dbe3844272f6d6abfd6692349a5d9de8c42e6c899327d3" exitCode=0 Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:27.562640 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-76bcd49fb6-54vvs" event={"ID":"6bfd5f51-8b37-4fea-8134-a2cdcd4f6049","Type":"ContainerDied","Data":"0a0d6f3815eb929a36dbe3844272f6d6abfd6692349a5d9de8c42e6c899327d3"} Dec 03 12:30:27 crc kubenswrapper[5052]: I1203 12:30:27.807618 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/neutron-56f9c7c6b6-kvxpr" podUID="5f522d90-036e-4565-8293-97097912fbb1" containerName="neutron-httpd" probeResult="failure" output="Get \"https://10.217.0.203:9696/\": dial tcp 10.217.0.203:9696: connect: connection refused" Dec 03 12:30:28 crc kubenswrapper[5052]: I1203 12:30:28.050792 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/barbican-api-ff6b8c44b-7ndgm" podUID="48831cad-910f-46f1-aefe-7cc4cbfa984f" containerName="barbican-api" probeResult="failure" output="Get \"https://10.217.0.204:9311/healthcheck\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Dec 03 12:30:28 crc kubenswrapper[5052]: I1203 12:30:28.051327 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/barbican-api-ff6b8c44b-7ndgm" podUID="48831cad-910f-46f1-aefe-7cc4cbfa984f" containerName="barbican-api-log" probeResult="failure" output="Get \"https://10.217.0.204:9311/healthcheck\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Dec 03 12:30:28 crc kubenswrapper[5052]: I1203 12:30:28.248841 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-564bc88b6-bvll5" Dec 03 12:30:28 crc kubenswrapper[5052]: I1203 12:30:28.259586 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-worker-679bfdf7b7-tg6h7" Dec 03 12:30:28 crc kubenswrapper[5052]: I1203 12:30:28.276470 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:30:28 crc kubenswrapper[5052]: I1203 12:30:28.280734 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:30:28 crc kubenswrapper[5052]: I1203 12:30:28.314791 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"nova-novncproxy-tls-certs\" (UniqueName: \"kubernetes.io/secret/f4e095cd-36d8-4136-b262-f6d52717a05c-nova-novncproxy-tls-certs\") pod \"f4e095cd-36d8-4136-b262-f6d52717a05c\" (UID: \"f4e095cd-36d8-4136-b262-f6d52717a05c\") " Dec 03 12:30:28 crc kubenswrapper[5052]: I1203 12:30:28.314890 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/c0ce9ced-33b0-4879-81a4-b5c066ec0631-scripts\") pod \"c0ce9ced-33b0-4879-81a4-b5c066ec0631\" (UID: \"c0ce9ced-33b0-4879-81a4-b5c066ec0631\") " Dec 03 12:30:28 crc kubenswrapper[5052]: I1203 12:30:28.314930 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/c0ce9ced-33b0-4879-81a4-b5c066ec0631-public-tls-certs\") pod \"c0ce9ced-33b0-4879-81a4-b5c066ec0631\" (UID: \"c0ce9ced-33b0-4879-81a4-b5c066ec0631\") " Dec 03 12:30:28 crc kubenswrapper[5052]: I1203 12:30:28.315015 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/12f39c79-eeb9-4cf4-8553-1a4d9a07ee82-galera-tls-certs\") pod \"12f39c79-eeb9-4cf4-8553-1a4d9a07ee82\" (UID: \"12f39c79-eeb9-4cf4-8553-1a4d9a07ee82\") " Dec 03 12:30:28 crc kubenswrapper[5052]: I1203 12:30:28.315051 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/12f39c79-eeb9-4cf4-8553-1a4d9a07ee82-operator-scripts\") pod \"12f39c79-eeb9-4cf4-8553-1a4d9a07ee82\" (UID: \"12f39c79-eeb9-4cf4-8553-1a4d9a07ee82\") " Dec 03 12:30:28 crc kubenswrapper[5052]: I1203 12:30:28.315078 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lr65w\" (UniqueName: \"kubernetes.io/projected/f4e095cd-36d8-4136-b262-f6d52717a05c-kube-api-access-lr65w\") pod \"f4e095cd-36d8-4136-b262-f6d52717a05c\" (UID: \"f4e095cd-36d8-4136-b262-f6d52717a05c\") " Dec 03 12:30:28 crc kubenswrapper[5052]: I1203 12:30:28.315098 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"vencrypt-tls-certs\" (UniqueName: \"kubernetes.io/secret/f4e095cd-36d8-4136-b262-f6d52717a05c-vencrypt-tls-certs\") pod \"f4e095cd-36d8-4136-b262-f6d52717a05c\" (UID: \"f4e095cd-36d8-4136-b262-f6d52717a05c\") " Dec 03 12:30:28 crc kubenswrapper[5052]: I1203 12:30:28.315128 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/72a32b4d-d61d-43a5-a45d-c1fd4bad672d-config-data-custom\") pod \"72a32b4d-d61d-43a5-a45d-c1fd4bad672d\" (UID: \"72a32b4d-d61d-43a5-a45d-c1fd4bad672d\") " Dec 03 12:30:28 crc kubenswrapper[5052]: I1203 12:30:28.315163 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/72a32b4d-d61d-43a5-a45d-c1fd4bad672d-combined-ca-bundle\") pod \"72a32b4d-d61d-43a5-a45d-c1fd4bad672d\" (UID: \"72a32b4d-d61d-43a5-a45d-c1fd4bad672d\") " Dec 03 12:30:28 crc kubenswrapper[5052]: I1203 12:30:28.315186 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/c0ce9ced-33b0-4879-81a4-b5c066ec0631-credential-keys\") pod \"c0ce9ced-33b0-4879-81a4-b5c066ec0631\" (UID: \"c0ce9ced-33b0-4879-81a4-b5c066ec0631\") " Dec 03 12:30:28 crc kubenswrapper[5052]: I1203 12:30:28.315217 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c689n\" (UniqueName: \"kubernetes.io/projected/12f39c79-eeb9-4cf4-8553-1a4d9a07ee82-kube-api-access-c689n\") pod \"12f39c79-eeb9-4cf4-8553-1a4d9a07ee82\" (UID: \"12f39c79-eeb9-4cf4-8553-1a4d9a07ee82\") " Dec 03 12:30:28 crc kubenswrapper[5052]: I1203 12:30:28.315251 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f4e095cd-36d8-4136-b262-f6d52717a05c-combined-ca-bundle\") pod \"f4e095cd-36d8-4136-b262-f6d52717a05c\" (UID: \"f4e095cd-36d8-4136-b262-f6d52717a05c\") " Dec 03 12:30:28 crc kubenswrapper[5052]: I1203 12:30:28.315275 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/72a32b4d-d61d-43a5-a45d-c1fd4bad672d-config-data\") pod \"72a32b4d-d61d-43a5-a45d-c1fd4bad672d\" (UID: \"72a32b4d-d61d-43a5-a45d-c1fd4bad672d\") " Dec 03 12:30:28 crc kubenswrapper[5052]: I1203 12:30:28.315315 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/12f39c79-eeb9-4cf4-8553-1a4d9a07ee82-combined-ca-bundle\") pod \"12f39c79-eeb9-4cf4-8553-1a4d9a07ee82\" (UID: \"12f39c79-eeb9-4cf4-8553-1a4d9a07ee82\") " Dec 03 12:30:28 crc kubenswrapper[5052]: I1203 12:30:28.315342 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/72a32b4d-d61d-43a5-a45d-c1fd4bad672d-logs\") pod \"72a32b4d-d61d-43a5-a45d-c1fd4bad672d\" (UID: \"72a32b4d-d61d-43a5-a45d-c1fd4bad672d\") " Dec 03 12:30:28 crc kubenswrapper[5052]: I1203 12:30:28.315365 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d57ht\" (UniqueName: \"kubernetes.io/projected/c0ce9ced-33b0-4879-81a4-b5c066ec0631-kube-api-access-d57ht\") pod \"c0ce9ced-33b0-4879-81a4-b5c066ec0631\" (UID: \"c0ce9ced-33b0-4879-81a4-b5c066ec0631\") " Dec 03 12:30:28 crc kubenswrapper[5052]: I1203 12:30:28.315410 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dg6kv\" (UniqueName: \"kubernetes.io/projected/72a32b4d-d61d-43a5-a45d-c1fd4bad672d-kube-api-access-dg6kv\") pod \"72a32b4d-d61d-43a5-a45d-c1fd4bad672d\" (UID: \"72a32b4d-d61d-43a5-a45d-c1fd4bad672d\") " Dec 03 12:30:28 crc kubenswrapper[5052]: I1203 12:30:28.315440 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/12f39c79-eeb9-4cf4-8553-1a4d9a07ee82-config-data-generated\") pod \"12f39c79-eeb9-4cf4-8553-1a4d9a07ee82\" (UID: \"12f39c79-eeb9-4cf4-8553-1a4d9a07ee82\") " Dec 03 12:30:28 crc kubenswrapper[5052]: I1203 12:30:28.315464 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/f4e095cd-36d8-4136-b262-f6d52717a05c-config-data\") pod \"f4e095cd-36d8-4136-b262-f6d52717a05c\" (UID: \"f4e095cd-36d8-4136-b262-f6d52717a05c\") " Dec 03 12:30:28 crc kubenswrapper[5052]: I1203 12:30:28.315490 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"mysql-db\" (UniqueName: \"kubernetes.io/local-volume/local-storage02-crc\") pod \"12f39c79-eeb9-4cf4-8553-1a4d9a07ee82\" (UID: \"12f39c79-eeb9-4cf4-8553-1a4d9a07ee82\") " Dec 03 12:30:28 crc kubenswrapper[5052]: I1203 12:30:28.315544 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/c0ce9ced-33b0-4879-81a4-b5c066ec0631-config-data\") pod \"c0ce9ced-33b0-4879-81a4-b5c066ec0631\" (UID: \"c0ce9ced-33b0-4879-81a4-b5c066ec0631\") " Dec 03 12:30:28 crc kubenswrapper[5052]: I1203 12:30:28.315572 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/12f39c79-eeb9-4cf4-8553-1a4d9a07ee82-kolla-config\") pod \"12f39c79-eeb9-4cf4-8553-1a4d9a07ee82\" (UID: \"12f39c79-eeb9-4cf4-8553-1a4d9a07ee82\") " Dec 03 12:30:28 crc kubenswrapper[5052]: I1203 12:30:28.315593 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/c0ce9ced-33b0-4879-81a4-b5c066ec0631-fernet-keys\") pod \"c0ce9ced-33b0-4879-81a4-b5c066ec0631\" (UID: \"c0ce9ced-33b0-4879-81a4-b5c066ec0631\") " Dec 03 12:30:28 crc kubenswrapper[5052]: I1203 12:30:28.315621 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/12f39c79-eeb9-4cf4-8553-1a4d9a07ee82-config-data-default\") pod \"12f39c79-eeb9-4cf4-8553-1a4d9a07ee82\" (UID: \"12f39c79-eeb9-4cf4-8553-1a4d9a07ee82\") " Dec 03 12:30:28 crc kubenswrapper[5052]: I1203 12:30:28.315640 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c0ce9ced-33b0-4879-81a4-b5c066ec0631-combined-ca-bundle\") pod \"c0ce9ced-33b0-4879-81a4-b5c066ec0631\" (UID: \"c0ce9ced-33b0-4879-81a4-b5c066ec0631\") " Dec 03 12:30:28 crc kubenswrapper[5052]: I1203 12:30:28.315662 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/c0ce9ced-33b0-4879-81a4-b5c066ec0631-internal-tls-certs\") pod \"c0ce9ced-33b0-4879-81a4-b5c066ec0631\" (UID: \"c0ce9ced-33b0-4879-81a4-b5c066ec0631\") " Dec 03 12:30:28 crc kubenswrapper[5052]: I1203 12:30:28.317998 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/12f39c79-eeb9-4cf4-8553-1a4d9a07ee82-config-data-generated" (OuterVolumeSpecName: "config-data-generated") pod "12f39c79-eeb9-4cf4-8553-1a4d9a07ee82" (UID: "12f39c79-eeb9-4cf4-8553-1a4d9a07ee82"). InnerVolumeSpecName "config-data-generated". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:30:28 crc kubenswrapper[5052]: I1203 12:30:28.320694 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/12f39c79-eeb9-4cf4-8553-1a4d9a07ee82-kolla-config" (OuterVolumeSpecName: "kolla-config") pod "12f39c79-eeb9-4cf4-8553-1a4d9a07ee82" (UID: "12f39c79-eeb9-4cf4-8553-1a4d9a07ee82"). InnerVolumeSpecName "kolla-config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:30:28 crc kubenswrapper[5052]: I1203 12:30:28.329943 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/72a32b4d-d61d-43a5-a45d-c1fd4bad672d-logs" (OuterVolumeSpecName: "logs") pod "72a32b4d-d61d-43a5-a45d-c1fd4bad672d" (UID: "72a32b4d-d61d-43a5-a45d-c1fd4bad672d"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:30:28 crc kubenswrapper[5052]: I1203 12:30:28.333322 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/12f39c79-eeb9-4cf4-8553-1a4d9a07ee82-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "12f39c79-eeb9-4cf4-8553-1a4d9a07ee82" (UID: "12f39c79-eeb9-4cf4-8553-1a4d9a07ee82"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:30:28 crc kubenswrapper[5052]: I1203 12:30:28.338493 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/12f39c79-eeb9-4cf4-8553-1a4d9a07ee82-config-data-default" (OuterVolumeSpecName: "config-data-default") pod "12f39c79-eeb9-4cf4-8553-1a4d9a07ee82" (UID: "12f39c79-eeb9-4cf4-8553-1a4d9a07ee82"). InnerVolumeSpecName "config-data-default". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:30:28 crc kubenswrapper[5052]: I1203 12:30:28.343730 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/12f39c79-eeb9-4cf4-8553-1a4d9a07ee82-kube-api-access-c689n" (OuterVolumeSpecName: "kube-api-access-c689n") pod "12f39c79-eeb9-4cf4-8553-1a4d9a07ee82" (UID: "12f39c79-eeb9-4cf4-8553-1a4d9a07ee82"). InnerVolumeSpecName "kube-api-access-c689n". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:30:28 crc kubenswrapper[5052]: I1203 12:30:28.341824 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c0ce9ced-33b0-4879-81a4-b5c066ec0631-scripts" (OuterVolumeSpecName: "scripts") pod "c0ce9ced-33b0-4879-81a4-b5c066ec0631" (UID: "c0ce9ced-33b0-4879-81a4-b5c066ec0631"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:28 crc kubenswrapper[5052]: I1203 12:30:28.351885 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/72a32b4d-d61d-43a5-a45d-c1fd4bad672d-kube-api-access-dg6kv" (OuterVolumeSpecName: "kube-api-access-dg6kv") pod "72a32b4d-d61d-43a5-a45d-c1fd4bad672d" (UID: "72a32b4d-d61d-43a5-a45d-c1fd4bad672d"). InnerVolumeSpecName "kube-api-access-dg6kv". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:30:28 crc kubenswrapper[5052]: I1203 12:30:28.352801 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c0ce9ced-33b0-4879-81a4-b5c066ec0631-credential-keys" (OuterVolumeSpecName: "credential-keys") pod "c0ce9ced-33b0-4879-81a4-b5c066ec0631" (UID: "c0ce9ced-33b0-4879-81a4-b5c066ec0631"). InnerVolumeSpecName "credential-keys". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:28 crc kubenswrapper[5052]: I1203 12:30:28.359008 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f4e095cd-36d8-4136-b262-f6d52717a05c-kube-api-access-lr65w" (OuterVolumeSpecName: "kube-api-access-lr65w") pod "f4e095cd-36d8-4136-b262-f6d52717a05c" (UID: "f4e095cd-36d8-4136-b262-f6d52717a05c"). InnerVolumeSpecName "kube-api-access-lr65w". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:30:28 crc kubenswrapper[5052]: I1203 12:30:28.369135 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c0ce9ced-33b0-4879-81a4-b5c066ec0631-kube-api-access-d57ht" (OuterVolumeSpecName: "kube-api-access-d57ht") pod "c0ce9ced-33b0-4879-81a4-b5c066ec0631" (UID: "c0ce9ced-33b0-4879-81a4-b5c066ec0631"). InnerVolumeSpecName "kube-api-access-d57ht". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:30:28 crc kubenswrapper[5052]: I1203 12:30:28.377679 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/72a32b4d-d61d-43a5-a45d-c1fd4bad672d-config-data-custom" (OuterVolumeSpecName: "config-data-custom") pod "72a32b4d-d61d-43a5-a45d-c1fd4bad672d" (UID: "72a32b4d-d61d-43a5-a45d-c1fd4bad672d"). InnerVolumeSpecName "config-data-custom". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:28 crc kubenswrapper[5052]: I1203 12:30:28.381048 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c0ce9ced-33b0-4879-81a4-b5c066ec0631-fernet-keys" (OuterVolumeSpecName: "fernet-keys") pod "c0ce9ced-33b0-4879-81a4-b5c066ec0631" (UID: "c0ce9ced-33b0-4879-81a4-b5c066ec0631"). InnerVolumeSpecName "fernet-keys". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:28 crc kubenswrapper[5052]: I1203 12:30:28.399810 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage02-crc" (OuterVolumeSpecName: "mysql-db") pod "12f39c79-eeb9-4cf4-8553-1a4d9a07ee82" (UID: "12f39c79-eeb9-4cf4-8553-1a4d9a07ee82"). InnerVolumeSpecName "local-storage02-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 12:30:28 crc kubenswrapper[5052]: I1203 12:30:28.418060 5052 reconciler_common.go:293] "Volume detached for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/12f39c79-eeb9-4cf4-8553-1a4d9a07ee82-kolla-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:28 crc kubenswrapper[5052]: I1203 12:30:28.418093 5052 reconciler_common.go:293] "Volume detached for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/c0ce9ced-33b0-4879-81a4-b5c066ec0631-fernet-keys\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:28 crc kubenswrapper[5052]: I1203 12:30:28.418102 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/12f39c79-eeb9-4cf4-8553-1a4d9a07ee82-config-data-default\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:28 crc kubenswrapper[5052]: I1203 12:30:28.418110 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/c0ce9ced-33b0-4879-81a4-b5c066ec0631-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:28 crc kubenswrapper[5052]: I1203 12:30:28.418120 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/12f39c79-eeb9-4cf4-8553-1a4d9a07ee82-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:28 crc kubenswrapper[5052]: I1203 12:30:28.418130 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-lr65w\" (UniqueName: \"kubernetes.io/projected/f4e095cd-36d8-4136-b262-f6d52717a05c-kube-api-access-lr65w\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:28 crc kubenswrapper[5052]: I1203 12:30:28.418140 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/72a32b4d-d61d-43a5-a45d-c1fd4bad672d-config-data-custom\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:28 crc kubenswrapper[5052]: I1203 12:30:28.418149 5052 reconciler_common.go:293] "Volume detached for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/c0ce9ced-33b0-4879-81a4-b5c066ec0631-credential-keys\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:28 crc kubenswrapper[5052]: I1203 12:30:28.418160 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-c689n\" (UniqueName: \"kubernetes.io/projected/12f39c79-eeb9-4cf4-8553-1a4d9a07ee82-kube-api-access-c689n\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:28 crc kubenswrapper[5052]: I1203 12:30:28.418169 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/72a32b4d-d61d-43a5-a45d-c1fd4bad672d-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:28 crc kubenswrapper[5052]: I1203 12:30:28.418177 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-d57ht\" (UniqueName: \"kubernetes.io/projected/c0ce9ced-33b0-4879-81a4-b5c066ec0631-kube-api-access-d57ht\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:28 crc kubenswrapper[5052]: I1203 12:30:28.418185 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-dg6kv\" (UniqueName: \"kubernetes.io/projected/72a32b4d-d61d-43a5-a45d-c1fd4bad672d-kube-api-access-dg6kv\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:28 crc kubenswrapper[5052]: I1203 12:30:28.418193 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/12f39c79-eeb9-4cf4-8553-1a4d9a07ee82-config-data-generated\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:28 crc kubenswrapper[5052]: I1203 12:30:28.418217 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage02-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage02-crc\") on node \"crc\" " Dec 03 12:30:28 crc kubenswrapper[5052]: I1203 12:30:28.420860 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/72a32b4d-d61d-43a5-a45d-c1fd4bad672d-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "72a32b4d-d61d-43a5-a45d-c1fd4bad672d" (UID: "72a32b4d-d61d-43a5-a45d-c1fd4bad672d"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:28 crc kubenswrapper[5052]: I1203 12:30:28.433130 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/12f39c79-eeb9-4cf4-8553-1a4d9a07ee82-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "12f39c79-eeb9-4cf4-8553-1a4d9a07ee82" (UID: "12f39c79-eeb9-4cf4-8553-1a4d9a07ee82"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:28 crc kubenswrapper[5052]: I1203 12:30:28.445262 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c0ce9ced-33b0-4879-81a4-b5c066ec0631-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "c0ce9ced-33b0-4879-81a4-b5c066ec0631" (UID: "c0ce9ced-33b0-4879-81a4-b5c066ec0631"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:28 crc kubenswrapper[5052]: I1203 12:30:28.494070 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c0ce9ced-33b0-4879-81a4-b5c066ec0631-config-data" (OuterVolumeSpecName: "config-data") pod "c0ce9ced-33b0-4879-81a4-b5c066ec0631" (UID: "c0ce9ced-33b0-4879-81a4-b5c066ec0631"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:28 crc kubenswrapper[5052]: I1203 12:30:28.514717 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage02-crc" (UniqueName: "kubernetes.io/local-volume/local-storage02-crc") on node "crc" Dec 03 12:30:28 crc kubenswrapper[5052]: I1203 12:30:28.516687 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f4e095cd-36d8-4136-b262-f6d52717a05c-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "f4e095cd-36d8-4136-b262-f6d52717a05c" (UID: "f4e095cd-36d8-4136-b262-f6d52717a05c"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:28 crc kubenswrapper[5052]: I1203 12:30:28.520403 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/72a32b4d-d61d-43a5-a45d-c1fd4bad672d-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:28 crc kubenswrapper[5052]: I1203 12:30:28.520434 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f4e095cd-36d8-4136-b262-f6d52717a05c-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:28 crc kubenswrapper[5052]: I1203 12:30:28.520443 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/12f39c79-eeb9-4cf4-8553-1a4d9a07ee82-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:28 crc kubenswrapper[5052]: I1203 12:30:28.520454 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage02-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage02-crc\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:28 crc kubenswrapper[5052]: I1203 12:30:28.520467 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/c0ce9ced-33b0-4879-81a4-b5c066ec0631-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:28 crc kubenswrapper[5052]: I1203 12:30:28.520475 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/c0ce9ced-33b0-4879-81a4-b5c066ec0631-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:28 crc kubenswrapper[5052]: I1203 12:30:28.522421 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c0ce9ced-33b0-4879-81a4-b5c066ec0631-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "c0ce9ced-33b0-4879-81a4-b5c066ec0631" (UID: "c0ce9ced-33b0-4879-81a4-b5c066ec0631"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:28 crc kubenswrapper[5052]: I1203 12:30:28.529623 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f4e095cd-36d8-4136-b262-f6d52717a05c-config-data" (OuterVolumeSpecName: "config-data") pod "f4e095cd-36d8-4136-b262-f6d52717a05c" (UID: "f4e095cd-36d8-4136-b262-f6d52717a05c"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:28 crc kubenswrapper[5052]: I1203 12:30:28.529743 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c0ce9ced-33b0-4879-81a4-b5c066ec0631-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "c0ce9ced-33b0-4879-81a4-b5c066ec0631" (UID: "c0ce9ced-33b0-4879-81a4-b5c066ec0631"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:28 crc kubenswrapper[5052]: I1203 12:30:28.532813 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f4e095cd-36d8-4136-b262-f6d52717a05c-vencrypt-tls-certs" (OuterVolumeSpecName: "vencrypt-tls-certs") pod "f4e095cd-36d8-4136-b262-f6d52717a05c" (UID: "f4e095cd-36d8-4136-b262-f6d52717a05c"). InnerVolumeSpecName "vencrypt-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:28 crc kubenswrapper[5052]: I1203 12:30:28.556487 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/12f39c79-eeb9-4cf4-8553-1a4d9a07ee82-galera-tls-certs" (OuterVolumeSpecName: "galera-tls-certs") pod "12f39c79-eeb9-4cf4-8553-1a4d9a07ee82" (UID: "12f39c79-eeb9-4cf4-8553-1a4d9a07ee82"). InnerVolumeSpecName "galera-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:28 crc kubenswrapper[5052]: I1203 12:30:28.568765 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f4e095cd-36d8-4136-b262-f6d52717a05c-nova-novncproxy-tls-certs" (OuterVolumeSpecName: "nova-novncproxy-tls-certs") pod "f4e095cd-36d8-4136-b262-f6d52717a05c" (UID: "f4e095cd-36d8-4136-b262-f6d52717a05c"). InnerVolumeSpecName "nova-novncproxy-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:28 crc kubenswrapper[5052]: I1203 12:30:28.587049 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-galera-0" event={"ID":"12f39c79-eeb9-4cf4-8553-1a4d9a07ee82","Type":"ContainerDied","Data":"066ac9e3b2ce853ae0441d00f7b5eee75bdffe353d6f1fb0ffad1c5f36411d87"} Dec 03 12:30:28 crc kubenswrapper[5052]: I1203 12:30:28.587106 5052 scope.go:117] "RemoveContainer" containerID="165a4ba65f86633615c852ba29ceb303c3bbd0dbd696084cf0c3a26afe0a4424" Dec 03 12:30:28 crc kubenswrapper[5052]: I1203 12:30:28.587216 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:30:28 crc kubenswrapper[5052]: I1203 12:30:28.590393 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/72a32b4d-d61d-43a5-a45d-c1fd4bad672d-config-data" (OuterVolumeSpecName: "config-data") pod "72a32b4d-d61d-43a5-a45d-c1fd4bad672d" (UID: "72a32b4d-d61d-43a5-a45d-c1fd4bad672d"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:28 crc kubenswrapper[5052]: I1203 12:30:28.592071 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-564bc88b6-bvll5" Dec 03 12:30:28 crc kubenswrapper[5052]: I1203 12:30:28.592732 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-564bc88b6-bvll5" event={"ID":"c0ce9ced-33b0-4879-81a4-b5c066ec0631","Type":"ContainerDied","Data":"7d60e6875dab88bfe727e3ecc5b73bec4dc81727c206ef38a830168d386edffa"} Dec 03 12:30:28 crc kubenswrapper[5052]: I1203 12:30:28.594075 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:30:28 crc kubenswrapper[5052]: I1203 12:30:28.594202 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" event={"ID":"f4e095cd-36d8-4136-b262-f6d52717a05c","Type":"ContainerDied","Data":"1e8fea595fbe0a6787690b161ee2b73ba9ae2e861acdb4233b860c488db1f389"} Dec 03 12:30:28 crc kubenswrapper[5052]: I1203 12:30:28.601231 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"a24b0824-26d0-4b81-b6cb-d617c36d45ab","Type":"ContainerStarted","Data":"e765959f449b54c23591a65f292f2f7a5099d71fa4f0ea45d8c2f901334b7206"} Dec 03 12:30:28 crc kubenswrapper[5052]: I1203 12:30:28.603003 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-worker-679bfdf7b7-tg6h7" event={"ID":"72a32b4d-d61d-43a5-a45d-c1fd4bad672d","Type":"ContainerDied","Data":"822106e884f3485cc561862c636fdd5bf103caa3793d11830c8ecdaa4d4430cc"} Dec 03 12:30:28 crc kubenswrapper[5052]: I1203 12:30:28.603068 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-worker-679bfdf7b7-tg6h7" Dec 03 12:30:28 crc kubenswrapper[5052]: I1203 12:30:28.628396 5052 reconciler_common.go:293] "Volume detached for volume \"nova-novncproxy-tls-certs\" (UniqueName: \"kubernetes.io/secret/f4e095cd-36d8-4136-b262-f6d52717a05c-nova-novncproxy-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:28 crc kubenswrapper[5052]: I1203 12:30:28.628429 5052 reconciler_common.go:293] "Volume detached for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/12f39c79-eeb9-4cf4-8553-1a4d9a07ee82-galera-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:28 crc kubenswrapper[5052]: I1203 12:30:28.628440 5052 reconciler_common.go:293] "Volume detached for volume \"vencrypt-tls-certs\" (UniqueName: \"kubernetes.io/secret/f4e095cd-36d8-4136-b262-f6d52717a05c-vencrypt-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:28 crc kubenswrapper[5052]: I1203 12:30:28.628448 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/72a32b4d-d61d-43a5-a45d-c1fd4bad672d-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:28 crc kubenswrapper[5052]: I1203 12:30:28.628459 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/f4e095cd-36d8-4136-b262-f6d52717a05c-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:28 crc kubenswrapper[5052]: I1203 12:30:28.628467 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c0ce9ced-33b0-4879-81a4-b5c066ec0631-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:28 crc kubenswrapper[5052]: I1203 12:30:28.628476 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/c0ce9ced-33b0-4879-81a4-b5c066ec0631-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:28 crc kubenswrapper[5052]: I1203 12:30:28.688433 5052 scope.go:117] "RemoveContainer" containerID="dcde059b067d1c9d50353c5be63ee2f4f97d8343ecfb247c42768f1aba1f469c" Dec 03 12:30:28 crc kubenswrapper[5052]: I1203 12:30:28.690495 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 12:30:28 crc kubenswrapper[5052]: I1203 12:30:28.731567 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 12:30:28 crc kubenswrapper[5052]: I1203 12:30:28.760475 5052 scope.go:117] "RemoveContainer" containerID="589675564f1f842a7ea700f8f790adcdebd983109491fa331d941686f179e466" Dec 03 12:30:28 crc kubenswrapper[5052]: I1203 12:30:28.794885 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/openstack-galera-0"] Dec 03 12:30:28 crc kubenswrapper[5052]: I1203 12:30:28.814471 5052 scope.go:117] "RemoveContainer" containerID="1e7025513d6b144fd4de246b9cff9dd24d587dcce1c3318a8c75d6b9165b00ff" Dec 03 12:30:28 crc kubenswrapper[5052]: I1203 12:30:28.859454 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 12:30:28 crc kubenswrapper[5052]: E1203 12:30:28.860086 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="f4e095cd-36d8-4136-b262-f6d52717a05c" containerName="nova-cell1-novncproxy-novncproxy" Dec 03 12:30:28 crc kubenswrapper[5052]: I1203 12:30:28.860146 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="f4e095cd-36d8-4136-b262-f6d52717a05c" containerName="nova-cell1-novncproxy-novncproxy" Dec 03 12:30:28 crc kubenswrapper[5052]: E1203 12:30:28.860235 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="12f39c79-eeb9-4cf4-8553-1a4d9a07ee82" containerName="mysql-bootstrap" Dec 03 12:30:28 crc kubenswrapper[5052]: I1203 12:30:28.860288 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="12f39c79-eeb9-4cf4-8553-1a4d9a07ee82" containerName="mysql-bootstrap" Dec 03 12:30:28 crc kubenswrapper[5052]: E1203 12:30:28.860357 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="72a32b4d-d61d-43a5-a45d-c1fd4bad672d" containerName="barbican-worker-log" Dec 03 12:30:28 crc kubenswrapper[5052]: I1203 12:30:28.860423 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="72a32b4d-d61d-43a5-a45d-c1fd4bad672d" containerName="barbican-worker-log" Dec 03 12:30:28 crc kubenswrapper[5052]: E1203 12:30:28.860557 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="72a32b4d-d61d-43a5-a45d-c1fd4bad672d" containerName="barbican-worker" Dec 03 12:30:28 crc kubenswrapper[5052]: I1203 12:30:28.860611 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="72a32b4d-d61d-43a5-a45d-c1fd4bad672d" containerName="barbican-worker" Dec 03 12:30:28 crc kubenswrapper[5052]: E1203 12:30:28.860679 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="c0ce9ced-33b0-4879-81a4-b5c066ec0631" containerName="keystone-api" Dec 03 12:30:28 crc kubenswrapper[5052]: I1203 12:30:28.860727 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="c0ce9ced-33b0-4879-81a4-b5c066ec0631" containerName="keystone-api" Dec 03 12:30:28 crc kubenswrapper[5052]: E1203 12:30:28.860784 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="12f39c79-eeb9-4cf4-8553-1a4d9a07ee82" containerName="galera" Dec 03 12:30:28 crc kubenswrapper[5052]: I1203 12:30:28.860830 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="12f39c79-eeb9-4cf4-8553-1a4d9a07ee82" containerName="galera" Dec 03 12:30:28 crc kubenswrapper[5052]: I1203 12:30:28.861088 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="f4e095cd-36d8-4136-b262-f6d52717a05c" containerName="nova-cell1-novncproxy-novncproxy" Dec 03 12:30:28 crc kubenswrapper[5052]: I1203 12:30:28.861148 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="72a32b4d-d61d-43a5-a45d-c1fd4bad672d" containerName="barbican-worker" Dec 03 12:30:28 crc kubenswrapper[5052]: I1203 12:30:28.861197 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="72a32b4d-d61d-43a5-a45d-c1fd4bad672d" containerName="barbican-worker-log" Dec 03 12:30:28 crc kubenswrapper[5052]: I1203 12:30:28.861267 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="c0ce9ced-33b0-4879-81a4-b5c066ec0631" containerName="keystone-api" Dec 03 12:30:28 crc kubenswrapper[5052]: I1203 12:30:28.861342 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="12f39c79-eeb9-4cf4-8553-1a4d9a07ee82" containerName="galera" Dec 03 12:30:28 crc kubenswrapper[5052]: I1203 12:30:28.862198 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:30:28 crc kubenswrapper[5052]: I1203 12:30:28.864919 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-nova-novncproxy-cell1-vencrypt" Dec 03 12:30:28 crc kubenswrapper[5052]: I1203 12:30:28.865320 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-nova-novncproxy-cell1-public-svc" Dec 03 12:30:28 crc kubenswrapper[5052]: I1203 12:30:28.867503 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell1-novncproxy-config-data" Dec 03 12:30:28 crc kubenswrapper[5052]: I1203 12:30:28.873140 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/openstack-galera-0"] Dec 03 12:30:28 crc kubenswrapper[5052]: I1203 12:30:28.899246 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 12:30:28 crc kubenswrapper[5052]: I1203 12:30:28.902835 5052 scope.go:117] "RemoveContainer" containerID="6d2a1c1187cb71ec6764a5792108b52b3d8a71f435dceee72f6a125558106e90" Dec 03 12:30:28 crc kubenswrapper[5052]: I1203 12:30:28.913770 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/openstack-galera-0"] Dec 03 12:30:28 crc kubenswrapper[5052]: I1203 12:30:28.915531 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:30:28 crc kubenswrapper[5052]: I1203 12:30:28.917261 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"galera-openstack-dockercfg-w7gzp" Dec 03 12:30:28 crc kubenswrapper[5052]: I1203 12:30:28.917979 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"openstack-config-data" Dec 03 12:30:28 crc kubenswrapper[5052]: I1203 12:30:28.918192 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"openstack-scripts" Dec 03 12:30:28 crc kubenswrapper[5052]: I1203 12:30:28.918248 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-galera-openstack-svc" Dec 03 12:30:28 crc kubenswrapper[5052]: I1203 12:30:28.925969 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/keystone-564bc88b6-bvll5"] Dec 03 12:30:28 crc kubenswrapper[5052]: I1203 12:30:28.936720 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/keystone-564bc88b6-bvll5"] Dec 03 12:30:28 crc kubenswrapper[5052]: I1203 12:30:28.946760 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/openstack-galera-0"] Dec 03 12:30:28 crc kubenswrapper[5052]: I1203 12:30:28.956465 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-api-68fc8f9776-n9gml"] Dec 03 12:30:28 crc kubenswrapper[5052]: I1203 12:30:28.965552 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-worker-679bfdf7b7-tg6h7"] Dec 03 12:30:28 crc kubenswrapper[5052]: I1203 12:30:28.974264 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/barbican-worker-679bfdf7b7-tg6h7"] Dec 03 12:30:28 crc kubenswrapper[5052]: I1203 12:30:28.974660 5052 scope.go:117] "RemoveContainer" containerID="902a5085d0a5d4b6d6bd379e7d79548b8cf29e5be3d818b61c582ef847c19f19" Dec 03 12:30:28 crc kubenswrapper[5052]: I1203 12:30:28.983736 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-api-64d69c4db4-hkscs"] Dec 03 12:30:28 crc kubenswrapper[5052]: I1203 12:30:28.993038 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-sb-0"] Dec 03 12:30:29 crc kubenswrapper[5052]: I1203 12:30:29.004347 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:30:29 crc kubenswrapper[5052]: I1203 12:30:29.013998 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ovn-northd-0"] Dec 03 12:30:29 crc kubenswrapper[5052]: I1203 12:30:29.026689 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-nb-0"] Dec 03 12:30:29 crc kubenswrapper[5052]: I1203 12:30:29.038178 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/openstack-cell1-galera-0"] Dec 03 12:30:29 crc kubenswrapper[5052]: I1203 12:30:29.039940 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"vencrypt-tls-certs\" (UniqueName: \"kubernetes.io/secret/d09b9817-6ea5-4328-9027-3311b7782157-vencrypt-tls-certs\") pod \"nova-cell1-novncproxy-0\" (UID: \"d09b9817-6ea5-4328-9027-3311b7782157\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:30:29 crc kubenswrapper[5052]: I1203 12:30:29.040155 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/6938204c-1fe8-4f03-ac16-ec9e19e090f8-config-data-default\") pod \"openstack-galera-0\" (UID: \"6938204c-1fe8-4f03-ac16-ec9e19e090f8\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:30:29 crc kubenswrapper[5052]: I1203 12:30:29.040197 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6938204c-1fe8-4f03-ac16-ec9e19e090f8-combined-ca-bundle\") pod \"openstack-galera-0\" (UID: \"6938204c-1fe8-4f03-ac16-ec9e19e090f8\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:30:29 crc kubenswrapper[5052]: I1203 12:30:29.040297 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/6938204c-1fe8-4f03-ac16-ec9e19e090f8-kolla-config\") pod \"openstack-galera-0\" (UID: \"6938204c-1fe8-4f03-ac16-ec9e19e090f8\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:30:29 crc kubenswrapper[5052]: I1203 12:30:29.040337 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/6938204c-1fe8-4f03-ac16-ec9e19e090f8-operator-scripts\") pod \"openstack-galera-0\" (UID: \"6938204c-1fe8-4f03-ac16-ec9e19e090f8\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:30:29 crc kubenswrapper[5052]: I1203 12:30:29.040400 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nova-novncproxy-tls-certs\" (UniqueName: \"kubernetes.io/secret/d09b9817-6ea5-4328-9027-3311b7782157-nova-novncproxy-tls-certs\") pod \"nova-cell1-novncproxy-0\" (UID: \"d09b9817-6ea5-4328-9027-3311b7782157\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:30:29 crc kubenswrapper[5052]: I1203 12:30:29.040446 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/6938204c-1fe8-4f03-ac16-ec9e19e090f8-galera-tls-certs\") pod \"openstack-galera-0\" (UID: \"6938204c-1fe8-4f03-ac16-ec9e19e090f8\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:30:29 crc kubenswrapper[5052]: I1203 12:30:29.040499 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rlwzr\" (UniqueName: \"kubernetes.io/projected/6938204c-1fe8-4f03-ac16-ec9e19e090f8-kube-api-access-rlwzr\") pod \"openstack-galera-0\" (UID: \"6938204c-1fe8-4f03-ac16-ec9e19e090f8\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:30:29 crc kubenswrapper[5052]: I1203 12:30:29.040602 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/d09b9817-6ea5-4328-9027-3311b7782157-config-data\") pod \"nova-cell1-novncproxy-0\" (UID: \"d09b9817-6ea5-4328-9027-3311b7782157\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:30:29 crc kubenswrapper[5052]: I1203 12:30:29.040808 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hhjkq\" (UniqueName: \"kubernetes.io/projected/d09b9817-6ea5-4328-9027-3311b7782157-kube-api-access-hhjkq\") pod \"nova-cell1-novncproxy-0\" (UID: \"d09b9817-6ea5-4328-9027-3311b7782157\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:30:29 crc kubenswrapper[5052]: I1203 12:30:29.040906 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/6938204c-1fe8-4f03-ac16-ec9e19e090f8-config-data-generated\") pod \"openstack-galera-0\" (UID: \"6938204c-1fe8-4f03-ac16-ec9e19e090f8\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:30:29 crc kubenswrapper[5052]: I1203 12:30:29.041554 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage02-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage02-crc\") pod \"openstack-galera-0\" (UID: \"6938204c-1fe8-4f03-ac16-ec9e19e090f8\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:30:29 crc kubenswrapper[5052]: I1203 12:30:29.041588 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d09b9817-6ea5-4328-9027-3311b7782157-combined-ca-bundle\") pod \"nova-cell1-novncproxy-0\" (UID: \"d09b9817-6ea5-4328-9027-3311b7782157\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:30:29 crc kubenswrapper[5052]: I1203 12:30:29.144060 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-hhjkq\" (UniqueName: \"kubernetes.io/projected/d09b9817-6ea5-4328-9027-3311b7782157-kube-api-access-hhjkq\") pod \"nova-cell1-novncproxy-0\" (UID: \"d09b9817-6ea5-4328-9027-3311b7782157\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:30:29 crc kubenswrapper[5052]: I1203 12:30:29.144402 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/6938204c-1fe8-4f03-ac16-ec9e19e090f8-config-data-generated\") pod \"openstack-galera-0\" (UID: \"6938204c-1fe8-4f03-ac16-ec9e19e090f8\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:30:29 crc kubenswrapper[5052]: I1203 12:30:29.144450 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage02-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage02-crc\") pod \"openstack-galera-0\" (UID: \"6938204c-1fe8-4f03-ac16-ec9e19e090f8\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:30:29 crc kubenswrapper[5052]: I1203 12:30:29.144486 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d09b9817-6ea5-4328-9027-3311b7782157-combined-ca-bundle\") pod \"nova-cell1-novncproxy-0\" (UID: \"d09b9817-6ea5-4328-9027-3311b7782157\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:30:29 crc kubenswrapper[5052]: I1203 12:30:29.144580 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"vencrypt-tls-certs\" (UniqueName: \"kubernetes.io/secret/d09b9817-6ea5-4328-9027-3311b7782157-vencrypt-tls-certs\") pod \"nova-cell1-novncproxy-0\" (UID: \"d09b9817-6ea5-4328-9027-3311b7782157\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:30:29 crc kubenswrapper[5052]: I1203 12:30:29.145125 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/6938204c-1fe8-4f03-ac16-ec9e19e090f8-config-data-generated\") pod \"openstack-galera-0\" (UID: \"6938204c-1fe8-4f03-ac16-ec9e19e090f8\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:30:29 crc kubenswrapper[5052]: I1203 12:30:29.145352 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage02-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage02-crc\") pod \"openstack-galera-0\" (UID: \"6938204c-1fe8-4f03-ac16-ec9e19e090f8\") device mount path \"/mnt/openstack/pv02\"" pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:30:29 crc kubenswrapper[5052]: I1203 12:30:29.148406 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/6938204c-1fe8-4f03-ac16-ec9e19e090f8-config-data-default\") pod \"openstack-galera-0\" (UID: \"6938204c-1fe8-4f03-ac16-ec9e19e090f8\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:30:29 crc kubenswrapper[5052]: I1203 12:30:29.148452 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6938204c-1fe8-4f03-ac16-ec9e19e090f8-combined-ca-bundle\") pod \"openstack-galera-0\" (UID: \"6938204c-1fe8-4f03-ac16-ec9e19e090f8\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:30:29 crc kubenswrapper[5052]: I1203 12:30:29.148546 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/6938204c-1fe8-4f03-ac16-ec9e19e090f8-kolla-config\") pod \"openstack-galera-0\" (UID: \"6938204c-1fe8-4f03-ac16-ec9e19e090f8\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:30:29 crc kubenswrapper[5052]: I1203 12:30:29.148583 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/6938204c-1fe8-4f03-ac16-ec9e19e090f8-operator-scripts\") pod \"openstack-galera-0\" (UID: \"6938204c-1fe8-4f03-ac16-ec9e19e090f8\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:30:29 crc kubenswrapper[5052]: I1203 12:30:29.148700 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"nova-novncproxy-tls-certs\" (UniqueName: \"kubernetes.io/secret/d09b9817-6ea5-4328-9027-3311b7782157-nova-novncproxy-tls-certs\") pod \"nova-cell1-novncproxy-0\" (UID: \"d09b9817-6ea5-4328-9027-3311b7782157\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:30:29 crc kubenswrapper[5052]: I1203 12:30:29.148788 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/6938204c-1fe8-4f03-ac16-ec9e19e090f8-galera-tls-certs\") pod \"openstack-galera-0\" (UID: \"6938204c-1fe8-4f03-ac16-ec9e19e090f8\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:30:29 crc kubenswrapper[5052]: I1203 12:30:29.148831 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-rlwzr\" (UniqueName: \"kubernetes.io/projected/6938204c-1fe8-4f03-ac16-ec9e19e090f8-kube-api-access-rlwzr\") pod \"openstack-galera-0\" (UID: \"6938204c-1fe8-4f03-ac16-ec9e19e090f8\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:30:29 crc kubenswrapper[5052]: I1203 12:30:29.148875 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/d09b9817-6ea5-4328-9027-3311b7782157-config-data\") pod \"nova-cell1-novncproxy-0\" (UID: \"d09b9817-6ea5-4328-9027-3311b7782157\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:30:29 crc kubenswrapper[5052]: I1203 12:30:29.149372 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/6938204c-1fe8-4f03-ac16-ec9e19e090f8-config-data-default\") pod \"openstack-galera-0\" (UID: \"6938204c-1fe8-4f03-ac16-ec9e19e090f8\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:30:29 crc kubenswrapper[5052]: I1203 12:30:29.151329 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/6938204c-1fe8-4f03-ac16-ec9e19e090f8-operator-scripts\") pod \"openstack-galera-0\" (UID: \"6938204c-1fe8-4f03-ac16-ec9e19e090f8\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:30:29 crc kubenswrapper[5052]: I1203 12:30:29.151429 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/6938204c-1fe8-4f03-ac16-ec9e19e090f8-kolla-config\") pod \"openstack-galera-0\" (UID: \"6938204c-1fe8-4f03-ac16-ec9e19e090f8\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:30:29 crc kubenswrapper[5052]: I1203 12:30:29.152236 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"vencrypt-tls-certs\" (UniqueName: \"kubernetes.io/secret/d09b9817-6ea5-4328-9027-3311b7782157-vencrypt-tls-certs\") pod \"nova-cell1-novncproxy-0\" (UID: \"d09b9817-6ea5-4328-9027-3311b7782157\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:30:29 crc kubenswrapper[5052]: I1203 12:30:29.152684 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d09b9817-6ea5-4328-9027-3311b7782157-combined-ca-bundle\") pod \"nova-cell1-novncproxy-0\" (UID: \"d09b9817-6ea5-4328-9027-3311b7782157\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:30:29 crc kubenswrapper[5052]: I1203 12:30:29.154499 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"nova-novncproxy-tls-certs\" (UniqueName: \"kubernetes.io/secret/d09b9817-6ea5-4328-9027-3311b7782157-nova-novncproxy-tls-certs\") pod \"nova-cell1-novncproxy-0\" (UID: \"d09b9817-6ea5-4328-9027-3311b7782157\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:30:29 crc kubenswrapper[5052]: I1203 12:30:29.155891 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/6938204c-1fe8-4f03-ac16-ec9e19e090f8-galera-tls-certs\") pod \"openstack-galera-0\" (UID: \"6938204c-1fe8-4f03-ac16-ec9e19e090f8\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:30:29 crc kubenswrapper[5052]: I1203 12:30:29.160561 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6938204c-1fe8-4f03-ac16-ec9e19e090f8-combined-ca-bundle\") pod \"openstack-galera-0\" (UID: \"6938204c-1fe8-4f03-ac16-ec9e19e090f8\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:30:29 crc kubenswrapper[5052]: I1203 12:30:29.183860 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-rlwzr\" (UniqueName: \"kubernetes.io/projected/6938204c-1fe8-4f03-ac16-ec9e19e090f8-kube-api-access-rlwzr\") pod \"openstack-galera-0\" (UID: \"6938204c-1fe8-4f03-ac16-ec9e19e090f8\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:30:29 crc kubenswrapper[5052]: I1203 12:30:29.184262 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-hhjkq\" (UniqueName: \"kubernetes.io/projected/d09b9817-6ea5-4328-9027-3311b7782157-kube-api-access-hhjkq\") pod \"nova-cell1-novncproxy-0\" (UID: \"d09b9817-6ea5-4328-9027-3311b7782157\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:30:29 crc kubenswrapper[5052]: I1203 12:30:29.184624 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/d09b9817-6ea5-4328-9027-3311b7782157-config-data\") pod \"nova-cell1-novncproxy-0\" (UID: \"d09b9817-6ea5-4328-9027-3311b7782157\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:30:29 crc kubenswrapper[5052]: I1203 12:30:29.222277 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage02-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage02-crc\") pod \"openstack-galera-0\" (UID: \"6938204c-1fe8-4f03-ac16-ec9e19e090f8\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:30:29 crc kubenswrapper[5052]: I1203 12:30:29.231085 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:30:29 crc kubenswrapper[5052]: I1203 12:30:29.400238 5052 kubelet.go:2542] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:30:29 crc kubenswrapper[5052]: I1203 12:30:29.401233 5052 scope.go:117] "RemoveContainer" containerID="96578ef7df608942efd7e88b79f20d9254a37ea8b6ab07704fddbcfab40445f1" Dec 03 12:30:29 crc kubenswrapper[5052]: E1203 12:30:29.401559 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"nova-cell0-conductor-conductor\" with CrashLoopBackOff: \"back-off 10s restarting failed container=nova-cell0-conductor-conductor pod=nova-cell0-conductor-0_openstack-kuttl-tests(e2718906-e52d-42fa-ad42-35626e99d461)\"" pod="openstack-kuttl-tests/nova-cell0-conductor-0" podUID="e2718906-e52d-42fa-ad42-35626e99d461" Dec 03 12:30:29 crc kubenswrapper[5052]: I1203 12:30:29.486755 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:30:29 crc kubenswrapper[5052]: I1203 12:30:29.616021 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"a65428bc-9326-4ace-a985-9f796b951304","Type":"ContainerStarted","Data":"14589c054b59a52bc21c00362eb9bafdebaddd153d062b3e5f87e8a4845e9a19"} Dec 03 12:30:29 crc kubenswrapper[5052]: I1203 12:30:29.620096 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-64d69c4db4-hkscs" event={"ID":"ff58ae3c-d75e-4038-be70-00f88178a18f","Type":"ContainerStarted","Data":"638897c2f0ffd643c8e7fa20a4d2aa99ceee7bfc03c3aa9760b40e97843cf3c8"} Dec 03 12:30:29 crc kubenswrapper[5052]: I1203 12:30:29.620137 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-64d69c4db4-hkscs" event={"ID":"ff58ae3c-d75e-4038-be70-00f88178a18f","Type":"ContainerStarted","Data":"85335d4cabbcc2d67511e1cbdbc856b320084a0d652a2d294770e9e25e81348f"} Dec 03 12:30:29 crc kubenswrapper[5052]: I1203 12:30:29.623334 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovn-northd-0" event={"ID":"a9f65397-a08a-4798-8532-646209c5d090","Type":"ContainerStarted","Data":"217fb246797b6f91c28f31f22b498303e560c0706448ba85948bf67b547612e3"} Dec 03 12:30:29 crc kubenswrapper[5052]: I1203 12:30:29.623590 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovn-northd-0" event={"ID":"a9f65397-a08a-4798-8532-646209c5d090","Type":"ContainerStarted","Data":"9a2f2f114e462f7d8496b44d7ae330b86bd3edb78f74e61fc5c568bae55d88e3"} Dec 03 12:30:29 crc kubenswrapper[5052]: I1203 12:30:29.624956 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-sb-0" event={"ID":"3fec3449-9615-4377-a600-69d816ec5416","Type":"ContainerStarted","Data":"19c04c4116df77c2accd0a6c361be38550f970c0c0f8032ebf2cd11805800978"} Dec 03 12:30:29 crc kubenswrapper[5052]: I1203 12:30:29.625010 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-sb-0" event={"ID":"3fec3449-9615-4377-a600-69d816ec5416","Type":"ContainerStarted","Data":"c575e780203e439005e13481039ff4a9ef25ce1a73d62c9d8c0ce5ed6f8bd46b"} Dec 03 12:30:29 crc kubenswrapper[5052]: I1203 12:30:29.627559 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-nb-0" event={"ID":"dccae7c1-009e-4b3b-b66a-ecbf6e53489a","Type":"ContainerStarted","Data":"da4594e855a05fe7faafd60c45e490f353a2622ba132821e7f56859f0a614ca4"} Dec 03 12:30:29 crc kubenswrapper[5052]: I1203 12:30:29.627608 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-nb-0" event={"ID":"dccae7c1-009e-4b3b-b66a-ecbf6e53489a","Type":"ContainerStarted","Data":"a0693dce3e63878da0e8a79833bdbb9e6496c63685edbeabed749e7bf6336079"} Dec 03 12:30:29 crc kubenswrapper[5052]: I1203 12:30:29.631600 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-cell1-galera-0" event={"ID":"aa55d0c8-56de-4edd-92b3-9252f169fed8","Type":"ContainerStarted","Data":"0a11b0f2d7f2318c289c08d3984727c92ca9c2e1900b8e4ebba02d7834859031"} Dec 03 12:30:29 crc kubenswrapper[5052]: I1203 12:30:29.631646 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-cell1-galera-0" event={"ID":"aa55d0c8-56de-4edd-92b3-9252f169fed8","Type":"ContainerStarted","Data":"e0fc2dcb7d273c04fa4e698e02a3ae70eb1854c856f4a52ccb677573d0cc28af"} Dec 03 12:30:29 crc kubenswrapper[5052]: I1203 12:30:29.633312 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-68fc8f9776-n9gml" event={"ID":"82f142ab-5be9-4727-b3c5-102d6f2401fb","Type":"ContainerStarted","Data":"4b1853107a72a495dcaac5832fb35d2f64fd74e87c388c0c4a4f6c88669f2664"} Dec 03 12:30:29 crc kubenswrapper[5052]: I1203 12:30:29.688545 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="12f39c79-eeb9-4cf4-8553-1a4d9a07ee82" path="/var/lib/kubelet/pods/12f39c79-eeb9-4cf4-8553-1a4d9a07ee82/volumes" Dec 03 12:30:29 crc kubenswrapper[5052]: I1203 12:30:29.689501 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="72a32b4d-d61d-43a5-a45d-c1fd4bad672d" path="/var/lib/kubelet/pods/72a32b4d-d61d-43a5-a45d-c1fd4bad672d/volumes" Dec 03 12:30:29 crc kubenswrapper[5052]: I1203 12:30:29.690177 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c0ce9ced-33b0-4879-81a4-b5c066ec0631" path="/var/lib/kubelet/pods/c0ce9ced-33b0-4879-81a4-b5c066ec0631/volumes" Dec 03 12:30:29 crc kubenswrapper[5052]: I1203 12:30:29.691277 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f4e095cd-36d8-4136-b262-f6d52717a05c" path="/var/lib/kubelet/pods/f4e095cd-36d8-4136-b262-f6d52717a05c/volumes" Dec 03 12:30:30 crc kubenswrapper[5052]: I1203 12:30:30.467825 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 12:30:30 crc kubenswrapper[5052]: W1203 12:30:30.472386 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd09b9817_6ea5_4328_9027_3311b7782157.slice/crio-f4e3e1af651c18cabeaf7d4b2c90fc4171ebe8b4d4dfa7e41b60c69c0093d8e8 WatchSource:0}: Error finding container f4e3e1af651c18cabeaf7d4b2c90fc4171ebe8b4d4dfa7e41b60c69c0093d8e8: Status 404 returned error can't find the container with id f4e3e1af651c18cabeaf7d4b2c90fc4171ebe8b4d4dfa7e41b60c69c0093d8e8 Dec 03 12:30:30 crc kubenswrapper[5052]: I1203 12:30:30.573592 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/openstack-galera-0"] Dec 03 12:30:30 crc kubenswrapper[5052]: W1203 12:30:30.581026 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6938204c_1fe8_4f03_ac16_ec9e19e090f8.slice/crio-b0a0323137cad0ba98d2d8e9fb51b9c46221cbc417d036f304ad0e9d99d1a0ad WatchSource:0}: Error finding container b0a0323137cad0ba98d2d8e9fb51b9c46221cbc417d036f304ad0e9d99d1a0ad: Status 404 returned error can't find the container with id b0a0323137cad0ba98d2d8e9fb51b9c46221cbc417d036f304ad0e9d99d1a0ad Dec 03 12:30:30 crc kubenswrapper[5052]: I1203 12:30:30.664554 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"a65428bc-9326-4ace-a985-9f796b951304","Type":"ContainerStarted","Data":"caa8dd02dbab945ca9d553d926ff4896db88a3b315e5384ca071cd388a2f2766"} Dec 03 12:30:30 crc kubenswrapper[5052]: I1203 12:30:30.670424 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-galera-0" event={"ID":"6938204c-1fe8-4f03-ac16-ec9e19e090f8","Type":"ContainerStarted","Data":"b0a0323137cad0ba98d2d8e9fb51b9c46221cbc417d036f304ad0e9d99d1a0ad"} Dec 03 12:30:30 crc kubenswrapper[5052]: I1203 12:30:30.683467 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" event={"ID":"d09b9817-6ea5-4328-9027-3311b7782157","Type":"ContainerStarted","Data":"f4e3e1af651c18cabeaf7d4b2c90fc4171ebe8b4d4dfa7e41b60c69c0093d8e8"} Dec 03 12:30:30 crc kubenswrapper[5052]: I1203 12:30:30.693329 5052 generic.go:334] "Generic (PLEG): container finished" podID="aa55d0c8-56de-4edd-92b3-9252f169fed8" containerID="0a11b0f2d7f2318c289c08d3984727c92ca9c2e1900b8e4ebba02d7834859031" exitCode=0 Dec 03 12:30:30 crc kubenswrapper[5052]: I1203 12:30:30.694083 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-cell1-galera-0" event={"ID":"aa55d0c8-56de-4edd-92b3-9252f169fed8","Type":"ContainerDied","Data":"0a11b0f2d7f2318c289c08d3984727c92ca9c2e1900b8e4ebba02d7834859031"} Dec 03 12:30:30 crc kubenswrapper[5052]: I1203 12:30:30.705552 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-68fc8f9776-n9gml" event={"ID":"82f142ab-5be9-4727-b3c5-102d6f2401fb","Type":"ContainerStarted","Data":"ca30ccff84d103c471a54af8538ca8e1d53df3754e5f8a3e9c4ab0869c3a23d6"} Dec 03 12:30:31 crc kubenswrapper[5052]: I1203 12:30:31.716797 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-nb-0" event={"ID":"dccae7c1-009e-4b3b-b66a-ecbf6e53489a","Type":"ContainerStarted","Data":"3b528f3a5ecffc67ae3f8e59c8348c99e1798d2e9c1b8ecf277ece85a83c0756"} Dec 03 12:30:31 crc kubenswrapper[5052]: I1203 12:30:31.719727 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"a24b0824-26d0-4b81-b6cb-d617c36d45ab","Type":"ContainerStarted","Data":"c477cd69a766929b221a6f0497205e288c5ded0f70fe9daadbd6669d7a943603"} Dec 03 12:30:31 crc kubenswrapper[5052]: I1203 12:30:31.721663 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-64d69c4db4-hkscs" event={"ID":"ff58ae3c-d75e-4038-be70-00f88178a18f","Type":"ContainerStarted","Data":"1165299b4299bb42e6bbea4345f1d0f3318f8b00847584b340a3eb83aecf87aa"} Dec 03 12:30:31 crc kubenswrapper[5052]: I1203 12:30:31.724367 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" event={"ID":"d09b9817-6ea5-4328-9027-3311b7782157","Type":"ContainerStarted","Data":"7e1821a6c6bd165b882e82fb2451aca61d98f3c04110b410f0845d91b6abfa0b"} Dec 03 12:30:31 crc kubenswrapper[5052]: I1203 12:30:31.727397 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-68fc8f9776-n9gml" event={"ID":"82f142ab-5be9-4727-b3c5-102d6f2401fb","Type":"ContainerStarted","Data":"74fe4216c2c90357dad59af2f1e20e908e8583d49e0f2a8d15cd86ca1fc755d6"} Dec 03 12:30:31 crc kubenswrapper[5052]: I1203 12:30:31.730576 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovn-northd-0" event={"ID":"a9f65397-a08a-4798-8532-646209c5d090","Type":"ContainerStarted","Data":"ea36b7c04ef3193b039e70bd5df384774204953e99d07610a4a191cab306432f"} Dec 03 12:30:31 crc kubenswrapper[5052]: I1203 12:30:31.734704 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-sb-0" event={"ID":"3fec3449-9615-4377-a600-69d816ec5416","Type":"ContainerStarted","Data":"21da3337b09e6b326697aef155af4bcd6a984d400eaed99ee26b1a3a045fba72"} Dec 03 12:30:31 crc kubenswrapper[5052]: I1203 12:30:31.737111 5052 generic.go:334] "Generic (PLEG): container finished" podID="4314928d-a43f-4b38-9c72-63977e59345b" containerID="55d873486c41fcc800af3aac0c5bdbf6786c715209a5758840bdce2e1c3fba02" exitCode=0 Dec 03 12:30:31 crc kubenswrapper[5052]: I1203 12:30:31.737145 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"4314928d-a43f-4b38-9c72-63977e59345b","Type":"ContainerDied","Data":"55d873486c41fcc800af3aac0c5bdbf6786c715209a5758840bdce2e1c3fba02"} Dec 03 12:30:32 crc kubenswrapper[5052]: I1203 12:30:32.172677 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/cinder-scheduler-0" podUID="bb1723c7-4e41-4b0e-b6b8-f8024dafdaee" containerName="cinder-scheduler" probeResult="failure" output="Get \"http://10.217.0.222:8080/\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Dec 03 12:30:32 crc kubenswrapper[5052]: I1203 12:30:32.526330 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:30:32 crc kubenswrapper[5052]: I1203 12:30:32.638111 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4314928d-a43f-4b38-9c72-63977e59345b-combined-ca-bundle\") pod \"4314928d-a43f-4b38-9c72-63977e59345b\" (UID: \"4314928d-a43f-4b38-9c72-63977e59345b\") " Dec 03 12:30:32 crc kubenswrapper[5052]: I1203 12:30:32.638167 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/4314928d-a43f-4b38-9c72-63977e59345b-public-tls-certs\") pod \"4314928d-a43f-4b38-9c72-63977e59345b\" (UID: \"4314928d-a43f-4b38-9c72-63977e59345b\") " Dec 03 12:30:32 crc kubenswrapper[5052]: I1203 12:30:32.638330 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/4314928d-a43f-4b38-9c72-63977e59345b-internal-tls-certs\") pod \"4314928d-a43f-4b38-9c72-63977e59345b\" (UID: \"4314928d-a43f-4b38-9c72-63977e59345b\") " Dec 03 12:30:32 crc kubenswrapper[5052]: I1203 12:30:32.638415 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/4314928d-a43f-4b38-9c72-63977e59345b-config-data\") pod \"4314928d-a43f-4b38-9c72-63977e59345b\" (UID: \"4314928d-a43f-4b38-9c72-63977e59345b\") " Dec 03 12:30:32 crc kubenswrapper[5052]: I1203 12:30:32.638555 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pfvmv\" (UniqueName: \"kubernetes.io/projected/4314928d-a43f-4b38-9c72-63977e59345b-kube-api-access-pfvmv\") pod \"4314928d-a43f-4b38-9c72-63977e59345b\" (UID: \"4314928d-a43f-4b38-9c72-63977e59345b\") " Dec 03 12:30:32 crc kubenswrapper[5052]: I1203 12:30:32.638598 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/4314928d-a43f-4b38-9c72-63977e59345b-logs\") pod \"4314928d-a43f-4b38-9c72-63977e59345b\" (UID: \"4314928d-a43f-4b38-9c72-63977e59345b\") " Dec 03 12:30:32 crc kubenswrapper[5052]: I1203 12:30:32.639424 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4314928d-a43f-4b38-9c72-63977e59345b-logs" (OuterVolumeSpecName: "logs") pod "4314928d-a43f-4b38-9c72-63977e59345b" (UID: "4314928d-a43f-4b38-9c72-63977e59345b"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:30:32 crc kubenswrapper[5052]: I1203 12:30:32.652771 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4314928d-a43f-4b38-9c72-63977e59345b-kube-api-access-pfvmv" (OuterVolumeSpecName: "kube-api-access-pfvmv") pod "4314928d-a43f-4b38-9c72-63977e59345b" (UID: "4314928d-a43f-4b38-9c72-63977e59345b"). InnerVolumeSpecName "kube-api-access-pfvmv". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:30:32 crc kubenswrapper[5052]: I1203 12:30:32.682528 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4314928d-a43f-4b38-9c72-63977e59345b-config-data" (OuterVolumeSpecName: "config-data") pod "4314928d-a43f-4b38-9c72-63977e59345b" (UID: "4314928d-a43f-4b38-9c72-63977e59345b"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:32 crc kubenswrapper[5052]: I1203 12:30:32.686312 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4314928d-a43f-4b38-9c72-63977e59345b-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "4314928d-a43f-4b38-9c72-63977e59345b" (UID: "4314928d-a43f-4b38-9c72-63977e59345b"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:32 crc kubenswrapper[5052]: I1203 12:30:32.709968 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4314928d-a43f-4b38-9c72-63977e59345b-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "4314928d-a43f-4b38-9c72-63977e59345b" (UID: "4314928d-a43f-4b38-9c72-63977e59345b"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:32 crc kubenswrapper[5052]: I1203 12:30:32.726676 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4314928d-a43f-4b38-9c72-63977e59345b-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "4314928d-a43f-4b38-9c72-63977e59345b" (UID: "4314928d-a43f-4b38-9c72-63977e59345b"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:32 crc kubenswrapper[5052]: I1203 12:30:32.742017 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/4314928d-a43f-4b38-9c72-63977e59345b-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:32 crc kubenswrapper[5052]: I1203 12:30:32.742045 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-pfvmv\" (UniqueName: \"kubernetes.io/projected/4314928d-a43f-4b38-9c72-63977e59345b-kube-api-access-pfvmv\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:32 crc kubenswrapper[5052]: I1203 12:30:32.742053 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/4314928d-a43f-4b38-9c72-63977e59345b-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:32 crc kubenswrapper[5052]: I1203 12:30:32.742064 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4314928d-a43f-4b38-9c72-63977e59345b-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:32 crc kubenswrapper[5052]: I1203 12:30:32.742072 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/4314928d-a43f-4b38-9c72-63977e59345b-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:32 crc kubenswrapper[5052]: I1203 12:30:32.742081 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/4314928d-a43f-4b38-9c72-63977e59345b-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:32 crc kubenswrapper[5052]: I1203 12:30:32.752120 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"4314928d-a43f-4b38-9c72-63977e59345b","Type":"ContainerDied","Data":"296ee69d2ec970208a8577c0e6c144ec5511a61fc7b242cf5d81b2653243ce31"} Dec 03 12:30:32 crc kubenswrapper[5052]: I1203 12:30:32.752148 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:30:32 crc kubenswrapper[5052]: I1203 12:30:32.752173 5052 scope.go:117] "RemoveContainer" containerID="55d873486c41fcc800af3aac0c5bdbf6786c715209a5758840bdce2e1c3fba02" Dec 03 12:30:32 crc kubenswrapper[5052]: I1203 12:30:32.754994 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-cell1-galera-0" event={"ID":"aa55d0c8-56de-4edd-92b3-9252f169fed8","Type":"ContainerStarted","Data":"8bf7dd5e7641b5ac8d567b37be404e10979d45b84ac534f9d893fee92ab89c5e"} Dec 03 12:30:32 crc kubenswrapper[5052]: I1203 12:30:32.760324 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"a65428bc-9326-4ace-a985-9f796b951304","Type":"ContainerStarted","Data":"ddde80cbe1c1205957f18817d4db06fabccc08242a25c2d74bcd5a4c61d8bd5d"} Dec 03 12:30:32 crc kubenswrapper[5052]: I1203 12:30:32.761655 5052 generic.go:334] "Generic (PLEG): container finished" podID="6938204c-1fe8-4f03-ac16-ec9e19e090f8" containerID="894ea95c17ab09554e637952aa3fd8f7d1432238350697dba009b9bb58995df9" exitCode=0 Dec 03 12:30:32 crc kubenswrapper[5052]: I1203 12:30:32.761720 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-galera-0" event={"ID":"6938204c-1fe8-4f03-ac16-ec9e19e090f8","Type":"ContainerDied","Data":"894ea95c17ab09554e637952aa3fd8f7d1432238350697dba009b9bb58995df9"} Dec 03 12:30:32 crc kubenswrapper[5052]: I1203 12:30:32.762399 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/barbican-api-64d69c4db4-hkscs" podUID="ff58ae3c-d75e-4038-be70-00f88178a18f" containerName="barbican-api-log" containerID="cri-o://638897c2f0ffd643c8e7fa20a4d2aa99ceee7bfc03c3aa9760b40e97843cf3c8" gracePeriod=30 Dec 03 12:30:32 crc kubenswrapper[5052]: I1203 12:30:32.762428 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/barbican-api-64d69c4db4-hkscs" podUID="ff58ae3c-d75e-4038-be70-00f88178a18f" containerName="barbican-api" containerID="cri-o://1165299b4299bb42e6bbea4345f1d0f3318f8b00847584b340a3eb83aecf87aa" gracePeriod=30 Dec 03 12:30:32 crc kubenswrapper[5052]: I1203 12:30:32.762449 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/barbican-api-68fc8f9776-n9gml" Dec 03 12:30:32 crc kubenswrapper[5052]: I1203 12:30:32.793193 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/openstack-cell1-galera-0" podStartSLOduration=15.793168699 podStartE2EDuration="15.793168699s" podCreationTimestamp="2025-12-03 12:30:17 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:30:32.785035147 +0000 UTC m=+4223.373276931" watchObservedRunningTime="2025-12-03 12:30:32.793168699 +0000 UTC m=+4223.381410463" Dec 03 12:30:32 crc kubenswrapper[5052]: I1203 12:30:32.823500 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/ovsdbserver-nb-0" podStartSLOduration=14.823483044 podStartE2EDuration="14.823483044s" podCreationTimestamp="2025-12-03 12:30:18 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:30:32.809990729 +0000 UTC m=+4223.398232513" watchObservedRunningTime="2025-12-03 12:30:32.823483044 +0000 UTC m=+4223.411724808" Dec 03 12:30:32 crc kubenswrapper[5052]: I1203 12:30:32.825661 5052 scope.go:117] "RemoveContainer" containerID="50954953a47c0484d7617bce920a6b97d031a9d9d0a476b236d1e1bdff5f0cfc" Dec 03 12:30:32 crc kubenswrapper[5052]: I1203 12:30:32.833875 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/barbican-api-68fc8f9776-n9gml" podStartSLOduration=11.83385501 podStartE2EDuration="11.83385501s" podCreationTimestamp="2025-12-03 12:30:21 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:30:32.830251367 +0000 UTC m=+4223.418493131" watchObservedRunningTime="2025-12-03 12:30:32.83385501 +0000 UTC m=+4223.422096774" Dec 03 12:30:32 crc kubenswrapper[5052]: I1203 12:30:32.930385 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/ovn-northd-0" podStartSLOduration=17.930365144 podStartE2EDuration="17.930365144s" podCreationTimestamp="2025-12-03 12:30:15 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:30:32.872098251 +0000 UTC m=+4223.460340015" watchObservedRunningTime="2025-12-03 12:30:32.930365144 +0000 UTC m=+4223.518606918" Dec 03 12:30:32 crc kubenswrapper[5052]: I1203 12:30:32.933171 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/barbican-api-64d69c4db4-hkscs" podStartSLOduration=11.933163564000001 podStartE2EDuration="11.933163564s" podCreationTimestamp="2025-12-03 12:30:21 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:30:32.901307335 +0000 UTC m=+4223.489549099" watchObservedRunningTime="2025-12-03 12:30:32.933163564 +0000 UTC m=+4223.521405328" Dec 03 12:30:32 crc kubenswrapper[5052]: I1203 12:30:32.972285 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/glance-default-external-api-0" podStartSLOduration=12.97226817 podStartE2EDuration="12.97226817s" podCreationTimestamp="2025-12-03 12:30:20 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:30:32.959853246 +0000 UTC m=+4223.548095010" watchObservedRunningTime="2025-12-03 12:30:32.97226817 +0000 UTC m=+4223.560509934" Dec 03 12:30:33 crc kubenswrapper[5052]: I1203 12:30:33.011967 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" podStartSLOduration=5.011947913 podStartE2EDuration="5.011947913s" podCreationTimestamp="2025-12-03 12:30:28 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:30:33.011628844 +0000 UTC m=+4223.599870608" watchObservedRunningTime="2025-12-03 12:30:33.011947913 +0000 UTC m=+4223.600189677" Dec 03 12:30:33 crc kubenswrapper[5052]: I1203 12:30:33.022054 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/ovsdbserver-sb-0" podStartSLOduration=15.022032721 podStartE2EDuration="15.022032721s" podCreationTimestamp="2025-12-03 12:30:18 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:30:32.992906339 +0000 UTC m=+4223.581148103" watchObservedRunningTime="2025-12-03 12:30:33.022032721 +0000 UTC m=+4223.610274485" Dec 03 12:30:33 crc kubenswrapper[5052]: I1203 12:30:33.035061 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:30:33 crc kubenswrapper[5052]: I1203 12:30:33.045734 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:30:33 crc kubenswrapper[5052]: I1203 12:30:33.052129 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/barbican-api-ff6b8c44b-7ndgm" podUID="48831cad-910f-46f1-aefe-7cc4cbfa984f" containerName="barbican-api-log" probeResult="failure" output="Get \"https://10.217.0.204:9311/healthcheck\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Dec 03 12:30:33 crc kubenswrapper[5052]: I1203 12:30:33.054916 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/barbican-api-ff6b8c44b-7ndgm" podUID="48831cad-910f-46f1-aefe-7cc4cbfa984f" containerName="barbican-api" probeResult="failure" output="Get \"https://10.217.0.204:9311/healthcheck\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Dec 03 12:30:33 crc kubenswrapper[5052]: I1203 12:30:33.062450 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:30:33 crc kubenswrapper[5052]: E1203 12:30:33.063013 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="4314928d-a43f-4b38-9c72-63977e59345b" containerName="nova-api-log" Dec 03 12:30:33 crc kubenswrapper[5052]: I1203 12:30:33.063031 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="4314928d-a43f-4b38-9c72-63977e59345b" containerName="nova-api-log" Dec 03 12:30:33 crc kubenswrapper[5052]: E1203 12:30:33.063043 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="4314928d-a43f-4b38-9c72-63977e59345b" containerName="nova-api-api" Dec 03 12:30:33 crc kubenswrapper[5052]: I1203 12:30:33.063051 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="4314928d-a43f-4b38-9c72-63977e59345b" containerName="nova-api-api" Dec 03 12:30:33 crc kubenswrapper[5052]: I1203 12:30:33.063237 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="4314928d-a43f-4b38-9c72-63977e59345b" containerName="nova-api-api" Dec 03 12:30:33 crc kubenswrapper[5052]: I1203 12:30:33.063270 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="4314928d-a43f-4b38-9c72-63977e59345b" containerName="nova-api-log" Dec 03 12:30:33 crc kubenswrapper[5052]: I1203 12:30:33.064436 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:30:33 crc kubenswrapper[5052]: I1203 12:30:33.068978 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-nova-public-svc" Dec 03 12:30:33 crc kubenswrapper[5052]: I1203 12:30:33.069128 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-nova-internal-svc" Dec 03 12:30:33 crc kubenswrapper[5052]: I1203 12:30:33.069305 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-api-config-data" Dec 03 12:30:33 crc kubenswrapper[5052]: I1203 12:30:33.090794 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:30:33 crc kubenswrapper[5052]: I1203 12:30:33.156973 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/7cbe1c2c-bc04-4232-9901-bc9d134df3d6-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"7cbe1c2c-bc04-4232-9901-bc9d134df3d6\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:30:33 crc kubenswrapper[5052]: I1203 12:30:33.157027 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/7cbe1c2c-bc04-4232-9901-bc9d134df3d6-internal-tls-certs\") pod \"nova-api-0\" (UID: \"7cbe1c2c-bc04-4232-9901-bc9d134df3d6\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:30:33 crc kubenswrapper[5052]: I1203 12:30:33.157113 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/7cbe1c2c-bc04-4232-9901-bc9d134df3d6-logs\") pod \"nova-api-0\" (UID: \"7cbe1c2c-bc04-4232-9901-bc9d134df3d6\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:30:33 crc kubenswrapper[5052]: I1203 12:30:33.157139 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9wpdp\" (UniqueName: \"kubernetes.io/projected/7cbe1c2c-bc04-4232-9901-bc9d134df3d6-kube-api-access-9wpdp\") pod \"nova-api-0\" (UID: \"7cbe1c2c-bc04-4232-9901-bc9d134df3d6\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:30:33 crc kubenswrapper[5052]: I1203 12:30:33.157160 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/7cbe1c2c-bc04-4232-9901-bc9d134df3d6-public-tls-certs\") pod \"nova-api-0\" (UID: \"7cbe1c2c-bc04-4232-9901-bc9d134df3d6\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:30:33 crc kubenswrapper[5052]: I1203 12:30:33.157179 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/7cbe1c2c-bc04-4232-9901-bc9d134df3d6-config-data\") pod \"nova-api-0\" (UID: \"7cbe1c2c-bc04-4232-9901-bc9d134df3d6\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:30:33 crc kubenswrapper[5052]: I1203 12:30:33.259031 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/7cbe1c2c-bc04-4232-9901-bc9d134df3d6-logs\") pod \"nova-api-0\" (UID: \"7cbe1c2c-bc04-4232-9901-bc9d134df3d6\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:30:33 crc kubenswrapper[5052]: I1203 12:30:33.259124 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-9wpdp\" (UniqueName: \"kubernetes.io/projected/7cbe1c2c-bc04-4232-9901-bc9d134df3d6-kube-api-access-9wpdp\") pod \"nova-api-0\" (UID: \"7cbe1c2c-bc04-4232-9901-bc9d134df3d6\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:30:33 crc kubenswrapper[5052]: I1203 12:30:33.259161 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/7cbe1c2c-bc04-4232-9901-bc9d134df3d6-public-tls-certs\") pod \"nova-api-0\" (UID: \"7cbe1c2c-bc04-4232-9901-bc9d134df3d6\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:30:33 crc kubenswrapper[5052]: I1203 12:30:33.259193 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/7cbe1c2c-bc04-4232-9901-bc9d134df3d6-config-data\") pod \"nova-api-0\" (UID: \"7cbe1c2c-bc04-4232-9901-bc9d134df3d6\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:30:33 crc kubenswrapper[5052]: I1203 12:30:33.260288 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/7cbe1c2c-bc04-4232-9901-bc9d134df3d6-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"7cbe1c2c-bc04-4232-9901-bc9d134df3d6\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:30:33 crc kubenswrapper[5052]: I1203 12:30:33.260319 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/7cbe1c2c-bc04-4232-9901-bc9d134df3d6-internal-tls-certs\") pod \"nova-api-0\" (UID: \"7cbe1c2c-bc04-4232-9901-bc9d134df3d6\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:30:33 crc kubenswrapper[5052]: I1203 12:30:33.259661 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/7cbe1c2c-bc04-4232-9901-bc9d134df3d6-logs\") pod \"nova-api-0\" (UID: \"7cbe1c2c-bc04-4232-9901-bc9d134df3d6\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:30:33 crc kubenswrapper[5052]: I1203 12:30:33.264074 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/7cbe1c2c-bc04-4232-9901-bc9d134df3d6-internal-tls-certs\") pod \"nova-api-0\" (UID: \"7cbe1c2c-bc04-4232-9901-bc9d134df3d6\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:30:33 crc kubenswrapper[5052]: I1203 12:30:33.264579 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/7cbe1c2c-bc04-4232-9901-bc9d134df3d6-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"7cbe1c2c-bc04-4232-9901-bc9d134df3d6\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:30:33 crc kubenswrapper[5052]: I1203 12:30:33.264838 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/7cbe1c2c-bc04-4232-9901-bc9d134df3d6-public-tls-certs\") pod \"nova-api-0\" (UID: \"7cbe1c2c-bc04-4232-9901-bc9d134df3d6\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:30:33 crc kubenswrapper[5052]: I1203 12:30:33.265266 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/7cbe1c2c-bc04-4232-9901-bc9d134df3d6-config-data\") pod \"nova-api-0\" (UID: \"7cbe1c2c-bc04-4232-9901-bc9d134df3d6\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:30:33 crc kubenswrapper[5052]: I1203 12:30:33.278546 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-9wpdp\" (UniqueName: \"kubernetes.io/projected/7cbe1c2c-bc04-4232-9901-bc9d134df3d6-kube-api-access-9wpdp\") pod \"nova-api-0\" (UID: \"7cbe1c2c-bc04-4232-9901-bc9d134df3d6\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:30:33 crc kubenswrapper[5052]: I1203 12:30:33.394910 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:30:33 crc kubenswrapper[5052]: I1203 12:30:33.540765 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:30:33 crc kubenswrapper[5052]: I1203 12:30:33.623486 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:30:33 crc kubenswrapper[5052]: I1203 12:30:33.623568 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:30:33 crc kubenswrapper[5052]: I1203 12:30:33.691874 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4314928d-a43f-4b38-9c72-63977e59345b" path="/var/lib/kubelet/pods/4314928d-a43f-4b38-9c72-63977e59345b/volumes" Dec 03 12:30:33 crc kubenswrapper[5052]: I1203 12:30:33.692862 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:30:33 crc kubenswrapper[5052]: I1203 12:30:33.810859 5052 generic.go:334] "Generic (PLEG): container finished" podID="aed41b09-319e-466c-9eb9-38208e1880a3" containerID="0a78aa778c6e6e1f2c1d7cc5f57fa0dc6dd213f54da357a449dda9efb5e3bc63" exitCode=137 Dec 03 12:30:33 crc kubenswrapper[5052]: I1203 12:30:33.810933 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-keystone-listener-77b6bbbdcb-4jlsc" event={"ID":"aed41b09-319e-466c-9eb9-38208e1880a3","Type":"ContainerDied","Data":"0a78aa778c6e6e1f2c1d7cc5f57fa0dc6dd213f54da357a449dda9efb5e3bc63"} Dec 03 12:30:33 crc kubenswrapper[5052]: I1203 12:30:33.814013 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-galera-0" event={"ID":"6938204c-1fe8-4f03-ac16-ec9e19e090f8","Type":"ContainerStarted","Data":"d94e0139dcad346a24d75f7bd285773c0fe1edad0369015fe7b299454967acdd"} Dec 03 12:30:33 crc kubenswrapper[5052]: I1203 12:30:33.822556 5052 generic.go:334] "Generic (PLEG): container finished" podID="a65652a1-62fa-40ff-8b8a-940b2cc795b7" containerID="f18b0f1af72b5634a7b8a197502ec544d35bc31178131d0cc5f858ca0d990b27" exitCode=137 Dec 03 12:30:33 crc kubenswrapper[5052]: I1203 12:30:33.822623 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-worker-6d687d9c57-4gplx" event={"ID":"a65652a1-62fa-40ff-8b8a-940b2cc795b7","Type":"ContainerDied","Data":"f18b0f1af72b5634a7b8a197502ec544d35bc31178131d0cc5f858ca0d990b27"} Dec 03 12:30:33 crc kubenswrapper[5052]: I1203 12:30:33.824540 5052 generic.go:334] "Generic (PLEG): container finished" podID="ff58ae3c-d75e-4038-be70-00f88178a18f" containerID="1165299b4299bb42e6bbea4345f1d0f3318f8b00847584b340a3eb83aecf87aa" exitCode=0 Dec 03 12:30:33 crc kubenswrapper[5052]: I1203 12:30:33.824567 5052 generic.go:334] "Generic (PLEG): container finished" podID="ff58ae3c-d75e-4038-be70-00f88178a18f" containerID="638897c2f0ffd643c8e7fa20a4d2aa99ceee7bfc03c3aa9760b40e97843cf3c8" exitCode=143 Dec 03 12:30:33 crc kubenswrapper[5052]: I1203 12:30:33.824709 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-64d69c4db4-hkscs" event={"ID":"ff58ae3c-d75e-4038-be70-00f88178a18f","Type":"ContainerDied","Data":"1165299b4299bb42e6bbea4345f1d0f3318f8b00847584b340a3eb83aecf87aa"} Dec 03 12:30:33 crc kubenswrapper[5052]: I1203 12:30:33.824728 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-64d69c4db4-hkscs" event={"ID":"ff58ae3c-d75e-4038-be70-00f88178a18f","Type":"ContainerDied","Data":"638897c2f0ffd643c8e7fa20a4d2aa99ceee7bfc03c3aa9760b40e97843cf3c8"} Dec 03 12:30:33 crc kubenswrapper[5052]: I1203 12:30:33.825486 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/barbican-api-68fc8f9776-n9gml" Dec 03 12:30:33 crc kubenswrapper[5052]: I1203 12:30:33.849049 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/openstack-galera-0" podStartSLOduration=5.849028935 podStartE2EDuration="5.849028935s" podCreationTimestamp="2025-12-03 12:30:28 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:30:33.842853439 +0000 UTC m=+4224.431095213" watchObservedRunningTime="2025-12-03 12:30:33.849028935 +0000 UTC m=+4224.437270689" Dec 03 12:30:33 crc kubenswrapper[5052]: I1203 12:30:33.944786 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:30:34 crc kubenswrapper[5052]: I1203 12:30:34.136175 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-api-64d69c4db4-hkscs" Dec 03 12:30:34 crc kubenswrapper[5052]: I1203 12:30:34.208476 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/ff58ae3c-d75e-4038-be70-00f88178a18f-config-data\") pod \"ff58ae3c-d75e-4038-be70-00f88178a18f\" (UID: \"ff58ae3c-d75e-4038-be70-00f88178a18f\") " Dec 03 12:30:34 crc kubenswrapper[5052]: I1203 12:30:34.208962 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ff58ae3c-d75e-4038-be70-00f88178a18f-combined-ca-bundle\") pod \"ff58ae3c-d75e-4038-be70-00f88178a18f\" (UID: \"ff58ae3c-d75e-4038-be70-00f88178a18f\") " Dec 03 12:30:34 crc kubenswrapper[5052]: I1203 12:30:34.209020 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/ff58ae3c-d75e-4038-be70-00f88178a18f-config-data-custom\") pod \"ff58ae3c-d75e-4038-be70-00f88178a18f\" (UID: \"ff58ae3c-d75e-4038-be70-00f88178a18f\") " Dec 03 12:30:34 crc kubenswrapper[5052]: I1203 12:30:34.209075 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/ff58ae3c-d75e-4038-be70-00f88178a18f-public-tls-certs\") pod \"ff58ae3c-d75e-4038-be70-00f88178a18f\" (UID: \"ff58ae3c-d75e-4038-be70-00f88178a18f\") " Dec 03 12:30:34 crc kubenswrapper[5052]: I1203 12:30:34.209156 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/ff58ae3c-d75e-4038-be70-00f88178a18f-internal-tls-certs\") pod \"ff58ae3c-d75e-4038-be70-00f88178a18f\" (UID: \"ff58ae3c-d75e-4038-be70-00f88178a18f\") " Dec 03 12:30:34 crc kubenswrapper[5052]: I1203 12:30:34.209183 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/ff58ae3c-d75e-4038-be70-00f88178a18f-logs\") pod \"ff58ae3c-d75e-4038-be70-00f88178a18f\" (UID: \"ff58ae3c-d75e-4038-be70-00f88178a18f\") " Dec 03 12:30:34 crc kubenswrapper[5052]: I1203 12:30:34.209232 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jm72m\" (UniqueName: \"kubernetes.io/projected/ff58ae3c-d75e-4038-be70-00f88178a18f-kube-api-access-jm72m\") pod \"ff58ae3c-d75e-4038-be70-00f88178a18f\" (UID: \"ff58ae3c-d75e-4038-be70-00f88178a18f\") " Dec 03 12:30:34 crc kubenswrapper[5052]: I1203 12:30:34.211037 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff58ae3c-d75e-4038-be70-00f88178a18f-logs" (OuterVolumeSpecName: "logs") pod "ff58ae3c-d75e-4038-be70-00f88178a18f" (UID: "ff58ae3c-d75e-4038-be70-00f88178a18f"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:30:34 crc kubenswrapper[5052]: I1203 12:30:34.216212 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ff58ae3c-d75e-4038-be70-00f88178a18f-kube-api-access-jm72m" (OuterVolumeSpecName: "kube-api-access-jm72m") pod "ff58ae3c-d75e-4038-be70-00f88178a18f" (UID: "ff58ae3c-d75e-4038-be70-00f88178a18f"). InnerVolumeSpecName "kube-api-access-jm72m". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:30:34 crc kubenswrapper[5052]: I1203 12:30:34.216917 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ff58ae3c-d75e-4038-be70-00f88178a18f-config-data-custom" (OuterVolumeSpecName: "config-data-custom") pod "ff58ae3c-d75e-4038-be70-00f88178a18f" (UID: "ff58ae3c-d75e-4038-be70-00f88178a18f"). InnerVolumeSpecName "config-data-custom". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:34 crc kubenswrapper[5052]: I1203 12:30:34.235775 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-metadata-0" podUID="39117149-c15d-4753-9324-8a42abe5b007" containerName="nova-metadata-metadata" probeResult="failure" output="Get \"https://10.217.0.213:8775/\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Dec 03 12:30:34 crc kubenswrapper[5052]: I1203 12:30:34.237050 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-metadata-0" podUID="39117149-c15d-4753-9324-8a42abe5b007" containerName="nova-metadata-log" probeResult="failure" output="Get \"https://10.217.0.213:8775/\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Dec 03 12:30:34 crc kubenswrapper[5052]: I1203 12:30:34.249721 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ff58ae3c-d75e-4038-be70-00f88178a18f-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "ff58ae3c-d75e-4038-be70-00f88178a18f" (UID: "ff58ae3c-d75e-4038-be70-00f88178a18f"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:34 crc kubenswrapper[5052]: I1203 12:30:34.264194 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-worker-6d687d9c57-4gplx" Dec 03 12:30:34 crc kubenswrapper[5052]: I1203 12:30:34.269864 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ff58ae3c-d75e-4038-be70-00f88178a18f-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "ff58ae3c-d75e-4038-be70-00f88178a18f" (UID: "ff58ae3c-d75e-4038-be70-00f88178a18f"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:34 crc kubenswrapper[5052]: I1203 12:30:34.302559 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ff58ae3c-d75e-4038-be70-00f88178a18f-config-data" (OuterVolumeSpecName: "config-data") pod "ff58ae3c-d75e-4038-be70-00f88178a18f" (UID: "ff58ae3c-d75e-4038-be70-00f88178a18f"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:34 crc kubenswrapper[5052]: I1203 12:30:34.304724 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ff58ae3c-d75e-4038-be70-00f88178a18f-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "ff58ae3c-d75e-4038-be70-00f88178a18f" (UID: "ff58ae3c-d75e-4038-be70-00f88178a18f"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:34 crc kubenswrapper[5052]: I1203 12:30:34.310929 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/a65652a1-62fa-40ff-8b8a-940b2cc795b7-logs\") pod \"a65652a1-62fa-40ff-8b8a-940b2cc795b7\" (UID: \"a65652a1-62fa-40ff-8b8a-940b2cc795b7\") " Dec 03 12:30:34 crc kubenswrapper[5052]: I1203 12:30:34.311057 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/a65652a1-62fa-40ff-8b8a-940b2cc795b7-config-data-custom\") pod \"a65652a1-62fa-40ff-8b8a-940b2cc795b7\" (UID: \"a65652a1-62fa-40ff-8b8a-940b2cc795b7\") " Dec 03 12:30:34 crc kubenswrapper[5052]: I1203 12:30:34.311104 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a65652a1-62fa-40ff-8b8a-940b2cc795b7-config-data\") pod \"a65652a1-62fa-40ff-8b8a-940b2cc795b7\" (UID: \"a65652a1-62fa-40ff-8b8a-940b2cc795b7\") " Dec 03 12:30:34 crc kubenswrapper[5052]: I1203 12:30:34.311174 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s4zxs\" (UniqueName: \"kubernetes.io/projected/a65652a1-62fa-40ff-8b8a-940b2cc795b7-kube-api-access-s4zxs\") pod \"a65652a1-62fa-40ff-8b8a-940b2cc795b7\" (UID: \"a65652a1-62fa-40ff-8b8a-940b2cc795b7\") " Dec 03 12:30:34 crc kubenswrapper[5052]: I1203 12:30:34.311277 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a65652a1-62fa-40ff-8b8a-940b2cc795b7-logs" (OuterVolumeSpecName: "logs") pod "a65652a1-62fa-40ff-8b8a-940b2cc795b7" (UID: "a65652a1-62fa-40ff-8b8a-940b2cc795b7"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:30:34 crc kubenswrapper[5052]: I1203 12:30:34.311293 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a65652a1-62fa-40ff-8b8a-940b2cc795b7-combined-ca-bundle\") pod \"a65652a1-62fa-40ff-8b8a-940b2cc795b7\" (UID: \"a65652a1-62fa-40ff-8b8a-940b2cc795b7\") " Dec 03 12:30:34 crc kubenswrapper[5052]: I1203 12:30:34.311934 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/ff58ae3c-d75e-4038-be70-00f88178a18f-config-data-custom\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:34 crc kubenswrapper[5052]: I1203 12:30:34.311953 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/ff58ae3c-d75e-4038-be70-00f88178a18f-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:34 crc kubenswrapper[5052]: I1203 12:30:34.311964 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/ff58ae3c-d75e-4038-be70-00f88178a18f-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:34 crc kubenswrapper[5052]: I1203 12:30:34.311977 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/ff58ae3c-d75e-4038-be70-00f88178a18f-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:34 crc kubenswrapper[5052]: I1203 12:30:34.311990 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-jm72m\" (UniqueName: \"kubernetes.io/projected/ff58ae3c-d75e-4038-be70-00f88178a18f-kube-api-access-jm72m\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:34 crc kubenswrapper[5052]: I1203 12:30:34.312006 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/ff58ae3c-d75e-4038-be70-00f88178a18f-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:34 crc kubenswrapper[5052]: I1203 12:30:34.312019 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/a65652a1-62fa-40ff-8b8a-940b2cc795b7-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:34 crc kubenswrapper[5052]: I1203 12:30:34.312030 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ff58ae3c-d75e-4038-be70-00f88178a18f-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:34 crc kubenswrapper[5052]: I1203 12:30:34.316483 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a65652a1-62fa-40ff-8b8a-940b2cc795b7-kube-api-access-s4zxs" (OuterVolumeSpecName: "kube-api-access-s4zxs") pod "a65652a1-62fa-40ff-8b8a-940b2cc795b7" (UID: "a65652a1-62fa-40ff-8b8a-940b2cc795b7"). InnerVolumeSpecName "kube-api-access-s4zxs". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:30:34 crc kubenswrapper[5052]: I1203 12:30:34.316891 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a65652a1-62fa-40ff-8b8a-940b2cc795b7-config-data-custom" (OuterVolumeSpecName: "config-data-custom") pod "a65652a1-62fa-40ff-8b8a-940b2cc795b7" (UID: "a65652a1-62fa-40ff-8b8a-940b2cc795b7"). InnerVolumeSpecName "config-data-custom". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:34 crc kubenswrapper[5052]: I1203 12:30:34.364828 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a65652a1-62fa-40ff-8b8a-940b2cc795b7-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "a65652a1-62fa-40ff-8b8a-940b2cc795b7" (UID: "a65652a1-62fa-40ff-8b8a-940b2cc795b7"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:34 crc kubenswrapper[5052]: I1203 12:30:34.414061 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a65652a1-62fa-40ff-8b8a-940b2cc795b7-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:34 crc kubenswrapper[5052]: I1203 12:30:34.414087 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/a65652a1-62fa-40ff-8b8a-940b2cc795b7-config-data-custom\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:34 crc kubenswrapper[5052]: I1203 12:30:34.414100 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-s4zxs\" (UniqueName: \"kubernetes.io/projected/a65652a1-62fa-40ff-8b8a-940b2cc795b7-kube-api-access-s4zxs\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:34 crc kubenswrapper[5052]: I1203 12:30:34.422474 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a65652a1-62fa-40ff-8b8a-940b2cc795b7-config-data" (OuterVolumeSpecName: "config-data") pod "a65652a1-62fa-40ff-8b8a-940b2cc795b7" (UID: "a65652a1-62fa-40ff-8b8a-940b2cc795b7"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:34 crc kubenswrapper[5052]: I1203 12:30:34.442890 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:30:34 crc kubenswrapper[5052]: I1203 12:30:34.442929 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:30:34 crc kubenswrapper[5052]: I1203 12:30:34.487670 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:30:34 crc kubenswrapper[5052]: I1203 12:30:34.511109 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:30:34 crc kubenswrapper[5052]: I1203 12:30:34.515474 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a65652a1-62fa-40ff-8b8a-940b2cc795b7-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:34 crc kubenswrapper[5052]: I1203 12:30:34.621948 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-keystone-listener-77b6bbbdcb-4jlsc" Dec 03 12:30:34 crc kubenswrapper[5052]: I1203 12:30:34.718247 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wv5t5\" (UniqueName: \"kubernetes.io/projected/aed41b09-319e-466c-9eb9-38208e1880a3-kube-api-access-wv5t5\") pod \"aed41b09-319e-466c-9eb9-38208e1880a3\" (UID: \"aed41b09-319e-466c-9eb9-38208e1880a3\") " Dec 03 12:30:34 crc kubenswrapper[5052]: I1203 12:30:34.718833 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/aed41b09-319e-466c-9eb9-38208e1880a3-combined-ca-bundle\") pod \"aed41b09-319e-466c-9eb9-38208e1880a3\" (UID: \"aed41b09-319e-466c-9eb9-38208e1880a3\") " Dec 03 12:30:34 crc kubenswrapper[5052]: I1203 12:30:34.718950 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/aed41b09-319e-466c-9eb9-38208e1880a3-config-data\") pod \"aed41b09-319e-466c-9eb9-38208e1880a3\" (UID: \"aed41b09-319e-466c-9eb9-38208e1880a3\") " Dec 03 12:30:34 crc kubenswrapper[5052]: I1203 12:30:34.719096 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/aed41b09-319e-466c-9eb9-38208e1880a3-logs\") pod \"aed41b09-319e-466c-9eb9-38208e1880a3\" (UID: \"aed41b09-319e-466c-9eb9-38208e1880a3\") " Dec 03 12:30:34 crc kubenswrapper[5052]: I1203 12:30:34.719429 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/aed41b09-319e-466c-9eb9-38208e1880a3-config-data-custom\") pod \"aed41b09-319e-466c-9eb9-38208e1880a3\" (UID: \"aed41b09-319e-466c-9eb9-38208e1880a3\") " Dec 03 12:30:34 crc kubenswrapper[5052]: I1203 12:30:34.719492 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aed41b09-319e-466c-9eb9-38208e1880a3-logs" (OuterVolumeSpecName: "logs") pod "aed41b09-319e-466c-9eb9-38208e1880a3" (UID: "aed41b09-319e-466c-9eb9-38208e1880a3"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:30:34 crc kubenswrapper[5052]: I1203 12:30:34.720133 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/aed41b09-319e-466c-9eb9-38208e1880a3-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:34 crc kubenswrapper[5052]: I1203 12:30:34.722111 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aed41b09-319e-466c-9eb9-38208e1880a3-kube-api-access-wv5t5" (OuterVolumeSpecName: "kube-api-access-wv5t5") pod "aed41b09-319e-466c-9eb9-38208e1880a3" (UID: "aed41b09-319e-466c-9eb9-38208e1880a3"). InnerVolumeSpecName "kube-api-access-wv5t5". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:30:34 crc kubenswrapper[5052]: I1203 12:30:34.725719 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aed41b09-319e-466c-9eb9-38208e1880a3-config-data-custom" (OuterVolumeSpecName: "config-data-custom") pod "aed41b09-319e-466c-9eb9-38208e1880a3" (UID: "aed41b09-319e-466c-9eb9-38208e1880a3"). InnerVolumeSpecName "config-data-custom". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:34 crc kubenswrapper[5052]: I1203 12:30:34.764718 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aed41b09-319e-466c-9eb9-38208e1880a3-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "aed41b09-319e-466c-9eb9-38208e1880a3" (UID: "aed41b09-319e-466c-9eb9-38208e1880a3"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:34 crc kubenswrapper[5052]: I1203 12:30:34.791974 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aed41b09-319e-466c-9eb9-38208e1880a3-config-data" (OuterVolumeSpecName: "config-data") pod "aed41b09-319e-466c-9eb9-38208e1880a3" (UID: "aed41b09-319e-466c-9eb9-38208e1880a3"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:34 crc kubenswrapper[5052]: I1203 12:30:34.822427 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/aed41b09-319e-466c-9eb9-38208e1880a3-config-data-custom\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:34 crc kubenswrapper[5052]: I1203 12:30:34.823533 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-wv5t5\" (UniqueName: \"kubernetes.io/projected/aed41b09-319e-466c-9eb9-38208e1880a3-kube-api-access-wv5t5\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:34 crc kubenswrapper[5052]: I1203 12:30:34.823631 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/aed41b09-319e-466c-9eb9-38208e1880a3-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:34 crc kubenswrapper[5052]: I1203 12:30:34.823727 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/aed41b09-319e-466c-9eb9-38208e1880a3-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:34 crc kubenswrapper[5052]: I1203 12:30:34.848528 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-worker-6d687d9c57-4gplx" event={"ID":"a65652a1-62fa-40ff-8b8a-940b2cc795b7","Type":"ContainerDied","Data":"9e5d3ebacee74cc3f19f073892deec26b2b6d3e604678ea5b944624e06141bd5"} Dec 03 12:30:34 crc kubenswrapper[5052]: I1203 12:30:34.848596 5052 scope.go:117] "RemoveContainer" containerID="f18b0f1af72b5634a7b8a197502ec544d35bc31178131d0cc5f858ca0d990b27" Dec 03 12:30:34 crc kubenswrapper[5052]: I1203 12:30:34.848598 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-worker-6d687d9c57-4gplx" Dec 03 12:30:34 crc kubenswrapper[5052]: I1203 12:30:34.853952 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-64d69c4db4-hkscs" event={"ID":"ff58ae3c-d75e-4038-be70-00f88178a18f","Type":"ContainerDied","Data":"85335d4cabbcc2d67511e1cbdbc856b320084a0d652a2d294770e9e25e81348f"} Dec 03 12:30:34 crc kubenswrapper[5052]: I1203 12:30:34.854085 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-api-64d69c4db4-hkscs" Dec 03 12:30:34 crc kubenswrapper[5052]: I1203 12:30:34.857307 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-keystone-listener-77b6bbbdcb-4jlsc" event={"ID":"aed41b09-319e-466c-9eb9-38208e1880a3","Type":"ContainerDied","Data":"d94de750247c1241b19be7117917b99bc882f3e7734d81c7bd3e29a29ec26f1c"} Dec 03 12:30:34 crc kubenswrapper[5052]: I1203 12:30:34.857803 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-keystone-listener-77b6bbbdcb-4jlsc" Dec 03 12:30:34 crc kubenswrapper[5052]: I1203 12:30:34.867550 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"7cbe1c2c-bc04-4232-9901-bc9d134df3d6","Type":"ContainerStarted","Data":"1d2a9d60a97780af3d81ede77692ed338720a0c0a5b32ac9f95fa608d8ace7d1"} Dec 03 12:30:34 crc kubenswrapper[5052]: I1203 12:30:34.867756 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"7cbe1c2c-bc04-4232-9901-bc9d134df3d6","Type":"ContainerStarted","Data":"b8128fd869a1a047b25167700c1bc9dedce09fba89a0659838f2d02e09eb65e6"} Dec 03 12:30:34 crc kubenswrapper[5052]: I1203 12:30:34.921706 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-worker-6d687d9c57-4gplx"] Dec 03 12:30:34 crc kubenswrapper[5052]: I1203 12:30:34.927001 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/barbican-api-ff6b8c44b-7ndgm" podUID="48831cad-910f-46f1-aefe-7cc4cbfa984f" containerName="barbican-api-log" probeResult="failure" output="Get \"https://10.217.0.204:9311/healthcheck\": read tcp 10.217.0.2:54548->10.217.0.204:9311: read: connection reset by peer" Dec 03 12:30:34 crc kubenswrapper[5052]: I1203 12:30:34.927163 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/barbican-api-ff6b8c44b-7ndgm" podUID="48831cad-910f-46f1-aefe-7cc4cbfa984f" containerName="barbican-api" probeResult="failure" output="Get \"https://10.217.0.204:9311/healthcheck\": read tcp 10.217.0.2:54560->10.217.0.204:9311: read: connection reset by peer" Dec 03 12:30:34 crc kubenswrapper[5052]: I1203 12:30:34.932051 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/barbican-worker-6d687d9c57-4gplx"] Dec 03 12:30:34 crc kubenswrapper[5052]: I1203 12:30:34.941261 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-keystone-listener-77b6bbbdcb-4jlsc"] Dec 03 12:30:34 crc kubenswrapper[5052]: I1203 12:30:34.950562 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/barbican-keystone-listener-77b6bbbdcb-4jlsc"] Dec 03 12:30:34 crc kubenswrapper[5052]: I1203 12:30:34.951354 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:30:34 crc kubenswrapper[5052]: I1203 12:30:34.958015 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:30:34 crc kubenswrapper[5052]: I1203 12:30:34.959578 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-api-64d69c4db4-hkscs"] Dec 03 12:30:34 crc kubenswrapper[5052]: I1203 12:30:34.968481 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/barbican-api-64d69c4db4-hkscs"] Dec 03 12:30:35 crc kubenswrapper[5052]: I1203 12:30:35.092860 5052 scope.go:117] "RemoveContainer" containerID="09f8bee6ba47ae352b63fc6a9643dadd544c69e4d30aec7017b6e2983d7db541" Dec 03 12:30:35 crc kubenswrapper[5052]: I1203 12:30:35.130943 5052 scope.go:117] "RemoveContainer" containerID="1165299b4299bb42e6bbea4345f1d0f3318f8b00847584b340a3eb83aecf87aa" Dec 03 12:30:35 crc kubenswrapper[5052]: I1203 12:30:35.187458 5052 scope.go:117] "RemoveContainer" containerID="638897c2f0ffd643c8e7fa20a4d2aa99ceee7bfc03c3aa9760b40e97843cf3c8" Dec 03 12:30:35 crc kubenswrapper[5052]: I1203 12:30:35.223826 5052 scope.go:117] "RemoveContainer" containerID="0a78aa778c6e6e1f2c1d7cc5f57fa0dc6dd213f54da357a449dda9efb5e3bc63" Dec 03 12:30:35 crc kubenswrapper[5052]: I1203 12:30:35.278269 5052 scope.go:117] "RemoveContainer" containerID="c4632c9d758da0d71f63ae0b7ef4c62003466443e6cb1e897bae7412a5ae794f" Dec 03 12:30:35 crc kubenswrapper[5052]: I1203 12:30:35.530889 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-api-ff6b8c44b-7ndgm" Dec 03 12:30:35 crc kubenswrapper[5052]: I1203 12:30:35.651132 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k84ln\" (UniqueName: \"kubernetes.io/projected/48831cad-910f-46f1-aefe-7cc4cbfa984f-kube-api-access-k84ln\") pod \"48831cad-910f-46f1-aefe-7cc4cbfa984f\" (UID: \"48831cad-910f-46f1-aefe-7cc4cbfa984f\") " Dec 03 12:30:35 crc kubenswrapper[5052]: I1203 12:30:35.651266 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/48831cad-910f-46f1-aefe-7cc4cbfa984f-public-tls-certs\") pod \"48831cad-910f-46f1-aefe-7cc4cbfa984f\" (UID: \"48831cad-910f-46f1-aefe-7cc4cbfa984f\") " Dec 03 12:30:35 crc kubenswrapper[5052]: I1203 12:30:35.651295 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/48831cad-910f-46f1-aefe-7cc4cbfa984f-config-data-custom\") pod \"48831cad-910f-46f1-aefe-7cc4cbfa984f\" (UID: \"48831cad-910f-46f1-aefe-7cc4cbfa984f\") " Dec 03 12:30:35 crc kubenswrapper[5052]: I1203 12:30:35.651375 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/48831cad-910f-46f1-aefe-7cc4cbfa984f-logs\") pod \"48831cad-910f-46f1-aefe-7cc4cbfa984f\" (UID: \"48831cad-910f-46f1-aefe-7cc4cbfa984f\") " Dec 03 12:30:35 crc kubenswrapper[5052]: I1203 12:30:35.651405 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/48831cad-910f-46f1-aefe-7cc4cbfa984f-combined-ca-bundle\") pod \"48831cad-910f-46f1-aefe-7cc4cbfa984f\" (UID: \"48831cad-910f-46f1-aefe-7cc4cbfa984f\") " Dec 03 12:30:35 crc kubenswrapper[5052]: I1203 12:30:35.651443 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/48831cad-910f-46f1-aefe-7cc4cbfa984f-config-data\") pod \"48831cad-910f-46f1-aefe-7cc4cbfa984f\" (UID: \"48831cad-910f-46f1-aefe-7cc4cbfa984f\") " Dec 03 12:30:35 crc kubenswrapper[5052]: I1203 12:30:35.651539 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/48831cad-910f-46f1-aefe-7cc4cbfa984f-internal-tls-certs\") pod \"48831cad-910f-46f1-aefe-7cc4cbfa984f\" (UID: \"48831cad-910f-46f1-aefe-7cc4cbfa984f\") " Dec 03 12:30:35 crc kubenswrapper[5052]: I1203 12:30:35.654258 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/48831cad-910f-46f1-aefe-7cc4cbfa984f-logs" (OuterVolumeSpecName: "logs") pod "48831cad-910f-46f1-aefe-7cc4cbfa984f" (UID: "48831cad-910f-46f1-aefe-7cc4cbfa984f"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:30:35 crc kubenswrapper[5052]: I1203 12:30:35.672499 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/48831cad-910f-46f1-aefe-7cc4cbfa984f-kube-api-access-k84ln" (OuterVolumeSpecName: "kube-api-access-k84ln") pod "48831cad-910f-46f1-aefe-7cc4cbfa984f" (UID: "48831cad-910f-46f1-aefe-7cc4cbfa984f"). InnerVolumeSpecName "kube-api-access-k84ln". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:30:35 crc kubenswrapper[5052]: I1203 12:30:35.690861 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/48831cad-910f-46f1-aefe-7cc4cbfa984f-config-data-custom" (OuterVolumeSpecName: "config-data-custom") pod "48831cad-910f-46f1-aefe-7cc4cbfa984f" (UID: "48831cad-910f-46f1-aefe-7cc4cbfa984f"). InnerVolumeSpecName "config-data-custom". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:35 crc kubenswrapper[5052]: I1203 12:30:35.702592 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a65652a1-62fa-40ff-8b8a-940b2cc795b7" path="/var/lib/kubelet/pods/a65652a1-62fa-40ff-8b8a-940b2cc795b7/volumes" Dec 03 12:30:35 crc kubenswrapper[5052]: I1203 12:30:35.703413 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aed41b09-319e-466c-9eb9-38208e1880a3" path="/var/lib/kubelet/pods/aed41b09-319e-466c-9eb9-38208e1880a3/volumes" Dec 03 12:30:35 crc kubenswrapper[5052]: I1203 12:30:35.704047 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ff58ae3c-d75e-4038-be70-00f88178a18f" path="/var/lib/kubelet/pods/ff58ae3c-d75e-4038-be70-00f88178a18f/volumes" Dec 03 12:30:35 crc kubenswrapper[5052]: I1203 12:30:35.734129 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/48831cad-910f-46f1-aefe-7cc4cbfa984f-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "48831cad-910f-46f1-aefe-7cc4cbfa984f" (UID: "48831cad-910f-46f1-aefe-7cc4cbfa984f"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:35 crc kubenswrapper[5052]: I1203 12:30:35.737095 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/48831cad-910f-46f1-aefe-7cc4cbfa984f-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "48831cad-910f-46f1-aefe-7cc4cbfa984f" (UID: "48831cad-910f-46f1-aefe-7cc4cbfa984f"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:35 crc kubenswrapper[5052]: I1203 12:30:35.740706 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/48831cad-910f-46f1-aefe-7cc4cbfa984f-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "48831cad-910f-46f1-aefe-7cc4cbfa984f" (UID: "48831cad-910f-46f1-aefe-7cc4cbfa984f"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:35 crc kubenswrapper[5052]: I1203 12:30:35.747189 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/48831cad-910f-46f1-aefe-7cc4cbfa984f-config-data" (OuterVolumeSpecName: "config-data") pod "48831cad-910f-46f1-aefe-7cc4cbfa984f" (UID: "48831cad-910f-46f1-aefe-7cc4cbfa984f"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:35 crc kubenswrapper[5052]: I1203 12:30:35.754096 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/48831cad-910f-46f1-aefe-7cc4cbfa984f-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:35 crc kubenswrapper[5052]: I1203 12:30:35.754128 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/48831cad-910f-46f1-aefe-7cc4cbfa984f-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:35 crc kubenswrapper[5052]: I1203 12:30:35.754164 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/48831cad-910f-46f1-aefe-7cc4cbfa984f-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:35 crc kubenswrapper[5052]: I1203 12:30:35.754175 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/48831cad-910f-46f1-aefe-7cc4cbfa984f-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:35 crc kubenswrapper[5052]: I1203 12:30:35.754184 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-k84ln\" (UniqueName: \"kubernetes.io/projected/48831cad-910f-46f1-aefe-7cc4cbfa984f-kube-api-access-k84ln\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:35 crc kubenswrapper[5052]: I1203 12:30:35.754193 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/48831cad-910f-46f1-aefe-7cc4cbfa984f-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:35 crc kubenswrapper[5052]: I1203 12:30:35.754201 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/48831cad-910f-46f1-aefe-7cc4cbfa984f-config-data-custom\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:35 crc kubenswrapper[5052]: I1203 12:30:35.876854 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"a24b0824-26d0-4b81-b6cb-d617c36d45ab","Type":"ContainerStarted","Data":"c16bebd7126caa95cf323caf300318d8818542b3e43480cb466bf73872c95fec"} Dec 03 12:30:35 crc kubenswrapper[5052]: I1203 12:30:35.877145 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="a24b0824-26d0-4b81-b6cb-d617c36d45ab" containerName="ceilometer-central-agent" containerID="cri-o://4b3af605b0d8c2dcd2ab4507ac26ec3ce43fc470db20730adce6dbd29442e123" gracePeriod=30 Dec 03 12:30:35 crc kubenswrapper[5052]: I1203 12:30:35.877819 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:30:35 crc kubenswrapper[5052]: I1203 12:30:35.878239 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="a24b0824-26d0-4b81-b6cb-d617c36d45ab" containerName="proxy-httpd" containerID="cri-o://c16bebd7126caa95cf323caf300318d8818542b3e43480cb466bf73872c95fec" gracePeriod=30 Dec 03 12:30:35 crc kubenswrapper[5052]: I1203 12:30:35.878325 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="a24b0824-26d0-4b81-b6cb-d617c36d45ab" containerName="sg-core" containerID="cri-o://c477cd69a766929b221a6f0497205e288c5ded0f70fe9daadbd6669d7a943603" gracePeriod=30 Dec 03 12:30:35 crc kubenswrapper[5052]: I1203 12:30:35.878437 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="a24b0824-26d0-4b81-b6cb-d617c36d45ab" containerName="ceilometer-notification-agent" containerID="cri-o://e765959f449b54c23591a65f292f2f7a5099d71fa4f0ea45d8c2f901334b7206" gracePeriod=30 Dec 03 12:30:35 crc kubenswrapper[5052]: I1203 12:30:35.890948 5052 generic.go:334] "Generic (PLEG): container finished" podID="48831cad-910f-46f1-aefe-7cc4cbfa984f" containerID="f6c28d3863e0d0d300bbffab66b3c5240cbeaa832b51d4164b9f0b3c092d3831" exitCode=0 Dec 03 12:30:35 crc kubenswrapper[5052]: I1203 12:30:35.891045 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-ff6b8c44b-7ndgm" event={"ID":"48831cad-910f-46f1-aefe-7cc4cbfa984f","Type":"ContainerDied","Data":"f6c28d3863e0d0d300bbffab66b3c5240cbeaa832b51d4164b9f0b3c092d3831"} Dec 03 12:30:35 crc kubenswrapper[5052]: I1203 12:30:35.891075 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-ff6b8c44b-7ndgm" event={"ID":"48831cad-910f-46f1-aefe-7cc4cbfa984f","Type":"ContainerDied","Data":"1f06f16f71b99e617383aa251524a1b2334b34f46ead54b868e09f13e83c19cc"} Dec 03 12:30:35 crc kubenswrapper[5052]: I1203 12:30:35.891095 5052 scope.go:117] "RemoveContainer" containerID="f6c28d3863e0d0d300bbffab66b3c5240cbeaa832b51d4164b9f0b3c092d3831" Dec 03 12:30:35 crc kubenswrapper[5052]: I1203 12:30:35.891224 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-api-ff6b8c44b-7ndgm" Dec 03 12:30:35 crc kubenswrapper[5052]: I1203 12:30:35.900451 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"7cbe1c2c-bc04-4232-9901-bc9d134df3d6","Type":"ContainerStarted","Data":"3c03aa60a5d69f404b768bb6cc6569fcdc7b56fcc98ed0721fb684712e773b3e"} Dec 03 12:30:35 crc kubenswrapper[5052]: I1203 12:30:35.921864 5052 scope.go:117] "RemoveContainer" containerID="b447fe4d34e0569d637c38fcee6f843a5b3a8e43d2bd2e661971752993efd16e" Dec 03 12:30:35 crc kubenswrapper[5052]: I1203 12:30:35.937043 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-api-0" podStartSLOduration=2.937004481 podStartE2EDuration="2.937004481s" podCreationTimestamp="2025-12-03 12:30:33 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:30:35.929778825 +0000 UTC m=+4226.518020589" watchObservedRunningTime="2025-12-03 12:30:35.937004481 +0000 UTC m=+4226.525246245" Dec 03 12:30:35 crc kubenswrapper[5052]: I1203 12:30:35.943399 5052 scope.go:117] "RemoveContainer" containerID="f6c28d3863e0d0d300bbffab66b3c5240cbeaa832b51d4164b9f0b3c092d3831" Dec 03 12:30:35 crc kubenswrapper[5052]: E1203 12:30:35.945538 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f6c28d3863e0d0d300bbffab66b3c5240cbeaa832b51d4164b9f0b3c092d3831\": container with ID starting with f6c28d3863e0d0d300bbffab66b3c5240cbeaa832b51d4164b9f0b3c092d3831 not found: ID does not exist" containerID="f6c28d3863e0d0d300bbffab66b3c5240cbeaa832b51d4164b9f0b3c092d3831" Dec 03 12:30:35 crc kubenswrapper[5052]: I1203 12:30:35.945682 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f6c28d3863e0d0d300bbffab66b3c5240cbeaa832b51d4164b9f0b3c092d3831"} err="failed to get container status \"f6c28d3863e0d0d300bbffab66b3c5240cbeaa832b51d4164b9f0b3c092d3831\": rpc error: code = NotFound desc = could not find container \"f6c28d3863e0d0d300bbffab66b3c5240cbeaa832b51d4164b9f0b3c092d3831\": container with ID starting with f6c28d3863e0d0d300bbffab66b3c5240cbeaa832b51d4164b9f0b3c092d3831 not found: ID does not exist" Dec 03 12:30:35 crc kubenswrapper[5052]: I1203 12:30:35.945787 5052 scope.go:117] "RemoveContainer" containerID="b447fe4d34e0569d637c38fcee6f843a5b3a8e43d2bd2e661971752993efd16e" Dec 03 12:30:35 crc kubenswrapper[5052]: E1203 12:30:35.947147 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b447fe4d34e0569d637c38fcee6f843a5b3a8e43d2bd2e661971752993efd16e\": container with ID starting with b447fe4d34e0569d637c38fcee6f843a5b3a8e43d2bd2e661971752993efd16e not found: ID does not exist" containerID="b447fe4d34e0569d637c38fcee6f843a5b3a8e43d2bd2e661971752993efd16e" Dec 03 12:30:35 crc kubenswrapper[5052]: I1203 12:30:35.947199 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b447fe4d34e0569d637c38fcee6f843a5b3a8e43d2bd2e661971752993efd16e"} err="failed to get container status \"b447fe4d34e0569d637c38fcee6f843a5b3a8e43d2bd2e661971752993efd16e\": rpc error: code = NotFound desc = could not find container \"b447fe4d34e0569d637c38fcee6f843a5b3a8e43d2bd2e661971752993efd16e\": container with ID starting with b447fe4d34e0569d637c38fcee6f843a5b3a8e43d2bd2e661971752993efd16e not found: ID does not exist" Dec 03 12:30:35 crc kubenswrapper[5052]: I1203 12:30:35.957480 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/ceilometer-0" podStartSLOduration=4.684215747 podStartE2EDuration="28.957382993s" podCreationTimestamp="2025-12-03 12:30:07 +0000 UTC" firstStartedPulling="2025-12-03 12:30:10.819915997 +0000 UTC m=+4201.408157761" lastFinishedPulling="2025-12-03 12:30:35.093083243 +0000 UTC m=+4225.681325007" observedRunningTime="2025-12-03 12:30:35.912658006 +0000 UTC m=+4226.500899770" watchObservedRunningTime="2025-12-03 12:30:35.957382993 +0000 UTC m=+4226.545624757" Dec 03 12:30:35 crc kubenswrapper[5052]: I1203 12:30:35.971655 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-api-ff6b8c44b-7ndgm"] Dec 03 12:30:35 crc kubenswrapper[5052]: I1203 12:30:35.984470 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/barbican-api-ff6b8c44b-7ndgm"] Dec 03 12:30:36 crc kubenswrapper[5052]: E1203 12:30:36.012529 5052 cadvisor_stats_provider.go:516] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod48831cad_910f_46f1_aefe_7cc4cbfa984f.slice\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod48831cad_910f_46f1_aefe_7cc4cbfa984f.slice/crio-1f06f16f71b99e617383aa251524a1b2334b34f46ead54b868e09f13e83c19cc\": RecentStats: unable to find data in memory cache]" Dec 03 12:30:36 crc kubenswrapper[5052]: I1203 12:30:36.184569 5052 pod_container_manager_linux.go:210] "Failed to delete cgroup paths" cgroupName=["kubepods","besteffort","pod82bcca5b-e2c3-43ce-aba3-eb879796dc3e"] err="unable to destroy cgroup paths for cgroup [kubepods besteffort pod82bcca5b-e2c3-43ce-aba3-eb879796dc3e] : Timed out while waiting for systemd to remove kubepods-besteffort-pod82bcca5b_e2c3_43ce_aba3_eb879796dc3e.slice" Dec 03 12:30:36 crc kubenswrapper[5052]: E1203 12:30:36.184622 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to delete cgroup paths for [kubepods besteffort pod82bcca5b-e2c3-43ce-aba3-eb879796dc3e] : unable to destroy cgroup paths for cgroup [kubepods besteffort pod82bcca5b-e2c3-43ce-aba3-eb879796dc3e] : Timed out while waiting for systemd to remove kubepods-besteffort-pod82bcca5b_e2c3_43ce_aba3_eb879796dc3e.slice" pod="openstack-kuttl-tests/glance-default-internal-api-0" podUID="82bcca5b-e2c3-43ce-aba3-eb879796dc3e" Dec 03 12:30:36 crc kubenswrapper[5052]: I1203 12:30:36.196738 5052 pod_container_manager_linux.go:210] "Failed to delete cgroup paths" cgroupName=["kubepods","besteffort","pod61cad274-57bd-4993-aaa6-3e6013900d88"] err="unable to destroy cgroup paths for cgroup [kubepods besteffort pod61cad274-57bd-4993-aaa6-3e6013900d88] : Timed out while waiting for systemd to remove kubepods-besteffort-pod61cad274_57bd_4993_aaa6_3e6013900d88.slice" Dec 03 12:30:36 crc kubenswrapper[5052]: E1203 12:30:36.196795 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to delete cgroup paths for [kubepods besteffort pod61cad274-57bd-4993-aaa6-3e6013900d88] : unable to destroy cgroup paths for cgroup [kubepods besteffort pod61cad274-57bd-4993-aaa6-3e6013900d88] : Timed out while waiting for systemd to remove kubepods-besteffort-pod61cad274_57bd_4993_aaa6_3e6013900d88.slice" pod="openstack-kuttl-tests/cinder-api-0" podUID="61cad274-57bd-4993-aaa6-3e6013900d88" Dec 03 12:30:36 crc kubenswrapper[5052]: I1203 12:30:36.205158 5052 pod_container_manager_linux.go:210] "Failed to delete cgroup paths" cgroupName=["kubepods","besteffort","pod82bcca5b-e2c3-43ce-aba3-eb879796dc3e"] err="unable to destroy cgroup paths for cgroup [kubepods besteffort pod82bcca5b-e2c3-43ce-aba3-eb879796dc3e] : Timed out while waiting for systemd to remove kubepods-besteffort-pod82bcca5b_e2c3_43ce_aba3_eb879796dc3e.slice" Dec 03 12:30:36 crc kubenswrapper[5052]: I1203 12:30:36.958463 5052 generic.go:334] "Generic (PLEG): container finished" podID="a24b0824-26d0-4b81-b6cb-d617c36d45ab" containerID="c16bebd7126caa95cf323caf300318d8818542b3e43480cb466bf73872c95fec" exitCode=0 Dec 03 12:30:36 crc kubenswrapper[5052]: I1203 12:30:36.958505 5052 generic.go:334] "Generic (PLEG): container finished" podID="a24b0824-26d0-4b81-b6cb-d617c36d45ab" containerID="c477cd69a766929b221a6f0497205e288c5ded0f70fe9daadbd6669d7a943603" exitCode=2 Dec 03 12:30:36 crc kubenswrapper[5052]: I1203 12:30:36.958550 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"a24b0824-26d0-4b81-b6cb-d617c36d45ab","Type":"ContainerDied","Data":"c16bebd7126caa95cf323caf300318d8818542b3e43480cb466bf73872c95fec"} Dec 03 12:30:36 crc kubenswrapper[5052]: I1203 12:30:36.958602 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"a24b0824-26d0-4b81-b6cb-d617c36d45ab","Type":"ContainerDied","Data":"c477cd69a766929b221a6f0497205e288c5ded0f70fe9daadbd6669d7a943603"} Dec 03 12:30:36 crc kubenswrapper[5052]: I1203 12:30:36.958622 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"a24b0824-26d0-4b81-b6cb-d617c36d45ab","Type":"ContainerDied","Data":"e765959f449b54c23591a65f292f2f7a5099d71fa4f0ea45d8c2f901334b7206"} Dec 03 12:30:36 crc kubenswrapper[5052]: I1203 12:30:36.958567 5052 generic.go:334] "Generic (PLEG): container finished" podID="a24b0824-26d0-4b81-b6cb-d617c36d45ab" containerID="e765959f449b54c23591a65f292f2f7a5099d71fa4f0ea45d8c2f901334b7206" exitCode=0 Dec 03 12:30:36 crc kubenswrapper[5052]: I1203 12:30:36.958672 5052 generic.go:334] "Generic (PLEG): container finished" podID="a24b0824-26d0-4b81-b6cb-d617c36d45ab" containerID="4b3af605b0d8c2dcd2ab4507ac26ec3ce43fc470db20730adce6dbd29442e123" exitCode=0 Dec 03 12:30:36 crc kubenswrapper[5052]: I1203 12:30:36.958726 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"a24b0824-26d0-4b81-b6cb-d617c36d45ab","Type":"ContainerDied","Data":"4b3af605b0d8c2dcd2ab4507ac26ec3ce43fc470db20730adce6dbd29442e123"} Dec 03 12:30:36 crc kubenswrapper[5052]: I1203 12:30:36.960733 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:30:36 crc kubenswrapper[5052]: I1203 12:30:36.960756 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:30:37 crc kubenswrapper[5052]: I1203 12:30:37.033913 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 12:30:37 crc kubenswrapper[5052]: I1203 12:30:37.064782 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 12:30:37 crc kubenswrapper[5052]: I1203 12:30:37.103676 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:30:37 crc kubenswrapper[5052]: I1203 12:30:37.119660 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:30:37 crc kubenswrapper[5052]: I1203 12:30:37.135080 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 12:30:37 crc kubenswrapper[5052]: E1203 12:30:37.135663 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="a65652a1-62fa-40ff-8b8a-940b2cc795b7" containerName="barbican-worker" Dec 03 12:30:37 crc kubenswrapper[5052]: I1203 12:30:37.135687 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="a65652a1-62fa-40ff-8b8a-940b2cc795b7" containerName="barbican-worker" Dec 03 12:30:37 crc kubenswrapper[5052]: E1203 12:30:37.135712 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="a65652a1-62fa-40ff-8b8a-940b2cc795b7" containerName="barbican-worker-log" Dec 03 12:30:37 crc kubenswrapper[5052]: I1203 12:30:37.135721 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="a65652a1-62fa-40ff-8b8a-940b2cc795b7" containerName="barbican-worker-log" Dec 03 12:30:37 crc kubenswrapper[5052]: E1203 12:30:37.135732 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="aed41b09-319e-466c-9eb9-38208e1880a3" containerName="barbican-keystone-listener" Dec 03 12:30:37 crc kubenswrapper[5052]: I1203 12:30:37.135739 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="aed41b09-319e-466c-9eb9-38208e1880a3" containerName="barbican-keystone-listener" Dec 03 12:30:37 crc kubenswrapper[5052]: E1203 12:30:37.135757 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="ff58ae3c-d75e-4038-be70-00f88178a18f" containerName="barbican-api-log" Dec 03 12:30:37 crc kubenswrapper[5052]: I1203 12:30:37.135764 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="ff58ae3c-d75e-4038-be70-00f88178a18f" containerName="barbican-api-log" Dec 03 12:30:37 crc kubenswrapper[5052]: E1203 12:30:37.135781 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="ff58ae3c-d75e-4038-be70-00f88178a18f" containerName="barbican-api" Dec 03 12:30:37 crc kubenswrapper[5052]: I1203 12:30:37.135788 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="ff58ae3c-d75e-4038-be70-00f88178a18f" containerName="barbican-api" Dec 03 12:30:37 crc kubenswrapper[5052]: E1203 12:30:37.135814 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="48831cad-910f-46f1-aefe-7cc4cbfa984f" containerName="barbican-api-log" Dec 03 12:30:37 crc kubenswrapper[5052]: I1203 12:30:37.135821 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="48831cad-910f-46f1-aefe-7cc4cbfa984f" containerName="barbican-api-log" Dec 03 12:30:37 crc kubenswrapper[5052]: E1203 12:30:37.135836 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="aed41b09-319e-466c-9eb9-38208e1880a3" containerName="barbican-keystone-listener-log" Dec 03 12:30:37 crc kubenswrapper[5052]: I1203 12:30:37.135843 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="aed41b09-319e-466c-9eb9-38208e1880a3" containerName="barbican-keystone-listener-log" Dec 03 12:30:37 crc kubenswrapper[5052]: E1203 12:30:37.135870 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="48831cad-910f-46f1-aefe-7cc4cbfa984f" containerName="barbican-api" Dec 03 12:30:37 crc kubenswrapper[5052]: I1203 12:30:37.135877 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="48831cad-910f-46f1-aefe-7cc4cbfa984f" containerName="barbican-api" Dec 03 12:30:37 crc kubenswrapper[5052]: I1203 12:30:37.136089 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="aed41b09-319e-466c-9eb9-38208e1880a3" containerName="barbican-keystone-listener-log" Dec 03 12:30:37 crc kubenswrapper[5052]: I1203 12:30:37.136153 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="a65652a1-62fa-40ff-8b8a-940b2cc795b7" containerName="barbican-worker-log" Dec 03 12:30:37 crc kubenswrapper[5052]: I1203 12:30:37.136174 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="a65652a1-62fa-40ff-8b8a-940b2cc795b7" containerName="barbican-worker" Dec 03 12:30:37 crc kubenswrapper[5052]: I1203 12:30:37.136187 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="48831cad-910f-46f1-aefe-7cc4cbfa984f" containerName="barbican-api-log" Dec 03 12:30:37 crc kubenswrapper[5052]: I1203 12:30:37.136195 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="48831cad-910f-46f1-aefe-7cc4cbfa984f" containerName="barbican-api" Dec 03 12:30:37 crc kubenswrapper[5052]: I1203 12:30:37.136224 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="ff58ae3c-d75e-4038-be70-00f88178a18f" containerName="barbican-api-log" Dec 03 12:30:37 crc kubenswrapper[5052]: I1203 12:30:37.136238 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="ff58ae3c-d75e-4038-be70-00f88178a18f" containerName="barbican-api" Dec 03 12:30:37 crc kubenswrapper[5052]: I1203 12:30:37.136253 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="aed41b09-319e-466c-9eb9-38208e1880a3" containerName="barbican-keystone-listener" Dec 03 12:30:37 crc kubenswrapper[5052]: I1203 12:30:37.137764 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:30:37 crc kubenswrapper[5052]: I1203 12:30:37.143418 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-cinder-internal-svc" Dec 03 12:30:37 crc kubenswrapper[5052]: I1203 12:30:37.143552 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-cinder-public-svc" Dec 03 12:30:37 crc kubenswrapper[5052]: I1203 12:30:37.146638 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cinder-api-config-data" Dec 03 12:30:37 crc kubenswrapper[5052]: I1203 12:30:37.157082 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:30:37 crc kubenswrapper[5052]: I1203 12:30:37.162782 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:30:37 crc kubenswrapper[5052]: I1203 12:30:37.167374 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-glance-default-internal-svc" Dec 03 12:30:37 crc kubenswrapper[5052]: I1203 12:30:37.167599 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-default-internal-config-data" Dec 03 12:30:37 crc kubenswrapper[5052]: I1203 12:30:37.173031 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 12:30:37 crc kubenswrapper[5052]: I1203 12:30:37.186376 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:30:37 crc kubenswrapper[5052]: I1203 12:30:37.192321 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6e0f19c7-34ae-4d8e-9a19-755ae1c20c1c-config-data\") pod \"cinder-api-0\" (UID: \"6e0f19c7-34ae-4d8e-9a19-755ae1c20c1c\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:30:37 crc kubenswrapper[5052]: I1203 12:30:37.192366 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/6e0f19c7-34ae-4d8e-9a19-755ae1c20c1c-logs\") pod \"cinder-api-0\" (UID: \"6e0f19c7-34ae-4d8e-9a19-755ae1c20c1c\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:30:37 crc kubenswrapper[5052]: I1203 12:30:37.192494 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/6e0f19c7-34ae-4d8e-9a19-755ae1c20c1c-etc-machine-id\") pod \"cinder-api-0\" (UID: \"6e0f19c7-34ae-4d8e-9a19-755ae1c20c1c\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:30:37 crc kubenswrapper[5052]: I1203 12:30:37.192636 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/6e0f19c7-34ae-4d8e-9a19-755ae1c20c1c-config-data-custom\") pod \"cinder-api-0\" (UID: \"6e0f19c7-34ae-4d8e-9a19-755ae1c20c1c\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:30:37 crc kubenswrapper[5052]: I1203 12:30:37.193248 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6e0f19c7-34ae-4d8e-9a19-755ae1c20c1c-combined-ca-bundle\") pod \"cinder-api-0\" (UID: \"6e0f19c7-34ae-4d8e-9a19-755ae1c20c1c\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:30:37 crc kubenswrapper[5052]: I1203 12:30:37.193323 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/6e0f19c7-34ae-4d8e-9a19-755ae1c20c1c-scripts\") pod \"cinder-api-0\" (UID: \"6e0f19c7-34ae-4d8e-9a19-755ae1c20c1c\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:30:37 crc kubenswrapper[5052]: I1203 12:30:37.222780 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/cinder-scheduler-0" podUID="bb1723c7-4e41-4b0e-b6b8-f8024dafdaee" containerName="cinder-scheduler" probeResult="failure" output="Get \"http://10.217.0.222:8080/\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Dec 03 12:30:37 crc kubenswrapper[5052]: I1203 12:30:37.294666 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/5737900d-97ae-4dcb-a65c-b40fe30cdc50-internal-tls-certs\") pod \"glance-default-internal-api-0\" (UID: \"5737900d-97ae-4dcb-a65c-b40fe30cdc50\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:30:37 crc kubenswrapper[5052]: I1203 12:30:37.294727 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/6e0f19c7-34ae-4d8e-9a19-755ae1c20c1c-scripts\") pod \"cinder-api-0\" (UID: \"6e0f19c7-34ae-4d8e-9a19-755ae1c20c1c\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:30:37 crc kubenswrapper[5052]: I1203 12:30:37.294767 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/5737900d-97ae-4dcb-a65c-b40fe30cdc50-httpd-run\") pod \"glance-default-internal-api-0\" (UID: \"5737900d-97ae-4dcb-a65c-b40fe30cdc50\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:30:37 crc kubenswrapper[5052]: I1203 12:30:37.294793 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/6e0f19c7-34ae-4d8e-9a19-755ae1c20c1c-public-tls-certs\") pod \"cinder-api-0\" (UID: \"6e0f19c7-34ae-4d8e-9a19-755ae1c20c1c\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:30:37 crc kubenswrapper[5052]: I1203 12:30:37.294834 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6e0f19c7-34ae-4d8e-9a19-755ae1c20c1c-config-data\") pod \"cinder-api-0\" (UID: \"6e0f19c7-34ae-4d8e-9a19-755ae1c20c1c\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:30:37 crc kubenswrapper[5052]: I1203 12:30:37.294850 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5737900d-97ae-4dcb-a65c-b40fe30cdc50-combined-ca-bundle\") pod \"glance-default-internal-api-0\" (UID: \"5737900d-97ae-4dcb-a65c-b40fe30cdc50\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:30:37 crc kubenswrapper[5052]: I1203 12:30:37.294878 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/6e0f19c7-34ae-4d8e-9a19-755ae1c20c1c-logs\") pod \"cinder-api-0\" (UID: \"6e0f19c7-34ae-4d8e-9a19-755ae1c20c1c\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:30:37 crc kubenswrapper[5052]: I1203 12:30:37.294908 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage09-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage09-crc\") pod \"glance-default-internal-api-0\" (UID: \"5737900d-97ae-4dcb-a65c-b40fe30cdc50\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:30:37 crc kubenswrapper[5052]: I1203 12:30:37.294946 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d2sc9\" (UniqueName: \"kubernetes.io/projected/5737900d-97ae-4dcb-a65c-b40fe30cdc50-kube-api-access-d2sc9\") pod \"glance-default-internal-api-0\" (UID: \"5737900d-97ae-4dcb-a65c-b40fe30cdc50\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:30:37 crc kubenswrapper[5052]: I1203 12:30:37.295012 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c6l8r\" (UniqueName: \"kubernetes.io/projected/6e0f19c7-34ae-4d8e-9a19-755ae1c20c1c-kube-api-access-c6l8r\") pod \"cinder-api-0\" (UID: \"6e0f19c7-34ae-4d8e-9a19-755ae1c20c1c\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:30:37 crc kubenswrapper[5052]: I1203 12:30:37.295046 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/5737900d-97ae-4dcb-a65c-b40fe30cdc50-logs\") pod \"glance-default-internal-api-0\" (UID: \"5737900d-97ae-4dcb-a65c-b40fe30cdc50\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:30:37 crc kubenswrapper[5052]: I1203 12:30:37.295071 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/6e0f19c7-34ae-4d8e-9a19-755ae1c20c1c-etc-machine-id\") pod \"cinder-api-0\" (UID: \"6e0f19c7-34ae-4d8e-9a19-755ae1c20c1c\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:30:37 crc kubenswrapper[5052]: I1203 12:30:37.295140 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/6e0f19c7-34ae-4d8e-9a19-755ae1c20c1c-internal-tls-certs\") pod \"cinder-api-0\" (UID: \"6e0f19c7-34ae-4d8e-9a19-755ae1c20c1c\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:30:37 crc kubenswrapper[5052]: I1203 12:30:37.295166 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/5737900d-97ae-4dcb-a65c-b40fe30cdc50-scripts\") pod \"glance-default-internal-api-0\" (UID: \"5737900d-97ae-4dcb-a65c-b40fe30cdc50\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:30:37 crc kubenswrapper[5052]: I1203 12:30:37.295196 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/6e0f19c7-34ae-4d8e-9a19-755ae1c20c1c-config-data-custom\") pod \"cinder-api-0\" (UID: \"6e0f19c7-34ae-4d8e-9a19-755ae1c20c1c\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:30:37 crc kubenswrapper[5052]: I1203 12:30:37.295238 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/5737900d-97ae-4dcb-a65c-b40fe30cdc50-config-data\") pod \"glance-default-internal-api-0\" (UID: \"5737900d-97ae-4dcb-a65c-b40fe30cdc50\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:30:37 crc kubenswrapper[5052]: I1203 12:30:37.295268 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6e0f19c7-34ae-4d8e-9a19-755ae1c20c1c-combined-ca-bundle\") pod \"cinder-api-0\" (UID: \"6e0f19c7-34ae-4d8e-9a19-755ae1c20c1c\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:30:37 crc kubenswrapper[5052]: I1203 12:30:37.299712 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/6e0f19c7-34ae-4d8e-9a19-755ae1c20c1c-etc-machine-id\") pod \"cinder-api-0\" (UID: \"6e0f19c7-34ae-4d8e-9a19-755ae1c20c1c\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:30:37 crc kubenswrapper[5052]: I1203 12:30:37.300102 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/6e0f19c7-34ae-4d8e-9a19-755ae1c20c1c-logs\") pod \"cinder-api-0\" (UID: \"6e0f19c7-34ae-4d8e-9a19-755ae1c20c1c\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:30:37 crc kubenswrapper[5052]: I1203 12:30:37.301277 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6e0f19c7-34ae-4d8e-9a19-755ae1c20c1c-combined-ca-bundle\") pod \"cinder-api-0\" (UID: \"6e0f19c7-34ae-4d8e-9a19-755ae1c20c1c\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:30:37 crc kubenswrapper[5052]: I1203 12:30:37.303402 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6e0f19c7-34ae-4d8e-9a19-755ae1c20c1c-config-data\") pod \"cinder-api-0\" (UID: \"6e0f19c7-34ae-4d8e-9a19-755ae1c20c1c\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:30:37 crc kubenswrapper[5052]: I1203 12:30:37.304957 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/6e0f19c7-34ae-4d8e-9a19-755ae1c20c1c-scripts\") pod \"cinder-api-0\" (UID: \"6e0f19c7-34ae-4d8e-9a19-755ae1c20c1c\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:30:37 crc kubenswrapper[5052]: I1203 12:30:37.305133 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/6e0f19c7-34ae-4d8e-9a19-755ae1c20c1c-config-data-custom\") pod \"cinder-api-0\" (UID: \"6e0f19c7-34ae-4d8e-9a19-755ae1c20c1c\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:30:37 crc kubenswrapper[5052]: I1203 12:30:37.397248 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/5737900d-97ae-4dcb-a65c-b40fe30cdc50-config-data\") pod \"glance-default-internal-api-0\" (UID: \"5737900d-97ae-4dcb-a65c-b40fe30cdc50\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:30:37 crc kubenswrapper[5052]: I1203 12:30:37.397301 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/5737900d-97ae-4dcb-a65c-b40fe30cdc50-internal-tls-certs\") pod \"glance-default-internal-api-0\" (UID: \"5737900d-97ae-4dcb-a65c-b40fe30cdc50\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:30:37 crc kubenswrapper[5052]: I1203 12:30:37.397337 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/5737900d-97ae-4dcb-a65c-b40fe30cdc50-httpd-run\") pod \"glance-default-internal-api-0\" (UID: \"5737900d-97ae-4dcb-a65c-b40fe30cdc50\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:30:37 crc kubenswrapper[5052]: I1203 12:30:37.397359 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/6e0f19c7-34ae-4d8e-9a19-755ae1c20c1c-public-tls-certs\") pod \"cinder-api-0\" (UID: \"6e0f19c7-34ae-4d8e-9a19-755ae1c20c1c\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:30:37 crc kubenswrapper[5052]: I1203 12:30:37.397395 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5737900d-97ae-4dcb-a65c-b40fe30cdc50-combined-ca-bundle\") pod \"glance-default-internal-api-0\" (UID: \"5737900d-97ae-4dcb-a65c-b40fe30cdc50\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:30:37 crc kubenswrapper[5052]: I1203 12:30:37.397432 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage09-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage09-crc\") pod \"glance-default-internal-api-0\" (UID: \"5737900d-97ae-4dcb-a65c-b40fe30cdc50\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:30:37 crc kubenswrapper[5052]: I1203 12:30:37.397459 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-d2sc9\" (UniqueName: \"kubernetes.io/projected/5737900d-97ae-4dcb-a65c-b40fe30cdc50-kube-api-access-d2sc9\") pod \"glance-default-internal-api-0\" (UID: \"5737900d-97ae-4dcb-a65c-b40fe30cdc50\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:30:37 crc kubenswrapper[5052]: I1203 12:30:37.397483 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-c6l8r\" (UniqueName: \"kubernetes.io/projected/6e0f19c7-34ae-4d8e-9a19-755ae1c20c1c-kube-api-access-c6l8r\") pod \"cinder-api-0\" (UID: \"6e0f19c7-34ae-4d8e-9a19-755ae1c20c1c\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:30:37 crc kubenswrapper[5052]: I1203 12:30:37.397504 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/5737900d-97ae-4dcb-a65c-b40fe30cdc50-logs\") pod \"glance-default-internal-api-0\" (UID: \"5737900d-97ae-4dcb-a65c-b40fe30cdc50\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:30:37 crc kubenswrapper[5052]: I1203 12:30:37.397566 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/6e0f19c7-34ae-4d8e-9a19-755ae1c20c1c-internal-tls-certs\") pod \"cinder-api-0\" (UID: \"6e0f19c7-34ae-4d8e-9a19-755ae1c20c1c\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:30:37 crc kubenswrapper[5052]: I1203 12:30:37.397584 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/5737900d-97ae-4dcb-a65c-b40fe30cdc50-scripts\") pod \"glance-default-internal-api-0\" (UID: \"5737900d-97ae-4dcb-a65c-b40fe30cdc50\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:30:37 crc kubenswrapper[5052]: I1203 12:30:37.397939 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/5737900d-97ae-4dcb-a65c-b40fe30cdc50-httpd-run\") pod \"glance-default-internal-api-0\" (UID: \"5737900d-97ae-4dcb-a65c-b40fe30cdc50\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:30:37 crc kubenswrapper[5052]: I1203 12:30:37.398250 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/5737900d-97ae-4dcb-a65c-b40fe30cdc50-logs\") pod \"glance-default-internal-api-0\" (UID: \"5737900d-97ae-4dcb-a65c-b40fe30cdc50\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:30:37 crc kubenswrapper[5052]: I1203 12:30:37.398554 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage09-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage09-crc\") pod \"glance-default-internal-api-0\" (UID: \"5737900d-97ae-4dcb-a65c-b40fe30cdc50\") device mount path \"/mnt/openstack/pv09\"" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:30:37 crc kubenswrapper[5052]: I1203 12:30:37.403084 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/6e0f19c7-34ae-4d8e-9a19-755ae1c20c1c-internal-tls-certs\") pod \"cinder-api-0\" (UID: \"6e0f19c7-34ae-4d8e-9a19-755ae1c20c1c\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:30:37 crc kubenswrapper[5052]: I1203 12:30:37.407318 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/5737900d-97ae-4dcb-a65c-b40fe30cdc50-internal-tls-certs\") pod \"glance-default-internal-api-0\" (UID: \"5737900d-97ae-4dcb-a65c-b40fe30cdc50\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:30:37 crc kubenswrapper[5052]: I1203 12:30:37.409440 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/5737900d-97ae-4dcb-a65c-b40fe30cdc50-config-data\") pod \"glance-default-internal-api-0\" (UID: \"5737900d-97ae-4dcb-a65c-b40fe30cdc50\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:30:37 crc kubenswrapper[5052]: I1203 12:30:37.412019 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/6e0f19c7-34ae-4d8e-9a19-755ae1c20c1c-public-tls-certs\") pod \"cinder-api-0\" (UID: \"6e0f19c7-34ae-4d8e-9a19-755ae1c20c1c\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:30:37 crc kubenswrapper[5052]: I1203 12:30:37.412693 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5737900d-97ae-4dcb-a65c-b40fe30cdc50-combined-ca-bundle\") pod \"glance-default-internal-api-0\" (UID: \"5737900d-97ae-4dcb-a65c-b40fe30cdc50\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:30:37 crc kubenswrapper[5052]: I1203 12:30:37.416045 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/5737900d-97ae-4dcb-a65c-b40fe30cdc50-scripts\") pod \"glance-default-internal-api-0\" (UID: \"5737900d-97ae-4dcb-a65c-b40fe30cdc50\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:30:37 crc kubenswrapper[5052]: I1203 12:30:37.420415 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-c6l8r\" (UniqueName: \"kubernetes.io/projected/6e0f19c7-34ae-4d8e-9a19-755ae1c20c1c-kube-api-access-c6l8r\") pod \"cinder-api-0\" (UID: \"6e0f19c7-34ae-4d8e-9a19-755ae1c20c1c\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:30:37 crc kubenswrapper[5052]: I1203 12:30:37.441448 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-d2sc9\" (UniqueName: \"kubernetes.io/projected/5737900d-97ae-4dcb-a65c-b40fe30cdc50-kube-api-access-d2sc9\") pod \"glance-default-internal-api-0\" (UID: \"5737900d-97ae-4dcb-a65c-b40fe30cdc50\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:30:37 crc kubenswrapper[5052]: I1203 12:30:37.443643 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage09-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage09-crc\") pod \"glance-default-internal-api-0\" (UID: \"5737900d-97ae-4dcb-a65c-b40fe30cdc50\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:30:37 crc kubenswrapper[5052]: I1203 12:30:37.462663 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:30:37 crc kubenswrapper[5052]: I1203 12:30:37.490028 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:30:37 crc kubenswrapper[5052]: I1203 12:30:37.640368 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:30:37 crc kubenswrapper[5052]: I1203 12:30:37.694578 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="48831cad-910f-46f1-aefe-7cc4cbfa984f" path="/var/lib/kubelet/pods/48831cad-910f-46f1-aefe-7cc4cbfa984f/volumes" Dec 03 12:30:37 crc kubenswrapper[5052]: I1203 12:30:37.695363 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="61cad274-57bd-4993-aaa6-3e6013900d88" path="/var/lib/kubelet/pods/61cad274-57bd-4993-aaa6-3e6013900d88/volumes" Dec 03 12:30:37 crc kubenswrapper[5052]: I1203 12:30:37.696290 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="82bcca5b-e2c3-43ce-aba3-eb879796dc3e" path="/var/lib/kubelet/pods/82bcca5b-e2c3-43ce-aba3-eb879796dc3e/volumes" Dec 03 12:30:37 crc kubenswrapper[5052]: I1203 12:30:37.722313 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a24b0824-26d0-4b81-b6cb-d617c36d45ab-combined-ca-bundle\") pod \"a24b0824-26d0-4b81-b6cb-d617c36d45ab\" (UID: \"a24b0824-26d0-4b81-b6cb-d617c36d45ab\") " Dec 03 12:30:37 crc kubenswrapper[5052]: I1203 12:30:37.722347 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a24b0824-26d0-4b81-b6cb-d617c36d45ab-config-data\") pod \"a24b0824-26d0-4b81-b6cb-d617c36d45ab\" (UID: \"a24b0824-26d0-4b81-b6cb-d617c36d45ab\") " Dec 03 12:30:37 crc kubenswrapper[5052]: I1203 12:30:37.722390 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/a24b0824-26d0-4b81-b6cb-d617c36d45ab-ceilometer-tls-certs\") pod \"a24b0824-26d0-4b81-b6cb-d617c36d45ab\" (UID: \"a24b0824-26d0-4b81-b6cb-d617c36d45ab\") " Dec 03 12:30:37 crc kubenswrapper[5052]: I1203 12:30:37.722537 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/a24b0824-26d0-4b81-b6cb-d617c36d45ab-run-httpd\") pod \"a24b0824-26d0-4b81-b6cb-d617c36d45ab\" (UID: \"a24b0824-26d0-4b81-b6cb-d617c36d45ab\") " Dec 03 12:30:37 crc kubenswrapper[5052]: I1203 12:30:37.722582 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/a24b0824-26d0-4b81-b6cb-d617c36d45ab-scripts\") pod \"a24b0824-26d0-4b81-b6cb-d617c36d45ab\" (UID: \"a24b0824-26d0-4b81-b6cb-d617c36d45ab\") " Dec 03 12:30:37 crc kubenswrapper[5052]: I1203 12:30:37.723015 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a24b0824-26d0-4b81-b6cb-d617c36d45ab-run-httpd" (OuterVolumeSpecName: "run-httpd") pod "a24b0824-26d0-4b81-b6cb-d617c36d45ab" (UID: "a24b0824-26d0-4b81-b6cb-d617c36d45ab"). InnerVolumeSpecName "run-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:30:37 crc kubenswrapper[5052]: I1203 12:30:37.723682 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/a24b0824-26d0-4b81-b6cb-d617c36d45ab-sg-core-conf-yaml\") pod \"a24b0824-26d0-4b81-b6cb-d617c36d45ab\" (UID: \"a24b0824-26d0-4b81-b6cb-d617c36d45ab\") " Dec 03 12:30:37 crc kubenswrapper[5052]: I1203 12:30:37.723749 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qr6nj\" (UniqueName: \"kubernetes.io/projected/a24b0824-26d0-4b81-b6cb-d617c36d45ab-kube-api-access-qr6nj\") pod \"a24b0824-26d0-4b81-b6cb-d617c36d45ab\" (UID: \"a24b0824-26d0-4b81-b6cb-d617c36d45ab\") " Dec 03 12:30:37 crc kubenswrapper[5052]: I1203 12:30:37.723770 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/a24b0824-26d0-4b81-b6cb-d617c36d45ab-log-httpd\") pod \"a24b0824-26d0-4b81-b6cb-d617c36d45ab\" (UID: \"a24b0824-26d0-4b81-b6cb-d617c36d45ab\") " Dec 03 12:30:37 crc kubenswrapper[5052]: I1203 12:30:37.724394 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a24b0824-26d0-4b81-b6cb-d617c36d45ab-log-httpd" (OuterVolumeSpecName: "log-httpd") pod "a24b0824-26d0-4b81-b6cb-d617c36d45ab" (UID: "a24b0824-26d0-4b81-b6cb-d617c36d45ab"). InnerVolumeSpecName "log-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:30:37 crc kubenswrapper[5052]: I1203 12:30:37.725539 5052 reconciler_common.go:293] "Volume detached for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/a24b0824-26d0-4b81-b6cb-d617c36d45ab-run-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:37 crc kubenswrapper[5052]: I1203 12:30:37.725558 5052 reconciler_common.go:293] "Volume detached for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/a24b0824-26d0-4b81-b6cb-d617c36d45ab-log-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:37 crc kubenswrapper[5052]: I1203 12:30:37.742838 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a24b0824-26d0-4b81-b6cb-d617c36d45ab-kube-api-access-qr6nj" (OuterVolumeSpecName: "kube-api-access-qr6nj") pod "a24b0824-26d0-4b81-b6cb-d617c36d45ab" (UID: "a24b0824-26d0-4b81-b6cb-d617c36d45ab"). InnerVolumeSpecName "kube-api-access-qr6nj". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:30:37 crc kubenswrapper[5052]: I1203 12:30:37.745393 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a24b0824-26d0-4b81-b6cb-d617c36d45ab-scripts" (OuterVolumeSpecName: "scripts") pod "a24b0824-26d0-4b81-b6cb-d617c36d45ab" (UID: "a24b0824-26d0-4b81-b6cb-d617c36d45ab"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:37 crc kubenswrapper[5052]: I1203 12:30:37.779704 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a24b0824-26d0-4b81-b6cb-d617c36d45ab-sg-core-conf-yaml" (OuterVolumeSpecName: "sg-core-conf-yaml") pod "a24b0824-26d0-4b81-b6cb-d617c36d45ab" (UID: "a24b0824-26d0-4b81-b6cb-d617c36d45ab"). InnerVolumeSpecName "sg-core-conf-yaml". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:37 crc kubenswrapper[5052]: I1203 12:30:37.823753 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a24b0824-26d0-4b81-b6cb-d617c36d45ab-ceilometer-tls-certs" (OuterVolumeSpecName: "ceilometer-tls-certs") pod "a24b0824-26d0-4b81-b6cb-d617c36d45ab" (UID: "a24b0824-26d0-4b81-b6cb-d617c36d45ab"). InnerVolumeSpecName "ceilometer-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:37 crc kubenswrapper[5052]: I1203 12:30:37.828816 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/a24b0824-26d0-4b81-b6cb-d617c36d45ab-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:37 crc kubenswrapper[5052]: I1203 12:30:37.828861 5052 reconciler_common.go:293] "Volume detached for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/a24b0824-26d0-4b81-b6cb-d617c36d45ab-sg-core-conf-yaml\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:37 crc kubenswrapper[5052]: I1203 12:30:37.828870 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-qr6nj\" (UniqueName: \"kubernetes.io/projected/a24b0824-26d0-4b81-b6cb-d617c36d45ab-kube-api-access-qr6nj\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:37 crc kubenswrapper[5052]: I1203 12:30:37.828880 5052 reconciler_common.go:293] "Volume detached for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/a24b0824-26d0-4b81-b6cb-d617c36d45ab-ceilometer-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:37 crc kubenswrapper[5052]: I1203 12:30:37.853189 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a24b0824-26d0-4b81-b6cb-d617c36d45ab-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "a24b0824-26d0-4b81-b6cb-d617c36d45ab" (UID: "a24b0824-26d0-4b81-b6cb-d617c36d45ab"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:37 crc kubenswrapper[5052]: I1203 12:30:37.884148 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a24b0824-26d0-4b81-b6cb-d617c36d45ab-config-data" (OuterVolumeSpecName: "config-data") pod "a24b0824-26d0-4b81-b6cb-d617c36d45ab" (UID: "a24b0824-26d0-4b81-b6cb-d617c36d45ab"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:37 crc kubenswrapper[5052]: I1203 12:30:37.930819 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a24b0824-26d0-4b81-b6cb-d617c36d45ab-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:37 crc kubenswrapper[5052]: I1203 12:30:37.930857 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a24b0824-26d0-4b81-b6cb-d617c36d45ab-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:37 crc kubenswrapper[5052]: I1203 12:30:37.982775 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_neutron-db474c7dd-tg7mq_886ea0a4-5096-4ea8-a74c-de243d1f4f40/neutron-api/0.log" Dec 03 12:30:37 crc kubenswrapper[5052]: I1203 12:30:37.982826 5052 generic.go:334] "Generic (PLEG): container finished" podID="886ea0a4-5096-4ea8-a74c-de243d1f4f40" containerID="36c7c94da2f3146bb43c39fdf96923c24534dd36b2033a6240c6be99921f9ca4" exitCode=137 Dec 03 12:30:37 crc kubenswrapper[5052]: I1203 12:30:37.982885 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-db474c7dd-tg7mq" event={"ID":"886ea0a4-5096-4ea8-a74c-de243d1f4f40","Type":"ContainerDied","Data":"36c7c94da2f3146bb43c39fdf96923c24534dd36b2033a6240c6be99921f9ca4"} Dec 03 12:30:37 crc kubenswrapper[5052]: I1203 12:30:37.986422 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"a24b0824-26d0-4b81-b6cb-d617c36d45ab","Type":"ContainerDied","Data":"a7a1fed3037d074c3c5826d734310aca486f80ded95efe62964a8c98bb827927"} Dec 03 12:30:37 crc kubenswrapper[5052]: I1203 12:30:37.986482 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:30:37 crc kubenswrapper[5052]: I1203 12:30:37.986552 5052 scope.go:117] "RemoveContainer" containerID="c16bebd7126caa95cf323caf300318d8818542b3e43480cb466bf73872c95fec" Dec 03 12:30:38 crc kubenswrapper[5052]: I1203 12:30:38.026888 5052 scope.go:117] "RemoveContainer" containerID="c477cd69a766929b221a6f0497205e288c5ded0f70fe9daadbd6669d7a943603" Dec 03 12:30:38 crc kubenswrapper[5052]: I1203 12:30:38.050030 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 12:30:38 crc kubenswrapper[5052]: I1203 12:30:38.063902 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:30:38 crc kubenswrapper[5052]: I1203 12:30:38.076824 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:30:38 crc kubenswrapper[5052]: I1203 12:30:38.084532 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:30:38 crc kubenswrapper[5052]: I1203 12:30:38.086730 5052 scope.go:117] "RemoveContainer" containerID="e765959f449b54c23591a65f292f2f7a5099d71fa4f0ea45d8c2f901334b7206" Dec 03 12:30:38 crc kubenswrapper[5052]: I1203 12:30:38.108896 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:30:38 crc kubenswrapper[5052]: E1203 12:30:38.109569 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="a24b0824-26d0-4b81-b6cb-d617c36d45ab" containerName="ceilometer-central-agent" Dec 03 12:30:38 crc kubenswrapper[5052]: I1203 12:30:38.109585 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="a24b0824-26d0-4b81-b6cb-d617c36d45ab" containerName="ceilometer-central-agent" Dec 03 12:30:38 crc kubenswrapper[5052]: E1203 12:30:38.109617 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="a24b0824-26d0-4b81-b6cb-d617c36d45ab" containerName="sg-core" Dec 03 12:30:38 crc kubenswrapper[5052]: I1203 12:30:38.109623 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="a24b0824-26d0-4b81-b6cb-d617c36d45ab" containerName="sg-core" Dec 03 12:30:38 crc kubenswrapper[5052]: E1203 12:30:38.109634 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="a24b0824-26d0-4b81-b6cb-d617c36d45ab" containerName="proxy-httpd" Dec 03 12:30:38 crc kubenswrapper[5052]: I1203 12:30:38.109640 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="a24b0824-26d0-4b81-b6cb-d617c36d45ab" containerName="proxy-httpd" Dec 03 12:30:38 crc kubenswrapper[5052]: E1203 12:30:38.109666 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="a24b0824-26d0-4b81-b6cb-d617c36d45ab" containerName="ceilometer-notification-agent" Dec 03 12:30:38 crc kubenswrapper[5052]: I1203 12:30:38.109673 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="a24b0824-26d0-4b81-b6cb-d617c36d45ab" containerName="ceilometer-notification-agent" Dec 03 12:30:38 crc kubenswrapper[5052]: I1203 12:30:38.109828 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="a24b0824-26d0-4b81-b6cb-d617c36d45ab" containerName="ceilometer-notification-agent" Dec 03 12:30:38 crc kubenswrapper[5052]: I1203 12:30:38.109856 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="a24b0824-26d0-4b81-b6cb-d617c36d45ab" containerName="proxy-httpd" Dec 03 12:30:38 crc kubenswrapper[5052]: I1203 12:30:38.109866 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="a24b0824-26d0-4b81-b6cb-d617c36d45ab" containerName="ceilometer-central-agent" Dec 03 12:30:38 crc kubenswrapper[5052]: I1203 12:30:38.109875 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="a24b0824-26d0-4b81-b6cb-d617c36d45ab" containerName="sg-core" Dec 03 12:30:38 crc kubenswrapper[5052]: I1203 12:30:38.111559 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:30:38 crc kubenswrapper[5052]: I1203 12:30:38.113878 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-ceilometer-internal-svc" Dec 03 12:30:38 crc kubenswrapper[5052]: I1203 12:30:38.114041 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-scripts" Dec 03 12:30:38 crc kubenswrapper[5052]: I1203 12:30:38.119379 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-config-data" Dec 03 12:30:38 crc kubenswrapper[5052]: I1203 12:30:38.121382 5052 scope.go:117] "RemoveContainer" containerID="4b3af605b0d8c2dcd2ab4507ac26ec3ce43fc470db20730adce6dbd29442e123" Dec 03 12:30:38 crc kubenswrapper[5052]: I1203 12:30:38.124164 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:30:38 crc kubenswrapper[5052]: I1203 12:30:38.137101 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/f9193a3b-a9c6-4343-a234-1616f73404cf-ceilometer-tls-certs\") pod \"ceilometer-0\" (UID: \"f9193a3b-a9c6-4343-a234-1616f73404cf\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:30:38 crc kubenswrapper[5052]: I1203 12:30:38.137163 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/f9193a3b-a9c6-4343-a234-1616f73404cf-scripts\") pod \"ceilometer-0\" (UID: \"f9193a3b-a9c6-4343-a234-1616f73404cf\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:30:38 crc kubenswrapper[5052]: I1203 12:30:38.137227 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vnbhc\" (UniqueName: \"kubernetes.io/projected/f9193a3b-a9c6-4343-a234-1616f73404cf-kube-api-access-vnbhc\") pod \"ceilometer-0\" (UID: \"f9193a3b-a9c6-4343-a234-1616f73404cf\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:30:38 crc kubenswrapper[5052]: I1203 12:30:38.137279 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/f9193a3b-a9c6-4343-a234-1616f73404cf-config-data\") pod \"ceilometer-0\" (UID: \"f9193a3b-a9c6-4343-a234-1616f73404cf\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:30:38 crc kubenswrapper[5052]: I1203 12:30:38.137356 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/f9193a3b-a9c6-4343-a234-1616f73404cf-run-httpd\") pod \"ceilometer-0\" (UID: \"f9193a3b-a9c6-4343-a234-1616f73404cf\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:30:38 crc kubenswrapper[5052]: I1203 12:30:38.137415 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/f9193a3b-a9c6-4343-a234-1616f73404cf-log-httpd\") pod \"ceilometer-0\" (UID: \"f9193a3b-a9c6-4343-a234-1616f73404cf\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:30:38 crc kubenswrapper[5052]: I1203 12:30:38.137437 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f9193a3b-a9c6-4343-a234-1616f73404cf-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"f9193a3b-a9c6-4343-a234-1616f73404cf\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:30:38 crc kubenswrapper[5052]: I1203 12:30:38.137464 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/f9193a3b-a9c6-4343-a234-1616f73404cf-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"f9193a3b-a9c6-4343-a234-1616f73404cf\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:30:38 crc kubenswrapper[5052]: I1203 12:30:38.239433 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/f9193a3b-a9c6-4343-a234-1616f73404cf-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"f9193a3b-a9c6-4343-a234-1616f73404cf\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:30:38 crc kubenswrapper[5052]: I1203 12:30:38.239790 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/f9193a3b-a9c6-4343-a234-1616f73404cf-ceilometer-tls-certs\") pod \"ceilometer-0\" (UID: \"f9193a3b-a9c6-4343-a234-1616f73404cf\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:30:38 crc kubenswrapper[5052]: I1203 12:30:38.239835 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/f9193a3b-a9c6-4343-a234-1616f73404cf-scripts\") pod \"ceilometer-0\" (UID: \"f9193a3b-a9c6-4343-a234-1616f73404cf\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:30:38 crc kubenswrapper[5052]: I1203 12:30:38.239878 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-vnbhc\" (UniqueName: \"kubernetes.io/projected/f9193a3b-a9c6-4343-a234-1616f73404cf-kube-api-access-vnbhc\") pod \"ceilometer-0\" (UID: \"f9193a3b-a9c6-4343-a234-1616f73404cf\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:30:38 crc kubenswrapper[5052]: I1203 12:30:38.239902 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/f9193a3b-a9c6-4343-a234-1616f73404cf-config-data\") pod \"ceilometer-0\" (UID: \"f9193a3b-a9c6-4343-a234-1616f73404cf\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:30:38 crc kubenswrapper[5052]: I1203 12:30:38.239962 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/f9193a3b-a9c6-4343-a234-1616f73404cf-run-httpd\") pod \"ceilometer-0\" (UID: \"f9193a3b-a9c6-4343-a234-1616f73404cf\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:30:38 crc kubenswrapper[5052]: I1203 12:30:38.239978 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/f9193a3b-a9c6-4343-a234-1616f73404cf-log-httpd\") pod \"ceilometer-0\" (UID: \"f9193a3b-a9c6-4343-a234-1616f73404cf\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:30:38 crc kubenswrapper[5052]: I1203 12:30:38.240003 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f9193a3b-a9c6-4343-a234-1616f73404cf-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"f9193a3b-a9c6-4343-a234-1616f73404cf\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:30:38 crc kubenswrapper[5052]: I1203 12:30:38.241810 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/f9193a3b-a9c6-4343-a234-1616f73404cf-run-httpd\") pod \"ceilometer-0\" (UID: \"f9193a3b-a9c6-4343-a234-1616f73404cf\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:30:38 crc kubenswrapper[5052]: I1203 12:30:38.242076 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/f9193a3b-a9c6-4343-a234-1616f73404cf-log-httpd\") pod \"ceilometer-0\" (UID: \"f9193a3b-a9c6-4343-a234-1616f73404cf\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:30:38 crc kubenswrapper[5052]: I1203 12:30:38.245869 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f9193a3b-a9c6-4343-a234-1616f73404cf-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"f9193a3b-a9c6-4343-a234-1616f73404cf\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:30:38 crc kubenswrapper[5052]: I1203 12:30:38.245956 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/f9193a3b-a9c6-4343-a234-1616f73404cf-scripts\") pod \"ceilometer-0\" (UID: \"f9193a3b-a9c6-4343-a234-1616f73404cf\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:30:38 crc kubenswrapper[5052]: I1203 12:30:38.246192 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/f9193a3b-a9c6-4343-a234-1616f73404cf-config-data\") pod \"ceilometer-0\" (UID: \"f9193a3b-a9c6-4343-a234-1616f73404cf\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:30:38 crc kubenswrapper[5052]: I1203 12:30:38.246793 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/f9193a3b-a9c6-4343-a234-1616f73404cf-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"f9193a3b-a9c6-4343-a234-1616f73404cf\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:30:38 crc kubenswrapper[5052]: I1203 12:30:38.256218 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/f9193a3b-a9c6-4343-a234-1616f73404cf-ceilometer-tls-certs\") pod \"ceilometer-0\" (UID: \"f9193a3b-a9c6-4343-a234-1616f73404cf\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:30:38 crc kubenswrapper[5052]: I1203 12:30:38.261159 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-vnbhc\" (UniqueName: \"kubernetes.io/projected/f9193a3b-a9c6-4343-a234-1616f73404cf-kube-api-access-vnbhc\") pod \"ceilometer-0\" (UID: \"f9193a3b-a9c6-4343-a234-1616f73404cf\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:30:38 crc kubenswrapper[5052]: I1203 12:30:38.370141 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_neutron-db474c7dd-tg7mq_886ea0a4-5096-4ea8-a74c-de243d1f4f40/neutron-api/0.log" Dec 03 12:30:38 crc kubenswrapper[5052]: I1203 12:30:38.370287 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-db474c7dd-tg7mq" Dec 03 12:30:38 crc kubenswrapper[5052]: I1203 12:30:38.446782 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:30:38 crc kubenswrapper[5052]: I1203 12:30:38.448944 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-czgzb\" (UniqueName: \"kubernetes.io/projected/886ea0a4-5096-4ea8-a74c-de243d1f4f40-kube-api-access-czgzb\") pod \"886ea0a4-5096-4ea8-a74c-de243d1f4f40\" (UID: \"886ea0a4-5096-4ea8-a74c-de243d1f4f40\") " Dec 03 12:30:38 crc kubenswrapper[5052]: I1203 12:30:38.449128 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/886ea0a4-5096-4ea8-a74c-de243d1f4f40-ovndb-tls-certs\") pod \"886ea0a4-5096-4ea8-a74c-de243d1f4f40\" (UID: \"886ea0a4-5096-4ea8-a74c-de243d1f4f40\") " Dec 03 12:30:38 crc kubenswrapper[5052]: I1203 12:30:38.449177 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/886ea0a4-5096-4ea8-a74c-de243d1f4f40-internal-tls-certs\") pod \"886ea0a4-5096-4ea8-a74c-de243d1f4f40\" (UID: \"886ea0a4-5096-4ea8-a74c-de243d1f4f40\") " Dec 03 12:30:38 crc kubenswrapper[5052]: I1203 12:30:38.449252 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/886ea0a4-5096-4ea8-a74c-de243d1f4f40-public-tls-certs\") pod \"886ea0a4-5096-4ea8-a74c-de243d1f4f40\" (UID: \"886ea0a4-5096-4ea8-a74c-de243d1f4f40\") " Dec 03 12:30:38 crc kubenswrapper[5052]: I1203 12:30:38.449296 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/886ea0a4-5096-4ea8-a74c-de243d1f4f40-config\") pod \"886ea0a4-5096-4ea8-a74c-de243d1f4f40\" (UID: \"886ea0a4-5096-4ea8-a74c-de243d1f4f40\") " Dec 03 12:30:38 crc kubenswrapper[5052]: I1203 12:30:38.449416 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/886ea0a4-5096-4ea8-a74c-de243d1f4f40-combined-ca-bundle\") pod \"886ea0a4-5096-4ea8-a74c-de243d1f4f40\" (UID: \"886ea0a4-5096-4ea8-a74c-de243d1f4f40\") " Dec 03 12:30:38 crc kubenswrapper[5052]: I1203 12:30:38.449453 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/886ea0a4-5096-4ea8-a74c-de243d1f4f40-httpd-config\") pod \"886ea0a4-5096-4ea8-a74c-de243d1f4f40\" (UID: \"886ea0a4-5096-4ea8-a74c-de243d1f4f40\") " Dec 03 12:30:38 crc kubenswrapper[5052]: I1203 12:30:38.462402 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/886ea0a4-5096-4ea8-a74c-de243d1f4f40-kube-api-access-czgzb" (OuterVolumeSpecName: "kube-api-access-czgzb") pod "886ea0a4-5096-4ea8-a74c-de243d1f4f40" (UID: "886ea0a4-5096-4ea8-a74c-de243d1f4f40"). InnerVolumeSpecName "kube-api-access-czgzb". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:30:38 crc kubenswrapper[5052]: I1203 12:30:38.464006 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/886ea0a4-5096-4ea8-a74c-de243d1f4f40-httpd-config" (OuterVolumeSpecName: "httpd-config") pod "886ea0a4-5096-4ea8-a74c-de243d1f4f40" (UID: "886ea0a4-5096-4ea8-a74c-de243d1f4f40"). InnerVolumeSpecName "httpd-config". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:38 crc kubenswrapper[5052]: I1203 12:30:38.550931 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/886ea0a4-5096-4ea8-a74c-de243d1f4f40-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "886ea0a4-5096-4ea8-a74c-de243d1f4f40" (UID: "886ea0a4-5096-4ea8-a74c-de243d1f4f40"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:38 crc kubenswrapper[5052]: I1203 12:30:38.553573 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/886ea0a4-5096-4ea8-a74c-de243d1f4f40-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "886ea0a4-5096-4ea8-a74c-de243d1f4f40" (UID: "886ea0a4-5096-4ea8-a74c-de243d1f4f40"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:38 crc kubenswrapper[5052]: I1203 12:30:38.553930 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-czgzb\" (UniqueName: \"kubernetes.io/projected/886ea0a4-5096-4ea8-a74c-de243d1f4f40-kube-api-access-czgzb\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:38 crc kubenswrapper[5052]: I1203 12:30:38.554012 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/886ea0a4-5096-4ea8-a74c-de243d1f4f40-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:38 crc kubenswrapper[5052]: I1203 12:30:38.554069 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/886ea0a4-5096-4ea8-a74c-de243d1f4f40-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:38 crc kubenswrapper[5052]: I1203 12:30:38.554125 5052 reconciler_common.go:293] "Volume detached for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/886ea0a4-5096-4ea8-a74c-de243d1f4f40-httpd-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:38 crc kubenswrapper[5052]: I1203 12:30:38.572491 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/886ea0a4-5096-4ea8-a74c-de243d1f4f40-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "886ea0a4-5096-4ea8-a74c-de243d1f4f40" (UID: "886ea0a4-5096-4ea8-a74c-de243d1f4f40"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:38 crc kubenswrapper[5052]: I1203 12:30:38.573717 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/886ea0a4-5096-4ea8-a74c-de243d1f4f40-config" (OuterVolumeSpecName: "config") pod "886ea0a4-5096-4ea8-a74c-de243d1f4f40" (UID: "886ea0a4-5096-4ea8-a74c-de243d1f4f40"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:38 crc kubenswrapper[5052]: I1203 12:30:38.613021 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/886ea0a4-5096-4ea8-a74c-de243d1f4f40-ovndb-tls-certs" (OuterVolumeSpecName: "ovndb-tls-certs") pod "886ea0a4-5096-4ea8-a74c-de243d1f4f40" (UID: "886ea0a4-5096-4ea8-a74c-de243d1f4f40"). InnerVolumeSpecName "ovndb-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:38 crc kubenswrapper[5052]: I1203 12:30:38.620719 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:30:38 crc kubenswrapper[5052]: I1203 12:30:38.621110 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:30:38 crc kubenswrapper[5052]: I1203 12:30:38.656104 5052 reconciler_common.go:293] "Volume detached for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/886ea0a4-5096-4ea8-a74c-de243d1f4f40-ovndb-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:38 crc kubenswrapper[5052]: I1203 12:30:38.656140 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/886ea0a4-5096-4ea8-a74c-de243d1f4f40-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:38 crc kubenswrapper[5052]: I1203 12:30:38.656150 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/886ea0a4-5096-4ea8-a74c-de243d1f4f40-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:38 crc kubenswrapper[5052]: I1203 12:30:38.673579 5052 scope.go:117] "RemoveContainer" containerID="421784ccacc82cea351bc14db49b9cace93a8a873e7c71fae104a1ba2d00d40c" Dec 03 12:30:38 crc kubenswrapper[5052]: I1203 12:30:38.727002 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:30:39 crc kubenswrapper[5052]: I1203 12:30:39.011711 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_neutron-db474c7dd-tg7mq_886ea0a4-5096-4ea8-a74c-de243d1f4f40/neutron-api/0.log" Dec 03 12:30:39 crc kubenswrapper[5052]: I1203 12:30:39.013140 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-db474c7dd-tg7mq" Dec 03 12:30:39 crc kubenswrapper[5052]: I1203 12:30:39.018230 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-db474c7dd-tg7mq" event={"ID":"886ea0a4-5096-4ea8-a74c-de243d1f4f40","Type":"ContainerDied","Data":"c4ff4fbe3dacec716771465b6ff8ae9204a17ae870e13c0be6a701515ebce0cc"} Dec 03 12:30:39 crc kubenswrapper[5052]: I1203 12:30:39.018357 5052 scope.go:117] "RemoveContainer" containerID="71e595450c5a81e73076dfa57785d77ddaafc47e00972e087f567c2fa539835b" Dec 03 12:30:39 crc kubenswrapper[5052]: I1203 12:30:39.026252 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"5737900d-97ae-4dcb-a65c-b40fe30cdc50","Type":"ContainerStarted","Data":"a973285d5356bd9dc51bc03e5f6f166c4ac926ebd9edb7f87c4de4de53693880"} Dec 03 12:30:39 crc kubenswrapper[5052]: I1203 12:30:39.026435 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"5737900d-97ae-4dcb-a65c-b40fe30cdc50","Type":"ContainerStarted","Data":"87ab449ce6632b6498e866cff688d31a02f7702881f0e2b6e60f3374d2103eab"} Dec 03 12:30:39 crc kubenswrapper[5052]: I1203 12:30:39.041568 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"6e0f19c7-34ae-4d8e-9a19-755ae1c20c1c","Type":"ContainerStarted","Data":"1b451d3cdf806d98af920f4369d1f2e05a91d7cd6f19439441f6f14eafe9ba01"} Dec 03 12:30:39 crc kubenswrapper[5052]: I1203 12:30:39.059329 5052 scope.go:117] "RemoveContainer" containerID="36c7c94da2f3146bb43c39fdf96923c24534dd36b2033a6240c6be99921f9ca4" Dec 03 12:30:39 crc kubenswrapper[5052]: I1203 12:30:39.068015 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:30:39 crc kubenswrapper[5052]: W1203 12:30:39.070436 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf9193a3b_a9c6_4343_a234_1616f73404cf.slice/crio-4eb1fdeb5b52ce5c389d9427885df5067888badb5bc0239fa6056a03765ab6ad WatchSource:0}: Error finding container 4eb1fdeb5b52ce5c389d9427885df5067888badb5bc0239fa6056a03765ab6ad: Status 404 returned error can't find the container with id 4eb1fdeb5b52ce5c389d9427885df5067888badb5bc0239fa6056a03765ab6ad Dec 03 12:30:39 crc kubenswrapper[5052]: I1203 12:30:39.074110 5052 provider.go:102] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Dec 03 12:30:39 crc kubenswrapper[5052]: I1203 12:30:39.079228 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/neutron-db474c7dd-tg7mq"] Dec 03 12:30:39 crc kubenswrapper[5052]: I1203 12:30:39.089910 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/neutron-db474c7dd-tg7mq"] Dec 03 12:30:39 crc kubenswrapper[5052]: I1203 12:30:39.231633 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:30:39 crc kubenswrapper[5052]: I1203 12:30:39.231684 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:30:39 crc kubenswrapper[5052]: I1203 12:30:39.295723 5052 prober.go:107] "Probe failed" probeType="Liveness" pod="openstack-kuttl-tests/barbican-api-68fc8f9776-n9gml" podUID="82f142ab-5be9-4727-b3c5-102d6f2401fb" containerName="barbican-api-log" probeResult="failure" output="Get \"https://10.217.0.233:9311/healthcheck\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Dec 03 12:30:39 crc kubenswrapper[5052]: I1203 12:30:39.487859 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:30:39 crc kubenswrapper[5052]: I1203 12:30:39.509997 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:30:39 crc kubenswrapper[5052]: I1203 12:30:39.699851 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="886ea0a4-5096-4ea8-a74c-de243d1f4f40" path="/var/lib/kubelet/pods/886ea0a4-5096-4ea8-a74c-de243d1f4f40/volumes" Dec 03 12:30:39 crc kubenswrapper[5052]: I1203 12:30:39.700640 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a24b0824-26d0-4b81-b6cb-d617c36d45ab" path="/var/lib/kubelet/pods/a24b0824-26d0-4b81-b6cb-d617c36d45ab/volumes" Dec 03 12:30:39 crc kubenswrapper[5052]: I1203 12:30:39.730482 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:30:39 crc kubenswrapper[5052]: I1203 12:30:39.808865 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:30:39 crc kubenswrapper[5052]: I1203 12:30:39.872704 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/barbican-api-68fc8f9776-n9gml" podUID="82f142ab-5be9-4727-b3c5-102d6f2401fb" containerName="barbican-api-log" probeResult="failure" output="Get \"https://10.217.0.233:9311/healthcheck\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Dec 03 12:30:40 crc kubenswrapper[5052]: I1203 12:30:40.050825 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"f9193a3b-a9c6-4343-a234-1616f73404cf","Type":"ContainerStarted","Data":"4eb1fdeb5b52ce5c389d9427885df5067888badb5bc0239fa6056a03765ab6ad"} Dec 03 12:30:40 crc kubenswrapper[5052]: I1203 12:30:40.052364 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"6e0f19c7-34ae-4d8e-9a19-755ae1c20c1c","Type":"ContainerStarted","Data":"fa8e6470aab88dd31d2b44427546df1694d629e5bea31eca80eadf84f064db9d"} Dec 03 12:30:40 crc kubenswrapper[5052]: I1203 12:30:40.054895 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-conductor-0" event={"ID":"2dfbec8a-3a04-407b-8145-e12a3afee2e0","Type":"ContainerStarted","Data":"9d08b86310d4fa1fec88a9c29833ad8e46ae7768dfb1a4176e3e8e54c54b84ea"} Dec 03 12:30:40 crc kubenswrapper[5052]: I1203 12:30:40.055081 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:30:40 crc kubenswrapper[5052]: I1203 12:30:40.091500 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:30:40 crc kubenswrapper[5052]: I1203 12:30:40.175448 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:30:41 crc kubenswrapper[5052]: I1203 12:30:41.105730 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:30:41 crc kubenswrapper[5052]: I1203 12:30:41.105965 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:30:41 crc kubenswrapper[5052]: I1203 12:30:41.110858 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"6e0f19c7-34ae-4d8e-9a19-755ae1c20c1c","Type":"ContainerStarted","Data":"e387dc9d87ea03a56050ef8cbe08349409d39a0cdecb55b0fe4e091ce7e2b2f4"} Dec 03 12:30:41 crc kubenswrapper[5052]: I1203 12:30:41.112026 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:30:41 crc kubenswrapper[5052]: I1203 12:30:41.130643 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"5737900d-97ae-4dcb-a65c-b40fe30cdc50","Type":"ContainerStarted","Data":"1840b34d3edddd62d0d38183e87fc8f3976fc72f58bdedd488ca0702b16fd6e7"} Dec 03 12:30:41 crc kubenswrapper[5052]: I1203 12:30:41.156258 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/cinder-api-0" podStartSLOduration=4.156233748 podStartE2EDuration="4.156233748s" podCreationTimestamp="2025-12-03 12:30:37 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:30:41.13386762 +0000 UTC m=+4231.722109384" watchObservedRunningTime="2025-12-03 12:30:41.156233748 +0000 UTC m=+4231.744475532" Dec 03 12:30:41 crc kubenswrapper[5052]: I1203 12:30:41.171850 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:30:41 crc kubenswrapper[5052]: I1203 12:30:41.176382 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:30:41 crc kubenswrapper[5052]: I1203 12:30:41.187194 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/glance-default-internal-api-0" podStartSLOduration=4.187174451 podStartE2EDuration="4.187174451s" podCreationTimestamp="2025-12-03 12:30:37 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:30:41.160748657 +0000 UTC m=+4231.748990431" watchObservedRunningTime="2025-12-03 12:30:41.187174451 +0000 UTC m=+4231.775416215" Dec 03 12:30:41 crc kubenswrapper[5052]: I1203 12:30:41.832321 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/barbican-api-68fc8f9776-n9gml" Dec 03 12:30:42 crc kubenswrapper[5052]: I1203 12:30:42.144540 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"f9193a3b-a9c6-4343-a234-1616f73404cf","Type":"ContainerStarted","Data":"29b7f054b1ac49030c5a40b904711ccba428b2c71382c48f025d25f4f0a30435"} Dec 03 12:30:42 crc kubenswrapper[5052]: I1203 12:30:42.144937 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:30:42 crc kubenswrapper[5052]: I1203 12:30:42.145182 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:30:42 crc kubenswrapper[5052]: I1203 12:30:42.265740 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/cinder-scheduler-0" podUID="bb1723c7-4e41-4b0e-b6b8-f8024dafdaee" containerName="cinder-scheduler" probeResult="failure" output="Get \"http://10.217.0.222:8080/\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Dec 03 12:30:42 crc kubenswrapper[5052]: I1203 12:30:42.294652 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/barbican-api-68fc8f9776-n9gml" podUID="82f142ab-5be9-4727-b3c5-102d6f2401fb" containerName="barbican-api" probeResult="failure" output="Get \"https://10.217.0.233:9311/healthcheck\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Dec 03 12:30:42 crc kubenswrapper[5052]: I1203 12:30:42.294970 5052 prober.go:107] "Probe failed" probeType="Liveness" pod="openstack-kuttl-tests/barbican-api-68fc8f9776-n9gml" podUID="82f142ab-5be9-4727-b3c5-102d6f2401fb" containerName="barbican-api" probeResult="failure" output="Get \"https://10.217.0.233:9311/healthcheck\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Dec 03 12:30:42 crc kubenswrapper[5052]: I1203 12:30:42.314129 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/barbican-api-68fc8f9776-n9gml" Dec 03 12:30:42 crc kubenswrapper[5052]: I1203 12:30:42.389820 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-api-6d8b99877d-pf4c5"] Dec 03 12:30:42 crc kubenswrapper[5052]: I1203 12:30:42.390297 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/barbican-api-6d8b99877d-pf4c5" podUID="2db0a4b0-75b7-4495-be94-761466b6f3b4" containerName="barbican-api-log" containerID="cri-o://a827f835542939adf9f746a906121547f47a7f5c3fb4c3ffc2ca5f8c6a877335" gracePeriod=30 Dec 03 12:30:42 crc kubenswrapper[5052]: I1203 12:30:42.390821 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/barbican-api-6d8b99877d-pf4c5" podUID="2db0a4b0-75b7-4495-be94-761466b6f3b4" containerName="barbican-api" containerID="cri-o://c56d4357a3132e1b0f7677a361c19f24c409f537f898fcb3c7fc4ac9bdcac64c" gracePeriod=30 Dec 03 12:30:42 crc kubenswrapper[5052]: I1203 12:30:42.522220 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/neutron-769cb78446-m6hzp" podUID="760aa987-6355-40cb-97c0-16c7eeddd135" containerName="neutron-httpd" probeResult="failure" output="Get \"https://10.217.0.224:9696/\": dial tcp 10.217.0.224:9696: connect: connection refused" Dec 03 12:30:42 crc kubenswrapper[5052]: I1203 12:30:42.673405 5052 scope.go:117] "RemoveContainer" containerID="96578ef7df608942efd7e88b79f20d9254a37ea8b6ab07704fddbcfab40445f1" Dec 03 12:30:42 crc kubenswrapper[5052]: I1203 12:30:42.952484 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_neutron-56f9c7c6b6-kvxpr_5f522d90-036e-4565-8293-97097912fbb1/neutron-api/0.log" Dec 03 12:30:42 crc kubenswrapper[5052]: I1203 12:30:42.952572 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-56f9c7c6b6-kvxpr" Dec 03 12:30:43 crc kubenswrapper[5052]: I1203 12:30:43.055282 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/5f522d90-036e-4565-8293-97097912fbb1-internal-tls-certs\") pod \"5f522d90-036e-4565-8293-97097912fbb1\" (UID: \"5f522d90-036e-4565-8293-97097912fbb1\") " Dec 03 12:30:43 crc kubenswrapper[5052]: I1203 12:30:43.055328 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/5f522d90-036e-4565-8293-97097912fbb1-public-tls-certs\") pod \"5f522d90-036e-4565-8293-97097912fbb1\" (UID: \"5f522d90-036e-4565-8293-97097912fbb1\") " Dec 03 12:30:43 crc kubenswrapper[5052]: I1203 12:30:43.055377 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xm6xf\" (UniqueName: \"kubernetes.io/projected/5f522d90-036e-4565-8293-97097912fbb1-kube-api-access-xm6xf\") pod \"5f522d90-036e-4565-8293-97097912fbb1\" (UID: \"5f522d90-036e-4565-8293-97097912fbb1\") " Dec 03 12:30:43 crc kubenswrapper[5052]: I1203 12:30:43.055406 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5f522d90-036e-4565-8293-97097912fbb1-combined-ca-bundle\") pod \"5f522d90-036e-4565-8293-97097912fbb1\" (UID: \"5f522d90-036e-4565-8293-97097912fbb1\") " Dec 03 12:30:43 crc kubenswrapper[5052]: I1203 12:30:43.056366 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/5f522d90-036e-4565-8293-97097912fbb1-config\") pod \"5f522d90-036e-4565-8293-97097912fbb1\" (UID: \"5f522d90-036e-4565-8293-97097912fbb1\") " Dec 03 12:30:43 crc kubenswrapper[5052]: I1203 12:30:43.056450 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/5f522d90-036e-4565-8293-97097912fbb1-httpd-config\") pod \"5f522d90-036e-4565-8293-97097912fbb1\" (UID: \"5f522d90-036e-4565-8293-97097912fbb1\") " Dec 03 12:30:43 crc kubenswrapper[5052]: I1203 12:30:43.056569 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/5f522d90-036e-4565-8293-97097912fbb1-ovndb-tls-certs\") pod \"5f522d90-036e-4565-8293-97097912fbb1\" (UID: \"5f522d90-036e-4565-8293-97097912fbb1\") " Dec 03 12:30:43 crc kubenswrapper[5052]: I1203 12:30:43.064246 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5f522d90-036e-4565-8293-97097912fbb1-kube-api-access-xm6xf" (OuterVolumeSpecName: "kube-api-access-xm6xf") pod "5f522d90-036e-4565-8293-97097912fbb1" (UID: "5f522d90-036e-4565-8293-97097912fbb1"). InnerVolumeSpecName "kube-api-access-xm6xf". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:30:43 crc kubenswrapper[5052]: I1203 12:30:43.065164 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5f522d90-036e-4565-8293-97097912fbb1-httpd-config" (OuterVolumeSpecName: "httpd-config") pod "5f522d90-036e-4565-8293-97097912fbb1" (UID: "5f522d90-036e-4565-8293-97097912fbb1"). InnerVolumeSpecName "httpd-config". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:43 crc kubenswrapper[5052]: I1203 12:30:43.132669 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5f522d90-036e-4565-8293-97097912fbb1-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "5f522d90-036e-4565-8293-97097912fbb1" (UID: "5f522d90-036e-4565-8293-97097912fbb1"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:43 crc kubenswrapper[5052]: I1203 12:30:43.154317 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5f522d90-036e-4565-8293-97097912fbb1-ovndb-tls-certs" (OuterVolumeSpecName: "ovndb-tls-certs") pod "5f522d90-036e-4565-8293-97097912fbb1" (UID: "5f522d90-036e-4565-8293-97097912fbb1"). InnerVolumeSpecName "ovndb-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:43 crc kubenswrapper[5052]: I1203 12:30:43.159037 5052 reconciler_common.go:293] "Volume detached for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/5f522d90-036e-4565-8293-97097912fbb1-httpd-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:43 crc kubenswrapper[5052]: I1203 12:30:43.159169 5052 reconciler_common.go:293] "Volume detached for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/5f522d90-036e-4565-8293-97097912fbb1-ovndb-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:43 crc kubenswrapper[5052]: I1203 12:30:43.159231 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-xm6xf\" (UniqueName: \"kubernetes.io/projected/5f522d90-036e-4565-8293-97097912fbb1-kube-api-access-xm6xf\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:43 crc kubenswrapper[5052]: I1203 12:30:43.159288 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5f522d90-036e-4565-8293-97097912fbb1-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:43 crc kubenswrapper[5052]: I1203 12:30:43.159705 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5f522d90-036e-4565-8293-97097912fbb1-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "5f522d90-036e-4565-8293-97097912fbb1" (UID: "5f522d90-036e-4565-8293-97097912fbb1"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:43 crc kubenswrapper[5052]: I1203 12:30:43.168794 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5f522d90-036e-4565-8293-97097912fbb1-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "5f522d90-036e-4565-8293-97097912fbb1" (UID: "5f522d90-036e-4565-8293-97097912fbb1"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:43 crc kubenswrapper[5052]: I1203 12:30:43.169799 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_neutron-56f9c7c6b6-kvxpr_5f522d90-036e-4565-8293-97097912fbb1/neutron-api/0.log" Dec 03 12:30:43 crc kubenswrapper[5052]: I1203 12:30:43.169843 5052 generic.go:334] "Generic (PLEG): container finished" podID="5f522d90-036e-4565-8293-97097912fbb1" containerID="83e0a3b478794ea6778ae35cae6f68d4eef32d5b5841dc00889915eed9b1dfbf" exitCode=137 Dec 03 12:30:43 crc kubenswrapper[5052]: I1203 12:30:43.169890 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-56f9c7c6b6-kvxpr" event={"ID":"5f522d90-036e-4565-8293-97097912fbb1","Type":"ContainerDied","Data":"83e0a3b478794ea6778ae35cae6f68d4eef32d5b5841dc00889915eed9b1dfbf"} Dec 03 12:30:43 crc kubenswrapper[5052]: I1203 12:30:43.169916 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-56f9c7c6b6-kvxpr" event={"ID":"5f522d90-036e-4565-8293-97097912fbb1","Type":"ContainerDied","Data":"1dda49dafe50f2de1258c72c294640965be08d91eee18557be9600402edf867a"} Dec 03 12:30:43 crc kubenswrapper[5052]: I1203 12:30:43.169932 5052 scope.go:117] "RemoveContainer" containerID="091f1e3cedc375760a2fcdc8a6147bfa5ecc673f83f421f7c9f903fa52ed8e8d" Dec 03 12:30:43 crc kubenswrapper[5052]: I1203 12:30:43.170047 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-56f9c7c6b6-kvxpr" Dec 03 12:30:43 crc kubenswrapper[5052]: I1203 12:30:43.194271 5052 generic.go:334] "Generic (PLEG): container finished" podID="2db0a4b0-75b7-4495-be94-761466b6f3b4" containerID="a827f835542939adf9f746a906121547f47a7f5c3fb4c3ffc2ca5f8c6a877335" exitCode=143 Dec 03 12:30:43 crc kubenswrapper[5052]: I1203 12:30:43.195294 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-6d8b99877d-pf4c5" event={"ID":"2db0a4b0-75b7-4495-be94-761466b6f3b4","Type":"ContainerDied","Data":"a827f835542939adf9f746a906121547f47a7f5c3fb4c3ffc2ca5f8c6a877335"} Dec 03 12:30:43 crc kubenswrapper[5052]: I1203 12:30:43.197029 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5f522d90-036e-4565-8293-97097912fbb1-config" (OuterVolumeSpecName: "config") pod "5f522d90-036e-4565-8293-97097912fbb1" (UID: "5f522d90-036e-4565-8293-97097912fbb1"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:43 crc kubenswrapper[5052]: I1203 12:30:43.261112 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/5f522d90-036e-4565-8293-97097912fbb1-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:43 crc kubenswrapper[5052]: I1203 12:30:43.261152 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/5f522d90-036e-4565-8293-97097912fbb1-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:43 crc kubenswrapper[5052]: I1203 12:30:43.261162 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/5f522d90-036e-4565-8293-97097912fbb1-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:43 crc kubenswrapper[5052]: I1203 12:30:43.266945 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/neutron-76bcd49fb6-54vvs" podUID="6bfd5f51-8b37-4fea-8134-a2cdcd4f6049" containerName="neutron-httpd" probeResult="failure" output="Get \"https://10.217.0.225:9696/\": dial tcp 10.217.0.225:9696: connect: connection refused" Dec 03 12:30:43 crc kubenswrapper[5052]: I1203 12:30:43.308583 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/keystone-6b8b855786-bltgj" Dec 03 12:30:43 crc kubenswrapper[5052]: I1203 12:30:43.377022 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/keystone-68cf4f75d7-6pwrw"] Dec 03 12:30:43 crc kubenswrapper[5052]: I1203 12:30:43.377266 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/keystone-68cf4f75d7-6pwrw" podUID="e381ca57-c1a2-4eb5-91db-760bc2c4f8d6" containerName="keystone-api" containerID="cri-o://80156d90682246d7cf6bcb1366f8a297c8108fbbbd9601898bba95b488041705" gracePeriod=30 Dec 03 12:30:43 crc kubenswrapper[5052]: I1203 12:30:43.395972 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:30:43 crc kubenswrapper[5052]: I1203 12:30:43.396014 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:30:43 crc kubenswrapper[5052]: I1203 12:30:43.512037 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/neutron-56f9c7c6b6-kvxpr"] Dec 03 12:30:43 crc kubenswrapper[5052]: I1203 12:30:43.520883 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/neutron-56f9c7c6b6-kvxpr"] Dec 03 12:30:43 crc kubenswrapper[5052]: I1203 12:30:43.605720 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:30:43 crc kubenswrapper[5052]: I1203 12:30:43.697661 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5f522d90-036e-4565-8293-97097912fbb1" path="/var/lib/kubelet/pods/5f522d90-036e-4565-8293-97097912fbb1/volumes" Dec 03 12:30:44 crc kubenswrapper[5052]: I1203 12:30:44.237657 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-metadata-0" podUID="39117149-c15d-4753-9324-8a42abe5b007" containerName="nova-metadata-metadata" probeResult="failure" output="Get \"https://10.217.0.213:8775/\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Dec 03 12:30:44 crc kubenswrapper[5052]: I1203 12:30:44.237699 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-metadata-0" podUID="39117149-c15d-4753-9324-8a42abe5b007" containerName="nova-metadata-log" probeResult="failure" output="Get \"https://10.217.0.213:8775/\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Dec 03 12:30:44 crc kubenswrapper[5052]: I1203 12:30:44.410812 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-api-0" podUID="7cbe1c2c-bc04-4232-9901-bc9d134df3d6" containerName="nova-api-api" probeResult="failure" output="Get \"https://10.217.0.236:8774/\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Dec 03 12:30:44 crc kubenswrapper[5052]: I1203 12:30:44.411167 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-api-0" podUID="7cbe1c2c-bc04-4232-9901-bc9d134df3d6" containerName="nova-api-log" probeResult="failure" output="Get \"https://10.217.0.236:8774/\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Dec 03 12:30:44 crc kubenswrapper[5052]: I1203 12:30:44.447809 5052 scope.go:117] "RemoveContainer" containerID="83e0a3b478794ea6778ae35cae6f68d4eef32d5b5841dc00889915eed9b1dfbf" Dec 03 12:30:44 crc kubenswrapper[5052]: I1203 12:30:44.501502 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/openstackclient"] Dec 03 12:30:44 crc kubenswrapper[5052]: E1203 12:30:44.502433 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="886ea0a4-5096-4ea8-a74c-de243d1f4f40" containerName="neutron-api" Dec 03 12:30:44 crc kubenswrapper[5052]: I1203 12:30:44.502483 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="886ea0a4-5096-4ea8-a74c-de243d1f4f40" containerName="neutron-api" Dec 03 12:30:44 crc kubenswrapper[5052]: E1203 12:30:44.502688 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="5f522d90-036e-4565-8293-97097912fbb1" containerName="neutron-api" Dec 03 12:30:44 crc kubenswrapper[5052]: I1203 12:30:44.502702 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f522d90-036e-4565-8293-97097912fbb1" containerName="neutron-api" Dec 03 12:30:44 crc kubenswrapper[5052]: E1203 12:30:44.502715 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="5f522d90-036e-4565-8293-97097912fbb1" containerName="neutron-httpd" Dec 03 12:30:44 crc kubenswrapper[5052]: I1203 12:30:44.502736 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f522d90-036e-4565-8293-97097912fbb1" containerName="neutron-httpd" Dec 03 12:30:44 crc kubenswrapper[5052]: E1203 12:30:44.502747 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="886ea0a4-5096-4ea8-a74c-de243d1f4f40" containerName="neutron-httpd" Dec 03 12:30:44 crc kubenswrapper[5052]: I1203 12:30:44.502753 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="886ea0a4-5096-4ea8-a74c-de243d1f4f40" containerName="neutron-httpd" Dec 03 12:30:44 crc kubenswrapper[5052]: I1203 12:30:44.502981 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="886ea0a4-5096-4ea8-a74c-de243d1f4f40" containerName="neutron-api" Dec 03 12:30:44 crc kubenswrapper[5052]: I1203 12:30:44.502994 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="5f522d90-036e-4565-8293-97097912fbb1" containerName="neutron-api" Dec 03 12:30:44 crc kubenswrapper[5052]: I1203 12:30:44.503001 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="886ea0a4-5096-4ea8-a74c-de243d1f4f40" containerName="neutron-httpd" Dec 03 12:30:44 crc kubenswrapper[5052]: I1203 12:30:44.503024 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="5f522d90-036e-4565-8293-97097912fbb1" containerName="neutron-httpd" Dec 03 12:30:44 crc kubenswrapper[5052]: I1203 12:30:44.504624 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstackclient" Dec 03 12:30:44 crc kubenswrapper[5052]: I1203 12:30:44.507326 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"openstack-config" Dec 03 12:30:44 crc kubenswrapper[5052]: I1203 12:30:44.509271 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"openstack-config-secret" Dec 03 12:30:44 crc kubenswrapper[5052]: I1203 12:30:44.514724 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"openstackclient-openstackclient-dockercfg-p42s9" Dec 03 12:30:44 crc kubenswrapper[5052]: I1203 12:30:44.526720 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/openstackclient"] Dec 03 12:30:44 crc kubenswrapper[5052]: I1203 12:30:44.553844 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/placement-65444bfc86-th5pw" podUID="cfd8af50-a644-4b24-8483-dd344f2218e4" containerName="placement-log" probeResult="failure" output="Get \"https://10.217.0.215:8778/\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Dec 03 12:30:44 crc kubenswrapper[5052]: I1203 12:30:44.599818 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openstack-config-secret\" (UniqueName: \"kubernetes.io/secret/2ed8a31a-9033-4601-84bf-9eaf57f1f155-openstack-config-secret\") pod \"openstackclient\" (UID: \"2ed8a31a-9033-4601-84bf-9eaf57f1f155\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 12:30:44 crc kubenswrapper[5052]: I1203 12:30:44.599893 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2ed8a31a-9033-4601-84bf-9eaf57f1f155-combined-ca-bundle\") pod \"openstackclient\" (UID: \"2ed8a31a-9033-4601-84bf-9eaf57f1f155\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 12:30:44 crc kubenswrapper[5052]: I1203 12:30:44.599933 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openstack-config\" (UniqueName: \"kubernetes.io/configmap/2ed8a31a-9033-4601-84bf-9eaf57f1f155-openstack-config\") pod \"openstackclient\" (UID: \"2ed8a31a-9033-4601-84bf-9eaf57f1f155\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 12:30:44 crc kubenswrapper[5052]: I1203 12:30:44.600045 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dvs55\" (UniqueName: \"kubernetes.io/projected/2ed8a31a-9033-4601-84bf-9eaf57f1f155-kube-api-access-dvs55\") pod \"openstackclient\" (UID: \"2ed8a31a-9033-4601-84bf-9eaf57f1f155\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 12:30:44 crc kubenswrapper[5052]: I1203 12:30:44.671224 5052 scope.go:117] "RemoveContainer" containerID="091f1e3cedc375760a2fcdc8a6147bfa5ecc673f83f421f7c9f903fa52ed8e8d" Dec 03 12:30:44 crc kubenswrapper[5052]: E1203 12:30:44.672155 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"091f1e3cedc375760a2fcdc8a6147bfa5ecc673f83f421f7c9f903fa52ed8e8d\": container with ID starting with 091f1e3cedc375760a2fcdc8a6147bfa5ecc673f83f421f7c9f903fa52ed8e8d not found: ID does not exist" containerID="091f1e3cedc375760a2fcdc8a6147bfa5ecc673f83f421f7c9f903fa52ed8e8d" Dec 03 12:30:44 crc kubenswrapper[5052]: I1203 12:30:44.672186 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"091f1e3cedc375760a2fcdc8a6147bfa5ecc673f83f421f7c9f903fa52ed8e8d"} err="failed to get container status \"091f1e3cedc375760a2fcdc8a6147bfa5ecc673f83f421f7c9f903fa52ed8e8d\": rpc error: code = NotFound desc = could not find container \"091f1e3cedc375760a2fcdc8a6147bfa5ecc673f83f421f7c9f903fa52ed8e8d\": container with ID starting with 091f1e3cedc375760a2fcdc8a6147bfa5ecc673f83f421f7c9f903fa52ed8e8d not found: ID does not exist" Dec 03 12:30:44 crc kubenswrapper[5052]: I1203 12:30:44.672204 5052 scope.go:117] "RemoveContainer" containerID="83e0a3b478794ea6778ae35cae6f68d4eef32d5b5841dc00889915eed9b1dfbf" Dec 03 12:30:44 crc kubenswrapper[5052]: E1203 12:30:44.672834 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"83e0a3b478794ea6778ae35cae6f68d4eef32d5b5841dc00889915eed9b1dfbf\": container with ID starting with 83e0a3b478794ea6778ae35cae6f68d4eef32d5b5841dc00889915eed9b1dfbf not found: ID does not exist" containerID="83e0a3b478794ea6778ae35cae6f68d4eef32d5b5841dc00889915eed9b1dfbf" Dec 03 12:30:44 crc kubenswrapper[5052]: I1203 12:30:44.672880 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"83e0a3b478794ea6778ae35cae6f68d4eef32d5b5841dc00889915eed9b1dfbf"} err="failed to get container status \"83e0a3b478794ea6778ae35cae6f68d4eef32d5b5841dc00889915eed9b1dfbf\": rpc error: code = NotFound desc = could not find container \"83e0a3b478794ea6778ae35cae6f68d4eef32d5b5841dc00889915eed9b1dfbf\": container with ID starting with 83e0a3b478794ea6778ae35cae6f68d4eef32d5b5841dc00889915eed9b1dfbf not found: ID does not exist" Dec 03 12:30:44 crc kubenswrapper[5052]: I1203 12:30:44.701891 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-dvs55\" (UniqueName: \"kubernetes.io/projected/2ed8a31a-9033-4601-84bf-9eaf57f1f155-kube-api-access-dvs55\") pod \"openstackclient\" (UID: \"2ed8a31a-9033-4601-84bf-9eaf57f1f155\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 12:30:44 crc kubenswrapper[5052]: I1203 12:30:44.701997 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"openstack-config-secret\" (UniqueName: \"kubernetes.io/secret/2ed8a31a-9033-4601-84bf-9eaf57f1f155-openstack-config-secret\") pod \"openstackclient\" (UID: \"2ed8a31a-9033-4601-84bf-9eaf57f1f155\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 12:30:44 crc kubenswrapper[5052]: I1203 12:30:44.702059 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2ed8a31a-9033-4601-84bf-9eaf57f1f155-combined-ca-bundle\") pod \"openstackclient\" (UID: \"2ed8a31a-9033-4601-84bf-9eaf57f1f155\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 12:30:44 crc kubenswrapper[5052]: I1203 12:30:44.702113 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"openstack-config\" (UniqueName: \"kubernetes.io/configmap/2ed8a31a-9033-4601-84bf-9eaf57f1f155-openstack-config\") pod \"openstackclient\" (UID: \"2ed8a31a-9033-4601-84bf-9eaf57f1f155\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 12:30:44 crc kubenswrapper[5052]: I1203 12:30:44.703237 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"openstack-config\" (UniqueName: \"kubernetes.io/configmap/2ed8a31a-9033-4601-84bf-9eaf57f1f155-openstack-config\") pod \"openstackclient\" (UID: \"2ed8a31a-9033-4601-84bf-9eaf57f1f155\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 12:30:44 crc kubenswrapper[5052]: I1203 12:30:44.712813 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"openstack-config-secret\" (UniqueName: \"kubernetes.io/secret/2ed8a31a-9033-4601-84bf-9eaf57f1f155-openstack-config-secret\") pod \"openstackclient\" (UID: \"2ed8a31a-9033-4601-84bf-9eaf57f1f155\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 12:30:44 crc kubenswrapper[5052]: I1203 12:30:44.713341 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2ed8a31a-9033-4601-84bf-9eaf57f1f155-combined-ca-bundle\") pod \"openstackclient\" (UID: \"2ed8a31a-9033-4601-84bf-9eaf57f1f155\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 12:30:44 crc kubenswrapper[5052]: I1203 12:30:44.724734 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-dvs55\" (UniqueName: \"kubernetes.io/projected/2ed8a31a-9033-4601-84bf-9eaf57f1f155-kube-api-access-dvs55\") pod \"openstackclient\" (UID: \"2ed8a31a-9033-4601-84bf-9eaf57f1f155\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 12:30:44 crc kubenswrapper[5052]: I1203 12:30:44.940269 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:30:44 crc kubenswrapper[5052]: I1203 12:30:44.940500 5052 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Dec 03 12:30:44 crc kubenswrapper[5052]: I1203 12:30:44.943067 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:30:44 crc kubenswrapper[5052]: I1203 12:30:44.984108 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstackclient" Dec 03 12:30:45 crc kubenswrapper[5052]: I1203 12:30:45.397889 5052 pod_container_manager_linux.go:210] "Failed to delete cgroup paths" cgroupName=["kubepods","besteffort","pod1d48b723-3f75-4d1d-83d5-8315ea5464c6"] err="unable to destroy cgroup paths for cgroup [kubepods besteffort pod1d48b723-3f75-4d1d-83d5-8315ea5464c6] : Timed out while waiting for systemd to remove kubepods-besteffort-pod1d48b723_3f75_4d1d_83d5_8315ea5464c6.slice" Dec 03 12:30:45 crc kubenswrapper[5052]: W1203 12:30:45.489752 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2ed8a31a_9033_4601_84bf_9eaf57f1f155.slice/crio-75a0ebc7b1e5db716f9f1486d32ca71a831c131400c6525e57e5d8e742b4305d WatchSource:0}: Error finding container 75a0ebc7b1e5db716f9f1486d32ca71a831c131400c6525e57e5d8e742b4305d: Status 404 returned error can't find the container with id 75a0ebc7b1e5db716f9f1486d32ca71a831c131400c6525e57e5d8e742b4305d Dec 03 12:30:45 crc kubenswrapper[5052]: I1203 12:30:45.496489 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/openstackclient"] Dec 03 12:30:45 crc kubenswrapper[5052]: I1203 12:30:45.577298 5052 prober.go:107] "Probe failed" probeType="Liveness" pod="openstack-kuttl-tests/neutron-58d7cc6794-9z49k" podUID="4b3e73a0-1320-427f-bcd7-d8062e315e39" containerName="neutron-httpd" probeResult="failure" output="HTTP probe failed with statuscode: 503" Dec 03 12:30:45 crc kubenswrapper[5052]: I1203 12:30:45.577776 5052 prober.go:107] "Probe failed" probeType="Liveness" pod="openstack-kuttl-tests/neutron-58d7cc6794-9z49k" podUID="4b3e73a0-1320-427f-bcd7-d8062e315e39" containerName="neutron-api" probeResult="failure" output="HTTP probe failed with statuscode: 503" Dec 03 12:30:45 crc kubenswrapper[5052]: I1203 12:30:45.581045 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/neutron-58d7cc6794-9z49k" podUID="4b3e73a0-1320-427f-bcd7-d8062e315e39" containerName="neutron-httpd" probeResult="failure" output="HTTP probe failed with statuscode: 503" Dec 03 12:30:46 crc kubenswrapper[5052]: I1203 12:30:46.253907 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-conductor-0" event={"ID":"e2718906-e52d-42fa-ad42-35626e99d461","Type":"ContainerStarted","Data":"db2496cc1591b72c5cf2fe6daf7733c7b2072a229f771d7bbe3144cf1984f444"} Dec 03 12:30:46 crc kubenswrapper[5052]: I1203 12:30:46.262970 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstackclient" event={"ID":"2ed8a31a-9033-4601-84bf-9eaf57f1f155","Type":"ContainerStarted","Data":"75a0ebc7b1e5db716f9f1486d32ca71a831c131400c6525e57e5d8e742b4305d"} Dec 03 12:30:46 crc kubenswrapper[5052]: I1203 12:30:46.266698 5052 generic.go:334] "Generic (PLEG): container finished" podID="2db0a4b0-75b7-4495-be94-761466b6f3b4" containerID="c56d4357a3132e1b0f7677a361c19f24c409f537f898fcb3c7fc4ac9bdcac64c" exitCode=0 Dec 03 12:30:46 crc kubenswrapper[5052]: I1203 12:30:46.266755 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-6d8b99877d-pf4c5" event={"ID":"2db0a4b0-75b7-4495-be94-761466b6f3b4","Type":"ContainerDied","Data":"c56d4357a3132e1b0f7677a361c19f24c409f537f898fcb3c7fc4ac9bdcac64c"} Dec 03 12:30:46 crc kubenswrapper[5052]: E1203 12:30:46.318050 5052 cadvisor_stats_provider.go:516] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2db0a4b0_75b7_4495_be94_761466b6f3b4.slice/crio-conmon-c56d4357a3132e1b0f7677a361c19f24c409f537f898fcb3c7fc4ac9bdcac64c.scope\": RecentStats: unable to find data in memory cache]" Dec 03 12:30:46 crc kubenswrapper[5052]: I1203 12:30:46.424070 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:30:46 crc kubenswrapper[5052]: I1203 12:30:46.516310 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/barbican-api-6d8b99877d-pf4c5" podUID="2db0a4b0-75b7-4495-be94-761466b6f3b4" containerName="barbican-api" probeResult="failure" output="Get \"https://10.217.0.149:9311/healthcheck\": dial tcp 10.217.0.149:9311: connect: connection refused" Dec 03 12:30:46 crc kubenswrapper[5052]: I1203 12:30:46.516938 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/barbican-api-6d8b99877d-pf4c5" podUID="2db0a4b0-75b7-4495-be94-761466b6f3b4" containerName="barbican-api-log" probeResult="failure" output="Get \"https://10.217.0.149:9311/healthcheck\": dial tcp 10.217.0.149:9311: connect: connection refused" Dec 03 12:30:47 crc kubenswrapper[5052]: I1203 12:30:47.284764 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"f9193a3b-a9c6-4343-a234-1616f73404cf","Type":"ContainerStarted","Data":"5eba7bcd1112f7d19e289be438b4a2d50309b5fe47adfc505eb50370db185f89"} Dec 03 12:30:47 crc kubenswrapper[5052]: I1203 12:30:47.287775 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstackclient" event={"ID":"2ed8a31a-9033-4601-84bf-9eaf57f1f155","Type":"ContainerStarted","Data":"220531c8139609a0b36f479504e8821b1f41b4615b00ee05432f9560ade3acb1"} Dec 03 12:30:47 crc kubenswrapper[5052]: I1203 12:30:47.287816 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:30:47 crc kubenswrapper[5052]: I1203 12:30:47.307692 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/cinder-scheduler-0" podUID="bb1723c7-4e41-4b0e-b6b8-f8024dafdaee" containerName="cinder-scheduler" probeResult="failure" output="Get \"http://10.217.0.222:8080/\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Dec 03 12:30:47 crc kubenswrapper[5052]: I1203 12:30:47.493941 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:30:47 crc kubenswrapper[5052]: I1203 12:30:47.494312 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:30:47 crc kubenswrapper[5052]: I1203 12:30:47.559976 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:30:47 crc kubenswrapper[5052]: I1203 12:30:47.568976 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:30:47 crc kubenswrapper[5052]: I1203 12:30:47.808814 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-api-6d8b99877d-pf4c5" Dec 03 12:30:47 crc kubenswrapper[5052]: I1203 12:30:47.876389 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-69jk5\" (UniqueName: \"kubernetes.io/projected/2db0a4b0-75b7-4495-be94-761466b6f3b4-kube-api-access-69jk5\") pod \"2db0a4b0-75b7-4495-be94-761466b6f3b4\" (UID: \"2db0a4b0-75b7-4495-be94-761466b6f3b4\") " Dec 03 12:30:47 crc kubenswrapper[5052]: I1203 12:30:47.876744 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2db0a4b0-75b7-4495-be94-761466b6f3b4-combined-ca-bundle\") pod \"2db0a4b0-75b7-4495-be94-761466b6f3b4\" (UID: \"2db0a4b0-75b7-4495-be94-761466b6f3b4\") " Dec 03 12:30:47 crc kubenswrapper[5052]: I1203 12:30:47.876888 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/2db0a4b0-75b7-4495-be94-761466b6f3b4-config-data\") pod \"2db0a4b0-75b7-4495-be94-761466b6f3b4\" (UID: \"2db0a4b0-75b7-4495-be94-761466b6f3b4\") " Dec 03 12:30:47 crc kubenswrapper[5052]: I1203 12:30:47.877054 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/2db0a4b0-75b7-4495-be94-761466b6f3b4-logs\") pod \"2db0a4b0-75b7-4495-be94-761466b6f3b4\" (UID: \"2db0a4b0-75b7-4495-be94-761466b6f3b4\") " Dec 03 12:30:47 crc kubenswrapper[5052]: I1203 12:30:47.877285 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/2db0a4b0-75b7-4495-be94-761466b6f3b4-internal-tls-certs\") pod \"2db0a4b0-75b7-4495-be94-761466b6f3b4\" (UID: \"2db0a4b0-75b7-4495-be94-761466b6f3b4\") " Dec 03 12:30:47 crc kubenswrapper[5052]: I1203 12:30:47.877461 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/2db0a4b0-75b7-4495-be94-761466b6f3b4-public-tls-certs\") pod \"2db0a4b0-75b7-4495-be94-761466b6f3b4\" (UID: \"2db0a4b0-75b7-4495-be94-761466b6f3b4\") " Dec 03 12:30:47 crc kubenswrapper[5052]: I1203 12:30:47.877648 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/2db0a4b0-75b7-4495-be94-761466b6f3b4-config-data-custom\") pod \"2db0a4b0-75b7-4495-be94-761466b6f3b4\" (UID: \"2db0a4b0-75b7-4495-be94-761466b6f3b4\") " Dec 03 12:30:47 crc kubenswrapper[5052]: I1203 12:30:47.880165 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2db0a4b0-75b7-4495-be94-761466b6f3b4-logs" (OuterVolumeSpecName: "logs") pod "2db0a4b0-75b7-4495-be94-761466b6f3b4" (UID: "2db0a4b0-75b7-4495-be94-761466b6f3b4"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:30:47 crc kubenswrapper[5052]: I1203 12:30:47.893963 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2db0a4b0-75b7-4495-be94-761466b6f3b4-kube-api-access-69jk5" (OuterVolumeSpecName: "kube-api-access-69jk5") pod "2db0a4b0-75b7-4495-be94-761466b6f3b4" (UID: "2db0a4b0-75b7-4495-be94-761466b6f3b4"). InnerVolumeSpecName "kube-api-access-69jk5". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:30:47 crc kubenswrapper[5052]: I1203 12:30:47.901138 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2db0a4b0-75b7-4495-be94-761466b6f3b4-config-data-custom" (OuterVolumeSpecName: "config-data-custom") pod "2db0a4b0-75b7-4495-be94-761466b6f3b4" (UID: "2db0a4b0-75b7-4495-be94-761466b6f3b4"). InnerVolumeSpecName "config-data-custom". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:47 crc kubenswrapper[5052]: I1203 12:30:47.920648 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2db0a4b0-75b7-4495-be94-761466b6f3b4-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "2db0a4b0-75b7-4495-be94-761466b6f3b4" (UID: "2db0a4b0-75b7-4495-be94-761466b6f3b4"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:47 crc kubenswrapper[5052]: I1203 12:30:47.926060 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2db0a4b0-75b7-4495-be94-761466b6f3b4-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "2db0a4b0-75b7-4495-be94-761466b6f3b4" (UID: "2db0a4b0-75b7-4495-be94-761466b6f3b4"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:47 crc kubenswrapper[5052]: I1203 12:30:47.953230 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2db0a4b0-75b7-4495-be94-761466b6f3b4-config-data" (OuterVolumeSpecName: "config-data") pod "2db0a4b0-75b7-4495-be94-761466b6f3b4" (UID: "2db0a4b0-75b7-4495-be94-761466b6f3b4"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:47 crc kubenswrapper[5052]: I1203 12:30:47.958423 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2db0a4b0-75b7-4495-be94-761466b6f3b4-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "2db0a4b0-75b7-4495-be94-761466b6f3b4" (UID: "2db0a4b0-75b7-4495-be94-761466b6f3b4"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:47 crc kubenswrapper[5052]: I1203 12:30:47.980527 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/2db0a4b0-75b7-4495-be94-761466b6f3b4-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:47 crc kubenswrapper[5052]: I1203 12:30:47.980567 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/2db0a4b0-75b7-4495-be94-761466b6f3b4-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:47 crc kubenswrapper[5052]: I1203 12:30:47.980584 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/2db0a4b0-75b7-4495-be94-761466b6f3b4-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:47 crc kubenswrapper[5052]: I1203 12:30:47.980597 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/2db0a4b0-75b7-4495-be94-761466b6f3b4-config-data-custom\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:47 crc kubenswrapper[5052]: I1203 12:30:47.980610 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-69jk5\" (UniqueName: \"kubernetes.io/projected/2db0a4b0-75b7-4495-be94-761466b6f3b4-kube-api-access-69jk5\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:47 crc kubenswrapper[5052]: I1203 12:30:47.980622 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2db0a4b0-75b7-4495-be94-761466b6f3b4-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:47 crc kubenswrapper[5052]: I1203 12:30:47.980634 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/2db0a4b0-75b7-4495-be94-761466b6f3b4-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:48 crc kubenswrapper[5052]: I1203 12:30:48.150339 5052 pod_container_manager_linux.go:210] "Failed to delete cgroup paths" cgroupName=["kubepods","besteffort","pod53fe6403-27db-44b7-a9a6-787afbc3bbb8"] err="unable to destroy cgroup paths for cgroup [kubepods besteffort pod53fe6403-27db-44b7-a9a6-787afbc3bbb8] : Timed out while waiting for systemd to remove kubepods-besteffort-pod53fe6403_27db_44b7_a9a6_787afbc3bbb8.slice" Dec 03 12:30:48 crc kubenswrapper[5052]: I1203 12:30:48.300138 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-api-6d8b99877d-pf4c5" Dec 03 12:30:48 crc kubenswrapper[5052]: I1203 12:30:48.300227 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-6d8b99877d-pf4c5" event={"ID":"2db0a4b0-75b7-4495-be94-761466b6f3b4","Type":"ContainerDied","Data":"6f52b869f59ba37ec0a917f4a7b67a1de39d21455470c71dd1d01458c3be2575"} Dec 03 12:30:48 crc kubenswrapper[5052]: I1203 12:30:48.300275 5052 scope.go:117] "RemoveContainer" containerID="c56d4357a3132e1b0f7677a361c19f24c409f537f898fcb3c7fc4ac9bdcac64c" Dec 03 12:30:48 crc kubenswrapper[5052]: I1203 12:30:48.302040 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:30:48 crc kubenswrapper[5052]: I1203 12:30:48.302075 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:30:48 crc kubenswrapper[5052]: I1203 12:30:48.346241 5052 scope.go:117] "RemoveContainer" containerID="a827f835542939adf9f746a906121547f47a7f5c3fb4c3ffc2ca5f8c6a877335" Dec 03 12:30:48 crc kubenswrapper[5052]: I1203 12:30:48.385527 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-api-6d8b99877d-pf4c5"] Dec 03 12:30:48 crc kubenswrapper[5052]: I1203 12:30:48.402558 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:30:48 crc kubenswrapper[5052]: I1203 12:30:48.402763 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/barbican-api-6d8b99877d-pf4c5"] Dec 03 12:30:49 crc kubenswrapper[5052]: I1203 12:30:49.314633 5052 generic.go:334] "Generic (PLEG): container finished" podID="e381ca57-c1a2-4eb5-91db-760bc2c4f8d6" containerID="80156d90682246d7cf6bcb1366f8a297c8108fbbbd9601898bba95b488041705" exitCode=0 Dec 03 12:30:49 crc kubenswrapper[5052]: I1203 12:30:49.315092 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-68cf4f75d7-6pwrw" event={"ID":"e381ca57-c1a2-4eb5-91db-760bc2c4f8d6","Type":"ContainerDied","Data":"80156d90682246d7cf6bcb1366f8a297c8108fbbbd9601898bba95b488041705"} Dec 03 12:30:49 crc kubenswrapper[5052]: I1203 12:30:49.338463 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/openstackclient" podStartSLOduration=5.338443416 podStartE2EDuration="5.338443416s" podCreationTimestamp="2025-12-03 12:30:44 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:30:49.330342074 +0000 UTC m=+4239.918583838" watchObservedRunningTime="2025-12-03 12:30:49.338443416 +0000 UTC m=+4239.926685180" Dec 03 12:30:49 crc kubenswrapper[5052]: I1203 12:30:49.411389 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/placement-65444bfc86-th5pw" Dec 03 12:30:49 crc kubenswrapper[5052]: I1203 12:30:49.411478 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/placement-65444bfc86-th5pw" Dec 03 12:30:49 crc kubenswrapper[5052]: I1203 12:30:49.492978 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/placement-7897d64b6b-j8w2r"] Dec 03 12:30:49 crc kubenswrapper[5052]: I1203 12:30:49.493491 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/placement-7897d64b6b-j8w2r" podUID="82331185-51eb-4ef5-8cf6-61dc74b73740" containerName="placement-log" containerID="cri-o://bdd07debdd83a2d2d982d8b066d0fefba55b8fee5901eb6d6479f72b63d3ae51" gracePeriod=30 Dec 03 12:30:49 crc kubenswrapper[5052]: I1203 12:30:49.493640 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/placement-7897d64b6b-j8w2r" podUID="82331185-51eb-4ef5-8cf6-61dc74b73740" containerName="placement-api" containerID="cri-o://8435a02270401c91707d9204caf291f3365f9acf4871f8d7542a8e53a028b8f7" gracePeriod=30 Dec 03 12:30:49 crc kubenswrapper[5052]: I1203 12:30:49.645183 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:30:49 crc kubenswrapper[5052]: I1203 12:30:49.696816 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2db0a4b0-75b7-4495-be94-761466b6f3b4" path="/var/lib/kubelet/pods/2db0a4b0-75b7-4495-be94-761466b6f3b4/volumes" Dec 03 12:30:50 crc kubenswrapper[5052]: I1203 12:30:50.021502 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-68cf4f75d7-6pwrw" Dec 03 12:30:50 crc kubenswrapper[5052]: I1203 12:30:50.054754 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/e381ca57-c1a2-4eb5-91db-760bc2c4f8d6-public-tls-certs\") pod \"e381ca57-c1a2-4eb5-91db-760bc2c4f8d6\" (UID: \"e381ca57-c1a2-4eb5-91db-760bc2c4f8d6\") " Dec 03 12:30:50 crc kubenswrapper[5052]: I1203 12:30:50.054843 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/e381ca57-c1a2-4eb5-91db-760bc2c4f8d6-scripts\") pod \"e381ca57-c1a2-4eb5-91db-760bc2c4f8d6\" (UID: \"e381ca57-c1a2-4eb5-91db-760bc2c4f8d6\") " Dec 03 12:30:50 crc kubenswrapper[5052]: I1203 12:30:50.054880 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e381ca57-c1a2-4eb5-91db-760bc2c4f8d6-combined-ca-bundle\") pod \"e381ca57-c1a2-4eb5-91db-760bc2c4f8d6\" (UID: \"e381ca57-c1a2-4eb5-91db-760bc2c4f8d6\") " Dec 03 12:30:50 crc kubenswrapper[5052]: I1203 12:30:50.055552 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/e381ca57-c1a2-4eb5-91db-760bc2c4f8d6-fernet-keys\") pod \"e381ca57-c1a2-4eb5-91db-760bc2c4f8d6\" (UID: \"e381ca57-c1a2-4eb5-91db-760bc2c4f8d6\") " Dec 03 12:30:50 crc kubenswrapper[5052]: I1203 12:30:50.055649 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/e381ca57-c1a2-4eb5-91db-760bc2c4f8d6-credential-keys\") pod \"e381ca57-c1a2-4eb5-91db-760bc2c4f8d6\" (UID: \"e381ca57-c1a2-4eb5-91db-760bc2c4f8d6\") " Dec 03 12:30:50 crc kubenswrapper[5052]: I1203 12:30:50.055683 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-55hzr\" (UniqueName: \"kubernetes.io/projected/e381ca57-c1a2-4eb5-91db-760bc2c4f8d6-kube-api-access-55hzr\") pod \"e381ca57-c1a2-4eb5-91db-760bc2c4f8d6\" (UID: \"e381ca57-c1a2-4eb5-91db-760bc2c4f8d6\") " Dec 03 12:30:50 crc kubenswrapper[5052]: I1203 12:30:50.055747 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/e381ca57-c1a2-4eb5-91db-760bc2c4f8d6-config-data\") pod \"e381ca57-c1a2-4eb5-91db-760bc2c4f8d6\" (UID: \"e381ca57-c1a2-4eb5-91db-760bc2c4f8d6\") " Dec 03 12:30:50 crc kubenswrapper[5052]: I1203 12:30:50.055770 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/e381ca57-c1a2-4eb5-91db-760bc2c4f8d6-internal-tls-certs\") pod \"e381ca57-c1a2-4eb5-91db-760bc2c4f8d6\" (UID: \"e381ca57-c1a2-4eb5-91db-760bc2c4f8d6\") " Dec 03 12:30:50 crc kubenswrapper[5052]: I1203 12:30:50.198357 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:30:50 crc kubenswrapper[5052]: I1203 12:30:50.327217 5052 generic.go:334] "Generic (PLEG): container finished" podID="82331185-51eb-4ef5-8cf6-61dc74b73740" containerID="bdd07debdd83a2d2d982d8b066d0fefba55b8fee5901eb6d6479f72b63d3ae51" exitCode=143 Dec 03 12:30:50 crc kubenswrapper[5052]: I1203 12:30:50.327271 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-7897d64b6b-j8w2r" event={"ID":"82331185-51eb-4ef5-8cf6-61dc74b73740","Type":"ContainerDied","Data":"bdd07debdd83a2d2d982d8b066d0fefba55b8fee5901eb6d6479f72b63d3ae51"} Dec 03 12:30:50 crc kubenswrapper[5052]: I1203 12:30:50.329087 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-68cf4f75d7-6pwrw" event={"ID":"e381ca57-c1a2-4eb5-91db-760bc2c4f8d6","Type":"ContainerDied","Data":"d19da63420f13dff7e86e98ffcfd9d0d5b7be226d32bac80e02cb52bd5e47040"} Dec 03 12:30:50 crc kubenswrapper[5052]: I1203 12:30:50.329138 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-68cf4f75d7-6pwrw" Dec 03 12:30:50 crc kubenswrapper[5052]: I1203 12:30:50.329153 5052 scope.go:117] "RemoveContainer" containerID="80156d90682246d7cf6bcb1366f8a297c8108fbbbd9601898bba95b488041705" Dec 03 12:30:50 crc kubenswrapper[5052]: I1203 12:30:50.424904 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e381ca57-c1a2-4eb5-91db-760bc2c4f8d6-scripts" (OuterVolumeSpecName: "scripts") pod "e381ca57-c1a2-4eb5-91db-760bc2c4f8d6" (UID: "e381ca57-c1a2-4eb5-91db-760bc2c4f8d6"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:50 crc kubenswrapper[5052]: I1203 12:30:50.449187 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e381ca57-c1a2-4eb5-91db-760bc2c4f8d6-fernet-keys" (OuterVolumeSpecName: "fernet-keys") pod "e381ca57-c1a2-4eb5-91db-760bc2c4f8d6" (UID: "e381ca57-c1a2-4eb5-91db-760bc2c4f8d6"). InnerVolumeSpecName "fernet-keys". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:50 crc kubenswrapper[5052]: I1203 12:30:50.449272 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e381ca57-c1a2-4eb5-91db-760bc2c4f8d6-credential-keys" (OuterVolumeSpecName: "credential-keys") pod "e381ca57-c1a2-4eb5-91db-760bc2c4f8d6" (UID: "e381ca57-c1a2-4eb5-91db-760bc2c4f8d6"). InnerVolumeSpecName "credential-keys". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:50 crc kubenswrapper[5052]: I1203 12:30:50.449285 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e381ca57-c1a2-4eb5-91db-760bc2c4f8d6-kube-api-access-55hzr" (OuterVolumeSpecName: "kube-api-access-55hzr") pod "e381ca57-c1a2-4eb5-91db-760bc2c4f8d6" (UID: "e381ca57-c1a2-4eb5-91db-760bc2c4f8d6"). InnerVolumeSpecName "kube-api-access-55hzr". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:30:50 crc kubenswrapper[5052]: I1203 12:30:50.463874 5052 reconciler_common.go:293] "Volume detached for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/e381ca57-c1a2-4eb5-91db-760bc2c4f8d6-fernet-keys\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:50 crc kubenswrapper[5052]: I1203 12:30:50.463905 5052 reconciler_common.go:293] "Volume detached for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/e381ca57-c1a2-4eb5-91db-760bc2c4f8d6-credential-keys\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:50 crc kubenswrapper[5052]: I1203 12:30:50.463916 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-55hzr\" (UniqueName: \"kubernetes.io/projected/e381ca57-c1a2-4eb5-91db-760bc2c4f8d6-kube-api-access-55hzr\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:50 crc kubenswrapper[5052]: I1203 12:30:50.463924 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/e381ca57-c1a2-4eb5-91db-760bc2c4f8d6-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:50 crc kubenswrapper[5052]: I1203 12:30:50.494053 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e381ca57-c1a2-4eb5-91db-760bc2c4f8d6-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "e381ca57-c1a2-4eb5-91db-760bc2c4f8d6" (UID: "e381ca57-c1a2-4eb5-91db-760bc2c4f8d6"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:50 crc kubenswrapper[5052]: I1203 12:30:50.500518 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e381ca57-c1a2-4eb5-91db-760bc2c4f8d6-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "e381ca57-c1a2-4eb5-91db-760bc2c4f8d6" (UID: "e381ca57-c1a2-4eb5-91db-760bc2c4f8d6"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:50 crc kubenswrapper[5052]: I1203 12:30:50.514308 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e381ca57-c1a2-4eb5-91db-760bc2c4f8d6-config-data" (OuterVolumeSpecName: "config-data") pod "e381ca57-c1a2-4eb5-91db-760bc2c4f8d6" (UID: "e381ca57-c1a2-4eb5-91db-760bc2c4f8d6"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:50 crc kubenswrapper[5052]: I1203 12:30:50.530979 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e381ca57-c1a2-4eb5-91db-760bc2c4f8d6-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "e381ca57-c1a2-4eb5-91db-760bc2c4f8d6" (UID: "e381ca57-c1a2-4eb5-91db-760bc2c4f8d6"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:50 crc kubenswrapper[5052]: I1203 12:30:50.565411 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/e381ca57-c1a2-4eb5-91db-760bc2c4f8d6-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:50 crc kubenswrapper[5052]: I1203 12:30:50.565443 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/e381ca57-c1a2-4eb5-91db-760bc2c4f8d6-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:50 crc kubenswrapper[5052]: I1203 12:30:50.565459 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/e381ca57-c1a2-4eb5-91db-760bc2c4f8d6-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:50 crc kubenswrapper[5052]: I1203 12:30:50.565471 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e381ca57-c1a2-4eb5-91db-760bc2c4f8d6-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:50 crc kubenswrapper[5052]: I1203 12:30:50.684375 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/keystone-68cf4f75d7-6pwrw"] Dec 03 12:30:50 crc kubenswrapper[5052]: I1203 12:30:50.694445 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/keystone-68cf4f75d7-6pwrw"] Dec 03 12:30:50 crc kubenswrapper[5052]: I1203 12:30:50.936043 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:30:50 crc kubenswrapper[5052]: I1203 12:30:50.936166 5052 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Dec 03 12:30:50 crc kubenswrapper[5052]: I1203 12:30:50.941574 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:30:51 crc kubenswrapper[5052]: I1203 12:30:51.682942 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e381ca57-c1a2-4eb5-91db-760bc2c4f8d6" path="/var/lib/kubelet/pods/e381ca57-c1a2-4eb5-91db-760bc2c4f8d6/volumes" Dec 03 12:30:52 crc kubenswrapper[5052]: I1203 12:30:52.178497 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/swift-proxy-55bdbb59c4-c4cjv"] Dec 03 12:30:52 crc kubenswrapper[5052]: E1203 12:30:52.178926 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="2db0a4b0-75b7-4495-be94-761466b6f3b4" containerName="barbican-api" Dec 03 12:30:52 crc kubenswrapper[5052]: I1203 12:30:52.178942 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="2db0a4b0-75b7-4495-be94-761466b6f3b4" containerName="barbican-api" Dec 03 12:30:52 crc kubenswrapper[5052]: E1203 12:30:52.178964 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="2db0a4b0-75b7-4495-be94-761466b6f3b4" containerName="barbican-api-log" Dec 03 12:30:52 crc kubenswrapper[5052]: I1203 12:30:52.178971 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="2db0a4b0-75b7-4495-be94-761466b6f3b4" containerName="barbican-api-log" Dec 03 12:30:52 crc kubenswrapper[5052]: E1203 12:30:52.179001 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="e381ca57-c1a2-4eb5-91db-760bc2c4f8d6" containerName="keystone-api" Dec 03 12:30:52 crc kubenswrapper[5052]: I1203 12:30:52.179009 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="e381ca57-c1a2-4eb5-91db-760bc2c4f8d6" containerName="keystone-api" Dec 03 12:30:52 crc kubenswrapper[5052]: I1203 12:30:52.179258 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="e381ca57-c1a2-4eb5-91db-760bc2c4f8d6" containerName="keystone-api" Dec 03 12:30:52 crc kubenswrapper[5052]: I1203 12:30:52.179282 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="2db0a4b0-75b7-4495-be94-761466b6f3b4" containerName="barbican-api-log" Dec 03 12:30:52 crc kubenswrapper[5052]: I1203 12:30:52.179291 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="2db0a4b0-75b7-4495-be94-761466b6f3b4" containerName="barbican-api" Dec 03 12:30:52 crc kubenswrapper[5052]: I1203 12:30:52.180365 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-proxy-55bdbb59c4-c4cjv" Dec 03 12:30:52 crc kubenswrapper[5052]: I1203 12:30:52.198713 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/swift-proxy-55bdbb59c4-c4cjv"] Dec 03 12:30:52 crc kubenswrapper[5052]: I1203 12:30:52.295565 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/16cac95c-d1d2-4542-bb8e-4bdc1923c684-internal-tls-certs\") pod \"swift-proxy-55bdbb59c4-c4cjv\" (UID: \"16cac95c-d1d2-4542-bb8e-4bdc1923c684\") " pod="openstack-kuttl-tests/swift-proxy-55bdbb59c4-c4cjv" Dec 03 12:30:52 crc kubenswrapper[5052]: I1203 12:30:52.295630 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/16cac95c-d1d2-4542-bb8e-4bdc1923c684-log-httpd\") pod \"swift-proxy-55bdbb59c4-c4cjv\" (UID: \"16cac95c-d1d2-4542-bb8e-4bdc1923c684\") " pod="openstack-kuttl-tests/swift-proxy-55bdbb59c4-c4cjv" Dec 03 12:30:52 crc kubenswrapper[5052]: I1203 12:30:52.295701 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s4mt4\" (UniqueName: \"kubernetes.io/projected/16cac95c-d1d2-4542-bb8e-4bdc1923c684-kube-api-access-s4mt4\") pod \"swift-proxy-55bdbb59c4-c4cjv\" (UID: \"16cac95c-d1d2-4542-bb8e-4bdc1923c684\") " pod="openstack-kuttl-tests/swift-proxy-55bdbb59c4-c4cjv" Dec 03 12:30:52 crc kubenswrapper[5052]: I1203 12:30:52.295737 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/16cac95c-d1d2-4542-bb8e-4bdc1923c684-run-httpd\") pod \"swift-proxy-55bdbb59c4-c4cjv\" (UID: \"16cac95c-d1d2-4542-bb8e-4bdc1923c684\") " pod="openstack-kuttl-tests/swift-proxy-55bdbb59c4-c4cjv" Dec 03 12:30:52 crc kubenswrapper[5052]: I1203 12:30:52.295792 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/16cac95c-d1d2-4542-bb8e-4bdc1923c684-etc-swift\") pod \"swift-proxy-55bdbb59c4-c4cjv\" (UID: \"16cac95c-d1d2-4542-bb8e-4bdc1923c684\") " pod="openstack-kuttl-tests/swift-proxy-55bdbb59c4-c4cjv" Dec 03 12:30:52 crc kubenswrapper[5052]: I1203 12:30:52.295818 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/16cac95c-d1d2-4542-bb8e-4bdc1923c684-combined-ca-bundle\") pod \"swift-proxy-55bdbb59c4-c4cjv\" (UID: \"16cac95c-d1d2-4542-bb8e-4bdc1923c684\") " pod="openstack-kuttl-tests/swift-proxy-55bdbb59c4-c4cjv" Dec 03 12:30:52 crc kubenswrapper[5052]: I1203 12:30:52.295934 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/16cac95c-d1d2-4542-bb8e-4bdc1923c684-public-tls-certs\") pod \"swift-proxy-55bdbb59c4-c4cjv\" (UID: \"16cac95c-d1d2-4542-bb8e-4bdc1923c684\") " pod="openstack-kuttl-tests/swift-proxy-55bdbb59c4-c4cjv" Dec 03 12:30:52 crc kubenswrapper[5052]: I1203 12:30:52.296082 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/16cac95c-d1d2-4542-bb8e-4bdc1923c684-config-data\") pod \"swift-proxy-55bdbb59c4-c4cjv\" (UID: \"16cac95c-d1d2-4542-bb8e-4bdc1923c684\") " pod="openstack-kuttl-tests/swift-proxy-55bdbb59c4-c4cjv" Dec 03 12:30:52 crc kubenswrapper[5052]: I1203 12:30:52.398418 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/16cac95c-d1d2-4542-bb8e-4bdc1923c684-config-data\") pod \"swift-proxy-55bdbb59c4-c4cjv\" (UID: \"16cac95c-d1d2-4542-bb8e-4bdc1923c684\") " pod="openstack-kuttl-tests/swift-proxy-55bdbb59c4-c4cjv" Dec 03 12:30:52 crc kubenswrapper[5052]: I1203 12:30:52.398619 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/16cac95c-d1d2-4542-bb8e-4bdc1923c684-internal-tls-certs\") pod \"swift-proxy-55bdbb59c4-c4cjv\" (UID: \"16cac95c-d1d2-4542-bb8e-4bdc1923c684\") " pod="openstack-kuttl-tests/swift-proxy-55bdbb59c4-c4cjv" Dec 03 12:30:52 crc kubenswrapper[5052]: I1203 12:30:52.398646 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/16cac95c-d1d2-4542-bb8e-4bdc1923c684-log-httpd\") pod \"swift-proxy-55bdbb59c4-c4cjv\" (UID: \"16cac95c-d1d2-4542-bb8e-4bdc1923c684\") " pod="openstack-kuttl-tests/swift-proxy-55bdbb59c4-c4cjv" Dec 03 12:30:52 crc kubenswrapper[5052]: I1203 12:30:52.398687 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-s4mt4\" (UniqueName: \"kubernetes.io/projected/16cac95c-d1d2-4542-bb8e-4bdc1923c684-kube-api-access-s4mt4\") pod \"swift-proxy-55bdbb59c4-c4cjv\" (UID: \"16cac95c-d1d2-4542-bb8e-4bdc1923c684\") " pod="openstack-kuttl-tests/swift-proxy-55bdbb59c4-c4cjv" Dec 03 12:30:52 crc kubenswrapper[5052]: I1203 12:30:52.398712 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/16cac95c-d1d2-4542-bb8e-4bdc1923c684-run-httpd\") pod \"swift-proxy-55bdbb59c4-c4cjv\" (UID: \"16cac95c-d1d2-4542-bb8e-4bdc1923c684\") " pod="openstack-kuttl-tests/swift-proxy-55bdbb59c4-c4cjv" Dec 03 12:30:52 crc kubenswrapper[5052]: I1203 12:30:52.398749 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/16cac95c-d1d2-4542-bb8e-4bdc1923c684-etc-swift\") pod \"swift-proxy-55bdbb59c4-c4cjv\" (UID: \"16cac95c-d1d2-4542-bb8e-4bdc1923c684\") " pod="openstack-kuttl-tests/swift-proxy-55bdbb59c4-c4cjv" Dec 03 12:30:52 crc kubenswrapper[5052]: I1203 12:30:52.398772 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/16cac95c-d1d2-4542-bb8e-4bdc1923c684-combined-ca-bundle\") pod \"swift-proxy-55bdbb59c4-c4cjv\" (UID: \"16cac95c-d1d2-4542-bb8e-4bdc1923c684\") " pod="openstack-kuttl-tests/swift-proxy-55bdbb59c4-c4cjv" Dec 03 12:30:52 crc kubenswrapper[5052]: I1203 12:30:52.398820 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/16cac95c-d1d2-4542-bb8e-4bdc1923c684-public-tls-certs\") pod \"swift-proxy-55bdbb59c4-c4cjv\" (UID: \"16cac95c-d1d2-4542-bb8e-4bdc1923c684\") " pod="openstack-kuttl-tests/swift-proxy-55bdbb59c4-c4cjv" Dec 03 12:30:52 crc kubenswrapper[5052]: I1203 12:30:52.399237 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/16cac95c-d1d2-4542-bb8e-4bdc1923c684-run-httpd\") pod \"swift-proxy-55bdbb59c4-c4cjv\" (UID: \"16cac95c-d1d2-4542-bb8e-4bdc1923c684\") " pod="openstack-kuttl-tests/swift-proxy-55bdbb59c4-c4cjv" Dec 03 12:30:52 crc kubenswrapper[5052]: I1203 12:30:52.399451 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/16cac95c-d1d2-4542-bb8e-4bdc1923c684-log-httpd\") pod \"swift-proxy-55bdbb59c4-c4cjv\" (UID: \"16cac95c-d1d2-4542-bb8e-4bdc1923c684\") " pod="openstack-kuttl-tests/swift-proxy-55bdbb59c4-c4cjv" Dec 03 12:30:52 crc kubenswrapper[5052]: I1203 12:30:52.409616 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/16cac95c-d1d2-4542-bb8e-4bdc1923c684-combined-ca-bundle\") pod \"swift-proxy-55bdbb59c4-c4cjv\" (UID: \"16cac95c-d1d2-4542-bb8e-4bdc1923c684\") " pod="openstack-kuttl-tests/swift-proxy-55bdbb59c4-c4cjv" Dec 03 12:30:52 crc kubenswrapper[5052]: I1203 12:30:52.410769 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/16cac95c-d1d2-4542-bb8e-4bdc1923c684-internal-tls-certs\") pod \"swift-proxy-55bdbb59c4-c4cjv\" (UID: \"16cac95c-d1d2-4542-bb8e-4bdc1923c684\") " pod="openstack-kuttl-tests/swift-proxy-55bdbb59c4-c4cjv" Dec 03 12:30:52 crc kubenswrapper[5052]: I1203 12:30:52.411456 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/16cac95c-d1d2-4542-bb8e-4bdc1923c684-config-data\") pod \"swift-proxy-55bdbb59c4-c4cjv\" (UID: \"16cac95c-d1d2-4542-bb8e-4bdc1923c684\") " pod="openstack-kuttl-tests/swift-proxy-55bdbb59c4-c4cjv" Dec 03 12:30:52 crc kubenswrapper[5052]: I1203 12:30:52.413437 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/16cac95c-d1d2-4542-bb8e-4bdc1923c684-etc-swift\") pod \"swift-proxy-55bdbb59c4-c4cjv\" (UID: \"16cac95c-d1d2-4542-bb8e-4bdc1923c684\") " pod="openstack-kuttl-tests/swift-proxy-55bdbb59c4-c4cjv" Dec 03 12:30:52 crc kubenswrapper[5052]: I1203 12:30:52.414062 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/16cac95c-d1d2-4542-bb8e-4bdc1923c684-public-tls-certs\") pod \"swift-proxy-55bdbb59c4-c4cjv\" (UID: \"16cac95c-d1d2-4542-bb8e-4bdc1923c684\") " pod="openstack-kuttl-tests/swift-proxy-55bdbb59c4-c4cjv" Dec 03 12:30:52 crc kubenswrapper[5052]: I1203 12:30:52.425941 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-s4mt4\" (UniqueName: \"kubernetes.io/projected/16cac95c-d1d2-4542-bb8e-4bdc1923c684-kube-api-access-s4mt4\") pod \"swift-proxy-55bdbb59c4-c4cjv\" (UID: \"16cac95c-d1d2-4542-bb8e-4bdc1923c684\") " pod="openstack-kuttl-tests/swift-proxy-55bdbb59c4-c4cjv" Dec 03 12:30:52 crc kubenswrapper[5052]: I1203 12:30:52.499421 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-proxy-55bdbb59c4-c4cjv" Dec 03 12:30:53 crc kubenswrapper[5052]: I1203 12:30:53.234636 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:30:53 crc kubenswrapper[5052]: I1203 12:30:53.235019 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:30:53 crc kubenswrapper[5052]: I1203 12:30:53.239468 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:30:53 crc kubenswrapper[5052]: I1203 12:30:53.250038 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:30:53 crc kubenswrapper[5052]: I1203 12:30:53.414007 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:30:53 crc kubenswrapper[5052]: I1203 12:30:53.414109 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:30:53 crc kubenswrapper[5052]: I1203 12:30:53.414699 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:30:53 crc kubenswrapper[5052]: I1203 12:30:53.414737 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:30:53 crc kubenswrapper[5052]: I1203 12:30:53.423815 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:30:53 crc kubenswrapper[5052]: I1203 12:30:53.425380 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:30:53 crc kubenswrapper[5052]: I1203 12:30:53.428014 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:30:54 crc kubenswrapper[5052]: I1203 12:30:54.384024 5052 generic.go:334] "Generic (PLEG): container finished" podID="82331185-51eb-4ef5-8cf6-61dc74b73740" containerID="8435a02270401c91707d9204caf291f3365f9acf4871f8d7542a8e53a028b8f7" exitCode=0 Dec 03 12:30:54 crc kubenswrapper[5052]: I1203 12:30:54.384247 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-7897d64b6b-j8w2r" event={"ID":"82331185-51eb-4ef5-8cf6-61dc74b73740","Type":"ContainerDied","Data":"8435a02270401c91707d9204caf291f3365f9acf4871f8d7542a8e53a028b8f7"} Dec 03 12:30:54 crc kubenswrapper[5052]: I1203 12:30:54.691913 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/swift-proxy-55bdbb59c4-c4cjv"] Dec 03 12:30:55 crc kubenswrapper[5052]: I1203 12:30:55.052500 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-7897d64b6b-j8w2r" Dec 03 12:30:55 crc kubenswrapper[5052]: I1203 12:30:55.121758 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/82331185-51eb-4ef5-8cf6-61dc74b73740-scripts\") pod \"82331185-51eb-4ef5-8cf6-61dc74b73740\" (UID: \"82331185-51eb-4ef5-8cf6-61dc74b73740\") " Dec 03 12:30:55 crc kubenswrapper[5052]: I1203 12:30:55.121814 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/82331185-51eb-4ef5-8cf6-61dc74b73740-public-tls-certs\") pod \"82331185-51eb-4ef5-8cf6-61dc74b73740\" (UID: \"82331185-51eb-4ef5-8cf6-61dc74b73740\") " Dec 03 12:30:55 crc kubenswrapper[5052]: I1203 12:30:55.121888 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/82331185-51eb-4ef5-8cf6-61dc74b73740-logs\") pod \"82331185-51eb-4ef5-8cf6-61dc74b73740\" (UID: \"82331185-51eb-4ef5-8cf6-61dc74b73740\") " Dec 03 12:30:55 crc kubenswrapper[5052]: I1203 12:30:55.122001 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-znzsm\" (UniqueName: \"kubernetes.io/projected/82331185-51eb-4ef5-8cf6-61dc74b73740-kube-api-access-znzsm\") pod \"82331185-51eb-4ef5-8cf6-61dc74b73740\" (UID: \"82331185-51eb-4ef5-8cf6-61dc74b73740\") " Dec 03 12:30:55 crc kubenswrapper[5052]: I1203 12:30:55.122032 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/82331185-51eb-4ef5-8cf6-61dc74b73740-combined-ca-bundle\") pod \"82331185-51eb-4ef5-8cf6-61dc74b73740\" (UID: \"82331185-51eb-4ef5-8cf6-61dc74b73740\") " Dec 03 12:30:55 crc kubenswrapper[5052]: I1203 12:30:55.122147 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/82331185-51eb-4ef5-8cf6-61dc74b73740-config-data\") pod \"82331185-51eb-4ef5-8cf6-61dc74b73740\" (UID: \"82331185-51eb-4ef5-8cf6-61dc74b73740\") " Dec 03 12:30:55 crc kubenswrapper[5052]: I1203 12:30:55.122169 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/82331185-51eb-4ef5-8cf6-61dc74b73740-internal-tls-certs\") pod \"82331185-51eb-4ef5-8cf6-61dc74b73740\" (UID: \"82331185-51eb-4ef5-8cf6-61dc74b73740\") " Dec 03 12:30:55 crc kubenswrapper[5052]: I1203 12:30:55.123023 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/82331185-51eb-4ef5-8cf6-61dc74b73740-logs" (OuterVolumeSpecName: "logs") pod "82331185-51eb-4ef5-8cf6-61dc74b73740" (UID: "82331185-51eb-4ef5-8cf6-61dc74b73740"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:30:55 crc kubenswrapper[5052]: I1203 12:30:55.130718 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/82331185-51eb-4ef5-8cf6-61dc74b73740-scripts" (OuterVolumeSpecName: "scripts") pod "82331185-51eb-4ef5-8cf6-61dc74b73740" (UID: "82331185-51eb-4ef5-8cf6-61dc74b73740"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:55 crc kubenswrapper[5052]: I1203 12:30:55.130981 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/82331185-51eb-4ef5-8cf6-61dc74b73740-kube-api-access-znzsm" (OuterVolumeSpecName: "kube-api-access-znzsm") pod "82331185-51eb-4ef5-8cf6-61dc74b73740" (UID: "82331185-51eb-4ef5-8cf6-61dc74b73740"). InnerVolumeSpecName "kube-api-access-znzsm". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:30:55 crc kubenswrapper[5052]: I1203 12:30:55.200491 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/82331185-51eb-4ef5-8cf6-61dc74b73740-config-data" (OuterVolumeSpecName: "config-data") pod "82331185-51eb-4ef5-8cf6-61dc74b73740" (UID: "82331185-51eb-4ef5-8cf6-61dc74b73740"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:55 crc kubenswrapper[5052]: I1203 12:30:55.203117 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/82331185-51eb-4ef5-8cf6-61dc74b73740-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "82331185-51eb-4ef5-8cf6-61dc74b73740" (UID: "82331185-51eb-4ef5-8cf6-61dc74b73740"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:55 crc kubenswrapper[5052]: I1203 12:30:55.224099 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/82331185-51eb-4ef5-8cf6-61dc74b73740-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:55 crc kubenswrapper[5052]: I1203 12:30:55.224137 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/82331185-51eb-4ef5-8cf6-61dc74b73740-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:55 crc kubenswrapper[5052]: I1203 12:30:55.224150 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-znzsm\" (UniqueName: \"kubernetes.io/projected/82331185-51eb-4ef5-8cf6-61dc74b73740-kube-api-access-znzsm\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:55 crc kubenswrapper[5052]: I1203 12:30:55.224165 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/82331185-51eb-4ef5-8cf6-61dc74b73740-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:55 crc kubenswrapper[5052]: I1203 12:30:55.224175 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/82331185-51eb-4ef5-8cf6-61dc74b73740-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:55 crc kubenswrapper[5052]: I1203 12:30:55.296441 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/82331185-51eb-4ef5-8cf6-61dc74b73740-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "82331185-51eb-4ef5-8cf6-61dc74b73740" (UID: "82331185-51eb-4ef5-8cf6-61dc74b73740"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:55 crc kubenswrapper[5052]: I1203 12:30:55.299984 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/82331185-51eb-4ef5-8cf6-61dc74b73740-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "82331185-51eb-4ef5-8cf6-61dc74b73740" (UID: "82331185-51eb-4ef5-8cf6-61dc74b73740"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:55 crc kubenswrapper[5052]: I1203 12:30:55.325981 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/82331185-51eb-4ef5-8cf6-61dc74b73740-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:55 crc kubenswrapper[5052]: I1203 12:30:55.326020 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/82331185-51eb-4ef5-8cf6-61dc74b73740-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:55 crc kubenswrapper[5052]: I1203 12:30:55.416504 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_neutron-769cb78446-m6hzp_760aa987-6355-40cb-97c0-16c7eeddd135/neutron-api/0.log" Dec 03 12:30:55 crc kubenswrapper[5052]: I1203 12:30:55.416568 5052 generic.go:334] "Generic (PLEG): container finished" podID="760aa987-6355-40cb-97c0-16c7eeddd135" containerID="1cdac21bc509913963cca59d86d5d3a547a3bc1e21fc0a5a1c2c9f6e47211913" exitCode=137 Dec 03 12:30:55 crc kubenswrapper[5052]: I1203 12:30:55.416617 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-769cb78446-m6hzp" event={"ID":"760aa987-6355-40cb-97c0-16c7eeddd135","Type":"ContainerDied","Data":"1cdac21bc509913963cca59d86d5d3a547a3bc1e21fc0a5a1c2c9f6e47211913"} Dec 03 12:30:55 crc kubenswrapper[5052]: I1203 12:30:55.419185 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-7897d64b6b-j8w2r" event={"ID":"82331185-51eb-4ef5-8cf6-61dc74b73740","Type":"ContainerDied","Data":"0cb65f56986c143c9460824320cb63c61956b2e8af1869d12d750f07524ab709"} Dec 03 12:30:55 crc kubenswrapper[5052]: I1203 12:30:55.419232 5052 scope.go:117] "RemoveContainer" containerID="8435a02270401c91707d9204caf291f3365f9acf4871f8d7542a8e53a028b8f7" Dec 03 12:30:55 crc kubenswrapper[5052]: I1203 12:30:55.419420 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-7897d64b6b-j8w2r" Dec 03 12:30:55 crc kubenswrapper[5052]: I1203 12:30:55.432853 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"f9193a3b-a9c6-4343-a234-1616f73404cf","Type":"ContainerStarted","Data":"aeb74a268ef4cbfcdaec7b12ce9c082930f2b926d389f3e0fb132ec3d9fa328c"} Dec 03 12:30:55 crc kubenswrapper[5052]: I1203 12:30:55.437241 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-proxy-55bdbb59c4-c4cjv" event={"ID":"16cac95c-d1d2-4542-bb8e-4bdc1923c684","Type":"ContainerStarted","Data":"e2917e8431b0de8f360760abc00d1838625c47e53148c72eb9f76598a47629bd"} Dec 03 12:30:55 crc kubenswrapper[5052]: I1203 12:30:55.437286 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-proxy-55bdbb59c4-c4cjv" event={"ID":"16cac95c-d1d2-4542-bb8e-4bdc1923c684","Type":"ContainerStarted","Data":"73c4644f0732be3df2511f0c00e7ed3cf60813f1e9870d75287eae749cc4992e"} Dec 03 12:30:55 crc kubenswrapper[5052]: I1203 12:30:55.464816 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/placement-7897d64b6b-j8w2r"] Dec 03 12:30:55 crc kubenswrapper[5052]: I1203 12:30:55.465263 5052 scope.go:117] "RemoveContainer" containerID="bdd07debdd83a2d2d982d8b066d0fefba55b8fee5901eb6d6479f72b63d3ae51" Dec 03 12:30:55 crc kubenswrapper[5052]: I1203 12:30:55.475182 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/placement-7897d64b6b-j8w2r"] Dec 03 12:30:55 crc kubenswrapper[5052]: I1203 12:30:55.698893 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="82331185-51eb-4ef5-8cf6-61dc74b73740" path="/var/lib/kubelet/pods/82331185-51eb-4ef5-8cf6-61dc74b73740/volumes" Dec 03 12:30:55 crc kubenswrapper[5052]: I1203 12:30:55.922803 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_neutron-769cb78446-m6hzp_760aa987-6355-40cb-97c0-16c7eeddd135/neutron-api/0.log" Dec 03 12:30:55 crc kubenswrapper[5052]: I1203 12:30:55.922890 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-769cb78446-m6hzp" Dec 03 12:30:56 crc kubenswrapper[5052]: I1203 12:30:56.043990 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/760aa987-6355-40cb-97c0-16c7eeddd135-httpd-config\") pod \"760aa987-6355-40cb-97c0-16c7eeddd135\" (UID: \"760aa987-6355-40cb-97c0-16c7eeddd135\") " Dec 03 12:30:56 crc kubenswrapper[5052]: I1203 12:30:56.044051 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/760aa987-6355-40cb-97c0-16c7eeddd135-config\") pod \"760aa987-6355-40cb-97c0-16c7eeddd135\" (UID: \"760aa987-6355-40cb-97c0-16c7eeddd135\") " Dec 03 12:30:56 crc kubenswrapper[5052]: I1203 12:30:56.044139 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7rmck\" (UniqueName: \"kubernetes.io/projected/760aa987-6355-40cb-97c0-16c7eeddd135-kube-api-access-7rmck\") pod \"760aa987-6355-40cb-97c0-16c7eeddd135\" (UID: \"760aa987-6355-40cb-97c0-16c7eeddd135\") " Dec 03 12:30:56 crc kubenswrapper[5052]: I1203 12:30:56.044271 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/760aa987-6355-40cb-97c0-16c7eeddd135-internal-tls-certs\") pod \"760aa987-6355-40cb-97c0-16c7eeddd135\" (UID: \"760aa987-6355-40cb-97c0-16c7eeddd135\") " Dec 03 12:30:56 crc kubenswrapper[5052]: I1203 12:30:56.044302 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/760aa987-6355-40cb-97c0-16c7eeddd135-combined-ca-bundle\") pod \"760aa987-6355-40cb-97c0-16c7eeddd135\" (UID: \"760aa987-6355-40cb-97c0-16c7eeddd135\") " Dec 03 12:30:56 crc kubenswrapper[5052]: I1203 12:30:56.044378 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/760aa987-6355-40cb-97c0-16c7eeddd135-ovndb-tls-certs\") pod \"760aa987-6355-40cb-97c0-16c7eeddd135\" (UID: \"760aa987-6355-40cb-97c0-16c7eeddd135\") " Dec 03 12:30:56 crc kubenswrapper[5052]: I1203 12:30:56.044423 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/760aa987-6355-40cb-97c0-16c7eeddd135-public-tls-certs\") pod \"760aa987-6355-40cb-97c0-16c7eeddd135\" (UID: \"760aa987-6355-40cb-97c0-16c7eeddd135\") " Dec 03 12:30:56 crc kubenswrapper[5052]: I1203 12:30:56.050252 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/760aa987-6355-40cb-97c0-16c7eeddd135-httpd-config" (OuterVolumeSpecName: "httpd-config") pod "760aa987-6355-40cb-97c0-16c7eeddd135" (UID: "760aa987-6355-40cb-97c0-16c7eeddd135"). InnerVolumeSpecName "httpd-config". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:56 crc kubenswrapper[5052]: I1203 12:30:56.052710 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/760aa987-6355-40cb-97c0-16c7eeddd135-kube-api-access-7rmck" (OuterVolumeSpecName: "kube-api-access-7rmck") pod "760aa987-6355-40cb-97c0-16c7eeddd135" (UID: "760aa987-6355-40cb-97c0-16c7eeddd135"). InnerVolumeSpecName "kube-api-access-7rmck". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:30:56 crc kubenswrapper[5052]: I1203 12:30:56.119328 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/760aa987-6355-40cb-97c0-16c7eeddd135-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "760aa987-6355-40cb-97c0-16c7eeddd135" (UID: "760aa987-6355-40cb-97c0-16c7eeddd135"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:56 crc kubenswrapper[5052]: I1203 12:30:56.129250 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/760aa987-6355-40cb-97c0-16c7eeddd135-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "760aa987-6355-40cb-97c0-16c7eeddd135" (UID: "760aa987-6355-40cb-97c0-16c7eeddd135"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:56 crc kubenswrapper[5052]: I1203 12:30:56.146941 5052 reconciler_common.go:293] "Volume detached for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/760aa987-6355-40cb-97c0-16c7eeddd135-httpd-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:56 crc kubenswrapper[5052]: I1203 12:30:56.146972 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-7rmck\" (UniqueName: \"kubernetes.io/projected/760aa987-6355-40cb-97c0-16c7eeddd135-kube-api-access-7rmck\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:56 crc kubenswrapper[5052]: I1203 12:30:56.146983 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/760aa987-6355-40cb-97c0-16c7eeddd135-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:56 crc kubenswrapper[5052]: I1203 12:30:56.146991 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/760aa987-6355-40cb-97c0-16c7eeddd135-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:56 crc kubenswrapper[5052]: I1203 12:30:56.149327 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/760aa987-6355-40cb-97c0-16c7eeddd135-ovndb-tls-certs" (OuterVolumeSpecName: "ovndb-tls-certs") pod "760aa987-6355-40cb-97c0-16c7eeddd135" (UID: "760aa987-6355-40cb-97c0-16c7eeddd135"). InnerVolumeSpecName "ovndb-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:56 crc kubenswrapper[5052]: I1203 12:30:56.153333 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/760aa987-6355-40cb-97c0-16c7eeddd135-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "760aa987-6355-40cb-97c0-16c7eeddd135" (UID: "760aa987-6355-40cb-97c0-16c7eeddd135"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:56 crc kubenswrapper[5052]: I1203 12:30:56.157808 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/760aa987-6355-40cb-97c0-16c7eeddd135-config" (OuterVolumeSpecName: "config") pod "760aa987-6355-40cb-97c0-16c7eeddd135" (UID: "760aa987-6355-40cb-97c0-16c7eeddd135"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:56 crc kubenswrapper[5052]: I1203 12:30:56.249356 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/760aa987-6355-40cb-97c0-16c7eeddd135-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:56 crc kubenswrapper[5052]: I1203 12:30:56.249391 5052 reconciler_common.go:293] "Volume detached for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/760aa987-6355-40cb-97c0-16c7eeddd135-ovndb-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:56 crc kubenswrapper[5052]: I1203 12:30:56.249404 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/760aa987-6355-40cb-97c0-16c7eeddd135-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:56 crc kubenswrapper[5052]: I1203 12:30:56.449894 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_neutron-76bcd49fb6-54vvs_6bfd5f51-8b37-4fea-8134-a2cdcd4f6049/neutron-api/0.log" Dec 03 12:30:56 crc kubenswrapper[5052]: I1203 12:30:56.449954 5052 generic.go:334] "Generic (PLEG): container finished" podID="6bfd5f51-8b37-4fea-8134-a2cdcd4f6049" containerID="01711fff696becc6dfb1b8494228bffbaa942ad3546265fb3e7434412310303c" exitCode=137 Dec 03 12:30:56 crc kubenswrapper[5052]: I1203 12:30:56.450004 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-76bcd49fb6-54vvs" event={"ID":"6bfd5f51-8b37-4fea-8134-a2cdcd4f6049","Type":"ContainerDied","Data":"01711fff696becc6dfb1b8494228bffbaa942ad3546265fb3e7434412310303c"} Dec 03 12:30:56 crc kubenswrapper[5052]: I1203 12:30:56.452878 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-proxy-55bdbb59c4-c4cjv" event={"ID":"16cac95c-d1d2-4542-bb8e-4bdc1923c684","Type":"ContainerStarted","Data":"3fd08a7accc11bddbab26e95f99cd86563dcbdee87c9faa5f4e155b8982c03de"} Dec 03 12:30:56 crc kubenswrapper[5052]: I1203 12:30:56.453027 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/swift-proxy-55bdbb59c4-c4cjv" Dec 03 12:30:56 crc kubenswrapper[5052]: I1203 12:30:56.453056 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/swift-proxy-55bdbb59c4-c4cjv" Dec 03 12:30:56 crc kubenswrapper[5052]: I1203 12:30:56.454654 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_neutron-769cb78446-m6hzp_760aa987-6355-40cb-97c0-16c7eeddd135/neutron-api/0.log" Dec 03 12:30:56 crc kubenswrapper[5052]: I1203 12:30:56.454710 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-769cb78446-m6hzp" event={"ID":"760aa987-6355-40cb-97c0-16c7eeddd135","Type":"ContainerDied","Data":"b2c4cb5fe70990b0b235abbc3aedfcf98912b1818086712902f3b4d14422a734"} Dec 03 12:30:56 crc kubenswrapper[5052]: I1203 12:30:56.454744 5052 scope.go:117] "RemoveContainer" containerID="bd93058f7e9a54e5b0b1a6eecb1d99267f8e0625ea9ee3a2cc7eea950b1986a3" Dec 03 12:30:56 crc kubenswrapper[5052]: I1203 12:30:56.454883 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-769cb78446-m6hzp" Dec 03 12:30:56 crc kubenswrapper[5052]: I1203 12:30:56.505343 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/swift-proxy-55bdbb59c4-c4cjv" podStartSLOduration=4.505324993 podStartE2EDuration="4.505324993s" podCreationTimestamp="2025-12-03 12:30:52 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:30:56.484393716 +0000 UTC m=+4247.072635490" watchObservedRunningTime="2025-12-03 12:30:56.505324993 +0000 UTC m=+4247.093566757" Dec 03 12:30:56 crc kubenswrapper[5052]: I1203 12:30:56.522827 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/neutron-769cb78446-m6hzp"] Dec 03 12:30:56 crc kubenswrapper[5052]: I1203 12:30:56.525693 5052 scope.go:117] "RemoveContainer" containerID="1cdac21bc509913963cca59d86d5d3a547a3bc1e21fc0a5a1c2c9f6e47211913" Dec 03 12:30:56 crc kubenswrapper[5052]: I1203 12:30:56.531875 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/neutron-769cb78446-m6hzp"] Dec 03 12:30:56 crc kubenswrapper[5052]: E1203 12:30:56.625609 5052 cadvisor_stats_provider.go:516] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod760aa987_6355_40cb_97c0_16c7eeddd135.slice\": RecentStats: unable to find data in memory cache]" Dec 03 12:30:56 crc kubenswrapper[5052]: I1203 12:30:56.876189 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_neutron-76bcd49fb6-54vvs_6bfd5f51-8b37-4fea-8134-a2cdcd4f6049/neutron-api/0.log" Dec 03 12:30:56 crc kubenswrapper[5052]: I1203 12:30:56.876614 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-76bcd49fb6-54vvs" Dec 03 12:30:56 crc kubenswrapper[5052]: I1203 12:30:56.962696 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/6bfd5f51-8b37-4fea-8134-a2cdcd4f6049-public-tls-certs\") pod \"6bfd5f51-8b37-4fea-8134-a2cdcd4f6049\" (UID: \"6bfd5f51-8b37-4fea-8134-a2cdcd4f6049\") " Dec 03 12:30:56 crc kubenswrapper[5052]: I1203 12:30:56.962761 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6bfd5f51-8b37-4fea-8134-a2cdcd4f6049-combined-ca-bundle\") pod \"6bfd5f51-8b37-4fea-8134-a2cdcd4f6049\" (UID: \"6bfd5f51-8b37-4fea-8134-a2cdcd4f6049\") " Dec 03 12:30:56 crc kubenswrapper[5052]: I1203 12:30:56.962803 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xt6hn\" (UniqueName: \"kubernetes.io/projected/6bfd5f51-8b37-4fea-8134-a2cdcd4f6049-kube-api-access-xt6hn\") pod \"6bfd5f51-8b37-4fea-8134-a2cdcd4f6049\" (UID: \"6bfd5f51-8b37-4fea-8134-a2cdcd4f6049\") " Dec 03 12:30:56 crc kubenswrapper[5052]: I1203 12:30:56.962903 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/6bfd5f51-8b37-4fea-8134-a2cdcd4f6049-httpd-config\") pod \"6bfd5f51-8b37-4fea-8134-a2cdcd4f6049\" (UID: \"6bfd5f51-8b37-4fea-8134-a2cdcd4f6049\") " Dec 03 12:30:56 crc kubenswrapper[5052]: I1203 12:30:56.962977 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/6bfd5f51-8b37-4fea-8134-a2cdcd4f6049-internal-tls-certs\") pod \"6bfd5f51-8b37-4fea-8134-a2cdcd4f6049\" (UID: \"6bfd5f51-8b37-4fea-8134-a2cdcd4f6049\") " Dec 03 12:30:56 crc kubenswrapper[5052]: I1203 12:30:56.963053 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/6bfd5f51-8b37-4fea-8134-a2cdcd4f6049-ovndb-tls-certs\") pod \"6bfd5f51-8b37-4fea-8134-a2cdcd4f6049\" (UID: \"6bfd5f51-8b37-4fea-8134-a2cdcd4f6049\") " Dec 03 12:30:56 crc kubenswrapper[5052]: I1203 12:30:56.963333 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/6bfd5f51-8b37-4fea-8134-a2cdcd4f6049-config\") pod \"6bfd5f51-8b37-4fea-8134-a2cdcd4f6049\" (UID: \"6bfd5f51-8b37-4fea-8134-a2cdcd4f6049\") " Dec 03 12:30:56 crc kubenswrapper[5052]: I1203 12:30:56.981466 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6bfd5f51-8b37-4fea-8134-a2cdcd4f6049-kube-api-access-xt6hn" (OuterVolumeSpecName: "kube-api-access-xt6hn") pod "6bfd5f51-8b37-4fea-8134-a2cdcd4f6049" (UID: "6bfd5f51-8b37-4fea-8134-a2cdcd4f6049"). InnerVolumeSpecName "kube-api-access-xt6hn". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:30:56 crc kubenswrapper[5052]: I1203 12:30:56.982490 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6bfd5f51-8b37-4fea-8134-a2cdcd4f6049-httpd-config" (OuterVolumeSpecName: "httpd-config") pod "6bfd5f51-8b37-4fea-8134-a2cdcd4f6049" (UID: "6bfd5f51-8b37-4fea-8134-a2cdcd4f6049"). InnerVolumeSpecName "httpd-config". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:57 crc kubenswrapper[5052]: I1203 12:30:57.036438 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6bfd5f51-8b37-4fea-8134-a2cdcd4f6049-config" (OuterVolumeSpecName: "config") pod "6bfd5f51-8b37-4fea-8134-a2cdcd4f6049" (UID: "6bfd5f51-8b37-4fea-8134-a2cdcd4f6049"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:57 crc kubenswrapper[5052]: I1203 12:30:57.069224 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6bfd5f51-8b37-4fea-8134-a2cdcd4f6049-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "6bfd5f51-8b37-4fea-8134-a2cdcd4f6049" (UID: "6bfd5f51-8b37-4fea-8134-a2cdcd4f6049"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:57 crc kubenswrapper[5052]: I1203 12:30:57.071896 5052 reconciler_common.go:293] "Volume detached for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/6bfd5f51-8b37-4fea-8134-a2cdcd4f6049-httpd-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:57 crc kubenswrapper[5052]: I1203 12:30:57.079806 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/6bfd5f51-8b37-4fea-8134-a2cdcd4f6049-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:57 crc kubenswrapper[5052]: I1203 12:30:57.079837 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/6bfd5f51-8b37-4fea-8134-a2cdcd4f6049-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:57 crc kubenswrapper[5052]: I1203 12:30:57.079860 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-xt6hn\" (UniqueName: \"kubernetes.io/projected/6bfd5f51-8b37-4fea-8134-a2cdcd4f6049-kube-api-access-xt6hn\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:57 crc kubenswrapper[5052]: I1203 12:30:57.123113 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6bfd5f51-8b37-4fea-8134-a2cdcd4f6049-ovndb-tls-certs" (OuterVolumeSpecName: "ovndb-tls-certs") pod "6bfd5f51-8b37-4fea-8134-a2cdcd4f6049" (UID: "6bfd5f51-8b37-4fea-8134-a2cdcd4f6049"). InnerVolumeSpecName "ovndb-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:57 crc kubenswrapper[5052]: I1203 12:30:57.125765 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6bfd5f51-8b37-4fea-8134-a2cdcd4f6049-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "6bfd5f51-8b37-4fea-8134-a2cdcd4f6049" (UID: "6bfd5f51-8b37-4fea-8134-a2cdcd4f6049"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:57 crc kubenswrapper[5052]: I1203 12:30:57.143595 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6bfd5f51-8b37-4fea-8134-a2cdcd4f6049-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "6bfd5f51-8b37-4fea-8134-a2cdcd4f6049" (UID: "6bfd5f51-8b37-4fea-8134-a2cdcd4f6049"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:30:57 crc kubenswrapper[5052]: I1203 12:30:57.182434 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/6bfd5f51-8b37-4fea-8134-a2cdcd4f6049-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:57 crc kubenswrapper[5052]: I1203 12:30:57.182476 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6bfd5f51-8b37-4fea-8134-a2cdcd4f6049-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:57 crc kubenswrapper[5052]: I1203 12:30:57.182490 5052 reconciler_common.go:293] "Volume detached for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/6bfd5f51-8b37-4fea-8134-a2cdcd4f6049-ovndb-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:30:57 crc kubenswrapper[5052]: I1203 12:30:57.467793 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_neutron-76bcd49fb6-54vvs_6bfd5f51-8b37-4fea-8134-a2cdcd4f6049/neutron-api/0.log" Dec 03 12:30:57 crc kubenswrapper[5052]: I1203 12:30:57.468064 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-76bcd49fb6-54vvs" event={"ID":"6bfd5f51-8b37-4fea-8134-a2cdcd4f6049","Type":"ContainerDied","Data":"1a42dc3b980f218afed77ce3e44ebc5314da8b49e81ad613c50769fa0114595f"} Dec 03 12:30:57 crc kubenswrapper[5052]: I1203 12:30:57.468143 5052 scope.go:117] "RemoveContainer" containerID="0a0d6f3815eb929a36dbe3844272f6d6abfd6692349a5d9de8c42e6c899327d3" Dec 03 12:30:57 crc kubenswrapper[5052]: I1203 12:30:57.468360 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-76bcd49fb6-54vvs" Dec 03 12:30:57 crc kubenswrapper[5052]: I1203 12:30:57.501761 5052 scope.go:117] "RemoveContainer" containerID="01711fff696becc6dfb1b8494228bffbaa942ad3546265fb3e7434412310303c" Dec 03 12:30:57 crc kubenswrapper[5052]: I1203 12:30:57.516662 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/neutron-76bcd49fb6-54vvs"] Dec 03 12:30:57 crc kubenswrapper[5052]: I1203 12:30:57.525919 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/neutron-76bcd49fb6-54vvs"] Dec 03 12:30:57 crc kubenswrapper[5052]: I1203 12:30:57.683013 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6bfd5f51-8b37-4fea-8134-a2cdcd4f6049" path="/var/lib/kubelet/pods/6bfd5f51-8b37-4fea-8134-a2cdcd4f6049/volumes" Dec 03 12:30:57 crc kubenswrapper[5052]: I1203 12:30:57.683727 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="760aa987-6355-40cb-97c0-16c7eeddd135" path="/var/lib/kubelet/pods/760aa987-6355-40cb-97c0-16c7eeddd135/volumes" Dec 03 12:30:58 crc kubenswrapper[5052]: I1203 12:30:58.102641 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/keystone-564bc88b6-bvll5" podUID="c0ce9ced-33b0-4879-81a4-b5c066ec0631" containerName="keystone-api" probeResult="failure" output="Get \"https://10.217.0.206:5000/v3\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Dec 03 12:30:59 crc kubenswrapper[5052]: I1203 12:30:59.379610 5052 patch_prober.go:28] interesting pod/machine-config-daemon-74bj6 container/machine-config-daemon namespace/openshift-machine-config-operator: Liveness probe status=failure output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" start-of-body= Dec 03 12:30:59 crc kubenswrapper[5052]: I1203 12:30:59.381494 5052 prober.go:107] "Probe failed" probeType="Liveness" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" containerName="machine-config-daemon" probeResult="failure" output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" Dec 03 12:30:59 crc kubenswrapper[5052]: I1203 12:30:59.502129 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"f9193a3b-a9c6-4343-a234-1616f73404cf","Type":"ContainerStarted","Data":"d5a506b8e44bdc2bb0161dbf172c464d8a512672b56f42caecb0a6ec89bd9715"} Dec 03 12:30:59 crc kubenswrapper[5052]: I1203 12:30:59.502386 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="f9193a3b-a9c6-4343-a234-1616f73404cf" containerName="ceilometer-central-agent" containerID="cri-o://29b7f054b1ac49030c5a40b904711ccba428b2c71382c48f025d25f4f0a30435" gracePeriod=30 Dec 03 12:30:59 crc kubenswrapper[5052]: I1203 12:30:59.502929 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:30:59 crc kubenswrapper[5052]: I1203 12:30:59.503363 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="f9193a3b-a9c6-4343-a234-1616f73404cf" containerName="sg-core" containerID="cri-o://aeb74a268ef4cbfcdaec7b12ce9c082930f2b926d389f3e0fb132ec3d9fa328c" gracePeriod=30 Dec 03 12:30:59 crc kubenswrapper[5052]: I1203 12:30:59.503432 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="f9193a3b-a9c6-4343-a234-1616f73404cf" containerName="proxy-httpd" containerID="cri-o://d5a506b8e44bdc2bb0161dbf172c464d8a512672b56f42caecb0a6ec89bd9715" gracePeriod=30 Dec 03 12:30:59 crc kubenswrapper[5052]: I1203 12:30:59.503497 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="f9193a3b-a9c6-4343-a234-1616f73404cf" containerName="ceilometer-notification-agent" containerID="cri-o://5eba7bcd1112f7d19e289be438b4a2d50309b5fe47adfc505eb50370db185f89" gracePeriod=30 Dec 03 12:30:59 crc kubenswrapper[5052]: I1203 12:30:59.537372 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/ceilometer-0" podStartSLOduration=1.9012294280000002 podStartE2EDuration="21.537346053s" podCreationTimestamp="2025-12-03 12:30:38 +0000 UTC" firstStartedPulling="2025-12-03 12:30:39.073853343 +0000 UTC m=+4229.662095107" lastFinishedPulling="2025-12-03 12:30:58.709969968 +0000 UTC m=+4249.298211732" observedRunningTime="2025-12-03 12:30:59.528770588 +0000 UTC m=+4250.117012352" watchObservedRunningTime="2025-12-03 12:30:59.537346053 +0000 UTC m=+4250.125587817" Dec 03 12:31:00 crc kubenswrapper[5052]: I1203 12:31:00.532358 5052 generic.go:334] "Generic (PLEG): container finished" podID="f9193a3b-a9c6-4343-a234-1616f73404cf" containerID="aeb74a268ef4cbfcdaec7b12ce9c082930f2b926d389f3e0fb132ec3d9fa328c" exitCode=2 Dec 03 12:31:00 crc kubenswrapper[5052]: I1203 12:31:00.532775 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"f9193a3b-a9c6-4343-a234-1616f73404cf","Type":"ContainerDied","Data":"aeb74a268ef4cbfcdaec7b12ce9c082930f2b926d389f3e0fb132ec3d9fa328c"} Dec 03 12:31:01 crc kubenswrapper[5052]: I1203 12:31:01.545253 5052 generic.go:334] "Generic (PLEG): container finished" podID="f9193a3b-a9c6-4343-a234-1616f73404cf" containerID="5eba7bcd1112f7d19e289be438b4a2d50309b5fe47adfc505eb50370db185f89" exitCode=0 Dec 03 12:31:01 crc kubenswrapper[5052]: I1203 12:31:01.545284 5052 generic.go:334] "Generic (PLEG): container finished" podID="f9193a3b-a9c6-4343-a234-1616f73404cf" containerID="29b7f054b1ac49030c5a40b904711ccba428b2c71382c48f025d25f4f0a30435" exitCode=0 Dec 03 12:31:01 crc kubenswrapper[5052]: I1203 12:31:01.545306 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"f9193a3b-a9c6-4343-a234-1616f73404cf","Type":"ContainerDied","Data":"5eba7bcd1112f7d19e289be438b4a2d50309b5fe47adfc505eb50370db185f89"} Dec 03 12:31:01 crc kubenswrapper[5052]: I1203 12:31:01.545333 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"f9193a3b-a9c6-4343-a234-1616f73404cf","Type":"ContainerDied","Data":"29b7f054b1ac49030c5a40b904711ccba428b2c71382c48f025d25f4f0a30435"} Dec 03 12:31:02 crc kubenswrapper[5052]: I1203 12:31:02.506221 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/swift-proxy-55bdbb59c4-c4cjv" Dec 03 12:31:02 crc kubenswrapper[5052]: I1203 12:31:02.506595 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/swift-proxy-55bdbb59c4-c4cjv" Dec 03 12:31:02 crc kubenswrapper[5052]: I1203 12:31:02.605243 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/swift-proxy-84cc784c4-9pgx5"] Dec 03 12:31:02 crc kubenswrapper[5052]: I1203 12:31:02.605473 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-proxy-84cc784c4-9pgx5" podUID="6af456b9-3c1e-4d13-ab42-843b94a6eccc" containerName="proxy-httpd" containerID="cri-o://e658cf1a323255cebd83f830ba316f88e6f46fb6e418f976a2cc18aafeb2c7b8" gracePeriod=30 Dec 03 12:31:02 crc kubenswrapper[5052]: I1203 12:31:02.605611 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-proxy-84cc784c4-9pgx5" podUID="6af456b9-3c1e-4d13-ab42-843b94a6eccc" containerName="proxy-server" containerID="cri-o://dc7840751c070b24059d5c6705a80932dcd86fe7d9711d447555b21056996478" gracePeriod=30 Dec 03 12:31:03 crc kubenswrapper[5052]: I1203 12:31:03.580652 5052 generic.go:334] "Generic (PLEG): container finished" podID="6af456b9-3c1e-4d13-ab42-843b94a6eccc" containerID="dc7840751c070b24059d5c6705a80932dcd86fe7d9711d447555b21056996478" exitCode=0 Dec 03 12:31:03 crc kubenswrapper[5052]: I1203 12:31:03.580999 5052 generic.go:334] "Generic (PLEG): container finished" podID="6af456b9-3c1e-4d13-ab42-843b94a6eccc" containerID="e658cf1a323255cebd83f830ba316f88e6f46fb6e418f976a2cc18aafeb2c7b8" exitCode=0 Dec 03 12:31:03 crc kubenswrapper[5052]: I1203 12:31:03.581026 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-proxy-84cc784c4-9pgx5" event={"ID":"6af456b9-3c1e-4d13-ab42-843b94a6eccc","Type":"ContainerDied","Data":"dc7840751c070b24059d5c6705a80932dcd86fe7d9711d447555b21056996478"} Dec 03 12:31:03 crc kubenswrapper[5052]: I1203 12:31:03.581063 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-proxy-84cc784c4-9pgx5" event={"ID":"6af456b9-3c1e-4d13-ab42-843b94a6eccc","Type":"ContainerDied","Data":"e658cf1a323255cebd83f830ba316f88e6f46fb6e418f976a2cc18aafeb2c7b8"} Dec 03 12:31:04 crc kubenswrapper[5052]: I1203 12:31:04.484605 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-proxy-84cc784c4-9pgx5" Dec 03 12:31:04 crc kubenswrapper[5052]: I1203 12:31:04.594537 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-proxy-84cc784c4-9pgx5" event={"ID":"6af456b9-3c1e-4d13-ab42-843b94a6eccc","Type":"ContainerDied","Data":"57747c3003398e289e5ac01543fab3e660cc94651b02052539b92d4fe11f2f31"} Dec 03 12:31:04 crc kubenswrapper[5052]: I1203 12:31:04.594814 5052 scope.go:117] "RemoveContainer" containerID="dc7840751c070b24059d5c6705a80932dcd86fe7d9711d447555b21056996478" Dec 03 12:31:04 crc kubenswrapper[5052]: I1203 12:31:04.594685 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-proxy-84cc784c4-9pgx5" Dec 03 12:31:04 crc kubenswrapper[5052]: I1203 12:31:04.616085 5052 scope.go:117] "RemoveContainer" containerID="e658cf1a323255cebd83f830ba316f88e6f46fb6e418f976a2cc18aafeb2c7b8" Dec 03 12:31:04 crc kubenswrapper[5052]: I1203 12:31:04.633329 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/6af456b9-3c1e-4d13-ab42-843b94a6eccc-public-tls-certs\") pod \"6af456b9-3c1e-4d13-ab42-843b94a6eccc\" (UID: \"6af456b9-3c1e-4d13-ab42-843b94a6eccc\") " Dec 03 12:31:04 crc kubenswrapper[5052]: I1203 12:31:04.633463 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/6af456b9-3c1e-4d13-ab42-843b94a6eccc-run-httpd\") pod \"6af456b9-3c1e-4d13-ab42-843b94a6eccc\" (UID: \"6af456b9-3c1e-4d13-ab42-843b94a6eccc\") " Dec 03 12:31:04 crc kubenswrapper[5052]: I1203 12:31:04.633548 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hg5ln\" (UniqueName: \"kubernetes.io/projected/6af456b9-3c1e-4d13-ab42-843b94a6eccc-kube-api-access-hg5ln\") pod \"6af456b9-3c1e-4d13-ab42-843b94a6eccc\" (UID: \"6af456b9-3c1e-4d13-ab42-843b94a6eccc\") " Dec 03 12:31:04 crc kubenswrapper[5052]: I1203 12:31:04.633615 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/6af456b9-3c1e-4d13-ab42-843b94a6eccc-internal-tls-certs\") pod \"6af456b9-3c1e-4d13-ab42-843b94a6eccc\" (UID: \"6af456b9-3c1e-4d13-ab42-843b94a6eccc\") " Dec 03 12:31:04 crc kubenswrapper[5052]: I1203 12:31:04.633693 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/6af456b9-3c1e-4d13-ab42-843b94a6eccc-etc-swift\") pod \"6af456b9-3c1e-4d13-ab42-843b94a6eccc\" (UID: \"6af456b9-3c1e-4d13-ab42-843b94a6eccc\") " Dec 03 12:31:04 crc kubenswrapper[5052]: I1203 12:31:04.633732 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/6af456b9-3c1e-4d13-ab42-843b94a6eccc-log-httpd\") pod \"6af456b9-3c1e-4d13-ab42-843b94a6eccc\" (UID: \"6af456b9-3c1e-4d13-ab42-843b94a6eccc\") " Dec 03 12:31:04 crc kubenswrapper[5052]: I1203 12:31:04.633767 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6af456b9-3c1e-4d13-ab42-843b94a6eccc-combined-ca-bundle\") pod \"6af456b9-3c1e-4d13-ab42-843b94a6eccc\" (UID: \"6af456b9-3c1e-4d13-ab42-843b94a6eccc\") " Dec 03 12:31:04 crc kubenswrapper[5052]: I1203 12:31:04.633810 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6af456b9-3c1e-4d13-ab42-843b94a6eccc-config-data\") pod \"6af456b9-3c1e-4d13-ab42-843b94a6eccc\" (UID: \"6af456b9-3c1e-4d13-ab42-843b94a6eccc\") " Dec 03 12:31:04 crc kubenswrapper[5052]: I1203 12:31:04.634527 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6af456b9-3c1e-4d13-ab42-843b94a6eccc-log-httpd" (OuterVolumeSpecName: "log-httpd") pod "6af456b9-3c1e-4d13-ab42-843b94a6eccc" (UID: "6af456b9-3c1e-4d13-ab42-843b94a6eccc"). InnerVolumeSpecName "log-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:31:04 crc kubenswrapper[5052]: I1203 12:31:04.634842 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6af456b9-3c1e-4d13-ab42-843b94a6eccc-run-httpd" (OuterVolumeSpecName: "run-httpd") pod "6af456b9-3c1e-4d13-ab42-843b94a6eccc" (UID: "6af456b9-3c1e-4d13-ab42-843b94a6eccc"). InnerVolumeSpecName "run-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:31:04 crc kubenswrapper[5052]: I1203 12:31:04.639780 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6af456b9-3c1e-4d13-ab42-843b94a6eccc-kube-api-access-hg5ln" (OuterVolumeSpecName: "kube-api-access-hg5ln") pod "6af456b9-3c1e-4d13-ab42-843b94a6eccc" (UID: "6af456b9-3c1e-4d13-ab42-843b94a6eccc"). InnerVolumeSpecName "kube-api-access-hg5ln". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:31:04 crc kubenswrapper[5052]: I1203 12:31:04.644563 5052 reconciler_common.go:293] "Volume detached for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/6af456b9-3c1e-4d13-ab42-843b94a6eccc-run-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:31:04 crc kubenswrapper[5052]: I1203 12:31:04.644600 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-hg5ln\" (UniqueName: \"kubernetes.io/projected/6af456b9-3c1e-4d13-ab42-843b94a6eccc-kube-api-access-hg5ln\") on node \"crc\" DevicePath \"\"" Dec 03 12:31:04 crc kubenswrapper[5052]: I1203 12:31:04.644615 5052 reconciler_common.go:293] "Volume detached for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/6af456b9-3c1e-4d13-ab42-843b94a6eccc-log-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:31:04 crc kubenswrapper[5052]: I1203 12:31:04.660761 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6af456b9-3c1e-4d13-ab42-843b94a6eccc-etc-swift" (OuterVolumeSpecName: "etc-swift") pod "6af456b9-3c1e-4d13-ab42-843b94a6eccc" (UID: "6af456b9-3c1e-4d13-ab42-843b94a6eccc"). InnerVolumeSpecName "etc-swift". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:31:04 crc kubenswrapper[5052]: I1203 12:31:04.698111 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6af456b9-3c1e-4d13-ab42-843b94a6eccc-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "6af456b9-3c1e-4d13-ab42-843b94a6eccc" (UID: "6af456b9-3c1e-4d13-ab42-843b94a6eccc"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:31:04 crc kubenswrapper[5052]: I1203 12:31:04.702462 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6af456b9-3c1e-4d13-ab42-843b94a6eccc-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "6af456b9-3c1e-4d13-ab42-843b94a6eccc" (UID: "6af456b9-3c1e-4d13-ab42-843b94a6eccc"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:31:04 crc kubenswrapper[5052]: I1203 12:31:04.707660 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6af456b9-3c1e-4d13-ab42-843b94a6eccc-config-data" (OuterVolumeSpecName: "config-data") pod "6af456b9-3c1e-4d13-ab42-843b94a6eccc" (UID: "6af456b9-3c1e-4d13-ab42-843b94a6eccc"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:31:04 crc kubenswrapper[5052]: I1203 12:31:04.713925 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6af456b9-3c1e-4d13-ab42-843b94a6eccc-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "6af456b9-3c1e-4d13-ab42-843b94a6eccc" (UID: "6af456b9-3c1e-4d13-ab42-843b94a6eccc"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:31:04 crc kubenswrapper[5052]: I1203 12:31:04.748865 5052 reconciler_common.go:293] "Volume detached for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/6af456b9-3c1e-4d13-ab42-843b94a6eccc-etc-swift\") on node \"crc\" DevicePath \"\"" Dec 03 12:31:04 crc kubenswrapper[5052]: I1203 12:31:04.748904 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6af456b9-3c1e-4d13-ab42-843b94a6eccc-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:31:04 crc kubenswrapper[5052]: I1203 12:31:04.748919 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6af456b9-3c1e-4d13-ab42-843b94a6eccc-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:31:04 crc kubenswrapper[5052]: I1203 12:31:04.748929 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/6af456b9-3c1e-4d13-ab42-843b94a6eccc-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:31:04 crc kubenswrapper[5052]: I1203 12:31:04.748941 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/6af456b9-3c1e-4d13-ab42-843b94a6eccc-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:31:04 crc kubenswrapper[5052]: I1203 12:31:04.927151 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/swift-proxy-84cc784c4-9pgx5"] Dec 03 12:31:04 crc kubenswrapper[5052]: I1203 12:31:04.935934 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/swift-proxy-84cc784c4-9pgx5"] Dec 03 12:31:05 crc kubenswrapper[5052]: I1203 12:31:05.686009 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6af456b9-3c1e-4d13-ab42-843b94a6eccc" path="/var/lib/kubelet/pods/6af456b9-3c1e-4d13-ab42-843b94a6eccc/volumes" Dec 03 12:31:15 crc kubenswrapper[5052]: I1203 12:31:15.572096 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/neutron-58d7cc6794-9z49k" Dec 03 12:31:15 crc kubenswrapper[5052]: I1203 12:31:15.641377 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/neutron-56ff9fdb74-cft5n"] Dec 03 12:31:15 crc kubenswrapper[5052]: I1203 12:31:15.641753 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/neutron-56ff9fdb74-cft5n" podUID="5c13362a-3cdf-4ff3-b57d-eb8c6cdcdc98" containerName="neutron-api" containerID="cri-o://7eb375afddb95b73f21f6b1d166ca117940d6174ccbd794be68d71b6d2fd2a74" gracePeriod=30 Dec 03 12:31:15 crc kubenswrapper[5052]: I1203 12:31:15.641872 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/neutron-56ff9fdb74-cft5n" podUID="5c13362a-3cdf-4ff3-b57d-eb8c6cdcdc98" containerName="neutron-httpd" containerID="cri-o://136d74558e9fead2cd7cf270320c4f5f5b870699d1758c2cc84506b5c369b70e" gracePeriod=30 Dec 03 12:31:16 crc kubenswrapper[5052]: I1203 12:31:16.735967 5052 generic.go:334] "Generic (PLEG): container finished" podID="5c13362a-3cdf-4ff3-b57d-eb8c6cdcdc98" containerID="136d74558e9fead2cd7cf270320c4f5f5b870699d1758c2cc84506b5c369b70e" exitCode=0 Dec 03 12:31:16 crc kubenswrapper[5052]: I1203 12:31:16.736110 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-56ff9fdb74-cft5n" event={"ID":"5c13362a-3cdf-4ff3-b57d-eb8c6cdcdc98","Type":"ContainerDied","Data":"136d74558e9fead2cd7cf270320c4f5f5b870699d1758c2cc84506b5c369b70e"} Dec 03 12:31:17 crc kubenswrapper[5052]: I1203 12:31:17.554389 5052 scope.go:117] "RemoveContainer" containerID="b1610ebac2aa0f8aa4b81967dc3ae3b5979590368ec9deafce7d6a3c16ed7f4b" Dec 03 12:31:17 crc kubenswrapper[5052]: I1203 12:31:17.580367 5052 scope.go:117] "RemoveContainer" containerID="00870692d2b50338582621d68d6c27be034aaac53091d9351d0a669d8366e448" Dec 03 12:31:17 crc kubenswrapper[5052]: I1203 12:31:17.601098 5052 scope.go:117] "RemoveContainer" containerID="f0af04bda2d702b68508554603573208064e8a15663ea9bba8a0ae02fde8add6" Dec 03 12:31:17 crc kubenswrapper[5052]: I1203 12:31:17.620842 5052 scope.go:117] "RemoveContainer" containerID="7a4d323cb6d6466b95c1a95a6d37b454a8e65321dde7e9773b5b6607401badac" Dec 03 12:31:17 crc kubenswrapper[5052]: I1203 12:31:17.659270 5052 scope.go:117] "RemoveContainer" containerID="333d5988d6ce193cc6cbf5c47b0c3154146fadf562614a1dddecfa687325ba71" Dec 03 12:31:17 crc kubenswrapper[5052]: I1203 12:31:17.686071 5052 scope.go:117] "RemoveContainer" containerID="bc7ed62ebecdd8949db875ccee04947a38e495a8effb4f2799788690d8f5353d" Dec 03 12:31:27 crc kubenswrapper[5052]: I1203 12:31:27.894016 5052 generic.go:334] "Generic (PLEG): container finished" podID="5c13362a-3cdf-4ff3-b57d-eb8c6cdcdc98" containerID="7eb375afddb95b73f21f6b1d166ca117940d6174ccbd794be68d71b6d2fd2a74" exitCode=0 Dec 03 12:31:27 crc kubenswrapper[5052]: I1203 12:31:27.894107 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-56ff9fdb74-cft5n" event={"ID":"5c13362a-3cdf-4ff3-b57d-eb8c6cdcdc98","Type":"ContainerDied","Data":"7eb375afddb95b73f21f6b1d166ca117940d6174ccbd794be68d71b6d2fd2a74"} Dec 03 12:31:28 crc kubenswrapper[5052]: I1203 12:31:28.198046 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-56ff9fdb74-cft5n" Dec 03 12:31:28 crc kubenswrapper[5052]: I1203 12:31:28.277007 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/5c13362a-3cdf-4ff3-b57d-eb8c6cdcdc98-public-tls-certs\") pod \"5c13362a-3cdf-4ff3-b57d-eb8c6cdcdc98\" (UID: \"5c13362a-3cdf-4ff3-b57d-eb8c6cdcdc98\") " Dec 03 12:31:28 crc kubenswrapper[5052]: I1203 12:31:28.277134 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/5c13362a-3cdf-4ff3-b57d-eb8c6cdcdc98-ovndb-tls-certs\") pod \"5c13362a-3cdf-4ff3-b57d-eb8c6cdcdc98\" (UID: \"5c13362a-3cdf-4ff3-b57d-eb8c6cdcdc98\") " Dec 03 12:31:28 crc kubenswrapper[5052]: I1203 12:31:28.277207 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5c13362a-3cdf-4ff3-b57d-eb8c6cdcdc98-combined-ca-bundle\") pod \"5c13362a-3cdf-4ff3-b57d-eb8c6cdcdc98\" (UID: \"5c13362a-3cdf-4ff3-b57d-eb8c6cdcdc98\") " Dec 03 12:31:28 crc kubenswrapper[5052]: I1203 12:31:28.277274 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/5c13362a-3cdf-4ff3-b57d-eb8c6cdcdc98-config\") pod \"5c13362a-3cdf-4ff3-b57d-eb8c6cdcdc98\" (UID: \"5c13362a-3cdf-4ff3-b57d-eb8c6cdcdc98\") " Dec 03 12:31:28 crc kubenswrapper[5052]: I1203 12:31:28.277299 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/5c13362a-3cdf-4ff3-b57d-eb8c6cdcdc98-httpd-config\") pod \"5c13362a-3cdf-4ff3-b57d-eb8c6cdcdc98\" (UID: \"5c13362a-3cdf-4ff3-b57d-eb8c6cdcdc98\") " Dec 03 12:31:28 crc kubenswrapper[5052]: I1203 12:31:28.277412 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/5c13362a-3cdf-4ff3-b57d-eb8c6cdcdc98-internal-tls-certs\") pod \"5c13362a-3cdf-4ff3-b57d-eb8c6cdcdc98\" (UID: \"5c13362a-3cdf-4ff3-b57d-eb8c6cdcdc98\") " Dec 03 12:31:28 crc kubenswrapper[5052]: I1203 12:31:28.277799 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pvvfv\" (UniqueName: \"kubernetes.io/projected/5c13362a-3cdf-4ff3-b57d-eb8c6cdcdc98-kube-api-access-pvvfv\") pod \"5c13362a-3cdf-4ff3-b57d-eb8c6cdcdc98\" (UID: \"5c13362a-3cdf-4ff3-b57d-eb8c6cdcdc98\") " Dec 03 12:31:28 crc kubenswrapper[5052]: I1203 12:31:28.285196 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5c13362a-3cdf-4ff3-b57d-eb8c6cdcdc98-httpd-config" (OuterVolumeSpecName: "httpd-config") pod "5c13362a-3cdf-4ff3-b57d-eb8c6cdcdc98" (UID: "5c13362a-3cdf-4ff3-b57d-eb8c6cdcdc98"). InnerVolumeSpecName "httpd-config". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:31:28 crc kubenswrapper[5052]: I1203 12:31:28.293687 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5c13362a-3cdf-4ff3-b57d-eb8c6cdcdc98-kube-api-access-pvvfv" (OuterVolumeSpecName: "kube-api-access-pvvfv") pod "5c13362a-3cdf-4ff3-b57d-eb8c6cdcdc98" (UID: "5c13362a-3cdf-4ff3-b57d-eb8c6cdcdc98"). InnerVolumeSpecName "kube-api-access-pvvfv". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:31:28 crc kubenswrapper[5052]: I1203 12:31:28.343580 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5c13362a-3cdf-4ff3-b57d-eb8c6cdcdc98-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "5c13362a-3cdf-4ff3-b57d-eb8c6cdcdc98" (UID: "5c13362a-3cdf-4ff3-b57d-eb8c6cdcdc98"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:31:28 crc kubenswrapper[5052]: I1203 12:31:28.347230 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5c13362a-3cdf-4ff3-b57d-eb8c6cdcdc98-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "5c13362a-3cdf-4ff3-b57d-eb8c6cdcdc98" (UID: "5c13362a-3cdf-4ff3-b57d-eb8c6cdcdc98"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:31:28 crc kubenswrapper[5052]: I1203 12:31:28.352193 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5c13362a-3cdf-4ff3-b57d-eb8c6cdcdc98-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "5c13362a-3cdf-4ff3-b57d-eb8c6cdcdc98" (UID: "5c13362a-3cdf-4ff3-b57d-eb8c6cdcdc98"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:31:28 crc kubenswrapper[5052]: I1203 12:31:28.371892 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5c13362a-3cdf-4ff3-b57d-eb8c6cdcdc98-config" (OuterVolumeSpecName: "config") pod "5c13362a-3cdf-4ff3-b57d-eb8c6cdcdc98" (UID: "5c13362a-3cdf-4ff3-b57d-eb8c6cdcdc98"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:31:28 crc kubenswrapper[5052]: I1203 12:31:28.380392 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/5c13362a-3cdf-4ff3-b57d-eb8c6cdcdc98-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:31:28 crc kubenswrapper[5052]: I1203 12:31:28.380431 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-pvvfv\" (UniqueName: \"kubernetes.io/projected/5c13362a-3cdf-4ff3-b57d-eb8c6cdcdc98-kube-api-access-pvvfv\") on node \"crc\" DevicePath \"\"" Dec 03 12:31:28 crc kubenswrapper[5052]: I1203 12:31:28.380443 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/5c13362a-3cdf-4ff3-b57d-eb8c6cdcdc98-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:31:28 crc kubenswrapper[5052]: I1203 12:31:28.380464 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5c13362a-3cdf-4ff3-b57d-eb8c6cdcdc98-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:31:28 crc kubenswrapper[5052]: I1203 12:31:28.380476 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/5c13362a-3cdf-4ff3-b57d-eb8c6cdcdc98-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:31:28 crc kubenswrapper[5052]: I1203 12:31:28.380487 5052 reconciler_common.go:293] "Volume detached for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/5c13362a-3cdf-4ff3-b57d-eb8c6cdcdc98-httpd-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:31:28 crc kubenswrapper[5052]: I1203 12:31:28.395155 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5c13362a-3cdf-4ff3-b57d-eb8c6cdcdc98-ovndb-tls-certs" (OuterVolumeSpecName: "ovndb-tls-certs") pod "5c13362a-3cdf-4ff3-b57d-eb8c6cdcdc98" (UID: "5c13362a-3cdf-4ff3-b57d-eb8c6cdcdc98"). InnerVolumeSpecName "ovndb-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:31:28 crc kubenswrapper[5052]: I1203 12:31:28.482165 5052 reconciler_common.go:293] "Volume detached for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/5c13362a-3cdf-4ff3-b57d-eb8c6cdcdc98-ovndb-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:31:28 crc kubenswrapper[5052]: I1203 12:31:28.906341 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-56ff9fdb74-cft5n" event={"ID":"5c13362a-3cdf-4ff3-b57d-eb8c6cdcdc98","Type":"ContainerDied","Data":"519ff8f87376a328beb7fed38c294edc36f7f254b41c60c34eefe2f7a8ebde99"} Dec 03 12:31:28 crc kubenswrapper[5052]: I1203 12:31:28.906401 5052 scope.go:117] "RemoveContainer" containerID="136d74558e9fead2cd7cf270320c4f5f5b870699d1758c2cc84506b5c369b70e" Dec 03 12:31:28 crc kubenswrapper[5052]: I1203 12:31:28.906448 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-56ff9fdb74-cft5n" Dec 03 12:31:28 crc kubenswrapper[5052]: I1203 12:31:28.939232 5052 scope.go:117] "RemoveContainer" containerID="7eb375afddb95b73f21f6b1d166ca117940d6174ccbd794be68d71b6d2fd2a74" Dec 03 12:31:28 crc kubenswrapper[5052]: I1203 12:31:28.941480 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/neutron-56ff9fdb74-cft5n"] Dec 03 12:31:28 crc kubenswrapper[5052]: I1203 12:31:28.951681 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/neutron-56ff9fdb74-cft5n"] Dec 03 12:31:29 crc kubenswrapper[5052]: I1203 12:31:29.378940 5052 patch_prober.go:28] interesting pod/machine-config-daemon-74bj6 container/machine-config-daemon namespace/openshift-machine-config-operator: Liveness probe status=failure output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" start-of-body= Dec 03 12:31:29 crc kubenswrapper[5052]: I1203 12:31:29.379298 5052 prober.go:107] "Probe failed" probeType="Liveness" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" containerName="machine-config-daemon" probeResult="failure" output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" Dec 03 12:31:29 crc kubenswrapper[5052]: I1203 12:31:29.685695 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5c13362a-3cdf-4ff3-b57d-eb8c6cdcdc98" path="/var/lib/kubelet/pods/5c13362a-3cdf-4ff3-b57d-eb8c6cdcdc98/volumes" Dec 03 12:31:29 crc kubenswrapper[5052]: I1203 12:31:29.920453 5052 generic.go:334] "Generic (PLEG): container finished" podID="f9193a3b-a9c6-4343-a234-1616f73404cf" containerID="d5a506b8e44bdc2bb0161dbf172c464d8a512672b56f42caecb0a6ec89bd9715" exitCode=137 Dec 03 12:31:29 crc kubenswrapper[5052]: I1203 12:31:29.920540 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"f9193a3b-a9c6-4343-a234-1616f73404cf","Type":"ContainerDied","Data":"d5a506b8e44bdc2bb0161dbf172c464d8a512672b56f42caecb0a6ec89bd9715"} Dec 03 12:31:30 crc kubenswrapper[5052]: I1203 12:31:30.551415 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:31:30 crc kubenswrapper[5052]: I1203 12:31:30.623211 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/f9193a3b-a9c6-4343-a234-1616f73404cf-log-httpd\") pod \"f9193a3b-a9c6-4343-a234-1616f73404cf\" (UID: \"f9193a3b-a9c6-4343-a234-1616f73404cf\") " Dec 03 12:31:30 crc kubenswrapper[5052]: I1203 12:31:30.623291 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vnbhc\" (UniqueName: \"kubernetes.io/projected/f9193a3b-a9c6-4343-a234-1616f73404cf-kube-api-access-vnbhc\") pod \"f9193a3b-a9c6-4343-a234-1616f73404cf\" (UID: \"f9193a3b-a9c6-4343-a234-1616f73404cf\") " Dec 03 12:31:30 crc kubenswrapper[5052]: I1203 12:31:30.623325 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/f9193a3b-a9c6-4343-a234-1616f73404cf-ceilometer-tls-certs\") pod \"f9193a3b-a9c6-4343-a234-1616f73404cf\" (UID: \"f9193a3b-a9c6-4343-a234-1616f73404cf\") " Dec 03 12:31:30 crc kubenswrapper[5052]: I1203 12:31:30.623376 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/f9193a3b-a9c6-4343-a234-1616f73404cf-run-httpd\") pod \"f9193a3b-a9c6-4343-a234-1616f73404cf\" (UID: \"f9193a3b-a9c6-4343-a234-1616f73404cf\") " Dec 03 12:31:30 crc kubenswrapper[5052]: I1203 12:31:30.623398 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/f9193a3b-a9c6-4343-a234-1616f73404cf-config-data\") pod \"f9193a3b-a9c6-4343-a234-1616f73404cf\" (UID: \"f9193a3b-a9c6-4343-a234-1616f73404cf\") " Dec 03 12:31:30 crc kubenswrapper[5052]: I1203 12:31:30.623476 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/f9193a3b-a9c6-4343-a234-1616f73404cf-sg-core-conf-yaml\") pod \"f9193a3b-a9c6-4343-a234-1616f73404cf\" (UID: \"f9193a3b-a9c6-4343-a234-1616f73404cf\") " Dec 03 12:31:30 crc kubenswrapper[5052]: I1203 12:31:30.623540 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f9193a3b-a9c6-4343-a234-1616f73404cf-combined-ca-bundle\") pod \"f9193a3b-a9c6-4343-a234-1616f73404cf\" (UID: \"f9193a3b-a9c6-4343-a234-1616f73404cf\") " Dec 03 12:31:30 crc kubenswrapper[5052]: I1203 12:31:30.623570 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/f9193a3b-a9c6-4343-a234-1616f73404cf-scripts\") pod \"f9193a3b-a9c6-4343-a234-1616f73404cf\" (UID: \"f9193a3b-a9c6-4343-a234-1616f73404cf\") " Dec 03 12:31:30 crc kubenswrapper[5052]: I1203 12:31:30.623804 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f9193a3b-a9c6-4343-a234-1616f73404cf-log-httpd" (OuterVolumeSpecName: "log-httpd") pod "f9193a3b-a9c6-4343-a234-1616f73404cf" (UID: "f9193a3b-a9c6-4343-a234-1616f73404cf"). InnerVolumeSpecName "log-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:31:30 crc kubenswrapper[5052]: I1203 12:31:30.624289 5052 reconciler_common.go:293] "Volume detached for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/f9193a3b-a9c6-4343-a234-1616f73404cf-log-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:31:30 crc kubenswrapper[5052]: I1203 12:31:30.624672 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f9193a3b-a9c6-4343-a234-1616f73404cf-run-httpd" (OuterVolumeSpecName: "run-httpd") pod "f9193a3b-a9c6-4343-a234-1616f73404cf" (UID: "f9193a3b-a9c6-4343-a234-1616f73404cf"). InnerVolumeSpecName "run-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:31:30 crc kubenswrapper[5052]: I1203 12:31:30.630857 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f9193a3b-a9c6-4343-a234-1616f73404cf-scripts" (OuterVolumeSpecName: "scripts") pod "f9193a3b-a9c6-4343-a234-1616f73404cf" (UID: "f9193a3b-a9c6-4343-a234-1616f73404cf"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:31:30 crc kubenswrapper[5052]: I1203 12:31:30.630927 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f9193a3b-a9c6-4343-a234-1616f73404cf-kube-api-access-vnbhc" (OuterVolumeSpecName: "kube-api-access-vnbhc") pod "f9193a3b-a9c6-4343-a234-1616f73404cf" (UID: "f9193a3b-a9c6-4343-a234-1616f73404cf"). InnerVolumeSpecName "kube-api-access-vnbhc". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:31:30 crc kubenswrapper[5052]: I1203 12:31:30.657869 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f9193a3b-a9c6-4343-a234-1616f73404cf-sg-core-conf-yaml" (OuterVolumeSpecName: "sg-core-conf-yaml") pod "f9193a3b-a9c6-4343-a234-1616f73404cf" (UID: "f9193a3b-a9c6-4343-a234-1616f73404cf"). InnerVolumeSpecName "sg-core-conf-yaml". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:31:30 crc kubenswrapper[5052]: I1203 12:31:30.683145 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f9193a3b-a9c6-4343-a234-1616f73404cf-ceilometer-tls-certs" (OuterVolumeSpecName: "ceilometer-tls-certs") pod "f9193a3b-a9c6-4343-a234-1616f73404cf" (UID: "f9193a3b-a9c6-4343-a234-1616f73404cf"). InnerVolumeSpecName "ceilometer-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:31:30 crc kubenswrapper[5052]: I1203 12:31:30.701389 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f9193a3b-a9c6-4343-a234-1616f73404cf-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "f9193a3b-a9c6-4343-a234-1616f73404cf" (UID: "f9193a3b-a9c6-4343-a234-1616f73404cf"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:31:30 crc kubenswrapper[5052]: I1203 12:31:30.725774 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f9193a3b-a9c6-4343-a234-1616f73404cf-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:31:30 crc kubenswrapper[5052]: I1203 12:31:30.725811 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/f9193a3b-a9c6-4343-a234-1616f73404cf-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:31:30 crc kubenswrapper[5052]: I1203 12:31:30.725824 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-vnbhc\" (UniqueName: \"kubernetes.io/projected/f9193a3b-a9c6-4343-a234-1616f73404cf-kube-api-access-vnbhc\") on node \"crc\" DevicePath \"\"" Dec 03 12:31:30 crc kubenswrapper[5052]: I1203 12:31:30.725837 5052 reconciler_common.go:293] "Volume detached for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/f9193a3b-a9c6-4343-a234-1616f73404cf-ceilometer-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:31:30 crc kubenswrapper[5052]: I1203 12:31:30.725848 5052 reconciler_common.go:293] "Volume detached for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/f9193a3b-a9c6-4343-a234-1616f73404cf-run-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:31:30 crc kubenswrapper[5052]: I1203 12:31:30.725860 5052 reconciler_common.go:293] "Volume detached for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/f9193a3b-a9c6-4343-a234-1616f73404cf-sg-core-conf-yaml\") on node \"crc\" DevicePath \"\"" Dec 03 12:31:30 crc kubenswrapper[5052]: I1203 12:31:30.741078 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f9193a3b-a9c6-4343-a234-1616f73404cf-config-data" (OuterVolumeSpecName: "config-data") pod "f9193a3b-a9c6-4343-a234-1616f73404cf" (UID: "f9193a3b-a9c6-4343-a234-1616f73404cf"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:31:30 crc kubenswrapper[5052]: I1203 12:31:30.827339 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/f9193a3b-a9c6-4343-a234-1616f73404cf-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:31:30 crc kubenswrapper[5052]: I1203 12:31:30.938745 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"f9193a3b-a9c6-4343-a234-1616f73404cf","Type":"ContainerDied","Data":"4eb1fdeb5b52ce5c389d9427885df5067888badb5bc0239fa6056a03765ab6ad"} Dec 03 12:31:30 crc kubenswrapper[5052]: I1203 12:31:30.938803 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:31:30 crc kubenswrapper[5052]: I1203 12:31:30.938822 5052 scope.go:117] "RemoveContainer" containerID="d5a506b8e44bdc2bb0161dbf172c464d8a512672b56f42caecb0a6ec89bd9715" Dec 03 12:31:30 crc kubenswrapper[5052]: I1203 12:31:30.966805 5052 scope.go:117] "RemoveContainer" containerID="aeb74a268ef4cbfcdaec7b12ce9c082930f2b926d389f3e0fb132ec3d9fa328c" Dec 03 12:31:30 crc kubenswrapper[5052]: I1203 12:31:30.989939 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:31:31 crc kubenswrapper[5052]: I1203 12:31:31.008424 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:31:31 crc kubenswrapper[5052]: I1203 12:31:31.011224 5052 scope.go:117] "RemoveContainer" containerID="5eba7bcd1112f7d19e289be438b4a2d50309b5fe47adfc505eb50370db185f89" Dec 03 12:31:31 crc kubenswrapper[5052]: I1203 12:31:31.026634 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:31:31 crc kubenswrapper[5052]: E1203 12:31:31.027172 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="6af456b9-3c1e-4d13-ab42-843b94a6eccc" containerName="proxy-server" Dec 03 12:31:31 crc kubenswrapper[5052]: I1203 12:31:31.027197 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="6af456b9-3c1e-4d13-ab42-843b94a6eccc" containerName="proxy-server" Dec 03 12:31:31 crc kubenswrapper[5052]: E1203 12:31:31.027222 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="5c13362a-3cdf-4ff3-b57d-eb8c6cdcdc98" containerName="neutron-api" Dec 03 12:31:31 crc kubenswrapper[5052]: I1203 12:31:31.027232 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c13362a-3cdf-4ff3-b57d-eb8c6cdcdc98" containerName="neutron-api" Dec 03 12:31:31 crc kubenswrapper[5052]: E1203 12:31:31.027250 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="82331185-51eb-4ef5-8cf6-61dc74b73740" containerName="placement-api" Dec 03 12:31:31 crc kubenswrapper[5052]: I1203 12:31:31.027260 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="82331185-51eb-4ef5-8cf6-61dc74b73740" containerName="placement-api" Dec 03 12:31:31 crc kubenswrapper[5052]: E1203 12:31:31.027283 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="f9193a3b-a9c6-4343-a234-1616f73404cf" containerName="ceilometer-notification-agent" Dec 03 12:31:31 crc kubenswrapper[5052]: I1203 12:31:31.027295 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="f9193a3b-a9c6-4343-a234-1616f73404cf" containerName="ceilometer-notification-agent" Dec 03 12:31:31 crc kubenswrapper[5052]: E1203 12:31:31.027322 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="5c13362a-3cdf-4ff3-b57d-eb8c6cdcdc98" containerName="neutron-httpd" Dec 03 12:31:31 crc kubenswrapper[5052]: I1203 12:31:31.027352 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c13362a-3cdf-4ff3-b57d-eb8c6cdcdc98" containerName="neutron-httpd" Dec 03 12:31:31 crc kubenswrapper[5052]: E1203 12:31:31.027378 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="f9193a3b-a9c6-4343-a234-1616f73404cf" containerName="proxy-httpd" Dec 03 12:31:31 crc kubenswrapper[5052]: I1203 12:31:31.027389 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="f9193a3b-a9c6-4343-a234-1616f73404cf" containerName="proxy-httpd" Dec 03 12:31:31 crc kubenswrapper[5052]: E1203 12:31:31.027407 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="6af456b9-3c1e-4d13-ab42-843b94a6eccc" containerName="proxy-httpd" Dec 03 12:31:31 crc kubenswrapper[5052]: I1203 12:31:31.027418 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="6af456b9-3c1e-4d13-ab42-843b94a6eccc" containerName="proxy-httpd" Dec 03 12:31:31 crc kubenswrapper[5052]: E1203 12:31:31.027437 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="f9193a3b-a9c6-4343-a234-1616f73404cf" containerName="ceilometer-central-agent" Dec 03 12:31:31 crc kubenswrapper[5052]: I1203 12:31:31.027451 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="f9193a3b-a9c6-4343-a234-1616f73404cf" containerName="ceilometer-central-agent" Dec 03 12:31:31 crc kubenswrapper[5052]: E1203 12:31:31.027481 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="760aa987-6355-40cb-97c0-16c7eeddd135" containerName="neutron-api" Dec 03 12:31:31 crc kubenswrapper[5052]: I1203 12:31:31.027489 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="760aa987-6355-40cb-97c0-16c7eeddd135" containerName="neutron-api" Dec 03 12:31:31 crc kubenswrapper[5052]: E1203 12:31:31.027506 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="6bfd5f51-8b37-4fea-8134-a2cdcd4f6049" containerName="neutron-api" Dec 03 12:31:31 crc kubenswrapper[5052]: I1203 12:31:31.027538 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="6bfd5f51-8b37-4fea-8134-a2cdcd4f6049" containerName="neutron-api" Dec 03 12:31:31 crc kubenswrapper[5052]: E1203 12:31:31.027561 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="760aa987-6355-40cb-97c0-16c7eeddd135" containerName="neutron-httpd" Dec 03 12:31:31 crc kubenswrapper[5052]: I1203 12:31:31.027570 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="760aa987-6355-40cb-97c0-16c7eeddd135" containerName="neutron-httpd" Dec 03 12:31:31 crc kubenswrapper[5052]: E1203 12:31:31.027591 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="6bfd5f51-8b37-4fea-8134-a2cdcd4f6049" containerName="neutron-httpd" Dec 03 12:31:31 crc kubenswrapper[5052]: I1203 12:31:31.027599 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="6bfd5f51-8b37-4fea-8134-a2cdcd4f6049" containerName="neutron-httpd" Dec 03 12:31:31 crc kubenswrapper[5052]: E1203 12:31:31.027614 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="f9193a3b-a9c6-4343-a234-1616f73404cf" containerName="sg-core" Dec 03 12:31:31 crc kubenswrapper[5052]: I1203 12:31:31.027623 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="f9193a3b-a9c6-4343-a234-1616f73404cf" containerName="sg-core" Dec 03 12:31:31 crc kubenswrapper[5052]: E1203 12:31:31.027636 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="82331185-51eb-4ef5-8cf6-61dc74b73740" containerName="placement-log" Dec 03 12:31:31 crc kubenswrapper[5052]: I1203 12:31:31.027645 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="82331185-51eb-4ef5-8cf6-61dc74b73740" containerName="placement-log" Dec 03 12:31:31 crc kubenswrapper[5052]: I1203 12:31:31.027922 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="760aa987-6355-40cb-97c0-16c7eeddd135" containerName="neutron-api" Dec 03 12:31:31 crc kubenswrapper[5052]: I1203 12:31:31.027942 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="f9193a3b-a9c6-4343-a234-1616f73404cf" containerName="ceilometer-central-agent" Dec 03 12:31:31 crc kubenswrapper[5052]: I1203 12:31:31.027959 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="5c13362a-3cdf-4ff3-b57d-eb8c6cdcdc98" containerName="neutron-api" Dec 03 12:31:31 crc kubenswrapper[5052]: I1203 12:31:31.027978 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="f9193a3b-a9c6-4343-a234-1616f73404cf" containerName="proxy-httpd" Dec 03 12:31:31 crc kubenswrapper[5052]: I1203 12:31:31.027995 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="5c13362a-3cdf-4ff3-b57d-eb8c6cdcdc98" containerName="neutron-httpd" Dec 03 12:31:31 crc kubenswrapper[5052]: I1203 12:31:31.028006 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="6af456b9-3c1e-4d13-ab42-843b94a6eccc" containerName="proxy-server" Dec 03 12:31:31 crc kubenswrapper[5052]: I1203 12:31:31.028017 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="760aa987-6355-40cb-97c0-16c7eeddd135" containerName="neutron-httpd" Dec 03 12:31:31 crc kubenswrapper[5052]: I1203 12:31:31.028038 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="f9193a3b-a9c6-4343-a234-1616f73404cf" containerName="ceilometer-notification-agent" Dec 03 12:31:31 crc kubenswrapper[5052]: I1203 12:31:31.028055 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="6bfd5f51-8b37-4fea-8134-a2cdcd4f6049" containerName="neutron-httpd" Dec 03 12:31:31 crc kubenswrapper[5052]: I1203 12:31:31.028066 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="f9193a3b-a9c6-4343-a234-1616f73404cf" containerName="sg-core" Dec 03 12:31:31 crc kubenswrapper[5052]: I1203 12:31:31.028082 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="6bfd5f51-8b37-4fea-8134-a2cdcd4f6049" containerName="neutron-api" Dec 03 12:31:31 crc kubenswrapper[5052]: I1203 12:31:31.028100 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="82331185-51eb-4ef5-8cf6-61dc74b73740" containerName="placement-api" Dec 03 12:31:31 crc kubenswrapper[5052]: I1203 12:31:31.028113 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="6af456b9-3c1e-4d13-ab42-843b94a6eccc" containerName="proxy-httpd" Dec 03 12:31:31 crc kubenswrapper[5052]: I1203 12:31:31.028126 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="82331185-51eb-4ef5-8cf6-61dc74b73740" containerName="placement-log" Dec 03 12:31:31 crc kubenswrapper[5052]: I1203 12:31:31.030896 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:31:31 crc kubenswrapper[5052]: I1203 12:31:31.036609 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:31:31 crc kubenswrapper[5052]: I1203 12:31:31.037103 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-scripts" Dec 03 12:31:31 crc kubenswrapper[5052]: I1203 12:31:31.038289 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-ceilometer-internal-svc" Dec 03 12:31:31 crc kubenswrapper[5052]: I1203 12:31:31.042125 5052 scope.go:117] "RemoveContainer" containerID="29b7f054b1ac49030c5a40b904711ccba428b2c71382c48f025d25f4f0a30435" Dec 03 12:31:31 crc kubenswrapper[5052]: I1203 12:31:31.046623 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-config-data" Dec 03 12:31:31 crc kubenswrapper[5052]: I1203 12:31:31.133561 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/21690a66-4bcf-4865-91c7-cfe1cdf32a61-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"21690a66-4bcf-4865-91c7-cfe1cdf32a61\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:31:31 crc kubenswrapper[5052]: I1203 12:31:31.133640 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/21690a66-4bcf-4865-91c7-cfe1cdf32a61-config-data\") pod \"ceilometer-0\" (UID: \"21690a66-4bcf-4865-91c7-cfe1cdf32a61\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:31:31 crc kubenswrapper[5052]: I1203 12:31:31.133709 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/21690a66-4bcf-4865-91c7-cfe1cdf32a61-run-httpd\") pod \"ceilometer-0\" (UID: \"21690a66-4bcf-4865-91c7-cfe1cdf32a61\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:31:31 crc kubenswrapper[5052]: I1203 12:31:31.133746 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/21690a66-4bcf-4865-91c7-cfe1cdf32a61-log-httpd\") pod \"ceilometer-0\" (UID: \"21690a66-4bcf-4865-91c7-cfe1cdf32a61\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:31:31 crc kubenswrapper[5052]: I1203 12:31:31.133786 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/21690a66-4bcf-4865-91c7-cfe1cdf32a61-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"21690a66-4bcf-4865-91c7-cfe1cdf32a61\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:31:31 crc kubenswrapper[5052]: I1203 12:31:31.134114 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/21690a66-4bcf-4865-91c7-cfe1cdf32a61-ceilometer-tls-certs\") pod \"ceilometer-0\" (UID: \"21690a66-4bcf-4865-91c7-cfe1cdf32a61\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:31:31 crc kubenswrapper[5052]: I1203 12:31:31.134257 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6bptq\" (UniqueName: \"kubernetes.io/projected/21690a66-4bcf-4865-91c7-cfe1cdf32a61-kube-api-access-6bptq\") pod \"ceilometer-0\" (UID: \"21690a66-4bcf-4865-91c7-cfe1cdf32a61\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:31:31 crc kubenswrapper[5052]: I1203 12:31:31.134612 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/21690a66-4bcf-4865-91c7-cfe1cdf32a61-scripts\") pod \"ceilometer-0\" (UID: \"21690a66-4bcf-4865-91c7-cfe1cdf32a61\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:31:31 crc kubenswrapper[5052]: I1203 12:31:31.236737 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/21690a66-4bcf-4865-91c7-cfe1cdf32a61-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"21690a66-4bcf-4865-91c7-cfe1cdf32a61\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:31:31 crc kubenswrapper[5052]: I1203 12:31:31.236809 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/21690a66-4bcf-4865-91c7-cfe1cdf32a61-config-data\") pod \"ceilometer-0\" (UID: \"21690a66-4bcf-4865-91c7-cfe1cdf32a61\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:31:31 crc kubenswrapper[5052]: I1203 12:31:31.236856 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/21690a66-4bcf-4865-91c7-cfe1cdf32a61-run-httpd\") pod \"ceilometer-0\" (UID: \"21690a66-4bcf-4865-91c7-cfe1cdf32a61\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:31:31 crc kubenswrapper[5052]: I1203 12:31:31.236888 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/21690a66-4bcf-4865-91c7-cfe1cdf32a61-log-httpd\") pod \"ceilometer-0\" (UID: \"21690a66-4bcf-4865-91c7-cfe1cdf32a61\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:31:31 crc kubenswrapper[5052]: I1203 12:31:31.236911 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/21690a66-4bcf-4865-91c7-cfe1cdf32a61-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"21690a66-4bcf-4865-91c7-cfe1cdf32a61\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:31:31 crc kubenswrapper[5052]: I1203 12:31:31.236959 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/21690a66-4bcf-4865-91c7-cfe1cdf32a61-ceilometer-tls-certs\") pod \"ceilometer-0\" (UID: \"21690a66-4bcf-4865-91c7-cfe1cdf32a61\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:31:31 crc kubenswrapper[5052]: I1203 12:31:31.236994 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-6bptq\" (UniqueName: \"kubernetes.io/projected/21690a66-4bcf-4865-91c7-cfe1cdf32a61-kube-api-access-6bptq\") pod \"ceilometer-0\" (UID: \"21690a66-4bcf-4865-91c7-cfe1cdf32a61\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:31:31 crc kubenswrapper[5052]: I1203 12:31:31.237019 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/21690a66-4bcf-4865-91c7-cfe1cdf32a61-scripts\") pod \"ceilometer-0\" (UID: \"21690a66-4bcf-4865-91c7-cfe1cdf32a61\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:31:31 crc kubenswrapper[5052]: I1203 12:31:31.237367 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/21690a66-4bcf-4865-91c7-cfe1cdf32a61-run-httpd\") pod \"ceilometer-0\" (UID: \"21690a66-4bcf-4865-91c7-cfe1cdf32a61\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:31:31 crc kubenswrapper[5052]: I1203 12:31:31.237422 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/21690a66-4bcf-4865-91c7-cfe1cdf32a61-log-httpd\") pod \"ceilometer-0\" (UID: \"21690a66-4bcf-4865-91c7-cfe1cdf32a61\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:31:31 crc kubenswrapper[5052]: I1203 12:31:31.242103 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/21690a66-4bcf-4865-91c7-cfe1cdf32a61-scripts\") pod \"ceilometer-0\" (UID: \"21690a66-4bcf-4865-91c7-cfe1cdf32a61\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:31:31 crc kubenswrapper[5052]: I1203 12:31:31.252145 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/21690a66-4bcf-4865-91c7-cfe1cdf32a61-ceilometer-tls-certs\") pod \"ceilometer-0\" (UID: \"21690a66-4bcf-4865-91c7-cfe1cdf32a61\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:31:31 crc kubenswrapper[5052]: I1203 12:31:31.252191 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/21690a66-4bcf-4865-91c7-cfe1cdf32a61-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"21690a66-4bcf-4865-91c7-cfe1cdf32a61\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:31:31 crc kubenswrapper[5052]: I1203 12:31:31.252708 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/21690a66-4bcf-4865-91c7-cfe1cdf32a61-config-data\") pod \"ceilometer-0\" (UID: \"21690a66-4bcf-4865-91c7-cfe1cdf32a61\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:31:31 crc kubenswrapper[5052]: I1203 12:31:31.254003 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/21690a66-4bcf-4865-91c7-cfe1cdf32a61-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"21690a66-4bcf-4865-91c7-cfe1cdf32a61\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:31:31 crc kubenswrapper[5052]: I1203 12:31:31.259044 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-6bptq\" (UniqueName: \"kubernetes.io/projected/21690a66-4bcf-4865-91c7-cfe1cdf32a61-kube-api-access-6bptq\") pod \"ceilometer-0\" (UID: \"21690a66-4bcf-4865-91c7-cfe1cdf32a61\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:31:31 crc kubenswrapper[5052]: I1203 12:31:31.355356 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:31:31 crc kubenswrapper[5052]: I1203 12:31:31.686174 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f9193a3b-a9c6-4343-a234-1616f73404cf" path="/var/lib/kubelet/pods/f9193a3b-a9c6-4343-a234-1616f73404cf/volumes" Dec 03 12:31:31 crc kubenswrapper[5052]: I1203 12:31:31.887747 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:31:31 crc kubenswrapper[5052]: I1203 12:31:31.949139 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"21690a66-4bcf-4865-91c7-cfe1cdf32a61","Type":"ContainerStarted","Data":"8b11486373024ed488c2261d2db65b77b031d3040b3593ca109abaa9ab812230"} Dec 03 12:31:36 crc kubenswrapper[5052]: I1203 12:31:36.008755 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"21690a66-4bcf-4865-91c7-cfe1cdf32a61","Type":"ContainerStarted","Data":"3c2c53aa7eb07d8c8a4be850fb78e1b62b32812e4e50fd988ba119d0f7c3a22a"} Dec 03 12:31:37 crc kubenswrapper[5052]: I1203 12:31:37.020155 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"21690a66-4bcf-4865-91c7-cfe1cdf32a61","Type":"ContainerStarted","Data":"6cd8e96f6ff38ec31288f641a86e7f8fed72b9e87063c8cee94ed8bad6b902cc"} Dec 03 12:31:39 crc kubenswrapper[5052]: I1203 12:31:39.043082 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"21690a66-4bcf-4865-91c7-cfe1cdf32a61","Type":"ContainerStarted","Data":"4588847c60a7e1e54c1d68283ca53c647caedbb74b422846f258c273285a8d29"} Dec 03 12:31:41 crc kubenswrapper[5052]: I1203 12:31:41.085090 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"21690a66-4bcf-4865-91c7-cfe1cdf32a61","Type":"ContainerStarted","Data":"fb673c0dc2e5ca2f8ef44bb06008daf9443cc163a93859ac48d11b2d619a8092"} Dec 03 12:31:41 crc kubenswrapper[5052]: I1203 12:31:41.085711 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:31:41 crc kubenswrapper[5052]: I1203 12:31:41.110119 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/ceilometer-0" podStartSLOduration=2.391754304 podStartE2EDuration="11.110099464s" podCreationTimestamp="2025-12-03 12:31:30 +0000 UTC" firstStartedPulling="2025-12-03 12:31:31.893603507 +0000 UTC m=+4282.481845271" lastFinishedPulling="2025-12-03 12:31:40.611948667 +0000 UTC m=+4291.200190431" observedRunningTime="2025-12-03 12:31:41.104436253 +0000 UTC m=+4291.692678017" watchObservedRunningTime="2025-12-03 12:31:41.110099464 +0000 UTC m=+4291.698341228" Dec 03 12:31:59 crc kubenswrapper[5052]: I1203 12:31:59.379213 5052 patch_prober.go:28] interesting pod/machine-config-daemon-74bj6 container/machine-config-daemon namespace/openshift-machine-config-operator: Liveness probe status=failure output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" start-of-body= Dec 03 12:31:59 crc kubenswrapper[5052]: I1203 12:31:59.380024 5052 prober.go:107] "Probe failed" probeType="Liveness" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" containerName="machine-config-daemon" probeResult="failure" output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" Dec 03 12:31:59 crc kubenswrapper[5052]: I1203 12:31:59.380091 5052 kubelet.go:2542] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" Dec 03 12:31:59 crc kubenswrapper[5052]: I1203 12:31:59.381313 5052 kuberuntime_manager.go:1027] "Message for Container of pod" containerName="machine-config-daemon" containerStatusID={"Type":"cri-o","ID":"e6edaf188f85ea4f11f487f11cc489c700f25fb42b5ee269f4f496f1e127d772"} pod="openshift-machine-config-operator/machine-config-daemon-74bj6" containerMessage="Container machine-config-daemon failed liveness probe, will be restarted" Dec 03 12:31:59 crc kubenswrapper[5052]: I1203 12:31:59.381396 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" containerName="machine-config-daemon" containerID="cri-o://e6edaf188f85ea4f11f487f11cc489c700f25fb42b5ee269f4f496f1e127d772" gracePeriod=600 Dec 03 12:32:01 crc kubenswrapper[5052]: I1203 12:32:01.363731 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:32:07 crc kubenswrapper[5052]: E1203 12:32:07.722105 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 12:32:08 crc kubenswrapper[5052]: I1203 12:32:08.411420 5052 generic.go:334] "Generic (PLEG): container finished" podID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" containerID="e6edaf188f85ea4f11f487f11cc489c700f25fb42b5ee269f4f496f1e127d772" exitCode=0 Dec 03 12:32:08 crc kubenswrapper[5052]: I1203 12:32:08.411499 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" event={"ID":"10c237bc-72e7-4da3-9f2c-b1688395bf6e","Type":"ContainerDied","Data":"e6edaf188f85ea4f11f487f11cc489c700f25fb42b5ee269f4f496f1e127d772"} Dec 03 12:32:08 crc kubenswrapper[5052]: I1203 12:32:08.411828 5052 scope.go:117] "RemoveContainer" containerID="658f4d3362e1d46f122d256822917d243f01c3aff5e32c450b855cf70ebcd2f9" Dec 03 12:32:08 crc kubenswrapper[5052]: I1203 12:32:08.412597 5052 scope.go:117] "RemoveContainer" containerID="e6edaf188f85ea4f11f487f11cc489c700f25fb42b5ee269f4f496f1e127d772" Dec 03 12:32:08 crc kubenswrapper[5052]: E1203 12:32:08.413012 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 12:32:18 crc kubenswrapper[5052]: I1203 12:32:18.265358 5052 scope.go:117] "RemoveContainer" containerID="1219127b0f28f232cebab108b969dfccbae824421c1a95a3735263eb1e8796a0" Dec 03 12:32:18 crc kubenswrapper[5052]: I1203 12:32:18.296453 5052 scope.go:117] "RemoveContainer" containerID="db9649b19c8444660353379457a673b760daed7d6142434b80ddd84a4d2f0f10" Dec 03 12:32:18 crc kubenswrapper[5052]: I1203 12:32:18.316875 5052 scope.go:117] "RemoveContainer" containerID="875ac2341ca0230e518265fea1fa835454614e579ba0e283663e6e5e5484220a" Dec 03 12:32:18 crc kubenswrapper[5052]: I1203 12:32:18.341494 5052 scope.go:117] "RemoveContainer" containerID="e9967011ef9d1750324e927a4b9e79d05b9a0a1c3f9835799be84de44b11270d" Dec 03 12:32:18 crc kubenswrapper[5052]: I1203 12:32:18.370300 5052 scope.go:117] "RemoveContainer" containerID="3627f5e2b19aa5df937c1c6ec05a72cfd712e75b011381fe1b0ec8c35bc881bf" Dec 03 12:32:18 crc kubenswrapper[5052]: I1203 12:32:18.628583 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/openstackclient"] Dec 03 12:32:18 crc kubenswrapper[5052]: I1203 12:32:18.628967 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/openstackclient" podUID="2ed8a31a-9033-4601-84bf-9eaf57f1f155" containerName="openstackclient" containerID="cri-o://220531c8139609a0b36f479504e8821b1f41b4615b00ee05432f9560ade3acb1" gracePeriod=2 Dec 03 12:32:18 crc kubenswrapper[5052]: I1203 12:32:18.651546 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/openstackclient"] Dec 03 12:32:18 crc kubenswrapper[5052]: I1203 12:32:18.884718 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/barbican-worker-c899dcc5-n6rdv"] Dec 03 12:32:18 crc kubenswrapper[5052]: E1203 12:32:18.885462 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="2ed8a31a-9033-4601-84bf-9eaf57f1f155" containerName="openstackclient" Dec 03 12:32:18 crc kubenswrapper[5052]: I1203 12:32:18.885477 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="2ed8a31a-9033-4601-84bf-9eaf57f1f155" containerName="openstackclient" Dec 03 12:32:18 crc kubenswrapper[5052]: I1203 12:32:18.885665 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="2ed8a31a-9033-4601-84bf-9eaf57f1f155" containerName="openstackclient" Dec 03 12:32:18 crc kubenswrapper[5052]: I1203 12:32:18.886768 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-worker-c899dcc5-n6rdv" Dec 03 12:32:18 crc kubenswrapper[5052]: I1203 12:32:18.923406 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/cc8c914a-cb83-48ae-97a7-fca144e96f52-logs\") pod \"barbican-worker-c899dcc5-n6rdv\" (UID: \"cc8c914a-cb83-48ae-97a7-fca144e96f52\") " pod="openstack-kuttl-tests/barbican-worker-c899dcc5-n6rdv" Dec 03 12:32:18 crc kubenswrapper[5052]: I1203 12:32:18.923463 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/cc8c914a-cb83-48ae-97a7-fca144e96f52-config-data-custom\") pod \"barbican-worker-c899dcc5-n6rdv\" (UID: \"cc8c914a-cb83-48ae-97a7-fca144e96f52\") " pod="openstack-kuttl-tests/barbican-worker-c899dcc5-n6rdv" Dec 03 12:32:18 crc kubenswrapper[5052]: I1203 12:32:18.923628 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/cc8c914a-cb83-48ae-97a7-fca144e96f52-combined-ca-bundle\") pod \"barbican-worker-c899dcc5-n6rdv\" (UID: \"cc8c914a-cb83-48ae-97a7-fca144e96f52\") " pod="openstack-kuttl-tests/barbican-worker-c899dcc5-n6rdv" Dec 03 12:32:18 crc kubenswrapper[5052]: I1203 12:32:18.923661 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/cc8c914a-cb83-48ae-97a7-fca144e96f52-config-data\") pod \"barbican-worker-c899dcc5-n6rdv\" (UID: \"cc8c914a-cb83-48ae-97a7-fca144e96f52\") " pod="openstack-kuttl-tests/barbican-worker-c899dcc5-n6rdv" Dec 03 12:32:18 crc kubenswrapper[5052]: I1203 12:32:18.923916 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tdw6g\" (UniqueName: \"kubernetes.io/projected/cc8c914a-cb83-48ae-97a7-fca144e96f52-kube-api-access-tdw6g\") pod \"barbican-worker-c899dcc5-n6rdv\" (UID: \"cc8c914a-cb83-48ae-97a7-fca144e96f52\") " pod="openstack-kuttl-tests/barbican-worker-c899dcc5-n6rdv" Dec 03 12:32:18 crc kubenswrapper[5052]: I1203 12:32:18.935745 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/barbican-keystone-listener-697474f89d-p7lwz"] Dec 03 12:32:18 crc kubenswrapper[5052]: I1203 12:32:18.941891 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-keystone-listener-697474f89d-p7lwz" Dec 03 12:32:18 crc kubenswrapper[5052]: I1203 12:32:18.968589 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/rabbitmq-server-0"] Dec 03 12:32:19 crc kubenswrapper[5052]: I1203 12:32:19.006266 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-worker-c899dcc5-n6rdv"] Dec 03 12:32:19 crc kubenswrapper[5052]: I1203 12:32:19.026844 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/cc8c914a-cb83-48ae-97a7-fca144e96f52-combined-ca-bundle\") pod \"barbican-worker-c899dcc5-n6rdv\" (UID: \"cc8c914a-cb83-48ae-97a7-fca144e96f52\") " pod="openstack-kuttl-tests/barbican-worker-c899dcc5-n6rdv" Dec 03 12:32:19 crc kubenswrapper[5052]: I1203 12:32:19.026902 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/cc8c914a-cb83-48ae-97a7-fca144e96f52-config-data\") pod \"barbican-worker-c899dcc5-n6rdv\" (UID: \"cc8c914a-cb83-48ae-97a7-fca144e96f52\") " pod="openstack-kuttl-tests/barbican-worker-c899dcc5-n6rdv" Dec 03 12:32:19 crc kubenswrapper[5052]: I1203 12:32:19.026957 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/045d578f-92c0-47c6-8356-59c489c57b9a-combined-ca-bundle\") pod \"barbican-keystone-listener-697474f89d-p7lwz\" (UID: \"045d578f-92c0-47c6-8356-59c489c57b9a\") " pod="openstack-kuttl-tests/barbican-keystone-listener-697474f89d-p7lwz" Dec 03 12:32:19 crc kubenswrapper[5052]: I1203 12:32:19.026996 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/045d578f-92c0-47c6-8356-59c489c57b9a-config-data\") pod \"barbican-keystone-listener-697474f89d-p7lwz\" (UID: \"045d578f-92c0-47c6-8356-59c489c57b9a\") " pod="openstack-kuttl-tests/barbican-keystone-listener-697474f89d-p7lwz" Dec 03 12:32:19 crc kubenswrapper[5052]: I1203 12:32:19.027021 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/045d578f-92c0-47c6-8356-59c489c57b9a-logs\") pod \"barbican-keystone-listener-697474f89d-p7lwz\" (UID: \"045d578f-92c0-47c6-8356-59c489c57b9a\") " pod="openstack-kuttl-tests/barbican-keystone-listener-697474f89d-p7lwz" Dec 03 12:32:19 crc kubenswrapper[5052]: I1203 12:32:19.027072 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-tdw6g\" (UniqueName: \"kubernetes.io/projected/cc8c914a-cb83-48ae-97a7-fca144e96f52-kube-api-access-tdw6g\") pod \"barbican-worker-c899dcc5-n6rdv\" (UID: \"cc8c914a-cb83-48ae-97a7-fca144e96f52\") " pod="openstack-kuttl-tests/barbican-worker-c899dcc5-n6rdv" Dec 03 12:32:19 crc kubenswrapper[5052]: I1203 12:32:19.027095 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/045d578f-92c0-47c6-8356-59c489c57b9a-config-data-custom\") pod \"barbican-keystone-listener-697474f89d-p7lwz\" (UID: \"045d578f-92c0-47c6-8356-59c489c57b9a\") " pod="openstack-kuttl-tests/barbican-keystone-listener-697474f89d-p7lwz" Dec 03 12:32:19 crc kubenswrapper[5052]: I1203 12:32:19.027144 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lflxx\" (UniqueName: \"kubernetes.io/projected/045d578f-92c0-47c6-8356-59c489c57b9a-kube-api-access-lflxx\") pod \"barbican-keystone-listener-697474f89d-p7lwz\" (UID: \"045d578f-92c0-47c6-8356-59c489c57b9a\") " pod="openstack-kuttl-tests/barbican-keystone-listener-697474f89d-p7lwz" Dec 03 12:32:19 crc kubenswrapper[5052]: I1203 12:32:19.027231 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/cc8c914a-cb83-48ae-97a7-fca144e96f52-logs\") pod \"barbican-worker-c899dcc5-n6rdv\" (UID: \"cc8c914a-cb83-48ae-97a7-fca144e96f52\") " pod="openstack-kuttl-tests/barbican-worker-c899dcc5-n6rdv" Dec 03 12:32:19 crc kubenswrapper[5052]: I1203 12:32:19.027268 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/cc8c914a-cb83-48ae-97a7-fca144e96f52-config-data-custom\") pod \"barbican-worker-c899dcc5-n6rdv\" (UID: \"cc8c914a-cb83-48ae-97a7-fca144e96f52\") " pod="openstack-kuttl-tests/barbican-worker-c899dcc5-n6rdv" Dec 03 12:32:19 crc kubenswrapper[5052]: E1203 12:32:19.028757 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/combined-ca-bundle: secret "combined-ca-bundle" not found Dec 03 12:32:19 crc kubenswrapper[5052]: E1203 12:32:19.028836 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cc8c914a-cb83-48ae-97a7-fca144e96f52-combined-ca-bundle podName:cc8c914a-cb83-48ae-97a7-fca144e96f52 nodeName:}" failed. No retries permitted until 2025-12-03 12:32:19.528813362 +0000 UTC m=+4330.117055126 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "combined-ca-bundle" (UniqueName: "kubernetes.io/secret/cc8c914a-cb83-48ae-97a7-fca144e96f52-combined-ca-bundle") pod "barbican-worker-c899dcc5-n6rdv" (UID: "cc8c914a-cb83-48ae-97a7-fca144e96f52") : secret "combined-ca-bundle" not found Dec 03 12:32:19 crc kubenswrapper[5052]: I1203 12:32:19.030093 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/cc8c914a-cb83-48ae-97a7-fca144e96f52-logs\") pod \"barbican-worker-c899dcc5-n6rdv\" (UID: \"cc8c914a-cb83-48ae-97a7-fca144e96f52\") " pod="openstack-kuttl-tests/barbican-worker-c899dcc5-n6rdv" Dec 03 12:32:19 crc kubenswrapper[5052]: I1203 12:32:19.059587 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/cc8c914a-cb83-48ae-97a7-fca144e96f52-config-data-custom\") pod \"barbican-worker-c899dcc5-n6rdv\" (UID: \"cc8c914a-cb83-48ae-97a7-fca144e96f52\") " pod="openstack-kuttl-tests/barbican-worker-c899dcc5-n6rdv" Dec 03 12:32:19 crc kubenswrapper[5052]: I1203 12:32:19.063505 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-keystone-listener-697474f89d-p7lwz"] Dec 03 12:32:19 crc kubenswrapper[5052]: I1203 12:32:19.084804 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/cc8c914a-cb83-48ae-97a7-fca144e96f52-config-data\") pod \"barbican-worker-c899dcc5-n6rdv\" (UID: \"cc8c914a-cb83-48ae-97a7-fca144e96f52\") " pod="openstack-kuttl-tests/barbican-worker-c899dcc5-n6rdv" Dec 03 12:32:19 crc kubenswrapper[5052]: I1203 12:32:19.161391 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/045d578f-92c0-47c6-8356-59c489c57b9a-combined-ca-bundle\") pod \"barbican-keystone-listener-697474f89d-p7lwz\" (UID: \"045d578f-92c0-47c6-8356-59c489c57b9a\") " pod="openstack-kuttl-tests/barbican-keystone-listener-697474f89d-p7lwz" Dec 03 12:32:19 crc kubenswrapper[5052]: I1203 12:32:19.161487 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/045d578f-92c0-47c6-8356-59c489c57b9a-config-data\") pod \"barbican-keystone-listener-697474f89d-p7lwz\" (UID: \"045d578f-92c0-47c6-8356-59c489c57b9a\") " pod="openstack-kuttl-tests/barbican-keystone-listener-697474f89d-p7lwz" Dec 03 12:32:19 crc kubenswrapper[5052]: I1203 12:32:19.161538 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/045d578f-92c0-47c6-8356-59c489c57b9a-logs\") pod \"barbican-keystone-listener-697474f89d-p7lwz\" (UID: \"045d578f-92c0-47c6-8356-59c489c57b9a\") " pod="openstack-kuttl-tests/barbican-keystone-listener-697474f89d-p7lwz" Dec 03 12:32:19 crc kubenswrapper[5052]: I1203 12:32:19.161617 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/045d578f-92c0-47c6-8356-59c489c57b9a-config-data-custom\") pod \"barbican-keystone-listener-697474f89d-p7lwz\" (UID: \"045d578f-92c0-47c6-8356-59c489c57b9a\") " pod="openstack-kuttl-tests/barbican-keystone-listener-697474f89d-p7lwz" Dec 03 12:32:19 crc kubenswrapper[5052]: I1203 12:32:19.161740 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-lflxx\" (UniqueName: \"kubernetes.io/projected/045d578f-92c0-47c6-8356-59c489c57b9a-kube-api-access-lflxx\") pod \"barbican-keystone-listener-697474f89d-p7lwz\" (UID: \"045d578f-92c0-47c6-8356-59c489c57b9a\") " pod="openstack-kuttl-tests/barbican-keystone-listener-697474f89d-p7lwz" Dec 03 12:32:19 crc kubenswrapper[5052]: E1203 12:32:19.162765 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/combined-ca-bundle: secret "combined-ca-bundle" not found Dec 03 12:32:19 crc kubenswrapper[5052]: E1203 12:32:19.162862 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/045d578f-92c0-47c6-8356-59c489c57b9a-combined-ca-bundle podName:045d578f-92c0-47c6-8356-59c489c57b9a nodeName:}" failed. No retries permitted until 2025-12-03 12:32:19.662840298 +0000 UTC m=+4330.251082062 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "combined-ca-bundle" (UniqueName: "kubernetes.io/secret/045d578f-92c0-47c6-8356-59c489c57b9a-combined-ca-bundle") pod "barbican-keystone-listener-697474f89d-p7lwz" (UID: "045d578f-92c0-47c6-8356-59c489c57b9a") : secret "combined-ca-bundle" not found Dec 03 12:32:19 crc kubenswrapper[5052]: I1203 12:32:19.164055 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-tdw6g\" (UniqueName: \"kubernetes.io/projected/cc8c914a-cb83-48ae-97a7-fca144e96f52-kube-api-access-tdw6g\") pod \"barbican-worker-c899dcc5-n6rdv\" (UID: \"cc8c914a-cb83-48ae-97a7-fca144e96f52\") " pod="openstack-kuttl-tests/barbican-worker-c899dcc5-n6rdv" Dec 03 12:32:19 crc kubenswrapper[5052]: E1203 12:32:19.164585 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/rabbitmq-config-data: configmap "rabbitmq-config-data" not found Dec 03 12:32:19 crc kubenswrapper[5052]: E1203 12:32:19.164648 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2c03f99a-98fd-4e86-ad29-fb69e94bd436-config-data podName:2c03f99a-98fd-4e86-ad29-fb69e94bd436 nodeName:}" failed. No retries permitted until 2025-12-03 12:32:19.664632109 +0000 UTC m=+4330.252873873 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "config-data" (UniqueName: "kubernetes.io/configmap/2c03f99a-98fd-4e86-ad29-fb69e94bd436-config-data") pod "rabbitmq-server-0" (UID: "2c03f99a-98fd-4e86-ad29-fb69e94bd436") : configmap "rabbitmq-config-data" not found Dec 03 12:32:19 crc kubenswrapper[5052]: I1203 12:32:19.170744 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/045d578f-92c0-47c6-8356-59c489c57b9a-logs\") pod \"barbican-keystone-listener-697474f89d-p7lwz\" (UID: \"045d578f-92c0-47c6-8356-59c489c57b9a\") " pod="openstack-kuttl-tests/barbican-keystone-listener-697474f89d-p7lwz" Dec 03 12:32:19 crc kubenswrapper[5052]: I1203 12:32:19.260195 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/045d578f-92c0-47c6-8356-59c489c57b9a-config-data\") pod \"barbican-keystone-listener-697474f89d-p7lwz\" (UID: \"045d578f-92c0-47c6-8356-59c489c57b9a\") " pod="openstack-kuttl-tests/barbican-keystone-listener-697474f89d-p7lwz" Dec 03 12:32:19 crc kubenswrapper[5052]: I1203 12:32:19.282748 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-lflxx\" (UniqueName: \"kubernetes.io/projected/045d578f-92c0-47c6-8356-59c489c57b9a-kube-api-access-lflxx\") pod \"barbican-keystone-listener-697474f89d-p7lwz\" (UID: \"045d578f-92c0-47c6-8356-59c489c57b9a\") " pod="openstack-kuttl-tests/barbican-keystone-listener-697474f89d-p7lwz" Dec 03 12:32:19 crc kubenswrapper[5052]: I1203 12:32:19.299415 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/045d578f-92c0-47c6-8356-59c489c57b9a-config-data-custom\") pod \"barbican-keystone-listener-697474f89d-p7lwz\" (UID: \"045d578f-92c0-47c6-8356-59c489c57b9a\") " pod="openstack-kuttl-tests/barbican-keystone-listener-697474f89d-p7lwz" Dec 03 12:32:19 crc kubenswrapper[5052]: I1203 12:32:19.372944 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/barbican-api-78dc8cddf6-kqfz7"] Dec 03 12:32:19 crc kubenswrapper[5052]: I1203 12:32:19.379088 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-api-78dc8cddf6-kqfz7" Dec 03 12:32:19 crc kubenswrapper[5052]: I1203 12:32:19.438068 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-api-78dc8cddf6-kqfz7"] Dec 03 12:32:19 crc kubenswrapper[5052]: I1203 12:32:19.464259 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-nb-0"] Dec 03 12:32:19 crc kubenswrapper[5052]: I1203 12:32:19.465116 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ovsdbserver-nb-0" podUID="dccae7c1-009e-4b3b-b66a-ecbf6e53489a" containerName="openstack-network-exporter" containerID="cri-o://3b528f3a5ecffc67ae3f8e59c8348c99e1798d2e9c1b8ecf277ece85a83c0756" gracePeriod=300 Dec 03 12:32:19 crc kubenswrapper[5052]: I1203 12:32:19.499573 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a3b48edc-15d0-4c5e-8a33-96bee20dd43e-config-data\") pod \"barbican-api-78dc8cddf6-kqfz7\" (UID: \"a3b48edc-15d0-4c5e-8a33-96bee20dd43e\") " pod="openstack-kuttl-tests/barbican-api-78dc8cddf6-kqfz7" Dec 03 12:32:19 crc kubenswrapper[5052]: I1203 12:32:19.499648 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/a3b48edc-15d0-4c5e-8a33-96bee20dd43e-config-data-custom\") pod \"barbican-api-78dc8cddf6-kqfz7\" (UID: \"a3b48edc-15d0-4c5e-8a33-96bee20dd43e\") " pod="openstack-kuttl-tests/barbican-api-78dc8cddf6-kqfz7" Dec 03 12:32:19 crc kubenswrapper[5052]: I1203 12:32:19.499867 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/a3b48edc-15d0-4c5e-8a33-96bee20dd43e-internal-tls-certs\") pod \"barbican-api-78dc8cddf6-kqfz7\" (UID: \"a3b48edc-15d0-4c5e-8a33-96bee20dd43e\") " pod="openstack-kuttl-tests/barbican-api-78dc8cddf6-kqfz7" Dec 03 12:32:19 crc kubenswrapper[5052]: I1203 12:32:19.499917 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/a3b48edc-15d0-4c5e-8a33-96bee20dd43e-logs\") pod \"barbican-api-78dc8cddf6-kqfz7\" (UID: \"a3b48edc-15d0-4c5e-8a33-96bee20dd43e\") " pod="openstack-kuttl-tests/barbican-api-78dc8cddf6-kqfz7" Dec 03 12:32:19 crc kubenswrapper[5052]: I1203 12:32:19.499952 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a3b48edc-15d0-4c5e-8a33-96bee20dd43e-combined-ca-bundle\") pod \"barbican-api-78dc8cddf6-kqfz7\" (UID: \"a3b48edc-15d0-4c5e-8a33-96bee20dd43e\") " pod="openstack-kuttl-tests/barbican-api-78dc8cddf6-kqfz7" Dec 03 12:32:19 crc kubenswrapper[5052]: I1203 12:32:19.500150 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fnt2q\" (UniqueName: \"kubernetes.io/projected/a3b48edc-15d0-4c5e-8a33-96bee20dd43e-kube-api-access-fnt2q\") pod \"barbican-api-78dc8cddf6-kqfz7\" (UID: \"a3b48edc-15d0-4c5e-8a33-96bee20dd43e\") " pod="openstack-kuttl-tests/barbican-api-78dc8cddf6-kqfz7" Dec 03 12:32:19 crc kubenswrapper[5052]: I1203 12:32:19.500186 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/a3b48edc-15d0-4c5e-8a33-96bee20dd43e-public-tls-certs\") pod \"barbican-api-78dc8cddf6-kqfz7\" (UID: \"a3b48edc-15d0-4c5e-8a33-96bee20dd43e\") " pod="openstack-kuttl-tests/barbican-api-78dc8cddf6-kqfz7" Dec 03 12:32:19 crc kubenswrapper[5052]: I1203 12:32:19.602130 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a3b48edc-15d0-4c5e-8a33-96bee20dd43e-combined-ca-bundle\") pod \"barbican-api-78dc8cddf6-kqfz7\" (UID: \"a3b48edc-15d0-4c5e-8a33-96bee20dd43e\") " pod="openstack-kuttl-tests/barbican-api-78dc8cddf6-kqfz7" Dec 03 12:32:19 crc kubenswrapper[5052]: I1203 12:32:19.602251 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/cc8c914a-cb83-48ae-97a7-fca144e96f52-combined-ca-bundle\") pod \"barbican-worker-c899dcc5-n6rdv\" (UID: \"cc8c914a-cb83-48ae-97a7-fca144e96f52\") " pod="openstack-kuttl-tests/barbican-worker-c899dcc5-n6rdv" Dec 03 12:32:19 crc kubenswrapper[5052]: I1203 12:32:19.602338 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-fnt2q\" (UniqueName: \"kubernetes.io/projected/a3b48edc-15d0-4c5e-8a33-96bee20dd43e-kube-api-access-fnt2q\") pod \"barbican-api-78dc8cddf6-kqfz7\" (UID: \"a3b48edc-15d0-4c5e-8a33-96bee20dd43e\") " pod="openstack-kuttl-tests/barbican-api-78dc8cddf6-kqfz7" Dec 03 12:32:19 crc kubenswrapper[5052]: I1203 12:32:19.602361 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/a3b48edc-15d0-4c5e-8a33-96bee20dd43e-public-tls-certs\") pod \"barbican-api-78dc8cddf6-kqfz7\" (UID: \"a3b48edc-15d0-4c5e-8a33-96bee20dd43e\") " pod="openstack-kuttl-tests/barbican-api-78dc8cddf6-kqfz7" Dec 03 12:32:19 crc kubenswrapper[5052]: I1203 12:32:19.602407 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a3b48edc-15d0-4c5e-8a33-96bee20dd43e-config-data\") pod \"barbican-api-78dc8cddf6-kqfz7\" (UID: \"a3b48edc-15d0-4c5e-8a33-96bee20dd43e\") " pod="openstack-kuttl-tests/barbican-api-78dc8cddf6-kqfz7" Dec 03 12:32:19 crc kubenswrapper[5052]: I1203 12:32:19.602433 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/a3b48edc-15d0-4c5e-8a33-96bee20dd43e-config-data-custom\") pod \"barbican-api-78dc8cddf6-kqfz7\" (UID: \"a3b48edc-15d0-4c5e-8a33-96bee20dd43e\") " pod="openstack-kuttl-tests/barbican-api-78dc8cddf6-kqfz7" Dec 03 12:32:19 crc kubenswrapper[5052]: I1203 12:32:19.602594 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/a3b48edc-15d0-4c5e-8a33-96bee20dd43e-internal-tls-certs\") pod \"barbican-api-78dc8cddf6-kqfz7\" (UID: \"a3b48edc-15d0-4c5e-8a33-96bee20dd43e\") " pod="openstack-kuttl-tests/barbican-api-78dc8cddf6-kqfz7" Dec 03 12:32:19 crc kubenswrapper[5052]: I1203 12:32:19.602638 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/a3b48edc-15d0-4c5e-8a33-96bee20dd43e-logs\") pod \"barbican-api-78dc8cddf6-kqfz7\" (UID: \"a3b48edc-15d0-4c5e-8a33-96bee20dd43e\") " pod="openstack-kuttl-tests/barbican-api-78dc8cddf6-kqfz7" Dec 03 12:32:19 crc kubenswrapper[5052]: I1203 12:32:19.603199 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/a3b48edc-15d0-4c5e-8a33-96bee20dd43e-logs\") pod \"barbican-api-78dc8cddf6-kqfz7\" (UID: \"a3b48edc-15d0-4c5e-8a33-96bee20dd43e\") " pod="openstack-kuttl-tests/barbican-api-78dc8cddf6-kqfz7" Dec 03 12:32:19 crc kubenswrapper[5052]: E1203 12:32:19.603333 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/combined-ca-bundle: secret "combined-ca-bundle" not found Dec 03 12:32:19 crc kubenswrapper[5052]: E1203 12:32:19.603398 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3b48edc-15d0-4c5e-8a33-96bee20dd43e-combined-ca-bundle podName:a3b48edc-15d0-4c5e-8a33-96bee20dd43e nodeName:}" failed. No retries permitted until 2025-12-03 12:32:20.103378782 +0000 UTC m=+4330.691620546 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "combined-ca-bundle" (UniqueName: "kubernetes.io/secret/a3b48edc-15d0-4c5e-8a33-96bee20dd43e-combined-ca-bundle") pod "barbican-api-78dc8cddf6-kqfz7" (UID: "a3b48edc-15d0-4c5e-8a33-96bee20dd43e") : secret "combined-ca-bundle" not found Dec 03 12:32:19 crc kubenswrapper[5052]: E1203 12:32:19.603742 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-barbican-internal-svc: secret "cert-barbican-internal-svc" not found Dec 03 12:32:19 crc kubenswrapper[5052]: E1203 12:32:19.603850 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3b48edc-15d0-4c5e-8a33-96bee20dd43e-internal-tls-certs podName:a3b48edc-15d0-4c5e-8a33-96bee20dd43e nodeName:}" failed. No retries permitted until 2025-12-03 12:32:20.103824374 +0000 UTC m=+4330.692066138 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "internal-tls-certs" (UniqueName: "kubernetes.io/secret/a3b48edc-15d0-4c5e-8a33-96bee20dd43e-internal-tls-certs") pod "barbican-api-78dc8cddf6-kqfz7" (UID: "a3b48edc-15d0-4c5e-8a33-96bee20dd43e") : secret "cert-barbican-internal-svc" not found Dec 03 12:32:19 crc kubenswrapper[5052]: E1203 12:32:19.603981 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/combined-ca-bundle: secret "combined-ca-bundle" not found Dec 03 12:32:19 crc kubenswrapper[5052]: E1203 12:32:19.604019 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cc8c914a-cb83-48ae-97a7-fca144e96f52-combined-ca-bundle podName:cc8c914a-cb83-48ae-97a7-fca144e96f52 nodeName:}" failed. No retries permitted until 2025-12-03 12:32:20.60401169 +0000 UTC m=+4331.192253454 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "combined-ca-bundle" (UniqueName: "kubernetes.io/secret/cc8c914a-cb83-48ae-97a7-fca144e96f52-combined-ca-bundle") pod "barbican-worker-c899dcc5-n6rdv" (UID: "cc8c914a-cb83-48ae-97a7-fca144e96f52") : secret "combined-ca-bundle" not found Dec 03 12:32:19 crc kubenswrapper[5052]: E1203 12:32:19.604087 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-barbican-public-svc: secret "cert-barbican-public-svc" not found Dec 03 12:32:19 crc kubenswrapper[5052]: E1203 12:32:19.604110 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3b48edc-15d0-4c5e-8a33-96bee20dd43e-public-tls-certs podName:a3b48edc-15d0-4c5e-8a33-96bee20dd43e nodeName:}" failed. No retries permitted until 2025-12-03 12:32:20.104103942 +0000 UTC m=+4330.692345706 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "public-tls-certs" (UniqueName: "kubernetes.io/secret/a3b48edc-15d0-4c5e-8a33-96bee20dd43e-public-tls-certs") pod "barbican-api-78dc8cddf6-kqfz7" (UID: "a3b48edc-15d0-4c5e-8a33-96bee20dd43e") : secret "cert-barbican-public-svc" not found Dec 03 12:32:19 crc kubenswrapper[5052]: I1203 12:32:19.609443 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/a3b48edc-15d0-4c5e-8a33-96bee20dd43e-config-data-custom\") pod \"barbican-api-78dc8cddf6-kqfz7\" (UID: \"a3b48edc-15d0-4c5e-8a33-96bee20dd43e\") " pod="openstack-kuttl-tests/barbican-api-78dc8cddf6-kqfz7" Dec 03 12:32:19 crc kubenswrapper[5052]: I1203 12:32:19.612620 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a3b48edc-15d0-4c5e-8a33-96bee20dd43e-config-data\") pod \"barbican-api-78dc8cddf6-kqfz7\" (UID: \"a3b48edc-15d0-4c5e-8a33-96bee20dd43e\") " pod="openstack-kuttl-tests/barbican-api-78dc8cddf6-kqfz7" Dec 03 12:32:19 crc kubenswrapper[5052]: I1203 12:32:19.624675 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ovsdbserver-nb-0" podUID="dccae7c1-009e-4b3b-b66a-ecbf6e53489a" containerName="ovsdbserver-nb" containerID="cri-o://da4594e855a05fe7faafd60c45e490f353a2622ba132821e7f56859f0a614ca4" gracePeriod=300 Dec 03 12:32:19 crc kubenswrapper[5052]: I1203 12:32:19.631242 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-fnt2q\" (UniqueName: \"kubernetes.io/projected/a3b48edc-15d0-4c5e-8a33-96bee20dd43e-kube-api-access-fnt2q\") pod \"barbican-api-78dc8cddf6-kqfz7\" (UID: \"a3b48edc-15d0-4c5e-8a33-96bee20dd43e\") " pod="openstack-kuttl-tests/barbican-api-78dc8cddf6-kqfz7" Dec 03 12:32:19 crc kubenswrapper[5052]: I1203 12:32:19.702590 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/rabbitmq-cell1-server-0"] Dec 03 12:32:19 crc kubenswrapper[5052]: I1203 12:32:19.704357 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/045d578f-92c0-47c6-8356-59c489c57b9a-combined-ca-bundle\") pod \"barbican-keystone-listener-697474f89d-p7lwz\" (UID: \"045d578f-92c0-47c6-8356-59c489c57b9a\") " pod="openstack-kuttl-tests/barbican-keystone-listener-697474f89d-p7lwz" Dec 03 12:32:19 crc kubenswrapper[5052]: E1203 12:32:19.704665 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/rabbitmq-config-data: configmap "rabbitmq-config-data" not found Dec 03 12:32:19 crc kubenswrapper[5052]: E1203 12:32:19.704712 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2c03f99a-98fd-4e86-ad29-fb69e94bd436-config-data podName:2c03f99a-98fd-4e86-ad29-fb69e94bd436 nodeName:}" failed. No retries permitted until 2025-12-03 12:32:20.704698694 +0000 UTC m=+4331.292940458 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "config-data" (UniqueName: "kubernetes.io/configmap/2c03f99a-98fd-4e86-ad29-fb69e94bd436-config-data") pod "rabbitmq-server-0" (UID: "2c03f99a-98fd-4e86-ad29-fb69e94bd436") : configmap "rabbitmq-config-data" not found Dec 03 12:32:19 crc kubenswrapper[5052]: E1203 12:32:19.704989 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/combined-ca-bundle: secret "combined-ca-bundle" not found Dec 03 12:32:19 crc kubenswrapper[5052]: E1203 12:32:19.705020 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/045d578f-92c0-47c6-8356-59c489c57b9a-combined-ca-bundle podName:045d578f-92c0-47c6-8356-59c489c57b9a nodeName:}" failed. No retries permitted until 2025-12-03 12:32:20.705011933 +0000 UTC m=+4331.293253697 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "combined-ca-bundle" (UniqueName: "kubernetes.io/secret/045d578f-92c0-47c6-8356-59c489c57b9a-combined-ca-bundle") pod "barbican-keystone-listener-697474f89d-p7lwz" (UID: "045d578f-92c0-47c6-8356-59c489c57b9a") : secret "combined-ca-bundle" not found Dec 03 12:32:19 crc kubenswrapper[5052]: I1203 12:32:19.779577 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/glance5b9b-account-delete-ggk6q"] Dec 03 12:32:19 crc kubenswrapper[5052]: I1203 12:32:19.781493 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance5b9b-account-delete-ggk6q" Dec 03 12:32:19 crc kubenswrapper[5052]: E1203 12:32:19.808009 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/rabbitmq-cell1-config-data: configmap "rabbitmq-cell1-config-data" not found Dec 03 12:32:19 crc kubenswrapper[5052]: E1203 12:32:19.808067 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b8613a94-1e2e-477f-8d52-713c70f5282c-config-data podName:b8613a94-1e2e-477f-8d52-713c70f5282c nodeName:}" failed. No retries permitted until 2025-12-03 12:32:20.308046663 +0000 UTC m=+4330.896288427 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "config-data" (UniqueName: "kubernetes.io/configmap/b8613a94-1e2e-477f-8d52-713c70f5282c-config-data") pod "rabbitmq-cell1-server-0" (UID: "b8613a94-1e2e-477f-8d52-713c70f5282c") : configmap "rabbitmq-cell1-config-data" not found Dec 03 12:32:19 crc kubenswrapper[5052]: E1203 12:32:19.809200 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cinder-config-data: secret "cinder-config-data" not found Dec 03 12:32:19 crc kubenswrapper[5052]: E1203 12:32:19.809240 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bb1723c7-4e41-4b0e-b6b8-f8024dafdaee-config-data podName:bb1723c7-4e41-4b0e-b6b8-f8024dafdaee nodeName:}" failed. No retries permitted until 2025-12-03 12:32:20.309229247 +0000 UTC m=+4330.897471011 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "config-data" (UniqueName: "kubernetes.io/secret/bb1723c7-4e41-4b0e-b6b8-f8024dafdaee-config-data") pod "cinder-scheduler-0" (UID: "bb1723c7-4e41-4b0e-b6b8-f8024dafdaee") : secret "cinder-config-data" not found Dec 03 12:32:19 crc kubenswrapper[5052]: E1203 12:32:19.809723 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cinder-scripts: secret "cinder-scripts" not found Dec 03 12:32:19 crc kubenswrapper[5052]: E1203 12:32:19.809797 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bb1723c7-4e41-4b0e-b6b8-f8024dafdaee-scripts podName:bb1723c7-4e41-4b0e-b6b8-f8024dafdaee nodeName:}" failed. No retries permitted until 2025-12-03 12:32:20.309777923 +0000 UTC m=+4330.898019687 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "scripts" (UniqueName: "kubernetes.io/secret/bb1723c7-4e41-4b0e-b6b8-f8024dafdaee-scripts") pod "cinder-scheduler-0" (UID: "bb1723c7-4e41-4b0e-b6b8-f8024dafdaee") : secret "cinder-scripts" not found Dec 03 12:32:19 crc kubenswrapper[5052]: E1203 12:32:19.809731 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/combined-ca-bundle: secret "combined-ca-bundle" not found Dec 03 12:32:19 crc kubenswrapper[5052]: E1203 12:32:19.810055 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bb1723c7-4e41-4b0e-b6b8-f8024dafdaee-combined-ca-bundle podName:bb1723c7-4e41-4b0e-b6b8-f8024dafdaee nodeName:}" failed. No retries permitted until 2025-12-03 12:32:20.31004505 +0000 UTC m=+4330.898286814 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "combined-ca-bundle" (UniqueName: "kubernetes.io/secret/bb1723c7-4e41-4b0e-b6b8-f8024dafdaee-combined-ca-bundle") pod "cinder-scheduler-0" (UID: "bb1723c7-4e41-4b0e-b6b8-f8024dafdaee") : secret "combined-ca-bundle" not found Dec 03 12:32:19 crc kubenswrapper[5052]: I1203 12:32:19.839411 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/neutron2174-account-delete-9dnxs"] Dec 03 12:32:19 crc kubenswrapper[5052]: I1203 12:32:19.841072 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron2174-account-delete-9dnxs" Dec 03 12:32:19 crc kubenswrapper[5052]: I1203 12:32:19.888094 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron2174-account-delete-9dnxs"] Dec 03 12:32:19 crc kubenswrapper[5052]: I1203 12:32:19.921149 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance5b9b-account-delete-ggk6q"] Dec 03 12:32:20 crc kubenswrapper[5052]: I1203 12:32:20.001971 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/placemente65a-account-delete-s7x52"] Dec 03 12:32:20 crc kubenswrapper[5052]: I1203 12:32:20.003583 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placemente65a-account-delete-s7x52" Dec 03 12:32:20 crc kubenswrapper[5052]: I1203 12:32:20.013875 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mwzzn\" (UniqueName: \"kubernetes.io/projected/ae9024f5-a2b4-49de-aed3-00cb9c806704-kube-api-access-mwzzn\") pod \"placemente65a-account-delete-s7x52\" (UID: \"ae9024f5-a2b4-49de-aed3-00cb9c806704\") " pod="openstack-kuttl-tests/placemente65a-account-delete-s7x52" Dec 03 12:32:20 crc kubenswrapper[5052]: I1203 12:32:20.013967 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/a2ef0804-75c7-43e3-95fa-ce901a561c9b-operator-scripts\") pod \"neutron2174-account-delete-9dnxs\" (UID: \"a2ef0804-75c7-43e3-95fa-ce901a561c9b\") " pod="openstack-kuttl-tests/neutron2174-account-delete-9dnxs" Dec 03 12:32:20 crc kubenswrapper[5052]: I1203 12:32:20.014071 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/ae9024f5-a2b4-49de-aed3-00cb9c806704-operator-scripts\") pod \"placemente65a-account-delete-s7x52\" (UID: \"ae9024f5-a2b4-49de-aed3-00cb9c806704\") " pod="openstack-kuttl-tests/placemente65a-account-delete-s7x52" Dec 03 12:32:20 crc kubenswrapper[5052]: I1203 12:32:20.014091 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rpq2v\" (UniqueName: \"kubernetes.io/projected/52bf0c8d-03b3-444b-80b6-13d41f5f9fb9-kube-api-access-rpq2v\") pod \"glance5b9b-account-delete-ggk6q\" (UID: \"52bf0c8d-03b3-444b-80b6-13d41f5f9fb9\") " pod="openstack-kuttl-tests/glance5b9b-account-delete-ggk6q" Dec 03 12:32:20 crc kubenswrapper[5052]: I1203 12:32:20.014196 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v2plr\" (UniqueName: \"kubernetes.io/projected/a2ef0804-75c7-43e3-95fa-ce901a561c9b-kube-api-access-v2plr\") pod \"neutron2174-account-delete-9dnxs\" (UID: \"a2ef0804-75c7-43e3-95fa-ce901a561c9b\") " pod="openstack-kuttl-tests/neutron2174-account-delete-9dnxs" Dec 03 12:32:20 crc kubenswrapper[5052]: I1203 12:32:20.014286 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/52bf0c8d-03b3-444b-80b6-13d41f5f9fb9-operator-scripts\") pod \"glance5b9b-account-delete-ggk6q\" (UID: \"52bf0c8d-03b3-444b-80b6-13d41f5f9fb9\") " pod="openstack-kuttl-tests/glance5b9b-account-delete-ggk6q" Dec 03 12:32:20 crc kubenswrapper[5052]: I1203 12:32:20.015906 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ovn-northd-0"] Dec 03 12:32:20 crc kubenswrapper[5052]: I1203 12:32:20.016192 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ovn-northd-0" podUID="a9f65397-a08a-4798-8532-646209c5d090" containerName="ovn-northd" containerID="cri-o://217fb246797b6f91c28f31f22b498303e560c0706448ba85948bf67b547612e3" gracePeriod=30 Dec 03 12:32:20 crc kubenswrapper[5052]: I1203 12:32:20.016286 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ovn-northd-0" podUID="a9f65397-a08a-4798-8532-646209c5d090" containerName="openstack-network-exporter" containerID="cri-o://ea36b7c04ef3193b039e70bd5df384774204953e99d07610a4a191cab306432f" gracePeriod=30 Dec 03 12:32:20 crc kubenswrapper[5052]: I1203 12:32:20.036338 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/cinderc151-account-delete-h4k5b"] Dec 03 12:32:20 crc kubenswrapper[5052]: I1203 12:32:20.045346 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinderc151-account-delete-h4k5b" Dec 03 12:32:20 crc kubenswrapper[5052]: I1203 12:32:20.069027 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/placemente65a-account-delete-s7x52"] Dec 03 12:32:20 crc kubenswrapper[5052]: I1203 12:32:20.116444 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-rpq2v\" (UniqueName: \"kubernetes.io/projected/52bf0c8d-03b3-444b-80b6-13d41f5f9fb9-kube-api-access-rpq2v\") pod \"glance5b9b-account-delete-ggk6q\" (UID: \"52bf0c8d-03b3-444b-80b6-13d41f5f9fb9\") " pod="openstack-kuttl-tests/glance5b9b-account-delete-ggk6q" Dec 03 12:32:20 crc kubenswrapper[5052]: I1203 12:32:20.116552 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/a3b48edc-15d0-4c5e-8a33-96bee20dd43e-public-tls-certs\") pod \"barbican-api-78dc8cddf6-kqfz7\" (UID: \"a3b48edc-15d0-4c5e-8a33-96bee20dd43e\") " pod="openstack-kuttl-tests/barbican-api-78dc8cddf6-kqfz7" Dec 03 12:32:20 crc kubenswrapper[5052]: I1203 12:32:20.116624 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/fc428005-b653-4a6a-a50d-133c2b4fb8b8-operator-scripts\") pod \"cinderc151-account-delete-h4k5b\" (UID: \"fc428005-b653-4a6a-a50d-133c2b4fb8b8\") " pod="openstack-kuttl-tests/cinderc151-account-delete-h4k5b" Dec 03 12:32:20 crc kubenswrapper[5052]: I1203 12:32:20.116673 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-v2plr\" (UniqueName: \"kubernetes.io/projected/a2ef0804-75c7-43e3-95fa-ce901a561c9b-kube-api-access-v2plr\") pod \"neutron2174-account-delete-9dnxs\" (UID: \"a2ef0804-75c7-43e3-95fa-ce901a561c9b\") " pod="openstack-kuttl-tests/neutron2174-account-delete-9dnxs" Dec 03 12:32:20 crc kubenswrapper[5052]: I1203 12:32:20.116701 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gcwnf\" (UniqueName: \"kubernetes.io/projected/fc428005-b653-4a6a-a50d-133c2b4fb8b8-kube-api-access-gcwnf\") pod \"cinderc151-account-delete-h4k5b\" (UID: \"fc428005-b653-4a6a-a50d-133c2b4fb8b8\") " pod="openstack-kuttl-tests/cinderc151-account-delete-h4k5b" Dec 03 12:32:20 crc kubenswrapper[5052]: I1203 12:32:20.116786 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/52bf0c8d-03b3-444b-80b6-13d41f5f9fb9-operator-scripts\") pod \"glance5b9b-account-delete-ggk6q\" (UID: \"52bf0c8d-03b3-444b-80b6-13d41f5f9fb9\") " pod="openstack-kuttl-tests/glance5b9b-account-delete-ggk6q" Dec 03 12:32:20 crc kubenswrapper[5052]: I1203 12:32:20.116814 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/a3b48edc-15d0-4c5e-8a33-96bee20dd43e-internal-tls-certs\") pod \"barbican-api-78dc8cddf6-kqfz7\" (UID: \"a3b48edc-15d0-4c5e-8a33-96bee20dd43e\") " pod="openstack-kuttl-tests/barbican-api-78dc8cddf6-kqfz7" Dec 03 12:32:20 crc kubenswrapper[5052]: I1203 12:32:20.116852 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a3b48edc-15d0-4c5e-8a33-96bee20dd43e-combined-ca-bundle\") pod \"barbican-api-78dc8cddf6-kqfz7\" (UID: \"a3b48edc-15d0-4c5e-8a33-96bee20dd43e\") " pod="openstack-kuttl-tests/barbican-api-78dc8cddf6-kqfz7" Dec 03 12:32:20 crc kubenswrapper[5052]: I1203 12:32:20.116888 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-mwzzn\" (UniqueName: \"kubernetes.io/projected/ae9024f5-a2b4-49de-aed3-00cb9c806704-kube-api-access-mwzzn\") pod \"placemente65a-account-delete-s7x52\" (UID: \"ae9024f5-a2b4-49de-aed3-00cb9c806704\") " pod="openstack-kuttl-tests/placemente65a-account-delete-s7x52" Dec 03 12:32:20 crc kubenswrapper[5052]: I1203 12:32:20.116944 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/a2ef0804-75c7-43e3-95fa-ce901a561c9b-operator-scripts\") pod \"neutron2174-account-delete-9dnxs\" (UID: \"a2ef0804-75c7-43e3-95fa-ce901a561c9b\") " pod="openstack-kuttl-tests/neutron2174-account-delete-9dnxs" Dec 03 12:32:20 crc kubenswrapper[5052]: I1203 12:32:20.116994 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/ae9024f5-a2b4-49de-aed3-00cb9c806704-operator-scripts\") pod \"placemente65a-account-delete-s7x52\" (UID: \"ae9024f5-a2b4-49de-aed3-00cb9c806704\") " pod="openstack-kuttl-tests/placemente65a-account-delete-s7x52" Dec 03 12:32:20 crc kubenswrapper[5052]: E1203 12:32:20.117817 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-barbican-public-svc: secret "cert-barbican-public-svc" not found Dec 03 12:32:20 crc kubenswrapper[5052]: E1203 12:32:20.117881 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3b48edc-15d0-4c5e-8a33-96bee20dd43e-public-tls-certs podName:a3b48edc-15d0-4c5e-8a33-96bee20dd43e nodeName:}" failed. No retries permitted until 2025-12-03 12:32:21.117861216 +0000 UTC m=+4331.706102970 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "public-tls-certs" (UniqueName: "kubernetes.io/secret/a3b48edc-15d0-4c5e-8a33-96bee20dd43e-public-tls-certs") pod "barbican-api-78dc8cddf6-kqfz7" (UID: "a3b48edc-15d0-4c5e-8a33-96bee20dd43e") : secret "cert-barbican-public-svc" not found Dec 03 12:32:20 crc kubenswrapper[5052]: E1203 12:32:20.118068 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/combined-ca-bundle: secret "combined-ca-bundle" not found Dec 03 12:32:20 crc kubenswrapper[5052]: E1203 12:32:20.118111 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3b48edc-15d0-4c5e-8a33-96bee20dd43e-combined-ca-bundle podName:a3b48edc-15d0-4c5e-8a33-96bee20dd43e nodeName:}" failed. No retries permitted until 2025-12-03 12:32:21.118099283 +0000 UTC m=+4331.706341047 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "combined-ca-bundle" (UniqueName: "kubernetes.io/secret/a3b48edc-15d0-4c5e-8a33-96bee20dd43e-combined-ca-bundle") pod "barbican-api-78dc8cddf6-kqfz7" (UID: "a3b48edc-15d0-4c5e-8a33-96bee20dd43e") : secret "combined-ca-bundle" not found Dec 03 12:32:20 crc kubenswrapper[5052]: I1203 12:32:20.118744 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/52bf0c8d-03b3-444b-80b6-13d41f5f9fb9-operator-scripts\") pod \"glance5b9b-account-delete-ggk6q\" (UID: \"52bf0c8d-03b3-444b-80b6-13d41f5f9fb9\") " pod="openstack-kuttl-tests/glance5b9b-account-delete-ggk6q" Dec 03 12:32:20 crc kubenswrapper[5052]: E1203 12:32:20.118822 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-barbican-internal-svc: secret "cert-barbican-internal-svc" not found Dec 03 12:32:20 crc kubenswrapper[5052]: E1203 12:32:20.118852 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3b48edc-15d0-4c5e-8a33-96bee20dd43e-internal-tls-certs podName:a3b48edc-15d0-4c5e-8a33-96bee20dd43e nodeName:}" failed. No retries permitted until 2025-12-03 12:32:21.118841864 +0000 UTC m=+4331.707083628 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "internal-tls-certs" (UniqueName: "kubernetes.io/secret/a3b48edc-15d0-4c5e-8a33-96bee20dd43e-internal-tls-certs") pod "barbican-api-78dc8cddf6-kqfz7" (UID: "a3b48edc-15d0-4c5e-8a33-96bee20dd43e") : secret "cert-barbican-internal-svc" not found Dec 03 12:32:20 crc kubenswrapper[5052]: I1203 12:32:20.119306 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/a2ef0804-75c7-43e3-95fa-ce901a561c9b-operator-scripts\") pod \"neutron2174-account-delete-9dnxs\" (UID: \"a2ef0804-75c7-43e3-95fa-ce901a561c9b\") " pod="openstack-kuttl-tests/neutron2174-account-delete-9dnxs" Dec 03 12:32:20 crc kubenswrapper[5052]: I1203 12:32:20.131303 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinderc151-account-delete-h4k5b"] Dec 03 12:32:20 crc kubenswrapper[5052]: I1203 12:32:20.131896 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/ae9024f5-a2b4-49de-aed3-00cb9c806704-operator-scripts\") pod \"placemente65a-account-delete-s7x52\" (UID: \"ae9024f5-a2b4-49de-aed3-00cb9c806704\") " pod="openstack-kuttl-tests/placemente65a-account-delete-s7x52" Dec 03 12:32:20 crc kubenswrapper[5052]: I1203 12:32:20.199631 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-rpq2v\" (UniqueName: \"kubernetes.io/projected/52bf0c8d-03b3-444b-80b6-13d41f5f9fb9-kube-api-access-rpq2v\") pod \"glance5b9b-account-delete-ggk6q\" (UID: \"52bf0c8d-03b3-444b-80b6-13d41f5f9fb9\") " pod="openstack-kuttl-tests/glance5b9b-account-delete-ggk6q" Dec 03 12:32:20 crc kubenswrapper[5052]: I1203 12:32:20.199669 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-v2plr\" (UniqueName: \"kubernetes.io/projected/a2ef0804-75c7-43e3-95fa-ce901a561c9b-kube-api-access-v2plr\") pod \"neutron2174-account-delete-9dnxs\" (UID: \"a2ef0804-75c7-43e3-95fa-ce901a561c9b\") " pod="openstack-kuttl-tests/neutron2174-account-delete-9dnxs" Dec 03 12:32:20 crc kubenswrapper[5052]: I1203 12:32:20.207044 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-sb-0"] Dec 03 12:32:20 crc kubenswrapper[5052]: I1203 12:32:20.207808 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ovsdbserver-sb-0" podUID="3fec3449-9615-4377-a600-69d816ec5416" containerName="openstack-network-exporter" containerID="cri-o://21da3337b09e6b326697aef155af4bcd6a984d400eaed99ee26b1a3a045fba72" gracePeriod=300 Dec 03 12:32:20 crc kubenswrapper[5052]: I1203 12:32:20.214780 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-mwzzn\" (UniqueName: \"kubernetes.io/projected/ae9024f5-a2b4-49de-aed3-00cb9c806704-kube-api-access-mwzzn\") pod \"placemente65a-account-delete-s7x52\" (UID: \"ae9024f5-a2b4-49de-aed3-00cb9c806704\") " pod="openstack-kuttl-tests/placemente65a-account-delete-s7x52" Dec 03 12:32:20 crc kubenswrapper[5052]: I1203 12:32:20.219612 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/fc428005-b653-4a6a-a50d-133c2b4fb8b8-operator-scripts\") pod \"cinderc151-account-delete-h4k5b\" (UID: \"fc428005-b653-4a6a-a50d-133c2b4fb8b8\") " pod="openstack-kuttl-tests/cinderc151-account-delete-h4k5b" Dec 03 12:32:20 crc kubenswrapper[5052]: I1203 12:32:20.219689 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-gcwnf\" (UniqueName: \"kubernetes.io/projected/fc428005-b653-4a6a-a50d-133c2b4fb8b8-kube-api-access-gcwnf\") pod \"cinderc151-account-delete-h4k5b\" (UID: \"fc428005-b653-4a6a-a50d-133c2b4fb8b8\") " pod="openstack-kuttl-tests/cinderc151-account-delete-h4k5b" Dec 03 12:32:20 crc kubenswrapper[5052]: I1203 12:32:20.220938 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/fc428005-b653-4a6a-a50d-133c2b4fb8b8-operator-scripts\") pod \"cinderc151-account-delete-h4k5b\" (UID: \"fc428005-b653-4a6a-a50d-133c2b4fb8b8\") " pod="openstack-kuttl-tests/cinderc151-account-delete-h4k5b" Dec 03 12:32:20 crc kubenswrapper[5052]: I1203 12:32:20.264395 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/placement-db-sync-fspvj"] Dec 03 12:32:20 crc kubenswrapper[5052]: I1203 12:32:20.286772 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-db-sync-5d6m8"] Dec 03 12:32:20 crc kubenswrapper[5052]: E1203 12:32:20.322066 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/rabbitmq-cell1-config-data: configmap "rabbitmq-cell1-config-data" not found Dec 03 12:32:20 crc kubenswrapper[5052]: E1203 12:32:20.322142 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b8613a94-1e2e-477f-8d52-713c70f5282c-config-data podName:b8613a94-1e2e-477f-8d52-713c70f5282c nodeName:}" failed. No retries permitted until 2025-12-03 12:32:21.322127086 +0000 UTC m=+4331.910368850 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "config-data" (UniqueName: "kubernetes.io/configmap/b8613a94-1e2e-477f-8d52-713c70f5282c-config-data") pod "rabbitmq-cell1-server-0" (UID: "b8613a94-1e2e-477f-8d52-713c70f5282c") : configmap "rabbitmq-cell1-config-data" not found Dec 03 12:32:20 crc kubenswrapper[5052]: E1203 12:32:20.322456 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cinder-config-data: secret "cinder-config-data" not found Dec 03 12:32:20 crc kubenswrapper[5052]: E1203 12:32:20.322480 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bb1723c7-4e41-4b0e-b6b8-f8024dafdaee-config-data podName:bb1723c7-4e41-4b0e-b6b8-f8024dafdaee nodeName:}" failed. No retries permitted until 2025-12-03 12:32:21.322472435 +0000 UTC m=+4331.910714199 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "config-data" (UniqueName: "kubernetes.io/secret/bb1723c7-4e41-4b0e-b6b8-f8024dafdaee-config-data") pod "cinder-scheduler-0" (UID: "bb1723c7-4e41-4b0e-b6b8-f8024dafdaee") : secret "cinder-config-data" not found Dec 03 12:32:20 crc kubenswrapper[5052]: E1203 12:32:20.322532 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cinder-scripts: secret "cinder-scripts" not found Dec 03 12:32:20 crc kubenswrapper[5052]: E1203 12:32:20.322551 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bb1723c7-4e41-4b0e-b6b8-f8024dafdaee-scripts podName:bb1723c7-4e41-4b0e-b6b8-f8024dafdaee nodeName:}" failed. No retries permitted until 2025-12-03 12:32:21.322545137 +0000 UTC m=+4331.910786901 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "scripts" (UniqueName: "kubernetes.io/secret/bb1723c7-4e41-4b0e-b6b8-f8024dafdaee-scripts") pod "cinder-scheduler-0" (UID: "bb1723c7-4e41-4b0e-b6b8-f8024dafdaee") : secret "cinder-scripts" not found Dec 03 12:32:20 crc kubenswrapper[5052]: E1203 12:32:20.322579 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/combined-ca-bundle: secret "combined-ca-bundle" not found Dec 03 12:32:20 crc kubenswrapper[5052]: E1203 12:32:20.322594 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bb1723c7-4e41-4b0e-b6b8-f8024dafdaee-combined-ca-bundle podName:bb1723c7-4e41-4b0e-b6b8-f8024dafdaee nodeName:}" failed. No retries permitted until 2025-12-03 12:32:21.322589559 +0000 UTC m=+4331.910831323 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "combined-ca-bundle" (UniqueName: "kubernetes.io/secret/bb1723c7-4e41-4b0e-b6b8-f8024dafdaee-combined-ca-bundle") pod "cinder-scheduler-0" (UID: "bb1723c7-4e41-4b0e-b6b8-f8024dafdaee") : secret "combined-ca-bundle" not found Dec 03 12:32:20 crc kubenswrapper[5052]: I1203 12:32:20.371591 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/placement-db-sync-fspvj"] Dec 03 12:32:20 crc kubenswrapper[5052]: I1203 12:32:20.377685 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placemente65a-account-delete-s7x52" Dec 03 12:32:20 crc kubenswrapper[5052]: I1203 12:32:20.413260 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/glance-db-sync-5d6m8"] Dec 03 12:32:20 crc kubenswrapper[5052]: I1203 12:32:20.441428 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance5b9b-account-delete-ggk6q" Dec 03 12:32:20 crc kubenswrapper[5052]: I1203 12:32:20.445358 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/barbican3d67-account-delete-qrp98"] Dec 03 12:32:20 crc kubenswrapper[5052]: I1203 12:32:20.446865 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican3d67-account-delete-qrp98" Dec 03 12:32:20 crc kubenswrapper[5052]: I1203 12:32:20.478565 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/neutron-db-sync-ggltp"] Dec 03 12:32:20 crc kubenswrapper[5052]: I1203 12:32:20.491972 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/neutron-db-sync-ggltp"] Dec 03 12:32:20 crc kubenswrapper[5052]: I1203 12:32:20.492528 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron2174-account-delete-9dnxs" Dec 03 12:32:20 crc kubenswrapper[5052]: I1203 12:32:20.505358 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder-db-sync-j76tm"] Dec 03 12:32:20 crc kubenswrapper[5052]: I1203 12:32:20.520060 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican3d67-account-delete-qrp98"] Dec 03 12:32:20 crc kubenswrapper[5052]: I1203 12:32:20.543660 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/cinder-db-sync-j76tm"] Dec 03 12:32:20 crc kubenswrapper[5052]: I1203 12:32:20.569640 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/novaapia918-account-delete-mk25j"] Dec 03 12:32:20 crc kubenswrapper[5052]: I1203 12:32:20.571943 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/novaapia918-account-delete-mk25j" Dec 03 12:32:20 crc kubenswrapper[5052]: I1203 12:32:20.589074 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/novaapia918-account-delete-mk25j"] Dec 03 12:32:20 crc kubenswrapper[5052]: I1203 12:32:20.609917 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/novacell046b8-account-delete-nqzlj"] Dec 03 12:32:20 crc kubenswrapper[5052]: I1203 12:32:20.611637 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/novacell046b8-account-delete-nqzlj" Dec 03 12:32:20 crc kubenswrapper[5052]: I1203 12:32:20.625801 5052 generic.go:334] "Generic (PLEG): container finished" podID="a9f65397-a08a-4798-8532-646209c5d090" containerID="ea36b7c04ef3193b039e70bd5df384774204953e99d07610a4a191cab306432f" exitCode=2 Dec 03 12:32:20 crc kubenswrapper[5052]: I1203 12:32:20.625857 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovn-northd-0" event={"ID":"a9f65397-a08a-4798-8532-646209c5d090","Type":"ContainerDied","Data":"ea36b7c04ef3193b039e70bd5df384774204953e99d07610a4a191cab306432f"} Dec 03 12:32:20 crc kubenswrapper[5052]: I1203 12:32:20.633816 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-db-sync-j9cr4"] Dec 03 12:32:20 crc kubenswrapper[5052]: I1203 12:32:20.634029 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/8a93d2af-9f8c-4282-8611-526869559ae2-operator-scripts\") pod \"barbican3d67-account-delete-qrp98\" (UID: \"8a93d2af-9f8c-4282-8611-526869559ae2\") " pod="openstack-kuttl-tests/barbican3d67-account-delete-qrp98" Dec 03 12:32:20 crc kubenswrapper[5052]: I1203 12:32:20.634224 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/cc8c914a-cb83-48ae-97a7-fca144e96f52-combined-ca-bundle\") pod \"barbican-worker-c899dcc5-n6rdv\" (UID: \"cc8c914a-cb83-48ae-97a7-fca144e96f52\") " pod="openstack-kuttl-tests/barbican-worker-c899dcc5-n6rdv" Dec 03 12:32:20 crc kubenswrapper[5052]: I1203 12:32:20.634314 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pqx8w\" (UniqueName: \"kubernetes.io/projected/8a93d2af-9f8c-4282-8611-526869559ae2-kube-api-access-pqx8w\") pod \"barbican3d67-account-delete-qrp98\" (UID: \"8a93d2af-9f8c-4282-8611-526869559ae2\") " pod="openstack-kuttl-tests/barbican3d67-account-delete-qrp98" Dec 03 12:32:20 crc kubenswrapper[5052]: E1203 12:32:20.634654 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/combined-ca-bundle: secret "combined-ca-bundle" not found Dec 03 12:32:20 crc kubenswrapper[5052]: E1203 12:32:20.634705 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cc8c914a-cb83-48ae-97a7-fca144e96f52-combined-ca-bundle podName:cc8c914a-cb83-48ae-97a7-fca144e96f52 nodeName:}" failed. No retries permitted until 2025-12-03 12:32:22.634687016 +0000 UTC m=+4333.222928780 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "combined-ca-bundle" (UniqueName: "kubernetes.io/secret/cc8c914a-cb83-48ae-97a7-fca144e96f52-combined-ca-bundle") pod "barbican-worker-c899dcc5-n6rdv" (UID: "cc8c914a-cb83-48ae-97a7-fca144e96f52") : secret "combined-ca-bundle" not found Dec 03 12:32:20 crc kubenswrapper[5052]: I1203 12:32:20.637767 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_ovsdbserver-nb-0_dccae7c1-009e-4b3b-b66a-ecbf6e53489a/ovsdbserver-nb/0.log" Dec 03 12:32:20 crc kubenswrapper[5052]: I1203 12:32:20.637945 5052 generic.go:334] "Generic (PLEG): container finished" podID="dccae7c1-009e-4b3b-b66a-ecbf6e53489a" containerID="3b528f3a5ecffc67ae3f8e59c8348c99e1798d2e9c1b8ecf277ece85a83c0756" exitCode=2 Dec 03 12:32:20 crc kubenswrapper[5052]: I1203 12:32:20.638023 5052 generic.go:334] "Generic (PLEG): container finished" podID="dccae7c1-009e-4b3b-b66a-ecbf6e53489a" containerID="da4594e855a05fe7faafd60c45e490f353a2622ba132821e7f56859f0a614ca4" exitCode=143 Dec 03 12:32:20 crc kubenswrapper[5052]: I1203 12:32:20.638090 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-nb-0" event={"ID":"dccae7c1-009e-4b3b-b66a-ecbf6e53489a","Type":"ContainerDied","Data":"3b528f3a5ecffc67ae3f8e59c8348c99e1798d2e9c1b8ecf277ece85a83c0756"} Dec 03 12:32:20 crc kubenswrapper[5052]: I1203 12:32:20.638158 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-nb-0" event={"ID":"dccae7c1-009e-4b3b-b66a-ecbf6e53489a","Type":"ContainerDied","Data":"da4594e855a05fe7faafd60c45e490f353a2622ba132821e7f56859f0a614ca4"} Dec 03 12:32:20 crc kubenswrapper[5052]: I1203 12:32:20.647083 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/swift-ring-rebalance-b9ssr"] Dec 03 12:32:20 crc kubenswrapper[5052]: I1203 12:32:20.665755 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/swift-storage-0"] Dec 03 12:32:20 crc kubenswrapper[5052]: I1203 12:32:20.666213 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="9a71429c-9d59-4863-8949-dcb4f79a35e2" containerName="account-server" containerID="cri-o://1637629a53175a4befbb5dad0d15dd5909adc30a1253d4051e99d888b51f7ccb" gracePeriod=30 Dec 03 12:32:20 crc kubenswrapper[5052]: I1203 12:32:20.666723 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="9a71429c-9d59-4863-8949-dcb4f79a35e2" containerName="container-updater" containerID="cri-o://e56722fdb2d4fc10478af03b1d3b9ea29d7757b10975bec562e15e2949b85364" gracePeriod=30 Dec 03 12:32:20 crc kubenswrapper[5052]: I1203 12:32:20.666821 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="9a71429c-9d59-4863-8949-dcb4f79a35e2" containerName="container-server" containerID="cri-o://44b04da4e7f0a025325c3b6375369b7ac347c2ad50331290bfb481efa7ef0736" gracePeriod=30 Dec 03 12:32:20 crc kubenswrapper[5052]: I1203 12:32:20.666943 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="9a71429c-9d59-4863-8949-dcb4f79a35e2" containerName="account-replicator" containerID="cri-o://9920899d34f69573106a85bde57a4bdf8c52b176a8195efa2c785d4039c638ea" gracePeriod=30 Dec 03 12:32:20 crc kubenswrapper[5052]: I1203 12:32:20.666784 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="9a71429c-9d59-4863-8949-dcb4f79a35e2" containerName="container-auditor" containerID="cri-o://844dc1daec944c795b946deb25e04e380651676c9d2e4f9f20f0b937c87cd282" gracePeriod=30 Dec 03 12:32:20 crc kubenswrapper[5052]: I1203 12:32:20.666753 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="9a71429c-9d59-4863-8949-dcb4f79a35e2" containerName="account-reaper" containerID="cri-o://9d3d20ffcb7e4c042c65cb9789046c8a52471258ecf145006245287d12709e44" gracePeriod=30 Dec 03 12:32:20 crc kubenswrapper[5052]: I1203 12:32:20.666799 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="9a71429c-9d59-4863-8949-dcb4f79a35e2" containerName="container-replicator" containerID="cri-o://7fe073f06854f186159ef18897942608baedecbba0d07d0c8013355c37619643" gracePeriod=30 Dec 03 12:32:20 crc kubenswrapper[5052]: I1203 12:32:20.667045 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="9a71429c-9d59-4863-8949-dcb4f79a35e2" containerName="object-updater" containerID="cri-o://59b11d0a23a20a97f6d34709bb5f894ff76b50c46bb80978918201b3b72f674c" gracePeriod=30 Dec 03 12:32:20 crc kubenswrapper[5052]: I1203 12:32:20.667062 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="9a71429c-9d59-4863-8949-dcb4f79a35e2" containerName="account-auditor" containerID="cri-o://be42b27cc3c5b3773e9ff8c15fd7487a0c826144d8760ad2e8dbee3a1515ac77" gracePeriod=30 Dec 03 12:32:20 crc kubenswrapper[5052]: I1203 12:32:20.667069 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="9a71429c-9d59-4863-8949-dcb4f79a35e2" containerName="rsync" containerID="cri-o://bdab1d1a8f6c5c7a6c89c49bf44da7d62eab1cd075defa615ab51c3d50e3986e" gracePeriod=30 Dec 03 12:32:20 crc kubenswrapper[5052]: I1203 12:32:20.667099 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="9a71429c-9d59-4863-8949-dcb4f79a35e2" containerName="object-expirer" containerID="cri-o://a6e47b8819de3aa2c92fe2ccff5f26450ecf089c945a22ec1bb902b9689f2e90" gracePeriod=30 Dec 03 12:32:20 crc kubenswrapper[5052]: I1203 12:32:20.667120 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="9a71429c-9d59-4863-8949-dcb4f79a35e2" containerName="object-replicator" containerID="cri-o://2a0e35f878dfc50c4b4c6c92dc0efc336af0f313d37084f949296ada2fc8a9a9" gracePeriod=30 Dec 03 12:32:20 crc kubenswrapper[5052]: I1203 12:32:20.667136 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="9a71429c-9d59-4863-8949-dcb4f79a35e2" containerName="object-auditor" containerID="cri-o://6d6d0a9e0c44982301515f6c27041d26e825a4e904631b906720c66939d7d046" gracePeriod=30 Dec 03 12:32:20 crc kubenswrapper[5052]: I1203 12:32:20.667172 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="9a71429c-9d59-4863-8949-dcb4f79a35e2" containerName="object-server" containerID="cri-o://1313894d0c0976392a7edf9b30f67f35fdafa36110b15f07dc7f5ffd68038ba8" gracePeriod=30 Dec 03 12:32:20 crc kubenswrapper[5052]: I1203 12:32:20.667336 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="9a71429c-9d59-4863-8949-dcb4f79a35e2" containerName="swift-recon-cron" containerID="cri-o://707deedabc8432d6991a028d80075701f8f2bd14b0998f6a88d8ce1c2703f42b" gracePeriod=30 Dec 03 12:32:20 crc kubenswrapper[5052]: I1203 12:32:20.679716 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/novacell046b8-account-delete-nqzlj"] Dec 03 12:32:20 crc kubenswrapper[5052]: I1203 12:32:20.691136 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/barbican-db-sync-j9cr4"] Dec 03 12:32:20 crc kubenswrapper[5052]: I1203 12:32:20.711523 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/novacell115a6-account-delete-pbwnc"] Dec 03 12:32:20 crc kubenswrapper[5052]: I1203 12:32:20.713854 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/novacell115a6-account-delete-pbwnc" Dec 03 12:32:20 crc kubenswrapper[5052]: I1203 12:32:20.731785 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-gcwnf\" (UniqueName: \"kubernetes.io/projected/fc428005-b653-4a6a-a50d-133c2b4fb8b8-kube-api-access-gcwnf\") pod \"cinderc151-account-delete-h4k5b\" (UID: \"fc428005-b653-4a6a-a50d-133c2b4fb8b8\") " pod="openstack-kuttl-tests/cinderc151-account-delete-h4k5b" Dec 03 12:32:20 crc kubenswrapper[5052]: I1203 12:32:20.737239 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/e6d87a68-5121-4088-a5d8-3f205a787b29-operator-scripts\") pod \"novaapia918-account-delete-mk25j\" (UID: \"e6d87a68-5121-4088-a5d8-3f205a787b29\") " pod="openstack-kuttl-tests/novaapia918-account-delete-mk25j" Dec 03 12:32:20 crc kubenswrapper[5052]: I1203 12:32:20.737601 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/8a93d2af-9f8c-4282-8611-526869559ae2-operator-scripts\") pod \"barbican3d67-account-delete-qrp98\" (UID: \"8a93d2af-9f8c-4282-8611-526869559ae2\") " pod="openstack-kuttl-tests/barbican3d67-account-delete-qrp98" Dec 03 12:32:20 crc kubenswrapper[5052]: I1203 12:32:20.737784 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nwmzn\" (UniqueName: \"kubernetes.io/projected/478e18e8-077e-406e-aa66-19bc1d28df46-kube-api-access-nwmzn\") pod \"novacell046b8-account-delete-nqzlj\" (UID: \"478e18e8-077e-406e-aa66-19bc1d28df46\") " pod="openstack-kuttl-tests/novacell046b8-account-delete-nqzlj" Dec 03 12:32:20 crc kubenswrapper[5052]: I1203 12:32:20.738004 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/478e18e8-077e-406e-aa66-19bc1d28df46-operator-scripts\") pod \"novacell046b8-account-delete-nqzlj\" (UID: \"478e18e8-077e-406e-aa66-19bc1d28df46\") " pod="openstack-kuttl-tests/novacell046b8-account-delete-nqzlj" Dec 03 12:32:20 crc kubenswrapper[5052]: I1203 12:32:20.738420 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/8a93d2af-9f8c-4282-8611-526869559ae2-operator-scripts\") pod \"barbican3d67-account-delete-qrp98\" (UID: \"8a93d2af-9f8c-4282-8611-526869559ae2\") " pod="openstack-kuttl-tests/barbican3d67-account-delete-qrp98" Dec 03 12:32:20 crc kubenswrapper[5052]: I1203 12:32:20.738589 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jjwdb\" (UniqueName: \"kubernetes.io/projected/e6d87a68-5121-4088-a5d8-3f205a787b29-kube-api-access-jjwdb\") pod \"novaapia918-account-delete-mk25j\" (UID: \"e6d87a68-5121-4088-a5d8-3f205a787b29\") " pod="openstack-kuttl-tests/novaapia918-account-delete-mk25j" Dec 03 12:32:20 crc kubenswrapper[5052]: I1203 12:32:20.738919 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/045d578f-92c0-47c6-8356-59c489c57b9a-combined-ca-bundle\") pod \"barbican-keystone-listener-697474f89d-p7lwz\" (UID: \"045d578f-92c0-47c6-8356-59c489c57b9a\") " pod="openstack-kuttl-tests/barbican-keystone-listener-697474f89d-p7lwz" Dec 03 12:32:20 crc kubenswrapper[5052]: I1203 12:32:20.739080 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-pqx8w\" (UniqueName: \"kubernetes.io/projected/8a93d2af-9f8c-4282-8611-526869559ae2-kube-api-access-pqx8w\") pod \"barbican3d67-account-delete-qrp98\" (UID: \"8a93d2af-9f8c-4282-8611-526869559ae2\") " pod="openstack-kuttl-tests/barbican3d67-account-delete-qrp98" Dec 03 12:32:20 crc kubenswrapper[5052]: E1203 12:32:20.739216 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/combined-ca-bundle: secret "combined-ca-bundle" not found Dec 03 12:32:20 crc kubenswrapper[5052]: E1203 12:32:20.739489 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/045d578f-92c0-47c6-8356-59c489c57b9a-combined-ca-bundle podName:045d578f-92c0-47c6-8356-59c489c57b9a nodeName:}" failed. No retries permitted until 2025-12-03 12:32:22.739472797 +0000 UTC m=+4333.327714561 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "combined-ca-bundle" (UniqueName: "kubernetes.io/secret/045d578f-92c0-47c6-8356-59c489c57b9a-combined-ca-bundle") pod "barbican-keystone-listener-697474f89d-p7lwz" (UID: "045d578f-92c0-47c6-8356-59c489c57b9a") : secret "combined-ca-bundle" not found Dec 03 12:32:20 crc kubenswrapper[5052]: E1203 12:32:20.739605 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/rabbitmq-config-data: configmap "rabbitmq-config-data" not found Dec 03 12:32:20 crc kubenswrapper[5052]: E1203 12:32:20.739699 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2c03f99a-98fd-4e86-ad29-fb69e94bd436-config-data podName:2c03f99a-98fd-4e86-ad29-fb69e94bd436 nodeName:}" failed. No retries permitted until 2025-12-03 12:32:22.739680623 +0000 UTC m=+4333.327922427 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "config-data" (UniqueName: "kubernetes.io/configmap/2c03f99a-98fd-4e86-ad29-fb69e94bd436-config-data") pod "rabbitmq-server-0" (UID: "2c03f99a-98fd-4e86-ad29-fb69e94bd436") : configmap "rabbitmq-config-data" not found Dec 03 12:32:20 crc kubenswrapper[5052]: I1203 12:32:20.746934 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/swift-ring-rebalance-b9ssr"] Dec 03 12:32:20 crc kubenswrapper[5052]: I1203 12:32:20.775465 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-pqx8w\" (UniqueName: \"kubernetes.io/projected/8a93d2af-9f8c-4282-8611-526869559ae2-kube-api-access-pqx8w\") pod \"barbican3d67-account-delete-qrp98\" (UID: \"8a93d2af-9f8c-4282-8611-526869559ae2\") " pod="openstack-kuttl-tests/barbican3d67-account-delete-qrp98" Dec 03 12:32:20 crc kubenswrapper[5052]: I1203 12:32:20.778152 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/novacell115a6-account-delete-pbwnc"] Dec 03 12:32:20 crc kubenswrapper[5052]: I1203 12:32:20.791444 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell0-cell-mapping-6nbfg"] Dec 03 12:32:20 crc kubenswrapper[5052]: I1203 12:32:20.802644 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/neutron-58d7cc6794-9z49k"] Dec 03 12:32:20 crc kubenswrapper[5052]: I1203 12:32:20.802951 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/neutron-58d7cc6794-9z49k" podUID="4b3e73a0-1320-427f-bcd7-d8062e315e39" containerName="neutron-api" containerID="cri-o://e1fe0ce8db29a1d1e0ea67a8eaf8d8a7753223ea2fe9336a061719ff4a0463c3" gracePeriod=30 Dec 03 12:32:20 crc kubenswrapper[5052]: I1203 12:32:20.803476 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/neutron-58d7cc6794-9z49k" podUID="4b3e73a0-1320-427f-bcd7-d8062e315e39" containerName="neutron-httpd" containerID="cri-o://82c2ce15a7d7e6c87d6ed20e3eaafec5eb24b16b4abc6508d88b41511f4560a0" gracePeriod=30 Dec 03 12:32:20 crc kubenswrapper[5052]: I1203 12:32:20.826323 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell0-cell-mapping-6nbfg"] Dec 03 12:32:20 crc kubenswrapper[5052]: I1203 12:32:20.841106 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/f12527ba-678e-4ed9-82ff-cec336f607f3-operator-scripts\") pod \"novacell115a6-account-delete-pbwnc\" (UID: \"f12527ba-678e-4ed9-82ff-cec336f607f3\") " pod="openstack-kuttl-tests/novacell115a6-account-delete-pbwnc" Dec 03 12:32:20 crc kubenswrapper[5052]: I1203 12:32:20.841227 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/e6d87a68-5121-4088-a5d8-3f205a787b29-operator-scripts\") pod \"novaapia918-account-delete-mk25j\" (UID: \"e6d87a68-5121-4088-a5d8-3f205a787b29\") " pod="openstack-kuttl-tests/novaapia918-account-delete-mk25j" Dec 03 12:32:20 crc kubenswrapper[5052]: I1203 12:32:20.841309 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wr9nk\" (UniqueName: \"kubernetes.io/projected/f12527ba-678e-4ed9-82ff-cec336f607f3-kube-api-access-wr9nk\") pod \"novacell115a6-account-delete-pbwnc\" (UID: \"f12527ba-678e-4ed9-82ff-cec336f607f3\") " pod="openstack-kuttl-tests/novacell115a6-account-delete-pbwnc" Dec 03 12:32:20 crc kubenswrapper[5052]: I1203 12:32:20.841371 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-nwmzn\" (UniqueName: \"kubernetes.io/projected/478e18e8-077e-406e-aa66-19bc1d28df46-kube-api-access-nwmzn\") pod \"novacell046b8-account-delete-nqzlj\" (UID: \"478e18e8-077e-406e-aa66-19bc1d28df46\") " pod="openstack-kuttl-tests/novacell046b8-account-delete-nqzlj" Dec 03 12:32:20 crc kubenswrapper[5052]: I1203 12:32:20.841459 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/478e18e8-077e-406e-aa66-19bc1d28df46-operator-scripts\") pod \"novacell046b8-account-delete-nqzlj\" (UID: \"478e18e8-077e-406e-aa66-19bc1d28df46\") " pod="openstack-kuttl-tests/novacell046b8-account-delete-nqzlj" Dec 03 12:32:20 crc kubenswrapper[5052]: I1203 12:32:20.841486 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-jjwdb\" (UniqueName: \"kubernetes.io/projected/e6d87a68-5121-4088-a5d8-3f205a787b29-kube-api-access-jjwdb\") pod \"novaapia918-account-delete-mk25j\" (UID: \"e6d87a68-5121-4088-a5d8-3f205a787b29\") " pod="openstack-kuttl-tests/novaapia918-account-delete-mk25j" Dec 03 12:32:20 crc kubenswrapper[5052]: I1203 12:32:20.844445 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/e6d87a68-5121-4088-a5d8-3f205a787b29-operator-scripts\") pod \"novaapia918-account-delete-mk25j\" (UID: \"e6d87a68-5121-4088-a5d8-3f205a787b29\") " pod="openstack-kuttl-tests/novaapia918-account-delete-mk25j" Dec 03 12:32:20 crc kubenswrapper[5052]: I1203 12:32:20.844685 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/478e18e8-077e-406e-aa66-19bc1d28df46-operator-scripts\") pod \"novacell046b8-account-delete-nqzlj\" (UID: \"478e18e8-077e-406e-aa66-19bc1d28df46\") " pod="openstack-kuttl-tests/novacell046b8-account-delete-nqzlj" Dec 03 12:32:20 crc kubenswrapper[5052]: I1203 12:32:20.858444 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:32:20 crc kubenswrapper[5052]: I1203 12:32:20.858725 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-internal-api-0" podUID="5737900d-97ae-4dcb-a65c-b40fe30cdc50" containerName="glance-log" containerID="cri-o://a973285d5356bd9dc51bc03e5f6f166c4ac926ebd9edb7f87c4de4de53693880" gracePeriod=30 Dec 03 12:32:20 crc kubenswrapper[5052]: I1203 12:32:20.859149 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-internal-api-0" podUID="5737900d-97ae-4dcb-a65c-b40fe30cdc50" containerName="glance-httpd" containerID="cri-o://1840b34d3edddd62d0d38183e87fc8f3976fc72f58bdedd488ca0702b16fd6e7" gracePeriod=30 Dec 03 12:32:20 crc kubenswrapper[5052]: I1203 12:32:20.860378 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-jjwdb\" (UniqueName: \"kubernetes.io/projected/e6d87a68-5121-4088-a5d8-3f205a787b29-kube-api-access-jjwdb\") pod \"novaapia918-account-delete-mk25j\" (UID: \"e6d87a68-5121-4088-a5d8-3f205a787b29\") " pod="openstack-kuttl-tests/novaapia918-account-delete-mk25j" Dec 03 12:32:20 crc kubenswrapper[5052]: I1203 12:32:20.872097 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell1-cell-mapping-7v2kh"] Dec 03 12:32:20 crc kubenswrapper[5052]: I1203 12:32:20.874290 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-nwmzn\" (UniqueName: \"kubernetes.io/projected/478e18e8-077e-406e-aa66-19bc1d28df46-kube-api-access-nwmzn\") pod \"novacell046b8-account-delete-nqzlj\" (UID: \"478e18e8-077e-406e-aa66-19bc1d28df46\") " pod="openstack-kuttl-tests/novacell046b8-account-delete-nqzlj" Dec 03 12:32:20 crc kubenswrapper[5052]: I1203 12:32:20.899172 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/placement-65444bfc86-th5pw"] Dec 03 12:32:20 crc kubenswrapper[5052]: I1203 12:32:20.900199 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/placement-65444bfc86-th5pw" podUID="cfd8af50-a644-4b24-8483-dd344f2218e4" containerName="placement-log" containerID="cri-o://dc83501080d7c2386bc4543b9befadc35e0e843ede852c450da98e997424e43e" gracePeriod=30 Dec 03 12:32:20 crc kubenswrapper[5052]: I1203 12:32:20.900668 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/placement-65444bfc86-th5pw" podUID="cfd8af50-a644-4b24-8483-dd344f2218e4" containerName="placement-api" containerID="cri-o://fefeb53cf7ddfecbec19b54d8ffa06426f1efa88fb8452a1c522b836c48d8085" gracePeriod=30 Dec 03 12:32:20 crc kubenswrapper[5052]: I1203 12:32:20.941694 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ovsdbserver-sb-0" podUID="3fec3449-9615-4377-a600-69d816ec5416" containerName="ovsdbserver-sb" containerID="cri-o://19c04c4116df77c2accd0a6c361be38550f970c0c0f8032ebf2cd11805800978" gracePeriod=300 Dec 03 12:32:20 crc kubenswrapper[5052]: I1203 12:32:20.952181 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/f12527ba-678e-4ed9-82ff-cec336f607f3-operator-scripts\") pod \"novacell115a6-account-delete-pbwnc\" (UID: \"f12527ba-678e-4ed9-82ff-cec336f607f3\") " pod="openstack-kuttl-tests/novacell115a6-account-delete-pbwnc" Dec 03 12:32:20 crc kubenswrapper[5052]: I1203 12:32:20.952394 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-wr9nk\" (UniqueName: \"kubernetes.io/projected/f12527ba-678e-4ed9-82ff-cec336f607f3-kube-api-access-wr9nk\") pod \"novacell115a6-account-delete-pbwnc\" (UID: \"f12527ba-678e-4ed9-82ff-cec336f607f3\") " pod="openstack-kuttl-tests/novacell115a6-account-delete-pbwnc" Dec 03 12:32:20 crc kubenswrapper[5052]: E1203 12:32:20.952763 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-cell1-scripts: configmap "openstack-cell1-scripts" not found Dec 03 12:32:20 crc kubenswrapper[5052]: E1203 12:32:20.952832 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f12527ba-678e-4ed9-82ff-cec336f607f3-operator-scripts podName:f12527ba-678e-4ed9-82ff-cec336f607f3 nodeName:}" failed. No retries permitted until 2025-12-03 12:32:21.452815497 +0000 UTC m=+4332.041057261 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/f12527ba-678e-4ed9-82ff-cec336f607f3-operator-scripts") pod "novacell115a6-account-delete-pbwnc" (UID: "f12527ba-678e-4ed9-82ff-cec336f607f3") : configmap "openstack-cell1-scripts" not found Dec 03 12:32:20 crc kubenswrapper[5052]: E1203 12:32:20.964933 5052 projected.go:194] Error preparing data for projected volume kube-api-access-wr9nk for pod openstack-kuttl-tests/novacell115a6-account-delete-pbwnc: failed to fetch token: serviceaccounts "galera-openstack-cell1" not found Dec 03 12:32:20 crc kubenswrapper[5052]: E1203 12:32:20.965008 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f12527ba-678e-4ed9-82ff-cec336f607f3-kube-api-access-wr9nk podName:f12527ba-678e-4ed9-82ff-cec336f607f3 nodeName:}" failed. No retries permitted until 2025-12-03 12:32:21.464987314 +0000 UTC m=+4332.053229078 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-wr9nk" (UniqueName: "kubernetes.io/projected/f12527ba-678e-4ed9-82ff-cec336f607f3-kube-api-access-wr9nk") pod "novacell115a6-account-delete-pbwnc" (UID: "f12527ba-678e-4ed9-82ff-cec336f607f3") : failed to fetch token: serviceaccounts "galera-openstack-cell1" not found Dec 03 12:32:20 crc kubenswrapper[5052]: I1203 12:32:20.998638 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell1-cell-mapping-7v2kh"] Dec 03 12:32:21 crc kubenswrapper[5052]: I1203 12:32:21.007476 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-hjfg4"] Dec 03 12:32:21 crc kubenswrapper[5052]: I1203 12:32:21.009941 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-hjfg4" Dec 03 12:32:21 crc kubenswrapper[5052]: I1203 12:32:21.029323 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinderc151-account-delete-h4k5b" Dec 03 12:32:21 crc kubenswrapper[5052]: I1203 12:32:21.043905 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-hjfg4"] Dec 03 12:32:21 crc kubenswrapper[5052]: I1203 12:32:21.056249 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/00bbba79-7529-4309-af53-b464acf28194-config\") pod \"dnsmasq-dnsmasq-84b9f45d47-hjfg4\" (UID: \"00bbba79-7529-4309-af53-b464acf28194\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-hjfg4" Dec 03 12:32:21 crc kubenswrapper[5052]: I1203 12:32:21.057358 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/00bbba79-7529-4309-af53-b464acf28194-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-84b9f45d47-hjfg4\" (UID: \"00bbba79-7529-4309-af53-b464acf28194\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-hjfg4" Dec 03 12:32:21 crc kubenswrapper[5052]: I1203 12:32:21.058100 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8pm8q\" (UniqueName: \"kubernetes.io/projected/00bbba79-7529-4309-af53-b464acf28194-kube-api-access-8pm8q\") pod \"dnsmasq-dnsmasq-84b9f45d47-hjfg4\" (UID: \"00bbba79-7529-4309-af53-b464acf28194\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-hjfg4" Dec 03 12:32:21 crc kubenswrapper[5052]: I1203 12:32:21.065194 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/rabbitmq-server-0"] Dec 03 12:32:21 crc kubenswrapper[5052]: I1203 12:32:21.122673 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 12:32:21 crc kubenswrapper[5052]: I1203 12:32:21.123669 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/cinder-scheduler-0" podUID="bb1723c7-4e41-4b0e-b6b8-f8024dafdaee" containerName="probe" containerID="cri-o://ab269ac11d99aed29d8c53741cb5bdd97ed1fe052bc19e5e9c3e0f4152eaeed4" gracePeriod=30 Dec 03 12:32:21 crc kubenswrapper[5052]: I1203 12:32:21.123628 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/cinder-scheduler-0" podUID="bb1723c7-4e41-4b0e-b6b8-f8024dafdaee" containerName="cinder-scheduler" containerID="cri-o://94df1fd3c0977441ba1cf70763e96ac140d6a232f2c0d6cfe09fb6aa6ae75dc3" gracePeriod=30 Dec 03 12:32:21 crc kubenswrapper[5052]: I1203 12:32:21.145395 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:32:21 crc kubenswrapper[5052]: I1203 12:32:21.145677 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-external-api-0" podUID="a65428bc-9326-4ace-a985-9f796b951304" containerName="glance-log" containerID="cri-o://caa8dd02dbab945ca9d553d926ff4896db88a3b315e5384ca071cd388a2f2766" gracePeriod=30 Dec 03 12:32:21 crc kubenswrapper[5052]: I1203 12:32:21.146200 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-external-api-0" podUID="a65428bc-9326-4ace-a985-9f796b951304" containerName="glance-httpd" containerID="cri-o://ddde80cbe1c1205957f18817d4db06fabccc08242a25c2d74bcd5a4c61d8bd5d" gracePeriod=30 Dec 03 12:32:21 crc kubenswrapper[5052]: I1203 12:32:21.149074 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/rabbitmq-server-0" podUID="2c03f99a-98fd-4e86-ad29-fb69e94bd436" containerName="rabbitmq" containerID="cri-o://33c00a517cac73b55f22b39dd9800951f3eddc4eb90411d1bc3e8a5477c02571" gracePeriod=604800 Dec 03 12:32:21 crc kubenswrapper[5052]: I1203 12:32:21.161274 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-8pm8q\" (UniqueName: \"kubernetes.io/projected/00bbba79-7529-4309-af53-b464acf28194-kube-api-access-8pm8q\") pod \"dnsmasq-dnsmasq-84b9f45d47-hjfg4\" (UID: \"00bbba79-7529-4309-af53-b464acf28194\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-hjfg4" Dec 03 12:32:21 crc kubenswrapper[5052]: I1203 12:32:21.161410 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/a3b48edc-15d0-4c5e-8a33-96bee20dd43e-public-tls-certs\") pod \"barbican-api-78dc8cddf6-kqfz7\" (UID: \"a3b48edc-15d0-4c5e-8a33-96bee20dd43e\") " pod="openstack-kuttl-tests/barbican-api-78dc8cddf6-kqfz7" Dec 03 12:32:21 crc kubenswrapper[5052]: I1203 12:32:21.161557 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/00bbba79-7529-4309-af53-b464acf28194-config\") pod \"dnsmasq-dnsmasq-84b9f45d47-hjfg4\" (UID: \"00bbba79-7529-4309-af53-b464acf28194\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-hjfg4" Dec 03 12:32:21 crc kubenswrapper[5052]: I1203 12:32:21.161689 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/00bbba79-7529-4309-af53-b464acf28194-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-84b9f45d47-hjfg4\" (UID: \"00bbba79-7529-4309-af53-b464acf28194\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-hjfg4" Dec 03 12:32:21 crc kubenswrapper[5052]: I1203 12:32:21.161716 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/a3b48edc-15d0-4c5e-8a33-96bee20dd43e-internal-tls-certs\") pod \"barbican-api-78dc8cddf6-kqfz7\" (UID: \"a3b48edc-15d0-4c5e-8a33-96bee20dd43e\") " pod="openstack-kuttl-tests/barbican-api-78dc8cddf6-kqfz7" Dec 03 12:32:21 crc kubenswrapper[5052]: I1203 12:32:21.161766 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a3b48edc-15d0-4c5e-8a33-96bee20dd43e-combined-ca-bundle\") pod \"barbican-api-78dc8cddf6-kqfz7\" (UID: \"a3b48edc-15d0-4c5e-8a33-96bee20dd43e\") " pod="openstack-kuttl-tests/barbican-api-78dc8cddf6-kqfz7" Dec 03 12:32:21 crc kubenswrapper[5052]: I1203 12:32:21.163821 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/00bbba79-7529-4309-af53-b464acf28194-config\") pod \"dnsmasq-dnsmasq-84b9f45d47-hjfg4\" (UID: \"00bbba79-7529-4309-af53-b464acf28194\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-hjfg4" Dec 03 12:32:21 crc kubenswrapper[5052]: I1203 12:32:21.163850 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/00bbba79-7529-4309-af53-b464acf28194-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-84b9f45d47-hjfg4\" (UID: \"00bbba79-7529-4309-af53-b464acf28194\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-hjfg4" Dec 03 12:32:21 crc kubenswrapper[5052]: E1203 12:32:21.171199 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-barbican-public-svc: secret "cert-barbican-public-svc" not found Dec 03 12:32:21 crc kubenswrapper[5052]: E1203 12:32:21.171253 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3b48edc-15d0-4c5e-8a33-96bee20dd43e-public-tls-certs podName:a3b48edc-15d0-4c5e-8a33-96bee20dd43e nodeName:}" failed. No retries permitted until 2025-12-03 12:32:23.171236511 +0000 UTC m=+4333.759478275 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "public-tls-certs" (UniqueName: "kubernetes.io/secret/a3b48edc-15d0-4c5e-8a33-96bee20dd43e-public-tls-certs") pod "barbican-api-78dc8cddf6-kqfz7" (UID: "a3b48edc-15d0-4c5e-8a33-96bee20dd43e") : secret "cert-barbican-public-svc" not found Dec 03 12:32:21 crc kubenswrapper[5052]: E1203 12:32:21.171292 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-barbican-internal-svc: secret "cert-barbican-internal-svc" not found Dec 03 12:32:21 crc kubenswrapper[5052]: E1203 12:32:21.171316 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3b48edc-15d0-4c5e-8a33-96bee20dd43e-internal-tls-certs podName:a3b48edc-15d0-4c5e-8a33-96bee20dd43e nodeName:}" failed. No retries permitted until 2025-12-03 12:32:23.171310083 +0000 UTC m=+4333.759551847 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "internal-tls-certs" (UniqueName: "kubernetes.io/secret/a3b48edc-15d0-4c5e-8a33-96bee20dd43e-internal-tls-certs") pod "barbican-api-78dc8cddf6-kqfz7" (UID: "a3b48edc-15d0-4c5e-8a33-96bee20dd43e") : secret "cert-barbican-internal-svc" not found Dec 03 12:32:21 crc kubenswrapper[5052]: E1203 12:32:21.171345 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/combined-ca-bundle: secret "combined-ca-bundle" not found Dec 03 12:32:21 crc kubenswrapper[5052]: E1203 12:32:21.171363 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3b48edc-15d0-4c5e-8a33-96bee20dd43e-combined-ca-bundle podName:a3b48edc-15d0-4c5e-8a33-96bee20dd43e nodeName:}" failed. No retries permitted until 2025-12-03 12:32:23.171357054 +0000 UTC m=+4333.759598818 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "combined-ca-bundle" (UniqueName: "kubernetes.io/secret/a3b48edc-15d0-4c5e-8a33-96bee20dd43e-combined-ca-bundle") pod "barbican-api-78dc8cddf6-kqfz7" (UID: "a3b48edc-15d0-4c5e-8a33-96bee20dd43e") : secret "combined-ca-bundle" not found Dec 03 12:32:21 crc kubenswrapper[5052]: I1203 12:32:21.211416 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-8pm8q\" (UniqueName: \"kubernetes.io/projected/00bbba79-7529-4309-af53-b464acf28194-kube-api-access-8pm8q\") pod \"dnsmasq-dnsmasq-84b9f45d47-hjfg4\" (UID: \"00bbba79-7529-4309-af53-b464acf28194\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-hjfg4" Dec 03 12:32:21 crc kubenswrapper[5052]: I1203 12:32:21.248812 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 12:32:21 crc kubenswrapper[5052]: I1203 12:32:21.249203 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/cinder-api-0" podUID="6e0f19c7-34ae-4d8e-9a19-755ae1c20c1c" containerName="cinder-api-log" containerID="cri-o://fa8e6470aab88dd31d2b44427546df1694d629e5bea31eca80eadf84f064db9d" gracePeriod=30 Dec 03 12:32:21 crc kubenswrapper[5052]: I1203 12:32:21.249274 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/cinder-api-0" podUID="6e0f19c7-34ae-4d8e-9a19-755ae1c20c1c" containerName="cinder-api" containerID="cri-o://e387dc9d87ea03a56050ef8cbe08349409d39a0cdecb55b0fe4e091ce7e2b2f4" gracePeriod=30 Dec 03 12:32:21 crc kubenswrapper[5052]: I1203 12:32:21.335808 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/swift-proxy-55bdbb59c4-c4cjv"] Dec 03 12:32:21 crc kubenswrapper[5052]: I1203 12:32:21.336270 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-proxy-55bdbb59c4-c4cjv" podUID="16cac95c-d1d2-4542-bb8e-4bdc1923c684" containerName="proxy-httpd" containerID="cri-o://e2917e8431b0de8f360760abc00d1838625c47e53148c72eb9f76598a47629bd" gracePeriod=30 Dec 03 12:32:21 crc kubenswrapper[5052]: I1203 12:32:21.336624 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-proxy-55bdbb59c4-c4cjv" podUID="16cac95c-d1d2-4542-bb8e-4bdc1923c684" containerName="proxy-server" containerID="cri-o://3fd08a7accc11bddbab26e95f99cd86563dcbdee87c9faa5f4e155b8982c03de" gracePeriod=30 Dec 03 12:32:21 crc kubenswrapper[5052]: I1203 12:32:21.342110 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican3d67-account-delete-qrp98" Dec 03 12:32:21 crc kubenswrapper[5052]: I1203 12:32:21.380349 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/novaapia918-account-delete-mk25j" Dec 03 12:32:21 crc kubenswrapper[5052]: E1203 12:32:21.387132 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/rabbitmq-cell1-config-data: configmap "rabbitmq-cell1-config-data" not found Dec 03 12:32:21 crc kubenswrapper[5052]: E1203 12:32:21.387189 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b8613a94-1e2e-477f-8d52-713c70f5282c-config-data podName:b8613a94-1e2e-477f-8d52-713c70f5282c nodeName:}" failed. No retries permitted until 2025-12-03 12:32:23.387175744 +0000 UTC m=+4333.975417508 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "config-data" (UniqueName: "kubernetes.io/configmap/b8613a94-1e2e-477f-8d52-713c70f5282c-config-data") pod "rabbitmq-cell1-server-0" (UID: "b8613a94-1e2e-477f-8d52-713c70f5282c") : configmap "rabbitmq-cell1-config-data" not found Dec 03 12:32:21 crc kubenswrapper[5052]: E1203 12:32:21.389803 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cinder-config-data: secret "cinder-config-data" not found Dec 03 12:32:21 crc kubenswrapper[5052]: E1203 12:32:21.389859 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bb1723c7-4e41-4b0e-b6b8-f8024dafdaee-config-data podName:bb1723c7-4e41-4b0e-b6b8-f8024dafdaee nodeName:}" failed. No retries permitted until 2025-12-03 12:32:23.389842191 +0000 UTC m=+4333.978083955 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "config-data" (UniqueName: "kubernetes.io/secret/bb1723c7-4e41-4b0e-b6b8-f8024dafdaee-config-data") pod "cinder-scheduler-0" (UID: "bb1723c7-4e41-4b0e-b6b8-f8024dafdaee") : secret "cinder-config-data" not found Dec 03 12:32:21 crc kubenswrapper[5052]: E1203 12:32:21.389897 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cinder-scripts: secret "cinder-scripts" not found Dec 03 12:32:21 crc kubenswrapper[5052]: E1203 12:32:21.389923 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bb1723c7-4e41-4b0e-b6b8-f8024dafdaee-scripts podName:bb1723c7-4e41-4b0e-b6b8-f8024dafdaee nodeName:}" failed. No retries permitted until 2025-12-03 12:32:23.389916403 +0000 UTC m=+4333.978158167 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "scripts" (UniqueName: "kubernetes.io/secret/bb1723c7-4e41-4b0e-b6b8-f8024dafdaee-scripts") pod "cinder-scheduler-0" (UID: "bb1723c7-4e41-4b0e-b6b8-f8024dafdaee") : secret "cinder-scripts" not found Dec 03 12:32:21 crc kubenswrapper[5052]: E1203 12:32:21.389956 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/combined-ca-bundle: secret "combined-ca-bundle" not found Dec 03 12:32:21 crc kubenswrapper[5052]: E1203 12:32:21.389975 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bb1723c7-4e41-4b0e-b6b8-f8024dafdaee-combined-ca-bundle podName:bb1723c7-4e41-4b0e-b6b8-f8024dafdaee nodeName:}" failed. No retries permitted until 2025-12-03 12:32:23.389970324 +0000 UTC m=+4333.978212088 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "combined-ca-bundle" (UniqueName: "kubernetes.io/secret/bb1723c7-4e41-4b0e-b6b8-f8024dafdaee-combined-ca-bundle") pod "cinder-scheduler-0" (UID: "bb1723c7-4e41-4b0e-b6b8-f8024dafdaee") : secret "combined-ca-bundle" not found Dec 03 12:32:21 crc kubenswrapper[5052]: I1203 12:32:21.393430 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:32:21 crc kubenswrapper[5052]: I1203 12:32:21.393757 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="21690a66-4bcf-4865-91c7-cfe1cdf32a61" containerName="ceilometer-central-agent" containerID="cri-o://3c2c53aa7eb07d8c8a4be850fb78e1b62b32812e4e50fd988ba119d0f7c3a22a" gracePeriod=30 Dec 03 12:32:21 crc kubenswrapper[5052]: I1203 12:32:21.394171 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="21690a66-4bcf-4865-91c7-cfe1cdf32a61" containerName="proxy-httpd" containerID="cri-o://fb673c0dc2e5ca2f8ef44bb06008daf9443cc163a93859ac48d11b2d619a8092" gracePeriod=30 Dec 03 12:32:21 crc kubenswrapper[5052]: I1203 12:32:21.394214 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="21690a66-4bcf-4865-91c7-cfe1cdf32a61" containerName="sg-core" containerID="cri-o://4588847c60a7e1e54c1d68283ca53c647caedbb74b422846f258c273285a8d29" gracePeriod=30 Dec 03 12:32:21 crc kubenswrapper[5052]: I1203 12:32:21.394246 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="21690a66-4bcf-4865-91c7-cfe1cdf32a61" containerName="ceilometer-notification-agent" containerID="cri-o://6cd8e96f6ff38ec31288f641a86e7f8fed72b9e87063c8cee94ed8bad6b902cc" gracePeriod=30 Dec 03 12:32:21 crc kubenswrapper[5052]: I1203 12:32:21.418430 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:32:21 crc kubenswrapper[5052]: I1203 12:32:21.418785 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-metadata-0" podUID="39117149-c15d-4753-9324-8a42abe5b007" containerName="nova-metadata-log" containerID="cri-o://53093380d15a12407af002e8218c329f263122e535b3f7a98513361ca8b1c783" gracePeriod=30 Dec 03 12:32:21 crc kubenswrapper[5052]: I1203 12:32:21.419289 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-metadata-0" podUID="39117149-c15d-4753-9324-8a42abe5b007" containerName="nova-metadata-metadata" containerID="cri-o://5b99120371dd1ea96363322c8c525108a18d3c48f5844ec07a58fedeb98a76df" gracePeriod=30 Dec 03 12:32:21 crc kubenswrapper[5052]: I1203 12:32:21.489408 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-wr9nk\" (UniqueName: \"kubernetes.io/projected/f12527ba-678e-4ed9-82ff-cec336f607f3-kube-api-access-wr9nk\") pod \"novacell115a6-account-delete-pbwnc\" (UID: \"f12527ba-678e-4ed9-82ff-cec336f607f3\") " pod="openstack-kuttl-tests/novacell115a6-account-delete-pbwnc" Dec 03 12:32:21 crc kubenswrapper[5052]: I1203 12:32:21.489604 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/f12527ba-678e-4ed9-82ff-cec336f607f3-operator-scripts\") pod \"novacell115a6-account-delete-pbwnc\" (UID: \"f12527ba-678e-4ed9-82ff-cec336f607f3\") " pod="openstack-kuttl-tests/novacell115a6-account-delete-pbwnc" Dec 03 12:32:21 crc kubenswrapper[5052]: E1203 12:32:21.489710 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-cell1-scripts: configmap "openstack-cell1-scripts" not found Dec 03 12:32:21 crc kubenswrapper[5052]: E1203 12:32:21.489757 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f12527ba-678e-4ed9-82ff-cec336f607f3-operator-scripts podName:f12527ba-678e-4ed9-82ff-cec336f607f3 nodeName:}" failed. No retries permitted until 2025-12-03 12:32:22.489744432 +0000 UTC m=+4333.077986196 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/f12527ba-678e-4ed9-82ff-cec336f607f3-operator-scripts") pod "novacell115a6-account-delete-pbwnc" (UID: "f12527ba-678e-4ed9-82ff-cec336f607f3") : configmap "openstack-cell1-scripts" not found Dec 03 12:32:21 crc kubenswrapper[5052]: E1203 12:32:21.494444 5052 projected.go:194] Error preparing data for projected volume kube-api-access-wr9nk for pod openstack-kuttl-tests/novacell115a6-account-delete-pbwnc: failed to fetch token: serviceaccounts "galera-openstack-cell1" not found Dec 03 12:32:21 crc kubenswrapper[5052]: E1203 12:32:21.494503 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f12527ba-678e-4ed9-82ff-cec336f607f3-kube-api-access-wr9nk podName:f12527ba-678e-4ed9-82ff-cec336f607f3 nodeName:}" failed. No retries permitted until 2025-12-03 12:32:22.494486607 +0000 UTC m=+4333.082728371 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-wr9nk" (UniqueName: "kubernetes.io/projected/f12527ba-678e-4ed9-82ff-cec336f607f3-kube-api-access-wr9nk") pod "novacell115a6-account-delete-pbwnc" (UID: "f12527ba-678e-4ed9-82ff-cec336f607f3") : failed to fetch token: serviceaccounts "galera-openstack-cell1" not found Dec 03 12:32:21 crc kubenswrapper[5052]: I1203 12:32:21.504554 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:32:21 crc kubenswrapper[5052]: I1203 12:32:21.507077 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-api-0" podUID="7cbe1c2c-bc04-4232-9901-bc9d134df3d6" containerName="nova-api-log" containerID="cri-o://1d2a9d60a97780af3d81ede77692ed338720a0c0a5b32ac9f95fa608d8ace7d1" gracePeriod=30 Dec 03 12:32:21 crc kubenswrapper[5052]: I1203 12:32:21.507766 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-api-0" podUID="7cbe1c2c-bc04-4232-9901-bc9d134df3d6" containerName="nova-api-api" containerID="cri-o://3c03aa60a5d69f404b768bb6cc6569fcdc7b56fcc98ed0721fb684712e773b3e" gracePeriod=30 Dec 03 12:32:21 crc kubenswrapper[5052]: I1203 12:32:21.540798 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/openstack-cell1-galera-0"] Dec 03 12:32:21 crc kubenswrapper[5052]: I1203 12:32:21.618630 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-api-78dc8cddf6-kqfz7"] Dec 03 12:32:21 crc kubenswrapper[5052]: I1203 12:32:21.630898 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-api-68fc8f9776-n9gml"] Dec 03 12:32:21 crc kubenswrapper[5052]: I1203 12:32:21.631294 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/barbican-api-68fc8f9776-n9gml" podUID="82f142ab-5be9-4727-b3c5-102d6f2401fb" containerName="barbican-api-log" containerID="cri-o://ca30ccff84d103c471a54af8538ca8e1d53df3754e5f8a3e9c4ab0869c3a23d6" gracePeriod=30 Dec 03 12:32:21 crc kubenswrapper[5052]: I1203 12:32:21.631891 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/barbican-api-68fc8f9776-n9gml" podUID="82f142ab-5be9-4727-b3c5-102d6f2401fb" containerName="barbican-api" containerID="cri-o://74fe4216c2c90357dad59af2f1e20e908e8583d49e0f2a8d15cd86ca1fc755d6" gracePeriod=30 Dec 03 12:32:21 crc kubenswrapper[5052]: E1203 12:32:21.664365 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[combined-ca-bundle internal-tls-certs public-tls-certs], unattached volumes=[], failed to process volumes=[]: context canceled" pod="openstack-kuttl-tests/barbican-api-78dc8cddf6-kqfz7" podUID="a3b48edc-15d0-4c5e-8a33-96bee20dd43e" Dec 03 12:32:21 crc kubenswrapper[5052]: I1203 12:32:21.674730 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-keystone-listener-cd5b58cfd-jf8vt"] Dec 03 12:32:21 crc kubenswrapper[5052]: I1203 12:32:21.680182 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/barbican-keystone-listener-cd5b58cfd-jf8vt" podUID="019689d4-d40b-4e9a-a028-e427f1aa9d9b" containerName="barbican-keystone-listener-log" containerID="cri-o://30570e4c9e23e90b86c1701d30a06063dcb7b48a31496a770e386fcaa5d3ac42" gracePeriod=30 Dec 03 12:32:21 crc kubenswrapper[5052]: I1203 12:32:21.681324 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/barbican-keystone-listener-cd5b58cfd-jf8vt" podUID="019689d4-d40b-4e9a-a028-e427f1aa9d9b" containerName="barbican-keystone-listener" containerID="cri-o://a988e23bbce42676cbdcb4abb64e1c8911c6df2c4d18faf34f5c6450d0879857" gracePeriod=30 Dec 03 12:32:21 crc kubenswrapper[5052]: I1203 12:32:21.743873 5052 generic.go:334] "Generic (PLEG): container finished" podID="5737900d-97ae-4dcb-a65c-b40fe30cdc50" containerID="a973285d5356bd9dc51bc03e5f6f166c4ac926ebd9edb7f87c4de4de53693880" exitCode=143 Dec 03 12:32:21 crc kubenswrapper[5052]: I1203 12:32:21.745551 5052 generic.go:334] "Generic (PLEG): container finished" podID="2ed8a31a-9033-4601-84bf-9eaf57f1f155" containerID="220531c8139609a0b36f479504e8821b1f41b4615b00ee05432f9560ade3acb1" exitCode=137 Dec 03 12:32:21 crc kubenswrapper[5052]: I1203 12:32:21.777880 5052 generic.go:334] "Generic (PLEG): container finished" podID="6e0f19c7-34ae-4d8e-9a19-755ae1c20c1c" containerID="fa8e6470aab88dd31d2b44427546df1694d629e5bea31eca80eadf84f064db9d" exitCode=143 Dec 03 12:32:21 crc kubenswrapper[5052]: I1203 12:32:21.877349 5052 generic.go:334] "Generic (PLEG): container finished" podID="21690a66-4bcf-4865-91c7-cfe1cdf32a61" containerID="4588847c60a7e1e54c1d68283ca53c647caedbb74b422846f258c273285a8d29" exitCode=2 Dec 03 12:32:21 crc kubenswrapper[5052]: I1203 12:32:21.908127 5052 generic.go:334] "Generic (PLEG): container finished" podID="39117149-c15d-4753-9324-8a42abe5b007" containerID="53093380d15a12407af002e8218c329f263122e535b3f7a98513361ca8b1c783" exitCode=143 Dec 03 12:32:21 crc kubenswrapper[5052]: I1203 12:32:21.932412 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="46e92959-0fd2-4e2b-8369-35cf14716af3" path="/var/lib/kubelet/pods/46e92959-0fd2-4e2b-8369-35cf14716af3/volumes" Dec 03 12:32:21 crc kubenswrapper[5052]: I1203 12:32:21.936615 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="516cc68a-d42b-40c8-8ca7-a093f47f10d5" path="/var/lib/kubelet/pods/516cc68a-d42b-40c8-8ca7-a093f47f10d5/volumes" Dec 03 12:32:21 crc kubenswrapper[5052]: I1203 12:32:21.937815 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="51d1c5f3-0622-4c29-a19a-f1ecd293654e" path="/var/lib/kubelet/pods/51d1c5f3-0622-4c29-a19a-f1ecd293654e/volumes" Dec 03 12:32:21 crc kubenswrapper[5052]: I1203 12:32:21.938801 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6af52f56-d687-43d2-a8da-1780e540ec92" path="/var/lib/kubelet/pods/6af52f56-d687-43d2-a8da-1780e540ec92/volumes" Dec 03 12:32:21 crc kubenswrapper[5052]: I1203 12:32:21.939928 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="778e86ee-3618-41fa-83c3-eb28609f50e2" path="/var/lib/kubelet/pods/778e86ee-3618-41fa-83c3-eb28609f50e2/volumes" Dec 03 12:32:21 crc kubenswrapper[5052]: I1203 12:32:21.941352 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b5a18429-60fa-4407-acb1-74e03616c939" path="/var/lib/kubelet/pods/b5a18429-60fa-4407-acb1-74e03616c939/volumes" Dec 03 12:32:21 crc kubenswrapper[5052]: I1203 12:32:21.942044 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e2d1b100-5918-4166-9fa1-5a094612727f" path="/var/lib/kubelet/pods/e2d1b100-5918-4166-9fa1-5a094612727f/volumes" Dec 03 12:32:21 crc kubenswrapper[5052]: I1203 12:32:21.946760 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fa8494cf-ad3c-4998-a3fd-a39863f72aff" path="/var/lib/kubelet/pods/fa8494cf-ad3c-4998-a3fd-a39863f72aff/volumes" Dec 03 12:32:21 crc kubenswrapper[5052]: I1203 12:32:21.947468 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"5737900d-97ae-4dcb-a65c-b40fe30cdc50","Type":"ContainerDied","Data":"a973285d5356bd9dc51bc03e5f6f166c4ac926ebd9edb7f87c4de4de53693880"} Dec 03 12:32:21 crc kubenswrapper[5052]: I1203 12:32:21.947497 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-keystone-listener-697474f89d-p7lwz"] Dec 03 12:32:21 crc kubenswrapper[5052]: I1203 12:32:21.984144 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/rabbitmq-cell1-server-0"] Dec 03 12:32:21 crc kubenswrapper[5052]: I1203 12:32:21.984162 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"6e0f19c7-34ae-4d8e-9a19-755ae1c20c1c","Type":"ContainerDied","Data":"fa8e6470aab88dd31d2b44427546df1694d629e5bea31eca80eadf84f064db9d"} Dec 03 12:32:21 crc kubenswrapper[5052]: I1203 12:32:21.984188 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"21690a66-4bcf-4865-91c7-cfe1cdf32a61","Type":"ContainerDied","Data":"4588847c60a7e1e54c1d68283ca53c647caedbb74b422846f258c273285a8d29"} Dec 03 12:32:21 crc kubenswrapper[5052]: I1203 12:32:21.984204 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"39117149-c15d-4753-9324-8a42abe5b007","Type":"ContainerDied","Data":"53093380d15a12407af002e8218c329f263122e535b3f7a98513361ca8b1c783"} Dec 03 12:32:21 crc kubenswrapper[5052]: E1203 12:32:21.985364 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[combined-ca-bundle], unattached volumes=[], failed to process volumes=[]: context canceled" pod="openstack-kuttl-tests/barbican-keystone-listener-697474f89d-p7lwz" podUID="045d578f-92c0-47c6-8356-59c489c57b9a" Dec 03 12:32:21 crc kubenswrapper[5052]: I1203 12:32:21.985647 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/kube-state-metrics-0"] Dec 03 12:32:21 crc kubenswrapper[5052]: I1203 12:32:21.985764 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/kube-state-metrics-0" podUID="44520845-0277-47ab-acc6-812aba4656ff" containerName="kube-state-metrics" containerID="cri-o://8048ed0425b57a9a994078140f513eae158b441a12e3e7142612e180f27c9cd3" gracePeriod=30 Dec 03 12:32:22 crc kubenswrapper[5052]: I1203 12:32:22.027257 5052 generic.go:334] "Generic (PLEG): container finished" podID="9a71429c-9d59-4863-8949-dcb4f79a35e2" containerID="bdab1d1a8f6c5c7a6c89c49bf44da7d62eab1cd075defa615ab51c3d50e3986e" exitCode=0 Dec 03 12:32:22 crc kubenswrapper[5052]: I1203 12:32:22.027282 5052 generic.go:334] "Generic (PLEG): container finished" podID="9a71429c-9d59-4863-8949-dcb4f79a35e2" containerID="a6e47b8819de3aa2c92fe2ccff5f26450ecf089c945a22ec1bb902b9689f2e90" exitCode=0 Dec 03 12:32:22 crc kubenswrapper[5052]: I1203 12:32:22.027290 5052 generic.go:334] "Generic (PLEG): container finished" podID="9a71429c-9d59-4863-8949-dcb4f79a35e2" containerID="59b11d0a23a20a97f6d34709bb5f894ff76b50c46bb80978918201b3b72f674c" exitCode=0 Dec 03 12:32:22 crc kubenswrapper[5052]: I1203 12:32:22.027297 5052 generic.go:334] "Generic (PLEG): container finished" podID="9a71429c-9d59-4863-8949-dcb4f79a35e2" containerID="6d6d0a9e0c44982301515f6c27041d26e825a4e904631b906720c66939d7d046" exitCode=0 Dec 03 12:32:22 crc kubenswrapper[5052]: I1203 12:32:22.027303 5052 generic.go:334] "Generic (PLEG): container finished" podID="9a71429c-9d59-4863-8949-dcb4f79a35e2" containerID="2a0e35f878dfc50c4b4c6c92dc0efc336af0f313d37084f949296ada2fc8a9a9" exitCode=0 Dec 03 12:32:22 crc kubenswrapper[5052]: I1203 12:32:22.027310 5052 generic.go:334] "Generic (PLEG): container finished" podID="9a71429c-9d59-4863-8949-dcb4f79a35e2" containerID="e56722fdb2d4fc10478af03b1d3b9ea29d7757b10975bec562e15e2949b85364" exitCode=0 Dec 03 12:32:22 crc kubenswrapper[5052]: I1203 12:32:22.027318 5052 generic.go:334] "Generic (PLEG): container finished" podID="9a71429c-9d59-4863-8949-dcb4f79a35e2" containerID="844dc1daec944c795b946deb25e04e380651676c9d2e4f9f20f0b937c87cd282" exitCode=0 Dec 03 12:32:22 crc kubenswrapper[5052]: I1203 12:32:22.027323 5052 generic.go:334] "Generic (PLEG): container finished" podID="9a71429c-9d59-4863-8949-dcb4f79a35e2" containerID="7fe073f06854f186159ef18897942608baedecbba0d07d0c8013355c37619643" exitCode=0 Dec 03 12:32:22 crc kubenswrapper[5052]: I1203 12:32:22.027329 5052 generic.go:334] "Generic (PLEG): container finished" podID="9a71429c-9d59-4863-8949-dcb4f79a35e2" containerID="44b04da4e7f0a025325c3b6375369b7ac347c2ad50331290bfb481efa7ef0736" exitCode=0 Dec 03 12:32:22 crc kubenswrapper[5052]: I1203 12:32:22.027336 5052 generic.go:334] "Generic (PLEG): container finished" podID="9a71429c-9d59-4863-8949-dcb4f79a35e2" containerID="9d3d20ffcb7e4c042c65cb9789046c8a52471258ecf145006245287d12709e44" exitCode=0 Dec 03 12:32:22 crc kubenswrapper[5052]: I1203 12:32:22.027342 5052 generic.go:334] "Generic (PLEG): container finished" podID="9a71429c-9d59-4863-8949-dcb4f79a35e2" containerID="be42b27cc3c5b3773e9ff8c15fd7487a0c826144d8760ad2e8dbee3a1515ac77" exitCode=0 Dec 03 12:32:22 crc kubenswrapper[5052]: I1203 12:32:22.027348 5052 generic.go:334] "Generic (PLEG): container finished" podID="9a71429c-9d59-4863-8949-dcb4f79a35e2" containerID="9920899d34f69573106a85bde57a4bdf8c52b176a8195efa2c785d4039c638ea" exitCode=0 Dec 03 12:32:22 crc kubenswrapper[5052]: I1203 12:32:22.027354 5052 generic.go:334] "Generic (PLEG): container finished" podID="9a71429c-9d59-4863-8949-dcb4f79a35e2" containerID="1637629a53175a4befbb5dad0d15dd5909adc30a1253d4051e99d888b51f7ccb" exitCode=0 Dec 03 12:32:22 crc kubenswrapper[5052]: I1203 12:32:22.027403 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"9a71429c-9d59-4863-8949-dcb4f79a35e2","Type":"ContainerDied","Data":"bdab1d1a8f6c5c7a6c89c49bf44da7d62eab1cd075defa615ab51c3d50e3986e"} Dec 03 12:32:22 crc kubenswrapper[5052]: I1203 12:32:22.027430 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"9a71429c-9d59-4863-8949-dcb4f79a35e2","Type":"ContainerDied","Data":"a6e47b8819de3aa2c92fe2ccff5f26450ecf089c945a22ec1bb902b9689f2e90"} Dec 03 12:32:22 crc kubenswrapper[5052]: I1203 12:32:22.027440 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"9a71429c-9d59-4863-8949-dcb4f79a35e2","Type":"ContainerDied","Data":"59b11d0a23a20a97f6d34709bb5f894ff76b50c46bb80978918201b3b72f674c"} Dec 03 12:32:22 crc kubenswrapper[5052]: I1203 12:32:22.027449 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"9a71429c-9d59-4863-8949-dcb4f79a35e2","Type":"ContainerDied","Data":"6d6d0a9e0c44982301515f6c27041d26e825a4e904631b906720c66939d7d046"} Dec 03 12:32:22 crc kubenswrapper[5052]: I1203 12:32:22.027468 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"9a71429c-9d59-4863-8949-dcb4f79a35e2","Type":"ContainerDied","Data":"2a0e35f878dfc50c4b4c6c92dc0efc336af0f313d37084f949296ada2fc8a9a9"} Dec 03 12:32:22 crc kubenswrapper[5052]: I1203 12:32:22.027477 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"9a71429c-9d59-4863-8949-dcb4f79a35e2","Type":"ContainerDied","Data":"e56722fdb2d4fc10478af03b1d3b9ea29d7757b10975bec562e15e2949b85364"} Dec 03 12:32:22 crc kubenswrapper[5052]: I1203 12:32:22.027485 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"9a71429c-9d59-4863-8949-dcb4f79a35e2","Type":"ContainerDied","Data":"844dc1daec944c795b946deb25e04e380651676c9d2e4f9f20f0b937c87cd282"} Dec 03 12:32:22 crc kubenswrapper[5052]: I1203 12:32:22.027494 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"9a71429c-9d59-4863-8949-dcb4f79a35e2","Type":"ContainerDied","Data":"7fe073f06854f186159ef18897942608baedecbba0d07d0c8013355c37619643"} Dec 03 12:32:22 crc kubenswrapper[5052]: I1203 12:32:22.027518 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"9a71429c-9d59-4863-8949-dcb4f79a35e2","Type":"ContainerDied","Data":"44b04da4e7f0a025325c3b6375369b7ac347c2ad50331290bfb481efa7ef0736"} Dec 03 12:32:22 crc kubenswrapper[5052]: I1203 12:32:22.027527 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"9a71429c-9d59-4863-8949-dcb4f79a35e2","Type":"ContainerDied","Data":"9d3d20ffcb7e4c042c65cb9789046c8a52471258ecf145006245287d12709e44"} Dec 03 12:32:22 crc kubenswrapper[5052]: I1203 12:32:22.027535 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"9a71429c-9d59-4863-8949-dcb4f79a35e2","Type":"ContainerDied","Data":"be42b27cc3c5b3773e9ff8c15fd7487a0c826144d8760ad2e8dbee3a1515ac77"} Dec 03 12:32:22 crc kubenswrapper[5052]: I1203 12:32:22.027545 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"9a71429c-9d59-4863-8949-dcb4f79a35e2","Type":"ContainerDied","Data":"9920899d34f69573106a85bde57a4bdf8c52b176a8195efa2c785d4039c638ea"} Dec 03 12:32:22 crc kubenswrapper[5052]: I1203 12:32:22.027553 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"9a71429c-9d59-4863-8949-dcb4f79a35e2","Type":"ContainerDied","Data":"1637629a53175a4befbb5dad0d15dd5909adc30a1253d4051e99d888b51f7ccb"} Dec 03 12:32:22 crc kubenswrapper[5052]: I1203 12:32:22.028898 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-worker-644558fbc5-4gtxj"] Dec 03 12:32:22 crc kubenswrapper[5052]: I1203 12:32:22.029109 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/barbican-worker-644558fbc5-4gtxj" podUID="8ac6e8f0-d49c-4a4c-846a-6cec226bd360" containerName="barbican-worker-log" containerID="cri-o://57de2e7ba65bad4f495dc483cc50efdd66d36654d3d53aadfa56507cc2faada6" gracePeriod=30 Dec 03 12:32:22 crc kubenswrapper[5052]: I1203 12:32:22.029462 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/barbican-worker-644558fbc5-4gtxj" podUID="8ac6e8f0-d49c-4a4c-846a-6cec226bd360" containerName="barbican-worker" containerID="cri-o://4893c6e7800c80d6b9f1c1b9d2a79b5fd298a43e5935cb9d7b3f6072575ccfc6" gracePeriod=30 Dec 03 12:32:22 crc kubenswrapper[5052]: I1203 12:32:22.038747 5052 generic.go:334] "Generic (PLEG): container finished" podID="cfd8af50-a644-4b24-8483-dd344f2218e4" containerID="dc83501080d7c2386bc4543b9befadc35e0e843ede852c450da98e997424e43e" exitCode=143 Dec 03 12:32:22 crc kubenswrapper[5052]: I1203 12:32:22.038805 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-65444bfc86-th5pw" event={"ID":"cfd8af50-a644-4b24-8483-dd344f2218e4","Type":"ContainerDied","Data":"dc83501080d7c2386bc4543b9befadc35e0e843ede852c450da98e997424e43e"} Dec 03 12:32:22 crc kubenswrapper[5052]: I1203 12:32:22.040115 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-worker-c899dcc5-n6rdv"] Dec 03 12:32:22 crc kubenswrapper[5052]: E1203 12:32:22.040877 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[combined-ca-bundle], unattached volumes=[], failed to process volumes=[]: context canceled" pod="openstack-kuttl-tests/barbican-worker-c899dcc5-n6rdv" podUID="cc8c914a-cb83-48ae-97a7-fca144e96f52" Dec 03 12:32:22 crc kubenswrapper[5052]: I1203 12:32:22.050638 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 12:32:22 crc kubenswrapper[5052]: I1203 12:32:22.050827 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" podUID="d09b9817-6ea5-4328-9027-3311b7782157" containerName="nova-cell1-novncproxy-novncproxy" containerID="cri-o://7e1821a6c6bd165b882e82fb2451aca61d98f3c04110b410f0845d91b6abfa0b" gracePeriod=30 Dec 03 12:32:22 crc kubenswrapper[5052]: I1203 12:32:22.061805 5052 generic.go:334] "Generic (PLEG): container finished" podID="4b3e73a0-1320-427f-bcd7-d8062e315e39" containerID="82c2ce15a7d7e6c87d6ed20e3eaafec5eb24b16b4abc6508d88b41511f4560a0" exitCode=0 Dec 03 12:32:22 crc kubenswrapper[5052]: I1203 12:32:22.061870 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-58d7cc6794-9z49k" event={"ID":"4b3e73a0-1320-427f-bcd7-d8062e315e39","Type":"ContainerDied","Data":"82c2ce15a7d7e6c87d6ed20e3eaafec5eb24b16b4abc6508d88b41511f4560a0"} Dec 03 12:32:22 crc kubenswrapper[5052]: I1203 12:32:22.075604 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell1-db-create-9vd8k"] Dec 03 12:32:22 crc kubenswrapper[5052]: I1203 12:32:22.090192 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_ovsdbserver-sb-0_3fec3449-9615-4377-a600-69d816ec5416/ovsdbserver-sb/0.log" Dec 03 12:32:22 crc kubenswrapper[5052]: I1203 12:32:22.090341 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-sb-0" event={"ID":"3fec3449-9615-4377-a600-69d816ec5416","Type":"ContainerDied","Data":"21da3337b09e6b326697aef155af4bcd6a984d400eaed99ee26b1a3a045fba72"} Dec 03 12:32:22 crc kubenswrapper[5052]: I1203 12:32:22.090381 5052 generic.go:334] "Generic (PLEG): container finished" podID="3fec3449-9615-4377-a600-69d816ec5416" containerID="21da3337b09e6b326697aef155af4bcd6a984d400eaed99ee26b1a3a045fba72" exitCode=2 Dec 03 12:32:22 crc kubenswrapper[5052]: I1203 12:32:22.090425 5052 generic.go:334] "Generic (PLEG): container finished" podID="3fec3449-9615-4377-a600-69d816ec5416" containerID="19c04c4116df77c2accd0a6c361be38550f970c0c0f8032ebf2cd11805800978" exitCode=143 Dec 03 12:32:22 crc kubenswrapper[5052]: I1203 12:32:22.090491 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-sb-0" event={"ID":"3fec3449-9615-4377-a600-69d816ec5416","Type":"ContainerDied","Data":"19c04c4116df77c2accd0a6c361be38550f970c0c0f8032ebf2cd11805800978"} Dec 03 12:32:22 crc kubenswrapper[5052]: I1203 12:32:22.093720 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" podUID="b8613a94-1e2e-477f-8d52-713c70f5282c" containerName="rabbitmq" containerID="cri-o://ecd92b1e5f2c4bb6955463da6c223b6fc821bf3dfbf16ddd65252cba7d35cea5" gracePeriod=604800 Dec 03 12:32:22 crc kubenswrapper[5052]: I1203 12:32:22.095085 5052 generic.go:334] "Generic (PLEG): container finished" podID="a65428bc-9326-4ace-a985-9f796b951304" containerID="caa8dd02dbab945ca9d553d926ff4896db88a3b315e5384ca071cd388a2f2766" exitCode=143 Dec 03 12:32:22 crc kubenswrapper[5052]: I1203 12:32:22.095150 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"a65428bc-9326-4ace-a985-9f796b951304","Type":"ContainerDied","Data":"caa8dd02dbab945ca9d553d926ff4896db88a3b315e5384ca071cd388a2f2766"} Dec 03 12:32:22 crc kubenswrapper[5052]: I1203 12:32:22.097084 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell1-db-create-9vd8k"] Dec 03 12:32:22 crc kubenswrapper[5052]: I1203 12:32:22.102047 5052 generic.go:334] "Generic (PLEG): container finished" podID="7cbe1c2c-bc04-4232-9901-bc9d134df3d6" containerID="1d2a9d60a97780af3d81ede77692ed338720a0c0a5b32ac9f95fa608d8ace7d1" exitCode=143 Dec 03 12:32:22 crc kubenswrapper[5052]: I1203 12:32:22.102409 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"7cbe1c2c-bc04-4232-9901-bc9d134df3d6","Type":"ContainerDied","Data":"1d2a9d60a97780af3d81ede77692ed338720a0c0a5b32ac9f95fa608d8ace7d1"} Dec 03 12:32:22 crc kubenswrapper[5052]: I1203 12:32:22.109758 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/novacell115a6-account-delete-pbwnc"] Dec 03 12:32:22 crc kubenswrapper[5052]: I1203 12:32:22.117753 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell1-15a6-account-create-update-z2tdn"] Dec 03 12:32:22 crc kubenswrapper[5052]: E1203 12:32:22.123524 5052 cadvisor_stats_provider.go:516] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9a71429c_9d59_4863_8949_dcb4f79a35e2.slice/crio-7fe073f06854f186159ef18897942608baedecbba0d07d0c8013355c37619643.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3fec3449_9615_4377_a600_69d816ec5416.slice/crio-conmon-21da3337b09e6b326697aef155af4bcd6a984d400eaed99ee26b1a3a045fba72.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5737900d_97ae_4dcb_a65c_b40fe30cdc50.slice/crio-conmon-a973285d5356bd9dc51bc03e5f6f166c4ac926ebd9edb7f87c4de4de53693880.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9a71429c_9d59_4863_8949_dcb4f79a35e2.slice/crio-1637629a53175a4befbb5dad0d15dd5909adc30a1253d4051e99d888b51f7ccb.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9a71429c_9d59_4863_8949_dcb4f79a35e2.slice/crio-9d3d20ffcb7e4c042c65cb9789046c8a52471258ecf145006245287d12709e44.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5737900d_97ae_4dcb_a65c_b40fe30cdc50.slice/crio-a973285d5356bd9dc51bc03e5f6f166c4ac926ebd9edb7f87c4de4de53693880.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9a71429c_9d59_4863_8949_dcb4f79a35e2.slice/crio-e56722fdb2d4fc10478af03b1d3b9ea29d7757b10975bec562e15e2949b85364.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9a71429c_9d59_4863_8949_dcb4f79a35e2.slice/crio-9920899d34f69573106a85bde57a4bdf8c52b176a8195efa2c785d4039c638ea.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2ed8a31a_9033_4601_84bf_9eaf57f1f155.slice/crio-220531c8139609a0b36f479504e8821b1f41b4615b00ee05432f9560ade3acb1.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9a71429c_9d59_4863_8949_dcb4f79a35e2.slice/crio-2a0e35f878dfc50c4b4c6c92dc0efc336af0f313d37084f949296ada2fc8a9a9.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcfd8af50_a644_4b24_8483_dd344f2218e4.slice/crio-dc83501080d7c2386bc4543b9befadc35e0e843ede852c450da98e997424e43e.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9a71429c_9d59_4863_8949_dcb4f79a35e2.slice/crio-conmon-6d6d0a9e0c44982301515f6c27041d26e825a4e904631b906720c66939d7d046.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9a71429c_9d59_4863_8949_dcb4f79a35e2.slice/crio-bdab1d1a8f6c5c7a6c89c49bf44da7d62eab1cd075defa615ab51c3d50e3986e.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9a71429c_9d59_4863_8949_dcb4f79a35e2.slice/crio-conmon-bdab1d1a8f6c5c7a6c89c49bf44da7d62eab1cd075defa615ab51c3d50e3986e.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9a71429c_9d59_4863_8949_dcb4f79a35e2.slice/crio-conmon-7fe073f06854f186159ef18897942608baedecbba0d07d0c8013355c37619643.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcfd8af50_a644_4b24_8483_dd344f2218e4.slice/crio-conmon-dc83501080d7c2386bc4543b9befadc35e0e843ede852c450da98e997424e43e.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9a71429c_9d59_4863_8949_dcb4f79a35e2.slice/crio-conmon-844dc1daec944c795b946deb25e04e380651676c9d2e4f9f20f0b937c87cd282.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6e0f19c7_34ae_4d8e_9a19_755ae1c20c1c.slice/crio-fa8e6470aab88dd31d2b44427546df1694d629e5bea31eca80eadf84f064db9d.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9a71429c_9d59_4863_8949_dcb4f79a35e2.slice/crio-a6e47b8819de3aa2c92fe2ccff5f26450ecf089c945a22ec1bb902b9689f2e90.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda65428bc_9326_4ace_a985_9f796b951304.slice/crio-caa8dd02dbab945ca9d553d926ff4896db88a3b315e5384ca071cd388a2f2766.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9a71429c_9d59_4863_8949_dcb4f79a35e2.slice/crio-conmon-59b11d0a23a20a97f6d34709bb5f894ff76b50c46bb80978918201b3b72f674c.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9a71429c_9d59_4863_8949_dcb4f79a35e2.slice/crio-conmon-a6e47b8819de3aa2c92fe2ccff5f26450ecf089c945a22ec1bb902b9689f2e90.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3fec3449_9615_4377_a600_69d816ec5416.slice/crio-19c04c4116df77c2accd0a6c361be38550f970c0c0f8032ebf2cd11805800978.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9a71429c_9d59_4863_8949_dcb4f79a35e2.slice/crio-conmon-9920899d34f69573106a85bde57a4bdf8c52b176a8195efa2c785d4039c638ea.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9a71429c_9d59_4863_8949_dcb4f79a35e2.slice/crio-844dc1daec944c795b946deb25e04e380651676c9d2e4f9f20f0b937c87cd282.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9a71429c_9d59_4863_8949_dcb4f79a35e2.slice/crio-be42b27cc3c5b3773e9ff8c15fd7487a0c826144d8760ad2e8dbee3a1515ac77.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3fec3449_9615_4377_a600_69d816ec5416.slice/crio-conmon-19c04c4116df77c2accd0a6c361be38550f970c0c0f8032ebf2cd11805800978.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9a71429c_9d59_4863_8949_dcb4f79a35e2.slice/crio-44b04da4e7f0a025325c3b6375369b7ac347c2ad50331290bfb481efa7ef0736.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9a71429c_9d59_4863_8949_dcb4f79a35e2.slice/crio-conmon-2a0e35f878dfc50c4b4c6c92dc0efc336af0f313d37084f949296ada2fc8a9a9.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod44520845_0277_47ab_acc6_812aba4656ff.slice/crio-8048ed0425b57a9a994078140f513eae158b441a12e3e7142612e180f27c9cd3.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9a71429c_9d59_4863_8949_dcb4f79a35e2.slice/crio-conmon-be42b27cc3c5b3773e9ff8c15fd7487a0c826144d8760ad2e8dbee3a1515ac77.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4b3e73a0_1320_427f_bcd7_d8062e315e39.slice/crio-82c2ce15a7d7e6c87d6ed20e3eaafec5eb24b16b4abc6508d88b41511f4560a0.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbb1723c7_4e41_4b0e_b6b8_f8024dafdaee.slice/crio-ab269ac11d99aed29d8c53741cb5bdd97ed1fe052bc19e5e9c3e0f4152eaeed4.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod21690a66_4bcf_4865_91c7_cfe1cdf32a61.slice/crio-conmon-fb673c0dc2e5ca2f8ef44bb06008daf9443cc163a93859ac48d11b2d619a8092.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2ed8a31a_9033_4601_84bf_9eaf57f1f155.slice/crio-conmon-220531c8139609a0b36f479504e8821b1f41b4615b00ee05432f9560ade3acb1.scope\": RecentStats: unable to find data in memory cache]" Dec 03 12:32:22 crc kubenswrapper[5052]: I1203 12:32:22.126562 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_ovsdbserver-nb-0_dccae7c1-009e-4b3b-b66a-ecbf6e53489a/ovsdbserver-nb/0.log" Dec 03 12:32:22 crc kubenswrapper[5052]: I1203 12:32:22.126620 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-nb-0" event={"ID":"dccae7c1-009e-4b3b-b66a-ecbf6e53489a","Type":"ContainerDied","Data":"a0693dce3e63878da0e8a79833bdbb9e6496c63685edbeabed749e7bf6336079"} Dec 03 12:32:22 crc kubenswrapper[5052]: I1203 12:32:22.126646 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a0693dce3e63878da0e8a79833bdbb9e6496c63685edbeabed749e7bf6336079" Dec 03 12:32:22 crc kubenswrapper[5052]: I1203 12:32:22.126964 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/openstack-cell1-galera-0" podUID="aa55d0c8-56de-4edd-92b3-9252f169fed8" containerName="galera" containerID="cri-o://8bf7dd5e7641b5ac8d567b37be404e10979d45b84ac534f9d893fee92ab89c5e" gracePeriod=30 Dec 03 12:32:22 crc kubenswrapper[5052]: I1203 12:32:22.127041 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell1-15a6-account-create-update-z2tdn"] Dec 03 12:32:22 crc kubenswrapper[5052]: I1203 12:32:22.138929 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/placemente65a-account-delete-s7x52"] Dec 03 12:32:22 crc kubenswrapper[5052]: I1203 12:32:22.161838 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/keystone-bootstrap-8nbqq"] Dec 03 12:32:22 crc kubenswrapper[5052]: I1203 12:32:22.180214 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/keystone-bootstrap-8nbqq"] Dec 03 12:32:22 crc kubenswrapper[5052]: I1203 12:32:22.182767 5052 kubelet_pods.go:1007] "Unable to retrieve pull secret, the image pull may not succeed." pod="openstack-kuttl-tests/novacell046b8-account-delete-nqzlj" secret="" err="secret \"galera-openstack-dockercfg-w7gzp\" not found" Dec 03 12:32:22 crc kubenswrapper[5052]: I1203 12:32:22.182801 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/novacell046b8-account-delete-nqzlj" Dec 03 12:32:22 crc kubenswrapper[5052]: I1203 12:32:22.191851 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/keystone-db-sync-z2t88"] Dec 03 12:32:22 crc kubenswrapper[5052]: E1203 12:32:22.198879 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-wr9nk operator-scripts], unattached volumes=[], failed to process volumes=[]: context canceled" pod="openstack-kuttl-tests/novacell115a6-account-delete-pbwnc" podUID="f12527ba-678e-4ed9-82ff-cec336f607f3" Dec 03 12:32:22 crc kubenswrapper[5052]: I1203 12:32:22.202590 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/keystone-db-sync-z2t88"] Dec 03 12:32:22 crc kubenswrapper[5052]: I1203 12:32:22.208714 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_ovsdbserver-nb-0_dccae7c1-009e-4b3b-b66a-ecbf6e53489a/ovsdbserver-nb/0.log" Dec 03 12:32:22 crc kubenswrapper[5052]: I1203 12:32:22.208783 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:32:22 crc kubenswrapper[5052]: I1203 12:32:22.222015 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/keystone-6b8b855786-bltgj"] Dec 03 12:32:22 crc kubenswrapper[5052]: I1203 12:32:22.222289 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/keystone-6b8b855786-bltgj" podUID="43933abc-db67-4ce5-9f30-6f8dbf3f59c8" containerName="keystone-api" containerID="cri-o://aec2d2ca3d9979ac47cbc62e58c9a1c7b06d1e13c5205fc585d92567ca0e4ec3" gracePeriod=30 Dec 03 12:32:22 crc kubenswrapper[5052]: I1203 12:32:22.257938 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/openstack-galera-0"] Dec 03 12:32:22 crc kubenswrapper[5052]: I1203 12:32:22.267839 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/dccae7c1-009e-4b3b-b66a-ecbf6e53489a-config\") pod \"dccae7c1-009e-4b3b-b66a-ecbf6e53489a\" (UID: \"dccae7c1-009e-4b3b-b66a-ecbf6e53489a\") " Dec 03 12:32:22 crc kubenswrapper[5052]: I1203 12:32:22.267915 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b9v9t\" (UniqueName: \"kubernetes.io/projected/dccae7c1-009e-4b3b-b66a-ecbf6e53489a-kube-api-access-b9v9t\") pod \"dccae7c1-009e-4b3b-b66a-ecbf6e53489a\" (UID: \"dccae7c1-009e-4b3b-b66a-ecbf6e53489a\") " Dec 03 12:32:22 crc kubenswrapper[5052]: I1203 12:32:22.267940 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/dccae7c1-009e-4b3b-b66a-ecbf6e53489a-ovsdb-rundir\") pod \"dccae7c1-009e-4b3b-b66a-ecbf6e53489a\" (UID: \"dccae7c1-009e-4b3b-b66a-ecbf6e53489a\") " Dec 03 12:32:22 crc kubenswrapper[5052]: I1203 12:32:22.267972 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/dccae7c1-009e-4b3b-b66a-ecbf6e53489a-combined-ca-bundle\") pod \"dccae7c1-009e-4b3b-b66a-ecbf6e53489a\" (UID: \"dccae7c1-009e-4b3b-b66a-ecbf6e53489a\") " Dec 03 12:32:22 crc kubenswrapper[5052]: I1203 12:32:22.268002 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/dccae7c1-009e-4b3b-b66a-ecbf6e53489a-metrics-certs-tls-certs\") pod \"dccae7c1-009e-4b3b-b66a-ecbf6e53489a\" (UID: \"dccae7c1-009e-4b3b-b66a-ecbf6e53489a\") " Dec 03 12:32:22 crc kubenswrapper[5052]: I1203 12:32:22.268086 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/dccae7c1-009e-4b3b-b66a-ecbf6e53489a-scripts\") pod \"dccae7c1-009e-4b3b-b66a-ecbf6e53489a\" (UID: \"dccae7c1-009e-4b3b-b66a-ecbf6e53489a\") " Dec 03 12:32:22 crc kubenswrapper[5052]: I1203 12:32:22.268149 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovndbcluster-nb-etc-ovn\" (UniqueName: \"kubernetes.io/local-volume/local-storage03-crc\") pod \"dccae7c1-009e-4b3b-b66a-ecbf6e53489a\" (UID: \"dccae7c1-009e-4b3b-b66a-ecbf6e53489a\") " Dec 03 12:32:22 crc kubenswrapper[5052]: I1203 12:32:22.268213 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovsdbserver-nb-tls-certs\" (UniqueName: \"kubernetes.io/secret/dccae7c1-009e-4b3b-b66a-ecbf6e53489a-ovsdbserver-nb-tls-certs\") pod \"dccae7c1-009e-4b3b-b66a-ecbf6e53489a\" (UID: \"dccae7c1-009e-4b3b-b66a-ecbf6e53489a\") " Dec 03 12:32:22 crc kubenswrapper[5052]: E1203 12:32:22.268863 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:32:22 crc kubenswrapper[5052]: E1203 12:32:22.268913 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/478e18e8-077e-406e-aa66-19bc1d28df46-operator-scripts podName:478e18e8-077e-406e-aa66-19bc1d28df46 nodeName:}" failed. No retries permitted until 2025-12-03 12:32:22.768898971 +0000 UTC m=+4333.357140735 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/478e18e8-077e-406e-aa66-19bc1d28df46-operator-scripts") pod "novacell046b8-account-delete-nqzlj" (UID: "478e18e8-077e-406e-aa66-19bc1d28df46") : configmap "openstack-scripts" not found Dec 03 12:32:22 crc kubenswrapper[5052]: I1203 12:32:22.270993 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dccae7c1-009e-4b3b-b66a-ecbf6e53489a-scripts" (OuterVolumeSpecName: "scripts") pod "dccae7c1-009e-4b3b-b66a-ecbf6e53489a" (UID: "dccae7c1-009e-4b3b-b66a-ecbf6e53489a"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:32:22 crc kubenswrapper[5052]: I1203 12:32:22.274252 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dccae7c1-009e-4b3b-b66a-ecbf6e53489a-ovsdb-rundir" (OuterVolumeSpecName: "ovsdb-rundir") pod "dccae7c1-009e-4b3b-b66a-ecbf6e53489a" (UID: "dccae7c1-009e-4b3b-b66a-ecbf6e53489a"). InnerVolumeSpecName "ovsdb-rundir". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:32:22 crc kubenswrapper[5052]: I1203 12:32:22.275203 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dccae7c1-009e-4b3b-b66a-ecbf6e53489a-config" (OuterVolumeSpecName: "config") pod "dccae7c1-009e-4b3b-b66a-ecbf6e53489a" (UID: "dccae7c1-009e-4b3b-b66a-ecbf6e53489a"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:32:22 crc kubenswrapper[5052]: I1203 12:32:22.296396 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dccae7c1-009e-4b3b-b66a-ecbf6e53489a-kube-api-access-b9v9t" (OuterVolumeSpecName: "kube-api-access-b9v9t") pod "dccae7c1-009e-4b3b-b66a-ecbf6e53489a" (UID: "dccae7c1-009e-4b3b-b66a-ecbf6e53489a"). InnerVolumeSpecName "kube-api-access-b9v9t". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:32:22 crc kubenswrapper[5052]: I1203 12:32:22.296622 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage03-crc" (OuterVolumeSpecName: "ovndbcluster-nb-etc-ovn") pod "dccae7c1-009e-4b3b-b66a-ecbf6e53489a" (UID: "dccae7c1-009e-4b3b-b66a-ecbf6e53489a"). InnerVolumeSpecName "local-storage03-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 12:32:22 crc kubenswrapper[5052]: I1203 12:32:22.343574 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/keystone-db-create-zrpdw"] Dec 03 12:32:22 crc kubenswrapper[5052]: I1203 12:32:22.367572 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/keystone-db-create-zrpdw"] Dec 03 12:32:22 crc kubenswrapper[5052]: I1203 12:32:22.371364 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/dccae7c1-009e-4b3b-b66a-ecbf6e53489a-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:22 crc kubenswrapper[5052]: I1203 12:32:22.371417 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage03-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage03-crc\") on node \"crc\" " Dec 03 12:32:22 crc kubenswrapper[5052]: I1203 12:32:22.371430 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/dccae7c1-009e-4b3b-b66a-ecbf6e53489a-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:22 crc kubenswrapper[5052]: I1203 12:32:22.371443 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-b9v9t\" (UniqueName: \"kubernetes.io/projected/dccae7c1-009e-4b3b-b66a-ecbf6e53489a-kube-api-access-b9v9t\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:22 crc kubenswrapper[5052]: I1203 12:32:22.371454 5052 reconciler_common.go:293] "Volume detached for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/dccae7c1-009e-4b3b-b66a-ecbf6e53489a-ovsdb-rundir\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:22 crc kubenswrapper[5052]: I1203 12:32:22.403831 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/keystone-39e2-account-create-update-74dh8"] Dec 03 12:32:22 crc kubenswrapper[5052]: I1203 12:32:22.407734 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage03-crc" (UniqueName: "kubernetes.io/local-volume/local-storage03-crc") on node "crc" Dec 03 12:32:22 crc kubenswrapper[5052]: I1203 12:32:22.412071 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/keystone-39e2-account-create-update-74dh8"] Dec 03 12:32:22 crc kubenswrapper[5052]: I1203 12:32:22.429432 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dccae7c1-009e-4b3b-b66a-ecbf6e53489a-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "dccae7c1-009e-4b3b-b66a-ecbf6e53489a" (UID: "dccae7c1-009e-4b3b-b66a-ecbf6e53489a"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:32:22 crc kubenswrapper[5052]: I1203 12:32:22.444693 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-hjfg4" Dec 03 12:32:22 crc kubenswrapper[5052]: I1203 12:32:22.456589 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron2174-account-delete-9dnxs"] Dec 03 12:32:22 crc kubenswrapper[5052]: I1203 12:32:22.474956 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/dccae7c1-009e-4b3b-b66a-ecbf6e53489a-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:22 crc kubenswrapper[5052]: I1203 12:32:22.474985 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage03-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage03-crc\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:22 crc kubenswrapper[5052]: I1203 12:32:22.475144 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstackclient" Dec 03 12:32:22 crc kubenswrapper[5052]: W1203 12:32:22.487370 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod52bf0c8d_03b3_444b_80b6_13d41f5f9fb9.slice/crio-e4dc3d0b73a5da661e1038b2fbe8fcf9820b4b233fb38d76f37ea7b29b323d19 WatchSource:0}: Error finding container e4dc3d0b73a5da661e1038b2fbe8fcf9820b4b233fb38d76f37ea7b29b323d19: Status 404 returned error can't find the container with id e4dc3d0b73a5da661e1038b2fbe8fcf9820b4b233fb38d76f37ea7b29b323d19 Dec 03 12:32:22 crc kubenswrapper[5052]: I1203 12:32:22.493881 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_ovsdbserver-sb-0_3fec3449-9615-4377-a600-69d816ec5416/ovsdbserver-sb/0.log" Dec 03 12:32:22 crc kubenswrapper[5052]: I1203 12:32:22.493966 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:32:22 crc kubenswrapper[5052]: I1203 12:32:22.504205 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/swift-proxy-55bdbb59c4-c4cjv" podUID="16cac95c-d1d2-4542-bb8e-4bdc1923c684" containerName="proxy-server" probeResult="failure" output="Get \"https://10.217.0.241:8080/healthcheck\": dial tcp 10.217.0.241:8080: connect: connection refused" Dec 03 12:32:22 crc kubenswrapper[5052]: I1203 12:32:22.504343 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/swift-proxy-55bdbb59c4-c4cjv" podUID="16cac95c-d1d2-4542-bb8e-4bdc1923c684" containerName="proxy-httpd" probeResult="failure" output="Get \"https://10.217.0.241:8080/healthcheck\": dial tcp 10.217.0.241:8080: connect: connection refused" Dec 03 12:32:22 crc kubenswrapper[5052]: I1203 12:32:22.504919 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance5b9b-account-delete-ggk6q"] Dec 03 12:32:22 crc kubenswrapper[5052]: I1203 12:32:22.583710 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"openstack-config-secret\" (UniqueName: \"kubernetes.io/secret/2ed8a31a-9033-4601-84bf-9eaf57f1f155-openstack-config-secret\") pod \"2ed8a31a-9033-4601-84bf-9eaf57f1f155\" (UID: \"2ed8a31a-9033-4601-84bf-9eaf57f1f155\") " Dec 03 12:32:22 crc kubenswrapper[5052]: I1203 12:32:22.584216 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovsdbserver-sb-tls-certs\" (UniqueName: \"kubernetes.io/secret/3fec3449-9615-4377-a600-69d816ec5416-ovsdbserver-sb-tls-certs\") pod \"3fec3449-9615-4377-a600-69d816ec5416\" (UID: \"3fec3449-9615-4377-a600-69d816ec5416\") " Dec 03 12:32:22 crc kubenswrapper[5052]: I1203 12:32:22.584317 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/3fec3449-9615-4377-a600-69d816ec5416-ovsdb-rundir\") pod \"3fec3449-9615-4377-a600-69d816ec5416\" (UID: \"3fec3449-9615-4377-a600-69d816ec5416\") " Dec 03 12:32:22 crc kubenswrapper[5052]: I1203 12:32:22.584370 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3fec3449-9615-4377-a600-69d816ec5416-config\") pod \"3fec3449-9615-4377-a600-69d816ec5416\" (UID: \"3fec3449-9615-4377-a600-69d816ec5416\") " Dec 03 12:32:22 crc kubenswrapper[5052]: I1203 12:32:22.584397 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8dr2d\" (UniqueName: \"kubernetes.io/projected/3fec3449-9615-4377-a600-69d816ec5416-kube-api-access-8dr2d\") pod \"3fec3449-9615-4377-a600-69d816ec5416\" (UID: \"3fec3449-9615-4377-a600-69d816ec5416\") " Dec 03 12:32:22 crc kubenswrapper[5052]: I1203 12:32:22.584467 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/3fec3449-9615-4377-a600-69d816ec5416-scripts\") pod \"3fec3449-9615-4377-a600-69d816ec5416\" (UID: \"3fec3449-9615-4377-a600-69d816ec5416\") " Dec 03 12:32:22 crc kubenswrapper[5052]: I1203 12:32:22.584492 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/3fec3449-9615-4377-a600-69d816ec5416-metrics-certs-tls-certs\") pod \"3fec3449-9615-4377-a600-69d816ec5416\" (UID: \"3fec3449-9615-4377-a600-69d816ec5416\") " Dec 03 12:32:22 crc kubenswrapper[5052]: I1203 12:32:22.584588 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2ed8a31a-9033-4601-84bf-9eaf57f1f155-combined-ca-bundle\") pod \"2ed8a31a-9033-4601-84bf-9eaf57f1f155\" (UID: \"2ed8a31a-9033-4601-84bf-9eaf57f1f155\") " Dec 03 12:32:22 crc kubenswrapper[5052]: I1203 12:32:22.584619 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dvs55\" (UniqueName: \"kubernetes.io/projected/2ed8a31a-9033-4601-84bf-9eaf57f1f155-kube-api-access-dvs55\") pod \"2ed8a31a-9033-4601-84bf-9eaf57f1f155\" (UID: \"2ed8a31a-9033-4601-84bf-9eaf57f1f155\") " Dec 03 12:32:22 crc kubenswrapper[5052]: I1203 12:32:22.584657 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovndbcluster-sb-etc-ovn\" (UniqueName: \"kubernetes.io/local-volume/local-storage08-crc\") pod \"3fec3449-9615-4377-a600-69d816ec5416\" (UID: \"3fec3449-9615-4377-a600-69d816ec5416\") " Dec 03 12:32:22 crc kubenswrapper[5052]: I1203 12:32:22.585757 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3fec3449-9615-4377-a600-69d816ec5416-ovsdb-rundir" (OuterVolumeSpecName: "ovsdb-rundir") pod "3fec3449-9615-4377-a600-69d816ec5416" (UID: "3fec3449-9615-4377-a600-69d816ec5416"). InnerVolumeSpecName "ovsdb-rundir". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:32:22 crc kubenswrapper[5052]: I1203 12:32:22.588655 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"openstack-config\" (UniqueName: \"kubernetes.io/configmap/2ed8a31a-9033-4601-84bf-9eaf57f1f155-openstack-config\") pod \"2ed8a31a-9033-4601-84bf-9eaf57f1f155\" (UID: \"2ed8a31a-9033-4601-84bf-9eaf57f1f155\") " Dec 03 12:32:22 crc kubenswrapper[5052]: I1203 12:32:22.588722 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3fec3449-9615-4377-a600-69d816ec5416-combined-ca-bundle\") pod \"3fec3449-9615-4377-a600-69d816ec5416\" (UID: \"3fec3449-9615-4377-a600-69d816ec5416\") " Dec 03 12:32:22 crc kubenswrapper[5052]: I1203 12:32:22.590034 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3fec3449-9615-4377-a600-69d816ec5416-scripts" (OuterVolumeSpecName: "scripts") pod "3fec3449-9615-4377-a600-69d816ec5416" (UID: "3fec3449-9615-4377-a600-69d816ec5416"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:32:22 crc kubenswrapper[5052]: I1203 12:32:22.590638 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3fec3449-9615-4377-a600-69d816ec5416-config" (OuterVolumeSpecName: "config") pod "3fec3449-9615-4377-a600-69d816ec5416" (UID: "3fec3449-9615-4377-a600-69d816ec5416"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:32:22 crc kubenswrapper[5052]: I1203 12:32:22.607832 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dccae7c1-009e-4b3b-b66a-ecbf6e53489a-metrics-certs-tls-certs" (OuterVolumeSpecName: "metrics-certs-tls-certs") pod "dccae7c1-009e-4b3b-b66a-ecbf6e53489a" (UID: "dccae7c1-009e-4b3b-b66a-ecbf6e53489a"). InnerVolumeSpecName "metrics-certs-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:32:22 crc kubenswrapper[5052]: I1203 12:32:22.608486 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-wr9nk\" (UniqueName: \"kubernetes.io/projected/f12527ba-678e-4ed9-82ff-cec336f607f3-kube-api-access-wr9nk\") pod \"novacell115a6-account-delete-pbwnc\" (UID: \"f12527ba-678e-4ed9-82ff-cec336f607f3\") " pod="openstack-kuttl-tests/novacell115a6-account-delete-pbwnc" Dec 03 12:32:22 crc kubenswrapper[5052]: I1203 12:32:22.608760 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage08-crc" (OuterVolumeSpecName: "ovndbcluster-sb-etc-ovn") pod "3fec3449-9615-4377-a600-69d816ec5416" (UID: "3fec3449-9615-4377-a600-69d816ec5416"). InnerVolumeSpecName "local-storage08-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 12:32:22 crc kubenswrapper[5052]: I1203 12:32:22.611161 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/f12527ba-678e-4ed9-82ff-cec336f607f3-operator-scripts\") pod \"novacell115a6-account-delete-pbwnc\" (UID: \"f12527ba-678e-4ed9-82ff-cec336f607f3\") " pod="openstack-kuttl-tests/novacell115a6-account-delete-pbwnc" Dec 03 12:32:22 crc kubenswrapper[5052]: I1203 12:32:22.611798 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3fec3449-9615-4377-a600-69d816ec5416-kube-api-access-8dr2d" (OuterVolumeSpecName: "kube-api-access-8dr2d") pod "3fec3449-9615-4377-a600-69d816ec5416" (UID: "3fec3449-9615-4377-a600-69d816ec5416"). InnerVolumeSpecName "kube-api-access-8dr2d". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:32:22 crc kubenswrapper[5052]: E1203 12:32:22.612365 5052 projected.go:194] Error preparing data for projected volume kube-api-access-wr9nk for pod openstack-kuttl-tests/novacell115a6-account-delete-pbwnc: failed to fetch token: serviceaccounts "galera-openstack-cell1" not found Dec 03 12:32:22 crc kubenswrapper[5052]: E1203 12:32:22.612853 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-cell1-scripts: configmap "openstack-cell1-scripts" not found Dec 03 12:32:22 crc kubenswrapper[5052]: E1203 12:32:22.612890 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f12527ba-678e-4ed9-82ff-cec336f607f3-operator-scripts podName:f12527ba-678e-4ed9-82ff-cec336f607f3 nodeName:}" failed. No retries permitted until 2025-12-03 12:32:24.612876919 +0000 UTC m=+4335.201118683 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/f12527ba-678e-4ed9-82ff-cec336f607f3-operator-scripts") pod "novacell115a6-account-delete-pbwnc" (UID: "f12527ba-678e-4ed9-82ff-cec336f607f3") : configmap "openstack-cell1-scripts" not found Dec 03 12:32:22 crc kubenswrapper[5052]: E1203 12:32:22.613573 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f12527ba-678e-4ed9-82ff-cec336f607f3-kube-api-access-wr9nk podName:f12527ba-678e-4ed9-82ff-cec336f607f3 nodeName:}" failed. No retries permitted until 2025-12-03 12:32:24.613543858 +0000 UTC m=+4335.201785702 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-wr9nk" (UniqueName: "kubernetes.io/projected/f12527ba-678e-4ed9-82ff-cec336f607f3-kube-api-access-wr9nk") pod "novacell115a6-account-delete-pbwnc" (UID: "f12527ba-678e-4ed9-82ff-cec336f607f3") : failed to fetch token: serviceaccounts "galera-openstack-cell1" not found Dec 03 12:32:22 crc kubenswrapper[5052]: I1203 12:32:22.613613 5052 reconciler_common.go:293] "Volume detached for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/3fec3449-9615-4377-a600-69d816ec5416-ovsdb-rundir\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:22 crc kubenswrapper[5052]: I1203 12:32:22.613631 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3fec3449-9615-4377-a600-69d816ec5416-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:22 crc kubenswrapper[5052]: I1203 12:32:22.613642 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/3fec3449-9615-4377-a600-69d816ec5416-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:22 crc kubenswrapper[5052]: I1203 12:32:22.613679 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage08-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage08-crc\") on node \"crc\" " Dec 03 12:32:22 crc kubenswrapper[5052]: I1203 12:32:22.613692 5052 reconciler_common.go:293] "Volume detached for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/dccae7c1-009e-4b3b-b66a-ecbf6e53489a-metrics-certs-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:22 crc kubenswrapper[5052]: I1203 12:32:22.614401 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2ed8a31a-9033-4601-84bf-9eaf57f1f155-kube-api-access-dvs55" (OuterVolumeSpecName: "kube-api-access-dvs55") pod "2ed8a31a-9033-4601-84bf-9eaf57f1f155" (UID: "2ed8a31a-9033-4601-84bf-9eaf57f1f155"). InnerVolumeSpecName "kube-api-access-dvs55". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:32:22 crc kubenswrapper[5052]: I1203 12:32:22.678870 5052 scope.go:117] "RemoveContainer" containerID="e6edaf188f85ea4f11f487f11cc489c700f25fb42b5ee269f4f496f1e127d772" Dec 03 12:32:22 crc kubenswrapper[5052]: E1203 12:32:22.679286 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 12:32:22 crc kubenswrapper[5052]: I1203 12:32:22.684724 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2ed8a31a-9033-4601-84bf-9eaf57f1f155-openstack-config" (OuterVolumeSpecName: "openstack-config") pod "2ed8a31a-9033-4601-84bf-9eaf57f1f155" (UID: "2ed8a31a-9033-4601-84bf-9eaf57f1f155"). InnerVolumeSpecName "openstack-config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:32:22 crc kubenswrapper[5052]: I1203 12:32:22.704794 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage08-crc" (UniqueName: "kubernetes.io/local-volume/local-storage08-crc") on node "crc" Dec 03 12:32:22 crc kubenswrapper[5052]: I1203 12:32:22.716530 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/cc8c914a-cb83-48ae-97a7-fca144e96f52-combined-ca-bundle\") pod \"barbican-worker-c899dcc5-n6rdv\" (UID: \"cc8c914a-cb83-48ae-97a7-fca144e96f52\") " pod="openstack-kuttl-tests/barbican-worker-c899dcc5-n6rdv" Dec 03 12:32:22 crc kubenswrapper[5052]: I1203 12:32:22.716680 5052 reconciler_common.go:293] "Volume detached for volume \"openstack-config\" (UniqueName: \"kubernetes.io/configmap/2ed8a31a-9033-4601-84bf-9eaf57f1f155-openstack-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:22 crc kubenswrapper[5052]: I1203 12:32:22.716696 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-8dr2d\" (UniqueName: \"kubernetes.io/projected/3fec3449-9615-4377-a600-69d816ec5416-kube-api-access-8dr2d\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:22 crc kubenswrapper[5052]: I1203 12:32:22.716707 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-dvs55\" (UniqueName: \"kubernetes.io/projected/2ed8a31a-9033-4601-84bf-9eaf57f1f155-kube-api-access-dvs55\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:22 crc kubenswrapper[5052]: I1203 12:32:22.716716 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage08-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage08-crc\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:22 crc kubenswrapper[5052]: E1203 12:32:22.716796 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/combined-ca-bundle: secret "combined-ca-bundle" not found Dec 03 12:32:22 crc kubenswrapper[5052]: E1203 12:32:22.716841 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cc8c914a-cb83-48ae-97a7-fca144e96f52-combined-ca-bundle podName:cc8c914a-cb83-48ae-97a7-fca144e96f52 nodeName:}" failed. No retries permitted until 2025-12-03 12:32:26.716827176 +0000 UTC m=+4337.305068940 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "combined-ca-bundle" (UniqueName: "kubernetes.io/secret/cc8c914a-cb83-48ae-97a7-fca144e96f52-combined-ca-bundle") pod "barbican-worker-c899dcc5-n6rdv" (UID: "cc8c914a-cb83-48ae-97a7-fca144e96f52") : secret "combined-ca-bundle" not found Dec 03 12:32:22 crc kubenswrapper[5052]: I1203 12:32:22.809883 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinderc151-account-delete-h4k5b"] Dec 03 12:32:22 crc kubenswrapper[5052]: I1203 12:32:22.819017 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/045d578f-92c0-47c6-8356-59c489c57b9a-combined-ca-bundle\") pod \"barbican-keystone-listener-697474f89d-p7lwz\" (UID: \"045d578f-92c0-47c6-8356-59c489c57b9a\") " pod="openstack-kuttl-tests/barbican-keystone-listener-697474f89d-p7lwz" Dec 03 12:32:22 crc kubenswrapper[5052]: E1203 12:32:22.819695 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/rabbitmq-config-data: configmap "rabbitmq-config-data" not found Dec 03 12:32:22 crc kubenswrapper[5052]: E1203 12:32:22.819769 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2c03f99a-98fd-4e86-ad29-fb69e94bd436-config-data podName:2c03f99a-98fd-4e86-ad29-fb69e94bd436 nodeName:}" failed. No retries permitted until 2025-12-03 12:32:26.819749884 +0000 UTC m=+4337.407991648 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "config-data" (UniqueName: "kubernetes.io/configmap/2c03f99a-98fd-4e86-ad29-fb69e94bd436-config-data") pod "rabbitmq-server-0" (UID: "2c03f99a-98fd-4e86-ad29-fb69e94bd436") : configmap "rabbitmq-config-data" not found Dec 03 12:32:22 crc kubenswrapper[5052]: E1203 12:32:22.819948 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:32:22 crc kubenswrapper[5052]: E1203 12:32:22.820129 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/478e18e8-077e-406e-aa66-19bc1d28df46-operator-scripts podName:478e18e8-077e-406e-aa66-19bc1d28df46 nodeName:}" failed. No retries permitted until 2025-12-03 12:32:23.820093753 +0000 UTC m=+4334.408335727 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/478e18e8-077e-406e-aa66-19bc1d28df46-operator-scripts") pod "novacell046b8-account-delete-nqzlj" (UID: "478e18e8-077e-406e-aa66-19bc1d28df46") : configmap "openstack-scripts" not found Dec 03 12:32:22 crc kubenswrapper[5052]: E1203 12:32:22.820176 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/combined-ca-bundle: secret "combined-ca-bundle" not found Dec 03 12:32:22 crc kubenswrapper[5052]: E1203 12:32:22.820212 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/045d578f-92c0-47c6-8356-59c489c57b9a-combined-ca-bundle podName:045d578f-92c0-47c6-8356-59c489c57b9a nodeName:}" failed. No retries permitted until 2025-12-03 12:32:26.820203127 +0000 UTC m=+4337.408444891 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "combined-ca-bundle" (UniqueName: "kubernetes.io/secret/045d578f-92c0-47c6-8356-59c489c57b9a-combined-ca-bundle") pod "barbican-keystone-listener-697474f89d-p7lwz" (UID: "045d578f-92c0-47c6-8356-59c489c57b9a") : secret "combined-ca-bundle" not found Dec 03 12:32:22 crc kubenswrapper[5052]: I1203 12:32:22.835667 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dccae7c1-009e-4b3b-b66a-ecbf6e53489a-ovsdbserver-nb-tls-certs" (OuterVolumeSpecName: "ovsdbserver-nb-tls-certs") pod "dccae7c1-009e-4b3b-b66a-ecbf6e53489a" (UID: "dccae7c1-009e-4b3b-b66a-ecbf6e53489a"). InnerVolumeSpecName "ovsdbserver-nb-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:32:22 crc kubenswrapper[5052]: I1203 12:32:22.836879 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2ed8a31a-9033-4601-84bf-9eaf57f1f155-openstack-config-secret" (OuterVolumeSpecName: "openstack-config-secret") pod "2ed8a31a-9033-4601-84bf-9eaf57f1f155" (UID: "2ed8a31a-9033-4601-84bf-9eaf57f1f155"). InnerVolumeSpecName "openstack-config-secret". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:32:22 crc kubenswrapper[5052]: I1203 12:32:22.841553 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2ed8a31a-9033-4601-84bf-9eaf57f1f155-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "2ed8a31a-9033-4601-84bf-9eaf57f1f155" (UID: "2ed8a31a-9033-4601-84bf-9eaf57f1f155"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:32:22 crc kubenswrapper[5052]: I1203 12:32:22.852426 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3fec3449-9615-4377-a600-69d816ec5416-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "3fec3449-9615-4377-a600-69d816ec5416" (UID: "3fec3449-9615-4377-a600-69d816ec5416"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:32:22 crc kubenswrapper[5052]: I1203 12:32:22.911702 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3fec3449-9615-4377-a600-69d816ec5416-metrics-certs-tls-certs" (OuterVolumeSpecName: "metrics-certs-tls-certs") pod "3fec3449-9615-4377-a600-69d816ec5416" (UID: "3fec3449-9615-4377-a600-69d816ec5416"). InnerVolumeSpecName "metrics-certs-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:32:22 crc kubenswrapper[5052]: I1203 12:32:22.922263 5052 reconciler_common.go:293] "Volume detached for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/3fec3449-9615-4377-a600-69d816ec5416-metrics-certs-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:22 crc kubenswrapper[5052]: I1203 12:32:22.922306 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2ed8a31a-9033-4601-84bf-9eaf57f1f155-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:22 crc kubenswrapper[5052]: I1203 12:32:22.922320 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3fec3449-9615-4377-a600-69d816ec5416-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:22 crc kubenswrapper[5052]: I1203 12:32:22.922332 5052 reconciler_common.go:293] "Volume detached for volume \"openstack-config-secret\" (UniqueName: \"kubernetes.io/secret/2ed8a31a-9033-4601-84bf-9eaf57f1f155-openstack-config-secret\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:22 crc kubenswrapper[5052]: I1203 12:32:22.922347 5052 reconciler_common.go:293] "Volume detached for volume \"ovsdbserver-nb-tls-certs\" (UniqueName: \"kubernetes.io/secret/dccae7c1-009e-4b3b-b66a-ecbf6e53489a-ovsdbserver-nb-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:22 crc kubenswrapper[5052]: I1203 12:32:22.982435 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/openstack-galera-0" podUID="6938204c-1fe8-4f03-ac16-ec9e19e090f8" containerName="galera" containerID="cri-o://d94e0139dcad346a24d75f7bd285773c0fe1edad0369015fe7b299454967acdd" gracePeriod=30 Dec 03 12:32:22 crc kubenswrapper[5052]: I1203 12:32:22.989812 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3fec3449-9615-4377-a600-69d816ec5416-ovsdbserver-sb-tls-certs" (OuterVolumeSpecName: "ovsdbserver-sb-tls-certs") pod "3fec3449-9615-4377-a600-69d816ec5416" (UID: "3fec3449-9615-4377-a600-69d816ec5416"). InnerVolumeSpecName "ovsdbserver-sb-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.024372 5052 reconciler_common.go:293] "Volume detached for volume \"ovsdbserver-sb-tls-certs\" (UniqueName: \"kubernetes.io/secret/3fec3449-9615-4377-a600-69d816ec5416-ovsdbserver-sb-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.125248 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.170632 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinderc151-account-delete-h4k5b" event={"ID":"fc428005-b653-4a6a-a50d-133c2b4fb8b8","Type":"ContainerStarted","Data":"36dff8165cc2991be6c31945f380c62eb1c6a1cf48803fd50d74fface1777ef7"} Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.174660 5052 generic.go:334] "Generic (PLEG): container finished" podID="8ac6e8f0-d49c-4a4c-846a-6cec226bd360" containerID="4893c6e7800c80d6b9f1c1b9d2a79b5fd298a43e5935cb9d7b3f6072575ccfc6" exitCode=0 Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.174708 5052 generic.go:334] "Generic (PLEG): container finished" podID="8ac6e8f0-d49c-4a4c-846a-6cec226bd360" containerID="57de2e7ba65bad4f495dc483cc50efdd66d36654d3d53aadfa56507cc2faada6" exitCode=143 Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.174769 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-worker-644558fbc5-4gtxj" event={"ID":"8ac6e8f0-d49c-4a4c-846a-6cec226bd360","Type":"ContainerDied","Data":"4893c6e7800c80d6b9f1c1b9d2a79b5fd298a43e5935cb9d7b3f6072575ccfc6"} Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.174799 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-worker-644558fbc5-4gtxj" event={"ID":"8ac6e8f0-d49c-4a4c-846a-6cec226bd360","Type":"ContainerDied","Data":"57de2e7ba65bad4f495dc483cc50efdd66d36654d3d53aadfa56507cc2faada6"} Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.180731 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-keystone-listener-cd5b58cfd-jf8vt" event={"ID":"019689d4-d40b-4e9a-a028-e427f1aa9d9b","Type":"ContainerDied","Data":"30570e4c9e23e90b86c1701d30a06063dcb7b48a31496a770e386fcaa5d3ac42"} Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.180733 5052 generic.go:334] "Generic (PLEG): container finished" podID="019689d4-d40b-4e9a-a028-e427f1aa9d9b" containerID="30570e4c9e23e90b86c1701d30a06063dcb7b48a31496a770e386fcaa5d3ac42" exitCode=143 Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.188043 5052 generic.go:334] "Generic (PLEG): container finished" podID="16cac95c-d1d2-4542-bb8e-4bdc1923c684" containerID="3fd08a7accc11bddbab26e95f99cd86563dcbdee87c9faa5f4e155b8982c03de" exitCode=0 Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.188071 5052 generic.go:334] "Generic (PLEG): container finished" podID="16cac95c-d1d2-4542-bb8e-4bdc1923c684" containerID="e2917e8431b0de8f360760abc00d1838625c47e53148c72eb9f76598a47629bd" exitCode=0 Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.188110 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-proxy-55bdbb59c4-c4cjv" event={"ID":"16cac95c-d1d2-4542-bb8e-4bdc1923c684","Type":"ContainerDied","Data":"3fd08a7accc11bddbab26e95f99cd86563dcbdee87c9faa5f4e155b8982c03de"} Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.188136 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-proxy-55bdbb59c4-c4cjv" event={"ID":"16cac95c-d1d2-4542-bb8e-4bdc1923c684","Type":"ContainerDied","Data":"e2917e8431b0de8f360760abc00d1838625c47e53148c72eb9f76598a47629bd"} Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.200949 5052 generic.go:334] "Generic (PLEG): container finished" podID="9a71429c-9d59-4863-8949-dcb4f79a35e2" containerID="1313894d0c0976392a7edf9b30f67f35fdafa36110b15f07dc7f5ffd68038ba8" exitCode=0 Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.201009 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"9a71429c-9d59-4863-8949-dcb4f79a35e2","Type":"ContainerDied","Data":"1313894d0c0976392a7edf9b30f67f35fdafa36110b15f07dc7f5ffd68038ba8"} Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.202616 5052 scope.go:117] "RemoveContainer" containerID="220531c8139609a0b36f479504e8821b1f41b4615b00ee05432f9560ade3acb1" Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.202762 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstackclient" Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.205778 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_ovsdbserver-sb-0_3fec3449-9615-4377-a600-69d816ec5416/ovsdbserver-sb/0.log" Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.205872 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-sb-0" event={"ID":"3fec3449-9615-4377-a600-69d816ec5416","Type":"ContainerDied","Data":"c575e780203e439005e13481039ff4a9ef25ce1a73d62c9d8c0ce5ed6f8bd46b"} Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.205928 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.210751 5052 generic.go:334] "Generic (PLEG): container finished" podID="44520845-0277-47ab-acc6-812aba4656ff" containerID="8048ed0425b57a9a994078140f513eae158b441a12e3e7142612e180f27c9cd3" exitCode=2 Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.211107 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/kube-state-metrics-0" event={"ID":"44520845-0277-47ab-acc6-812aba4656ff","Type":"ContainerDied","Data":"8048ed0425b57a9a994078140f513eae158b441a12e3e7142612e180f27c9cd3"} Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.211149 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/kube-state-metrics-0" event={"ID":"44520845-0277-47ab-acc6-812aba4656ff","Type":"ContainerDied","Data":"fa0dca2ac03e31aa8e3e1bbce341f29f20144a5c8b32c9f9ac776cb15fc046ba"} Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.211228 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.214752 5052 generic.go:334] "Generic (PLEG): container finished" podID="d09b9817-6ea5-4328-9027-3311b7782157" containerID="7e1821a6c6bd165b882e82fb2451aca61d98f3c04110b410f0845d91b6abfa0b" exitCode=0 Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.214830 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" event={"ID":"d09b9817-6ea5-4328-9027-3311b7782157","Type":"ContainerDied","Data":"7e1821a6c6bd165b882e82fb2451aca61d98f3c04110b410f0845d91b6abfa0b"} Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.217049 5052 generic.go:334] "Generic (PLEG): container finished" podID="aa55d0c8-56de-4edd-92b3-9252f169fed8" containerID="8bf7dd5e7641b5ac8d567b37be404e10979d45b84ac534f9d893fee92ab89c5e" exitCode=0 Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.217110 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-cell1-galera-0" event={"ID":"aa55d0c8-56de-4edd-92b3-9252f169fed8","Type":"ContainerDied","Data":"8bf7dd5e7641b5ac8d567b37be404e10979d45b84ac534f9d893fee92ab89c5e"} Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.219649 5052 generic.go:334] "Generic (PLEG): container finished" podID="bb1723c7-4e41-4b0e-b6b8-f8024dafdaee" containerID="ab269ac11d99aed29d8c53741cb5bdd97ed1fe052bc19e5e9c3e0f4152eaeed4" exitCode=0 Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.219717 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"bb1723c7-4e41-4b0e-b6b8-f8024dafdaee","Type":"ContainerDied","Data":"ab269ac11d99aed29d8c53741cb5bdd97ed1fe052bc19e5e9c3e0f4152eaeed4"} Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.221267 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron2174-account-delete-9dnxs" event={"ID":"a2ef0804-75c7-43e3-95fa-ce901a561c9b","Type":"ContainerStarted","Data":"ce2b978378221fa56c56acc841ab2e8fbc8ded006c98024e04d4c568e3f73c7a"} Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.223394 5052 generic.go:334] "Generic (PLEG): container finished" podID="21690a66-4bcf-4865-91c7-cfe1cdf32a61" containerID="fb673c0dc2e5ca2f8ef44bb06008daf9443cc163a93859ac48d11b2d619a8092" exitCode=0 Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.223479 5052 generic.go:334] "Generic (PLEG): container finished" podID="21690a66-4bcf-4865-91c7-cfe1cdf32a61" containerID="3c2c53aa7eb07d8c8a4be850fb78e1b62b32812e4e50fd988ba119d0f7c3a22a" exitCode=0 Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.223597 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"21690a66-4bcf-4865-91c7-cfe1cdf32a61","Type":"ContainerDied","Data":"fb673c0dc2e5ca2f8ef44bb06008daf9443cc163a93859ac48d11b2d619a8092"} Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.223677 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"21690a66-4bcf-4865-91c7-cfe1cdf32a61","Type":"ContainerDied","Data":"3c2c53aa7eb07d8c8a4be850fb78e1b62b32812e4e50fd988ba119d0f7c3a22a"} Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.231576 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance5b9b-account-delete-ggk6q" event={"ID":"52bf0c8d-03b3-444b-80b6-13d41f5f9fb9","Type":"ContainerStarted","Data":"e4dc3d0b73a5da661e1038b2fbe8fcf9820b4b233fb38d76f37ea7b29b323d19"} Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.231875 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/44520845-0277-47ab-acc6-812aba4656ff-combined-ca-bundle\") pod \"44520845-0277-47ab-acc6-812aba4656ff\" (UID: \"44520845-0277-47ab-acc6-812aba4656ff\") " Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.232108 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ht8f4\" (UniqueName: \"kubernetes.io/projected/44520845-0277-47ab-acc6-812aba4656ff-kube-api-access-ht8f4\") pod \"44520845-0277-47ab-acc6-812aba4656ff\" (UID: \"44520845-0277-47ab-acc6-812aba4656ff\") " Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.232182 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-state-metrics-tls-config\" (UniqueName: \"kubernetes.io/secret/44520845-0277-47ab-acc6-812aba4656ff-kube-state-metrics-tls-config\") pod \"44520845-0277-47ab-acc6-812aba4656ff\" (UID: \"44520845-0277-47ab-acc6-812aba4656ff\") " Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.232209 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-state-metrics-tls-certs\" (UniqueName: \"kubernetes.io/secret/44520845-0277-47ab-acc6-812aba4656ff-kube-state-metrics-tls-certs\") pod \"44520845-0277-47ab-acc6-812aba4656ff\" (UID: \"44520845-0277-47ab-acc6-812aba4656ff\") " Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.232629 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/a3b48edc-15d0-4c5e-8a33-96bee20dd43e-internal-tls-certs\") pod \"barbican-api-78dc8cddf6-kqfz7\" (UID: \"a3b48edc-15d0-4c5e-8a33-96bee20dd43e\") " pod="openstack-kuttl-tests/barbican-api-78dc8cddf6-kqfz7" Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.232664 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a3b48edc-15d0-4c5e-8a33-96bee20dd43e-combined-ca-bundle\") pod \"barbican-api-78dc8cddf6-kqfz7\" (UID: \"a3b48edc-15d0-4c5e-8a33-96bee20dd43e\") " pod="openstack-kuttl-tests/barbican-api-78dc8cddf6-kqfz7" Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.232799 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/a3b48edc-15d0-4c5e-8a33-96bee20dd43e-public-tls-certs\") pod \"barbican-api-78dc8cddf6-kqfz7\" (UID: \"a3b48edc-15d0-4c5e-8a33-96bee20dd43e\") " pod="openstack-kuttl-tests/barbican-api-78dc8cddf6-kqfz7" Dec 03 12:32:23 crc kubenswrapper[5052]: E1203 12:32:23.233608 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-barbican-public-svc: secret "cert-barbican-public-svc" not found Dec 03 12:32:23 crc kubenswrapper[5052]: E1203 12:32:23.233675 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3b48edc-15d0-4c5e-8a33-96bee20dd43e-public-tls-certs podName:a3b48edc-15d0-4c5e-8a33-96bee20dd43e nodeName:}" failed. No retries permitted until 2025-12-03 12:32:27.233656568 +0000 UTC m=+4337.821898332 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "public-tls-certs" (UniqueName: "kubernetes.io/secret/a3b48edc-15d0-4c5e-8a33-96bee20dd43e-public-tls-certs") pod "barbican-api-78dc8cddf6-kqfz7" (UID: "a3b48edc-15d0-4c5e-8a33-96bee20dd43e") : secret "cert-barbican-public-svc" not found Dec 03 12:32:23 crc kubenswrapper[5052]: E1203 12:32:23.239962 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/combined-ca-bundle: secret "combined-ca-bundle" not found Dec 03 12:32:23 crc kubenswrapper[5052]: E1203 12:32:23.240035 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3b48edc-15d0-4c5e-8a33-96bee20dd43e-combined-ca-bundle podName:a3b48edc-15d0-4c5e-8a33-96bee20dd43e nodeName:}" failed. No retries permitted until 2025-12-03 12:32:27.240016539 +0000 UTC m=+4337.828258303 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "combined-ca-bundle" (UniqueName: "kubernetes.io/secret/a3b48edc-15d0-4c5e-8a33-96bee20dd43e-combined-ca-bundle") pod "barbican-api-78dc8cddf6-kqfz7" (UID: "a3b48edc-15d0-4c5e-8a33-96bee20dd43e") : secret "combined-ca-bundle" not found Dec 03 12:32:23 crc kubenswrapper[5052]: E1203 12:32:23.240076 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-barbican-internal-svc: secret "cert-barbican-internal-svc" not found Dec 03 12:32:23 crc kubenswrapper[5052]: E1203 12:32:23.240103 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3b48edc-15d0-4c5e-8a33-96bee20dd43e-internal-tls-certs podName:a3b48edc-15d0-4c5e-8a33-96bee20dd43e nodeName:}" failed. No retries permitted until 2025-12-03 12:32:27.240094621 +0000 UTC m=+4337.828336385 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "internal-tls-certs" (UniqueName: "kubernetes.io/secret/a3b48edc-15d0-4c5e-8a33-96bee20dd43e-internal-tls-certs") pod "barbican-api-78dc8cddf6-kqfz7" (UID: "a3b48edc-15d0-4c5e-8a33-96bee20dd43e") : secret "cert-barbican-internal-svc" not found Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.241265 5052 scope.go:117] "RemoveContainer" containerID="21da3337b09e6b326697aef155af4bcd6a984d400eaed99ee26b1a3a045fba72" Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.244609 5052 generic.go:334] "Generic (PLEG): container finished" podID="82f142ab-5be9-4727-b3c5-102d6f2401fb" containerID="ca30ccff84d103c471a54af8538ca8e1d53df3754e5f8a3e9c4ab0869c3a23d6" exitCode=143 Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.244680 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-68fc8f9776-n9gml" event={"ID":"82f142ab-5be9-4727-b3c5-102d6f2401fb","Type":"ContainerDied","Data":"ca30ccff84d103c471a54af8538ca8e1d53df3754e5f8a3e9c4ab0869c3a23d6"} Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.255102 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-worker-c899dcc5-n6rdv" Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.256086 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-keystone-listener-697474f89d-p7lwz" Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.256243 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/novacell115a6-account-delete-pbwnc" Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.259328 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placemente65a-account-delete-s7x52" event={"ID":"ae9024f5-a2b4-49de-aed3-00cb9c806704","Type":"ContainerStarted","Data":"96cd7d92781623b5aa19e13255fcc36f068db61739a1f3c2145635b5d11cc87a"} Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.259388 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placemente65a-account-delete-s7x52" event={"ID":"ae9024f5-a2b4-49de-aed3-00cb9c806704","Type":"ContainerStarted","Data":"560d2d91b6fbb85cac7eccb7681c0db9b995668b33f0e3c26747232ed5e22999"} Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.259473 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.260059 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-api-78dc8cddf6-kqfz7" Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.260181 5052 kubelet_pods.go:1007] "Unable to retrieve pull secret, the image pull may not succeed." pod="openstack-kuttl-tests/placemente65a-account-delete-s7x52" secret="" err="secret \"galera-openstack-dockercfg-w7gzp\" not found" Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.270990 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-sb-0"] Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.277978 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/44520845-0277-47ab-acc6-812aba4656ff-kube-api-access-ht8f4" (OuterVolumeSpecName: "kube-api-access-ht8f4") pod "44520845-0277-47ab-acc6-812aba4656ff" (UID: "44520845-0277-47ab-acc6-812aba4656ff"). InnerVolumeSpecName "kube-api-access-ht8f4". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.286801 5052 scope.go:117] "RemoveContainer" containerID="19c04c4116df77c2accd0a6c361be38550f970c0c0f8032ebf2cd11805800978" Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.290445 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-worker-c899dcc5-n6rdv" Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.292206 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-keystone-listener-697474f89d-p7lwz" Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.296302 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-sb-0"] Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.308739 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/placemente65a-account-delete-s7x52" podStartSLOduration=4.30871382 podStartE2EDuration="4.30871382s" podCreationTimestamp="2025-12-03 12:32:19 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:32:23.301911186 +0000 UTC m=+4333.890152960" watchObservedRunningTime="2025-12-03 12:32:23.30871382 +0000 UTC m=+4333.896955584" Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.323861 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/novacell115a6-account-delete-pbwnc" Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.331958 5052 scope.go:117] "RemoveContainer" containerID="8048ed0425b57a9a994078140f513eae158b441a12e3e7142612e180f27c9cd3" Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.337808 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-ht8f4\" (UniqueName: \"kubernetes.io/projected/44520845-0277-47ab-acc6-812aba4656ff-kube-api-access-ht8f4\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:23 crc kubenswrapper[5052]: E1203 12:32:23.337925 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:32:23 crc kubenswrapper[5052]: E1203 12:32:23.338015 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ae9024f5-a2b4-49de-aed3-00cb9c806704-operator-scripts podName:ae9024f5-a2b4-49de-aed3-00cb9c806704 nodeName:}" failed. No retries permitted until 2025-12-03 12:32:23.837992686 +0000 UTC m=+4334.426234450 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/ae9024f5-a2b4-49de-aed3-00cb9c806704-operator-scripts") pod "placemente65a-account-delete-s7x52" (UID: "ae9024f5-a2b4-49de-aed3-00cb9c806704") : configmap "openstack-scripts" not found Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.340704 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/44520845-0277-47ab-acc6-812aba4656ff-kube-state-metrics-tls-certs" (OuterVolumeSpecName: "kube-state-metrics-tls-certs") pod "44520845-0277-47ab-acc6-812aba4656ff" (UID: "44520845-0277-47ab-acc6-812aba4656ff"). InnerVolumeSpecName "kube-state-metrics-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.343148 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/44520845-0277-47ab-acc6-812aba4656ff-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "44520845-0277-47ab-acc6-812aba4656ff" (UID: "44520845-0277-47ab-acc6-812aba4656ff"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.351203 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-api-78dc8cddf6-kqfz7" Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.356730 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-nb-0"] Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.365717 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-nb-0"] Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.378264 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/44520845-0277-47ab-acc6-812aba4656ff-kube-state-metrics-tls-config" (OuterVolumeSpecName: "kube-state-metrics-tls-config") pod "44520845-0277-47ab-acc6-812aba4656ff" (UID: "44520845-0277-47ab-acc6-812aba4656ff"). InnerVolumeSpecName "kube-state-metrics-tls-config". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.378875 5052 scope.go:117] "RemoveContainer" containerID="8048ed0425b57a9a994078140f513eae158b441a12e3e7142612e180f27c9cd3" Dec 03 12:32:23 crc kubenswrapper[5052]: E1203 12:32:23.380175 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8048ed0425b57a9a994078140f513eae158b441a12e3e7142612e180f27c9cd3\": container with ID starting with 8048ed0425b57a9a994078140f513eae158b441a12e3e7142612e180f27c9cd3 not found: ID does not exist" containerID="8048ed0425b57a9a994078140f513eae158b441a12e3e7142612e180f27c9cd3" Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.380253 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8048ed0425b57a9a994078140f513eae158b441a12e3e7142612e180f27c9cd3"} err="failed to get container status \"8048ed0425b57a9a994078140f513eae158b441a12e3e7142612e180f27c9cd3\": rpc error: code = NotFound desc = could not find container \"8048ed0425b57a9a994078140f513eae158b441a12e3e7142612e180f27c9cd3\": container with ID starting with 8048ed0425b57a9a994078140f513eae158b441a12e3e7142612e180f27c9cd3 not found: ID does not exist" Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.445197 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/a3b48edc-15d0-4c5e-8a33-96bee20dd43e-logs\") pod \"a3b48edc-15d0-4c5e-8a33-96bee20dd43e\" (UID: \"a3b48edc-15d0-4c5e-8a33-96bee20dd43e\") " Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.445257 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a3b48edc-15d0-4c5e-8a33-96bee20dd43e-config-data\") pod \"a3b48edc-15d0-4c5e-8a33-96bee20dd43e\" (UID: \"a3b48edc-15d0-4c5e-8a33-96bee20dd43e\") " Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.445305 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/a3b48edc-15d0-4c5e-8a33-96bee20dd43e-config-data-custom\") pod \"a3b48edc-15d0-4c5e-8a33-96bee20dd43e\" (UID: \"a3b48edc-15d0-4c5e-8a33-96bee20dd43e\") " Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.445452 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fnt2q\" (UniqueName: \"kubernetes.io/projected/a3b48edc-15d0-4c5e-8a33-96bee20dd43e-kube-api-access-fnt2q\") pod \"a3b48edc-15d0-4c5e-8a33-96bee20dd43e\" (UID: \"a3b48edc-15d0-4c5e-8a33-96bee20dd43e\") " Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.445528 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/045d578f-92c0-47c6-8356-59c489c57b9a-logs\") pod \"045d578f-92c0-47c6-8356-59c489c57b9a\" (UID: \"045d578f-92c0-47c6-8356-59c489c57b9a\") " Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.445568 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lflxx\" (UniqueName: \"kubernetes.io/projected/045d578f-92c0-47c6-8356-59c489c57b9a-kube-api-access-lflxx\") pod \"045d578f-92c0-47c6-8356-59c489c57b9a\" (UID: \"045d578f-92c0-47c6-8356-59c489c57b9a\") " Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.445615 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/cc8c914a-cb83-48ae-97a7-fca144e96f52-config-data-custom\") pod \"cc8c914a-cb83-48ae-97a7-fca144e96f52\" (UID: \"cc8c914a-cb83-48ae-97a7-fca144e96f52\") " Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.445665 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/cc8c914a-cb83-48ae-97a7-fca144e96f52-logs\") pod \"cc8c914a-cb83-48ae-97a7-fca144e96f52\" (UID: \"cc8c914a-cb83-48ae-97a7-fca144e96f52\") " Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.445697 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/045d578f-92c0-47c6-8356-59c489c57b9a-config-data-custom\") pod \"045d578f-92c0-47c6-8356-59c489c57b9a\" (UID: \"045d578f-92c0-47c6-8356-59c489c57b9a\") " Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.445725 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tdw6g\" (UniqueName: \"kubernetes.io/projected/cc8c914a-cb83-48ae-97a7-fca144e96f52-kube-api-access-tdw6g\") pod \"cc8c914a-cb83-48ae-97a7-fca144e96f52\" (UID: \"cc8c914a-cb83-48ae-97a7-fca144e96f52\") " Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.445749 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/cc8c914a-cb83-48ae-97a7-fca144e96f52-config-data\") pod \"cc8c914a-cb83-48ae-97a7-fca144e96f52\" (UID: \"cc8c914a-cb83-48ae-97a7-fca144e96f52\") " Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.445862 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/045d578f-92c0-47c6-8356-59c489c57b9a-config-data\") pod \"045d578f-92c0-47c6-8356-59c489c57b9a\" (UID: \"045d578f-92c0-47c6-8356-59c489c57b9a\") " Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.446498 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/44520845-0277-47ab-acc6-812aba4656ff-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.446543 5052 reconciler_common.go:293] "Volume detached for volume \"kube-state-metrics-tls-config\" (UniqueName: \"kubernetes.io/secret/44520845-0277-47ab-acc6-812aba4656ff-kube-state-metrics-tls-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.446559 5052 reconciler_common.go:293] "Volume detached for volume \"kube-state-metrics-tls-certs\" (UniqueName: \"kubernetes.io/secret/44520845-0277-47ab-acc6-812aba4656ff-kube-state-metrics-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:23 crc kubenswrapper[5052]: E1203 12:32:23.446646 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cinder-config-data: secret "cinder-config-data" not found Dec 03 12:32:23 crc kubenswrapper[5052]: E1203 12:32:23.446703 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bb1723c7-4e41-4b0e-b6b8-f8024dafdaee-config-data podName:bb1723c7-4e41-4b0e-b6b8-f8024dafdaee nodeName:}" failed. No retries permitted until 2025-12-03 12:32:27.446685748 +0000 UTC m=+4338.034927522 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "config-data" (UniqueName: "kubernetes.io/secret/bb1723c7-4e41-4b0e-b6b8-f8024dafdaee-config-data") pod "cinder-scheduler-0" (UID: "bb1723c7-4e41-4b0e-b6b8-f8024dafdaee") : secret "cinder-config-data" not found Dec 03 12:32:23 crc kubenswrapper[5052]: E1203 12:32:23.448464 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cinder-scripts: secret "cinder-scripts" not found Dec 03 12:32:23 crc kubenswrapper[5052]: E1203 12:32:23.449914 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bb1723c7-4e41-4b0e-b6b8-f8024dafdaee-scripts podName:bb1723c7-4e41-4b0e-b6b8-f8024dafdaee nodeName:}" failed. No retries permitted until 2025-12-03 12:32:27.449890359 +0000 UTC m=+4338.038132193 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "scripts" (UniqueName: "kubernetes.io/secret/bb1723c7-4e41-4b0e-b6b8-f8024dafdaee-scripts") pod "cinder-scheduler-0" (UID: "bb1723c7-4e41-4b0e-b6b8-f8024dafdaee") : secret "cinder-scripts" not found Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.451500 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cc8c914a-cb83-48ae-97a7-fca144e96f52-logs" (OuterVolumeSpecName: "logs") pod "cc8c914a-cb83-48ae-97a7-fca144e96f52" (UID: "cc8c914a-cb83-48ae-97a7-fca144e96f52"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:32:23 crc kubenswrapper[5052]: E1203 12:32:23.452425 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/combined-ca-bundle: secret "combined-ca-bundle" not found Dec 03 12:32:23 crc kubenswrapper[5052]: E1203 12:32:23.452506 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bb1723c7-4e41-4b0e-b6b8-f8024dafdaee-combined-ca-bundle podName:bb1723c7-4e41-4b0e-b6b8-f8024dafdaee nodeName:}" failed. No retries permitted until 2025-12-03 12:32:27.452483464 +0000 UTC m=+4338.040725298 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "combined-ca-bundle" (UniqueName: "kubernetes.io/secret/bb1723c7-4e41-4b0e-b6b8-f8024dafdaee-combined-ca-bundle") pod "cinder-scheduler-0" (UID: "bb1723c7-4e41-4b0e-b6b8-f8024dafdaee") : secret "combined-ca-bundle" not found Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.453114 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/045d578f-92c0-47c6-8356-59c489c57b9a-config-data-custom" (OuterVolumeSpecName: "config-data-custom") pod "045d578f-92c0-47c6-8356-59c489c57b9a" (UID: "045d578f-92c0-47c6-8356-59c489c57b9a"). InnerVolumeSpecName "config-data-custom". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.448781 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3b48edc-15d0-4c5e-8a33-96bee20dd43e-logs" (OuterVolumeSpecName: "logs") pod "a3b48edc-15d0-4c5e-8a33-96bee20dd43e" (UID: "a3b48edc-15d0-4c5e-8a33-96bee20dd43e"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.449015 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/045d578f-92c0-47c6-8356-59c489c57b9a-logs" (OuterVolumeSpecName: "logs") pod "045d578f-92c0-47c6-8356-59c489c57b9a" (UID: "045d578f-92c0-47c6-8356-59c489c57b9a"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:32:23 crc kubenswrapper[5052]: E1203 12:32:23.455591 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/rabbitmq-cell1-config-data: configmap "rabbitmq-cell1-config-data" not found Dec 03 12:32:23 crc kubenswrapper[5052]: E1203 12:32:23.455645 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b8613a94-1e2e-477f-8d52-713c70f5282c-config-data podName:b8613a94-1e2e-477f-8d52-713c70f5282c nodeName:}" failed. No retries permitted until 2025-12-03 12:32:27.455627403 +0000 UTC m=+4338.043869167 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "config-data" (UniqueName: "kubernetes.io/configmap/b8613a94-1e2e-477f-8d52-713c70f5282c-config-data") pod "rabbitmq-cell1-server-0" (UID: "b8613a94-1e2e-477f-8d52-713c70f5282c") : configmap "rabbitmq-cell1-config-data" not found Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.461835 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/045d578f-92c0-47c6-8356-59c489c57b9a-kube-api-access-lflxx" (OuterVolumeSpecName: "kube-api-access-lflxx") pod "045d578f-92c0-47c6-8356-59c489c57b9a" (UID: "045d578f-92c0-47c6-8356-59c489c57b9a"). InnerVolumeSpecName "kube-api-access-lflxx". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.462528 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cc8c914a-cb83-48ae-97a7-fca144e96f52-config-data-custom" (OuterVolumeSpecName: "config-data-custom") pod "cc8c914a-cb83-48ae-97a7-fca144e96f52" (UID: "cc8c914a-cb83-48ae-97a7-fca144e96f52"). InnerVolumeSpecName "config-data-custom". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.472711 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/045d578f-92c0-47c6-8356-59c489c57b9a-config-data" (OuterVolumeSpecName: "config-data") pod "045d578f-92c0-47c6-8356-59c489c57b9a" (UID: "045d578f-92c0-47c6-8356-59c489c57b9a"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.472744 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a3b48edc-15d0-4c5e-8a33-96bee20dd43e-kube-api-access-fnt2q" (OuterVolumeSpecName: "kube-api-access-fnt2q") pod "a3b48edc-15d0-4c5e-8a33-96bee20dd43e" (UID: "a3b48edc-15d0-4c5e-8a33-96bee20dd43e"). InnerVolumeSpecName "kube-api-access-fnt2q". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.472717 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cc8c914a-cb83-48ae-97a7-fca144e96f52-kube-api-access-tdw6g" (OuterVolumeSpecName: "kube-api-access-tdw6g") pod "cc8c914a-cb83-48ae-97a7-fca144e96f52" (UID: "cc8c914a-cb83-48ae-97a7-fca144e96f52"). InnerVolumeSpecName "kube-api-access-tdw6g". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.479810 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cc8c914a-cb83-48ae-97a7-fca144e96f52-config-data" (OuterVolumeSpecName: "config-data") pod "cc8c914a-cb83-48ae-97a7-fca144e96f52" (UID: "cc8c914a-cb83-48ae-97a7-fca144e96f52"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.479940 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a3b48edc-15d0-4c5e-8a33-96bee20dd43e-config-data" (OuterVolumeSpecName: "config-data") pod "a3b48edc-15d0-4c5e-8a33-96bee20dd43e" (UID: "a3b48edc-15d0-4c5e-8a33-96bee20dd43e"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.492230 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a3b48edc-15d0-4c5e-8a33-96bee20dd43e-config-data-custom" (OuterVolumeSpecName: "config-data-custom") pod "a3b48edc-15d0-4c5e-8a33-96bee20dd43e" (UID: "a3b48edc-15d0-4c5e-8a33-96bee20dd43e"). InnerVolumeSpecName "config-data-custom". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.548086 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-fnt2q\" (UniqueName: \"kubernetes.io/projected/a3b48edc-15d0-4c5e-8a33-96bee20dd43e-kube-api-access-fnt2q\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.548114 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/045d578f-92c0-47c6-8356-59c489c57b9a-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.548123 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-lflxx\" (UniqueName: \"kubernetes.io/projected/045d578f-92c0-47c6-8356-59c489c57b9a-kube-api-access-lflxx\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.548134 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/cc8c914a-cb83-48ae-97a7-fca144e96f52-config-data-custom\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.548143 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/cc8c914a-cb83-48ae-97a7-fca144e96f52-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.548151 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-tdw6g\" (UniqueName: \"kubernetes.io/projected/cc8c914a-cb83-48ae-97a7-fca144e96f52-kube-api-access-tdw6g\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.548160 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/045d578f-92c0-47c6-8356-59c489c57b9a-config-data-custom\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.548168 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/cc8c914a-cb83-48ae-97a7-fca144e96f52-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.548176 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/045d578f-92c0-47c6-8356-59c489c57b9a-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.548187 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/a3b48edc-15d0-4c5e-8a33-96bee20dd43e-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.548215 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a3b48edc-15d0-4c5e-8a33-96bee20dd43e-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.548224 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/a3b48edc-15d0-4c5e-8a33-96bee20dd43e-config-data-custom\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:23 crc kubenswrapper[5052]: E1203 12:32:23.548338 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="217fb246797b6f91c28f31f22b498303e560c0706448ba85948bf67b547612e3" cmd=["/usr/local/bin/container-scripts/status_check.sh"] Dec 03 12:32:23 crc kubenswrapper[5052]: E1203 12:32:23.573239 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="217fb246797b6f91c28f31f22b498303e560c0706448ba85948bf67b547612e3" cmd=["/usr/local/bin/container-scripts/status_check.sh"] Dec 03 12:32:23 crc kubenswrapper[5052]: E1203 12:32:23.575868 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="217fb246797b6f91c28f31f22b498303e560c0706448ba85948bf67b547612e3" cmd=["/usr/local/bin/container-scripts/status_check.sh"] Dec 03 12:32:23 crc kubenswrapper[5052]: E1203 12:32:23.575973 5052 prober.go:104] "Probe errored" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" probeType="Readiness" pod="openstack-kuttl-tests/ovn-northd-0" podUID="a9f65397-a08a-4798-8532-646209c5d090" containerName="ovn-northd" Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.583104 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/kube-state-metrics-0"] Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.590579 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/kube-state-metrics-0"] Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.598481 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.611327 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-proxy-55bdbb59c4-c4cjv" Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.652653 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-worker-644558fbc5-4gtxj" Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.668285 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.704491 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2ed8a31a-9033-4601-84bf-9eaf57f1f155" path="/var/lib/kubelet/pods/2ed8a31a-9033-4601-84bf-9eaf57f1f155/volumes" Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.705284 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="37da25a2-1fa7-4a14-b8d6-1ba47412c6cd" path="/var/lib/kubelet/pods/37da25a2-1fa7-4a14-b8d6-1ba47412c6cd/volumes" Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.706112 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3fec3449-9615-4377-a600-69d816ec5416" path="/var/lib/kubelet/pods/3fec3449-9615-4377-a600-69d816ec5416/volumes" Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.707451 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="44520845-0277-47ab-acc6-812aba4656ff" path="/var/lib/kubelet/pods/44520845-0277-47ab-acc6-812aba4656ff/volumes" Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.708116 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="47087a5b-7fd2-498e-b9d0-0bc3c6939637" path="/var/lib/kubelet/pods/47087a5b-7fd2-498e-b9d0-0bc3c6939637/volumes" Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.712250 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5bbad3a2-f21b-4ce1-8e37-5cf432d806ef" path="/var/lib/kubelet/pods/5bbad3a2-f21b-4ce1-8e37-5cf432d806ef/volumes" Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.719623 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8ab8eb50-664d-4a76-b74c-c043f8891dce" path="/var/lib/kubelet/pods/8ab8eb50-664d-4a76-b74c-c043f8891dce/volumes" Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.720205 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="beb9c625-5acb-4d1e-9044-7fb5287e8cfe" path="/var/lib/kubelet/pods/beb9c625-5acb-4d1e-9044-7fb5287e8cfe/volumes" Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.720931 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dccae7c1-009e-4b3b-b66a-ecbf6e53489a" path="/var/lib/kubelet/pods/dccae7c1-009e-4b3b-b66a-ecbf6e53489a/volumes" Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.722047 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ee394270-24f0-4687-9129-a5937efe9826" path="/var/lib/kubelet/pods/ee394270-24f0-4687-9129-a5937efe9826/volumes" Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.769630 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/aa55d0c8-56de-4edd-92b3-9252f169fed8-combined-ca-bundle\") pod \"aa55d0c8-56de-4edd-92b3-9252f169fed8\" (UID: \"aa55d0c8-56de-4edd-92b3-9252f169fed8\") " Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.769700 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/16cac95c-d1d2-4542-bb8e-4bdc1923c684-log-httpd\") pod \"16cac95c-d1d2-4542-bb8e-4bdc1923c684\" (UID: \"16cac95c-d1d2-4542-bb8e-4bdc1923c684\") " Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.769796 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"vencrypt-tls-certs\" (UniqueName: \"kubernetes.io/secret/d09b9817-6ea5-4328-9027-3311b7782157-vencrypt-tls-certs\") pod \"d09b9817-6ea5-4328-9027-3311b7782157\" (UID: \"d09b9817-6ea5-4328-9027-3311b7782157\") " Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.769833 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d09b9817-6ea5-4328-9027-3311b7782157-combined-ca-bundle\") pod \"d09b9817-6ea5-4328-9027-3311b7782157\" (UID: \"d09b9817-6ea5-4328-9027-3311b7782157\") " Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.769867 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/d09b9817-6ea5-4328-9027-3311b7782157-config-data\") pod \"d09b9817-6ea5-4328-9027-3311b7782157\" (UID: \"d09b9817-6ea5-4328-9027-3311b7782157\") " Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.769898 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/aa55d0c8-56de-4edd-92b3-9252f169fed8-config-data-generated\") pod \"aa55d0c8-56de-4edd-92b3-9252f169fed8\" (UID: \"aa55d0c8-56de-4edd-92b3-9252f169fed8\") " Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.769925 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/16cac95c-d1d2-4542-bb8e-4bdc1923c684-config-data\") pod \"16cac95c-d1d2-4542-bb8e-4bdc1923c684\" (UID: \"16cac95c-d1d2-4542-bb8e-4bdc1923c684\") " Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.769951 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/16cac95c-d1d2-4542-bb8e-4bdc1923c684-combined-ca-bundle\") pod \"16cac95c-d1d2-4542-bb8e-4bdc1923c684\" (UID: \"16cac95c-d1d2-4542-bb8e-4bdc1923c684\") " Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.769984 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hhjkq\" (UniqueName: \"kubernetes.io/projected/d09b9817-6ea5-4328-9027-3311b7782157-kube-api-access-hhjkq\") pod \"d09b9817-6ea5-4328-9027-3311b7782157\" (UID: \"d09b9817-6ea5-4328-9027-3311b7782157\") " Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.770050 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/8ac6e8f0-d49c-4a4c-846a-6cec226bd360-config-data\") pod \"8ac6e8f0-d49c-4a4c-846a-6cec226bd360\" (UID: \"8ac6e8f0-d49c-4a4c-846a-6cec226bd360\") " Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.770086 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/aa55d0c8-56de-4edd-92b3-9252f169fed8-kolla-config\") pod \"aa55d0c8-56de-4edd-92b3-9252f169fed8\" (UID: \"aa55d0c8-56de-4edd-92b3-9252f169fed8\") " Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.770139 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-94jwv\" (UniqueName: \"kubernetes.io/projected/aa55d0c8-56de-4edd-92b3-9252f169fed8-kube-api-access-94jwv\") pod \"aa55d0c8-56de-4edd-92b3-9252f169fed8\" (UID: \"aa55d0c8-56de-4edd-92b3-9252f169fed8\") " Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.770170 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/8ac6e8f0-d49c-4a4c-846a-6cec226bd360-config-data-custom\") pod \"8ac6e8f0-d49c-4a4c-846a-6cec226bd360\" (UID: \"8ac6e8f0-d49c-4a4c-846a-6cec226bd360\") " Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.770209 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/16cac95c-d1d2-4542-bb8e-4bdc1923c684-internal-tls-certs\") pod \"16cac95c-d1d2-4542-bb8e-4bdc1923c684\" (UID: \"16cac95c-d1d2-4542-bb8e-4bdc1923c684\") " Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.770238 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/16cac95c-d1d2-4542-bb8e-4bdc1923c684-etc-swift\") pod \"16cac95c-d1d2-4542-bb8e-4bdc1923c684\" (UID: \"16cac95c-d1d2-4542-bb8e-4bdc1923c684\") " Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.770290 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/aa55d0c8-56de-4edd-92b3-9252f169fed8-galera-tls-certs\") pod \"aa55d0c8-56de-4edd-92b3-9252f169fed8\" (UID: \"aa55d0c8-56de-4edd-92b3-9252f169fed8\") " Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.770330 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/16cac95c-d1d2-4542-bb8e-4bdc1923c684-run-httpd\") pod \"16cac95c-d1d2-4542-bb8e-4bdc1923c684\" (UID: \"16cac95c-d1d2-4542-bb8e-4bdc1923c684\") " Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.770360 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s4mt4\" (UniqueName: \"kubernetes.io/projected/16cac95c-d1d2-4542-bb8e-4bdc1923c684-kube-api-access-s4mt4\") pod \"16cac95c-d1d2-4542-bb8e-4bdc1923c684\" (UID: \"16cac95c-d1d2-4542-bb8e-4bdc1923c684\") " Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.770407 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/aa55d0c8-56de-4edd-92b3-9252f169fed8-operator-scripts\") pod \"aa55d0c8-56de-4edd-92b3-9252f169fed8\" (UID: \"aa55d0c8-56de-4edd-92b3-9252f169fed8\") " Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.770427 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8ac6e8f0-d49c-4a4c-846a-6cec226bd360-combined-ca-bundle\") pod \"8ac6e8f0-d49c-4a4c-846a-6cec226bd360\" (UID: \"8ac6e8f0-d49c-4a4c-846a-6cec226bd360\") " Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.770457 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"nova-novncproxy-tls-certs\" (UniqueName: \"kubernetes.io/secret/d09b9817-6ea5-4328-9027-3311b7782157-nova-novncproxy-tls-certs\") pod \"d09b9817-6ea5-4328-9027-3311b7782157\" (UID: \"d09b9817-6ea5-4328-9027-3311b7782157\") " Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.770479 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/16cac95c-d1d2-4542-bb8e-4bdc1923c684-public-tls-certs\") pod \"16cac95c-d1d2-4542-bb8e-4bdc1923c684\" (UID: \"16cac95c-d1d2-4542-bb8e-4bdc1923c684\") " Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.770543 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"mysql-db\" (UniqueName: \"kubernetes.io/local-volume/local-storage05-crc\") pod \"aa55d0c8-56de-4edd-92b3-9252f169fed8\" (UID: \"aa55d0c8-56de-4edd-92b3-9252f169fed8\") " Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.770581 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/8ac6e8f0-d49c-4a4c-846a-6cec226bd360-logs\") pod \"8ac6e8f0-d49c-4a4c-846a-6cec226bd360\" (UID: \"8ac6e8f0-d49c-4a4c-846a-6cec226bd360\") " Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.770613 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fqrfd\" (UniqueName: \"kubernetes.io/projected/8ac6e8f0-d49c-4a4c-846a-6cec226bd360-kube-api-access-fqrfd\") pod \"8ac6e8f0-d49c-4a4c-846a-6cec226bd360\" (UID: \"8ac6e8f0-d49c-4a4c-846a-6cec226bd360\") " Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.770645 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/aa55d0c8-56de-4edd-92b3-9252f169fed8-config-data-default\") pod \"aa55d0c8-56de-4edd-92b3-9252f169fed8\" (UID: \"aa55d0c8-56de-4edd-92b3-9252f169fed8\") " Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.772338 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aa55d0c8-56de-4edd-92b3-9252f169fed8-config-data-default" (OuterVolumeSpecName: "config-data-default") pod "aa55d0c8-56de-4edd-92b3-9252f169fed8" (UID: "aa55d0c8-56de-4edd-92b3-9252f169fed8"). InnerVolumeSpecName "config-data-default". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.772853 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/16cac95c-d1d2-4542-bb8e-4bdc1923c684-run-httpd" (OuterVolumeSpecName: "run-httpd") pod "16cac95c-d1d2-4542-bb8e-4bdc1923c684" (UID: "16cac95c-d1d2-4542-bb8e-4bdc1923c684"). InnerVolumeSpecName "run-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.778604 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aa55d0c8-56de-4edd-92b3-9252f169fed8-kolla-config" (OuterVolumeSpecName: "kolla-config") pod "aa55d0c8-56de-4edd-92b3-9252f169fed8" (UID: "aa55d0c8-56de-4edd-92b3-9252f169fed8"). InnerVolumeSpecName "kolla-config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.778955 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8ac6e8f0-d49c-4a4c-846a-6cec226bd360-logs" (OuterVolumeSpecName: "logs") pod "8ac6e8f0-d49c-4a4c-846a-6cec226bd360" (UID: "8ac6e8f0-d49c-4a4c-846a-6cec226bd360"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.782563 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d09b9817-6ea5-4328-9027-3311b7782157-kube-api-access-hhjkq" (OuterVolumeSpecName: "kube-api-access-hhjkq") pod "d09b9817-6ea5-4328-9027-3311b7782157" (UID: "d09b9817-6ea5-4328-9027-3311b7782157"). InnerVolumeSpecName "kube-api-access-hhjkq". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.788564 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/16cac95c-d1d2-4542-bb8e-4bdc1923c684-log-httpd" (OuterVolumeSpecName: "log-httpd") pod "16cac95c-d1d2-4542-bb8e-4bdc1923c684" (UID: "16cac95c-d1d2-4542-bb8e-4bdc1923c684"). InnerVolumeSpecName "log-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.788995 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa55d0c8-56de-4edd-92b3-9252f169fed8-config-data-generated" (OuterVolumeSpecName: "config-data-generated") pod "aa55d0c8-56de-4edd-92b3-9252f169fed8" (UID: "aa55d0c8-56de-4edd-92b3-9252f169fed8"). InnerVolumeSpecName "config-data-generated". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.789171 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aa55d0c8-56de-4edd-92b3-9252f169fed8-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "aa55d0c8-56de-4edd-92b3-9252f169fed8" (UID: "aa55d0c8-56de-4edd-92b3-9252f169fed8"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.790077 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8ac6e8f0-d49c-4a4c-846a-6cec226bd360-kube-api-access-fqrfd" (OuterVolumeSpecName: "kube-api-access-fqrfd") pod "8ac6e8f0-d49c-4a4c-846a-6cec226bd360" (UID: "8ac6e8f0-d49c-4a4c-846a-6cec226bd360"). InnerVolumeSpecName "kube-api-access-fqrfd". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.798827 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8ac6e8f0-d49c-4a4c-846a-6cec226bd360-config-data-custom" (OuterVolumeSpecName: "config-data-custom") pod "8ac6e8f0-d49c-4a4c-846a-6cec226bd360" (UID: "8ac6e8f0-d49c-4a4c-846a-6cec226bd360"). InnerVolumeSpecName "config-data-custom". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.809320 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage05-crc" (OuterVolumeSpecName: "mysql-db") pod "aa55d0c8-56de-4edd-92b3-9252f169fed8" (UID: "aa55d0c8-56de-4edd-92b3-9252f169fed8"). InnerVolumeSpecName "local-storage05-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.814775 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aa55d0c8-56de-4edd-92b3-9252f169fed8-kube-api-access-94jwv" (OuterVolumeSpecName: "kube-api-access-94jwv") pod "aa55d0c8-56de-4edd-92b3-9252f169fed8" (UID: "aa55d0c8-56de-4edd-92b3-9252f169fed8"). InnerVolumeSpecName "kube-api-access-94jwv". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.818022 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/16cac95c-d1d2-4542-bb8e-4bdc1923c684-etc-swift" (OuterVolumeSpecName: "etc-swift") pod "16cac95c-d1d2-4542-bb8e-4bdc1923c684" (UID: "16cac95c-d1d2-4542-bb8e-4bdc1923c684"). InnerVolumeSpecName "etc-swift". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.862305 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d09b9817-6ea5-4328-9027-3311b7782157-config-data" (OuterVolumeSpecName: "config-data") pod "d09b9817-6ea5-4328-9027-3311b7782157" (UID: "d09b9817-6ea5-4328-9027-3311b7782157"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.862860 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/16cac95c-d1d2-4542-bb8e-4bdc1923c684-kube-api-access-s4mt4" (OuterVolumeSpecName: "kube-api-access-s4mt4") pod "16cac95c-d1d2-4542-bb8e-4bdc1923c684" (UID: "16cac95c-d1d2-4542-bb8e-4bdc1923c684"). InnerVolumeSpecName "kube-api-access-s4mt4". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:32:23 crc kubenswrapper[5052]: E1203 12:32:23.873281 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:32:23 crc kubenswrapper[5052]: E1203 12:32:23.873356 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/478e18e8-077e-406e-aa66-19bc1d28df46-operator-scripts podName:478e18e8-077e-406e-aa66-19bc1d28df46 nodeName:}" failed. No retries permitted until 2025-12-03 12:32:25.873338505 +0000 UTC m=+4336.461580269 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/478e18e8-077e-406e-aa66-19bc1d28df46-operator-scripts") pod "novacell046b8-account-delete-nqzlj" (UID: "478e18e8-077e-406e-aa66-19bc1d28df46") : configmap "openstack-scripts" not found Dec 03 12:32:23 crc kubenswrapper[5052]: E1203 12:32:23.873931 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:32:23 crc kubenswrapper[5052]: E1203 12:32:23.874048 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ae9024f5-a2b4-49de-aed3-00cb9c806704-operator-scripts podName:ae9024f5-a2b4-49de-aed3-00cb9c806704 nodeName:}" failed. No retries permitted until 2025-12-03 12:32:24.873969813 +0000 UTC m=+4335.462211567 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/ae9024f5-a2b4-49de-aed3-00cb9c806704-operator-scripts") pod "placemente65a-account-delete-s7x52" (UID: "ae9024f5-a2b4-49de-aed3-00cb9c806704") : configmap "openstack-scripts" not found Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.875175 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/aa55d0c8-56de-4edd-92b3-9252f169fed8-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.875308 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage05-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage05-crc\") on node \"crc\" " Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.875412 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/8ac6e8f0-d49c-4a4c-846a-6cec226bd360-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.875581 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-fqrfd\" (UniqueName: \"kubernetes.io/projected/8ac6e8f0-d49c-4a4c-846a-6cec226bd360-kube-api-access-fqrfd\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.876109 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/aa55d0c8-56de-4edd-92b3-9252f169fed8-config-data-default\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.876216 5052 reconciler_common.go:293] "Volume detached for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/16cac95c-d1d2-4542-bb8e-4bdc1923c684-log-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.876286 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/d09b9817-6ea5-4328-9027-3311b7782157-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.876349 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/aa55d0c8-56de-4edd-92b3-9252f169fed8-config-data-generated\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.876416 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-hhjkq\" (UniqueName: \"kubernetes.io/projected/d09b9817-6ea5-4328-9027-3311b7782157-kube-api-access-hhjkq\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.876481 5052 reconciler_common.go:293] "Volume detached for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/aa55d0c8-56de-4edd-92b3-9252f169fed8-kolla-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.876573 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-94jwv\" (UniqueName: \"kubernetes.io/projected/aa55d0c8-56de-4edd-92b3-9252f169fed8-kube-api-access-94jwv\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.876763 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/8ac6e8f0-d49c-4a4c-846a-6cec226bd360-config-data-custom\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.876898 5052 reconciler_common.go:293] "Volume detached for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/16cac95c-d1d2-4542-bb8e-4bdc1923c684-etc-swift\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.878758 5052 reconciler_common.go:293] "Volume detached for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/16cac95c-d1d2-4542-bb8e-4bdc1923c684-run-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.878870 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-s4mt4\" (UniqueName: \"kubernetes.io/projected/16cac95c-d1d2-4542-bb8e-4bdc1923c684-kube-api-access-s4mt4\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.924613 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8ac6e8f0-d49c-4a4c-846a-6cec226bd360-config-data" (OuterVolumeSpecName: "config-data") pod "8ac6e8f0-d49c-4a4c-846a-6cec226bd360" (UID: "8ac6e8f0-d49c-4a4c-846a-6cec226bd360"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.927248 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/16cac95c-d1d2-4542-bb8e-4bdc1923c684-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "16cac95c-d1d2-4542-bb8e-4bdc1923c684" (UID: "16cac95c-d1d2-4542-bb8e-4bdc1923c684"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.936713 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aa55d0c8-56de-4edd-92b3-9252f169fed8-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "aa55d0c8-56de-4edd-92b3-9252f169fed8" (UID: "aa55d0c8-56de-4edd-92b3-9252f169fed8"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.937418 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8ac6e8f0-d49c-4a4c-846a-6cec226bd360-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "8ac6e8f0-d49c-4a4c-846a-6cec226bd360" (UID: "8ac6e8f0-d49c-4a4c-846a-6cec226bd360"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.938231 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aa55d0c8-56de-4edd-92b3-9252f169fed8-galera-tls-certs" (OuterVolumeSpecName: "galera-tls-certs") pod "aa55d0c8-56de-4edd-92b3-9252f169fed8" (UID: "aa55d0c8-56de-4edd-92b3-9252f169fed8"). InnerVolumeSpecName "galera-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.953382 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/16cac95c-d1d2-4542-bb8e-4bdc1923c684-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "16cac95c-d1d2-4542-bb8e-4bdc1923c684" (UID: "16cac95c-d1d2-4542-bb8e-4bdc1923c684"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.961126 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage05-crc" (UniqueName: "kubernetes.io/local-volume/local-storage05-crc") on node "crc" Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.974939 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d09b9817-6ea5-4328-9027-3311b7782157-vencrypt-tls-certs" (OuterVolumeSpecName: "vencrypt-tls-certs") pod "d09b9817-6ea5-4328-9027-3311b7782157" (UID: "d09b9817-6ea5-4328-9027-3311b7782157"). InnerVolumeSpecName "vencrypt-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.982525 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d09b9817-6ea5-4328-9027-3311b7782157-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "d09b9817-6ea5-4328-9027-3311b7782157" (UID: "d09b9817-6ea5-4328-9027-3311b7782157"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.983140 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/8ac6e8f0-d49c-4a4c-846a-6cec226bd360-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.983170 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/16cac95c-d1d2-4542-bb8e-4bdc1923c684-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.983180 5052 reconciler_common.go:293] "Volume detached for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/aa55d0c8-56de-4edd-92b3-9252f169fed8-galera-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.983190 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8ac6e8f0-d49c-4a4c-846a-6cec226bd360-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.983199 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage05-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage05-crc\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.983207 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/aa55d0c8-56de-4edd-92b3-9252f169fed8-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.983218 5052 reconciler_common.go:293] "Volume detached for volume \"vencrypt-tls-certs\" (UniqueName: \"kubernetes.io/secret/d09b9817-6ea5-4328-9027-3311b7782157-vencrypt-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.983228 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d09b9817-6ea5-4328-9027-3311b7782157-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.983236 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/16cac95c-d1d2-4542-bb8e-4bdc1923c684-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.992314 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/16cac95c-d1d2-4542-bb8e-4bdc1923c684-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "16cac95c-d1d2-4542-bb8e-4bdc1923c684" (UID: "16cac95c-d1d2-4542-bb8e-4bdc1923c684"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:32:23 crc kubenswrapper[5052]: I1203 12:32:23.994181 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/16cac95c-d1d2-4542-bb8e-4bdc1923c684-config-data" (OuterVolumeSpecName: "config-data") pod "16cac95c-d1d2-4542-bb8e-4bdc1923c684" (UID: "16cac95c-d1d2-4542-bb8e-4bdc1923c684"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:32:24 crc kubenswrapper[5052]: I1203 12:32:24.035146 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d09b9817-6ea5-4328-9027-3311b7782157-nova-novncproxy-tls-certs" (OuterVolumeSpecName: "nova-novncproxy-tls-certs") pod "d09b9817-6ea5-4328-9027-3311b7782157" (UID: "d09b9817-6ea5-4328-9027-3311b7782157"). InnerVolumeSpecName "nova-novncproxy-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:32:24 crc kubenswrapper[5052]: I1203 12:32:24.084772 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/16cac95c-d1d2-4542-bb8e-4bdc1923c684-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:24 crc kubenswrapper[5052]: I1203 12:32:24.084798 5052 reconciler_common.go:293] "Volume detached for volume \"nova-novncproxy-tls-certs\" (UniqueName: \"kubernetes.io/secret/d09b9817-6ea5-4328-9027-3311b7782157-nova-novncproxy-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:24 crc kubenswrapper[5052]: I1203 12:32:24.084809 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/16cac95c-d1d2-4542-bb8e-4bdc1923c684-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:24 crc kubenswrapper[5052]: I1203 12:32:24.110951 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/novaapia918-account-delete-mk25j"] Dec 03 12:32:24 crc kubenswrapper[5052]: I1203 12:32:24.110986 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican3d67-account-delete-qrp98"] Dec 03 12:32:24 crc kubenswrapper[5052]: I1203 12:32:24.111000 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/novacell046b8-account-delete-nqzlj"] Dec 03 12:32:24 crc kubenswrapper[5052]: I1203 12:32:24.111020 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-hjfg4"] Dec 03 12:32:24 crc kubenswrapper[5052]: I1203 12:32:24.253022 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/rabbitmq-server-0" podUID="2c03f99a-98fd-4e86-ad29-fb69e94bd436" containerName="rabbitmq" probeResult="failure" output="dial tcp 10.217.0.142:5671: connect: connection refused" Dec 03 12:32:24 crc kubenswrapper[5052]: I1203 12:32:24.263884 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-worker-644558fbc5-4gtxj" Dec 03 12:32:24 crc kubenswrapper[5052]: I1203 12:32:24.264757 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-worker-644558fbc5-4gtxj" event={"ID":"8ac6e8f0-d49c-4a4c-846a-6cec226bd360","Type":"ContainerDied","Data":"4546b826c99134ff60d23f10a99bb12e6e7977932e01290b96ec7338744adb66"} Dec 03 12:32:24 crc kubenswrapper[5052]: I1203 12:32:24.264858 5052 scope.go:117] "RemoveContainer" containerID="4893c6e7800c80d6b9f1c1b9d2a79b5fd298a43e5935cb9d7b3f6072575ccfc6" Dec 03 12:32:24 crc kubenswrapper[5052]: I1203 12:32:24.267227 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-hjfg4" event={"ID":"00bbba79-7529-4309-af53-b464acf28194","Type":"ContainerStarted","Data":"d728e258976c599d37747130930f52da15a04d3a7b62248cfb1dfef0b9361827"} Dec 03 12:32:24 crc kubenswrapper[5052]: I1203 12:32:24.271215 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:32:24 crc kubenswrapper[5052]: I1203 12:32:24.271247 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-cell1-galera-0" event={"ID":"aa55d0c8-56de-4edd-92b3-9252f169fed8","Type":"ContainerDied","Data":"e0fc2dcb7d273c04fa4e698e02a3ae70eb1854c856f4a52ccb677573d0cc28af"} Dec 03 12:32:24 crc kubenswrapper[5052]: I1203 12:32:24.273347 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" event={"ID":"d09b9817-6ea5-4328-9027-3311b7782157","Type":"ContainerDied","Data":"f4e3e1af651c18cabeaf7d4b2c90fc4171ebe8b4d4dfa7e41b60c69c0093d8e8"} Dec 03 12:32:24 crc kubenswrapper[5052]: I1203 12:32:24.273467 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:32:24 crc kubenswrapper[5052]: I1203 12:32:24.290074 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-proxy-55bdbb59c4-c4cjv" Dec 03 12:32:24 crc kubenswrapper[5052]: I1203 12:32:24.291774 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-proxy-55bdbb59c4-c4cjv" event={"ID":"16cac95c-d1d2-4542-bb8e-4bdc1923c684","Type":"ContainerDied","Data":"73c4644f0732be3df2511f0c00e7ed3cf60813f1e9870d75287eae749cc4992e"} Dec 03 12:32:24 crc kubenswrapper[5052]: I1203 12:32:24.295010 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/novaapia918-account-delete-mk25j" event={"ID":"e6d87a68-5121-4088-a5d8-3f205a787b29","Type":"ContainerStarted","Data":"11243a11f3f92fefd822d688612a3f588dc1510f45d518e358b4b10cfe36371c"} Dec 03 12:32:24 crc kubenswrapper[5052]: I1203 12:32:24.295084 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/novaapia918-account-delete-mk25j" event={"ID":"e6d87a68-5121-4088-a5d8-3f205a787b29","Type":"ContainerStarted","Data":"4375fafb1b150e12234333b1938d3140be223ca4add0a6d78137dbc69efc6d17"} Dec 03 12:32:24 crc kubenswrapper[5052]: I1203 12:32:24.296061 5052 scope.go:117] "RemoveContainer" containerID="57de2e7ba65bad4f495dc483cc50efdd66d36654d3d53aadfa56507cc2faada6" Dec 03 12:32:24 crc kubenswrapper[5052]: I1203 12:32:24.313811 5052 kubelet_pods.go:1007] "Unable to retrieve pull secret, the image pull may not succeed." pod="openstack-kuttl-tests/novaapia918-account-delete-mk25j" secret="" err="secret \"galera-openstack-dockercfg-w7gzp\" not found" Dec 03 12:32:24 crc kubenswrapper[5052]: I1203 12:32:24.323816 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance5b9b-account-delete-ggk6q" event={"ID":"52bf0c8d-03b3-444b-80b6-13d41f5f9fb9","Type":"ContainerStarted","Data":"af107e2e82074f5c5aba6f4fcc9960e916fe7e1e038d4e5c3098122ab4c96e1b"} Dec 03 12:32:24 crc kubenswrapper[5052]: I1203 12:32:24.328750 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican3d67-account-delete-qrp98" event={"ID":"8a93d2af-9f8c-4282-8611-526869559ae2","Type":"ContainerStarted","Data":"a4c45df8d4b52dec58435965365d638aca4c5865f260d4a90c6878d5dc66331c"} Dec 03 12:32:24 crc kubenswrapper[5052]: I1203 12:32:24.335883 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinderc151-account-delete-h4k5b" event={"ID":"fc428005-b653-4a6a-a50d-133c2b4fb8b8","Type":"ContainerStarted","Data":"f45508992c6ce3df52ea0e08b462396de8f82c3d69fcce583cf8d26d1c4c3946"} Dec 03 12:32:24 crc kubenswrapper[5052]: I1203 12:32:24.338218 5052 kubelet_pods.go:1007] "Unable to retrieve pull secret, the image pull may not succeed." pod="openstack-kuttl-tests/glance5b9b-account-delete-ggk6q" secret="" err="secret \"galera-openstack-dockercfg-w7gzp\" not found" Dec 03 12:32:24 crc kubenswrapper[5052]: I1203 12:32:24.344663 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/novaapia918-account-delete-mk25j" podStartSLOduration=4.3446479270000005 podStartE2EDuration="4.344647927s" podCreationTimestamp="2025-12-03 12:32:20 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:32:24.318341876 +0000 UTC m=+4334.906583640" watchObservedRunningTime="2025-12-03 12:32:24.344647927 +0000 UTC m=+4334.932889691" Dec 03 12:32:24 crc kubenswrapper[5052]: I1203 12:32:24.350109 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/glance5b9b-account-delete-ggk6q" podStartSLOduration=5.350089753 podStartE2EDuration="5.350089753s" podCreationTimestamp="2025-12-03 12:32:19 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:32:24.344775731 +0000 UTC m=+4334.933017495" watchObservedRunningTime="2025-12-03 12:32:24.350089753 +0000 UTC m=+4334.938331517" Dec 03 12:32:24 crc kubenswrapper[5052]: I1203 12:32:24.354902 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron2174-account-delete-9dnxs" event={"ID":"a2ef0804-75c7-43e3-95fa-ce901a561c9b","Type":"ContainerStarted","Data":"1527475535a0407481384ca6a15c4f2f9da6de7e6bcda257db051f93a93b1aa9"} Dec 03 12:32:24 crc kubenswrapper[5052]: I1203 12:32:24.358777 5052 kubelet_pods.go:1007] "Unable to retrieve pull secret, the image pull may not succeed." pod="openstack-kuttl-tests/neutron2174-account-delete-9dnxs" secret="" err="secret \"galera-openstack-dockercfg-w7gzp\" not found" Dec 03 12:32:24 crc kubenswrapper[5052]: I1203 12:32:24.360157 5052 kubelet_pods.go:1007] "Unable to retrieve pull secret, the image pull may not succeed." pod="openstack-kuttl-tests/cinderc151-account-delete-h4k5b" secret="" err="secret \"galera-openstack-dockercfg-w7gzp\" not found" Dec 03 12:32:24 crc kubenswrapper[5052]: I1203 12:32:24.372152 5052 generic.go:334] "Generic (PLEG): container finished" podID="cfd8af50-a644-4b24-8483-dd344f2218e4" containerID="fefeb53cf7ddfecbec19b54d8ffa06426f1efa88fb8452a1c522b836c48d8085" exitCode=0 Dec 03 12:32:24 crc kubenswrapper[5052]: I1203 12:32:24.372237 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-65444bfc86-th5pw" event={"ID":"cfd8af50-a644-4b24-8483-dd344f2218e4","Type":"ContainerDied","Data":"fefeb53cf7ddfecbec19b54d8ffa06426f1efa88fb8452a1c522b836c48d8085"} Dec 03 12:32:24 crc kubenswrapper[5052]: I1203 12:32:24.376838 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/novacell046b8-account-delete-nqzlj" event={"ID":"478e18e8-077e-406e-aa66-19bc1d28df46","Type":"ContainerStarted","Data":"53db9bec4c5fa77c3e23669a8d7cb3e397825fa6ec1b6c508ab5eed920339068"} Dec 03 12:32:24 crc kubenswrapper[5052]: I1203 12:32:24.378132 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/cinderc151-account-delete-h4k5b" podStartSLOduration=5.378109932 podStartE2EDuration="5.378109932s" podCreationTimestamp="2025-12-03 12:32:19 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:32:24.359424169 +0000 UTC m=+4334.947665933" watchObservedRunningTime="2025-12-03 12:32:24.378109932 +0000 UTC m=+4334.966351696" Dec 03 12:32:24 crc kubenswrapper[5052]: I1203 12:32:24.381974 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/novacell115a6-account-delete-pbwnc" Dec 03 12:32:24 crc kubenswrapper[5052]: I1203 12:32:24.383085 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-api-78dc8cddf6-kqfz7" Dec 03 12:32:24 crc kubenswrapper[5052]: I1203 12:32:24.383314 5052 kubelet_pods.go:1007] "Unable to retrieve pull secret, the image pull may not succeed." pod="openstack-kuttl-tests/placemente65a-account-delete-s7x52" secret="" err="secret \"galera-openstack-dockercfg-w7gzp\" not found" Dec 03 12:32:24 crc kubenswrapper[5052]: I1203 12:32:24.383348 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-keystone-listener-697474f89d-p7lwz" Dec 03 12:32:24 crc kubenswrapper[5052]: I1203 12:32:24.384804 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-worker-c899dcc5-n6rdv" Dec 03 12:32:24 crc kubenswrapper[5052]: E1203 12:32:24.395604 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:32:24 crc kubenswrapper[5052]: E1203 12:32:24.395682 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a2ef0804-75c7-43e3-95fa-ce901a561c9b-operator-scripts podName:a2ef0804-75c7-43e3-95fa-ce901a561c9b nodeName:}" failed. No retries permitted until 2025-12-03 12:32:24.895656353 +0000 UTC m=+4335.483898107 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/a2ef0804-75c7-43e3-95fa-ce901a561c9b-operator-scripts") pod "neutron2174-account-delete-9dnxs" (UID: "a2ef0804-75c7-43e3-95fa-ce901a561c9b") : configmap "openstack-scripts" not found Dec 03 12:32:24 crc kubenswrapper[5052]: I1203 12:32:24.399626 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/neutron2174-account-delete-9dnxs" podStartSLOduration=5.399603246 podStartE2EDuration="5.399603246s" podCreationTimestamp="2025-12-03 12:32:19 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:32:24.379257555 +0000 UTC m=+4334.967499319" watchObservedRunningTime="2025-12-03 12:32:24.399603246 +0000 UTC m=+4334.987845030" Dec 03 12:32:24 crc kubenswrapper[5052]: E1203 12:32:24.499747 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:32:24 crc kubenswrapper[5052]: E1203 12:32:24.499814 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/fc428005-b653-4a6a-a50d-133c2b4fb8b8-operator-scripts podName:fc428005-b653-4a6a-a50d-133c2b4fb8b8 nodeName:}" failed. No retries permitted until 2025-12-03 12:32:24.999799276 +0000 UTC m=+4335.588041040 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/fc428005-b653-4a6a-a50d-133c2b4fb8b8-operator-scripts") pod "cinderc151-account-delete-h4k5b" (UID: "fc428005-b653-4a6a-a50d-133c2b4fb8b8") : configmap "openstack-scripts" not found Dec 03 12:32:24 crc kubenswrapper[5052]: E1203 12:32:24.500293 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:32:24 crc kubenswrapper[5052]: E1203 12:32:24.500344 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e6d87a68-5121-4088-a5d8-3f205a787b29-operator-scripts podName:e6d87a68-5121-4088-a5d8-3f205a787b29 nodeName:}" failed. No retries permitted until 2025-12-03 12:32:25.000332901 +0000 UTC m=+4335.588574665 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/e6d87a68-5121-4088-a5d8-3f205a787b29-operator-scripts") pod "novaapia918-account-delete-mk25j" (UID: "e6d87a68-5121-4088-a5d8-3f205a787b29") : configmap "openstack-scripts" not found Dec 03 12:32:24 crc kubenswrapper[5052]: E1203 12:32:24.500396 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:32:24 crc kubenswrapper[5052]: E1203 12:32:24.500427 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/52bf0c8d-03b3-444b-80b6-13d41f5f9fb9-operator-scripts podName:52bf0c8d-03b3-444b-80b6-13d41f5f9fb9 nodeName:}" failed. No retries permitted until 2025-12-03 12:32:25.000417963 +0000 UTC m=+4335.588659727 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/52bf0c8d-03b3-444b-80b6-13d41f5f9fb9-operator-scripts") pod "glance5b9b-account-delete-ggk6q" (UID: "52bf0c8d-03b3-444b-80b6-13d41f5f9fb9") : configmap "openstack-scripts" not found Dec 03 12:32:24 crc kubenswrapper[5052]: I1203 12:32:24.529560 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/placement-db-create-s2jk2"] Dec 03 12:32:24 crc kubenswrapper[5052]: I1203 12:32:24.535845 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/cinder-api-0" podUID="6e0f19c7-34ae-4d8e-9a19-755ae1c20c1c" containerName="cinder-api" probeResult="failure" output="Get \"https://10.217.0.237:8776/healthcheck\": read tcp 10.217.0.2:39096->10.217.0.237:8776: read: connection reset by peer" Dec 03 12:32:24 crc kubenswrapper[5052]: I1203 12:32:24.546392 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/placement-db-create-s2jk2"] Dec 03 12:32:24 crc kubenswrapper[5052]: I1203 12:32:24.570336 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/placement-e65a-account-create-update-8vzns"] Dec 03 12:32:24 crc kubenswrapper[5052]: I1203 12:32:24.577752 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/nova-metadata-0" podUID="39117149-c15d-4753-9324-8a42abe5b007" containerName="nova-metadata-log" probeResult="failure" output="Get \"https://10.217.0.213:8775/\": read tcp 10.217.0.2:35184->10.217.0.213:8775: read: connection reset by peer" Dec 03 12:32:24 crc kubenswrapper[5052]: I1203 12:32:24.578036 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/nova-metadata-0" podUID="39117149-c15d-4753-9324-8a42abe5b007" containerName="nova-metadata-metadata" probeResult="failure" output="Get \"https://10.217.0.213:8775/\": read tcp 10.217.0.2:35192->10.217.0.213:8775: read: connection reset by peer" Dec 03 12:32:24 crc kubenswrapper[5052]: I1203 12:32:24.583933 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/placemente65a-account-delete-s7x52"] Dec 03 12:32:24 crc kubenswrapper[5052]: I1203 12:32:24.591494 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/placement-e65a-account-create-update-8vzns"] Dec 03 12:32:24 crc kubenswrapper[5052]: I1203 12:32:24.611403 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-db-create-gssmd"] Dec 03 12:32:24 crc kubenswrapper[5052]: I1203 12:32:24.635773 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/glance-db-create-gssmd"] Dec 03 12:32:24 crc kubenswrapper[5052]: I1203 12:32:24.668273 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-5b9b-account-create-update-5qk9p"] Dec 03 12:32:24 crc kubenswrapper[5052]: I1203 12:32:24.679755 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance5b9b-account-delete-ggk6q"] Dec 03 12:32:24 crc kubenswrapper[5052]: I1203 12:32:24.693630 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/glance-5b9b-account-create-update-5qk9p"] Dec 03 12:32:24 crc kubenswrapper[5052]: I1203 12:32:24.705281 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/neutron-db-create-bmmmd"] Dec 03 12:32:24 crc kubenswrapper[5052]: I1203 12:32:24.708858 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/f12527ba-678e-4ed9-82ff-cec336f607f3-operator-scripts\") pod \"novacell115a6-account-delete-pbwnc\" (UID: \"f12527ba-678e-4ed9-82ff-cec336f607f3\") " pod="openstack-kuttl-tests/novacell115a6-account-delete-pbwnc" Dec 03 12:32:24 crc kubenswrapper[5052]: I1203 12:32:24.708952 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-wr9nk\" (UniqueName: \"kubernetes.io/projected/f12527ba-678e-4ed9-82ff-cec336f607f3-kube-api-access-wr9nk\") pod \"novacell115a6-account-delete-pbwnc\" (UID: \"f12527ba-678e-4ed9-82ff-cec336f607f3\") " pod="openstack-kuttl-tests/novacell115a6-account-delete-pbwnc" Dec 03 12:32:24 crc kubenswrapper[5052]: E1203 12:32:24.710151 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-cell1-scripts: object "openstack-kuttl-tests"/"openstack-cell1-scripts" not registered Dec 03 12:32:24 crc kubenswrapper[5052]: E1203 12:32:24.710195 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f12527ba-678e-4ed9-82ff-cec336f607f3-operator-scripts podName:f12527ba-678e-4ed9-82ff-cec336f607f3 nodeName:}" failed. No retries permitted until 2025-12-03 12:32:28.71018158 +0000 UTC m=+4339.298423344 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/f12527ba-678e-4ed9-82ff-cec336f607f3-operator-scripts") pod "novacell115a6-account-delete-pbwnc" (UID: "f12527ba-678e-4ed9-82ff-cec336f607f3") : object "openstack-kuttl-tests"/"openstack-cell1-scripts" not registered Dec 03 12:32:24 crc kubenswrapper[5052]: E1203 12:32:24.716453 5052 projected.go:194] Error preparing data for projected volume kube-api-access-wr9nk for pod openstack-kuttl-tests/novacell115a6-account-delete-pbwnc: failed to fetch token: serviceaccounts "galera-openstack-cell1" not found Dec 03 12:32:24 crc kubenswrapper[5052]: E1203 12:32:24.716531 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f12527ba-678e-4ed9-82ff-cec336f607f3-kube-api-access-wr9nk podName:f12527ba-678e-4ed9-82ff-cec336f607f3 nodeName:}" failed. No retries permitted until 2025-12-03 12:32:28.716501761 +0000 UTC m=+4339.304743525 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-wr9nk" (UniqueName: "kubernetes.io/projected/f12527ba-678e-4ed9-82ff-cec336f607f3-kube-api-access-wr9nk") pod "novacell115a6-account-delete-pbwnc" (UID: "f12527ba-678e-4ed9-82ff-cec336f607f3") : failed to fetch token: serviceaccounts "galera-openstack-cell1" not found Dec 03 12:32:24 crc kubenswrapper[5052]: I1203 12:32:24.720689 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/neutron-db-create-bmmmd"] Dec 03 12:32:24 crc kubenswrapper[5052]: I1203 12:32:24.748111 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/neutron2174-account-delete-9dnxs"] Dec 03 12:32:24 crc kubenswrapper[5052]: I1203 12:32:24.753958 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/neutron-2174-account-create-update-m4lrb"] Dec 03 12:32:24 crc kubenswrapper[5052]: I1203 12:32:24.761185 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/neutron-2174-account-create-update-m4lrb"] Dec 03 12:32:24 crc kubenswrapper[5052]: I1203 12:32:24.818984 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/barbican-api-68fc8f9776-n9gml" podUID="82f142ab-5be9-4727-b3c5-102d6f2401fb" containerName="barbican-api" probeResult="failure" output="Get \"https://10.217.0.233:9311/healthcheck\": read tcp 10.217.0.2:45948->10.217.0.233:9311: read: connection reset by peer" Dec 03 12:32:24 crc kubenswrapper[5052]: I1203 12:32:24.819023 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/barbican-api-68fc8f9776-n9gml" podUID="82f142ab-5be9-4727-b3c5-102d6f2401fb" containerName="barbican-api-log" probeResult="failure" output="Get \"https://10.217.0.233:9311/healthcheck\": read tcp 10.217.0.2:45952->10.217.0.233:9311: read: connection reset by peer" Dec 03 12:32:24 crc kubenswrapper[5052]: E1203 12:32:24.928736 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:32:24 crc kubenswrapper[5052]: E1203 12:32:24.929199 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a2ef0804-75c7-43e3-95fa-ce901a561c9b-operator-scripts podName:a2ef0804-75c7-43e3-95fa-ce901a561c9b nodeName:}" failed. No retries permitted until 2025-12-03 12:32:25.929175771 +0000 UTC m=+4336.517417545 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/a2ef0804-75c7-43e3-95fa-ce901a561c9b-operator-scripts") pod "neutron2174-account-delete-9dnxs" (UID: "a2ef0804-75c7-43e3-95fa-ce901a561c9b") : configmap "openstack-scripts" not found Dec 03 12:32:24 crc kubenswrapper[5052]: E1203 12:32:24.928887 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:32:24 crc kubenswrapper[5052]: E1203 12:32:24.929281 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ae9024f5-a2b4-49de-aed3-00cb9c806704-operator-scripts podName:ae9024f5-a2b4-49de-aed3-00cb9c806704 nodeName:}" failed. No retries permitted until 2025-12-03 12:32:26.929272094 +0000 UTC m=+4337.517513948 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/ae9024f5-a2b4-49de-aed3-00cb9c806704-operator-scripts") pod "placemente65a-account-delete-s7x52" (UID: "ae9024f5-a2b4-49de-aed3-00cb9c806704") : configmap "openstack-scripts" not found Dec 03 12:32:24 crc kubenswrapper[5052]: I1203 12:32:24.934473 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder-db-create-7rgm6"] Dec 03 12:32:24 crc kubenswrapper[5052]: I1203 12:32:24.954635 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/cinder-db-create-7rgm6"] Dec 03 12:32:24 crc kubenswrapper[5052]: I1203 12:32:24.971743 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder-c151-account-create-update-6ltql"] Dec 03 12:32:24 crc kubenswrapper[5052]: I1203 12:32:24.978246 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/cinder-c151-account-create-update-6ltql"] Dec 03 12:32:24 crc kubenswrapper[5052]: I1203 12:32:24.987395 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinderc151-account-delete-h4k5b"] Dec 03 12:32:25 crc kubenswrapper[5052]: E1203 12:32:25.031097 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:32:25 crc kubenswrapper[5052]: E1203 12:32:25.031580 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:32:25 crc kubenswrapper[5052]: E1203 12:32:25.031855 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:32:25 crc kubenswrapper[5052]: E1203 12:32:25.032548 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/fc428005-b653-4a6a-a50d-133c2b4fb8b8-operator-scripts podName:fc428005-b653-4a6a-a50d-133c2b4fb8b8 nodeName:}" failed. No retries permitted until 2025-12-03 12:32:26.032516901 +0000 UTC m=+4336.620758675 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/fc428005-b653-4a6a-a50d-133c2b4fb8b8-operator-scripts") pod "cinderc151-account-delete-h4k5b" (UID: "fc428005-b653-4a6a-a50d-133c2b4fb8b8") : configmap "openstack-scripts" not found Dec 03 12:32:25 crc kubenswrapper[5052]: E1203 12:32:25.032593 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e6d87a68-5121-4088-a5d8-3f205a787b29-operator-scripts podName:e6d87a68-5121-4088-a5d8-3f205a787b29 nodeName:}" failed. No retries permitted until 2025-12-03 12:32:26.032584273 +0000 UTC m=+4336.620826117 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/e6d87a68-5121-4088-a5d8-3f205a787b29-operator-scripts") pod "novaapia918-account-delete-mk25j" (UID: "e6d87a68-5121-4088-a5d8-3f205a787b29") : configmap "openstack-scripts" not found Dec 03 12:32:25 crc kubenswrapper[5052]: E1203 12:32:25.032607 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/52bf0c8d-03b3-444b-80b6-13d41f5f9fb9-operator-scripts podName:52bf0c8d-03b3-444b-80b6-13d41f5f9fb9 nodeName:}" failed. No retries permitted until 2025-12-03 12:32:26.032600133 +0000 UTC m=+4336.620841897 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/52bf0c8d-03b3-444b-80b6-13d41f5f9fb9-operator-scripts") pod "glance5b9b-account-delete-ggk6q" (UID: "52bf0c8d-03b3-444b-80b6-13d41f5f9fb9") : configmap "openstack-scripts" not found Dec 03 12:32:25 crc kubenswrapper[5052]: I1203 12:32:25.138300 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-db-create-8h8nv"] Dec 03 12:32:25 crc kubenswrapper[5052]: I1203 12:32:25.165133 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/barbican-db-create-8h8nv"] Dec 03 12:32:25 crc kubenswrapper[5052]: I1203 12:32:25.212600 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-3d67-account-create-update-wmkzq"] Dec 03 12:32:25 crc kubenswrapper[5052]: I1203 12:32:25.224238 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/barbican-3d67-account-create-update-wmkzq"] Dec 03 12:32:25 crc kubenswrapper[5052]: I1203 12:32:25.239764 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican3d67-account-delete-qrp98"] Dec 03 12:32:25 crc kubenswrapper[5052]: I1203 12:32:25.287446 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-api-db-create-846mj"] Dec 03 12:32:25 crc kubenswrapper[5052]: I1203 12:32:25.317557 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-api-db-create-846mj"] Dec 03 12:32:25 crc kubenswrapper[5052]: I1203 12:32:25.348679 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-api-a918-account-create-update-s6fjd"] Dec 03 12:32:25 crc kubenswrapper[5052]: I1203 12:32:25.359842 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-api-a918-account-create-update-s6fjd"] Dec 03 12:32:25 crc kubenswrapper[5052]: I1203 12:32:25.373767 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/novaapia918-account-delete-mk25j"] Dec 03 12:32:25 crc kubenswrapper[5052]: I1203 12:32:25.380698 5052 scope.go:117] "RemoveContainer" containerID="8bf7dd5e7641b5ac8d567b37be404e10979d45b84ac534f9d893fee92ab89c5e" Dec 03 12:32:25 crc kubenswrapper[5052]: I1203 12:32:25.390900 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell0-db-create-5fgfv"] Dec 03 12:32:25 crc kubenswrapper[5052]: I1203 12:32:25.415105 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell0-db-create-5fgfv"] Dec 03 12:32:25 crc kubenswrapper[5052]: I1203 12:32:25.435969 5052 generic.go:334] "Generic (PLEG): container finished" podID="82f142ab-5be9-4727-b3c5-102d6f2401fb" containerID="74fe4216c2c90357dad59af2f1e20e908e8583d49e0f2a8d15cd86ca1fc755d6" exitCode=0 Dec 03 12:32:25 crc kubenswrapper[5052]: I1203 12:32:25.436433 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-68fc8f9776-n9gml" event={"ID":"82f142ab-5be9-4727-b3c5-102d6f2401fb","Type":"ContainerDied","Data":"74fe4216c2c90357dad59af2f1e20e908e8583d49e0f2a8d15cd86ca1fc755d6"} Dec 03 12:32:25 crc kubenswrapper[5052]: I1203 12:32:25.447029 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/novacell046b8-account-delete-nqzlj"] Dec 03 12:32:25 crc kubenswrapper[5052]: I1203 12:32:25.451658 5052 generic.go:334] "Generic (PLEG): container finished" podID="bb1723c7-4e41-4b0e-b6b8-f8024dafdaee" containerID="94df1fd3c0977441ba1cf70763e96ac140d6a232f2c0d6cfe09fb6aa6ae75dc3" exitCode=0 Dec 03 12:32:25 crc kubenswrapper[5052]: I1203 12:32:25.451783 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"bb1723c7-4e41-4b0e-b6b8-f8024dafdaee","Type":"ContainerDied","Data":"94df1fd3c0977441ba1cf70763e96ac140d6a232f2c0d6cfe09fb6aa6ae75dc3"} Dec 03 12:32:25 crc kubenswrapper[5052]: I1203 12:32:25.457771 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell0-46b8-account-create-update-g9c2m"] Dec 03 12:32:25 crc kubenswrapper[5052]: I1203 12:32:25.507636 5052 scope.go:117] "RemoveContainer" containerID="0a11b0f2d7f2318c289c08d3984727c92ca9c2e1900b8e4ebba02d7834859031" Dec 03 12:32:25 crc kubenswrapper[5052]: I1203 12:32:25.516780 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell0-46b8-account-create-update-g9c2m"] Dec 03 12:32:25 crc kubenswrapper[5052]: I1203 12:32:25.541295 5052 generic.go:334] "Generic (PLEG): container finished" podID="a65428bc-9326-4ace-a985-9f796b951304" containerID="ddde80cbe1c1205957f18817d4db06fabccc08242a25c2d74bcd5a4c61d8bd5d" exitCode=0 Dec 03 12:32:25 crc kubenswrapper[5052]: I1203 12:32:25.541327 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"a65428bc-9326-4ace-a985-9f796b951304","Type":"ContainerDied","Data":"ddde80cbe1c1205957f18817d4db06fabccc08242a25c2d74bcd5a4c61d8bd5d"} Dec 03 12:32:25 crc kubenswrapper[5052]: I1203 12:32:25.586294 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-worker-644558fbc5-4gtxj"] Dec 03 12:32:25 crc kubenswrapper[5052]: I1203 12:32:25.600759 5052 generic.go:334] "Generic (PLEG): container finished" podID="39117149-c15d-4753-9324-8a42abe5b007" containerID="5b99120371dd1ea96363322c8c525108a18d3c48f5844ec07a58fedeb98a76df" exitCode=0 Dec 03 12:32:25 crc kubenswrapper[5052]: I1203 12:32:25.600870 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"39117149-c15d-4753-9324-8a42abe5b007","Type":"ContainerDied","Data":"5b99120371dd1ea96363322c8c525108a18d3c48f5844ec07a58fedeb98a76df"} Dec 03 12:32:25 crc kubenswrapper[5052]: I1203 12:32:25.622312 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/barbican-worker-644558fbc5-4gtxj"] Dec 03 12:32:25 crc kubenswrapper[5052]: I1203 12:32:25.655140 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/openstack-cell1-galera-0"] Dec 03 12:32:25 crc kubenswrapper[5052]: I1203 12:32:25.670241 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/openstack-cell1-galera-0"] Dec 03 12:32:25 crc kubenswrapper[5052]: I1203 12:32:25.672177 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" podUID="b8613a94-1e2e-477f-8d52-713c70f5282c" containerName="rabbitmq" probeResult="failure" output="dial tcp 10.217.0.145:5671: connect: connection refused" Dec 03 12:32:25 crc kubenswrapper[5052]: I1203 12:32:25.704293 5052 generic.go:334] "Generic (PLEG): container finished" podID="5737900d-97ae-4dcb-a65c-b40fe30cdc50" containerID="1840b34d3edddd62d0d38183e87fc8f3976fc72f58bdedd488ca0702b16fd6e7" exitCode=0 Dec 03 12:32:25 crc kubenswrapper[5052]: I1203 12:32:25.749633 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-65444bfc86-th5pw" Dec 03 12:32:25 crc kubenswrapper[5052]: I1203 12:32:25.749832 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0160de5f-930d-4766-9302-28c89c2fcf14" path="/var/lib/kubelet/pods/0160de5f-930d-4766-9302-28c89c2fcf14/volumes" Dec 03 12:32:25 crc kubenswrapper[5052]: I1203 12:32:25.750172 5052 scope.go:117] "RemoveContainer" containerID="7e1821a6c6bd165b882e82fb2451aca61d98f3c04110b410f0845d91b6abfa0b" Dec 03 12:32:25 crc kubenswrapper[5052]: I1203 12:32:25.752310 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="027c0ef8-c154-4de5-961e-d822ab32a761" path="/var/lib/kubelet/pods/027c0ef8-c154-4de5-961e-d822ab32a761/volumes" Dec 03 12:32:25 crc kubenswrapper[5052]: I1203 12:32:25.753797 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="35767293-d6c6-4fba-92bb-053cde681af2" path="/var/lib/kubelet/pods/35767293-d6c6-4fba-92bb-053cde681af2/volumes" Dec 03 12:32:25 crc kubenswrapper[5052]: I1203 12:32:25.754694 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4ea65f95-6ea7-4362-a361-514ace4fbc44" path="/var/lib/kubelet/pods/4ea65f95-6ea7-4362-a361-514ace4fbc44/volumes" Dec 03 12:32:25 crc kubenswrapper[5052]: I1203 12:32:25.755657 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="514089fc-57f6-44d3-a961-e5f3325576fb" path="/var/lib/kubelet/pods/514089fc-57f6-44d3-a961-e5f3325576fb/volumes" Dec 03 12:32:25 crc kubenswrapper[5052]: I1203 12:32:25.766491 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="59f35833-49e4-4b5d-9646-8c481aa4a9a7" path="/var/lib/kubelet/pods/59f35833-49e4-4b5d-9646-8c481aa4a9a7/volumes" Dec 03 12:32:25 crc kubenswrapper[5052]: I1203 12:32:25.767260 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="60ba2a3e-26ac-4265-b206-cb9ea4c88584" path="/var/lib/kubelet/pods/60ba2a3e-26ac-4265-b206-cb9ea4c88584/volumes" Dec 03 12:32:25 crc kubenswrapper[5052]: I1203 12:32:25.770698 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6b47bc28-48e6-4c70-b4a8-de8251b9a06e" path="/var/lib/kubelet/pods/6b47bc28-48e6-4c70-b4a8-de8251b9a06e/volumes" Dec 03 12:32:25 crc kubenswrapper[5052]: I1203 12:32:25.771930 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8ac6e8f0-d49c-4a4c-846a-6cec226bd360" path="/var/lib/kubelet/pods/8ac6e8f0-d49c-4a4c-846a-6cec226bd360/volumes" Dec 03 12:32:25 crc kubenswrapper[5052]: I1203 12:32:25.772652 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a1ac8f66-94c3-4ac5-b039-fe29314ed99a" path="/var/lib/kubelet/pods/a1ac8f66-94c3-4ac5-b039-fe29314ed99a/volumes" Dec 03 12:32:25 crc kubenswrapper[5052]: I1203 12:32:25.774330 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aa55d0c8-56de-4edd-92b3-9252f169fed8" path="/var/lib/kubelet/pods/aa55d0c8-56de-4edd-92b3-9252f169fed8/volumes" Dec 03 12:32:25 crc kubenswrapper[5052]: I1203 12:32:25.775166 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bd34573a-a034-4c22-8df3-bab80ed70f45" path="/var/lib/kubelet/pods/bd34573a-a034-4c22-8df3-bab80ed70f45/volumes" Dec 03 12:32:25 crc kubenswrapper[5052]: I1203 12:32:25.775885 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c8f2575e-56ef-4583-93ac-ea09f2004a79" path="/var/lib/kubelet/pods/c8f2575e-56ef-4583-93ac-ea09f2004a79/volumes" Dec 03 12:32:25 crc kubenswrapper[5052]: I1203 12:32:25.777086 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d04ffd07-eb13-41b2-a0ed-058afc5c8d73" path="/var/lib/kubelet/pods/d04ffd07-eb13-41b2-a0ed-058afc5c8d73/volumes" Dec 03 12:32:25 crc kubenswrapper[5052]: I1203 12:32:25.777970 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e01a4c75-3d41-4784-a957-681fc74f9f7d" path="/var/lib/kubelet/pods/e01a4c75-3d41-4784-a957-681fc74f9f7d/volumes" Dec 03 12:32:25 crc kubenswrapper[5052]: I1203 12:32:25.778828 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f3c78563-0630-4849-85ae-8b5d5ad6795c" path="/var/lib/kubelet/pods/f3c78563-0630-4849-85ae-8b5d5ad6795c/volumes" Dec 03 12:32:25 crc kubenswrapper[5052]: I1203 12:32:25.779676 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-keystone-listener-697474f89d-p7lwz"] Dec 03 12:32:25 crc kubenswrapper[5052]: I1203 12:32:25.779702 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/barbican-keystone-listener-697474f89d-p7lwz"] Dec 03 12:32:25 crc kubenswrapper[5052]: I1203 12:32:25.779724 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"5737900d-97ae-4dcb-a65c-b40fe30cdc50","Type":"ContainerDied","Data":"1840b34d3edddd62d0d38183e87fc8f3976fc72f58bdedd488ca0702b16fd6e7"} Dec 03 12:32:25 crc kubenswrapper[5052]: I1203 12:32:25.835964 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/cfd8af50-a644-4b24-8483-dd344f2218e4-config-data\") pod \"cfd8af50-a644-4b24-8483-dd344f2218e4\" (UID: \"cfd8af50-a644-4b24-8483-dd344f2218e4\") " Dec 03 12:32:25 crc kubenswrapper[5052]: I1203 12:32:25.836023 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/cfd8af50-a644-4b24-8483-dd344f2218e4-scripts\") pod \"cfd8af50-a644-4b24-8483-dd344f2218e4\" (UID: \"cfd8af50-a644-4b24-8483-dd344f2218e4\") " Dec 03 12:32:25 crc kubenswrapper[5052]: I1203 12:32:25.836081 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/cfd8af50-a644-4b24-8483-dd344f2218e4-combined-ca-bundle\") pod \"cfd8af50-a644-4b24-8483-dd344f2218e4\" (UID: \"cfd8af50-a644-4b24-8483-dd344f2218e4\") " Dec 03 12:32:25 crc kubenswrapper[5052]: I1203 12:32:25.836121 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jkbfb\" (UniqueName: \"kubernetes.io/projected/cfd8af50-a644-4b24-8483-dd344f2218e4-kube-api-access-jkbfb\") pod \"cfd8af50-a644-4b24-8483-dd344f2218e4\" (UID: \"cfd8af50-a644-4b24-8483-dd344f2218e4\") " Dec 03 12:32:25 crc kubenswrapper[5052]: I1203 12:32:25.836243 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/cfd8af50-a644-4b24-8483-dd344f2218e4-internal-tls-certs\") pod \"cfd8af50-a644-4b24-8483-dd344f2218e4\" (UID: \"cfd8af50-a644-4b24-8483-dd344f2218e4\") " Dec 03 12:32:25 crc kubenswrapper[5052]: I1203 12:32:25.836275 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/cfd8af50-a644-4b24-8483-dd344f2218e4-logs\") pod \"cfd8af50-a644-4b24-8483-dd344f2218e4\" (UID: \"cfd8af50-a644-4b24-8483-dd344f2218e4\") " Dec 03 12:32:25 crc kubenswrapper[5052]: I1203 12:32:25.836305 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/cfd8af50-a644-4b24-8483-dd344f2218e4-public-tls-certs\") pod \"cfd8af50-a644-4b24-8483-dd344f2218e4\" (UID: \"cfd8af50-a644-4b24-8483-dd344f2218e4\") " Dec 03 12:32:25 crc kubenswrapper[5052]: I1203 12:32:25.873221 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/045d578f-92c0-47c6-8356-59c489c57b9a-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:25 crc kubenswrapper[5052]: I1203 12:32:25.875684 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cfd8af50-a644-4b24-8483-dd344f2218e4-logs" (OuterVolumeSpecName: "logs") pod "cfd8af50-a644-4b24-8483-dd344f2218e4" (UID: "cfd8af50-a644-4b24-8483-dd344f2218e4"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:32:25 crc kubenswrapper[5052]: I1203 12:32:25.877736 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cfd8af50-a644-4b24-8483-dd344f2218e4-scripts" (OuterVolumeSpecName: "scripts") pod "cfd8af50-a644-4b24-8483-dd344f2218e4" (UID: "cfd8af50-a644-4b24-8483-dd344f2218e4"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:32:25 crc kubenswrapper[5052]: I1203 12:32:25.890884 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cfd8af50-a644-4b24-8483-dd344f2218e4-kube-api-access-jkbfb" (OuterVolumeSpecName: "kube-api-access-jkbfb") pod "cfd8af50-a644-4b24-8483-dd344f2218e4" (UID: "cfd8af50-a644-4b24-8483-dd344f2218e4"). InnerVolumeSpecName "kube-api-access-jkbfb". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:32:25 crc kubenswrapper[5052]: I1203 12:32:25.893958 5052 generic.go:334] "Generic (PLEG): container finished" podID="6e0f19c7-34ae-4d8e-9a19-755ae1c20c1c" containerID="e387dc9d87ea03a56050ef8cbe08349409d39a0cdecb55b0fe4e091ce7e2b2f4" exitCode=0 Dec 03 12:32:25 crc kubenswrapper[5052]: I1203 12:32:25.894038 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"6e0f19c7-34ae-4d8e-9a19-755ae1c20c1c","Type":"ContainerDied","Data":"e387dc9d87ea03a56050ef8cbe08349409d39a0cdecb55b0fe4e091ce7e2b2f4"} Dec 03 12:32:25 crc kubenswrapper[5052]: I1203 12:32:25.906747 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-worker-c899dcc5-n6rdv"] Dec 03 12:32:25 crc kubenswrapper[5052]: I1203 12:32:25.910029 5052 generic.go:334] "Generic (PLEG): container finished" podID="00bbba79-7529-4309-af53-b464acf28194" containerID="767245499b2d11a292bcbfa756fc57b438b73c163713457389fa39a70f97a888" exitCode=0 Dec 03 12:32:25 crc kubenswrapper[5052]: I1203 12:32:25.910121 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-hjfg4" event={"ID":"00bbba79-7529-4309-af53-b464acf28194","Type":"ContainerDied","Data":"767245499b2d11a292bcbfa756fc57b438b73c163713457389fa39a70f97a888"} Dec 03 12:32:25 crc kubenswrapper[5052]: I1203 12:32:25.918892 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/novacell046b8-account-delete-nqzlj" event={"ID":"478e18e8-077e-406e-aa66-19bc1d28df46","Type":"ContainerStarted","Data":"7abe534b28e0d727623c30c2ee8d6feee7f3db4244b68e027625e6a8eab08cc4"} Dec 03 12:32:25 crc kubenswrapper[5052]: I1203 12:32:25.919621 5052 kubelet_pods.go:1007] "Unable to retrieve pull secret, the image pull may not succeed." pod="openstack-kuttl-tests/novacell046b8-account-delete-nqzlj" secret="" err="secret \"galera-openstack-dockercfg-w7gzp\" not found" Dec 03 12:32:25 crc kubenswrapper[5052]: I1203 12:32:25.924608 5052 generic.go:334] "Generic (PLEG): container finished" podID="7cbe1c2c-bc04-4232-9901-bc9d134df3d6" containerID="3c03aa60a5d69f404b768bb6cc6569fcdc7b56fcc98ed0721fb684712e773b3e" exitCode=0 Dec 03 12:32:25 crc kubenswrapper[5052]: I1203 12:32:25.924663 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"7cbe1c2c-bc04-4232-9901-bc9d134df3d6","Type":"ContainerDied","Data":"3c03aa60a5d69f404b768bb6cc6569fcdc7b56fcc98ed0721fb684712e773b3e"} Dec 03 12:32:25 crc kubenswrapper[5052]: I1203 12:32:25.928563 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican3d67-account-delete-qrp98" event={"ID":"8a93d2af-9f8c-4282-8611-526869559ae2","Type":"ContainerStarted","Data":"444eb8a103a3fd9c5c270baa3559817c3b3a7da9d570c02241c847c5911ad88f"} Dec 03 12:32:25 crc kubenswrapper[5052]: I1203 12:32:25.929887 5052 kubelet_pods.go:1007] "Unable to retrieve pull secret, the image pull may not succeed." pod="openstack-kuttl-tests/barbican3d67-account-delete-qrp98" secret="" err="secret \"galera-openstack-dockercfg-w7gzp\" not found" Dec 03 12:32:25 crc kubenswrapper[5052]: I1203 12:32:25.930083 5052 kubelet_pods.go:1007] "Unable to retrieve pull secret, the image pull may not succeed." pod="openstack-kuttl-tests/novaapia918-account-delete-mk25j" secret="" err="secret \"galera-openstack-dockercfg-w7gzp\" not found" Dec 03 12:32:25 crc kubenswrapper[5052]: I1203 12:32:25.930929 5052 kubelet_pods.go:1007] "Unable to retrieve pull secret, the image pull may not succeed." pod="openstack-kuttl-tests/neutron2174-account-delete-9dnxs" secret="" err="secret \"galera-openstack-dockercfg-w7gzp\" not found" Dec 03 12:32:25 crc kubenswrapper[5052]: I1203 12:32:25.931326 5052 kubelet_pods.go:1007] "Unable to retrieve pull secret, the image pull may not succeed." pod="openstack-kuttl-tests/glance5b9b-account-delete-ggk6q" secret="" err="secret \"galera-openstack-dockercfg-w7gzp\" not found" Dec 03 12:32:25 crc kubenswrapper[5052]: I1203 12:32:25.932218 5052 kubelet_pods.go:1007] "Unable to retrieve pull secret, the image pull may not succeed." pod="openstack-kuttl-tests/cinderc151-account-delete-h4k5b" secret="" err="secret \"galera-openstack-dockercfg-w7gzp\" not found" Dec 03 12:32:25 crc kubenswrapper[5052]: I1203 12:32:25.932350 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/placemente65a-account-delete-s7x52" podUID="ae9024f5-a2b4-49de-aed3-00cb9c806704" containerName="mariadb-account-delete" containerID="cri-o://96cd7d92781623b5aa19e13255fcc36f068db61739a1f3c2145635b5d11cc87a" gracePeriod=30 Dec 03 12:32:25 crc kubenswrapper[5052]: I1203 12:32:25.940599 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/barbican-worker-c899dcc5-n6rdv"] Dec 03 12:32:25 crc kubenswrapper[5052]: I1203 12:32:25.970594 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/novacell115a6-account-delete-pbwnc"] Dec 03 12:32:25 crc kubenswrapper[5052]: I1203 12:32:25.970936 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cfd8af50-a644-4b24-8483-dd344f2218e4-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "cfd8af50-a644-4b24-8483-dd344f2218e4" (UID: "cfd8af50-a644-4b24-8483-dd344f2218e4"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:32:25 crc kubenswrapper[5052]: E1203 12:32:25.974747 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:32:25 crc kubenswrapper[5052]: E1203 12:32:25.974810 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/478e18e8-077e-406e-aa66-19bc1d28df46-operator-scripts podName:478e18e8-077e-406e-aa66-19bc1d28df46 nodeName:}" failed. No retries permitted until 2025-12-03 12:32:29.974792295 +0000 UTC m=+4340.563034059 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/478e18e8-077e-406e-aa66-19bc1d28df46-operator-scripts") pod "novacell046b8-account-delete-nqzlj" (UID: "478e18e8-077e-406e-aa66-19bc1d28df46") : configmap "openstack-scripts" not found Dec 03 12:32:25 crc kubenswrapper[5052]: E1203 12:32:25.974856 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:32:25 crc kubenswrapper[5052]: E1203 12:32:25.974923 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a2ef0804-75c7-43e3-95fa-ce901a561c9b-operator-scripts podName:a2ef0804-75c7-43e3-95fa-ce901a561c9b nodeName:}" failed. No retries permitted until 2025-12-03 12:32:27.974891358 +0000 UTC m=+4338.563133122 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/a2ef0804-75c7-43e3-95fa-ce901a561c9b-operator-scripts") pod "neutron2174-account-delete-9dnxs" (UID: "a2ef0804-75c7-43e3-95fa-ce901a561c9b") : configmap "openstack-scripts" not found Dec 03 12:32:25 crc kubenswrapper[5052]: I1203 12:32:25.975200 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/cfd8af50-a644-4b24-8483-dd344f2218e4-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:25 crc kubenswrapper[5052]: I1203 12:32:25.975216 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/cfd8af50-a644-4b24-8483-dd344f2218e4-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:25 crc kubenswrapper[5052]: I1203 12:32:25.975241 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-jkbfb\" (UniqueName: \"kubernetes.io/projected/cfd8af50-a644-4b24-8483-dd344f2218e4-kube-api-access-jkbfb\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:25 crc kubenswrapper[5052]: I1203 12:32:25.975250 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/cfd8af50-a644-4b24-8483-dd344f2218e4-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:25 crc kubenswrapper[5052]: I1203 12:32:25.980571 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cfd8af50-a644-4b24-8483-dd344f2218e4-config-data" (OuterVolumeSpecName: "config-data") pod "cfd8af50-a644-4b24-8483-dd344f2218e4" (UID: "cfd8af50-a644-4b24-8483-dd344f2218e4"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:32:25 crc kubenswrapper[5052]: I1203 12:32:25.991846 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/novacell115a6-account-delete-pbwnc"] Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.021810 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/swift-proxy-55bdbb59c4-c4cjv"] Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.041674 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cfd8af50-a644-4b24-8483-dd344f2218e4-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "cfd8af50-a644-4b24-8483-dd344f2218e4" (UID: "cfd8af50-a644-4b24-8483-dd344f2218e4"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.060714 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/swift-proxy-55bdbb59c4-c4cjv"] Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.078332 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/cfd8af50-a644-4b24-8483-dd344f2218e4-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:26 crc kubenswrapper[5052]: E1203 12:32:26.078388 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:32:26 crc kubenswrapper[5052]: E1203 12:32:26.078444 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/52bf0c8d-03b3-444b-80b6-13d41f5f9fb9-operator-scripts podName:52bf0c8d-03b3-444b-80b6-13d41f5f9fb9 nodeName:}" failed. No retries permitted until 2025-12-03 12:32:28.078424133 +0000 UTC m=+4338.666665967 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/52bf0c8d-03b3-444b-80b6-13d41f5f9fb9-operator-scripts") pod "glance5b9b-account-delete-ggk6q" (UID: "52bf0c8d-03b3-444b-80b6-13d41f5f9fb9") : configmap "openstack-scripts" not found Dec 03 12:32:26 crc kubenswrapper[5052]: E1203 12:32:26.079152 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:32:26 crc kubenswrapper[5052]: E1203 12:32:26.079899 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/fc428005-b653-4a6a-a50d-133c2b4fb8b8-operator-scripts podName:fc428005-b653-4a6a-a50d-133c2b4fb8b8 nodeName:}" failed. No retries permitted until 2025-12-03 12:32:28.079882995 +0000 UTC m=+4338.668124839 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/fc428005-b653-4a6a-a50d-133c2b4fb8b8-operator-scripts") pod "cinderc151-account-delete-h4k5b" (UID: "fc428005-b653-4a6a-a50d-133c2b4fb8b8") : configmap "openstack-scripts" not found Dec 03 12:32:26 crc kubenswrapper[5052]: E1203 12:32:26.079936 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:32:26 crc kubenswrapper[5052]: E1203 12:32:26.079961 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e6d87a68-5121-4088-a5d8-3f205a787b29-operator-scripts podName:e6d87a68-5121-4088-a5d8-3f205a787b29 nodeName:}" failed. No retries permitted until 2025-12-03 12:32:28.079953377 +0000 UTC m=+4338.668195211 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/e6d87a68-5121-4088-a5d8-3f205a787b29-operator-scripts") pod "novaapia918-account-delete-mk25j" (UID: "e6d87a68-5121-4088-a5d8-3f205a787b29") : configmap "openstack-scripts" not found Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.079986 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/cc8c914a-cb83-48ae-97a7-fca144e96f52-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:26 crc kubenswrapper[5052]: E1203 12:32:26.081632 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:32:26 crc kubenswrapper[5052]: E1203 12:32:26.081677 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8a93d2af-9f8c-4282-8611-526869559ae2-operator-scripts podName:8a93d2af-9f8c-4282-8611-526869559ae2 nodeName:}" failed. No retries permitted until 2025-12-03 12:32:26.581665976 +0000 UTC m=+4337.169907740 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/8a93d2af-9f8c-4282-8611-526869559ae2-operator-scripts") pod "barbican3d67-account-delete-qrp98" (UID: "8a93d2af-9f8c-4282-8611-526869559ae2") : configmap "openstack-scripts" not found Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.081698 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/f12527ba-678e-4ed9-82ff-cec336f607f3-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.081735 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-wr9nk\" (UniqueName: \"kubernetes.io/projected/f12527ba-678e-4ed9-82ff-cec336f607f3-kube-api-access-wr9nk\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.081752 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/cfd8af50-a644-4b24-8483-dd344f2218e4-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.088016 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cfd8af50-a644-4b24-8483-dd344f2218e4-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "cfd8af50-a644-4b24-8483-dd344f2218e4" (UID: "cfd8af50-a644-4b24-8483-dd344f2218e4"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.118391 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-api-78dc8cddf6-kqfz7"] Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.131103 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/barbican-api-78dc8cddf6-kqfz7"] Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.145656 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.152843 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.167778 5052 scope.go:117] "RemoveContainer" containerID="3fd08a7accc11bddbab26e95f99cd86563dcbdee87c9faa5f4e155b8982c03de" Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.184118 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/novacell046b8-account-delete-nqzlj" podStartSLOduration=6.18410024 podStartE2EDuration="6.18410024s" podCreationTimestamp="2025-12-03 12:32:20 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:32:25.963796572 +0000 UTC m=+4336.552038336" watchObservedRunningTime="2025-12-03 12:32:26.18410024 +0000 UTC m=+4336.772342004" Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.187778 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a3b48edc-15d0-4c5e-8a33-96bee20dd43e-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.187800 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/cfd8af50-a644-4b24-8483-dd344f2218e4-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.187809 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/a3b48edc-15d0-4c5e-8a33-96bee20dd43e-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.187818 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/a3b48edc-15d0-4c5e-8a33-96bee20dd43e-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.193562 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/barbican3d67-account-delete-qrp98" podStartSLOduration=6.193543849 podStartE2EDuration="6.193543849s" podCreationTimestamp="2025-12-03 12:32:20 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:32:25.998027999 +0000 UTC m=+4336.586269763" watchObservedRunningTime="2025-12-03 12:32:26.193543849 +0000 UTC m=+4336.781785613" Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.227554 5052 scope.go:117] "RemoveContainer" containerID="e2917e8431b0de8f360760abc00d1838625c47e53148c72eb9f76598a47629bd" Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.232385 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.242636 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.257742 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.259559 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.295163 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tcf7d\" (UniqueName: \"kubernetes.io/projected/39117149-c15d-4753-9324-8a42abe5b007-kube-api-access-tcf7d\") pod \"39117149-c15d-4753-9324-8a42abe5b007\" (UID: \"39117149-c15d-4753-9324-8a42abe5b007\") " Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.295222 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/39117149-c15d-4753-9324-8a42abe5b007-config-data\") pod \"39117149-c15d-4753-9324-8a42abe5b007\" (UID: \"39117149-c15d-4753-9324-8a42abe5b007\") " Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.295291 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/39117149-c15d-4753-9324-8a42abe5b007-nova-metadata-tls-certs\") pod \"39117149-c15d-4753-9324-8a42abe5b007\" (UID: \"39117149-c15d-4753-9324-8a42abe5b007\") " Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.295337 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a65428bc-9326-4ace-a985-9f796b951304-config-data\") pod \"a65428bc-9326-4ace-a985-9f796b951304\" (UID: \"a65428bc-9326-4ace-a985-9f796b951304\") " Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.295380 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/a65428bc-9326-4ace-a985-9f796b951304-public-tls-certs\") pod \"a65428bc-9326-4ace-a985-9f796b951304\" (UID: \"a65428bc-9326-4ace-a985-9f796b951304\") " Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.295427 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/39117149-c15d-4753-9324-8a42abe5b007-combined-ca-bundle\") pod \"39117149-c15d-4753-9324-8a42abe5b007\" (UID: \"39117149-c15d-4753-9324-8a42abe5b007\") " Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.295461 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a65428bc-9326-4ace-a985-9f796b951304-combined-ca-bundle\") pod \"a65428bc-9326-4ace-a985-9f796b951304\" (UID: \"a65428bc-9326-4ace-a985-9f796b951304\") " Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.295527 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/a65428bc-9326-4ace-a985-9f796b951304-scripts\") pod \"a65428bc-9326-4ace-a985-9f796b951304\" (UID: \"a65428bc-9326-4ace-a985-9f796b951304\") " Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.295563 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/a65428bc-9326-4ace-a985-9f796b951304-httpd-run\") pod \"a65428bc-9326-4ace-a985-9f796b951304\" (UID: \"a65428bc-9326-4ace-a985-9f796b951304\") " Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.295594 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/39117149-c15d-4753-9324-8a42abe5b007-logs\") pod \"39117149-c15d-4753-9324-8a42abe5b007\" (UID: \"39117149-c15d-4753-9324-8a42abe5b007\") " Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.295646 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"glance\" (UniqueName: \"kubernetes.io/local-volume/local-storage07-crc\") pod \"a65428bc-9326-4ace-a985-9f796b951304\" (UID: \"a65428bc-9326-4ace-a985-9f796b951304\") " Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.295701 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/a65428bc-9326-4ace-a985-9f796b951304-logs\") pod \"a65428bc-9326-4ace-a985-9f796b951304\" (UID: \"a65428bc-9326-4ace-a985-9f796b951304\") " Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.295752 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-npmcg\" (UniqueName: \"kubernetes.io/projected/a65428bc-9326-4ace-a985-9f796b951304-kube-api-access-npmcg\") pod \"a65428bc-9326-4ace-a985-9f796b951304\" (UID: \"a65428bc-9326-4ace-a985-9f796b951304\") " Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.299665 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/39117149-c15d-4753-9324-8a42abe5b007-logs" (OuterVolumeSpecName: "logs") pod "39117149-c15d-4753-9324-8a42abe5b007" (UID: "39117149-c15d-4753-9324-8a42abe5b007"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.299930 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a65428bc-9326-4ace-a985-9f796b951304-httpd-run" (OuterVolumeSpecName: "httpd-run") pod "a65428bc-9326-4ace-a985-9f796b951304" (UID: "a65428bc-9326-4ace-a985-9f796b951304"). InnerVolumeSpecName "httpd-run". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.300363 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a65428bc-9326-4ace-a985-9f796b951304-logs" (OuterVolumeSpecName: "logs") pod "a65428bc-9326-4ace-a985-9f796b951304" (UID: "a65428bc-9326-4ace-a985-9f796b951304"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.302631 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/39117149-c15d-4753-9324-8a42abe5b007-kube-api-access-tcf7d" (OuterVolumeSpecName: "kube-api-access-tcf7d") pod "39117149-c15d-4753-9324-8a42abe5b007" (UID: "39117149-c15d-4753-9324-8a42abe5b007"). InnerVolumeSpecName "kube-api-access-tcf7d". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.303041 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a65428bc-9326-4ace-a985-9f796b951304-scripts" (OuterVolumeSpecName: "scripts") pod "a65428bc-9326-4ace-a985-9f796b951304" (UID: "a65428bc-9326-4ace-a985-9f796b951304"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.304775 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a65428bc-9326-4ace-a985-9f796b951304-kube-api-access-npmcg" (OuterVolumeSpecName: "kube-api-access-npmcg") pod "a65428bc-9326-4ace-a985-9f796b951304" (UID: "a65428bc-9326-4ace-a985-9f796b951304"). InnerVolumeSpecName "kube-api-access-npmcg". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.317574 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-api-68fc8f9776-n9gml" Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.346406 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage07-crc" (OuterVolumeSpecName: "glance") pod "a65428bc-9326-4ace-a985-9f796b951304" (UID: "a65428bc-9326-4ace-a985-9f796b951304"). InnerVolumeSpecName "local-storage07-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.400694 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/5737900d-97ae-4dcb-a65c-b40fe30cdc50-config-data\") pod \"5737900d-97ae-4dcb-a65c-b40fe30cdc50\" (UID: \"5737900d-97ae-4dcb-a65c-b40fe30cdc50\") " Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.400740 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d2sc9\" (UniqueName: \"kubernetes.io/projected/5737900d-97ae-4dcb-a65c-b40fe30cdc50-kube-api-access-d2sc9\") pod \"5737900d-97ae-4dcb-a65c-b40fe30cdc50\" (UID: \"5737900d-97ae-4dcb-a65c-b40fe30cdc50\") " Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.400773 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/5737900d-97ae-4dcb-a65c-b40fe30cdc50-httpd-run\") pod \"5737900d-97ae-4dcb-a65c-b40fe30cdc50\" (UID: \"5737900d-97ae-4dcb-a65c-b40fe30cdc50\") " Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.400818 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6e0f19c7-34ae-4d8e-9a19-755ae1c20c1c-combined-ca-bundle\") pod \"6e0f19c7-34ae-4d8e-9a19-755ae1c20c1c\" (UID: \"6e0f19c7-34ae-4d8e-9a19-755ae1c20c1c\") " Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.400863 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/82f142ab-5be9-4727-b3c5-102d6f2401fb-combined-ca-bundle\") pod \"82f142ab-5be9-4727-b3c5-102d6f2401fb\" (UID: \"82f142ab-5be9-4727-b3c5-102d6f2401fb\") " Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.400889 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/82f142ab-5be9-4727-b3c5-102d6f2401fb-config-data-custom\") pod \"82f142ab-5be9-4727-b3c5-102d6f2401fb\" (UID: \"82f142ab-5be9-4727-b3c5-102d6f2401fb\") " Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.400932 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"glance\" (UniqueName: \"kubernetes.io/local-volume/local-storage09-crc\") pod \"5737900d-97ae-4dcb-a65c-b40fe30cdc50\" (UID: \"5737900d-97ae-4dcb-a65c-b40fe30cdc50\") " Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.400968 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6e0f19c7-34ae-4d8e-9a19-755ae1c20c1c-config-data\") pod \"6e0f19c7-34ae-4d8e-9a19-755ae1c20c1c\" (UID: \"6e0f19c7-34ae-4d8e-9a19-755ae1c20c1c\") " Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.401009 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/6e0f19c7-34ae-4d8e-9a19-755ae1c20c1c-internal-tls-certs\") pod \"6e0f19c7-34ae-4d8e-9a19-755ae1c20c1c\" (UID: \"6e0f19c7-34ae-4d8e-9a19-755ae1c20c1c\") " Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.401054 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/6e0f19c7-34ae-4d8e-9a19-755ae1c20c1c-etc-machine-id\") pod \"6e0f19c7-34ae-4d8e-9a19-755ae1c20c1c\" (UID: \"6e0f19c7-34ae-4d8e-9a19-755ae1c20c1c\") " Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.401073 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/6e0f19c7-34ae-4d8e-9a19-755ae1c20c1c-logs\") pod \"6e0f19c7-34ae-4d8e-9a19-755ae1c20c1c\" (UID: \"6e0f19c7-34ae-4d8e-9a19-755ae1c20c1c\") " Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.401101 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/82f142ab-5be9-4727-b3c5-102d6f2401fb-logs\") pod \"82f142ab-5be9-4727-b3c5-102d6f2401fb\" (UID: \"82f142ab-5be9-4727-b3c5-102d6f2401fb\") " Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.401131 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-md4n8\" (UniqueName: \"kubernetes.io/projected/82f142ab-5be9-4727-b3c5-102d6f2401fb-kube-api-access-md4n8\") pod \"82f142ab-5be9-4727-b3c5-102d6f2401fb\" (UID: \"82f142ab-5be9-4727-b3c5-102d6f2401fb\") " Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.401178 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/5737900d-97ae-4dcb-a65c-b40fe30cdc50-scripts\") pod \"5737900d-97ae-4dcb-a65c-b40fe30cdc50\" (UID: \"5737900d-97ae-4dcb-a65c-b40fe30cdc50\") " Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.401219 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/82f142ab-5be9-4727-b3c5-102d6f2401fb-public-tls-certs\") pod \"82f142ab-5be9-4727-b3c5-102d6f2401fb\" (UID: \"82f142ab-5be9-4727-b3c5-102d6f2401fb\") " Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.401249 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c6l8r\" (UniqueName: \"kubernetes.io/projected/6e0f19c7-34ae-4d8e-9a19-755ae1c20c1c-kube-api-access-c6l8r\") pod \"6e0f19c7-34ae-4d8e-9a19-755ae1c20c1c\" (UID: \"6e0f19c7-34ae-4d8e-9a19-755ae1c20c1c\") " Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.401290 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/82f142ab-5be9-4727-b3c5-102d6f2401fb-config-data\") pod \"82f142ab-5be9-4727-b3c5-102d6f2401fb\" (UID: \"82f142ab-5be9-4727-b3c5-102d6f2401fb\") " Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.401313 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/6e0f19c7-34ae-4d8e-9a19-755ae1c20c1c-scripts\") pod \"6e0f19c7-34ae-4d8e-9a19-755ae1c20c1c\" (UID: \"6e0f19c7-34ae-4d8e-9a19-755ae1c20c1c\") " Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.401357 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5737900d-97ae-4dcb-a65c-b40fe30cdc50-combined-ca-bundle\") pod \"5737900d-97ae-4dcb-a65c-b40fe30cdc50\" (UID: \"5737900d-97ae-4dcb-a65c-b40fe30cdc50\") " Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.401384 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/5737900d-97ae-4dcb-a65c-b40fe30cdc50-internal-tls-certs\") pod \"5737900d-97ae-4dcb-a65c-b40fe30cdc50\" (UID: \"5737900d-97ae-4dcb-a65c-b40fe30cdc50\") " Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.401409 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/6e0f19c7-34ae-4d8e-9a19-755ae1c20c1c-public-tls-certs\") pod \"6e0f19c7-34ae-4d8e-9a19-755ae1c20c1c\" (UID: \"6e0f19c7-34ae-4d8e-9a19-755ae1c20c1c\") " Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.401440 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/5737900d-97ae-4dcb-a65c-b40fe30cdc50-logs\") pod \"5737900d-97ae-4dcb-a65c-b40fe30cdc50\" (UID: \"5737900d-97ae-4dcb-a65c-b40fe30cdc50\") " Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.401482 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/82f142ab-5be9-4727-b3c5-102d6f2401fb-internal-tls-certs\") pod \"82f142ab-5be9-4727-b3c5-102d6f2401fb\" (UID: \"82f142ab-5be9-4727-b3c5-102d6f2401fb\") " Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.401575 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/6e0f19c7-34ae-4d8e-9a19-755ae1c20c1c-config-data-custom\") pod \"6e0f19c7-34ae-4d8e-9a19-755ae1c20c1c\" (UID: \"6e0f19c7-34ae-4d8e-9a19-755ae1c20c1c\") " Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.402189 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/a65428bc-9326-4ace-a985-9f796b951304-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.402210 5052 reconciler_common.go:293] "Volume detached for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/a65428bc-9326-4ace-a985-9f796b951304-httpd-run\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.402222 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/39117149-c15d-4753-9324-8a42abe5b007-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.402247 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage07-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage07-crc\") on node \"crc\" " Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.402261 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/a65428bc-9326-4ace-a985-9f796b951304-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.402275 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-npmcg\" (UniqueName: \"kubernetes.io/projected/a65428bc-9326-4ace-a985-9f796b951304-kube-api-access-npmcg\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.402290 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-tcf7d\" (UniqueName: \"kubernetes.io/projected/39117149-c15d-4753-9324-8a42abe5b007-kube-api-access-tcf7d\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.403459 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/82f142ab-5be9-4727-b3c5-102d6f2401fb-logs" (OuterVolumeSpecName: "logs") pod "82f142ab-5be9-4727-b3c5-102d6f2401fb" (UID: "82f142ab-5be9-4727-b3c5-102d6f2401fb"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.409286 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6e0f19c7-34ae-4d8e-9a19-755ae1c20c1c-scripts" (OuterVolumeSpecName: "scripts") pod "6e0f19c7-34ae-4d8e-9a19-755ae1c20c1c" (UID: "6e0f19c7-34ae-4d8e-9a19-755ae1c20c1c"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.418682 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5737900d-97ae-4dcb-a65c-b40fe30cdc50-logs" (OuterVolumeSpecName: "logs") pod "5737900d-97ae-4dcb-a65c-b40fe30cdc50" (UID: "5737900d-97ae-4dcb-a65c-b40fe30cdc50"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.422790 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5737900d-97ae-4dcb-a65c-b40fe30cdc50-httpd-run" (OuterVolumeSpecName: "httpd-run") pod "5737900d-97ae-4dcb-a65c-b40fe30cdc50" (UID: "5737900d-97ae-4dcb-a65c-b40fe30cdc50"). InnerVolumeSpecName "httpd-run". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.424984 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/39117149-c15d-4753-9324-8a42abe5b007-config-data" (OuterVolumeSpecName: "config-data") pod "39117149-c15d-4753-9324-8a42abe5b007" (UID: "39117149-c15d-4753-9324-8a42abe5b007"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.425086 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/82f142ab-5be9-4727-b3c5-102d6f2401fb-kube-api-access-md4n8" (OuterVolumeSpecName: "kube-api-access-md4n8") pod "82f142ab-5be9-4727-b3c5-102d6f2401fb" (UID: "82f142ab-5be9-4727-b3c5-102d6f2401fb"). InnerVolumeSpecName "kube-api-access-md4n8". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.425279 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/6e0f19c7-34ae-4d8e-9a19-755ae1c20c1c-etc-machine-id" (OuterVolumeSpecName: "etc-machine-id") pod "6e0f19c7-34ae-4d8e-9a19-755ae1c20c1c" (UID: "6e0f19c7-34ae-4d8e-9a19-755ae1c20c1c"). InnerVolumeSpecName "etc-machine-id". PluginName "kubernetes.io/host-path", VolumeGidValue "" Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.429476 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage09-crc" (OuterVolumeSpecName: "glance") pod "5737900d-97ae-4dcb-a65c-b40fe30cdc50" (UID: "5737900d-97ae-4dcb-a65c-b40fe30cdc50"). InnerVolumeSpecName "local-storage09-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.428182 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e0f19c7-34ae-4d8e-9a19-755ae1c20c1c-logs" (OuterVolumeSpecName: "logs") pod "6e0f19c7-34ae-4d8e-9a19-755ae1c20c1c" (UID: "6e0f19c7-34ae-4d8e-9a19-755ae1c20c1c"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.437987 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6e0f19c7-34ae-4d8e-9a19-755ae1c20c1c-kube-api-access-c6l8r" (OuterVolumeSpecName: "kube-api-access-c6l8r") pod "6e0f19c7-34ae-4d8e-9a19-755ae1c20c1c" (UID: "6e0f19c7-34ae-4d8e-9a19-755ae1c20c1c"). InnerVolumeSpecName "kube-api-access-c6l8r". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.461197 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5737900d-97ae-4dcb-a65c-b40fe30cdc50-kube-api-access-d2sc9" (OuterVolumeSpecName: "kube-api-access-d2sc9") pod "5737900d-97ae-4dcb-a65c-b40fe30cdc50" (UID: "5737900d-97ae-4dcb-a65c-b40fe30cdc50"). InnerVolumeSpecName "kube-api-access-d2sc9". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.462988 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6e0f19c7-34ae-4d8e-9a19-755ae1c20c1c-config-data-custom" (OuterVolumeSpecName: "config-data-custom") pod "6e0f19c7-34ae-4d8e-9a19-755ae1c20c1c" (UID: "6e0f19c7-34ae-4d8e-9a19-755ae1c20c1c"). InnerVolumeSpecName "config-data-custom". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.465329 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5737900d-97ae-4dcb-a65c-b40fe30cdc50-scripts" (OuterVolumeSpecName: "scripts") pod "5737900d-97ae-4dcb-a65c-b40fe30cdc50" (UID: "5737900d-97ae-4dcb-a65c-b40fe30cdc50"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.465411 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/82f142ab-5be9-4727-b3c5-102d6f2401fb-config-data-custom" (OuterVolumeSpecName: "config-data-custom") pod "82f142ab-5be9-4727-b3c5-102d6f2401fb" (UID: "82f142ab-5be9-4727-b3c5-102d6f2401fb"). InnerVolumeSpecName "config-data-custom". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.475063 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.485013 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage07-crc" (UniqueName: "kubernetes.io/local-volume/local-storage07-crc") on node "crc" Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.498413 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a65428bc-9326-4ace-a985-9f796b951304-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "a65428bc-9326-4ace-a985-9f796b951304" (UID: "a65428bc-9326-4ace-a985-9f796b951304"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.504094 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/39117149-c15d-4753-9324-8a42abe5b007-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.504121 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/6e0f19c7-34ae-4d8e-9a19-755ae1c20c1c-config-data-custom\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.504135 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-d2sc9\" (UniqueName: \"kubernetes.io/projected/5737900d-97ae-4dcb-a65c-b40fe30cdc50-kube-api-access-d2sc9\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.504147 5052 reconciler_common.go:293] "Volume detached for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/5737900d-97ae-4dcb-a65c-b40fe30cdc50-httpd-run\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.504157 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/82f142ab-5be9-4727-b3c5-102d6f2401fb-config-data-custom\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.504181 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage09-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage09-crc\") on node \"crc\" " Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.504192 5052 reconciler_common.go:293] "Volume detached for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/6e0f19c7-34ae-4d8e-9a19-755ae1c20c1c-etc-machine-id\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.504200 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/6e0f19c7-34ae-4d8e-9a19-755ae1c20c1c-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.504209 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/82f142ab-5be9-4727-b3c5-102d6f2401fb-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.504219 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-md4n8\" (UniqueName: \"kubernetes.io/projected/82f142ab-5be9-4727-b3c5-102d6f2401fb-kube-api-access-md4n8\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.504229 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a65428bc-9326-4ace-a985-9f796b951304-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.504237 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/5737900d-97ae-4dcb-a65c-b40fe30cdc50-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.504246 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-c6l8r\" (UniqueName: \"kubernetes.io/projected/6e0f19c7-34ae-4d8e-9a19-755ae1c20c1c-kube-api-access-c6l8r\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.504255 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage07-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage07-crc\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.504265 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/6e0f19c7-34ae-4d8e-9a19-755ae1c20c1c-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.504273 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/5737900d-97ae-4dcb-a65c-b40fe30cdc50-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.515677 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/39117149-c15d-4753-9324-8a42abe5b007-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "39117149-c15d-4753-9324-8a42abe5b007" (UID: "39117149-c15d-4753-9324-8a42abe5b007"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.564044 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.594660 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage09-crc" (UniqueName: "kubernetes.io/local-volume/local-storage09-crc") on node "crc" Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.605017 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5737900d-97ae-4dcb-a65c-b40fe30cdc50-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "5737900d-97ae-4dcb-a65c-b40fe30cdc50" (UID: "5737900d-97ae-4dcb-a65c-b40fe30cdc50"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.605536 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/7cbe1c2c-bc04-4232-9901-bc9d134df3d6-public-tls-certs\") pod \"7cbe1c2c-bc04-4232-9901-bc9d134df3d6\" (UID: \"7cbe1c2c-bc04-4232-9901-bc9d134df3d6\") " Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.605643 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/7cbe1c2c-bc04-4232-9901-bc9d134df3d6-logs\") pod \"7cbe1c2c-bc04-4232-9901-bc9d134df3d6\" (UID: \"7cbe1c2c-bc04-4232-9901-bc9d134df3d6\") " Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.605674 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/7cbe1c2c-bc04-4232-9901-bc9d134df3d6-config-data\") pod \"7cbe1c2c-bc04-4232-9901-bc9d134df3d6\" (UID: \"7cbe1c2c-bc04-4232-9901-bc9d134df3d6\") " Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.605720 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9wpdp\" (UniqueName: \"kubernetes.io/projected/7cbe1c2c-bc04-4232-9901-bc9d134df3d6-kube-api-access-9wpdp\") pod \"7cbe1c2c-bc04-4232-9901-bc9d134df3d6\" (UID: \"7cbe1c2c-bc04-4232-9901-bc9d134df3d6\") " Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.605771 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/7cbe1c2c-bc04-4232-9901-bc9d134df3d6-combined-ca-bundle\") pod \"7cbe1c2c-bc04-4232-9901-bc9d134df3d6\" (UID: \"7cbe1c2c-bc04-4232-9901-bc9d134df3d6\") " Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.605857 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/7cbe1c2c-bc04-4232-9901-bc9d134df3d6-internal-tls-certs\") pod \"7cbe1c2c-bc04-4232-9901-bc9d134df3d6\" (UID: \"7cbe1c2c-bc04-4232-9901-bc9d134df3d6\") " Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.607185 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7cbe1c2c-bc04-4232-9901-bc9d134df3d6-logs" (OuterVolumeSpecName: "logs") pod "7cbe1c2c-bc04-4232-9901-bc9d134df3d6" (UID: "7cbe1c2c-bc04-4232-9901-bc9d134df3d6"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.610280 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/39117149-c15d-4753-9324-8a42abe5b007-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.610389 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5737900d-97ae-4dcb-a65c-b40fe30cdc50-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.610462 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/7cbe1c2c-bc04-4232-9901-bc9d134df3d6-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.610575 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage09-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage09-crc\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:26 crc kubenswrapper[5052]: E1203 12:32:26.610334 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:32:26 crc kubenswrapper[5052]: E1203 12:32:26.610732 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8a93d2af-9f8c-4282-8611-526869559ae2-operator-scripts podName:8a93d2af-9f8c-4282-8611-526869559ae2 nodeName:}" failed. No retries permitted until 2025-12-03 12:32:27.610715826 +0000 UTC m=+4338.198957590 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/8a93d2af-9f8c-4282-8611-526869559ae2-operator-scripts") pod "barbican3d67-account-delete-qrp98" (UID: "8a93d2af-9f8c-4282-8611-526869559ae2") : configmap "openstack-scripts" not found Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.624696 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7cbe1c2c-bc04-4232-9901-bc9d134df3d6-kube-api-access-9wpdp" (OuterVolumeSpecName: "kube-api-access-9wpdp") pod "7cbe1c2c-bc04-4232-9901-bc9d134df3d6" (UID: "7cbe1c2c-bc04-4232-9901-bc9d134df3d6"). InnerVolumeSpecName "kube-api-access-9wpdp". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.629130 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a65428bc-9326-4ace-a985-9f796b951304-config-data" (OuterVolumeSpecName: "config-data") pod "a65428bc-9326-4ace-a985-9f796b951304" (UID: "a65428bc-9326-4ace-a985-9f796b951304"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.640098 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/39117149-c15d-4753-9324-8a42abe5b007-nova-metadata-tls-certs" (OuterVolumeSpecName: "nova-metadata-tls-certs") pod "39117149-c15d-4753-9324-8a42abe5b007" (UID: "39117149-c15d-4753-9324-8a42abe5b007"). InnerVolumeSpecName "nova-metadata-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.652855 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6e0f19c7-34ae-4d8e-9a19-755ae1c20c1c-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "6e0f19c7-34ae-4d8e-9a19-755ae1c20c1c" (UID: "6e0f19c7-34ae-4d8e-9a19-755ae1c20c1c"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.678133 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/82f142ab-5be9-4727-b3c5-102d6f2401fb-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "82f142ab-5be9-4727-b3c5-102d6f2401fb" (UID: "82f142ab-5be9-4727-b3c5-102d6f2401fb"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.678412 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-6b8b855786-bltgj" Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.693985 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a65428bc-9326-4ace-a985-9f796b951304-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "a65428bc-9326-4ace-a985-9f796b951304" (UID: "a65428bc-9326-4ace-a985-9f796b951304"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.703000 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7cbe1c2c-bc04-4232-9901-bc9d134df3d6-config-data" (OuterVolumeSpecName: "config-data") pod "7cbe1c2c-bc04-4232-9901-bc9d134df3d6" (UID: "7cbe1c2c-bc04-4232-9901-bc9d134df3d6"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.708339 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6e0f19c7-34ae-4d8e-9a19-755ae1c20c1c-config-data" (OuterVolumeSpecName: "config-data") pod "6e0f19c7-34ae-4d8e-9a19-755ae1c20c1c" (UID: "6e0f19c7-34ae-4d8e-9a19-755ae1c20c1c"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.711367 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/bb1723c7-4e41-4b0e-b6b8-f8024dafdaee-scripts\") pod \"bb1723c7-4e41-4b0e-b6b8-f8024dafdaee\" (UID: \"bb1723c7-4e41-4b0e-b6b8-f8024dafdaee\") " Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.711597 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/bb1723c7-4e41-4b0e-b6b8-f8024dafdaee-etc-machine-id\") pod \"bb1723c7-4e41-4b0e-b6b8-f8024dafdaee\" (UID: \"bb1723c7-4e41-4b0e-b6b8-f8024dafdaee\") " Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.711699 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bz7zz\" (UniqueName: \"kubernetes.io/projected/bb1723c7-4e41-4b0e-b6b8-f8024dafdaee-kube-api-access-bz7zz\") pod \"bb1723c7-4e41-4b0e-b6b8-f8024dafdaee\" (UID: \"bb1723c7-4e41-4b0e-b6b8-f8024dafdaee\") " Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.711925 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/bb1723c7-4e41-4b0e-b6b8-f8024dafdaee-config-data-custom\") pod \"bb1723c7-4e41-4b0e-b6b8-f8024dafdaee\" (UID: \"bb1723c7-4e41-4b0e-b6b8-f8024dafdaee\") " Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.712057 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/bb1723c7-4e41-4b0e-b6b8-f8024dafdaee-config-data\") pod \"bb1723c7-4e41-4b0e-b6b8-f8024dafdaee\" (UID: \"bb1723c7-4e41-4b0e-b6b8-f8024dafdaee\") " Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.712137 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/bb1723c7-4e41-4b0e-b6b8-f8024dafdaee-combined-ca-bundle\") pod \"bb1723c7-4e41-4b0e-b6b8-f8024dafdaee\" (UID: \"bb1723c7-4e41-4b0e-b6b8-f8024dafdaee\") " Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.713215 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a65428bc-9326-4ace-a985-9f796b951304-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.713321 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6e0f19c7-34ae-4d8e-9a19-755ae1c20c1c-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.713410 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/a65428bc-9326-4ace-a985-9f796b951304-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.713492 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6e0f19c7-34ae-4d8e-9a19-755ae1c20c1c-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.713601 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/7cbe1c2c-bc04-4232-9901-bc9d134df3d6-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.713678 5052 reconciler_common.go:293] "Volume detached for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/39117149-c15d-4753-9324-8a42abe5b007-nova-metadata-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.713750 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/82f142ab-5be9-4727-b3c5-102d6f2401fb-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.713831 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-9wpdp\" (UniqueName: \"kubernetes.io/projected/7cbe1c2c-bc04-4232-9901-bc9d134df3d6-kube-api-access-9wpdp\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.714050 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/bb1723c7-4e41-4b0e-b6b8-f8024dafdaee-etc-machine-id" (OuterVolumeSpecName: "etc-machine-id") pod "bb1723c7-4e41-4b0e-b6b8-f8024dafdaee" (UID: "bb1723c7-4e41-4b0e-b6b8-f8024dafdaee"). InnerVolumeSpecName "etc-machine-id". PluginName "kubernetes.io/host-path", VolumeGidValue "" Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.722303 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bb1723c7-4e41-4b0e-b6b8-f8024dafdaee-config-data-custom" (OuterVolumeSpecName: "config-data-custom") pod "bb1723c7-4e41-4b0e-b6b8-f8024dafdaee" (UID: "bb1723c7-4e41-4b0e-b6b8-f8024dafdaee"). InnerVolumeSpecName "config-data-custom". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.729350 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bb1723c7-4e41-4b0e-b6b8-f8024dafdaee-scripts" (OuterVolumeSpecName: "scripts") pod "bb1723c7-4e41-4b0e-b6b8-f8024dafdaee" (UID: "bb1723c7-4e41-4b0e-b6b8-f8024dafdaee"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.734957 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/82f142ab-5be9-4727-b3c5-102d6f2401fb-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "82f142ab-5be9-4727-b3c5-102d6f2401fb" (UID: "82f142ab-5be9-4727-b3c5-102d6f2401fb"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.734916 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bb1723c7-4e41-4b0e-b6b8-f8024dafdaee-kube-api-access-bz7zz" (OuterVolumeSpecName: "kube-api-access-bz7zz") pod "bb1723c7-4e41-4b0e-b6b8-f8024dafdaee" (UID: "bb1723c7-4e41-4b0e-b6b8-f8024dafdaee"). InnerVolumeSpecName "kube-api-access-bz7zz". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.735072 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5737900d-97ae-4dcb-a65c-b40fe30cdc50-config-data" (OuterVolumeSpecName: "config-data") pod "5737900d-97ae-4dcb-a65c-b40fe30cdc50" (UID: "5737900d-97ae-4dcb-a65c-b40fe30cdc50"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.736673 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7cbe1c2c-bc04-4232-9901-bc9d134df3d6-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "7cbe1c2c-bc04-4232-9901-bc9d134df3d6" (UID: "7cbe1c2c-bc04-4232-9901-bc9d134df3d6"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.753145 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6e0f19c7-34ae-4d8e-9a19-755ae1c20c1c-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "6e0f19c7-34ae-4d8e-9a19-755ae1c20c1c" (UID: "6e0f19c7-34ae-4d8e-9a19-755ae1c20c1c"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.756278 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/82f142ab-5be9-4727-b3c5-102d6f2401fb-config-data" (OuterVolumeSpecName: "config-data") pod "82f142ab-5be9-4727-b3c5-102d6f2401fb" (UID: "82f142ab-5be9-4727-b3c5-102d6f2401fb"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.758536 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6e0f19c7-34ae-4d8e-9a19-755ae1c20c1c-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "6e0f19c7-34ae-4d8e-9a19-755ae1c20c1c" (UID: "6e0f19c7-34ae-4d8e-9a19-755ae1c20c1c"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.761762 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/82f142ab-5be9-4727-b3c5-102d6f2401fb-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "82f142ab-5be9-4727-b3c5-102d6f2401fb" (UID: "82f142ab-5be9-4727-b3c5-102d6f2401fb"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.770484 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7cbe1c2c-bc04-4232-9901-bc9d134df3d6-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "7cbe1c2c-bc04-4232-9901-bc9d134df3d6" (UID: "7cbe1c2c-bc04-4232-9901-bc9d134df3d6"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.777015 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5737900d-97ae-4dcb-a65c-b40fe30cdc50-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "5737900d-97ae-4dcb-a65c-b40fe30cdc50" (UID: "5737900d-97ae-4dcb-a65c-b40fe30cdc50"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.800665 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7cbe1c2c-bc04-4232-9901-bc9d134df3d6-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "7cbe1c2c-bc04-4232-9901-bc9d134df3d6" (UID: "7cbe1c2c-bc04-4232-9901-bc9d134df3d6"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.801128 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bb1723c7-4e41-4b0e-b6b8-f8024dafdaee-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "bb1723c7-4e41-4b0e-b6b8-f8024dafdaee" (UID: "bb1723c7-4e41-4b0e-b6b8-f8024dafdaee"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.815478 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/43933abc-db67-4ce5-9f30-6f8dbf3f59c8-credential-keys\") pod \"43933abc-db67-4ce5-9f30-6f8dbf3f59c8\" (UID: \"43933abc-db67-4ce5-9f30-6f8dbf3f59c8\") " Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.815629 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/43933abc-db67-4ce5-9f30-6f8dbf3f59c8-config-data\") pod \"43933abc-db67-4ce5-9f30-6f8dbf3f59c8\" (UID: \"43933abc-db67-4ce5-9f30-6f8dbf3f59c8\") " Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.816177 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/43933abc-db67-4ce5-9f30-6f8dbf3f59c8-scripts\") pod \"43933abc-db67-4ce5-9f30-6f8dbf3f59c8\" (UID: \"43933abc-db67-4ce5-9f30-6f8dbf3f59c8\") " Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.816222 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/43933abc-db67-4ce5-9f30-6f8dbf3f59c8-internal-tls-certs\") pod \"43933abc-db67-4ce5-9f30-6f8dbf3f59c8\" (UID: \"43933abc-db67-4ce5-9f30-6f8dbf3f59c8\") " Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.816482 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/43933abc-db67-4ce5-9f30-6f8dbf3f59c8-public-tls-certs\") pod \"43933abc-db67-4ce5-9f30-6f8dbf3f59c8\" (UID: \"43933abc-db67-4ce5-9f30-6f8dbf3f59c8\") " Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.816566 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/43933abc-db67-4ce5-9f30-6f8dbf3f59c8-fernet-keys\") pod \"43933abc-db67-4ce5-9f30-6f8dbf3f59c8\" (UID: \"43933abc-db67-4ce5-9f30-6f8dbf3f59c8\") " Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.816635 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xq8cj\" (UniqueName: \"kubernetes.io/projected/43933abc-db67-4ce5-9f30-6f8dbf3f59c8-kube-api-access-xq8cj\") pod \"43933abc-db67-4ce5-9f30-6f8dbf3f59c8\" (UID: \"43933abc-db67-4ce5-9f30-6f8dbf3f59c8\") " Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.816690 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/43933abc-db67-4ce5-9f30-6f8dbf3f59c8-combined-ca-bundle\") pod \"43933abc-db67-4ce5-9f30-6f8dbf3f59c8\" (UID: \"43933abc-db67-4ce5-9f30-6f8dbf3f59c8\") " Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.818080 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/82f142ab-5be9-4727-b3c5-102d6f2401fb-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.818133 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/7cbe1c2c-bc04-4232-9901-bc9d134df3d6-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.818147 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/bb1723c7-4e41-4b0e-b6b8-f8024dafdaee-config-data-custom\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.818160 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/5737900d-97ae-4dcb-a65c-b40fe30cdc50-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.818171 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/bb1723c7-4e41-4b0e-b6b8-f8024dafdaee-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.818206 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/7cbe1c2c-bc04-4232-9901-bc9d134df3d6-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.818217 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/6e0f19c7-34ae-4d8e-9a19-755ae1c20c1c-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.818228 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/bb1723c7-4e41-4b0e-b6b8-f8024dafdaee-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.818239 5052 reconciler_common.go:293] "Volume detached for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/bb1723c7-4e41-4b0e-b6b8-f8024dafdaee-etc-machine-id\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.818252 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-bz7zz\" (UniqueName: \"kubernetes.io/projected/bb1723c7-4e41-4b0e-b6b8-f8024dafdaee-kube-api-access-bz7zz\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.818289 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/82f142ab-5be9-4727-b3c5-102d6f2401fb-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.818300 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/7cbe1c2c-bc04-4232-9901-bc9d134df3d6-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.818315 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/82f142ab-5be9-4727-b3c5-102d6f2401fb-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.818327 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/5737900d-97ae-4dcb-a65c-b40fe30cdc50-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.818339 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/6e0f19c7-34ae-4d8e-9a19-755ae1c20c1c-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.818961 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/43933abc-db67-4ce5-9f30-6f8dbf3f59c8-scripts" (OuterVolumeSpecName: "scripts") pod "43933abc-db67-4ce5-9f30-6f8dbf3f59c8" (UID: "43933abc-db67-4ce5-9f30-6f8dbf3f59c8"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.819107 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/43933abc-db67-4ce5-9f30-6f8dbf3f59c8-credential-keys" (OuterVolumeSpecName: "credential-keys") pod "43933abc-db67-4ce5-9f30-6f8dbf3f59c8" (UID: "43933abc-db67-4ce5-9f30-6f8dbf3f59c8"). InnerVolumeSpecName "credential-keys". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.821489 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/43933abc-db67-4ce5-9f30-6f8dbf3f59c8-kube-api-access-xq8cj" (OuterVolumeSpecName: "kube-api-access-xq8cj") pod "43933abc-db67-4ce5-9f30-6f8dbf3f59c8" (UID: "43933abc-db67-4ce5-9f30-6f8dbf3f59c8"). InnerVolumeSpecName "kube-api-access-xq8cj". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.822671 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/43933abc-db67-4ce5-9f30-6f8dbf3f59c8-fernet-keys" (OuterVolumeSpecName: "fernet-keys") pod "43933abc-db67-4ce5-9f30-6f8dbf3f59c8" (UID: "43933abc-db67-4ce5-9f30-6f8dbf3f59c8"). InnerVolumeSpecName "fernet-keys". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.852582 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bb1723c7-4e41-4b0e-b6b8-f8024dafdaee-config-data" (OuterVolumeSpecName: "config-data") pod "bb1723c7-4e41-4b0e-b6b8-f8024dafdaee" (UID: "bb1723c7-4e41-4b0e-b6b8-f8024dafdaee"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.854952 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/43933abc-db67-4ce5-9f30-6f8dbf3f59c8-config-data" (OuterVolumeSpecName: "config-data") pod "43933abc-db67-4ce5-9f30-6f8dbf3f59c8" (UID: "43933abc-db67-4ce5-9f30-6f8dbf3f59c8"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.855595 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/43933abc-db67-4ce5-9f30-6f8dbf3f59c8-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "43933abc-db67-4ce5-9f30-6f8dbf3f59c8" (UID: "43933abc-db67-4ce5-9f30-6f8dbf3f59c8"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.870612 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/43933abc-db67-4ce5-9f30-6f8dbf3f59c8-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "43933abc-db67-4ce5-9f30-6f8dbf3f59c8" (UID: "43933abc-db67-4ce5-9f30-6f8dbf3f59c8"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.872883 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/43933abc-db67-4ce5-9f30-6f8dbf3f59c8-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "43933abc-db67-4ce5-9f30-6f8dbf3f59c8" (UID: "43933abc-db67-4ce5-9f30-6f8dbf3f59c8"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.921182 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/43933abc-db67-4ce5-9f30-6f8dbf3f59c8-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.921236 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/43933abc-db67-4ce5-9f30-6f8dbf3f59c8-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.921250 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/bb1723c7-4e41-4b0e-b6b8-f8024dafdaee-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.921264 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/43933abc-db67-4ce5-9f30-6f8dbf3f59c8-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.921276 5052 reconciler_common.go:293] "Volume detached for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/43933abc-db67-4ce5-9f30-6f8dbf3f59c8-fernet-keys\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.921292 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-xq8cj\" (UniqueName: \"kubernetes.io/projected/43933abc-db67-4ce5-9f30-6f8dbf3f59c8-kube-api-access-xq8cj\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.921305 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/43933abc-db67-4ce5-9f30-6f8dbf3f59c8-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.921317 5052 reconciler_common.go:293] "Volume detached for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/43933abc-db67-4ce5-9f30-6f8dbf3f59c8-credential-keys\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.921329 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/43933abc-db67-4ce5-9f30-6f8dbf3f59c8-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:26 crc kubenswrapper[5052]: E1203 12:32:26.921405 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/rabbitmq-config-data: configmap "rabbitmq-config-data" not found Dec 03 12:32:26 crc kubenswrapper[5052]: E1203 12:32:26.921497 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2c03f99a-98fd-4e86-ad29-fb69e94bd436-config-data podName:2c03f99a-98fd-4e86-ad29-fb69e94bd436 nodeName:}" failed. No retries permitted until 2025-12-03 12:32:34.921477886 +0000 UTC m=+4345.509719660 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "config-data" (UniqueName: "kubernetes.io/configmap/2c03f99a-98fd-4e86-ad29-fb69e94bd436-config-data") pod "rabbitmq-server-0" (UID: "2c03f99a-98fd-4e86-ad29-fb69e94bd436") : configmap "rabbitmq-config-data" not found Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.952507 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"bb1723c7-4e41-4b0e-b6b8-f8024dafdaee","Type":"ContainerDied","Data":"0416fe74f81dcb0b84e39cd58ebd5f9449a6bd5290a02aed47a52edaf01b8dda"} Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.952574 5052 scope.go:117] "RemoveContainer" containerID="ab269ac11d99aed29d8c53741cb5bdd97ed1fe052bc19e5e9c3e0f4152eaeed4" Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.952701 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.961703 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"a65428bc-9326-4ace-a985-9f796b951304","Type":"ContainerDied","Data":"14589c054b59a52bc21c00362eb9bafdebaddd153d062b3e5f87e8a4845e9a19"} Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.961825 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.977026 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-68fc8f9776-n9gml" event={"ID":"82f142ab-5be9-4727-b3c5-102d6f2401fb","Type":"ContainerDied","Data":"4b1853107a72a495dcaac5832fb35d2f64fd74e87c388c0c4a4f6c88669f2664"} Dec 03 12:32:26 crc kubenswrapper[5052]: I1203 12:32:26.977060 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-api-68fc8f9776-n9gml" Dec 03 12:32:27 crc kubenswrapper[5052]: I1203 12:32:27.005856 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"6e0f19c7-34ae-4d8e-9a19-755ae1c20c1c","Type":"ContainerDied","Data":"1b451d3cdf806d98af920f4369d1f2e05a91d7cd6f19439441f6f14eafe9ba01"} Dec 03 12:32:27 crc kubenswrapper[5052]: I1203 12:32:27.006035 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:32:27 crc kubenswrapper[5052]: I1203 12:32:27.024065 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-hjfg4" event={"ID":"00bbba79-7529-4309-af53-b464acf28194","Type":"ContainerStarted","Data":"856900df1677628daad9107fd3739a17cd9695e68c3f78d8a83df16b291e65ea"} Dec 03 12:32:27 crc kubenswrapper[5052]: E1203 12:32:27.025744 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:32:27 crc kubenswrapper[5052]: E1203 12:32:27.025816 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ae9024f5-a2b4-49de-aed3-00cb9c806704-operator-scripts podName:ae9024f5-a2b4-49de-aed3-00cb9c806704 nodeName:}" failed. No retries permitted until 2025-12-03 12:32:31.025801604 +0000 UTC m=+4341.614043368 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/ae9024f5-a2b4-49de-aed3-00cb9c806704-operator-scripts") pod "placemente65a-account-delete-s7x52" (UID: "ae9024f5-a2b4-49de-aed3-00cb9c806704") : configmap "openstack-scripts" not found Dec 03 12:32:27 crc kubenswrapper[5052]: I1203 12:32:27.025830 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-hjfg4" Dec 03 12:32:27 crc kubenswrapper[5052]: I1203 12:32:27.030715 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"39117149-c15d-4753-9324-8a42abe5b007","Type":"ContainerDied","Data":"4e54eebec6cc33b44308d0d2910a2dc282c24d4bd3abab322cc14f37bec326e1"} Dec 03 12:32:27 crc kubenswrapper[5052]: I1203 12:32:27.032685 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:32:27 crc kubenswrapper[5052]: I1203 12:32:27.033599 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-65444bfc86-th5pw" event={"ID":"cfd8af50-a644-4b24-8483-dd344f2218e4","Type":"ContainerDied","Data":"3d7601570722d86f8224664e7bb57ae02c41b3b00b91c1982ff975c685f871f9"} Dec 03 12:32:27 crc kubenswrapper[5052]: I1203 12:32:27.033719 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-65444bfc86-th5pw" Dec 03 12:32:27 crc kubenswrapper[5052]: I1203 12:32:27.042465 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"5737900d-97ae-4dcb-a65c-b40fe30cdc50","Type":"ContainerDied","Data":"87ab449ce6632b6498e866cff688d31a02f7702881f0e2b6e60f3374d2103eab"} Dec 03 12:32:27 crc kubenswrapper[5052]: I1203 12:32:27.042572 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:32:27 crc kubenswrapper[5052]: I1203 12:32:27.052195 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-hjfg4" podStartSLOduration=7.052169216 podStartE2EDuration="7.052169216s" podCreationTimestamp="2025-12-03 12:32:20 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:32:27.046305179 +0000 UTC m=+4337.634546943" watchObservedRunningTime="2025-12-03 12:32:27.052169216 +0000 UTC m=+4337.640410980" Dec 03 12:32:27 crc kubenswrapper[5052]: I1203 12:32:27.056795 5052 generic.go:334] "Generic (PLEG): container finished" podID="43933abc-db67-4ce5-9f30-6f8dbf3f59c8" containerID="aec2d2ca3d9979ac47cbc62e58c9a1c7b06d1e13c5205fc585d92567ca0e4ec3" exitCode=0 Dec 03 12:32:27 crc kubenswrapper[5052]: I1203 12:32:27.056871 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-6b8b855786-bltgj" event={"ID":"43933abc-db67-4ce5-9f30-6f8dbf3f59c8","Type":"ContainerDied","Data":"aec2d2ca3d9979ac47cbc62e58c9a1c7b06d1e13c5205fc585d92567ca0e4ec3"} Dec 03 12:32:27 crc kubenswrapper[5052]: I1203 12:32:27.056902 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-6b8b855786-bltgj" event={"ID":"43933abc-db67-4ce5-9f30-6f8dbf3f59c8","Type":"ContainerDied","Data":"d2387624a050b94b6cfa3ba971af84ee298e7c7fed409795b6b82257adc33a22"} Dec 03 12:32:27 crc kubenswrapper[5052]: I1203 12:32:27.056968 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-6b8b855786-bltgj" Dec 03 12:32:27 crc kubenswrapper[5052]: I1203 12:32:27.061476 5052 generic.go:334] "Generic (PLEG): container finished" podID="6938204c-1fe8-4f03-ac16-ec9e19e090f8" containerID="d94e0139dcad346a24d75f7bd285773c0fe1edad0369015fe7b299454967acdd" exitCode=0 Dec 03 12:32:27 crc kubenswrapper[5052]: I1203 12:32:27.061556 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-galera-0" event={"ID":"6938204c-1fe8-4f03-ac16-ec9e19e090f8","Type":"ContainerDied","Data":"d94e0139dcad346a24d75f7bd285773c0fe1edad0369015fe7b299454967acdd"} Dec 03 12:32:27 crc kubenswrapper[5052]: I1203 12:32:27.063906 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:32:27 crc kubenswrapper[5052]: I1203 12:32:27.063898 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"7cbe1c2c-bc04-4232-9901-bc9d134df3d6","Type":"ContainerDied","Data":"b8128fd869a1a047b25167700c1bc9dedce09fba89a0659838f2d02e09eb65e6"} Dec 03 12:32:27 crc kubenswrapper[5052]: I1203 12:32:27.065588 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/cinderc151-account-delete-h4k5b" podUID="fc428005-b653-4a6a-a50d-133c2b4fb8b8" containerName="mariadb-account-delete" containerID="cri-o://f45508992c6ce3df52ea0e08b462396de8f82c3d69fcce583cf8d26d1c4c3946" gracePeriod=30 Dec 03 12:32:27 crc kubenswrapper[5052]: I1203 12:32:27.065760 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/barbican3d67-account-delete-qrp98" podUID="8a93d2af-9f8c-4282-8611-526869559ae2" containerName="mariadb-account-delete" containerID="cri-o://444eb8a103a3fd9c5c270baa3559817c3b3a7da9d570c02241c847c5911ad88f" gracePeriod=30 Dec 03 12:32:27 crc kubenswrapper[5052]: I1203 12:32:27.066776 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/novacell046b8-account-delete-nqzlj" podUID="478e18e8-077e-406e-aa66-19bc1d28df46" containerName="mariadb-account-delete" containerID="cri-o://7abe534b28e0d727623c30c2ee8d6feee7f3db4244b68e027625e6a8eab08cc4" gracePeriod=30 Dec 03 12:32:27 crc kubenswrapper[5052]: I1203 12:32:27.066919 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance5b9b-account-delete-ggk6q" podUID="52bf0c8d-03b3-444b-80b6-13d41f5f9fb9" containerName="mariadb-account-delete" containerID="cri-o://af107e2e82074f5c5aba6f4fcc9960e916fe7e1e038d4e5c3098122ab4c96e1b" gracePeriod=30 Dec 03 12:32:27 crc kubenswrapper[5052]: I1203 12:32:27.067012 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/novaapia918-account-delete-mk25j" podUID="e6d87a68-5121-4088-a5d8-3f205a787b29" containerName="mariadb-account-delete" containerID="cri-o://11243a11f3f92fefd822d688612a3f588dc1510f45d518e358b4b10cfe36371c" gracePeriod=30 Dec 03 12:32:27 crc kubenswrapper[5052]: I1203 12:32:27.067107 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/neutron2174-account-delete-9dnxs" podUID="a2ef0804-75c7-43e3-95fa-ce901a561c9b" containerName="mariadb-account-delete" containerID="cri-o://1527475535a0407481384ca6a15c4f2f9da6de7e6bcda257db051f93a93b1aa9" gracePeriod=30 Dec 03 12:32:27 crc kubenswrapper[5052]: I1203 12:32:27.195927 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:32:27 crc kubenswrapper[5052]: I1203 12:32:27.205728 5052 scope.go:117] "RemoveContainer" containerID="94df1fd3c0977441ba1cf70763e96ac140d6a232f2c0d6cfe09fb6aa6ae75dc3" Dec 03 12:32:27 crc kubenswrapper[5052]: I1203 12:32:27.246827 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 12:32:27 crc kubenswrapper[5052]: I1203 12:32:27.254078 5052 scope.go:117] "RemoveContainer" containerID="ddde80cbe1c1205957f18817d4db06fabccc08242a25c2d74bcd5a4c61d8bd5d" Dec 03 12:32:27 crc kubenswrapper[5052]: I1203 12:32:27.257855 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 12:32:27 crc kubenswrapper[5052]: I1203 12:32:27.276991 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/placement-65444bfc86-th5pw"] Dec 03 12:32:27 crc kubenswrapper[5052]: I1203 12:32:27.290397 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/placement-65444bfc86-th5pw"] Dec 03 12:32:27 crc kubenswrapper[5052]: I1203 12:32:27.297537 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-api-68fc8f9776-n9gml"] Dec 03 12:32:27 crc kubenswrapper[5052]: I1203 12:32:27.298309 5052 scope.go:117] "RemoveContainer" containerID="caa8dd02dbab945ca9d553d926ff4896db88a3b315e5384ca071cd388a2f2766" Dec 03 12:32:27 crc kubenswrapper[5052]: I1203 12:32:27.304557 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/barbican-api-68fc8f9776-n9gml"] Dec 03 12:32:27 crc kubenswrapper[5052]: I1203 12:32:27.318473 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:32:27 crc kubenswrapper[5052]: I1203 12:32:27.328952 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:32:27 crc kubenswrapper[5052]: I1203 12:32:27.331831 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"mysql-db\" (UniqueName: \"kubernetes.io/local-volume/local-storage02-crc\") pod \"6938204c-1fe8-4f03-ac16-ec9e19e090f8\" (UID: \"6938204c-1fe8-4f03-ac16-ec9e19e090f8\") " Dec 03 12:32:27 crc kubenswrapper[5052]: I1203 12:32:27.331948 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/6938204c-1fe8-4f03-ac16-ec9e19e090f8-config-data-generated\") pod \"6938204c-1fe8-4f03-ac16-ec9e19e090f8\" (UID: \"6938204c-1fe8-4f03-ac16-ec9e19e090f8\") " Dec 03 12:32:27 crc kubenswrapper[5052]: I1203 12:32:27.332004 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/6938204c-1fe8-4f03-ac16-ec9e19e090f8-kolla-config\") pod \"6938204c-1fe8-4f03-ac16-ec9e19e090f8\" (UID: \"6938204c-1fe8-4f03-ac16-ec9e19e090f8\") " Dec 03 12:32:27 crc kubenswrapper[5052]: I1203 12:32:27.332045 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/6938204c-1fe8-4f03-ac16-ec9e19e090f8-config-data-default\") pod \"6938204c-1fe8-4f03-ac16-ec9e19e090f8\" (UID: \"6938204c-1fe8-4f03-ac16-ec9e19e090f8\") " Dec 03 12:32:27 crc kubenswrapper[5052]: I1203 12:32:27.332118 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/6938204c-1fe8-4f03-ac16-ec9e19e090f8-operator-scripts\") pod \"6938204c-1fe8-4f03-ac16-ec9e19e090f8\" (UID: \"6938204c-1fe8-4f03-ac16-ec9e19e090f8\") " Dec 03 12:32:27 crc kubenswrapper[5052]: I1203 12:32:27.334910 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rlwzr\" (UniqueName: \"kubernetes.io/projected/6938204c-1fe8-4f03-ac16-ec9e19e090f8-kube-api-access-rlwzr\") pod \"6938204c-1fe8-4f03-ac16-ec9e19e090f8\" (UID: \"6938204c-1fe8-4f03-ac16-ec9e19e090f8\") " Dec 03 12:32:27 crc kubenswrapper[5052]: I1203 12:32:27.334964 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/6938204c-1fe8-4f03-ac16-ec9e19e090f8-galera-tls-certs\") pod \"6938204c-1fe8-4f03-ac16-ec9e19e090f8\" (UID: \"6938204c-1fe8-4f03-ac16-ec9e19e090f8\") " Dec 03 12:32:27 crc kubenswrapper[5052]: I1203 12:32:27.334991 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6938204c-1fe8-4f03-ac16-ec9e19e090f8-combined-ca-bundle\") pod \"6938204c-1fe8-4f03-ac16-ec9e19e090f8\" (UID: \"6938204c-1fe8-4f03-ac16-ec9e19e090f8\") " Dec 03 12:32:27 crc kubenswrapper[5052]: I1203 12:32:27.335024 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6938204c-1fe8-4f03-ac16-ec9e19e090f8-kolla-config" (OuterVolumeSpecName: "kolla-config") pod "6938204c-1fe8-4f03-ac16-ec9e19e090f8" (UID: "6938204c-1fe8-4f03-ac16-ec9e19e090f8"). InnerVolumeSpecName "kolla-config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:32:27 crc kubenswrapper[5052]: I1203 12:32:27.335321 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6938204c-1fe8-4f03-ac16-ec9e19e090f8-config-data-generated" (OuterVolumeSpecName: "config-data-generated") pod "6938204c-1fe8-4f03-ac16-ec9e19e090f8" (UID: "6938204c-1fe8-4f03-ac16-ec9e19e090f8"). InnerVolumeSpecName "config-data-generated". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:32:27 crc kubenswrapper[5052]: I1203 12:32:27.335909 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6938204c-1fe8-4f03-ac16-ec9e19e090f8-config-data-default" (OuterVolumeSpecName: "config-data-default") pod "6938204c-1fe8-4f03-ac16-ec9e19e090f8" (UID: "6938204c-1fe8-4f03-ac16-ec9e19e090f8"). InnerVolumeSpecName "config-data-default". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:32:27 crc kubenswrapper[5052]: I1203 12:32:27.336009 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6938204c-1fe8-4f03-ac16-ec9e19e090f8-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "6938204c-1fe8-4f03-ac16-ec9e19e090f8" (UID: "6938204c-1fe8-4f03-ac16-ec9e19e090f8"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:32:27 crc kubenswrapper[5052]: I1203 12:32:27.336359 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/6938204c-1fe8-4f03-ac16-ec9e19e090f8-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:27 crc kubenswrapper[5052]: I1203 12:32:27.336380 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/6938204c-1fe8-4f03-ac16-ec9e19e090f8-config-data-generated\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:27 crc kubenswrapper[5052]: I1203 12:32:27.336394 5052 reconciler_common.go:293] "Volume detached for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/6938204c-1fe8-4f03-ac16-ec9e19e090f8-kolla-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:27 crc kubenswrapper[5052]: I1203 12:32:27.336406 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/6938204c-1fe8-4f03-ac16-ec9e19e090f8-config-data-default\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:27 crc kubenswrapper[5052]: I1203 12:32:27.337337 5052 scope.go:117] "RemoveContainer" containerID="74fe4216c2c90357dad59af2f1e20e908e8583d49e0f2a8d15cd86ca1fc755d6" Dec 03 12:32:27 crc kubenswrapper[5052]: I1203 12:32:27.342059 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:32:27 crc kubenswrapper[5052]: I1203 12:32:27.342364 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6938204c-1fe8-4f03-ac16-ec9e19e090f8-kube-api-access-rlwzr" (OuterVolumeSpecName: "kube-api-access-rlwzr") pod "6938204c-1fe8-4f03-ac16-ec9e19e090f8" (UID: "6938204c-1fe8-4f03-ac16-ec9e19e090f8"). InnerVolumeSpecName "kube-api-access-rlwzr". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:32:27 crc kubenswrapper[5052]: I1203 12:32:27.355656 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage02-crc" (OuterVolumeSpecName: "mysql-db") pod "6938204c-1fe8-4f03-ac16-ec9e19e090f8" (UID: "6938204c-1fe8-4f03-ac16-ec9e19e090f8"). InnerVolumeSpecName "local-storage02-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 12:32:27 crc kubenswrapper[5052]: I1203 12:32:27.359600 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:32:27 crc kubenswrapper[5052]: I1203 12:32:27.378833 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6938204c-1fe8-4f03-ac16-ec9e19e090f8-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "6938204c-1fe8-4f03-ac16-ec9e19e090f8" (UID: "6938204c-1fe8-4f03-ac16-ec9e19e090f8"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:32:27 crc kubenswrapper[5052]: I1203 12:32:27.382336 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/keystone-6b8b855786-bltgj"] Dec 03 12:32:27 crc kubenswrapper[5052]: I1203 12:32:27.389741 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/keystone-6b8b855786-bltgj"] Dec 03 12:32:27 crc kubenswrapper[5052]: I1203 12:32:27.396554 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:32:27 crc kubenswrapper[5052]: I1203 12:32:27.402837 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:32:27 crc kubenswrapper[5052]: I1203 12:32:27.402945 5052 scope.go:117] "RemoveContainer" containerID="ca30ccff84d103c471a54af8538ca8e1d53df3754e5f8a3e9c4ab0869c3a23d6" Dec 03 12:32:27 crc kubenswrapper[5052]: I1203 12:32:27.405793 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6938204c-1fe8-4f03-ac16-ec9e19e090f8-galera-tls-certs" (OuterVolumeSpecName: "galera-tls-certs") pod "6938204c-1fe8-4f03-ac16-ec9e19e090f8" (UID: "6938204c-1fe8-4f03-ac16-ec9e19e090f8"). InnerVolumeSpecName "galera-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:32:27 crc kubenswrapper[5052]: I1203 12:32:27.409573 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:32:27 crc kubenswrapper[5052]: I1203 12:32:27.421562 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:32:27 crc kubenswrapper[5052]: I1203 12:32:27.426423 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 12:32:27 crc kubenswrapper[5052]: I1203 12:32:27.427644 5052 scope.go:117] "RemoveContainer" containerID="e387dc9d87ea03a56050ef8cbe08349409d39a0cdecb55b0fe4e091ce7e2b2f4" Dec 03 12:32:27 crc kubenswrapper[5052]: I1203 12:32:27.434724 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 12:32:27 crc kubenswrapper[5052]: I1203 12:32:27.438862 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage02-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage02-crc\") on node \"crc\" " Dec 03 12:32:27 crc kubenswrapper[5052]: I1203 12:32:27.439105 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-rlwzr\" (UniqueName: \"kubernetes.io/projected/6938204c-1fe8-4f03-ac16-ec9e19e090f8-kube-api-access-rlwzr\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:27 crc kubenswrapper[5052]: I1203 12:32:27.439198 5052 reconciler_common.go:293] "Volume detached for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/6938204c-1fe8-4f03-ac16-ec9e19e090f8-galera-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:27 crc kubenswrapper[5052]: I1203 12:32:27.439282 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6938204c-1fe8-4f03-ac16-ec9e19e090f8-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:27 crc kubenswrapper[5052]: I1203 12:32:27.456215 5052 scope.go:117] "RemoveContainer" containerID="fa8e6470aab88dd31d2b44427546df1694d629e5bea31eca80eadf84f064db9d" Dec 03 12:32:27 crc kubenswrapper[5052]: I1203 12:32:27.478283 5052 scope.go:117] "RemoveContainer" containerID="5b99120371dd1ea96363322c8c525108a18d3c48f5844ec07a58fedeb98a76df" Dec 03 12:32:27 crc kubenswrapper[5052]: I1203 12:32:27.499582 5052 scope.go:117] "RemoveContainer" containerID="53093380d15a12407af002e8218c329f263122e535b3f7a98513361ca8b1c783" Dec 03 12:32:27 crc kubenswrapper[5052]: I1203 12:32:27.517555 5052 scope.go:117] "RemoveContainer" containerID="fefeb53cf7ddfecbec19b54d8ffa06426f1efa88fb8452a1c522b836c48d8085" Dec 03 12:32:27 crc kubenswrapper[5052]: I1203 12:32:27.538888 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage02-crc" (UniqueName: "kubernetes.io/local-volume/local-storage02-crc") on node "crc" Dec 03 12:32:27 crc kubenswrapper[5052]: I1203 12:32:27.540932 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage02-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage02-crc\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:27 crc kubenswrapper[5052]: E1203 12:32:27.541166 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/rabbitmq-cell1-config-data: configmap "rabbitmq-cell1-config-data" not found Dec 03 12:32:27 crc kubenswrapper[5052]: E1203 12:32:27.541292 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b8613a94-1e2e-477f-8d52-713c70f5282c-config-data podName:b8613a94-1e2e-477f-8d52-713c70f5282c nodeName:}" failed. No retries permitted until 2025-12-03 12:32:35.541274036 +0000 UTC m=+4346.129515810 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "config-data" (UniqueName: "kubernetes.io/configmap/b8613a94-1e2e-477f-8d52-713c70f5282c-config-data") pod "rabbitmq-cell1-server-0" (UID: "b8613a94-1e2e-477f-8d52-713c70f5282c") : configmap "rabbitmq-cell1-config-data" not found Dec 03 12:32:27 crc kubenswrapper[5052]: I1203 12:32:27.557422 5052 scope.go:117] "RemoveContainer" containerID="dc83501080d7c2386bc4543b9befadc35e0e843ede852c450da98e997424e43e" Dec 03 12:32:27 crc kubenswrapper[5052]: I1203 12:32:27.598621 5052 scope.go:117] "RemoveContainer" containerID="1840b34d3edddd62d0d38183e87fc8f3976fc72f58bdedd488ca0702b16fd6e7" Dec 03 12:32:27 crc kubenswrapper[5052]: I1203 12:32:27.623818 5052 scope.go:117] "RemoveContainer" containerID="a973285d5356bd9dc51bc03e5f6f166c4ac926ebd9edb7f87c4de4de53693880" Dec 03 12:32:27 crc kubenswrapper[5052]: E1203 12:32:27.643402 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:32:27 crc kubenswrapper[5052]: E1203 12:32:27.644152 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8a93d2af-9f8c-4282-8611-526869559ae2-operator-scripts podName:8a93d2af-9f8c-4282-8611-526869559ae2 nodeName:}" failed. No retries permitted until 2025-12-03 12:32:29.643472733 +0000 UTC m=+4340.231714497 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/8a93d2af-9f8c-4282-8611-526869559ae2-operator-scripts") pod "barbican3d67-account-delete-qrp98" (UID: "8a93d2af-9f8c-4282-8611-526869559ae2") : configmap "openstack-scripts" not found Dec 03 12:32:27 crc kubenswrapper[5052]: I1203 12:32:27.646492 5052 scope.go:117] "RemoveContainer" containerID="aec2d2ca3d9979ac47cbc62e58c9a1c7b06d1e13c5205fc585d92567ca0e4ec3" Dec 03 12:32:27 crc kubenswrapper[5052]: I1203 12:32:27.682645 5052 scope.go:117] "RemoveContainer" containerID="aec2d2ca3d9979ac47cbc62e58c9a1c7b06d1e13c5205fc585d92567ca0e4ec3" Dec 03 12:32:27 crc kubenswrapper[5052]: E1203 12:32:27.683213 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aec2d2ca3d9979ac47cbc62e58c9a1c7b06d1e13c5205fc585d92567ca0e4ec3\": container with ID starting with aec2d2ca3d9979ac47cbc62e58c9a1c7b06d1e13c5205fc585d92567ca0e4ec3 not found: ID does not exist" containerID="aec2d2ca3d9979ac47cbc62e58c9a1c7b06d1e13c5205fc585d92567ca0e4ec3" Dec 03 12:32:27 crc kubenswrapper[5052]: I1203 12:32:27.683268 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aec2d2ca3d9979ac47cbc62e58c9a1c7b06d1e13c5205fc585d92567ca0e4ec3"} err="failed to get container status \"aec2d2ca3d9979ac47cbc62e58c9a1c7b06d1e13c5205fc585d92567ca0e4ec3\": rpc error: code = NotFound desc = could not find container \"aec2d2ca3d9979ac47cbc62e58c9a1c7b06d1e13c5205fc585d92567ca0e4ec3\": container with ID starting with aec2d2ca3d9979ac47cbc62e58c9a1c7b06d1e13c5205fc585d92567ca0e4ec3 not found: ID does not exist" Dec 03 12:32:27 crc kubenswrapper[5052]: I1203 12:32:27.683294 5052 scope.go:117] "RemoveContainer" containerID="3c03aa60a5d69f404b768bb6cc6569fcdc7b56fcc98ed0721fb684712e773b3e" Dec 03 12:32:27 crc kubenswrapper[5052]: I1203 12:32:27.689659 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="045d578f-92c0-47c6-8356-59c489c57b9a" path="/var/lib/kubelet/pods/045d578f-92c0-47c6-8356-59c489c57b9a/volumes" Dec 03 12:32:27 crc kubenswrapper[5052]: I1203 12:32:27.690189 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="16cac95c-d1d2-4542-bb8e-4bdc1923c684" path="/var/lib/kubelet/pods/16cac95c-d1d2-4542-bb8e-4bdc1923c684/volumes" Dec 03 12:32:27 crc kubenswrapper[5052]: I1203 12:32:27.691450 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="39117149-c15d-4753-9324-8a42abe5b007" path="/var/lib/kubelet/pods/39117149-c15d-4753-9324-8a42abe5b007/volumes" Dec 03 12:32:27 crc kubenswrapper[5052]: I1203 12:32:27.693783 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="43933abc-db67-4ce5-9f30-6f8dbf3f59c8" path="/var/lib/kubelet/pods/43933abc-db67-4ce5-9f30-6f8dbf3f59c8/volumes" Dec 03 12:32:27 crc kubenswrapper[5052]: I1203 12:32:27.695253 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5737900d-97ae-4dcb-a65c-b40fe30cdc50" path="/var/lib/kubelet/pods/5737900d-97ae-4dcb-a65c-b40fe30cdc50/volumes" Dec 03 12:32:27 crc kubenswrapper[5052]: I1203 12:32:27.696253 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6e0f19c7-34ae-4d8e-9a19-755ae1c20c1c" path="/var/lib/kubelet/pods/6e0f19c7-34ae-4d8e-9a19-755ae1c20c1c/volumes" Dec 03 12:32:27 crc kubenswrapper[5052]: I1203 12:32:27.704270 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7cbe1c2c-bc04-4232-9901-bc9d134df3d6" path="/var/lib/kubelet/pods/7cbe1c2c-bc04-4232-9901-bc9d134df3d6/volumes" Dec 03 12:32:27 crc kubenswrapper[5052]: I1203 12:32:27.705105 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="82f142ab-5be9-4727-b3c5-102d6f2401fb" path="/var/lib/kubelet/pods/82f142ab-5be9-4727-b3c5-102d6f2401fb/volumes" Dec 03 12:32:27 crc kubenswrapper[5052]: I1203 12:32:27.705900 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a3b48edc-15d0-4c5e-8a33-96bee20dd43e" path="/var/lib/kubelet/pods/a3b48edc-15d0-4c5e-8a33-96bee20dd43e/volumes" Dec 03 12:32:27 crc kubenswrapper[5052]: I1203 12:32:27.707327 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a65428bc-9326-4ace-a985-9f796b951304" path="/var/lib/kubelet/pods/a65428bc-9326-4ace-a985-9f796b951304/volumes" Dec 03 12:32:27 crc kubenswrapper[5052]: I1203 12:32:27.708229 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bb1723c7-4e41-4b0e-b6b8-f8024dafdaee" path="/var/lib/kubelet/pods/bb1723c7-4e41-4b0e-b6b8-f8024dafdaee/volumes" Dec 03 12:32:27 crc kubenswrapper[5052]: I1203 12:32:27.709339 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cc8c914a-cb83-48ae-97a7-fca144e96f52" path="/var/lib/kubelet/pods/cc8c914a-cb83-48ae-97a7-fca144e96f52/volumes" Dec 03 12:32:27 crc kubenswrapper[5052]: I1203 12:32:27.710438 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cfd8af50-a644-4b24-8483-dd344f2218e4" path="/var/lib/kubelet/pods/cfd8af50-a644-4b24-8483-dd344f2218e4/volumes" Dec 03 12:32:27 crc kubenswrapper[5052]: I1203 12:32:27.711177 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d09b9817-6ea5-4328-9027-3311b7782157" path="/var/lib/kubelet/pods/d09b9817-6ea5-4328-9027-3311b7782157/volumes" Dec 03 12:32:27 crc kubenswrapper[5052]: I1203 12:32:27.711307 5052 scope.go:117] "RemoveContainer" containerID="1d2a9d60a97780af3d81ede77692ed338720a0c0a5b32ac9f95fa608d8ace7d1" Dec 03 12:32:27 crc kubenswrapper[5052]: I1203 12:32:27.715092 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f12527ba-678e-4ed9-82ff-cec336f607f3" path="/var/lib/kubelet/pods/f12527ba-678e-4ed9-82ff-cec336f607f3/volumes" Dec 03 12:32:28 crc kubenswrapper[5052]: E1203 12:32:28.052805 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:32:28 crc kubenswrapper[5052]: E1203 12:32:28.053191 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a2ef0804-75c7-43e3-95fa-ce901a561c9b-operator-scripts podName:a2ef0804-75c7-43e3-95fa-ce901a561c9b nodeName:}" failed. No retries permitted until 2025-12-03 12:32:32.053149216 +0000 UTC m=+4342.641390980 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/a2ef0804-75c7-43e3-95fa-ce901a561c9b-operator-scripts") pod "neutron2174-account-delete-9dnxs" (UID: "a2ef0804-75c7-43e3-95fa-ce901a561c9b") : configmap "openstack-scripts" not found Dec 03 12:32:28 crc kubenswrapper[5052]: I1203 12:32:28.085859 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:32:28 crc kubenswrapper[5052]: I1203 12:32:28.085852 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-galera-0" event={"ID":"6938204c-1fe8-4f03-ac16-ec9e19e090f8","Type":"ContainerDied","Data":"b0a0323137cad0ba98d2d8e9fb51b9c46221cbc417d036f304ad0e9d99d1a0ad"} Dec 03 12:32:28 crc kubenswrapper[5052]: I1203 12:32:28.086027 5052 scope.go:117] "RemoveContainer" containerID="d94e0139dcad346a24d75f7bd285773c0fe1edad0369015fe7b299454967acdd" Dec 03 12:32:28 crc kubenswrapper[5052]: I1203 12:32:28.088705 5052 generic.go:334] "Generic (PLEG): container finished" podID="2c03f99a-98fd-4e86-ad29-fb69e94bd436" containerID="33c00a517cac73b55f22b39dd9800951f3eddc4eb90411d1bc3e8a5477c02571" exitCode=0 Dec 03 12:32:28 crc kubenswrapper[5052]: I1203 12:32:28.088771 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-server-0" event={"ID":"2c03f99a-98fd-4e86-ad29-fb69e94bd436","Type":"ContainerDied","Data":"33c00a517cac73b55f22b39dd9800951f3eddc4eb90411d1bc3e8a5477c02571"} Dec 03 12:32:28 crc kubenswrapper[5052]: I1203 12:32:28.094723 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_ovn-northd-0_a9f65397-a08a-4798-8532-646209c5d090/ovn-northd/0.log" Dec 03 12:32:28 crc kubenswrapper[5052]: I1203 12:32:28.094773 5052 generic.go:334] "Generic (PLEG): container finished" podID="a9f65397-a08a-4798-8532-646209c5d090" containerID="217fb246797b6f91c28f31f22b498303e560c0706448ba85948bf67b547612e3" exitCode=139 Dec 03 12:32:28 crc kubenswrapper[5052]: I1203 12:32:28.094849 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovn-northd-0" event={"ID":"a9f65397-a08a-4798-8532-646209c5d090","Type":"ContainerDied","Data":"217fb246797b6f91c28f31f22b498303e560c0706448ba85948bf67b547612e3"} Dec 03 12:32:28 crc kubenswrapper[5052]: I1203 12:32:28.152941 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_ovn-northd-0_a9f65397-a08a-4798-8532-646209c5d090/ovn-northd/0.log" Dec 03 12:32:28 crc kubenswrapper[5052]: I1203 12:32:28.153033 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:32:28 crc kubenswrapper[5052]: E1203 12:32:28.154537 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:32:28 crc kubenswrapper[5052]: E1203 12:32:28.154612 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:32:28 crc kubenswrapper[5052]: E1203 12:32:28.154627 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/52bf0c8d-03b3-444b-80b6-13d41f5f9fb9-operator-scripts podName:52bf0c8d-03b3-444b-80b6-13d41f5f9fb9 nodeName:}" failed. No retries permitted until 2025-12-03 12:32:32.154607281 +0000 UTC m=+4342.742849045 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/52bf0c8d-03b3-444b-80b6-13d41f5f9fb9-operator-scripts") pod "glance5b9b-account-delete-ggk6q" (UID: "52bf0c8d-03b3-444b-80b6-13d41f5f9fb9") : configmap "openstack-scripts" not found Dec 03 12:32:28 crc kubenswrapper[5052]: E1203 12:32:28.154662 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:32:28 crc kubenswrapper[5052]: E1203 12:32:28.154705 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/fc428005-b653-4a6a-a50d-133c2b4fb8b8-operator-scripts podName:fc428005-b653-4a6a-a50d-133c2b4fb8b8 nodeName:}" failed. No retries permitted until 2025-12-03 12:32:32.154679753 +0000 UTC m=+4342.742921557 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/fc428005-b653-4a6a-a50d-133c2b4fb8b8-operator-scripts") pod "cinderc151-account-delete-h4k5b" (UID: "fc428005-b653-4a6a-a50d-133c2b4fb8b8") : configmap "openstack-scripts" not found Dec 03 12:32:28 crc kubenswrapper[5052]: E1203 12:32:28.154728 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e6d87a68-5121-4088-a5d8-3f205a787b29-operator-scripts podName:e6d87a68-5121-4088-a5d8-3f205a787b29 nodeName:}" failed. No retries permitted until 2025-12-03 12:32:32.154718715 +0000 UTC m=+4342.742960589 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/e6d87a68-5121-4088-a5d8-3f205a787b29-operator-scripts") pod "novaapia918-account-delete-mk25j" (UID: "e6d87a68-5121-4088-a5d8-3f205a787b29") : configmap "openstack-scripts" not found Dec 03 12:32:28 crc kubenswrapper[5052]: I1203 12:32:28.160648 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/openstack-galera-0"] Dec 03 12:32:28 crc kubenswrapper[5052]: I1203 12:32:28.171729 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/openstack-galera-0"] Dec 03 12:32:28 crc kubenswrapper[5052]: I1203 12:32:28.179350 5052 scope.go:117] "RemoveContainer" containerID="894ea95c17ab09554e637952aa3fd8f7d1432238350697dba009b9bb58995df9" Dec 03 12:32:28 crc kubenswrapper[5052]: I1203 12:32:28.255115 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x5d8s\" (UniqueName: \"kubernetes.io/projected/a9f65397-a08a-4798-8532-646209c5d090-kube-api-access-x5d8s\") pod \"a9f65397-a08a-4798-8532-646209c5d090\" (UID: \"a9f65397-a08a-4798-8532-646209c5d090\") " Dec 03 12:32:28 crc kubenswrapper[5052]: I1203 12:32:28.255248 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/a9f65397-a08a-4798-8532-646209c5d090-scripts\") pod \"a9f65397-a08a-4798-8532-646209c5d090\" (UID: \"a9f65397-a08a-4798-8532-646209c5d090\") " Dec 03 12:32:28 crc kubenswrapper[5052]: I1203 12:32:28.255297 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/a9f65397-a08a-4798-8532-646209c5d090-metrics-certs-tls-certs\") pod \"a9f65397-a08a-4798-8532-646209c5d090\" (UID: \"a9f65397-a08a-4798-8532-646209c5d090\") " Dec 03 12:32:28 crc kubenswrapper[5052]: I1203 12:32:28.255327 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovn-northd-tls-certs\" (UniqueName: \"kubernetes.io/secret/a9f65397-a08a-4798-8532-646209c5d090-ovn-northd-tls-certs\") pod \"a9f65397-a08a-4798-8532-646209c5d090\" (UID: \"a9f65397-a08a-4798-8532-646209c5d090\") " Dec 03 12:32:28 crc kubenswrapper[5052]: I1203 12:32:28.255359 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovn-rundir\" (UniqueName: \"kubernetes.io/empty-dir/a9f65397-a08a-4798-8532-646209c5d090-ovn-rundir\") pod \"a9f65397-a08a-4798-8532-646209c5d090\" (UID: \"a9f65397-a08a-4798-8532-646209c5d090\") " Dec 03 12:32:28 crc kubenswrapper[5052]: I1203 12:32:28.255441 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a9f65397-a08a-4798-8532-646209c5d090-config\") pod \"a9f65397-a08a-4798-8532-646209c5d090\" (UID: \"a9f65397-a08a-4798-8532-646209c5d090\") " Dec 03 12:32:28 crc kubenswrapper[5052]: I1203 12:32:28.255519 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a9f65397-a08a-4798-8532-646209c5d090-combined-ca-bundle\") pod \"a9f65397-a08a-4798-8532-646209c5d090\" (UID: \"a9f65397-a08a-4798-8532-646209c5d090\") " Dec 03 12:32:28 crc kubenswrapper[5052]: I1203 12:32:28.255795 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a9f65397-a08a-4798-8532-646209c5d090-ovn-rundir" (OuterVolumeSpecName: "ovn-rundir") pod "a9f65397-a08a-4798-8532-646209c5d090" (UID: "a9f65397-a08a-4798-8532-646209c5d090"). InnerVolumeSpecName "ovn-rundir". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:32:28 crc kubenswrapper[5052]: I1203 12:32:28.255864 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a9f65397-a08a-4798-8532-646209c5d090-scripts" (OuterVolumeSpecName: "scripts") pod "a9f65397-a08a-4798-8532-646209c5d090" (UID: "a9f65397-a08a-4798-8532-646209c5d090"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:32:28 crc kubenswrapper[5052]: I1203 12:32:28.256989 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a9f65397-a08a-4798-8532-646209c5d090-config" (OuterVolumeSpecName: "config") pod "a9f65397-a08a-4798-8532-646209c5d090" (UID: "a9f65397-a08a-4798-8532-646209c5d090"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:32:28 crc kubenswrapper[5052]: I1203 12:32:28.359564 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a9f65397-a08a-4798-8532-646209c5d090-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:28 crc kubenswrapper[5052]: I1203 12:32:28.359599 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/a9f65397-a08a-4798-8532-646209c5d090-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:28 crc kubenswrapper[5052]: I1203 12:32:28.359611 5052 reconciler_common.go:293] "Volume detached for volume \"ovn-rundir\" (UniqueName: \"kubernetes.io/empty-dir/a9f65397-a08a-4798-8532-646209c5d090-ovn-rundir\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:29 crc kubenswrapper[5052]: I1203 12:32:29.025064 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a9f65397-a08a-4798-8532-646209c5d090-kube-api-access-x5d8s" (OuterVolumeSpecName: "kube-api-access-x5d8s") pod "a9f65397-a08a-4798-8532-646209c5d090" (UID: "a9f65397-a08a-4798-8532-646209c5d090"). InnerVolumeSpecName "kube-api-access-x5d8s". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:32:29 crc kubenswrapper[5052]: I1203 12:32:29.078939 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-x5d8s\" (UniqueName: \"kubernetes.io/projected/a9f65397-a08a-4798-8532-646209c5d090-kube-api-access-x5d8s\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:29 crc kubenswrapper[5052]: I1203 12:32:29.144826 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_ovn-northd-0_a9f65397-a08a-4798-8532-646209c5d090/ovn-northd/0.log" Dec 03 12:32:29 crc kubenswrapper[5052]: I1203 12:32:29.145020 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:32:29 crc kubenswrapper[5052]: I1203 12:32:29.145017 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovn-northd-0" event={"ID":"a9f65397-a08a-4798-8532-646209c5d090","Type":"ContainerDied","Data":"9a2f2f114e462f7d8496b44d7ae330b86bd3edb78f74e61fc5c568bae55d88e3"} Dec 03 12:32:29 crc kubenswrapper[5052]: I1203 12:32:29.145105 5052 scope.go:117] "RemoveContainer" containerID="ea36b7c04ef3193b039e70bd5df384774204953e99d07610a4a191cab306432f" Dec 03 12:32:29 crc kubenswrapper[5052]: I1203 12:32:29.148943 5052 generic.go:334] "Generic (PLEG): container finished" podID="b8613a94-1e2e-477f-8d52-713c70f5282c" containerID="ecd92b1e5f2c4bb6955463da6c223b6fc821bf3dfbf16ddd65252cba7d35cea5" exitCode=0 Dec 03 12:32:29 crc kubenswrapper[5052]: I1203 12:32:29.149060 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" event={"ID":"b8613a94-1e2e-477f-8d52-713c70f5282c","Type":"ContainerDied","Data":"ecd92b1e5f2c4bb6955463da6c223b6fc821bf3dfbf16ddd65252cba7d35cea5"} Dec 03 12:32:29 crc kubenswrapper[5052]: I1203 12:32:29.155484 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-server-0" event={"ID":"2c03f99a-98fd-4e86-ad29-fb69e94bd436","Type":"ContainerDied","Data":"8d68154bede68b4efe4328c546c86de3b50b2f79ee843aa36a7c95ffbde2f980"} Dec 03 12:32:29 crc kubenswrapper[5052]: I1203 12:32:29.155605 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8d68154bede68b4efe4328c546c86de3b50b2f79ee843aa36a7c95ffbde2f980" Dec 03 12:32:29 crc kubenswrapper[5052]: I1203 12:32:29.159115 5052 generic.go:334] "Generic (PLEG): container finished" podID="019689d4-d40b-4e9a-a028-e427f1aa9d9b" containerID="a988e23bbce42676cbdcb4abb64e1c8911c6df2c4d18faf34f5c6450d0879857" exitCode=0 Dec 03 12:32:29 crc kubenswrapper[5052]: I1203 12:32:29.159153 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-keystone-listener-cd5b58cfd-jf8vt" event={"ID":"019689d4-d40b-4e9a-a028-e427f1aa9d9b","Type":"ContainerDied","Data":"a988e23bbce42676cbdcb4abb64e1c8911c6df2c4d18faf34f5c6450d0879857"} Dec 03 12:32:29 crc kubenswrapper[5052]: I1203 12:32:29.248864 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a9f65397-a08a-4798-8532-646209c5d090-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "a9f65397-a08a-4798-8532-646209c5d090" (UID: "a9f65397-a08a-4798-8532-646209c5d090"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:32:29 crc kubenswrapper[5052]: I1203 12:32:29.284089 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a9f65397-a08a-4798-8532-646209c5d090-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:29 crc kubenswrapper[5052]: I1203 12:32:29.330848 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a9f65397-a08a-4798-8532-646209c5d090-ovn-northd-tls-certs" (OuterVolumeSpecName: "ovn-northd-tls-certs") pod "a9f65397-a08a-4798-8532-646209c5d090" (UID: "a9f65397-a08a-4798-8532-646209c5d090"). InnerVolumeSpecName "ovn-northd-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:32:29 crc kubenswrapper[5052]: I1203 12:32:29.340986 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a9f65397-a08a-4798-8532-646209c5d090-metrics-certs-tls-certs" (OuterVolumeSpecName: "metrics-certs-tls-certs") pod "a9f65397-a08a-4798-8532-646209c5d090" (UID: "a9f65397-a08a-4798-8532-646209c5d090"). InnerVolumeSpecName "metrics-certs-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:32:29 crc kubenswrapper[5052]: I1203 12:32:29.386113 5052 reconciler_common.go:293] "Volume detached for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/a9f65397-a08a-4798-8532-646209c5d090-metrics-certs-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:29 crc kubenswrapper[5052]: I1203 12:32:29.386916 5052 reconciler_common.go:293] "Volume detached for volume \"ovn-northd-tls-certs\" (UniqueName: \"kubernetes.io/secret/a9f65397-a08a-4798-8532-646209c5d090-ovn-northd-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:29 crc kubenswrapper[5052]: I1203 12:32:29.606768 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:32:29 crc kubenswrapper[5052]: I1203 12:32:29.613434 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-keystone-listener-cd5b58cfd-jf8vt" Dec 03 12:32:29 crc kubenswrapper[5052]: I1203 12:32:29.625307 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ovn-northd-0"] Dec 03 12:32:29 crc kubenswrapper[5052]: I1203 12:32:29.631926 5052 scope.go:117] "RemoveContainer" containerID="217fb246797b6f91c28f31f22b498303e560c0706448ba85948bf67b547612e3" Dec 03 12:32:29 crc kubenswrapper[5052]: I1203 12:32:29.640256 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ovn-northd-0"] Dec 03 12:32:29 crc kubenswrapper[5052]: I1203 12:32:29.651792 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:32:29 crc kubenswrapper[5052]: I1203 12:32:29.658497 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:32:29 crc kubenswrapper[5052]: I1203 12:32:29.684354 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6938204c-1fe8-4f03-ac16-ec9e19e090f8" path="/var/lib/kubelet/pods/6938204c-1fe8-4f03-ac16-ec9e19e090f8/volumes" Dec 03 12:32:29 crc kubenswrapper[5052]: I1203 12:32:29.685063 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a9f65397-a08a-4798-8532-646209c5d090" path="/var/lib/kubelet/pods/a9f65397-a08a-4798-8532-646209c5d090/volumes" Dec 03 12:32:29 crc kubenswrapper[5052]: I1203 12:32:29.690587 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"rabbitmq-plugins\" (UniqueName: \"kubernetes.io/empty-dir/2c03f99a-98fd-4e86-ad29-fb69e94bd436-rabbitmq-plugins\") pod \"2c03f99a-98fd-4e86-ad29-fb69e94bd436\" (UID: \"2c03f99a-98fd-4e86-ad29-fb69e94bd436\") " Dec 03 12:32:29 crc kubenswrapper[5052]: I1203 12:32:29.690633 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/21690a66-4bcf-4865-91c7-cfe1cdf32a61-log-httpd\") pod \"21690a66-4bcf-4865-91c7-cfe1cdf32a61\" (UID: \"21690a66-4bcf-4865-91c7-cfe1cdf32a61\") " Dec 03 12:32:29 crc kubenswrapper[5052]: I1203 12:32:29.690693 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/019689d4-d40b-4e9a-a028-e427f1aa9d9b-config-data-custom\") pod \"019689d4-d40b-4e9a-a028-e427f1aa9d9b\" (UID: \"019689d4-d40b-4e9a-a028-e427f1aa9d9b\") " Dec 03 12:32:29 crc kubenswrapper[5052]: I1203 12:32:29.690715 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"plugins-conf\" (UniqueName: \"kubernetes.io/configmap/2c03f99a-98fd-4e86-ad29-fb69e94bd436-plugins-conf\") pod \"2c03f99a-98fd-4e86-ad29-fb69e94bd436\" (UID: \"2c03f99a-98fd-4e86-ad29-fb69e94bd436\") " Dec 03 12:32:29 crc kubenswrapper[5052]: I1203 12:32:29.690740 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/019689d4-d40b-4e9a-a028-e427f1aa9d9b-logs\") pod \"019689d4-d40b-4e9a-a028-e427f1aa9d9b\" (UID: \"019689d4-d40b-4e9a-a028-e427f1aa9d9b\") " Dec 03 12:32:29 crc kubenswrapper[5052]: I1203 12:32:29.690773 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"pod-info\" (UniqueName: \"kubernetes.io/downward-api/2c03f99a-98fd-4e86-ad29-fb69e94bd436-pod-info\") pod \"2c03f99a-98fd-4e86-ad29-fb69e94bd436\" (UID: \"2c03f99a-98fd-4e86-ad29-fb69e94bd436\") " Dec 03 12:32:29 crc kubenswrapper[5052]: I1203 12:32:29.690792 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/21690a66-4bcf-4865-91c7-cfe1cdf32a61-combined-ca-bundle\") pod \"21690a66-4bcf-4865-91c7-cfe1cdf32a61\" (UID: \"21690a66-4bcf-4865-91c7-cfe1cdf32a61\") " Dec 03 12:32:29 crc kubenswrapper[5052]: I1203 12:32:29.690831 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"server-conf\" (UniqueName: \"kubernetes.io/configmap/2c03f99a-98fd-4e86-ad29-fb69e94bd436-server-conf\") pod \"2c03f99a-98fd-4e86-ad29-fb69e94bd436\" (UID: \"2c03f99a-98fd-4e86-ad29-fb69e94bd436\") " Dec 03 12:32:29 crc kubenswrapper[5052]: I1203 12:32:29.690860 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/21690a66-4bcf-4865-91c7-cfe1cdf32a61-config-data\") pod \"21690a66-4bcf-4865-91c7-cfe1cdf32a61\" (UID: \"21690a66-4bcf-4865-91c7-cfe1cdf32a61\") " Dec 03 12:32:29 crc kubenswrapper[5052]: I1203 12:32:29.690880 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"rabbitmq-erlang-cookie\" (UniqueName: \"kubernetes.io/empty-dir/2c03f99a-98fd-4e86-ad29-fb69e94bd436-rabbitmq-erlang-cookie\") pod \"2c03f99a-98fd-4e86-ad29-fb69e94bd436\" (UID: \"2c03f99a-98fd-4e86-ad29-fb69e94bd436\") " Dec 03 12:32:29 crc kubenswrapper[5052]: I1203 12:32:29.690914 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/2c03f99a-98fd-4e86-ad29-fb69e94bd436-config-data\") pod \"2c03f99a-98fd-4e86-ad29-fb69e94bd436\" (UID: \"2c03f99a-98fd-4e86-ad29-fb69e94bd436\") " Dec 03 12:32:29 crc kubenswrapper[5052]: I1203 12:32:29.690935 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6rggl\" (UniqueName: \"kubernetes.io/projected/019689d4-d40b-4e9a-a028-e427f1aa9d9b-kube-api-access-6rggl\") pod \"019689d4-d40b-4e9a-a028-e427f1aa9d9b\" (UID: \"019689d4-d40b-4e9a-a028-e427f1aa9d9b\") " Dec 03 12:32:29 crc kubenswrapper[5052]: I1203 12:32:29.690958 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/019689d4-d40b-4e9a-a028-e427f1aa9d9b-combined-ca-bundle\") pod \"019689d4-d40b-4e9a-a028-e427f1aa9d9b\" (UID: \"019689d4-d40b-4e9a-a028-e427f1aa9d9b\") " Dec 03 12:32:29 crc kubenswrapper[5052]: I1203 12:32:29.690981 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/21690a66-4bcf-4865-91c7-cfe1cdf32a61-sg-core-conf-yaml\") pod \"21690a66-4bcf-4865-91c7-cfe1cdf32a61\" (UID: \"21690a66-4bcf-4865-91c7-cfe1cdf32a61\") " Dec 03 12:32:29 crc kubenswrapper[5052]: I1203 12:32:29.691000 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/019689d4-d40b-4e9a-a028-e427f1aa9d9b-config-data\") pod \"019689d4-d40b-4e9a-a028-e427f1aa9d9b\" (UID: \"019689d4-d40b-4e9a-a028-e427f1aa9d9b\") " Dec 03 12:32:29 crc kubenswrapper[5052]: I1203 12:32:29.691015 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6bptq\" (UniqueName: \"kubernetes.io/projected/21690a66-4bcf-4865-91c7-cfe1cdf32a61-kube-api-access-6bptq\") pod \"21690a66-4bcf-4865-91c7-cfe1cdf32a61\" (UID: \"21690a66-4bcf-4865-91c7-cfe1cdf32a61\") " Dec 03 12:32:29 crc kubenswrapper[5052]: I1203 12:32:29.691031 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"rabbitmq-tls\" (UniqueName: \"kubernetes.io/projected/2c03f99a-98fd-4e86-ad29-fb69e94bd436-rabbitmq-tls\") pod \"2c03f99a-98fd-4e86-ad29-fb69e94bd436\" (UID: \"2c03f99a-98fd-4e86-ad29-fb69e94bd436\") " Dec 03 12:32:29 crc kubenswrapper[5052]: I1203 12:32:29.691050 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"persistence\" (UniqueName: \"kubernetes.io/local-volume/local-storage19-crc\") pod \"2c03f99a-98fd-4e86-ad29-fb69e94bd436\" (UID: \"2c03f99a-98fd-4e86-ad29-fb69e94bd436\") " Dec 03 12:32:29 crc kubenswrapper[5052]: I1203 12:32:29.691073 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/21690a66-4bcf-4865-91c7-cfe1cdf32a61-scripts\") pod \"21690a66-4bcf-4865-91c7-cfe1cdf32a61\" (UID: \"21690a66-4bcf-4865-91c7-cfe1cdf32a61\") " Dec 03 12:32:29 crc kubenswrapper[5052]: I1203 12:32:29.691113 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cgp57\" (UniqueName: \"kubernetes.io/projected/2c03f99a-98fd-4e86-ad29-fb69e94bd436-kube-api-access-cgp57\") pod \"2c03f99a-98fd-4e86-ad29-fb69e94bd436\" (UID: \"2c03f99a-98fd-4e86-ad29-fb69e94bd436\") " Dec 03 12:32:29 crc kubenswrapper[5052]: I1203 12:32:29.691129 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/21690a66-4bcf-4865-91c7-cfe1cdf32a61-ceilometer-tls-certs\") pod \"21690a66-4bcf-4865-91c7-cfe1cdf32a61\" (UID: \"21690a66-4bcf-4865-91c7-cfe1cdf32a61\") " Dec 03 12:32:29 crc kubenswrapper[5052]: I1203 12:32:29.691149 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"erlang-cookie-secret\" (UniqueName: \"kubernetes.io/secret/2c03f99a-98fd-4e86-ad29-fb69e94bd436-erlang-cookie-secret\") pod \"2c03f99a-98fd-4e86-ad29-fb69e94bd436\" (UID: \"2c03f99a-98fd-4e86-ad29-fb69e94bd436\") " Dec 03 12:32:29 crc kubenswrapper[5052]: I1203 12:32:29.691165 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/21690a66-4bcf-4865-91c7-cfe1cdf32a61-run-httpd\") pod \"21690a66-4bcf-4865-91c7-cfe1cdf32a61\" (UID: \"21690a66-4bcf-4865-91c7-cfe1cdf32a61\") " Dec 03 12:32:29 crc kubenswrapper[5052]: I1203 12:32:29.691188 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"rabbitmq-confd\" (UniqueName: \"kubernetes.io/projected/2c03f99a-98fd-4e86-ad29-fb69e94bd436-rabbitmq-confd\") pod \"2c03f99a-98fd-4e86-ad29-fb69e94bd436\" (UID: \"2c03f99a-98fd-4e86-ad29-fb69e94bd436\") " Dec 03 12:32:29 crc kubenswrapper[5052]: I1203 12:32:29.691706 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c03f99a-98fd-4e86-ad29-fb69e94bd436-rabbitmq-plugins" (OuterVolumeSpecName: "rabbitmq-plugins") pod "2c03f99a-98fd-4e86-ad29-fb69e94bd436" (UID: "2c03f99a-98fd-4e86-ad29-fb69e94bd436"). InnerVolumeSpecName "rabbitmq-plugins". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:32:29 crc kubenswrapper[5052]: I1203 12:32:29.691744 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/019689d4-d40b-4e9a-a028-e427f1aa9d9b-logs" (OuterVolumeSpecName: "logs") pod "019689d4-d40b-4e9a-a028-e427f1aa9d9b" (UID: "019689d4-d40b-4e9a-a028-e427f1aa9d9b"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:32:29 crc kubenswrapper[5052]: I1203 12:32:29.691823 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c03f99a-98fd-4e86-ad29-fb69e94bd436-rabbitmq-erlang-cookie" (OuterVolumeSpecName: "rabbitmq-erlang-cookie") pod "2c03f99a-98fd-4e86-ad29-fb69e94bd436" (UID: "2c03f99a-98fd-4e86-ad29-fb69e94bd436"). InnerVolumeSpecName "rabbitmq-erlang-cookie". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:32:29 crc kubenswrapper[5052]: I1203 12:32:29.691913 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2c03f99a-98fd-4e86-ad29-fb69e94bd436-plugins-conf" (OuterVolumeSpecName: "plugins-conf") pod "2c03f99a-98fd-4e86-ad29-fb69e94bd436" (UID: "2c03f99a-98fd-4e86-ad29-fb69e94bd436"). InnerVolumeSpecName "plugins-conf". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:32:29 crc kubenswrapper[5052]: I1203 12:32:29.692369 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/21690a66-4bcf-4865-91c7-cfe1cdf32a61-log-httpd" (OuterVolumeSpecName: "log-httpd") pod "21690a66-4bcf-4865-91c7-cfe1cdf32a61" (UID: "21690a66-4bcf-4865-91c7-cfe1cdf32a61"). InnerVolumeSpecName "log-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:32:29 crc kubenswrapper[5052]: I1203 12:32:29.695433 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2c03f99a-98fd-4e86-ad29-fb69e94bd436-pod-info" (OuterVolumeSpecName: "pod-info") pod "2c03f99a-98fd-4e86-ad29-fb69e94bd436" (UID: "2c03f99a-98fd-4e86-ad29-fb69e94bd436"). InnerVolumeSpecName "pod-info". PluginName "kubernetes.io/downward-api", VolumeGidValue "" Dec 03 12:32:29 crc kubenswrapper[5052]: I1203 12:32:29.695699 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage19-crc" (OuterVolumeSpecName: "persistence") pod "2c03f99a-98fd-4e86-ad29-fb69e94bd436" (UID: "2c03f99a-98fd-4e86-ad29-fb69e94bd436"). InnerVolumeSpecName "local-storage19-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 12:32:29 crc kubenswrapper[5052]: I1203 12:32:29.696699 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/019689d4-d40b-4e9a-a028-e427f1aa9d9b-config-data-custom" (OuterVolumeSpecName: "config-data-custom") pod "019689d4-d40b-4e9a-a028-e427f1aa9d9b" (UID: "019689d4-d40b-4e9a-a028-e427f1aa9d9b"). InnerVolumeSpecName "config-data-custom". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:32:29 crc kubenswrapper[5052]: I1203 12:32:29.698063 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/21690a66-4bcf-4865-91c7-cfe1cdf32a61-scripts" (OuterVolumeSpecName: "scripts") pod "21690a66-4bcf-4865-91c7-cfe1cdf32a61" (UID: "21690a66-4bcf-4865-91c7-cfe1cdf32a61"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:32:29 crc kubenswrapper[5052]: I1203 12:32:29.698425 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/21690a66-4bcf-4865-91c7-cfe1cdf32a61-run-httpd" (OuterVolumeSpecName: "run-httpd") pod "21690a66-4bcf-4865-91c7-cfe1cdf32a61" (UID: "21690a66-4bcf-4865-91c7-cfe1cdf32a61"). InnerVolumeSpecName "run-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:32:29 crc kubenswrapper[5052]: I1203 12:32:29.698883 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/21690a66-4bcf-4865-91c7-cfe1cdf32a61-kube-api-access-6bptq" (OuterVolumeSpecName: "kube-api-access-6bptq") pod "21690a66-4bcf-4865-91c7-cfe1cdf32a61" (UID: "21690a66-4bcf-4865-91c7-cfe1cdf32a61"). InnerVolumeSpecName "kube-api-access-6bptq". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:32:29 crc kubenswrapper[5052]: I1203 12:32:29.699454 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2c03f99a-98fd-4e86-ad29-fb69e94bd436-rabbitmq-tls" (OuterVolumeSpecName: "rabbitmq-tls") pod "2c03f99a-98fd-4e86-ad29-fb69e94bd436" (UID: "2c03f99a-98fd-4e86-ad29-fb69e94bd436"). InnerVolumeSpecName "rabbitmq-tls". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:32:29 crc kubenswrapper[5052]: I1203 12:32:29.700093 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-6bptq\" (UniqueName: \"kubernetes.io/projected/21690a66-4bcf-4865-91c7-cfe1cdf32a61-kube-api-access-6bptq\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:29 crc kubenswrapper[5052]: I1203 12:32:29.700120 5052 reconciler_common.go:293] "Volume detached for volume \"rabbitmq-tls\" (UniqueName: \"kubernetes.io/projected/2c03f99a-98fd-4e86-ad29-fb69e94bd436-rabbitmq-tls\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:29 crc kubenswrapper[5052]: I1203 12:32:29.700146 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage19-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage19-crc\") on node \"crc\" " Dec 03 12:32:29 crc kubenswrapper[5052]: I1203 12:32:29.700158 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/21690a66-4bcf-4865-91c7-cfe1cdf32a61-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:29 crc kubenswrapper[5052]: I1203 12:32:29.700171 5052 reconciler_common.go:293] "Volume detached for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/21690a66-4bcf-4865-91c7-cfe1cdf32a61-run-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:29 crc kubenswrapper[5052]: I1203 12:32:29.700183 5052 reconciler_common.go:293] "Volume detached for volume \"rabbitmq-plugins\" (UniqueName: \"kubernetes.io/empty-dir/2c03f99a-98fd-4e86-ad29-fb69e94bd436-rabbitmq-plugins\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:29 crc kubenswrapper[5052]: I1203 12:32:29.700194 5052 reconciler_common.go:293] "Volume detached for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/21690a66-4bcf-4865-91c7-cfe1cdf32a61-log-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:29 crc kubenswrapper[5052]: I1203 12:32:29.700207 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/019689d4-d40b-4e9a-a028-e427f1aa9d9b-config-data-custom\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:29 crc kubenswrapper[5052]: I1203 12:32:29.700217 5052 reconciler_common.go:293] "Volume detached for volume \"plugins-conf\" (UniqueName: \"kubernetes.io/configmap/2c03f99a-98fd-4e86-ad29-fb69e94bd436-plugins-conf\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:29 crc kubenswrapper[5052]: I1203 12:32:29.700229 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/019689d4-d40b-4e9a-a028-e427f1aa9d9b-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:29 crc kubenswrapper[5052]: I1203 12:32:29.700240 5052 reconciler_common.go:293] "Volume detached for volume \"pod-info\" (UniqueName: \"kubernetes.io/downward-api/2c03f99a-98fd-4e86-ad29-fb69e94bd436-pod-info\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:29 crc kubenswrapper[5052]: I1203 12:32:29.700252 5052 reconciler_common.go:293] "Volume detached for volume \"rabbitmq-erlang-cookie\" (UniqueName: \"kubernetes.io/empty-dir/2c03f99a-98fd-4e86-ad29-fb69e94bd436-rabbitmq-erlang-cookie\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:29 crc kubenswrapper[5052]: I1203 12:32:29.700248 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/019689d4-d40b-4e9a-a028-e427f1aa9d9b-kube-api-access-6rggl" (OuterVolumeSpecName: "kube-api-access-6rggl") pod "019689d4-d40b-4e9a-a028-e427f1aa9d9b" (UID: "019689d4-d40b-4e9a-a028-e427f1aa9d9b"). InnerVolumeSpecName "kube-api-access-6rggl". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:32:29 crc kubenswrapper[5052]: E1203 12:32:29.700379 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:32:29 crc kubenswrapper[5052]: E1203 12:32:29.700439 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8a93d2af-9f8c-4282-8611-526869559ae2-operator-scripts podName:8a93d2af-9f8c-4282-8611-526869559ae2 nodeName:}" failed. No retries permitted until 2025-12-03 12:32:33.700416912 +0000 UTC m=+4344.288658756 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/8a93d2af-9f8c-4282-8611-526869559ae2-operator-scripts") pod "barbican3d67-account-delete-qrp98" (UID: "8a93d2af-9f8c-4282-8611-526869559ae2") : configmap "openstack-scripts" not found Dec 03 12:32:29 crc kubenswrapper[5052]: I1203 12:32:29.701637 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2c03f99a-98fd-4e86-ad29-fb69e94bd436-erlang-cookie-secret" (OuterVolumeSpecName: "erlang-cookie-secret") pod "2c03f99a-98fd-4e86-ad29-fb69e94bd436" (UID: "2c03f99a-98fd-4e86-ad29-fb69e94bd436"). InnerVolumeSpecName "erlang-cookie-secret". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:32:29 crc kubenswrapper[5052]: I1203 12:32:29.737283 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage19-crc" (UniqueName: "kubernetes.io/local-volume/local-storage19-crc") on node "crc" Dec 03 12:32:29 crc kubenswrapper[5052]: I1203 12:32:29.801613 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"pod-info\" (UniqueName: \"kubernetes.io/downward-api/b8613a94-1e2e-477f-8d52-713c70f5282c-pod-info\") pod \"b8613a94-1e2e-477f-8d52-713c70f5282c\" (UID: \"b8613a94-1e2e-477f-8d52-713c70f5282c\") " Dec 03 12:32:29 crc kubenswrapper[5052]: I1203 12:32:29.801672 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"server-conf\" (UniqueName: \"kubernetes.io/configmap/b8613a94-1e2e-477f-8d52-713c70f5282c-server-conf\") pod \"b8613a94-1e2e-477f-8d52-713c70f5282c\" (UID: \"b8613a94-1e2e-477f-8d52-713c70f5282c\") " Dec 03 12:32:29 crc kubenswrapper[5052]: I1203 12:32:29.801781 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"rabbitmq-erlang-cookie\" (UniqueName: \"kubernetes.io/empty-dir/b8613a94-1e2e-477f-8d52-713c70f5282c-rabbitmq-erlang-cookie\") pod \"b8613a94-1e2e-477f-8d52-713c70f5282c\" (UID: \"b8613a94-1e2e-477f-8d52-713c70f5282c\") " Dec 03 12:32:29 crc kubenswrapper[5052]: I1203 12:32:29.801863 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/b8613a94-1e2e-477f-8d52-713c70f5282c-config-data\") pod \"b8613a94-1e2e-477f-8d52-713c70f5282c\" (UID: \"b8613a94-1e2e-477f-8d52-713c70f5282c\") " Dec 03 12:32:29 crc kubenswrapper[5052]: I1203 12:32:29.801970 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6sgkj\" (UniqueName: \"kubernetes.io/projected/b8613a94-1e2e-477f-8d52-713c70f5282c-kube-api-access-6sgkj\") pod \"b8613a94-1e2e-477f-8d52-713c70f5282c\" (UID: \"b8613a94-1e2e-477f-8d52-713c70f5282c\") " Dec 03 12:32:29 crc kubenswrapper[5052]: I1203 12:32:29.802017 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"persistence\" (UniqueName: \"kubernetes.io/local-volume/local-storage01-crc\") pod \"b8613a94-1e2e-477f-8d52-713c70f5282c\" (UID: \"b8613a94-1e2e-477f-8d52-713c70f5282c\") " Dec 03 12:32:29 crc kubenswrapper[5052]: I1203 12:32:29.802071 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"rabbitmq-tls\" (UniqueName: \"kubernetes.io/projected/b8613a94-1e2e-477f-8d52-713c70f5282c-rabbitmq-tls\") pod \"b8613a94-1e2e-477f-8d52-713c70f5282c\" (UID: \"b8613a94-1e2e-477f-8d52-713c70f5282c\") " Dec 03 12:32:29 crc kubenswrapper[5052]: I1203 12:32:29.802104 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"rabbitmq-plugins\" (UniqueName: \"kubernetes.io/empty-dir/b8613a94-1e2e-477f-8d52-713c70f5282c-rabbitmq-plugins\") pod \"b8613a94-1e2e-477f-8d52-713c70f5282c\" (UID: \"b8613a94-1e2e-477f-8d52-713c70f5282c\") " Dec 03 12:32:29 crc kubenswrapper[5052]: I1203 12:32:29.802126 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"plugins-conf\" (UniqueName: \"kubernetes.io/configmap/b8613a94-1e2e-477f-8d52-713c70f5282c-plugins-conf\") pod \"b8613a94-1e2e-477f-8d52-713c70f5282c\" (UID: \"b8613a94-1e2e-477f-8d52-713c70f5282c\") " Dec 03 12:32:29 crc kubenswrapper[5052]: I1203 12:32:29.802146 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"erlang-cookie-secret\" (UniqueName: \"kubernetes.io/secret/b8613a94-1e2e-477f-8d52-713c70f5282c-erlang-cookie-secret\") pod \"b8613a94-1e2e-477f-8d52-713c70f5282c\" (UID: \"b8613a94-1e2e-477f-8d52-713c70f5282c\") " Dec 03 12:32:29 crc kubenswrapper[5052]: I1203 12:32:29.802200 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"rabbitmq-confd\" (UniqueName: \"kubernetes.io/projected/b8613a94-1e2e-477f-8d52-713c70f5282c-rabbitmq-confd\") pod \"b8613a94-1e2e-477f-8d52-713c70f5282c\" (UID: \"b8613a94-1e2e-477f-8d52-713c70f5282c\") " Dec 03 12:32:29 crc kubenswrapper[5052]: I1203 12:32:29.802663 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b8613a94-1e2e-477f-8d52-713c70f5282c-rabbitmq-plugins" (OuterVolumeSpecName: "rabbitmq-plugins") pod "b8613a94-1e2e-477f-8d52-713c70f5282c" (UID: "b8613a94-1e2e-477f-8d52-713c70f5282c"). InnerVolumeSpecName "rabbitmq-plugins". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:32:29 crc kubenswrapper[5052]: I1203 12:32:29.803348 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-6rggl\" (UniqueName: \"kubernetes.io/projected/019689d4-d40b-4e9a-a028-e427f1aa9d9b-kube-api-access-6rggl\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:29 crc kubenswrapper[5052]: I1203 12:32:29.803370 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage19-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage19-crc\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:29 crc kubenswrapper[5052]: I1203 12:32:29.803381 5052 reconciler_common.go:293] "Volume detached for volume \"erlang-cookie-secret\" (UniqueName: \"kubernetes.io/secret/2c03f99a-98fd-4e86-ad29-fb69e94bd436-erlang-cookie-secret\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:29 crc kubenswrapper[5052]: I1203 12:32:29.803391 5052 reconciler_common.go:293] "Volume detached for volume \"rabbitmq-plugins\" (UniqueName: \"kubernetes.io/empty-dir/b8613a94-1e2e-477f-8d52-713c70f5282c-rabbitmq-plugins\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:29 crc kubenswrapper[5052]: I1203 12:32:29.803445 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b8613a94-1e2e-477f-8d52-713c70f5282c-plugins-conf" (OuterVolumeSpecName: "plugins-conf") pod "b8613a94-1e2e-477f-8d52-713c70f5282c" (UID: "b8613a94-1e2e-477f-8d52-713c70f5282c"). InnerVolumeSpecName "plugins-conf". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:32:29 crc kubenswrapper[5052]: I1203 12:32:29.803906 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b8613a94-1e2e-477f-8d52-713c70f5282c-rabbitmq-erlang-cookie" (OuterVolumeSpecName: "rabbitmq-erlang-cookie") pod "b8613a94-1e2e-477f-8d52-713c70f5282c" (UID: "b8613a94-1e2e-477f-8d52-713c70f5282c"). InnerVolumeSpecName "rabbitmq-erlang-cookie". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:32:29 crc kubenswrapper[5052]: I1203 12:32:29.822701 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b8613a94-1e2e-477f-8d52-713c70f5282c-erlang-cookie-secret" (OuterVolumeSpecName: "erlang-cookie-secret") pod "b8613a94-1e2e-477f-8d52-713c70f5282c" (UID: "b8613a94-1e2e-477f-8d52-713c70f5282c"). InnerVolumeSpecName "erlang-cookie-secret". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:32:29 crc kubenswrapper[5052]: I1203 12:32:29.823291 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2c03f99a-98fd-4e86-ad29-fb69e94bd436-kube-api-access-cgp57" (OuterVolumeSpecName: "kube-api-access-cgp57") pod "2c03f99a-98fd-4e86-ad29-fb69e94bd436" (UID: "2c03f99a-98fd-4e86-ad29-fb69e94bd436"). InnerVolumeSpecName "kube-api-access-cgp57". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:32:29 crc kubenswrapper[5052]: I1203 12:32:29.824115 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2c03f99a-98fd-4e86-ad29-fb69e94bd436-config-data" (OuterVolumeSpecName: "config-data") pod "2c03f99a-98fd-4e86-ad29-fb69e94bd436" (UID: "2c03f99a-98fd-4e86-ad29-fb69e94bd436"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:32:29 crc kubenswrapper[5052]: I1203 12:32:29.824227 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b8613a94-1e2e-477f-8d52-713c70f5282c-kube-api-access-6sgkj" (OuterVolumeSpecName: "kube-api-access-6sgkj") pod "b8613a94-1e2e-477f-8d52-713c70f5282c" (UID: "b8613a94-1e2e-477f-8d52-713c70f5282c"). InnerVolumeSpecName "kube-api-access-6sgkj". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:32:29 crc kubenswrapper[5052]: I1203 12:32:29.824288 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage01-crc" (OuterVolumeSpecName: "persistence") pod "b8613a94-1e2e-477f-8d52-713c70f5282c" (UID: "b8613a94-1e2e-477f-8d52-713c70f5282c"). InnerVolumeSpecName "local-storage01-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 12:32:29 crc kubenswrapper[5052]: I1203 12:32:29.826957 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/019689d4-d40b-4e9a-a028-e427f1aa9d9b-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "019689d4-d40b-4e9a-a028-e427f1aa9d9b" (UID: "019689d4-d40b-4e9a-a028-e427f1aa9d9b"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:32:29 crc kubenswrapper[5052]: I1203 12:32:29.827683 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b8613a94-1e2e-477f-8d52-713c70f5282c-rabbitmq-tls" (OuterVolumeSpecName: "rabbitmq-tls") pod "b8613a94-1e2e-477f-8d52-713c70f5282c" (UID: "b8613a94-1e2e-477f-8d52-713c70f5282c"). InnerVolumeSpecName "rabbitmq-tls". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:32:29 crc kubenswrapper[5052]: I1203 12:32:29.827924 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/21690a66-4bcf-4865-91c7-cfe1cdf32a61-sg-core-conf-yaml" (OuterVolumeSpecName: "sg-core-conf-yaml") pod "21690a66-4bcf-4865-91c7-cfe1cdf32a61" (UID: "21690a66-4bcf-4865-91c7-cfe1cdf32a61"). InnerVolumeSpecName "sg-core-conf-yaml". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:32:29 crc kubenswrapper[5052]: I1203 12:32:29.836156 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b8613a94-1e2e-477f-8d52-713c70f5282c-config-data" (OuterVolumeSpecName: "config-data") pod "b8613a94-1e2e-477f-8d52-713c70f5282c" (UID: "b8613a94-1e2e-477f-8d52-713c70f5282c"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:32:29 crc kubenswrapper[5052]: I1203 12:32:29.841945 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b8613a94-1e2e-477f-8d52-713c70f5282c-pod-info" (OuterVolumeSpecName: "pod-info") pod "b8613a94-1e2e-477f-8d52-713c70f5282c" (UID: "b8613a94-1e2e-477f-8d52-713c70f5282c"). InnerVolumeSpecName "pod-info". PluginName "kubernetes.io/downward-api", VolumeGidValue "" Dec 03 12:32:29 crc kubenswrapper[5052]: I1203 12:32:29.849724 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/21690a66-4bcf-4865-91c7-cfe1cdf32a61-ceilometer-tls-certs" (OuterVolumeSpecName: "ceilometer-tls-certs") pod "21690a66-4bcf-4865-91c7-cfe1cdf32a61" (UID: "21690a66-4bcf-4865-91c7-cfe1cdf32a61"). InnerVolumeSpecName "ceilometer-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:32:29 crc kubenswrapper[5052]: I1203 12:32:29.853872 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/019689d4-d40b-4e9a-a028-e427f1aa9d9b-config-data" (OuterVolumeSpecName: "config-data") pod "019689d4-d40b-4e9a-a028-e427f1aa9d9b" (UID: "019689d4-d40b-4e9a-a028-e427f1aa9d9b"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:32:29 crc kubenswrapper[5052]: I1203 12:32:29.854096 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2c03f99a-98fd-4e86-ad29-fb69e94bd436-server-conf" (OuterVolumeSpecName: "server-conf") pod "2c03f99a-98fd-4e86-ad29-fb69e94bd436" (UID: "2c03f99a-98fd-4e86-ad29-fb69e94bd436"). InnerVolumeSpecName "server-conf". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:32:29 crc kubenswrapper[5052]: I1203 12:32:29.872325 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b8613a94-1e2e-477f-8d52-713c70f5282c-server-conf" (OuterVolumeSpecName: "server-conf") pod "b8613a94-1e2e-477f-8d52-713c70f5282c" (UID: "b8613a94-1e2e-477f-8d52-713c70f5282c"). InnerVolumeSpecName "server-conf". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:32:29 crc kubenswrapper[5052]: I1203 12:32:29.891794 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/21690a66-4bcf-4865-91c7-cfe1cdf32a61-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "21690a66-4bcf-4865-91c7-cfe1cdf32a61" (UID: "21690a66-4bcf-4865-91c7-cfe1cdf32a61"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:32:29 crc kubenswrapper[5052]: I1203 12:32:29.904710 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage01-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage01-crc\") on node \"crc\" " Dec 03 12:32:29 crc kubenswrapper[5052]: I1203 12:32:29.904743 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-cgp57\" (UniqueName: \"kubernetes.io/projected/2c03f99a-98fd-4e86-ad29-fb69e94bd436-kube-api-access-cgp57\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:29 crc kubenswrapper[5052]: I1203 12:32:29.904754 5052 reconciler_common.go:293] "Volume detached for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/21690a66-4bcf-4865-91c7-cfe1cdf32a61-ceilometer-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:29 crc kubenswrapper[5052]: I1203 12:32:29.904762 5052 reconciler_common.go:293] "Volume detached for volume \"rabbitmq-tls\" (UniqueName: \"kubernetes.io/projected/b8613a94-1e2e-477f-8d52-713c70f5282c-rabbitmq-tls\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:29 crc kubenswrapper[5052]: I1203 12:32:29.904771 5052 reconciler_common.go:293] "Volume detached for volume \"plugins-conf\" (UniqueName: \"kubernetes.io/configmap/b8613a94-1e2e-477f-8d52-713c70f5282c-plugins-conf\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:29 crc kubenswrapper[5052]: I1203 12:32:29.904781 5052 reconciler_common.go:293] "Volume detached for volume \"erlang-cookie-secret\" (UniqueName: \"kubernetes.io/secret/b8613a94-1e2e-477f-8d52-713c70f5282c-erlang-cookie-secret\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:29 crc kubenswrapper[5052]: I1203 12:32:29.904789 5052 reconciler_common.go:293] "Volume detached for volume \"pod-info\" (UniqueName: \"kubernetes.io/downward-api/b8613a94-1e2e-477f-8d52-713c70f5282c-pod-info\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:29 crc kubenswrapper[5052]: I1203 12:32:29.904803 5052 reconciler_common.go:293] "Volume detached for volume \"server-conf\" (UniqueName: \"kubernetes.io/configmap/b8613a94-1e2e-477f-8d52-713c70f5282c-server-conf\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:29 crc kubenswrapper[5052]: I1203 12:32:29.904811 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/21690a66-4bcf-4865-91c7-cfe1cdf32a61-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:29 crc kubenswrapper[5052]: I1203 12:32:29.904819 5052 reconciler_common.go:293] "Volume detached for volume \"rabbitmq-erlang-cookie\" (UniqueName: \"kubernetes.io/empty-dir/b8613a94-1e2e-477f-8d52-713c70f5282c-rabbitmq-erlang-cookie\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:29 crc kubenswrapper[5052]: I1203 12:32:29.904829 5052 reconciler_common.go:293] "Volume detached for volume \"server-conf\" (UniqueName: \"kubernetes.io/configmap/2c03f99a-98fd-4e86-ad29-fb69e94bd436-server-conf\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:29 crc kubenswrapper[5052]: I1203 12:32:29.904838 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/b8613a94-1e2e-477f-8d52-713c70f5282c-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:29 crc kubenswrapper[5052]: I1203 12:32:29.904846 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/2c03f99a-98fd-4e86-ad29-fb69e94bd436-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:29 crc kubenswrapper[5052]: I1203 12:32:29.904855 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/019689d4-d40b-4e9a-a028-e427f1aa9d9b-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:29 crc kubenswrapper[5052]: I1203 12:32:29.904863 5052 reconciler_common.go:293] "Volume detached for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/21690a66-4bcf-4865-91c7-cfe1cdf32a61-sg-core-conf-yaml\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:29 crc kubenswrapper[5052]: I1203 12:32:29.904871 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/019689d4-d40b-4e9a-a028-e427f1aa9d9b-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:29 crc kubenswrapper[5052]: I1203 12:32:29.904880 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-6sgkj\" (UniqueName: \"kubernetes.io/projected/b8613a94-1e2e-477f-8d52-713c70f5282c-kube-api-access-6sgkj\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:29 crc kubenswrapper[5052]: I1203 12:32:29.910425 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/21690a66-4bcf-4865-91c7-cfe1cdf32a61-config-data" (OuterVolumeSpecName: "config-data") pod "21690a66-4bcf-4865-91c7-cfe1cdf32a61" (UID: "21690a66-4bcf-4865-91c7-cfe1cdf32a61"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:32:29 crc kubenswrapper[5052]: I1203 12:32:29.923171 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage01-crc" (UniqueName: "kubernetes.io/local-volume/local-storage01-crc") on node "crc" Dec 03 12:32:29 crc kubenswrapper[5052]: I1203 12:32:29.924045 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b8613a94-1e2e-477f-8d52-713c70f5282c-rabbitmq-confd" (OuterVolumeSpecName: "rabbitmq-confd") pod "b8613a94-1e2e-477f-8d52-713c70f5282c" (UID: "b8613a94-1e2e-477f-8d52-713c70f5282c"). InnerVolumeSpecName "rabbitmq-confd". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:32:29 crc kubenswrapper[5052]: I1203 12:32:29.933876 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-58d7cc6794-9z49k" Dec 03 12:32:29 crc kubenswrapper[5052]: I1203 12:32:29.941234 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2c03f99a-98fd-4e86-ad29-fb69e94bd436-rabbitmq-confd" (OuterVolumeSpecName: "rabbitmq-confd") pod "2c03f99a-98fd-4e86-ad29-fb69e94bd436" (UID: "2c03f99a-98fd-4e86-ad29-fb69e94bd436"). InnerVolumeSpecName "rabbitmq-confd". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:32:30 crc kubenswrapper[5052]: I1203 12:32:30.006691 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4b3e73a0-1320-427f-bcd7-d8062e315e39-combined-ca-bundle\") pod \"4b3e73a0-1320-427f-bcd7-d8062e315e39\" (UID: \"4b3e73a0-1320-427f-bcd7-d8062e315e39\") " Dec 03 12:32:30 crc kubenswrapper[5052]: I1203 12:32:30.006783 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/4b3e73a0-1320-427f-bcd7-d8062e315e39-ovndb-tls-certs\") pod \"4b3e73a0-1320-427f-bcd7-d8062e315e39\" (UID: \"4b3e73a0-1320-427f-bcd7-d8062e315e39\") " Dec 03 12:32:30 crc kubenswrapper[5052]: I1203 12:32:30.006958 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/4b3e73a0-1320-427f-bcd7-d8062e315e39-httpd-config\") pod \"4b3e73a0-1320-427f-bcd7-d8062e315e39\" (UID: \"4b3e73a0-1320-427f-bcd7-d8062e315e39\") " Dec 03 12:32:30 crc kubenswrapper[5052]: I1203 12:32:30.007070 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/4b3e73a0-1320-427f-bcd7-d8062e315e39-public-tls-certs\") pod \"4b3e73a0-1320-427f-bcd7-d8062e315e39\" (UID: \"4b3e73a0-1320-427f-bcd7-d8062e315e39\") " Dec 03 12:32:30 crc kubenswrapper[5052]: I1203 12:32:30.007117 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/4b3e73a0-1320-427f-bcd7-d8062e315e39-internal-tls-certs\") pod \"4b3e73a0-1320-427f-bcd7-d8062e315e39\" (UID: \"4b3e73a0-1320-427f-bcd7-d8062e315e39\") " Dec 03 12:32:30 crc kubenswrapper[5052]: I1203 12:32:30.007171 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wmkts\" (UniqueName: \"kubernetes.io/projected/4b3e73a0-1320-427f-bcd7-d8062e315e39-kube-api-access-wmkts\") pod \"4b3e73a0-1320-427f-bcd7-d8062e315e39\" (UID: \"4b3e73a0-1320-427f-bcd7-d8062e315e39\") " Dec 03 12:32:30 crc kubenswrapper[5052]: I1203 12:32:30.007310 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/4b3e73a0-1320-427f-bcd7-d8062e315e39-config\") pod \"4b3e73a0-1320-427f-bcd7-d8062e315e39\" (UID: \"4b3e73a0-1320-427f-bcd7-d8062e315e39\") " Dec 03 12:32:30 crc kubenswrapper[5052]: E1203 12:32:30.007966 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:32:30 crc kubenswrapper[5052]: E1203 12:32:30.008058 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/478e18e8-077e-406e-aa66-19bc1d28df46-operator-scripts podName:478e18e8-077e-406e-aa66-19bc1d28df46 nodeName:}" failed. No retries permitted until 2025-12-03 12:32:38.008033463 +0000 UTC m=+4348.596275227 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/478e18e8-077e-406e-aa66-19bc1d28df46-operator-scripts") pod "novacell046b8-account-delete-nqzlj" (UID: "478e18e8-077e-406e-aa66-19bc1d28df46") : configmap "openstack-scripts" not found Dec 03 12:32:30 crc kubenswrapper[5052]: I1203 12:32:30.008327 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/21690a66-4bcf-4865-91c7-cfe1cdf32a61-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:30 crc kubenswrapper[5052]: I1203 12:32:30.008351 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage01-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage01-crc\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:30 crc kubenswrapper[5052]: I1203 12:32:30.008365 5052 reconciler_common.go:293] "Volume detached for volume \"rabbitmq-confd\" (UniqueName: \"kubernetes.io/projected/2c03f99a-98fd-4e86-ad29-fb69e94bd436-rabbitmq-confd\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:30 crc kubenswrapper[5052]: I1203 12:32:30.008378 5052 reconciler_common.go:293] "Volume detached for volume \"rabbitmq-confd\" (UniqueName: \"kubernetes.io/projected/b8613a94-1e2e-477f-8d52-713c70f5282c-rabbitmq-confd\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:30 crc kubenswrapper[5052]: I1203 12:32:30.009844 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4b3e73a0-1320-427f-bcd7-d8062e315e39-httpd-config" (OuterVolumeSpecName: "httpd-config") pod "4b3e73a0-1320-427f-bcd7-d8062e315e39" (UID: "4b3e73a0-1320-427f-bcd7-d8062e315e39"). InnerVolumeSpecName "httpd-config". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:32:30 crc kubenswrapper[5052]: I1203 12:32:30.012452 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4b3e73a0-1320-427f-bcd7-d8062e315e39-kube-api-access-wmkts" (OuterVolumeSpecName: "kube-api-access-wmkts") pod "4b3e73a0-1320-427f-bcd7-d8062e315e39" (UID: "4b3e73a0-1320-427f-bcd7-d8062e315e39"). InnerVolumeSpecName "kube-api-access-wmkts". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:32:30 crc kubenswrapper[5052]: I1203 12:32:30.055043 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4b3e73a0-1320-427f-bcd7-d8062e315e39-config" (OuterVolumeSpecName: "config") pod "4b3e73a0-1320-427f-bcd7-d8062e315e39" (UID: "4b3e73a0-1320-427f-bcd7-d8062e315e39"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:32:30 crc kubenswrapper[5052]: I1203 12:32:30.055660 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4b3e73a0-1320-427f-bcd7-d8062e315e39-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "4b3e73a0-1320-427f-bcd7-d8062e315e39" (UID: "4b3e73a0-1320-427f-bcd7-d8062e315e39"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:32:30 crc kubenswrapper[5052]: I1203 12:32:30.057189 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4b3e73a0-1320-427f-bcd7-d8062e315e39-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "4b3e73a0-1320-427f-bcd7-d8062e315e39" (UID: "4b3e73a0-1320-427f-bcd7-d8062e315e39"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:32:30 crc kubenswrapper[5052]: I1203 12:32:30.075318 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4b3e73a0-1320-427f-bcd7-d8062e315e39-ovndb-tls-certs" (OuterVolumeSpecName: "ovndb-tls-certs") pod "4b3e73a0-1320-427f-bcd7-d8062e315e39" (UID: "4b3e73a0-1320-427f-bcd7-d8062e315e39"). InnerVolumeSpecName "ovndb-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:32:30 crc kubenswrapper[5052]: I1203 12:32:30.077169 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4b3e73a0-1320-427f-bcd7-d8062e315e39-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "4b3e73a0-1320-427f-bcd7-d8062e315e39" (UID: "4b3e73a0-1320-427f-bcd7-d8062e315e39"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:32:30 crc kubenswrapper[5052]: I1203 12:32:30.110055 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/4b3e73a0-1320-427f-bcd7-d8062e315e39-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:30 crc kubenswrapper[5052]: I1203 12:32:30.110157 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4b3e73a0-1320-427f-bcd7-d8062e315e39-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:30 crc kubenswrapper[5052]: I1203 12:32:30.110187 5052 reconciler_common.go:293] "Volume detached for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/4b3e73a0-1320-427f-bcd7-d8062e315e39-ovndb-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:30 crc kubenswrapper[5052]: I1203 12:32:30.110244 5052 reconciler_common.go:293] "Volume detached for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/4b3e73a0-1320-427f-bcd7-d8062e315e39-httpd-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:30 crc kubenswrapper[5052]: I1203 12:32:30.110272 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/4b3e73a0-1320-427f-bcd7-d8062e315e39-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:30 crc kubenswrapper[5052]: I1203 12:32:30.110284 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/4b3e73a0-1320-427f-bcd7-d8062e315e39-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:30 crc kubenswrapper[5052]: I1203 12:32:30.110297 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-wmkts\" (UniqueName: \"kubernetes.io/projected/4b3e73a0-1320-427f-bcd7-d8062e315e39-kube-api-access-wmkts\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:30 crc kubenswrapper[5052]: I1203 12:32:30.176423 5052 generic.go:334] "Generic (PLEG): container finished" podID="21690a66-4bcf-4865-91c7-cfe1cdf32a61" containerID="6cd8e96f6ff38ec31288f641a86e7f8fed72b9e87063c8cee94ed8bad6b902cc" exitCode=0 Dec 03 12:32:30 crc kubenswrapper[5052]: I1203 12:32:30.176497 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"21690a66-4bcf-4865-91c7-cfe1cdf32a61","Type":"ContainerDied","Data":"6cd8e96f6ff38ec31288f641a86e7f8fed72b9e87063c8cee94ed8bad6b902cc"} Dec 03 12:32:30 crc kubenswrapper[5052]: I1203 12:32:30.176541 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:32:30 crc kubenswrapper[5052]: I1203 12:32:30.176559 5052 scope.go:117] "RemoveContainer" containerID="fb673c0dc2e5ca2f8ef44bb06008daf9443cc163a93859ac48d11b2d619a8092" Dec 03 12:32:30 crc kubenswrapper[5052]: I1203 12:32:30.176545 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"21690a66-4bcf-4865-91c7-cfe1cdf32a61","Type":"ContainerDied","Data":"8b11486373024ed488c2261d2db65b77b031d3040b3593ca109abaa9ab812230"} Dec 03 12:32:30 crc kubenswrapper[5052]: I1203 12:32:30.184362 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-keystone-listener-cd5b58cfd-jf8vt" Dec 03 12:32:30 crc kubenswrapper[5052]: I1203 12:32:30.184347 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-keystone-listener-cd5b58cfd-jf8vt" event={"ID":"019689d4-d40b-4e9a-a028-e427f1aa9d9b","Type":"ContainerDied","Data":"1388cc58677cb99a408eeb9662787e5c6d0865f5f7b0080c83b290fe93da2278"} Dec 03 12:32:30 crc kubenswrapper[5052]: I1203 12:32:30.191502 5052 generic.go:334] "Generic (PLEG): container finished" podID="4b3e73a0-1320-427f-bcd7-d8062e315e39" containerID="e1fe0ce8db29a1d1e0ea67a8eaf8d8a7753223ea2fe9336a061719ff4a0463c3" exitCode=0 Dec 03 12:32:30 crc kubenswrapper[5052]: I1203 12:32:30.191594 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-58d7cc6794-9z49k" event={"ID":"4b3e73a0-1320-427f-bcd7-d8062e315e39","Type":"ContainerDied","Data":"e1fe0ce8db29a1d1e0ea67a8eaf8d8a7753223ea2fe9336a061719ff4a0463c3"} Dec 03 12:32:30 crc kubenswrapper[5052]: I1203 12:32:30.191629 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-58d7cc6794-9z49k" event={"ID":"4b3e73a0-1320-427f-bcd7-d8062e315e39","Type":"ContainerDied","Data":"409382895fc1e13c5f55bec79f9858c93d095ab609a5e7c29543e7085095770b"} Dec 03 12:32:30 crc kubenswrapper[5052]: I1203 12:32:30.191657 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-58d7cc6794-9z49k" Dec 03 12:32:30 crc kubenswrapper[5052]: I1203 12:32:30.206113 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:32:30 crc kubenswrapper[5052]: I1203 12:32:30.206140 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" event={"ID":"b8613a94-1e2e-477f-8d52-713c70f5282c","Type":"ContainerDied","Data":"655770c6b6b108506058af5b98eb7807e27572e48bb338cd2a061b9f4c2ce826"} Dec 03 12:32:30 crc kubenswrapper[5052]: I1203 12:32:30.206118 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:32:30 crc kubenswrapper[5052]: I1203 12:32:30.218361 5052 scope.go:117] "RemoveContainer" containerID="4588847c60a7e1e54c1d68283ca53c647caedbb74b422846f258c273285a8d29" Dec 03 12:32:30 crc kubenswrapper[5052]: I1203 12:32:30.249190 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:32:30 crc kubenswrapper[5052]: I1203 12:32:30.259280 5052 scope.go:117] "RemoveContainer" containerID="6cd8e96f6ff38ec31288f641a86e7f8fed72b9e87063c8cee94ed8bad6b902cc" Dec 03 12:32:30 crc kubenswrapper[5052]: I1203 12:32:30.269611 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:32:30 crc kubenswrapper[5052]: I1203 12:32:30.284404 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/neutron-58d7cc6794-9z49k"] Dec 03 12:32:30 crc kubenswrapper[5052]: I1203 12:32:30.292454 5052 scope.go:117] "RemoveContainer" containerID="3c2c53aa7eb07d8c8a4be850fb78e1b62b32812e4e50fd988ba119d0f7c3a22a" Dec 03 12:32:30 crc kubenswrapper[5052]: I1203 12:32:30.293236 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/neutron-58d7cc6794-9z49k"] Dec 03 12:32:30 crc kubenswrapper[5052]: I1203 12:32:30.303640 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-keystone-listener-cd5b58cfd-jf8vt"] Dec 03 12:32:30 crc kubenswrapper[5052]: I1203 12:32:30.318229 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/barbican-keystone-listener-cd5b58cfd-jf8vt"] Dec 03 12:32:30 crc kubenswrapper[5052]: I1203 12:32:30.328545 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/rabbitmq-server-0"] Dec 03 12:32:30 crc kubenswrapper[5052]: I1203 12:32:30.330549 5052 scope.go:117] "RemoveContainer" containerID="fb673c0dc2e5ca2f8ef44bb06008daf9443cc163a93859ac48d11b2d619a8092" Dec 03 12:32:30 crc kubenswrapper[5052]: E1203 12:32:30.330987 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fb673c0dc2e5ca2f8ef44bb06008daf9443cc163a93859ac48d11b2d619a8092\": container with ID starting with fb673c0dc2e5ca2f8ef44bb06008daf9443cc163a93859ac48d11b2d619a8092 not found: ID does not exist" containerID="fb673c0dc2e5ca2f8ef44bb06008daf9443cc163a93859ac48d11b2d619a8092" Dec 03 12:32:30 crc kubenswrapper[5052]: I1203 12:32:30.331117 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fb673c0dc2e5ca2f8ef44bb06008daf9443cc163a93859ac48d11b2d619a8092"} err="failed to get container status \"fb673c0dc2e5ca2f8ef44bb06008daf9443cc163a93859ac48d11b2d619a8092\": rpc error: code = NotFound desc = could not find container \"fb673c0dc2e5ca2f8ef44bb06008daf9443cc163a93859ac48d11b2d619a8092\": container with ID starting with fb673c0dc2e5ca2f8ef44bb06008daf9443cc163a93859ac48d11b2d619a8092 not found: ID does not exist" Dec 03 12:32:30 crc kubenswrapper[5052]: I1203 12:32:30.331225 5052 scope.go:117] "RemoveContainer" containerID="4588847c60a7e1e54c1d68283ca53c647caedbb74b422846f258c273285a8d29" Dec 03 12:32:30 crc kubenswrapper[5052]: E1203 12:32:30.331558 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4588847c60a7e1e54c1d68283ca53c647caedbb74b422846f258c273285a8d29\": container with ID starting with 4588847c60a7e1e54c1d68283ca53c647caedbb74b422846f258c273285a8d29 not found: ID does not exist" containerID="4588847c60a7e1e54c1d68283ca53c647caedbb74b422846f258c273285a8d29" Dec 03 12:32:30 crc kubenswrapper[5052]: I1203 12:32:30.331664 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4588847c60a7e1e54c1d68283ca53c647caedbb74b422846f258c273285a8d29"} err="failed to get container status \"4588847c60a7e1e54c1d68283ca53c647caedbb74b422846f258c273285a8d29\": rpc error: code = NotFound desc = could not find container \"4588847c60a7e1e54c1d68283ca53c647caedbb74b422846f258c273285a8d29\": container with ID starting with 4588847c60a7e1e54c1d68283ca53c647caedbb74b422846f258c273285a8d29 not found: ID does not exist" Dec 03 12:32:30 crc kubenswrapper[5052]: I1203 12:32:30.332042 5052 scope.go:117] "RemoveContainer" containerID="6cd8e96f6ff38ec31288f641a86e7f8fed72b9e87063c8cee94ed8bad6b902cc" Dec 03 12:32:30 crc kubenswrapper[5052]: E1203 12:32:30.332426 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6cd8e96f6ff38ec31288f641a86e7f8fed72b9e87063c8cee94ed8bad6b902cc\": container with ID starting with 6cd8e96f6ff38ec31288f641a86e7f8fed72b9e87063c8cee94ed8bad6b902cc not found: ID does not exist" containerID="6cd8e96f6ff38ec31288f641a86e7f8fed72b9e87063c8cee94ed8bad6b902cc" Dec 03 12:32:30 crc kubenswrapper[5052]: I1203 12:32:30.332542 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6cd8e96f6ff38ec31288f641a86e7f8fed72b9e87063c8cee94ed8bad6b902cc"} err="failed to get container status \"6cd8e96f6ff38ec31288f641a86e7f8fed72b9e87063c8cee94ed8bad6b902cc\": rpc error: code = NotFound desc = could not find container \"6cd8e96f6ff38ec31288f641a86e7f8fed72b9e87063c8cee94ed8bad6b902cc\": container with ID starting with 6cd8e96f6ff38ec31288f641a86e7f8fed72b9e87063c8cee94ed8bad6b902cc not found: ID does not exist" Dec 03 12:32:30 crc kubenswrapper[5052]: I1203 12:32:30.332647 5052 scope.go:117] "RemoveContainer" containerID="3c2c53aa7eb07d8c8a4be850fb78e1b62b32812e4e50fd988ba119d0f7c3a22a" Dec 03 12:32:30 crc kubenswrapper[5052]: E1203 12:32:30.332980 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3c2c53aa7eb07d8c8a4be850fb78e1b62b32812e4e50fd988ba119d0f7c3a22a\": container with ID starting with 3c2c53aa7eb07d8c8a4be850fb78e1b62b32812e4e50fd988ba119d0f7c3a22a not found: ID does not exist" containerID="3c2c53aa7eb07d8c8a4be850fb78e1b62b32812e4e50fd988ba119d0f7c3a22a" Dec 03 12:32:30 crc kubenswrapper[5052]: I1203 12:32:30.333097 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3c2c53aa7eb07d8c8a4be850fb78e1b62b32812e4e50fd988ba119d0f7c3a22a"} err="failed to get container status \"3c2c53aa7eb07d8c8a4be850fb78e1b62b32812e4e50fd988ba119d0f7c3a22a\": rpc error: code = NotFound desc = could not find container \"3c2c53aa7eb07d8c8a4be850fb78e1b62b32812e4e50fd988ba119d0f7c3a22a\": container with ID starting with 3c2c53aa7eb07d8c8a4be850fb78e1b62b32812e4e50fd988ba119d0f7c3a22a not found: ID does not exist" Dec 03 12:32:30 crc kubenswrapper[5052]: I1203 12:32:30.333162 5052 scope.go:117] "RemoveContainer" containerID="a988e23bbce42676cbdcb4abb64e1c8911c6df2c4d18faf34f5c6450d0879857" Dec 03 12:32:30 crc kubenswrapper[5052]: I1203 12:32:30.340684 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/rabbitmq-server-0"] Dec 03 12:32:30 crc kubenswrapper[5052]: I1203 12:32:30.349998 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/rabbitmq-cell1-server-0"] Dec 03 12:32:30 crc kubenswrapper[5052]: I1203 12:32:30.352258 5052 scope.go:117] "RemoveContainer" containerID="30570e4c9e23e90b86c1701d30a06063dcb7b48a31496a770e386fcaa5d3ac42" Dec 03 12:32:30 crc kubenswrapper[5052]: I1203 12:32:30.357783 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/rabbitmq-cell1-server-0"] Dec 03 12:32:30 crc kubenswrapper[5052]: I1203 12:32:30.372525 5052 scope.go:117] "RemoveContainer" containerID="82c2ce15a7d7e6c87d6ed20e3eaafec5eb24b16b4abc6508d88b41511f4560a0" Dec 03 12:32:30 crc kubenswrapper[5052]: I1203 12:32:30.393634 5052 scope.go:117] "RemoveContainer" containerID="e1fe0ce8db29a1d1e0ea67a8eaf8d8a7753223ea2fe9336a061719ff4a0463c3" Dec 03 12:32:30 crc kubenswrapper[5052]: I1203 12:32:30.416409 5052 scope.go:117] "RemoveContainer" containerID="82c2ce15a7d7e6c87d6ed20e3eaafec5eb24b16b4abc6508d88b41511f4560a0" Dec 03 12:32:30 crc kubenswrapper[5052]: E1203 12:32:30.418594 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"82c2ce15a7d7e6c87d6ed20e3eaafec5eb24b16b4abc6508d88b41511f4560a0\": container with ID starting with 82c2ce15a7d7e6c87d6ed20e3eaafec5eb24b16b4abc6508d88b41511f4560a0 not found: ID does not exist" containerID="82c2ce15a7d7e6c87d6ed20e3eaafec5eb24b16b4abc6508d88b41511f4560a0" Dec 03 12:32:30 crc kubenswrapper[5052]: I1203 12:32:30.418635 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"82c2ce15a7d7e6c87d6ed20e3eaafec5eb24b16b4abc6508d88b41511f4560a0"} err="failed to get container status \"82c2ce15a7d7e6c87d6ed20e3eaafec5eb24b16b4abc6508d88b41511f4560a0\": rpc error: code = NotFound desc = could not find container \"82c2ce15a7d7e6c87d6ed20e3eaafec5eb24b16b4abc6508d88b41511f4560a0\": container with ID starting with 82c2ce15a7d7e6c87d6ed20e3eaafec5eb24b16b4abc6508d88b41511f4560a0 not found: ID does not exist" Dec 03 12:32:30 crc kubenswrapper[5052]: I1203 12:32:30.418663 5052 scope.go:117] "RemoveContainer" containerID="e1fe0ce8db29a1d1e0ea67a8eaf8d8a7753223ea2fe9336a061719ff4a0463c3" Dec 03 12:32:30 crc kubenswrapper[5052]: E1203 12:32:30.419032 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e1fe0ce8db29a1d1e0ea67a8eaf8d8a7753223ea2fe9336a061719ff4a0463c3\": container with ID starting with e1fe0ce8db29a1d1e0ea67a8eaf8d8a7753223ea2fe9336a061719ff4a0463c3 not found: ID does not exist" containerID="e1fe0ce8db29a1d1e0ea67a8eaf8d8a7753223ea2fe9336a061719ff4a0463c3" Dec 03 12:32:30 crc kubenswrapper[5052]: I1203 12:32:30.419088 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e1fe0ce8db29a1d1e0ea67a8eaf8d8a7753223ea2fe9336a061719ff4a0463c3"} err="failed to get container status \"e1fe0ce8db29a1d1e0ea67a8eaf8d8a7753223ea2fe9336a061719ff4a0463c3\": rpc error: code = NotFound desc = could not find container \"e1fe0ce8db29a1d1e0ea67a8eaf8d8a7753223ea2fe9336a061719ff4a0463c3\": container with ID starting with e1fe0ce8db29a1d1e0ea67a8eaf8d8a7753223ea2fe9336a061719ff4a0463c3 not found: ID does not exist" Dec 03 12:32:30 crc kubenswrapper[5052]: I1203 12:32:30.419107 5052 scope.go:117] "RemoveContainer" containerID="ecd92b1e5f2c4bb6955463da6c223b6fc821bf3dfbf16ddd65252cba7d35cea5" Dec 03 12:32:30 crc kubenswrapper[5052]: I1203 12:32:30.454938 5052 scope.go:117] "RemoveContainer" containerID="fdf73317a2bb37c60cc14493b2e7c924d803a06e05cbba2abbe853ddbf7c4fb6" Dec 03 12:32:30 crc kubenswrapper[5052]: I1203 12:32:30.524903 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:32:30 crc kubenswrapper[5052]: I1203 12:32:30.525336 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-scheduler-0" podUID="f23969c4-775b-49b1-a88f-6fe74fd1db75" containerName="nova-scheduler-scheduler" containerID="cri-o://9f8a3f17d990e268d789edd61d07026ea31ff500db1611d7cec6ad35c4fc914c" gracePeriod=30 Dec 03 12:32:30 crc kubenswrapper[5052]: I1203 12:32:30.671794 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/memcached-0"] Dec 03 12:32:30 crc kubenswrapper[5052]: I1203 12:32:30.672317 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/memcached-0" podUID="19fe6517-a9bd-4463-b2b1-e006bd5d8145" containerName="memcached" containerID="cri-o://486a415678ff9d664168e49847f09106017f5e5fe69b855bfe47c5bb2d5e7dcd" gracePeriod=30 Dec 03 12:32:30 crc kubenswrapper[5052]: I1203 12:32:30.683839 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-0"] Dec 03 12:32:30 crc kubenswrapper[5052]: I1203 12:32:30.684063 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-cell1-conductor-0" podUID="2dfbec8a-3a04-407b-8145-e12a3afee2e0" containerName="nova-cell1-conductor-conductor" containerID="cri-o://9d08b86310d4fa1fec88a9c29833ad8e46ae7768dfb1a4176e3e8e54c54b84ea" gracePeriod=30 Dec 03 12:32:30 crc kubenswrapper[5052]: I1203 12:32:30.691068 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-db-sync-f6zrf"] Dec 03 12:32:30 crc kubenswrapper[5052]: I1203 12:32:30.703194 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-db-sync-f6zrf"] Dec 03 12:32:30 crc kubenswrapper[5052]: I1203 12:32:30.726019 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-db-sync-gpqnk"] Dec 03 12:32:30 crc kubenswrapper[5052]: I1203 12:32:30.726263 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-db-sync-gpqnk"] Dec 03 12:32:30 crc kubenswrapper[5052]: I1203 12:32:30.726366 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-0"] Dec 03 12:32:30 crc kubenswrapper[5052]: I1203 12:32:30.726599 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-cell0-conductor-0" podUID="e2718906-e52d-42fa-ad42-35626e99d461" containerName="nova-cell0-conductor-conductor" containerID="cri-o://db2496cc1591b72c5cf2fe6daf7733c7b2072a229f771d7bbe3144cf1984f444" gracePeriod=30 Dec 03 12:32:31 crc kubenswrapper[5052]: E1203 12:32:31.030778 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:32:31 crc kubenswrapper[5052]: E1203 12:32:31.031116 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ae9024f5-a2b4-49de-aed3-00cb9c806704-operator-scripts podName:ae9024f5-a2b4-49de-aed3-00cb9c806704 nodeName:}" failed. No retries permitted until 2025-12-03 12:32:39.031101442 +0000 UTC m=+4349.619343206 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/ae9024f5-a2b4-49de-aed3-00cb9c806704-operator-scripts") pod "placemente65a-account-delete-s7x52" (UID: "ae9024f5-a2b4-49de-aed3-00cb9c806704") : configmap "openstack-scripts" not found Dec 03 12:32:31 crc kubenswrapper[5052]: I1203 12:32:31.684712 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="019689d4-d40b-4e9a-a028-e427f1aa9d9b" path="/var/lib/kubelet/pods/019689d4-d40b-4e9a-a028-e427f1aa9d9b/volumes" Dec 03 12:32:31 crc kubenswrapper[5052]: I1203 12:32:31.685341 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="21690a66-4bcf-4865-91c7-cfe1cdf32a61" path="/var/lib/kubelet/pods/21690a66-4bcf-4865-91c7-cfe1cdf32a61/volumes" Dec 03 12:32:31 crc kubenswrapper[5052]: I1203 12:32:31.686870 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2c03f99a-98fd-4e86-ad29-fb69e94bd436" path="/var/lib/kubelet/pods/2c03f99a-98fd-4e86-ad29-fb69e94bd436/volumes" Dec 03 12:32:31 crc kubenswrapper[5052]: I1203 12:32:31.687497 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3fd1fe87-a439-480b-a74e-854e7003b7a8" path="/var/lib/kubelet/pods/3fd1fe87-a439-480b-a74e-854e7003b7a8/volumes" Dec 03 12:32:31 crc kubenswrapper[5052]: I1203 12:32:31.688006 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4b3e73a0-1320-427f-bcd7-d8062e315e39" path="/var/lib/kubelet/pods/4b3e73a0-1320-427f-bcd7-d8062e315e39/volumes" Dec 03 12:32:31 crc kubenswrapper[5052]: I1203 12:32:31.689234 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b8613a94-1e2e-477f-8d52-713c70f5282c" path="/var/lib/kubelet/pods/b8613a94-1e2e-477f-8d52-713c70f5282c/volumes" Dec 03 12:32:31 crc kubenswrapper[5052]: I1203 12:32:31.689832 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f18600d3-8cdb-48e0-9b40-4773286fdd82" path="/var/lib/kubelet/pods/f18600d3-8cdb-48e0-9b40-4773286fdd82/volumes" Dec 03 12:32:32 crc kubenswrapper[5052]: E1203 12:32:32.152921 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:32:32 crc kubenswrapper[5052]: E1203 12:32:32.152989 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a2ef0804-75c7-43e3-95fa-ce901a561c9b-operator-scripts podName:a2ef0804-75c7-43e3-95fa-ce901a561c9b nodeName:}" failed. No retries permitted until 2025-12-03 12:32:40.152972113 +0000 UTC m=+4350.741213887 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/a2ef0804-75c7-43e3-95fa-ce901a561c9b-operator-scripts") pod "neutron2174-account-delete-9dnxs" (UID: "a2ef0804-75c7-43e3-95fa-ce901a561c9b") : configmap "openstack-scripts" not found Dec 03 12:32:32 crc kubenswrapper[5052]: I1203 12:32:32.237629 5052 generic.go:334] "Generic (PLEG): container finished" podID="19fe6517-a9bd-4463-b2b1-e006bd5d8145" containerID="486a415678ff9d664168e49847f09106017f5e5fe69b855bfe47c5bb2d5e7dcd" exitCode=0 Dec 03 12:32:32 crc kubenswrapper[5052]: I1203 12:32:32.237675 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/memcached-0" event={"ID":"19fe6517-a9bd-4463-b2b1-e006bd5d8145","Type":"ContainerDied","Data":"486a415678ff9d664168e49847f09106017f5e5fe69b855bfe47c5bb2d5e7dcd"} Dec 03 12:32:32 crc kubenswrapper[5052]: E1203 12:32:32.254088 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:32:32 crc kubenswrapper[5052]: E1203 12:32:32.254176 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/52bf0c8d-03b3-444b-80b6-13d41f5f9fb9-operator-scripts podName:52bf0c8d-03b3-444b-80b6-13d41f5f9fb9 nodeName:}" failed. No retries permitted until 2025-12-03 12:32:40.254161291 +0000 UTC m=+4350.842403055 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/52bf0c8d-03b3-444b-80b6-13d41f5f9fb9-operator-scripts") pod "glance5b9b-account-delete-ggk6q" (UID: "52bf0c8d-03b3-444b-80b6-13d41f5f9fb9") : configmap "openstack-scripts" not found Dec 03 12:32:32 crc kubenswrapper[5052]: E1203 12:32:32.254706 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:32:32 crc kubenswrapper[5052]: E1203 12:32:32.254735 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/fc428005-b653-4a6a-a50d-133c2b4fb8b8-operator-scripts podName:fc428005-b653-4a6a-a50d-133c2b4fb8b8 nodeName:}" failed. No retries permitted until 2025-12-03 12:32:40.254727437 +0000 UTC m=+4350.842969201 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/fc428005-b653-4a6a-a50d-133c2b4fb8b8-operator-scripts") pod "cinderc151-account-delete-h4k5b" (UID: "fc428005-b653-4a6a-a50d-133c2b4fb8b8") : configmap "openstack-scripts" not found Dec 03 12:32:32 crc kubenswrapper[5052]: E1203 12:32:32.254761 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:32:32 crc kubenswrapper[5052]: E1203 12:32:32.254777 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e6d87a68-5121-4088-a5d8-3f205a787b29-operator-scripts podName:e6d87a68-5121-4088-a5d8-3f205a787b29 nodeName:}" failed. No retries permitted until 2025-12-03 12:32:40.254772409 +0000 UTC m=+4350.843014163 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/e6d87a68-5121-4088-a5d8-3f205a787b29-operator-scripts") pod "novaapia918-account-delete-mk25j" (UID: "e6d87a68-5121-4088-a5d8-3f205a787b29") : configmap "openstack-scripts" not found Dec 03 12:32:32 crc kubenswrapper[5052]: I1203 12:32:32.384406 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/memcached-0" Dec 03 12:32:32 crc kubenswrapper[5052]: I1203 12:32:32.447694 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-hjfg4" Dec 03 12:32:32 crc kubenswrapper[5052]: I1203 12:32:32.456446 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/19fe6517-a9bd-4463-b2b1-e006bd5d8145-kolla-config\") pod \"19fe6517-a9bd-4463-b2b1-e006bd5d8145\" (UID: \"19fe6517-a9bd-4463-b2b1-e006bd5d8145\") " Dec 03 12:32:32 crc kubenswrapper[5052]: I1203 12:32:32.456541 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/19fe6517-a9bd-4463-b2b1-e006bd5d8145-combined-ca-bundle\") pod \"19fe6517-a9bd-4463-b2b1-e006bd5d8145\" (UID: \"19fe6517-a9bd-4463-b2b1-e006bd5d8145\") " Dec 03 12:32:32 crc kubenswrapper[5052]: I1203 12:32:32.456651 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gcvg4\" (UniqueName: \"kubernetes.io/projected/19fe6517-a9bd-4463-b2b1-e006bd5d8145-kube-api-access-gcvg4\") pod \"19fe6517-a9bd-4463-b2b1-e006bd5d8145\" (UID: \"19fe6517-a9bd-4463-b2b1-e006bd5d8145\") " Dec 03 12:32:32 crc kubenswrapper[5052]: I1203 12:32:32.456671 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/19fe6517-a9bd-4463-b2b1-e006bd5d8145-config-data\") pod \"19fe6517-a9bd-4463-b2b1-e006bd5d8145\" (UID: \"19fe6517-a9bd-4463-b2b1-e006bd5d8145\") " Dec 03 12:32:32 crc kubenswrapper[5052]: I1203 12:32:32.456747 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"memcached-tls-certs\" (UniqueName: \"kubernetes.io/secret/19fe6517-a9bd-4463-b2b1-e006bd5d8145-memcached-tls-certs\") pod \"19fe6517-a9bd-4463-b2b1-e006bd5d8145\" (UID: \"19fe6517-a9bd-4463-b2b1-e006bd5d8145\") " Dec 03 12:32:32 crc kubenswrapper[5052]: I1203 12:32:32.457430 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/19fe6517-a9bd-4463-b2b1-e006bd5d8145-kolla-config" (OuterVolumeSpecName: "kolla-config") pod "19fe6517-a9bd-4463-b2b1-e006bd5d8145" (UID: "19fe6517-a9bd-4463-b2b1-e006bd5d8145"). InnerVolumeSpecName "kolla-config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:32:32 crc kubenswrapper[5052]: I1203 12:32:32.457989 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/19fe6517-a9bd-4463-b2b1-e006bd5d8145-config-data" (OuterVolumeSpecName: "config-data") pod "19fe6517-a9bd-4463-b2b1-e006bd5d8145" (UID: "19fe6517-a9bd-4463-b2b1-e006bd5d8145"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:32:32 crc kubenswrapper[5052]: I1203 12:32:32.478894 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/19fe6517-a9bd-4463-b2b1-e006bd5d8145-kube-api-access-gcvg4" (OuterVolumeSpecName: "kube-api-access-gcvg4") pod "19fe6517-a9bd-4463-b2b1-e006bd5d8145" (UID: "19fe6517-a9bd-4463-b2b1-e006bd5d8145"). InnerVolumeSpecName "kube-api-access-gcvg4". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:32:32 crc kubenswrapper[5052]: I1203 12:32:32.525495 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-7f9fd59985-48h54"] Dec 03 12:32:32 crc kubenswrapper[5052]: I1203 12:32:32.525748 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-7f9fd59985-48h54" podUID="0f665fa3-6863-43d7-a43c-7e9cf3975345" containerName="dnsmasq-dns" containerID="cri-o://93b52a7aa013e6c6f3027aaba01303c05198f39c6fc6938bc82fb3dc438f99a6" gracePeriod=10 Dec 03 12:32:32 crc kubenswrapper[5052]: I1203 12:32:32.532099 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/19fe6517-a9bd-4463-b2b1-e006bd5d8145-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "19fe6517-a9bd-4463-b2b1-e006bd5d8145" (UID: "19fe6517-a9bd-4463-b2b1-e006bd5d8145"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:32:32 crc kubenswrapper[5052]: I1203 12:32:32.539735 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/19fe6517-a9bd-4463-b2b1-e006bd5d8145-memcached-tls-certs" (OuterVolumeSpecName: "memcached-tls-certs") pod "19fe6517-a9bd-4463-b2b1-e006bd5d8145" (UID: "19fe6517-a9bd-4463-b2b1-e006bd5d8145"). InnerVolumeSpecName "memcached-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:32:32 crc kubenswrapper[5052]: I1203 12:32:32.559420 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/19fe6517-a9bd-4463-b2b1-e006bd5d8145-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:32 crc kubenswrapper[5052]: I1203 12:32:32.559486 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-gcvg4\" (UniqueName: \"kubernetes.io/projected/19fe6517-a9bd-4463-b2b1-e006bd5d8145-kube-api-access-gcvg4\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:32 crc kubenswrapper[5052]: I1203 12:32:32.559568 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/19fe6517-a9bd-4463-b2b1-e006bd5d8145-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:32 crc kubenswrapper[5052]: I1203 12:32:32.559585 5052 reconciler_common.go:293] "Volume detached for volume \"memcached-tls-certs\" (UniqueName: \"kubernetes.io/secret/19fe6517-a9bd-4463-b2b1-e006bd5d8145-memcached-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:32 crc kubenswrapper[5052]: I1203 12:32:32.559597 5052 reconciler_common.go:293] "Volume detached for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/19fe6517-a9bd-4463-b2b1-e006bd5d8145-kolla-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:32 crc kubenswrapper[5052]: I1203 12:32:32.922046 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-7f9fd59985-48h54" Dec 03 12:32:32 crc kubenswrapper[5052]: I1203 12:32:32.969977 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qrf7l\" (UniqueName: \"kubernetes.io/projected/0f665fa3-6863-43d7-a43c-7e9cf3975345-kube-api-access-qrf7l\") pod \"0f665fa3-6863-43d7-a43c-7e9cf3975345\" (UID: \"0f665fa3-6863-43d7-a43c-7e9cf3975345\") " Dec 03 12:32:32 crc kubenswrapper[5052]: I1203 12:32:32.970032 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/0f665fa3-6863-43d7-a43c-7e9cf3975345-dnsmasq-svc\") pod \"0f665fa3-6863-43d7-a43c-7e9cf3975345\" (UID: \"0f665fa3-6863-43d7-a43c-7e9cf3975345\") " Dec 03 12:32:32 crc kubenswrapper[5052]: I1203 12:32:32.970085 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"dns-swift-storage-0\" (UniqueName: \"kubernetes.io/configmap/0f665fa3-6863-43d7-a43c-7e9cf3975345-dns-swift-storage-0\") pod \"0f665fa3-6863-43d7-a43c-7e9cf3975345\" (UID: \"0f665fa3-6863-43d7-a43c-7e9cf3975345\") " Dec 03 12:32:32 crc kubenswrapper[5052]: I1203 12:32:32.970132 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0f665fa3-6863-43d7-a43c-7e9cf3975345-config\") pod \"0f665fa3-6863-43d7-a43c-7e9cf3975345\" (UID: \"0f665fa3-6863-43d7-a43c-7e9cf3975345\") " Dec 03 12:32:32 crc kubenswrapper[5052]: I1203 12:32:32.973689 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0f665fa3-6863-43d7-a43c-7e9cf3975345-kube-api-access-qrf7l" (OuterVolumeSpecName: "kube-api-access-qrf7l") pod "0f665fa3-6863-43d7-a43c-7e9cf3975345" (UID: "0f665fa3-6863-43d7-a43c-7e9cf3975345"). InnerVolumeSpecName "kube-api-access-qrf7l". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:32:33 crc kubenswrapper[5052]: I1203 12:32:33.010553 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0f665fa3-6863-43d7-a43c-7e9cf3975345-dns-swift-storage-0" (OuterVolumeSpecName: "dns-swift-storage-0") pod "0f665fa3-6863-43d7-a43c-7e9cf3975345" (UID: "0f665fa3-6863-43d7-a43c-7e9cf3975345"). InnerVolumeSpecName "dns-swift-storage-0". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:32:33 crc kubenswrapper[5052]: I1203 12:32:33.020994 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0f665fa3-6863-43d7-a43c-7e9cf3975345-config" (OuterVolumeSpecName: "config") pod "0f665fa3-6863-43d7-a43c-7e9cf3975345" (UID: "0f665fa3-6863-43d7-a43c-7e9cf3975345"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:32:33 crc kubenswrapper[5052]: I1203 12:32:33.025755 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0f665fa3-6863-43d7-a43c-7e9cf3975345-dnsmasq-svc" (OuterVolumeSpecName: "dnsmasq-svc") pod "0f665fa3-6863-43d7-a43c-7e9cf3975345" (UID: "0f665fa3-6863-43d7-a43c-7e9cf3975345"). InnerVolumeSpecName "dnsmasq-svc". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:32:33 crc kubenswrapper[5052]: I1203 12:32:33.071780 5052 reconciler_common.go:293] "Volume detached for volume \"dns-swift-storage-0\" (UniqueName: \"kubernetes.io/configmap/0f665fa3-6863-43d7-a43c-7e9cf3975345-dns-swift-storage-0\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:33 crc kubenswrapper[5052]: I1203 12:32:33.071819 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0f665fa3-6863-43d7-a43c-7e9cf3975345-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:33 crc kubenswrapper[5052]: I1203 12:32:33.071835 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-qrf7l\" (UniqueName: \"kubernetes.io/projected/0f665fa3-6863-43d7-a43c-7e9cf3975345-kube-api-access-qrf7l\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:33 crc kubenswrapper[5052]: I1203 12:32:33.071850 5052 reconciler_common.go:293] "Volume detached for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/0f665fa3-6863-43d7-a43c-7e9cf3975345-dnsmasq-svc\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:33 crc kubenswrapper[5052]: I1203 12:32:33.079945 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:32:33 crc kubenswrapper[5052]: I1203 12:32:33.173000 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/2dfbec8a-3a04-407b-8145-e12a3afee2e0-config-data\") pod \"2dfbec8a-3a04-407b-8145-e12a3afee2e0\" (UID: \"2dfbec8a-3a04-407b-8145-e12a3afee2e0\") " Dec 03 12:32:33 crc kubenswrapper[5052]: I1203 12:32:33.173090 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sd9m4\" (UniqueName: \"kubernetes.io/projected/2dfbec8a-3a04-407b-8145-e12a3afee2e0-kube-api-access-sd9m4\") pod \"2dfbec8a-3a04-407b-8145-e12a3afee2e0\" (UID: \"2dfbec8a-3a04-407b-8145-e12a3afee2e0\") " Dec 03 12:32:33 crc kubenswrapper[5052]: I1203 12:32:33.173122 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2dfbec8a-3a04-407b-8145-e12a3afee2e0-combined-ca-bundle\") pod \"2dfbec8a-3a04-407b-8145-e12a3afee2e0\" (UID: \"2dfbec8a-3a04-407b-8145-e12a3afee2e0\") " Dec 03 12:32:33 crc kubenswrapper[5052]: I1203 12:32:33.176672 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2dfbec8a-3a04-407b-8145-e12a3afee2e0-kube-api-access-sd9m4" (OuterVolumeSpecName: "kube-api-access-sd9m4") pod "2dfbec8a-3a04-407b-8145-e12a3afee2e0" (UID: "2dfbec8a-3a04-407b-8145-e12a3afee2e0"). InnerVolumeSpecName "kube-api-access-sd9m4". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:32:33 crc kubenswrapper[5052]: I1203 12:32:33.192665 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2dfbec8a-3a04-407b-8145-e12a3afee2e0-config-data" (OuterVolumeSpecName: "config-data") pod "2dfbec8a-3a04-407b-8145-e12a3afee2e0" (UID: "2dfbec8a-3a04-407b-8145-e12a3afee2e0"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:32:33 crc kubenswrapper[5052]: I1203 12:32:33.194217 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2dfbec8a-3a04-407b-8145-e12a3afee2e0-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "2dfbec8a-3a04-407b-8145-e12a3afee2e0" (UID: "2dfbec8a-3a04-407b-8145-e12a3afee2e0"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:32:33 crc kubenswrapper[5052]: I1203 12:32:33.249381 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/memcached-0" event={"ID":"19fe6517-a9bd-4463-b2b1-e006bd5d8145","Type":"ContainerDied","Data":"c05d4bc250127b98e1df0c953be791f5ea8cd76589505431daa66756603ad4be"} Dec 03 12:32:33 crc kubenswrapper[5052]: I1203 12:32:33.249439 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/memcached-0" Dec 03 12:32:33 crc kubenswrapper[5052]: I1203 12:32:33.249459 5052 scope.go:117] "RemoveContainer" containerID="486a415678ff9d664168e49847f09106017f5e5fe69b855bfe47c5bb2d5e7dcd" Dec 03 12:32:33 crc kubenswrapper[5052]: I1203 12:32:33.252009 5052 generic.go:334] "Generic (PLEG): container finished" podID="2dfbec8a-3a04-407b-8145-e12a3afee2e0" containerID="9d08b86310d4fa1fec88a9c29833ad8e46ae7768dfb1a4176e3e8e54c54b84ea" exitCode=0 Dec 03 12:32:33 crc kubenswrapper[5052]: I1203 12:32:33.252095 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-conductor-0" event={"ID":"2dfbec8a-3a04-407b-8145-e12a3afee2e0","Type":"ContainerDied","Data":"9d08b86310d4fa1fec88a9c29833ad8e46ae7768dfb1a4176e3e8e54c54b84ea"} Dec 03 12:32:33 crc kubenswrapper[5052]: I1203 12:32:33.252123 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-conductor-0" event={"ID":"2dfbec8a-3a04-407b-8145-e12a3afee2e0","Type":"ContainerDied","Data":"1cdb30cee2fbfa2eeec820d62852f60c9458350e3fa13e3998acbbd4a219ff59"} Dec 03 12:32:33 crc kubenswrapper[5052]: I1203 12:32:33.252153 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:32:33 crc kubenswrapper[5052]: I1203 12:32:33.254789 5052 generic.go:334] "Generic (PLEG): container finished" podID="0f665fa3-6863-43d7-a43c-7e9cf3975345" containerID="93b52a7aa013e6c6f3027aaba01303c05198f39c6fc6938bc82fb3dc438f99a6" exitCode=0 Dec 03 12:32:33 crc kubenswrapper[5052]: I1203 12:32:33.254832 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-7f9fd59985-48h54" event={"ID":"0f665fa3-6863-43d7-a43c-7e9cf3975345","Type":"ContainerDied","Data":"93b52a7aa013e6c6f3027aaba01303c05198f39c6fc6938bc82fb3dc438f99a6"} Dec 03 12:32:33 crc kubenswrapper[5052]: I1203 12:32:33.254847 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-7f9fd59985-48h54" Dec 03 12:32:33 crc kubenswrapper[5052]: I1203 12:32:33.254861 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-7f9fd59985-48h54" event={"ID":"0f665fa3-6863-43d7-a43c-7e9cf3975345","Type":"ContainerDied","Data":"365f3653c2568d37b362668e23549a4e914687211c12879b235adf576a5b6e58"} Dec 03 12:32:33 crc kubenswrapper[5052]: I1203 12:32:33.271268 5052 scope.go:117] "RemoveContainer" containerID="9d08b86310d4fa1fec88a9c29833ad8e46ae7768dfb1a4176e3e8e54c54b84ea" Dec 03 12:32:33 crc kubenswrapper[5052]: I1203 12:32:33.276292 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-sd9m4\" (UniqueName: \"kubernetes.io/projected/2dfbec8a-3a04-407b-8145-e12a3afee2e0-kube-api-access-sd9m4\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:33 crc kubenswrapper[5052]: I1203 12:32:33.276326 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2dfbec8a-3a04-407b-8145-e12a3afee2e0-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:33 crc kubenswrapper[5052]: I1203 12:32:33.276574 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/2dfbec8a-3a04-407b-8145-e12a3afee2e0-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:33 crc kubenswrapper[5052]: I1203 12:32:33.290146 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/memcached-0"] Dec 03 12:32:33 crc kubenswrapper[5052]: I1203 12:32:33.300995 5052 scope.go:117] "RemoveContainer" containerID="421784ccacc82cea351bc14db49b9cace93a8a873e7c71fae104a1ba2d00d40c" Dec 03 12:32:33 crc kubenswrapper[5052]: I1203 12:32:33.302220 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/memcached-0"] Dec 03 12:32:33 crc kubenswrapper[5052]: I1203 12:32:33.313639 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-0"] Dec 03 12:32:33 crc kubenswrapper[5052]: I1203 12:32:33.319416 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-0"] Dec 03 12:32:33 crc kubenswrapper[5052]: I1203 12:32:33.325207 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-7f9fd59985-48h54"] Dec 03 12:32:33 crc kubenswrapper[5052]: I1203 12:32:33.332113 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-7f9fd59985-48h54"] Dec 03 12:32:33 crc kubenswrapper[5052]: I1203 12:32:33.336590 5052 scope.go:117] "RemoveContainer" containerID="9d08b86310d4fa1fec88a9c29833ad8e46ae7768dfb1a4176e3e8e54c54b84ea" Dec 03 12:32:33 crc kubenswrapper[5052]: E1203 12:32:33.337034 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9d08b86310d4fa1fec88a9c29833ad8e46ae7768dfb1a4176e3e8e54c54b84ea\": container with ID starting with 9d08b86310d4fa1fec88a9c29833ad8e46ae7768dfb1a4176e3e8e54c54b84ea not found: ID does not exist" containerID="9d08b86310d4fa1fec88a9c29833ad8e46ae7768dfb1a4176e3e8e54c54b84ea" Dec 03 12:32:33 crc kubenswrapper[5052]: I1203 12:32:33.337077 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9d08b86310d4fa1fec88a9c29833ad8e46ae7768dfb1a4176e3e8e54c54b84ea"} err="failed to get container status \"9d08b86310d4fa1fec88a9c29833ad8e46ae7768dfb1a4176e3e8e54c54b84ea\": rpc error: code = NotFound desc = could not find container \"9d08b86310d4fa1fec88a9c29833ad8e46ae7768dfb1a4176e3e8e54c54b84ea\": container with ID starting with 9d08b86310d4fa1fec88a9c29833ad8e46ae7768dfb1a4176e3e8e54c54b84ea not found: ID does not exist" Dec 03 12:32:33 crc kubenswrapper[5052]: I1203 12:32:33.337104 5052 scope.go:117] "RemoveContainer" containerID="421784ccacc82cea351bc14db49b9cace93a8a873e7c71fae104a1ba2d00d40c" Dec 03 12:32:33 crc kubenswrapper[5052]: E1203 12:32:33.337419 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"421784ccacc82cea351bc14db49b9cace93a8a873e7c71fae104a1ba2d00d40c\": container with ID starting with 421784ccacc82cea351bc14db49b9cace93a8a873e7c71fae104a1ba2d00d40c not found: ID does not exist" containerID="421784ccacc82cea351bc14db49b9cace93a8a873e7c71fae104a1ba2d00d40c" Dec 03 12:32:33 crc kubenswrapper[5052]: I1203 12:32:33.337451 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"421784ccacc82cea351bc14db49b9cace93a8a873e7c71fae104a1ba2d00d40c"} err="failed to get container status \"421784ccacc82cea351bc14db49b9cace93a8a873e7c71fae104a1ba2d00d40c\": rpc error: code = NotFound desc = could not find container \"421784ccacc82cea351bc14db49b9cace93a8a873e7c71fae104a1ba2d00d40c\": container with ID starting with 421784ccacc82cea351bc14db49b9cace93a8a873e7c71fae104a1ba2d00d40c not found: ID does not exist" Dec 03 12:32:33 crc kubenswrapper[5052]: I1203 12:32:33.337475 5052 scope.go:117] "RemoveContainer" containerID="93b52a7aa013e6c6f3027aaba01303c05198f39c6fc6938bc82fb3dc438f99a6" Dec 03 12:32:33 crc kubenswrapper[5052]: I1203 12:32:33.354799 5052 scope.go:117] "RemoveContainer" containerID="b8429722e957a292c9a5cae7e6de3f9027900fdd6e4da3df96e4d483cfac94ec" Dec 03 12:32:33 crc kubenswrapper[5052]: I1203 12:32:33.380190 5052 scope.go:117] "RemoveContainer" containerID="93b52a7aa013e6c6f3027aaba01303c05198f39c6fc6938bc82fb3dc438f99a6" Dec 03 12:32:33 crc kubenswrapper[5052]: E1203 12:32:33.380751 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"93b52a7aa013e6c6f3027aaba01303c05198f39c6fc6938bc82fb3dc438f99a6\": container with ID starting with 93b52a7aa013e6c6f3027aaba01303c05198f39c6fc6938bc82fb3dc438f99a6 not found: ID does not exist" containerID="93b52a7aa013e6c6f3027aaba01303c05198f39c6fc6938bc82fb3dc438f99a6" Dec 03 12:32:33 crc kubenswrapper[5052]: I1203 12:32:33.380786 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"93b52a7aa013e6c6f3027aaba01303c05198f39c6fc6938bc82fb3dc438f99a6"} err="failed to get container status \"93b52a7aa013e6c6f3027aaba01303c05198f39c6fc6938bc82fb3dc438f99a6\": rpc error: code = NotFound desc = could not find container \"93b52a7aa013e6c6f3027aaba01303c05198f39c6fc6938bc82fb3dc438f99a6\": container with ID starting with 93b52a7aa013e6c6f3027aaba01303c05198f39c6fc6938bc82fb3dc438f99a6 not found: ID does not exist" Dec 03 12:32:33 crc kubenswrapper[5052]: I1203 12:32:33.380812 5052 scope.go:117] "RemoveContainer" containerID="b8429722e957a292c9a5cae7e6de3f9027900fdd6e4da3df96e4d483cfac94ec" Dec 03 12:32:33 crc kubenswrapper[5052]: E1203 12:32:33.381227 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b8429722e957a292c9a5cae7e6de3f9027900fdd6e4da3df96e4d483cfac94ec\": container with ID starting with b8429722e957a292c9a5cae7e6de3f9027900fdd6e4da3df96e4d483cfac94ec not found: ID does not exist" containerID="b8429722e957a292c9a5cae7e6de3f9027900fdd6e4da3df96e4d483cfac94ec" Dec 03 12:32:33 crc kubenswrapper[5052]: I1203 12:32:33.381283 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b8429722e957a292c9a5cae7e6de3f9027900fdd6e4da3df96e4d483cfac94ec"} err="failed to get container status \"b8429722e957a292c9a5cae7e6de3f9027900fdd6e4da3df96e4d483cfac94ec\": rpc error: code = NotFound desc = could not find container \"b8429722e957a292c9a5cae7e6de3f9027900fdd6e4da3df96e4d483cfac94ec\": container with ID starting with b8429722e957a292c9a5cae7e6de3f9027900fdd6e4da3df96e4d483cfac94ec not found: ID does not exist" Dec 03 12:32:33 crc kubenswrapper[5052]: E1203 12:32:33.401480 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="db2496cc1591b72c5cf2fe6daf7733c7b2072a229f771d7bbe3144cf1984f444" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:32:33 crc kubenswrapper[5052]: E1203 12:32:33.403094 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="db2496cc1591b72c5cf2fe6daf7733c7b2072a229f771d7bbe3144cf1984f444" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:32:33 crc kubenswrapper[5052]: E1203 12:32:33.405278 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="db2496cc1591b72c5cf2fe6daf7733c7b2072a229f771d7bbe3144cf1984f444" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:32:33 crc kubenswrapper[5052]: E1203 12:32:33.405357 5052 prober.go:104] "Probe errored" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" probeType="Readiness" pod="openstack-kuttl-tests/nova-cell0-conductor-0" podUID="e2718906-e52d-42fa-ad42-35626e99d461" containerName="nova-cell0-conductor-conductor" Dec 03 12:32:33 crc kubenswrapper[5052]: E1203 12:32:33.617589 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="9f8a3f17d990e268d789edd61d07026ea31ff500db1611d7cec6ad35c4fc914c" cmd=["/usr/bin/pgrep","-r","DRST","nova-scheduler"] Dec 03 12:32:33 crc kubenswrapper[5052]: E1203 12:32:33.618993 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="9f8a3f17d990e268d789edd61d07026ea31ff500db1611d7cec6ad35c4fc914c" cmd=["/usr/bin/pgrep","-r","DRST","nova-scheduler"] Dec 03 12:32:33 crc kubenswrapper[5052]: E1203 12:32:33.620154 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="9f8a3f17d990e268d789edd61d07026ea31ff500db1611d7cec6ad35c4fc914c" cmd=["/usr/bin/pgrep","-r","DRST","nova-scheduler"] Dec 03 12:32:33 crc kubenswrapper[5052]: E1203 12:32:33.620206 5052 prober.go:104] "Probe errored" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" probeType="Readiness" pod="openstack-kuttl-tests/nova-scheduler-0" podUID="f23969c4-775b-49b1-a88f-6fe74fd1db75" containerName="nova-scheduler-scheduler" Dec 03 12:32:33 crc kubenswrapper[5052]: I1203 12:32:33.683074 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0f665fa3-6863-43d7-a43c-7e9cf3975345" path="/var/lib/kubelet/pods/0f665fa3-6863-43d7-a43c-7e9cf3975345/volumes" Dec 03 12:32:33 crc kubenswrapper[5052]: I1203 12:32:33.683736 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="19fe6517-a9bd-4463-b2b1-e006bd5d8145" path="/var/lib/kubelet/pods/19fe6517-a9bd-4463-b2b1-e006bd5d8145/volumes" Dec 03 12:32:33 crc kubenswrapper[5052]: I1203 12:32:33.684191 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2dfbec8a-3a04-407b-8145-e12a3afee2e0" path="/var/lib/kubelet/pods/2dfbec8a-3a04-407b-8145-e12a3afee2e0/volumes" Dec 03 12:32:33 crc kubenswrapper[5052]: E1203 12:32:33.786496 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:32:33 crc kubenswrapper[5052]: E1203 12:32:33.786697 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8a93d2af-9f8c-4282-8611-526869559ae2-operator-scripts podName:8a93d2af-9f8c-4282-8611-526869559ae2 nodeName:}" failed. No retries permitted until 2025-12-03 12:32:41.786667272 +0000 UTC m=+4352.374909036 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/8a93d2af-9f8c-4282-8611-526869559ae2-operator-scripts") pod "barbican3d67-account-delete-qrp98" (UID: "8a93d2af-9f8c-4282-8611-526869559ae2") : configmap "openstack-scripts" not found Dec 03 12:32:36 crc kubenswrapper[5052]: I1203 12:32:36.231168 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:32:36 crc kubenswrapper[5052]: I1203 12:32:36.287931 5052 generic.go:334] "Generic (PLEG): container finished" podID="f23969c4-775b-49b1-a88f-6fe74fd1db75" containerID="9f8a3f17d990e268d789edd61d07026ea31ff500db1611d7cec6ad35c4fc914c" exitCode=0 Dec 03 12:32:36 crc kubenswrapper[5052]: I1203 12:32:36.288344 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"f23969c4-775b-49b1-a88f-6fe74fd1db75","Type":"ContainerDied","Data":"9f8a3f17d990e268d789edd61d07026ea31ff500db1611d7cec6ad35c4fc914c"} Dec 03 12:32:36 crc kubenswrapper[5052]: I1203 12:32:36.288375 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"f23969c4-775b-49b1-a88f-6fe74fd1db75","Type":"ContainerDied","Data":"fa68fb091076369591b072c2f0542f7384d3920d776d3e6a905fb2a53ebd501d"} Dec 03 12:32:36 crc kubenswrapper[5052]: I1203 12:32:36.288393 5052 scope.go:117] "RemoveContainer" containerID="9f8a3f17d990e268d789edd61d07026ea31ff500db1611d7cec6ad35c4fc914c" Dec 03 12:32:36 crc kubenswrapper[5052]: I1203 12:32:36.288551 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:32:36 crc kubenswrapper[5052]: I1203 12:32:36.313219 5052 scope.go:117] "RemoveContainer" containerID="9f8a3f17d990e268d789edd61d07026ea31ff500db1611d7cec6ad35c4fc914c" Dec 03 12:32:36 crc kubenswrapper[5052]: E1203 12:32:36.315897 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9f8a3f17d990e268d789edd61d07026ea31ff500db1611d7cec6ad35c4fc914c\": container with ID starting with 9f8a3f17d990e268d789edd61d07026ea31ff500db1611d7cec6ad35c4fc914c not found: ID does not exist" containerID="9f8a3f17d990e268d789edd61d07026ea31ff500db1611d7cec6ad35c4fc914c" Dec 03 12:32:36 crc kubenswrapper[5052]: I1203 12:32:36.315945 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9f8a3f17d990e268d789edd61d07026ea31ff500db1611d7cec6ad35c4fc914c"} err="failed to get container status \"9f8a3f17d990e268d789edd61d07026ea31ff500db1611d7cec6ad35c4fc914c\": rpc error: code = NotFound desc = could not find container \"9f8a3f17d990e268d789edd61d07026ea31ff500db1611d7cec6ad35c4fc914c\": container with ID starting with 9f8a3f17d990e268d789edd61d07026ea31ff500db1611d7cec6ad35c4fc914c not found: ID does not exist" Dec 03 12:32:36 crc kubenswrapper[5052]: I1203 12:32:36.341831 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/f23969c4-775b-49b1-a88f-6fe74fd1db75-config-data\") pod \"f23969c4-775b-49b1-a88f-6fe74fd1db75\" (UID: \"f23969c4-775b-49b1-a88f-6fe74fd1db75\") " Dec 03 12:32:36 crc kubenswrapper[5052]: I1203 12:32:36.341949 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4b2md\" (UniqueName: \"kubernetes.io/projected/f23969c4-775b-49b1-a88f-6fe74fd1db75-kube-api-access-4b2md\") pod \"f23969c4-775b-49b1-a88f-6fe74fd1db75\" (UID: \"f23969c4-775b-49b1-a88f-6fe74fd1db75\") " Dec 03 12:32:36 crc kubenswrapper[5052]: I1203 12:32:36.342000 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f23969c4-775b-49b1-a88f-6fe74fd1db75-combined-ca-bundle\") pod \"f23969c4-775b-49b1-a88f-6fe74fd1db75\" (UID: \"f23969c4-775b-49b1-a88f-6fe74fd1db75\") " Dec 03 12:32:36 crc kubenswrapper[5052]: I1203 12:32:36.347857 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f23969c4-775b-49b1-a88f-6fe74fd1db75-kube-api-access-4b2md" (OuterVolumeSpecName: "kube-api-access-4b2md") pod "f23969c4-775b-49b1-a88f-6fe74fd1db75" (UID: "f23969c4-775b-49b1-a88f-6fe74fd1db75"). InnerVolumeSpecName "kube-api-access-4b2md". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:32:36 crc kubenswrapper[5052]: I1203 12:32:36.368632 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f23969c4-775b-49b1-a88f-6fe74fd1db75-config-data" (OuterVolumeSpecName: "config-data") pod "f23969c4-775b-49b1-a88f-6fe74fd1db75" (UID: "f23969c4-775b-49b1-a88f-6fe74fd1db75"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:32:36 crc kubenswrapper[5052]: I1203 12:32:36.374249 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f23969c4-775b-49b1-a88f-6fe74fd1db75-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "f23969c4-775b-49b1-a88f-6fe74fd1db75" (UID: "f23969c4-775b-49b1-a88f-6fe74fd1db75"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:32:36 crc kubenswrapper[5052]: I1203 12:32:36.445048 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-4b2md\" (UniqueName: \"kubernetes.io/projected/f23969c4-775b-49b1-a88f-6fe74fd1db75-kube-api-access-4b2md\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:36 crc kubenswrapper[5052]: I1203 12:32:36.445107 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f23969c4-775b-49b1-a88f-6fe74fd1db75-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:36 crc kubenswrapper[5052]: I1203 12:32:36.445126 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/f23969c4-775b-49b1-a88f-6fe74fd1db75-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:36 crc kubenswrapper[5052]: I1203 12:32:36.625381 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:32:36 crc kubenswrapper[5052]: I1203 12:32:36.632245 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:32:36 crc kubenswrapper[5052]: I1203 12:32:36.672958 5052 scope.go:117] "RemoveContainer" containerID="e6edaf188f85ea4f11f487f11cc489c700f25fb42b5ee269f4f496f1e127d772" Dec 03 12:32:36 crc kubenswrapper[5052]: E1203 12:32:36.673244 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 12:32:37 crc kubenswrapper[5052]: I1203 12:32:37.194177 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:32:37 crc kubenswrapper[5052]: I1203 12:32:37.299706 5052 generic.go:334] "Generic (PLEG): container finished" podID="e2718906-e52d-42fa-ad42-35626e99d461" containerID="db2496cc1591b72c5cf2fe6daf7733c7b2072a229f771d7bbe3144cf1984f444" exitCode=0 Dec 03 12:32:37 crc kubenswrapper[5052]: I1203 12:32:37.299775 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:32:37 crc kubenswrapper[5052]: I1203 12:32:37.299789 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-conductor-0" event={"ID":"e2718906-e52d-42fa-ad42-35626e99d461","Type":"ContainerDied","Data":"db2496cc1591b72c5cf2fe6daf7733c7b2072a229f771d7bbe3144cf1984f444"} Dec 03 12:32:37 crc kubenswrapper[5052]: I1203 12:32:37.299845 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-conductor-0" event={"ID":"e2718906-e52d-42fa-ad42-35626e99d461","Type":"ContainerDied","Data":"b62a7c39d4bca0804816b2a547791ef7327cceb21e0af84713fe3c446866867d"} Dec 03 12:32:37 crc kubenswrapper[5052]: I1203 12:32:37.299862 5052 scope.go:117] "RemoveContainer" containerID="db2496cc1591b72c5cf2fe6daf7733c7b2072a229f771d7bbe3144cf1984f444" Dec 03 12:32:37 crc kubenswrapper[5052]: I1203 12:32:37.323972 5052 scope.go:117] "RemoveContainer" containerID="96578ef7df608942efd7e88b79f20d9254a37ea8b6ab07704fddbcfab40445f1" Dec 03 12:32:37 crc kubenswrapper[5052]: I1203 12:32:37.357605 5052 scope.go:117] "RemoveContainer" containerID="db2496cc1591b72c5cf2fe6daf7733c7b2072a229f771d7bbe3144cf1984f444" Dec 03 12:32:37 crc kubenswrapper[5052]: E1203 12:32:37.358198 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"db2496cc1591b72c5cf2fe6daf7733c7b2072a229f771d7bbe3144cf1984f444\": container with ID starting with db2496cc1591b72c5cf2fe6daf7733c7b2072a229f771d7bbe3144cf1984f444 not found: ID does not exist" containerID="db2496cc1591b72c5cf2fe6daf7733c7b2072a229f771d7bbe3144cf1984f444" Dec 03 12:32:37 crc kubenswrapper[5052]: I1203 12:32:37.358251 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"db2496cc1591b72c5cf2fe6daf7733c7b2072a229f771d7bbe3144cf1984f444"} err="failed to get container status \"db2496cc1591b72c5cf2fe6daf7733c7b2072a229f771d7bbe3144cf1984f444\": rpc error: code = NotFound desc = could not find container \"db2496cc1591b72c5cf2fe6daf7733c7b2072a229f771d7bbe3144cf1984f444\": container with ID starting with db2496cc1591b72c5cf2fe6daf7733c7b2072a229f771d7bbe3144cf1984f444 not found: ID does not exist" Dec 03 12:32:37 crc kubenswrapper[5052]: I1203 12:32:37.358285 5052 scope.go:117] "RemoveContainer" containerID="96578ef7df608942efd7e88b79f20d9254a37ea8b6ab07704fddbcfab40445f1" Dec 03 12:32:37 crc kubenswrapper[5052]: E1203 12:32:37.358881 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"96578ef7df608942efd7e88b79f20d9254a37ea8b6ab07704fddbcfab40445f1\": container with ID starting with 96578ef7df608942efd7e88b79f20d9254a37ea8b6ab07704fddbcfab40445f1 not found: ID does not exist" containerID="96578ef7df608942efd7e88b79f20d9254a37ea8b6ab07704fddbcfab40445f1" Dec 03 12:32:37 crc kubenswrapper[5052]: I1203 12:32:37.358955 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"96578ef7df608942efd7e88b79f20d9254a37ea8b6ab07704fddbcfab40445f1"} err="failed to get container status \"96578ef7df608942efd7e88b79f20d9254a37ea8b6ab07704fddbcfab40445f1\": rpc error: code = NotFound desc = could not find container \"96578ef7df608942efd7e88b79f20d9254a37ea8b6ab07704fddbcfab40445f1\": container with ID starting with 96578ef7df608942efd7e88b79f20d9254a37ea8b6ab07704fddbcfab40445f1 not found: ID does not exist" Dec 03 12:32:37 crc kubenswrapper[5052]: I1203 12:32:37.361647 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e2718906-e52d-42fa-ad42-35626e99d461-combined-ca-bundle\") pod \"e2718906-e52d-42fa-ad42-35626e99d461\" (UID: \"e2718906-e52d-42fa-ad42-35626e99d461\") " Dec 03 12:32:37 crc kubenswrapper[5052]: I1203 12:32:37.361689 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/e2718906-e52d-42fa-ad42-35626e99d461-config-data\") pod \"e2718906-e52d-42fa-ad42-35626e99d461\" (UID: \"e2718906-e52d-42fa-ad42-35626e99d461\") " Dec 03 12:32:37 crc kubenswrapper[5052]: I1203 12:32:37.362250 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4jn4f\" (UniqueName: \"kubernetes.io/projected/e2718906-e52d-42fa-ad42-35626e99d461-kube-api-access-4jn4f\") pod \"e2718906-e52d-42fa-ad42-35626e99d461\" (UID: \"e2718906-e52d-42fa-ad42-35626e99d461\") " Dec 03 12:32:37 crc kubenswrapper[5052]: I1203 12:32:37.367043 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e2718906-e52d-42fa-ad42-35626e99d461-kube-api-access-4jn4f" (OuterVolumeSpecName: "kube-api-access-4jn4f") pod "e2718906-e52d-42fa-ad42-35626e99d461" (UID: "e2718906-e52d-42fa-ad42-35626e99d461"). InnerVolumeSpecName "kube-api-access-4jn4f". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:32:37 crc kubenswrapper[5052]: I1203 12:32:37.381890 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e2718906-e52d-42fa-ad42-35626e99d461-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "e2718906-e52d-42fa-ad42-35626e99d461" (UID: "e2718906-e52d-42fa-ad42-35626e99d461"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:32:37 crc kubenswrapper[5052]: I1203 12:32:37.390986 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e2718906-e52d-42fa-ad42-35626e99d461-config-data" (OuterVolumeSpecName: "config-data") pod "e2718906-e52d-42fa-ad42-35626e99d461" (UID: "e2718906-e52d-42fa-ad42-35626e99d461"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:32:37 crc kubenswrapper[5052]: I1203 12:32:37.463954 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-4jn4f\" (UniqueName: \"kubernetes.io/projected/e2718906-e52d-42fa-ad42-35626e99d461-kube-api-access-4jn4f\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:37 crc kubenswrapper[5052]: I1203 12:32:37.463996 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e2718906-e52d-42fa-ad42-35626e99d461-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:37 crc kubenswrapper[5052]: I1203 12:32:37.464009 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/e2718906-e52d-42fa-ad42-35626e99d461-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:37 crc kubenswrapper[5052]: I1203 12:32:37.633838 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-0"] Dec 03 12:32:37 crc kubenswrapper[5052]: I1203 12:32:37.639903 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-0"] Dec 03 12:32:37 crc kubenswrapper[5052]: I1203 12:32:37.683478 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e2718906-e52d-42fa-ad42-35626e99d461" path="/var/lib/kubelet/pods/e2718906-e52d-42fa-ad42-35626e99d461/volumes" Dec 03 12:32:37 crc kubenswrapper[5052]: I1203 12:32:37.684255 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f23969c4-775b-49b1-a88f-6fe74fd1db75" path="/var/lib/kubelet/pods/f23969c4-775b-49b1-a88f-6fe74fd1db75/volumes" Dec 03 12:32:38 crc kubenswrapper[5052]: E1203 12:32:38.072582 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:32:38 crc kubenswrapper[5052]: E1203 12:32:38.072669 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/478e18e8-077e-406e-aa66-19bc1d28df46-operator-scripts podName:478e18e8-077e-406e-aa66-19bc1d28df46 nodeName:}" failed. No retries permitted until 2025-12-03 12:32:54.072651983 +0000 UTC m=+4364.660893747 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/478e18e8-077e-406e-aa66-19bc1d28df46-operator-scripts") pod "novacell046b8-account-delete-nqzlj" (UID: "478e18e8-077e-406e-aa66-19bc1d28df46") : configmap "openstack-scripts" not found Dec 03 12:32:39 crc kubenswrapper[5052]: E1203 12:32:39.088693 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:32:39 crc kubenswrapper[5052]: E1203 12:32:39.088792 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ae9024f5-a2b4-49de-aed3-00cb9c806704-operator-scripts podName:ae9024f5-a2b4-49de-aed3-00cb9c806704 nodeName:}" failed. No retries permitted until 2025-12-03 12:32:55.088770045 +0000 UTC m=+4365.677011819 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/ae9024f5-a2b4-49de-aed3-00cb9c806704-operator-scripts") pod "placemente65a-account-delete-s7x52" (UID: "ae9024f5-a2b4-49de-aed3-00cb9c806704") : configmap "openstack-scripts" not found Dec 03 12:32:40 crc kubenswrapper[5052]: E1203 12:32:40.206449 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:32:40 crc kubenswrapper[5052]: E1203 12:32:40.206784 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a2ef0804-75c7-43e3-95fa-ce901a561c9b-operator-scripts podName:a2ef0804-75c7-43e3-95fa-ce901a561c9b nodeName:}" failed. No retries permitted until 2025-12-03 12:32:56.206767325 +0000 UTC m=+4366.795009089 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/a2ef0804-75c7-43e3-95fa-ce901a561c9b-operator-scripts") pod "neutron2174-account-delete-9dnxs" (UID: "a2ef0804-75c7-43e3-95fa-ce901a561c9b") : configmap "openstack-scripts" not found Dec 03 12:32:40 crc kubenswrapper[5052]: E1203 12:32:40.308485 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:32:40 crc kubenswrapper[5052]: E1203 12:32:40.308548 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:32:40 crc kubenswrapper[5052]: E1203 12:32:40.308597 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/fc428005-b653-4a6a-a50d-133c2b4fb8b8-operator-scripts podName:fc428005-b653-4a6a-a50d-133c2b4fb8b8 nodeName:}" failed. No retries permitted until 2025-12-03 12:32:56.308578731 +0000 UTC m=+4366.896820495 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/fc428005-b653-4a6a-a50d-133c2b4fb8b8-operator-scripts") pod "cinderc151-account-delete-h4k5b" (UID: "fc428005-b653-4a6a-a50d-133c2b4fb8b8") : configmap "openstack-scripts" not found Dec 03 12:32:40 crc kubenswrapper[5052]: E1203 12:32:40.308608 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:32:40 crc kubenswrapper[5052]: E1203 12:32:40.308655 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e6d87a68-5121-4088-a5d8-3f205a787b29-operator-scripts podName:e6d87a68-5121-4088-a5d8-3f205a787b29 nodeName:}" failed. No retries permitted until 2025-12-03 12:32:56.308615832 +0000 UTC m=+4366.896857656 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/e6d87a68-5121-4088-a5d8-3f205a787b29-operator-scripts") pod "novaapia918-account-delete-mk25j" (UID: "e6d87a68-5121-4088-a5d8-3f205a787b29") : configmap "openstack-scripts" not found Dec 03 12:32:40 crc kubenswrapper[5052]: E1203 12:32:40.308706 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/52bf0c8d-03b3-444b-80b6-13d41f5f9fb9-operator-scripts podName:52bf0c8d-03b3-444b-80b6-13d41f5f9fb9 nodeName:}" failed. No retries permitted until 2025-12-03 12:32:56.308682604 +0000 UTC m=+4366.896924388 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/52bf0c8d-03b3-444b-80b6-13d41f5f9fb9-operator-scripts") pod "glance5b9b-account-delete-ggk6q" (UID: "52bf0c8d-03b3-444b-80b6-13d41f5f9fb9") : configmap "openstack-scripts" not found Dec 03 12:32:41 crc kubenswrapper[5052]: E1203 12:32:41.835167 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:32:41 crc kubenswrapper[5052]: E1203 12:32:41.835336 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8a93d2af-9f8c-4282-8611-526869559ae2-operator-scripts podName:8a93d2af-9f8c-4282-8611-526869559ae2 nodeName:}" failed. No retries permitted until 2025-12-03 12:32:57.835255065 +0000 UTC m=+4368.423496859 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/8a93d2af-9f8c-4282-8611-526869559ae2-operator-scripts") pod "barbican3d67-account-delete-qrp98" (UID: "8a93d2af-9f8c-4282-8611-526869559ae2") : configmap "openstack-scripts" not found Dec 03 12:32:50 crc kubenswrapper[5052]: I1203 12:32:50.839346 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openshift-marketplace/community-operators-x22v4"] Dec 03 12:32:50 crc kubenswrapper[5052]: E1203 12:32:50.840323 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="2dfbec8a-3a04-407b-8145-e12a3afee2e0" containerName="nova-cell1-conductor-conductor" Dec 03 12:32:50 crc kubenswrapper[5052]: I1203 12:32:50.840338 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="2dfbec8a-3a04-407b-8145-e12a3afee2e0" containerName="nova-cell1-conductor-conductor" Dec 03 12:32:50 crc kubenswrapper[5052]: E1203 12:32:50.840353 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="21690a66-4bcf-4865-91c7-cfe1cdf32a61" containerName="sg-core" Dec 03 12:32:50 crc kubenswrapper[5052]: I1203 12:32:50.840360 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="21690a66-4bcf-4865-91c7-cfe1cdf32a61" containerName="sg-core" Dec 03 12:32:50 crc kubenswrapper[5052]: E1203 12:32:50.840375 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="cfd8af50-a644-4b24-8483-dd344f2218e4" containerName="placement-api" Dec 03 12:32:50 crc kubenswrapper[5052]: I1203 12:32:50.840384 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="cfd8af50-a644-4b24-8483-dd344f2218e4" containerName="placement-api" Dec 03 12:32:50 crc kubenswrapper[5052]: E1203 12:32:50.840460 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="43933abc-db67-4ce5-9f30-6f8dbf3f59c8" containerName="keystone-api" Dec 03 12:32:50 crc kubenswrapper[5052]: I1203 12:32:50.840468 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="43933abc-db67-4ce5-9f30-6f8dbf3f59c8" containerName="keystone-api" Dec 03 12:32:50 crc kubenswrapper[5052]: E1203 12:32:50.840487 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="0f665fa3-6863-43d7-a43c-7e9cf3975345" containerName="init" Dec 03 12:32:50 crc kubenswrapper[5052]: I1203 12:32:50.840496 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="0f665fa3-6863-43d7-a43c-7e9cf3975345" containerName="init" Dec 03 12:32:50 crc kubenswrapper[5052]: E1203 12:32:50.840530 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="a9f65397-a08a-4798-8532-646209c5d090" containerName="ovn-northd" Dec 03 12:32:50 crc kubenswrapper[5052]: I1203 12:32:50.840539 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="a9f65397-a08a-4798-8532-646209c5d090" containerName="ovn-northd" Dec 03 12:32:50 crc kubenswrapper[5052]: E1203 12:32:50.840549 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="6e0f19c7-34ae-4d8e-9a19-755ae1c20c1c" containerName="cinder-api-log" Dec 03 12:32:50 crc kubenswrapper[5052]: I1203 12:32:50.840556 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="6e0f19c7-34ae-4d8e-9a19-755ae1c20c1c" containerName="cinder-api-log" Dec 03 12:32:50 crc kubenswrapper[5052]: E1203 12:32:50.840564 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="b8613a94-1e2e-477f-8d52-713c70f5282c" containerName="setup-container" Dec 03 12:32:50 crc kubenswrapper[5052]: I1203 12:32:50.840576 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="b8613a94-1e2e-477f-8d52-713c70f5282c" containerName="setup-container" Dec 03 12:32:50 crc kubenswrapper[5052]: E1203 12:32:50.840592 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="16cac95c-d1d2-4542-bb8e-4bdc1923c684" containerName="proxy-server" Dec 03 12:32:50 crc kubenswrapper[5052]: I1203 12:32:50.840599 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="16cac95c-d1d2-4542-bb8e-4bdc1923c684" containerName="proxy-server" Dec 03 12:32:50 crc kubenswrapper[5052]: E1203 12:32:50.840614 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="bb1723c7-4e41-4b0e-b6b8-f8024dafdaee" containerName="probe" Dec 03 12:32:50 crc kubenswrapper[5052]: I1203 12:32:50.840621 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb1723c7-4e41-4b0e-b6b8-f8024dafdaee" containerName="probe" Dec 03 12:32:50 crc kubenswrapper[5052]: E1203 12:32:50.840633 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="e2718906-e52d-42fa-ad42-35626e99d461" containerName="nova-cell0-conductor-conductor" Dec 03 12:32:50 crc kubenswrapper[5052]: I1203 12:32:50.840645 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="e2718906-e52d-42fa-ad42-35626e99d461" containerName="nova-cell0-conductor-conductor" Dec 03 12:32:50 crc kubenswrapper[5052]: E1203 12:32:50.840658 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="8ac6e8f0-d49c-4a4c-846a-6cec226bd360" containerName="barbican-worker" Dec 03 12:32:50 crc kubenswrapper[5052]: I1203 12:32:50.840666 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="8ac6e8f0-d49c-4a4c-846a-6cec226bd360" containerName="barbican-worker" Dec 03 12:32:50 crc kubenswrapper[5052]: E1203 12:32:50.840676 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="0f665fa3-6863-43d7-a43c-7e9cf3975345" containerName="dnsmasq-dns" Dec 03 12:32:50 crc kubenswrapper[5052]: I1203 12:32:50.840686 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="0f665fa3-6863-43d7-a43c-7e9cf3975345" containerName="dnsmasq-dns" Dec 03 12:32:50 crc kubenswrapper[5052]: E1203 12:32:50.840699 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="dccae7c1-009e-4b3b-b66a-ecbf6e53489a" containerName="openstack-network-exporter" Dec 03 12:32:50 crc kubenswrapper[5052]: I1203 12:32:50.840707 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="dccae7c1-009e-4b3b-b66a-ecbf6e53489a" containerName="openstack-network-exporter" Dec 03 12:32:50 crc kubenswrapper[5052]: E1203 12:32:50.840718 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="16cac95c-d1d2-4542-bb8e-4bdc1923c684" containerName="proxy-httpd" Dec 03 12:32:50 crc kubenswrapper[5052]: I1203 12:32:50.840725 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="16cac95c-d1d2-4542-bb8e-4bdc1923c684" containerName="proxy-httpd" Dec 03 12:32:50 crc kubenswrapper[5052]: E1203 12:32:50.840740 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="8ac6e8f0-d49c-4a4c-846a-6cec226bd360" containerName="barbican-worker-log" Dec 03 12:32:50 crc kubenswrapper[5052]: I1203 12:32:50.840748 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="8ac6e8f0-d49c-4a4c-846a-6cec226bd360" containerName="barbican-worker-log" Dec 03 12:32:50 crc kubenswrapper[5052]: E1203 12:32:50.840764 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="44520845-0277-47ab-acc6-812aba4656ff" containerName="kube-state-metrics" Dec 03 12:32:50 crc kubenswrapper[5052]: I1203 12:32:50.840773 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="44520845-0277-47ab-acc6-812aba4656ff" containerName="kube-state-metrics" Dec 03 12:32:50 crc kubenswrapper[5052]: E1203 12:32:50.840783 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="5737900d-97ae-4dcb-a65c-b40fe30cdc50" containerName="glance-log" Dec 03 12:32:50 crc kubenswrapper[5052]: I1203 12:32:50.840790 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="5737900d-97ae-4dcb-a65c-b40fe30cdc50" containerName="glance-log" Dec 03 12:32:50 crc kubenswrapper[5052]: E1203 12:32:50.840799 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="a9f65397-a08a-4798-8532-646209c5d090" containerName="openstack-network-exporter" Dec 03 12:32:50 crc kubenswrapper[5052]: I1203 12:32:50.840807 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="a9f65397-a08a-4798-8532-646209c5d090" containerName="openstack-network-exporter" Dec 03 12:32:50 crc kubenswrapper[5052]: E1203 12:32:50.840824 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="019689d4-d40b-4e9a-a028-e427f1aa9d9b" containerName="barbican-keystone-listener-log" Dec 03 12:32:50 crc kubenswrapper[5052]: I1203 12:32:50.840832 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="019689d4-d40b-4e9a-a028-e427f1aa9d9b" containerName="barbican-keystone-listener-log" Dec 03 12:32:50 crc kubenswrapper[5052]: E1203 12:32:50.840848 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="bb1723c7-4e41-4b0e-b6b8-f8024dafdaee" containerName="cinder-scheduler" Dec 03 12:32:50 crc kubenswrapper[5052]: I1203 12:32:50.840856 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb1723c7-4e41-4b0e-b6b8-f8024dafdaee" containerName="cinder-scheduler" Dec 03 12:32:50 crc kubenswrapper[5052]: E1203 12:32:50.840863 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="e2718906-e52d-42fa-ad42-35626e99d461" containerName="nova-cell0-conductor-conductor" Dec 03 12:32:50 crc kubenswrapper[5052]: I1203 12:32:50.840871 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="e2718906-e52d-42fa-ad42-35626e99d461" containerName="nova-cell0-conductor-conductor" Dec 03 12:32:50 crc kubenswrapper[5052]: E1203 12:32:50.840885 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="3fec3449-9615-4377-a600-69d816ec5416" containerName="openstack-network-exporter" Dec 03 12:32:50 crc kubenswrapper[5052]: I1203 12:32:50.840892 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="3fec3449-9615-4377-a600-69d816ec5416" containerName="openstack-network-exporter" Dec 03 12:32:50 crc kubenswrapper[5052]: E1203 12:32:50.840905 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="cfd8af50-a644-4b24-8483-dd344f2218e4" containerName="placement-log" Dec 03 12:32:50 crc kubenswrapper[5052]: I1203 12:32:50.840913 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="cfd8af50-a644-4b24-8483-dd344f2218e4" containerName="placement-log" Dec 03 12:32:50 crc kubenswrapper[5052]: E1203 12:32:50.840922 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="2c03f99a-98fd-4e86-ad29-fb69e94bd436" containerName="rabbitmq" Dec 03 12:32:50 crc kubenswrapper[5052]: I1203 12:32:50.840929 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c03f99a-98fd-4e86-ad29-fb69e94bd436" containerName="rabbitmq" Dec 03 12:32:50 crc kubenswrapper[5052]: E1203 12:32:50.840941 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="6e0f19c7-34ae-4d8e-9a19-755ae1c20c1c" containerName="cinder-api" Dec 03 12:32:50 crc kubenswrapper[5052]: I1203 12:32:50.840948 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="6e0f19c7-34ae-4d8e-9a19-755ae1c20c1c" containerName="cinder-api" Dec 03 12:32:50 crc kubenswrapper[5052]: E1203 12:32:50.840965 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="4b3e73a0-1320-427f-bcd7-d8062e315e39" containerName="neutron-httpd" Dec 03 12:32:50 crc kubenswrapper[5052]: I1203 12:32:50.840973 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="4b3e73a0-1320-427f-bcd7-d8062e315e39" containerName="neutron-httpd" Dec 03 12:32:50 crc kubenswrapper[5052]: E1203 12:32:50.840988 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="e2718906-e52d-42fa-ad42-35626e99d461" containerName="nova-cell0-conductor-conductor" Dec 03 12:32:50 crc kubenswrapper[5052]: I1203 12:32:50.840996 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="e2718906-e52d-42fa-ad42-35626e99d461" containerName="nova-cell0-conductor-conductor" Dec 03 12:32:50 crc kubenswrapper[5052]: E1203 12:32:50.841004 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="2dfbec8a-3a04-407b-8145-e12a3afee2e0" containerName="nova-cell1-conductor-conductor" Dec 03 12:32:50 crc kubenswrapper[5052]: I1203 12:32:50.841012 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="2dfbec8a-3a04-407b-8145-e12a3afee2e0" containerName="nova-cell1-conductor-conductor" Dec 03 12:32:50 crc kubenswrapper[5052]: E1203 12:32:50.841023 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="b8613a94-1e2e-477f-8d52-713c70f5282c" containerName="rabbitmq" Dec 03 12:32:50 crc kubenswrapper[5052]: I1203 12:32:50.841030 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="b8613a94-1e2e-477f-8d52-713c70f5282c" containerName="rabbitmq" Dec 03 12:32:50 crc kubenswrapper[5052]: E1203 12:32:50.841043 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="21690a66-4bcf-4865-91c7-cfe1cdf32a61" containerName="ceilometer-central-agent" Dec 03 12:32:50 crc kubenswrapper[5052]: I1203 12:32:50.841051 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="21690a66-4bcf-4865-91c7-cfe1cdf32a61" containerName="ceilometer-central-agent" Dec 03 12:32:50 crc kubenswrapper[5052]: E1203 12:32:50.841065 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="5737900d-97ae-4dcb-a65c-b40fe30cdc50" containerName="glance-httpd" Dec 03 12:32:50 crc kubenswrapper[5052]: I1203 12:32:50.841072 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="5737900d-97ae-4dcb-a65c-b40fe30cdc50" containerName="glance-httpd" Dec 03 12:32:50 crc kubenswrapper[5052]: E1203 12:32:50.841088 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="21690a66-4bcf-4865-91c7-cfe1cdf32a61" containerName="proxy-httpd" Dec 03 12:32:50 crc kubenswrapper[5052]: I1203 12:32:50.841095 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="21690a66-4bcf-4865-91c7-cfe1cdf32a61" containerName="proxy-httpd" Dec 03 12:32:50 crc kubenswrapper[5052]: E1203 12:32:50.841110 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="2c03f99a-98fd-4e86-ad29-fb69e94bd436" containerName="setup-container" Dec 03 12:32:50 crc kubenswrapper[5052]: I1203 12:32:50.841117 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c03f99a-98fd-4e86-ad29-fb69e94bd436" containerName="setup-container" Dec 03 12:32:50 crc kubenswrapper[5052]: E1203 12:32:50.841132 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="d09b9817-6ea5-4328-9027-3311b7782157" containerName="nova-cell1-novncproxy-novncproxy" Dec 03 12:32:50 crc kubenswrapper[5052]: I1203 12:32:50.841139 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="d09b9817-6ea5-4328-9027-3311b7782157" containerName="nova-cell1-novncproxy-novncproxy" Dec 03 12:32:50 crc kubenswrapper[5052]: E1203 12:32:50.841154 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="39117149-c15d-4753-9324-8a42abe5b007" containerName="nova-metadata-log" Dec 03 12:32:50 crc kubenswrapper[5052]: I1203 12:32:50.841162 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="39117149-c15d-4753-9324-8a42abe5b007" containerName="nova-metadata-log" Dec 03 12:32:50 crc kubenswrapper[5052]: E1203 12:32:50.841177 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="dccae7c1-009e-4b3b-b66a-ecbf6e53489a" containerName="ovsdbserver-nb" Dec 03 12:32:50 crc kubenswrapper[5052]: I1203 12:32:50.841184 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="dccae7c1-009e-4b3b-b66a-ecbf6e53489a" containerName="ovsdbserver-nb" Dec 03 12:32:50 crc kubenswrapper[5052]: E1203 12:32:50.841195 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="a65428bc-9326-4ace-a985-9f796b951304" containerName="glance-log" Dec 03 12:32:50 crc kubenswrapper[5052]: I1203 12:32:50.841202 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="a65428bc-9326-4ace-a985-9f796b951304" containerName="glance-log" Dec 03 12:32:50 crc kubenswrapper[5052]: E1203 12:32:50.841216 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="7cbe1c2c-bc04-4232-9901-bc9d134df3d6" containerName="nova-api-api" Dec 03 12:32:50 crc kubenswrapper[5052]: I1203 12:32:50.841223 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="7cbe1c2c-bc04-4232-9901-bc9d134df3d6" containerName="nova-api-api" Dec 03 12:32:50 crc kubenswrapper[5052]: E1203 12:32:50.841236 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="82f142ab-5be9-4727-b3c5-102d6f2401fb" containerName="barbican-api" Dec 03 12:32:50 crc kubenswrapper[5052]: I1203 12:32:50.841244 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="82f142ab-5be9-4727-b3c5-102d6f2401fb" containerName="barbican-api" Dec 03 12:32:50 crc kubenswrapper[5052]: E1203 12:32:50.841259 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="aa55d0c8-56de-4edd-92b3-9252f169fed8" containerName="galera" Dec 03 12:32:50 crc kubenswrapper[5052]: I1203 12:32:50.841265 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa55d0c8-56de-4edd-92b3-9252f169fed8" containerName="galera" Dec 03 12:32:50 crc kubenswrapper[5052]: E1203 12:32:50.841283 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="a65428bc-9326-4ace-a985-9f796b951304" containerName="glance-httpd" Dec 03 12:32:50 crc kubenswrapper[5052]: I1203 12:32:50.841290 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="a65428bc-9326-4ace-a985-9f796b951304" containerName="glance-httpd" Dec 03 12:32:50 crc kubenswrapper[5052]: E1203 12:32:50.841299 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="4b3e73a0-1320-427f-bcd7-d8062e315e39" containerName="neutron-api" Dec 03 12:32:50 crc kubenswrapper[5052]: I1203 12:32:50.841306 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="4b3e73a0-1320-427f-bcd7-d8062e315e39" containerName="neutron-api" Dec 03 12:32:50 crc kubenswrapper[5052]: E1203 12:32:50.841320 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="21690a66-4bcf-4865-91c7-cfe1cdf32a61" containerName="ceilometer-notification-agent" Dec 03 12:32:50 crc kubenswrapper[5052]: I1203 12:32:50.841328 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="21690a66-4bcf-4865-91c7-cfe1cdf32a61" containerName="ceilometer-notification-agent" Dec 03 12:32:50 crc kubenswrapper[5052]: E1203 12:32:50.841340 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="f23969c4-775b-49b1-a88f-6fe74fd1db75" containerName="nova-scheduler-scheduler" Dec 03 12:32:50 crc kubenswrapper[5052]: I1203 12:32:50.841347 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="f23969c4-775b-49b1-a88f-6fe74fd1db75" containerName="nova-scheduler-scheduler" Dec 03 12:32:50 crc kubenswrapper[5052]: E1203 12:32:50.841356 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="7cbe1c2c-bc04-4232-9901-bc9d134df3d6" containerName="nova-api-log" Dec 03 12:32:50 crc kubenswrapper[5052]: I1203 12:32:50.841362 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="7cbe1c2c-bc04-4232-9901-bc9d134df3d6" containerName="nova-api-log" Dec 03 12:32:50 crc kubenswrapper[5052]: E1203 12:32:50.841372 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="82f142ab-5be9-4727-b3c5-102d6f2401fb" containerName="barbican-api-log" Dec 03 12:32:50 crc kubenswrapper[5052]: I1203 12:32:50.841379 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="82f142ab-5be9-4727-b3c5-102d6f2401fb" containerName="barbican-api-log" Dec 03 12:32:50 crc kubenswrapper[5052]: E1203 12:32:50.841392 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="6938204c-1fe8-4f03-ac16-ec9e19e090f8" containerName="mysql-bootstrap" Dec 03 12:32:50 crc kubenswrapper[5052]: I1203 12:32:50.841399 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="6938204c-1fe8-4f03-ac16-ec9e19e090f8" containerName="mysql-bootstrap" Dec 03 12:32:50 crc kubenswrapper[5052]: E1203 12:32:50.841410 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="6938204c-1fe8-4f03-ac16-ec9e19e090f8" containerName="galera" Dec 03 12:32:50 crc kubenswrapper[5052]: I1203 12:32:50.841417 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="6938204c-1fe8-4f03-ac16-ec9e19e090f8" containerName="galera" Dec 03 12:32:50 crc kubenswrapper[5052]: E1203 12:32:50.841429 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="aa55d0c8-56de-4edd-92b3-9252f169fed8" containerName="mysql-bootstrap" Dec 03 12:32:50 crc kubenswrapper[5052]: I1203 12:32:50.841437 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa55d0c8-56de-4edd-92b3-9252f169fed8" containerName="mysql-bootstrap" Dec 03 12:32:50 crc kubenswrapper[5052]: E1203 12:32:50.841446 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="39117149-c15d-4753-9324-8a42abe5b007" containerName="nova-metadata-metadata" Dec 03 12:32:50 crc kubenswrapper[5052]: I1203 12:32:50.841453 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="39117149-c15d-4753-9324-8a42abe5b007" containerName="nova-metadata-metadata" Dec 03 12:32:50 crc kubenswrapper[5052]: E1203 12:32:50.841465 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="19fe6517-a9bd-4463-b2b1-e006bd5d8145" containerName="memcached" Dec 03 12:32:50 crc kubenswrapper[5052]: I1203 12:32:50.841472 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="19fe6517-a9bd-4463-b2b1-e006bd5d8145" containerName="memcached" Dec 03 12:32:50 crc kubenswrapper[5052]: E1203 12:32:50.841485 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="019689d4-d40b-4e9a-a028-e427f1aa9d9b" containerName="barbican-keystone-listener" Dec 03 12:32:50 crc kubenswrapper[5052]: I1203 12:32:50.841493 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="019689d4-d40b-4e9a-a028-e427f1aa9d9b" containerName="barbican-keystone-listener" Dec 03 12:32:50 crc kubenswrapper[5052]: E1203 12:32:50.841525 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="3fec3449-9615-4377-a600-69d816ec5416" containerName="ovsdbserver-sb" Dec 03 12:32:50 crc kubenswrapper[5052]: I1203 12:32:50.841536 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="3fec3449-9615-4377-a600-69d816ec5416" containerName="ovsdbserver-sb" Dec 03 12:32:50 crc kubenswrapper[5052]: I1203 12:32:50.841756 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="82f142ab-5be9-4727-b3c5-102d6f2401fb" containerName="barbican-api-log" Dec 03 12:32:50 crc kubenswrapper[5052]: I1203 12:32:50.841766 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="dccae7c1-009e-4b3b-b66a-ecbf6e53489a" containerName="openstack-network-exporter" Dec 03 12:32:50 crc kubenswrapper[5052]: I1203 12:32:50.841782 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="16cac95c-d1d2-4542-bb8e-4bdc1923c684" containerName="proxy-httpd" Dec 03 12:32:50 crc kubenswrapper[5052]: I1203 12:32:50.841794 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="8ac6e8f0-d49c-4a4c-846a-6cec226bd360" containerName="barbican-worker" Dec 03 12:32:50 crc kubenswrapper[5052]: I1203 12:32:50.841806 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="39117149-c15d-4753-9324-8a42abe5b007" containerName="nova-metadata-log" Dec 03 12:32:50 crc kubenswrapper[5052]: I1203 12:32:50.841816 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="21690a66-4bcf-4865-91c7-cfe1cdf32a61" containerName="ceilometer-central-agent" Dec 03 12:32:50 crc kubenswrapper[5052]: I1203 12:32:50.841830 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="2c03f99a-98fd-4e86-ad29-fb69e94bd436" containerName="rabbitmq" Dec 03 12:32:50 crc kubenswrapper[5052]: I1203 12:32:50.841844 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="3fec3449-9615-4377-a600-69d816ec5416" containerName="openstack-network-exporter" Dec 03 12:32:50 crc kubenswrapper[5052]: I1203 12:32:50.841858 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="a9f65397-a08a-4798-8532-646209c5d090" containerName="ovn-northd" Dec 03 12:32:50 crc kubenswrapper[5052]: I1203 12:32:50.841869 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="2dfbec8a-3a04-407b-8145-e12a3afee2e0" containerName="nova-cell1-conductor-conductor" Dec 03 12:32:50 crc kubenswrapper[5052]: I1203 12:32:50.841879 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="e2718906-e52d-42fa-ad42-35626e99d461" containerName="nova-cell0-conductor-conductor" Dec 03 12:32:50 crc kubenswrapper[5052]: I1203 12:32:50.841888 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="a65428bc-9326-4ace-a985-9f796b951304" containerName="glance-httpd" Dec 03 12:32:50 crc kubenswrapper[5052]: I1203 12:32:50.841899 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="bb1723c7-4e41-4b0e-b6b8-f8024dafdaee" containerName="probe" Dec 03 12:32:50 crc kubenswrapper[5052]: I1203 12:32:50.841914 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="16cac95c-d1d2-4542-bb8e-4bdc1923c684" containerName="proxy-server" Dec 03 12:32:50 crc kubenswrapper[5052]: I1203 12:32:50.841928 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="0f665fa3-6863-43d7-a43c-7e9cf3975345" containerName="dnsmasq-dns" Dec 03 12:32:50 crc kubenswrapper[5052]: I1203 12:32:50.841938 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="39117149-c15d-4753-9324-8a42abe5b007" containerName="nova-metadata-metadata" Dec 03 12:32:50 crc kubenswrapper[5052]: I1203 12:32:50.841949 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="3fec3449-9615-4377-a600-69d816ec5416" containerName="ovsdbserver-sb" Dec 03 12:32:50 crc kubenswrapper[5052]: I1203 12:32:50.841960 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="6938204c-1fe8-4f03-ac16-ec9e19e090f8" containerName="galera" Dec 03 12:32:50 crc kubenswrapper[5052]: I1203 12:32:50.841970 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="aa55d0c8-56de-4edd-92b3-9252f169fed8" containerName="galera" Dec 03 12:32:50 crc kubenswrapper[5052]: I1203 12:32:50.841980 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="21690a66-4bcf-4865-91c7-cfe1cdf32a61" containerName="proxy-httpd" Dec 03 12:32:50 crc kubenswrapper[5052]: I1203 12:32:50.841990 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="019689d4-d40b-4e9a-a028-e427f1aa9d9b" containerName="barbican-keystone-listener" Dec 03 12:32:50 crc kubenswrapper[5052]: I1203 12:32:50.842000 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="e2718906-e52d-42fa-ad42-35626e99d461" containerName="nova-cell0-conductor-conductor" Dec 03 12:32:50 crc kubenswrapper[5052]: I1203 12:32:50.842009 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="2dfbec8a-3a04-407b-8145-e12a3afee2e0" containerName="nova-cell1-conductor-conductor" Dec 03 12:32:50 crc kubenswrapper[5052]: I1203 12:32:50.842020 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="cfd8af50-a644-4b24-8483-dd344f2218e4" containerName="placement-log" Dec 03 12:32:50 crc kubenswrapper[5052]: I1203 12:32:50.842031 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="5737900d-97ae-4dcb-a65c-b40fe30cdc50" containerName="glance-httpd" Dec 03 12:32:50 crc kubenswrapper[5052]: I1203 12:32:50.842039 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="4b3e73a0-1320-427f-bcd7-d8062e315e39" containerName="neutron-api" Dec 03 12:32:50 crc kubenswrapper[5052]: I1203 12:32:50.842046 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="44520845-0277-47ab-acc6-812aba4656ff" containerName="kube-state-metrics" Dec 03 12:32:50 crc kubenswrapper[5052]: I1203 12:32:50.842058 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="7cbe1c2c-bc04-4232-9901-bc9d134df3d6" containerName="nova-api-api" Dec 03 12:32:50 crc kubenswrapper[5052]: I1203 12:32:50.842067 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="dccae7c1-009e-4b3b-b66a-ecbf6e53489a" containerName="ovsdbserver-nb" Dec 03 12:32:50 crc kubenswrapper[5052]: I1203 12:32:50.842079 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="a9f65397-a08a-4798-8532-646209c5d090" containerName="openstack-network-exporter" Dec 03 12:32:50 crc kubenswrapper[5052]: I1203 12:32:50.842091 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="4b3e73a0-1320-427f-bcd7-d8062e315e39" containerName="neutron-httpd" Dec 03 12:32:50 crc kubenswrapper[5052]: I1203 12:32:50.842099 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="6e0f19c7-34ae-4d8e-9a19-755ae1c20c1c" containerName="cinder-api-log" Dec 03 12:32:50 crc kubenswrapper[5052]: I1203 12:32:50.842110 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="43933abc-db67-4ce5-9f30-6f8dbf3f59c8" containerName="keystone-api" Dec 03 12:32:50 crc kubenswrapper[5052]: I1203 12:32:50.842119 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="7cbe1c2c-bc04-4232-9901-bc9d134df3d6" containerName="nova-api-log" Dec 03 12:32:50 crc kubenswrapper[5052]: I1203 12:32:50.842129 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="cfd8af50-a644-4b24-8483-dd344f2218e4" containerName="placement-api" Dec 03 12:32:50 crc kubenswrapper[5052]: I1203 12:32:50.842143 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="82f142ab-5be9-4727-b3c5-102d6f2401fb" containerName="barbican-api" Dec 03 12:32:50 crc kubenswrapper[5052]: I1203 12:32:50.842158 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="d09b9817-6ea5-4328-9027-3311b7782157" containerName="nova-cell1-novncproxy-novncproxy" Dec 03 12:32:50 crc kubenswrapper[5052]: I1203 12:32:50.842172 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="19fe6517-a9bd-4463-b2b1-e006bd5d8145" containerName="memcached" Dec 03 12:32:50 crc kubenswrapper[5052]: I1203 12:32:50.842180 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="6e0f19c7-34ae-4d8e-9a19-755ae1c20c1c" containerName="cinder-api" Dec 03 12:32:50 crc kubenswrapper[5052]: I1203 12:32:50.842193 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="f23969c4-775b-49b1-a88f-6fe74fd1db75" containerName="nova-scheduler-scheduler" Dec 03 12:32:50 crc kubenswrapper[5052]: I1203 12:32:50.842205 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="bb1723c7-4e41-4b0e-b6b8-f8024dafdaee" containerName="cinder-scheduler" Dec 03 12:32:50 crc kubenswrapper[5052]: I1203 12:32:50.842217 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="21690a66-4bcf-4865-91c7-cfe1cdf32a61" containerName="ceilometer-notification-agent" Dec 03 12:32:50 crc kubenswrapper[5052]: I1203 12:32:50.842230 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="a65428bc-9326-4ace-a985-9f796b951304" containerName="glance-log" Dec 03 12:32:50 crc kubenswrapper[5052]: I1203 12:32:50.842239 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="8ac6e8f0-d49c-4a4c-846a-6cec226bd360" containerName="barbican-worker-log" Dec 03 12:32:50 crc kubenswrapper[5052]: I1203 12:32:50.842249 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="b8613a94-1e2e-477f-8d52-713c70f5282c" containerName="rabbitmq" Dec 03 12:32:50 crc kubenswrapper[5052]: I1203 12:32:50.842259 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="5737900d-97ae-4dcb-a65c-b40fe30cdc50" containerName="glance-log" Dec 03 12:32:50 crc kubenswrapper[5052]: I1203 12:32:50.842270 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="21690a66-4bcf-4865-91c7-cfe1cdf32a61" containerName="sg-core" Dec 03 12:32:50 crc kubenswrapper[5052]: I1203 12:32:50.842280 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="019689d4-d40b-4e9a-a028-e427f1aa9d9b" containerName="barbican-keystone-listener-log" Dec 03 12:32:50 crc kubenswrapper[5052]: E1203 12:32:50.842473 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="2dfbec8a-3a04-407b-8145-e12a3afee2e0" containerName="nova-cell1-conductor-conductor" Dec 03 12:32:50 crc kubenswrapper[5052]: I1203 12:32:50.842483 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="2dfbec8a-3a04-407b-8145-e12a3afee2e0" containerName="nova-cell1-conductor-conductor" Dec 03 12:32:50 crc kubenswrapper[5052]: I1203 12:32:50.842768 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="2dfbec8a-3a04-407b-8145-e12a3afee2e0" containerName="nova-cell1-conductor-conductor" Dec 03 12:32:50 crc kubenswrapper[5052]: I1203 12:32:50.842800 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="e2718906-e52d-42fa-ad42-35626e99d461" containerName="nova-cell0-conductor-conductor" Dec 03 12:32:50 crc kubenswrapper[5052]: I1203 12:32:50.843891 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/community-operators-x22v4" Dec 03 12:32:50 crc kubenswrapper[5052]: I1203 12:32:50.853859 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/community-operators-x22v4"] Dec 03 12:32:50 crc kubenswrapper[5052]: I1203 12:32:50.986337 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/79e33cc2-8e67-4b03-a8cb-f7c257116087-catalog-content\") pod \"community-operators-x22v4\" (UID: \"79e33cc2-8e67-4b03-a8cb-f7c257116087\") " pod="openshift-marketplace/community-operators-x22v4" Dec 03 12:32:50 crc kubenswrapper[5052]: I1203 12:32:50.986451 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2cfmn\" (UniqueName: \"kubernetes.io/projected/79e33cc2-8e67-4b03-a8cb-f7c257116087-kube-api-access-2cfmn\") pod \"community-operators-x22v4\" (UID: \"79e33cc2-8e67-4b03-a8cb-f7c257116087\") " pod="openshift-marketplace/community-operators-x22v4" Dec 03 12:32:50 crc kubenswrapper[5052]: I1203 12:32:50.986467 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/79e33cc2-8e67-4b03-a8cb-f7c257116087-utilities\") pod \"community-operators-x22v4\" (UID: \"79e33cc2-8e67-4b03-a8cb-f7c257116087\") " pod="openshift-marketplace/community-operators-x22v4" Dec 03 12:32:51 crc kubenswrapper[5052]: I1203 12:32:51.089059 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/79e33cc2-8e67-4b03-a8cb-f7c257116087-catalog-content\") pod \"community-operators-x22v4\" (UID: \"79e33cc2-8e67-4b03-a8cb-f7c257116087\") " pod="openshift-marketplace/community-operators-x22v4" Dec 03 12:32:51 crc kubenswrapper[5052]: I1203 12:32:51.089345 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-2cfmn\" (UniqueName: \"kubernetes.io/projected/79e33cc2-8e67-4b03-a8cb-f7c257116087-kube-api-access-2cfmn\") pod \"community-operators-x22v4\" (UID: \"79e33cc2-8e67-4b03-a8cb-f7c257116087\") " pod="openshift-marketplace/community-operators-x22v4" Dec 03 12:32:51 crc kubenswrapper[5052]: I1203 12:32:51.089383 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/79e33cc2-8e67-4b03-a8cb-f7c257116087-utilities\") pod \"community-operators-x22v4\" (UID: \"79e33cc2-8e67-4b03-a8cb-f7c257116087\") " pod="openshift-marketplace/community-operators-x22v4" Dec 03 12:32:51 crc kubenswrapper[5052]: I1203 12:32:51.089690 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/79e33cc2-8e67-4b03-a8cb-f7c257116087-catalog-content\") pod \"community-operators-x22v4\" (UID: \"79e33cc2-8e67-4b03-a8cb-f7c257116087\") " pod="openshift-marketplace/community-operators-x22v4" Dec 03 12:32:51 crc kubenswrapper[5052]: I1203 12:32:51.090272 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/79e33cc2-8e67-4b03-a8cb-f7c257116087-utilities\") pod \"community-operators-x22v4\" (UID: \"79e33cc2-8e67-4b03-a8cb-f7c257116087\") " pod="openshift-marketplace/community-operators-x22v4" Dec 03 12:32:51 crc kubenswrapper[5052]: I1203 12:32:51.224331 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-2cfmn\" (UniqueName: \"kubernetes.io/projected/79e33cc2-8e67-4b03-a8cb-f7c257116087-kube-api-access-2cfmn\") pod \"community-operators-x22v4\" (UID: \"79e33cc2-8e67-4b03-a8cb-f7c257116087\") " pod="openshift-marketplace/community-operators-x22v4" Dec 03 12:32:51 crc kubenswrapper[5052]: I1203 12:32:51.462231 5052 generic.go:334] "Generic (PLEG): container finished" podID="9a71429c-9d59-4863-8949-dcb4f79a35e2" containerID="707deedabc8432d6991a028d80075701f8f2bd14b0998f6a88d8ce1c2703f42b" exitCode=137 Dec 03 12:32:51 crc kubenswrapper[5052]: I1203 12:32:51.462295 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"9a71429c-9d59-4863-8949-dcb4f79a35e2","Type":"ContainerDied","Data":"707deedabc8432d6991a028d80075701f8f2bd14b0998f6a88d8ce1c2703f42b"} Dec 03 12:32:51 crc kubenswrapper[5052]: I1203 12:32:51.474753 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/community-operators-x22v4" Dec 03 12:32:51 crc kubenswrapper[5052]: I1203 12:32:51.674269 5052 scope.go:117] "RemoveContainer" containerID="e6edaf188f85ea4f11f487f11cc489c700f25fb42b5ee269f4f496f1e127d772" Dec 03 12:32:51 crc kubenswrapper[5052]: E1203 12:32:51.674915 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 12:32:51 crc kubenswrapper[5052]: I1203 12:32:51.963238 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/community-operators-x22v4"] Dec 03 12:32:52 crc kubenswrapper[5052]: I1203 12:32:52.343357 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:32:52 crc kubenswrapper[5052]: I1203 12:32:52.481161 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"9a71429c-9d59-4863-8949-dcb4f79a35e2","Type":"ContainerDied","Data":"30effbbdf9935462e10a9a1d9efeda543d42e320e8af45e4e06802d2c0e922c2"} Dec 03 12:32:52 crc kubenswrapper[5052]: I1203 12:32:52.481241 5052 scope.go:117] "RemoveContainer" containerID="707deedabc8432d6991a028d80075701f8f2bd14b0998f6a88d8ce1c2703f42b" Dec 03 12:32:52 crc kubenswrapper[5052]: I1203 12:32:52.481578 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:32:52 crc kubenswrapper[5052]: I1203 12:32:52.483634 5052 generic.go:334] "Generic (PLEG): container finished" podID="79e33cc2-8e67-4b03-a8cb-f7c257116087" containerID="597920b9b2b45564d61b5cdfeef47ee3c428fc23b90fbb938197c9a481902574" exitCode=0 Dec 03 12:32:52 crc kubenswrapper[5052]: I1203 12:32:52.483747 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/community-operators-x22v4" event={"ID":"79e33cc2-8e67-4b03-a8cb-f7c257116087","Type":"ContainerDied","Data":"597920b9b2b45564d61b5cdfeef47ee3c428fc23b90fbb938197c9a481902574"} Dec 03 12:32:52 crc kubenswrapper[5052]: I1203 12:32:52.483955 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/community-operators-x22v4" event={"ID":"79e33cc2-8e67-4b03-a8cb-f7c257116087","Type":"ContainerStarted","Data":"524f61b224402b926a7a03d475c254e612088f0550e11356984503e6da740453"} Dec 03 12:32:52 crc kubenswrapper[5052]: I1203 12:32:52.514196 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/9a71429c-9d59-4863-8949-dcb4f79a35e2-etc-swift\") pod \"9a71429c-9d59-4863-8949-dcb4f79a35e2\" (UID: \"9a71429c-9d59-4863-8949-dcb4f79a35e2\") " Dec 03 12:32:52 crc kubenswrapper[5052]: I1203 12:32:52.514336 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"lock\" (UniqueName: \"kubernetes.io/empty-dir/9a71429c-9d59-4863-8949-dcb4f79a35e2-lock\") pod \"9a71429c-9d59-4863-8949-dcb4f79a35e2\" (UID: \"9a71429c-9d59-4863-8949-dcb4f79a35e2\") " Dec 03 12:32:52 crc kubenswrapper[5052]: I1203 12:32:52.514447 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"swift\" (UniqueName: \"kubernetes.io/local-volume/local-storage06-crc\") pod \"9a71429c-9d59-4863-8949-dcb4f79a35e2\" (UID: \"9a71429c-9d59-4863-8949-dcb4f79a35e2\") " Dec 03 12:32:52 crc kubenswrapper[5052]: I1203 12:32:52.514565 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d5r28\" (UniqueName: \"kubernetes.io/projected/9a71429c-9d59-4863-8949-dcb4f79a35e2-kube-api-access-d5r28\") pod \"9a71429c-9d59-4863-8949-dcb4f79a35e2\" (UID: \"9a71429c-9d59-4863-8949-dcb4f79a35e2\") " Dec 03 12:32:52 crc kubenswrapper[5052]: I1203 12:32:52.514654 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"cache\" (UniqueName: \"kubernetes.io/empty-dir/9a71429c-9d59-4863-8949-dcb4f79a35e2-cache\") pod \"9a71429c-9d59-4863-8949-dcb4f79a35e2\" (UID: \"9a71429c-9d59-4863-8949-dcb4f79a35e2\") " Dec 03 12:32:52 crc kubenswrapper[5052]: I1203 12:32:52.515485 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9a71429c-9d59-4863-8949-dcb4f79a35e2-cache" (OuterVolumeSpecName: "cache") pod "9a71429c-9d59-4863-8949-dcb4f79a35e2" (UID: "9a71429c-9d59-4863-8949-dcb4f79a35e2"). InnerVolumeSpecName "cache". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:32:52 crc kubenswrapper[5052]: I1203 12:32:52.515598 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9a71429c-9d59-4863-8949-dcb4f79a35e2-lock" (OuterVolumeSpecName: "lock") pod "9a71429c-9d59-4863-8949-dcb4f79a35e2" (UID: "9a71429c-9d59-4863-8949-dcb4f79a35e2"). InnerVolumeSpecName "lock". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:32:52 crc kubenswrapper[5052]: I1203 12:32:52.515906 5052 reconciler_common.go:293] "Volume detached for volume \"cache\" (UniqueName: \"kubernetes.io/empty-dir/9a71429c-9d59-4863-8949-dcb4f79a35e2-cache\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:52 crc kubenswrapper[5052]: I1203 12:32:52.515930 5052 reconciler_common.go:293] "Volume detached for volume \"lock\" (UniqueName: \"kubernetes.io/empty-dir/9a71429c-9d59-4863-8949-dcb4f79a35e2-lock\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:52 crc kubenswrapper[5052]: I1203 12:32:52.522277 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage06-crc" (OuterVolumeSpecName: "swift") pod "9a71429c-9d59-4863-8949-dcb4f79a35e2" (UID: "9a71429c-9d59-4863-8949-dcb4f79a35e2"). InnerVolumeSpecName "local-storage06-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 12:32:52 crc kubenswrapper[5052]: I1203 12:32:52.522822 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9a71429c-9d59-4863-8949-dcb4f79a35e2-etc-swift" (OuterVolumeSpecName: "etc-swift") pod "9a71429c-9d59-4863-8949-dcb4f79a35e2" (UID: "9a71429c-9d59-4863-8949-dcb4f79a35e2"). InnerVolumeSpecName "etc-swift". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:32:52 crc kubenswrapper[5052]: I1203 12:32:52.522929 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9a71429c-9d59-4863-8949-dcb4f79a35e2-kube-api-access-d5r28" (OuterVolumeSpecName: "kube-api-access-d5r28") pod "9a71429c-9d59-4863-8949-dcb4f79a35e2" (UID: "9a71429c-9d59-4863-8949-dcb4f79a35e2"). InnerVolumeSpecName "kube-api-access-d5r28". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:32:52 crc kubenswrapper[5052]: I1203 12:32:52.523988 5052 scope.go:117] "RemoveContainer" containerID="bdab1d1a8f6c5c7a6c89c49bf44da7d62eab1cd075defa615ab51c3d50e3986e" Dec 03 12:32:52 crc kubenswrapper[5052]: I1203 12:32:52.600976 5052 scope.go:117] "RemoveContainer" containerID="a6e47b8819de3aa2c92fe2ccff5f26450ecf089c945a22ec1bb902b9689f2e90" Dec 03 12:32:52 crc kubenswrapper[5052]: I1203 12:32:52.618362 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-d5r28\" (UniqueName: \"kubernetes.io/projected/9a71429c-9d59-4863-8949-dcb4f79a35e2-kube-api-access-d5r28\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:52 crc kubenswrapper[5052]: I1203 12:32:52.618462 5052 reconciler_common.go:293] "Volume detached for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/9a71429c-9d59-4863-8949-dcb4f79a35e2-etc-swift\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:52 crc kubenswrapper[5052]: I1203 12:32:52.618528 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage06-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage06-crc\") on node \"crc\" " Dec 03 12:32:52 crc kubenswrapper[5052]: I1203 12:32:52.636328 5052 scope.go:117] "RemoveContainer" containerID="59b11d0a23a20a97f6d34709bb5f894ff76b50c46bb80978918201b3b72f674c" Dec 03 12:32:52 crc kubenswrapper[5052]: I1203 12:32:52.665215 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage06-crc" (UniqueName: "kubernetes.io/local-volume/local-storage06-crc") on node "crc" Dec 03 12:32:52 crc kubenswrapper[5052]: I1203 12:32:52.715931 5052 scope.go:117] "RemoveContainer" containerID="6d6d0a9e0c44982301515f6c27041d26e825a4e904631b906720c66939d7d046" Dec 03 12:32:52 crc kubenswrapper[5052]: I1203 12:32:52.720042 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage06-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage06-crc\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:52 crc kubenswrapper[5052]: I1203 12:32:52.758110 5052 scope.go:117] "RemoveContainer" containerID="2a0e35f878dfc50c4b4c6c92dc0efc336af0f313d37084f949296ada2fc8a9a9" Dec 03 12:32:52 crc kubenswrapper[5052]: I1203 12:32:52.789224 5052 scope.go:117] "RemoveContainer" containerID="1313894d0c0976392a7edf9b30f67f35fdafa36110b15f07dc7f5ffd68038ba8" Dec 03 12:32:52 crc kubenswrapper[5052]: I1203 12:32:52.818792 5052 scope.go:117] "RemoveContainer" containerID="e56722fdb2d4fc10478af03b1d3b9ea29d7757b10975bec562e15e2949b85364" Dec 03 12:32:52 crc kubenswrapper[5052]: I1203 12:32:52.831123 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/swift-storage-0"] Dec 03 12:32:52 crc kubenswrapper[5052]: I1203 12:32:52.837298 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/swift-storage-0"] Dec 03 12:32:52 crc kubenswrapper[5052]: I1203 12:32:52.847468 5052 scope.go:117] "RemoveContainer" containerID="844dc1daec944c795b946deb25e04e380651676c9d2e4f9f20f0b937c87cd282" Dec 03 12:32:52 crc kubenswrapper[5052]: I1203 12:32:52.876160 5052 scope.go:117] "RemoveContainer" containerID="7fe073f06854f186159ef18897942608baedecbba0d07d0c8013355c37619643" Dec 03 12:32:52 crc kubenswrapper[5052]: I1203 12:32:52.898124 5052 scope.go:117] "RemoveContainer" containerID="44b04da4e7f0a025325c3b6375369b7ac347c2ad50331290bfb481efa7ef0736" Dec 03 12:32:52 crc kubenswrapper[5052]: E1203 12:32:52.924386 5052 cadvisor_stats_provider.go:516] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9a71429c_9d59_4863_8949_dcb4f79a35e2.slice/crio-30effbbdf9935462e10a9a1d9efeda543d42e320e8af45e4e06802d2c0e922c2\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9a71429c_9d59_4863_8949_dcb4f79a35e2.slice\": RecentStats: unable to find data in memory cache]" Dec 03 12:32:52 crc kubenswrapper[5052]: I1203 12:32:52.925975 5052 scope.go:117] "RemoveContainer" containerID="9d3d20ffcb7e4c042c65cb9789046c8a52471258ecf145006245287d12709e44" Dec 03 12:32:52 crc kubenswrapper[5052]: I1203 12:32:52.974330 5052 scope.go:117] "RemoveContainer" containerID="be42b27cc3c5b3773e9ff8c15fd7487a0c826144d8760ad2e8dbee3a1515ac77" Dec 03 12:32:52 crc kubenswrapper[5052]: I1203 12:32:52.995954 5052 scope.go:117] "RemoveContainer" containerID="9920899d34f69573106a85bde57a4bdf8c52b176a8195efa2c785d4039c638ea" Dec 03 12:32:53 crc kubenswrapper[5052]: I1203 12:32:53.014370 5052 scope.go:117] "RemoveContainer" containerID="1637629a53175a4befbb5dad0d15dd5909adc30a1253d4051e99d888b51f7ccb" Dec 03 12:32:53 crc kubenswrapper[5052]: I1203 12:32:53.496057 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/community-operators-x22v4" event={"ID":"79e33cc2-8e67-4b03-a8cb-f7c257116087","Type":"ContainerStarted","Data":"167f97e954de88b693249438c1a13638c936688a52268f8c206876e013e07b0e"} Dec 03 12:32:53 crc kubenswrapper[5052]: I1203 12:32:53.684737 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9a71429c-9d59-4863-8949-dcb4f79a35e2" path="/var/lib/kubelet/pods/9a71429c-9d59-4863-8949-dcb4f79a35e2/volumes" Dec 03 12:32:54 crc kubenswrapper[5052]: E1203 12:32:54.144201 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:32:54 crc kubenswrapper[5052]: E1203 12:32:54.144958 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/478e18e8-077e-406e-aa66-19bc1d28df46-operator-scripts podName:478e18e8-077e-406e-aa66-19bc1d28df46 nodeName:}" failed. No retries permitted until 2025-12-03 12:33:26.144940318 +0000 UTC m=+4396.733182082 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/478e18e8-077e-406e-aa66-19bc1d28df46-operator-scripts") pod "novacell046b8-account-delete-nqzlj" (UID: "478e18e8-077e-406e-aa66-19bc1d28df46") : configmap "openstack-scripts" not found Dec 03 12:32:54 crc kubenswrapper[5052]: I1203 12:32:54.507221 5052 generic.go:334] "Generic (PLEG): container finished" podID="79e33cc2-8e67-4b03-a8cb-f7c257116087" containerID="167f97e954de88b693249438c1a13638c936688a52268f8c206876e013e07b0e" exitCode=0 Dec 03 12:32:54 crc kubenswrapper[5052]: I1203 12:32:54.507319 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/community-operators-x22v4" event={"ID":"79e33cc2-8e67-4b03-a8cb-f7c257116087","Type":"ContainerDied","Data":"167f97e954de88b693249438c1a13638c936688a52268f8c206876e013e07b0e"} Dec 03 12:32:55 crc kubenswrapper[5052]: E1203 12:32:55.161747 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:32:55 crc kubenswrapper[5052]: E1203 12:32:55.162810 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ae9024f5-a2b4-49de-aed3-00cb9c806704-operator-scripts podName:ae9024f5-a2b4-49de-aed3-00cb9c806704 nodeName:}" failed. No retries permitted until 2025-12-03 12:33:27.162759829 +0000 UTC m=+4397.751001743 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/ae9024f5-a2b4-49de-aed3-00cb9c806704-operator-scripts") pod "placemente65a-account-delete-s7x52" (UID: "ae9024f5-a2b4-49de-aed3-00cb9c806704") : configmap "openstack-scripts" not found Dec 03 12:32:55 crc kubenswrapper[5052]: I1203 12:32:55.520178 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/community-operators-x22v4" event={"ID":"79e33cc2-8e67-4b03-a8cb-f7c257116087","Type":"ContainerStarted","Data":"25af0e2e5e24042272a7471b5ffad7790b58b631217e05bee64670256f1b73ee"} Dec 03 12:32:55 crc kubenswrapper[5052]: I1203 12:32:55.540430 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-marketplace/community-operators-x22v4" podStartSLOduration=2.875500556 podStartE2EDuration="5.540415759s" podCreationTimestamp="2025-12-03 12:32:50 +0000 UTC" firstStartedPulling="2025-12-03 12:32:52.485333119 +0000 UTC m=+4363.073574903" lastFinishedPulling="2025-12-03 12:32:55.150248342 +0000 UTC m=+4365.738490106" observedRunningTime="2025-12-03 12:32:55.536896818 +0000 UTC m=+4366.125138582" watchObservedRunningTime="2025-12-03 12:32:55.540415759 +0000 UTC m=+4366.128657513" Dec 03 12:32:56 crc kubenswrapper[5052]: E1203 12:32:56.278198 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:32:56 crc kubenswrapper[5052]: E1203 12:32:56.278541 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a2ef0804-75c7-43e3-95fa-ce901a561c9b-operator-scripts podName:a2ef0804-75c7-43e3-95fa-ce901a561c9b nodeName:}" failed. No retries permitted until 2025-12-03 12:33:28.278523415 +0000 UTC m=+4398.866765179 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/a2ef0804-75c7-43e3-95fa-ce901a561c9b-operator-scripts") pod "neutron2174-account-delete-9dnxs" (UID: "a2ef0804-75c7-43e3-95fa-ce901a561c9b") : configmap "openstack-scripts" not found Dec 03 12:32:56 crc kubenswrapper[5052]: E1203 12:32:56.379359 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:32:56 crc kubenswrapper[5052]: E1203 12:32:56.379410 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:32:56 crc kubenswrapper[5052]: E1203 12:32:56.379442 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/fc428005-b653-4a6a-a50d-133c2b4fb8b8-operator-scripts podName:fc428005-b653-4a6a-a50d-133c2b4fb8b8 nodeName:}" failed. No retries permitted until 2025-12-03 12:33:28.379421825 +0000 UTC m=+4398.967663589 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/fc428005-b653-4a6a-a50d-133c2b4fb8b8-operator-scripts") pod "cinderc151-account-delete-h4k5b" (UID: "fc428005-b653-4a6a-a50d-133c2b4fb8b8") : configmap "openstack-scripts" not found Dec 03 12:32:56 crc kubenswrapper[5052]: E1203 12:32:56.379459 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e6d87a68-5121-4088-a5d8-3f205a787b29-operator-scripts podName:e6d87a68-5121-4088-a5d8-3f205a787b29 nodeName:}" failed. No retries permitted until 2025-12-03 12:33:28.379452056 +0000 UTC m=+4398.967693900 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/e6d87a68-5121-4088-a5d8-3f205a787b29-operator-scripts") pod "novaapia918-account-delete-mk25j" (UID: "e6d87a68-5121-4088-a5d8-3f205a787b29") : configmap "openstack-scripts" not found Dec 03 12:32:56 crc kubenswrapper[5052]: E1203 12:32:56.379474 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:32:56 crc kubenswrapper[5052]: E1203 12:32:56.379570 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/52bf0c8d-03b3-444b-80b6-13d41f5f9fb9-operator-scripts podName:52bf0c8d-03b3-444b-80b6-13d41f5f9fb9 nodeName:}" failed. No retries permitted until 2025-12-03 12:33:28.379553559 +0000 UTC m=+4398.967795383 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/52bf0c8d-03b3-444b-80b6-13d41f5f9fb9-operator-scripts") pod "glance5b9b-account-delete-ggk6q" (UID: "52bf0c8d-03b3-444b-80b6-13d41f5f9fb9") : configmap "openstack-scripts" not found Dec 03 12:32:56 crc kubenswrapper[5052]: I1203 12:32:56.438911 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placemente65a-account-delete-s7x52" Dec 03 12:32:56 crc kubenswrapper[5052]: I1203 12:32:56.530871 5052 generic.go:334] "Generic (PLEG): container finished" podID="ae9024f5-a2b4-49de-aed3-00cb9c806704" containerID="96cd7d92781623b5aa19e13255fcc36f068db61739a1f3c2145635b5d11cc87a" exitCode=137 Dec 03 12:32:56 crc kubenswrapper[5052]: I1203 12:32:56.530983 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placemente65a-account-delete-s7x52" event={"ID":"ae9024f5-a2b4-49de-aed3-00cb9c806704","Type":"ContainerDied","Data":"96cd7d92781623b5aa19e13255fcc36f068db61739a1f3c2145635b5d11cc87a"} Dec 03 12:32:56 crc kubenswrapper[5052]: I1203 12:32:56.531043 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placemente65a-account-delete-s7x52" Dec 03 12:32:56 crc kubenswrapper[5052]: I1203 12:32:56.531063 5052 scope.go:117] "RemoveContainer" containerID="96cd7d92781623b5aa19e13255fcc36f068db61739a1f3c2145635b5d11cc87a" Dec 03 12:32:56 crc kubenswrapper[5052]: I1203 12:32:56.531034 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placemente65a-account-delete-s7x52" event={"ID":"ae9024f5-a2b4-49de-aed3-00cb9c806704","Type":"ContainerDied","Data":"560d2d91b6fbb85cac7eccb7681c0db9b995668b33f0e3c26747232ed5e22999"} Dec 03 12:32:56 crc kubenswrapper[5052]: I1203 12:32:56.552320 5052 scope.go:117] "RemoveContainer" containerID="96cd7d92781623b5aa19e13255fcc36f068db61739a1f3c2145635b5d11cc87a" Dec 03 12:32:56 crc kubenswrapper[5052]: E1203 12:32:56.552807 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"96cd7d92781623b5aa19e13255fcc36f068db61739a1f3c2145635b5d11cc87a\": container with ID starting with 96cd7d92781623b5aa19e13255fcc36f068db61739a1f3c2145635b5d11cc87a not found: ID does not exist" containerID="96cd7d92781623b5aa19e13255fcc36f068db61739a1f3c2145635b5d11cc87a" Dec 03 12:32:56 crc kubenswrapper[5052]: I1203 12:32:56.552850 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"96cd7d92781623b5aa19e13255fcc36f068db61739a1f3c2145635b5d11cc87a"} err="failed to get container status \"96cd7d92781623b5aa19e13255fcc36f068db61739a1f3c2145635b5d11cc87a\": rpc error: code = NotFound desc = could not find container \"96cd7d92781623b5aa19e13255fcc36f068db61739a1f3c2145635b5d11cc87a\": container with ID starting with 96cd7d92781623b5aa19e13255fcc36f068db61739a1f3c2145635b5d11cc87a not found: ID does not exist" Dec 03 12:32:56 crc kubenswrapper[5052]: I1203 12:32:56.581420 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/ae9024f5-a2b4-49de-aed3-00cb9c806704-operator-scripts\") pod \"ae9024f5-a2b4-49de-aed3-00cb9c806704\" (UID: \"ae9024f5-a2b4-49de-aed3-00cb9c806704\") " Dec 03 12:32:56 crc kubenswrapper[5052]: I1203 12:32:56.581524 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mwzzn\" (UniqueName: \"kubernetes.io/projected/ae9024f5-a2b4-49de-aed3-00cb9c806704-kube-api-access-mwzzn\") pod \"ae9024f5-a2b4-49de-aed3-00cb9c806704\" (UID: \"ae9024f5-a2b4-49de-aed3-00cb9c806704\") " Dec 03 12:32:56 crc kubenswrapper[5052]: I1203 12:32:56.582307 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ae9024f5-a2b4-49de-aed3-00cb9c806704-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "ae9024f5-a2b4-49de-aed3-00cb9c806704" (UID: "ae9024f5-a2b4-49de-aed3-00cb9c806704"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:32:56 crc kubenswrapper[5052]: I1203 12:32:56.587230 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ae9024f5-a2b4-49de-aed3-00cb9c806704-kube-api-access-mwzzn" (OuterVolumeSpecName: "kube-api-access-mwzzn") pod "ae9024f5-a2b4-49de-aed3-00cb9c806704" (UID: "ae9024f5-a2b4-49de-aed3-00cb9c806704"). InnerVolumeSpecName "kube-api-access-mwzzn". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:32:56 crc kubenswrapper[5052]: I1203 12:32:56.683626 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/ae9024f5-a2b4-49de-aed3-00cb9c806704-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:56 crc kubenswrapper[5052]: I1203 12:32:56.683673 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-mwzzn\" (UniqueName: \"kubernetes.io/projected/ae9024f5-a2b4-49de-aed3-00cb9c806704-kube-api-access-mwzzn\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:56 crc kubenswrapper[5052]: I1203 12:32:56.863741 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/placemente65a-account-delete-s7x52"] Dec 03 12:32:56 crc kubenswrapper[5052]: I1203 12:32:56.872827 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/placemente65a-account-delete-s7x52"] Dec 03 12:32:57 crc kubenswrapper[5052]: I1203 12:32:57.552394 5052 generic.go:334] "Generic (PLEG): container finished" podID="e6d87a68-5121-4088-a5d8-3f205a787b29" containerID="11243a11f3f92fefd822d688612a3f588dc1510f45d518e358b4b10cfe36371c" exitCode=137 Dec 03 12:32:57 crc kubenswrapper[5052]: I1203 12:32:57.552480 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/novaapia918-account-delete-mk25j" event={"ID":"e6d87a68-5121-4088-a5d8-3f205a787b29","Type":"ContainerDied","Data":"11243a11f3f92fefd822d688612a3f588dc1510f45d518e358b4b10cfe36371c"} Dec 03 12:32:57 crc kubenswrapper[5052]: I1203 12:32:57.555319 5052 generic.go:334] "Generic (PLEG): container finished" podID="8a93d2af-9f8c-4282-8611-526869559ae2" containerID="444eb8a103a3fd9c5c270baa3559817c3b3a7da9d570c02241c847c5911ad88f" exitCode=137 Dec 03 12:32:57 crc kubenswrapper[5052]: I1203 12:32:57.555361 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican3d67-account-delete-qrp98" event={"ID":"8a93d2af-9f8c-4282-8611-526869559ae2","Type":"ContainerDied","Data":"444eb8a103a3fd9c5c270baa3559817c3b3a7da9d570c02241c847c5911ad88f"} Dec 03 12:32:57 crc kubenswrapper[5052]: I1203 12:32:57.560019 5052 generic.go:334] "Generic (PLEG): container finished" podID="52bf0c8d-03b3-444b-80b6-13d41f5f9fb9" containerID="af107e2e82074f5c5aba6f4fcc9960e916fe7e1e038d4e5c3098122ab4c96e1b" exitCode=137 Dec 03 12:32:57 crc kubenswrapper[5052]: I1203 12:32:57.560109 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance5b9b-account-delete-ggk6q" event={"ID":"52bf0c8d-03b3-444b-80b6-13d41f5f9fb9","Type":"ContainerDied","Data":"af107e2e82074f5c5aba6f4fcc9960e916fe7e1e038d4e5c3098122ab4c96e1b"} Dec 03 12:32:57 crc kubenswrapper[5052]: I1203 12:32:57.562622 5052 generic.go:334] "Generic (PLEG): container finished" podID="a2ef0804-75c7-43e3-95fa-ce901a561c9b" containerID="1527475535a0407481384ca6a15c4f2f9da6de7e6bcda257db051f93a93b1aa9" exitCode=137 Dec 03 12:32:57 crc kubenswrapper[5052]: I1203 12:32:57.562734 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron2174-account-delete-9dnxs" event={"ID":"a2ef0804-75c7-43e3-95fa-ce901a561c9b","Type":"ContainerDied","Data":"1527475535a0407481384ca6a15c4f2f9da6de7e6bcda257db051f93a93b1aa9"} Dec 03 12:32:57 crc kubenswrapper[5052]: I1203 12:32:57.565438 5052 generic.go:334] "Generic (PLEG): container finished" podID="478e18e8-077e-406e-aa66-19bc1d28df46" containerID="7abe534b28e0d727623c30c2ee8d6feee7f3db4244b68e027625e6a8eab08cc4" exitCode=137 Dec 03 12:32:57 crc kubenswrapper[5052]: I1203 12:32:57.565567 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/novacell046b8-account-delete-nqzlj" event={"ID":"478e18e8-077e-406e-aa66-19bc1d28df46","Type":"ContainerDied","Data":"7abe534b28e0d727623c30c2ee8d6feee7f3db4244b68e027625e6a8eab08cc4"} Dec 03 12:32:57 crc kubenswrapper[5052]: I1203 12:32:57.567766 5052 generic.go:334] "Generic (PLEG): container finished" podID="fc428005-b653-4a6a-a50d-133c2b4fb8b8" containerID="f45508992c6ce3df52ea0e08b462396de8f82c3d69fcce583cf8d26d1c4c3946" exitCode=137 Dec 03 12:32:57 crc kubenswrapper[5052]: I1203 12:32:57.567810 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinderc151-account-delete-h4k5b" event={"ID":"fc428005-b653-4a6a-a50d-133c2b4fb8b8","Type":"ContainerDied","Data":"f45508992c6ce3df52ea0e08b462396de8f82c3d69fcce583cf8d26d1c4c3946"} Dec 03 12:32:57 crc kubenswrapper[5052]: I1203 12:32:57.686376 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ae9024f5-a2b4-49de-aed3-00cb9c806704" path="/var/lib/kubelet/pods/ae9024f5-a2b4-49de-aed3-00cb9c806704/volumes" Dec 03 12:32:57 crc kubenswrapper[5052]: E1203 12:32:57.901891 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:32:57 crc kubenswrapper[5052]: E1203 12:32:57.901975 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8a93d2af-9f8c-4282-8611-526869559ae2-operator-scripts podName:8a93d2af-9f8c-4282-8611-526869559ae2 nodeName:}" failed. No retries permitted until 2025-12-03 12:33:29.901957412 +0000 UTC m=+4400.490199176 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/8a93d2af-9f8c-4282-8611-526869559ae2-operator-scripts") pod "barbican3d67-account-delete-qrp98" (UID: "8a93d2af-9f8c-4282-8611-526869559ae2") : configmap "openstack-scripts" not found Dec 03 12:32:58 crc kubenswrapper[5052]: I1203 12:32:58.020415 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance5b9b-account-delete-ggk6q" Dec 03 12:32:58 crc kubenswrapper[5052]: I1203 12:32:58.105331 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/52bf0c8d-03b3-444b-80b6-13d41f5f9fb9-operator-scripts\") pod \"52bf0c8d-03b3-444b-80b6-13d41f5f9fb9\" (UID: \"52bf0c8d-03b3-444b-80b6-13d41f5f9fb9\") " Dec 03 12:32:58 crc kubenswrapper[5052]: I1203 12:32:58.105486 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rpq2v\" (UniqueName: \"kubernetes.io/projected/52bf0c8d-03b3-444b-80b6-13d41f5f9fb9-kube-api-access-rpq2v\") pod \"52bf0c8d-03b3-444b-80b6-13d41f5f9fb9\" (UID: \"52bf0c8d-03b3-444b-80b6-13d41f5f9fb9\") " Dec 03 12:32:58 crc kubenswrapper[5052]: I1203 12:32:58.107635 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/52bf0c8d-03b3-444b-80b6-13d41f5f9fb9-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "52bf0c8d-03b3-444b-80b6-13d41f5f9fb9" (UID: "52bf0c8d-03b3-444b-80b6-13d41f5f9fb9"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:32:58 crc kubenswrapper[5052]: I1203 12:32:58.153799 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/52bf0c8d-03b3-444b-80b6-13d41f5f9fb9-kube-api-access-rpq2v" (OuterVolumeSpecName: "kube-api-access-rpq2v") pod "52bf0c8d-03b3-444b-80b6-13d41f5f9fb9" (UID: "52bf0c8d-03b3-444b-80b6-13d41f5f9fb9"). InnerVolumeSpecName "kube-api-access-rpq2v". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:32:58 crc kubenswrapper[5052]: I1203 12:32:58.194670 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/novacell046b8-account-delete-nqzlj" Dec 03 12:32:58 crc kubenswrapper[5052]: I1203 12:32:58.202814 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron2174-account-delete-9dnxs" Dec 03 12:32:58 crc kubenswrapper[5052]: I1203 12:32:58.206949 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/52bf0c8d-03b3-444b-80b6-13d41f5f9fb9-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:58 crc kubenswrapper[5052]: I1203 12:32:58.206972 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-rpq2v\" (UniqueName: \"kubernetes.io/projected/52bf0c8d-03b3-444b-80b6-13d41f5f9fb9-kube-api-access-rpq2v\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:58 crc kubenswrapper[5052]: I1203 12:32:58.210313 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/novaapia918-account-delete-mk25j" Dec 03 12:32:58 crc kubenswrapper[5052]: I1203 12:32:58.233388 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinderc151-account-delete-h4k5b" Dec 03 12:32:58 crc kubenswrapper[5052]: I1203 12:32:58.237771 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican3d67-account-delete-qrp98" Dec 03 12:32:58 crc kubenswrapper[5052]: I1203 12:32:58.308066 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/e6d87a68-5121-4088-a5d8-3f205a787b29-operator-scripts\") pod \"e6d87a68-5121-4088-a5d8-3f205a787b29\" (UID: \"e6d87a68-5121-4088-a5d8-3f205a787b29\") " Dec 03 12:32:58 crc kubenswrapper[5052]: I1203 12:32:58.308125 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pqx8w\" (UniqueName: \"kubernetes.io/projected/8a93d2af-9f8c-4282-8611-526869559ae2-kube-api-access-pqx8w\") pod \"8a93d2af-9f8c-4282-8611-526869559ae2\" (UID: \"8a93d2af-9f8c-4282-8611-526869559ae2\") " Dec 03 12:32:58 crc kubenswrapper[5052]: I1203 12:32:58.308164 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nwmzn\" (UniqueName: \"kubernetes.io/projected/478e18e8-077e-406e-aa66-19bc1d28df46-kube-api-access-nwmzn\") pod \"478e18e8-077e-406e-aa66-19bc1d28df46\" (UID: \"478e18e8-077e-406e-aa66-19bc1d28df46\") " Dec 03 12:32:58 crc kubenswrapper[5052]: I1203 12:32:58.308192 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/478e18e8-077e-406e-aa66-19bc1d28df46-operator-scripts\") pod \"478e18e8-077e-406e-aa66-19bc1d28df46\" (UID: \"478e18e8-077e-406e-aa66-19bc1d28df46\") " Dec 03 12:32:58 crc kubenswrapper[5052]: I1203 12:32:58.308226 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/fc428005-b653-4a6a-a50d-133c2b4fb8b8-operator-scripts\") pod \"fc428005-b653-4a6a-a50d-133c2b4fb8b8\" (UID: \"fc428005-b653-4a6a-a50d-133c2b4fb8b8\") " Dec 03 12:32:58 crc kubenswrapper[5052]: I1203 12:32:58.308288 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/8a93d2af-9f8c-4282-8611-526869559ae2-operator-scripts\") pod \"8a93d2af-9f8c-4282-8611-526869559ae2\" (UID: \"8a93d2af-9f8c-4282-8611-526869559ae2\") " Dec 03 12:32:58 crc kubenswrapper[5052]: I1203 12:32:58.308329 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/a2ef0804-75c7-43e3-95fa-ce901a561c9b-operator-scripts\") pod \"a2ef0804-75c7-43e3-95fa-ce901a561c9b\" (UID: \"a2ef0804-75c7-43e3-95fa-ce901a561c9b\") " Dec 03 12:32:58 crc kubenswrapper[5052]: I1203 12:32:58.308349 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gcwnf\" (UniqueName: \"kubernetes.io/projected/fc428005-b653-4a6a-a50d-133c2b4fb8b8-kube-api-access-gcwnf\") pod \"fc428005-b653-4a6a-a50d-133c2b4fb8b8\" (UID: \"fc428005-b653-4a6a-a50d-133c2b4fb8b8\") " Dec 03 12:32:58 crc kubenswrapper[5052]: I1203 12:32:58.308375 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v2plr\" (UniqueName: \"kubernetes.io/projected/a2ef0804-75c7-43e3-95fa-ce901a561c9b-kube-api-access-v2plr\") pod \"a2ef0804-75c7-43e3-95fa-ce901a561c9b\" (UID: \"a2ef0804-75c7-43e3-95fa-ce901a561c9b\") " Dec 03 12:32:58 crc kubenswrapper[5052]: I1203 12:32:58.308426 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jjwdb\" (UniqueName: \"kubernetes.io/projected/e6d87a68-5121-4088-a5d8-3f205a787b29-kube-api-access-jjwdb\") pod \"e6d87a68-5121-4088-a5d8-3f205a787b29\" (UID: \"e6d87a68-5121-4088-a5d8-3f205a787b29\") " Dec 03 12:32:58 crc kubenswrapper[5052]: I1203 12:32:58.309488 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fc428005-b653-4a6a-a50d-133c2b4fb8b8-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "fc428005-b653-4a6a-a50d-133c2b4fb8b8" (UID: "fc428005-b653-4a6a-a50d-133c2b4fb8b8"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:32:58 crc kubenswrapper[5052]: I1203 12:32:58.311796 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e6d87a68-5121-4088-a5d8-3f205a787b29-kube-api-access-jjwdb" (OuterVolumeSpecName: "kube-api-access-jjwdb") pod "e6d87a68-5121-4088-a5d8-3f205a787b29" (UID: "e6d87a68-5121-4088-a5d8-3f205a787b29"). InnerVolumeSpecName "kube-api-access-jjwdb". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:32:58 crc kubenswrapper[5052]: I1203 12:32:58.312500 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8a93d2af-9f8c-4282-8611-526869559ae2-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "8a93d2af-9f8c-4282-8611-526869559ae2" (UID: "8a93d2af-9f8c-4282-8611-526869559ae2"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:32:58 crc kubenswrapper[5052]: I1203 12:32:58.312573 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/478e18e8-077e-406e-aa66-19bc1d28df46-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "478e18e8-077e-406e-aa66-19bc1d28df46" (UID: "478e18e8-077e-406e-aa66-19bc1d28df46"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:32:58 crc kubenswrapper[5052]: I1203 12:32:58.312821 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/478e18e8-077e-406e-aa66-19bc1d28df46-kube-api-access-nwmzn" (OuterVolumeSpecName: "kube-api-access-nwmzn") pod "478e18e8-077e-406e-aa66-19bc1d28df46" (UID: "478e18e8-077e-406e-aa66-19bc1d28df46"). InnerVolumeSpecName "kube-api-access-nwmzn". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:32:58 crc kubenswrapper[5052]: I1203 12:32:58.312843 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a2ef0804-75c7-43e3-95fa-ce901a561c9b-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "a2ef0804-75c7-43e3-95fa-ce901a561c9b" (UID: "a2ef0804-75c7-43e3-95fa-ce901a561c9b"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:32:58 crc kubenswrapper[5052]: I1203 12:32:58.313482 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e6d87a68-5121-4088-a5d8-3f205a787b29-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "e6d87a68-5121-4088-a5d8-3f205a787b29" (UID: "e6d87a68-5121-4088-a5d8-3f205a787b29"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:32:58 crc kubenswrapper[5052]: I1203 12:32:58.313725 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8a93d2af-9f8c-4282-8611-526869559ae2-kube-api-access-pqx8w" (OuterVolumeSpecName: "kube-api-access-pqx8w") pod "8a93d2af-9f8c-4282-8611-526869559ae2" (UID: "8a93d2af-9f8c-4282-8611-526869559ae2"). InnerVolumeSpecName "kube-api-access-pqx8w". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:32:58 crc kubenswrapper[5052]: I1203 12:32:58.314963 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fc428005-b653-4a6a-a50d-133c2b4fb8b8-kube-api-access-gcwnf" (OuterVolumeSpecName: "kube-api-access-gcwnf") pod "fc428005-b653-4a6a-a50d-133c2b4fb8b8" (UID: "fc428005-b653-4a6a-a50d-133c2b4fb8b8"). InnerVolumeSpecName "kube-api-access-gcwnf". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:32:58 crc kubenswrapper[5052]: I1203 12:32:58.315170 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a2ef0804-75c7-43e3-95fa-ce901a561c9b-kube-api-access-v2plr" (OuterVolumeSpecName: "kube-api-access-v2plr") pod "a2ef0804-75c7-43e3-95fa-ce901a561c9b" (UID: "a2ef0804-75c7-43e3-95fa-ce901a561c9b"). InnerVolumeSpecName "kube-api-access-v2plr". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:32:58 crc kubenswrapper[5052]: I1203 12:32:58.410651 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/478e18e8-077e-406e-aa66-19bc1d28df46-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:58 crc kubenswrapper[5052]: I1203 12:32:58.410686 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/fc428005-b653-4a6a-a50d-133c2b4fb8b8-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:58 crc kubenswrapper[5052]: I1203 12:32:58.410697 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/8a93d2af-9f8c-4282-8611-526869559ae2-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:58 crc kubenswrapper[5052]: I1203 12:32:58.410706 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/a2ef0804-75c7-43e3-95fa-ce901a561c9b-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:58 crc kubenswrapper[5052]: I1203 12:32:58.410717 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-gcwnf\" (UniqueName: \"kubernetes.io/projected/fc428005-b653-4a6a-a50d-133c2b4fb8b8-kube-api-access-gcwnf\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:58 crc kubenswrapper[5052]: I1203 12:32:58.410728 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-v2plr\" (UniqueName: \"kubernetes.io/projected/a2ef0804-75c7-43e3-95fa-ce901a561c9b-kube-api-access-v2plr\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:58 crc kubenswrapper[5052]: I1203 12:32:58.410738 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-jjwdb\" (UniqueName: \"kubernetes.io/projected/e6d87a68-5121-4088-a5d8-3f205a787b29-kube-api-access-jjwdb\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:58 crc kubenswrapper[5052]: I1203 12:32:58.410746 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/e6d87a68-5121-4088-a5d8-3f205a787b29-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:58 crc kubenswrapper[5052]: I1203 12:32:58.410755 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-pqx8w\" (UniqueName: \"kubernetes.io/projected/8a93d2af-9f8c-4282-8611-526869559ae2-kube-api-access-pqx8w\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:58 crc kubenswrapper[5052]: I1203 12:32:58.410763 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-nwmzn\" (UniqueName: \"kubernetes.io/projected/478e18e8-077e-406e-aa66-19bc1d28df46-kube-api-access-nwmzn\") on node \"crc\" DevicePath \"\"" Dec 03 12:32:58 crc kubenswrapper[5052]: I1203 12:32:58.578579 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/novacell046b8-account-delete-nqzlj" event={"ID":"478e18e8-077e-406e-aa66-19bc1d28df46","Type":"ContainerDied","Data":"53db9bec4c5fa77c3e23669a8d7cb3e397825fa6ec1b6c508ab5eed920339068"} Dec 03 12:32:58 crc kubenswrapper[5052]: I1203 12:32:58.578611 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/novacell046b8-account-delete-nqzlj" Dec 03 12:32:58 crc kubenswrapper[5052]: I1203 12:32:58.578731 5052 scope.go:117] "RemoveContainer" containerID="7abe534b28e0d727623c30c2ee8d6feee7f3db4244b68e027625e6a8eab08cc4" Dec 03 12:32:58 crc kubenswrapper[5052]: I1203 12:32:58.580895 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinderc151-account-delete-h4k5b" event={"ID":"fc428005-b653-4a6a-a50d-133c2b4fb8b8","Type":"ContainerDied","Data":"36dff8165cc2991be6c31945f380c62eb1c6a1cf48803fd50d74fface1777ef7"} Dec 03 12:32:58 crc kubenswrapper[5052]: I1203 12:32:58.581155 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinderc151-account-delete-h4k5b" Dec 03 12:32:58 crc kubenswrapper[5052]: I1203 12:32:58.584780 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/novaapia918-account-delete-mk25j" event={"ID":"e6d87a68-5121-4088-a5d8-3f205a787b29","Type":"ContainerDied","Data":"4375fafb1b150e12234333b1938d3140be223ca4add0a6d78137dbc69efc6d17"} Dec 03 12:32:58 crc kubenswrapper[5052]: I1203 12:32:58.585026 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/novaapia918-account-delete-mk25j" Dec 03 12:32:58 crc kubenswrapper[5052]: I1203 12:32:58.587939 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican3d67-account-delete-qrp98" event={"ID":"8a93d2af-9f8c-4282-8611-526869559ae2","Type":"ContainerDied","Data":"a4c45df8d4b52dec58435965365d638aca4c5865f260d4a90c6878d5dc66331c"} Dec 03 12:32:58 crc kubenswrapper[5052]: I1203 12:32:58.588041 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican3d67-account-delete-qrp98" Dec 03 12:32:58 crc kubenswrapper[5052]: I1203 12:32:58.596964 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance5b9b-account-delete-ggk6q" event={"ID":"52bf0c8d-03b3-444b-80b6-13d41f5f9fb9","Type":"ContainerDied","Data":"e4dc3d0b73a5da661e1038b2fbe8fcf9820b4b233fb38d76f37ea7b29b323d19"} Dec 03 12:32:58 crc kubenswrapper[5052]: I1203 12:32:58.596998 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance5b9b-account-delete-ggk6q" Dec 03 12:32:58 crc kubenswrapper[5052]: I1203 12:32:58.598500 5052 scope.go:117] "RemoveContainer" containerID="f45508992c6ce3df52ea0e08b462396de8f82c3d69fcce583cf8d26d1c4c3946" Dec 03 12:32:58 crc kubenswrapper[5052]: I1203 12:32:58.599703 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron2174-account-delete-9dnxs" event={"ID":"a2ef0804-75c7-43e3-95fa-ce901a561c9b","Type":"ContainerDied","Data":"ce2b978378221fa56c56acc841ab2e8fbc8ded006c98024e04d4c568e3f73c7a"} Dec 03 12:32:58 crc kubenswrapper[5052]: I1203 12:32:58.599788 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron2174-account-delete-9dnxs" Dec 03 12:32:58 crc kubenswrapper[5052]: I1203 12:32:58.626282 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/novacell046b8-account-delete-nqzlj"] Dec 03 12:32:58 crc kubenswrapper[5052]: I1203 12:32:58.638821 5052 scope.go:117] "RemoveContainer" containerID="11243a11f3f92fefd822d688612a3f588dc1510f45d518e358b4b10cfe36371c" Dec 03 12:32:58 crc kubenswrapper[5052]: I1203 12:32:58.644591 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/novacell046b8-account-delete-nqzlj"] Dec 03 12:32:58 crc kubenswrapper[5052]: I1203 12:32:58.661403 5052 scope.go:117] "RemoveContainer" containerID="444eb8a103a3fd9c5c270baa3559817c3b3a7da9d570c02241c847c5911ad88f" Dec 03 12:32:58 crc kubenswrapper[5052]: I1203 12:32:58.680731 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance5b9b-account-delete-ggk6q"] Dec 03 12:32:58 crc kubenswrapper[5052]: I1203 12:32:58.691103 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/glance5b9b-account-delete-ggk6q"] Dec 03 12:32:58 crc kubenswrapper[5052]: I1203 12:32:58.698409 5052 scope.go:117] "RemoveContainer" containerID="af107e2e82074f5c5aba6f4fcc9960e916fe7e1e038d4e5c3098122ab4c96e1b" Dec 03 12:32:58 crc kubenswrapper[5052]: I1203 12:32:58.701763 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/novaapia918-account-delete-mk25j"] Dec 03 12:32:58 crc kubenswrapper[5052]: I1203 12:32:58.709645 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/novaapia918-account-delete-mk25j"] Dec 03 12:32:58 crc kubenswrapper[5052]: I1203 12:32:58.714625 5052 scope.go:117] "RemoveContainer" containerID="1527475535a0407481384ca6a15c4f2f9da6de7e6bcda257db051f93a93b1aa9" Dec 03 12:32:58 crc kubenswrapper[5052]: I1203 12:32:58.718622 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/neutron2174-account-delete-9dnxs"] Dec 03 12:32:58 crc kubenswrapper[5052]: I1203 12:32:58.723093 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/neutron2174-account-delete-9dnxs"] Dec 03 12:32:58 crc kubenswrapper[5052]: I1203 12:32:58.728306 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican3d67-account-delete-qrp98"] Dec 03 12:32:58 crc kubenswrapper[5052]: I1203 12:32:58.734353 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/barbican3d67-account-delete-qrp98"] Dec 03 12:32:58 crc kubenswrapper[5052]: I1203 12:32:58.740859 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinderc151-account-delete-h4k5b"] Dec 03 12:32:58 crc kubenswrapper[5052]: I1203 12:32:58.747695 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/cinderc151-account-delete-h4k5b"] Dec 03 12:32:59 crc kubenswrapper[5052]: I1203 12:32:59.681997 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="478e18e8-077e-406e-aa66-19bc1d28df46" path="/var/lib/kubelet/pods/478e18e8-077e-406e-aa66-19bc1d28df46/volumes" Dec 03 12:32:59 crc kubenswrapper[5052]: I1203 12:32:59.682915 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="52bf0c8d-03b3-444b-80b6-13d41f5f9fb9" path="/var/lib/kubelet/pods/52bf0c8d-03b3-444b-80b6-13d41f5f9fb9/volumes" Dec 03 12:32:59 crc kubenswrapper[5052]: I1203 12:32:59.683607 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8a93d2af-9f8c-4282-8611-526869559ae2" path="/var/lib/kubelet/pods/8a93d2af-9f8c-4282-8611-526869559ae2/volumes" Dec 03 12:32:59 crc kubenswrapper[5052]: I1203 12:32:59.684141 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a2ef0804-75c7-43e3-95fa-ce901a561c9b" path="/var/lib/kubelet/pods/a2ef0804-75c7-43e3-95fa-ce901a561c9b/volumes" Dec 03 12:32:59 crc kubenswrapper[5052]: I1203 12:32:59.685228 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e6d87a68-5121-4088-a5d8-3f205a787b29" path="/var/lib/kubelet/pods/e6d87a68-5121-4088-a5d8-3f205a787b29/volumes" Dec 03 12:32:59 crc kubenswrapper[5052]: I1203 12:32:59.685766 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fc428005-b653-4a6a-a50d-133c2b4fb8b8" path="/var/lib/kubelet/pods/fc428005-b653-4a6a-a50d-133c2b4fb8b8/volumes" Dec 03 12:33:01 crc kubenswrapper[5052]: I1203 12:33:01.474912 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-marketplace/community-operators-x22v4" Dec 03 12:33:01 crc kubenswrapper[5052]: I1203 12:33:01.475259 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openshift-marketplace/community-operators-x22v4" Dec 03 12:33:01 crc kubenswrapper[5052]: I1203 12:33:01.957148 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-marketplace/community-operators-x22v4" Dec 03 12:33:02 crc kubenswrapper[5052]: I1203 12:33:02.001928 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-marketplace/community-operators-x22v4" Dec 03 12:33:03 crc kubenswrapper[5052]: I1203 12:33:03.654584 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["crc-storage/crc-storage-crc-b6699"] Dec 03 12:33:03 crc kubenswrapper[5052]: I1203 12:33:03.667345 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["crc-storage/crc-storage-crc-b6699"] Dec 03 12:33:03 crc kubenswrapper[5052]: I1203 12:33:03.682008 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="889bee72-4f6a-4c9c-817e-dfce966f6fd5" path="/var/lib/kubelet/pods/889bee72-4f6a-4c9c-817e-dfce966f6fd5/volumes" Dec 03 12:33:03 crc kubenswrapper[5052]: I1203 12:33:03.795988 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["crc-storage/crc-storage-crc-nfl8c"] Dec 03 12:33:03 crc kubenswrapper[5052]: E1203 12:33:03.796530 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="9a71429c-9d59-4863-8949-dcb4f79a35e2" containerName="swift-recon-cron" Dec 03 12:33:03 crc kubenswrapper[5052]: I1203 12:33:03.796554 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="9a71429c-9d59-4863-8949-dcb4f79a35e2" containerName="swift-recon-cron" Dec 03 12:33:03 crc kubenswrapper[5052]: E1203 12:33:03.796569 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="478e18e8-077e-406e-aa66-19bc1d28df46" containerName="mariadb-account-delete" Dec 03 12:33:03 crc kubenswrapper[5052]: I1203 12:33:03.796576 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="478e18e8-077e-406e-aa66-19bc1d28df46" containerName="mariadb-account-delete" Dec 03 12:33:03 crc kubenswrapper[5052]: E1203 12:33:03.796590 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="9a71429c-9d59-4863-8949-dcb4f79a35e2" containerName="object-auditor" Dec 03 12:33:03 crc kubenswrapper[5052]: I1203 12:33:03.796597 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="9a71429c-9d59-4863-8949-dcb4f79a35e2" containerName="object-auditor" Dec 03 12:33:03 crc kubenswrapper[5052]: E1203 12:33:03.796608 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="e6d87a68-5121-4088-a5d8-3f205a787b29" containerName="mariadb-account-delete" Dec 03 12:33:03 crc kubenswrapper[5052]: I1203 12:33:03.796614 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="e6d87a68-5121-4088-a5d8-3f205a787b29" containerName="mariadb-account-delete" Dec 03 12:33:03 crc kubenswrapper[5052]: E1203 12:33:03.796628 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="9a71429c-9d59-4863-8949-dcb4f79a35e2" containerName="container-replicator" Dec 03 12:33:03 crc kubenswrapper[5052]: I1203 12:33:03.796635 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="9a71429c-9d59-4863-8949-dcb4f79a35e2" containerName="container-replicator" Dec 03 12:33:03 crc kubenswrapper[5052]: E1203 12:33:03.796648 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="9a71429c-9d59-4863-8949-dcb4f79a35e2" containerName="account-auditor" Dec 03 12:33:03 crc kubenswrapper[5052]: I1203 12:33:03.796659 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="9a71429c-9d59-4863-8949-dcb4f79a35e2" containerName="account-auditor" Dec 03 12:33:03 crc kubenswrapper[5052]: E1203 12:33:03.796687 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="9a71429c-9d59-4863-8949-dcb4f79a35e2" containerName="account-reaper" Dec 03 12:33:03 crc kubenswrapper[5052]: I1203 12:33:03.796696 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="9a71429c-9d59-4863-8949-dcb4f79a35e2" containerName="account-reaper" Dec 03 12:33:03 crc kubenswrapper[5052]: E1203 12:33:03.796709 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="9a71429c-9d59-4863-8949-dcb4f79a35e2" containerName="object-updater" Dec 03 12:33:03 crc kubenswrapper[5052]: I1203 12:33:03.796715 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="9a71429c-9d59-4863-8949-dcb4f79a35e2" containerName="object-updater" Dec 03 12:33:03 crc kubenswrapper[5052]: E1203 12:33:03.796724 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="9a71429c-9d59-4863-8949-dcb4f79a35e2" containerName="container-server" Dec 03 12:33:03 crc kubenswrapper[5052]: I1203 12:33:03.796732 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="9a71429c-9d59-4863-8949-dcb4f79a35e2" containerName="container-server" Dec 03 12:33:03 crc kubenswrapper[5052]: E1203 12:33:03.796745 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="a2ef0804-75c7-43e3-95fa-ce901a561c9b" containerName="mariadb-account-delete" Dec 03 12:33:03 crc kubenswrapper[5052]: I1203 12:33:03.796751 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="a2ef0804-75c7-43e3-95fa-ce901a561c9b" containerName="mariadb-account-delete" Dec 03 12:33:03 crc kubenswrapper[5052]: E1203 12:33:03.796765 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="9a71429c-9d59-4863-8949-dcb4f79a35e2" containerName="rsync" Dec 03 12:33:03 crc kubenswrapper[5052]: I1203 12:33:03.796772 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="9a71429c-9d59-4863-8949-dcb4f79a35e2" containerName="rsync" Dec 03 12:33:03 crc kubenswrapper[5052]: E1203 12:33:03.796781 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="8a93d2af-9f8c-4282-8611-526869559ae2" containerName="mariadb-account-delete" Dec 03 12:33:03 crc kubenswrapper[5052]: I1203 12:33:03.796788 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="8a93d2af-9f8c-4282-8611-526869559ae2" containerName="mariadb-account-delete" Dec 03 12:33:03 crc kubenswrapper[5052]: E1203 12:33:03.796804 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="9a71429c-9d59-4863-8949-dcb4f79a35e2" containerName="container-updater" Dec 03 12:33:03 crc kubenswrapper[5052]: I1203 12:33:03.796810 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="9a71429c-9d59-4863-8949-dcb4f79a35e2" containerName="container-updater" Dec 03 12:33:03 crc kubenswrapper[5052]: E1203 12:33:03.796826 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="9a71429c-9d59-4863-8949-dcb4f79a35e2" containerName="object-expirer" Dec 03 12:33:03 crc kubenswrapper[5052]: I1203 12:33:03.796834 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="9a71429c-9d59-4863-8949-dcb4f79a35e2" containerName="object-expirer" Dec 03 12:33:03 crc kubenswrapper[5052]: E1203 12:33:03.796841 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="9a71429c-9d59-4863-8949-dcb4f79a35e2" containerName="object-replicator" Dec 03 12:33:03 crc kubenswrapper[5052]: I1203 12:33:03.796847 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="9a71429c-9d59-4863-8949-dcb4f79a35e2" containerName="object-replicator" Dec 03 12:33:03 crc kubenswrapper[5052]: E1203 12:33:03.796857 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="fc428005-b653-4a6a-a50d-133c2b4fb8b8" containerName="mariadb-account-delete" Dec 03 12:33:03 crc kubenswrapper[5052]: I1203 12:33:03.796863 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="fc428005-b653-4a6a-a50d-133c2b4fb8b8" containerName="mariadb-account-delete" Dec 03 12:33:03 crc kubenswrapper[5052]: E1203 12:33:03.796873 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="9a71429c-9d59-4863-8949-dcb4f79a35e2" containerName="object-server" Dec 03 12:33:03 crc kubenswrapper[5052]: I1203 12:33:03.796879 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="9a71429c-9d59-4863-8949-dcb4f79a35e2" containerName="object-server" Dec 03 12:33:03 crc kubenswrapper[5052]: E1203 12:33:03.796889 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="9a71429c-9d59-4863-8949-dcb4f79a35e2" containerName="account-server" Dec 03 12:33:03 crc kubenswrapper[5052]: I1203 12:33:03.796895 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="9a71429c-9d59-4863-8949-dcb4f79a35e2" containerName="account-server" Dec 03 12:33:03 crc kubenswrapper[5052]: E1203 12:33:03.796907 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="9a71429c-9d59-4863-8949-dcb4f79a35e2" containerName="account-replicator" Dec 03 12:33:03 crc kubenswrapper[5052]: I1203 12:33:03.796914 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="9a71429c-9d59-4863-8949-dcb4f79a35e2" containerName="account-replicator" Dec 03 12:33:03 crc kubenswrapper[5052]: E1203 12:33:03.796926 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="52bf0c8d-03b3-444b-80b6-13d41f5f9fb9" containerName="mariadb-account-delete" Dec 03 12:33:03 crc kubenswrapper[5052]: I1203 12:33:03.796933 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="52bf0c8d-03b3-444b-80b6-13d41f5f9fb9" containerName="mariadb-account-delete" Dec 03 12:33:03 crc kubenswrapper[5052]: E1203 12:33:03.796943 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="9a71429c-9d59-4863-8949-dcb4f79a35e2" containerName="container-auditor" Dec 03 12:33:03 crc kubenswrapper[5052]: I1203 12:33:03.796951 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="9a71429c-9d59-4863-8949-dcb4f79a35e2" containerName="container-auditor" Dec 03 12:33:03 crc kubenswrapper[5052]: E1203 12:33:03.796960 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="ae9024f5-a2b4-49de-aed3-00cb9c806704" containerName="mariadb-account-delete" Dec 03 12:33:03 crc kubenswrapper[5052]: I1203 12:33:03.796970 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae9024f5-a2b4-49de-aed3-00cb9c806704" containerName="mariadb-account-delete" Dec 03 12:33:03 crc kubenswrapper[5052]: I1203 12:33:03.797126 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="9a71429c-9d59-4863-8949-dcb4f79a35e2" containerName="account-replicator" Dec 03 12:33:03 crc kubenswrapper[5052]: I1203 12:33:03.797141 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="52bf0c8d-03b3-444b-80b6-13d41f5f9fb9" containerName="mariadb-account-delete" Dec 03 12:33:03 crc kubenswrapper[5052]: I1203 12:33:03.797152 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="9a71429c-9d59-4863-8949-dcb4f79a35e2" containerName="object-auditor" Dec 03 12:33:03 crc kubenswrapper[5052]: I1203 12:33:03.797168 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="fc428005-b653-4a6a-a50d-133c2b4fb8b8" containerName="mariadb-account-delete" Dec 03 12:33:03 crc kubenswrapper[5052]: I1203 12:33:03.797187 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="9a71429c-9d59-4863-8949-dcb4f79a35e2" containerName="rsync" Dec 03 12:33:03 crc kubenswrapper[5052]: I1203 12:33:03.797204 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="9a71429c-9d59-4863-8949-dcb4f79a35e2" containerName="object-updater" Dec 03 12:33:03 crc kubenswrapper[5052]: I1203 12:33:03.797219 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="9a71429c-9d59-4863-8949-dcb4f79a35e2" containerName="container-server" Dec 03 12:33:03 crc kubenswrapper[5052]: I1203 12:33:03.797230 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="9a71429c-9d59-4863-8949-dcb4f79a35e2" containerName="account-reaper" Dec 03 12:33:03 crc kubenswrapper[5052]: I1203 12:33:03.797242 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="9a71429c-9d59-4863-8949-dcb4f79a35e2" containerName="container-auditor" Dec 03 12:33:03 crc kubenswrapper[5052]: I1203 12:33:03.797255 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="ae9024f5-a2b4-49de-aed3-00cb9c806704" containerName="mariadb-account-delete" Dec 03 12:33:03 crc kubenswrapper[5052]: I1203 12:33:03.797265 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="a2ef0804-75c7-43e3-95fa-ce901a561c9b" containerName="mariadb-account-delete" Dec 03 12:33:03 crc kubenswrapper[5052]: I1203 12:33:03.797276 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="9a71429c-9d59-4863-8949-dcb4f79a35e2" containerName="account-server" Dec 03 12:33:03 crc kubenswrapper[5052]: I1203 12:33:03.797290 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="9a71429c-9d59-4863-8949-dcb4f79a35e2" containerName="account-auditor" Dec 03 12:33:03 crc kubenswrapper[5052]: I1203 12:33:03.797302 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="478e18e8-077e-406e-aa66-19bc1d28df46" containerName="mariadb-account-delete" Dec 03 12:33:03 crc kubenswrapper[5052]: I1203 12:33:03.797309 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="9a71429c-9d59-4863-8949-dcb4f79a35e2" containerName="object-expirer" Dec 03 12:33:03 crc kubenswrapper[5052]: I1203 12:33:03.797321 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="8a93d2af-9f8c-4282-8611-526869559ae2" containerName="mariadb-account-delete" Dec 03 12:33:03 crc kubenswrapper[5052]: I1203 12:33:03.797329 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="9a71429c-9d59-4863-8949-dcb4f79a35e2" containerName="container-updater" Dec 03 12:33:03 crc kubenswrapper[5052]: I1203 12:33:03.797339 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="9a71429c-9d59-4863-8949-dcb4f79a35e2" containerName="object-server" Dec 03 12:33:03 crc kubenswrapper[5052]: I1203 12:33:03.797349 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="9a71429c-9d59-4863-8949-dcb4f79a35e2" containerName="object-replicator" Dec 03 12:33:03 crc kubenswrapper[5052]: I1203 12:33:03.797363 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="9a71429c-9d59-4863-8949-dcb4f79a35e2" containerName="swift-recon-cron" Dec 03 12:33:03 crc kubenswrapper[5052]: I1203 12:33:03.797371 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="9a71429c-9d59-4863-8949-dcb4f79a35e2" containerName="container-replicator" Dec 03 12:33:03 crc kubenswrapper[5052]: I1203 12:33:03.797386 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="e6d87a68-5121-4088-a5d8-3f205a787b29" containerName="mariadb-account-delete" Dec 03 12:33:03 crc kubenswrapper[5052]: I1203 12:33:03.798122 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="crc-storage/crc-storage-crc-nfl8c" Dec 03 12:33:03 crc kubenswrapper[5052]: I1203 12:33:03.800096 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"crc-storage"/"crc-storage" Dec 03 12:33:03 crc kubenswrapper[5052]: I1203 12:33:03.801303 5052 reflector.go:368] Caches populated for *v1.Secret from object-"crc-storage"/"crc-storage-dockercfg-mf76g" Dec 03 12:33:03 crc kubenswrapper[5052]: I1203 12:33:03.801652 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"crc-storage"/"openshift-service-ca.crt" Dec 03 12:33:03 crc kubenswrapper[5052]: I1203 12:33:03.801815 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"crc-storage"/"kube-root-ca.crt" Dec 03 12:33:03 crc kubenswrapper[5052]: I1203 12:33:03.806698 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["crc-storage/crc-storage-crc-nfl8c"] Dec 03 12:33:03 crc kubenswrapper[5052]: I1203 12:33:03.894011 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/a0a19377-663c-4f13-a568-c3b35b12b3a8-node-mnt\") pod \"crc-storage-crc-nfl8c\" (UID: \"a0a19377-663c-4f13-a568-c3b35b12b3a8\") " pod="crc-storage/crc-storage-crc-nfl8c" Dec 03 12:33:03 crc kubenswrapper[5052]: I1203 12:33:03.894062 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7skjx\" (UniqueName: \"kubernetes.io/projected/a0a19377-663c-4f13-a568-c3b35b12b3a8-kube-api-access-7skjx\") pod \"crc-storage-crc-nfl8c\" (UID: \"a0a19377-663c-4f13-a568-c3b35b12b3a8\") " pod="crc-storage/crc-storage-crc-nfl8c" Dec 03 12:33:03 crc kubenswrapper[5052]: I1203 12:33:03.894350 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/a0a19377-663c-4f13-a568-c3b35b12b3a8-crc-storage\") pod \"crc-storage-crc-nfl8c\" (UID: \"a0a19377-663c-4f13-a568-c3b35b12b3a8\") " pod="crc-storage/crc-storage-crc-nfl8c" Dec 03 12:33:03 crc kubenswrapper[5052]: I1203 12:33:03.996199 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/a0a19377-663c-4f13-a568-c3b35b12b3a8-node-mnt\") pod \"crc-storage-crc-nfl8c\" (UID: \"a0a19377-663c-4f13-a568-c3b35b12b3a8\") " pod="crc-storage/crc-storage-crc-nfl8c" Dec 03 12:33:03 crc kubenswrapper[5052]: I1203 12:33:03.996257 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-7skjx\" (UniqueName: \"kubernetes.io/projected/a0a19377-663c-4f13-a568-c3b35b12b3a8-kube-api-access-7skjx\") pod \"crc-storage-crc-nfl8c\" (UID: \"a0a19377-663c-4f13-a568-c3b35b12b3a8\") " pod="crc-storage/crc-storage-crc-nfl8c" Dec 03 12:33:03 crc kubenswrapper[5052]: I1203 12:33:03.996330 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/a0a19377-663c-4f13-a568-c3b35b12b3a8-crc-storage\") pod \"crc-storage-crc-nfl8c\" (UID: \"a0a19377-663c-4f13-a568-c3b35b12b3a8\") " pod="crc-storage/crc-storage-crc-nfl8c" Dec 03 12:33:03 crc kubenswrapper[5052]: I1203 12:33:03.996937 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/a0a19377-663c-4f13-a568-c3b35b12b3a8-node-mnt\") pod \"crc-storage-crc-nfl8c\" (UID: \"a0a19377-663c-4f13-a568-c3b35b12b3a8\") " pod="crc-storage/crc-storage-crc-nfl8c" Dec 03 12:33:03 crc kubenswrapper[5052]: I1203 12:33:03.997652 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/a0a19377-663c-4f13-a568-c3b35b12b3a8-crc-storage\") pod \"crc-storage-crc-nfl8c\" (UID: \"a0a19377-663c-4f13-a568-c3b35b12b3a8\") " pod="crc-storage/crc-storage-crc-nfl8c" Dec 03 12:33:04 crc kubenswrapper[5052]: I1203 12:33:04.023062 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-7skjx\" (UniqueName: \"kubernetes.io/projected/a0a19377-663c-4f13-a568-c3b35b12b3a8-kube-api-access-7skjx\") pod \"crc-storage-crc-nfl8c\" (UID: \"a0a19377-663c-4f13-a568-c3b35b12b3a8\") " pod="crc-storage/crc-storage-crc-nfl8c" Dec 03 12:33:04 crc kubenswrapper[5052]: I1203 12:33:04.116726 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="crc-storage/crc-storage-crc-nfl8c" Dec 03 12:33:04 crc kubenswrapper[5052]: I1203 12:33:04.553275 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["crc-storage/crc-storage-crc-nfl8c"] Dec 03 12:33:04 crc kubenswrapper[5052]: I1203 12:33:04.667764 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="crc-storage/crc-storage-crc-nfl8c" event={"ID":"a0a19377-663c-4f13-a568-c3b35b12b3a8","Type":"ContainerStarted","Data":"db9f12632fa42ad38de340217d7550942b729170a819769224731453c9d9d9ab"} Dec 03 12:33:04 crc kubenswrapper[5052]: I1203 12:33:04.673821 5052 scope.go:117] "RemoveContainer" containerID="e6edaf188f85ea4f11f487f11cc489c700f25fb42b5ee269f4f496f1e127d772" Dec 03 12:33:04 crc kubenswrapper[5052]: E1203 12:33:04.674342 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 12:33:04 crc kubenswrapper[5052]: I1203 12:33:04.845056 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openshift-marketplace/redhat-marketplace-4bdgr"] Dec 03 12:33:04 crc kubenswrapper[5052]: I1203 12:33:04.846892 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/redhat-marketplace-4bdgr" Dec 03 12:33:04 crc kubenswrapper[5052]: I1203 12:33:04.854832 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/redhat-marketplace-4bdgr"] Dec 03 12:33:04 crc kubenswrapper[5052]: I1203 12:33:04.910142 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/f1d6c616-b537-4665-92fc-7ffdfad1a210-utilities\") pod \"redhat-marketplace-4bdgr\" (UID: \"f1d6c616-b537-4665-92fc-7ffdfad1a210\") " pod="openshift-marketplace/redhat-marketplace-4bdgr" Dec 03 12:33:04 crc kubenswrapper[5052]: I1203 12:33:04.910206 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gz4fj\" (UniqueName: \"kubernetes.io/projected/f1d6c616-b537-4665-92fc-7ffdfad1a210-kube-api-access-gz4fj\") pod \"redhat-marketplace-4bdgr\" (UID: \"f1d6c616-b537-4665-92fc-7ffdfad1a210\") " pod="openshift-marketplace/redhat-marketplace-4bdgr" Dec 03 12:33:04 crc kubenswrapper[5052]: I1203 12:33:04.910465 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/f1d6c616-b537-4665-92fc-7ffdfad1a210-catalog-content\") pod \"redhat-marketplace-4bdgr\" (UID: \"f1d6c616-b537-4665-92fc-7ffdfad1a210\") " pod="openshift-marketplace/redhat-marketplace-4bdgr" Dec 03 12:33:05 crc kubenswrapper[5052]: I1203 12:33:05.011701 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/f1d6c616-b537-4665-92fc-7ffdfad1a210-utilities\") pod \"redhat-marketplace-4bdgr\" (UID: \"f1d6c616-b537-4665-92fc-7ffdfad1a210\") " pod="openshift-marketplace/redhat-marketplace-4bdgr" Dec 03 12:33:05 crc kubenswrapper[5052]: I1203 12:33:05.011992 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-gz4fj\" (UniqueName: \"kubernetes.io/projected/f1d6c616-b537-4665-92fc-7ffdfad1a210-kube-api-access-gz4fj\") pod \"redhat-marketplace-4bdgr\" (UID: \"f1d6c616-b537-4665-92fc-7ffdfad1a210\") " pod="openshift-marketplace/redhat-marketplace-4bdgr" Dec 03 12:33:05 crc kubenswrapper[5052]: I1203 12:33:05.012059 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/f1d6c616-b537-4665-92fc-7ffdfad1a210-catalog-content\") pod \"redhat-marketplace-4bdgr\" (UID: \"f1d6c616-b537-4665-92fc-7ffdfad1a210\") " pod="openshift-marketplace/redhat-marketplace-4bdgr" Dec 03 12:33:05 crc kubenswrapper[5052]: I1203 12:33:05.012224 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/f1d6c616-b537-4665-92fc-7ffdfad1a210-utilities\") pod \"redhat-marketplace-4bdgr\" (UID: \"f1d6c616-b537-4665-92fc-7ffdfad1a210\") " pod="openshift-marketplace/redhat-marketplace-4bdgr" Dec 03 12:33:05 crc kubenswrapper[5052]: I1203 12:33:05.012364 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/f1d6c616-b537-4665-92fc-7ffdfad1a210-catalog-content\") pod \"redhat-marketplace-4bdgr\" (UID: \"f1d6c616-b537-4665-92fc-7ffdfad1a210\") " pod="openshift-marketplace/redhat-marketplace-4bdgr" Dec 03 12:33:05 crc kubenswrapper[5052]: I1203 12:33:05.036687 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-gz4fj\" (UniqueName: \"kubernetes.io/projected/f1d6c616-b537-4665-92fc-7ffdfad1a210-kube-api-access-gz4fj\") pod \"redhat-marketplace-4bdgr\" (UID: \"f1d6c616-b537-4665-92fc-7ffdfad1a210\") " pod="openshift-marketplace/redhat-marketplace-4bdgr" Dec 03 12:33:05 crc kubenswrapper[5052]: I1203 12:33:05.176587 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/redhat-marketplace-4bdgr" Dec 03 12:33:05 crc kubenswrapper[5052]: I1203 12:33:05.646566 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/redhat-marketplace-4bdgr"] Dec 03 12:33:05 crc kubenswrapper[5052]: W1203 12:33:05.654907 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1d6c616_b537_4665_92fc_7ffdfad1a210.slice/crio-189c37102e3a3ec0a2ab4959ea8b11dd1ca24b5dec52babfb2429b9c9f224297 WatchSource:0}: Error finding container 189c37102e3a3ec0a2ab4959ea8b11dd1ca24b5dec52babfb2429b9c9f224297: Status 404 returned error can't find the container with id 189c37102e3a3ec0a2ab4959ea8b11dd1ca24b5dec52babfb2429b9c9f224297 Dec 03 12:33:05 crc kubenswrapper[5052]: I1203 12:33:05.690231 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-marketplace-4bdgr" event={"ID":"f1d6c616-b537-4665-92fc-7ffdfad1a210","Type":"ContainerStarted","Data":"189c37102e3a3ec0a2ab4959ea8b11dd1ca24b5dec52babfb2429b9c9f224297"} Dec 03 12:33:07 crc kubenswrapper[5052]: I1203 12:33:07.724621 5052 generic.go:334] "Generic (PLEG): container finished" podID="a0a19377-663c-4f13-a568-c3b35b12b3a8" containerID="7a510d263ec955ad2d5ebb00bcc48080e47a93e217cc757ad503fbe9cf666300" exitCode=0 Dec 03 12:33:07 crc kubenswrapper[5052]: I1203 12:33:07.724678 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="crc-storage/crc-storage-crc-nfl8c" event={"ID":"a0a19377-663c-4f13-a568-c3b35b12b3a8","Type":"ContainerDied","Data":"7a510d263ec955ad2d5ebb00bcc48080e47a93e217cc757ad503fbe9cf666300"} Dec 03 12:33:07 crc kubenswrapper[5052]: I1203 12:33:07.727763 5052 generic.go:334] "Generic (PLEG): container finished" podID="f1d6c616-b537-4665-92fc-7ffdfad1a210" containerID="7ea9373ebd9b66c25f34c4df4a9bff72c9038e8b6188f33a5bdd30c7671e419f" exitCode=0 Dec 03 12:33:07 crc kubenswrapper[5052]: I1203 12:33:07.727814 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-marketplace-4bdgr" event={"ID":"f1d6c616-b537-4665-92fc-7ffdfad1a210","Type":"ContainerDied","Data":"7ea9373ebd9b66c25f34c4df4a9bff72c9038e8b6188f33a5bdd30c7671e419f"} Dec 03 12:33:08 crc kubenswrapper[5052]: I1203 12:33:08.744995 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-marketplace-4bdgr" event={"ID":"f1d6c616-b537-4665-92fc-7ffdfad1a210","Type":"ContainerStarted","Data":"941bcc621dfc1d9644cd04b4a3f9e091c1ae4b9d9a9f9ae0fdfd252ff6f9c868"} Dec 03 12:33:09 crc kubenswrapper[5052]: I1203 12:33:09.071229 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="crc-storage/crc-storage-crc-nfl8c" Dec 03 12:33:09 crc kubenswrapper[5052]: I1203 12:33:09.185953 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7skjx\" (UniqueName: \"kubernetes.io/projected/a0a19377-663c-4f13-a568-c3b35b12b3a8-kube-api-access-7skjx\") pod \"a0a19377-663c-4f13-a568-c3b35b12b3a8\" (UID: \"a0a19377-663c-4f13-a568-c3b35b12b3a8\") " Dec 03 12:33:09 crc kubenswrapper[5052]: I1203 12:33:09.186035 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/a0a19377-663c-4f13-a568-c3b35b12b3a8-crc-storage\") pod \"a0a19377-663c-4f13-a568-c3b35b12b3a8\" (UID: \"a0a19377-663c-4f13-a568-c3b35b12b3a8\") " Dec 03 12:33:09 crc kubenswrapper[5052]: I1203 12:33:09.186129 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/a0a19377-663c-4f13-a568-c3b35b12b3a8-node-mnt\") pod \"a0a19377-663c-4f13-a568-c3b35b12b3a8\" (UID: \"a0a19377-663c-4f13-a568-c3b35b12b3a8\") " Dec 03 12:33:09 crc kubenswrapper[5052]: I1203 12:33:09.186562 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/a0a19377-663c-4f13-a568-c3b35b12b3a8-node-mnt" (OuterVolumeSpecName: "node-mnt") pod "a0a19377-663c-4f13-a568-c3b35b12b3a8" (UID: "a0a19377-663c-4f13-a568-c3b35b12b3a8"). InnerVolumeSpecName "node-mnt". PluginName "kubernetes.io/host-path", VolumeGidValue "" Dec 03 12:33:09 crc kubenswrapper[5052]: I1203 12:33:09.192721 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a0a19377-663c-4f13-a568-c3b35b12b3a8-kube-api-access-7skjx" (OuterVolumeSpecName: "kube-api-access-7skjx") pod "a0a19377-663c-4f13-a568-c3b35b12b3a8" (UID: "a0a19377-663c-4f13-a568-c3b35b12b3a8"). InnerVolumeSpecName "kube-api-access-7skjx". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:33:09 crc kubenswrapper[5052]: I1203 12:33:09.207151 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a0a19377-663c-4f13-a568-c3b35b12b3a8-crc-storage" (OuterVolumeSpecName: "crc-storage") pod "a0a19377-663c-4f13-a568-c3b35b12b3a8" (UID: "a0a19377-663c-4f13-a568-c3b35b12b3a8"). InnerVolumeSpecName "crc-storage". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:33:09 crc kubenswrapper[5052]: I1203 12:33:09.288170 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-7skjx\" (UniqueName: \"kubernetes.io/projected/a0a19377-663c-4f13-a568-c3b35b12b3a8-kube-api-access-7skjx\") on node \"crc\" DevicePath \"\"" Dec 03 12:33:09 crc kubenswrapper[5052]: I1203 12:33:09.288214 5052 reconciler_common.go:293] "Volume detached for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/a0a19377-663c-4f13-a568-c3b35b12b3a8-crc-storage\") on node \"crc\" DevicePath \"\"" Dec 03 12:33:09 crc kubenswrapper[5052]: I1203 12:33:09.288228 5052 reconciler_common.go:293] "Volume detached for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/a0a19377-663c-4f13-a568-c3b35b12b3a8-node-mnt\") on node \"crc\" DevicePath \"\"" Dec 03 12:33:09 crc kubenswrapper[5052]: I1203 12:33:09.756829 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="crc-storage/crc-storage-crc-nfl8c" Dec 03 12:33:09 crc kubenswrapper[5052]: I1203 12:33:09.756845 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="crc-storage/crc-storage-crc-nfl8c" event={"ID":"a0a19377-663c-4f13-a568-c3b35b12b3a8","Type":"ContainerDied","Data":"db9f12632fa42ad38de340217d7550942b729170a819769224731453c9d9d9ab"} Dec 03 12:33:09 crc kubenswrapper[5052]: I1203 12:33:09.756966 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="db9f12632fa42ad38de340217d7550942b729170a819769224731453c9d9d9ab" Dec 03 12:33:09 crc kubenswrapper[5052]: I1203 12:33:09.759943 5052 generic.go:334] "Generic (PLEG): container finished" podID="f1d6c616-b537-4665-92fc-7ffdfad1a210" containerID="941bcc621dfc1d9644cd04b4a3f9e091c1ae4b9d9a9f9ae0fdfd252ff6f9c868" exitCode=0 Dec 03 12:33:09 crc kubenswrapper[5052]: I1203 12:33:09.759972 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-marketplace-4bdgr" event={"ID":"f1d6c616-b537-4665-92fc-7ffdfad1a210","Type":"ContainerDied","Data":"941bcc621dfc1d9644cd04b4a3f9e091c1ae4b9d9a9f9ae0fdfd252ff6f9c868"} Dec 03 12:33:10 crc kubenswrapper[5052]: I1203 12:33:10.233915 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openshift-marketplace/community-operators-x22v4"] Dec 03 12:33:10 crc kubenswrapper[5052]: I1203 12:33:10.234466 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openshift-marketplace/community-operators-x22v4" podUID="79e33cc2-8e67-4b03-a8cb-f7c257116087" containerName="registry-server" containerID="cri-o://25af0e2e5e24042272a7471b5ffad7790b58b631217e05bee64670256f1b73ee" gracePeriod=2 Dec 03 12:33:10 crc kubenswrapper[5052]: I1203 12:33:10.769885 5052 generic.go:334] "Generic (PLEG): container finished" podID="79e33cc2-8e67-4b03-a8cb-f7c257116087" containerID="25af0e2e5e24042272a7471b5ffad7790b58b631217e05bee64670256f1b73ee" exitCode=0 Dec 03 12:33:10 crc kubenswrapper[5052]: I1203 12:33:10.769935 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/community-operators-x22v4" event={"ID":"79e33cc2-8e67-4b03-a8cb-f7c257116087","Type":"ContainerDied","Data":"25af0e2e5e24042272a7471b5ffad7790b58b631217e05bee64670256f1b73ee"} Dec 03 12:33:11 crc kubenswrapper[5052]: I1203 12:33:11.334010 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/community-operators-x22v4" Dec 03 12:33:11 crc kubenswrapper[5052]: I1203 12:33:11.421411 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/79e33cc2-8e67-4b03-a8cb-f7c257116087-utilities\") pod \"79e33cc2-8e67-4b03-a8cb-f7c257116087\" (UID: \"79e33cc2-8e67-4b03-a8cb-f7c257116087\") " Dec 03 12:33:11 crc kubenswrapper[5052]: I1203 12:33:11.421544 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2cfmn\" (UniqueName: \"kubernetes.io/projected/79e33cc2-8e67-4b03-a8cb-f7c257116087-kube-api-access-2cfmn\") pod \"79e33cc2-8e67-4b03-a8cb-f7c257116087\" (UID: \"79e33cc2-8e67-4b03-a8cb-f7c257116087\") " Dec 03 12:33:11 crc kubenswrapper[5052]: I1203 12:33:11.421683 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/79e33cc2-8e67-4b03-a8cb-f7c257116087-catalog-content\") pod \"79e33cc2-8e67-4b03-a8cb-f7c257116087\" (UID: \"79e33cc2-8e67-4b03-a8cb-f7c257116087\") " Dec 03 12:33:11 crc kubenswrapper[5052]: I1203 12:33:11.422489 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/79e33cc2-8e67-4b03-a8cb-f7c257116087-utilities" (OuterVolumeSpecName: "utilities") pod "79e33cc2-8e67-4b03-a8cb-f7c257116087" (UID: "79e33cc2-8e67-4b03-a8cb-f7c257116087"). InnerVolumeSpecName "utilities". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:33:11 crc kubenswrapper[5052]: I1203 12:33:11.433815 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/79e33cc2-8e67-4b03-a8cb-f7c257116087-kube-api-access-2cfmn" (OuterVolumeSpecName: "kube-api-access-2cfmn") pod "79e33cc2-8e67-4b03-a8cb-f7c257116087" (UID: "79e33cc2-8e67-4b03-a8cb-f7c257116087"). InnerVolumeSpecName "kube-api-access-2cfmn". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:33:11 crc kubenswrapper[5052]: I1203 12:33:11.477710 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/79e33cc2-8e67-4b03-a8cb-f7c257116087-catalog-content" (OuterVolumeSpecName: "catalog-content") pod "79e33cc2-8e67-4b03-a8cb-f7c257116087" (UID: "79e33cc2-8e67-4b03-a8cb-f7c257116087"). InnerVolumeSpecName "catalog-content". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:33:11 crc kubenswrapper[5052]: I1203 12:33:11.523099 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-2cfmn\" (UniqueName: \"kubernetes.io/projected/79e33cc2-8e67-4b03-a8cb-f7c257116087-kube-api-access-2cfmn\") on node \"crc\" DevicePath \"\"" Dec 03 12:33:11 crc kubenswrapper[5052]: I1203 12:33:11.523150 5052 reconciler_common.go:293] "Volume detached for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/79e33cc2-8e67-4b03-a8cb-f7c257116087-catalog-content\") on node \"crc\" DevicePath \"\"" Dec 03 12:33:11 crc kubenswrapper[5052]: I1203 12:33:11.523165 5052 reconciler_common.go:293] "Volume detached for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/79e33cc2-8e67-4b03-a8cb-f7c257116087-utilities\") on node \"crc\" DevicePath \"\"" Dec 03 12:33:11 crc kubenswrapper[5052]: I1203 12:33:11.782333 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/community-operators-x22v4" event={"ID":"79e33cc2-8e67-4b03-a8cb-f7c257116087","Type":"ContainerDied","Data":"524f61b224402b926a7a03d475c254e612088f0550e11356984503e6da740453"} Dec 03 12:33:11 crc kubenswrapper[5052]: I1203 12:33:11.782728 5052 scope.go:117] "RemoveContainer" containerID="25af0e2e5e24042272a7471b5ffad7790b58b631217e05bee64670256f1b73ee" Dec 03 12:33:11 crc kubenswrapper[5052]: I1203 12:33:11.782671 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/community-operators-x22v4" Dec 03 12:33:11 crc kubenswrapper[5052]: I1203 12:33:11.787056 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-marketplace-4bdgr" event={"ID":"f1d6c616-b537-4665-92fc-7ffdfad1a210","Type":"ContainerStarted","Data":"ccfc67ee1f22fd58354c95aff664abc75b3b7dc3d87eee6e3724bc6b2bd8f696"} Dec 03 12:33:11 crc kubenswrapper[5052]: I1203 12:33:11.808196 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-marketplace/redhat-marketplace-4bdgr" podStartSLOduration=5.335732615 podStartE2EDuration="7.808177084s" podCreationTimestamp="2025-12-03 12:33:04 +0000 UTC" firstStartedPulling="2025-12-03 12:33:07.72949218 +0000 UTC m=+4378.317733944" lastFinishedPulling="2025-12-03 12:33:10.201936639 +0000 UTC m=+4380.790178413" observedRunningTime="2025-12-03 12:33:11.806785125 +0000 UTC m=+4382.395026889" watchObservedRunningTime="2025-12-03 12:33:11.808177084 +0000 UTC m=+4382.396418848" Dec 03 12:33:11 crc kubenswrapper[5052]: I1203 12:33:11.808444 5052 scope.go:117] "RemoveContainer" containerID="167f97e954de88b693249438c1a13638c936688a52268f8c206876e013e07b0e" Dec 03 12:33:11 crc kubenswrapper[5052]: I1203 12:33:11.827821 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openshift-marketplace/community-operators-x22v4"] Dec 03 12:33:11 crc kubenswrapper[5052]: I1203 12:33:11.833465 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openshift-marketplace/community-operators-x22v4"] Dec 03 12:33:12 crc kubenswrapper[5052]: I1203 12:33:12.239136 5052 scope.go:117] "RemoveContainer" containerID="597920b9b2b45564d61b5cdfeef47ee3c428fc23b90fbb938197c9a481902574" Dec 03 12:33:13 crc kubenswrapper[5052]: I1203 12:33:13.097432 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["crc-storage/crc-storage-crc-nfl8c"] Dec 03 12:33:13 crc kubenswrapper[5052]: I1203 12:33:13.104617 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["crc-storage/crc-storage-crc-nfl8c"] Dec 03 12:33:13 crc kubenswrapper[5052]: I1203 12:33:13.263080 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["crc-storage/crc-storage-crc-mz9xv"] Dec 03 12:33:13 crc kubenswrapper[5052]: E1203 12:33:13.263452 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="79e33cc2-8e67-4b03-a8cb-f7c257116087" containerName="extract-content" Dec 03 12:33:13 crc kubenswrapper[5052]: I1203 12:33:13.263463 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="79e33cc2-8e67-4b03-a8cb-f7c257116087" containerName="extract-content" Dec 03 12:33:13 crc kubenswrapper[5052]: E1203 12:33:13.263484 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="a0a19377-663c-4f13-a568-c3b35b12b3a8" containerName="storage" Dec 03 12:33:13 crc kubenswrapper[5052]: I1203 12:33:13.263491 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="a0a19377-663c-4f13-a568-c3b35b12b3a8" containerName="storage" Dec 03 12:33:13 crc kubenswrapper[5052]: E1203 12:33:13.263502 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="79e33cc2-8e67-4b03-a8cb-f7c257116087" containerName="extract-utilities" Dec 03 12:33:13 crc kubenswrapper[5052]: I1203 12:33:13.263532 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="79e33cc2-8e67-4b03-a8cb-f7c257116087" containerName="extract-utilities" Dec 03 12:33:13 crc kubenswrapper[5052]: E1203 12:33:13.263547 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="79e33cc2-8e67-4b03-a8cb-f7c257116087" containerName="registry-server" Dec 03 12:33:13 crc kubenswrapper[5052]: I1203 12:33:13.263553 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="79e33cc2-8e67-4b03-a8cb-f7c257116087" containerName="registry-server" Dec 03 12:33:13 crc kubenswrapper[5052]: I1203 12:33:13.263736 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="a0a19377-663c-4f13-a568-c3b35b12b3a8" containerName="storage" Dec 03 12:33:13 crc kubenswrapper[5052]: I1203 12:33:13.263752 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="79e33cc2-8e67-4b03-a8cb-f7c257116087" containerName="registry-server" Dec 03 12:33:13 crc kubenswrapper[5052]: I1203 12:33:13.264366 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="crc-storage/crc-storage-crc-mz9xv" Dec 03 12:33:13 crc kubenswrapper[5052]: I1203 12:33:13.267409 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"crc-storage"/"kube-root-ca.crt" Dec 03 12:33:13 crc kubenswrapper[5052]: I1203 12:33:13.267423 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"crc-storage"/"openshift-service-ca.crt" Dec 03 12:33:13 crc kubenswrapper[5052]: I1203 12:33:13.267647 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"crc-storage"/"crc-storage" Dec 03 12:33:13 crc kubenswrapper[5052]: I1203 12:33:13.267662 5052 reflector.go:368] Caches populated for *v1.Secret from object-"crc-storage"/"crc-storage-dockercfg-mf76g" Dec 03 12:33:13 crc kubenswrapper[5052]: I1203 12:33:13.274221 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["crc-storage/crc-storage-crc-mz9xv"] Dec 03 12:33:13 crc kubenswrapper[5052]: I1203 12:33:13.365838 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7vb2b\" (UniqueName: \"kubernetes.io/projected/b6dc9243-9b95-4639-b4b9-a0adce5f25b8-kube-api-access-7vb2b\") pod \"crc-storage-crc-mz9xv\" (UID: \"b6dc9243-9b95-4639-b4b9-a0adce5f25b8\") " pod="crc-storage/crc-storage-crc-mz9xv" Dec 03 12:33:13 crc kubenswrapper[5052]: I1203 12:33:13.366566 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/b6dc9243-9b95-4639-b4b9-a0adce5f25b8-crc-storage\") pod \"crc-storage-crc-mz9xv\" (UID: \"b6dc9243-9b95-4639-b4b9-a0adce5f25b8\") " pod="crc-storage/crc-storage-crc-mz9xv" Dec 03 12:33:13 crc kubenswrapper[5052]: I1203 12:33:13.366616 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/b6dc9243-9b95-4639-b4b9-a0adce5f25b8-node-mnt\") pod \"crc-storage-crc-mz9xv\" (UID: \"b6dc9243-9b95-4639-b4b9-a0adce5f25b8\") " pod="crc-storage/crc-storage-crc-mz9xv" Dec 03 12:33:13 crc kubenswrapper[5052]: I1203 12:33:13.468209 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-7vb2b\" (UniqueName: \"kubernetes.io/projected/b6dc9243-9b95-4639-b4b9-a0adce5f25b8-kube-api-access-7vb2b\") pod \"crc-storage-crc-mz9xv\" (UID: \"b6dc9243-9b95-4639-b4b9-a0adce5f25b8\") " pod="crc-storage/crc-storage-crc-mz9xv" Dec 03 12:33:13 crc kubenswrapper[5052]: I1203 12:33:13.468369 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/b6dc9243-9b95-4639-b4b9-a0adce5f25b8-crc-storage\") pod \"crc-storage-crc-mz9xv\" (UID: \"b6dc9243-9b95-4639-b4b9-a0adce5f25b8\") " pod="crc-storage/crc-storage-crc-mz9xv" Dec 03 12:33:13 crc kubenswrapper[5052]: I1203 12:33:13.468412 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/b6dc9243-9b95-4639-b4b9-a0adce5f25b8-node-mnt\") pod \"crc-storage-crc-mz9xv\" (UID: \"b6dc9243-9b95-4639-b4b9-a0adce5f25b8\") " pod="crc-storage/crc-storage-crc-mz9xv" Dec 03 12:33:13 crc kubenswrapper[5052]: I1203 12:33:13.468755 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/b6dc9243-9b95-4639-b4b9-a0adce5f25b8-node-mnt\") pod \"crc-storage-crc-mz9xv\" (UID: \"b6dc9243-9b95-4639-b4b9-a0adce5f25b8\") " pod="crc-storage/crc-storage-crc-mz9xv" Dec 03 12:33:13 crc kubenswrapper[5052]: I1203 12:33:13.469907 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/b6dc9243-9b95-4639-b4b9-a0adce5f25b8-crc-storage\") pod \"crc-storage-crc-mz9xv\" (UID: \"b6dc9243-9b95-4639-b4b9-a0adce5f25b8\") " pod="crc-storage/crc-storage-crc-mz9xv" Dec 03 12:33:13 crc kubenswrapper[5052]: I1203 12:33:13.515316 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-7vb2b\" (UniqueName: \"kubernetes.io/projected/b6dc9243-9b95-4639-b4b9-a0adce5f25b8-kube-api-access-7vb2b\") pod \"crc-storage-crc-mz9xv\" (UID: \"b6dc9243-9b95-4639-b4b9-a0adce5f25b8\") " pod="crc-storage/crc-storage-crc-mz9xv" Dec 03 12:33:13 crc kubenswrapper[5052]: I1203 12:33:13.620857 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="crc-storage/crc-storage-crc-mz9xv" Dec 03 12:33:13 crc kubenswrapper[5052]: I1203 12:33:13.684254 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="79e33cc2-8e67-4b03-a8cb-f7c257116087" path="/var/lib/kubelet/pods/79e33cc2-8e67-4b03-a8cb-f7c257116087/volumes" Dec 03 12:33:13 crc kubenswrapper[5052]: I1203 12:33:13.684910 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a0a19377-663c-4f13-a568-c3b35b12b3a8" path="/var/lib/kubelet/pods/a0a19377-663c-4f13-a568-c3b35b12b3a8/volumes" Dec 03 12:33:14 crc kubenswrapper[5052]: I1203 12:33:14.049897 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["crc-storage/crc-storage-crc-mz9xv"] Dec 03 12:33:14 crc kubenswrapper[5052]: I1203 12:33:14.822403 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="crc-storage/crc-storage-crc-mz9xv" event={"ID":"b6dc9243-9b95-4639-b4b9-a0adce5f25b8","Type":"ContainerStarted","Data":"9416323ce87521c95dccf3ffd0aad8e361473d33580677d82a91d034a9105a26"} Dec 03 12:33:15 crc kubenswrapper[5052]: I1203 12:33:15.177456 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-marketplace/redhat-marketplace-4bdgr" Dec 03 12:33:15 crc kubenswrapper[5052]: I1203 12:33:15.177748 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openshift-marketplace/redhat-marketplace-4bdgr" Dec 03 12:33:15 crc kubenswrapper[5052]: I1203 12:33:15.225990 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-marketplace/redhat-marketplace-4bdgr" Dec 03 12:33:15 crc kubenswrapper[5052]: I1203 12:33:15.832474 5052 generic.go:334] "Generic (PLEG): container finished" podID="b6dc9243-9b95-4639-b4b9-a0adce5f25b8" containerID="80c94eaeae20bd3937789c76d0b2f5e78f8cdbd6d6a0f6314ae1b3855ffcc22b" exitCode=0 Dec 03 12:33:15 crc kubenswrapper[5052]: I1203 12:33:15.832553 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="crc-storage/crc-storage-crc-mz9xv" event={"ID":"b6dc9243-9b95-4639-b4b9-a0adce5f25b8","Type":"ContainerDied","Data":"80c94eaeae20bd3937789c76d0b2f5e78f8cdbd6d6a0f6314ae1b3855ffcc22b"} Dec 03 12:33:15 crc kubenswrapper[5052]: I1203 12:33:15.872795 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-marketplace/redhat-marketplace-4bdgr" Dec 03 12:33:16 crc kubenswrapper[5052]: I1203 12:33:16.233158 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openshift-marketplace/redhat-marketplace-4bdgr"] Dec 03 12:33:17 crc kubenswrapper[5052]: I1203 12:33:17.156061 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="crc-storage/crc-storage-crc-mz9xv" Dec 03 12:33:17 crc kubenswrapper[5052]: I1203 12:33:17.225583 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7vb2b\" (UniqueName: \"kubernetes.io/projected/b6dc9243-9b95-4639-b4b9-a0adce5f25b8-kube-api-access-7vb2b\") pod \"b6dc9243-9b95-4639-b4b9-a0adce5f25b8\" (UID: \"b6dc9243-9b95-4639-b4b9-a0adce5f25b8\") " Dec 03 12:33:17 crc kubenswrapper[5052]: I1203 12:33:17.225697 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/b6dc9243-9b95-4639-b4b9-a0adce5f25b8-node-mnt\") pod \"b6dc9243-9b95-4639-b4b9-a0adce5f25b8\" (UID: \"b6dc9243-9b95-4639-b4b9-a0adce5f25b8\") " Dec 03 12:33:17 crc kubenswrapper[5052]: I1203 12:33:17.225783 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/b6dc9243-9b95-4639-b4b9-a0adce5f25b8-crc-storage\") pod \"b6dc9243-9b95-4639-b4b9-a0adce5f25b8\" (UID: \"b6dc9243-9b95-4639-b4b9-a0adce5f25b8\") " Dec 03 12:33:17 crc kubenswrapper[5052]: I1203 12:33:17.226338 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/b6dc9243-9b95-4639-b4b9-a0adce5f25b8-node-mnt" (OuterVolumeSpecName: "node-mnt") pod "b6dc9243-9b95-4639-b4b9-a0adce5f25b8" (UID: "b6dc9243-9b95-4639-b4b9-a0adce5f25b8"). InnerVolumeSpecName "node-mnt". PluginName "kubernetes.io/host-path", VolumeGidValue "" Dec 03 12:33:17 crc kubenswrapper[5052]: I1203 12:33:17.232134 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b6dc9243-9b95-4639-b4b9-a0adce5f25b8-kube-api-access-7vb2b" (OuterVolumeSpecName: "kube-api-access-7vb2b") pod "b6dc9243-9b95-4639-b4b9-a0adce5f25b8" (UID: "b6dc9243-9b95-4639-b4b9-a0adce5f25b8"). InnerVolumeSpecName "kube-api-access-7vb2b". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:33:17 crc kubenswrapper[5052]: I1203 12:33:17.245586 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b6dc9243-9b95-4639-b4b9-a0adce5f25b8-crc-storage" (OuterVolumeSpecName: "crc-storage") pod "b6dc9243-9b95-4639-b4b9-a0adce5f25b8" (UID: "b6dc9243-9b95-4639-b4b9-a0adce5f25b8"). InnerVolumeSpecName "crc-storage". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:33:17 crc kubenswrapper[5052]: I1203 12:33:17.327871 5052 reconciler_common.go:293] "Volume detached for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/b6dc9243-9b95-4639-b4b9-a0adce5f25b8-node-mnt\") on node \"crc\" DevicePath \"\"" Dec 03 12:33:17 crc kubenswrapper[5052]: I1203 12:33:17.327911 5052 reconciler_common.go:293] "Volume detached for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/b6dc9243-9b95-4639-b4b9-a0adce5f25b8-crc-storage\") on node \"crc\" DevicePath \"\"" Dec 03 12:33:17 crc kubenswrapper[5052]: I1203 12:33:17.327924 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-7vb2b\" (UniqueName: \"kubernetes.io/projected/b6dc9243-9b95-4639-b4b9-a0adce5f25b8-kube-api-access-7vb2b\") on node \"crc\" DevicePath \"\"" Dec 03 12:33:17 crc kubenswrapper[5052]: I1203 12:33:17.672663 5052 scope.go:117] "RemoveContainer" containerID="e6edaf188f85ea4f11f487f11cc489c700f25fb42b5ee269f4f496f1e127d772" Dec 03 12:33:17 crc kubenswrapper[5052]: E1203 12:33:17.672990 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 12:33:17 crc kubenswrapper[5052]: I1203 12:33:17.856685 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openshift-marketplace/redhat-marketplace-4bdgr" podUID="f1d6c616-b537-4665-92fc-7ffdfad1a210" containerName="registry-server" containerID="cri-o://ccfc67ee1f22fd58354c95aff664abc75b3b7dc3d87eee6e3724bc6b2bd8f696" gracePeriod=2 Dec 03 12:33:17 crc kubenswrapper[5052]: I1203 12:33:17.856796 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="crc-storage/crc-storage-crc-mz9xv" Dec 03 12:33:17 crc kubenswrapper[5052]: I1203 12:33:17.857440 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="crc-storage/crc-storage-crc-mz9xv" event={"ID":"b6dc9243-9b95-4639-b4b9-a0adce5f25b8","Type":"ContainerDied","Data":"9416323ce87521c95dccf3ffd0aad8e361473d33580677d82a91d034a9105a26"} Dec 03 12:33:17 crc kubenswrapper[5052]: I1203 12:33:17.857466 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9416323ce87521c95dccf3ffd0aad8e361473d33580677d82a91d034a9105a26" Dec 03 12:33:18 crc kubenswrapper[5052]: I1203 12:33:18.367481 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/redhat-marketplace-4bdgr" Dec 03 12:33:18 crc kubenswrapper[5052]: I1203 12:33:18.446445 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/f1d6c616-b537-4665-92fc-7ffdfad1a210-utilities\") pod \"f1d6c616-b537-4665-92fc-7ffdfad1a210\" (UID: \"f1d6c616-b537-4665-92fc-7ffdfad1a210\") " Dec 03 12:33:18 crc kubenswrapper[5052]: I1203 12:33:18.446558 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gz4fj\" (UniqueName: \"kubernetes.io/projected/f1d6c616-b537-4665-92fc-7ffdfad1a210-kube-api-access-gz4fj\") pod \"f1d6c616-b537-4665-92fc-7ffdfad1a210\" (UID: \"f1d6c616-b537-4665-92fc-7ffdfad1a210\") " Dec 03 12:33:18 crc kubenswrapper[5052]: I1203 12:33:18.446609 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/f1d6c616-b537-4665-92fc-7ffdfad1a210-catalog-content\") pod \"f1d6c616-b537-4665-92fc-7ffdfad1a210\" (UID: \"f1d6c616-b537-4665-92fc-7ffdfad1a210\") " Dec 03 12:33:18 crc kubenswrapper[5052]: I1203 12:33:18.447737 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f1d6c616-b537-4665-92fc-7ffdfad1a210-utilities" (OuterVolumeSpecName: "utilities") pod "f1d6c616-b537-4665-92fc-7ffdfad1a210" (UID: "f1d6c616-b537-4665-92fc-7ffdfad1a210"). InnerVolumeSpecName "utilities". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:33:18 crc kubenswrapper[5052]: I1203 12:33:18.460977 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f1d6c616-b537-4665-92fc-7ffdfad1a210-kube-api-access-gz4fj" (OuterVolumeSpecName: "kube-api-access-gz4fj") pod "f1d6c616-b537-4665-92fc-7ffdfad1a210" (UID: "f1d6c616-b537-4665-92fc-7ffdfad1a210"). InnerVolumeSpecName "kube-api-access-gz4fj". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:33:18 crc kubenswrapper[5052]: I1203 12:33:18.469689 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f1d6c616-b537-4665-92fc-7ffdfad1a210-catalog-content" (OuterVolumeSpecName: "catalog-content") pod "f1d6c616-b537-4665-92fc-7ffdfad1a210" (UID: "f1d6c616-b537-4665-92fc-7ffdfad1a210"). InnerVolumeSpecName "catalog-content". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:33:18 crc kubenswrapper[5052]: I1203 12:33:18.548604 5052 reconciler_common.go:293] "Volume detached for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/f1d6c616-b537-4665-92fc-7ffdfad1a210-utilities\") on node \"crc\" DevicePath \"\"" Dec 03 12:33:18 crc kubenswrapper[5052]: I1203 12:33:18.548646 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-gz4fj\" (UniqueName: \"kubernetes.io/projected/f1d6c616-b537-4665-92fc-7ffdfad1a210-kube-api-access-gz4fj\") on node \"crc\" DevicePath \"\"" Dec 03 12:33:18 crc kubenswrapper[5052]: I1203 12:33:18.548663 5052 reconciler_common.go:293] "Volume detached for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/f1d6c616-b537-4665-92fc-7ffdfad1a210-catalog-content\") on node \"crc\" DevicePath \"\"" Dec 03 12:33:18 crc kubenswrapper[5052]: I1203 12:33:18.804231 5052 scope.go:117] "RemoveContainer" containerID="76ce33028e7542af6d393c3f4a62e893bf649320e40a27297e557281f0c774d0" Dec 03 12:33:18 crc kubenswrapper[5052]: I1203 12:33:18.827717 5052 scope.go:117] "RemoveContainer" containerID="5acdaf1de9637eb9a2dcd430e89e4aa32e1830b27b6a59fec9de95f2f8f0c88f" Dec 03 12:33:18 crc kubenswrapper[5052]: I1203 12:33:18.869898 5052 scope.go:117] "RemoveContainer" containerID="94b056c86cf3f39157a4c778b82ad7f6fc7c02b88c1e228893f8382b18fe8241" Dec 03 12:33:18 crc kubenswrapper[5052]: I1203 12:33:18.882829 5052 generic.go:334] "Generic (PLEG): container finished" podID="f1d6c616-b537-4665-92fc-7ffdfad1a210" containerID="ccfc67ee1f22fd58354c95aff664abc75b3b7dc3d87eee6e3724bc6b2bd8f696" exitCode=0 Dec 03 12:33:18 crc kubenswrapper[5052]: I1203 12:33:18.882863 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-marketplace-4bdgr" event={"ID":"f1d6c616-b537-4665-92fc-7ffdfad1a210","Type":"ContainerDied","Data":"ccfc67ee1f22fd58354c95aff664abc75b3b7dc3d87eee6e3724bc6b2bd8f696"} Dec 03 12:33:18 crc kubenswrapper[5052]: I1203 12:33:18.882885 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-marketplace-4bdgr" event={"ID":"f1d6c616-b537-4665-92fc-7ffdfad1a210","Type":"ContainerDied","Data":"189c37102e3a3ec0a2ab4959ea8b11dd1ca24b5dec52babfb2429b9c9f224297"} Dec 03 12:33:18 crc kubenswrapper[5052]: I1203 12:33:18.882901 5052 scope.go:117] "RemoveContainer" containerID="ccfc67ee1f22fd58354c95aff664abc75b3b7dc3d87eee6e3724bc6b2bd8f696" Dec 03 12:33:18 crc kubenswrapper[5052]: I1203 12:33:18.883019 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/redhat-marketplace-4bdgr" Dec 03 12:33:18 crc kubenswrapper[5052]: I1203 12:33:18.895408 5052 scope.go:117] "RemoveContainer" containerID="ef3ec7b7f63c95d670ec6dce26e58fcf598dfd84857be75804ebf18715f6be7d" Dec 03 12:33:18 crc kubenswrapper[5052]: I1203 12:33:18.906086 5052 scope.go:117] "RemoveContainer" containerID="941bcc621dfc1d9644cd04b4a3f9e091c1ae4b9d9a9f9ae0fdfd252ff6f9c868" Dec 03 12:33:18 crc kubenswrapper[5052]: I1203 12:33:18.920993 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openshift-marketplace/redhat-marketplace-4bdgr"] Dec 03 12:33:18 crc kubenswrapper[5052]: I1203 12:33:18.926853 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openshift-marketplace/redhat-marketplace-4bdgr"] Dec 03 12:33:18 crc kubenswrapper[5052]: I1203 12:33:18.945366 5052 scope.go:117] "RemoveContainer" containerID="ca583556bd7883d15cb27eea37734da610b1438d1bc770d94b10a6407a260036" Dec 03 12:33:18 crc kubenswrapper[5052]: I1203 12:33:18.959222 5052 scope.go:117] "RemoveContainer" containerID="7ea9373ebd9b66c25f34c4df4a9bff72c9038e8b6188f33a5bdd30c7671e419f" Dec 03 12:33:18 crc kubenswrapper[5052]: I1203 12:33:18.972286 5052 scope.go:117] "RemoveContainer" containerID="46ad9f6912447d4884f1470a5dcbdc70613a23efc296c65a89788b0461ace5fb" Dec 03 12:33:18 crc kubenswrapper[5052]: I1203 12:33:18.984260 5052 scope.go:117] "RemoveContainer" containerID="ccfc67ee1f22fd58354c95aff664abc75b3b7dc3d87eee6e3724bc6b2bd8f696" Dec 03 12:33:18 crc kubenswrapper[5052]: E1203 12:33:18.984931 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ccfc67ee1f22fd58354c95aff664abc75b3b7dc3d87eee6e3724bc6b2bd8f696\": container with ID starting with ccfc67ee1f22fd58354c95aff664abc75b3b7dc3d87eee6e3724bc6b2bd8f696 not found: ID does not exist" containerID="ccfc67ee1f22fd58354c95aff664abc75b3b7dc3d87eee6e3724bc6b2bd8f696" Dec 03 12:33:18 crc kubenswrapper[5052]: I1203 12:33:18.984979 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ccfc67ee1f22fd58354c95aff664abc75b3b7dc3d87eee6e3724bc6b2bd8f696"} err="failed to get container status \"ccfc67ee1f22fd58354c95aff664abc75b3b7dc3d87eee6e3724bc6b2bd8f696\": rpc error: code = NotFound desc = could not find container \"ccfc67ee1f22fd58354c95aff664abc75b3b7dc3d87eee6e3724bc6b2bd8f696\": container with ID starting with ccfc67ee1f22fd58354c95aff664abc75b3b7dc3d87eee6e3724bc6b2bd8f696 not found: ID does not exist" Dec 03 12:33:18 crc kubenswrapper[5052]: I1203 12:33:18.985012 5052 scope.go:117] "RemoveContainer" containerID="941bcc621dfc1d9644cd04b4a3f9e091c1ae4b9d9a9f9ae0fdfd252ff6f9c868" Dec 03 12:33:18 crc kubenswrapper[5052]: E1203 12:33:18.985398 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"941bcc621dfc1d9644cd04b4a3f9e091c1ae4b9d9a9f9ae0fdfd252ff6f9c868\": container with ID starting with 941bcc621dfc1d9644cd04b4a3f9e091c1ae4b9d9a9f9ae0fdfd252ff6f9c868 not found: ID does not exist" containerID="941bcc621dfc1d9644cd04b4a3f9e091c1ae4b9d9a9f9ae0fdfd252ff6f9c868" Dec 03 12:33:18 crc kubenswrapper[5052]: I1203 12:33:18.985437 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"941bcc621dfc1d9644cd04b4a3f9e091c1ae4b9d9a9f9ae0fdfd252ff6f9c868"} err="failed to get container status \"941bcc621dfc1d9644cd04b4a3f9e091c1ae4b9d9a9f9ae0fdfd252ff6f9c868\": rpc error: code = NotFound desc = could not find container \"941bcc621dfc1d9644cd04b4a3f9e091c1ae4b9d9a9f9ae0fdfd252ff6f9c868\": container with ID starting with 941bcc621dfc1d9644cd04b4a3f9e091c1ae4b9d9a9f9ae0fdfd252ff6f9c868 not found: ID does not exist" Dec 03 12:33:18 crc kubenswrapper[5052]: I1203 12:33:18.985463 5052 scope.go:117] "RemoveContainer" containerID="7ea9373ebd9b66c25f34c4df4a9bff72c9038e8b6188f33a5bdd30c7671e419f" Dec 03 12:33:18 crc kubenswrapper[5052]: E1203 12:33:18.985799 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7ea9373ebd9b66c25f34c4df4a9bff72c9038e8b6188f33a5bdd30c7671e419f\": container with ID starting with 7ea9373ebd9b66c25f34c4df4a9bff72c9038e8b6188f33a5bdd30c7671e419f not found: ID does not exist" containerID="7ea9373ebd9b66c25f34c4df4a9bff72c9038e8b6188f33a5bdd30c7671e419f" Dec 03 12:33:18 crc kubenswrapper[5052]: I1203 12:33:18.985832 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7ea9373ebd9b66c25f34c4df4a9bff72c9038e8b6188f33a5bdd30c7671e419f"} err="failed to get container status \"7ea9373ebd9b66c25f34c4df4a9bff72c9038e8b6188f33a5bdd30c7671e419f\": rpc error: code = NotFound desc = could not find container \"7ea9373ebd9b66c25f34c4df4a9bff72c9038e8b6188f33a5bdd30c7671e419f\": container with ID starting with 7ea9373ebd9b66c25f34c4df4a9bff72c9038e8b6188f33a5bdd30c7671e419f not found: ID does not exist" Dec 03 12:33:18 crc kubenswrapper[5052]: I1203 12:33:18.996164 5052 scope.go:117] "RemoveContainer" containerID="3cc4322f202b3e31f0e15cf554b9c3df35c358e0a3edc6031c1321e38a0851d3" Dec 03 12:33:19 crc kubenswrapper[5052]: I1203 12:33:19.016439 5052 scope.go:117] "RemoveContainer" containerID="17e6a6fdbd72ea6cd16402a2b231dc4e88079b9588859838d5611cfd4bb232c5" Dec 03 12:33:19 crc kubenswrapper[5052]: I1203 12:33:19.036171 5052 scope.go:117] "RemoveContainer" containerID="a46cb3d56ad689ac1d90543412ee1d3884621a7da5a8f143e20e1ff107ef6902" Dec 03 12:33:19 crc kubenswrapper[5052]: I1203 12:33:19.071536 5052 scope.go:117] "RemoveContainer" containerID="92379b4ee05501138180215d0ddb2588eed029321c921b317ba693c25f8551e2" Dec 03 12:33:19 crc kubenswrapper[5052]: I1203 12:33:19.097292 5052 scope.go:117] "RemoveContainer" containerID="16f725234e733e8166ae824e324c0f1ea6f2d967a6fb37526435b3960d3691cc" Dec 03 12:33:19 crc kubenswrapper[5052]: I1203 12:33:19.120683 5052 scope.go:117] "RemoveContainer" containerID="2cf3827a312792b3066340c95e19c6fcb87688eafedbc9bb41c1a04fa7aebf0c" Dec 03 12:33:19 crc kubenswrapper[5052]: I1203 12:33:19.157234 5052 scope.go:117] "RemoveContainer" containerID="8a99003bc5144e66c0c428e9b1815efa135128f34c7b263b18770d41d3b547ec" Dec 03 12:33:19 crc kubenswrapper[5052]: I1203 12:33:19.177283 5052 scope.go:117] "RemoveContainer" containerID="2586d14ec5172cd8887cf742de44efde3b182ee97f4136dec4ca8ec79d2e0521" Dec 03 12:33:19 crc kubenswrapper[5052]: I1203 12:33:19.198253 5052 scope.go:117] "RemoveContainer" containerID="63734bc615818a6cadecc9f3d7235c2f0339297c97215ad371cef23fc29afd3c" Dec 03 12:33:19 crc kubenswrapper[5052]: I1203 12:33:19.217732 5052 scope.go:117] "RemoveContainer" containerID="4da5e593296a628772a4119fc20ad8f06ce5e05f848f19bd5f817b1a9b9fb997" Dec 03 12:33:19 crc kubenswrapper[5052]: I1203 12:33:19.262213 5052 scope.go:117] "RemoveContainer" containerID="0eb00ee36fea8f5a8d36eaf31a8098b39f40b6cf42acfc1c7a923e4f7ae3d122" Dec 03 12:33:19 crc kubenswrapper[5052]: I1203 12:33:19.309316 5052 scope.go:117] "RemoveContainer" containerID="1866d9e97345f3e78cc8443ebbf0430328e7ab4a4ed32a162d7d7f17a4786a03" Dec 03 12:33:19 crc kubenswrapper[5052]: I1203 12:33:19.362680 5052 scope.go:117] "RemoveContainer" containerID="0fb21147d3c613c2babe2b6f711cd697be71aba6d40349fe2ecac629ffec60fa" Dec 03 12:33:19 crc kubenswrapper[5052]: I1203 12:33:19.385135 5052 scope.go:117] "RemoveContainer" containerID="d4a5529f77a7bdcc5d5d76b2cb904eb4403a956000dcfbbc8e4cb0a549f64b5f" Dec 03 12:33:19 crc kubenswrapper[5052]: I1203 12:33:19.429825 5052 scope.go:117] "RemoveContainer" containerID="387c7b8ee696cb426191ea26fa0317bb28fa52a78581eb4197de94eee8d0a4a6" Dec 03 12:33:19 crc kubenswrapper[5052]: I1203 12:33:19.454615 5052 scope.go:117] "RemoveContainer" containerID="2e86dc76cd3c6e8f0ef1db1ef4ca376f07e13b1e654e9d9b8f0bae1e0f3a5a4f" Dec 03 12:33:19 crc kubenswrapper[5052]: I1203 12:33:19.482138 5052 scope.go:117] "RemoveContainer" containerID="3418fbee6952b48c95c35c9758d124d82b2c04833e3b31186c3190edd275b7cd" Dec 03 12:33:19 crc kubenswrapper[5052]: I1203 12:33:19.500974 5052 scope.go:117] "RemoveContainer" containerID="6dd77861dd24def3a954ac96db60a8061925af3b1fbe5c7d29936dd201e6dea4" Dec 03 12:33:19 crc kubenswrapper[5052]: I1203 12:33:19.520695 5052 scope.go:117] "RemoveContainer" containerID="059f15379c22e9cbd4b778847ac6fc1ba70f4d80d669d8256de7257a96a80dbb" Dec 03 12:33:19 crc kubenswrapper[5052]: I1203 12:33:19.537986 5052 scope.go:117] "RemoveContainer" containerID="fdfab1b488effd7f6ef251f831aebda5da17360079975291ad8ed38f57cc9bd2" Dec 03 12:33:19 crc kubenswrapper[5052]: I1203 12:33:19.559046 5052 scope.go:117] "RemoveContainer" containerID="7e368d1baaf0ba061b0efb437531205a49da54db72f9bde12964c76141ba7147" Dec 03 12:33:19 crc kubenswrapper[5052]: I1203 12:33:19.582855 5052 scope.go:117] "RemoveContainer" containerID="dae22d6ddc7e6009c3d2ef035b575fe16294b7d9baf876c20c60db3ce6c60b06" Dec 03 12:33:19 crc kubenswrapper[5052]: I1203 12:33:19.605083 5052 scope.go:117] "RemoveContainer" containerID="22275b3672b8b4add52b127bbe276c42c2a5e8d7f524d0cf5e1961f6aee9186c" Dec 03 12:33:19 crc kubenswrapper[5052]: I1203 12:33:19.630874 5052 scope.go:117] "RemoveContainer" containerID="281db729b55e9d4d2851c2134872d5a117b0767da33f8a3837143567bbe3f595" Dec 03 12:33:19 crc kubenswrapper[5052]: I1203 12:33:19.649023 5052 scope.go:117] "RemoveContainer" containerID="245c1a42883f9d4981abbc2944dd195440c9014036cc110f8c69aeadb6a35ea1" Dec 03 12:33:19 crc kubenswrapper[5052]: I1203 12:33:19.675243 5052 scope.go:117] "RemoveContainer" containerID="28fe6a9fcdf74a06f01f23fda2bdb34246e51f9604f3cd36102d78d52be9d6f6" Dec 03 12:33:19 crc kubenswrapper[5052]: I1203 12:33:19.692915 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f1d6c616-b537-4665-92fc-7ffdfad1a210" path="/var/lib/kubelet/pods/f1d6c616-b537-4665-92fc-7ffdfad1a210/volumes" Dec 03 12:33:19 crc kubenswrapper[5052]: I1203 12:33:19.706759 5052 scope.go:117] "RemoveContainer" containerID="4263fd69a4c9b5f25263e2be807fba706bb60e8af115e188221406a2cb517c61" Dec 03 12:33:19 crc kubenswrapper[5052]: I1203 12:33:19.734989 5052 scope.go:117] "RemoveContainer" containerID="de982574a6d170dab76a64098f7410a6d612c5fad55aa7ca04f5090fda64d238" Dec 03 12:33:19 crc kubenswrapper[5052]: I1203 12:33:19.774413 5052 scope.go:117] "RemoveContainer" containerID="e0aacbc34641afb12682d4842ddff06302fd96d916828aad6ac07df6cad52fcf" Dec 03 12:33:19 crc kubenswrapper[5052]: I1203 12:33:19.817903 5052 scope.go:117] "RemoveContainer" containerID="c1d51e836a79bfee8be38357907cb5f002159100bca556fe09a10fe835ba32b6" Dec 03 12:33:28 crc kubenswrapper[5052]: I1203 12:33:28.672979 5052 scope.go:117] "RemoveContainer" containerID="e6edaf188f85ea4f11f487f11cc489c700f25fb42b5ee269f4f496f1e127d772" Dec 03 12:33:28 crc kubenswrapper[5052]: E1203 12:33:28.673923 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 12:33:30 crc kubenswrapper[5052]: I1203 12:33:30.911372 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/rabbitmq-server-0"] Dec 03 12:33:30 crc kubenswrapper[5052]: E1203 12:33:30.912181 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="f1d6c616-b537-4665-92fc-7ffdfad1a210" containerName="extract-content" Dec 03 12:33:30 crc kubenswrapper[5052]: I1203 12:33:30.912198 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="f1d6c616-b537-4665-92fc-7ffdfad1a210" containerName="extract-content" Dec 03 12:33:30 crc kubenswrapper[5052]: E1203 12:33:30.912228 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="b6dc9243-9b95-4639-b4b9-a0adce5f25b8" containerName="storage" Dec 03 12:33:30 crc kubenswrapper[5052]: I1203 12:33:30.912236 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="b6dc9243-9b95-4639-b4b9-a0adce5f25b8" containerName="storage" Dec 03 12:33:30 crc kubenswrapper[5052]: E1203 12:33:30.912248 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="f1d6c616-b537-4665-92fc-7ffdfad1a210" containerName="extract-utilities" Dec 03 12:33:30 crc kubenswrapper[5052]: I1203 12:33:30.912260 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="f1d6c616-b537-4665-92fc-7ffdfad1a210" containerName="extract-utilities" Dec 03 12:33:30 crc kubenswrapper[5052]: E1203 12:33:30.912282 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="f1d6c616-b537-4665-92fc-7ffdfad1a210" containerName="registry-server" Dec 03 12:33:30 crc kubenswrapper[5052]: I1203 12:33:30.912289 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="f1d6c616-b537-4665-92fc-7ffdfad1a210" containerName="registry-server" Dec 03 12:33:30 crc kubenswrapper[5052]: I1203 12:33:30.912471 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="b6dc9243-9b95-4639-b4b9-a0adce5f25b8" containerName="storage" Dec 03 12:33:30 crc kubenswrapper[5052]: I1203 12:33:30.912488 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="f1d6c616-b537-4665-92fc-7ffdfad1a210" containerName="registry-server" Dec 03 12:33:30 crc kubenswrapper[5052]: I1203 12:33:30.916648 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:33:30 crc kubenswrapper[5052]: I1203 12:33:30.920365 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"rabbitmq-server-dockercfg-6xpcg" Dec 03 12:33:30 crc kubenswrapper[5052]: I1203 12:33:30.920678 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"rabbitmq-default-user" Dec 03 12:33:30 crc kubenswrapper[5052]: I1203 12:33:30.921055 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-rabbitmq-svc" Dec 03 12:33:30 crc kubenswrapper[5052]: I1203 12:33:30.921765 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"rabbitmq-config-data" Dec 03 12:33:30 crc kubenswrapper[5052]: I1203 12:33:30.921999 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"rabbitmq-plugins-conf" Dec 03 12:33:30 crc kubenswrapper[5052]: I1203 12:33:30.922636 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"rabbitmq-server-conf" Dec 03 12:33:30 crc kubenswrapper[5052]: I1203 12:33:30.928066 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"rabbitmq-erlang-cookie" Dec 03 12:33:30 crc kubenswrapper[5052]: I1203 12:33:30.934468 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/rabbitmq-server-0"] Dec 03 12:33:31 crc kubenswrapper[5052]: I1203 12:33:31.042888 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"erlang-cookie-secret\" (UniqueName: \"kubernetes.io/secret/d82ef8ea-0c3b-4d89-9c78-22095940f4da-erlang-cookie-secret\") pod \"rabbitmq-server-0\" (UID: \"d82ef8ea-0c3b-4d89-9c78-22095940f4da\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:33:31 crc kubenswrapper[5052]: I1203 12:33:31.042933 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/d82ef8ea-0c3b-4d89-9c78-22095940f4da-config-data\") pod \"rabbitmq-server-0\" (UID: \"d82ef8ea-0c3b-4d89-9c78-22095940f4da\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:33:31 crc kubenswrapper[5052]: I1203 12:33:31.042958 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"rabbitmq-erlang-cookie\" (UniqueName: \"kubernetes.io/empty-dir/d82ef8ea-0c3b-4d89-9c78-22095940f4da-rabbitmq-erlang-cookie\") pod \"rabbitmq-server-0\" (UID: \"d82ef8ea-0c3b-4d89-9c78-22095940f4da\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:33:31 crc kubenswrapper[5052]: I1203 12:33:31.042978 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"rabbitmq-tls\" (UniqueName: \"kubernetes.io/projected/d82ef8ea-0c3b-4d89-9c78-22095940f4da-rabbitmq-tls\") pod \"rabbitmq-server-0\" (UID: \"d82ef8ea-0c3b-4d89-9c78-22095940f4da\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:33:31 crc kubenswrapper[5052]: I1203 12:33:31.043016 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage06-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage06-crc\") pod \"rabbitmq-server-0\" (UID: \"d82ef8ea-0c3b-4d89-9c78-22095940f4da\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:33:31 crc kubenswrapper[5052]: I1203 12:33:31.043051 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"pod-info\" (UniqueName: \"kubernetes.io/downward-api/d82ef8ea-0c3b-4d89-9c78-22095940f4da-pod-info\") pod \"rabbitmq-server-0\" (UID: \"d82ef8ea-0c3b-4d89-9c78-22095940f4da\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:33:31 crc kubenswrapper[5052]: I1203 12:33:31.043092 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-conf\" (UniqueName: \"kubernetes.io/configmap/d82ef8ea-0c3b-4d89-9c78-22095940f4da-plugins-conf\") pod \"rabbitmq-server-0\" (UID: \"d82ef8ea-0c3b-4d89-9c78-22095940f4da\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:33:31 crc kubenswrapper[5052]: I1203 12:33:31.043107 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"rabbitmq-plugins\" (UniqueName: \"kubernetes.io/empty-dir/d82ef8ea-0c3b-4d89-9c78-22095940f4da-rabbitmq-plugins\") pod \"rabbitmq-server-0\" (UID: \"d82ef8ea-0c3b-4d89-9c78-22095940f4da\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:33:31 crc kubenswrapper[5052]: I1203 12:33:31.043132 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"server-conf\" (UniqueName: \"kubernetes.io/configmap/d82ef8ea-0c3b-4d89-9c78-22095940f4da-server-conf\") pod \"rabbitmq-server-0\" (UID: \"d82ef8ea-0c3b-4d89-9c78-22095940f4da\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:33:31 crc kubenswrapper[5052]: I1203 12:33:31.043148 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"rabbitmq-confd\" (UniqueName: \"kubernetes.io/projected/d82ef8ea-0c3b-4d89-9c78-22095940f4da-rabbitmq-confd\") pod \"rabbitmq-server-0\" (UID: \"d82ef8ea-0c3b-4d89-9c78-22095940f4da\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:33:31 crc kubenswrapper[5052]: I1203 12:33:31.043170 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x46l5\" (UniqueName: \"kubernetes.io/projected/d82ef8ea-0c3b-4d89-9c78-22095940f4da-kube-api-access-x46l5\") pod \"rabbitmq-server-0\" (UID: \"d82ef8ea-0c3b-4d89-9c78-22095940f4da\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:33:31 crc kubenswrapper[5052]: I1203 12:33:31.144380 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-x46l5\" (UniqueName: \"kubernetes.io/projected/d82ef8ea-0c3b-4d89-9c78-22095940f4da-kube-api-access-x46l5\") pod \"rabbitmq-server-0\" (UID: \"d82ef8ea-0c3b-4d89-9c78-22095940f4da\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:33:31 crc kubenswrapper[5052]: I1203 12:33:31.144449 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"erlang-cookie-secret\" (UniqueName: \"kubernetes.io/secret/d82ef8ea-0c3b-4d89-9c78-22095940f4da-erlang-cookie-secret\") pod \"rabbitmq-server-0\" (UID: \"d82ef8ea-0c3b-4d89-9c78-22095940f4da\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:33:31 crc kubenswrapper[5052]: I1203 12:33:31.144468 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/d82ef8ea-0c3b-4d89-9c78-22095940f4da-config-data\") pod \"rabbitmq-server-0\" (UID: \"d82ef8ea-0c3b-4d89-9c78-22095940f4da\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:33:31 crc kubenswrapper[5052]: I1203 12:33:31.144490 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"rabbitmq-erlang-cookie\" (UniqueName: \"kubernetes.io/empty-dir/d82ef8ea-0c3b-4d89-9c78-22095940f4da-rabbitmq-erlang-cookie\") pod \"rabbitmq-server-0\" (UID: \"d82ef8ea-0c3b-4d89-9c78-22095940f4da\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:33:31 crc kubenswrapper[5052]: I1203 12:33:31.144533 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"rabbitmq-tls\" (UniqueName: \"kubernetes.io/projected/d82ef8ea-0c3b-4d89-9c78-22095940f4da-rabbitmq-tls\") pod \"rabbitmq-server-0\" (UID: \"d82ef8ea-0c3b-4d89-9c78-22095940f4da\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:33:31 crc kubenswrapper[5052]: I1203 12:33:31.144557 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage06-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage06-crc\") pod \"rabbitmq-server-0\" (UID: \"d82ef8ea-0c3b-4d89-9c78-22095940f4da\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:33:31 crc kubenswrapper[5052]: I1203 12:33:31.144601 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"pod-info\" (UniqueName: \"kubernetes.io/downward-api/d82ef8ea-0c3b-4d89-9c78-22095940f4da-pod-info\") pod \"rabbitmq-server-0\" (UID: \"d82ef8ea-0c3b-4d89-9c78-22095940f4da\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:33:31 crc kubenswrapper[5052]: I1203 12:33:31.144633 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"plugins-conf\" (UniqueName: \"kubernetes.io/configmap/d82ef8ea-0c3b-4d89-9c78-22095940f4da-plugins-conf\") pod \"rabbitmq-server-0\" (UID: \"d82ef8ea-0c3b-4d89-9c78-22095940f4da\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:33:31 crc kubenswrapper[5052]: I1203 12:33:31.144649 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"rabbitmq-plugins\" (UniqueName: \"kubernetes.io/empty-dir/d82ef8ea-0c3b-4d89-9c78-22095940f4da-rabbitmq-plugins\") pod \"rabbitmq-server-0\" (UID: \"d82ef8ea-0c3b-4d89-9c78-22095940f4da\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:33:31 crc kubenswrapper[5052]: I1203 12:33:31.144676 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"server-conf\" (UniqueName: \"kubernetes.io/configmap/d82ef8ea-0c3b-4d89-9c78-22095940f4da-server-conf\") pod \"rabbitmq-server-0\" (UID: \"d82ef8ea-0c3b-4d89-9c78-22095940f4da\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:33:31 crc kubenswrapper[5052]: I1203 12:33:31.144692 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"rabbitmq-confd\" (UniqueName: \"kubernetes.io/projected/d82ef8ea-0c3b-4d89-9c78-22095940f4da-rabbitmq-confd\") pod \"rabbitmq-server-0\" (UID: \"d82ef8ea-0c3b-4d89-9c78-22095940f4da\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:33:31 crc kubenswrapper[5052]: I1203 12:33:31.145899 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage06-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage06-crc\") pod \"rabbitmq-server-0\" (UID: \"d82ef8ea-0c3b-4d89-9c78-22095940f4da\") device mount path \"/mnt/openstack/pv06\"" pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:33:31 crc kubenswrapper[5052]: I1203 12:33:31.145937 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"plugins-conf\" (UniqueName: \"kubernetes.io/configmap/d82ef8ea-0c3b-4d89-9c78-22095940f4da-plugins-conf\") pod \"rabbitmq-server-0\" (UID: \"d82ef8ea-0c3b-4d89-9c78-22095940f4da\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:33:31 crc kubenswrapper[5052]: I1203 12:33:31.145969 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/d82ef8ea-0c3b-4d89-9c78-22095940f4da-config-data\") pod \"rabbitmq-server-0\" (UID: \"d82ef8ea-0c3b-4d89-9c78-22095940f4da\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:33:31 crc kubenswrapper[5052]: I1203 12:33:31.146043 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"rabbitmq-erlang-cookie\" (UniqueName: \"kubernetes.io/empty-dir/d82ef8ea-0c3b-4d89-9c78-22095940f4da-rabbitmq-erlang-cookie\") pod \"rabbitmq-server-0\" (UID: \"d82ef8ea-0c3b-4d89-9c78-22095940f4da\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:33:31 crc kubenswrapper[5052]: I1203 12:33:31.146257 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"rabbitmq-plugins\" (UniqueName: \"kubernetes.io/empty-dir/d82ef8ea-0c3b-4d89-9c78-22095940f4da-rabbitmq-plugins\") pod \"rabbitmq-server-0\" (UID: \"d82ef8ea-0c3b-4d89-9c78-22095940f4da\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:33:31 crc kubenswrapper[5052]: I1203 12:33:31.146931 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"server-conf\" (UniqueName: \"kubernetes.io/configmap/d82ef8ea-0c3b-4d89-9c78-22095940f4da-server-conf\") pod \"rabbitmq-server-0\" (UID: \"d82ef8ea-0c3b-4d89-9c78-22095940f4da\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:33:31 crc kubenswrapper[5052]: I1203 12:33:31.151340 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"erlang-cookie-secret\" (UniqueName: \"kubernetes.io/secret/d82ef8ea-0c3b-4d89-9c78-22095940f4da-erlang-cookie-secret\") pod \"rabbitmq-server-0\" (UID: \"d82ef8ea-0c3b-4d89-9c78-22095940f4da\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:33:31 crc kubenswrapper[5052]: I1203 12:33:31.153923 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"rabbitmq-tls\" (UniqueName: \"kubernetes.io/projected/d82ef8ea-0c3b-4d89-9c78-22095940f4da-rabbitmq-tls\") pod \"rabbitmq-server-0\" (UID: \"d82ef8ea-0c3b-4d89-9c78-22095940f4da\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:33:31 crc kubenswrapper[5052]: I1203 12:33:31.157328 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"pod-info\" (UniqueName: \"kubernetes.io/downward-api/d82ef8ea-0c3b-4d89-9c78-22095940f4da-pod-info\") pod \"rabbitmq-server-0\" (UID: \"d82ef8ea-0c3b-4d89-9c78-22095940f4da\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:33:31 crc kubenswrapper[5052]: I1203 12:33:31.160530 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"rabbitmq-confd\" (UniqueName: \"kubernetes.io/projected/d82ef8ea-0c3b-4d89-9c78-22095940f4da-rabbitmq-confd\") pod \"rabbitmq-server-0\" (UID: \"d82ef8ea-0c3b-4d89-9c78-22095940f4da\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:33:31 crc kubenswrapper[5052]: I1203 12:33:31.176008 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage06-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage06-crc\") pod \"rabbitmq-server-0\" (UID: \"d82ef8ea-0c3b-4d89-9c78-22095940f4da\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:33:31 crc kubenswrapper[5052]: I1203 12:33:31.179260 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-x46l5\" (UniqueName: \"kubernetes.io/projected/d82ef8ea-0c3b-4d89-9c78-22095940f4da-kube-api-access-x46l5\") pod \"rabbitmq-server-0\" (UID: \"d82ef8ea-0c3b-4d89-9c78-22095940f4da\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:33:31 crc kubenswrapper[5052]: I1203 12:33:31.232880 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/rabbitmq-cell1-server-0"] Dec 03 12:33:31 crc kubenswrapper[5052]: I1203 12:33:31.234638 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:33:31 crc kubenswrapper[5052]: I1203 12:33:31.236552 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"rabbitmq-cell1-default-user" Dec 03 12:33:31 crc kubenswrapper[5052]: I1203 12:33:31.237128 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"rabbitmq-cell1-erlang-cookie" Dec 03 12:33:31 crc kubenswrapper[5052]: I1203 12:33:31.237193 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"rabbitmq-cell1-config-data" Dec 03 12:33:31 crc kubenswrapper[5052]: I1203 12:33:31.237215 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"rabbitmq-cell1-server-dockercfg-d5n7x" Dec 03 12:33:31 crc kubenswrapper[5052]: I1203 12:33:31.237272 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"rabbitmq-cell1-plugins-conf" Dec 03 12:33:31 crc kubenswrapper[5052]: I1203 12:33:31.237331 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"rabbitmq-cell1-server-conf" Dec 03 12:33:31 crc kubenswrapper[5052]: I1203 12:33:31.237346 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-rabbitmq-cell1-svc" Dec 03 12:33:31 crc kubenswrapper[5052]: I1203 12:33:31.243140 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/rabbitmq-cell1-server-0"] Dec 03 12:33:31 crc kubenswrapper[5052]: I1203 12:33:31.251987 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:33:31 crc kubenswrapper[5052]: I1203 12:33:31.355783 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage04-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage04-crc\") pod \"rabbitmq-cell1-server-0\" (UID: \"f9bac64c-518e-4630-8889-20e09d5b7a39\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:33:31 crc kubenswrapper[5052]: I1203 12:33:31.355834 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-conf\" (UniqueName: \"kubernetes.io/configmap/f9bac64c-518e-4630-8889-20e09d5b7a39-plugins-conf\") pod \"rabbitmq-cell1-server-0\" (UID: \"f9bac64c-518e-4630-8889-20e09d5b7a39\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:33:31 crc kubenswrapper[5052]: I1203 12:33:31.355867 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-69sgs\" (UniqueName: \"kubernetes.io/projected/f9bac64c-518e-4630-8889-20e09d5b7a39-kube-api-access-69sgs\") pod \"rabbitmq-cell1-server-0\" (UID: \"f9bac64c-518e-4630-8889-20e09d5b7a39\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:33:31 crc kubenswrapper[5052]: I1203 12:33:31.355887 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"pod-info\" (UniqueName: \"kubernetes.io/downward-api/f9bac64c-518e-4630-8889-20e09d5b7a39-pod-info\") pod \"rabbitmq-cell1-server-0\" (UID: \"f9bac64c-518e-4630-8889-20e09d5b7a39\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:33:31 crc kubenswrapper[5052]: I1203 12:33:31.355909 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"rabbitmq-tls\" (UniqueName: \"kubernetes.io/projected/f9bac64c-518e-4630-8889-20e09d5b7a39-rabbitmq-tls\") pod \"rabbitmq-cell1-server-0\" (UID: \"f9bac64c-518e-4630-8889-20e09d5b7a39\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:33:31 crc kubenswrapper[5052]: I1203 12:33:31.356148 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"rabbitmq-confd\" (UniqueName: \"kubernetes.io/projected/f9bac64c-518e-4630-8889-20e09d5b7a39-rabbitmq-confd\") pod \"rabbitmq-cell1-server-0\" (UID: \"f9bac64c-518e-4630-8889-20e09d5b7a39\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:33:31 crc kubenswrapper[5052]: I1203 12:33:31.356261 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"server-conf\" (UniqueName: \"kubernetes.io/configmap/f9bac64c-518e-4630-8889-20e09d5b7a39-server-conf\") pod \"rabbitmq-cell1-server-0\" (UID: \"f9bac64c-518e-4630-8889-20e09d5b7a39\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:33:31 crc kubenswrapper[5052]: I1203 12:33:31.356325 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"erlang-cookie-secret\" (UniqueName: \"kubernetes.io/secret/f9bac64c-518e-4630-8889-20e09d5b7a39-erlang-cookie-secret\") pod \"rabbitmq-cell1-server-0\" (UID: \"f9bac64c-518e-4630-8889-20e09d5b7a39\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:33:31 crc kubenswrapper[5052]: I1203 12:33:31.356378 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"rabbitmq-erlang-cookie\" (UniqueName: \"kubernetes.io/empty-dir/f9bac64c-518e-4630-8889-20e09d5b7a39-rabbitmq-erlang-cookie\") pod \"rabbitmq-cell1-server-0\" (UID: \"f9bac64c-518e-4630-8889-20e09d5b7a39\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:33:31 crc kubenswrapper[5052]: I1203 12:33:31.356467 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/f9bac64c-518e-4630-8889-20e09d5b7a39-config-data\") pod \"rabbitmq-cell1-server-0\" (UID: \"f9bac64c-518e-4630-8889-20e09d5b7a39\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:33:31 crc kubenswrapper[5052]: I1203 12:33:31.356552 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"rabbitmq-plugins\" (UniqueName: \"kubernetes.io/empty-dir/f9bac64c-518e-4630-8889-20e09d5b7a39-rabbitmq-plugins\") pod \"rabbitmq-cell1-server-0\" (UID: \"f9bac64c-518e-4630-8889-20e09d5b7a39\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:33:31 crc kubenswrapper[5052]: I1203 12:33:31.458110 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"rabbitmq-confd\" (UniqueName: \"kubernetes.io/projected/f9bac64c-518e-4630-8889-20e09d5b7a39-rabbitmq-confd\") pod \"rabbitmq-cell1-server-0\" (UID: \"f9bac64c-518e-4630-8889-20e09d5b7a39\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:33:31 crc kubenswrapper[5052]: I1203 12:33:31.458177 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"server-conf\" (UniqueName: \"kubernetes.io/configmap/f9bac64c-518e-4630-8889-20e09d5b7a39-server-conf\") pod \"rabbitmq-cell1-server-0\" (UID: \"f9bac64c-518e-4630-8889-20e09d5b7a39\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:33:31 crc kubenswrapper[5052]: I1203 12:33:31.458206 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"erlang-cookie-secret\" (UniqueName: \"kubernetes.io/secret/f9bac64c-518e-4630-8889-20e09d5b7a39-erlang-cookie-secret\") pod \"rabbitmq-cell1-server-0\" (UID: \"f9bac64c-518e-4630-8889-20e09d5b7a39\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:33:31 crc kubenswrapper[5052]: I1203 12:33:31.458231 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"rabbitmq-erlang-cookie\" (UniqueName: \"kubernetes.io/empty-dir/f9bac64c-518e-4630-8889-20e09d5b7a39-rabbitmq-erlang-cookie\") pod \"rabbitmq-cell1-server-0\" (UID: \"f9bac64c-518e-4630-8889-20e09d5b7a39\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:33:31 crc kubenswrapper[5052]: I1203 12:33:31.458280 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/f9bac64c-518e-4630-8889-20e09d5b7a39-config-data\") pod \"rabbitmq-cell1-server-0\" (UID: \"f9bac64c-518e-4630-8889-20e09d5b7a39\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:33:31 crc kubenswrapper[5052]: I1203 12:33:31.458327 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"rabbitmq-plugins\" (UniqueName: \"kubernetes.io/empty-dir/f9bac64c-518e-4630-8889-20e09d5b7a39-rabbitmq-plugins\") pod \"rabbitmq-cell1-server-0\" (UID: \"f9bac64c-518e-4630-8889-20e09d5b7a39\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:33:31 crc kubenswrapper[5052]: I1203 12:33:31.458372 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage04-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage04-crc\") pod \"rabbitmq-cell1-server-0\" (UID: \"f9bac64c-518e-4630-8889-20e09d5b7a39\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:33:31 crc kubenswrapper[5052]: I1203 12:33:31.458402 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"plugins-conf\" (UniqueName: \"kubernetes.io/configmap/f9bac64c-518e-4630-8889-20e09d5b7a39-plugins-conf\") pod \"rabbitmq-cell1-server-0\" (UID: \"f9bac64c-518e-4630-8889-20e09d5b7a39\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:33:31 crc kubenswrapper[5052]: I1203 12:33:31.458439 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-69sgs\" (UniqueName: \"kubernetes.io/projected/f9bac64c-518e-4630-8889-20e09d5b7a39-kube-api-access-69sgs\") pod \"rabbitmq-cell1-server-0\" (UID: \"f9bac64c-518e-4630-8889-20e09d5b7a39\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:33:31 crc kubenswrapper[5052]: I1203 12:33:31.458463 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"pod-info\" (UniqueName: \"kubernetes.io/downward-api/f9bac64c-518e-4630-8889-20e09d5b7a39-pod-info\") pod \"rabbitmq-cell1-server-0\" (UID: \"f9bac64c-518e-4630-8889-20e09d5b7a39\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:33:31 crc kubenswrapper[5052]: I1203 12:33:31.458494 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"rabbitmq-tls\" (UniqueName: \"kubernetes.io/projected/f9bac64c-518e-4630-8889-20e09d5b7a39-rabbitmq-tls\") pod \"rabbitmq-cell1-server-0\" (UID: \"f9bac64c-518e-4630-8889-20e09d5b7a39\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:33:31 crc kubenswrapper[5052]: I1203 12:33:31.459273 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage04-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage04-crc\") pod \"rabbitmq-cell1-server-0\" (UID: \"f9bac64c-518e-4630-8889-20e09d5b7a39\") device mount path \"/mnt/openstack/pv04\"" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:33:31 crc kubenswrapper[5052]: I1203 12:33:31.460059 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/f9bac64c-518e-4630-8889-20e09d5b7a39-config-data\") pod \"rabbitmq-cell1-server-0\" (UID: \"f9bac64c-518e-4630-8889-20e09d5b7a39\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:33:31 crc kubenswrapper[5052]: I1203 12:33:31.460333 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"rabbitmq-erlang-cookie\" (UniqueName: \"kubernetes.io/empty-dir/f9bac64c-518e-4630-8889-20e09d5b7a39-rabbitmq-erlang-cookie\") pod \"rabbitmq-cell1-server-0\" (UID: \"f9bac64c-518e-4630-8889-20e09d5b7a39\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:33:31 crc kubenswrapper[5052]: I1203 12:33:31.460477 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"rabbitmq-plugins\" (UniqueName: \"kubernetes.io/empty-dir/f9bac64c-518e-4630-8889-20e09d5b7a39-rabbitmq-plugins\") pod \"rabbitmq-cell1-server-0\" (UID: \"f9bac64c-518e-4630-8889-20e09d5b7a39\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:33:31 crc kubenswrapper[5052]: I1203 12:33:31.460926 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"server-conf\" (UniqueName: \"kubernetes.io/configmap/f9bac64c-518e-4630-8889-20e09d5b7a39-server-conf\") pod \"rabbitmq-cell1-server-0\" (UID: \"f9bac64c-518e-4630-8889-20e09d5b7a39\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:33:31 crc kubenswrapper[5052]: I1203 12:33:31.462369 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"plugins-conf\" (UniqueName: \"kubernetes.io/configmap/f9bac64c-518e-4630-8889-20e09d5b7a39-plugins-conf\") pod \"rabbitmq-cell1-server-0\" (UID: \"f9bac64c-518e-4630-8889-20e09d5b7a39\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:33:31 crc kubenswrapper[5052]: I1203 12:33:31.462619 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"rabbitmq-tls\" (UniqueName: \"kubernetes.io/projected/f9bac64c-518e-4630-8889-20e09d5b7a39-rabbitmq-tls\") pod \"rabbitmq-cell1-server-0\" (UID: \"f9bac64c-518e-4630-8889-20e09d5b7a39\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:33:31 crc kubenswrapper[5052]: I1203 12:33:31.463254 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"rabbitmq-confd\" (UniqueName: \"kubernetes.io/projected/f9bac64c-518e-4630-8889-20e09d5b7a39-rabbitmq-confd\") pod \"rabbitmq-cell1-server-0\" (UID: \"f9bac64c-518e-4630-8889-20e09d5b7a39\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:33:31 crc kubenswrapper[5052]: I1203 12:33:31.463566 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"erlang-cookie-secret\" (UniqueName: \"kubernetes.io/secret/f9bac64c-518e-4630-8889-20e09d5b7a39-erlang-cookie-secret\") pod \"rabbitmq-cell1-server-0\" (UID: \"f9bac64c-518e-4630-8889-20e09d5b7a39\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:33:31 crc kubenswrapper[5052]: I1203 12:33:31.465383 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"pod-info\" (UniqueName: \"kubernetes.io/downward-api/f9bac64c-518e-4630-8889-20e09d5b7a39-pod-info\") pod \"rabbitmq-cell1-server-0\" (UID: \"f9bac64c-518e-4630-8889-20e09d5b7a39\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:33:31 crc kubenswrapper[5052]: I1203 12:33:31.488026 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-69sgs\" (UniqueName: \"kubernetes.io/projected/f9bac64c-518e-4630-8889-20e09d5b7a39-kube-api-access-69sgs\") pod \"rabbitmq-cell1-server-0\" (UID: \"f9bac64c-518e-4630-8889-20e09d5b7a39\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:33:31 crc kubenswrapper[5052]: I1203 12:33:31.501034 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage04-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage04-crc\") pod \"rabbitmq-cell1-server-0\" (UID: \"f9bac64c-518e-4630-8889-20e09d5b7a39\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:33:31 crc kubenswrapper[5052]: I1203 12:33:31.573998 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:33:31 crc kubenswrapper[5052]: I1203 12:33:31.695703 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/rabbitmq-server-0"] Dec 03 12:33:31 crc kubenswrapper[5052]: W1203 12:33:31.718005 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd82ef8ea_0c3b_4d89_9c78_22095940f4da.slice/crio-5c73705bdec2516967e4b29b1ee550293c51c115bca310b791b246711b0cea24 WatchSource:0}: Error finding container 5c73705bdec2516967e4b29b1ee550293c51c115bca310b791b246711b0cea24: Status 404 returned error can't find the container with id 5c73705bdec2516967e4b29b1ee550293c51c115bca310b791b246711b0cea24 Dec 03 12:33:32 crc kubenswrapper[5052]: I1203 12:33:32.104301 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/rabbitmq-cell1-server-0"] Dec 03 12:33:32 crc kubenswrapper[5052]: W1203 12:33:32.106698 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf9bac64c_518e_4630_8889_20e09d5b7a39.slice/crio-a2b1a35a346db4af01b43450f89d01305b46c2cd55396705ef6ff7ffa849bc0a WatchSource:0}: Error finding container a2b1a35a346db4af01b43450f89d01305b46c2cd55396705ef6ff7ffa849bc0a: Status 404 returned error can't find the container with id a2b1a35a346db4af01b43450f89d01305b46c2cd55396705ef6ff7ffa849bc0a Dec 03 12:33:32 crc kubenswrapper[5052]: I1203 12:33:32.167629 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-server-0" event={"ID":"d82ef8ea-0c3b-4d89-9c78-22095940f4da","Type":"ContainerStarted","Data":"5c73705bdec2516967e4b29b1ee550293c51c115bca310b791b246711b0cea24"} Dec 03 12:33:32 crc kubenswrapper[5052]: I1203 12:33:32.169275 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" event={"ID":"f9bac64c-518e-4630-8889-20e09d5b7a39","Type":"ContainerStarted","Data":"a2b1a35a346db4af01b43450f89d01305b46c2cd55396705ef6ff7ffa849bc0a"} Dec 03 12:33:32 crc kubenswrapper[5052]: I1203 12:33:32.774626 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/openstack-galera-0"] Dec 03 12:33:32 crc kubenswrapper[5052]: I1203 12:33:32.776783 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:33:32 crc kubenswrapper[5052]: I1203 12:33:32.782639 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"openstack-scripts" Dec 03 12:33:32 crc kubenswrapper[5052]: I1203 12:33:32.782884 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"openstack-config-data" Dec 03 12:33:32 crc kubenswrapper[5052]: I1203 12:33:32.783131 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"galera-openstack-dockercfg-hsg7j" Dec 03 12:33:32 crc kubenswrapper[5052]: I1203 12:33:32.788152 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-galera-openstack-svc" Dec 03 12:33:32 crc kubenswrapper[5052]: I1203 12:33:32.788384 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/openstack-galera-0"] Dec 03 12:33:32 crc kubenswrapper[5052]: I1203 12:33:32.796216 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"combined-ca-bundle" Dec 03 12:33:32 crc kubenswrapper[5052]: I1203 12:33:32.880198 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/01fb7120-3521-4969-a2c4-b2637e0e2414-kolla-config\") pod \"openstack-galera-0\" (UID: \"01fb7120-3521-4969-a2c4-b2637e0e2414\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:33:32 crc kubenswrapper[5052]: I1203 12:33:32.880276 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/01fb7120-3521-4969-a2c4-b2637e0e2414-galera-tls-certs\") pod \"openstack-galera-0\" (UID: \"01fb7120-3521-4969-a2c4-b2637e0e2414\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:33:32 crc kubenswrapper[5052]: I1203 12:33:32.880313 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage07-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage07-crc\") pod \"openstack-galera-0\" (UID: \"01fb7120-3521-4969-a2c4-b2637e0e2414\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:33:32 crc kubenswrapper[5052]: I1203 12:33:32.880373 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/01fb7120-3521-4969-a2c4-b2637e0e2414-config-data-generated\") pod \"openstack-galera-0\" (UID: \"01fb7120-3521-4969-a2c4-b2637e0e2414\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:33:32 crc kubenswrapper[5052]: I1203 12:33:32.880402 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/01fb7120-3521-4969-a2c4-b2637e0e2414-config-data-default\") pod \"openstack-galera-0\" (UID: \"01fb7120-3521-4969-a2c4-b2637e0e2414\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:33:32 crc kubenswrapper[5052]: I1203 12:33:32.880422 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wksnx\" (UniqueName: \"kubernetes.io/projected/01fb7120-3521-4969-a2c4-b2637e0e2414-kube-api-access-wksnx\") pod \"openstack-galera-0\" (UID: \"01fb7120-3521-4969-a2c4-b2637e0e2414\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:33:32 crc kubenswrapper[5052]: I1203 12:33:32.880455 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/01fb7120-3521-4969-a2c4-b2637e0e2414-operator-scripts\") pod \"openstack-galera-0\" (UID: \"01fb7120-3521-4969-a2c4-b2637e0e2414\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:33:32 crc kubenswrapper[5052]: I1203 12:33:32.880499 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/01fb7120-3521-4969-a2c4-b2637e0e2414-combined-ca-bundle\") pod \"openstack-galera-0\" (UID: \"01fb7120-3521-4969-a2c4-b2637e0e2414\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:33:32 crc kubenswrapper[5052]: I1203 12:33:32.982450 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/01fb7120-3521-4969-a2c4-b2637e0e2414-galera-tls-certs\") pod \"openstack-galera-0\" (UID: \"01fb7120-3521-4969-a2c4-b2637e0e2414\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:33:32 crc kubenswrapper[5052]: I1203 12:33:32.982849 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage07-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage07-crc\") pod \"openstack-galera-0\" (UID: \"01fb7120-3521-4969-a2c4-b2637e0e2414\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:33:32 crc kubenswrapper[5052]: I1203 12:33:32.982923 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/01fb7120-3521-4969-a2c4-b2637e0e2414-config-data-generated\") pod \"openstack-galera-0\" (UID: \"01fb7120-3521-4969-a2c4-b2637e0e2414\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:33:32 crc kubenswrapper[5052]: I1203 12:33:32.982957 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/01fb7120-3521-4969-a2c4-b2637e0e2414-config-data-default\") pod \"openstack-galera-0\" (UID: \"01fb7120-3521-4969-a2c4-b2637e0e2414\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:33:32 crc kubenswrapper[5052]: I1203 12:33:32.982981 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-wksnx\" (UniqueName: \"kubernetes.io/projected/01fb7120-3521-4969-a2c4-b2637e0e2414-kube-api-access-wksnx\") pod \"openstack-galera-0\" (UID: \"01fb7120-3521-4969-a2c4-b2637e0e2414\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:33:32 crc kubenswrapper[5052]: I1203 12:33:32.983010 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/01fb7120-3521-4969-a2c4-b2637e0e2414-operator-scripts\") pod \"openstack-galera-0\" (UID: \"01fb7120-3521-4969-a2c4-b2637e0e2414\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:33:32 crc kubenswrapper[5052]: I1203 12:33:32.983059 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/01fb7120-3521-4969-a2c4-b2637e0e2414-combined-ca-bundle\") pod \"openstack-galera-0\" (UID: \"01fb7120-3521-4969-a2c4-b2637e0e2414\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:33:32 crc kubenswrapper[5052]: I1203 12:33:32.983123 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage07-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage07-crc\") pod \"openstack-galera-0\" (UID: \"01fb7120-3521-4969-a2c4-b2637e0e2414\") device mount path \"/mnt/openstack/pv07\"" pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:33:32 crc kubenswrapper[5052]: I1203 12:33:32.983145 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/01fb7120-3521-4969-a2c4-b2637e0e2414-kolla-config\") pod \"openstack-galera-0\" (UID: \"01fb7120-3521-4969-a2c4-b2637e0e2414\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:33:32 crc kubenswrapper[5052]: I1203 12:33:32.983442 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/01fb7120-3521-4969-a2c4-b2637e0e2414-config-data-generated\") pod \"openstack-galera-0\" (UID: \"01fb7120-3521-4969-a2c4-b2637e0e2414\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:33:32 crc kubenswrapper[5052]: I1203 12:33:32.983906 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/01fb7120-3521-4969-a2c4-b2637e0e2414-kolla-config\") pod \"openstack-galera-0\" (UID: \"01fb7120-3521-4969-a2c4-b2637e0e2414\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:33:32 crc kubenswrapper[5052]: I1203 12:33:32.984169 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/01fb7120-3521-4969-a2c4-b2637e0e2414-config-data-default\") pod \"openstack-galera-0\" (UID: \"01fb7120-3521-4969-a2c4-b2637e0e2414\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:33:32 crc kubenswrapper[5052]: I1203 12:33:32.984776 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/01fb7120-3521-4969-a2c4-b2637e0e2414-operator-scripts\") pod \"openstack-galera-0\" (UID: \"01fb7120-3521-4969-a2c4-b2637e0e2414\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:33:33 crc kubenswrapper[5052]: I1203 12:33:33.024827 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/01fb7120-3521-4969-a2c4-b2637e0e2414-galera-tls-certs\") pod \"openstack-galera-0\" (UID: \"01fb7120-3521-4969-a2c4-b2637e0e2414\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:33:33 crc kubenswrapper[5052]: I1203 12:33:33.029020 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/01fb7120-3521-4969-a2c4-b2637e0e2414-combined-ca-bundle\") pod \"openstack-galera-0\" (UID: \"01fb7120-3521-4969-a2c4-b2637e0e2414\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:33:33 crc kubenswrapper[5052]: I1203 12:33:33.029393 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-wksnx\" (UniqueName: \"kubernetes.io/projected/01fb7120-3521-4969-a2c4-b2637e0e2414-kube-api-access-wksnx\") pod \"openstack-galera-0\" (UID: \"01fb7120-3521-4969-a2c4-b2637e0e2414\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:33:33 crc kubenswrapper[5052]: I1203 12:33:33.051575 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage07-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage07-crc\") pod \"openstack-galera-0\" (UID: \"01fb7120-3521-4969-a2c4-b2637e0e2414\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:33:33 crc kubenswrapper[5052]: I1203 12:33:33.094321 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:33:33 crc kubenswrapper[5052]: I1203 12:33:33.187807 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-server-0" event={"ID":"d82ef8ea-0c3b-4d89-9c78-22095940f4da","Type":"ContainerStarted","Data":"69ee48f37ae71f152a64da3b3da0915d74d05a1eb1b6c9a0996fd2a43065ecba"} Dec 03 12:33:33 crc kubenswrapper[5052]: I1203 12:33:33.597893 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/openstack-galera-0"] Dec 03 12:33:33 crc kubenswrapper[5052]: W1203 12:33:33.599113 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod01fb7120_3521_4969_a2c4_b2637e0e2414.slice/crio-f9456d80cb676adb94c83be749d62019435a6816deaec5cc23878d4aed8c335f WatchSource:0}: Error finding container f9456d80cb676adb94c83be749d62019435a6816deaec5cc23878d4aed8c335f: Status 404 returned error can't find the container with id f9456d80cb676adb94c83be749d62019435a6816deaec5cc23878d4aed8c335f Dec 03 12:33:34 crc kubenswrapper[5052]: I1203 12:33:34.090859 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/openstack-cell1-galera-0"] Dec 03 12:33:34 crc kubenswrapper[5052]: I1203 12:33:34.101815 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:33:34 crc kubenswrapper[5052]: I1203 12:33:34.109052 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-galera-openstack-cell1-svc" Dec 03 12:33:34 crc kubenswrapper[5052]: I1203 12:33:34.114952 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"galera-openstack-cell1-dockercfg-4wff5" Dec 03 12:33:34 crc kubenswrapper[5052]: I1203 12:33:34.115492 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"openstack-cell1-scripts" Dec 03 12:33:34 crc kubenswrapper[5052]: I1203 12:33:34.116197 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"openstack-cell1-config-data" Dec 03 12:33:34 crc kubenswrapper[5052]: I1203 12:33:34.130756 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/openstack-cell1-galera-0"] Dec 03 12:33:34 crc kubenswrapper[5052]: I1203 12:33:34.197666 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" event={"ID":"f9bac64c-518e-4630-8889-20e09d5b7a39","Type":"ContainerStarted","Data":"806aad7eebb3067b147df9728513f4f5c522d884aded6480c84b0af0d1a1afd4"} Dec 03 12:33:34 crc kubenswrapper[5052]: I1203 12:33:34.202955 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-galera-0" event={"ID":"01fb7120-3521-4969-a2c4-b2637e0e2414","Type":"ContainerStarted","Data":"7cdb8d3632d0751bac948ca612e51336988e363391bc868fdb6600511091056c"} Dec 03 12:33:34 crc kubenswrapper[5052]: I1203 12:33:34.203009 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-galera-0" event={"ID":"01fb7120-3521-4969-a2c4-b2637e0e2414","Type":"ContainerStarted","Data":"f9456d80cb676adb94c83be749d62019435a6816deaec5cc23878d4aed8c335f"} Dec 03 12:33:34 crc kubenswrapper[5052]: I1203 12:33:34.210333 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/1cc13873-68c8-4c70-89dc-caefaeb40e25-galera-tls-certs\") pod \"openstack-cell1-galera-0\" (UID: \"1cc13873-68c8-4c70-89dc-caefaeb40e25\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:33:34 crc kubenswrapper[5052]: I1203 12:33:34.210413 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/1cc13873-68c8-4c70-89dc-caefaeb40e25-config-data-default\") pod \"openstack-cell1-galera-0\" (UID: \"1cc13873-68c8-4c70-89dc-caefaeb40e25\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:33:34 crc kubenswrapper[5052]: I1203 12:33:34.210479 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/1cc13873-68c8-4c70-89dc-caefaeb40e25-config-data-generated\") pod \"openstack-cell1-galera-0\" (UID: \"1cc13873-68c8-4c70-89dc-caefaeb40e25\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:33:34 crc kubenswrapper[5052]: I1203 12:33:34.210531 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wskcq\" (UniqueName: \"kubernetes.io/projected/1cc13873-68c8-4c70-89dc-caefaeb40e25-kube-api-access-wskcq\") pod \"openstack-cell1-galera-0\" (UID: \"1cc13873-68c8-4c70-89dc-caefaeb40e25\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:33:34 crc kubenswrapper[5052]: I1203 12:33:34.210560 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1cc13873-68c8-4c70-89dc-caefaeb40e25-combined-ca-bundle\") pod \"openstack-cell1-galera-0\" (UID: \"1cc13873-68c8-4c70-89dc-caefaeb40e25\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:33:34 crc kubenswrapper[5052]: I1203 12:33:34.210604 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage11-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage11-crc\") pod \"openstack-cell1-galera-0\" (UID: \"1cc13873-68c8-4c70-89dc-caefaeb40e25\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:33:34 crc kubenswrapper[5052]: I1203 12:33:34.210681 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/1cc13873-68c8-4c70-89dc-caefaeb40e25-kolla-config\") pod \"openstack-cell1-galera-0\" (UID: \"1cc13873-68c8-4c70-89dc-caefaeb40e25\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:33:34 crc kubenswrapper[5052]: I1203 12:33:34.210749 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/1cc13873-68c8-4c70-89dc-caefaeb40e25-operator-scripts\") pod \"openstack-cell1-galera-0\" (UID: \"1cc13873-68c8-4c70-89dc-caefaeb40e25\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:33:34 crc kubenswrapper[5052]: I1203 12:33:34.261113 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/memcached-0"] Dec 03 12:33:34 crc kubenswrapper[5052]: I1203 12:33:34.262374 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/memcached-0" Dec 03 12:33:34 crc kubenswrapper[5052]: I1203 12:33:34.271122 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-memcached-svc" Dec 03 12:33:34 crc kubenswrapper[5052]: I1203 12:33:34.271790 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"memcached-config-data" Dec 03 12:33:34 crc kubenswrapper[5052]: I1203 12:33:34.271800 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"memcached-memcached-dockercfg-wq48p" Dec 03 12:33:34 crc kubenswrapper[5052]: I1203 12:33:34.298488 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/memcached-0"] Dec 03 12:33:34 crc kubenswrapper[5052]: I1203 12:33:34.314872 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/1cc13873-68c8-4c70-89dc-caefaeb40e25-config-data-default\") pod \"openstack-cell1-galera-0\" (UID: \"1cc13873-68c8-4c70-89dc-caefaeb40e25\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:33:34 crc kubenswrapper[5052]: I1203 12:33:34.314940 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5bcb6f6e-9a7c-4313-977a-64868597c373-combined-ca-bundle\") pod \"memcached-0\" (UID: \"5bcb6f6e-9a7c-4313-977a-64868597c373\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:33:34 crc kubenswrapper[5052]: I1203 12:33:34.314997 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/1cc13873-68c8-4c70-89dc-caefaeb40e25-config-data-generated\") pod \"openstack-cell1-galera-0\" (UID: \"1cc13873-68c8-4c70-89dc-caefaeb40e25\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:33:34 crc kubenswrapper[5052]: I1203 12:33:34.315046 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"memcached-tls-certs\" (UniqueName: \"kubernetes.io/secret/5bcb6f6e-9a7c-4313-977a-64868597c373-memcached-tls-certs\") pod \"memcached-0\" (UID: \"5bcb6f6e-9a7c-4313-977a-64868597c373\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:33:34 crc kubenswrapper[5052]: I1203 12:33:34.315083 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-wskcq\" (UniqueName: \"kubernetes.io/projected/1cc13873-68c8-4c70-89dc-caefaeb40e25-kube-api-access-wskcq\") pod \"openstack-cell1-galera-0\" (UID: \"1cc13873-68c8-4c70-89dc-caefaeb40e25\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:33:34 crc kubenswrapper[5052]: I1203 12:33:34.315130 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1cc13873-68c8-4c70-89dc-caefaeb40e25-combined-ca-bundle\") pod \"openstack-cell1-galera-0\" (UID: \"1cc13873-68c8-4c70-89dc-caefaeb40e25\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:33:34 crc kubenswrapper[5052]: I1203 12:33:34.315171 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/5bcb6f6e-9a7c-4313-977a-64868597c373-config-data\") pod \"memcached-0\" (UID: \"5bcb6f6e-9a7c-4313-977a-64868597c373\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:33:34 crc kubenswrapper[5052]: I1203 12:33:34.315293 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage11-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage11-crc\") pod \"openstack-cell1-galera-0\" (UID: \"1cc13873-68c8-4c70-89dc-caefaeb40e25\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:33:34 crc kubenswrapper[5052]: I1203 12:33:34.315353 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/5bcb6f6e-9a7c-4313-977a-64868597c373-kolla-config\") pod \"memcached-0\" (UID: \"5bcb6f6e-9a7c-4313-977a-64868597c373\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:33:34 crc kubenswrapper[5052]: I1203 12:33:34.315383 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/1cc13873-68c8-4c70-89dc-caefaeb40e25-kolla-config\") pod \"openstack-cell1-galera-0\" (UID: \"1cc13873-68c8-4c70-89dc-caefaeb40e25\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:33:34 crc kubenswrapper[5052]: I1203 12:33:34.315455 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wkhcj\" (UniqueName: \"kubernetes.io/projected/5bcb6f6e-9a7c-4313-977a-64868597c373-kube-api-access-wkhcj\") pod \"memcached-0\" (UID: \"5bcb6f6e-9a7c-4313-977a-64868597c373\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:33:34 crc kubenswrapper[5052]: I1203 12:33:34.315553 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/1cc13873-68c8-4c70-89dc-caefaeb40e25-operator-scripts\") pod \"openstack-cell1-galera-0\" (UID: \"1cc13873-68c8-4c70-89dc-caefaeb40e25\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:33:34 crc kubenswrapper[5052]: I1203 12:33:34.315591 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/1cc13873-68c8-4c70-89dc-caefaeb40e25-galera-tls-certs\") pod \"openstack-cell1-galera-0\" (UID: \"1cc13873-68c8-4c70-89dc-caefaeb40e25\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:33:34 crc kubenswrapper[5052]: I1203 12:33:34.317359 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/1cc13873-68c8-4c70-89dc-caefaeb40e25-config-data-generated\") pod \"openstack-cell1-galera-0\" (UID: \"1cc13873-68c8-4c70-89dc-caefaeb40e25\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:33:34 crc kubenswrapper[5052]: I1203 12:33:34.317763 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/1cc13873-68c8-4c70-89dc-caefaeb40e25-config-data-default\") pod \"openstack-cell1-galera-0\" (UID: \"1cc13873-68c8-4c70-89dc-caefaeb40e25\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:33:34 crc kubenswrapper[5052]: I1203 12:33:34.317801 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage11-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage11-crc\") pod \"openstack-cell1-galera-0\" (UID: \"1cc13873-68c8-4c70-89dc-caefaeb40e25\") device mount path \"/mnt/openstack/pv11\"" pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:33:34 crc kubenswrapper[5052]: I1203 12:33:34.318683 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/1cc13873-68c8-4c70-89dc-caefaeb40e25-kolla-config\") pod \"openstack-cell1-galera-0\" (UID: \"1cc13873-68c8-4c70-89dc-caefaeb40e25\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:33:34 crc kubenswrapper[5052]: I1203 12:33:34.319318 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/1cc13873-68c8-4c70-89dc-caefaeb40e25-operator-scripts\") pod \"openstack-cell1-galera-0\" (UID: \"1cc13873-68c8-4c70-89dc-caefaeb40e25\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:33:34 crc kubenswrapper[5052]: I1203 12:33:34.322860 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1cc13873-68c8-4c70-89dc-caefaeb40e25-combined-ca-bundle\") pod \"openstack-cell1-galera-0\" (UID: \"1cc13873-68c8-4c70-89dc-caefaeb40e25\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:33:34 crc kubenswrapper[5052]: I1203 12:33:34.338459 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-wskcq\" (UniqueName: \"kubernetes.io/projected/1cc13873-68c8-4c70-89dc-caefaeb40e25-kube-api-access-wskcq\") pod \"openstack-cell1-galera-0\" (UID: \"1cc13873-68c8-4c70-89dc-caefaeb40e25\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:33:34 crc kubenswrapper[5052]: I1203 12:33:34.341204 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/1cc13873-68c8-4c70-89dc-caefaeb40e25-galera-tls-certs\") pod \"openstack-cell1-galera-0\" (UID: \"1cc13873-68c8-4c70-89dc-caefaeb40e25\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:33:34 crc kubenswrapper[5052]: I1203 12:33:34.351087 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage11-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage11-crc\") pod \"openstack-cell1-galera-0\" (UID: \"1cc13873-68c8-4c70-89dc-caefaeb40e25\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:33:34 crc kubenswrapper[5052]: I1203 12:33:34.416985 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-wkhcj\" (UniqueName: \"kubernetes.io/projected/5bcb6f6e-9a7c-4313-977a-64868597c373-kube-api-access-wkhcj\") pod \"memcached-0\" (UID: \"5bcb6f6e-9a7c-4313-977a-64868597c373\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:33:34 crc kubenswrapper[5052]: I1203 12:33:34.417137 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5bcb6f6e-9a7c-4313-977a-64868597c373-combined-ca-bundle\") pod \"memcached-0\" (UID: \"5bcb6f6e-9a7c-4313-977a-64868597c373\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:33:34 crc kubenswrapper[5052]: I1203 12:33:34.417232 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"memcached-tls-certs\" (UniqueName: \"kubernetes.io/secret/5bcb6f6e-9a7c-4313-977a-64868597c373-memcached-tls-certs\") pod \"memcached-0\" (UID: \"5bcb6f6e-9a7c-4313-977a-64868597c373\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:33:34 crc kubenswrapper[5052]: I1203 12:33:34.417281 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/5bcb6f6e-9a7c-4313-977a-64868597c373-config-data\") pod \"memcached-0\" (UID: \"5bcb6f6e-9a7c-4313-977a-64868597c373\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:33:34 crc kubenswrapper[5052]: I1203 12:33:34.417370 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/5bcb6f6e-9a7c-4313-977a-64868597c373-kolla-config\") pod \"memcached-0\" (UID: \"5bcb6f6e-9a7c-4313-977a-64868597c373\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:33:34 crc kubenswrapper[5052]: I1203 12:33:34.418280 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/5bcb6f6e-9a7c-4313-977a-64868597c373-kolla-config\") pod \"memcached-0\" (UID: \"5bcb6f6e-9a7c-4313-977a-64868597c373\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:33:34 crc kubenswrapper[5052]: I1203 12:33:34.418784 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/5bcb6f6e-9a7c-4313-977a-64868597c373-config-data\") pod \"memcached-0\" (UID: \"5bcb6f6e-9a7c-4313-977a-64868597c373\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:33:34 crc kubenswrapper[5052]: I1203 12:33:34.421027 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5bcb6f6e-9a7c-4313-977a-64868597c373-combined-ca-bundle\") pod \"memcached-0\" (UID: \"5bcb6f6e-9a7c-4313-977a-64868597c373\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:33:34 crc kubenswrapper[5052]: I1203 12:33:34.421223 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"memcached-tls-certs\" (UniqueName: \"kubernetes.io/secret/5bcb6f6e-9a7c-4313-977a-64868597c373-memcached-tls-certs\") pod \"memcached-0\" (UID: \"5bcb6f6e-9a7c-4313-977a-64868597c373\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:33:34 crc kubenswrapper[5052]: I1203 12:33:34.431856 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:33:34 crc kubenswrapper[5052]: I1203 12:33:34.436270 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-wkhcj\" (UniqueName: \"kubernetes.io/projected/5bcb6f6e-9a7c-4313-977a-64868597c373-kube-api-access-wkhcj\") pod \"memcached-0\" (UID: \"5bcb6f6e-9a7c-4313-977a-64868597c373\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:33:34 crc kubenswrapper[5052]: I1203 12:33:34.598131 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/memcached-0" Dec 03 12:33:34 crc kubenswrapper[5052]: I1203 12:33:34.980398 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/openstack-cell1-galera-0"] Dec 03 12:33:35 crc kubenswrapper[5052]: I1203 12:33:35.121759 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/memcached-0"] Dec 03 12:33:35 crc kubenswrapper[5052]: I1203 12:33:35.209505 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/memcached-0" event={"ID":"5bcb6f6e-9a7c-4313-977a-64868597c373","Type":"ContainerStarted","Data":"d5a768662ffa04134b167c3280d10cff7f65b675ac9e11faec0729044b46979c"} Dec 03 12:33:35 crc kubenswrapper[5052]: I1203 12:33:35.211837 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-cell1-galera-0" event={"ID":"1cc13873-68c8-4c70-89dc-caefaeb40e25","Type":"ContainerStarted","Data":"940708945e467089d948362e1fc3b34404c0c0c4806d349291101b66943d6391"} Dec 03 12:33:35 crc kubenswrapper[5052]: I1203 12:33:35.211879 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-cell1-galera-0" event={"ID":"1cc13873-68c8-4c70-89dc-caefaeb40e25","Type":"ContainerStarted","Data":"26dfde733a8efe039801ca8fa714dd066e4aef78697aadfba4be17b4393e5118"} Dec 03 12:33:36 crc kubenswrapper[5052]: I1203 12:33:36.132373 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/kube-state-metrics-0"] Dec 03 12:33:36 crc kubenswrapper[5052]: I1203 12:33:36.134318 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:33:36 crc kubenswrapper[5052]: I1203 12:33:36.137011 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"telemetry-ceilometer-dockercfg-k2d6g" Dec 03 12:33:36 crc kubenswrapper[5052]: I1203 12:33:36.143074 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/kube-state-metrics-0"] Dec 03 12:33:36 crc kubenswrapper[5052]: I1203 12:33:36.219538 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/memcached-0" event={"ID":"5bcb6f6e-9a7c-4313-977a-64868597c373","Type":"ContainerStarted","Data":"8a9cce80bd63ba1cdb7efaac129f52c4051a34e072ec1a2b01113e833ffc26df"} Dec 03 12:33:36 crc kubenswrapper[5052]: I1203 12:33:36.241090 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/memcached-0" podStartSLOduration=2.241069789 podStartE2EDuration="2.241069789s" podCreationTimestamp="2025-12-03 12:33:34 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:33:36.233611586 +0000 UTC m=+4406.821853350" watchObservedRunningTime="2025-12-03 12:33:36.241069789 +0000 UTC m=+4406.829311553" Dec 03 12:33:36 crc kubenswrapper[5052]: I1203 12:33:36.250218 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2vbwg\" (UniqueName: \"kubernetes.io/projected/5c9ee44f-4958-4cc9-a8a4-e5c023b65baf-kube-api-access-2vbwg\") pod \"kube-state-metrics-0\" (UID: \"5c9ee44f-4958-4cc9-a8a4-e5c023b65baf\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:33:36 crc kubenswrapper[5052]: I1203 12:33:36.352695 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-2vbwg\" (UniqueName: \"kubernetes.io/projected/5c9ee44f-4958-4cc9-a8a4-e5c023b65baf-kube-api-access-2vbwg\") pod \"kube-state-metrics-0\" (UID: \"5c9ee44f-4958-4cc9-a8a4-e5c023b65baf\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:33:36 crc kubenswrapper[5052]: I1203 12:33:36.378208 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-2vbwg\" (UniqueName: \"kubernetes.io/projected/5c9ee44f-4958-4cc9-a8a4-e5c023b65baf-kube-api-access-2vbwg\") pod \"kube-state-metrics-0\" (UID: \"5c9ee44f-4958-4cc9-a8a4-e5c023b65baf\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:33:36 crc kubenswrapper[5052]: I1203 12:33:36.454074 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:33:36 crc kubenswrapper[5052]: W1203 12:33:36.894460 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5c9ee44f_4958_4cc9_a8a4_e5c023b65baf.slice/crio-309901a267add4c45800bd5475e7376f43e64ee259ae43f27b80c7fdea6e239c WatchSource:0}: Error finding container 309901a267add4c45800bd5475e7376f43e64ee259ae43f27b80c7fdea6e239c: Status 404 returned error can't find the container with id 309901a267add4c45800bd5475e7376f43e64ee259ae43f27b80c7fdea6e239c Dec 03 12:33:36 crc kubenswrapper[5052]: I1203 12:33:36.902997 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/kube-state-metrics-0"] Dec 03 12:33:37 crc kubenswrapper[5052]: I1203 12:33:37.230333 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/kube-state-metrics-0" event={"ID":"5c9ee44f-4958-4cc9-a8a4-e5c023b65baf","Type":"ContainerStarted","Data":"309901a267add4c45800bd5475e7376f43e64ee259ae43f27b80c7fdea6e239c"} Dec 03 12:33:37 crc kubenswrapper[5052]: I1203 12:33:37.230482 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/memcached-0" Dec 03 12:33:38 crc kubenswrapper[5052]: I1203 12:33:38.241182 5052 generic.go:334] "Generic (PLEG): container finished" podID="01fb7120-3521-4969-a2c4-b2637e0e2414" containerID="7cdb8d3632d0751bac948ca612e51336988e363391bc868fdb6600511091056c" exitCode=0 Dec 03 12:33:38 crc kubenswrapper[5052]: I1203 12:33:38.241270 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-galera-0" event={"ID":"01fb7120-3521-4969-a2c4-b2637e0e2414","Type":"ContainerDied","Data":"7cdb8d3632d0751bac948ca612e51336988e363391bc868fdb6600511091056c"} Dec 03 12:33:38 crc kubenswrapper[5052]: I1203 12:33:38.243956 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/kube-state-metrics-0" event={"ID":"5c9ee44f-4958-4cc9-a8a4-e5c023b65baf","Type":"ContainerStarted","Data":"079d43fdf2f0ff185ac5f2d5c966c26353a478c914bb29cf74d63cf1f61876bd"} Dec 03 12:33:38 crc kubenswrapper[5052]: I1203 12:33:38.244102 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:33:38 crc kubenswrapper[5052]: I1203 12:33:38.287535 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/kube-state-metrics-0" podStartSLOduration=1.933484475 podStartE2EDuration="2.287497399s" podCreationTimestamp="2025-12-03 12:33:36 +0000 UTC" firstStartedPulling="2025-12-03 12:33:36.896718923 +0000 UTC m=+4407.484960687" lastFinishedPulling="2025-12-03 12:33:37.250731847 +0000 UTC m=+4407.838973611" observedRunningTime="2025-12-03 12:33:38.279154381 +0000 UTC m=+4408.867396145" watchObservedRunningTime="2025-12-03 12:33:38.287497399 +0000 UTC m=+4408.875739173" Dec 03 12:33:39 crc kubenswrapper[5052]: I1203 12:33:39.254790 5052 generic.go:334] "Generic (PLEG): container finished" podID="1cc13873-68c8-4c70-89dc-caefaeb40e25" containerID="940708945e467089d948362e1fc3b34404c0c0c4806d349291101b66943d6391" exitCode=0 Dec 03 12:33:39 crc kubenswrapper[5052]: I1203 12:33:39.254928 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-cell1-galera-0" event={"ID":"1cc13873-68c8-4c70-89dc-caefaeb40e25","Type":"ContainerDied","Data":"940708945e467089d948362e1fc3b34404c0c0c4806d349291101b66943d6391"} Dec 03 12:33:39 crc kubenswrapper[5052]: I1203 12:33:39.258671 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-galera-0" event={"ID":"01fb7120-3521-4969-a2c4-b2637e0e2414","Type":"ContainerStarted","Data":"71d046155554dd75f42c3796f0e8202ff1e0f2c95466f0def0740b661fba207f"} Dec 03 12:33:39 crc kubenswrapper[5052]: I1203 12:33:39.314855 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/openstack-galera-0" podStartSLOduration=8.314836431 podStartE2EDuration="8.314836431s" podCreationTimestamp="2025-12-03 12:33:31 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:33:39.309712675 +0000 UTC m=+4409.897954449" watchObservedRunningTime="2025-12-03 12:33:39.314836431 +0000 UTC m=+4409.903078195" Dec 03 12:33:39 crc kubenswrapper[5052]: I1203 12:33:39.582980 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ovsdbserver-nb-0"] Dec 03 12:33:39 crc kubenswrapper[5052]: I1203 12:33:39.585438 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:33:39 crc kubenswrapper[5052]: I1203 12:33:39.595181 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"ovndbcluster-nb-scripts" Dec 03 12:33:39 crc kubenswrapper[5052]: I1203 12:33:39.595544 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"ovndbcluster-nb-config" Dec 03 12:33:39 crc kubenswrapper[5052]: I1203 12:33:39.595691 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ovncluster-ovndbcluster-nb-dockercfg-xg67d" Dec 03 12:33:39 crc kubenswrapper[5052]: I1203 12:33:39.595839 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-ovndbcluster-nb-ovndbs" Dec 03 12:33:39 crc kubenswrapper[5052]: I1203 12:33:39.597823 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-ovn-metrics" Dec 03 12:33:39 crc kubenswrapper[5052]: I1203 12:33:39.602192 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-nb-0"] Dec 03 12:33:39 crc kubenswrapper[5052]: I1203 12:33:39.716935 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3da53770-41d7-4590-9278-21c1a82203b1-combined-ca-bundle\") pod \"ovsdbserver-nb-0\" (UID: \"3da53770-41d7-4590-9278-21c1a82203b1\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:33:39 crc kubenswrapper[5052]: I1203 12:33:39.717243 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/3da53770-41d7-4590-9278-21c1a82203b1-metrics-certs-tls-certs\") pod \"ovsdbserver-nb-0\" (UID: \"3da53770-41d7-4590-9278-21c1a82203b1\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:33:39 crc kubenswrapper[5052]: I1203 12:33:39.717371 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bqbwq\" (UniqueName: \"kubernetes.io/projected/3da53770-41d7-4590-9278-21c1a82203b1-kube-api-access-bqbwq\") pod \"ovsdbserver-nb-0\" (UID: \"3da53770-41d7-4590-9278-21c1a82203b1\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:33:39 crc kubenswrapper[5052]: I1203 12:33:39.717487 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage01-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage01-crc\") pod \"ovsdbserver-nb-0\" (UID: \"3da53770-41d7-4590-9278-21c1a82203b1\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:33:39 crc kubenswrapper[5052]: I1203 12:33:39.717610 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3da53770-41d7-4590-9278-21c1a82203b1-config\") pod \"ovsdbserver-nb-0\" (UID: \"3da53770-41d7-4590-9278-21c1a82203b1\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:33:39 crc kubenswrapper[5052]: I1203 12:33:39.717770 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovsdbserver-nb-tls-certs\" (UniqueName: \"kubernetes.io/secret/3da53770-41d7-4590-9278-21c1a82203b1-ovsdbserver-nb-tls-certs\") pod \"ovsdbserver-nb-0\" (UID: \"3da53770-41d7-4590-9278-21c1a82203b1\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:33:39 crc kubenswrapper[5052]: I1203 12:33:39.717955 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/3da53770-41d7-4590-9278-21c1a82203b1-scripts\") pod \"ovsdbserver-nb-0\" (UID: \"3da53770-41d7-4590-9278-21c1a82203b1\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:33:39 crc kubenswrapper[5052]: I1203 12:33:39.718128 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/3da53770-41d7-4590-9278-21c1a82203b1-ovsdb-rundir\") pod \"ovsdbserver-nb-0\" (UID: \"3da53770-41d7-4590-9278-21c1a82203b1\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:33:39 crc kubenswrapper[5052]: I1203 12:33:39.820024 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3da53770-41d7-4590-9278-21c1a82203b1-config\") pod \"ovsdbserver-nb-0\" (UID: \"3da53770-41d7-4590-9278-21c1a82203b1\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:33:39 crc kubenswrapper[5052]: I1203 12:33:39.820537 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovsdbserver-nb-tls-certs\" (UniqueName: \"kubernetes.io/secret/3da53770-41d7-4590-9278-21c1a82203b1-ovsdbserver-nb-tls-certs\") pod \"ovsdbserver-nb-0\" (UID: \"3da53770-41d7-4590-9278-21c1a82203b1\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:33:39 crc kubenswrapper[5052]: I1203 12:33:39.821315 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3da53770-41d7-4590-9278-21c1a82203b1-config\") pod \"ovsdbserver-nb-0\" (UID: \"3da53770-41d7-4590-9278-21c1a82203b1\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:33:39 crc kubenswrapper[5052]: I1203 12:33:39.821664 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/3da53770-41d7-4590-9278-21c1a82203b1-scripts\") pod \"ovsdbserver-nb-0\" (UID: \"3da53770-41d7-4590-9278-21c1a82203b1\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:33:39 crc kubenswrapper[5052]: I1203 12:33:39.821961 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/3da53770-41d7-4590-9278-21c1a82203b1-ovsdb-rundir\") pod \"ovsdbserver-nb-0\" (UID: \"3da53770-41d7-4590-9278-21c1a82203b1\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:33:39 crc kubenswrapper[5052]: I1203 12:33:39.822328 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3da53770-41d7-4590-9278-21c1a82203b1-combined-ca-bundle\") pod \"ovsdbserver-nb-0\" (UID: \"3da53770-41d7-4590-9278-21c1a82203b1\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:33:39 crc kubenswrapper[5052]: I1203 12:33:39.822503 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/3da53770-41d7-4590-9278-21c1a82203b1-metrics-certs-tls-certs\") pod \"ovsdbserver-nb-0\" (UID: \"3da53770-41d7-4590-9278-21c1a82203b1\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:33:39 crc kubenswrapper[5052]: I1203 12:33:39.822643 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-bqbwq\" (UniqueName: \"kubernetes.io/projected/3da53770-41d7-4590-9278-21c1a82203b1-kube-api-access-bqbwq\") pod \"ovsdbserver-nb-0\" (UID: \"3da53770-41d7-4590-9278-21c1a82203b1\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:33:39 crc kubenswrapper[5052]: I1203 12:33:39.822699 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage01-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage01-crc\") pod \"ovsdbserver-nb-0\" (UID: \"3da53770-41d7-4590-9278-21c1a82203b1\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:33:39 crc kubenswrapper[5052]: I1203 12:33:39.822711 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/3da53770-41d7-4590-9278-21c1a82203b1-scripts\") pod \"ovsdbserver-nb-0\" (UID: \"3da53770-41d7-4590-9278-21c1a82203b1\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:33:39 crc kubenswrapper[5052]: I1203 12:33:39.823131 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage01-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage01-crc\") pod \"ovsdbserver-nb-0\" (UID: \"3da53770-41d7-4590-9278-21c1a82203b1\") device mount path \"/mnt/openstack/pv01\"" pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:33:39 crc kubenswrapper[5052]: I1203 12:33:39.825029 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/3da53770-41d7-4590-9278-21c1a82203b1-ovsdb-rundir\") pod \"ovsdbserver-nb-0\" (UID: \"3da53770-41d7-4590-9278-21c1a82203b1\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:33:39 crc kubenswrapper[5052]: I1203 12:33:39.826531 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovsdbserver-nb-tls-certs\" (UniqueName: \"kubernetes.io/secret/3da53770-41d7-4590-9278-21c1a82203b1-ovsdbserver-nb-tls-certs\") pod \"ovsdbserver-nb-0\" (UID: \"3da53770-41d7-4590-9278-21c1a82203b1\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:33:39 crc kubenswrapper[5052]: I1203 12:33:39.828891 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3da53770-41d7-4590-9278-21c1a82203b1-combined-ca-bundle\") pod \"ovsdbserver-nb-0\" (UID: \"3da53770-41d7-4590-9278-21c1a82203b1\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:33:39 crc kubenswrapper[5052]: I1203 12:33:39.838866 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/3da53770-41d7-4590-9278-21c1a82203b1-metrics-certs-tls-certs\") pod \"ovsdbserver-nb-0\" (UID: \"3da53770-41d7-4590-9278-21c1a82203b1\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:33:39 crc kubenswrapper[5052]: I1203 12:33:39.849016 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage01-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage01-crc\") pod \"ovsdbserver-nb-0\" (UID: \"3da53770-41d7-4590-9278-21c1a82203b1\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:33:39 crc kubenswrapper[5052]: I1203 12:33:39.851582 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-bqbwq\" (UniqueName: \"kubernetes.io/projected/3da53770-41d7-4590-9278-21c1a82203b1-kube-api-access-bqbwq\") pod \"ovsdbserver-nb-0\" (UID: \"3da53770-41d7-4590-9278-21c1a82203b1\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:33:39 crc kubenswrapper[5052]: I1203 12:33:39.912701 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:33:40 crc kubenswrapper[5052]: I1203 12:33:40.269884 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-cell1-galera-0" event={"ID":"1cc13873-68c8-4c70-89dc-caefaeb40e25","Type":"ContainerStarted","Data":"34697f0f6773a06387fe7e2f667823c92555f8d2c3bcc6f8942a7d5f2769f8ed"} Dec 03 12:33:40 crc kubenswrapper[5052]: I1203 12:33:40.300410 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/openstack-cell1-galera-0" podStartSLOduration=7.300388241 podStartE2EDuration="7.300388241s" podCreationTimestamp="2025-12-03 12:33:33 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:33:40.294681598 +0000 UTC m=+4410.882923392" watchObservedRunningTime="2025-12-03 12:33:40.300388241 +0000 UTC m=+4410.888630005" Dec 03 12:33:40 crc kubenswrapper[5052]: W1203 12:33:40.392750 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3da53770_41d7_4590_9278_21c1a82203b1.slice/crio-58f5b345e908a96e302d405c9daeb577751c8a424016870cafde74b07319caa5 WatchSource:0}: Error finding container 58f5b345e908a96e302d405c9daeb577751c8a424016870cafde74b07319caa5: Status 404 returned error can't find the container with id 58f5b345e908a96e302d405c9daeb577751c8a424016870cafde74b07319caa5 Dec 03 12:33:40 crc kubenswrapper[5052]: I1203 12:33:40.392796 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-nb-0"] Dec 03 12:33:41 crc kubenswrapper[5052]: I1203 12:33:41.279488 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-nb-0" event={"ID":"3da53770-41d7-4590-9278-21c1a82203b1","Type":"ContainerStarted","Data":"5aa617ab3930326bfed819c0605942aa0091d0586b0d47d203001eb7f25457f5"} Dec 03 12:33:41 crc kubenswrapper[5052]: I1203 12:33:41.279870 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-nb-0" event={"ID":"3da53770-41d7-4590-9278-21c1a82203b1","Type":"ContainerStarted","Data":"44b1bf53e148e3c548b1c0d9c822dcf34d9f2cab27ee6241c47780a6ed9ccca8"} Dec 03 12:33:41 crc kubenswrapper[5052]: I1203 12:33:41.279887 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-nb-0" event={"ID":"3da53770-41d7-4590-9278-21c1a82203b1","Type":"ContainerStarted","Data":"58f5b345e908a96e302d405c9daeb577751c8a424016870cafde74b07319caa5"} Dec 03 12:33:41 crc kubenswrapper[5052]: I1203 12:33:41.297611 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/ovsdbserver-nb-0" podStartSLOduration=3.297594363 podStartE2EDuration="3.297594363s" podCreationTimestamp="2025-12-03 12:33:38 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:33:41.295954187 +0000 UTC m=+4411.884195961" watchObservedRunningTime="2025-12-03 12:33:41.297594363 +0000 UTC m=+4411.885836127" Dec 03 12:33:42 crc kubenswrapper[5052]: I1203 12:33:42.325225 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ovsdbserver-sb-0"] Dec 03 12:33:42 crc kubenswrapper[5052]: I1203 12:33:42.326964 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:33:42 crc kubenswrapper[5052]: I1203 12:33:42.329237 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-ovndbcluster-sb-ovndbs" Dec 03 12:33:42 crc kubenswrapper[5052]: I1203 12:33:42.329375 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"ovndbcluster-sb-scripts" Dec 03 12:33:42 crc kubenswrapper[5052]: I1203 12:33:42.329608 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ovncluster-ovndbcluster-sb-dockercfg-q8jfj" Dec 03 12:33:42 crc kubenswrapper[5052]: I1203 12:33:42.330910 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"ovndbcluster-sb-config" Dec 03 12:33:42 crc kubenswrapper[5052]: I1203 12:33:42.343663 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-sb-0"] Dec 03 12:33:42 crc kubenswrapper[5052]: I1203 12:33:42.473355 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/143d0ab6-f833-41a9-aacd-34d20b35f8ed-ovsdb-rundir\") pod \"ovsdbserver-sb-0\" (UID: \"143d0ab6-f833-41a9-aacd-34d20b35f8ed\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:33:42 crc kubenswrapper[5052]: I1203 12:33:42.473437 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovsdbserver-sb-tls-certs\" (UniqueName: \"kubernetes.io/secret/143d0ab6-f833-41a9-aacd-34d20b35f8ed-ovsdbserver-sb-tls-certs\") pod \"ovsdbserver-sb-0\" (UID: \"143d0ab6-f833-41a9-aacd-34d20b35f8ed\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:33:42 crc kubenswrapper[5052]: I1203 12:33:42.473501 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sppmc\" (UniqueName: \"kubernetes.io/projected/143d0ab6-f833-41a9-aacd-34d20b35f8ed-kube-api-access-sppmc\") pod \"ovsdbserver-sb-0\" (UID: \"143d0ab6-f833-41a9-aacd-34d20b35f8ed\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:33:42 crc kubenswrapper[5052]: I1203 12:33:42.473552 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage09-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage09-crc\") pod \"ovsdbserver-sb-0\" (UID: \"143d0ab6-f833-41a9-aacd-34d20b35f8ed\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:33:42 crc kubenswrapper[5052]: I1203 12:33:42.473571 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/143d0ab6-f833-41a9-aacd-34d20b35f8ed-metrics-certs-tls-certs\") pod \"ovsdbserver-sb-0\" (UID: \"143d0ab6-f833-41a9-aacd-34d20b35f8ed\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:33:42 crc kubenswrapper[5052]: I1203 12:33:42.473594 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/143d0ab6-f833-41a9-aacd-34d20b35f8ed-combined-ca-bundle\") pod \"ovsdbserver-sb-0\" (UID: \"143d0ab6-f833-41a9-aacd-34d20b35f8ed\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:33:42 crc kubenswrapper[5052]: I1203 12:33:42.473632 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/143d0ab6-f833-41a9-aacd-34d20b35f8ed-scripts\") pod \"ovsdbserver-sb-0\" (UID: \"143d0ab6-f833-41a9-aacd-34d20b35f8ed\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:33:42 crc kubenswrapper[5052]: I1203 12:33:42.473654 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/143d0ab6-f833-41a9-aacd-34d20b35f8ed-config\") pod \"ovsdbserver-sb-0\" (UID: \"143d0ab6-f833-41a9-aacd-34d20b35f8ed\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:33:42 crc kubenswrapper[5052]: I1203 12:33:42.575225 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/143d0ab6-f833-41a9-aacd-34d20b35f8ed-ovsdb-rundir\") pod \"ovsdbserver-sb-0\" (UID: \"143d0ab6-f833-41a9-aacd-34d20b35f8ed\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:33:42 crc kubenswrapper[5052]: I1203 12:33:42.575288 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovsdbserver-sb-tls-certs\" (UniqueName: \"kubernetes.io/secret/143d0ab6-f833-41a9-aacd-34d20b35f8ed-ovsdbserver-sb-tls-certs\") pod \"ovsdbserver-sb-0\" (UID: \"143d0ab6-f833-41a9-aacd-34d20b35f8ed\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:33:42 crc kubenswrapper[5052]: I1203 12:33:42.575346 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-sppmc\" (UniqueName: \"kubernetes.io/projected/143d0ab6-f833-41a9-aacd-34d20b35f8ed-kube-api-access-sppmc\") pod \"ovsdbserver-sb-0\" (UID: \"143d0ab6-f833-41a9-aacd-34d20b35f8ed\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:33:42 crc kubenswrapper[5052]: I1203 12:33:42.575372 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/143d0ab6-f833-41a9-aacd-34d20b35f8ed-metrics-certs-tls-certs\") pod \"ovsdbserver-sb-0\" (UID: \"143d0ab6-f833-41a9-aacd-34d20b35f8ed\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:33:42 crc kubenswrapper[5052]: I1203 12:33:42.575397 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage09-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage09-crc\") pod \"ovsdbserver-sb-0\" (UID: \"143d0ab6-f833-41a9-aacd-34d20b35f8ed\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:33:42 crc kubenswrapper[5052]: I1203 12:33:42.575423 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/143d0ab6-f833-41a9-aacd-34d20b35f8ed-combined-ca-bundle\") pod \"ovsdbserver-sb-0\" (UID: \"143d0ab6-f833-41a9-aacd-34d20b35f8ed\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:33:42 crc kubenswrapper[5052]: I1203 12:33:42.575457 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/143d0ab6-f833-41a9-aacd-34d20b35f8ed-config\") pod \"ovsdbserver-sb-0\" (UID: \"143d0ab6-f833-41a9-aacd-34d20b35f8ed\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:33:42 crc kubenswrapper[5052]: I1203 12:33:42.575481 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/143d0ab6-f833-41a9-aacd-34d20b35f8ed-scripts\") pod \"ovsdbserver-sb-0\" (UID: \"143d0ab6-f833-41a9-aacd-34d20b35f8ed\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:33:42 crc kubenswrapper[5052]: I1203 12:33:42.575800 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage09-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage09-crc\") pod \"ovsdbserver-sb-0\" (UID: \"143d0ab6-f833-41a9-aacd-34d20b35f8ed\") device mount path \"/mnt/openstack/pv09\"" pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:33:42 crc kubenswrapper[5052]: I1203 12:33:42.576371 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/143d0ab6-f833-41a9-aacd-34d20b35f8ed-ovsdb-rundir\") pod \"ovsdbserver-sb-0\" (UID: \"143d0ab6-f833-41a9-aacd-34d20b35f8ed\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:33:42 crc kubenswrapper[5052]: I1203 12:33:42.576799 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/143d0ab6-f833-41a9-aacd-34d20b35f8ed-config\") pod \"ovsdbserver-sb-0\" (UID: \"143d0ab6-f833-41a9-aacd-34d20b35f8ed\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:33:42 crc kubenswrapper[5052]: I1203 12:33:42.578184 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/143d0ab6-f833-41a9-aacd-34d20b35f8ed-scripts\") pod \"ovsdbserver-sb-0\" (UID: \"143d0ab6-f833-41a9-aacd-34d20b35f8ed\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:33:42 crc kubenswrapper[5052]: I1203 12:33:42.625589 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/143d0ab6-f833-41a9-aacd-34d20b35f8ed-metrics-certs-tls-certs\") pod \"ovsdbserver-sb-0\" (UID: \"143d0ab6-f833-41a9-aacd-34d20b35f8ed\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:33:42 crc kubenswrapper[5052]: I1203 12:33:42.625956 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/143d0ab6-f833-41a9-aacd-34d20b35f8ed-combined-ca-bundle\") pod \"ovsdbserver-sb-0\" (UID: \"143d0ab6-f833-41a9-aacd-34d20b35f8ed\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:33:42 crc kubenswrapper[5052]: I1203 12:33:42.625969 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovsdbserver-sb-tls-certs\" (UniqueName: \"kubernetes.io/secret/143d0ab6-f833-41a9-aacd-34d20b35f8ed-ovsdbserver-sb-tls-certs\") pod \"ovsdbserver-sb-0\" (UID: \"143d0ab6-f833-41a9-aacd-34d20b35f8ed\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:33:42 crc kubenswrapper[5052]: I1203 12:33:42.627012 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-sppmc\" (UniqueName: \"kubernetes.io/projected/143d0ab6-f833-41a9-aacd-34d20b35f8ed-kube-api-access-sppmc\") pod \"ovsdbserver-sb-0\" (UID: \"143d0ab6-f833-41a9-aacd-34d20b35f8ed\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:33:42 crc kubenswrapper[5052]: E1203 12:33:42.739845 5052 upgradeaware.go:427] Error proxying data from client to backend: readfrom tcp 38.129.56.252:51142->38.129.56.252:34849: write tcp 38.129.56.252:51142->38.129.56.252:34849: write: broken pipe Dec 03 12:33:42 crc kubenswrapper[5052]: I1203 12:33:42.844173 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage09-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage09-crc\") pod \"ovsdbserver-sb-0\" (UID: \"143d0ab6-f833-41a9-aacd-34d20b35f8ed\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:33:42 crc kubenswrapper[5052]: I1203 12:33:42.913842 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:33:42 crc kubenswrapper[5052]: I1203 12:33:42.947550 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:33:43 crc kubenswrapper[5052]: I1203 12:33:43.095293 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:33:43 crc kubenswrapper[5052]: I1203 12:33:43.095704 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:33:43 crc kubenswrapper[5052]: I1203 12:33:43.475776 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-sb-0"] Dec 03 12:33:43 crc kubenswrapper[5052]: I1203 12:33:43.673810 5052 scope.go:117] "RemoveContainer" containerID="e6edaf188f85ea4f11f487f11cc489c700f25fb42b5ee269f4f496f1e127d772" Dec 03 12:33:43 crc kubenswrapper[5052]: E1203 12:33:43.674467 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 12:33:43 crc kubenswrapper[5052]: I1203 12:33:43.795930 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:33:43 crc kubenswrapper[5052]: I1203 12:33:43.881700 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:33:44 crc kubenswrapper[5052]: I1203 12:33:44.305942 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-sb-0" event={"ID":"143d0ab6-f833-41a9-aacd-34d20b35f8ed","Type":"ContainerStarted","Data":"66eb2dc35973bd557c9042a9e0b7776f13f8d8df1e970eff2cb7d9509c8c414a"} Dec 03 12:33:44 crc kubenswrapper[5052]: I1203 12:33:44.306290 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-sb-0" event={"ID":"143d0ab6-f833-41a9-aacd-34d20b35f8ed","Type":"ContainerStarted","Data":"f9ed4255fa67b2a8f659f0dd13829e2e2da10d2d67a6417060a75e2605964162"} Dec 03 12:33:44 crc kubenswrapper[5052]: I1203 12:33:44.306306 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-sb-0" event={"ID":"143d0ab6-f833-41a9-aacd-34d20b35f8ed","Type":"ContainerStarted","Data":"6c526ec1bf96c63f15a87a02a06a0464499e31aef3337c2671be8c4343b95929"} Dec 03 12:33:44 crc kubenswrapper[5052]: I1203 12:33:44.333187 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/ovsdbserver-sb-0" podStartSLOduration=3.333163615 podStartE2EDuration="3.333163615s" podCreationTimestamp="2025-12-03 12:33:41 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:33:44.325233129 +0000 UTC m=+4414.913474903" watchObservedRunningTime="2025-12-03 12:33:44.333163615 +0000 UTC m=+4414.921405379" Dec 03 12:33:44 crc kubenswrapper[5052]: I1203 12:33:44.432388 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:33:44 crc kubenswrapper[5052]: I1203 12:33:44.432838 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:33:44 crc kubenswrapper[5052]: I1203 12:33:44.571628 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/placement-db-create-mzssh"] Dec 03 12:33:44 crc kubenswrapper[5052]: I1203 12:33:44.572883 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-db-create-mzssh" Dec 03 12:33:44 crc kubenswrapper[5052]: I1203 12:33:44.582492 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/placement-db-create-mzssh"] Dec 03 12:33:44 crc kubenswrapper[5052]: I1203 12:33:44.601639 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/memcached-0" Dec 03 12:33:44 crc kubenswrapper[5052]: I1203 12:33:44.627161 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jjzvk\" (UniqueName: \"kubernetes.io/projected/fc2e5ba2-92e9-4fbe-80f6-d977d0e1d11b-kube-api-access-jjzvk\") pod \"placement-db-create-mzssh\" (UID: \"fc2e5ba2-92e9-4fbe-80f6-d977d0e1d11b\") " pod="openstack-kuttl-tests/placement-db-create-mzssh" Dec 03 12:33:44 crc kubenswrapper[5052]: I1203 12:33:44.627274 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/fc2e5ba2-92e9-4fbe-80f6-d977d0e1d11b-operator-scripts\") pod \"placement-db-create-mzssh\" (UID: \"fc2e5ba2-92e9-4fbe-80f6-d977d0e1d11b\") " pod="openstack-kuttl-tests/placement-db-create-mzssh" Dec 03 12:33:44 crc kubenswrapper[5052]: I1203 12:33:44.707556 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/placement-7f30-account-create-update-zx4ld"] Dec 03 12:33:44 crc kubenswrapper[5052]: I1203 12:33:44.708635 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-7f30-account-create-update-zx4ld" Dec 03 12:33:44 crc kubenswrapper[5052]: I1203 12:33:44.712149 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"placement-db-secret" Dec 03 12:33:44 crc kubenswrapper[5052]: I1203 12:33:44.728359 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-jjzvk\" (UniqueName: \"kubernetes.io/projected/fc2e5ba2-92e9-4fbe-80f6-d977d0e1d11b-kube-api-access-jjzvk\") pod \"placement-db-create-mzssh\" (UID: \"fc2e5ba2-92e9-4fbe-80f6-d977d0e1d11b\") " pod="openstack-kuttl-tests/placement-db-create-mzssh" Dec 03 12:33:44 crc kubenswrapper[5052]: I1203 12:33:44.728665 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/fc2e5ba2-92e9-4fbe-80f6-d977d0e1d11b-operator-scripts\") pod \"placement-db-create-mzssh\" (UID: \"fc2e5ba2-92e9-4fbe-80f6-d977d0e1d11b\") " pod="openstack-kuttl-tests/placement-db-create-mzssh" Dec 03 12:33:44 crc kubenswrapper[5052]: I1203 12:33:44.729274 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/fc2e5ba2-92e9-4fbe-80f6-d977d0e1d11b-operator-scripts\") pod \"placement-db-create-mzssh\" (UID: \"fc2e5ba2-92e9-4fbe-80f6-d977d0e1d11b\") " pod="openstack-kuttl-tests/placement-db-create-mzssh" Dec 03 12:33:44 crc kubenswrapper[5052]: I1203 12:33:44.730145 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/placement-7f30-account-create-update-zx4ld"] Dec 03 12:33:44 crc kubenswrapper[5052]: I1203 12:33:44.760411 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-jjzvk\" (UniqueName: \"kubernetes.io/projected/fc2e5ba2-92e9-4fbe-80f6-d977d0e1d11b-kube-api-access-jjzvk\") pod \"placement-db-create-mzssh\" (UID: \"fc2e5ba2-92e9-4fbe-80f6-d977d0e1d11b\") " pod="openstack-kuttl-tests/placement-db-create-mzssh" Dec 03 12:33:44 crc kubenswrapper[5052]: I1203 12:33:44.799659 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/glance-db-create-tsdrr"] Dec 03 12:33:44 crc kubenswrapper[5052]: I1203 12:33:44.800834 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-db-create-tsdrr" Dec 03 12:33:44 crc kubenswrapper[5052]: I1203 12:33:44.815668 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-db-create-tsdrr"] Dec 03 12:33:44 crc kubenswrapper[5052]: I1203 12:33:44.830304 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/cebde9f9-4c41-4628-b014-6b7771299e50-operator-scripts\") pod \"placement-7f30-account-create-update-zx4ld\" (UID: \"cebde9f9-4c41-4628-b014-6b7771299e50\") " pod="openstack-kuttl-tests/placement-7f30-account-create-update-zx4ld" Dec 03 12:33:44 crc kubenswrapper[5052]: I1203 12:33:44.830491 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-67qmw\" (UniqueName: \"kubernetes.io/projected/cebde9f9-4c41-4628-b014-6b7771299e50-kube-api-access-67qmw\") pod \"placement-7f30-account-create-update-zx4ld\" (UID: \"cebde9f9-4c41-4628-b014-6b7771299e50\") " pod="openstack-kuttl-tests/placement-7f30-account-create-update-zx4ld" Dec 03 12:33:44 crc kubenswrapper[5052]: I1203 12:33:44.901623 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/glance-c94f-account-create-update-jjfpq"] Dec 03 12:33:44 crc kubenswrapper[5052]: I1203 12:33:44.902873 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-c94f-account-create-update-jjfpq" Dec 03 12:33:44 crc kubenswrapper[5052]: I1203 12:33:44.905003 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-db-secret" Dec 03 12:33:44 crc kubenswrapper[5052]: I1203 12:33:44.911800 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-c94f-account-create-update-jjfpq"] Dec 03 12:33:44 crc kubenswrapper[5052]: I1203 12:33:44.912999 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:33:44 crc kubenswrapper[5052]: I1203 12:33:44.932571 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lqjs5\" (UniqueName: \"kubernetes.io/projected/dc792e82-3490-43dc-a435-8847a5d3940e-kube-api-access-lqjs5\") pod \"glance-db-create-tsdrr\" (UID: \"dc792e82-3490-43dc-a435-8847a5d3940e\") " pod="openstack-kuttl-tests/glance-db-create-tsdrr" Dec 03 12:33:44 crc kubenswrapper[5052]: I1203 12:33:44.932631 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/dc792e82-3490-43dc-a435-8847a5d3940e-operator-scripts\") pod \"glance-db-create-tsdrr\" (UID: \"dc792e82-3490-43dc-a435-8847a5d3940e\") " pod="openstack-kuttl-tests/glance-db-create-tsdrr" Dec 03 12:33:44 crc kubenswrapper[5052]: I1203 12:33:44.933261 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-67qmw\" (UniqueName: \"kubernetes.io/projected/cebde9f9-4c41-4628-b014-6b7771299e50-kube-api-access-67qmw\") pod \"placement-7f30-account-create-update-zx4ld\" (UID: \"cebde9f9-4c41-4628-b014-6b7771299e50\") " pod="openstack-kuttl-tests/placement-7f30-account-create-update-zx4ld" Dec 03 12:33:44 crc kubenswrapper[5052]: I1203 12:33:44.933327 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/cebde9f9-4c41-4628-b014-6b7771299e50-operator-scripts\") pod \"placement-7f30-account-create-update-zx4ld\" (UID: \"cebde9f9-4c41-4628-b014-6b7771299e50\") " pod="openstack-kuttl-tests/placement-7f30-account-create-update-zx4ld" Dec 03 12:33:44 crc kubenswrapper[5052]: I1203 12:33:44.934167 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/cebde9f9-4c41-4628-b014-6b7771299e50-operator-scripts\") pod \"placement-7f30-account-create-update-zx4ld\" (UID: \"cebde9f9-4c41-4628-b014-6b7771299e50\") " pod="openstack-kuttl-tests/placement-7f30-account-create-update-zx4ld" Dec 03 12:33:44 crc kubenswrapper[5052]: I1203 12:33:44.938583 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-db-create-mzssh" Dec 03 12:33:44 crc kubenswrapper[5052]: I1203 12:33:44.953153 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-67qmw\" (UniqueName: \"kubernetes.io/projected/cebde9f9-4c41-4628-b014-6b7771299e50-kube-api-access-67qmw\") pod \"placement-7f30-account-create-update-zx4ld\" (UID: \"cebde9f9-4c41-4628-b014-6b7771299e50\") " pod="openstack-kuttl-tests/placement-7f30-account-create-update-zx4ld" Dec 03 12:33:45 crc kubenswrapper[5052]: I1203 12:33:45.032410 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-7f30-account-create-update-zx4ld" Dec 03 12:33:45 crc kubenswrapper[5052]: I1203 12:33:45.034587 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w87c8\" (UniqueName: \"kubernetes.io/projected/41a48918-de1d-4463-94ed-3d19cc52ead8-kube-api-access-w87c8\") pod \"glance-c94f-account-create-update-jjfpq\" (UID: \"41a48918-de1d-4463-94ed-3d19cc52ead8\") " pod="openstack-kuttl-tests/glance-c94f-account-create-update-jjfpq" Dec 03 12:33:45 crc kubenswrapper[5052]: I1203 12:33:45.034626 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-lqjs5\" (UniqueName: \"kubernetes.io/projected/dc792e82-3490-43dc-a435-8847a5d3940e-kube-api-access-lqjs5\") pod \"glance-db-create-tsdrr\" (UID: \"dc792e82-3490-43dc-a435-8847a5d3940e\") " pod="openstack-kuttl-tests/glance-db-create-tsdrr" Dec 03 12:33:45 crc kubenswrapper[5052]: I1203 12:33:45.034668 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/dc792e82-3490-43dc-a435-8847a5d3940e-operator-scripts\") pod \"glance-db-create-tsdrr\" (UID: \"dc792e82-3490-43dc-a435-8847a5d3940e\") " pod="openstack-kuttl-tests/glance-db-create-tsdrr" Dec 03 12:33:45 crc kubenswrapper[5052]: I1203 12:33:45.034688 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/41a48918-de1d-4463-94ed-3d19cc52ead8-operator-scripts\") pod \"glance-c94f-account-create-update-jjfpq\" (UID: \"41a48918-de1d-4463-94ed-3d19cc52ead8\") " pod="openstack-kuttl-tests/glance-c94f-account-create-update-jjfpq" Dec 03 12:33:45 crc kubenswrapper[5052]: I1203 12:33:45.036259 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/dc792e82-3490-43dc-a435-8847a5d3940e-operator-scripts\") pod \"glance-db-create-tsdrr\" (UID: \"dc792e82-3490-43dc-a435-8847a5d3940e\") " pod="openstack-kuttl-tests/glance-db-create-tsdrr" Dec 03 12:33:45 crc kubenswrapper[5052]: I1203 12:33:45.066784 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-lqjs5\" (UniqueName: \"kubernetes.io/projected/dc792e82-3490-43dc-a435-8847a5d3940e-kube-api-access-lqjs5\") pod \"glance-db-create-tsdrr\" (UID: \"dc792e82-3490-43dc-a435-8847a5d3940e\") " pod="openstack-kuttl-tests/glance-db-create-tsdrr" Dec 03 12:33:45 crc kubenswrapper[5052]: I1203 12:33:45.121102 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-db-create-tsdrr" Dec 03 12:33:45 crc kubenswrapper[5052]: I1203 12:33:45.136079 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-w87c8\" (UniqueName: \"kubernetes.io/projected/41a48918-de1d-4463-94ed-3d19cc52ead8-kube-api-access-w87c8\") pod \"glance-c94f-account-create-update-jjfpq\" (UID: \"41a48918-de1d-4463-94ed-3d19cc52ead8\") " pod="openstack-kuttl-tests/glance-c94f-account-create-update-jjfpq" Dec 03 12:33:45 crc kubenswrapper[5052]: I1203 12:33:45.136143 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/41a48918-de1d-4463-94ed-3d19cc52ead8-operator-scripts\") pod \"glance-c94f-account-create-update-jjfpq\" (UID: \"41a48918-de1d-4463-94ed-3d19cc52ead8\") " pod="openstack-kuttl-tests/glance-c94f-account-create-update-jjfpq" Dec 03 12:33:45 crc kubenswrapper[5052]: I1203 12:33:45.136899 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/41a48918-de1d-4463-94ed-3d19cc52ead8-operator-scripts\") pod \"glance-c94f-account-create-update-jjfpq\" (UID: \"41a48918-de1d-4463-94ed-3d19cc52ead8\") " pod="openstack-kuttl-tests/glance-c94f-account-create-update-jjfpq" Dec 03 12:33:45 crc kubenswrapper[5052]: I1203 12:33:45.153393 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-w87c8\" (UniqueName: \"kubernetes.io/projected/41a48918-de1d-4463-94ed-3d19cc52ead8-kube-api-access-w87c8\") pod \"glance-c94f-account-create-update-jjfpq\" (UID: \"41a48918-de1d-4463-94ed-3d19cc52ead8\") " pod="openstack-kuttl-tests/glance-c94f-account-create-update-jjfpq" Dec 03 12:33:45 crc kubenswrapper[5052]: I1203 12:33:45.222526 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-c94f-account-create-update-jjfpq" Dec 03 12:33:45 crc kubenswrapper[5052]: I1203 12:33:45.389171 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/placement-db-create-mzssh"] Dec 03 12:33:45 crc kubenswrapper[5052]: W1203 12:33:45.394589 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfc2e5ba2_92e9_4fbe_80f6_d977d0e1d11b.slice/crio-00fddd7bb75ca79ec173d0896f6b6e4a3ffaee22d9e254de8ed62313492a629c WatchSource:0}: Error finding container 00fddd7bb75ca79ec173d0896f6b6e4a3ffaee22d9e254de8ed62313492a629c: Status 404 returned error can't find the container with id 00fddd7bb75ca79ec173d0896f6b6e4a3ffaee22d9e254de8ed62313492a629c Dec 03 12:33:45 crc kubenswrapper[5052]: I1203 12:33:45.535212 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/placement-7f30-account-create-update-zx4ld"] Dec 03 12:33:45 crc kubenswrapper[5052]: W1203 12:33:45.541328 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcebde9f9_4c41_4628_b014_6b7771299e50.slice/crio-6ba65ebd85209235903c29ad5b793cf27b7c1efc4985c8ca5cb85d7d0474211d WatchSource:0}: Error finding container 6ba65ebd85209235903c29ad5b793cf27b7c1efc4985c8ca5cb85d7d0474211d: Status 404 returned error can't find the container with id 6ba65ebd85209235903c29ad5b793cf27b7c1efc4985c8ca5cb85d7d0474211d Dec 03 12:33:45 crc kubenswrapper[5052]: I1203 12:33:45.632472 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-db-create-tsdrr"] Dec 03 12:33:45 crc kubenswrapper[5052]: W1203 12:33:45.635327 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddc792e82_3490_43dc_a435_8847a5d3940e.slice/crio-b8424553ffe296f5b1c3a36004e98816c238288dbbf60709fdb534e9cf7306fb WatchSource:0}: Error finding container b8424553ffe296f5b1c3a36004e98816c238288dbbf60709fdb534e9cf7306fb: Status 404 returned error can't find the container with id b8424553ffe296f5b1c3a36004e98816c238288dbbf60709fdb534e9cf7306fb Dec 03 12:33:45 crc kubenswrapper[5052]: I1203 12:33:45.842788 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-c94f-account-create-update-jjfpq"] Dec 03 12:33:45 crc kubenswrapper[5052]: I1203 12:33:45.947639 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:33:46 crc kubenswrapper[5052]: I1203 12:33:46.105530 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:33:46 crc kubenswrapper[5052]: I1203 12:33:46.152384 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:33:46 crc kubenswrapper[5052]: I1203 12:33:46.331974 5052 generic.go:334] "Generic (PLEG): container finished" podID="cebde9f9-4c41-4628-b014-6b7771299e50" containerID="95f57bf535752ee63cba070f80b83c1452bf047674a4d81bbcd2c9f86b68b156" exitCode=0 Dec 03 12:33:46 crc kubenswrapper[5052]: I1203 12:33:46.332039 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-7f30-account-create-update-zx4ld" event={"ID":"cebde9f9-4c41-4628-b014-6b7771299e50","Type":"ContainerDied","Data":"95f57bf535752ee63cba070f80b83c1452bf047674a4d81bbcd2c9f86b68b156"} Dec 03 12:33:46 crc kubenswrapper[5052]: I1203 12:33:46.332069 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-7f30-account-create-update-zx4ld" event={"ID":"cebde9f9-4c41-4628-b014-6b7771299e50","Type":"ContainerStarted","Data":"6ba65ebd85209235903c29ad5b793cf27b7c1efc4985c8ca5cb85d7d0474211d"} Dec 03 12:33:46 crc kubenswrapper[5052]: I1203 12:33:46.334403 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-db-create-mzssh" event={"ID":"fc2e5ba2-92e9-4fbe-80f6-d977d0e1d11b","Type":"ContainerStarted","Data":"fe3736c7ba04bcc1974c0d7f89521fdb2f73dec260f35708f011fe12a31b1f16"} Dec 03 12:33:46 crc kubenswrapper[5052]: I1203 12:33:46.334444 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-db-create-mzssh" event={"ID":"fc2e5ba2-92e9-4fbe-80f6-d977d0e1d11b","Type":"ContainerStarted","Data":"00fddd7bb75ca79ec173d0896f6b6e4a3ffaee22d9e254de8ed62313492a629c"} Dec 03 12:33:46 crc kubenswrapper[5052]: I1203 12:33:46.335854 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-db-create-tsdrr" event={"ID":"dc792e82-3490-43dc-a435-8847a5d3940e","Type":"ContainerStarted","Data":"ae7fadfe5a8df990d9027b5c721db8c6366fb8ccd2d58034fa4422a3ec06e20a"} Dec 03 12:33:46 crc kubenswrapper[5052]: I1203 12:33:46.335909 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-db-create-tsdrr" event={"ID":"dc792e82-3490-43dc-a435-8847a5d3940e","Type":"ContainerStarted","Data":"b8424553ffe296f5b1c3a36004e98816c238288dbbf60709fdb534e9cf7306fb"} Dec 03 12:33:46 crc kubenswrapper[5052]: I1203 12:33:46.337650 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-c94f-account-create-update-jjfpq" event={"ID":"41a48918-de1d-4463-94ed-3d19cc52ead8","Type":"ContainerStarted","Data":"affe246afdd79fb48a7eb807b7a9eee6026dac2baf6ba1d47fd16150e4137cd4"} Dec 03 12:33:46 crc kubenswrapper[5052]: I1203 12:33:46.337692 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-c94f-account-create-update-jjfpq" event={"ID":"41a48918-de1d-4463-94ed-3d19cc52ead8","Type":"ContainerStarted","Data":"3def9e651e90841466395e2cce6e486126d67cfb87376cfee64d788490779248"} Dec 03 12:33:46 crc kubenswrapper[5052]: I1203 12:33:46.369624 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/glance-c94f-account-create-update-jjfpq" podStartSLOduration=2.369609249 podStartE2EDuration="2.369609249s" podCreationTimestamp="2025-12-03 12:33:44 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:33:46.367802357 +0000 UTC m=+4416.956044121" watchObservedRunningTime="2025-12-03 12:33:46.369609249 +0000 UTC m=+4416.957851013" Dec 03 12:33:46 crc kubenswrapper[5052]: I1203 12:33:46.427066 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/glance-db-create-tsdrr" podStartSLOduration=2.427040708 podStartE2EDuration="2.427040708s" podCreationTimestamp="2025-12-03 12:33:44 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:33:46.387192271 +0000 UTC m=+4416.975434035" watchObservedRunningTime="2025-12-03 12:33:46.427040708 +0000 UTC m=+4417.015282482" Dec 03 12:33:46 crc kubenswrapper[5052]: I1203 12:33:46.448741 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/placement-db-create-mzssh" podStartSLOduration=2.448717337 podStartE2EDuration="2.448717337s" podCreationTimestamp="2025-12-03 12:33:44 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:33:46.42044803 +0000 UTC m=+4417.008689804" watchObservedRunningTime="2025-12-03 12:33:46.448717337 +0000 UTC m=+4417.036959111" Dec 03 12:33:46 crc kubenswrapper[5052]: I1203 12:33:46.473040 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:33:46 crc kubenswrapper[5052]: I1203 12:33:46.635771 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:33:46 crc kubenswrapper[5052]: I1203 12:33:46.727661 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:33:47 crc kubenswrapper[5052]: I1203 12:33:47.347794 5052 generic.go:334] "Generic (PLEG): container finished" podID="fc2e5ba2-92e9-4fbe-80f6-d977d0e1d11b" containerID="fe3736c7ba04bcc1974c0d7f89521fdb2f73dec260f35708f011fe12a31b1f16" exitCode=0 Dec 03 12:33:47 crc kubenswrapper[5052]: I1203 12:33:47.348088 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-db-create-mzssh" event={"ID":"fc2e5ba2-92e9-4fbe-80f6-d977d0e1d11b","Type":"ContainerDied","Data":"fe3736c7ba04bcc1974c0d7f89521fdb2f73dec260f35708f011fe12a31b1f16"} Dec 03 12:33:47 crc kubenswrapper[5052]: I1203 12:33:47.350778 5052 generic.go:334] "Generic (PLEG): container finished" podID="dc792e82-3490-43dc-a435-8847a5d3940e" containerID="ae7fadfe5a8df990d9027b5c721db8c6366fb8ccd2d58034fa4422a3ec06e20a" exitCode=0 Dec 03 12:33:47 crc kubenswrapper[5052]: I1203 12:33:47.350911 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-db-create-tsdrr" event={"ID":"dc792e82-3490-43dc-a435-8847a5d3940e","Type":"ContainerDied","Data":"ae7fadfe5a8df990d9027b5c721db8c6366fb8ccd2d58034fa4422a3ec06e20a"} Dec 03 12:33:47 crc kubenswrapper[5052]: I1203 12:33:47.354254 5052 generic.go:334] "Generic (PLEG): container finished" podID="41a48918-de1d-4463-94ed-3d19cc52ead8" containerID="affe246afdd79fb48a7eb807b7a9eee6026dac2baf6ba1d47fd16150e4137cd4" exitCode=0 Dec 03 12:33:47 crc kubenswrapper[5052]: I1203 12:33:47.355540 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-c94f-account-create-update-jjfpq" event={"ID":"41a48918-de1d-4463-94ed-3d19cc52ead8","Type":"ContainerDied","Data":"affe246afdd79fb48a7eb807b7a9eee6026dac2baf6ba1d47fd16150e4137cd4"} Dec 03 12:33:47 crc kubenswrapper[5052]: I1203 12:33:47.464423 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/swift-storage-0"] Dec 03 12:33:47 crc kubenswrapper[5052]: I1203 12:33:47.478275 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:33:47 crc kubenswrapper[5052]: I1203 12:33:47.480709 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"swift-ring-files" Dec 03 12:33:47 crc kubenswrapper[5052]: I1203 12:33:47.481190 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"swift-conf" Dec 03 12:33:47 crc kubenswrapper[5052]: I1203 12:33:47.481296 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"swift-swift-dockercfg-8csdn" Dec 03 12:33:47 crc kubenswrapper[5052]: I1203 12:33:47.481577 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"swift-storage-config-data" Dec 03 12:33:47 crc kubenswrapper[5052]: I1203 12:33:47.481718 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/swift-storage-0"] Dec 03 12:33:47 crc kubenswrapper[5052]: I1203 12:33:47.504007 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage13-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage13-crc\") pod \"swift-storage-0\" (UID: \"c7d598df-f336-4798-b381-9b813068f271\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:33:47 crc kubenswrapper[5052]: I1203 12:33:47.504083 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ghbrw\" (UniqueName: \"kubernetes.io/projected/c7d598df-f336-4798-b381-9b813068f271-kube-api-access-ghbrw\") pod \"swift-storage-0\" (UID: \"c7d598df-f336-4798-b381-9b813068f271\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:33:47 crc kubenswrapper[5052]: I1203 12:33:47.504112 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/c7d598df-f336-4798-b381-9b813068f271-etc-swift\") pod \"swift-storage-0\" (UID: \"c7d598df-f336-4798-b381-9b813068f271\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:33:47 crc kubenswrapper[5052]: I1203 12:33:47.504161 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cache\" (UniqueName: \"kubernetes.io/empty-dir/c7d598df-f336-4798-b381-9b813068f271-cache\") pod \"swift-storage-0\" (UID: \"c7d598df-f336-4798-b381-9b813068f271\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:33:47 crc kubenswrapper[5052]: I1203 12:33:47.504292 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lock\" (UniqueName: \"kubernetes.io/empty-dir/c7d598df-f336-4798-b381-9b813068f271-lock\") pod \"swift-storage-0\" (UID: \"c7d598df-f336-4798-b381-9b813068f271\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:33:47 crc kubenswrapper[5052]: I1203 12:33:47.612812 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"cache\" (UniqueName: \"kubernetes.io/empty-dir/c7d598df-f336-4798-b381-9b813068f271-cache\") pod \"swift-storage-0\" (UID: \"c7d598df-f336-4798-b381-9b813068f271\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:33:47 crc kubenswrapper[5052]: I1203 12:33:47.612989 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"lock\" (UniqueName: \"kubernetes.io/empty-dir/c7d598df-f336-4798-b381-9b813068f271-lock\") pod \"swift-storage-0\" (UID: \"c7d598df-f336-4798-b381-9b813068f271\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:33:47 crc kubenswrapper[5052]: I1203 12:33:47.613032 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage13-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage13-crc\") pod \"swift-storage-0\" (UID: \"c7d598df-f336-4798-b381-9b813068f271\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:33:47 crc kubenswrapper[5052]: I1203 12:33:47.613066 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-ghbrw\" (UniqueName: \"kubernetes.io/projected/c7d598df-f336-4798-b381-9b813068f271-kube-api-access-ghbrw\") pod \"swift-storage-0\" (UID: \"c7d598df-f336-4798-b381-9b813068f271\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:33:47 crc kubenswrapper[5052]: I1203 12:33:47.613092 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/c7d598df-f336-4798-b381-9b813068f271-etc-swift\") pod \"swift-storage-0\" (UID: \"c7d598df-f336-4798-b381-9b813068f271\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:33:47 crc kubenswrapper[5052]: E1203 12:33:47.613227 5052 projected.go:288] Couldn't get configMap openstack-kuttl-tests/swift-ring-files: configmap "swift-ring-files" not found Dec 03 12:33:47 crc kubenswrapper[5052]: E1203 12:33:47.613243 5052 projected.go:194] Error preparing data for projected volume etc-swift for pod openstack-kuttl-tests/swift-storage-0: configmap "swift-ring-files" not found Dec 03 12:33:47 crc kubenswrapper[5052]: E1203 12:33:47.613300 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c7d598df-f336-4798-b381-9b813068f271-etc-swift podName:c7d598df-f336-4798-b381-9b813068f271 nodeName:}" failed. No retries permitted until 2025-12-03 12:33:48.113280266 +0000 UTC m=+4418.701522030 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "etc-swift" (UniqueName: "kubernetes.io/projected/c7d598df-f336-4798-b381-9b813068f271-etc-swift") pod "swift-storage-0" (UID: "c7d598df-f336-4798-b381-9b813068f271") : configmap "swift-ring-files" not found Dec 03 12:33:47 crc kubenswrapper[5052]: I1203 12:33:47.613550 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage13-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage13-crc\") pod \"swift-storage-0\" (UID: \"c7d598df-f336-4798-b381-9b813068f271\") device mount path \"/mnt/openstack/pv13\"" pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:33:47 crc kubenswrapper[5052]: I1203 12:33:47.613592 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"cache\" (UniqueName: \"kubernetes.io/empty-dir/c7d598df-f336-4798-b381-9b813068f271-cache\") pod \"swift-storage-0\" (UID: \"c7d598df-f336-4798-b381-9b813068f271\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:33:47 crc kubenswrapper[5052]: I1203 12:33:47.613708 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"lock\" (UniqueName: \"kubernetes.io/empty-dir/c7d598df-f336-4798-b381-9b813068f271-lock\") pod \"swift-storage-0\" (UID: \"c7d598df-f336-4798-b381-9b813068f271\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:33:47 crc kubenswrapper[5052]: I1203 12:33:47.636558 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-ghbrw\" (UniqueName: \"kubernetes.io/projected/c7d598df-f336-4798-b381-9b813068f271-kube-api-access-ghbrw\") pod \"swift-storage-0\" (UID: \"c7d598df-f336-4798-b381-9b813068f271\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:33:47 crc kubenswrapper[5052]: I1203 12:33:47.648142 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage13-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage13-crc\") pod \"swift-storage-0\" (UID: \"c7d598df-f336-4798-b381-9b813068f271\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:33:47 crc kubenswrapper[5052]: I1203 12:33:47.704460 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-7f30-account-create-update-zx4ld" Dec 03 12:33:47 crc kubenswrapper[5052]: I1203 12:33:47.815990 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-67qmw\" (UniqueName: \"kubernetes.io/projected/cebde9f9-4c41-4628-b014-6b7771299e50-kube-api-access-67qmw\") pod \"cebde9f9-4c41-4628-b014-6b7771299e50\" (UID: \"cebde9f9-4c41-4628-b014-6b7771299e50\") " Dec 03 12:33:47 crc kubenswrapper[5052]: I1203 12:33:47.816086 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/cebde9f9-4c41-4628-b014-6b7771299e50-operator-scripts\") pod \"cebde9f9-4c41-4628-b014-6b7771299e50\" (UID: \"cebde9f9-4c41-4628-b014-6b7771299e50\") " Dec 03 12:33:47 crc kubenswrapper[5052]: I1203 12:33:47.817042 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cebde9f9-4c41-4628-b014-6b7771299e50-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "cebde9f9-4c41-4628-b014-6b7771299e50" (UID: "cebde9f9-4c41-4628-b014-6b7771299e50"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:33:47 crc kubenswrapper[5052]: I1203 12:33:47.820376 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cebde9f9-4c41-4628-b014-6b7771299e50-kube-api-access-67qmw" (OuterVolumeSpecName: "kube-api-access-67qmw") pod "cebde9f9-4c41-4628-b014-6b7771299e50" (UID: "cebde9f9-4c41-4628-b014-6b7771299e50"). InnerVolumeSpecName "kube-api-access-67qmw". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:33:47 crc kubenswrapper[5052]: I1203 12:33:47.918813 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-67qmw\" (UniqueName: \"kubernetes.io/projected/cebde9f9-4c41-4628-b014-6b7771299e50-kube-api-access-67qmw\") on node \"crc\" DevicePath \"\"" Dec 03 12:33:47 crc kubenswrapper[5052]: I1203 12:33:47.918861 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/cebde9f9-4c41-4628-b014-6b7771299e50-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:33:47 crc kubenswrapper[5052]: I1203 12:33:47.948004 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:33:48 crc kubenswrapper[5052]: I1203 12:33:48.122093 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/c7d598df-f336-4798-b381-9b813068f271-etc-swift\") pod \"swift-storage-0\" (UID: \"c7d598df-f336-4798-b381-9b813068f271\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:33:48 crc kubenswrapper[5052]: E1203 12:33:48.122315 5052 projected.go:288] Couldn't get configMap openstack-kuttl-tests/swift-ring-files: configmap "swift-ring-files" not found Dec 03 12:33:48 crc kubenswrapper[5052]: E1203 12:33:48.122345 5052 projected.go:194] Error preparing data for projected volume etc-swift for pod openstack-kuttl-tests/swift-storage-0: configmap "swift-ring-files" not found Dec 03 12:33:48 crc kubenswrapper[5052]: E1203 12:33:48.122413 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c7d598df-f336-4798-b381-9b813068f271-etc-swift podName:c7d598df-f336-4798-b381-9b813068f271 nodeName:}" failed. No retries permitted until 2025-12-03 12:33:49.122388937 +0000 UTC m=+4419.710630701 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "etc-swift" (UniqueName: "kubernetes.io/projected/c7d598df-f336-4798-b381-9b813068f271-etc-swift") pod "swift-storage-0" (UID: "c7d598df-f336-4798-b381-9b813068f271") : configmap "swift-ring-files" not found Dec 03 12:33:48 crc kubenswrapper[5052]: I1203 12:33:48.364262 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-7f30-account-create-update-zx4ld" event={"ID":"cebde9f9-4c41-4628-b014-6b7771299e50","Type":"ContainerDied","Data":"6ba65ebd85209235903c29ad5b793cf27b7c1efc4985c8ca5cb85d7d0474211d"} Dec 03 12:33:48 crc kubenswrapper[5052]: I1203 12:33:48.364690 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6ba65ebd85209235903c29ad5b793cf27b7c1efc4985c8ca5cb85d7d0474211d" Dec 03 12:33:48 crc kubenswrapper[5052]: I1203 12:33:48.364310 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-7f30-account-create-update-zx4ld" Dec 03 12:33:48 crc kubenswrapper[5052]: I1203 12:33:48.782275 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-db-create-mzssh" Dec 03 12:33:48 crc kubenswrapper[5052]: I1203 12:33:48.847012 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jjzvk\" (UniqueName: \"kubernetes.io/projected/fc2e5ba2-92e9-4fbe-80f6-d977d0e1d11b-kube-api-access-jjzvk\") pod \"fc2e5ba2-92e9-4fbe-80f6-d977d0e1d11b\" (UID: \"fc2e5ba2-92e9-4fbe-80f6-d977d0e1d11b\") " Dec 03 12:33:48 crc kubenswrapper[5052]: I1203 12:33:48.847191 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/fc2e5ba2-92e9-4fbe-80f6-d977d0e1d11b-operator-scripts\") pod \"fc2e5ba2-92e9-4fbe-80f6-d977d0e1d11b\" (UID: \"fc2e5ba2-92e9-4fbe-80f6-d977d0e1d11b\") " Dec 03 12:33:48 crc kubenswrapper[5052]: I1203 12:33:48.847977 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fc2e5ba2-92e9-4fbe-80f6-d977d0e1d11b-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "fc2e5ba2-92e9-4fbe-80f6-d977d0e1d11b" (UID: "fc2e5ba2-92e9-4fbe-80f6-d977d0e1d11b"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:33:48 crc kubenswrapper[5052]: I1203 12:33:48.860498 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fc2e5ba2-92e9-4fbe-80f6-d977d0e1d11b-kube-api-access-jjzvk" (OuterVolumeSpecName: "kube-api-access-jjzvk") pod "fc2e5ba2-92e9-4fbe-80f6-d977d0e1d11b" (UID: "fc2e5ba2-92e9-4fbe-80f6-d977d0e1d11b"). InnerVolumeSpecName "kube-api-access-jjzvk". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:33:48 crc kubenswrapper[5052]: I1203 12:33:48.932785 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-c94f-account-create-update-jjfpq" Dec 03 12:33:48 crc kubenswrapper[5052]: I1203 12:33:48.937503 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-db-create-tsdrr" Dec 03 12:33:48 crc kubenswrapper[5052]: I1203 12:33:48.953164 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w87c8\" (UniqueName: \"kubernetes.io/projected/41a48918-de1d-4463-94ed-3d19cc52ead8-kube-api-access-w87c8\") pod \"41a48918-de1d-4463-94ed-3d19cc52ead8\" (UID: \"41a48918-de1d-4463-94ed-3d19cc52ead8\") " Dec 03 12:33:48 crc kubenswrapper[5052]: I1203 12:33:48.953457 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lqjs5\" (UniqueName: \"kubernetes.io/projected/dc792e82-3490-43dc-a435-8847a5d3940e-kube-api-access-lqjs5\") pod \"dc792e82-3490-43dc-a435-8847a5d3940e\" (UID: \"dc792e82-3490-43dc-a435-8847a5d3940e\") " Dec 03 12:33:48 crc kubenswrapper[5052]: I1203 12:33:48.953543 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/dc792e82-3490-43dc-a435-8847a5d3940e-operator-scripts\") pod \"dc792e82-3490-43dc-a435-8847a5d3940e\" (UID: \"dc792e82-3490-43dc-a435-8847a5d3940e\") " Dec 03 12:33:48 crc kubenswrapper[5052]: I1203 12:33:48.953664 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/41a48918-de1d-4463-94ed-3d19cc52ead8-operator-scripts\") pod \"41a48918-de1d-4463-94ed-3d19cc52ead8\" (UID: \"41a48918-de1d-4463-94ed-3d19cc52ead8\") " Dec 03 12:33:48 crc kubenswrapper[5052]: I1203 12:33:48.954038 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-jjzvk\" (UniqueName: \"kubernetes.io/projected/fc2e5ba2-92e9-4fbe-80f6-d977d0e1d11b-kube-api-access-jjzvk\") on node \"crc\" DevicePath \"\"" Dec 03 12:33:48 crc kubenswrapper[5052]: I1203 12:33:48.954055 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/fc2e5ba2-92e9-4fbe-80f6-d977d0e1d11b-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:33:48 crc kubenswrapper[5052]: I1203 12:33:48.954446 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/41a48918-de1d-4463-94ed-3d19cc52ead8-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "41a48918-de1d-4463-94ed-3d19cc52ead8" (UID: "41a48918-de1d-4463-94ed-3d19cc52ead8"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:33:48 crc kubenswrapper[5052]: I1203 12:33:48.954677 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dc792e82-3490-43dc-a435-8847a5d3940e-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "dc792e82-3490-43dc-a435-8847a5d3940e" (UID: "dc792e82-3490-43dc-a435-8847a5d3940e"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:33:48 crc kubenswrapper[5052]: I1203 12:33:48.958131 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/41a48918-de1d-4463-94ed-3d19cc52ead8-kube-api-access-w87c8" (OuterVolumeSpecName: "kube-api-access-w87c8") pod "41a48918-de1d-4463-94ed-3d19cc52ead8" (UID: "41a48918-de1d-4463-94ed-3d19cc52ead8"). InnerVolumeSpecName "kube-api-access-w87c8". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:33:48 crc kubenswrapper[5052]: I1203 12:33:48.959806 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dc792e82-3490-43dc-a435-8847a5d3940e-kube-api-access-lqjs5" (OuterVolumeSpecName: "kube-api-access-lqjs5") pod "dc792e82-3490-43dc-a435-8847a5d3940e" (UID: "dc792e82-3490-43dc-a435-8847a5d3940e"). InnerVolumeSpecName "kube-api-access-lqjs5". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:33:49 crc kubenswrapper[5052]: I1203 12:33:49.000408 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:33:49 crc kubenswrapper[5052]: I1203 12:33:49.039569 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:33:49 crc kubenswrapper[5052]: I1203 12:33:49.055904 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/41a48918-de1d-4463-94ed-3d19cc52ead8-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:33:49 crc kubenswrapper[5052]: I1203 12:33:49.055945 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-w87c8\" (UniqueName: \"kubernetes.io/projected/41a48918-de1d-4463-94ed-3d19cc52ead8-kube-api-access-w87c8\") on node \"crc\" DevicePath \"\"" Dec 03 12:33:49 crc kubenswrapper[5052]: I1203 12:33:49.055958 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-lqjs5\" (UniqueName: \"kubernetes.io/projected/dc792e82-3490-43dc-a435-8847a5d3940e-kube-api-access-lqjs5\") on node \"crc\" DevicePath \"\"" Dec 03 12:33:49 crc kubenswrapper[5052]: I1203 12:33:49.055970 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/dc792e82-3490-43dc-a435-8847a5d3940e-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:33:49 crc kubenswrapper[5052]: I1203 12:33:49.157138 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/c7d598df-f336-4798-b381-9b813068f271-etc-swift\") pod \"swift-storage-0\" (UID: \"c7d598df-f336-4798-b381-9b813068f271\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:33:49 crc kubenswrapper[5052]: E1203 12:33:49.157310 5052 projected.go:288] Couldn't get configMap openstack-kuttl-tests/swift-ring-files: configmap "swift-ring-files" not found Dec 03 12:33:49 crc kubenswrapper[5052]: E1203 12:33:49.157324 5052 projected.go:194] Error preparing data for projected volume etc-swift for pod openstack-kuttl-tests/swift-storage-0: configmap "swift-ring-files" not found Dec 03 12:33:49 crc kubenswrapper[5052]: E1203 12:33:49.157373 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c7d598df-f336-4798-b381-9b813068f271-etc-swift podName:c7d598df-f336-4798-b381-9b813068f271 nodeName:}" failed. No retries permitted until 2025-12-03 12:33:51.157358698 +0000 UTC m=+4421.745600462 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "etc-swift" (UniqueName: "kubernetes.io/projected/c7d598df-f336-4798-b381-9b813068f271-etc-swift") pod "swift-storage-0" (UID: "c7d598df-f336-4798-b381-9b813068f271") : configmap "swift-ring-files" not found Dec 03 12:33:49 crc kubenswrapper[5052]: I1203 12:33:49.244820 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ovn-northd-0"] Dec 03 12:33:49 crc kubenswrapper[5052]: E1203 12:33:49.245308 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="fc2e5ba2-92e9-4fbe-80f6-d977d0e1d11b" containerName="mariadb-database-create" Dec 03 12:33:49 crc kubenswrapper[5052]: I1203 12:33:49.245331 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="fc2e5ba2-92e9-4fbe-80f6-d977d0e1d11b" containerName="mariadb-database-create" Dec 03 12:33:49 crc kubenswrapper[5052]: E1203 12:33:49.245354 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="41a48918-de1d-4463-94ed-3d19cc52ead8" containerName="mariadb-account-create-update" Dec 03 12:33:49 crc kubenswrapper[5052]: I1203 12:33:49.245362 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="41a48918-de1d-4463-94ed-3d19cc52ead8" containerName="mariadb-account-create-update" Dec 03 12:33:49 crc kubenswrapper[5052]: E1203 12:33:49.245375 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="dc792e82-3490-43dc-a435-8847a5d3940e" containerName="mariadb-database-create" Dec 03 12:33:49 crc kubenswrapper[5052]: I1203 12:33:49.245382 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="dc792e82-3490-43dc-a435-8847a5d3940e" containerName="mariadb-database-create" Dec 03 12:33:49 crc kubenswrapper[5052]: E1203 12:33:49.245422 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="cebde9f9-4c41-4628-b014-6b7771299e50" containerName="mariadb-account-create-update" Dec 03 12:33:49 crc kubenswrapper[5052]: I1203 12:33:49.245428 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="cebde9f9-4c41-4628-b014-6b7771299e50" containerName="mariadb-account-create-update" Dec 03 12:33:49 crc kubenswrapper[5052]: I1203 12:33:49.246112 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="dc792e82-3490-43dc-a435-8847a5d3940e" containerName="mariadb-database-create" Dec 03 12:33:49 crc kubenswrapper[5052]: I1203 12:33:49.246193 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="41a48918-de1d-4463-94ed-3d19cc52ead8" containerName="mariadb-account-create-update" Dec 03 12:33:49 crc kubenswrapper[5052]: I1203 12:33:49.246223 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="fc2e5ba2-92e9-4fbe-80f6-d977d0e1d11b" containerName="mariadb-database-create" Dec 03 12:33:49 crc kubenswrapper[5052]: I1203 12:33:49.246259 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="cebde9f9-4c41-4628-b014-6b7771299e50" containerName="mariadb-account-create-update" Dec 03 12:33:49 crc kubenswrapper[5052]: I1203 12:33:49.260149 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:33:49 crc kubenswrapper[5052]: I1203 12:33:49.268990 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ovnnorthd-ovnnorthd-dockercfg-56wrf" Dec 03 12:33:49 crc kubenswrapper[5052]: I1203 12:33:49.269158 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-ovnnorthd-ovndbs" Dec 03 12:33:49 crc kubenswrapper[5052]: I1203 12:33:49.269220 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"ovnnorthd-config" Dec 03 12:33:49 crc kubenswrapper[5052]: I1203 12:33:49.272390 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ovn-northd-0"] Dec 03 12:33:49 crc kubenswrapper[5052]: I1203 12:33:49.272695 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"ovnnorthd-scripts" Dec 03 12:33:49 crc kubenswrapper[5052]: I1203 12:33:49.360737 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/92dadf05-a318-4491-bb1c-1fa23cb293fa-metrics-certs-tls-certs\") pod \"ovn-northd-0\" (UID: \"92dadf05-a318-4491-bb1c-1fa23cb293fa\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:33:49 crc kubenswrapper[5052]: I1203 12:33:49.360809 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/92dadf05-a318-4491-bb1c-1fa23cb293fa-config\") pod \"ovn-northd-0\" (UID: \"92dadf05-a318-4491-bb1c-1fa23cb293fa\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:33:49 crc kubenswrapper[5052]: I1203 12:33:49.360856 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/92dadf05-a318-4491-bb1c-1fa23cb293fa-scripts\") pod \"ovn-northd-0\" (UID: \"92dadf05-a318-4491-bb1c-1fa23cb293fa\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:33:49 crc kubenswrapper[5052]: I1203 12:33:49.360890 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-rundir\" (UniqueName: \"kubernetes.io/empty-dir/92dadf05-a318-4491-bb1c-1fa23cb293fa-ovn-rundir\") pod \"ovn-northd-0\" (UID: \"92dadf05-a318-4491-bb1c-1fa23cb293fa\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:33:49 crc kubenswrapper[5052]: I1203 12:33:49.360929 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xrncn\" (UniqueName: \"kubernetes.io/projected/92dadf05-a318-4491-bb1c-1fa23cb293fa-kube-api-access-xrncn\") pod \"ovn-northd-0\" (UID: \"92dadf05-a318-4491-bb1c-1fa23cb293fa\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:33:49 crc kubenswrapper[5052]: I1203 12:33:49.361068 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/92dadf05-a318-4491-bb1c-1fa23cb293fa-combined-ca-bundle\") pod \"ovn-northd-0\" (UID: \"92dadf05-a318-4491-bb1c-1fa23cb293fa\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:33:49 crc kubenswrapper[5052]: I1203 12:33:49.361183 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-northd-tls-certs\" (UniqueName: \"kubernetes.io/secret/92dadf05-a318-4491-bb1c-1fa23cb293fa-ovn-northd-tls-certs\") pod \"ovn-northd-0\" (UID: \"92dadf05-a318-4491-bb1c-1fa23cb293fa\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:33:49 crc kubenswrapper[5052]: I1203 12:33:49.373373 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-db-create-tsdrr" event={"ID":"dc792e82-3490-43dc-a435-8847a5d3940e","Type":"ContainerDied","Data":"b8424553ffe296f5b1c3a36004e98816c238288dbbf60709fdb534e9cf7306fb"} Dec 03 12:33:49 crc kubenswrapper[5052]: I1203 12:33:49.373422 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b8424553ffe296f5b1c3a36004e98816c238288dbbf60709fdb534e9cf7306fb" Dec 03 12:33:49 crc kubenswrapper[5052]: I1203 12:33:49.373395 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-db-create-tsdrr" Dec 03 12:33:49 crc kubenswrapper[5052]: I1203 12:33:49.374857 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-c94f-account-create-update-jjfpq" event={"ID":"41a48918-de1d-4463-94ed-3d19cc52ead8","Type":"ContainerDied","Data":"3def9e651e90841466395e2cce6e486126d67cfb87376cfee64d788490779248"} Dec 03 12:33:49 crc kubenswrapper[5052]: I1203 12:33:49.374879 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-c94f-account-create-update-jjfpq" Dec 03 12:33:49 crc kubenswrapper[5052]: I1203 12:33:49.374882 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3def9e651e90841466395e2cce6e486126d67cfb87376cfee64d788490779248" Dec 03 12:33:49 crc kubenswrapper[5052]: I1203 12:33:49.377926 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-db-create-mzssh" event={"ID":"fc2e5ba2-92e9-4fbe-80f6-d977d0e1d11b","Type":"ContainerDied","Data":"00fddd7bb75ca79ec173d0896f6b6e4a3ffaee22d9e254de8ed62313492a629c"} Dec 03 12:33:49 crc kubenswrapper[5052]: I1203 12:33:49.377954 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="00fddd7bb75ca79ec173d0896f6b6e4a3ffaee22d9e254de8ed62313492a629c" Dec 03 12:33:49 crc kubenswrapper[5052]: I1203 12:33:49.378124 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-db-create-mzssh" Dec 03 12:33:49 crc kubenswrapper[5052]: I1203 12:33:49.462816 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/92dadf05-a318-4491-bb1c-1fa23cb293fa-combined-ca-bundle\") pod \"ovn-northd-0\" (UID: \"92dadf05-a318-4491-bb1c-1fa23cb293fa\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:33:49 crc kubenswrapper[5052]: I1203 12:33:49.462921 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovn-northd-tls-certs\" (UniqueName: \"kubernetes.io/secret/92dadf05-a318-4491-bb1c-1fa23cb293fa-ovn-northd-tls-certs\") pod \"ovn-northd-0\" (UID: \"92dadf05-a318-4491-bb1c-1fa23cb293fa\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:33:49 crc kubenswrapper[5052]: I1203 12:33:49.462980 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/92dadf05-a318-4491-bb1c-1fa23cb293fa-metrics-certs-tls-certs\") pod \"ovn-northd-0\" (UID: \"92dadf05-a318-4491-bb1c-1fa23cb293fa\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:33:49 crc kubenswrapper[5052]: I1203 12:33:49.463005 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/92dadf05-a318-4491-bb1c-1fa23cb293fa-config\") pod \"ovn-northd-0\" (UID: \"92dadf05-a318-4491-bb1c-1fa23cb293fa\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:33:49 crc kubenswrapper[5052]: I1203 12:33:49.463040 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/92dadf05-a318-4491-bb1c-1fa23cb293fa-scripts\") pod \"ovn-northd-0\" (UID: \"92dadf05-a318-4491-bb1c-1fa23cb293fa\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:33:49 crc kubenswrapper[5052]: I1203 12:33:49.463073 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovn-rundir\" (UniqueName: \"kubernetes.io/empty-dir/92dadf05-a318-4491-bb1c-1fa23cb293fa-ovn-rundir\") pod \"ovn-northd-0\" (UID: \"92dadf05-a318-4491-bb1c-1fa23cb293fa\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:33:49 crc kubenswrapper[5052]: I1203 12:33:49.463102 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-xrncn\" (UniqueName: \"kubernetes.io/projected/92dadf05-a318-4491-bb1c-1fa23cb293fa-kube-api-access-xrncn\") pod \"ovn-northd-0\" (UID: \"92dadf05-a318-4491-bb1c-1fa23cb293fa\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:33:49 crc kubenswrapper[5052]: I1203 12:33:49.464064 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovn-rundir\" (UniqueName: \"kubernetes.io/empty-dir/92dadf05-a318-4491-bb1c-1fa23cb293fa-ovn-rundir\") pod \"ovn-northd-0\" (UID: \"92dadf05-a318-4491-bb1c-1fa23cb293fa\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:33:49 crc kubenswrapper[5052]: I1203 12:33:49.464100 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/92dadf05-a318-4491-bb1c-1fa23cb293fa-config\") pod \"ovn-northd-0\" (UID: \"92dadf05-a318-4491-bb1c-1fa23cb293fa\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:33:49 crc kubenswrapper[5052]: I1203 12:33:49.464410 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/92dadf05-a318-4491-bb1c-1fa23cb293fa-scripts\") pod \"ovn-northd-0\" (UID: \"92dadf05-a318-4491-bb1c-1fa23cb293fa\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:33:49 crc kubenswrapper[5052]: I1203 12:33:49.467762 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovn-northd-tls-certs\" (UniqueName: \"kubernetes.io/secret/92dadf05-a318-4491-bb1c-1fa23cb293fa-ovn-northd-tls-certs\") pod \"ovn-northd-0\" (UID: \"92dadf05-a318-4491-bb1c-1fa23cb293fa\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:33:49 crc kubenswrapper[5052]: I1203 12:33:49.467787 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/92dadf05-a318-4491-bb1c-1fa23cb293fa-combined-ca-bundle\") pod \"ovn-northd-0\" (UID: \"92dadf05-a318-4491-bb1c-1fa23cb293fa\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:33:49 crc kubenswrapper[5052]: I1203 12:33:49.468327 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/92dadf05-a318-4491-bb1c-1fa23cb293fa-metrics-certs-tls-certs\") pod \"ovn-northd-0\" (UID: \"92dadf05-a318-4491-bb1c-1fa23cb293fa\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:33:49 crc kubenswrapper[5052]: I1203 12:33:49.481656 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-xrncn\" (UniqueName: \"kubernetes.io/projected/92dadf05-a318-4491-bb1c-1fa23cb293fa-kube-api-access-xrncn\") pod \"ovn-northd-0\" (UID: \"92dadf05-a318-4491-bb1c-1fa23cb293fa\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:33:49 crc kubenswrapper[5052]: I1203 12:33:49.589933 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:33:50 crc kubenswrapper[5052]: I1203 12:33:50.077859 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ovn-northd-0"] Dec 03 12:33:50 crc kubenswrapper[5052]: I1203 12:33:50.386579 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovn-northd-0" event={"ID":"92dadf05-a318-4491-bb1c-1fa23cb293fa","Type":"ContainerStarted","Data":"e9b77740c5b7971ec0cda24631811a45f606f26233dd0f84362e91bcc82dc8c1"} Dec 03 12:33:51 crc kubenswrapper[5052]: I1203 12:33:51.191971 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/c7d598df-f336-4798-b381-9b813068f271-etc-swift\") pod \"swift-storage-0\" (UID: \"c7d598df-f336-4798-b381-9b813068f271\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:33:51 crc kubenswrapper[5052]: E1203 12:33:51.192152 5052 projected.go:288] Couldn't get configMap openstack-kuttl-tests/swift-ring-files: configmap "swift-ring-files" not found Dec 03 12:33:51 crc kubenswrapper[5052]: E1203 12:33:51.192407 5052 projected.go:194] Error preparing data for projected volume etc-swift for pod openstack-kuttl-tests/swift-storage-0: configmap "swift-ring-files" not found Dec 03 12:33:51 crc kubenswrapper[5052]: E1203 12:33:51.192463 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c7d598df-f336-4798-b381-9b813068f271-etc-swift podName:c7d598df-f336-4798-b381-9b813068f271 nodeName:}" failed. No retries permitted until 2025-12-03 12:33:55.192445573 +0000 UTC m=+4425.780687337 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "etc-swift" (UniqueName: "kubernetes.io/projected/c7d598df-f336-4798-b381-9b813068f271-etc-swift") pod "swift-storage-0" (UID: "c7d598df-f336-4798-b381-9b813068f271") : configmap "swift-ring-files" not found Dec 03 12:33:51 crc kubenswrapper[5052]: I1203 12:33:51.333568 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/swift-ring-rebalance-jstcn"] Dec 03 12:33:51 crc kubenswrapper[5052]: I1203 12:33:51.335140 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-ring-rebalance-jstcn" Dec 03 12:33:51 crc kubenswrapper[5052]: I1203 12:33:51.337091 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"swift-proxy-config-data" Dec 03 12:33:51 crc kubenswrapper[5052]: I1203 12:33:51.337389 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"swift-ring-config-data" Dec 03 12:33:51 crc kubenswrapper[5052]: I1203 12:33:51.337672 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"swift-ring-scripts" Dec 03 12:33:51 crc kubenswrapper[5052]: I1203 12:33:51.362747 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/swift-ring-rebalance-jstcn"] Dec 03 12:33:51 crc kubenswrapper[5052]: I1203 12:33:51.397731 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovn-northd-0" event={"ID":"92dadf05-a318-4491-bb1c-1fa23cb293fa","Type":"ContainerStarted","Data":"94a951ac5d9669865bf707580490a8df30f61838bbfebbf2fccdc24d07f308a3"} Dec 03 12:33:51 crc kubenswrapper[5052]: I1203 12:33:51.397802 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovn-northd-0" event={"ID":"92dadf05-a318-4491-bb1c-1fa23cb293fa","Type":"ContainerStarted","Data":"4e823788ee83fdbbba2177f783a9bb4380ce320612fdfe974cb17f48a899577b"} Dec 03 12:33:51 crc kubenswrapper[5052]: I1203 12:33:51.397898 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:33:51 crc kubenswrapper[5052]: I1203 12:33:51.501828 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f3af01db-f2de-4cd4-aeda-3cc200498cc7-combined-ca-bundle\") pod \"swift-ring-rebalance-jstcn\" (UID: \"f3af01db-f2de-4cd4-aeda-3cc200498cc7\") " pod="openstack-kuttl-tests/swift-ring-rebalance-jstcn" Dec 03 12:33:51 crc kubenswrapper[5052]: I1203 12:33:51.502079 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"swiftconf\" (UniqueName: \"kubernetes.io/secret/f3af01db-f2de-4cd4-aeda-3cc200498cc7-swiftconf\") pod \"swift-ring-rebalance-jstcn\" (UID: \"f3af01db-f2de-4cd4-aeda-3cc200498cc7\") " pod="openstack-kuttl-tests/swift-ring-rebalance-jstcn" Dec 03 12:33:51 crc kubenswrapper[5052]: I1203 12:33:51.502291 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dispersionconf\" (UniqueName: \"kubernetes.io/secret/f3af01db-f2de-4cd4-aeda-3cc200498cc7-dispersionconf\") pod \"swift-ring-rebalance-jstcn\" (UID: \"f3af01db-f2de-4cd4-aeda-3cc200498cc7\") " pod="openstack-kuttl-tests/swift-ring-rebalance-jstcn" Dec 03 12:33:51 crc kubenswrapper[5052]: I1203 12:33:51.502442 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gzh6c\" (UniqueName: \"kubernetes.io/projected/f3af01db-f2de-4cd4-aeda-3cc200498cc7-kube-api-access-gzh6c\") pod \"swift-ring-rebalance-jstcn\" (UID: \"f3af01db-f2de-4cd4-aeda-3cc200498cc7\") " pod="openstack-kuttl-tests/swift-ring-rebalance-jstcn" Dec 03 12:33:51 crc kubenswrapper[5052]: I1203 12:33:51.502553 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/empty-dir/f3af01db-f2de-4cd4-aeda-3cc200498cc7-etc-swift\") pod \"swift-ring-rebalance-jstcn\" (UID: \"f3af01db-f2de-4cd4-aeda-3cc200498cc7\") " pod="openstack-kuttl-tests/swift-ring-rebalance-jstcn" Dec 03 12:33:51 crc kubenswrapper[5052]: I1203 12:33:51.502630 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/f3af01db-f2de-4cd4-aeda-3cc200498cc7-scripts\") pod \"swift-ring-rebalance-jstcn\" (UID: \"f3af01db-f2de-4cd4-aeda-3cc200498cc7\") " pod="openstack-kuttl-tests/swift-ring-rebalance-jstcn" Dec 03 12:33:51 crc kubenswrapper[5052]: I1203 12:33:51.502722 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ring-data-devices\" (UniqueName: \"kubernetes.io/configmap/f3af01db-f2de-4cd4-aeda-3cc200498cc7-ring-data-devices\") pod \"swift-ring-rebalance-jstcn\" (UID: \"f3af01db-f2de-4cd4-aeda-3cc200498cc7\") " pod="openstack-kuttl-tests/swift-ring-rebalance-jstcn" Dec 03 12:33:51 crc kubenswrapper[5052]: I1203 12:33:51.603923 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/empty-dir/f3af01db-f2de-4cd4-aeda-3cc200498cc7-etc-swift\") pod \"swift-ring-rebalance-jstcn\" (UID: \"f3af01db-f2de-4cd4-aeda-3cc200498cc7\") " pod="openstack-kuttl-tests/swift-ring-rebalance-jstcn" Dec 03 12:33:51 crc kubenswrapper[5052]: I1203 12:33:51.603972 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/f3af01db-f2de-4cd4-aeda-3cc200498cc7-scripts\") pod \"swift-ring-rebalance-jstcn\" (UID: \"f3af01db-f2de-4cd4-aeda-3cc200498cc7\") " pod="openstack-kuttl-tests/swift-ring-rebalance-jstcn" Dec 03 12:33:51 crc kubenswrapper[5052]: I1203 12:33:51.604007 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ring-data-devices\" (UniqueName: \"kubernetes.io/configmap/f3af01db-f2de-4cd4-aeda-3cc200498cc7-ring-data-devices\") pod \"swift-ring-rebalance-jstcn\" (UID: \"f3af01db-f2de-4cd4-aeda-3cc200498cc7\") " pod="openstack-kuttl-tests/swift-ring-rebalance-jstcn" Dec 03 12:33:51 crc kubenswrapper[5052]: I1203 12:33:51.604055 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f3af01db-f2de-4cd4-aeda-3cc200498cc7-combined-ca-bundle\") pod \"swift-ring-rebalance-jstcn\" (UID: \"f3af01db-f2de-4cd4-aeda-3cc200498cc7\") " pod="openstack-kuttl-tests/swift-ring-rebalance-jstcn" Dec 03 12:33:51 crc kubenswrapper[5052]: I1203 12:33:51.604123 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"swiftconf\" (UniqueName: \"kubernetes.io/secret/f3af01db-f2de-4cd4-aeda-3cc200498cc7-swiftconf\") pod \"swift-ring-rebalance-jstcn\" (UID: \"f3af01db-f2de-4cd4-aeda-3cc200498cc7\") " pod="openstack-kuttl-tests/swift-ring-rebalance-jstcn" Dec 03 12:33:51 crc kubenswrapper[5052]: I1203 12:33:51.604141 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"dispersionconf\" (UniqueName: \"kubernetes.io/secret/f3af01db-f2de-4cd4-aeda-3cc200498cc7-dispersionconf\") pod \"swift-ring-rebalance-jstcn\" (UID: \"f3af01db-f2de-4cd4-aeda-3cc200498cc7\") " pod="openstack-kuttl-tests/swift-ring-rebalance-jstcn" Dec 03 12:33:51 crc kubenswrapper[5052]: I1203 12:33:51.604161 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-gzh6c\" (UniqueName: \"kubernetes.io/projected/f3af01db-f2de-4cd4-aeda-3cc200498cc7-kube-api-access-gzh6c\") pod \"swift-ring-rebalance-jstcn\" (UID: \"f3af01db-f2de-4cd4-aeda-3cc200498cc7\") " pod="openstack-kuttl-tests/swift-ring-rebalance-jstcn" Dec 03 12:33:51 crc kubenswrapper[5052]: I1203 12:33:51.604315 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"etc-swift\" (UniqueName: \"kubernetes.io/empty-dir/f3af01db-f2de-4cd4-aeda-3cc200498cc7-etc-swift\") pod \"swift-ring-rebalance-jstcn\" (UID: \"f3af01db-f2de-4cd4-aeda-3cc200498cc7\") " pod="openstack-kuttl-tests/swift-ring-rebalance-jstcn" Dec 03 12:33:51 crc kubenswrapper[5052]: I1203 12:33:51.604854 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/f3af01db-f2de-4cd4-aeda-3cc200498cc7-scripts\") pod \"swift-ring-rebalance-jstcn\" (UID: \"f3af01db-f2de-4cd4-aeda-3cc200498cc7\") " pod="openstack-kuttl-tests/swift-ring-rebalance-jstcn" Dec 03 12:33:51 crc kubenswrapper[5052]: I1203 12:33:51.605247 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ring-data-devices\" (UniqueName: \"kubernetes.io/configmap/f3af01db-f2de-4cd4-aeda-3cc200498cc7-ring-data-devices\") pod \"swift-ring-rebalance-jstcn\" (UID: \"f3af01db-f2de-4cd4-aeda-3cc200498cc7\") " pod="openstack-kuttl-tests/swift-ring-rebalance-jstcn" Dec 03 12:33:51 crc kubenswrapper[5052]: I1203 12:33:51.609216 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"swiftconf\" (UniqueName: \"kubernetes.io/secret/f3af01db-f2de-4cd4-aeda-3cc200498cc7-swiftconf\") pod \"swift-ring-rebalance-jstcn\" (UID: \"f3af01db-f2de-4cd4-aeda-3cc200498cc7\") " pod="openstack-kuttl-tests/swift-ring-rebalance-jstcn" Dec 03 12:33:51 crc kubenswrapper[5052]: I1203 12:33:51.609395 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f3af01db-f2de-4cd4-aeda-3cc200498cc7-combined-ca-bundle\") pod \"swift-ring-rebalance-jstcn\" (UID: \"f3af01db-f2de-4cd4-aeda-3cc200498cc7\") " pod="openstack-kuttl-tests/swift-ring-rebalance-jstcn" Dec 03 12:33:51 crc kubenswrapper[5052]: I1203 12:33:51.614732 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"dispersionconf\" (UniqueName: \"kubernetes.io/secret/f3af01db-f2de-4cd4-aeda-3cc200498cc7-dispersionconf\") pod \"swift-ring-rebalance-jstcn\" (UID: \"f3af01db-f2de-4cd4-aeda-3cc200498cc7\") " pod="openstack-kuttl-tests/swift-ring-rebalance-jstcn" Dec 03 12:33:51 crc kubenswrapper[5052]: I1203 12:33:51.625707 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-gzh6c\" (UniqueName: \"kubernetes.io/projected/f3af01db-f2de-4cd4-aeda-3cc200498cc7-kube-api-access-gzh6c\") pod \"swift-ring-rebalance-jstcn\" (UID: \"f3af01db-f2de-4cd4-aeda-3cc200498cc7\") " pod="openstack-kuttl-tests/swift-ring-rebalance-jstcn" Dec 03 12:33:51 crc kubenswrapper[5052]: I1203 12:33:51.663626 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-ring-rebalance-jstcn" Dec 03 12:33:52 crc kubenswrapper[5052]: I1203 12:33:52.068357 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/ovn-northd-0" podStartSLOduration=3.068339173 podStartE2EDuration="3.068339173s" podCreationTimestamp="2025-12-03 12:33:49 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:33:51.426221905 +0000 UTC m=+4422.014463669" watchObservedRunningTime="2025-12-03 12:33:52.068339173 +0000 UTC m=+4422.656580937" Dec 03 12:33:52 crc kubenswrapper[5052]: I1203 12:33:52.076911 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/swift-ring-rebalance-jstcn"] Dec 03 12:33:52 crc kubenswrapper[5052]: W1203 12:33:52.228083 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf3af01db_f2de_4cd4_aeda_3cc200498cc7.slice/crio-d7d949b9ba6f9e4269ffc313b1a08bcd56301d0aaa692641607ba1a2c0bd17f0 WatchSource:0}: Error finding container d7d949b9ba6f9e4269ffc313b1a08bcd56301d0aaa692641607ba1a2c0bd17f0: Status 404 returned error can't find the container with id d7d949b9ba6f9e4269ffc313b1a08bcd56301d0aaa692641607ba1a2c0bd17f0 Dec 03 12:33:52 crc kubenswrapper[5052]: I1203 12:33:52.405728 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-ring-rebalance-jstcn" event={"ID":"f3af01db-f2de-4cd4-aeda-3cc200498cc7","Type":"ContainerStarted","Data":"d7d949b9ba6f9e4269ffc313b1a08bcd56301d0aaa692641607ba1a2c0bd17f0"} Dec 03 12:33:53 crc kubenswrapper[5052]: I1203 12:33:53.416183 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-ring-rebalance-jstcn" event={"ID":"f3af01db-f2de-4cd4-aeda-3cc200498cc7","Type":"ContainerStarted","Data":"0f4cc658cedc4570aad8a710600353eb501f638bf059e857fdb8c388153955ff"} Dec 03 12:33:53 crc kubenswrapper[5052]: I1203 12:33:53.439879 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/swift-ring-rebalance-jstcn" podStartSLOduration=2.439860688 podStartE2EDuration="2.439860688s" podCreationTimestamp="2025-12-03 12:33:51 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:33:53.432371524 +0000 UTC m=+4424.020613288" watchObservedRunningTime="2025-12-03 12:33:53.439860688 +0000 UTC m=+4424.028102452" Dec 03 12:33:54 crc kubenswrapper[5052]: I1203 12:33:54.194156 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/keystone-db-create-mvj42"] Dec 03 12:33:54 crc kubenswrapper[5052]: I1203 12:33:54.195984 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-db-create-mvj42" Dec 03 12:33:54 crc kubenswrapper[5052]: I1203 12:33:54.221795 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-db-create-mvj42"] Dec 03 12:33:54 crc kubenswrapper[5052]: I1203 12:33:54.305957 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/keystone-4ba6-account-create-update-nr8wd"] Dec 03 12:33:54 crc kubenswrapper[5052]: I1203 12:33:54.307351 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-4ba6-account-create-update-nr8wd" Dec 03 12:33:54 crc kubenswrapper[5052]: I1203 12:33:54.310088 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone-db-secret" Dec 03 12:33:54 crc kubenswrapper[5052]: I1203 12:33:54.323094 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-4ba6-account-create-update-nr8wd"] Dec 03 12:33:54 crc kubenswrapper[5052]: I1203 12:33:54.352411 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/8179fa49-7bb4-4c12-b863-a080cbabb765-operator-scripts\") pod \"keystone-db-create-mvj42\" (UID: \"8179fa49-7bb4-4c12-b863-a080cbabb765\") " pod="openstack-kuttl-tests/keystone-db-create-mvj42" Dec 03 12:33:54 crc kubenswrapper[5052]: I1203 12:33:54.352703 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-45s52\" (UniqueName: \"kubernetes.io/projected/8179fa49-7bb4-4c12-b863-a080cbabb765-kube-api-access-45s52\") pod \"keystone-db-create-mvj42\" (UID: \"8179fa49-7bb4-4c12-b863-a080cbabb765\") " pod="openstack-kuttl-tests/keystone-db-create-mvj42" Dec 03 12:33:54 crc kubenswrapper[5052]: I1203 12:33:54.454465 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-45s52\" (UniqueName: \"kubernetes.io/projected/8179fa49-7bb4-4c12-b863-a080cbabb765-kube-api-access-45s52\") pod \"keystone-db-create-mvj42\" (UID: \"8179fa49-7bb4-4c12-b863-a080cbabb765\") " pod="openstack-kuttl-tests/keystone-db-create-mvj42" Dec 03 12:33:54 crc kubenswrapper[5052]: I1203 12:33:54.454614 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/8179fa49-7bb4-4c12-b863-a080cbabb765-operator-scripts\") pod \"keystone-db-create-mvj42\" (UID: \"8179fa49-7bb4-4c12-b863-a080cbabb765\") " pod="openstack-kuttl-tests/keystone-db-create-mvj42" Dec 03 12:33:54 crc kubenswrapper[5052]: I1203 12:33:54.454688 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m8fdl\" (UniqueName: \"kubernetes.io/projected/13e6f571-4161-442d-b45e-f30f6ea05843-kube-api-access-m8fdl\") pod \"keystone-4ba6-account-create-update-nr8wd\" (UID: \"13e6f571-4161-442d-b45e-f30f6ea05843\") " pod="openstack-kuttl-tests/keystone-4ba6-account-create-update-nr8wd" Dec 03 12:33:54 crc kubenswrapper[5052]: I1203 12:33:54.454787 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/13e6f571-4161-442d-b45e-f30f6ea05843-operator-scripts\") pod \"keystone-4ba6-account-create-update-nr8wd\" (UID: \"13e6f571-4161-442d-b45e-f30f6ea05843\") " pod="openstack-kuttl-tests/keystone-4ba6-account-create-update-nr8wd" Dec 03 12:33:54 crc kubenswrapper[5052]: I1203 12:33:54.455652 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/8179fa49-7bb4-4c12-b863-a080cbabb765-operator-scripts\") pod \"keystone-db-create-mvj42\" (UID: \"8179fa49-7bb4-4c12-b863-a080cbabb765\") " pod="openstack-kuttl-tests/keystone-db-create-mvj42" Dec 03 12:33:54 crc kubenswrapper[5052]: I1203 12:33:54.475113 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-45s52\" (UniqueName: \"kubernetes.io/projected/8179fa49-7bb4-4c12-b863-a080cbabb765-kube-api-access-45s52\") pod \"keystone-db-create-mvj42\" (UID: \"8179fa49-7bb4-4c12-b863-a080cbabb765\") " pod="openstack-kuttl-tests/keystone-db-create-mvj42" Dec 03 12:33:54 crc kubenswrapper[5052]: I1203 12:33:54.520487 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-db-create-mvj42" Dec 03 12:33:54 crc kubenswrapper[5052]: I1203 12:33:54.556445 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-m8fdl\" (UniqueName: \"kubernetes.io/projected/13e6f571-4161-442d-b45e-f30f6ea05843-kube-api-access-m8fdl\") pod \"keystone-4ba6-account-create-update-nr8wd\" (UID: \"13e6f571-4161-442d-b45e-f30f6ea05843\") " pod="openstack-kuttl-tests/keystone-4ba6-account-create-update-nr8wd" Dec 03 12:33:54 crc kubenswrapper[5052]: I1203 12:33:54.556588 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/13e6f571-4161-442d-b45e-f30f6ea05843-operator-scripts\") pod \"keystone-4ba6-account-create-update-nr8wd\" (UID: \"13e6f571-4161-442d-b45e-f30f6ea05843\") " pod="openstack-kuttl-tests/keystone-4ba6-account-create-update-nr8wd" Dec 03 12:33:54 crc kubenswrapper[5052]: I1203 12:33:54.557591 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/13e6f571-4161-442d-b45e-f30f6ea05843-operator-scripts\") pod \"keystone-4ba6-account-create-update-nr8wd\" (UID: \"13e6f571-4161-442d-b45e-f30f6ea05843\") " pod="openstack-kuttl-tests/keystone-4ba6-account-create-update-nr8wd" Dec 03 12:33:54 crc kubenswrapper[5052]: I1203 12:33:54.581116 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-m8fdl\" (UniqueName: \"kubernetes.io/projected/13e6f571-4161-442d-b45e-f30f6ea05843-kube-api-access-m8fdl\") pod \"keystone-4ba6-account-create-update-nr8wd\" (UID: \"13e6f571-4161-442d-b45e-f30f6ea05843\") " pod="openstack-kuttl-tests/keystone-4ba6-account-create-update-nr8wd" Dec 03 12:33:54 crc kubenswrapper[5052]: I1203 12:33:54.622434 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-4ba6-account-create-update-nr8wd" Dec 03 12:33:54 crc kubenswrapper[5052]: I1203 12:33:54.956553 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/glance-db-sync-rxhl2"] Dec 03 12:33:54 crc kubenswrapper[5052]: I1203 12:33:54.958295 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-db-sync-rxhl2" Dec 03 12:33:54 crc kubenswrapper[5052]: I1203 12:33:54.960131 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-config-data" Dec 03 12:33:54 crc kubenswrapper[5052]: I1203 12:33:54.962288 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-glance-dockercfg-xkmzv" Dec 03 12:33:54 crc kubenswrapper[5052]: I1203 12:33:54.963143 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-db-sync-rxhl2"] Dec 03 12:33:55 crc kubenswrapper[5052]: I1203 12:33:55.032438 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-db-create-mvj42"] Dec 03 12:33:55 crc kubenswrapper[5052]: W1203 12:33:55.037252 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8179fa49_7bb4_4c12_b863_a080cbabb765.slice/crio-56facf6f938ce71404d322a0669d3afa06594c2e1a7f6ddf21d3a52b8dab8804 WatchSource:0}: Error finding container 56facf6f938ce71404d322a0669d3afa06594c2e1a7f6ddf21d3a52b8dab8804: Status 404 returned error can't find the container with id 56facf6f938ce71404d322a0669d3afa06594c2e1a7f6ddf21d3a52b8dab8804 Dec 03 12:33:55 crc kubenswrapper[5052]: I1203 12:33:55.065105 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/6cc1f6de-7848-429c-9f14-ea763a359583-db-sync-config-data\") pod \"glance-db-sync-rxhl2\" (UID: \"6cc1f6de-7848-429c-9f14-ea763a359583\") " pod="openstack-kuttl-tests/glance-db-sync-rxhl2" Dec 03 12:33:55 crc kubenswrapper[5052]: I1203 12:33:55.065262 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6cc1f6de-7848-429c-9f14-ea763a359583-combined-ca-bundle\") pod \"glance-db-sync-rxhl2\" (UID: \"6cc1f6de-7848-429c-9f14-ea763a359583\") " pod="openstack-kuttl-tests/glance-db-sync-rxhl2" Dec 03 12:33:55 crc kubenswrapper[5052]: I1203 12:33:55.065298 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7d4kb\" (UniqueName: \"kubernetes.io/projected/6cc1f6de-7848-429c-9f14-ea763a359583-kube-api-access-7d4kb\") pod \"glance-db-sync-rxhl2\" (UID: \"6cc1f6de-7848-429c-9f14-ea763a359583\") " pod="openstack-kuttl-tests/glance-db-sync-rxhl2" Dec 03 12:33:55 crc kubenswrapper[5052]: I1203 12:33:55.065375 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6cc1f6de-7848-429c-9f14-ea763a359583-config-data\") pod \"glance-db-sync-rxhl2\" (UID: \"6cc1f6de-7848-429c-9f14-ea763a359583\") " pod="openstack-kuttl-tests/glance-db-sync-rxhl2" Dec 03 12:33:55 crc kubenswrapper[5052]: I1203 12:33:55.149966 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-4ba6-account-create-update-nr8wd"] Dec 03 12:33:55 crc kubenswrapper[5052]: I1203 12:33:55.166626 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6cc1f6de-7848-429c-9f14-ea763a359583-combined-ca-bundle\") pod \"glance-db-sync-rxhl2\" (UID: \"6cc1f6de-7848-429c-9f14-ea763a359583\") " pod="openstack-kuttl-tests/glance-db-sync-rxhl2" Dec 03 12:33:55 crc kubenswrapper[5052]: I1203 12:33:55.166686 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-7d4kb\" (UniqueName: \"kubernetes.io/projected/6cc1f6de-7848-429c-9f14-ea763a359583-kube-api-access-7d4kb\") pod \"glance-db-sync-rxhl2\" (UID: \"6cc1f6de-7848-429c-9f14-ea763a359583\") " pod="openstack-kuttl-tests/glance-db-sync-rxhl2" Dec 03 12:33:55 crc kubenswrapper[5052]: I1203 12:33:55.166761 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6cc1f6de-7848-429c-9f14-ea763a359583-config-data\") pod \"glance-db-sync-rxhl2\" (UID: \"6cc1f6de-7848-429c-9f14-ea763a359583\") " pod="openstack-kuttl-tests/glance-db-sync-rxhl2" Dec 03 12:33:55 crc kubenswrapper[5052]: I1203 12:33:55.166807 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/6cc1f6de-7848-429c-9f14-ea763a359583-db-sync-config-data\") pod \"glance-db-sync-rxhl2\" (UID: \"6cc1f6de-7848-429c-9f14-ea763a359583\") " pod="openstack-kuttl-tests/glance-db-sync-rxhl2" Dec 03 12:33:55 crc kubenswrapper[5052]: I1203 12:33:55.172690 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/6cc1f6de-7848-429c-9f14-ea763a359583-db-sync-config-data\") pod \"glance-db-sync-rxhl2\" (UID: \"6cc1f6de-7848-429c-9f14-ea763a359583\") " pod="openstack-kuttl-tests/glance-db-sync-rxhl2" Dec 03 12:33:55 crc kubenswrapper[5052]: I1203 12:33:55.172703 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6cc1f6de-7848-429c-9f14-ea763a359583-combined-ca-bundle\") pod \"glance-db-sync-rxhl2\" (UID: \"6cc1f6de-7848-429c-9f14-ea763a359583\") " pod="openstack-kuttl-tests/glance-db-sync-rxhl2" Dec 03 12:33:55 crc kubenswrapper[5052]: I1203 12:33:55.172845 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6cc1f6de-7848-429c-9f14-ea763a359583-config-data\") pod \"glance-db-sync-rxhl2\" (UID: \"6cc1f6de-7848-429c-9f14-ea763a359583\") " pod="openstack-kuttl-tests/glance-db-sync-rxhl2" Dec 03 12:33:55 crc kubenswrapper[5052]: I1203 12:33:55.184892 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-7d4kb\" (UniqueName: \"kubernetes.io/projected/6cc1f6de-7848-429c-9f14-ea763a359583-kube-api-access-7d4kb\") pod \"glance-db-sync-rxhl2\" (UID: \"6cc1f6de-7848-429c-9f14-ea763a359583\") " pod="openstack-kuttl-tests/glance-db-sync-rxhl2" Dec 03 12:33:55 crc kubenswrapper[5052]: I1203 12:33:55.268671 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/c7d598df-f336-4798-b381-9b813068f271-etc-swift\") pod \"swift-storage-0\" (UID: \"c7d598df-f336-4798-b381-9b813068f271\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:33:55 crc kubenswrapper[5052]: E1203 12:33:55.269410 5052 projected.go:288] Couldn't get configMap openstack-kuttl-tests/swift-ring-files: configmap "swift-ring-files" not found Dec 03 12:33:55 crc kubenswrapper[5052]: E1203 12:33:55.269428 5052 projected.go:194] Error preparing data for projected volume etc-swift for pod openstack-kuttl-tests/swift-storage-0: configmap "swift-ring-files" not found Dec 03 12:33:55 crc kubenswrapper[5052]: E1203 12:33:55.269480 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c7d598df-f336-4798-b381-9b813068f271-etc-swift podName:c7d598df-f336-4798-b381-9b813068f271 nodeName:}" failed. No retries permitted until 2025-12-03 12:34:03.269462729 +0000 UTC m=+4433.857704503 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "etc-swift" (UniqueName: "kubernetes.io/projected/c7d598df-f336-4798-b381-9b813068f271-etc-swift") pod "swift-storage-0" (UID: "c7d598df-f336-4798-b381-9b813068f271") : configmap "swift-ring-files" not found Dec 03 12:33:55 crc kubenswrapper[5052]: I1203 12:33:55.288800 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-db-sync-rxhl2" Dec 03 12:33:55 crc kubenswrapper[5052]: I1203 12:33:55.436486 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-4ba6-account-create-update-nr8wd" event={"ID":"13e6f571-4161-442d-b45e-f30f6ea05843","Type":"ContainerStarted","Data":"37205114d12d1cc0882510cbd6d7f46eb0e500325b061ad47f151d3e67478480"} Dec 03 12:33:55 crc kubenswrapper[5052]: I1203 12:33:55.436546 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-4ba6-account-create-update-nr8wd" event={"ID":"13e6f571-4161-442d-b45e-f30f6ea05843","Type":"ContainerStarted","Data":"4bef600bcf18c8b70b73ea354812aeb2423cd1975e82e86266dcda34c08a29a2"} Dec 03 12:33:55 crc kubenswrapper[5052]: I1203 12:33:55.438245 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-db-create-mvj42" event={"ID":"8179fa49-7bb4-4c12-b863-a080cbabb765","Type":"ContainerStarted","Data":"ee2f15d5cf56ceed957dec77523e3537f6db3f8aa6f0663cfe5da91979d7d23c"} Dec 03 12:33:55 crc kubenswrapper[5052]: I1203 12:33:55.438291 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-db-create-mvj42" event={"ID":"8179fa49-7bb4-4c12-b863-a080cbabb765","Type":"ContainerStarted","Data":"56facf6f938ce71404d322a0669d3afa06594c2e1a7f6ddf21d3a52b8dab8804"} Dec 03 12:33:55 crc kubenswrapper[5052]: I1203 12:33:55.464052 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/keystone-4ba6-account-create-update-nr8wd" podStartSLOduration=1.464025663 podStartE2EDuration="1.464025663s" podCreationTimestamp="2025-12-03 12:33:54 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:33:55.45869163 +0000 UTC m=+4426.046933394" watchObservedRunningTime="2025-12-03 12:33:55.464025663 +0000 UTC m=+4426.052267427" Dec 03 12:33:55 crc kubenswrapper[5052]: I1203 12:33:55.487072 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/keystone-db-create-mvj42" podStartSLOduration=1.4870461 podStartE2EDuration="1.4870461s" podCreationTimestamp="2025-12-03 12:33:54 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:33:55.473771821 +0000 UTC m=+4426.062013585" watchObservedRunningTime="2025-12-03 12:33:55.4870461 +0000 UTC m=+4426.075287864" Dec 03 12:33:55 crc kubenswrapper[5052]: I1203 12:33:55.774246 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-db-sync-rxhl2"] Dec 03 12:33:56 crc kubenswrapper[5052]: I1203 12:33:56.450186 5052 generic.go:334] "Generic (PLEG): container finished" podID="8179fa49-7bb4-4c12-b863-a080cbabb765" containerID="ee2f15d5cf56ceed957dec77523e3537f6db3f8aa6f0663cfe5da91979d7d23c" exitCode=0 Dec 03 12:33:56 crc kubenswrapper[5052]: I1203 12:33:56.450313 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-db-create-mvj42" event={"ID":"8179fa49-7bb4-4c12-b863-a080cbabb765","Type":"ContainerDied","Data":"ee2f15d5cf56ceed957dec77523e3537f6db3f8aa6f0663cfe5da91979d7d23c"} Dec 03 12:33:56 crc kubenswrapper[5052]: I1203 12:33:56.454416 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-db-sync-rxhl2" event={"ID":"6cc1f6de-7848-429c-9f14-ea763a359583","Type":"ContainerStarted","Data":"2253171c4c8bd199a61de0bcd7cfe694d9c538afb521022af5f3e5fd80739aea"} Dec 03 12:33:56 crc kubenswrapper[5052]: I1203 12:33:56.454461 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-db-sync-rxhl2" event={"ID":"6cc1f6de-7848-429c-9f14-ea763a359583","Type":"ContainerStarted","Data":"8d1bd1d3b2d34ed0fd4a5869ac1d601db57c94c26aa27738af8b3e5f6f1c6b44"} Dec 03 12:33:56 crc kubenswrapper[5052]: I1203 12:33:56.457215 5052 generic.go:334] "Generic (PLEG): container finished" podID="13e6f571-4161-442d-b45e-f30f6ea05843" containerID="37205114d12d1cc0882510cbd6d7f46eb0e500325b061ad47f151d3e67478480" exitCode=0 Dec 03 12:33:56 crc kubenswrapper[5052]: I1203 12:33:56.457254 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-4ba6-account-create-update-nr8wd" event={"ID":"13e6f571-4161-442d-b45e-f30f6ea05843","Type":"ContainerDied","Data":"37205114d12d1cc0882510cbd6d7f46eb0e500325b061ad47f151d3e67478480"} Dec 03 12:33:56 crc kubenswrapper[5052]: I1203 12:33:56.484987 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/glance-db-sync-rxhl2" podStartSLOduration=2.484965942 podStartE2EDuration="2.484965942s" podCreationTimestamp="2025-12-03 12:33:54 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:33:56.478496498 +0000 UTC m=+4427.066738262" watchObservedRunningTime="2025-12-03 12:33:56.484965942 +0000 UTC m=+4427.073207706" Dec 03 12:33:56 crc kubenswrapper[5052]: I1203 12:33:56.673076 5052 scope.go:117] "RemoveContainer" containerID="e6edaf188f85ea4f11f487f11cc489c700f25fb42b5ee269f4f496f1e127d772" Dec 03 12:33:56 crc kubenswrapper[5052]: E1203 12:33:56.673362 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 12:33:57 crc kubenswrapper[5052]: I1203 12:33:57.833285 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-db-create-mvj42" Dec 03 12:33:57 crc kubenswrapper[5052]: I1203 12:33:57.908856 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-4ba6-account-create-update-nr8wd" Dec 03 12:33:58 crc kubenswrapper[5052]: I1203 12:33:58.027235 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-45s52\" (UniqueName: \"kubernetes.io/projected/8179fa49-7bb4-4c12-b863-a080cbabb765-kube-api-access-45s52\") pod \"8179fa49-7bb4-4c12-b863-a080cbabb765\" (UID: \"8179fa49-7bb4-4c12-b863-a080cbabb765\") " Dec 03 12:33:58 crc kubenswrapper[5052]: I1203 12:33:58.027300 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/8179fa49-7bb4-4c12-b863-a080cbabb765-operator-scripts\") pod \"8179fa49-7bb4-4c12-b863-a080cbabb765\" (UID: \"8179fa49-7bb4-4c12-b863-a080cbabb765\") " Dec 03 12:33:58 crc kubenswrapper[5052]: I1203 12:33:58.027321 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/13e6f571-4161-442d-b45e-f30f6ea05843-operator-scripts\") pod \"13e6f571-4161-442d-b45e-f30f6ea05843\" (UID: \"13e6f571-4161-442d-b45e-f30f6ea05843\") " Dec 03 12:33:58 crc kubenswrapper[5052]: I1203 12:33:58.027448 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m8fdl\" (UniqueName: \"kubernetes.io/projected/13e6f571-4161-442d-b45e-f30f6ea05843-kube-api-access-m8fdl\") pod \"13e6f571-4161-442d-b45e-f30f6ea05843\" (UID: \"13e6f571-4161-442d-b45e-f30f6ea05843\") " Dec 03 12:33:58 crc kubenswrapper[5052]: I1203 12:33:58.028250 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8179fa49-7bb4-4c12-b863-a080cbabb765-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "8179fa49-7bb4-4c12-b863-a080cbabb765" (UID: "8179fa49-7bb4-4c12-b863-a080cbabb765"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:33:58 crc kubenswrapper[5052]: I1203 12:33:58.028249 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/13e6f571-4161-442d-b45e-f30f6ea05843-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "13e6f571-4161-442d-b45e-f30f6ea05843" (UID: "13e6f571-4161-442d-b45e-f30f6ea05843"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:33:58 crc kubenswrapper[5052]: I1203 12:33:58.032970 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/13e6f571-4161-442d-b45e-f30f6ea05843-kube-api-access-m8fdl" (OuterVolumeSpecName: "kube-api-access-m8fdl") pod "13e6f571-4161-442d-b45e-f30f6ea05843" (UID: "13e6f571-4161-442d-b45e-f30f6ea05843"). InnerVolumeSpecName "kube-api-access-m8fdl". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:33:58 crc kubenswrapper[5052]: I1203 12:33:58.033324 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8179fa49-7bb4-4c12-b863-a080cbabb765-kube-api-access-45s52" (OuterVolumeSpecName: "kube-api-access-45s52") pod "8179fa49-7bb4-4c12-b863-a080cbabb765" (UID: "8179fa49-7bb4-4c12-b863-a080cbabb765"). InnerVolumeSpecName "kube-api-access-45s52". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:33:58 crc kubenswrapper[5052]: I1203 12:33:58.129416 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-45s52\" (UniqueName: \"kubernetes.io/projected/8179fa49-7bb4-4c12-b863-a080cbabb765-kube-api-access-45s52\") on node \"crc\" DevicePath \"\"" Dec 03 12:33:58 crc kubenswrapper[5052]: I1203 12:33:58.129455 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/8179fa49-7bb4-4c12-b863-a080cbabb765-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:33:58 crc kubenswrapper[5052]: I1203 12:33:58.129465 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/13e6f571-4161-442d-b45e-f30f6ea05843-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:33:58 crc kubenswrapper[5052]: I1203 12:33:58.129474 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-m8fdl\" (UniqueName: \"kubernetes.io/projected/13e6f571-4161-442d-b45e-f30f6ea05843-kube-api-access-m8fdl\") on node \"crc\" DevicePath \"\"" Dec 03 12:33:58 crc kubenswrapper[5052]: I1203 12:33:58.494466 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-db-create-mvj42" event={"ID":"8179fa49-7bb4-4c12-b863-a080cbabb765","Type":"ContainerDied","Data":"56facf6f938ce71404d322a0669d3afa06594c2e1a7f6ddf21d3a52b8dab8804"} Dec 03 12:33:58 crc kubenswrapper[5052]: I1203 12:33:58.494506 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="56facf6f938ce71404d322a0669d3afa06594c2e1a7f6ddf21d3a52b8dab8804" Dec 03 12:33:58 crc kubenswrapper[5052]: I1203 12:33:58.494577 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-db-create-mvj42" Dec 03 12:33:58 crc kubenswrapper[5052]: I1203 12:33:58.504156 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-4ba6-account-create-update-nr8wd" event={"ID":"13e6f571-4161-442d-b45e-f30f6ea05843","Type":"ContainerDied","Data":"4bef600bcf18c8b70b73ea354812aeb2423cd1975e82e86266dcda34c08a29a2"} Dec 03 12:33:58 crc kubenswrapper[5052]: I1203 12:33:58.504239 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4bef600bcf18c8b70b73ea354812aeb2423cd1975e82e86266dcda34c08a29a2" Dec 03 12:33:58 crc kubenswrapper[5052]: I1203 12:33:58.504347 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-4ba6-account-create-update-nr8wd" Dec 03 12:33:59 crc kubenswrapper[5052]: I1203 12:33:59.520256 5052 generic.go:334] "Generic (PLEG): container finished" podID="f3af01db-f2de-4cd4-aeda-3cc200498cc7" containerID="0f4cc658cedc4570aad8a710600353eb501f638bf059e857fdb8c388153955ff" exitCode=0 Dec 03 12:33:59 crc kubenswrapper[5052]: I1203 12:33:59.521645 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-ring-rebalance-jstcn" event={"ID":"f3af01db-f2de-4cd4-aeda-3cc200498cc7","Type":"ContainerDied","Data":"0f4cc658cedc4570aad8a710600353eb501f638bf059e857fdb8c388153955ff"} Dec 03 12:34:00 crc kubenswrapper[5052]: I1203 12:34:00.918712 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-ring-rebalance-jstcn" Dec 03 12:34:01 crc kubenswrapper[5052]: I1203 12:34:01.088097 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gzh6c\" (UniqueName: \"kubernetes.io/projected/f3af01db-f2de-4cd4-aeda-3cc200498cc7-kube-api-access-gzh6c\") pod \"f3af01db-f2de-4cd4-aeda-3cc200498cc7\" (UID: \"f3af01db-f2de-4cd4-aeda-3cc200498cc7\") " Dec 03 12:34:01 crc kubenswrapper[5052]: I1203 12:34:01.088183 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"dispersionconf\" (UniqueName: \"kubernetes.io/secret/f3af01db-f2de-4cd4-aeda-3cc200498cc7-dispersionconf\") pod \"f3af01db-f2de-4cd4-aeda-3cc200498cc7\" (UID: \"f3af01db-f2de-4cd4-aeda-3cc200498cc7\") " Dec 03 12:34:01 crc kubenswrapper[5052]: I1203 12:34:01.088211 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f3af01db-f2de-4cd4-aeda-3cc200498cc7-combined-ca-bundle\") pod \"f3af01db-f2de-4cd4-aeda-3cc200498cc7\" (UID: \"f3af01db-f2de-4cd4-aeda-3cc200498cc7\") " Dec 03 12:34:01 crc kubenswrapper[5052]: I1203 12:34:01.088239 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/empty-dir/f3af01db-f2de-4cd4-aeda-3cc200498cc7-etc-swift\") pod \"f3af01db-f2de-4cd4-aeda-3cc200498cc7\" (UID: \"f3af01db-f2de-4cd4-aeda-3cc200498cc7\") " Dec 03 12:34:01 crc kubenswrapper[5052]: I1203 12:34:01.088393 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/f3af01db-f2de-4cd4-aeda-3cc200498cc7-scripts\") pod \"f3af01db-f2de-4cd4-aeda-3cc200498cc7\" (UID: \"f3af01db-f2de-4cd4-aeda-3cc200498cc7\") " Dec 03 12:34:01 crc kubenswrapper[5052]: I1203 12:34:01.088456 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"swiftconf\" (UniqueName: \"kubernetes.io/secret/f3af01db-f2de-4cd4-aeda-3cc200498cc7-swiftconf\") pod \"f3af01db-f2de-4cd4-aeda-3cc200498cc7\" (UID: \"f3af01db-f2de-4cd4-aeda-3cc200498cc7\") " Dec 03 12:34:01 crc kubenswrapper[5052]: I1203 12:34:01.088483 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ring-data-devices\" (UniqueName: \"kubernetes.io/configmap/f3af01db-f2de-4cd4-aeda-3cc200498cc7-ring-data-devices\") pod \"f3af01db-f2de-4cd4-aeda-3cc200498cc7\" (UID: \"f3af01db-f2de-4cd4-aeda-3cc200498cc7\") " Dec 03 12:34:01 crc kubenswrapper[5052]: I1203 12:34:01.089390 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f3af01db-f2de-4cd4-aeda-3cc200498cc7-etc-swift" (OuterVolumeSpecName: "etc-swift") pod "f3af01db-f2de-4cd4-aeda-3cc200498cc7" (UID: "f3af01db-f2de-4cd4-aeda-3cc200498cc7"). InnerVolumeSpecName "etc-swift". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:34:01 crc kubenswrapper[5052]: I1203 12:34:01.089338 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f3af01db-f2de-4cd4-aeda-3cc200498cc7-ring-data-devices" (OuterVolumeSpecName: "ring-data-devices") pod "f3af01db-f2de-4cd4-aeda-3cc200498cc7" (UID: "f3af01db-f2de-4cd4-aeda-3cc200498cc7"). InnerVolumeSpecName "ring-data-devices". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:34:01 crc kubenswrapper[5052]: I1203 12:34:01.105502 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f3af01db-f2de-4cd4-aeda-3cc200498cc7-kube-api-access-gzh6c" (OuterVolumeSpecName: "kube-api-access-gzh6c") pod "f3af01db-f2de-4cd4-aeda-3cc200498cc7" (UID: "f3af01db-f2de-4cd4-aeda-3cc200498cc7"). InnerVolumeSpecName "kube-api-access-gzh6c". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:34:01 crc kubenswrapper[5052]: I1203 12:34:01.110972 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f3af01db-f2de-4cd4-aeda-3cc200498cc7-dispersionconf" (OuterVolumeSpecName: "dispersionconf") pod "f3af01db-f2de-4cd4-aeda-3cc200498cc7" (UID: "f3af01db-f2de-4cd4-aeda-3cc200498cc7"). InnerVolumeSpecName "dispersionconf". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:34:01 crc kubenswrapper[5052]: I1203 12:34:01.117680 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f3af01db-f2de-4cd4-aeda-3cc200498cc7-scripts" (OuterVolumeSpecName: "scripts") pod "f3af01db-f2de-4cd4-aeda-3cc200498cc7" (UID: "f3af01db-f2de-4cd4-aeda-3cc200498cc7"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:34:01 crc kubenswrapper[5052]: I1203 12:34:01.119207 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f3af01db-f2de-4cd4-aeda-3cc200498cc7-swiftconf" (OuterVolumeSpecName: "swiftconf") pod "f3af01db-f2de-4cd4-aeda-3cc200498cc7" (UID: "f3af01db-f2de-4cd4-aeda-3cc200498cc7"). InnerVolumeSpecName "swiftconf". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:34:01 crc kubenswrapper[5052]: I1203 12:34:01.120944 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f3af01db-f2de-4cd4-aeda-3cc200498cc7-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "f3af01db-f2de-4cd4-aeda-3cc200498cc7" (UID: "f3af01db-f2de-4cd4-aeda-3cc200498cc7"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:34:01 crc kubenswrapper[5052]: I1203 12:34:01.190396 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/f3af01db-f2de-4cd4-aeda-3cc200498cc7-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:34:01 crc kubenswrapper[5052]: I1203 12:34:01.190430 5052 reconciler_common.go:293] "Volume detached for volume \"swiftconf\" (UniqueName: \"kubernetes.io/secret/f3af01db-f2de-4cd4-aeda-3cc200498cc7-swiftconf\") on node \"crc\" DevicePath \"\"" Dec 03 12:34:01 crc kubenswrapper[5052]: I1203 12:34:01.190442 5052 reconciler_common.go:293] "Volume detached for volume \"ring-data-devices\" (UniqueName: \"kubernetes.io/configmap/f3af01db-f2de-4cd4-aeda-3cc200498cc7-ring-data-devices\") on node \"crc\" DevicePath \"\"" Dec 03 12:34:01 crc kubenswrapper[5052]: I1203 12:34:01.190454 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-gzh6c\" (UniqueName: \"kubernetes.io/projected/f3af01db-f2de-4cd4-aeda-3cc200498cc7-kube-api-access-gzh6c\") on node \"crc\" DevicePath \"\"" Dec 03 12:34:01 crc kubenswrapper[5052]: I1203 12:34:01.190464 5052 reconciler_common.go:293] "Volume detached for volume \"dispersionconf\" (UniqueName: \"kubernetes.io/secret/f3af01db-f2de-4cd4-aeda-3cc200498cc7-dispersionconf\") on node \"crc\" DevicePath \"\"" Dec 03 12:34:01 crc kubenswrapper[5052]: I1203 12:34:01.190474 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f3af01db-f2de-4cd4-aeda-3cc200498cc7-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:34:01 crc kubenswrapper[5052]: I1203 12:34:01.190483 5052 reconciler_common.go:293] "Volume detached for volume \"etc-swift\" (UniqueName: \"kubernetes.io/empty-dir/f3af01db-f2de-4cd4-aeda-3cc200498cc7-etc-swift\") on node \"crc\" DevicePath \"\"" Dec 03 12:34:01 crc kubenswrapper[5052]: I1203 12:34:01.553501 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-ring-rebalance-jstcn" event={"ID":"f3af01db-f2de-4cd4-aeda-3cc200498cc7","Type":"ContainerDied","Data":"d7d949b9ba6f9e4269ffc313b1a08bcd56301d0aaa692641607ba1a2c0bd17f0"} Dec 03 12:34:01 crc kubenswrapper[5052]: I1203 12:34:01.553637 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-ring-rebalance-jstcn" Dec 03 12:34:01 crc kubenswrapper[5052]: I1203 12:34:01.553668 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d7d949b9ba6f9e4269ffc313b1a08bcd56301d0aaa692641607ba1a2c0bd17f0" Dec 03 12:34:03 crc kubenswrapper[5052]: I1203 12:34:03.330906 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/c7d598df-f336-4798-b381-9b813068f271-etc-swift\") pod \"swift-storage-0\" (UID: \"c7d598df-f336-4798-b381-9b813068f271\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:34:03 crc kubenswrapper[5052]: I1203 12:34:03.528598 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/c7d598df-f336-4798-b381-9b813068f271-etc-swift\") pod \"swift-storage-0\" (UID: \"c7d598df-f336-4798-b381-9b813068f271\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:34:03 crc kubenswrapper[5052]: I1203 12:34:03.709291 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:34:04 crc kubenswrapper[5052]: I1203 12:34:04.202631 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/swift-storage-0"] Dec 03 12:34:04 crc kubenswrapper[5052]: I1203 12:34:04.584751 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"c7d598df-f336-4798-b381-9b813068f271","Type":"ContainerStarted","Data":"cabbc82eac0ab602656d651e0b196849ff956b6dd323581aac8d857b23fae0e7"} Dec 03 12:34:04 crc kubenswrapper[5052]: I1203 12:34:04.585013 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"c7d598df-f336-4798-b381-9b813068f271","Type":"ContainerStarted","Data":"7fd8d3e9755be0491e71f4771e79219dfa4803774a0535e6af7be76023acf022"} Dec 03 12:34:04 crc kubenswrapper[5052]: I1203 12:34:04.585025 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"c7d598df-f336-4798-b381-9b813068f271","Type":"ContainerStarted","Data":"f82206729c6907aa4b14611e3738346c1d95ff29c1214f31750b8be9aefd7692"} Dec 03 12:34:04 crc kubenswrapper[5052]: I1203 12:34:04.587806 5052 generic.go:334] "Generic (PLEG): container finished" podID="6cc1f6de-7848-429c-9f14-ea763a359583" containerID="2253171c4c8bd199a61de0bcd7cfe694d9c538afb521022af5f3e5fd80739aea" exitCode=0 Dec 03 12:34:04 crc kubenswrapper[5052]: I1203 12:34:04.587835 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-db-sync-rxhl2" event={"ID":"6cc1f6de-7848-429c-9f14-ea763a359583","Type":"ContainerDied","Data":"2253171c4c8bd199a61de0bcd7cfe694d9c538afb521022af5f3e5fd80739aea"} Dec 03 12:34:04 crc kubenswrapper[5052]: I1203 12:34:04.677844 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:34:05 crc kubenswrapper[5052]: I1203 12:34:05.598829 5052 generic.go:334] "Generic (PLEG): container finished" podID="d82ef8ea-0c3b-4d89-9c78-22095940f4da" containerID="69ee48f37ae71f152a64da3b3da0915d74d05a1eb1b6c9a0996fd2a43065ecba" exitCode=0 Dec 03 12:34:05 crc kubenswrapper[5052]: I1203 12:34:05.598926 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-server-0" event={"ID":"d82ef8ea-0c3b-4d89-9c78-22095940f4da","Type":"ContainerDied","Data":"69ee48f37ae71f152a64da3b3da0915d74d05a1eb1b6c9a0996fd2a43065ecba"} Dec 03 12:34:05 crc kubenswrapper[5052]: I1203 12:34:05.606666 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"c7d598df-f336-4798-b381-9b813068f271","Type":"ContainerStarted","Data":"bf315758fc67c50f371749f4f7bbec4a1f92fbcaefd1d1b439352a3050edde3f"} Dec 03 12:34:05 crc kubenswrapper[5052]: I1203 12:34:05.606707 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"c7d598df-f336-4798-b381-9b813068f271","Type":"ContainerStarted","Data":"aa711f046bfc3e94d6ec5597e4da6fb83450c27a3396cf24f0175b9d7508c25a"} Dec 03 12:34:05 crc kubenswrapper[5052]: I1203 12:34:05.606721 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"c7d598df-f336-4798-b381-9b813068f271","Type":"ContainerStarted","Data":"490483344d3233731e445298ec4b3a6f3b3d6ba0822ed43ceb5cdbc637a73f2c"} Dec 03 12:34:05 crc kubenswrapper[5052]: I1203 12:34:05.606732 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"c7d598df-f336-4798-b381-9b813068f271","Type":"ContainerStarted","Data":"13466840574a6516f401dae001c130c3bbb09775106363deef64c61f44c71bac"} Dec 03 12:34:05 crc kubenswrapper[5052]: I1203 12:34:05.606743 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"c7d598df-f336-4798-b381-9b813068f271","Type":"ContainerStarted","Data":"93f01c0cccc7bb298548365c64b66abc7b40d0c114c2d14e3eaed3c673c6da66"} Dec 03 12:34:05 crc kubenswrapper[5052]: I1203 12:34:05.917091 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-db-sync-rxhl2" Dec 03 12:34:06 crc kubenswrapper[5052]: I1203 12:34:06.078523 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7d4kb\" (UniqueName: \"kubernetes.io/projected/6cc1f6de-7848-429c-9f14-ea763a359583-kube-api-access-7d4kb\") pod \"6cc1f6de-7848-429c-9f14-ea763a359583\" (UID: \"6cc1f6de-7848-429c-9f14-ea763a359583\") " Dec 03 12:34:06 crc kubenswrapper[5052]: I1203 12:34:06.078592 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6cc1f6de-7848-429c-9f14-ea763a359583-combined-ca-bundle\") pod \"6cc1f6de-7848-429c-9f14-ea763a359583\" (UID: \"6cc1f6de-7848-429c-9f14-ea763a359583\") " Dec 03 12:34:06 crc kubenswrapper[5052]: I1203 12:34:06.078652 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/6cc1f6de-7848-429c-9f14-ea763a359583-db-sync-config-data\") pod \"6cc1f6de-7848-429c-9f14-ea763a359583\" (UID: \"6cc1f6de-7848-429c-9f14-ea763a359583\") " Dec 03 12:34:06 crc kubenswrapper[5052]: I1203 12:34:06.078694 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6cc1f6de-7848-429c-9f14-ea763a359583-config-data\") pod \"6cc1f6de-7848-429c-9f14-ea763a359583\" (UID: \"6cc1f6de-7848-429c-9f14-ea763a359583\") " Dec 03 12:34:06 crc kubenswrapper[5052]: I1203 12:34:06.082450 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6cc1f6de-7848-429c-9f14-ea763a359583-db-sync-config-data" (OuterVolumeSpecName: "db-sync-config-data") pod "6cc1f6de-7848-429c-9f14-ea763a359583" (UID: "6cc1f6de-7848-429c-9f14-ea763a359583"). InnerVolumeSpecName "db-sync-config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:34:06 crc kubenswrapper[5052]: I1203 12:34:06.085091 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6cc1f6de-7848-429c-9f14-ea763a359583-kube-api-access-7d4kb" (OuterVolumeSpecName: "kube-api-access-7d4kb") pod "6cc1f6de-7848-429c-9f14-ea763a359583" (UID: "6cc1f6de-7848-429c-9f14-ea763a359583"). InnerVolumeSpecName "kube-api-access-7d4kb". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:34:06 crc kubenswrapper[5052]: I1203 12:34:06.109206 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6cc1f6de-7848-429c-9f14-ea763a359583-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "6cc1f6de-7848-429c-9f14-ea763a359583" (UID: "6cc1f6de-7848-429c-9f14-ea763a359583"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:34:06 crc kubenswrapper[5052]: I1203 12:34:06.133323 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6cc1f6de-7848-429c-9f14-ea763a359583-config-data" (OuterVolumeSpecName: "config-data") pod "6cc1f6de-7848-429c-9f14-ea763a359583" (UID: "6cc1f6de-7848-429c-9f14-ea763a359583"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:34:06 crc kubenswrapper[5052]: I1203 12:34:06.180746 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-7d4kb\" (UniqueName: \"kubernetes.io/projected/6cc1f6de-7848-429c-9f14-ea763a359583-kube-api-access-7d4kb\") on node \"crc\" DevicePath \"\"" Dec 03 12:34:06 crc kubenswrapper[5052]: I1203 12:34:06.180784 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6cc1f6de-7848-429c-9f14-ea763a359583-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:34:06 crc kubenswrapper[5052]: I1203 12:34:06.180794 5052 reconciler_common.go:293] "Volume detached for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/6cc1f6de-7848-429c-9f14-ea763a359583-db-sync-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:34:06 crc kubenswrapper[5052]: I1203 12:34:06.180805 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6cc1f6de-7848-429c-9f14-ea763a359583-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:34:06 crc kubenswrapper[5052]: I1203 12:34:06.627831 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"c7d598df-f336-4798-b381-9b813068f271","Type":"ContainerStarted","Data":"3668ba2994a5322df4fc232407f1ab911e9742d99a5ce860e7891029d6b3262b"} Dec 03 12:34:06 crc kubenswrapper[5052]: I1203 12:34:06.628150 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"c7d598df-f336-4798-b381-9b813068f271","Type":"ContainerStarted","Data":"2bccd4a808d4dfbe6d20fb21dfd50899da53a3d24dee822e5b066fe75d15e745"} Dec 03 12:34:06 crc kubenswrapper[5052]: I1203 12:34:06.628276 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"c7d598df-f336-4798-b381-9b813068f271","Type":"ContainerStarted","Data":"5e022d3b3c656463516700245609bcc85dbdb9a341c772fa6cfcb4273bc6c15d"} Dec 03 12:34:06 crc kubenswrapper[5052]: I1203 12:34:06.628288 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"c7d598df-f336-4798-b381-9b813068f271","Type":"ContainerStarted","Data":"67aad15627897c77a5ca09bba14532fa30813bf015f5d0017d2d2538795a5666"} Dec 03 12:34:06 crc kubenswrapper[5052]: I1203 12:34:06.628299 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"c7d598df-f336-4798-b381-9b813068f271","Type":"ContainerStarted","Data":"e82e8cd6cac0e3fe75cbcd377821970ca81cbeff69756a18c653617637c572e3"} Dec 03 12:34:06 crc kubenswrapper[5052]: I1203 12:34:06.628307 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"c7d598df-f336-4798-b381-9b813068f271","Type":"ContainerStarted","Data":"e1915bece2ea7e993dda7b1e15e670ab02f1ff041682e537f1d13ff72de48727"} Dec 03 12:34:06 crc kubenswrapper[5052]: I1203 12:34:06.637324 5052 generic.go:334] "Generic (PLEG): container finished" podID="f9bac64c-518e-4630-8889-20e09d5b7a39" containerID="806aad7eebb3067b147df9728513f4f5c522d884aded6480c84b0af0d1a1afd4" exitCode=0 Dec 03 12:34:06 crc kubenswrapper[5052]: I1203 12:34:06.637427 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" event={"ID":"f9bac64c-518e-4630-8889-20e09d5b7a39","Type":"ContainerDied","Data":"806aad7eebb3067b147df9728513f4f5c522d884aded6480c84b0af0d1a1afd4"} Dec 03 12:34:06 crc kubenswrapper[5052]: I1203 12:34:06.646777 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-db-sync-rxhl2" Dec 03 12:34:06 crc kubenswrapper[5052]: I1203 12:34:06.646802 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-db-sync-rxhl2" event={"ID":"6cc1f6de-7848-429c-9f14-ea763a359583","Type":"ContainerDied","Data":"8d1bd1d3b2d34ed0fd4a5869ac1d601db57c94c26aa27738af8b3e5f6f1c6b44"} Dec 03 12:34:06 crc kubenswrapper[5052]: I1203 12:34:06.646837 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8d1bd1d3b2d34ed0fd4a5869ac1d601db57c94c26aa27738af8b3e5f6f1c6b44" Dec 03 12:34:06 crc kubenswrapper[5052]: I1203 12:34:06.648458 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-server-0" event={"ID":"d82ef8ea-0c3b-4d89-9c78-22095940f4da","Type":"ContainerStarted","Data":"ed184550d5a8d5611c9220d80018d6040a97d60ffbd768b55b87c5964020d439"} Dec 03 12:34:06 crc kubenswrapper[5052]: I1203 12:34:06.648731 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:34:06 crc kubenswrapper[5052]: I1203 12:34:06.712025 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/rabbitmq-server-0" podStartSLOduration=37.712003962 podStartE2EDuration="37.712003962s" podCreationTimestamp="2025-12-03 12:33:29 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:34:06.699712601 +0000 UTC m=+4437.287954375" watchObservedRunningTime="2025-12-03 12:34:06.712003962 +0000 UTC m=+4437.300245716" Dec 03 12:34:07 crc kubenswrapper[5052]: I1203 12:34:07.662717 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"c7d598df-f336-4798-b381-9b813068f271","Type":"ContainerStarted","Data":"2c19477455e7e676ad01ea83d0e26999fb852f2f3e3c20bf2a784b76c5930dfc"} Dec 03 12:34:07 crc kubenswrapper[5052]: I1203 12:34:07.663032 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"c7d598df-f336-4798-b381-9b813068f271","Type":"ContainerStarted","Data":"34d7b3c41a14d274d9f82dc6e97d783bfeea4af12baf996aefeb1b3be4f773dd"} Dec 03 12:34:07 crc kubenswrapper[5052]: I1203 12:34:07.665769 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" event={"ID":"f9bac64c-518e-4630-8889-20e09d5b7a39","Type":"ContainerStarted","Data":"ed967e0e50724d3854ca971b7a125ee6c019b59d594a733d91f837aeb86bc924"} Dec 03 12:34:07 crc kubenswrapper[5052]: I1203 12:34:07.666254 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:34:07 crc kubenswrapper[5052]: I1203 12:34:07.735661 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/swift-storage-0" podStartSLOduration=21.735639758 podStartE2EDuration="21.735639758s" podCreationTimestamp="2025-12-03 12:33:46 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:34:07.726273891 +0000 UTC m=+4438.314515655" watchObservedRunningTime="2025-12-03 12:34:07.735639758 +0000 UTC m=+4438.323881522" Dec 03 12:34:07 crc kubenswrapper[5052]: I1203 12:34:07.764006 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" podStartSLOduration=37.763979267 podStartE2EDuration="37.763979267s" podCreationTimestamp="2025-12-03 12:33:30 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:34:07.752126739 +0000 UTC m=+4438.340368513" watchObservedRunningTime="2025-12-03 12:34:07.763979267 +0000 UTC m=+4438.352221031" Dec 03 12:34:07 crc kubenswrapper[5052]: I1203 12:34:07.904032 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-57d8c988fc-78nns"] Dec 03 12:34:07 crc kubenswrapper[5052]: E1203 12:34:07.904389 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="f3af01db-f2de-4cd4-aeda-3cc200498cc7" containerName="swift-ring-rebalance" Dec 03 12:34:07 crc kubenswrapper[5052]: I1203 12:34:07.904405 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="f3af01db-f2de-4cd4-aeda-3cc200498cc7" containerName="swift-ring-rebalance" Dec 03 12:34:07 crc kubenswrapper[5052]: E1203 12:34:07.904421 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="13e6f571-4161-442d-b45e-f30f6ea05843" containerName="mariadb-account-create-update" Dec 03 12:34:07 crc kubenswrapper[5052]: I1203 12:34:07.904428 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="13e6f571-4161-442d-b45e-f30f6ea05843" containerName="mariadb-account-create-update" Dec 03 12:34:07 crc kubenswrapper[5052]: E1203 12:34:07.904438 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="6cc1f6de-7848-429c-9f14-ea763a359583" containerName="glance-db-sync" Dec 03 12:34:07 crc kubenswrapper[5052]: I1203 12:34:07.904444 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="6cc1f6de-7848-429c-9f14-ea763a359583" containerName="glance-db-sync" Dec 03 12:34:07 crc kubenswrapper[5052]: E1203 12:34:07.904451 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="8179fa49-7bb4-4c12-b863-a080cbabb765" containerName="mariadb-database-create" Dec 03 12:34:07 crc kubenswrapper[5052]: I1203 12:34:07.904457 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="8179fa49-7bb4-4c12-b863-a080cbabb765" containerName="mariadb-database-create" Dec 03 12:34:07 crc kubenswrapper[5052]: I1203 12:34:07.904649 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="6cc1f6de-7848-429c-9f14-ea763a359583" containerName="glance-db-sync" Dec 03 12:34:07 crc kubenswrapper[5052]: I1203 12:34:07.904662 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="f3af01db-f2de-4cd4-aeda-3cc200498cc7" containerName="swift-ring-rebalance" Dec 03 12:34:07 crc kubenswrapper[5052]: I1203 12:34:07.904680 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="13e6f571-4161-442d-b45e-f30f6ea05843" containerName="mariadb-account-create-update" Dec 03 12:34:07 crc kubenswrapper[5052]: I1203 12:34:07.904696 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="8179fa49-7bb4-4c12-b863-a080cbabb765" containerName="mariadb-database-create" Dec 03 12:34:07 crc kubenswrapper[5052]: I1203 12:34:07.905617 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-57d8c988fc-78nns" Dec 03 12:34:07 crc kubenswrapper[5052]: I1203 12:34:07.908299 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"dns-swift-storage-0" Dec 03 12:34:07 crc kubenswrapper[5052]: I1203 12:34:07.916067 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-57d8c988fc-78nns"] Dec 03 12:34:08 crc kubenswrapper[5052]: I1203 12:34:08.017055 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/1fcf641b-a032-4179-9dde-ad76e9d86925-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-57d8c988fc-78nns\" (UID: \"1fcf641b-a032-4179-9dde-ad76e9d86925\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-57d8c988fc-78nns" Dec 03 12:34:08 crc kubenswrapper[5052]: I1203 12:34:08.017167 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mhfdp\" (UniqueName: \"kubernetes.io/projected/1fcf641b-a032-4179-9dde-ad76e9d86925-kube-api-access-mhfdp\") pod \"dnsmasq-dnsmasq-57d8c988fc-78nns\" (UID: \"1fcf641b-a032-4179-9dde-ad76e9d86925\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-57d8c988fc-78nns" Dec 03 12:34:08 crc kubenswrapper[5052]: I1203 12:34:08.017214 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1fcf641b-a032-4179-9dde-ad76e9d86925-config\") pod \"dnsmasq-dnsmasq-57d8c988fc-78nns\" (UID: \"1fcf641b-a032-4179-9dde-ad76e9d86925\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-57d8c988fc-78nns" Dec 03 12:34:08 crc kubenswrapper[5052]: I1203 12:34:08.017369 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dns-swift-storage-0\" (UniqueName: \"kubernetes.io/configmap/1fcf641b-a032-4179-9dde-ad76e9d86925-dns-swift-storage-0\") pod \"dnsmasq-dnsmasq-57d8c988fc-78nns\" (UID: \"1fcf641b-a032-4179-9dde-ad76e9d86925\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-57d8c988fc-78nns" Dec 03 12:34:08 crc kubenswrapper[5052]: I1203 12:34:08.118846 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-mhfdp\" (UniqueName: \"kubernetes.io/projected/1fcf641b-a032-4179-9dde-ad76e9d86925-kube-api-access-mhfdp\") pod \"dnsmasq-dnsmasq-57d8c988fc-78nns\" (UID: \"1fcf641b-a032-4179-9dde-ad76e9d86925\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-57d8c988fc-78nns" Dec 03 12:34:08 crc kubenswrapper[5052]: I1203 12:34:08.118923 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1fcf641b-a032-4179-9dde-ad76e9d86925-config\") pod \"dnsmasq-dnsmasq-57d8c988fc-78nns\" (UID: \"1fcf641b-a032-4179-9dde-ad76e9d86925\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-57d8c988fc-78nns" Dec 03 12:34:08 crc kubenswrapper[5052]: I1203 12:34:08.119003 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"dns-swift-storage-0\" (UniqueName: \"kubernetes.io/configmap/1fcf641b-a032-4179-9dde-ad76e9d86925-dns-swift-storage-0\") pod \"dnsmasq-dnsmasq-57d8c988fc-78nns\" (UID: \"1fcf641b-a032-4179-9dde-ad76e9d86925\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-57d8c988fc-78nns" Dec 03 12:34:08 crc kubenswrapper[5052]: I1203 12:34:08.119062 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/1fcf641b-a032-4179-9dde-ad76e9d86925-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-57d8c988fc-78nns\" (UID: \"1fcf641b-a032-4179-9dde-ad76e9d86925\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-57d8c988fc-78nns" Dec 03 12:34:08 crc kubenswrapper[5052]: I1203 12:34:08.120004 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/1fcf641b-a032-4179-9dde-ad76e9d86925-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-57d8c988fc-78nns\" (UID: \"1fcf641b-a032-4179-9dde-ad76e9d86925\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-57d8c988fc-78nns" Dec 03 12:34:08 crc kubenswrapper[5052]: I1203 12:34:08.121134 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1fcf641b-a032-4179-9dde-ad76e9d86925-config\") pod \"dnsmasq-dnsmasq-57d8c988fc-78nns\" (UID: \"1fcf641b-a032-4179-9dde-ad76e9d86925\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-57d8c988fc-78nns" Dec 03 12:34:08 crc kubenswrapper[5052]: I1203 12:34:08.122071 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"dns-swift-storage-0\" (UniqueName: \"kubernetes.io/configmap/1fcf641b-a032-4179-9dde-ad76e9d86925-dns-swift-storage-0\") pod \"dnsmasq-dnsmasq-57d8c988fc-78nns\" (UID: \"1fcf641b-a032-4179-9dde-ad76e9d86925\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-57d8c988fc-78nns" Dec 03 12:34:08 crc kubenswrapper[5052]: I1203 12:34:08.143591 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-mhfdp\" (UniqueName: \"kubernetes.io/projected/1fcf641b-a032-4179-9dde-ad76e9d86925-kube-api-access-mhfdp\") pod \"dnsmasq-dnsmasq-57d8c988fc-78nns\" (UID: \"1fcf641b-a032-4179-9dde-ad76e9d86925\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-57d8c988fc-78nns" Dec 03 12:34:08 crc kubenswrapper[5052]: I1203 12:34:08.222804 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-57d8c988fc-78nns" Dec 03 12:34:08 crc kubenswrapper[5052]: I1203 12:34:08.672853 5052 scope.go:117] "RemoveContainer" containerID="e6edaf188f85ea4f11f487f11cc489c700f25fb42b5ee269f4f496f1e127d772" Dec 03 12:34:08 crc kubenswrapper[5052]: E1203 12:34:08.673322 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 12:34:08 crc kubenswrapper[5052]: I1203 12:34:08.703408 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-57d8c988fc-78nns"] Dec 03 12:34:09 crc kubenswrapper[5052]: I1203 12:34:09.694074 5052 generic.go:334] "Generic (PLEG): container finished" podID="1fcf641b-a032-4179-9dde-ad76e9d86925" containerID="817dd5a079a81242fe031d9189c03112a82a5c61f2ee81b7e6a114983bb4f990" exitCode=0 Dec 03 12:34:09 crc kubenswrapper[5052]: I1203 12:34:09.694144 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-57d8c988fc-78nns" event={"ID":"1fcf641b-a032-4179-9dde-ad76e9d86925","Type":"ContainerDied","Data":"817dd5a079a81242fe031d9189c03112a82a5c61f2ee81b7e6a114983bb4f990"} Dec 03 12:34:09 crc kubenswrapper[5052]: I1203 12:34:09.694582 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-57d8c988fc-78nns" event={"ID":"1fcf641b-a032-4179-9dde-ad76e9d86925","Type":"ContainerStarted","Data":"df2a18a663b3a561fd9b9b69b618387618f19977fe89926626b99ddeb8b1fe35"} Dec 03 12:34:10 crc kubenswrapper[5052]: I1203 12:34:10.704629 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-57d8c988fc-78nns" event={"ID":"1fcf641b-a032-4179-9dde-ad76e9d86925","Type":"ContainerStarted","Data":"9e329fad0d27b1a8701714781d22f98230c08ebcc9a5d105a727094e58b0a0ee"} Dec 03 12:34:10 crc kubenswrapper[5052]: I1203 12:34:10.705184 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-57d8c988fc-78nns" Dec 03 12:34:10 crc kubenswrapper[5052]: I1203 12:34:10.724747 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-57d8c988fc-78nns" podStartSLOduration=3.7247267539999998 podStartE2EDuration="3.724726754s" podCreationTimestamp="2025-12-03 12:34:07 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:34:10.718181947 +0000 UTC m=+4441.306423711" watchObservedRunningTime="2025-12-03 12:34:10.724726754 +0000 UTC m=+4441.312968518" Dec 03 12:34:18 crc kubenswrapper[5052]: I1203 12:34:18.224721 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-57d8c988fc-78nns" Dec 03 12:34:18 crc kubenswrapper[5052]: I1203 12:34:18.281976 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-hjfg4"] Dec 03 12:34:18 crc kubenswrapper[5052]: I1203 12:34:18.282271 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-hjfg4" podUID="00bbba79-7529-4309-af53-b464acf28194" containerName="dnsmasq-dns" containerID="cri-o://856900df1677628daad9107fd3739a17cd9695e68c3f78d8a83df16b291e65ea" gracePeriod=10 Dec 03 12:34:18 crc kubenswrapper[5052]: I1203 12:34:18.732442 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-hjfg4" Dec 03 12:34:18 crc kubenswrapper[5052]: I1203 12:34:18.774950 5052 generic.go:334] "Generic (PLEG): container finished" podID="00bbba79-7529-4309-af53-b464acf28194" containerID="856900df1677628daad9107fd3739a17cd9695e68c3f78d8a83df16b291e65ea" exitCode=0 Dec 03 12:34:18 crc kubenswrapper[5052]: I1203 12:34:18.774991 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-hjfg4" event={"ID":"00bbba79-7529-4309-af53-b464acf28194","Type":"ContainerDied","Data":"856900df1677628daad9107fd3739a17cd9695e68c3f78d8a83df16b291e65ea"} Dec 03 12:34:18 crc kubenswrapper[5052]: I1203 12:34:18.775023 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-hjfg4" event={"ID":"00bbba79-7529-4309-af53-b464acf28194","Type":"ContainerDied","Data":"d728e258976c599d37747130930f52da15a04d3a7b62248cfb1dfef0b9361827"} Dec 03 12:34:18 crc kubenswrapper[5052]: I1203 12:34:18.775068 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-hjfg4" Dec 03 12:34:18 crc kubenswrapper[5052]: I1203 12:34:18.775077 5052 scope.go:117] "RemoveContainer" containerID="856900df1677628daad9107fd3739a17cd9695e68c3f78d8a83df16b291e65ea" Dec 03 12:34:18 crc kubenswrapper[5052]: I1203 12:34:18.801129 5052 scope.go:117] "RemoveContainer" containerID="767245499b2d11a292bcbfa756fc57b438b73c163713457389fa39a70f97a888" Dec 03 12:34:18 crc kubenswrapper[5052]: I1203 12:34:18.818559 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/00bbba79-7529-4309-af53-b464acf28194-config\") pod \"00bbba79-7529-4309-af53-b464acf28194\" (UID: \"00bbba79-7529-4309-af53-b464acf28194\") " Dec 03 12:34:18 crc kubenswrapper[5052]: I1203 12:34:18.818892 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8pm8q\" (UniqueName: \"kubernetes.io/projected/00bbba79-7529-4309-af53-b464acf28194-kube-api-access-8pm8q\") pod \"00bbba79-7529-4309-af53-b464acf28194\" (UID: \"00bbba79-7529-4309-af53-b464acf28194\") " Dec 03 12:34:18 crc kubenswrapper[5052]: I1203 12:34:18.820010 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/00bbba79-7529-4309-af53-b464acf28194-dnsmasq-svc\") pod \"00bbba79-7529-4309-af53-b464acf28194\" (UID: \"00bbba79-7529-4309-af53-b464acf28194\") " Dec 03 12:34:18 crc kubenswrapper[5052]: I1203 12:34:18.821595 5052 scope.go:117] "RemoveContainer" containerID="856900df1677628daad9107fd3739a17cd9695e68c3f78d8a83df16b291e65ea" Dec 03 12:34:18 crc kubenswrapper[5052]: E1203 12:34:18.822641 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"856900df1677628daad9107fd3739a17cd9695e68c3f78d8a83df16b291e65ea\": container with ID starting with 856900df1677628daad9107fd3739a17cd9695e68c3f78d8a83df16b291e65ea not found: ID does not exist" containerID="856900df1677628daad9107fd3739a17cd9695e68c3f78d8a83df16b291e65ea" Dec 03 12:34:18 crc kubenswrapper[5052]: I1203 12:34:18.822680 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"856900df1677628daad9107fd3739a17cd9695e68c3f78d8a83df16b291e65ea"} err="failed to get container status \"856900df1677628daad9107fd3739a17cd9695e68c3f78d8a83df16b291e65ea\": rpc error: code = NotFound desc = could not find container \"856900df1677628daad9107fd3739a17cd9695e68c3f78d8a83df16b291e65ea\": container with ID starting with 856900df1677628daad9107fd3739a17cd9695e68c3f78d8a83df16b291e65ea not found: ID does not exist" Dec 03 12:34:18 crc kubenswrapper[5052]: I1203 12:34:18.822706 5052 scope.go:117] "RemoveContainer" containerID="767245499b2d11a292bcbfa756fc57b438b73c163713457389fa39a70f97a888" Dec 03 12:34:18 crc kubenswrapper[5052]: E1203 12:34:18.823140 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"767245499b2d11a292bcbfa756fc57b438b73c163713457389fa39a70f97a888\": container with ID starting with 767245499b2d11a292bcbfa756fc57b438b73c163713457389fa39a70f97a888 not found: ID does not exist" containerID="767245499b2d11a292bcbfa756fc57b438b73c163713457389fa39a70f97a888" Dec 03 12:34:18 crc kubenswrapper[5052]: I1203 12:34:18.823176 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"767245499b2d11a292bcbfa756fc57b438b73c163713457389fa39a70f97a888"} err="failed to get container status \"767245499b2d11a292bcbfa756fc57b438b73c163713457389fa39a70f97a888\": rpc error: code = NotFound desc = could not find container \"767245499b2d11a292bcbfa756fc57b438b73c163713457389fa39a70f97a888\": container with ID starting with 767245499b2d11a292bcbfa756fc57b438b73c163713457389fa39a70f97a888 not found: ID does not exist" Dec 03 12:34:18 crc kubenswrapper[5052]: I1203 12:34:18.823758 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/00bbba79-7529-4309-af53-b464acf28194-kube-api-access-8pm8q" (OuterVolumeSpecName: "kube-api-access-8pm8q") pod "00bbba79-7529-4309-af53-b464acf28194" (UID: "00bbba79-7529-4309-af53-b464acf28194"). InnerVolumeSpecName "kube-api-access-8pm8q". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:34:18 crc kubenswrapper[5052]: I1203 12:34:18.859338 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/00bbba79-7529-4309-af53-b464acf28194-config" (OuterVolumeSpecName: "config") pod "00bbba79-7529-4309-af53-b464acf28194" (UID: "00bbba79-7529-4309-af53-b464acf28194"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:34:18 crc kubenswrapper[5052]: I1203 12:34:18.860482 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/00bbba79-7529-4309-af53-b464acf28194-dnsmasq-svc" (OuterVolumeSpecName: "dnsmasq-svc") pod "00bbba79-7529-4309-af53-b464acf28194" (UID: "00bbba79-7529-4309-af53-b464acf28194"). InnerVolumeSpecName "dnsmasq-svc". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:34:18 crc kubenswrapper[5052]: I1203 12:34:18.923078 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/00bbba79-7529-4309-af53-b464acf28194-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:34:18 crc kubenswrapper[5052]: I1203 12:34:18.923111 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-8pm8q\" (UniqueName: \"kubernetes.io/projected/00bbba79-7529-4309-af53-b464acf28194-kube-api-access-8pm8q\") on node \"crc\" DevicePath \"\"" Dec 03 12:34:18 crc kubenswrapper[5052]: I1203 12:34:18.923124 5052 reconciler_common.go:293] "Volume detached for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/00bbba79-7529-4309-af53-b464acf28194-dnsmasq-svc\") on node \"crc\" DevicePath \"\"" Dec 03 12:34:19 crc kubenswrapper[5052]: I1203 12:34:19.107317 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-hjfg4"] Dec 03 12:34:19 crc kubenswrapper[5052]: I1203 12:34:19.116631 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-hjfg4"] Dec 03 12:34:19 crc kubenswrapper[5052]: I1203 12:34:19.701144 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="00bbba79-7529-4309-af53-b464acf28194" path="/var/lib/kubelet/pods/00bbba79-7529-4309-af53-b464acf28194/volumes" Dec 03 12:34:20 crc kubenswrapper[5052]: I1203 12:34:20.886757 5052 scope.go:117] "RemoveContainer" containerID="df1192fd6167f45271208db821d1f0909fc60a2b10aada89f5e4d0015c104317" Dec 03 12:34:20 crc kubenswrapper[5052]: I1203 12:34:20.908394 5052 scope.go:117] "RemoveContainer" containerID="8a517664093b91e9ef8c95cddf9f11b7aba074cb44d5aee9b5c50411254e47b7" Dec 03 12:34:20 crc kubenswrapper[5052]: I1203 12:34:20.959697 5052 scope.go:117] "RemoveContainer" containerID="33c00a517cac73b55f22b39dd9800951f3eddc4eb90411d1bc3e8a5477c02571" Dec 03 12:34:20 crc kubenswrapper[5052]: I1203 12:34:20.981321 5052 scope.go:117] "RemoveContainer" containerID="e1e47b47265fb722fd8606f3ed96e8cbc15ed3bc36658d9fdd76dc06cbe471c8" Dec 03 12:34:21 crc kubenswrapper[5052]: I1203 12:34:21.006862 5052 scope.go:117] "RemoveContainer" containerID="0680111a7796404e507144aefcc56ae790dfd5e497642c8b08e59f43944a5d46" Dec 03 12:34:21 crc kubenswrapper[5052]: I1203 12:34:21.079519 5052 scope.go:117] "RemoveContainer" containerID="c1a01685a9df4a046418f219f4791232079759a5dfaf1dcd3a5230170ee077ab" Dec 03 12:34:21 crc kubenswrapper[5052]: I1203 12:34:21.254685 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:34:21 crc kubenswrapper[5052]: I1203 12:34:21.543797 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/cinder-db-create-kc855"] Dec 03 12:34:21 crc kubenswrapper[5052]: E1203 12:34:21.544478 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="00bbba79-7529-4309-af53-b464acf28194" containerName="dnsmasq-dns" Dec 03 12:34:21 crc kubenswrapper[5052]: I1203 12:34:21.544497 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="00bbba79-7529-4309-af53-b464acf28194" containerName="dnsmasq-dns" Dec 03 12:34:21 crc kubenswrapper[5052]: E1203 12:34:21.544630 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="00bbba79-7529-4309-af53-b464acf28194" containerName="init" Dec 03 12:34:21 crc kubenswrapper[5052]: I1203 12:34:21.544638 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="00bbba79-7529-4309-af53-b464acf28194" containerName="init" Dec 03 12:34:21 crc kubenswrapper[5052]: I1203 12:34:21.544786 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="00bbba79-7529-4309-af53-b464acf28194" containerName="dnsmasq-dns" Dec 03 12:34:21 crc kubenswrapper[5052]: I1203 12:34:21.545318 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-db-create-kc855" Dec 03 12:34:21 crc kubenswrapper[5052]: I1203 12:34:21.556938 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-db-create-kc855"] Dec 03 12:34:21 crc kubenswrapper[5052]: I1203 12:34:21.573739 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/dde85075-240d-43d9-a002-cc0206cfebf8-operator-scripts\") pod \"cinder-db-create-kc855\" (UID: \"dde85075-240d-43d9-a002-cc0206cfebf8\") " pod="openstack-kuttl-tests/cinder-db-create-kc855" Dec 03 12:34:21 crc kubenswrapper[5052]: I1203 12:34:21.573843 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6ffml\" (UniqueName: \"kubernetes.io/projected/dde85075-240d-43d9-a002-cc0206cfebf8-kube-api-access-6ffml\") pod \"cinder-db-create-kc855\" (UID: \"dde85075-240d-43d9-a002-cc0206cfebf8\") " pod="openstack-kuttl-tests/cinder-db-create-kc855" Dec 03 12:34:21 crc kubenswrapper[5052]: I1203 12:34:21.592672 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:34:21 crc kubenswrapper[5052]: I1203 12:34:21.675406 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/dde85075-240d-43d9-a002-cc0206cfebf8-operator-scripts\") pod \"cinder-db-create-kc855\" (UID: \"dde85075-240d-43d9-a002-cc0206cfebf8\") " pod="openstack-kuttl-tests/cinder-db-create-kc855" Dec 03 12:34:21 crc kubenswrapper[5052]: I1203 12:34:21.675594 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-6ffml\" (UniqueName: \"kubernetes.io/projected/dde85075-240d-43d9-a002-cc0206cfebf8-kube-api-access-6ffml\") pod \"cinder-db-create-kc855\" (UID: \"dde85075-240d-43d9-a002-cc0206cfebf8\") " pod="openstack-kuttl-tests/cinder-db-create-kc855" Dec 03 12:34:21 crc kubenswrapper[5052]: I1203 12:34:21.676752 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/dde85075-240d-43d9-a002-cc0206cfebf8-operator-scripts\") pod \"cinder-db-create-kc855\" (UID: \"dde85075-240d-43d9-a002-cc0206cfebf8\") " pod="openstack-kuttl-tests/cinder-db-create-kc855" Dec 03 12:34:21 crc kubenswrapper[5052]: I1203 12:34:21.696583 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/barbican-db-create-j9nw5"] Dec 03 12:34:21 crc kubenswrapper[5052]: I1203 12:34:21.707293 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/cinder-3a0d-account-create-update-wn6fm"] Dec 03 12:34:21 crc kubenswrapper[5052]: I1203 12:34:21.707421 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-db-create-j9nw5" Dec 03 12:34:21 crc kubenswrapper[5052]: I1203 12:34:21.708697 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-3a0d-account-create-update-wn6fm" Dec 03 12:34:21 crc kubenswrapper[5052]: I1203 12:34:21.710680 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cinder-db-secret" Dec 03 12:34:21 crc kubenswrapper[5052]: I1203 12:34:21.723234 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-3a0d-account-create-update-wn6fm"] Dec 03 12:34:21 crc kubenswrapper[5052]: I1203 12:34:21.740672 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-db-create-j9nw5"] Dec 03 12:34:21 crc kubenswrapper[5052]: I1203 12:34:21.780404 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-67z7x\" (UniqueName: \"kubernetes.io/projected/2f122603-9186-43bc-b3c7-b21307f24c5a-kube-api-access-67z7x\") pod \"barbican-db-create-j9nw5\" (UID: \"2f122603-9186-43bc-b3c7-b21307f24c5a\") " pod="openstack-kuttl-tests/barbican-db-create-j9nw5" Dec 03 12:34:21 crc kubenswrapper[5052]: I1203 12:34:21.780462 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/2f122603-9186-43bc-b3c7-b21307f24c5a-operator-scripts\") pod \"barbican-db-create-j9nw5\" (UID: \"2f122603-9186-43bc-b3c7-b21307f24c5a\") " pod="openstack-kuttl-tests/barbican-db-create-j9nw5" Dec 03 12:34:21 crc kubenswrapper[5052]: I1203 12:34:21.780576 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zfdhw\" (UniqueName: \"kubernetes.io/projected/a6e548d7-7349-413c-a4f7-914f4df80c4b-kube-api-access-zfdhw\") pod \"cinder-3a0d-account-create-update-wn6fm\" (UID: \"a6e548d7-7349-413c-a4f7-914f4df80c4b\") " pod="openstack-kuttl-tests/cinder-3a0d-account-create-update-wn6fm" Dec 03 12:34:21 crc kubenswrapper[5052]: I1203 12:34:21.780747 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/a6e548d7-7349-413c-a4f7-914f4df80c4b-operator-scripts\") pod \"cinder-3a0d-account-create-update-wn6fm\" (UID: \"a6e548d7-7349-413c-a4f7-914f4df80c4b\") " pod="openstack-kuttl-tests/cinder-3a0d-account-create-update-wn6fm" Dec 03 12:34:21 crc kubenswrapper[5052]: I1203 12:34:21.795885 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/barbican-1171-account-create-update-gm2mn"] Dec 03 12:34:21 crc kubenswrapper[5052]: I1203 12:34:21.797071 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-1171-account-create-update-gm2mn" Dec 03 12:34:21 crc kubenswrapper[5052]: I1203 12:34:21.801808 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"barbican-db-secret" Dec 03 12:34:21 crc kubenswrapper[5052]: I1203 12:34:21.804024 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-1171-account-create-update-gm2mn"] Dec 03 12:34:21 crc kubenswrapper[5052]: I1203 12:34:21.826766 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-6ffml\" (UniqueName: \"kubernetes.io/projected/dde85075-240d-43d9-a002-cc0206cfebf8-kube-api-access-6ffml\") pod \"cinder-db-create-kc855\" (UID: \"dde85075-240d-43d9-a002-cc0206cfebf8\") " pod="openstack-kuttl-tests/cinder-db-create-kc855" Dec 03 12:34:21 crc kubenswrapper[5052]: I1203 12:34:21.854616 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/neutron-db-create-78h7j"] Dec 03 12:34:21 crc kubenswrapper[5052]: I1203 12:34:21.856353 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-db-create-78h7j" Dec 03 12:34:21 crc kubenswrapper[5052]: I1203 12:34:21.869118 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-db-create-78h7j"] Dec 03 12:34:21 crc kubenswrapper[5052]: I1203 12:34:21.882460 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-zfdhw\" (UniqueName: \"kubernetes.io/projected/a6e548d7-7349-413c-a4f7-914f4df80c4b-kube-api-access-zfdhw\") pod \"cinder-3a0d-account-create-update-wn6fm\" (UID: \"a6e548d7-7349-413c-a4f7-914f4df80c4b\") " pod="openstack-kuttl-tests/cinder-3a0d-account-create-update-wn6fm" Dec 03 12:34:21 crc kubenswrapper[5052]: I1203 12:34:21.882599 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z99zx\" (UniqueName: \"kubernetes.io/projected/65fef887-25c4-46ee-8a9b-cc82f32dcbe3-kube-api-access-z99zx\") pod \"barbican-1171-account-create-update-gm2mn\" (UID: \"65fef887-25c4-46ee-8a9b-cc82f32dcbe3\") " pod="openstack-kuttl-tests/barbican-1171-account-create-update-gm2mn" Dec 03 12:34:21 crc kubenswrapper[5052]: I1203 12:34:21.882629 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/a6e548d7-7349-413c-a4f7-914f4df80c4b-operator-scripts\") pod \"cinder-3a0d-account-create-update-wn6fm\" (UID: \"a6e548d7-7349-413c-a4f7-914f4df80c4b\") " pod="openstack-kuttl-tests/cinder-3a0d-account-create-update-wn6fm" Dec 03 12:34:21 crc kubenswrapper[5052]: I1203 12:34:21.882677 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/65fef887-25c4-46ee-8a9b-cc82f32dcbe3-operator-scripts\") pod \"barbican-1171-account-create-update-gm2mn\" (UID: \"65fef887-25c4-46ee-8a9b-cc82f32dcbe3\") " pod="openstack-kuttl-tests/barbican-1171-account-create-update-gm2mn" Dec 03 12:34:21 crc kubenswrapper[5052]: I1203 12:34:21.882712 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-67z7x\" (UniqueName: \"kubernetes.io/projected/2f122603-9186-43bc-b3c7-b21307f24c5a-kube-api-access-67z7x\") pod \"barbican-db-create-j9nw5\" (UID: \"2f122603-9186-43bc-b3c7-b21307f24c5a\") " pod="openstack-kuttl-tests/barbican-db-create-j9nw5" Dec 03 12:34:21 crc kubenswrapper[5052]: I1203 12:34:21.882738 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/2f122603-9186-43bc-b3c7-b21307f24c5a-operator-scripts\") pod \"barbican-db-create-j9nw5\" (UID: \"2f122603-9186-43bc-b3c7-b21307f24c5a\") " pod="openstack-kuttl-tests/barbican-db-create-j9nw5" Dec 03 12:34:21 crc kubenswrapper[5052]: I1203 12:34:21.882778 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m9865\" (UniqueName: \"kubernetes.io/projected/2a13e864-04af-42a7-bfbb-9995231b567d-kube-api-access-m9865\") pod \"neutron-db-create-78h7j\" (UID: \"2a13e864-04af-42a7-bfbb-9995231b567d\") " pod="openstack-kuttl-tests/neutron-db-create-78h7j" Dec 03 12:34:21 crc kubenswrapper[5052]: I1203 12:34:21.882816 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/2a13e864-04af-42a7-bfbb-9995231b567d-operator-scripts\") pod \"neutron-db-create-78h7j\" (UID: \"2a13e864-04af-42a7-bfbb-9995231b567d\") " pod="openstack-kuttl-tests/neutron-db-create-78h7j" Dec 03 12:34:21 crc kubenswrapper[5052]: I1203 12:34:21.884348 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/a6e548d7-7349-413c-a4f7-914f4df80c4b-operator-scripts\") pod \"cinder-3a0d-account-create-update-wn6fm\" (UID: \"a6e548d7-7349-413c-a4f7-914f4df80c4b\") " pod="openstack-kuttl-tests/cinder-3a0d-account-create-update-wn6fm" Dec 03 12:34:21 crc kubenswrapper[5052]: I1203 12:34:21.884715 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/2f122603-9186-43bc-b3c7-b21307f24c5a-operator-scripts\") pod \"barbican-db-create-j9nw5\" (UID: \"2f122603-9186-43bc-b3c7-b21307f24c5a\") " pod="openstack-kuttl-tests/barbican-db-create-j9nw5" Dec 03 12:34:21 crc kubenswrapper[5052]: I1203 12:34:21.889268 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-db-create-kc855" Dec 03 12:34:21 crc kubenswrapper[5052]: I1203 12:34:21.912446 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-zfdhw\" (UniqueName: \"kubernetes.io/projected/a6e548d7-7349-413c-a4f7-914f4df80c4b-kube-api-access-zfdhw\") pod \"cinder-3a0d-account-create-update-wn6fm\" (UID: \"a6e548d7-7349-413c-a4f7-914f4df80c4b\") " pod="openstack-kuttl-tests/cinder-3a0d-account-create-update-wn6fm" Dec 03 12:34:21 crc kubenswrapper[5052]: I1203 12:34:21.916417 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-67z7x\" (UniqueName: \"kubernetes.io/projected/2f122603-9186-43bc-b3c7-b21307f24c5a-kube-api-access-67z7x\") pod \"barbican-db-create-j9nw5\" (UID: \"2f122603-9186-43bc-b3c7-b21307f24c5a\") " pod="openstack-kuttl-tests/barbican-db-create-j9nw5" Dec 03 12:34:21 crc kubenswrapper[5052]: I1203 12:34:21.930793 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/keystone-db-sync-v5287"] Dec 03 12:34:21 crc kubenswrapper[5052]: I1203 12:34:21.932272 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-db-sync-v5287" Dec 03 12:34:21 crc kubenswrapper[5052]: I1203 12:34:21.939016 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone-keystone-dockercfg-fvfwv" Dec 03 12:34:21 crc kubenswrapper[5052]: I1203 12:34:21.939273 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone-scripts" Dec 03 12:34:21 crc kubenswrapper[5052]: I1203 12:34:21.939508 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone-config-data" Dec 03 12:34:21 crc kubenswrapper[5052]: I1203 12:34:21.946889 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone" Dec 03 12:34:21 crc kubenswrapper[5052]: I1203 12:34:21.958629 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-db-sync-v5287"] Dec 03 12:34:21 crc kubenswrapper[5052]: I1203 12:34:21.975915 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/neutron-d24e-account-create-update-gskxw"] Dec 03 12:34:21 crc kubenswrapper[5052]: I1203 12:34:21.977192 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-d24e-account-create-update-gskxw" Dec 03 12:34:21 crc kubenswrapper[5052]: I1203 12:34:21.978798 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"neutron-db-secret" Dec 03 12:34:21 crc kubenswrapper[5052]: I1203 12:34:21.985619 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/65fef887-25c4-46ee-8a9b-cc82f32dcbe3-operator-scripts\") pod \"barbican-1171-account-create-update-gm2mn\" (UID: \"65fef887-25c4-46ee-8a9b-cc82f32dcbe3\") " pod="openstack-kuttl-tests/barbican-1171-account-create-update-gm2mn" Dec 03 12:34:21 crc kubenswrapper[5052]: I1203 12:34:21.985696 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-m9865\" (UniqueName: \"kubernetes.io/projected/2a13e864-04af-42a7-bfbb-9995231b567d-kube-api-access-m9865\") pod \"neutron-db-create-78h7j\" (UID: \"2a13e864-04af-42a7-bfbb-9995231b567d\") " pod="openstack-kuttl-tests/neutron-db-create-78h7j" Dec 03 12:34:21 crc kubenswrapper[5052]: I1203 12:34:21.985730 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/2a13e864-04af-42a7-bfbb-9995231b567d-operator-scripts\") pod \"neutron-db-create-78h7j\" (UID: \"2a13e864-04af-42a7-bfbb-9995231b567d\") " pod="openstack-kuttl-tests/neutron-db-create-78h7j" Dec 03 12:34:21 crc kubenswrapper[5052]: I1203 12:34:21.985813 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-z99zx\" (UniqueName: \"kubernetes.io/projected/65fef887-25c4-46ee-8a9b-cc82f32dcbe3-kube-api-access-z99zx\") pod \"barbican-1171-account-create-update-gm2mn\" (UID: \"65fef887-25c4-46ee-8a9b-cc82f32dcbe3\") " pod="openstack-kuttl-tests/barbican-1171-account-create-update-gm2mn" Dec 03 12:34:21 crc kubenswrapper[5052]: I1203 12:34:21.986628 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/65fef887-25c4-46ee-8a9b-cc82f32dcbe3-operator-scripts\") pod \"barbican-1171-account-create-update-gm2mn\" (UID: \"65fef887-25c4-46ee-8a9b-cc82f32dcbe3\") " pod="openstack-kuttl-tests/barbican-1171-account-create-update-gm2mn" Dec 03 12:34:21 crc kubenswrapper[5052]: I1203 12:34:21.986816 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/2a13e864-04af-42a7-bfbb-9995231b567d-operator-scripts\") pod \"neutron-db-create-78h7j\" (UID: \"2a13e864-04af-42a7-bfbb-9995231b567d\") " pod="openstack-kuttl-tests/neutron-db-create-78h7j" Dec 03 12:34:22 crc kubenswrapper[5052]: I1203 12:34:22.013180 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-z99zx\" (UniqueName: \"kubernetes.io/projected/65fef887-25c4-46ee-8a9b-cc82f32dcbe3-kube-api-access-z99zx\") pod \"barbican-1171-account-create-update-gm2mn\" (UID: \"65fef887-25c4-46ee-8a9b-cc82f32dcbe3\") " pod="openstack-kuttl-tests/barbican-1171-account-create-update-gm2mn" Dec 03 12:34:22 crc kubenswrapper[5052]: I1203 12:34:22.023503 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-d24e-account-create-update-gskxw"] Dec 03 12:34:22 crc kubenswrapper[5052]: I1203 12:34:22.038204 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-m9865\" (UniqueName: \"kubernetes.io/projected/2a13e864-04af-42a7-bfbb-9995231b567d-kube-api-access-m9865\") pod \"neutron-db-create-78h7j\" (UID: \"2a13e864-04af-42a7-bfbb-9995231b567d\") " pod="openstack-kuttl-tests/neutron-db-create-78h7j" Dec 03 12:34:22 crc kubenswrapper[5052]: I1203 12:34:22.050032 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-db-create-j9nw5" Dec 03 12:34:22 crc kubenswrapper[5052]: I1203 12:34:22.057128 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-3a0d-account-create-update-wn6fm" Dec 03 12:34:22 crc kubenswrapper[5052]: I1203 12:34:22.093033 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/88758015-0ff4-4b36-bb8b-3b914824871c-combined-ca-bundle\") pod \"keystone-db-sync-v5287\" (UID: \"88758015-0ff4-4b36-bb8b-3b914824871c\") " pod="openstack-kuttl-tests/keystone-db-sync-v5287" Dec 03 12:34:22 crc kubenswrapper[5052]: I1203 12:34:22.093153 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/88758015-0ff4-4b36-bb8b-3b914824871c-config-data\") pod \"keystone-db-sync-v5287\" (UID: \"88758015-0ff4-4b36-bb8b-3b914824871c\") " pod="openstack-kuttl-tests/keystone-db-sync-v5287" Dec 03 12:34:22 crc kubenswrapper[5052]: I1203 12:34:22.093192 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9v7fx\" (UniqueName: \"kubernetes.io/projected/f5e0e5f2-c417-4ebc-b903-25e3b3803964-kube-api-access-9v7fx\") pod \"neutron-d24e-account-create-update-gskxw\" (UID: \"f5e0e5f2-c417-4ebc-b903-25e3b3803964\") " pod="openstack-kuttl-tests/neutron-d24e-account-create-update-gskxw" Dec 03 12:34:22 crc kubenswrapper[5052]: I1203 12:34:22.093216 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g465c\" (UniqueName: \"kubernetes.io/projected/88758015-0ff4-4b36-bb8b-3b914824871c-kube-api-access-g465c\") pod \"keystone-db-sync-v5287\" (UID: \"88758015-0ff4-4b36-bb8b-3b914824871c\") " pod="openstack-kuttl-tests/keystone-db-sync-v5287" Dec 03 12:34:22 crc kubenswrapper[5052]: I1203 12:34:22.093238 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/f5e0e5f2-c417-4ebc-b903-25e3b3803964-operator-scripts\") pod \"neutron-d24e-account-create-update-gskxw\" (UID: \"f5e0e5f2-c417-4ebc-b903-25e3b3803964\") " pod="openstack-kuttl-tests/neutron-d24e-account-create-update-gskxw" Dec 03 12:34:22 crc kubenswrapper[5052]: I1203 12:34:22.115867 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-1171-account-create-update-gm2mn" Dec 03 12:34:22 crc kubenswrapper[5052]: I1203 12:34:22.195074 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/88758015-0ff4-4b36-bb8b-3b914824871c-config-data\") pod \"keystone-db-sync-v5287\" (UID: \"88758015-0ff4-4b36-bb8b-3b914824871c\") " pod="openstack-kuttl-tests/keystone-db-sync-v5287" Dec 03 12:34:22 crc kubenswrapper[5052]: I1203 12:34:22.195620 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-9v7fx\" (UniqueName: \"kubernetes.io/projected/f5e0e5f2-c417-4ebc-b903-25e3b3803964-kube-api-access-9v7fx\") pod \"neutron-d24e-account-create-update-gskxw\" (UID: \"f5e0e5f2-c417-4ebc-b903-25e3b3803964\") " pod="openstack-kuttl-tests/neutron-d24e-account-create-update-gskxw" Dec 03 12:34:22 crc kubenswrapper[5052]: I1203 12:34:22.195722 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-g465c\" (UniqueName: \"kubernetes.io/projected/88758015-0ff4-4b36-bb8b-3b914824871c-kube-api-access-g465c\") pod \"keystone-db-sync-v5287\" (UID: \"88758015-0ff4-4b36-bb8b-3b914824871c\") " pod="openstack-kuttl-tests/keystone-db-sync-v5287" Dec 03 12:34:22 crc kubenswrapper[5052]: I1203 12:34:22.195815 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/f5e0e5f2-c417-4ebc-b903-25e3b3803964-operator-scripts\") pod \"neutron-d24e-account-create-update-gskxw\" (UID: \"f5e0e5f2-c417-4ebc-b903-25e3b3803964\") " pod="openstack-kuttl-tests/neutron-d24e-account-create-update-gskxw" Dec 03 12:34:22 crc kubenswrapper[5052]: I1203 12:34:22.195921 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/88758015-0ff4-4b36-bb8b-3b914824871c-combined-ca-bundle\") pod \"keystone-db-sync-v5287\" (UID: \"88758015-0ff4-4b36-bb8b-3b914824871c\") " pod="openstack-kuttl-tests/keystone-db-sync-v5287" Dec 03 12:34:22 crc kubenswrapper[5052]: I1203 12:34:22.199879 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/f5e0e5f2-c417-4ebc-b903-25e3b3803964-operator-scripts\") pod \"neutron-d24e-account-create-update-gskxw\" (UID: \"f5e0e5f2-c417-4ebc-b903-25e3b3803964\") " pod="openstack-kuttl-tests/neutron-d24e-account-create-update-gskxw" Dec 03 12:34:22 crc kubenswrapper[5052]: I1203 12:34:22.200550 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-db-create-78h7j" Dec 03 12:34:22 crc kubenswrapper[5052]: I1203 12:34:22.209365 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/88758015-0ff4-4b36-bb8b-3b914824871c-config-data\") pod \"keystone-db-sync-v5287\" (UID: \"88758015-0ff4-4b36-bb8b-3b914824871c\") " pod="openstack-kuttl-tests/keystone-db-sync-v5287" Dec 03 12:34:22 crc kubenswrapper[5052]: I1203 12:34:22.216844 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/88758015-0ff4-4b36-bb8b-3b914824871c-combined-ca-bundle\") pod \"keystone-db-sync-v5287\" (UID: \"88758015-0ff4-4b36-bb8b-3b914824871c\") " pod="openstack-kuttl-tests/keystone-db-sync-v5287" Dec 03 12:34:22 crc kubenswrapper[5052]: I1203 12:34:22.233083 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-g465c\" (UniqueName: \"kubernetes.io/projected/88758015-0ff4-4b36-bb8b-3b914824871c-kube-api-access-g465c\") pod \"keystone-db-sync-v5287\" (UID: \"88758015-0ff4-4b36-bb8b-3b914824871c\") " pod="openstack-kuttl-tests/keystone-db-sync-v5287" Dec 03 12:34:22 crc kubenswrapper[5052]: I1203 12:34:22.234285 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-9v7fx\" (UniqueName: \"kubernetes.io/projected/f5e0e5f2-c417-4ebc-b903-25e3b3803964-kube-api-access-9v7fx\") pod \"neutron-d24e-account-create-update-gskxw\" (UID: \"f5e0e5f2-c417-4ebc-b903-25e3b3803964\") " pod="openstack-kuttl-tests/neutron-d24e-account-create-update-gskxw" Dec 03 12:34:22 crc kubenswrapper[5052]: I1203 12:34:22.473874 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-db-create-kc855"] Dec 03 12:34:22 crc kubenswrapper[5052]: W1203 12:34:22.493336 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddde85075_240d_43d9_a002_cc0206cfebf8.slice/crio-6a32aebcbab55e6358804b53077194bc539d2e228e666937073b0c4710496416 WatchSource:0}: Error finding container 6a32aebcbab55e6358804b53077194bc539d2e228e666937073b0c4710496416: Status 404 returned error can't find the container with id 6a32aebcbab55e6358804b53077194bc539d2e228e666937073b0c4710496416 Dec 03 12:34:22 crc kubenswrapper[5052]: I1203 12:34:22.493618 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-d24e-account-create-update-gskxw" Dec 03 12:34:22 crc kubenswrapper[5052]: I1203 12:34:22.493618 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-db-sync-v5287" Dec 03 12:34:22 crc kubenswrapper[5052]: I1203 12:34:22.622114 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-3a0d-account-create-update-wn6fm"] Dec 03 12:34:22 crc kubenswrapper[5052]: W1203 12:34:22.639067 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda6e548d7_7349_413c_a4f7_914f4df80c4b.slice/crio-ad9bc64435aa9055c85c9de11436cab7587cba1b60a5a38d32fa3cb8b4631bdd WatchSource:0}: Error finding container ad9bc64435aa9055c85c9de11436cab7587cba1b60a5a38d32fa3cb8b4631bdd: Status 404 returned error can't find the container with id ad9bc64435aa9055c85c9de11436cab7587cba1b60a5a38d32fa3cb8b4631bdd Dec 03 12:34:22 crc kubenswrapper[5052]: I1203 12:34:22.674259 5052 scope.go:117] "RemoveContainer" containerID="e6edaf188f85ea4f11f487f11cc489c700f25fb42b5ee269f4f496f1e127d772" Dec 03 12:34:22 crc kubenswrapper[5052]: E1203 12:34:22.674504 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 12:34:22 crc kubenswrapper[5052]: I1203 12:34:22.685834 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-db-create-j9nw5"] Dec 03 12:34:22 crc kubenswrapper[5052]: I1203 12:34:22.857271 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-3a0d-account-create-update-wn6fm" event={"ID":"a6e548d7-7349-413c-a4f7-914f4df80c4b","Type":"ContainerStarted","Data":"ad9bc64435aa9055c85c9de11436cab7587cba1b60a5a38d32fa3cb8b4631bdd"} Dec 03 12:34:22 crc kubenswrapper[5052]: I1203 12:34:22.867937 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-db-create-j9nw5" event={"ID":"2f122603-9186-43bc-b3c7-b21307f24c5a","Type":"ContainerStarted","Data":"8535b50e00fe002204958ab12203f06b6481259971f25557fdfbcd0782b2f14c"} Dec 03 12:34:22 crc kubenswrapper[5052]: I1203 12:34:22.876995 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-1171-account-create-update-gm2mn"] Dec 03 12:34:22 crc kubenswrapper[5052]: I1203 12:34:22.877797 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-db-create-kc855" event={"ID":"dde85075-240d-43d9-a002-cc0206cfebf8","Type":"ContainerStarted","Data":"6a32aebcbab55e6358804b53077194bc539d2e228e666937073b0c4710496416"} Dec 03 12:34:22 crc kubenswrapper[5052]: W1203 12:34:22.885911 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod65fef887_25c4_46ee_8a9b_cc82f32dcbe3.slice/crio-2e79a03a9fcbe48818be5e3aaa4ca5f3716fc0bf3fede3881b5d91be6789289c WatchSource:0}: Error finding container 2e79a03a9fcbe48818be5e3aaa4ca5f3716fc0bf3fede3881b5d91be6789289c: Status 404 returned error can't find the container with id 2e79a03a9fcbe48818be5e3aaa4ca5f3716fc0bf3fede3881b5d91be6789289c Dec 03 12:34:22 crc kubenswrapper[5052]: I1203 12:34:22.893784 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-db-create-78h7j"] Dec 03 12:34:22 crc kubenswrapper[5052]: W1203 12:34:22.895163 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2a13e864_04af_42a7_bfbb_9995231b567d.slice/crio-005bd557bb69c320b11a555a11a32e8c6fafb462fa596024a4d6ec96516b0f26 WatchSource:0}: Error finding container 005bd557bb69c320b11a555a11a32e8c6fafb462fa596024a4d6ec96516b0f26: Status 404 returned error can't find the container with id 005bd557bb69c320b11a555a11a32e8c6fafb462fa596024a4d6ec96516b0f26 Dec 03 12:34:23 crc kubenswrapper[5052]: I1203 12:34:23.050669 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-db-sync-v5287"] Dec 03 12:34:23 crc kubenswrapper[5052]: I1203 12:34:23.064660 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-d24e-account-create-update-gskxw"] Dec 03 12:34:24 crc kubenswrapper[5052]: I1203 12:34:24.739350 5052 generic.go:334] "Generic (PLEG): container finished" podID="dde85075-240d-43d9-a002-cc0206cfebf8" containerID="2276cac3a17897b8e09aaa253d140d85fe4fd6c6e7497729651aa093fde9b97f" exitCode=0 Dec 03 12:34:24 crc kubenswrapper[5052]: I1203 12:34:24.739718 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-db-create-kc855" event={"ID":"dde85075-240d-43d9-a002-cc0206cfebf8","Type":"ContainerDied","Data":"2276cac3a17897b8e09aaa253d140d85fe4fd6c6e7497729651aa093fde9b97f"} Dec 03 12:34:24 crc kubenswrapper[5052]: I1203 12:34:24.757659 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-db-sync-v5287" event={"ID":"88758015-0ff4-4b36-bb8b-3b914824871c","Type":"ContainerStarted","Data":"a528d1d26acedffbb3df81e3ae121d9580353eb5c588f2c9e283415c9bf7b940"} Dec 03 12:34:24 crc kubenswrapper[5052]: I1203 12:34:24.775620 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-1171-account-create-update-gm2mn" event={"ID":"65fef887-25c4-46ee-8a9b-cc82f32dcbe3","Type":"ContainerStarted","Data":"2e79a03a9fcbe48818be5e3aaa4ca5f3716fc0bf3fede3881b5d91be6789289c"} Dec 03 12:34:24 crc kubenswrapper[5052]: I1203 12:34:24.802137 5052 generic.go:334] "Generic (PLEG): container finished" podID="a6e548d7-7349-413c-a4f7-914f4df80c4b" containerID="85c2b1d766a8ae5a49909c56750cc7a4bbef2172c6a1149c8104f3518834e4b6" exitCode=0 Dec 03 12:34:24 crc kubenswrapper[5052]: I1203 12:34:24.802267 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-3a0d-account-create-update-wn6fm" event={"ID":"a6e548d7-7349-413c-a4f7-914f4df80c4b","Type":"ContainerDied","Data":"85c2b1d766a8ae5a49909c56750cc7a4bbef2172c6a1149c8104f3518834e4b6"} Dec 03 12:34:24 crc kubenswrapper[5052]: I1203 12:34:24.828142 5052 generic.go:334] "Generic (PLEG): container finished" podID="2f122603-9186-43bc-b3c7-b21307f24c5a" containerID="f179d5351b61865d029b504ef4c3a0f25f02e76884020ed26f254aca9ad99586" exitCode=0 Dec 03 12:34:24 crc kubenswrapper[5052]: I1203 12:34:24.828259 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-db-create-j9nw5" event={"ID":"2f122603-9186-43bc-b3c7-b21307f24c5a","Type":"ContainerDied","Data":"f179d5351b61865d029b504ef4c3a0f25f02e76884020ed26f254aca9ad99586"} Dec 03 12:34:24 crc kubenswrapper[5052]: I1203 12:34:24.852922 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-d24e-account-create-update-gskxw" event={"ID":"f5e0e5f2-c417-4ebc-b903-25e3b3803964","Type":"ContainerStarted","Data":"bb0d9568816f0f4a22b9f8b03e583c31944d718b4e507d39d59b602c744df16a"} Dec 03 12:34:24 crc kubenswrapper[5052]: I1203 12:34:24.854844 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-db-create-78h7j" event={"ID":"2a13e864-04af-42a7-bfbb-9995231b567d","Type":"ContainerStarted","Data":"005bd557bb69c320b11a555a11a32e8c6fafb462fa596024a4d6ec96516b0f26"} Dec 03 12:34:25 crc kubenswrapper[5052]: E1203 12:34:25.137171 5052 cadvisor_stats_provider.go:516] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddde85075_240d_43d9_a002_cc0206cfebf8.slice/crio-2276cac3a17897b8e09aaa253d140d85fe4fd6c6e7497729651aa093fde9b97f.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda6e548d7_7349_413c_a4f7_914f4df80c4b.slice/crio-85c2b1d766a8ae5a49909c56750cc7a4bbef2172c6a1149c8104f3518834e4b6.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddde85075_240d_43d9_a002_cc0206cfebf8.slice/crio-conmon-2276cac3a17897b8e09aaa253d140d85fe4fd6c6e7497729651aa093fde9b97f.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda6e548d7_7349_413c_a4f7_914f4df80c4b.slice/crio-conmon-85c2b1d766a8ae5a49909c56750cc7a4bbef2172c6a1149c8104f3518834e4b6.scope\": RecentStats: unable to find data in memory cache]" Dec 03 12:34:25 crc kubenswrapper[5052]: I1203 12:34:25.866014 5052 generic.go:334] "Generic (PLEG): container finished" podID="65fef887-25c4-46ee-8a9b-cc82f32dcbe3" containerID="5bb5bef178a578ef338d8e4c3a444c4fd52a248e069fcb9184e3dd53a4555372" exitCode=0 Dec 03 12:34:25 crc kubenswrapper[5052]: I1203 12:34:25.866106 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-1171-account-create-update-gm2mn" event={"ID":"65fef887-25c4-46ee-8a9b-cc82f32dcbe3","Type":"ContainerDied","Data":"5bb5bef178a578ef338d8e4c3a444c4fd52a248e069fcb9184e3dd53a4555372"} Dec 03 12:34:25 crc kubenswrapper[5052]: I1203 12:34:25.869423 5052 generic.go:334] "Generic (PLEG): container finished" podID="f5e0e5f2-c417-4ebc-b903-25e3b3803964" containerID="c2385170deb299c29625c309b58a0737df77e5b8f042cdb7e41b0523b06cd3ed" exitCode=0 Dec 03 12:34:25 crc kubenswrapper[5052]: I1203 12:34:25.869448 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-d24e-account-create-update-gskxw" event={"ID":"f5e0e5f2-c417-4ebc-b903-25e3b3803964","Type":"ContainerDied","Data":"c2385170deb299c29625c309b58a0737df77e5b8f042cdb7e41b0523b06cd3ed"} Dec 03 12:34:25 crc kubenswrapper[5052]: I1203 12:34:25.871094 5052 generic.go:334] "Generic (PLEG): container finished" podID="2a13e864-04af-42a7-bfbb-9995231b567d" containerID="1e61208c62ab46fd9d699374a5c8b4391a3017ecb2c3822df67c5b7f305c2abc" exitCode=0 Dec 03 12:34:25 crc kubenswrapper[5052]: I1203 12:34:25.871133 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-db-create-78h7j" event={"ID":"2a13e864-04af-42a7-bfbb-9995231b567d","Type":"ContainerDied","Data":"1e61208c62ab46fd9d699374a5c8b4391a3017ecb2c3822df67c5b7f305c2abc"} Dec 03 12:34:25 crc kubenswrapper[5052]: I1203 12:34:25.873680 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-db-sync-v5287" event={"ID":"88758015-0ff4-4b36-bb8b-3b914824871c","Type":"ContainerStarted","Data":"b9fb898ae650a25991bcaf8b4202e50ad89d244003a528574792da3d36870f29"} Dec 03 12:34:25 crc kubenswrapper[5052]: I1203 12:34:25.935632 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/keystone-db-sync-v5287" podStartSLOduration=4.935613353 podStartE2EDuration="4.935613353s" podCreationTimestamp="2025-12-03 12:34:21 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:34:25.934934973 +0000 UTC m=+4456.523176737" watchObservedRunningTime="2025-12-03 12:34:25.935613353 +0000 UTC m=+4456.523855117" Dec 03 12:34:26 crc kubenswrapper[5052]: I1203 12:34:26.383518 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-db-create-j9nw5" Dec 03 12:34:26 crc kubenswrapper[5052]: I1203 12:34:26.475701 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-67z7x\" (UniqueName: \"kubernetes.io/projected/2f122603-9186-43bc-b3c7-b21307f24c5a-kube-api-access-67z7x\") pod \"2f122603-9186-43bc-b3c7-b21307f24c5a\" (UID: \"2f122603-9186-43bc-b3c7-b21307f24c5a\") " Dec 03 12:34:26 crc kubenswrapper[5052]: I1203 12:34:26.475815 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/2f122603-9186-43bc-b3c7-b21307f24c5a-operator-scripts\") pod \"2f122603-9186-43bc-b3c7-b21307f24c5a\" (UID: \"2f122603-9186-43bc-b3c7-b21307f24c5a\") " Dec 03 12:34:26 crc kubenswrapper[5052]: I1203 12:34:26.476864 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2f122603-9186-43bc-b3c7-b21307f24c5a-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "2f122603-9186-43bc-b3c7-b21307f24c5a" (UID: "2f122603-9186-43bc-b3c7-b21307f24c5a"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:34:26 crc kubenswrapper[5052]: I1203 12:34:26.482326 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2f122603-9186-43bc-b3c7-b21307f24c5a-kube-api-access-67z7x" (OuterVolumeSpecName: "kube-api-access-67z7x") pod "2f122603-9186-43bc-b3c7-b21307f24c5a" (UID: "2f122603-9186-43bc-b3c7-b21307f24c5a"). InnerVolumeSpecName "kube-api-access-67z7x". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:34:26 crc kubenswrapper[5052]: I1203 12:34:26.559593 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-db-create-kc855" Dec 03 12:34:26 crc kubenswrapper[5052]: I1203 12:34:26.566812 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-3a0d-account-create-update-wn6fm" Dec 03 12:34:26 crc kubenswrapper[5052]: I1203 12:34:26.576974 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zfdhw\" (UniqueName: \"kubernetes.io/projected/a6e548d7-7349-413c-a4f7-914f4df80c4b-kube-api-access-zfdhw\") pod \"a6e548d7-7349-413c-a4f7-914f4df80c4b\" (UID: \"a6e548d7-7349-413c-a4f7-914f4df80c4b\") " Dec 03 12:34:26 crc kubenswrapper[5052]: I1203 12:34:26.577117 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6ffml\" (UniqueName: \"kubernetes.io/projected/dde85075-240d-43d9-a002-cc0206cfebf8-kube-api-access-6ffml\") pod \"dde85075-240d-43d9-a002-cc0206cfebf8\" (UID: \"dde85075-240d-43d9-a002-cc0206cfebf8\") " Dec 03 12:34:26 crc kubenswrapper[5052]: I1203 12:34:26.577260 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/a6e548d7-7349-413c-a4f7-914f4df80c4b-operator-scripts\") pod \"a6e548d7-7349-413c-a4f7-914f4df80c4b\" (UID: \"a6e548d7-7349-413c-a4f7-914f4df80c4b\") " Dec 03 12:34:26 crc kubenswrapper[5052]: I1203 12:34:26.577323 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/dde85075-240d-43d9-a002-cc0206cfebf8-operator-scripts\") pod \"dde85075-240d-43d9-a002-cc0206cfebf8\" (UID: \"dde85075-240d-43d9-a002-cc0206cfebf8\") " Dec 03 12:34:26 crc kubenswrapper[5052]: I1203 12:34:26.577711 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/2f122603-9186-43bc-b3c7-b21307f24c5a-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:34:26 crc kubenswrapper[5052]: I1203 12:34:26.577736 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-67z7x\" (UniqueName: \"kubernetes.io/projected/2f122603-9186-43bc-b3c7-b21307f24c5a-kube-api-access-67z7x\") on node \"crc\" DevicePath \"\"" Dec 03 12:34:26 crc kubenswrapper[5052]: I1203 12:34:26.578036 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a6e548d7-7349-413c-a4f7-914f4df80c4b-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "a6e548d7-7349-413c-a4f7-914f4df80c4b" (UID: "a6e548d7-7349-413c-a4f7-914f4df80c4b"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:34:26 crc kubenswrapper[5052]: I1203 12:34:26.578353 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dde85075-240d-43d9-a002-cc0206cfebf8-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "dde85075-240d-43d9-a002-cc0206cfebf8" (UID: "dde85075-240d-43d9-a002-cc0206cfebf8"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:34:26 crc kubenswrapper[5052]: I1203 12:34:26.601584 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a6e548d7-7349-413c-a4f7-914f4df80c4b-kube-api-access-zfdhw" (OuterVolumeSpecName: "kube-api-access-zfdhw") pod "a6e548d7-7349-413c-a4f7-914f4df80c4b" (UID: "a6e548d7-7349-413c-a4f7-914f4df80c4b"). InnerVolumeSpecName "kube-api-access-zfdhw". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:34:26 crc kubenswrapper[5052]: I1203 12:34:26.609779 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dde85075-240d-43d9-a002-cc0206cfebf8-kube-api-access-6ffml" (OuterVolumeSpecName: "kube-api-access-6ffml") pod "dde85075-240d-43d9-a002-cc0206cfebf8" (UID: "dde85075-240d-43d9-a002-cc0206cfebf8"). InnerVolumeSpecName "kube-api-access-6ffml". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:34:26 crc kubenswrapper[5052]: I1203 12:34:26.678667 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-6ffml\" (UniqueName: \"kubernetes.io/projected/dde85075-240d-43d9-a002-cc0206cfebf8-kube-api-access-6ffml\") on node \"crc\" DevicePath \"\"" Dec 03 12:34:26 crc kubenswrapper[5052]: I1203 12:34:26.678717 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/a6e548d7-7349-413c-a4f7-914f4df80c4b-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:34:26 crc kubenswrapper[5052]: I1203 12:34:26.678729 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/dde85075-240d-43d9-a002-cc0206cfebf8-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:34:26 crc kubenswrapper[5052]: I1203 12:34:26.678741 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-zfdhw\" (UniqueName: \"kubernetes.io/projected/a6e548d7-7349-413c-a4f7-914f4df80c4b-kube-api-access-zfdhw\") on node \"crc\" DevicePath \"\"" Dec 03 12:34:26 crc kubenswrapper[5052]: I1203 12:34:26.883358 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-3a0d-account-create-update-wn6fm" event={"ID":"a6e548d7-7349-413c-a4f7-914f4df80c4b","Type":"ContainerDied","Data":"ad9bc64435aa9055c85c9de11436cab7587cba1b60a5a38d32fa3cb8b4631bdd"} Dec 03 12:34:26 crc kubenswrapper[5052]: I1203 12:34:26.883736 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ad9bc64435aa9055c85c9de11436cab7587cba1b60a5a38d32fa3cb8b4631bdd" Dec 03 12:34:26 crc kubenswrapper[5052]: I1203 12:34:26.883380 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-3a0d-account-create-update-wn6fm" Dec 03 12:34:26 crc kubenswrapper[5052]: I1203 12:34:26.885261 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-db-create-j9nw5" event={"ID":"2f122603-9186-43bc-b3c7-b21307f24c5a","Type":"ContainerDied","Data":"8535b50e00fe002204958ab12203f06b6481259971f25557fdfbcd0782b2f14c"} Dec 03 12:34:26 crc kubenswrapper[5052]: I1203 12:34:26.885297 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8535b50e00fe002204958ab12203f06b6481259971f25557fdfbcd0782b2f14c" Dec 03 12:34:26 crc kubenswrapper[5052]: I1203 12:34:26.885714 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-db-create-j9nw5" Dec 03 12:34:26 crc kubenswrapper[5052]: I1203 12:34:26.886610 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-db-create-kc855" Dec 03 12:34:26 crc kubenswrapper[5052]: I1203 12:34:26.886639 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-db-create-kc855" event={"ID":"dde85075-240d-43d9-a002-cc0206cfebf8","Type":"ContainerDied","Data":"6a32aebcbab55e6358804b53077194bc539d2e228e666937073b0c4710496416"} Dec 03 12:34:26 crc kubenswrapper[5052]: I1203 12:34:26.886658 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6a32aebcbab55e6358804b53077194bc539d2e228e666937073b0c4710496416" Dec 03 12:34:27 crc kubenswrapper[5052]: I1203 12:34:27.244574 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-1171-account-create-update-gm2mn" Dec 03 12:34:27 crc kubenswrapper[5052]: I1203 12:34:27.295271 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/65fef887-25c4-46ee-8a9b-cc82f32dcbe3-operator-scripts\") pod \"65fef887-25c4-46ee-8a9b-cc82f32dcbe3\" (UID: \"65fef887-25c4-46ee-8a9b-cc82f32dcbe3\") " Dec 03 12:34:27 crc kubenswrapper[5052]: I1203 12:34:27.295479 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z99zx\" (UniqueName: \"kubernetes.io/projected/65fef887-25c4-46ee-8a9b-cc82f32dcbe3-kube-api-access-z99zx\") pod \"65fef887-25c4-46ee-8a9b-cc82f32dcbe3\" (UID: \"65fef887-25c4-46ee-8a9b-cc82f32dcbe3\") " Dec 03 12:34:27 crc kubenswrapper[5052]: I1203 12:34:27.296995 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/65fef887-25c4-46ee-8a9b-cc82f32dcbe3-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "65fef887-25c4-46ee-8a9b-cc82f32dcbe3" (UID: "65fef887-25c4-46ee-8a9b-cc82f32dcbe3"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:34:27 crc kubenswrapper[5052]: I1203 12:34:27.299574 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/65fef887-25c4-46ee-8a9b-cc82f32dcbe3-kube-api-access-z99zx" (OuterVolumeSpecName: "kube-api-access-z99zx") pod "65fef887-25c4-46ee-8a9b-cc82f32dcbe3" (UID: "65fef887-25c4-46ee-8a9b-cc82f32dcbe3"). InnerVolumeSpecName "kube-api-access-z99zx". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:34:27 crc kubenswrapper[5052]: I1203 12:34:27.396650 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-db-create-78h7j" Dec 03 12:34:27 crc kubenswrapper[5052]: I1203 12:34:27.398612 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/65fef887-25c4-46ee-8a9b-cc82f32dcbe3-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:34:27 crc kubenswrapper[5052]: I1203 12:34:27.398981 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-z99zx\" (UniqueName: \"kubernetes.io/projected/65fef887-25c4-46ee-8a9b-cc82f32dcbe3-kube-api-access-z99zx\") on node \"crc\" DevicePath \"\"" Dec 03 12:34:27 crc kubenswrapper[5052]: I1203 12:34:27.406920 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-d24e-account-create-update-gskxw" Dec 03 12:34:27 crc kubenswrapper[5052]: I1203 12:34:27.501029 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m9865\" (UniqueName: \"kubernetes.io/projected/2a13e864-04af-42a7-bfbb-9995231b567d-kube-api-access-m9865\") pod \"2a13e864-04af-42a7-bfbb-9995231b567d\" (UID: \"2a13e864-04af-42a7-bfbb-9995231b567d\") " Dec 03 12:34:27 crc kubenswrapper[5052]: I1203 12:34:27.501161 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/2a13e864-04af-42a7-bfbb-9995231b567d-operator-scripts\") pod \"2a13e864-04af-42a7-bfbb-9995231b567d\" (UID: \"2a13e864-04af-42a7-bfbb-9995231b567d\") " Dec 03 12:34:27 crc kubenswrapper[5052]: I1203 12:34:27.501290 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9v7fx\" (UniqueName: \"kubernetes.io/projected/f5e0e5f2-c417-4ebc-b903-25e3b3803964-kube-api-access-9v7fx\") pod \"f5e0e5f2-c417-4ebc-b903-25e3b3803964\" (UID: \"f5e0e5f2-c417-4ebc-b903-25e3b3803964\") " Dec 03 12:34:27 crc kubenswrapper[5052]: I1203 12:34:27.501335 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/f5e0e5f2-c417-4ebc-b903-25e3b3803964-operator-scripts\") pod \"f5e0e5f2-c417-4ebc-b903-25e3b3803964\" (UID: \"f5e0e5f2-c417-4ebc-b903-25e3b3803964\") " Dec 03 12:34:27 crc kubenswrapper[5052]: I1203 12:34:27.502071 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2a13e864-04af-42a7-bfbb-9995231b567d-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "2a13e864-04af-42a7-bfbb-9995231b567d" (UID: "2a13e864-04af-42a7-bfbb-9995231b567d"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:34:27 crc kubenswrapper[5052]: I1203 12:34:27.502368 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f5e0e5f2-c417-4ebc-b903-25e3b3803964-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "f5e0e5f2-c417-4ebc-b903-25e3b3803964" (UID: "f5e0e5f2-c417-4ebc-b903-25e3b3803964"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:34:27 crc kubenswrapper[5052]: I1203 12:34:27.505943 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2a13e864-04af-42a7-bfbb-9995231b567d-kube-api-access-m9865" (OuterVolumeSpecName: "kube-api-access-m9865") pod "2a13e864-04af-42a7-bfbb-9995231b567d" (UID: "2a13e864-04af-42a7-bfbb-9995231b567d"). InnerVolumeSpecName "kube-api-access-m9865". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:34:27 crc kubenswrapper[5052]: I1203 12:34:27.507776 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f5e0e5f2-c417-4ebc-b903-25e3b3803964-kube-api-access-9v7fx" (OuterVolumeSpecName: "kube-api-access-9v7fx") pod "f5e0e5f2-c417-4ebc-b903-25e3b3803964" (UID: "f5e0e5f2-c417-4ebc-b903-25e3b3803964"). InnerVolumeSpecName "kube-api-access-9v7fx". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:34:27 crc kubenswrapper[5052]: I1203 12:34:27.603654 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-m9865\" (UniqueName: \"kubernetes.io/projected/2a13e864-04af-42a7-bfbb-9995231b567d-kube-api-access-m9865\") on node \"crc\" DevicePath \"\"" Dec 03 12:34:27 crc kubenswrapper[5052]: I1203 12:34:27.603709 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/2a13e864-04af-42a7-bfbb-9995231b567d-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:34:27 crc kubenswrapper[5052]: I1203 12:34:27.603724 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-9v7fx\" (UniqueName: \"kubernetes.io/projected/f5e0e5f2-c417-4ebc-b903-25e3b3803964-kube-api-access-9v7fx\") on node \"crc\" DevicePath \"\"" Dec 03 12:34:27 crc kubenswrapper[5052]: I1203 12:34:27.603737 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/f5e0e5f2-c417-4ebc-b903-25e3b3803964-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:34:27 crc kubenswrapper[5052]: I1203 12:34:27.897066 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-d24e-account-create-update-gskxw" Dec 03 12:34:27 crc kubenswrapper[5052]: I1203 12:34:27.897007 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-d24e-account-create-update-gskxw" event={"ID":"f5e0e5f2-c417-4ebc-b903-25e3b3803964","Type":"ContainerDied","Data":"bb0d9568816f0f4a22b9f8b03e583c31944d718b4e507d39d59b602c744df16a"} Dec 03 12:34:27 crc kubenswrapper[5052]: I1203 12:34:27.897301 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bb0d9568816f0f4a22b9f8b03e583c31944d718b4e507d39d59b602c744df16a" Dec 03 12:34:27 crc kubenswrapper[5052]: I1203 12:34:27.899278 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-db-create-78h7j" Dec 03 12:34:27 crc kubenswrapper[5052]: I1203 12:34:27.899270 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-db-create-78h7j" event={"ID":"2a13e864-04af-42a7-bfbb-9995231b567d","Type":"ContainerDied","Data":"005bd557bb69c320b11a555a11a32e8c6fafb462fa596024a4d6ec96516b0f26"} Dec 03 12:34:27 crc kubenswrapper[5052]: I1203 12:34:27.899495 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="005bd557bb69c320b11a555a11a32e8c6fafb462fa596024a4d6ec96516b0f26" Dec 03 12:34:27 crc kubenswrapper[5052]: I1203 12:34:27.901928 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-1171-account-create-update-gm2mn" event={"ID":"65fef887-25c4-46ee-8a9b-cc82f32dcbe3","Type":"ContainerDied","Data":"2e79a03a9fcbe48818be5e3aaa4ca5f3716fc0bf3fede3881b5d91be6789289c"} Dec 03 12:34:27 crc kubenswrapper[5052]: I1203 12:34:27.901971 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-1171-account-create-update-gm2mn" Dec 03 12:34:27 crc kubenswrapper[5052]: I1203 12:34:27.901976 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2e79a03a9fcbe48818be5e3aaa4ca5f3716fc0bf3fede3881b5d91be6789289c" Dec 03 12:34:29 crc kubenswrapper[5052]: I1203 12:34:29.916354 5052 generic.go:334] "Generic (PLEG): container finished" podID="88758015-0ff4-4b36-bb8b-3b914824871c" containerID="b9fb898ae650a25991bcaf8b4202e50ad89d244003a528574792da3d36870f29" exitCode=0 Dec 03 12:34:29 crc kubenswrapper[5052]: I1203 12:34:29.916456 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-db-sync-v5287" event={"ID":"88758015-0ff4-4b36-bb8b-3b914824871c","Type":"ContainerDied","Data":"b9fb898ae650a25991bcaf8b4202e50ad89d244003a528574792da3d36870f29"} Dec 03 12:34:31 crc kubenswrapper[5052]: I1203 12:34:31.265298 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-db-sync-v5287" Dec 03 12:34:31 crc kubenswrapper[5052]: I1203 12:34:31.362572 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g465c\" (UniqueName: \"kubernetes.io/projected/88758015-0ff4-4b36-bb8b-3b914824871c-kube-api-access-g465c\") pod \"88758015-0ff4-4b36-bb8b-3b914824871c\" (UID: \"88758015-0ff4-4b36-bb8b-3b914824871c\") " Dec 03 12:34:31 crc kubenswrapper[5052]: I1203 12:34:31.362689 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/88758015-0ff4-4b36-bb8b-3b914824871c-combined-ca-bundle\") pod \"88758015-0ff4-4b36-bb8b-3b914824871c\" (UID: \"88758015-0ff4-4b36-bb8b-3b914824871c\") " Dec 03 12:34:31 crc kubenswrapper[5052]: I1203 12:34:31.362790 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/88758015-0ff4-4b36-bb8b-3b914824871c-config-data\") pod \"88758015-0ff4-4b36-bb8b-3b914824871c\" (UID: \"88758015-0ff4-4b36-bb8b-3b914824871c\") " Dec 03 12:34:31 crc kubenswrapper[5052]: I1203 12:34:31.368884 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/88758015-0ff4-4b36-bb8b-3b914824871c-kube-api-access-g465c" (OuterVolumeSpecName: "kube-api-access-g465c") pod "88758015-0ff4-4b36-bb8b-3b914824871c" (UID: "88758015-0ff4-4b36-bb8b-3b914824871c"). InnerVolumeSpecName "kube-api-access-g465c". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:34:31 crc kubenswrapper[5052]: I1203 12:34:31.392646 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/88758015-0ff4-4b36-bb8b-3b914824871c-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "88758015-0ff4-4b36-bb8b-3b914824871c" (UID: "88758015-0ff4-4b36-bb8b-3b914824871c"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:34:31 crc kubenswrapper[5052]: I1203 12:34:31.408844 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/88758015-0ff4-4b36-bb8b-3b914824871c-config-data" (OuterVolumeSpecName: "config-data") pod "88758015-0ff4-4b36-bb8b-3b914824871c" (UID: "88758015-0ff4-4b36-bb8b-3b914824871c"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:34:31 crc kubenswrapper[5052]: I1203 12:34:31.463999 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-g465c\" (UniqueName: \"kubernetes.io/projected/88758015-0ff4-4b36-bb8b-3b914824871c-kube-api-access-g465c\") on node \"crc\" DevicePath \"\"" Dec 03 12:34:31 crc kubenswrapper[5052]: I1203 12:34:31.464040 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/88758015-0ff4-4b36-bb8b-3b914824871c-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:34:31 crc kubenswrapper[5052]: I1203 12:34:31.464052 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/88758015-0ff4-4b36-bb8b-3b914824871c-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:34:31 crc kubenswrapper[5052]: I1203 12:34:31.937077 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-db-sync-v5287" event={"ID":"88758015-0ff4-4b36-bb8b-3b914824871c","Type":"ContainerDied","Data":"a528d1d26acedffbb3df81e3ae121d9580353eb5c588f2c9e283415c9bf7b940"} Dec 03 12:34:31 crc kubenswrapper[5052]: I1203 12:34:31.937116 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-db-sync-v5287" Dec 03 12:34:31 crc kubenswrapper[5052]: I1203 12:34:31.937125 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a528d1d26acedffbb3df81e3ae121d9580353eb5c588f2c9e283415c9bf7b940" Dec 03 12:34:32 crc kubenswrapper[5052]: I1203 12:34:32.057423 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/keystone-bootstrap-hvfxb"] Dec 03 12:34:32 crc kubenswrapper[5052]: E1203 12:34:32.057837 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="2a13e864-04af-42a7-bfbb-9995231b567d" containerName="mariadb-database-create" Dec 03 12:34:32 crc kubenswrapper[5052]: I1203 12:34:32.057851 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="2a13e864-04af-42a7-bfbb-9995231b567d" containerName="mariadb-database-create" Dec 03 12:34:32 crc kubenswrapper[5052]: E1203 12:34:32.057891 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="65fef887-25c4-46ee-8a9b-cc82f32dcbe3" containerName="mariadb-account-create-update" Dec 03 12:34:32 crc kubenswrapper[5052]: I1203 12:34:32.057898 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="65fef887-25c4-46ee-8a9b-cc82f32dcbe3" containerName="mariadb-account-create-update" Dec 03 12:34:32 crc kubenswrapper[5052]: E1203 12:34:32.057908 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="a6e548d7-7349-413c-a4f7-914f4df80c4b" containerName="mariadb-account-create-update" Dec 03 12:34:32 crc kubenswrapper[5052]: I1203 12:34:32.057914 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="a6e548d7-7349-413c-a4f7-914f4df80c4b" containerName="mariadb-account-create-update" Dec 03 12:34:32 crc kubenswrapper[5052]: E1203 12:34:32.057933 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="f5e0e5f2-c417-4ebc-b903-25e3b3803964" containerName="mariadb-account-create-update" Dec 03 12:34:32 crc kubenswrapper[5052]: I1203 12:34:32.057939 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="f5e0e5f2-c417-4ebc-b903-25e3b3803964" containerName="mariadb-account-create-update" Dec 03 12:34:32 crc kubenswrapper[5052]: E1203 12:34:32.057951 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="2f122603-9186-43bc-b3c7-b21307f24c5a" containerName="mariadb-database-create" Dec 03 12:34:32 crc kubenswrapper[5052]: I1203 12:34:32.057958 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f122603-9186-43bc-b3c7-b21307f24c5a" containerName="mariadb-database-create" Dec 03 12:34:32 crc kubenswrapper[5052]: E1203 12:34:32.057967 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="88758015-0ff4-4b36-bb8b-3b914824871c" containerName="keystone-db-sync" Dec 03 12:34:32 crc kubenswrapper[5052]: I1203 12:34:32.057973 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="88758015-0ff4-4b36-bb8b-3b914824871c" containerName="keystone-db-sync" Dec 03 12:34:32 crc kubenswrapper[5052]: E1203 12:34:32.057981 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="dde85075-240d-43d9-a002-cc0206cfebf8" containerName="mariadb-database-create" Dec 03 12:34:32 crc kubenswrapper[5052]: I1203 12:34:32.057987 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="dde85075-240d-43d9-a002-cc0206cfebf8" containerName="mariadb-database-create" Dec 03 12:34:32 crc kubenswrapper[5052]: I1203 12:34:32.058141 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="65fef887-25c4-46ee-8a9b-cc82f32dcbe3" containerName="mariadb-account-create-update" Dec 03 12:34:32 crc kubenswrapper[5052]: I1203 12:34:32.058157 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="2a13e864-04af-42a7-bfbb-9995231b567d" containerName="mariadb-database-create" Dec 03 12:34:32 crc kubenswrapper[5052]: I1203 12:34:32.058172 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="88758015-0ff4-4b36-bb8b-3b914824871c" containerName="keystone-db-sync" Dec 03 12:34:32 crc kubenswrapper[5052]: I1203 12:34:32.058180 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="a6e548d7-7349-413c-a4f7-914f4df80c4b" containerName="mariadb-account-create-update" Dec 03 12:34:32 crc kubenswrapper[5052]: I1203 12:34:32.058192 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="dde85075-240d-43d9-a002-cc0206cfebf8" containerName="mariadb-database-create" Dec 03 12:34:32 crc kubenswrapper[5052]: I1203 12:34:32.058205 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="2f122603-9186-43bc-b3c7-b21307f24c5a" containerName="mariadb-database-create" Dec 03 12:34:32 crc kubenswrapper[5052]: I1203 12:34:32.058216 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="f5e0e5f2-c417-4ebc-b903-25e3b3803964" containerName="mariadb-account-create-update" Dec 03 12:34:32 crc kubenswrapper[5052]: I1203 12:34:32.062852 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-bootstrap-hvfxb" Dec 03 12:34:32 crc kubenswrapper[5052]: I1203 12:34:32.065739 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"osp-secret" Dec 03 12:34:32 crc kubenswrapper[5052]: I1203 12:34:32.065782 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone-keystone-dockercfg-fvfwv" Dec 03 12:34:32 crc kubenswrapper[5052]: I1203 12:34:32.065948 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone" Dec 03 12:34:32 crc kubenswrapper[5052]: I1203 12:34:32.067171 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone-config-data" Dec 03 12:34:32 crc kubenswrapper[5052]: I1203 12:34:32.067302 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone-scripts" Dec 03 12:34:32 crc kubenswrapper[5052]: I1203 12:34:32.074681 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/80bd8d5d-a98f-455b-977a-e96317d7f525-combined-ca-bundle\") pod \"keystone-bootstrap-hvfxb\" (UID: \"80bd8d5d-a98f-455b-977a-e96317d7f525\") " pod="openstack-kuttl-tests/keystone-bootstrap-hvfxb" Dec 03 12:34:32 crc kubenswrapper[5052]: I1203 12:34:32.074746 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/80bd8d5d-a98f-455b-977a-e96317d7f525-fernet-keys\") pod \"keystone-bootstrap-hvfxb\" (UID: \"80bd8d5d-a98f-455b-977a-e96317d7f525\") " pod="openstack-kuttl-tests/keystone-bootstrap-hvfxb" Dec 03 12:34:32 crc kubenswrapper[5052]: I1203 12:34:32.074877 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-82qfd\" (UniqueName: \"kubernetes.io/projected/80bd8d5d-a98f-455b-977a-e96317d7f525-kube-api-access-82qfd\") pod \"keystone-bootstrap-hvfxb\" (UID: \"80bd8d5d-a98f-455b-977a-e96317d7f525\") " pod="openstack-kuttl-tests/keystone-bootstrap-hvfxb" Dec 03 12:34:32 crc kubenswrapper[5052]: I1203 12:34:32.074948 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/80bd8d5d-a98f-455b-977a-e96317d7f525-credential-keys\") pod \"keystone-bootstrap-hvfxb\" (UID: \"80bd8d5d-a98f-455b-977a-e96317d7f525\") " pod="openstack-kuttl-tests/keystone-bootstrap-hvfxb" Dec 03 12:34:32 crc kubenswrapper[5052]: I1203 12:34:32.074981 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/80bd8d5d-a98f-455b-977a-e96317d7f525-scripts\") pod \"keystone-bootstrap-hvfxb\" (UID: \"80bd8d5d-a98f-455b-977a-e96317d7f525\") " pod="openstack-kuttl-tests/keystone-bootstrap-hvfxb" Dec 03 12:34:32 crc kubenswrapper[5052]: I1203 12:34:32.075034 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/80bd8d5d-a98f-455b-977a-e96317d7f525-config-data\") pod \"keystone-bootstrap-hvfxb\" (UID: \"80bd8d5d-a98f-455b-977a-e96317d7f525\") " pod="openstack-kuttl-tests/keystone-bootstrap-hvfxb" Dec 03 12:34:32 crc kubenswrapper[5052]: I1203 12:34:32.077265 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-bootstrap-hvfxb"] Dec 03 12:34:32 crc kubenswrapper[5052]: I1203 12:34:32.176419 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/80bd8d5d-a98f-455b-977a-e96317d7f525-combined-ca-bundle\") pod \"keystone-bootstrap-hvfxb\" (UID: \"80bd8d5d-a98f-455b-977a-e96317d7f525\") " pod="openstack-kuttl-tests/keystone-bootstrap-hvfxb" Dec 03 12:34:32 crc kubenswrapper[5052]: I1203 12:34:32.176476 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/80bd8d5d-a98f-455b-977a-e96317d7f525-fernet-keys\") pod \"keystone-bootstrap-hvfxb\" (UID: \"80bd8d5d-a98f-455b-977a-e96317d7f525\") " pod="openstack-kuttl-tests/keystone-bootstrap-hvfxb" Dec 03 12:34:32 crc kubenswrapper[5052]: I1203 12:34:32.176528 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-82qfd\" (UniqueName: \"kubernetes.io/projected/80bd8d5d-a98f-455b-977a-e96317d7f525-kube-api-access-82qfd\") pod \"keystone-bootstrap-hvfxb\" (UID: \"80bd8d5d-a98f-455b-977a-e96317d7f525\") " pod="openstack-kuttl-tests/keystone-bootstrap-hvfxb" Dec 03 12:34:32 crc kubenswrapper[5052]: I1203 12:34:32.176579 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/80bd8d5d-a98f-455b-977a-e96317d7f525-credential-keys\") pod \"keystone-bootstrap-hvfxb\" (UID: \"80bd8d5d-a98f-455b-977a-e96317d7f525\") " pod="openstack-kuttl-tests/keystone-bootstrap-hvfxb" Dec 03 12:34:32 crc kubenswrapper[5052]: I1203 12:34:32.176601 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/80bd8d5d-a98f-455b-977a-e96317d7f525-scripts\") pod \"keystone-bootstrap-hvfxb\" (UID: \"80bd8d5d-a98f-455b-977a-e96317d7f525\") " pod="openstack-kuttl-tests/keystone-bootstrap-hvfxb" Dec 03 12:34:32 crc kubenswrapper[5052]: I1203 12:34:32.176627 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/80bd8d5d-a98f-455b-977a-e96317d7f525-config-data\") pod \"keystone-bootstrap-hvfxb\" (UID: \"80bd8d5d-a98f-455b-977a-e96317d7f525\") " pod="openstack-kuttl-tests/keystone-bootstrap-hvfxb" Dec 03 12:34:32 crc kubenswrapper[5052]: I1203 12:34:32.224995 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/80bd8d5d-a98f-455b-977a-e96317d7f525-scripts\") pod \"keystone-bootstrap-hvfxb\" (UID: \"80bd8d5d-a98f-455b-977a-e96317d7f525\") " pod="openstack-kuttl-tests/keystone-bootstrap-hvfxb" Dec 03 12:34:32 crc kubenswrapper[5052]: I1203 12:34:32.225750 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/80bd8d5d-a98f-455b-977a-e96317d7f525-config-data\") pod \"keystone-bootstrap-hvfxb\" (UID: \"80bd8d5d-a98f-455b-977a-e96317d7f525\") " pod="openstack-kuttl-tests/keystone-bootstrap-hvfxb" Dec 03 12:34:32 crc kubenswrapper[5052]: I1203 12:34:32.226091 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/80bd8d5d-a98f-455b-977a-e96317d7f525-fernet-keys\") pod \"keystone-bootstrap-hvfxb\" (UID: \"80bd8d5d-a98f-455b-977a-e96317d7f525\") " pod="openstack-kuttl-tests/keystone-bootstrap-hvfxb" Dec 03 12:34:32 crc kubenswrapper[5052]: I1203 12:34:32.226174 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/80bd8d5d-a98f-455b-977a-e96317d7f525-credential-keys\") pod \"keystone-bootstrap-hvfxb\" (UID: \"80bd8d5d-a98f-455b-977a-e96317d7f525\") " pod="openstack-kuttl-tests/keystone-bootstrap-hvfxb" Dec 03 12:34:32 crc kubenswrapper[5052]: I1203 12:34:32.226422 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-82qfd\" (UniqueName: \"kubernetes.io/projected/80bd8d5d-a98f-455b-977a-e96317d7f525-kube-api-access-82qfd\") pod \"keystone-bootstrap-hvfxb\" (UID: \"80bd8d5d-a98f-455b-977a-e96317d7f525\") " pod="openstack-kuttl-tests/keystone-bootstrap-hvfxb" Dec 03 12:34:32 crc kubenswrapper[5052]: I1203 12:34:32.226940 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/80bd8d5d-a98f-455b-977a-e96317d7f525-combined-ca-bundle\") pod \"keystone-bootstrap-hvfxb\" (UID: \"80bd8d5d-a98f-455b-977a-e96317d7f525\") " pod="openstack-kuttl-tests/keystone-bootstrap-hvfxb" Dec 03 12:34:32 crc kubenswrapper[5052]: I1203 12:34:32.260056 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:34:32 crc kubenswrapper[5052]: I1203 12:34:32.262101 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:34:32 crc kubenswrapper[5052]: I1203 12:34:32.264069 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-scripts" Dec 03 12:34:32 crc kubenswrapper[5052]: I1203 12:34:32.264166 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-config-data" Dec 03 12:34:32 crc kubenswrapper[5052]: I1203 12:34:32.277909 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6e18c0ba-8f07-4d54-b383-3d0164780725-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"6e18c0ba-8f07-4d54-b383-3d0164780725\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:34:32 crc kubenswrapper[5052]: I1203 12:34:32.277973 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/6e18c0ba-8f07-4d54-b383-3d0164780725-scripts\") pod \"ceilometer-0\" (UID: \"6e18c0ba-8f07-4d54-b383-3d0164780725\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:34:32 crc kubenswrapper[5052]: I1203 12:34:32.277995 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/6e18c0ba-8f07-4d54-b383-3d0164780725-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"6e18c0ba-8f07-4d54-b383-3d0164780725\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:34:32 crc kubenswrapper[5052]: I1203 12:34:32.278176 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/6e18c0ba-8f07-4d54-b383-3d0164780725-run-httpd\") pod \"ceilometer-0\" (UID: \"6e18c0ba-8f07-4d54-b383-3d0164780725\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:34:32 crc kubenswrapper[5052]: I1203 12:34:32.278239 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fxdfv\" (UniqueName: \"kubernetes.io/projected/6e18c0ba-8f07-4d54-b383-3d0164780725-kube-api-access-fxdfv\") pod \"ceilometer-0\" (UID: \"6e18c0ba-8f07-4d54-b383-3d0164780725\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:34:32 crc kubenswrapper[5052]: I1203 12:34:32.278361 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/6e18c0ba-8f07-4d54-b383-3d0164780725-log-httpd\") pod \"ceilometer-0\" (UID: \"6e18c0ba-8f07-4d54-b383-3d0164780725\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:34:32 crc kubenswrapper[5052]: I1203 12:34:32.278393 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6e18c0ba-8f07-4d54-b383-3d0164780725-config-data\") pod \"ceilometer-0\" (UID: \"6e18c0ba-8f07-4d54-b383-3d0164780725\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:34:32 crc kubenswrapper[5052]: I1203 12:34:32.289261 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:34:32 crc kubenswrapper[5052]: I1203 12:34:32.362975 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/cinder-db-sync-s42b9"] Dec 03 12:34:32 crc kubenswrapper[5052]: I1203 12:34:32.364459 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-db-sync-s42b9" Dec 03 12:34:32 crc kubenswrapper[5052]: I1203 12:34:32.367148 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cinder-scripts" Dec 03 12:34:32 crc kubenswrapper[5052]: I1203 12:34:32.367256 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cinder-cinder-dockercfg-8v2d7" Dec 03 12:34:32 crc kubenswrapper[5052]: I1203 12:34:32.376100 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cinder-config-data" Dec 03 12:34:32 crc kubenswrapper[5052]: I1203 12:34:32.379252 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/55ccf323-53e4-47fe-bea8-a17273b8587c-db-sync-config-data\") pod \"cinder-db-sync-s42b9\" (UID: \"55ccf323-53e4-47fe-bea8-a17273b8587c\") " pod="openstack-kuttl-tests/cinder-db-sync-s42b9" Dec 03 12:34:32 crc kubenswrapper[5052]: I1203 12:34:32.379327 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6e18c0ba-8f07-4d54-b383-3d0164780725-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"6e18c0ba-8f07-4d54-b383-3d0164780725\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:34:32 crc kubenswrapper[5052]: I1203 12:34:32.379468 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/55ccf323-53e4-47fe-bea8-a17273b8587c-etc-machine-id\") pod \"cinder-db-sync-s42b9\" (UID: \"55ccf323-53e4-47fe-bea8-a17273b8587c\") " pod="openstack-kuttl-tests/cinder-db-sync-s42b9" Dec 03 12:34:32 crc kubenswrapper[5052]: I1203 12:34:32.379595 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/6e18c0ba-8f07-4d54-b383-3d0164780725-scripts\") pod \"ceilometer-0\" (UID: \"6e18c0ba-8f07-4d54-b383-3d0164780725\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:34:32 crc kubenswrapper[5052]: I1203 12:34:32.379663 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/6e18c0ba-8f07-4d54-b383-3d0164780725-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"6e18c0ba-8f07-4d54-b383-3d0164780725\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:34:32 crc kubenswrapper[5052]: I1203 12:34:32.379761 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/6e18c0ba-8f07-4d54-b383-3d0164780725-run-httpd\") pod \"ceilometer-0\" (UID: \"6e18c0ba-8f07-4d54-b383-3d0164780725\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:34:32 crc kubenswrapper[5052]: I1203 12:34:32.379801 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/55ccf323-53e4-47fe-bea8-a17273b8587c-scripts\") pod \"cinder-db-sync-s42b9\" (UID: \"55ccf323-53e4-47fe-bea8-a17273b8587c\") " pod="openstack-kuttl-tests/cinder-db-sync-s42b9" Dec 03 12:34:32 crc kubenswrapper[5052]: I1203 12:34:32.379831 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-fxdfv\" (UniqueName: \"kubernetes.io/projected/6e18c0ba-8f07-4d54-b383-3d0164780725-kube-api-access-fxdfv\") pod \"ceilometer-0\" (UID: \"6e18c0ba-8f07-4d54-b383-3d0164780725\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:34:32 crc kubenswrapper[5052]: I1203 12:34:32.379867 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nwfzf\" (UniqueName: \"kubernetes.io/projected/55ccf323-53e4-47fe-bea8-a17273b8587c-kube-api-access-nwfzf\") pod \"cinder-db-sync-s42b9\" (UID: \"55ccf323-53e4-47fe-bea8-a17273b8587c\") " pod="openstack-kuttl-tests/cinder-db-sync-s42b9" Dec 03 12:34:32 crc kubenswrapper[5052]: I1203 12:34:32.379935 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/6e18c0ba-8f07-4d54-b383-3d0164780725-log-httpd\") pod \"ceilometer-0\" (UID: \"6e18c0ba-8f07-4d54-b383-3d0164780725\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:34:32 crc kubenswrapper[5052]: I1203 12:34:32.379959 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6e18c0ba-8f07-4d54-b383-3d0164780725-config-data\") pod \"ceilometer-0\" (UID: \"6e18c0ba-8f07-4d54-b383-3d0164780725\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:34:32 crc kubenswrapper[5052]: I1203 12:34:32.380006 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/55ccf323-53e4-47fe-bea8-a17273b8587c-config-data\") pod \"cinder-db-sync-s42b9\" (UID: \"55ccf323-53e4-47fe-bea8-a17273b8587c\") " pod="openstack-kuttl-tests/cinder-db-sync-s42b9" Dec 03 12:34:32 crc kubenswrapper[5052]: I1203 12:34:32.380085 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/55ccf323-53e4-47fe-bea8-a17273b8587c-combined-ca-bundle\") pod \"cinder-db-sync-s42b9\" (UID: \"55ccf323-53e4-47fe-bea8-a17273b8587c\") " pod="openstack-kuttl-tests/cinder-db-sync-s42b9" Dec 03 12:34:32 crc kubenswrapper[5052]: I1203 12:34:32.380225 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/6e18c0ba-8f07-4d54-b383-3d0164780725-run-httpd\") pod \"ceilometer-0\" (UID: \"6e18c0ba-8f07-4d54-b383-3d0164780725\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:34:32 crc kubenswrapper[5052]: I1203 12:34:32.380661 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/6e18c0ba-8f07-4d54-b383-3d0164780725-log-httpd\") pod \"ceilometer-0\" (UID: \"6e18c0ba-8f07-4d54-b383-3d0164780725\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:34:32 crc kubenswrapper[5052]: I1203 12:34:32.382768 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-bootstrap-hvfxb" Dec 03 12:34:32 crc kubenswrapper[5052]: I1203 12:34:32.384386 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6e18c0ba-8f07-4d54-b383-3d0164780725-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"6e18c0ba-8f07-4d54-b383-3d0164780725\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:34:32 crc kubenswrapper[5052]: I1203 12:34:32.387029 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/6e18c0ba-8f07-4d54-b383-3d0164780725-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"6e18c0ba-8f07-4d54-b383-3d0164780725\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:34:32 crc kubenswrapper[5052]: I1203 12:34:32.390831 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6e18c0ba-8f07-4d54-b383-3d0164780725-config-data\") pod \"ceilometer-0\" (UID: \"6e18c0ba-8f07-4d54-b383-3d0164780725\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:34:32 crc kubenswrapper[5052]: I1203 12:34:32.394000 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/6e18c0ba-8f07-4d54-b383-3d0164780725-scripts\") pod \"ceilometer-0\" (UID: \"6e18c0ba-8f07-4d54-b383-3d0164780725\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:34:32 crc kubenswrapper[5052]: I1203 12:34:32.396759 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/neutron-db-sync-5kww4"] Dec 03 12:34:32 crc kubenswrapper[5052]: I1203 12:34:32.397986 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-db-sync-5kww4" Dec 03 12:34:32 crc kubenswrapper[5052]: I1203 12:34:32.414992 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"neutron-httpd-config" Dec 03 12:34:32 crc kubenswrapper[5052]: I1203 12:34:32.415744 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"neutron-neutron-dockercfg-7kmvw" Dec 03 12:34:32 crc kubenswrapper[5052]: I1203 12:34:32.417340 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"neutron-config" Dec 03 12:34:32 crc kubenswrapper[5052]: I1203 12:34:32.422153 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-fxdfv\" (UniqueName: \"kubernetes.io/projected/6e18c0ba-8f07-4d54-b383-3d0164780725-kube-api-access-fxdfv\") pod \"ceilometer-0\" (UID: \"6e18c0ba-8f07-4d54-b383-3d0164780725\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:34:32 crc kubenswrapper[5052]: I1203 12:34:32.428470 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-db-sync-s42b9"] Dec 03 12:34:32 crc kubenswrapper[5052]: I1203 12:34:32.451673 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-db-sync-5kww4"] Dec 03 12:34:32 crc kubenswrapper[5052]: I1203 12:34:32.472798 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/barbican-db-sync-nj49v"] Dec 03 12:34:32 crc kubenswrapper[5052]: I1203 12:34:32.474346 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-db-sync-nj49v" Dec 03 12:34:32 crc kubenswrapper[5052]: I1203 12:34:32.479579 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"barbican-barbican-dockercfg-n9q58" Dec 03 12:34:32 crc kubenswrapper[5052]: I1203 12:34:32.480116 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"barbican-config-data" Dec 03 12:34:32 crc kubenswrapper[5052]: I1203 12:34:32.484395 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/55ccf323-53e4-47fe-bea8-a17273b8587c-scripts\") pod \"cinder-db-sync-s42b9\" (UID: \"55ccf323-53e4-47fe-bea8-a17273b8587c\") " pod="openstack-kuttl-tests/cinder-db-sync-s42b9" Dec 03 12:34:32 crc kubenswrapper[5052]: I1203 12:34:32.484456 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-nwfzf\" (UniqueName: \"kubernetes.io/projected/55ccf323-53e4-47fe-bea8-a17273b8587c-kube-api-access-nwfzf\") pod \"cinder-db-sync-s42b9\" (UID: \"55ccf323-53e4-47fe-bea8-a17273b8587c\") " pod="openstack-kuttl-tests/cinder-db-sync-s42b9" Dec 03 12:34:32 crc kubenswrapper[5052]: I1203 12:34:32.484500 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/55ccf323-53e4-47fe-bea8-a17273b8587c-config-data\") pod \"cinder-db-sync-s42b9\" (UID: \"55ccf323-53e4-47fe-bea8-a17273b8587c\") " pod="openstack-kuttl-tests/cinder-db-sync-s42b9" Dec 03 12:34:32 crc kubenswrapper[5052]: I1203 12:34:32.484553 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/55ccf323-53e4-47fe-bea8-a17273b8587c-combined-ca-bundle\") pod \"cinder-db-sync-s42b9\" (UID: \"55ccf323-53e4-47fe-bea8-a17273b8587c\") " pod="openstack-kuttl-tests/cinder-db-sync-s42b9" Dec 03 12:34:32 crc kubenswrapper[5052]: I1203 12:34:32.484606 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/55ccf323-53e4-47fe-bea8-a17273b8587c-db-sync-config-data\") pod \"cinder-db-sync-s42b9\" (UID: \"55ccf323-53e4-47fe-bea8-a17273b8587c\") " pod="openstack-kuttl-tests/cinder-db-sync-s42b9" Dec 03 12:34:32 crc kubenswrapper[5052]: I1203 12:34:32.484665 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/55ccf323-53e4-47fe-bea8-a17273b8587c-etc-machine-id\") pod \"cinder-db-sync-s42b9\" (UID: \"55ccf323-53e4-47fe-bea8-a17273b8587c\") " pod="openstack-kuttl-tests/cinder-db-sync-s42b9" Dec 03 12:34:32 crc kubenswrapper[5052]: I1203 12:34:32.484776 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/55ccf323-53e4-47fe-bea8-a17273b8587c-etc-machine-id\") pod \"cinder-db-sync-s42b9\" (UID: \"55ccf323-53e4-47fe-bea8-a17273b8587c\") " pod="openstack-kuttl-tests/cinder-db-sync-s42b9" Dec 03 12:34:32 crc kubenswrapper[5052]: I1203 12:34:32.503762 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/placement-db-sync-fcpth"] Dec 03 12:34:32 crc kubenswrapper[5052]: I1203 12:34:32.505031 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-db-sync-fcpth" Dec 03 12:34:32 crc kubenswrapper[5052]: I1203 12:34:32.506380 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/55ccf323-53e4-47fe-bea8-a17273b8587c-config-data\") pod \"cinder-db-sync-s42b9\" (UID: \"55ccf323-53e4-47fe-bea8-a17273b8587c\") " pod="openstack-kuttl-tests/cinder-db-sync-s42b9" Dec 03 12:34:32 crc kubenswrapper[5052]: I1203 12:34:32.506626 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"placement-config-data" Dec 03 12:34:32 crc kubenswrapper[5052]: I1203 12:34:32.507238 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/55ccf323-53e4-47fe-bea8-a17273b8587c-combined-ca-bundle\") pod \"cinder-db-sync-s42b9\" (UID: \"55ccf323-53e4-47fe-bea8-a17273b8587c\") " pod="openstack-kuttl-tests/cinder-db-sync-s42b9" Dec 03 12:34:32 crc kubenswrapper[5052]: I1203 12:34:32.507919 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/55ccf323-53e4-47fe-bea8-a17273b8587c-db-sync-config-data\") pod \"cinder-db-sync-s42b9\" (UID: \"55ccf323-53e4-47fe-bea8-a17273b8587c\") " pod="openstack-kuttl-tests/cinder-db-sync-s42b9" Dec 03 12:34:32 crc kubenswrapper[5052]: I1203 12:34:32.506660 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"placement-scripts" Dec 03 12:34:32 crc kubenswrapper[5052]: I1203 12:34:32.506778 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"placement-placement-dockercfg-642c9" Dec 03 12:34:32 crc kubenswrapper[5052]: I1203 12:34:32.521163 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-nwfzf\" (UniqueName: \"kubernetes.io/projected/55ccf323-53e4-47fe-bea8-a17273b8587c-kube-api-access-nwfzf\") pod \"cinder-db-sync-s42b9\" (UID: \"55ccf323-53e4-47fe-bea8-a17273b8587c\") " pod="openstack-kuttl-tests/cinder-db-sync-s42b9" Dec 03 12:34:32 crc kubenswrapper[5052]: I1203 12:34:32.521708 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/55ccf323-53e4-47fe-bea8-a17273b8587c-scripts\") pod \"cinder-db-sync-s42b9\" (UID: \"55ccf323-53e4-47fe-bea8-a17273b8587c\") " pod="openstack-kuttl-tests/cinder-db-sync-s42b9" Dec 03 12:34:32 crc kubenswrapper[5052]: I1203 12:34:32.533026 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/placement-db-sync-fcpth"] Dec 03 12:34:32 crc kubenswrapper[5052]: I1203 12:34:32.555476 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:34:32 crc kubenswrapper[5052]: I1203 12:34:32.551467 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-db-sync-nj49v"] Dec 03 12:34:32 crc kubenswrapper[5052]: I1203 12:34:32.578338 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-db-sync-s42b9" Dec 03 12:34:32 crc kubenswrapper[5052]: I1203 12:34:32.587010 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mns4w\" (UniqueName: \"kubernetes.io/projected/d9523127-c2ea-4f5f-98ae-0288681fb93c-kube-api-access-mns4w\") pod \"barbican-db-sync-nj49v\" (UID: \"d9523127-c2ea-4f5f-98ae-0288681fb93c\") " pod="openstack-kuttl-tests/barbican-db-sync-nj49v" Dec 03 12:34:32 crc kubenswrapper[5052]: I1203 12:34:32.587185 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/348487de-7480-462b-9f22-56a555321be8-config\") pod \"neutron-db-sync-5kww4\" (UID: \"348487de-7480-462b-9f22-56a555321be8\") " pod="openstack-kuttl-tests/neutron-db-sync-5kww4" Dec 03 12:34:32 crc kubenswrapper[5052]: I1203 12:34:32.587287 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/348487de-7480-462b-9f22-56a555321be8-combined-ca-bundle\") pod \"neutron-db-sync-5kww4\" (UID: \"348487de-7480-462b-9f22-56a555321be8\") " pod="openstack-kuttl-tests/neutron-db-sync-5kww4" Dec 03 12:34:32 crc kubenswrapper[5052]: I1203 12:34:32.587395 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/d9523127-c2ea-4f5f-98ae-0288681fb93c-db-sync-config-data\") pod \"barbican-db-sync-nj49v\" (UID: \"d9523127-c2ea-4f5f-98ae-0288681fb93c\") " pod="openstack-kuttl-tests/barbican-db-sync-nj49v" Dec 03 12:34:32 crc kubenswrapper[5052]: I1203 12:34:32.587483 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-spzbr\" (UniqueName: \"kubernetes.io/projected/348487de-7480-462b-9f22-56a555321be8-kube-api-access-spzbr\") pod \"neutron-db-sync-5kww4\" (UID: \"348487de-7480-462b-9f22-56a555321be8\") " pod="openstack-kuttl-tests/neutron-db-sync-5kww4" Dec 03 12:34:32 crc kubenswrapper[5052]: I1203 12:34:32.587652 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d9523127-c2ea-4f5f-98ae-0288681fb93c-combined-ca-bundle\") pod \"barbican-db-sync-nj49v\" (UID: \"d9523127-c2ea-4f5f-98ae-0288681fb93c\") " pod="openstack-kuttl-tests/barbican-db-sync-nj49v" Dec 03 12:34:32 crc kubenswrapper[5052]: I1203 12:34:32.689916 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-mns4w\" (UniqueName: \"kubernetes.io/projected/d9523127-c2ea-4f5f-98ae-0288681fb93c-kube-api-access-mns4w\") pod \"barbican-db-sync-nj49v\" (UID: \"d9523127-c2ea-4f5f-98ae-0288681fb93c\") " pod="openstack-kuttl-tests/barbican-db-sync-nj49v" Dec 03 12:34:32 crc kubenswrapper[5052]: I1203 12:34:32.690384 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/a4de7484-329f-412e-b365-e373a4cab966-logs\") pod \"placement-db-sync-fcpth\" (UID: \"a4de7484-329f-412e-b365-e373a4cab966\") " pod="openstack-kuttl-tests/placement-db-sync-fcpth" Dec 03 12:34:32 crc kubenswrapper[5052]: I1203 12:34:32.690409 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/348487de-7480-462b-9f22-56a555321be8-config\") pod \"neutron-db-sync-5kww4\" (UID: \"348487de-7480-462b-9f22-56a555321be8\") " pod="openstack-kuttl-tests/neutron-db-sync-5kww4" Dec 03 12:34:32 crc kubenswrapper[5052]: I1203 12:34:32.690445 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fhbjq\" (UniqueName: \"kubernetes.io/projected/a4de7484-329f-412e-b365-e373a4cab966-kube-api-access-fhbjq\") pod \"placement-db-sync-fcpth\" (UID: \"a4de7484-329f-412e-b365-e373a4cab966\") " pod="openstack-kuttl-tests/placement-db-sync-fcpth" Dec 03 12:34:32 crc kubenswrapper[5052]: I1203 12:34:32.690465 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/348487de-7480-462b-9f22-56a555321be8-combined-ca-bundle\") pod \"neutron-db-sync-5kww4\" (UID: \"348487de-7480-462b-9f22-56a555321be8\") " pod="openstack-kuttl-tests/neutron-db-sync-5kww4" Dec 03 12:34:32 crc kubenswrapper[5052]: I1203 12:34:32.690586 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/d9523127-c2ea-4f5f-98ae-0288681fb93c-db-sync-config-data\") pod \"barbican-db-sync-nj49v\" (UID: \"d9523127-c2ea-4f5f-98ae-0288681fb93c\") " pod="openstack-kuttl-tests/barbican-db-sync-nj49v" Dec 03 12:34:32 crc kubenswrapper[5052]: I1203 12:34:32.690617 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/a4de7484-329f-412e-b365-e373a4cab966-scripts\") pod \"placement-db-sync-fcpth\" (UID: \"a4de7484-329f-412e-b365-e373a4cab966\") " pod="openstack-kuttl-tests/placement-db-sync-fcpth" Dec 03 12:34:32 crc kubenswrapper[5052]: I1203 12:34:32.690644 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-spzbr\" (UniqueName: \"kubernetes.io/projected/348487de-7480-462b-9f22-56a555321be8-kube-api-access-spzbr\") pod \"neutron-db-sync-5kww4\" (UID: \"348487de-7480-462b-9f22-56a555321be8\") " pod="openstack-kuttl-tests/neutron-db-sync-5kww4" Dec 03 12:34:32 crc kubenswrapper[5052]: I1203 12:34:32.690670 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a4de7484-329f-412e-b365-e373a4cab966-config-data\") pod \"placement-db-sync-fcpth\" (UID: \"a4de7484-329f-412e-b365-e373a4cab966\") " pod="openstack-kuttl-tests/placement-db-sync-fcpth" Dec 03 12:34:32 crc kubenswrapper[5052]: I1203 12:34:32.690686 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a4de7484-329f-412e-b365-e373a4cab966-combined-ca-bundle\") pod \"placement-db-sync-fcpth\" (UID: \"a4de7484-329f-412e-b365-e373a4cab966\") " pod="openstack-kuttl-tests/placement-db-sync-fcpth" Dec 03 12:34:32 crc kubenswrapper[5052]: I1203 12:34:32.690708 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d9523127-c2ea-4f5f-98ae-0288681fb93c-combined-ca-bundle\") pod \"barbican-db-sync-nj49v\" (UID: \"d9523127-c2ea-4f5f-98ae-0288681fb93c\") " pod="openstack-kuttl-tests/barbican-db-sync-nj49v" Dec 03 12:34:32 crc kubenswrapper[5052]: I1203 12:34:32.701115 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/d9523127-c2ea-4f5f-98ae-0288681fb93c-db-sync-config-data\") pod \"barbican-db-sync-nj49v\" (UID: \"d9523127-c2ea-4f5f-98ae-0288681fb93c\") " pod="openstack-kuttl-tests/barbican-db-sync-nj49v" Dec 03 12:34:32 crc kubenswrapper[5052]: I1203 12:34:32.702925 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d9523127-c2ea-4f5f-98ae-0288681fb93c-combined-ca-bundle\") pod \"barbican-db-sync-nj49v\" (UID: \"d9523127-c2ea-4f5f-98ae-0288681fb93c\") " pod="openstack-kuttl-tests/barbican-db-sync-nj49v" Dec 03 12:34:32 crc kubenswrapper[5052]: I1203 12:34:32.705145 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/348487de-7480-462b-9f22-56a555321be8-combined-ca-bundle\") pod \"neutron-db-sync-5kww4\" (UID: \"348487de-7480-462b-9f22-56a555321be8\") " pod="openstack-kuttl-tests/neutron-db-sync-5kww4" Dec 03 12:34:32 crc kubenswrapper[5052]: I1203 12:34:32.706733 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/348487de-7480-462b-9f22-56a555321be8-config\") pod \"neutron-db-sync-5kww4\" (UID: \"348487de-7480-462b-9f22-56a555321be8\") " pod="openstack-kuttl-tests/neutron-db-sync-5kww4" Dec 03 12:34:32 crc kubenswrapper[5052]: I1203 12:34:32.709868 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-mns4w\" (UniqueName: \"kubernetes.io/projected/d9523127-c2ea-4f5f-98ae-0288681fb93c-kube-api-access-mns4w\") pod \"barbican-db-sync-nj49v\" (UID: \"d9523127-c2ea-4f5f-98ae-0288681fb93c\") " pod="openstack-kuttl-tests/barbican-db-sync-nj49v" Dec 03 12:34:32 crc kubenswrapper[5052]: I1203 12:34:32.714720 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-spzbr\" (UniqueName: \"kubernetes.io/projected/348487de-7480-462b-9f22-56a555321be8-kube-api-access-spzbr\") pod \"neutron-db-sync-5kww4\" (UID: \"348487de-7480-462b-9f22-56a555321be8\") " pod="openstack-kuttl-tests/neutron-db-sync-5kww4" Dec 03 12:34:32 crc kubenswrapper[5052]: I1203 12:34:32.793055 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/a4de7484-329f-412e-b365-e373a4cab966-scripts\") pod \"placement-db-sync-fcpth\" (UID: \"a4de7484-329f-412e-b365-e373a4cab966\") " pod="openstack-kuttl-tests/placement-db-sync-fcpth" Dec 03 12:34:32 crc kubenswrapper[5052]: I1203 12:34:32.793193 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a4de7484-329f-412e-b365-e373a4cab966-config-data\") pod \"placement-db-sync-fcpth\" (UID: \"a4de7484-329f-412e-b365-e373a4cab966\") " pod="openstack-kuttl-tests/placement-db-sync-fcpth" Dec 03 12:34:32 crc kubenswrapper[5052]: I1203 12:34:32.794011 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a4de7484-329f-412e-b365-e373a4cab966-combined-ca-bundle\") pod \"placement-db-sync-fcpth\" (UID: \"a4de7484-329f-412e-b365-e373a4cab966\") " pod="openstack-kuttl-tests/placement-db-sync-fcpth" Dec 03 12:34:32 crc kubenswrapper[5052]: I1203 12:34:32.794805 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/a4de7484-329f-412e-b365-e373a4cab966-logs\") pod \"placement-db-sync-fcpth\" (UID: \"a4de7484-329f-412e-b365-e373a4cab966\") " pod="openstack-kuttl-tests/placement-db-sync-fcpth" Dec 03 12:34:32 crc kubenswrapper[5052]: I1203 12:34:32.794886 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-fhbjq\" (UniqueName: \"kubernetes.io/projected/a4de7484-329f-412e-b365-e373a4cab966-kube-api-access-fhbjq\") pod \"placement-db-sync-fcpth\" (UID: \"a4de7484-329f-412e-b365-e373a4cab966\") " pod="openstack-kuttl-tests/placement-db-sync-fcpth" Dec 03 12:34:32 crc kubenswrapper[5052]: I1203 12:34:32.795918 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/a4de7484-329f-412e-b365-e373a4cab966-logs\") pod \"placement-db-sync-fcpth\" (UID: \"a4de7484-329f-412e-b365-e373a4cab966\") " pod="openstack-kuttl-tests/placement-db-sync-fcpth" Dec 03 12:34:32 crc kubenswrapper[5052]: I1203 12:34:32.796618 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/a4de7484-329f-412e-b365-e373a4cab966-scripts\") pod \"placement-db-sync-fcpth\" (UID: \"a4de7484-329f-412e-b365-e373a4cab966\") " pod="openstack-kuttl-tests/placement-db-sync-fcpth" Dec 03 12:34:32 crc kubenswrapper[5052]: I1203 12:34:32.798692 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a4de7484-329f-412e-b365-e373a4cab966-config-data\") pod \"placement-db-sync-fcpth\" (UID: \"a4de7484-329f-412e-b365-e373a4cab966\") " pod="openstack-kuttl-tests/placement-db-sync-fcpth" Dec 03 12:34:32 crc kubenswrapper[5052]: I1203 12:34:32.799392 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a4de7484-329f-412e-b365-e373a4cab966-combined-ca-bundle\") pod \"placement-db-sync-fcpth\" (UID: \"a4de7484-329f-412e-b365-e373a4cab966\") " pod="openstack-kuttl-tests/placement-db-sync-fcpth" Dec 03 12:34:32 crc kubenswrapper[5052]: I1203 12:34:32.814746 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-fhbjq\" (UniqueName: \"kubernetes.io/projected/a4de7484-329f-412e-b365-e373a4cab966-kube-api-access-fhbjq\") pod \"placement-db-sync-fcpth\" (UID: \"a4de7484-329f-412e-b365-e373a4cab966\") " pod="openstack-kuttl-tests/placement-db-sync-fcpth" Dec 03 12:34:32 crc kubenswrapper[5052]: I1203 12:34:32.905428 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-db-sync-5kww4" Dec 03 12:34:32 crc kubenswrapper[5052]: I1203 12:34:32.921130 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-db-sync-nj49v" Dec 03 12:34:32 crc kubenswrapper[5052]: I1203 12:34:32.938931 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-db-sync-fcpth" Dec 03 12:34:33 crc kubenswrapper[5052]: I1203 12:34:33.006940 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-bootstrap-hvfxb"] Dec 03 12:34:33 crc kubenswrapper[5052]: W1203 12:34:33.008856 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod80bd8d5d_a98f_455b_977a_e96317d7f525.slice/crio-18897b1cf1b0797253e333d8b9d11c44cfe74e0ff25736ac787bfde87f18d1c8 WatchSource:0}: Error finding container 18897b1cf1b0797253e333d8b9d11c44cfe74e0ff25736ac787bfde87f18d1c8: Status 404 returned error can't find the container with id 18897b1cf1b0797253e333d8b9d11c44cfe74e0ff25736ac787bfde87f18d1c8 Dec 03 12:34:33 crc kubenswrapper[5052]: I1203 12:34:33.017844 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-db-sync-s42b9"] Dec 03 12:34:33 crc kubenswrapper[5052]: W1203 12:34:33.035572 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod55ccf323_53e4_47fe_bea8_a17273b8587c.slice/crio-1a703be9bb20d985462a2e0555794ece28a51df5b49803a0b4a8f1399091a5d2 WatchSource:0}: Error finding container 1a703be9bb20d985462a2e0555794ece28a51df5b49803a0b4a8f1399091a5d2: Status 404 returned error can't find the container with id 1a703be9bb20d985462a2e0555794ece28a51df5b49803a0b4a8f1399091a5d2 Dec 03 12:34:33 crc kubenswrapper[5052]: I1203 12:34:33.096952 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:34:33 crc kubenswrapper[5052]: W1203 12:34:33.115031 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6e18c0ba_8f07_4d54_b383_3d0164780725.slice/crio-dca0ec34dd849975bff714d71f1f4011e087a03ca11b0afe20e086380117598c WatchSource:0}: Error finding container dca0ec34dd849975bff714d71f1f4011e087a03ca11b0afe20e086380117598c: Status 404 returned error can't find the container with id dca0ec34dd849975bff714d71f1f4011e087a03ca11b0afe20e086380117598c Dec 03 12:34:33 crc kubenswrapper[5052]: I1203 12:34:33.183036 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:34:33 crc kubenswrapper[5052]: I1203 12:34:33.186095 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:34:33 crc kubenswrapper[5052]: I1203 12:34:33.189689 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-scripts" Dec 03 12:34:33 crc kubenswrapper[5052]: I1203 12:34:33.189962 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-glance-dockercfg-xkmzv" Dec 03 12:34:33 crc kubenswrapper[5052]: I1203 12:34:33.190139 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-default-internal-config-data" Dec 03 12:34:33 crc kubenswrapper[5052]: I1203 12:34:33.191627 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-glance-default-internal-svc" Dec 03 12:34:33 crc kubenswrapper[5052]: I1203 12:34:33.208229 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:34:33 crc kubenswrapper[5052]: I1203 12:34:33.272416 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:34:33 crc kubenswrapper[5052]: I1203 12:34:33.275686 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:34:33 crc kubenswrapper[5052]: I1203 12:34:33.281333 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-glance-default-public-svc" Dec 03 12:34:33 crc kubenswrapper[5052]: I1203 12:34:33.282538 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-default-external-config-data" Dec 03 12:34:33 crc kubenswrapper[5052]: I1203 12:34:33.290289 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:34:33 crc kubenswrapper[5052]: I1203 12:34:33.314390 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage18-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage18-crc\") pod \"glance-default-internal-api-0\" (UID: \"fa6dddd6-6ac5-4baa-9437-9a8ea5f46031\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:34:33 crc kubenswrapper[5052]: I1203 12:34:33.314439 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/fa6dddd6-6ac5-4baa-9437-9a8ea5f46031-combined-ca-bundle\") pod \"glance-default-internal-api-0\" (UID: \"fa6dddd6-6ac5-4baa-9437-9a8ea5f46031\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:34:33 crc kubenswrapper[5052]: I1203 12:34:33.314466 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-44pcb\" (UniqueName: \"kubernetes.io/projected/fa6dddd6-6ac5-4baa-9437-9a8ea5f46031-kube-api-access-44pcb\") pod \"glance-default-internal-api-0\" (UID: \"fa6dddd6-6ac5-4baa-9437-9a8ea5f46031\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:34:33 crc kubenswrapper[5052]: I1203 12:34:33.314574 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/fa6dddd6-6ac5-4baa-9437-9a8ea5f46031-logs\") pod \"glance-default-internal-api-0\" (UID: \"fa6dddd6-6ac5-4baa-9437-9a8ea5f46031\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:34:33 crc kubenswrapper[5052]: I1203 12:34:33.314618 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/fa6dddd6-6ac5-4baa-9437-9a8ea5f46031-internal-tls-certs\") pod \"glance-default-internal-api-0\" (UID: \"fa6dddd6-6ac5-4baa-9437-9a8ea5f46031\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:34:33 crc kubenswrapper[5052]: I1203 12:34:33.314633 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/fa6dddd6-6ac5-4baa-9437-9a8ea5f46031-httpd-run\") pod \"glance-default-internal-api-0\" (UID: \"fa6dddd6-6ac5-4baa-9437-9a8ea5f46031\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:34:33 crc kubenswrapper[5052]: I1203 12:34:33.314675 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/fa6dddd6-6ac5-4baa-9437-9a8ea5f46031-scripts\") pod \"glance-default-internal-api-0\" (UID: \"fa6dddd6-6ac5-4baa-9437-9a8ea5f46031\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:34:33 crc kubenswrapper[5052]: I1203 12:34:33.314708 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/fa6dddd6-6ac5-4baa-9437-9a8ea5f46031-config-data\") pod \"glance-default-internal-api-0\" (UID: \"fa6dddd6-6ac5-4baa-9437-9a8ea5f46031\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:34:33 crc kubenswrapper[5052]: I1203 12:34:33.416201 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/4471a563-0245-40e9-bca8-acc180198abe-public-tls-certs\") pod \"glance-default-external-api-0\" (UID: \"4471a563-0245-40e9-bca8-acc180198abe\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:34:33 crc kubenswrapper[5052]: I1203 12:34:33.416258 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/fa6dddd6-6ac5-4baa-9437-9a8ea5f46031-internal-tls-certs\") pod \"glance-default-internal-api-0\" (UID: \"fa6dddd6-6ac5-4baa-9437-9a8ea5f46031\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:34:33 crc kubenswrapper[5052]: I1203 12:34:33.416287 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/fa6dddd6-6ac5-4baa-9437-9a8ea5f46031-httpd-run\") pod \"glance-default-internal-api-0\" (UID: \"fa6dddd6-6ac5-4baa-9437-9a8ea5f46031\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:34:33 crc kubenswrapper[5052]: I1203 12:34:33.416319 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j7s7w\" (UniqueName: \"kubernetes.io/projected/4471a563-0245-40e9-bca8-acc180198abe-kube-api-access-j7s7w\") pod \"glance-default-external-api-0\" (UID: \"4471a563-0245-40e9-bca8-acc180198abe\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:34:33 crc kubenswrapper[5052]: I1203 12:34:33.416347 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4471a563-0245-40e9-bca8-acc180198abe-combined-ca-bundle\") pod \"glance-default-external-api-0\" (UID: \"4471a563-0245-40e9-bca8-acc180198abe\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:34:33 crc kubenswrapper[5052]: I1203 12:34:33.416400 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/fa6dddd6-6ac5-4baa-9437-9a8ea5f46031-scripts\") pod \"glance-default-internal-api-0\" (UID: \"fa6dddd6-6ac5-4baa-9437-9a8ea5f46031\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:34:33 crc kubenswrapper[5052]: I1203 12:34:33.416440 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/fa6dddd6-6ac5-4baa-9437-9a8ea5f46031-config-data\") pod \"glance-default-internal-api-0\" (UID: \"fa6dddd6-6ac5-4baa-9437-9a8ea5f46031\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:34:33 crc kubenswrapper[5052]: I1203 12:34:33.416467 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage18-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage18-crc\") pod \"glance-default-internal-api-0\" (UID: \"fa6dddd6-6ac5-4baa-9437-9a8ea5f46031\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:34:33 crc kubenswrapper[5052]: I1203 12:34:33.416499 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/fa6dddd6-6ac5-4baa-9437-9a8ea5f46031-combined-ca-bundle\") pod \"glance-default-internal-api-0\" (UID: \"fa6dddd6-6ac5-4baa-9437-9a8ea5f46031\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:34:33 crc kubenswrapper[5052]: I1203 12:34:33.416540 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/4471a563-0245-40e9-bca8-acc180198abe-httpd-run\") pod \"glance-default-external-api-0\" (UID: \"4471a563-0245-40e9-bca8-acc180198abe\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:34:33 crc kubenswrapper[5052]: I1203 12:34:33.416567 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-44pcb\" (UniqueName: \"kubernetes.io/projected/fa6dddd6-6ac5-4baa-9437-9a8ea5f46031-kube-api-access-44pcb\") pod \"glance-default-internal-api-0\" (UID: \"fa6dddd6-6ac5-4baa-9437-9a8ea5f46031\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:34:33 crc kubenswrapper[5052]: I1203 12:34:33.416605 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/4471a563-0245-40e9-bca8-acc180198abe-config-data\") pod \"glance-default-external-api-0\" (UID: \"4471a563-0245-40e9-bca8-acc180198abe\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:34:33 crc kubenswrapper[5052]: I1203 12:34:33.416639 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage12-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage12-crc\") pod \"glance-default-external-api-0\" (UID: \"4471a563-0245-40e9-bca8-acc180198abe\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:34:33 crc kubenswrapper[5052]: I1203 12:34:33.416670 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/4471a563-0245-40e9-bca8-acc180198abe-logs\") pod \"glance-default-external-api-0\" (UID: \"4471a563-0245-40e9-bca8-acc180198abe\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:34:33 crc kubenswrapper[5052]: I1203 12:34:33.416723 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/fa6dddd6-6ac5-4baa-9437-9a8ea5f46031-logs\") pod \"glance-default-internal-api-0\" (UID: \"fa6dddd6-6ac5-4baa-9437-9a8ea5f46031\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:34:33 crc kubenswrapper[5052]: I1203 12:34:33.416748 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/4471a563-0245-40e9-bca8-acc180198abe-scripts\") pod \"glance-default-external-api-0\" (UID: \"4471a563-0245-40e9-bca8-acc180198abe\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:34:33 crc kubenswrapper[5052]: I1203 12:34:33.416770 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/fa6dddd6-6ac5-4baa-9437-9a8ea5f46031-httpd-run\") pod \"glance-default-internal-api-0\" (UID: \"fa6dddd6-6ac5-4baa-9437-9a8ea5f46031\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:34:33 crc kubenswrapper[5052]: I1203 12:34:33.417741 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/fa6dddd6-6ac5-4baa-9437-9a8ea5f46031-logs\") pod \"glance-default-internal-api-0\" (UID: \"fa6dddd6-6ac5-4baa-9437-9a8ea5f46031\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:34:33 crc kubenswrapper[5052]: I1203 12:34:33.417979 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage18-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage18-crc\") pod \"glance-default-internal-api-0\" (UID: \"fa6dddd6-6ac5-4baa-9437-9a8ea5f46031\") device mount path \"/mnt/openstack/pv18\"" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:34:33 crc kubenswrapper[5052]: I1203 12:34:33.505695 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-db-sync-5kww4"] Dec 03 12:34:33 crc kubenswrapper[5052]: I1203 12:34:33.518695 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/placement-db-sync-fcpth"] Dec 03 12:34:33 crc kubenswrapper[5052]: I1203 12:34:33.519351 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/4471a563-0245-40e9-bca8-acc180198abe-scripts\") pod \"glance-default-external-api-0\" (UID: \"4471a563-0245-40e9-bca8-acc180198abe\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:34:33 crc kubenswrapper[5052]: I1203 12:34:33.519414 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/4471a563-0245-40e9-bca8-acc180198abe-public-tls-certs\") pod \"glance-default-external-api-0\" (UID: \"4471a563-0245-40e9-bca8-acc180198abe\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:34:33 crc kubenswrapper[5052]: I1203 12:34:33.519461 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-j7s7w\" (UniqueName: \"kubernetes.io/projected/4471a563-0245-40e9-bca8-acc180198abe-kube-api-access-j7s7w\") pod \"glance-default-external-api-0\" (UID: \"4471a563-0245-40e9-bca8-acc180198abe\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:34:33 crc kubenswrapper[5052]: I1203 12:34:33.519487 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4471a563-0245-40e9-bca8-acc180198abe-combined-ca-bundle\") pod \"glance-default-external-api-0\" (UID: \"4471a563-0245-40e9-bca8-acc180198abe\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:34:33 crc kubenswrapper[5052]: I1203 12:34:33.519590 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/4471a563-0245-40e9-bca8-acc180198abe-httpd-run\") pod \"glance-default-external-api-0\" (UID: \"4471a563-0245-40e9-bca8-acc180198abe\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:34:33 crc kubenswrapper[5052]: I1203 12:34:33.519636 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/4471a563-0245-40e9-bca8-acc180198abe-config-data\") pod \"glance-default-external-api-0\" (UID: \"4471a563-0245-40e9-bca8-acc180198abe\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:34:33 crc kubenswrapper[5052]: I1203 12:34:33.519662 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage12-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage12-crc\") pod \"glance-default-external-api-0\" (UID: \"4471a563-0245-40e9-bca8-acc180198abe\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:34:33 crc kubenswrapper[5052]: I1203 12:34:33.519686 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/4471a563-0245-40e9-bca8-acc180198abe-logs\") pod \"glance-default-external-api-0\" (UID: \"4471a563-0245-40e9-bca8-acc180198abe\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:34:33 crc kubenswrapper[5052]: I1203 12:34:33.520165 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/4471a563-0245-40e9-bca8-acc180198abe-logs\") pod \"glance-default-external-api-0\" (UID: \"4471a563-0245-40e9-bca8-acc180198abe\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:34:33 crc kubenswrapper[5052]: I1203 12:34:33.521493 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/4471a563-0245-40e9-bca8-acc180198abe-httpd-run\") pod \"glance-default-external-api-0\" (UID: \"4471a563-0245-40e9-bca8-acc180198abe\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:34:33 crc kubenswrapper[5052]: I1203 12:34:33.521924 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage12-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage12-crc\") pod \"glance-default-external-api-0\" (UID: \"4471a563-0245-40e9-bca8-acc180198abe\") device mount path \"/mnt/openstack/pv12\"" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:34:33 crc kubenswrapper[5052]: I1203 12:34:33.673342 5052 scope.go:117] "RemoveContainer" containerID="e6edaf188f85ea4f11f487f11cc489c700f25fb42b5ee269f4f496f1e127d772" Dec 03 12:34:33 crc kubenswrapper[5052]: E1203 12:34:33.673676 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 12:34:33 crc kubenswrapper[5052]: I1203 12:34:33.700064 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-db-sync-nj49v"] Dec 03 12:34:33 crc kubenswrapper[5052]: I1203 12:34:33.823931 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/fa6dddd6-6ac5-4baa-9437-9a8ea5f46031-combined-ca-bundle\") pod \"glance-default-internal-api-0\" (UID: \"fa6dddd6-6ac5-4baa-9437-9a8ea5f46031\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:34:33 crc kubenswrapper[5052]: I1203 12:34:33.824437 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/fa6dddd6-6ac5-4baa-9437-9a8ea5f46031-config-data\") pod \"glance-default-internal-api-0\" (UID: \"fa6dddd6-6ac5-4baa-9437-9a8ea5f46031\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:34:33 crc kubenswrapper[5052]: W1203 12:34:33.828153 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod348487de_7480_462b_9f22_56a555321be8.slice/crio-705a14d8cdcd09593362ce67b9e841465fa1498b9c423b5b91d3146581895ab3 WatchSource:0}: Error finding container 705a14d8cdcd09593362ce67b9e841465fa1498b9c423b5b91d3146581895ab3: Status 404 returned error can't find the container with id 705a14d8cdcd09593362ce67b9e841465fa1498b9c423b5b91d3146581895ab3 Dec 03 12:34:33 crc kubenswrapper[5052]: I1203 12:34:33.848248 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/fa6dddd6-6ac5-4baa-9437-9a8ea5f46031-scripts\") pod \"glance-default-internal-api-0\" (UID: \"fa6dddd6-6ac5-4baa-9437-9a8ea5f46031\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:34:33 crc kubenswrapper[5052]: I1203 12:34:33.848573 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/fa6dddd6-6ac5-4baa-9437-9a8ea5f46031-internal-tls-certs\") pod \"glance-default-internal-api-0\" (UID: \"fa6dddd6-6ac5-4baa-9437-9a8ea5f46031\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:34:33 crc kubenswrapper[5052]: I1203 12:34:33.860603 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4471a563-0245-40e9-bca8-acc180198abe-combined-ca-bundle\") pod \"glance-default-external-api-0\" (UID: \"4471a563-0245-40e9-bca8-acc180198abe\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:34:33 crc kubenswrapper[5052]: I1203 12:34:33.860838 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/4471a563-0245-40e9-bca8-acc180198abe-public-tls-certs\") pod \"glance-default-external-api-0\" (UID: \"4471a563-0245-40e9-bca8-acc180198abe\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:34:33 crc kubenswrapper[5052]: I1203 12:34:33.861156 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/4471a563-0245-40e9-bca8-acc180198abe-scripts\") pod \"glance-default-external-api-0\" (UID: \"4471a563-0245-40e9-bca8-acc180198abe\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:34:33 crc kubenswrapper[5052]: I1203 12:34:33.861405 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-44pcb\" (UniqueName: \"kubernetes.io/projected/fa6dddd6-6ac5-4baa-9437-9a8ea5f46031-kube-api-access-44pcb\") pod \"glance-default-internal-api-0\" (UID: \"fa6dddd6-6ac5-4baa-9437-9a8ea5f46031\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:34:33 crc kubenswrapper[5052]: I1203 12:34:33.871623 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/4471a563-0245-40e9-bca8-acc180198abe-config-data\") pod \"glance-default-external-api-0\" (UID: \"4471a563-0245-40e9-bca8-acc180198abe\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:34:33 crc kubenswrapper[5052]: I1203 12:34:33.872400 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-j7s7w\" (UniqueName: \"kubernetes.io/projected/4471a563-0245-40e9-bca8-acc180198abe-kube-api-access-j7s7w\") pod \"glance-default-external-api-0\" (UID: \"4471a563-0245-40e9-bca8-acc180198abe\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:34:34 crc kubenswrapper[5052]: I1203 12:34:34.034307 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-bootstrap-hvfxb" event={"ID":"80bd8d5d-a98f-455b-977a-e96317d7f525","Type":"ContainerStarted","Data":"e7b490b7c7957636f9cffce74b4da96fad7f27b7e1fc5e91d3a9fc13f1f970f9"} Dec 03 12:34:34 crc kubenswrapper[5052]: I1203 12:34:34.034388 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-bootstrap-hvfxb" event={"ID":"80bd8d5d-a98f-455b-977a-e96317d7f525","Type":"ContainerStarted","Data":"18897b1cf1b0797253e333d8b9d11c44cfe74e0ff25736ac787bfde87f18d1c8"} Dec 03 12:34:34 crc kubenswrapper[5052]: I1203 12:34:34.089680 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-db-sync-5kww4" event={"ID":"348487de-7480-462b-9f22-56a555321be8","Type":"ContainerStarted","Data":"705a14d8cdcd09593362ce67b9e841465fa1498b9c423b5b91d3146581895ab3"} Dec 03 12:34:34 crc kubenswrapper[5052]: I1203 12:34:34.101142 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/keystone-bootstrap-hvfxb" podStartSLOduration=2.101122003 podStartE2EDuration="2.101122003s" podCreationTimestamp="2025-12-03 12:34:32 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:34:34.072027663 +0000 UTC m=+4464.660269427" watchObservedRunningTime="2025-12-03 12:34:34.101122003 +0000 UTC m=+4464.689363767" Dec 03 12:34:34 crc kubenswrapper[5052]: I1203 12:34:34.123012 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage12-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage12-crc\") pod \"glance-default-external-api-0\" (UID: \"4471a563-0245-40e9-bca8-acc180198abe\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:34:34 crc kubenswrapper[5052]: I1203 12:34:34.123009 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-db-sync-nj49v" event={"ID":"d9523127-c2ea-4f5f-98ae-0288681fb93c","Type":"ContainerStarted","Data":"cf44f0b7617a4ca325e895dca0ebefb4380bccd2b4dd6bfa58930351ecdfa3ea"} Dec 03 12:34:34 crc kubenswrapper[5052]: I1203 12:34:34.129064 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage18-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage18-crc\") pod \"glance-default-internal-api-0\" (UID: \"fa6dddd6-6ac5-4baa-9437-9a8ea5f46031\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:34:34 crc kubenswrapper[5052]: I1203 12:34:34.130207 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"6e18c0ba-8f07-4d54-b383-3d0164780725","Type":"ContainerStarted","Data":"dca0ec34dd849975bff714d71f1f4011e087a03ca11b0afe20e086380117598c"} Dec 03 12:34:34 crc kubenswrapper[5052]: I1203 12:34:34.135467 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-db-sync-fcpth" event={"ID":"a4de7484-329f-412e-b365-e373a4cab966","Type":"ContainerStarted","Data":"5d3f626046e9992f492882d32dd038cdb6e5114c45ef88706662380b3f0fbab7"} Dec 03 12:34:34 crc kubenswrapper[5052]: I1203 12:34:34.145188 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-db-sync-s42b9" event={"ID":"55ccf323-53e4-47fe-bea8-a17273b8587c","Type":"ContainerStarted","Data":"1a703be9bb20d985462a2e0555794ece28a51df5b49803a0b4a8f1399091a5d2"} Dec 03 12:34:34 crc kubenswrapper[5052]: I1203 12:34:34.250462 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:34:34 crc kubenswrapper[5052]: I1203 12:34:34.427832 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:34:34 crc kubenswrapper[5052]: I1203 12:34:34.831350 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:34:34 crc kubenswrapper[5052]: I1203 12:34:34.840757 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:34:34 crc kubenswrapper[5052]: W1203 12:34:34.842228 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4471a563_0245_40e9_bca8_acc180198abe.slice/crio-d13b3a67f8563d7e793096cf47112a67210713ed9e6278b964338b63b110080e WatchSource:0}: Error finding container d13b3a67f8563d7e793096cf47112a67210713ed9e6278b964338b63b110080e: Status 404 returned error can't find the container with id d13b3a67f8563d7e793096cf47112a67210713ed9e6278b964338b63b110080e Dec 03 12:34:34 crc kubenswrapper[5052]: I1203 12:34:34.926187 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:34:35 crc kubenswrapper[5052]: I1203 12:34:35.067167 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:34:35 crc kubenswrapper[5052]: I1203 12:34:35.092742 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:34:35 crc kubenswrapper[5052]: I1203 12:34:35.176956 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"fa6dddd6-6ac5-4baa-9437-9a8ea5f46031","Type":"ContainerStarted","Data":"234c8849dc8fbd932aa3cdc694845cf522ad66b4ec8ac7bcf25ed35f95621749"} Dec 03 12:34:35 crc kubenswrapper[5052]: I1203 12:34:35.199798 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-db-sync-nj49v" event={"ID":"d9523127-c2ea-4f5f-98ae-0288681fb93c","Type":"ContainerStarted","Data":"bf727997773d9131797166f237393d3a676ff8ba1aaa81a1ab8bdb9c481d21ae"} Dec 03 12:34:35 crc kubenswrapper[5052]: I1203 12:34:35.225413 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"4471a563-0245-40e9-bca8-acc180198abe","Type":"ContainerStarted","Data":"d13b3a67f8563d7e793096cf47112a67210713ed9e6278b964338b63b110080e"} Dec 03 12:34:35 crc kubenswrapper[5052]: I1203 12:34:35.244906 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/barbican-db-sync-nj49v" podStartSLOduration=3.244885239 podStartE2EDuration="3.244885239s" podCreationTimestamp="2025-12-03 12:34:32 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:34:35.225770073 +0000 UTC m=+4465.814011847" watchObservedRunningTime="2025-12-03 12:34:35.244885239 +0000 UTC m=+4465.833126993" Dec 03 12:34:35 crc kubenswrapper[5052]: I1203 12:34:35.250852 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"6e18c0ba-8f07-4d54-b383-3d0164780725","Type":"ContainerStarted","Data":"6ccc4a54d174219d793d5a774b64b4fc2e9d4e6aded52b78b522a8ccc368b7ba"} Dec 03 12:34:35 crc kubenswrapper[5052]: I1203 12:34:35.282043 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-db-sync-fcpth" event={"ID":"a4de7484-329f-412e-b365-e373a4cab966","Type":"ContainerStarted","Data":"a10bf847b707e0b7acccc05608dc70ede33326a069a0b09b022880875d8e19af"} Dec 03 12:34:35 crc kubenswrapper[5052]: I1203 12:34:35.307101 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-db-sync-s42b9" event={"ID":"55ccf323-53e4-47fe-bea8-a17273b8587c","Type":"ContainerStarted","Data":"d8dc47bd79781784d95621d45bf044b65727ec3ede2bffae178e8e91d46cc7b7"} Dec 03 12:34:35 crc kubenswrapper[5052]: I1203 12:34:35.326780 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-db-sync-5kww4" event={"ID":"348487de-7480-462b-9f22-56a555321be8","Type":"ContainerStarted","Data":"de7ec0b74eb7ee449efa4ae5a8ddecacb3b2510555a213842ba699fc19a15a76"} Dec 03 12:34:35 crc kubenswrapper[5052]: I1203 12:34:35.340620 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/placement-db-sync-fcpth" podStartSLOduration=3.340597481 podStartE2EDuration="3.340597481s" podCreationTimestamp="2025-12-03 12:34:32 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:34:35.297660675 +0000 UTC m=+4465.885902439" watchObservedRunningTime="2025-12-03 12:34:35.340597481 +0000 UTC m=+4465.928839245" Dec 03 12:34:35 crc kubenswrapper[5052]: I1203 12:34:35.360240 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/cinder-db-sync-s42b9" podStartSLOduration=3.360220011 podStartE2EDuration="3.360220011s" podCreationTimestamp="2025-12-03 12:34:32 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:34:35.334308031 +0000 UTC m=+4465.922549795" watchObservedRunningTime="2025-12-03 12:34:35.360220011 +0000 UTC m=+4465.948461775" Dec 03 12:34:35 crc kubenswrapper[5052]: I1203 12:34:35.369654 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/neutron-db-sync-5kww4" podStartSLOduration=3.36963418 podStartE2EDuration="3.36963418s" podCreationTimestamp="2025-12-03 12:34:32 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:34:35.358871112 +0000 UTC m=+4465.947112876" watchObservedRunningTime="2025-12-03 12:34:35.36963418 +0000 UTC m=+4465.957875974" Dec 03 12:34:36 crc kubenswrapper[5052]: I1203 12:34:36.352333 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"fa6dddd6-6ac5-4baa-9437-9a8ea5f46031","Type":"ContainerStarted","Data":"d7eb9f4115d7d6306f186d3b2fbe06f4f4dc72eb897d6223291040cc9555fc52"} Dec 03 12:34:36 crc kubenswrapper[5052]: I1203 12:34:36.357334 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"4471a563-0245-40e9-bca8-acc180198abe","Type":"ContainerStarted","Data":"23dee71ebadb046e0741ad3e14e8fb30a1c575ae880897e7d46ff43378f34975"} Dec 03 12:34:36 crc kubenswrapper[5052]: I1203 12:34:36.359478 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"6e18c0ba-8f07-4d54-b383-3d0164780725","Type":"ContainerStarted","Data":"e3b4fe5b61ea893e2ab1fcac0c2d3a2ab7831afe1733b8ce810a8002ffa09adb"} Dec 03 12:34:37 crc kubenswrapper[5052]: I1203 12:34:37.370013 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"fa6dddd6-6ac5-4baa-9437-9a8ea5f46031","Type":"ContainerStarted","Data":"3f4e36bc1892c98a17ba8a8eba8081d1019b9a3b0aa4a0caaa6572f3eb9973c3"} Dec 03 12:34:37 crc kubenswrapper[5052]: I1203 12:34:37.370135 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-internal-api-0" podUID="fa6dddd6-6ac5-4baa-9437-9a8ea5f46031" containerName="glance-log" containerID="cri-o://d7eb9f4115d7d6306f186d3b2fbe06f4f4dc72eb897d6223291040cc9555fc52" gracePeriod=30 Dec 03 12:34:37 crc kubenswrapper[5052]: I1203 12:34:37.370193 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-internal-api-0" podUID="fa6dddd6-6ac5-4baa-9437-9a8ea5f46031" containerName="glance-httpd" containerID="cri-o://3f4e36bc1892c98a17ba8a8eba8081d1019b9a3b0aa4a0caaa6572f3eb9973c3" gracePeriod=30 Dec 03 12:34:37 crc kubenswrapper[5052]: I1203 12:34:37.372575 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"4471a563-0245-40e9-bca8-acc180198abe","Type":"ContainerStarted","Data":"8d2d6c3c01e2a7a78f8703f800cba0b98ebc53688cda012fb8bfa1e2ce132b72"} Dec 03 12:34:37 crc kubenswrapper[5052]: I1203 12:34:37.372710 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-external-api-0" podUID="4471a563-0245-40e9-bca8-acc180198abe" containerName="glance-log" containerID="cri-o://23dee71ebadb046e0741ad3e14e8fb30a1c575ae880897e7d46ff43378f34975" gracePeriod=30 Dec 03 12:34:37 crc kubenswrapper[5052]: I1203 12:34:37.372800 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-external-api-0" podUID="4471a563-0245-40e9-bca8-acc180198abe" containerName="glance-httpd" containerID="cri-o://8d2d6c3c01e2a7a78f8703f800cba0b98ebc53688cda012fb8bfa1e2ce132b72" gracePeriod=30 Dec 03 12:34:37 crc kubenswrapper[5052]: I1203 12:34:37.379900 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"6e18c0ba-8f07-4d54-b383-3d0164780725","Type":"ContainerStarted","Data":"604e88a6f9d5f31113fd9e61926ba1170e23369e894c60276bf937a45384854e"} Dec 03 12:34:37 crc kubenswrapper[5052]: I1203 12:34:37.406598 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/glance-default-internal-api-0" podStartSLOduration=5.406576388 podStartE2EDuration="5.406576388s" podCreationTimestamp="2025-12-03 12:34:32 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:34:37.402970205 +0000 UTC m=+4467.991211969" watchObservedRunningTime="2025-12-03 12:34:37.406576388 +0000 UTC m=+4467.994818152" Dec 03 12:34:37 crc kubenswrapper[5052]: I1203 12:34:37.423427 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/glance-default-external-api-0" podStartSLOduration=5.423408748 podStartE2EDuration="5.423408748s" podCreationTimestamp="2025-12-03 12:34:32 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:34:37.42278123 +0000 UTC m=+4468.011023004" watchObservedRunningTime="2025-12-03 12:34:37.423408748 +0000 UTC m=+4468.011650512" Dec 03 12:34:37 crc kubenswrapper[5052]: I1203 12:34:37.982079 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.060840 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.081636 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/fa6dddd6-6ac5-4baa-9437-9a8ea5f46031-logs\") pod \"fa6dddd6-6ac5-4baa-9437-9a8ea5f46031\" (UID: \"fa6dddd6-6ac5-4baa-9437-9a8ea5f46031\") " Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.081705 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/fa6dddd6-6ac5-4baa-9437-9a8ea5f46031-scripts\") pod \"fa6dddd6-6ac5-4baa-9437-9a8ea5f46031\" (UID: \"fa6dddd6-6ac5-4baa-9437-9a8ea5f46031\") " Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.081732 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/fa6dddd6-6ac5-4baa-9437-9a8ea5f46031-httpd-run\") pod \"fa6dddd6-6ac5-4baa-9437-9a8ea5f46031\" (UID: \"fa6dddd6-6ac5-4baa-9437-9a8ea5f46031\") " Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.081756 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/fa6dddd6-6ac5-4baa-9437-9a8ea5f46031-combined-ca-bundle\") pod \"fa6dddd6-6ac5-4baa-9437-9a8ea5f46031\" (UID: \"fa6dddd6-6ac5-4baa-9437-9a8ea5f46031\") " Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.081778 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/fa6dddd6-6ac5-4baa-9437-9a8ea5f46031-internal-tls-certs\") pod \"fa6dddd6-6ac5-4baa-9437-9a8ea5f46031\" (UID: \"fa6dddd6-6ac5-4baa-9437-9a8ea5f46031\") " Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.081796 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"glance\" (UniqueName: \"kubernetes.io/local-volume/local-storage18-crc\") pod \"fa6dddd6-6ac5-4baa-9437-9a8ea5f46031\" (UID: \"fa6dddd6-6ac5-4baa-9437-9a8ea5f46031\") " Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.081922 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-44pcb\" (UniqueName: \"kubernetes.io/projected/fa6dddd6-6ac5-4baa-9437-9a8ea5f46031-kube-api-access-44pcb\") pod \"fa6dddd6-6ac5-4baa-9437-9a8ea5f46031\" (UID: \"fa6dddd6-6ac5-4baa-9437-9a8ea5f46031\") " Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.081961 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/fa6dddd6-6ac5-4baa-9437-9a8ea5f46031-config-data\") pod \"fa6dddd6-6ac5-4baa-9437-9a8ea5f46031\" (UID: \"fa6dddd6-6ac5-4baa-9437-9a8ea5f46031\") " Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.083205 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fa6dddd6-6ac5-4baa-9437-9a8ea5f46031-logs" (OuterVolumeSpecName: "logs") pod "fa6dddd6-6ac5-4baa-9437-9a8ea5f46031" (UID: "fa6dddd6-6ac5-4baa-9437-9a8ea5f46031"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.084543 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fa6dddd6-6ac5-4baa-9437-9a8ea5f46031-httpd-run" (OuterVolumeSpecName: "httpd-run") pod "fa6dddd6-6ac5-4baa-9437-9a8ea5f46031" (UID: "fa6dddd6-6ac5-4baa-9437-9a8ea5f46031"). InnerVolumeSpecName "httpd-run". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.087440 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fa6dddd6-6ac5-4baa-9437-9a8ea5f46031-kube-api-access-44pcb" (OuterVolumeSpecName: "kube-api-access-44pcb") pod "fa6dddd6-6ac5-4baa-9437-9a8ea5f46031" (UID: "fa6dddd6-6ac5-4baa-9437-9a8ea5f46031"). InnerVolumeSpecName "kube-api-access-44pcb". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.088619 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage18-crc" (OuterVolumeSpecName: "glance") pod "fa6dddd6-6ac5-4baa-9437-9a8ea5f46031" (UID: "fa6dddd6-6ac5-4baa-9437-9a8ea5f46031"). InnerVolumeSpecName "local-storage18-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.088640 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fa6dddd6-6ac5-4baa-9437-9a8ea5f46031-scripts" (OuterVolumeSpecName: "scripts") pod "fa6dddd6-6ac5-4baa-9437-9a8ea5f46031" (UID: "fa6dddd6-6ac5-4baa-9437-9a8ea5f46031"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.114593 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fa6dddd6-6ac5-4baa-9437-9a8ea5f46031-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "fa6dddd6-6ac5-4baa-9437-9a8ea5f46031" (UID: "fa6dddd6-6ac5-4baa-9437-9a8ea5f46031"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.132837 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fa6dddd6-6ac5-4baa-9437-9a8ea5f46031-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "fa6dddd6-6ac5-4baa-9437-9a8ea5f46031" (UID: "fa6dddd6-6ac5-4baa-9437-9a8ea5f46031"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.133244 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fa6dddd6-6ac5-4baa-9437-9a8ea5f46031-config-data" (OuterVolumeSpecName: "config-data") pod "fa6dddd6-6ac5-4baa-9437-9a8ea5f46031" (UID: "fa6dddd6-6ac5-4baa-9437-9a8ea5f46031"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.183404 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/4471a563-0245-40e9-bca8-acc180198abe-config-data\") pod \"4471a563-0245-40e9-bca8-acc180198abe\" (UID: \"4471a563-0245-40e9-bca8-acc180198abe\") " Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.183718 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j7s7w\" (UniqueName: \"kubernetes.io/projected/4471a563-0245-40e9-bca8-acc180198abe-kube-api-access-j7s7w\") pod \"4471a563-0245-40e9-bca8-acc180198abe\" (UID: \"4471a563-0245-40e9-bca8-acc180198abe\") " Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.183755 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/4471a563-0245-40e9-bca8-acc180198abe-logs\") pod \"4471a563-0245-40e9-bca8-acc180198abe\" (UID: \"4471a563-0245-40e9-bca8-acc180198abe\") " Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.183816 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/4471a563-0245-40e9-bca8-acc180198abe-public-tls-certs\") pod \"4471a563-0245-40e9-bca8-acc180198abe\" (UID: \"4471a563-0245-40e9-bca8-acc180198abe\") " Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.183840 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4471a563-0245-40e9-bca8-acc180198abe-combined-ca-bundle\") pod \"4471a563-0245-40e9-bca8-acc180198abe\" (UID: \"4471a563-0245-40e9-bca8-acc180198abe\") " Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.183937 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"glance\" (UniqueName: \"kubernetes.io/local-volume/local-storage12-crc\") pod \"4471a563-0245-40e9-bca8-acc180198abe\" (UID: \"4471a563-0245-40e9-bca8-acc180198abe\") " Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.183998 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/4471a563-0245-40e9-bca8-acc180198abe-httpd-run\") pod \"4471a563-0245-40e9-bca8-acc180198abe\" (UID: \"4471a563-0245-40e9-bca8-acc180198abe\") " Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.184049 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/4471a563-0245-40e9-bca8-acc180198abe-scripts\") pod \"4471a563-0245-40e9-bca8-acc180198abe\" (UID: \"4471a563-0245-40e9-bca8-acc180198abe\") " Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.184416 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-44pcb\" (UniqueName: \"kubernetes.io/projected/fa6dddd6-6ac5-4baa-9437-9a8ea5f46031-kube-api-access-44pcb\") on node \"crc\" DevicePath \"\"" Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.184433 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/fa6dddd6-6ac5-4baa-9437-9a8ea5f46031-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.184442 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/fa6dddd6-6ac5-4baa-9437-9a8ea5f46031-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.184451 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/fa6dddd6-6ac5-4baa-9437-9a8ea5f46031-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.184459 5052 reconciler_common.go:293] "Volume detached for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/fa6dddd6-6ac5-4baa-9437-9a8ea5f46031-httpd-run\") on node \"crc\" DevicePath \"\"" Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.184467 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/fa6dddd6-6ac5-4baa-9437-9a8ea5f46031-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.184476 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/fa6dddd6-6ac5-4baa-9437-9a8ea5f46031-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.184494 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage18-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage18-crc\") on node \"crc\" " Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.184957 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4471a563-0245-40e9-bca8-acc180198abe-httpd-run" (OuterVolumeSpecName: "httpd-run") pod "4471a563-0245-40e9-bca8-acc180198abe" (UID: "4471a563-0245-40e9-bca8-acc180198abe"). InnerVolumeSpecName "httpd-run". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.185000 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4471a563-0245-40e9-bca8-acc180198abe-logs" (OuterVolumeSpecName: "logs") pod "4471a563-0245-40e9-bca8-acc180198abe" (UID: "4471a563-0245-40e9-bca8-acc180198abe"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.190652 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4471a563-0245-40e9-bca8-acc180198abe-scripts" (OuterVolumeSpecName: "scripts") pod "4471a563-0245-40e9-bca8-acc180198abe" (UID: "4471a563-0245-40e9-bca8-acc180198abe"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.190662 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage12-crc" (OuterVolumeSpecName: "glance") pod "4471a563-0245-40e9-bca8-acc180198abe" (UID: "4471a563-0245-40e9-bca8-acc180198abe"). InnerVolumeSpecName "local-storage12-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.190753 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4471a563-0245-40e9-bca8-acc180198abe-kube-api-access-j7s7w" (OuterVolumeSpecName: "kube-api-access-j7s7w") pod "4471a563-0245-40e9-bca8-acc180198abe" (UID: "4471a563-0245-40e9-bca8-acc180198abe"). InnerVolumeSpecName "kube-api-access-j7s7w". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.202699 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage18-crc" (UniqueName: "kubernetes.io/local-volume/local-storage18-crc") on node "crc" Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.204887 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4471a563-0245-40e9-bca8-acc180198abe-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "4471a563-0245-40e9-bca8-acc180198abe" (UID: "4471a563-0245-40e9-bca8-acc180198abe"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.229812 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4471a563-0245-40e9-bca8-acc180198abe-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "4471a563-0245-40e9-bca8-acc180198abe" (UID: "4471a563-0245-40e9-bca8-acc180198abe"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.234864 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4471a563-0245-40e9-bca8-acc180198abe-config-data" (OuterVolumeSpecName: "config-data") pod "4471a563-0245-40e9-bca8-acc180198abe" (UID: "4471a563-0245-40e9-bca8-acc180198abe"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.286551 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/4471a563-0245-40e9-bca8-acc180198abe-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.286587 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage18-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage18-crc\") on node \"crc\" DevicePath \"\"" Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.286598 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/4471a563-0245-40e9-bca8-acc180198abe-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.286613 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-j7s7w\" (UniqueName: \"kubernetes.io/projected/4471a563-0245-40e9-bca8-acc180198abe-kube-api-access-j7s7w\") on node \"crc\" DevicePath \"\"" Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.286622 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/4471a563-0245-40e9-bca8-acc180198abe-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.286630 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/4471a563-0245-40e9-bca8-acc180198abe-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.286639 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4471a563-0245-40e9-bca8-acc180198abe-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.286673 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage12-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage12-crc\") on node \"crc\" " Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.286683 5052 reconciler_common.go:293] "Volume detached for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/4471a563-0245-40e9-bca8-acc180198abe-httpd-run\") on node \"crc\" DevicePath \"\"" Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.303080 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage12-crc" (UniqueName: "kubernetes.io/local-volume/local-storage12-crc") on node "crc" Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.387849 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage12-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage12-crc\") on node \"crc\" DevicePath \"\"" Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.391862 5052 generic.go:334] "Generic (PLEG): container finished" podID="80bd8d5d-a98f-455b-977a-e96317d7f525" containerID="e7b490b7c7957636f9cffce74b4da96fad7f27b7e1fc5e91d3a9fc13f1f970f9" exitCode=0 Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.392040 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-bootstrap-hvfxb" event={"ID":"80bd8d5d-a98f-455b-977a-e96317d7f525","Type":"ContainerDied","Data":"e7b490b7c7957636f9cffce74b4da96fad7f27b7e1fc5e91d3a9fc13f1f970f9"} Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.394645 5052 generic.go:334] "Generic (PLEG): container finished" podID="fa6dddd6-6ac5-4baa-9437-9a8ea5f46031" containerID="3f4e36bc1892c98a17ba8a8eba8081d1019b9a3b0aa4a0caaa6572f3eb9973c3" exitCode=0 Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.394677 5052 generic.go:334] "Generic (PLEG): container finished" podID="fa6dddd6-6ac5-4baa-9437-9a8ea5f46031" containerID="d7eb9f4115d7d6306f186d3b2fbe06f4f4dc72eb897d6223291040cc9555fc52" exitCode=143 Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.394723 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"fa6dddd6-6ac5-4baa-9437-9a8ea5f46031","Type":"ContainerDied","Data":"3f4e36bc1892c98a17ba8a8eba8081d1019b9a3b0aa4a0caaa6572f3eb9973c3"} Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.394741 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.394765 5052 scope.go:117] "RemoveContainer" containerID="3f4e36bc1892c98a17ba8a8eba8081d1019b9a3b0aa4a0caaa6572f3eb9973c3" Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.394752 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"fa6dddd6-6ac5-4baa-9437-9a8ea5f46031","Type":"ContainerDied","Data":"d7eb9f4115d7d6306f186d3b2fbe06f4f4dc72eb897d6223291040cc9555fc52"} Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.394943 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"fa6dddd6-6ac5-4baa-9437-9a8ea5f46031","Type":"ContainerDied","Data":"234c8849dc8fbd932aa3cdc694845cf522ad66b4ec8ac7bcf25ed35f95621749"} Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.399793 5052 generic.go:334] "Generic (PLEG): container finished" podID="4471a563-0245-40e9-bca8-acc180198abe" containerID="8d2d6c3c01e2a7a78f8703f800cba0b98ebc53688cda012fb8bfa1e2ce132b72" exitCode=0 Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.399815 5052 generic.go:334] "Generic (PLEG): container finished" podID="4471a563-0245-40e9-bca8-acc180198abe" containerID="23dee71ebadb046e0741ad3e14e8fb30a1c575ae880897e7d46ff43378f34975" exitCode=143 Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.399849 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"4471a563-0245-40e9-bca8-acc180198abe","Type":"ContainerDied","Data":"8d2d6c3c01e2a7a78f8703f800cba0b98ebc53688cda012fb8bfa1e2ce132b72"} Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.399867 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"4471a563-0245-40e9-bca8-acc180198abe","Type":"ContainerDied","Data":"23dee71ebadb046e0741ad3e14e8fb30a1c575ae880897e7d46ff43378f34975"} Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.399876 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"4471a563-0245-40e9-bca8-acc180198abe","Type":"ContainerDied","Data":"d13b3a67f8563d7e793096cf47112a67210713ed9e6278b964338b63b110080e"} Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.399922 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.421887 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"6e18c0ba-8f07-4d54-b383-3d0164780725","Type":"ContainerStarted","Data":"c03570ebb8b22d098415dfbe8a4b795a6bec4bf731ed35226369bf3a10c7348f"} Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.422032 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.422007 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="6e18c0ba-8f07-4d54-b383-3d0164780725" containerName="ceilometer-central-agent" containerID="cri-o://6ccc4a54d174219d793d5a774b64b4fc2e9d4e6aded52b78b522a8ccc368b7ba" gracePeriod=30 Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.422062 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="6e18c0ba-8f07-4d54-b383-3d0164780725" containerName="proxy-httpd" containerID="cri-o://c03570ebb8b22d098415dfbe8a4b795a6bec4bf731ed35226369bf3a10c7348f" gracePeriod=30 Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.422120 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="6e18c0ba-8f07-4d54-b383-3d0164780725" containerName="ceilometer-notification-agent" containerID="cri-o://e3b4fe5b61ea893e2ab1fcac0c2d3a2ab7831afe1733b8ce810a8002ffa09adb" gracePeriod=30 Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.422081 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="6e18c0ba-8f07-4d54-b383-3d0164780725" containerName="sg-core" containerID="cri-o://604e88a6f9d5f31113fd9e61926ba1170e23369e894c60276bf937a45384854e" gracePeriod=30 Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.430020 5052 generic.go:334] "Generic (PLEG): container finished" podID="a4de7484-329f-412e-b365-e373a4cab966" containerID="a10bf847b707e0b7acccc05608dc70ede33326a069a0b09b022880875d8e19af" exitCode=0 Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.430072 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-db-sync-fcpth" event={"ID":"a4de7484-329f-412e-b365-e373a4cab966","Type":"ContainerDied","Data":"a10bf847b707e0b7acccc05608dc70ede33326a069a0b09b022880875d8e19af"} Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.443885 5052 scope.go:117] "RemoveContainer" containerID="d7eb9f4115d7d6306f186d3b2fbe06f4f4dc72eb897d6223291040cc9555fc52" Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.457333 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/ceilometer-0" podStartSLOduration=1.857239372 podStartE2EDuration="6.457312048s" podCreationTimestamp="2025-12-03 12:34:32 +0000 UTC" firstStartedPulling="2025-12-03 12:34:33.123033836 +0000 UTC m=+4463.711275600" lastFinishedPulling="2025-12-03 12:34:37.723106512 +0000 UTC m=+4468.311348276" observedRunningTime="2025-12-03 12:34:38.447379745 +0000 UTC m=+4469.035621529" watchObservedRunningTime="2025-12-03 12:34:38.457312048 +0000 UTC m=+4469.045553812" Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.498255 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.507971 5052 scope.go:117] "RemoveContainer" containerID="3f4e36bc1892c98a17ba8a8eba8081d1019b9a3b0aa4a0caaa6572f3eb9973c3" Dec 03 12:34:38 crc kubenswrapper[5052]: E1203 12:34:38.517395 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3f4e36bc1892c98a17ba8a8eba8081d1019b9a3b0aa4a0caaa6572f3eb9973c3\": container with ID starting with 3f4e36bc1892c98a17ba8a8eba8081d1019b9a3b0aa4a0caaa6572f3eb9973c3 not found: ID does not exist" containerID="3f4e36bc1892c98a17ba8a8eba8081d1019b9a3b0aa4a0caaa6572f3eb9973c3" Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.517607 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3f4e36bc1892c98a17ba8a8eba8081d1019b9a3b0aa4a0caaa6572f3eb9973c3"} err="failed to get container status \"3f4e36bc1892c98a17ba8a8eba8081d1019b9a3b0aa4a0caaa6572f3eb9973c3\": rpc error: code = NotFound desc = could not find container \"3f4e36bc1892c98a17ba8a8eba8081d1019b9a3b0aa4a0caaa6572f3eb9973c3\": container with ID starting with 3f4e36bc1892c98a17ba8a8eba8081d1019b9a3b0aa4a0caaa6572f3eb9973c3 not found: ID does not exist" Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.517639 5052 scope.go:117] "RemoveContainer" containerID="d7eb9f4115d7d6306f186d3b2fbe06f4f4dc72eb897d6223291040cc9555fc52" Dec 03 12:34:38 crc kubenswrapper[5052]: E1203 12:34:38.520999 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d7eb9f4115d7d6306f186d3b2fbe06f4f4dc72eb897d6223291040cc9555fc52\": container with ID starting with d7eb9f4115d7d6306f186d3b2fbe06f4f4dc72eb897d6223291040cc9555fc52 not found: ID does not exist" containerID="d7eb9f4115d7d6306f186d3b2fbe06f4f4dc72eb897d6223291040cc9555fc52" Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.521042 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d7eb9f4115d7d6306f186d3b2fbe06f4f4dc72eb897d6223291040cc9555fc52"} err="failed to get container status \"d7eb9f4115d7d6306f186d3b2fbe06f4f4dc72eb897d6223291040cc9555fc52\": rpc error: code = NotFound desc = could not find container \"d7eb9f4115d7d6306f186d3b2fbe06f4f4dc72eb897d6223291040cc9555fc52\": container with ID starting with d7eb9f4115d7d6306f186d3b2fbe06f4f4dc72eb897d6223291040cc9555fc52 not found: ID does not exist" Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.521070 5052 scope.go:117] "RemoveContainer" containerID="3f4e36bc1892c98a17ba8a8eba8081d1019b9a3b0aa4a0caaa6572f3eb9973c3" Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.522738 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3f4e36bc1892c98a17ba8a8eba8081d1019b9a3b0aa4a0caaa6572f3eb9973c3"} err="failed to get container status \"3f4e36bc1892c98a17ba8a8eba8081d1019b9a3b0aa4a0caaa6572f3eb9973c3\": rpc error: code = NotFound desc = could not find container \"3f4e36bc1892c98a17ba8a8eba8081d1019b9a3b0aa4a0caaa6572f3eb9973c3\": container with ID starting with 3f4e36bc1892c98a17ba8a8eba8081d1019b9a3b0aa4a0caaa6572f3eb9973c3 not found: ID does not exist" Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.522767 5052 scope.go:117] "RemoveContainer" containerID="d7eb9f4115d7d6306f186d3b2fbe06f4f4dc72eb897d6223291040cc9555fc52" Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.525985 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d7eb9f4115d7d6306f186d3b2fbe06f4f4dc72eb897d6223291040cc9555fc52"} err="failed to get container status \"d7eb9f4115d7d6306f186d3b2fbe06f4f4dc72eb897d6223291040cc9555fc52\": rpc error: code = NotFound desc = could not find container \"d7eb9f4115d7d6306f186d3b2fbe06f4f4dc72eb897d6223291040cc9555fc52\": container with ID starting with d7eb9f4115d7d6306f186d3b2fbe06f4f4dc72eb897d6223291040cc9555fc52 not found: ID does not exist" Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.526033 5052 scope.go:117] "RemoveContainer" containerID="8d2d6c3c01e2a7a78f8703f800cba0b98ebc53688cda012fb8bfa1e2ce132b72" Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.526595 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.567060 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:34:38 crc kubenswrapper[5052]: E1203 12:34:38.567557 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="fa6dddd6-6ac5-4baa-9437-9a8ea5f46031" containerName="glance-httpd" Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.567575 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="fa6dddd6-6ac5-4baa-9437-9a8ea5f46031" containerName="glance-httpd" Dec 03 12:34:38 crc kubenswrapper[5052]: E1203 12:34:38.567605 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="fa6dddd6-6ac5-4baa-9437-9a8ea5f46031" containerName="glance-log" Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.567612 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="fa6dddd6-6ac5-4baa-9437-9a8ea5f46031" containerName="glance-log" Dec 03 12:34:38 crc kubenswrapper[5052]: E1203 12:34:38.567631 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="4471a563-0245-40e9-bca8-acc180198abe" containerName="glance-httpd" Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.567638 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="4471a563-0245-40e9-bca8-acc180198abe" containerName="glance-httpd" Dec 03 12:34:38 crc kubenswrapper[5052]: E1203 12:34:38.567660 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="4471a563-0245-40e9-bca8-acc180198abe" containerName="glance-log" Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.567667 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="4471a563-0245-40e9-bca8-acc180198abe" containerName="glance-log" Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.567869 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="4471a563-0245-40e9-bca8-acc180198abe" containerName="glance-log" Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.567884 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="fa6dddd6-6ac5-4baa-9437-9a8ea5f46031" containerName="glance-log" Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.567906 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="fa6dddd6-6ac5-4baa-9437-9a8ea5f46031" containerName="glance-httpd" Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.567917 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="4471a563-0245-40e9-bca8-acc180198abe" containerName="glance-httpd" Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.569020 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.572955 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-scripts" Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.573018 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-glance-default-public-svc" Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.573460 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-glance-dockercfg-xkmzv" Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.573798 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-default-external-config-data" Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.576918 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.585417 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.595177 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.604137 5052 scope.go:117] "RemoveContainer" containerID="23dee71ebadb046e0741ad3e14e8fb30a1c575ae880897e7d46ff43378f34975" Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.610178 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.613549 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.615559 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-glance-default-internal-svc" Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.616747 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-default-internal-config-data" Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.627383 5052 scope.go:117] "RemoveContainer" containerID="8d2d6c3c01e2a7a78f8703f800cba0b98ebc53688cda012fb8bfa1e2ce132b72" Dec 03 12:34:38 crc kubenswrapper[5052]: E1203 12:34:38.630970 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8d2d6c3c01e2a7a78f8703f800cba0b98ebc53688cda012fb8bfa1e2ce132b72\": container with ID starting with 8d2d6c3c01e2a7a78f8703f800cba0b98ebc53688cda012fb8bfa1e2ce132b72 not found: ID does not exist" containerID="8d2d6c3c01e2a7a78f8703f800cba0b98ebc53688cda012fb8bfa1e2ce132b72" Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.631532 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8d2d6c3c01e2a7a78f8703f800cba0b98ebc53688cda012fb8bfa1e2ce132b72"} err="failed to get container status \"8d2d6c3c01e2a7a78f8703f800cba0b98ebc53688cda012fb8bfa1e2ce132b72\": rpc error: code = NotFound desc = could not find container \"8d2d6c3c01e2a7a78f8703f800cba0b98ebc53688cda012fb8bfa1e2ce132b72\": container with ID starting with 8d2d6c3c01e2a7a78f8703f800cba0b98ebc53688cda012fb8bfa1e2ce132b72 not found: ID does not exist" Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.631649 5052 scope.go:117] "RemoveContainer" containerID="23dee71ebadb046e0741ad3e14e8fb30a1c575ae880897e7d46ff43378f34975" Dec 03 12:34:38 crc kubenswrapper[5052]: E1203 12:34:38.633890 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"23dee71ebadb046e0741ad3e14e8fb30a1c575ae880897e7d46ff43378f34975\": container with ID starting with 23dee71ebadb046e0741ad3e14e8fb30a1c575ae880897e7d46ff43378f34975 not found: ID does not exist" containerID="23dee71ebadb046e0741ad3e14e8fb30a1c575ae880897e7d46ff43378f34975" Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.634126 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"23dee71ebadb046e0741ad3e14e8fb30a1c575ae880897e7d46ff43378f34975"} err="failed to get container status \"23dee71ebadb046e0741ad3e14e8fb30a1c575ae880897e7d46ff43378f34975\": rpc error: code = NotFound desc = could not find container \"23dee71ebadb046e0741ad3e14e8fb30a1c575ae880897e7d46ff43378f34975\": container with ID starting with 23dee71ebadb046e0741ad3e14e8fb30a1c575ae880897e7d46ff43378f34975 not found: ID does not exist" Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.634370 5052 scope.go:117] "RemoveContainer" containerID="8d2d6c3c01e2a7a78f8703f800cba0b98ebc53688cda012fb8bfa1e2ce132b72" Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.638787 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8d2d6c3c01e2a7a78f8703f800cba0b98ebc53688cda012fb8bfa1e2ce132b72"} err="failed to get container status \"8d2d6c3c01e2a7a78f8703f800cba0b98ebc53688cda012fb8bfa1e2ce132b72\": rpc error: code = NotFound desc = could not find container \"8d2d6c3c01e2a7a78f8703f800cba0b98ebc53688cda012fb8bfa1e2ce132b72\": container with ID starting with 8d2d6c3c01e2a7a78f8703f800cba0b98ebc53688cda012fb8bfa1e2ce132b72 not found: ID does not exist" Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.638840 5052 scope.go:117] "RemoveContainer" containerID="23dee71ebadb046e0741ad3e14e8fb30a1c575ae880897e7d46ff43378f34975" Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.639314 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"23dee71ebadb046e0741ad3e14e8fb30a1c575ae880897e7d46ff43378f34975"} err="failed to get container status \"23dee71ebadb046e0741ad3e14e8fb30a1c575ae880897e7d46ff43378f34975\": rpc error: code = NotFound desc = could not find container \"23dee71ebadb046e0741ad3e14e8fb30a1c575ae880897e7d46ff43378f34975\": container with ID starting with 23dee71ebadb046e0741ad3e14e8fb30a1c575ae880897e7d46ff43378f34975 not found: ID does not exist" Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.646574 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.698673 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/6e96674e-4a20-4d8f-9118-90532fa75788-scripts\") pod \"glance-default-external-api-0\" (UID: \"6e96674e-4a20-4d8f-9118-90532fa75788\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.698754 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/6e96674e-4a20-4d8f-9118-90532fa75788-httpd-run\") pod \"glance-default-external-api-0\" (UID: \"6e96674e-4a20-4d8f-9118-90532fa75788\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.698781 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6e96674e-4a20-4d8f-9118-90532fa75788-combined-ca-bundle\") pod \"glance-default-external-api-0\" (UID: \"6e96674e-4a20-4d8f-9118-90532fa75788\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.698834 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/6e96674e-4a20-4d8f-9118-90532fa75788-logs\") pod \"glance-default-external-api-0\" (UID: \"6e96674e-4a20-4d8f-9118-90532fa75788\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.698866 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6e96674e-4a20-4d8f-9118-90532fa75788-config-data\") pod \"glance-default-external-api-0\" (UID: \"6e96674e-4a20-4d8f-9118-90532fa75788\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.698909 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/25498b02-c179-4555-ad04-1ef6445df4ab-combined-ca-bundle\") pod \"glance-default-internal-api-0\" (UID: \"25498b02-c179-4555-ad04-1ef6445df4ab\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.698944 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage18-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage18-crc\") pod \"glance-default-internal-api-0\" (UID: \"25498b02-c179-4555-ad04-1ef6445df4ab\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.698972 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/25498b02-c179-4555-ad04-1ef6445df4ab-httpd-run\") pod \"glance-default-internal-api-0\" (UID: \"25498b02-c179-4555-ad04-1ef6445df4ab\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.698992 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mbwvx\" (UniqueName: \"kubernetes.io/projected/25498b02-c179-4555-ad04-1ef6445df4ab-kube-api-access-mbwvx\") pod \"glance-default-internal-api-0\" (UID: \"25498b02-c179-4555-ad04-1ef6445df4ab\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.699024 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/25498b02-c179-4555-ad04-1ef6445df4ab-scripts\") pod \"glance-default-internal-api-0\" (UID: \"25498b02-c179-4555-ad04-1ef6445df4ab\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.699041 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage12-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage12-crc\") pod \"glance-default-external-api-0\" (UID: \"6e96674e-4a20-4d8f-9118-90532fa75788\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.699058 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/25498b02-c179-4555-ad04-1ef6445df4ab-config-data\") pod \"glance-default-internal-api-0\" (UID: \"25498b02-c179-4555-ad04-1ef6445df4ab\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.699081 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h87dc\" (UniqueName: \"kubernetes.io/projected/6e96674e-4a20-4d8f-9118-90532fa75788-kube-api-access-h87dc\") pod \"glance-default-external-api-0\" (UID: \"6e96674e-4a20-4d8f-9118-90532fa75788\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.699109 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/25498b02-c179-4555-ad04-1ef6445df4ab-logs\") pod \"glance-default-internal-api-0\" (UID: \"25498b02-c179-4555-ad04-1ef6445df4ab\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.699123 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/25498b02-c179-4555-ad04-1ef6445df4ab-internal-tls-certs\") pod \"glance-default-internal-api-0\" (UID: \"25498b02-c179-4555-ad04-1ef6445df4ab\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.699144 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/6e96674e-4a20-4d8f-9118-90532fa75788-public-tls-certs\") pod \"glance-default-external-api-0\" (UID: \"6e96674e-4a20-4d8f-9118-90532fa75788\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.801166 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6e96674e-4a20-4d8f-9118-90532fa75788-combined-ca-bundle\") pod \"glance-default-external-api-0\" (UID: \"6e96674e-4a20-4d8f-9118-90532fa75788\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.801320 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/6e96674e-4a20-4d8f-9118-90532fa75788-httpd-run\") pod \"glance-default-external-api-0\" (UID: \"6e96674e-4a20-4d8f-9118-90532fa75788\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.801410 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/6e96674e-4a20-4d8f-9118-90532fa75788-logs\") pod \"glance-default-external-api-0\" (UID: \"6e96674e-4a20-4d8f-9118-90532fa75788\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.801501 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6e96674e-4a20-4d8f-9118-90532fa75788-config-data\") pod \"glance-default-external-api-0\" (UID: \"6e96674e-4a20-4d8f-9118-90532fa75788\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.801609 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/25498b02-c179-4555-ad04-1ef6445df4ab-combined-ca-bundle\") pod \"glance-default-internal-api-0\" (UID: \"25498b02-c179-4555-ad04-1ef6445df4ab\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.801708 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage18-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage18-crc\") pod \"glance-default-internal-api-0\" (UID: \"25498b02-c179-4555-ad04-1ef6445df4ab\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.801802 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/25498b02-c179-4555-ad04-1ef6445df4ab-httpd-run\") pod \"glance-default-internal-api-0\" (UID: \"25498b02-c179-4555-ad04-1ef6445df4ab\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.801887 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-mbwvx\" (UniqueName: \"kubernetes.io/projected/25498b02-c179-4555-ad04-1ef6445df4ab-kube-api-access-mbwvx\") pod \"glance-default-internal-api-0\" (UID: \"25498b02-c179-4555-ad04-1ef6445df4ab\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.801980 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/25498b02-c179-4555-ad04-1ef6445df4ab-scripts\") pod \"glance-default-internal-api-0\" (UID: \"25498b02-c179-4555-ad04-1ef6445df4ab\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.802053 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage12-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage12-crc\") pod \"glance-default-external-api-0\" (UID: \"6e96674e-4a20-4d8f-9118-90532fa75788\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.802135 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/25498b02-c179-4555-ad04-1ef6445df4ab-config-data\") pod \"glance-default-internal-api-0\" (UID: \"25498b02-c179-4555-ad04-1ef6445df4ab\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.802225 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-h87dc\" (UniqueName: \"kubernetes.io/projected/6e96674e-4a20-4d8f-9118-90532fa75788-kube-api-access-h87dc\") pod \"glance-default-external-api-0\" (UID: \"6e96674e-4a20-4d8f-9118-90532fa75788\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.802311 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/25498b02-c179-4555-ad04-1ef6445df4ab-internal-tls-certs\") pod \"glance-default-internal-api-0\" (UID: \"25498b02-c179-4555-ad04-1ef6445df4ab\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.802375 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/6e96674e-4a20-4d8f-9118-90532fa75788-logs\") pod \"glance-default-external-api-0\" (UID: \"6e96674e-4a20-4d8f-9118-90532fa75788\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.802385 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/25498b02-c179-4555-ad04-1ef6445df4ab-logs\") pod \"glance-default-internal-api-0\" (UID: \"25498b02-c179-4555-ad04-1ef6445df4ab\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.802438 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/6e96674e-4a20-4d8f-9118-90532fa75788-public-tls-certs\") pod \"glance-default-external-api-0\" (UID: \"6e96674e-4a20-4d8f-9118-90532fa75788\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.801847 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/6e96674e-4a20-4d8f-9118-90532fa75788-httpd-run\") pod \"glance-default-external-api-0\" (UID: \"6e96674e-4a20-4d8f-9118-90532fa75788\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.802578 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/6e96674e-4a20-4d8f-9118-90532fa75788-scripts\") pod \"glance-default-external-api-0\" (UID: \"6e96674e-4a20-4d8f-9118-90532fa75788\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.802601 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/25498b02-c179-4555-ad04-1ef6445df4ab-httpd-run\") pod \"glance-default-internal-api-0\" (UID: \"25498b02-c179-4555-ad04-1ef6445df4ab\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.802705 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage18-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage18-crc\") pod \"glance-default-internal-api-0\" (UID: \"25498b02-c179-4555-ad04-1ef6445df4ab\") device mount path \"/mnt/openstack/pv18\"" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.803020 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/25498b02-c179-4555-ad04-1ef6445df4ab-logs\") pod \"glance-default-internal-api-0\" (UID: \"25498b02-c179-4555-ad04-1ef6445df4ab\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.803488 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage12-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage12-crc\") pod \"glance-default-external-api-0\" (UID: \"6e96674e-4a20-4d8f-9118-90532fa75788\") device mount path \"/mnt/openstack/pv12\"" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.806150 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/6e96674e-4a20-4d8f-9118-90532fa75788-scripts\") pod \"glance-default-external-api-0\" (UID: \"6e96674e-4a20-4d8f-9118-90532fa75788\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.806289 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6e96674e-4a20-4d8f-9118-90532fa75788-combined-ca-bundle\") pod \"glance-default-external-api-0\" (UID: \"6e96674e-4a20-4d8f-9118-90532fa75788\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.806786 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/25498b02-c179-4555-ad04-1ef6445df4ab-scripts\") pod \"glance-default-internal-api-0\" (UID: \"25498b02-c179-4555-ad04-1ef6445df4ab\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.807390 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/6e96674e-4a20-4d8f-9118-90532fa75788-public-tls-certs\") pod \"glance-default-external-api-0\" (UID: \"6e96674e-4a20-4d8f-9118-90532fa75788\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.808484 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/25498b02-c179-4555-ad04-1ef6445df4ab-config-data\") pod \"glance-default-internal-api-0\" (UID: \"25498b02-c179-4555-ad04-1ef6445df4ab\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.810170 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/25498b02-c179-4555-ad04-1ef6445df4ab-combined-ca-bundle\") pod \"glance-default-internal-api-0\" (UID: \"25498b02-c179-4555-ad04-1ef6445df4ab\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.819396 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-h87dc\" (UniqueName: \"kubernetes.io/projected/6e96674e-4a20-4d8f-9118-90532fa75788-kube-api-access-h87dc\") pod \"glance-default-external-api-0\" (UID: \"6e96674e-4a20-4d8f-9118-90532fa75788\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.819442 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-mbwvx\" (UniqueName: \"kubernetes.io/projected/25498b02-c179-4555-ad04-1ef6445df4ab-kube-api-access-mbwvx\") pod \"glance-default-internal-api-0\" (UID: \"25498b02-c179-4555-ad04-1ef6445df4ab\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.827802 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6e96674e-4a20-4d8f-9118-90532fa75788-config-data\") pod \"glance-default-external-api-0\" (UID: \"6e96674e-4a20-4d8f-9118-90532fa75788\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.827987 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/25498b02-c179-4555-ad04-1ef6445df4ab-internal-tls-certs\") pod \"glance-default-internal-api-0\" (UID: \"25498b02-c179-4555-ad04-1ef6445df4ab\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.836111 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage12-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage12-crc\") pod \"glance-default-external-api-0\" (UID: \"6e96674e-4a20-4d8f-9118-90532fa75788\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.840361 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage18-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage18-crc\") pod \"glance-default-internal-api-0\" (UID: \"25498b02-c179-4555-ad04-1ef6445df4ab\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.903380 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:34:38 crc kubenswrapper[5052]: I1203 12:34:38.971931 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:34:39 crc kubenswrapper[5052]: I1203 12:34:39.342502 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:34:39 crc kubenswrapper[5052]: I1203 12:34:39.445158 5052 generic.go:334] "Generic (PLEG): container finished" podID="6e18c0ba-8f07-4d54-b383-3d0164780725" containerID="c03570ebb8b22d098415dfbe8a4b795a6bec4bf731ed35226369bf3a10c7348f" exitCode=0 Dec 03 12:34:39 crc kubenswrapper[5052]: I1203 12:34:39.445185 5052 generic.go:334] "Generic (PLEG): container finished" podID="6e18c0ba-8f07-4d54-b383-3d0164780725" containerID="604e88a6f9d5f31113fd9e61926ba1170e23369e894c60276bf937a45384854e" exitCode=2 Dec 03 12:34:39 crc kubenswrapper[5052]: I1203 12:34:39.445192 5052 generic.go:334] "Generic (PLEG): container finished" podID="6e18c0ba-8f07-4d54-b383-3d0164780725" containerID="e3b4fe5b61ea893e2ab1fcac0c2d3a2ab7831afe1733b8ce810a8002ffa09adb" exitCode=0 Dec 03 12:34:39 crc kubenswrapper[5052]: I1203 12:34:39.445201 5052 generic.go:334] "Generic (PLEG): container finished" podID="6e18c0ba-8f07-4d54-b383-3d0164780725" containerID="6ccc4a54d174219d793d5a774b64b4fc2e9d4e6aded52b78b522a8ccc368b7ba" exitCode=0 Dec 03 12:34:39 crc kubenswrapper[5052]: I1203 12:34:39.445203 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"6e18c0ba-8f07-4d54-b383-3d0164780725","Type":"ContainerDied","Data":"c03570ebb8b22d098415dfbe8a4b795a6bec4bf731ed35226369bf3a10c7348f"} Dec 03 12:34:39 crc kubenswrapper[5052]: I1203 12:34:39.445250 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"6e18c0ba-8f07-4d54-b383-3d0164780725","Type":"ContainerDied","Data":"604e88a6f9d5f31113fd9e61926ba1170e23369e894c60276bf937a45384854e"} Dec 03 12:34:39 crc kubenswrapper[5052]: I1203 12:34:39.445254 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:34:39 crc kubenswrapper[5052]: I1203 12:34:39.445278 5052 scope.go:117] "RemoveContainer" containerID="c03570ebb8b22d098415dfbe8a4b795a6bec4bf731ed35226369bf3a10c7348f" Dec 03 12:34:39 crc kubenswrapper[5052]: I1203 12:34:39.445266 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"6e18c0ba-8f07-4d54-b383-3d0164780725","Type":"ContainerDied","Data":"e3b4fe5b61ea893e2ab1fcac0c2d3a2ab7831afe1733b8ce810a8002ffa09adb"} Dec 03 12:34:39 crc kubenswrapper[5052]: I1203 12:34:39.445326 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"6e18c0ba-8f07-4d54-b383-3d0164780725","Type":"ContainerDied","Data":"6ccc4a54d174219d793d5a774b64b4fc2e9d4e6aded52b78b522a8ccc368b7ba"} Dec 03 12:34:39 crc kubenswrapper[5052]: I1203 12:34:39.445357 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"6e18c0ba-8f07-4d54-b383-3d0164780725","Type":"ContainerDied","Data":"dca0ec34dd849975bff714d71f1f4011e087a03ca11b0afe20e086380117598c"} Dec 03 12:34:39 crc kubenswrapper[5052]: I1203 12:34:39.448773 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/6e18c0ba-8f07-4d54-b383-3d0164780725-log-httpd\") pod \"6e18c0ba-8f07-4d54-b383-3d0164780725\" (UID: \"6e18c0ba-8f07-4d54-b383-3d0164780725\") " Dec 03 12:34:39 crc kubenswrapper[5052]: I1203 12:34:39.448837 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/6e18c0ba-8f07-4d54-b383-3d0164780725-scripts\") pod \"6e18c0ba-8f07-4d54-b383-3d0164780725\" (UID: \"6e18c0ba-8f07-4d54-b383-3d0164780725\") " Dec 03 12:34:39 crc kubenswrapper[5052]: I1203 12:34:39.448886 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/6e18c0ba-8f07-4d54-b383-3d0164780725-sg-core-conf-yaml\") pod \"6e18c0ba-8f07-4d54-b383-3d0164780725\" (UID: \"6e18c0ba-8f07-4d54-b383-3d0164780725\") " Dec 03 12:34:39 crc kubenswrapper[5052]: I1203 12:34:39.448968 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/6e18c0ba-8f07-4d54-b383-3d0164780725-run-httpd\") pod \"6e18c0ba-8f07-4d54-b383-3d0164780725\" (UID: \"6e18c0ba-8f07-4d54-b383-3d0164780725\") " Dec 03 12:34:39 crc kubenswrapper[5052]: I1203 12:34:39.449048 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6e18c0ba-8f07-4d54-b383-3d0164780725-combined-ca-bundle\") pod \"6e18c0ba-8f07-4d54-b383-3d0164780725\" (UID: \"6e18c0ba-8f07-4d54-b383-3d0164780725\") " Dec 03 12:34:39 crc kubenswrapper[5052]: I1203 12:34:39.449094 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fxdfv\" (UniqueName: \"kubernetes.io/projected/6e18c0ba-8f07-4d54-b383-3d0164780725-kube-api-access-fxdfv\") pod \"6e18c0ba-8f07-4d54-b383-3d0164780725\" (UID: \"6e18c0ba-8f07-4d54-b383-3d0164780725\") " Dec 03 12:34:39 crc kubenswrapper[5052]: I1203 12:34:39.449203 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6e18c0ba-8f07-4d54-b383-3d0164780725-config-data\") pod \"6e18c0ba-8f07-4d54-b383-3d0164780725\" (UID: \"6e18c0ba-8f07-4d54-b383-3d0164780725\") " Dec 03 12:34:39 crc kubenswrapper[5052]: I1203 12:34:39.449504 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e18c0ba-8f07-4d54-b383-3d0164780725-log-httpd" (OuterVolumeSpecName: "log-httpd") pod "6e18c0ba-8f07-4d54-b383-3d0164780725" (UID: "6e18c0ba-8f07-4d54-b383-3d0164780725"). InnerVolumeSpecName "log-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:34:39 crc kubenswrapper[5052]: I1203 12:34:39.449651 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e18c0ba-8f07-4d54-b383-3d0164780725-run-httpd" (OuterVolumeSpecName: "run-httpd") pod "6e18c0ba-8f07-4d54-b383-3d0164780725" (UID: "6e18c0ba-8f07-4d54-b383-3d0164780725"). InnerVolumeSpecName "run-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:34:39 crc kubenswrapper[5052]: I1203 12:34:39.450539 5052 reconciler_common.go:293] "Volume detached for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/6e18c0ba-8f07-4d54-b383-3d0164780725-run-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:34:39 crc kubenswrapper[5052]: I1203 12:34:39.450558 5052 reconciler_common.go:293] "Volume detached for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/6e18c0ba-8f07-4d54-b383-3d0164780725-log-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:34:39 crc kubenswrapper[5052]: I1203 12:34:39.455698 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6e18c0ba-8f07-4d54-b383-3d0164780725-scripts" (OuterVolumeSpecName: "scripts") pod "6e18c0ba-8f07-4d54-b383-3d0164780725" (UID: "6e18c0ba-8f07-4d54-b383-3d0164780725"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:34:39 crc kubenswrapper[5052]: I1203 12:34:39.456860 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6e18c0ba-8f07-4d54-b383-3d0164780725-kube-api-access-fxdfv" (OuterVolumeSpecName: "kube-api-access-fxdfv") pod "6e18c0ba-8f07-4d54-b383-3d0164780725" (UID: "6e18c0ba-8f07-4d54-b383-3d0164780725"). InnerVolumeSpecName "kube-api-access-fxdfv". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:34:39 crc kubenswrapper[5052]: I1203 12:34:39.471398 5052 scope.go:117] "RemoveContainer" containerID="604e88a6f9d5f31113fd9e61926ba1170e23369e894c60276bf937a45384854e" Dec 03 12:34:39 crc kubenswrapper[5052]: I1203 12:34:39.489689 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6e18c0ba-8f07-4d54-b383-3d0164780725-sg-core-conf-yaml" (OuterVolumeSpecName: "sg-core-conf-yaml") pod "6e18c0ba-8f07-4d54-b383-3d0164780725" (UID: "6e18c0ba-8f07-4d54-b383-3d0164780725"). InnerVolumeSpecName "sg-core-conf-yaml". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:34:39 crc kubenswrapper[5052]: I1203 12:34:39.502491 5052 scope.go:117] "RemoveContainer" containerID="e3b4fe5b61ea893e2ab1fcac0c2d3a2ab7831afe1733b8ce810a8002ffa09adb" Dec 03 12:34:39 crc kubenswrapper[5052]: I1203 12:34:39.561488 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-fxdfv\" (UniqueName: \"kubernetes.io/projected/6e18c0ba-8f07-4d54-b383-3d0164780725-kube-api-access-fxdfv\") on node \"crc\" DevicePath \"\"" Dec 03 12:34:39 crc kubenswrapper[5052]: I1203 12:34:39.561865 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/6e18c0ba-8f07-4d54-b383-3d0164780725-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:34:39 crc kubenswrapper[5052]: I1203 12:34:39.561880 5052 reconciler_common.go:293] "Volume detached for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/6e18c0ba-8f07-4d54-b383-3d0164780725-sg-core-conf-yaml\") on node \"crc\" DevicePath \"\"" Dec 03 12:34:39 crc kubenswrapper[5052]: I1203 12:34:39.576486 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6e18c0ba-8f07-4d54-b383-3d0164780725-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "6e18c0ba-8f07-4d54-b383-3d0164780725" (UID: "6e18c0ba-8f07-4d54-b383-3d0164780725"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:34:39 crc kubenswrapper[5052]: I1203 12:34:39.618312 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6e18c0ba-8f07-4d54-b383-3d0164780725-config-data" (OuterVolumeSpecName: "config-data") pod "6e18c0ba-8f07-4d54-b383-3d0164780725" (UID: "6e18c0ba-8f07-4d54-b383-3d0164780725"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:34:39 crc kubenswrapper[5052]: I1203 12:34:39.621758 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:34:39 crc kubenswrapper[5052]: I1203 12:34:39.634271 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:34:39 crc kubenswrapper[5052]: I1203 12:34:39.664733 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6e18c0ba-8f07-4d54-b383-3d0164780725-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:34:39 crc kubenswrapper[5052]: I1203 12:34:39.664782 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6e18c0ba-8f07-4d54-b383-3d0164780725-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:34:39 crc kubenswrapper[5052]: I1203 12:34:39.700597 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4471a563-0245-40e9-bca8-acc180198abe" path="/var/lib/kubelet/pods/4471a563-0245-40e9-bca8-acc180198abe/volumes" Dec 03 12:34:39 crc kubenswrapper[5052]: I1203 12:34:39.706587 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fa6dddd6-6ac5-4baa-9437-9a8ea5f46031" path="/var/lib/kubelet/pods/fa6dddd6-6ac5-4baa-9437-9a8ea5f46031/volumes" Dec 03 12:34:39 crc kubenswrapper[5052]: I1203 12:34:39.709265 5052 scope.go:117] "RemoveContainer" containerID="6ccc4a54d174219d793d5a774b64b4fc2e9d4e6aded52b78b522a8ccc368b7ba" Dec 03 12:34:39 crc kubenswrapper[5052]: I1203 12:34:39.817631 5052 scope.go:117] "RemoveContainer" containerID="c03570ebb8b22d098415dfbe8a4b795a6bec4bf731ed35226369bf3a10c7348f" Dec 03 12:34:39 crc kubenswrapper[5052]: E1203 12:34:39.819897 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c03570ebb8b22d098415dfbe8a4b795a6bec4bf731ed35226369bf3a10c7348f\": container with ID starting with c03570ebb8b22d098415dfbe8a4b795a6bec4bf731ed35226369bf3a10c7348f not found: ID does not exist" containerID="c03570ebb8b22d098415dfbe8a4b795a6bec4bf731ed35226369bf3a10c7348f" Dec 03 12:34:39 crc kubenswrapper[5052]: I1203 12:34:39.819942 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c03570ebb8b22d098415dfbe8a4b795a6bec4bf731ed35226369bf3a10c7348f"} err="failed to get container status \"c03570ebb8b22d098415dfbe8a4b795a6bec4bf731ed35226369bf3a10c7348f\": rpc error: code = NotFound desc = could not find container \"c03570ebb8b22d098415dfbe8a4b795a6bec4bf731ed35226369bf3a10c7348f\": container with ID starting with c03570ebb8b22d098415dfbe8a4b795a6bec4bf731ed35226369bf3a10c7348f not found: ID does not exist" Dec 03 12:34:39 crc kubenswrapper[5052]: I1203 12:34:39.819969 5052 scope.go:117] "RemoveContainer" containerID="604e88a6f9d5f31113fd9e61926ba1170e23369e894c60276bf937a45384854e" Dec 03 12:34:39 crc kubenswrapper[5052]: E1203 12:34:39.840438 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"604e88a6f9d5f31113fd9e61926ba1170e23369e894c60276bf937a45384854e\": container with ID starting with 604e88a6f9d5f31113fd9e61926ba1170e23369e894c60276bf937a45384854e not found: ID does not exist" containerID="604e88a6f9d5f31113fd9e61926ba1170e23369e894c60276bf937a45384854e" Dec 03 12:34:39 crc kubenswrapper[5052]: I1203 12:34:39.840481 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"604e88a6f9d5f31113fd9e61926ba1170e23369e894c60276bf937a45384854e"} err="failed to get container status \"604e88a6f9d5f31113fd9e61926ba1170e23369e894c60276bf937a45384854e\": rpc error: code = NotFound desc = could not find container \"604e88a6f9d5f31113fd9e61926ba1170e23369e894c60276bf937a45384854e\": container with ID starting with 604e88a6f9d5f31113fd9e61926ba1170e23369e894c60276bf937a45384854e not found: ID does not exist" Dec 03 12:34:39 crc kubenswrapper[5052]: I1203 12:34:39.840524 5052 scope.go:117] "RemoveContainer" containerID="e3b4fe5b61ea893e2ab1fcac0c2d3a2ab7831afe1733b8ce810a8002ffa09adb" Dec 03 12:34:39 crc kubenswrapper[5052]: E1203 12:34:39.842080 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e3b4fe5b61ea893e2ab1fcac0c2d3a2ab7831afe1733b8ce810a8002ffa09adb\": container with ID starting with e3b4fe5b61ea893e2ab1fcac0c2d3a2ab7831afe1733b8ce810a8002ffa09adb not found: ID does not exist" containerID="e3b4fe5b61ea893e2ab1fcac0c2d3a2ab7831afe1733b8ce810a8002ffa09adb" Dec 03 12:34:39 crc kubenswrapper[5052]: I1203 12:34:39.842105 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e3b4fe5b61ea893e2ab1fcac0c2d3a2ab7831afe1733b8ce810a8002ffa09adb"} err="failed to get container status \"e3b4fe5b61ea893e2ab1fcac0c2d3a2ab7831afe1733b8ce810a8002ffa09adb\": rpc error: code = NotFound desc = could not find container \"e3b4fe5b61ea893e2ab1fcac0c2d3a2ab7831afe1733b8ce810a8002ffa09adb\": container with ID starting with e3b4fe5b61ea893e2ab1fcac0c2d3a2ab7831afe1733b8ce810a8002ffa09adb not found: ID does not exist" Dec 03 12:34:39 crc kubenswrapper[5052]: I1203 12:34:39.842119 5052 scope.go:117] "RemoveContainer" containerID="6ccc4a54d174219d793d5a774b64b4fc2e9d4e6aded52b78b522a8ccc368b7ba" Dec 03 12:34:39 crc kubenswrapper[5052]: E1203 12:34:39.842734 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6ccc4a54d174219d793d5a774b64b4fc2e9d4e6aded52b78b522a8ccc368b7ba\": container with ID starting with 6ccc4a54d174219d793d5a774b64b4fc2e9d4e6aded52b78b522a8ccc368b7ba not found: ID does not exist" containerID="6ccc4a54d174219d793d5a774b64b4fc2e9d4e6aded52b78b522a8ccc368b7ba" Dec 03 12:34:39 crc kubenswrapper[5052]: I1203 12:34:39.842759 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6ccc4a54d174219d793d5a774b64b4fc2e9d4e6aded52b78b522a8ccc368b7ba"} err="failed to get container status \"6ccc4a54d174219d793d5a774b64b4fc2e9d4e6aded52b78b522a8ccc368b7ba\": rpc error: code = NotFound desc = could not find container \"6ccc4a54d174219d793d5a774b64b4fc2e9d4e6aded52b78b522a8ccc368b7ba\": container with ID starting with 6ccc4a54d174219d793d5a774b64b4fc2e9d4e6aded52b78b522a8ccc368b7ba not found: ID does not exist" Dec 03 12:34:39 crc kubenswrapper[5052]: I1203 12:34:39.842771 5052 scope.go:117] "RemoveContainer" containerID="c03570ebb8b22d098415dfbe8a4b795a6bec4bf731ed35226369bf3a10c7348f" Dec 03 12:34:39 crc kubenswrapper[5052]: I1203 12:34:39.842980 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c03570ebb8b22d098415dfbe8a4b795a6bec4bf731ed35226369bf3a10c7348f"} err="failed to get container status \"c03570ebb8b22d098415dfbe8a4b795a6bec4bf731ed35226369bf3a10c7348f\": rpc error: code = NotFound desc = could not find container \"c03570ebb8b22d098415dfbe8a4b795a6bec4bf731ed35226369bf3a10c7348f\": container with ID starting with c03570ebb8b22d098415dfbe8a4b795a6bec4bf731ed35226369bf3a10c7348f not found: ID does not exist" Dec 03 12:34:39 crc kubenswrapper[5052]: I1203 12:34:39.842993 5052 scope.go:117] "RemoveContainer" containerID="604e88a6f9d5f31113fd9e61926ba1170e23369e894c60276bf937a45384854e" Dec 03 12:34:39 crc kubenswrapper[5052]: I1203 12:34:39.843270 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"604e88a6f9d5f31113fd9e61926ba1170e23369e894c60276bf937a45384854e"} err="failed to get container status \"604e88a6f9d5f31113fd9e61926ba1170e23369e894c60276bf937a45384854e\": rpc error: code = NotFound desc = could not find container \"604e88a6f9d5f31113fd9e61926ba1170e23369e894c60276bf937a45384854e\": container with ID starting with 604e88a6f9d5f31113fd9e61926ba1170e23369e894c60276bf937a45384854e not found: ID does not exist" Dec 03 12:34:39 crc kubenswrapper[5052]: I1203 12:34:39.843283 5052 scope.go:117] "RemoveContainer" containerID="e3b4fe5b61ea893e2ab1fcac0c2d3a2ab7831afe1733b8ce810a8002ffa09adb" Dec 03 12:34:39 crc kubenswrapper[5052]: I1203 12:34:39.843470 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e3b4fe5b61ea893e2ab1fcac0c2d3a2ab7831afe1733b8ce810a8002ffa09adb"} err="failed to get container status \"e3b4fe5b61ea893e2ab1fcac0c2d3a2ab7831afe1733b8ce810a8002ffa09adb\": rpc error: code = NotFound desc = could not find container \"e3b4fe5b61ea893e2ab1fcac0c2d3a2ab7831afe1733b8ce810a8002ffa09adb\": container with ID starting with e3b4fe5b61ea893e2ab1fcac0c2d3a2ab7831afe1733b8ce810a8002ffa09adb not found: ID does not exist" Dec 03 12:34:39 crc kubenswrapper[5052]: I1203 12:34:39.843491 5052 scope.go:117] "RemoveContainer" containerID="6ccc4a54d174219d793d5a774b64b4fc2e9d4e6aded52b78b522a8ccc368b7ba" Dec 03 12:34:39 crc kubenswrapper[5052]: I1203 12:34:39.846568 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6ccc4a54d174219d793d5a774b64b4fc2e9d4e6aded52b78b522a8ccc368b7ba"} err="failed to get container status \"6ccc4a54d174219d793d5a774b64b4fc2e9d4e6aded52b78b522a8ccc368b7ba\": rpc error: code = NotFound desc = could not find container \"6ccc4a54d174219d793d5a774b64b4fc2e9d4e6aded52b78b522a8ccc368b7ba\": container with ID starting with 6ccc4a54d174219d793d5a774b64b4fc2e9d4e6aded52b78b522a8ccc368b7ba not found: ID does not exist" Dec 03 12:34:39 crc kubenswrapper[5052]: I1203 12:34:39.846625 5052 scope.go:117] "RemoveContainer" containerID="c03570ebb8b22d098415dfbe8a4b795a6bec4bf731ed35226369bf3a10c7348f" Dec 03 12:34:39 crc kubenswrapper[5052]: I1203 12:34:39.847365 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c03570ebb8b22d098415dfbe8a4b795a6bec4bf731ed35226369bf3a10c7348f"} err="failed to get container status \"c03570ebb8b22d098415dfbe8a4b795a6bec4bf731ed35226369bf3a10c7348f\": rpc error: code = NotFound desc = could not find container \"c03570ebb8b22d098415dfbe8a4b795a6bec4bf731ed35226369bf3a10c7348f\": container with ID starting with c03570ebb8b22d098415dfbe8a4b795a6bec4bf731ed35226369bf3a10c7348f not found: ID does not exist" Dec 03 12:34:39 crc kubenswrapper[5052]: I1203 12:34:39.847390 5052 scope.go:117] "RemoveContainer" containerID="604e88a6f9d5f31113fd9e61926ba1170e23369e894c60276bf937a45384854e" Dec 03 12:34:39 crc kubenswrapper[5052]: I1203 12:34:39.849683 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"604e88a6f9d5f31113fd9e61926ba1170e23369e894c60276bf937a45384854e"} err="failed to get container status \"604e88a6f9d5f31113fd9e61926ba1170e23369e894c60276bf937a45384854e\": rpc error: code = NotFound desc = could not find container \"604e88a6f9d5f31113fd9e61926ba1170e23369e894c60276bf937a45384854e\": container with ID starting with 604e88a6f9d5f31113fd9e61926ba1170e23369e894c60276bf937a45384854e not found: ID does not exist" Dec 03 12:34:39 crc kubenswrapper[5052]: I1203 12:34:39.849706 5052 scope.go:117] "RemoveContainer" containerID="e3b4fe5b61ea893e2ab1fcac0c2d3a2ab7831afe1733b8ce810a8002ffa09adb" Dec 03 12:34:39 crc kubenswrapper[5052]: I1203 12:34:39.855313 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:34:39 crc kubenswrapper[5052]: I1203 12:34:39.857705 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e3b4fe5b61ea893e2ab1fcac0c2d3a2ab7831afe1733b8ce810a8002ffa09adb"} err="failed to get container status \"e3b4fe5b61ea893e2ab1fcac0c2d3a2ab7831afe1733b8ce810a8002ffa09adb\": rpc error: code = NotFound desc = could not find container \"e3b4fe5b61ea893e2ab1fcac0c2d3a2ab7831afe1733b8ce810a8002ffa09adb\": container with ID starting with e3b4fe5b61ea893e2ab1fcac0c2d3a2ab7831afe1733b8ce810a8002ffa09adb not found: ID does not exist" Dec 03 12:34:39 crc kubenswrapper[5052]: I1203 12:34:39.857745 5052 scope.go:117] "RemoveContainer" containerID="6ccc4a54d174219d793d5a774b64b4fc2e9d4e6aded52b78b522a8ccc368b7ba" Dec 03 12:34:39 crc kubenswrapper[5052]: I1203 12:34:39.859975 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6ccc4a54d174219d793d5a774b64b4fc2e9d4e6aded52b78b522a8ccc368b7ba"} err="failed to get container status \"6ccc4a54d174219d793d5a774b64b4fc2e9d4e6aded52b78b522a8ccc368b7ba\": rpc error: code = NotFound desc = could not find container \"6ccc4a54d174219d793d5a774b64b4fc2e9d4e6aded52b78b522a8ccc368b7ba\": container with ID starting with 6ccc4a54d174219d793d5a774b64b4fc2e9d4e6aded52b78b522a8ccc368b7ba not found: ID does not exist" Dec 03 12:34:39 crc kubenswrapper[5052]: I1203 12:34:39.860025 5052 scope.go:117] "RemoveContainer" containerID="c03570ebb8b22d098415dfbe8a4b795a6bec4bf731ed35226369bf3a10c7348f" Dec 03 12:34:39 crc kubenswrapper[5052]: I1203 12:34:39.860300 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c03570ebb8b22d098415dfbe8a4b795a6bec4bf731ed35226369bf3a10c7348f"} err="failed to get container status \"c03570ebb8b22d098415dfbe8a4b795a6bec4bf731ed35226369bf3a10c7348f\": rpc error: code = NotFound desc = could not find container \"c03570ebb8b22d098415dfbe8a4b795a6bec4bf731ed35226369bf3a10c7348f\": container with ID starting with c03570ebb8b22d098415dfbe8a4b795a6bec4bf731ed35226369bf3a10c7348f not found: ID does not exist" Dec 03 12:34:39 crc kubenswrapper[5052]: I1203 12:34:39.860325 5052 scope.go:117] "RemoveContainer" containerID="604e88a6f9d5f31113fd9e61926ba1170e23369e894c60276bf937a45384854e" Dec 03 12:34:39 crc kubenswrapper[5052]: I1203 12:34:39.860632 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"604e88a6f9d5f31113fd9e61926ba1170e23369e894c60276bf937a45384854e"} err="failed to get container status \"604e88a6f9d5f31113fd9e61926ba1170e23369e894c60276bf937a45384854e\": rpc error: code = NotFound desc = could not find container \"604e88a6f9d5f31113fd9e61926ba1170e23369e894c60276bf937a45384854e\": container with ID starting with 604e88a6f9d5f31113fd9e61926ba1170e23369e894c60276bf937a45384854e not found: ID does not exist" Dec 03 12:34:39 crc kubenswrapper[5052]: I1203 12:34:39.860659 5052 scope.go:117] "RemoveContainer" containerID="e3b4fe5b61ea893e2ab1fcac0c2d3a2ab7831afe1733b8ce810a8002ffa09adb" Dec 03 12:34:39 crc kubenswrapper[5052]: I1203 12:34:39.861373 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e3b4fe5b61ea893e2ab1fcac0c2d3a2ab7831afe1733b8ce810a8002ffa09adb"} err="failed to get container status \"e3b4fe5b61ea893e2ab1fcac0c2d3a2ab7831afe1733b8ce810a8002ffa09adb\": rpc error: code = NotFound desc = could not find container \"e3b4fe5b61ea893e2ab1fcac0c2d3a2ab7831afe1733b8ce810a8002ffa09adb\": container with ID starting with e3b4fe5b61ea893e2ab1fcac0c2d3a2ab7831afe1733b8ce810a8002ffa09adb not found: ID does not exist" Dec 03 12:34:39 crc kubenswrapper[5052]: I1203 12:34:39.861402 5052 scope.go:117] "RemoveContainer" containerID="6ccc4a54d174219d793d5a774b64b4fc2e9d4e6aded52b78b522a8ccc368b7ba" Dec 03 12:34:39 crc kubenswrapper[5052]: I1203 12:34:39.861717 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6ccc4a54d174219d793d5a774b64b4fc2e9d4e6aded52b78b522a8ccc368b7ba"} err="failed to get container status \"6ccc4a54d174219d793d5a774b64b4fc2e9d4e6aded52b78b522a8ccc368b7ba\": rpc error: code = NotFound desc = could not find container \"6ccc4a54d174219d793d5a774b64b4fc2e9d4e6aded52b78b522a8ccc368b7ba\": container with ID starting with 6ccc4a54d174219d793d5a774b64b4fc2e9d4e6aded52b78b522a8ccc368b7ba not found: ID does not exist" Dec 03 12:34:39 crc kubenswrapper[5052]: I1203 12:34:39.881593 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:34:39 crc kubenswrapper[5052]: I1203 12:34:39.904940 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:34:39 crc kubenswrapper[5052]: E1203 12:34:39.905315 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="6e18c0ba-8f07-4d54-b383-3d0164780725" containerName="sg-core" Dec 03 12:34:39 crc kubenswrapper[5052]: I1203 12:34:39.905338 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="6e18c0ba-8f07-4d54-b383-3d0164780725" containerName="sg-core" Dec 03 12:34:39 crc kubenswrapper[5052]: E1203 12:34:39.905356 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="6e18c0ba-8f07-4d54-b383-3d0164780725" containerName="ceilometer-central-agent" Dec 03 12:34:39 crc kubenswrapper[5052]: I1203 12:34:39.905364 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="6e18c0ba-8f07-4d54-b383-3d0164780725" containerName="ceilometer-central-agent" Dec 03 12:34:39 crc kubenswrapper[5052]: E1203 12:34:39.905387 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="6e18c0ba-8f07-4d54-b383-3d0164780725" containerName="ceilometer-notification-agent" Dec 03 12:34:39 crc kubenswrapper[5052]: I1203 12:34:39.905422 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="6e18c0ba-8f07-4d54-b383-3d0164780725" containerName="ceilometer-notification-agent" Dec 03 12:34:39 crc kubenswrapper[5052]: E1203 12:34:39.905472 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="6e18c0ba-8f07-4d54-b383-3d0164780725" containerName="proxy-httpd" Dec 03 12:34:39 crc kubenswrapper[5052]: I1203 12:34:39.905480 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="6e18c0ba-8f07-4d54-b383-3d0164780725" containerName="proxy-httpd" Dec 03 12:34:39 crc kubenswrapper[5052]: I1203 12:34:39.908500 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="6e18c0ba-8f07-4d54-b383-3d0164780725" containerName="ceilometer-notification-agent" Dec 03 12:34:39 crc kubenswrapper[5052]: I1203 12:34:39.908558 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="6e18c0ba-8f07-4d54-b383-3d0164780725" containerName="ceilometer-central-agent" Dec 03 12:34:39 crc kubenswrapper[5052]: I1203 12:34:39.908578 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="6e18c0ba-8f07-4d54-b383-3d0164780725" containerName="sg-core" Dec 03 12:34:39 crc kubenswrapper[5052]: I1203 12:34:39.908585 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="6e18c0ba-8f07-4d54-b383-3d0164780725" containerName="proxy-httpd" Dec 03 12:34:39 crc kubenswrapper[5052]: I1203 12:34:39.910952 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:34:39 crc kubenswrapper[5052]: I1203 12:34:39.914367 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-scripts" Dec 03 12:34:39 crc kubenswrapper[5052]: I1203 12:34:39.914506 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-config-data" Dec 03 12:34:39 crc kubenswrapper[5052]: I1203 12:34:39.917308 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-bootstrap-hvfxb" Dec 03 12:34:39 crc kubenswrapper[5052]: I1203 12:34:39.930567 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:34:40 crc kubenswrapper[5052]: I1203 12:34:40.065390 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-db-sync-fcpth" Dec 03 12:34:40 crc kubenswrapper[5052]: I1203 12:34:40.096950 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/80bd8d5d-a98f-455b-977a-e96317d7f525-scripts\") pod \"80bd8d5d-a98f-455b-977a-e96317d7f525\" (UID: \"80bd8d5d-a98f-455b-977a-e96317d7f525\") " Dec 03 12:34:40 crc kubenswrapper[5052]: I1203 12:34:40.097002 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/80bd8d5d-a98f-455b-977a-e96317d7f525-config-data\") pod \"80bd8d5d-a98f-455b-977a-e96317d7f525\" (UID: \"80bd8d5d-a98f-455b-977a-e96317d7f525\") " Dec 03 12:34:40 crc kubenswrapper[5052]: I1203 12:34:40.097070 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/80bd8d5d-a98f-455b-977a-e96317d7f525-fernet-keys\") pod \"80bd8d5d-a98f-455b-977a-e96317d7f525\" (UID: \"80bd8d5d-a98f-455b-977a-e96317d7f525\") " Dec 03 12:34:40 crc kubenswrapper[5052]: I1203 12:34:40.097094 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/80bd8d5d-a98f-455b-977a-e96317d7f525-credential-keys\") pod \"80bd8d5d-a98f-455b-977a-e96317d7f525\" (UID: \"80bd8d5d-a98f-455b-977a-e96317d7f525\") " Dec 03 12:34:40 crc kubenswrapper[5052]: I1203 12:34:40.097155 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/80bd8d5d-a98f-455b-977a-e96317d7f525-combined-ca-bundle\") pod \"80bd8d5d-a98f-455b-977a-e96317d7f525\" (UID: \"80bd8d5d-a98f-455b-977a-e96317d7f525\") " Dec 03 12:34:40 crc kubenswrapper[5052]: I1203 12:34:40.097192 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-82qfd\" (UniqueName: \"kubernetes.io/projected/80bd8d5d-a98f-455b-977a-e96317d7f525-kube-api-access-82qfd\") pod \"80bd8d5d-a98f-455b-977a-e96317d7f525\" (UID: \"80bd8d5d-a98f-455b-977a-e96317d7f525\") " Dec 03 12:34:40 crc kubenswrapper[5052]: I1203 12:34:40.097411 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/f45ca29a-a9fb-4644-be06-f728cc56d453-scripts\") pod \"ceilometer-0\" (UID: \"f45ca29a-a9fb-4644-be06-f728cc56d453\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:34:40 crc kubenswrapper[5052]: I1203 12:34:40.097463 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/f45ca29a-a9fb-4644-be06-f728cc56d453-run-httpd\") pod \"ceilometer-0\" (UID: \"f45ca29a-a9fb-4644-be06-f728cc56d453\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:34:40 crc kubenswrapper[5052]: I1203 12:34:40.097488 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w52f7\" (UniqueName: \"kubernetes.io/projected/f45ca29a-a9fb-4644-be06-f728cc56d453-kube-api-access-w52f7\") pod \"ceilometer-0\" (UID: \"f45ca29a-a9fb-4644-be06-f728cc56d453\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:34:40 crc kubenswrapper[5052]: I1203 12:34:40.097561 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/f45ca29a-a9fb-4644-be06-f728cc56d453-log-httpd\") pod \"ceilometer-0\" (UID: \"f45ca29a-a9fb-4644-be06-f728cc56d453\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:34:40 crc kubenswrapper[5052]: I1203 12:34:40.097591 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/f45ca29a-a9fb-4644-be06-f728cc56d453-config-data\") pod \"ceilometer-0\" (UID: \"f45ca29a-a9fb-4644-be06-f728cc56d453\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:34:40 crc kubenswrapper[5052]: I1203 12:34:40.097621 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f45ca29a-a9fb-4644-be06-f728cc56d453-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"f45ca29a-a9fb-4644-be06-f728cc56d453\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:34:40 crc kubenswrapper[5052]: I1203 12:34:40.097646 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/f45ca29a-a9fb-4644-be06-f728cc56d453-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"f45ca29a-a9fb-4644-be06-f728cc56d453\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:34:40 crc kubenswrapper[5052]: I1203 12:34:40.102028 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/80bd8d5d-a98f-455b-977a-e96317d7f525-credential-keys" (OuterVolumeSpecName: "credential-keys") pod "80bd8d5d-a98f-455b-977a-e96317d7f525" (UID: "80bd8d5d-a98f-455b-977a-e96317d7f525"). InnerVolumeSpecName "credential-keys". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:34:40 crc kubenswrapper[5052]: I1203 12:34:40.102039 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/80bd8d5d-a98f-455b-977a-e96317d7f525-scripts" (OuterVolumeSpecName: "scripts") pod "80bd8d5d-a98f-455b-977a-e96317d7f525" (UID: "80bd8d5d-a98f-455b-977a-e96317d7f525"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:34:40 crc kubenswrapper[5052]: I1203 12:34:40.102234 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/80bd8d5d-a98f-455b-977a-e96317d7f525-fernet-keys" (OuterVolumeSpecName: "fernet-keys") pod "80bd8d5d-a98f-455b-977a-e96317d7f525" (UID: "80bd8d5d-a98f-455b-977a-e96317d7f525"). InnerVolumeSpecName "fernet-keys". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:34:40 crc kubenswrapper[5052]: I1203 12:34:40.110052 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/80bd8d5d-a98f-455b-977a-e96317d7f525-kube-api-access-82qfd" (OuterVolumeSpecName: "kube-api-access-82qfd") pod "80bd8d5d-a98f-455b-977a-e96317d7f525" (UID: "80bd8d5d-a98f-455b-977a-e96317d7f525"). InnerVolumeSpecName "kube-api-access-82qfd". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:34:40 crc kubenswrapper[5052]: I1203 12:34:40.125103 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/80bd8d5d-a98f-455b-977a-e96317d7f525-config-data" (OuterVolumeSpecName: "config-data") pod "80bd8d5d-a98f-455b-977a-e96317d7f525" (UID: "80bd8d5d-a98f-455b-977a-e96317d7f525"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:34:40 crc kubenswrapper[5052]: I1203 12:34:40.138134 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/80bd8d5d-a98f-455b-977a-e96317d7f525-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "80bd8d5d-a98f-455b-977a-e96317d7f525" (UID: "80bd8d5d-a98f-455b-977a-e96317d7f525"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:34:40 crc kubenswrapper[5052]: I1203 12:34:40.200132 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a4de7484-329f-412e-b365-e373a4cab966-combined-ca-bundle\") pod \"a4de7484-329f-412e-b365-e373a4cab966\" (UID: \"a4de7484-329f-412e-b365-e373a4cab966\") " Dec 03 12:34:40 crc kubenswrapper[5052]: I1203 12:34:40.200224 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a4de7484-329f-412e-b365-e373a4cab966-config-data\") pod \"a4de7484-329f-412e-b365-e373a4cab966\" (UID: \"a4de7484-329f-412e-b365-e373a4cab966\") " Dec 03 12:34:40 crc kubenswrapper[5052]: I1203 12:34:40.200850 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/a4de7484-329f-412e-b365-e373a4cab966-logs\") pod \"a4de7484-329f-412e-b365-e373a4cab966\" (UID: \"a4de7484-329f-412e-b365-e373a4cab966\") " Dec 03 12:34:40 crc kubenswrapper[5052]: I1203 12:34:40.201036 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/a4de7484-329f-412e-b365-e373a4cab966-scripts\") pod \"a4de7484-329f-412e-b365-e373a4cab966\" (UID: \"a4de7484-329f-412e-b365-e373a4cab966\") " Dec 03 12:34:40 crc kubenswrapper[5052]: I1203 12:34:40.201069 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fhbjq\" (UniqueName: \"kubernetes.io/projected/a4de7484-329f-412e-b365-e373a4cab966-kube-api-access-fhbjq\") pod \"a4de7484-329f-412e-b365-e373a4cab966\" (UID: \"a4de7484-329f-412e-b365-e373a4cab966\") " Dec 03 12:34:40 crc kubenswrapper[5052]: I1203 12:34:40.201366 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4de7484-329f-412e-b365-e373a4cab966-logs" (OuterVolumeSpecName: "logs") pod "a4de7484-329f-412e-b365-e373a4cab966" (UID: "a4de7484-329f-412e-b365-e373a4cab966"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:34:40 crc kubenswrapper[5052]: I1203 12:34:40.202158 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f45ca29a-a9fb-4644-be06-f728cc56d453-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"f45ca29a-a9fb-4644-be06-f728cc56d453\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:34:40 crc kubenswrapper[5052]: I1203 12:34:40.202368 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/f45ca29a-a9fb-4644-be06-f728cc56d453-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"f45ca29a-a9fb-4644-be06-f728cc56d453\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:34:40 crc kubenswrapper[5052]: I1203 12:34:40.202629 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/f45ca29a-a9fb-4644-be06-f728cc56d453-scripts\") pod \"ceilometer-0\" (UID: \"f45ca29a-a9fb-4644-be06-f728cc56d453\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:34:40 crc kubenswrapper[5052]: I1203 12:34:40.202997 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/f45ca29a-a9fb-4644-be06-f728cc56d453-run-httpd\") pod \"ceilometer-0\" (UID: \"f45ca29a-a9fb-4644-be06-f728cc56d453\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:34:40 crc kubenswrapper[5052]: I1203 12:34:40.203153 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-w52f7\" (UniqueName: \"kubernetes.io/projected/f45ca29a-a9fb-4644-be06-f728cc56d453-kube-api-access-w52f7\") pod \"ceilometer-0\" (UID: \"f45ca29a-a9fb-4644-be06-f728cc56d453\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:34:40 crc kubenswrapper[5052]: I1203 12:34:40.203927 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/f45ca29a-a9fb-4644-be06-f728cc56d453-run-httpd\") pod \"ceilometer-0\" (UID: \"f45ca29a-a9fb-4644-be06-f728cc56d453\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:34:40 crc kubenswrapper[5052]: I1203 12:34:40.205885 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/f45ca29a-a9fb-4644-be06-f728cc56d453-log-httpd\") pod \"ceilometer-0\" (UID: \"f45ca29a-a9fb-4644-be06-f728cc56d453\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:34:40 crc kubenswrapper[5052]: I1203 12:34:40.206044 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a4de7484-329f-412e-b365-e373a4cab966-scripts" (OuterVolumeSpecName: "scripts") pod "a4de7484-329f-412e-b365-e373a4cab966" (UID: "a4de7484-329f-412e-b365-e373a4cab966"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:34:40 crc kubenswrapper[5052]: I1203 12:34:40.206245 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/f45ca29a-a9fb-4644-be06-f728cc56d453-config-data\") pod \"ceilometer-0\" (UID: \"f45ca29a-a9fb-4644-be06-f728cc56d453\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:34:40 crc kubenswrapper[5052]: I1203 12:34:40.206540 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-82qfd\" (UniqueName: \"kubernetes.io/projected/80bd8d5d-a98f-455b-977a-e96317d7f525-kube-api-access-82qfd\") on node \"crc\" DevicePath \"\"" Dec 03 12:34:40 crc kubenswrapper[5052]: I1203 12:34:40.206690 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/a4de7484-329f-412e-b365-e373a4cab966-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:34:40 crc kubenswrapper[5052]: I1203 12:34:40.209962 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/80bd8d5d-a98f-455b-977a-e96317d7f525-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:34:40 crc kubenswrapper[5052]: I1203 12:34:40.210194 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/80bd8d5d-a98f-455b-977a-e96317d7f525-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:34:40 crc kubenswrapper[5052]: I1203 12:34:40.210337 5052 reconciler_common.go:293] "Volume detached for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/80bd8d5d-a98f-455b-977a-e96317d7f525-fernet-keys\") on node \"crc\" DevicePath \"\"" Dec 03 12:34:40 crc kubenswrapper[5052]: I1203 12:34:40.210442 5052 reconciler_common.go:293] "Volume detached for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/80bd8d5d-a98f-455b-977a-e96317d7f525-credential-keys\") on node \"crc\" DevicePath \"\"" Dec 03 12:34:40 crc kubenswrapper[5052]: I1203 12:34:40.210563 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/80bd8d5d-a98f-455b-977a-e96317d7f525-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:34:40 crc kubenswrapper[5052]: I1203 12:34:40.210665 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/a4de7484-329f-412e-b365-e373a4cab966-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:34:40 crc kubenswrapper[5052]: I1203 12:34:40.206904 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/f45ca29a-a9fb-4644-be06-f728cc56d453-log-httpd\") pod \"ceilometer-0\" (UID: \"f45ca29a-a9fb-4644-be06-f728cc56d453\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:34:40 crc kubenswrapper[5052]: I1203 12:34:40.210923 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/f45ca29a-a9fb-4644-be06-f728cc56d453-config-data\") pod \"ceilometer-0\" (UID: \"f45ca29a-a9fb-4644-be06-f728cc56d453\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:34:40 crc kubenswrapper[5052]: I1203 12:34:40.216653 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a4de7484-329f-412e-b365-e373a4cab966-kube-api-access-fhbjq" (OuterVolumeSpecName: "kube-api-access-fhbjq") pod "a4de7484-329f-412e-b365-e373a4cab966" (UID: "a4de7484-329f-412e-b365-e373a4cab966"). InnerVolumeSpecName "kube-api-access-fhbjq". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:34:40 crc kubenswrapper[5052]: I1203 12:34:40.223337 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/f45ca29a-a9fb-4644-be06-f728cc56d453-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"f45ca29a-a9fb-4644-be06-f728cc56d453\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:34:40 crc kubenswrapper[5052]: I1203 12:34:40.223394 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f45ca29a-a9fb-4644-be06-f728cc56d453-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"f45ca29a-a9fb-4644-be06-f728cc56d453\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:34:40 crc kubenswrapper[5052]: I1203 12:34:40.223976 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-w52f7\" (UniqueName: \"kubernetes.io/projected/f45ca29a-a9fb-4644-be06-f728cc56d453-kube-api-access-w52f7\") pod \"ceilometer-0\" (UID: \"f45ca29a-a9fb-4644-be06-f728cc56d453\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:34:40 crc kubenswrapper[5052]: I1203 12:34:40.226277 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/f45ca29a-a9fb-4644-be06-f728cc56d453-scripts\") pod \"ceilometer-0\" (UID: \"f45ca29a-a9fb-4644-be06-f728cc56d453\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:34:40 crc kubenswrapper[5052]: I1203 12:34:40.227548 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a4de7484-329f-412e-b365-e373a4cab966-config-data" (OuterVolumeSpecName: "config-data") pod "a4de7484-329f-412e-b365-e373a4cab966" (UID: "a4de7484-329f-412e-b365-e373a4cab966"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:34:40 crc kubenswrapper[5052]: I1203 12:34:40.233359 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a4de7484-329f-412e-b365-e373a4cab966-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "a4de7484-329f-412e-b365-e373a4cab966" (UID: "a4de7484-329f-412e-b365-e373a4cab966"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:34:40 crc kubenswrapper[5052]: I1203 12:34:40.240989 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:34:40 crc kubenswrapper[5052]: I1203 12:34:40.312623 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-fhbjq\" (UniqueName: \"kubernetes.io/projected/a4de7484-329f-412e-b365-e373a4cab966-kube-api-access-fhbjq\") on node \"crc\" DevicePath \"\"" Dec 03 12:34:40 crc kubenswrapper[5052]: I1203 12:34:40.312651 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a4de7484-329f-412e-b365-e373a4cab966-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:34:40 crc kubenswrapper[5052]: I1203 12:34:40.312661 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a4de7484-329f-412e-b365-e373a4cab966-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:34:40 crc kubenswrapper[5052]: I1203 12:34:40.465871 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"25498b02-c179-4555-ad04-1ef6445df4ab","Type":"ContainerStarted","Data":"b1eb3705336137fdb58a75000976f8917d3ae6bc93b6459c35f5a5a4494f5dc5"} Dec 03 12:34:40 crc kubenswrapper[5052]: I1203 12:34:40.466232 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"25498b02-c179-4555-ad04-1ef6445df4ab","Type":"ContainerStarted","Data":"32653e85f448eef3d2da7d0786700be61d651c987dd3d9b0c1da81c0cd2bfba5"} Dec 03 12:34:40 crc kubenswrapper[5052]: I1203 12:34:40.469067 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-db-sync-fcpth" Dec 03 12:34:40 crc kubenswrapper[5052]: I1203 12:34:40.469119 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-db-sync-fcpth" event={"ID":"a4de7484-329f-412e-b365-e373a4cab966","Type":"ContainerDied","Data":"5d3f626046e9992f492882d32dd038cdb6e5114c45ef88706662380b3f0fbab7"} Dec 03 12:34:40 crc kubenswrapper[5052]: I1203 12:34:40.469174 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5d3f626046e9992f492882d32dd038cdb6e5114c45ef88706662380b3f0fbab7" Dec 03 12:34:40 crc kubenswrapper[5052]: I1203 12:34:40.472675 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"6e96674e-4a20-4d8f-9118-90532fa75788","Type":"ContainerStarted","Data":"ddcff7ae2062661e408d5c15760d3bc57eb9460bcc46af6105ea25f9411be361"} Dec 03 12:34:40 crc kubenswrapper[5052]: I1203 12:34:40.472750 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"6e96674e-4a20-4d8f-9118-90532fa75788","Type":"ContainerStarted","Data":"e958eed2d4d943b337051a8a21db34d2de99782e4a9056908e19e538adaf56f9"} Dec 03 12:34:40 crc kubenswrapper[5052]: I1203 12:34:40.479218 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-bootstrap-hvfxb" event={"ID":"80bd8d5d-a98f-455b-977a-e96317d7f525","Type":"ContainerDied","Data":"18897b1cf1b0797253e333d8b9d11c44cfe74e0ff25736ac787bfde87f18d1c8"} Dec 03 12:34:40 crc kubenswrapper[5052]: I1203 12:34:40.479292 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="18897b1cf1b0797253e333d8b9d11c44cfe74e0ff25736ac787bfde87f18d1c8" Dec 03 12:34:40 crc kubenswrapper[5052]: I1203 12:34:40.479473 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-bootstrap-hvfxb" Dec 03 12:34:40 crc kubenswrapper[5052]: I1203 12:34:40.558446 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/keystone-bootstrap-hvfxb"] Dec 03 12:34:40 crc kubenswrapper[5052]: I1203 12:34:40.567259 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/keystone-bootstrap-hvfxb"] Dec 03 12:34:40 crc kubenswrapper[5052]: I1203 12:34:40.679601 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/keystone-bootstrap-9grd9"] Dec 03 12:34:40 crc kubenswrapper[5052]: E1203 12:34:40.680438 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="80bd8d5d-a98f-455b-977a-e96317d7f525" containerName="keystone-bootstrap" Dec 03 12:34:40 crc kubenswrapper[5052]: I1203 12:34:40.680455 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="80bd8d5d-a98f-455b-977a-e96317d7f525" containerName="keystone-bootstrap" Dec 03 12:34:40 crc kubenswrapper[5052]: E1203 12:34:40.680498 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="a4de7484-329f-412e-b365-e373a4cab966" containerName="placement-db-sync" Dec 03 12:34:40 crc kubenswrapper[5052]: I1203 12:34:40.680507 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="a4de7484-329f-412e-b365-e373a4cab966" containerName="placement-db-sync" Dec 03 12:34:40 crc kubenswrapper[5052]: I1203 12:34:40.680732 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="a4de7484-329f-412e-b365-e373a4cab966" containerName="placement-db-sync" Dec 03 12:34:40 crc kubenswrapper[5052]: I1203 12:34:40.680750 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="80bd8d5d-a98f-455b-977a-e96317d7f525" containerName="keystone-bootstrap" Dec 03 12:34:40 crc kubenswrapper[5052]: I1203 12:34:40.681534 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-bootstrap-9grd9" Dec 03 12:34:40 crc kubenswrapper[5052]: I1203 12:34:40.684347 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone-keystone-dockercfg-fvfwv" Dec 03 12:34:40 crc kubenswrapper[5052]: I1203 12:34:40.684689 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone" Dec 03 12:34:40 crc kubenswrapper[5052]: I1203 12:34:40.684803 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone-scripts" Dec 03 12:34:40 crc kubenswrapper[5052]: I1203 12:34:40.684940 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"osp-secret" Dec 03 12:34:40 crc kubenswrapper[5052]: I1203 12:34:40.687684 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone-config-data" Dec 03 12:34:40 crc kubenswrapper[5052]: I1203 12:34:40.697051 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/placement-6cd995dc74-hpvzm"] Dec 03 12:34:40 crc kubenswrapper[5052]: I1203 12:34:40.698823 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-6cd995dc74-hpvzm" Dec 03 12:34:40 crc kubenswrapper[5052]: I1203 12:34:40.701134 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"placement-placement-dockercfg-642c9" Dec 03 12:34:40 crc kubenswrapper[5052]: I1203 12:34:40.710484 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-placement-internal-svc" Dec 03 12:34:40 crc kubenswrapper[5052]: I1203 12:34:40.710541 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"placement-scripts" Dec 03 12:34:40 crc kubenswrapper[5052]: I1203 12:34:40.710705 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-placement-public-svc" Dec 03 12:34:40 crc kubenswrapper[5052]: I1203 12:34:40.710813 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"placement-config-data" Dec 03 12:34:40 crc kubenswrapper[5052]: I1203 12:34:40.715391 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-bootstrap-9grd9"] Dec 03 12:34:40 crc kubenswrapper[5052]: I1203 12:34:40.724589 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/placement-6cd995dc74-hpvzm"] Dec 03 12:34:40 crc kubenswrapper[5052]: I1203 12:34:40.739549 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/38bfd61a-669e-4d69-b667-69270b5ec99d-config-data\") pod \"placement-6cd995dc74-hpvzm\" (UID: \"38bfd61a-669e-4d69-b667-69270b5ec99d\") " pod="openstack-kuttl-tests/placement-6cd995dc74-hpvzm" Dec 03 12:34:40 crc kubenswrapper[5052]: I1203 12:34:40.739605 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/38bfd61a-669e-4d69-b667-69270b5ec99d-public-tls-certs\") pod \"placement-6cd995dc74-hpvzm\" (UID: \"38bfd61a-669e-4d69-b667-69270b5ec99d\") " pod="openstack-kuttl-tests/placement-6cd995dc74-hpvzm" Dec 03 12:34:40 crc kubenswrapper[5052]: I1203 12:34:40.739633 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/0a3ff09e-7256-4090-9060-099706213333-credential-keys\") pod \"keystone-bootstrap-9grd9\" (UID: \"0a3ff09e-7256-4090-9060-099706213333\") " pod="openstack-kuttl-tests/keystone-bootstrap-9grd9" Dec 03 12:34:40 crc kubenswrapper[5052]: I1203 12:34:40.739657 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0a3ff09e-7256-4090-9060-099706213333-combined-ca-bundle\") pod \"keystone-bootstrap-9grd9\" (UID: \"0a3ff09e-7256-4090-9060-099706213333\") " pod="openstack-kuttl-tests/keystone-bootstrap-9grd9" Dec 03 12:34:40 crc kubenswrapper[5052]: I1203 12:34:40.739702 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nvhjx\" (UniqueName: \"kubernetes.io/projected/38bfd61a-669e-4d69-b667-69270b5ec99d-kube-api-access-nvhjx\") pod \"placement-6cd995dc74-hpvzm\" (UID: \"38bfd61a-669e-4d69-b667-69270b5ec99d\") " pod="openstack-kuttl-tests/placement-6cd995dc74-hpvzm" Dec 03 12:34:40 crc kubenswrapper[5052]: I1203 12:34:40.739723 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/38bfd61a-669e-4d69-b667-69270b5ec99d-internal-tls-certs\") pod \"placement-6cd995dc74-hpvzm\" (UID: \"38bfd61a-669e-4d69-b667-69270b5ec99d\") " pod="openstack-kuttl-tests/placement-6cd995dc74-hpvzm" Dec 03 12:34:40 crc kubenswrapper[5052]: I1203 12:34:40.739759 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/0a3ff09e-7256-4090-9060-099706213333-config-data\") pod \"keystone-bootstrap-9grd9\" (UID: \"0a3ff09e-7256-4090-9060-099706213333\") " pod="openstack-kuttl-tests/keystone-bootstrap-9grd9" Dec 03 12:34:40 crc kubenswrapper[5052]: I1203 12:34:40.739779 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/38bfd61a-669e-4d69-b667-69270b5ec99d-combined-ca-bundle\") pod \"placement-6cd995dc74-hpvzm\" (UID: \"38bfd61a-669e-4d69-b667-69270b5ec99d\") " pod="openstack-kuttl-tests/placement-6cd995dc74-hpvzm" Dec 03 12:34:40 crc kubenswrapper[5052]: I1203 12:34:40.739824 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/0a3ff09e-7256-4090-9060-099706213333-scripts\") pod \"keystone-bootstrap-9grd9\" (UID: \"0a3ff09e-7256-4090-9060-099706213333\") " pod="openstack-kuttl-tests/keystone-bootstrap-9grd9" Dec 03 12:34:40 crc kubenswrapper[5052]: I1203 12:34:40.739843 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/38bfd61a-669e-4d69-b667-69270b5ec99d-logs\") pod \"placement-6cd995dc74-hpvzm\" (UID: \"38bfd61a-669e-4d69-b667-69270b5ec99d\") " pod="openstack-kuttl-tests/placement-6cd995dc74-hpvzm" Dec 03 12:34:40 crc kubenswrapper[5052]: I1203 12:34:40.739893 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/0a3ff09e-7256-4090-9060-099706213333-fernet-keys\") pod \"keystone-bootstrap-9grd9\" (UID: \"0a3ff09e-7256-4090-9060-099706213333\") " pod="openstack-kuttl-tests/keystone-bootstrap-9grd9" Dec 03 12:34:40 crc kubenswrapper[5052]: I1203 12:34:40.739936 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/38bfd61a-669e-4d69-b667-69270b5ec99d-scripts\") pod \"placement-6cd995dc74-hpvzm\" (UID: \"38bfd61a-669e-4d69-b667-69270b5ec99d\") " pod="openstack-kuttl-tests/placement-6cd995dc74-hpvzm" Dec 03 12:34:40 crc kubenswrapper[5052]: I1203 12:34:40.739951 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q2h8j\" (UniqueName: \"kubernetes.io/projected/0a3ff09e-7256-4090-9060-099706213333-kube-api-access-q2h8j\") pod \"keystone-bootstrap-9grd9\" (UID: \"0a3ff09e-7256-4090-9060-099706213333\") " pod="openstack-kuttl-tests/keystone-bootstrap-9grd9" Dec 03 12:34:40 crc kubenswrapper[5052]: I1203 12:34:40.781692 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:34:40 crc kubenswrapper[5052]: I1203 12:34:40.841900 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/0a3ff09e-7256-4090-9060-099706213333-fernet-keys\") pod \"keystone-bootstrap-9grd9\" (UID: \"0a3ff09e-7256-4090-9060-099706213333\") " pod="openstack-kuttl-tests/keystone-bootstrap-9grd9" Dec 03 12:34:40 crc kubenswrapper[5052]: I1203 12:34:40.841972 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/38bfd61a-669e-4d69-b667-69270b5ec99d-scripts\") pod \"placement-6cd995dc74-hpvzm\" (UID: \"38bfd61a-669e-4d69-b667-69270b5ec99d\") " pod="openstack-kuttl-tests/placement-6cd995dc74-hpvzm" Dec 03 12:34:40 crc kubenswrapper[5052]: I1203 12:34:40.842006 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-q2h8j\" (UniqueName: \"kubernetes.io/projected/0a3ff09e-7256-4090-9060-099706213333-kube-api-access-q2h8j\") pod \"keystone-bootstrap-9grd9\" (UID: \"0a3ff09e-7256-4090-9060-099706213333\") " pod="openstack-kuttl-tests/keystone-bootstrap-9grd9" Dec 03 12:34:40 crc kubenswrapper[5052]: I1203 12:34:40.842031 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/38bfd61a-669e-4d69-b667-69270b5ec99d-config-data\") pod \"placement-6cd995dc74-hpvzm\" (UID: \"38bfd61a-669e-4d69-b667-69270b5ec99d\") " pod="openstack-kuttl-tests/placement-6cd995dc74-hpvzm" Dec 03 12:34:40 crc kubenswrapper[5052]: I1203 12:34:40.842060 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/38bfd61a-669e-4d69-b667-69270b5ec99d-public-tls-certs\") pod \"placement-6cd995dc74-hpvzm\" (UID: \"38bfd61a-669e-4d69-b667-69270b5ec99d\") " pod="openstack-kuttl-tests/placement-6cd995dc74-hpvzm" Dec 03 12:34:40 crc kubenswrapper[5052]: I1203 12:34:40.842087 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/0a3ff09e-7256-4090-9060-099706213333-credential-keys\") pod \"keystone-bootstrap-9grd9\" (UID: \"0a3ff09e-7256-4090-9060-099706213333\") " pod="openstack-kuttl-tests/keystone-bootstrap-9grd9" Dec 03 12:34:40 crc kubenswrapper[5052]: I1203 12:34:40.842117 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0a3ff09e-7256-4090-9060-099706213333-combined-ca-bundle\") pod \"keystone-bootstrap-9grd9\" (UID: \"0a3ff09e-7256-4090-9060-099706213333\") " pod="openstack-kuttl-tests/keystone-bootstrap-9grd9" Dec 03 12:34:40 crc kubenswrapper[5052]: I1203 12:34:40.842149 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-nvhjx\" (UniqueName: \"kubernetes.io/projected/38bfd61a-669e-4d69-b667-69270b5ec99d-kube-api-access-nvhjx\") pod \"placement-6cd995dc74-hpvzm\" (UID: \"38bfd61a-669e-4d69-b667-69270b5ec99d\") " pod="openstack-kuttl-tests/placement-6cd995dc74-hpvzm" Dec 03 12:34:40 crc kubenswrapper[5052]: I1203 12:34:40.842182 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/38bfd61a-669e-4d69-b667-69270b5ec99d-internal-tls-certs\") pod \"placement-6cd995dc74-hpvzm\" (UID: \"38bfd61a-669e-4d69-b667-69270b5ec99d\") " pod="openstack-kuttl-tests/placement-6cd995dc74-hpvzm" Dec 03 12:34:40 crc kubenswrapper[5052]: I1203 12:34:40.842223 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/0a3ff09e-7256-4090-9060-099706213333-config-data\") pod \"keystone-bootstrap-9grd9\" (UID: \"0a3ff09e-7256-4090-9060-099706213333\") " pod="openstack-kuttl-tests/keystone-bootstrap-9grd9" Dec 03 12:34:40 crc kubenswrapper[5052]: I1203 12:34:40.842252 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/38bfd61a-669e-4d69-b667-69270b5ec99d-combined-ca-bundle\") pod \"placement-6cd995dc74-hpvzm\" (UID: \"38bfd61a-669e-4d69-b667-69270b5ec99d\") " pod="openstack-kuttl-tests/placement-6cd995dc74-hpvzm" Dec 03 12:34:40 crc kubenswrapper[5052]: I1203 12:34:40.842292 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/0a3ff09e-7256-4090-9060-099706213333-scripts\") pod \"keystone-bootstrap-9grd9\" (UID: \"0a3ff09e-7256-4090-9060-099706213333\") " pod="openstack-kuttl-tests/keystone-bootstrap-9grd9" Dec 03 12:34:40 crc kubenswrapper[5052]: I1203 12:34:40.842322 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/38bfd61a-669e-4d69-b667-69270b5ec99d-logs\") pod \"placement-6cd995dc74-hpvzm\" (UID: \"38bfd61a-669e-4d69-b667-69270b5ec99d\") " pod="openstack-kuttl-tests/placement-6cd995dc74-hpvzm" Dec 03 12:34:40 crc kubenswrapper[5052]: I1203 12:34:40.842791 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/38bfd61a-669e-4d69-b667-69270b5ec99d-logs\") pod \"placement-6cd995dc74-hpvzm\" (UID: \"38bfd61a-669e-4d69-b667-69270b5ec99d\") " pod="openstack-kuttl-tests/placement-6cd995dc74-hpvzm" Dec 03 12:34:40 crc kubenswrapper[5052]: I1203 12:34:40.848297 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0a3ff09e-7256-4090-9060-099706213333-combined-ca-bundle\") pod \"keystone-bootstrap-9grd9\" (UID: \"0a3ff09e-7256-4090-9060-099706213333\") " pod="openstack-kuttl-tests/keystone-bootstrap-9grd9" Dec 03 12:34:40 crc kubenswrapper[5052]: I1203 12:34:40.848358 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/0a3ff09e-7256-4090-9060-099706213333-fernet-keys\") pod \"keystone-bootstrap-9grd9\" (UID: \"0a3ff09e-7256-4090-9060-099706213333\") " pod="openstack-kuttl-tests/keystone-bootstrap-9grd9" Dec 03 12:34:40 crc kubenswrapper[5052]: I1203 12:34:40.852694 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/38bfd61a-669e-4d69-b667-69270b5ec99d-scripts\") pod \"placement-6cd995dc74-hpvzm\" (UID: \"38bfd61a-669e-4d69-b667-69270b5ec99d\") " pod="openstack-kuttl-tests/placement-6cd995dc74-hpvzm" Dec 03 12:34:40 crc kubenswrapper[5052]: I1203 12:34:40.855088 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/38bfd61a-669e-4d69-b667-69270b5ec99d-public-tls-certs\") pod \"placement-6cd995dc74-hpvzm\" (UID: \"38bfd61a-669e-4d69-b667-69270b5ec99d\") " pod="openstack-kuttl-tests/placement-6cd995dc74-hpvzm" Dec 03 12:34:40 crc kubenswrapper[5052]: I1203 12:34:40.857004 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/38bfd61a-669e-4d69-b667-69270b5ec99d-combined-ca-bundle\") pod \"placement-6cd995dc74-hpvzm\" (UID: \"38bfd61a-669e-4d69-b667-69270b5ec99d\") " pod="openstack-kuttl-tests/placement-6cd995dc74-hpvzm" Dec 03 12:34:40 crc kubenswrapper[5052]: I1203 12:34:40.857177 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/0a3ff09e-7256-4090-9060-099706213333-scripts\") pod \"keystone-bootstrap-9grd9\" (UID: \"0a3ff09e-7256-4090-9060-099706213333\") " pod="openstack-kuttl-tests/keystone-bootstrap-9grd9" Dec 03 12:34:40 crc kubenswrapper[5052]: I1203 12:34:40.857284 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/0a3ff09e-7256-4090-9060-099706213333-config-data\") pod \"keystone-bootstrap-9grd9\" (UID: \"0a3ff09e-7256-4090-9060-099706213333\") " pod="openstack-kuttl-tests/keystone-bootstrap-9grd9" Dec 03 12:34:40 crc kubenswrapper[5052]: I1203 12:34:40.858229 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/38bfd61a-669e-4d69-b667-69270b5ec99d-config-data\") pod \"placement-6cd995dc74-hpvzm\" (UID: \"38bfd61a-669e-4d69-b667-69270b5ec99d\") " pod="openstack-kuttl-tests/placement-6cd995dc74-hpvzm" Dec 03 12:34:40 crc kubenswrapper[5052]: I1203 12:34:40.858286 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/38bfd61a-669e-4d69-b667-69270b5ec99d-internal-tls-certs\") pod \"placement-6cd995dc74-hpvzm\" (UID: \"38bfd61a-669e-4d69-b667-69270b5ec99d\") " pod="openstack-kuttl-tests/placement-6cd995dc74-hpvzm" Dec 03 12:34:40 crc kubenswrapper[5052]: I1203 12:34:40.867599 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/0a3ff09e-7256-4090-9060-099706213333-credential-keys\") pod \"keystone-bootstrap-9grd9\" (UID: \"0a3ff09e-7256-4090-9060-099706213333\") " pod="openstack-kuttl-tests/keystone-bootstrap-9grd9" Dec 03 12:34:40 crc kubenswrapper[5052]: I1203 12:34:40.868524 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-nvhjx\" (UniqueName: \"kubernetes.io/projected/38bfd61a-669e-4d69-b667-69270b5ec99d-kube-api-access-nvhjx\") pod \"placement-6cd995dc74-hpvzm\" (UID: \"38bfd61a-669e-4d69-b667-69270b5ec99d\") " pod="openstack-kuttl-tests/placement-6cd995dc74-hpvzm" Dec 03 12:34:40 crc kubenswrapper[5052]: I1203 12:34:40.869628 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-q2h8j\" (UniqueName: \"kubernetes.io/projected/0a3ff09e-7256-4090-9060-099706213333-kube-api-access-q2h8j\") pod \"keystone-bootstrap-9grd9\" (UID: \"0a3ff09e-7256-4090-9060-099706213333\") " pod="openstack-kuttl-tests/keystone-bootstrap-9grd9" Dec 03 12:34:41 crc kubenswrapper[5052]: I1203 12:34:41.024925 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-bootstrap-9grd9" Dec 03 12:34:41 crc kubenswrapper[5052]: I1203 12:34:41.026528 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-6cd995dc74-hpvzm" Dec 03 12:34:41 crc kubenswrapper[5052]: I1203 12:34:41.499615 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"f45ca29a-a9fb-4644-be06-f728cc56d453","Type":"ContainerStarted","Data":"5a5a7b78a52c9eab5e3b6d904e527d910f137fc77d871a1e48dfcc358909e2de"} Dec 03 12:34:41 crc kubenswrapper[5052]: I1203 12:34:41.502319 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"6e96674e-4a20-4d8f-9118-90532fa75788","Type":"ContainerStarted","Data":"f2a8405809435951d9d11632bcafb46cf42cd5e5fa8d7b9e91d1ba64a3ac3dca"} Dec 03 12:34:41 crc kubenswrapper[5052]: I1203 12:34:41.506071 5052 generic.go:334] "Generic (PLEG): container finished" podID="d9523127-c2ea-4f5f-98ae-0288681fb93c" containerID="bf727997773d9131797166f237393d3a676ff8ba1aaa81a1ab8bdb9c481d21ae" exitCode=0 Dec 03 12:34:41 crc kubenswrapper[5052]: I1203 12:34:41.506195 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-db-sync-nj49v" event={"ID":"d9523127-c2ea-4f5f-98ae-0288681fb93c","Type":"ContainerDied","Data":"bf727997773d9131797166f237393d3a676ff8ba1aaa81a1ab8bdb9c481d21ae"} Dec 03 12:34:41 crc kubenswrapper[5052]: I1203 12:34:41.509965 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"25498b02-c179-4555-ad04-1ef6445df4ab","Type":"ContainerStarted","Data":"7e310e8333b7259c202284ca732fa44b22dde345a4ca324e0799f37aa77c10db"} Dec 03 12:34:41 crc kubenswrapper[5052]: I1203 12:34:41.553851 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-bootstrap-9grd9"] Dec 03 12:34:41 crc kubenswrapper[5052]: I1203 12:34:41.562785 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/glance-default-external-api-0" podStartSLOduration=3.562765412 podStartE2EDuration="3.562765412s" podCreationTimestamp="2025-12-03 12:34:38 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:34:41.526538576 +0000 UTC m=+4472.114780340" watchObservedRunningTime="2025-12-03 12:34:41.562765412 +0000 UTC m=+4472.151007176" Dec 03 12:34:41 crc kubenswrapper[5052]: I1203 12:34:41.583447 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/glance-default-internal-api-0" podStartSLOduration=3.583409792 podStartE2EDuration="3.583409792s" podCreationTimestamp="2025-12-03 12:34:38 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:34:41.572881661 +0000 UTC m=+4472.161123425" watchObservedRunningTime="2025-12-03 12:34:41.583409792 +0000 UTC m=+4472.171651546" Dec 03 12:34:41 crc kubenswrapper[5052]: I1203 12:34:41.649908 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/placement-6cd995dc74-hpvzm"] Dec 03 12:34:41 crc kubenswrapper[5052]: I1203 12:34:41.688823 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6e18c0ba-8f07-4d54-b383-3d0164780725" path="/var/lib/kubelet/pods/6e18c0ba-8f07-4d54-b383-3d0164780725/volumes" Dec 03 12:34:41 crc kubenswrapper[5052]: I1203 12:34:41.689817 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="80bd8d5d-a98f-455b-977a-e96317d7f525" path="/var/lib/kubelet/pods/80bd8d5d-a98f-455b-977a-e96317d7f525/volumes" Dec 03 12:34:42 crc kubenswrapper[5052]: I1203 12:34:42.522439 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"f45ca29a-a9fb-4644-be06-f728cc56d453","Type":"ContainerStarted","Data":"8b655d6718df3641de13c30530434a2989b46b2da6946e2ca33442a75f43cf88"} Dec 03 12:34:42 crc kubenswrapper[5052]: I1203 12:34:42.524331 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-6cd995dc74-hpvzm" event={"ID":"38bfd61a-669e-4d69-b667-69270b5ec99d","Type":"ContainerStarted","Data":"54bc9dba691e3f9bc3fd33b6f2ed864845a91b45a88e3b273a06f1bc0b2bce85"} Dec 03 12:34:42 crc kubenswrapper[5052]: I1203 12:34:42.524373 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-6cd995dc74-hpvzm" event={"ID":"38bfd61a-669e-4d69-b667-69270b5ec99d","Type":"ContainerStarted","Data":"473883e07ac4b625b4c47f00b264f64fef22cf82efbcd9bdb6eb7bc82576beb7"} Dec 03 12:34:42 crc kubenswrapper[5052]: I1203 12:34:42.524388 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-6cd995dc74-hpvzm" event={"ID":"38bfd61a-669e-4d69-b667-69270b5ec99d","Type":"ContainerStarted","Data":"6035e3dec571cc358328664f1d0d08e0f5e176cb17551bba680b7f05828dd099"} Dec 03 12:34:42 crc kubenswrapper[5052]: I1203 12:34:42.525855 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/placement-6cd995dc74-hpvzm" Dec 03 12:34:42 crc kubenswrapper[5052]: I1203 12:34:42.525882 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/placement-6cd995dc74-hpvzm" Dec 03 12:34:42 crc kubenswrapper[5052]: I1203 12:34:42.528306 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-bootstrap-9grd9" event={"ID":"0a3ff09e-7256-4090-9060-099706213333","Type":"ContainerStarted","Data":"a8d5c79e35703d7180a5fe276802f60ef6ba0eabb63c93544208b5e293dcdf4e"} Dec 03 12:34:42 crc kubenswrapper[5052]: I1203 12:34:42.528327 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-bootstrap-9grd9" event={"ID":"0a3ff09e-7256-4090-9060-099706213333","Type":"ContainerStarted","Data":"9c71a35a9a24539d037ad2d5bd1b4d452359582ad28f32dff522ce818b0f1f5a"} Dec 03 12:34:42 crc kubenswrapper[5052]: I1203 12:34:42.555261 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/placement-6cd995dc74-hpvzm" podStartSLOduration=2.5552334549999998 podStartE2EDuration="2.555233455s" podCreationTimestamp="2025-12-03 12:34:40 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:34:42.549615415 +0000 UTC m=+4473.137857229" watchObservedRunningTime="2025-12-03 12:34:42.555233455 +0000 UTC m=+4473.143475239" Dec 03 12:34:42 crc kubenswrapper[5052]: I1203 12:34:42.577611 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/keystone-bootstrap-9grd9" podStartSLOduration=2.577591875 podStartE2EDuration="2.577591875s" podCreationTimestamp="2025-12-03 12:34:40 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:34:42.577005358 +0000 UTC m=+4473.165247122" watchObservedRunningTime="2025-12-03 12:34:42.577591875 +0000 UTC m=+4473.165833639" Dec 03 12:34:42 crc kubenswrapper[5052]: I1203 12:34:42.972779 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-db-sync-nj49v" Dec 03 12:34:43 crc kubenswrapper[5052]: I1203 12:34:43.086384 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d9523127-c2ea-4f5f-98ae-0288681fb93c-combined-ca-bundle\") pod \"d9523127-c2ea-4f5f-98ae-0288681fb93c\" (UID: \"d9523127-c2ea-4f5f-98ae-0288681fb93c\") " Dec 03 12:34:43 crc kubenswrapper[5052]: I1203 12:34:43.086642 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mns4w\" (UniqueName: \"kubernetes.io/projected/d9523127-c2ea-4f5f-98ae-0288681fb93c-kube-api-access-mns4w\") pod \"d9523127-c2ea-4f5f-98ae-0288681fb93c\" (UID: \"d9523127-c2ea-4f5f-98ae-0288681fb93c\") " Dec 03 12:34:43 crc kubenswrapper[5052]: I1203 12:34:43.086691 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/d9523127-c2ea-4f5f-98ae-0288681fb93c-db-sync-config-data\") pod \"d9523127-c2ea-4f5f-98ae-0288681fb93c\" (UID: \"d9523127-c2ea-4f5f-98ae-0288681fb93c\") " Dec 03 12:34:43 crc kubenswrapper[5052]: I1203 12:34:43.091084 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d9523127-c2ea-4f5f-98ae-0288681fb93c-db-sync-config-data" (OuterVolumeSpecName: "db-sync-config-data") pod "d9523127-c2ea-4f5f-98ae-0288681fb93c" (UID: "d9523127-c2ea-4f5f-98ae-0288681fb93c"). InnerVolumeSpecName "db-sync-config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:34:43 crc kubenswrapper[5052]: I1203 12:34:43.091113 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d9523127-c2ea-4f5f-98ae-0288681fb93c-kube-api-access-mns4w" (OuterVolumeSpecName: "kube-api-access-mns4w") pod "d9523127-c2ea-4f5f-98ae-0288681fb93c" (UID: "d9523127-c2ea-4f5f-98ae-0288681fb93c"). InnerVolumeSpecName "kube-api-access-mns4w". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:34:43 crc kubenswrapper[5052]: I1203 12:34:43.110180 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d9523127-c2ea-4f5f-98ae-0288681fb93c-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "d9523127-c2ea-4f5f-98ae-0288681fb93c" (UID: "d9523127-c2ea-4f5f-98ae-0288681fb93c"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:34:43 crc kubenswrapper[5052]: I1203 12:34:43.188855 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-mns4w\" (UniqueName: \"kubernetes.io/projected/d9523127-c2ea-4f5f-98ae-0288681fb93c-kube-api-access-mns4w\") on node \"crc\" DevicePath \"\"" Dec 03 12:34:43 crc kubenswrapper[5052]: I1203 12:34:43.188909 5052 reconciler_common.go:293] "Volume detached for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/d9523127-c2ea-4f5f-98ae-0288681fb93c-db-sync-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:34:43 crc kubenswrapper[5052]: I1203 12:34:43.188919 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d9523127-c2ea-4f5f-98ae-0288681fb93c-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:34:43 crc kubenswrapper[5052]: I1203 12:34:43.537038 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-db-sync-nj49v" Dec 03 12:34:43 crc kubenswrapper[5052]: I1203 12:34:43.538597 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-db-sync-nj49v" event={"ID":"d9523127-c2ea-4f5f-98ae-0288681fb93c","Type":"ContainerDied","Data":"cf44f0b7617a4ca325e895dca0ebefb4380bccd2b4dd6bfa58930351ecdfa3ea"} Dec 03 12:34:43 crc kubenswrapper[5052]: I1203 12:34:43.538640 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cf44f0b7617a4ca325e895dca0ebefb4380bccd2b4dd6bfa58930351ecdfa3ea" Dec 03 12:34:43 crc kubenswrapper[5052]: I1203 12:34:43.540709 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"f45ca29a-a9fb-4644-be06-f728cc56d453","Type":"ContainerStarted","Data":"5c1d2761e93325105dd9e13409ca25432fbfd12fd75000d1efd160e5758aa9e0"} Dec 03 12:34:43 crc kubenswrapper[5052]: I1203 12:34:43.540758 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"f45ca29a-a9fb-4644-be06-f728cc56d453","Type":"ContainerStarted","Data":"dfad5acef7a373a71ef748493cbad2e7c868171969fa948b880b61be5af6d286"} Dec 03 12:34:43 crc kubenswrapper[5052]: I1203 12:34:43.822825 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/barbican-worker-7bd4f6f96f-w4glb"] Dec 03 12:34:43 crc kubenswrapper[5052]: E1203 12:34:43.823277 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="d9523127-c2ea-4f5f-98ae-0288681fb93c" containerName="barbican-db-sync" Dec 03 12:34:43 crc kubenswrapper[5052]: I1203 12:34:43.823298 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="d9523127-c2ea-4f5f-98ae-0288681fb93c" containerName="barbican-db-sync" Dec 03 12:34:43 crc kubenswrapper[5052]: I1203 12:34:43.823498 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="d9523127-c2ea-4f5f-98ae-0288681fb93c" containerName="barbican-db-sync" Dec 03 12:34:43 crc kubenswrapper[5052]: I1203 12:34:43.824466 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-worker-7bd4f6f96f-w4glb" Dec 03 12:34:43 crc kubenswrapper[5052]: I1203 12:34:43.828453 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"barbican-barbican-dockercfg-n9q58" Dec 03 12:34:43 crc kubenswrapper[5052]: I1203 12:34:43.828814 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/barbican-keystone-listener-8d6c85ffd-x99nt"] Dec 03 12:34:43 crc kubenswrapper[5052]: I1203 12:34:43.829584 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"barbican-worker-config-data" Dec 03 12:34:43 crc kubenswrapper[5052]: I1203 12:34:43.829797 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"barbican-config-data" Dec 03 12:34:43 crc kubenswrapper[5052]: I1203 12:34:43.831237 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-keystone-listener-8d6c85ffd-x99nt" Dec 03 12:34:43 crc kubenswrapper[5052]: I1203 12:34:43.836752 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"barbican-keystone-listener-config-data" Dec 03 12:34:43 crc kubenswrapper[5052]: I1203 12:34:43.842109 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-worker-7bd4f6f96f-w4glb"] Dec 03 12:34:43 crc kubenswrapper[5052]: I1203 12:34:43.855422 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-keystone-listener-8d6c85ffd-x99nt"] Dec 03 12:34:43 crc kubenswrapper[5052]: I1203 12:34:43.905696 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/15415292-9b86-462f-afe6-7f688c84805b-logs\") pod \"barbican-worker-7bd4f6f96f-w4glb\" (UID: \"15415292-9b86-462f-afe6-7f688c84805b\") " pod="openstack-kuttl-tests/barbican-worker-7bd4f6f96f-w4glb" Dec 03 12:34:43 crc kubenswrapper[5052]: I1203 12:34:43.905785 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/15415292-9b86-462f-afe6-7f688c84805b-config-data\") pod \"barbican-worker-7bd4f6f96f-w4glb\" (UID: \"15415292-9b86-462f-afe6-7f688c84805b\") " pod="openstack-kuttl-tests/barbican-worker-7bd4f6f96f-w4glb" Dec 03 12:34:43 crc kubenswrapper[5052]: I1203 12:34:43.905872 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qjhwk\" (UniqueName: \"kubernetes.io/projected/15415292-9b86-462f-afe6-7f688c84805b-kube-api-access-qjhwk\") pod \"barbican-worker-7bd4f6f96f-w4glb\" (UID: \"15415292-9b86-462f-afe6-7f688c84805b\") " pod="openstack-kuttl-tests/barbican-worker-7bd4f6f96f-w4glb" Dec 03 12:34:43 crc kubenswrapper[5052]: I1203 12:34:43.905902 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/15415292-9b86-462f-afe6-7f688c84805b-combined-ca-bundle\") pod \"barbican-worker-7bd4f6f96f-w4glb\" (UID: \"15415292-9b86-462f-afe6-7f688c84805b\") " pod="openstack-kuttl-tests/barbican-worker-7bd4f6f96f-w4glb" Dec 03 12:34:43 crc kubenswrapper[5052]: I1203 12:34:43.905943 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/15415292-9b86-462f-afe6-7f688c84805b-config-data-custom\") pod \"barbican-worker-7bd4f6f96f-w4glb\" (UID: \"15415292-9b86-462f-afe6-7f688c84805b\") " pod="openstack-kuttl-tests/barbican-worker-7bd4f6f96f-w4glb" Dec 03 12:34:43 crc kubenswrapper[5052]: I1203 12:34:43.987136 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/barbican-api-5656894b88-g5p25"] Dec 03 12:34:43 crc kubenswrapper[5052]: I1203 12:34:43.989073 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-api-5656894b88-g5p25" Dec 03 12:34:43 crc kubenswrapper[5052]: I1203 12:34:43.990954 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"barbican-api-config-data" Dec 03 12:34:44 crc kubenswrapper[5052]: I1203 12:34:44.008327 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-qjhwk\" (UniqueName: \"kubernetes.io/projected/15415292-9b86-462f-afe6-7f688c84805b-kube-api-access-qjhwk\") pod \"barbican-worker-7bd4f6f96f-w4glb\" (UID: \"15415292-9b86-462f-afe6-7f688c84805b\") " pod="openstack-kuttl-tests/barbican-worker-7bd4f6f96f-w4glb" Dec 03 12:34:44 crc kubenswrapper[5052]: I1203 12:34:44.008391 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/15415292-9b86-462f-afe6-7f688c84805b-combined-ca-bundle\") pod \"barbican-worker-7bd4f6f96f-w4glb\" (UID: \"15415292-9b86-462f-afe6-7f688c84805b\") " pod="openstack-kuttl-tests/barbican-worker-7bd4f6f96f-w4glb" Dec 03 12:34:44 crc kubenswrapper[5052]: I1203 12:34:44.008437 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d6ln8\" (UniqueName: \"kubernetes.io/projected/f8e97a34-9689-4f0f-b43e-f87b43a115c8-kube-api-access-d6ln8\") pod \"barbican-keystone-listener-8d6c85ffd-x99nt\" (UID: \"f8e97a34-9689-4f0f-b43e-f87b43a115c8\") " pod="openstack-kuttl-tests/barbican-keystone-listener-8d6c85ffd-x99nt" Dec 03 12:34:44 crc kubenswrapper[5052]: I1203 12:34:44.008468 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/f8e97a34-9689-4f0f-b43e-f87b43a115c8-config-data\") pod \"barbican-keystone-listener-8d6c85ffd-x99nt\" (UID: \"f8e97a34-9689-4f0f-b43e-f87b43a115c8\") " pod="openstack-kuttl-tests/barbican-keystone-listener-8d6c85ffd-x99nt" Dec 03 12:34:44 crc kubenswrapper[5052]: I1203 12:34:44.008495 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/15415292-9b86-462f-afe6-7f688c84805b-config-data-custom\") pod \"barbican-worker-7bd4f6f96f-w4glb\" (UID: \"15415292-9b86-462f-afe6-7f688c84805b\") " pod="openstack-kuttl-tests/barbican-worker-7bd4f6f96f-w4glb" Dec 03 12:34:44 crc kubenswrapper[5052]: I1203 12:34:44.008585 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/f8e97a34-9689-4f0f-b43e-f87b43a115c8-config-data-custom\") pod \"barbican-keystone-listener-8d6c85ffd-x99nt\" (UID: \"f8e97a34-9689-4f0f-b43e-f87b43a115c8\") " pod="openstack-kuttl-tests/barbican-keystone-listener-8d6c85ffd-x99nt" Dec 03 12:34:44 crc kubenswrapper[5052]: I1203 12:34:44.008617 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/15415292-9b86-462f-afe6-7f688c84805b-logs\") pod \"barbican-worker-7bd4f6f96f-w4glb\" (UID: \"15415292-9b86-462f-afe6-7f688c84805b\") " pod="openstack-kuttl-tests/barbican-worker-7bd4f6f96f-w4glb" Dec 03 12:34:44 crc kubenswrapper[5052]: I1203 12:34:44.008662 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/f8e97a34-9689-4f0f-b43e-f87b43a115c8-logs\") pod \"barbican-keystone-listener-8d6c85ffd-x99nt\" (UID: \"f8e97a34-9689-4f0f-b43e-f87b43a115c8\") " pod="openstack-kuttl-tests/barbican-keystone-listener-8d6c85ffd-x99nt" Dec 03 12:34:44 crc kubenswrapper[5052]: I1203 12:34:44.008695 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/15415292-9b86-462f-afe6-7f688c84805b-config-data\") pod \"barbican-worker-7bd4f6f96f-w4glb\" (UID: \"15415292-9b86-462f-afe6-7f688c84805b\") " pod="openstack-kuttl-tests/barbican-worker-7bd4f6f96f-w4glb" Dec 03 12:34:44 crc kubenswrapper[5052]: I1203 12:34:44.008742 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f8e97a34-9689-4f0f-b43e-f87b43a115c8-combined-ca-bundle\") pod \"barbican-keystone-listener-8d6c85ffd-x99nt\" (UID: \"f8e97a34-9689-4f0f-b43e-f87b43a115c8\") " pod="openstack-kuttl-tests/barbican-keystone-listener-8d6c85ffd-x99nt" Dec 03 12:34:44 crc kubenswrapper[5052]: I1203 12:34:44.013963 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/15415292-9b86-462f-afe6-7f688c84805b-logs\") pod \"barbican-worker-7bd4f6f96f-w4glb\" (UID: \"15415292-9b86-462f-afe6-7f688c84805b\") " pod="openstack-kuttl-tests/barbican-worker-7bd4f6f96f-w4glb" Dec 03 12:34:44 crc kubenswrapper[5052]: I1203 12:34:44.014664 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-api-5656894b88-g5p25"] Dec 03 12:34:44 crc kubenswrapper[5052]: I1203 12:34:44.032207 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/15415292-9b86-462f-afe6-7f688c84805b-combined-ca-bundle\") pod \"barbican-worker-7bd4f6f96f-w4glb\" (UID: \"15415292-9b86-462f-afe6-7f688c84805b\") " pod="openstack-kuttl-tests/barbican-worker-7bd4f6f96f-w4glb" Dec 03 12:34:44 crc kubenswrapper[5052]: I1203 12:34:44.033555 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/15415292-9b86-462f-afe6-7f688c84805b-config-data\") pod \"barbican-worker-7bd4f6f96f-w4glb\" (UID: \"15415292-9b86-462f-afe6-7f688c84805b\") " pod="openstack-kuttl-tests/barbican-worker-7bd4f6f96f-w4glb" Dec 03 12:34:44 crc kubenswrapper[5052]: I1203 12:34:44.037401 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-qjhwk\" (UniqueName: \"kubernetes.io/projected/15415292-9b86-462f-afe6-7f688c84805b-kube-api-access-qjhwk\") pod \"barbican-worker-7bd4f6f96f-w4glb\" (UID: \"15415292-9b86-462f-afe6-7f688c84805b\") " pod="openstack-kuttl-tests/barbican-worker-7bd4f6f96f-w4glb" Dec 03 12:34:44 crc kubenswrapper[5052]: I1203 12:34:44.038103 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/15415292-9b86-462f-afe6-7f688c84805b-config-data-custom\") pod \"barbican-worker-7bd4f6f96f-w4glb\" (UID: \"15415292-9b86-462f-afe6-7f688c84805b\") " pod="openstack-kuttl-tests/barbican-worker-7bd4f6f96f-w4glb" Dec 03 12:34:44 crc kubenswrapper[5052]: I1203 12:34:44.110719 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5j2k8\" (UniqueName: \"kubernetes.io/projected/8a22a52b-d45b-4678-900d-a9d6d5113da3-kube-api-access-5j2k8\") pod \"barbican-api-5656894b88-g5p25\" (UID: \"8a22a52b-d45b-4678-900d-a9d6d5113da3\") " pod="openstack-kuttl-tests/barbican-api-5656894b88-g5p25" Dec 03 12:34:44 crc kubenswrapper[5052]: I1203 12:34:44.111063 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f8e97a34-9689-4f0f-b43e-f87b43a115c8-combined-ca-bundle\") pod \"barbican-keystone-listener-8d6c85ffd-x99nt\" (UID: \"f8e97a34-9689-4f0f-b43e-f87b43a115c8\") " pod="openstack-kuttl-tests/barbican-keystone-listener-8d6c85ffd-x99nt" Dec 03 12:34:44 crc kubenswrapper[5052]: I1203 12:34:44.111102 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/8a22a52b-d45b-4678-900d-a9d6d5113da3-config-data-custom\") pod \"barbican-api-5656894b88-g5p25\" (UID: \"8a22a52b-d45b-4678-900d-a9d6d5113da3\") " pod="openstack-kuttl-tests/barbican-api-5656894b88-g5p25" Dec 03 12:34:44 crc kubenswrapper[5052]: I1203 12:34:44.111178 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-d6ln8\" (UniqueName: \"kubernetes.io/projected/f8e97a34-9689-4f0f-b43e-f87b43a115c8-kube-api-access-d6ln8\") pod \"barbican-keystone-listener-8d6c85ffd-x99nt\" (UID: \"f8e97a34-9689-4f0f-b43e-f87b43a115c8\") " pod="openstack-kuttl-tests/barbican-keystone-listener-8d6c85ffd-x99nt" Dec 03 12:34:44 crc kubenswrapper[5052]: I1203 12:34:44.111213 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/f8e97a34-9689-4f0f-b43e-f87b43a115c8-config-data\") pod \"barbican-keystone-listener-8d6c85ffd-x99nt\" (UID: \"f8e97a34-9689-4f0f-b43e-f87b43a115c8\") " pod="openstack-kuttl-tests/barbican-keystone-listener-8d6c85ffd-x99nt" Dec 03 12:34:44 crc kubenswrapper[5052]: I1203 12:34:44.111264 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/8a22a52b-d45b-4678-900d-a9d6d5113da3-config-data\") pod \"barbican-api-5656894b88-g5p25\" (UID: \"8a22a52b-d45b-4678-900d-a9d6d5113da3\") " pod="openstack-kuttl-tests/barbican-api-5656894b88-g5p25" Dec 03 12:34:44 crc kubenswrapper[5052]: I1203 12:34:44.111312 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/f8e97a34-9689-4f0f-b43e-f87b43a115c8-config-data-custom\") pod \"barbican-keystone-listener-8d6c85ffd-x99nt\" (UID: \"f8e97a34-9689-4f0f-b43e-f87b43a115c8\") " pod="openstack-kuttl-tests/barbican-keystone-listener-8d6c85ffd-x99nt" Dec 03 12:34:44 crc kubenswrapper[5052]: I1203 12:34:44.111370 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/8a22a52b-d45b-4678-900d-a9d6d5113da3-logs\") pod \"barbican-api-5656894b88-g5p25\" (UID: \"8a22a52b-d45b-4678-900d-a9d6d5113da3\") " pod="openstack-kuttl-tests/barbican-api-5656894b88-g5p25" Dec 03 12:34:44 crc kubenswrapper[5052]: I1203 12:34:44.111431 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/f8e97a34-9689-4f0f-b43e-f87b43a115c8-logs\") pod \"barbican-keystone-listener-8d6c85ffd-x99nt\" (UID: \"f8e97a34-9689-4f0f-b43e-f87b43a115c8\") " pod="openstack-kuttl-tests/barbican-keystone-listener-8d6c85ffd-x99nt" Dec 03 12:34:44 crc kubenswrapper[5052]: I1203 12:34:44.111478 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8a22a52b-d45b-4678-900d-a9d6d5113da3-combined-ca-bundle\") pod \"barbican-api-5656894b88-g5p25\" (UID: \"8a22a52b-d45b-4678-900d-a9d6d5113da3\") " pod="openstack-kuttl-tests/barbican-api-5656894b88-g5p25" Dec 03 12:34:44 crc kubenswrapper[5052]: I1203 12:34:44.111857 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/f8e97a34-9689-4f0f-b43e-f87b43a115c8-logs\") pod \"barbican-keystone-listener-8d6c85ffd-x99nt\" (UID: \"f8e97a34-9689-4f0f-b43e-f87b43a115c8\") " pod="openstack-kuttl-tests/barbican-keystone-listener-8d6c85ffd-x99nt" Dec 03 12:34:44 crc kubenswrapper[5052]: I1203 12:34:44.116133 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f8e97a34-9689-4f0f-b43e-f87b43a115c8-combined-ca-bundle\") pod \"barbican-keystone-listener-8d6c85ffd-x99nt\" (UID: \"f8e97a34-9689-4f0f-b43e-f87b43a115c8\") " pod="openstack-kuttl-tests/barbican-keystone-listener-8d6c85ffd-x99nt" Dec 03 12:34:44 crc kubenswrapper[5052]: I1203 12:34:44.116137 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/f8e97a34-9689-4f0f-b43e-f87b43a115c8-config-data-custom\") pod \"barbican-keystone-listener-8d6c85ffd-x99nt\" (UID: \"f8e97a34-9689-4f0f-b43e-f87b43a115c8\") " pod="openstack-kuttl-tests/barbican-keystone-listener-8d6c85ffd-x99nt" Dec 03 12:34:44 crc kubenswrapper[5052]: I1203 12:34:44.117453 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/f8e97a34-9689-4f0f-b43e-f87b43a115c8-config-data\") pod \"barbican-keystone-listener-8d6c85ffd-x99nt\" (UID: \"f8e97a34-9689-4f0f-b43e-f87b43a115c8\") " pod="openstack-kuttl-tests/barbican-keystone-listener-8d6c85ffd-x99nt" Dec 03 12:34:44 crc kubenswrapper[5052]: I1203 12:34:44.130003 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-d6ln8\" (UniqueName: \"kubernetes.io/projected/f8e97a34-9689-4f0f-b43e-f87b43a115c8-kube-api-access-d6ln8\") pod \"barbican-keystone-listener-8d6c85ffd-x99nt\" (UID: \"f8e97a34-9689-4f0f-b43e-f87b43a115c8\") " pod="openstack-kuttl-tests/barbican-keystone-listener-8d6c85ffd-x99nt" Dec 03 12:34:44 crc kubenswrapper[5052]: I1203 12:34:44.156160 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-worker-7bd4f6f96f-w4glb" Dec 03 12:34:44 crc kubenswrapper[5052]: I1203 12:34:44.176195 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-keystone-listener-8d6c85ffd-x99nt" Dec 03 12:34:44 crc kubenswrapper[5052]: I1203 12:34:44.212830 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-5j2k8\" (UniqueName: \"kubernetes.io/projected/8a22a52b-d45b-4678-900d-a9d6d5113da3-kube-api-access-5j2k8\") pod \"barbican-api-5656894b88-g5p25\" (UID: \"8a22a52b-d45b-4678-900d-a9d6d5113da3\") " pod="openstack-kuttl-tests/barbican-api-5656894b88-g5p25" Dec 03 12:34:44 crc kubenswrapper[5052]: I1203 12:34:44.212903 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/8a22a52b-d45b-4678-900d-a9d6d5113da3-config-data-custom\") pod \"barbican-api-5656894b88-g5p25\" (UID: \"8a22a52b-d45b-4678-900d-a9d6d5113da3\") " pod="openstack-kuttl-tests/barbican-api-5656894b88-g5p25" Dec 03 12:34:44 crc kubenswrapper[5052]: I1203 12:34:44.213092 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/8a22a52b-d45b-4678-900d-a9d6d5113da3-config-data\") pod \"barbican-api-5656894b88-g5p25\" (UID: \"8a22a52b-d45b-4678-900d-a9d6d5113da3\") " pod="openstack-kuttl-tests/barbican-api-5656894b88-g5p25" Dec 03 12:34:44 crc kubenswrapper[5052]: I1203 12:34:44.213673 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/8a22a52b-d45b-4678-900d-a9d6d5113da3-logs\") pod \"barbican-api-5656894b88-g5p25\" (UID: \"8a22a52b-d45b-4678-900d-a9d6d5113da3\") " pod="openstack-kuttl-tests/barbican-api-5656894b88-g5p25" Dec 03 12:34:44 crc kubenswrapper[5052]: I1203 12:34:44.213743 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8a22a52b-d45b-4678-900d-a9d6d5113da3-combined-ca-bundle\") pod \"barbican-api-5656894b88-g5p25\" (UID: \"8a22a52b-d45b-4678-900d-a9d6d5113da3\") " pod="openstack-kuttl-tests/barbican-api-5656894b88-g5p25" Dec 03 12:34:44 crc kubenswrapper[5052]: I1203 12:34:44.214279 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/8a22a52b-d45b-4678-900d-a9d6d5113da3-logs\") pod \"barbican-api-5656894b88-g5p25\" (UID: \"8a22a52b-d45b-4678-900d-a9d6d5113da3\") " pod="openstack-kuttl-tests/barbican-api-5656894b88-g5p25" Dec 03 12:34:44 crc kubenswrapper[5052]: I1203 12:34:44.216961 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/8a22a52b-d45b-4678-900d-a9d6d5113da3-config-data-custom\") pod \"barbican-api-5656894b88-g5p25\" (UID: \"8a22a52b-d45b-4678-900d-a9d6d5113da3\") " pod="openstack-kuttl-tests/barbican-api-5656894b88-g5p25" Dec 03 12:34:44 crc kubenswrapper[5052]: I1203 12:34:44.217670 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/8a22a52b-d45b-4678-900d-a9d6d5113da3-config-data\") pod \"barbican-api-5656894b88-g5p25\" (UID: \"8a22a52b-d45b-4678-900d-a9d6d5113da3\") " pod="openstack-kuttl-tests/barbican-api-5656894b88-g5p25" Dec 03 12:34:44 crc kubenswrapper[5052]: I1203 12:34:44.218478 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8a22a52b-d45b-4678-900d-a9d6d5113da3-combined-ca-bundle\") pod \"barbican-api-5656894b88-g5p25\" (UID: \"8a22a52b-d45b-4678-900d-a9d6d5113da3\") " pod="openstack-kuttl-tests/barbican-api-5656894b88-g5p25" Dec 03 12:34:44 crc kubenswrapper[5052]: I1203 12:34:44.228631 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-5j2k8\" (UniqueName: \"kubernetes.io/projected/8a22a52b-d45b-4678-900d-a9d6d5113da3-kube-api-access-5j2k8\") pod \"barbican-api-5656894b88-g5p25\" (UID: \"8a22a52b-d45b-4678-900d-a9d6d5113da3\") " pod="openstack-kuttl-tests/barbican-api-5656894b88-g5p25" Dec 03 12:34:46 crc kubenswrapper[5052]: I1203 12:34:44.319048 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-api-5656894b88-g5p25" Dec 03 12:34:46 crc kubenswrapper[5052]: I1203 12:34:44.672818 5052 scope.go:117] "RemoveContainer" containerID="e6edaf188f85ea4f11f487f11cc489c700f25fb42b5ee269f4f496f1e127d772" Dec 03 12:34:46 crc kubenswrapper[5052]: E1203 12:34:44.673326 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 12:34:46 crc kubenswrapper[5052]: I1203 12:34:45.568757 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"f45ca29a-a9fb-4644-be06-f728cc56d453","Type":"ContainerStarted","Data":"064f270e84004ca2ace7f9545a79dfb6f74dcab89088dbb2aa2b43e7663886fa"} Dec 03 12:34:46 crc kubenswrapper[5052]: I1203 12:34:45.569359 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:34:46 crc kubenswrapper[5052]: I1203 12:34:45.576220 5052 generic.go:334] "Generic (PLEG): container finished" podID="55ccf323-53e4-47fe-bea8-a17273b8587c" containerID="d8dc47bd79781784d95621d45bf044b65727ec3ede2bffae178e8e91d46cc7b7" exitCode=0 Dec 03 12:34:46 crc kubenswrapper[5052]: I1203 12:34:45.576268 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-db-sync-s42b9" event={"ID":"55ccf323-53e4-47fe-bea8-a17273b8587c","Type":"ContainerDied","Data":"d8dc47bd79781784d95621d45bf044b65727ec3ede2bffae178e8e91d46cc7b7"} Dec 03 12:34:46 crc kubenswrapper[5052]: I1203 12:34:45.619984 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/ceilometer-0" podStartSLOduration=2.347912839 podStartE2EDuration="6.619959801s" podCreationTimestamp="2025-12-03 12:34:39 +0000 UTC" firstStartedPulling="2025-12-03 12:34:40.792633018 +0000 UTC m=+4471.380874782" lastFinishedPulling="2025-12-03 12:34:45.06467998 +0000 UTC m=+4475.652921744" observedRunningTime="2025-12-03 12:34:45.606431614 +0000 UTC m=+4476.194673378" watchObservedRunningTime="2025-12-03 12:34:45.619959801 +0000 UTC m=+4476.208201555" Dec 03 12:34:46 crc kubenswrapper[5052]: I1203 12:34:46.427400 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-api-5656894b88-g5p25"] Dec 03 12:34:46 crc kubenswrapper[5052]: I1203 12:34:46.437262 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-keystone-listener-8d6c85ffd-x99nt"] Dec 03 12:34:46 crc kubenswrapper[5052]: I1203 12:34:46.445587 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-worker-7bd4f6f96f-w4glb"] Dec 03 12:34:46 crc kubenswrapper[5052]: W1203 12:34:46.445832 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf8e97a34_9689_4f0f_b43e_f87b43a115c8.slice/crio-44aa20016b5f599a6556489aaea176b866aa982026fffe9d162ac81e13c9045d WatchSource:0}: Error finding container 44aa20016b5f599a6556489aaea176b866aa982026fffe9d162ac81e13c9045d: Status 404 returned error can't find the container with id 44aa20016b5f599a6556489aaea176b866aa982026fffe9d162ac81e13c9045d Dec 03 12:34:46 crc kubenswrapper[5052]: I1203 12:34:46.592643 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-keystone-listener-8d6c85ffd-x99nt" event={"ID":"f8e97a34-9689-4f0f-b43e-f87b43a115c8","Type":"ContainerStarted","Data":"44aa20016b5f599a6556489aaea176b866aa982026fffe9d162ac81e13c9045d"} Dec 03 12:34:46 crc kubenswrapper[5052]: I1203 12:34:46.597050 5052 generic.go:334] "Generic (PLEG): container finished" podID="0a3ff09e-7256-4090-9060-099706213333" containerID="a8d5c79e35703d7180a5fe276802f60ef6ba0eabb63c93544208b5e293dcdf4e" exitCode=0 Dec 03 12:34:46 crc kubenswrapper[5052]: I1203 12:34:46.597126 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-bootstrap-9grd9" event={"ID":"0a3ff09e-7256-4090-9060-099706213333","Type":"ContainerDied","Data":"a8d5c79e35703d7180a5fe276802f60ef6ba0eabb63c93544208b5e293dcdf4e"} Dec 03 12:34:46 crc kubenswrapper[5052]: I1203 12:34:46.601298 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-5656894b88-g5p25" event={"ID":"8a22a52b-d45b-4678-900d-a9d6d5113da3","Type":"ContainerStarted","Data":"3eb18214b19b85281e2ecd516827f1dd427fa02f1c919d5210f68825fc8fa32c"} Dec 03 12:34:46 crc kubenswrapper[5052]: I1203 12:34:46.603127 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-worker-7bd4f6f96f-w4glb" event={"ID":"15415292-9b86-462f-afe6-7f688c84805b","Type":"ContainerStarted","Data":"b304eb2fead6ebc0b2296b60138ce48ee9146a3adbb09731cb0639c884649eff"} Dec 03 12:34:46 crc kubenswrapper[5052]: I1203 12:34:46.657192 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/barbican-api-b857b994d-6xzbl"] Dec 03 12:34:46 crc kubenswrapper[5052]: I1203 12:34:46.659609 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-api-b857b994d-6xzbl" Dec 03 12:34:46 crc kubenswrapper[5052]: I1203 12:34:46.662990 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-barbican-internal-svc" Dec 03 12:34:46 crc kubenswrapper[5052]: I1203 12:34:46.663148 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-barbican-public-svc" Dec 03 12:34:46 crc kubenswrapper[5052]: I1203 12:34:46.677882 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-api-b857b994d-6xzbl"] Dec 03 12:34:46 crc kubenswrapper[5052]: I1203 12:34:46.771848 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/3ceac470-11eb-4d69-a683-2e9bc4e8f617-public-tls-certs\") pod \"barbican-api-b857b994d-6xzbl\" (UID: \"3ceac470-11eb-4d69-a683-2e9bc4e8f617\") " pod="openstack-kuttl-tests/barbican-api-b857b994d-6xzbl" Dec 03 12:34:46 crc kubenswrapper[5052]: I1203 12:34:46.771995 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/3ceac470-11eb-4d69-a683-2e9bc4e8f617-config-data-custom\") pod \"barbican-api-b857b994d-6xzbl\" (UID: \"3ceac470-11eb-4d69-a683-2e9bc4e8f617\") " pod="openstack-kuttl-tests/barbican-api-b857b994d-6xzbl" Dec 03 12:34:46 crc kubenswrapper[5052]: I1203 12:34:46.772029 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/3ceac470-11eb-4d69-a683-2e9bc4e8f617-internal-tls-certs\") pod \"barbican-api-b857b994d-6xzbl\" (UID: \"3ceac470-11eb-4d69-a683-2e9bc4e8f617\") " pod="openstack-kuttl-tests/barbican-api-b857b994d-6xzbl" Dec 03 12:34:46 crc kubenswrapper[5052]: I1203 12:34:46.773211 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3ceac470-11eb-4d69-a683-2e9bc4e8f617-combined-ca-bundle\") pod \"barbican-api-b857b994d-6xzbl\" (UID: \"3ceac470-11eb-4d69-a683-2e9bc4e8f617\") " pod="openstack-kuttl-tests/barbican-api-b857b994d-6xzbl" Dec 03 12:34:46 crc kubenswrapper[5052]: I1203 12:34:46.773451 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/3ceac470-11eb-4d69-a683-2e9bc4e8f617-logs\") pod \"barbican-api-b857b994d-6xzbl\" (UID: \"3ceac470-11eb-4d69-a683-2e9bc4e8f617\") " pod="openstack-kuttl-tests/barbican-api-b857b994d-6xzbl" Dec 03 12:34:46 crc kubenswrapper[5052]: I1203 12:34:46.773622 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/3ceac470-11eb-4d69-a683-2e9bc4e8f617-config-data\") pod \"barbican-api-b857b994d-6xzbl\" (UID: \"3ceac470-11eb-4d69-a683-2e9bc4e8f617\") " pod="openstack-kuttl-tests/barbican-api-b857b994d-6xzbl" Dec 03 12:34:46 crc kubenswrapper[5052]: I1203 12:34:46.773661 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-khg4m\" (UniqueName: \"kubernetes.io/projected/3ceac470-11eb-4d69-a683-2e9bc4e8f617-kube-api-access-khg4m\") pod \"barbican-api-b857b994d-6xzbl\" (UID: \"3ceac470-11eb-4d69-a683-2e9bc4e8f617\") " pod="openstack-kuttl-tests/barbican-api-b857b994d-6xzbl" Dec 03 12:34:46 crc kubenswrapper[5052]: I1203 12:34:46.875250 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-khg4m\" (UniqueName: \"kubernetes.io/projected/3ceac470-11eb-4d69-a683-2e9bc4e8f617-kube-api-access-khg4m\") pod \"barbican-api-b857b994d-6xzbl\" (UID: \"3ceac470-11eb-4d69-a683-2e9bc4e8f617\") " pod="openstack-kuttl-tests/barbican-api-b857b994d-6xzbl" Dec 03 12:34:46 crc kubenswrapper[5052]: I1203 12:34:46.875286 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/3ceac470-11eb-4d69-a683-2e9bc4e8f617-public-tls-certs\") pod \"barbican-api-b857b994d-6xzbl\" (UID: \"3ceac470-11eb-4d69-a683-2e9bc4e8f617\") " pod="openstack-kuttl-tests/barbican-api-b857b994d-6xzbl" Dec 03 12:34:46 crc kubenswrapper[5052]: I1203 12:34:46.875314 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/3ceac470-11eb-4d69-a683-2e9bc4e8f617-config-data-custom\") pod \"barbican-api-b857b994d-6xzbl\" (UID: \"3ceac470-11eb-4d69-a683-2e9bc4e8f617\") " pod="openstack-kuttl-tests/barbican-api-b857b994d-6xzbl" Dec 03 12:34:46 crc kubenswrapper[5052]: I1203 12:34:46.875332 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/3ceac470-11eb-4d69-a683-2e9bc4e8f617-internal-tls-certs\") pod \"barbican-api-b857b994d-6xzbl\" (UID: \"3ceac470-11eb-4d69-a683-2e9bc4e8f617\") " pod="openstack-kuttl-tests/barbican-api-b857b994d-6xzbl" Dec 03 12:34:46 crc kubenswrapper[5052]: I1203 12:34:46.875408 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3ceac470-11eb-4d69-a683-2e9bc4e8f617-combined-ca-bundle\") pod \"barbican-api-b857b994d-6xzbl\" (UID: \"3ceac470-11eb-4d69-a683-2e9bc4e8f617\") " pod="openstack-kuttl-tests/barbican-api-b857b994d-6xzbl" Dec 03 12:34:46 crc kubenswrapper[5052]: I1203 12:34:46.875461 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/3ceac470-11eb-4d69-a683-2e9bc4e8f617-logs\") pod \"barbican-api-b857b994d-6xzbl\" (UID: \"3ceac470-11eb-4d69-a683-2e9bc4e8f617\") " pod="openstack-kuttl-tests/barbican-api-b857b994d-6xzbl" Dec 03 12:34:46 crc kubenswrapper[5052]: I1203 12:34:46.875503 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/3ceac470-11eb-4d69-a683-2e9bc4e8f617-config-data\") pod \"barbican-api-b857b994d-6xzbl\" (UID: \"3ceac470-11eb-4d69-a683-2e9bc4e8f617\") " pod="openstack-kuttl-tests/barbican-api-b857b994d-6xzbl" Dec 03 12:34:46 crc kubenswrapper[5052]: I1203 12:34:46.876645 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/3ceac470-11eb-4d69-a683-2e9bc4e8f617-logs\") pod \"barbican-api-b857b994d-6xzbl\" (UID: \"3ceac470-11eb-4d69-a683-2e9bc4e8f617\") " pod="openstack-kuttl-tests/barbican-api-b857b994d-6xzbl" Dec 03 12:34:46 crc kubenswrapper[5052]: I1203 12:34:46.898239 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/3ceac470-11eb-4d69-a683-2e9bc4e8f617-internal-tls-certs\") pod \"barbican-api-b857b994d-6xzbl\" (UID: \"3ceac470-11eb-4d69-a683-2e9bc4e8f617\") " pod="openstack-kuttl-tests/barbican-api-b857b994d-6xzbl" Dec 03 12:34:46 crc kubenswrapper[5052]: I1203 12:34:46.898925 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/3ceac470-11eb-4d69-a683-2e9bc4e8f617-config-data-custom\") pod \"barbican-api-b857b994d-6xzbl\" (UID: \"3ceac470-11eb-4d69-a683-2e9bc4e8f617\") " pod="openstack-kuttl-tests/barbican-api-b857b994d-6xzbl" Dec 03 12:34:46 crc kubenswrapper[5052]: I1203 12:34:46.898931 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3ceac470-11eb-4d69-a683-2e9bc4e8f617-combined-ca-bundle\") pod \"barbican-api-b857b994d-6xzbl\" (UID: \"3ceac470-11eb-4d69-a683-2e9bc4e8f617\") " pod="openstack-kuttl-tests/barbican-api-b857b994d-6xzbl" Dec 03 12:34:46 crc kubenswrapper[5052]: I1203 12:34:46.899944 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/3ceac470-11eb-4d69-a683-2e9bc4e8f617-config-data\") pod \"barbican-api-b857b994d-6xzbl\" (UID: \"3ceac470-11eb-4d69-a683-2e9bc4e8f617\") " pod="openstack-kuttl-tests/barbican-api-b857b994d-6xzbl" Dec 03 12:34:46 crc kubenswrapper[5052]: I1203 12:34:46.901845 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-khg4m\" (UniqueName: \"kubernetes.io/projected/3ceac470-11eb-4d69-a683-2e9bc4e8f617-kube-api-access-khg4m\") pod \"barbican-api-b857b994d-6xzbl\" (UID: \"3ceac470-11eb-4d69-a683-2e9bc4e8f617\") " pod="openstack-kuttl-tests/barbican-api-b857b994d-6xzbl" Dec 03 12:34:46 crc kubenswrapper[5052]: I1203 12:34:46.902257 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/3ceac470-11eb-4d69-a683-2e9bc4e8f617-public-tls-certs\") pod \"barbican-api-b857b994d-6xzbl\" (UID: \"3ceac470-11eb-4d69-a683-2e9bc4e8f617\") " pod="openstack-kuttl-tests/barbican-api-b857b994d-6xzbl" Dec 03 12:34:46 crc kubenswrapper[5052]: I1203 12:34:46.977778 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-db-sync-s42b9" Dec 03 12:34:46 crc kubenswrapper[5052]: I1203 12:34:46.991390 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-api-b857b994d-6xzbl" Dec 03 12:34:47 crc kubenswrapper[5052]: I1203 12:34:47.079733 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nwfzf\" (UniqueName: \"kubernetes.io/projected/55ccf323-53e4-47fe-bea8-a17273b8587c-kube-api-access-nwfzf\") pod \"55ccf323-53e4-47fe-bea8-a17273b8587c\" (UID: \"55ccf323-53e4-47fe-bea8-a17273b8587c\") " Dec 03 12:34:47 crc kubenswrapper[5052]: I1203 12:34:47.079822 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/55ccf323-53e4-47fe-bea8-a17273b8587c-etc-machine-id\") pod \"55ccf323-53e4-47fe-bea8-a17273b8587c\" (UID: \"55ccf323-53e4-47fe-bea8-a17273b8587c\") " Dec 03 12:34:47 crc kubenswrapper[5052]: I1203 12:34:47.079874 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/55ccf323-53e4-47fe-bea8-a17273b8587c-combined-ca-bundle\") pod \"55ccf323-53e4-47fe-bea8-a17273b8587c\" (UID: \"55ccf323-53e4-47fe-bea8-a17273b8587c\") " Dec 03 12:34:47 crc kubenswrapper[5052]: I1203 12:34:47.079936 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/55ccf323-53e4-47fe-bea8-a17273b8587c-db-sync-config-data\") pod \"55ccf323-53e4-47fe-bea8-a17273b8587c\" (UID: \"55ccf323-53e4-47fe-bea8-a17273b8587c\") " Dec 03 12:34:47 crc kubenswrapper[5052]: I1203 12:34:47.080107 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/55ccf323-53e4-47fe-bea8-a17273b8587c-config-data\") pod \"55ccf323-53e4-47fe-bea8-a17273b8587c\" (UID: \"55ccf323-53e4-47fe-bea8-a17273b8587c\") " Dec 03 12:34:47 crc kubenswrapper[5052]: I1203 12:34:47.080199 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/55ccf323-53e4-47fe-bea8-a17273b8587c-scripts\") pod \"55ccf323-53e4-47fe-bea8-a17273b8587c\" (UID: \"55ccf323-53e4-47fe-bea8-a17273b8587c\") " Dec 03 12:34:47 crc kubenswrapper[5052]: I1203 12:34:47.085208 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/55ccf323-53e4-47fe-bea8-a17273b8587c-etc-machine-id" (OuterVolumeSpecName: "etc-machine-id") pod "55ccf323-53e4-47fe-bea8-a17273b8587c" (UID: "55ccf323-53e4-47fe-bea8-a17273b8587c"). InnerVolumeSpecName "etc-machine-id". PluginName "kubernetes.io/host-path", VolumeGidValue "" Dec 03 12:34:47 crc kubenswrapper[5052]: I1203 12:34:47.090120 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/55ccf323-53e4-47fe-bea8-a17273b8587c-scripts" (OuterVolumeSpecName: "scripts") pod "55ccf323-53e4-47fe-bea8-a17273b8587c" (UID: "55ccf323-53e4-47fe-bea8-a17273b8587c"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:34:47 crc kubenswrapper[5052]: I1203 12:34:47.092612 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/55ccf323-53e4-47fe-bea8-a17273b8587c-db-sync-config-data" (OuterVolumeSpecName: "db-sync-config-data") pod "55ccf323-53e4-47fe-bea8-a17273b8587c" (UID: "55ccf323-53e4-47fe-bea8-a17273b8587c"). InnerVolumeSpecName "db-sync-config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:34:47 crc kubenswrapper[5052]: I1203 12:34:47.092783 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/55ccf323-53e4-47fe-bea8-a17273b8587c-kube-api-access-nwfzf" (OuterVolumeSpecName: "kube-api-access-nwfzf") pod "55ccf323-53e4-47fe-bea8-a17273b8587c" (UID: "55ccf323-53e4-47fe-bea8-a17273b8587c"). InnerVolumeSpecName "kube-api-access-nwfzf". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:34:47 crc kubenswrapper[5052]: I1203 12:34:47.119919 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/55ccf323-53e4-47fe-bea8-a17273b8587c-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "55ccf323-53e4-47fe-bea8-a17273b8587c" (UID: "55ccf323-53e4-47fe-bea8-a17273b8587c"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:34:47 crc kubenswrapper[5052]: I1203 12:34:47.136945 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/55ccf323-53e4-47fe-bea8-a17273b8587c-config-data" (OuterVolumeSpecName: "config-data") pod "55ccf323-53e4-47fe-bea8-a17273b8587c" (UID: "55ccf323-53e4-47fe-bea8-a17273b8587c"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:34:47 crc kubenswrapper[5052]: I1203 12:34:47.184077 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/55ccf323-53e4-47fe-bea8-a17273b8587c-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:34:47 crc kubenswrapper[5052]: I1203 12:34:47.184151 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-nwfzf\" (UniqueName: \"kubernetes.io/projected/55ccf323-53e4-47fe-bea8-a17273b8587c-kube-api-access-nwfzf\") on node \"crc\" DevicePath \"\"" Dec 03 12:34:47 crc kubenswrapper[5052]: I1203 12:34:47.184170 5052 reconciler_common.go:293] "Volume detached for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/55ccf323-53e4-47fe-bea8-a17273b8587c-etc-machine-id\") on node \"crc\" DevicePath \"\"" Dec 03 12:34:47 crc kubenswrapper[5052]: I1203 12:34:47.184186 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/55ccf323-53e4-47fe-bea8-a17273b8587c-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:34:47 crc kubenswrapper[5052]: I1203 12:34:47.184200 5052 reconciler_common.go:293] "Volume detached for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/55ccf323-53e4-47fe-bea8-a17273b8587c-db-sync-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:34:47 crc kubenswrapper[5052]: I1203 12:34:47.184213 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/55ccf323-53e4-47fe-bea8-a17273b8587c-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:34:47 crc kubenswrapper[5052]: W1203 12:34:47.463446 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3ceac470_11eb_4d69_a683_2e9bc4e8f617.slice/crio-2da51349cc9b8a1c7a1cb9bf175483e9aae3f05e945d00d68bc9cef218210bc8 WatchSource:0}: Error finding container 2da51349cc9b8a1c7a1cb9bf175483e9aae3f05e945d00d68bc9cef218210bc8: Status 404 returned error can't find the container with id 2da51349cc9b8a1c7a1cb9bf175483e9aae3f05e945d00d68bc9cef218210bc8 Dec 03 12:34:47 crc kubenswrapper[5052]: I1203 12:34:47.463942 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-api-b857b994d-6xzbl"] Dec 03 12:34:47 crc kubenswrapper[5052]: I1203 12:34:47.612468 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-keystone-listener-8d6c85ffd-x99nt" event={"ID":"f8e97a34-9689-4f0f-b43e-f87b43a115c8","Type":"ContainerStarted","Data":"122c1c84abc5c24f0d3838dc8447ee559106f50fd8abb559d37b19cf2c31cffa"} Dec 03 12:34:47 crc kubenswrapper[5052]: I1203 12:34:47.612548 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-keystone-listener-8d6c85ffd-x99nt" event={"ID":"f8e97a34-9689-4f0f-b43e-f87b43a115c8","Type":"ContainerStarted","Data":"cdb963a5681bb97da6125afe6f345d268c65ca0e93d883fb5df0751413fccc4e"} Dec 03 12:34:47 crc kubenswrapper[5052]: I1203 12:34:47.614779 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-5656894b88-g5p25" event={"ID":"8a22a52b-d45b-4678-900d-a9d6d5113da3","Type":"ContainerStarted","Data":"81f54b8a5b8caf3d328e09e17a0f8fcd598cbfa32b1af39536bc30c4637e930a"} Dec 03 12:34:47 crc kubenswrapper[5052]: I1203 12:34:47.614802 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-5656894b88-g5p25" event={"ID":"8a22a52b-d45b-4678-900d-a9d6d5113da3","Type":"ContainerStarted","Data":"cca3f4b446ecb9d98e6d2c384dbfc5000a4f123199962b788a74ab88d347c76b"} Dec 03 12:34:47 crc kubenswrapper[5052]: I1203 12:34:47.614879 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/barbican-api-5656894b88-g5p25" Dec 03 12:34:47 crc kubenswrapper[5052]: I1203 12:34:47.614905 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/barbican-api-5656894b88-g5p25" Dec 03 12:34:47 crc kubenswrapper[5052]: I1203 12:34:47.616139 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-b857b994d-6xzbl" event={"ID":"3ceac470-11eb-4d69-a683-2e9bc4e8f617","Type":"ContainerStarted","Data":"2da51349cc9b8a1c7a1cb9bf175483e9aae3f05e945d00d68bc9cef218210bc8"} Dec 03 12:34:47 crc kubenswrapper[5052]: I1203 12:34:47.617575 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-worker-7bd4f6f96f-w4glb" event={"ID":"15415292-9b86-462f-afe6-7f688c84805b","Type":"ContainerStarted","Data":"0dc0f37a65ed55aa8edaf8d39bd7ee49b39f6566a3fd737dfc57db3b421f9551"} Dec 03 12:34:47 crc kubenswrapper[5052]: I1203 12:34:47.617598 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-worker-7bd4f6f96f-w4glb" event={"ID":"15415292-9b86-462f-afe6-7f688c84805b","Type":"ContainerStarted","Data":"0f04069bde4fecc56a28786a8ae81b79a9b8cacf126174aabf4a5e2108942a49"} Dec 03 12:34:47 crc kubenswrapper[5052]: I1203 12:34:47.630739 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-db-sync-s42b9" Dec 03 12:34:47 crc kubenswrapper[5052]: I1203 12:34:47.630845 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-db-sync-s42b9" event={"ID":"55ccf323-53e4-47fe-bea8-a17273b8587c","Type":"ContainerDied","Data":"1a703be9bb20d985462a2e0555794ece28a51df5b49803a0b4a8f1399091a5d2"} Dec 03 12:34:47 crc kubenswrapper[5052]: I1203 12:34:47.630881 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1a703be9bb20d985462a2e0555794ece28a51df5b49803a0b4a8f1399091a5d2" Dec 03 12:34:47 crc kubenswrapper[5052]: I1203 12:34:47.703876 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/barbican-keystone-listener-8d6c85ffd-x99nt" podStartSLOduration=4.703846896 podStartE2EDuration="4.703846896s" podCreationTimestamp="2025-12-03 12:34:43 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:34:47.649339207 +0000 UTC m=+4478.237580991" watchObservedRunningTime="2025-12-03 12:34:47.703846896 +0000 UTC m=+4478.292088660" Dec 03 12:34:47 crc kubenswrapper[5052]: I1203 12:34:47.733017 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/barbican-worker-7bd4f6f96f-w4glb" podStartSLOduration=4.732996579 podStartE2EDuration="4.732996579s" podCreationTimestamp="2025-12-03 12:34:43 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:34:47.699043518 +0000 UTC m=+4478.287285282" watchObservedRunningTime="2025-12-03 12:34:47.732996579 +0000 UTC m=+4478.321238343" Dec 03 12:34:47 crc kubenswrapper[5052]: I1203 12:34:47.745732 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/barbican-api-5656894b88-g5p25" podStartSLOduration=4.745711393 podStartE2EDuration="4.745711393s" podCreationTimestamp="2025-12-03 12:34:43 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:34:47.727183823 +0000 UTC m=+4478.315425577" watchObservedRunningTime="2025-12-03 12:34:47.745711393 +0000 UTC m=+4478.333953157" Dec 03 12:34:47 crc kubenswrapper[5052]: I1203 12:34:47.913615 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 12:34:47 crc kubenswrapper[5052]: E1203 12:34:47.920833 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="55ccf323-53e4-47fe-bea8-a17273b8587c" containerName="cinder-db-sync" Dec 03 12:34:47 crc kubenswrapper[5052]: I1203 12:34:47.920863 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="55ccf323-53e4-47fe-bea8-a17273b8587c" containerName="cinder-db-sync" Dec 03 12:34:47 crc kubenswrapper[5052]: I1203 12:34:47.922505 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="55ccf323-53e4-47fe-bea8-a17273b8587c" containerName="cinder-db-sync" Dec 03 12:34:47 crc kubenswrapper[5052]: I1203 12:34:47.923862 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:34:47 crc kubenswrapper[5052]: I1203 12:34:47.935654 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cinder-cinder-dockercfg-8v2d7" Dec 03 12:34:47 crc kubenswrapper[5052]: I1203 12:34:47.935870 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cinder-scheduler-config-data" Dec 03 12:34:47 crc kubenswrapper[5052]: I1203 12:34:47.935964 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cinder-config-data" Dec 03 12:34:47 crc kubenswrapper[5052]: I1203 12:34:47.938822 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cinder-scripts" Dec 03 12:34:47 crc kubenswrapper[5052]: I1203 12:34:47.975145 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 12:34:48 crc kubenswrapper[5052]: I1203 12:34:48.083373 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 12:34:48 crc kubenswrapper[5052]: I1203 12:34:48.085081 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:34:48 crc kubenswrapper[5052]: I1203 12:34:48.086760 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cinder-api-config-data" Dec 03 12:34:48 crc kubenswrapper[5052]: I1203 12:34:48.091809 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 12:34:48 crc kubenswrapper[5052]: I1203 12:34:48.108403 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/2793c42c-363d-4495-88d8-4385bfc9c3fc-config-data\") pod \"cinder-scheduler-0\" (UID: \"2793c42c-363d-4495-88d8-4385bfc9c3fc\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:34:48 crc kubenswrapper[5052]: I1203 12:34:48.108452 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/2793c42c-363d-4495-88d8-4385bfc9c3fc-scripts\") pod \"cinder-scheduler-0\" (UID: \"2793c42c-363d-4495-88d8-4385bfc9c3fc\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:34:48 crc kubenswrapper[5052]: I1203 12:34:48.108477 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/2793c42c-363d-4495-88d8-4385bfc9c3fc-config-data-custom\") pod \"cinder-scheduler-0\" (UID: \"2793c42c-363d-4495-88d8-4385bfc9c3fc\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:34:48 crc kubenswrapper[5052]: I1203 12:34:48.108505 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2793c42c-363d-4495-88d8-4385bfc9c3fc-combined-ca-bundle\") pod \"cinder-scheduler-0\" (UID: \"2793c42c-363d-4495-88d8-4385bfc9c3fc\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:34:48 crc kubenswrapper[5052]: I1203 12:34:48.110906 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7kmjb\" (UniqueName: \"kubernetes.io/projected/2793c42c-363d-4495-88d8-4385bfc9c3fc-kube-api-access-7kmjb\") pod \"cinder-scheduler-0\" (UID: \"2793c42c-363d-4495-88d8-4385bfc9c3fc\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:34:48 crc kubenswrapper[5052]: I1203 12:34:48.111052 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/2793c42c-363d-4495-88d8-4385bfc9c3fc-etc-machine-id\") pod \"cinder-scheduler-0\" (UID: \"2793c42c-363d-4495-88d8-4385bfc9c3fc\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:34:48 crc kubenswrapper[5052]: I1203 12:34:48.183183 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-bootstrap-9grd9" Dec 03 12:34:48 crc kubenswrapper[5052]: I1203 12:34:48.214679 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/e5a85223-b8cf-471c-92d1-a4da3eae9649-config-data\") pod \"cinder-api-0\" (UID: \"e5a85223-b8cf-471c-92d1-a4da3eae9649\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:34:48 crc kubenswrapper[5052]: I1203 12:34:48.214737 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/2793c42c-363d-4495-88d8-4385bfc9c3fc-config-data\") pod \"cinder-scheduler-0\" (UID: \"2793c42c-363d-4495-88d8-4385bfc9c3fc\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:34:48 crc kubenswrapper[5052]: I1203 12:34:48.214770 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/e5a85223-b8cf-471c-92d1-a4da3eae9649-logs\") pod \"cinder-api-0\" (UID: \"e5a85223-b8cf-471c-92d1-a4da3eae9649\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:34:48 crc kubenswrapper[5052]: I1203 12:34:48.214920 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/2793c42c-363d-4495-88d8-4385bfc9c3fc-scripts\") pod \"cinder-scheduler-0\" (UID: \"2793c42c-363d-4495-88d8-4385bfc9c3fc\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:34:48 crc kubenswrapper[5052]: I1203 12:34:48.214988 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/2793c42c-363d-4495-88d8-4385bfc9c3fc-config-data-custom\") pod \"cinder-scheduler-0\" (UID: \"2793c42c-363d-4495-88d8-4385bfc9c3fc\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:34:48 crc kubenswrapper[5052]: I1203 12:34:48.215034 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/e5a85223-b8cf-471c-92d1-a4da3eae9649-etc-machine-id\") pod \"cinder-api-0\" (UID: \"e5a85223-b8cf-471c-92d1-a4da3eae9649\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:34:48 crc kubenswrapper[5052]: I1203 12:34:48.215095 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2793c42c-363d-4495-88d8-4385bfc9c3fc-combined-ca-bundle\") pod \"cinder-scheduler-0\" (UID: \"2793c42c-363d-4495-88d8-4385bfc9c3fc\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:34:48 crc kubenswrapper[5052]: I1203 12:34:48.215314 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/e5a85223-b8cf-471c-92d1-a4da3eae9649-scripts\") pod \"cinder-api-0\" (UID: \"e5a85223-b8cf-471c-92d1-a4da3eae9649\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:34:48 crc kubenswrapper[5052]: I1203 12:34:48.215839 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-7kmjb\" (UniqueName: \"kubernetes.io/projected/2793c42c-363d-4495-88d8-4385bfc9c3fc-kube-api-access-7kmjb\") pod \"cinder-scheduler-0\" (UID: \"2793c42c-363d-4495-88d8-4385bfc9c3fc\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:34:48 crc kubenswrapper[5052]: I1203 12:34:48.215982 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/e5a85223-b8cf-471c-92d1-a4da3eae9649-config-data-custom\") pod \"cinder-api-0\" (UID: \"e5a85223-b8cf-471c-92d1-a4da3eae9649\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:34:48 crc kubenswrapper[5052]: I1203 12:34:48.216002 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tbptv\" (UniqueName: \"kubernetes.io/projected/e5a85223-b8cf-471c-92d1-a4da3eae9649-kube-api-access-tbptv\") pod \"cinder-api-0\" (UID: \"e5a85223-b8cf-471c-92d1-a4da3eae9649\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:34:48 crc kubenswrapper[5052]: I1203 12:34:48.216028 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/2793c42c-363d-4495-88d8-4385bfc9c3fc-etc-machine-id\") pod \"cinder-scheduler-0\" (UID: \"2793c42c-363d-4495-88d8-4385bfc9c3fc\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:34:48 crc kubenswrapper[5052]: I1203 12:34:48.216069 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e5a85223-b8cf-471c-92d1-a4da3eae9649-combined-ca-bundle\") pod \"cinder-api-0\" (UID: \"e5a85223-b8cf-471c-92d1-a4da3eae9649\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:34:48 crc kubenswrapper[5052]: I1203 12:34:48.216737 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/2793c42c-363d-4495-88d8-4385bfc9c3fc-etc-machine-id\") pod \"cinder-scheduler-0\" (UID: \"2793c42c-363d-4495-88d8-4385bfc9c3fc\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:34:48 crc kubenswrapper[5052]: I1203 12:34:48.219534 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/2793c42c-363d-4495-88d8-4385bfc9c3fc-scripts\") pod \"cinder-scheduler-0\" (UID: \"2793c42c-363d-4495-88d8-4385bfc9c3fc\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:34:48 crc kubenswrapper[5052]: I1203 12:34:48.219840 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/2793c42c-363d-4495-88d8-4385bfc9c3fc-config-data-custom\") pod \"cinder-scheduler-0\" (UID: \"2793c42c-363d-4495-88d8-4385bfc9c3fc\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:34:48 crc kubenswrapper[5052]: I1203 12:34:48.220588 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/2793c42c-363d-4495-88d8-4385bfc9c3fc-config-data\") pod \"cinder-scheduler-0\" (UID: \"2793c42c-363d-4495-88d8-4385bfc9c3fc\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:34:48 crc kubenswrapper[5052]: I1203 12:34:48.221414 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2793c42c-363d-4495-88d8-4385bfc9c3fc-combined-ca-bundle\") pod \"cinder-scheduler-0\" (UID: \"2793c42c-363d-4495-88d8-4385bfc9c3fc\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:34:48 crc kubenswrapper[5052]: I1203 12:34:48.237001 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-7kmjb\" (UniqueName: \"kubernetes.io/projected/2793c42c-363d-4495-88d8-4385bfc9c3fc-kube-api-access-7kmjb\") pod \"cinder-scheduler-0\" (UID: \"2793c42c-363d-4495-88d8-4385bfc9c3fc\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:34:48 crc kubenswrapper[5052]: I1203 12:34:48.285873 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:34:48 crc kubenswrapper[5052]: I1203 12:34:48.318448 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q2h8j\" (UniqueName: \"kubernetes.io/projected/0a3ff09e-7256-4090-9060-099706213333-kube-api-access-q2h8j\") pod \"0a3ff09e-7256-4090-9060-099706213333\" (UID: \"0a3ff09e-7256-4090-9060-099706213333\") " Dec 03 12:34:48 crc kubenswrapper[5052]: I1203 12:34:48.318536 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/0a3ff09e-7256-4090-9060-099706213333-credential-keys\") pod \"0a3ff09e-7256-4090-9060-099706213333\" (UID: \"0a3ff09e-7256-4090-9060-099706213333\") " Dec 03 12:34:48 crc kubenswrapper[5052]: I1203 12:34:48.318574 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/0a3ff09e-7256-4090-9060-099706213333-config-data\") pod \"0a3ff09e-7256-4090-9060-099706213333\" (UID: \"0a3ff09e-7256-4090-9060-099706213333\") " Dec 03 12:34:48 crc kubenswrapper[5052]: I1203 12:34:48.318696 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0a3ff09e-7256-4090-9060-099706213333-combined-ca-bundle\") pod \"0a3ff09e-7256-4090-9060-099706213333\" (UID: \"0a3ff09e-7256-4090-9060-099706213333\") " Dec 03 12:34:48 crc kubenswrapper[5052]: I1203 12:34:48.318730 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/0a3ff09e-7256-4090-9060-099706213333-scripts\") pod \"0a3ff09e-7256-4090-9060-099706213333\" (UID: \"0a3ff09e-7256-4090-9060-099706213333\") " Dec 03 12:34:48 crc kubenswrapper[5052]: I1203 12:34:48.318768 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/0a3ff09e-7256-4090-9060-099706213333-fernet-keys\") pod \"0a3ff09e-7256-4090-9060-099706213333\" (UID: \"0a3ff09e-7256-4090-9060-099706213333\") " Dec 03 12:34:48 crc kubenswrapper[5052]: I1203 12:34:48.318996 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/e5a85223-b8cf-471c-92d1-a4da3eae9649-logs\") pod \"cinder-api-0\" (UID: \"e5a85223-b8cf-471c-92d1-a4da3eae9649\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:34:48 crc kubenswrapper[5052]: I1203 12:34:48.319050 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/e5a85223-b8cf-471c-92d1-a4da3eae9649-etc-machine-id\") pod \"cinder-api-0\" (UID: \"e5a85223-b8cf-471c-92d1-a4da3eae9649\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:34:48 crc kubenswrapper[5052]: I1203 12:34:48.319143 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/e5a85223-b8cf-471c-92d1-a4da3eae9649-scripts\") pod \"cinder-api-0\" (UID: \"e5a85223-b8cf-471c-92d1-a4da3eae9649\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:34:48 crc kubenswrapper[5052]: I1203 12:34:48.319205 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/e5a85223-b8cf-471c-92d1-a4da3eae9649-etc-machine-id\") pod \"cinder-api-0\" (UID: \"e5a85223-b8cf-471c-92d1-a4da3eae9649\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:34:48 crc kubenswrapper[5052]: I1203 12:34:48.319230 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/e5a85223-b8cf-471c-92d1-a4da3eae9649-config-data-custom\") pod \"cinder-api-0\" (UID: \"e5a85223-b8cf-471c-92d1-a4da3eae9649\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:34:48 crc kubenswrapper[5052]: I1203 12:34:48.319262 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-tbptv\" (UniqueName: \"kubernetes.io/projected/e5a85223-b8cf-471c-92d1-a4da3eae9649-kube-api-access-tbptv\") pod \"cinder-api-0\" (UID: \"e5a85223-b8cf-471c-92d1-a4da3eae9649\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:34:48 crc kubenswrapper[5052]: I1203 12:34:48.319304 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e5a85223-b8cf-471c-92d1-a4da3eae9649-combined-ca-bundle\") pod \"cinder-api-0\" (UID: \"e5a85223-b8cf-471c-92d1-a4da3eae9649\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:34:48 crc kubenswrapper[5052]: I1203 12:34:48.319372 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/e5a85223-b8cf-471c-92d1-a4da3eae9649-config-data\") pod \"cinder-api-0\" (UID: \"e5a85223-b8cf-471c-92d1-a4da3eae9649\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:34:48 crc kubenswrapper[5052]: I1203 12:34:48.319416 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/e5a85223-b8cf-471c-92d1-a4da3eae9649-logs\") pod \"cinder-api-0\" (UID: \"e5a85223-b8cf-471c-92d1-a4da3eae9649\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:34:48 crc kubenswrapper[5052]: I1203 12:34:48.328111 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0a3ff09e-7256-4090-9060-099706213333-kube-api-access-q2h8j" (OuterVolumeSpecName: "kube-api-access-q2h8j") pod "0a3ff09e-7256-4090-9060-099706213333" (UID: "0a3ff09e-7256-4090-9060-099706213333"). InnerVolumeSpecName "kube-api-access-q2h8j". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:34:48 crc kubenswrapper[5052]: I1203 12:34:48.328094 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0a3ff09e-7256-4090-9060-099706213333-fernet-keys" (OuterVolumeSpecName: "fernet-keys") pod "0a3ff09e-7256-4090-9060-099706213333" (UID: "0a3ff09e-7256-4090-9060-099706213333"). InnerVolumeSpecName "fernet-keys". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:34:48 crc kubenswrapper[5052]: I1203 12:34:48.329011 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/e5a85223-b8cf-471c-92d1-a4da3eae9649-config-data\") pod \"cinder-api-0\" (UID: \"e5a85223-b8cf-471c-92d1-a4da3eae9649\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:34:48 crc kubenswrapper[5052]: I1203 12:34:48.329164 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e5a85223-b8cf-471c-92d1-a4da3eae9649-combined-ca-bundle\") pod \"cinder-api-0\" (UID: \"e5a85223-b8cf-471c-92d1-a4da3eae9649\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:34:48 crc kubenswrapper[5052]: I1203 12:34:48.330745 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/e5a85223-b8cf-471c-92d1-a4da3eae9649-config-data-custom\") pod \"cinder-api-0\" (UID: \"e5a85223-b8cf-471c-92d1-a4da3eae9649\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:34:48 crc kubenswrapper[5052]: I1203 12:34:48.335272 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0a3ff09e-7256-4090-9060-099706213333-credential-keys" (OuterVolumeSpecName: "credential-keys") pod "0a3ff09e-7256-4090-9060-099706213333" (UID: "0a3ff09e-7256-4090-9060-099706213333"). InnerVolumeSpecName "credential-keys". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:34:48 crc kubenswrapper[5052]: I1203 12:34:48.338394 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-tbptv\" (UniqueName: \"kubernetes.io/projected/e5a85223-b8cf-471c-92d1-a4da3eae9649-kube-api-access-tbptv\") pod \"cinder-api-0\" (UID: \"e5a85223-b8cf-471c-92d1-a4da3eae9649\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:34:48 crc kubenswrapper[5052]: I1203 12:34:48.340376 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0a3ff09e-7256-4090-9060-099706213333-scripts" (OuterVolumeSpecName: "scripts") pod "0a3ff09e-7256-4090-9060-099706213333" (UID: "0a3ff09e-7256-4090-9060-099706213333"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:34:48 crc kubenswrapper[5052]: I1203 12:34:48.341772 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/e5a85223-b8cf-471c-92d1-a4da3eae9649-scripts\") pod \"cinder-api-0\" (UID: \"e5a85223-b8cf-471c-92d1-a4da3eae9649\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:34:48 crc kubenswrapper[5052]: I1203 12:34:48.358195 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0a3ff09e-7256-4090-9060-099706213333-config-data" (OuterVolumeSpecName: "config-data") pod "0a3ff09e-7256-4090-9060-099706213333" (UID: "0a3ff09e-7256-4090-9060-099706213333"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:34:48 crc kubenswrapper[5052]: I1203 12:34:48.358884 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0a3ff09e-7256-4090-9060-099706213333-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "0a3ff09e-7256-4090-9060-099706213333" (UID: "0a3ff09e-7256-4090-9060-099706213333"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:34:48 crc kubenswrapper[5052]: I1203 12:34:48.421754 5052 reconciler_common.go:293] "Volume detached for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/0a3ff09e-7256-4090-9060-099706213333-fernet-keys\") on node \"crc\" DevicePath \"\"" Dec 03 12:34:48 crc kubenswrapper[5052]: I1203 12:34:48.421792 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-q2h8j\" (UniqueName: \"kubernetes.io/projected/0a3ff09e-7256-4090-9060-099706213333-kube-api-access-q2h8j\") on node \"crc\" DevicePath \"\"" Dec 03 12:34:48 crc kubenswrapper[5052]: I1203 12:34:48.421808 5052 reconciler_common.go:293] "Volume detached for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/0a3ff09e-7256-4090-9060-099706213333-credential-keys\") on node \"crc\" DevicePath \"\"" Dec 03 12:34:48 crc kubenswrapper[5052]: I1203 12:34:48.421819 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/0a3ff09e-7256-4090-9060-099706213333-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:34:48 crc kubenswrapper[5052]: I1203 12:34:48.421830 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0a3ff09e-7256-4090-9060-099706213333-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:34:48 crc kubenswrapper[5052]: I1203 12:34:48.422074 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/0a3ff09e-7256-4090-9060-099706213333-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:34:48 crc kubenswrapper[5052]: I1203 12:34:48.439451 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:34:48 crc kubenswrapper[5052]: I1203 12:34:48.659930 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-bootstrap-9grd9" event={"ID":"0a3ff09e-7256-4090-9060-099706213333","Type":"ContainerDied","Data":"9c71a35a9a24539d037ad2d5bd1b4d452359582ad28f32dff522ce818b0f1f5a"} Dec 03 12:34:48 crc kubenswrapper[5052]: I1203 12:34:48.659998 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9c71a35a9a24539d037ad2d5bd1b4d452359582ad28f32dff522ce818b0f1f5a" Dec 03 12:34:48 crc kubenswrapper[5052]: I1203 12:34:48.660603 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-bootstrap-9grd9" Dec 03 12:34:48 crc kubenswrapper[5052]: I1203 12:34:48.662422 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-b857b994d-6xzbl" event={"ID":"3ceac470-11eb-4d69-a683-2e9bc4e8f617","Type":"ContainerStarted","Data":"fc0930236187285aa345d3839c9e4d90b9b3da00e5015ee57851c76c3228e9b7"} Dec 03 12:34:48 crc kubenswrapper[5052]: I1203 12:34:48.662480 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-b857b994d-6xzbl" event={"ID":"3ceac470-11eb-4d69-a683-2e9bc4e8f617","Type":"ContainerStarted","Data":"1fe0c8f8d952b7b1bbdf2ddc25bf96bcb1eeeec47a75464c77ed9ef77f45fc86"} Dec 03 12:34:48 crc kubenswrapper[5052]: I1203 12:34:48.734177 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/barbican-api-b857b994d-6xzbl" podStartSLOduration=2.734155821 podStartE2EDuration="2.734155821s" podCreationTimestamp="2025-12-03 12:34:46 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:34:48.689825643 +0000 UTC m=+4479.278067407" watchObservedRunningTime="2025-12-03 12:34:48.734155821 +0000 UTC m=+4479.322397585" Dec 03 12:34:48 crc kubenswrapper[5052]: I1203 12:34:48.771247 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 12:34:48 crc kubenswrapper[5052]: W1203 12:34:48.778180 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2793c42c_363d_4495_88d8_4385bfc9c3fc.slice/crio-7634a1171510b5a7027061193ec392fdd1fab2f90deb2accca4c473913c2b91c WatchSource:0}: Error finding container 7634a1171510b5a7027061193ec392fdd1fab2f90deb2accca4c473913c2b91c: Status 404 returned error can't find the container with id 7634a1171510b5a7027061193ec392fdd1fab2f90deb2accca4c473913c2b91c Dec 03 12:34:48 crc kubenswrapper[5052]: I1203 12:34:48.846063 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/keystone-7476b7fc95-6cxvn"] Dec 03 12:34:48 crc kubenswrapper[5052]: E1203 12:34:48.846611 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="0a3ff09e-7256-4090-9060-099706213333" containerName="keystone-bootstrap" Dec 03 12:34:48 crc kubenswrapper[5052]: I1203 12:34:48.846634 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a3ff09e-7256-4090-9060-099706213333" containerName="keystone-bootstrap" Dec 03 12:34:48 crc kubenswrapper[5052]: I1203 12:34:48.846845 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="0a3ff09e-7256-4090-9060-099706213333" containerName="keystone-bootstrap" Dec 03 12:34:48 crc kubenswrapper[5052]: I1203 12:34:48.847665 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-7476b7fc95-6cxvn" Dec 03 12:34:48 crc kubenswrapper[5052]: I1203 12:34:48.854058 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone-scripts" Dec 03 12:34:48 crc kubenswrapper[5052]: I1203 12:34:48.854297 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone" Dec 03 12:34:48 crc kubenswrapper[5052]: I1203 12:34:48.854701 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-keystone-internal-svc" Dec 03 12:34:48 crc kubenswrapper[5052]: I1203 12:34:48.854854 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-keystone-public-svc" Dec 03 12:34:48 crc kubenswrapper[5052]: I1203 12:34:48.855008 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone-keystone-dockercfg-fvfwv" Dec 03 12:34:48 crc kubenswrapper[5052]: I1203 12:34:48.855159 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone-config-data" Dec 03 12:34:48 crc kubenswrapper[5052]: I1203 12:34:48.879712 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-7476b7fc95-6cxvn"] Dec 03 12:34:48 crc kubenswrapper[5052]: I1203 12:34:48.905130 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:34:48 crc kubenswrapper[5052]: I1203 12:34:48.905180 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:34:48 crc kubenswrapper[5052]: I1203 12:34:48.938892 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/3f5299f3-7867-4a8a-9d64-f5a60526053a-scripts\") pod \"keystone-7476b7fc95-6cxvn\" (UID: \"3f5299f3-7867-4a8a-9d64-f5a60526053a\") " pod="openstack-kuttl-tests/keystone-7476b7fc95-6cxvn" Dec 03 12:34:48 crc kubenswrapper[5052]: I1203 12:34:48.938960 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/3f5299f3-7867-4a8a-9d64-f5a60526053a-credential-keys\") pod \"keystone-7476b7fc95-6cxvn\" (UID: \"3f5299f3-7867-4a8a-9d64-f5a60526053a\") " pod="openstack-kuttl-tests/keystone-7476b7fc95-6cxvn" Dec 03 12:34:48 crc kubenswrapper[5052]: I1203 12:34:48.939008 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3f5299f3-7867-4a8a-9d64-f5a60526053a-combined-ca-bundle\") pod \"keystone-7476b7fc95-6cxvn\" (UID: \"3f5299f3-7867-4a8a-9d64-f5a60526053a\") " pod="openstack-kuttl-tests/keystone-7476b7fc95-6cxvn" Dec 03 12:34:48 crc kubenswrapper[5052]: I1203 12:34:48.939128 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/3f5299f3-7867-4a8a-9d64-f5a60526053a-internal-tls-certs\") pod \"keystone-7476b7fc95-6cxvn\" (UID: \"3f5299f3-7867-4a8a-9d64-f5a60526053a\") " pod="openstack-kuttl-tests/keystone-7476b7fc95-6cxvn" Dec 03 12:34:48 crc kubenswrapper[5052]: I1203 12:34:48.939155 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/3f5299f3-7867-4a8a-9d64-f5a60526053a-config-data\") pod \"keystone-7476b7fc95-6cxvn\" (UID: \"3f5299f3-7867-4a8a-9d64-f5a60526053a\") " pod="openstack-kuttl-tests/keystone-7476b7fc95-6cxvn" Dec 03 12:34:48 crc kubenswrapper[5052]: I1203 12:34:48.939219 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/3f5299f3-7867-4a8a-9d64-f5a60526053a-fernet-keys\") pod \"keystone-7476b7fc95-6cxvn\" (UID: \"3f5299f3-7867-4a8a-9d64-f5a60526053a\") " pod="openstack-kuttl-tests/keystone-7476b7fc95-6cxvn" Dec 03 12:34:48 crc kubenswrapper[5052]: I1203 12:34:48.939274 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6g9s7\" (UniqueName: \"kubernetes.io/projected/3f5299f3-7867-4a8a-9d64-f5a60526053a-kube-api-access-6g9s7\") pod \"keystone-7476b7fc95-6cxvn\" (UID: \"3f5299f3-7867-4a8a-9d64-f5a60526053a\") " pod="openstack-kuttl-tests/keystone-7476b7fc95-6cxvn" Dec 03 12:34:48 crc kubenswrapper[5052]: I1203 12:34:48.939303 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/3f5299f3-7867-4a8a-9d64-f5a60526053a-public-tls-certs\") pod \"keystone-7476b7fc95-6cxvn\" (UID: \"3f5299f3-7867-4a8a-9d64-f5a60526053a\") " pod="openstack-kuttl-tests/keystone-7476b7fc95-6cxvn" Dec 03 12:34:48 crc kubenswrapper[5052]: I1203 12:34:48.953064 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:34:48 crc kubenswrapper[5052]: I1203 12:34:48.953868 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 12:34:48 crc kubenswrapper[5052]: I1203 12:34:48.962075 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:34:48 crc kubenswrapper[5052]: I1203 12:34:48.976395 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:34:48 crc kubenswrapper[5052]: I1203 12:34:48.977568 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:34:49 crc kubenswrapper[5052]: I1203 12:34:49.022706 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:34:49 crc kubenswrapper[5052]: I1203 12:34:49.040942 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/3f5299f3-7867-4a8a-9d64-f5a60526053a-config-data\") pod \"keystone-7476b7fc95-6cxvn\" (UID: \"3f5299f3-7867-4a8a-9d64-f5a60526053a\") " pod="openstack-kuttl-tests/keystone-7476b7fc95-6cxvn" Dec 03 12:34:49 crc kubenswrapper[5052]: I1203 12:34:49.040978 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/3f5299f3-7867-4a8a-9d64-f5a60526053a-internal-tls-certs\") pod \"keystone-7476b7fc95-6cxvn\" (UID: \"3f5299f3-7867-4a8a-9d64-f5a60526053a\") " pod="openstack-kuttl-tests/keystone-7476b7fc95-6cxvn" Dec 03 12:34:49 crc kubenswrapper[5052]: I1203 12:34:49.041030 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/3f5299f3-7867-4a8a-9d64-f5a60526053a-fernet-keys\") pod \"keystone-7476b7fc95-6cxvn\" (UID: \"3f5299f3-7867-4a8a-9d64-f5a60526053a\") " pod="openstack-kuttl-tests/keystone-7476b7fc95-6cxvn" Dec 03 12:34:49 crc kubenswrapper[5052]: I1203 12:34:49.041077 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-6g9s7\" (UniqueName: \"kubernetes.io/projected/3f5299f3-7867-4a8a-9d64-f5a60526053a-kube-api-access-6g9s7\") pod \"keystone-7476b7fc95-6cxvn\" (UID: \"3f5299f3-7867-4a8a-9d64-f5a60526053a\") " pod="openstack-kuttl-tests/keystone-7476b7fc95-6cxvn" Dec 03 12:34:49 crc kubenswrapper[5052]: I1203 12:34:49.041104 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/3f5299f3-7867-4a8a-9d64-f5a60526053a-public-tls-certs\") pod \"keystone-7476b7fc95-6cxvn\" (UID: \"3f5299f3-7867-4a8a-9d64-f5a60526053a\") " pod="openstack-kuttl-tests/keystone-7476b7fc95-6cxvn" Dec 03 12:34:49 crc kubenswrapper[5052]: I1203 12:34:49.041145 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/3f5299f3-7867-4a8a-9d64-f5a60526053a-scripts\") pod \"keystone-7476b7fc95-6cxvn\" (UID: \"3f5299f3-7867-4a8a-9d64-f5a60526053a\") " pod="openstack-kuttl-tests/keystone-7476b7fc95-6cxvn" Dec 03 12:34:49 crc kubenswrapper[5052]: I1203 12:34:49.041165 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/3f5299f3-7867-4a8a-9d64-f5a60526053a-credential-keys\") pod \"keystone-7476b7fc95-6cxvn\" (UID: \"3f5299f3-7867-4a8a-9d64-f5a60526053a\") " pod="openstack-kuttl-tests/keystone-7476b7fc95-6cxvn" Dec 03 12:34:49 crc kubenswrapper[5052]: I1203 12:34:49.041193 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3f5299f3-7867-4a8a-9d64-f5a60526053a-combined-ca-bundle\") pod \"keystone-7476b7fc95-6cxvn\" (UID: \"3f5299f3-7867-4a8a-9d64-f5a60526053a\") " pod="openstack-kuttl-tests/keystone-7476b7fc95-6cxvn" Dec 03 12:34:49 crc kubenswrapper[5052]: I1203 12:34:49.046374 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/3f5299f3-7867-4a8a-9d64-f5a60526053a-public-tls-certs\") pod \"keystone-7476b7fc95-6cxvn\" (UID: \"3f5299f3-7867-4a8a-9d64-f5a60526053a\") " pod="openstack-kuttl-tests/keystone-7476b7fc95-6cxvn" Dec 03 12:34:49 crc kubenswrapper[5052]: I1203 12:34:49.047174 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3f5299f3-7867-4a8a-9d64-f5a60526053a-combined-ca-bundle\") pod \"keystone-7476b7fc95-6cxvn\" (UID: \"3f5299f3-7867-4a8a-9d64-f5a60526053a\") " pod="openstack-kuttl-tests/keystone-7476b7fc95-6cxvn" Dec 03 12:34:49 crc kubenswrapper[5052]: I1203 12:34:49.048033 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/3f5299f3-7867-4a8a-9d64-f5a60526053a-internal-tls-certs\") pod \"keystone-7476b7fc95-6cxvn\" (UID: \"3f5299f3-7867-4a8a-9d64-f5a60526053a\") " pod="openstack-kuttl-tests/keystone-7476b7fc95-6cxvn" Dec 03 12:34:49 crc kubenswrapper[5052]: I1203 12:34:49.048668 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/3f5299f3-7867-4a8a-9d64-f5a60526053a-credential-keys\") pod \"keystone-7476b7fc95-6cxvn\" (UID: \"3f5299f3-7867-4a8a-9d64-f5a60526053a\") " pod="openstack-kuttl-tests/keystone-7476b7fc95-6cxvn" Dec 03 12:34:49 crc kubenswrapper[5052]: I1203 12:34:49.049493 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/3f5299f3-7867-4a8a-9d64-f5a60526053a-fernet-keys\") pod \"keystone-7476b7fc95-6cxvn\" (UID: \"3f5299f3-7867-4a8a-9d64-f5a60526053a\") " pod="openstack-kuttl-tests/keystone-7476b7fc95-6cxvn" Dec 03 12:34:49 crc kubenswrapper[5052]: I1203 12:34:49.049841 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/3f5299f3-7867-4a8a-9d64-f5a60526053a-config-data\") pod \"keystone-7476b7fc95-6cxvn\" (UID: \"3f5299f3-7867-4a8a-9d64-f5a60526053a\") " pod="openstack-kuttl-tests/keystone-7476b7fc95-6cxvn" Dec 03 12:34:49 crc kubenswrapper[5052]: I1203 12:34:49.054491 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/3f5299f3-7867-4a8a-9d64-f5a60526053a-scripts\") pod \"keystone-7476b7fc95-6cxvn\" (UID: \"3f5299f3-7867-4a8a-9d64-f5a60526053a\") " pod="openstack-kuttl-tests/keystone-7476b7fc95-6cxvn" Dec 03 12:34:49 crc kubenswrapper[5052]: I1203 12:34:49.054910 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:34:49 crc kubenswrapper[5052]: I1203 12:34:49.066158 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-6g9s7\" (UniqueName: \"kubernetes.io/projected/3f5299f3-7867-4a8a-9d64-f5a60526053a-kube-api-access-6g9s7\") pod \"keystone-7476b7fc95-6cxvn\" (UID: \"3f5299f3-7867-4a8a-9d64-f5a60526053a\") " pod="openstack-kuttl-tests/keystone-7476b7fc95-6cxvn" Dec 03 12:34:49 crc kubenswrapper[5052]: I1203 12:34:49.219711 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-7476b7fc95-6cxvn" Dec 03 12:34:49 crc kubenswrapper[5052]: I1203 12:34:49.729694 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"e5a85223-b8cf-471c-92d1-a4da3eae9649","Type":"ContainerStarted","Data":"a92d28ae6a790758e6fb4593a0ccedb09bf362eaf24606e0197b57fcf073ac3a"} Dec 03 12:34:49 crc kubenswrapper[5052]: I1203 12:34:49.738117 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"2793c42c-363d-4495-88d8-4385bfc9c3fc","Type":"ContainerStarted","Data":"b95bd6a82cabea6eab48bbfdbb5b46e696325061b2a6bdf6f39cab85c095c7dd"} Dec 03 12:34:49 crc kubenswrapper[5052]: I1203 12:34:49.738190 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:34:49 crc kubenswrapper[5052]: I1203 12:34:49.738295 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/barbican-api-b857b994d-6xzbl" Dec 03 12:34:49 crc kubenswrapper[5052]: I1203 12:34:49.738318 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:34:49 crc kubenswrapper[5052]: I1203 12:34:49.738339 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"2793c42c-363d-4495-88d8-4385bfc9c3fc","Type":"ContainerStarted","Data":"7634a1171510b5a7027061193ec392fdd1fab2f90deb2accca4c473913c2b91c"} Dec 03 12:34:49 crc kubenswrapper[5052]: I1203 12:34:49.738352 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:34:49 crc kubenswrapper[5052]: I1203 12:34:49.738784 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:34:49 crc kubenswrapper[5052]: I1203 12:34:49.738991 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/barbican-api-b857b994d-6xzbl" Dec 03 12:34:49 crc kubenswrapper[5052]: I1203 12:34:49.767297 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-7476b7fc95-6cxvn"] Dec 03 12:34:50 crc kubenswrapper[5052]: I1203 12:34:50.694014 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 12:34:50 crc kubenswrapper[5052]: I1203 12:34:50.753061 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"2793c42c-363d-4495-88d8-4385bfc9c3fc","Type":"ContainerStarted","Data":"2d2114af30c1e322b378894db0939eaa23284ec865c947f9868cf108b43d4e16"} Dec 03 12:34:50 crc kubenswrapper[5052]: I1203 12:34:50.756861 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-7476b7fc95-6cxvn" event={"ID":"3f5299f3-7867-4a8a-9d64-f5a60526053a","Type":"ContainerStarted","Data":"531c4d1034757d8c13569682d86b9abe1a7d5520dc20a2e62198919870a9b33c"} Dec 03 12:34:50 crc kubenswrapper[5052]: I1203 12:34:50.756920 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-7476b7fc95-6cxvn" event={"ID":"3f5299f3-7867-4a8a-9d64-f5a60526053a","Type":"ContainerStarted","Data":"edbcb5ee93daf0c7c41dbed35f926ee32d334282b63827a19e6be8eb3cb238ac"} Dec 03 12:34:50 crc kubenswrapper[5052]: I1203 12:34:50.756980 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/keystone-7476b7fc95-6cxvn" Dec 03 12:34:50 crc kubenswrapper[5052]: I1203 12:34:50.770289 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"e5a85223-b8cf-471c-92d1-a4da3eae9649","Type":"ContainerStarted","Data":"427e2bce666042e75cf585ffad6630d9e9a13bd7a2570f6489796f3fd63bee90"} Dec 03 12:34:50 crc kubenswrapper[5052]: I1203 12:34:50.778710 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/cinder-scheduler-0" podStartSLOduration=3.778696221 podStartE2EDuration="3.778696221s" podCreationTimestamp="2025-12-03 12:34:47 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:34:50.775310844 +0000 UTC m=+4481.363552608" watchObservedRunningTime="2025-12-03 12:34:50.778696221 +0000 UTC m=+4481.366937985" Dec 03 12:34:50 crc kubenswrapper[5052]: I1203 12:34:50.802598 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/keystone-7476b7fc95-6cxvn" podStartSLOduration=2.8025748139999997 podStartE2EDuration="2.802574814s" podCreationTimestamp="2025-12-03 12:34:48 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:34:50.796694035 +0000 UTC m=+4481.384935819" watchObservedRunningTime="2025-12-03 12:34:50.802574814 +0000 UTC m=+4481.390816578" Dec 03 12:34:51 crc kubenswrapper[5052]: I1203 12:34:51.782554 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/cinder-api-0" podUID="e5a85223-b8cf-471c-92d1-a4da3eae9649" containerName="cinder-api-log" containerID="cri-o://427e2bce666042e75cf585ffad6630d9e9a13bd7a2570f6489796f3fd63bee90" gracePeriod=30 Dec 03 12:34:51 crc kubenswrapper[5052]: I1203 12:34:51.783138 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"e5a85223-b8cf-471c-92d1-a4da3eae9649","Type":"ContainerStarted","Data":"3db4e46cf19d4d3f80a83572acbded51e7f977ba431b3484bce748d945f16b3a"} Dec 03 12:34:51 crc kubenswrapper[5052]: I1203 12:34:51.784047 5052 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Dec 03 12:34:51 crc kubenswrapper[5052]: I1203 12:34:51.784290 5052 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Dec 03 12:34:51 crc kubenswrapper[5052]: I1203 12:34:51.784745 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:34:51 crc kubenswrapper[5052]: I1203 12:34:51.785046 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/cinder-api-0" podUID="e5a85223-b8cf-471c-92d1-a4da3eae9649" containerName="cinder-api" containerID="cri-o://3db4e46cf19d4d3f80a83572acbded51e7f977ba431b3484bce748d945f16b3a" gracePeriod=30 Dec 03 12:34:51 crc kubenswrapper[5052]: I1203 12:34:51.825616 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/cinder-api-0" podStartSLOduration=3.82559332 podStartE2EDuration="3.82559332s" podCreationTimestamp="2025-12-03 12:34:48 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:34:51.810942452 +0000 UTC m=+4482.399184256" watchObservedRunningTime="2025-12-03 12:34:51.82559332 +0000 UTC m=+4482.413835094" Dec 03 12:34:51 crc kubenswrapper[5052]: I1203 12:34:51.963001 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:34:51 crc kubenswrapper[5052]: I1203 12:34:51.963081 5052 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Dec 03 12:34:52 crc kubenswrapper[5052]: I1203 12:34:52.042679 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:34:52 crc kubenswrapper[5052]: I1203 12:34:52.077042 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:34:52 crc kubenswrapper[5052]: I1203 12:34:52.086252 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:34:52 crc kubenswrapper[5052]: I1203 12:34:52.545800 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:34:52 crc kubenswrapper[5052]: I1203 12:34:52.719914 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tbptv\" (UniqueName: \"kubernetes.io/projected/e5a85223-b8cf-471c-92d1-a4da3eae9649-kube-api-access-tbptv\") pod \"e5a85223-b8cf-471c-92d1-a4da3eae9649\" (UID: \"e5a85223-b8cf-471c-92d1-a4da3eae9649\") " Dec 03 12:34:52 crc kubenswrapper[5052]: I1203 12:34:52.720174 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e5a85223-b8cf-471c-92d1-a4da3eae9649-combined-ca-bundle\") pod \"e5a85223-b8cf-471c-92d1-a4da3eae9649\" (UID: \"e5a85223-b8cf-471c-92d1-a4da3eae9649\") " Dec 03 12:34:52 crc kubenswrapper[5052]: I1203 12:34:52.720368 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/e5a85223-b8cf-471c-92d1-a4da3eae9649-config-data-custom\") pod \"e5a85223-b8cf-471c-92d1-a4da3eae9649\" (UID: \"e5a85223-b8cf-471c-92d1-a4da3eae9649\") " Dec 03 12:34:52 crc kubenswrapper[5052]: I1203 12:34:52.720646 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/e5a85223-b8cf-471c-92d1-a4da3eae9649-config-data\") pod \"e5a85223-b8cf-471c-92d1-a4da3eae9649\" (UID: \"e5a85223-b8cf-471c-92d1-a4da3eae9649\") " Dec 03 12:34:52 crc kubenswrapper[5052]: I1203 12:34:52.720805 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/e5a85223-b8cf-471c-92d1-a4da3eae9649-scripts\") pod \"e5a85223-b8cf-471c-92d1-a4da3eae9649\" (UID: \"e5a85223-b8cf-471c-92d1-a4da3eae9649\") " Dec 03 12:34:52 crc kubenswrapper[5052]: I1203 12:34:52.721011 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/e5a85223-b8cf-471c-92d1-a4da3eae9649-etc-machine-id\") pod \"e5a85223-b8cf-471c-92d1-a4da3eae9649\" (UID: \"e5a85223-b8cf-471c-92d1-a4da3eae9649\") " Dec 03 12:34:52 crc kubenswrapper[5052]: I1203 12:34:52.721106 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/e5a85223-b8cf-471c-92d1-a4da3eae9649-etc-machine-id" (OuterVolumeSpecName: "etc-machine-id") pod "e5a85223-b8cf-471c-92d1-a4da3eae9649" (UID: "e5a85223-b8cf-471c-92d1-a4da3eae9649"). InnerVolumeSpecName "etc-machine-id". PluginName "kubernetes.io/host-path", VolumeGidValue "" Dec 03 12:34:52 crc kubenswrapper[5052]: I1203 12:34:52.721382 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/e5a85223-b8cf-471c-92d1-a4da3eae9649-logs\") pod \"e5a85223-b8cf-471c-92d1-a4da3eae9649\" (UID: \"e5a85223-b8cf-471c-92d1-a4da3eae9649\") " Dec 03 12:34:52 crc kubenswrapper[5052]: I1203 12:34:52.721672 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e5a85223-b8cf-471c-92d1-a4da3eae9649-logs" (OuterVolumeSpecName: "logs") pod "e5a85223-b8cf-471c-92d1-a4da3eae9649" (UID: "e5a85223-b8cf-471c-92d1-a4da3eae9649"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:34:52 crc kubenswrapper[5052]: I1203 12:34:52.722322 5052 reconciler_common.go:293] "Volume detached for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/e5a85223-b8cf-471c-92d1-a4da3eae9649-etc-machine-id\") on node \"crc\" DevicePath \"\"" Dec 03 12:34:52 crc kubenswrapper[5052]: I1203 12:34:52.722433 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/e5a85223-b8cf-471c-92d1-a4da3eae9649-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:34:52 crc kubenswrapper[5052]: I1203 12:34:52.726070 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e5a85223-b8cf-471c-92d1-a4da3eae9649-kube-api-access-tbptv" (OuterVolumeSpecName: "kube-api-access-tbptv") pod "e5a85223-b8cf-471c-92d1-a4da3eae9649" (UID: "e5a85223-b8cf-471c-92d1-a4da3eae9649"). InnerVolumeSpecName "kube-api-access-tbptv". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:34:52 crc kubenswrapper[5052]: I1203 12:34:52.726835 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e5a85223-b8cf-471c-92d1-a4da3eae9649-scripts" (OuterVolumeSpecName: "scripts") pod "e5a85223-b8cf-471c-92d1-a4da3eae9649" (UID: "e5a85223-b8cf-471c-92d1-a4da3eae9649"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:34:52 crc kubenswrapper[5052]: I1203 12:34:52.738742 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e5a85223-b8cf-471c-92d1-a4da3eae9649-config-data-custom" (OuterVolumeSpecName: "config-data-custom") pod "e5a85223-b8cf-471c-92d1-a4da3eae9649" (UID: "e5a85223-b8cf-471c-92d1-a4da3eae9649"). InnerVolumeSpecName "config-data-custom". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:34:52 crc kubenswrapper[5052]: I1203 12:34:52.777661 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e5a85223-b8cf-471c-92d1-a4da3eae9649-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "e5a85223-b8cf-471c-92d1-a4da3eae9649" (UID: "e5a85223-b8cf-471c-92d1-a4da3eae9649"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:34:52 crc kubenswrapper[5052]: I1203 12:34:52.790455 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e5a85223-b8cf-471c-92d1-a4da3eae9649-config-data" (OuterVolumeSpecName: "config-data") pod "e5a85223-b8cf-471c-92d1-a4da3eae9649" (UID: "e5a85223-b8cf-471c-92d1-a4da3eae9649"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:34:52 crc kubenswrapper[5052]: I1203 12:34:52.794304 5052 generic.go:334] "Generic (PLEG): container finished" podID="e5a85223-b8cf-471c-92d1-a4da3eae9649" containerID="3db4e46cf19d4d3f80a83572acbded51e7f977ba431b3484bce748d945f16b3a" exitCode=0 Dec 03 12:34:52 crc kubenswrapper[5052]: I1203 12:34:52.794348 5052 generic.go:334] "Generic (PLEG): container finished" podID="e5a85223-b8cf-471c-92d1-a4da3eae9649" containerID="427e2bce666042e75cf585ffad6630d9e9a13bd7a2570f6489796f3fd63bee90" exitCode=143 Dec 03 12:34:52 crc kubenswrapper[5052]: I1203 12:34:52.795466 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:34:52 crc kubenswrapper[5052]: I1203 12:34:52.795630 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"e5a85223-b8cf-471c-92d1-a4da3eae9649","Type":"ContainerDied","Data":"3db4e46cf19d4d3f80a83572acbded51e7f977ba431b3484bce748d945f16b3a"} Dec 03 12:34:52 crc kubenswrapper[5052]: I1203 12:34:52.795668 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"e5a85223-b8cf-471c-92d1-a4da3eae9649","Type":"ContainerDied","Data":"427e2bce666042e75cf585ffad6630d9e9a13bd7a2570f6489796f3fd63bee90"} Dec 03 12:34:52 crc kubenswrapper[5052]: I1203 12:34:52.795681 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"e5a85223-b8cf-471c-92d1-a4da3eae9649","Type":"ContainerDied","Data":"a92d28ae6a790758e6fb4593a0ccedb09bf362eaf24606e0197b57fcf073ac3a"} Dec 03 12:34:52 crc kubenswrapper[5052]: I1203 12:34:52.795701 5052 scope.go:117] "RemoveContainer" containerID="3db4e46cf19d4d3f80a83572acbded51e7f977ba431b3484bce748d945f16b3a" Dec 03 12:34:52 crc kubenswrapper[5052]: I1203 12:34:52.820366 5052 scope.go:117] "RemoveContainer" containerID="427e2bce666042e75cf585ffad6630d9e9a13bd7a2570f6489796f3fd63bee90" Dec 03 12:34:52 crc kubenswrapper[5052]: I1203 12:34:52.825420 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-tbptv\" (UniqueName: \"kubernetes.io/projected/e5a85223-b8cf-471c-92d1-a4da3eae9649-kube-api-access-tbptv\") on node \"crc\" DevicePath \"\"" Dec 03 12:34:52 crc kubenswrapper[5052]: I1203 12:34:52.825447 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e5a85223-b8cf-471c-92d1-a4da3eae9649-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:34:52 crc kubenswrapper[5052]: I1203 12:34:52.825456 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/e5a85223-b8cf-471c-92d1-a4da3eae9649-config-data-custom\") on node \"crc\" DevicePath \"\"" Dec 03 12:34:52 crc kubenswrapper[5052]: I1203 12:34:52.825465 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/e5a85223-b8cf-471c-92d1-a4da3eae9649-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:34:52 crc kubenswrapper[5052]: I1203 12:34:52.825474 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/e5a85223-b8cf-471c-92d1-a4da3eae9649-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:34:52 crc kubenswrapper[5052]: I1203 12:34:52.831626 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 12:34:52 crc kubenswrapper[5052]: I1203 12:34:52.844723 5052 scope.go:117] "RemoveContainer" containerID="3db4e46cf19d4d3f80a83572acbded51e7f977ba431b3484bce748d945f16b3a" Dec 03 12:34:52 crc kubenswrapper[5052]: E1203 12:34:52.845268 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3db4e46cf19d4d3f80a83572acbded51e7f977ba431b3484bce748d945f16b3a\": container with ID starting with 3db4e46cf19d4d3f80a83572acbded51e7f977ba431b3484bce748d945f16b3a not found: ID does not exist" containerID="3db4e46cf19d4d3f80a83572acbded51e7f977ba431b3484bce748d945f16b3a" Dec 03 12:34:52 crc kubenswrapper[5052]: I1203 12:34:52.845307 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3db4e46cf19d4d3f80a83572acbded51e7f977ba431b3484bce748d945f16b3a"} err="failed to get container status \"3db4e46cf19d4d3f80a83572acbded51e7f977ba431b3484bce748d945f16b3a\": rpc error: code = NotFound desc = could not find container \"3db4e46cf19d4d3f80a83572acbded51e7f977ba431b3484bce748d945f16b3a\": container with ID starting with 3db4e46cf19d4d3f80a83572acbded51e7f977ba431b3484bce748d945f16b3a not found: ID does not exist" Dec 03 12:34:52 crc kubenswrapper[5052]: I1203 12:34:52.845345 5052 scope.go:117] "RemoveContainer" containerID="427e2bce666042e75cf585ffad6630d9e9a13bd7a2570f6489796f3fd63bee90" Dec 03 12:34:52 crc kubenswrapper[5052]: E1203 12:34:52.845911 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"427e2bce666042e75cf585ffad6630d9e9a13bd7a2570f6489796f3fd63bee90\": container with ID starting with 427e2bce666042e75cf585ffad6630d9e9a13bd7a2570f6489796f3fd63bee90 not found: ID does not exist" containerID="427e2bce666042e75cf585ffad6630d9e9a13bd7a2570f6489796f3fd63bee90" Dec 03 12:34:52 crc kubenswrapper[5052]: I1203 12:34:52.845959 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"427e2bce666042e75cf585ffad6630d9e9a13bd7a2570f6489796f3fd63bee90"} err="failed to get container status \"427e2bce666042e75cf585ffad6630d9e9a13bd7a2570f6489796f3fd63bee90\": rpc error: code = NotFound desc = could not find container \"427e2bce666042e75cf585ffad6630d9e9a13bd7a2570f6489796f3fd63bee90\": container with ID starting with 427e2bce666042e75cf585ffad6630d9e9a13bd7a2570f6489796f3fd63bee90 not found: ID does not exist" Dec 03 12:34:52 crc kubenswrapper[5052]: I1203 12:34:52.845985 5052 scope.go:117] "RemoveContainer" containerID="3db4e46cf19d4d3f80a83572acbded51e7f977ba431b3484bce748d945f16b3a" Dec 03 12:34:52 crc kubenswrapper[5052]: I1203 12:34:52.846364 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 12:34:52 crc kubenswrapper[5052]: I1203 12:34:52.846389 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3db4e46cf19d4d3f80a83572acbded51e7f977ba431b3484bce748d945f16b3a"} err="failed to get container status \"3db4e46cf19d4d3f80a83572acbded51e7f977ba431b3484bce748d945f16b3a\": rpc error: code = NotFound desc = could not find container \"3db4e46cf19d4d3f80a83572acbded51e7f977ba431b3484bce748d945f16b3a\": container with ID starting with 3db4e46cf19d4d3f80a83572acbded51e7f977ba431b3484bce748d945f16b3a not found: ID does not exist" Dec 03 12:34:52 crc kubenswrapper[5052]: I1203 12:34:52.846470 5052 scope.go:117] "RemoveContainer" containerID="427e2bce666042e75cf585ffad6630d9e9a13bd7a2570f6489796f3fd63bee90" Dec 03 12:34:52 crc kubenswrapper[5052]: I1203 12:34:52.846867 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"427e2bce666042e75cf585ffad6630d9e9a13bd7a2570f6489796f3fd63bee90"} err="failed to get container status \"427e2bce666042e75cf585ffad6630d9e9a13bd7a2570f6489796f3fd63bee90\": rpc error: code = NotFound desc = could not find container \"427e2bce666042e75cf585ffad6630d9e9a13bd7a2570f6489796f3fd63bee90\": container with ID starting with 427e2bce666042e75cf585ffad6630d9e9a13bd7a2570f6489796f3fd63bee90 not found: ID does not exist" Dec 03 12:34:52 crc kubenswrapper[5052]: I1203 12:34:52.859678 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 12:34:52 crc kubenswrapper[5052]: E1203 12:34:52.860424 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="e5a85223-b8cf-471c-92d1-a4da3eae9649" containerName="cinder-api" Dec 03 12:34:52 crc kubenswrapper[5052]: I1203 12:34:52.860450 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="e5a85223-b8cf-471c-92d1-a4da3eae9649" containerName="cinder-api" Dec 03 12:34:52 crc kubenswrapper[5052]: E1203 12:34:52.864319 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="e5a85223-b8cf-471c-92d1-a4da3eae9649" containerName="cinder-api-log" Dec 03 12:34:52 crc kubenswrapper[5052]: I1203 12:34:52.864355 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="e5a85223-b8cf-471c-92d1-a4da3eae9649" containerName="cinder-api-log" Dec 03 12:34:52 crc kubenswrapper[5052]: I1203 12:34:52.865054 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="e5a85223-b8cf-471c-92d1-a4da3eae9649" containerName="cinder-api-log" Dec 03 12:34:52 crc kubenswrapper[5052]: I1203 12:34:52.865105 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="e5a85223-b8cf-471c-92d1-a4da3eae9649" containerName="cinder-api" Dec 03 12:34:52 crc kubenswrapper[5052]: I1203 12:34:52.866913 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:34:52 crc kubenswrapper[5052]: I1203 12:34:52.875684 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 12:34:52 crc kubenswrapper[5052]: I1203 12:34:52.889121 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-cinder-internal-svc" Dec 03 12:34:52 crc kubenswrapper[5052]: I1203 12:34:52.891040 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-cinder-public-svc" Dec 03 12:34:52 crc kubenswrapper[5052]: I1203 12:34:52.895354 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cinder-api-config-data" Dec 03 12:34:53 crc kubenswrapper[5052]: I1203 12:34:53.036148 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/8a16aa42-962f-48cc-a7a8-d0e189164013-public-tls-certs\") pod \"cinder-api-0\" (UID: \"8a16aa42-962f-48cc-a7a8-d0e189164013\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:34:53 crc kubenswrapper[5052]: I1203 12:34:53.036216 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/8a16aa42-962f-48cc-a7a8-d0e189164013-internal-tls-certs\") pod \"cinder-api-0\" (UID: \"8a16aa42-962f-48cc-a7a8-d0e189164013\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:34:53 crc kubenswrapper[5052]: I1203 12:34:53.036258 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/8a16aa42-962f-48cc-a7a8-d0e189164013-etc-machine-id\") pod \"cinder-api-0\" (UID: \"8a16aa42-962f-48cc-a7a8-d0e189164013\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:34:53 crc kubenswrapper[5052]: I1203 12:34:53.036340 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/8a16aa42-962f-48cc-a7a8-d0e189164013-config-data\") pod \"cinder-api-0\" (UID: \"8a16aa42-962f-48cc-a7a8-d0e189164013\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:34:53 crc kubenswrapper[5052]: I1203 12:34:53.036365 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/8a16aa42-962f-48cc-a7a8-d0e189164013-logs\") pod \"cinder-api-0\" (UID: \"8a16aa42-962f-48cc-a7a8-d0e189164013\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:34:53 crc kubenswrapper[5052]: I1203 12:34:53.036438 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/8a16aa42-962f-48cc-a7a8-d0e189164013-scripts\") pod \"cinder-api-0\" (UID: \"8a16aa42-962f-48cc-a7a8-d0e189164013\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:34:53 crc kubenswrapper[5052]: I1203 12:34:53.036563 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/8a16aa42-962f-48cc-a7a8-d0e189164013-config-data-custom\") pod \"cinder-api-0\" (UID: \"8a16aa42-962f-48cc-a7a8-d0e189164013\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:34:53 crc kubenswrapper[5052]: I1203 12:34:53.036602 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-npsxr\" (UniqueName: \"kubernetes.io/projected/8a16aa42-962f-48cc-a7a8-d0e189164013-kube-api-access-npsxr\") pod \"cinder-api-0\" (UID: \"8a16aa42-962f-48cc-a7a8-d0e189164013\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:34:53 crc kubenswrapper[5052]: I1203 12:34:53.036633 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8a16aa42-962f-48cc-a7a8-d0e189164013-combined-ca-bundle\") pod \"cinder-api-0\" (UID: \"8a16aa42-962f-48cc-a7a8-d0e189164013\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:34:53 crc kubenswrapper[5052]: I1203 12:34:53.138528 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/8a16aa42-962f-48cc-a7a8-d0e189164013-scripts\") pod \"cinder-api-0\" (UID: \"8a16aa42-962f-48cc-a7a8-d0e189164013\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:34:53 crc kubenswrapper[5052]: I1203 12:34:53.138620 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/8a16aa42-962f-48cc-a7a8-d0e189164013-config-data-custom\") pod \"cinder-api-0\" (UID: \"8a16aa42-962f-48cc-a7a8-d0e189164013\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:34:53 crc kubenswrapper[5052]: I1203 12:34:53.138649 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-npsxr\" (UniqueName: \"kubernetes.io/projected/8a16aa42-962f-48cc-a7a8-d0e189164013-kube-api-access-npsxr\") pod \"cinder-api-0\" (UID: \"8a16aa42-962f-48cc-a7a8-d0e189164013\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:34:53 crc kubenswrapper[5052]: I1203 12:34:53.138685 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8a16aa42-962f-48cc-a7a8-d0e189164013-combined-ca-bundle\") pod \"cinder-api-0\" (UID: \"8a16aa42-962f-48cc-a7a8-d0e189164013\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:34:53 crc kubenswrapper[5052]: I1203 12:34:53.138732 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/8a16aa42-962f-48cc-a7a8-d0e189164013-public-tls-certs\") pod \"cinder-api-0\" (UID: \"8a16aa42-962f-48cc-a7a8-d0e189164013\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:34:53 crc kubenswrapper[5052]: I1203 12:34:53.138751 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/8a16aa42-962f-48cc-a7a8-d0e189164013-internal-tls-certs\") pod \"cinder-api-0\" (UID: \"8a16aa42-962f-48cc-a7a8-d0e189164013\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:34:53 crc kubenswrapper[5052]: I1203 12:34:53.138774 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/8a16aa42-962f-48cc-a7a8-d0e189164013-etc-machine-id\") pod \"cinder-api-0\" (UID: \"8a16aa42-962f-48cc-a7a8-d0e189164013\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:34:53 crc kubenswrapper[5052]: I1203 12:34:53.138808 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/8a16aa42-962f-48cc-a7a8-d0e189164013-config-data\") pod \"cinder-api-0\" (UID: \"8a16aa42-962f-48cc-a7a8-d0e189164013\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:34:53 crc kubenswrapper[5052]: I1203 12:34:53.138826 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/8a16aa42-962f-48cc-a7a8-d0e189164013-logs\") pod \"cinder-api-0\" (UID: \"8a16aa42-962f-48cc-a7a8-d0e189164013\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:34:53 crc kubenswrapper[5052]: I1203 12:34:53.139134 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/8a16aa42-962f-48cc-a7a8-d0e189164013-logs\") pod \"cinder-api-0\" (UID: \"8a16aa42-962f-48cc-a7a8-d0e189164013\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:34:53 crc kubenswrapper[5052]: I1203 12:34:53.140482 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/8a16aa42-962f-48cc-a7a8-d0e189164013-etc-machine-id\") pod \"cinder-api-0\" (UID: \"8a16aa42-962f-48cc-a7a8-d0e189164013\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:34:53 crc kubenswrapper[5052]: I1203 12:34:53.144077 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/8a16aa42-962f-48cc-a7a8-d0e189164013-internal-tls-certs\") pod \"cinder-api-0\" (UID: \"8a16aa42-962f-48cc-a7a8-d0e189164013\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:34:53 crc kubenswrapper[5052]: I1203 12:34:53.147828 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/8a16aa42-962f-48cc-a7a8-d0e189164013-config-data-custom\") pod \"cinder-api-0\" (UID: \"8a16aa42-962f-48cc-a7a8-d0e189164013\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:34:53 crc kubenswrapper[5052]: I1203 12:34:53.148227 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/8a16aa42-962f-48cc-a7a8-d0e189164013-public-tls-certs\") pod \"cinder-api-0\" (UID: \"8a16aa42-962f-48cc-a7a8-d0e189164013\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:34:53 crc kubenswrapper[5052]: I1203 12:34:53.150111 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/8a16aa42-962f-48cc-a7a8-d0e189164013-config-data\") pod \"cinder-api-0\" (UID: \"8a16aa42-962f-48cc-a7a8-d0e189164013\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:34:53 crc kubenswrapper[5052]: I1203 12:34:53.159162 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8a16aa42-962f-48cc-a7a8-d0e189164013-combined-ca-bundle\") pod \"cinder-api-0\" (UID: \"8a16aa42-962f-48cc-a7a8-d0e189164013\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:34:53 crc kubenswrapper[5052]: I1203 12:34:53.165733 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/8a16aa42-962f-48cc-a7a8-d0e189164013-scripts\") pod \"cinder-api-0\" (UID: \"8a16aa42-962f-48cc-a7a8-d0e189164013\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:34:53 crc kubenswrapper[5052]: I1203 12:34:53.176104 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-npsxr\" (UniqueName: \"kubernetes.io/projected/8a16aa42-962f-48cc-a7a8-d0e189164013-kube-api-access-npsxr\") pod \"cinder-api-0\" (UID: \"8a16aa42-962f-48cc-a7a8-d0e189164013\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:34:53 crc kubenswrapper[5052]: I1203 12:34:53.199474 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:34:53 crc kubenswrapper[5052]: I1203 12:34:53.286895 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:34:53 crc kubenswrapper[5052]: I1203 12:34:53.710798 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e5a85223-b8cf-471c-92d1-a4da3eae9649" path="/var/lib/kubelet/pods/e5a85223-b8cf-471c-92d1-a4da3eae9649/volumes" Dec 03 12:34:53 crc kubenswrapper[5052]: I1203 12:34:53.890087 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 12:34:53 crc kubenswrapper[5052]: I1203 12:34:53.996254 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/barbican-api-b857b994d-6xzbl" Dec 03 12:34:54 crc kubenswrapper[5052]: I1203 12:34:54.841668 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"8a16aa42-962f-48cc-a7a8-d0e189164013","Type":"ContainerStarted","Data":"adfc4f6fc4db6a74ee3543f31c610dc26c11391eefc7a3ac50cbb3115dbc02c5"} Dec 03 12:34:54 crc kubenswrapper[5052]: I1203 12:34:54.841717 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"8a16aa42-962f-48cc-a7a8-d0e189164013","Type":"ContainerStarted","Data":"747e157d7c371ec2fa20cd7081d12642550aa5ded0bd95490c4ab3beaf0b5bbf"} Dec 03 12:34:55 crc kubenswrapper[5052]: I1203 12:34:55.587093 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/barbican-api-b857b994d-6xzbl" Dec 03 12:34:55 crc kubenswrapper[5052]: I1203 12:34:55.662949 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-api-5656894b88-g5p25"] Dec 03 12:34:55 crc kubenswrapper[5052]: I1203 12:34:55.663242 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/barbican-api-5656894b88-g5p25" podUID="8a22a52b-d45b-4678-900d-a9d6d5113da3" containerName="barbican-api-log" containerID="cri-o://cca3f4b446ecb9d98e6d2c384dbfc5000a4f123199962b788a74ab88d347c76b" gracePeriod=30 Dec 03 12:34:55 crc kubenswrapper[5052]: I1203 12:34:55.663813 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/barbican-api-5656894b88-g5p25" podUID="8a22a52b-d45b-4678-900d-a9d6d5113da3" containerName="barbican-api" containerID="cri-o://81f54b8a5b8caf3d328e09e17a0f8fcd598cbfa32b1af39536bc30c4637e930a" gracePeriod=30 Dec 03 12:34:55 crc kubenswrapper[5052]: I1203 12:34:55.668047 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/barbican-api-5656894b88-g5p25" podUID="8a22a52b-d45b-4678-900d-a9d6d5113da3" containerName="barbican-api" probeResult="failure" output="Get \"http://10.217.1.45:9311/healthcheck\": EOF" Dec 03 12:34:55 crc kubenswrapper[5052]: I1203 12:34:55.668134 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/barbican-api-5656894b88-g5p25" podUID="8a22a52b-d45b-4678-900d-a9d6d5113da3" containerName="barbican-api-log" probeResult="failure" output="Get \"http://10.217.1.45:9311/healthcheck\": EOF" Dec 03 12:34:55 crc kubenswrapper[5052]: I1203 12:34:55.858106 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"8a16aa42-962f-48cc-a7a8-d0e189164013","Type":"ContainerStarted","Data":"0c4e092293ea6adf3158ffe86080aa757175ad4a21f6d1b968c71ccec2b3a499"} Dec 03 12:34:55 crc kubenswrapper[5052]: I1203 12:34:55.858182 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:34:55 crc kubenswrapper[5052]: I1203 12:34:55.859621 5052 generic.go:334] "Generic (PLEG): container finished" podID="8a22a52b-d45b-4678-900d-a9d6d5113da3" containerID="cca3f4b446ecb9d98e6d2c384dbfc5000a4f123199962b788a74ab88d347c76b" exitCode=143 Dec 03 12:34:55 crc kubenswrapper[5052]: I1203 12:34:55.859651 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-5656894b88-g5p25" event={"ID":"8a22a52b-d45b-4678-900d-a9d6d5113da3","Type":"ContainerDied","Data":"cca3f4b446ecb9d98e6d2c384dbfc5000a4f123199962b788a74ab88d347c76b"} Dec 03 12:34:55 crc kubenswrapper[5052]: I1203 12:34:55.888878 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/cinder-api-0" podStartSLOduration=3.8888530230000002 podStartE2EDuration="3.888853023s" podCreationTimestamp="2025-12-03 12:34:52 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:34:55.878444455 +0000 UTC m=+4486.466686239" watchObservedRunningTime="2025-12-03 12:34:55.888853023 +0000 UTC m=+4486.477094777" Dec 03 12:34:56 crc kubenswrapper[5052]: I1203 12:34:56.673155 5052 scope.go:117] "RemoveContainer" containerID="e6edaf188f85ea4f11f487f11cc489c700f25fb42b5ee269f4f496f1e127d772" Dec 03 12:34:56 crc kubenswrapper[5052]: E1203 12:34:56.673641 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 12:34:57 crc kubenswrapper[5052]: I1203 12:34:57.079243 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/barbican-api-5656894b88-g5p25" Dec 03 12:34:57 crc kubenswrapper[5052]: I1203 12:34:57.136051 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/barbican-api-5656894b88-g5p25" Dec 03 12:34:58 crc kubenswrapper[5052]: I1203 12:34:58.537858 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:34:58 crc kubenswrapper[5052]: I1203 12:34:58.597486 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 12:34:58 crc kubenswrapper[5052]: I1203 12:34:58.900636 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/cinder-scheduler-0" podUID="2793c42c-363d-4495-88d8-4385bfc9c3fc" containerName="cinder-scheduler" containerID="cri-o://b95bd6a82cabea6eab48bbfdbb5b46e696325061b2a6bdf6f39cab85c095c7dd" gracePeriod=30 Dec 03 12:34:58 crc kubenswrapper[5052]: I1203 12:34:58.901356 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/cinder-scheduler-0" podUID="2793c42c-363d-4495-88d8-4385bfc9c3fc" containerName="probe" containerID="cri-o://2d2114af30c1e322b378894db0939eaa23284ec865c947f9868cf108b43d4e16" gracePeriod=30 Dec 03 12:34:59 crc kubenswrapper[5052]: I1203 12:34:59.915951 5052 generic.go:334] "Generic (PLEG): container finished" podID="2793c42c-363d-4495-88d8-4385bfc9c3fc" containerID="2d2114af30c1e322b378894db0939eaa23284ec865c947f9868cf108b43d4e16" exitCode=0 Dec 03 12:34:59 crc kubenswrapper[5052]: I1203 12:34:59.916203 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"2793c42c-363d-4495-88d8-4385bfc9c3fc","Type":"ContainerDied","Data":"2d2114af30c1e322b378894db0939eaa23284ec865c947f9868cf108b43d4e16"} Dec 03 12:35:01 crc kubenswrapper[5052]: I1203 12:35:01.076642 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/barbican-api-5656894b88-g5p25" podUID="8a22a52b-d45b-4678-900d-a9d6d5113da3" containerName="barbican-api" probeResult="failure" output="Get \"http://10.217.1.45:9311/healthcheck\": read tcp 10.217.0.2:60644->10.217.1.45:9311: read: connection reset by peer" Dec 03 12:35:01 crc kubenswrapper[5052]: I1203 12:35:01.076690 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/barbican-api-5656894b88-g5p25" podUID="8a22a52b-d45b-4678-900d-a9d6d5113da3" containerName="barbican-api-log" probeResult="failure" output="Get \"http://10.217.1.45:9311/healthcheck\": read tcp 10.217.0.2:60650->10.217.1.45:9311: read: connection reset by peer" Dec 03 12:35:01 crc kubenswrapper[5052]: I1203 12:35:01.528995 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-api-5656894b88-g5p25" Dec 03 12:35:01 crc kubenswrapper[5052]: I1203 12:35:01.623645 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/8a22a52b-d45b-4678-900d-a9d6d5113da3-logs\") pod \"8a22a52b-d45b-4678-900d-a9d6d5113da3\" (UID: \"8a22a52b-d45b-4678-900d-a9d6d5113da3\") " Dec 03 12:35:01 crc kubenswrapper[5052]: I1203 12:35:01.623717 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/8a22a52b-d45b-4678-900d-a9d6d5113da3-config-data\") pod \"8a22a52b-d45b-4678-900d-a9d6d5113da3\" (UID: \"8a22a52b-d45b-4678-900d-a9d6d5113da3\") " Dec 03 12:35:01 crc kubenswrapper[5052]: I1203 12:35:01.623838 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/8a22a52b-d45b-4678-900d-a9d6d5113da3-config-data-custom\") pod \"8a22a52b-d45b-4678-900d-a9d6d5113da3\" (UID: \"8a22a52b-d45b-4678-900d-a9d6d5113da3\") " Dec 03 12:35:01 crc kubenswrapper[5052]: I1203 12:35:01.623914 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5j2k8\" (UniqueName: \"kubernetes.io/projected/8a22a52b-d45b-4678-900d-a9d6d5113da3-kube-api-access-5j2k8\") pod \"8a22a52b-d45b-4678-900d-a9d6d5113da3\" (UID: \"8a22a52b-d45b-4678-900d-a9d6d5113da3\") " Dec 03 12:35:01 crc kubenswrapper[5052]: I1203 12:35:01.623972 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8a22a52b-d45b-4678-900d-a9d6d5113da3-combined-ca-bundle\") pod \"8a22a52b-d45b-4678-900d-a9d6d5113da3\" (UID: \"8a22a52b-d45b-4678-900d-a9d6d5113da3\") " Dec 03 12:35:01 crc kubenswrapper[5052]: I1203 12:35:01.624237 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8a22a52b-d45b-4678-900d-a9d6d5113da3-logs" (OuterVolumeSpecName: "logs") pod "8a22a52b-d45b-4678-900d-a9d6d5113da3" (UID: "8a22a52b-d45b-4678-900d-a9d6d5113da3"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:35:01 crc kubenswrapper[5052]: I1203 12:35:01.624691 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/8a22a52b-d45b-4678-900d-a9d6d5113da3-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:35:01 crc kubenswrapper[5052]: I1203 12:35:01.628907 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8a22a52b-d45b-4678-900d-a9d6d5113da3-config-data-custom" (OuterVolumeSpecName: "config-data-custom") pod "8a22a52b-d45b-4678-900d-a9d6d5113da3" (UID: "8a22a52b-d45b-4678-900d-a9d6d5113da3"). InnerVolumeSpecName "config-data-custom". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:35:01 crc kubenswrapper[5052]: I1203 12:35:01.629374 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8a22a52b-d45b-4678-900d-a9d6d5113da3-kube-api-access-5j2k8" (OuterVolumeSpecName: "kube-api-access-5j2k8") pod "8a22a52b-d45b-4678-900d-a9d6d5113da3" (UID: "8a22a52b-d45b-4678-900d-a9d6d5113da3"). InnerVolumeSpecName "kube-api-access-5j2k8". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:35:01 crc kubenswrapper[5052]: I1203 12:35:01.648514 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8a22a52b-d45b-4678-900d-a9d6d5113da3-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "8a22a52b-d45b-4678-900d-a9d6d5113da3" (UID: "8a22a52b-d45b-4678-900d-a9d6d5113da3"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:35:01 crc kubenswrapper[5052]: I1203 12:35:01.675422 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8a22a52b-d45b-4678-900d-a9d6d5113da3-config-data" (OuterVolumeSpecName: "config-data") pod "8a22a52b-d45b-4678-900d-a9d6d5113da3" (UID: "8a22a52b-d45b-4678-900d-a9d6d5113da3"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:35:01 crc kubenswrapper[5052]: I1203 12:35:01.727041 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/8a22a52b-d45b-4678-900d-a9d6d5113da3-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:35:01 crc kubenswrapper[5052]: I1203 12:35:01.727099 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/8a22a52b-d45b-4678-900d-a9d6d5113da3-config-data-custom\") on node \"crc\" DevicePath \"\"" Dec 03 12:35:01 crc kubenswrapper[5052]: I1203 12:35:01.727115 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-5j2k8\" (UniqueName: \"kubernetes.io/projected/8a22a52b-d45b-4678-900d-a9d6d5113da3-kube-api-access-5j2k8\") on node \"crc\" DevicePath \"\"" Dec 03 12:35:01 crc kubenswrapper[5052]: I1203 12:35:01.727130 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8a22a52b-d45b-4678-900d-a9d6d5113da3-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:35:01 crc kubenswrapper[5052]: I1203 12:35:01.936672 5052 generic.go:334] "Generic (PLEG): container finished" podID="8a22a52b-d45b-4678-900d-a9d6d5113da3" containerID="81f54b8a5b8caf3d328e09e17a0f8fcd598cbfa32b1af39536bc30c4637e930a" exitCode=0 Dec 03 12:35:01 crc kubenswrapper[5052]: I1203 12:35:01.936712 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-5656894b88-g5p25" event={"ID":"8a22a52b-d45b-4678-900d-a9d6d5113da3","Type":"ContainerDied","Data":"81f54b8a5b8caf3d328e09e17a0f8fcd598cbfa32b1af39536bc30c4637e930a"} Dec 03 12:35:01 crc kubenswrapper[5052]: I1203 12:35:01.936757 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-5656894b88-g5p25" event={"ID":"8a22a52b-d45b-4678-900d-a9d6d5113da3","Type":"ContainerDied","Data":"3eb18214b19b85281e2ecd516827f1dd427fa02f1c919d5210f68825fc8fa32c"} Dec 03 12:35:01 crc kubenswrapper[5052]: I1203 12:35:01.936776 5052 scope.go:117] "RemoveContainer" containerID="81f54b8a5b8caf3d328e09e17a0f8fcd598cbfa32b1af39536bc30c4637e930a" Dec 03 12:35:01 crc kubenswrapper[5052]: I1203 12:35:01.936780 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-api-5656894b88-g5p25" Dec 03 12:35:01 crc kubenswrapper[5052]: I1203 12:35:01.939761 5052 generic.go:334] "Generic (PLEG): container finished" podID="348487de-7480-462b-9f22-56a555321be8" containerID="de7ec0b74eb7ee449efa4ae5a8ddecacb3b2510555a213842ba699fc19a15a76" exitCode=0 Dec 03 12:35:01 crc kubenswrapper[5052]: I1203 12:35:01.939805 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-db-sync-5kww4" event={"ID":"348487de-7480-462b-9f22-56a555321be8","Type":"ContainerDied","Data":"de7ec0b74eb7ee449efa4ae5a8ddecacb3b2510555a213842ba699fc19a15a76"} Dec 03 12:35:01 crc kubenswrapper[5052]: I1203 12:35:01.961851 5052 scope.go:117] "RemoveContainer" containerID="cca3f4b446ecb9d98e6d2c384dbfc5000a4f123199962b788a74ab88d347c76b" Dec 03 12:35:01 crc kubenswrapper[5052]: I1203 12:35:01.987045 5052 scope.go:117] "RemoveContainer" containerID="81f54b8a5b8caf3d328e09e17a0f8fcd598cbfa32b1af39536bc30c4637e930a" Dec 03 12:35:01 crc kubenswrapper[5052]: E1203 12:35:01.987590 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"81f54b8a5b8caf3d328e09e17a0f8fcd598cbfa32b1af39536bc30c4637e930a\": container with ID starting with 81f54b8a5b8caf3d328e09e17a0f8fcd598cbfa32b1af39536bc30c4637e930a not found: ID does not exist" containerID="81f54b8a5b8caf3d328e09e17a0f8fcd598cbfa32b1af39536bc30c4637e930a" Dec 03 12:35:01 crc kubenswrapper[5052]: I1203 12:35:01.987620 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"81f54b8a5b8caf3d328e09e17a0f8fcd598cbfa32b1af39536bc30c4637e930a"} err="failed to get container status \"81f54b8a5b8caf3d328e09e17a0f8fcd598cbfa32b1af39536bc30c4637e930a\": rpc error: code = NotFound desc = could not find container \"81f54b8a5b8caf3d328e09e17a0f8fcd598cbfa32b1af39536bc30c4637e930a\": container with ID starting with 81f54b8a5b8caf3d328e09e17a0f8fcd598cbfa32b1af39536bc30c4637e930a not found: ID does not exist" Dec 03 12:35:01 crc kubenswrapper[5052]: I1203 12:35:01.987642 5052 scope.go:117] "RemoveContainer" containerID="cca3f4b446ecb9d98e6d2c384dbfc5000a4f123199962b788a74ab88d347c76b" Dec 03 12:35:01 crc kubenswrapper[5052]: E1203 12:35:01.987918 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cca3f4b446ecb9d98e6d2c384dbfc5000a4f123199962b788a74ab88d347c76b\": container with ID starting with cca3f4b446ecb9d98e6d2c384dbfc5000a4f123199962b788a74ab88d347c76b not found: ID does not exist" containerID="cca3f4b446ecb9d98e6d2c384dbfc5000a4f123199962b788a74ab88d347c76b" Dec 03 12:35:01 crc kubenswrapper[5052]: I1203 12:35:01.987945 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cca3f4b446ecb9d98e6d2c384dbfc5000a4f123199962b788a74ab88d347c76b"} err="failed to get container status \"cca3f4b446ecb9d98e6d2c384dbfc5000a4f123199962b788a74ab88d347c76b\": rpc error: code = NotFound desc = could not find container \"cca3f4b446ecb9d98e6d2c384dbfc5000a4f123199962b788a74ab88d347c76b\": container with ID starting with cca3f4b446ecb9d98e6d2c384dbfc5000a4f123199962b788a74ab88d347c76b not found: ID does not exist" Dec 03 12:35:01 crc kubenswrapper[5052]: I1203 12:35:01.988691 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-api-5656894b88-g5p25"] Dec 03 12:35:02 crc kubenswrapper[5052]: I1203 12:35:02.000497 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/barbican-api-5656894b88-g5p25"] Dec 03 12:35:02 crc kubenswrapper[5052]: I1203 12:35:02.969595 5052 generic.go:334] "Generic (PLEG): container finished" podID="2793c42c-363d-4495-88d8-4385bfc9c3fc" containerID="b95bd6a82cabea6eab48bbfdbb5b46e696325061b2a6bdf6f39cab85c095c7dd" exitCode=0 Dec 03 12:35:02 crc kubenswrapper[5052]: I1203 12:35:02.969637 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"2793c42c-363d-4495-88d8-4385bfc9c3fc","Type":"ContainerDied","Data":"b95bd6a82cabea6eab48bbfdbb5b46e696325061b2a6bdf6f39cab85c095c7dd"} Dec 03 12:35:03 crc kubenswrapper[5052]: I1203 12:35:03.098222 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:35:03 crc kubenswrapper[5052]: I1203 12:35:03.255074 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/2793c42c-363d-4495-88d8-4385bfc9c3fc-config-data-custom\") pod \"2793c42c-363d-4495-88d8-4385bfc9c3fc\" (UID: \"2793c42c-363d-4495-88d8-4385bfc9c3fc\") " Dec 03 12:35:03 crc kubenswrapper[5052]: I1203 12:35:03.255396 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2793c42c-363d-4495-88d8-4385bfc9c3fc-combined-ca-bundle\") pod \"2793c42c-363d-4495-88d8-4385bfc9c3fc\" (UID: \"2793c42c-363d-4495-88d8-4385bfc9c3fc\") " Dec 03 12:35:03 crc kubenswrapper[5052]: I1203 12:35:03.255439 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7kmjb\" (UniqueName: \"kubernetes.io/projected/2793c42c-363d-4495-88d8-4385bfc9c3fc-kube-api-access-7kmjb\") pod \"2793c42c-363d-4495-88d8-4385bfc9c3fc\" (UID: \"2793c42c-363d-4495-88d8-4385bfc9c3fc\") " Dec 03 12:35:03 crc kubenswrapper[5052]: I1203 12:35:03.255484 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/2793c42c-363d-4495-88d8-4385bfc9c3fc-scripts\") pod \"2793c42c-363d-4495-88d8-4385bfc9c3fc\" (UID: \"2793c42c-363d-4495-88d8-4385bfc9c3fc\") " Dec 03 12:35:03 crc kubenswrapper[5052]: I1203 12:35:03.255574 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/2793c42c-363d-4495-88d8-4385bfc9c3fc-etc-machine-id\") pod \"2793c42c-363d-4495-88d8-4385bfc9c3fc\" (UID: \"2793c42c-363d-4495-88d8-4385bfc9c3fc\") " Dec 03 12:35:03 crc kubenswrapper[5052]: I1203 12:35:03.255615 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/2793c42c-363d-4495-88d8-4385bfc9c3fc-config-data\") pod \"2793c42c-363d-4495-88d8-4385bfc9c3fc\" (UID: \"2793c42c-363d-4495-88d8-4385bfc9c3fc\") " Dec 03 12:35:03 crc kubenswrapper[5052]: I1203 12:35:03.256389 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/2793c42c-363d-4495-88d8-4385bfc9c3fc-etc-machine-id" (OuterVolumeSpecName: "etc-machine-id") pod "2793c42c-363d-4495-88d8-4385bfc9c3fc" (UID: "2793c42c-363d-4495-88d8-4385bfc9c3fc"). InnerVolumeSpecName "etc-machine-id". PluginName "kubernetes.io/host-path", VolumeGidValue "" Dec 03 12:35:03 crc kubenswrapper[5052]: I1203 12:35:03.274997 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2793c42c-363d-4495-88d8-4385bfc9c3fc-scripts" (OuterVolumeSpecName: "scripts") pod "2793c42c-363d-4495-88d8-4385bfc9c3fc" (UID: "2793c42c-363d-4495-88d8-4385bfc9c3fc"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:35:03 crc kubenswrapper[5052]: I1203 12:35:03.276468 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2793c42c-363d-4495-88d8-4385bfc9c3fc-config-data-custom" (OuterVolumeSpecName: "config-data-custom") pod "2793c42c-363d-4495-88d8-4385bfc9c3fc" (UID: "2793c42c-363d-4495-88d8-4385bfc9c3fc"). InnerVolumeSpecName "config-data-custom". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:35:03 crc kubenswrapper[5052]: I1203 12:35:03.282865 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2793c42c-363d-4495-88d8-4385bfc9c3fc-kube-api-access-7kmjb" (OuterVolumeSpecName: "kube-api-access-7kmjb") pod "2793c42c-363d-4495-88d8-4385bfc9c3fc" (UID: "2793c42c-363d-4495-88d8-4385bfc9c3fc"). InnerVolumeSpecName "kube-api-access-7kmjb". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:35:03 crc kubenswrapper[5052]: I1203 12:35:03.309961 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2793c42c-363d-4495-88d8-4385bfc9c3fc-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "2793c42c-363d-4495-88d8-4385bfc9c3fc" (UID: "2793c42c-363d-4495-88d8-4385bfc9c3fc"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:35:03 crc kubenswrapper[5052]: I1203 12:35:03.358229 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/2793c42c-363d-4495-88d8-4385bfc9c3fc-config-data-custom\") on node \"crc\" DevicePath \"\"" Dec 03 12:35:03 crc kubenswrapper[5052]: I1203 12:35:03.358277 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2793c42c-363d-4495-88d8-4385bfc9c3fc-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:35:03 crc kubenswrapper[5052]: I1203 12:35:03.358295 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-7kmjb\" (UniqueName: \"kubernetes.io/projected/2793c42c-363d-4495-88d8-4385bfc9c3fc-kube-api-access-7kmjb\") on node \"crc\" DevicePath \"\"" Dec 03 12:35:03 crc kubenswrapper[5052]: I1203 12:35:03.358308 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/2793c42c-363d-4495-88d8-4385bfc9c3fc-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:35:03 crc kubenswrapper[5052]: I1203 12:35:03.358321 5052 reconciler_common.go:293] "Volume detached for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/2793c42c-363d-4495-88d8-4385bfc9c3fc-etc-machine-id\") on node \"crc\" DevicePath \"\"" Dec 03 12:35:03 crc kubenswrapper[5052]: I1203 12:35:03.375975 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2793c42c-363d-4495-88d8-4385bfc9c3fc-config-data" (OuterVolumeSpecName: "config-data") pod "2793c42c-363d-4495-88d8-4385bfc9c3fc" (UID: "2793c42c-363d-4495-88d8-4385bfc9c3fc"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:35:03 crc kubenswrapper[5052]: I1203 12:35:03.440927 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-db-sync-5kww4" Dec 03 12:35:03 crc kubenswrapper[5052]: I1203 12:35:03.469744 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/2793c42c-363d-4495-88d8-4385bfc9c3fc-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:35:03 crc kubenswrapper[5052]: I1203 12:35:03.570785 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-spzbr\" (UniqueName: \"kubernetes.io/projected/348487de-7480-462b-9f22-56a555321be8-kube-api-access-spzbr\") pod \"348487de-7480-462b-9f22-56a555321be8\" (UID: \"348487de-7480-462b-9f22-56a555321be8\") " Dec 03 12:35:03 crc kubenswrapper[5052]: I1203 12:35:03.570871 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/348487de-7480-462b-9f22-56a555321be8-combined-ca-bundle\") pod \"348487de-7480-462b-9f22-56a555321be8\" (UID: \"348487de-7480-462b-9f22-56a555321be8\") " Dec 03 12:35:03 crc kubenswrapper[5052]: I1203 12:35:03.570985 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/348487de-7480-462b-9f22-56a555321be8-config\") pod \"348487de-7480-462b-9f22-56a555321be8\" (UID: \"348487de-7480-462b-9f22-56a555321be8\") " Dec 03 12:35:03 crc kubenswrapper[5052]: I1203 12:35:03.574244 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/348487de-7480-462b-9f22-56a555321be8-kube-api-access-spzbr" (OuterVolumeSpecName: "kube-api-access-spzbr") pod "348487de-7480-462b-9f22-56a555321be8" (UID: "348487de-7480-462b-9f22-56a555321be8"). InnerVolumeSpecName "kube-api-access-spzbr". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:35:03 crc kubenswrapper[5052]: I1203 12:35:03.598289 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/348487de-7480-462b-9f22-56a555321be8-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "348487de-7480-462b-9f22-56a555321be8" (UID: "348487de-7480-462b-9f22-56a555321be8"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:35:03 crc kubenswrapper[5052]: I1203 12:35:03.599991 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/348487de-7480-462b-9f22-56a555321be8-config" (OuterVolumeSpecName: "config") pod "348487de-7480-462b-9f22-56a555321be8" (UID: "348487de-7480-462b-9f22-56a555321be8"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:35:03 crc kubenswrapper[5052]: I1203 12:35:03.673644 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-spzbr\" (UniqueName: \"kubernetes.io/projected/348487de-7480-462b-9f22-56a555321be8-kube-api-access-spzbr\") on node \"crc\" DevicePath \"\"" Dec 03 12:35:03 crc kubenswrapper[5052]: I1203 12:35:03.673923 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/348487de-7480-462b-9f22-56a555321be8-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:35:03 crc kubenswrapper[5052]: I1203 12:35:03.674004 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/348487de-7480-462b-9f22-56a555321be8-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:35:03 crc kubenswrapper[5052]: I1203 12:35:03.704568 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8a22a52b-d45b-4678-900d-a9d6d5113da3" path="/var/lib/kubelet/pods/8a22a52b-d45b-4678-900d-a9d6d5113da3/volumes" Dec 03 12:35:04 crc kubenswrapper[5052]: I1203 12:35:04.007677 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"2793c42c-363d-4495-88d8-4385bfc9c3fc","Type":"ContainerDied","Data":"7634a1171510b5a7027061193ec392fdd1fab2f90deb2accca4c473913c2b91c"} Dec 03 12:35:04 crc kubenswrapper[5052]: I1203 12:35:04.007736 5052 scope.go:117] "RemoveContainer" containerID="2d2114af30c1e322b378894db0939eaa23284ec865c947f9868cf108b43d4e16" Dec 03 12:35:04 crc kubenswrapper[5052]: I1203 12:35:04.007744 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:35:04 crc kubenswrapper[5052]: I1203 12:35:04.009996 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-db-sync-5kww4" event={"ID":"348487de-7480-462b-9f22-56a555321be8","Type":"ContainerDied","Data":"705a14d8cdcd09593362ce67b9e841465fa1498b9c423b5b91d3146581895ab3"} Dec 03 12:35:04 crc kubenswrapper[5052]: I1203 12:35:04.010033 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="705a14d8cdcd09593362ce67b9e841465fa1498b9c423b5b91d3146581895ab3" Dec 03 12:35:04 crc kubenswrapper[5052]: I1203 12:35:04.010118 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-db-sync-5kww4" Dec 03 12:35:04 crc kubenswrapper[5052]: I1203 12:35:04.048726 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 12:35:04 crc kubenswrapper[5052]: I1203 12:35:04.052705 5052 scope.go:117] "RemoveContainer" containerID="b95bd6a82cabea6eab48bbfdbb5b46e696325061b2a6bdf6f39cab85c095c7dd" Dec 03 12:35:04 crc kubenswrapper[5052]: I1203 12:35:04.059142 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 12:35:04 crc kubenswrapper[5052]: I1203 12:35:04.083216 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 12:35:04 crc kubenswrapper[5052]: E1203 12:35:04.083627 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="8a22a52b-d45b-4678-900d-a9d6d5113da3" containerName="barbican-api" Dec 03 12:35:04 crc kubenswrapper[5052]: I1203 12:35:04.083651 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="8a22a52b-d45b-4678-900d-a9d6d5113da3" containerName="barbican-api" Dec 03 12:35:04 crc kubenswrapper[5052]: E1203 12:35:04.083671 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="8a22a52b-d45b-4678-900d-a9d6d5113da3" containerName="barbican-api-log" Dec 03 12:35:04 crc kubenswrapper[5052]: I1203 12:35:04.083683 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="8a22a52b-d45b-4678-900d-a9d6d5113da3" containerName="barbican-api-log" Dec 03 12:35:04 crc kubenswrapper[5052]: E1203 12:35:04.083700 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="348487de-7480-462b-9f22-56a555321be8" containerName="neutron-db-sync" Dec 03 12:35:04 crc kubenswrapper[5052]: I1203 12:35:04.083707 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="348487de-7480-462b-9f22-56a555321be8" containerName="neutron-db-sync" Dec 03 12:35:04 crc kubenswrapper[5052]: E1203 12:35:04.083723 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="2793c42c-363d-4495-88d8-4385bfc9c3fc" containerName="cinder-scheduler" Dec 03 12:35:04 crc kubenswrapper[5052]: I1203 12:35:04.083729 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="2793c42c-363d-4495-88d8-4385bfc9c3fc" containerName="cinder-scheduler" Dec 03 12:35:04 crc kubenswrapper[5052]: E1203 12:35:04.083753 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="2793c42c-363d-4495-88d8-4385bfc9c3fc" containerName="probe" Dec 03 12:35:04 crc kubenswrapper[5052]: I1203 12:35:04.083760 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="2793c42c-363d-4495-88d8-4385bfc9c3fc" containerName="probe" Dec 03 12:35:04 crc kubenswrapper[5052]: I1203 12:35:04.083931 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="2793c42c-363d-4495-88d8-4385bfc9c3fc" containerName="cinder-scheduler" Dec 03 12:35:04 crc kubenswrapper[5052]: I1203 12:35:04.083948 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="2793c42c-363d-4495-88d8-4385bfc9c3fc" containerName="probe" Dec 03 12:35:04 crc kubenswrapper[5052]: I1203 12:35:04.083993 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="348487de-7480-462b-9f22-56a555321be8" containerName="neutron-db-sync" Dec 03 12:35:04 crc kubenswrapper[5052]: I1203 12:35:04.084011 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="8a22a52b-d45b-4678-900d-a9d6d5113da3" containerName="barbican-api" Dec 03 12:35:04 crc kubenswrapper[5052]: I1203 12:35:04.084020 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="8a22a52b-d45b-4678-900d-a9d6d5113da3" containerName="barbican-api-log" Dec 03 12:35:04 crc kubenswrapper[5052]: I1203 12:35:04.085182 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:35:04 crc kubenswrapper[5052]: I1203 12:35:04.087911 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cinder-scheduler-config-data" Dec 03 12:35:04 crc kubenswrapper[5052]: I1203 12:35:04.104496 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 12:35:04 crc kubenswrapper[5052]: I1203 12:35:04.185071 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/0a42958a-1720-45c8-a4d9-d38a24a246d5-config-data-custom\") pod \"cinder-scheduler-0\" (UID: \"0a42958a-1720-45c8-a4d9-d38a24a246d5\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:35:04 crc kubenswrapper[5052]: I1203 12:35:04.185166 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/0a42958a-1720-45c8-a4d9-d38a24a246d5-etc-machine-id\") pod \"cinder-scheduler-0\" (UID: \"0a42958a-1720-45c8-a4d9-d38a24a246d5\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:35:04 crc kubenswrapper[5052]: I1203 12:35:04.185199 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/0a42958a-1720-45c8-a4d9-d38a24a246d5-scripts\") pod \"cinder-scheduler-0\" (UID: \"0a42958a-1720-45c8-a4d9-d38a24a246d5\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:35:04 crc kubenswrapper[5052]: I1203 12:35:04.185232 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0a42958a-1720-45c8-a4d9-d38a24a246d5-combined-ca-bundle\") pod \"cinder-scheduler-0\" (UID: \"0a42958a-1720-45c8-a4d9-d38a24a246d5\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:35:04 crc kubenswrapper[5052]: I1203 12:35:04.185267 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/0a42958a-1720-45c8-a4d9-d38a24a246d5-config-data\") pod \"cinder-scheduler-0\" (UID: \"0a42958a-1720-45c8-a4d9-d38a24a246d5\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:35:04 crc kubenswrapper[5052]: I1203 12:35:04.185380 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-68c2k\" (UniqueName: \"kubernetes.io/projected/0a42958a-1720-45c8-a4d9-d38a24a246d5-kube-api-access-68c2k\") pod \"cinder-scheduler-0\" (UID: \"0a42958a-1720-45c8-a4d9-d38a24a246d5\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:35:04 crc kubenswrapper[5052]: I1203 12:35:04.192892 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/neutron-9464b48df-szjp9"] Dec 03 12:35:04 crc kubenswrapper[5052]: I1203 12:35:04.194641 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-9464b48df-szjp9" Dec 03 12:35:04 crc kubenswrapper[5052]: I1203 12:35:04.196471 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"neutron-config" Dec 03 12:35:04 crc kubenswrapper[5052]: I1203 12:35:04.196832 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"neutron-neutron-dockercfg-7kmvw" Dec 03 12:35:04 crc kubenswrapper[5052]: I1203 12:35:04.197087 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"neutron-httpd-config" Dec 03 12:35:04 crc kubenswrapper[5052]: I1203 12:35:04.197266 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-neutron-ovndbs" Dec 03 12:35:04 crc kubenswrapper[5052]: I1203 12:35:04.209435 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-9464b48df-szjp9"] Dec 03 12:35:04 crc kubenswrapper[5052]: I1203 12:35:04.286639 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/0a42958a-1720-45c8-a4d9-d38a24a246d5-etc-machine-id\") pod \"cinder-scheduler-0\" (UID: \"0a42958a-1720-45c8-a4d9-d38a24a246d5\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:35:04 crc kubenswrapper[5052]: I1203 12:35:04.286687 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/0a42958a-1720-45c8-a4d9-d38a24a246d5-scripts\") pod \"cinder-scheduler-0\" (UID: \"0a42958a-1720-45c8-a4d9-d38a24a246d5\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:35:04 crc kubenswrapper[5052]: I1203 12:35:04.286720 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0a42958a-1720-45c8-a4d9-d38a24a246d5-combined-ca-bundle\") pod \"cinder-scheduler-0\" (UID: \"0a42958a-1720-45c8-a4d9-d38a24a246d5\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:35:04 crc kubenswrapper[5052]: I1203 12:35:04.286750 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/0a42958a-1720-45c8-a4d9-d38a24a246d5-config-data\") pod \"cinder-scheduler-0\" (UID: \"0a42958a-1720-45c8-a4d9-d38a24a246d5\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:35:04 crc kubenswrapper[5052]: I1203 12:35:04.286810 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-68c2k\" (UniqueName: \"kubernetes.io/projected/0a42958a-1720-45c8-a4d9-d38a24a246d5-kube-api-access-68c2k\") pod \"cinder-scheduler-0\" (UID: \"0a42958a-1720-45c8-a4d9-d38a24a246d5\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:35:04 crc kubenswrapper[5052]: I1203 12:35:04.286833 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/f83ec921-2a86-43a8-8d18-3c66c694682e-ovndb-tls-certs\") pod \"neutron-9464b48df-szjp9\" (UID: \"f83ec921-2a86-43a8-8d18-3c66c694682e\") " pod="openstack-kuttl-tests/neutron-9464b48df-szjp9" Dec 03 12:35:04 crc kubenswrapper[5052]: I1203 12:35:04.286857 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/f83ec921-2a86-43a8-8d18-3c66c694682e-config\") pod \"neutron-9464b48df-szjp9\" (UID: \"f83ec921-2a86-43a8-8d18-3c66c694682e\") " pod="openstack-kuttl-tests/neutron-9464b48df-szjp9" Dec 03 12:35:04 crc kubenswrapper[5052]: I1203 12:35:04.286891 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bl86d\" (UniqueName: \"kubernetes.io/projected/f83ec921-2a86-43a8-8d18-3c66c694682e-kube-api-access-bl86d\") pod \"neutron-9464b48df-szjp9\" (UID: \"f83ec921-2a86-43a8-8d18-3c66c694682e\") " pod="openstack-kuttl-tests/neutron-9464b48df-szjp9" Dec 03 12:35:04 crc kubenswrapper[5052]: I1203 12:35:04.286932 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/f83ec921-2a86-43a8-8d18-3c66c694682e-httpd-config\") pod \"neutron-9464b48df-szjp9\" (UID: \"f83ec921-2a86-43a8-8d18-3c66c694682e\") " pod="openstack-kuttl-tests/neutron-9464b48df-szjp9" Dec 03 12:35:04 crc kubenswrapper[5052]: I1203 12:35:04.286953 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/0a42958a-1720-45c8-a4d9-d38a24a246d5-config-data-custom\") pod \"cinder-scheduler-0\" (UID: \"0a42958a-1720-45c8-a4d9-d38a24a246d5\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:35:04 crc kubenswrapper[5052]: I1203 12:35:04.286971 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f83ec921-2a86-43a8-8d18-3c66c694682e-combined-ca-bundle\") pod \"neutron-9464b48df-szjp9\" (UID: \"f83ec921-2a86-43a8-8d18-3c66c694682e\") " pod="openstack-kuttl-tests/neutron-9464b48df-szjp9" Dec 03 12:35:04 crc kubenswrapper[5052]: I1203 12:35:04.287079 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/0a42958a-1720-45c8-a4d9-d38a24a246d5-etc-machine-id\") pod \"cinder-scheduler-0\" (UID: \"0a42958a-1720-45c8-a4d9-d38a24a246d5\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:35:04 crc kubenswrapper[5052]: I1203 12:35:04.292212 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/0a42958a-1720-45c8-a4d9-d38a24a246d5-config-data-custom\") pod \"cinder-scheduler-0\" (UID: \"0a42958a-1720-45c8-a4d9-d38a24a246d5\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:35:04 crc kubenswrapper[5052]: I1203 12:35:04.295097 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/0a42958a-1720-45c8-a4d9-d38a24a246d5-scripts\") pod \"cinder-scheduler-0\" (UID: \"0a42958a-1720-45c8-a4d9-d38a24a246d5\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:35:04 crc kubenswrapper[5052]: I1203 12:35:04.295485 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/0a42958a-1720-45c8-a4d9-d38a24a246d5-config-data\") pod \"cinder-scheduler-0\" (UID: \"0a42958a-1720-45c8-a4d9-d38a24a246d5\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:35:04 crc kubenswrapper[5052]: I1203 12:35:04.295482 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0a42958a-1720-45c8-a4d9-d38a24a246d5-combined-ca-bundle\") pod \"cinder-scheduler-0\" (UID: \"0a42958a-1720-45c8-a4d9-d38a24a246d5\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:35:04 crc kubenswrapper[5052]: I1203 12:35:04.309312 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-68c2k\" (UniqueName: \"kubernetes.io/projected/0a42958a-1720-45c8-a4d9-d38a24a246d5-kube-api-access-68c2k\") pod \"cinder-scheduler-0\" (UID: \"0a42958a-1720-45c8-a4d9-d38a24a246d5\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:35:04 crc kubenswrapper[5052]: I1203 12:35:04.388716 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/f83ec921-2a86-43a8-8d18-3c66c694682e-ovndb-tls-certs\") pod \"neutron-9464b48df-szjp9\" (UID: \"f83ec921-2a86-43a8-8d18-3c66c694682e\") " pod="openstack-kuttl-tests/neutron-9464b48df-szjp9" Dec 03 12:35:04 crc kubenswrapper[5052]: I1203 12:35:04.388771 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/f83ec921-2a86-43a8-8d18-3c66c694682e-config\") pod \"neutron-9464b48df-szjp9\" (UID: \"f83ec921-2a86-43a8-8d18-3c66c694682e\") " pod="openstack-kuttl-tests/neutron-9464b48df-szjp9" Dec 03 12:35:04 crc kubenswrapper[5052]: I1203 12:35:04.389087 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-bl86d\" (UniqueName: \"kubernetes.io/projected/f83ec921-2a86-43a8-8d18-3c66c694682e-kube-api-access-bl86d\") pod \"neutron-9464b48df-szjp9\" (UID: \"f83ec921-2a86-43a8-8d18-3c66c694682e\") " pod="openstack-kuttl-tests/neutron-9464b48df-szjp9" Dec 03 12:35:04 crc kubenswrapper[5052]: I1203 12:35:04.389137 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/f83ec921-2a86-43a8-8d18-3c66c694682e-httpd-config\") pod \"neutron-9464b48df-szjp9\" (UID: \"f83ec921-2a86-43a8-8d18-3c66c694682e\") " pod="openstack-kuttl-tests/neutron-9464b48df-szjp9" Dec 03 12:35:04 crc kubenswrapper[5052]: I1203 12:35:04.389168 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f83ec921-2a86-43a8-8d18-3c66c694682e-combined-ca-bundle\") pod \"neutron-9464b48df-szjp9\" (UID: \"f83ec921-2a86-43a8-8d18-3c66c694682e\") " pod="openstack-kuttl-tests/neutron-9464b48df-szjp9" Dec 03 12:35:04 crc kubenswrapper[5052]: I1203 12:35:04.392830 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f83ec921-2a86-43a8-8d18-3c66c694682e-combined-ca-bundle\") pod \"neutron-9464b48df-szjp9\" (UID: \"f83ec921-2a86-43a8-8d18-3c66c694682e\") " pod="openstack-kuttl-tests/neutron-9464b48df-szjp9" Dec 03 12:35:04 crc kubenswrapper[5052]: I1203 12:35:04.392874 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/f83ec921-2a86-43a8-8d18-3c66c694682e-httpd-config\") pod \"neutron-9464b48df-szjp9\" (UID: \"f83ec921-2a86-43a8-8d18-3c66c694682e\") " pod="openstack-kuttl-tests/neutron-9464b48df-szjp9" Dec 03 12:35:04 crc kubenswrapper[5052]: I1203 12:35:04.393430 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/f83ec921-2a86-43a8-8d18-3c66c694682e-ovndb-tls-certs\") pod \"neutron-9464b48df-szjp9\" (UID: \"f83ec921-2a86-43a8-8d18-3c66c694682e\") " pod="openstack-kuttl-tests/neutron-9464b48df-szjp9" Dec 03 12:35:04 crc kubenswrapper[5052]: I1203 12:35:04.393551 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/f83ec921-2a86-43a8-8d18-3c66c694682e-config\") pod \"neutron-9464b48df-szjp9\" (UID: \"f83ec921-2a86-43a8-8d18-3c66c694682e\") " pod="openstack-kuttl-tests/neutron-9464b48df-szjp9" Dec 03 12:35:04 crc kubenswrapper[5052]: I1203 12:35:04.410277 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-bl86d\" (UniqueName: \"kubernetes.io/projected/f83ec921-2a86-43a8-8d18-3c66c694682e-kube-api-access-bl86d\") pod \"neutron-9464b48df-szjp9\" (UID: \"f83ec921-2a86-43a8-8d18-3c66c694682e\") " pod="openstack-kuttl-tests/neutron-9464b48df-szjp9" Dec 03 12:35:04 crc kubenswrapper[5052]: I1203 12:35:04.425177 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:35:04 crc kubenswrapper[5052]: I1203 12:35:04.522159 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-9464b48df-szjp9" Dec 03 12:35:04 crc kubenswrapper[5052]: I1203 12:35:04.916046 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 12:35:05 crc kubenswrapper[5052]: I1203 12:35:05.032437 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"0a42958a-1720-45c8-a4d9-d38a24a246d5","Type":"ContainerStarted","Data":"4e1e8789b29e739db27c88cba0630606de7a397bef539d4c80735c38b80e05b2"} Dec 03 12:35:05 crc kubenswrapper[5052]: W1203 12:35:05.055487 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf83ec921_2a86_43a8_8d18_3c66c694682e.slice/crio-0603b91b3658322a1469c7c9070914904c3e2a3e6f752ab6f1357bb2fe5a6c46 WatchSource:0}: Error finding container 0603b91b3658322a1469c7c9070914904c3e2a3e6f752ab6f1357bb2fe5a6c46: Status 404 returned error can't find the container with id 0603b91b3658322a1469c7c9070914904c3e2a3e6f752ab6f1357bb2fe5a6c46 Dec 03 12:35:05 crc kubenswrapper[5052]: I1203 12:35:05.062811 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-9464b48df-szjp9"] Dec 03 12:35:05 crc kubenswrapper[5052]: I1203 12:35:05.274985 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:35:05 crc kubenswrapper[5052]: I1203 12:35:05.684615 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2793c42c-363d-4495-88d8-4385bfc9c3fc" path="/var/lib/kubelet/pods/2793c42c-363d-4495-88d8-4385bfc9c3fc/volumes" Dec 03 12:35:06 crc kubenswrapper[5052]: I1203 12:35:06.051850 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"0a42958a-1720-45c8-a4d9-d38a24a246d5","Type":"ContainerStarted","Data":"57014359344dd7eed908e5209d6cd12ff8506d0d739a8eaa92efff47d08930e9"} Dec 03 12:35:06 crc kubenswrapper[5052]: I1203 12:35:06.053593 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-9464b48df-szjp9" event={"ID":"f83ec921-2a86-43a8-8d18-3c66c694682e","Type":"ContainerStarted","Data":"4aac3f358c0e898b24af2679d198e356f499489d60cbf8a19caaf910f13f9de4"} Dec 03 12:35:06 crc kubenswrapper[5052]: I1203 12:35:06.053619 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-9464b48df-szjp9" event={"ID":"f83ec921-2a86-43a8-8d18-3c66c694682e","Type":"ContainerStarted","Data":"0603b91b3658322a1469c7c9070914904c3e2a3e6f752ab6f1357bb2fe5a6c46"} Dec 03 12:35:06 crc kubenswrapper[5052]: I1203 12:35:06.630655 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/neutron-5d67d5bcdb-klgzj"] Dec 03 12:35:06 crc kubenswrapper[5052]: I1203 12:35:06.635300 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-5d67d5bcdb-klgzj" Dec 03 12:35:06 crc kubenswrapper[5052]: I1203 12:35:06.640454 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-5d67d5bcdb-klgzj"] Dec 03 12:35:06 crc kubenswrapper[5052]: I1203 12:35:06.642317 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-neutron-internal-svc" Dec 03 12:35:06 crc kubenswrapper[5052]: I1203 12:35:06.642745 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-neutron-public-svc" Dec 03 12:35:06 crc kubenswrapper[5052]: I1203 12:35:06.745970 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/4426a4f1-76e6-4746-a88a-4f4efead3977-internal-tls-certs\") pod \"neutron-5d67d5bcdb-klgzj\" (UID: \"4426a4f1-76e6-4746-a88a-4f4efead3977\") " pod="openstack-kuttl-tests/neutron-5d67d5bcdb-klgzj" Dec 03 12:35:06 crc kubenswrapper[5052]: I1203 12:35:06.746069 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/4426a4f1-76e6-4746-a88a-4f4efead3977-config\") pod \"neutron-5d67d5bcdb-klgzj\" (UID: \"4426a4f1-76e6-4746-a88a-4f4efead3977\") " pod="openstack-kuttl-tests/neutron-5d67d5bcdb-klgzj" Dec 03 12:35:06 crc kubenswrapper[5052]: I1203 12:35:06.746114 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/4426a4f1-76e6-4746-a88a-4f4efead3977-ovndb-tls-certs\") pod \"neutron-5d67d5bcdb-klgzj\" (UID: \"4426a4f1-76e6-4746-a88a-4f4efead3977\") " pod="openstack-kuttl-tests/neutron-5d67d5bcdb-klgzj" Dec 03 12:35:06 crc kubenswrapper[5052]: I1203 12:35:06.746157 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/4426a4f1-76e6-4746-a88a-4f4efead3977-public-tls-certs\") pod \"neutron-5d67d5bcdb-klgzj\" (UID: \"4426a4f1-76e6-4746-a88a-4f4efead3977\") " pod="openstack-kuttl-tests/neutron-5d67d5bcdb-klgzj" Dec 03 12:35:06 crc kubenswrapper[5052]: I1203 12:35:06.746181 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4426a4f1-76e6-4746-a88a-4f4efead3977-combined-ca-bundle\") pod \"neutron-5d67d5bcdb-klgzj\" (UID: \"4426a4f1-76e6-4746-a88a-4f4efead3977\") " pod="openstack-kuttl-tests/neutron-5d67d5bcdb-klgzj" Dec 03 12:35:06 crc kubenswrapper[5052]: I1203 12:35:06.746259 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/4426a4f1-76e6-4746-a88a-4f4efead3977-httpd-config\") pod \"neutron-5d67d5bcdb-klgzj\" (UID: \"4426a4f1-76e6-4746-a88a-4f4efead3977\") " pod="openstack-kuttl-tests/neutron-5d67d5bcdb-klgzj" Dec 03 12:35:06 crc kubenswrapper[5052]: I1203 12:35:06.746324 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-slhbg\" (UniqueName: \"kubernetes.io/projected/4426a4f1-76e6-4746-a88a-4f4efead3977-kube-api-access-slhbg\") pod \"neutron-5d67d5bcdb-klgzj\" (UID: \"4426a4f1-76e6-4746-a88a-4f4efead3977\") " pod="openstack-kuttl-tests/neutron-5d67d5bcdb-klgzj" Dec 03 12:35:06 crc kubenswrapper[5052]: I1203 12:35:06.848025 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/4426a4f1-76e6-4746-a88a-4f4efead3977-internal-tls-certs\") pod \"neutron-5d67d5bcdb-klgzj\" (UID: \"4426a4f1-76e6-4746-a88a-4f4efead3977\") " pod="openstack-kuttl-tests/neutron-5d67d5bcdb-klgzj" Dec 03 12:35:06 crc kubenswrapper[5052]: I1203 12:35:06.848095 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/4426a4f1-76e6-4746-a88a-4f4efead3977-config\") pod \"neutron-5d67d5bcdb-klgzj\" (UID: \"4426a4f1-76e6-4746-a88a-4f4efead3977\") " pod="openstack-kuttl-tests/neutron-5d67d5bcdb-klgzj" Dec 03 12:35:06 crc kubenswrapper[5052]: I1203 12:35:06.848128 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/4426a4f1-76e6-4746-a88a-4f4efead3977-ovndb-tls-certs\") pod \"neutron-5d67d5bcdb-klgzj\" (UID: \"4426a4f1-76e6-4746-a88a-4f4efead3977\") " pod="openstack-kuttl-tests/neutron-5d67d5bcdb-klgzj" Dec 03 12:35:06 crc kubenswrapper[5052]: I1203 12:35:06.848153 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/4426a4f1-76e6-4746-a88a-4f4efead3977-public-tls-certs\") pod \"neutron-5d67d5bcdb-klgzj\" (UID: \"4426a4f1-76e6-4746-a88a-4f4efead3977\") " pod="openstack-kuttl-tests/neutron-5d67d5bcdb-klgzj" Dec 03 12:35:06 crc kubenswrapper[5052]: I1203 12:35:06.848172 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4426a4f1-76e6-4746-a88a-4f4efead3977-combined-ca-bundle\") pod \"neutron-5d67d5bcdb-klgzj\" (UID: \"4426a4f1-76e6-4746-a88a-4f4efead3977\") " pod="openstack-kuttl-tests/neutron-5d67d5bcdb-klgzj" Dec 03 12:35:06 crc kubenswrapper[5052]: I1203 12:35:06.848207 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/4426a4f1-76e6-4746-a88a-4f4efead3977-httpd-config\") pod \"neutron-5d67d5bcdb-klgzj\" (UID: \"4426a4f1-76e6-4746-a88a-4f4efead3977\") " pod="openstack-kuttl-tests/neutron-5d67d5bcdb-klgzj" Dec 03 12:35:06 crc kubenswrapper[5052]: I1203 12:35:06.848246 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-slhbg\" (UniqueName: \"kubernetes.io/projected/4426a4f1-76e6-4746-a88a-4f4efead3977-kube-api-access-slhbg\") pod \"neutron-5d67d5bcdb-klgzj\" (UID: \"4426a4f1-76e6-4746-a88a-4f4efead3977\") " pod="openstack-kuttl-tests/neutron-5d67d5bcdb-klgzj" Dec 03 12:35:06 crc kubenswrapper[5052]: I1203 12:35:06.854161 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/4426a4f1-76e6-4746-a88a-4f4efead3977-internal-tls-certs\") pod \"neutron-5d67d5bcdb-klgzj\" (UID: \"4426a4f1-76e6-4746-a88a-4f4efead3977\") " pod="openstack-kuttl-tests/neutron-5d67d5bcdb-klgzj" Dec 03 12:35:06 crc kubenswrapper[5052]: I1203 12:35:06.854411 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/4426a4f1-76e6-4746-a88a-4f4efead3977-config\") pod \"neutron-5d67d5bcdb-klgzj\" (UID: \"4426a4f1-76e6-4746-a88a-4f4efead3977\") " pod="openstack-kuttl-tests/neutron-5d67d5bcdb-klgzj" Dec 03 12:35:06 crc kubenswrapper[5052]: I1203 12:35:06.854778 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/4426a4f1-76e6-4746-a88a-4f4efead3977-public-tls-certs\") pod \"neutron-5d67d5bcdb-klgzj\" (UID: \"4426a4f1-76e6-4746-a88a-4f4efead3977\") " pod="openstack-kuttl-tests/neutron-5d67d5bcdb-klgzj" Dec 03 12:35:06 crc kubenswrapper[5052]: I1203 12:35:06.855122 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4426a4f1-76e6-4746-a88a-4f4efead3977-combined-ca-bundle\") pod \"neutron-5d67d5bcdb-klgzj\" (UID: \"4426a4f1-76e6-4746-a88a-4f4efead3977\") " pod="openstack-kuttl-tests/neutron-5d67d5bcdb-klgzj" Dec 03 12:35:06 crc kubenswrapper[5052]: I1203 12:35:06.855766 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/4426a4f1-76e6-4746-a88a-4f4efead3977-ovndb-tls-certs\") pod \"neutron-5d67d5bcdb-klgzj\" (UID: \"4426a4f1-76e6-4746-a88a-4f4efead3977\") " pod="openstack-kuttl-tests/neutron-5d67d5bcdb-klgzj" Dec 03 12:35:06 crc kubenswrapper[5052]: I1203 12:35:06.869642 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/4426a4f1-76e6-4746-a88a-4f4efead3977-httpd-config\") pod \"neutron-5d67d5bcdb-klgzj\" (UID: \"4426a4f1-76e6-4746-a88a-4f4efead3977\") " pod="openstack-kuttl-tests/neutron-5d67d5bcdb-klgzj" Dec 03 12:35:06 crc kubenswrapper[5052]: I1203 12:35:06.885564 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-slhbg\" (UniqueName: \"kubernetes.io/projected/4426a4f1-76e6-4746-a88a-4f4efead3977-kube-api-access-slhbg\") pod \"neutron-5d67d5bcdb-klgzj\" (UID: \"4426a4f1-76e6-4746-a88a-4f4efead3977\") " pod="openstack-kuttl-tests/neutron-5d67d5bcdb-klgzj" Dec 03 12:35:06 crc kubenswrapper[5052]: I1203 12:35:06.961297 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-5d67d5bcdb-klgzj" Dec 03 12:35:07 crc kubenswrapper[5052]: I1203 12:35:07.072078 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"0a42958a-1720-45c8-a4d9-d38a24a246d5","Type":"ContainerStarted","Data":"bf2ad3eb69b0929f58698c3117f643b29a34469bb0e38c60eb24ebaa0a6a5ea1"} Dec 03 12:35:07 crc kubenswrapper[5052]: I1203 12:35:07.080018 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-9464b48df-szjp9" event={"ID":"f83ec921-2a86-43a8-8d18-3c66c694682e","Type":"ContainerStarted","Data":"96457f722879e7f4b70f63c2fb15496d232c98398206c25a1f690ebf24ca728b"} Dec 03 12:35:07 crc kubenswrapper[5052]: I1203 12:35:07.080926 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/neutron-9464b48df-szjp9" Dec 03 12:35:07 crc kubenswrapper[5052]: I1203 12:35:07.111857 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/cinder-scheduler-0" podStartSLOduration=3.11183391 podStartE2EDuration="3.11183391s" podCreationTimestamp="2025-12-03 12:35:04 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:35:07.098078777 +0000 UTC m=+4497.686320541" watchObservedRunningTime="2025-12-03 12:35:07.11183391 +0000 UTC m=+4497.700075674" Dec 03 12:35:07 crc kubenswrapper[5052]: I1203 12:35:07.121663 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/neutron-9464b48df-szjp9" podStartSLOduration=3.121636581 podStartE2EDuration="3.121636581s" podCreationTimestamp="2025-12-03 12:35:04 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:35:07.116977047 +0000 UTC m=+4497.705218811" watchObservedRunningTime="2025-12-03 12:35:07.121636581 +0000 UTC m=+4497.709878345" Dec 03 12:35:07 crc kubenswrapper[5052]: I1203 12:35:07.446640 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-5d67d5bcdb-klgzj"] Dec 03 12:35:07 crc kubenswrapper[5052]: W1203 12:35:07.447904 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4426a4f1_76e6_4746_a88a_4f4efead3977.slice/crio-de9dd63521e94c20c668e631d3ac331c247e96c7bb5868ebc0db0547180dc83c WatchSource:0}: Error finding container de9dd63521e94c20c668e631d3ac331c247e96c7bb5868ebc0db0547180dc83c: Status 404 returned error can't find the container with id de9dd63521e94c20c668e631d3ac331c247e96c7bb5868ebc0db0547180dc83c Dec 03 12:35:08 crc kubenswrapper[5052]: I1203 12:35:08.095233 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-5d67d5bcdb-klgzj" event={"ID":"4426a4f1-76e6-4746-a88a-4f4efead3977","Type":"ContainerStarted","Data":"1a7463968b45581a86dc3faca6452efdbcfd78fb18a9ccb7360af588a2bd68a5"} Dec 03 12:35:08 crc kubenswrapper[5052]: I1203 12:35:08.096837 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/neutron-5d67d5bcdb-klgzj" Dec 03 12:35:08 crc kubenswrapper[5052]: I1203 12:35:08.096948 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-5d67d5bcdb-klgzj" event={"ID":"4426a4f1-76e6-4746-a88a-4f4efead3977","Type":"ContainerStarted","Data":"dde274eb8e16ff85ae1b2ee940add419f52d4e53a71e9f14e7931dcb8897d9fe"} Dec 03 12:35:08 crc kubenswrapper[5052]: I1203 12:35:08.097059 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-5d67d5bcdb-klgzj" event={"ID":"4426a4f1-76e6-4746-a88a-4f4efead3977","Type":"ContainerStarted","Data":"de9dd63521e94c20c668e631d3ac331c247e96c7bb5868ebc0db0547180dc83c"} Dec 03 12:35:08 crc kubenswrapper[5052]: I1203 12:35:08.122761 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/neutron-5d67d5bcdb-klgzj" podStartSLOduration=2.122738811 podStartE2EDuration="2.122738811s" podCreationTimestamp="2025-12-03 12:35:06 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:35:08.122059481 +0000 UTC m=+4498.710301245" watchObservedRunningTime="2025-12-03 12:35:08.122738811 +0000 UTC m=+4498.710980575" Dec 03 12:35:09 crc kubenswrapper[5052]: I1203 12:35:09.426400 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:35:10 crc kubenswrapper[5052]: I1203 12:35:10.248157 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:35:11 crc kubenswrapper[5052]: I1203 12:35:11.673173 5052 scope.go:117] "RemoveContainer" containerID="e6edaf188f85ea4f11f487f11cc489c700f25fb42b5ee269f4f496f1e127d772" Dec 03 12:35:11 crc kubenswrapper[5052]: E1203 12:35:11.673683 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 12:35:12 crc kubenswrapper[5052]: I1203 12:35:12.362626 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/placement-6cd995dc74-hpvzm" Dec 03 12:35:12 crc kubenswrapper[5052]: I1203 12:35:12.364450 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/placement-6cd995dc74-hpvzm" Dec 03 12:35:14 crc kubenswrapper[5052]: I1203 12:35:14.633308 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:35:20 crc kubenswrapper[5052]: I1203 12:35:20.818643 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/keystone-7476b7fc95-6cxvn" Dec 03 12:35:21 crc kubenswrapper[5052]: I1203 12:35:21.232125 5052 scope.go:117] "RemoveContainer" containerID="cda20389ae104a8a76ec7b7e03e816da57afd5a07e90e7f0e1bf79a5704aa60d" Dec 03 12:35:21 crc kubenswrapper[5052]: I1203 12:35:21.253891 5052 scope.go:117] "RemoveContainer" containerID="e2e4d22232c587b84e25f9aceb08f0b3c39f38badcaa667d82792f3314b6a1b9" Dec 03 12:35:21 crc kubenswrapper[5052]: I1203 12:35:21.276253 5052 scope.go:117] "RemoveContainer" containerID="203cfb7bbd64d8e75d9f87a48f8ac62ffc001878165757ef3e41902cf4c16eca" Dec 03 12:35:21 crc kubenswrapper[5052]: I1203 12:35:21.296136 5052 scope.go:117] "RemoveContainer" containerID="102038f6978c676a1d8853f850be294a8aea8d6b74a6f1949c24231d714a3608" Dec 03 12:35:21 crc kubenswrapper[5052]: I1203 12:35:21.314708 5052 scope.go:117] "RemoveContainer" containerID="a6f27ec51db1c9719c74272ae88d2ed48068153f9cc4d46a92cce3985e31c2e5" Dec 03 12:35:21 crc kubenswrapper[5052]: I1203 12:35:21.334158 5052 scope.go:117] "RemoveContainer" containerID="bc91adc346315718fd765cd7a7a58ccc06d83103b40be9d46a7d2e14981369da" Dec 03 12:35:21 crc kubenswrapper[5052]: I1203 12:35:21.355534 5052 scope.go:117] "RemoveContainer" containerID="bf3135f41d57b4319b491c492f40b5262bdbbb4439ae6f42157eab9d6b44e24e" Dec 03 12:35:21 crc kubenswrapper[5052]: I1203 12:35:21.379459 5052 scope.go:117] "RemoveContainer" containerID="486f511e2e18cfc8ea29b34eeca83447c73622373f7fc22a86772a5bb4c258ab" Dec 03 12:35:21 crc kubenswrapper[5052]: I1203 12:35:21.408467 5052 scope.go:117] "RemoveContainer" containerID="069e9d7b5055dc14bd89a6f68f4d0ccea363e1ff3aad490e83aff00d8a0ad411" Dec 03 12:35:21 crc kubenswrapper[5052]: I1203 12:35:21.439018 5052 scope.go:117] "RemoveContainer" containerID="9b7bab21260f028f58b297b4ef87a5852a2f31508dfedff95c3c79316a694a8f" Dec 03 12:35:21 crc kubenswrapper[5052]: I1203 12:35:21.470965 5052 scope.go:117] "RemoveContainer" containerID="08dfbdd50f0698222b0397a521292a1d56421652d4a20cca08cf4aa13cdeb4db" Dec 03 12:35:21 crc kubenswrapper[5052]: I1203 12:35:21.498781 5052 scope.go:117] "RemoveContainer" containerID="da3d10f0fb54aeb0fa5fdc57f022b12303d52fa87da0b230d95a66aac9d2be59" Dec 03 12:35:22 crc kubenswrapper[5052]: I1203 12:35:22.673684 5052 scope.go:117] "RemoveContainer" containerID="e6edaf188f85ea4f11f487f11cc489c700f25fb42b5ee269f4f496f1e127d772" Dec 03 12:35:22 crc kubenswrapper[5052]: E1203 12:35:22.674205 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 12:35:25 crc kubenswrapper[5052]: I1203 12:35:25.006576 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/swift-proxy-55f4df7cd-4gjcp"] Dec 03 12:35:25 crc kubenswrapper[5052]: I1203 12:35:25.008740 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-proxy-55f4df7cd-4gjcp" Dec 03 12:35:25 crc kubenswrapper[5052]: I1203 12:35:25.014898 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/swift-proxy-55f4df7cd-4gjcp"] Dec 03 12:35:25 crc kubenswrapper[5052]: I1203 12:35:25.018307 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-swift-public-svc" Dec 03 12:35:25 crc kubenswrapper[5052]: I1203 12:35:25.018779 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"swift-proxy-config-data" Dec 03 12:35:25 crc kubenswrapper[5052]: I1203 12:35:25.018974 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-swift-internal-svc" Dec 03 12:35:25 crc kubenswrapper[5052]: I1203 12:35:25.128545 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/06b2a08b-6b91-44f2-b630-169938fe5771-public-tls-certs\") pod \"swift-proxy-55f4df7cd-4gjcp\" (UID: \"06b2a08b-6b91-44f2-b630-169938fe5771\") " pod="openstack-kuttl-tests/swift-proxy-55f4df7cd-4gjcp" Dec 03 12:35:25 crc kubenswrapper[5052]: I1203 12:35:25.128604 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ndmt6\" (UniqueName: \"kubernetes.io/projected/06b2a08b-6b91-44f2-b630-169938fe5771-kube-api-access-ndmt6\") pod \"swift-proxy-55f4df7cd-4gjcp\" (UID: \"06b2a08b-6b91-44f2-b630-169938fe5771\") " pod="openstack-kuttl-tests/swift-proxy-55f4df7cd-4gjcp" Dec 03 12:35:25 crc kubenswrapper[5052]: I1203 12:35:25.128649 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/06b2a08b-6b91-44f2-b630-169938fe5771-run-httpd\") pod \"swift-proxy-55f4df7cd-4gjcp\" (UID: \"06b2a08b-6b91-44f2-b630-169938fe5771\") " pod="openstack-kuttl-tests/swift-proxy-55f4df7cd-4gjcp" Dec 03 12:35:25 crc kubenswrapper[5052]: I1203 12:35:25.129092 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/06b2a08b-6b91-44f2-b630-169938fe5771-combined-ca-bundle\") pod \"swift-proxy-55f4df7cd-4gjcp\" (UID: \"06b2a08b-6b91-44f2-b630-169938fe5771\") " pod="openstack-kuttl-tests/swift-proxy-55f4df7cd-4gjcp" Dec 03 12:35:25 crc kubenswrapper[5052]: I1203 12:35:25.129175 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/06b2a08b-6b91-44f2-b630-169938fe5771-config-data\") pod \"swift-proxy-55f4df7cd-4gjcp\" (UID: \"06b2a08b-6b91-44f2-b630-169938fe5771\") " pod="openstack-kuttl-tests/swift-proxy-55f4df7cd-4gjcp" Dec 03 12:35:25 crc kubenswrapper[5052]: I1203 12:35:25.129234 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/06b2a08b-6b91-44f2-b630-169938fe5771-log-httpd\") pod \"swift-proxy-55f4df7cd-4gjcp\" (UID: \"06b2a08b-6b91-44f2-b630-169938fe5771\") " pod="openstack-kuttl-tests/swift-proxy-55f4df7cd-4gjcp" Dec 03 12:35:25 crc kubenswrapper[5052]: I1203 12:35:25.129252 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/06b2a08b-6b91-44f2-b630-169938fe5771-internal-tls-certs\") pod \"swift-proxy-55f4df7cd-4gjcp\" (UID: \"06b2a08b-6b91-44f2-b630-169938fe5771\") " pod="openstack-kuttl-tests/swift-proxy-55f4df7cd-4gjcp" Dec 03 12:35:25 crc kubenswrapper[5052]: I1203 12:35:25.129268 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/06b2a08b-6b91-44f2-b630-169938fe5771-etc-swift\") pod \"swift-proxy-55f4df7cd-4gjcp\" (UID: \"06b2a08b-6b91-44f2-b630-169938fe5771\") " pod="openstack-kuttl-tests/swift-proxy-55f4df7cd-4gjcp" Dec 03 12:35:25 crc kubenswrapper[5052]: I1203 12:35:25.231201 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/06b2a08b-6b91-44f2-b630-169938fe5771-log-httpd\") pod \"swift-proxy-55f4df7cd-4gjcp\" (UID: \"06b2a08b-6b91-44f2-b630-169938fe5771\") " pod="openstack-kuttl-tests/swift-proxy-55f4df7cd-4gjcp" Dec 03 12:35:25 crc kubenswrapper[5052]: I1203 12:35:25.231262 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/06b2a08b-6b91-44f2-b630-169938fe5771-internal-tls-certs\") pod \"swift-proxy-55f4df7cd-4gjcp\" (UID: \"06b2a08b-6b91-44f2-b630-169938fe5771\") " pod="openstack-kuttl-tests/swift-proxy-55f4df7cd-4gjcp" Dec 03 12:35:25 crc kubenswrapper[5052]: I1203 12:35:25.231291 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/06b2a08b-6b91-44f2-b630-169938fe5771-etc-swift\") pod \"swift-proxy-55f4df7cd-4gjcp\" (UID: \"06b2a08b-6b91-44f2-b630-169938fe5771\") " pod="openstack-kuttl-tests/swift-proxy-55f4df7cd-4gjcp" Dec 03 12:35:25 crc kubenswrapper[5052]: I1203 12:35:25.231376 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/06b2a08b-6b91-44f2-b630-169938fe5771-public-tls-certs\") pod \"swift-proxy-55f4df7cd-4gjcp\" (UID: \"06b2a08b-6b91-44f2-b630-169938fe5771\") " pod="openstack-kuttl-tests/swift-proxy-55f4df7cd-4gjcp" Dec 03 12:35:25 crc kubenswrapper[5052]: I1203 12:35:25.231412 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-ndmt6\" (UniqueName: \"kubernetes.io/projected/06b2a08b-6b91-44f2-b630-169938fe5771-kube-api-access-ndmt6\") pod \"swift-proxy-55f4df7cd-4gjcp\" (UID: \"06b2a08b-6b91-44f2-b630-169938fe5771\") " pod="openstack-kuttl-tests/swift-proxy-55f4df7cd-4gjcp" Dec 03 12:35:25 crc kubenswrapper[5052]: I1203 12:35:25.231447 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/06b2a08b-6b91-44f2-b630-169938fe5771-run-httpd\") pod \"swift-proxy-55f4df7cd-4gjcp\" (UID: \"06b2a08b-6b91-44f2-b630-169938fe5771\") " pod="openstack-kuttl-tests/swift-proxy-55f4df7cd-4gjcp" Dec 03 12:35:25 crc kubenswrapper[5052]: I1203 12:35:25.231611 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/06b2a08b-6b91-44f2-b630-169938fe5771-combined-ca-bundle\") pod \"swift-proxy-55f4df7cd-4gjcp\" (UID: \"06b2a08b-6b91-44f2-b630-169938fe5771\") " pod="openstack-kuttl-tests/swift-proxy-55f4df7cd-4gjcp" Dec 03 12:35:25 crc kubenswrapper[5052]: I1203 12:35:25.231713 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/06b2a08b-6b91-44f2-b630-169938fe5771-config-data\") pod \"swift-proxy-55f4df7cd-4gjcp\" (UID: \"06b2a08b-6b91-44f2-b630-169938fe5771\") " pod="openstack-kuttl-tests/swift-proxy-55f4df7cd-4gjcp" Dec 03 12:35:25 crc kubenswrapper[5052]: I1203 12:35:25.231823 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/06b2a08b-6b91-44f2-b630-169938fe5771-log-httpd\") pod \"swift-proxy-55f4df7cd-4gjcp\" (UID: \"06b2a08b-6b91-44f2-b630-169938fe5771\") " pod="openstack-kuttl-tests/swift-proxy-55f4df7cd-4gjcp" Dec 03 12:35:25 crc kubenswrapper[5052]: I1203 12:35:25.232066 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/06b2a08b-6b91-44f2-b630-169938fe5771-run-httpd\") pod \"swift-proxy-55f4df7cd-4gjcp\" (UID: \"06b2a08b-6b91-44f2-b630-169938fe5771\") " pod="openstack-kuttl-tests/swift-proxy-55f4df7cd-4gjcp" Dec 03 12:35:25 crc kubenswrapper[5052]: I1203 12:35:25.523939 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/06b2a08b-6b91-44f2-b630-169938fe5771-public-tls-certs\") pod \"swift-proxy-55f4df7cd-4gjcp\" (UID: \"06b2a08b-6b91-44f2-b630-169938fe5771\") " pod="openstack-kuttl-tests/swift-proxy-55f4df7cd-4gjcp" Dec 03 12:35:25 crc kubenswrapper[5052]: I1203 12:35:25.523989 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/06b2a08b-6b91-44f2-b630-169938fe5771-config-data\") pod \"swift-proxy-55f4df7cd-4gjcp\" (UID: \"06b2a08b-6b91-44f2-b630-169938fe5771\") " pod="openstack-kuttl-tests/swift-proxy-55f4df7cd-4gjcp" Dec 03 12:35:25 crc kubenswrapper[5052]: I1203 12:35:25.524001 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/06b2a08b-6b91-44f2-b630-169938fe5771-combined-ca-bundle\") pod \"swift-proxy-55f4df7cd-4gjcp\" (UID: \"06b2a08b-6b91-44f2-b630-169938fe5771\") " pod="openstack-kuttl-tests/swift-proxy-55f4df7cd-4gjcp" Dec 03 12:35:25 crc kubenswrapper[5052]: I1203 12:35:25.523947 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/06b2a08b-6b91-44f2-b630-169938fe5771-internal-tls-certs\") pod \"swift-proxy-55f4df7cd-4gjcp\" (UID: \"06b2a08b-6b91-44f2-b630-169938fe5771\") " pod="openstack-kuttl-tests/swift-proxy-55f4df7cd-4gjcp" Dec 03 12:35:25 crc kubenswrapper[5052]: I1203 12:35:25.529241 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-ndmt6\" (UniqueName: \"kubernetes.io/projected/06b2a08b-6b91-44f2-b630-169938fe5771-kube-api-access-ndmt6\") pod \"swift-proxy-55f4df7cd-4gjcp\" (UID: \"06b2a08b-6b91-44f2-b630-169938fe5771\") " pod="openstack-kuttl-tests/swift-proxy-55f4df7cd-4gjcp" Dec 03 12:35:25 crc kubenswrapper[5052]: I1203 12:35:25.529556 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/06b2a08b-6b91-44f2-b630-169938fe5771-etc-swift\") pod \"swift-proxy-55f4df7cd-4gjcp\" (UID: \"06b2a08b-6b91-44f2-b630-169938fe5771\") " pod="openstack-kuttl-tests/swift-proxy-55f4df7cd-4gjcp" Dec 03 12:35:25 crc kubenswrapper[5052]: I1203 12:35:25.649022 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-proxy-55f4df7cd-4gjcp" Dec 03 12:35:25 crc kubenswrapper[5052]: I1203 12:35:25.685376 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:35:25 crc kubenswrapper[5052]: I1203 12:35:25.685949 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="f45ca29a-a9fb-4644-be06-f728cc56d453" containerName="ceilometer-central-agent" containerID="cri-o://8b655d6718df3641de13c30530434a2989b46b2da6946e2ca33442a75f43cf88" gracePeriod=30 Dec 03 12:35:25 crc kubenswrapper[5052]: I1203 12:35:25.686317 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="f45ca29a-a9fb-4644-be06-f728cc56d453" containerName="proxy-httpd" containerID="cri-o://064f270e84004ca2ace7f9545a79dfb6f74dcab89088dbb2aa2b43e7663886fa" gracePeriod=30 Dec 03 12:35:25 crc kubenswrapper[5052]: I1203 12:35:25.686368 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="f45ca29a-a9fb-4644-be06-f728cc56d453" containerName="sg-core" containerID="cri-o://5c1d2761e93325105dd9e13409ca25432fbfd12fd75000d1efd160e5758aa9e0" gracePeriod=30 Dec 03 12:35:25 crc kubenswrapper[5052]: I1203 12:35:25.686447 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="f45ca29a-a9fb-4644-be06-f728cc56d453" containerName="ceilometer-notification-agent" containerID="cri-o://dfad5acef7a373a71ef748493cbad2e7c868171969fa948b880b61be5af6d286" gracePeriod=30 Dec 03 12:35:25 crc kubenswrapper[5052]: I1203 12:35:25.909568 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/openstackclient"] Dec 03 12:35:25 crc kubenswrapper[5052]: I1203 12:35:25.911026 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstackclient" Dec 03 12:35:25 crc kubenswrapper[5052]: I1203 12:35:25.913817 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"openstack-config" Dec 03 12:35:25 crc kubenswrapper[5052]: I1203 12:35:25.913957 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"openstack-config-secret" Dec 03 12:35:25 crc kubenswrapper[5052]: I1203 12:35:25.923674 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"openstackclient-openstackclient-dockercfg-hsm88" Dec 03 12:35:25 crc kubenswrapper[5052]: I1203 12:35:25.948079 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openstack-config-secret\" (UniqueName: \"kubernetes.io/secret/24444a47-099d-4057-a118-669a0b8a7367-openstack-config-secret\") pod \"openstackclient\" (UID: \"24444a47-099d-4057-a118-669a0b8a7367\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 12:35:25 crc kubenswrapper[5052]: I1203 12:35:25.948485 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sxvvg\" (UniqueName: \"kubernetes.io/projected/24444a47-099d-4057-a118-669a0b8a7367-kube-api-access-sxvvg\") pod \"openstackclient\" (UID: \"24444a47-099d-4057-a118-669a0b8a7367\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 12:35:25 crc kubenswrapper[5052]: I1203 12:35:25.948791 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/24444a47-099d-4057-a118-669a0b8a7367-combined-ca-bundle\") pod \"openstackclient\" (UID: \"24444a47-099d-4057-a118-669a0b8a7367\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 12:35:25 crc kubenswrapper[5052]: I1203 12:35:25.948934 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openstack-config\" (UniqueName: \"kubernetes.io/configmap/24444a47-099d-4057-a118-669a0b8a7367-openstack-config\") pod \"openstackclient\" (UID: \"24444a47-099d-4057-a118-669a0b8a7367\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 12:35:25 crc kubenswrapper[5052]: I1203 12:35:25.958927 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/openstackclient"] Dec 03 12:35:26 crc kubenswrapper[5052]: I1203 12:35:26.051123 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/24444a47-099d-4057-a118-669a0b8a7367-combined-ca-bundle\") pod \"openstackclient\" (UID: \"24444a47-099d-4057-a118-669a0b8a7367\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 12:35:26 crc kubenswrapper[5052]: I1203 12:35:26.051547 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"openstack-config\" (UniqueName: \"kubernetes.io/configmap/24444a47-099d-4057-a118-669a0b8a7367-openstack-config\") pod \"openstackclient\" (UID: \"24444a47-099d-4057-a118-669a0b8a7367\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 12:35:26 crc kubenswrapper[5052]: I1203 12:35:26.051638 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"openstack-config-secret\" (UniqueName: \"kubernetes.io/secret/24444a47-099d-4057-a118-669a0b8a7367-openstack-config-secret\") pod \"openstackclient\" (UID: \"24444a47-099d-4057-a118-669a0b8a7367\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 12:35:26 crc kubenswrapper[5052]: I1203 12:35:26.051672 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-sxvvg\" (UniqueName: \"kubernetes.io/projected/24444a47-099d-4057-a118-669a0b8a7367-kube-api-access-sxvvg\") pod \"openstackclient\" (UID: \"24444a47-099d-4057-a118-669a0b8a7367\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 12:35:26 crc kubenswrapper[5052]: I1203 12:35:26.052692 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"openstack-config\" (UniqueName: \"kubernetes.io/configmap/24444a47-099d-4057-a118-669a0b8a7367-openstack-config\") pod \"openstackclient\" (UID: \"24444a47-099d-4057-a118-669a0b8a7367\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 12:35:26 crc kubenswrapper[5052]: I1203 12:35:26.057190 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"openstack-config-secret\" (UniqueName: \"kubernetes.io/secret/24444a47-099d-4057-a118-669a0b8a7367-openstack-config-secret\") pod \"openstackclient\" (UID: \"24444a47-099d-4057-a118-669a0b8a7367\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 12:35:26 crc kubenswrapper[5052]: I1203 12:35:26.059533 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/24444a47-099d-4057-a118-669a0b8a7367-combined-ca-bundle\") pod \"openstackclient\" (UID: \"24444a47-099d-4057-a118-669a0b8a7367\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 12:35:26 crc kubenswrapper[5052]: I1203 12:35:26.078117 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-sxvvg\" (UniqueName: \"kubernetes.io/projected/24444a47-099d-4057-a118-669a0b8a7367-kube-api-access-sxvvg\") pod \"openstackclient\" (UID: \"24444a47-099d-4057-a118-669a0b8a7367\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 12:35:26 crc kubenswrapper[5052]: I1203 12:35:26.166775 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/swift-proxy-55f4df7cd-4gjcp"] Dec 03 12:35:26 crc kubenswrapper[5052]: W1203 12:35:26.171260 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod06b2a08b_6b91_44f2_b630_169938fe5771.slice/crio-a3501750ee3c86ae7c02bc63bcbfc468c8155e083b2729cd978109b695842fe9 WatchSource:0}: Error finding container a3501750ee3c86ae7c02bc63bcbfc468c8155e083b2729cd978109b695842fe9: Status 404 returned error can't find the container with id a3501750ee3c86ae7c02bc63bcbfc468c8155e083b2729cd978109b695842fe9 Dec 03 12:35:26 crc kubenswrapper[5052]: I1203 12:35:26.247643 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstackclient" Dec 03 12:35:26 crc kubenswrapper[5052]: I1203 12:35:26.308177 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-proxy-55f4df7cd-4gjcp" event={"ID":"06b2a08b-6b91-44f2-b630-169938fe5771","Type":"ContainerStarted","Data":"a3501750ee3c86ae7c02bc63bcbfc468c8155e083b2729cd978109b695842fe9"} Dec 03 12:35:26 crc kubenswrapper[5052]: I1203 12:35:26.311095 5052 generic.go:334] "Generic (PLEG): container finished" podID="f45ca29a-a9fb-4644-be06-f728cc56d453" containerID="064f270e84004ca2ace7f9545a79dfb6f74dcab89088dbb2aa2b43e7663886fa" exitCode=0 Dec 03 12:35:26 crc kubenswrapper[5052]: I1203 12:35:26.311123 5052 generic.go:334] "Generic (PLEG): container finished" podID="f45ca29a-a9fb-4644-be06-f728cc56d453" containerID="5c1d2761e93325105dd9e13409ca25432fbfd12fd75000d1efd160e5758aa9e0" exitCode=2 Dec 03 12:35:26 crc kubenswrapper[5052]: I1203 12:35:26.311170 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"f45ca29a-a9fb-4644-be06-f728cc56d453","Type":"ContainerDied","Data":"064f270e84004ca2ace7f9545a79dfb6f74dcab89088dbb2aa2b43e7663886fa"} Dec 03 12:35:26 crc kubenswrapper[5052]: I1203 12:35:26.311211 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"f45ca29a-a9fb-4644-be06-f728cc56d453","Type":"ContainerDied","Data":"5c1d2761e93325105dd9e13409ca25432fbfd12fd75000d1efd160e5758aa9e0"} Dec 03 12:35:26 crc kubenswrapper[5052]: I1203 12:35:26.311225 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"f45ca29a-a9fb-4644-be06-f728cc56d453","Type":"ContainerDied","Data":"8b655d6718df3641de13c30530434a2989b46b2da6946e2ca33442a75f43cf88"} Dec 03 12:35:26 crc kubenswrapper[5052]: I1203 12:35:26.311179 5052 generic.go:334] "Generic (PLEG): container finished" podID="f45ca29a-a9fb-4644-be06-f728cc56d453" containerID="8b655d6718df3641de13c30530434a2989b46b2da6946e2ca33442a75f43cf88" exitCode=0 Dec 03 12:35:26 crc kubenswrapper[5052]: W1203 12:35:26.732673 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod24444a47_099d_4057_a118_669a0b8a7367.slice/crio-5ad770201751b3d290beab02cfed3f0a09455c512e46f25c71ef148fa308869f WatchSource:0}: Error finding container 5ad770201751b3d290beab02cfed3f0a09455c512e46f25c71ef148fa308869f: Status 404 returned error can't find the container with id 5ad770201751b3d290beab02cfed3f0a09455c512e46f25c71ef148fa308869f Dec 03 12:35:26 crc kubenswrapper[5052]: I1203 12:35:26.748967 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/openstackclient"] Dec 03 12:35:26 crc kubenswrapper[5052]: I1203 12:35:26.805461 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-api-db-create-k2lkr"] Dec 03 12:35:26 crc kubenswrapper[5052]: I1203 12:35:26.807346 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-db-create-k2lkr" Dec 03 12:35:26 crc kubenswrapper[5052]: I1203 12:35:26.845577 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-db-create-k2lkr"] Dec 03 12:35:26 crc kubenswrapper[5052]: I1203 12:35:26.868344 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/ffa2902f-c174-44ec-98dc-de610f30b64f-operator-scripts\") pod \"nova-api-db-create-k2lkr\" (UID: \"ffa2902f-c174-44ec-98dc-de610f30b64f\") " pod="openstack-kuttl-tests/nova-api-db-create-k2lkr" Dec 03 12:35:26 crc kubenswrapper[5052]: I1203 12:35:26.868578 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nb849\" (UniqueName: \"kubernetes.io/projected/ffa2902f-c174-44ec-98dc-de610f30b64f-kube-api-access-nb849\") pod \"nova-api-db-create-k2lkr\" (UID: \"ffa2902f-c174-44ec-98dc-de610f30b64f\") " pod="openstack-kuttl-tests/nova-api-db-create-k2lkr" Dec 03 12:35:26 crc kubenswrapper[5052]: I1203 12:35:26.905336 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell0-db-create-4cj5q"] Dec 03 12:35:26 crc kubenswrapper[5052]: I1203 12:35:26.906684 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-db-create-4cj5q" Dec 03 12:35:26 crc kubenswrapper[5052]: I1203 12:35:26.917190 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-api-b163-account-create-update-d97hs"] Dec 03 12:35:26 crc kubenswrapper[5052]: I1203 12:35:26.918857 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-b163-account-create-update-d97hs" Dec 03 12:35:26 crc kubenswrapper[5052]: I1203 12:35:26.922868 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-api-db-secret" Dec 03 12:35:26 crc kubenswrapper[5052]: I1203 12:35:26.951577 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-b163-account-create-update-d97hs"] Dec 03 12:35:26 crc kubenswrapper[5052]: I1203 12:35:26.961065 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell0-db-create-4cj5q"] Dec 03 12:35:26 crc kubenswrapper[5052]: I1203 12:35:26.971245 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/44c9835e-4fd7-4713-8e42-2d3f0a82fbfe-operator-scripts\") pod \"nova-cell0-db-create-4cj5q\" (UID: \"44c9835e-4fd7-4713-8e42-2d3f0a82fbfe\") " pod="openstack-kuttl-tests/nova-cell0-db-create-4cj5q" Dec 03 12:35:26 crc kubenswrapper[5052]: I1203 12:35:26.971340 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/ffa2902f-c174-44ec-98dc-de610f30b64f-operator-scripts\") pod \"nova-api-db-create-k2lkr\" (UID: \"ffa2902f-c174-44ec-98dc-de610f30b64f\") " pod="openstack-kuttl-tests/nova-api-db-create-k2lkr" Dec 03 12:35:26 crc kubenswrapper[5052]: I1203 12:35:26.971382 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5vbpp\" (UniqueName: \"kubernetes.io/projected/44c9835e-4fd7-4713-8e42-2d3f0a82fbfe-kube-api-access-5vbpp\") pod \"nova-cell0-db-create-4cj5q\" (UID: \"44c9835e-4fd7-4713-8e42-2d3f0a82fbfe\") " pod="openstack-kuttl-tests/nova-cell0-db-create-4cj5q" Dec 03 12:35:26 crc kubenswrapper[5052]: I1203 12:35:26.971533 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/2a2e1ec2-a657-4a31-93fd-712d45fab72b-operator-scripts\") pod \"nova-api-b163-account-create-update-d97hs\" (UID: \"2a2e1ec2-a657-4a31-93fd-712d45fab72b\") " pod="openstack-kuttl-tests/nova-api-b163-account-create-update-d97hs" Dec 03 12:35:26 crc kubenswrapper[5052]: I1203 12:35:26.971568 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4g27p\" (UniqueName: \"kubernetes.io/projected/2a2e1ec2-a657-4a31-93fd-712d45fab72b-kube-api-access-4g27p\") pod \"nova-api-b163-account-create-update-d97hs\" (UID: \"2a2e1ec2-a657-4a31-93fd-712d45fab72b\") " pod="openstack-kuttl-tests/nova-api-b163-account-create-update-d97hs" Dec 03 12:35:26 crc kubenswrapper[5052]: I1203 12:35:26.971714 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-nb849\" (UniqueName: \"kubernetes.io/projected/ffa2902f-c174-44ec-98dc-de610f30b64f-kube-api-access-nb849\") pod \"nova-api-db-create-k2lkr\" (UID: \"ffa2902f-c174-44ec-98dc-de610f30b64f\") " pod="openstack-kuttl-tests/nova-api-db-create-k2lkr" Dec 03 12:35:26 crc kubenswrapper[5052]: I1203 12:35:26.974654 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/ffa2902f-c174-44ec-98dc-de610f30b64f-operator-scripts\") pod \"nova-api-db-create-k2lkr\" (UID: \"ffa2902f-c174-44ec-98dc-de610f30b64f\") " pod="openstack-kuttl-tests/nova-api-db-create-k2lkr" Dec 03 12:35:26 crc kubenswrapper[5052]: I1203 12:35:26.996669 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-nb849\" (UniqueName: \"kubernetes.io/projected/ffa2902f-c174-44ec-98dc-de610f30b64f-kube-api-access-nb849\") pod \"nova-api-db-create-k2lkr\" (UID: \"ffa2902f-c174-44ec-98dc-de610f30b64f\") " pod="openstack-kuttl-tests/nova-api-db-create-k2lkr" Dec 03 12:35:27 crc kubenswrapper[5052]: I1203 12:35:27.002925 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell1-db-create-22ctd"] Dec 03 12:35:27 crc kubenswrapper[5052]: I1203 12:35:27.004204 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-db-create-22ctd" Dec 03 12:35:27 crc kubenswrapper[5052]: I1203 12:35:27.014933 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-db-create-22ctd"] Dec 03 12:35:27 crc kubenswrapper[5052]: I1203 12:35:27.078148 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/44c9835e-4fd7-4713-8e42-2d3f0a82fbfe-operator-scripts\") pod \"nova-cell0-db-create-4cj5q\" (UID: \"44c9835e-4fd7-4713-8e42-2d3f0a82fbfe\") " pod="openstack-kuttl-tests/nova-cell0-db-create-4cj5q" Dec 03 12:35:27 crc kubenswrapper[5052]: I1203 12:35:27.078234 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-5vbpp\" (UniqueName: \"kubernetes.io/projected/44c9835e-4fd7-4713-8e42-2d3f0a82fbfe-kube-api-access-5vbpp\") pod \"nova-cell0-db-create-4cj5q\" (UID: \"44c9835e-4fd7-4713-8e42-2d3f0a82fbfe\") " pod="openstack-kuttl-tests/nova-cell0-db-create-4cj5q" Dec 03 12:35:27 crc kubenswrapper[5052]: I1203 12:35:27.078310 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-77k77\" (UniqueName: \"kubernetes.io/projected/51f3388b-292f-40a9-9d41-ad60bb51ab68-kube-api-access-77k77\") pod \"nova-cell1-db-create-22ctd\" (UID: \"51f3388b-292f-40a9-9d41-ad60bb51ab68\") " pod="openstack-kuttl-tests/nova-cell1-db-create-22ctd" Dec 03 12:35:27 crc kubenswrapper[5052]: I1203 12:35:27.078348 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/2a2e1ec2-a657-4a31-93fd-712d45fab72b-operator-scripts\") pod \"nova-api-b163-account-create-update-d97hs\" (UID: \"2a2e1ec2-a657-4a31-93fd-712d45fab72b\") " pod="openstack-kuttl-tests/nova-api-b163-account-create-update-d97hs" Dec 03 12:35:27 crc kubenswrapper[5052]: I1203 12:35:27.078375 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-4g27p\" (UniqueName: \"kubernetes.io/projected/2a2e1ec2-a657-4a31-93fd-712d45fab72b-kube-api-access-4g27p\") pod \"nova-api-b163-account-create-update-d97hs\" (UID: \"2a2e1ec2-a657-4a31-93fd-712d45fab72b\") " pod="openstack-kuttl-tests/nova-api-b163-account-create-update-d97hs" Dec 03 12:35:27 crc kubenswrapper[5052]: I1203 12:35:27.078412 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/51f3388b-292f-40a9-9d41-ad60bb51ab68-operator-scripts\") pod \"nova-cell1-db-create-22ctd\" (UID: \"51f3388b-292f-40a9-9d41-ad60bb51ab68\") " pod="openstack-kuttl-tests/nova-cell1-db-create-22ctd" Dec 03 12:35:27 crc kubenswrapper[5052]: I1203 12:35:27.079579 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/44c9835e-4fd7-4713-8e42-2d3f0a82fbfe-operator-scripts\") pod \"nova-cell0-db-create-4cj5q\" (UID: \"44c9835e-4fd7-4713-8e42-2d3f0a82fbfe\") " pod="openstack-kuttl-tests/nova-cell0-db-create-4cj5q" Dec 03 12:35:27 crc kubenswrapper[5052]: I1203 12:35:27.080018 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/2a2e1ec2-a657-4a31-93fd-712d45fab72b-operator-scripts\") pod \"nova-api-b163-account-create-update-d97hs\" (UID: \"2a2e1ec2-a657-4a31-93fd-712d45fab72b\") " pod="openstack-kuttl-tests/nova-api-b163-account-create-update-d97hs" Dec 03 12:35:27 crc kubenswrapper[5052]: I1203 12:35:27.104006 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-5vbpp\" (UniqueName: \"kubernetes.io/projected/44c9835e-4fd7-4713-8e42-2d3f0a82fbfe-kube-api-access-5vbpp\") pod \"nova-cell0-db-create-4cj5q\" (UID: \"44c9835e-4fd7-4713-8e42-2d3f0a82fbfe\") " pod="openstack-kuttl-tests/nova-cell0-db-create-4cj5q" Dec 03 12:35:27 crc kubenswrapper[5052]: I1203 12:35:27.106159 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-4g27p\" (UniqueName: \"kubernetes.io/projected/2a2e1ec2-a657-4a31-93fd-712d45fab72b-kube-api-access-4g27p\") pod \"nova-api-b163-account-create-update-d97hs\" (UID: \"2a2e1ec2-a657-4a31-93fd-712d45fab72b\") " pod="openstack-kuttl-tests/nova-api-b163-account-create-update-d97hs" Dec 03 12:35:27 crc kubenswrapper[5052]: I1203 12:35:27.136803 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell0-7298-account-create-update-5sx49"] Dec 03 12:35:27 crc kubenswrapper[5052]: I1203 12:35:27.138281 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-7298-account-create-update-5sx49" Dec 03 12:35:27 crc kubenswrapper[5052]: I1203 12:35:27.141214 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell0-db-secret" Dec 03 12:35:27 crc kubenswrapper[5052]: I1203 12:35:27.165677 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell0-7298-account-create-update-5sx49"] Dec 03 12:35:27 crc kubenswrapper[5052]: I1203 12:35:27.181217 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-77k77\" (UniqueName: \"kubernetes.io/projected/51f3388b-292f-40a9-9d41-ad60bb51ab68-kube-api-access-77k77\") pod \"nova-cell1-db-create-22ctd\" (UID: \"51f3388b-292f-40a9-9d41-ad60bb51ab68\") " pod="openstack-kuttl-tests/nova-cell1-db-create-22ctd" Dec 03 12:35:27 crc kubenswrapper[5052]: I1203 12:35:27.181316 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/51f3388b-292f-40a9-9d41-ad60bb51ab68-operator-scripts\") pod \"nova-cell1-db-create-22ctd\" (UID: \"51f3388b-292f-40a9-9d41-ad60bb51ab68\") " pod="openstack-kuttl-tests/nova-cell1-db-create-22ctd" Dec 03 12:35:27 crc kubenswrapper[5052]: I1203 12:35:27.181449 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8qkvj\" (UniqueName: \"kubernetes.io/projected/6daea253-ee08-4aa8-a900-39552cae7bd5-kube-api-access-8qkvj\") pod \"nova-cell0-7298-account-create-update-5sx49\" (UID: \"6daea253-ee08-4aa8-a900-39552cae7bd5\") " pod="openstack-kuttl-tests/nova-cell0-7298-account-create-update-5sx49" Dec 03 12:35:27 crc kubenswrapper[5052]: I1203 12:35:27.181507 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/6daea253-ee08-4aa8-a900-39552cae7bd5-operator-scripts\") pod \"nova-cell0-7298-account-create-update-5sx49\" (UID: \"6daea253-ee08-4aa8-a900-39552cae7bd5\") " pod="openstack-kuttl-tests/nova-cell0-7298-account-create-update-5sx49" Dec 03 12:35:27 crc kubenswrapper[5052]: I1203 12:35:27.182418 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/51f3388b-292f-40a9-9d41-ad60bb51ab68-operator-scripts\") pod \"nova-cell1-db-create-22ctd\" (UID: \"51f3388b-292f-40a9-9d41-ad60bb51ab68\") " pod="openstack-kuttl-tests/nova-cell1-db-create-22ctd" Dec 03 12:35:27 crc kubenswrapper[5052]: I1203 12:35:27.199237 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-77k77\" (UniqueName: \"kubernetes.io/projected/51f3388b-292f-40a9-9d41-ad60bb51ab68-kube-api-access-77k77\") pod \"nova-cell1-db-create-22ctd\" (UID: \"51f3388b-292f-40a9-9d41-ad60bb51ab68\") " pod="openstack-kuttl-tests/nova-cell1-db-create-22ctd" Dec 03 12:35:27 crc kubenswrapper[5052]: I1203 12:35:27.252261 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-db-create-k2lkr" Dec 03 12:35:27 crc kubenswrapper[5052]: I1203 12:35:27.283470 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-8qkvj\" (UniqueName: \"kubernetes.io/projected/6daea253-ee08-4aa8-a900-39552cae7bd5-kube-api-access-8qkvj\") pod \"nova-cell0-7298-account-create-update-5sx49\" (UID: \"6daea253-ee08-4aa8-a900-39552cae7bd5\") " pod="openstack-kuttl-tests/nova-cell0-7298-account-create-update-5sx49" Dec 03 12:35:27 crc kubenswrapper[5052]: I1203 12:35:27.284195 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/6daea253-ee08-4aa8-a900-39552cae7bd5-operator-scripts\") pod \"nova-cell0-7298-account-create-update-5sx49\" (UID: \"6daea253-ee08-4aa8-a900-39552cae7bd5\") " pod="openstack-kuttl-tests/nova-cell0-7298-account-create-update-5sx49" Dec 03 12:35:27 crc kubenswrapper[5052]: I1203 12:35:27.283609 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-db-create-4cj5q" Dec 03 12:35:27 crc kubenswrapper[5052]: I1203 12:35:27.289985 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/6daea253-ee08-4aa8-a900-39552cae7bd5-operator-scripts\") pod \"nova-cell0-7298-account-create-update-5sx49\" (UID: \"6daea253-ee08-4aa8-a900-39552cae7bd5\") " pod="openstack-kuttl-tests/nova-cell0-7298-account-create-update-5sx49" Dec 03 12:35:27 crc kubenswrapper[5052]: I1203 12:35:27.293100 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell1-692e-account-create-update-44rrb"] Dec 03 12:35:27 crc kubenswrapper[5052]: I1203 12:35:27.294672 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-692e-account-create-update-44rrb" Dec 03 12:35:27 crc kubenswrapper[5052]: I1203 12:35:27.300623 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell1-db-secret" Dec 03 12:35:27 crc kubenswrapper[5052]: I1203 12:35:27.305268 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-b163-account-create-update-d97hs" Dec 03 12:35:27 crc kubenswrapper[5052]: I1203 12:35:27.317911 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-8qkvj\" (UniqueName: \"kubernetes.io/projected/6daea253-ee08-4aa8-a900-39552cae7bd5-kube-api-access-8qkvj\") pod \"nova-cell0-7298-account-create-update-5sx49\" (UID: \"6daea253-ee08-4aa8-a900-39552cae7bd5\") " pod="openstack-kuttl-tests/nova-cell0-7298-account-create-update-5sx49" Dec 03 12:35:27 crc kubenswrapper[5052]: I1203 12:35:27.332495 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-692e-account-create-update-44rrb"] Dec 03 12:35:27 crc kubenswrapper[5052]: I1203 12:35:27.335832 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-db-create-22ctd" Dec 03 12:35:27 crc kubenswrapper[5052]: I1203 12:35:27.340240 5052 generic.go:334] "Generic (PLEG): container finished" podID="f45ca29a-a9fb-4644-be06-f728cc56d453" containerID="dfad5acef7a373a71ef748493cbad2e7c868171969fa948b880b61be5af6d286" exitCode=0 Dec 03 12:35:27 crc kubenswrapper[5052]: I1203 12:35:27.340369 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"f45ca29a-a9fb-4644-be06-f728cc56d453","Type":"ContainerDied","Data":"dfad5acef7a373a71ef748493cbad2e7c868171969fa948b880b61be5af6d286"} Dec 03 12:35:27 crc kubenswrapper[5052]: I1203 12:35:27.351791 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-proxy-55f4df7cd-4gjcp" event={"ID":"06b2a08b-6b91-44f2-b630-169938fe5771","Type":"ContainerStarted","Data":"d08bfd89e72c8e8b387d02b98de513fbe980b80423758f0f559173df742846d9"} Dec 03 12:35:27 crc kubenswrapper[5052]: I1203 12:35:27.351854 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-proxy-55f4df7cd-4gjcp" event={"ID":"06b2a08b-6b91-44f2-b630-169938fe5771","Type":"ContainerStarted","Data":"cf437da4e9616b7fcbb31740024db41704bd4de4a657e7d26a4bdbbb2a642eb7"} Dec 03 12:35:27 crc kubenswrapper[5052]: I1203 12:35:27.352051 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/swift-proxy-55f4df7cd-4gjcp" Dec 03 12:35:27 crc kubenswrapper[5052]: I1203 12:35:27.361960 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstackclient" event={"ID":"24444a47-099d-4057-a118-669a0b8a7367","Type":"ContainerStarted","Data":"dbee67def34d5df95d62c1a89f2db75ed8290851a65f7a468f58b7a707d3ad20"} Dec 03 12:35:27 crc kubenswrapper[5052]: I1203 12:35:27.362005 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstackclient" event={"ID":"24444a47-099d-4057-a118-669a0b8a7367","Type":"ContainerStarted","Data":"5ad770201751b3d290beab02cfed3f0a09455c512e46f25c71ef148fa308869f"} Dec 03 12:35:27 crc kubenswrapper[5052]: I1203 12:35:27.387696 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2dp8m\" (UniqueName: \"kubernetes.io/projected/c2dfe7a5-4ce2-4d6e-98e5-3ea6a4d3f3ca-kube-api-access-2dp8m\") pod \"nova-cell1-692e-account-create-update-44rrb\" (UID: \"c2dfe7a5-4ce2-4d6e-98e5-3ea6a4d3f3ca\") " pod="openstack-kuttl-tests/nova-cell1-692e-account-create-update-44rrb" Dec 03 12:35:27 crc kubenswrapper[5052]: I1203 12:35:27.387932 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/c2dfe7a5-4ce2-4d6e-98e5-3ea6a4d3f3ca-operator-scripts\") pod \"nova-cell1-692e-account-create-update-44rrb\" (UID: \"c2dfe7a5-4ce2-4d6e-98e5-3ea6a4d3f3ca\") " pod="openstack-kuttl-tests/nova-cell1-692e-account-create-update-44rrb" Dec 03 12:35:27 crc kubenswrapper[5052]: I1203 12:35:27.389990 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/swift-proxy-55f4df7cd-4gjcp" podStartSLOduration=3.389972559 podStartE2EDuration="3.389972559s" podCreationTimestamp="2025-12-03 12:35:24 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:35:27.373881759 +0000 UTC m=+4517.962123523" watchObservedRunningTime="2025-12-03 12:35:27.389972559 +0000 UTC m=+4517.978214333" Dec 03 12:35:27 crc kubenswrapper[5052]: I1203 12:35:27.420648 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/openstackclient" podStartSLOduration=2.420625595 podStartE2EDuration="2.420625595s" podCreationTimestamp="2025-12-03 12:35:25 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:35:27.390985718 +0000 UTC m=+4517.979227482" watchObservedRunningTime="2025-12-03 12:35:27.420625595 +0000 UTC m=+4518.008867379" Dec 03 12:35:27 crc kubenswrapper[5052]: I1203 12:35:27.463396 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-7298-account-create-update-5sx49" Dec 03 12:35:27 crc kubenswrapper[5052]: I1203 12:35:27.490408 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-2dp8m\" (UniqueName: \"kubernetes.io/projected/c2dfe7a5-4ce2-4d6e-98e5-3ea6a4d3f3ca-kube-api-access-2dp8m\") pod \"nova-cell1-692e-account-create-update-44rrb\" (UID: \"c2dfe7a5-4ce2-4d6e-98e5-3ea6a4d3f3ca\") " pod="openstack-kuttl-tests/nova-cell1-692e-account-create-update-44rrb" Dec 03 12:35:27 crc kubenswrapper[5052]: I1203 12:35:27.490497 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/c2dfe7a5-4ce2-4d6e-98e5-3ea6a4d3f3ca-operator-scripts\") pod \"nova-cell1-692e-account-create-update-44rrb\" (UID: \"c2dfe7a5-4ce2-4d6e-98e5-3ea6a4d3f3ca\") " pod="openstack-kuttl-tests/nova-cell1-692e-account-create-update-44rrb" Dec 03 12:35:27 crc kubenswrapper[5052]: I1203 12:35:27.491364 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/c2dfe7a5-4ce2-4d6e-98e5-3ea6a4d3f3ca-operator-scripts\") pod \"nova-cell1-692e-account-create-update-44rrb\" (UID: \"c2dfe7a5-4ce2-4d6e-98e5-3ea6a4d3f3ca\") " pod="openstack-kuttl-tests/nova-cell1-692e-account-create-update-44rrb" Dec 03 12:35:27 crc kubenswrapper[5052]: I1203 12:35:27.514138 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-2dp8m\" (UniqueName: \"kubernetes.io/projected/c2dfe7a5-4ce2-4d6e-98e5-3ea6a4d3f3ca-kube-api-access-2dp8m\") pod \"nova-cell1-692e-account-create-update-44rrb\" (UID: \"c2dfe7a5-4ce2-4d6e-98e5-3ea6a4d3f3ca\") " pod="openstack-kuttl-tests/nova-cell1-692e-account-create-update-44rrb" Dec 03 12:35:27 crc kubenswrapper[5052]: I1203 12:35:27.632304 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-692e-account-create-update-44rrb" Dec 03 12:35:27 crc kubenswrapper[5052]: I1203 12:35:27.808145 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-db-create-k2lkr"] Dec 03 12:35:28 crc kubenswrapper[5052]: I1203 12:35:28.027142 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:35:28 crc kubenswrapper[5052]: I1203 12:35:28.104150 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell0-db-create-4cj5q"] Dec 03 12:35:28 crc kubenswrapper[5052]: I1203 12:35:28.110125 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w52f7\" (UniqueName: \"kubernetes.io/projected/f45ca29a-a9fb-4644-be06-f728cc56d453-kube-api-access-w52f7\") pod \"f45ca29a-a9fb-4644-be06-f728cc56d453\" (UID: \"f45ca29a-a9fb-4644-be06-f728cc56d453\") " Dec 03 12:35:28 crc kubenswrapper[5052]: I1203 12:35:28.110178 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f45ca29a-a9fb-4644-be06-f728cc56d453-combined-ca-bundle\") pod \"f45ca29a-a9fb-4644-be06-f728cc56d453\" (UID: \"f45ca29a-a9fb-4644-be06-f728cc56d453\") " Dec 03 12:35:28 crc kubenswrapper[5052]: I1203 12:35:28.110333 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/f45ca29a-a9fb-4644-be06-f728cc56d453-log-httpd\") pod \"f45ca29a-a9fb-4644-be06-f728cc56d453\" (UID: \"f45ca29a-a9fb-4644-be06-f728cc56d453\") " Dec 03 12:35:28 crc kubenswrapper[5052]: I1203 12:35:28.110397 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/f45ca29a-a9fb-4644-be06-f728cc56d453-run-httpd\") pod \"f45ca29a-a9fb-4644-be06-f728cc56d453\" (UID: \"f45ca29a-a9fb-4644-be06-f728cc56d453\") " Dec 03 12:35:28 crc kubenswrapper[5052]: I1203 12:35:28.110424 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/f45ca29a-a9fb-4644-be06-f728cc56d453-sg-core-conf-yaml\") pod \"f45ca29a-a9fb-4644-be06-f728cc56d453\" (UID: \"f45ca29a-a9fb-4644-be06-f728cc56d453\") " Dec 03 12:35:28 crc kubenswrapper[5052]: I1203 12:35:28.110447 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/f45ca29a-a9fb-4644-be06-f728cc56d453-config-data\") pod \"f45ca29a-a9fb-4644-be06-f728cc56d453\" (UID: \"f45ca29a-a9fb-4644-be06-f728cc56d453\") " Dec 03 12:35:28 crc kubenswrapper[5052]: I1203 12:35:28.110479 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/f45ca29a-a9fb-4644-be06-f728cc56d453-scripts\") pod \"f45ca29a-a9fb-4644-be06-f728cc56d453\" (UID: \"f45ca29a-a9fb-4644-be06-f728cc56d453\") " Dec 03 12:35:28 crc kubenswrapper[5052]: I1203 12:35:28.111053 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f45ca29a-a9fb-4644-be06-f728cc56d453-log-httpd" (OuterVolumeSpecName: "log-httpd") pod "f45ca29a-a9fb-4644-be06-f728cc56d453" (UID: "f45ca29a-a9fb-4644-be06-f728cc56d453"). InnerVolumeSpecName "log-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:35:28 crc kubenswrapper[5052]: I1203 12:35:28.115565 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f45ca29a-a9fb-4644-be06-f728cc56d453-run-httpd" (OuterVolumeSpecName: "run-httpd") pod "f45ca29a-a9fb-4644-be06-f728cc56d453" (UID: "f45ca29a-a9fb-4644-be06-f728cc56d453"). InnerVolumeSpecName "run-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:35:28 crc kubenswrapper[5052]: I1203 12:35:28.173762 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f45ca29a-a9fb-4644-be06-f728cc56d453-scripts" (OuterVolumeSpecName: "scripts") pod "f45ca29a-a9fb-4644-be06-f728cc56d453" (UID: "f45ca29a-a9fb-4644-be06-f728cc56d453"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:35:28 crc kubenswrapper[5052]: I1203 12:35:28.174225 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f45ca29a-a9fb-4644-be06-f728cc56d453-kube-api-access-w52f7" (OuterVolumeSpecName: "kube-api-access-w52f7") pod "f45ca29a-a9fb-4644-be06-f728cc56d453" (UID: "f45ca29a-a9fb-4644-be06-f728cc56d453"). InnerVolumeSpecName "kube-api-access-w52f7". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:35:28 crc kubenswrapper[5052]: I1203 12:35:28.213038 5052 reconciler_common.go:293] "Volume detached for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/f45ca29a-a9fb-4644-be06-f728cc56d453-log-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:35:28 crc kubenswrapper[5052]: I1203 12:35:28.213085 5052 reconciler_common.go:293] "Volume detached for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/f45ca29a-a9fb-4644-be06-f728cc56d453-run-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:35:28 crc kubenswrapper[5052]: I1203 12:35:28.213101 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/f45ca29a-a9fb-4644-be06-f728cc56d453-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:35:28 crc kubenswrapper[5052]: I1203 12:35:28.213115 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-w52f7\" (UniqueName: \"kubernetes.io/projected/f45ca29a-a9fb-4644-be06-f728cc56d453-kube-api-access-w52f7\") on node \"crc\" DevicePath \"\"" Dec 03 12:35:28 crc kubenswrapper[5052]: I1203 12:35:28.263962 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f45ca29a-a9fb-4644-be06-f728cc56d453-sg-core-conf-yaml" (OuterVolumeSpecName: "sg-core-conf-yaml") pod "f45ca29a-a9fb-4644-be06-f728cc56d453" (UID: "f45ca29a-a9fb-4644-be06-f728cc56d453"). InnerVolumeSpecName "sg-core-conf-yaml". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:35:28 crc kubenswrapper[5052]: I1203 12:35:28.272354 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-db-create-22ctd"] Dec 03 12:35:28 crc kubenswrapper[5052]: I1203 12:35:28.287402 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell0-7298-account-create-update-5sx49"] Dec 03 12:35:28 crc kubenswrapper[5052]: I1203 12:35:28.296470 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-b163-account-create-update-d97hs"] Dec 03 12:35:28 crc kubenswrapper[5052]: I1203 12:35:28.319774 5052 reconciler_common.go:293] "Volume detached for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/f45ca29a-a9fb-4644-be06-f728cc56d453-sg-core-conf-yaml\") on node \"crc\" DevicePath \"\"" Dec 03 12:35:28 crc kubenswrapper[5052]: I1203 12:35:28.441604 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"f45ca29a-a9fb-4644-be06-f728cc56d453","Type":"ContainerDied","Data":"5a5a7b78a52c9eab5e3b6d904e527d910f137fc77d871a1e48dfcc358909e2de"} Dec 03 12:35:28 crc kubenswrapper[5052]: I1203 12:35:28.441653 5052 scope.go:117] "RemoveContainer" containerID="064f270e84004ca2ace7f9545a79dfb6f74dcab89088dbb2aa2b43e7663886fa" Dec 03 12:35:28 crc kubenswrapper[5052]: I1203 12:35:28.441773 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:35:28 crc kubenswrapper[5052]: I1203 12:35:28.448527 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f45ca29a-a9fb-4644-be06-f728cc56d453-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "f45ca29a-a9fb-4644-be06-f728cc56d453" (UID: "f45ca29a-a9fb-4644-be06-f728cc56d453"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:35:28 crc kubenswrapper[5052]: I1203 12:35:28.450771 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-db-create-4cj5q" event={"ID":"44c9835e-4fd7-4713-8e42-2d3f0a82fbfe","Type":"ContainerStarted","Data":"28c2e9820def366e06686c0d1645513ab9116fd3167a1e8f1bfe06ef99d116b3"} Dec 03 12:35:28 crc kubenswrapper[5052]: I1203 12:35:28.464802 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-692e-account-create-update-44rrb"] Dec 03 12:35:28 crc kubenswrapper[5052]: I1203 12:35:28.471131 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-db-create-k2lkr" event={"ID":"ffa2902f-c174-44ec-98dc-de610f30b64f","Type":"ContainerStarted","Data":"1729d01686e7da24ac8eadd64fe8d08a11e6f6f31264cea2c7dd0a578e797dfb"} Dec 03 12:35:28 crc kubenswrapper[5052]: I1203 12:35:28.471181 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-db-create-k2lkr" event={"ID":"ffa2902f-c174-44ec-98dc-de610f30b64f","Type":"ContainerStarted","Data":"23f4286158cac12ee1c3f0fdbabba7d8eea04143035c21b79b41e1f2ac089be7"} Dec 03 12:35:28 crc kubenswrapper[5052]: I1203 12:35:28.476099 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-7298-account-create-update-5sx49" event={"ID":"6daea253-ee08-4aa8-a900-39552cae7bd5","Type":"ContainerStarted","Data":"3bdfd0a433dc5b6735ea2071c8fe5a8238869e13696292b23922925fb5d85641"} Dec 03 12:35:28 crc kubenswrapper[5052]: I1203 12:35:28.479139 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-b163-account-create-update-d97hs" event={"ID":"2a2e1ec2-a657-4a31-93fd-712d45fab72b","Type":"ContainerStarted","Data":"889cfe5d0933a701f06918f1859adb3c9f31937a7eb380bb157176f9d2069801"} Dec 03 12:35:28 crc kubenswrapper[5052]: I1203 12:35:28.484850 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-db-create-22ctd" event={"ID":"51f3388b-292f-40a9-9d41-ad60bb51ab68","Type":"ContainerStarted","Data":"1f198ffb113b77f26ebc8588f32b3fbd3bbf98d228487035be86e73e18c98e05"} Dec 03 12:35:28 crc kubenswrapper[5052]: I1203 12:35:28.485778 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/swift-proxy-55f4df7cd-4gjcp" Dec 03 12:35:28 crc kubenswrapper[5052]: I1203 12:35:28.489441 5052 scope.go:117] "RemoveContainer" containerID="5c1d2761e93325105dd9e13409ca25432fbfd12fd75000d1efd160e5758aa9e0" Dec 03 12:35:28 crc kubenswrapper[5052]: I1203 12:35:28.495344 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-api-db-create-k2lkr" podStartSLOduration=2.49532615 podStartE2EDuration="2.49532615s" podCreationTimestamp="2025-12-03 12:35:26 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:35:28.489928636 +0000 UTC m=+4519.078170390" watchObservedRunningTime="2025-12-03 12:35:28.49532615 +0000 UTC m=+4519.083567904" Dec 03 12:35:28 crc kubenswrapper[5052]: I1203 12:35:28.517293 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f45ca29a-a9fb-4644-be06-f728cc56d453-config-data" (OuterVolumeSpecName: "config-data") pod "f45ca29a-a9fb-4644-be06-f728cc56d453" (UID: "f45ca29a-a9fb-4644-be06-f728cc56d453"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:35:28 crc kubenswrapper[5052]: I1203 12:35:28.525125 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/f45ca29a-a9fb-4644-be06-f728cc56d453-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:35:28 crc kubenswrapper[5052]: I1203 12:35:28.525146 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f45ca29a-a9fb-4644-be06-f728cc56d453-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:35:28 crc kubenswrapper[5052]: I1203 12:35:28.536628 5052 scope.go:117] "RemoveContainer" containerID="dfad5acef7a373a71ef748493cbad2e7c868171969fa948b880b61be5af6d286" Dec 03 12:35:28 crc kubenswrapper[5052]: I1203 12:35:28.744945 5052 scope.go:117] "RemoveContainer" containerID="8b655d6718df3641de13c30530434a2989b46b2da6946e2ca33442a75f43cf88" Dec 03 12:35:28 crc kubenswrapper[5052]: I1203 12:35:28.779084 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:35:28 crc kubenswrapper[5052]: I1203 12:35:28.794565 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:35:28 crc kubenswrapper[5052]: I1203 12:35:28.806447 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:35:28 crc kubenswrapper[5052]: E1203 12:35:28.806934 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="f45ca29a-a9fb-4644-be06-f728cc56d453" containerName="proxy-httpd" Dec 03 12:35:28 crc kubenswrapper[5052]: I1203 12:35:28.806950 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="f45ca29a-a9fb-4644-be06-f728cc56d453" containerName="proxy-httpd" Dec 03 12:35:28 crc kubenswrapper[5052]: E1203 12:35:28.806984 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="f45ca29a-a9fb-4644-be06-f728cc56d453" containerName="ceilometer-central-agent" Dec 03 12:35:28 crc kubenswrapper[5052]: I1203 12:35:28.806991 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="f45ca29a-a9fb-4644-be06-f728cc56d453" containerName="ceilometer-central-agent" Dec 03 12:35:28 crc kubenswrapper[5052]: E1203 12:35:28.806997 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="f45ca29a-a9fb-4644-be06-f728cc56d453" containerName="sg-core" Dec 03 12:35:28 crc kubenswrapper[5052]: I1203 12:35:28.807003 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="f45ca29a-a9fb-4644-be06-f728cc56d453" containerName="sg-core" Dec 03 12:35:28 crc kubenswrapper[5052]: E1203 12:35:28.807016 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="f45ca29a-a9fb-4644-be06-f728cc56d453" containerName="ceilometer-notification-agent" Dec 03 12:35:28 crc kubenswrapper[5052]: I1203 12:35:28.807021 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="f45ca29a-a9fb-4644-be06-f728cc56d453" containerName="ceilometer-notification-agent" Dec 03 12:35:28 crc kubenswrapper[5052]: I1203 12:35:28.807200 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="f45ca29a-a9fb-4644-be06-f728cc56d453" containerName="ceilometer-central-agent" Dec 03 12:35:28 crc kubenswrapper[5052]: I1203 12:35:28.807211 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="f45ca29a-a9fb-4644-be06-f728cc56d453" containerName="sg-core" Dec 03 12:35:28 crc kubenswrapper[5052]: I1203 12:35:28.807222 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="f45ca29a-a9fb-4644-be06-f728cc56d453" containerName="ceilometer-notification-agent" Dec 03 12:35:28 crc kubenswrapper[5052]: I1203 12:35:28.807239 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="f45ca29a-a9fb-4644-be06-f728cc56d453" containerName="proxy-httpd" Dec 03 12:35:28 crc kubenswrapper[5052]: I1203 12:35:28.809045 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:35:28 crc kubenswrapper[5052]: I1203 12:35:28.812100 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-config-data" Dec 03 12:35:28 crc kubenswrapper[5052]: I1203 12:35:28.812423 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-scripts" Dec 03 12:35:28 crc kubenswrapper[5052]: I1203 12:35:28.828120 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:35:28 crc kubenswrapper[5052]: I1203 12:35:28.834536 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/97784b0f-4557-4bf2-b422-8d20fe37be9b-scripts\") pod \"ceilometer-0\" (UID: \"97784b0f-4557-4bf2-b422-8d20fe37be9b\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:35:28 crc kubenswrapper[5052]: I1203 12:35:28.834644 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jrqzr\" (UniqueName: \"kubernetes.io/projected/97784b0f-4557-4bf2-b422-8d20fe37be9b-kube-api-access-jrqzr\") pod \"ceilometer-0\" (UID: \"97784b0f-4557-4bf2-b422-8d20fe37be9b\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:35:28 crc kubenswrapper[5052]: I1203 12:35:28.834674 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/97784b0f-4557-4bf2-b422-8d20fe37be9b-log-httpd\") pod \"ceilometer-0\" (UID: \"97784b0f-4557-4bf2-b422-8d20fe37be9b\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:35:28 crc kubenswrapper[5052]: I1203 12:35:28.834721 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/97784b0f-4557-4bf2-b422-8d20fe37be9b-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"97784b0f-4557-4bf2-b422-8d20fe37be9b\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:35:28 crc kubenswrapper[5052]: I1203 12:35:28.834824 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/97784b0f-4557-4bf2-b422-8d20fe37be9b-config-data\") pod \"ceilometer-0\" (UID: \"97784b0f-4557-4bf2-b422-8d20fe37be9b\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:35:28 crc kubenswrapper[5052]: I1203 12:35:28.834890 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/97784b0f-4557-4bf2-b422-8d20fe37be9b-run-httpd\") pod \"ceilometer-0\" (UID: \"97784b0f-4557-4bf2-b422-8d20fe37be9b\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:35:28 crc kubenswrapper[5052]: I1203 12:35:28.834948 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/97784b0f-4557-4bf2-b422-8d20fe37be9b-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"97784b0f-4557-4bf2-b422-8d20fe37be9b\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:35:28 crc kubenswrapper[5052]: I1203 12:35:28.938763 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/97784b0f-4557-4bf2-b422-8d20fe37be9b-scripts\") pod \"ceilometer-0\" (UID: \"97784b0f-4557-4bf2-b422-8d20fe37be9b\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:35:28 crc kubenswrapper[5052]: I1203 12:35:28.938843 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-jrqzr\" (UniqueName: \"kubernetes.io/projected/97784b0f-4557-4bf2-b422-8d20fe37be9b-kube-api-access-jrqzr\") pod \"ceilometer-0\" (UID: \"97784b0f-4557-4bf2-b422-8d20fe37be9b\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:35:28 crc kubenswrapper[5052]: I1203 12:35:28.938871 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/97784b0f-4557-4bf2-b422-8d20fe37be9b-log-httpd\") pod \"ceilometer-0\" (UID: \"97784b0f-4557-4bf2-b422-8d20fe37be9b\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:35:28 crc kubenswrapper[5052]: I1203 12:35:28.938908 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/97784b0f-4557-4bf2-b422-8d20fe37be9b-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"97784b0f-4557-4bf2-b422-8d20fe37be9b\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:35:28 crc kubenswrapper[5052]: I1203 12:35:28.938985 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/97784b0f-4557-4bf2-b422-8d20fe37be9b-config-data\") pod \"ceilometer-0\" (UID: \"97784b0f-4557-4bf2-b422-8d20fe37be9b\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:35:28 crc kubenswrapper[5052]: I1203 12:35:28.939037 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/97784b0f-4557-4bf2-b422-8d20fe37be9b-run-httpd\") pod \"ceilometer-0\" (UID: \"97784b0f-4557-4bf2-b422-8d20fe37be9b\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:35:28 crc kubenswrapper[5052]: I1203 12:35:28.939081 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/97784b0f-4557-4bf2-b422-8d20fe37be9b-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"97784b0f-4557-4bf2-b422-8d20fe37be9b\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:35:28 crc kubenswrapper[5052]: I1203 12:35:28.940360 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/97784b0f-4557-4bf2-b422-8d20fe37be9b-log-httpd\") pod \"ceilometer-0\" (UID: \"97784b0f-4557-4bf2-b422-8d20fe37be9b\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:35:28 crc kubenswrapper[5052]: I1203 12:35:28.942824 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/97784b0f-4557-4bf2-b422-8d20fe37be9b-run-httpd\") pod \"ceilometer-0\" (UID: \"97784b0f-4557-4bf2-b422-8d20fe37be9b\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:35:28 crc kubenswrapper[5052]: I1203 12:35:28.946388 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/97784b0f-4557-4bf2-b422-8d20fe37be9b-scripts\") pod \"ceilometer-0\" (UID: \"97784b0f-4557-4bf2-b422-8d20fe37be9b\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:35:28 crc kubenswrapper[5052]: I1203 12:35:28.946851 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/97784b0f-4557-4bf2-b422-8d20fe37be9b-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"97784b0f-4557-4bf2-b422-8d20fe37be9b\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:35:28 crc kubenswrapper[5052]: I1203 12:35:28.947678 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/97784b0f-4557-4bf2-b422-8d20fe37be9b-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"97784b0f-4557-4bf2-b422-8d20fe37be9b\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:35:28 crc kubenswrapper[5052]: I1203 12:35:28.959093 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-jrqzr\" (UniqueName: \"kubernetes.io/projected/97784b0f-4557-4bf2-b422-8d20fe37be9b-kube-api-access-jrqzr\") pod \"ceilometer-0\" (UID: \"97784b0f-4557-4bf2-b422-8d20fe37be9b\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:35:28 crc kubenswrapper[5052]: I1203 12:35:28.959683 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/97784b0f-4557-4bf2-b422-8d20fe37be9b-config-data\") pod \"ceilometer-0\" (UID: \"97784b0f-4557-4bf2-b422-8d20fe37be9b\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:35:29 crc kubenswrapper[5052]: I1203 12:35:29.079376 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:35:29 crc kubenswrapper[5052]: I1203 12:35:29.493736 5052 generic.go:334] "Generic (PLEG): container finished" podID="6daea253-ee08-4aa8-a900-39552cae7bd5" containerID="c3d94fea1d9f1a0f6f228595edb362c6bd0e2d49784198212c905e4bc050a564" exitCode=0 Dec 03 12:35:29 crc kubenswrapper[5052]: I1203 12:35:29.493886 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-7298-account-create-update-5sx49" event={"ID":"6daea253-ee08-4aa8-a900-39552cae7bd5","Type":"ContainerDied","Data":"c3d94fea1d9f1a0f6f228595edb362c6bd0e2d49784198212c905e4bc050a564"} Dec 03 12:35:29 crc kubenswrapper[5052]: I1203 12:35:29.497363 5052 generic.go:334] "Generic (PLEG): container finished" podID="c2dfe7a5-4ce2-4d6e-98e5-3ea6a4d3f3ca" containerID="7f51aca2747cb14558df5b7a3e191c04da61ef6ed6f56e77fe911e095658488b" exitCode=0 Dec 03 12:35:29 crc kubenswrapper[5052]: I1203 12:35:29.497409 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-692e-account-create-update-44rrb" event={"ID":"c2dfe7a5-4ce2-4d6e-98e5-3ea6a4d3f3ca","Type":"ContainerDied","Data":"7f51aca2747cb14558df5b7a3e191c04da61ef6ed6f56e77fe911e095658488b"} Dec 03 12:35:29 crc kubenswrapper[5052]: I1203 12:35:29.497427 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-692e-account-create-update-44rrb" event={"ID":"c2dfe7a5-4ce2-4d6e-98e5-3ea6a4d3f3ca","Type":"ContainerStarted","Data":"a46a4337632986ac6348aa0f72173051c920c215327e463d3bbf330a0d67c116"} Dec 03 12:35:29 crc kubenswrapper[5052]: I1203 12:35:29.499860 5052 generic.go:334] "Generic (PLEG): container finished" podID="2a2e1ec2-a657-4a31-93fd-712d45fab72b" containerID="00c3870e2cd22f1219f98f3fe1c8ae16c25295f882978dced379eafbfab9d78e" exitCode=0 Dec 03 12:35:29 crc kubenswrapper[5052]: I1203 12:35:29.499906 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-b163-account-create-update-d97hs" event={"ID":"2a2e1ec2-a657-4a31-93fd-712d45fab72b","Type":"ContainerDied","Data":"00c3870e2cd22f1219f98f3fe1c8ae16c25295f882978dced379eafbfab9d78e"} Dec 03 12:35:29 crc kubenswrapper[5052]: I1203 12:35:29.501487 5052 generic.go:334] "Generic (PLEG): container finished" podID="51f3388b-292f-40a9-9d41-ad60bb51ab68" containerID="8f30cdd770e803faba76b2be2f9c6ae0cba884a14e491e0151c3400a085e89d1" exitCode=0 Dec 03 12:35:29 crc kubenswrapper[5052]: I1203 12:35:29.501541 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-db-create-22ctd" event={"ID":"51f3388b-292f-40a9-9d41-ad60bb51ab68","Type":"ContainerDied","Data":"8f30cdd770e803faba76b2be2f9c6ae0cba884a14e491e0151c3400a085e89d1"} Dec 03 12:35:29 crc kubenswrapper[5052]: I1203 12:35:29.504666 5052 generic.go:334] "Generic (PLEG): container finished" podID="44c9835e-4fd7-4713-8e42-2d3f0a82fbfe" containerID="7cb1a81184710de145b19d3051c5761c627e9207bc129e10fab4d1acd4fc4174" exitCode=0 Dec 03 12:35:29 crc kubenswrapper[5052]: I1203 12:35:29.504852 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-db-create-4cj5q" event={"ID":"44c9835e-4fd7-4713-8e42-2d3f0a82fbfe","Type":"ContainerDied","Data":"7cb1a81184710de145b19d3051c5761c627e9207bc129e10fab4d1acd4fc4174"} Dec 03 12:35:29 crc kubenswrapper[5052]: I1203 12:35:29.512015 5052 generic.go:334] "Generic (PLEG): container finished" podID="ffa2902f-c174-44ec-98dc-de610f30b64f" containerID="1729d01686e7da24ac8eadd64fe8d08a11e6f6f31264cea2c7dd0a578e797dfb" exitCode=0 Dec 03 12:35:29 crc kubenswrapper[5052]: I1203 12:35:29.512643 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-db-create-k2lkr" event={"ID":"ffa2902f-c174-44ec-98dc-de610f30b64f","Type":"ContainerDied","Data":"1729d01686e7da24ac8eadd64fe8d08a11e6f6f31264cea2c7dd0a578e797dfb"} Dec 03 12:35:29 crc kubenswrapper[5052]: I1203 12:35:29.566268 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:35:29 crc kubenswrapper[5052]: W1203 12:35:29.580248 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod97784b0f_4557_4bf2_b422_8d20fe37be9b.slice/crio-9b72bcce6e077331593c7e5397c3e841799171a4ba34d83ac876a90ca5bcf4c5 WatchSource:0}: Error finding container 9b72bcce6e077331593c7e5397c3e841799171a4ba34d83ac876a90ca5bcf4c5: Status 404 returned error can't find the container with id 9b72bcce6e077331593c7e5397c3e841799171a4ba34d83ac876a90ca5bcf4c5 Dec 03 12:35:29 crc kubenswrapper[5052]: I1203 12:35:29.689157 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f45ca29a-a9fb-4644-be06-f728cc56d453" path="/var/lib/kubelet/pods/f45ca29a-a9fb-4644-be06-f728cc56d453/volumes" Dec 03 12:35:30 crc kubenswrapper[5052]: I1203 12:35:30.526897 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"97784b0f-4557-4bf2-b422-8d20fe37be9b","Type":"ContainerStarted","Data":"9b72bcce6e077331593c7e5397c3e841799171a4ba34d83ac876a90ca5bcf4c5"} Dec 03 12:35:30 crc kubenswrapper[5052]: I1203 12:35:30.975672 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-db-create-k2lkr" Dec 03 12:35:30 crc kubenswrapper[5052]: I1203 12:35:30.979617 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/ffa2902f-c174-44ec-98dc-de610f30b64f-operator-scripts\") pod \"ffa2902f-c174-44ec-98dc-de610f30b64f\" (UID: \"ffa2902f-c174-44ec-98dc-de610f30b64f\") " Dec 03 12:35:30 crc kubenswrapper[5052]: I1203 12:35:30.979707 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nb849\" (UniqueName: \"kubernetes.io/projected/ffa2902f-c174-44ec-98dc-de610f30b64f-kube-api-access-nb849\") pod \"ffa2902f-c174-44ec-98dc-de610f30b64f\" (UID: \"ffa2902f-c174-44ec-98dc-de610f30b64f\") " Dec 03 12:35:30 crc kubenswrapper[5052]: I1203 12:35:30.980115 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ffa2902f-c174-44ec-98dc-de610f30b64f-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "ffa2902f-c174-44ec-98dc-de610f30b64f" (UID: "ffa2902f-c174-44ec-98dc-de610f30b64f"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:35:30 crc kubenswrapper[5052]: I1203 12:35:30.980278 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/ffa2902f-c174-44ec-98dc-de610f30b64f-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:35:30 crc kubenswrapper[5052]: I1203 12:35:30.989778 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ffa2902f-c174-44ec-98dc-de610f30b64f-kube-api-access-nb849" (OuterVolumeSpecName: "kube-api-access-nb849") pod "ffa2902f-c174-44ec-98dc-de610f30b64f" (UID: "ffa2902f-c174-44ec-98dc-de610f30b64f"). InnerVolumeSpecName "kube-api-access-nb849". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:35:31 crc kubenswrapper[5052]: I1203 12:35:31.083810 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-nb849\" (UniqueName: \"kubernetes.io/projected/ffa2902f-c174-44ec-98dc-de610f30b64f-kube-api-access-nb849\") on node \"crc\" DevicePath \"\"" Dec 03 12:35:31 crc kubenswrapper[5052]: I1203 12:35:31.256012 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-db-create-4cj5q" Dec 03 12:35:31 crc kubenswrapper[5052]: I1203 12:35:31.264193 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-b163-account-create-update-d97hs" Dec 03 12:35:31 crc kubenswrapper[5052]: I1203 12:35:31.276667 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-7298-account-create-update-5sx49" Dec 03 12:35:31 crc kubenswrapper[5052]: I1203 12:35:31.286067 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-db-create-22ctd" Dec 03 12:35:31 crc kubenswrapper[5052]: I1203 12:35:31.287213 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/2a2e1ec2-a657-4a31-93fd-712d45fab72b-operator-scripts\") pod \"2a2e1ec2-a657-4a31-93fd-712d45fab72b\" (UID: \"2a2e1ec2-a657-4a31-93fd-712d45fab72b\") " Dec 03 12:35:31 crc kubenswrapper[5052]: I1203 12:35:31.287332 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4g27p\" (UniqueName: \"kubernetes.io/projected/2a2e1ec2-a657-4a31-93fd-712d45fab72b-kube-api-access-4g27p\") pod \"2a2e1ec2-a657-4a31-93fd-712d45fab72b\" (UID: \"2a2e1ec2-a657-4a31-93fd-712d45fab72b\") " Dec 03 12:35:31 crc kubenswrapper[5052]: I1203 12:35:31.287389 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/6daea253-ee08-4aa8-a900-39552cae7bd5-operator-scripts\") pod \"6daea253-ee08-4aa8-a900-39552cae7bd5\" (UID: \"6daea253-ee08-4aa8-a900-39552cae7bd5\") " Dec 03 12:35:31 crc kubenswrapper[5052]: I1203 12:35:31.287490 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8qkvj\" (UniqueName: \"kubernetes.io/projected/6daea253-ee08-4aa8-a900-39552cae7bd5-kube-api-access-8qkvj\") pod \"6daea253-ee08-4aa8-a900-39552cae7bd5\" (UID: \"6daea253-ee08-4aa8-a900-39552cae7bd5\") " Dec 03 12:35:31 crc kubenswrapper[5052]: I1203 12:35:31.287631 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5vbpp\" (UniqueName: \"kubernetes.io/projected/44c9835e-4fd7-4713-8e42-2d3f0a82fbfe-kube-api-access-5vbpp\") pod \"44c9835e-4fd7-4713-8e42-2d3f0a82fbfe\" (UID: \"44c9835e-4fd7-4713-8e42-2d3f0a82fbfe\") " Dec 03 12:35:31 crc kubenswrapper[5052]: I1203 12:35:31.287670 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/44c9835e-4fd7-4713-8e42-2d3f0a82fbfe-operator-scripts\") pod \"44c9835e-4fd7-4713-8e42-2d3f0a82fbfe\" (UID: \"44c9835e-4fd7-4713-8e42-2d3f0a82fbfe\") " Dec 03 12:35:31 crc kubenswrapper[5052]: I1203 12:35:31.288526 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/44c9835e-4fd7-4713-8e42-2d3f0a82fbfe-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "44c9835e-4fd7-4713-8e42-2d3f0a82fbfe" (UID: "44c9835e-4fd7-4713-8e42-2d3f0a82fbfe"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:35:31 crc kubenswrapper[5052]: I1203 12:35:31.288870 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6daea253-ee08-4aa8-a900-39552cae7bd5-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "6daea253-ee08-4aa8-a900-39552cae7bd5" (UID: "6daea253-ee08-4aa8-a900-39552cae7bd5"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:35:31 crc kubenswrapper[5052]: I1203 12:35:31.289391 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2a2e1ec2-a657-4a31-93fd-712d45fab72b-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "2a2e1ec2-a657-4a31-93fd-712d45fab72b" (UID: "2a2e1ec2-a657-4a31-93fd-712d45fab72b"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:35:31 crc kubenswrapper[5052]: I1203 12:35:31.292488 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6daea253-ee08-4aa8-a900-39552cae7bd5-kube-api-access-8qkvj" (OuterVolumeSpecName: "kube-api-access-8qkvj") pod "6daea253-ee08-4aa8-a900-39552cae7bd5" (UID: "6daea253-ee08-4aa8-a900-39552cae7bd5"). InnerVolumeSpecName "kube-api-access-8qkvj". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:35:31 crc kubenswrapper[5052]: I1203 12:35:31.310388 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-692e-account-create-update-44rrb" Dec 03 12:35:31 crc kubenswrapper[5052]: I1203 12:35:31.330918 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/44c9835e-4fd7-4713-8e42-2d3f0a82fbfe-kube-api-access-5vbpp" (OuterVolumeSpecName: "kube-api-access-5vbpp") pod "44c9835e-4fd7-4713-8e42-2d3f0a82fbfe" (UID: "44c9835e-4fd7-4713-8e42-2d3f0a82fbfe"). InnerVolumeSpecName "kube-api-access-5vbpp". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:35:31 crc kubenswrapper[5052]: I1203 12:35:31.332687 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2a2e1ec2-a657-4a31-93fd-712d45fab72b-kube-api-access-4g27p" (OuterVolumeSpecName: "kube-api-access-4g27p") pod "2a2e1ec2-a657-4a31-93fd-712d45fab72b" (UID: "2a2e1ec2-a657-4a31-93fd-712d45fab72b"). InnerVolumeSpecName "kube-api-access-4g27p". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:35:31 crc kubenswrapper[5052]: I1203 12:35:31.389927 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/51f3388b-292f-40a9-9d41-ad60bb51ab68-operator-scripts\") pod \"51f3388b-292f-40a9-9d41-ad60bb51ab68\" (UID: \"51f3388b-292f-40a9-9d41-ad60bb51ab68\") " Dec 03 12:35:31 crc kubenswrapper[5052]: I1203 12:35:31.390151 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-77k77\" (UniqueName: \"kubernetes.io/projected/51f3388b-292f-40a9-9d41-ad60bb51ab68-kube-api-access-77k77\") pod \"51f3388b-292f-40a9-9d41-ad60bb51ab68\" (UID: \"51f3388b-292f-40a9-9d41-ad60bb51ab68\") " Dec 03 12:35:31 crc kubenswrapper[5052]: I1203 12:35:31.390217 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/c2dfe7a5-4ce2-4d6e-98e5-3ea6a4d3f3ca-operator-scripts\") pod \"c2dfe7a5-4ce2-4d6e-98e5-3ea6a4d3f3ca\" (UID: \"c2dfe7a5-4ce2-4d6e-98e5-3ea6a4d3f3ca\") " Dec 03 12:35:31 crc kubenswrapper[5052]: I1203 12:35:31.390247 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2dp8m\" (UniqueName: \"kubernetes.io/projected/c2dfe7a5-4ce2-4d6e-98e5-3ea6a4d3f3ca-kube-api-access-2dp8m\") pod \"c2dfe7a5-4ce2-4d6e-98e5-3ea6a4d3f3ca\" (UID: \"c2dfe7a5-4ce2-4d6e-98e5-3ea6a4d3f3ca\") " Dec 03 12:35:31 crc kubenswrapper[5052]: I1203 12:35:31.390818 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-8qkvj\" (UniqueName: \"kubernetes.io/projected/6daea253-ee08-4aa8-a900-39552cae7bd5-kube-api-access-8qkvj\") on node \"crc\" DevicePath \"\"" Dec 03 12:35:31 crc kubenswrapper[5052]: I1203 12:35:31.390850 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-5vbpp\" (UniqueName: \"kubernetes.io/projected/44c9835e-4fd7-4713-8e42-2d3f0a82fbfe-kube-api-access-5vbpp\") on node \"crc\" DevicePath \"\"" Dec 03 12:35:31 crc kubenswrapper[5052]: I1203 12:35:31.390865 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/44c9835e-4fd7-4713-8e42-2d3f0a82fbfe-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:35:31 crc kubenswrapper[5052]: I1203 12:35:31.390878 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/2a2e1ec2-a657-4a31-93fd-712d45fab72b-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:35:31 crc kubenswrapper[5052]: I1203 12:35:31.390890 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-4g27p\" (UniqueName: \"kubernetes.io/projected/2a2e1ec2-a657-4a31-93fd-712d45fab72b-kube-api-access-4g27p\") on node \"crc\" DevicePath \"\"" Dec 03 12:35:31 crc kubenswrapper[5052]: I1203 12:35:31.390902 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/6daea253-ee08-4aa8-a900-39552cae7bd5-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:35:31 crc kubenswrapper[5052]: I1203 12:35:31.391324 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/51f3388b-292f-40a9-9d41-ad60bb51ab68-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "51f3388b-292f-40a9-9d41-ad60bb51ab68" (UID: "51f3388b-292f-40a9-9d41-ad60bb51ab68"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:35:31 crc kubenswrapper[5052]: I1203 12:35:31.391712 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c2dfe7a5-4ce2-4d6e-98e5-3ea6a4d3f3ca-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "c2dfe7a5-4ce2-4d6e-98e5-3ea6a4d3f3ca" (UID: "c2dfe7a5-4ce2-4d6e-98e5-3ea6a4d3f3ca"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:35:31 crc kubenswrapper[5052]: I1203 12:35:31.394956 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/51f3388b-292f-40a9-9d41-ad60bb51ab68-kube-api-access-77k77" (OuterVolumeSpecName: "kube-api-access-77k77") pod "51f3388b-292f-40a9-9d41-ad60bb51ab68" (UID: "51f3388b-292f-40a9-9d41-ad60bb51ab68"). InnerVolumeSpecName "kube-api-access-77k77". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:35:31 crc kubenswrapper[5052]: I1203 12:35:31.397833 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c2dfe7a5-4ce2-4d6e-98e5-3ea6a4d3f3ca-kube-api-access-2dp8m" (OuterVolumeSpecName: "kube-api-access-2dp8m") pod "c2dfe7a5-4ce2-4d6e-98e5-3ea6a4d3f3ca" (UID: "c2dfe7a5-4ce2-4d6e-98e5-3ea6a4d3f3ca"). InnerVolumeSpecName "kube-api-access-2dp8m". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:35:31 crc kubenswrapper[5052]: I1203 12:35:31.492488 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/c2dfe7a5-4ce2-4d6e-98e5-3ea6a4d3f3ca-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:35:31 crc kubenswrapper[5052]: I1203 12:35:31.492553 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-2dp8m\" (UniqueName: \"kubernetes.io/projected/c2dfe7a5-4ce2-4d6e-98e5-3ea6a4d3f3ca-kube-api-access-2dp8m\") on node \"crc\" DevicePath \"\"" Dec 03 12:35:31 crc kubenswrapper[5052]: I1203 12:35:31.492571 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/51f3388b-292f-40a9-9d41-ad60bb51ab68-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:35:31 crc kubenswrapper[5052]: I1203 12:35:31.492583 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-77k77\" (UniqueName: \"kubernetes.io/projected/51f3388b-292f-40a9-9d41-ad60bb51ab68-kube-api-access-77k77\") on node \"crc\" DevicePath \"\"" Dec 03 12:35:31 crc kubenswrapper[5052]: I1203 12:35:31.542555 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"97784b0f-4557-4bf2-b422-8d20fe37be9b","Type":"ContainerStarted","Data":"b4fd34b4f0c343db45fbd1bdb710f411c64de0872931440a1736bf965e2c9511"} Dec 03 12:35:31 crc kubenswrapper[5052]: I1203 12:35:31.542632 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"97784b0f-4557-4bf2-b422-8d20fe37be9b","Type":"ContainerStarted","Data":"aebafbcf69a07088182f53df7cedef96e0d49a5015e5751ce65b67a668fa738a"} Dec 03 12:35:31 crc kubenswrapper[5052]: I1203 12:35:31.544611 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-db-create-4cj5q" event={"ID":"44c9835e-4fd7-4713-8e42-2d3f0a82fbfe","Type":"ContainerDied","Data":"28c2e9820def366e06686c0d1645513ab9116fd3167a1e8f1bfe06ef99d116b3"} Dec 03 12:35:31 crc kubenswrapper[5052]: I1203 12:35:31.544645 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-db-create-4cj5q" Dec 03 12:35:31 crc kubenswrapper[5052]: I1203 12:35:31.544666 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="28c2e9820def366e06686c0d1645513ab9116fd3167a1e8f1bfe06ef99d116b3" Dec 03 12:35:31 crc kubenswrapper[5052]: I1203 12:35:31.546758 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-db-create-k2lkr" event={"ID":"ffa2902f-c174-44ec-98dc-de610f30b64f","Type":"ContainerDied","Data":"23f4286158cac12ee1c3f0fdbabba7d8eea04143035c21b79b41e1f2ac089be7"} Dec 03 12:35:31 crc kubenswrapper[5052]: I1203 12:35:31.546791 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-db-create-k2lkr" Dec 03 12:35:31 crc kubenswrapper[5052]: I1203 12:35:31.546808 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="23f4286158cac12ee1c3f0fdbabba7d8eea04143035c21b79b41e1f2ac089be7" Dec 03 12:35:31 crc kubenswrapper[5052]: I1203 12:35:31.548907 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-7298-account-create-update-5sx49" Dec 03 12:35:31 crc kubenswrapper[5052]: I1203 12:35:31.548924 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-7298-account-create-update-5sx49" event={"ID":"6daea253-ee08-4aa8-a900-39552cae7bd5","Type":"ContainerDied","Data":"3bdfd0a433dc5b6735ea2071c8fe5a8238869e13696292b23922925fb5d85641"} Dec 03 12:35:31 crc kubenswrapper[5052]: I1203 12:35:31.549153 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3bdfd0a433dc5b6735ea2071c8fe5a8238869e13696292b23922925fb5d85641" Dec 03 12:35:31 crc kubenswrapper[5052]: I1203 12:35:31.551640 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-692e-account-create-update-44rrb" Dec 03 12:35:31 crc kubenswrapper[5052]: I1203 12:35:31.551632 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-692e-account-create-update-44rrb" event={"ID":"c2dfe7a5-4ce2-4d6e-98e5-3ea6a4d3f3ca","Type":"ContainerDied","Data":"a46a4337632986ac6348aa0f72173051c920c215327e463d3bbf330a0d67c116"} Dec 03 12:35:31 crc kubenswrapper[5052]: I1203 12:35:31.551823 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a46a4337632986ac6348aa0f72173051c920c215327e463d3bbf330a0d67c116" Dec 03 12:35:31 crc kubenswrapper[5052]: I1203 12:35:31.565753 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-b163-account-create-update-d97hs" event={"ID":"2a2e1ec2-a657-4a31-93fd-712d45fab72b","Type":"ContainerDied","Data":"889cfe5d0933a701f06918f1859adb3c9f31937a7eb380bb157176f9d2069801"} Dec 03 12:35:31 crc kubenswrapper[5052]: I1203 12:35:31.565800 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="889cfe5d0933a701f06918f1859adb3c9f31937a7eb380bb157176f9d2069801" Dec 03 12:35:31 crc kubenswrapper[5052]: I1203 12:35:31.565951 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-b163-account-create-update-d97hs" Dec 03 12:35:31 crc kubenswrapper[5052]: I1203 12:35:31.580103 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-db-create-22ctd" event={"ID":"51f3388b-292f-40a9-9d41-ad60bb51ab68","Type":"ContainerDied","Data":"1f198ffb113b77f26ebc8588f32b3fbd3bbf98d228487035be86e73e18c98e05"} Dec 03 12:35:31 crc kubenswrapper[5052]: I1203 12:35:31.580437 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1f198ffb113b77f26ebc8588f32b3fbd3bbf98d228487035be86e73e18c98e05" Dec 03 12:35:31 crc kubenswrapper[5052]: I1203 12:35:31.580520 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-db-create-22ctd" Dec 03 12:35:32 crc kubenswrapper[5052]: I1203 12:35:32.592911 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"97784b0f-4557-4bf2-b422-8d20fe37be9b","Type":"ContainerStarted","Data":"d5995ac179c0fdb768efad6a24e81f03f3330db9b408b2b1cba70e9567a6bccb"} Dec 03 12:35:34 crc kubenswrapper[5052]: I1203 12:35:34.535644 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/neutron-9464b48df-szjp9" Dec 03 12:35:34 crc kubenswrapper[5052]: I1203 12:35:34.627942 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"97784b0f-4557-4bf2-b422-8d20fe37be9b","Type":"ContainerStarted","Data":"b240a09ead7e759bb5e70d1f913a5ad02d019dd404dd2a114be948b72358a3ee"} Dec 03 12:35:34 crc kubenswrapper[5052]: I1203 12:35:34.628103 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:35:34 crc kubenswrapper[5052]: I1203 12:35:34.652603 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/ceilometer-0" podStartSLOduration=2.400618669 podStartE2EDuration="6.652585647s" podCreationTimestamp="2025-12-03 12:35:28 +0000 UTC" firstStartedPulling="2025-12-03 12:35:29.582696527 +0000 UTC m=+4520.170938291" lastFinishedPulling="2025-12-03 12:35:33.834663505 +0000 UTC m=+4524.422905269" observedRunningTime="2025-12-03 12:35:34.649781076 +0000 UTC m=+4525.238022830" watchObservedRunningTime="2025-12-03 12:35:34.652585647 +0000 UTC m=+4525.240827401" Dec 03 12:35:34 crc kubenswrapper[5052]: I1203 12:35:34.672522 5052 scope.go:117] "RemoveContainer" containerID="e6edaf188f85ea4f11f487f11cc489c700f25fb42b5ee269f4f496f1e127d772" Dec 03 12:35:34 crc kubenswrapper[5052]: E1203 12:35:34.672745 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 12:35:35 crc kubenswrapper[5052]: I1203 12:35:35.627912 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:35:35 crc kubenswrapper[5052]: I1203 12:35:35.653494 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/swift-proxy-55f4df7cd-4gjcp" Dec 03 12:35:35 crc kubenswrapper[5052]: I1203 12:35:35.654441 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/swift-proxy-55f4df7cd-4gjcp" Dec 03 12:35:36 crc kubenswrapper[5052]: I1203 12:35:36.646374 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="97784b0f-4557-4bf2-b422-8d20fe37be9b" containerName="ceilometer-central-agent" containerID="cri-o://aebafbcf69a07088182f53df7cedef96e0d49a5015e5751ce65b67a668fa738a" gracePeriod=30 Dec 03 12:35:36 crc kubenswrapper[5052]: I1203 12:35:36.646408 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="97784b0f-4557-4bf2-b422-8d20fe37be9b" containerName="sg-core" containerID="cri-o://d5995ac179c0fdb768efad6a24e81f03f3330db9b408b2b1cba70e9567a6bccb" gracePeriod=30 Dec 03 12:35:36 crc kubenswrapper[5052]: I1203 12:35:36.646555 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="97784b0f-4557-4bf2-b422-8d20fe37be9b" containerName="proxy-httpd" containerID="cri-o://b240a09ead7e759bb5e70d1f913a5ad02d019dd404dd2a114be948b72358a3ee" gracePeriod=30 Dec 03 12:35:36 crc kubenswrapper[5052]: I1203 12:35:36.646501 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="97784b0f-4557-4bf2-b422-8d20fe37be9b" containerName="ceilometer-notification-agent" containerID="cri-o://b4fd34b4f0c343db45fbd1bdb710f411c64de0872931440a1736bf965e2c9511" gracePeriod=30 Dec 03 12:35:36 crc kubenswrapper[5052]: I1203 12:35:36.980977 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/neutron-5d67d5bcdb-klgzj" Dec 03 12:35:37 crc kubenswrapper[5052]: I1203 12:35:37.108888 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/neutron-9464b48df-szjp9"] Dec 03 12:35:37 crc kubenswrapper[5052]: I1203 12:35:37.109107 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/neutron-9464b48df-szjp9" podUID="f83ec921-2a86-43a8-8d18-3c66c694682e" containerName="neutron-api" containerID="cri-o://4aac3f358c0e898b24af2679d198e356f499489d60cbf8a19caaf910f13f9de4" gracePeriod=30 Dec 03 12:35:37 crc kubenswrapper[5052]: I1203 12:35:37.109227 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/neutron-9464b48df-szjp9" podUID="f83ec921-2a86-43a8-8d18-3c66c694682e" containerName="neutron-httpd" containerID="cri-o://96457f722879e7f4b70f63c2fb15496d232c98398206c25a1f690ebf24ca728b" gracePeriod=30 Dec 03 12:35:37 crc kubenswrapper[5052]: E1203 12:35:37.260872 5052 cadvisor_stats_provider.go:516] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf83ec921_2a86_43a8_8d18_3c66c694682e.slice/crio-96457f722879e7f4b70f63c2fb15496d232c98398206c25a1f690ebf24ca728b.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod97784b0f_4557_4bf2_b422_8d20fe37be9b.slice/crio-b4fd34b4f0c343db45fbd1bdb710f411c64de0872931440a1736bf965e2c9511.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod97784b0f_4557_4bf2_b422_8d20fe37be9b.slice/crio-conmon-aebafbcf69a07088182f53df7cedef96e0d49a5015e5751ce65b67a668fa738a.scope\": RecentStats: unable to find data in memory cache]" Dec 03 12:35:37 crc kubenswrapper[5052]: I1203 12:35:37.389901 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-db-sync-dpcff"] Dec 03 12:35:37 crc kubenswrapper[5052]: E1203 12:35:37.390869 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="ffa2902f-c174-44ec-98dc-de610f30b64f" containerName="mariadb-database-create" Dec 03 12:35:37 crc kubenswrapper[5052]: I1203 12:35:37.390891 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="ffa2902f-c174-44ec-98dc-de610f30b64f" containerName="mariadb-database-create" Dec 03 12:35:37 crc kubenswrapper[5052]: E1203 12:35:37.390902 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="6daea253-ee08-4aa8-a900-39552cae7bd5" containerName="mariadb-account-create-update" Dec 03 12:35:37 crc kubenswrapper[5052]: I1203 12:35:37.390909 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="6daea253-ee08-4aa8-a900-39552cae7bd5" containerName="mariadb-account-create-update" Dec 03 12:35:37 crc kubenswrapper[5052]: E1203 12:35:37.390924 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="c2dfe7a5-4ce2-4d6e-98e5-3ea6a4d3f3ca" containerName="mariadb-account-create-update" Dec 03 12:35:37 crc kubenswrapper[5052]: I1203 12:35:37.390934 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="c2dfe7a5-4ce2-4d6e-98e5-3ea6a4d3f3ca" containerName="mariadb-account-create-update" Dec 03 12:35:37 crc kubenswrapper[5052]: E1203 12:35:37.390961 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="51f3388b-292f-40a9-9d41-ad60bb51ab68" containerName="mariadb-database-create" Dec 03 12:35:37 crc kubenswrapper[5052]: I1203 12:35:37.390969 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="51f3388b-292f-40a9-9d41-ad60bb51ab68" containerName="mariadb-database-create" Dec 03 12:35:37 crc kubenswrapper[5052]: E1203 12:35:37.390984 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="44c9835e-4fd7-4713-8e42-2d3f0a82fbfe" containerName="mariadb-database-create" Dec 03 12:35:37 crc kubenswrapper[5052]: I1203 12:35:37.390992 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="44c9835e-4fd7-4713-8e42-2d3f0a82fbfe" containerName="mariadb-database-create" Dec 03 12:35:37 crc kubenswrapper[5052]: E1203 12:35:37.391009 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="2a2e1ec2-a657-4a31-93fd-712d45fab72b" containerName="mariadb-account-create-update" Dec 03 12:35:37 crc kubenswrapper[5052]: I1203 12:35:37.391018 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="2a2e1ec2-a657-4a31-93fd-712d45fab72b" containerName="mariadb-account-create-update" Dec 03 12:35:37 crc kubenswrapper[5052]: I1203 12:35:37.391229 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="44c9835e-4fd7-4713-8e42-2d3f0a82fbfe" containerName="mariadb-database-create" Dec 03 12:35:37 crc kubenswrapper[5052]: I1203 12:35:37.391260 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="2a2e1ec2-a657-4a31-93fd-712d45fab72b" containerName="mariadb-account-create-update" Dec 03 12:35:37 crc kubenswrapper[5052]: I1203 12:35:37.391283 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="51f3388b-292f-40a9-9d41-ad60bb51ab68" containerName="mariadb-database-create" Dec 03 12:35:37 crc kubenswrapper[5052]: I1203 12:35:37.391296 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="ffa2902f-c174-44ec-98dc-de610f30b64f" containerName="mariadb-database-create" Dec 03 12:35:37 crc kubenswrapper[5052]: I1203 12:35:37.391307 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="c2dfe7a5-4ce2-4d6e-98e5-3ea6a4d3f3ca" containerName="mariadb-account-create-update" Dec 03 12:35:37 crc kubenswrapper[5052]: I1203 12:35:37.391332 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="6daea253-ee08-4aa8-a900-39552cae7bd5" containerName="mariadb-account-create-update" Dec 03 12:35:37 crc kubenswrapper[5052]: I1203 12:35:37.392177 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-dpcff" Dec 03 12:35:37 crc kubenswrapper[5052]: I1203 12:35:37.396921 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell0-conductor-scripts" Dec 03 12:35:37 crc kubenswrapper[5052]: I1203 12:35:37.397203 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell0-conductor-config-data" Dec 03 12:35:37 crc kubenswrapper[5052]: I1203 12:35:37.397321 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-nova-dockercfg-t94dq" Dec 03 12:35:37 crc kubenswrapper[5052]: I1203 12:35:37.398383 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-db-sync-dpcff"] Dec 03 12:35:37 crc kubenswrapper[5052]: I1203 12:35:37.417620 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fcjbh\" (UniqueName: \"kubernetes.io/projected/63821a5d-93bd-41a3-9c7d-979723bfc035-kube-api-access-fcjbh\") pod \"nova-cell0-conductor-db-sync-dpcff\" (UID: \"63821a5d-93bd-41a3-9c7d-979723bfc035\") " pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-dpcff" Dec 03 12:35:37 crc kubenswrapper[5052]: I1203 12:35:37.417725 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/63821a5d-93bd-41a3-9c7d-979723bfc035-combined-ca-bundle\") pod \"nova-cell0-conductor-db-sync-dpcff\" (UID: \"63821a5d-93bd-41a3-9c7d-979723bfc035\") " pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-dpcff" Dec 03 12:35:37 crc kubenswrapper[5052]: I1203 12:35:37.417749 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/63821a5d-93bd-41a3-9c7d-979723bfc035-config-data\") pod \"nova-cell0-conductor-db-sync-dpcff\" (UID: \"63821a5d-93bd-41a3-9c7d-979723bfc035\") " pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-dpcff" Dec 03 12:35:37 crc kubenswrapper[5052]: I1203 12:35:37.417779 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/63821a5d-93bd-41a3-9c7d-979723bfc035-scripts\") pod \"nova-cell0-conductor-db-sync-dpcff\" (UID: \"63821a5d-93bd-41a3-9c7d-979723bfc035\") " pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-dpcff" Dec 03 12:35:37 crc kubenswrapper[5052]: I1203 12:35:37.447027 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:35:37 crc kubenswrapper[5052]: I1203 12:35:37.519418 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/63821a5d-93bd-41a3-9c7d-979723bfc035-combined-ca-bundle\") pod \"nova-cell0-conductor-db-sync-dpcff\" (UID: \"63821a5d-93bd-41a3-9c7d-979723bfc035\") " pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-dpcff" Dec 03 12:35:37 crc kubenswrapper[5052]: I1203 12:35:37.519473 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/63821a5d-93bd-41a3-9c7d-979723bfc035-config-data\") pod \"nova-cell0-conductor-db-sync-dpcff\" (UID: \"63821a5d-93bd-41a3-9c7d-979723bfc035\") " pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-dpcff" Dec 03 12:35:37 crc kubenswrapper[5052]: I1203 12:35:37.519532 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/63821a5d-93bd-41a3-9c7d-979723bfc035-scripts\") pod \"nova-cell0-conductor-db-sync-dpcff\" (UID: \"63821a5d-93bd-41a3-9c7d-979723bfc035\") " pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-dpcff" Dec 03 12:35:37 crc kubenswrapper[5052]: I1203 12:35:37.519648 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-fcjbh\" (UniqueName: \"kubernetes.io/projected/63821a5d-93bd-41a3-9c7d-979723bfc035-kube-api-access-fcjbh\") pod \"nova-cell0-conductor-db-sync-dpcff\" (UID: \"63821a5d-93bd-41a3-9c7d-979723bfc035\") " pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-dpcff" Dec 03 12:35:37 crc kubenswrapper[5052]: I1203 12:35:37.526261 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/63821a5d-93bd-41a3-9c7d-979723bfc035-combined-ca-bundle\") pod \"nova-cell0-conductor-db-sync-dpcff\" (UID: \"63821a5d-93bd-41a3-9c7d-979723bfc035\") " pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-dpcff" Dec 03 12:35:37 crc kubenswrapper[5052]: I1203 12:35:37.526371 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/63821a5d-93bd-41a3-9c7d-979723bfc035-config-data\") pod \"nova-cell0-conductor-db-sync-dpcff\" (UID: \"63821a5d-93bd-41a3-9c7d-979723bfc035\") " pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-dpcff" Dec 03 12:35:37 crc kubenswrapper[5052]: I1203 12:35:37.527086 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/63821a5d-93bd-41a3-9c7d-979723bfc035-scripts\") pod \"nova-cell0-conductor-db-sync-dpcff\" (UID: \"63821a5d-93bd-41a3-9c7d-979723bfc035\") " pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-dpcff" Dec 03 12:35:37 crc kubenswrapper[5052]: I1203 12:35:37.536091 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-fcjbh\" (UniqueName: \"kubernetes.io/projected/63821a5d-93bd-41a3-9c7d-979723bfc035-kube-api-access-fcjbh\") pod \"nova-cell0-conductor-db-sync-dpcff\" (UID: \"63821a5d-93bd-41a3-9c7d-979723bfc035\") " pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-dpcff" Dec 03 12:35:37 crc kubenswrapper[5052]: I1203 12:35:37.621687 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/97784b0f-4557-4bf2-b422-8d20fe37be9b-combined-ca-bundle\") pod \"97784b0f-4557-4bf2-b422-8d20fe37be9b\" (UID: \"97784b0f-4557-4bf2-b422-8d20fe37be9b\") " Dec 03 12:35:37 crc kubenswrapper[5052]: I1203 12:35:37.621756 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/97784b0f-4557-4bf2-b422-8d20fe37be9b-log-httpd\") pod \"97784b0f-4557-4bf2-b422-8d20fe37be9b\" (UID: \"97784b0f-4557-4bf2-b422-8d20fe37be9b\") " Dec 03 12:35:37 crc kubenswrapper[5052]: I1203 12:35:37.621799 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/97784b0f-4557-4bf2-b422-8d20fe37be9b-scripts\") pod \"97784b0f-4557-4bf2-b422-8d20fe37be9b\" (UID: \"97784b0f-4557-4bf2-b422-8d20fe37be9b\") " Dec 03 12:35:37 crc kubenswrapper[5052]: I1203 12:35:37.621841 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/97784b0f-4557-4bf2-b422-8d20fe37be9b-run-httpd\") pod \"97784b0f-4557-4bf2-b422-8d20fe37be9b\" (UID: \"97784b0f-4557-4bf2-b422-8d20fe37be9b\") " Dec 03 12:35:37 crc kubenswrapper[5052]: I1203 12:35:37.622027 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/97784b0f-4557-4bf2-b422-8d20fe37be9b-config-data\") pod \"97784b0f-4557-4bf2-b422-8d20fe37be9b\" (UID: \"97784b0f-4557-4bf2-b422-8d20fe37be9b\") " Dec 03 12:35:37 crc kubenswrapper[5052]: I1203 12:35:37.622070 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/97784b0f-4557-4bf2-b422-8d20fe37be9b-sg-core-conf-yaml\") pod \"97784b0f-4557-4bf2-b422-8d20fe37be9b\" (UID: \"97784b0f-4557-4bf2-b422-8d20fe37be9b\") " Dec 03 12:35:37 crc kubenswrapper[5052]: I1203 12:35:37.622097 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jrqzr\" (UniqueName: \"kubernetes.io/projected/97784b0f-4557-4bf2-b422-8d20fe37be9b-kube-api-access-jrqzr\") pod \"97784b0f-4557-4bf2-b422-8d20fe37be9b\" (UID: \"97784b0f-4557-4bf2-b422-8d20fe37be9b\") " Dec 03 12:35:37 crc kubenswrapper[5052]: I1203 12:35:37.622299 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/97784b0f-4557-4bf2-b422-8d20fe37be9b-run-httpd" (OuterVolumeSpecName: "run-httpd") pod "97784b0f-4557-4bf2-b422-8d20fe37be9b" (UID: "97784b0f-4557-4bf2-b422-8d20fe37be9b"). InnerVolumeSpecName "run-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:35:37 crc kubenswrapper[5052]: I1203 12:35:37.622365 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/97784b0f-4557-4bf2-b422-8d20fe37be9b-log-httpd" (OuterVolumeSpecName: "log-httpd") pod "97784b0f-4557-4bf2-b422-8d20fe37be9b" (UID: "97784b0f-4557-4bf2-b422-8d20fe37be9b"). InnerVolumeSpecName "log-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:35:37 crc kubenswrapper[5052]: I1203 12:35:37.622739 5052 reconciler_common.go:293] "Volume detached for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/97784b0f-4557-4bf2-b422-8d20fe37be9b-log-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:35:37 crc kubenswrapper[5052]: I1203 12:35:37.622765 5052 reconciler_common.go:293] "Volume detached for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/97784b0f-4557-4bf2-b422-8d20fe37be9b-run-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:35:37 crc kubenswrapper[5052]: I1203 12:35:37.625729 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/97784b0f-4557-4bf2-b422-8d20fe37be9b-kube-api-access-jrqzr" (OuterVolumeSpecName: "kube-api-access-jrqzr") pod "97784b0f-4557-4bf2-b422-8d20fe37be9b" (UID: "97784b0f-4557-4bf2-b422-8d20fe37be9b"). InnerVolumeSpecName "kube-api-access-jrqzr". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:35:37 crc kubenswrapper[5052]: I1203 12:35:37.626107 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/97784b0f-4557-4bf2-b422-8d20fe37be9b-scripts" (OuterVolumeSpecName: "scripts") pod "97784b0f-4557-4bf2-b422-8d20fe37be9b" (UID: "97784b0f-4557-4bf2-b422-8d20fe37be9b"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:35:37 crc kubenswrapper[5052]: I1203 12:35:37.680062 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/97784b0f-4557-4bf2-b422-8d20fe37be9b-sg-core-conf-yaml" (OuterVolumeSpecName: "sg-core-conf-yaml") pod "97784b0f-4557-4bf2-b422-8d20fe37be9b" (UID: "97784b0f-4557-4bf2-b422-8d20fe37be9b"). InnerVolumeSpecName "sg-core-conf-yaml". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:35:37 crc kubenswrapper[5052]: I1203 12:35:37.680905 5052 generic.go:334] "Generic (PLEG): container finished" podID="97784b0f-4557-4bf2-b422-8d20fe37be9b" containerID="b240a09ead7e759bb5e70d1f913a5ad02d019dd404dd2a114be948b72358a3ee" exitCode=0 Dec 03 12:35:37 crc kubenswrapper[5052]: I1203 12:35:37.680935 5052 generic.go:334] "Generic (PLEG): container finished" podID="97784b0f-4557-4bf2-b422-8d20fe37be9b" containerID="d5995ac179c0fdb768efad6a24e81f03f3330db9b408b2b1cba70e9567a6bccb" exitCode=2 Dec 03 12:35:37 crc kubenswrapper[5052]: I1203 12:35:37.680947 5052 generic.go:334] "Generic (PLEG): container finished" podID="97784b0f-4557-4bf2-b422-8d20fe37be9b" containerID="b4fd34b4f0c343db45fbd1bdb710f411c64de0872931440a1736bf965e2c9511" exitCode=0 Dec 03 12:35:37 crc kubenswrapper[5052]: I1203 12:35:37.680958 5052 generic.go:334] "Generic (PLEG): container finished" podID="97784b0f-4557-4bf2-b422-8d20fe37be9b" containerID="aebafbcf69a07088182f53df7cedef96e0d49a5015e5751ce65b67a668fa738a" exitCode=0 Dec 03 12:35:37 crc kubenswrapper[5052]: I1203 12:35:37.681098 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:35:37 crc kubenswrapper[5052]: I1203 12:35:37.684917 5052 generic.go:334] "Generic (PLEG): container finished" podID="f83ec921-2a86-43a8-8d18-3c66c694682e" containerID="96457f722879e7f4b70f63c2fb15496d232c98398206c25a1f690ebf24ca728b" exitCode=0 Dec 03 12:35:37 crc kubenswrapper[5052]: I1203 12:35:37.699052 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"97784b0f-4557-4bf2-b422-8d20fe37be9b","Type":"ContainerDied","Data":"b240a09ead7e759bb5e70d1f913a5ad02d019dd404dd2a114be948b72358a3ee"} Dec 03 12:35:37 crc kubenswrapper[5052]: I1203 12:35:37.699096 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"97784b0f-4557-4bf2-b422-8d20fe37be9b","Type":"ContainerDied","Data":"d5995ac179c0fdb768efad6a24e81f03f3330db9b408b2b1cba70e9567a6bccb"} Dec 03 12:35:37 crc kubenswrapper[5052]: I1203 12:35:37.699108 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"97784b0f-4557-4bf2-b422-8d20fe37be9b","Type":"ContainerDied","Data":"b4fd34b4f0c343db45fbd1bdb710f411c64de0872931440a1736bf965e2c9511"} Dec 03 12:35:37 crc kubenswrapper[5052]: I1203 12:35:37.699118 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"97784b0f-4557-4bf2-b422-8d20fe37be9b","Type":"ContainerDied","Data":"aebafbcf69a07088182f53df7cedef96e0d49a5015e5751ce65b67a668fa738a"} Dec 03 12:35:37 crc kubenswrapper[5052]: I1203 12:35:37.699129 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"97784b0f-4557-4bf2-b422-8d20fe37be9b","Type":"ContainerDied","Data":"9b72bcce6e077331593c7e5397c3e841799171a4ba34d83ac876a90ca5bcf4c5"} Dec 03 12:35:37 crc kubenswrapper[5052]: I1203 12:35:37.699139 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-9464b48df-szjp9" event={"ID":"f83ec921-2a86-43a8-8d18-3c66c694682e","Type":"ContainerDied","Data":"96457f722879e7f4b70f63c2fb15496d232c98398206c25a1f690ebf24ca728b"} Dec 03 12:35:37 crc kubenswrapper[5052]: I1203 12:35:37.699160 5052 scope.go:117] "RemoveContainer" containerID="b240a09ead7e759bb5e70d1f913a5ad02d019dd404dd2a114be948b72358a3ee" Dec 03 12:35:37 crc kubenswrapper[5052]: I1203 12:35:37.720290 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-dpcff" Dec 03 12:35:37 crc kubenswrapper[5052]: I1203 12:35:37.726235 5052 reconciler_common.go:293] "Volume detached for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/97784b0f-4557-4bf2-b422-8d20fe37be9b-sg-core-conf-yaml\") on node \"crc\" DevicePath \"\"" Dec 03 12:35:37 crc kubenswrapper[5052]: I1203 12:35:37.726268 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-jrqzr\" (UniqueName: \"kubernetes.io/projected/97784b0f-4557-4bf2-b422-8d20fe37be9b-kube-api-access-jrqzr\") on node \"crc\" DevicePath \"\"" Dec 03 12:35:37 crc kubenswrapper[5052]: I1203 12:35:37.726278 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/97784b0f-4557-4bf2-b422-8d20fe37be9b-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:35:37 crc kubenswrapper[5052]: I1203 12:35:37.730675 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/97784b0f-4557-4bf2-b422-8d20fe37be9b-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "97784b0f-4557-4bf2-b422-8d20fe37be9b" (UID: "97784b0f-4557-4bf2-b422-8d20fe37be9b"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:35:37 crc kubenswrapper[5052]: I1203 12:35:37.736895 5052 scope.go:117] "RemoveContainer" containerID="d5995ac179c0fdb768efad6a24e81f03f3330db9b408b2b1cba70e9567a6bccb" Dec 03 12:35:37 crc kubenswrapper[5052]: I1203 12:35:37.791874 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/97784b0f-4557-4bf2-b422-8d20fe37be9b-config-data" (OuterVolumeSpecName: "config-data") pod "97784b0f-4557-4bf2-b422-8d20fe37be9b" (UID: "97784b0f-4557-4bf2-b422-8d20fe37be9b"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:35:37 crc kubenswrapper[5052]: I1203 12:35:37.821906 5052 scope.go:117] "RemoveContainer" containerID="b4fd34b4f0c343db45fbd1bdb710f411c64de0872931440a1736bf965e2c9511" Dec 03 12:35:37 crc kubenswrapper[5052]: I1203 12:35:37.827189 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/97784b0f-4557-4bf2-b422-8d20fe37be9b-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:35:37 crc kubenswrapper[5052]: I1203 12:35:37.827217 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/97784b0f-4557-4bf2-b422-8d20fe37be9b-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:35:37 crc kubenswrapper[5052]: I1203 12:35:37.911404 5052 scope.go:117] "RemoveContainer" containerID="aebafbcf69a07088182f53df7cedef96e0d49a5015e5751ce65b67a668fa738a" Dec 03 12:35:37 crc kubenswrapper[5052]: I1203 12:35:37.943610 5052 scope.go:117] "RemoveContainer" containerID="b240a09ead7e759bb5e70d1f913a5ad02d019dd404dd2a114be948b72358a3ee" Dec 03 12:35:37 crc kubenswrapper[5052]: E1203 12:35:37.944786 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b240a09ead7e759bb5e70d1f913a5ad02d019dd404dd2a114be948b72358a3ee\": container with ID starting with b240a09ead7e759bb5e70d1f913a5ad02d019dd404dd2a114be948b72358a3ee not found: ID does not exist" containerID="b240a09ead7e759bb5e70d1f913a5ad02d019dd404dd2a114be948b72358a3ee" Dec 03 12:35:37 crc kubenswrapper[5052]: I1203 12:35:37.944822 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b240a09ead7e759bb5e70d1f913a5ad02d019dd404dd2a114be948b72358a3ee"} err="failed to get container status \"b240a09ead7e759bb5e70d1f913a5ad02d019dd404dd2a114be948b72358a3ee\": rpc error: code = NotFound desc = could not find container \"b240a09ead7e759bb5e70d1f913a5ad02d019dd404dd2a114be948b72358a3ee\": container with ID starting with b240a09ead7e759bb5e70d1f913a5ad02d019dd404dd2a114be948b72358a3ee not found: ID does not exist" Dec 03 12:35:37 crc kubenswrapper[5052]: I1203 12:35:37.944850 5052 scope.go:117] "RemoveContainer" containerID="d5995ac179c0fdb768efad6a24e81f03f3330db9b408b2b1cba70e9567a6bccb" Dec 03 12:35:37 crc kubenswrapper[5052]: E1203 12:35:37.945325 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d5995ac179c0fdb768efad6a24e81f03f3330db9b408b2b1cba70e9567a6bccb\": container with ID starting with d5995ac179c0fdb768efad6a24e81f03f3330db9b408b2b1cba70e9567a6bccb not found: ID does not exist" containerID="d5995ac179c0fdb768efad6a24e81f03f3330db9b408b2b1cba70e9567a6bccb" Dec 03 12:35:37 crc kubenswrapper[5052]: I1203 12:35:37.945377 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d5995ac179c0fdb768efad6a24e81f03f3330db9b408b2b1cba70e9567a6bccb"} err="failed to get container status \"d5995ac179c0fdb768efad6a24e81f03f3330db9b408b2b1cba70e9567a6bccb\": rpc error: code = NotFound desc = could not find container \"d5995ac179c0fdb768efad6a24e81f03f3330db9b408b2b1cba70e9567a6bccb\": container with ID starting with d5995ac179c0fdb768efad6a24e81f03f3330db9b408b2b1cba70e9567a6bccb not found: ID does not exist" Dec 03 12:35:37 crc kubenswrapper[5052]: I1203 12:35:37.945411 5052 scope.go:117] "RemoveContainer" containerID="b4fd34b4f0c343db45fbd1bdb710f411c64de0872931440a1736bf965e2c9511" Dec 03 12:35:37 crc kubenswrapper[5052]: E1203 12:35:37.946002 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b4fd34b4f0c343db45fbd1bdb710f411c64de0872931440a1736bf965e2c9511\": container with ID starting with b4fd34b4f0c343db45fbd1bdb710f411c64de0872931440a1736bf965e2c9511 not found: ID does not exist" containerID="b4fd34b4f0c343db45fbd1bdb710f411c64de0872931440a1736bf965e2c9511" Dec 03 12:35:37 crc kubenswrapper[5052]: I1203 12:35:37.946032 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b4fd34b4f0c343db45fbd1bdb710f411c64de0872931440a1736bf965e2c9511"} err="failed to get container status \"b4fd34b4f0c343db45fbd1bdb710f411c64de0872931440a1736bf965e2c9511\": rpc error: code = NotFound desc = could not find container \"b4fd34b4f0c343db45fbd1bdb710f411c64de0872931440a1736bf965e2c9511\": container with ID starting with b4fd34b4f0c343db45fbd1bdb710f411c64de0872931440a1736bf965e2c9511 not found: ID does not exist" Dec 03 12:35:37 crc kubenswrapper[5052]: I1203 12:35:37.946052 5052 scope.go:117] "RemoveContainer" containerID="aebafbcf69a07088182f53df7cedef96e0d49a5015e5751ce65b67a668fa738a" Dec 03 12:35:37 crc kubenswrapper[5052]: E1203 12:35:37.946969 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aebafbcf69a07088182f53df7cedef96e0d49a5015e5751ce65b67a668fa738a\": container with ID starting with aebafbcf69a07088182f53df7cedef96e0d49a5015e5751ce65b67a668fa738a not found: ID does not exist" containerID="aebafbcf69a07088182f53df7cedef96e0d49a5015e5751ce65b67a668fa738a" Dec 03 12:35:37 crc kubenswrapper[5052]: I1203 12:35:37.946999 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aebafbcf69a07088182f53df7cedef96e0d49a5015e5751ce65b67a668fa738a"} err="failed to get container status \"aebafbcf69a07088182f53df7cedef96e0d49a5015e5751ce65b67a668fa738a\": rpc error: code = NotFound desc = could not find container \"aebafbcf69a07088182f53df7cedef96e0d49a5015e5751ce65b67a668fa738a\": container with ID starting with aebafbcf69a07088182f53df7cedef96e0d49a5015e5751ce65b67a668fa738a not found: ID does not exist" Dec 03 12:35:37 crc kubenswrapper[5052]: I1203 12:35:37.947016 5052 scope.go:117] "RemoveContainer" containerID="b240a09ead7e759bb5e70d1f913a5ad02d019dd404dd2a114be948b72358a3ee" Dec 03 12:35:37 crc kubenswrapper[5052]: I1203 12:35:37.947300 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b240a09ead7e759bb5e70d1f913a5ad02d019dd404dd2a114be948b72358a3ee"} err="failed to get container status \"b240a09ead7e759bb5e70d1f913a5ad02d019dd404dd2a114be948b72358a3ee\": rpc error: code = NotFound desc = could not find container \"b240a09ead7e759bb5e70d1f913a5ad02d019dd404dd2a114be948b72358a3ee\": container with ID starting with b240a09ead7e759bb5e70d1f913a5ad02d019dd404dd2a114be948b72358a3ee not found: ID does not exist" Dec 03 12:35:37 crc kubenswrapper[5052]: I1203 12:35:37.947324 5052 scope.go:117] "RemoveContainer" containerID="d5995ac179c0fdb768efad6a24e81f03f3330db9b408b2b1cba70e9567a6bccb" Dec 03 12:35:37 crc kubenswrapper[5052]: I1203 12:35:37.947882 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d5995ac179c0fdb768efad6a24e81f03f3330db9b408b2b1cba70e9567a6bccb"} err="failed to get container status \"d5995ac179c0fdb768efad6a24e81f03f3330db9b408b2b1cba70e9567a6bccb\": rpc error: code = NotFound desc = could not find container \"d5995ac179c0fdb768efad6a24e81f03f3330db9b408b2b1cba70e9567a6bccb\": container with ID starting with d5995ac179c0fdb768efad6a24e81f03f3330db9b408b2b1cba70e9567a6bccb not found: ID does not exist" Dec 03 12:35:37 crc kubenswrapper[5052]: I1203 12:35:37.947920 5052 scope.go:117] "RemoveContainer" containerID="b4fd34b4f0c343db45fbd1bdb710f411c64de0872931440a1736bf965e2c9511" Dec 03 12:35:37 crc kubenswrapper[5052]: I1203 12:35:37.949093 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b4fd34b4f0c343db45fbd1bdb710f411c64de0872931440a1736bf965e2c9511"} err="failed to get container status \"b4fd34b4f0c343db45fbd1bdb710f411c64de0872931440a1736bf965e2c9511\": rpc error: code = NotFound desc = could not find container \"b4fd34b4f0c343db45fbd1bdb710f411c64de0872931440a1736bf965e2c9511\": container with ID starting with b4fd34b4f0c343db45fbd1bdb710f411c64de0872931440a1736bf965e2c9511 not found: ID does not exist" Dec 03 12:35:37 crc kubenswrapper[5052]: I1203 12:35:37.949118 5052 scope.go:117] "RemoveContainer" containerID="aebafbcf69a07088182f53df7cedef96e0d49a5015e5751ce65b67a668fa738a" Dec 03 12:35:37 crc kubenswrapper[5052]: I1203 12:35:37.949712 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aebafbcf69a07088182f53df7cedef96e0d49a5015e5751ce65b67a668fa738a"} err="failed to get container status \"aebafbcf69a07088182f53df7cedef96e0d49a5015e5751ce65b67a668fa738a\": rpc error: code = NotFound desc = could not find container \"aebafbcf69a07088182f53df7cedef96e0d49a5015e5751ce65b67a668fa738a\": container with ID starting with aebafbcf69a07088182f53df7cedef96e0d49a5015e5751ce65b67a668fa738a not found: ID does not exist" Dec 03 12:35:37 crc kubenswrapper[5052]: I1203 12:35:37.949736 5052 scope.go:117] "RemoveContainer" containerID="b240a09ead7e759bb5e70d1f913a5ad02d019dd404dd2a114be948b72358a3ee" Dec 03 12:35:37 crc kubenswrapper[5052]: I1203 12:35:37.950013 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b240a09ead7e759bb5e70d1f913a5ad02d019dd404dd2a114be948b72358a3ee"} err="failed to get container status \"b240a09ead7e759bb5e70d1f913a5ad02d019dd404dd2a114be948b72358a3ee\": rpc error: code = NotFound desc = could not find container \"b240a09ead7e759bb5e70d1f913a5ad02d019dd404dd2a114be948b72358a3ee\": container with ID starting with b240a09ead7e759bb5e70d1f913a5ad02d019dd404dd2a114be948b72358a3ee not found: ID does not exist" Dec 03 12:35:37 crc kubenswrapper[5052]: I1203 12:35:37.950031 5052 scope.go:117] "RemoveContainer" containerID="d5995ac179c0fdb768efad6a24e81f03f3330db9b408b2b1cba70e9567a6bccb" Dec 03 12:35:37 crc kubenswrapper[5052]: I1203 12:35:37.950278 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d5995ac179c0fdb768efad6a24e81f03f3330db9b408b2b1cba70e9567a6bccb"} err="failed to get container status \"d5995ac179c0fdb768efad6a24e81f03f3330db9b408b2b1cba70e9567a6bccb\": rpc error: code = NotFound desc = could not find container \"d5995ac179c0fdb768efad6a24e81f03f3330db9b408b2b1cba70e9567a6bccb\": container with ID starting with d5995ac179c0fdb768efad6a24e81f03f3330db9b408b2b1cba70e9567a6bccb not found: ID does not exist" Dec 03 12:35:37 crc kubenswrapper[5052]: I1203 12:35:37.950299 5052 scope.go:117] "RemoveContainer" containerID="b4fd34b4f0c343db45fbd1bdb710f411c64de0872931440a1736bf965e2c9511" Dec 03 12:35:37 crc kubenswrapper[5052]: I1203 12:35:37.951133 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b4fd34b4f0c343db45fbd1bdb710f411c64de0872931440a1736bf965e2c9511"} err="failed to get container status \"b4fd34b4f0c343db45fbd1bdb710f411c64de0872931440a1736bf965e2c9511\": rpc error: code = NotFound desc = could not find container \"b4fd34b4f0c343db45fbd1bdb710f411c64de0872931440a1736bf965e2c9511\": container with ID starting with b4fd34b4f0c343db45fbd1bdb710f411c64de0872931440a1736bf965e2c9511 not found: ID does not exist" Dec 03 12:35:37 crc kubenswrapper[5052]: I1203 12:35:37.951155 5052 scope.go:117] "RemoveContainer" containerID="aebafbcf69a07088182f53df7cedef96e0d49a5015e5751ce65b67a668fa738a" Dec 03 12:35:37 crc kubenswrapper[5052]: I1203 12:35:37.951590 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aebafbcf69a07088182f53df7cedef96e0d49a5015e5751ce65b67a668fa738a"} err="failed to get container status \"aebafbcf69a07088182f53df7cedef96e0d49a5015e5751ce65b67a668fa738a\": rpc error: code = NotFound desc = could not find container \"aebafbcf69a07088182f53df7cedef96e0d49a5015e5751ce65b67a668fa738a\": container with ID starting with aebafbcf69a07088182f53df7cedef96e0d49a5015e5751ce65b67a668fa738a not found: ID does not exist" Dec 03 12:35:37 crc kubenswrapper[5052]: I1203 12:35:37.951608 5052 scope.go:117] "RemoveContainer" containerID="b240a09ead7e759bb5e70d1f913a5ad02d019dd404dd2a114be948b72358a3ee" Dec 03 12:35:37 crc kubenswrapper[5052]: I1203 12:35:37.951810 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b240a09ead7e759bb5e70d1f913a5ad02d019dd404dd2a114be948b72358a3ee"} err="failed to get container status \"b240a09ead7e759bb5e70d1f913a5ad02d019dd404dd2a114be948b72358a3ee\": rpc error: code = NotFound desc = could not find container \"b240a09ead7e759bb5e70d1f913a5ad02d019dd404dd2a114be948b72358a3ee\": container with ID starting with b240a09ead7e759bb5e70d1f913a5ad02d019dd404dd2a114be948b72358a3ee not found: ID does not exist" Dec 03 12:35:37 crc kubenswrapper[5052]: I1203 12:35:37.951825 5052 scope.go:117] "RemoveContainer" containerID="d5995ac179c0fdb768efad6a24e81f03f3330db9b408b2b1cba70e9567a6bccb" Dec 03 12:35:37 crc kubenswrapper[5052]: I1203 12:35:37.952123 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d5995ac179c0fdb768efad6a24e81f03f3330db9b408b2b1cba70e9567a6bccb"} err="failed to get container status \"d5995ac179c0fdb768efad6a24e81f03f3330db9b408b2b1cba70e9567a6bccb\": rpc error: code = NotFound desc = could not find container \"d5995ac179c0fdb768efad6a24e81f03f3330db9b408b2b1cba70e9567a6bccb\": container with ID starting with d5995ac179c0fdb768efad6a24e81f03f3330db9b408b2b1cba70e9567a6bccb not found: ID does not exist" Dec 03 12:35:37 crc kubenswrapper[5052]: I1203 12:35:37.952138 5052 scope.go:117] "RemoveContainer" containerID="b4fd34b4f0c343db45fbd1bdb710f411c64de0872931440a1736bf965e2c9511" Dec 03 12:35:37 crc kubenswrapper[5052]: I1203 12:35:37.952378 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b4fd34b4f0c343db45fbd1bdb710f411c64de0872931440a1736bf965e2c9511"} err="failed to get container status \"b4fd34b4f0c343db45fbd1bdb710f411c64de0872931440a1736bf965e2c9511\": rpc error: code = NotFound desc = could not find container \"b4fd34b4f0c343db45fbd1bdb710f411c64de0872931440a1736bf965e2c9511\": container with ID starting with b4fd34b4f0c343db45fbd1bdb710f411c64de0872931440a1736bf965e2c9511 not found: ID does not exist" Dec 03 12:35:37 crc kubenswrapper[5052]: I1203 12:35:37.952395 5052 scope.go:117] "RemoveContainer" containerID="aebafbcf69a07088182f53df7cedef96e0d49a5015e5751ce65b67a668fa738a" Dec 03 12:35:37 crc kubenswrapper[5052]: I1203 12:35:37.952961 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aebafbcf69a07088182f53df7cedef96e0d49a5015e5751ce65b67a668fa738a"} err="failed to get container status \"aebafbcf69a07088182f53df7cedef96e0d49a5015e5751ce65b67a668fa738a\": rpc error: code = NotFound desc = could not find container \"aebafbcf69a07088182f53df7cedef96e0d49a5015e5751ce65b67a668fa738a\": container with ID starting with aebafbcf69a07088182f53df7cedef96e0d49a5015e5751ce65b67a668fa738a not found: ID does not exist" Dec 03 12:35:38 crc kubenswrapper[5052]: I1203 12:35:38.022993 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:35:38 crc kubenswrapper[5052]: I1203 12:35:38.032553 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:35:38 crc kubenswrapper[5052]: I1203 12:35:38.066338 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:35:38 crc kubenswrapper[5052]: E1203 12:35:38.066845 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="97784b0f-4557-4bf2-b422-8d20fe37be9b" containerName="ceilometer-notification-agent" Dec 03 12:35:38 crc kubenswrapper[5052]: I1203 12:35:38.066869 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="97784b0f-4557-4bf2-b422-8d20fe37be9b" containerName="ceilometer-notification-agent" Dec 03 12:35:38 crc kubenswrapper[5052]: E1203 12:35:38.066896 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="97784b0f-4557-4bf2-b422-8d20fe37be9b" containerName="ceilometer-central-agent" Dec 03 12:35:38 crc kubenswrapper[5052]: I1203 12:35:38.066905 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="97784b0f-4557-4bf2-b422-8d20fe37be9b" containerName="ceilometer-central-agent" Dec 03 12:35:38 crc kubenswrapper[5052]: E1203 12:35:38.066919 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="97784b0f-4557-4bf2-b422-8d20fe37be9b" containerName="sg-core" Dec 03 12:35:38 crc kubenswrapper[5052]: I1203 12:35:38.066927 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="97784b0f-4557-4bf2-b422-8d20fe37be9b" containerName="sg-core" Dec 03 12:35:38 crc kubenswrapper[5052]: E1203 12:35:38.066941 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="97784b0f-4557-4bf2-b422-8d20fe37be9b" containerName="proxy-httpd" Dec 03 12:35:38 crc kubenswrapper[5052]: I1203 12:35:38.066950 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="97784b0f-4557-4bf2-b422-8d20fe37be9b" containerName="proxy-httpd" Dec 03 12:35:38 crc kubenswrapper[5052]: I1203 12:35:38.067172 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="97784b0f-4557-4bf2-b422-8d20fe37be9b" containerName="ceilometer-notification-agent" Dec 03 12:35:38 crc kubenswrapper[5052]: I1203 12:35:38.067196 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="97784b0f-4557-4bf2-b422-8d20fe37be9b" containerName="proxy-httpd" Dec 03 12:35:38 crc kubenswrapper[5052]: I1203 12:35:38.067206 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="97784b0f-4557-4bf2-b422-8d20fe37be9b" containerName="ceilometer-central-agent" Dec 03 12:35:38 crc kubenswrapper[5052]: I1203 12:35:38.067227 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="97784b0f-4557-4bf2-b422-8d20fe37be9b" containerName="sg-core" Dec 03 12:35:38 crc kubenswrapper[5052]: I1203 12:35:38.068934 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:35:38 crc kubenswrapper[5052]: I1203 12:35:38.071708 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-scripts" Dec 03 12:35:38 crc kubenswrapper[5052]: I1203 12:35:38.072259 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-config-data" Dec 03 12:35:38 crc kubenswrapper[5052]: I1203 12:35:38.095830 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:35:38 crc kubenswrapper[5052]: I1203 12:35:38.135045 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/86ddb247-3041-44d5-aa85-dad99d370fff-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"86ddb247-3041-44d5-aa85-dad99d370fff\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:35:38 crc kubenswrapper[5052]: I1203 12:35:38.135154 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/86ddb247-3041-44d5-aa85-dad99d370fff-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"86ddb247-3041-44d5-aa85-dad99d370fff\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:35:38 crc kubenswrapper[5052]: I1203 12:35:38.135394 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/86ddb247-3041-44d5-aa85-dad99d370fff-log-httpd\") pod \"ceilometer-0\" (UID: \"86ddb247-3041-44d5-aa85-dad99d370fff\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:35:38 crc kubenswrapper[5052]: I1203 12:35:38.135454 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xkqvz\" (UniqueName: \"kubernetes.io/projected/86ddb247-3041-44d5-aa85-dad99d370fff-kube-api-access-xkqvz\") pod \"ceilometer-0\" (UID: \"86ddb247-3041-44d5-aa85-dad99d370fff\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:35:38 crc kubenswrapper[5052]: I1203 12:35:38.135541 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/86ddb247-3041-44d5-aa85-dad99d370fff-run-httpd\") pod \"ceilometer-0\" (UID: \"86ddb247-3041-44d5-aa85-dad99d370fff\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:35:38 crc kubenswrapper[5052]: I1203 12:35:38.135574 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/86ddb247-3041-44d5-aa85-dad99d370fff-scripts\") pod \"ceilometer-0\" (UID: \"86ddb247-3041-44d5-aa85-dad99d370fff\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:35:38 crc kubenswrapper[5052]: I1203 12:35:38.135717 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/86ddb247-3041-44d5-aa85-dad99d370fff-config-data\") pod \"ceilometer-0\" (UID: \"86ddb247-3041-44d5-aa85-dad99d370fff\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:35:38 crc kubenswrapper[5052]: I1203 12:35:38.194208 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-db-sync-dpcff"] Dec 03 12:35:38 crc kubenswrapper[5052]: W1203 12:35:38.203526 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod63821a5d_93bd_41a3_9c7d_979723bfc035.slice/crio-ae1fbd48e118f8aacf6561ab68e82755a7906595b39f4dd49fd54b325a932883 WatchSource:0}: Error finding container ae1fbd48e118f8aacf6561ab68e82755a7906595b39f4dd49fd54b325a932883: Status 404 returned error can't find the container with id ae1fbd48e118f8aacf6561ab68e82755a7906595b39f4dd49fd54b325a932883 Dec 03 12:35:38 crc kubenswrapper[5052]: I1203 12:35:38.238291 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/86ddb247-3041-44d5-aa85-dad99d370fff-config-data\") pod \"ceilometer-0\" (UID: \"86ddb247-3041-44d5-aa85-dad99d370fff\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:35:38 crc kubenswrapper[5052]: I1203 12:35:38.238969 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/86ddb247-3041-44d5-aa85-dad99d370fff-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"86ddb247-3041-44d5-aa85-dad99d370fff\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:35:38 crc kubenswrapper[5052]: I1203 12:35:38.239087 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/86ddb247-3041-44d5-aa85-dad99d370fff-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"86ddb247-3041-44d5-aa85-dad99d370fff\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:35:38 crc kubenswrapper[5052]: I1203 12:35:38.239173 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/86ddb247-3041-44d5-aa85-dad99d370fff-log-httpd\") pod \"ceilometer-0\" (UID: \"86ddb247-3041-44d5-aa85-dad99d370fff\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:35:38 crc kubenswrapper[5052]: I1203 12:35:38.239218 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-xkqvz\" (UniqueName: \"kubernetes.io/projected/86ddb247-3041-44d5-aa85-dad99d370fff-kube-api-access-xkqvz\") pod \"ceilometer-0\" (UID: \"86ddb247-3041-44d5-aa85-dad99d370fff\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:35:38 crc kubenswrapper[5052]: I1203 12:35:38.239266 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/86ddb247-3041-44d5-aa85-dad99d370fff-run-httpd\") pod \"ceilometer-0\" (UID: \"86ddb247-3041-44d5-aa85-dad99d370fff\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:35:38 crc kubenswrapper[5052]: I1203 12:35:38.239299 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/86ddb247-3041-44d5-aa85-dad99d370fff-scripts\") pod \"ceilometer-0\" (UID: \"86ddb247-3041-44d5-aa85-dad99d370fff\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:35:38 crc kubenswrapper[5052]: I1203 12:35:38.240089 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/86ddb247-3041-44d5-aa85-dad99d370fff-log-httpd\") pod \"ceilometer-0\" (UID: \"86ddb247-3041-44d5-aa85-dad99d370fff\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:35:38 crc kubenswrapper[5052]: I1203 12:35:38.240558 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/86ddb247-3041-44d5-aa85-dad99d370fff-run-httpd\") pod \"ceilometer-0\" (UID: \"86ddb247-3041-44d5-aa85-dad99d370fff\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:35:38 crc kubenswrapper[5052]: I1203 12:35:38.246246 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/86ddb247-3041-44d5-aa85-dad99d370fff-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"86ddb247-3041-44d5-aa85-dad99d370fff\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:35:38 crc kubenswrapper[5052]: I1203 12:35:38.248788 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/86ddb247-3041-44d5-aa85-dad99d370fff-scripts\") pod \"ceilometer-0\" (UID: \"86ddb247-3041-44d5-aa85-dad99d370fff\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:35:38 crc kubenswrapper[5052]: I1203 12:35:38.248868 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/86ddb247-3041-44d5-aa85-dad99d370fff-config-data\") pod \"ceilometer-0\" (UID: \"86ddb247-3041-44d5-aa85-dad99d370fff\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:35:38 crc kubenswrapper[5052]: I1203 12:35:38.258352 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/86ddb247-3041-44d5-aa85-dad99d370fff-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"86ddb247-3041-44d5-aa85-dad99d370fff\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:35:38 crc kubenswrapper[5052]: I1203 12:35:38.268312 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-xkqvz\" (UniqueName: \"kubernetes.io/projected/86ddb247-3041-44d5-aa85-dad99d370fff-kube-api-access-xkqvz\") pod \"ceilometer-0\" (UID: \"86ddb247-3041-44d5-aa85-dad99d370fff\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:35:38 crc kubenswrapper[5052]: I1203 12:35:38.384156 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:35:38 crc kubenswrapper[5052]: I1203 12:35:38.694633 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-dpcff" event={"ID":"63821a5d-93bd-41a3-9c7d-979723bfc035","Type":"ContainerStarted","Data":"9330194596eda8836d7765112355fa440ad87fd5443eb2c1506c02b1767075eb"} Dec 03 12:35:38 crc kubenswrapper[5052]: I1203 12:35:38.695051 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-dpcff" event={"ID":"63821a5d-93bd-41a3-9c7d-979723bfc035","Type":"ContainerStarted","Data":"ae1fbd48e118f8aacf6561ab68e82755a7906595b39f4dd49fd54b325a932883"} Dec 03 12:35:38 crc kubenswrapper[5052]: I1203 12:35:38.717868 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-dpcff" podStartSLOduration=1.717853826 podStartE2EDuration="1.717853826s" podCreationTimestamp="2025-12-03 12:35:37 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:35:38.716896569 +0000 UTC m=+4529.305138333" watchObservedRunningTime="2025-12-03 12:35:38.717853826 +0000 UTC m=+4529.306095590" Dec 03 12:35:38 crc kubenswrapper[5052]: I1203 12:35:38.837849 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:35:38 crc kubenswrapper[5052]: W1203 12:35:38.838599 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod86ddb247_3041_44d5_aa85_dad99d370fff.slice/crio-f9619b85e60754f9d7e9ed84fb60cd8a8a719b345930168ee4192840af22b96c WatchSource:0}: Error finding container f9619b85e60754f9d7e9ed84fb60cd8a8a719b345930168ee4192840af22b96c: Status 404 returned error can't find the container with id f9619b85e60754f9d7e9ed84fb60cd8a8a719b345930168ee4192840af22b96c Dec 03 12:35:39 crc kubenswrapper[5052]: I1203 12:35:39.576688 5052 provider.go:102] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Dec 03 12:35:39 crc kubenswrapper[5052]: I1203 12:35:39.700282 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="97784b0f-4557-4bf2-b422-8d20fe37be9b" path="/var/lib/kubelet/pods/97784b0f-4557-4bf2-b422-8d20fe37be9b/volumes" Dec 03 12:35:39 crc kubenswrapper[5052]: I1203 12:35:39.719640 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"86ddb247-3041-44d5-aa85-dad99d370fff","Type":"ContainerStarted","Data":"9ee7a5065be7f5713680e21754e8ad61a263d7c54861980ccdf4094d374d1af1"} Dec 03 12:35:39 crc kubenswrapper[5052]: I1203 12:35:39.719716 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"86ddb247-3041-44d5-aa85-dad99d370fff","Type":"ContainerStarted","Data":"f9619b85e60754f9d7e9ed84fb60cd8a8a719b345930168ee4192840af22b96c"} Dec 03 12:35:40 crc kubenswrapper[5052]: I1203 12:35:40.747643 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"86ddb247-3041-44d5-aa85-dad99d370fff","Type":"ContainerStarted","Data":"4559674fd0ebae4b6c6a91c9926fba76caa1021c14f8396d3aee06b7d0209af6"} Dec 03 12:35:43 crc kubenswrapper[5052]: I1203 12:35:43.279479 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-9464b48df-szjp9" Dec 03 12:35:43 crc kubenswrapper[5052]: I1203 12:35:43.450535 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/f83ec921-2a86-43a8-8d18-3c66c694682e-httpd-config\") pod \"f83ec921-2a86-43a8-8d18-3c66c694682e\" (UID: \"f83ec921-2a86-43a8-8d18-3c66c694682e\") " Dec 03 12:35:43 crc kubenswrapper[5052]: I1203 12:35:43.450905 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f83ec921-2a86-43a8-8d18-3c66c694682e-combined-ca-bundle\") pod \"f83ec921-2a86-43a8-8d18-3c66c694682e\" (UID: \"f83ec921-2a86-43a8-8d18-3c66c694682e\") " Dec 03 12:35:43 crc kubenswrapper[5052]: I1203 12:35:43.450960 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/f83ec921-2a86-43a8-8d18-3c66c694682e-ovndb-tls-certs\") pod \"f83ec921-2a86-43a8-8d18-3c66c694682e\" (UID: \"f83ec921-2a86-43a8-8d18-3c66c694682e\") " Dec 03 12:35:43 crc kubenswrapper[5052]: I1203 12:35:43.451014 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bl86d\" (UniqueName: \"kubernetes.io/projected/f83ec921-2a86-43a8-8d18-3c66c694682e-kube-api-access-bl86d\") pod \"f83ec921-2a86-43a8-8d18-3c66c694682e\" (UID: \"f83ec921-2a86-43a8-8d18-3c66c694682e\") " Dec 03 12:35:43 crc kubenswrapper[5052]: I1203 12:35:43.451087 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/f83ec921-2a86-43a8-8d18-3c66c694682e-config\") pod \"f83ec921-2a86-43a8-8d18-3c66c694682e\" (UID: \"f83ec921-2a86-43a8-8d18-3c66c694682e\") " Dec 03 12:35:43 crc kubenswrapper[5052]: I1203 12:35:43.457795 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f83ec921-2a86-43a8-8d18-3c66c694682e-kube-api-access-bl86d" (OuterVolumeSpecName: "kube-api-access-bl86d") pod "f83ec921-2a86-43a8-8d18-3c66c694682e" (UID: "f83ec921-2a86-43a8-8d18-3c66c694682e"). InnerVolumeSpecName "kube-api-access-bl86d". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:35:43 crc kubenswrapper[5052]: I1203 12:35:43.458486 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f83ec921-2a86-43a8-8d18-3c66c694682e-httpd-config" (OuterVolumeSpecName: "httpd-config") pod "f83ec921-2a86-43a8-8d18-3c66c694682e" (UID: "f83ec921-2a86-43a8-8d18-3c66c694682e"). InnerVolumeSpecName "httpd-config". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:35:43 crc kubenswrapper[5052]: I1203 12:35:43.498544 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f83ec921-2a86-43a8-8d18-3c66c694682e-config" (OuterVolumeSpecName: "config") pod "f83ec921-2a86-43a8-8d18-3c66c694682e" (UID: "f83ec921-2a86-43a8-8d18-3c66c694682e"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:35:43 crc kubenswrapper[5052]: I1203 12:35:43.513555 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f83ec921-2a86-43a8-8d18-3c66c694682e-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "f83ec921-2a86-43a8-8d18-3c66c694682e" (UID: "f83ec921-2a86-43a8-8d18-3c66c694682e"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:35:43 crc kubenswrapper[5052]: I1203 12:35:43.530437 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f83ec921-2a86-43a8-8d18-3c66c694682e-ovndb-tls-certs" (OuterVolumeSpecName: "ovndb-tls-certs") pod "f83ec921-2a86-43a8-8d18-3c66c694682e" (UID: "f83ec921-2a86-43a8-8d18-3c66c694682e"). InnerVolumeSpecName "ovndb-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:35:43 crc kubenswrapper[5052]: I1203 12:35:43.552906 5052 reconciler_common.go:293] "Volume detached for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/f83ec921-2a86-43a8-8d18-3c66c694682e-httpd-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:35:43 crc kubenswrapper[5052]: I1203 12:35:43.552937 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f83ec921-2a86-43a8-8d18-3c66c694682e-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:35:43 crc kubenswrapper[5052]: I1203 12:35:43.552951 5052 reconciler_common.go:293] "Volume detached for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/f83ec921-2a86-43a8-8d18-3c66c694682e-ovndb-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:35:43 crc kubenswrapper[5052]: I1203 12:35:43.552961 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-bl86d\" (UniqueName: \"kubernetes.io/projected/f83ec921-2a86-43a8-8d18-3c66c694682e-kube-api-access-bl86d\") on node \"crc\" DevicePath \"\"" Dec 03 12:35:43 crc kubenswrapper[5052]: I1203 12:35:43.552972 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/f83ec921-2a86-43a8-8d18-3c66c694682e-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:35:43 crc kubenswrapper[5052]: I1203 12:35:43.778223 5052 generic.go:334] "Generic (PLEG): container finished" podID="f83ec921-2a86-43a8-8d18-3c66c694682e" containerID="4aac3f358c0e898b24af2679d198e356f499489d60cbf8a19caaf910f13f9de4" exitCode=0 Dec 03 12:35:43 crc kubenswrapper[5052]: I1203 12:35:43.778295 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-9464b48df-szjp9" event={"ID":"f83ec921-2a86-43a8-8d18-3c66c694682e","Type":"ContainerDied","Data":"4aac3f358c0e898b24af2679d198e356f499489d60cbf8a19caaf910f13f9de4"} Dec 03 12:35:43 crc kubenswrapper[5052]: I1203 12:35:43.778323 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-9464b48df-szjp9" event={"ID":"f83ec921-2a86-43a8-8d18-3c66c694682e","Type":"ContainerDied","Data":"0603b91b3658322a1469c7c9070914904c3e2a3e6f752ab6f1357bb2fe5a6c46"} Dec 03 12:35:43 crc kubenswrapper[5052]: I1203 12:35:43.778338 5052 scope.go:117] "RemoveContainer" containerID="96457f722879e7f4b70f63c2fb15496d232c98398206c25a1f690ebf24ca728b" Dec 03 12:35:43 crc kubenswrapper[5052]: I1203 12:35:43.778659 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-9464b48df-szjp9" Dec 03 12:35:43 crc kubenswrapper[5052]: I1203 12:35:43.784540 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"86ddb247-3041-44d5-aa85-dad99d370fff","Type":"ContainerStarted","Data":"b71c247170e563f1d81a4f881000a75630545bdb5e9efdc0aa6fea1d65d71767"} Dec 03 12:35:43 crc kubenswrapper[5052]: I1203 12:35:43.828353 5052 scope.go:117] "RemoveContainer" containerID="4aac3f358c0e898b24af2679d198e356f499489d60cbf8a19caaf910f13f9de4" Dec 03 12:35:43 crc kubenswrapper[5052]: I1203 12:35:43.841487 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/neutron-9464b48df-szjp9"] Dec 03 12:35:43 crc kubenswrapper[5052]: I1203 12:35:43.850817 5052 scope.go:117] "RemoveContainer" containerID="96457f722879e7f4b70f63c2fb15496d232c98398206c25a1f690ebf24ca728b" Dec 03 12:35:43 crc kubenswrapper[5052]: E1203 12:35:43.851256 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"96457f722879e7f4b70f63c2fb15496d232c98398206c25a1f690ebf24ca728b\": container with ID starting with 96457f722879e7f4b70f63c2fb15496d232c98398206c25a1f690ebf24ca728b not found: ID does not exist" containerID="96457f722879e7f4b70f63c2fb15496d232c98398206c25a1f690ebf24ca728b" Dec 03 12:35:43 crc kubenswrapper[5052]: I1203 12:35:43.851288 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"96457f722879e7f4b70f63c2fb15496d232c98398206c25a1f690ebf24ca728b"} err="failed to get container status \"96457f722879e7f4b70f63c2fb15496d232c98398206c25a1f690ebf24ca728b\": rpc error: code = NotFound desc = could not find container \"96457f722879e7f4b70f63c2fb15496d232c98398206c25a1f690ebf24ca728b\": container with ID starting with 96457f722879e7f4b70f63c2fb15496d232c98398206c25a1f690ebf24ca728b not found: ID does not exist" Dec 03 12:35:43 crc kubenswrapper[5052]: I1203 12:35:43.851308 5052 scope.go:117] "RemoveContainer" containerID="4aac3f358c0e898b24af2679d198e356f499489d60cbf8a19caaf910f13f9de4" Dec 03 12:35:43 crc kubenswrapper[5052]: E1203 12:35:43.851639 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4aac3f358c0e898b24af2679d198e356f499489d60cbf8a19caaf910f13f9de4\": container with ID starting with 4aac3f358c0e898b24af2679d198e356f499489d60cbf8a19caaf910f13f9de4 not found: ID does not exist" containerID="4aac3f358c0e898b24af2679d198e356f499489d60cbf8a19caaf910f13f9de4" Dec 03 12:35:43 crc kubenswrapper[5052]: I1203 12:35:43.851660 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4aac3f358c0e898b24af2679d198e356f499489d60cbf8a19caaf910f13f9de4"} err="failed to get container status \"4aac3f358c0e898b24af2679d198e356f499489d60cbf8a19caaf910f13f9de4\": rpc error: code = NotFound desc = could not find container \"4aac3f358c0e898b24af2679d198e356f499489d60cbf8a19caaf910f13f9de4\": container with ID starting with 4aac3f358c0e898b24af2679d198e356f499489d60cbf8a19caaf910f13f9de4 not found: ID does not exist" Dec 03 12:35:43 crc kubenswrapper[5052]: I1203 12:35:43.860046 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/neutron-9464b48df-szjp9"] Dec 03 12:35:44 crc kubenswrapper[5052]: I1203 12:35:44.809604 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"86ddb247-3041-44d5-aa85-dad99d370fff","Type":"ContainerStarted","Data":"774f900f1eb0e5123e1a20db782a46c7c22f7e16f06e33df311cc81243d73d99"} Dec 03 12:35:44 crc kubenswrapper[5052]: I1203 12:35:44.809918 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:35:44 crc kubenswrapper[5052]: I1203 12:35:44.846887 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/ceilometer-0" podStartSLOduration=1.391413089 podStartE2EDuration="6.846863355s" podCreationTimestamp="2025-12-03 12:35:38 +0000 UTC" firstStartedPulling="2025-12-03 12:35:38.840745681 +0000 UTC m=+4529.428987445" lastFinishedPulling="2025-12-03 12:35:44.296195947 +0000 UTC m=+4534.884437711" observedRunningTime="2025-12-03 12:35:44.835582742 +0000 UTC m=+4535.423824516" watchObservedRunningTime="2025-12-03 12:35:44.846863355 +0000 UTC m=+4535.435105129" Dec 03 12:35:45 crc kubenswrapper[5052]: I1203 12:35:45.684039 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f83ec921-2a86-43a8-8d18-3c66c694682e" path="/var/lib/kubelet/pods/f83ec921-2a86-43a8-8d18-3c66c694682e/volumes" Dec 03 12:35:47 crc kubenswrapper[5052]: I1203 12:35:47.677294 5052 scope.go:117] "RemoveContainer" containerID="e6edaf188f85ea4f11f487f11cc489c700f25fb42b5ee269f4f496f1e127d772" Dec 03 12:35:47 crc kubenswrapper[5052]: E1203 12:35:47.677944 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 12:35:48 crc kubenswrapper[5052]: I1203 12:35:48.754895 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:35:48 crc kubenswrapper[5052]: I1203 12:35:48.755264 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="86ddb247-3041-44d5-aa85-dad99d370fff" containerName="ceilometer-central-agent" containerID="cri-o://9ee7a5065be7f5713680e21754e8ad61a263d7c54861980ccdf4094d374d1af1" gracePeriod=30 Dec 03 12:35:48 crc kubenswrapper[5052]: I1203 12:35:48.755306 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="86ddb247-3041-44d5-aa85-dad99d370fff" containerName="proxy-httpd" containerID="cri-o://774f900f1eb0e5123e1a20db782a46c7c22f7e16f06e33df311cc81243d73d99" gracePeriod=30 Dec 03 12:35:48 crc kubenswrapper[5052]: I1203 12:35:48.755341 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="86ddb247-3041-44d5-aa85-dad99d370fff" containerName="sg-core" containerID="cri-o://b71c247170e563f1d81a4f881000a75630545bdb5e9efdc0aa6fea1d65d71767" gracePeriod=30 Dec 03 12:35:48 crc kubenswrapper[5052]: I1203 12:35:48.755377 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="86ddb247-3041-44d5-aa85-dad99d370fff" containerName="ceilometer-notification-agent" containerID="cri-o://4559674fd0ebae4b6c6a91c9926fba76caa1021c14f8396d3aee06b7d0209af6" gracePeriod=30 Dec 03 12:35:49 crc kubenswrapper[5052]: I1203 12:35:49.800153 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:35:49 crc kubenswrapper[5052]: I1203 12:35:49.860383 5052 generic.go:334] "Generic (PLEG): container finished" podID="86ddb247-3041-44d5-aa85-dad99d370fff" containerID="774f900f1eb0e5123e1a20db782a46c7c22f7e16f06e33df311cc81243d73d99" exitCode=0 Dec 03 12:35:49 crc kubenswrapper[5052]: I1203 12:35:49.860418 5052 generic.go:334] "Generic (PLEG): container finished" podID="86ddb247-3041-44d5-aa85-dad99d370fff" containerID="b71c247170e563f1d81a4f881000a75630545bdb5e9efdc0aa6fea1d65d71767" exitCode=2 Dec 03 12:35:49 crc kubenswrapper[5052]: I1203 12:35:49.860428 5052 generic.go:334] "Generic (PLEG): container finished" podID="86ddb247-3041-44d5-aa85-dad99d370fff" containerID="4559674fd0ebae4b6c6a91c9926fba76caa1021c14f8396d3aee06b7d0209af6" exitCode=0 Dec 03 12:35:49 crc kubenswrapper[5052]: I1203 12:35:49.860438 5052 generic.go:334] "Generic (PLEG): container finished" podID="86ddb247-3041-44d5-aa85-dad99d370fff" containerID="9ee7a5065be7f5713680e21754e8ad61a263d7c54861980ccdf4094d374d1af1" exitCode=0 Dec 03 12:35:49 crc kubenswrapper[5052]: I1203 12:35:49.860451 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:35:49 crc kubenswrapper[5052]: I1203 12:35:49.860467 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"86ddb247-3041-44d5-aa85-dad99d370fff","Type":"ContainerDied","Data":"774f900f1eb0e5123e1a20db782a46c7c22f7e16f06e33df311cc81243d73d99"} Dec 03 12:35:49 crc kubenswrapper[5052]: I1203 12:35:49.860528 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"86ddb247-3041-44d5-aa85-dad99d370fff","Type":"ContainerDied","Data":"b71c247170e563f1d81a4f881000a75630545bdb5e9efdc0aa6fea1d65d71767"} Dec 03 12:35:49 crc kubenswrapper[5052]: I1203 12:35:49.860543 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"86ddb247-3041-44d5-aa85-dad99d370fff","Type":"ContainerDied","Data":"4559674fd0ebae4b6c6a91c9926fba76caa1021c14f8396d3aee06b7d0209af6"} Dec 03 12:35:49 crc kubenswrapper[5052]: I1203 12:35:49.860555 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"86ddb247-3041-44d5-aa85-dad99d370fff","Type":"ContainerDied","Data":"9ee7a5065be7f5713680e21754e8ad61a263d7c54861980ccdf4094d374d1af1"} Dec 03 12:35:49 crc kubenswrapper[5052]: I1203 12:35:49.860565 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"86ddb247-3041-44d5-aa85-dad99d370fff","Type":"ContainerDied","Data":"f9619b85e60754f9d7e9ed84fb60cd8a8a719b345930168ee4192840af22b96c"} Dec 03 12:35:49 crc kubenswrapper[5052]: I1203 12:35:49.860584 5052 scope.go:117] "RemoveContainer" containerID="774f900f1eb0e5123e1a20db782a46c7c22f7e16f06e33df311cc81243d73d99" Dec 03 12:35:49 crc kubenswrapper[5052]: I1203 12:35:49.918650 5052 scope.go:117] "RemoveContainer" containerID="b71c247170e563f1d81a4f881000a75630545bdb5e9efdc0aa6fea1d65d71767" Dec 03 12:35:49 crc kubenswrapper[5052]: I1203 12:35:49.936674 5052 scope.go:117] "RemoveContainer" containerID="4559674fd0ebae4b6c6a91c9926fba76caa1021c14f8396d3aee06b7d0209af6" Dec 03 12:35:49 crc kubenswrapper[5052]: I1203 12:35:49.960185 5052 scope.go:117] "RemoveContainer" containerID="9ee7a5065be7f5713680e21754e8ad61a263d7c54861980ccdf4094d374d1af1" Dec 03 12:35:49 crc kubenswrapper[5052]: I1203 12:35:49.970748 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xkqvz\" (UniqueName: \"kubernetes.io/projected/86ddb247-3041-44d5-aa85-dad99d370fff-kube-api-access-xkqvz\") pod \"86ddb247-3041-44d5-aa85-dad99d370fff\" (UID: \"86ddb247-3041-44d5-aa85-dad99d370fff\") " Dec 03 12:35:49 crc kubenswrapper[5052]: I1203 12:35:49.970802 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/86ddb247-3041-44d5-aa85-dad99d370fff-combined-ca-bundle\") pod \"86ddb247-3041-44d5-aa85-dad99d370fff\" (UID: \"86ddb247-3041-44d5-aa85-dad99d370fff\") " Dec 03 12:35:49 crc kubenswrapper[5052]: I1203 12:35:49.970835 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/86ddb247-3041-44d5-aa85-dad99d370fff-log-httpd\") pod \"86ddb247-3041-44d5-aa85-dad99d370fff\" (UID: \"86ddb247-3041-44d5-aa85-dad99d370fff\") " Dec 03 12:35:49 crc kubenswrapper[5052]: I1203 12:35:49.970862 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/86ddb247-3041-44d5-aa85-dad99d370fff-scripts\") pod \"86ddb247-3041-44d5-aa85-dad99d370fff\" (UID: \"86ddb247-3041-44d5-aa85-dad99d370fff\") " Dec 03 12:35:49 crc kubenswrapper[5052]: I1203 12:35:49.970911 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/86ddb247-3041-44d5-aa85-dad99d370fff-config-data\") pod \"86ddb247-3041-44d5-aa85-dad99d370fff\" (UID: \"86ddb247-3041-44d5-aa85-dad99d370fff\") " Dec 03 12:35:49 crc kubenswrapper[5052]: I1203 12:35:49.970958 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/86ddb247-3041-44d5-aa85-dad99d370fff-sg-core-conf-yaml\") pod \"86ddb247-3041-44d5-aa85-dad99d370fff\" (UID: \"86ddb247-3041-44d5-aa85-dad99d370fff\") " Dec 03 12:35:49 crc kubenswrapper[5052]: I1203 12:35:49.971026 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/86ddb247-3041-44d5-aa85-dad99d370fff-run-httpd\") pod \"86ddb247-3041-44d5-aa85-dad99d370fff\" (UID: \"86ddb247-3041-44d5-aa85-dad99d370fff\") " Dec 03 12:35:49 crc kubenswrapper[5052]: I1203 12:35:49.971711 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/86ddb247-3041-44d5-aa85-dad99d370fff-run-httpd" (OuterVolumeSpecName: "run-httpd") pod "86ddb247-3041-44d5-aa85-dad99d370fff" (UID: "86ddb247-3041-44d5-aa85-dad99d370fff"). InnerVolumeSpecName "run-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:35:49 crc kubenswrapper[5052]: I1203 12:35:49.972444 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/86ddb247-3041-44d5-aa85-dad99d370fff-log-httpd" (OuterVolumeSpecName: "log-httpd") pod "86ddb247-3041-44d5-aa85-dad99d370fff" (UID: "86ddb247-3041-44d5-aa85-dad99d370fff"). InnerVolumeSpecName "log-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:35:49 crc kubenswrapper[5052]: I1203 12:35:49.987775 5052 scope.go:117] "RemoveContainer" containerID="774f900f1eb0e5123e1a20db782a46c7c22f7e16f06e33df311cc81243d73d99" Dec 03 12:35:49 crc kubenswrapper[5052]: E1203 12:35:49.988138 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"774f900f1eb0e5123e1a20db782a46c7c22f7e16f06e33df311cc81243d73d99\": container with ID starting with 774f900f1eb0e5123e1a20db782a46c7c22f7e16f06e33df311cc81243d73d99 not found: ID does not exist" containerID="774f900f1eb0e5123e1a20db782a46c7c22f7e16f06e33df311cc81243d73d99" Dec 03 12:35:49 crc kubenswrapper[5052]: I1203 12:35:49.988166 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"774f900f1eb0e5123e1a20db782a46c7c22f7e16f06e33df311cc81243d73d99"} err="failed to get container status \"774f900f1eb0e5123e1a20db782a46c7c22f7e16f06e33df311cc81243d73d99\": rpc error: code = NotFound desc = could not find container \"774f900f1eb0e5123e1a20db782a46c7c22f7e16f06e33df311cc81243d73d99\": container with ID starting with 774f900f1eb0e5123e1a20db782a46c7c22f7e16f06e33df311cc81243d73d99 not found: ID does not exist" Dec 03 12:35:49 crc kubenswrapper[5052]: I1203 12:35:49.988188 5052 scope.go:117] "RemoveContainer" containerID="b71c247170e563f1d81a4f881000a75630545bdb5e9efdc0aa6fea1d65d71767" Dec 03 12:35:49 crc kubenswrapper[5052]: E1203 12:35:49.988376 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b71c247170e563f1d81a4f881000a75630545bdb5e9efdc0aa6fea1d65d71767\": container with ID starting with b71c247170e563f1d81a4f881000a75630545bdb5e9efdc0aa6fea1d65d71767 not found: ID does not exist" containerID="b71c247170e563f1d81a4f881000a75630545bdb5e9efdc0aa6fea1d65d71767" Dec 03 12:35:49 crc kubenswrapper[5052]: I1203 12:35:49.988395 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b71c247170e563f1d81a4f881000a75630545bdb5e9efdc0aa6fea1d65d71767"} err="failed to get container status \"b71c247170e563f1d81a4f881000a75630545bdb5e9efdc0aa6fea1d65d71767\": rpc error: code = NotFound desc = could not find container \"b71c247170e563f1d81a4f881000a75630545bdb5e9efdc0aa6fea1d65d71767\": container with ID starting with b71c247170e563f1d81a4f881000a75630545bdb5e9efdc0aa6fea1d65d71767 not found: ID does not exist" Dec 03 12:35:49 crc kubenswrapper[5052]: I1203 12:35:49.988406 5052 scope.go:117] "RemoveContainer" containerID="4559674fd0ebae4b6c6a91c9926fba76caa1021c14f8396d3aee06b7d0209af6" Dec 03 12:35:49 crc kubenswrapper[5052]: E1203 12:35:49.988602 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4559674fd0ebae4b6c6a91c9926fba76caa1021c14f8396d3aee06b7d0209af6\": container with ID starting with 4559674fd0ebae4b6c6a91c9926fba76caa1021c14f8396d3aee06b7d0209af6 not found: ID does not exist" containerID="4559674fd0ebae4b6c6a91c9926fba76caa1021c14f8396d3aee06b7d0209af6" Dec 03 12:35:49 crc kubenswrapper[5052]: I1203 12:35:49.988627 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4559674fd0ebae4b6c6a91c9926fba76caa1021c14f8396d3aee06b7d0209af6"} err="failed to get container status \"4559674fd0ebae4b6c6a91c9926fba76caa1021c14f8396d3aee06b7d0209af6\": rpc error: code = NotFound desc = could not find container \"4559674fd0ebae4b6c6a91c9926fba76caa1021c14f8396d3aee06b7d0209af6\": container with ID starting with 4559674fd0ebae4b6c6a91c9926fba76caa1021c14f8396d3aee06b7d0209af6 not found: ID does not exist" Dec 03 12:35:49 crc kubenswrapper[5052]: I1203 12:35:49.988639 5052 scope.go:117] "RemoveContainer" containerID="9ee7a5065be7f5713680e21754e8ad61a263d7c54861980ccdf4094d374d1af1" Dec 03 12:35:49 crc kubenswrapper[5052]: I1203 12:35:49.988743 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/86ddb247-3041-44d5-aa85-dad99d370fff-scripts" (OuterVolumeSpecName: "scripts") pod "86ddb247-3041-44d5-aa85-dad99d370fff" (UID: "86ddb247-3041-44d5-aa85-dad99d370fff"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:35:49 crc kubenswrapper[5052]: E1203 12:35:49.989021 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9ee7a5065be7f5713680e21754e8ad61a263d7c54861980ccdf4094d374d1af1\": container with ID starting with 9ee7a5065be7f5713680e21754e8ad61a263d7c54861980ccdf4094d374d1af1 not found: ID does not exist" containerID="9ee7a5065be7f5713680e21754e8ad61a263d7c54861980ccdf4094d374d1af1" Dec 03 12:35:49 crc kubenswrapper[5052]: I1203 12:35:49.989043 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9ee7a5065be7f5713680e21754e8ad61a263d7c54861980ccdf4094d374d1af1"} err="failed to get container status \"9ee7a5065be7f5713680e21754e8ad61a263d7c54861980ccdf4094d374d1af1\": rpc error: code = NotFound desc = could not find container \"9ee7a5065be7f5713680e21754e8ad61a263d7c54861980ccdf4094d374d1af1\": container with ID starting with 9ee7a5065be7f5713680e21754e8ad61a263d7c54861980ccdf4094d374d1af1 not found: ID does not exist" Dec 03 12:35:49 crc kubenswrapper[5052]: I1203 12:35:49.989056 5052 scope.go:117] "RemoveContainer" containerID="774f900f1eb0e5123e1a20db782a46c7c22f7e16f06e33df311cc81243d73d99" Dec 03 12:35:49 crc kubenswrapper[5052]: I1203 12:35:49.989347 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"774f900f1eb0e5123e1a20db782a46c7c22f7e16f06e33df311cc81243d73d99"} err="failed to get container status \"774f900f1eb0e5123e1a20db782a46c7c22f7e16f06e33df311cc81243d73d99\": rpc error: code = NotFound desc = could not find container \"774f900f1eb0e5123e1a20db782a46c7c22f7e16f06e33df311cc81243d73d99\": container with ID starting with 774f900f1eb0e5123e1a20db782a46c7c22f7e16f06e33df311cc81243d73d99 not found: ID does not exist" Dec 03 12:35:49 crc kubenswrapper[5052]: I1203 12:35:49.989390 5052 scope.go:117] "RemoveContainer" containerID="b71c247170e563f1d81a4f881000a75630545bdb5e9efdc0aa6fea1d65d71767" Dec 03 12:35:49 crc kubenswrapper[5052]: I1203 12:35:49.989806 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b71c247170e563f1d81a4f881000a75630545bdb5e9efdc0aa6fea1d65d71767"} err="failed to get container status \"b71c247170e563f1d81a4f881000a75630545bdb5e9efdc0aa6fea1d65d71767\": rpc error: code = NotFound desc = could not find container \"b71c247170e563f1d81a4f881000a75630545bdb5e9efdc0aa6fea1d65d71767\": container with ID starting with b71c247170e563f1d81a4f881000a75630545bdb5e9efdc0aa6fea1d65d71767 not found: ID does not exist" Dec 03 12:35:49 crc kubenswrapper[5052]: I1203 12:35:49.989833 5052 scope.go:117] "RemoveContainer" containerID="4559674fd0ebae4b6c6a91c9926fba76caa1021c14f8396d3aee06b7d0209af6" Dec 03 12:35:49 crc kubenswrapper[5052]: I1203 12:35:49.990104 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4559674fd0ebae4b6c6a91c9926fba76caa1021c14f8396d3aee06b7d0209af6"} err="failed to get container status \"4559674fd0ebae4b6c6a91c9926fba76caa1021c14f8396d3aee06b7d0209af6\": rpc error: code = NotFound desc = could not find container \"4559674fd0ebae4b6c6a91c9926fba76caa1021c14f8396d3aee06b7d0209af6\": container with ID starting with 4559674fd0ebae4b6c6a91c9926fba76caa1021c14f8396d3aee06b7d0209af6 not found: ID does not exist" Dec 03 12:35:49 crc kubenswrapper[5052]: I1203 12:35:49.990136 5052 scope.go:117] "RemoveContainer" containerID="9ee7a5065be7f5713680e21754e8ad61a263d7c54861980ccdf4094d374d1af1" Dec 03 12:35:49 crc kubenswrapper[5052]: I1203 12:35:49.990516 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9ee7a5065be7f5713680e21754e8ad61a263d7c54861980ccdf4094d374d1af1"} err="failed to get container status \"9ee7a5065be7f5713680e21754e8ad61a263d7c54861980ccdf4094d374d1af1\": rpc error: code = NotFound desc = could not find container \"9ee7a5065be7f5713680e21754e8ad61a263d7c54861980ccdf4094d374d1af1\": container with ID starting with 9ee7a5065be7f5713680e21754e8ad61a263d7c54861980ccdf4094d374d1af1 not found: ID does not exist" Dec 03 12:35:49 crc kubenswrapper[5052]: I1203 12:35:49.990546 5052 scope.go:117] "RemoveContainer" containerID="774f900f1eb0e5123e1a20db782a46c7c22f7e16f06e33df311cc81243d73d99" Dec 03 12:35:49 crc kubenswrapper[5052]: I1203 12:35:49.990785 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"774f900f1eb0e5123e1a20db782a46c7c22f7e16f06e33df311cc81243d73d99"} err="failed to get container status \"774f900f1eb0e5123e1a20db782a46c7c22f7e16f06e33df311cc81243d73d99\": rpc error: code = NotFound desc = could not find container \"774f900f1eb0e5123e1a20db782a46c7c22f7e16f06e33df311cc81243d73d99\": container with ID starting with 774f900f1eb0e5123e1a20db782a46c7c22f7e16f06e33df311cc81243d73d99 not found: ID does not exist" Dec 03 12:35:49 crc kubenswrapper[5052]: I1203 12:35:49.990808 5052 scope.go:117] "RemoveContainer" containerID="b71c247170e563f1d81a4f881000a75630545bdb5e9efdc0aa6fea1d65d71767" Dec 03 12:35:49 crc kubenswrapper[5052]: I1203 12:35:49.990941 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/86ddb247-3041-44d5-aa85-dad99d370fff-kube-api-access-xkqvz" (OuterVolumeSpecName: "kube-api-access-xkqvz") pod "86ddb247-3041-44d5-aa85-dad99d370fff" (UID: "86ddb247-3041-44d5-aa85-dad99d370fff"). InnerVolumeSpecName "kube-api-access-xkqvz". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:35:49 crc kubenswrapper[5052]: I1203 12:35:49.994736 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b71c247170e563f1d81a4f881000a75630545bdb5e9efdc0aa6fea1d65d71767"} err="failed to get container status \"b71c247170e563f1d81a4f881000a75630545bdb5e9efdc0aa6fea1d65d71767\": rpc error: code = NotFound desc = could not find container \"b71c247170e563f1d81a4f881000a75630545bdb5e9efdc0aa6fea1d65d71767\": container with ID starting with b71c247170e563f1d81a4f881000a75630545bdb5e9efdc0aa6fea1d65d71767 not found: ID does not exist" Dec 03 12:35:49 crc kubenswrapper[5052]: I1203 12:35:49.994783 5052 scope.go:117] "RemoveContainer" containerID="4559674fd0ebae4b6c6a91c9926fba76caa1021c14f8396d3aee06b7d0209af6" Dec 03 12:35:49 crc kubenswrapper[5052]: I1203 12:35:49.999763 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4559674fd0ebae4b6c6a91c9926fba76caa1021c14f8396d3aee06b7d0209af6"} err="failed to get container status \"4559674fd0ebae4b6c6a91c9926fba76caa1021c14f8396d3aee06b7d0209af6\": rpc error: code = NotFound desc = could not find container \"4559674fd0ebae4b6c6a91c9926fba76caa1021c14f8396d3aee06b7d0209af6\": container with ID starting with 4559674fd0ebae4b6c6a91c9926fba76caa1021c14f8396d3aee06b7d0209af6 not found: ID does not exist" Dec 03 12:35:49 crc kubenswrapper[5052]: I1203 12:35:49.999805 5052 scope.go:117] "RemoveContainer" containerID="9ee7a5065be7f5713680e21754e8ad61a263d7c54861980ccdf4094d374d1af1" Dec 03 12:35:50 crc kubenswrapper[5052]: I1203 12:35:50.000624 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9ee7a5065be7f5713680e21754e8ad61a263d7c54861980ccdf4094d374d1af1"} err="failed to get container status \"9ee7a5065be7f5713680e21754e8ad61a263d7c54861980ccdf4094d374d1af1\": rpc error: code = NotFound desc = could not find container \"9ee7a5065be7f5713680e21754e8ad61a263d7c54861980ccdf4094d374d1af1\": container with ID starting with 9ee7a5065be7f5713680e21754e8ad61a263d7c54861980ccdf4094d374d1af1 not found: ID does not exist" Dec 03 12:35:50 crc kubenswrapper[5052]: I1203 12:35:50.000663 5052 scope.go:117] "RemoveContainer" containerID="774f900f1eb0e5123e1a20db782a46c7c22f7e16f06e33df311cc81243d73d99" Dec 03 12:35:50 crc kubenswrapper[5052]: I1203 12:35:50.000966 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"774f900f1eb0e5123e1a20db782a46c7c22f7e16f06e33df311cc81243d73d99"} err="failed to get container status \"774f900f1eb0e5123e1a20db782a46c7c22f7e16f06e33df311cc81243d73d99\": rpc error: code = NotFound desc = could not find container \"774f900f1eb0e5123e1a20db782a46c7c22f7e16f06e33df311cc81243d73d99\": container with ID starting with 774f900f1eb0e5123e1a20db782a46c7c22f7e16f06e33df311cc81243d73d99 not found: ID does not exist" Dec 03 12:35:50 crc kubenswrapper[5052]: I1203 12:35:50.000992 5052 scope.go:117] "RemoveContainer" containerID="b71c247170e563f1d81a4f881000a75630545bdb5e9efdc0aa6fea1d65d71767" Dec 03 12:35:50 crc kubenswrapper[5052]: I1203 12:35:50.001638 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b71c247170e563f1d81a4f881000a75630545bdb5e9efdc0aa6fea1d65d71767"} err="failed to get container status \"b71c247170e563f1d81a4f881000a75630545bdb5e9efdc0aa6fea1d65d71767\": rpc error: code = NotFound desc = could not find container \"b71c247170e563f1d81a4f881000a75630545bdb5e9efdc0aa6fea1d65d71767\": container with ID starting with b71c247170e563f1d81a4f881000a75630545bdb5e9efdc0aa6fea1d65d71767 not found: ID does not exist" Dec 03 12:35:50 crc kubenswrapper[5052]: I1203 12:35:50.001800 5052 scope.go:117] "RemoveContainer" containerID="4559674fd0ebae4b6c6a91c9926fba76caa1021c14f8396d3aee06b7d0209af6" Dec 03 12:35:50 crc kubenswrapper[5052]: I1203 12:35:50.002231 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4559674fd0ebae4b6c6a91c9926fba76caa1021c14f8396d3aee06b7d0209af6"} err="failed to get container status \"4559674fd0ebae4b6c6a91c9926fba76caa1021c14f8396d3aee06b7d0209af6\": rpc error: code = NotFound desc = could not find container \"4559674fd0ebae4b6c6a91c9926fba76caa1021c14f8396d3aee06b7d0209af6\": container with ID starting with 4559674fd0ebae4b6c6a91c9926fba76caa1021c14f8396d3aee06b7d0209af6 not found: ID does not exist" Dec 03 12:35:50 crc kubenswrapper[5052]: I1203 12:35:50.002783 5052 scope.go:117] "RemoveContainer" containerID="9ee7a5065be7f5713680e21754e8ad61a263d7c54861980ccdf4094d374d1af1" Dec 03 12:35:50 crc kubenswrapper[5052]: I1203 12:35:50.003209 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/86ddb247-3041-44d5-aa85-dad99d370fff-sg-core-conf-yaml" (OuterVolumeSpecName: "sg-core-conf-yaml") pod "86ddb247-3041-44d5-aa85-dad99d370fff" (UID: "86ddb247-3041-44d5-aa85-dad99d370fff"). InnerVolumeSpecName "sg-core-conf-yaml". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:35:50 crc kubenswrapper[5052]: I1203 12:35:50.003816 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9ee7a5065be7f5713680e21754e8ad61a263d7c54861980ccdf4094d374d1af1"} err="failed to get container status \"9ee7a5065be7f5713680e21754e8ad61a263d7c54861980ccdf4094d374d1af1\": rpc error: code = NotFound desc = could not find container \"9ee7a5065be7f5713680e21754e8ad61a263d7c54861980ccdf4094d374d1af1\": container with ID starting with 9ee7a5065be7f5713680e21754e8ad61a263d7c54861980ccdf4094d374d1af1 not found: ID does not exist" Dec 03 12:35:50 crc kubenswrapper[5052]: I1203 12:35:50.044216 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/86ddb247-3041-44d5-aa85-dad99d370fff-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "86ddb247-3041-44d5-aa85-dad99d370fff" (UID: "86ddb247-3041-44d5-aa85-dad99d370fff"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:35:50 crc kubenswrapper[5052]: I1203 12:35:50.072787 5052 reconciler_common.go:293] "Volume detached for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/86ddb247-3041-44d5-aa85-dad99d370fff-run-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:35:50 crc kubenswrapper[5052]: I1203 12:35:50.072824 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-xkqvz\" (UniqueName: \"kubernetes.io/projected/86ddb247-3041-44d5-aa85-dad99d370fff-kube-api-access-xkqvz\") on node \"crc\" DevicePath \"\"" Dec 03 12:35:50 crc kubenswrapper[5052]: I1203 12:35:50.072834 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/86ddb247-3041-44d5-aa85-dad99d370fff-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:35:50 crc kubenswrapper[5052]: I1203 12:35:50.072844 5052 reconciler_common.go:293] "Volume detached for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/86ddb247-3041-44d5-aa85-dad99d370fff-log-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:35:50 crc kubenswrapper[5052]: I1203 12:35:50.072853 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/86ddb247-3041-44d5-aa85-dad99d370fff-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:35:50 crc kubenswrapper[5052]: I1203 12:35:50.072861 5052 reconciler_common.go:293] "Volume detached for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/86ddb247-3041-44d5-aa85-dad99d370fff-sg-core-conf-yaml\") on node \"crc\" DevicePath \"\"" Dec 03 12:35:50 crc kubenswrapper[5052]: I1203 12:35:50.106959 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/86ddb247-3041-44d5-aa85-dad99d370fff-config-data" (OuterVolumeSpecName: "config-data") pod "86ddb247-3041-44d5-aa85-dad99d370fff" (UID: "86ddb247-3041-44d5-aa85-dad99d370fff"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:35:50 crc kubenswrapper[5052]: I1203 12:35:50.175057 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/86ddb247-3041-44d5-aa85-dad99d370fff-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:35:50 crc kubenswrapper[5052]: I1203 12:35:50.202117 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:35:50 crc kubenswrapper[5052]: I1203 12:35:50.213554 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:35:50 crc kubenswrapper[5052]: I1203 12:35:50.222072 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:35:50 crc kubenswrapper[5052]: E1203 12:35:50.222404 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="f83ec921-2a86-43a8-8d18-3c66c694682e" containerName="neutron-httpd" Dec 03 12:35:50 crc kubenswrapper[5052]: I1203 12:35:50.222422 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="f83ec921-2a86-43a8-8d18-3c66c694682e" containerName="neutron-httpd" Dec 03 12:35:50 crc kubenswrapper[5052]: E1203 12:35:50.222434 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="f83ec921-2a86-43a8-8d18-3c66c694682e" containerName="neutron-api" Dec 03 12:35:50 crc kubenswrapper[5052]: I1203 12:35:50.222439 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="f83ec921-2a86-43a8-8d18-3c66c694682e" containerName="neutron-api" Dec 03 12:35:50 crc kubenswrapper[5052]: E1203 12:35:50.222451 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="86ddb247-3041-44d5-aa85-dad99d370fff" containerName="proxy-httpd" Dec 03 12:35:50 crc kubenswrapper[5052]: I1203 12:35:50.222458 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="86ddb247-3041-44d5-aa85-dad99d370fff" containerName="proxy-httpd" Dec 03 12:35:50 crc kubenswrapper[5052]: E1203 12:35:50.222489 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="86ddb247-3041-44d5-aa85-dad99d370fff" containerName="ceilometer-notification-agent" Dec 03 12:35:50 crc kubenswrapper[5052]: I1203 12:35:50.222495 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="86ddb247-3041-44d5-aa85-dad99d370fff" containerName="ceilometer-notification-agent" Dec 03 12:35:50 crc kubenswrapper[5052]: E1203 12:35:50.222521 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="86ddb247-3041-44d5-aa85-dad99d370fff" containerName="sg-core" Dec 03 12:35:50 crc kubenswrapper[5052]: I1203 12:35:50.222527 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="86ddb247-3041-44d5-aa85-dad99d370fff" containerName="sg-core" Dec 03 12:35:50 crc kubenswrapper[5052]: E1203 12:35:50.222538 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="86ddb247-3041-44d5-aa85-dad99d370fff" containerName="ceilometer-central-agent" Dec 03 12:35:50 crc kubenswrapper[5052]: I1203 12:35:50.222543 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="86ddb247-3041-44d5-aa85-dad99d370fff" containerName="ceilometer-central-agent" Dec 03 12:35:50 crc kubenswrapper[5052]: I1203 12:35:50.222713 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="f83ec921-2a86-43a8-8d18-3c66c694682e" containerName="neutron-api" Dec 03 12:35:50 crc kubenswrapper[5052]: I1203 12:35:50.222732 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="86ddb247-3041-44d5-aa85-dad99d370fff" containerName="ceilometer-notification-agent" Dec 03 12:35:50 crc kubenswrapper[5052]: I1203 12:35:50.222743 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="86ddb247-3041-44d5-aa85-dad99d370fff" containerName="ceilometer-central-agent" Dec 03 12:35:50 crc kubenswrapper[5052]: I1203 12:35:50.222750 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="86ddb247-3041-44d5-aa85-dad99d370fff" containerName="sg-core" Dec 03 12:35:50 crc kubenswrapper[5052]: I1203 12:35:50.222758 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="f83ec921-2a86-43a8-8d18-3c66c694682e" containerName="neutron-httpd" Dec 03 12:35:50 crc kubenswrapper[5052]: I1203 12:35:50.222768 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="86ddb247-3041-44d5-aa85-dad99d370fff" containerName="proxy-httpd" Dec 03 12:35:50 crc kubenswrapper[5052]: I1203 12:35:50.224275 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:35:50 crc kubenswrapper[5052]: I1203 12:35:50.226268 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-scripts" Dec 03 12:35:50 crc kubenswrapper[5052]: I1203 12:35:50.228089 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-config-data" Dec 03 12:35:50 crc kubenswrapper[5052]: I1203 12:35:50.240270 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:35:50 crc kubenswrapper[5052]: I1203 12:35:50.377807 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/083c7d71-e993-4f55-9192-4ec777f861e0-run-httpd\") pod \"ceilometer-0\" (UID: \"083c7d71-e993-4f55-9192-4ec777f861e0\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:35:50 crc kubenswrapper[5052]: I1203 12:35:50.377879 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/083c7d71-e993-4f55-9192-4ec777f861e0-config-data\") pod \"ceilometer-0\" (UID: \"083c7d71-e993-4f55-9192-4ec777f861e0\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:35:50 crc kubenswrapper[5052]: I1203 12:35:50.378208 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/083c7d71-e993-4f55-9192-4ec777f861e0-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"083c7d71-e993-4f55-9192-4ec777f861e0\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:35:50 crc kubenswrapper[5052]: I1203 12:35:50.378250 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kg6jq\" (UniqueName: \"kubernetes.io/projected/083c7d71-e993-4f55-9192-4ec777f861e0-kube-api-access-kg6jq\") pod \"ceilometer-0\" (UID: \"083c7d71-e993-4f55-9192-4ec777f861e0\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:35:50 crc kubenswrapper[5052]: I1203 12:35:50.378269 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/083c7d71-e993-4f55-9192-4ec777f861e0-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"083c7d71-e993-4f55-9192-4ec777f861e0\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:35:50 crc kubenswrapper[5052]: I1203 12:35:50.378290 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/083c7d71-e993-4f55-9192-4ec777f861e0-scripts\") pod \"ceilometer-0\" (UID: \"083c7d71-e993-4f55-9192-4ec777f861e0\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:35:50 crc kubenswrapper[5052]: I1203 12:35:50.378304 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/083c7d71-e993-4f55-9192-4ec777f861e0-log-httpd\") pod \"ceilometer-0\" (UID: \"083c7d71-e993-4f55-9192-4ec777f861e0\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:35:50 crc kubenswrapper[5052]: I1203 12:35:50.479992 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-kg6jq\" (UniqueName: \"kubernetes.io/projected/083c7d71-e993-4f55-9192-4ec777f861e0-kube-api-access-kg6jq\") pod \"ceilometer-0\" (UID: \"083c7d71-e993-4f55-9192-4ec777f861e0\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:35:50 crc kubenswrapper[5052]: I1203 12:35:50.480036 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/083c7d71-e993-4f55-9192-4ec777f861e0-scripts\") pod \"ceilometer-0\" (UID: \"083c7d71-e993-4f55-9192-4ec777f861e0\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:35:50 crc kubenswrapper[5052]: I1203 12:35:50.480064 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/083c7d71-e993-4f55-9192-4ec777f861e0-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"083c7d71-e993-4f55-9192-4ec777f861e0\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:35:50 crc kubenswrapper[5052]: I1203 12:35:50.480086 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/083c7d71-e993-4f55-9192-4ec777f861e0-log-httpd\") pod \"ceilometer-0\" (UID: \"083c7d71-e993-4f55-9192-4ec777f861e0\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:35:50 crc kubenswrapper[5052]: I1203 12:35:50.480142 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/083c7d71-e993-4f55-9192-4ec777f861e0-run-httpd\") pod \"ceilometer-0\" (UID: \"083c7d71-e993-4f55-9192-4ec777f861e0\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:35:50 crc kubenswrapper[5052]: I1203 12:35:50.480181 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/083c7d71-e993-4f55-9192-4ec777f861e0-config-data\") pod \"ceilometer-0\" (UID: \"083c7d71-e993-4f55-9192-4ec777f861e0\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:35:50 crc kubenswrapper[5052]: I1203 12:35:50.480290 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/083c7d71-e993-4f55-9192-4ec777f861e0-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"083c7d71-e993-4f55-9192-4ec777f861e0\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:35:50 crc kubenswrapper[5052]: I1203 12:35:50.480645 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/083c7d71-e993-4f55-9192-4ec777f861e0-log-httpd\") pod \"ceilometer-0\" (UID: \"083c7d71-e993-4f55-9192-4ec777f861e0\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:35:50 crc kubenswrapper[5052]: I1203 12:35:50.480700 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/083c7d71-e993-4f55-9192-4ec777f861e0-run-httpd\") pod \"ceilometer-0\" (UID: \"083c7d71-e993-4f55-9192-4ec777f861e0\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:35:50 crc kubenswrapper[5052]: I1203 12:35:50.483869 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/083c7d71-e993-4f55-9192-4ec777f861e0-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"083c7d71-e993-4f55-9192-4ec777f861e0\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:35:50 crc kubenswrapper[5052]: I1203 12:35:50.484026 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/083c7d71-e993-4f55-9192-4ec777f861e0-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"083c7d71-e993-4f55-9192-4ec777f861e0\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:35:50 crc kubenswrapper[5052]: I1203 12:35:50.484077 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/083c7d71-e993-4f55-9192-4ec777f861e0-scripts\") pod \"ceilometer-0\" (UID: \"083c7d71-e993-4f55-9192-4ec777f861e0\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:35:50 crc kubenswrapper[5052]: I1203 12:35:50.484350 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/083c7d71-e993-4f55-9192-4ec777f861e0-config-data\") pod \"ceilometer-0\" (UID: \"083c7d71-e993-4f55-9192-4ec777f861e0\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:35:50 crc kubenswrapper[5052]: I1203 12:35:50.499293 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-kg6jq\" (UniqueName: \"kubernetes.io/projected/083c7d71-e993-4f55-9192-4ec777f861e0-kube-api-access-kg6jq\") pod \"ceilometer-0\" (UID: \"083c7d71-e993-4f55-9192-4ec777f861e0\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:35:50 crc kubenswrapper[5052]: I1203 12:35:50.540598 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:35:51 crc kubenswrapper[5052]: I1203 12:35:51.011268 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:35:51 crc kubenswrapper[5052]: I1203 12:35:51.402338 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:35:51 crc kubenswrapper[5052]: I1203 12:35:51.684182 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="86ddb247-3041-44d5-aa85-dad99d370fff" path="/var/lib/kubelet/pods/86ddb247-3041-44d5-aa85-dad99d370fff/volumes" Dec 03 12:35:51 crc kubenswrapper[5052]: I1203 12:35:51.882308 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"083c7d71-e993-4f55-9192-4ec777f861e0","Type":"ContainerStarted","Data":"cb557aed42badd5a8e35a87c5d43012c96b278f7e1b453964099db1e2f21204a"} Dec 03 12:35:51 crc kubenswrapper[5052]: I1203 12:35:51.882354 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"083c7d71-e993-4f55-9192-4ec777f861e0","Type":"ContainerStarted","Data":"a2f6c0cf27000f6ee8c784094b3435a973922ac368358efee947496b783d7284"} Dec 03 12:35:52 crc kubenswrapper[5052]: I1203 12:35:52.894162 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"083c7d71-e993-4f55-9192-4ec777f861e0","Type":"ContainerStarted","Data":"c3d595431e6fb1c94db66f3c11a76dc94cb634f25c27480a4930edb720dae2d3"} Dec 03 12:35:53 crc kubenswrapper[5052]: I1203 12:35:53.906558 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"083c7d71-e993-4f55-9192-4ec777f861e0","Type":"ContainerStarted","Data":"a6139bafd67bbba3d8cd83eeb249e20fea6767a144eb504613ee6b7dd06877e9"} Dec 03 12:35:56 crc kubenswrapper[5052]: I1203 12:35:56.951302 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"083c7d71-e993-4f55-9192-4ec777f861e0","Type":"ContainerStarted","Data":"2cbd14b600699bfe1e099da97363c805b86566346724df2d5ac0c9d8b33841f8"} Dec 03 12:35:56 crc kubenswrapper[5052]: I1203 12:35:56.952031 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="083c7d71-e993-4f55-9192-4ec777f861e0" containerName="ceilometer-central-agent" containerID="cri-o://cb557aed42badd5a8e35a87c5d43012c96b278f7e1b453964099db1e2f21204a" gracePeriod=30 Dec 03 12:35:56 crc kubenswrapper[5052]: I1203 12:35:56.952154 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:35:56 crc kubenswrapper[5052]: I1203 12:35:56.952182 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="083c7d71-e993-4f55-9192-4ec777f861e0" containerName="proxy-httpd" containerID="cri-o://2cbd14b600699bfe1e099da97363c805b86566346724df2d5ac0c9d8b33841f8" gracePeriod=30 Dec 03 12:35:56 crc kubenswrapper[5052]: I1203 12:35:56.952239 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="083c7d71-e993-4f55-9192-4ec777f861e0" containerName="sg-core" containerID="cri-o://a6139bafd67bbba3d8cd83eeb249e20fea6767a144eb504613ee6b7dd06877e9" gracePeriod=30 Dec 03 12:35:56 crc kubenswrapper[5052]: I1203 12:35:56.952310 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="083c7d71-e993-4f55-9192-4ec777f861e0" containerName="ceilometer-notification-agent" containerID="cri-o://c3d595431e6fb1c94db66f3c11a76dc94cb634f25c27480a4930edb720dae2d3" gracePeriod=30 Dec 03 12:35:56 crc kubenswrapper[5052]: I1203 12:35:56.979287 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/ceilometer-0" podStartSLOduration=2.4211113060000002 podStartE2EDuration="6.97926112s" podCreationTimestamp="2025-12-03 12:35:50 +0000 UTC" firstStartedPulling="2025-12-03 12:35:51.033330247 +0000 UTC m=+4541.621572011" lastFinishedPulling="2025-12-03 12:35:55.591480021 +0000 UTC m=+4546.179721825" observedRunningTime="2025-12-03 12:35:56.972229849 +0000 UTC m=+4547.560471613" watchObservedRunningTime="2025-12-03 12:35:56.97926112 +0000 UTC m=+4547.567502884" Dec 03 12:35:57 crc kubenswrapper[5052]: I1203 12:35:57.967527 5052 generic.go:334] "Generic (PLEG): container finished" podID="083c7d71-e993-4f55-9192-4ec777f861e0" containerID="2cbd14b600699bfe1e099da97363c805b86566346724df2d5ac0c9d8b33841f8" exitCode=0 Dec 03 12:35:57 crc kubenswrapper[5052]: I1203 12:35:57.968015 5052 generic.go:334] "Generic (PLEG): container finished" podID="083c7d71-e993-4f55-9192-4ec777f861e0" containerID="a6139bafd67bbba3d8cd83eeb249e20fea6767a144eb504613ee6b7dd06877e9" exitCode=2 Dec 03 12:35:57 crc kubenswrapper[5052]: I1203 12:35:57.968023 5052 generic.go:334] "Generic (PLEG): container finished" podID="083c7d71-e993-4f55-9192-4ec777f861e0" containerID="c3d595431e6fb1c94db66f3c11a76dc94cb634f25c27480a4930edb720dae2d3" exitCode=0 Dec 03 12:35:57 crc kubenswrapper[5052]: I1203 12:35:57.968089 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"083c7d71-e993-4f55-9192-4ec777f861e0","Type":"ContainerDied","Data":"2cbd14b600699bfe1e099da97363c805b86566346724df2d5ac0c9d8b33841f8"} Dec 03 12:35:57 crc kubenswrapper[5052]: I1203 12:35:57.968116 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"083c7d71-e993-4f55-9192-4ec777f861e0","Type":"ContainerDied","Data":"a6139bafd67bbba3d8cd83eeb249e20fea6767a144eb504613ee6b7dd06877e9"} Dec 03 12:35:57 crc kubenswrapper[5052]: I1203 12:35:57.968127 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"083c7d71-e993-4f55-9192-4ec777f861e0","Type":"ContainerDied","Data":"c3d595431e6fb1c94db66f3c11a76dc94cb634f25c27480a4930edb720dae2d3"} Dec 03 12:35:57 crc kubenswrapper[5052]: I1203 12:35:57.970482 5052 generic.go:334] "Generic (PLEG): container finished" podID="63821a5d-93bd-41a3-9c7d-979723bfc035" containerID="9330194596eda8836d7765112355fa440ad87fd5443eb2c1506c02b1767075eb" exitCode=0 Dec 03 12:35:57 crc kubenswrapper[5052]: I1203 12:35:57.970525 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-dpcff" event={"ID":"63821a5d-93bd-41a3-9c7d-979723bfc035","Type":"ContainerDied","Data":"9330194596eda8836d7765112355fa440ad87fd5443eb2c1506c02b1767075eb"} Dec 03 12:35:59 crc kubenswrapper[5052]: I1203 12:35:59.335185 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-dpcff" Dec 03 12:35:59 crc kubenswrapper[5052]: I1203 12:35:59.441993 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/63821a5d-93bd-41a3-9c7d-979723bfc035-config-data\") pod \"63821a5d-93bd-41a3-9c7d-979723bfc035\" (UID: \"63821a5d-93bd-41a3-9c7d-979723bfc035\") " Dec 03 12:35:59 crc kubenswrapper[5052]: I1203 12:35:59.442829 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/63821a5d-93bd-41a3-9c7d-979723bfc035-scripts\") pod \"63821a5d-93bd-41a3-9c7d-979723bfc035\" (UID: \"63821a5d-93bd-41a3-9c7d-979723bfc035\") " Dec 03 12:35:59 crc kubenswrapper[5052]: I1203 12:35:59.442882 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/63821a5d-93bd-41a3-9c7d-979723bfc035-combined-ca-bundle\") pod \"63821a5d-93bd-41a3-9c7d-979723bfc035\" (UID: \"63821a5d-93bd-41a3-9c7d-979723bfc035\") " Dec 03 12:35:59 crc kubenswrapper[5052]: I1203 12:35:59.442936 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fcjbh\" (UniqueName: \"kubernetes.io/projected/63821a5d-93bd-41a3-9c7d-979723bfc035-kube-api-access-fcjbh\") pod \"63821a5d-93bd-41a3-9c7d-979723bfc035\" (UID: \"63821a5d-93bd-41a3-9c7d-979723bfc035\") " Dec 03 12:35:59 crc kubenswrapper[5052]: I1203 12:35:59.448494 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/63821a5d-93bd-41a3-9c7d-979723bfc035-scripts" (OuterVolumeSpecName: "scripts") pod "63821a5d-93bd-41a3-9c7d-979723bfc035" (UID: "63821a5d-93bd-41a3-9c7d-979723bfc035"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:35:59 crc kubenswrapper[5052]: I1203 12:35:59.450231 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/63821a5d-93bd-41a3-9c7d-979723bfc035-kube-api-access-fcjbh" (OuterVolumeSpecName: "kube-api-access-fcjbh") pod "63821a5d-93bd-41a3-9c7d-979723bfc035" (UID: "63821a5d-93bd-41a3-9c7d-979723bfc035"). InnerVolumeSpecName "kube-api-access-fcjbh". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:35:59 crc kubenswrapper[5052]: I1203 12:35:59.472471 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/63821a5d-93bd-41a3-9c7d-979723bfc035-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "63821a5d-93bd-41a3-9c7d-979723bfc035" (UID: "63821a5d-93bd-41a3-9c7d-979723bfc035"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:35:59 crc kubenswrapper[5052]: I1203 12:35:59.480482 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/63821a5d-93bd-41a3-9c7d-979723bfc035-config-data" (OuterVolumeSpecName: "config-data") pod "63821a5d-93bd-41a3-9c7d-979723bfc035" (UID: "63821a5d-93bd-41a3-9c7d-979723bfc035"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:35:59 crc kubenswrapper[5052]: I1203 12:35:59.544767 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/63821a5d-93bd-41a3-9c7d-979723bfc035-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:35:59 crc kubenswrapper[5052]: I1203 12:35:59.545054 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/63821a5d-93bd-41a3-9c7d-979723bfc035-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:35:59 crc kubenswrapper[5052]: I1203 12:35:59.545065 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/63821a5d-93bd-41a3-9c7d-979723bfc035-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:35:59 crc kubenswrapper[5052]: I1203 12:35:59.545076 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-fcjbh\" (UniqueName: \"kubernetes.io/projected/63821a5d-93bd-41a3-9c7d-979723bfc035-kube-api-access-fcjbh\") on node \"crc\" DevicePath \"\"" Dec 03 12:35:59 crc kubenswrapper[5052]: I1203 12:35:59.989034 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-dpcff" event={"ID":"63821a5d-93bd-41a3-9c7d-979723bfc035","Type":"ContainerDied","Data":"ae1fbd48e118f8aacf6561ab68e82755a7906595b39f4dd49fd54b325a932883"} Dec 03 12:35:59 crc kubenswrapper[5052]: I1203 12:35:59.989075 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ae1fbd48e118f8aacf6561ab68e82755a7906595b39f4dd49fd54b325a932883" Dec 03 12:35:59 crc kubenswrapper[5052]: I1203 12:35:59.989111 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-dpcff" Dec 03 12:36:00 crc kubenswrapper[5052]: I1203 12:36:00.070328 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-0"] Dec 03 12:36:00 crc kubenswrapper[5052]: E1203 12:36:00.070765 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="63821a5d-93bd-41a3-9c7d-979723bfc035" containerName="nova-cell0-conductor-db-sync" Dec 03 12:36:00 crc kubenswrapper[5052]: I1203 12:36:00.070780 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="63821a5d-93bd-41a3-9c7d-979723bfc035" containerName="nova-cell0-conductor-db-sync" Dec 03 12:36:00 crc kubenswrapper[5052]: I1203 12:36:00.071026 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="63821a5d-93bd-41a3-9c7d-979723bfc035" containerName="nova-cell0-conductor-db-sync" Dec 03 12:36:00 crc kubenswrapper[5052]: I1203 12:36:00.071677 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:36:00 crc kubenswrapper[5052]: I1203 12:36:00.074079 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell0-conductor-config-data" Dec 03 12:36:00 crc kubenswrapper[5052]: I1203 12:36:00.074334 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-nova-dockercfg-t94dq" Dec 03 12:36:00 crc kubenswrapper[5052]: I1203 12:36:00.082502 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-0"] Dec 03 12:36:00 crc kubenswrapper[5052]: I1203 12:36:00.256487 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/1498054d-22e0-426e-b85d-144cf7c4180f-config-data\") pod \"nova-cell0-conductor-0\" (UID: \"1498054d-22e0-426e-b85d-144cf7c4180f\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:36:00 crc kubenswrapper[5052]: I1203 12:36:00.256616 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1498054d-22e0-426e-b85d-144cf7c4180f-combined-ca-bundle\") pod \"nova-cell0-conductor-0\" (UID: \"1498054d-22e0-426e-b85d-144cf7c4180f\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:36:00 crc kubenswrapper[5052]: I1203 12:36:00.256687 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7cxn7\" (UniqueName: \"kubernetes.io/projected/1498054d-22e0-426e-b85d-144cf7c4180f-kube-api-access-7cxn7\") pod \"nova-cell0-conductor-0\" (UID: \"1498054d-22e0-426e-b85d-144cf7c4180f\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:36:00 crc kubenswrapper[5052]: I1203 12:36:00.358025 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1498054d-22e0-426e-b85d-144cf7c4180f-combined-ca-bundle\") pod \"nova-cell0-conductor-0\" (UID: \"1498054d-22e0-426e-b85d-144cf7c4180f\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:36:00 crc kubenswrapper[5052]: I1203 12:36:00.358115 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-7cxn7\" (UniqueName: \"kubernetes.io/projected/1498054d-22e0-426e-b85d-144cf7c4180f-kube-api-access-7cxn7\") pod \"nova-cell0-conductor-0\" (UID: \"1498054d-22e0-426e-b85d-144cf7c4180f\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:36:00 crc kubenswrapper[5052]: I1203 12:36:00.358258 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/1498054d-22e0-426e-b85d-144cf7c4180f-config-data\") pod \"nova-cell0-conductor-0\" (UID: \"1498054d-22e0-426e-b85d-144cf7c4180f\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:36:00 crc kubenswrapper[5052]: I1203 12:36:00.362929 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1498054d-22e0-426e-b85d-144cf7c4180f-combined-ca-bundle\") pod \"nova-cell0-conductor-0\" (UID: \"1498054d-22e0-426e-b85d-144cf7c4180f\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:36:00 crc kubenswrapper[5052]: I1203 12:36:00.363009 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/1498054d-22e0-426e-b85d-144cf7c4180f-config-data\") pod \"nova-cell0-conductor-0\" (UID: \"1498054d-22e0-426e-b85d-144cf7c4180f\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:36:00 crc kubenswrapper[5052]: I1203 12:36:00.373457 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-7cxn7\" (UniqueName: \"kubernetes.io/projected/1498054d-22e0-426e-b85d-144cf7c4180f-kube-api-access-7cxn7\") pod \"nova-cell0-conductor-0\" (UID: \"1498054d-22e0-426e-b85d-144cf7c4180f\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:36:00 crc kubenswrapper[5052]: I1203 12:36:00.401236 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:36:00 crc kubenswrapper[5052]: I1203 12:36:00.849578 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-0"] Dec 03 12:36:00 crc kubenswrapper[5052]: W1203 12:36:00.854201 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1498054d_22e0_426e_b85d_144cf7c4180f.slice/crio-8e7655ebc8c6aaae5798bf385240ef8b2645e57121844e670143a45fbaf97325 WatchSource:0}: Error finding container 8e7655ebc8c6aaae5798bf385240ef8b2645e57121844e670143a45fbaf97325: Status 404 returned error can't find the container with id 8e7655ebc8c6aaae5798bf385240ef8b2645e57121844e670143a45fbaf97325 Dec 03 12:36:00 crc kubenswrapper[5052]: I1203 12:36:00.999829 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-conductor-0" event={"ID":"1498054d-22e0-426e-b85d-144cf7c4180f","Type":"ContainerStarted","Data":"8e7655ebc8c6aaae5798bf385240ef8b2645e57121844e670143a45fbaf97325"} Dec 03 12:36:01 crc kubenswrapper[5052]: I1203 12:36:01.724501 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:36:01 crc kubenswrapper[5052]: I1203 12:36:01.892868 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/083c7d71-e993-4f55-9192-4ec777f861e0-scripts\") pod \"083c7d71-e993-4f55-9192-4ec777f861e0\" (UID: \"083c7d71-e993-4f55-9192-4ec777f861e0\") " Dec 03 12:36:01 crc kubenswrapper[5052]: I1203 12:36:01.892974 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/083c7d71-e993-4f55-9192-4ec777f861e0-log-httpd\") pod \"083c7d71-e993-4f55-9192-4ec777f861e0\" (UID: \"083c7d71-e993-4f55-9192-4ec777f861e0\") " Dec 03 12:36:01 crc kubenswrapper[5052]: I1203 12:36:01.893007 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/083c7d71-e993-4f55-9192-4ec777f861e0-sg-core-conf-yaml\") pod \"083c7d71-e993-4f55-9192-4ec777f861e0\" (UID: \"083c7d71-e993-4f55-9192-4ec777f861e0\") " Dec 03 12:36:01 crc kubenswrapper[5052]: I1203 12:36:01.893030 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kg6jq\" (UniqueName: \"kubernetes.io/projected/083c7d71-e993-4f55-9192-4ec777f861e0-kube-api-access-kg6jq\") pod \"083c7d71-e993-4f55-9192-4ec777f861e0\" (UID: \"083c7d71-e993-4f55-9192-4ec777f861e0\") " Dec 03 12:36:01 crc kubenswrapper[5052]: I1203 12:36:01.893070 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/083c7d71-e993-4f55-9192-4ec777f861e0-run-httpd\") pod \"083c7d71-e993-4f55-9192-4ec777f861e0\" (UID: \"083c7d71-e993-4f55-9192-4ec777f861e0\") " Dec 03 12:36:01 crc kubenswrapper[5052]: I1203 12:36:01.893098 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/083c7d71-e993-4f55-9192-4ec777f861e0-config-data\") pod \"083c7d71-e993-4f55-9192-4ec777f861e0\" (UID: \"083c7d71-e993-4f55-9192-4ec777f861e0\") " Dec 03 12:36:01 crc kubenswrapper[5052]: I1203 12:36:01.893118 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/083c7d71-e993-4f55-9192-4ec777f861e0-combined-ca-bundle\") pod \"083c7d71-e993-4f55-9192-4ec777f861e0\" (UID: \"083c7d71-e993-4f55-9192-4ec777f861e0\") " Dec 03 12:36:01 crc kubenswrapper[5052]: I1203 12:36:01.894599 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/083c7d71-e993-4f55-9192-4ec777f861e0-run-httpd" (OuterVolumeSpecName: "run-httpd") pod "083c7d71-e993-4f55-9192-4ec777f861e0" (UID: "083c7d71-e993-4f55-9192-4ec777f861e0"). InnerVolumeSpecName "run-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:36:01 crc kubenswrapper[5052]: I1203 12:36:01.894840 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/083c7d71-e993-4f55-9192-4ec777f861e0-log-httpd" (OuterVolumeSpecName: "log-httpd") pod "083c7d71-e993-4f55-9192-4ec777f861e0" (UID: "083c7d71-e993-4f55-9192-4ec777f861e0"). InnerVolumeSpecName "log-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:36:01 crc kubenswrapper[5052]: I1203 12:36:01.898896 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/083c7d71-e993-4f55-9192-4ec777f861e0-scripts" (OuterVolumeSpecName: "scripts") pod "083c7d71-e993-4f55-9192-4ec777f861e0" (UID: "083c7d71-e993-4f55-9192-4ec777f861e0"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:36:01 crc kubenswrapper[5052]: I1203 12:36:01.900736 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/083c7d71-e993-4f55-9192-4ec777f861e0-kube-api-access-kg6jq" (OuterVolumeSpecName: "kube-api-access-kg6jq") pod "083c7d71-e993-4f55-9192-4ec777f861e0" (UID: "083c7d71-e993-4f55-9192-4ec777f861e0"). InnerVolumeSpecName "kube-api-access-kg6jq". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:36:01 crc kubenswrapper[5052]: I1203 12:36:01.927775 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/083c7d71-e993-4f55-9192-4ec777f861e0-sg-core-conf-yaml" (OuterVolumeSpecName: "sg-core-conf-yaml") pod "083c7d71-e993-4f55-9192-4ec777f861e0" (UID: "083c7d71-e993-4f55-9192-4ec777f861e0"). InnerVolumeSpecName "sg-core-conf-yaml". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:36:01 crc kubenswrapper[5052]: I1203 12:36:01.984667 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/083c7d71-e993-4f55-9192-4ec777f861e0-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "083c7d71-e993-4f55-9192-4ec777f861e0" (UID: "083c7d71-e993-4f55-9192-4ec777f861e0"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:36:01 crc kubenswrapper[5052]: I1203 12:36:01.991361 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/083c7d71-e993-4f55-9192-4ec777f861e0-config-data" (OuterVolumeSpecName: "config-data") pod "083c7d71-e993-4f55-9192-4ec777f861e0" (UID: "083c7d71-e993-4f55-9192-4ec777f861e0"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:36:01 crc kubenswrapper[5052]: I1203 12:36:01.995460 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/083c7d71-e993-4f55-9192-4ec777f861e0-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:36:01 crc kubenswrapper[5052]: I1203 12:36:01.995690 5052 reconciler_common.go:293] "Volume detached for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/083c7d71-e993-4f55-9192-4ec777f861e0-log-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:36:01 crc kubenswrapper[5052]: I1203 12:36:01.995778 5052 reconciler_common.go:293] "Volume detached for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/083c7d71-e993-4f55-9192-4ec777f861e0-sg-core-conf-yaml\") on node \"crc\" DevicePath \"\"" Dec 03 12:36:01 crc kubenswrapper[5052]: I1203 12:36:01.995859 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-kg6jq\" (UniqueName: \"kubernetes.io/projected/083c7d71-e993-4f55-9192-4ec777f861e0-kube-api-access-kg6jq\") on node \"crc\" DevicePath \"\"" Dec 03 12:36:01 crc kubenswrapper[5052]: I1203 12:36:01.995953 5052 reconciler_common.go:293] "Volume detached for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/083c7d71-e993-4f55-9192-4ec777f861e0-run-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:36:01 crc kubenswrapper[5052]: I1203 12:36:01.996037 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/083c7d71-e993-4f55-9192-4ec777f861e0-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:36:01 crc kubenswrapper[5052]: I1203 12:36:01.996104 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/083c7d71-e993-4f55-9192-4ec777f861e0-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:36:02 crc kubenswrapper[5052]: I1203 12:36:02.011125 5052 generic.go:334] "Generic (PLEG): container finished" podID="083c7d71-e993-4f55-9192-4ec777f861e0" containerID="cb557aed42badd5a8e35a87c5d43012c96b278f7e1b453964099db1e2f21204a" exitCode=0 Dec 03 12:36:02 crc kubenswrapper[5052]: I1203 12:36:02.011331 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"083c7d71-e993-4f55-9192-4ec777f861e0","Type":"ContainerDied","Data":"cb557aed42badd5a8e35a87c5d43012c96b278f7e1b453964099db1e2f21204a"} Dec 03 12:36:02 crc kubenswrapper[5052]: I1203 12:36:02.011490 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"083c7d71-e993-4f55-9192-4ec777f861e0","Type":"ContainerDied","Data":"a2f6c0cf27000f6ee8c784094b3435a973922ac368358efee947496b783d7284"} Dec 03 12:36:02 crc kubenswrapper[5052]: I1203 12:36:02.011418 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:36:02 crc kubenswrapper[5052]: I1203 12:36:02.011548 5052 scope.go:117] "RemoveContainer" containerID="2cbd14b600699bfe1e099da97363c805b86566346724df2d5ac0c9d8b33841f8" Dec 03 12:36:02 crc kubenswrapper[5052]: I1203 12:36:02.013026 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-conductor-0" event={"ID":"1498054d-22e0-426e-b85d-144cf7c4180f","Type":"ContainerStarted","Data":"1faeb3363fa7f1cd2f7286bb26522aec6db5cbbb5ff191e8414c9a7b738bf6e4"} Dec 03 12:36:02 crc kubenswrapper[5052]: I1203 12:36:02.013768 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:36:02 crc kubenswrapper[5052]: I1203 12:36:02.034787 5052 scope.go:117] "RemoveContainer" containerID="a6139bafd67bbba3d8cd83eeb249e20fea6767a144eb504613ee6b7dd06877e9" Dec 03 12:36:02 crc kubenswrapper[5052]: I1203 12:36:02.035152 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-cell0-conductor-0" podStartSLOduration=2.035139719 podStartE2EDuration="2.035139719s" podCreationTimestamp="2025-12-03 12:36:00 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:36:02.031863195 +0000 UTC m=+4552.620104969" watchObservedRunningTime="2025-12-03 12:36:02.035139719 +0000 UTC m=+4552.623381483" Dec 03 12:36:02 crc kubenswrapper[5052]: I1203 12:36:02.067902 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:36:02 crc kubenswrapper[5052]: I1203 12:36:02.072084 5052 scope.go:117] "RemoveContainer" containerID="c3d595431e6fb1c94db66f3c11a76dc94cb634f25c27480a4930edb720dae2d3" Dec 03 12:36:02 crc kubenswrapper[5052]: I1203 12:36:02.079797 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:36:02 crc kubenswrapper[5052]: I1203 12:36:02.100239 5052 scope.go:117] "RemoveContainer" containerID="cb557aed42badd5a8e35a87c5d43012c96b278f7e1b453964099db1e2f21204a" Dec 03 12:36:02 crc kubenswrapper[5052]: I1203 12:36:02.104845 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:36:02 crc kubenswrapper[5052]: E1203 12:36:02.105263 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="083c7d71-e993-4f55-9192-4ec777f861e0" containerName="sg-core" Dec 03 12:36:02 crc kubenswrapper[5052]: I1203 12:36:02.105282 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="083c7d71-e993-4f55-9192-4ec777f861e0" containerName="sg-core" Dec 03 12:36:02 crc kubenswrapper[5052]: E1203 12:36:02.105300 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="083c7d71-e993-4f55-9192-4ec777f861e0" containerName="ceilometer-notification-agent" Dec 03 12:36:02 crc kubenswrapper[5052]: I1203 12:36:02.105307 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="083c7d71-e993-4f55-9192-4ec777f861e0" containerName="ceilometer-notification-agent" Dec 03 12:36:02 crc kubenswrapper[5052]: E1203 12:36:02.105328 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="083c7d71-e993-4f55-9192-4ec777f861e0" containerName="proxy-httpd" Dec 03 12:36:02 crc kubenswrapper[5052]: I1203 12:36:02.105334 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="083c7d71-e993-4f55-9192-4ec777f861e0" containerName="proxy-httpd" Dec 03 12:36:02 crc kubenswrapper[5052]: E1203 12:36:02.105346 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="083c7d71-e993-4f55-9192-4ec777f861e0" containerName="ceilometer-central-agent" Dec 03 12:36:02 crc kubenswrapper[5052]: I1203 12:36:02.105352 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="083c7d71-e993-4f55-9192-4ec777f861e0" containerName="ceilometer-central-agent" Dec 03 12:36:02 crc kubenswrapper[5052]: I1203 12:36:02.105524 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="083c7d71-e993-4f55-9192-4ec777f861e0" containerName="sg-core" Dec 03 12:36:02 crc kubenswrapper[5052]: I1203 12:36:02.105540 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="083c7d71-e993-4f55-9192-4ec777f861e0" containerName="ceilometer-central-agent" Dec 03 12:36:02 crc kubenswrapper[5052]: I1203 12:36:02.105554 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="083c7d71-e993-4f55-9192-4ec777f861e0" containerName="proxy-httpd" Dec 03 12:36:02 crc kubenswrapper[5052]: I1203 12:36:02.105564 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="083c7d71-e993-4f55-9192-4ec777f861e0" containerName="ceilometer-notification-agent" Dec 03 12:36:02 crc kubenswrapper[5052]: I1203 12:36:02.108000 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:36:02 crc kubenswrapper[5052]: I1203 12:36:02.113443 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-scripts" Dec 03 12:36:02 crc kubenswrapper[5052]: I1203 12:36:02.114473 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-config-data" Dec 03 12:36:02 crc kubenswrapper[5052]: I1203 12:36:02.115641 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:36:02 crc kubenswrapper[5052]: I1203 12:36:02.155257 5052 scope.go:117] "RemoveContainer" containerID="2cbd14b600699bfe1e099da97363c805b86566346724df2d5ac0c9d8b33841f8" Dec 03 12:36:02 crc kubenswrapper[5052]: E1203 12:36:02.155889 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2cbd14b600699bfe1e099da97363c805b86566346724df2d5ac0c9d8b33841f8\": container with ID starting with 2cbd14b600699bfe1e099da97363c805b86566346724df2d5ac0c9d8b33841f8 not found: ID does not exist" containerID="2cbd14b600699bfe1e099da97363c805b86566346724df2d5ac0c9d8b33841f8" Dec 03 12:36:02 crc kubenswrapper[5052]: I1203 12:36:02.155989 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2cbd14b600699bfe1e099da97363c805b86566346724df2d5ac0c9d8b33841f8"} err="failed to get container status \"2cbd14b600699bfe1e099da97363c805b86566346724df2d5ac0c9d8b33841f8\": rpc error: code = NotFound desc = could not find container \"2cbd14b600699bfe1e099da97363c805b86566346724df2d5ac0c9d8b33841f8\": container with ID starting with 2cbd14b600699bfe1e099da97363c805b86566346724df2d5ac0c9d8b33841f8 not found: ID does not exist" Dec 03 12:36:02 crc kubenswrapper[5052]: I1203 12:36:02.156074 5052 scope.go:117] "RemoveContainer" containerID="a6139bafd67bbba3d8cd83eeb249e20fea6767a144eb504613ee6b7dd06877e9" Dec 03 12:36:02 crc kubenswrapper[5052]: E1203 12:36:02.156408 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a6139bafd67bbba3d8cd83eeb249e20fea6767a144eb504613ee6b7dd06877e9\": container with ID starting with a6139bafd67bbba3d8cd83eeb249e20fea6767a144eb504613ee6b7dd06877e9 not found: ID does not exist" containerID="a6139bafd67bbba3d8cd83eeb249e20fea6767a144eb504613ee6b7dd06877e9" Dec 03 12:36:02 crc kubenswrapper[5052]: I1203 12:36:02.156455 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a6139bafd67bbba3d8cd83eeb249e20fea6767a144eb504613ee6b7dd06877e9"} err="failed to get container status \"a6139bafd67bbba3d8cd83eeb249e20fea6767a144eb504613ee6b7dd06877e9\": rpc error: code = NotFound desc = could not find container \"a6139bafd67bbba3d8cd83eeb249e20fea6767a144eb504613ee6b7dd06877e9\": container with ID starting with a6139bafd67bbba3d8cd83eeb249e20fea6767a144eb504613ee6b7dd06877e9 not found: ID does not exist" Dec 03 12:36:02 crc kubenswrapper[5052]: I1203 12:36:02.156473 5052 scope.go:117] "RemoveContainer" containerID="c3d595431e6fb1c94db66f3c11a76dc94cb634f25c27480a4930edb720dae2d3" Dec 03 12:36:02 crc kubenswrapper[5052]: E1203 12:36:02.156863 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c3d595431e6fb1c94db66f3c11a76dc94cb634f25c27480a4930edb720dae2d3\": container with ID starting with c3d595431e6fb1c94db66f3c11a76dc94cb634f25c27480a4930edb720dae2d3 not found: ID does not exist" containerID="c3d595431e6fb1c94db66f3c11a76dc94cb634f25c27480a4930edb720dae2d3" Dec 03 12:36:02 crc kubenswrapper[5052]: I1203 12:36:02.156908 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c3d595431e6fb1c94db66f3c11a76dc94cb634f25c27480a4930edb720dae2d3"} err="failed to get container status \"c3d595431e6fb1c94db66f3c11a76dc94cb634f25c27480a4930edb720dae2d3\": rpc error: code = NotFound desc = could not find container \"c3d595431e6fb1c94db66f3c11a76dc94cb634f25c27480a4930edb720dae2d3\": container with ID starting with c3d595431e6fb1c94db66f3c11a76dc94cb634f25c27480a4930edb720dae2d3 not found: ID does not exist" Dec 03 12:36:02 crc kubenswrapper[5052]: I1203 12:36:02.156921 5052 scope.go:117] "RemoveContainer" containerID="cb557aed42badd5a8e35a87c5d43012c96b278f7e1b453964099db1e2f21204a" Dec 03 12:36:02 crc kubenswrapper[5052]: E1203 12:36:02.157283 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cb557aed42badd5a8e35a87c5d43012c96b278f7e1b453964099db1e2f21204a\": container with ID starting with cb557aed42badd5a8e35a87c5d43012c96b278f7e1b453964099db1e2f21204a not found: ID does not exist" containerID="cb557aed42badd5a8e35a87c5d43012c96b278f7e1b453964099db1e2f21204a" Dec 03 12:36:02 crc kubenswrapper[5052]: I1203 12:36:02.157304 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cb557aed42badd5a8e35a87c5d43012c96b278f7e1b453964099db1e2f21204a"} err="failed to get container status \"cb557aed42badd5a8e35a87c5d43012c96b278f7e1b453964099db1e2f21204a\": rpc error: code = NotFound desc = could not find container \"cb557aed42badd5a8e35a87c5d43012c96b278f7e1b453964099db1e2f21204a\": container with ID starting with cb557aed42badd5a8e35a87c5d43012c96b278f7e1b453964099db1e2f21204a not found: ID does not exist" Dec 03 12:36:02 crc kubenswrapper[5052]: I1203 12:36:02.302305 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/ed1c5151-73cb-4c20-b801-cf77509c7088-log-httpd\") pod \"ceilometer-0\" (UID: \"ed1c5151-73cb-4c20-b801-cf77509c7088\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:36:02 crc kubenswrapper[5052]: I1203 12:36:02.302369 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/ed1c5151-73cb-4c20-b801-cf77509c7088-run-httpd\") pod \"ceilometer-0\" (UID: \"ed1c5151-73cb-4c20-b801-cf77509c7088\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:36:02 crc kubenswrapper[5052]: I1203 12:36:02.302396 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/ed1c5151-73cb-4c20-b801-cf77509c7088-scripts\") pod \"ceilometer-0\" (UID: \"ed1c5151-73cb-4c20-b801-cf77509c7088\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:36:02 crc kubenswrapper[5052]: I1203 12:36:02.302677 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ed1c5151-73cb-4c20-b801-cf77509c7088-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"ed1c5151-73cb-4c20-b801-cf77509c7088\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:36:02 crc kubenswrapper[5052]: I1203 12:36:02.302780 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/ed1c5151-73cb-4c20-b801-cf77509c7088-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"ed1c5151-73cb-4c20-b801-cf77509c7088\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:36:02 crc kubenswrapper[5052]: I1203 12:36:02.302887 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hp4j8\" (UniqueName: \"kubernetes.io/projected/ed1c5151-73cb-4c20-b801-cf77509c7088-kube-api-access-hp4j8\") pod \"ceilometer-0\" (UID: \"ed1c5151-73cb-4c20-b801-cf77509c7088\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:36:02 crc kubenswrapper[5052]: I1203 12:36:02.302919 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/ed1c5151-73cb-4c20-b801-cf77509c7088-config-data\") pod \"ceilometer-0\" (UID: \"ed1c5151-73cb-4c20-b801-cf77509c7088\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:36:02 crc kubenswrapper[5052]: I1203 12:36:02.405259 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-hp4j8\" (UniqueName: \"kubernetes.io/projected/ed1c5151-73cb-4c20-b801-cf77509c7088-kube-api-access-hp4j8\") pod \"ceilometer-0\" (UID: \"ed1c5151-73cb-4c20-b801-cf77509c7088\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:36:02 crc kubenswrapper[5052]: I1203 12:36:02.405303 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/ed1c5151-73cb-4c20-b801-cf77509c7088-config-data\") pod \"ceilometer-0\" (UID: \"ed1c5151-73cb-4c20-b801-cf77509c7088\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:36:02 crc kubenswrapper[5052]: I1203 12:36:02.405408 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/ed1c5151-73cb-4c20-b801-cf77509c7088-log-httpd\") pod \"ceilometer-0\" (UID: \"ed1c5151-73cb-4c20-b801-cf77509c7088\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:36:02 crc kubenswrapper[5052]: I1203 12:36:02.405436 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/ed1c5151-73cb-4c20-b801-cf77509c7088-run-httpd\") pod \"ceilometer-0\" (UID: \"ed1c5151-73cb-4c20-b801-cf77509c7088\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:36:02 crc kubenswrapper[5052]: I1203 12:36:02.405456 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/ed1c5151-73cb-4c20-b801-cf77509c7088-scripts\") pod \"ceilometer-0\" (UID: \"ed1c5151-73cb-4c20-b801-cf77509c7088\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:36:02 crc kubenswrapper[5052]: I1203 12:36:02.405506 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ed1c5151-73cb-4c20-b801-cf77509c7088-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"ed1c5151-73cb-4c20-b801-cf77509c7088\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:36:02 crc kubenswrapper[5052]: I1203 12:36:02.405545 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/ed1c5151-73cb-4c20-b801-cf77509c7088-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"ed1c5151-73cb-4c20-b801-cf77509c7088\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:36:02 crc kubenswrapper[5052]: I1203 12:36:02.406836 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/ed1c5151-73cb-4c20-b801-cf77509c7088-log-httpd\") pod \"ceilometer-0\" (UID: \"ed1c5151-73cb-4c20-b801-cf77509c7088\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:36:02 crc kubenswrapper[5052]: I1203 12:36:02.407159 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/ed1c5151-73cb-4c20-b801-cf77509c7088-run-httpd\") pod \"ceilometer-0\" (UID: \"ed1c5151-73cb-4c20-b801-cf77509c7088\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:36:02 crc kubenswrapper[5052]: I1203 12:36:02.411770 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/ed1c5151-73cb-4c20-b801-cf77509c7088-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"ed1c5151-73cb-4c20-b801-cf77509c7088\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:36:02 crc kubenswrapper[5052]: I1203 12:36:02.412019 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ed1c5151-73cb-4c20-b801-cf77509c7088-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"ed1c5151-73cb-4c20-b801-cf77509c7088\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:36:02 crc kubenswrapper[5052]: I1203 12:36:02.412019 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/ed1c5151-73cb-4c20-b801-cf77509c7088-scripts\") pod \"ceilometer-0\" (UID: \"ed1c5151-73cb-4c20-b801-cf77509c7088\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:36:02 crc kubenswrapper[5052]: I1203 12:36:02.428726 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/ed1c5151-73cb-4c20-b801-cf77509c7088-config-data\") pod \"ceilometer-0\" (UID: \"ed1c5151-73cb-4c20-b801-cf77509c7088\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:36:02 crc kubenswrapper[5052]: I1203 12:36:02.440910 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-hp4j8\" (UniqueName: \"kubernetes.io/projected/ed1c5151-73cb-4c20-b801-cf77509c7088-kube-api-access-hp4j8\") pod \"ceilometer-0\" (UID: \"ed1c5151-73cb-4c20-b801-cf77509c7088\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:36:02 crc kubenswrapper[5052]: I1203 12:36:02.673317 5052 scope.go:117] "RemoveContainer" containerID="e6edaf188f85ea4f11f487f11cc489c700f25fb42b5ee269f4f496f1e127d772" Dec 03 12:36:02 crc kubenswrapper[5052]: E1203 12:36:02.673618 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 12:36:02 crc kubenswrapper[5052]: I1203 12:36:02.741471 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:36:03 crc kubenswrapper[5052]: I1203 12:36:03.186706 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:36:03 crc kubenswrapper[5052]: W1203 12:36:03.193639 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poded1c5151_73cb_4c20_b801_cf77509c7088.slice/crio-9497695f831e94f22d1d3ea8130a32e4964a34d77e7538607c165b98b87813ae WatchSource:0}: Error finding container 9497695f831e94f22d1d3ea8130a32e4964a34d77e7538607c165b98b87813ae: Status 404 returned error can't find the container with id 9497695f831e94f22d1d3ea8130a32e4964a34d77e7538607c165b98b87813ae Dec 03 12:36:03 crc kubenswrapper[5052]: I1203 12:36:03.685011 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="083c7d71-e993-4f55-9192-4ec777f861e0" path="/var/lib/kubelet/pods/083c7d71-e993-4f55-9192-4ec777f861e0/volumes" Dec 03 12:36:04 crc kubenswrapper[5052]: I1203 12:36:04.034453 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"ed1c5151-73cb-4c20-b801-cf77509c7088","Type":"ContainerStarted","Data":"ba21a758c0eafa87981092d11a759465be8fe7d7c3f030cbfc2a0ec4049f8450"} Dec 03 12:36:04 crc kubenswrapper[5052]: I1203 12:36:04.034562 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"ed1c5151-73cb-4c20-b801-cf77509c7088","Type":"ContainerStarted","Data":"9497695f831e94f22d1d3ea8130a32e4964a34d77e7538607c165b98b87813ae"} Dec 03 12:36:05 crc kubenswrapper[5052]: I1203 12:36:05.047119 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"ed1c5151-73cb-4c20-b801-cf77509c7088","Type":"ContainerStarted","Data":"3fdeef99541d7da1c01a8d9a5170a7f180d3f1cc9d21e3baa51812d4cb02033c"} Dec 03 12:36:06 crc kubenswrapper[5052]: I1203 12:36:06.058232 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"ed1c5151-73cb-4c20-b801-cf77509c7088","Type":"ContainerStarted","Data":"3300d0f26c3e473950a0c44c09d7cd9b59119f7fd171515efd3ee4a1105f3249"} Dec 03 12:36:06 crc kubenswrapper[5052]: I1203 12:36:06.466705 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:36:06 crc kubenswrapper[5052]: I1203 12:36:06.466923 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-external-api-0" podUID="6e96674e-4a20-4d8f-9118-90532fa75788" containerName="glance-log" containerID="cri-o://ddcff7ae2062661e408d5c15760d3bc57eb9460bcc46af6105ea25f9411be361" gracePeriod=30 Dec 03 12:36:06 crc kubenswrapper[5052]: I1203 12:36:06.467372 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-external-api-0" podUID="6e96674e-4a20-4d8f-9118-90532fa75788" containerName="glance-httpd" containerID="cri-o://f2a8405809435951d9d11632bcafb46cf42cd5e5fa8d7b9e91d1ba64a3ac3dca" gracePeriod=30 Dec 03 12:36:07 crc kubenswrapper[5052]: I1203 12:36:07.068206 5052 generic.go:334] "Generic (PLEG): container finished" podID="6e96674e-4a20-4d8f-9118-90532fa75788" containerID="ddcff7ae2062661e408d5c15760d3bc57eb9460bcc46af6105ea25f9411be361" exitCode=143 Dec 03 12:36:07 crc kubenswrapper[5052]: I1203 12:36:07.068258 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"6e96674e-4a20-4d8f-9118-90532fa75788","Type":"ContainerDied","Data":"ddcff7ae2062661e408d5c15760d3bc57eb9460bcc46af6105ea25f9411be361"} Dec 03 12:36:07 crc kubenswrapper[5052]: I1203 12:36:07.407914 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-0"] Dec 03 12:36:07 crc kubenswrapper[5052]: I1203 12:36:07.408354 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-cell0-conductor-0" podUID="1498054d-22e0-426e-b85d-144cf7c4180f" containerName="nova-cell0-conductor-conductor" containerID="cri-o://1faeb3363fa7f1cd2f7286bb26522aec6db5cbbb5ff191e8414c9a7b738bf6e4" gracePeriod=30 Dec 03 12:36:07 crc kubenswrapper[5052]: E1203 12:36:07.412858 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="1faeb3363fa7f1cd2f7286bb26522aec6db5cbbb5ff191e8414c9a7b738bf6e4" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:36:07 crc kubenswrapper[5052]: E1203 12:36:07.417231 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="1faeb3363fa7f1cd2f7286bb26522aec6db5cbbb5ff191e8414c9a7b738bf6e4" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:36:07 crc kubenswrapper[5052]: E1203 12:36:07.426478 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="1faeb3363fa7f1cd2f7286bb26522aec6db5cbbb5ff191e8414c9a7b738bf6e4" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:36:07 crc kubenswrapper[5052]: E1203 12:36:07.426575 5052 prober.go:104] "Probe errored" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" probeType="Readiness" pod="openstack-kuttl-tests/nova-cell0-conductor-0" podUID="1498054d-22e0-426e-b85d-144cf7c4180f" containerName="nova-cell0-conductor-conductor" Dec 03 12:36:07 crc kubenswrapper[5052]: I1203 12:36:07.469637 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:36:07 crc kubenswrapper[5052]: I1203 12:36:07.469860 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-internal-api-0" podUID="25498b02-c179-4555-ad04-1ef6445df4ab" containerName="glance-log" containerID="cri-o://b1eb3705336137fdb58a75000976f8917d3ae6bc93b6459c35f5a5a4494f5dc5" gracePeriod=30 Dec 03 12:36:07 crc kubenswrapper[5052]: I1203 12:36:07.470616 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-internal-api-0" podUID="25498b02-c179-4555-ad04-1ef6445df4ab" containerName="glance-httpd" containerID="cri-o://7e310e8333b7259c202284ca732fa44b22dde345a4ca324e0799f37aa77c10db" gracePeriod=30 Dec 03 12:36:08 crc kubenswrapper[5052]: I1203 12:36:08.081336 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"ed1c5151-73cb-4c20-b801-cf77509c7088","Type":"ContainerStarted","Data":"41dc33620881a66066ffa87ac1dc04a00aba4a0067bba8e0d874f761f849a738"} Dec 03 12:36:08 crc kubenswrapper[5052]: I1203 12:36:08.081700 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:36:08 crc kubenswrapper[5052]: I1203 12:36:08.083854 5052 generic.go:334] "Generic (PLEG): container finished" podID="25498b02-c179-4555-ad04-1ef6445df4ab" containerID="b1eb3705336137fdb58a75000976f8917d3ae6bc93b6459c35f5a5a4494f5dc5" exitCode=143 Dec 03 12:36:08 crc kubenswrapper[5052]: I1203 12:36:08.083905 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"25498b02-c179-4555-ad04-1ef6445df4ab","Type":"ContainerDied","Data":"b1eb3705336137fdb58a75000976f8917d3ae6bc93b6459c35f5a5a4494f5dc5"} Dec 03 12:36:08 crc kubenswrapper[5052]: I1203 12:36:08.114046 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/ceilometer-0" podStartSLOduration=1.946100919 podStartE2EDuration="6.114028044s" podCreationTimestamp="2025-12-03 12:36:02 +0000 UTC" firstStartedPulling="2025-12-03 12:36:03.195526784 +0000 UTC m=+4553.783768548" lastFinishedPulling="2025-12-03 12:36:07.363453909 +0000 UTC m=+4557.951695673" observedRunningTime="2025-12-03 12:36:08.105276164 +0000 UTC m=+4558.693517948" watchObservedRunningTime="2025-12-03 12:36:08.114028044 +0000 UTC m=+4558.702269798" Dec 03 12:36:08 crc kubenswrapper[5052]: I1203 12:36:08.700751 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:36:08 crc kubenswrapper[5052]: I1203 12:36:08.703905 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:36:08 crc kubenswrapper[5052]: I1203 12:36:08.822487 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7cxn7\" (UniqueName: \"kubernetes.io/projected/1498054d-22e0-426e-b85d-144cf7c4180f-kube-api-access-7cxn7\") pod \"1498054d-22e0-426e-b85d-144cf7c4180f\" (UID: \"1498054d-22e0-426e-b85d-144cf7c4180f\") " Dec 03 12:36:08 crc kubenswrapper[5052]: I1203 12:36:08.822606 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1498054d-22e0-426e-b85d-144cf7c4180f-combined-ca-bundle\") pod \"1498054d-22e0-426e-b85d-144cf7c4180f\" (UID: \"1498054d-22e0-426e-b85d-144cf7c4180f\") " Dec 03 12:36:08 crc kubenswrapper[5052]: I1203 12:36:08.822687 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/1498054d-22e0-426e-b85d-144cf7c4180f-config-data\") pod \"1498054d-22e0-426e-b85d-144cf7c4180f\" (UID: \"1498054d-22e0-426e-b85d-144cf7c4180f\") " Dec 03 12:36:08 crc kubenswrapper[5052]: I1203 12:36:08.835725 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1498054d-22e0-426e-b85d-144cf7c4180f-kube-api-access-7cxn7" (OuterVolumeSpecName: "kube-api-access-7cxn7") pod "1498054d-22e0-426e-b85d-144cf7c4180f" (UID: "1498054d-22e0-426e-b85d-144cf7c4180f"). InnerVolumeSpecName "kube-api-access-7cxn7". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:36:08 crc kubenswrapper[5052]: I1203 12:36:08.859497 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1498054d-22e0-426e-b85d-144cf7c4180f-config-data" (OuterVolumeSpecName: "config-data") pod "1498054d-22e0-426e-b85d-144cf7c4180f" (UID: "1498054d-22e0-426e-b85d-144cf7c4180f"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:36:08 crc kubenswrapper[5052]: I1203 12:36:08.866672 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1498054d-22e0-426e-b85d-144cf7c4180f-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "1498054d-22e0-426e-b85d-144cf7c4180f" (UID: "1498054d-22e0-426e-b85d-144cf7c4180f"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:36:08 crc kubenswrapper[5052]: I1203 12:36:08.937738 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/1498054d-22e0-426e-b85d-144cf7c4180f-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:36:08 crc kubenswrapper[5052]: I1203 12:36:08.937936 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-7cxn7\" (UniqueName: \"kubernetes.io/projected/1498054d-22e0-426e-b85d-144cf7c4180f-kube-api-access-7cxn7\") on node \"crc\" DevicePath \"\"" Dec 03 12:36:08 crc kubenswrapper[5052]: I1203 12:36:08.937957 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1498054d-22e0-426e-b85d-144cf7c4180f-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:36:09 crc kubenswrapper[5052]: I1203 12:36:09.093364 5052 generic.go:334] "Generic (PLEG): container finished" podID="1498054d-22e0-426e-b85d-144cf7c4180f" containerID="1faeb3363fa7f1cd2f7286bb26522aec6db5cbbb5ff191e8414c9a7b738bf6e4" exitCode=0 Dec 03 12:36:09 crc kubenswrapper[5052]: I1203 12:36:09.093447 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-conductor-0" event={"ID":"1498054d-22e0-426e-b85d-144cf7c4180f","Type":"ContainerDied","Data":"1faeb3363fa7f1cd2f7286bb26522aec6db5cbbb5ff191e8414c9a7b738bf6e4"} Dec 03 12:36:09 crc kubenswrapper[5052]: I1203 12:36:09.093482 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-conductor-0" event={"ID":"1498054d-22e0-426e-b85d-144cf7c4180f","Type":"ContainerDied","Data":"8e7655ebc8c6aaae5798bf385240ef8b2645e57121844e670143a45fbaf97325"} Dec 03 12:36:09 crc kubenswrapper[5052]: I1203 12:36:09.093541 5052 scope.go:117] "RemoveContainer" containerID="1faeb3363fa7f1cd2f7286bb26522aec6db5cbbb5ff191e8414c9a7b738bf6e4" Dec 03 12:36:09 crc kubenswrapper[5052]: I1203 12:36:09.093559 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:36:09 crc kubenswrapper[5052]: I1203 12:36:09.127755 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-0"] Dec 03 12:36:09 crc kubenswrapper[5052]: I1203 12:36:09.140198 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-0"] Dec 03 12:36:09 crc kubenswrapper[5052]: I1203 12:36:09.143146 5052 scope.go:117] "RemoveContainer" containerID="1faeb3363fa7f1cd2f7286bb26522aec6db5cbbb5ff191e8414c9a7b738bf6e4" Dec 03 12:36:09 crc kubenswrapper[5052]: E1203 12:36:09.143473 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1faeb3363fa7f1cd2f7286bb26522aec6db5cbbb5ff191e8414c9a7b738bf6e4\": container with ID starting with 1faeb3363fa7f1cd2f7286bb26522aec6db5cbbb5ff191e8414c9a7b738bf6e4 not found: ID does not exist" containerID="1faeb3363fa7f1cd2f7286bb26522aec6db5cbbb5ff191e8414c9a7b738bf6e4" Dec 03 12:36:09 crc kubenswrapper[5052]: I1203 12:36:09.143502 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1faeb3363fa7f1cd2f7286bb26522aec6db5cbbb5ff191e8414c9a7b738bf6e4"} err="failed to get container status \"1faeb3363fa7f1cd2f7286bb26522aec6db5cbbb5ff191e8414c9a7b738bf6e4\": rpc error: code = NotFound desc = could not find container \"1faeb3363fa7f1cd2f7286bb26522aec6db5cbbb5ff191e8414c9a7b738bf6e4\": container with ID starting with 1faeb3363fa7f1cd2f7286bb26522aec6db5cbbb5ff191e8414c9a7b738bf6e4 not found: ID does not exist" Dec 03 12:36:09 crc kubenswrapper[5052]: I1203 12:36:09.150965 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-0"] Dec 03 12:36:09 crc kubenswrapper[5052]: E1203 12:36:09.151463 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="1498054d-22e0-426e-b85d-144cf7c4180f" containerName="nova-cell0-conductor-conductor" Dec 03 12:36:09 crc kubenswrapper[5052]: I1203 12:36:09.151485 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="1498054d-22e0-426e-b85d-144cf7c4180f" containerName="nova-cell0-conductor-conductor" Dec 03 12:36:09 crc kubenswrapper[5052]: I1203 12:36:09.151772 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="1498054d-22e0-426e-b85d-144cf7c4180f" containerName="nova-cell0-conductor-conductor" Dec 03 12:36:09 crc kubenswrapper[5052]: I1203 12:36:09.152607 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:36:09 crc kubenswrapper[5052]: I1203 12:36:09.161708 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-0"] Dec 03 12:36:09 crc kubenswrapper[5052]: I1203 12:36:09.184309 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell0-conductor-config-data" Dec 03 12:36:09 crc kubenswrapper[5052]: I1203 12:36:09.184641 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-nova-dockercfg-t94dq" Dec 03 12:36:09 crc kubenswrapper[5052]: I1203 12:36:09.346675 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fmnfd\" (UniqueName: \"kubernetes.io/projected/72a59b27-7692-41b5-96d1-59ffa7d5858a-kube-api-access-fmnfd\") pod \"nova-cell0-conductor-0\" (UID: \"72a59b27-7692-41b5-96d1-59ffa7d5858a\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:36:09 crc kubenswrapper[5052]: I1203 12:36:09.347001 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/72a59b27-7692-41b5-96d1-59ffa7d5858a-config-data\") pod \"nova-cell0-conductor-0\" (UID: \"72a59b27-7692-41b5-96d1-59ffa7d5858a\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:36:09 crc kubenswrapper[5052]: I1203 12:36:09.347184 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/72a59b27-7692-41b5-96d1-59ffa7d5858a-combined-ca-bundle\") pod \"nova-cell0-conductor-0\" (UID: \"72a59b27-7692-41b5-96d1-59ffa7d5858a\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:36:09 crc kubenswrapper[5052]: I1203 12:36:09.448928 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-fmnfd\" (UniqueName: \"kubernetes.io/projected/72a59b27-7692-41b5-96d1-59ffa7d5858a-kube-api-access-fmnfd\") pod \"nova-cell0-conductor-0\" (UID: \"72a59b27-7692-41b5-96d1-59ffa7d5858a\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:36:09 crc kubenswrapper[5052]: I1203 12:36:09.449022 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/72a59b27-7692-41b5-96d1-59ffa7d5858a-config-data\") pod \"nova-cell0-conductor-0\" (UID: \"72a59b27-7692-41b5-96d1-59ffa7d5858a\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:36:09 crc kubenswrapper[5052]: I1203 12:36:09.449091 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/72a59b27-7692-41b5-96d1-59ffa7d5858a-combined-ca-bundle\") pod \"nova-cell0-conductor-0\" (UID: \"72a59b27-7692-41b5-96d1-59ffa7d5858a\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:36:09 crc kubenswrapper[5052]: I1203 12:36:09.454079 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/72a59b27-7692-41b5-96d1-59ffa7d5858a-combined-ca-bundle\") pod \"nova-cell0-conductor-0\" (UID: \"72a59b27-7692-41b5-96d1-59ffa7d5858a\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:36:09 crc kubenswrapper[5052]: I1203 12:36:09.454577 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/72a59b27-7692-41b5-96d1-59ffa7d5858a-config-data\") pod \"nova-cell0-conductor-0\" (UID: \"72a59b27-7692-41b5-96d1-59ffa7d5858a\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:36:09 crc kubenswrapper[5052]: I1203 12:36:09.471529 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-fmnfd\" (UniqueName: \"kubernetes.io/projected/72a59b27-7692-41b5-96d1-59ffa7d5858a-kube-api-access-fmnfd\") pod \"nova-cell0-conductor-0\" (UID: \"72a59b27-7692-41b5-96d1-59ffa7d5858a\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:36:09 crc kubenswrapper[5052]: I1203 12:36:09.500232 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:36:09 crc kubenswrapper[5052]: I1203 12:36:09.711754 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1498054d-22e0-426e-b85d-144cf7c4180f" path="/var/lib/kubelet/pods/1498054d-22e0-426e-b85d-144cf7c4180f/volumes" Dec 03 12:36:09 crc kubenswrapper[5052]: I1203 12:36:09.768230 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/glance-default-external-api-0" podUID="6e96674e-4a20-4d8f-9118-90532fa75788" containerName="glance-httpd" probeResult="failure" output="Get \"https://10.217.1.38:9292/healthcheck\": read tcp 10.217.0.2:44788->10.217.1.38:9292: read: connection reset by peer" Dec 03 12:36:09 crc kubenswrapper[5052]: I1203 12:36:09.768724 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/glance-default-external-api-0" podUID="6e96674e-4a20-4d8f-9118-90532fa75788" containerName="glance-log" probeResult="failure" output="Get \"https://10.217.1.38:9292/healthcheck\": read tcp 10.217.0.2:44798->10.217.1.38:9292: read: connection reset by peer" Dec 03 12:36:09 crc kubenswrapper[5052]: I1203 12:36:09.964834 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-0"] Dec 03 12:36:10 crc kubenswrapper[5052]: I1203 12:36:10.102745 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:36:10 crc kubenswrapper[5052]: I1203 12:36:10.109031 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-conductor-0" event={"ID":"72a59b27-7692-41b5-96d1-59ffa7d5858a","Type":"ContainerStarted","Data":"2139fed74c4c8e0b222acfd683f6009cc0edd28af1bb2ff84cb4b4e1a09bd7b4"} Dec 03 12:36:10 crc kubenswrapper[5052]: I1203 12:36:10.110723 5052 generic.go:334] "Generic (PLEG): container finished" podID="6e96674e-4a20-4d8f-9118-90532fa75788" containerID="f2a8405809435951d9d11632bcafb46cf42cd5e5fa8d7b9e91d1ba64a3ac3dca" exitCode=0 Dec 03 12:36:10 crc kubenswrapper[5052]: I1203 12:36:10.110767 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"6e96674e-4a20-4d8f-9118-90532fa75788","Type":"ContainerDied","Data":"f2a8405809435951d9d11632bcafb46cf42cd5e5fa8d7b9e91d1ba64a3ac3dca"} Dec 03 12:36:10 crc kubenswrapper[5052]: I1203 12:36:10.110813 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:36:10 crc kubenswrapper[5052]: I1203 12:36:10.110834 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"6e96674e-4a20-4d8f-9118-90532fa75788","Type":"ContainerDied","Data":"e958eed2d4d943b337051a8a21db34d2de99782e4a9056908e19e538adaf56f9"} Dec 03 12:36:10 crc kubenswrapper[5052]: I1203 12:36:10.110859 5052 scope.go:117] "RemoveContainer" containerID="f2a8405809435951d9d11632bcafb46cf42cd5e5fa8d7b9e91d1ba64a3ac3dca" Dec 03 12:36:10 crc kubenswrapper[5052]: I1203 12:36:10.110951 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="ed1c5151-73cb-4c20-b801-cf77509c7088" containerName="ceilometer-central-agent" containerID="cri-o://ba21a758c0eafa87981092d11a759465be8fe7d7c3f030cbfc2a0ec4049f8450" gracePeriod=30 Dec 03 12:36:10 crc kubenswrapper[5052]: I1203 12:36:10.110990 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="ed1c5151-73cb-4c20-b801-cf77509c7088" containerName="proxy-httpd" containerID="cri-o://41dc33620881a66066ffa87ac1dc04a00aba4a0067bba8e0d874f761f849a738" gracePeriod=30 Dec 03 12:36:10 crc kubenswrapper[5052]: I1203 12:36:10.111018 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="ed1c5151-73cb-4c20-b801-cf77509c7088" containerName="sg-core" containerID="cri-o://3300d0f26c3e473950a0c44c09d7cd9b59119f7fd171515efd3ee4a1105f3249" gracePeriod=30 Dec 03 12:36:10 crc kubenswrapper[5052]: I1203 12:36:10.111054 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="ed1c5151-73cb-4c20-b801-cf77509c7088" containerName="ceilometer-notification-agent" containerID="cri-o://3fdeef99541d7da1c01a8d9a5170a7f180d3f1cc9d21e3baa51812d4cb02033c" gracePeriod=30 Dec 03 12:36:10 crc kubenswrapper[5052]: I1203 12:36:10.149216 5052 scope.go:117] "RemoveContainer" containerID="ddcff7ae2062661e408d5c15760d3bc57eb9460bcc46af6105ea25f9411be361" Dec 03 12:36:10 crc kubenswrapper[5052]: I1203 12:36:10.178660 5052 scope.go:117] "RemoveContainer" containerID="f2a8405809435951d9d11632bcafb46cf42cd5e5fa8d7b9e91d1ba64a3ac3dca" Dec 03 12:36:10 crc kubenswrapper[5052]: E1203 12:36:10.182566 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f2a8405809435951d9d11632bcafb46cf42cd5e5fa8d7b9e91d1ba64a3ac3dca\": container with ID starting with f2a8405809435951d9d11632bcafb46cf42cd5e5fa8d7b9e91d1ba64a3ac3dca not found: ID does not exist" containerID="f2a8405809435951d9d11632bcafb46cf42cd5e5fa8d7b9e91d1ba64a3ac3dca" Dec 03 12:36:10 crc kubenswrapper[5052]: I1203 12:36:10.182612 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f2a8405809435951d9d11632bcafb46cf42cd5e5fa8d7b9e91d1ba64a3ac3dca"} err="failed to get container status \"f2a8405809435951d9d11632bcafb46cf42cd5e5fa8d7b9e91d1ba64a3ac3dca\": rpc error: code = NotFound desc = could not find container \"f2a8405809435951d9d11632bcafb46cf42cd5e5fa8d7b9e91d1ba64a3ac3dca\": container with ID starting with f2a8405809435951d9d11632bcafb46cf42cd5e5fa8d7b9e91d1ba64a3ac3dca not found: ID does not exist" Dec 03 12:36:10 crc kubenswrapper[5052]: I1203 12:36:10.182641 5052 scope.go:117] "RemoveContainer" containerID="ddcff7ae2062661e408d5c15760d3bc57eb9460bcc46af6105ea25f9411be361" Dec 03 12:36:10 crc kubenswrapper[5052]: E1203 12:36:10.183048 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ddcff7ae2062661e408d5c15760d3bc57eb9460bcc46af6105ea25f9411be361\": container with ID starting with ddcff7ae2062661e408d5c15760d3bc57eb9460bcc46af6105ea25f9411be361 not found: ID does not exist" containerID="ddcff7ae2062661e408d5c15760d3bc57eb9460bcc46af6105ea25f9411be361" Dec 03 12:36:10 crc kubenswrapper[5052]: I1203 12:36:10.183081 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ddcff7ae2062661e408d5c15760d3bc57eb9460bcc46af6105ea25f9411be361"} err="failed to get container status \"ddcff7ae2062661e408d5c15760d3bc57eb9460bcc46af6105ea25f9411be361\": rpc error: code = NotFound desc = could not find container \"ddcff7ae2062661e408d5c15760d3bc57eb9460bcc46af6105ea25f9411be361\": container with ID starting with ddcff7ae2062661e408d5c15760d3bc57eb9460bcc46af6105ea25f9411be361 not found: ID does not exist" Dec 03 12:36:10 crc kubenswrapper[5052]: I1203 12:36:10.263483 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/6e96674e-4a20-4d8f-9118-90532fa75788-public-tls-certs\") pod \"6e96674e-4a20-4d8f-9118-90532fa75788\" (UID: \"6e96674e-4a20-4d8f-9118-90532fa75788\") " Dec 03 12:36:10 crc kubenswrapper[5052]: I1203 12:36:10.263664 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/6e96674e-4a20-4d8f-9118-90532fa75788-logs\") pod \"6e96674e-4a20-4d8f-9118-90532fa75788\" (UID: \"6e96674e-4a20-4d8f-9118-90532fa75788\") " Dec 03 12:36:10 crc kubenswrapper[5052]: I1203 12:36:10.263695 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"glance\" (UniqueName: \"kubernetes.io/local-volume/local-storage12-crc\") pod \"6e96674e-4a20-4d8f-9118-90532fa75788\" (UID: \"6e96674e-4a20-4d8f-9118-90532fa75788\") " Dec 03 12:36:10 crc kubenswrapper[5052]: I1203 12:36:10.263758 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6e96674e-4a20-4d8f-9118-90532fa75788-combined-ca-bundle\") pod \"6e96674e-4a20-4d8f-9118-90532fa75788\" (UID: \"6e96674e-4a20-4d8f-9118-90532fa75788\") " Dec 03 12:36:10 crc kubenswrapper[5052]: I1203 12:36:10.263793 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h87dc\" (UniqueName: \"kubernetes.io/projected/6e96674e-4a20-4d8f-9118-90532fa75788-kube-api-access-h87dc\") pod \"6e96674e-4a20-4d8f-9118-90532fa75788\" (UID: \"6e96674e-4a20-4d8f-9118-90532fa75788\") " Dec 03 12:36:10 crc kubenswrapper[5052]: I1203 12:36:10.263866 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/6e96674e-4a20-4d8f-9118-90532fa75788-httpd-run\") pod \"6e96674e-4a20-4d8f-9118-90532fa75788\" (UID: \"6e96674e-4a20-4d8f-9118-90532fa75788\") " Dec 03 12:36:10 crc kubenswrapper[5052]: I1203 12:36:10.263885 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6e96674e-4a20-4d8f-9118-90532fa75788-config-data\") pod \"6e96674e-4a20-4d8f-9118-90532fa75788\" (UID: \"6e96674e-4a20-4d8f-9118-90532fa75788\") " Dec 03 12:36:10 crc kubenswrapper[5052]: I1203 12:36:10.263940 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/6e96674e-4a20-4d8f-9118-90532fa75788-scripts\") pod \"6e96674e-4a20-4d8f-9118-90532fa75788\" (UID: \"6e96674e-4a20-4d8f-9118-90532fa75788\") " Dec 03 12:36:10 crc kubenswrapper[5052]: I1203 12:36:10.264039 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e96674e-4a20-4d8f-9118-90532fa75788-logs" (OuterVolumeSpecName: "logs") pod "6e96674e-4a20-4d8f-9118-90532fa75788" (UID: "6e96674e-4a20-4d8f-9118-90532fa75788"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:36:10 crc kubenswrapper[5052]: I1203 12:36:10.264316 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/6e96674e-4a20-4d8f-9118-90532fa75788-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:36:10 crc kubenswrapper[5052]: I1203 12:36:10.265046 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e96674e-4a20-4d8f-9118-90532fa75788-httpd-run" (OuterVolumeSpecName: "httpd-run") pod "6e96674e-4a20-4d8f-9118-90532fa75788" (UID: "6e96674e-4a20-4d8f-9118-90532fa75788"). InnerVolumeSpecName "httpd-run". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:36:10 crc kubenswrapper[5052]: I1203 12:36:10.268352 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6e96674e-4a20-4d8f-9118-90532fa75788-kube-api-access-h87dc" (OuterVolumeSpecName: "kube-api-access-h87dc") pod "6e96674e-4a20-4d8f-9118-90532fa75788" (UID: "6e96674e-4a20-4d8f-9118-90532fa75788"). InnerVolumeSpecName "kube-api-access-h87dc". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:36:10 crc kubenswrapper[5052]: I1203 12:36:10.274731 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage12-crc" (OuterVolumeSpecName: "glance") pod "6e96674e-4a20-4d8f-9118-90532fa75788" (UID: "6e96674e-4a20-4d8f-9118-90532fa75788"). InnerVolumeSpecName "local-storage12-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 12:36:10 crc kubenswrapper[5052]: I1203 12:36:10.274693 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6e96674e-4a20-4d8f-9118-90532fa75788-scripts" (OuterVolumeSpecName: "scripts") pod "6e96674e-4a20-4d8f-9118-90532fa75788" (UID: "6e96674e-4a20-4d8f-9118-90532fa75788"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:36:10 crc kubenswrapper[5052]: I1203 12:36:10.296973 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6e96674e-4a20-4d8f-9118-90532fa75788-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "6e96674e-4a20-4d8f-9118-90532fa75788" (UID: "6e96674e-4a20-4d8f-9118-90532fa75788"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:36:10 crc kubenswrapper[5052]: I1203 12:36:10.320549 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6e96674e-4a20-4d8f-9118-90532fa75788-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "6e96674e-4a20-4d8f-9118-90532fa75788" (UID: "6e96674e-4a20-4d8f-9118-90532fa75788"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:36:10 crc kubenswrapper[5052]: I1203 12:36:10.336646 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6e96674e-4a20-4d8f-9118-90532fa75788-config-data" (OuterVolumeSpecName: "config-data") pod "6e96674e-4a20-4d8f-9118-90532fa75788" (UID: "6e96674e-4a20-4d8f-9118-90532fa75788"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:36:10 crc kubenswrapper[5052]: I1203 12:36:10.366418 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/6e96674e-4a20-4d8f-9118-90532fa75788-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:36:10 crc kubenswrapper[5052]: I1203 12:36:10.366448 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/6e96674e-4a20-4d8f-9118-90532fa75788-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:36:10 crc kubenswrapper[5052]: I1203 12:36:10.366488 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage12-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage12-crc\") on node \"crc\" " Dec 03 12:36:10 crc kubenswrapper[5052]: I1203 12:36:10.366502 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6e96674e-4a20-4d8f-9118-90532fa75788-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:36:10 crc kubenswrapper[5052]: I1203 12:36:10.366559 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-h87dc\" (UniqueName: \"kubernetes.io/projected/6e96674e-4a20-4d8f-9118-90532fa75788-kube-api-access-h87dc\") on node \"crc\" DevicePath \"\"" Dec 03 12:36:10 crc kubenswrapper[5052]: I1203 12:36:10.366572 5052 reconciler_common.go:293] "Volume detached for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/6e96674e-4a20-4d8f-9118-90532fa75788-httpd-run\") on node \"crc\" DevicePath \"\"" Dec 03 12:36:10 crc kubenswrapper[5052]: I1203 12:36:10.366582 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6e96674e-4a20-4d8f-9118-90532fa75788-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:36:10 crc kubenswrapper[5052]: I1203 12:36:10.385461 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage12-crc" (UniqueName: "kubernetes.io/local-volume/local-storage12-crc") on node "crc" Dec 03 12:36:10 crc kubenswrapper[5052]: I1203 12:36:10.452080 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:36:10 crc kubenswrapper[5052]: I1203 12:36:10.460357 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:36:10 crc kubenswrapper[5052]: I1203 12:36:10.468230 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage12-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage12-crc\") on node \"crc\" DevicePath \"\"" Dec 03 12:36:10 crc kubenswrapper[5052]: I1203 12:36:10.469776 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:36:10 crc kubenswrapper[5052]: E1203 12:36:10.470163 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="6e96674e-4a20-4d8f-9118-90532fa75788" containerName="glance-httpd" Dec 03 12:36:10 crc kubenswrapper[5052]: I1203 12:36:10.470182 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="6e96674e-4a20-4d8f-9118-90532fa75788" containerName="glance-httpd" Dec 03 12:36:10 crc kubenswrapper[5052]: E1203 12:36:10.470215 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="6e96674e-4a20-4d8f-9118-90532fa75788" containerName="glance-log" Dec 03 12:36:10 crc kubenswrapper[5052]: I1203 12:36:10.470221 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="6e96674e-4a20-4d8f-9118-90532fa75788" containerName="glance-log" Dec 03 12:36:10 crc kubenswrapper[5052]: I1203 12:36:10.470387 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="6e96674e-4a20-4d8f-9118-90532fa75788" containerName="glance-log" Dec 03 12:36:10 crc kubenswrapper[5052]: I1203 12:36:10.470401 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="6e96674e-4a20-4d8f-9118-90532fa75788" containerName="glance-httpd" Dec 03 12:36:10 crc kubenswrapper[5052]: I1203 12:36:10.471439 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:36:10 crc kubenswrapper[5052]: I1203 12:36:10.473901 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-glance-default-public-svc" Dec 03 12:36:10 crc kubenswrapper[5052]: I1203 12:36:10.474206 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-default-external-config-data" Dec 03 12:36:10 crc kubenswrapper[5052]: I1203 12:36:10.485333 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:36:10 crc kubenswrapper[5052]: I1203 12:36:10.570396 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/9dccead0-7ff2-4255-ac00-10a9b357c2db-logs\") pod \"glance-default-external-api-0\" (UID: \"9dccead0-7ff2-4255-ac00-10a9b357c2db\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:36:10 crc kubenswrapper[5052]: I1203 12:36:10.570475 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9dccead0-7ff2-4255-ac00-10a9b357c2db-combined-ca-bundle\") pod \"glance-default-external-api-0\" (UID: \"9dccead0-7ff2-4255-ac00-10a9b357c2db\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:36:10 crc kubenswrapper[5052]: I1203 12:36:10.570525 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/9dccead0-7ff2-4255-ac00-10a9b357c2db-config-data\") pod \"glance-default-external-api-0\" (UID: \"9dccead0-7ff2-4255-ac00-10a9b357c2db\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:36:10 crc kubenswrapper[5052]: I1203 12:36:10.570547 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/9dccead0-7ff2-4255-ac00-10a9b357c2db-public-tls-certs\") pod \"glance-default-external-api-0\" (UID: \"9dccead0-7ff2-4255-ac00-10a9b357c2db\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:36:10 crc kubenswrapper[5052]: I1203 12:36:10.570768 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage12-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage12-crc\") pod \"glance-default-external-api-0\" (UID: \"9dccead0-7ff2-4255-ac00-10a9b357c2db\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:36:10 crc kubenswrapper[5052]: I1203 12:36:10.570838 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-phrpm\" (UniqueName: \"kubernetes.io/projected/9dccead0-7ff2-4255-ac00-10a9b357c2db-kube-api-access-phrpm\") pod \"glance-default-external-api-0\" (UID: \"9dccead0-7ff2-4255-ac00-10a9b357c2db\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:36:10 crc kubenswrapper[5052]: I1203 12:36:10.570924 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/9dccead0-7ff2-4255-ac00-10a9b357c2db-scripts\") pod \"glance-default-external-api-0\" (UID: \"9dccead0-7ff2-4255-ac00-10a9b357c2db\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:36:10 crc kubenswrapper[5052]: I1203 12:36:10.570950 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/9dccead0-7ff2-4255-ac00-10a9b357c2db-httpd-run\") pod \"glance-default-external-api-0\" (UID: \"9dccead0-7ff2-4255-ac00-10a9b357c2db\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:36:10 crc kubenswrapper[5052]: I1203 12:36:10.616730 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/glance-default-internal-api-0" podUID="25498b02-c179-4555-ad04-1ef6445df4ab" containerName="glance-httpd" probeResult="failure" output="Get \"https://10.217.1.39:9292/healthcheck\": read tcp 10.217.0.2:53750->10.217.1.39:9292: read: connection reset by peer" Dec 03 12:36:10 crc kubenswrapper[5052]: I1203 12:36:10.616730 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/glance-default-internal-api-0" podUID="25498b02-c179-4555-ad04-1ef6445df4ab" containerName="glance-log" probeResult="failure" output="Get \"https://10.217.1.39:9292/healthcheck\": read tcp 10.217.0.2:53754->10.217.1.39:9292: read: connection reset by peer" Dec 03 12:36:10 crc kubenswrapper[5052]: I1203 12:36:10.673283 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/9dccead0-7ff2-4255-ac00-10a9b357c2db-logs\") pod \"glance-default-external-api-0\" (UID: \"9dccead0-7ff2-4255-ac00-10a9b357c2db\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:36:10 crc kubenswrapper[5052]: I1203 12:36:10.673359 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9dccead0-7ff2-4255-ac00-10a9b357c2db-combined-ca-bundle\") pod \"glance-default-external-api-0\" (UID: \"9dccead0-7ff2-4255-ac00-10a9b357c2db\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:36:10 crc kubenswrapper[5052]: I1203 12:36:10.673397 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/9dccead0-7ff2-4255-ac00-10a9b357c2db-config-data\") pod \"glance-default-external-api-0\" (UID: \"9dccead0-7ff2-4255-ac00-10a9b357c2db\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:36:10 crc kubenswrapper[5052]: I1203 12:36:10.673424 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/9dccead0-7ff2-4255-ac00-10a9b357c2db-public-tls-certs\") pod \"glance-default-external-api-0\" (UID: \"9dccead0-7ff2-4255-ac00-10a9b357c2db\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:36:10 crc kubenswrapper[5052]: I1203 12:36:10.673536 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage12-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage12-crc\") pod \"glance-default-external-api-0\" (UID: \"9dccead0-7ff2-4255-ac00-10a9b357c2db\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:36:10 crc kubenswrapper[5052]: I1203 12:36:10.673566 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-phrpm\" (UniqueName: \"kubernetes.io/projected/9dccead0-7ff2-4255-ac00-10a9b357c2db-kube-api-access-phrpm\") pod \"glance-default-external-api-0\" (UID: \"9dccead0-7ff2-4255-ac00-10a9b357c2db\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:36:10 crc kubenswrapper[5052]: I1203 12:36:10.673611 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/9dccead0-7ff2-4255-ac00-10a9b357c2db-scripts\") pod \"glance-default-external-api-0\" (UID: \"9dccead0-7ff2-4255-ac00-10a9b357c2db\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:36:10 crc kubenswrapper[5052]: I1203 12:36:10.673636 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/9dccead0-7ff2-4255-ac00-10a9b357c2db-httpd-run\") pod \"glance-default-external-api-0\" (UID: \"9dccead0-7ff2-4255-ac00-10a9b357c2db\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:36:10 crc kubenswrapper[5052]: I1203 12:36:10.673857 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/9dccead0-7ff2-4255-ac00-10a9b357c2db-logs\") pod \"glance-default-external-api-0\" (UID: \"9dccead0-7ff2-4255-ac00-10a9b357c2db\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:36:10 crc kubenswrapper[5052]: I1203 12:36:10.673975 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/9dccead0-7ff2-4255-ac00-10a9b357c2db-httpd-run\") pod \"glance-default-external-api-0\" (UID: \"9dccead0-7ff2-4255-ac00-10a9b357c2db\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:36:10 crc kubenswrapper[5052]: I1203 12:36:10.674096 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage12-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage12-crc\") pod \"glance-default-external-api-0\" (UID: \"9dccead0-7ff2-4255-ac00-10a9b357c2db\") device mount path \"/mnt/openstack/pv12\"" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:36:10 crc kubenswrapper[5052]: I1203 12:36:10.679155 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/9dccead0-7ff2-4255-ac00-10a9b357c2db-public-tls-certs\") pod \"glance-default-external-api-0\" (UID: \"9dccead0-7ff2-4255-ac00-10a9b357c2db\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:36:10 crc kubenswrapper[5052]: I1203 12:36:10.679267 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/9dccead0-7ff2-4255-ac00-10a9b357c2db-scripts\") pod \"glance-default-external-api-0\" (UID: \"9dccead0-7ff2-4255-ac00-10a9b357c2db\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:36:10 crc kubenswrapper[5052]: I1203 12:36:10.681399 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9dccead0-7ff2-4255-ac00-10a9b357c2db-combined-ca-bundle\") pod \"glance-default-external-api-0\" (UID: \"9dccead0-7ff2-4255-ac00-10a9b357c2db\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:36:10 crc kubenswrapper[5052]: I1203 12:36:10.691531 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/9dccead0-7ff2-4255-ac00-10a9b357c2db-config-data\") pod \"glance-default-external-api-0\" (UID: \"9dccead0-7ff2-4255-ac00-10a9b357c2db\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:36:10 crc kubenswrapper[5052]: I1203 12:36:10.693140 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-phrpm\" (UniqueName: \"kubernetes.io/projected/9dccead0-7ff2-4255-ac00-10a9b357c2db-kube-api-access-phrpm\") pod \"glance-default-external-api-0\" (UID: \"9dccead0-7ff2-4255-ac00-10a9b357c2db\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:36:10 crc kubenswrapper[5052]: I1203 12:36:10.713246 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage12-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage12-crc\") pod \"glance-default-external-api-0\" (UID: \"9dccead0-7ff2-4255-ac00-10a9b357c2db\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:36:10 crc kubenswrapper[5052]: I1203 12:36:10.842911 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:36:11 crc kubenswrapper[5052]: I1203 12:36:11.076812 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:36:11 crc kubenswrapper[5052]: I1203 12:36:11.134016 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-conductor-0" event={"ID":"72a59b27-7692-41b5-96d1-59ffa7d5858a","Type":"ContainerStarted","Data":"23a9059e0881bace3a1446302321f9b380f5df1f67c2f1949c4627c97df7dceb"} Dec 03 12:36:11 crc kubenswrapper[5052]: I1203 12:36:11.134114 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:36:11 crc kubenswrapper[5052]: I1203 12:36:11.140797 5052 generic.go:334] "Generic (PLEG): container finished" podID="25498b02-c179-4555-ad04-1ef6445df4ab" containerID="7e310e8333b7259c202284ca732fa44b22dde345a4ca324e0799f37aa77c10db" exitCode=0 Dec 03 12:36:11 crc kubenswrapper[5052]: I1203 12:36:11.140879 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:36:11 crc kubenswrapper[5052]: I1203 12:36:11.140908 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"25498b02-c179-4555-ad04-1ef6445df4ab","Type":"ContainerDied","Data":"7e310e8333b7259c202284ca732fa44b22dde345a4ca324e0799f37aa77c10db"} Dec 03 12:36:11 crc kubenswrapper[5052]: I1203 12:36:11.140943 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"25498b02-c179-4555-ad04-1ef6445df4ab","Type":"ContainerDied","Data":"32653e85f448eef3d2da7d0786700be61d651c987dd3d9b0c1da81c0cd2bfba5"} Dec 03 12:36:11 crc kubenswrapper[5052]: I1203 12:36:11.140966 5052 scope.go:117] "RemoveContainer" containerID="7e310e8333b7259c202284ca732fa44b22dde345a4ca324e0799f37aa77c10db" Dec 03 12:36:11 crc kubenswrapper[5052]: I1203 12:36:11.155975 5052 generic.go:334] "Generic (PLEG): container finished" podID="ed1c5151-73cb-4c20-b801-cf77509c7088" containerID="41dc33620881a66066ffa87ac1dc04a00aba4a0067bba8e0d874f761f849a738" exitCode=0 Dec 03 12:36:11 crc kubenswrapper[5052]: I1203 12:36:11.156375 5052 generic.go:334] "Generic (PLEG): container finished" podID="ed1c5151-73cb-4c20-b801-cf77509c7088" containerID="3300d0f26c3e473950a0c44c09d7cd9b59119f7fd171515efd3ee4a1105f3249" exitCode=2 Dec 03 12:36:11 crc kubenswrapper[5052]: I1203 12:36:11.156395 5052 generic.go:334] "Generic (PLEG): container finished" podID="ed1c5151-73cb-4c20-b801-cf77509c7088" containerID="3fdeef99541d7da1c01a8d9a5170a7f180d3f1cc9d21e3baa51812d4cb02033c" exitCode=0 Dec 03 12:36:11 crc kubenswrapper[5052]: I1203 12:36:11.156467 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"ed1c5151-73cb-4c20-b801-cf77509c7088","Type":"ContainerDied","Data":"41dc33620881a66066ffa87ac1dc04a00aba4a0067bba8e0d874f761f849a738"} Dec 03 12:36:11 crc kubenswrapper[5052]: I1203 12:36:11.156499 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"ed1c5151-73cb-4c20-b801-cf77509c7088","Type":"ContainerDied","Data":"3300d0f26c3e473950a0c44c09d7cd9b59119f7fd171515efd3ee4a1105f3249"} Dec 03 12:36:11 crc kubenswrapper[5052]: I1203 12:36:11.156541 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"ed1c5151-73cb-4c20-b801-cf77509c7088","Type":"ContainerDied","Data":"3fdeef99541d7da1c01a8d9a5170a7f180d3f1cc9d21e3baa51812d4cb02033c"} Dec 03 12:36:11 crc kubenswrapper[5052]: I1203 12:36:11.170918 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-cell0-conductor-0" podStartSLOduration=2.170895815 podStartE2EDuration="2.170895815s" podCreationTimestamp="2025-12-03 12:36:09 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:36:11.155129034 +0000 UTC m=+4561.743370798" watchObservedRunningTime="2025-12-03 12:36:11.170895815 +0000 UTC m=+4561.759137579" Dec 03 12:36:11 crc kubenswrapper[5052]: I1203 12:36:11.186686 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/25498b02-c179-4555-ad04-1ef6445df4ab-combined-ca-bundle\") pod \"25498b02-c179-4555-ad04-1ef6445df4ab\" (UID: \"25498b02-c179-4555-ad04-1ef6445df4ab\") " Dec 03 12:36:11 crc kubenswrapper[5052]: I1203 12:36:11.186857 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/25498b02-c179-4555-ad04-1ef6445df4ab-internal-tls-certs\") pod \"25498b02-c179-4555-ad04-1ef6445df4ab\" (UID: \"25498b02-c179-4555-ad04-1ef6445df4ab\") " Dec 03 12:36:11 crc kubenswrapper[5052]: I1203 12:36:11.186977 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/25498b02-c179-4555-ad04-1ef6445df4ab-config-data\") pod \"25498b02-c179-4555-ad04-1ef6445df4ab\" (UID: \"25498b02-c179-4555-ad04-1ef6445df4ab\") " Dec 03 12:36:11 crc kubenswrapper[5052]: I1203 12:36:11.187048 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"glance\" (UniqueName: \"kubernetes.io/local-volume/local-storage18-crc\") pod \"25498b02-c179-4555-ad04-1ef6445df4ab\" (UID: \"25498b02-c179-4555-ad04-1ef6445df4ab\") " Dec 03 12:36:11 crc kubenswrapper[5052]: I1203 12:36:11.187220 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mbwvx\" (UniqueName: \"kubernetes.io/projected/25498b02-c179-4555-ad04-1ef6445df4ab-kube-api-access-mbwvx\") pod \"25498b02-c179-4555-ad04-1ef6445df4ab\" (UID: \"25498b02-c179-4555-ad04-1ef6445df4ab\") " Dec 03 12:36:11 crc kubenswrapper[5052]: I1203 12:36:11.187329 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/25498b02-c179-4555-ad04-1ef6445df4ab-httpd-run\") pod \"25498b02-c179-4555-ad04-1ef6445df4ab\" (UID: \"25498b02-c179-4555-ad04-1ef6445df4ab\") " Dec 03 12:36:11 crc kubenswrapper[5052]: I1203 12:36:11.187414 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/25498b02-c179-4555-ad04-1ef6445df4ab-scripts\") pod \"25498b02-c179-4555-ad04-1ef6445df4ab\" (UID: \"25498b02-c179-4555-ad04-1ef6445df4ab\") " Dec 03 12:36:11 crc kubenswrapper[5052]: I1203 12:36:11.187572 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/25498b02-c179-4555-ad04-1ef6445df4ab-logs\") pod \"25498b02-c179-4555-ad04-1ef6445df4ab\" (UID: \"25498b02-c179-4555-ad04-1ef6445df4ab\") " Dec 03 12:36:11 crc kubenswrapper[5052]: I1203 12:36:11.188695 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/25498b02-c179-4555-ad04-1ef6445df4ab-logs" (OuterVolumeSpecName: "logs") pod "25498b02-c179-4555-ad04-1ef6445df4ab" (UID: "25498b02-c179-4555-ad04-1ef6445df4ab"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:36:11 crc kubenswrapper[5052]: I1203 12:36:11.188765 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/25498b02-c179-4555-ad04-1ef6445df4ab-httpd-run" (OuterVolumeSpecName: "httpd-run") pod "25498b02-c179-4555-ad04-1ef6445df4ab" (UID: "25498b02-c179-4555-ad04-1ef6445df4ab"). InnerVolumeSpecName "httpd-run". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:36:11 crc kubenswrapper[5052]: I1203 12:36:11.189187 5052 scope.go:117] "RemoveContainer" containerID="b1eb3705336137fdb58a75000976f8917d3ae6bc93b6459c35f5a5a4494f5dc5" Dec 03 12:36:11 crc kubenswrapper[5052]: I1203 12:36:11.193968 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage18-crc" (OuterVolumeSpecName: "glance") pod "25498b02-c179-4555-ad04-1ef6445df4ab" (UID: "25498b02-c179-4555-ad04-1ef6445df4ab"). InnerVolumeSpecName "local-storage18-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 12:36:11 crc kubenswrapper[5052]: I1203 12:36:11.194652 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/25498b02-c179-4555-ad04-1ef6445df4ab-scripts" (OuterVolumeSpecName: "scripts") pod "25498b02-c179-4555-ad04-1ef6445df4ab" (UID: "25498b02-c179-4555-ad04-1ef6445df4ab"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:36:11 crc kubenswrapper[5052]: I1203 12:36:11.194916 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/25498b02-c179-4555-ad04-1ef6445df4ab-kube-api-access-mbwvx" (OuterVolumeSpecName: "kube-api-access-mbwvx") pod "25498b02-c179-4555-ad04-1ef6445df4ab" (UID: "25498b02-c179-4555-ad04-1ef6445df4ab"). InnerVolumeSpecName "kube-api-access-mbwvx". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:36:11 crc kubenswrapper[5052]: I1203 12:36:11.218888 5052 scope.go:117] "RemoveContainer" containerID="7e310e8333b7259c202284ca732fa44b22dde345a4ca324e0799f37aa77c10db" Dec 03 12:36:11 crc kubenswrapper[5052]: E1203 12:36:11.219459 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7e310e8333b7259c202284ca732fa44b22dde345a4ca324e0799f37aa77c10db\": container with ID starting with 7e310e8333b7259c202284ca732fa44b22dde345a4ca324e0799f37aa77c10db not found: ID does not exist" containerID="7e310e8333b7259c202284ca732fa44b22dde345a4ca324e0799f37aa77c10db" Dec 03 12:36:11 crc kubenswrapper[5052]: I1203 12:36:11.219554 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7e310e8333b7259c202284ca732fa44b22dde345a4ca324e0799f37aa77c10db"} err="failed to get container status \"7e310e8333b7259c202284ca732fa44b22dde345a4ca324e0799f37aa77c10db\": rpc error: code = NotFound desc = could not find container \"7e310e8333b7259c202284ca732fa44b22dde345a4ca324e0799f37aa77c10db\": container with ID starting with 7e310e8333b7259c202284ca732fa44b22dde345a4ca324e0799f37aa77c10db not found: ID does not exist" Dec 03 12:36:11 crc kubenswrapper[5052]: I1203 12:36:11.219588 5052 scope.go:117] "RemoveContainer" containerID="b1eb3705336137fdb58a75000976f8917d3ae6bc93b6459c35f5a5a4494f5dc5" Dec 03 12:36:11 crc kubenswrapper[5052]: E1203 12:36:11.220134 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b1eb3705336137fdb58a75000976f8917d3ae6bc93b6459c35f5a5a4494f5dc5\": container with ID starting with b1eb3705336137fdb58a75000976f8917d3ae6bc93b6459c35f5a5a4494f5dc5 not found: ID does not exist" containerID="b1eb3705336137fdb58a75000976f8917d3ae6bc93b6459c35f5a5a4494f5dc5" Dec 03 12:36:11 crc kubenswrapper[5052]: I1203 12:36:11.220203 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b1eb3705336137fdb58a75000976f8917d3ae6bc93b6459c35f5a5a4494f5dc5"} err="failed to get container status \"b1eb3705336137fdb58a75000976f8917d3ae6bc93b6459c35f5a5a4494f5dc5\": rpc error: code = NotFound desc = could not find container \"b1eb3705336137fdb58a75000976f8917d3ae6bc93b6459c35f5a5a4494f5dc5\": container with ID starting with b1eb3705336137fdb58a75000976f8917d3ae6bc93b6459c35f5a5a4494f5dc5 not found: ID does not exist" Dec 03 12:36:11 crc kubenswrapper[5052]: I1203 12:36:11.240790 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/25498b02-c179-4555-ad04-1ef6445df4ab-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "25498b02-c179-4555-ad04-1ef6445df4ab" (UID: "25498b02-c179-4555-ad04-1ef6445df4ab"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:36:11 crc kubenswrapper[5052]: I1203 12:36:11.258229 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/25498b02-c179-4555-ad04-1ef6445df4ab-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "25498b02-c179-4555-ad04-1ef6445df4ab" (UID: "25498b02-c179-4555-ad04-1ef6445df4ab"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:36:11 crc kubenswrapper[5052]: I1203 12:36:11.276697 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/25498b02-c179-4555-ad04-1ef6445df4ab-config-data" (OuterVolumeSpecName: "config-data") pod "25498b02-c179-4555-ad04-1ef6445df4ab" (UID: "25498b02-c179-4555-ad04-1ef6445df4ab"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:36:11 crc kubenswrapper[5052]: I1203 12:36:11.289489 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/25498b02-c179-4555-ad04-1ef6445df4ab-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:36:11 crc kubenswrapper[5052]: I1203 12:36:11.289548 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage18-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage18-crc\") on node \"crc\" " Dec 03 12:36:11 crc kubenswrapper[5052]: I1203 12:36:11.289560 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-mbwvx\" (UniqueName: \"kubernetes.io/projected/25498b02-c179-4555-ad04-1ef6445df4ab-kube-api-access-mbwvx\") on node \"crc\" DevicePath \"\"" Dec 03 12:36:11 crc kubenswrapper[5052]: I1203 12:36:11.289569 5052 reconciler_common.go:293] "Volume detached for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/25498b02-c179-4555-ad04-1ef6445df4ab-httpd-run\") on node \"crc\" DevicePath \"\"" Dec 03 12:36:11 crc kubenswrapper[5052]: I1203 12:36:11.289578 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/25498b02-c179-4555-ad04-1ef6445df4ab-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:36:11 crc kubenswrapper[5052]: I1203 12:36:11.289585 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/25498b02-c179-4555-ad04-1ef6445df4ab-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:36:11 crc kubenswrapper[5052]: I1203 12:36:11.289595 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/25498b02-c179-4555-ad04-1ef6445df4ab-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:36:11 crc kubenswrapper[5052]: I1203 12:36:11.289606 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/25498b02-c179-4555-ad04-1ef6445df4ab-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:36:11 crc kubenswrapper[5052]: I1203 12:36:11.308000 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage18-crc" (UniqueName: "kubernetes.io/local-volume/local-storage18-crc") on node "crc" Dec 03 12:36:11 crc kubenswrapper[5052]: I1203 12:36:11.391727 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage18-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage18-crc\") on node \"crc\" DevicePath \"\"" Dec 03 12:36:11 crc kubenswrapper[5052]: W1203 12:36:11.398449 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9dccead0_7ff2_4255_ac00_10a9b357c2db.slice/crio-99a719d77c1c2a1f956f5aba0b3d775f4d82fe13fa8a0f238262600b3e4746e4 WatchSource:0}: Error finding container 99a719d77c1c2a1f956f5aba0b3d775f4d82fe13fa8a0f238262600b3e4746e4: Status 404 returned error can't find the container with id 99a719d77c1c2a1f956f5aba0b3d775f4d82fe13fa8a0f238262600b3e4746e4 Dec 03 12:36:11 crc kubenswrapper[5052]: I1203 12:36:11.399147 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:36:11 crc kubenswrapper[5052]: I1203 12:36:11.585786 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:36:11 crc kubenswrapper[5052]: I1203 12:36:11.599697 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:36:11 crc kubenswrapper[5052]: I1203 12:36:11.616970 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:36:11 crc kubenswrapper[5052]: E1203 12:36:11.617476 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="25498b02-c179-4555-ad04-1ef6445df4ab" containerName="glance-httpd" Dec 03 12:36:11 crc kubenswrapper[5052]: I1203 12:36:11.617497 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="25498b02-c179-4555-ad04-1ef6445df4ab" containerName="glance-httpd" Dec 03 12:36:11 crc kubenswrapper[5052]: E1203 12:36:11.617540 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="25498b02-c179-4555-ad04-1ef6445df4ab" containerName="glance-log" Dec 03 12:36:11 crc kubenswrapper[5052]: I1203 12:36:11.617550 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="25498b02-c179-4555-ad04-1ef6445df4ab" containerName="glance-log" Dec 03 12:36:11 crc kubenswrapper[5052]: I1203 12:36:11.617784 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="25498b02-c179-4555-ad04-1ef6445df4ab" containerName="glance-httpd" Dec 03 12:36:11 crc kubenswrapper[5052]: I1203 12:36:11.617807 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="25498b02-c179-4555-ad04-1ef6445df4ab" containerName="glance-log" Dec 03 12:36:11 crc kubenswrapper[5052]: I1203 12:36:11.619001 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:36:11 crc kubenswrapper[5052]: I1203 12:36:11.624402 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-glance-default-internal-svc" Dec 03 12:36:11 crc kubenswrapper[5052]: I1203 12:36:11.628799 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:36:11 crc kubenswrapper[5052]: I1203 12:36:11.628880 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-default-internal-config-data" Dec 03 12:36:11 crc kubenswrapper[5052]: I1203 12:36:11.684757 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="25498b02-c179-4555-ad04-1ef6445df4ab" path="/var/lib/kubelet/pods/25498b02-c179-4555-ad04-1ef6445df4ab/volumes" Dec 03 12:36:11 crc kubenswrapper[5052]: I1203 12:36:11.686198 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6e96674e-4a20-4d8f-9118-90532fa75788" path="/var/lib/kubelet/pods/6e96674e-4a20-4d8f-9118-90532fa75788/volumes" Dec 03 12:36:11 crc kubenswrapper[5052]: I1203 12:36:11.798135 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/42e9c6bc-62d9-442a-a5f0-d885eff829cb-combined-ca-bundle\") pod \"glance-default-internal-api-0\" (UID: \"42e9c6bc-62d9-442a-a5f0-d885eff829cb\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:36:11 crc kubenswrapper[5052]: I1203 12:36:11.798183 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage18-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage18-crc\") pod \"glance-default-internal-api-0\" (UID: \"42e9c6bc-62d9-442a-a5f0-d885eff829cb\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:36:11 crc kubenswrapper[5052]: I1203 12:36:11.798214 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/42e9c6bc-62d9-442a-a5f0-d885eff829cb-internal-tls-certs\") pod \"glance-default-internal-api-0\" (UID: \"42e9c6bc-62d9-442a-a5f0-d885eff829cb\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:36:11 crc kubenswrapper[5052]: I1203 12:36:11.798241 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/42e9c6bc-62d9-442a-a5f0-d885eff829cb-logs\") pod \"glance-default-internal-api-0\" (UID: \"42e9c6bc-62d9-442a-a5f0-d885eff829cb\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:36:11 crc kubenswrapper[5052]: I1203 12:36:11.798268 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/42e9c6bc-62d9-442a-a5f0-d885eff829cb-config-data\") pod \"glance-default-internal-api-0\" (UID: \"42e9c6bc-62d9-442a-a5f0-d885eff829cb\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:36:11 crc kubenswrapper[5052]: I1203 12:36:11.798321 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/42e9c6bc-62d9-442a-a5f0-d885eff829cb-httpd-run\") pod \"glance-default-internal-api-0\" (UID: \"42e9c6bc-62d9-442a-a5f0-d885eff829cb\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:36:11 crc kubenswrapper[5052]: I1203 12:36:11.798426 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/42e9c6bc-62d9-442a-a5f0-d885eff829cb-scripts\") pod \"glance-default-internal-api-0\" (UID: \"42e9c6bc-62d9-442a-a5f0-d885eff829cb\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:36:11 crc kubenswrapper[5052]: I1203 12:36:11.799423 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kmsm9\" (UniqueName: \"kubernetes.io/projected/42e9c6bc-62d9-442a-a5f0-d885eff829cb-kube-api-access-kmsm9\") pod \"glance-default-internal-api-0\" (UID: \"42e9c6bc-62d9-442a-a5f0-d885eff829cb\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:36:11 crc kubenswrapper[5052]: I1203 12:36:11.901542 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/42e9c6bc-62d9-442a-a5f0-d885eff829cb-scripts\") pod \"glance-default-internal-api-0\" (UID: \"42e9c6bc-62d9-442a-a5f0-d885eff829cb\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:36:11 crc kubenswrapper[5052]: I1203 12:36:11.901915 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-kmsm9\" (UniqueName: \"kubernetes.io/projected/42e9c6bc-62d9-442a-a5f0-d885eff829cb-kube-api-access-kmsm9\") pod \"glance-default-internal-api-0\" (UID: \"42e9c6bc-62d9-442a-a5f0-d885eff829cb\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:36:11 crc kubenswrapper[5052]: I1203 12:36:11.902029 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/42e9c6bc-62d9-442a-a5f0-d885eff829cb-combined-ca-bundle\") pod \"glance-default-internal-api-0\" (UID: \"42e9c6bc-62d9-442a-a5f0-d885eff829cb\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:36:11 crc kubenswrapper[5052]: I1203 12:36:11.902113 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage18-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage18-crc\") pod \"glance-default-internal-api-0\" (UID: \"42e9c6bc-62d9-442a-a5f0-d885eff829cb\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:36:11 crc kubenswrapper[5052]: I1203 12:36:11.902206 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/42e9c6bc-62d9-442a-a5f0-d885eff829cb-internal-tls-certs\") pod \"glance-default-internal-api-0\" (UID: \"42e9c6bc-62d9-442a-a5f0-d885eff829cb\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:36:11 crc kubenswrapper[5052]: I1203 12:36:11.902285 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/42e9c6bc-62d9-442a-a5f0-d885eff829cb-logs\") pod \"glance-default-internal-api-0\" (UID: \"42e9c6bc-62d9-442a-a5f0-d885eff829cb\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:36:11 crc kubenswrapper[5052]: I1203 12:36:11.902361 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/42e9c6bc-62d9-442a-a5f0-d885eff829cb-config-data\") pod \"glance-default-internal-api-0\" (UID: \"42e9c6bc-62d9-442a-a5f0-d885eff829cb\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:36:11 crc kubenswrapper[5052]: I1203 12:36:11.902493 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/42e9c6bc-62d9-442a-a5f0-d885eff829cb-httpd-run\") pod \"glance-default-internal-api-0\" (UID: \"42e9c6bc-62d9-442a-a5f0-d885eff829cb\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:36:11 crc kubenswrapper[5052]: I1203 12:36:11.902769 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/42e9c6bc-62d9-442a-a5f0-d885eff829cb-logs\") pod \"glance-default-internal-api-0\" (UID: \"42e9c6bc-62d9-442a-a5f0-d885eff829cb\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:36:11 crc kubenswrapper[5052]: I1203 12:36:11.902397 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage18-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage18-crc\") pod \"glance-default-internal-api-0\" (UID: \"42e9c6bc-62d9-442a-a5f0-d885eff829cb\") device mount path \"/mnt/openstack/pv18\"" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:36:11 crc kubenswrapper[5052]: I1203 12:36:11.903054 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/42e9c6bc-62d9-442a-a5f0-d885eff829cb-httpd-run\") pod \"glance-default-internal-api-0\" (UID: \"42e9c6bc-62d9-442a-a5f0-d885eff829cb\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:36:11 crc kubenswrapper[5052]: I1203 12:36:11.906317 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/42e9c6bc-62d9-442a-a5f0-d885eff829cb-internal-tls-certs\") pod \"glance-default-internal-api-0\" (UID: \"42e9c6bc-62d9-442a-a5f0-d885eff829cb\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:36:11 crc kubenswrapper[5052]: I1203 12:36:11.906437 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/42e9c6bc-62d9-442a-a5f0-d885eff829cb-scripts\") pod \"glance-default-internal-api-0\" (UID: \"42e9c6bc-62d9-442a-a5f0-d885eff829cb\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:36:11 crc kubenswrapper[5052]: I1203 12:36:11.906446 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/42e9c6bc-62d9-442a-a5f0-d885eff829cb-combined-ca-bundle\") pod \"glance-default-internal-api-0\" (UID: \"42e9c6bc-62d9-442a-a5f0-d885eff829cb\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:36:11 crc kubenswrapper[5052]: I1203 12:36:11.920121 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/42e9c6bc-62d9-442a-a5f0-d885eff829cb-config-data\") pod \"glance-default-internal-api-0\" (UID: \"42e9c6bc-62d9-442a-a5f0-d885eff829cb\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:36:11 crc kubenswrapper[5052]: I1203 12:36:11.928272 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage18-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage18-crc\") pod \"glance-default-internal-api-0\" (UID: \"42e9c6bc-62d9-442a-a5f0-d885eff829cb\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:36:11 crc kubenswrapper[5052]: I1203 12:36:11.936256 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-kmsm9\" (UniqueName: \"kubernetes.io/projected/42e9c6bc-62d9-442a-a5f0-d885eff829cb-kube-api-access-kmsm9\") pod \"glance-default-internal-api-0\" (UID: \"42e9c6bc-62d9-442a-a5f0-d885eff829cb\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:36:11 crc kubenswrapper[5052]: I1203 12:36:11.947004 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:36:12 crc kubenswrapper[5052]: I1203 12:36:12.174300 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"9dccead0-7ff2-4255-ac00-10a9b357c2db","Type":"ContainerStarted","Data":"96e6068a8658ed240b81b5a1db15585f3efc1468e4c44b15dd5b3a8a124b6f8e"} Dec 03 12:36:12 crc kubenswrapper[5052]: I1203 12:36:12.174617 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"9dccead0-7ff2-4255-ac00-10a9b357c2db","Type":"ContainerStarted","Data":"99a719d77c1c2a1f956f5aba0b3d775f4d82fe13fa8a0f238262600b3e4746e4"} Dec 03 12:36:12 crc kubenswrapper[5052]: I1203 12:36:12.198704 5052 generic.go:334] "Generic (PLEG): container finished" podID="ed1c5151-73cb-4c20-b801-cf77509c7088" containerID="ba21a758c0eafa87981092d11a759465be8fe7d7c3f030cbfc2a0ec4049f8450" exitCode=0 Dec 03 12:36:12 crc kubenswrapper[5052]: I1203 12:36:12.198792 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"ed1c5151-73cb-4c20-b801-cf77509c7088","Type":"ContainerDied","Data":"ba21a758c0eafa87981092d11a759465be8fe7d7c3f030cbfc2a0ec4049f8450"} Dec 03 12:36:12 crc kubenswrapper[5052]: I1203 12:36:12.376165 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:36:12 crc kubenswrapper[5052]: W1203 12:36:12.387842 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod42e9c6bc_62d9_442a_a5f0_d885eff829cb.slice/crio-adca9569dc698688f1f02450e166a38ba90740456648576356a082d180255bd0 WatchSource:0}: Error finding container adca9569dc698688f1f02450e166a38ba90740456648576356a082d180255bd0: Status 404 returned error can't find the container with id adca9569dc698688f1f02450e166a38ba90740456648576356a082d180255bd0 Dec 03 12:36:12 crc kubenswrapper[5052]: I1203 12:36:12.450401 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:36:12 crc kubenswrapper[5052]: I1203 12:36:12.615647 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hp4j8\" (UniqueName: \"kubernetes.io/projected/ed1c5151-73cb-4c20-b801-cf77509c7088-kube-api-access-hp4j8\") pod \"ed1c5151-73cb-4c20-b801-cf77509c7088\" (UID: \"ed1c5151-73cb-4c20-b801-cf77509c7088\") " Dec 03 12:36:12 crc kubenswrapper[5052]: I1203 12:36:12.615702 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/ed1c5151-73cb-4c20-b801-cf77509c7088-scripts\") pod \"ed1c5151-73cb-4c20-b801-cf77509c7088\" (UID: \"ed1c5151-73cb-4c20-b801-cf77509c7088\") " Dec 03 12:36:12 crc kubenswrapper[5052]: I1203 12:36:12.616322 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/ed1c5151-73cb-4c20-b801-cf77509c7088-config-data\") pod \"ed1c5151-73cb-4c20-b801-cf77509c7088\" (UID: \"ed1c5151-73cb-4c20-b801-cf77509c7088\") " Dec 03 12:36:12 crc kubenswrapper[5052]: I1203 12:36:12.616416 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ed1c5151-73cb-4c20-b801-cf77509c7088-combined-ca-bundle\") pod \"ed1c5151-73cb-4c20-b801-cf77509c7088\" (UID: \"ed1c5151-73cb-4c20-b801-cf77509c7088\") " Dec 03 12:36:12 crc kubenswrapper[5052]: I1203 12:36:12.616581 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/ed1c5151-73cb-4c20-b801-cf77509c7088-log-httpd\") pod \"ed1c5151-73cb-4c20-b801-cf77509c7088\" (UID: \"ed1c5151-73cb-4c20-b801-cf77509c7088\") " Dec 03 12:36:12 crc kubenswrapper[5052]: I1203 12:36:12.616741 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/ed1c5151-73cb-4c20-b801-cf77509c7088-run-httpd\") pod \"ed1c5151-73cb-4c20-b801-cf77509c7088\" (UID: \"ed1c5151-73cb-4c20-b801-cf77509c7088\") " Dec 03 12:36:12 crc kubenswrapper[5052]: I1203 12:36:12.616869 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed1c5151-73cb-4c20-b801-cf77509c7088-log-httpd" (OuterVolumeSpecName: "log-httpd") pod "ed1c5151-73cb-4c20-b801-cf77509c7088" (UID: "ed1c5151-73cb-4c20-b801-cf77509c7088"). InnerVolumeSpecName "log-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:36:12 crc kubenswrapper[5052]: I1203 12:36:12.617026 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/ed1c5151-73cb-4c20-b801-cf77509c7088-sg-core-conf-yaml\") pod \"ed1c5151-73cb-4c20-b801-cf77509c7088\" (UID: \"ed1c5151-73cb-4c20-b801-cf77509c7088\") " Dec 03 12:36:12 crc kubenswrapper[5052]: I1203 12:36:12.617160 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed1c5151-73cb-4c20-b801-cf77509c7088-run-httpd" (OuterVolumeSpecName: "run-httpd") pod "ed1c5151-73cb-4c20-b801-cf77509c7088" (UID: "ed1c5151-73cb-4c20-b801-cf77509c7088"). InnerVolumeSpecName "run-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:36:12 crc kubenswrapper[5052]: I1203 12:36:12.617602 5052 reconciler_common.go:293] "Volume detached for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/ed1c5151-73cb-4c20-b801-cf77509c7088-log-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:36:12 crc kubenswrapper[5052]: I1203 12:36:12.617695 5052 reconciler_common.go:293] "Volume detached for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/ed1c5151-73cb-4c20-b801-cf77509c7088-run-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:36:12 crc kubenswrapper[5052]: I1203 12:36:12.619801 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ed1c5151-73cb-4c20-b801-cf77509c7088-scripts" (OuterVolumeSpecName: "scripts") pod "ed1c5151-73cb-4c20-b801-cf77509c7088" (UID: "ed1c5151-73cb-4c20-b801-cf77509c7088"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:36:12 crc kubenswrapper[5052]: I1203 12:36:12.619853 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ed1c5151-73cb-4c20-b801-cf77509c7088-kube-api-access-hp4j8" (OuterVolumeSpecName: "kube-api-access-hp4j8") pod "ed1c5151-73cb-4c20-b801-cf77509c7088" (UID: "ed1c5151-73cb-4c20-b801-cf77509c7088"). InnerVolumeSpecName "kube-api-access-hp4j8". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:36:12 crc kubenswrapper[5052]: I1203 12:36:12.644787 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ed1c5151-73cb-4c20-b801-cf77509c7088-sg-core-conf-yaml" (OuterVolumeSpecName: "sg-core-conf-yaml") pod "ed1c5151-73cb-4c20-b801-cf77509c7088" (UID: "ed1c5151-73cb-4c20-b801-cf77509c7088"). InnerVolumeSpecName "sg-core-conf-yaml". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:36:12 crc kubenswrapper[5052]: I1203 12:36:12.702946 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ed1c5151-73cb-4c20-b801-cf77509c7088-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "ed1c5151-73cb-4c20-b801-cf77509c7088" (UID: "ed1c5151-73cb-4c20-b801-cf77509c7088"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:36:12 crc kubenswrapper[5052]: I1203 12:36:12.714079 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ed1c5151-73cb-4c20-b801-cf77509c7088-config-data" (OuterVolumeSpecName: "config-data") pod "ed1c5151-73cb-4c20-b801-cf77509c7088" (UID: "ed1c5151-73cb-4c20-b801-cf77509c7088"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:36:12 crc kubenswrapper[5052]: I1203 12:36:12.720177 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ed1c5151-73cb-4c20-b801-cf77509c7088-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:36:12 crc kubenswrapper[5052]: I1203 12:36:12.720214 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/ed1c5151-73cb-4c20-b801-cf77509c7088-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:36:12 crc kubenswrapper[5052]: I1203 12:36:12.720226 5052 reconciler_common.go:293] "Volume detached for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/ed1c5151-73cb-4c20-b801-cf77509c7088-sg-core-conf-yaml\") on node \"crc\" DevicePath \"\"" Dec 03 12:36:12 crc kubenswrapper[5052]: I1203 12:36:12.720276 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-hp4j8\" (UniqueName: \"kubernetes.io/projected/ed1c5151-73cb-4c20-b801-cf77509c7088-kube-api-access-hp4j8\") on node \"crc\" DevicePath \"\"" Dec 03 12:36:12 crc kubenswrapper[5052]: I1203 12:36:12.720294 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/ed1c5151-73cb-4c20-b801-cf77509c7088-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:36:13 crc kubenswrapper[5052]: I1203 12:36:13.210907 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"9dccead0-7ff2-4255-ac00-10a9b357c2db","Type":"ContainerStarted","Data":"ee1a1b3c296fc80f39512ea1317cd9d3123e9fe3d41bf5b71488a44c14937d12"} Dec 03 12:36:13 crc kubenswrapper[5052]: I1203 12:36:13.213601 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"42e9c6bc-62d9-442a-a5f0-d885eff829cb","Type":"ContainerStarted","Data":"256155caf5dc190b0f04698e3cd7ad88f707a09926115e6b668bb753698f6ddf"} Dec 03 12:36:13 crc kubenswrapper[5052]: I1203 12:36:13.213644 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"42e9c6bc-62d9-442a-a5f0-d885eff829cb","Type":"ContainerStarted","Data":"adca9569dc698688f1f02450e166a38ba90740456648576356a082d180255bd0"} Dec 03 12:36:13 crc kubenswrapper[5052]: I1203 12:36:13.217489 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"ed1c5151-73cb-4c20-b801-cf77509c7088","Type":"ContainerDied","Data":"9497695f831e94f22d1d3ea8130a32e4964a34d77e7538607c165b98b87813ae"} Dec 03 12:36:13 crc kubenswrapper[5052]: I1203 12:36:13.217556 5052 scope.go:117] "RemoveContainer" containerID="41dc33620881a66066ffa87ac1dc04a00aba4a0067bba8e0d874f761f849a738" Dec 03 12:36:13 crc kubenswrapper[5052]: I1203 12:36:13.217561 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:36:13 crc kubenswrapper[5052]: I1203 12:36:13.239939 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/glance-default-external-api-0" podStartSLOduration=3.239921255 podStartE2EDuration="3.239921255s" podCreationTimestamp="2025-12-03 12:36:10 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:36:13.237111685 +0000 UTC m=+4563.825353469" watchObservedRunningTime="2025-12-03 12:36:13.239921255 +0000 UTC m=+4563.828163019" Dec 03 12:36:13 crc kubenswrapper[5052]: I1203 12:36:13.249659 5052 scope.go:117] "RemoveContainer" containerID="3300d0f26c3e473950a0c44c09d7cd9b59119f7fd171515efd3ee4a1105f3249" Dec 03 12:36:13 crc kubenswrapper[5052]: I1203 12:36:13.279861 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:36:13 crc kubenswrapper[5052]: I1203 12:36:13.297555 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:36:13 crc kubenswrapper[5052]: I1203 12:36:13.312302 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:36:13 crc kubenswrapper[5052]: E1203 12:36:13.322093 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="ed1c5151-73cb-4c20-b801-cf77509c7088" containerName="sg-core" Dec 03 12:36:13 crc kubenswrapper[5052]: I1203 12:36:13.322345 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="ed1c5151-73cb-4c20-b801-cf77509c7088" containerName="sg-core" Dec 03 12:36:13 crc kubenswrapper[5052]: E1203 12:36:13.322412 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="ed1c5151-73cb-4c20-b801-cf77509c7088" containerName="ceilometer-central-agent" Dec 03 12:36:13 crc kubenswrapper[5052]: I1203 12:36:13.322466 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="ed1c5151-73cb-4c20-b801-cf77509c7088" containerName="ceilometer-central-agent" Dec 03 12:36:13 crc kubenswrapper[5052]: E1203 12:36:13.322598 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="ed1c5151-73cb-4c20-b801-cf77509c7088" containerName="proxy-httpd" Dec 03 12:36:13 crc kubenswrapper[5052]: I1203 12:36:13.322657 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="ed1c5151-73cb-4c20-b801-cf77509c7088" containerName="proxy-httpd" Dec 03 12:36:13 crc kubenswrapper[5052]: E1203 12:36:13.322724 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="ed1c5151-73cb-4c20-b801-cf77509c7088" containerName="ceilometer-notification-agent" Dec 03 12:36:13 crc kubenswrapper[5052]: I1203 12:36:13.322784 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="ed1c5151-73cb-4c20-b801-cf77509c7088" containerName="ceilometer-notification-agent" Dec 03 12:36:13 crc kubenswrapper[5052]: I1203 12:36:13.323073 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="ed1c5151-73cb-4c20-b801-cf77509c7088" containerName="ceilometer-central-agent" Dec 03 12:36:13 crc kubenswrapper[5052]: I1203 12:36:13.323165 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="ed1c5151-73cb-4c20-b801-cf77509c7088" containerName="ceilometer-notification-agent" Dec 03 12:36:13 crc kubenswrapper[5052]: I1203 12:36:13.323235 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="ed1c5151-73cb-4c20-b801-cf77509c7088" containerName="proxy-httpd" Dec 03 12:36:13 crc kubenswrapper[5052]: I1203 12:36:13.323302 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="ed1c5151-73cb-4c20-b801-cf77509c7088" containerName="sg-core" Dec 03 12:36:13 crc kubenswrapper[5052]: I1203 12:36:13.324899 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:36:13 crc kubenswrapper[5052]: I1203 12:36:13.325057 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:36:13 crc kubenswrapper[5052]: I1203 12:36:13.328197 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-config-data" Dec 03 12:36:13 crc kubenswrapper[5052]: I1203 12:36:13.328451 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-scripts" Dec 03 12:36:13 crc kubenswrapper[5052]: I1203 12:36:13.360624 5052 scope.go:117] "RemoveContainer" containerID="3fdeef99541d7da1c01a8d9a5170a7f180d3f1cc9d21e3baa51812d4cb02033c" Dec 03 12:36:13 crc kubenswrapper[5052]: I1203 12:36:13.394064 5052 scope.go:117] "RemoveContainer" containerID="ba21a758c0eafa87981092d11a759465be8fe7d7c3f030cbfc2a0ec4049f8450" Dec 03 12:36:13 crc kubenswrapper[5052]: I1203 12:36:13.444984 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/36028611-5675-4c85-af0d-6ba873b30da7-run-httpd\") pod \"ceilometer-0\" (UID: \"36028611-5675-4c85-af0d-6ba873b30da7\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:36:13 crc kubenswrapper[5052]: I1203 12:36:13.445058 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/36028611-5675-4c85-af0d-6ba873b30da7-config-data\") pod \"ceilometer-0\" (UID: \"36028611-5675-4c85-af0d-6ba873b30da7\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:36:13 crc kubenswrapper[5052]: I1203 12:36:13.445077 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/36028611-5675-4c85-af0d-6ba873b30da7-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"36028611-5675-4c85-af0d-6ba873b30da7\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:36:13 crc kubenswrapper[5052]: I1203 12:36:13.445118 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/36028611-5675-4c85-af0d-6ba873b30da7-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"36028611-5675-4c85-af0d-6ba873b30da7\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:36:13 crc kubenswrapper[5052]: I1203 12:36:13.445135 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/36028611-5675-4c85-af0d-6ba873b30da7-scripts\") pod \"ceilometer-0\" (UID: \"36028611-5675-4c85-af0d-6ba873b30da7\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:36:13 crc kubenswrapper[5052]: I1203 12:36:13.445294 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9f27c\" (UniqueName: \"kubernetes.io/projected/36028611-5675-4c85-af0d-6ba873b30da7-kube-api-access-9f27c\") pod \"ceilometer-0\" (UID: \"36028611-5675-4c85-af0d-6ba873b30da7\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:36:13 crc kubenswrapper[5052]: I1203 12:36:13.445641 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/36028611-5675-4c85-af0d-6ba873b30da7-log-httpd\") pod \"ceilometer-0\" (UID: \"36028611-5675-4c85-af0d-6ba873b30da7\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:36:13 crc kubenswrapper[5052]: I1203 12:36:13.546961 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/36028611-5675-4c85-af0d-6ba873b30da7-run-httpd\") pod \"ceilometer-0\" (UID: \"36028611-5675-4c85-af0d-6ba873b30da7\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:36:13 crc kubenswrapper[5052]: I1203 12:36:13.547661 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/36028611-5675-4c85-af0d-6ba873b30da7-config-data\") pod \"ceilometer-0\" (UID: \"36028611-5675-4c85-af0d-6ba873b30da7\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:36:13 crc kubenswrapper[5052]: I1203 12:36:13.547762 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/36028611-5675-4c85-af0d-6ba873b30da7-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"36028611-5675-4c85-af0d-6ba873b30da7\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:36:13 crc kubenswrapper[5052]: I1203 12:36:13.548310 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/36028611-5675-4c85-af0d-6ba873b30da7-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"36028611-5675-4c85-af0d-6ba873b30da7\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:36:13 crc kubenswrapper[5052]: I1203 12:36:13.548413 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/36028611-5675-4c85-af0d-6ba873b30da7-scripts\") pod \"ceilometer-0\" (UID: \"36028611-5675-4c85-af0d-6ba873b30da7\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:36:13 crc kubenswrapper[5052]: I1203 12:36:13.548608 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-9f27c\" (UniqueName: \"kubernetes.io/projected/36028611-5675-4c85-af0d-6ba873b30da7-kube-api-access-9f27c\") pod \"ceilometer-0\" (UID: \"36028611-5675-4c85-af0d-6ba873b30da7\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:36:13 crc kubenswrapper[5052]: I1203 12:36:13.548744 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/36028611-5675-4c85-af0d-6ba873b30da7-log-httpd\") pod \"ceilometer-0\" (UID: \"36028611-5675-4c85-af0d-6ba873b30da7\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:36:13 crc kubenswrapper[5052]: I1203 12:36:13.549080 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/36028611-5675-4c85-af0d-6ba873b30da7-log-httpd\") pod \"ceilometer-0\" (UID: \"36028611-5675-4c85-af0d-6ba873b30da7\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:36:13 crc kubenswrapper[5052]: I1203 12:36:13.547613 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/36028611-5675-4c85-af0d-6ba873b30da7-run-httpd\") pod \"ceilometer-0\" (UID: \"36028611-5675-4c85-af0d-6ba873b30da7\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:36:13 crc kubenswrapper[5052]: I1203 12:36:13.552834 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/36028611-5675-4c85-af0d-6ba873b30da7-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"36028611-5675-4c85-af0d-6ba873b30da7\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:36:13 crc kubenswrapper[5052]: I1203 12:36:13.553493 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/36028611-5675-4c85-af0d-6ba873b30da7-scripts\") pod \"ceilometer-0\" (UID: \"36028611-5675-4c85-af0d-6ba873b30da7\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:36:13 crc kubenswrapper[5052]: I1203 12:36:13.553717 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/36028611-5675-4c85-af0d-6ba873b30da7-config-data\") pod \"ceilometer-0\" (UID: \"36028611-5675-4c85-af0d-6ba873b30da7\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:36:13 crc kubenswrapper[5052]: I1203 12:36:13.555495 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/36028611-5675-4c85-af0d-6ba873b30da7-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"36028611-5675-4c85-af0d-6ba873b30da7\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:36:13 crc kubenswrapper[5052]: I1203 12:36:13.566998 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-9f27c\" (UniqueName: \"kubernetes.io/projected/36028611-5675-4c85-af0d-6ba873b30da7-kube-api-access-9f27c\") pod \"ceilometer-0\" (UID: \"36028611-5675-4c85-af0d-6ba873b30da7\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:36:13 crc kubenswrapper[5052]: I1203 12:36:13.662872 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:36:13 crc kubenswrapper[5052]: I1203 12:36:13.684792 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ed1c5151-73cb-4c20-b801-cf77509c7088" path="/var/lib/kubelet/pods/ed1c5151-73cb-4c20-b801-cf77509c7088/volumes" Dec 03 12:36:14 crc kubenswrapper[5052]: I1203 12:36:14.134634 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:36:14 crc kubenswrapper[5052]: W1203 12:36:14.138095 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod36028611_5675_4c85_af0d_6ba873b30da7.slice/crio-edd8622abc71df59fb09ec409b84ea2f22620d7ac699ba96b789afe070bb03d5 WatchSource:0}: Error finding container edd8622abc71df59fb09ec409b84ea2f22620d7ac699ba96b789afe070bb03d5: Status 404 returned error can't find the container with id edd8622abc71df59fb09ec409b84ea2f22620d7ac699ba96b789afe070bb03d5 Dec 03 12:36:14 crc kubenswrapper[5052]: I1203 12:36:14.230973 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"42e9c6bc-62d9-442a-a5f0-d885eff829cb","Type":"ContainerStarted","Data":"24924091dd14c68a3a47c176dd1bea56c02b9f160a15ed0661a72c290a768b27"} Dec 03 12:36:14 crc kubenswrapper[5052]: I1203 12:36:14.232548 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"36028611-5675-4c85-af0d-6ba873b30da7","Type":"ContainerStarted","Data":"edd8622abc71df59fb09ec409b84ea2f22620d7ac699ba96b789afe070bb03d5"} Dec 03 12:36:14 crc kubenswrapper[5052]: I1203 12:36:14.254868 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/glance-default-internal-api-0" podStartSLOduration=3.254850781 podStartE2EDuration="3.254850781s" podCreationTimestamp="2025-12-03 12:36:11 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:36:14.25201689 +0000 UTC m=+4564.840258654" watchObservedRunningTime="2025-12-03 12:36:14.254850781 +0000 UTC m=+4564.843092545" Dec 03 12:36:15 crc kubenswrapper[5052]: I1203 12:36:15.246417 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"36028611-5675-4c85-af0d-6ba873b30da7","Type":"ContainerStarted","Data":"dd0ce7d71b0c277f456d230103929c6b85104381dfe61d5d51d3dbfbc10a484a"} Dec 03 12:36:16 crc kubenswrapper[5052]: I1203 12:36:16.260624 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"36028611-5675-4c85-af0d-6ba873b30da7","Type":"ContainerStarted","Data":"1ba41230cf7215eee34d6d8ea8681f728e95087f0ac46229ec46ae822a73d0e3"} Dec 03 12:36:17 crc kubenswrapper[5052]: I1203 12:36:17.270369 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"36028611-5675-4c85-af0d-6ba873b30da7","Type":"ContainerStarted","Data":"999ebc2e5d21eb2c7377ec909cc77ed7c15f0032bb17311e3331c347b89b4ade"} Dec 03 12:36:17 crc kubenswrapper[5052]: I1203 12:36:17.673536 5052 scope.go:117] "RemoveContainer" containerID="e6edaf188f85ea4f11f487f11cc489c700f25fb42b5ee269f4f496f1e127d772" Dec 03 12:36:17 crc kubenswrapper[5052]: E1203 12:36:17.673963 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 12:36:18 crc kubenswrapper[5052]: I1203 12:36:18.281556 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"36028611-5675-4c85-af0d-6ba873b30da7","Type":"ContainerStarted","Data":"5debcb02a261adaa17b1df9a4bfe8eef2137d454c7652df878fe48809d10cad3"} Dec 03 12:36:18 crc kubenswrapper[5052]: I1203 12:36:18.282299 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:36:18 crc kubenswrapper[5052]: I1203 12:36:18.309531 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/ceilometer-0" podStartSLOduration=1.5242868459999999 podStartE2EDuration="5.309516767s" podCreationTimestamp="2025-12-03 12:36:13 +0000 UTC" firstStartedPulling="2025-12-03 12:36:14.143316271 +0000 UTC m=+4564.731558035" lastFinishedPulling="2025-12-03 12:36:17.928546172 +0000 UTC m=+4568.516787956" observedRunningTime="2025-12-03 12:36:18.301454047 +0000 UTC m=+4568.889695811" watchObservedRunningTime="2025-12-03 12:36:18.309516767 +0000 UTC m=+4568.897758531" Dec 03 12:36:19 crc kubenswrapper[5052]: I1203 12:36:19.529987 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:36:20 crc kubenswrapper[5052]: I1203 12:36:20.002405 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell0-cell-mapping-x6pl4"] Dec 03 12:36:20 crc kubenswrapper[5052]: I1203 12:36:20.003752 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-cell-mapping-x6pl4" Dec 03 12:36:20 crc kubenswrapper[5052]: I1203 12:36:20.006151 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell0-manage-config-data" Dec 03 12:36:20 crc kubenswrapper[5052]: I1203 12:36:20.006866 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell0-manage-scripts" Dec 03 12:36:20 crc kubenswrapper[5052]: I1203 12:36:20.015143 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell0-cell-mapping-x6pl4"] Dec 03 12:36:20 crc kubenswrapper[5052]: I1203 12:36:20.081246 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/772e7623-3c44-4375-9f34-5ba8683d5fe1-scripts\") pod \"nova-cell0-cell-mapping-x6pl4\" (UID: \"772e7623-3c44-4375-9f34-5ba8683d5fe1\") " pod="openstack-kuttl-tests/nova-cell0-cell-mapping-x6pl4" Dec 03 12:36:20 crc kubenswrapper[5052]: I1203 12:36:20.081311 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8p8r6\" (UniqueName: \"kubernetes.io/projected/772e7623-3c44-4375-9f34-5ba8683d5fe1-kube-api-access-8p8r6\") pod \"nova-cell0-cell-mapping-x6pl4\" (UID: \"772e7623-3c44-4375-9f34-5ba8683d5fe1\") " pod="openstack-kuttl-tests/nova-cell0-cell-mapping-x6pl4" Dec 03 12:36:20 crc kubenswrapper[5052]: I1203 12:36:20.081469 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/772e7623-3c44-4375-9f34-5ba8683d5fe1-config-data\") pod \"nova-cell0-cell-mapping-x6pl4\" (UID: \"772e7623-3c44-4375-9f34-5ba8683d5fe1\") " pod="openstack-kuttl-tests/nova-cell0-cell-mapping-x6pl4" Dec 03 12:36:20 crc kubenswrapper[5052]: I1203 12:36:20.081694 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/772e7623-3c44-4375-9f34-5ba8683d5fe1-combined-ca-bundle\") pod \"nova-cell0-cell-mapping-x6pl4\" (UID: \"772e7623-3c44-4375-9f34-5ba8683d5fe1\") " pod="openstack-kuttl-tests/nova-cell0-cell-mapping-x6pl4" Dec 03 12:36:20 crc kubenswrapper[5052]: I1203 12:36:20.146776 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:36:20 crc kubenswrapper[5052]: I1203 12:36:20.153292 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:36:20 crc kubenswrapper[5052]: I1203 12:36:20.160011 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-scheduler-config-data" Dec 03 12:36:20 crc kubenswrapper[5052]: I1203 12:36:20.173642 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:36:20 crc kubenswrapper[5052]: I1203 12:36:20.175589 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:36:20 crc kubenswrapper[5052]: I1203 12:36:20.182083 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-metadata-config-data" Dec 03 12:36:20 crc kubenswrapper[5052]: I1203 12:36:20.183404 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/772e7623-3c44-4375-9f34-5ba8683d5fe1-scripts\") pod \"nova-cell0-cell-mapping-x6pl4\" (UID: \"772e7623-3c44-4375-9f34-5ba8683d5fe1\") " pod="openstack-kuttl-tests/nova-cell0-cell-mapping-x6pl4" Dec 03 12:36:20 crc kubenswrapper[5052]: I1203 12:36:20.183459 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-8p8r6\" (UniqueName: \"kubernetes.io/projected/772e7623-3c44-4375-9f34-5ba8683d5fe1-kube-api-access-8p8r6\") pod \"nova-cell0-cell-mapping-x6pl4\" (UID: \"772e7623-3c44-4375-9f34-5ba8683d5fe1\") " pod="openstack-kuttl-tests/nova-cell0-cell-mapping-x6pl4" Dec 03 12:36:20 crc kubenswrapper[5052]: I1203 12:36:20.183489 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/772e7623-3c44-4375-9f34-5ba8683d5fe1-config-data\") pod \"nova-cell0-cell-mapping-x6pl4\" (UID: \"772e7623-3c44-4375-9f34-5ba8683d5fe1\") " pod="openstack-kuttl-tests/nova-cell0-cell-mapping-x6pl4" Dec 03 12:36:20 crc kubenswrapper[5052]: I1203 12:36:20.183538 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/772e7623-3c44-4375-9f34-5ba8683d5fe1-combined-ca-bundle\") pod \"nova-cell0-cell-mapping-x6pl4\" (UID: \"772e7623-3c44-4375-9f34-5ba8683d5fe1\") " pod="openstack-kuttl-tests/nova-cell0-cell-mapping-x6pl4" Dec 03 12:36:20 crc kubenswrapper[5052]: I1203 12:36:20.192480 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/772e7623-3c44-4375-9f34-5ba8683d5fe1-config-data\") pod \"nova-cell0-cell-mapping-x6pl4\" (UID: \"772e7623-3c44-4375-9f34-5ba8683d5fe1\") " pod="openstack-kuttl-tests/nova-cell0-cell-mapping-x6pl4" Dec 03 12:36:20 crc kubenswrapper[5052]: I1203 12:36:20.194897 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/772e7623-3c44-4375-9f34-5ba8683d5fe1-combined-ca-bundle\") pod \"nova-cell0-cell-mapping-x6pl4\" (UID: \"772e7623-3c44-4375-9f34-5ba8683d5fe1\") " pod="openstack-kuttl-tests/nova-cell0-cell-mapping-x6pl4" Dec 03 12:36:20 crc kubenswrapper[5052]: I1203 12:36:20.201221 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/772e7623-3c44-4375-9f34-5ba8683d5fe1-scripts\") pod \"nova-cell0-cell-mapping-x6pl4\" (UID: \"772e7623-3c44-4375-9f34-5ba8683d5fe1\") " pod="openstack-kuttl-tests/nova-cell0-cell-mapping-x6pl4" Dec 03 12:36:20 crc kubenswrapper[5052]: I1203 12:36:20.215916 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:36:20 crc kubenswrapper[5052]: I1203 12:36:20.247193 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-8p8r6\" (UniqueName: \"kubernetes.io/projected/772e7623-3c44-4375-9f34-5ba8683d5fe1-kube-api-access-8p8r6\") pod \"nova-cell0-cell-mapping-x6pl4\" (UID: \"772e7623-3c44-4375-9f34-5ba8683d5fe1\") " pod="openstack-kuttl-tests/nova-cell0-cell-mapping-x6pl4" Dec 03 12:36:20 crc kubenswrapper[5052]: I1203 12:36:20.247596 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:36:20 crc kubenswrapper[5052]: I1203 12:36:20.285236 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/31ee3320-d8b9-4f85-9500-3016e2edf1f7-combined-ca-bundle\") pod \"nova-scheduler-0\" (UID: \"31ee3320-d8b9-4f85-9500-3016e2edf1f7\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:36:20 crc kubenswrapper[5052]: I1203 12:36:20.285281 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5j9x5\" (UniqueName: \"kubernetes.io/projected/9dc44369-33d6-48a5-a072-2f9f4311f8f4-kube-api-access-5j9x5\") pod \"nova-metadata-0\" (UID: \"9dc44369-33d6-48a5-a072-2f9f4311f8f4\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:36:20 crc kubenswrapper[5052]: I1203 12:36:20.285369 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/9dc44369-33d6-48a5-a072-2f9f4311f8f4-logs\") pod \"nova-metadata-0\" (UID: \"9dc44369-33d6-48a5-a072-2f9f4311f8f4\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:36:20 crc kubenswrapper[5052]: I1203 12:36:20.285417 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/31ee3320-d8b9-4f85-9500-3016e2edf1f7-config-data\") pod \"nova-scheduler-0\" (UID: \"31ee3320-d8b9-4f85-9500-3016e2edf1f7\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:36:20 crc kubenswrapper[5052]: I1203 12:36:20.285450 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9dc44369-33d6-48a5-a072-2f9f4311f8f4-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"9dc44369-33d6-48a5-a072-2f9f4311f8f4\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:36:20 crc kubenswrapper[5052]: I1203 12:36:20.285469 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-478rx\" (UniqueName: \"kubernetes.io/projected/31ee3320-d8b9-4f85-9500-3016e2edf1f7-kube-api-access-478rx\") pod \"nova-scheduler-0\" (UID: \"31ee3320-d8b9-4f85-9500-3016e2edf1f7\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:36:20 crc kubenswrapper[5052]: I1203 12:36:20.285485 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/9dc44369-33d6-48a5-a072-2f9f4311f8f4-config-data\") pod \"nova-metadata-0\" (UID: \"9dc44369-33d6-48a5-a072-2f9f4311f8f4\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:36:20 crc kubenswrapper[5052]: I1203 12:36:20.322171 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-cell-mapping-x6pl4" Dec 03 12:36:20 crc kubenswrapper[5052]: I1203 12:36:20.349593 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 12:36:20 crc kubenswrapper[5052]: I1203 12:36:20.351351 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:36:20 crc kubenswrapper[5052]: I1203 12:36:20.366943 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell1-novncproxy-config-data" Dec 03 12:36:20 crc kubenswrapper[5052]: I1203 12:36:20.385857 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:36:20 crc kubenswrapper[5052]: I1203 12:36:20.387436 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:36:20 crc kubenswrapper[5052]: I1203 12:36:20.388836 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/31ee3320-d8b9-4f85-9500-3016e2edf1f7-config-data\") pod \"nova-scheduler-0\" (UID: \"31ee3320-d8b9-4f85-9500-3016e2edf1f7\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:36:20 crc kubenswrapper[5052]: I1203 12:36:20.388884 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9dc44369-33d6-48a5-a072-2f9f4311f8f4-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"9dc44369-33d6-48a5-a072-2f9f4311f8f4\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:36:20 crc kubenswrapper[5052]: I1203 12:36:20.388906 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-478rx\" (UniqueName: \"kubernetes.io/projected/31ee3320-d8b9-4f85-9500-3016e2edf1f7-kube-api-access-478rx\") pod \"nova-scheduler-0\" (UID: \"31ee3320-d8b9-4f85-9500-3016e2edf1f7\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:36:20 crc kubenswrapper[5052]: I1203 12:36:20.388926 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/9dc44369-33d6-48a5-a072-2f9f4311f8f4-config-data\") pod \"nova-metadata-0\" (UID: \"9dc44369-33d6-48a5-a072-2f9f4311f8f4\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:36:20 crc kubenswrapper[5052]: I1203 12:36:20.388966 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-5j9x5\" (UniqueName: \"kubernetes.io/projected/9dc44369-33d6-48a5-a072-2f9f4311f8f4-kube-api-access-5j9x5\") pod \"nova-metadata-0\" (UID: \"9dc44369-33d6-48a5-a072-2f9f4311f8f4\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:36:20 crc kubenswrapper[5052]: I1203 12:36:20.389006 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/31ee3320-d8b9-4f85-9500-3016e2edf1f7-combined-ca-bundle\") pod \"nova-scheduler-0\" (UID: \"31ee3320-d8b9-4f85-9500-3016e2edf1f7\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:36:20 crc kubenswrapper[5052]: I1203 12:36:20.389087 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/9dc44369-33d6-48a5-a072-2f9f4311f8f4-logs\") pod \"nova-metadata-0\" (UID: \"9dc44369-33d6-48a5-a072-2f9f4311f8f4\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:36:20 crc kubenswrapper[5052]: I1203 12:36:20.389127 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rdmx2\" (UniqueName: \"kubernetes.io/projected/a3a9c880-4b1f-40e9-a45d-4fe8025c5199-kube-api-access-rdmx2\") pod \"nova-cell1-novncproxy-0\" (UID: \"a3a9c880-4b1f-40e9-a45d-4fe8025c5199\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:36:20 crc kubenswrapper[5052]: I1203 12:36:20.389143 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a3a9c880-4b1f-40e9-a45d-4fe8025c5199-config-data\") pod \"nova-cell1-novncproxy-0\" (UID: \"a3a9c880-4b1f-40e9-a45d-4fe8025c5199\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:36:20 crc kubenswrapper[5052]: I1203 12:36:20.389163 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a3a9c880-4b1f-40e9-a45d-4fe8025c5199-combined-ca-bundle\") pod \"nova-cell1-novncproxy-0\" (UID: \"a3a9c880-4b1f-40e9-a45d-4fe8025c5199\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:36:20 crc kubenswrapper[5052]: I1203 12:36:20.391477 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-api-config-data" Dec 03 12:36:20 crc kubenswrapper[5052]: I1203 12:36:20.392011 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/9dc44369-33d6-48a5-a072-2f9f4311f8f4-logs\") pod \"nova-metadata-0\" (UID: \"9dc44369-33d6-48a5-a072-2f9f4311f8f4\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:36:20 crc kubenswrapper[5052]: I1203 12:36:20.396258 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/31ee3320-d8b9-4f85-9500-3016e2edf1f7-combined-ca-bundle\") pod \"nova-scheduler-0\" (UID: \"31ee3320-d8b9-4f85-9500-3016e2edf1f7\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:36:20 crc kubenswrapper[5052]: I1203 12:36:20.398092 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/31ee3320-d8b9-4f85-9500-3016e2edf1f7-config-data\") pod \"nova-scheduler-0\" (UID: \"31ee3320-d8b9-4f85-9500-3016e2edf1f7\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:36:20 crc kubenswrapper[5052]: I1203 12:36:20.403146 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9dc44369-33d6-48a5-a072-2f9f4311f8f4-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"9dc44369-33d6-48a5-a072-2f9f4311f8f4\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:36:20 crc kubenswrapper[5052]: I1203 12:36:20.409708 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/9dc44369-33d6-48a5-a072-2f9f4311f8f4-config-data\") pod \"nova-metadata-0\" (UID: \"9dc44369-33d6-48a5-a072-2f9f4311f8f4\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:36:20 crc kubenswrapper[5052]: I1203 12:36:20.428643 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-478rx\" (UniqueName: \"kubernetes.io/projected/31ee3320-d8b9-4f85-9500-3016e2edf1f7-kube-api-access-478rx\") pod \"nova-scheduler-0\" (UID: \"31ee3320-d8b9-4f85-9500-3016e2edf1f7\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:36:20 crc kubenswrapper[5052]: I1203 12:36:20.434718 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-5j9x5\" (UniqueName: \"kubernetes.io/projected/9dc44369-33d6-48a5-a072-2f9f4311f8f4-kube-api-access-5j9x5\") pod \"nova-metadata-0\" (UID: \"9dc44369-33d6-48a5-a072-2f9f4311f8f4\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:36:20 crc kubenswrapper[5052]: I1203 12:36:20.437124 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:36:20 crc kubenswrapper[5052]: I1203 12:36:20.449920 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 12:36:20 crc kubenswrapper[5052]: I1203 12:36:20.480604 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:36:20 crc kubenswrapper[5052]: I1203 12:36:20.494765 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ft9k8\" (UniqueName: \"kubernetes.io/projected/7b6bb151-d113-40e6-97a3-644ca7e4e5f3-kube-api-access-ft9k8\") pod \"nova-api-0\" (UID: \"7b6bb151-d113-40e6-97a3-644ca7e4e5f3\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:36:20 crc kubenswrapper[5052]: I1203 12:36:20.495109 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-rdmx2\" (UniqueName: \"kubernetes.io/projected/a3a9c880-4b1f-40e9-a45d-4fe8025c5199-kube-api-access-rdmx2\") pod \"nova-cell1-novncproxy-0\" (UID: \"a3a9c880-4b1f-40e9-a45d-4fe8025c5199\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:36:20 crc kubenswrapper[5052]: I1203 12:36:20.495169 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a3a9c880-4b1f-40e9-a45d-4fe8025c5199-config-data\") pod \"nova-cell1-novncproxy-0\" (UID: \"a3a9c880-4b1f-40e9-a45d-4fe8025c5199\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:36:20 crc kubenswrapper[5052]: I1203 12:36:20.495200 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a3a9c880-4b1f-40e9-a45d-4fe8025c5199-combined-ca-bundle\") pod \"nova-cell1-novncproxy-0\" (UID: \"a3a9c880-4b1f-40e9-a45d-4fe8025c5199\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:36:20 crc kubenswrapper[5052]: I1203 12:36:20.495239 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/7b6bb151-d113-40e6-97a3-644ca7e4e5f3-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"7b6bb151-d113-40e6-97a3-644ca7e4e5f3\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:36:20 crc kubenswrapper[5052]: I1203 12:36:20.495355 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/7b6bb151-d113-40e6-97a3-644ca7e4e5f3-config-data\") pod \"nova-api-0\" (UID: \"7b6bb151-d113-40e6-97a3-644ca7e4e5f3\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:36:20 crc kubenswrapper[5052]: I1203 12:36:20.495398 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/7b6bb151-d113-40e6-97a3-644ca7e4e5f3-logs\") pod \"nova-api-0\" (UID: \"7b6bb151-d113-40e6-97a3-644ca7e4e5f3\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:36:20 crc kubenswrapper[5052]: I1203 12:36:20.500396 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a3a9c880-4b1f-40e9-a45d-4fe8025c5199-config-data\") pod \"nova-cell1-novncproxy-0\" (UID: \"a3a9c880-4b1f-40e9-a45d-4fe8025c5199\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:36:20 crc kubenswrapper[5052]: I1203 12:36:20.505733 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a3a9c880-4b1f-40e9-a45d-4fe8025c5199-combined-ca-bundle\") pod \"nova-cell1-novncproxy-0\" (UID: \"a3a9c880-4b1f-40e9-a45d-4fe8025c5199\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:36:20 crc kubenswrapper[5052]: I1203 12:36:20.529584 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-rdmx2\" (UniqueName: \"kubernetes.io/projected/a3a9c880-4b1f-40e9-a45d-4fe8025c5199-kube-api-access-rdmx2\") pod \"nova-cell1-novncproxy-0\" (UID: \"a3a9c880-4b1f-40e9-a45d-4fe8025c5199\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:36:20 crc kubenswrapper[5052]: I1203 12:36:20.597172 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/7b6bb151-d113-40e6-97a3-644ca7e4e5f3-config-data\") pod \"nova-api-0\" (UID: \"7b6bb151-d113-40e6-97a3-644ca7e4e5f3\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:36:20 crc kubenswrapper[5052]: I1203 12:36:20.597412 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/7b6bb151-d113-40e6-97a3-644ca7e4e5f3-logs\") pod \"nova-api-0\" (UID: \"7b6bb151-d113-40e6-97a3-644ca7e4e5f3\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:36:20 crc kubenswrapper[5052]: I1203 12:36:20.597539 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-ft9k8\" (UniqueName: \"kubernetes.io/projected/7b6bb151-d113-40e6-97a3-644ca7e4e5f3-kube-api-access-ft9k8\") pod \"nova-api-0\" (UID: \"7b6bb151-d113-40e6-97a3-644ca7e4e5f3\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:36:20 crc kubenswrapper[5052]: I1203 12:36:20.609890 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/7b6bb151-d113-40e6-97a3-644ca7e4e5f3-logs\") pod \"nova-api-0\" (UID: \"7b6bb151-d113-40e6-97a3-644ca7e4e5f3\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:36:20 crc kubenswrapper[5052]: I1203 12:36:20.610002 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/7b6bb151-d113-40e6-97a3-644ca7e4e5f3-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"7b6bb151-d113-40e6-97a3-644ca7e4e5f3\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:36:20 crc kubenswrapper[5052]: I1203 12:36:20.614461 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/7b6bb151-d113-40e6-97a3-644ca7e4e5f3-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"7b6bb151-d113-40e6-97a3-644ca7e4e5f3\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:36:20 crc kubenswrapper[5052]: I1203 12:36:20.614580 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/7b6bb151-d113-40e6-97a3-644ca7e4e5f3-config-data\") pod \"nova-api-0\" (UID: \"7b6bb151-d113-40e6-97a3-644ca7e4e5f3\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:36:20 crc kubenswrapper[5052]: I1203 12:36:20.619995 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:36:20 crc kubenswrapper[5052]: I1203 12:36:20.626550 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-ft9k8\" (UniqueName: \"kubernetes.io/projected/7b6bb151-d113-40e6-97a3-644ca7e4e5f3-kube-api-access-ft9k8\") pod \"nova-api-0\" (UID: \"7b6bb151-d113-40e6-97a3-644ca7e4e5f3\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:36:20 crc kubenswrapper[5052]: I1203 12:36:20.795193 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:36:20 crc kubenswrapper[5052]: I1203 12:36:20.811074 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:36:20 crc kubenswrapper[5052]: I1203 12:36:20.846797 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:36:20 crc kubenswrapper[5052]: I1203 12:36:20.846857 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:36:20 crc kubenswrapper[5052]: I1203 12:36:20.883999 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:36:20 crc kubenswrapper[5052]: I1203 12:36:20.896920 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:36:20 crc kubenswrapper[5052]: W1203 12:36:20.902331 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod31ee3320_d8b9_4f85_9500_3016e2edf1f7.slice/crio-1055239fd3d01ec31db4e05253ae205fa3c1f67e2d0d5298a42bfeae540194d3 WatchSource:0}: Error finding container 1055239fd3d01ec31db4e05253ae205fa3c1f67e2d0d5298a42bfeae540194d3: Status 404 returned error can't find the container with id 1055239fd3d01ec31db4e05253ae205fa3c1f67e2d0d5298a42bfeae540194d3 Dec 03 12:36:20 crc kubenswrapper[5052]: I1203 12:36:20.941817 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:36:20 crc kubenswrapper[5052]: I1203 12:36:20.977091 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell0-cell-mapping-x6pl4"] Dec 03 12:36:21 crc kubenswrapper[5052]: W1203 12:36:21.002851 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod772e7623_3c44_4375_9f34_5ba8683d5fe1.slice/crio-a1df95494e68580aaf70278576fe02128cecde30611a4b551f6ccb08ea9754d5 WatchSource:0}: Error finding container a1df95494e68580aaf70278576fe02128cecde30611a4b551f6ccb08ea9754d5: Status 404 returned error can't find the container with id a1df95494e68580aaf70278576fe02128cecde30611a4b551f6ccb08ea9754d5 Dec 03 12:36:21 crc kubenswrapper[5052]: I1203 12:36:21.213406 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:36:21 crc kubenswrapper[5052]: I1203 12:36:21.243233 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 12:36:21 crc kubenswrapper[5052]: W1203 12:36:21.301900 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7b6bb151_d113_40e6_97a3_644ca7e4e5f3.slice/crio-d541dc9f23ceb98afa8eee4e8260db9f66d999f810603fa6a0b01f017ee0803b WatchSource:0}: Error finding container d541dc9f23ceb98afa8eee4e8260db9f66d999f810603fa6a0b01f017ee0803b: Status 404 returned error can't find the container with id d541dc9f23ceb98afa8eee4e8260db9f66d999f810603fa6a0b01f017ee0803b Dec 03 12:36:21 crc kubenswrapper[5052]: I1203 12:36:21.311809 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:36:21 crc kubenswrapper[5052]: I1203 12:36:21.350081 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"7b6bb151-d113-40e6-97a3-644ca7e4e5f3","Type":"ContainerStarted","Data":"d541dc9f23ceb98afa8eee4e8260db9f66d999f810603fa6a0b01f017ee0803b"} Dec 03 12:36:21 crc kubenswrapper[5052]: I1203 12:36:21.350885 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-cell-mapping-x6pl4" event={"ID":"772e7623-3c44-4375-9f34-5ba8683d5fe1","Type":"ContainerStarted","Data":"a1df95494e68580aaf70278576fe02128cecde30611a4b551f6ccb08ea9754d5"} Dec 03 12:36:21 crc kubenswrapper[5052]: I1203 12:36:21.354537 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" event={"ID":"a3a9c880-4b1f-40e9-a45d-4fe8025c5199","Type":"ContainerStarted","Data":"1aba86d1faff97e4db3a48ec42087d1c1ab1ee31638fb854f36c8829427549ff"} Dec 03 12:36:21 crc kubenswrapper[5052]: I1203 12:36:21.381000 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"9dc44369-33d6-48a5-a072-2f9f4311f8f4","Type":"ContainerStarted","Data":"65c0fc20efc15f025ef37633ff680684cd2b18d9ba848d9a6fc16871acb96389"} Dec 03 12:36:21 crc kubenswrapper[5052]: I1203 12:36:21.383847 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"31ee3320-d8b9-4f85-9500-3016e2edf1f7","Type":"ContainerStarted","Data":"fb06a935923602d2b3fa6510b0f711cbdca186b1223cbdab3975ad3887386af8"} Dec 03 12:36:21 crc kubenswrapper[5052]: I1203 12:36:21.383898 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"31ee3320-d8b9-4f85-9500-3016e2edf1f7","Type":"ContainerStarted","Data":"1055239fd3d01ec31db4e05253ae205fa3c1f67e2d0d5298a42bfeae540194d3"} Dec 03 12:36:21 crc kubenswrapper[5052]: I1203 12:36:21.384408 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:36:21 crc kubenswrapper[5052]: I1203 12:36:21.384438 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:36:21 crc kubenswrapper[5052]: I1203 12:36:21.401497 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-db-sync-2v6n2"] Dec 03 12:36:21 crc kubenswrapper[5052]: I1203 12:36:21.404295 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-2v6n2" Dec 03 12:36:21 crc kubenswrapper[5052]: I1203 12:36:21.415311 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell1-conductor-scripts" Dec 03 12:36:21 crc kubenswrapper[5052]: I1203 12:36:21.415572 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell1-conductor-config-data" Dec 03 12:36:21 crc kubenswrapper[5052]: I1203 12:36:21.424568 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-db-sync-2v6n2"] Dec 03 12:36:21 crc kubenswrapper[5052]: I1203 12:36:21.443333 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-scheduler-0" podStartSLOduration=1.443310138 podStartE2EDuration="1.443310138s" podCreationTimestamp="2025-12-03 12:36:20 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:36:21.417193671 +0000 UTC m=+4572.005435455" watchObservedRunningTime="2025-12-03 12:36:21.443310138 +0000 UTC m=+4572.031551902" Dec 03 12:36:21 crc kubenswrapper[5052]: I1203 12:36:21.557259 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bzzvw\" (UniqueName: \"kubernetes.io/projected/32d4c4ba-c6dd-49cf-80a3-4d7ed71cb6b5-kube-api-access-bzzvw\") pod \"nova-cell1-conductor-db-sync-2v6n2\" (UID: \"32d4c4ba-c6dd-49cf-80a3-4d7ed71cb6b5\") " pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-2v6n2" Dec 03 12:36:21 crc kubenswrapper[5052]: I1203 12:36:21.557583 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/32d4c4ba-c6dd-49cf-80a3-4d7ed71cb6b5-config-data\") pod \"nova-cell1-conductor-db-sync-2v6n2\" (UID: \"32d4c4ba-c6dd-49cf-80a3-4d7ed71cb6b5\") " pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-2v6n2" Dec 03 12:36:21 crc kubenswrapper[5052]: I1203 12:36:21.557632 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/32d4c4ba-c6dd-49cf-80a3-4d7ed71cb6b5-scripts\") pod \"nova-cell1-conductor-db-sync-2v6n2\" (UID: \"32d4c4ba-c6dd-49cf-80a3-4d7ed71cb6b5\") " pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-2v6n2" Dec 03 12:36:21 crc kubenswrapper[5052]: I1203 12:36:21.557690 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/32d4c4ba-c6dd-49cf-80a3-4d7ed71cb6b5-combined-ca-bundle\") pod \"nova-cell1-conductor-db-sync-2v6n2\" (UID: \"32d4c4ba-c6dd-49cf-80a3-4d7ed71cb6b5\") " pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-2v6n2" Dec 03 12:36:21 crc kubenswrapper[5052]: I1203 12:36:21.660126 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-bzzvw\" (UniqueName: \"kubernetes.io/projected/32d4c4ba-c6dd-49cf-80a3-4d7ed71cb6b5-kube-api-access-bzzvw\") pod \"nova-cell1-conductor-db-sync-2v6n2\" (UID: \"32d4c4ba-c6dd-49cf-80a3-4d7ed71cb6b5\") " pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-2v6n2" Dec 03 12:36:21 crc kubenswrapper[5052]: I1203 12:36:21.660193 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/32d4c4ba-c6dd-49cf-80a3-4d7ed71cb6b5-config-data\") pod \"nova-cell1-conductor-db-sync-2v6n2\" (UID: \"32d4c4ba-c6dd-49cf-80a3-4d7ed71cb6b5\") " pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-2v6n2" Dec 03 12:36:21 crc kubenswrapper[5052]: I1203 12:36:21.660230 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/32d4c4ba-c6dd-49cf-80a3-4d7ed71cb6b5-scripts\") pod \"nova-cell1-conductor-db-sync-2v6n2\" (UID: \"32d4c4ba-c6dd-49cf-80a3-4d7ed71cb6b5\") " pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-2v6n2" Dec 03 12:36:21 crc kubenswrapper[5052]: I1203 12:36:21.660286 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/32d4c4ba-c6dd-49cf-80a3-4d7ed71cb6b5-combined-ca-bundle\") pod \"nova-cell1-conductor-db-sync-2v6n2\" (UID: \"32d4c4ba-c6dd-49cf-80a3-4d7ed71cb6b5\") " pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-2v6n2" Dec 03 12:36:21 crc kubenswrapper[5052]: I1203 12:36:21.669449 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/32d4c4ba-c6dd-49cf-80a3-4d7ed71cb6b5-config-data\") pod \"nova-cell1-conductor-db-sync-2v6n2\" (UID: \"32d4c4ba-c6dd-49cf-80a3-4d7ed71cb6b5\") " pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-2v6n2" Dec 03 12:36:21 crc kubenswrapper[5052]: I1203 12:36:21.671325 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/32d4c4ba-c6dd-49cf-80a3-4d7ed71cb6b5-scripts\") pod \"nova-cell1-conductor-db-sync-2v6n2\" (UID: \"32d4c4ba-c6dd-49cf-80a3-4d7ed71cb6b5\") " pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-2v6n2" Dec 03 12:36:21 crc kubenswrapper[5052]: I1203 12:36:21.671942 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/32d4c4ba-c6dd-49cf-80a3-4d7ed71cb6b5-combined-ca-bundle\") pod \"nova-cell1-conductor-db-sync-2v6n2\" (UID: \"32d4c4ba-c6dd-49cf-80a3-4d7ed71cb6b5\") " pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-2v6n2" Dec 03 12:36:21 crc kubenswrapper[5052]: I1203 12:36:21.695156 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-bzzvw\" (UniqueName: \"kubernetes.io/projected/32d4c4ba-c6dd-49cf-80a3-4d7ed71cb6b5-kube-api-access-bzzvw\") pod \"nova-cell1-conductor-db-sync-2v6n2\" (UID: \"32d4c4ba-c6dd-49cf-80a3-4d7ed71cb6b5\") " pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-2v6n2" Dec 03 12:36:21 crc kubenswrapper[5052]: I1203 12:36:21.761802 5052 scope.go:117] "RemoveContainer" containerID="c1a2fa107302843e2249473b85f61687344a151311126ad578b5760b51b1b60e" Dec 03 12:36:21 crc kubenswrapper[5052]: I1203 12:36:21.781913 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-2v6n2" Dec 03 12:36:21 crc kubenswrapper[5052]: I1203 12:36:21.843695 5052 scope.go:117] "RemoveContainer" containerID="6f35ce14790c9ffb5796ad1467515b2be517ae60eb2573a1571ba5f99f630cad" Dec 03 12:36:21 crc kubenswrapper[5052]: I1203 12:36:21.904420 5052 scope.go:117] "RemoveContainer" containerID="009d530d399c78fb2e9155e989a69050f595b2f0a1be3bf13247c7f1d91a158a" Dec 03 12:36:21 crc kubenswrapper[5052]: I1203 12:36:21.948567 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:36:21 crc kubenswrapper[5052]: I1203 12:36:21.949718 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:36:21 crc kubenswrapper[5052]: I1203 12:36:21.949719 5052 scope.go:117] "RemoveContainer" containerID="b4d3741161a3d95e91cfc6ceb2e9626438d026ce0c88039796b48eae93f58046" Dec 03 12:36:22 crc kubenswrapper[5052]: I1203 12:36:22.047792 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:36:22 crc kubenswrapper[5052]: I1203 12:36:22.077424 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:36:22 crc kubenswrapper[5052]: I1203 12:36:22.105101 5052 scope.go:117] "RemoveContainer" containerID="b5aa97990bcd4bfc80db39c69ed703786808679eda611dc607c177d51eeb3707" Dec 03 12:36:22 crc kubenswrapper[5052]: I1203 12:36:22.197993 5052 scope.go:117] "RemoveContainer" containerID="e9076e00de7ae30a9d54e2e79b337633b061bc7f8a01e8aa825185ef328777b0" Dec 03 12:36:22 crc kubenswrapper[5052]: I1203 12:36:22.295796 5052 scope.go:117] "RemoveContainer" containerID="b4461aa30a7576cad9c21f4e4005920d51c63b774d33068ff773dbfc856ca202" Dec 03 12:36:22 crc kubenswrapper[5052]: I1203 12:36:22.309424 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-db-sync-2v6n2"] Dec 03 12:36:22 crc kubenswrapper[5052]: I1203 12:36:22.396989 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-cell-mapping-x6pl4" event={"ID":"772e7623-3c44-4375-9f34-5ba8683d5fe1","Type":"ContainerStarted","Data":"e732642ec3d2e2c1b77a5c5cf4d43de4178312bd401dae5fa03708d2e3c9248f"} Dec 03 12:36:22 crc kubenswrapper[5052]: I1203 12:36:22.399635 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" event={"ID":"a3a9c880-4b1f-40e9-a45d-4fe8025c5199","Type":"ContainerStarted","Data":"d8d2cdf9ed5e3b9d8f2e7e571bc75f70731f37d526b3207d40db7dc36e682c8c"} Dec 03 12:36:22 crc kubenswrapper[5052]: I1203 12:36:22.411446 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"9dc44369-33d6-48a5-a072-2f9f4311f8f4","Type":"ContainerStarted","Data":"d671d6c5362c51b3bc62b92cfede167be6efb1c9b8ce80868491d117252d0e6b"} Dec 03 12:36:22 crc kubenswrapper[5052]: I1203 12:36:22.411527 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"9dc44369-33d6-48a5-a072-2f9f4311f8f4","Type":"ContainerStarted","Data":"203a8e011f4c47248b78234223150b6dbec13d98ffa7ce6afb0307f2e7edf50f"} Dec 03 12:36:22 crc kubenswrapper[5052]: I1203 12:36:22.415037 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-cell0-cell-mapping-x6pl4" podStartSLOduration=3.415021348 podStartE2EDuration="3.415021348s" podCreationTimestamp="2025-12-03 12:36:19 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:36:22.409930062 +0000 UTC m=+4572.998171826" watchObservedRunningTime="2025-12-03 12:36:22.415021348 +0000 UTC m=+4573.003263112" Dec 03 12:36:22 crc kubenswrapper[5052]: I1203 12:36:22.434959 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"7b6bb151-d113-40e6-97a3-644ca7e4e5f3","Type":"ContainerStarted","Data":"0b2f979945fddbeb900f47fb601b5b1707af68be110b9c7e7a0ec564f0e78e54"} Dec 03 12:36:22 crc kubenswrapper[5052]: I1203 12:36:22.435030 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"7b6bb151-d113-40e6-97a3-644ca7e4e5f3","Type":"ContainerStarted","Data":"62321816c49aa66fe160c689b7c7f7a1345ff820851a3c51ec682204697cbbe5"} Dec 03 12:36:22 crc kubenswrapper[5052]: I1203 12:36:22.436542 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:36:22 crc kubenswrapper[5052]: I1203 12:36:22.436703 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:36:22 crc kubenswrapper[5052]: I1203 12:36:22.446508 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" podStartSLOduration=2.4464878580000002 podStartE2EDuration="2.446487858s" podCreationTimestamp="2025-12-03 12:36:20 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:36:22.441275788 +0000 UTC m=+4573.029517552" watchObservedRunningTime="2025-12-03 12:36:22.446487858 +0000 UTC m=+4573.034729622" Dec 03 12:36:22 crc kubenswrapper[5052]: I1203 12:36:22.473131 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-metadata-0" podStartSLOduration=2.473112249 podStartE2EDuration="2.473112249s" podCreationTimestamp="2025-12-03 12:36:20 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:36:22.471985267 +0000 UTC m=+4573.060227031" watchObservedRunningTime="2025-12-03 12:36:22.473112249 +0000 UTC m=+4573.061354013" Dec 03 12:36:22 crc kubenswrapper[5052]: I1203 12:36:22.505387 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-api-0" podStartSLOduration=2.505365911 podStartE2EDuration="2.505365911s" podCreationTimestamp="2025-12-03 12:36:20 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:36:22.495423177 +0000 UTC m=+4573.083664941" watchObservedRunningTime="2025-12-03 12:36:22.505365911 +0000 UTC m=+4573.093607675" Dec 03 12:36:23 crc kubenswrapper[5052]: I1203 12:36:23.445051 5052 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Dec 03 12:36:23 crc kubenswrapper[5052]: I1203 12:36:23.445084 5052 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Dec 03 12:36:23 crc kubenswrapper[5052]: W1203 12:36:23.632582 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod32d4c4ba_c6dd_49cf_80a3_4d7ed71cb6b5.slice/crio-b05f948f48338fd7c1f6cd1e438d6262d9d99ba06e412f5140cf8b8bf463c8cf WatchSource:0}: Error finding container b05f948f48338fd7c1f6cd1e438d6262d9d99ba06e412f5140cf8b8bf463c8cf: Status 404 returned error can't find the container with id b05f948f48338fd7c1f6cd1e438d6262d9d99ba06e412f5140cf8b8bf463c8cf Dec 03 12:36:23 crc kubenswrapper[5052]: I1203 12:36:23.648350 5052 scope.go:117] "RemoveContainer" containerID="e56245b2be2e929328f0828e016ea41db39854cc67eb6cfae7d314651f18c3c0" Dec 03 12:36:23 crc kubenswrapper[5052]: I1203 12:36:23.771024 5052 scope.go:117] "RemoveContainer" containerID="fc5e4fca262b35b92bb410a508d650dc4bdca0fc0848d2a90abb1ede218d5a33" Dec 03 12:36:24 crc kubenswrapper[5052]: I1203 12:36:24.471973 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-2v6n2" event={"ID":"32d4c4ba-c6dd-49cf-80a3-4d7ed71cb6b5","Type":"ContainerStarted","Data":"ffc601b182d6f61d49cc3f4cc073d39e73625c22b23b4c46e4b1d092e911918c"} Dec 03 12:36:24 crc kubenswrapper[5052]: I1203 12:36:24.472334 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-2v6n2" event={"ID":"32d4c4ba-c6dd-49cf-80a3-4d7ed71cb6b5","Type":"ContainerStarted","Data":"b05f948f48338fd7c1f6cd1e438d6262d9d99ba06e412f5140cf8b8bf463c8cf"} Dec 03 12:36:24 crc kubenswrapper[5052]: I1203 12:36:24.492910 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-2v6n2" podStartSLOduration=3.492889201 podStartE2EDuration="3.492889201s" podCreationTimestamp="2025-12-03 12:36:21 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:36:24.491417248 +0000 UTC m=+4575.079659012" watchObservedRunningTime="2025-12-03 12:36:24.492889201 +0000 UTC m=+4575.081130965" Dec 03 12:36:24 crc kubenswrapper[5052]: I1203 12:36:24.580585 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 12:36:24 crc kubenswrapper[5052]: I1203 12:36:24.580818 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" podUID="a3a9c880-4b1f-40e9-a45d-4fe8025c5199" containerName="nova-cell1-novncproxy-novncproxy" containerID="cri-o://d8d2cdf9ed5e3b9d8f2e7e571bc75f70731f37d526b3207d40db7dc36e682c8c" gracePeriod=30 Dec 03 12:36:24 crc kubenswrapper[5052]: I1203 12:36:24.627762 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:36:24 crc kubenswrapper[5052]: I1203 12:36:24.627983 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-metadata-0" podUID="9dc44369-33d6-48a5-a072-2f9f4311f8f4" containerName="nova-metadata-log" containerID="cri-o://203a8e011f4c47248b78234223150b6dbec13d98ffa7ce6afb0307f2e7edf50f" gracePeriod=30 Dec 03 12:36:24 crc kubenswrapper[5052]: I1203 12:36:24.628053 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-metadata-0" podUID="9dc44369-33d6-48a5-a072-2f9f4311f8f4" containerName="nova-metadata-metadata" containerID="cri-o://d671d6c5362c51b3bc62b92cfede167be6efb1c9b8ce80868491d117252d0e6b" gracePeriod=30 Dec 03 12:36:24 crc kubenswrapper[5052]: I1203 12:36:24.663898 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:36:24 crc kubenswrapper[5052]: I1203 12:36:24.664256 5052 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Dec 03 12:36:24 crc kubenswrapper[5052]: I1203 12:36:24.965855 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:36:24 crc kubenswrapper[5052]: I1203 12:36:24.965944 5052 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Dec 03 12:36:24 crc kubenswrapper[5052]: I1203 12:36:24.969110 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:36:25 crc kubenswrapper[5052]: I1203 12:36:25.321314 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:36:25 crc kubenswrapper[5052]: I1203 12:36:25.481817 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:36:25 crc kubenswrapper[5052]: I1203 12:36:25.483758 5052 generic.go:334] "Generic (PLEG): container finished" podID="9dc44369-33d6-48a5-a072-2f9f4311f8f4" containerID="d671d6c5362c51b3bc62b92cfede167be6efb1c9b8ce80868491d117252d0e6b" exitCode=0 Dec 03 12:36:25 crc kubenswrapper[5052]: I1203 12:36:25.483805 5052 generic.go:334] "Generic (PLEG): container finished" podID="9dc44369-33d6-48a5-a072-2f9f4311f8f4" containerID="203a8e011f4c47248b78234223150b6dbec13d98ffa7ce6afb0307f2e7edf50f" exitCode=143 Dec 03 12:36:25 crc kubenswrapper[5052]: I1203 12:36:25.483932 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"9dc44369-33d6-48a5-a072-2f9f4311f8f4","Type":"ContainerDied","Data":"d671d6c5362c51b3bc62b92cfede167be6efb1c9b8ce80868491d117252d0e6b"} Dec 03 12:36:25 crc kubenswrapper[5052]: I1203 12:36:25.483972 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"9dc44369-33d6-48a5-a072-2f9f4311f8f4","Type":"ContainerDied","Data":"203a8e011f4c47248b78234223150b6dbec13d98ffa7ce6afb0307f2e7edf50f"} Dec 03 12:36:25 crc kubenswrapper[5052]: I1203 12:36:25.621791 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:36:25 crc kubenswrapper[5052]: I1203 12:36:25.621928 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:36:25 crc kubenswrapper[5052]: I1203 12:36:25.797404 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:36:25 crc kubenswrapper[5052]: I1203 12:36:25.867734 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:36:25 crc kubenswrapper[5052]: I1203 12:36:25.957726 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/9dc44369-33d6-48a5-a072-2f9f4311f8f4-config-data\") pod \"9dc44369-33d6-48a5-a072-2f9f4311f8f4\" (UID: \"9dc44369-33d6-48a5-a072-2f9f4311f8f4\") " Dec 03 12:36:25 crc kubenswrapper[5052]: I1203 12:36:25.957871 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5j9x5\" (UniqueName: \"kubernetes.io/projected/9dc44369-33d6-48a5-a072-2f9f4311f8f4-kube-api-access-5j9x5\") pod \"9dc44369-33d6-48a5-a072-2f9f4311f8f4\" (UID: \"9dc44369-33d6-48a5-a072-2f9f4311f8f4\") " Dec 03 12:36:25 crc kubenswrapper[5052]: I1203 12:36:25.957991 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/9dc44369-33d6-48a5-a072-2f9f4311f8f4-logs\") pod \"9dc44369-33d6-48a5-a072-2f9f4311f8f4\" (UID: \"9dc44369-33d6-48a5-a072-2f9f4311f8f4\") " Dec 03 12:36:25 crc kubenswrapper[5052]: I1203 12:36:25.958063 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9dc44369-33d6-48a5-a072-2f9f4311f8f4-combined-ca-bundle\") pod \"9dc44369-33d6-48a5-a072-2f9f4311f8f4\" (UID: \"9dc44369-33d6-48a5-a072-2f9f4311f8f4\") " Dec 03 12:36:25 crc kubenswrapper[5052]: I1203 12:36:25.962920 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9dc44369-33d6-48a5-a072-2f9f4311f8f4-logs" (OuterVolumeSpecName: "logs") pod "9dc44369-33d6-48a5-a072-2f9f4311f8f4" (UID: "9dc44369-33d6-48a5-a072-2f9f4311f8f4"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:36:25 crc kubenswrapper[5052]: I1203 12:36:25.969738 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9dc44369-33d6-48a5-a072-2f9f4311f8f4-kube-api-access-5j9x5" (OuterVolumeSpecName: "kube-api-access-5j9x5") pod "9dc44369-33d6-48a5-a072-2f9f4311f8f4" (UID: "9dc44369-33d6-48a5-a072-2f9f4311f8f4"). InnerVolumeSpecName "kube-api-access-5j9x5". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:36:25 crc kubenswrapper[5052]: I1203 12:36:25.991576 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9dc44369-33d6-48a5-a072-2f9f4311f8f4-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "9dc44369-33d6-48a5-a072-2f9f4311f8f4" (UID: "9dc44369-33d6-48a5-a072-2f9f4311f8f4"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:36:25 crc kubenswrapper[5052]: I1203 12:36:25.992446 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9dc44369-33d6-48a5-a072-2f9f4311f8f4-config-data" (OuterVolumeSpecName: "config-data") pod "9dc44369-33d6-48a5-a072-2f9f4311f8f4" (UID: "9dc44369-33d6-48a5-a072-2f9f4311f8f4"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:36:26 crc kubenswrapper[5052]: I1203 12:36:26.060836 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/9dc44369-33d6-48a5-a072-2f9f4311f8f4-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:36:26 crc kubenswrapper[5052]: I1203 12:36:26.060870 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-5j9x5\" (UniqueName: \"kubernetes.io/projected/9dc44369-33d6-48a5-a072-2f9f4311f8f4-kube-api-access-5j9x5\") on node \"crc\" DevicePath \"\"" Dec 03 12:36:26 crc kubenswrapper[5052]: I1203 12:36:26.060882 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/9dc44369-33d6-48a5-a072-2f9f4311f8f4-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:36:26 crc kubenswrapper[5052]: I1203 12:36:26.060895 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9dc44369-33d6-48a5-a072-2f9f4311f8f4-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:36:26 crc kubenswrapper[5052]: I1203 12:36:26.086174 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:36:26 crc kubenswrapper[5052]: I1203 12:36:26.161822 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a3a9c880-4b1f-40e9-a45d-4fe8025c5199-config-data\") pod \"a3a9c880-4b1f-40e9-a45d-4fe8025c5199\" (UID: \"a3a9c880-4b1f-40e9-a45d-4fe8025c5199\") " Dec 03 12:36:26 crc kubenswrapper[5052]: I1203 12:36:26.161969 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a3a9c880-4b1f-40e9-a45d-4fe8025c5199-combined-ca-bundle\") pod \"a3a9c880-4b1f-40e9-a45d-4fe8025c5199\" (UID: \"a3a9c880-4b1f-40e9-a45d-4fe8025c5199\") " Dec 03 12:36:26 crc kubenswrapper[5052]: I1203 12:36:26.162429 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rdmx2\" (UniqueName: \"kubernetes.io/projected/a3a9c880-4b1f-40e9-a45d-4fe8025c5199-kube-api-access-rdmx2\") pod \"a3a9c880-4b1f-40e9-a45d-4fe8025c5199\" (UID: \"a3a9c880-4b1f-40e9-a45d-4fe8025c5199\") " Dec 03 12:36:26 crc kubenswrapper[5052]: I1203 12:36:26.165744 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a3a9c880-4b1f-40e9-a45d-4fe8025c5199-kube-api-access-rdmx2" (OuterVolumeSpecName: "kube-api-access-rdmx2") pod "a3a9c880-4b1f-40e9-a45d-4fe8025c5199" (UID: "a3a9c880-4b1f-40e9-a45d-4fe8025c5199"). InnerVolumeSpecName "kube-api-access-rdmx2". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:36:26 crc kubenswrapper[5052]: I1203 12:36:26.198135 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a3a9c880-4b1f-40e9-a45d-4fe8025c5199-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "a3a9c880-4b1f-40e9-a45d-4fe8025c5199" (UID: "a3a9c880-4b1f-40e9-a45d-4fe8025c5199"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:36:26 crc kubenswrapper[5052]: I1203 12:36:26.204366 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a3a9c880-4b1f-40e9-a45d-4fe8025c5199-config-data" (OuterVolumeSpecName: "config-data") pod "a3a9c880-4b1f-40e9-a45d-4fe8025c5199" (UID: "a3a9c880-4b1f-40e9-a45d-4fe8025c5199"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:36:26 crc kubenswrapper[5052]: I1203 12:36:26.264012 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-rdmx2\" (UniqueName: \"kubernetes.io/projected/a3a9c880-4b1f-40e9-a45d-4fe8025c5199-kube-api-access-rdmx2\") on node \"crc\" DevicePath \"\"" Dec 03 12:36:26 crc kubenswrapper[5052]: I1203 12:36:26.264061 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a3a9c880-4b1f-40e9-a45d-4fe8025c5199-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:36:26 crc kubenswrapper[5052]: I1203 12:36:26.264074 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a3a9c880-4b1f-40e9-a45d-4fe8025c5199-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:36:26 crc kubenswrapper[5052]: I1203 12:36:26.496200 5052 generic.go:334] "Generic (PLEG): container finished" podID="a3a9c880-4b1f-40e9-a45d-4fe8025c5199" containerID="d8d2cdf9ed5e3b9d8f2e7e571bc75f70731f37d526b3207d40db7dc36e682c8c" exitCode=0 Dec 03 12:36:26 crc kubenswrapper[5052]: I1203 12:36:26.496299 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" event={"ID":"a3a9c880-4b1f-40e9-a45d-4fe8025c5199","Type":"ContainerDied","Data":"d8d2cdf9ed5e3b9d8f2e7e571bc75f70731f37d526b3207d40db7dc36e682c8c"} Dec 03 12:36:26 crc kubenswrapper[5052]: I1203 12:36:26.496332 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" event={"ID":"a3a9c880-4b1f-40e9-a45d-4fe8025c5199","Type":"ContainerDied","Data":"1aba86d1faff97e4db3a48ec42087d1c1ab1ee31638fb854f36c8829427549ff"} Dec 03 12:36:26 crc kubenswrapper[5052]: I1203 12:36:26.496351 5052 scope.go:117] "RemoveContainer" containerID="d8d2cdf9ed5e3b9d8f2e7e571bc75f70731f37d526b3207d40db7dc36e682c8c" Dec 03 12:36:26 crc kubenswrapper[5052]: I1203 12:36:26.496577 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:36:26 crc kubenswrapper[5052]: I1203 12:36:26.507169 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"9dc44369-33d6-48a5-a072-2f9f4311f8f4","Type":"ContainerDied","Data":"65c0fc20efc15f025ef37633ff680684cd2b18d9ba848d9a6fc16871acb96389"} Dec 03 12:36:26 crc kubenswrapper[5052]: I1203 12:36:26.507355 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:36:26 crc kubenswrapper[5052]: I1203 12:36:26.536754 5052 scope.go:117] "RemoveContainer" containerID="d8d2cdf9ed5e3b9d8f2e7e571bc75f70731f37d526b3207d40db7dc36e682c8c" Dec 03 12:36:26 crc kubenswrapper[5052]: E1203 12:36:26.537343 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d8d2cdf9ed5e3b9d8f2e7e571bc75f70731f37d526b3207d40db7dc36e682c8c\": container with ID starting with d8d2cdf9ed5e3b9d8f2e7e571bc75f70731f37d526b3207d40db7dc36e682c8c not found: ID does not exist" containerID="d8d2cdf9ed5e3b9d8f2e7e571bc75f70731f37d526b3207d40db7dc36e682c8c" Dec 03 12:36:26 crc kubenswrapper[5052]: I1203 12:36:26.537397 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d8d2cdf9ed5e3b9d8f2e7e571bc75f70731f37d526b3207d40db7dc36e682c8c"} err="failed to get container status \"d8d2cdf9ed5e3b9d8f2e7e571bc75f70731f37d526b3207d40db7dc36e682c8c\": rpc error: code = NotFound desc = could not find container \"d8d2cdf9ed5e3b9d8f2e7e571bc75f70731f37d526b3207d40db7dc36e682c8c\": container with ID starting with d8d2cdf9ed5e3b9d8f2e7e571bc75f70731f37d526b3207d40db7dc36e682c8c not found: ID does not exist" Dec 03 12:36:26 crc kubenswrapper[5052]: I1203 12:36:26.537425 5052 scope.go:117] "RemoveContainer" containerID="d671d6c5362c51b3bc62b92cfede167be6efb1c9b8ce80868491d117252d0e6b" Dec 03 12:36:26 crc kubenswrapper[5052]: I1203 12:36:26.552651 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 12:36:26 crc kubenswrapper[5052]: I1203 12:36:26.569835 5052 scope.go:117] "RemoveContainer" containerID="203a8e011f4c47248b78234223150b6dbec13d98ffa7ce6afb0307f2e7edf50f" Dec 03 12:36:26 crc kubenswrapper[5052]: I1203 12:36:26.580084 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 12:36:26 crc kubenswrapper[5052]: I1203 12:36:26.599579 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 12:36:26 crc kubenswrapper[5052]: E1203 12:36:26.600089 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="9dc44369-33d6-48a5-a072-2f9f4311f8f4" containerName="nova-metadata-metadata" Dec 03 12:36:26 crc kubenswrapper[5052]: I1203 12:36:26.600112 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="9dc44369-33d6-48a5-a072-2f9f4311f8f4" containerName="nova-metadata-metadata" Dec 03 12:36:26 crc kubenswrapper[5052]: E1203 12:36:26.600142 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="9dc44369-33d6-48a5-a072-2f9f4311f8f4" containerName="nova-metadata-log" Dec 03 12:36:26 crc kubenswrapper[5052]: I1203 12:36:26.600152 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="9dc44369-33d6-48a5-a072-2f9f4311f8f4" containerName="nova-metadata-log" Dec 03 12:36:26 crc kubenswrapper[5052]: E1203 12:36:26.600176 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="a3a9c880-4b1f-40e9-a45d-4fe8025c5199" containerName="nova-cell1-novncproxy-novncproxy" Dec 03 12:36:26 crc kubenswrapper[5052]: I1203 12:36:26.600186 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="a3a9c880-4b1f-40e9-a45d-4fe8025c5199" containerName="nova-cell1-novncproxy-novncproxy" Dec 03 12:36:26 crc kubenswrapper[5052]: I1203 12:36:26.600400 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="a3a9c880-4b1f-40e9-a45d-4fe8025c5199" containerName="nova-cell1-novncproxy-novncproxy" Dec 03 12:36:26 crc kubenswrapper[5052]: I1203 12:36:26.600427 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="9dc44369-33d6-48a5-a072-2f9f4311f8f4" containerName="nova-metadata-metadata" Dec 03 12:36:26 crc kubenswrapper[5052]: I1203 12:36:26.600437 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="9dc44369-33d6-48a5-a072-2f9f4311f8f4" containerName="nova-metadata-log" Dec 03 12:36:26 crc kubenswrapper[5052]: I1203 12:36:26.601208 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:36:26 crc kubenswrapper[5052]: I1203 12:36:26.603111 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-nova-novncproxy-cell1-public-svc" Dec 03 12:36:26 crc kubenswrapper[5052]: I1203 12:36:26.604228 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-nova-novncproxy-cell1-vencrypt" Dec 03 12:36:26 crc kubenswrapper[5052]: I1203 12:36:26.605650 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell1-novncproxy-config-data" Dec 03 12:36:26 crc kubenswrapper[5052]: I1203 12:36:26.609824 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:36:26 crc kubenswrapper[5052]: I1203 12:36:26.625838 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:36:26 crc kubenswrapper[5052]: I1203 12:36:26.639069 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 12:36:26 crc kubenswrapper[5052]: I1203 12:36:26.652779 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:36:26 crc kubenswrapper[5052]: I1203 12:36:26.654840 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:36:26 crc kubenswrapper[5052]: I1203 12:36:26.659154 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-nova-metadata-internal-svc" Dec 03 12:36:26 crc kubenswrapper[5052]: I1203 12:36:26.659742 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-metadata-config-data" Dec 03 12:36:26 crc kubenswrapper[5052]: I1203 12:36:26.683923 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/f5644eaa-5e51-45cf-a209-832bf54059e8-logs\") pod \"nova-metadata-0\" (UID: \"f5644eaa-5e51-45cf-a209-832bf54059e8\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:36:26 crc kubenswrapper[5052]: I1203 12:36:26.684005 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/f5644eaa-5e51-45cf-a209-832bf54059e8-nova-metadata-tls-certs\") pod \"nova-metadata-0\" (UID: \"f5644eaa-5e51-45cf-a209-832bf54059e8\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:36:26 crc kubenswrapper[5052]: I1203 12:36:26.684024 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nova-novncproxy-tls-certs\" (UniqueName: \"kubernetes.io/secret/a36c62b8-e624-450a-8b71-a33fc4395eee-nova-novncproxy-tls-certs\") pod \"nova-cell1-novncproxy-0\" (UID: \"a36c62b8-e624-450a-8b71-a33fc4395eee\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:36:26 crc kubenswrapper[5052]: I1203 12:36:26.684044 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/f5644eaa-5e51-45cf-a209-832bf54059e8-config-data\") pod \"nova-metadata-0\" (UID: \"f5644eaa-5e51-45cf-a209-832bf54059e8\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:36:26 crc kubenswrapper[5052]: I1203 12:36:26.684060 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a36c62b8-e624-450a-8b71-a33fc4395eee-combined-ca-bundle\") pod \"nova-cell1-novncproxy-0\" (UID: \"a36c62b8-e624-450a-8b71-a33fc4395eee\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:36:26 crc kubenswrapper[5052]: I1203 12:36:26.684104 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fpvzn\" (UniqueName: \"kubernetes.io/projected/f5644eaa-5e51-45cf-a209-832bf54059e8-kube-api-access-fpvzn\") pod \"nova-metadata-0\" (UID: \"f5644eaa-5e51-45cf-a209-832bf54059e8\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:36:26 crc kubenswrapper[5052]: I1203 12:36:26.684120 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a36c62b8-e624-450a-8b71-a33fc4395eee-config-data\") pod \"nova-cell1-novncproxy-0\" (UID: \"a36c62b8-e624-450a-8b71-a33fc4395eee\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:36:26 crc kubenswrapper[5052]: I1203 12:36:26.684159 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x227p\" (UniqueName: \"kubernetes.io/projected/a36c62b8-e624-450a-8b71-a33fc4395eee-kube-api-access-x227p\") pod \"nova-cell1-novncproxy-0\" (UID: \"a36c62b8-e624-450a-8b71-a33fc4395eee\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:36:26 crc kubenswrapper[5052]: I1203 12:36:26.684204 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"vencrypt-tls-certs\" (UniqueName: \"kubernetes.io/secret/a36c62b8-e624-450a-8b71-a33fc4395eee-vencrypt-tls-certs\") pod \"nova-cell1-novncproxy-0\" (UID: \"a36c62b8-e624-450a-8b71-a33fc4395eee\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:36:26 crc kubenswrapper[5052]: I1203 12:36:26.684233 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f5644eaa-5e51-45cf-a209-832bf54059e8-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"f5644eaa-5e51-45cf-a209-832bf54059e8\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:36:26 crc kubenswrapper[5052]: I1203 12:36:26.688759 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:36:26 crc kubenswrapper[5052]: I1203 12:36:26.785753 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-fpvzn\" (UniqueName: \"kubernetes.io/projected/f5644eaa-5e51-45cf-a209-832bf54059e8-kube-api-access-fpvzn\") pod \"nova-metadata-0\" (UID: \"f5644eaa-5e51-45cf-a209-832bf54059e8\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:36:26 crc kubenswrapper[5052]: I1203 12:36:26.785805 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a36c62b8-e624-450a-8b71-a33fc4395eee-config-data\") pod \"nova-cell1-novncproxy-0\" (UID: \"a36c62b8-e624-450a-8b71-a33fc4395eee\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:36:26 crc kubenswrapper[5052]: I1203 12:36:26.785840 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-x227p\" (UniqueName: \"kubernetes.io/projected/a36c62b8-e624-450a-8b71-a33fc4395eee-kube-api-access-x227p\") pod \"nova-cell1-novncproxy-0\" (UID: \"a36c62b8-e624-450a-8b71-a33fc4395eee\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:36:26 crc kubenswrapper[5052]: I1203 12:36:26.785892 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"vencrypt-tls-certs\" (UniqueName: \"kubernetes.io/secret/a36c62b8-e624-450a-8b71-a33fc4395eee-vencrypt-tls-certs\") pod \"nova-cell1-novncproxy-0\" (UID: \"a36c62b8-e624-450a-8b71-a33fc4395eee\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:36:26 crc kubenswrapper[5052]: I1203 12:36:26.785919 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f5644eaa-5e51-45cf-a209-832bf54059e8-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"f5644eaa-5e51-45cf-a209-832bf54059e8\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:36:26 crc kubenswrapper[5052]: I1203 12:36:26.785998 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/f5644eaa-5e51-45cf-a209-832bf54059e8-logs\") pod \"nova-metadata-0\" (UID: \"f5644eaa-5e51-45cf-a209-832bf54059e8\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:36:26 crc kubenswrapper[5052]: I1203 12:36:26.786046 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/f5644eaa-5e51-45cf-a209-832bf54059e8-nova-metadata-tls-certs\") pod \"nova-metadata-0\" (UID: \"f5644eaa-5e51-45cf-a209-832bf54059e8\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:36:26 crc kubenswrapper[5052]: I1203 12:36:26.786062 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"nova-novncproxy-tls-certs\" (UniqueName: \"kubernetes.io/secret/a36c62b8-e624-450a-8b71-a33fc4395eee-nova-novncproxy-tls-certs\") pod \"nova-cell1-novncproxy-0\" (UID: \"a36c62b8-e624-450a-8b71-a33fc4395eee\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:36:26 crc kubenswrapper[5052]: I1203 12:36:26.786090 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/f5644eaa-5e51-45cf-a209-832bf54059e8-config-data\") pod \"nova-metadata-0\" (UID: \"f5644eaa-5e51-45cf-a209-832bf54059e8\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:36:26 crc kubenswrapper[5052]: I1203 12:36:26.786107 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a36c62b8-e624-450a-8b71-a33fc4395eee-combined-ca-bundle\") pod \"nova-cell1-novncproxy-0\" (UID: \"a36c62b8-e624-450a-8b71-a33fc4395eee\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:36:26 crc kubenswrapper[5052]: I1203 12:36:26.787456 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/f5644eaa-5e51-45cf-a209-832bf54059e8-logs\") pod \"nova-metadata-0\" (UID: \"f5644eaa-5e51-45cf-a209-832bf54059e8\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:36:26 crc kubenswrapper[5052]: I1203 12:36:26.790107 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/f5644eaa-5e51-45cf-a209-832bf54059e8-nova-metadata-tls-certs\") pod \"nova-metadata-0\" (UID: \"f5644eaa-5e51-45cf-a209-832bf54059e8\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:36:26 crc kubenswrapper[5052]: I1203 12:36:26.792085 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/f5644eaa-5e51-45cf-a209-832bf54059e8-config-data\") pod \"nova-metadata-0\" (UID: \"f5644eaa-5e51-45cf-a209-832bf54059e8\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:36:26 crc kubenswrapper[5052]: I1203 12:36:26.792341 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a36c62b8-e624-450a-8b71-a33fc4395eee-config-data\") pod \"nova-cell1-novncproxy-0\" (UID: \"a36c62b8-e624-450a-8b71-a33fc4395eee\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:36:26 crc kubenswrapper[5052]: I1203 12:36:26.792742 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a36c62b8-e624-450a-8b71-a33fc4395eee-combined-ca-bundle\") pod \"nova-cell1-novncproxy-0\" (UID: \"a36c62b8-e624-450a-8b71-a33fc4395eee\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:36:26 crc kubenswrapper[5052]: I1203 12:36:26.794958 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"nova-novncproxy-tls-certs\" (UniqueName: \"kubernetes.io/secret/a36c62b8-e624-450a-8b71-a33fc4395eee-nova-novncproxy-tls-certs\") pod \"nova-cell1-novncproxy-0\" (UID: \"a36c62b8-e624-450a-8b71-a33fc4395eee\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:36:26 crc kubenswrapper[5052]: I1203 12:36:26.795197 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f5644eaa-5e51-45cf-a209-832bf54059e8-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"f5644eaa-5e51-45cf-a209-832bf54059e8\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:36:26 crc kubenswrapper[5052]: I1203 12:36:26.804211 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"vencrypt-tls-certs\" (UniqueName: \"kubernetes.io/secret/a36c62b8-e624-450a-8b71-a33fc4395eee-vencrypt-tls-certs\") pod \"nova-cell1-novncproxy-0\" (UID: \"a36c62b8-e624-450a-8b71-a33fc4395eee\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:36:26 crc kubenswrapper[5052]: I1203 12:36:26.807716 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-x227p\" (UniqueName: \"kubernetes.io/projected/a36c62b8-e624-450a-8b71-a33fc4395eee-kube-api-access-x227p\") pod \"nova-cell1-novncproxy-0\" (UID: \"a36c62b8-e624-450a-8b71-a33fc4395eee\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:36:26 crc kubenswrapper[5052]: I1203 12:36:26.808084 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-fpvzn\" (UniqueName: \"kubernetes.io/projected/f5644eaa-5e51-45cf-a209-832bf54059e8-kube-api-access-fpvzn\") pod \"nova-metadata-0\" (UID: \"f5644eaa-5e51-45cf-a209-832bf54059e8\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:36:26 crc kubenswrapper[5052]: I1203 12:36:26.938785 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:36:26 crc kubenswrapper[5052]: I1203 12:36:26.982536 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:36:27 crc kubenswrapper[5052]: I1203 12:36:27.496333 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 12:36:27 crc kubenswrapper[5052]: I1203 12:36:27.535636 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" event={"ID":"a36c62b8-e624-450a-8b71-a33fc4395eee","Type":"ContainerStarted","Data":"91e3d68773399efb138163bf94575ecfabb42bf591d318e7b557f618025a1ff2"} Dec 03 12:36:27 crc kubenswrapper[5052]: I1203 12:36:27.587043 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:36:27 crc kubenswrapper[5052]: W1203 12:36:27.588135 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf5644eaa_5e51_45cf_a209_832bf54059e8.slice/crio-81d1f20c541ac71456e0eb2239f15f38d64a708b55c52ce57c344042f0f83b32 WatchSource:0}: Error finding container 81d1f20c541ac71456e0eb2239f15f38d64a708b55c52ce57c344042f0f83b32: Status 404 returned error can't find the container with id 81d1f20c541ac71456e0eb2239f15f38d64a708b55c52ce57c344042f0f83b32 Dec 03 12:36:27 crc kubenswrapper[5052]: I1203 12:36:27.707929 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9dc44369-33d6-48a5-a072-2f9f4311f8f4" path="/var/lib/kubelet/pods/9dc44369-33d6-48a5-a072-2f9f4311f8f4/volumes" Dec 03 12:36:27 crc kubenswrapper[5052]: I1203 12:36:27.708876 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a3a9c880-4b1f-40e9-a45d-4fe8025c5199" path="/var/lib/kubelet/pods/a3a9c880-4b1f-40e9-a45d-4fe8025c5199/volumes" Dec 03 12:36:28 crc kubenswrapper[5052]: I1203 12:36:28.553452 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" event={"ID":"a36c62b8-e624-450a-8b71-a33fc4395eee","Type":"ContainerStarted","Data":"5970858020fb206863e1145b9231a3f86c94c1e2d5ed380dc57653d6868c9b58"} Dec 03 12:36:28 crc kubenswrapper[5052]: I1203 12:36:28.562423 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"f5644eaa-5e51-45cf-a209-832bf54059e8","Type":"ContainerStarted","Data":"97f59a6afe168b8d7ae74029ad90982e6b480edbeb2a63449ea23b3a3491d800"} Dec 03 12:36:28 crc kubenswrapper[5052]: I1203 12:36:28.562995 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"f5644eaa-5e51-45cf-a209-832bf54059e8","Type":"ContainerStarted","Data":"30fcf10ad9445fe61aacd4dec74cceae98c6012f42a28ae265f12ad48acc0c92"} Dec 03 12:36:28 crc kubenswrapper[5052]: I1203 12:36:28.563010 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"f5644eaa-5e51-45cf-a209-832bf54059e8","Type":"ContainerStarted","Data":"81d1f20c541ac71456e0eb2239f15f38d64a708b55c52ce57c344042f0f83b32"} Dec 03 12:36:28 crc kubenswrapper[5052]: I1203 12:36:28.584138 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" podStartSLOduration=2.584119342 podStartE2EDuration="2.584119342s" podCreationTimestamp="2025-12-03 12:36:26 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:36:28.582909778 +0000 UTC m=+4579.171151542" watchObservedRunningTime="2025-12-03 12:36:28.584119342 +0000 UTC m=+4579.172361106" Dec 03 12:36:28 crc kubenswrapper[5052]: I1203 12:36:28.643972 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-metadata-0" podStartSLOduration=2.643953334 podStartE2EDuration="2.643953334s" podCreationTimestamp="2025-12-03 12:36:26 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:36:28.628823631 +0000 UTC m=+4579.217065395" watchObservedRunningTime="2025-12-03 12:36:28.643953334 +0000 UTC m=+4579.232195098" Dec 03 12:36:29 crc kubenswrapper[5052]: I1203 12:36:29.679789 5052 scope.go:117] "RemoveContainer" containerID="e6edaf188f85ea4f11f487f11cc489c700f25fb42b5ee269f4f496f1e127d772" Dec 03 12:36:29 crc kubenswrapper[5052]: E1203 12:36:29.682339 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 12:36:30 crc kubenswrapper[5052]: I1203 12:36:30.482580 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:36:30 crc kubenswrapper[5052]: I1203 12:36:30.512060 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:36:30 crc kubenswrapper[5052]: I1203 12:36:30.584587 5052 generic.go:334] "Generic (PLEG): container finished" podID="772e7623-3c44-4375-9f34-5ba8683d5fe1" containerID="e732642ec3d2e2c1b77a5c5cf4d43de4178312bd401dae5fa03708d2e3c9248f" exitCode=0 Dec 03 12:36:30 crc kubenswrapper[5052]: I1203 12:36:30.584926 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-cell-mapping-x6pl4" event={"ID":"772e7623-3c44-4375-9f34-5ba8683d5fe1","Type":"ContainerDied","Data":"e732642ec3d2e2c1b77a5c5cf4d43de4178312bd401dae5fa03708d2e3c9248f"} Dec 03 12:36:30 crc kubenswrapper[5052]: I1203 12:36:30.613985 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:36:30 crc kubenswrapper[5052]: I1203 12:36:30.811873 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:36:30 crc kubenswrapper[5052]: I1203 12:36:30.811959 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:36:31 crc kubenswrapper[5052]: I1203 12:36:31.893858 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-api-0" podUID="7b6bb151-d113-40e6-97a3-644ca7e4e5f3" containerName="nova-api-api" probeResult="failure" output="Get \"http://10.217.1.76:8774/\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Dec 03 12:36:31 crc kubenswrapper[5052]: I1203 12:36:31.893844 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-api-0" podUID="7b6bb151-d113-40e6-97a3-644ca7e4e5f3" containerName="nova-api-log" probeResult="failure" output="Get \"http://10.217.1.76:8774/\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Dec 03 12:36:31 crc kubenswrapper[5052]: I1203 12:36:31.940984 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:36:31 crc kubenswrapper[5052]: I1203 12:36:31.982952 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:36:31 crc kubenswrapper[5052]: I1203 12:36:31.983496 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:36:32 crc kubenswrapper[5052]: I1203 12:36:32.025976 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-cell-mapping-x6pl4" Dec 03 12:36:32 crc kubenswrapper[5052]: I1203 12:36:32.113656 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/772e7623-3c44-4375-9f34-5ba8683d5fe1-combined-ca-bundle\") pod \"772e7623-3c44-4375-9f34-5ba8683d5fe1\" (UID: \"772e7623-3c44-4375-9f34-5ba8683d5fe1\") " Dec 03 12:36:32 crc kubenswrapper[5052]: I1203 12:36:32.113726 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/772e7623-3c44-4375-9f34-5ba8683d5fe1-scripts\") pod \"772e7623-3c44-4375-9f34-5ba8683d5fe1\" (UID: \"772e7623-3c44-4375-9f34-5ba8683d5fe1\") " Dec 03 12:36:32 crc kubenswrapper[5052]: I1203 12:36:32.113846 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8p8r6\" (UniqueName: \"kubernetes.io/projected/772e7623-3c44-4375-9f34-5ba8683d5fe1-kube-api-access-8p8r6\") pod \"772e7623-3c44-4375-9f34-5ba8683d5fe1\" (UID: \"772e7623-3c44-4375-9f34-5ba8683d5fe1\") " Dec 03 12:36:32 crc kubenswrapper[5052]: I1203 12:36:32.113929 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/772e7623-3c44-4375-9f34-5ba8683d5fe1-config-data\") pod \"772e7623-3c44-4375-9f34-5ba8683d5fe1\" (UID: \"772e7623-3c44-4375-9f34-5ba8683d5fe1\") " Dec 03 12:36:32 crc kubenswrapper[5052]: I1203 12:36:32.119246 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/772e7623-3c44-4375-9f34-5ba8683d5fe1-scripts" (OuterVolumeSpecName: "scripts") pod "772e7623-3c44-4375-9f34-5ba8683d5fe1" (UID: "772e7623-3c44-4375-9f34-5ba8683d5fe1"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:36:32 crc kubenswrapper[5052]: I1203 12:36:32.119738 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/772e7623-3c44-4375-9f34-5ba8683d5fe1-kube-api-access-8p8r6" (OuterVolumeSpecName: "kube-api-access-8p8r6") pod "772e7623-3c44-4375-9f34-5ba8683d5fe1" (UID: "772e7623-3c44-4375-9f34-5ba8683d5fe1"). InnerVolumeSpecName "kube-api-access-8p8r6". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:36:32 crc kubenswrapper[5052]: I1203 12:36:32.144669 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/772e7623-3c44-4375-9f34-5ba8683d5fe1-config-data" (OuterVolumeSpecName: "config-data") pod "772e7623-3c44-4375-9f34-5ba8683d5fe1" (UID: "772e7623-3c44-4375-9f34-5ba8683d5fe1"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:36:32 crc kubenswrapper[5052]: I1203 12:36:32.147673 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/772e7623-3c44-4375-9f34-5ba8683d5fe1-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "772e7623-3c44-4375-9f34-5ba8683d5fe1" (UID: "772e7623-3c44-4375-9f34-5ba8683d5fe1"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:36:32 crc kubenswrapper[5052]: I1203 12:36:32.217924 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/772e7623-3c44-4375-9f34-5ba8683d5fe1-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:36:32 crc kubenswrapper[5052]: I1203 12:36:32.217975 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/772e7623-3c44-4375-9f34-5ba8683d5fe1-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:36:32 crc kubenswrapper[5052]: I1203 12:36:32.217992 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-8p8r6\" (UniqueName: \"kubernetes.io/projected/772e7623-3c44-4375-9f34-5ba8683d5fe1-kube-api-access-8p8r6\") on node \"crc\" DevicePath \"\"" Dec 03 12:36:32 crc kubenswrapper[5052]: I1203 12:36:32.218006 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/772e7623-3c44-4375-9f34-5ba8683d5fe1-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:36:32 crc kubenswrapper[5052]: I1203 12:36:32.606784 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-cell-mapping-x6pl4" event={"ID":"772e7623-3c44-4375-9f34-5ba8683d5fe1","Type":"ContainerDied","Data":"a1df95494e68580aaf70278576fe02128cecde30611a4b551f6ccb08ea9754d5"} Dec 03 12:36:32 crc kubenswrapper[5052]: I1203 12:36:32.606833 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a1df95494e68580aaf70278576fe02128cecde30611a4b551f6ccb08ea9754d5" Dec 03 12:36:32 crc kubenswrapper[5052]: I1203 12:36:32.606853 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-cell-mapping-x6pl4" Dec 03 12:36:32 crc kubenswrapper[5052]: I1203 12:36:32.802227 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:36:32 crc kubenswrapper[5052]: I1203 12:36:32.802473 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-api-0" podUID="7b6bb151-d113-40e6-97a3-644ca7e4e5f3" containerName="nova-api-log" containerID="cri-o://62321816c49aa66fe160c689b7c7f7a1345ff820851a3c51ec682204697cbbe5" gracePeriod=30 Dec 03 12:36:32 crc kubenswrapper[5052]: I1203 12:36:32.802548 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-api-0" podUID="7b6bb151-d113-40e6-97a3-644ca7e4e5f3" containerName="nova-api-api" containerID="cri-o://0b2f979945fddbeb900f47fb601b5b1707af68be110b9c7e7a0ec564f0e78e54" gracePeriod=30 Dec 03 12:36:32 crc kubenswrapper[5052]: I1203 12:36:32.815901 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:36:32 crc kubenswrapper[5052]: I1203 12:36:32.816081 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-scheduler-0" podUID="31ee3320-d8b9-4f85-9500-3016e2edf1f7" containerName="nova-scheduler-scheduler" containerID="cri-o://fb06a935923602d2b3fa6510b0f711cbdca186b1223cbdab3975ad3887386af8" gracePeriod=30 Dec 03 12:36:32 crc kubenswrapper[5052]: I1203 12:36:32.866135 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:36:32 crc kubenswrapper[5052]: I1203 12:36:32.866400 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-metadata-0" podUID="f5644eaa-5e51-45cf-a209-832bf54059e8" containerName="nova-metadata-log" containerID="cri-o://30fcf10ad9445fe61aacd4dec74cceae98c6012f42a28ae265f12ad48acc0c92" gracePeriod=30 Dec 03 12:36:32 crc kubenswrapper[5052]: I1203 12:36:32.866895 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-metadata-0" podUID="f5644eaa-5e51-45cf-a209-832bf54059e8" containerName="nova-metadata-metadata" containerID="cri-o://97f59a6afe168b8d7ae74029ad90982e6b480edbeb2a63449ea23b3a3491d800" gracePeriod=30 Dec 03 12:36:33 crc kubenswrapper[5052]: I1203 12:36:33.623941 5052 generic.go:334] "Generic (PLEG): container finished" podID="f5644eaa-5e51-45cf-a209-832bf54059e8" containerID="97f59a6afe168b8d7ae74029ad90982e6b480edbeb2a63449ea23b3a3491d800" exitCode=0 Dec 03 12:36:33 crc kubenswrapper[5052]: I1203 12:36:33.624274 5052 generic.go:334] "Generic (PLEG): container finished" podID="f5644eaa-5e51-45cf-a209-832bf54059e8" containerID="30fcf10ad9445fe61aacd4dec74cceae98c6012f42a28ae265f12ad48acc0c92" exitCode=143 Dec 03 12:36:33 crc kubenswrapper[5052]: I1203 12:36:33.624084 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"f5644eaa-5e51-45cf-a209-832bf54059e8","Type":"ContainerDied","Data":"97f59a6afe168b8d7ae74029ad90982e6b480edbeb2a63449ea23b3a3491d800"} Dec 03 12:36:33 crc kubenswrapper[5052]: I1203 12:36:33.624372 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"f5644eaa-5e51-45cf-a209-832bf54059e8","Type":"ContainerDied","Data":"30fcf10ad9445fe61aacd4dec74cceae98c6012f42a28ae265f12ad48acc0c92"} Dec 03 12:36:33 crc kubenswrapper[5052]: I1203 12:36:33.627067 5052 generic.go:334] "Generic (PLEG): container finished" podID="7b6bb151-d113-40e6-97a3-644ca7e4e5f3" containerID="62321816c49aa66fe160c689b7c7f7a1345ff820851a3c51ec682204697cbbe5" exitCode=143 Dec 03 12:36:33 crc kubenswrapper[5052]: I1203 12:36:33.627106 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"7b6bb151-d113-40e6-97a3-644ca7e4e5f3","Type":"ContainerDied","Data":"62321816c49aa66fe160c689b7c7f7a1345ff820851a3c51ec682204697cbbe5"} Dec 03 12:36:34 crc kubenswrapper[5052]: I1203 12:36:34.234380 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:36:34 crc kubenswrapper[5052]: I1203 12:36:34.361073 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fpvzn\" (UniqueName: \"kubernetes.io/projected/f5644eaa-5e51-45cf-a209-832bf54059e8-kube-api-access-fpvzn\") pod \"f5644eaa-5e51-45cf-a209-832bf54059e8\" (UID: \"f5644eaa-5e51-45cf-a209-832bf54059e8\") " Dec 03 12:36:34 crc kubenswrapper[5052]: I1203 12:36:34.361340 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/f5644eaa-5e51-45cf-a209-832bf54059e8-nova-metadata-tls-certs\") pod \"f5644eaa-5e51-45cf-a209-832bf54059e8\" (UID: \"f5644eaa-5e51-45cf-a209-832bf54059e8\") " Dec 03 12:36:34 crc kubenswrapper[5052]: I1203 12:36:34.361395 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/f5644eaa-5e51-45cf-a209-832bf54059e8-config-data\") pod \"f5644eaa-5e51-45cf-a209-832bf54059e8\" (UID: \"f5644eaa-5e51-45cf-a209-832bf54059e8\") " Dec 03 12:36:34 crc kubenswrapper[5052]: I1203 12:36:34.361443 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/f5644eaa-5e51-45cf-a209-832bf54059e8-logs\") pod \"f5644eaa-5e51-45cf-a209-832bf54059e8\" (UID: \"f5644eaa-5e51-45cf-a209-832bf54059e8\") " Dec 03 12:36:34 crc kubenswrapper[5052]: I1203 12:36:34.362047 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f5644eaa-5e51-45cf-a209-832bf54059e8-combined-ca-bundle\") pod \"f5644eaa-5e51-45cf-a209-832bf54059e8\" (UID: \"f5644eaa-5e51-45cf-a209-832bf54059e8\") " Dec 03 12:36:34 crc kubenswrapper[5052]: I1203 12:36:34.363430 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f5644eaa-5e51-45cf-a209-832bf54059e8-logs" (OuterVolumeSpecName: "logs") pod "f5644eaa-5e51-45cf-a209-832bf54059e8" (UID: "f5644eaa-5e51-45cf-a209-832bf54059e8"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:36:34 crc kubenswrapper[5052]: I1203 12:36:34.424822 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f5644eaa-5e51-45cf-a209-832bf54059e8-kube-api-access-fpvzn" (OuterVolumeSpecName: "kube-api-access-fpvzn") pod "f5644eaa-5e51-45cf-a209-832bf54059e8" (UID: "f5644eaa-5e51-45cf-a209-832bf54059e8"). InnerVolumeSpecName "kube-api-access-fpvzn". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:36:34 crc kubenswrapper[5052]: I1203 12:36:34.429336 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f5644eaa-5e51-45cf-a209-832bf54059e8-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "f5644eaa-5e51-45cf-a209-832bf54059e8" (UID: "f5644eaa-5e51-45cf-a209-832bf54059e8"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:36:34 crc kubenswrapper[5052]: I1203 12:36:34.443954 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f5644eaa-5e51-45cf-a209-832bf54059e8-config-data" (OuterVolumeSpecName: "config-data") pod "f5644eaa-5e51-45cf-a209-832bf54059e8" (UID: "f5644eaa-5e51-45cf-a209-832bf54059e8"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:36:34 crc kubenswrapper[5052]: I1203 12:36:34.466729 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-fpvzn\" (UniqueName: \"kubernetes.io/projected/f5644eaa-5e51-45cf-a209-832bf54059e8-kube-api-access-fpvzn\") on node \"crc\" DevicePath \"\"" Dec 03 12:36:34 crc kubenswrapper[5052]: I1203 12:36:34.467062 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/f5644eaa-5e51-45cf-a209-832bf54059e8-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:36:34 crc kubenswrapper[5052]: I1203 12:36:34.467127 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/f5644eaa-5e51-45cf-a209-832bf54059e8-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:36:34 crc kubenswrapper[5052]: I1203 12:36:34.467183 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f5644eaa-5e51-45cf-a209-832bf54059e8-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:36:34 crc kubenswrapper[5052]: I1203 12:36:34.583726 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f5644eaa-5e51-45cf-a209-832bf54059e8-nova-metadata-tls-certs" (OuterVolumeSpecName: "nova-metadata-tls-certs") pod "f5644eaa-5e51-45cf-a209-832bf54059e8" (UID: "f5644eaa-5e51-45cf-a209-832bf54059e8"). InnerVolumeSpecName "nova-metadata-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:36:34 crc kubenswrapper[5052]: I1203 12:36:34.621881 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:36:34 crc kubenswrapper[5052]: I1203 12:36:34.667722 5052 generic.go:334] "Generic (PLEG): container finished" podID="32d4c4ba-c6dd-49cf-80a3-4d7ed71cb6b5" containerID="ffc601b182d6f61d49cc3f4cc073d39e73625c22b23b4c46e4b1d092e911918c" exitCode=0 Dec 03 12:36:34 crc kubenswrapper[5052]: I1203 12:36:34.667834 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-2v6n2" event={"ID":"32d4c4ba-c6dd-49cf-80a3-4d7ed71cb6b5","Type":"ContainerDied","Data":"ffc601b182d6f61d49cc3f4cc073d39e73625c22b23b4c46e4b1d092e911918c"} Dec 03 12:36:34 crc kubenswrapper[5052]: I1203 12:36:34.674868 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/31ee3320-d8b9-4f85-9500-3016e2edf1f7-combined-ca-bundle\") pod \"31ee3320-d8b9-4f85-9500-3016e2edf1f7\" (UID: \"31ee3320-d8b9-4f85-9500-3016e2edf1f7\") " Dec 03 12:36:34 crc kubenswrapper[5052]: I1203 12:36:34.674958 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/31ee3320-d8b9-4f85-9500-3016e2edf1f7-config-data\") pod \"31ee3320-d8b9-4f85-9500-3016e2edf1f7\" (UID: \"31ee3320-d8b9-4f85-9500-3016e2edf1f7\") " Dec 03 12:36:34 crc kubenswrapper[5052]: I1203 12:36:34.675022 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-478rx\" (UniqueName: \"kubernetes.io/projected/31ee3320-d8b9-4f85-9500-3016e2edf1f7-kube-api-access-478rx\") pod \"31ee3320-d8b9-4f85-9500-3016e2edf1f7\" (UID: \"31ee3320-d8b9-4f85-9500-3016e2edf1f7\") " Dec 03 12:36:34 crc kubenswrapper[5052]: I1203 12:36:34.675481 5052 reconciler_common.go:293] "Volume detached for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/f5644eaa-5e51-45cf-a209-832bf54059e8-nova-metadata-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:36:34 crc kubenswrapper[5052]: I1203 12:36:34.690830 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/31ee3320-d8b9-4f85-9500-3016e2edf1f7-kube-api-access-478rx" (OuterVolumeSpecName: "kube-api-access-478rx") pod "31ee3320-d8b9-4f85-9500-3016e2edf1f7" (UID: "31ee3320-d8b9-4f85-9500-3016e2edf1f7"). InnerVolumeSpecName "kube-api-access-478rx". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:36:34 crc kubenswrapper[5052]: I1203 12:36:34.695925 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"f5644eaa-5e51-45cf-a209-832bf54059e8","Type":"ContainerDied","Data":"81d1f20c541ac71456e0eb2239f15f38d64a708b55c52ce57c344042f0f83b32"} Dec 03 12:36:34 crc kubenswrapper[5052]: I1203 12:36:34.695988 5052 scope.go:117] "RemoveContainer" containerID="97f59a6afe168b8d7ae74029ad90982e6b480edbeb2a63449ea23b3a3491d800" Dec 03 12:36:34 crc kubenswrapper[5052]: I1203 12:36:34.696307 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:36:34 crc kubenswrapper[5052]: I1203 12:36:34.718321 5052 generic.go:334] "Generic (PLEG): container finished" podID="31ee3320-d8b9-4f85-9500-3016e2edf1f7" containerID="fb06a935923602d2b3fa6510b0f711cbdca186b1223cbdab3975ad3887386af8" exitCode=0 Dec 03 12:36:34 crc kubenswrapper[5052]: I1203 12:36:34.718366 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"31ee3320-d8b9-4f85-9500-3016e2edf1f7","Type":"ContainerDied","Data":"fb06a935923602d2b3fa6510b0f711cbdca186b1223cbdab3975ad3887386af8"} Dec 03 12:36:34 crc kubenswrapper[5052]: I1203 12:36:34.718395 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"31ee3320-d8b9-4f85-9500-3016e2edf1f7","Type":"ContainerDied","Data":"1055239fd3d01ec31db4e05253ae205fa3c1f67e2d0d5298a42bfeae540194d3"} Dec 03 12:36:34 crc kubenswrapper[5052]: I1203 12:36:34.718444 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:36:34 crc kubenswrapper[5052]: I1203 12:36:34.749400 5052 scope.go:117] "RemoveContainer" containerID="30fcf10ad9445fe61aacd4dec74cceae98c6012f42a28ae265f12ad48acc0c92" Dec 03 12:36:34 crc kubenswrapper[5052]: I1203 12:36:34.776559 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:36:34 crc kubenswrapper[5052]: I1203 12:36:34.779600 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-478rx\" (UniqueName: \"kubernetes.io/projected/31ee3320-d8b9-4f85-9500-3016e2edf1f7-kube-api-access-478rx\") on node \"crc\" DevicePath \"\"" Dec 03 12:36:34 crc kubenswrapper[5052]: I1203 12:36:34.786989 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:36:34 crc kubenswrapper[5052]: I1203 12:36:34.800700 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/31ee3320-d8b9-4f85-9500-3016e2edf1f7-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "31ee3320-d8b9-4f85-9500-3016e2edf1f7" (UID: "31ee3320-d8b9-4f85-9500-3016e2edf1f7"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:36:34 crc kubenswrapper[5052]: I1203 12:36:34.807890 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:36:34 crc kubenswrapper[5052]: E1203 12:36:34.808262 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="f5644eaa-5e51-45cf-a209-832bf54059e8" containerName="nova-metadata-log" Dec 03 12:36:34 crc kubenswrapper[5052]: I1203 12:36:34.808273 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="f5644eaa-5e51-45cf-a209-832bf54059e8" containerName="nova-metadata-log" Dec 03 12:36:34 crc kubenswrapper[5052]: E1203 12:36:34.808307 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="31ee3320-d8b9-4f85-9500-3016e2edf1f7" containerName="nova-scheduler-scheduler" Dec 03 12:36:34 crc kubenswrapper[5052]: I1203 12:36:34.808313 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="31ee3320-d8b9-4f85-9500-3016e2edf1f7" containerName="nova-scheduler-scheduler" Dec 03 12:36:34 crc kubenswrapper[5052]: E1203 12:36:34.808330 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="772e7623-3c44-4375-9f34-5ba8683d5fe1" containerName="nova-manage" Dec 03 12:36:34 crc kubenswrapper[5052]: I1203 12:36:34.808337 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="772e7623-3c44-4375-9f34-5ba8683d5fe1" containerName="nova-manage" Dec 03 12:36:34 crc kubenswrapper[5052]: E1203 12:36:34.808351 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="f5644eaa-5e51-45cf-a209-832bf54059e8" containerName="nova-metadata-metadata" Dec 03 12:36:34 crc kubenswrapper[5052]: I1203 12:36:34.808358 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="f5644eaa-5e51-45cf-a209-832bf54059e8" containerName="nova-metadata-metadata" Dec 03 12:36:34 crc kubenswrapper[5052]: I1203 12:36:34.808525 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="f5644eaa-5e51-45cf-a209-832bf54059e8" containerName="nova-metadata-metadata" Dec 03 12:36:34 crc kubenswrapper[5052]: I1203 12:36:34.808552 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="f5644eaa-5e51-45cf-a209-832bf54059e8" containerName="nova-metadata-log" Dec 03 12:36:34 crc kubenswrapper[5052]: I1203 12:36:34.808573 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="31ee3320-d8b9-4f85-9500-3016e2edf1f7" containerName="nova-scheduler-scheduler" Dec 03 12:36:34 crc kubenswrapper[5052]: I1203 12:36:34.808592 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="772e7623-3c44-4375-9f34-5ba8683d5fe1" containerName="nova-manage" Dec 03 12:36:34 crc kubenswrapper[5052]: I1203 12:36:34.809759 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:36:34 crc kubenswrapper[5052]: I1203 12:36:34.817071 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/31ee3320-d8b9-4f85-9500-3016e2edf1f7-config-data" (OuterVolumeSpecName: "config-data") pod "31ee3320-d8b9-4f85-9500-3016e2edf1f7" (UID: "31ee3320-d8b9-4f85-9500-3016e2edf1f7"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:36:34 crc kubenswrapper[5052]: I1203 12:36:34.817549 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-nova-metadata-internal-svc" Dec 03 12:36:34 crc kubenswrapper[5052]: I1203 12:36:34.817660 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-metadata-config-data" Dec 03 12:36:34 crc kubenswrapper[5052]: I1203 12:36:34.818477 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:36:34 crc kubenswrapper[5052]: I1203 12:36:34.829704 5052 scope.go:117] "RemoveContainer" containerID="fb06a935923602d2b3fa6510b0f711cbdca186b1223cbdab3975ad3887386af8" Dec 03 12:36:34 crc kubenswrapper[5052]: I1203 12:36:34.860582 5052 scope.go:117] "RemoveContainer" containerID="fb06a935923602d2b3fa6510b0f711cbdca186b1223cbdab3975ad3887386af8" Dec 03 12:36:34 crc kubenswrapper[5052]: E1203 12:36:34.860887 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fb06a935923602d2b3fa6510b0f711cbdca186b1223cbdab3975ad3887386af8\": container with ID starting with fb06a935923602d2b3fa6510b0f711cbdca186b1223cbdab3975ad3887386af8 not found: ID does not exist" containerID="fb06a935923602d2b3fa6510b0f711cbdca186b1223cbdab3975ad3887386af8" Dec 03 12:36:34 crc kubenswrapper[5052]: I1203 12:36:34.860911 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fb06a935923602d2b3fa6510b0f711cbdca186b1223cbdab3975ad3887386af8"} err="failed to get container status \"fb06a935923602d2b3fa6510b0f711cbdca186b1223cbdab3975ad3887386af8\": rpc error: code = NotFound desc = could not find container \"fb06a935923602d2b3fa6510b0f711cbdca186b1223cbdab3975ad3887386af8\": container with ID starting with fb06a935923602d2b3fa6510b0f711cbdca186b1223cbdab3975ad3887386af8 not found: ID does not exist" Dec 03 12:36:34 crc kubenswrapper[5052]: I1203 12:36:34.881643 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/69502269-3194-440d-aa90-8a9a1821a297-nova-metadata-tls-certs\") pod \"nova-metadata-0\" (UID: \"69502269-3194-440d-aa90-8a9a1821a297\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:36:34 crc kubenswrapper[5052]: I1203 12:36:34.881786 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/69502269-3194-440d-aa90-8a9a1821a297-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"69502269-3194-440d-aa90-8a9a1821a297\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:36:34 crc kubenswrapper[5052]: I1203 12:36:34.881975 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/69502269-3194-440d-aa90-8a9a1821a297-logs\") pod \"nova-metadata-0\" (UID: \"69502269-3194-440d-aa90-8a9a1821a297\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:36:34 crc kubenswrapper[5052]: I1203 12:36:34.882241 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bnl48\" (UniqueName: \"kubernetes.io/projected/69502269-3194-440d-aa90-8a9a1821a297-kube-api-access-bnl48\") pod \"nova-metadata-0\" (UID: \"69502269-3194-440d-aa90-8a9a1821a297\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:36:34 crc kubenswrapper[5052]: I1203 12:36:34.882360 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/69502269-3194-440d-aa90-8a9a1821a297-config-data\") pod \"nova-metadata-0\" (UID: \"69502269-3194-440d-aa90-8a9a1821a297\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:36:34 crc kubenswrapper[5052]: I1203 12:36:34.882589 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/31ee3320-d8b9-4f85-9500-3016e2edf1f7-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:36:34 crc kubenswrapper[5052]: I1203 12:36:34.882609 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/31ee3320-d8b9-4f85-9500-3016e2edf1f7-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:36:34 crc kubenswrapper[5052]: I1203 12:36:34.984065 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/69502269-3194-440d-aa90-8a9a1821a297-config-data\") pod \"nova-metadata-0\" (UID: \"69502269-3194-440d-aa90-8a9a1821a297\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:36:34 crc kubenswrapper[5052]: I1203 12:36:34.984455 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/69502269-3194-440d-aa90-8a9a1821a297-nova-metadata-tls-certs\") pod \"nova-metadata-0\" (UID: \"69502269-3194-440d-aa90-8a9a1821a297\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:36:34 crc kubenswrapper[5052]: I1203 12:36:34.984512 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/69502269-3194-440d-aa90-8a9a1821a297-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"69502269-3194-440d-aa90-8a9a1821a297\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:36:34 crc kubenswrapper[5052]: I1203 12:36:34.984579 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/69502269-3194-440d-aa90-8a9a1821a297-logs\") pod \"nova-metadata-0\" (UID: \"69502269-3194-440d-aa90-8a9a1821a297\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:36:34 crc kubenswrapper[5052]: I1203 12:36:34.984704 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-bnl48\" (UniqueName: \"kubernetes.io/projected/69502269-3194-440d-aa90-8a9a1821a297-kube-api-access-bnl48\") pod \"nova-metadata-0\" (UID: \"69502269-3194-440d-aa90-8a9a1821a297\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:36:34 crc kubenswrapper[5052]: I1203 12:36:34.985243 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/69502269-3194-440d-aa90-8a9a1821a297-logs\") pod \"nova-metadata-0\" (UID: \"69502269-3194-440d-aa90-8a9a1821a297\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:36:34 crc kubenswrapper[5052]: I1203 12:36:34.988910 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/69502269-3194-440d-aa90-8a9a1821a297-config-data\") pod \"nova-metadata-0\" (UID: \"69502269-3194-440d-aa90-8a9a1821a297\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:36:34 crc kubenswrapper[5052]: I1203 12:36:34.989601 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/69502269-3194-440d-aa90-8a9a1821a297-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"69502269-3194-440d-aa90-8a9a1821a297\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:36:34 crc kubenswrapper[5052]: I1203 12:36:34.991215 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/69502269-3194-440d-aa90-8a9a1821a297-nova-metadata-tls-certs\") pod \"nova-metadata-0\" (UID: \"69502269-3194-440d-aa90-8a9a1821a297\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:36:35 crc kubenswrapper[5052]: I1203 12:36:35.001230 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-bnl48\" (UniqueName: \"kubernetes.io/projected/69502269-3194-440d-aa90-8a9a1821a297-kube-api-access-bnl48\") pod \"nova-metadata-0\" (UID: \"69502269-3194-440d-aa90-8a9a1821a297\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:36:35 crc kubenswrapper[5052]: I1203 12:36:35.111139 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:36:35 crc kubenswrapper[5052]: I1203 12:36:35.136695 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:36:35 crc kubenswrapper[5052]: I1203 12:36:35.143988 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:36:35 crc kubenswrapper[5052]: I1203 12:36:35.149480 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:36:35 crc kubenswrapper[5052]: I1203 12:36:35.150999 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:36:35 crc kubenswrapper[5052]: I1203 12:36:35.155074 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-scheduler-config-data" Dec 03 12:36:35 crc kubenswrapper[5052]: I1203 12:36:35.164521 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:36:35 crc kubenswrapper[5052]: I1203 12:36:35.290931 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r46rj\" (UniqueName: \"kubernetes.io/projected/1316edb7-1894-427d-92bf-dab018e056fd-kube-api-access-r46rj\") pod \"nova-scheduler-0\" (UID: \"1316edb7-1894-427d-92bf-dab018e056fd\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:36:35 crc kubenswrapper[5052]: I1203 12:36:35.291093 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/1316edb7-1894-427d-92bf-dab018e056fd-config-data\") pod \"nova-scheduler-0\" (UID: \"1316edb7-1894-427d-92bf-dab018e056fd\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:36:35 crc kubenswrapper[5052]: I1203 12:36:35.291166 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1316edb7-1894-427d-92bf-dab018e056fd-combined-ca-bundle\") pod \"nova-scheduler-0\" (UID: \"1316edb7-1894-427d-92bf-dab018e056fd\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:36:35 crc kubenswrapper[5052]: I1203 12:36:35.393204 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1316edb7-1894-427d-92bf-dab018e056fd-combined-ca-bundle\") pod \"nova-scheduler-0\" (UID: \"1316edb7-1894-427d-92bf-dab018e056fd\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:36:35 crc kubenswrapper[5052]: I1203 12:36:35.393582 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-r46rj\" (UniqueName: \"kubernetes.io/projected/1316edb7-1894-427d-92bf-dab018e056fd-kube-api-access-r46rj\") pod \"nova-scheduler-0\" (UID: \"1316edb7-1894-427d-92bf-dab018e056fd\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:36:35 crc kubenswrapper[5052]: I1203 12:36:35.393773 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/1316edb7-1894-427d-92bf-dab018e056fd-config-data\") pod \"nova-scheduler-0\" (UID: \"1316edb7-1894-427d-92bf-dab018e056fd\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:36:35 crc kubenswrapper[5052]: I1203 12:36:35.613455 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:36:35 crc kubenswrapper[5052]: I1203 12:36:35.685142 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="31ee3320-d8b9-4f85-9500-3016e2edf1f7" path="/var/lib/kubelet/pods/31ee3320-d8b9-4f85-9500-3016e2edf1f7/volumes" Dec 03 12:36:35 crc kubenswrapper[5052]: I1203 12:36:35.685988 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f5644eaa-5e51-45cf-a209-832bf54059e8" path="/var/lib/kubelet/pods/f5644eaa-5e51-45cf-a209-832bf54059e8/volumes" Dec 03 12:36:35 crc kubenswrapper[5052]: I1203 12:36:35.724273 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/1316edb7-1894-427d-92bf-dab018e056fd-config-data\") pod \"nova-scheduler-0\" (UID: \"1316edb7-1894-427d-92bf-dab018e056fd\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:36:35 crc kubenswrapper[5052]: I1203 12:36:35.725233 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-r46rj\" (UniqueName: \"kubernetes.io/projected/1316edb7-1894-427d-92bf-dab018e056fd-kube-api-access-r46rj\") pod \"nova-scheduler-0\" (UID: \"1316edb7-1894-427d-92bf-dab018e056fd\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:36:35 crc kubenswrapper[5052]: I1203 12:36:35.727737 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1316edb7-1894-427d-92bf-dab018e056fd-combined-ca-bundle\") pod \"nova-scheduler-0\" (UID: \"1316edb7-1894-427d-92bf-dab018e056fd\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:36:35 crc kubenswrapper[5052]: W1203 12:36:35.730204 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod69502269_3194_440d_aa90_8a9a1821a297.slice/crio-6a2d13b006cb0eb1850f1ce57e0b8be7759b8e5ca66aca71f042c5738a6afa97 WatchSource:0}: Error finding container 6a2d13b006cb0eb1850f1ce57e0b8be7759b8e5ca66aca71f042c5738a6afa97: Status 404 returned error can't find the container with id 6a2d13b006cb0eb1850f1ce57e0b8be7759b8e5ca66aca71f042c5738a6afa97 Dec 03 12:36:35 crc kubenswrapper[5052]: I1203 12:36:35.767477 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:36:36 crc kubenswrapper[5052]: I1203 12:36:36.031041 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-2v6n2" Dec 03 12:36:36 crc kubenswrapper[5052]: I1203 12:36:36.108684 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/32d4c4ba-c6dd-49cf-80a3-4d7ed71cb6b5-config-data\") pod \"32d4c4ba-c6dd-49cf-80a3-4d7ed71cb6b5\" (UID: \"32d4c4ba-c6dd-49cf-80a3-4d7ed71cb6b5\") " Dec 03 12:36:36 crc kubenswrapper[5052]: I1203 12:36:36.108743 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/32d4c4ba-c6dd-49cf-80a3-4d7ed71cb6b5-combined-ca-bundle\") pod \"32d4c4ba-c6dd-49cf-80a3-4d7ed71cb6b5\" (UID: \"32d4c4ba-c6dd-49cf-80a3-4d7ed71cb6b5\") " Dec 03 12:36:36 crc kubenswrapper[5052]: I1203 12:36:36.108909 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bzzvw\" (UniqueName: \"kubernetes.io/projected/32d4c4ba-c6dd-49cf-80a3-4d7ed71cb6b5-kube-api-access-bzzvw\") pod \"32d4c4ba-c6dd-49cf-80a3-4d7ed71cb6b5\" (UID: \"32d4c4ba-c6dd-49cf-80a3-4d7ed71cb6b5\") " Dec 03 12:36:36 crc kubenswrapper[5052]: I1203 12:36:36.108940 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/32d4c4ba-c6dd-49cf-80a3-4d7ed71cb6b5-scripts\") pod \"32d4c4ba-c6dd-49cf-80a3-4d7ed71cb6b5\" (UID: \"32d4c4ba-c6dd-49cf-80a3-4d7ed71cb6b5\") " Dec 03 12:36:36 crc kubenswrapper[5052]: I1203 12:36:36.112814 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/32d4c4ba-c6dd-49cf-80a3-4d7ed71cb6b5-scripts" (OuterVolumeSpecName: "scripts") pod "32d4c4ba-c6dd-49cf-80a3-4d7ed71cb6b5" (UID: "32d4c4ba-c6dd-49cf-80a3-4d7ed71cb6b5"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:36:36 crc kubenswrapper[5052]: I1203 12:36:36.112824 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/32d4c4ba-c6dd-49cf-80a3-4d7ed71cb6b5-kube-api-access-bzzvw" (OuterVolumeSpecName: "kube-api-access-bzzvw") pod "32d4c4ba-c6dd-49cf-80a3-4d7ed71cb6b5" (UID: "32d4c4ba-c6dd-49cf-80a3-4d7ed71cb6b5"). InnerVolumeSpecName "kube-api-access-bzzvw". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:36:36 crc kubenswrapper[5052]: E1203 12:36:36.140802 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/32d4c4ba-c6dd-49cf-80a3-4d7ed71cb6b5-combined-ca-bundle podName:32d4c4ba-c6dd-49cf-80a3-4d7ed71cb6b5 nodeName:}" failed. No retries permitted until 2025-12-03 12:36:36.640275955 +0000 UTC m=+4587.228517739 (durationBeforeRetry 500ms). Error: error cleaning subPath mounts for volume "combined-ca-bundle" (UniqueName: "kubernetes.io/secret/32d4c4ba-c6dd-49cf-80a3-4d7ed71cb6b5-combined-ca-bundle") pod "32d4c4ba-c6dd-49cf-80a3-4d7ed71cb6b5" (UID: "32d4c4ba-c6dd-49cf-80a3-4d7ed71cb6b5") : error deleting /var/lib/kubelet/pods/32d4c4ba-c6dd-49cf-80a3-4d7ed71cb6b5/volume-subpaths: remove /var/lib/kubelet/pods/32d4c4ba-c6dd-49cf-80a3-4d7ed71cb6b5/volume-subpaths: no such file or directory Dec 03 12:36:36 crc kubenswrapper[5052]: I1203 12:36:36.145460 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/32d4c4ba-c6dd-49cf-80a3-4d7ed71cb6b5-config-data" (OuterVolumeSpecName: "config-data") pod "32d4c4ba-c6dd-49cf-80a3-4d7ed71cb6b5" (UID: "32d4c4ba-c6dd-49cf-80a3-4d7ed71cb6b5"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:36:36 crc kubenswrapper[5052]: I1203 12:36:36.211623 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-bzzvw\" (UniqueName: \"kubernetes.io/projected/32d4c4ba-c6dd-49cf-80a3-4d7ed71cb6b5-kube-api-access-bzzvw\") on node \"crc\" DevicePath \"\"" Dec 03 12:36:36 crc kubenswrapper[5052]: I1203 12:36:36.211674 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/32d4c4ba-c6dd-49cf-80a3-4d7ed71cb6b5-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:36:36 crc kubenswrapper[5052]: I1203 12:36:36.211685 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/32d4c4ba-c6dd-49cf-80a3-4d7ed71cb6b5-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:36:36 crc kubenswrapper[5052]: I1203 12:36:36.260003 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:36:36 crc kubenswrapper[5052]: I1203 12:36:36.723679 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/32d4c4ba-c6dd-49cf-80a3-4d7ed71cb6b5-combined-ca-bundle\") pod \"32d4c4ba-c6dd-49cf-80a3-4d7ed71cb6b5\" (UID: \"32d4c4ba-c6dd-49cf-80a3-4d7ed71cb6b5\") " Dec 03 12:36:36 crc kubenswrapper[5052]: I1203 12:36:36.728547 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/32d4c4ba-c6dd-49cf-80a3-4d7ed71cb6b5-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "32d4c4ba-c6dd-49cf-80a3-4d7ed71cb6b5" (UID: "32d4c4ba-c6dd-49cf-80a3-4d7ed71cb6b5"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:36:36 crc kubenswrapper[5052]: I1203 12:36:36.744149 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"1316edb7-1894-427d-92bf-dab018e056fd","Type":"ContainerStarted","Data":"40c2a8b3592d4fb942b76bd47d5c522c0f6eac013387e752b7ac5f8e344a5983"} Dec 03 12:36:36 crc kubenswrapper[5052]: I1203 12:36:36.744204 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"1316edb7-1894-427d-92bf-dab018e056fd","Type":"ContainerStarted","Data":"032ba52f8e13c68cd39376b0e7f50f353a40ac4cdb56ab0ff1ba8f9f7494f68f"} Dec 03 12:36:36 crc kubenswrapper[5052]: I1203 12:36:36.748246 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-2v6n2" event={"ID":"32d4c4ba-c6dd-49cf-80a3-4d7ed71cb6b5","Type":"ContainerDied","Data":"b05f948f48338fd7c1f6cd1e438d6262d9d99ba06e412f5140cf8b8bf463c8cf"} Dec 03 12:36:36 crc kubenswrapper[5052]: I1203 12:36:36.748298 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b05f948f48338fd7c1f6cd1e438d6262d9d99ba06e412f5140cf8b8bf463c8cf" Dec 03 12:36:36 crc kubenswrapper[5052]: I1203 12:36:36.748322 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-2v6n2" Dec 03 12:36:36 crc kubenswrapper[5052]: I1203 12:36:36.757158 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"69502269-3194-440d-aa90-8a9a1821a297","Type":"ContainerStarted","Data":"6340ef46e764a405f61fe84125c42eb8a5422a7ba57320782ecc1f6addbfbe0c"} Dec 03 12:36:36 crc kubenswrapper[5052]: I1203 12:36:36.757208 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"69502269-3194-440d-aa90-8a9a1821a297","Type":"ContainerStarted","Data":"8158be457c8bd67dfca9154299916c22a6ad78859653cc60a7a7925603aaf518"} Dec 03 12:36:36 crc kubenswrapper[5052]: I1203 12:36:36.757221 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"69502269-3194-440d-aa90-8a9a1821a297","Type":"ContainerStarted","Data":"6a2d13b006cb0eb1850f1ce57e0b8be7759b8e5ca66aca71f042c5738a6afa97"} Dec 03 12:36:36 crc kubenswrapper[5052]: I1203 12:36:36.765668 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-scheduler-0" podStartSLOduration=1.7656508400000002 podStartE2EDuration="1.76565084s" podCreationTimestamp="2025-12-03 12:36:35 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:36:36.765029592 +0000 UTC m=+4587.353271366" watchObservedRunningTime="2025-12-03 12:36:36.76565084 +0000 UTC m=+4587.353892594" Dec 03 12:36:36 crc kubenswrapper[5052]: I1203 12:36:36.793990 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-0"] Dec 03 12:36:36 crc kubenswrapper[5052]: E1203 12:36:36.794771 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="32d4c4ba-c6dd-49cf-80a3-4d7ed71cb6b5" containerName="nova-cell1-conductor-db-sync" Dec 03 12:36:36 crc kubenswrapper[5052]: I1203 12:36:36.794872 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="32d4c4ba-c6dd-49cf-80a3-4d7ed71cb6b5" containerName="nova-cell1-conductor-db-sync" Dec 03 12:36:36 crc kubenswrapper[5052]: I1203 12:36:36.795217 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="32d4c4ba-c6dd-49cf-80a3-4d7ed71cb6b5" containerName="nova-cell1-conductor-db-sync" Dec 03 12:36:36 crc kubenswrapper[5052]: I1203 12:36:36.796959 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:36:36 crc kubenswrapper[5052]: I1203 12:36:36.799280 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell1-conductor-config-data" Dec 03 12:36:36 crc kubenswrapper[5052]: I1203 12:36:36.810696 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-0"] Dec 03 12:36:36 crc kubenswrapper[5052]: I1203 12:36:36.814159 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-metadata-0" podStartSLOduration=2.814143357 podStartE2EDuration="2.814143357s" podCreationTimestamp="2025-12-03 12:36:34 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:36:36.792052005 +0000 UTC m=+4587.380293769" watchObservedRunningTime="2025-12-03 12:36:36.814143357 +0000 UTC m=+4587.402385111" Dec 03 12:36:36 crc kubenswrapper[5052]: I1203 12:36:36.825974 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/32d4c4ba-c6dd-49cf-80a3-4d7ed71cb6b5-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:36:36 crc kubenswrapper[5052]: I1203 12:36:36.927723 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/0ffa4505-bb77-4d31-8e7d-7926eae343fd-config-data\") pod \"nova-cell1-conductor-0\" (UID: \"0ffa4505-bb77-4d31-8e7d-7926eae343fd\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:36:36 crc kubenswrapper[5052]: I1203 12:36:36.927900 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pmkx7\" (UniqueName: \"kubernetes.io/projected/0ffa4505-bb77-4d31-8e7d-7926eae343fd-kube-api-access-pmkx7\") pod \"nova-cell1-conductor-0\" (UID: \"0ffa4505-bb77-4d31-8e7d-7926eae343fd\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:36:36 crc kubenswrapper[5052]: I1203 12:36:36.927946 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0ffa4505-bb77-4d31-8e7d-7926eae343fd-combined-ca-bundle\") pod \"nova-cell1-conductor-0\" (UID: \"0ffa4505-bb77-4d31-8e7d-7926eae343fd\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:36:36 crc kubenswrapper[5052]: I1203 12:36:36.940666 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:36:36 crc kubenswrapper[5052]: I1203 12:36:36.962267 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:36:37 crc kubenswrapper[5052]: I1203 12:36:37.030570 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-pmkx7\" (UniqueName: \"kubernetes.io/projected/0ffa4505-bb77-4d31-8e7d-7926eae343fd-kube-api-access-pmkx7\") pod \"nova-cell1-conductor-0\" (UID: \"0ffa4505-bb77-4d31-8e7d-7926eae343fd\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:36:37 crc kubenswrapper[5052]: I1203 12:36:37.030623 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0ffa4505-bb77-4d31-8e7d-7926eae343fd-combined-ca-bundle\") pod \"nova-cell1-conductor-0\" (UID: \"0ffa4505-bb77-4d31-8e7d-7926eae343fd\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:36:37 crc kubenswrapper[5052]: I1203 12:36:37.030706 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/0ffa4505-bb77-4d31-8e7d-7926eae343fd-config-data\") pod \"nova-cell1-conductor-0\" (UID: \"0ffa4505-bb77-4d31-8e7d-7926eae343fd\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:36:37 crc kubenswrapper[5052]: I1203 12:36:37.036000 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0ffa4505-bb77-4d31-8e7d-7926eae343fd-combined-ca-bundle\") pod \"nova-cell1-conductor-0\" (UID: \"0ffa4505-bb77-4d31-8e7d-7926eae343fd\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:36:37 crc kubenswrapper[5052]: I1203 12:36:37.037164 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/0ffa4505-bb77-4d31-8e7d-7926eae343fd-config-data\") pod \"nova-cell1-conductor-0\" (UID: \"0ffa4505-bb77-4d31-8e7d-7926eae343fd\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:36:37 crc kubenswrapper[5052]: I1203 12:36:37.050788 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-pmkx7\" (UniqueName: \"kubernetes.io/projected/0ffa4505-bb77-4d31-8e7d-7926eae343fd-kube-api-access-pmkx7\") pod \"nova-cell1-conductor-0\" (UID: \"0ffa4505-bb77-4d31-8e7d-7926eae343fd\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:36:37 crc kubenswrapper[5052]: I1203 12:36:37.116437 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:36:37 crc kubenswrapper[5052]: I1203 12:36:37.578084 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-0"] Dec 03 12:36:37 crc kubenswrapper[5052]: I1203 12:36:37.669062 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:36:37 crc kubenswrapper[5052]: I1203 12:36:37.742841 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ft9k8\" (UniqueName: \"kubernetes.io/projected/7b6bb151-d113-40e6-97a3-644ca7e4e5f3-kube-api-access-ft9k8\") pod \"7b6bb151-d113-40e6-97a3-644ca7e4e5f3\" (UID: \"7b6bb151-d113-40e6-97a3-644ca7e4e5f3\") " Dec 03 12:36:37 crc kubenswrapper[5052]: I1203 12:36:37.743007 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/7b6bb151-d113-40e6-97a3-644ca7e4e5f3-logs\") pod \"7b6bb151-d113-40e6-97a3-644ca7e4e5f3\" (UID: \"7b6bb151-d113-40e6-97a3-644ca7e4e5f3\") " Dec 03 12:36:37 crc kubenswrapper[5052]: I1203 12:36:37.743053 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/7b6bb151-d113-40e6-97a3-644ca7e4e5f3-combined-ca-bundle\") pod \"7b6bb151-d113-40e6-97a3-644ca7e4e5f3\" (UID: \"7b6bb151-d113-40e6-97a3-644ca7e4e5f3\") " Dec 03 12:36:37 crc kubenswrapper[5052]: I1203 12:36:37.743097 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/7b6bb151-d113-40e6-97a3-644ca7e4e5f3-config-data\") pod \"7b6bb151-d113-40e6-97a3-644ca7e4e5f3\" (UID: \"7b6bb151-d113-40e6-97a3-644ca7e4e5f3\") " Dec 03 12:36:37 crc kubenswrapper[5052]: I1203 12:36:37.745507 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7b6bb151-d113-40e6-97a3-644ca7e4e5f3-logs" (OuterVolumeSpecName: "logs") pod "7b6bb151-d113-40e6-97a3-644ca7e4e5f3" (UID: "7b6bb151-d113-40e6-97a3-644ca7e4e5f3"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:36:37 crc kubenswrapper[5052]: I1203 12:36:37.757882 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7b6bb151-d113-40e6-97a3-644ca7e4e5f3-kube-api-access-ft9k8" (OuterVolumeSpecName: "kube-api-access-ft9k8") pod "7b6bb151-d113-40e6-97a3-644ca7e4e5f3" (UID: "7b6bb151-d113-40e6-97a3-644ca7e4e5f3"). InnerVolumeSpecName "kube-api-access-ft9k8". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:36:37 crc kubenswrapper[5052]: I1203 12:36:37.773393 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-conductor-0" event={"ID":"0ffa4505-bb77-4d31-8e7d-7926eae343fd","Type":"ContainerStarted","Data":"0f8b4c9ec56b842fda10122fea469713824491466b5906c88f5d6467555c4737"} Dec 03 12:36:37 crc kubenswrapper[5052]: I1203 12:36:37.775293 5052 generic.go:334] "Generic (PLEG): container finished" podID="7b6bb151-d113-40e6-97a3-644ca7e4e5f3" containerID="0b2f979945fddbeb900f47fb601b5b1707af68be110b9c7e7a0ec564f0e78e54" exitCode=0 Dec 03 12:36:37 crc kubenswrapper[5052]: I1203 12:36:37.775409 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:36:37 crc kubenswrapper[5052]: I1203 12:36:37.775489 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"7b6bb151-d113-40e6-97a3-644ca7e4e5f3","Type":"ContainerDied","Data":"0b2f979945fddbeb900f47fb601b5b1707af68be110b9c7e7a0ec564f0e78e54"} Dec 03 12:36:37 crc kubenswrapper[5052]: I1203 12:36:37.775553 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"7b6bb151-d113-40e6-97a3-644ca7e4e5f3","Type":"ContainerDied","Data":"d541dc9f23ceb98afa8eee4e8260db9f66d999f810603fa6a0b01f017ee0803b"} Dec 03 12:36:37 crc kubenswrapper[5052]: I1203 12:36:37.775584 5052 scope.go:117] "RemoveContainer" containerID="0b2f979945fddbeb900f47fb601b5b1707af68be110b9c7e7a0ec564f0e78e54" Dec 03 12:36:37 crc kubenswrapper[5052]: I1203 12:36:37.791081 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7b6bb151-d113-40e6-97a3-644ca7e4e5f3-config-data" (OuterVolumeSpecName: "config-data") pod "7b6bb151-d113-40e6-97a3-644ca7e4e5f3" (UID: "7b6bb151-d113-40e6-97a3-644ca7e4e5f3"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:36:37 crc kubenswrapper[5052]: I1203 12:36:37.794742 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:36:37 crc kubenswrapper[5052]: I1203 12:36:37.803952 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7b6bb151-d113-40e6-97a3-644ca7e4e5f3-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "7b6bb151-d113-40e6-97a3-644ca7e4e5f3" (UID: "7b6bb151-d113-40e6-97a3-644ca7e4e5f3"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:36:37 crc kubenswrapper[5052]: I1203 12:36:37.816929 5052 scope.go:117] "RemoveContainer" containerID="62321816c49aa66fe160c689b7c7f7a1345ff820851a3c51ec682204697cbbe5" Dec 03 12:36:37 crc kubenswrapper[5052]: I1203 12:36:37.846658 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/7b6bb151-d113-40e6-97a3-644ca7e4e5f3-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:36:37 crc kubenswrapper[5052]: I1203 12:36:37.847239 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/7b6bb151-d113-40e6-97a3-644ca7e4e5f3-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:36:37 crc kubenswrapper[5052]: I1203 12:36:37.847331 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/7b6bb151-d113-40e6-97a3-644ca7e4e5f3-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:36:37 crc kubenswrapper[5052]: I1203 12:36:37.847413 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-ft9k8\" (UniqueName: \"kubernetes.io/projected/7b6bb151-d113-40e6-97a3-644ca7e4e5f3-kube-api-access-ft9k8\") on node \"crc\" DevicePath \"\"" Dec 03 12:36:37 crc kubenswrapper[5052]: I1203 12:36:37.871174 5052 scope.go:117] "RemoveContainer" containerID="0b2f979945fddbeb900f47fb601b5b1707af68be110b9c7e7a0ec564f0e78e54" Dec 03 12:36:37 crc kubenswrapper[5052]: E1203 12:36:37.873190 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0b2f979945fddbeb900f47fb601b5b1707af68be110b9c7e7a0ec564f0e78e54\": container with ID starting with 0b2f979945fddbeb900f47fb601b5b1707af68be110b9c7e7a0ec564f0e78e54 not found: ID does not exist" containerID="0b2f979945fddbeb900f47fb601b5b1707af68be110b9c7e7a0ec564f0e78e54" Dec 03 12:36:37 crc kubenswrapper[5052]: I1203 12:36:37.873263 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0b2f979945fddbeb900f47fb601b5b1707af68be110b9c7e7a0ec564f0e78e54"} err="failed to get container status \"0b2f979945fddbeb900f47fb601b5b1707af68be110b9c7e7a0ec564f0e78e54\": rpc error: code = NotFound desc = could not find container \"0b2f979945fddbeb900f47fb601b5b1707af68be110b9c7e7a0ec564f0e78e54\": container with ID starting with 0b2f979945fddbeb900f47fb601b5b1707af68be110b9c7e7a0ec564f0e78e54 not found: ID does not exist" Dec 03 12:36:37 crc kubenswrapper[5052]: I1203 12:36:37.873312 5052 scope.go:117] "RemoveContainer" containerID="62321816c49aa66fe160c689b7c7f7a1345ff820851a3c51ec682204697cbbe5" Dec 03 12:36:37 crc kubenswrapper[5052]: E1203 12:36:37.874839 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"62321816c49aa66fe160c689b7c7f7a1345ff820851a3c51ec682204697cbbe5\": container with ID starting with 62321816c49aa66fe160c689b7c7f7a1345ff820851a3c51ec682204697cbbe5 not found: ID does not exist" containerID="62321816c49aa66fe160c689b7c7f7a1345ff820851a3c51ec682204697cbbe5" Dec 03 12:36:37 crc kubenswrapper[5052]: I1203 12:36:37.874864 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"62321816c49aa66fe160c689b7c7f7a1345ff820851a3c51ec682204697cbbe5"} err="failed to get container status \"62321816c49aa66fe160c689b7c7f7a1345ff820851a3c51ec682204697cbbe5\": rpc error: code = NotFound desc = could not find container \"62321816c49aa66fe160c689b7c7f7a1345ff820851a3c51ec682204697cbbe5\": container with ID starting with 62321816c49aa66fe160c689b7c7f7a1345ff820851a3c51ec682204697cbbe5 not found: ID does not exist" Dec 03 12:36:38 crc kubenswrapper[5052]: I1203 12:36:38.128370 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:36:38 crc kubenswrapper[5052]: I1203 12:36:38.139170 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:36:38 crc kubenswrapper[5052]: I1203 12:36:38.150697 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:36:38 crc kubenswrapper[5052]: E1203 12:36:38.151294 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="7b6bb151-d113-40e6-97a3-644ca7e4e5f3" containerName="nova-api-log" Dec 03 12:36:38 crc kubenswrapper[5052]: I1203 12:36:38.151318 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="7b6bb151-d113-40e6-97a3-644ca7e4e5f3" containerName="nova-api-log" Dec 03 12:36:38 crc kubenswrapper[5052]: E1203 12:36:38.151341 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="7b6bb151-d113-40e6-97a3-644ca7e4e5f3" containerName="nova-api-api" Dec 03 12:36:38 crc kubenswrapper[5052]: I1203 12:36:38.151349 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="7b6bb151-d113-40e6-97a3-644ca7e4e5f3" containerName="nova-api-api" Dec 03 12:36:38 crc kubenswrapper[5052]: I1203 12:36:38.151605 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="7b6bb151-d113-40e6-97a3-644ca7e4e5f3" containerName="nova-api-log" Dec 03 12:36:38 crc kubenswrapper[5052]: I1203 12:36:38.151637 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="7b6bb151-d113-40e6-97a3-644ca7e4e5f3" containerName="nova-api-api" Dec 03 12:36:38 crc kubenswrapper[5052]: I1203 12:36:38.153096 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:36:38 crc kubenswrapper[5052]: I1203 12:36:38.158586 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-api-config-data" Dec 03 12:36:38 crc kubenswrapper[5052]: I1203 12:36:38.177458 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:36:38 crc kubenswrapper[5052]: I1203 12:36:38.254615 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/7e7d9a94-b50b-4065-a178-557deda1100c-config-data\") pod \"nova-api-0\" (UID: \"7e7d9a94-b50b-4065-a178-557deda1100c\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:36:38 crc kubenswrapper[5052]: I1203 12:36:38.254733 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hmgbn\" (UniqueName: \"kubernetes.io/projected/7e7d9a94-b50b-4065-a178-557deda1100c-kube-api-access-hmgbn\") pod \"nova-api-0\" (UID: \"7e7d9a94-b50b-4065-a178-557deda1100c\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:36:38 crc kubenswrapper[5052]: I1203 12:36:38.254773 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/7e7d9a94-b50b-4065-a178-557deda1100c-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"7e7d9a94-b50b-4065-a178-557deda1100c\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:36:38 crc kubenswrapper[5052]: I1203 12:36:38.254834 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/7e7d9a94-b50b-4065-a178-557deda1100c-logs\") pod \"nova-api-0\" (UID: \"7e7d9a94-b50b-4065-a178-557deda1100c\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:36:38 crc kubenswrapper[5052]: I1203 12:36:38.357215 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-hmgbn\" (UniqueName: \"kubernetes.io/projected/7e7d9a94-b50b-4065-a178-557deda1100c-kube-api-access-hmgbn\") pod \"nova-api-0\" (UID: \"7e7d9a94-b50b-4065-a178-557deda1100c\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:36:38 crc kubenswrapper[5052]: I1203 12:36:38.357754 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/7e7d9a94-b50b-4065-a178-557deda1100c-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"7e7d9a94-b50b-4065-a178-557deda1100c\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:36:38 crc kubenswrapper[5052]: I1203 12:36:38.358071 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/7e7d9a94-b50b-4065-a178-557deda1100c-logs\") pod \"nova-api-0\" (UID: \"7e7d9a94-b50b-4065-a178-557deda1100c\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:36:38 crc kubenswrapper[5052]: I1203 12:36:38.358362 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/7e7d9a94-b50b-4065-a178-557deda1100c-config-data\") pod \"nova-api-0\" (UID: \"7e7d9a94-b50b-4065-a178-557deda1100c\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:36:38 crc kubenswrapper[5052]: I1203 12:36:38.358834 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/7e7d9a94-b50b-4065-a178-557deda1100c-logs\") pod \"nova-api-0\" (UID: \"7e7d9a94-b50b-4065-a178-557deda1100c\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:36:38 crc kubenswrapper[5052]: I1203 12:36:38.364110 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/7e7d9a94-b50b-4065-a178-557deda1100c-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"7e7d9a94-b50b-4065-a178-557deda1100c\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:36:38 crc kubenswrapper[5052]: I1203 12:36:38.364422 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/7e7d9a94-b50b-4065-a178-557deda1100c-config-data\") pod \"nova-api-0\" (UID: \"7e7d9a94-b50b-4065-a178-557deda1100c\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:36:38 crc kubenswrapper[5052]: I1203 12:36:38.405569 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-hmgbn\" (UniqueName: \"kubernetes.io/projected/7e7d9a94-b50b-4065-a178-557deda1100c-kube-api-access-hmgbn\") pod \"nova-api-0\" (UID: \"7e7d9a94-b50b-4065-a178-557deda1100c\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:36:38 crc kubenswrapper[5052]: I1203 12:36:38.472902 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:36:38 crc kubenswrapper[5052]: I1203 12:36:38.793095 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-conductor-0" event={"ID":"0ffa4505-bb77-4d31-8e7d-7926eae343fd","Type":"ContainerStarted","Data":"faa257c722aaf243fc59be292d2c265e25f1a8976cfba25553abe4b2ea8c30f6"} Dec 03 12:36:38 crc kubenswrapper[5052]: I1203 12:36:38.793665 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:36:38 crc kubenswrapper[5052]: I1203 12:36:38.968018 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-cell1-conductor-0" podStartSLOduration=2.967990263 podStartE2EDuration="2.967990263s" podCreationTimestamp="2025-12-03 12:36:36 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:36:38.837005687 +0000 UTC m=+4589.425247451" watchObservedRunningTime="2025-12-03 12:36:38.967990263 +0000 UTC m=+4589.556232027" Dec 03 12:36:38 crc kubenswrapper[5052]: I1203 12:36:38.979659 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:36:39 crc kubenswrapper[5052]: I1203 12:36:39.698719 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7b6bb151-d113-40e6-97a3-644ca7e4e5f3" path="/var/lib/kubelet/pods/7b6bb151-d113-40e6-97a3-644ca7e4e5f3/volumes" Dec 03 12:36:39 crc kubenswrapper[5052]: I1203 12:36:39.803685 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"7e7d9a94-b50b-4065-a178-557deda1100c","Type":"ContainerStarted","Data":"2de47cacb469eb9f4ef023ca0e0c9a3f1c57692a0ef8c8f65261831c30d72149"} Dec 03 12:36:39 crc kubenswrapper[5052]: I1203 12:36:39.804782 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"7e7d9a94-b50b-4065-a178-557deda1100c","Type":"ContainerStarted","Data":"e151b80f440784eeb61f00e0a399f2ccb9b03fb4ee3fee6bac9ffb707dde5f36"} Dec 03 12:36:39 crc kubenswrapper[5052]: I1203 12:36:39.804892 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"7e7d9a94-b50b-4065-a178-557deda1100c","Type":"ContainerStarted","Data":"4fcec97c6d61d6ecc7ab69b60c08d2f8ddf98269e8c21e82ffe9d6ebb860e8da"} Dec 03 12:36:39 crc kubenswrapper[5052]: I1203 12:36:39.841777 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-api-0" podStartSLOduration=1.841759011 podStartE2EDuration="1.841759011s" podCreationTimestamp="2025-12-03 12:36:38 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:36:39.837664654 +0000 UTC m=+4590.425906428" watchObservedRunningTime="2025-12-03 12:36:39.841759011 +0000 UTC m=+4590.430000775" Dec 03 12:36:40 crc kubenswrapper[5052]: I1203 12:36:40.144249 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:36:40 crc kubenswrapper[5052]: I1203 12:36:40.144301 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:36:40 crc kubenswrapper[5052]: I1203 12:36:40.674009 5052 scope.go:117] "RemoveContainer" containerID="e6edaf188f85ea4f11f487f11cc489c700f25fb42b5ee269f4f496f1e127d772" Dec 03 12:36:40 crc kubenswrapper[5052]: E1203 12:36:40.674265 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 12:36:40 crc kubenswrapper[5052]: I1203 12:36:40.768432 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:36:43 crc kubenswrapper[5052]: I1203 12:36:43.669889 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:36:45 crc kubenswrapper[5052]: I1203 12:36:45.144153 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:36:45 crc kubenswrapper[5052]: I1203 12:36:45.144382 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:36:45 crc kubenswrapper[5052]: I1203 12:36:45.767897 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:36:45 crc kubenswrapper[5052]: I1203 12:36:45.793643 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:36:45 crc kubenswrapper[5052]: I1203 12:36:45.884578 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:36:46 crc kubenswrapper[5052]: I1203 12:36:46.155717 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-metadata-0" podUID="69502269-3194-440d-aa90-8a9a1821a297" containerName="nova-metadata-log" probeResult="failure" output="Get \"https://10.217.1.80:8775/\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Dec 03 12:36:46 crc kubenswrapper[5052]: I1203 12:36:46.155755 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-metadata-0" podUID="69502269-3194-440d-aa90-8a9a1821a297" containerName="nova-metadata-metadata" probeResult="failure" output="Get \"https://10.217.1.80:8775/\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Dec 03 12:36:47 crc kubenswrapper[5052]: I1203 12:36:47.150732 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:36:47 crc kubenswrapper[5052]: I1203 12:36:47.790089 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell1-cell-mapping-fj7tx"] Dec 03 12:36:47 crc kubenswrapper[5052]: I1203 12:36:47.795053 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-cell-mapping-fj7tx" Dec 03 12:36:47 crc kubenswrapper[5052]: I1203 12:36:47.798012 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell1-manage-scripts" Dec 03 12:36:47 crc kubenswrapper[5052]: I1203 12:36:47.798064 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell1-manage-config-data" Dec 03 12:36:47 crc kubenswrapper[5052]: I1203 12:36:47.802794 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-cell-mapping-fj7tx"] Dec 03 12:36:47 crc kubenswrapper[5052]: I1203 12:36:47.859157 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/47136a96-c6f5-428d-a11a-fa9eeb4910e7-combined-ca-bundle\") pod \"nova-cell1-cell-mapping-fj7tx\" (UID: \"47136a96-c6f5-428d-a11a-fa9eeb4910e7\") " pod="openstack-kuttl-tests/nova-cell1-cell-mapping-fj7tx" Dec 03 12:36:47 crc kubenswrapper[5052]: I1203 12:36:47.859263 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2rsnc\" (UniqueName: \"kubernetes.io/projected/47136a96-c6f5-428d-a11a-fa9eeb4910e7-kube-api-access-2rsnc\") pod \"nova-cell1-cell-mapping-fj7tx\" (UID: \"47136a96-c6f5-428d-a11a-fa9eeb4910e7\") " pod="openstack-kuttl-tests/nova-cell1-cell-mapping-fj7tx" Dec 03 12:36:47 crc kubenswrapper[5052]: I1203 12:36:47.859342 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/47136a96-c6f5-428d-a11a-fa9eeb4910e7-config-data\") pod \"nova-cell1-cell-mapping-fj7tx\" (UID: \"47136a96-c6f5-428d-a11a-fa9eeb4910e7\") " pod="openstack-kuttl-tests/nova-cell1-cell-mapping-fj7tx" Dec 03 12:36:47 crc kubenswrapper[5052]: I1203 12:36:47.859364 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/47136a96-c6f5-428d-a11a-fa9eeb4910e7-scripts\") pod \"nova-cell1-cell-mapping-fj7tx\" (UID: \"47136a96-c6f5-428d-a11a-fa9eeb4910e7\") " pod="openstack-kuttl-tests/nova-cell1-cell-mapping-fj7tx" Dec 03 12:36:47 crc kubenswrapper[5052]: I1203 12:36:47.962219 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/47136a96-c6f5-428d-a11a-fa9eeb4910e7-combined-ca-bundle\") pod \"nova-cell1-cell-mapping-fj7tx\" (UID: \"47136a96-c6f5-428d-a11a-fa9eeb4910e7\") " pod="openstack-kuttl-tests/nova-cell1-cell-mapping-fj7tx" Dec 03 12:36:47 crc kubenswrapper[5052]: I1203 12:36:47.962383 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-2rsnc\" (UniqueName: \"kubernetes.io/projected/47136a96-c6f5-428d-a11a-fa9eeb4910e7-kube-api-access-2rsnc\") pod \"nova-cell1-cell-mapping-fj7tx\" (UID: \"47136a96-c6f5-428d-a11a-fa9eeb4910e7\") " pod="openstack-kuttl-tests/nova-cell1-cell-mapping-fj7tx" Dec 03 12:36:47 crc kubenswrapper[5052]: I1203 12:36:47.962491 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/47136a96-c6f5-428d-a11a-fa9eeb4910e7-config-data\") pod \"nova-cell1-cell-mapping-fj7tx\" (UID: \"47136a96-c6f5-428d-a11a-fa9eeb4910e7\") " pod="openstack-kuttl-tests/nova-cell1-cell-mapping-fj7tx" Dec 03 12:36:47 crc kubenswrapper[5052]: I1203 12:36:47.962532 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/47136a96-c6f5-428d-a11a-fa9eeb4910e7-scripts\") pod \"nova-cell1-cell-mapping-fj7tx\" (UID: \"47136a96-c6f5-428d-a11a-fa9eeb4910e7\") " pod="openstack-kuttl-tests/nova-cell1-cell-mapping-fj7tx" Dec 03 12:36:47 crc kubenswrapper[5052]: I1203 12:36:47.970153 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/47136a96-c6f5-428d-a11a-fa9eeb4910e7-config-data\") pod \"nova-cell1-cell-mapping-fj7tx\" (UID: \"47136a96-c6f5-428d-a11a-fa9eeb4910e7\") " pod="openstack-kuttl-tests/nova-cell1-cell-mapping-fj7tx" Dec 03 12:36:47 crc kubenswrapper[5052]: I1203 12:36:47.973350 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/47136a96-c6f5-428d-a11a-fa9eeb4910e7-combined-ca-bundle\") pod \"nova-cell1-cell-mapping-fj7tx\" (UID: \"47136a96-c6f5-428d-a11a-fa9eeb4910e7\") " pod="openstack-kuttl-tests/nova-cell1-cell-mapping-fj7tx" Dec 03 12:36:47 crc kubenswrapper[5052]: I1203 12:36:47.974538 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/47136a96-c6f5-428d-a11a-fa9eeb4910e7-scripts\") pod \"nova-cell1-cell-mapping-fj7tx\" (UID: \"47136a96-c6f5-428d-a11a-fa9eeb4910e7\") " pod="openstack-kuttl-tests/nova-cell1-cell-mapping-fj7tx" Dec 03 12:36:47 crc kubenswrapper[5052]: I1203 12:36:47.982607 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-2rsnc\" (UniqueName: \"kubernetes.io/projected/47136a96-c6f5-428d-a11a-fa9eeb4910e7-kube-api-access-2rsnc\") pod \"nova-cell1-cell-mapping-fj7tx\" (UID: \"47136a96-c6f5-428d-a11a-fa9eeb4910e7\") " pod="openstack-kuttl-tests/nova-cell1-cell-mapping-fj7tx" Dec 03 12:36:48 crc kubenswrapper[5052]: I1203 12:36:48.012015 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/kube-state-metrics-0"] Dec 03 12:36:48 crc kubenswrapper[5052]: I1203 12:36:48.012226 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/kube-state-metrics-0" podUID="5c9ee44f-4958-4cc9-a8a4-e5c023b65baf" containerName="kube-state-metrics" containerID="cri-o://079d43fdf2f0ff185ac5f2d5c966c26353a478c914bb29cf74d63cf1f61876bd" gracePeriod=30 Dec 03 12:36:48 crc kubenswrapper[5052]: I1203 12:36:48.156968 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-cell-mapping-fj7tx" Dec 03 12:36:48 crc kubenswrapper[5052]: I1203 12:36:48.473650 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:36:48 crc kubenswrapper[5052]: I1203 12:36:48.474894 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:36:48 crc kubenswrapper[5052]: I1203 12:36:48.543135 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:36:48 crc kubenswrapper[5052]: I1203 12:36:48.574735 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2vbwg\" (UniqueName: \"kubernetes.io/projected/5c9ee44f-4958-4cc9-a8a4-e5c023b65baf-kube-api-access-2vbwg\") pod \"5c9ee44f-4958-4cc9-a8a4-e5c023b65baf\" (UID: \"5c9ee44f-4958-4cc9-a8a4-e5c023b65baf\") " Dec 03 12:36:48 crc kubenswrapper[5052]: I1203 12:36:48.580251 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5c9ee44f-4958-4cc9-a8a4-e5c023b65baf-kube-api-access-2vbwg" (OuterVolumeSpecName: "kube-api-access-2vbwg") pod "5c9ee44f-4958-4cc9-a8a4-e5c023b65baf" (UID: "5c9ee44f-4958-4cc9-a8a4-e5c023b65baf"). InnerVolumeSpecName "kube-api-access-2vbwg". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:36:48 crc kubenswrapper[5052]: I1203 12:36:48.677775 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-2vbwg\" (UniqueName: \"kubernetes.io/projected/5c9ee44f-4958-4cc9-a8a4-e5c023b65baf-kube-api-access-2vbwg\") on node \"crc\" DevicePath \"\"" Dec 03 12:36:48 crc kubenswrapper[5052]: I1203 12:36:48.695289 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-cell-mapping-fj7tx"] Dec 03 12:36:48 crc kubenswrapper[5052]: I1203 12:36:48.885424 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-cell-mapping-fj7tx" event={"ID":"47136a96-c6f5-428d-a11a-fa9eeb4910e7","Type":"ContainerStarted","Data":"89fab5932f39924d83d4ab819b2183e190ed6a7efae6f78ee3791c1efe360659"} Dec 03 12:36:48 crc kubenswrapper[5052]: I1203 12:36:48.885760 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-cell-mapping-fj7tx" event={"ID":"47136a96-c6f5-428d-a11a-fa9eeb4910e7","Type":"ContainerStarted","Data":"225cf34a2ab533a9646e28b5830daa206e7eeee9515b766f4f51f4995cc2809f"} Dec 03 12:36:48 crc kubenswrapper[5052]: I1203 12:36:48.888445 5052 generic.go:334] "Generic (PLEG): container finished" podID="5c9ee44f-4958-4cc9-a8a4-e5c023b65baf" containerID="079d43fdf2f0ff185ac5f2d5c966c26353a478c914bb29cf74d63cf1f61876bd" exitCode=2 Dec 03 12:36:48 crc kubenswrapper[5052]: I1203 12:36:48.888784 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:36:48 crc kubenswrapper[5052]: I1203 12:36:48.889207 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/kube-state-metrics-0" event={"ID":"5c9ee44f-4958-4cc9-a8a4-e5c023b65baf","Type":"ContainerDied","Data":"079d43fdf2f0ff185ac5f2d5c966c26353a478c914bb29cf74d63cf1f61876bd"} Dec 03 12:36:48 crc kubenswrapper[5052]: I1203 12:36:48.889238 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/kube-state-metrics-0" event={"ID":"5c9ee44f-4958-4cc9-a8a4-e5c023b65baf","Type":"ContainerDied","Data":"309901a267add4c45800bd5475e7376f43e64ee259ae43f27b80c7fdea6e239c"} Dec 03 12:36:48 crc kubenswrapper[5052]: I1203 12:36:48.889255 5052 scope.go:117] "RemoveContainer" containerID="079d43fdf2f0ff185ac5f2d5c966c26353a478c914bb29cf74d63cf1f61876bd" Dec 03 12:36:48 crc kubenswrapper[5052]: I1203 12:36:48.930995 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-cell1-cell-mapping-fj7tx" podStartSLOduration=1.930972036 podStartE2EDuration="1.930972036s" podCreationTimestamp="2025-12-03 12:36:47 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:36:48.913316491 +0000 UTC m=+4599.501558255" watchObservedRunningTime="2025-12-03 12:36:48.930972036 +0000 UTC m=+4599.519213800" Dec 03 12:36:48 crc kubenswrapper[5052]: I1203 12:36:48.931132 5052 scope.go:117] "RemoveContainer" containerID="079d43fdf2f0ff185ac5f2d5c966c26353a478c914bb29cf74d63cf1f61876bd" Dec 03 12:36:48 crc kubenswrapper[5052]: E1203 12:36:48.933141 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"079d43fdf2f0ff185ac5f2d5c966c26353a478c914bb29cf74d63cf1f61876bd\": container with ID starting with 079d43fdf2f0ff185ac5f2d5c966c26353a478c914bb29cf74d63cf1f61876bd not found: ID does not exist" containerID="079d43fdf2f0ff185ac5f2d5c966c26353a478c914bb29cf74d63cf1f61876bd" Dec 03 12:36:48 crc kubenswrapper[5052]: I1203 12:36:48.933186 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"079d43fdf2f0ff185ac5f2d5c966c26353a478c914bb29cf74d63cf1f61876bd"} err="failed to get container status \"079d43fdf2f0ff185ac5f2d5c966c26353a478c914bb29cf74d63cf1f61876bd\": rpc error: code = NotFound desc = could not find container \"079d43fdf2f0ff185ac5f2d5c966c26353a478c914bb29cf74d63cf1f61876bd\": container with ID starting with 079d43fdf2f0ff185ac5f2d5c966c26353a478c914bb29cf74d63cf1f61876bd not found: ID does not exist" Dec 03 12:36:48 crc kubenswrapper[5052]: I1203 12:36:48.950905 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/kube-state-metrics-0"] Dec 03 12:36:48 crc kubenswrapper[5052]: I1203 12:36:48.991735 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/kube-state-metrics-0"] Dec 03 12:36:49 crc kubenswrapper[5052]: I1203 12:36:49.007956 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/kube-state-metrics-0"] Dec 03 12:36:49 crc kubenswrapper[5052]: E1203 12:36:49.008412 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="5c9ee44f-4958-4cc9-a8a4-e5c023b65baf" containerName="kube-state-metrics" Dec 03 12:36:49 crc kubenswrapper[5052]: I1203 12:36:49.008424 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c9ee44f-4958-4cc9-a8a4-e5c023b65baf" containerName="kube-state-metrics" Dec 03 12:36:49 crc kubenswrapper[5052]: I1203 12:36:49.008656 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="5c9ee44f-4958-4cc9-a8a4-e5c023b65baf" containerName="kube-state-metrics" Dec 03 12:36:49 crc kubenswrapper[5052]: I1203 12:36:49.010686 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:36:49 crc kubenswrapper[5052]: I1203 12:36:49.014879 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"kube-state-metrics-tls-config" Dec 03 12:36:49 crc kubenswrapper[5052]: I1203 12:36:49.014912 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-kube-state-metrics-svc" Dec 03 12:36:49 crc kubenswrapper[5052]: I1203 12:36:49.019563 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/kube-state-metrics-0"] Dec 03 12:36:49 crc kubenswrapper[5052]: I1203 12:36:49.184298 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gvsj7\" (UniqueName: \"kubernetes.io/projected/b7defe22-28b4-4fec-8fe5-ebde8e658c6b-kube-api-access-gvsj7\") pod \"kube-state-metrics-0\" (UID: \"b7defe22-28b4-4fec-8fe5-ebde8e658c6b\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:36:49 crc kubenswrapper[5052]: I1203 12:36:49.184410 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b7defe22-28b4-4fec-8fe5-ebde8e658c6b-combined-ca-bundle\") pod \"kube-state-metrics-0\" (UID: \"b7defe22-28b4-4fec-8fe5-ebde8e658c6b\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:36:49 crc kubenswrapper[5052]: I1203 12:36:49.184435 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls-certs\" (UniqueName: \"kubernetes.io/secret/b7defe22-28b4-4fec-8fe5-ebde8e658c6b-kube-state-metrics-tls-certs\") pod \"kube-state-metrics-0\" (UID: \"b7defe22-28b4-4fec-8fe5-ebde8e658c6b\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:36:49 crc kubenswrapper[5052]: I1203 12:36:49.184511 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls-config\" (UniqueName: \"kubernetes.io/secret/b7defe22-28b4-4fec-8fe5-ebde8e658c6b-kube-state-metrics-tls-config\") pod \"kube-state-metrics-0\" (UID: \"b7defe22-28b4-4fec-8fe5-ebde8e658c6b\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:36:49 crc kubenswrapper[5052]: I1203 12:36:49.286345 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls-config\" (UniqueName: \"kubernetes.io/secret/b7defe22-28b4-4fec-8fe5-ebde8e658c6b-kube-state-metrics-tls-config\") pod \"kube-state-metrics-0\" (UID: \"b7defe22-28b4-4fec-8fe5-ebde8e658c6b\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:36:49 crc kubenswrapper[5052]: I1203 12:36:49.286681 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-gvsj7\" (UniqueName: \"kubernetes.io/projected/b7defe22-28b4-4fec-8fe5-ebde8e658c6b-kube-api-access-gvsj7\") pod \"kube-state-metrics-0\" (UID: \"b7defe22-28b4-4fec-8fe5-ebde8e658c6b\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:36:49 crc kubenswrapper[5052]: I1203 12:36:49.286740 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b7defe22-28b4-4fec-8fe5-ebde8e658c6b-combined-ca-bundle\") pod \"kube-state-metrics-0\" (UID: \"b7defe22-28b4-4fec-8fe5-ebde8e658c6b\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:36:49 crc kubenswrapper[5052]: I1203 12:36:49.286758 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls-certs\" (UniqueName: \"kubernetes.io/secret/b7defe22-28b4-4fec-8fe5-ebde8e658c6b-kube-state-metrics-tls-certs\") pod \"kube-state-metrics-0\" (UID: \"b7defe22-28b4-4fec-8fe5-ebde8e658c6b\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:36:49 crc kubenswrapper[5052]: I1203 12:36:49.291370 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b7defe22-28b4-4fec-8fe5-ebde8e658c6b-combined-ca-bundle\") pod \"kube-state-metrics-0\" (UID: \"b7defe22-28b4-4fec-8fe5-ebde8e658c6b\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:36:49 crc kubenswrapper[5052]: I1203 12:36:49.292049 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls-certs\" (UniqueName: \"kubernetes.io/secret/b7defe22-28b4-4fec-8fe5-ebde8e658c6b-kube-state-metrics-tls-certs\") pod \"kube-state-metrics-0\" (UID: \"b7defe22-28b4-4fec-8fe5-ebde8e658c6b\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:36:49 crc kubenswrapper[5052]: I1203 12:36:49.303618 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls-config\" (UniqueName: \"kubernetes.io/secret/b7defe22-28b4-4fec-8fe5-ebde8e658c6b-kube-state-metrics-tls-config\") pod \"kube-state-metrics-0\" (UID: \"b7defe22-28b4-4fec-8fe5-ebde8e658c6b\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:36:49 crc kubenswrapper[5052]: I1203 12:36:49.308859 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-gvsj7\" (UniqueName: \"kubernetes.io/projected/b7defe22-28b4-4fec-8fe5-ebde8e658c6b-kube-api-access-gvsj7\") pod \"kube-state-metrics-0\" (UID: \"b7defe22-28b4-4fec-8fe5-ebde8e658c6b\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:36:49 crc kubenswrapper[5052]: I1203 12:36:49.354130 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:36:49 crc kubenswrapper[5052]: I1203 12:36:49.558667 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-api-0" podUID="7e7d9a94-b50b-4065-a178-557deda1100c" containerName="nova-api-api" probeResult="failure" output="Get \"http://10.217.1.83:8774/\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Dec 03 12:36:49 crc kubenswrapper[5052]: I1203 12:36:49.558997 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-api-0" podUID="7e7d9a94-b50b-4065-a178-557deda1100c" containerName="nova-api-log" probeResult="failure" output="Get \"http://10.217.1.83:8774/\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Dec 03 12:36:49 crc kubenswrapper[5052]: I1203 12:36:49.691266 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5c9ee44f-4958-4cc9-a8a4-e5c023b65baf" path="/var/lib/kubelet/pods/5c9ee44f-4958-4cc9-a8a4-e5c023b65baf/volumes" Dec 03 12:36:49 crc kubenswrapper[5052]: W1203 12:36:49.840394 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb7defe22_28b4_4fec_8fe5_ebde8e658c6b.slice/crio-d64051f0917a85a737352a0ca1c2523190331331b68cf6f466ba17250c4ba9ff WatchSource:0}: Error finding container d64051f0917a85a737352a0ca1c2523190331331b68cf6f466ba17250c4ba9ff: Status 404 returned error can't find the container with id d64051f0917a85a737352a0ca1c2523190331331b68cf6f466ba17250c4ba9ff Dec 03 12:36:49 crc kubenswrapper[5052]: I1203 12:36:49.851580 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/kube-state-metrics-0"] Dec 03 12:36:49 crc kubenswrapper[5052]: I1203 12:36:49.907419 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/kube-state-metrics-0" event={"ID":"b7defe22-28b4-4fec-8fe5-ebde8e658c6b","Type":"ContainerStarted","Data":"d64051f0917a85a737352a0ca1c2523190331331b68cf6f466ba17250c4ba9ff"} Dec 03 12:36:50 crc kubenswrapper[5052]: I1203 12:36:50.199061 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:36:50 crc kubenswrapper[5052]: I1203 12:36:50.199611 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="36028611-5675-4c85-af0d-6ba873b30da7" containerName="ceilometer-central-agent" containerID="cri-o://dd0ce7d71b0c277f456d230103929c6b85104381dfe61d5d51d3dbfbc10a484a" gracePeriod=30 Dec 03 12:36:50 crc kubenswrapper[5052]: I1203 12:36:50.199669 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="36028611-5675-4c85-af0d-6ba873b30da7" containerName="proxy-httpd" containerID="cri-o://5debcb02a261adaa17b1df9a4bfe8eef2137d454c7652df878fe48809d10cad3" gracePeriod=30 Dec 03 12:36:50 crc kubenswrapper[5052]: I1203 12:36:50.199754 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="36028611-5675-4c85-af0d-6ba873b30da7" containerName="ceilometer-notification-agent" containerID="cri-o://1ba41230cf7215eee34d6d8ea8681f728e95087f0ac46229ec46ae822a73d0e3" gracePeriod=30 Dec 03 12:36:50 crc kubenswrapper[5052]: I1203 12:36:50.199731 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="36028611-5675-4c85-af0d-6ba873b30da7" containerName="sg-core" containerID="cri-o://999ebc2e5d21eb2c7377ec909cc77ed7c15f0032bb17311e3331c347b89b4ade" gracePeriod=30 Dec 03 12:36:50 crc kubenswrapper[5052]: I1203 12:36:50.943879 5052 generic.go:334] "Generic (PLEG): container finished" podID="36028611-5675-4c85-af0d-6ba873b30da7" containerID="5debcb02a261adaa17b1df9a4bfe8eef2137d454c7652df878fe48809d10cad3" exitCode=0 Dec 03 12:36:50 crc kubenswrapper[5052]: I1203 12:36:50.944144 5052 generic.go:334] "Generic (PLEG): container finished" podID="36028611-5675-4c85-af0d-6ba873b30da7" containerID="999ebc2e5d21eb2c7377ec909cc77ed7c15f0032bb17311e3331c347b89b4ade" exitCode=2 Dec 03 12:36:50 crc kubenswrapper[5052]: I1203 12:36:50.944156 5052 generic.go:334] "Generic (PLEG): container finished" podID="36028611-5675-4c85-af0d-6ba873b30da7" containerID="dd0ce7d71b0c277f456d230103929c6b85104381dfe61d5d51d3dbfbc10a484a" exitCode=0 Dec 03 12:36:50 crc kubenswrapper[5052]: I1203 12:36:50.944192 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"36028611-5675-4c85-af0d-6ba873b30da7","Type":"ContainerDied","Data":"5debcb02a261adaa17b1df9a4bfe8eef2137d454c7652df878fe48809d10cad3"} Dec 03 12:36:50 crc kubenswrapper[5052]: I1203 12:36:50.944217 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"36028611-5675-4c85-af0d-6ba873b30da7","Type":"ContainerDied","Data":"999ebc2e5d21eb2c7377ec909cc77ed7c15f0032bb17311e3331c347b89b4ade"} Dec 03 12:36:50 crc kubenswrapper[5052]: I1203 12:36:50.944226 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"36028611-5675-4c85-af0d-6ba873b30da7","Type":"ContainerDied","Data":"dd0ce7d71b0c277f456d230103929c6b85104381dfe61d5d51d3dbfbc10a484a"} Dec 03 12:36:50 crc kubenswrapper[5052]: I1203 12:36:50.945893 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/kube-state-metrics-0" event={"ID":"b7defe22-28b4-4fec-8fe5-ebde8e658c6b","Type":"ContainerStarted","Data":"26de7dd666c3dea50b83a35cff57a4ece7ec413a749af698ab13d88f7a800f76"} Dec 03 12:36:50 crc kubenswrapper[5052]: I1203 12:36:50.946600 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:36:50 crc kubenswrapper[5052]: I1203 12:36:50.966682 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/kube-state-metrics-0" podStartSLOduration=2.608168831 podStartE2EDuration="2.966664964s" podCreationTimestamp="2025-12-03 12:36:48 +0000 UTC" firstStartedPulling="2025-12-03 12:36:49.844027358 +0000 UTC m=+4600.432269122" lastFinishedPulling="2025-12-03 12:36:50.202523491 +0000 UTC m=+4600.790765255" observedRunningTime="2025-12-03 12:36:50.964744579 +0000 UTC m=+4601.552986343" watchObservedRunningTime="2025-12-03 12:36:50.966664964 +0000 UTC m=+4601.554906728" Dec 03 12:36:54 crc kubenswrapper[5052]: I1203 12:36:54.998067 5052 generic.go:334] "Generic (PLEG): container finished" podID="47136a96-c6f5-428d-a11a-fa9eeb4910e7" containerID="89fab5932f39924d83d4ab819b2183e190ed6a7efae6f78ee3791c1efe360659" exitCode=0 Dec 03 12:36:54 crc kubenswrapper[5052]: I1203 12:36:54.998152 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-cell-mapping-fj7tx" event={"ID":"47136a96-c6f5-428d-a11a-fa9eeb4910e7","Type":"ContainerDied","Data":"89fab5932f39924d83d4ab819b2183e190ed6a7efae6f78ee3791c1efe360659"} Dec 03 12:36:55 crc kubenswrapper[5052]: I1203 12:36:55.006870 5052 generic.go:334] "Generic (PLEG): container finished" podID="36028611-5675-4c85-af0d-6ba873b30da7" containerID="1ba41230cf7215eee34d6d8ea8681f728e95087f0ac46229ec46ae822a73d0e3" exitCode=0 Dec 03 12:36:55 crc kubenswrapper[5052]: I1203 12:36:55.006918 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"36028611-5675-4c85-af0d-6ba873b30da7","Type":"ContainerDied","Data":"1ba41230cf7215eee34d6d8ea8681f728e95087f0ac46229ec46ae822a73d0e3"} Dec 03 12:36:55 crc kubenswrapper[5052]: I1203 12:36:55.006946 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"36028611-5675-4c85-af0d-6ba873b30da7","Type":"ContainerDied","Data":"edd8622abc71df59fb09ec409b84ea2f22620d7ac699ba96b789afe070bb03d5"} Dec 03 12:36:55 crc kubenswrapper[5052]: I1203 12:36:55.006957 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="edd8622abc71df59fb09ec409b84ea2f22620d7ac699ba96b789afe070bb03d5" Dec 03 12:36:55 crc kubenswrapper[5052]: I1203 12:36:55.149644 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:36:55 crc kubenswrapper[5052]: I1203 12:36:55.150337 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:36:55 crc kubenswrapper[5052]: I1203 12:36:55.153802 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:36:55 crc kubenswrapper[5052]: I1203 12:36:55.564734 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:36:55 crc kubenswrapper[5052]: I1203 12:36:55.673483 5052 scope.go:117] "RemoveContainer" containerID="e6edaf188f85ea4f11f487f11cc489c700f25fb42b5ee269f4f496f1e127d772" Dec 03 12:36:55 crc kubenswrapper[5052]: E1203 12:36:55.673983 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 12:36:55 crc kubenswrapper[5052]: I1203 12:36:55.709552 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9f27c\" (UniqueName: \"kubernetes.io/projected/36028611-5675-4c85-af0d-6ba873b30da7-kube-api-access-9f27c\") pod \"36028611-5675-4c85-af0d-6ba873b30da7\" (UID: \"36028611-5675-4c85-af0d-6ba873b30da7\") " Dec 03 12:36:55 crc kubenswrapper[5052]: I1203 12:36:55.709654 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/36028611-5675-4c85-af0d-6ba873b30da7-run-httpd\") pod \"36028611-5675-4c85-af0d-6ba873b30da7\" (UID: \"36028611-5675-4c85-af0d-6ba873b30da7\") " Dec 03 12:36:55 crc kubenswrapper[5052]: I1203 12:36:55.709716 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/36028611-5675-4c85-af0d-6ba873b30da7-config-data\") pod \"36028611-5675-4c85-af0d-6ba873b30da7\" (UID: \"36028611-5675-4c85-af0d-6ba873b30da7\") " Dec 03 12:36:55 crc kubenswrapper[5052]: I1203 12:36:55.709746 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/36028611-5675-4c85-af0d-6ba873b30da7-combined-ca-bundle\") pod \"36028611-5675-4c85-af0d-6ba873b30da7\" (UID: \"36028611-5675-4c85-af0d-6ba873b30da7\") " Dec 03 12:36:55 crc kubenswrapper[5052]: I1203 12:36:55.709937 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/36028611-5675-4c85-af0d-6ba873b30da7-sg-core-conf-yaml\") pod \"36028611-5675-4c85-af0d-6ba873b30da7\" (UID: \"36028611-5675-4c85-af0d-6ba873b30da7\") " Dec 03 12:36:55 crc kubenswrapper[5052]: I1203 12:36:55.709982 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/36028611-5675-4c85-af0d-6ba873b30da7-log-httpd\") pod \"36028611-5675-4c85-af0d-6ba873b30da7\" (UID: \"36028611-5675-4c85-af0d-6ba873b30da7\") " Dec 03 12:36:55 crc kubenswrapper[5052]: I1203 12:36:55.710054 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/36028611-5675-4c85-af0d-6ba873b30da7-scripts\") pod \"36028611-5675-4c85-af0d-6ba873b30da7\" (UID: \"36028611-5675-4c85-af0d-6ba873b30da7\") " Dec 03 12:36:55 crc kubenswrapper[5052]: I1203 12:36:55.710134 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36028611-5675-4c85-af0d-6ba873b30da7-run-httpd" (OuterVolumeSpecName: "run-httpd") pod "36028611-5675-4c85-af0d-6ba873b30da7" (UID: "36028611-5675-4c85-af0d-6ba873b30da7"). InnerVolumeSpecName "run-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:36:55 crc kubenswrapper[5052]: I1203 12:36:55.710593 5052 reconciler_common.go:293] "Volume detached for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/36028611-5675-4c85-af0d-6ba873b30da7-run-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:36:55 crc kubenswrapper[5052]: I1203 12:36:55.711373 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36028611-5675-4c85-af0d-6ba873b30da7-log-httpd" (OuterVolumeSpecName: "log-httpd") pod "36028611-5675-4c85-af0d-6ba873b30da7" (UID: "36028611-5675-4c85-af0d-6ba873b30da7"). InnerVolumeSpecName "log-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:36:55 crc kubenswrapper[5052]: I1203 12:36:55.723039 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/36028611-5675-4c85-af0d-6ba873b30da7-scripts" (OuterVolumeSpecName: "scripts") pod "36028611-5675-4c85-af0d-6ba873b30da7" (UID: "36028611-5675-4c85-af0d-6ba873b30da7"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:36:55 crc kubenswrapper[5052]: I1203 12:36:55.814250 5052 reconciler_common.go:293] "Volume detached for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/36028611-5675-4c85-af0d-6ba873b30da7-log-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:36:55 crc kubenswrapper[5052]: I1203 12:36:55.814300 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/36028611-5675-4c85-af0d-6ba873b30da7-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:36:55 crc kubenswrapper[5052]: I1203 12:36:55.923893 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/36028611-5675-4c85-af0d-6ba873b30da7-kube-api-access-9f27c" (OuterVolumeSpecName: "kube-api-access-9f27c") pod "36028611-5675-4c85-af0d-6ba873b30da7" (UID: "36028611-5675-4c85-af0d-6ba873b30da7"). InnerVolumeSpecName "kube-api-access-9f27c". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:36:55 crc kubenswrapper[5052]: I1203 12:36:55.927384 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/36028611-5675-4c85-af0d-6ba873b30da7-sg-core-conf-yaml" (OuterVolumeSpecName: "sg-core-conf-yaml") pod "36028611-5675-4c85-af0d-6ba873b30da7" (UID: "36028611-5675-4c85-af0d-6ba873b30da7"). InnerVolumeSpecName "sg-core-conf-yaml". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:36:55 crc kubenswrapper[5052]: I1203 12:36:55.985965 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/36028611-5675-4c85-af0d-6ba873b30da7-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "36028611-5675-4c85-af0d-6ba873b30da7" (UID: "36028611-5675-4c85-af0d-6ba873b30da7"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:36:56 crc kubenswrapper[5052]: I1203 12:36:56.010177 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/36028611-5675-4c85-af0d-6ba873b30da7-config-data" (OuterVolumeSpecName: "config-data") pod "36028611-5675-4c85-af0d-6ba873b30da7" (UID: "36028611-5675-4c85-af0d-6ba873b30da7"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:36:56 crc kubenswrapper[5052]: I1203 12:36:56.019754 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:36:56 crc kubenswrapper[5052]: I1203 12:36:56.024140 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/36028611-5675-4c85-af0d-6ba873b30da7-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:36:56 crc kubenswrapper[5052]: I1203 12:36:56.028291 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/36028611-5675-4c85-af0d-6ba873b30da7-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:36:56 crc kubenswrapper[5052]: I1203 12:36:56.028410 5052 reconciler_common.go:293] "Volume detached for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/36028611-5675-4c85-af0d-6ba873b30da7-sg-core-conf-yaml\") on node \"crc\" DevicePath \"\"" Dec 03 12:36:56 crc kubenswrapper[5052]: I1203 12:36:56.028504 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-9f27c\" (UniqueName: \"kubernetes.io/projected/36028611-5675-4c85-af0d-6ba873b30da7-kube-api-access-9f27c\") on node \"crc\" DevicePath \"\"" Dec 03 12:36:56 crc kubenswrapper[5052]: I1203 12:36:56.028995 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:36:56 crc kubenswrapper[5052]: I1203 12:36:56.092268 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:36:56 crc kubenswrapper[5052]: I1203 12:36:56.105464 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:36:56 crc kubenswrapper[5052]: I1203 12:36:56.132355 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:36:56 crc kubenswrapper[5052]: E1203 12:36:56.132867 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="36028611-5675-4c85-af0d-6ba873b30da7" containerName="sg-core" Dec 03 12:36:56 crc kubenswrapper[5052]: I1203 12:36:56.132892 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="36028611-5675-4c85-af0d-6ba873b30da7" containerName="sg-core" Dec 03 12:36:56 crc kubenswrapper[5052]: E1203 12:36:56.132901 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="36028611-5675-4c85-af0d-6ba873b30da7" containerName="proxy-httpd" Dec 03 12:36:56 crc kubenswrapper[5052]: I1203 12:36:56.132910 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="36028611-5675-4c85-af0d-6ba873b30da7" containerName="proxy-httpd" Dec 03 12:36:56 crc kubenswrapper[5052]: E1203 12:36:56.132928 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="36028611-5675-4c85-af0d-6ba873b30da7" containerName="ceilometer-notification-agent" Dec 03 12:36:56 crc kubenswrapper[5052]: I1203 12:36:56.132935 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="36028611-5675-4c85-af0d-6ba873b30da7" containerName="ceilometer-notification-agent" Dec 03 12:36:56 crc kubenswrapper[5052]: E1203 12:36:56.132946 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="36028611-5675-4c85-af0d-6ba873b30da7" containerName="ceilometer-central-agent" Dec 03 12:36:56 crc kubenswrapper[5052]: I1203 12:36:56.132953 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="36028611-5675-4c85-af0d-6ba873b30da7" containerName="ceilometer-central-agent" Dec 03 12:36:56 crc kubenswrapper[5052]: I1203 12:36:56.133145 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="36028611-5675-4c85-af0d-6ba873b30da7" containerName="proxy-httpd" Dec 03 12:36:56 crc kubenswrapper[5052]: I1203 12:36:56.133177 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="36028611-5675-4c85-af0d-6ba873b30da7" containerName="ceilometer-notification-agent" Dec 03 12:36:56 crc kubenswrapper[5052]: I1203 12:36:56.133189 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="36028611-5675-4c85-af0d-6ba873b30da7" containerName="ceilometer-central-agent" Dec 03 12:36:56 crc kubenswrapper[5052]: I1203 12:36:56.133195 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="36028611-5675-4c85-af0d-6ba873b30da7" containerName="sg-core" Dec 03 12:36:56 crc kubenswrapper[5052]: I1203 12:36:56.135048 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:36:56 crc kubenswrapper[5052]: I1203 12:36:56.138916 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-scripts" Dec 03 12:36:56 crc kubenswrapper[5052]: I1203 12:36:56.139237 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-ceilometer-internal-svc" Dec 03 12:36:56 crc kubenswrapper[5052]: I1203 12:36:56.139424 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-config-data" Dec 03 12:36:56 crc kubenswrapper[5052]: I1203 12:36:56.144529 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:36:56 crc kubenswrapper[5052]: I1203 12:36:56.232132 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/42839582-fefc-4756-933b-d0ba9fb9334c-ceilometer-tls-certs\") pod \"ceilometer-0\" (UID: \"42839582-fefc-4756-933b-d0ba9fb9334c\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:36:56 crc kubenswrapper[5052]: I1203 12:36:56.232180 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/42839582-fefc-4756-933b-d0ba9fb9334c-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"42839582-fefc-4756-933b-d0ba9fb9334c\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:36:56 crc kubenswrapper[5052]: I1203 12:36:56.232242 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4pmqp\" (UniqueName: \"kubernetes.io/projected/42839582-fefc-4756-933b-d0ba9fb9334c-kube-api-access-4pmqp\") pod \"ceilometer-0\" (UID: \"42839582-fefc-4756-933b-d0ba9fb9334c\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:36:56 crc kubenswrapper[5052]: I1203 12:36:56.232351 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/42839582-fefc-4756-933b-d0ba9fb9334c-scripts\") pod \"ceilometer-0\" (UID: \"42839582-fefc-4756-933b-d0ba9fb9334c\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:36:56 crc kubenswrapper[5052]: I1203 12:36:56.232393 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/42839582-fefc-4756-933b-d0ba9fb9334c-config-data\") pod \"ceilometer-0\" (UID: \"42839582-fefc-4756-933b-d0ba9fb9334c\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:36:56 crc kubenswrapper[5052]: I1203 12:36:56.232444 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/42839582-fefc-4756-933b-d0ba9fb9334c-run-httpd\") pod \"ceilometer-0\" (UID: \"42839582-fefc-4756-933b-d0ba9fb9334c\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:36:56 crc kubenswrapper[5052]: I1203 12:36:56.232547 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/42839582-fefc-4756-933b-d0ba9fb9334c-log-httpd\") pod \"ceilometer-0\" (UID: \"42839582-fefc-4756-933b-d0ba9fb9334c\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:36:56 crc kubenswrapper[5052]: I1203 12:36:56.232573 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/42839582-fefc-4756-933b-d0ba9fb9334c-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"42839582-fefc-4756-933b-d0ba9fb9334c\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:36:56 crc kubenswrapper[5052]: I1203 12:36:56.334139 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/42839582-fefc-4756-933b-d0ba9fb9334c-ceilometer-tls-certs\") pod \"ceilometer-0\" (UID: \"42839582-fefc-4756-933b-d0ba9fb9334c\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:36:56 crc kubenswrapper[5052]: I1203 12:36:56.334507 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/42839582-fefc-4756-933b-d0ba9fb9334c-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"42839582-fefc-4756-933b-d0ba9fb9334c\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:36:56 crc kubenswrapper[5052]: I1203 12:36:56.334598 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-4pmqp\" (UniqueName: \"kubernetes.io/projected/42839582-fefc-4756-933b-d0ba9fb9334c-kube-api-access-4pmqp\") pod \"ceilometer-0\" (UID: \"42839582-fefc-4756-933b-d0ba9fb9334c\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:36:56 crc kubenswrapper[5052]: I1203 12:36:56.334664 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/42839582-fefc-4756-933b-d0ba9fb9334c-scripts\") pod \"ceilometer-0\" (UID: \"42839582-fefc-4756-933b-d0ba9fb9334c\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:36:56 crc kubenswrapper[5052]: I1203 12:36:56.334696 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/42839582-fefc-4756-933b-d0ba9fb9334c-config-data\") pod \"ceilometer-0\" (UID: \"42839582-fefc-4756-933b-d0ba9fb9334c\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:36:56 crc kubenswrapper[5052]: I1203 12:36:56.334745 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/42839582-fefc-4756-933b-d0ba9fb9334c-run-httpd\") pod \"ceilometer-0\" (UID: \"42839582-fefc-4756-933b-d0ba9fb9334c\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:36:56 crc kubenswrapper[5052]: I1203 12:36:56.334812 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/42839582-fefc-4756-933b-d0ba9fb9334c-log-httpd\") pod \"ceilometer-0\" (UID: \"42839582-fefc-4756-933b-d0ba9fb9334c\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:36:56 crc kubenswrapper[5052]: I1203 12:36:56.334839 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/42839582-fefc-4756-933b-d0ba9fb9334c-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"42839582-fefc-4756-933b-d0ba9fb9334c\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:36:56 crc kubenswrapper[5052]: I1203 12:36:56.338822 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/42839582-fefc-4756-933b-d0ba9fb9334c-ceilometer-tls-certs\") pod \"ceilometer-0\" (UID: \"42839582-fefc-4756-933b-d0ba9fb9334c\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:36:56 crc kubenswrapper[5052]: I1203 12:36:56.338822 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/42839582-fefc-4756-933b-d0ba9fb9334c-scripts\") pod \"ceilometer-0\" (UID: \"42839582-fefc-4756-933b-d0ba9fb9334c\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:36:56 crc kubenswrapper[5052]: I1203 12:36:56.339691 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/42839582-fefc-4756-933b-d0ba9fb9334c-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"42839582-fefc-4756-933b-d0ba9fb9334c\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:36:56 crc kubenswrapper[5052]: I1203 12:36:56.344038 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/42839582-fefc-4756-933b-d0ba9fb9334c-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"42839582-fefc-4756-933b-d0ba9fb9334c\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:36:56 crc kubenswrapper[5052]: I1203 12:36:56.344316 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/42839582-fefc-4756-933b-d0ba9fb9334c-log-httpd\") pod \"ceilometer-0\" (UID: \"42839582-fefc-4756-933b-d0ba9fb9334c\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:36:56 crc kubenswrapper[5052]: I1203 12:36:56.344566 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/42839582-fefc-4756-933b-d0ba9fb9334c-run-httpd\") pod \"ceilometer-0\" (UID: \"42839582-fefc-4756-933b-d0ba9fb9334c\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:36:56 crc kubenswrapper[5052]: I1203 12:36:56.350135 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/42839582-fefc-4756-933b-d0ba9fb9334c-config-data\") pod \"ceilometer-0\" (UID: \"42839582-fefc-4756-933b-d0ba9fb9334c\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:36:56 crc kubenswrapper[5052]: I1203 12:36:56.362531 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-4pmqp\" (UniqueName: \"kubernetes.io/projected/42839582-fefc-4756-933b-d0ba9fb9334c-kube-api-access-4pmqp\") pod \"ceilometer-0\" (UID: \"42839582-fefc-4756-933b-d0ba9fb9334c\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:36:56 crc kubenswrapper[5052]: I1203 12:36:56.396636 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-cell-mapping-fj7tx" Dec 03 12:36:56 crc kubenswrapper[5052]: I1203 12:36:56.435499 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2rsnc\" (UniqueName: \"kubernetes.io/projected/47136a96-c6f5-428d-a11a-fa9eeb4910e7-kube-api-access-2rsnc\") pod \"47136a96-c6f5-428d-a11a-fa9eeb4910e7\" (UID: \"47136a96-c6f5-428d-a11a-fa9eeb4910e7\") " Dec 03 12:36:56 crc kubenswrapper[5052]: I1203 12:36:56.435571 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/47136a96-c6f5-428d-a11a-fa9eeb4910e7-config-data\") pod \"47136a96-c6f5-428d-a11a-fa9eeb4910e7\" (UID: \"47136a96-c6f5-428d-a11a-fa9eeb4910e7\") " Dec 03 12:36:56 crc kubenswrapper[5052]: I1203 12:36:56.435605 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/47136a96-c6f5-428d-a11a-fa9eeb4910e7-combined-ca-bundle\") pod \"47136a96-c6f5-428d-a11a-fa9eeb4910e7\" (UID: \"47136a96-c6f5-428d-a11a-fa9eeb4910e7\") " Dec 03 12:36:56 crc kubenswrapper[5052]: I1203 12:36:56.435626 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/47136a96-c6f5-428d-a11a-fa9eeb4910e7-scripts\") pod \"47136a96-c6f5-428d-a11a-fa9eeb4910e7\" (UID: \"47136a96-c6f5-428d-a11a-fa9eeb4910e7\") " Dec 03 12:36:56 crc kubenswrapper[5052]: I1203 12:36:56.439424 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/47136a96-c6f5-428d-a11a-fa9eeb4910e7-scripts" (OuterVolumeSpecName: "scripts") pod "47136a96-c6f5-428d-a11a-fa9eeb4910e7" (UID: "47136a96-c6f5-428d-a11a-fa9eeb4910e7"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:36:56 crc kubenswrapper[5052]: I1203 12:36:56.440962 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/47136a96-c6f5-428d-a11a-fa9eeb4910e7-kube-api-access-2rsnc" (OuterVolumeSpecName: "kube-api-access-2rsnc") pod "47136a96-c6f5-428d-a11a-fa9eeb4910e7" (UID: "47136a96-c6f5-428d-a11a-fa9eeb4910e7"). InnerVolumeSpecName "kube-api-access-2rsnc". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:36:56 crc kubenswrapper[5052]: I1203 12:36:56.460185 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/47136a96-c6f5-428d-a11a-fa9eeb4910e7-config-data" (OuterVolumeSpecName: "config-data") pod "47136a96-c6f5-428d-a11a-fa9eeb4910e7" (UID: "47136a96-c6f5-428d-a11a-fa9eeb4910e7"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:36:56 crc kubenswrapper[5052]: I1203 12:36:56.463727 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:36:56 crc kubenswrapper[5052]: I1203 12:36:56.466122 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/47136a96-c6f5-428d-a11a-fa9eeb4910e7-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "47136a96-c6f5-428d-a11a-fa9eeb4910e7" (UID: "47136a96-c6f5-428d-a11a-fa9eeb4910e7"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:36:56 crc kubenswrapper[5052]: I1203 12:36:56.538274 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-2rsnc\" (UniqueName: \"kubernetes.io/projected/47136a96-c6f5-428d-a11a-fa9eeb4910e7-kube-api-access-2rsnc\") on node \"crc\" DevicePath \"\"" Dec 03 12:36:56 crc kubenswrapper[5052]: I1203 12:36:56.538308 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/47136a96-c6f5-428d-a11a-fa9eeb4910e7-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:36:56 crc kubenswrapper[5052]: I1203 12:36:56.538320 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/47136a96-c6f5-428d-a11a-fa9eeb4910e7-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:36:56 crc kubenswrapper[5052]: I1203 12:36:56.538328 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/47136a96-c6f5-428d-a11a-fa9eeb4910e7-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:36:56 crc kubenswrapper[5052]: I1203 12:36:56.932495 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:36:57 crc kubenswrapper[5052]: I1203 12:36:57.031736 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-cell-mapping-fj7tx" Dec 03 12:36:57 crc kubenswrapper[5052]: I1203 12:36:57.031731 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-cell-mapping-fj7tx" event={"ID":"47136a96-c6f5-428d-a11a-fa9eeb4910e7","Type":"ContainerDied","Data":"225cf34a2ab533a9646e28b5830daa206e7eeee9515b766f4f51f4995cc2809f"} Dec 03 12:36:57 crc kubenswrapper[5052]: I1203 12:36:57.031884 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="225cf34a2ab533a9646e28b5830daa206e7eeee9515b766f4f51f4995cc2809f" Dec 03 12:36:57 crc kubenswrapper[5052]: I1203 12:36:57.033219 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"42839582-fefc-4756-933b-d0ba9fb9334c","Type":"ContainerStarted","Data":"60e7f9dafeb8dfe87186dd910f6d0baf15a2512f1932425263e48763f3382764"} Dec 03 12:36:57 crc kubenswrapper[5052]: I1203 12:36:57.111785 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:36:57 crc kubenswrapper[5052]: I1203 12:36:57.112062 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-api-0" podUID="7e7d9a94-b50b-4065-a178-557deda1100c" containerName="nova-api-log" containerID="cri-o://e151b80f440784eeb61f00e0a399f2ccb9b03fb4ee3fee6bac9ffb707dde5f36" gracePeriod=30 Dec 03 12:36:57 crc kubenswrapper[5052]: I1203 12:36:57.112101 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-api-0" podUID="7e7d9a94-b50b-4065-a178-557deda1100c" containerName="nova-api-api" containerID="cri-o://2de47cacb469eb9f4ef023ca0e0c9a3f1c57692a0ef8c8f65261831c30d72149" gracePeriod=30 Dec 03 12:36:57 crc kubenswrapper[5052]: I1203 12:36:57.120460 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:36:57 crc kubenswrapper[5052]: I1203 12:36:57.120650 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-scheduler-0" podUID="1316edb7-1894-427d-92bf-dab018e056fd" containerName="nova-scheduler-scheduler" containerID="cri-o://40c2a8b3592d4fb942b76bd47d5c522c0f6eac013387e752b7ac5f8e344a5983" gracePeriod=30 Dec 03 12:36:57 crc kubenswrapper[5052]: I1203 12:36:57.170868 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:36:57 crc kubenswrapper[5052]: I1203 12:36:57.683702 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="36028611-5675-4c85-af0d-6ba873b30da7" path="/var/lib/kubelet/pods/36028611-5675-4c85-af0d-6ba873b30da7/volumes" Dec 03 12:36:58 crc kubenswrapper[5052]: I1203 12:36:58.044101 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"42839582-fefc-4756-933b-d0ba9fb9334c","Type":"ContainerStarted","Data":"e95078911abc9ff62ef4de1494f5c34277564504f3550e7458213fe28afe676b"} Dec 03 12:36:58 crc kubenswrapper[5052]: I1203 12:36:58.045584 5052 generic.go:334] "Generic (PLEG): container finished" podID="1316edb7-1894-427d-92bf-dab018e056fd" containerID="40c2a8b3592d4fb942b76bd47d5c522c0f6eac013387e752b7ac5f8e344a5983" exitCode=0 Dec 03 12:36:58 crc kubenswrapper[5052]: I1203 12:36:58.045640 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"1316edb7-1894-427d-92bf-dab018e056fd","Type":"ContainerDied","Data":"40c2a8b3592d4fb942b76bd47d5c522c0f6eac013387e752b7ac5f8e344a5983"} Dec 03 12:36:58 crc kubenswrapper[5052]: I1203 12:36:58.047276 5052 generic.go:334] "Generic (PLEG): container finished" podID="7e7d9a94-b50b-4065-a178-557deda1100c" containerID="e151b80f440784eeb61f00e0a399f2ccb9b03fb4ee3fee6bac9ffb707dde5f36" exitCode=143 Dec 03 12:36:58 crc kubenswrapper[5052]: I1203 12:36:58.047363 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"7e7d9a94-b50b-4065-a178-557deda1100c","Type":"ContainerDied","Data":"e151b80f440784eeb61f00e0a399f2ccb9b03fb4ee3fee6bac9ffb707dde5f36"} Dec 03 12:36:58 crc kubenswrapper[5052]: I1203 12:36:58.358560 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:36:58 crc kubenswrapper[5052]: I1203 12:36:58.388534 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1316edb7-1894-427d-92bf-dab018e056fd-combined-ca-bundle\") pod \"1316edb7-1894-427d-92bf-dab018e056fd\" (UID: \"1316edb7-1894-427d-92bf-dab018e056fd\") " Dec 03 12:36:58 crc kubenswrapper[5052]: I1203 12:36:58.388585 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/1316edb7-1894-427d-92bf-dab018e056fd-config-data\") pod \"1316edb7-1894-427d-92bf-dab018e056fd\" (UID: \"1316edb7-1894-427d-92bf-dab018e056fd\") " Dec 03 12:36:58 crc kubenswrapper[5052]: I1203 12:36:58.388617 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r46rj\" (UniqueName: \"kubernetes.io/projected/1316edb7-1894-427d-92bf-dab018e056fd-kube-api-access-r46rj\") pod \"1316edb7-1894-427d-92bf-dab018e056fd\" (UID: \"1316edb7-1894-427d-92bf-dab018e056fd\") " Dec 03 12:36:58 crc kubenswrapper[5052]: I1203 12:36:58.403732 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1316edb7-1894-427d-92bf-dab018e056fd-kube-api-access-r46rj" (OuterVolumeSpecName: "kube-api-access-r46rj") pod "1316edb7-1894-427d-92bf-dab018e056fd" (UID: "1316edb7-1894-427d-92bf-dab018e056fd"). InnerVolumeSpecName "kube-api-access-r46rj". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:36:58 crc kubenswrapper[5052]: I1203 12:36:58.434391 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1316edb7-1894-427d-92bf-dab018e056fd-config-data" (OuterVolumeSpecName: "config-data") pod "1316edb7-1894-427d-92bf-dab018e056fd" (UID: "1316edb7-1894-427d-92bf-dab018e056fd"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:36:58 crc kubenswrapper[5052]: I1203 12:36:58.452978 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1316edb7-1894-427d-92bf-dab018e056fd-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "1316edb7-1894-427d-92bf-dab018e056fd" (UID: "1316edb7-1894-427d-92bf-dab018e056fd"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:36:58 crc kubenswrapper[5052]: I1203 12:36:58.490610 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/1316edb7-1894-427d-92bf-dab018e056fd-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:36:58 crc kubenswrapper[5052]: I1203 12:36:58.490650 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-r46rj\" (UniqueName: \"kubernetes.io/projected/1316edb7-1894-427d-92bf-dab018e056fd-kube-api-access-r46rj\") on node \"crc\" DevicePath \"\"" Dec 03 12:36:58 crc kubenswrapper[5052]: I1203 12:36:58.490664 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1316edb7-1894-427d-92bf-dab018e056fd-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:36:59 crc kubenswrapper[5052]: I1203 12:36:59.059450 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"42839582-fefc-4756-933b-d0ba9fb9334c","Type":"ContainerStarted","Data":"7b420af945e0dc26ceeb21ff59642d3f3e96828f457c6d91a7c64ec8f9111c11"} Dec 03 12:36:59 crc kubenswrapper[5052]: I1203 12:36:59.063483 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"1316edb7-1894-427d-92bf-dab018e056fd","Type":"ContainerDied","Data":"032ba52f8e13c68cd39376b0e7f50f353a40ac4cdb56ab0ff1ba8f9f7494f68f"} Dec 03 12:36:59 crc kubenswrapper[5052]: I1203 12:36:59.063526 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:36:59 crc kubenswrapper[5052]: I1203 12:36:59.063557 5052 scope.go:117] "RemoveContainer" containerID="40c2a8b3592d4fb942b76bd47d5c522c0f6eac013387e752b7ac5f8e344a5983" Dec 03 12:36:59 crc kubenswrapper[5052]: I1203 12:36:59.063671 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-metadata-0" podUID="69502269-3194-440d-aa90-8a9a1821a297" containerName="nova-metadata-log" containerID="cri-o://8158be457c8bd67dfca9154299916c22a6ad78859653cc60a7a7925603aaf518" gracePeriod=30 Dec 03 12:36:59 crc kubenswrapper[5052]: I1203 12:36:59.063807 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-metadata-0" podUID="69502269-3194-440d-aa90-8a9a1821a297" containerName="nova-metadata-metadata" containerID="cri-o://6340ef46e764a405f61fe84125c42eb8a5422a7ba57320782ecc1f6addbfbe0c" gracePeriod=30 Dec 03 12:36:59 crc kubenswrapper[5052]: I1203 12:36:59.217138 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:36:59 crc kubenswrapper[5052]: I1203 12:36:59.227297 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:36:59 crc kubenswrapper[5052]: I1203 12:36:59.245140 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:36:59 crc kubenswrapper[5052]: E1203 12:36:59.245711 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="47136a96-c6f5-428d-a11a-fa9eeb4910e7" containerName="nova-manage" Dec 03 12:36:59 crc kubenswrapper[5052]: I1203 12:36:59.245733 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="47136a96-c6f5-428d-a11a-fa9eeb4910e7" containerName="nova-manage" Dec 03 12:36:59 crc kubenswrapper[5052]: E1203 12:36:59.245757 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="1316edb7-1894-427d-92bf-dab018e056fd" containerName="nova-scheduler-scheduler" Dec 03 12:36:59 crc kubenswrapper[5052]: I1203 12:36:59.245765 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="1316edb7-1894-427d-92bf-dab018e056fd" containerName="nova-scheduler-scheduler" Dec 03 12:36:59 crc kubenswrapper[5052]: I1203 12:36:59.246015 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="1316edb7-1894-427d-92bf-dab018e056fd" containerName="nova-scheduler-scheduler" Dec 03 12:36:59 crc kubenswrapper[5052]: I1203 12:36:59.246037 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="47136a96-c6f5-428d-a11a-fa9eeb4910e7" containerName="nova-manage" Dec 03 12:36:59 crc kubenswrapper[5052]: I1203 12:36:59.246892 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:36:59 crc kubenswrapper[5052]: I1203 12:36:59.250065 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-scheduler-config-data" Dec 03 12:36:59 crc kubenswrapper[5052]: I1203 12:36:59.285151 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:36:59 crc kubenswrapper[5052]: I1203 12:36:59.329370 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m2kgw\" (UniqueName: \"kubernetes.io/projected/b0baf287-dc5f-45d0-b09d-324933082b9e-kube-api-access-m2kgw\") pod \"nova-scheduler-0\" (UID: \"b0baf287-dc5f-45d0-b09d-324933082b9e\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:36:59 crc kubenswrapper[5052]: I1203 12:36:59.329486 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b0baf287-dc5f-45d0-b09d-324933082b9e-combined-ca-bundle\") pod \"nova-scheduler-0\" (UID: \"b0baf287-dc5f-45d0-b09d-324933082b9e\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:36:59 crc kubenswrapper[5052]: I1203 12:36:59.329634 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/b0baf287-dc5f-45d0-b09d-324933082b9e-config-data\") pod \"nova-scheduler-0\" (UID: \"b0baf287-dc5f-45d0-b09d-324933082b9e\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:36:59 crc kubenswrapper[5052]: I1203 12:36:59.367620 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:36:59 crc kubenswrapper[5052]: I1203 12:36:59.430997 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/b0baf287-dc5f-45d0-b09d-324933082b9e-config-data\") pod \"nova-scheduler-0\" (UID: \"b0baf287-dc5f-45d0-b09d-324933082b9e\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:36:59 crc kubenswrapper[5052]: I1203 12:36:59.431099 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-m2kgw\" (UniqueName: \"kubernetes.io/projected/b0baf287-dc5f-45d0-b09d-324933082b9e-kube-api-access-m2kgw\") pod \"nova-scheduler-0\" (UID: \"b0baf287-dc5f-45d0-b09d-324933082b9e\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:36:59 crc kubenswrapper[5052]: I1203 12:36:59.431371 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b0baf287-dc5f-45d0-b09d-324933082b9e-combined-ca-bundle\") pod \"nova-scheduler-0\" (UID: \"b0baf287-dc5f-45d0-b09d-324933082b9e\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:36:59 crc kubenswrapper[5052]: I1203 12:36:59.438149 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/b0baf287-dc5f-45d0-b09d-324933082b9e-config-data\") pod \"nova-scheduler-0\" (UID: \"b0baf287-dc5f-45d0-b09d-324933082b9e\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:36:59 crc kubenswrapper[5052]: I1203 12:36:59.440299 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b0baf287-dc5f-45d0-b09d-324933082b9e-combined-ca-bundle\") pod \"nova-scheduler-0\" (UID: \"b0baf287-dc5f-45d0-b09d-324933082b9e\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:36:59 crc kubenswrapper[5052]: I1203 12:36:59.457458 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-m2kgw\" (UniqueName: \"kubernetes.io/projected/b0baf287-dc5f-45d0-b09d-324933082b9e-kube-api-access-m2kgw\") pod \"nova-scheduler-0\" (UID: \"b0baf287-dc5f-45d0-b09d-324933082b9e\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:36:59 crc kubenswrapper[5052]: I1203 12:36:59.567309 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:36:59 crc kubenswrapper[5052]: I1203 12:36:59.688918 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1316edb7-1894-427d-92bf-dab018e056fd" path="/var/lib/kubelet/pods/1316edb7-1894-427d-92bf-dab018e056fd/volumes" Dec 03 12:37:00 crc kubenswrapper[5052]: W1203 12:37:00.030846 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb0baf287_dc5f_45d0_b09d_324933082b9e.slice/crio-369b7fc23b87c3915b9767772260a1ef51bf180abe5bb11dc2acd6fba66b6884 WatchSource:0}: Error finding container 369b7fc23b87c3915b9767772260a1ef51bf180abe5bb11dc2acd6fba66b6884: Status 404 returned error can't find the container with id 369b7fc23b87c3915b9767772260a1ef51bf180abe5bb11dc2acd6fba66b6884 Dec 03 12:37:00 crc kubenswrapper[5052]: I1203 12:37:00.041059 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:37:00 crc kubenswrapper[5052]: I1203 12:37:00.076289 5052 generic.go:334] "Generic (PLEG): container finished" podID="69502269-3194-440d-aa90-8a9a1821a297" containerID="8158be457c8bd67dfca9154299916c22a6ad78859653cc60a7a7925603aaf518" exitCode=143 Dec 03 12:37:00 crc kubenswrapper[5052]: I1203 12:37:00.076353 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"69502269-3194-440d-aa90-8a9a1821a297","Type":"ContainerDied","Data":"8158be457c8bd67dfca9154299916c22a6ad78859653cc60a7a7925603aaf518"} Dec 03 12:37:00 crc kubenswrapper[5052]: I1203 12:37:00.078809 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"42839582-fefc-4756-933b-d0ba9fb9334c","Type":"ContainerStarted","Data":"b05a5920f0129ccca27e993e2d3d6d889d654dc1612338a0709892206efad711"} Dec 03 12:37:00 crc kubenswrapper[5052]: I1203 12:37:00.080781 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"b0baf287-dc5f-45d0-b09d-324933082b9e","Type":"ContainerStarted","Data":"369b7fc23b87c3915b9767772260a1ef51bf180abe5bb11dc2acd6fba66b6884"} Dec 03 12:37:00 crc kubenswrapper[5052]: I1203 12:37:00.593263 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:37:00 crc kubenswrapper[5052]: I1203 12:37:00.652535 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/7e7d9a94-b50b-4065-a178-557deda1100c-logs\") pod \"7e7d9a94-b50b-4065-a178-557deda1100c\" (UID: \"7e7d9a94-b50b-4065-a178-557deda1100c\") " Dec 03 12:37:00 crc kubenswrapper[5052]: I1203 12:37:00.652611 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/7e7d9a94-b50b-4065-a178-557deda1100c-combined-ca-bundle\") pod \"7e7d9a94-b50b-4065-a178-557deda1100c\" (UID: \"7e7d9a94-b50b-4065-a178-557deda1100c\") " Dec 03 12:37:00 crc kubenswrapper[5052]: I1203 12:37:00.652750 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hmgbn\" (UniqueName: \"kubernetes.io/projected/7e7d9a94-b50b-4065-a178-557deda1100c-kube-api-access-hmgbn\") pod \"7e7d9a94-b50b-4065-a178-557deda1100c\" (UID: \"7e7d9a94-b50b-4065-a178-557deda1100c\") " Dec 03 12:37:00 crc kubenswrapper[5052]: I1203 12:37:00.652782 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/7e7d9a94-b50b-4065-a178-557deda1100c-config-data\") pod \"7e7d9a94-b50b-4065-a178-557deda1100c\" (UID: \"7e7d9a94-b50b-4065-a178-557deda1100c\") " Dec 03 12:37:00 crc kubenswrapper[5052]: I1203 12:37:00.654078 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e7d9a94-b50b-4065-a178-557deda1100c-logs" (OuterVolumeSpecName: "logs") pod "7e7d9a94-b50b-4065-a178-557deda1100c" (UID: "7e7d9a94-b50b-4065-a178-557deda1100c"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:37:00 crc kubenswrapper[5052]: I1203 12:37:00.663889 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7e7d9a94-b50b-4065-a178-557deda1100c-kube-api-access-hmgbn" (OuterVolumeSpecName: "kube-api-access-hmgbn") pod "7e7d9a94-b50b-4065-a178-557deda1100c" (UID: "7e7d9a94-b50b-4065-a178-557deda1100c"). InnerVolumeSpecName "kube-api-access-hmgbn". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:37:00 crc kubenswrapper[5052]: I1203 12:37:00.686223 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7e7d9a94-b50b-4065-a178-557deda1100c-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "7e7d9a94-b50b-4065-a178-557deda1100c" (UID: "7e7d9a94-b50b-4065-a178-557deda1100c"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:37:00 crc kubenswrapper[5052]: I1203 12:37:00.692581 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7e7d9a94-b50b-4065-a178-557deda1100c-config-data" (OuterVolumeSpecName: "config-data") pod "7e7d9a94-b50b-4065-a178-557deda1100c" (UID: "7e7d9a94-b50b-4065-a178-557deda1100c"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:37:00 crc kubenswrapper[5052]: I1203 12:37:00.755568 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/7e7d9a94-b50b-4065-a178-557deda1100c-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:37:00 crc kubenswrapper[5052]: I1203 12:37:00.755810 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/7e7d9a94-b50b-4065-a178-557deda1100c-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:37:00 crc kubenswrapper[5052]: I1203 12:37:00.755821 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-hmgbn\" (UniqueName: \"kubernetes.io/projected/7e7d9a94-b50b-4065-a178-557deda1100c-kube-api-access-hmgbn\") on node \"crc\" DevicePath \"\"" Dec 03 12:37:00 crc kubenswrapper[5052]: I1203 12:37:00.755841 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/7e7d9a94-b50b-4065-a178-557deda1100c-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:37:01 crc kubenswrapper[5052]: I1203 12:37:01.094471 5052 generic.go:334] "Generic (PLEG): container finished" podID="7e7d9a94-b50b-4065-a178-557deda1100c" containerID="2de47cacb469eb9f4ef023ca0e0c9a3f1c57692a0ef8c8f65261831c30d72149" exitCode=0 Dec 03 12:37:01 crc kubenswrapper[5052]: I1203 12:37:01.094566 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"7e7d9a94-b50b-4065-a178-557deda1100c","Type":"ContainerDied","Data":"2de47cacb469eb9f4ef023ca0e0c9a3f1c57692a0ef8c8f65261831c30d72149"} Dec 03 12:37:01 crc kubenswrapper[5052]: I1203 12:37:01.094598 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"7e7d9a94-b50b-4065-a178-557deda1100c","Type":"ContainerDied","Data":"4fcec97c6d61d6ecc7ab69b60c08d2f8ddf98269e8c21e82ffe9d6ebb860e8da"} Dec 03 12:37:01 crc kubenswrapper[5052]: I1203 12:37:01.094617 5052 scope.go:117] "RemoveContainer" containerID="2de47cacb469eb9f4ef023ca0e0c9a3f1c57692a0ef8c8f65261831c30d72149" Dec 03 12:37:01 crc kubenswrapper[5052]: I1203 12:37:01.094742 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:37:01 crc kubenswrapper[5052]: I1203 12:37:01.114156 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"42839582-fefc-4756-933b-d0ba9fb9334c","Type":"ContainerStarted","Data":"0c7c6a8fc3b361c9c2ddb52582cd40de33c720adb452b47945f1cda5a836d730"} Dec 03 12:37:01 crc kubenswrapper[5052]: I1203 12:37:01.114535 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:37:01 crc kubenswrapper[5052]: I1203 12:37:01.117018 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"b0baf287-dc5f-45d0-b09d-324933082b9e","Type":"ContainerStarted","Data":"441e495d6dff11b2b90f0f4528ee17c8d1710a586189a25097a602eff7902fd7"} Dec 03 12:37:01 crc kubenswrapper[5052]: I1203 12:37:01.125988 5052 scope.go:117] "RemoveContainer" containerID="e151b80f440784eeb61f00e0a399f2ccb9b03fb4ee3fee6bac9ffb707dde5f36" Dec 03 12:37:01 crc kubenswrapper[5052]: I1203 12:37:01.148526 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/ceilometer-0" podStartSLOduration=1.302698483 podStartE2EDuration="5.148484036s" podCreationTimestamp="2025-12-03 12:36:56 +0000 UTC" firstStartedPulling="2025-12-03 12:36:56.93508578 +0000 UTC m=+4607.523327544" lastFinishedPulling="2025-12-03 12:37:00.780871333 +0000 UTC m=+4611.369113097" observedRunningTime="2025-12-03 12:37:01.139582631 +0000 UTC m=+4611.727824405" watchObservedRunningTime="2025-12-03 12:37:01.148484036 +0000 UTC m=+4611.736725800" Dec 03 12:37:01 crc kubenswrapper[5052]: I1203 12:37:01.177811 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-scheduler-0" podStartSLOduration=2.177782124 podStartE2EDuration="2.177782124s" podCreationTimestamp="2025-12-03 12:36:59 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:37:01.167034316 +0000 UTC m=+4611.755276080" watchObservedRunningTime="2025-12-03 12:37:01.177782124 +0000 UTC m=+4611.766023888" Dec 03 12:37:01 crc kubenswrapper[5052]: I1203 12:37:01.178675 5052 scope.go:117] "RemoveContainer" containerID="2de47cacb469eb9f4ef023ca0e0c9a3f1c57692a0ef8c8f65261831c30d72149" Dec 03 12:37:01 crc kubenswrapper[5052]: E1203 12:37:01.179609 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2de47cacb469eb9f4ef023ca0e0c9a3f1c57692a0ef8c8f65261831c30d72149\": container with ID starting with 2de47cacb469eb9f4ef023ca0e0c9a3f1c57692a0ef8c8f65261831c30d72149 not found: ID does not exist" containerID="2de47cacb469eb9f4ef023ca0e0c9a3f1c57692a0ef8c8f65261831c30d72149" Dec 03 12:37:01 crc kubenswrapper[5052]: I1203 12:37:01.179656 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2de47cacb469eb9f4ef023ca0e0c9a3f1c57692a0ef8c8f65261831c30d72149"} err="failed to get container status \"2de47cacb469eb9f4ef023ca0e0c9a3f1c57692a0ef8c8f65261831c30d72149\": rpc error: code = NotFound desc = could not find container \"2de47cacb469eb9f4ef023ca0e0c9a3f1c57692a0ef8c8f65261831c30d72149\": container with ID starting with 2de47cacb469eb9f4ef023ca0e0c9a3f1c57692a0ef8c8f65261831c30d72149 not found: ID does not exist" Dec 03 12:37:01 crc kubenswrapper[5052]: I1203 12:37:01.179688 5052 scope.go:117] "RemoveContainer" containerID="e151b80f440784eeb61f00e0a399f2ccb9b03fb4ee3fee6bac9ffb707dde5f36" Dec 03 12:37:01 crc kubenswrapper[5052]: E1203 12:37:01.180015 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e151b80f440784eeb61f00e0a399f2ccb9b03fb4ee3fee6bac9ffb707dde5f36\": container with ID starting with e151b80f440784eeb61f00e0a399f2ccb9b03fb4ee3fee6bac9ffb707dde5f36 not found: ID does not exist" containerID="e151b80f440784eeb61f00e0a399f2ccb9b03fb4ee3fee6bac9ffb707dde5f36" Dec 03 12:37:01 crc kubenswrapper[5052]: I1203 12:37:01.180045 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e151b80f440784eeb61f00e0a399f2ccb9b03fb4ee3fee6bac9ffb707dde5f36"} err="failed to get container status \"e151b80f440784eeb61f00e0a399f2ccb9b03fb4ee3fee6bac9ffb707dde5f36\": rpc error: code = NotFound desc = could not find container \"e151b80f440784eeb61f00e0a399f2ccb9b03fb4ee3fee6bac9ffb707dde5f36\": container with ID starting with e151b80f440784eeb61f00e0a399f2ccb9b03fb4ee3fee6bac9ffb707dde5f36 not found: ID does not exist" Dec 03 12:37:01 crc kubenswrapper[5052]: I1203 12:37:01.208367 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:37:01 crc kubenswrapper[5052]: I1203 12:37:01.220288 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:37:01 crc kubenswrapper[5052]: I1203 12:37:01.235176 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:37:01 crc kubenswrapper[5052]: E1203 12:37:01.235700 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="7e7d9a94-b50b-4065-a178-557deda1100c" containerName="nova-api-log" Dec 03 12:37:01 crc kubenswrapper[5052]: I1203 12:37:01.235724 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="7e7d9a94-b50b-4065-a178-557deda1100c" containerName="nova-api-log" Dec 03 12:37:01 crc kubenswrapper[5052]: E1203 12:37:01.235763 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="7e7d9a94-b50b-4065-a178-557deda1100c" containerName="nova-api-api" Dec 03 12:37:01 crc kubenswrapper[5052]: I1203 12:37:01.235773 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="7e7d9a94-b50b-4065-a178-557deda1100c" containerName="nova-api-api" Dec 03 12:37:01 crc kubenswrapper[5052]: I1203 12:37:01.236051 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="7e7d9a94-b50b-4065-a178-557deda1100c" containerName="nova-api-api" Dec 03 12:37:01 crc kubenswrapper[5052]: I1203 12:37:01.236073 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="7e7d9a94-b50b-4065-a178-557deda1100c" containerName="nova-api-log" Dec 03 12:37:01 crc kubenswrapper[5052]: I1203 12:37:01.237303 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:37:01 crc kubenswrapper[5052]: I1203 12:37:01.241211 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-api-config-data" Dec 03 12:37:01 crc kubenswrapper[5052]: I1203 12:37:01.244697 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:37:01 crc kubenswrapper[5052]: I1203 12:37:01.269376 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/cc1f111d-2478-4099-a14d-988741d7eca4-config-data\") pod \"nova-api-0\" (UID: \"cc1f111d-2478-4099-a14d-988741d7eca4\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:37:01 crc kubenswrapper[5052]: I1203 12:37:01.269473 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/cc1f111d-2478-4099-a14d-988741d7eca4-logs\") pod \"nova-api-0\" (UID: \"cc1f111d-2478-4099-a14d-988741d7eca4\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:37:01 crc kubenswrapper[5052]: I1203 12:37:01.269766 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pkkbx\" (UniqueName: \"kubernetes.io/projected/cc1f111d-2478-4099-a14d-988741d7eca4-kube-api-access-pkkbx\") pod \"nova-api-0\" (UID: \"cc1f111d-2478-4099-a14d-988741d7eca4\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:37:01 crc kubenswrapper[5052]: I1203 12:37:01.269923 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/cc1f111d-2478-4099-a14d-988741d7eca4-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"cc1f111d-2478-4099-a14d-988741d7eca4\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:37:01 crc kubenswrapper[5052]: I1203 12:37:01.376787 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/cc1f111d-2478-4099-a14d-988741d7eca4-config-data\") pod \"nova-api-0\" (UID: \"cc1f111d-2478-4099-a14d-988741d7eca4\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:37:01 crc kubenswrapper[5052]: I1203 12:37:01.377282 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/cc1f111d-2478-4099-a14d-988741d7eca4-logs\") pod \"nova-api-0\" (UID: \"cc1f111d-2478-4099-a14d-988741d7eca4\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:37:01 crc kubenswrapper[5052]: I1203 12:37:01.377576 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-pkkbx\" (UniqueName: \"kubernetes.io/projected/cc1f111d-2478-4099-a14d-988741d7eca4-kube-api-access-pkkbx\") pod \"nova-api-0\" (UID: \"cc1f111d-2478-4099-a14d-988741d7eca4\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:37:01 crc kubenswrapper[5052]: I1203 12:37:01.377814 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/cc1f111d-2478-4099-a14d-988741d7eca4-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"cc1f111d-2478-4099-a14d-988741d7eca4\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:37:01 crc kubenswrapper[5052]: I1203 12:37:01.379997 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/cc1f111d-2478-4099-a14d-988741d7eca4-logs\") pod \"nova-api-0\" (UID: \"cc1f111d-2478-4099-a14d-988741d7eca4\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:37:01 crc kubenswrapper[5052]: I1203 12:37:01.389258 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/cc1f111d-2478-4099-a14d-988741d7eca4-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"cc1f111d-2478-4099-a14d-988741d7eca4\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:37:01 crc kubenswrapper[5052]: I1203 12:37:01.400706 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/cc1f111d-2478-4099-a14d-988741d7eca4-config-data\") pod \"nova-api-0\" (UID: \"cc1f111d-2478-4099-a14d-988741d7eca4\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:37:01 crc kubenswrapper[5052]: I1203 12:37:01.401123 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-pkkbx\" (UniqueName: \"kubernetes.io/projected/cc1f111d-2478-4099-a14d-988741d7eca4-kube-api-access-pkkbx\") pod \"nova-api-0\" (UID: \"cc1f111d-2478-4099-a14d-988741d7eca4\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:37:01 crc kubenswrapper[5052]: I1203 12:37:01.574682 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:37:01 crc kubenswrapper[5052]: I1203 12:37:01.687917 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7e7d9a94-b50b-4065-a178-557deda1100c" path="/var/lib/kubelet/pods/7e7d9a94-b50b-4065-a178-557deda1100c/volumes" Dec 03 12:37:02 crc kubenswrapper[5052]: I1203 12:37:02.057899 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:37:02 crc kubenswrapper[5052]: I1203 12:37:02.132246 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"cc1f111d-2478-4099-a14d-988741d7eca4","Type":"ContainerStarted","Data":"6e33f68e9af21ec2b12cfda08b1e8254316f5e30bae8f061dbc965fd85230ec3"} Dec 03 12:37:02 crc kubenswrapper[5052]: I1203 12:37:02.208965 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/nova-metadata-0" podUID="69502269-3194-440d-aa90-8a9a1821a297" containerName="nova-metadata-metadata" probeResult="failure" output="Get \"https://10.217.1.80:8775/\": read tcp 10.217.0.2:50886->10.217.1.80:8775: read: connection reset by peer" Dec 03 12:37:02 crc kubenswrapper[5052]: I1203 12:37:02.208999 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/nova-metadata-0" podUID="69502269-3194-440d-aa90-8a9a1821a297" containerName="nova-metadata-log" probeResult="failure" output="Get \"https://10.217.1.80:8775/\": read tcp 10.217.0.2:50900->10.217.1.80:8775: read: connection reset by peer" Dec 03 12:37:02 crc kubenswrapper[5052]: I1203 12:37:02.598228 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:37:02 crc kubenswrapper[5052]: I1203 12:37:02.706366 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/69502269-3194-440d-aa90-8a9a1821a297-nova-metadata-tls-certs\") pod \"69502269-3194-440d-aa90-8a9a1821a297\" (UID: \"69502269-3194-440d-aa90-8a9a1821a297\") " Dec 03 12:37:02 crc kubenswrapper[5052]: I1203 12:37:02.706410 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/69502269-3194-440d-aa90-8a9a1821a297-combined-ca-bundle\") pod \"69502269-3194-440d-aa90-8a9a1821a297\" (UID: \"69502269-3194-440d-aa90-8a9a1821a297\") " Dec 03 12:37:02 crc kubenswrapper[5052]: I1203 12:37:02.706529 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/69502269-3194-440d-aa90-8a9a1821a297-config-data\") pod \"69502269-3194-440d-aa90-8a9a1821a297\" (UID: \"69502269-3194-440d-aa90-8a9a1821a297\") " Dec 03 12:37:02 crc kubenswrapper[5052]: I1203 12:37:02.706557 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/69502269-3194-440d-aa90-8a9a1821a297-logs\") pod \"69502269-3194-440d-aa90-8a9a1821a297\" (UID: \"69502269-3194-440d-aa90-8a9a1821a297\") " Dec 03 12:37:02 crc kubenswrapper[5052]: I1203 12:37:02.706585 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bnl48\" (UniqueName: \"kubernetes.io/projected/69502269-3194-440d-aa90-8a9a1821a297-kube-api-access-bnl48\") pod \"69502269-3194-440d-aa90-8a9a1821a297\" (UID: \"69502269-3194-440d-aa90-8a9a1821a297\") " Dec 03 12:37:02 crc kubenswrapper[5052]: I1203 12:37:02.707645 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/69502269-3194-440d-aa90-8a9a1821a297-logs" (OuterVolumeSpecName: "logs") pod "69502269-3194-440d-aa90-8a9a1821a297" (UID: "69502269-3194-440d-aa90-8a9a1821a297"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:37:02 crc kubenswrapper[5052]: I1203 12:37:02.730379 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/69502269-3194-440d-aa90-8a9a1821a297-kube-api-access-bnl48" (OuterVolumeSpecName: "kube-api-access-bnl48") pod "69502269-3194-440d-aa90-8a9a1821a297" (UID: "69502269-3194-440d-aa90-8a9a1821a297"). InnerVolumeSpecName "kube-api-access-bnl48". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:37:02 crc kubenswrapper[5052]: I1203 12:37:02.774718 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/69502269-3194-440d-aa90-8a9a1821a297-config-data" (OuterVolumeSpecName: "config-data") pod "69502269-3194-440d-aa90-8a9a1821a297" (UID: "69502269-3194-440d-aa90-8a9a1821a297"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:37:02 crc kubenswrapper[5052]: I1203 12:37:02.811054 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/69502269-3194-440d-aa90-8a9a1821a297-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:37:02 crc kubenswrapper[5052]: I1203 12:37:02.811095 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/69502269-3194-440d-aa90-8a9a1821a297-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:37:02 crc kubenswrapper[5052]: I1203 12:37:02.811108 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-bnl48\" (UniqueName: \"kubernetes.io/projected/69502269-3194-440d-aa90-8a9a1821a297-kube-api-access-bnl48\") on node \"crc\" DevicePath \"\"" Dec 03 12:37:02 crc kubenswrapper[5052]: I1203 12:37:02.832662 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/69502269-3194-440d-aa90-8a9a1821a297-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "69502269-3194-440d-aa90-8a9a1821a297" (UID: "69502269-3194-440d-aa90-8a9a1821a297"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:37:02 crc kubenswrapper[5052]: I1203 12:37:02.871666 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/69502269-3194-440d-aa90-8a9a1821a297-nova-metadata-tls-certs" (OuterVolumeSpecName: "nova-metadata-tls-certs") pod "69502269-3194-440d-aa90-8a9a1821a297" (UID: "69502269-3194-440d-aa90-8a9a1821a297"). InnerVolumeSpecName "nova-metadata-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:37:02 crc kubenswrapper[5052]: I1203 12:37:02.912726 5052 reconciler_common.go:293] "Volume detached for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/69502269-3194-440d-aa90-8a9a1821a297-nova-metadata-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:37:02 crc kubenswrapper[5052]: I1203 12:37:02.912778 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/69502269-3194-440d-aa90-8a9a1821a297-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:37:03 crc kubenswrapper[5052]: I1203 12:37:03.144385 5052 generic.go:334] "Generic (PLEG): container finished" podID="69502269-3194-440d-aa90-8a9a1821a297" containerID="6340ef46e764a405f61fe84125c42eb8a5422a7ba57320782ecc1f6addbfbe0c" exitCode=0 Dec 03 12:37:03 crc kubenswrapper[5052]: I1203 12:37:03.144474 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:37:03 crc kubenswrapper[5052]: I1203 12:37:03.144501 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"69502269-3194-440d-aa90-8a9a1821a297","Type":"ContainerDied","Data":"6340ef46e764a405f61fe84125c42eb8a5422a7ba57320782ecc1f6addbfbe0c"} Dec 03 12:37:03 crc kubenswrapper[5052]: I1203 12:37:03.144586 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"69502269-3194-440d-aa90-8a9a1821a297","Type":"ContainerDied","Data":"6a2d13b006cb0eb1850f1ce57e0b8be7759b8e5ca66aca71f042c5738a6afa97"} Dec 03 12:37:03 crc kubenswrapper[5052]: I1203 12:37:03.144608 5052 scope.go:117] "RemoveContainer" containerID="6340ef46e764a405f61fe84125c42eb8a5422a7ba57320782ecc1f6addbfbe0c" Dec 03 12:37:03 crc kubenswrapper[5052]: I1203 12:37:03.159404 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"cc1f111d-2478-4099-a14d-988741d7eca4","Type":"ContainerStarted","Data":"8bad61b6aaa957bae99b2ed91564feeb230212df20f792436bb8a05805b9b30a"} Dec 03 12:37:03 crc kubenswrapper[5052]: I1203 12:37:03.159453 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"cc1f111d-2478-4099-a14d-988741d7eca4","Type":"ContainerStarted","Data":"378db679e7cf9cf650c98215b9d41617133014bbdb526a7bdeb0bdb1fee34f07"} Dec 03 12:37:03 crc kubenswrapper[5052]: I1203 12:37:03.192074 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-api-0" podStartSLOduration=2.192054498 podStartE2EDuration="2.192054498s" podCreationTimestamp="2025-12-03 12:37:01 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:37:03.186674484 +0000 UTC m=+4613.774916248" watchObservedRunningTime="2025-12-03 12:37:03.192054498 +0000 UTC m=+4613.780296262" Dec 03 12:37:03 crc kubenswrapper[5052]: I1203 12:37:03.193000 5052 scope.go:117] "RemoveContainer" containerID="8158be457c8bd67dfca9154299916c22a6ad78859653cc60a7a7925603aaf518" Dec 03 12:37:03 crc kubenswrapper[5052]: I1203 12:37:03.237826 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:37:03 crc kubenswrapper[5052]: I1203 12:37:03.249343 5052 scope.go:117] "RemoveContainer" containerID="6340ef46e764a405f61fe84125c42eb8a5422a7ba57320782ecc1f6addbfbe0c" Dec 03 12:37:03 crc kubenswrapper[5052]: E1203 12:37:03.252921 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6340ef46e764a405f61fe84125c42eb8a5422a7ba57320782ecc1f6addbfbe0c\": container with ID starting with 6340ef46e764a405f61fe84125c42eb8a5422a7ba57320782ecc1f6addbfbe0c not found: ID does not exist" containerID="6340ef46e764a405f61fe84125c42eb8a5422a7ba57320782ecc1f6addbfbe0c" Dec 03 12:37:03 crc kubenswrapper[5052]: I1203 12:37:03.252968 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6340ef46e764a405f61fe84125c42eb8a5422a7ba57320782ecc1f6addbfbe0c"} err="failed to get container status \"6340ef46e764a405f61fe84125c42eb8a5422a7ba57320782ecc1f6addbfbe0c\": rpc error: code = NotFound desc = could not find container \"6340ef46e764a405f61fe84125c42eb8a5422a7ba57320782ecc1f6addbfbe0c\": container with ID starting with 6340ef46e764a405f61fe84125c42eb8a5422a7ba57320782ecc1f6addbfbe0c not found: ID does not exist" Dec 03 12:37:03 crc kubenswrapper[5052]: I1203 12:37:03.252994 5052 scope.go:117] "RemoveContainer" containerID="8158be457c8bd67dfca9154299916c22a6ad78859653cc60a7a7925603aaf518" Dec 03 12:37:03 crc kubenswrapper[5052]: I1203 12:37:03.253087 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:37:03 crc kubenswrapper[5052]: E1203 12:37:03.254564 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8158be457c8bd67dfca9154299916c22a6ad78859653cc60a7a7925603aaf518\": container with ID starting with 8158be457c8bd67dfca9154299916c22a6ad78859653cc60a7a7925603aaf518 not found: ID does not exist" containerID="8158be457c8bd67dfca9154299916c22a6ad78859653cc60a7a7925603aaf518" Dec 03 12:37:03 crc kubenswrapper[5052]: I1203 12:37:03.254618 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8158be457c8bd67dfca9154299916c22a6ad78859653cc60a7a7925603aaf518"} err="failed to get container status \"8158be457c8bd67dfca9154299916c22a6ad78859653cc60a7a7925603aaf518\": rpc error: code = NotFound desc = could not find container \"8158be457c8bd67dfca9154299916c22a6ad78859653cc60a7a7925603aaf518\": container with ID starting with 8158be457c8bd67dfca9154299916c22a6ad78859653cc60a7a7925603aaf518 not found: ID does not exist" Dec 03 12:37:03 crc kubenswrapper[5052]: I1203 12:37:03.276520 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:37:03 crc kubenswrapper[5052]: E1203 12:37:03.276948 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="69502269-3194-440d-aa90-8a9a1821a297" containerName="nova-metadata-metadata" Dec 03 12:37:03 crc kubenswrapper[5052]: I1203 12:37:03.276961 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="69502269-3194-440d-aa90-8a9a1821a297" containerName="nova-metadata-metadata" Dec 03 12:37:03 crc kubenswrapper[5052]: E1203 12:37:03.276977 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="69502269-3194-440d-aa90-8a9a1821a297" containerName="nova-metadata-log" Dec 03 12:37:03 crc kubenswrapper[5052]: I1203 12:37:03.276983 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="69502269-3194-440d-aa90-8a9a1821a297" containerName="nova-metadata-log" Dec 03 12:37:03 crc kubenswrapper[5052]: I1203 12:37:03.277177 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="69502269-3194-440d-aa90-8a9a1821a297" containerName="nova-metadata-log" Dec 03 12:37:03 crc kubenswrapper[5052]: I1203 12:37:03.277207 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="69502269-3194-440d-aa90-8a9a1821a297" containerName="nova-metadata-metadata" Dec 03 12:37:03 crc kubenswrapper[5052]: I1203 12:37:03.278161 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:37:03 crc kubenswrapper[5052]: I1203 12:37:03.278320 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:37:03 crc kubenswrapper[5052]: I1203 12:37:03.280325 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-metadata-config-data" Dec 03 12:37:03 crc kubenswrapper[5052]: I1203 12:37:03.281239 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-nova-metadata-internal-svc" Dec 03 12:37:03 crc kubenswrapper[5052]: I1203 12:37:03.320412 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/18ab5c99-e604-43d2-9b49-5f94b526a476-nova-metadata-tls-certs\") pod \"nova-metadata-0\" (UID: \"18ab5c99-e604-43d2-9b49-5f94b526a476\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:37:03 crc kubenswrapper[5052]: I1203 12:37:03.320502 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/18ab5c99-e604-43d2-9b49-5f94b526a476-logs\") pod \"nova-metadata-0\" (UID: \"18ab5c99-e604-43d2-9b49-5f94b526a476\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:37:03 crc kubenswrapper[5052]: I1203 12:37:03.320563 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/18ab5c99-e604-43d2-9b49-5f94b526a476-config-data\") pod \"nova-metadata-0\" (UID: \"18ab5c99-e604-43d2-9b49-5f94b526a476\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:37:03 crc kubenswrapper[5052]: I1203 12:37:03.320634 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-28dkm\" (UniqueName: \"kubernetes.io/projected/18ab5c99-e604-43d2-9b49-5f94b526a476-kube-api-access-28dkm\") pod \"nova-metadata-0\" (UID: \"18ab5c99-e604-43d2-9b49-5f94b526a476\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:37:03 crc kubenswrapper[5052]: I1203 12:37:03.320657 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/18ab5c99-e604-43d2-9b49-5f94b526a476-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"18ab5c99-e604-43d2-9b49-5f94b526a476\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:37:03 crc kubenswrapper[5052]: I1203 12:37:03.422368 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/18ab5c99-e604-43d2-9b49-5f94b526a476-nova-metadata-tls-certs\") pod \"nova-metadata-0\" (UID: \"18ab5c99-e604-43d2-9b49-5f94b526a476\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:37:03 crc kubenswrapper[5052]: I1203 12:37:03.422443 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/18ab5c99-e604-43d2-9b49-5f94b526a476-logs\") pod \"nova-metadata-0\" (UID: \"18ab5c99-e604-43d2-9b49-5f94b526a476\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:37:03 crc kubenswrapper[5052]: I1203 12:37:03.422471 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/18ab5c99-e604-43d2-9b49-5f94b526a476-config-data\") pod \"nova-metadata-0\" (UID: \"18ab5c99-e604-43d2-9b49-5f94b526a476\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:37:03 crc kubenswrapper[5052]: I1203 12:37:03.422528 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-28dkm\" (UniqueName: \"kubernetes.io/projected/18ab5c99-e604-43d2-9b49-5f94b526a476-kube-api-access-28dkm\") pod \"nova-metadata-0\" (UID: \"18ab5c99-e604-43d2-9b49-5f94b526a476\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:37:03 crc kubenswrapper[5052]: I1203 12:37:03.422545 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/18ab5c99-e604-43d2-9b49-5f94b526a476-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"18ab5c99-e604-43d2-9b49-5f94b526a476\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:37:03 crc kubenswrapper[5052]: I1203 12:37:03.423410 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/18ab5c99-e604-43d2-9b49-5f94b526a476-logs\") pod \"nova-metadata-0\" (UID: \"18ab5c99-e604-43d2-9b49-5f94b526a476\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:37:03 crc kubenswrapper[5052]: I1203 12:37:03.623969 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/18ab5c99-e604-43d2-9b49-5f94b526a476-nova-metadata-tls-certs\") pod \"nova-metadata-0\" (UID: \"18ab5c99-e604-43d2-9b49-5f94b526a476\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:37:03 crc kubenswrapper[5052]: I1203 12:37:03.624054 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/18ab5c99-e604-43d2-9b49-5f94b526a476-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"18ab5c99-e604-43d2-9b49-5f94b526a476\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:37:03 crc kubenswrapper[5052]: I1203 12:37:03.623988 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/18ab5c99-e604-43d2-9b49-5f94b526a476-config-data\") pod \"nova-metadata-0\" (UID: \"18ab5c99-e604-43d2-9b49-5f94b526a476\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:37:03 crc kubenswrapper[5052]: I1203 12:37:03.625762 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-28dkm\" (UniqueName: \"kubernetes.io/projected/18ab5c99-e604-43d2-9b49-5f94b526a476-kube-api-access-28dkm\") pod \"nova-metadata-0\" (UID: \"18ab5c99-e604-43d2-9b49-5f94b526a476\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:37:03 crc kubenswrapper[5052]: I1203 12:37:03.685542 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="69502269-3194-440d-aa90-8a9a1821a297" path="/var/lib/kubelet/pods/69502269-3194-440d-aa90-8a9a1821a297/volumes" Dec 03 12:37:03 crc kubenswrapper[5052]: I1203 12:37:03.899116 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:37:04 crc kubenswrapper[5052]: I1203 12:37:04.376578 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:37:04 crc kubenswrapper[5052]: I1203 12:37:04.568464 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:37:05 crc kubenswrapper[5052]: I1203 12:37:05.183176 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"18ab5c99-e604-43d2-9b49-5f94b526a476","Type":"ContainerStarted","Data":"c607f1aa6bd5710d36211e17d6bbe0611dda60c14807926a6aedb78693ce269e"} Dec 03 12:37:05 crc kubenswrapper[5052]: I1203 12:37:05.183477 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"18ab5c99-e604-43d2-9b49-5f94b526a476","Type":"ContainerStarted","Data":"2df29b41503ca0f4f1423f5cc4f05df03f748b38d67ba8937757f85fe2765311"} Dec 03 12:37:05 crc kubenswrapper[5052]: I1203 12:37:05.183493 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"18ab5c99-e604-43d2-9b49-5f94b526a476","Type":"ContainerStarted","Data":"c408720fa0382dab55f15320d95b7781c9a9f7b1b327be3f3e2133182b2da8fb"} Dec 03 12:37:05 crc kubenswrapper[5052]: I1203 12:37:05.204725 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-metadata-0" podStartSLOduration=2.204703157 podStartE2EDuration="2.204703157s" podCreationTimestamp="2025-12-03 12:37:03 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:37:05.201476554 +0000 UTC m=+4615.789718348" watchObservedRunningTime="2025-12-03 12:37:05.204703157 +0000 UTC m=+4615.792944921" Dec 03 12:37:05 crc kubenswrapper[5052]: I1203 12:37:05.878918 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openshift-marketplace/redhat-operators-bdc4s"] Dec 03 12:37:05 crc kubenswrapper[5052]: I1203 12:37:05.880810 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/redhat-operators-bdc4s" Dec 03 12:37:05 crc kubenswrapper[5052]: I1203 12:37:05.896396 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/redhat-operators-bdc4s"] Dec 03 12:37:05 crc kubenswrapper[5052]: I1203 12:37:05.980948 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/22dc0bdb-80fd-4b3c-82a5-f9ec16797c9f-catalog-content\") pod \"redhat-operators-bdc4s\" (UID: \"22dc0bdb-80fd-4b3c-82a5-f9ec16797c9f\") " pod="openshift-marketplace/redhat-operators-bdc4s" Dec 03 12:37:05 crc kubenswrapper[5052]: I1203 12:37:05.981030 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/22dc0bdb-80fd-4b3c-82a5-f9ec16797c9f-utilities\") pod \"redhat-operators-bdc4s\" (UID: \"22dc0bdb-80fd-4b3c-82a5-f9ec16797c9f\") " pod="openshift-marketplace/redhat-operators-bdc4s" Dec 03 12:37:05 crc kubenswrapper[5052]: I1203 12:37:05.981086 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cml8v\" (UniqueName: \"kubernetes.io/projected/22dc0bdb-80fd-4b3c-82a5-f9ec16797c9f-kube-api-access-cml8v\") pod \"redhat-operators-bdc4s\" (UID: \"22dc0bdb-80fd-4b3c-82a5-f9ec16797c9f\") " pod="openshift-marketplace/redhat-operators-bdc4s" Dec 03 12:37:06 crc kubenswrapper[5052]: I1203 12:37:06.083143 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/22dc0bdb-80fd-4b3c-82a5-f9ec16797c9f-catalog-content\") pod \"redhat-operators-bdc4s\" (UID: \"22dc0bdb-80fd-4b3c-82a5-f9ec16797c9f\") " pod="openshift-marketplace/redhat-operators-bdc4s" Dec 03 12:37:06 crc kubenswrapper[5052]: I1203 12:37:06.083221 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/22dc0bdb-80fd-4b3c-82a5-f9ec16797c9f-utilities\") pod \"redhat-operators-bdc4s\" (UID: \"22dc0bdb-80fd-4b3c-82a5-f9ec16797c9f\") " pod="openshift-marketplace/redhat-operators-bdc4s" Dec 03 12:37:06 crc kubenswrapper[5052]: I1203 12:37:06.083247 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-cml8v\" (UniqueName: \"kubernetes.io/projected/22dc0bdb-80fd-4b3c-82a5-f9ec16797c9f-kube-api-access-cml8v\") pod \"redhat-operators-bdc4s\" (UID: \"22dc0bdb-80fd-4b3c-82a5-f9ec16797c9f\") " pod="openshift-marketplace/redhat-operators-bdc4s" Dec 03 12:37:06 crc kubenswrapper[5052]: I1203 12:37:06.083655 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/22dc0bdb-80fd-4b3c-82a5-f9ec16797c9f-catalog-content\") pod \"redhat-operators-bdc4s\" (UID: \"22dc0bdb-80fd-4b3c-82a5-f9ec16797c9f\") " pod="openshift-marketplace/redhat-operators-bdc4s" Dec 03 12:37:06 crc kubenswrapper[5052]: I1203 12:37:06.083903 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/22dc0bdb-80fd-4b3c-82a5-f9ec16797c9f-utilities\") pod \"redhat-operators-bdc4s\" (UID: \"22dc0bdb-80fd-4b3c-82a5-f9ec16797c9f\") " pod="openshift-marketplace/redhat-operators-bdc4s" Dec 03 12:37:06 crc kubenswrapper[5052]: I1203 12:37:06.105203 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-cml8v\" (UniqueName: \"kubernetes.io/projected/22dc0bdb-80fd-4b3c-82a5-f9ec16797c9f-kube-api-access-cml8v\") pod \"redhat-operators-bdc4s\" (UID: \"22dc0bdb-80fd-4b3c-82a5-f9ec16797c9f\") " pod="openshift-marketplace/redhat-operators-bdc4s" Dec 03 12:37:06 crc kubenswrapper[5052]: I1203 12:37:06.204718 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/redhat-operators-bdc4s" Dec 03 12:37:06 crc kubenswrapper[5052]: I1203 12:37:06.677663 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/redhat-operators-bdc4s"] Dec 03 12:37:06 crc kubenswrapper[5052]: W1203 12:37:06.680380 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod22dc0bdb_80fd_4b3c_82a5_f9ec16797c9f.slice/crio-777bfcfa8ed5b6dd2774e52e4e074e531f3c837179bcaa2c77575cf8897f116a WatchSource:0}: Error finding container 777bfcfa8ed5b6dd2774e52e4e074e531f3c837179bcaa2c77575cf8897f116a: Status 404 returned error can't find the container with id 777bfcfa8ed5b6dd2774e52e4e074e531f3c837179bcaa2c77575cf8897f116a Dec 03 12:37:07 crc kubenswrapper[5052]: I1203 12:37:07.204294 5052 generic.go:334] "Generic (PLEG): container finished" podID="22dc0bdb-80fd-4b3c-82a5-f9ec16797c9f" containerID="13abcd4f48d227745f5286025ab1fb8561f6cd358b99c48362ff934c2f1255bc" exitCode=0 Dec 03 12:37:07 crc kubenswrapper[5052]: I1203 12:37:07.204383 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-operators-bdc4s" event={"ID":"22dc0bdb-80fd-4b3c-82a5-f9ec16797c9f","Type":"ContainerDied","Data":"13abcd4f48d227745f5286025ab1fb8561f6cd358b99c48362ff934c2f1255bc"} Dec 03 12:37:07 crc kubenswrapper[5052]: I1203 12:37:07.204624 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-operators-bdc4s" event={"ID":"22dc0bdb-80fd-4b3c-82a5-f9ec16797c9f","Type":"ContainerStarted","Data":"777bfcfa8ed5b6dd2774e52e4e074e531f3c837179bcaa2c77575cf8897f116a"} Dec 03 12:37:07 crc kubenswrapper[5052]: I1203 12:37:07.673676 5052 scope.go:117] "RemoveContainer" containerID="e6edaf188f85ea4f11f487f11cc489c700f25fb42b5ee269f4f496f1e127d772" Dec 03 12:37:08 crc kubenswrapper[5052]: I1203 12:37:08.222878 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" event={"ID":"10c237bc-72e7-4da3-9f2c-b1688395bf6e","Type":"ContainerStarted","Data":"bccf12735e965b0db2889952a34202c267bb425fd32c44f3ef490bba9c0424d8"} Dec 03 12:37:08 crc kubenswrapper[5052]: I1203 12:37:08.238837 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-operators-bdc4s" event={"ID":"22dc0bdb-80fd-4b3c-82a5-f9ec16797c9f","Type":"ContainerStarted","Data":"d4d5c394816fa4cba25c2a585ea7dcbc241dc7600eed59d25cd1f77bb90fc1e9"} Dec 03 12:37:08 crc kubenswrapper[5052]: I1203 12:37:08.899938 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:37:08 crc kubenswrapper[5052]: I1203 12:37:08.900008 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:37:09 crc kubenswrapper[5052]: I1203 12:37:09.567818 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:37:09 crc kubenswrapper[5052]: I1203 12:37:09.659244 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:37:10 crc kubenswrapper[5052]: I1203 12:37:10.292898 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:37:11 crc kubenswrapper[5052]: I1203 12:37:11.573979 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:37:11 crc kubenswrapper[5052]: I1203 12:37:11.576329 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:37:12 crc kubenswrapper[5052]: I1203 12:37:12.657729 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-api-0" podUID="cc1f111d-2478-4099-a14d-988741d7eca4" containerName="nova-api-api" probeResult="failure" output="Get \"http://10.217.1.88:8774/\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Dec 03 12:37:12 crc kubenswrapper[5052]: I1203 12:37:12.657722 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-api-0" podUID="cc1f111d-2478-4099-a14d-988741d7eca4" containerName="nova-api-log" probeResult="failure" output="Get \"http://10.217.1.88:8774/\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Dec 03 12:37:13 crc kubenswrapper[5052]: I1203 12:37:13.295650 5052 generic.go:334] "Generic (PLEG): container finished" podID="22dc0bdb-80fd-4b3c-82a5-f9ec16797c9f" containerID="d4d5c394816fa4cba25c2a585ea7dcbc241dc7600eed59d25cd1f77bb90fc1e9" exitCode=0 Dec 03 12:37:13 crc kubenswrapper[5052]: I1203 12:37:13.295700 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-operators-bdc4s" event={"ID":"22dc0bdb-80fd-4b3c-82a5-f9ec16797c9f","Type":"ContainerDied","Data":"d4d5c394816fa4cba25c2a585ea7dcbc241dc7600eed59d25cd1f77bb90fc1e9"} Dec 03 12:37:13 crc kubenswrapper[5052]: I1203 12:37:13.899887 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:37:13 crc kubenswrapper[5052]: I1203 12:37:13.900071 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:37:14 crc kubenswrapper[5052]: I1203 12:37:14.310680 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-operators-bdc4s" event={"ID":"22dc0bdb-80fd-4b3c-82a5-f9ec16797c9f","Type":"ContainerStarted","Data":"686c781c0528102edc3fc3910fd3255804c9368e2704cafee53621e65e09bfb7"} Dec 03 12:37:14 crc kubenswrapper[5052]: I1203 12:37:14.334053 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-marketplace/redhat-operators-bdc4s" podStartSLOduration=2.769762192 podStartE2EDuration="9.334033219s" podCreationTimestamp="2025-12-03 12:37:05 +0000 UTC" firstStartedPulling="2025-12-03 12:37:07.205845515 +0000 UTC m=+4617.794087279" lastFinishedPulling="2025-12-03 12:37:13.770116542 +0000 UTC m=+4624.358358306" observedRunningTime="2025-12-03 12:37:14.330713984 +0000 UTC m=+4624.918955748" watchObservedRunningTime="2025-12-03 12:37:14.334033219 +0000 UTC m=+4624.922274993" Dec 03 12:37:14 crc kubenswrapper[5052]: I1203 12:37:14.917989 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-metadata-0" podUID="18ab5c99-e604-43d2-9b49-5f94b526a476" containerName="nova-metadata-log" probeResult="failure" output="Get \"https://10.217.1.89:8775/\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Dec 03 12:37:14 crc kubenswrapper[5052]: I1203 12:37:14.918381 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-metadata-0" podUID="18ab5c99-e604-43d2-9b49-5f94b526a476" containerName="nova-metadata-metadata" probeResult="failure" output="Get \"https://10.217.1.89:8775/\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Dec 03 12:37:16 crc kubenswrapper[5052]: I1203 12:37:16.205088 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-marketplace/redhat-operators-bdc4s" Dec 03 12:37:16 crc kubenswrapper[5052]: I1203 12:37:16.205167 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openshift-marketplace/redhat-operators-bdc4s" Dec 03 12:37:17 crc kubenswrapper[5052]: I1203 12:37:17.252442 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openshift-marketplace/redhat-operators-bdc4s" podUID="22dc0bdb-80fd-4b3c-82a5-f9ec16797c9f" containerName="registry-server" probeResult="failure" output=< Dec 03 12:37:17 crc kubenswrapper[5052]: timeout: failed to connect service ":50051" within 1s Dec 03 12:37:17 crc kubenswrapper[5052]: > Dec 03 12:37:21 crc kubenswrapper[5052]: I1203 12:37:21.578967 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:37:21 crc kubenswrapper[5052]: I1203 12:37:21.579805 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:37:21 crc kubenswrapper[5052]: I1203 12:37:21.581915 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:37:21 crc kubenswrapper[5052]: I1203 12:37:21.592681 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:37:22 crc kubenswrapper[5052]: I1203 12:37:22.397240 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:37:22 crc kubenswrapper[5052]: I1203 12:37:22.401270 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:37:23 crc kubenswrapper[5052]: I1203 12:37:23.908199 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:37:23 crc kubenswrapper[5052]: I1203 12:37:23.908733 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:37:23 crc kubenswrapper[5052]: I1203 12:37:23.919790 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:37:23 crc kubenswrapper[5052]: I1203 12:37:23.921338 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:37:24 crc kubenswrapper[5052]: I1203 12:37:24.240161 5052 scope.go:117] "RemoveContainer" containerID="3b528f3a5ecffc67ae3f8e59c8348c99e1798d2e9c1b8ecf277ece85a83c0756" Dec 03 12:37:24 crc kubenswrapper[5052]: I1203 12:37:24.265002 5052 scope.go:117] "RemoveContainer" containerID="da4594e855a05fe7faafd60c45e490f353a2622ba132821e7f56859f0a614ca4" Dec 03 12:37:24 crc kubenswrapper[5052]: I1203 12:37:24.801170 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:37:24 crc kubenswrapper[5052]: I1203 12:37:24.883555 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:37:24 crc kubenswrapper[5052]: I1203 12:37:24.883857 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="42839582-fefc-4756-933b-d0ba9fb9334c" containerName="ceilometer-central-agent" containerID="cri-o://e95078911abc9ff62ef4de1494f5c34277564504f3550e7458213fe28afe676b" gracePeriod=30 Dec 03 12:37:24 crc kubenswrapper[5052]: I1203 12:37:24.883929 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="42839582-fefc-4756-933b-d0ba9fb9334c" containerName="ceilometer-notification-agent" containerID="cri-o://7b420af945e0dc26ceeb21ff59642d3f3e96828f457c6d91a7c64ec8f9111c11" gracePeriod=30 Dec 03 12:37:24 crc kubenswrapper[5052]: I1203 12:37:24.883953 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="42839582-fefc-4756-933b-d0ba9fb9334c" containerName="proxy-httpd" containerID="cri-o://0c7c6a8fc3b361c9c2ddb52582cd40de33c720adb452b47945f1cda5a836d730" gracePeriod=30 Dec 03 12:37:24 crc kubenswrapper[5052]: I1203 12:37:24.883930 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="42839582-fefc-4756-933b-d0ba9fb9334c" containerName="sg-core" containerID="cri-o://b05a5920f0129ccca27e993e2d3d6d889d654dc1612338a0709892206efad711" gracePeriod=30 Dec 03 12:37:24 crc kubenswrapper[5052]: I1203 12:37:24.906979 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/ceilometer-0" podUID="42839582-fefc-4756-933b-d0ba9fb9334c" containerName="proxy-httpd" probeResult="failure" output="HTTP probe failed with statuscode: 503" Dec 03 12:37:25 crc kubenswrapper[5052]: I1203 12:37:25.437830 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"42839582-fefc-4756-933b-d0ba9fb9334c","Type":"ContainerDied","Data":"0c7c6a8fc3b361c9c2ddb52582cd40de33c720adb452b47945f1cda5a836d730"} Dec 03 12:37:25 crc kubenswrapper[5052]: I1203 12:37:25.437756 5052 generic.go:334] "Generic (PLEG): container finished" podID="42839582-fefc-4756-933b-d0ba9fb9334c" containerID="0c7c6a8fc3b361c9c2ddb52582cd40de33c720adb452b47945f1cda5a836d730" exitCode=0 Dec 03 12:37:25 crc kubenswrapper[5052]: I1203 12:37:25.437894 5052 generic.go:334] "Generic (PLEG): container finished" podID="42839582-fefc-4756-933b-d0ba9fb9334c" containerID="b05a5920f0129ccca27e993e2d3d6d889d654dc1612338a0709892206efad711" exitCode=2 Dec 03 12:37:25 crc kubenswrapper[5052]: I1203 12:37:25.437905 5052 generic.go:334] "Generic (PLEG): container finished" podID="42839582-fefc-4756-933b-d0ba9fb9334c" containerID="e95078911abc9ff62ef4de1494f5c34277564504f3550e7458213fe28afe676b" exitCode=0 Dec 03 12:37:25 crc kubenswrapper[5052]: I1203 12:37:25.437936 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"42839582-fefc-4756-933b-d0ba9fb9334c","Type":"ContainerDied","Data":"b05a5920f0129ccca27e993e2d3d6d889d654dc1612338a0709892206efad711"} Dec 03 12:37:25 crc kubenswrapper[5052]: I1203 12:37:25.437958 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"42839582-fefc-4756-933b-d0ba9fb9334c","Type":"ContainerDied","Data":"e95078911abc9ff62ef4de1494f5c34277564504f3550e7458213fe28afe676b"} Dec 03 12:37:25 crc kubenswrapper[5052]: I1203 12:37:25.438127 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-api-0" podUID="cc1f111d-2478-4099-a14d-988741d7eca4" containerName="nova-api-log" containerID="cri-o://378db679e7cf9cf650c98215b9d41617133014bbdb526a7bdeb0bdb1fee34f07" gracePeriod=30 Dec 03 12:37:25 crc kubenswrapper[5052]: I1203 12:37:25.438173 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-api-0" podUID="cc1f111d-2478-4099-a14d-988741d7eca4" containerName="nova-api-api" containerID="cri-o://8bad61b6aaa957bae99b2ed91564feeb230212df20f792436bb8a05805b9b30a" gracePeriod=30 Dec 03 12:37:26 crc kubenswrapper[5052]: I1203 12:37:26.261214 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-marketplace/redhat-operators-bdc4s" Dec 03 12:37:26 crc kubenswrapper[5052]: I1203 12:37:26.309368 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-marketplace/redhat-operators-bdc4s" Dec 03 12:37:26 crc kubenswrapper[5052]: I1203 12:37:26.449229 5052 generic.go:334] "Generic (PLEG): container finished" podID="cc1f111d-2478-4099-a14d-988741d7eca4" containerID="378db679e7cf9cf650c98215b9d41617133014bbdb526a7bdeb0bdb1fee34f07" exitCode=143 Dec 03 12:37:26 crc kubenswrapper[5052]: I1203 12:37:26.449310 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"cc1f111d-2478-4099-a14d-988741d7eca4","Type":"ContainerDied","Data":"378db679e7cf9cf650c98215b9d41617133014bbdb526a7bdeb0bdb1fee34f07"} Dec 03 12:37:26 crc kubenswrapper[5052]: I1203 12:37:26.452463 5052 generic.go:334] "Generic (PLEG): container finished" podID="42839582-fefc-4756-933b-d0ba9fb9334c" containerID="7b420af945e0dc26ceeb21ff59642d3f3e96828f457c6d91a7c64ec8f9111c11" exitCode=0 Dec 03 12:37:26 crc kubenswrapper[5052]: I1203 12:37:26.452535 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"42839582-fefc-4756-933b-d0ba9fb9334c","Type":"ContainerDied","Data":"7b420af945e0dc26ceeb21ff59642d3f3e96828f457c6d91a7c64ec8f9111c11"} Dec 03 12:37:26 crc kubenswrapper[5052]: I1203 12:37:26.465276 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/ceilometer-0" podUID="42839582-fefc-4756-933b-d0ba9fb9334c" containerName="proxy-httpd" probeResult="failure" output="Get \"https://10.217.1.86:3000/\": dial tcp 10.217.1.86:3000: connect: connection refused" Dec 03 12:37:26 crc kubenswrapper[5052]: I1203 12:37:26.503119 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openshift-marketplace/redhat-operators-bdc4s"] Dec 03 12:37:27 crc kubenswrapper[5052]: I1203 12:37:27.461787 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openshift-marketplace/redhat-operators-bdc4s" podUID="22dc0bdb-80fd-4b3c-82a5-f9ec16797c9f" containerName="registry-server" containerID="cri-o://686c781c0528102edc3fc3910fd3255804c9368e2704cafee53621e65e09bfb7" gracePeriod=2 Dec 03 12:37:29 crc kubenswrapper[5052]: I1203 12:37:27.843504 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:37:29 crc kubenswrapper[5052]: I1203 12:37:27.944156 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/42839582-fefc-4756-933b-d0ba9fb9334c-sg-core-conf-yaml\") pod \"42839582-fefc-4756-933b-d0ba9fb9334c\" (UID: \"42839582-fefc-4756-933b-d0ba9fb9334c\") " Dec 03 12:37:29 crc kubenswrapper[5052]: I1203 12:37:27.944268 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/42839582-fefc-4756-933b-d0ba9fb9334c-combined-ca-bundle\") pod \"42839582-fefc-4756-933b-d0ba9fb9334c\" (UID: \"42839582-fefc-4756-933b-d0ba9fb9334c\") " Dec 03 12:37:29 crc kubenswrapper[5052]: I1203 12:37:27.944366 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/42839582-fefc-4756-933b-d0ba9fb9334c-ceilometer-tls-certs\") pod \"42839582-fefc-4756-933b-d0ba9fb9334c\" (UID: \"42839582-fefc-4756-933b-d0ba9fb9334c\") " Dec 03 12:37:29 crc kubenswrapper[5052]: I1203 12:37:27.944428 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/42839582-fefc-4756-933b-d0ba9fb9334c-config-data\") pod \"42839582-fefc-4756-933b-d0ba9fb9334c\" (UID: \"42839582-fefc-4756-933b-d0ba9fb9334c\") " Dec 03 12:37:29 crc kubenswrapper[5052]: I1203 12:37:27.944486 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/42839582-fefc-4756-933b-d0ba9fb9334c-log-httpd\") pod \"42839582-fefc-4756-933b-d0ba9fb9334c\" (UID: \"42839582-fefc-4756-933b-d0ba9fb9334c\") " Dec 03 12:37:29 crc kubenswrapper[5052]: I1203 12:37:27.944563 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/42839582-fefc-4756-933b-d0ba9fb9334c-scripts\") pod \"42839582-fefc-4756-933b-d0ba9fb9334c\" (UID: \"42839582-fefc-4756-933b-d0ba9fb9334c\") " Dec 03 12:37:29 crc kubenswrapper[5052]: I1203 12:37:27.944600 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/42839582-fefc-4756-933b-d0ba9fb9334c-run-httpd\") pod \"42839582-fefc-4756-933b-d0ba9fb9334c\" (UID: \"42839582-fefc-4756-933b-d0ba9fb9334c\") " Dec 03 12:37:29 crc kubenswrapper[5052]: I1203 12:37:27.944627 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4pmqp\" (UniqueName: \"kubernetes.io/projected/42839582-fefc-4756-933b-d0ba9fb9334c-kube-api-access-4pmqp\") pod \"42839582-fefc-4756-933b-d0ba9fb9334c\" (UID: \"42839582-fefc-4756-933b-d0ba9fb9334c\") " Dec 03 12:37:29 crc kubenswrapper[5052]: I1203 12:37:27.945149 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/42839582-fefc-4756-933b-d0ba9fb9334c-run-httpd" (OuterVolumeSpecName: "run-httpd") pod "42839582-fefc-4756-933b-d0ba9fb9334c" (UID: "42839582-fefc-4756-933b-d0ba9fb9334c"). InnerVolumeSpecName "run-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:37:29 crc kubenswrapper[5052]: I1203 12:37:27.945262 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/42839582-fefc-4756-933b-d0ba9fb9334c-log-httpd" (OuterVolumeSpecName: "log-httpd") pod "42839582-fefc-4756-933b-d0ba9fb9334c" (UID: "42839582-fefc-4756-933b-d0ba9fb9334c"). InnerVolumeSpecName "log-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:37:29 crc kubenswrapper[5052]: I1203 12:37:27.949863 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/42839582-fefc-4756-933b-d0ba9fb9334c-scripts" (OuterVolumeSpecName: "scripts") pod "42839582-fefc-4756-933b-d0ba9fb9334c" (UID: "42839582-fefc-4756-933b-d0ba9fb9334c"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:37:29 crc kubenswrapper[5052]: I1203 12:37:27.950898 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/42839582-fefc-4756-933b-d0ba9fb9334c-kube-api-access-4pmqp" (OuterVolumeSpecName: "kube-api-access-4pmqp") pod "42839582-fefc-4756-933b-d0ba9fb9334c" (UID: "42839582-fefc-4756-933b-d0ba9fb9334c"). InnerVolumeSpecName "kube-api-access-4pmqp". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:37:29 crc kubenswrapper[5052]: I1203 12:37:27.988785 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/42839582-fefc-4756-933b-d0ba9fb9334c-sg-core-conf-yaml" (OuterVolumeSpecName: "sg-core-conf-yaml") pod "42839582-fefc-4756-933b-d0ba9fb9334c" (UID: "42839582-fefc-4756-933b-d0ba9fb9334c"). InnerVolumeSpecName "sg-core-conf-yaml". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:37:29 crc kubenswrapper[5052]: I1203 12:37:28.003640 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/42839582-fefc-4756-933b-d0ba9fb9334c-ceilometer-tls-certs" (OuterVolumeSpecName: "ceilometer-tls-certs") pod "42839582-fefc-4756-933b-d0ba9fb9334c" (UID: "42839582-fefc-4756-933b-d0ba9fb9334c"). InnerVolumeSpecName "ceilometer-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:37:29 crc kubenswrapper[5052]: I1203 12:37:28.026077 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/42839582-fefc-4756-933b-d0ba9fb9334c-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "42839582-fefc-4756-933b-d0ba9fb9334c" (UID: "42839582-fefc-4756-933b-d0ba9fb9334c"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:37:29 crc kubenswrapper[5052]: I1203 12:37:28.046991 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/42839582-fefc-4756-933b-d0ba9fb9334c-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:37:29 crc kubenswrapper[5052]: I1203 12:37:28.047022 5052 reconciler_common.go:293] "Volume detached for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/42839582-fefc-4756-933b-d0ba9fb9334c-ceilometer-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:37:29 crc kubenswrapper[5052]: I1203 12:37:28.047063 5052 reconciler_common.go:293] "Volume detached for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/42839582-fefc-4756-933b-d0ba9fb9334c-log-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:37:29 crc kubenswrapper[5052]: I1203 12:37:28.047074 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/42839582-fefc-4756-933b-d0ba9fb9334c-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:37:29 crc kubenswrapper[5052]: I1203 12:37:28.047087 5052 reconciler_common.go:293] "Volume detached for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/42839582-fefc-4756-933b-d0ba9fb9334c-run-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:37:29 crc kubenswrapper[5052]: I1203 12:37:28.047102 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-4pmqp\" (UniqueName: \"kubernetes.io/projected/42839582-fefc-4756-933b-d0ba9fb9334c-kube-api-access-4pmqp\") on node \"crc\" DevicePath \"\"" Dec 03 12:37:29 crc kubenswrapper[5052]: I1203 12:37:28.047114 5052 reconciler_common.go:293] "Volume detached for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/42839582-fefc-4756-933b-d0ba9fb9334c-sg-core-conf-yaml\") on node \"crc\" DevicePath \"\"" Dec 03 12:37:29 crc kubenswrapper[5052]: I1203 12:37:28.048612 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/42839582-fefc-4756-933b-d0ba9fb9334c-config-data" (OuterVolumeSpecName: "config-data") pod "42839582-fefc-4756-933b-d0ba9fb9334c" (UID: "42839582-fefc-4756-933b-d0ba9fb9334c"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:37:29 crc kubenswrapper[5052]: I1203 12:37:28.150038 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/42839582-fefc-4756-933b-d0ba9fb9334c-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:37:29 crc kubenswrapper[5052]: I1203 12:37:28.478598 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:37:29 crc kubenswrapper[5052]: I1203 12:37:28.478554 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"42839582-fefc-4756-933b-d0ba9fb9334c","Type":"ContainerDied","Data":"60e7f9dafeb8dfe87186dd910f6d0baf15a2512f1932425263e48763f3382764"} Dec 03 12:37:29 crc kubenswrapper[5052]: I1203 12:37:28.478777 5052 scope.go:117] "RemoveContainer" containerID="0c7c6a8fc3b361c9c2ddb52582cd40de33c720adb452b47945f1cda5a836d730" Dec 03 12:37:29 crc kubenswrapper[5052]: I1203 12:37:28.482929 5052 generic.go:334] "Generic (PLEG): container finished" podID="22dc0bdb-80fd-4b3c-82a5-f9ec16797c9f" containerID="686c781c0528102edc3fc3910fd3255804c9368e2704cafee53621e65e09bfb7" exitCode=0 Dec 03 12:37:29 crc kubenswrapper[5052]: I1203 12:37:28.482952 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-operators-bdc4s" event={"ID":"22dc0bdb-80fd-4b3c-82a5-f9ec16797c9f","Type":"ContainerDied","Data":"686c781c0528102edc3fc3910fd3255804c9368e2704cafee53621e65e09bfb7"} Dec 03 12:37:29 crc kubenswrapper[5052]: I1203 12:37:28.507721 5052 scope.go:117] "RemoveContainer" containerID="b05a5920f0129ccca27e993e2d3d6d889d654dc1612338a0709892206efad711" Dec 03 12:37:29 crc kubenswrapper[5052]: I1203 12:37:28.511602 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:37:29 crc kubenswrapper[5052]: I1203 12:37:28.526029 5052 scope.go:117] "RemoveContainer" containerID="7b420af945e0dc26ceeb21ff59642d3f3e96828f457c6d91a7c64ec8f9111c11" Dec 03 12:37:29 crc kubenswrapper[5052]: I1203 12:37:28.533619 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:37:29 crc kubenswrapper[5052]: I1203 12:37:28.557656 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:37:29 crc kubenswrapper[5052]: E1203 12:37:28.558167 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="42839582-fefc-4756-933b-d0ba9fb9334c" containerName="ceilometer-central-agent" Dec 03 12:37:29 crc kubenswrapper[5052]: I1203 12:37:28.558181 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="42839582-fefc-4756-933b-d0ba9fb9334c" containerName="ceilometer-central-agent" Dec 03 12:37:29 crc kubenswrapper[5052]: E1203 12:37:28.558204 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="42839582-fefc-4756-933b-d0ba9fb9334c" containerName="sg-core" Dec 03 12:37:29 crc kubenswrapper[5052]: I1203 12:37:28.558210 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="42839582-fefc-4756-933b-d0ba9fb9334c" containerName="sg-core" Dec 03 12:37:29 crc kubenswrapper[5052]: E1203 12:37:28.558227 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="42839582-fefc-4756-933b-d0ba9fb9334c" containerName="proxy-httpd" Dec 03 12:37:29 crc kubenswrapper[5052]: I1203 12:37:28.558233 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="42839582-fefc-4756-933b-d0ba9fb9334c" containerName="proxy-httpd" Dec 03 12:37:29 crc kubenswrapper[5052]: E1203 12:37:28.558255 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="42839582-fefc-4756-933b-d0ba9fb9334c" containerName="ceilometer-notification-agent" Dec 03 12:37:29 crc kubenswrapper[5052]: I1203 12:37:28.558261 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="42839582-fefc-4756-933b-d0ba9fb9334c" containerName="ceilometer-notification-agent" Dec 03 12:37:29 crc kubenswrapper[5052]: I1203 12:37:28.558448 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="42839582-fefc-4756-933b-d0ba9fb9334c" containerName="sg-core" Dec 03 12:37:29 crc kubenswrapper[5052]: I1203 12:37:28.558467 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="42839582-fefc-4756-933b-d0ba9fb9334c" containerName="ceilometer-notification-agent" Dec 03 12:37:29 crc kubenswrapper[5052]: I1203 12:37:28.558481 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="42839582-fefc-4756-933b-d0ba9fb9334c" containerName="proxy-httpd" Dec 03 12:37:29 crc kubenswrapper[5052]: I1203 12:37:28.558496 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="42839582-fefc-4756-933b-d0ba9fb9334c" containerName="ceilometer-central-agent" Dec 03 12:37:29 crc kubenswrapper[5052]: I1203 12:37:28.560644 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:37:29 crc kubenswrapper[5052]: I1203 12:37:28.563168 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-config-data" Dec 03 12:37:29 crc kubenswrapper[5052]: I1203 12:37:28.563318 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-ceilometer-internal-svc" Dec 03 12:37:29 crc kubenswrapper[5052]: I1203 12:37:28.563505 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-scripts" Dec 03 12:37:29 crc kubenswrapper[5052]: I1203 12:37:28.583700 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:37:29 crc kubenswrapper[5052]: I1203 12:37:28.609614 5052 scope.go:117] "RemoveContainer" containerID="e95078911abc9ff62ef4de1494f5c34277564504f3550e7458213fe28afe676b" Dec 03 12:37:29 crc kubenswrapper[5052]: I1203 12:37:28.675552 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/68678c3f-bf25-488f-a422-1bf8586703b3-run-httpd\") pod \"ceilometer-0\" (UID: \"68678c3f-bf25-488f-a422-1bf8586703b3\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:37:29 crc kubenswrapper[5052]: I1203 12:37:28.675638 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/68678c3f-bf25-488f-a422-1bf8586703b3-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"68678c3f-bf25-488f-a422-1bf8586703b3\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:37:29 crc kubenswrapper[5052]: I1203 12:37:28.675707 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/68678c3f-bf25-488f-a422-1bf8586703b3-config-data\") pod \"ceilometer-0\" (UID: \"68678c3f-bf25-488f-a422-1bf8586703b3\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:37:29 crc kubenswrapper[5052]: I1203 12:37:28.675746 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/68678c3f-bf25-488f-a422-1bf8586703b3-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"68678c3f-bf25-488f-a422-1bf8586703b3\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:37:29 crc kubenswrapper[5052]: I1203 12:37:28.675789 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/68678c3f-bf25-488f-a422-1bf8586703b3-scripts\") pod \"ceilometer-0\" (UID: \"68678c3f-bf25-488f-a422-1bf8586703b3\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:37:29 crc kubenswrapper[5052]: I1203 12:37:28.675854 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jdvlz\" (UniqueName: \"kubernetes.io/projected/68678c3f-bf25-488f-a422-1bf8586703b3-kube-api-access-jdvlz\") pod \"ceilometer-0\" (UID: \"68678c3f-bf25-488f-a422-1bf8586703b3\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:37:29 crc kubenswrapper[5052]: I1203 12:37:28.675908 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/68678c3f-bf25-488f-a422-1bf8586703b3-log-httpd\") pod \"ceilometer-0\" (UID: \"68678c3f-bf25-488f-a422-1bf8586703b3\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:37:29 crc kubenswrapper[5052]: I1203 12:37:28.675975 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/68678c3f-bf25-488f-a422-1bf8586703b3-ceilometer-tls-certs\") pod \"ceilometer-0\" (UID: \"68678c3f-bf25-488f-a422-1bf8586703b3\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:37:29 crc kubenswrapper[5052]: I1203 12:37:28.777542 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/68678c3f-bf25-488f-a422-1bf8586703b3-run-httpd\") pod \"ceilometer-0\" (UID: \"68678c3f-bf25-488f-a422-1bf8586703b3\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:37:29 crc kubenswrapper[5052]: I1203 12:37:28.777610 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/68678c3f-bf25-488f-a422-1bf8586703b3-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"68678c3f-bf25-488f-a422-1bf8586703b3\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:37:29 crc kubenswrapper[5052]: I1203 12:37:28.777685 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/68678c3f-bf25-488f-a422-1bf8586703b3-config-data\") pod \"ceilometer-0\" (UID: \"68678c3f-bf25-488f-a422-1bf8586703b3\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:37:29 crc kubenswrapper[5052]: I1203 12:37:28.777737 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/68678c3f-bf25-488f-a422-1bf8586703b3-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"68678c3f-bf25-488f-a422-1bf8586703b3\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:37:29 crc kubenswrapper[5052]: I1203 12:37:28.777761 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/68678c3f-bf25-488f-a422-1bf8586703b3-scripts\") pod \"ceilometer-0\" (UID: \"68678c3f-bf25-488f-a422-1bf8586703b3\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:37:29 crc kubenswrapper[5052]: I1203 12:37:28.777810 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-jdvlz\" (UniqueName: \"kubernetes.io/projected/68678c3f-bf25-488f-a422-1bf8586703b3-kube-api-access-jdvlz\") pod \"ceilometer-0\" (UID: \"68678c3f-bf25-488f-a422-1bf8586703b3\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:37:29 crc kubenswrapper[5052]: I1203 12:37:28.777875 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/68678c3f-bf25-488f-a422-1bf8586703b3-log-httpd\") pod \"ceilometer-0\" (UID: \"68678c3f-bf25-488f-a422-1bf8586703b3\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:37:29 crc kubenswrapper[5052]: I1203 12:37:28.777928 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/68678c3f-bf25-488f-a422-1bf8586703b3-ceilometer-tls-certs\") pod \"ceilometer-0\" (UID: \"68678c3f-bf25-488f-a422-1bf8586703b3\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:37:29 crc kubenswrapper[5052]: I1203 12:37:28.779226 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/68678c3f-bf25-488f-a422-1bf8586703b3-run-httpd\") pod \"ceilometer-0\" (UID: \"68678c3f-bf25-488f-a422-1bf8586703b3\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:37:29 crc kubenswrapper[5052]: I1203 12:37:28.780196 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/68678c3f-bf25-488f-a422-1bf8586703b3-log-httpd\") pod \"ceilometer-0\" (UID: \"68678c3f-bf25-488f-a422-1bf8586703b3\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:37:29 crc kubenswrapper[5052]: I1203 12:37:28.786288 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/68678c3f-bf25-488f-a422-1bf8586703b3-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"68678c3f-bf25-488f-a422-1bf8586703b3\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:37:29 crc kubenswrapper[5052]: I1203 12:37:28.788439 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/68678c3f-bf25-488f-a422-1bf8586703b3-scripts\") pod \"ceilometer-0\" (UID: \"68678c3f-bf25-488f-a422-1bf8586703b3\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:37:29 crc kubenswrapper[5052]: I1203 12:37:28.788963 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/68678c3f-bf25-488f-a422-1bf8586703b3-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"68678c3f-bf25-488f-a422-1bf8586703b3\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:37:29 crc kubenswrapper[5052]: I1203 12:37:28.799437 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/68678c3f-bf25-488f-a422-1bf8586703b3-ceilometer-tls-certs\") pod \"ceilometer-0\" (UID: \"68678c3f-bf25-488f-a422-1bf8586703b3\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:37:29 crc kubenswrapper[5052]: I1203 12:37:28.799491 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/68678c3f-bf25-488f-a422-1bf8586703b3-config-data\") pod \"ceilometer-0\" (UID: \"68678c3f-bf25-488f-a422-1bf8586703b3\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:37:29 crc kubenswrapper[5052]: I1203 12:37:28.811316 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-jdvlz\" (UniqueName: \"kubernetes.io/projected/68678c3f-bf25-488f-a422-1bf8586703b3-kube-api-access-jdvlz\") pod \"ceilometer-0\" (UID: \"68678c3f-bf25-488f-a422-1bf8586703b3\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:37:29 crc kubenswrapper[5052]: I1203 12:37:28.894489 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:37:29 crc kubenswrapper[5052]: I1203 12:37:29.425568 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:37:29 crc kubenswrapper[5052]: I1203 12:37:29.495320 5052 generic.go:334] "Generic (PLEG): container finished" podID="cc1f111d-2478-4099-a14d-988741d7eca4" containerID="8bad61b6aaa957bae99b2ed91564feeb230212df20f792436bb8a05805b9b30a" exitCode=0 Dec 03 12:37:29 crc kubenswrapper[5052]: I1203 12:37:29.495360 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"cc1f111d-2478-4099-a14d-988741d7eca4","Type":"ContainerDied","Data":"8bad61b6aaa957bae99b2ed91564feeb230212df20f792436bb8a05805b9b30a"} Dec 03 12:37:29 crc kubenswrapper[5052]: I1203 12:37:29.694716 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="42839582-fefc-4756-933b-d0ba9fb9334c" path="/var/lib/kubelet/pods/42839582-fefc-4756-933b-d0ba9fb9334c/volumes" Dec 03 12:37:29 crc kubenswrapper[5052]: I1203 12:37:29.971793 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/redhat-operators-bdc4s" Dec 03 12:37:29 crc kubenswrapper[5052]: I1203 12:37:29.976169 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:37:30 crc kubenswrapper[5052]: I1203 12:37:30.116479 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pkkbx\" (UniqueName: \"kubernetes.io/projected/cc1f111d-2478-4099-a14d-988741d7eca4-kube-api-access-pkkbx\") pod \"cc1f111d-2478-4099-a14d-988741d7eca4\" (UID: \"cc1f111d-2478-4099-a14d-988741d7eca4\") " Dec 03 12:37:30 crc kubenswrapper[5052]: I1203 12:37:30.116636 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/22dc0bdb-80fd-4b3c-82a5-f9ec16797c9f-catalog-content\") pod \"22dc0bdb-80fd-4b3c-82a5-f9ec16797c9f\" (UID: \"22dc0bdb-80fd-4b3c-82a5-f9ec16797c9f\") " Dec 03 12:37:30 crc kubenswrapper[5052]: I1203 12:37:30.116700 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/22dc0bdb-80fd-4b3c-82a5-f9ec16797c9f-utilities\") pod \"22dc0bdb-80fd-4b3c-82a5-f9ec16797c9f\" (UID: \"22dc0bdb-80fd-4b3c-82a5-f9ec16797c9f\") " Dec 03 12:37:30 crc kubenswrapper[5052]: I1203 12:37:30.116746 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cml8v\" (UniqueName: \"kubernetes.io/projected/22dc0bdb-80fd-4b3c-82a5-f9ec16797c9f-kube-api-access-cml8v\") pod \"22dc0bdb-80fd-4b3c-82a5-f9ec16797c9f\" (UID: \"22dc0bdb-80fd-4b3c-82a5-f9ec16797c9f\") " Dec 03 12:37:30 crc kubenswrapper[5052]: I1203 12:37:30.116884 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/cc1f111d-2478-4099-a14d-988741d7eca4-config-data\") pod \"cc1f111d-2478-4099-a14d-988741d7eca4\" (UID: \"cc1f111d-2478-4099-a14d-988741d7eca4\") " Dec 03 12:37:30 crc kubenswrapper[5052]: I1203 12:37:30.116970 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/cc1f111d-2478-4099-a14d-988741d7eca4-combined-ca-bundle\") pod \"cc1f111d-2478-4099-a14d-988741d7eca4\" (UID: \"cc1f111d-2478-4099-a14d-988741d7eca4\") " Dec 03 12:37:30 crc kubenswrapper[5052]: I1203 12:37:30.117021 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/cc1f111d-2478-4099-a14d-988741d7eca4-logs\") pod \"cc1f111d-2478-4099-a14d-988741d7eca4\" (UID: \"cc1f111d-2478-4099-a14d-988741d7eca4\") " Dec 03 12:37:30 crc kubenswrapper[5052]: I1203 12:37:30.118478 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cc1f111d-2478-4099-a14d-988741d7eca4-logs" (OuterVolumeSpecName: "logs") pod "cc1f111d-2478-4099-a14d-988741d7eca4" (UID: "cc1f111d-2478-4099-a14d-988741d7eca4"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:37:30 crc kubenswrapper[5052]: I1203 12:37:30.118790 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/22dc0bdb-80fd-4b3c-82a5-f9ec16797c9f-utilities" (OuterVolumeSpecName: "utilities") pod "22dc0bdb-80fd-4b3c-82a5-f9ec16797c9f" (UID: "22dc0bdb-80fd-4b3c-82a5-f9ec16797c9f"). InnerVolumeSpecName "utilities". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:37:30 crc kubenswrapper[5052]: I1203 12:37:30.123618 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/22dc0bdb-80fd-4b3c-82a5-f9ec16797c9f-kube-api-access-cml8v" (OuterVolumeSpecName: "kube-api-access-cml8v") pod "22dc0bdb-80fd-4b3c-82a5-f9ec16797c9f" (UID: "22dc0bdb-80fd-4b3c-82a5-f9ec16797c9f"). InnerVolumeSpecName "kube-api-access-cml8v". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:37:30 crc kubenswrapper[5052]: I1203 12:37:30.126078 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cc1f111d-2478-4099-a14d-988741d7eca4-kube-api-access-pkkbx" (OuterVolumeSpecName: "kube-api-access-pkkbx") pod "cc1f111d-2478-4099-a14d-988741d7eca4" (UID: "cc1f111d-2478-4099-a14d-988741d7eca4"). InnerVolumeSpecName "kube-api-access-pkkbx". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:37:30 crc kubenswrapper[5052]: I1203 12:37:30.147121 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cc1f111d-2478-4099-a14d-988741d7eca4-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "cc1f111d-2478-4099-a14d-988741d7eca4" (UID: "cc1f111d-2478-4099-a14d-988741d7eca4"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:37:30 crc kubenswrapper[5052]: I1203 12:37:30.157904 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cc1f111d-2478-4099-a14d-988741d7eca4-config-data" (OuterVolumeSpecName: "config-data") pod "cc1f111d-2478-4099-a14d-988741d7eca4" (UID: "cc1f111d-2478-4099-a14d-988741d7eca4"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:37:30 crc kubenswrapper[5052]: I1203 12:37:30.218970 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/cc1f111d-2478-4099-a14d-988741d7eca4-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:37:30 crc kubenswrapper[5052]: I1203 12:37:30.219016 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/cc1f111d-2478-4099-a14d-988741d7eca4-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:37:30 crc kubenswrapper[5052]: I1203 12:37:30.219028 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/cc1f111d-2478-4099-a14d-988741d7eca4-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:37:30 crc kubenswrapper[5052]: I1203 12:37:30.219038 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-pkkbx\" (UniqueName: \"kubernetes.io/projected/cc1f111d-2478-4099-a14d-988741d7eca4-kube-api-access-pkkbx\") on node \"crc\" DevicePath \"\"" Dec 03 12:37:30 crc kubenswrapper[5052]: I1203 12:37:30.219049 5052 reconciler_common.go:293] "Volume detached for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/22dc0bdb-80fd-4b3c-82a5-f9ec16797c9f-utilities\") on node \"crc\" DevicePath \"\"" Dec 03 12:37:30 crc kubenswrapper[5052]: I1203 12:37:30.219057 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-cml8v\" (UniqueName: \"kubernetes.io/projected/22dc0bdb-80fd-4b3c-82a5-f9ec16797c9f-kube-api-access-cml8v\") on node \"crc\" DevicePath \"\"" Dec 03 12:37:30 crc kubenswrapper[5052]: I1203 12:37:30.229117 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/22dc0bdb-80fd-4b3c-82a5-f9ec16797c9f-catalog-content" (OuterVolumeSpecName: "catalog-content") pod "22dc0bdb-80fd-4b3c-82a5-f9ec16797c9f" (UID: "22dc0bdb-80fd-4b3c-82a5-f9ec16797c9f"). InnerVolumeSpecName "catalog-content". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:37:30 crc kubenswrapper[5052]: I1203 12:37:30.320429 5052 reconciler_common.go:293] "Volume detached for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/22dc0bdb-80fd-4b3c-82a5-f9ec16797c9f-catalog-content\") on node \"crc\" DevicePath \"\"" Dec 03 12:37:30 crc kubenswrapper[5052]: I1203 12:37:30.503618 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"68678c3f-bf25-488f-a422-1bf8586703b3","Type":"ContainerStarted","Data":"1ca98836ce5312c9ad7a59adc1b315a3e898632296c8e075ea169e1daa505732"} Dec 03 12:37:30 crc kubenswrapper[5052]: I1203 12:37:30.506253 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"cc1f111d-2478-4099-a14d-988741d7eca4","Type":"ContainerDied","Data":"6e33f68e9af21ec2b12cfda08b1e8254316f5e30bae8f061dbc965fd85230ec3"} Dec 03 12:37:30 crc kubenswrapper[5052]: I1203 12:37:30.506297 5052 scope.go:117] "RemoveContainer" containerID="8bad61b6aaa957bae99b2ed91564feeb230212df20f792436bb8a05805b9b30a" Dec 03 12:37:30 crc kubenswrapper[5052]: I1203 12:37:30.506458 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:37:30 crc kubenswrapper[5052]: I1203 12:37:30.513202 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-operators-bdc4s" event={"ID":"22dc0bdb-80fd-4b3c-82a5-f9ec16797c9f","Type":"ContainerDied","Data":"777bfcfa8ed5b6dd2774e52e4e074e531f3c837179bcaa2c77575cf8897f116a"} Dec 03 12:37:30 crc kubenswrapper[5052]: I1203 12:37:30.513321 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/redhat-operators-bdc4s" Dec 03 12:37:30 crc kubenswrapper[5052]: I1203 12:37:30.544197 5052 scope.go:117] "RemoveContainer" containerID="378db679e7cf9cf650c98215b9d41617133014bbdb526a7bdeb0bdb1fee34f07" Dec 03 12:37:30 crc kubenswrapper[5052]: I1203 12:37:30.558345 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:37:30 crc kubenswrapper[5052]: I1203 12:37:30.577489 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:37:30 crc kubenswrapper[5052]: I1203 12:37:30.585700 5052 scope.go:117] "RemoveContainer" containerID="686c781c0528102edc3fc3910fd3255804c9368e2704cafee53621e65e09bfb7" Dec 03 12:37:30 crc kubenswrapper[5052]: I1203 12:37:30.603673 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:37:30 crc kubenswrapper[5052]: E1203 12:37:30.604210 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="cc1f111d-2478-4099-a14d-988741d7eca4" containerName="nova-api-log" Dec 03 12:37:30 crc kubenswrapper[5052]: I1203 12:37:30.604224 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="cc1f111d-2478-4099-a14d-988741d7eca4" containerName="nova-api-log" Dec 03 12:37:30 crc kubenswrapper[5052]: E1203 12:37:30.604243 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="22dc0bdb-80fd-4b3c-82a5-f9ec16797c9f" containerName="extract-utilities" Dec 03 12:37:30 crc kubenswrapper[5052]: I1203 12:37:30.604250 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="22dc0bdb-80fd-4b3c-82a5-f9ec16797c9f" containerName="extract-utilities" Dec 03 12:37:30 crc kubenswrapper[5052]: E1203 12:37:30.604283 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="22dc0bdb-80fd-4b3c-82a5-f9ec16797c9f" containerName="extract-content" Dec 03 12:37:30 crc kubenswrapper[5052]: I1203 12:37:30.604291 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="22dc0bdb-80fd-4b3c-82a5-f9ec16797c9f" containerName="extract-content" Dec 03 12:37:30 crc kubenswrapper[5052]: E1203 12:37:30.604309 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="22dc0bdb-80fd-4b3c-82a5-f9ec16797c9f" containerName="registry-server" Dec 03 12:37:30 crc kubenswrapper[5052]: I1203 12:37:30.604318 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="22dc0bdb-80fd-4b3c-82a5-f9ec16797c9f" containerName="registry-server" Dec 03 12:37:30 crc kubenswrapper[5052]: E1203 12:37:30.604329 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="cc1f111d-2478-4099-a14d-988741d7eca4" containerName="nova-api-api" Dec 03 12:37:30 crc kubenswrapper[5052]: I1203 12:37:30.604335 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="cc1f111d-2478-4099-a14d-988741d7eca4" containerName="nova-api-api" Dec 03 12:37:30 crc kubenswrapper[5052]: I1203 12:37:30.604598 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="22dc0bdb-80fd-4b3c-82a5-f9ec16797c9f" containerName="registry-server" Dec 03 12:37:30 crc kubenswrapper[5052]: I1203 12:37:30.604621 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="cc1f111d-2478-4099-a14d-988741d7eca4" containerName="nova-api-api" Dec 03 12:37:30 crc kubenswrapper[5052]: I1203 12:37:30.604636 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="cc1f111d-2478-4099-a14d-988741d7eca4" containerName="nova-api-log" Dec 03 12:37:30 crc kubenswrapper[5052]: I1203 12:37:30.605753 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:37:30 crc kubenswrapper[5052]: I1203 12:37:30.612036 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-api-config-data" Dec 03 12:37:30 crc kubenswrapper[5052]: I1203 12:37:30.613855 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-nova-public-svc" Dec 03 12:37:30 crc kubenswrapper[5052]: I1203 12:37:30.614022 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-nova-internal-svc" Dec 03 12:37:30 crc kubenswrapper[5052]: I1203 12:37:30.638910 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:37:30 crc kubenswrapper[5052]: I1203 12:37:30.641652 5052 scope.go:117] "RemoveContainer" containerID="d4d5c394816fa4cba25c2a585ea7dcbc241dc7600eed59d25cd1f77bb90fc1e9" Dec 03 12:37:30 crc kubenswrapper[5052]: I1203 12:37:30.651843 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openshift-marketplace/redhat-operators-bdc4s"] Dec 03 12:37:30 crc kubenswrapper[5052]: I1203 12:37:30.668595 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openshift-marketplace/redhat-operators-bdc4s"] Dec 03 12:37:30 crc kubenswrapper[5052]: I1203 12:37:30.696210 5052 scope.go:117] "RemoveContainer" containerID="13abcd4f48d227745f5286025ab1fb8561f6cd358b99c48362ff934c2f1255bc" Dec 03 12:37:30 crc kubenswrapper[5052]: I1203 12:37:30.731858 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/4a200957-c941-47a9-b881-6e9dbc8327b7-config-data\") pod \"nova-api-0\" (UID: \"4a200957-c941-47a9-b881-6e9dbc8327b7\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:37:30 crc kubenswrapper[5052]: I1203 12:37:30.731978 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/4a200957-c941-47a9-b881-6e9dbc8327b7-public-tls-certs\") pod \"nova-api-0\" (UID: \"4a200957-c941-47a9-b881-6e9dbc8327b7\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:37:30 crc kubenswrapper[5052]: I1203 12:37:30.732034 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/4a200957-c941-47a9-b881-6e9dbc8327b7-logs\") pod \"nova-api-0\" (UID: \"4a200957-c941-47a9-b881-6e9dbc8327b7\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:37:30 crc kubenswrapper[5052]: I1203 12:37:30.732063 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xsqww\" (UniqueName: \"kubernetes.io/projected/4a200957-c941-47a9-b881-6e9dbc8327b7-kube-api-access-xsqww\") pod \"nova-api-0\" (UID: \"4a200957-c941-47a9-b881-6e9dbc8327b7\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:37:30 crc kubenswrapper[5052]: I1203 12:37:30.732109 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/4a200957-c941-47a9-b881-6e9dbc8327b7-internal-tls-certs\") pod \"nova-api-0\" (UID: \"4a200957-c941-47a9-b881-6e9dbc8327b7\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:37:30 crc kubenswrapper[5052]: I1203 12:37:30.732133 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4a200957-c941-47a9-b881-6e9dbc8327b7-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"4a200957-c941-47a9-b881-6e9dbc8327b7\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:37:30 crc kubenswrapper[5052]: I1203 12:37:30.834013 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/4a200957-c941-47a9-b881-6e9dbc8327b7-config-data\") pod \"nova-api-0\" (UID: \"4a200957-c941-47a9-b881-6e9dbc8327b7\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:37:30 crc kubenswrapper[5052]: I1203 12:37:30.834430 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/4a200957-c941-47a9-b881-6e9dbc8327b7-public-tls-certs\") pod \"nova-api-0\" (UID: \"4a200957-c941-47a9-b881-6e9dbc8327b7\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:37:30 crc kubenswrapper[5052]: I1203 12:37:30.834467 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/4a200957-c941-47a9-b881-6e9dbc8327b7-logs\") pod \"nova-api-0\" (UID: \"4a200957-c941-47a9-b881-6e9dbc8327b7\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:37:30 crc kubenswrapper[5052]: I1203 12:37:30.834494 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-xsqww\" (UniqueName: \"kubernetes.io/projected/4a200957-c941-47a9-b881-6e9dbc8327b7-kube-api-access-xsqww\") pod \"nova-api-0\" (UID: \"4a200957-c941-47a9-b881-6e9dbc8327b7\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:37:30 crc kubenswrapper[5052]: I1203 12:37:30.834892 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/4a200957-c941-47a9-b881-6e9dbc8327b7-logs\") pod \"nova-api-0\" (UID: \"4a200957-c941-47a9-b881-6e9dbc8327b7\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:37:30 crc kubenswrapper[5052]: I1203 12:37:30.835037 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/4a200957-c941-47a9-b881-6e9dbc8327b7-internal-tls-certs\") pod \"nova-api-0\" (UID: \"4a200957-c941-47a9-b881-6e9dbc8327b7\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:37:30 crc kubenswrapper[5052]: I1203 12:37:30.835070 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4a200957-c941-47a9-b881-6e9dbc8327b7-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"4a200957-c941-47a9-b881-6e9dbc8327b7\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:37:30 crc kubenswrapper[5052]: I1203 12:37:30.838338 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/4a200957-c941-47a9-b881-6e9dbc8327b7-config-data\") pod \"nova-api-0\" (UID: \"4a200957-c941-47a9-b881-6e9dbc8327b7\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:37:30 crc kubenswrapper[5052]: I1203 12:37:30.838741 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/4a200957-c941-47a9-b881-6e9dbc8327b7-public-tls-certs\") pod \"nova-api-0\" (UID: \"4a200957-c941-47a9-b881-6e9dbc8327b7\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:37:30 crc kubenswrapper[5052]: I1203 12:37:30.840456 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/4a200957-c941-47a9-b881-6e9dbc8327b7-internal-tls-certs\") pod \"nova-api-0\" (UID: \"4a200957-c941-47a9-b881-6e9dbc8327b7\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:37:30 crc kubenswrapper[5052]: I1203 12:37:30.841030 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4a200957-c941-47a9-b881-6e9dbc8327b7-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"4a200957-c941-47a9-b881-6e9dbc8327b7\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:37:30 crc kubenswrapper[5052]: I1203 12:37:30.854919 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-xsqww\" (UniqueName: \"kubernetes.io/projected/4a200957-c941-47a9-b881-6e9dbc8327b7-kube-api-access-xsqww\") pod \"nova-api-0\" (UID: \"4a200957-c941-47a9-b881-6e9dbc8327b7\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:37:30 crc kubenswrapper[5052]: I1203 12:37:30.938882 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:37:31 crc kubenswrapper[5052]: I1203 12:37:31.395093 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:37:31 crc kubenswrapper[5052]: I1203 12:37:31.543000 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"4a200957-c941-47a9-b881-6e9dbc8327b7","Type":"ContainerStarted","Data":"21ed13a14f539cb2dc7294be8549c5b0103c4956a74455657743bc81528f5687"} Dec 03 12:37:31 crc kubenswrapper[5052]: I1203 12:37:31.545860 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"68678c3f-bf25-488f-a422-1bf8586703b3","Type":"ContainerStarted","Data":"aece19546c52d53c88212e467860c80e8703242aed2c7ccbcbed127d5a4cd806"} Dec 03 12:37:31 crc kubenswrapper[5052]: I1203 12:37:31.684950 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="22dc0bdb-80fd-4b3c-82a5-f9ec16797c9f" path="/var/lib/kubelet/pods/22dc0bdb-80fd-4b3c-82a5-f9ec16797c9f/volumes" Dec 03 12:37:31 crc kubenswrapper[5052]: I1203 12:37:31.686815 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cc1f111d-2478-4099-a14d-988741d7eca4" path="/var/lib/kubelet/pods/cc1f111d-2478-4099-a14d-988741d7eca4/volumes" Dec 03 12:37:32 crc kubenswrapper[5052]: I1203 12:37:32.558301 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"68678c3f-bf25-488f-a422-1bf8586703b3","Type":"ContainerStarted","Data":"2c6d15a1288e67b74a8c91f85763a3f1a0710e092f088b7defedb0435024e3ee"} Dec 03 12:37:32 crc kubenswrapper[5052]: I1203 12:37:32.558660 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"68678c3f-bf25-488f-a422-1bf8586703b3","Type":"ContainerStarted","Data":"75aabd113fe3f8c31645077c72c336a2d111ebe7269663fa192668bdab05f74c"} Dec 03 12:37:32 crc kubenswrapper[5052]: I1203 12:37:32.560758 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"4a200957-c941-47a9-b881-6e9dbc8327b7","Type":"ContainerStarted","Data":"c489ec06ee93875a69927e3ababa6ff83e2ec14d6d1790c0dd006a349910b023"} Dec 03 12:37:32 crc kubenswrapper[5052]: I1203 12:37:32.560791 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"4a200957-c941-47a9-b881-6e9dbc8327b7","Type":"ContainerStarted","Data":"767ce5808c23166231a144b01e05e10cc1fe868247442d07a0b8b9a42c438b78"} Dec 03 12:37:32 crc kubenswrapper[5052]: I1203 12:37:32.584290 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-api-0" podStartSLOduration=2.584272024 podStartE2EDuration="2.584272024s" podCreationTimestamp="2025-12-03 12:37:30 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:37:32.578178739 +0000 UTC m=+4643.166420533" watchObservedRunningTime="2025-12-03 12:37:32.584272024 +0000 UTC m=+4643.172513788" Dec 03 12:37:34 crc kubenswrapper[5052]: I1203 12:37:34.609048 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"68678c3f-bf25-488f-a422-1bf8586703b3","Type":"ContainerStarted","Data":"a50f4e422eaeb77c3e3ddb88c72b4b83fabdf60cb1ad6b1d7fbca14f74ef6cc3"} Dec 03 12:37:34 crc kubenswrapper[5052]: I1203 12:37:34.609580 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:37:34 crc kubenswrapper[5052]: I1203 12:37:34.672187 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/ceilometer-0" podStartSLOduration=2.694567222 podStartE2EDuration="6.672164414s" podCreationTimestamp="2025-12-03 12:37:28 +0000 UTC" firstStartedPulling="2025-12-03 12:37:29.862873417 +0000 UTC m=+4640.451115181" lastFinishedPulling="2025-12-03 12:37:33.840470609 +0000 UTC m=+4644.428712373" observedRunningTime="2025-12-03 12:37:34.662891579 +0000 UTC m=+4645.251133343" watchObservedRunningTime="2025-12-03 12:37:34.672164414 +0000 UTC m=+4645.260406178" Dec 03 12:37:40 crc kubenswrapper[5052]: I1203 12:37:40.939979 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:37:40 crc kubenswrapper[5052]: I1203 12:37:40.941283 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:37:41 crc kubenswrapper[5052]: I1203 12:37:41.951769 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-api-0" podUID="4a200957-c941-47a9-b881-6e9dbc8327b7" containerName="nova-api-api" probeResult="failure" output="Get \"https://10.217.1.92:8774/\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Dec 03 12:37:41 crc kubenswrapper[5052]: I1203 12:37:41.951790 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-api-0" podUID="4a200957-c941-47a9-b881-6e9dbc8327b7" containerName="nova-api-log" probeResult="failure" output="Get \"https://10.217.1.92:8774/\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Dec 03 12:37:50 crc kubenswrapper[5052]: I1203 12:37:50.946492 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:37:50 crc kubenswrapper[5052]: I1203 12:37:50.946996 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:37:50 crc kubenswrapper[5052]: I1203 12:37:50.948737 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:37:50 crc kubenswrapper[5052]: I1203 12:37:50.948767 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:37:50 crc kubenswrapper[5052]: I1203 12:37:50.953678 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:37:50 crc kubenswrapper[5052]: I1203 12:37:50.953967 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:37:58 crc kubenswrapper[5052]: I1203 12:37:58.906219 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:38:23 crc kubenswrapper[5052]: E1203 12:38:23.770012 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-galera-openstack-svc: secret "cert-galera-openstack-svc" not found Dec 03 12:38:23 crc kubenswrapper[5052]: E1203 12:38:23.770692 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/01fb7120-3521-4969-a2c4-b2637e0e2414-galera-tls-certs podName:01fb7120-3521-4969-a2c4-b2637e0e2414 nodeName:}" failed. No retries permitted until 2025-12-03 12:38:24.270669282 +0000 UTC m=+4694.858911066 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "galera-tls-certs" (UniqueName: "kubernetes.io/secret/01fb7120-3521-4969-a2c4-b2637e0e2414-galera-tls-certs") pod "openstack-galera-0" (UID: "01fb7120-3521-4969-a2c4-b2637e0e2414") : secret "cert-galera-openstack-svc" not found Dec 03 12:38:24 crc kubenswrapper[5052]: E1203 12:38:24.279128 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-galera-openstack-svc: secret "cert-galera-openstack-svc" not found Dec 03 12:38:24 crc kubenswrapper[5052]: E1203 12:38:24.279270 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/01fb7120-3521-4969-a2c4-b2637e0e2414-galera-tls-certs podName:01fb7120-3521-4969-a2c4-b2637e0e2414 nodeName:}" failed. No retries permitted until 2025-12-03 12:38:25.279243556 +0000 UTC m=+4695.867485320 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "galera-tls-certs" (UniqueName: "kubernetes.io/secret/01fb7120-3521-4969-a2c4-b2637e0e2414-galera-tls-certs") pod "openstack-galera-0" (UID: "01fb7120-3521-4969-a2c4-b2637e0e2414") : secret "cert-galera-openstack-svc" not found Dec 03 12:38:25 crc kubenswrapper[5052]: E1203 12:38:25.294207 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-galera-openstack-svc: secret "cert-galera-openstack-svc" not found Dec 03 12:38:25 crc kubenswrapper[5052]: E1203 12:38:25.294648 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/01fb7120-3521-4969-a2c4-b2637e0e2414-galera-tls-certs podName:01fb7120-3521-4969-a2c4-b2637e0e2414 nodeName:}" failed. No retries permitted until 2025-12-03 12:38:27.294623524 +0000 UTC m=+4697.882865338 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "galera-tls-certs" (UniqueName: "kubernetes.io/secret/01fb7120-3521-4969-a2c4-b2637e0e2414-galera-tls-certs") pod "openstack-galera-0" (UID: "01fb7120-3521-4969-a2c4-b2637e0e2414") : secret "cert-galera-openstack-svc" not found Dec 03 12:38:25 crc kubenswrapper[5052]: I1203 12:38:25.693682 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/barbican-api-744cb8cb64-cqprx"] Dec 03 12:38:25 crc kubenswrapper[5052]: I1203 12:38:25.695382 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-api-744cb8cb64-cqprx" Dec 03 12:38:25 crc kubenswrapper[5052]: I1203 12:38:25.716626 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-api-744cb8cb64-cqprx"] Dec 03 12:38:25 crc kubenswrapper[5052]: I1203 12:38:25.806146 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/96b339d1-2cd9-4285-ad3c-abcc5e733a90-logs\") pod \"barbican-api-744cb8cb64-cqprx\" (UID: \"96b339d1-2cd9-4285-ad3c-abcc5e733a90\") " pod="openstack-kuttl-tests/barbican-api-744cb8cb64-cqprx" Dec 03 12:38:25 crc kubenswrapper[5052]: I1203 12:38:25.806265 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-86dr2\" (UniqueName: \"kubernetes.io/projected/96b339d1-2cd9-4285-ad3c-abcc5e733a90-kube-api-access-86dr2\") pod \"barbican-api-744cb8cb64-cqprx\" (UID: \"96b339d1-2cd9-4285-ad3c-abcc5e733a90\") " pod="openstack-kuttl-tests/barbican-api-744cb8cb64-cqprx" Dec 03 12:38:25 crc kubenswrapper[5052]: I1203 12:38:25.806298 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/96b339d1-2cd9-4285-ad3c-abcc5e733a90-public-tls-certs\") pod \"barbican-api-744cb8cb64-cqprx\" (UID: \"96b339d1-2cd9-4285-ad3c-abcc5e733a90\") " pod="openstack-kuttl-tests/barbican-api-744cb8cb64-cqprx" Dec 03 12:38:25 crc kubenswrapper[5052]: I1203 12:38:25.806393 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/96b339d1-2cd9-4285-ad3c-abcc5e733a90-internal-tls-certs\") pod \"barbican-api-744cb8cb64-cqprx\" (UID: \"96b339d1-2cd9-4285-ad3c-abcc5e733a90\") " pod="openstack-kuttl-tests/barbican-api-744cb8cb64-cqprx" Dec 03 12:38:25 crc kubenswrapper[5052]: I1203 12:38:25.806496 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/96b339d1-2cd9-4285-ad3c-abcc5e733a90-combined-ca-bundle\") pod \"barbican-api-744cb8cb64-cqprx\" (UID: \"96b339d1-2cd9-4285-ad3c-abcc5e733a90\") " pod="openstack-kuttl-tests/barbican-api-744cb8cb64-cqprx" Dec 03 12:38:25 crc kubenswrapper[5052]: I1203 12:38:25.806556 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/96b339d1-2cd9-4285-ad3c-abcc5e733a90-config-data\") pod \"barbican-api-744cb8cb64-cqprx\" (UID: \"96b339d1-2cd9-4285-ad3c-abcc5e733a90\") " pod="openstack-kuttl-tests/barbican-api-744cb8cb64-cqprx" Dec 03 12:38:25 crc kubenswrapper[5052]: I1203 12:38:25.806595 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/96b339d1-2cd9-4285-ad3c-abcc5e733a90-config-data-custom\") pod \"barbican-api-744cb8cb64-cqprx\" (UID: \"96b339d1-2cd9-4285-ad3c-abcc5e733a90\") " pod="openstack-kuttl-tests/barbican-api-744cb8cb64-cqprx" Dec 03 12:38:25 crc kubenswrapper[5052]: E1203 12:38:25.806729 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-kube-state-metrics-svc: secret "cert-kube-state-metrics-svc" not found Dec 03 12:38:25 crc kubenswrapper[5052]: E1203 12:38:25.806797 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b7defe22-28b4-4fec-8fe5-ebde8e658c6b-kube-state-metrics-tls-certs podName:b7defe22-28b4-4fec-8fe5-ebde8e658c6b nodeName:}" failed. No retries permitted until 2025-12-03 12:38:26.306778751 +0000 UTC m=+4696.895020515 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls-certs" (UniqueName: "kubernetes.io/secret/b7defe22-28b4-4fec-8fe5-ebde8e658c6b-kube-state-metrics-tls-certs") pod "kube-state-metrics-0" (UID: "b7defe22-28b4-4fec-8fe5-ebde8e658c6b") : secret "cert-kube-state-metrics-svc" not found Dec 03 12:38:25 crc kubenswrapper[5052]: I1203 12:38:25.907987 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/96b339d1-2cd9-4285-ad3c-abcc5e733a90-combined-ca-bundle\") pod \"barbican-api-744cb8cb64-cqprx\" (UID: \"96b339d1-2cd9-4285-ad3c-abcc5e733a90\") " pod="openstack-kuttl-tests/barbican-api-744cb8cb64-cqprx" Dec 03 12:38:25 crc kubenswrapper[5052]: I1203 12:38:25.908074 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/96b339d1-2cd9-4285-ad3c-abcc5e733a90-config-data\") pod \"barbican-api-744cb8cb64-cqprx\" (UID: \"96b339d1-2cd9-4285-ad3c-abcc5e733a90\") " pod="openstack-kuttl-tests/barbican-api-744cb8cb64-cqprx" Dec 03 12:38:25 crc kubenswrapper[5052]: I1203 12:38:25.908118 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/96b339d1-2cd9-4285-ad3c-abcc5e733a90-config-data-custom\") pod \"barbican-api-744cb8cb64-cqprx\" (UID: \"96b339d1-2cd9-4285-ad3c-abcc5e733a90\") " pod="openstack-kuttl-tests/barbican-api-744cb8cb64-cqprx" Dec 03 12:38:25 crc kubenswrapper[5052]: I1203 12:38:25.908163 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/96b339d1-2cd9-4285-ad3c-abcc5e733a90-logs\") pod \"barbican-api-744cb8cb64-cqprx\" (UID: \"96b339d1-2cd9-4285-ad3c-abcc5e733a90\") " pod="openstack-kuttl-tests/barbican-api-744cb8cb64-cqprx" Dec 03 12:38:25 crc kubenswrapper[5052]: I1203 12:38:25.908227 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-86dr2\" (UniqueName: \"kubernetes.io/projected/96b339d1-2cd9-4285-ad3c-abcc5e733a90-kube-api-access-86dr2\") pod \"barbican-api-744cb8cb64-cqprx\" (UID: \"96b339d1-2cd9-4285-ad3c-abcc5e733a90\") " pod="openstack-kuttl-tests/barbican-api-744cb8cb64-cqprx" Dec 03 12:38:25 crc kubenswrapper[5052]: I1203 12:38:25.908262 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/96b339d1-2cd9-4285-ad3c-abcc5e733a90-public-tls-certs\") pod \"barbican-api-744cb8cb64-cqprx\" (UID: \"96b339d1-2cd9-4285-ad3c-abcc5e733a90\") " pod="openstack-kuttl-tests/barbican-api-744cb8cb64-cqprx" Dec 03 12:38:25 crc kubenswrapper[5052]: I1203 12:38:25.908331 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/96b339d1-2cd9-4285-ad3c-abcc5e733a90-internal-tls-certs\") pod \"barbican-api-744cb8cb64-cqprx\" (UID: \"96b339d1-2cd9-4285-ad3c-abcc5e733a90\") " pod="openstack-kuttl-tests/barbican-api-744cb8cb64-cqprx" Dec 03 12:38:25 crc kubenswrapper[5052]: I1203 12:38:25.908708 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/96b339d1-2cd9-4285-ad3c-abcc5e733a90-logs\") pod \"barbican-api-744cb8cb64-cqprx\" (UID: \"96b339d1-2cd9-4285-ad3c-abcc5e733a90\") " pod="openstack-kuttl-tests/barbican-api-744cb8cb64-cqprx" Dec 03 12:38:26 crc kubenswrapper[5052]: E1203 12:38:26.315607 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-kube-state-metrics-svc: secret "cert-kube-state-metrics-svc" not found Dec 03 12:38:26 crc kubenswrapper[5052]: E1203 12:38:26.315685 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b7defe22-28b4-4fec-8fe5-ebde8e658c6b-kube-state-metrics-tls-certs podName:b7defe22-28b4-4fec-8fe5-ebde8e658c6b nodeName:}" failed. No retries permitted until 2025-12-03 12:38:27.315668664 +0000 UTC m=+4697.903910428 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls-certs" (UniqueName: "kubernetes.io/secret/b7defe22-28b4-4fec-8fe5-ebde8e658c6b-kube-state-metrics-tls-certs") pod "kube-state-metrics-0" (UID: "b7defe22-28b4-4fec-8fe5-ebde8e658c6b") : secret "cert-kube-state-metrics-svc" not found Dec 03 12:38:26 crc kubenswrapper[5052]: I1203 12:38:26.323752 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/96b339d1-2cd9-4285-ad3c-abcc5e733a90-public-tls-certs\") pod \"barbican-api-744cb8cb64-cqprx\" (UID: \"96b339d1-2cd9-4285-ad3c-abcc5e733a90\") " pod="openstack-kuttl-tests/barbican-api-744cb8cb64-cqprx" Dec 03 12:38:26 crc kubenswrapper[5052]: I1203 12:38:26.324076 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/96b339d1-2cd9-4285-ad3c-abcc5e733a90-combined-ca-bundle\") pod \"barbican-api-744cb8cb64-cqprx\" (UID: \"96b339d1-2cd9-4285-ad3c-abcc5e733a90\") " pod="openstack-kuttl-tests/barbican-api-744cb8cb64-cqprx" Dec 03 12:38:26 crc kubenswrapper[5052]: I1203 12:38:26.324144 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/96b339d1-2cd9-4285-ad3c-abcc5e733a90-config-data-custom\") pod \"barbican-api-744cb8cb64-cqprx\" (UID: \"96b339d1-2cd9-4285-ad3c-abcc5e733a90\") " pod="openstack-kuttl-tests/barbican-api-744cb8cb64-cqprx" Dec 03 12:38:26 crc kubenswrapper[5052]: I1203 12:38:26.324736 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/96b339d1-2cd9-4285-ad3c-abcc5e733a90-config-data\") pod \"barbican-api-744cb8cb64-cqprx\" (UID: \"96b339d1-2cd9-4285-ad3c-abcc5e733a90\") " pod="openstack-kuttl-tests/barbican-api-744cb8cb64-cqprx" Dec 03 12:38:26 crc kubenswrapper[5052]: I1203 12:38:26.324883 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/96b339d1-2cd9-4285-ad3c-abcc5e733a90-internal-tls-certs\") pod \"barbican-api-744cb8cb64-cqprx\" (UID: \"96b339d1-2cd9-4285-ad3c-abcc5e733a90\") " pod="openstack-kuttl-tests/barbican-api-744cb8cb64-cqprx" Dec 03 12:38:26 crc kubenswrapper[5052]: I1203 12:38:26.325378 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-86dr2\" (UniqueName: \"kubernetes.io/projected/96b339d1-2cd9-4285-ad3c-abcc5e733a90-kube-api-access-86dr2\") pod \"barbican-api-744cb8cb64-cqprx\" (UID: \"96b339d1-2cd9-4285-ad3c-abcc5e733a90\") " pod="openstack-kuttl-tests/barbican-api-744cb8cb64-cqprx" Dec 03 12:38:26 crc kubenswrapper[5052]: I1203 12:38:26.616927 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-api-744cb8cb64-cqprx" Dec 03 12:38:27 crc kubenswrapper[5052]: I1203 12:38:27.067503 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-api-744cb8cb64-cqprx"] Dec 03 12:38:27 crc kubenswrapper[5052]: I1203 12:38:27.151203 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-744cb8cb64-cqprx" event={"ID":"96b339d1-2cd9-4285-ad3c-abcc5e733a90","Type":"ContainerStarted","Data":"a39a960bfea3b3352f097e277937b02c0265a10fe75d9cc345b5e6a304e5b370"} Dec 03 12:38:27 crc kubenswrapper[5052]: E1203 12:38:27.334390 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-kube-state-metrics-svc: secret "cert-kube-state-metrics-svc" not found Dec 03 12:38:27 crc kubenswrapper[5052]: E1203 12:38:27.334426 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-galera-openstack-svc: secret "cert-galera-openstack-svc" not found Dec 03 12:38:27 crc kubenswrapper[5052]: E1203 12:38:27.334468 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b7defe22-28b4-4fec-8fe5-ebde8e658c6b-kube-state-metrics-tls-certs podName:b7defe22-28b4-4fec-8fe5-ebde8e658c6b nodeName:}" failed. No retries permitted until 2025-12-03 12:38:29.33444798 +0000 UTC m=+4699.922689744 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls-certs" (UniqueName: "kubernetes.io/secret/b7defe22-28b4-4fec-8fe5-ebde8e658c6b-kube-state-metrics-tls-certs") pod "kube-state-metrics-0" (UID: "b7defe22-28b4-4fec-8fe5-ebde8e658c6b") : secret "cert-kube-state-metrics-svc" not found Dec 03 12:38:27 crc kubenswrapper[5052]: E1203 12:38:27.334495 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/01fb7120-3521-4969-a2c4-b2637e0e2414-galera-tls-certs podName:01fb7120-3521-4969-a2c4-b2637e0e2414 nodeName:}" failed. No retries permitted until 2025-12-03 12:38:31.334482791 +0000 UTC m=+4701.922724555 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "galera-tls-certs" (UniqueName: "kubernetes.io/secret/01fb7120-3521-4969-a2c4-b2637e0e2414-galera-tls-certs") pod "openstack-galera-0" (UID: "01fb7120-3521-4969-a2c4-b2637e0e2414") : secret "cert-galera-openstack-svc" not found Dec 03 12:38:27 crc kubenswrapper[5052]: I1203 12:38:27.855210 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 12:38:27 crc kubenswrapper[5052]: I1203 12:38:27.856001 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/cinder-scheduler-0" podUID="0a42958a-1720-45c8-a4d9-d38a24a246d5" containerName="probe" containerID="cri-o://bf2ad3eb69b0929f58698c3117f643b29a34469bb0e38c60eb24ebaa0a6a5ea1" gracePeriod=30 Dec 03 12:38:27 crc kubenswrapper[5052]: I1203 12:38:27.855861 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/cinder-scheduler-0" podUID="0a42958a-1720-45c8-a4d9-d38a24a246d5" containerName="cinder-scheduler" containerID="cri-o://57014359344dd7eed908e5209d6cd12ff8506d0d739a8eaa92efff47d08930e9" gracePeriod=30 Dec 03 12:38:27 crc kubenswrapper[5052]: I1203 12:38:27.875326 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/barbican-worker-7bd785845-99jqj"] Dec 03 12:38:27 crc kubenswrapper[5052]: I1203 12:38:27.877101 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-worker-7bd785845-99jqj" Dec 03 12:38:27 crc kubenswrapper[5052]: I1203 12:38:27.899409 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-worker-7bd785845-99jqj"] Dec 03 12:38:27 crc kubenswrapper[5052]: I1203 12:38:27.920284 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/barbican-keystone-listener-74b68b767b-8zpnm"] Dec 03 12:38:27 crc kubenswrapper[5052]: I1203 12:38:27.922120 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-keystone-listener-74b68b767b-8zpnm" Dec 03 12:38:27 crc kubenswrapper[5052]: I1203 12:38:27.960402 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-keystone-listener-74b68b767b-8zpnm"] Dec 03 12:38:28 crc kubenswrapper[5052]: I1203 12:38:28.001117 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-api-744cb8cb64-cqprx"] Dec 03 12:38:28 crc kubenswrapper[5052]: I1203 12:38:28.026006 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/barbican-api-9bbb48f5b-4tfjv"] Dec 03 12:38:28 crc kubenswrapper[5052]: I1203 12:38:28.028340 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-api-9bbb48f5b-4tfjv" Dec 03 12:38:28 crc kubenswrapper[5052]: I1203 12:38:28.046554 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-api-9bbb48f5b-4tfjv"] Dec 03 12:38:28 crc kubenswrapper[5052]: I1203 12:38:28.047604 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/18a62c9b-4032-4639-be99-3520b515313b-combined-ca-bundle\") pod \"barbican-worker-7bd785845-99jqj\" (UID: \"18a62c9b-4032-4639-be99-3520b515313b\") " pod="openstack-kuttl-tests/barbican-worker-7bd785845-99jqj" Dec 03 12:38:28 crc kubenswrapper[5052]: I1203 12:38:28.047652 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a998e1b6-f576-4463-b5ba-9f539f6939ee-config-data\") pod \"barbican-api-9bbb48f5b-4tfjv\" (UID: \"a998e1b6-f576-4463-b5ba-9f539f6939ee\") " pod="openstack-kuttl-tests/barbican-api-9bbb48f5b-4tfjv" Dec 03 12:38:28 crc kubenswrapper[5052]: I1203 12:38:28.047715 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a998e1b6-f576-4463-b5ba-9f539f6939ee-combined-ca-bundle\") pod \"barbican-api-9bbb48f5b-4tfjv\" (UID: \"a998e1b6-f576-4463-b5ba-9f539f6939ee\") " pod="openstack-kuttl-tests/barbican-api-9bbb48f5b-4tfjv" Dec 03 12:38:28 crc kubenswrapper[5052]: I1203 12:38:28.047750 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hw2xp\" (UniqueName: \"kubernetes.io/projected/18a62c9b-4032-4639-be99-3520b515313b-kube-api-access-hw2xp\") pod \"barbican-worker-7bd785845-99jqj\" (UID: \"18a62c9b-4032-4639-be99-3520b515313b\") " pod="openstack-kuttl-tests/barbican-worker-7bd785845-99jqj" Dec 03 12:38:28 crc kubenswrapper[5052]: I1203 12:38:28.047862 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/18a62c9b-4032-4639-be99-3520b515313b-config-data\") pod \"barbican-worker-7bd785845-99jqj\" (UID: \"18a62c9b-4032-4639-be99-3520b515313b\") " pod="openstack-kuttl-tests/barbican-worker-7bd785845-99jqj" Dec 03 12:38:28 crc kubenswrapper[5052]: I1203 12:38:28.047918 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/bc0b4316-9c87-4c72-bc06-b4b526c2070e-config-data\") pod \"barbican-keystone-listener-74b68b767b-8zpnm\" (UID: \"bc0b4316-9c87-4c72-bc06-b4b526c2070e\") " pod="openstack-kuttl-tests/barbican-keystone-listener-74b68b767b-8zpnm" Dec 03 12:38:28 crc kubenswrapper[5052]: I1203 12:38:28.048035 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lkrkl\" (UniqueName: \"kubernetes.io/projected/bc0b4316-9c87-4c72-bc06-b4b526c2070e-kube-api-access-lkrkl\") pod \"barbican-keystone-listener-74b68b767b-8zpnm\" (UID: \"bc0b4316-9c87-4c72-bc06-b4b526c2070e\") " pod="openstack-kuttl-tests/barbican-keystone-listener-74b68b767b-8zpnm" Dec 03 12:38:28 crc kubenswrapper[5052]: I1203 12:38:28.048170 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/bc0b4316-9c87-4c72-bc06-b4b526c2070e-combined-ca-bundle\") pod \"barbican-keystone-listener-74b68b767b-8zpnm\" (UID: \"bc0b4316-9c87-4c72-bc06-b4b526c2070e\") " pod="openstack-kuttl-tests/barbican-keystone-listener-74b68b767b-8zpnm" Dec 03 12:38:28 crc kubenswrapper[5052]: I1203 12:38:28.048241 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/a998e1b6-f576-4463-b5ba-9f539f6939ee-public-tls-certs\") pod \"barbican-api-9bbb48f5b-4tfjv\" (UID: \"a998e1b6-f576-4463-b5ba-9f539f6939ee\") " pod="openstack-kuttl-tests/barbican-api-9bbb48f5b-4tfjv" Dec 03 12:38:28 crc kubenswrapper[5052]: I1203 12:38:28.048277 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/a998e1b6-f576-4463-b5ba-9f539f6939ee-internal-tls-certs\") pod \"barbican-api-9bbb48f5b-4tfjv\" (UID: \"a998e1b6-f576-4463-b5ba-9f539f6939ee\") " pod="openstack-kuttl-tests/barbican-api-9bbb48f5b-4tfjv" Dec 03 12:38:28 crc kubenswrapper[5052]: I1203 12:38:28.048333 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x5vhp\" (UniqueName: \"kubernetes.io/projected/a998e1b6-f576-4463-b5ba-9f539f6939ee-kube-api-access-x5vhp\") pod \"barbican-api-9bbb48f5b-4tfjv\" (UID: \"a998e1b6-f576-4463-b5ba-9f539f6939ee\") " pod="openstack-kuttl-tests/barbican-api-9bbb48f5b-4tfjv" Dec 03 12:38:28 crc kubenswrapper[5052]: I1203 12:38:28.048781 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/a998e1b6-f576-4463-b5ba-9f539f6939ee-config-data-custom\") pod \"barbican-api-9bbb48f5b-4tfjv\" (UID: \"a998e1b6-f576-4463-b5ba-9f539f6939ee\") " pod="openstack-kuttl-tests/barbican-api-9bbb48f5b-4tfjv" Dec 03 12:38:28 crc kubenswrapper[5052]: I1203 12:38:28.048884 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/bc0b4316-9c87-4c72-bc06-b4b526c2070e-logs\") pod \"barbican-keystone-listener-74b68b767b-8zpnm\" (UID: \"bc0b4316-9c87-4c72-bc06-b4b526c2070e\") " pod="openstack-kuttl-tests/barbican-keystone-listener-74b68b767b-8zpnm" Dec 03 12:38:28 crc kubenswrapper[5052]: I1203 12:38:28.049162 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/a998e1b6-f576-4463-b5ba-9f539f6939ee-logs\") pod \"barbican-api-9bbb48f5b-4tfjv\" (UID: \"a998e1b6-f576-4463-b5ba-9f539f6939ee\") " pod="openstack-kuttl-tests/barbican-api-9bbb48f5b-4tfjv" Dec 03 12:38:28 crc kubenswrapper[5052]: I1203 12:38:28.049284 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/18a62c9b-4032-4639-be99-3520b515313b-logs\") pod \"barbican-worker-7bd785845-99jqj\" (UID: \"18a62c9b-4032-4639-be99-3520b515313b\") " pod="openstack-kuttl-tests/barbican-worker-7bd785845-99jqj" Dec 03 12:38:28 crc kubenswrapper[5052]: I1203 12:38:28.049337 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/18a62c9b-4032-4639-be99-3520b515313b-config-data-custom\") pod \"barbican-worker-7bd785845-99jqj\" (UID: \"18a62c9b-4032-4639-be99-3520b515313b\") " pod="openstack-kuttl-tests/barbican-worker-7bd785845-99jqj" Dec 03 12:38:28 crc kubenswrapper[5052]: I1203 12:38:28.049366 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/bc0b4316-9c87-4c72-bc06-b4b526c2070e-config-data-custom\") pod \"barbican-keystone-listener-74b68b767b-8zpnm\" (UID: \"bc0b4316-9c87-4c72-bc06-b4b526c2070e\") " pod="openstack-kuttl-tests/barbican-keystone-listener-74b68b767b-8zpnm" Dec 03 12:38:28 crc kubenswrapper[5052]: I1203 12:38:28.151082 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-lkrkl\" (UniqueName: \"kubernetes.io/projected/bc0b4316-9c87-4c72-bc06-b4b526c2070e-kube-api-access-lkrkl\") pod \"barbican-keystone-listener-74b68b767b-8zpnm\" (UID: \"bc0b4316-9c87-4c72-bc06-b4b526c2070e\") " pod="openstack-kuttl-tests/barbican-keystone-listener-74b68b767b-8zpnm" Dec 03 12:38:28 crc kubenswrapper[5052]: I1203 12:38:28.151160 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/bc0b4316-9c87-4c72-bc06-b4b526c2070e-combined-ca-bundle\") pod \"barbican-keystone-listener-74b68b767b-8zpnm\" (UID: \"bc0b4316-9c87-4c72-bc06-b4b526c2070e\") " pod="openstack-kuttl-tests/barbican-keystone-listener-74b68b767b-8zpnm" Dec 03 12:38:28 crc kubenswrapper[5052]: I1203 12:38:28.151218 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/a998e1b6-f576-4463-b5ba-9f539f6939ee-public-tls-certs\") pod \"barbican-api-9bbb48f5b-4tfjv\" (UID: \"a998e1b6-f576-4463-b5ba-9f539f6939ee\") " pod="openstack-kuttl-tests/barbican-api-9bbb48f5b-4tfjv" Dec 03 12:38:28 crc kubenswrapper[5052]: I1203 12:38:28.151247 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/a998e1b6-f576-4463-b5ba-9f539f6939ee-internal-tls-certs\") pod \"barbican-api-9bbb48f5b-4tfjv\" (UID: \"a998e1b6-f576-4463-b5ba-9f539f6939ee\") " pod="openstack-kuttl-tests/barbican-api-9bbb48f5b-4tfjv" Dec 03 12:38:28 crc kubenswrapper[5052]: I1203 12:38:28.151309 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-x5vhp\" (UniqueName: \"kubernetes.io/projected/a998e1b6-f576-4463-b5ba-9f539f6939ee-kube-api-access-x5vhp\") pod \"barbican-api-9bbb48f5b-4tfjv\" (UID: \"a998e1b6-f576-4463-b5ba-9f539f6939ee\") " pod="openstack-kuttl-tests/barbican-api-9bbb48f5b-4tfjv" Dec 03 12:38:28 crc kubenswrapper[5052]: I1203 12:38:28.151348 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/a998e1b6-f576-4463-b5ba-9f539f6939ee-config-data-custom\") pod \"barbican-api-9bbb48f5b-4tfjv\" (UID: \"a998e1b6-f576-4463-b5ba-9f539f6939ee\") " pod="openstack-kuttl-tests/barbican-api-9bbb48f5b-4tfjv" Dec 03 12:38:28 crc kubenswrapper[5052]: I1203 12:38:28.151376 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/bc0b4316-9c87-4c72-bc06-b4b526c2070e-logs\") pod \"barbican-keystone-listener-74b68b767b-8zpnm\" (UID: \"bc0b4316-9c87-4c72-bc06-b4b526c2070e\") " pod="openstack-kuttl-tests/barbican-keystone-listener-74b68b767b-8zpnm" Dec 03 12:38:28 crc kubenswrapper[5052]: I1203 12:38:28.151424 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/a998e1b6-f576-4463-b5ba-9f539f6939ee-logs\") pod \"barbican-api-9bbb48f5b-4tfjv\" (UID: \"a998e1b6-f576-4463-b5ba-9f539f6939ee\") " pod="openstack-kuttl-tests/barbican-api-9bbb48f5b-4tfjv" Dec 03 12:38:28 crc kubenswrapper[5052]: I1203 12:38:28.151473 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/18a62c9b-4032-4639-be99-3520b515313b-logs\") pod \"barbican-worker-7bd785845-99jqj\" (UID: \"18a62c9b-4032-4639-be99-3520b515313b\") " pod="openstack-kuttl-tests/barbican-worker-7bd785845-99jqj" Dec 03 12:38:28 crc kubenswrapper[5052]: I1203 12:38:28.151550 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/18a62c9b-4032-4639-be99-3520b515313b-config-data-custom\") pod \"barbican-worker-7bd785845-99jqj\" (UID: \"18a62c9b-4032-4639-be99-3520b515313b\") " pod="openstack-kuttl-tests/barbican-worker-7bd785845-99jqj" Dec 03 12:38:28 crc kubenswrapper[5052]: I1203 12:38:28.151585 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/bc0b4316-9c87-4c72-bc06-b4b526c2070e-config-data-custom\") pod \"barbican-keystone-listener-74b68b767b-8zpnm\" (UID: \"bc0b4316-9c87-4c72-bc06-b4b526c2070e\") " pod="openstack-kuttl-tests/barbican-keystone-listener-74b68b767b-8zpnm" Dec 03 12:38:28 crc kubenswrapper[5052]: I1203 12:38:28.151628 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/18a62c9b-4032-4639-be99-3520b515313b-combined-ca-bundle\") pod \"barbican-worker-7bd785845-99jqj\" (UID: \"18a62c9b-4032-4639-be99-3520b515313b\") " pod="openstack-kuttl-tests/barbican-worker-7bd785845-99jqj" Dec 03 12:38:28 crc kubenswrapper[5052]: I1203 12:38:28.151650 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a998e1b6-f576-4463-b5ba-9f539f6939ee-config-data\") pod \"barbican-api-9bbb48f5b-4tfjv\" (UID: \"a998e1b6-f576-4463-b5ba-9f539f6939ee\") " pod="openstack-kuttl-tests/barbican-api-9bbb48f5b-4tfjv" Dec 03 12:38:28 crc kubenswrapper[5052]: I1203 12:38:28.151687 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a998e1b6-f576-4463-b5ba-9f539f6939ee-combined-ca-bundle\") pod \"barbican-api-9bbb48f5b-4tfjv\" (UID: \"a998e1b6-f576-4463-b5ba-9f539f6939ee\") " pod="openstack-kuttl-tests/barbican-api-9bbb48f5b-4tfjv" Dec 03 12:38:28 crc kubenswrapper[5052]: I1203 12:38:28.151714 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-hw2xp\" (UniqueName: \"kubernetes.io/projected/18a62c9b-4032-4639-be99-3520b515313b-kube-api-access-hw2xp\") pod \"barbican-worker-7bd785845-99jqj\" (UID: \"18a62c9b-4032-4639-be99-3520b515313b\") " pod="openstack-kuttl-tests/barbican-worker-7bd785845-99jqj" Dec 03 12:38:28 crc kubenswrapper[5052]: I1203 12:38:28.151792 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/18a62c9b-4032-4639-be99-3520b515313b-config-data\") pod \"barbican-worker-7bd785845-99jqj\" (UID: \"18a62c9b-4032-4639-be99-3520b515313b\") " pod="openstack-kuttl-tests/barbican-worker-7bd785845-99jqj" Dec 03 12:38:28 crc kubenswrapper[5052]: I1203 12:38:28.151820 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/bc0b4316-9c87-4c72-bc06-b4b526c2070e-config-data\") pod \"barbican-keystone-listener-74b68b767b-8zpnm\" (UID: \"bc0b4316-9c87-4c72-bc06-b4b526c2070e\") " pod="openstack-kuttl-tests/barbican-keystone-listener-74b68b767b-8zpnm" Dec 03 12:38:28 crc kubenswrapper[5052]: I1203 12:38:28.151982 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/a998e1b6-f576-4463-b5ba-9f539f6939ee-logs\") pod \"barbican-api-9bbb48f5b-4tfjv\" (UID: \"a998e1b6-f576-4463-b5ba-9f539f6939ee\") " pod="openstack-kuttl-tests/barbican-api-9bbb48f5b-4tfjv" Dec 03 12:38:28 crc kubenswrapper[5052]: I1203 12:38:28.152007 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/bc0b4316-9c87-4c72-bc06-b4b526c2070e-logs\") pod \"barbican-keystone-listener-74b68b767b-8zpnm\" (UID: \"bc0b4316-9c87-4c72-bc06-b4b526c2070e\") " pod="openstack-kuttl-tests/barbican-keystone-listener-74b68b767b-8zpnm" Dec 03 12:38:28 crc kubenswrapper[5052]: I1203 12:38:28.159445 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/18a62c9b-4032-4639-be99-3520b515313b-logs\") pod \"barbican-worker-7bd785845-99jqj\" (UID: \"18a62c9b-4032-4639-be99-3520b515313b\") " pod="openstack-kuttl-tests/barbican-worker-7bd785845-99jqj" Dec 03 12:38:28 crc kubenswrapper[5052]: I1203 12:38:28.166807 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-744cb8cb64-cqprx" event={"ID":"96b339d1-2cd9-4285-ad3c-abcc5e733a90","Type":"ContainerStarted","Data":"dbcfd144f0d294d7748d8b5a8f8656e377a499c796a3b6b9cd715c85e7d362cc"} Dec 03 12:38:28 crc kubenswrapper[5052]: I1203 12:38:28.166866 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-744cb8cb64-cqprx" event={"ID":"96b339d1-2cd9-4285-ad3c-abcc5e733a90","Type":"ContainerStarted","Data":"2ba2018de4a4e65cd9d02e1ce298bc6f417cc9863859aa31fe14b1edd2915066"} Dec 03 12:38:28 crc kubenswrapper[5052]: I1203 12:38:28.167024 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/barbican-api-744cb8cb64-cqprx" Dec 03 12:38:28 crc kubenswrapper[5052]: I1203 12:38:28.167053 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/barbican-api-744cb8cb64-cqprx" Dec 03 12:38:28 crc kubenswrapper[5052]: I1203 12:38:28.199066 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/barbican-api-744cb8cb64-cqprx" podStartSLOduration=3.199042956 podStartE2EDuration="3.199042956s" podCreationTimestamp="2025-12-03 12:38:25 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:38:28.193569209 +0000 UTC m=+4698.781810993" watchObservedRunningTime="2025-12-03 12:38:28.199042956 +0000 UTC m=+4698.787284720" Dec 03 12:38:28 crc kubenswrapper[5052]: I1203 12:38:28.324194 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/bc0b4316-9c87-4c72-bc06-b4b526c2070e-combined-ca-bundle\") pod \"barbican-keystone-listener-74b68b767b-8zpnm\" (UID: \"bc0b4316-9c87-4c72-bc06-b4b526c2070e\") " pod="openstack-kuttl-tests/barbican-keystone-listener-74b68b767b-8zpnm" Dec 03 12:38:28 crc kubenswrapper[5052]: I1203 12:38:28.324656 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/a998e1b6-f576-4463-b5ba-9f539f6939ee-public-tls-certs\") pod \"barbican-api-9bbb48f5b-4tfjv\" (UID: \"a998e1b6-f576-4463-b5ba-9f539f6939ee\") " pod="openstack-kuttl-tests/barbican-api-9bbb48f5b-4tfjv" Dec 03 12:38:28 crc kubenswrapper[5052]: I1203 12:38:28.325412 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/bc0b4316-9c87-4c72-bc06-b4b526c2070e-config-data\") pod \"barbican-keystone-listener-74b68b767b-8zpnm\" (UID: \"bc0b4316-9c87-4c72-bc06-b4b526c2070e\") " pod="openstack-kuttl-tests/barbican-keystone-listener-74b68b767b-8zpnm" Dec 03 12:38:28 crc kubenswrapper[5052]: I1203 12:38:28.328351 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/18a62c9b-4032-4639-be99-3520b515313b-config-data-custom\") pod \"barbican-worker-7bd785845-99jqj\" (UID: \"18a62c9b-4032-4639-be99-3520b515313b\") " pod="openstack-kuttl-tests/barbican-worker-7bd785845-99jqj" Dec 03 12:38:28 crc kubenswrapper[5052]: I1203 12:38:28.328526 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/a998e1b6-f576-4463-b5ba-9f539f6939ee-internal-tls-certs\") pod \"barbican-api-9bbb48f5b-4tfjv\" (UID: \"a998e1b6-f576-4463-b5ba-9f539f6939ee\") " pod="openstack-kuttl-tests/barbican-api-9bbb48f5b-4tfjv" Dec 03 12:38:28 crc kubenswrapper[5052]: I1203 12:38:28.328926 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/a998e1b6-f576-4463-b5ba-9f539f6939ee-config-data-custom\") pod \"barbican-api-9bbb48f5b-4tfjv\" (UID: \"a998e1b6-f576-4463-b5ba-9f539f6939ee\") " pod="openstack-kuttl-tests/barbican-api-9bbb48f5b-4tfjv" Dec 03 12:38:28 crc kubenswrapper[5052]: I1203 12:38:28.329411 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/bc0b4316-9c87-4c72-bc06-b4b526c2070e-config-data-custom\") pod \"barbican-keystone-listener-74b68b767b-8zpnm\" (UID: \"bc0b4316-9c87-4c72-bc06-b4b526c2070e\") " pod="openstack-kuttl-tests/barbican-keystone-listener-74b68b767b-8zpnm" Dec 03 12:38:28 crc kubenswrapper[5052]: I1203 12:38:28.330064 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a998e1b6-f576-4463-b5ba-9f539f6939ee-combined-ca-bundle\") pod \"barbican-api-9bbb48f5b-4tfjv\" (UID: \"a998e1b6-f576-4463-b5ba-9f539f6939ee\") " pod="openstack-kuttl-tests/barbican-api-9bbb48f5b-4tfjv" Dec 03 12:38:28 crc kubenswrapper[5052]: I1203 12:38:28.330806 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/18a62c9b-4032-4639-be99-3520b515313b-config-data\") pod \"barbican-worker-7bd785845-99jqj\" (UID: \"18a62c9b-4032-4639-be99-3520b515313b\") " pod="openstack-kuttl-tests/barbican-worker-7bd785845-99jqj" Dec 03 12:38:28 crc kubenswrapper[5052]: I1203 12:38:28.331098 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-x5vhp\" (UniqueName: \"kubernetes.io/projected/a998e1b6-f576-4463-b5ba-9f539f6939ee-kube-api-access-x5vhp\") pod \"barbican-api-9bbb48f5b-4tfjv\" (UID: \"a998e1b6-f576-4463-b5ba-9f539f6939ee\") " pod="openstack-kuttl-tests/barbican-api-9bbb48f5b-4tfjv" Dec 03 12:38:28 crc kubenswrapper[5052]: I1203 12:38:28.331110 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-hw2xp\" (UniqueName: \"kubernetes.io/projected/18a62c9b-4032-4639-be99-3520b515313b-kube-api-access-hw2xp\") pod \"barbican-worker-7bd785845-99jqj\" (UID: \"18a62c9b-4032-4639-be99-3520b515313b\") " pod="openstack-kuttl-tests/barbican-worker-7bd785845-99jqj" Dec 03 12:38:28 crc kubenswrapper[5052]: I1203 12:38:28.331165 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-lkrkl\" (UniqueName: \"kubernetes.io/projected/bc0b4316-9c87-4c72-bc06-b4b526c2070e-kube-api-access-lkrkl\") pod \"barbican-keystone-listener-74b68b767b-8zpnm\" (UID: \"bc0b4316-9c87-4c72-bc06-b4b526c2070e\") " pod="openstack-kuttl-tests/barbican-keystone-listener-74b68b767b-8zpnm" Dec 03 12:38:28 crc kubenswrapper[5052]: I1203 12:38:28.331215 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a998e1b6-f576-4463-b5ba-9f539f6939ee-config-data\") pod \"barbican-api-9bbb48f5b-4tfjv\" (UID: \"a998e1b6-f576-4463-b5ba-9f539f6939ee\") " pod="openstack-kuttl-tests/barbican-api-9bbb48f5b-4tfjv" Dec 03 12:38:28 crc kubenswrapper[5052]: I1203 12:38:28.331299 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/18a62c9b-4032-4639-be99-3520b515313b-combined-ca-bundle\") pod \"barbican-worker-7bd785845-99jqj\" (UID: \"18a62c9b-4032-4639-be99-3520b515313b\") " pod="openstack-kuttl-tests/barbican-worker-7bd785845-99jqj" Dec 03 12:38:28 crc kubenswrapper[5052]: I1203 12:38:28.347492 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-api-9bbb48f5b-4tfjv" Dec 03 12:38:28 crc kubenswrapper[5052]: I1203 12:38:28.498472 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-worker-7bd785845-99jqj" Dec 03 12:38:28 crc kubenswrapper[5052]: I1203 12:38:28.549163 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-keystone-listener-74b68b767b-8zpnm" Dec 03 12:38:28 crc kubenswrapper[5052]: I1203 12:38:28.834823 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-api-9bbb48f5b-4tfjv"] Dec 03 12:38:28 crc kubenswrapper[5052]: E1203 12:38:28.874305 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-swift-public-svc: secret "cert-swift-public-svc" not found Dec 03 12:38:28 crc kubenswrapper[5052]: E1203 12:38:28.874384 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/06b2a08b-6b91-44f2-b630-169938fe5771-public-tls-certs podName:06b2a08b-6b91-44f2-b630-169938fe5771 nodeName:}" failed. No retries permitted until 2025-12-03 12:38:29.374364478 +0000 UTC m=+4699.962606242 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "public-tls-certs" (UniqueName: "kubernetes.io/secret/06b2a08b-6b91-44f2-b630-169938fe5771-public-tls-certs") pod "swift-proxy-55f4df7cd-4gjcp" (UID: "06b2a08b-6b91-44f2-b630-169938fe5771") : secret "cert-swift-public-svc" not found Dec 03 12:38:28 crc kubenswrapper[5052]: E1203 12:38:28.874589 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-swift-internal-svc: secret "cert-swift-internal-svc" not found Dec 03 12:38:28 crc kubenswrapper[5052]: E1203 12:38:28.874631 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/06b2a08b-6b91-44f2-b630-169938fe5771-internal-tls-certs podName:06b2a08b-6b91-44f2-b630-169938fe5771 nodeName:}" failed. No retries permitted until 2025-12-03 12:38:29.374618945 +0000 UTC m=+4699.962860809 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "internal-tls-certs" (UniqueName: "kubernetes.io/secret/06b2a08b-6b91-44f2-b630-169938fe5771-internal-tls-certs") pod "swift-proxy-55f4df7cd-4gjcp" (UID: "06b2a08b-6b91-44f2-b630-169938fe5771") : secret "cert-swift-internal-svc" not found Dec 03 12:38:29 crc kubenswrapper[5052]: I1203 12:38:29.005746 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-worker-7bd785845-99jqj"] Dec 03 12:38:29 crc kubenswrapper[5052]: W1203 12:38:29.081721 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod18a62c9b_4032_4639_be99_3520b515313b.slice/crio-acc2228edb501fe2e9a4a38f3f06480f14fabf6f9bde01adac460f118021a3d0 WatchSource:0}: Error finding container acc2228edb501fe2e9a4a38f3f06480f14fabf6f9bde01adac460f118021a3d0: Status 404 returned error can't find the container with id acc2228edb501fe2e9a4a38f3f06480f14fabf6f9bde01adac460f118021a3d0 Dec 03 12:38:29 crc kubenswrapper[5052]: I1203 12:38:29.105595 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-keystone-listener-74b68b767b-8zpnm"] Dec 03 12:38:29 crc kubenswrapper[5052]: W1203 12:38:29.115491 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbc0b4316_9c87_4c72_bc06_b4b526c2070e.slice/crio-845b3d6b4f24339abf3ed376dbf591ebb1a98cc2405345e7336eb05dd9f9d999 WatchSource:0}: Error finding container 845b3d6b4f24339abf3ed376dbf591ebb1a98cc2405345e7336eb05dd9f9d999: Status 404 returned error can't find the container with id 845b3d6b4f24339abf3ed376dbf591ebb1a98cc2405345e7336eb05dd9f9d999 Dec 03 12:38:29 crc kubenswrapper[5052]: I1203 12:38:29.179613 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-keystone-listener-74b68b767b-8zpnm" event={"ID":"bc0b4316-9c87-4c72-bc06-b4b526c2070e","Type":"ContainerStarted","Data":"845b3d6b4f24339abf3ed376dbf591ebb1a98cc2405345e7336eb05dd9f9d999"} Dec 03 12:38:29 crc kubenswrapper[5052]: I1203 12:38:29.181616 5052 generic.go:334] "Generic (PLEG): container finished" podID="0a42958a-1720-45c8-a4d9-d38a24a246d5" containerID="bf2ad3eb69b0929f58698c3117f643b29a34469bb0e38c60eb24ebaa0a6a5ea1" exitCode=0 Dec 03 12:38:29 crc kubenswrapper[5052]: I1203 12:38:29.181632 5052 generic.go:334] "Generic (PLEG): container finished" podID="0a42958a-1720-45c8-a4d9-d38a24a246d5" containerID="57014359344dd7eed908e5209d6cd12ff8506d0d739a8eaa92efff47d08930e9" exitCode=0 Dec 03 12:38:29 crc kubenswrapper[5052]: I1203 12:38:29.181665 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"0a42958a-1720-45c8-a4d9-d38a24a246d5","Type":"ContainerDied","Data":"bf2ad3eb69b0929f58698c3117f643b29a34469bb0e38c60eb24ebaa0a6a5ea1"} Dec 03 12:38:29 crc kubenswrapper[5052]: I1203 12:38:29.181684 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"0a42958a-1720-45c8-a4d9-d38a24a246d5","Type":"ContainerDied","Data":"57014359344dd7eed908e5209d6cd12ff8506d0d739a8eaa92efff47d08930e9"} Dec 03 12:38:29 crc kubenswrapper[5052]: I1203 12:38:29.187239 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-worker-7bd785845-99jqj" event={"ID":"18a62c9b-4032-4639-be99-3520b515313b","Type":"ContainerStarted","Data":"acc2228edb501fe2e9a4a38f3f06480f14fabf6f9bde01adac460f118021a3d0"} Dec 03 12:38:29 crc kubenswrapper[5052]: I1203 12:38:29.188881 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/barbican-api-744cb8cb64-cqprx" podUID="96b339d1-2cd9-4285-ad3c-abcc5e733a90" containerName="barbican-api-log" containerID="cri-o://2ba2018de4a4e65cd9d02e1ce298bc6f417cc9863859aa31fe14b1edd2915066" gracePeriod=30 Dec 03 12:38:29 crc kubenswrapper[5052]: I1203 12:38:29.189191 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-9bbb48f5b-4tfjv" event={"ID":"a998e1b6-f576-4463-b5ba-9f539f6939ee","Type":"ContainerStarted","Data":"2f2c98f514a16f9410da4d782bafec1bccda88bd4b1ae8222e7d00e250b0991c"} Dec 03 12:38:29 crc kubenswrapper[5052]: I1203 12:38:29.189215 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-9bbb48f5b-4tfjv" event={"ID":"a998e1b6-f576-4463-b5ba-9f539f6939ee","Type":"ContainerStarted","Data":"f12cbd23c70cc8b4cadb90311161d8d4682966a9d562eadf8036ddbae5f994ed"} Dec 03 12:38:29 crc kubenswrapper[5052]: I1203 12:38:29.189584 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/barbican-api-744cb8cb64-cqprx" podUID="96b339d1-2cd9-4285-ad3c-abcc5e733a90" containerName="barbican-api" containerID="cri-o://dbcfd144f0d294d7748d8b5a8f8656e377a499c796a3b6b9cd715c85e7d362cc" gracePeriod=30 Dec 03 12:38:29 crc kubenswrapper[5052]: E1203 12:38:29.384731 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-swift-public-svc: secret "cert-swift-public-svc" not found Dec 03 12:38:29 crc kubenswrapper[5052]: E1203 12:38:29.384801 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/06b2a08b-6b91-44f2-b630-169938fe5771-public-tls-certs podName:06b2a08b-6b91-44f2-b630-169938fe5771 nodeName:}" failed. No retries permitted until 2025-12-03 12:38:30.384784685 +0000 UTC m=+4700.973026449 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "public-tls-certs" (UniqueName: "kubernetes.io/secret/06b2a08b-6b91-44f2-b630-169938fe5771-public-tls-certs") pod "swift-proxy-55f4df7cd-4gjcp" (UID: "06b2a08b-6b91-44f2-b630-169938fe5771") : secret "cert-swift-public-svc" not found Dec 03 12:38:29 crc kubenswrapper[5052]: E1203 12:38:29.384868 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-swift-internal-svc: secret "cert-swift-internal-svc" not found Dec 03 12:38:29 crc kubenswrapper[5052]: E1203 12:38:29.384942 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/06b2a08b-6b91-44f2-b630-169938fe5771-internal-tls-certs podName:06b2a08b-6b91-44f2-b630-169938fe5771 nodeName:}" failed. No retries permitted until 2025-12-03 12:38:30.384927459 +0000 UTC m=+4700.973169223 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "internal-tls-certs" (UniqueName: "kubernetes.io/secret/06b2a08b-6b91-44f2-b630-169938fe5771-internal-tls-certs") pod "swift-proxy-55f4df7cd-4gjcp" (UID: "06b2a08b-6b91-44f2-b630-169938fe5771") : secret "cert-swift-internal-svc" not found Dec 03 12:38:29 crc kubenswrapper[5052]: E1203 12:38:29.384997 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-kube-state-metrics-svc: secret "cert-kube-state-metrics-svc" not found Dec 03 12:38:29 crc kubenswrapper[5052]: E1203 12:38:29.385020 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b7defe22-28b4-4fec-8fe5-ebde8e658c6b-kube-state-metrics-tls-certs podName:b7defe22-28b4-4fec-8fe5-ebde8e658c6b nodeName:}" failed. No retries permitted until 2025-12-03 12:38:33.385012832 +0000 UTC m=+4703.973254596 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls-certs" (UniqueName: "kubernetes.io/secret/b7defe22-28b4-4fec-8fe5-ebde8e658c6b-kube-state-metrics-tls-certs") pod "kube-state-metrics-0" (UID: "b7defe22-28b4-4fec-8fe5-ebde8e658c6b") : secret "cert-kube-state-metrics-svc" not found Dec 03 12:38:29 crc kubenswrapper[5052]: I1203 12:38:29.433240 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:38:29 crc kubenswrapper[5052]: I1203 12:38:29.590100 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-68c2k\" (UniqueName: \"kubernetes.io/projected/0a42958a-1720-45c8-a4d9-d38a24a246d5-kube-api-access-68c2k\") pod \"0a42958a-1720-45c8-a4d9-d38a24a246d5\" (UID: \"0a42958a-1720-45c8-a4d9-d38a24a246d5\") " Dec 03 12:38:29 crc kubenswrapper[5052]: I1203 12:38:29.590172 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/0a42958a-1720-45c8-a4d9-d38a24a246d5-config-data\") pod \"0a42958a-1720-45c8-a4d9-d38a24a246d5\" (UID: \"0a42958a-1720-45c8-a4d9-d38a24a246d5\") " Dec 03 12:38:29 crc kubenswrapper[5052]: I1203 12:38:29.590195 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/0a42958a-1720-45c8-a4d9-d38a24a246d5-scripts\") pod \"0a42958a-1720-45c8-a4d9-d38a24a246d5\" (UID: \"0a42958a-1720-45c8-a4d9-d38a24a246d5\") " Dec 03 12:38:29 crc kubenswrapper[5052]: I1203 12:38:29.590271 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0a42958a-1720-45c8-a4d9-d38a24a246d5-combined-ca-bundle\") pod \"0a42958a-1720-45c8-a4d9-d38a24a246d5\" (UID: \"0a42958a-1720-45c8-a4d9-d38a24a246d5\") " Dec 03 12:38:29 crc kubenswrapper[5052]: I1203 12:38:29.590312 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/0a42958a-1720-45c8-a4d9-d38a24a246d5-config-data-custom\") pod \"0a42958a-1720-45c8-a4d9-d38a24a246d5\" (UID: \"0a42958a-1720-45c8-a4d9-d38a24a246d5\") " Dec 03 12:38:29 crc kubenswrapper[5052]: I1203 12:38:29.590344 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/0a42958a-1720-45c8-a4d9-d38a24a246d5-etc-machine-id\") pod \"0a42958a-1720-45c8-a4d9-d38a24a246d5\" (UID: \"0a42958a-1720-45c8-a4d9-d38a24a246d5\") " Dec 03 12:38:29 crc kubenswrapper[5052]: I1203 12:38:29.590807 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/0a42958a-1720-45c8-a4d9-d38a24a246d5-etc-machine-id" (OuterVolumeSpecName: "etc-machine-id") pod "0a42958a-1720-45c8-a4d9-d38a24a246d5" (UID: "0a42958a-1720-45c8-a4d9-d38a24a246d5"). InnerVolumeSpecName "etc-machine-id". PluginName "kubernetes.io/host-path", VolumeGidValue "" Dec 03 12:38:29 crc kubenswrapper[5052]: I1203 12:38:29.615622 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0a42958a-1720-45c8-a4d9-d38a24a246d5-scripts" (OuterVolumeSpecName: "scripts") pod "0a42958a-1720-45c8-a4d9-d38a24a246d5" (UID: "0a42958a-1720-45c8-a4d9-d38a24a246d5"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:38:29 crc kubenswrapper[5052]: I1203 12:38:29.621760 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0a42958a-1720-45c8-a4d9-d38a24a246d5-config-data-custom" (OuterVolumeSpecName: "config-data-custom") pod "0a42958a-1720-45c8-a4d9-d38a24a246d5" (UID: "0a42958a-1720-45c8-a4d9-d38a24a246d5"). InnerVolumeSpecName "config-data-custom". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:38:29 crc kubenswrapper[5052]: I1203 12:38:29.622334 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0a42958a-1720-45c8-a4d9-d38a24a246d5-kube-api-access-68c2k" (OuterVolumeSpecName: "kube-api-access-68c2k") pod "0a42958a-1720-45c8-a4d9-d38a24a246d5" (UID: "0a42958a-1720-45c8-a4d9-d38a24a246d5"). InnerVolumeSpecName "kube-api-access-68c2k". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:38:29 crc kubenswrapper[5052]: I1203 12:38:29.692178 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-68c2k\" (UniqueName: \"kubernetes.io/projected/0a42958a-1720-45c8-a4d9-d38a24a246d5-kube-api-access-68c2k\") on node \"crc\" DevicePath \"\"" Dec 03 12:38:29 crc kubenswrapper[5052]: I1203 12:38:29.692707 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/0a42958a-1720-45c8-a4d9-d38a24a246d5-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:38:29 crc kubenswrapper[5052]: I1203 12:38:29.692724 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/0a42958a-1720-45c8-a4d9-d38a24a246d5-config-data-custom\") on node \"crc\" DevicePath \"\"" Dec 03 12:38:29 crc kubenswrapper[5052]: I1203 12:38:29.692773 5052 reconciler_common.go:293] "Volume detached for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/0a42958a-1720-45c8-a4d9-d38a24a246d5-etc-machine-id\") on node \"crc\" DevicePath \"\"" Dec 03 12:38:29 crc kubenswrapper[5052]: I1203 12:38:29.750635 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0a42958a-1720-45c8-a4d9-d38a24a246d5-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "0a42958a-1720-45c8-a4d9-d38a24a246d5" (UID: "0a42958a-1720-45c8-a4d9-d38a24a246d5"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:38:29 crc kubenswrapper[5052]: I1203 12:38:29.795853 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0a42958a-1720-45c8-a4d9-d38a24a246d5-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:38:29 crc kubenswrapper[5052]: I1203 12:38:29.815476 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0a42958a-1720-45c8-a4d9-d38a24a246d5-config-data" (OuterVolumeSpecName: "config-data") pod "0a42958a-1720-45c8-a4d9-d38a24a246d5" (UID: "0a42958a-1720-45c8-a4d9-d38a24a246d5"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:38:29 crc kubenswrapper[5052]: I1203 12:38:29.896506 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-api-744cb8cb64-cqprx" Dec 03 12:38:29 crc kubenswrapper[5052]: I1203 12:38:29.900324 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/0a42958a-1720-45c8-a4d9-d38a24a246d5-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:38:30 crc kubenswrapper[5052]: I1203 12:38:30.002010 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/96b339d1-2cd9-4285-ad3c-abcc5e733a90-config-data-custom\") pod \"96b339d1-2cd9-4285-ad3c-abcc5e733a90\" (UID: \"96b339d1-2cd9-4285-ad3c-abcc5e733a90\") " Dec 03 12:38:30 crc kubenswrapper[5052]: I1203 12:38:30.002197 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/96b339d1-2cd9-4285-ad3c-abcc5e733a90-public-tls-certs\") pod \"96b339d1-2cd9-4285-ad3c-abcc5e733a90\" (UID: \"96b339d1-2cd9-4285-ad3c-abcc5e733a90\") " Dec 03 12:38:30 crc kubenswrapper[5052]: I1203 12:38:30.002241 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/96b339d1-2cd9-4285-ad3c-abcc5e733a90-config-data\") pod \"96b339d1-2cd9-4285-ad3c-abcc5e733a90\" (UID: \"96b339d1-2cd9-4285-ad3c-abcc5e733a90\") " Dec 03 12:38:30 crc kubenswrapper[5052]: I1203 12:38:30.002317 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/96b339d1-2cd9-4285-ad3c-abcc5e733a90-combined-ca-bundle\") pod \"96b339d1-2cd9-4285-ad3c-abcc5e733a90\" (UID: \"96b339d1-2cd9-4285-ad3c-abcc5e733a90\") " Dec 03 12:38:30 crc kubenswrapper[5052]: I1203 12:38:30.002378 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/96b339d1-2cd9-4285-ad3c-abcc5e733a90-logs\") pod \"96b339d1-2cd9-4285-ad3c-abcc5e733a90\" (UID: \"96b339d1-2cd9-4285-ad3c-abcc5e733a90\") " Dec 03 12:38:30 crc kubenswrapper[5052]: I1203 12:38:30.002425 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/96b339d1-2cd9-4285-ad3c-abcc5e733a90-internal-tls-certs\") pod \"96b339d1-2cd9-4285-ad3c-abcc5e733a90\" (UID: \"96b339d1-2cd9-4285-ad3c-abcc5e733a90\") " Dec 03 12:38:30 crc kubenswrapper[5052]: I1203 12:38:30.002448 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-86dr2\" (UniqueName: \"kubernetes.io/projected/96b339d1-2cd9-4285-ad3c-abcc5e733a90-kube-api-access-86dr2\") pod \"96b339d1-2cd9-4285-ad3c-abcc5e733a90\" (UID: \"96b339d1-2cd9-4285-ad3c-abcc5e733a90\") " Dec 03 12:38:30 crc kubenswrapper[5052]: I1203 12:38:30.003508 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/96b339d1-2cd9-4285-ad3c-abcc5e733a90-logs" (OuterVolumeSpecName: "logs") pod "96b339d1-2cd9-4285-ad3c-abcc5e733a90" (UID: "96b339d1-2cd9-4285-ad3c-abcc5e733a90"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:38:30 crc kubenswrapper[5052]: I1203 12:38:30.007030 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/96b339d1-2cd9-4285-ad3c-abcc5e733a90-kube-api-access-86dr2" (OuterVolumeSpecName: "kube-api-access-86dr2") pod "96b339d1-2cd9-4285-ad3c-abcc5e733a90" (UID: "96b339d1-2cd9-4285-ad3c-abcc5e733a90"). InnerVolumeSpecName "kube-api-access-86dr2". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:38:30 crc kubenswrapper[5052]: I1203 12:38:30.008407 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/96b339d1-2cd9-4285-ad3c-abcc5e733a90-config-data-custom" (OuterVolumeSpecName: "config-data-custom") pod "96b339d1-2cd9-4285-ad3c-abcc5e733a90" (UID: "96b339d1-2cd9-4285-ad3c-abcc5e733a90"). InnerVolumeSpecName "config-data-custom". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:38:30 crc kubenswrapper[5052]: I1203 12:38:30.032318 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/96b339d1-2cd9-4285-ad3c-abcc5e733a90-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "96b339d1-2cd9-4285-ad3c-abcc5e733a90" (UID: "96b339d1-2cd9-4285-ad3c-abcc5e733a90"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:38:30 crc kubenswrapper[5052]: I1203 12:38:30.053303 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/96b339d1-2cd9-4285-ad3c-abcc5e733a90-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "96b339d1-2cd9-4285-ad3c-abcc5e733a90" (UID: "96b339d1-2cd9-4285-ad3c-abcc5e733a90"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:38:30 crc kubenswrapper[5052]: I1203 12:38:30.054804 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/96b339d1-2cd9-4285-ad3c-abcc5e733a90-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "96b339d1-2cd9-4285-ad3c-abcc5e733a90" (UID: "96b339d1-2cd9-4285-ad3c-abcc5e733a90"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:38:30 crc kubenswrapper[5052]: I1203 12:38:30.069096 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/96b339d1-2cd9-4285-ad3c-abcc5e733a90-config-data" (OuterVolumeSpecName: "config-data") pod "96b339d1-2cd9-4285-ad3c-abcc5e733a90" (UID: "96b339d1-2cd9-4285-ad3c-abcc5e733a90"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:38:30 crc kubenswrapper[5052]: I1203 12:38:30.104694 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/96b339d1-2cd9-4285-ad3c-abcc5e733a90-config-data-custom\") on node \"crc\" DevicePath \"\"" Dec 03 12:38:30 crc kubenswrapper[5052]: I1203 12:38:30.104754 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/96b339d1-2cd9-4285-ad3c-abcc5e733a90-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:38:30 crc kubenswrapper[5052]: I1203 12:38:30.104768 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/96b339d1-2cd9-4285-ad3c-abcc5e733a90-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:38:30 crc kubenswrapper[5052]: I1203 12:38:30.104781 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/96b339d1-2cd9-4285-ad3c-abcc5e733a90-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:38:30 crc kubenswrapper[5052]: I1203 12:38:30.104797 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/96b339d1-2cd9-4285-ad3c-abcc5e733a90-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:38:30 crc kubenswrapper[5052]: I1203 12:38:30.104811 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/96b339d1-2cd9-4285-ad3c-abcc5e733a90-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:38:30 crc kubenswrapper[5052]: I1203 12:38:30.104824 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-86dr2\" (UniqueName: \"kubernetes.io/projected/96b339d1-2cd9-4285-ad3c-abcc5e733a90-kube-api-access-86dr2\") on node \"crc\" DevicePath \"\"" Dec 03 12:38:30 crc kubenswrapper[5052]: I1203 12:38:30.199641 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-9bbb48f5b-4tfjv" event={"ID":"a998e1b6-f576-4463-b5ba-9f539f6939ee","Type":"ContainerStarted","Data":"3ad84be36d758109b2b6d9b25c2c91696eb0fa36fbcff38476460061f16a372e"} Dec 03 12:38:30 crc kubenswrapper[5052]: I1203 12:38:30.200119 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/barbican-api-9bbb48f5b-4tfjv" Dec 03 12:38:30 crc kubenswrapper[5052]: I1203 12:38:30.202111 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-keystone-listener-74b68b767b-8zpnm" event={"ID":"bc0b4316-9c87-4c72-bc06-b4b526c2070e","Type":"ContainerStarted","Data":"a19585c85edfd20edc4392a9229786e6495d1e999b33f47c80f6aec8b04d2bf2"} Dec 03 12:38:30 crc kubenswrapper[5052]: I1203 12:38:30.202149 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-keystone-listener-74b68b767b-8zpnm" event={"ID":"bc0b4316-9c87-4c72-bc06-b4b526c2070e","Type":"ContainerStarted","Data":"f6309d5be7e4db13ae452596e6a4fcddfc94ab7011d5e22b0f3ce37272e5a6eb"} Dec 03 12:38:30 crc kubenswrapper[5052]: I1203 12:38:30.204813 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"0a42958a-1720-45c8-a4d9-d38a24a246d5","Type":"ContainerDied","Data":"4e1e8789b29e739db27c88cba0630606de7a397bef539d4c80735c38b80e05b2"} Dec 03 12:38:30 crc kubenswrapper[5052]: I1203 12:38:30.204839 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:38:30 crc kubenswrapper[5052]: I1203 12:38:30.204919 5052 scope.go:117] "RemoveContainer" containerID="bf2ad3eb69b0929f58698c3117f643b29a34469bb0e38c60eb24ebaa0a6a5ea1" Dec 03 12:38:30 crc kubenswrapper[5052]: I1203 12:38:30.215897 5052 generic.go:334] "Generic (PLEG): container finished" podID="96b339d1-2cd9-4285-ad3c-abcc5e733a90" containerID="dbcfd144f0d294d7748d8b5a8f8656e377a499c796a3b6b9cd715c85e7d362cc" exitCode=0 Dec 03 12:38:30 crc kubenswrapper[5052]: I1203 12:38:30.215932 5052 generic.go:334] "Generic (PLEG): container finished" podID="96b339d1-2cd9-4285-ad3c-abcc5e733a90" containerID="2ba2018de4a4e65cd9d02e1ce298bc6f417cc9863859aa31fe14b1edd2915066" exitCode=143 Dec 03 12:38:30 crc kubenswrapper[5052]: I1203 12:38:30.216038 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-744cb8cb64-cqprx" event={"ID":"96b339d1-2cd9-4285-ad3c-abcc5e733a90","Type":"ContainerDied","Data":"dbcfd144f0d294d7748d8b5a8f8656e377a499c796a3b6b9cd715c85e7d362cc"} Dec 03 12:38:30 crc kubenswrapper[5052]: I1203 12:38:30.216073 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-744cb8cb64-cqprx" event={"ID":"96b339d1-2cd9-4285-ad3c-abcc5e733a90","Type":"ContainerDied","Data":"2ba2018de4a4e65cd9d02e1ce298bc6f417cc9863859aa31fe14b1edd2915066"} Dec 03 12:38:30 crc kubenswrapper[5052]: I1203 12:38:30.216090 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-744cb8cb64-cqprx" event={"ID":"96b339d1-2cd9-4285-ad3c-abcc5e733a90","Type":"ContainerDied","Data":"a39a960bfea3b3352f097e277937b02c0265a10fe75d9cc345b5e6a304e5b370"} Dec 03 12:38:30 crc kubenswrapper[5052]: I1203 12:38:30.216161 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-api-744cb8cb64-cqprx" Dec 03 12:38:30 crc kubenswrapper[5052]: I1203 12:38:30.224505 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-worker-7bd785845-99jqj" event={"ID":"18a62c9b-4032-4639-be99-3520b515313b","Type":"ContainerStarted","Data":"0aea18d8ca92744fcf1bb8068e5ff8a5eb48807d9cf65983d90eb513bcd33ddb"} Dec 03 12:38:30 crc kubenswrapper[5052]: I1203 12:38:30.224568 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-worker-7bd785845-99jqj" event={"ID":"18a62c9b-4032-4639-be99-3520b515313b","Type":"ContainerStarted","Data":"c45f971177244262381100c68ce2c891e45f06d82ee0da8bc330ce4a83260099"} Dec 03 12:38:30 crc kubenswrapper[5052]: I1203 12:38:30.228110 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/barbican-api-9bbb48f5b-4tfjv" podStartSLOduration=3.228096383 podStartE2EDuration="3.228096383s" podCreationTimestamp="2025-12-03 12:38:27 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:38:30.219446106 +0000 UTC m=+4700.807687870" watchObservedRunningTime="2025-12-03 12:38:30.228096383 +0000 UTC m=+4700.816338147" Dec 03 12:38:30 crc kubenswrapper[5052]: I1203 12:38:30.246840 5052 scope.go:117] "RemoveContainer" containerID="57014359344dd7eed908e5209d6cd12ff8506d0d739a8eaa92efff47d08930e9" Dec 03 12:38:30 crc kubenswrapper[5052]: I1203 12:38:30.252454 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/barbican-keystone-listener-74b68b767b-8zpnm" podStartSLOduration=3.252436529 podStartE2EDuration="3.252436529s" podCreationTimestamp="2025-12-03 12:38:27 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:38:30.241327141 +0000 UTC m=+4700.829568925" watchObservedRunningTime="2025-12-03 12:38:30.252436529 +0000 UTC m=+4700.840678293" Dec 03 12:38:30 crc kubenswrapper[5052]: I1203 12:38:30.285022 5052 scope.go:117] "RemoveContainer" containerID="dbcfd144f0d294d7748d8b5a8f8656e377a499c796a3b6b9cd715c85e7d362cc" Dec 03 12:38:30 crc kubenswrapper[5052]: I1203 12:38:30.290826 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/barbican-worker-7bd785845-99jqj" podStartSLOduration=3.290802186 podStartE2EDuration="3.290802186s" podCreationTimestamp="2025-12-03 12:38:27 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:38:30.259381258 +0000 UTC m=+4700.847623032" watchObservedRunningTime="2025-12-03 12:38:30.290802186 +0000 UTC m=+4700.879043960" Dec 03 12:38:30 crc kubenswrapper[5052]: I1203 12:38:30.309588 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-keystone-listener-8d6c85ffd-x99nt"] Dec 03 12:38:30 crc kubenswrapper[5052]: I1203 12:38:30.309879 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/barbican-keystone-listener-8d6c85ffd-x99nt" podUID="f8e97a34-9689-4f0f-b43e-f87b43a115c8" containerName="barbican-keystone-listener-log" containerID="cri-o://cdb963a5681bb97da6125afe6f345d268c65ca0e93d883fb5df0751413fccc4e" gracePeriod=30 Dec 03 12:38:30 crc kubenswrapper[5052]: I1203 12:38:30.310314 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/barbican-keystone-listener-8d6c85ffd-x99nt" podUID="f8e97a34-9689-4f0f-b43e-f87b43a115c8" containerName="barbican-keystone-listener" containerID="cri-o://122c1c84abc5c24f0d3838dc8447ee559106f50fd8abb559d37b19cf2c31cffa" gracePeriod=30 Dec 03 12:38:30 crc kubenswrapper[5052]: I1203 12:38:30.322330 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-api-744cb8cb64-cqprx"] Dec 03 12:38:30 crc kubenswrapper[5052]: I1203 12:38:30.330571 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/barbican-api-744cb8cb64-cqprx"] Dec 03 12:38:30 crc kubenswrapper[5052]: I1203 12:38:30.331539 5052 scope.go:117] "RemoveContainer" containerID="2ba2018de4a4e65cd9d02e1ce298bc6f417cc9863859aa31fe14b1edd2915066" Dec 03 12:38:30 crc kubenswrapper[5052]: I1203 12:38:30.346137 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-worker-7bd4f6f96f-w4glb"] Dec 03 12:38:30 crc kubenswrapper[5052]: I1203 12:38:30.346414 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/barbican-worker-7bd4f6f96f-w4glb" podUID="15415292-9b86-462f-afe6-7f688c84805b" containerName="barbican-worker-log" containerID="cri-o://0f04069bde4fecc56a28786a8ae81b79a9b8cacf126174aabf4a5e2108942a49" gracePeriod=30 Dec 03 12:38:30 crc kubenswrapper[5052]: I1203 12:38:30.346909 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/barbican-worker-7bd4f6f96f-w4glb" podUID="15415292-9b86-462f-afe6-7f688c84805b" containerName="barbican-worker" containerID="cri-o://0dc0f37a65ed55aa8edaf8d39bd7ee49b39f6566a3fd737dfc57db3b421f9551" gracePeriod=30 Dec 03 12:38:30 crc kubenswrapper[5052]: I1203 12:38:30.369420 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 12:38:30 crc kubenswrapper[5052]: I1203 12:38:30.369764 5052 scope.go:117] "RemoveContainer" containerID="dbcfd144f0d294d7748d8b5a8f8656e377a499c796a3b6b9cd715c85e7d362cc" Dec 03 12:38:30 crc kubenswrapper[5052]: E1203 12:38:30.371160 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dbcfd144f0d294d7748d8b5a8f8656e377a499c796a3b6b9cd715c85e7d362cc\": container with ID starting with dbcfd144f0d294d7748d8b5a8f8656e377a499c796a3b6b9cd715c85e7d362cc not found: ID does not exist" containerID="dbcfd144f0d294d7748d8b5a8f8656e377a499c796a3b6b9cd715c85e7d362cc" Dec 03 12:38:30 crc kubenswrapper[5052]: I1203 12:38:30.371194 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dbcfd144f0d294d7748d8b5a8f8656e377a499c796a3b6b9cd715c85e7d362cc"} err="failed to get container status \"dbcfd144f0d294d7748d8b5a8f8656e377a499c796a3b6b9cd715c85e7d362cc\": rpc error: code = NotFound desc = could not find container \"dbcfd144f0d294d7748d8b5a8f8656e377a499c796a3b6b9cd715c85e7d362cc\": container with ID starting with dbcfd144f0d294d7748d8b5a8f8656e377a499c796a3b6b9cd715c85e7d362cc not found: ID does not exist" Dec 03 12:38:30 crc kubenswrapper[5052]: I1203 12:38:30.371215 5052 scope.go:117] "RemoveContainer" containerID="2ba2018de4a4e65cd9d02e1ce298bc6f417cc9863859aa31fe14b1edd2915066" Dec 03 12:38:30 crc kubenswrapper[5052]: E1203 12:38:30.371433 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2ba2018de4a4e65cd9d02e1ce298bc6f417cc9863859aa31fe14b1edd2915066\": container with ID starting with 2ba2018de4a4e65cd9d02e1ce298bc6f417cc9863859aa31fe14b1edd2915066 not found: ID does not exist" containerID="2ba2018de4a4e65cd9d02e1ce298bc6f417cc9863859aa31fe14b1edd2915066" Dec 03 12:38:30 crc kubenswrapper[5052]: I1203 12:38:30.371453 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2ba2018de4a4e65cd9d02e1ce298bc6f417cc9863859aa31fe14b1edd2915066"} err="failed to get container status \"2ba2018de4a4e65cd9d02e1ce298bc6f417cc9863859aa31fe14b1edd2915066\": rpc error: code = NotFound desc = could not find container \"2ba2018de4a4e65cd9d02e1ce298bc6f417cc9863859aa31fe14b1edd2915066\": container with ID starting with 2ba2018de4a4e65cd9d02e1ce298bc6f417cc9863859aa31fe14b1edd2915066 not found: ID does not exist" Dec 03 12:38:30 crc kubenswrapper[5052]: I1203 12:38:30.371466 5052 scope.go:117] "RemoveContainer" containerID="dbcfd144f0d294d7748d8b5a8f8656e377a499c796a3b6b9cd715c85e7d362cc" Dec 03 12:38:30 crc kubenswrapper[5052]: I1203 12:38:30.371723 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dbcfd144f0d294d7748d8b5a8f8656e377a499c796a3b6b9cd715c85e7d362cc"} err="failed to get container status \"dbcfd144f0d294d7748d8b5a8f8656e377a499c796a3b6b9cd715c85e7d362cc\": rpc error: code = NotFound desc = could not find container \"dbcfd144f0d294d7748d8b5a8f8656e377a499c796a3b6b9cd715c85e7d362cc\": container with ID starting with dbcfd144f0d294d7748d8b5a8f8656e377a499c796a3b6b9cd715c85e7d362cc not found: ID does not exist" Dec 03 12:38:30 crc kubenswrapper[5052]: I1203 12:38:30.371740 5052 scope.go:117] "RemoveContainer" containerID="2ba2018de4a4e65cd9d02e1ce298bc6f417cc9863859aa31fe14b1edd2915066" Dec 03 12:38:30 crc kubenswrapper[5052]: I1203 12:38:30.371795 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 12:38:30 crc kubenswrapper[5052]: I1203 12:38:30.371982 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2ba2018de4a4e65cd9d02e1ce298bc6f417cc9863859aa31fe14b1edd2915066"} err="failed to get container status \"2ba2018de4a4e65cd9d02e1ce298bc6f417cc9863859aa31fe14b1edd2915066\": rpc error: code = NotFound desc = could not find container \"2ba2018de4a4e65cd9d02e1ce298bc6f417cc9863859aa31fe14b1edd2915066\": container with ID starting with 2ba2018de4a4e65cd9d02e1ce298bc6f417cc9863859aa31fe14b1edd2915066 not found: ID does not exist" Dec 03 12:38:30 crc kubenswrapper[5052]: I1203 12:38:30.402049 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 12:38:30 crc kubenswrapper[5052]: E1203 12:38:30.402609 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="0a42958a-1720-45c8-a4d9-d38a24a246d5" containerName="probe" Dec 03 12:38:30 crc kubenswrapper[5052]: I1203 12:38:30.402624 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a42958a-1720-45c8-a4d9-d38a24a246d5" containerName="probe" Dec 03 12:38:30 crc kubenswrapper[5052]: E1203 12:38:30.402648 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="96b339d1-2cd9-4285-ad3c-abcc5e733a90" containerName="barbican-api-log" Dec 03 12:38:30 crc kubenswrapper[5052]: I1203 12:38:30.402657 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="96b339d1-2cd9-4285-ad3c-abcc5e733a90" containerName="barbican-api-log" Dec 03 12:38:30 crc kubenswrapper[5052]: E1203 12:38:30.402677 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="0a42958a-1720-45c8-a4d9-d38a24a246d5" containerName="cinder-scheduler" Dec 03 12:38:30 crc kubenswrapper[5052]: I1203 12:38:30.402686 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a42958a-1720-45c8-a4d9-d38a24a246d5" containerName="cinder-scheduler" Dec 03 12:38:30 crc kubenswrapper[5052]: E1203 12:38:30.402702 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="96b339d1-2cd9-4285-ad3c-abcc5e733a90" containerName="barbican-api" Dec 03 12:38:30 crc kubenswrapper[5052]: I1203 12:38:30.402709 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="96b339d1-2cd9-4285-ad3c-abcc5e733a90" containerName="barbican-api" Dec 03 12:38:30 crc kubenswrapper[5052]: I1203 12:38:30.402937 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="96b339d1-2cd9-4285-ad3c-abcc5e733a90" containerName="barbican-api" Dec 03 12:38:30 crc kubenswrapper[5052]: I1203 12:38:30.402967 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="0a42958a-1720-45c8-a4d9-d38a24a246d5" containerName="cinder-scheduler" Dec 03 12:38:30 crc kubenswrapper[5052]: I1203 12:38:30.402991 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="0a42958a-1720-45c8-a4d9-d38a24a246d5" containerName="probe" Dec 03 12:38:30 crc kubenswrapper[5052]: I1203 12:38:30.403005 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="96b339d1-2cd9-4285-ad3c-abcc5e733a90" containerName="barbican-api-log" Dec 03 12:38:30 crc kubenswrapper[5052]: I1203 12:38:30.404265 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:38:30 crc kubenswrapper[5052]: I1203 12:38:30.406960 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cinder-scheduler-config-data" Dec 03 12:38:30 crc kubenswrapper[5052]: I1203 12:38:30.413093 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/d855c655-3b2f-4e4f-9b27-cf4e43a92045-config-data\") pod \"cinder-scheduler-0\" (UID: \"d855c655-3b2f-4e4f-9b27-cf4e43a92045\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:38:30 crc kubenswrapper[5052]: I1203 12:38:30.413192 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/d855c655-3b2f-4e4f-9b27-cf4e43a92045-etc-machine-id\") pod \"cinder-scheduler-0\" (UID: \"d855c655-3b2f-4e4f-9b27-cf4e43a92045\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:38:30 crc kubenswrapper[5052]: I1203 12:38:30.413314 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9qtk7\" (UniqueName: \"kubernetes.io/projected/d855c655-3b2f-4e4f-9b27-cf4e43a92045-kube-api-access-9qtk7\") pod \"cinder-scheduler-0\" (UID: \"d855c655-3b2f-4e4f-9b27-cf4e43a92045\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:38:30 crc kubenswrapper[5052]: I1203 12:38:30.413357 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/d855c655-3b2f-4e4f-9b27-cf4e43a92045-config-data-custom\") pod \"cinder-scheduler-0\" (UID: \"d855c655-3b2f-4e4f-9b27-cf4e43a92045\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:38:30 crc kubenswrapper[5052]: I1203 12:38:30.413383 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d855c655-3b2f-4e4f-9b27-cf4e43a92045-combined-ca-bundle\") pod \"cinder-scheduler-0\" (UID: \"d855c655-3b2f-4e4f-9b27-cf4e43a92045\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:38:30 crc kubenswrapper[5052]: I1203 12:38:30.414671 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/d855c655-3b2f-4e4f-9b27-cf4e43a92045-scripts\") pod \"cinder-scheduler-0\" (UID: \"d855c655-3b2f-4e4f-9b27-cf4e43a92045\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:38:30 crc kubenswrapper[5052]: E1203 12:38:30.415609 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-swift-public-svc: secret "cert-swift-public-svc" not found Dec 03 12:38:30 crc kubenswrapper[5052]: E1203 12:38:30.415662 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/06b2a08b-6b91-44f2-b630-169938fe5771-public-tls-certs podName:06b2a08b-6b91-44f2-b630-169938fe5771 nodeName:}" failed. No retries permitted until 2025-12-03 12:38:32.415648527 +0000 UTC m=+4703.003890291 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "public-tls-certs" (UniqueName: "kubernetes.io/secret/06b2a08b-6b91-44f2-b630-169938fe5771-public-tls-certs") pod "swift-proxy-55f4df7cd-4gjcp" (UID: "06b2a08b-6b91-44f2-b630-169938fe5771") : secret "cert-swift-public-svc" not found Dec 03 12:38:30 crc kubenswrapper[5052]: E1203 12:38:30.415932 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-swift-internal-svc: secret "cert-swift-internal-svc" not found Dec 03 12:38:30 crc kubenswrapper[5052]: E1203 12:38:30.415970 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/06b2a08b-6b91-44f2-b630-169938fe5771-internal-tls-certs podName:06b2a08b-6b91-44f2-b630-169938fe5771 nodeName:}" failed. No retries permitted until 2025-12-03 12:38:32.415961926 +0000 UTC m=+4703.004203680 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "internal-tls-certs" (UniqueName: "kubernetes.io/secret/06b2a08b-6b91-44f2-b630-169938fe5771-internal-tls-certs") pod "swift-proxy-55f4df7cd-4gjcp" (UID: "06b2a08b-6b91-44f2-b630-169938fe5771") : secret "cert-swift-internal-svc" not found Dec 03 12:38:30 crc kubenswrapper[5052]: I1203 12:38:30.419189 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 12:38:30 crc kubenswrapper[5052]: I1203 12:38:30.515461 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/d855c655-3b2f-4e4f-9b27-cf4e43a92045-config-data\") pod \"cinder-scheduler-0\" (UID: \"d855c655-3b2f-4e4f-9b27-cf4e43a92045\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:38:30 crc kubenswrapper[5052]: I1203 12:38:30.515541 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/d855c655-3b2f-4e4f-9b27-cf4e43a92045-etc-machine-id\") pod \"cinder-scheduler-0\" (UID: \"d855c655-3b2f-4e4f-9b27-cf4e43a92045\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:38:30 crc kubenswrapper[5052]: I1203 12:38:30.515611 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-9qtk7\" (UniqueName: \"kubernetes.io/projected/d855c655-3b2f-4e4f-9b27-cf4e43a92045-kube-api-access-9qtk7\") pod \"cinder-scheduler-0\" (UID: \"d855c655-3b2f-4e4f-9b27-cf4e43a92045\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:38:30 crc kubenswrapper[5052]: I1203 12:38:30.515642 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/d855c655-3b2f-4e4f-9b27-cf4e43a92045-config-data-custom\") pod \"cinder-scheduler-0\" (UID: \"d855c655-3b2f-4e4f-9b27-cf4e43a92045\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:38:30 crc kubenswrapper[5052]: I1203 12:38:30.515659 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d855c655-3b2f-4e4f-9b27-cf4e43a92045-combined-ca-bundle\") pod \"cinder-scheduler-0\" (UID: \"d855c655-3b2f-4e4f-9b27-cf4e43a92045\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:38:30 crc kubenswrapper[5052]: I1203 12:38:30.515698 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/d855c655-3b2f-4e4f-9b27-cf4e43a92045-scripts\") pod \"cinder-scheduler-0\" (UID: \"d855c655-3b2f-4e4f-9b27-cf4e43a92045\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:38:30 crc kubenswrapper[5052]: I1203 12:38:30.516655 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/d855c655-3b2f-4e4f-9b27-cf4e43a92045-etc-machine-id\") pod \"cinder-scheduler-0\" (UID: \"d855c655-3b2f-4e4f-9b27-cf4e43a92045\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:38:30 crc kubenswrapper[5052]: I1203 12:38:30.520613 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/d855c655-3b2f-4e4f-9b27-cf4e43a92045-scripts\") pod \"cinder-scheduler-0\" (UID: \"d855c655-3b2f-4e4f-9b27-cf4e43a92045\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:38:30 crc kubenswrapper[5052]: I1203 12:38:30.520950 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/d855c655-3b2f-4e4f-9b27-cf4e43a92045-config-data-custom\") pod \"cinder-scheduler-0\" (UID: \"d855c655-3b2f-4e4f-9b27-cf4e43a92045\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:38:30 crc kubenswrapper[5052]: I1203 12:38:30.525325 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d855c655-3b2f-4e4f-9b27-cf4e43a92045-combined-ca-bundle\") pod \"cinder-scheduler-0\" (UID: \"d855c655-3b2f-4e4f-9b27-cf4e43a92045\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:38:30 crc kubenswrapper[5052]: I1203 12:38:30.525437 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/d855c655-3b2f-4e4f-9b27-cf4e43a92045-config-data\") pod \"cinder-scheduler-0\" (UID: \"d855c655-3b2f-4e4f-9b27-cf4e43a92045\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:38:30 crc kubenswrapper[5052]: I1203 12:38:30.532753 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-9qtk7\" (UniqueName: \"kubernetes.io/projected/d855c655-3b2f-4e4f-9b27-cf4e43a92045-kube-api-access-9qtk7\") pod \"cinder-scheduler-0\" (UID: \"d855c655-3b2f-4e4f-9b27-cf4e43a92045\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:38:30 crc kubenswrapper[5052]: I1203 12:38:30.735422 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:38:31 crc kubenswrapper[5052]: I1203 12:38:31.018966 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/memcached-0"] Dec 03 12:38:31 crc kubenswrapper[5052]: I1203 12:38:31.019559 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/memcached-0" podUID="5bcb6f6e-9a7c-4313-977a-64868597c373" containerName="memcached" containerID="cri-o://8a9cce80bd63ba1cdb7efaac129f52c4051a34e072ec1a2b01113e833ffc26df" gracePeriod=30 Dec 03 12:38:31 crc kubenswrapper[5052]: I1203 12:38:31.063062 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-0"] Dec 03 12:38:31 crc kubenswrapper[5052]: I1203 12:38:31.063367 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-cell1-conductor-0" podUID="0ffa4505-bb77-4d31-8e7d-7926eae343fd" containerName="nova-cell1-conductor-conductor" containerID="cri-o://faa257c722aaf243fc59be292d2c265e25f1a8976cfba25553abe4b2ea8c30f6" gracePeriod=30 Dec 03 12:38:31 crc kubenswrapper[5052]: I1203 12:38:31.088831 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:38:31 crc kubenswrapper[5052]: I1203 12:38:31.089106 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-scheduler-0" podUID="b0baf287-dc5f-45d0-b09d-324933082b9e" containerName="nova-scheduler-scheduler" containerID="cri-o://441e495d6dff11b2b90f0f4528ee17c8d1710a586189a25097a602eff7902fd7" gracePeriod=30 Dec 03 12:38:31 crc kubenswrapper[5052]: I1203 12:38:31.106294 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:38:31 crc kubenswrapper[5052]: I1203 12:38:31.106853 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-metadata-0" podUID="18ab5c99-e604-43d2-9b49-5f94b526a476" containerName="nova-metadata-log" containerID="cri-o://2df29b41503ca0f4f1423f5cc4f05df03f748b38d67ba8937757f85fe2765311" gracePeriod=30 Dec 03 12:38:31 crc kubenswrapper[5052]: I1203 12:38:31.106928 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-metadata-0" podUID="18ab5c99-e604-43d2-9b49-5f94b526a476" containerName="nova-metadata-metadata" containerID="cri-o://c607f1aa6bd5710d36211e17d6bbe0611dda60c14807926a6aedb78693ce269e" gracePeriod=30 Dec 03 12:38:31 crc kubenswrapper[5052]: I1203 12:38:31.202166 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 12:38:31 crc kubenswrapper[5052]: I1203 12:38:31.254914 5052 generic.go:334] "Generic (PLEG): container finished" podID="15415292-9b86-462f-afe6-7f688c84805b" containerID="0f04069bde4fecc56a28786a8ae81b79a9b8cacf126174aabf4a5e2108942a49" exitCode=143 Dec 03 12:38:31 crc kubenswrapper[5052]: I1203 12:38:31.254996 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-worker-7bd4f6f96f-w4glb" event={"ID":"15415292-9b86-462f-afe6-7f688c84805b","Type":"ContainerDied","Data":"0f04069bde4fecc56a28786a8ae81b79a9b8cacf126174aabf4a5e2108942a49"} Dec 03 12:38:31 crc kubenswrapper[5052]: I1203 12:38:31.258006 5052 generic.go:334] "Generic (PLEG): container finished" podID="f8e97a34-9689-4f0f-b43e-f87b43a115c8" containerID="cdb963a5681bb97da6125afe6f345d268c65ca0e93d883fb5df0751413fccc4e" exitCode=143 Dec 03 12:38:31 crc kubenswrapper[5052]: I1203 12:38:31.258093 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-keystone-listener-8d6c85ffd-x99nt" event={"ID":"f8e97a34-9689-4f0f-b43e-f87b43a115c8","Type":"ContainerDied","Data":"cdb963a5681bb97da6125afe6f345d268c65ca0e93d883fb5df0751413fccc4e"} Dec 03 12:38:31 crc kubenswrapper[5052]: I1203 12:38:31.269100 5052 generic.go:334] "Generic (PLEG): container finished" podID="18ab5c99-e604-43d2-9b49-5f94b526a476" containerID="2df29b41503ca0f4f1423f5cc4f05df03f748b38d67ba8937757f85fe2765311" exitCode=143 Dec 03 12:38:31 crc kubenswrapper[5052]: I1203 12:38:31.269148 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"18ab5c99-e604-43d2-9b49-5f94b526a476","Type":"ContainerDied","Data":"2df29b41503ca0f4f1423f5cc4f05df03f748b38d67ba8937757f85fe2765311"} Dec 03 12:38:31 crc kubenswrapper[5052]: I1203 12:38:31.275217 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"d855c655-3b2f-4e4f-9b27-cf4e43a92045","Type":"ContainerStarted","Data":"cbf4a90f447a46d3524bfa43c1ebb6ebb7b1a7f4d967436abf0fd6d4b0a6d875"} Dec 03 12:38:31 crc kubenswrapper[5052]: I1203 12:38:31.277879 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/barbican-api-9bbb48f5b-4tfjv" Dec 03 12:38:31 crc kubenswrapper[5052]: I1203 12:38:31.282730 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-0"] Dec 03 12:38:31 crc kubenswrapper[5052]: I1203 12:38:31.282934 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-cell0-conductor-0" podUID="72a59b27-7692-41b5-96d1-59ffa7d5858a" containerName="nova-cell0-conductor-conductor" containerID="cri-o://23a9059e0881bace3a1446302321f9b380f5df1f67c2f1949c4627c97df7dceb" gracePeriod=30 Dec 03 12:38:31 crc kubenswrapper[5052]: E1203 12:38:31.349562 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-galera-openstack-svc: secret "cert-galera-openstack-svc" not found Dec 03 12:38:31 crc kubenswrapper[5052]: E1203 12:38:31.349932 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/01fb7120-3521-4969-a2c4-b2637e0e2414-galera-tls-certs podName:01fb7120-3521-4969-a2c4-b2637e0e2414 nodeName:}" failed. No retries permitted until 2025-12-03 12:38:39.349909915 +0000 UTC m=+4709.938151739 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "galera-tls-certs" (UniqueName: "kubernetes.io/secret/01fb7120-3521-4969-a2c4-b2637e0e2414-galera-tls-certs") pod "openstack-galera-0" (UID: "01fb7120-3521-4969-a2c4-b2637e0e2414") : secret "cert-galera-openstack-svc" not found Dec 03 12:38:31 crc kubenswrapper[5052]: I1203 12:38:31.690553 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0a42958a-1720-45c8-a4d9-d38a24a246d5" path="/var/lib/kubelet/pods/0a42958a-1720-45c8-a4d9-d38a24a246d5/volumes" Dec 03 12:38:31 crc kubenswrapper[5052]: I1203 12:38:31.691697 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="96b339d1-2cd9-4285-ad3c-abcc5e733a90" path="/var/lib/kubelet/pods/96b339d1-2cd9-4285-ad3c-abcc5e733a90/volumes" Dec 03 12:38:32 crc kubenswrapper[5052]: E1203 12:38:32.122455 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="faa257c722aaf243fc59be292d2c265e25f1a8976cfba25553abe4b2ea8c30f6" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:38:32 crc kubenswrapper[5052]: E1203 12:38:32.125759 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="faa257c722aaf243fc59be292d2c265e25f1a8976cfba25553abe4b2ea8c30f6" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:38:32 crc kubenswrapper[5052]: E1203 12:38:32.127756 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="faa257c722aaf243fc59be292d2c265e25f1a8976cfba25553abe4b2ea8c30f6" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:38:32 crc kubenswrapper[5052]: E1203 12:38:32.127818 5052 prober.go:104] "Probe errored" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" probeType="Readiness" pod="openstack-kuttl-tests/nova-cell1-conductor-0" podUID="0ffa4505-bb77-4d31-8e7d-7926eae343fd" containerName="nova-cell1-conductor-conductor" Dec 03 12:38:32 crc kubenswrapper[5052]: I1203 12:38:32.298819 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"d855c655-3b2f-4e4f-9b27-cf4e43a92045","Type":"ContainerStarted","Data":"c25218175c330c36ee6dc83936ebcb6c9c017af62ee188f8b82f5fd416a29e9c"} Dec 03 12:38:32 crc kubenswrapper[5052]: I1203 12:38:32.309660 5052 generic.go:334] "Generic (PLEG): container finished" podID="5bcb6f6e-9a7c-4313-977a-64868597c373" containerID="8a9cce80bd63ba1cdb7efaac129f52c4051a34e072ec1a2b01113e833ffc26df" exitCode=0 Dec 03 12:38:32 crc kubenswrapper[5052]: I1203 12:38:32.310888 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/memcached-0" event={"ID":"5bcb6f6e-9a7c-4313-977a-64868597c373","Type":"ContainerDied","Data":"8a9cce80bd63ba1cdb7efaac129f52c4051a34e072ec1a2b01113e833ffc26df"} Dec 03 12:38:32 crc kubenswrapper[5052]: I1203 12:38:32.413975 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/memcached-0" Dec 03 12:38:32 crc kubenswrapper[5052]: I1203 12:38:32.474599 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"memcached-tls-certs\" (UniqueName: \"kubernetes.io/secret/5bcb6f6e-9a7c-4313-977a-64868597c373-memcached-tls-certs\") pod \"5bcb6f6e-9a7c-4313-977a-64868597c373\" (UID: \"5bcb6f6e-9a7c-4313-977a-64868597c373\") " Dec 03 12:38:32 crc kubenswrapper[5052]: I1203 12:38:32.474712 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/5bcb6f6e-9a7c-4313-977a-64868597c373-kolla-config\") pod \"5bcb6f6e-9a7c-4313-977a-64868597c373\" (UID: \"5bcb6f6e-9a7c-4313-977a-64868597c373\") " Dec 03 12:38:32 crc kubenswrapper[5052]: I1203 12:38:32.474744 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5bcb6f6e-9a7c-4313-977a-64868597c373-combined-ca-bundle\") pod \"5bcb6f6e-9a7c-4313-977a-64868597c373\" (UID: \"5bcb6f6e-9a7c-4313-977a-64868597c373\") " Dec 03 12:38:32 crc kubenswrapper[5052]: I1203 12:38:32.474867 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wkhcj\" (UniqueName: \"kubernetes.io/projected/5bcb6f6e-9a7c-4313-977a-64868597c373-kube-api-access-wkhcj\") pod \"5bcb6f6e-9a7c-4313-977a-64868597c373\" (UID: \"5bcb6f6e-9a7c-4313-977a-64868597c373\") " Dec 03 12:38:32 crc kubenswrapper[5052]: I1203 12:38:32.475350 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/5bcb6f6e-9a7c-4313-977a-64868597c373-config-data\") pod \"5bcb6f6e-9a7c-4313-977a-64868597c373\" (UID: \"5bcb6f6e-9a7c-4313-977a-64868597c373\") " Dec 03 12:38:32 crc kubenswrapper[5052]: I1203 12:38:32.475529 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5bcb6f6e-9a7c-4313-977a-64868597c373-kolla-config" (OuterVolumeSpecName: "kolla-config") pod "5bcb6f6e-9a7c-4313-977a-64868597c373" (UID: "5bcb6f6e-9a7c-4313-977a-64868597c373"). InnerVolumeSpecName "kolla-config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:38:32 crc kubenswrapper[5052]: I1203 12:38:32.475853 5052 reconciler_common.go:293] "Volume detached for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/5bcb6f6e-9a7c-4313-977a-64868597c373-kolla-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:38:32 crc kubenswrapper[5052]: E1203 12:38:32.475928 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-swift-public-svc: secret "cert-swift-public-svc" not found Dec 03 12:38:32 crc kubenswrapper[5052]: E1203 12:38:32.475970 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/06b2a08b-6b91-44f2-b630-169938fe5771-public-tls-certs podName:06b2a08b-6b91-44f2-b630-169938fe5771 nodeName:}" failed. No retries permitted until 2025-12-03 12:38:36.475957497 +0000 UTC m=+4707.064199261 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "public-tls-certs" (UniqueName: "kubernetes.io/secret/06b2a08b-6b91-44f2-b630-169938fe5771-public-tls-certs") pod "swift-proxy-55f4df7cd-4gjcp" (UID: "06b2a08b-6b91-44f2-b630-169938fe5771") : secret "cert-swift-public-svc" not found Dec 03 12:38:32 crc kubenswrapper[5052]: E1203 12:38:32.476278 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-swift-internal-svc: secret "cert-swift-internal-svc" not found Dec 03 12:38:32 crc kubenswrapper[5052]: E1203 12:38:32.476301 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/06b2a08b-6b91-44f2-b630-169938fe5771-internal-tls-certs podName:06b2a08b-6b91-44f2-b630-169938fe5771 nodeName:}" failed. No retries permitted until 2025-12-03 12:38:36.476293527 +0000 UTC m=+4707.064535281 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "internal-tls-certs" (UniqueName: "kubernetes.io/secret/06b2a08b-6b91-44f2-b630-169938fe5771-internal-tls-certs") pod "swift-proxy-55f4df7cd-4gjcp" (UID: "06b2a08b-6b91-44f2-b630-169938fe5771") : secret "cert-swift-internal-svc" not found Dec 03 12:38:32 crc kubenswrapper[5052]: I1203 12:38:32.476645 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5bcb6f6e-9a7c-4313-977a-64868597c373-config-data" (OuterVolumeSpecName: "config-data") pod "5bcb6f6e-9a7c-4313-977a-64868597c373" (UID: "5bcb6f6e-9a7c-4313-977a-64868597c373"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:38:32 crc kubenswrapper[5052]: I1203 12:38:32.493818 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5bcb6f6e-9a7c-4313-977a-64868597c373-kube-api-access-wkhcj" (OuterVolumeSpecName: "kube-api-access-wkhcj") pod "5bcb6f6e-9a7c-4313-977a-64868597c373" (UID: "5bcb6f6e-9a7c-4313-977a-64868597c373"). InnerVolumeSpecName "kube-api-access-wkhcj". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:38:32 crc kubenswrapper[5052]: I1203 12:38:32.530389 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5bcb6f6e-9a7c-4313-977a-64868597c373-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "5bcb6f6e-9a7c-4313-977a-64868597c373" (UID: "5bcb6f6e-9a7c-4313-977a-64868597c373"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:38:32 crc kubenswrapper[5052]: I1203 12:38:32.564870 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5bcb6f6e-9a7c-4313-977a-64868597c373-memcached-tls-certs" (OuterVolumeSpecName: "memcached-tls-certs") pod "5bcb6f6e-9a7c-4313-977a-64868597c373" (UID: "5bcb6f6e-9a7c-4313-977a-64868597c373"). InnerVolumeSpecName "memcached-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:38:32 crc kubenswrapper[5052]: I1203 12:38:32.577410 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/5bcb6f6e-9a7c-4313-977a-64868597c373-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:38:32 crc kubenswrapper[5052]: I1203 12:38:32.577464 5052 reconciler_common.go:293] "Volume detached for volume \"memcached-tls-certs\" (UniqueName: \"kubernetes.io/secret/5bcb6f6e-9a7c-4313-977a-64868597c373-memcached-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:38:32 crc kubenswrapper[5052]: I1203 12:38:32.577479 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5bcb6f6e-9a7c-4313-977a-64868597c373-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:38:32 crc kubenswrapper[5052]: I1203 12:38:32.577496 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-wkhcj\" (UniqueName: \"kubernetes.io/projected/5bcb6f6e-9a7c-4313-977a-64868597c373-kube-api-access-wkhcj\") on node \"crc\" DevicePath \"\"" Dec 03 12:38:32 crc kubenswrapper[5052]: E1203 12:38:32.781095 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-ovndbcluster-sb-ovndbs: secret "cert-ovndbcluster-sb-ovndbs" not found Dec 03 12:38:32 crc kubenswrapper[5052]: E1203 12:38:32.782497 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/143d0ab6-f833-41a9-aacd-34d20b35f8ed-ovsdbserver-sb-tls-certs podName:143d0ab6-f833-41a9-aacd-34d20b35f8ed nodeName:}" failed. No retries permitted until 2025-12-03 12:38:33.282468143 +0000 UTC m=+4703.870709907 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "ovsdbserver-sb-tls-certs" (UniqueName: "kubernetes.io/secret/143d0ab6-f833-41a9-aacd-34d20b35f8ed-ovsdbserver-sb-tls-certs") pod "ovsdbserver-sb-0" (UID: "143d0ab6-f833-41a9-aacd-34d20b35f8ed") : secret "cert-ovndbcluster-sb-ovndbs" not found Dec 03 12:38:32 crc kubenswrapper[5052]: E1203 12:38:32.781165 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-ovn-metrics: secret "cert-ovn-metrics" not found Dec 03 12:38:32 crc kubenswrapper[5052]: E1203 12:38:32.782852 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/143d0ab6-f833-41a9-aacd-34d20b35f8ed-metrics-certs-tls-certs podName:143d0ab6-f833-41a9-aacd-34d20b35f8ed nodeName:}" failed. No retries permitted until 2025-12-03 12:38:33.282831503 +0000 UTC m=+4703.871073267 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs-tls-certs" (UniqueName: "kubernetes.io/secret/143d0ab6-f833-41a9-aacd-34d20b35f8ed-metrics-certs-tls-certs") pod "ovsdbserver-sb-0" (UID: "143d0ab6-f833-41a9-aacd-34d20b35f8ed") : secret "cert-ovn-metrics" not found Dec 03 12:38:33 crc kubenswrapper[5052]: E1203 12:38:33.287616 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-ovndbcluster-sb-ovndbs: secret "cert-ovndbcluster-sb-ovndbs" not found Dec 03 12:38:33 crc kubenswrapper[5052]: E1203 12:38:33.287633 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-ovn-metrics: secret "cert-ovn-metrics" not found Dec 03 12:38:33 crc kubenswrapper[5052]: E1203 12:38:33.287968 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/143d0ab6-f833-41a9-aacd-34d20b35f8ed-ovsdbserver-sb-tls-certs podName:143d0ab6-f833-41a9-aacd-34d20b35f8ed nodeName:}" failed. No retries permitted until 2025-12-03 12:38:34.287926388 +0000 UTC m=+4704.876168152 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "ovsdbserver-sb-tls-certs" (UniqueName: "kubernetes.io/secret/143d0ab6-f833-41a9-aacd-34d20b35f8ed-ovsdbserver-sb-tls-certs") pod "ovsdbserver-sb-0" (UID: "143d0ab6-f833-41a9-aacd-34d20b35f8ed") : secret "cert-ovndbcluster-sb-ovndbs" not found Dec 03 12:38:33 crc kubenswrapper[5052]: E1203 12:38:33.288058 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/143d0ab6-f833-41a9-aacd-34d20b35f8ed-metrics-certs-tls-certs podName:143d0ab6-f833-41a9-aacd-34d20b35f8ed nodeName:}" failed. No retries permitted until 2025-12-03 12:38:34.288023121 +0000 UTC m=+4704.876264945 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs-tls-certs" (UniqueName: "kubernetes.io/secret/143d0ab6-f833-41a9-aacd-34d20b35f8ed-metrics-certs-tls-certs") pod "ovsdbserver-sb-0" (UID: "143d0ab6-f833-41a9-aacd-34d20b35f8ed") : secret "cert-ovn-metrics" not found Dec 03 12:38:33 crc kubenswrapper[5052]: I1203 12:38:33.320652 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/memcached-0" Dec 03 12:38:33 crc kubenswrapper[5052]: I1203 12:38:33.320737 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/memcached-0" event={"ID":"5bcb6f6e-9a7c-4313-977a-64868597c373","Type":"ContainerDied","Data":"d5a768662ffa04134b167c3280d10cff7f65b675ac9e11faec0729044b46979c"} Dec 03 12:38:33 crc kubenswrapper[5052]: I1203 12:38:33.321277 5052 scope.go:117] "RemoveContainer" containerID="8a9cce80bd63ba1cdb7efaac129f52c4051a34e072ec1a2b01113e833ffc26df" Dec 03 12:38:33 crc kubenswrapper[5052]: I1203 12:38:33.323958 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"d855c655-3b2f-4e4f-9b27-cf4e43a92045","Type":"ContainerStarted","Data":"323efac3746af7ff7702e46c55dedadf7e2d8d45f1a20d6b7d1fa2753913a05d"} Dec 03 12:38:33 crc kubenswrapper[5052]: I1203 12:38:33.364009 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/cinder-scheduler-0" podStartSLOduration=3.363988114 podStartE2EDuration="3.363988114s" podCreationTimestamp="2025-12-03 12:38:30 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:38:33.356204301 +0000 UTC m=+4703.944446065" watchObservedRunningTime="2025-12-03 12:38:33.363988114 +0000 UTC m=+4703.952229878" Dec 03 12:38:33 crc kubenswrapper[5052]: E1203 12:38:33.390989 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-kube-state-metrics-svc: secret "cert-kube-state-metrics-svc" not found Dec 03 12:38:33 crc kubenswrapper[5052]: E1203 12:38:33.391043 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b7defe22-28b4-4fec-8fe5-ebde8e658c6b-kube-state-metrics-tls-certs podName:b7defe22-28b4-4fec-8fe5-ebde8e658c6b nodeName:}" failed. No retries permitted until 2025-12-03 12:38:41.391029997 +0000 UTC m=+4711.979271761 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls-certs" (UniqueName: "kubernetes.io/secret/b7defe22-28b4-4fec-8fe5-ebde8e658c6b-kube-state-metrics-tls-certs") pod "kube-state-metrics-0" (UID: "b7defe22-28b4-4fec-8fe5-ebde8e658c6b") : secret "cert-kube-state-metrics-svc" not found Dec 03 12:38:33 crc kubenswrapper[5052]: I1203 12:38:33.398772 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/memcached-0"] Dec 03 12:38:33 crc kubenswrapper[5052]: I1203 12:38:33.420287 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/memcached-0"] Dec 03 12:38:33 crc kubenswrapper[5052]: I1203 12:38:33.432947 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/memcached-0"] Dec 03 12:38:33 crc kubenswrapper[5052]: E1203 12:38:33.433570 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="5bcb6f6e-9a7c-4313-977a-64868597c373" containerName="memcached" Dec 03 12:38:33 crc kubenswrapper[5052]: I1203 12:38:33.438631 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="5bcb6f6e-9a7c-4313-977a-64868597c373" containerName="memcached" Dec 03 12:38:33 crc kubenswrapper[5052]: I1203 12:38:33.439241 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="5bcb6f6e-9a7c-4313-977a-64868597c373" containerName="memcached" Dec 03 12:38:33 crc kubenswrapper[5052]: I1203 12:38:33.440218 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/memcached-0" Dec 03 12:38:33 crc kubenswrapper[5052]: I1203 12:38:33.442350 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/memcached-0"] Dec 03 12:38:33 crc kubenswrapper[5052]: I1203 12:38:33.445184 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-memcached-svc" Dec 03 12:38:33 crc kubenswrapper[5052]: I1203 12:38:33.445844 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"memcached-memcached-dockercfg-wq48p" Dec 03 12:38:33 crc kubenswrapper[5052]: I1203 12:38:33.448315 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"memcached-config-data" Dec 03 12:38:33 crc kubenswrapper[5052]: I1203 12:38:33.492940 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r5fsw\" (UniqueName: \"kubernetes.io/projected/3a55dfe2-de85-40db-84f5-78404cd4a72b-kube-api-access-r5fsw\") pod \"memcached-0\" (UID: \"3a55dfe2-de85-40db-84f5-78404cd4a72b\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:38:33 crc kubenswrapper[5052]: I1203 12:38:33.493112 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/3a55dfe2-de85-40db-84f5-78404cd4a72b-config-data\") pod \"memcached-0\" (UID: \"3a55dfe2-de85-40db-84f5-78404cd4a72b\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:38:33 crc kubenswrapper[5052]: I1203 12:38:33.493133 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/3a55dfe2-de85-40db-84f5-78404cd4a72b-kolla-config\") pod \"memcached-0\" (UID: \"3a55dfe2-de85-40db-84f5-78404cd4a72b\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:38:33 crc kubenswrapper[5052]: I1203 12:38:33.493149 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3a55dfe2-de85-40db-84f5-78404cd4a72b-combined-ca-bundle\") pod \"memcached-0\" (UID: \"3a55dfe2-de85-40db-84f5-78404cd4a72b\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:38:33 crc kubenswrapper[5052]: I1203 12:38:33.493183 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"memcached-tls-certs\" (UniqueName: \"kubernetes.io/secret/3a55dfe2-de85-40db-84f5-78404cd4a72b-memcached-tls-certs\") pod \"memcached-0\" (UID: \"3a55dfe2-de85-40db-84f5-78404cd4a72b\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:38:33 crc kubenswrapper[5052]: I1203 12:38:33.597551 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-r5fsw\" (UniqueName: \"kubernetes.io/projected/3a55dfe2-de85-40db-84f5-78404cd4a72b-kube-api-access-r5fsw\") pod \"memcached-0\" (UID: \"3a55dfe2-de85-40db-84f5-78404cd4a72b\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:38:33 crc kubenswrapper[5052]: I1203 12:38:33.597779 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/3a55dfe2-de85-40db-84f5-78404cd4a72b-config-data\") pod \"memcached-0\" (UID: \"3a55dfe2-de85-40db-84f5-78404cd4a72b\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:38:33 crc kubenswrapper[5052]: I1203 12:38:33.597807 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/3a55dfe2-de85-40db-84f5-78404cd4a72b-kolla-config\") pod \"memcached-0\" (UID: \"3a55dfe2-de85-40db-84f5-78404cd4a72b\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:38:33 crc kubenswrapper[5052]: I1203 12:38:33.597828 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3a55dfe2-de85-40db-84f5-78404cd4a72b-combined-ca-bundle\") pod \"memcached-0\" (UID: \"3a55dfe2-de85-40db-84f5-78404cd4a72b\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:38:33 crc kubenswrapper[5052]: I1203 12:38:33.597862 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"memcached-tls-certs\" (UniqueName: \"kubernetes.io/secret/3a55dfe2-de85-40db-84f5-78404cd4a72b-memcached-tls-certs\") pod \"memcached-0\" (UID: \"3a55dfe2-de85-40db-84f5-78404cd4a72b\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:38:33 crc kubenswrapper[5052]: I1203 12:38:33.599128 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/3a55dfe2-de85-40db-84f5-78404cd4a72b-config-data\") pod \"memcached-0\" (UID: \"3a55dfe2-de85-40db-84f5-78404cd4a72b\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:38:33 crc kubenswrapper[5052]: I1203 12:38:33.599230 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/3a55dfe2-de85-40db-84f5-78404cd4a72b-kolla-config\") pod \"memcached-0\" (UID: \"3a55dfe2-de85-40db-84f5-78404cd4a72b\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:38:33 crc kubenswrapper[5052]: I1203 12:38:33.603647 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3a55dfe2-de85-40db-84f5-78404cd4a72b-combined-ca-bundle\") pod \"memcached-0\" (UID: \"3a55dfe2-de85-40db-84f5-78404cd4a72b\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:38:33 crc kubenswrapper[5052]: I1203 12:38:33.603777 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"memcached-tls-certs\" (UniqueName: \"kubernetes.io/secret/3a55dfe2-de85-40db-84f5-78404cd4a72b-memcached-tls-certs\") pod \"memcached-0\" (UID: \"3a55dfe2-de85-40db-84f5-78404cd4a72b\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:38:33 crc kubenswrapper[5052]: I1203 12:38:33.615969 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-r5fsw\" (UniqueName: \"kubernetes.io/projected/3a55dfe2-de85-40db-84f5-78404cd4a72b-kube-api-access-r5fsw\") pod \"memcached-0\" (UID: \"3a55dfe2-de85-40db-84f5-78404cd4a72b\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:38:33 crc kubenswrapper[5052]: I1203 12:38:33.717115 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5bcb6f6e-9a7c-4313-977a-64868597c373" path="/var/lib/kubelet/pods/5bcb6f6e-9a7c-4313-977a-64868597c373/volumes" Dec 03 12:38:33 crc kubenswrapper[5052]: I1203 12:38:33.869302 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:38:33 crc kubenswrapper[5052]: I1203 12:38:33.872729 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/memcached-0" Dec 03 12:38:33 crc kubenswrapper[5052]: I1203 12:38:33.928901 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/72a59b27-7692-41b5-96d1-59ffa7d5858a-combined-ca-bundle\") pod \"72a59b27-7692-41b5-96d1-59ffa7d5858a\" (UID: \"72a59b27-7692-41b5-96d1-59ffa7d5858a\") " Dec 03 12:38:33 crc kubenswrapper[5052]: I1203 12:38:33.929151 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/72a59b27-7692-41b5-96d1-59ffa7d5858a-config-data\") pod \"72a59b27-7692-41b5-96d1-59ffa7d5858a\" (UID: \"72a59b27-7692-41b5-96d1-59ffa7d5858a\") " Dec 03 12:38:33 crc kubenswrapper[5052]: I1203 12:38:33.929350 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fmnfd\" (UniqueName: \"kubernetes.io/projected/72a59b27-7692-41b5-96d1-59ffa7d5858a-kube-api-access-fmnfd\") pod \"72a59b27-7692-41b5-96d1-59ffa7d5858a\" (UID: \"72a59b27-7692-41b5-96d1-59ffa7d5858a\") " Dec 03 12:38:33 crc kubenswrapper[5052]: I1203 12:38:33.933856 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/72a59b27-7692-41b5-96d1-59ffa7d5858a-kube-api-access-fmnfd" (OuterVolumeSpecName: "kube-api-access-fmnfd") pod "72a59b27-7692-41b5-96d1-59ffa7d5858a" (UID: "72a59b27-7692-41b5-96d1-59ffa7d5858a"). InnerVolumeSpecName "kube-api-access-fmnfd". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:38:33 crc kubenswrapper[5052]: I1203 12:38:33.975373 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/72a59b27-7692-41b5-96d1-59ffa7d5858a-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "72a59b27-7692-41b5-96d1-59ffa7d5858a" (UID: "72a59b27-7692-41b5-96d1-59ffa7d5858a"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:38:33 crc kubenswrapper[5052]: I1203 12:38:33.984405 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/72a59b27-7692-41b5-96d1-59ffa7d5858a-config-data" (OuterVolumeSpecName: "config-data") pod "72a59b27-7692-41b5-96d1-59ffa7d5858a" (UID: "72a59b27-7692-41b5-96d1-59ffa7d5858a"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:38:34 crc kubenswrapper[5052]: I1203 12:38:34.031859 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/72a59b27-7692-41b5-96d1-59ffa7d5858a-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:38:34 crc kubenswrapper[5052]: I1203 12:38:34.031903 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-fmnfd\" (UniqueName: \"kubernetes.io/projected/72a59b27-7692-41b5-96d1-59ffa7d5858a-kube-api-access-fmnfd\") on node \"crc\" DevicePath \"\"" Dec 03 12:38:34 crc kubenswrapper[5052]: I1203 12:38:34.031919 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/72a59b27-7692-41b5-96d1-59ffa7d5858a-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:38:34 crc kubenswrapper[5052]: I1203 12:38:34.336944 5052 generic.go:334] "Generic (PLEG): container finished" podID="72a59b27-7692-41b5-96d1-59ffa7d5858a" containerID="23a9059e0881bace3a1446302321f9b380f5df1f67c2f1949c4627c97df7dceb" exitCode=0 Dec 03 12:38:34 crc kubenswrapper[5052]: I1203 12:38:34.337010 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:38:34 crc kubenswrapper[5052]: I1203 12:38:34.337046 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-conductor-0" event={"ID":"72a59b27-7692-41b5-96d1-59ffa7d5858a","Type":"ContainerDied","Data":"23a9059e0881bace3a1446302321f9b380f5df1f67c2f1949c4627c97df7dceb"} Dec 03 12:38:34 crc kubenswrapper[5052]: I1203 12:38:34.337074 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-conductor-0" event={"ID":"72a59b27-7692-41b5-96d1-59ffa7d5858a","Type":"ContainerDied","Data":"2139fed74c4c8e0b222acfd683f6009cc0edd28af1bb2ff84cb4b4e1a09bd7b4"} Dec 03 12:38:34 crc kubenswrapper[5052]: I1203 12:38:34.337091 5052 scope.go:117] "RemoveContainer" containerID="23a9059e0881bace3a1446302321f9b380f5df1f67c2f1949c4627c97df7dceb" Dec 03 12:38:34 crc kubenswrapper[5052]: E1203 12:38:34.337289 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-ovndbcluster-sb-ovndbs: secret "cert-ovndbcluster-sb-ovndbs" not found Dec 03 12:38:34 crc kubenswrapper[5052]: E1203 12:38:34.337344 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/143d0ab6-f833-41a9-aacd-34d20b35f8ed-ovsdbserver-sb-tls-certs podName:143d0ab6-f833-41a9-aacd-34d20b35f8ed nodeName:}" failed. No retries permitted until 2025-12-03 12:38:36.33732508 +0000 UTC m=+4706.925566834 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "ovsdbserver-sb-tls-certs" (UniqueName: "kubernetes.io/secret/143d0ab6-f833-41a9-aacd-34d20b35f8ed-ovsdbserver-sb-tls-certs") pod "ovsdbserver-sb-0" (UID: "143d0ab6-f833-41a9-aacd-34d20b35f8ed") : secret "cert-ovndbcluster-sb-ovndbs" not found Dec 03 12:38:34 crc kubenswrapper[5052]: E1203 12:38:34.337401 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-ovn-metrics: secret "cert-ovn-metrics" not found Dec 03 12:38:34 crc kubenswrapper[5052]: E1203 12:38:34.337470 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/143d0ab6-f833-41a9-aacd-34d20b35f8ed-metrics-certs-tls-certs podName:143d0ab6-f833-41a9-aacd-34d20b35f8ed nodeName:}" failed. No retries permitted until 2025-12-03 12:38:36.337452403 +0000 UTC m=+4706.925694257 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs-tls-certs" (UniqueName: "kubernetes.io/secret/143d0ab6-f833-41a9-aacd-34d20b35f8ed-metrics-certs-tls-certs") pod "ovsdbserver-sb-0" (UID: "143d0ab6-f833-41a9-aacd-34d20b35f8ed") : secret "cert-ovn-metrics" not found Dec 03 12:38:34 crc kubenswrapper[5052]: I1203 12:38:34.347859 5052 generic.go:334] "Generic (PLEG): container finished" podID="15415292-9b86-462f-afe6-7f688c84805b" containerID="0dc0f37a65ed55aa8edaf8d39bd7ee49b39f6566a3fd737dfc57db3b421f9551" exitCode=0 Dec 03 12:38:34 crc kubenswrapper[5052]: I1203 12:38:34.348567 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-worker-7bd4f6f96f-w4glb" event={"ID":"15415292-9b86-462f-afe6-7f688c84805b","Type":"ContainerDied","Data":"0dc0f37a65ed55aa8edaf8d39bd7ee49b39f6566a3fd737dfc57db3b421f9551"} Dec 03 12:38:34 crc kubenswrapper[5052]: I1203 12:38:34.373088 5052 scope.go:117] "RemoveContainer" containerID="23a9059e0881bace3a1446302321f9b380f5df1f67c2f1949c4627c97df7dceb" Dec 03 12:38:34 crc kubenswrapper[5052]: E1203 12:38:34.373706 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"23a9059e0881bace3a1446302321f9b380f5df1f67c2f1949c4627c97df7dceb\": container with ID starting with 23a9059e0881bace3a1446302321f9b380f5df1f67c2f1949c4627c97df7dceb not found: ID does not exist" containerID="23a9059e0881bace3a1446302321f9b380f5df1f67c2f1949c4627c97df7dceb" Dec 03 12:38:34 crc kubenswrapper[5052]: I1203 12:38:34.373743 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"23a9059e0881bace3a1446302321f9b380f5df1f67c2f1949c4627c97df7dceb"} err="failed to get container status \"23a9059e0881bace3a1446302321f9b380f5df1f67c2f1949c4627c97df7dceb\": rpc error: code = NotFound desc = could not find container \"23a9059e0881bace3a1446302321f9b380f5df1f67c2f1949c4627c97df7dceb\": container with ID starting with 23a9059e0881bace3a1446302321f9b380f5df1f67c2f1949c4627c97df7dceb not found: ID does not exist" Dec 03 12:38:34 crc kubenswrapper[5052]: I1203 12:38:34.379358 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/memcached-0"] Dec 03 12:38:34 crc kubenswrapper[5052]: I1203 12:38:34.400959 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-0"] Dec 03 12:38:34 crc kubenswrapper[5052]: I1203 12:38:34.416596 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-0"] Dec 03 12:38:34 crc kubenswrapper[5052]: I1203 12:38:34.428085 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-0"] Dec 03 12:38:34 crc kubenswrapper[5052]: E1203 12:38:34.428615 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="72a59b27-7692-41b5-96d1-59ffa7d5858a" containerName="nova-cell0-conductor-conductor" Dec 03 12:38:34 crc kubenswrapper[5052]: I1203 12:38:34.428637 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="72a59b27-7692-41b5-96d1-59ffa7d5858a" containerName="nova-cell0-conductor-conductor" Dec 03 12:38:34 crc kubenswrapper[5052]: I1203 12:38:34.428888 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="72a59b27-7692-41b5-96d1-59ffa7d5858a" containerName="nova-cell0-conductor-conductor" Dec 03 12:38:34 crc kubenswrapper[5052]: I1203 12:38:34.429764 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:38:34 crc kubenswrapper[5052]: I1203 12:38:34.432675 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell0-conductor-config-data" Dec 03 12:38:34 crc kubenswrapper[5052]: I1203 12:38:34.469620 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-0"] Dec 03 12:38:34 crc kubenswrapper[5052]: I1203 12:38:34.483062 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 12:38:34 crc kubenswrapper[5052]: I1203 12:38:34.483340 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/cinder-api-0" podUID="8a16aa42-962f-48cc-a7a8-d0e189164013" containerName="cinder-api-log" containerID="cri-o://adfc4f6fc4db6a74ee3543f31c610dc26c11391eefc7a3ac50cbb3115dbc02c5" gracePeriod=30 Dec 03 12:38:34 crc kubenswrapper[5052]: I1203 12:38:34.483655 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/cinder-api-0" podUID="8a16aa42-962f-48cc-a7a8-d0e189164013" containerName="cinder-api" containerID="cri-o://0c4e092293ea6adf3158ffe86080aa757175ad4a21f6d1b968c71ccec2b3a499" gracePeriod=30 Dec 03 12:38:34 crc kubenswrapper[5052]: I1203 12:38:34.505858 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/nova-metadata-0" podUID="18ab5c99-e604-43d2-9b49-5f94b526a476" containerName="nova-metadata-metadata" probeResult="failure" output="Get \"https://10.217.1.89:8775/\": read tcp 10.217.0.2:50130->10.217.1.89:8775: read: connection reset by peer" Dec 03 12:38:34 crc kubenswrapper[5052]: I1203 12:38:34.505874 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/nova-metadata-0" podUID="18ab5c99-e604-43d2-9b49-5f94b526a476" containerName="nova-metadata-log" probeResult="failure" output="Get \"https://10.217.1.89:8775/\": read tcp 10.217.0.2:50132->10.217.1.89:8775: read: connection reset by peer" Dec 03 12:38:34 crc kubenswrapper[5052]: I1203 12:38:34.549607 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/e9af392d-1940-4203-961e-39da1ac5f3da-config-data\") pod \"nova-cell0-conductor-0\" (UID: \"e9af392d-1940-4203-961e-39da1ac5f3da\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:38:34 crc kubenswrapper[5052]: I1203 12:38:34.549823 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gphkk\" (UniqueName: \"kubernetes.io/projected/e9af392d-1940-4203-961e-39da1ac5f3da-kube-api-access-gphkk\") pod \"nova-cell0-conductor-0\" (UID: \"e9af392d-1940-4203-961e-39da1ac5f3da\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:38:34 crc kubenswrapper[5052]: I1203 12:38:34.549861 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e9af392d-1940-4203-961e-39da1ac5f3da-combined-ca-bundle\") pod \"nova-cell0-conductor-0\" (UID: \"e9af392d-1940-4203-961e-39da1ac5f3da\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:38:34 crc kubenswrapper[5052]: E1203 12:38:34.569825 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="441e495d6dff11b2b90f0f4528ee17c8d1710a586189a25097a602eff7902fd7" cmd=["/usr/bin/pgrep","-r","DRST","nova-scheduler"] Dec 03 12:38:34 crc kubenswrapper[5052]: E1203 12:38:34.574083 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="441e495d6dff11b2b90f0f4528ee17c8d1710a586189a25097a602eff7902fd7" cmd=["/usr/bin/pgrep","-r","DRST","nova-scheduler"] Dec 03 12:38:34 crc kubenswrapper[5052]: E1203 12:38:34.576779 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="441e495d6dff11b2b90f0f4528ee17c8d1710a586189a25097a602eff7902fd7" cmd=["/usr/bin/pgrep","-r","DRST","nova-scheduler"] Dec 03 12:38:34 crc kubenswrapper[5052]: E1203 12:38:34.576842 5052 prober.go:104] "Probe errored" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" probeType="Readiness" pod="openstack-kuttl-tests/nova-scheduler-0" podUID="b0baf287-dc5f-45d0-b09d-324933082b9e" containerName="nova-scheduler-scheduler" Dec 03 12:38:34 crc kubenswrapper[5052]: I1203 12:38:34.654254 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/e9af392d-1940-4203-961e-39da1ac5f3da-config-data\") pod \"nova-cell0-conductor-0\" (UID: \"e9af392d-1940-4203-961e-39da1ac5f3da\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:38:34 crc kubenswrapper[5052]: I1203 12:38:34.654530 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-gphkk\" (UniqueName: \"kubernetes.io/projected/e9af392d-1940-4203-961e-39da1ac5f3da-kube-api-access-gphkk\") pod \"nova-cell0-conductor-0\" (UID: \"e9af392d-1940-4203-961e-39da1ac5f3da\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:38:34 crc kubenswrapper[5052]: I1203 12:38:34.654575 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e9af392d-1940-4203-961e-39da1ac5f3da-combined-ca-bundle\") pod \"nova-cell0-conductor-0\" (UID: \"e9af392d-1940-4203-961e-39da1ac5f3da\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:38:34 crc kubenswrapper[5052]: I1203 12:38:34.663367 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/e9af392d-1940-4203-961e-39da1ac5f3da-config-data\") pod \"nova-cell0-conductor-0\" (UID: \"e9af392d-1940-4203-961e-39da1ac5f3da\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:38:34 crc kubenswrapper[5052]: I1203 12:38:34.667339 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e9af392d-1940-4203-961e-39da1ac5f3da-combined-ca-bundle\") pod \"nova-cell0-conductor-0\" (UID: \"e9af392d-1940-4203-961e-39da1ac5f3da\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:38:34 crc kubenswrapper[5052]: I1203 12:38:34.672926 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-gphkk\" (UniqueName: \"kubernetes.io/projected/e9af392d-1940-4203-961e-39da1ac5f3da-kube-api-access-gphkk\") pod \"nova-cell0-conductor-0\" (UID: \"e9af392d-1940-4203-961e-39da1ac5f3da\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:38:34 crc kubenswrapper[5052]: I1203 12:38:34.777846 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:38:35 crc kubenswrapper[5052]: I1203 12:38:35.006749 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:38:35 crc kubenswrapper[5052]: I1203 12:38:35.064391 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/18ab5c99-e604-43d2-9b49-5f94b526a476-logs\") pod \"18ab5c99-e604-43d2-9b49-5f94b526a476\" (UID: \"18ab5c99-e604-43d2-9b49-5f94b526a476\") " Dec 03 12:38:35 crc kubenswrapper[5052]: I1203 12:38:35.064473 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/18ab5c99-e604-43d2-9b49-5f94b526a476-config-data\") pod \"18ab5c99-e604-43d2-9b49-5f94b526a476\" (UID: \"18ab5c99-e604-43d2-9b49-5f94b526a476\") " Dec 03 12:38:35 crc kubenswrapper[5052]: I1203 12:38:35.064548 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-28dkm\" (UniqueName: \"kubernetes.io/projected/18ab5c99-e604-43d2-9b49-5f94b526a476-kube-api-access-28dkm\") pod \"18ab5c99-e604-43d2-9b49-5f94b526a476\" (UID: \"18ab5c99-e604-43d2-9b49-5f94b526a476\") " Dec 03 12:38:35 crc kubenswrapper[5052]: I1203 12:38:35.064600 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/18ab5c99-e604-43d2-9b49-5f94b526a476-combined-ca-bundle\") pod \"18ab5c99-e604-43d2-9b49-5f94b526a476\" (UID: \"18ab5c99-e604-43d2-9b49-5f94b526a476\") " Dec 03 12:38:35 crc kubenswrapper[5052]: I1203 12:38:35.064629 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/18ab5c99-e604-43d2-9b49-5f94b526a476-nova-metadata-tls-certs\") pod \"18ab5c99-e604-43d2-9b49-5f94b526a476\" (UID: \"18ab5c99-e604-43d2-9b49-5f94b526a476\") " Dec 03 12:38:35 crc kubenswrapper[5052]: I1203 12:38:35.065198 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/18ab5c99-e604-43d2-9b49-5f94b526a476-logs" (OuterVolumeSpecName: "logs") pod "18ab5c99-e604-43d2-9b49-5f94b526a476" (UID: "18ab5c99-e604-43d2-9b49-5f94b526a476"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:38:35 crc kubenswrapper[5052]: I1203 12:38:35.066848 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/18ab5c99-e604-43d2-9b49-5f94b526a476-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:38:35 crc kubenswrapper[5052]: I1203 12:38:35.078098 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/18ab5c99-e604-43d2-9b49-5f94b526a476-kube-api-access-28dkm" (OuterVolumeSpecName: "kube-api-access-28dkm") pod "18ab5c99-e604-43d2-9b49-5f94b526a476" (UID: "18ab5c99-e604-43d2-9b49-5f94b526a476"). InnerVolumeSpecName "kube-api-access-28dkm". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:38:35 crc kubenswrapper[5052]: I1203 12:38:35.131640 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/18ab5c99-e604-43d2-9b49-5f94b526a476-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "18ab5c99-e604-43d2-9b49-5f94b526a476" (UID: "18ab5c99-e604-43d2-9b49-5f94b526a476"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:38:35 crc kubenswrapper[5052]: I1203 12:38:35.134727 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-worker-7bd4f6f96f-w4glb" Dec 03 12:38:35 crc kubenswrapper[5052]: I1203 12:38:35.154607 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/18ab5c99-e604-43d2-9b49-5f94b526a476-config-data" (OuterVolumeSpecName: "config-data") pod "18ab5c99-e604-43d2-9b49-5f94b526a476" (UID: "18ab5c99-e604-43d2-9b49-5f94b526a476"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:38:35 crc kubenswrapper[5052]: I1203 12:38:35.167678 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/18ab5c99-e604-43d2-9b49-5f94b526a476-nova-metadata-tls-certs" (OuterVolumeSpecName: "nova-metadata-tls-certs") pod "18ab5c99-e604-43d2-9b49-5f94b526a476" (UID: "18ab5c99-e604-43d2-9b49-5f94b526a476"). InnerVolumeSpecName "nova-metadata-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:38:35 crc kubenswrapper[5052]: I1203 12:38:35.168652 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qjhwk\" (UniqueName: \"kubernetes.io/projected/15415292-9b86-462f-afe6-7f688c84805b-kube-api-access-qjhwk\") pod \"15415292-9b86-462f-afe6-7f688c84805b\" (UID: \"15415292-9b86-462f-afe6-7f688c84805b\") " Dec 03 12:38:35 crc kubenswrapper[5052]: I1203 12:38:35.168805 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/15415292-9b86-462f-afe6-7f688c84805b-config-data-custom\") pod \"15415292-9b86-462f-afe6-7f688c84805b\" (UID: \"15415292-9b86-462f-afe6-7f688c84805b\") " Dec 03 12:38:35 crc kubenswrapper[5052]: I1203 12:38:35.168943 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/15415292-9b86-462f-afe6-7f688c84805b-combined-ca-bundle\") pod \"15415292-9b86-462f-afe6-7f688c84805b\" (UID: \"15415292-9b86-462f-afe6-7f688c84805b\") " Dec 03 12:38:35 crc kubenswrapper[5052]: I1203 12:38:35.168977 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/15415292-9b86-462f-afe6-7f688c84805b-logs\") pod \"15415292-9b86-462f-afe6-7f688c84805b\" (UID: \"15415292-9b86-462f-afe6-7f688c84805b\") " Dec 03 12:38:35 crc kubenswrapper[5052]: I1203 12:38:35.169001 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/15415292-9b86-462f-afe6-7f688c84805b-config-data\") pod \"15415292-9b86-462f-afe6-7f688c84805b\" (UID: \"15415292-9b86-462f-afe6-7f688c84805b\") " Dec 03 12:38:35 crc kubenswrapper[5052]: I1203 12:38:35.169798 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/18ab5c99-e604-43d2-9b49-5f94b526a476-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:38:35 crc kubenswrapper[5052]: I1203 12:38:35.169838 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-28dkm\" (UniqueName: \"kubernetes.io/projected/18ab5c99-e604-43d2-9b49-5f94b526a476-kube-api-access-28dkm\") on node \"crc\" DevicePath \"\"" Dec 03 12:38:35 crc kubenswrapper[5052]: I1203 12:38:35.169850 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/18ab5c99-e604-43d2-9b49-5f94b526a476-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:38:35 crc kubenswrapper[5052]: I1203 12:38:35.169859 5052 reconciler_common.go:293] "Volume detached for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/18ab5c99-e604-43d2-9b49-5f94b526a476-nova-metadata-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:38:35 crc kubenswrapper[5052]: I1203 12:38:35.170885 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/15415292-9b86-462f-afe6-7f688c84805b-logs" (OuterVolumeSpecName: "logs") pod "15415292-9b86-462f-afe6-7f688c84805b" (UID: "15415292-9b86-462f-afe6-7f688c84805b"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:38:35 crc kubenswrapper[5052]: I1203 12:38:35.172579 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/15415292-9b86-462f-afe6-7f688c84805b-kube-api-access-qjhwk" (OuterVolumeSpecName: "kube-api-access-qjhwk") pod "15415292-9b86-462f-afe6-7f688c84805b" (UID: "15415292-9b86-462f-afe6-7f688c84805b"). InnerVolumeSpecName "kube-api-access-qjhwk". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:38:35 crc kubenswrapper[5052]: I1203 12:38:35.174325 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/15415292-9b86-462f-afe6-7f688c84805b-config-data-custom" (OuterVolumeSpecName: "config-data-custom") pod "15415292-9b86-462f-afe6-7f688c84805b" (UID: "15415292-9b86-462f-afe6-7f688c84805b"). InnerVolumeSpecName "config-data-custom". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:38:35 crc kubenswrapper[5052]: I1203 12:38:35.181724 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-keystone-listener-8d6c85ffd-x99nt" Dec 03 12:38:35 crc kubenswrapper[5052]: I1203 12:38:35.206229 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/15415292-9b86-462f-afe6-7f688c84805b-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "15415292-9b86-462f-afe6-7f688c84805b" (UID: "15415292-9b86-462f-afe6-7f688c84805b"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:38:35 crc kubenswrapper[5052]: I1203 12:38:35.251068 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/15415292-9b86-462f-afe6-7f688c84805b-config-data" (OuterVolumeSpecName: "config-data") pod "15415292-9b86-462f-afe6-7f688c84805b" (UID: "15415292-9b86-462f-afe6-7f688c84805b"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:38:35 crc kubenswrapper[5052]: I1203 12:38:35.271706 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f8e97a34-9689-4f0f-b43e-f87b43a115c8-combined-ca-bundle\") pod \"f8e97a34-9689-4f0f-b43e-f87b43a115c8\" (UID: \"f8e97a34-9689-4f0f-b43e-f87b43a115c8\") " Dec 03 12:38:35 crc kubenswrapper[5052]: I1203 12:38:35.271764 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/f8e97a34-9689-4f0f-b43e-f87b43a115c8-logs\") pod \"f8e97a34-9689-4f0f-b43e-f87b43a115c8\" (UID: \"f8e97a34-9689-4f0f-b43e-f87b43a115c8\") " Dec 03 12:38:35 crc kubenswrapper[5052]: I1203 12:38:35.271861 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d6ln8\" (UniqueName: \"kubernetes.io/projected/f8e97a34-9689-4f0f-b43e-f87b43a115c8-kube-api-access-d6ln8\") pod \"f8e97a34-9689-4f0f-b43e-f87b43a115c8\" (UID: \"f8e97a34-9689-4f0f-b43e-f87b43a115c8\") " Dec 03 12:38:35 crc kubenswrapper[5052]: I1203 12:38:35.271973 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/f8e97a34-9689-4f0f-b43e-f87b43a115c8-config-data-custom\") pod \"f8e97a34-9689-4f0f-b43e-f87b43a115c8\" (UID: \"f8e97a34-9689-4f0f-b43e-f87b43a115c8\") " Dec 03 12:38:35 crc kubenswrapper[5052]: I1203 12:38:35.272546 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f8e97a34-9689-4f0f-b43e-f87b43a115c8-logs" (OuterVolumeSpecName: "logs") pod "f8e97a34-9689-4f0f-b43e-f87b43a115c8" (UID: "f8e97a34-9689-4f0f-b43e-f87b43a115c8"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:38:35 crc kubenswrapper[5052]: I1203 12:38:35.274731 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/f8e97a34-9689-4f0f-b43e-f87b43a115c8-config-data\") pod \"f8e97a34-9689-4f0f-b43e-f87b43a115c8\" (UID: \"f8e97a34-9689-4f0f-b43e-f87b43a115c8\") " Dec 03 12:38:35 crc kubenswrapper[5052]: I1203 12:38:35.275735 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/15415292-9b86-462f-afe6-7f688c84805b-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:38:35 crc kubenswrapper[5052]: I1203 12:38:35.275762 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/15415292-9b86-462f-afe6-7f688c84805b-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:38:35 crc kubenswrapper[5052]: I1203 12:38:35.275775 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/15415292-9b86-462f-afe6-7f688c84805b-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:38:35 crc kubenswrapper[5052]: I1203 12:38:35.275786 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/f8e97a34-9689-4f0f-b43e-f87b43a115c8-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:38:35 crc kubenswrapper[5052]: I1203 12:38:35.275798 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-qjhwk\" (UniqueName: \"kubernetes.io/projected/15415292-9b86-462f-afe6-7f688c84805b-kube-api-access-qjhwk\") on node \"crc\" DevicePath \"\"" Dec 03 12:38:35 crc kubenswrapper[5052]: I1203 12:38:35.275808 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/15415292-9b86-462f-afe6-7f688c84805b-config-data-custom\") on node \"crc\" DevicePath \"\"" Dec 03 12:38:35 crc kubenswrapper[5052]: I1203 12:38:35.277239 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f8e97a34-9689-4f0f-b43e-f87b43a115c8-kube-api-access-d6ln8" (OuterVolumeSpecName: "kube-api-access-d6ln8") pod "f8e97a34-9689-4f0f-b43e-f87b43a115c8" (UID: "f8e97a34-9689-4f0f-b43e-f87b43a115c8"). InnerVolumeSpecName "kube-api-access-d6ln8". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:38:35 crc kubenswrapper[5052]: I1203 12:38:35.277233 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f8e97a34-9689-4f0f-b43e-f87b43a115c8-config-data-custom" (OuterVolumeSpecName: "config-data-custom") pod "f8e97a34-9689-4f0f-b43e-f87b43a115c8" (UID: "f8e97a34-9689-4f0f-b43e-f87b43a115c8"). InnerVolumeSpecName "config-data-custom". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:38:35 crc kubenswrapper[5052]: I1203 12:38:35.304055 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f8e97a34-9689-4f0f-b43e-f87b43a115c8-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "f8e97a34-9689-4f0f-b43e-f87b43a115c8" (UID: "f8e97a34-9689-4f0f-b43e-f87b43a115c8"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:38:35 crc kubenswrapper[5052]: I1203 12:38:35.329569 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f8e97a34-9689-4f0f-b43e-f87b43a115c8-config-data" (OuterVolumeSpecName: "config-data") pod "f8e97a34-9689-4f0f-b43e-f87b43a115c8" (UID: "f8e97a34-9689-4f0f-b43e-f87b43a115c8"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:38:35 crc kubenswrapper[5052]: I1203 12:38:35.360433 5052 generic.go:334] "Generic (PLEG): container finished" podID="f8e97a34-9689-4f0f-b43e-f87b43a115c8" containerID="122c1c84abc5c24f0d3838dc8447ee559106f50fd8abb559d37b19cf2c31cffa" exitCode=0 Dec 03 12:38:35 crc kubenswrapper[5052]: I1203 12:38:35.360541 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-keystone-listener-8d6c85ffd-x99nt" event={"ID":"f8e97a34-9689-4f0f-b43e-f87b43a115c8","Type":"ContainerDied","Data":"122c1c84abc5c24f0d3838dc8447ee559106f50fd8abb559d37b19cf2c31cffa"} Dec 03 12:38:35 crc kubenswrapper[5052]: I1203 12:38:35.360546 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-keystone-listener-8d6c85ffd-x99nt" Dec 03 12:38:35 crc kubenswrapper[5052]: I1203 12:38:35.360651 5052 scope.go:117] "RemoveContainer" containerID="122c1c84abc5c24f0d3838dc8447ee559106f50fd8abb559d37b19cf2c31cffa" Dec 03 12:38:35 crc kubenswrapper[5052]: I1203 12:38:35.360636 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-keystone-listener-8d6c85ffd-x99nt" event={"ID":"f8e97a34-9689-4f0f-b43e-f87b43a115c8","Type":"ContainerDied","Data":"44aa20016b5f599a6556489aaea176b866aa982026fffe9d162ac81e13c9045d"} Dec 03 12:38:35 crc kubenswrapper[5052]: I1203 12:38:35.363309 5052 generic.go:334] "Generic (PLEG): container finished" podID="8a16aa42-962f-48cc-a7a8-d0e189164013" containerID="adfc4f6fc4db6a74ee3543f31c610dc26c11391eefc7a3ac50cbb3115dbc02c5" exitCode=143 Dec 03 12:38:35 crc kubenswrapper[5052]: I1203 12:38:35.363400 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"8a16aa42-962f-48cc-a7a8-d0e189164013","Type":"ContainerDied","Data":"adfc4f6fc4db6a74ee3543f31c610dc26c11391eefc7a3ac50cbb3115dbc02c5"} Dec 03 12:38:35 crc kubenswrapper[5052]: I1203 12:38:35.365607 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/memcached-0" event={"ID":"3a55dfe2-de85-40db-84f5-78404cd4a72b","Type":"ContainerStarted","Data":"46967c848876329b0210e5ac3f401315001310250adc99a9447969c57cb0bc34"} Dec 03 12:38:35 crc kubenswrapper[5052]: I1203 12:38:35.365631 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/memcached-0" event={"ID":"3a55dfe2-de85-40db-84f5-78404cd4a72b","Type":"ContainerStarted","Data":"5e0b929315e5735100ddbc38bb312ce05f6d6479429a8694e01de3688e14e9bb"} Dec 03 12:38:35 crc kubenswrapper[5052]: I1203 12:38:35.367270 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/memcached-0" Dec 03 12:38:35 crc kubenswrapper[5052]: I1203 12:38:35.376058 5052 generic.go:334] "Generic (PLEG): container finished" podID="18ab5c99-e604-43d2-9b49-5f94b526a476" containerID="c607f1aa6bd5710d36211e17d6bbe0611dda60c14807926a6aedb78693ce269e" exitCode=0 Dec 03 12:38:35 crc kubenswrapper[5052]: I1203 12:38:35.376132 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"18ab5c99-e604-43d2-9b49-5f94b526a476","Type":"ContainerDied","Data":"c607f1aa6bd5710d36211e17d6bbe0611dda60c14807926a6aedb78693ce269e"} Dec 03 12:38:35 crc kubenswrapper[5052]: I1203 12:38:35.376164 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"18ab5c99-e604-43d2-9b49-5f94b526a476","Type":"ContainerDied","Data":"c408720fa0382dab55f15320d95b7781c9a9f7b1b327be3f3e2133182b2da8fb"} Dec 03 12:38:35 crc kubenswrapper[5052]: I1203 12:38:35.376330 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:38:35 crc kubenswrapper[5052]: I1203 12:38:35.379146 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-d6ln8\" (UniqueName: \"kubernetes.io/projected/f8e97a34-9689-4f0f-b43e-f87b43a115c8-kube-api-access-d6ln8\") on node \"crc\" DevicePath \"\"" Dec 03 12:38:35 crc kubenswrapper[5052]: I1203 12:38:35.379194 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/f8e97a34-9689-4f0f-b43e-f87b43a115c8-config-data-custom\") on node \"crc\" DevicePath \"\"" Dec 03 12:38:35 crc kubenswrapper[5052]: I1203 12:38:35.379203 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/f8e97a34-9689-4f0f-b43e-f87b43a115c8-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:38:35 crc kubenswrapper[5052]: I1203 12:38:35.379212 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f8e97a34-9689-4f0f-b43e-f87b43a115c8-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:38:35 crc kubenswrapper[5052]: I1203 12:38:35.383427 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-worker-7bd4f6f96f-w4glb" Dec 03 12:38:35 crc kubenswrapper[5052]: I1203 12:38:35.383184 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-worker-7bd4f6f96f-w4glb" event={"ID":"15415292-9b86-462f-afe6-7f688c84805b","Type":"ContainerDied","Data":"b304eb2fead6ebc0b2296b60138ce48ee9146a3adbb09731cb0639c884649eff"} Dec 03 12:38:35 crc kubenswrapper[5052]: I1203 12:38:35.389979 5052 scope.go:117] "RemoveContainer" containerID="cdb963a5681bb97da6125afe6f345d268c65ca0e93d883fb5df0751413fccc4e" Dec 03 12:38:35 crc kubenswrapper[5052]: I1203 12:38:35.401113 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/memcached-0" podStartSLOduration=2.401091632 podStartE2EDuration="2.401091632s" podCreationTimestamp="2025-12-03 12:38:33 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:38:35.394120722 +0000 UTC m=+4705.982362487" watchObservedRunningTime="2025-12-03 12:38:35.401091632 +0000 UTC m=+4705.989333396" Dec 03 12:38:35 crc kubenswrapper[5052]: I1203 12:38:35.427106 5052 scope.go:117] "RemoveContainer" containerID="122c1c84abc5c24f0d3838dc8447ee559106f50fd8abb559d37b19cf2c31cffa" Dec 03 12:38:35 crc kubenswrapper[5052]: E1203 12:38:35.428563 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"122c1c84abc5c24f0d3838dc8447ee559106f50fd8abb559d37b19cf2c31cffa\": container with ID starting with 122c1c84abc5c24f0d3838dc8447ee559106f50fd8abb559d37b19cf2c31cffa not found: ID does not exist" containerID="122c1c84abc5c24f0d3838dc8447ee559106f50fd8abb559d37b19cf2c31cffa" Dec 03 12:38:35 crc kubenswrapper[5052]: I1203 12:38:35.428645 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"122c1c84abc5c24f0d3838dc8447ee559106f50fd8abb559d37b19cf2c31cffa"} err="failed to get container status \"122c1c84abc5c24f0d3838dc8447ee559106f50fd8abb559d37b19cf2c31cffa\": rpc error: code = NotFound desc = could not find container \"122c1c84abc5c24f0d3838dc8447ee559106f50fd8abb559d37b19cf2c31cffa\": container with ID starting with 122c1c84abc5c24f0d3838dc8447ee559106f50fd8abb559d37b19cf2c31cffa not found: ID does not exist" Dec 03 12:38:35 crc kubenswrapper[5052]: I1203 12:38:35.428675 5052 scope.go:117] "RemoveContainer" containerID="cdb963a5681bb97da6125afe6f345d268c65ca0e93d883fb5df0751413fccc4e" Dec 03 12:38:35 crc kubenswrapper[5052]: I1203 12:38:35.429857 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-keystone-listener-8d6c85ffd-x99nt"] Dec 03 12:38:35 crc kubenswrapper[5052]: E1203 12:38:35.430242 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cdb963a5681bb97da6125afe6f345d268c65ca0e93d883fb5df0751413fccc4e\": container with ID starting with cdb963a5681bb97da6125afe6f345d268c65ca0e93d883fb5df0751413fccc4e not found: ID does not exist" containerID="cdb963a5681bb97da6125afe6f345d268c65ca0e93d883fb5df0751413fccc4e" Dec 03 12:38:35 crc kubenswrapper[5052]: I1203 12:38:35.430310 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cdb963a5681bb97da6125afe6f345d268c65ca0e93d883fb5df0751413fccc4e"} err="failed to get container status \"cdb963a5681bb97da6125afe6f345d268c65ca0e93d883fb5df0751413fccc4e\": rpc error: code = NotFound desc = could not find container \"cdb963a5681bb97da6125afe6f345d268c65ca0e93d883fb5df0751413fccc4e\": container with ID starting with cdb963a5681bb97da6125afe6f345d268c65ca0e93d883fb5df0751413fccc4e not found: ID does not exist" Dec 03 12:38:35 crc kubenswrapper[5052]: I1203 12:38:35.430357 5052 scope.go:117] "RemoveContainer" containerID="c607f1aa6bd5710d36211e17d6bbe0611dda60c14807926a6aedb78693ce269e" Dec 03 12:38:35 crc kubenswrapper[5052]: W1203 12:38:35.439876 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode9af392d_1940_4203_961e_39da1ac5f3da.slice/crio-9ef9370a5e9488feb6b7342fd954b62b84654eebee2633b913f8032d2a5c23f2 WatchSource:0}: Error finding container 9ef9370a5e9488feb6b7342fd954b62b84654eebee2633b913f8032d2a5c23f2: Status 404 returned error can't find the container with id 9ef9370a5e9488feb6b7342fd954b62b84654eebee2633b913f8032d2a5c23f2 Dec 03 12:38:35 crc kubenswrapper[5052]: I1203 12:38:35.459573 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/barbican-keystone-listener-8d6c85ffd-x99nt"] Dec 03 12:38:35 crc kubenswrapper[5052]: I1203 12:38:35.480804 5052 scope.go:117] "RemoveContainer" containerID="2df29b41503ca0f4f1423f5cc4f05df03f748b38d67ba8937757f85fe2765311" Dec 03 12:38:35 crc kubenswrapper[5052]: I1203 12:38:35.481109 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-0"] Dec 03 12:38:35 crc kubenswrapper[5052]: I1203 12:38:35.525800 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-worker-7bd4f6f96f-w4glb"] Dec 03 12:38:35 crc kubenswrapper[5052]: I1203 12:38:35.527675 5052 scope.go:117] "RemoveContainer" containerID="c607f1aa6bd5710d36211e17d6bbe0611dda60c14807926a6aedb78693ce269e" Dec 03 12:38:35 crc kubenswrapper[5052]: E1203 12:38:35.531903 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c607f1aa6bd5710d36211e17d6bbe0611dda60c14807926a6aedb78693ce269e\": container with ID starting with c607f1aa6bd5710d36211e17d6bbe0611dda60c14807926a6aedb78693ce269e not found: ID does not exist" containerID="c607f1aa6bd5710d36211e17d6bbe0611dda60c14807926a6aedb78693ce269e" Dec 03 12:38:35 crc kubenswrapper[5052]: I1203 12:38:35.531939 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c607f1aa6bd5710d36211e17d6bbe0611dda60c14807926a6aedb78693ce269e"} err="failed to get container status \"c607f1aa6bd5710d36211e17d6bbe0611dda60c14807926a6aedb78693ce269e\": rpc error: code = NotFound desc = could not find container \"c607f1aa6bd5710d36211e17d6bbe0611dda60c14807926a6aedb78693ce269e\": container with ID starting with c607f1aa6bd5710d36211e17d6bbe0611dda60c14807926a6aedb78693ce269e not found: ID does not exist" Dec 03 12:38:35 crc kubenswrapper[5052]: I1203 12:38:35.531966 5052 scope.go:117] "RemoveContainer" containerID="2df29b41503ca0f4f1423f5cc4f05df03f748b38d67ba8937757f85fe2765311" Dec 03 12:38:35 crc kubenswrapper[5052]: I1203 12:38:35.534750 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/barbican-worker-7bd4f6f96f-w4glb"] Dec 03 12:38:35 crc kubenswrapper[5052]: E1203 12:38:35.537637 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2df29b41503ca0f4f1423f5cc4f05df03f748b38d67ba8937757f85fe2765311\": container with ID starting with 2df29b41503ca0f4f1423f5cc4f05df03f748b38d67ba8937757f85fe2765311 not found: ID does not exist" containerID="2df29b41503ca0f4f1423f5cc4f05df03f748b38d67ba8937757f85fe2765311" Dec 03 12:38:35 crc kubenswrapper[5052]: I1203 12:38:35.537678 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2df29b41503ca0f4f1423f5cc4f05df03f748b38d67ba8937757f85fe2765311"} err="failed to get container status \"2df29b41503ca0f4f1423f5cc4f05df03f748b38d67ba8937757f85fe2765311\": rpc error: code = NotFound desc = could not find container \"2df29b41503ca0f4f1423f5cc4f05df03f748b38d67ba8937757f85fe2765311\": container with ID starting with 2df29b41503ca0f4f1423f5cc4f05df03f748b38d67ba8937757f85fe2765311 not found: ID does not exist" Dec 03 12:38:35 crc kubenswrapper[5052]: I1203 12:38:35.537705 5052 scope.go:117] "RemoveContainer" containerID="0dc0f37a65ed55aa8edaf8d39bd7ee49b39f6566a3fd737dfc57db3b421f9551" Dec 03 12:38:35 crc kubenswrapper[5052]: I1203 12:38:35.546654 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:38:35 crc kubenswrapper[5052]: I1203 12:38:35.562178 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:38:35 crc kubenswrapper[5052]: I1203 12:38:35.573310 5052 scope.go:117] "RemoveContainer" containerID="0f04069bde4fecc56a28786a8ae81b79a9b8cacf126174aabf4a5e2108942a49" Dec 03 12:38:35 crc kubenswrapper[5052]: I1203 12:38:35.574986 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:38:35 crc kubenswrapper[5052]: E1203 12:38:35.575402 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="15415292-9b86-462f-afe6-7f688c84805b" containerName="barbican-worker" Dec 03 12:38:35 crc kubenswrapper[5052]: I1203 12:38:35.575423 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="15415292-9b86-462f-afe6-7f688c84805b" containerName="barbican-worker" Dec 03 12:38:35 crc kubenswrapper[5052]: E1203 12:38:35.575457 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="15415292-9b86-462f-afe6-7f688c84805b" containerName="barbican-worker-log" Dec 03 12:38:35 crc kubenswrapper[5052]: I1203 12:38:35.575465 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="15415292-9b86-462f-afe6-7f688c84805b" containerName="barbican-worker-log" Dec 03 12:38:35 crc kubenswrapper[5052]: E1203 12:38:35.575479 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="f8e97a34-9689-4f0f-b43e-f87b43a115c8" containerName="barbican-keystone-listener" Dec 03 12:38:35 crc kubenswrapper[5052]: I1203 12:38:35.575486 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="f8e97a34-9689-4f0f-b43e-f87b43a115c8" containerName="barbican-keystone-listener" Dec 03 12:38:35 crc kubenswrapper[5052]: E1203 12:38:35.575522 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="f8e97a34-9689-4f0f-b43e-f87b43a115c8" containerName="barbican-keystone-listener-log" Dec 03 12:38:35 crc kubenswrapper[5052]: I1203 12:38:35.575528 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="f8e97a34-9689-4f0f-b43e-f87b43a115c8" containerName="barbican-keystone-listener-log" Dec 03 12:38:35 crc kubenswrapper[5052]: E1203 12:38:35.575534 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="18ab5c99-e604-43d2-9b49-5f94b526a476" containerName="nova-metadata-metadata" Dec 03 12:38:35 crc kubenswrapper[5052]: I1203 12:38:35.575539 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="18ab5c99-e604-43d2-9b49-5f94b526a476" containerName="nova-metadata-metadata" Dec 03 12:38:35 crc kubenswrapper[5052]: E1203 12:38:35.575559 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="18ab5c99-e604-43d2-9b49-5f94b526a476" containerName="nova-metadata-log" Dec 03 12:38:35 crc kubenswrapper[5052]: I1203 12:38:35.575566 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="18ab5c99-e604-43d2-9b49-5f94b526a476" containerName="nova-metadata-log" Dec 03 12:38:35 crc kubenswrapper[5052]: I1203 12:38:35.575802 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="f8e97a34-9689-4f0f-b43e-f87b43a115c8" containerName="barbican-keystone-listener-log" Dec 03 12:38:35 crc kubenswrapper[5052]: I1203 12:38:35.575827 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="15415292-9b86-462f-afe6-7f688c84805b" containerName="barbican-worker" Dec 03 12:38:35 crc kubenswrapper[5052]: I1203 12:38:35.575841 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="18ab5c99-e604-43d2-9b49-5f94b526a476" containerName="nova-metadata-log" Dec 03 12:38:35 crc kubenswrapper[5052]: I1203 12:38:35.575864 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="18ab5c99-e604-43d2-9b49-5f94b526a476" containerName="nova-metadata-metadata" Dec 03 12:38:35 crc kubenswrapper[5052]: I1203 12:38:35.575886 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="f8e97a34-9689-4f0f-b43e-f87b43a115c8" containerName="barbican-keystone-listener" Dec 03 12:38:35 crc kubenswrapper[5052]: I1203 12:38:35.575900 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="15415292-9b86-462f-afe6-7f688c84805b" containerName="barbican-worker-log" Dec 03 12:38:35 crc kubenswrapper[5052]: I1203 12:38:35.577144 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:38:35 crc kubenswrapper[5052]: I1203 12:38:35.582968 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-nova-metadata-internal-svc" Dec 03 12:38:35 crc kubenswrapper[5052]: I1203 12:38:35.583212 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-metadata-config-data" Dec 03 12:38:35 crc kubenswrapper[5052]: I1203 12:38:35.587983 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:38:35 crc kubenswrapper[5052]: I1203 12:38:35.687527 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/edea8e53-340d-4f1e-99e5-082268c64ceb-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"edea8e53-340d-4f1e-99e5-082268c64ceb\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:38:35 crc kubenswrapper[5052]: I1203 12:38:35.687710 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dgnqg\" (UniqueName: \"kubernetes.io/projected/edea8e53-340d-4f1e-99e5-082268c64ceb-kube-api-access-dgnqg\") pod \"nova-metadata-0\" (UID: \"edea8e53-340d-4f1e-99e5-082268c64ceb\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:38:35 crc kubenswrapper[5052]: I1203 12:38:35.687775 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/edea8e53-340d-4f1e-99e5-082268c64ceb-logs\") pod \"nova-metadata-0\" (UID: \"edea8e53-340d-4f1e-99e5-082268c64ceb\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:38:35 crc kubenswrapper[5052]: I1203 12:38:35.696947 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/edea8e53-340d-4f1e-99e5-082268c64ceb-nova-metadata-tls-certs\") pod \"nova-metadata-0\" (UID: \"edea8e53-340d-4f1e-99e5-082268c64ceb\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:38:35 crc kubenswrapper[5052]: I1203 12:38:35.697077 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/edea8e53-340d-4f1e-99e5-082268c64ceb-config-data\") pod \"nova-metadata-0\" (UID: \"edea8e53-340d-4f1e-99e5-082268c64ceb\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:38:35 crc kubenswrapper[5052]: I1203 12:38:35.722352 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="15415292-9b86-462f-afe6-7f688c84805b" path="/var/lib/kubelet/pods/15415292-9b86-462f-afe6-7f688c84805b/volumes" Dec 03 12:38:35 crc kubenswrapper[5052]: I1203 12:38:35.723432 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="18ab5c99-e604-43d2-9b49-5f94b526a476" path="/var/lib/kubelet/pods/18ab5c99-e604-43d2-9b49-5f94b526a476/volumes" Dec 03 12:38:35 crc kubenswrapper[5052]: I1203 12:38:35.724397 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="72a59b27-7692-41b5-96d1-59ffa7d5858a" path="/var/lib/kubelet/pods/72a59b27-7692-41b5-96d1-59ffa7d5858a/volumes" Dec 03 12:38:35 crc kubenswrapper[5052]: I1203 12:38:35.727788 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f8e97a34-9689-4f0f-b43e-f87b43a115c8" path="/var/lib/kubelet/pods/f8e97a34-9689-4f0f-b43e-f87b43a115c8/volumes" Dec 03 12:38:35 crc kubenswrapper[5052]: I1203 12:38:35.736270 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:38:35 crc kubenswrapper[5052]: I1203 12:38:35.798998 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-dgnqg\" (UniqueName: \"kubernetes.io/projected/edea8e53-340d-4f1e-99e5-082268c64ceb-kube-api-access-dgnqg\") pod \"nova-metadata-0\" (UID: \"edea8e53-340d-4f1e-99e5-082268c64ceb\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:38:35 crc kubenswrapper[5052]: I1203 12:38:35.799329 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/edea8e53-340d-4f1e-99e5-082268c64ceb-logs\") pod \"nova-metadata-0\" (UID: \"edea8e53-340d-4f1e-99e5-082268c64ceb\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:38:35 crc kubenswrapper[5052]: I1203 12:38:35.799482 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/edea8e53-340d-4f1e-99e5-082268c64ceb-nova-metadata-tls-certs\") pod \"nova-metadata-0\" (UID: \"edea8e53-340d-4f1e-99e5-082268c64ceb\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:38:35 crc kubenswrapper[5052]: I1203 12:38:35.799626 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/edea8e53-340d-4f1e-99e5-082268c64ceb-config-data\") pod \"nova-metadata-0\" (UID: \"edea8e53-340d-4f1e-99e5-082268c64ceb\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:38:35 crc kubenswrapper[5052]: I1203 12:38:35.799823 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/edea8e53-340d-4f1e-99e5-082268c64ceb-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"edea8e53-340d-4f1e-99e5-082268c64ceb\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:38:35 crc kubenswrapper[5052]: I1203 12:38:35.799922 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/edea8e53-340d-4f1e-99e5-082268c64ceb-logs\") pod \"nova-metadata-0\" (UID: \"edea8e53-340d-4f1e-99e5-082268c64ceb\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:38:35 crc kubenswrapper[5052]: E1203 12:38:35.799378 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-glance-default-public-svc: secret "cert-glance-default-public-svc" not found Dec 03 12:38:35 crc kubenswrapper[5052]: E1203 12:38:35.801635 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9dccead0-7ff2-4255-ac00-10a9b357c2db-public-tls-certs podName:9dccead0-7ff2-4255-ac00-10a9b357c2db nodeName:}" failed. No retries permitted until 2025-12-03 12:38:36.301610626 +0000 UTC m=+4706.889852450 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "public-tls-certs" (UniqueName: "kubernetes.io/secret/9dccead0-7ff2-4255-ac00-10a9b357c2db-public-tls-certs") pod "glance-default-external-api-0" (UID: "9dccead0-7ff2-4255-ac00-10a9b357c2db") : secret "cert-glance-default-public-svc" not found Dec 03 12:38:35 crc kubenswrapper[5052]: I1203 12:38:35.805892 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/edea8e53-340d-4f1e-99e5-082268c64ceb-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"edea8e53-340d-4f1e-99e5-082268c64ceb\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:38:35 crc kubenswrapper[5052]: I1203 12:38:35.806798 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/edea8e53-340d-4f1e-99e5-082268c64ceb-nova-metadata-tls-certs\") pod \"nova-metadata-0\" (UID: \"edea8e53-340d-4f1e-99e5-082268c64ceb\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:38:35 crc kubenswrapper[5052]: I1203 12:38:35.807229 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/edea8e53-340d-4f1e-99e5-082268c64ceb-config-data\") pod \"nova-metadata-0\" (UID: \"edea8e53-340d-4f1e-99e5-082268c64ceb\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:38:35 crc kubenswrapper[5052]: I1203 12:38:35.821448 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-dgnqg\" (UniqueName: \"kubernetes.io/projected/edea8e53-340d-4f1e-99e5-082268c64ceb-kube-api-access-dgnqg\") pod \"nova-metadata-0\" (UID: \"edea8e53-340d-4f1e-99e5-082268c64ceb\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:38:35 crc kubenswrapper[5052]: I1203 12:38:35.938885 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:38:36 crc kubenswrapper[5052]: I1203 12:38:36.102014 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/openstack-cell1-galera-0"] Dec 03 12:38:36 crc kubenswrapper[5052]: E1203 12:38:36.317490 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-glance-default-public-svc: secret "cert-glance-default-public-svc" not found Dec 03 12:38:36 crc kubenswrapper[5052]: E1203 12:38:36.317605 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9dccead0-7ff2-4255-ac00-10a9b357c2db-public-tls-certs podName:9dccead0-7ff2-4255-ac00-10a9b357c2db nodeName:}" failed. No retries permitted until 2025-12-03 12:38:37.317589521 +0000 UTC m=+4707.905831285 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "public-tls-certs" (UniqueName: "kubernetes.io/secret/9dccead0-7ff2-4255-ac00-10a9b357c2db-public-tls-certs") pod "glance-default-external-api-0" (UID: "9dccead0-7ff2-4255-ac00-10a9b357c2db") : secret "cert-glance-default-public-svc" not found Dec 03 12:38:36 crc kubenswrapper[5052]: I1203 12:38:36.400617 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-conductor-0" event={"ID":"e9af392d-1940-4203-961e-39da1ac5f3da","Type":"ContainerStarted","Data":"88bb1e9d52b42c7f2f99966c163df2f71ff6d2b3bec16e67d60c4dbcaea76bb6"} Dec 03 12:38:36 crc kubenswrapper[5052]: I1203 12:38:36.400662 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-conductor-0" event={"ID":"e9af392d-1940-4203-961e-39da1ac5f3da","Type":"ContainerStarted","Data":"9ef9370a5e9488feb6b7342fd954b62b84654eebee2633b913f8032d2a5c23f2"} Dec 03 12:38:36 crc kubenswrapper[5052]: I1203 12:38:36.401668 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:38:36 crc kubenswrapper[5052]: E1203 12:38:36.422029 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-ovn-metrics: secret "cert-ovn-metrics" not found Dec 03 12:38:36 crc kubenswrapper[5052]: E1203 12:38:36.422116 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/143d0ab6-f833-41a9-aacd-34d20b35f8ed-metrics-certs-tls-certs podName:143d0ab6-f833-41a9-aacd-34d20b35f8ed nodeName:}" failed. No retries permitted until 2025-12-03 12:38:40.42209581 +0000 UTC m=+4711.010337574 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs-tls-certs" (UniqueName: "kubernetes.io/secret/143d0ab6-f833-41a9-aacd-34d20b35f8ed-metrics-certs-tls-certs") pod "ovsdbserver-sb-0" (UID: "143d0ab6-f833-41a9-aacd-34d20b35f8ed") : secret "cert-ovn-metrics" not found Dec 03 12:38:36 crc kubenswrapper[5052]: E1203 12:38:36.422741 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-ovndbcluster-sb-ovndbs: secret "cert-ovndbcluster-sb-ovndbs" not found Dec 03 12:38:36 crc kubenswrapper[5052]: E1203 12:38:36.422783 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/143d0ab6-f833-41a9-aacd-34d20b35f8ed-ovsdbserver-sb-tls-certs podName:143d0ab6-f833-41a9-aacd-34d20b35f8ed nodeName:}" failed. No retries permitted until 2025-12-03 12:38:40.422772679 +0000 UTC m=+4711.011014443 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "ovsdbserver-sb-tls-certs" (UniqueName: "kubernetes.io/secret/143d0ab6-f833-41a9-aacd-34d20b35f8ed-ovsdbserver-sb-tls-certs") pod "ovsdbserver-sb-0" (UID: "143d0ab6-f833-41a9-aacd-34d20b35f8ed") : secret "cert-ovndbcluster-sb-ovndbs" not found Dec 03 12:38:36 crc kubenswrapper[5052]: I1203 12:38:36.439704 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-cell0-conductor-0" podStartSLOduration=2.439679623 podStartE2EDuration="2.439679623s" podCreationTimestamp="2025-12-03 12:38:34 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:38:36.426944919 +0000 UTC m=+4707.015186683" watchObservedRunningTime="2025-12-03 12:38:36.439679623 +0000 UTC m=+4707.027921397" Dec 03 12:38:36 crc kubenswrapper[5052]: I1203 12:38:36.509984 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:38:36 crc kubenswrapper[5052]: E1203 12:38:36.538656 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-swift-internal-svc: secret "cert-swift-internal-svc" not found Dec 03 12:38:36 crc kubenswrapper[5052]: E1203 12:38:36.538762 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/06b2a08b-6b91-44f2-b630-169938fe5771-internal-tls-certs podName:06b2a08b-6b91-44f2-b630-169938fe5771 nodeName:}" failed. No retries permitted until 2025-12-03 12:38:44.538742466 +0000 UTC m=+4715.126984230 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "internal-tls-certs" (UniqueName: "kubernetes.io/secret/06b2a08b-6b91-44f2-b630-169938fe5771-internal-tls-certs") pod "swift-proxy-55f4df7cd-4gjcp" (UID: "06b2a08b-6b91-44f2-b630-169938fe5771") : secret "cert-swift-internal-svc" not found Dec 03 12:38:36 crc kubenswrapper[5052]: E1203 12:38:36.539214 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-swift-public-svc: secret "cert-swift-public-svc" not found Dec 03 12:38:36 crc kubenswrapper[5052]: E1203 12:38:36.539314 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/06b2a08b-6b91-44f2-b630-169938fe5771-public-tls-certs podName:06b2a08b-6b91-44f2-b630-169938fe5771 nodeName:}" failed. No retries permitted until 2025-12-03 12:38:44.539304302 +0000 UTC m=+4715.127546146 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "public-tls-certs" (UniqueName: "kubernetes.io/secret/06b2a08b-6b91-44f2-b630-169938fe5771-public-tls-certs") pod "swift-proxy-55f4df7cd-4gjcp" (UID: "06b2a08b-6b91-44f2-b630-169938fe5771") : secret "cert-swift-public-svc" not found Dec 03 12:38:36 crc kubenswrapper[5052]: I1203 12:38:36.653024 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/openstack-cell1-galera-0" podUID="1cc13873-68c8-4c70-89dc-caefaeb40e25" containerName="galera" containerID="cri-o://34697f0f6773a06387fe7e2f667823c92555f8d2c3bcc6f8942a7d5f2769f8ed" gracePeriod=30 Dec 03 12:38:37 crc kubenswrapper[5052]: E1203 12:38:37.117778 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of faa257c722aaf243fc59be292d2c265e25f1a8976cfba25553abe4b2ea8c30f6 is running failed: container process not found" containerID="faa257c722aaf243fc59be292d2c265e25f1a8976cfba25553abe4b2ea8c30f6" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:38:37 crc kubenswrapper[5052]: E1203 12:38:37.118259 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of faa257c722aaf243fc59be292d2c265e25f1a8976cfba25553abe4b2ea8c30f6 is running failed: container process not found" containerID="faa257c722aaf243fc59be292d2c265e25f1a8976cfba25553abe4b2ea8c30f6" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:38:37 crc kubenswrapper[5052]: E1203 12:38:37.118775 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of faa257c722aaf243fc59be292d2c265e25f1a8976cfba25553abe4b2ea8c30f6 is running failed: container process not found" containerID="faa257c722aaf243fc59be292d2c265e25f1a8976cfba25553abe4b2ea8c30f6" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:38:37 crc kubenswrapper[5052]: E1203 12:38:37.118814 5052 prober.go:104] "Probe errored" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of faa257c722aaf243fc59be292d2c265e25f1a8976cfba25553abe4b2ea8c30f6 is running failed: container process not found" probeType="Readiness" pod="openstack-kuttl-tests/nova-cell1-conductor-0" podUID="0ffa4505-bb77-4d31-8e7d-7926eae343fd" containerName="nova-cell1-conductor-conductor" Dec 03 12:38:37 crc kubenswrapper[5052]: I1203 12:38:37.153958 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:38:37 crc kubenswrapper[5052]: I1203 12:38:37.251683 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0ffa4505-bb77-4d31-8e7d-7926eae343fd-combined-ca-bundle\") pod \"0ffa4505-bb77-4d31-8e7d-7926eae343fd\" (UID: \"0ffa4505-bb77-4d31-8e7d-7926eae343fd\") " Dec 03 12:38:37 crc kubenswrapper[5052]: I1203 12:38:37.251977 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/0ffa4505-bb77-4d31-8e7d-7926eae343fd-config-data\") pod \"0ffa4505-bb77-4d31-8e7d-7926eae343fd\" (UID: \"0ffa4505-bb77-4d31-8e7d-7926eae343fd\") " Dec 03 12:38:37 crc kubenswrapper[5052]: I1203 12:38:37.252043 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pmkx7\" (UniqueName: \"kubernetes.io/projected/0ffa4505-bb77-4d31-8e7d-7926eae343fd-kube-api-access-pmkx7\") pod \"0ffa4505-bb77-4d31-8e7d-7926eae343fd\" (UID: \"0ffa4505-bb77-4d31-8e7d-7926eae343fd\") " Dec 03 12:38:37 crc kubenswrapper[5052]: I1203 12:38:37.259785 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0ffa4505-bb77-4d31-8e7d-7926eae343fd-kube-api-access-pmkx7" (OuterVolumeSpecName: "kube-api-access-pmkx7") pod "0ffa4505-bb77-4d31-8e7d-7926eae343fd" (UID: "0ffa4505-bb77-4d31-8e7d-7926eae343fd"). InnerVolumeSpecName "kube-api-access-pmkx7". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:38:37 crc kubenswrapper[5052]: I1203 12:38:37.282837 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0ffa4505-bb77-4d31-8e7d-7926eae343fd-config-data" (OuterVolumeSpecName: "config-data") pod "0ffa4505-bb77-4d31-8e7d-7926eae343fd" (UID: "0ffa4505-bb77-4d31-8e7d-7926eae343fd"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:38:37 crc kubenswrapper[5052]: I1203 12:38:37.291140 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0ffa4505-bb77-4d31-8e7d-7926eae343fd-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "0ffa4505-bb77-4d31-8e7d-7926eae343fd" (UID: "0ffa4505-bb77-4d31-8e7d-7926eae343fd"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:38:37 crc kubenswrapper[5052]: I1203 12:38:37.354460 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/0ffa4505-bb77-4d31-8e7d-7926eae343fd-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:38:37 crc kubenswrapper[5052]: I1203 12:38:37.354492 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-pmkx7\" (UniqueName: \"kubernetes.io/projected/0ffa4505-bb77-4d31-8e7d-7926eae343fd-kube-api-access-pmkx7\") on node \"crc\" DevicePath \"\"" Dec 03 12:38:37 crc kubenswrapper[5052]: I1203 12:38:37.354507 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0ffa4505-bb77-4d31-8e7d-7926eae343fd-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:38:37 crc kubenswrapper[5052]: E1203 12:38:37.354631 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-glance-default-public-svc: secret "cert-glance-default-public-svc" not found Dec 03 12:38:37 crc kubenswrapper[5052]: E1203 12:38:37.354680 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9dccead0-7ff2-4255-ac00-10a9b357c2db-public-tls-certs podName:9dccead0-7ff2-4255-ac00-10a9b357c2db nodeName:}" failed. No retries permitted until 2025-12-03 12:38:39.35466412 +0000 UTC m=+4709.942905884 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "public-tls-certs" (UniqueName: "kubernetes.io/secret/9dccead0-7ff2-4255-ac00-10a9b357c2db-public-tls-certs") pod "glance-default-external-api-0" (UID: "9dccead0-7ff2-4255-ac00-10a9b357c2db") : secret "cert-glance-default-public-svc" not found Dec 03 12:38:37 crc kubenswrapper[5052]: I1203 12:38:37.432772 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"edea8e53-340d-4f1e-99e5-082268c64ceb","Type":"ContainerStarted","Data":"e60c32d29dd74ca93adc4e272d8de99081bbf8e5961fe48ef0a2093869b9682e"} Dec 03 12:38:37 crc kubenswrapper[5052]: I1203 12:38:37.432821 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"edea8e53-340d-4f1e-99e5-082268c64ceb","Type":"ContainerStarted","Data":"ebef114a01de4517efb29228cf3e1863ae4862f9108584e0350c29841b2315c3"} Dec 03 12:38:37 crc kubenswrapper[5052]: I1203 12:38:37.432834 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"edea8e53-340d-4f1e-99e5-082268c64ceb","Type":"ContainerStarted","Data":"31fbf4be90f62cd778687f590b181dfa7ad9d21ee0c96fd40c93756d8d87ddbc"} Dec 03 12:38:37 crc kubenswrapper[5052]: I1203 12:38:37.438099 5052 generic.go:334] "Generic (PLEG): container finished" podID="1cc13873-68c8-4c70-89dc-caefaeb40e25" containerID="34697f0f6773a06387fe7e2f667823c92555f8d2c3bcc6f8942a7d5f2769f8ed" exitCode=0 Dec 03 12:38:37 crc kubenswrapper[5052]: I1203 12:38:37.438209 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-cell1-galera-0" event={"ID":"1cc13873-68c8-4c70-89dc-caefaeb40e25","Type":"ContainerDied","Data":"34697f0f6773a06387fe7e2f667823c92555f8d2c3bcc6f8942a7d5f2769f8ed"} Dec 03 12:38:37 crc kubenswrapper[5052]: I1203 12:38:37.455786 5052 generic.go:334] "Generic (PLEG): container finished" podID="0ffa4505-bb77-4d31-8e7d-7926eae343fd" containerID="faa257c722aaf243fc59be292d2c265e25f1a8976cfba25553abe4b2ea8c30f6" exitCode=0 Dec 03 12:38:37 crc kubenswrapper[5052]: I1203 12:38:37.456870 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:38:37 crc kubenswrapper[5052]: I1203 12:38:37.457052 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-metadata-0" podStartSLOduration=2.457040958 podStartE2EDuration="2.457040958s" podCreationTimestamp="2025-12-03 12:38:35 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:38:37.450000137 +0000 UTC m=+4708.038241901" watchObservedRunningTime="2025-12-03 12:38:37.457040958 +0000 UTC m=+4708.045282722" Dec 03 12:38:37 crc kubenswrapper[5052]: I1203 12:38:37.457137 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-conductor-0" event={"ID":"0ffa4505-bb77-4d31-8e7d-7926eae343fd","Type":"ContainerDied","Data":"faa257c722aaf243fc59be292d2c265e25f1a8976cfba25553abe4b2ea8c30f6"} Dec 03 12:38:37 crc kubenswrapper[5052]: I1203 12:38:37.457191 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-conductor-0" event={"ID":"0ffa4505-bb77-4d31-8e7d-7926eae343fd","Type":"ContainerDied","Data":"0f8b4c9ec56b842fda10122fea469713824491466b5906c88f5d6467555c4737"} Dec 03 12:38:37 crc kubenswrapper[5052]: I1203 12:38:37.457212 5052 scope.go:117] "RemoveContainer" containerID="faa257c722aaf243fc59be292d2c265e25f1a8976cfba25553abe4b2ea8c30f6" Dec 03 12:38:37 crc kubenswrapper[5052]: I1203 12:38:37.499162 5052 scope.go:117] "RemoveContainer" containerID="faa257c722aaf243fc59be292d2c265e25f1a8976cfba25553abe4b2ea8c30f6" Dec 03 12:38:37 crc kubenswrapper[5052]: E1203 12:38:37.509184 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"faa257c722aaf243fc59be292d2c265e25f1a8976cfba25553abe4b2ea8c30f6\": container with ID starting with faa257c722aaf243fc59be292d2c265e25f1a8976cfba25553abe4b2ea8c30f6 not found: ID does not exist" containerID="faa257c722aaf243fc59be292d2c265e25f1a8976cfba25553abe4b2ea8c30f6" Dec 03 12:38:37 crc kubenswrapper[5052]: I1203 12:38:37.509235 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"faa257c722aaf243fc59be292d2c265e25f1a8976cfba25553abe4b2ea8c30f6"} err="failed to get container status \"faa257c722aaf243fc59be292d2c265e25f1a8976cfba25553abe4b2ea8c30f6\": rpc error: code = NotFound desc = could not find container \"faa257c722aaf243fc59be292d2c265e25f1a8976cfba25553abe4b2ea8c30f6\": container with ID starting with faa257c722aaf243fc59be292d2c265e25f1a8976cfba25553abe4b2ea8c30f6 not found: ID does not exist" Dec 03 12:38:37 crc kubenswrapper[5052]: I1203 12:38:37.512114 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-0"] Dec 03 12:38:37 crc kubenswrapper[5052]: I1203 12:38:37.545005 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-0"] Dec 03 12:38:37 crc kubenswrapper[5052]: I1203 12:38:37.564897 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-0"] Dec 03 12:38:37 crc kubenswrapper[5052]: E1203 12:38:37.565556 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="0ffa4505-bb77-4d31-8e7d-7926eae343fd" containerName="nova-cell1-conductor-conductor" Dec 03 12:38:37 crc kubenswrapper[5052]: I1203 12:38:37.565582 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="0ffa4505-bb77-4d31-8e7d-7926eae343fd" containerName="nova-cell1-conductor-conductor" Dec 03 12:38:37 crc kubenswrapper[5052]: I1203 12:38:37.565950 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="0ffa4505-bb77-4d31-8e7d-7926eae343fd" containerName="nova-cell1-conductor-conductor" Dec 03 12:38:37 crc kubenswrapper[5052]: I1203 12:38:37.566749 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:38:37 crc kubenswrapper[5052]: I1203 12:38:37.569491 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell1-conductor-config-data" Dec 03 12:38:37 crc kubenswrapper[5052]: I1203 12:38:37.584752 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-0"] Dec 03 12:38:37 crc kubenswrapper[5052]: I1203 12:38:37.666375 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0556e9f0-e431-4193-9822-6079cb8a1e44-combined-ca-bundle\") pod \"nova-cell1-conductor-0\" (UID: \"0556e9f0-e431-4193-9822-6079cb8a1e44\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:38:37 crc kubenswrapper[5052]: I1203 12:38:37.666571 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-prg4s\" (UniqueName: \"kubernetes.io/projected/0556e9f0-e431-4193-9822-6079cb8a1e44-kube-api-access-prg4s\") pod \"nova-cell1-conductor-0\" (UID: \"0556e9f0-e431-4193-9822-6079cb8a1e44\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:38:37 crc kubenswrapper[5052]: I1203 12:38:37.666615 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/0556e9f0-e431-4193-9822-6079cb8a1e44-config-data\") pod \"nova-cell1-conductor-0\" (UID: \"0556e9f0-e431-4193-9822-6079cb8a1e44\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:38:37 crc kubenswrapper[5052]: I1203 12:38:37.701226 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0ffa4505-bb77-4d31-8e7d-7926eae343fd" path="/var/lib/kubelet/pods/0ffa4505-bb77-4d31-8e7d-7926eae343fd/volumes" Dec 03 12:38:37 crc kubenswrapper[5052]: I1203 12:38:37.768309 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0556e9f0-e431-4193-9822-6079cb8a1e44-combined-ca-bundle\") pod \"nova-cell1-conductor-0\" (UID: \"0556e9f0-e431-4193-9822-6079cb8a1e44\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:38:37 crc kubenswrapper[5052]: I1203 12:38:37.768702 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-prg4s\" (UniqueName: \"kubernetes.io/projected/0556e9f0-e431-4193-9822-6079cb8a1e44-kube-api-access-prg4s\") pod \"nova-cell1-conductor-0\" (UID: \"0556e9f0-e431-4193-9822-6079cb8a1e44\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:38:37 crc kubenswrapper[5052]: I1203 12:38:37.768802 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/0556e9f0-e431-4193-9822-6079cb8a1e44-config-data\") pod \"nova-cell1-conductor-0\" (UID: \"0556e9f0-e431-4193-9822-6079cb8a1e44\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:38:37 crc kubenswrapper[5052]: I1203 12:38:37.775441 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0556e9f0-e431-4193-9822-6079cb8a1e44-combined-ca-bundle\") pod \"nova-cell1-conductor-0\" (UID: \"0556e9f0-e431-4193-9822-6079cb8a1e44\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:38:37 crc kubenswrapper[5052]: I1203 12:38:37.778539 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/0556e9f0-e431-4193-9822-6079cb8a1e44-config-data\") pod \"nova-cell1-conductor-0\" (UID: \"0556e9f0-e431-4193-9822-6079cb8a1e44\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:38:37 crc kubenswrapper[5052]: I1203 12:38:37.791694 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-prg4s\" (UniqueName: \"kubernetes.io/projected/0556e9f0-e431-4193-9822-6079cb8a1e44-kube-api-access-prg4s\") pod \"nova-cell1-conductor-0\" (UID: \"0556e9f0-e431-4193-9822-6079cb8a1e44\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:38:37 crc kubenswrapper[5052]: I1203 12:38:37.902095 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:38:38 crc kubenswrapper[5052]: I1203 12:38:38.484877 5052 generic.go:334] "Generic (PLEG): container finished" podID="8a16aa42-962f-48cc-a7a8-d0e189164013" containerID="0c4e092293ea6adf3158ffe86080aa757175ad4a21f6d1b968c71ccec2b3a499" exitCode=0 Dec 03 12:38:38 crc kubenswrapper[5052]: I1203 12:38:38.484968 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"8a16aa42-962f-48cc-a7a8-d0e189164013","Type":"ContainerDied","Data":"0c4e092293ea6adf3158ffe86080aa757175ad4a21f6d1b968c71ccec2b3a499"} Dec 03 12:38:38 crc kubenswrapper[5052]: I1203 12:38:38.485276 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"8a16aa42-962f-48cc-a7a8-d0e189164013","Type":"ContainerDied","Data":"747e157d7c371ec2fa20cd7081d12642550aa5ded0bd95490c4ab3beaf0b5bbf"} Dec 03 12:38:38 crc kubenswrapper[5052]: I1203 12:38:38.485289 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="747e157d7c371ec2fa20cd7081d12642550aa5ded0bd95490c4ab3beaf0b5bbf" Dec 03 12:38:38 crc kubenswrapper[5052]: I1203 12:38:38.488076 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-cell1-galera-0" event={"ID":"1cc13873-68c8-4c70-89dc-caefaeb40e25","Type":"ContainerDied","Data":"26dfde733a8efe039801ca8fa714dd066e4aef78697aadfba4be17b4393e5118"} Dec 03 12:38:38 crc kubenswrapper[5052]: I1203 12:38:38.488117 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="26dfde733a8efe039801ca8fa714dd066e4aef78697aadfba4be17b4393e5118" Dec 03 12:38:38 crc kubenswrapper[5052]: I1203 12:38:38.683668 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:38:38 crc kubenswrapper[5052]: I1203 12:38:38.701973 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:38:38 crc kubenswrapper[5052]: I1203 12:38:38.802494 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1cc13873-68c8-4c70-89dc-caefaeb40e25-combined-ca-bundle\") pod \"1cc13873-68c8-4c70-89dc-caefaeb40e25\" (UID: \"1cc13873-68c8-4c70-89dc-caefaeb40e25\") " Dec 03 12:38:38 crc kubenswrapper[5052]: I1203 12:38:38.802595 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/1cc13873-68c8-4c70-89dc-caefaeb40e25-galera-tls-certs\") pod \"1cc13873-68c8-4c70-89dc-caefaeb40e25\" (UID: \"1cc13873-68c8-4c70-89dc-caefaeb40e25\") " Dec 03 12:38:38 crc kubenswrapper[5052]: I1203 12:38:38.802632 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/8a16aa42-962f-48cc-a7a8-d0e189164013-config-data\") pod \"8a16aa42-962f-48cc-a7a8-d0e189164013\" (UID: \"8a16aa42-962f-48cc-a7a8-d0e189164013\") " Dec 03 12:38:38 crc kubenswrapper[5052]: I1203 12:38:38.802667 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8a16aa42-962f-48cc-a7a8-d0e189164013-combined-ca-bundle\") pod \"8a16aa42-962f-48cc-a7a8-d0e189164013\" (UID: \"8a16aa42-962f-48cc-a7a8-d0e189164013\") " Dec 03 12:38:38 crc kubenswrapper[5052]: I1203 12:38:38.802717 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"mysql-db\" (UniqueName: \"kubernetes.io/local-volume/local-storage11-crc\") pod \"1cc13873-68c8-4c70-89dc-caefaeb40e25\" (UID: \"1cc13873-68c8-4c70-89dc-caefaeb40e25\") " Dec 03 12:38:38 crc kubenswrapper[5052]: I1203 12:38:38.802786 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/1cc13873-68c8-4c70-89dc-caefaeb40e25-config-data-default\") pod \"1cc13873-68c8-4c70-89dc-caefaeb40e25\" (UID: \"1cc13873-68c8-4c70-89dc-caefaeb40e25\") " Dec 03 12:38:38 crc kubenswrapper[5052]: I1203 12:38:38.802854 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/8a16aa42-962f-48cc-a7a8-d0e189164013-config-data-custom\") pod \"8a16aa42-962f-48cc-a7a8-d0e189164013\" (UID: \"8a16aa42-962f-48cc-a7a8-d0e189164013\") " Dec 03 12:38:38 crc kubenswrapper[5052]: I1203 12:38:38.802884 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/8a16aa42-962f-48cc-a7a8-d0e189164013-public-tls-certs\") pod \"8a16aa42-962f-48cc-a7a8-d0e189164013\" (UID: \"8a16aa42-962f-48cc-a7a8-d0e189164013\") " Dec 03 12:38:38 crc kubenswrapper[5052]: I1203 12:38:38.802944 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/1cc13873-68c8-4c70-89dc-caefaeb40e25-config-data-generated\") pod \"1cc13873-68c8-4c70-89dc-caefaeb40e25\" (UID: \"1cc13873-68c8-4c70-89dc-caefaeb40e25\") " Dec 03 12:38:38 crc kubenswrapper[5052]: I1203 12:38:38.802972 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/8a16aa42-962f-48cc-a7a8-d0e189164013-internal-tls-certs\") pod \"8a16aa42-962f-48cc-a7a8-d0e189164013\" (UID: \"8a16aa42-962f-48cc-a7a8-d0e189164013\") " Dec 03 12:38:38 crc kubenswrapper[5052]: I1203 12:38:38.803028 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wskcq\" (UniqueName: \"kubernetes.io/projected/1cc13873-68c8-4c70-89dc-caefaeb40e25-kube-api-access-wskcq\") pod \"1cc13873-68c8-4c70-89dc-caefaeb40e25\" (UID: \"1cc13873-68c8-4c70-89dc-caefaeb40e25\") " Dec 03 12:38:38 crc kubenswrapper[5052]: I1203 12:38:38.803725 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1cc13873-68c8-4c70-89dc-caefaeb40e25-config-data-default" (OuterVolumeSpecName: "config-data-default") pod "1cc13873-68c8-4c70-89dc-caefaeb40e25" (UID: "1cc13873-68c8-4c70-89dc-caefaeb40e25"). InnerVolumeSpecName "config-data-default". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:38:38 crc kubenswrapper[5052]: I1203 12:38:38.803849 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1cc13873-68c8-4c70-89dc-caefaeb40e25-config-data-generated" (OuterVolumeSpecName: "config-data-generated") pod "1cc13873-68c8-4c70-89dc-caefaeb40e25" (UID: "1cc13873-68c8-4c70-89dc-caefaeb40e25"). InnerVolumeSpecName "config-data-generated". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:38:38 crc kubenswrapper[5052]: I1203 12:38:38.807641 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/1cc13873-68c8-4c70-89dc-caefaeb40e25-kolla-config\") pod \"1cc13873-68c8-4c70-89dc-caefaeb40e25\" (UID: \"1cc13873-68c8-4c70-89dc-caefaeb40e25\") " Dec 03 12:38:38 crc kubenswrapper[5052]: I1203 12:38:38.807966 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/8a16aa42-962f-48cc-a7a8-d0e189164013-etc-machine-id\") pod \"8a16aa42-962f-48cc-a7a8-d0e189164013\" (UID: \"8a16aa42-962f-48cc-a7a8-d0e189164013\") " Dec 03 12:38:38 crc kubenswrapper[5052]: I1203 12:38:38.808010 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/1cc13873-68c8-4c70-89dc-caefaeb40e25-operator-scripts\") pod \"1cc13873-68c8-4c70-89dc-caefaeb40e25\" (UID: \"1cc13873-68c8-4c70-89dc-caefaeb40e25\") " Dec 03 12:38:38 crc kubenswrapper[5052]: I1203 12:38:38.808051 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-npsxr\" (UniqueName: \"kubernetes.io/projected/8a16aa42-962f-48cc-a7a8-d0e189164013-kube-api-access-npsxr\") pod \"8a16aa42-962f-48cc-a7a8-d0e189164013\" (UID: \"8a16aa42-962f-48cc-a7a8-d0e189164013\") " Dec 03 12:38:38 crc kubenswrapper[5052]: I1203 12:38:38.808100 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/8a16aa42-962f-48cc-a7a8-d0e189164013-scripts\") pod \"8a16aa42-962f-48cc-a7a8-d0e189164013\" (UID: \"8a16aa42-962f-48cc-a7a8-d0e189164013\") " Dec 03 12:38:38 crc kubenswrapper[5052]: I1203 12:38:38.808175 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/8a16aa42-962f-48cc-a7a8-d0e189164013-logs\") pod \"8a16aa42-962f-48cc-a7a8-d0e189164013\" (UID: \"8a16aa42-962f-48cc-a7a8-d0e189164013\") " Dec 03 12:38:38 crc kubenswrapper[5052]: I1203 12:38:38.809285 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/1cc13873-68c8-4c70-89dc-caefaeb40e25-config-data-default\") on node \"crc\" DevicePath \"\"" Dec 03 12:38:38 crc kubenswrapper[5052]: I1203 12:38:38.809313 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/1cc13873-68c8-4c70-89dc-caefaeb40e25-config-data-generated\") on node \"crc\" DevicePath \"\"" Dec 03 12:38:38 crc kubenswrapper[5052]: I1203 12:38:38.812605 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/8a16aa42-962f-48cc-a7a8-d0e189164013-etc-machine-id" (OuterVolumeSpecName: "etc-machine-id") pod "8a16aa42-962f-48cc-a7a8-d0e189164013" (UID: "8a16aa42-962f-48cc-a7a8-d0e189164013"). InnerVolumeSpecName "etc-machine-id". PluginName "kubernetes.io/host-path", VolumeGidValue "" Dec 03 12:38:38 crc kubenswrapper[5052]: I1203 12:38:38.813000 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1cc13873-68c8-4c70-89dc-caefaeb40e25-kolla-config" (OuterVolumeSpecName: "kolla-config") pod "1cc13873-68c8-4c70-89dc-caefaeb40e25" (UID: "1cc13873-68c8-4c70-89dc-caefaeb40e25"). InnerVolumeSpecName "kolla-config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:38:38 crc kubenswrapper[5052]: I1203 12:38:38.813557 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1cc13873-68c8-4c70-89dc-caefaeb40e25-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "1cc13873-68c8-4c70-89dc-caefaeb40e25" (UID: "1cc13873-68c8-4c70-89dc-caefaeb40e25"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:38:38 crc kubenswrapper[5052]: I1203 12:38:38.814660 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8a16aa42-962f-48cc-a7a8-d0e189164013-logs" (OuterVolumeSpecName: "logs") pod "8a16aa42-962f-48cc-a7a8-d0e189164013" (UID: "8a16aa42-962f-48cc-a7a8-d0e189164013"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:38:38 crc kubenswrapper[5052]: I1203 12:38:38.819904 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1cc13873-68c8-4c70-89dc-caefaeb40e25-kube-api-access-wskcq" (OuterVolumeSpecName: "kube-api-access-wskcq") pod "1cc13873-68c8-4c70-89dc-caefaeb40e25" (UID: "1cc13873-68c8-4c70-89dc-caefaeb40e25"). InnerVolumeSpecName "kube-api-access-wskcq". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:38:38 crc kubenswrapper[5052]: I1203 12:38:38.821754 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8a16aa42-962f-48cc-a7a8-d0e189164013-config-data-custom" (OuterVolumeSpecName: "config-data-custom") pod "8a16aa42-962f-48cc-a7a8-d0e189164013" (UID: "8a16aa42-962f-48cc-a7a8-d0e189164013"). InnerVolumeSpecName "config-data-custom". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:38:38 crc kubenswrapper[5052]: I1203 12:38:38.822723 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8a16aa42-962f-48cc-a7a8-d0e189164013-kube-api-access-npsxr" (OuterVolumeSpecName: "kube-api-access-npsxr") pod "8a16aa42-962f-48cc-a7a8-d0e189164013" (UID: "8a16aa42-962f-48cc-a7a8-d0e189164013"). InnerVolumeSpecName "kube-api-access-npsxr". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:38:38 crc kubenswrapper[5052]: I1203 12:38:38.835673 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8a16aa42-962f-48cc-a7a8-d0e189164013-scripts" (OuterVolumeSpecName: "scripts") pod "8a16aa42-962f-48cc-a7a8-d0e189164013" (UID: "8a16aa42-962f-48cc-a7a8-d0e189164013"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:38:38 crc kubenswrapper[5052]: I1203 12:38:38.856180 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage11-crc" (OuterVolumeSpecName: "mysql-db") pod "1cc13873-68c8-4c70-89dc-caefaeb40e25" (UID: "1cc13873-68c8-4c70-89dc-caefaeb40e25"). InnerVolumeSpecName "local-storage11-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 12:38:38 crc kubenswrapper[5052]: I1203 12:38:38.856670 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8a16aa42-962f-48cc-a7a8-d0e189164013-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "8a16aa42-962f-48cc-a7a8-d0e189164013" (UID: "8a16aa42-962f-48cc-a7a8-d0e189164013"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:38:38 crc kubenswrapper[5052]: I1203 12:38:38.874356 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1cc13873-68c8-4c70-89dc-caefaeb40e25-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "1cc13873-68c8-4c70-89dc-caefaeb40e25" (UID: "1cc13873-68c8-4c70-89dc-caefaeb40e25"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:38:38 crc kubenswrapper[5052]: I1203 12:38:38.898665 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8a16aa42-962f-48cc-a7a8-d0e189164013-config-data" (OuterVolumeSpecName: "config-data") pod "8a16aa42-962f-48cc-a7a8-d0e189164013" (UID: "8a16aa42-962f-48cc-a7a8-d0e189164013"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:38:38 crc kubenswrapper[5052]: I1203 12:38:38.914955 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/8a16aa42-962f-48cc-a7a8-d0e189164013-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:38:38 crc kubenswrapper[5052]: I1203 12:38:38.914990 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8a16aa42-962f-48cc-a7a8-d0e189164013-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:38:38 crc kubenswrapper[5052]: I1203 12:38:38.915011 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage11-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage11-crc\") on node \"crc\" " Dec 03 12:38:38 crc kubenswrapper[5052]: I1203 12:38:38.915021 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/8a16aa42-962f-48cc-a7a8-d0e189164013-config-data-custom\") on node \"crc\" DevicePath \"\"" Dec 03 12:38:38 crc kubenswrapper[5052]: I1203 12:38:38.915033 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-wskcq\" (UniqueName: \"kubernetes.io/projected/1cc13873-68c8-4c70-89dc-caefaeb40e25-kube-api-access-wskcq\") on node \"crc\" DevicePath \"\"" Dec 03 12:38:38 crc kubenswrapper[5052]: I1203 12:38:38.915043 5052 reconciler_common.go:293] "Volume detached for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/1cc13873-68c8-4c70-89dc-caefaeb40e25-kolla-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:38:38 crc kubenswrapper[5052]: I1203 12:38:38.915051 5052 reconciler_common.go:293] "Volume detached for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/8a16aa42-962f-48cc-a7a8-d0e189164013-etc-machine-id\") on node \"crc\" DevicePath \"\"" Dec 03 12:38:38 crc kubenswrapper[5052]: I1203 12:38:38.915060 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/1cc13873-68c8-4c70-89dc-caefaeb40e25-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:38:38 crc kubenswrapper[5052]: I1203 12:38:38.915068 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-npsxr\" (UniqueName: \"kubernetes.io/projected/8a16aa42-962f-48cc-a7a8-d0e189164013-kube-api-access-npsxr\") on node \"crc\" DevicePath \"\"" Dec 03 12:38:38 crc kubenswrapper[5052]: I1203 12:38:38.915076 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/8a16aa42-962f-48cc-a7a8-d0e189164013-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:38:38 crc kubenswrapper[5052]: I1203 12:38:38.915084 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/8a16aa42-962f-48cc-a7a8-d0e189164013-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:38:38 crc kubenswrapper[5052]: I1203 12:38:38.915092 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1cc13873-68c8-4c70-89dc-caefaeb40e25-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:38:38 crc kubenswrapper[5052]: I1203 12:38:38.919188 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1cc13873-68c8-4c70-89dc-caefaeb40e25-galera-tls-certs" (OuterVolumeSpecName: "galera-tls-certs") pod "1cc13873-68c8-4c70-89dc-caefaeb40e25" (UID: "1cc13873-68c8-4c70-89dc-caefaeb40e25"). InnerVolumeSpecName "galera-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:38:38 crc kubenswrapper[5052]: I1203 12:38:38.933168 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-0"] Dec 03 12:38:38 crc kubenswrapper[5052]: I1203 12:38:38.964342 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage11-crc" (UniqueName: "kubernetes.io/local-volume/local-storage11-crc") on node "crc" Dec 03 12:38:38 crc kubenswrapper[5052]: I1203 12:38:38.994349 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8a16aa42-962f-48cc-a7a8-d0e189164013-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "8a16aa42-962f-48cc-a7a8-d0e189164013" (UID: "8a16aa42-962f-48cc-a7a8-d0e189164013"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:38:38 crc kubenswrapper[5052]: I1203 12:38:38.999042 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8a16aa42-962f-48cc-a7a8-d0e189164013-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "8a16aa42-962f-48cc-a7a8-d0e189164013" (UID: "8a16aa42-962f-48cc-a7a8-d0e189164013"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:38:39 crc kubenswrapper[5052]: I1203 12:38:39.018647 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage11-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage11-crc\") on node \"crc\" DevicePath \"\"" Dec 03 12:38:39 crc kubenswrapper[5052]: I1203 12:38:39.018684 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/8a16aa42-962f-48cc-a7a8-d0e189164013-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:38:39 crc kubenswrapper[5052]: I1203 12:38:39.018698 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/8a16aa42-962f-48cc-a7a8-d0e189164013-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:38:39 crc kubenswrapper[5052]: I1203 12:38:39.018709 5052 reconciler_common.go:293] "Volume detached for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/1cc13873-68c8-4c70-89dc-caefaeb40e25-galera-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:38:39 crc kubenswrapper[5052]: I1203 12:38:39.377554 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:38:39 crc kubenswrapper[5052]: I1203 12:38:39.426639 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/b0baf287-dc5f-45d0-b09d-324933082b9e-config-data\") pod \"b0baf287-dc5f-45d0-b09d-324933082b9e\" (UID: \"b0baf287-dc5f-45d0-b09d-324933082b9e\") " Dec 03 12:38:39 crc kubenswrapper[5052]: I1203 12:38:39.426820 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b0baf287-dc5f-45d0-b09d-324933082b9e-combined-ca-bundle\") pod \"b0baf287-dc5f-45d0-b09d-324933082b9e\" (UID: \"b0baf287-dc5f-45d0-b09d-324933082b9e\") " Dec 03 12:38:39 crc kubenswrapper[5052]: I1203 12:38:39.426876 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m2kgw\" (UniqueName: \"kubernetes.io/projected/b0baf287-dc5f-45d0-b09d-324933082b9e-kube-api-access-m2kgw\") pod \"b0baf287-dc5f-45d0-b09d-324933082b9e\" (UID: \"b0baf287-dc5f-45d0-b09d-324933082b9e\") " Dec 03 12:38:39 crc kubenswrapper[5052]: E1203 12:38:39.427466 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-galera-openstack-svc: secret "cert-galera-openstack-svc" not found Dec 03 12:38:39 crc kubenswrapper[5052]: E1203 12:38:39.427535 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/01fb7120-3521-4969-a2c4-b2637e0e2414-galera-tls-certs podName:01fb7120-3521-4969-a2c4-b2637e0e2414 nodeName:}" failed. No retries permitted until 2025-12-03 12:38:55.42750594 +0000 UTC m=+4726.015747704 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "galera-tls-certs" (UniqueName: "kubernetes.io/secret/01fb7120-3521-4969-a2c4-b2637e0e2414-galera-tls-certs") pod "openstack-galera-0" (UID: "01fb7120-3521-4969-a2c4-b2637e0e2414") : secret "cert-galera-openstack-svc" not found Dec 03 12:38:39 crc kubenswrapper[5052]: E1203 12:38:39.427940 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-glance-default-public-svc: secret "cert-glance-default-public-svc" not found Dec 03 12:38:39 crc kubenswrapper[5052]: E1203 12:38:39.427971 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9dccead0-7ff2-4255-ac00-10a9b357c2db-public-tls-certs podName:9dccead0-7ff2-4255-ac00-10a9b357c2db nodeName:}" failed. No retries permitted until 2025-12-03 12:38:43.427963153 +0000 UTC m=+4714.016204917 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "public-tls-certs" (UniqueName: "kubernetes.io/secret/9dccead0-7ff2-4255-ac00-10a9b357c2db-public-tls-certs") pod "glance-default-external-api-0" (UID: "9dccead0-7ff2-4255-ac00-10a9b357c2db") : secret "cert-glance-default-public-svc" not found Dec 03 12:38:39 crc kubenswrapper[5052]: I1203 12:38:39.432632 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b0baf287-dc5f-45d0-b09d-324933082b9e-kube-api-access-m2kgw" (OuterVolumeSpecName: "kube-api-access-m2kgw") pod "b0baf287-dc5f-45d0-b09d-324933082b9e" (UID: "b0baf287-dc5f-45d0-b09d-324933082b9e"). InnerVolumeSpecName "kube-api-access-m2kgw". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:38:39 crc kubenswrapper[5052]: I1203 12:38:39.454980 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b0baf287-dc5f-45d0-b09d-324933082b9e-config-data" (OuterVolumeSpecName: "config-data") pod "b0baf287-dc5f-45d0-b09d-324933082b9e" (UID: "b0baf287-dc5f-45d0-b09d-324933082b9e"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:38:39 crc kubenswrapper[5052]: I1203 12:38:39.460490 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b0baf287-dc5f-45d0-b09d-324933082b9e-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "b0baf287-dc5f-45d0-b09d-324933082b9e" (UID: "b0baf287-dc5f-45d0-b09d-324933082b9e"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:38:39 crc kubenswrapper[5052]: I1203 12:38:39.505372 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-conductor-0" event={"ID":"0556e9f0-e431-4193-9822-6079cb8a1e44","Type":"ContainerStarted","Data":"382891affaf25ff9b098ddaf284f3fa5667c14b93f3487974c23c2cdae354300"} Dec 03 12:38:39 crc kubenswrapper[5052]: I1203 12:38:39.505479 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-conductor-0" event={"ID":"0556e9f0-e431-4193-9822-6079cb8a1e44","Type":"ContainerStarted","Data":"712de07ce5eeaee31029705ab3d33cfdf189affff7d973b2e31342976b5cb79c"} Dec 03 12:38:39 crc kubenswrapper[5052]: I1203 12:38:39.506895 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:38:39 crc kubenswrapper[5052]: I1203 12:38:39.508946 5052 generic.go:334] "Generic (PLEG): container finished" podID="b0baf287-dc5f-45d0-b09d-324933082b9e" containerID="441e495d6dff11b2b90f0f4528ee17c8d1710a586189a25097a602eff7902fd7" exitCode=0 Dec 03 12:38:39 crc kubenswrapper[5052]: I1203 12:38:39.509044 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:38:39 crc kubenswrapper[5052]: I1203 12:38:39.509691 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:38:39 crc kubenswrapper[5052]: I1203 12:38:39.509806 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:38:39 crc kubenswrapper[5052]: I1203 12:38:39.521233 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"b0baf287-dc5f-45d0-b09d-324933082b9e","Type":"ContainerDied","Data":"441e495d6dff11b2b90f0f4528ee17c8d1710a586189a25097a602eff7902fd7"} Dec 03 12:38:39 crc kubenswrapper[5052]: I1203 12:38:39.521554 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"b0baf287-dc5f-45d0-b09d-324933082b9e","Type":"ContainerDied","Data":"369b7fc23b87c3915b9767772260a1ef51bf180abe5bb11dc2acd6fba66b6884"} Dec 03 12:38:39 crc kubenswrapper[5052]: I1203 12:38:39.521608 5052 scope.go:117] "RemoveContainer" containerID="441e495d6dff11b2b90f0f4528ee17c8d1710a586189a25097a602eff7902fd7" Dec 03 12:38:39 crc kubenswrapper[5052]: I1203 12:38:39.529868 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/b0baf287-dc5f-45d0-b09d-324933082b9e-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:38:39 crc kubenswrapper[5052]: I1203 12:38:39.529910 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b0baf287-dc5f-45d0-b09d-324933082b9e-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:38:39 crc kubenswrapper[5052]: I1203 12:38:39.529954 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-m2kgw\" (UniqueName: \"kubernetes.io/projected/b0baf287-dc5f-45d0-b09d-324933082b9e-kube-api-access-m2kgw\") on node \"crc\" DevicePath \"\"" Dec 03 12:38:39 crc kubenswrapper[5052]: I1203 12:38:39.543234 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-cell1-conductor-0" podStartSLOduration=2.5432119589999997 podStartE2EDuration="2.543211959s" podCreationTimestamp="2025-12-03 12:38:37 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:38:39.524882475 +0000 UTC m=+4710.113124259" watchObservedRunningTime="2025-12-03 12:38:39.543211959 +0000 UTC m=+4710.131453733" Dec 03 12:38:39 crc kubenswrapper[5052]: I1203 12:38:39.562685 5052 scope.go:117] "RemoveContainer" containerID="441e495d6dff11b2b90f0f4528ee17c8d1710a586189a25097a602eff7902fd7" Dec 03 12:38:39 crc kubenswrapper[5052]: E1203 12:38:39.566185 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"441e495d6dff11b2b90f0f4528ee17c8d1710a586189a25097a602eff7902fd7\": container with ID starting with 441e495d6dff11b2b90f0f4528ee17c8d1710a586189a25097a602eff7902fd7 not found: ID does not exist" containerID="441e495d6dff11b2b90f0f4528ee17c8d1710a586189a25097a602eff7902fd7" Dec 03 12:38:39 crc kubenswrapper[5052]: I1203 12:38:39.566227 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"441e495d6dff11b2b90f0f4528ee17c8d1710a586189a25097a602eff7902fd7"} err="failed to get container status \"441e495d6dff11b2b90f0f4528ee17c8d1710a586189a25097a602eff7902fd7\": rpc error: code = NotFound desc = could not find container \"441e495d6dff11b2b90f0f4528ee17c8d1710a586189a25097a602eff7902fd7\": container with ID starting with 441e495d6dff11b2b90f0f4528ee17c8d1710a586189a25097a602eff7902fd7 not found: ID does not exist" Dec 03 12:38:39 crc kubenswrapper[5052]: I1203 12:38:39.574104 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/openstack-cell1-galera-0"] Dec 03 12:38:39 crc kubenswrapper[5052]: I1203 12:38:39.600896 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/openstack-cell1-galera-0"] Dec 03 12:38:39 crc kubenswrapper[5052]: I1203 12:38:39.620897 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:38:39 crc kubenswrapper[5052]: I1203 12:38:39.633822 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:38:39 crc kubenswrapper[5052]: I1203 12:38:39.694668 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1cc13873-68c8-4c70-89dc-caefaeb40e25" path="/var/lib/kubelet/pods/1cc13873-68c8-4c70-89dc-caefaeb40e25/volumes" Dec 03 12:38:39 crc kubenswrapper[5052]: I1203 12:38:39.701085 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b0baf287-dc5f-45d0-b09d-324933082b9e" path="/var/lib/kubelet/pods/b0baf287-dc5f-45d0-b09d-324933082b9e/volumes" Dec 03 12:38:39 crc kubenswrapper[5052]: I1203 12:38:39.701783 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/openstack-cell1-galera-0"] Dec 03 12:38:39 crc kubenswrapper[5052]: E1203 12:38:39.702124 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="8a16aa42-962f-48cc-a7a8-d0e189164013" containerName="cinder-api-log" Dec 03 12:38:39 crc kubenswrapper[5052]: I1203 12:38:39.702142 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="8a16aa42-962f-48cc-a7a8-d0e189164013" containerName="cinder-api-log" Dec 03 12:38:39 crc kubenswrapper[5052]: E1203 12:38:39.702155 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="b0baf287-dc5f-45d0-b09d-324933082b9e" containerName="nova-scheduler-scheduler" Dec 03 12:38:39 crc kubenswrapper[5052]: I1203 12:38:39.702162 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="b0baf287-dc5f-45d0-b09d-324933082b9e" containerName="nova-scheduler-scheduler" Dec 03 12:38:39 crc kubenswrapper[5052]: E1203 12:38:39.702177 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="1cc13873-68c8-4c70-89dc-caefaeb40e25" containerName="galera" Dec 03 12:38:39 crc kubenswrapper[5052]: I1203 12:38:39.702183 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="1cc13873-68c8-4c70-89dc-caefaeb40e25" containerName="galera" Dec 03 12:38:39 crc kubenswrapper[5052]: E1203 12:38:39.702195 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="8a16aa42-962f-48cc-a7a8-d0e189164013" containerName="cinder-api" Dec 03 12:38:39 crc kubenswrapper[5052]: I1203 12:38:39.702201 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="8a16aa42-962f-48cc-a7a8-d0e189164013" containerName="cinder-api" Dec 03 12:38:39 crc kubenswrapper[5052]: E1203 12:38:39.702233 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="1cc13873-68c8-4c70-89dc-caefaeb40e25" containerName="mysql-bootstrap" Dec 03 12:38:39 crc kubenswrapper[5052]: I1203 12:38:39.702239 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="1cc13873-68c8-4c70-89dc-caefaeb40e25" containerName="mysql-bootstrap" Dec 03 12:38:39 crc kubenswrapper[5052]: I1203 12:38:39.702423 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="b0baf287-dc5f-45d0-b09d-324933082b9e" containerName="nova-scheduler-scheduler" Dec 03 12:38:39 crc kubenswrapper[5052]: I1203 12:38:39.702451 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="8a16aa42-962f-48cc-a7a8-d0e189164013" containerName="cinder-api-log" Dec 03 12:38:39 crc kubenswrapper[5052]: I1203 12:38:39.702463 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="1cc13873-68c8-4c70-89dc-caefaeb40e25" containerName="galera" Dec 03 12:38:39 crc kubenswrapper[5052]: I1203 12:38:39.702475 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="8a16aa42-962f-48cc-a7a8-d0e189164013" containerName="cinder-api" Dec 03 12:38:39 crc kubenswrapper[5052]: I1203 12:38:39.705030 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 12:38:39 crc kubenswrapper[5052]: I1203 12:38:39.705138 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:38:39 crc kubenswrapper[5052]: I1203 12:38:39.707077 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 12:38:39 crc kubenswrapper[5052]: I1203 12:38:39.708313 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"galera-openstack-cell1-dockercfg-4wff5" Dec 03 12:38:39 crc kubenswrapper[5052]: I1203 12:38:39.714583 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"openstack-cell1-config-data" Dec 03 12:38:39 crc kubenswrapper[5052]: I1203 12:38:39.714803 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-galera-openstack-cell1-svc" Dec 03 12:38:39 crc kubenswrapper[5052]: I1203 12:38:39.714931 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"openstack-cell1-scripts" Dec 03 12:38:39 crc kubenswrapper[5052]: I1203 12:38:39.719337 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:38:39 crc kubenswrapper[5052]: I1203 12:38:39.720580 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:38:39 crc kubenswrapper[5052]: I1203 12:38:39.722851 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-scheduler-config-data" Dec 03 12:38:39 crc kubenswrapper[5052]: I1203 12:38:39.745193 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:38:39 crc kubenswrapper[5052]: I1203 12:38:39.760455 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/openstack-cell1-galera-0"] Dec 03 12:38:39 crc kubenswrapper[5052]: I1203 12:38:39.780315 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 12:38:39 crc kubenswrapper[5052]: I1203 12:38:39.782775 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:38:39 crc kubenswrapper[5052]: I1203 12:38:39.785386 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-cinder-internal-svc" Dec 03 12:38:39 crc kubenswrapper[5052]: I1203 12:38:39.786256 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-cinder-public-svc" Dec 03 12:38:39 crc kubenswrapper[5052]: I1203 12:38:39.787179 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cinder-api-config-data" Dec 03 12:38:39 crc kubenswrapper[5052]: I1203 12:38:39.790120 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 12:38:39 crc kubenswrapper[5052]: I1203 12:38:39.843813 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/2253cde7-04d1-41a2-82d3-9c22cc6099c4-galera-tls-certs\") pod \"openstack-cell1-galera-0\" (UID: \"2253cde7-04d1-41a2-82d3-9c22cc6099c4\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:38:39 crc kubenswrapper[5052]: I1203 12:38:39.843960 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/93664241-e156-4225-980c-4dbe8aff3353-public-tls-certs\") pod \"cinder-api-0\" (UID: \"93664241-e156-4225-980c-4dbe8aff3353\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:38:39 crc kubenswrapper[5052]: I1203 12:38:39.843987 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9e3bfbf3-a39c-48d2-bb1d-062c90c0ef10-combined-ca-bundle\") pod \"nova-scheduler-0\" (UID: \"9e3bfbf3-a39c-48d2-bb1d-062c90c0ef10\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:38:39 crc kubenswrapper[5052]: I1203 12:38:39.844019 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/93664241-e156-4225-980c-4dbe8aff3353-internal-tls-certs\") pod \"cinder-api-0\" (UID: \"93664241-e156-4225-980c-4dbe8aff3353\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:38:39 crc kubenswrapper[5052]: I1203 12:38:39.844053 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/2253cde7-04d1-41a2-82d3-9c22cc6099c4-config-data-default\") pod \"openstack-cell1-galera-0\" (UID: \"2253cde7-04d1-41a2-82d3-9c22cc6099c4\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:38:39 crc kubenswrapper[5052]: I1203 12:38:39.844101 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage11-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage11-crc\") pod \"openstack-cell1-galera-0\" (UID: \"2253cde7-04d1-41a2-82d3-9c22cc6099c4\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:38:39 crc kubenswrapper[5052]: I1203 12:38:39.844127 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2253cde7-04d1-41a2-82d3-9c22cc6099c4-combined-ca-bundle\") pod \"openstack-cell1-galera-0\" (UID: \"2253cde7-04d1-41a2-82d3-9c22cc6099c4\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:38:39 crc kubenswrapper[5052]: I1203 12:38:39.844172 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xsp7z\" (UniqueName: \"kubernetes.io/projected/93664241-e156-4225-980c-4dbe8aff3353-kube-api-access-xsp7z\") pod \"cinder-api-0\" (UID: \"93664241-e156-4225-980c-4dbe8aff3353\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:38:39 crc kubenswrapper[5052]: I1203 12:38:39.844198 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/2253cde7-04d1-41a2-82d3-9c22cc6099c4-operator-scripts\") pod \"openstack-cell1-galera-0\" (UID: \"2253cde7-04d1-41a2-82d3-9c22cc6099c4\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:38:39 crc kubenswrapper[5052]: I1203 12:38:39.844221 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/93664241-e156-4225-980c-4dbe8aff3353-logs\") pod \"cinder-api-0\" (UID: \"93664241-e156-4225-980c-4dbe8aff3353\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:38:39 crc kubenswrapper[5052]: I1203 12:38:39.844246 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/93664241-e156-4225-980c-4dbe8aff3353-config-data\") pod \"cinder-api-0\" (UID: \"93664241-e156-4225-980c-4dbe8aff3353\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:38:39 crc kubenswrapper[5052]: I1203 12:38:39.844272 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/93664241-e156-4225-980c-4dbe8aff3353-combined-ca-bundle\") pod \"cinder-api-0\" (UID: \"93664241-e156-4225-980c-4dbe8aff3353\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:38:39 crc kubenswrapper[5052]: I1203 12:38:39.844312 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wj2jk\" (UniqueName: \"kubernetes.io/projected/9e3bfbf3-a39c-48d2-bb1d-062c90c0ef10-kube-api-access-wj2jk\") pod \"nova-scheduler-0\" (UID: \"9e3bfbf3-a39c-48d2-bb1d-062c90c0ef10\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:38:39 crc kubenswrapper[5052]: I1203 12:38:39.844345 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/2253cde7-04d1-41a2-82d3-9c22cc6099c4-kolla-config\") pod \"openstack-cell1-galera-0\" (UID: \"2253cde7-04d1-41a2-82d3-9c22cc6099c4\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:38:39 crc kubenswrapper[5052]: I1203 12:38:39.844381 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/93664241-e156-4225-980c-4dbe8aff3353-config-data-custom\") pod \"cinder-api-0\" (UID: \"93664241-e156-4225-980c-4dbe8aff3353\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:38:39 crc kubenswrapper[5052]: I1203 12:38:39.844415 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/9e3bfbf3-a39c-48d2-bb1d-062c90c0ef10-config-data\") pod \"nova-scheduler-0\" (UID: \"9e3bfbf3-a39c-48d2-bb1d-062c90c0ef10\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:38:39 crc kubenswrapper[5052]: I1203 12:38:39.844439 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ll69p\" (UniqueName: \"kubernetes.io/projected/2253cde7-04d1-41a2-82d3-9c22cc6099c4-kube-api-access-ll69p\") pod \"openstack-cell1-galera-0\" (UID: \"2253cde7-04d1-41a2-82d3-9c22cc6099c4\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:38:39 crc kubenswrapper[5052]: I1203 12:38:39.844464 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/93664241-e156-4225-980c-4dbe8aff3353-scripts\") pod \"cinder-api-0\" (UID: \"93664241-e156-4225-980c-4dbe8aff3353\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:38:39 crc kubenswrapper[5052]: I1203 12:38:39.844490 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/2253cde7-04d1-41a2-82d3-9c22cc6099c4-config-data-generated\") pod \"openstack-cell1-galera-0\" (UID: \"2253cde7-04d1-41a2-82d3-9c22cc6099c4\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:38:39 crc kubenswrapper[5052]: I1203 12:38:39.844596 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/93664241-e156-4225-980c-4dbe8aff3353-etc-machine-id\") pod \"cinder-api-0\" (UID: \"93664241-e156-4225-980c-4dbe8aff3353\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:38:39 crc kubenswrapper[5052]: I1203 12:38:39.946432 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-xsp7z\" (UniqueName: \"kubernetes.io/projected/93664241-e156-4225-980c-4dbe8aff3353-kube-api-access-xsp7z\") pod \"cinder-api-0\" (UID: \"93664241-e156-4225-980c-4dbe8aff3353\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:38:39 crc kubenswrapper[5052]: I1203 12:38:39.946486 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/2253cde7-04d1-41a2-82d3-9c22cc6099c4-operator-scripts\") pod \"openstack-cell1-galera-0\" (UID: \"2253cde7-04d1-41a2-82d3-9c22cc6099c4\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:38:39 crc kubenswrapper[5052]: I1203 12:38:39.946525 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/93664241-e156-4225-980c-4dbe8aff3353-logs\") pod \"cinder-api-0\" (UID: \"93664241-e156-4225-980c-4dbe8aff3353\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:38:39 crc kubenswrapper[5052]: I1203 12:38:39.946545 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/93664241-e156-4225-980c-4dbe8aff3353-config-data\") pod \"cinder-api-0\" (UID: \"93664241-e156-4225-980c-4dbe8aff3353\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:38:39 crc kubenswrapper[5052]: I1203 12:38:39.947181 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/93664241-e156-4225-980c-4dbe8aff3353-logs\") pod \"cinder-api-0\" (UID: \"93664241-e156-4225-980c-4dbe8aff3353\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:38:39 crc kubenswrapper[5052]: I1203 12:38:39.946568 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/93664241-e156-4225-980c-4dbe8aff3353-combined-ca-bundle\") pod \"cinder-api-0\" (UID: \"93664241-e156-4225-980c-4dbe8aff3353\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:38:39 crc kubenswrapper[5052]: I1203 12:38:39.947900 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-wj2jk\" (UniqueName: \"kubernetes.io/projected/9e3bfbf3-a39c-48d2-bb1d-062c90c0ef10-kube-api-access-wj2jk\") pod \"nova-scheduler-0\" (UID: \"9e3bfbf3-a39c-48d2-bb1d-062c90c0ef10\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:38:39 crc kubenswrapper[5052]: I1203 12:38:39.947943 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/2253cde7-04d1-41a2-82d3-9c22cc6099c4-kolla-config\") pod \"openstack-cell1-galera-0\" (UID: \"2253cde7-04d1-41a2-82d3-9c22cc6099c4\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:38:39 crc kubenswrapper[5052]: I1203 12:38:39.947980 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/93664241-e156-4225-980c-4dbe8aff3353-config-data-custom\") pod \"cinder-api-0\" (UID: \"93664241-e156-4225-980c-4dbe8aff3353\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:38:39 crc kubenswrapper[5052]: I1203 12:38:39.948009 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/9e3bfbf3-a39c-48d2-bb1d-062c90c0ef10-config-data\") pod \"nova-scheduler-0\" (UID: \"9e3bfbf3-a39c-48d2-bb1d-062c90c0ef10\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:38:39 crc kubenswrapper[5052]: I1203 12:38:39.948030 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-ll69p\" (UniqueName: \"kubernetes.io/projected/2253cde7-04d1-41a2-82d3-9c22cc6099c4-kube-api-access-ll69p\") pod \"openstack-cell1-galera-0\" (UID: \"2253cde7-04d1-41a2-82d3-9c22cc6099c4\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:38:39 crc kubenswrapper[5052]: I1203 12:38:39.948047 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/93664241-e156-4225-980c-4dbe8aff3353-scripts\") pod \"cinder-api-0\" (UID: \"93664241-e156-4225-980c-4dbe8aff3353\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:38:39 crc kubenswrapper[5052]: I1203 12:38:39.948063 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/2253cde7-04d1-41a2-82d3-9c22cc6099c4-config-data-generated\") pod \"openstack-cell1-galera-0\" (UID: \"2253cde7-04d1-41a2-82d3-9c22cc6099c4\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:38:39 crc kubenswrapper[5052]: I1203 12:38:39.948133 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/93664241-e156-4225-980c-4dbe8aff3353-etc-machine-id\") pod \"cinder-api-0\" (UID: \"93664241-e156-4225-980c-4dbe8aff3353\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:38:39 crc kubenswrapper[5052]: I1203 12:38:39.949442 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/2253cde7-04d1-41a2-82d3-9c22cc6099c4-kolla-config\") pod \"openstack-cell1-galera-0\" (UID: \"2253cde7-04d1-41a2-82d3-9c22cc6099c4\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:38:39 crc kubenswrapper[5052]: I1203 12:38:39.949800 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/2253cde7-04d1-41a2-82d3-9c22cc6099c4-config-data-generated\") pod \"openstack-cell1-galera-0\" (UID: \"2253cde7-04d1-41a2-82d3-9c22cc6099c4\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:38:39 crc kubenswrapper[5052]: I1203 12:38:39.951290 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/93664241-e156-4225-980c-4dbe8aff3353-etc-machine-id\") pod \"cinder-api-0\" (UID: \"93664241-e156-4225-980c-4dbe8aff3353\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:38:39 crc kubenswrapper[5052]: I1203 12:38:39.956359 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/2253cde7-04d1-41a2-82d3-9c22cc6099c4-operator-scripts\") pod \"openstack-cell1-galera-0\" (UID: \"2253cde7-04d1-41a2-82d3-9c22cc6099c4\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:38:39 crc kubenswrapper[5052]: I1203 12:38:39.956541 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/2253cde7-04d1-41a2-82d3-9c22cc6099c4-galera-tls-certs\") pod \"openstack-cell1-galera-0\" (UID: \"2253cde7-04d1-41a2-82d3-9c22cc6099c4\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:38:39 crc kubenswrapper[5052]: I1203 12:38:39.957238 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/93664241-e156-4225-980c-4dbe8aff3353-combined-ca-bundle\") pod \"cinder-api-0\" (UID: \"93664241-e156-4225-980c-4dbe8aff3353\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:38:39 crc kubenswrapper[5052]: I1203 12:38:39.957369 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/93664241-e156-4225-980c-4dbe8aff3353-public-tls-certs\") pod \"cinder-api-0\" (UID: \"93664241-e156-4225-980c-4dbe8aff3353\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:38:39 crc kubenswrapper[5052]: I1203 12:38:39.957399 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9e3bfbf3-a39c-48d2-bb1d-062c90c0ef10-combined-ca-bundle\") pod \"nova-scheduler-0\" (UID: \"9e3bfbf3-a39c-48d2-bb1d-062c90c0ef10\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:38:39 crc kubenswrapper[5052]: I1203 12:38:39.957446 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/93664241-e156-4225-980c-4dbe8aff3353-internal-tls-certs\") pod \"cinder-api-0\" (UID: \"93664241-e156-4225-980c-4dbe8aff3353\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:38:39 crc kubenswrapper[5052]: I1203 12:38:39.957491 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/2253cde7-04d1-41a2-82d3-9c22cc6099c4-config-data-default\") pod \"openstack-cell1-galera-0\" (UID: \"2253cde7-04d1-41a2-82d3-9c22cc6099c4\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:38:39 crc kubenswrapper[5052]: I1203 12:38:39.957577 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage11-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage11-crc\") pod \"openstack-cell1-galera-0\" (UID: \"2253cde7-04d1-41a2-82d3-9c22cc6099c4\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:38:39 crc kubenswrapper[5052]: I1203 12:38:39.957615 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2253cde7-04d1-41a2-82d3-9c22cc6099c4-combined-ca-bundle\") pod \"openstack-cell1-galera-0\" (UID: \"2253cde7-04d1-41a2-82d3-9c22cc6099c4\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:38:39 crc kubenswrapper[5052]: I1203 12:38:39.959651 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/2253cde7-04d1-41a2-82d3-9c22cc6099c4-config-data-default\") pod \"openstack-cell1-galera-0\" (UID: \"2253cde7-04d1-41a2-82d3-9c22cc6099c4\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:38:39 crc kubenswrapper[5052]: I1203 12:38:39.963199 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/2253cde7-04d1-41a2-82d3-9c22cc6099c4-galera-tls-certs\") pod \"openstack-cell1-galera-0\" (UID: \"2253cde7-04d1-41a2-82d3-9c22cc6099c4\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:38:39 crc kubenswrapper[5052]: I1203 12:38:39.967416 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/9e3bfbf3-a39c-48d2-bb1d-062c90c0ef10-config-data\") pod \"nova-scheduler-0\" (UID: \"9e3bfbf3-a39c-48d2-bb1d-062c90c0ef10\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:38:39 crc kubenswrapper[5052]: I1203 12:38:39.967486 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/93664241-e156-4225-980c-4dbe8aff3353-internal-tls-certs\") pod \"cinder-api-0\" (UID: \"93664241-e156-4225-980c-4dbe8aff3353\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:38:39 crc kubenswrapper[5052]: I1203 12:38:39.969362 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/93664241-e156-4225-980c-4dbe8aff3353-config-data\") pod \"cinder-api-0\" (UID: \"93664241-e156-4225-980c-4dbe8aff3353\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:38:39 crc kubenswrapper[5052]: I1203 12:38:39.969713 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/93664241-e156-4225-980c-4dbe8aff3353-scripts\") pod \"cinder-api-0\" (UID: \"93664241-e156-4225-980c-4dbe8aff3353\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:38:39 crc kubenswrapper[5052]: I1203 12:38:39.969795 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9e3bfbf3-a39c-48d2-bb1d-062c90c0ef10-combined-ca-bundle\") pod \"nova-scheduler-0\" (UID: \"9e3bfbf3-a39c-48d2-bb1d-062c90c0ef10\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:38:39 crc kubenswrapper[5052]: I1203 12:38:39.969980 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2253cde7-04d1-41a2-82d3-9c22cc6099c4-combined-ca-bundle\") pod \"openstack-cell1-galera-0\" (UID: \"2253cde7-04d1-41a2-82d3-9c22cc6099c4\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:38:39 crc kubenswrapper[5052]: I1203 12:38:39.970173 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/93664241-e156-4225-980c-4dbe8aff3353-public-tls-certs\") pod \"cinder-api-0\" (UID: \"93664241-e156-4225-980c-4dbe8aff3353\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:38:39 crc kubenswrapper[5052]: I1203 12:38:39.971752 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-wj2jk\" (UniqueName: \"kubernetes.io/projected/9e3bfbf3-a39c-48d2-bb1d-062c90c0ef10-kube-api-access-wj2jk\") pod \"nova-scheduler-0\" (UID: \"9e3bfbf3-a39c-48d2-bb1d-062c90c0ef10\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:38:39 crc kubenswrapper[5052]: I1203 12:38:39.971873 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage11-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage11-crc\") pod \"openstack-cell1-galera-0\" (UID: \"2253cde7-04d1-41a2-82d3-9c22cc6099c4\") device mount path \"/mnt/openstack/pv11\"" pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:38:39 crc kubenswrapper[5052]: I1203 12:38:39.978494 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-ll69p\" (UniqueName: \"kubernetes.io/projected/2253cde7-04d1-41a2-82d3-9c22cc6099c4-kube-api-access-ll69p\") pod \"openstack-cell1-galera-0\" (UID: \"2253cde7-04d1-41a2-82d3-9c22cc6099c4\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:38:39 crc kubenswrapper[5052]: I1203 12:38:39.980445 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/93664241-e156-4225-980c-4dbe8aff3353-config-data-custom\") pod \"cinder-api-0\" (UID: \"93664241-e156-4225-980c-4dbe8aff3353\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:38:39 crc kubenswrapper[5052]: I1203 12:38:39.983155 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-xsp7z\" (UniqueName: \"kubernetes.io/projected/93664241-e156-4225-980c-4dbe8aff3353-kube-api-access-xsp7z\") pod \"cinder-api-0\" (UID: \"93664241-e156-4225-980c-4dbe8aff3353\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:38:40 crc kubenswrapper[5052]: I1203 12:38:40.016658 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage11-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage11-crc\") pod \"openstack-cell1-galera-0\" (UID: \"2253cde7-04d1-41a2-82d3-9c22cc6099c4\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:38:40 crc kubenswrapper[5052]: I1203 12:38:40.036569 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:38:40 crc kubenswrapper[5052]: I1203 12:38:40.059668 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:38:40 crc kubenswrapper[5052]: I1203 12:38:40.099346 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:38:40 crc kubenswrapper[5052]: I1203 12:38:40.292597 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/barbican-api-9bbb48f5b-4tfjv" Dec 03 12:38:40 crc kubenswrapper[5052]: I1203 12:38:40.306890 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/barbican-api-9bbb48f5b-4tfjv" Dec 03 12:38:40 crc kubenswrapper[5052]: I1203 12:38:40.411155 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-api-b857b994d-6xzbl"] Dec 03 12:38:40 crc kubenswrapper[5052]: I1203 12:38:40.411644 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/barbican-api-b857b994d-6xzbl" podUID="3ceac470-11eb-4d69-a683-2e9bc4e8f617" containerName="barbican-api-log" containerID="cri-o://1fe0c8f8d952b7b1bbdf2ddc25bf96bcb1eeeec47a75464c77ed9ef77f45fc86" gracePeriod=30 Dec 03 12:38:40 crc kubenswrapper[5052]: I1203 12:38:40.411718 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/barbican-api-b857b994d-6xzbl" podUID="3ceac470-11eb-4d69-a683-2e9bc4e8f617" containerName="barbican-api" containerID="cri-o://fc0930236187285aa345d3839c9e4d90b9b3da00e5015ee57851c76c3228e9b7" gracePeriod=30 Dec 03 12:38:40 crc kubenswrapper[5052]: E1203 12:38:40.469070 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-ovn-metrics: secret "cert-ovn-metrics" not found Dec 03 12:38:40 crc kubenswrapper[5052]: E1203 12:38:40.469139 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/143d0ab6-f833-41a9-aacd-34d20b35f8ed-metrics-certs-tls-certs podName:143d0ab6-f833-41a9-aacd-34d20b35f8ed nodeName:}" failed. No retries permitted until 2025-12-03 12:38:48.469125938 +0000 UTC m=+4719.057367692 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs-tls-certs" (UniqueName: "kubernetes.io/secret/143d0ab6-f833-41a9-aacd-34d20b35f8ed-metrics-certs-tls-certs") pod "ovsdbserver-sb-0" (UID: "143d0ab6-f833-41a9-aacd-34d20b35f8ed") : secret "cert-ovn-metrics" not found Dec 03 12:38:40 crc kubenswrapper[5052]: E1203 12:38:40.469484 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-ovndbcluster-sb-ovndbs: secret "cert-ovndbcluster-sb-ovndbs" not found Dec 03 12:38:40 crc kubenswrapper[5052]: E1203 12:38:40.469594 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/143d0ab6-f833-41a9-aacd-34d20b35f8ed-ovsdbserver-sb-tls-certs podName:143d0ab6-f833-41a9-aacd-34d20b35f8ed nodeName:}" failed. No retries permitted until 2025-12-03 12:38:48.469584251 +0000 UTC m=+4719.057826015 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "ovsdbserver-sb-tls-certs" (UniqueName: "kubernetes.io/secret/143d0ab6-f833-41a9-aacd-34d20b35f8ed-ovsdbserver-sb-tls-certs") pod "ovsdbserver-sb-0" (UID: "143d0ab6-f833-41a9-aacd-34d20b35f8ed") : secret "cert-ovndbcluster-sb-ovndbs" not found Dec 03 12:38:40 crc kubenswrapper[5052]: I1203 12:38:40.598039 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/openstack-cell1-galera-0"] Dec 03 12:38:40 crc kubenswrapper[5052]: W1203 12:38:40.610576 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2253cde7_04d1_41a2_82d3_9c22cc6099c4.slice/crio-3b64affb676a87f9e7e47b6b73e6e439f09cdb8f0e1aff76678a39a1611a3360 WatchSource:0}: Error finding container 3b64affb676a87f9e7e47b6b73e6e439f09cdb8f0e1aff76678a39a1611a3360: Status 404 returned error can't find the container with id 3b64affb676a87f9e7e47b6b73e6e439f09cdb8f0e1aff76678a39a1611a3360 Dec 03 12:38:40 crc kubenswrapper[5052]: W1203 12:38:40.741455 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod93664241_e156_4225_980c_4dbe8aff3353.slice/crio-e9aab40643fa964dffd0ed38e2df1e663e805d74053f0e1478502101a719137a WatchSource:0}: Error finding container e9aab40643fa964dffd0ed38e2df1e663e805d74053f0e1478502101a719137a: Status 404 returned error can't find the container with id e9aab40643fa964dffd0ed38e2df1e663e805d74053f0e1478502101a719137a Dec 03 12:38:40 crc kubenswrapper[5052]: I1203 12:38:40.743434 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:38:40 crc kubenswrapper[5052]: I1203 12:38:40.811375 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 12:38:40 crc kubenswrapper[5052]: I1203 12:38:40.958808 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:38:40 crc kubenswrapper[5052]: I1203 12:38:40.959637 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:38:41 crc kubenswrapper[5052]: I1203 12:38:41.319167 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:38:41 crc kubenswrapper[5052]: E1203 12:38:41.392822 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-kube-state-metrics-svc: secret "cert-kube-state-metrics-svc" not found Dec 03 12:38:41 crc kubenswrapper[5052]: E1203 12:38:41.392909 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b7defe22-28b4-4fec-8fe5-ebde8e658c6b-kube-state-metrics-tls-certs podName:b7defe22-28b4-4fec-8fe5-ebde8e658c6b nodeName:}" failed. No retries permitted until 2025-12-03 12:38:57.392887676 +0000 UTC m=+4727.981129470 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls-certs" (UniqueName: "kubernetes.io/secret/b7defe22-28b4-4fec-8fe5-ebde8e658c6b-kube-state-metrics-tls-certs") pod "kube-state-metrics-0" (UID: "b7defe22-28b4-4fec-8fe5-ebde8e658c6b") : secret "cert-kube-state-metrics-svc" not found Dec 03 12:38:41 crc kubenswrapper[5052]: I1203 12:38:41.542740 5052 generic.go:334] "Generic (PLEG): container finished" podID="3ceac470-11eb-4d69-a683-2e9bc4e8f617" containerID="1fe0c8f8d952b7b1bbdf2ddc25bf96bcb1eeeec47a75464c77ed9ef77f45fc86" exitCode=143 Dec 03 12:38:41 crc kubenswrapper[5052]: I1203 12:38:41.542862 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-b857b994d-6xzbl" event={"ID":"3ceac470-11eb-4d69-a683-2e9bc4e8f617","Type":"ContainerDied","Data":"1fe0c8f8d952b7b1bbdf2ddc25bf96bcb1eeeec47a75464c77ed9ef77f45fc86"} Dec 03 12:38:41 crc kubenswrapper[5052]: I1203 12:38:41.544602 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"93664241-e156-4225-980c-4dbe8aff3353","Type":"ContainerStarted","Data":"e9aab40643fa964dffd0ed38e2df1e663e805d74053f0e1478502101a719137a"} Dec 03 12:38:41 crc kubenswrapper[5052]: I1203 12:38:41.547272 5052 generic.go:334] "Generic (PLEG): container finished" podID="2253cde7-04d1-41a2-82d3-9c22cc6099c4" containerID="cef98b05ac915fca7faa283789b624d495e993a12f3a6d014308f6a6c2b30343" exitCode=0 Dec 03 12:38:41 crc kubenswrapper[5052]: I1203 12:38:41.547339 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-cell1-galera-0" event={"ID":"2253cde7-04d1-41a2-82d3-9c22cc6099c4","Type":"ContainerDied","Data":"cef98b05ac915fca7faa283789b624d495e993a12f3a6d014308f6a6c2b30343"} Dec 03 12:38:41 crc kubenswrapper[5052]: I1203 12:38:41.547364 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-cell1-galera-0" event={"ID":"2253cde7-04d1-41a2-82d3-9c22cc6099c4","Type":"ContainerStarted","Data":"3b64affb676a87f9e7e47b6b73e6e439f09cdb8f0e1aff76678a39a1611a3360"} Dec 03 12:38:41 crc kubenswrapper[5052]: I1203 12:38:41.557148 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"9e3bfbf3-a39c-48d2-bb1d-062c90c0ef10","Type":"ContainerStarted","Data":"b1ff1f57b80e78690f7e46060a438671dfcfacd08f98a788c2bd60d1eb2bbdf1"} Dec 03 12:38:41 crc kubenswrapper[5052]: I1203 12:38:41.557187 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"9e3bfbf3-a39c-48d2-bb1d-062c90c0ef10","Type":"ContainerStarted","Data":"e9d74f51a3c4d43463e19a86dda53575e91a595d1878cc45e8a213c710b19eab"} Dec 03 12:38:41 crc kubenswrapper[5052]: I1203 12:38:41.609800 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-scheduler-0" podStartSLOduration=2.609780409 podStartE2EDuration="2.609780409s" podCreationTimestamp="2025-12-03 12:38:39 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:38:41.600696269 +0000 UTC m=+4712.188938033" watchObservedRunningTime="2025-12-03 12:38:41.609780409 +0000 UTC m=+4712.198022183" Dec 03 12:38:41 crc kubenswrapper[5052]: I1203 12:38:41.691088 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8a16aa42-962f-48cc-a7a8-d0e189164013" path="/var/lib/kubelet/pods/8a16aa42-962f-48cc-a7a8-d0e189164013/volumes" Dec 03 12:38:41 crc kubenswrapper[5052]: E1203 12:38:41.701007 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-ovnnorthd-ovndbs: secret "cert-ovnnorthd-ovndbs" not found Dec 03 12:38:41 crc kubenswrapper[5052]: E1203 12:38:41.703210 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/92dadf05-a318-4491-bb1c-1fa23cb293fa-ovn-northd-tls-certs podName:92dadf05-a318-4491-bb1c-1fa23cb293fa nodeName:}" failed. No retries permitted until 2025-12-03 12:38:42.203194061 +0000 UTC m=+4712.791435825 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "ovn-northd-tls-certs" (UniqueName: "kubernetes.io/secret/92dadf05-a318-4491-bb1c-1fa23cb293fa-ovn-northd-tls-certs") pod "ovn-northd-0" (UID: "92dadf05-a318-4491-bb1c-1fa23cb293fa") : secret "cert-ovnnorthd-ovndbs" not found Dec 03 12:38:41 crc kubenswrapper[5052]: I1203 12:38:41.807675 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-nb-0"] Dec 03 12:38:41 crc kubenswrapper[5052]: I1203 12:38:41.809729 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ovsdbserver-nb-0" podUID="3da53770-41d7-4590-9278-21c1a82203b1" containerName="openstack-network-exporter" containerID="cri-o://5aa617ab3930326bfed819c0605942aa0091d0586b0d47d203001eb7f25457f5" gracePeriod=300 Dec 03 12:38:41 crc kubenswrapper[5052]: I1203 12:38:41.870112 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ovsdbserver-nb-0" podUID="3da53770-41d7-4590-9278-21c1a82203b1" containerName="ovsdbserver-nb" containerID="cri-o://44b1bf53e148e3c548b1c0d9c822dcf34d9f2cab27ee6241c47780a6ed9ccca8" gracePeriod=300 Dec 03 12:38:42 crc kubenswrapper[5052]: E1203 12:38:42.210697 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-ovnnorthd-ovndbs: secret "cert-ovnnorthd-ovndbs" not found Dec 03 12:38:42 crc kubenswrapper[5052]: E1203 12:38:42.211139 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/92dadf05-a318-4491-bb1c-1fa23cb293fa-ovn-northd-tls-certs podName:92dadf05-a318-4491-bb1c-1fa23cb293fa nodeName:}" failed. No retries permitted until 2025-12-03 12:38:43.211115396 +0000 UTC m=+4713.799357220 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "ovn-northd-tls-certs" (UniqueName: "kubernetes.io/secret/92dadf05-a318-4491-bb1c-1fa23cb293fa-ovn-northd-tls-certs") pod "ovn-northd-0" (UID: "92dadf05-a318-4491-bb1c-1fa23cb293fa") : secret "cert-ovnnorthd-ovndbs" not found Dec 03 12:38:42 crc kubenswrapper[5052]: I1203 12:38:42.396371 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_ovsdbserver-nb-0_3da53770-41d7-4590-9278-21c1a82203b1/ovsdbserver-nb/0.log" Dec 03 12:38:42 crc kubenswrapper[5052]: I1203 12:38:42.396441 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:38:42 crc kubenswrapper[5052]: I1203 12:38:42.516443 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovndbcluster-nb-etc-ovn\" (UniqueName: \"kubernetes.io/local-volume/local-storage01-crc\") pod \"3da53770-41d7-4590-9278-21c1a82203b1\" (UID: \"3da53770-41d7-4590-9278-21c1a82203b1\") " Dec 03 12:38:42 crc kubenswrapper[5052]: I1203 12:38:42.516612 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/3da53770-41d7-4590-9278-21c1a82203b1-ovsdb-rundir\") pod \"3da53770-41d7-4590-9278-21c1a82203b1\" (UID: \"3da53770-41d7-4590-9278-21c1a82203b1\") " Dec 03 12:38:42 crc kubenswrapper[5052]: I1203 12:38:42.516661 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3da53770-41d7-4590-9278-21c1a82203b1-config\") pod \"3da53770-41d7-4590-9278-21c1a82203b1\" (UID: \"3da53770-41d7-4590-9278-21c1a82203b1\") " Dec 03 12:38:42 crc kubenswrapper[5052]: I1203 12:38:42.516706 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovsdbserver-nb-tls-certs\" (UniqueName: \"kubernetes.io/secret/3da53770-41d7-4590-9278-21c1a82203b1-ovsdbserver-nb-tls-certs\") pod \"3da53770-41d7-4590-9278-21c1a82203b1\" (UID: \"3da53770-41d7-4590-9278-21c1a82203b1\") " Dec 03 12:38:42 crc kubenswrapper[5052]: I1203 12:38:42.516737 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/3da53770-41d7-4590-9278-21c1a82203b1-scripts\") pod \"3da53770-41d7-4590-9278-21c1a82203b1\" (UID: \"3da53770-41d7-4590-9278-21c1a82203b1\") " Dec 03 12:38:42 crc kubenswrapper[5052]: I1203 12:38:42.516763 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3da53770-41d7-4590-9278-21c1a82203b1-combined-ca-bundle\") pod \"3da53770-41d7-4590-9278-21c1a82203b1\" (UID: \"3da53770-41d7-4590-9278-21c1a82203b1\") " Dec 03 12:38:42 crc kubenswrapper[5052]: I1203 12:38:42.516804 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/3da53770-41d7-4590-9278-21c1a82203b1-metrics-certs-tls-certs\") pod \"3da53770-41d7-4590-9278-21c1a82203b1\" (UID: \"3da53770-41d7-4590-9278-21c1a82203b1\") " Dec 03 12:38:42 crc kubenswrapper[5052]: I1203 12:38:42.516863 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bqbwq\" (UniqueName: \"kubernetes.io/projected/3da53770-41d7-4590-9278-21c1a82203b1-kube-api-access-bqbwq\") pod \"3da53770-41d7-4590-9278-21c1a82203b1\" (UID: \"3da53770-41d7-4590-9278-21c1a82203b1\") " Dec 03 12:38:42 crc kubenswrapper[5052]: I1203 12:38:42.517290 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3da53770-41d7-4590-9278-21c1a82203b1-ovsdb-rundir" (OuterVolumeSpecName: "ovsdb-rundir") pod "3da53770-41d7-4590-9278-21c1a82203b1" (UID: "3da53770-41d7-4590-9278-21c1a82203b1"). InnerVolumeSpecName "ovsdb-rundir". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:38:42 crc kubenswrapper[5052]: I1203 12:38:42.518746 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3da53770-41d7-4590-9278-21c1a82203b1-config" (OuterVolumeSpecName: "config") pod "3da53770-41d7-4590-9278-21c1a82203b1" (UID: "3da53770-41d7-4590-9278-21c1a82203b1"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:38:42 crc kubenswrapper[5052]: I1203 12:38:42.519465 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3da53770-41d7-4590-9278-21c1a82203b1-scripts" (OuterVolumeSpecName: "scripts") pod "3da53770-41d7-4590-9278-21c1a82203b1" (UID: "3da53770-41d7-4590-9278-21c1a82203b1"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:38:42 crc kubenswrapper[5052]: I1203 12:38:42.522456 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage01-crc" (OuterVolumeSpecName: "ovndbcluster-nb-etc-ovn") pod "3da53770-41d7-4590-9278-21c1a82203b1" (UID: "3da53770-41d7-4590-9278-21c1a82203b1"). InnerVolumeSpecName "local-storage01-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 12:38:42 crc kubenswrapper[5052]: I1203 12:38:42.526663 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3da53770-41d7-4590-9278-21c1a82203b1-kube-api-access-bqbwq" (OuterVolumeSpecName: "kube-api-access-bqbwq") pod "3da53770-41d7-4590-9278-21c1a82203b1" (UID: "3da53770-41d7-4590-9278-21c1a82203b1"). InnerVolumeSpecName "kube-api-access-bqbwq". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:38:42 crc kubenswrapper[5052]: I1203 12:38:42.547866 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3da53770-41d7-4590-9278-21c1a82203b1-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "3da53770-41d7-4590-9278-21c1a82203b1" (UID: "3da53770-41d7-4590-9278-21c1a82203b1"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:38:42 crc kubenswrapper[5052]: I1203 12:38:42.567929 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_ovsdbserver-nb-0_3da53770-41d7-4590-9278-21c1a82203b1/ovsdbserver-nb/0.log" Dec 03 12:38:42 crc kubenswrapper[5052]: I1203 12:38:42.567982 5052 generic.go:334] "Generic (PLEG): container finished" podID="3da53770-41d7-4590-9278-21c1a82203b1" containerID="5aa617ab3930326bfed819c0605942aa0091d0586b0d47d203001eb7f25457f5" exitCode=2 Dec 03 12:38:42 crc kubenswrapper[5052]: I1203 12:38:42.567996 5052 generic.go:334] "Generic (PLEG): container finished" podID="3da53770-41d7-4590-9278-21c1a82203b1" containerID="44b1bf53e148e3c548b1c0d9c822dcf34d9f2cab27ee6241c47780a6ed9ccca8" exitCode=143 Dec 03 12:38:42 crc kubenswrapper[5052]: I1203 12:38:42.568055 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-nb-0" event={"ID":"3da53770-41d7-4590-9278-21c1a82203b1","Type":"ContainerDied","Data":"5aa617ab3930326bfed819c0605942aa0091d0586b0d47d203001eb7f25457f5"} Dec 03 12:38:42 crc kubenswrapper[5052]: I1203 12:38:42.568081 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-nb-0" event={"ID":"3da53770-41d7-4590-9278-21c1a82203b1","Type":"ContainerDied","Data":"44b1bf53e148e3c548b1c0d9c822dcf34d9f2cab27ee6241c47780a6ed9ccca8"} Dec 03 12:38:42 crc kubenswrapper[5052]: I1203 12:38:42.568092 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-nb-0" event={"ID":"3da53770-41d7-4590-9278-21c1a82203b1","Type":"ContainerDied","Data":"58f5b345e908a96e302d405c9daeb577751c8a424016870cafde74b07319caa5"} Dec 03 12:38:42 crc kubenswrapper[5052]: I1203 12:38:42.568108 5052 scope.go:117] "RemoveContainer" containerID="5aa617ab3930326bfed819c0605942aa0091d0586b0d47d203001eb7f25457f5" Dec 03 12:38:42 crc kubenswrapper[5052]: I1203 12:38:42.568107 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:38:42 crc kubenswrapper[5052]: I1203 12:38:42.571621 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"93664241-e156-4225-980c-4dbe8aff3353","Type":"ContainerStarted","Data":"165fa18efd5297210936aec88fa2048a36107465b7ba46db18221643e103d851"} Dec 03 12:38:42 crc kubenswrapper[5052]: I1203 12:38:42.575058 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-cell1-galera-0" event={"ID":"2253cde7-04d1-41a2-82d3-9c22cc6099c4","Type":"ContainerStarted","Data":"422d4b6e0b64601b76bee8009b403a22fc44a6ee2de6d96f6fdfc586119b7cfe"} Dec 03 12:38:42 crc kubenswrapper[5052]: I1203 12:38:42.601744 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3da53770-41d7-4590-9278-21c1a82203b1-ovsdbserver-nb-tls-certs" (OuterVolumeSpecName: "ovsdbserver-nb-tls-certs") pod "3da53770-41d7-4590-9278-21c1a82203b1" (UID: "3da53770-41d7-4590-9278-21c1a82203b1"). InnerVolumeSpecName "ovsdbserver-nb-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:38:42 crc kubenswrapper[5052]: I1203 12:38:42.611616 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3da53770-41d7-4590-9278-21c1a82203b1-metrics-certs-tls-certs" (OuterVolumeSpecName: "metrics-certs-tls-certs") pod "3da53770-41d7-4590-9278-21c1a82203b1" (UID: "3da53770-41d7-4590-9278-21c1a82203b1"). InnerVolumeSpecName "metrics-certs-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:38:42 crc kubenswrapper[5052]: I1203 12:38:42.611892 5052 scope.go:117] "RemoveContainer" containerID="44b1bf53e148e3c548b1c0d9c822dcf34d9f2cab27ee6241c47780a6ed9ccca8" Dec 03 12:38:42 crc kubenswrapper[5052]: I1203 12:38:42.618057 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/openstack-cell1-galera-0" podStartSLOduration=3.618037174 podStartE2EDuration="3.618037174s" podCreationTimestamp="2025-12-03 12:38:39 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:38:42.598760592 +0000 UTC m=+4713.187002376" watchObservedRunningTime="2025-12-03 12:38:42.618037174 +0000 UTC m=+4713.206278958" Dec 03 12:38:42 crc kubenswrapper[5052]: I1203 12:38:42.619172 5052 reconciler_common.go:293] "Volume detached for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/3da53770-41d7-4590-9278-21c1a82203b1-ovsdb-rundir\") on node \"crc\" DevicePath \"\"" Dec 03 12:38:42 crc kubenswrapper[5052]: I1203 12:38:42.619218 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3da53770-41d7-4590-9278-21c1a82203b1-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:38:42 crc kubenswrapper[5052]: I1203 12:38:42.619229 5052 reconciler_common.go:293] "Volume detached for volume \"ovsdbserver-nb-tls-certs\" (UniqueName: \"kubernetes.io/secret/3da53770-41d7-4590-9278-21c1a82203b1-ovsdbserver-nb-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:38:42 crc kubenswrapper[5052]: I1203 12:38:42.619238 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/3da53770-41d7-4590-9278-21c1a82203b1-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:38:42 crc kubenswrapper[5052]: I1203 12:38:42.619248 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3da53770-41d7-4590-9278-21c1a82203b1-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:38:42 crc kubenswrapper[5052]: I1203 12:38:42.619976 5052 reconciler_common.go:293] "Volume detached for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/3da53770-41d7-4590-9278-21c1a82203b1-metrics-certs-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:38:42 crc kubenswrapper[5052]: I1203 12:38:42.619993 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-bqbwq\" (UniqueName: \"kubernetes.io/projected/3da53770-41d7-4590-9278-21c1a82203b1-kube-api-access-bqbwq\") on node \"crc\" DevicePath \"\"" Dec 03 12:38:42 crc kubenswrapper[5052]: I1203 12:38:42.620022 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage01-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage01-crc\") on node \"crc\" " Dec 03 12:38:42 crc kubenswrapper[5052]: I1203 12:38:42.642991 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage01-crc" (UniqueName: "kubernetes.io/local-volume/local-storage01-crc") on node "crc" Dec 03 12:38:42 crc kubenswrapper[5052]: I1203 12:38:42.643965 5052 scope.go:117] "RemoveContainer" containerID="5aa617ab3930326bfed819c0605942aa0091d0586b0d47d203001eb7f25457f5" Dec 03 12:38:42 crc kubenswrapper[5052]: E1203 12:38:42.644670 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5aa617ab3930326bfed819c0605942aa0091d0586b0d47d203001eb7f25457f5\": container with ID starting with 5aa617ab3930326bfed819c0605942aa0091d0586b0d47d203001eb7f25457f5 not found: ID does not exist" containerID="5aa617ab3930326bfed819c0605942aa0091d0586b0d47d203001eb7f25457f5" Dec 03 12:38:42 crc kubenswrapper[5052]: I1203 12:38:42.644782 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5aa617ab3930326bfed819c0605942aa0091d0586b0d47d203001eb7f25457f5"} err="failed to get container status \"5aa617ab3930326bfed819c0605942aa0091d0586b0d47d203001eb7f25457f5\": rpc error: code = NotFound desc = could not find container \"5aa617ab3930326bfed819c0605942aa0091d0586b0d47d203001eb7f25457f5\": container with ID starting with 5aa617ab3930326bfed819c0605942aa0091d0586b0d47d203001eb7f25457f5 not found: ID does not exist" Dec 03 12:38:42 crc kubenswrapper[5052]: I1203 12:38:42.644878 5052 scope.go:117] "RemoveContainer" containerID="44b1bf53e148e3c548b1c0d9c822dcf34d9f2cab27ee6241c47780a6ed9ccca8" Dec 03 12:38:42 crc kubenswrapper[5052]: E1203 12:38:42.645280 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"44b1bf53e148e3c548b1c0d9c822dcf34d9f2cab27ee6241c47780a6ed9ccca8\": container with ID starting with 44b1bf53e148e3c548b1c0d9c822dcf34d9f2cab27ee6241c47780a6ed9ccca8 not found: ID does not exist" containerID="44b1bf53e148e3c548b1c0d9c822dcf34d9f2cab27ee6241c47780a6ed9ccca8" Dec 03 12:38:42 crc kubenswrapper[5052]: I1203 12:38:42.645318 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"44b1bf53e148e3c548b1c0d9c822dcf34d9f2cab27ee6241c47780a6ed9ccca8"} err="failed to get container status \"44b1bf53e148e3c548b1c0d9c822dcf34d9f2cab27ee6241c47780a6ed9ccca8\": rpc error: code = NotFound desc = could not find container \"44b1bf53e148e3c548b1c0d9c822dcf34d9f2cab27ee6241c47780a6ed9ccca8\": container with ID starting with 44b1bf53e148e3c548b1c0d9c822dcf34d9f2cab27ee6241c47780a6ed9ccca8 not found: ID does not exist" Dec 03 12:38:42 crc kubenswrapper[5052]: I1203 12:38:42.645332 5052 scope.go:117] "RemoveContainer" containerID="5aa617ab3930326bfed819c0605942aa0091d0586b0d47d203001eb7f25457f5" Dec 03 12:38:42 crc kubenswrapper[5052]: I1203 12:38:42.645676 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5aa617ab3930326bfed819c0605942aa0091d0586b0d47d203001eb7f25457f5"} err="failed to get container status \"5aa617ab3930326bfed819c0605942aa0091d0586b0d47d203001eb7f25457f5\": rpc error: code = NotFound desc = could not find container \"5aa617ab3930326bfed819c0605942aa0091d0586b0d47d203001eb7f25457f5\": container with ID starting with 5aa617ab3930326bfed819c0605942aa0091d0586b0d47d203001eb7f25457f5 not found: ID does not exist" Dec 03 12:38:42 crc kubenswrapper[5052]: I1203 12:38:42.645701 5052 scope.go:117] "RemoveContainer" containerID="44b1bf53e148e3c548b1c0d9c822dcf34d9f2cab27ee6241c47780a6ed9ccca8" Dec 03 12:38:42 crc kubenswrapper[5052]: I1203 12:38:42.646144 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"44b1bf53e148e3c548b1c0d9c822dcf34d9f2cab27ee6241c47780a6ed9ccca8"} err="failed to get container status \"44b1bf53e148e3c548b1c0d9c822dcf34d9f2cab27ee6241c47780a6ed9ccca8\": rpc error: code = NotFound desc = could not find container \"44b1bf53e148e3c548b1c0d9c822dcf34d9f2cab27ee6241c47780a6ed9ccca8\": container with ID starting with 44b1bf53e148e3c548b1c0d9c822dcf34d9f2cab27ee6241c47780a6ed9ccca8 not found: ID does not exist" Dec 03 12:38:42 crc kubenswrapper[5052]: I1203 12:38:42.722481 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage01-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage01-crc\") on node \"crc\" DevicePath \"\"" Dec 03 12:38:42 crc kubenswrapper[5052]: I1203 12:38:42.741324 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-sb-0"] Dec 03 12:38:42 crc kubenswrapper[5052]: I1203 12:38:42.741700 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ovsdbserver-sb-0" podUID="143d0ab6-f833-41a9-aacd-34d20b35f8ed" containerName="openstack-network-exporter" containerID="cri-o://66eb2dc35973bd557c9042a9e0b7776f13f8d8df1e970eff2cb7d9509c8c414a" gracePeriod=300 Dec 03 12:38:42 crc kubenswrapper[5052]: I1203 12:38:42.808939 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ovsdbserver-sb-0" podUID="143d0ab6-f833-41a9-aacd-34d20b35f8ed" containerName="ovsdbserver-sb" containerID="cri-o://f9ed4255fa67b2a8f659f0dd13829e2e2da10d2d67a6417060a75e2605964162" gracePeriod=300 Dec 03 12:38:42 crc kubenswrapper[5052]: E1203 12:38:42.949312 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of f9ed4255fa67b2a8f659f0dd13829e2e2da10d2d67a6417060a75e2605964162 is running failed: container process not found" containerID="f9ed4255fa67b2a8f659f0dd13829e2e2da10d2d67a6417060a75e2605964162" cmd=["/usr/bin/pidof","ovsdb-server"] Dec 03 12:38:42 crc kubenswrapper[5052]: E1203 12:38:42.949657 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of f9ed4255fa67b2a8f659f0dd13829e2e2da10d2d67a6417060a75e2605964162 is running failed: container process not found" containerID="f9ed4255fa67b2a8f659f0dd13829e2e2da10d2d67a6417060a75e2605964162" cmd=["/usr/bin/pidof","ovsdb-server"] Dec 03 12:38:42 crc kubenswrapper[5052]: E1203 12:38:42.949995 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of f9ed4255fa67b2a8f659f0dd13829e2e2da10d2d67a6417060a75e2605964162 is running failed: container process not found" containerID="f9ed4255fa67b2a8f659f0dd13829e2e2da10d2d67a6417060a75e2605964162" cmd=["/usr/bin/pidof","ovsdb-server"] Dec 03 12:38:42 crc kubenswrapper[5052]: E1203 12:38:42.950025 5052 prober.go:104] "Probe errored" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of f9ed4255fa67b2a8f659f0dd13829e2e2da10d2d67a6417060a75e2605964162 is running failed: container process not found" probeType="Readiness" pod="openstack-kuttl-tests/ovsdbserver-sb-0" podUID="143d0ab6-f833-41a9-aacd-34d20b35f8ed" containerName="ovsdbserver-sb" Dec 03 12:38:43 crc kubenswrapper[5052]: I1203 12:38:43.098820 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-nb-0"] Dec 03 12:38:43 crc kubenswrapper[5052]: I1203 12:38:43.118921 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-nb-0"] Dec 03 12:38:43 crc kubenswrapper[5052]: I1203 12:38:43.146701 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ovsdbserver-nb-0"] Dec 03 12:38:43 crc kubenswrapper[5052]: E1203 12:38:43.147242 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="3da53770-41d7-4590-9278-21c1a82203b1" containerName="ovsdbserver-nb" Dec 03 12:38:43 crc kubenswrapper[5052]: I1203 12:38:43.147258 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="3da53770-41d7-4590-9278-21c1a82203b1" containerName="ovsdbserver-nb" Dec 03 12:38:43 crc kubenswrapper[5052]: E1203 12:38:43.147284 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="3da53770-41d7-4590-9278-21c1a82203b1" containerName="openstack-network-exporter" Dec 03 12:38:43 crc kubenswrapper[5052]: I1203 12:38:43.147290 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="3da53770-41d7-4590-9278-21c1a82203b1" containerName="openstack-network-exporter" Dec 03 12:38:43 crc kubenswrapper[5052]: I1203 12:38:43.147486 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="3da53770-41d7-4590-9278-21c1a82203b1" containerName="ovsdbserver-nb" Dec 03 12:38:43 crc kubenswrapper[5052]: I1203 12:38:43.147495 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="3da53770-41d7-4590-9278-21c1a82203b1" containerName="openstack-network-exporter" Dec 03 12:38:43 crc kubenswrapper[5052]: I1203 12:38:43.149083 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:38:43 crc kubenswrapper[5052]: I1203 12:38:43.155961 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-ovndbcluster-nb-ovndbs" Dec 03 12:38:43 crc kubenswrapper[5052]: I1203 12:38:43.156183 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"ovndbcluster-nb-scripts" Dec 03 12:38:43 crc kubenswrapper[5052]: I1203 12:38:43.156300 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ovncluster-ovndbcluster-nb-dockercfg-xg67d" Dec 03 12:38:43 crc kubenswrapper[5052]: I1203 12:38:43.156465 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"ovndbcluster-nb-config" Dec 03 12:38:43 crc kubenswrapper[5052]: I1203 12:38:43.204886 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/cinder-api-0" podUID="8a16aa42-962f-48cc-a7a8-d0e189164013" containerName="cinder-api" probeResult="failure" output="Get \"https://10.217.1.50:8776/healthcheck\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Dec 03 12:38:43 crc kubenswrapper[5052]: I1203 12:38:43.217338 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-nb-0"] Dec 03 12:38:43 crc kubenswrapper[5052]: I1203 12:38:43.233907 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/46c590ef-2b7c-412d-8582-dfdc311e6778-ovsdb-rundir\") pod \"ovsdbserver-nb-0\" (UID: \"46c590ef-2b7c-412d-8582-dfdc311e6778\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:38:43 crc kubenswrapper[5052]: I1203 12:38:43.233975 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage01-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage01-crc\") pod \"ovsdbserver-nb-0\" (UID: \"46c590ef-2b7c-412d-8582-dfdc311e6778\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:38:43 crc kubenswrapper[5052]: I1203 12:38:43.234029 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/46c590ef-2b7c-412d-8582-dfdc311e6778-scripts\") pod \"ovsdbserver-nb-0\" (UID: \"46c590ef-2b7c-412d-8582-dfdc311e6778\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:38:43 crc kubenswrapper[5052]: I1203 12:38:43.234098 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/46c590ef-2b7c-412d-8582-dfdc311e6778-metrics-certs-tls-certs\") pod \"ovsdbserver-nb-0\" (UID: \"46c590ef-2b7c-412d-8582-dfdc311e6778\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:38:43 crc kubenswrapper[5052]: I1203 12:38:43.234138 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/46c590ef-2b7c-412d-8582-dfdc311e6778-combined-ca-bundle\") pod \"ovsdbserver-nb-0\" (UID: \"46c590ef-2b7c-412d-8582-dfdc311e6778\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:38:43 crc kubenswrapper[5052]: E1203 12:38:43.234231 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-ovnnorthd-ovndbs: secret "cert-ovnnorthd-ovndbs" not found Dec 03 12:38:43 crc kubenswrapper[5052]: E1203 12:38:43.234349 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/92dadf05-a318-4491-bb1c-1fa23cb293fa-ovn-northd-tls-certs podName:92dadf05-a318-4491-bb1c-1fa23cb293fa nodeName:}" failed. No retries permitted until 2025-12-03 12:38:45.234300477 +0000 UTC m=+4715.822542301 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "ovn-northd-tls-certs" (UniqueName: "kubernetes.io/secret/92dadf05-a318-4491-bb1c-1fa23cb293fa-ovn-northd-tls-certs") pod "ovn-northd-0" (UID: "92dadf05-a318-4491-bb1c-1fa23cb293fa") : secret "cert-ovnnorthd-ovndbs" not found Dec 03 12:38:43 crc kubenswrapper[5052]: I1203 12:38:43.234446 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/46c590ef-2b7c-412d-8582-dfdc311e6778-config\") pod \"ovsdbserver-nb-0\" (UID: \"46c590ef-2b7c-412d-8582-dfdc311e6778\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:38:43 crc kubenswrapper[5052]: I1203 12:38:43.234563 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-npwz9\" (UniqueName: \"kubernetes.io/projected/46c590ef-2b7c-412d-8582-dfdc311e6778-kube-api-access-npwz9\") pod \"ovsdbserver-nb-0\" (UID: \"46c590ef-2b7c-412d-8582-dfdc311e6778\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:38:43 crc kubenswrapper[5052]: I1203 12:38:43.234900 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovsdbserver-nb-tls-certs\" (UniqueName: \"kubernetes.io/secret/46c590ef-2b7c-412d-8582-dfdc311e6778-ovsdbserver-nb-tls-certs\") pod \"ovsdbserver-nb-0\" (UID: \"46c590ef-2b7c-412d-8582-dfdc311e6778\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:38:43 crc kubenswrapper[5052]: I1203 12:38:43.285091 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_ovsdbserver-sb-0_143d0ab6-f833-41a9-aacd-34d20b35f8ed/ovsdbserver-sb/0.log" Dec 03 12:38:43 crc kubenswrapper[5052]: I1203 12:38:43.285162 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:38:43 crc kubenswrapper[5052]: I1203 12:38:43.336453 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/46c590ef-2b7c-412d-8582-dfdc311e6778-ovsdb-rundir\") pod \"ovsdbserver-nb-0\" (UID: \"46c590ef-2b7c-412d-8582-dfdc311e6778\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:38:43 crc kubenswrapper[5052]: I1203 12:38:43.336544 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage01-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage01-crc\") pod \"ovsdbserver-nb-0\" (UID: \"46c590ef-2b7c-412d-8582-dfdc311e6778\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:38:43 crc kubenswrapper[5052]: I1203 12:38:43.336579 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/46c590ef-2b7c-412d-8582-dfdc311e6778-scripts\") pod \"ovsdbserver-nb-0\" (UID: \"46c590ef-2b7c-412d-8582-dfdc311e6778\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:38:43 crc kubenswrapper[5052]: I1203 12:38:43.336621 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/46c590ef-2b7c-412d-8582-dfdc311e6778-metrics-certs-tls-certs\") pod \"ovsdbserver-nb-0\" (UID: \"46c590ef-2b7c-412d-8582-dfdc311e6778\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:38:43 crc kubenswrapper[5052]: I1203 12:38:43.336652 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/46c590ef-2b7c-412d-8582-dfdc311e6778-combined-ca-bundle\") pod \"ovsdbserver-nb-0\" (UID: \"46c590ef-2b7c-412d-8582-dfdc311e6778\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:38:43 crc kubenswrapper[5052]: I1203 12:38:43.336702 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/46c590ef-2b7c-412d-8582-dfdc311e6778-config\") pod \"ovsdbserver-nb-0\" (UID: \"46c590ef-2b7c-412d-8582-dfdc311e6778\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:38:43 crc kubenswrapper[5052]: I1203 12:38:43.336737 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-npwz9\" (UniqueName: \"kubernetes.io/projected/46c590ef-2b7c-412d-8582-dfdc311e6778-kube-api-access-npwz9\") pod \"ovsdbserver-nb-0\" (UID: \"46c590ef-2b7c-412d-8582-dfdc311e6778\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:38:43 crc kubenswrapper[5052]: I1203 12:38:43.336840 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovsdbserver-nb-tls-certs\" (UniqueName: \"kubernetes.io/secret/46c590ef-2b7c-412d-8582-dfdc311e6778-ovsdbserver-nb-tls-certs\") pod \"ovsdbserver-nb-0\" (UID: \"46c590ef-2b7c-412d-8582-dfdc311e6778\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:38:43 crc kubenswrapper[5052]: I1203 12:38:43.336988 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/46c590ef-2b7c-412d-8582-dfdc311e6778-ovsdb-rundir\") pod \"ovsdbserver-nb-0\" (UID: \"46c590ef-2b7c-412d-8582-dfdc311e6778\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:38:43 crc kubenswrapper[5052]: I1203 12:38:43.337916 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage01-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage01-crc\") pod \"ovsdbserver-nb-0\" (UID: \"46c590ef-2b7c-412d-8582-dfdc311e6778\") device mount path \"/mnt/openstack/pv01\"" pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:38:43 crc kubenswrapper[5052]: I1203 12:38:43.339117 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/46c590ef-2b7c-412d-8582-dfdc311e6778-config\") pod \"ovsdbserver-nb-0\" (UID: \"46c590ef-2b7c-412d-8582-dfdc311e6778\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:38:43 crc kubenswrapper[5052]: I1203 12:38:43.341166 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/46c590ef-2b7c-412d-8582-dfdc311e6778-scripts\") pod \"ovsdbserver-nb-0\" (UID: \"46c590ef-2b7c-412d-8582-dfdc311e6778\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:38:43 crc kubenswrapper[5052]: I1203 12:38:43.344611 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/46c590ef-2b7c-412d-8582-dfdc311e6778-metrics-certs-tls-certs\") pod \"ovsdbserver-nb-0\" (UID: \"46c590ef-2b7c-412d-8582-dfdc311e6778\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:38:43 crc kubenswrapper[5052]: I1203 12:38:43.345300 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovsdbserver-nb-tls-certs\" (UniqueName: \"kubernetes.io/secret/46c590ef-2b7c-412d-8582-dfdc311e6778-ovsdbserver-nb-tls-certs\") pod \"ovsdbserver-nb-0\" (UID: \"46c590ef-2b7c-412d-8582-dfdc311e6778\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:38:43 crc kubenswrapper[5052]: I1203 12:38:43.346247 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/46c590ef-2b7c-412d-8582-dfdc311e6778-combined-ca-bundle\") pod \"ovsdbserver-nb-0\" (UID: \"46c590ef-2b7c-412d-8582-dfdc311e6778\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:38:43 crc kubenswrapper[5052]: I1203 12:38:43.362642 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-npwz9\" (UniqueName: \"kubernetes.io/projected/46c590ef-2b7c-412d-8582-dfdc311e6778-kube-api-access-npwz9\") pod \"ovsdbserver-nb-0\" (UID: \"46c590ef-2b7c-412d-8582-dfdc311e6778\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:38:43 crc kubenswrapper[5052]: I1203 12:38:43.370921 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage01-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage01-crc\") pod \"ovsdbserver-nb-0\" (UID: \"46c590ef-2b7c-412d-8582-dfdc311e6778\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:38:43 crc kubenswrapper[5052]: I1203 12:38:43.438160 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/143d0ab6-f833-41a9-aacd-34d20b35f8ed-scripts\") pod \"143d0ab6-f833-41a9-aacd-34d20b35f8ed\" (UID: \"143d0ab6-f833-41a9-aacd-34d20b35f8ed\") " Dec 03 12:38:43 crc kubenswrapper[5052]: I1203 12:38:43.438259 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/143d0ab6-f833-41a9-aacd-34d20b35f8ed-config\") pod \"143d0ab6-f833-41a9-aacd-34d20b35f8ed\" (UID: \"143d0ab6-f833-41a9-aacd-34d20b35f8ed\") " Dec 03 12:38:43 crc kubenswrapper[5052]: I1203 12:38:43.438355 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovndbcluster-sb-etc-ovn\" (UniqueName: \"kubernetes.io/local-volume/local-storage09-crc\") pod \"143d0ab6-f833-41a9-aacd-34d20b35f8ed\" (UID: \"143d0ab6-f833-41a9-aacd-34d20b35f8ed\") " Dec 03 12:38:43 crc kubenswrapper[5052]: I1203 12:38:43.438473 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/143d0ab6-f833-41a9-aacd-34d20b35f8ed-ovsdb-rundir\") pod \"143d0ab6-f833-41a9-aacd-34d20b35f8ed\" (UID: \"143d0ab6-f833-41a9-aacd-34d20b35f8ed\") " Dec 03 12:38:43 crc kubenswrapper[5052]: I1203 12:38:43.438546 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovsdbserver-sb-tls-certs\" (UniqueName: \"kubernetes.io/secret/143d0ab6-f833-41a9-aacd-34d20b35f8ed-ovsdbserver-sb-tls-certs\") pod \"143d0ab6-f833-41a9-aacd-34d20b35f8ed\" (UID: \"143d0ab6-f833-41a9-aacd-34d20b35f8ed\") " Dec 03 12:38:43 crc kubenswrapper[5052]: I1203 12:38:43.438592 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/143d0ab6-f833-41a9-aacd-34d20b35f8ed-metrics-certs-tls-certs\") pod \"143d0ab6-f833-41a9-aacd-34d20b35f8ed\" (UID: \"143d0ab6-f833-41a9-aacd-34d20b35f8ed\") " Dec 03 12:38:43 crc kubenswrapper[5052]: I1203 12:38:43.438618 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/143d0ab6-f833-41a9-aacd-34d20b35f8ed-combined-ca-bundle\") pod \"143d0ab6-f833-41a9-aacd-34d20b35f8ed\" (UID: \"143d0ab6-f833-41a9-aacd-34d20b35f8ed\") " Dec 03 12:38:43 crc kubenswrapper[5052]: I1203 12:38:43.438661 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sppmc\" (UniqueName: \"kubernetes.io/projected/143d0ab6-f833-41a9-aacd-34d20b35f8ed-kube-api-access-sppmc\") pod \"143d0ab6-f833-41a9-aacd-34d20b35f8ed\" (UID: \"143d0ab6-f833-41a9-aacd-34d20b35f8ed\") " Dec 03 12:38:43 crc kubenswrapper[5052]: I1203 12:38:43.438957 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/143d0ab6-f833-41a9-aacd-34d20b35f8ed-config" (OuterVolumeSpecName: "config") pod "143d0ab6-f833-41a9-aacd-34d20b35f8ed" (UID: "143d0ab6-f833-41a9-aacd-34d20b35f8ed"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:38:43 crc kubenswrapper[5052]: I1203 12:38:43.439597 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/143d0ab6-f833-41a9-aacd-34d20b35f8ed-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:38:43 crc kubenswrapper[5052]: E1203 12:38:43.439676 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-glance-default-public-svc: secret "cert-glance-default-public-svc" not found Dec 03 12:38:43 crc kubenswrapper[5052]: E1203 12:38:43.439725 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9dccead0-7ff2-4255-ac00-10a9b357c2db-public-tls-certs podName:9dccead0-7ff2-4255-ac00-10a9b357c2db nodeName:}" failed. No retries permitted until 2025-12-03 12:38:51.439712181 +0000 UTC m=+4722.027953935 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "public-tls-certs" (UniqueName: "kubernetes.io/secret/9dccead0-7ff2-4255-ac00-10a9b357c2db-public-tls-certs") pod "glance-default-external-api-0" (UID: "9dccead0-7ff2-4255-ac00-10a9b357c2db") : secret "cert-glance-default-public-svc" not found Dec 03 12:38:43 crc kubenswrapper[5052]: I1203 12:38:43.440728 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/143d0ab6-f833-41a9-aacd-34d20b35f8ed-ovsdb-rundir" (OuterVolumeSpecName: "ovsdb-rundir") pod "143d0ab6-f833-41a9-aacd-34d20b35f8ed" (UID: "143d0ab6-f833-41a9-aacd-34d20b35f8ed"). InnerVolumeSpecName "ovsdb-rundir". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:38:43 crc kubenswrapper[5052]: I1203 12:38:43.442007 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/143d0ab6-f833-41a9-aacd-34d20b35f8ed-scripts" (OuterVolumeSpecName: "scripts") pod "143d0ab6-f833-41a9-aacd-34d20b35f8ed" (UID: "143d0ab6-f833-41a9-aacd-34d20b35f8ed"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:38:43 crc kubenswrapper[5052]: I1203 12:38:43.443249 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/143d0ab6-f833-41a9-aacd-34d20b35f8ed-kube-api-access-sppmc" (OuterVolumeSpecName: "kube-api-access-sppmc") pod "143d0ab6-f833-41a9-aacd-34d20b35f8ed" (UID: "143d0ab6-f833-41a9-aacd-34d20b35f8ed"). InnerVolumeSpecName "kube-api-access-sppmc". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:38:43 crc kubenswrapper[5052]: I1203 12:38:43.445647 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage09-crc" (OuterVolumeSpecName: "ovndbcluster-sb-etc-ovn") pod "143d0ab6-f833-41a9-aacd-34d20b35f8ed" (UID: "143d0ab6-f833-41a9-aacd-34d20b35f8ed"). InnerVolumeSpecName "local-storage09-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 12:38:43 crc kubenswrapper[5052]: I1203 12:38:43.469819 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/143d0ab6-f833-41a9-aacd-34d20b35f8ed-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "143d0ab6-f833-41a9-aacd-34d20b35f8ed" (UID: "143d0ab6-f833-41a9-aacd-34d20b35f8ed"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:38:43 crc kubenswrapper[5052]: I1203 12:38:43.490048 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:38:43 crc kubenswrapper[5052]: I1203 12:38:43.517671 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/143d0ab6-f833-41a9-aacd-34d20b35f8ed-ovsdbserver-sb-tls-certs" (OuterVolumeSpecName: "ovsdbserver-sb-tls-certs") pod "143d0ab6-f833-41a9-aacd-34d20b35f8ed" (UID: "143d0ab6-f833-41a9-aacd-34d20b35f8ed"). InnerVolumeSpecName "ovsdbserver-sb-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:38:43 crc kubenswrapper[5052]: I1203 12:38:43.526592 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/143d0ab6-f833-41a9-aacd-34d20b35f8ed-metrics-certs-tls-certs" (OuterVolumeSpecName: "metrics-certs-tls-certs") pod "143d0ab6-f833-41a9-aacd-34d20b35f8ed" (UID: "143d0ab6-f833-41a9-aacd-34d20b35f8ed"). InnerVolumeSpecName "metrics-certs-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:38:43 crc kubenswrapper[5052]: I1203 12:38:43.542239 5052 reconciler_common.go:293] "Volume detached for volume \"ovsdbserver-sb-tls-certs\" (UniqueName: \"kubernetes.io/secret/143d0ab6-f833-41a9-aacd-34d20b35f8ed-ovsdbserver-sb-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:38:43 crc kubenswrapper[5052]: I1203 12:38:43.542289 5052 reconciler_common.go:293] "Volume detached for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/143d0ab6-f833-41a9-aacd-34d20b35f8ed-metrics-certs-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:38:43 crc kubenswrapper[5052]: I1203 12:38:43.542303 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/143d0ab6-f833-41a9-aacd-34d20b35f8ed-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:38:43 crc kubenswrapper[5052]: I1203 12:38:43.542316 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-sppmc\" (UniqueName: \"kubernetes.io/projected/143d0ab6-f833-41a9-aacd-34d20b35f8ed-kube-api-access-sppmc\") on node \"crc\" DevicePath \"\"" Dec 03 12:38:43 crc kubenswrapper[5052]: I1203 12:38:43.542328 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/143d0ab6-f833-41a9-aacd-34d20b35f8ed-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:38:43 crc kubenswrapper[5052]: I1203 12:38:43.542364 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage09-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage09-crc\") on node \"crc\" " Dec 03 12:38:43 crc kubenswrapper[5052]: I1203 12:38:43.542380 5052 reconciler_common.go:293] "Volume detached for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/143d0ab6-f833-41a9-aacd-34d20b35f8ed-ovsdb-rundir\") on node \"crc\" DevicePath \"\"" Dec 03 12:38:43 crc kubenswrapper[5052]: I1203 12:38:43.573461 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage09-crc" (UniqueName: "kubernetes.io/local-volume/local-storage09-crc") on node "crc" Dec 03 12:38:43 crc kubenswrapper[5052]: I1203 12:38:43.594418 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"93664241-e156-4225-980c-4dbe8aff3353","Type":"ContainerStarted","Data":"305768045d12def6ad73ffe7ea66855fa5cc0e381b3a31f58d215ce49facba41"} Dec 03 12:38:43 crc kubenswrapper[5052]: I1203 12:38:43.594773 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:38:43 crc kubenswrapper[5052]: I1203 12:38:43.603901 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/barbican-api-b857b994d-6xzbl" podUID="3ceac470-11eb-4d69-a683-2e9bc4e8f617" containerName="barbican-api-log" probeResult="failure" output="Get \"https://10.217.1.46:9311/healthcheck\": read tcp 10.217.0.2:50934->10.217.1.46:9311: read: connection reset by peer" Dec 03 12:38:43 crc kubenswrapper[5052]: I1203 12:38:43.603905 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/barbican-api-b857b994d-6xzbl" podUID="3ceac470-11eb-4d69-a683-2e9bc4e8f617" containerName="barbican-api" probeResult="failure" output="Get \"https://10.217.1.46:9311/healthcheck\": read tcp 10.217.0.2:50936->10.217.1.46:9311: read: connection reset by peer" Dec 03 12:38:43 crc kubenswrapper[5052]: I1203 12:38:43.604383 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_ovsdbserver-sb-0_143d0ab6-f833-41a9-aacd-34d20b35f8ed/ovsdbserver-sb/0.log" Dec 03 12:38:43 crc kubenswrapper[5052]: I1203 12:38:43.604446 5052 generic.go:334] "Generic (PLEG): container finished" podID="143d0ab6-f833-41a9-aacd-34d20b35f8ed" containerID="66eb2dc35973bd557c9042a9e0b7776f13f8d8df1e970eff2cb7d9509c8c414a" exitCode=2 Dec 03 12:38:43 crc kubenswrapper[5052]: I1203 12:38:43.604467 5052 generic.go:334] "Generic (PLEG): container finished" podID="143d0ab6-f833-41a9-aacd-34d20b35f8ed" containerID="f9ed4255fa67b2a8f659f0dd13829e2e2da10d2d67a6417060a75e2605964162" exitCode=143 Dec 03 12:38:43 crc kubenswrapper[5052]: I1203 12:38:43.604565 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:38:43 crc kubenswrapper[5052]: I1203 12:38:43.604575 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-sb-0" event={"ID":"143d0ab6-f833-41a9-aacd-34d20b35f8ed","Type":"ContainerDied","Data":"66eb2dc35973bd557c9042a9e0b7776f13f8d8df1e970eff2cb7d9509c8c414a"} Dec 03 12:38:43 crc kubenswrapper[5052]: I1203 12:38:43.604621 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-sb-0" event={"ID":"143d0ab6-f833-41a9-aacd-34d20b35f8ed","Type":"ContainerDied","Data":"f9ed4255fa67b2a8f659f0dd13829e2e2da10d2d67a6417060a75e2605964162"} Dec 03 12:38:43 crc kubenswrapper[5052]: I1203 12:38:43.604639 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-sb-0" event={"ID":"143d0ab6-f833-41a9-aacd-34d20b35f8ed","Type":"ContainerDied","Data":"6c526ec1bf96c63f15a87a02a06a0464499e31aef3337c2671be8c4343b95929"} Dec 03 12:38:43 crc kubenswrapper[5052]: I1203 12:38:43.604658 5052 scope.go:117] "RemoveContainer" containerID="66eb2dc35973bd557c9042a9e0b7776f13f8d8df1e970eff2cb7d9509c8c414a" Dec 03 12:38:43 crc kubenswrapper[5052]: I1203 12:38:43.616199 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/cinder-api-0" podStartSLOduration=4.616175518 podStartE2EDuration="4.616175518s" podCreationTimestamp="2025-12-03 12:38:39 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:38:43.615085687 +0000 UTC m=+4714.203327471" watchObservedRunningTime="2025-12-03 12:38:43.616175518 +0000 UTC m=+4714.204417282" Dec 03 12:38:43 crc kubenswrapper[5052]: I1203 12:38:43.644627 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage09-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage09-crc\") on node \"crc\" DevicePath \"\"" Dec 03 12:38:43 crc kubenswrapper[5052]: I1203 12:38:43.701002 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3da53770-41d7-4590-9278-21c1a82203b1" path="/var/lib/kubelet/pods/3da53770-41d7-4590-9278-21c1a82203b1/volumes" Dec 03 12:38:43 crc kubenswrapper[5052]: I1203 12:38:43.702280 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-sb-0"] Dec 03 12:38:43 crc kubenswrapper[5052]: I1203 12:38:43.728911 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-sb-0"] Dec 03 12:38:43 crc kubenswrapper[5052]: I1203 12:38:43.741961 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ovsdbserver-sb-0"] Dec 03 12:38:43 crc kubenswrapper[5052]: E1203 12:38:43.742414 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="143d0ab6-f833-41a9-aacd-34d20b35f8ed" containerName="openstack-network-exporter" Dec 03 12:38:43 crc kubenswrapper[5052]: I1203 12:38:43.742431 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="143d0ab6-f833-41a9-aacd-34d20b35f8ed" containerName="openstack-network-exporter" Dec 03 12:38:43 crc kubenswrapper[5052]: E1203 12:38:43.742481 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="143d0ab6-f833-41a9-aacd-34d20b35f8ed" containerName="ovsdbserver-sb" Dec 03 12:38:43 crc kubenswrapper[5052]: I1203 12:38:43.742488 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="143d0ab6-f833-41a9-aacd-34d20b35f8ed" containerName="ovsdbserver-sb" Dec 03 12:38:43 crc kubenswrapper[5052]: I1203 12:38:43.742686 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="143d0ab6-f833-41a9-aacd-34d20b35f8ed" containerName="openstack-network-exporter" Dec 03 12:38:43 crc kubenswrapper[5052]: I1203 12:38:43.742698 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="143d0ab6-f833-41a9-aacd-34d20b35f8ed" containerName="ovsdbserver-sb" Dec 03 12:38:43 crc kubenswrapper[5052]: I1203 12:38:43.743864 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:38:43 crc kubenswrapper[5052]: I1203 12:38:43.747766 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"ovndbcluster-sb-config" Dec 03 12:38:43 crc kubenswrapper[5052]: I1203 12:38:43.747823 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-ovndbcluster-sb-ovndbs" Dec 03 12:38:43 crc kubenswrapper[5052]: I1203 12:38:43.748319 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"ovndbcluster-sb-scripts" Dec 03 12:38:43 crc kubenswrapper[5052]: I1203 12:38:43.748406 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ovncluster-ovndbcluster-sb-dockercfg-q8jfj" Dec 03 12:38:43 crc kubenswrapper[5052]: I1203 12:38:43.759298 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-sb-0"] Dec 03 12:38:43 crc kubenswrapper[5052]: I1203 12:38:43.763978 5052 scope.go:117] "RemoveContainer" containerID="f9ed4255fa67b2a8f659f0dd13829e2e2da10d2d67a6417060a75e2605964162" Dec 03 12:38:43 crc kubenswrapper[5052]: I1203 12:38:43.803038 5052 scope.go:117] "RemoveContainer" containerID="66eb2dc35973bd557c9042a9e0b7776f13f8d8df1e970eff2cb7d9509c8c414a" Dec 03 12:38:43 crc kubenswrapper[5052]: E1203 12:38:43.803433 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"66eb2dc35973bd557c9042a9e0b7776f13f8d8df1e970eff2cb7d9509c8c414a\": container with ID starting with 66eb2dc35973bd557c9042a9e0b7776f13f8d8df1e970eff2cb7d9509c8c414a not found: ID does not exist" containerID="66eb2dc35973bd557c9042a9e0b7776f13f8d8df1e970eff2cb7d9509c8c414a" Dec 03 12:38:43 crc kubenswrapper[5052]: I1203 12:38:43.803461 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"66eb2dc35973bd557c9042a9e0b7776f13f8d8df1e970eff2cb7d9509c8c414a"} err="failed to get container status \"66eb2dc35973bd557c9042a9e0b7776f13f8d8df1e970eff2cb7d9509c8c414a\": rpc error: code = NotFound desc = could not find container \"66eb2dc35973bd557c9042a9e0b7776f13f8d8df1e970eff2cb7d9509c8c414a\": container with ID starting with 66eb2dc35973bd557c9042a9e0b7776f13f8d8df1e970eff2cb7d9509c8c414a not found: ID does not exist" Dec 03 12:38:43 crc kubenswrapper[5052]: I1203 12:38:43.803482 5052 scope.go:117] "RemoveContainer" containerID="f9ed4255fa67b2a8f659f0dd13829e2e2da10d2d67a6417060a75e2605964162" Dec 03 12:38:43 crc kubenswrapper[5052]: E1203 12:38:43.803836 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f9ed4255fa67b2a8f659f0dd13829e2e2da10d2d67a6417060a75e2605964162\": container with ID starting with f9ed4255fa67b2a8f659f0dd13829e2e2da10d2d67a6417060a75e2605964162 not found: ID does not exist" containerID="f9ed4255fa67b2a8f659f0dd13829e2e2da10d2d67a6417060a75e2605964162" Dec 03 12:38:43 crc kubenswrapper[5052]: I1203 12:38:43.803856 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f9ed4255fa67b2a8f659f0dd13829e2e2da10d2d67a6417060a75e2605964162"} err="failed to get container status \"f9ed4255fa67b2a8f659f0dd13829e2e2da10d2d67a6417060a75e2605964162\": rpc error: code = NotFound desc = could not find container \"f9ed4255fa67b2a8f659f0dd13829e2e2da10d2d67a6417060a75e2605964162\": container with ID starting with f9ed4255fa67b2a8f659f0dd13829e2e2da10d2d67a6417060a75e2605964162 not found: ID does not exist" Dec 03 12:38:43 crc kubenswrapper[5052]: I1203 12:38:43.803880 5052 scope.go:117] "RemoveContainer" containerID="66eb2dc35973bd557c9042a9e0b7776f13f8d8df1e970eff2cb7d9509c8c414a" Dec 03 12:38:43 crc kubenswrapper[5052]: I1203 12:38:43.804064 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"66eb2dc35973bd557c9042a9e0b7776f13f8d8df1e970eff2cb7d9509c8c414a"} err="failed to get container status \"66eb2dc35973bd557c9042a9e0b7776f13f8d8df1e970eff2cb7d9509c8c414a\": rpc error: code = NotFound desc = could not find container \"66eb2dc35973bd557c9042a9e0b7776f13f8d8df1e970eff2cb7d9509c8c414a\": container with ID starting with 66eb2dc35973bd557c9042a9e0b7776f13f8d8df1e970eff2cb7d9509c8c414a not found: ID does not exist" Dec 03 12:38:43 crc kubenswrapper[5052]: I1203 12:38:43.804088 5052 scope.go:117] "RemoveContainer" containerID="f9ed4255fa67b2a8f659f0dd13829e2e2da10d2d67a6417060a75e2605964162" Dec 03 12:38:43 crc kubenswrapper[5052]: I1203 12:38:43.804264 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f9ed4255fa67b2a8f659f0dd13829e2e2da10d2d67a6417060a75e2605964162"} err="failed to get container status \"f9ed4255fa67b2a8f659f0dd13829e2e2da10d2d67a6417060a75e2605964162\": rpc error: code = NotFound desc = could not find container \"f9ed4255fa67b2a8f659f0dd13829e2e2da10d2d67a6417060a75e2605964162\": container with ID starting with f9ed4255fa67b2a8f659f0dd13829e2e2da10d2d67a6417060a75e2605964162 not found: ID does not exist" Dec 03 12:38:43 crc kubenswrapper[5052]: I1203 12:38:43.851579 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage09-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage09-crc\") pod \"ovsdbserver-sb-0\" (UID: \"d6bedb3e-52e6-44a9-afd9-a164b0730f52\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:38:43 crc kubenswrapper[5052]: I1203 12:38:43.851679 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cbd64\" (UniqueName: \"kubernetes.io/projected/d6bedb3e-52e6-44a9-afd9-a164b0730f52-kube-api-access-cbd64\") pod \"ovsdbserver-sb-0\" (UID: \"d6bedb3e-52e6-44a9-afd9-a164b0730f52\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:38:43 crc kubenswrapper[5052]: I1203 12:38:43.851710 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/d6bedb3e-52e6-44a9-afd9-a164b0730f52-scripts\") pod \"ovsdbserver-sb-0\" (UID: \"d6bedb3e-52e6-44a9-afd9-a164b0730f52\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:38:43 crc kubenswrapper[5052]: I1203 12:38:43.851814 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d6bedb3e-52e6-44a9-afd9-a164b0730f52-combined-ca-bundle\") pod \"ovsdbserver-sb-0\" (UID: \"d6bedb3e-52e6-44a9-afd9-a164b0730f52\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:38:43 crc kubenswrapper[5052]: I1203 12:38:43.851897 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/d6bedb3e-52e6-44a9-afd9-a164b0730f52-metrics-certs-tls-certs\") pod \"ovsdbserver-sb-0\" (UID: \"d6bedb3e-52e6-44a9-afd9-a164b0730f52\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:38:43 crc kubenswrapper[5052]: I1203 12:38:43.851923 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d6bedb3e-52e6-44a9-afd9-a164b0730f52-config\") pod \"ovsdbserver-sb-0\" (UID: \"d6bedb3e-52e6-44a9-afd9-a164b0730f52\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:38:43 crc kubenswrapper[5052]: I1203 12:38:43.852475 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovsdbserver-sb-tls-certs\" (UniqueName: \"kubernetes.io/secret/d6bedb3e-52e6-44a9-afd9-a164b0730f52-ovsdbserver-sb-tls-certs\") pod \"ovsdbserver-sb-0\" (UID: \"d6bedb3e-52e6-44a9-afd9-a164b0730f52\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:38:43 crc kubenswrapper[5052]: I1203 12:38:43.852585 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/d6bedb3e-52e6-44a9-afd9-a164b0730f52-ovsdb-rundir\") pod \"ovsdbserver-sb-0\" (UID: \"d6bedb3e-52e6-44a9-afd9-a164b0730f52\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:38:43 crc kubenswrapper[5052]: I1203 12:38:43.876669 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/memcached-0" Dec 03 12:38:43 crc kubenswrapper[5052]: I1203 12:38:43.954293 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/d6bedb3e-52e6-44a9-afd9-a164b0730f52-metrics-certs-tls-certs\") pod \"ovsdbserver-sb-0\" (UID: \"d6bedb3e-52e6-44a9-afd9-a164b0730f52\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:38:43 crc kubenswrapper[5052]: I1203 12:38:43.954358 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d6bedb3e-52e6-44a9-afd9-a164b0730f52-config\") pod \"ovsdbserver-sb-0\" (UID: \"d6bedb3e-52e6-44a9-afd9-a164b0730f52\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:38:43 crc kubenswrapper[5052]: I1203 12:38:43.954464 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovsdbserver-sb-tls-certs\" (UniqueName: \"kubernetes.io/secret/d6bedb3e-52e6-44a9-afd9-a164b0730f52-ovsdbserver-sb-tls-certs\") pod \"ovsdbserver-sb-0\" (UID: \"d6bedb3e-52e6-44a9-afd9-a164b0730f52\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:38:43 crc kubenswrapper[5052]: I1203 12:38:43.954495 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/d6bedb3e-52e6-44a9-afd9-a164b0730f52-ovsdb-rundir\") pod \"ovsdbserver-sb-0\" (UID: \"d6bedb3e-52e6-44a9-afd9-a164b0730f52\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:38:43 crc kubenswrapper[5052]: I1203 12:38:43.954543 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage09-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage09-crc\") pod \"ovsdbserver-sb-0\" (UID: \"d6bedb3e-52e6-44a9-afd9-a164b0730f52\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:38:43 crc kubenswrapper[5052]: I1203 12:38:43.954566 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-cbd64\" (UniqueName: \"kubernetes.io/projected/d6bedb3e-52e6-44a9-afd9-a164b0730f52-kube-api-access-cbd64\") pod \"ovsdbserver-sb-0\" (UID: \"d6bedb3e-52e6-44a9-afd9-a164b0730f52\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:38:43 crc kubenswrapper[5052]: I1203 12:38:43.954595 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/d6bedb3e-52e6-44a9-afd9-a164b0730f52-scripts\") pod \"ovsdbserver-sb-0\" (UID: \"d6bedb3e-52e6-44a9-afd9-a164b0730f52\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:38:43 crc kubenswrapper[5052]: I1203 12:38:43.954759 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d6bedb3e-52e6-44a9-afd9-a164b0730f52-combined-ca-bundle\") pod \"ovsdbserver-sb-0\" (UID: \"d6bedb3e-52e6-44a9-afd9-a164b0730f52\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:38:43 crc kubenswrapper[5052]: I1203 12:38:43.956028 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage09-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage09-crc\") pod \"ovsdbserver-sb-0\" (UID: \"d6bedb3e-52e6-44a9-afd9-a164b0730f52\") device mount path \"/mnt/openstack/pv09\"" pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:38:43 crc kubenswrapper[5052]: I1203 12:38:43.957691 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/d6bedb3e-52e6-44a9-afd9-a164b0730f52-scripts\") pod \"ovsdbserver-sb-0\" (UID: \"d6bedb3e-52e6-44a9-afd9-a164b0730f52\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:38:43 crc kubenswrapper[5052]: I1203 12:38:43.958328 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d6bedb3e-52e6-44a9-afd9-a164b0730f52-config\") pod \"ovsdbserver-sb-0\" (UID: \"d6bedb3e-52e6-44a9-afd9-a164b0730f52\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:38:43 crc kubenswrapper[5052]: I1203 12:38:43.959122 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/d6bedb3e-52e6-44a9-afd9-a164b0730f52-ovsdb-rundir\") pod \"ovsdbserver-sb-0\" (UID: \"d6bedb3e-52e6-44a9-afd9-a164b0730f52\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:38:43 crc kubenswrapper[5052]: I1203 12:38:43.965872 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/d6bedb3e-52e6-44a9-afd9-a164b0730f52-metrics-certs-tls-certs\") pod \"ovsdbserver-sb-0\" (UID: \"d6bedb3e-52e6-44a9-afd9-a164b0730f52\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:38:43 crc kubenswrapper[5052]: I1203 12:38:43.969393 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovsdbserver-sb-tls-certs\" (UniqueName: \"kubernetes.io/secret/d6bedb3e-52e6-44a9-afd9-a164b0730f52-ovsdbserver-sb-tls-certs\") pod \"ovsdbserver-sb-0\" (UID: \"d6bedb3e-52e6-44a9-afd9-a164b0730f52\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:38:43 crc kubenswrapper[5052]: I1203 12:38:43.976341 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d6bedb3e-52e6-44a9-afd9-a164b0730f52-combined-ca-bundle\") pod \"ovsdbserver-sb-0\" (UID: \"d6bedb3e-52e6-44a9-afd9-a164b0730f52\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:38:43 crc kubenswrapper[5052]: I1203 12:38:43.994233 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-cbd64\" (UniqueName: \"kubernetes.io/projected/d6bedb3e-52e6-44a9-afd9-a164b0730f52-kube-api-access-cbd64\") pod \"ovsdbserver-sb-0\" (UID: \"d6bedb3e-52e6-44a9-afd9-a164b0730f52\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:38:44 crc kubenswrapper[5052]: I1203 12:38:44.034965 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:38:44 crc kubenswrapper[5052]: I1203 12:38:44.035196 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-api-0" podUID="4a200957-c941-47a9-b881-6e9dbc8327b7" containerName="nova-api-log" containerID="cri-o://767ce5808c23166231a144b01e05e10cc1fe868247442d07a0b8b9a42c438b78" gracePeriod=30 Dec 03 12:38:44 crc kubenswrapper[5052]: I1203 12:38:44.035905 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-api-0" podUID="4a200957-c941-47a9-b881-6e9dbc8327b7" containerName="nova-api-api" containerID="cri-o://c489ec06ee93875a69927e3ababa6ff83e2ec14d6d1790c0dd006a349910b023" gracePeriod=30 Dec 03 12:38:44 crc kubenswrapper[5052]: I1203 12:38:44.060036 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage09-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage09-crc\") pod \"ovsdbserver-sb-0\" (UID: \"d6bedb3e-52e6-44a9-afd9-a164b0730f52\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:38:44 crc kubenswrapper[5052]: I1203 12:38:44.076017 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:38:44 crc kubenswrapper[5052]: I1203 12:38:44.104558 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 12:38:44 crc kubenswrapper[5052]: I1203 12:38:44.104820 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" podUID="a36c62b8-e624-450a-8b71-a33fc4395eee" containerName="nova-cell1-novncproxy-novncproxy" containerID="cri-o://5970858020fb206863e1145b9231a3f86c94c1e2d5ed380dc57653d6868c9b58" gracePeriod=30 Dec 03 12:38:44 crc kubenswrapper[5052]: W1203 12:38:44.128273 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod46c590ef_2b7c_412d_8582_dfdc311e6778.slice/crio-7ff432b7958a2acdc47a781956d13dd5c2c28341d37af9335f8ed7019cdf4272 WatchSource:0}: Error finding container 7ff432b7958a2acdc47a781956d13dd5c2c28341d37af9335f8ed7019cdf4272: Status 404 returned error can't find the container with id 7ff432b7958a2acdc47a781956d13dd5c2c28341d37af9335f8ed7019cdf4272 Dec 03 12:38:44 crc kubenswrapper[5052]: I1203 12:38:44.136893 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-nb-0"] Dec 03 12:38:44 crc kubenswrapper[5052]: I1203 12:38:44.222543 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/neutron-544c66dd6d-75q8t"] Dec 03 12:38:44 crc kubenswrapper[5052]: I1203 12:38:44.224691 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-544c66dd6d-75q8t" Dec 03 12:38:44 crc kubenswrapper[5052]: I1203 12:38:44.258233 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-api-b857b994d-6xzbl" Dec 03 12:38:44 crc kubenswrapper[5052]: I1203 12:38:44.266639 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8zvdv\" (UniqueName: \"kubernetes.io/projected/c34523d4-d634-4d76-a238-36566ef6a682-kube-api-access-8zvdv\") pod \"neutron-544c66dd6d-75q8t\" (UID: \"c34523d4-d634-4d76-a238-36566ef6a682\") " pod="openstack-kuttl-tests/neutron-544c66dd6d-75q8t" Dec 03 12:38:44 crc kubenswrapper[5052]: I1203 12:38:44.266694 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/c34523d4-d634-4d76-a238-36566ef6a682-ovndb-tls-certs\") pod \"neutron-544c66dd6d-75q8t\" (UID: \"c34523d4-d634-4d76-a238-36566ef6a682\") " pod="openstack-kuttl-tests/neutron-544c66dd6d-75q8t" Dec 03 12:38:44 crc kubenswrapper[5052]: I1203 12:38:44.266769 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/c34523d4-d634-4d76-a238-36566ef6a682-public-tls-certs\") pod \"neutron-544c66dd6d-75q8t\" (UID: \"c34523d4-d634-4d76-a238-36566ef6a682\") " pod="openstack-kuttl-tests/neutron-544c66dd6d-75q8t" Dec 03 12:38:44 crc kubenswrapper[5052]: I1203 12:38:44.267908 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/c34523d4-d634-4d76-a238-36566ef6a682-httpd-config\") pod \"neutron-544c66dd6d-75q8t\" (UID: \"c34523d4-d634-4d76-a238-36566ef6a682\") " pod="openstack-kuttl-tests/neutron-544c66dd6d-75q8t" Dec 03 12:38:44 crc kubenswrapper[5052]: I1203 12:38:44.268406 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/c34523d4-d634-4d76-a238-36566ef6a682-config\") pod \"neutron-544c66dd6d-75q8t\" (UID: \"c34523d4-d634-4d76-a238-36566ef6a682\") " pod="openstack-kuttl-tests/neutron-544c66dd6d-75q8t" Dec 03 12:38:44 crc kubenswrapper[5052]: I1203 12:38:44.268503 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/c34523d4-d634-4d76-a238-36566ef6a682-internal-tls-certs\") pod \"neutron-544c66dd6d-75q8t\" (UID: \"c34523d4-d634-4d76-a238-36566ef6a682\") " pod="openstack-kuttl-tests/neutron-544c66dd6d-75q8t" Dec 03 12:38:44 crc kubenswrapper[5052]: I1203 12:38:44.268694 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c34523d4-d634-4d76-a238-36566ef6a682-combined-ca-bundle\") pod \"neutron-544c66dd6d-75q8t\" (UID: \"c34523d4-d634-4d76-a238-36566ef6a682\") " pod="openstack-kuttl-tests/neutron-544c66dd6d-75q8t" Dec 03 12:38:44 crc kubenswrapper[5052]: I1203 12:38:44.276860 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-544c66dd6d-75q8t"] Dec 03 12:38:44 crc kubenswrapper[5052]: I1203 12:38:44.370783 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/3ceac470-11eb-4d69-a683-2e9bc4e8f617-internal-tls-certs\") pod \"3ceac470-11eb-4d69-a683-2e9bc4e8f617\" (UID: \"3ceac470-11eb-4d69-a683-2e9bc4e8f617\") " Dec 03 12:38:44 crc kubenswrapper[5052]: I1203 12:38:44.370934 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-khg4m\" (UniqueName: \"kubernetes.io/projected/3ceac470-11eb-4d69-a683-2e9bc4e8f617-kube-api-access-khg4m\") pod \"3ceac470-11eb-4d69-a683-2e9bc4e8f617\" (UID: \"3ceac470-11eb-4d69-a683-2e9bc4e8f617\") " Dec 03 12:38:44 crc kubenswrapper[5052]: I1203 12:38:44.370974 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/3ceac470-11eb-4d69-a683-2e9bc4e8f617-config-data-custom\") pod \"3ceac470-11eb-4d69-a683-2e9bc4e8f617\" (UID: \"3ceac470-11eb-4d69-a683-2e9bc4e8f617\") " Dec 03 12:38:44 crc kubenswrapper[5052]: I1203 12:38:44.371128 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3ceac470-11eb-4d69-a683-2e9bc4e8f617-combined-ca-bundle\") pod \"3ceac470-11eb-4d69-a683-2e9bc4e8f617\" (UID: \"3ceac470-11eb-4d69-a683-2e9bc4e8f617\") " Dec 03 12:38:44 crc kubenswrapper[5052]: I1203 12:38:44.371286 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/3ceac470-11eb-4d69-a683-2e9bc4e8f617-logs\") pod \"3ceac470-11eb-4d69-a683-2e9bc4e8f617\" (UID: \"3ceac470-11eb-4d69-a683-2e9bc4e8f617\") " Dec 03 12:38:44 crc kubenswrapper[5052]: I1203 12:38:44.371326 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/3ceac470-11eb-4d69-a683-2e9bc4e8f617-public-tls-certs\") pod \"3ceac470-11eb-4d69-a683-2e9bc4e8f617\" (UID: \"3ceac470-11eb-4d69-a683-2e9bc4e8f617\") " Dec 03 12:38:44 crc kubenswrapper[5052]: I1203 12:38:44.371350 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/3ceac470-11eb-4d69-a683-2e9bc4e8f617-config-data\") pod \"3ceac470-11eb-4d69-a683-2e9bc4e8f617\" (UID: \"3ceac470-11eb-4d69-a683-2e9bc4e8f617\") " Dec 03 12:38:44 crc kubenswrapper[5052]: I1203 12:38:44.371694 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c34523d4-d634-4d76-a238-36566ef6a682-combined-ca-bundle\") pod \"neutron-544c66dd6d-75q8t\" (UID: \"c34523d4-d634-4d76-a238-36566ef6a682\") " pod="openstack-kuttl-tests/neutron-544c66dd6d-75q8t" Dec 03 12:38:44 crc kubenswrapper[5052]: I1203 12:38:44.371811 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-8zvdv\" (UniqueName: \"kubernetes.io/projected/c34523d4-d634-4d76-a238-36566ef6a682-kube-api-access-8zvdv\") pod \"neutron-544c66dd6d-75q8t\" (UID: \"c34523d4-d634-4d76-a238-36566ef6a682\") " pod="openstack-kuttl-tests/neutron-544c66dd6d-75q8t" Dec 03 12:38:44 crc kubenswrapper[5052]: I1203 12:38:44.371839 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/c34523d4-d634-4d76-a238-36566ef6a682-ovndb-tls-certs\") pod \"neutron-544c66dd6d-75q8t\" (UID: \"c34523d4-d634-4d76-a238-36566ef6a682\") " pod="openstack-kuttl-tests/neutron-544c66dd6d-75q8t" Dec 03 12:38:44 crc kubenswrapper[5052]: I1203 12:38:44.371866 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/c34523d4-d634-4d76-a238-36566ef6a682-public-tls-certs\") pod \"neutron-544c66dd6d-75q8t\" (UID: \"c34523d4-d634-4d76-a238-36566ef6a682\") " pod="openstack-kuttl-tests/neutron-544c66dd6d-75q8t" Dec 03 12:38:44 crc kubenswrapper[5052]: I1203 12:38:44.371931 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/c34523d4-d634-4d76-a238-36566ef6a682-httpd-config\") pod \"neutron-544c66dd6d-75q8t\" (UID: \"c34523d4-d634-4d76-a238-36566ef6a682\") " pod="openstack-kuttl-tests/neutron-544c66dd6d-75q8t" Dec 03 12:38:44 crc kubenswrapper[5052]: I1203 12:38:44.372027 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/c34523d4-d634-4d76-a238-36566ef6a682-config\") pod \"neutron-544c66dd6d-75q8t\" (UID: \"c34523d4-d634-4d76-a238-36566ef6a682\") " pod="openstack-kuttl-tests/neutron-544c66dd6d-75q8t" Dec 03 12:38:44 crc kubenswrapper[5052]: I1203 12:38:44.372058 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/c34523d4-d634-4d76-a238-36566ef6a682-internal-tls-certs\") pod \"neutron-544c66dd6d-75q8t\" (UID: \"c34523d4-d634-4d76-a238-36566ef6a682\") " pod="openstack-kuttl-tests/neutron-544c66dd6d-75q8t" Dec 03 12:38:44 crc kubenswrapper[5052]: I1203 12:38:44.372188 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3ceac470-11eb-4d69-a683-2e9bc4e8f617-logs" (OuterVolumeSpecName: "logs") pod "3ceac470-11eb-4d69-a683-2e9bc4e8f617" (UID: "3ceac470-11eb-4d69-a683-2e9bc4e8f617"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:38:44 crc kubenswrapper[5052]: I1203 12:38:44.383021 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3ceac470-11eb-4d69-a683-2e9bc4e8f617-config-data-custom" (OuterVolumeSpecName: "config-data-custom") pod "3ceac470-11eb-4d69-a683-2e9bc4e8f617" (UID: "3ceac470-11eb-4d69-a683-2e9bc4e8f617"). InnerVolumeSpecName "config-data-custom". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:38:44 crc kubenswrapper[5052]: I1203 12:38:44.384098 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/c34523d4-d634-4d76-a238-36566ef6a682-internal-tls-certs\") pod \"neutron-544c66dd6d-75q8t\" (UID: \"c34523d4-d634-4d76-a238-36566ef6a682\") " pod="openstack-kuttl-tests/neutron-544c66dd6d-75q8t" Dec 03 12:38:44 crc kubenswrapper[5052]: I1203 12:38:44.385660 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3ceac470-11eb-4d69-a683-2e9bc4e8f617-kube-api-access-khg4m" (OuterVolumeSpecName: "kube-api-access-khg4m") pod "3ceac470-11eb-4d69-a683-2e9bc4e8f617" (UID: "3ceac470-11eb-4d69-a683-2e9bc4e8f617"). InnerVolumeSpecName "kube-api-access-khg4m". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:38:44 crc kubenswrapper[5052]: I1203 12:38:44.391885 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/c34523d4-d634-4d76-a238-36566ef6a682-public-tls-certs\") pod \"neutron-544c66dd6d-75q8t\" (UID: \"c34523d4-d634-4d76-a238-36566ef6a682\") " pod="openstack-kuttl-tests/neutron-544c66dd6d-75q8t" Dec 03 12:38:44 crc kubenswrapper[5052]: I1203 12:38:44.393024 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-8zvdv\" (UniqueName: \"kubernetes.io/projected/c34523d4-d634-4d76-a238-36566ef6a682-kube-api-access-8zvdv\") pod \"neutron-544c66dd6d-75q8t\" (UID: \"c34523d4-d634-4d76-a238-36566ef6a682\") " pod="openstack-kuttl-tests/neutron-544c66dd6d-75q8t" Dec 03 12:38:44 crc kubenswrapper[5052]: I1203 12:38:44.393855 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/c34523d4-d634-4d76-a238-36566ef6a682-httpd-config\") pod \"neutron-544c66dd6d-75q8t\" (UID: \"c34523d4-d634-4d76-a238-36566ef6a682\") " pod="openstack-kuttl-tests/neutron-544c66dd6d-75q8t" Dec 03 12:38:44 crc kubenswrapper[5052]: I1203 12:38:44.395538 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/c34523d4-d634-4d76-a238-36566ef6a682-config\") pod \"neutron-544c66dd6d-75q8t\" (UID: \"c34523d4-d634-4d76-a238-36566ef6a682\") " pod="openstack-kuttl-tests/neutron-544c66dd6d-75q8t" Dec 03 12:38:44 crc kubenswrapper[5052]: I1203 12:38:44.402847 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/c34523d4-d634-4d76-a238-36566ef6a682-ovndb-tls-certs\") pod \"neutron-544c66dd6d-75q8t\" (UID: \"c34523d4-d634-4d76-a238-36566ef6a682\") " pod="openstack-kuttl-tests/neutron-544c66dd6d-75q8t" Dec 03 12:38:44 crc kubenswrapper[5052]: I1203 12:38:44.403805 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c34523d4-d634-4d76-a238-36566ef6a682-combined-ca-bundle\") pod \"neutron-544c66dd6d-75q8t\" (UID: \"c34523d4-d634-4d76-a238-36566ef6a682\") " pod="openstack-kuttl-tests/neutron-544c66dd6d-75q8t" Dec 03 12:38:44 crc kubenswrapper[5052]: I1203 12:38:44.413305 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3ceac470-11eb-4d69-a683-2e9bc4e8f617-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "3ceac470-11eb-4d69-a683-2e9bc4e8f617" (UID: "3ceac470-11eb-4d69-a683-2e9bc4e8f617"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:38:44 crc kubenswrapper[5052]: I1203 12:38:44.454042 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3ceac470-11eb-4d69-a683-2e9bc4e8f617-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "3ceac470-11eb-4d69-a683-2e9bc4e8f617" (UID: "3ceac470-11eb-4d69-a683-2e9bc4e8f617"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:38:44 crc kubenswrapper[5052]: I1203 12:38:44.462280 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3ceac470-11eb-4d69-a683-2e9bc4e8f617-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "3ceac470-11eb-4d69-a683-2e9bc4e8f617" (UID: "3ceac470-11eb-4d69-a683-2e9bc4e8f617"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:38:44 crc kubenswrapper[5052]: I1203 12:38:44.464608 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3ceac470-11eb-4d69-a683-2e9bc4e8f617-config-data" (OuterVolumeSpecName: "config-data") pod "3ceac470-11eb-4d69-a683-2e9bc4e8f617" (UID: "3ceac470-11eb-4d69-a683-2e9bc4e8f617"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:38:44 crc kubenswrapper[5052]: I1203 12:38:44.473960 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-khg4m\" (UniqueName: \"kubernetes.io/projected/3ceac470-11eb-4d69-a683-2e9bc4e8f617-kube-api-access-khg4m\") on node \"crc\" DevicePath \"\"" Dec 03 12:38:44 crc kubenswrapper[5052]: I1203 12:38:44.473996 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/3ceac470-11eb-4d69-a683-2e9bc4e8f617-config-data-custom\") on node \"crc\" DevicePath \"\"" Dec 03 12:38:44 crc kubenswrapper[5052]: I1203 12:38:44.474005 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3ceac470-11eb-4d69-a683-2e9bc4e8f617-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:38:44 crc kubenswrapper[5052]: I1203 12:38:44.474014 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/3ceac470-11eb-4d69-a683-2e9bc4e8f617-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:38:44 crc kubenswrapper[5052]: I1203 12:38:44.474025 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/3ceac470-11eb-4d69-a683-2e9bc4e8f617-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:38:44 crc kubenswrapper[5052]: I1203 12:38:44.474035 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/3ceac470-11eb-4d69-a683-2e9bc4e8f617-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:38:44 crc kubenswrapper[5052]: I1203 12:38:44.474045 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/3ceac470-11eb-4d69-a683-2e9bc4e8f617-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:38:44 crc kubenswrapper[5052]: I1203 12:38:44.570595 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-544c66dd6d-75q8t" Dec 03 12:38:44 crc kubenswrapper[5052]: E1203 12:38:44.575330 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-swift-public-svc: secret "cert-swift-public-svc" not found Dec 03 12:38:44 crc kubenswrapper[5052]: E1203 12:38:44.575408 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/06b2a08b-6b91-44f2-b630-169938fe5771-public-tls-certs podName:06b2a08b-6b91-44f2-b630-169938fe5771 nodeName:}" failed. No retries permitted until 2025-12-03 12:39:00.57538903 +0000 UTC m=+4731.163630794 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "public-tls-certs" (UniqueName: "kubernetes.io/secret/06b2a08b-6b91-44f2-b630-169938fe5771-public-tls-certs") pod "swift-proxy-55f4df7cd-4gjcp" (UID: "06b2a08b-6b91-44f2-b630-169938fe5771") : secret "cert-swift-public-svc" not found Dec 03 12:38:44 crc kubenswrapper[5052]: E1203 12:38:44.575812 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-swift-internal-svc: secret "cert-swift-internal-svc" not found Dec 03 12:38:44 crc kubenswrapper[5052]: E1203 12:38:44.575852 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/06b2a08b-6b91-44f2-b630-169938fe5771-internal-tls-certs podName:06b2a08b-6b91-44f2-b630-169938fe5771 nodeName:}" failed. No retries permitted until 2025-12-03 12:39:00.575842053 +0000 UTC m=+4731.164083817 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "internal-tls-certs" (UniqueName: "kubernetes.io/secret/06b2a08b-6b91-44f2-b630-169938fe5771-internal-tls-certs") pod "swift-proxy-55f4df7cd-4gjcp" (UID: "06b2a08b-6b91-44f2-b630-169938fe5771") : secret "cert-swift-internal-svc" not found Dec 03 12:38:44 crc kubenswrapper[5052]: I1203 12:38:44.626816 5052 generic.go:334] "Generic (PLEG): container finished" podID="4a200957-c941-47a9-b881-6e9dbc8327b7" containerID="767ce5808c23166231a144b01e05e10cc1fe868247442d07a0b8b9a42c438b78" exitCode=143 Dec 03 12:38:44 crc kubenswrapper[5052]: I1203 12:38:44.626874 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"4a200957-c941-47a9-b881-6e9dbc8327b7","Type":"ContainerDied","Data":"767ce5808c23166231a144b01e05e10cc1fe868247442d07a0b8b9a42c438b78"} Dec 03 12:38:44 crc kubenswrapper[5052]: I1203 12:38:44.630199 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-nb-0" event={"ID":"46c590ef-2b7c-412d-8582-dfdc311e6778","Type":"ContainerStarted","Data":"7ff432b7958a2acdc47a781956d13dd5c2c28341d37af9335f8ed7019cdf4272"} Dec 03 12:38:44 crc kubenswrapper[5052]: I1203 12:38:44.634004 5052 generic.go:334] "Generic (PLEG): container finished" podID="3ceac470-11eb-4d69-a683-2e9bc4e8f617" containerID="fc0930236187285aa345d3839c9e4d90b9b3da00e5015ee57851c76c3228e9b7" exitCode=0 Dec 03 12:38:44 crc kubenswrapper[5052]: I1203 12:38:44.635132 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-api-b857b994d-6xzbl" Dec 03 12:38:44 crc kubenswrapper[5052]: I1203 12:38:44.637558 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-b857b994d-6xzbl" event={"ID":"3ceac470-11eb-4d69-a683-2e9bc4e8f617","Type":"ContainerDied","Data":"fc0930236187285aa345d3839c9e4d90b9b3da00e5015ee57851c76c3228e9b7"} Dec 03 12:38:44 crc kubenswrapper[5052]: I1203 12:38:44.637692 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-b857b994d-6xzbl" event={"ID":"3ceac470-11eb-4d69-a683-2e9bc4e8f617","Type":"ContainerDied","Data":"2da51349cc9b8a1c7a1cb9bf175483e9aae3f05e945d00d68bc9cef218210bc8"} Dec 03 12:38:44 crc kubenswrapper[5052]: I1203 12:38:44.637779 5052 scope.go:117] "RemoveContainer" containerID="fc0930236187285aa345d3839c9e4d90b9b3da00e5015ee57851c76c3228e9b7" Dec 03 12:38:44 crc kubenswrapper[5052]: I1203 12:38:44.693443 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-api-b857b994d-6xzbl"] Dec 03 12:38:44 crc kubenswrapper[5052]: I1203 12:38:44.704792 5052 scope.go:117] "RemoveContainer" containerID="1fe0c8f8d952b7b1bbdf2ddc25bf96bcb1eeeec47a75464c77ed9ef77f45fc86" Dec 03 12:38:44 crc kubenswrapper[5052]: I1203 12:38:44.713880 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/barbican-api-b857b994d-6xzbl"] Dec 03 12:38:44 crc kubenswrapper[5052]: I1203 12:38:44.741036 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-sb-0"] Dec 03 12:38:44 crc kubenswrapper[5052]: I1203 12:38:44.743968 5052 scope.go:117] "RemoveContainer" containerID="fc0930236187285aa345d3839c9e4d90b9b3da00e5015ee57851c76c3228e9b7" Dec 03 12:38:44 crc kubenswrapper[5052]: E1203 12:38:44.746865 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fc0930236187285aa345d3839c9e4d90b9b3da00e5015ee57851c76c3228e9b7\": container with ID starting with fc0930236187285aa345d3839c9e4d90b9b3da00e5015ee57851c76c3228e9b7 not found: ID does not exist" containerID="fc0930236187285aa345d3839c9e4d90b9b3da00e5015ee57851c76c3228e9b7" Dec 03 12:38:44 crc kubenswrapper[5052]: I1203 12:38:44.746923 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fc0930236187285aa345d3839c9e4d90b9b3da00e5015ee57851c76c3228e9b7"} err="failed to get container status \"fc0930236187285aa345d3839c9e4d90b9b3da00e5015ee57851c76c3228e9b7\": rpc error: code = NotFound desc = could not find container \"fc0930236187285aa345d3839c9e4d90b9b3da00e5015ee57851c76c3228e9b7\": container with ID starting with fc0930236187285aa345d3839c9e4d90b9b3da00e5015ee57851c76c3228e9b7 not found: ID does not exist" Dec 03 12:38:44 crc kubenswrapper[5052]: I1203 12:38:44.746945 5052 scope.go:117] "RemoveContainer" containerID="1fe0c8f8d952b7b1bbdf2ddc25bf96bcb1eeeec47a75464c77ed9ef77f45fc86" Dec 03 12:38:44 crc kubenswrapper[5052]: E1203 12:38:44.747915 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1fe0c8f8d952b7b1bbdf2ddc25bf96bcb1eeeec47a75464c77ed9ef77f45fc86\": container with ID starting with 1fe0c8f8d952b7b1bbdf2ddc25bf96bcb1eeeec47a75464c77ed9ef77f45fc86 not found: ID does not exist" containerID="1fe0c8f8d952b7b1bbdf2ddc25bf96bcb1eeeec47a75464c77ed9ef77f45fc86" Dec 03 12:38:44 crc kubenswrapper[5052]: I1203 12:38:44.747942 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1fe0c8f8d952b7b1bbdf2ddc25bf96bcb1eeeec47a75464c77ed9ef77f45fc86"} err="failed to get container status \"1fe0c8f8d952b7b1bbdf2ddc25bf96bcb1eeeec47a75464c77ed9ef77f45fc86\": rpc error: code = NotFound desc = could not find container \"1fe0c8f8d952b7b1bbdf2ddc25bf96bcb1eeeec47a75464c77ed9ef77f45fc86\": container with ID starting with 1fe0c8f8d952b7b1bbdf2ddc25bf96bcb1eeeec47a75464c77ed9ef77f45fc86 not found: ID does not exist" Dec 03 12:38:44 crc kubenswrapper[5052]: I1203 12:38:44.827149 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:38:45 crc kubenswrapper[5052]: I1203 12:38:45.063896 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:38:45 crc kubenswrapper[5052]: I1203 12:38:45.068549 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-544c66dd6d-75q8t"] Dec 03 12:38:45 crc kubenswrapper[5052]: W1203 12:38:45.077216 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc34523d4_d634_4d76_a238_36566ef6a682.slice/crio-2da7df7c61cec0233bc5232f83d4c248600d2c0d8ce92aed8c21d58a826fea98 WatchSource:0}: Error finding container 2da7df7c61cec0233bc5232f83d4c248600d2c0d8ce92aed8c21d58a826fea98: Status 404 returned error can't find the container with id 2da7df7c61cec0233bc5232f83d4c248600d2c0d8ce92aed8c21d58a826fea98 Dec 03 12:38:45 crc kubenswrapper[5052]: E1203 12:38:45.236450 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-ovnnorthd-ovndbs: secret "cert-ovnnorthd-ovndbs" not found Dec 03 12:38:45 crc kubenswrapper[5052]: E1203 12:38:45.237876 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/92dadf05-a318-4491-bb1c-1fa23cb293fa-ovn-northd-tls-certs podName:92dadf05-a318-4491-bb1c-1fa23cb293fa nodeName:}" failed. No retries permitted until 2025-12-03 12:38:49.237853456 +0000 UTC m=+4719.826095280 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "ovn-northd-tls-certs" (UniqueName: "kubernetes.io/secret/92dadf05-a318-4491-bb1c-1fa23cb293fa-ovn-northd-tls-certs") pod "ovn-northd-0" (UID: "92dadf05-a318-4491-bb1c-1fa23cb293fa") : secret "cert-ovnnorthd-ovndbs" not found Dec 03 12:38:45 crc kubenswrapper[5052]: I1203 12:38:45.294159 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ovn-northd-0"] Dec 03 12:38:45 crc kubenswrapper[5052]: I1203 12:38:45.294435 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ovn-northd-0" podUID="92dadf05-a318-4491-bb1c-1fa23cb293fa" containerName="ovn-northd" containerID="cri-o://4e823788ee83fdbbba2177f783a9bb4380ce320612fdfe974cb17f48a899577b" gracePeriod=30 Dec 03 12:38:45 crc kubenswrapper[5052]: I1203 12:38:45.294562 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ovn-northd-0" podUID="92dadf05-a318-4491-bb1c-1fa23cb293fa" containerName="openstack-network-exporter" containerID="cri-o://94a951ac5d9669865bf707580490a8df30f61838bbfebbf2fccdc24d07f308a3" gracePeriod=30 Dec 03 12:38:45 crc kubenswrapper[5052]: I1203 12:38:45.664090 5052 generic.go:334] "Generic (PLEG): container finished" podID="92dadf05-a318-4491-bb1c-1fa23cb293fa" containerID="94a951ac5d9669865bf707580490a8df30f61838bbfebbf2fccdc24d07f308a3" exitCode=2 Dec 03 12:38:45 crc kubenswrapper[5052]: I1203 12:38:45.664351 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovn-northd-0" event={"ID":"92dadf05-a318-4491-bb1c-1fa23cb293fa","Type":"ContainerDied","Data":"94a951ac5d9669865bf707580490a8df30f61838bbfebbf2fccdc24d07f308a3"} Dec 03 12:38:45 crc kubenswrapper[5052]: I1203 12:38:45.666605 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-nb-0" event={"ID":"46c590ef-2b7c-412d-8582-dfdc311e6778","Type":"ContainerStarted","Data":"e918d411b918ed8ca93043d08217369893147e6405ff8425f54ff29adf217db7"} Dec 03 12:38:45 crc kubenswrapper[5052]: I1203 12:38:45.666637 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-nb-0" event={"ID":"46c590ef-2b7c-412d-8582-dfdc311e6778","Type":"ContainerStarted","Data":"db6fc581a8df4ec1884c71516e39b9792e494f950b83f3b0b693c01758dfa0f1"} Dec 03 12:38:45 crc kubenswrapper[5052]: I1203 12:38:45.668317 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-sb-0" event={"ID":"d6bedb3e-52e6-44a9-afd9-a164b0730f52","Type":"ContainerStarted","Data":"3f73a35540704d1c4ac272256521dc836a1d429fbe1cee0a507bfe3859a50b06"} Dec 03 12:38:45 crc kubenswrapper[5052]: I1203 12:38:45.668346 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-sb-0" event={"ID":"d6bedb3e-52e6-44a9-afd9-a164b0730f52","Type":"ContainerStarted","Data":"87e14300e8697a4e0c8594bb1dde5f610d618d2da77d5478a1bb33029c23a5e0"} Dec 03 12:38:45 crc kubenswrapper[5052]: I1203 12:38:45.668355 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-sb-0" event={"ID":"d6bedb3e-52e6-44a9-afd9-a164b0730f52","Type":"ContainerStarted","Data":"ac810b46f2960a4c26ef9798138b1d1d954bd6def2c577bfa4e87659f07846f6"} Dec 03 12:38:45 crc kubenswrapper[5052]: I1203 12:38:45.672059 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-544c66dd6d-75q8t" event={"ID":"c34523d4-d634-4d76-a238-36566ef6a682","Type":"ContainerStarted","Data":"2bddb69fe53e0c8a15b016cc5ddde5164332e92554c29e75c2ee6664a3dd67a7"} Dec 03 12:38:45 crc kubenswrapper[5052]: I1203 12:38:45.673562 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:38:45 crc kubenswrapper[5052]: I1203 12:38:45.674051 5052 generic.go:334] "Generic (PLEG): container finished" podID="a36c62b8-e624-450a-8b71-a33fc4395eee" containerID="5970858020fb206863e1145b9231a3f86c94c1e2d5ed380dc57653d6868c9b58" exitCode=0 Dec 03 12:38:45 crc kubenswrapper[5052]: I1203 12:38:45.689971 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="143d0ab6-f833-41a9-aacd-34d20b35f8ed" path="/var/lib/kubelet/pods/143d0ab6-f833-41a9-aacd-34d20b35f8ed/volumes" Dec 03 12:38:45 crc kubenswrapper[5052]: I1203 12:38:45.691106 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3ceac470-11eb-4d69-a683-2e9bc4e8f617" path="/var/lib/kubelet/pods/3ceac470-11eb-4d69-a683-2e9bc4e8f617/volumes" Dec 03 12:38:45 crc kubenswrapper[5052]: I1203 12:38:45.710744 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-544c66dd6d-75q8t" event={"ID":"c34523d4-d634-4d76-a238-36566ef6a682","Type":"ContainerStarted","Data":"2da7df7c61cec0233bc5232f83d4c248600d2c0d8ce92aed8c21d58a826fea98"} Dec 03 12:38:45 crc kubenswrapper[5052]: I1203 12:38:45.710790 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" event={"ID":"a36c62b8-e624-450a-8b71-a33fc4395eee","Type":"ContainerDied","Data":"5970858020fb206863e1145b9231a3f86c94c1e2d5ed380dc57653d6868c9b58"} Dec 03 12:38:45 crc kubenswrapper[5052]: I1203 12:38:45.710818 5052 scope.go:117] "RemoveContainer" containerID="5970858020fb206863e1145b9231a3f86c94c1e2d5ed380dc57653d6868c9b58" Dec 03 12:38:45 crc kubenswrapper[5052]: I1203 12:38:45.745606 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x227p\" (UniqueName: \"kubernetes.io/projected/a36c62b8-e624-450a-8b71-a33fc4395eee-kube-api-access-x227p\") pod \"a36c62b8-e624-450a-8b71-a33fc4395eee\" (UID: \"a36c62b8-e624-450a-8b71-a33fc4395eee\") " Dec 03 12:38:45 crc kubenswrapper[5052]: I1203 12:38:45.745676 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"nova-novncproxy-tls-certs\" (UniqueName: \"kubernetes.io/secret/a36c62b8-e624-450a-8b71-a33fc4395eee-nova-novncproxy-tls-certs\") pod \"a36c62b8-e624-450a-8b71-a33fc4395eee\" (UID: \"a36c62b8-e624-450a-8b71-a33fc4395eee\") " Dec 03 12:38:45 crc kubenswrapper[5052]: I1203 12:38:45.745743 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"vencrypt-tls-certs\" (UniqueName: \"kubernetes.io/secret/a36c62b8-e624-450a-8b71-a33fc4395eee-vencrypt-tls-certs\") pod \"a36c62b8-e624-450a-8b71-a33fc4395eee\" (UID: \"a36c62b8-e624-450a-8b71-a33fc4395eee\") " Dec 03 12:38:45 crc kubenswrapper[5052]: I1203 12:38:45.745786 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a36c62b8-e624-450a-8b71-a33fc4395eee-config-data\") pod \"a36c62b8-e624-450a-8b71-a33fc4395eee\" (UID: \"a36c62b8-e624-450a-8b71-a33fc4395eee\") " Dec 03 12:38:45 crc kubenswrapper[5052]: I1203 12:38:45.745842 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a36c62b8-e624-450a-8b71-a33fc4395eee-combined-ca-bundle\") pod \"a36c62b8-e624-450a-8b71-a33fc4395eee\" (UID: \"a36c62b8-e624-450a-8b71-a33fc4395eee\") " Dec 03 12:38:45 crc kubenswrapper[5052]: I1203 12:38:45.764037 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a36c62b8-e624-450a-8b71-a33fc4395eee-kube-api-access-x227p" (OuterVolumeSpecName: "kube-api-access-x227p") pod "a36c62b8-e624-450a-8b71-a33fc4395eee" (UID: "a36c62b8-e624-450a-8b71-a33fc4395eee"). InnerVolumeSpecName "kube-api-access-x227p". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:38:45 crc kubenswrapper[5052]: I1203 12:38:45.836411 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a36c62b8-e624-450a-8b71-a33fc4395eee-config-data" (OuterVolumeSpecName: "config-data") pod "a36c62b8-e624-450a-8b71-a33fc4395eee" (UID: "a36c62b8-e624-450a-8b71-a33fc4395eee"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:38:45 crc kubenswrapper[5052]: I1203 12:38:45.848883 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-x227p\" (UniqueName: \"kubernetes.io/projected/a36c62b8-e624-450a-8b71-a33fc4395eee-kube-api-access-x227p\") on node \"crc\" DevicePath \"\"" Dec 03 12:38:45 crc kubenswrapper[5052]: I1203 12:38:45.849116 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a36c62b8-e624-450a-8b71-a33fc4395eee-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:38:45 crc kubenswrapper[5052]: I1203 12:38:45.851944 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a36c62b8-e624-450a-8b71-a33fc4395eee-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "a36c62b8-e624-450a-8b71-a33fc4395eee" (UID: "a36c62b8-e624-450a-8b71-a33fc4395eee"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:38:45 crc kubenswrapper[5052]: I1203 12:38:45.861442 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a36c62b8-e624-450a-8b71-a33fc4395eee-nova-novncproxy-tls-certs" (OuterVolumeSpecName: "nova-novncproxy-tls-certs") pod "a36c62b8-e624-450a-8b71-a33fc4395eee" (UID: "a36c62b8-e624-450a-8b71-a33fc4395eee"). InnerVolumeSpecName "nova-novncproxy-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:38:45 crc kubenswrapper[5052]: I1203 12:38:45.872054 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a36c62b8-e624-450a-8b71-a33fc4395eee-vencrypt-tls-certs" (OuterVolumeSpecName: "vencrypt-tls-certs") pod "a36c62b8-e624-450a-8b71-a33fc4395eee" (UID: "a36c62b8-e624-450a-8b71-a33fc4395eee"). InnerVolumeSpecName "vencrypt-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:38:45 crc kubenswrapper[5052]: I1203 12:38:45.939992 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:38:45 crc kubenswrapper[5052]: I1203 12:38:45.940036 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:38:45 crc kubenswrapper[5052]: I1203 12:38:45.953528 5052 reconciler_common.go:293] "Volume detached for volume \"nova-novncproxy-tls-certs\" (UniqueName: \"kubernetes.io/secret/a36c62b8-e624-450a-8b71-a33fc4395eee-nova-novncproxy-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:38:45 crc kubenswrapper[5052]: I1203 12:38:45.953740 5052 reconciler_common.go:293] "Volume detached for volume \"vencrypt-tls-certs\" (UniqueName: \"kubernetes.io/secret/a36c62b8-e624-450a-8b71-a33fc4395eee-vencrypt-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:38:45 crc kubenswrapper[5052]: I1203 12:38:45.953761 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a36c62b8-e624-450a-8b71-a33fc4395eee-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:38:46 crc kubenswrapper[5052]: I1203 12:38:46.169305 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/placement-76bf6b6758-lvtjf"] Dec 03 12:38:46 crc kubenswrapper[5052]: E1203 12:38:46.169781 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="3ceac470-11eb-4d69-a683-2e9bc4e8f617" containerName="barbican-api-log" Dec 03 12:38:46 crc kubenswrapper[5052]: I1203 12:38:46.169798 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="3ceac470-11eb-4d69-a683-2e9bc4e8f617" containerName="barbican-api-log" Dec 03 12:38:46 crc kubenswrapper[5052]: E1203 12:38:46.169850 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="a36c62b8-e624-450a-8b71-a33fc4395eee" containerName="nova-cell1-novncproxy-novncproxy" Dec 03 12:38:46 crc kubenswrapper[5052]: I1203 12:38:46.169858 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="a36c62b8-e624-450a-8b71-a33fc4395eee" containerName="nova-cell1-novncproxy-novncproxy" Dec 03 12:38:46 crc kubenswrapper[5052]: E1203 12:38:46.169874 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="3ceac470-11eb-4d69-a683-2e9bc4e8f617" containerName="barbican-api" Dec 03 12:38:46 crc kubenswrapper[5052]: I1203 12:38:46.169883 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="3ceac470-11eb-4d69-a683-2e9bc4e8f617" containerName="barbican-api" Dec 03 12:38:46 crc kubenswrapper[5052]: I1203 12:38:46.170127 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="3ceac470-11eb-4d69-a683-2e9bc4e8f617" containerName="barbican-api" Dec 03 12:38:46 crc kubenswrapper[5052]: I1203 12:38:46.170146 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="3ceac470-11eb-4d69-a683-2e9bc4e8f617" containerName="barbican-api-log" Dec 03 12:38:46 crc kubenswrapper[5052]: I1203 12:38:46.170166 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="a36c62b8-e624-450a-8b71-a33fc4395eee" containerName="nova-cell1-novncproxy-novncproxy" Dec 03 12:38:46 crc kubenswrapper[5052]: I1203 12:38:46.172299 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-76bf6b6758-lvtjf" Dec 03 12:38:46 crc kubenswrapper[5052]: I1203 12:38:46.216457 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/placement-76bf6b6758-lvtjf"] Dec 03 12:38:46 crc kubenswrapper[5052]: I1203 12:38:46.262684 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/0519e3a0-a23c-45c6-a45b-b913f4983532-public-tls-certs\") pod \"placement-76bf6b6758-lvtjf\" (UID: \"0519e3a0-a23c-45c6-a45b-b913f4983532\") " pod="openstack-kuttl-tests/placement-76bf6b6758-lvtjf" Dec 03 12:38:46 crc kubenswrapper[5052]: I1203 12:38:46.262845 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0519e3a0-a23c-45c6-a45b-b913f4983532-combined-ca-bundle\") pod \"placement-76bf6b6758-lvtjf\" (UID: \"0519e3a0-a23c-45c6-a45b-b913f4983532\") " pod="openstack-kuttl-tests/placement-76bf6b6758-lvtjf" Dec 03 12:38:46 crc kubenswrapper[5052]: I1203 12:38:46.262904 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/0519e3a0-a23c-45c6-a45b-b913f4983532-internal-tls-certs\") pod \"placement-76bf6b6758-lvtjf\" (UID: \"0519e3a0-a23c-45c6-a45b-b913f4983532\") " pod="openstack-kuttl-tests/placement-76bf6b6758-lvtjf" Dec 03 12:38:46 crc kubenswrapper[5052]: I1203 12:38:46.263326 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/0519e3a0-a23c-45c6-a45b-b913f4983532-scripts\") pod \"placement-76bf6b6758-lvtjf\" (UID: \"0519e3a0-a23c-45c6-a45b-b913f4983532\") " pod="openstack-kuttl-tests/placement-76bf6b6758-lvtjf" Dec 03 12:38:46 crc kubenswrapper[5052]: I1203 12:38:46.263458 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lt4sm\" (UniqueName: \"kubernetes.io/projected/0519e3a0-a23c-45c6-a45b-b913f4983532-kube-api-access-lt4sm\") pod \"placement-76bf6b6758-lvtjf\" (UID: \"0519e3a0-a23c-45c6-a45b-b913f4983532\") " pod="openstack-kuttl-tests/placement-76bf6b6758-lvtjf" Dec 03 12:38:46 crc kubenswrapper[5052]: I1203 12:38:46.263649 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/0519e3a0-a23c-45c6-a45b-b913f4983532-logs\") pod \"placement-76bf6b6758-lvtjf\" (UID: \"0519e3a0-a23c-45c6-a45b-b913f4983532\") " pod="openstack-kuttl-tests/placement-76bf6b6758-lvtjf" Dec 03 12:38:46 crc kubenswrapper[5052]: I1203 12:38:46.263868 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/0519e3a0-a23c-45c6-a45b-b913f4983532-config-data\") pod \"placement-76bf6b6758-lvtjf\" (UID: \"0519e3a0-a23c-45c6-a45b-b913f4983532\") " pod="openstack-kuttl-tests/placement-76bf6b6758-lvtjf" Dec 03 12:38:46 crc kubenswrapper[5052]: I1203 12:38:46.365681 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/0519e3a0-a23c-45c6-a45b-b913f4983532-public-tls-certs\") pod \"placement-76bf6b6758-lvtjf\" (UID: \"0519e3a0-a23c-45c6-a45b-b913f4983532\") " pod="openstack-kuttl-tests/placement-76bf6b6758-lvtjf" Dec 03 12:38:46 crc kubenswrapper[5052]: I1203 12:38:46.365801 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0519e3a0-a23c-45c6-a45b-b913f4983532-combined-ca-bundle\") pod \"placement-76bf6b6758-lvtjf\" (UID: \"0519e3a0-a23c-45c6-a45b-b913f4983532\") " pod="openstack-kuttl-tests/placement-76bf6b6758-lvtjf" Dec 03 12:38:46 crc kubenswrapper[5052]: I1203 12:38:46.365830 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/0519e3a0-a23c-45c6-a45b-b913f4983532-internal-tls-certs\") pod \"placement-76bf6b6758-lvtjf\" (UID: \"0519e3a0-a23c-45c6-a45b-b913f4983532\") " pod="openstack-kuttl-tests/placement-76bf6b6758-lvtjf" Dec 03 12:38:46 crc kubenswrapper[5052]: I1203 12:38:46.365884 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/0519e3a0-a23c-45c6-a45b-b913f4983532-scripts\") pod \"placement-76bf6b6758-lvtjf\" (UID: \"0519e3a0-a23c-45c6-a45b-b913f4983532\") " pod="openstack-kuttl-tests/placement-76bf6b6758-lvtjf" Dec 03 12:38:46 crc kubenswrapper[5052]: I1203 12:38:46.365934 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-lt4sm\" (UniqueName: \"kubernetes.io/projected/0519e3a0-a23c-45c6-a45b-b913f4983532-kube-api-access-lt4sm\") pod \"placement-76bf6b6758-lvtjf\" (UID: \"0519e3a0-a23c-45c6-a45b-b913f4983532\") " pod="openstack-kuttl-tests/placement-76bf6b6758-lvtjf" Dec 03 12:38:46 crc kubenswrapper[5052]: I1203 12:38:46.366011 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/0519e3a0-a23c-45c6-a45b-b913f4983532-logs\") pod \"placement-76bf6b6758-lvtjf\" (UID: \"0519e3a0-a23c-45c6-a45b-b913f4983532\") " pod="openstack-kuttl-tests/placement-76bf6b6758-lvtjf" Dec 03 12:38:46 crc kubenswrapper[5052]: I1203 12:38:46.366096 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/0519e3a0-a23c-45c6-a45b-b913f4983532-config-data\") pod \"placement-76bf6b6758-lvtjf\" (UID: \"0519e3a0-a23c-45c6-a45b-b913f4983532\") " pod="openstack-kuttl-tests/placement-76bf6b6758-lvtjf" Dec 03 12:38:46 crc kubenswrapper[5052]: I1203 12:38:46.366578 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/0519e3a0-a23c-45c6-a45b-b913f4983532-logs\") pod \"placement-76bf6b6758-lvtjf\" (UID: \"0519e3a0-a23c-45c6-a45b-b913f4983532\") " pod="openstack-kuttl-tests/placement-76bf6b6758-lvtjf" Dec 03 12:38:46 crc kubenswrapper[5052]: I1203 12:38:46.369610 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/0519e3a0-a23c-45c6-a45b-b913f4983532-scripts\") pod \"placement-76bf6b6758-lvtjf\" (UID: \"0519e3a0-a23c-45c6-a45b-b913f4983532\") " pod="openstack-kuttl-tests/placement-76bf6b6758-lvtjf" Dec 03 12:38:46 crc kubenswrapper[5052]: I1203 12:38:46.370022 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/0519e3a0-a23c-45c6-a45b-b913f4983532-config-data\") pod \"placement-76bf6b6758-lvtjf\" (UID: \"0519e3a0-a23c-45c6-a45b-b913f4983532\") " pod="openstack-kuttl-tests/placement-76bf6b6758-lvtjf" Dec 03 12:38:46 crc kubenswrapper[5052]: I1203 12:38:46.371566 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0519e3a0-a23c-45c6-a45b-b913f4983532-combined-ca-bundle\") pod \"placement-76bf6b6758-lvtjf\" (UID: \"0519e3a0-a23c-45c6-a45b-b913f4983532\") " pod="openstack-kuttl-tests/placement-76bf6b6758-lvtjf" Dec 03 12:38:46 crc kubenswrapper[5052]: I1203 12:38:46.372412 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/0519e3a0-a23c-45c6-a45b-b913f4983532-public-tls-certs\") pod \"placement-76bf6b6758-lvtjf\" (UID: \"0519e3a0-a23c-45c6-a45b-b913f4983532\") " pod="openstack-kuttl-tests/placement-76bf6b6758-lvtjf" Dec 03 12:38:46 crc kubenswrapper[5052]: I1203 12:38:46.372577 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/0519e3a0-a23c-45c6-a45b-b913f4983532-internal-tls-certs\") pod \"placement-76bf6b6758-lvtjf\" (UID: \"0519e3a0-a23c-45c6-a45b-b913f4983532\") " pod="openstack-kuttl-tests/placement-76bf6b6758-lvtjf" Dec 03 12:38:46 crc kubenswrapper[5052]: I1203 12:38:46.391405 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-lt4sm\" (UniqueName: \"kubernetes.io/projected/0519e3a0-a23c-45c6-a45b-b913f4983532-kube-api-access-lt4sm\") pod \"placement-76bf6b6758-lvtjf\" (UID: \"0519e3a0-a23c-45c6-a45b-b913f4983532\") " pod="openstack-kuttl-tests/placement-76bf6b6758-lvtjf" Dec 03 12:38:46 crc kubenswrapper[5052]: I1203 12:38:46.517388 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-76bf6b6758-lvtjf" Dec 03 12:38:46 crc kubenswrapper[5052]: I1203 12:38:46.688498 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-544c66dd6d-75q8t" event={"ID":"c34523d4-d634-4d76-a238-36566ef6a682","Type":"ContainerStarted","Data":"575b5e45ed0a1dfb51c75c86c63be7eabec2113683e893c6ee22d23b04e78576"} Dec 03 12:38:46 crc kubenswrapper[5052]: I1203 12:38:46.688671 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/neutron-544c66dd6d-75q8t" Dec 03 12:38:46 crc kubenswrapper[5052]: I1203 12:38:46.690972 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" event={"ID":"a36c62b8-e624-450a-8b71-a33fc4395eee","Type":"ContainerDied","Data":"91e3d68773399efb138163bf94575ecfabb42bf591d318e7b557f618025a1ff2"} Dec 03 12:38:46 crc kubenswrapper[5052]: I1203 12:38:46.690982 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:38:46 crc kubenswrapper[5052]: I1203 12:38:46.752120 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/ovsdbserver-nb-0" podStartSLOduration=3.75209569 podStartE2EDuration="3.75209569s" podCreationTimestamp="2025-12-03 12:38:43 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:38:46.745918823 +0000 UTC m=+4717.334160587" watchObservedRunningTime="2025-12-03 12:38:46.75209569 +0000 UTC m=+4717.340337454" Dec 03 12:38:46 crc kubenswrapper[5052]: I1203 12:38:46.754203 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/neutron-544c66dd6d-75q8t" podStartSLOduration=2.75418801 podStartE2EDuration="2.75418801s" podCreationTimestamp="2025-12-03 12:38:44 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:38:46.718820879 +0000 UTC m=+4717.307062643" watchObservedRunningTime="2025-12-03 12:38:46.75418801 +0000 UTC m=+4717.342429774" Dec 03 12:38:46 crc kubenswrapper[5052]: I1203 12:38:46.775334 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/ovsdbserver-sb-0" podStartSLOduration=3.775313084 podStartE2EDuration="3.775313084s" podCreationTimestamp="2025-12-03 12:38:43 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:38:46.767212782 +0000 UTC m=+4717.355454546" watchObservedRunningTime="2025-12-03 12:38:46.775313084 +0000 UTC m=+4717.363554848" Dec 03 12:38:46 crc kubenswrapper[5052]: I1203 12:38:46.951749 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-metadata-0" podUID="edea8e53-340d-4f1e-99e5-082268c64ceb" containerName="nova-metadata-metadata" probeResult="failure" output="Get \"https://10.217.1.100:8775/\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Dec 03 12:38:46 crc kubenswrapper[5052]: I1203 12:38:46.952393 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-metadata-0" podUID="edea8e53-340d-4f1e-99e5-082268c64ceb" containerName="nova-metadata-log" probeResult="failure" output="Get \"https://10.217.1.100:8775/\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Dec 03 12:38:46 crc kubenswrapper[5052]: I1203 12:38:46.952446 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 12:38:46 crc kubenswrapper[5052]: I1203 12:38:46.981578 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 12:38:46 crc kubenswrapper[5052]: I1203 12:38:46.998598 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 12:38:47 crc kubenswrapper[5052]: I1203 12:38:47.006558 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:38:47 crc kubenswrapper[5052]: I1203 12:38:47.007777 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 12:38:47 crc kubenswrapper[5052]: I1203 12:38:47.014024 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-nova-novncproxy-cell1-public-svc" Dec 03 12:38:47 crc kubenswrapper[5052]: I1203 12:38:47.014256 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-nova-novncproxy-cell1-vencrypt" Dec 03 12:38:47 crc kubenswrapper[5052]: I1203 12:38:47.014468 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell1-novncproxy-config-data" Dec 03 12:38:47 crc kubenswrapper[5052]: I1203 12:38:47.077623 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:38:47 crc kubenswrapper[5052]: I1203 12:38:47.082506 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p7vnw\" (UniqueName: \"kubernetes.io/projected/20e672a5-6f07-415f-bf4d-51ef7bf20a0d-kube-api-access-p7vnw\") pod \"nova-cell1-novncproxy-0\" (UID: \"20e672a5-6f07-415f-bf4d-51ef7bf20a0d\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:38:47 crc kubenswrapper[5052]: I1203 12:38:47.082632 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/20e672a5-6f07-415f-bf4d-51ef7bf20a0d-combined-ca-bundle\") pod \"nova-cell1-novncproxy-0\" (UID: \"20e672a5-6f07-415f-bf4d-51ef7bf20a0d\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:38:47 crc kubenswrapper[5052]: I1203 12:38:47.082702 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nova-novncproxy-tls-certs\" (UniqueName: \"kubernetes.io/secret/20e672a5-6f07-415f-bf4d-51ef7bf20a0d-nova-novncproxy-tls-certs\") pod \"nova-cell1-novncproxy-0\" (UID: \"20e672a5-6f07-415f-bf4d-51ef7bf20a0d\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:38:47 crc kubenswrapper[5052]: I1203 12:38:47.082861 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/20e672a5-6f07-415f-bf4d-51ef7bf20a0d-config-data\") pod \"nova-cell1-novncproxy-0\" (UID: \"20e672a5-6f07-415f-bf4d-51ef7bf20a0d\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:38:47 crc kubenswrapper[5052]: I1203 12:38:47.082891 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"vencrypt-tls-certs\" (UniqueName: \"kubernetes.io/secret/20e672a5-6f07-415f-bf4d-51ef7bf20a0d-vencrypt-tls-certs\") pod \"nova-cell1-novncproxy-0\" (UID: \"20e672a5-6f07-415f-bf4d-51ef7bf20a0d\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:38:47 crc kubenswrapper[5052]: I1203 12:38:47.185177 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/20e672a5-6f07-415f-bf4d-51ef7bf20a0d-combined-ca-bundle\") pod \"nova-cell1-novncproxy-0\" (UID: \"20e672a5-6f07-415f-bf4d-51ef7bf20a0d\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:38:47 crc kubenswrapper[5052]: I1203 12:38:47.185256 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"nova-novncproxy-tls-certs\" (UniqueName: \"kubernetes.io/secret/20e672a5-6f07-415f-bf4d-51ef7bf20a0d-nova-novncproxy-tls-certs\") pod \"nova-cell1-novncproxy-0\" (UID: \"20e672a5-6f07-415f-bf4d-51ef7bf20a0d\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:38:47 crc kubenswrapper[5052]: I1203 12:38:47.185385 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/20e672a5-6f07-415f-bf4d-51ef7bf20a0d-config-data\") pod \"nova-cell1-novncproxy-0\" (UID: \"20e672a5-6f07-415f-bf4d-51ef7bf20a0d\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:38:47 crc kubenswrapper[5052]: I1203 12:38:47.185887 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"vencrypt-tls-certs\" (UniqueName: \"kubernetes.io/secret/20e672a5-6f07-415f-bf4d-51ef7bf20a0d-vencrypt-tls-certs\") pod \"nova-cell1-novncproxy-0\" (UID: \"20e672a5-6f07-415f-bf4d-51ef7bf20a0d\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:38:47 crc kubenswrapper[5052]: I1203 12:38:47.186093 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-p7vnw\" (UniqueName: \"kubernetes.io/projected/20e672a5-6f07-415f-bf4d-51ef7bf20a0d-kube-api-access-p7vnw\") pod \"nova-cell1-novncproxy-0\" (UID: \"20e672a5-6f07-415f-bf4d-51ef7bf20a0d\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:38:47 crc kubenswrapper[5052]: I1203 12:38:47.192781 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/20e672a5-6f07-415f-bf4d-51ef7bf20a0d-config-data\") pod \"nova-cell1-novncproxy-0\" (UID: \"20e672a5-6f07-415f-bf4d-51ef7bf20a0d\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:38:47 crc kubenswrapper[5052]: I1203 12:38:47.192949 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/20e672a5-6f07-415f-bf4d-51ef7bf20a0d-combined-ca-bundle\") pod \"nova-cell1-novncproxy-0\" (UID: \"20e672a5-6f07-415f-bf4d-51ef7bf20a0d\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:38:47 crc kubenswrapper[5052]: I1203 12:38:47.203190 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"nova-novncproxy-tls-certs\" (UniqueName: \"kubernetes.io/secret/20e672a5-6f07-415f-bf4d-51ef7bf20a0d-nova-novncproxy-tls-certs\") pod \"nova-cell1-novncproxy-0\" (UID: \"20e672a5-6f07-415f-bf4d-51ef7bf20a0d\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:38:47 crc kubenswrapper[5052]: I1203 12:38:47.203404 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-p7vnw\" (UniqueName: \"kubernetes.io/projected/20e672a5-6f07-415f-bf4d-51ef7bf20a0d-kube-api-access-p7vnw\") pod \"nova-cell1-novncproxy-0\" (UID: \"20e672a5-6f07-415f-bf4d-51ef7bf20a0d\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:38:47 crc kubenswrapper[5052]: I1203 12:38:47.207425 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"vencrypt-tls-certs\" (UniqueName: \"kubernetes.io/secret/20e672a5-6f07-415f-bf4d-51ef7bf20a0d-vencrypt-tls-certs\") pod \"nova-cell1-novncproxy-0\" (UID: \"20e672a5-6f07-415f-bf4d-51ef7bf20a0d\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:38:47 crc kubenswrapper[5052]: I1203 12:38:47.315373 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/placement-76bf6b6758-lvtjf"] Dec 03 12:38:47 crc kubenswrapper[5052]: I1203 12:38:47.338212 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:38:47 crc kubenswrapper[5052]: I1203 12:38:47.697378 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a36c62b8-e624-450a-8b71-a33fc4395eee" path="/var/lib/kubelet/pods/a36c62b8-e624-450a-8b71-a33fc4395eee/volumes" Dec 03 12:38:47 crc kubenswrapper[5052]: I1203 12:38:47.736798 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-76bf6b6758-lvtjf" event={"ID":"0519e3a0-a23c-45c6-a45b-b913f4983532","Type":"ContainerStarted","Data":"47656fe9f6350bd895b8133e6f1e7fa7c0d77e1477a3062813d0d11f8a69fee9"} Dec 03 12:38:47 crc kubenswrapper[5052]: I1203 12:38:47.739782 5052 generic.go:334] "Generic (PLEG): container finished" podID="4a200957-c941-47a9-b881-6e9dbc8327b7" containerID="c489ec06ee93875a69927e3ababa6ff83e2ec14d6d1790c0dd006a349910b023" exitCode=0 Dec 03 12:38:47 crc kubenswrapper[5052]: I1203 12:38:47.739882 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"4a200957-c941-47a9-b881-6e9dbc8327b7","Type":"ContainerDied","Data":"c489ec06ee93875a69927e3ababa6ff83e2ec14d6d1790c0dd006a349910b023"} Dec 03 12:38:48 crc kubenswrapper[5052]: I1203 12:38:47.934196 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:38:48 crc kubenswrapper[5052]: I1203 12:38:48.009671 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 12:38:48 crc kubenswrapper[5052]: I1203 12:38:48.490733 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:38:48 crc kubenswrapper[5052]: E1203 12:38:48.733869 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-glance-default-internal-svc: secret "cert-glance-default-internal-svc" not found Dec 03 12:38:48 crc kubenswrapper[5052]: E1203 12:38:48.733959 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/42e9c6bc-62d9-442a-a5f0-d885eff829cb-internal-tls-certs podName:42e9c6bc-62d9-442a-a5f0-d885eff829cb nodeName:}" failed. No retries permitted until 2025-12-03 12:38:49.233939168 +0000 UTC m=+4719.822180932 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "internal-tls-certs" (UniqueName: "kubernetes.io/secret/42e9c6bc-62d9-442a-a5f0-d885eff829cb-internal-tls-certs") pod "glance-default-internal-api-0" (UID: "42e9c6bc-62d9-442a-a5f0-d885eff829cb") : secret "cert-glance-default-internal-svc" not found Dec 03 12:38:48 crc kubenswrapper[5052]: I1203 12:38:48.762457 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-76bf6b6758-lvtjf" event={"ID":"0519e3a0-a23c-45c6-a45b-b913f4983532","Type":"ContainerStarted","Data":"f388a5399432c5ccb31c169b3ad931f092c9dc3131b2e25657fd938dda7893db"} Dec 03 12:38:48 crc kubenswrapper[5052]: I1203 12:38:48.762498 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-76bf6b6758-lvtjf" event={"ID":"0519e3a0-a23c-45c6-a45b-b913f4983532","Type":"ContainerStarted","Data":"9378b56786328ddd0407e98056ad23855b875424e631516f969dc644a1efb7e4"} Dec 03 12:38:48 crc kubenswrapper[5052]: I1203 12:38:48.762807 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/placement-76bf6b6758-lvtjf" Dec 03 12:38:48 crc kubenswrapper[5052]: I1203 12:38:48.763016 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/placement-76bf6b6758-lvtjf" Dec 03 12:38:48 crc kubenswrapper[5052]: I1203 12:38:48.766378 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" event={"ID":"20e672a5-6f07-415f-bf4d-51ef7bf20a0d","Type":"ContainerStarted","Data":"a2e11648a96c675347a628bf0acd6dbbc0ea6280cad79e45c8b977540157aa57"} Dec 03 12:38:48 crc kubenswrapper[5052]: I1203 12:38:48.766424 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" event={"ID":"20e672a5-6f07-415f-bf4d-51ef7bf20a0d","Type":"ContainerStarted","Data":"706d339f4fe6f91d8e28ba1d3aee0ca842a3b0d84d2564ef0e7af37ed0750be3"} Dec 03 12:38:48 crc kubenswrapper[5052]: I1203 12:38:48.787555 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/placement-76bf6b6758-lvtjf" podStartSLOduration=2.78753224 podStartE2EDuration="2.78753224s" podCreationTimestamp="2025-12-03 12:38:46 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:38:48.783634649 +0000 UTC m=+4719.371876413" watchObservedRunningTime="2025-12-03 12:38:48.78753224 +0000 UTC m=+4719.375774004" Dec 03 12:38:48 crc kubenswrapper[5052]: I1203 12:38:48.792520 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:38:48 crc kubenswrapper[5052]: I1203 12:38:48.811836 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" podStartSLOduration=2.811817905 podStartE2EDuration="2.811817905s" podCreationTimestamp="2025-12-03 12:38:46 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:38:48.80185286 +0000 UTC m=+4719.390094624" watchObservedRunningTime="2025-12-03 12:38:48.811817905 +0000 UTC m=+4719.400059669" Dec 03 12:38:48 crc kubenswrapper[5052]: I1203 12:38:48.834503 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/4a200957-c941-47a9-b881-6e9dbc8327b7-public-tls-certs\") pod \"4a200957-c941-47a9-b881-6e9dbc8327b7\" (UID: \"4a200957-c941-47a9-b881-6e9dbc8327b7\") " Dec 03 12:38:48 crc kubenswrapper[5052]: I1203 12:38:48.834596 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/4a200957-c941-47a9-b881-6e9dbc8327b7-config-data\") pod \"4a200957-c941-47a9-b881-6e9dbc8327b7\" (UID: \"4a200957-c941-47a9-b881-6e9dbc8327b7\") " Dec 03 12:38:48 crc kubenswrapper[5052]: I1203 12:38:48.834639 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xsqww\" (UniqueName: \"kubernetes.io/projected/4a200957-c941-47a9-b881-6e9dbc8327b7-kube-api-access-xsqww\") pod \"4a200957-c941-47a9-b881-6e9dbc8327b7\" (UID: \"4a200957-c941-47a9-b881-6e9dbc8327b7\") " Dec 03 12:38:48 crc kubenswrapper[5052]: I1203 12:38:48.834677 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4a200957-c941-47a9-b881-6e9dbc8327b7-combined-ca-bundle\") pod \"4a200957-c941-47a9-b881-6e9dbc8327b7\" (UID: \"4a200957-c941-47a9-b881-6e9dbc8327b7\") " Dec 03 12:38:48 crc kubenswrapper[5052]: I1203 12:38:48.834728 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/4a200957-c941-47a9-b881-6e9dbc8327b7-internal-tls-certs\") pod \"4a200957-c941-47a9-b881-6e9dbc8327b7\" (UID: \"4a200957-c941-47a9-b881-6e9dbc8327b7\") " Dec 03 12:38:48 crc kubenswrapper[5052]: I1203 12:38:48.834781 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/4a200957-c941-47a9-b881-6e9dbc8327b7-logs\") pod \"4a200957-c941-47a9-b881-6e9dbc8327b7\" (UID: \"4a200957-c941-47a9-b881-6e9dbc8327b7\") " Dec 03 12:38:48 crc kubenswrapper[5052]: I1203 12:38:48.838058 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a200957-c941-47a9-b881-6e9dbc8327b7-logs" (OuterVolumeSpecName: "logs") pod "4a200957-c941-47a9-b881-6e9dbc8327b7" (UID: "4a200957-c941-47a9-b881-6e9dbc8327b7"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:38:48 crc kubenswrapper[5052]: I1203 12:38:48.869863 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4a200957-c941-47a9-b881-6e9dbc8327b7-kube-api-access-xsqww" (OuterVolumeSpecName: "kube-api-access-xsqww") pod "4a200957-c941-47a9-b881-6e9dbc8327b7" (UID: "4a200957-c941-47a9-b881-6e9dbc8327b7"). InnerVolumeSpecName "kube-api-access-xsqww". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:38:48 crc kubenswrapper[5052]: I1203 12:38:48.882596 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4a200957-c941-47a9-b881-6e9dbc8327b7-config-data" (OuterVolumeSpecName: "config-data") pod "4a200957-c941-47a9-b881-6e9dbc8327b7" (UID: "4a200957-c941-47a9-b881-6e9dbc8327b7"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:38:48 crc kubenswrapper[5052]: I1203 12:38:48.908653 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4a200957-c941-47a9-b881-6e9dbc8327b7-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "4a200957-c941-47a9-b881-6e9dbc8327b7" (UID: "4a200957-c941-47a9-b881-6e9dbc8327b7"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:38:48 crc kubenswrapper[5052]: I1203 12:38:48.921251 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4a200957-c941-47a9-b881-6e9dbc8327b7-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "4a200957-c941-47a9-b881-6e9dbc8327b7" (UID: "4a200957-c941-47a9-b881-6e9dbc8327b7"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:38:48 crc kubenswrapper[5052]: I1203 12:38:48.942231 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4a200957-c941-47a9-b881-6e9dbc8327b7-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:38:48 crc kubenswrapper[5052]: I1203 12:38:48.942275 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/4a200957-c941-47a9-b881-6e9dbc8327b7-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:38:48 crc kubenswrapper[5052]: I1203 12:38:48.942287 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/4a200957-c941-47a9-b881-6e9dbc8327b7-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:38:48 crc kubenswrapper[5052]: I1203 12:38:48.942297 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/4a200957-c941-47a9-b881-6e9dbc8327b7-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:38:48 crc kubenswrapper[5052]: I1203 12:38:48.942314 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-xsqww\" (UniqueName: \"kubernetes.io/projected/4a200957-c941-47a9-b881-6e9dbc8327b7-kube-api-access-xsqww\") on node \"crc\" DevicePath \"\"" Dec 03 12:38:48 crc kubenswrapper[5052]: I1203 12:38:48.949941 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4a200957-c941-47a9-b881-6e9dbc8327b7-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "4a200957-c941-47a9-b881-6e9dbc8327b7" (UID: "4a200957-c941-47a9-b881-6e9dbc8327b7"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:38:49 crc kubenswrapper[5052]: I1203 12:38:49.044617 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/4a200957-c941-47a9-b881-6e9dbc8327b7-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:38:49 crc kubenswrapper[5052]: I1203 12:38:49.077420 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:38:49 crc kubenswrapper[5052]: E1203 12:38:49.249278 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-glance-default-internal-svc: secret "cert-glance-default-internal-svc" not found Dec 03 12:38:49 crc kubenswrapper[5052]: E1203 12:38:49.249358 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/42e9c6bc-62d9-442a-a5f0-d885eff829cb-internal-tls-certs podName:42e9c6bc-62d9-442a-a5f0-d885eff829cb nodeName:}" failed. No retries permitted until 2025-12-03 12:38:50.249341117 +0000 UTC m=+4720.837582871 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "internal-tls-certs" (UniqueName: "kubernetes.io/secret/42e9c6bc-62d9-442a-a5f0-d885eff829cb-internal-tls-certs") pod "glance-default-internal-api-0" (UID: "42e9c6bc-62d9-442a-a5f0-d885eff829cb") : secret "cert-glance-default-internal-svc" not found Dec 03 12:38:49 crc kubenswrapper[5052]: I1203 12:38:49.490725 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:38:49 crc kubenswrapper[5052]: I1203 12:38:49.549553 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:38:49 crc kubenswrapper[5052]: E1203 12:38:49.602816 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="4e823788ee83fdbbba2177f783a9bb4380ce320612fdfe974cb17f48a899577b" cmd=["/usr/local/bin/container-scripts/status_check.sh"] Dec 03 12:38:49 crc kubenswrapper[5052]: E1203 12:38:49.606010 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="4e823788ee83fdbbba2177f783a9bb4380ce320612fdfe974cb17f48a899577b" cmd=["/usr/local/bin/container-scripts/status_check.sh"] Dec 03 12:38:49 crc kubenswrapper[5052]: E1203 12:38:49.607559 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="4e823788ee83fdbbba2177f783a9bb4380ce320612fdfe974cb17f48a899577b" cmd=["/usr/local/bin/container-scripts/status_check.sh"] Dec 03 12:38:49 crc kubenswrapper[5052]: E1203 12:38:49.607606 5052 prober.go:104] "Probe errored" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" probeType="Readiness" pod="openstack-kuttl-tests/ovn-northd-0" podUID="92dadf05-a318-4491-bb1c-1fa23cb293fa" containerName="ovn-northd" Dec 03 12:38:49 crc kubenswrapper[5052]: I1203 12:38:49.778416 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"4a200957-c941-47a9-b881-6e9dbc8327b7","Type":"ContainerDied","Data":"21ed13a14f539cb2dc7294be8549c5b0103c4956a74455657743bc81528f5687"} Dec 03 12:38:49 crc kubenswrapper[5052]: I1203 12:38:49.778780 5052 scope.go:117] "RemoveContainer" containerID="c489ec06ee93875a69927e3ababa6ff83e2ec14d6d1790c0dd006a349910b023" Dec 03 12:38:49 crc kubenswrapper[5052]: I1203 12:38:49.778526 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:38:49 crc kubenswrapper[5052]: I1203 12:38:49.826138 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:38:49 crc kubenswrapper[5052]: I1203 12:38:49.841254 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:38:49 crc kubenswrapper[5052]: I1203 12:38:49.841613 5052 scope.go:117] "RemoveContainer" containerID="767ce5808c23166231a144b01e05e10cc1fe868247442d07a0b8b9a42c438b78" Dec 03 12:38:49 crc kubenswrapper[5052]: I1203 12:38:49.882235 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:38:49 crc kubenswrapper[5052]: E1203 12:38:49.883199 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="4a200957-c941-47a9-b881-6e9dbc8327b7" containerName="nova-api-api" Dec 03 12:38:49 crc kubenswrapper[5052]: I1203 12:38:49.883345 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="4a200957-c941-47a9-b881-6e9dbc8327b7" containerName="nova-api-api" Dec 03 12:38:49 crc kubenswrapper[5052]: E1203 12:38:49.883521 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="4a200957-c941-47a9-b881-6e9dbc8327b7" containerName="nova-api-log" Dec 03 12:38:49 crc kubenswrapper[5052]: I1203 12:38:49.883595 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="4a200957-c941-47a9-b881-6e9dbc8327b7" containerName="nova-api-log" Dec 03 12:38:49 crc kubenswrapper[5052]: I1203 12:38:49.883923 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="4a200957-c941-47a9-b881-6e9dbc8327b7" containerName="nova-api-log" Dec 03 12:38:49 crc kubenswrapper[5052]: I1203 12:38:49.884030 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="4a200957-c941-47a9-b881-6e9dbc8327b7" containerName="nova-api-api" Dec 03 12:38:49 crc kubenswrapper[5052]: I1203 12:38:49.885892 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:38:49 crc kubenswrapper[5052]: I1203 12:38:49.888450 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-nova-internal-svc" Dec 03 12:38:49 crc kubenswrapper[5052]: I1203 12:38:49.888751 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-nova-public-svc" Dec 03 12:38:49 crc kubenswrapper[5052]: I1203 12:38:49.889267 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-api-config-data" Dec 03 12:38:49 crc kubenswrapper[5052]: I1203 12:38:49.891926 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:38:49 crc kubenswrapper[5052]: I1203 12:38:49.963472 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/c74fd61a-bf75-4a68-b802-cdc7f4b48be9-internal-tls-certs\") pod \"nova-api-0\" (UID: \"c74fd61a-bf75-4a68-b802-cdc7f4b48be9\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:38:49 crc kubenswrapper[5052]: I1203 12:38:49.963676 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c74fd61a-bf75-4a68-b802-cdc7f4b48be9-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"c74fd61a-bf75-4a68-b802-cdc7f4b48be9\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:38:49 crc kubenswrapper[5052]: I1203 12:38:49.963750 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/c74fd61a-bf75-4a68-b802-cdc7f4b48be9-config-data\") pod \"nova-api-0\" (UID: \"c74fd61a-bf75-4a68-b802-cdc7f4b48be9\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:38:49 crc kubenswrapper[5052]: I1203 12:38:49.963820 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v9t9t\" (UniqueName: \"kubernetes.io/projected/c74fd61a-bf75-4a68-b802-cdc7f4b48be9-kube-api-access-v9t9t\") pod \"nova-api-0\" (UID: \"c74fd61a-bf75-4a68-b802-cdc7f4b48be9\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:38:49 crc kubenswrapper[5052]: I1203 12:38:49.963881 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/c74fd61a-bf75-4a68-b802-cdc7f4b48be9-logs\") pod \"nova-api-0\" (UID: \"c74fd61a-bf75-4a68-b802-cdc7f4b48be9\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:38:49 crc kubenswrapper[5052]: I1203 12:38:49.963954 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/c74fd61a-bf75-4a68-b802-cdc7f4b48be9-public-tls-certs\") pod \"nova-api-0\" (UID: \"c74fd61a-bf75-4a68-b802-cdc7f4b48be9\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:38:50 crc kubenswrapper[5052]: I1203 12:38:50.038165 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:38:50 crc kubenswrapper[5052]: I1203 12:38:50.038244 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:38:50 crc kubenswrapper[5052]: I1203 12:38:50.061241 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:38:50 crc kubenswrapper[5052]: I1203 12:38:50.065895 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c74fd61a-bf75-4a68-b802-cdc7f4b48be9-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"c74fd61a-bf75-4a68-b802-cdc7f4b48be9\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:38:50 crc kubenswrapper[5052]: I1203 12:38:50.065971 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/c74fd61a-bf75-4a68-b802-cdc7f4b48be9-config-data\") pod \"nova-api-0\" (UID: \"c74fd61a-bf75-4a68-b802-cdc7f4b48be9\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:38:50 crc kubenswrapper[5052]: I1203 12:38:50.066013 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-v9t9t\" (UniqueName: \"kubernetes.io/projected/c74fd61a-bf75-4a68-b802-cdc7f4b48be9-kube-api-access-v9t9t\") pod \"nova-api-0\" (UID: \"c74fd61a-bf75-4a68-b802-cdc7f4b48be9\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:38:50 crc kubenswrapper[5052]: I1203 12:38:50.066052 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/c74fd61a-bf75-4a68-b802-cdc7f4b48be9-logs\") pod \"nova-api-0\" (UID: \"c74fd61a-bf75-4a68-b802-cdc7f4b48be9\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:38:50 crc kubenswrapper[5052]: I1203 12:38:50.066086 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/c74fd61a-bf75-4a68-b802-cdc7f4b48be9-public-tls-certs\") pod \"nova-api-0\" (UID: \"c74fd61a-bf75-4a68-b802-cdc7f4b48be9\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:38:50 crc kubenswrapper[5052]: I1203 12:38:50.066137 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/c74fd61a-bf75-4a68-b802-cdc7f4b48be9-internal-tls-certs\") pod \"nova-api-0\" (UID: \"c74fd61a-bf75-4a68-b802-cdc7f4b48be9\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:38:50 crc kubenswrapper[5052]: I1203 12:38:50.066855 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/c74fd61a-bf75-4a68-b802-cdc7f4b48be9-logs\") pod \"nova-api-0\" (UID: \"c74fd61a-bf75-4a68-b802-cdc7f4b48be9\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:38:50 crc kubenswrapper[5052]: I1203 12:38:50.072119 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/c74fd61a-bf75-4a68-b802-cdc7f4b48be9-config-data\") pod \"nova-api-0\" (UID: \"c74fd61a-bf75-4a68-b802-cdc7f4b48be9\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:38:50 crc kubenswrapper[5052]: I1203 12:38:50.087150 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/c74fd61a-bf75-4a68-b802-cdc7f4b48be9-public-tls-certs\") pod \"nova-api-0\" (UID: \"c74fd61a-bf75-4a68-b802-cdc7f4b48be9\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:38:50 crc kubenswrapper[5052]: I1203 12:38:50.087595 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c74fd61a-bf75-4a68-b802-cdc7f4b48be9-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"c74fd61a-bf75-4a68-b802-cdc7f4b48be9\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:38:50 crc kubenswrapper[5052]: I1203 12:38:50.094237 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-v9t9t\" (UniqueName: \"kubernetes.io/projected/c74fd61a-bf75-4a68-b802-cdc7f4b48be9-kube-api-access-v9t9t\") pod \"nova-api-0\" (UID: \"c74fd61a-bf75-4a68-b802-cdc7f4b48be9\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:38:50 crc kubenswrapper[5052]: I1203 12:38:50.099034 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:38:50 crc kubenswrapper[5052]: I1203 12:38:50.099760 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/c74fd61a-bf75-4a68-b802-cdc7f4b48be9-internal-tls-certs\") pod \"nova-api-0\" (UID: \"c74fd61a-bf75-4a68-b802-cdc7f4b48be9\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:38:50 crc kubenswrapper[5052]: I1203 12:38:50.136039 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:38:50 crc kubenswrapper[5052]: I1203 12:38:50.210543 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:38:50 crc kubenswrapper[5052]: E1203 12:38:50.271898 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-glance-default-internal-svc: secret "cert-glance-default-internal-svc" not found Dec 03 12:38:50 crc kubenswrapper[5052]: E1203 12:38:50.272286 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/42e9c6bc-62d9-442a-a5f0-d885eff829cb-internal-tls-certs podName:42e9c6bc-62d9-442a-a5f0-d885eff829cb nodeName:}" failed. No retries permitted until 2025-12-03 12:38:52.272271272 +0000 UTC m=+4722.860513036 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "internal-tls-certs" (UniqueName: "kubernetes.io/secret/42e9c6bc-62d9-442a-a5f0-d885eff829cb-internal-tls-certs") pod "glance-default-internal-api-0" (UID: "42e9c6bc-62d9-442a-a5f0-d885eff829cb") : secret "cert-glance-default-internal-svc" not found Dec 03 12:38:50 crc kubenswrapper[5052]: I1203 12:38:50.731663 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:38:50 crc kubenswrapper[5052]: I1203 12:38:50.876160 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:38:50 crc kubenswrapper[5052]: I1203 12:38:50.942681 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:38:50 crc kubenswrapper[5052]: I1203 12:38:50.945852 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:38:51 crc kubenswrapper[5052]: I1203 12:38:51.241350 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:38:51 crc kubenswrapper[5052]: E1203 12:38:51.498665 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-glance-default-public-svc: secret "cert-glance-default-public-svc" not found Dec 03 12:38:51 crc kubenswrapper[5052]: E1203 12:38:51.498746 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9dccead0-7ff2-4255-ac00-10a9b357c2db-public-tls-certs podName:9dccead0-7ff2-4255-ac00-10a9b357c2db nodeName:}" failed. No retries permitted until 2025-12-03 12:39:07.498729625 +0000 UTC m=+4738.086971399 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "public-tls-certs" (UniqueName: "kubernetes.io/secret/9dccead0-7ff2-4255-ac00-10a9b357c2db-public-tls-certs") pod "glance-default-external-api-0" (UID: "9dccead0-7ff2-4255-ac00-10a9b357c2db") : secret "cert-glance-default-public-svc" not found Dec 03 12:38:51 crc kubenswrapper[5052]: I1203 12:38:51.686819 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4a200957-c941-47a9-b881-6e9dbc8327b7" path="/var/lib/kubelet/pods/4a200957-c941-47a9-b881-6e9dbc8327b7/volumes" Dec 03 12:38:51 crc kubenswrapper[5052]: I1203 12:38:51.800867 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"c74fd61a-bf75-4a68-b802-cdc7f4b48be9","Type":"ContainerStarted","Data":"f74dbe94a7303ed5a5e53c969e42792ff7d1835379c81468a7fb60a94a0d4124"} Dec 03 12:38:51 crc kubenswrapper[5052]: I1203 12:38:51.800909 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"c74fd61a-bf75-4a68-b802-cdc7f4b48be9","Type":"ContainerStarted","Data":"4f43d5ebe24ab8b588b14aa860f7d67bd1fcbcefa5469616bcdd5d550ee70012"} Dec 03 12:38:51 crc kubenswrapper[5052]: I1203 12:38:51.805443 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_ovn-northd-0_92dadf05-a318-4491-bb1c-1fa23cb293fa/ovn-northd/0.log" Dec 03 12:38:51 crc kubenswrapper[5052]: I1203 12:38:51.805809 5052 generic.go:334] "Generic (PLEG): container finished" podID="92dadf05-a318-4491-bb1c-1fa23cb293fa" containerID="4e823788ee83fdbbba2177f783a9bb4380ce320612fdfe974cb17f48a899577b" exitCode=139 Dec 03 12:38:51 crc kubenswrapper[5052]: I1203 12:38:51.806775 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovn-northd-0" event={"ID":"92dadf05-a318-4491-bb1c-1fa23cb293fa","Type":"ContainerDied","Data":"4e823788ee83fdbbba2177f783a9bb4380ce320612fdfe974cb17f48a899577b"} Dec 03 12:38:52 crc kubenswrapper[5052]: I1203 12:38:52.116012 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:38:52 crc kubenswrapper[5052]: E1203 12:38:52.317438 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-glance-default-internal-svc: secret "cert-glance-default-internal-svc" not found Dec 03 12:38:52 crc kubenswrapper[5052]: E1203 12:38:52.317863 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/42e9c6bc-62d9-442a-a5f0-d885eff829cb-internal-tls-certs podName:42e9c6bc-62d9-442a-a5f0-d885eff829cb nodeName:}" failed. No retries permitted until 2025-12-03 12:38:56.317837711 +0000 UTC m=+4726.906079475 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "internal-tls-certs" (UniqueName: "kubernetes.io/secret/42e9c6bc-62d9-442a-a5f0-d885eff829cb-internal-tls-certs") pod "glance-default-internal-api-0" (UID: "42e9c6bc-62d9-442a-a5f0-d885eff829cb") : secret "cert-glance-default-internal-svc" not found Dec 03 12:38:52 crc kubenswrapper[5052]: I1203 12:38:52.338867 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:38:52 crc kubenswrapper[5052]: I1203 12:38:52.550222 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_ovn-northd-0_92dadf05-a318-4491-bb1c-1fa23cb293fa/ovn-northd/0.log" Dec 03 12:38:52 crc kubenswrapper[5052]: I1203 12:38:52.550284 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:38:52 crc kubenswrapper[5052]: I1203 12:38:52.621646 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovn-northd-tls-certs\" (UniqueName: \"kubernetes.io/secret/92dadf05-a318-4491-bb1c-1fa23cb293fa-ovn-northd-tls-certs\") pod \"92dadf05-a318-4491-bb1c-1fa23cb293fa\" (UID: \"92dadf05-a318-4491-bb1c-1fa23cb293fa\") " Dec 03 12:38:52 crc kubenswrapper[5052]: I1203 12:38:52.621697 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/92dadf05-a318-4491-bb1c-1fa23cb293fa-combined-ca-bundle\") pod \"92dadf05-a318-4491-bb1c-1fa23cb293fa\" (UID: \"92dadf05-a318-4491-bb1c-1fa23cb293fa\") " Dec 03 12:38:52 crc kubenswrapper[5052]: I1203 12:38:52.621799 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/92dadf05-a318-4491-bb1c-1fa23cb293fa-metrics-certs-tls-certs\") pod \"92dadf05-a318-4491-bb1c-1fa23cb293fa\" (UID: \"92dadf05-a318-4491-bb1c-1fa23cb293fa\") " Dec 03 12:38:52 crc kubenswrapper[5052]: I1203 12:38:52.621872 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/92dadf05-a318-4491-bb1c-1fa23cb293fa-config\") pod \"92dadf05-a318-4491-bb1c-1fa23cb293fa\" (UID: \"92dadf05-a318-4491-bb1c-1fa23cb293fa\") " Dec 03 12:38:52 crc kubenswrapper[5052]: I1203 12:38:52.621935 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xrncn\" (UniqueName: \"kubernetes.io/projected/92dadf05-a318-4491-bb1c-1fa23cb293fa-kube-api-access-xrncn\") pod \"92dadf05-a318-4491-bb1c-1fa23cb293fa\" (UID: \"92dadf05-a318-4491-bb1c-1fa23cb293fa\") " Dec 03 12:38:52 crc kubenswrapper[5052]: I1203 12:38:52.621970 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/92dadf05-a318-4491-bb1c-1fa23cb293fa-scripts\") pod \"92dadf05-a318-4491-bb1c-1fa23cb293fa\" (UID: \"92dadf05-a318-4491-bb1c-1fa23cb293fa\") " Dec 03 12:38:52 crc kubenswrapper[5052]: I1203 12:38:52.622056 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovn-rundir\" (UniqueName: \"kubernetes.io/empty-dir/92dadf05-a318-4491-bb1c-1fa23cb293fa-ovn-rundir\") pod \"92dadf05-a318-4491-bb1c-1fa23cb293fa\" (UID: \"92dadf05-a318-4491-bb1c-1fa23cb293fa\") " Dec 03 12:38:52 crc kubenswrapper[5052]: I1203 12:38:52.623764 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/92dadf05-a318-4491-bb1c-1fa23cb293fa-ovn-rundir" (OuterVolumeSpecName: "ovn-rundir") pod "92dadf05-a318-4491-bb1c-1fa23cb293fa" (UID: "92dadf05-a318-4491-bb1c-1fa23cb293fa"). InnerVolumeSpecName "ovn-rundir". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:38:52 crc kubenswrapper[5052]: I1203 12:38:52.624258 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/92dadf05-a318-4491-bb1c-1fa23cb293fa-config" (OuterVolumeSpecName: "config") pod "92dadf05-a318-4491-bb1c-1fa23cb293fa" (UID: "92dadf05-a318-4491-bb1c-1fa23cb293fa"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:38:52 crc kubenswrapper[5052]: I1203 12:38:52.624703 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/92dadf05-a318-4491-bb1c-1fa23cb293fa-scripts" (OuterVolumeSpecName: "scripts") pod "92dadf05-a318-4491-bb1c-1fa23cb293fa" (UID: "92dadf05-a318-4491-bb1c-1fa23cb293fa"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:38:52 crc kubenswrapper[5052]: I1203 12:38:52.632344 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/92dadf05-a318-4491-bb1c-1fa23cb293fa-kube-api-access-xrncn" (OuterVolumeSpecName: "kube-api-access-xrncn") pod "92dadf05-a318-4491-bb1c-1fa23cb293fa" (UID: "92dadf05-a318-4491-bb1c-1fa23cb293fa"). InnerVolumeSpecName "kube-api-access-xrncn". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:38:52 crc kubenswrapper[5052]: I1203 12:38:52.680796 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/92dadf05-a318-4491-bb1c-1fa23cb293fa-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "92dadf05-a318-4491-bb1c-1fa23cb293fa" (UID: "92dadf05-a318-4491-bb1c-1fa23cb293fa"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:38:52 crc kubenswrapper[5052]: I1203 12:38:52.715243 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/92dadf05-a318-4491-bb1c-1fa23cb293fa-metrics-certs-tls-certs" (OuterVolumeSpecName: "metrics-certs-tls-certs") pod "92dadf05-a318-4491-bb1c-1fa23cb293fa" (UID: "92dadf05-a318-4491-bb1c-1fa23cb293fa"). InnerVolumeSpecName "metrics-certs-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:38:52 crc kubenswrapper[5052]: I1203 12:38:52.724402 5052 reconciler_common.go:293] "Volume detached for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/92dadf05-a318-4491-bb1c-1fa23cb293fa-metrics-certs-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:38:52 crc kubenswrapper[5052]: I1203 12:38:52.724434 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/92dadf05-a318-4491-bb1c-1fa23cb293fa-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:38:52 crc kubenswrapper[5052]: I1203 12:38:52.724443 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-xrncn\" (UniqueName: \"kubernetes.io/projected/92dadf05-a318-4491-bb1c-1fa23cb293fa-kube-api-access-xrncn\") on node \"crc\" DevicePath \"\"" Dec 03 12:38:52 crc kubenswrapper[5052]: I1203 12:38:52.724452 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/92dadf05-a318-4491-bb1c-1fa23cb293fa-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:38:52 crc kubenswrapper[5052]: I1203 12:38:52.724462 5052 reconciler_common.go:293] "Volume detached for volume \"ovn-rundir\" (UniqueName: \"kubernetes.io/empty-dir/92dadf05-a318-4491-bb1c-1fa23cb293fa-ovn-rundir\") on node \"crc\" DevicePath \"\"" Dec 03 12:38:52 crc kubenswrapper[5052]: I1203 12:38:52.724473 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/92dadf05-a318-4491-bb1c-1fa23cb293fa-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:38:52 crc kubenswrapper[5052]: I1203 12:38:52.728565 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/92dadf05-a318-4491-bb1c-1fa23cb293fa-ovn-northd-tls-certs" (OuterVolumeSpecName: "ovn-northd-tls-certs") pod "92dadf05-a318-4491-bb1c-1fa23cb293fa" (UID: "92dadf05-a318-4491-bb1c-1fa23cb293fa"). InnerVolumeSpecName "ovn-northd-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:38:52 crc kubenswrapper[5052]: I1203 12:38:52.816890 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_ovn-northd-0_92dadf05-a318-4491-bb1c-1fa23cb293fa/ovn-northd/0.log" Dec 03 12:38:52 crc kubenswrapper[5052]: I1203 12:38:52.816951 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovn-northd-0" event={"ID":"92dadf05-a318-4491-bb1c-1fa23cb293fa","Type":"ContainerDied","Data":"e9b77740c5b7971ec0cda24631811a45f606f26233dd0f84362e91bcc82dc8c1"} Dec 03 12:38:52 crc kubenswrapper[5052]: I1203 12:38:52.816987 5052 scope.go:117] "RemoveContainer" containerID="94a951ac5d9669865bf707580490a8df30f61838bbfebbf2fccdc24d07f308a3" Dec 03 12:38:52 crc kubenswrapper[5052]: I1203 12:38:52.817103 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:38:52 crc kubenswrapper[5052]: I1203 12:38:52.824460 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"c74fd61a-bf75-4a68-b802-cdc7f4b48be9","Type":"ContainerStarted","Data":"4233bebd5d63d6736a338fd2b32760cdc34fecfde4113939f42d9e6e5b01fa67"} Dec 03 12:38:52 crc kubenswrapper[5052]: I1203 12:38:52.827328 5052 reconciler_common.go:293] "Volume detached for volume \"ovn-northd-tls-certs\" (UniqueName: \"kubernetes.io/secret/92dadf05-a318-4491-bb1c-1fa23cb293fa-ovn-northd-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:38:52 crc kubenswrapper[5052]: I1203 12:38:52.847331 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-api-0" podStartSLOduration=3.847315643 podStartE2EDuration="3.847315643s" podCreationTimestamp="2025-12-03 12:38:49 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:38:52.842933418 +0000 UTC m=+4723.431175182" watchObservedRunningTime="2025-12-03 12:38:52.847315643 +0000 UTC m=+4723.435557407" Dec 03 12:38:52 crc kubenswrapper[5052]: I1203 12:38:52.864358 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ovn-northd-0"] Dec 03 12:38:52 crc kubenswrapper[5052]: I1203 12:38:52.872645 5052 scope.go:117] "RemoveContainer" containerID="4e823788ee83fdbbba2177f783a9bb4380ce320612fdfe974cb17f48a899577b" Dec 03 12:38:52 crc kubenswrapper[5052]: I1203 12:38:52.875390 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ovn-northd-0"] Dec 03 12:38:52 crc kubenswrapper[5052]: I1203 12:38:52.894380 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ovn-northd-0"] Dec 03 12:38:52 crc kubenswrapper[5052]: E1203 12:38:52.894890 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="92dadf05-a318-4491-bb1c-1fa23cb293fa" containerName="openstack-network-exporter" Dec 03 12:38:52 crc kubenswrapper[5052]: I1203 12:38:52.894915 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="92dadf05-a318-4491-bb1c-1fa23cb293fa" containerName="openstack-network-exporter" Dec 03 12:38:52 crc kubenswrapper[5052]: E1203 12:38:52.894962 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="92dadf05-a318-4491-bb1c-1fa23cb293fa" containerName="ovn-northd" Dec 03 12:38:52 crc kubenswrapper[5052]: I1203 12:38:52.894971 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="92dadf05-a318-4491-bb1c-1fa23cb293fa" containerName="ovn-northd" Dec 03 12:38:52 crc kubenswrapper[5052]: I1203 12:38:52.895190 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="92dadf05-a318-4491-bb1c-1fa23cb293fa" containerName="ovn-northd" Dec 03 12:38:52 crc kubenswrapper[5052]: I1203 12:38:52.895223 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="92dadf05-a318-4491-bb1c-1fa23cb293fa" containerName="openstack-network-exporter" Dec 03 12:38:52 crc kubenswrapper[5052]: I1203 12:38:52.896271 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:38:52 crc kubenswrapper[5052]: I1203 12:38:52.902254 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ovnnorthd-ovnnorthd-dockercfg-56wrf" Dec 03 12:38:52 crc kubenswrapper[5052]: I1203 12:38:52.907325 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"ovnnorthd-scripts" Dec 03 12:38:52 crc kubenswrapper[5052]: I1203 12:38:52.907607 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"ovnnorthd-config" Dec 03 12:38:52 crc kubenswrapper[5052]: I1203 12:38:52.908261 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-ovnnorthd-ovndbs" Dec 03 12:38:52 crc kubenswrapper[5052]: I1203 12:38:52.916091 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ovn-northd-0"] Dec 03 12:38:52 crc kubenswrapper[5052]: I1203 12:38:52.934827 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5pfwg\" (UniqueName: \"kubernetes.io/projected/ea4cd237-7ec6-49f7-9afd-1ea159b9b23b-kube-api-access-5pfwg\") pod \"ovn-northd-0\" (UID: \"ea4cd237-7ec6-49f7-9afd-1ea159b9b23b\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:38:52 crc kubenswrapper[5052]: I1203 12:38:52.934914 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/ea4cd237-7ec6-49f7-9afd-1ea159b9b23b-scripts\") pod \"ovn-northd-0\" (UID: \"ea4cd237-7ec6-49f7-9afd-1ea159b9b23b\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:38:52 crc kubenswrapper[5052]: I1203 12:38:52.934953 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ea4cd237-7ec6-49f7-9afd-1ea159b9b23b-combined-ca-bundle\") pod \"ovn-northd-0\" (UID: \"ea4cd237-7ec6-49f7-9afd-1ea159b9b23b\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:38:52 crc kubenswrapper[5052]: I1203 12:38:52.934978 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-northd-tls-certs\" (UniqueName: \"kubernetes.io/secret/ea4cd237-7ec6-49f7-9afd-1ea159b9b23b-ovn-northd-tls-certs\") pod \"ovn-northd-0\" (UID: \"ea4cd237-7ec6-49f7-9afd-1ea159b9b23b\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:38:52 crc kubenswrapper[5052]: I1203 12:38:52.935062 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/ea4cd237-7ec6-49f7-9afd-1ea159b9b23b-metrics-certs-tls-certs\") pod \"ovn-northd-0\" (UID: \"ea4cd237-7ec6-49f7-9afd-1ea159b9b23b\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:38:52 crc kubenswrapper[5052]: I1203 12:38:52.935139 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ea4cd237-7ec6-49f7-9afd-1ea159b9b23b-config\") pod \"ovn-northd-0\" (UID: \"ea4cd237-7ec6-49f7-9afd-1ea159b9b23b\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:38:52 crc kubenswrapper[5052]: I1203 12:38:52.935281 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-rundir\" (UniqueName: \"kubernetes.io/empty-dir/ea4cd237-7ec6-49f7-9afd-1ea159b9b23b-ovn-rundir\") pod \"ovn-northd-0\" (UID: \"ea4cd237-7ec6-49f7-9afd-1ea159b9b23b\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:38:53 crc kubenswrapper[5052]: I1203 12:38:53.045835 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-5pfwg\" (UniqueName: \"kubernetes.io/projected/ea4cd237-7ec6-49f7-9afd-1ea159b9b23b-kube-api-access-5pfwg\") pod \"ovn-northd-0\" (UID: \"ea4cd237-7ec6-49f7-9afd-1ea159b9b23b\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:38:53 crc kubenswrapper[5052]: I1203 12:38:53.045955 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/ea4cd237-7ec6-49f7-9afd-1ea159b9b23b-scripts\") pod \"ovn-northd-0\" (UID: \"ea4cd237-7ec6-49f7-9afd-1ea159b9b23b\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:38:53 crc kubenswrapper[5052]: I1203 12:38:53.045980 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ea4cd237-7ec6-49f7-9afd-1ea159b9b23b-combined-ca-bundle\") pod \"ovn-northd-0\" (UID: \"ea4cd237-7ec6-49f7-9afd-1ea159b9b23b\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:38:53 crc kubenswrapper[5052]: I1203 12:38:53.046013 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovn-northd-tls-certs\" (UniqueName: \"kubernetes.io/secret/ea4cd237-7ec6-49f7-9afd-1ea159b9b23b-ovn-northd-tls-certs\") pod \"ovn-northd-0\" (UID: \"ea4cd237-7ec6-49f7-9afd-1ea159b9b23b\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:38:53 crc kubenswrapper[5052]: I1203 12:38:53.046141 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/ea4cd237-7ec6-49f7-9afd-1ea159b9b23b-metrics-certs-tls-certs\") pod \"ovn-northd-0\" (UID: \"ea4cd237-7ec6-49f7-9afd-1ea159b9b23b\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:38:53 crc kubenswrapper[5052]: I1203 12:38:53.046249 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ea4cd237-7ec6-49f7-9afd-1ea159b9b23b-config\") pod \"ovn-northd-0\" (UID: \"ea4cd237-7ec6-49f7-9afd-1ea159b9b23b\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:38:53 crc kubenswrapper[5052]: I1203 12:38:53.046483 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovn-rundir\" (UniqueName: \"kubernetes.io/empty-dir/ea4cd237-7ec6-49f7-9afd-1ea159b9b23b-ovn-rundir\") pod \"ovn-northd-0\" (UID: \"ea4cd237-7ec6-49f7-9afd-1ea159b9b23b\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:38:53 crc kubenswrapper[5052]: I1203 12:38:53.047043 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovn-rundir\" (UniqueName: \"kubernetes.io/empty-dir/ea4cd237-7ec6-49f7-9afd-1ea159b9b23b-ovn-rundir\") pod \"ovn-northd-0\" (UID: \"ea4cd237-7ec6-49f7-9afd-1ea159b9b23b\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:38:53 crc kubenswrapper[5052]: I1203 12:38:53.048186 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/ea4cd237-7ec6-49f7-9afd-1ea159b9b23b-scripts\") pod \"ovn-northd-0\" (UID: \"ea4cd237-7ec6-49f7-9afd-1ea159b9b23b\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:38:53 crc kubenswrapper[5052]: I1203 12:38:53.048497 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ea4cd237-7ec6-49f7-9afd-1ea159b9b23b-config\") pod \"ovn-northd-0\" (UID: \"ea4cd237-7ec6-49f7-9afd-1ea159b9b23b\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:38:53 crc kubenswrapper[5052]: I1203 12:38:53.053341 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovn-northd-tls-certs\" (UniqueName: \"kubernetes.io/secret/ea4cd237-7ec6-49f7-9afd-1ea159b9b23b-ovn-northd-tls-certs\") pod \"ovn-northd-0\" (UID: \"ea4cd237-7ec6-49f7-9afd-1ea159b9b23b\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:38:53 crc kubenswrapper[5052]: I1203 12:38:53.054196 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/ea4cd237-7ec6-49f7-9afd-1ea159b9b23b-metrics-certs-tls-certs\") pod \"ovn-northd-0\" (UID: \"ea4cd237-7ec6-49f7-9afd-1ea159b9b23b\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:38:53 crc kubenswrapper[5052]: I1203 12:38:53.059498 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ea4cd237-7ec6-49f7-9afd-1ea159b9b23b-combined-ca-bundle\") pod \"ovn-northd-0\" (UID: \"ea4cd237-7ec6-49f7-9afd-1ea159b9b23b\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:38:53 crc kubenswrapper[5052]: I1203 12:38:53.065050 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-5pfwg\" (UniqueName: \"kubernetes.io/projected/ea4cd237-7ec6-49f7-9afd-1ea159b9b23b-kube-api-access-5pfwg\") pod \"ovn-northd-0\" (UID: \"ea4cd237-7ec6-49f7-9afd-1ea159b9b23b\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:38:53 crc kubenswrapper[5052]: I1203 12:38:53.227905 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:38:53 crc kubenswrapper[5052]: I1203 12:38:53.535371 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:38:53 crc kubenswrapper[5052]: I1203 12:38:53.684924 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="92dadf05-a318-4491-bb1c-1fa23cb293fa" path="/var/lib/kubelet/pods/92dadf05-a318-4491-bb1c-1fa23cb293fa/volumes" Dec 03 12:38:53 crc kubenswrapper[5052]: I1203 12:38:53.699891 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ovn-northd-0"] Dec 03 12:38:53 crc kubenswrapper[5052]: I1203 12:38:53.842540 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovn-northd-0" event={"ID":"ea4cd237-7ec6-49f7-9afd-1ea159b9b23b","Type":"ContainerStarted","Data":"bb3afddf44a1f77efa151215ac24c463f8c74211df666d92c6489ae6bd78979b"} Dec 03 12:38:53 crc kubenswrapper[5052]: I1203 12:38:53.847769 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_neutron-544c66dd6d-75q8t_c34523d4-d634-4d76-a238-36566ef6a682/neutron-api/0.log" Dec 03 12:38:53 crc kubenswrapper[5052]: I1203 12:38:53.847838 5052 generic.go:334] "Generic (PLEG): container finished" podID="c34523d4-d634-4d76-a238-36566ef6a682" containerID="2bddb69fe53e0c8a15b016cc5ddde5164332e92554c29e75c2ee6664a3dd67a7" exitCode=1 Dec 03 12:38:53 crc kubenswrapper[5052]: I1203 12:38:53.847924 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-544c66dd6d-75q8t" event={"ID":"c34523d4-d634-4d76-a238-36566ef6a682","Type":"ContainerDied","Data":"2bddb69fe53e0c8a15b016cc5ddde5164332e92554c29e75c2ee6664a3dd67a7"} Dec 03 12:38:53 crc kubenswrapper[5052]: I1203 12:38:53.849103 5052 scope.go:117] "RemoveContainer" containerID="2bddb69fe53e0c8a15b016cc5ddde5164332e92554c29e75c2ee6664a3dd67a7" Dec 03 12:38:53 crc kubenswrapper[5052]: I1203 12:38:53.857027 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/neutron-544c66dd6d-75q8t" podUID="c34523d4-d634-4d76-a238-36566ef6a682" containerName="neutron-httpd" probeResult="failure" output="HTTP probe failed with statuscode: 503" Dec 03 12:38:54 crc kubenswrapper[5052]: I1203 12:38:54.860596 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovn-northd-0" event={"ID":"ea4cd237-7ec6-49f7-9afd-1ea159b9b23b","Type":"ContainerStarted","Data":"2e4992cc2e8905b7fe793a8b028dc60f77b2fc53b9e9aa87bd2e3fbe6b4f4d4c"} Dec 03 12:38:54 crc kubenswrapper[5052]: I1203 12:38:54.860958 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovn-northd-0" event={"ID":"ea4cd237-7ec6-49f7-9afd-1ea159b9b23b","Type":"ContainerStarted","Data":"b2f117c03e1adda1bc8b58e2d0870125a9b0fd3dffffabf7fb3480969ee908b3"} Dec 03 12:38:54 crc kubenswrapper[5052]: I1203 12:38:54.860981 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:38:54 crc kubenswrapper[5052]: I1203 12:38:54.863923 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_neutron-544c66dd6d-75q8t_c34523d4-d634-4d76-a238-36566ef6a682/neutron-api/0.log" Dec 03 12:38:54 crc kubenswrapper[5052]: I1203 12:38:54.863976 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-544c66dd6d-75q8t" event={"ID":"c34523d4-d634-4d76-a238-36566ef6a682","Type":"ContainerStarted","Data":"a71490515349faed4ab3d5147860c7b66eb2e343b2b37fce8896d878d5c4382f"} Dec 03 12:38:54 crc kubenswrapper[5052]: I1203 12:38:54.870182 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/neutron-544c66dd6d-75q8t" podUID="c34523d4-d634-4d76-a238-36566ef6a682" containerName="neutron-httpd" probeResult="failure" output="HTTP probe failed with statuscode: 503" Dec 03 12:38:54 crc kubenswrapper[5052]: I1203 12:38:54.891622 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/ovn-northd-0" podStartSLOduration=2.891597365 podStartE2EDuration="2.891597365s" podCreationTimestamp="2025-12-03 12:38:52 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:38:54.878531572 +0000 UTC m=+4725.466773336" watchObservedRunningTime="2025-12-03 12:38:54.891597365 +0000 UTC m=+4725.479839129" Dec 03 12:38:55 crc kubenswrapper[5052]: E1203 12:38:55.498716 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-galera-openstack-svc: secret "cert-galera-openstack-svc" not found Dec 03 12:38:55 crc kubenswrapper[5052]: E1203 12:38:55.498795 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/01fb7120-3521-4969-a2c4-b2637e0e2414-galera-tls-certs podName:01fb7120-3521-4969-a2c4-b2637e0e2414 nodeName:}" failed. No retries permitted until 2025-12-03 12:39:27.49877588 +0000 UTC m=+4758.087017644 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "galera-tls-certs" (UniqueName: "kubernetes.io/secret/01fb7120-3521-4969-a2c4-b2637e0e2414-galera-tls-certs") pod "openstack-galera-0" (UID: "01fb7120-3521-4969-a2c4-b2637e0e2414") : secret "cert-galera-openstack-svc" not found Dec 03 12:38:56 crc kubenswrapper[5052]: I1203 12:38:56.440029 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:38:56 crc kubenswrapper[5052]: I1203 12:38:56.440339 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-external-api-0" podUID="9dccead0-7ff2-4255-ac00-10a9b357c2db" containerName="glance-log" containerID="cri-o://96e6068a8658ed240b81b5a1db15585f3efc1468e4c44b15dd5b3a8a124b6f8e" gracePeriod=30 Dec 03 12:38:56 crc kubenswrapper[5052]: I1203 12:38:56.440400 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-external-api-0" podUID="9dccead0-7ff2-4255-ac00-10a9b357c2db" containerName="glance-httpd" containerID="cri-o://ee1a1b3c296fc80f39512ea1317cd9d3123e9fe3d41bf5b71488a44c14937d12" gracePeriod=30 Dec 03 12:38:56 crc kubenswrapper[5052]: I1203 12:38:56.484496 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:38:56 crc kubenswrapper[5052]: I1203 12:38:56.484775 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-internal-api-0" podUID="42e9c6bc-62d9-442a-a5f0-d885eff829cb" containerName="glance-log" containerID="cri-o://256155caf5dc190b0f04698e3cd7ad88f707a09926115e6b668bb753698f6ddf" gracePeriod=30 Dec 03 12:38:56 crc kubenswrapper[5052]: I1203 12:38:56.484865 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-internal-api-0" podUID="42e9c6bc-62d9-442a-a5f0-d885eff829cb" containerName="glance-httpd" containerID="cri-o://24924091dd14c68a3a47c176dd1bea56c02b9f160a15ed0661a72c290a768b27" gracePeriod=30 Dec 03 12:38:56 crc kubenswrapper[5052]: I1203 12:38:56.769106 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/openstack-galera-0"] Dec 03 12:38:56 crc kubenswrapper[5052]: I1203 12:38:56.884080 5052 generic.go:334] "Generic (PLEG): container finished" podID="9dccead0-7ff2-4255-ac00-10a9b357c2db" containerID="96e6068a8658ed240b81b5a1db15585f3efc1468e4c44b15dd5b3a8a124b6f8e" exitCode=143 Dec 03 12:38:56 crc kubenswrapper[5052]: I1203 12:38:56.884147 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"9dccead0-7ff2-4255-ac00-10a9b357c2db","Type":"ContainerDied","Data":"96e6068a8658ed240b81b5a1db15585f3efc1468e4c44b15dd5b3a8a124b6f8e"} Dec 03 12:38:56 crc kubenswrapper[5052]: I1203 12:38:56.886467 5052 generic.go:334] "Generic (PLEG): container finished" podID="42e9c6bc-62d9-442a-a5f0-d885eff829cb" containerID="256155caf5dc190b0f04698e3cd7ad88f707a09926115e6b668bb753698f6ddf" exitCode=143 Dec 03 12:38:56 crc kubenswrapper[5052]: I1203 12:38:56.886521 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"42e9c6bc-62d9-442a-a5f0-d885eff829cb","Type":"ContainerDied","Data":"256155caf5dc190b0f04698e3cd7ad88f707a09926115e6b668bb753698f6ddf"} Dec 03 12:38:56 crc kubenswrapper[5052]: I1203 12:38:56.896612 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/openstack-galera-0" podUID="01fb7120-3521-4969-a2c4-b2637e0e2414" containerName="galera" containerID="cri-o://71d046155554dd75f42c3796f0e8202ff1e0f2c95466f0def0740b661fba207f" gracePeriod=30 Dec 03 12:38:56 crc kubenswrapper[5052]: I1203 12:38:56.951691 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-metadata-0" podUID="edea8e53-340d-4f1e-99e5-082268c64ceb" containerName="nova-metadata-log" probeResult="failure" output="Get \"https://10.217.1.100:8775/\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Dec 03 12:38:56 crc kubenswrapper[5052]: I1203 12:38:56.951719 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-metadata-0" podUID="edea8e53-340d-4f1e-99e5-082268c64ceb" containerName="nova-metadata-metadata" probeResult="failure" output="Get \"https://10.217.1.100:8775/\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Dec 03 12:38:57 crc kubenswrapper[5052]: I1203 12:38:57.339907 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:38:57 crc kubenswrapper[5052]: I1203 12:38:57.365774 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:38:57 crc kubenswrapper[5052]: E1203 12:38:57.434632 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-kube-state-metrics-svc: secret "cert-kube-state-metrics-svc" not found Dec 03 12:38:57 crc kubenswrapper[5052]: E1203 12:38:57.434734 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b7defe22-28b4-4fec-8fe5-ebde8e658c6b-kube-state-metrics-tls-certs podName:b7defe22-28b4-4fec-8fe5-ebde8e658c6b nodeName:}" failed. No retries permitted until 2025-12-03 12:39:29.434714165 +0000 UTC m=+4760.022955929 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls-certs" (UniqueName: "kubernetes.io/secret/b7defe22-28b4-4fec-8fe5-ebde8e658c6b-kube-state-metrics-tls-certs") pod "kube-state-metrics-0" (UID: "b7defe22-28b4-4fec-8fe5-ebde8e658c6b") : secret "cert-kube-state-metrics-svc" not found Dec 03 12:38:57 crc kubenswrapper[5052]: I1203 12:38:57.911053 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:38:57 crc kubenswrapper[5052]: I1203 12:38:57.945744 5052 prober.go:107] "Probe failed" probeType="Liveness" pod="openstack-kuttl-tests/cinder-scheduler-0" podUID="d855c655-3b2f-4e4f-9b27-cf4e43a92045" containerName="cinder-scheduler" probeResult="failure" output="HTTP probe failed with statuscode: 500" Dec 03 12:38:59 crc kubenswrapper[5052]: I1203 12:38:59.912456 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:38:59 crc kubenswrapper[5052]: I1203 12:38:59.915492 5052 generic.go:334] "Generic (PLEG): container finished" podID="9dccead0-7ff2-4255-ac00-10a9b357c2db" containerID="ee1a1b3c296fc80f39512ea1317cd9d3123e9fe3d41bf5b71488a44c14937d12" exitCode=0 Dec 03 12:38:59 crc kubenswrapper[5052]: I1203 12:38:59.915574 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"9dccead0-7ff2-4255-ac00-10a9b357c2db","Type":"ContainerDied","Data":"ee1a1b3c296fc80f39512ea1317cd9d3123e9fe3d41bf5b71488a44c14937d12"} Dec 03 12:38:59 crc kubenswrapper[5052]: I1203 12:38:59.917949 5052 generic.go:334] "Generic (PLEG): container finished" podID="42e9c6bc-62d9-442a-a5f0-d885eff829cb" containerID="24924091dd14c68a3a47c176dd1bea56c02b9f160a15ed0661a72c290a768b27" exitCode=0 Dec 03 12:38:59 crc kubenswrapper[5052]: I1203 12:38:59.918008 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"42e9c6bc-62d9-442a-a5f0-d885eff829cb","Type":"ContainerDied","Data":"24924091dd14c68a3a47c176dd1bea56c02b9f160a15ed0661a72c290a768b27"} Dec 03 12:38:59 crc kubenswrapper[5052]: I1203 12:38:59.918064 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"42e9c6bc-62d9-442a-a5f0-d885eff829cb","Type":"ContainerDied","Data":"adca9569dc698688f1f02450e166a38ba90740456648576356a082d180255bd0"} Dec 03 12:38:59 crc kubenswrapper[5052]: I1203 12:38:59.918076 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="adca9569dc698688f1f02450e166a38ba90740456648576356a082d180255bd0" Dec 03 12:38:59 crc kubenswrapper[5052]: I1203 12:38:59.953950 5052 generic.go:334] "Generic (PLEG): container finished" podID="01fb7120-3521-4969-a2c4-b2637e0e2414" containerID="71d046155554dd75f42c3796f0e8202ff1e0f2c95466f0def0740b661fba207f" exitCode=0 Dec 03 12:38:59 crc kubenswrapper[5052]: I1203 12:38:59.954003 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-galera-0" event={"ID":"01fb7120-3521-4969-a2c4-b2637e0e2414","Type":"ContainerDied","Data":"71d046155554dd75f42c3796f0e8202ff1e0f2c95466f0def0740b661fba207f"} Dec 03 12:38:59 crc kubenswrapper[5052]: I1203 12:38:59.954041 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-galera-0" event={"ID":"01fb7120-3521-4969-a2c4-b2637e0e2414","Type":"ContainerDied","Data":"f9456d80cb676adb94c83be749d62019435a6816deaec5cc23878d4aed8c335f"} Dec 03 12:38:59 crc kubenswrapper[5052]: I1203 12:38:59.954063 5052 scope.go:117] "RemoveContainer" containerID="71d046155554dd75f42c3796f0e8202ff1e0f2c95466f0def0740b661fba207f" Dec 03 12:38:59 crc kubenswrapper[5052]: I1203 12:38:59.954262 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:38:59 crc kubenswrapper[5052]: I1203 12:38:59.969826 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:39:00 crc kubenswrapper[5052]: I1203 12:39:00.000247 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/42e9c6bc-62d9-442a-a5f0-d885eff829cb-internal-tls-certs\") pod \"42e9c6bc-62d9-442a-a5f0-d885eff829cb\" (UID: \"42e9c6bc-62d9-442a-a5f0-d885eff829cb\") " Dec 03 12:39:00 crc kubenswrapper[5052]: I1203 12:39:00.000677 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/42e9c6bc-62d9-442a-a5f0-d885eff829cb-logs\") pod \"42e9c6bc-62d9-442a-a5f0-d885eff829cb\" (UID: \"42e9c6bc-62d9-442a-a5f0-d885eff829cb\") " Dec 03 12:39:00 crc kubenswrapper[5052]: I1203 12:39:00.000733 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/01fb7120-3521-4969-a2c4-b2637e0e2414-operator-scripts\") pod \"01fb7120-3521-4969-a2c4-b2637e0e2414\" (UID: \"01fb7120-3521-4969-a2c4-b2637e0e2414\") " Dec 03 12:39:00 crc kubenswrapper[5052]: I1203 12:39:00.000789 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/01fb7120-3521-4969-a2c4-b2637e0e2414-galera-tls-certs\") pod \"01fb7120-3521-4969-a2c4-b2637e0e2414\" (UID: \"01fb7120-3521-4969-a2c4-b2637e0e2414\") " Dec 03 12:39:00 crc kubenswrapper[5052]: I1203 12:39:00.000847 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"mysql-db\" (UniqueName: \"kubernetes.io/local-volume/local-storage07-crc\") pod \"01fb7120-3521-4969-a2c4-b2637e0e2414\" (UID: \"01fb7120-3521-4969-a2c4-b2637e0e2414\") " Dec 03 12:39:00 crc kubenswrapper[5052]: I1203 12:39:00.000882 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/01fb7120-3521-4969-a2c4-b2637e0e2414-config-data-default\") pod \"01fb7120-3521-4969-a2c4-b2637e0e2414\" (UID: \"01fb7120-3521-4969-a2c4-b2637e0e2414\") " Dec 03 12:39:00 crc kubenswrapper[5052]: I1203 12:39:00.000913 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kmsm9\" (UniqueName: \"kubernetes.io/projected/42e9c6bc-62d9-442a-a5f0-d885eff829cb-kube-api-access-kmsm9\") pod \"42e9c6bc-62d9-442a-a5f0-d885eff829cb\" (UID: \"42e9c6bc-62d9-442a-a5f0-d885eff829cb\") " Dec 03 12:39:00 crc kubenswrapper[5052]: I1203 12:39:00.000934 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/01fb7120-3521-4969-a2c4-b2637e0e2414-combined-ca-bundle\") pod \"01fb7120-3521-4969-a2c4-b2637e0e2414\" (UID: \"01fb7120-3521-4969-a2c4-b2637e0e2414\") " Dec 03 12:39:00 crc kubenswrapper[5052]: I1203 12:39:00.000973 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/42e9c6bc-62d9-442a-a5f0-d885eff829cb-config-data\") pod \"42e9c6bc-62d9-442a-a5f0-d885eff829cb\" (UID: \"42e9c6bc-62d9-442a-a5f0-d885eff829cb\") " Dec 03 12:39:00 crc kubenswrapper[5052]: I1203 12:39:00.001048 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/01fb7120-3521-4969-a2c4-b2637e0e2414-config-data-generated\") pod \"01fb7120-3521-4969-a2c4-b2637e0e2414\" (UID: \"01fb7120-3521-4969-a2c4-b2637e0e2414\") " Dec 03 12:39:00 crc kubenswrapper[5052]: I1203 12:39:00.001090 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/42e9c6bc-62d9-442a-a5f0-d885eff829cb-scripts\") pod \"42e9c6bc-62d9-442a-a5f0-d885eff829cb\" (UID: \"42e9c6bc-62d9-442a-a5f0-d885eff829cb\") " Dec 03 12:39:00 crc kubenswrapper[5052]: I1203 12:39:00.001131 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/42e9c6bc-62d9-442a-a5f0-d885eff829cb-httpd-run\") pod \"42e9c6bc-62d9-442a-a5f0-d885eff829cb\" (UID: \"42e9c6bc-62d9-442a-a5f0-d885eff829cb\") " Dec 03 12:39:00 crc kubenswrapper[5052]: I1203 12:39:00.001162 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wksnx\" (UniqueName: \"kubernetes.io/projected/01fb7120-3521-4969-a2c4-b2637e0e2414-kube-api-access-wksnx\") pod \"01fb7120-3521-4969-a2c4-b2637e0e2414\" (UID: \"01fb7120-3521-4969-a2c4-b2637e0e2414\") " Dec 03 12:39:00 crc kubenswrapper[5052]: I1203 12:39:00.001183 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/01fb7120-3521-4969-a2c4-b2637e0e2414-kolla-config\") pod \"01fb7120-3521-4969-a2c4-b2637e0e2414\" (UID: \"01fb7120-3521-4969-a2c4-b2637e0e2414\") " Dec 03 12:39:00 crc kubenswrapper[5052]: I1203 12:39:00.001215 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/42e9c6bc-62d9-442a-a5f0-d885eff829cb-combined-ca-bundle\") pod \"42e9c6bc-62d9-442a-a5f0-d885eff829cb\" (UID: \"42e9c6bc-62d9-442a-a5f0-d885eff829cb\") " Dec 03 12:39:00 crc kubenswrapper[5052]: I1203 12:39:00.001240 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"glance\" (UniqueName: \"kubernetes.io/local-volume/local-storage18-crc\") pod \"42e9c6bc-62d9-442a-a5f0-d885eff829cb\" (UID: \"42e9c6bc-62d9-442a-a5f0-d885eff829cb\") " Dec 03 12:39:00 crc kubenswrapper[5052]: I1203 12:39:00.003008 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/42e9c6bc-62d9-442a-a5f0-d885eff829cb-logs" (OuterVolumeSpecName: "logs") pod "42e9c6bc-62d9-442a-a5f0-d885eff829cb" (UID: "42e9c6bc-62d9-442a-a5f0-d885eff829cb"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:39:00 crc kubenswrapper[5052]: I1203 12:39:00.003251 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/01fb7120-3521-4969-a2c4-b2637e0e2414-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "01fb7120-3521-4969-a2c4-b2637e0e2414" (UID: "01fb7120-3521-4969-a2c4-b2637e0e2414"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:39:00 crc kubenswrapper[5052]: I1203 12:39:00.003855 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/01fb7120-3521-4969-a2c4-b2637e0e2414-config-data-generated" (OuterVolumeSpecName: "config-data-generated") pod "01fb7120-3521-4969-a2c4-b2637e0e2414" (UID: "01fb7120-3521-4969-a2c4-b2637e0e2414"). InnerVolumeSpecName "config-data-generated". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:39:00 crc kubenswrapper[5052]: I1203 12:39:00.004153 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/01fb7120-3521-4969-a2c4-b2637e0e2414-config-data-default" (OuterVolumeSpecName: "config-data-default") pod "01fb7120-3521-4969-a2c4-b2637e0e2414" (UID: "01fb7120-3521-4969-a2c4-b2637e0e2414"). InnerVolumeSpecName "config-data-default". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:39:00 crc kubenswrapper[5052]: I1203 12:39:00.005182 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/01fb7120-3521-4969-a2c4-b2637e0e2414-kolla-config" (OuterVolumeSpecName: "kolla-config") pod "01fb7120-3521-4969-a2c4-b2637e0e2414" (UID: "01fb7120-3521-4969-a2c4-b2637e0e2414"). InnerVolumeSpecName "kolla-config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:39:00 crc kubenswrapper[5052]: I1203 12:39:00.025654 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/42e9c6bc-62d9-442a-a5f0-d885eff829cb-httpd-run" (OuterVolumeSpecName: "httpd-run") pod "42e9c6bc-62d9-442a-a5f0-d885eff829cb" (UID: "42e9c6bc-62d9-442a-a5f0-d885eff829cb"). InnerVolumeSpecName "httpd-run". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:39:00 crc kubenswrapper[5052]: I1203 12:39:00.056894 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage18-crc" (OuterVolumeSpecName: "glance") pod "42e9c6bc-62d9-442a-a5f0-d885eff829cb" (UID: "42e9c6bc-62d9-442a-a5f0-d885eff829cb"). InnerVolumeSpecName "local-storage18-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 12:39:00 crc kubenswrapper[5052]: I1203 12:39:00.058460 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/42e9c6bc-62d9-442a-a5f0-d885eff829cb-scripts" (OuterVolumeSpecName: "scripts") pod "42e9c6bc-62d9-442a-a5f0-d885eff829cb" (UID: "42e9c6bc-62d9-442a-a5f0-d885eff829cb"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:39:00 crc kubenswrapper[5052]: I1203 12:39:00.066832 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/01fb7120-3521-4969-a2c4-b2637e0e2414-kube-api-access-wksnx" (OuterVolumeSpecName: "kube-api-access-wksnx") pod "01fb7120-3521-4969-a2c4-b2637e0e2414" (UID: "01fb7120-3521-4969-a2c4-b2637e0e2414"). InnerVolumeSpecName "kube-api-access-wksnx". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:39:00 crc kubenswrapper[5052]: I1203 12:39:00.075825 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/42e9c6bc-62d9-442a-a5f0-d885eff829cb-kube-api-access-kmsm9" (OuterVolumeSpecName: "kube-api-access-kmsm9") pod "42e9c6bc-62d9-442a-a5f0-d885eff829cb" (UID: "42e9c6bc-62d9-442a-a5f0-d885eff829cb"). InnerVolumeSpecName "kube-api-access-kmsm9". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:39:00 crc kubenswrapper[5052]: I1203 12:39:00.105959 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/42e9c6bc-62d9-442a-a5f0-d885eff829cb-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "42e9c6bc-62d9-442a-a5f0-d885eff829cb" (UID: "42e9c6bc-62d9-442a-a5f0-d885eff829cb"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:39:00 crc kubenswrapper[5052]: I1203 12:39:00.106850 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/01fb7120-3521-4969-a2c4-b2637e0e2414-config-data-default\") on node \"crc\" DevicePath \"\"" Dec 03 12:39:00 crc kubenswrapper[5052]: I1203 12:39:00.106886 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-kmsm9\" (UniqueName: \"kubernetes.io/projected/42e9c6bc-62d9-442a-a5f0-d885eff829cb-kube-api-access-kmsm9\") on node \"crc\" DevicePath \"\"" Dec 03 12:39:00 crc kubenswrapper[5052]: I1203 12:39:00.106902 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/01fb7120-3521-4969-a2c4-b2637e0e2414-config-data-generated\") on node \"crc\" DevicePath \"\"" Dec 03 12:39:00 crc kubenswrapper[5052]: I1203 12:39:00.106914 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/42e9c6bc-62d9-442a-a5f0-d885eff829cb-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:39:00 crc kubenswrapper[5052]: I1203 12:39:00.106924 5052 reconciler_common.go:293] "Volume detached for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/42e9c6bc-62d9-442a-a5f0-d885eff829cb-httpd-run\") on node \"crc\" DevicePath \"\"" Dec 03 12:39:00 crc kubenswrapper[5052]: I1203 12:39:00.106935 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-wksnx\" (UniqueName: \"kubernetes.io/projected/01fb7120-3521-4969-a2c4-b2637e0e2414-kube-api-access-wksnx\") on node \"crc\" DevicePath \"\"" Dec 03 12:39:00 crc kubenswrapper[5052]: I1203 12:39:00.106946 5052 reconciler_common.go:293] "Volume detached for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/01fb7120-3521-4969-a2c4-b2637e0e2414-kolla-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:39:00 crc kubenswrapper[5052]: I1203 12:39:00.106958 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/42e9c6bc-62d9-442a-a5f0-d885eff829cb-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:39:00 crc kubenswrapper[5052]: I1203 12:39:00.106983 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage18-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage18-crc\") on node \"crc\" " Dec 03 12:39:00 crc kubenswrapper[5052]: I1203 12:39:00.106997 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/42e9c6bc-62d9-442a-a5f0-d885eff829cb-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:39:00 crc kubenswrapper[5052]: I1203 12:39:00.107010 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/01fb7120-3521-4969-a2c4-b2637e0e2414-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:39:00 crc kubenswrapper[5052]: I1203 12:39:00.108440 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/01fb7120-3521-4969-a2c4-b2637e0e2414-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "01fb7120-3521-4969-a2c4-b2637e0e2414" (UID: "01fb7120-3521-4969-a2c4-b2637e0e2414"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:39:00 crc kubenswrapper[5052]: I1203 12:39:00.114857 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage07-crc" (OuterVolumeSpecName: "mysql-db") pod "01fb7120-3521-4969-a2c4-b2637e0e2414" (UID: "01fb7120-3521-4969-a2c4-b2637e0e2414"). InnerVolumeSpecName "local-storage07-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 12:39:00 crc kubenswrapper[5052]: I1203 12:39:00.131994 5052 scope.go:117] "RemoveContainer" containerID="7cdb8d3632d0751bac948ca612e51336988e363391bc868fdb6600511091056c" Dec 03 12:39:00 crc kubenswrapper[5052]: I1203 12:39:00.208743 5052 scope.go:117] "RemoveContainer" containerID="71d046155554dd75f42c3796f0e8202ff1e0f2c95466f0def0740b661fba207f" Dec 03 12:39:00 crc kubenswrapper[5052]: I1203 12:39:00.210807 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage07-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage07-crc\") on node \"crc\" " Dec 03 12:39:00 crc kubenswrapper[5052]: I1203 12:39:00.210851 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/01fb7120-3521-4969-a2c4-b2637e0e2414-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:39:00 crc kubenswrapper[5052]: I1203 12:39:00.216194 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:39:00 crc kubenswrapper[5052]: I1203 12:39:00.216244 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:39:00 crc kubenswrapper[5052]: E1203 12:39:00.222681 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"71d046155554dd75f42c3796f0e8202ff1e0f2c95466f0def0740b661fba207f\": container with ID starting with 71d046155554dd75f42c3796f0e8202ff1e0f2c95466f0def0740b661fba207f not found: ID does not exist" containerID="71d046155554dd75f42c3796f0e8202ff1e0f2c95466f0def0740b661fba207f" Dec 03 12:39:00 crc kubenswrapper[5052]: I1203 12:39:00.222724 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"71d046155554dd75f42c3796f0e8202ff1e0f2c95466f0def0740b661fba207f"} err="failed to get container status \"71d046155554dd75f42c3796f0e8202ff1e0f2c95466f0def0740b661fba207f\": rpc error: code = NotFound desc = could not find container \"71d046155554dd75f42c3796f0e8202ff1e0f2c95466f0def0740b661fba207f\": container with ID starting with 71d046155554dd75f42c3796f0e8202ff1e0f2c95466f0def0740b661fba207f not found: ID does not exist" Dec 03 12:39:00 crc kubenswrapper[5052]: I1203 12:39:00.222752 5052 scope.go:117] "RemoveContainer" containerID="7cdb8d3632d0751bac948ca612e51336988e363391bc868fdb6600511091056c" Dec 03 12:39:00 crc kubenswrapper[5052]: E1203 12:39:00.224626 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7cdb8d3632d0751bac948ca612e51336988e363391bc868fdb6600511091056c\": container with ID starting with 7cdb8d3632d0751bac948ca612e51336988e363391bc868fdb6600511091056c not found: ID does not exist" containerID="7cdb8d3632d0751bac948ca612e51336988e363391bc868fdb6600511091056c" Dec 03 12:39:00 crc kubenswrapper[5052]: I1203 12:39:00.224673 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7cdb8d3632d0751bac948ca612e51336988e363391bc868fdb6600511091056c"} err="failed to get container status \"7cdb8d3632d0751bac948ca612e51336988e363391bc868fdb6600511091056c\": rpc error: code = NotFound desc = could not find container \"7cdb8d3632d0751bac948ca612e51336988e363391bc868fdb6600511091056c\": container with ID starting with 7cdb8d3632d0751bac948ca612e51336988e363391bc868fdb6600511091056c not found: ID does not exist" Dec 03 12:39:00 crc kubenswrapper[5052]: I1203 12:39:00.242328 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:39:00 crc kubenswrapper[5052]: I1203 12:39:00.270304 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage18-crc" (UniqueName: "kubernetes.io/local-volume/local-storage18-crc") on node "crc" Dec 03 12:39:00 crc kubenswrapper[5052]: I1203 12:39:00.296654 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/42e9c6bc-62d9-442a-a5f0-d885eff829cb-config-data" (OuterVolumeSpecName: "config-data") pod "42e9c6bc-62d9-442a-a5f0-d885eff829cb" (UID: "42e9c6bc-62d9-442a-a5f0-d885eff829cb"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:39:00 crc kubenswrapper[5052]: I1203 12:39:00.318649 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage07-crc" (UniqueName: "kubernetes.io/local-volume/local-storage07-crc") on node "crc" Dec 03 12:39:00 crc kubenswrapper[5052]: I1203 12:39:00.319475 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage18-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage18-crc\") on node \"crc\" DevicePath \"\"" Dec 03 12:39:00 crc kubenswrapper[5052]: I1203 12:39:00.319770 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage07-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage07-crc\") on node \"crc\" DevicePath \"\"" Dec 03 12:39:00 crc kubenswrapper[5052]: I1203 12:39:00.319842 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/42e9c6bc-62d9-442a-a5f0-d885eff829cb-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:39:00 crc kubenswrapper[5052]: I1203 12:39:00.367626 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/01fb7120-3521-4969-a2c4-b2637e0e2414-galera-tls-certs" (OuterVolumeSpecName: "galera-tls-certs") pod "01fb7120-3521-4969-a2c4-b2637e0e2414" (UID: "01fb7120-3521-4969-a2c4-b2637e0e2414"). InnerVolumeSpecName "galera-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:39:00 crc kubenswrapper[5052]: I1203 12:39:00.383859 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/42e9c6bc-62d9-442a-a5f0-d885eff829cb-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "42e9c6bc-62d9-442a-a5f0-d885eff829cb" (UID: "42e9c6bc-62d9-442a-a5f0-d885eff829cb"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:39:00 crc kubenswrapper[5052]: I1203 12:39:00.421243 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-phrpm\" (UniqueName: \"kubernetes.io/projected/9dccead0-7ff2-4255-ac00-10a9b357c2db-kube-api-access-phrpm\") pod \"9dccead0-7ff2-4255-ac00-10a9b357c2db\" (UID: \"9dccead0-7ff2-4255-ac00-10a9b357c2db\") " Dec 03 12:39:00 crc kubenswrapper[5052]: I1203 12:39:00.421321 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/9dccead0-7ff2-4255-ac00-10a9b357c2db-scripts\") pod \"9dccead0-7ff2-4255-ac00-10a9b357c2db\" (UID: \"9dccead0-7ff2-4255-ac00-10a9b357c2db\") " Dec 03 12:39:00 crc kubenswrapper[5052]: I1203 12:39:00.421361 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/9dccead0-7ff2-4255-ac00-10a9b357c2db-logs\") pod \"9dccead0-7ff2-4255-ac00-10a9b357c2db\" (UID: \"9dccead0-7ff2-4255-ac00-10a9b357c2db\") " Dec 03 12:39:00 crc kubenswrapper[5052]: I1203 12:39:00.421471 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/9dccead0-7ff2-4255-ac00-10a9b357c2db-config-data\") pod \"9dccead0-7ff2-4255-ac00-10a9b357c2db\" (UID: \"9dccead0-7ff2-4255-ac00-10a9b357c2db\") " Dec 03 12:39:00 crc kubenswrapper[5052]: I1203 12:39:00.421552 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9dccead0-7ff2-4255-ac00-10a9b357c2db-combined-ca-bundle\") pod \"9dccead0-7ff2-4255-ac00-10a9b357c2db\" (UID: \"9dccead0-7ff2-4255-ac00-10a9b357c2db\") " Dec 03 12:39:00 crc kubenswrapper[5052]: I1203 12:39:00.421576 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/9dccead0-7ff2-4255-ac00-10a9b357c2db-httpd-run\") pod \"9dccead0-7ff2-4255-ac00-10a9b357c2db\" (UID: \"9dccead0-7ff2-4255-ac00-10a9b357c2db\") " Dec 03 12:39:00 crc kubenswrapper[5052]: I1203 12:39:00.421607 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"glance\" (UniqueName: \"kubernetes.io/local-volume/local-storage12-crc\") pod \"9dccead0-7ff2-4255-ac00-10a9b357c2db\" (UID: \"9dccead0-7ff2-4255-ac00-10a9b357c2db\") " Dec 03 12:39:00 crc kubenswrapper[5052]: I1203 12:39:00.421751 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/9dccead0-7ff2-4255-ac00-10a9b357c2db-public-tls-certs\") pod \"9dccead0-7ff2-4255-ac00-10a9b357c2db\" (UID: \"9dccead0-7ff2-4255-ac00-10a9b357c2db\") " Dec 03 12:39:00 crc kubenswrapper[5052]: I1203 12:39:00.422295 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9dccead0-7ff2-4255-ac00-10a9b357c2db-httpd-run" (OuterVolumeSpecName: "httpd-run") pod "9dccead0-7ff2-4255-ac00-10a9b357c2db" (UID: "9dccead0-7ff2-4255-ac00-10a9b357c2db"). InnerVolumeSpecName "httpd-run". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:39:00 crc kubenswrapper[5052]: I1203 12:39:00.422383 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/42e9c6bc-62d9-442a-a5f0-d885eff829cb-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:39:00 crc kubenswrapper[5052]: I1203 12:39:00.422402 5052 reconciler_common.go:293] "Volume detached for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/01fb7120-3521-4969-a2c4-b2637e0e2414-galera-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:39:00 crc kubenswrapper[5052]: I1203 12:39:00.422416 5052 reconciler_common.go:293] "Volume detached for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/9dccead0-7ff2-4255-ac00-10a9b357c2db-httpd-run\") on node \"crc\" DevicePath \"\"" Dec 03 12:39:00 crc kubenswrapper[5052]: I1203 12:39:00.424736 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9dccead0-7ff2-4255-ac00-10a9b357c2db-logs" (OuterVolumeSpecName: "logs") pod "9dccead0-7ff2-4255-ac00-10a9b357c2db" (UID: "9dccead0-7ff2-4255-ac00-10a9b357c2db"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:39:00 crc kubenswrapper[5052]: I1203 12:39:00.425666 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9dccead0-7ff2-4255-ac00-10a9b357c2db-scripts" (OuterVolumeSpecName: "scripts") pod "9dccead0-7ff2-4255-ac00-10a9b357c2db" (UID: "9dccead0-7ff2-4255-ac00-10a9b357c2db"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:39:00 crc kubenswrapper[5052]: I1203 12:39:00.427136 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage12-crc" (OuterVolumeSpecName: "glance") pod "9dccead0-7ff2-4255-ac00-10a9b357c2db" (UID: "9dccead0-7ff2-4255-ac00-10a9b357c2db"). InnerVolumeSpecName "local-storage12-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 12:39:00 crc kubenswrapper[5052]: I1203 12:39:00.442874 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9dccead0-7ff2-4255-ac00-10a9b357c2db-kube-api-access-phrpm" (OuterVolumeSpecName: "kube-api-access-phrpm") pod "9dccead0-7ff2-4255-ac00-10a9b357c2db" (UID: "9dccead0-7ff2-4255-ac00-10a9b357c2db"). InnerVolumeSpecName "kube-api-access-phrpm". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:39:00 crc kubenswrapper[5052]: I1203 12:39:00.477589 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9dccead0-7ff2-4255-ac00-10a9b357c2db-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "9dccead0-7ff2-4255-ac00-10a9b357c2db" (UID: "9dccead0-7ff2-4255-ac00-10a9b357c2db"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:39:00 crc kubenswrapper[5052]: I1203 12:39:00.488742 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9dccead0-7ff2-4255-ac00-10a9b357c2db-config-data" (OuterVolumeSpecName: "config-data") pod "9dccead0-7ff2-4255-ac00-10a9b357c2db" (UID: "9dccead0-7ff2-4255-ac00-10a9b357c2db"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:39:00 crc kubenswrapper[5052]: I1203 12:39:00.505536 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9dccead0-7ff2-4255-ac00-10a9b357c2db-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "9dccead0-7ff2-4255-ac00-10a9b357c2db" (UID: "9dccead0-7ff2-4255-ac00-10a9b357c2db"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:39:00 crc kubenswrapper[5052]: I1203 12:39:00.523904 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/9dccead0-7ff2-4255-ac00-10a9b357c2db-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:39:00 crc kubenswrapper[5052]: I1203 12:39:00.523950 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-phrpm\" (UniqueName: \"kubernetes.io/projected/9dccead0-7ff2-4255-ac00-10a9b357c2db-kube-api-access-phrpm\") on node \"crc\" DevicePath \"\"" Dec 03 12:39:00 crc kubenswrapper[5052]: I1203 12:39:00.523968 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/9dccead0-7ff2-4255-ac00-10a9b357c2db-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:39:00 crc kubenswrapper[5052]: I1203 12:39:00.523979 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/9dccead0-7ff2-4255-ac00-10a9b357c2db-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:39:00 crc kubenswrapper[5052]: I1203 12:39:00.523988 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/9dccead0-7ff2-4255-ac00-10a9b357c2db-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:39:00 crc kubenswrapper[5052]: I1203 12:39:00.523996 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9dccead0-7ff2-4255-ac00-10a9b357c2db-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:39:00 crc kubenswrapper[5052]: I1203 12:39:00.524026 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage12-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage12-crc\") on node \"crc\" " Dec 03 12:39:00 crc kubenswrapper[5052]: I1203 12:39:00.548790 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage12-crc" (UniqueName: "kubernetes.io/local-volume/local-storage12-crc") on node "crc" Dec 03 12:39:00 crc kubenswrapper[5052]: I1203 12:39:00.578504 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:39:00 crc kubenswrapper[5052]: I1203 12:39:00.585251 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="68678c3f-bf25-488f-a422-1bf8586703b3" containerName="ceilometer-central-agent" containerID="cri-o://aece19546c52d53c88212e467860c80e8703242aed2c7ccbcbed127d5a4cd806" gracePeriod=30 Dec 03 12:39:00 crc kubenswrapper[5052]: I1203 12:39:00.585272 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="68678c3f-bf25-488f-a422-1bf8586703b3" containerName="sg-core" containerID="cri-o://2c6d15a1288e67b74a8c91f85763a3f1a0710e092f088b7defedb0435024e3ee" gracePeriod=30 Dec 03 12:39:00 crc kubenswrapper[5052]: I1203 12:39:00.585312 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="68678c3f-bf25-488f-a422-1bf8586703b3" containerName="ceilometer-notification-agent" containerID="cri-o://75aabd113fe3f8c31645077c72c336a2d111ebe7269663fa192668bdab05f74c" gracePeriod=30 Dec 03 12:39:00 crc kubenswrapper[5052]: I1203 12:39:00.585311 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="68678c3f-bf25-488f-a422-1bf8586703b3" containerName="proxy-httpd" containerID="cri-o://a50f4e422eaeb77c3e3ddb88c72b4b83fabdf60cb1ad6b1d7fbca14f74ef6cc3" gracePeriod=30 Dec 03 12:39:00 crc kubenswrapper[5052]: I1203 12:39:00.615881 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/openstack-galera-0"] Dec 03 12:39:00 crc kubenswrapper[5052]: I1203 12:39:00.626059 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/openstack-galera-0"] Dec 03 12:39:00 crc kubenswrapper[5052]: I1203 12:39:00.630247 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage12-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage12-crc\") on node \"crc\" DevicePath \"\"" Dec 03 12:39:00 crc kubenswrapper[5052]: I1203 12:39:00.674300 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/openstack-galera-0"] Dec 03 12:39:00 crc kubenswrapper[5052]: E1203 12:39:00.675841 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="01fb7120-3521-4969-a2c4-b2637e0e2414" containerName="mysql-bootstrap" Dec 03 12:39:00 crc kubenswrapper[5052]: I1203 12:39:00.675858 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="01fb7120-3521-4969-a2c4-b2637e0e2414" containerName="mysql-bootstrap" Dec 03 12:39:00 crc kubenswrapper[5052]: E1203 12:39:00.675886 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="01fb7120-3521-4969-a2c4-b2637e0e2414" containerName="galera" Dec 03 12:39:00 crc kubenswrapper[5052]: I1203 12:39:00.675892 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="01fb7120-3521-4969-a2c4-b2637e0e2414" containerName="galera" Dec 03 12:39:00 crc kubenswrapper[5052]: E1203 12:39:00.675902 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="42e9c6bc-62d9-442a-a5f0-d885eff829cb" containerName="glance-httpd" Dec 03 12:39:00 crc kubenswrapper[5052]: I1203 12:39:00.675913 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="42e9c6bc-62d9-442a-a5f0-d885eff829cb" containerName="glance-httpd" Dec 03 12:39:00 crc kubenswrapper[5052]: E1203 12:39:00.675940 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="42e9c6bc-62d9-442a-a5f0-d885eff829cb" containerName="glance-log" Dec 03 12:39:00 crc kubenswrapper[5052]: I1203 12:39:00.675946 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="42e9c6bc-62d9-442a-a5f0-d885eff829cb" containerName="glance-log" Dec 03 12:39:00 crc kubenswrapper[5052]: E1203 12:39:00.675968 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="9dccead0-7ff2-4255-ac00-10a9b357c2db" containerName="glance-httpd" Dec 03 12:39:00 crc kubenswrapper[5052]: I1203 12:39:00.675975 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="9dccead0-7ff2-4255-ac00-10a9b357c2db" containerName="glance-httpd" Dec 03 12:39:00 crc kubenswrapper[5052]: E1203 12:39:00.676001 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="9dccead0-7ff2-4255-ac00-10a9b357c2db" containerName="glance-log" Dec 03 12:39:00 crc kubenswrapper[5052]: I1203 12:39:00.676007 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="9dccead0-7ff2-4255-ac00-10a9b357c2db" containerName="glance-log" Dec 03 12:39:00 crc kubenswrapper[5052]: I1203 12:39:00.676557 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="9dccead0-7ff2-4255-ac00-10a9b357c2db" containerName="glance-log" Dec 03 12:39:00 crc kubenswrapper[5052]: I1203 12:39:00.676636 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="42e9c6bc-62d9-442a-a5f0-d885eff829cb" containerName="glance-httpd" Dec 03 12:39:00 crc kubenswrapper[5052]: I1203 12:39:00.676657 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="42e9c6bc-62d9-442a-a5f0-d885eff829cb" containerName="glance-log" Dec 03 12:39:00 crc kubenswrapper[5052]: I1203 12:39:00.676829 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="01fb7120-3521-4969-a2c4-b2637e0e2414" containerName="galera" Dec 03 12:39:00 crc kubenswrapper[5052]: I1203 12:39:00.676840 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="9dccead0-7ff2-4255-ac00-10a9b357c2db" containerName="glance-httpd" Dec 03 12:39:00 crc kubenswrapper[5052]: I1203 12:39:00.693483 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:39:00 crc kubenswrapper[5052]: I1203 12:39:00.696550 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"galera-openstack-dockercfg-hsg7j" Dec 03 12:39:00 crc kubenswrapper[5052]: I1203 12:39:00.696794 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-galera-openstack-svc" Dec 03 12:39:00 crc kubenswrapper[5052]: I1203 12:39:00.697026 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"openstack-scripts" Dec 03 12:39:00 crc kubenswrapper[5052]: I1203 12:39:00.697340 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"openstack-config-data" Dec 03 12:39:00 crc kubenswrapper[5052]: I1203 12:39:00.697844 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/openstack-galera-0"] Dec 03 12:39:00 crc kubenswrapper[5052]: I1203 12:39:00.767190 5052 prober.go:107] "Probe failed" probeType="Liveness" pod="openstack-kuttl-tests/cinder-scheduler-0" podUID="d855c655-3b2f-4e4f-9b27-cf4e43a92045" containerName="cinder-scheduler" probeResult="failure" output="HTTP probe failed with statuscode: 500" Dec 03 12:39:00 crc kubenswrapper[5052]: I1203 12:39:00.835202 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a7ae5e02-8418-40e2-b65d-7b74625fe5b3-combined-ca-bundle\") pod \"openstack-galera-0\" (UID: \"a7ae5e02-8418-40e2-b65d-7b74625fe5b3\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:39:00 crc kubenswrapper[5052]: I1203 12:39:00.835405 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/a7ae5e02-8418-40e2-b65d-7b74625fe5b3-kolla-config\") pod \"openstack-galera-0\" (UID: \"a7ae5e02-8418-40e2-b65d-7b74625fe5b3\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:39:00 crc kubenswrapper[5052]: I1203 12:39:00.835541 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/a7ae5e02-8418-40e2-b65d-7b74625fe5b3-operator-scripts\") pod \"openstack-galera-0\" (UID: \"a7ae5e02-8418-40e2-b65d-7b74625fe5b3\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:39:00 crc kubenswrapper[5052]: I1203 12:39:00.835674 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/a7ae5e02-8418-40e2-b65d-7b74625fe5b3-galera-tls-certs\") pod \"openstack-galera-0\" (UID: \"a7ae5e02-8418-40e2-b65d-7b74625fe5b3\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:39:00 crc kubenswrapper[5052]: I1203 12:39:00.835736 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-84lr2\" (UniqueName: \"kubernetes.io/projected/a7ae5e02-8418-40e2-b65d-7b74625fe5b3-kube-api-access-84lr2\") pod \"openstack-galera-0\" (UID: \"a7ae5e02-8418-40e2-b65d-7b74625fe5b3\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:39:00 crc kubenswrapper[5052]: I1203 12:39:00.835777 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage07-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage07-crc\") pod \"openstack-galera-0\" (UID: \"a7ae5e02-8418-40e2-b65d-7b74625fe5b3\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:39:00 crc kubenswrapper[5052]: I1203 12:39:00.835831 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/a7ae5e02-8418-40e2-b65d-7b74625fe5b3-config-data-generated\") pod \"openstack-galera-0\" (UID: \"a7ae5e02-8418-40e2-b65d-7b74625fe5b3\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:39:00 crc kubenswrapper[5052]: I1203 12:39:00.835949 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/a7ae5e02-8418-40e2-b65d-7b74625fe5b3-config-data-default\") pod \"openstack-galera-0\" (UID: \"a7ae5e02-8418-40e2-b65d-7b74625fe5b3\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:39:00 crc kubenswrapper[5052]: I1203 12:39:00.937679 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/a7ae5e02-8418-40e2-b65d-7b74625fe5b3-galera-tls-certs\") pod \"openstack-galera-0\" (UID: \"a7ae5e02-8418-40e2-b65d-7b74625fe5b3\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:39:00 crc kubenswrapper[5052]: I1203 12:39:00.937754 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-84lr2\" (UniqueName: \"kubernetes.io/projected/a7ae5e02-8418-40e2-b65d-7b74625fe5b3-kube-api-access-84lr2\") pod \"openstack-galera-0\" (UID: \"a7ae5e02-8418-40e2-b65d-7b74625fe5b3\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:39:00 crc kubenswrapper[5052]: I1203 12:39:00.937788 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage07-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage07-crc\") pod \"openstack-galera-0\" (UID: \"a7ae5e02-8418-40e2-b65d-7b74625fe5b3\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:39:00 crc kubenswrapper[5052]: I1203 12:39:00.937832 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/a7ae5e02-8418-40e2-b65d-7b74625fe5b3-config-data-generated\") pod \"openstack-galera-0\" (UID: \"a7ae5e02-8418-40e2-b65d-7b74625fe5b3\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:39:00 crc kubenswrapper[5052]: I1203 12:39:00.937860 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/a7ae5e02-8418-40e2-b65d-7b74625fe5b3-config-data-default\") pod \"openstack-galera-0\" (UID: \"a7ae5e02-8418-40e2-b65d-7b74625fe5b3\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:39:00 crc kubenswrapper[5052]: I1203 12:39:00.937900 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a7ae5e02-8418-40e2-b65d-7b74625fe5b3-combined-ca-bundle\") pod \"openstack-galera-0\" (UID: \"a7ae5e02-8418-40e2-b65d-7b74625fe5b3\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:39:00 crc kubenswrapper[5052]: I1203 12:39:00.937979 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/a7ae5e02-8418-40e2-b65d-7b74625fe5b3-kolla-config\") pod \"openstack-galera-0\" (UID: \"a7ae5e02-8418-40e2-b65d-7b74625fe5b3\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:39:00 crc kubenswrapper[5052]: I1203 12:39:00.938026 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/a7ae5e02-8418-40e2-b65d-7b74625fe5b3-operator-scripts\") pod \"openstack-galera-0\" (UID: \"a7ae5e02-8418-40e2-b65d-7b74625fe5b3\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:39:00 crc kubenswrapper[5052]: I1203 12:39:00.938629 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage07-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage07-crc\") pod \"openstack-galera-0\" (UID: \"a7ae5e02-8418-40e2-b65d-7b74625fe5b3\") device mount path \"/mnt/openstack/pv07\"" pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:39:00 crc kubenswrapper[5052]: I1203 12:39:00.939117 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/a7ae5e02-8418-40e2-b65d-7b74625fe5b3-config-data-default\") pod \"openstack-galera-0\" (UID: \"a7ae5e02-8418-40e2-b65d-7b74625fe5b3\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:39:00 crc kubenswrapper[5052]: I1203 12:39:00.939457 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/a7ae5e02-8418-40e2-b65d-7b74625fe5b3-config-data-generated\") pod \"openstack-galera-0\" (UID: \"a7ae5e02-8418-40e2-b65d-7b74625fe5b3\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:39:00 crc kubenswrapper[5052]: I1203 12:39:00.939630 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/a7ae5e02-8418-40e2-b65d-7b74625fe5b3-operator-scripts\") pod \"openstack-galera-0\" (UID: \"a7ae5e02-8418-40e2-b65d-7b74625fe5b3\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:39:00 crc kubenswrapper[5052]: I1203 12:39:00.940133 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/a7ae5e02-8418-40e2-b65d-7b74625fe5b3-kolla-config\") pod \"openstack-galera-0\" (UID: \"a7ae5e02-8418-40e2-b65d-7b74625fe5b3\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:39:00 crc kubenswrapper[5052]: I1203 12:39:00.942397 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a7ae5e02-8418-40e2-b65d-7b74625fe5b3-combined-ca-bundle\") pod \"openstack-galera-0\" (UID: \"a7ae5e02-8418-40e2-b65d-7b74625fe5b3\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:39:00 crc kubenswrapper[5052]: I1203 12:39:00.942976 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/a7ae5e02-8418-40e2-b65d-7b74625fe5b3-galera-tls-certs\") pod \"openstack-galera-0\" (UID: \"a7ae5e02-8418-40e2-b65d-7b74625fe5b3\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:39:00 crc kubenswrapper[5052]: I1203 12:39:00.956946 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-84lr2\" (UniqueName: \"kubernetes.io/projected/a7ae5e02-8418-40e2-b65d-7b74625fe5b3-kube-api-access-84lr2\") pod \"openstack-galera-0\" (UID: \"a7ae5e02-8418-40e2-b65d-7b74625fe5b3\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:39:00 crc kubenswrapper[5052]: I1203 12:39:00.967687 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"9dccead0-7ff2-4255-ac00-10a9b357c2db","Type":"ContainerDied","Data":"99a719d77c1c2a1f956f5aba0b3d775f4d82fe13fa8a0f238262600b3e4746e4"} Dec 03 12:39:00 crc kubenswrapper[5052]: I1203 12:39:00.967735 5052 scope.go:117] "RemoveContainer" containerID="ee1a1b3c296fc80f39512ea1317cd9d3123e9fe3d41bf5b71488a44c14937d12" Dec 03 12:39:00 crc kubenswrapper[5052]: I1203 12:39:00.967894 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:39:00 crc kubenswrapper[5052]: I1203 12:39:00.969901 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage07-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage07-crc\") pod \"openstack-galera-0\" (UID: \"a7ae5e02-8418-40e2-b65d-7b74625fe5b3\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:39:00 crc kubenswrapper[5052]: I1203 12:39:00.976809 5052 generic.go:334] "Generic (PLEG): container finished" podID="68678c3f-bf25-488f-a422-1bf8586703b3" containerID="a50f4e422eaeb77c3e3ddb88c72b4b83fabdf60cb1ad6b1d7fbca14f74ef6cc3" exitCode=0 Dec 03 12:39:00 crc kubenswrapper[5052]: I1203 12:39:00.976845 5052 generic.go:334] "Generic (PLEG): container finished" podID="68678c3f-bf25-488f-a422-1bf8586703b3" containerID="2c6d15a1288e67b74a8c91f85763a3f1a0710e092f088b7defedb0435024e3ee" exitCode=2 Dec 03 12:39:00 crc kubenswrapper[5052]: I1203 12:39:00.976879 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"68678c3f-bf25-488f-a422-1bf8586703b3","Type":"ContainerDied","Data":"a50f4e422eaeb77c3e3ddb88c72b4b83fabdf60cb1ad6b1d7fbca14f74ef6cc3"} Dec 03 12:39:00 crc kubenswrapper[5052]: I1203 12:39:00.976917 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"68678c3f-bf25-488f-a422-1bf8586703b3","Type":"ContainerDied","Data":"2c6d15a1288e67b74a8c91f85763a3f1a0710e092f088b7defedb0435024e3ee"} Dec 03 12:39:00 crc kubenswrapper[5052]: I1203 12:39:00.976944 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:39:01 crc kubenswrapper[5052]: I1203 12:39:01.007426 5052 scope.go:117] "RemoveContainer" containerID="96e6068a8658ed240b81b5a1db15585f3efc1468e4c44b15dd5b3a8a124b6f8e" Dec 03 12:39:01 crc kubenswrapper[5052]: I1203 12:39:01.031814 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:39:01 crc kubenswrapper[5052]: I1203 12:39:01.054434 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:39:01 crc kubenswrapper[5052]: I1203 12:39:01.061765 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:39:01 crc kubenswrapper[5052]: I1203 12:39:01.084873 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:39:01 crc kubenswrapper[5052]: I1203 12:39:01.102803 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:39:01 crc kubenswrapper[5052]: I1203 12:39:01.117248 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:39:01 crc kubenswrapper[5052]: I1203 12:39:01.119055 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:39:01 crc kubenswrapper[5052]: I1203 12:39:01.122138 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-glance-dockercfg-xkmzv" Dec 03 12:39:01 crc kubenswrapper[5052]: I1203 12:39:01.122448 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-scripts" Dec 03 12:39:01 crc kubenswrapper[5052]: I1203 12:39:01.122673 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-default-external-config-data" Dec 03 12:39:01 crc kubenswrapper[5052]: I1203 12:39:01.122760 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-glance-default-public-svc" Dec 03 12:39:01 crc kubenswrapper[5052]: I1203 12:39:01.129459 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:39:01 crc kubenswrapper[5052]: I1203 12:39:01.132409 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:39:01 crc kubenswrapper[5052]: I1203 12:39:01.140960 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-default-internal-config-data" Dec 03 12:39:01 crc kubenswrapper[5052]: I1203 12:39:01.141155 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-glance-default-internal-svc" Dec 03 12:39:01 crc kubenswrapper[5052]: I1203 12:39:01.145012 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:39:01 crc kubenswrapper[5052]: I1203 12:39:01.177084 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:39:01 crc kubenswrapper[5052]: I1203 12:39:01.245178 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5d4a61d7-2d2c-4186-a923-f59636bc0994-combined-ca-bundle\") pod \"glance-default-internal-api-0\" (UID: \"5d4a61d7-2d2c-4186-a923-f59636bc0994\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:39:01 crc kubenswrapper[5052]: I1203 12:39:01.245571 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/5d4a61d7-2d2c-4186-a923-f59636bc0994-logs\") pod \"glance-default-internal-api-0\" (UID: \"5d4a61d7-2d2c-4186-a923-f59636bc0994\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:39:01 crc kubenswrapper[5052]: I1203 12:39:01.245607 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mqvdm\" (UniqueName: \"kubernetes.io/projected/5d4a61d7-2d2c-4186-a923-f59636bc0994-kube-api-access-mqvdm\") pod \"glance-default-internal-api-0\" (UID: \"5d4a61d7-2d2c-4186-a923-f59636bc0994\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:39:01 crc kubenswrapper[5052]: I1203 12:39:01.245643 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4mtft\" (UniqueName: \"kubernetes.io/projected/02726407-aa1e-400f-aedb-78d896c83113-kube-api-access-4mtft\") pod \"glance-default-external-api-0\" (UID: \"02726407-aa1e-400f-aedb-78d896c83113\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:39:01 crc kubenswrapper[5052]: I1203 12:39:01.245730 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/02726407-aa1e-400f-aedb-78d896c83113-combined-ca-bundle\") pod \"glance-default-external-api-0\" (UID: \"02726407-aa1e-400f-aedb-78d896c83113\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:39:01 crc kubenswrapper[5052]: I1203 12:39:01.245756 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/5d4a61d7-2d2c-4186-a923-f59636bc0994-httpd-run\") pod \"glance-default-internal-api-0\" (UID: \"5d4a61d7-2d2c-4186-a923-f59636bc0994\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:39:01 crc kubenswrapper[5052]: I1203 12:39:01.245782 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/5d4a61d7-2d2c-4186-a923-f59636bc0994-scripts\") pod \"glance-default-internal-api-0\" (UID: \"5d4a61d7-2d2c-4186-a923-f59636bc0994\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:39:01 crc kubenswrapper[5052]: I1203 12:39:01.245802 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/02726407-aa1e-400f-aedb-78d896c83113-public-tls-certs\") pod \"glance-default-external-api-0\" (UID: \"02726407-aa1e-400f-aedb-78d896c83113\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:39:01 crc kubenswrapper[5052]: I1203 12:39:01.245830 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage18-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage18-crc\") pod \"glance-default-internal-api-0\" (UID: \"5d4a61d7-2d2c-4186-a923-f59636bc0994\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:39:01 crc kubenswrapper[5052]: I1203 12:39:01.245863 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/02726407-aa1e-400f-aedb-78d896c83113-httpd-run\") pod \"glance-default-external-api-0\" (UID: \"02726407-aa1e-400f-aedb-78d896c83113\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:39:01 crc kubenswrapper[5052]: I1203 12:39:01.245893 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/5d4a61d7-2d2c-4186-a923-f59636bc0994-internal-tls-certs\") pod \"glance-default-internal-api-0\" (UID: \"5d4a61d7-2d2c-4186-a923-f59636bc0994\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:39:01 crc kubenswrapper[5052]: I1203 12:39:01.246100 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage12-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage12-crc\") pod \"glance-default-external-api-0\" (UID: \"02726407-aa1e-400f-aedb-78d896c83113\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:39:01 crc kubenswrapper[5052]: I1203 12:39:01.246274 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/02726407-aa1e-400f-aedb-78d896c83113-logs\") pod \"glance-default-external-api-0\" (UID: \"02726407-aa1e-400f-aedb-78d896c83113\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:39:01 crc kubenswrapper[5052]: I1203 12:39:01.247132 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/02726407-aa1e-400f-aedb-78d896c83113-config-data\") pod \"glance-default-external-api-0\" (UID: \"02726407-aa1e-400f-aedb-78d896c83113\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:39:01 crc kubenswrapper[5052]: I1203 12:39:01.247178 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/02726407-aa1e-400f-aedb-78d896c83113-scripts\") pod \"glance-default-external-api-0\" (UID: \"02726407-aa1e-400f-aedb-78d896c83113\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:39:01 crc kubenswrapper[5052]: I1203 12:39:01.247240 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/5d4a61d7-2d2c-4186-a923-f59636bc0994-config-data\") pod \"glance-default-internal-api-0\" (UID: \"5d4a61d7-2d2c-4186-a923-f59636bc0994\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:39:01 crc kubenswrapper[5052]: I1203 12:39:01.278833 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-api-0" podUID="c74fd61a-bf75-4a68-b802-cdc7f4b48be9" containerName="nova-api-log" probeResult="failure" output="Get \"https://10.217.1.110:8774/\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Dec 03 12:39:01 crc kubenswrapper[5052]: I1203 12:39:01.278930 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-api-0" podUID="c74fd61a-bf75-4a68-b802-cdc7f4b48be9" containerName="nova-api-api" probeResult="failure" output="Get \"https://10.217.1.110:8774/\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Dec 03 12:39:01 crc kubenswrapper[5052]: I1203 12:39:01.349451 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-4mtft\" (UniqueName: \"kubernetes.io/projected/02726407-aa1e-400f-aedb-78d896c83113-kube-api-access-4mtft\") pod \"glance-default-external-api-0\" (UID: \"02726407-aa1e-400f-aedb-78d896c83113\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:39:01 crc kubenswrapper[5052]: I1203 12:39:01.349659 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/02726407-aa1e-400f-aedb-78d896c83113-combined-ca-bundle\") pod \"glance-default-external-api-0\" (UID: \"02726407-aa1e-400f-aedb-78d896c83113\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:39:01 crc kubenswrapper[5052]: I1203 12:39:01.349688 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/5d4a61d7-2d2c-4186-a923-f59636bc0994-httpd-run\") pod \"glance-default-internal-api-0\" (UID: \"5d4a61d7-2d2c-4186-a923-f59636bc0994\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:39:01 crc kubenswrapper[5052]: I1203 12:39:01.349730 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/5d4a61d7-2d2c-4186-a923-f59636bc0994-scripts\") pod \"glance-default-internal-api-0\" (UID: \"5d4a61d7-2d2c-4186-a923-f59636bc0994\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:39:01 crc kubenswrapper[5052]: I1203 12:39:01.349750 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/02726407-aa1e-400f-aedb-78d896c83113-public-tls-certs\") pod \"glance-default-external-api-0\" (UID: \"02726407-aa1e-400f-aedb-78d896c83113\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:39:01 crc kubenswrapper[5052]: I1203 12:39:01.349799 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage18-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage18-crc\") pod \"glance-default-internal-api-0\" (UID: \"5d4a61d7-2d2c-4186-a923-f59636bc0994\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:39:01 crc kubenswrapper[5052]: I1203 12:39:01.349829 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/02726407-aa1e-400f-aedb-78d896c83113-httpd-run\") pod \"glance-default-external-api-0\" (UID: \"02726407-aa1e-400f-aedb-78d896c83113\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:39:01 crc kubenswrapper[5052]: I1203 12:39:01.349873 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/5d4a61d7-2d2c-4186-a923-f59636bc0994-internal-tls-certs\") pod \"glance-default-internal-api-0\" (UID: \"5d4a61d7-2d2c-4186-a923-f59636bc0994\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:39:01 crc kubenswrapper[5052]: I1203 12:39:01.349916 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage12-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage12-crc\") pod \"glance-default-external-api-0\" (UID: \"02726407-aa1e-400f-aedb-78d896c83113\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:39:01 crc kubenswrapper[5052]: I1203 12:39:01.349986 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/02726407-aa1e-400f-aedb-78d896c83113-logs\") pod \"glance-default-external-api-0\" (UID: \"02726407-aa1e-400f-aedb-78d896c83113\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:39:01 crc kubenswrapper[5052]: I1203 12:39:01.350049 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/02726407-aa1e-400f-aedb-78d896c83113-config-data\") pod \"glance-default-external-api-0\" (UID: \"02726407-aa1e-400f-aedb-78d896c83113\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:39:01 crc kubenswrapper[5052]: I1203 12:39:01.350075 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/02726407-aa1e-400f-aedb-78d896c83113-scripts\") pod \"glance-default-external-api-0\" (UID: \"02726407-aa1e-400f-aedb-78d896c83113\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:39:01 crc kubenswrapper[5052]: I1203 12:39:01.350132 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/5d4a61d7-2d2c-4186-a923-f59636bc0994-config-data\") pod \"glance-default-internal-api-0\" (UID: \"5d4a61d7-2d2c-4186-a923-f59636bc0994\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:39:01 crc kubenswrapper[5052]: I1203 12:39:01.350166 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5d4a61d7-2d2c-4186-a923-f59636bc0994-combined-ca-bundle\") pod \"glance-default-internal-api-0\" (UID: \"5d4a61d7-2d2c-4186-a923-f59636bc0994\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:39:01 crc kubenswrapper[5052]: I1203 12:39:01.350207 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/5d4a61d7-2d2c-4186-a923-f59636bc0994-logs\") pod \"glance-default-internal-api-0\" (UID: \"5d4a61d7-2d2c-4186-a923-f59636bc0994\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:39:01 crc kubenswrapper[5052]: I1203 12:39:01.350227 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-mqvdm\" (UniqueName: \"kubernetes.io/projected/5d4a61d7-2d2c-4186-a923-f59636bc0994-kube-api-access-mqvdm\") pod \"glance-default-internal-api-0\" (UID: \"5d4a61d7-2d2c-4186-a923-f59636bc0994\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:39:01 crc kubenswrapper[5052]: I1203 12:39:01.350944 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage12-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage12-crc\") pod \"glance-default-external-api-0\" (UID: \"02726407-aa1e-400f-aedb-78d896c83113\") device mount path \"/mnt/openstack/pv12\"" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:39:01 crc kubenswrapper[5052]: I1203 12:39:01.353859 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage18-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage18-crc\") pod \"glance-default-internal-api-0\" (UID: \"5d4a61d7-2d2c-4186-a923-f59636bc0994\") device mount path \"/mnt/openstack/pv18\"" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:39:01 crc kubenswrapper[5052]: I1203 12:39:01.365857 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/5d4a61d7-2d2c-4186-a923-f59636bc0994-logs\") pod \"glance-default-internal-api-0\" (UID: \"5d4a61d7-2d2c-4186-a923-f59636bc0994\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:39:01 crc kubenswrapper[5052]: I1203 12:39:01.353981 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/02726407-aa1e-400f-aedb-78d896c83113-logs\") pod \"glance-default-external-api-0\" (UID: \"02726407-aa1e-400f-aedb-78d896c83113\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:39:01 crc kubenswrapper[5052]: I1203 12:39:01.368532 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/02726407-aa1e-400f-aedb-78d896c83113-httpd-run\") pod \"glance-default-external-api-0\" (UID: \"02726407-aa1e-400f-aedb-78d896c83113\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:39:01 crc kubenswrapper[5052]: I1203 12:39:01.368698 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/5d4a61d7-2d2c-4186-a923-f59636bc0994-httpd-run\") pod \"glance-default-internal-api-0\" (UID: \"5d4a61d7-2d2c-4186-a923-f59636bc0994\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:39:01 crc kubenswrapper[5052]: I1203 12:39:01.369925 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/02726407-aa1e-400f-aedb-78d896c83113-config-data\") pod \"glance-default-external-api-0\" (UID: \"02726407-aa1e-400f-aedb-78d896c83113\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:39:01 crc kubenswrapper[5052]: I1203 12:39:01.369949 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/5d4a61d7-2d2c-4186-a923-f59636bc0994-internal-tls-certs\") pod \"glance-default-internal-api-0\" (UID: \"5d4a61d7-2d2c-4186-a923-f59636bc0994\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:39:01 crc kubenswrapper[5052]: I1203 12:39:01.370203 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/5d4a61d7-2d2c-4186-a923-f59636bc0994-scripts\") pod \"glance-default-internal-api-0\" (UID: \"5d4a61d7-2d2c-4186-a923-f59636bc0994\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:39:01 crc kubenswrapper[5052]: I1203 12:39:01.371085 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-4mtft\" (UniqueName: \"kubernetes.io/projected/02726407-aa1e-400f-aedb-78d896c83113-kube-api-access-4mtft\") pod \"glance-default-external-api-0\" (UID: \"02726407-aa1e-400f-aedb-78d896c83113\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:39:01 crc kubenswrapper[5052]: I1203 12:39:01.371134 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/02726407-aa1e-400f-aedb-78d896c83113-public-tls-certs\") pod \"glance-default-external-api-0\" (UID: \"02726407-aa1e-400f-aedb-78d896c83113\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:39:01 crc kubenswrapper[5052]: I1203 12:39:01.371645 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/02726407-aa1e-400f-aedb-78d896c83113-scripts\") pod \"glance-default-external-api-0\" (UID: \"02726407-aa1e-400f-aedb-78d896c83113\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:39:01 crc kubenswrapper[5052]: I1203 12:39:01.372023 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/02726407-aa1e-400f-aedb-78d896c83113-combined-ca-bundle\") pod \"glance-default-external-api-0\" (UID: \"02726407-aa1e-400f-aedb-78d896c83113\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:39:01 crc kubenswrapper[5052]: I1203 12:39:01.381065 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5d4a61d7-2d2c-4186-a923-f59636bc0994-combined-ca-bundle\") pod \"glance-default-internal-api-0\" (UID: \"5d4a61d7-2d2c-4186-a923-f59636bc0994\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:39:01 crc kubenswrapper[5052]: I1203 12:39:01.399200 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/5d4a61d7-2d2c-4186-a923-f59636bc0994-config-data\") pod \"glance-default-internal-api-0\" (UID: \"5d4a61d7-2d2c-4186-a923-f59636bc0994\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:39:01 crc kubenswrapper[5052]: I1203 12:39:01.404385 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-mqvdm\" (UniqueName: \"kubernetes.io/projected/5d4a61d7-2d2c-4186-a923-f59636bc0994-kube-api-access-mqvdm\") pod \"glance-default-internal-api-0\" (UID: \"5d4a61d7-2d2c-4186-a923-f59636bc0994\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:39:01 crc kubenswrapper[5052]: I1203 12:39:01.406918 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/openstack-galera-0"] Dec 03 12:39:01 crc kubenswrapper[5052]: W1203 12:39:01.425923 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda7ae5e02_8418_40e2_b65d_7b74625fe5b3.slice/crio-b28f75b111ff37c19eb949a2cf88f6f1e38e525c57756893899aa9bc9fc86996 WatchSource:0}: Error finding container b28f75b111ff37c19eb949a2cf88f6f1e38e525c57756893899aa9bc9fc86996: Status 404 returned error can't find the container with id b28f75b111ff37c19eb949a2cf88f6f1e38e525c57756893899aa9bc9fc86996 Dec 03 12:39:01 crc kubenswrapper[5052]: I1203 12:39:01.437737 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage12-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage12-crc\") pod \"glance-default-external-api-0\" (UID: \"02726407-aa1e-400f-aedb-78d896c83113\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:39:01 crc kubenswrapper[5052]: I1203 12:39:01.463047 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:39:01 crc kubenswrapper[5052]: I1203 12:39:01.473142 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage18-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage18-crc\") pod \"glance-default-internal-api-0\" (UID: \"5d4a61d7-2d2c-4186-a923-f59636bc0994\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:39:01 crc kubenswrapper[5052]: I1203 12:39:01.687714 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="01fb7120-3521-4969-a2c4-b2637e0e2414" path="/var/lib/kubelet/pods/01fb7120-3521-4969-a2c4-b2637e0e2414/volumes" Dec 03 12:39:01 crc kubenswrapper[5052]: I1203 12:39:01.695928 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="42e9c6bc-62d9-442a-a5f0-d885eff829cb" path="/var/lib/kubelet/pods/42e9c6bc-62d9-442a-a5f0-d885eff829cb/volumes" Dec 03 12:39:01 crc kubenswrapper[5052]: I1203 12:39:01.697305 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9dccead0-7ff2-4255-ac00-10a9b357c2db" path="/var/lib/kubelet/pods/9dccead0-7ff2-4255-ac00-10a9b357c2db/volumes" Dec 03 12:39:01 crc kubenswrapper[5052]: I1203 12:39:01.771141 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:39:01 crc kubenswrapper[5052]: I1203 12:39:01.966634 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:39:02 crc kubenswrapper[5052]: I1203 12:39:02.002777 5052 generic.go:334] "Generic (PLEG): container finished" podID="a7ae5e02-8418-40e2-b65d-7b74625fe5b3" containerID="7010c3ab9f848b0d7bcc5baa0f7b66961b913b599dc72e2ae5037fc9fa66ce73" exitCode=0 Dec 03 12:39:02 crc kubenswrapper[5052]: I1203 12:39:02.002866 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-galera-0" event={"ID":"a7ae5e02-8418-40e2-b65d-7b74625fe5b3","Type":"ContainerDied","Data":"7010c3ab9f848b0d7bcc5baa0f7b66961b913b599dc72e2ae5037fc9fa66ce73"} Dec 03 12:39:02 crc kubenswrapper[5052]: I1203 12:39:02.002917 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-galera-0" event={"ID":"a7ae5e02-8418-40e2-b65d-7b74625fe5b3","Type":"ContainerStarted","Data":"b28f75b111ff37c19eb949a2cf88f6f1e38e525c57756893899aa9bc9fc86996"} Dec 03 12:39:02 crc kubenswrapper[5052]: I1203 12:39:02.006590 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"02726407-aa1e-400f-aedb-78d896c83113","Type":"ContainerStarted","Data":"b35d25d646491d5ba914899b4a1958f71c9b4b047d5f01932e519ac36f40ddcf"} Dec 03 12:39:02 crc kubenswrapper[5052]: I1203 12:39:02.032784 5052 generic.go:334] "Generic (PLEG): container finished" podID="68678c3f-bf25-488f-a422-1bf8586703b3" containerID="aece19546c52d53c88212e467860c80e8703242aed2c7ccbcbed127d5a4cd806" exitCode=0 Dec 03 12:39:02 crc kubenswrapper[5052]: I1203 12:39:02.032838 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"68678c3f-bf25-488f-a422-1bf8586703b3","Type":"ContainerDied","Data":"aece19546c52d53c88212e467860c80e8703242aed2c7ccbcbed127d5a4cd806"} Dec 03 12:39:02 crc kubenswrapper[5052]: I1203 12:39:02.224336 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:39:02 crc kubenswrapper[5052]: W1203 12:39:02.235698 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5d4a61d7_2d2c_4186_a923_f59636bc0994.slice/crio-fa4221771479fd3430e53e3dbc74221c74fd7652dcd22d0b3f64f1871afeca86 WatchSource:0}: Error finding container fa4221771479fd3430e53e3dbc74221c74fd7652dcd22d0b3f64f1871afeca86: Status 404 returned error can't find the container with id fa4221771479fd3430e53e3dbc74221c74fd7652dcd22d0b3f64f1871afeca86 Dec 03 12:39:03 crc kubenswrapper[5052]: I1203 12:39:03.046818 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"5d4a61d7-2d2c-4186-a923-f59636bc0994","Type":"ContainerStarted","Data":"fa4221771479fd3430e53e3dbc74221c74fd7652dcd22d0b3f64f1871afeca86"} Dec 03 12:39:03 crc kubenswrapper[5052]: I1203 12:39:03.049100 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-galera-0" event={"ID":"a7ae5e02-8418-40e2-b65d-7b74625fe5b3","Type":"ContainerStarted","Data":"fb98f0e6943bcf2cc568bb5bb0901b5c57179b59a118343bafcea6218d35b382"} Dec 03 12:39:03 crc kubenswrapper[5052]: I1203 12:39:03.051334 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"02726407-aa1e-400f-aedb-78d896c83113","Type":"ContainerStarted","Data":"df1cbd0a5a0599784a42f96943876ae3db85b427e8a37874b50da6af3c45e953"} Dec 03 12:39:03 crc kubenswrapper[5052]: I1203 12:39:03.072559 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/openstack-galera-0" podStartSLOduration=3.072536466 podStartE2EDuration="3.072536466s" podCreationTimestamp="2025-12-03 12:39:00 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:39:03.069295843 +0000 UTC m=+4733.657537617" watchObservedRunningTime="2025-12-03 12:39:03.072536466 +0000 UTC m=+4733.660778230" Dec 03 12:39:03 crc kubenswrapper[5052]: I1203 12:39:03.219147 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/keystone-7b8cddb768-5snf9"] Dec 03 12:39:03 crc kubenswrapper[5052]: I1203 12:39:03.223240 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-7b8cddb768-5snf9" Dec 03 12:39:03 crc kubenswrapper[5052]: I1203 12:39:03.260464 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-7b8cddb768-5snf9"] Dec 03 12:39:03 crc kubenswrapper[5052]: I1203 12:39:03.308043 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:39:03 crc kubenswrapper[5052]: I1203 12:39:03.346193 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/2972cf0c-5378-4e71-b308-0235608d5bef-public-tls-certs\") pod \"keystone-7b8cddb768-5snf9\" (UID: \"2972cf0c-5378-4e71-b308-0235608d5bef\") " pod="openstack-kuttl-tests/keystone-7b8cddb768-5snf9" Dec 03 12:39:03 crc kubenswrapper[5052]: I1203 12:39:03.346751 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/2972cf0c-5378-4e71-b308-0235608d5bef-credential-keys\") pod \"keystone-7b8cddb768-5snf9\" (UID: \"2972cf0c-5378-4e71-b308-0235608d5bef\") " pod="openstack-kuttl-tests/keystone-7b8cddb768-5snf9" Dec 03 12:39:03 crc kubenswrapper[5052]: I1203 12:39:03.346865 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k799r\" (UniqueName: \"kubernetes.io/projected/2972cf0c-5378-4e71-b308-0235608d5bef-kube-api-access-k799r\") pod \"keystone-7b8cddb768-5snf9\" (UID: \"2972cf0c-5378-4e71-b308-0235608d5bef\") " pod="openstack-kuttl-tests/keystone-7b8cddb768-5snf9" Dec 03 12:39:03 crc kubenswrapper[5052]: I1203 12:39:03.346987 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/2972cf0c-5378-4e71-b308-0235608d5bef-scripts\") pod \"keystone-7b8cddb768-5snf9\" (UID: \"2972cf0c-5378-4e71-b308-0235608d5bef\") " pod="openstack-kuttl-tests/keystone-7b8cddb768-5snf9" Dec 03 12:39:03 crc kubenswrapper[5052]: I1203 12:39:03.347186 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/2972cf0c-5378-4e71-b308-0235608d5bef-fernet-keys\") pod \"keystone-7b8cddb768-5snf9\" (UID: \"2972cf0c-5378-4e71-b308-0235608d5bef\") " pod="openstack-kuttl-tests/keystone-7b8cddb768-5snf9" Dec 03 12:39:03 crc kubenswrapper[5052]: I1203 12:39:03.347255 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/2972cf0c-5378-4e71-b308-0235608d5bef-config-data\") pod \"keystone-7b8cddb768-5snf9\" (UID: \"2972cf0c-5378-4e71-b308-0235608d5bef\") " pod="openstack-kuttl-tests/keystone-7b8cddb768-5snf9" Dec 03 12:39:03 crc kubenswrapper[5052]: I1203 12:39:03.347396 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2972cf0c-5378-4e71-b308-0235608d5bef-combined-ca-bundle\") pod \"keystone-7b8cddb768-5snf9\" (UID: \"2972cf0c-5378-4e71-b308-0235608d5bef\") " pod="openstack-kuttl-tests/keystone-7b8cddb768-5snf9" Dec 03 12:39:03 crc kubenswrapper[5052]: I1203 12:39:03.347592 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/2972cf0c-5378-4e71-b308-0235608d5bef-internal-tls-certs\") pod \"keystone-7b8cddb768-5snf9\" (UID: \"2972cf0c-5378-4e71-b308-0235608d5bef\") " pod="openstack-kuttl-tests/keystone-7b8cddb768-5snf9" Dec 03 12:39:03 crc kubenswrapper[5052]: I1203 12:39:03.451900 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/2972cf0c-5378-4e71-b308-0235608d5bef-credential-keys\") pod \"keystone-7b8cddb768-5snf9\" (UID: \"2972cf0c-5378-4e71-b308-0235608d5bef\") " pod="openstack-kuttl-tests/keystone-7b8cddb768-5snf9" Dec 03 12:39:03 crc kubenswrapper[5052]: I1203 12:39:03.451980 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-k799r\" (UniqueName: \"kubernetes.io/projected/2972cf0c-5378-4e71-b308-0235608d5bef-kube-api-access-k799r\") pod \"keystone-7b8cddb768-5snf9\" (UID: \"2972cf0c-5378-4e71-b308-0235608d5bef\") " pod="openstack-kuttl-tests/keystone-7b8cddb768-5snf9" Dec 03 12:39:03 crc kubenswrapper[5052]: I1203 12:39:03.452018 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/2972cf0c-5378-4e71-b308-0235608d5bef-scripts\") pod \"keystone-7b8cddb768-5snf9\" (UID: \"2972cf0c-5378-4e71-b308-0235608d5bef\") " pod="openstack-kuttl-tests/keystone-7b8cddb768-5snf9" Dec 03 12:39:03 crc kubenswrapper[5052]: I1203 12:39:03.452068 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/2972cf0c-5378-4e71-b308-0235608d5bef-fernet-keys\") pod \"keystone-7b8cddb768-5snf9\" (UID: \"2972cf0c-5378-4e71-b308-0235608d5bef\") " pod="openstack-kuttl-tests/keystone-7b8cddb768-5snf9" Dec 03 12:39:03 crc kubenswrapper[5052]: I1203 12:39:03.452105 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/2972cf0c-5378-4e71-b308-0235608d5bef-config-data\") pod \"keystone-7b8cddb768-5snf9\" (UID: \"2972cf0c-5378-4e71-b308-0235608d5bef\") " pod="openstack-kuttl-tests/keystone-7b8cddb768-5snf9" Dec 03 12:39:03 crc kubenswrapper[5052]: I1203 12:39:03.452174 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2972cf0c-5378-4e71-b308-0235608d5bef-combined-ca-bundle\") pod \"keystone-7b8cddb768-5snf9\" (UID: \"2972cf0c-5378-4e71-b308-0235608d5bef\") " pod="openstack-kuttl-tests/keystone-7b8cddb768-5snf9" Dec 03 12:39:03 crc kubenswrapper[5052]: I1203 12:39:03.452204 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/2972cf0c-5378-4e71-b308-0235608d5bef-internal-tls-certs\") pod \"keystone-7b8cddb768-5snf9\" (UID: \"2972cf0c-5378-4e71-b308-0235608d5bef\") " pod="openstack-kuttl-tests/keystone-7b8cddb768-5snf9" Dec 03 12:39:03 crc kubenswrapper[5052]: I1203 12:39:03.452275 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/2972cf0c-5378-4e71-b308-0235608d5bef-public-tls-certs\") pod \"keystone-7b8cddb768-5snf9\" (UID: \"2972cf0c-5378-4e71-b308-0235608d5bef\") " pod="openstack-kuttl-tests/keystone-7b8cddb768-5snf9" Dec 03 12:39:03 crc kubenswrapper[5052]: I1203 12:39:03.459896 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/2972cf0c-5378-4e71-b308-0235608d5bef-public-tls-certs\") pod \"keystone-7b8cddb768-5snf9\" (UID: \"2972cf0c-5378-4e71-b308-0235608d5bef\") " pod="openstack-kuttl-tests/keystone-7b8cddb768-5snf9" Dec 03 12:39:03 crc kubenswrapper[5052]: I1203 12:39:03.461297 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/2972cf0c-5378-4e71-b308-0235608d5bef-config-data\") pod \"keystone-7b8cddb768-5snf9\" (UID: \"2972cf0c-5378-4e71-b308-0235608d5bef\") " pod="openstack-kuttl-tests/keystone-7b8cddb768-5snf9" Dec 03 12:39:03 crc kubenswrapper[5052]: I1203 12:39:03.464853 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2972cf0c-5378-4e71-b308-0235608d5bef-combined-ca-bundle\") pod \"keystone-7b8cddb768-5snf9\" (UID: \"2972cf0c-5378-4e71-b308-0235608d5bef\") " pod="openstack-kuttl-tests/keystone-7b8cddb768-5snf9" Dec 03 12:39:03 crc kubenswrapper[5052]: I1203 12:39:03.470704 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/2972cf0c-5378-4e71-b308-0235608d5bef-fernet-keys\") pod \"keystone-7b8cddb768-5snf9\" (UID: \"2972cf0c-5378-4e71-b308-0235608d5bef\") " pod="openstack-kuttl-tests/keystone-7b8cddb768-5snf9" Dec 03 12:39:03 crc kubenswrapper[5052]: I1203 12:39:03.471797 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/2972cf0c-5378-4e71-b308-0235608d5bef-scripts\") pod \"keystone-7b8cddb768-5snf9\" (UID: \"2972cf0c-5378-4e71-b308-0235608d5bef\") " pod="openstack-kuttl-tests/keystone-7b8cddb768-5snf9" Dec 03 12:39:03 crc kubenswrapper[5052]: I1203 12:39:03.474471 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/2972cf0c-5378-4e71-b308-0235608d5bef-internal-tls-certs\") pod \"keystone-7b8cddb768-5snf9\" (UID: \"2972cf0c-5378-4e71-b308-0235608d5bef\") " pod="openstack-kuttl-tests/keystone-7b8cddb768-5snf9" Dec 03 12:39:03 crc kubenswrapper[5052]: I1203 12:39:03.474490 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-k799r\" (UniqueName: \"kubernetes.io/projected/2972cf0c-5378-4e71-b308-0235608d5bef-kube-api-access-k799r\") pod \"keystone-7b8cddb768-5snf9\" (UID: \"2972cf0c-5378-4e71-b308-0235608d5bef\") " pod="openstack-kuttl-tests/keystone-7b8cddb768-5snf9" Dec 03 12:39:03 crc kubenswrapper[5052]: I1203 12:39:03.476347 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/2972cf0c-5378-4e71-b308-0235608d5bef-credential-keys\") pod \"keystone-7b8cddb768-5snf9\" (UID: \"2972cf0c-5378-4e71-b308-0235608d5bef\") " pod="openstack-kuttl-tests/keystone-7b8cddb768-5snf9" Dec 03 12:39:03 crc kubenswrapper[5052]: I1203 12:39:03.550072 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-7b8cddb768-5snf9" Dec 03 12:39:03 crc kubenswrapper[5052]: I1203 12:39:03.776051 5052 prober.go:107] "Probe failed" probeType="Liveness" pod="openstack-kuttl-tests/cinder-scheduler-0" podUID="d855c655-3b2f-4e4f-9b27-cf4e43a92045" containerName="cinder-scheduler" probeResult="failure" output="HTTP probe failed with statuscode: 500" Dec 03 12:39:03 crc kubenswrapper[5052]: I1203 12:39:03.776143 5052 kubelet.go:2542] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:39:03 crc kubenswrapper[5052]: I1203 12:39:03.777668 5052 kuberuntime_manager.go:1027] "Message for Container of pod" containerName="cinder-scheduler" containerStatusID={"Type":"cri-o","ID":"c25218175c330c36ee6dc83936ebcb6c9c017af62ee188f8b82f5fd416a29e9c"} pod="openstack-kuttl-tests/cinder-scheduler-0" containerMessage="Container cinder-scheduler failed liveness probe, will be restarted" Dec 03 12:39:03 crc kubenswrapper[5052]: I1203 12:39:03.777732 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/cinder-scheduler-0" podUID="d855c655-3b2f-4e4f-9b27-cf4e43a92045" containerName="cinder-scheduler" containerID="cri-o://c25218175c330c36ee6dc83936ebcb6c9c017af62ee188f8b82f5fd416a29e9c" gracePeriod=30 Dec 03 12:39:04 crc kubenswrapper[5052]: I1203 12:39:04.042654 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-7b8cddb768-5snf9"] Dec 03 12:39:04 crc kubenswrapper[5052]: I1203 12:39:04.068055 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"02726407-aa1e-400f-aedb-78d896c83113","Type":"ContainerStarted","Data":"4407c34f059e21949a882b9b730e10be07f5e7b9fb9786cf9259647fe69f7a12"} Dec 03 12:39:04 crc kubenswrapper[5052]: I1203 12:39:04.072482 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"5d4a61d7-2d2c-4186-a923-f59636bc0994","Type":"ContainerStarted","Data":"2981d01a1325fdf3537f8cf2d9ade2af4707530bbb2ce6666c3b3503c416127d"} Dec 03 12:39:04 crc kubenswrapper[5052]: I1203 12:39:04.073647 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"5d4a61d7-2d2c-4186-a923-f59636bc0994","Type":"ContainerStarted","Data":"7ebd36be6c9083bc844d5b32e9c85265741962dbf955670e0381dd41fb13e4d3"} Dec 03 12:39:04 crc kubenswrapper[5052]: I1203 12:39:04.103101 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/glance-default-external-api-0" podStartSLOduration=3.103082198 podStartE2EDuration="3.103082198s" podCreationTimestamp="2025-12-03 12:39:01 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:39:04.087098251 +0000 UTC m=+4734.675340035" watchObservedRunningTime="2025-12-03 12:39:04.103082198 +0000 UTC m=+4734.691323952" Dec 03 12:39:04 crc kubenswrapper[5052]: I1203 12:39:04.115849 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/glance-default-internal-api-0" podStartSLOduration=3.115827753 podStartE2EDuration="3.115827753s" podCreationTimestamp="2025-12-03 12:39:01 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:39:04.109492431 +0000 UTC m=+4734.697734205" watchObservedRunningTime="2025-12-03 12:39:04.115827753 +0000 UTC m=+4734.704069537" Dec 03 12:39:05 crc kubenswrapper[5052]: I1203 12:39:05.084866 5052 generic.go:334] "Generic (PLEG): container finished" podID="68678c3f-bf25-488f-a422-1bf8586703b3" containerID="75aabd113fe3f8c31645077c72c336a2d111ebe7269663fa192668bdab05f74c" exitCode=0 Dec 03 12:39:05 crc kubenswrapper[5052]: I1203 12:39:05.085050 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"68678c3f-bf25-488f-a422-1bf8586703b3","Type":"ContainerDied","Data":"75aabd113fe3f8c31645077c72c336a2d111ebe7269663fa192668bdab05f74c"} Dec 03 12:39:05 crc kubenswrapper[5052]: I1203 12:39:05.085720 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"68678c3f-bf25-488f-a422-1bf8586703b3","Type":"ContainerDied","Data":"1ca98836ce5312c9ad7a59adc1b315a3e898632296c8e075ea169e1daa505732"} Dec 03 12:39:05 crc kubenswrapper[5052]: I1203 12:39:05.085736 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1ca98836ce5312c9ad7a59adc1b315a3e898632296c8e075ea169e1daa505732" Dec 03 12:39:05 crc kubenswrapper[5052]: I1203 12:39:05.087546 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-7b8cddb768-5snf9" event={"ID":"2972cf0c-5378-4e71-b308-0235608d5bef","Type":"ContainerStarted","Data":"1e9c082372288c89143c066dc4a3ffdcb4a743df94ac5b3327dcb6b74a2be2a7"} Dec 03 12:39:05 crc kubenswrapper[5052]: I1203 12:39:05.087628 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-7b8cddb768-5snf9" event={"ID":"2972cf0c-5378-4e71-b308-0235608d5bef","Type":"ContainerStarted","Data":"9e8c8cfe7e17bc4dae78f6e83c14daf6847b88ce114876f007358139a98604c8"} Dec 03 12:39:05 crc kubenswrapper[5052]: I1203 12:39:05.110883 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/keystone-7b8cddb768-5snf9" podStartSLOduration=2.110865878 podStartE2EDuration="2.110865878s" podCreationTimestamp="2025-12-03 12:39:03 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:39:05.105059452 +0000 UTC m=+4735.693301216" watchObservedRunningTime="2025-12-03 12:39:05.110865878 +0000 UTC m=+4735.699107632" Dec 03 12:39:05 crc kubenswrapper[5052]: I1203 12:39:05.120729 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:39:05 crc kubenswrapper[5052]: I1203 12:39:05.192445 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/68678c3f-bf25-488f-a422-1bf8586703b3-run-httpd\") pod \"68678c3f-bf25-488f-a422-1bf8586703b3\" (UID: \"68678c3f-bf25-488f-a422-1bf8586703b3\") " Dec 03 12:39:05 crc kubenswrapper[5052]: I1203 12:39:05.192551 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/68678c3f-bf25-488f-a422-1bf8586703b3-combined-ca-bundle\") pod \"68678c3f-bf25-488f-a422-1bf8586703b3\" (UID: \"68678c3f-bf25-488f-a422-1bf8586703b3\") " Dec 03 12:39:05 crc kubenswrapper[5052]: I1203 12:39:05.192655 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/68678c3f-bf25-488f-a422-1bf8586703b3-ceilometer-tls-certs\") pod \"68678c3f-bf25-488f-a422-1bf8586703b3\" (UID: \"68678c3f-bf25-488f-a422-1bf8586703b3\") " Dec 03 12:39:05 crc kubenswrapper[5052]: I1203 12:39:05.192730 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/68678c3f-bf25-488f-a422-1bf8586703b3-scripts\") pod \"68678c3f-bf25-488f-a422-1bf8586703b3\" (UID: \"68678c3f-bf25-488f-a422-1bf8586703b3\") " Dec 03 12:39:05 crc kubenswrapper[5052]: I1203 12:39:05.192776 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jdvlz\" (UniqueName: \"kubernetes.io/projected/68678c3f-bf25-488f-a422-1bf8586703b3-kube-api-access-jdvlz\") pod \"68678c3f-bf25-488f-a422-1bf8586703b3\" (UID: \"68678c3f-bf25-488f-a422-1bf8586703b3\") " Dec 03 12:39:05 crc kubenswrapper[5052]: I1203 12:39:05.192814 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/68678c3f-bf25-488f-a422-1bf8586703b3-log-httpd\") pod \"68678c3f-bf25-488f-a422-1bf8586703b3\" (UID: \"68678c3f-bf25-488f-a422-1bf8586703b3\") " Dec 03 12:39:05 crc kubenswrapper[5052]: I1203 12:39:05.192821 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/68678c3f-bf25-488f-a422-1bf8586703b3-run-httpd" (OuterVolumeSpecName: "run-httpd") pod "68678c3f-bf25-488f-a422-1bf8586703b3" (UID: "68678c3f-bf25-488f-a422-1bf8586703b3"). InnerVolumeSpecName "run-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:39:05 crc kubenswrapper[5052]: I1203 12:39:05.192855 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/68678c3f-bf25-488f-a422-1bf8586703b3-config-data\") pod \"68678c3f-bf25-488f-a422-1bf8586703b3\" (UID: \"68678c3f-bf25-488f-a422-1bf8586703b3\") " Dec 03 12:39:05 crc kubenswrapper[5052]: I1203 12:39:05.192955 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/68678c3f-bf25-488f-a422-1bf8586703b3-sg-core-conf-yaml\") pod \"68678c3f-bf25-488f-a422-1bf8586703b3\" (UID: \"68678c3f-bf25-488f-a422-1bf8586703b3\") " Dec 03 12:39:05 crc kubenswrapper[5052]: I1203 12:39:05.193692 5052 reconciler_common.go:293] "Volume detached for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/68678c3f-bf25-488f-a422-1bf8586703b3-run-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:39:05 crc kubenswrapper[5052]: I1203 12:39:05.196831 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/68678c3f-bf25-488f-a422-1bf8586703b3-log-httpd" (OuterVolumeSpecName: "log-httpd") pod "68678c3f-bf25-488f-a422-1bf8586703b3" (UID: "68678c3f-bf25-488f-a422-1bf8586703b3"). InnerVolumeSpecName "log-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:39:05 crc kubenswrapper[5052]: I1203 12:39:05.206507 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/68678c3f-bf25-488f-a422-1bf8586703b3-scripts" (OuterVolumeSpecName: "scripts") pod "68678c3f-bf25-488f-a422-1bf8586703b3" (UID: "68678c3f-bf25-488f-a422-1bf8586703b3"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:39:05 crc kubenswrapper[5052]: I1203 12:39:05.224704 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/68678c3f-bf25-488f-a422-1bf8586703b3-kube-api-access-jdvlz" (OuterVolumeSpecName: "kube-api-access-jdvlz") pod "68678c3f-bf25-488f-a422-1bf8586703b3" (UID: "68678c3f-bf25-488f-a422-1bf8586703b3"). InnerVolumeSpecName "kube-api-access-jdvlz". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:39:05 crc kubenswrapper[5052]: I1203 12:39:05.236741 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/68678c3f-bf25-488f-a422-1bf8586703b3-sg-core-conf-yaml" (OuterVolumeSpecName: "sg-core-conf-yaml") pod "68678c3f-bf25-488f-a422-1bf8586703b3" (UID: "68678c3f-bf25-488f-a422-1bf8586703b3"). InnerVolumeSpecName "sg-core-conf-yaml". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:39:05 crc kubenswrapper[5052]: I1203 12:39:05.281723 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/68678c3f-bf25-488f-a422-1bf8586703b3-ceilometer-tls-certs" (OuterVolumeSpecName: "ceilometer-tls-certs") pod "68678c3f-bf25-488f-a422-1bf8586703b3" (UID: "68678c3f-bf25-488f-a422-1bf8586703b3"). InnerVolumeSpecName "ceilometer-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:39:05 crc kubenswrapper[5052]: I1203 12:39:05.297149 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/68678c3f-bf25-488f-a422-1bf8586703b3-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:39:05 crc kubenswrapper[5052]: I1203 12:39:05.297383 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-jdvlz\" (UniqueName: \"kubernetes.io/projected/68678c3f-bf25-488f-a422-1bf8586703b3-kube-api-access-jdvlz\") on node \"crc\" DevicePath \"\"" Dec 03 12:39:05 crc kubenswrapper[5052]: I1203 12:39:05.297443 5052 reconciler_common.go:293] "Volume detached for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/68678c3f-bf25-488f-a422-1bf8586703b3-log-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:39:05 crc kubenswrapper[5052]: I1203 12:39:05.297495 5052 reconciler_common.go:293] "Volume detached for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/68678c3f-bf25-488f-a422-1bf8586703b3-sg-core-conf-yaml\") on node \"crc\" DevicePath \"\"" Dec 03 12:39:05 crc kubenswrapper[5052]: I1203 12:39:05.297575 5052 reconciler_common.go:293] "Volume detached for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/68678c3f-bf25-488f-a422-1bf8586703b3-ceilometer-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:39:05 crc kubenswrapper[5052]: I1203 12:39:05.306288 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/68678c3f-bf25-488f-a422-1bf8586703b3-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "68678c3f-bf25-488f-a422-1bf8586703b3" (UID: "68678c3f-bf25-488f-a422-1bf8586703b3"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:39:05 crc kubenswrapper[5052]: I1203 12:39:05.330561 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/68678c3f-bf25-488f-a422-1bf8586703b3-config-data" (OuterVolumeSpecName: "config-data") pod "68678c3f-bf25-488f-a422-1bf8586703b3" (UID: "68678c3f-bf25-488f-a422-1bf8586703b3"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:39:05 crc kubenswrapper[5052]: I1203 12:39:05.399792 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/68678c3f-bf25-488f-a422-1bf8586703b3-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:39:05 crc kubenswrapper[5052]: I1203 12:39:05.400021 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/68678c3f-bf25-488f-a422-1bf8586703b3-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:39:06 crc kubenswrapper[5052]: I1203 12:39:06.097696 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:39:06 crc kubenswrapper[5052]: I1203 12:39:06.098698 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/keystone-7b8cddb768-5snf9" Dec 03 12:39:06 crc kubenswrapper[5052]: I1203 12:39:06.119731 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:39:06 crc kubenswrapper[5052]: I1203 12:39:06.129648 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:39:06 crc kubenswrapper[5052]: I1203 12:39:06.148032 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:39:06 crc kubenswrapper[5052]: E1203 12:39:06.148503 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="68678c3f-bf25-488f-a422-1bf8586703b3" containerName="sg-core" Dec 03 12:39:06 crc kubenswrapper[5052]: I1203 12:39:06.148536 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="68678c3f-bf25-488f-a422-1bf8586703b3" containerName="sg-core" Dec 03 12:39:06 crc kubenswrapper[5052]: E1203 12:39:06.148549 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="68678c3f-bf25-488f-a422-1bf8586703b3" containerName="ceilometer-central-agent" Dec 03 12:39:06 crc kubenswrapper[5052]: I1203 12:39:06.148557 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="68678c3f-bf25-488f-a422-1bf8586703b3" containerName="ceilometer-central-agent" Dec 03 12:39:06 crc kubenswrapper[5052]: E1203 12:39:06.148579 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="68678c3f-bf25-488f-a422-1bf8586703b3" containerName="proxy-httpd" Dec 03 12:39:06 crc kubenswrapper[5052]: I1203 12:39:06.148585 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="68678c3f-bf25-488f-a422-1bf8586703b3" containerName="proxy-httpd" Dec 03 12:39:06 crc kubenswrapper[5052]: E1203 12:39:06.148596 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="68678c3f-bf25-488f-a422-1bf8586703b3" containerName="ceilometer-notification-agent" Dec 03 12:39:06 crc kubenswrapper[5052]: I1203 12:39:06.148601 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="68678c3f-bf25-488f-a422-1bf8586703b3" containerName="ceilometer-notification-agent" Dec 03 12:39:06 crc kubenswrapper[5052]: I1203 12:39:06.148793 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="68678c3f-bf25-488f-a422-1bf8586703b3" containerName="proxy-httpd" Dec 03 12:39:06 crc kubenswrapper[5052]: I1203 12:39:06.148810 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="68678c3f-bf25-488f-a422-1bf8586703b3" containerName="sg-core" Dec 03 12:39:06 crc kubenswrapper[5052]: I1203 12:39:06.148822 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="68678c3f-bf25-488f-a422-1bf8586703b3" containerName="ceilometer-notification-agent" Dec 03 12:39:06 crc kubenswrapper[5052]: I1203 12:39:06.148834 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="68678c3f-bf25-488f-a422-1bf8586703b3" containerName="ceilometer-central-agent" Dec 03 12:39:06 crc kubenswrapper[5052]: I1203 12:39:06.151037 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:39:06 crc kubenswrapper[5052]: I1203 12:39:06.153265 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-ceilometer-internal-svc" Dec 03 12:39:06 crc kubenswrapper[5052]: I1203 12:39:06.153774 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-scripts" Dec 03 12:39:06 crc kubenswrapper[5052]: I1203 12:39:06.154178 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-config-data" Dec 03 12:39:06 crc kubenswrapper[5052]: I1203 12:39:06.163144 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:39:06 crc kubenswrapper[5052]: I1203 12:39:06.216166 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/59dcc834-7abb-4dff-a10f-6127276698ee-ceilometer-tls-certs\") pod \"ceilometer-0\" (UID: \"59dcc834-7abb-4dff-a10f-6127276698ee\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:39:06 crc kubenswrapper[5052]: I1203 12:39:06.216207 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/59dcc834-7abb-4dff-a10f-6127276698ee-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"59dcc834-7abb-4dff-a10f-6127276698ee\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:39:06 crc kubenswrapper[5052]: I1203 12:39:06.216249 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/59dcc834-7abb-4dff-a10f-6127276698ee-scripts\") pod \"ceilometer-0\" (UID: \"59dcc834-7abb-4dff-a10f-6127276698ee\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:39:06 crc kubenswrapper[5052]: I1203 12:39:06.216274 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/59dcc834-7abb-4dff-a10f-6127276698ee-run-httpd\") pod \"ceilometer-0\" (UID: \"59dcc834-7abb-4dff-a10f-6127276698ee\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:39:06 crc kubenswrapper[5052]: I1203 12:39:06.216295 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/59dcc834-7abb-4dff-a10f-6127276698ee-config-data\") pod \"ceilometer-0\" (UID: \"59dcc834-7abb-4dff-a10f-6127276698ee\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:39:06 crc kubenswrapper[5052]: I1203 12:39:06.216337 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/59dcc834-7abb-4dff-a10f-6127276698ee-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"59dcc834-7abb-4dff-a10f-6127276698ee\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:39:06 crc kubenswrapper[5052]: I1203 12:39:06.216635 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/59dcc834-7abb-4dff-a10f-6127276698ee-log-httpd\") pod \"ceilometer-0\" (UID: \"59dcc834-7abb-4dff-a10f-6127276698ee\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:39:06 crc kubenswrapper[5052]: I1203 12:39:06.217204 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2n9rs\" (UniqueName: \"kubernetes.io/projected/59dcc834-7abb-4dff-a10f-6127276698ee-kube-api-access-2n9rs\") pod \"ceilometer-0\" (UID: \"59dcc834-7abb-4dff-a10f-6127276698ee\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:39:06 crc kubenswrapper[5052]: I1203 12:39:06.318816 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-2n9rs\" (UniqueName: \"kubernetes.io/projected/59dcc834-7abb-4dff-a10f-6127276698ee-kube-api-access-2n9rs\") pod \"ceilometer-0\" (UID: \"59dcc834-7abb-4dff-a10f-6127276698ee\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:39:06 crc kubenswrapper[5052]: I1203 12:39:06.318897 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/59dcc834-7abb-4dff-a10f-6127276698ee-ceilometer-tls-certs\") pod \"ceilometer-0\" (UID: \"59dcc834-7abb-4dff-a10f-6127276698ee\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:39:06 crc kubenswrapper[5052]: I1203 12:39:06.318922 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/59dcc834-7abb-4dff-a10f-6127276698ee-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"59dcc834-7abb-4dff-a10f-6127276698ee\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:39:06 crc kubenswrapper[5052]: I1203 12:39:06.318963 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/59dcc834-7abb-4dff-a10f-6127276698ee-scripts\") pod \"ceilometer-0\" (UID: \"59dcc834-7abb-4dff-a10f-6127276698ee\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:39:06 crc kubenswrapper[5052]: I1203 12:39:06.318994 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/59dcc834-7abb-4dff-a10f-6127276698ee-run-httpd\") pod \"ceilometer-0\" (UID: \"59dcc834-7abb-4dff-a10f-6127276698ee\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:39:06 crc kubenswrapper[5052]: I1203 12:39:06.319019 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/59dcc834-7abb-4dff-a10f-6127276698ee-config-data\") pod \"ceilometer-0\" (UID: \"59dcc834-7abb-4dff-a10f-6127276698ee\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:39:06 crc kubenswrapper[5052]: I1203 12:39:06.319060 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/59dcc834-7abb-4dff-a10f-6127276698ee-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"59dcc834-7abb-4dff-a10f-6127276698ee\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:39:06 crc kubenswrapper[5052]: I1203 12:39:06.319078 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/59dcc834-7abb-4dff-a10f-6127276698ee-log-httpd\") pod \"ceilometer-0\" (UID: \"59dcc834-7abb-4dff-a10f-6127276698ee\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:39:06 crc kubenswrapper[5052]: I1203 12:39:06.319626 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/59dcc834-7abb-4dff-a10f-6127276698ee-log-httpd\") pod \"ceilometer-0\" (UID: \"59dcc834-7abb-4dff-a10f-6127276698ee\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:39:06 crc kubenswrapper[5052]: I1203 12:39:06.319768 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/59dcc834-7abb-4dff-a10f-6127276698ee-run-httpd\") pod \"ceilometer-0\" (UID: \"59dcc834-7abb-4dff-a10f-6127276698ee\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:39:06 crc kubenswrapper[5052]: I1203 12:39:06.323752 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/59dcc834-7abb-4dff-a10f-6127276698ee-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"59dcc834-7abb-4dff-a10f-6127276698ee\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:39:06 crc kubenswrapper[5052]: I1203 12:39:06.323855 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/59dcc834-7abb-4dff-a10f-6127276698ee-ceilometer-tls-certs\") pod \"ceilometer-0\" (UID: \"59dcc834-7abb-4dff-a10f-6127276698ee\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:39:06 crc kubenswrapper[5052]: I1203 12:39:06.323881 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/59dcc834-7abb-4dff-a10f-6127276698ee-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"59dcc834-7abb-4dff-a10f-6127276698ee\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:39:06 crc kubenswrapper[5052]: I1203 12:39:06.324839 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/59dcc834-7abb-4dff-a10f-6127276698ee-scripts\") pod \"ceilometer-0\" (UID: \"59dcc834-7abb-4dff-a10f-6127276698ee\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:39:06 crc kubenswrapper[5052]: I1203 12:39:06.324857 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/59dcc834-7abb-4dff-a10f-6127276698ee-config-data\") pod \"ceilometer-0\" (UID: \"59dcc834-7abb-4dff-a10f-6127276698ee\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:39:06 crc kubenswrapper[5052]: I1203 12:39:06.338332 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-2n9rs\" (UniqueName: \"kubernetes.io/projected/59dcc834-7abb-4dff-a10f-6127276698ee-kube-api-access-2n9rs\") pod \"ceilometer-0\" (UID: \"59dcc834-7abb-4dff-a10f-6127276698ee\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:39:06 crc kubenswrapper[5052]: I1203 12:39:06.468593 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:39:06 crc kubenswrapper[5052]: I1203 12:39:06.905358 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:39:06 crc kubenswrapper[5052]: W1203 12:39:06.907638 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod59dcc834_7abb_4dff_a10f_6127276698ee.slice/crio-0ef98e7e01f4e7dd908ead07500cc88e4cfb923646967b80612c75f59e6f532d WatchSource:0}: Error finding container 0ef98e7e01f4e7dd908ead07500cc88e4cfb923646967b80612c75f59e6f532d: Status 404 returned error can't find the container with id 0ef98e7e01f4e7dd908ead07500cc88e4cfb923646967b80612c75f59e6f532d Dec 03 12:39:06 crc kubenswrapper[5052]: I1203 12:39:06.947666 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-metadata-0" podUID="edea8e53-340d-4f1e-99e5-082268c64ceb" containerName="nova-metadata-log" probeResult="failure" output="Get \"https://10.217.1.100:8775/\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Dec 03 12:39:06 crc kubenswrapper[5052]: I1203 12:39:06.947721 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-metadata-0" podUID="edea8e53-340d-4f1e-99e5-082268c64ceb" containerName="nova-metadata-metadata" probeResult="failure" output="Get \"https://10.217.1.100:8775/\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Dec 03 12:39:07 crc kubenswrapper[5052]: I1203 12:39:07.109169 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"59dcc834-7abb-4dff-a10f-6127276698ee","Type":"ContainerStarted","Data":"0ef98e7e01f4e7dd908ead07500cc88e4cfb923646967b80612c75f59e6f532d"} Dec 03 12:39:07 crc kubenswrapper[5052]: I1203 12:39:07.685827 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="68678c3f-bf25-488f-a422-1bf8586703b3" path="/var/lib/kubelet/pods/68678c3f-bf25-488f-a422-1bf8586703b3/volumes" Dec 03 12:39:08 crc kubenswrapper[5052]: I1203 12:39:08.120865 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"59dcc834-7abb-4dff-a10f-6127276698ee","Type":"ContainerStarted","Data":"1537061d8a4905c4818df97a435c7501964181424870fcee0c47aed528717497"} Dec 03 12:39:09 crc kubenswrapper[5052]: I1203 12:39:09.133022 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"59dcc834-7abb-4dff-a10f-6127276698ee","Type":"ContainerStarted","Data":"fab91a75f95875869bda13fb508f1578ec0f067822bbe9b4d97201b302a40035"} Dec 03 12:39:09 crc kubenswrapper[5052]: I1203 12:39:09.561788 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/memcached-0"] Dec 03 12:39:09 crc kubenswrapper[5052]: I1203 12:39:09.562379 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/memcached-0" podUID="3a55dfe2-de85-40db-84f5-78404cd4a72b" containerName="memcached" containerID="cri-o://46967c848876329b0210e5ac3f401315001310250adc99a9447969c57cb0bc34" gracePeriod=30 Dec 03 12:39:10 crc kubenswrapper[5052]: I1203 12:39:10.151818 5052 generic.go:334] "Generic (PLEG): container finished" podID="d855c655-3b2f-4e4f-9b27-cf4e43a92045" containerID="c25218175c330c36ee6dc83936ebcb6c9c017af62ee188f8b82f5fd416a29e9c" exitCode=0 Dec 03 12:39:10 crc kubenswrapper[5052]: I1203 12:39:10.152143 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"d855c655-3b2f-4e4f-9b27-cf4e43a92045","Type":"ContainerDied","Data":"c25218175c330c36ee6dc83936ebcb6c9c017af62ee188f8b82f5fd416a29e9c"} Dec 03 12:39:10 crc kubenswrapper[5052]: I1203 12:39:10.154996 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"59dcc834-7abb-4dff-a10f-6127276698ee","Type":"ContainerStarted","Data":"8694cf282fe33e5849ce697fe875d63a8f6e6daaa6c0de9a9adcdfe4163c63d3"} Dec 03 12:39:11 crc kubenswrapper[5052]: I1203 12:39:11.062831 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:39:11 crc kubenswrapper[5052]: I1203 12:39:11.062923 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:39:11 crc kubenswrapper[5052]: I1203 12:39:11.173912 5052 generic.go:334] "Generic (PLEG): container finished" podID="3a55dfe2-de85-40db-84f5-78404cd4a72b" containerID="46967c848876329b0210e5ac3f401315001310250adc99a9447969c57cb0bc34" exitCode=0 Dec 03 12:39:11 crc kubenswrapper[5052]: I1203 12:39:11.173995 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/memcached-0" event={"ID":"3a55dfe2-de85-40db-84f5-78404cd4a72b","Type":"ContainerDied","Data":"46967c848876329b0210e5ac3f401315001310250adc99a9447969c57cb0bc34"} Dec 03 12:39:11 crc kubenswrapper[5052]: I1203 12:39:11.177404 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"59dcc834-7abb-4dff-a10f-6127276698ee","Type":"ContainerStarted","Data":"be761ab2be8f4c31fcd16de111710f2bfb1713804672136a770634cfde4d0e6b"} Dec 03 12:39:11 crc kubenswrapper[5052]: I1203 12:39:11.177681 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:39:11 crc kubenswrapper[5052]: I1203 12:39:11.182893 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:39:11 crc kubenswrapper[5052]: I1203 12:39:11.217924 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/ceilometer-0" podStartSLOduration=1.540868092 podStartE2EDuration="5.217905529s" podCreationTimestamp="2025-12-03 12:39:06 +0000 UTC" firstStartedPulling="2025-12-03 12:39:06.909728003 +0000 UTC m=+4737.497969757" lastFinishedPulling="2025-12-03 12:39:10.58676543 +0000 UTC m=+4741.175007194" observedRunningTime="2025-12-03 12:39:11.20706916 +0000 UTC m=+4741.795310934" watchObservedRunningTime="2025-12-03 12:39:11.217905529 +0000 UTC m=+4741.806147293" Dec 03 12:39:11 crc kubenswrapper[5052]: I1203 12:39:11.256674 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-api-0" podUID="c74fd61a-bf75-4a68-b802-cdc7f4b48be9" containerName="nova-api-log" probeResult="failure" output="Get \"https://10.217.1.110:8774/\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Dec 03 12:39:11 crc kubenswrapper[5052]: I1203 12:39:11.256787 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-api-0" podUID="c74fd61a-bf75-4a68-b802-cdc7f4b48be9" containerName="nova-api-api" probeResult="failure" output="Get \"https://10.217.1.110:8774/\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Dec 03 12:39:11 crc kubenswrapper[5052]: I1203 12:39:11.293793 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:39:11 crc kubenswrapper[5052]: I1203 12:39:11.464134 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:39:11 crc kubenswrapper[5052]: I1203 12:39:11.464472 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:39:11 crc kubenswrapper[5052]: I1203 12:39:11.500532 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:39:11 crc kubenswrapper[5052]: I1203 12:39:11.500739 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/memcached-0" Dec 03 12:39:11 crc kubenswrapper[5052]: I1203 12:39:11.515473 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:39:11 crc kubenswrapper[5052]: I1203 12:39:11.522501 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/3a55dfe2-de85-40db-84f5-78404cd4a72b-kolla-config\") pod \"3a55dfe2-de85-40db-84f5-78404cd4a72b\" (UID: \"3a55dfe2-de85-40db-84f5-78404cd4a72b\") " Dec 03 12:39:11 crc kubenswrapper[5052]: I1203 12:39:11.522549 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/3a55dfe2-de85-40db-84f5-78404cd4a72b-config-data\") pod \"3a55dfe2-de85-40db-84f5-78404cd4a72b\" (UID: \"3a55dfe2-de85-40db-84f5-78404cd4a72b\") " Dec 03 12:39:11 crc kubenswrapper[5052]: I1203 12:39:11.522585 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r5fsw\" (UniqueName: \"kubernetes.io/projected/3a55dfe2-de85-40db-84f5-78404cd4a72b-kube-api-access-r5fsw\") pod \"3a55dfe2-de85-40db-84f5-78404cd4a72b\" (UID: \"3a55dfe2-de85-40db-84f5-78404cd4a72b\") " Dec 03 12:39:11 crc kubenswrapper[5052]: I1203 12:39:11.522829 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3a55dfe2-de85-40db-84f5-78404cd4a72b-combined-ca-bundle\") pod \"3a55dfe2-de85-40db-84f5-78404cd4a72b\" (UID: \"3a55dfe2-de85-40db-84f5-78404cd4a72b\") " Dec 03 12:39:11 crc kubenswrapper[5052]: I1203 12:39:11.522893 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"memcached-tls-certs\" (UniqueName: \"kubernetes.io/secret/3a55dfe2-de85-40db-84f5-78404cd4a72b-memcached-tls-certs\") pod \"3a55dfe2-de85-40db-84f5-78404cd4a72b\" (UID: \"3a55dfe2-de85-40db-84f5-78404cd4a72b\") " Dec 03 12:39:11 crc kubenswrapper[5052]: I1203 12:39:11.523979 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3a55dfe2-de85-40db-84f5-78404cd4a72b-kolla-config" (OuterVolumeSpecName: "kolla-config") pod "3a55dfe2-de85-40db-84f5-78404cd4a72b" (UID: "3a55dfe2-de85-40db-84f5-78404cd4a72b"). InnerVolumeSpecName "kolla-config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:39:11 crc kubenswrapper[5052]: I1203 12:39:11.524147 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3a55dfe2-de85-40db-84f5-78404cd4a72b-config-data" (OuterVolumeSpecName: "config-data") pod "3a55dfe2-de85-40db-84f5-78404cd4a72b" (UID: "3a55dfe2-de85-40db-84f5-78404cd4a72b"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:39:11 crc kubenswrapper[5052]: I1203 12:39:11.527888 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3a55dfe2-de85-40db-84f5-78404cd4a72b-kube-api-access-r5fsw" (OuterVolumeSpecName: "kube-api-access-r5fsw") pod "3a55dfe2-de85-40db-84f5-78404cd4a72b" (UID: "3a55dfe2-de85-40db-84f5-78404cd4a72b"). InnerVolumeSpecName "kube-api-access-r5fsw". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:39:11 crc kubenswrapper[5052]: I1203 12:39:11.611849 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3a55dfe2-de85-40db-84f5-78404cd4a72b-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "3a55dfe2-de85-40db-84f5-78404cd4a72b" (UID: "3a55dfe2-de85-40db-84f5-78404cd4a72b"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:39:11 crc kubenswrapper[5052]: I1203 12:39:11.626561 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3a55dfe2-de85-40db-84f5-78404cd4a72b-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:39:11 crc kubenswrapper[5052]: I1203 12:39:11.626596 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/3a55dfe2-de85-40db-84f5-78404cd4a72b-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:39:11 crc kubenswrapper[5052]: I1203 12:39:11.626605 5052 reconciler_common.go:293] "Volume detached for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/3a55dfe2-de85-40db-84f5-78404cd4a72b-kolla-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:39:11 crc kubenswrapper[5052]: I1203 12:39:11.626614 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-r5fsw\" (UniqueName: \"kubernetes.io/projected/3a55dfe2-de85-40db-84f5-78404cd4a72b-kube-api-access-r5fsw\") on node \"crc\" DevicePath \"\"" Dec 03 12:39:11 crc kubenswrapper[5052]: I1203 12:39:11.646730 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3a55dfe2-de85-40db-84f5-78404cd4a72b-memcached-tls-certs" (OuterVolumeSpecName: "memcached-tls-certs") pod "3a55dfe2-de85-40db-84f5-78404cd4a72b" (UID: "3a55dfe2-de85-40db-84f5-78404cd4a72b"). InnerVolumeSpecName "memcached-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:39:11 crc kubenswrapper[5052]: I1203 12:39:11.730745 5052 reconciler_common.go:293] "Volume detached for volume \"memcached-tls-certs\" (UniqueName: \"kubernetes.io/secret/3a55dfe2-de85-40db-84f5-78404cd4a72b-memcached-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:39:11 crc kubenswrapper[5052]: I1203 12:39:11.772180 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:39:11 crc kubenswrapper[5052]: I1203 12:39:11.772233 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:39:11 crc kubenswrapper[5052]: I1203 12:39:11.811071 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:39:11 crc kubenswrapper[5052]: I1203 12:39:11.828832 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:39:12 crc kubenswrapper[5052]: I1203 12:39:12.191564 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"d855c655-3b2f-4e4f-9b27-cf4e43a92045","Type":"ContainerStarted","Data":"84ca5b96b0f3458adcf7e677440d6f30818b02e536633792be662d5c24cf8b44"} Dec 03 12:39:12 crc kubenswrapper[5052]: I1203 12:39:12.193681 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/memcached-0" event={"ID":"3a55dfe2-de85-40db-84f5-78404cd4a72b","Type":"ContainerDied","Data":"5e0b929315e5735100ddbc38bb312ce05f6d6479429a8694e01de3688e14e9bb"} Dec 03 12:39:12 crc kubenswrapper[5052]: I1203 12:39:12.193725 5052 scope.go:117] "RemoveContainer" containerID="46967c848876329b0210e5ac3f401315001310250adc99a9447969c57cb0bc34" Dec 03 12:39:12 crc kubenswrapper[5052]: I1203 12:39:12.194636 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/memcached-0" Dec 03 12:39:12 crc kubenswrapper[5052]: I1203 12:39:12.196004 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:39:12 crc kubenswrapper[5052]: I1203 12:39:12.196035 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:39:12 crc kubenswrapper[5052]: I1203 12:39:12.196115 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:39:12 crc kubenswrapper[5052]: I1203 12:39:12.196134 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:39:12 crc kubenswrapper[5052]: I1203 12:39:12.250177 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/memcached-0"] Dec 03 12:39:12 crc kubenswrapper[5052]: I1203 12:39:12.261174 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/memcached-0"] Dec 03 12:39:12 crc kubenswrapper[5052]: I1203 12:39:12.296820 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/memcached-0"] Dec 03 12:39:12 crc kubenswrapper[5052]: E1203 12:39:12.297533 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="3a55dfe2-de85-40db-84f5-78404cd4a72b" containerName="memcached" Dec 03 12:39:12 crc kubenswrapper[5052]: I1203 12:39:12.297649 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="3a55dfe2-de85-40db-84f5-78404cd4a72b" containerName="memcached" Dec 03 12:39:12 crc kubenswrapper[5052]: I1203 12:39:12.297907 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="3a55dfe2-de85-40db-84f5-78404cd4a72b" containerName="memcached" Dec 03 12:39:12 crc kubenswrapper[5052]: I1203 12:39:12.298980 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/memcached-0" Dec 03 12:39:12 crc kubenswrapper[5052]: I1203 12:39:12.302544 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"memcached-memcached-dockercfg-wq48p" Dec 03 12:39:12 crc kubenswrapper[5052]: I1203 12:39:12.318069 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-memcached-svc" Dec 03 12:39:12 crc kubenswrapper[5052]: I1203 12:39:12.318104 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"memcached-config-data" Dec 03 12:39:12 crc kubenswrapper[5052]: I1203 12:39:12.325347 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/memcached-0"] Dec 03 12:39:12 crc kubenswrapper[5052]: I1203 12:39:12.450084 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ggq2r\" (UniqueName: \"kubernetes.io/projected/5d19811b-4e82-42b2-b6ec-f16e707484f2-kube-api-access-ggq2r\") pod \"memcached-0\" (UID: \"5d19811b-4e82-42b2-b6ec-f16e707484f2\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:39:12 crc kubenswrapper[5052]: I1203 12:39:12.450904 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/5d19811b-4e82-42b2-b6ec-f16e707484f2-kolla-config\") pod \"memcached-0\" (UID: \"5d19811b-4e82-42b2-b6ec-f16e707484f2\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:39:12 crc kubenswrapper[5052]: I1203 12:39:12.451001 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"memcached-tls-certs\" (UniqueName: \"kubernetes.io/secret/5d19811b-4e82-42b2-b6ec-f16e707484f2-memcached-tls-certs\") pod \"memcached-0\" (UID: \"5d19811b-4e82-42b2-b6ec-f16e707484f2\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:39:12 crc kubenswrapper[5052]: I1203 12:39:12.451093 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/5d19811b-4e82-42b2-b6ec-f16e707484f2-config-data\") pod \"memcached-0\" (UID: \"5d19811b-4e82-42b2-b6ec-f16e707484f2\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:39:12 crc kubenswrapper[5052]: I1203 12:39:12.451215 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5d19811b-4e82-42b2-b6ec-f16e707484f2-combined-ca-bundle\") pod \"memcached-0\" (UID: \"5d19811b-4e82-42b2-b6ec-f16e707484f2\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:39:12 crc kubenswrapper[5052]: I1203 12:39:12.553490 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5d19811b-4e82-42b2-b6ec-f16e707484f2-combined-ca-bundle\") pod \"memcached-0\" (UID: \"5d19811b-4e82-42b2-b6ec-f16e707484f2\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:39:12 crc kubenswrapper[5052]: I1203 12:39:12.553647 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-ggq2r\" (UniqueName: \"kubernetes.io/projected/5d19811b-4e82-42b2-b6ec-f16e707484f2-kube-api-access-ggq2r\") pod \"memcached-0\" (UID: \"5d19811b-4e82-42b2-b6ec-f16e707484f2\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:39:12 crc kubenswrapper[5052]: I1203 12:39:12.553758 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/5d19811b-4e82-42b2-b6ec-f16e707484f2-kolla-config\") pod \"memcached-0\" (UID: \"5d19811b-4e82-42b2-b6ec-f16e707484f2\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:39:12 crc kubenswrapper[5052]: I1203 12:39:12.553785 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"memcached-tls-certs\" (UniqueName: \"kubernetes.io/secret/5d19811b-4e82-42b2-b6ec-f16e707484f2-memcached-tls-certs\") pod \"memcached-0\" (UID: \"5d19811b-4e82-42b2-b6ec-f16e707484f2\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:39:12 crc kubenswrapper[5052]: I1203 12:39:12.553825 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/5d19811b-4e82-42b2-b6ec-f16e707484f2-config-data\") pod \"memcached-0\" (UID: \"5d19811b-4e82-42b2-b6ec-f16e707484f2\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:39:12 crc kubenswrapper[5052]: I1203 12:39:12.554667 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/5d19811b-4e82-42b2-b6ec-f16e707484f2-kolla-config\") pod \"memcached-0\" (UID: \"5d19811b-4e82-42b2-b6ec-f16e707484f2\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:39:12 crc kubenswrapper[5052]: I1203 12:39:12.554868 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/5d19811b-4e82-42b2-b6ec-f16e707484f2-config-data\") pod \"memcached-0\" (UID: \"5d19811b-4e82-42b2-b6ec-f16e707484f2\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:39:12 crc kubenswrapper[5052]: I1203 12:39:12.558855 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5d19811b-4e82-42b2-b6ec-f16e707484f2-combined-ca-bundle\") pod \"memcached-0\" (UID: \"5d19811b-4e82-42b2-b6ec-f16e707484f2\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:39:12 crc kubenswrapper[5052]: I1203 12:39:12.575688 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"memcached-tls-certs\" (UniqueName: \"kubernetes.io/secret/5d19811b-4e82-42b2-b6ec-f16e707484f2-memcached-tls-certs\") pod \"memcached-0\" (UID: \"5d19811b-4e82-42b2-b6ec-f16e707484f2\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:39:12 crc kubenswrapper[5052]: I1203 12:39:12.602331 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-ggq2r\" (UniqueName: \"kubernetes.io/projected/5d19811b-4e82-42b2-b6ec-f16e707484f2-kube-api-access-ggq2r\") pod \"memcached-0\" (UID: \"5d19811b-4e82-42b2-b6ec-f16e707484f2\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:39:12 crc kubenswrapper[5052]: I1203 12:39:12.616173 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/memcached-0" Dec 03 12:39:13 crc kubenswrapper[5052]: I1203 12:39:13.230651 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/memcached-0"] Dec 03 12:39:13 crc kubenswrapper[5052]: I1203 12:39:13.686138 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3a55dfe2-de85-40db-84f5-78404cd4a72b" path="/var/lib/kubelet/pods/3a55dfe2-de85-40db-84f5-78404cd4a72b/volumes" Dec 03 12:39:13 crc kubenswrapper[5052]: I1203 12:39:13.705156 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/openstack-cell1-galera-0"] Dec 03 12:39:13 crc kubenswrapper[5052]: I1203 12:39:13.972682 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/openstack-cell1-galera-0" podUID="2253cde7-04d1-41a2-82d3-9c22cc6099c4" containerName="galera" containerID="cri-o://422d4b6e0b64601b76bee8009b403a22fc44a6ee2de6d96f6fdfc586119b7cfe" gracePeriod=30 Dec 03 12:39:14 crc kubenswrapper[5052]: I1203 12:39:14.220854 5052 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Dec 03 12:39:14 crc kubenswrapper[5052]: I1203 12:39:14.220880 5052 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Dec 03 12:39:14 crc kubenswrapper[5052]: I1203 12:39:14.222562 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/memcached-0" event={"ID":"5d19811b-4e82-42b2-b6ec-f16e707484f2","Type":"ContainerStarted","Data":"78246e0cecbf2891b1eda71870c4a06e4588bbbe3e1fc591176960ad3a92ef57"} Dec 03 12:39:14 crc kubenswrapper[5052]: I1203 12:39:14.222594 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/memcached-0" event={"ID":"5d19811b-4e82-42b2-b6ec-f16e707484f2","Type":"ContainerStarted","Data":"3c4772e5ec1181020af33392aa008e4601d530e327e896e9a07400e27341f7d5"} Dec 03 12:39:14 crc kubenswrapper[5052]: I1203 12:39:14.222619 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/memcached-0" Dec 03 12:39:14 crc kubenswrapper[5052]: I1203 12:39:14.578306 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/neutron-544c66dd6d-75q8t" podUID="c34523d4-d634-4d76-a238-36566ef6a682" containerName="neutron-httpd" probeResult="failure" output="HTTP probe failed with statuscode: 503" Dec 03 12:39:14 crc kubenswrapper[5052]: I1203 12:39:14.578927 5052 prober.go:107] "Probe failed" probeType="Liveness" pod="openstack-kuttl-tests/neutron-544c66dd6d-75q8t" podUID="c34523d4-d634-4d76-a238-36566ef6a682" containerName="neutron-httpd" probeResult="failure" output="HTTP probe failed with statuscode: 503" Dec 03 12:39:14 crc kubenswrapper[5052]: I1203 12:39:14.581253 5052 prober.go:107] "Probe failed" probeType="Liveness" pod="openstack-kuttl-tests/neutron-544c66dd6d-75q8t" podUID="c34523d4-d634-4d76-a238-36566ef6a682" containerName="neutron-api" probeResult="failure" output="HTTP probe failed with statuscode: 503" Dec 03 12:39:14 crc kubenswrapper[5052]: I1203 12:39:14.959867 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:39:14 crc kubenswrapper[5052]: I1203 12:39:14.991682 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/memcached-0" podStartSLOduration=2.991665363 podStartE2EDuration="2.991665363s" podCreationTimestamp="2025-12-03 12:39:12 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:39:14.248825988 +0000 UTC m=+4744.837067752" watchObservedRunningTime="2025-12-03 12:39:14.991665363 +0000 UTC m=+4745.579907127" Dec 03 12:39:15 crc kubenswrapper[5052]: I1203 12:39:15.113589 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:39:15 crc kubenswrapper[5052]: I1203 12:39:15.139646 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:39:15 crc kubenswrapper[5052]: I1203 12:39:15.212485 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/2253cde7-04d1-41a2-82d3-9c22cc6099c4-kolla-config\") pod \"2253cde7-04d1-41a2-82d3-9c22cc6099c4\" (UID: \"2253cde7-04d1-41a2-82d3-9c22cc6099c4\") " Dec 03 12:39:15 crc kubenswrapper[5052]: I1203 12:39:15.213023 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/2253cde7-04d1-41a2-82d3-9c22cc6099c4-operator-scripts\") pod \"2253cde7-04d1-41a2-82d3-9c22cc6099c4\" (UID: \"2253cde7-04d1-41a2-82d3-9c22cc6099c4\") " Dec 03 12:39:15 crc kubenswrapper[5052]: I1203 12:39:15.213082 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/2253cde7-04d1-41a2-82d3-9c22cc6099c4-config-data-generated\") pod \"2253cde7-04d1-41a2-82d3-9c22cc6099c4\" (UID: \"2253cde7-04d1-41a2-82d3-9c22cc6099c4\") " Dec 03 12:39:15 crc kubenswrapper[5052]: I1203 12:39:15.213210 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/2253cde7-04d1-41a2-82d3-9c22cc6099c4-config-data-default\") pod \"2253cde7-04d1-41a2-82d3-9c22cc6099c4\" (UID: \"2253cde7-04d1-41a2-82d3-9c22cc6099c4\") " Dec 03 12:39:15 crc kubenswrapper[5052]: I1203 12:39:15.213341 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2253cde7-04d1-41a2-82d3-9c22cc6099c4-combined-ca-bundle\") pod \"2253cde7-04d1-41a2-82d3-9c22cc6099c4\" (UID: \"2253cde7-04d1-41a2-82d3-9c22cc6099c4\") " Dec 03 12:39:15 crc kubenswrapper[5052]: I1203 12:39:15.213373 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/2253cde7-04d1-41a2-82d3-9c22cc6099c4-galera-tls-certs\") pod \"2253cde7-04d1-41a2-82d3-9c22cc6099c4\" (UID: \"2253cde7-04d1-41a2-82d3-9c22cc6099c4\") " Dec 03 12:39:15 crc kubenswrapper[5052]: I1203 12:39:15.213486 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"mysql-db\" (UniqueName: \"kubernetes.io/local-volume/local-storage11-crc\") pod \"2253cde7-04d1-41a2-82d3-9c22cc6099c4\" (UID: \"2253cde7-04d1-41a2-82d3-9c22cc6099c4\") " Dec 03 12:39:15 crc kubenswrapper[5052]: I1203 12:39:15.213551 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ll69p\" (UniqueName: \"kubernetes.io/projected/2253cde7-04d1-41a2-82d3-9c22cc6099c4-kube-api-access-ll69p\") pod \"2253cde7-04d1-41a2-82d3-9c22cc6099c4\" (UID: \"2253cde7-04d1-41a2-82d3-9c22cc6099c4\") " Dec 03 12:39:15 crc kubenswrapper[5052]: I1203 12:39:15.213773 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2253cde7-04d1-41a2-82d3-9c22cc6099c4-config-data-generated" (OuterVolumeSpecName: "config-data-generated") pod "2253cde7-04d1-41a2-82d3-9c22cc6099c4" (UID: "2253cde7-04d1-41a2-82d3-9c22cc6099c4"). InnerVolumeSpecName "config-data-generated". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:39:15 crc kubenswrapper[5052]: I1203 12:39:15.214336 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/2253cde7-04d1-41a2-82d3-9c22cc6099c4-config-data-generated\") on node \"crc\" DevicePath \"\"" Dec 03 12:39:15 crc kubenswrapper[5052]: I1203 12:39:15.214453 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2253cde7-04d1-41a2-82d3-9c22cc6099c4-kolla-config" (OuterVolumeSpecName: "kolla-config") pod "2253cde7-04d1-41a2-82d3-9c22cc6099c4" (UID: "2253cde7-04d1-41a2-82d3-9c22cc6099c4"). InnerVolumeSpecName "kolla-config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:39:15 crc kubenswrapper[5052]: I1203 12:39:15.214551 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2253cde7-04d1-41a2-82d3-9c22cc6099c4-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "2253cde7-04d1-41a2-82d3-9c22cc6099c4" (UID: "2253cde7-04d1-41a2-82d3-9c22cc6099c4"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:39:15 crc kubenswrapper[5052]: I1203 12:39:15.217829 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2253cde7-04d1-41a2-82d3-9c22cc6099c4-config-data-default" (OuterVolumeSpecName: "config-data-default") pod "2253cde7-04d1-41a2-82d3-9c22cc6099c4" (UID: "2253cde7-04d1-41a2-82d3-9c22cc6099c4"). InnerVolumeSpecName "config-data-default". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:39:15 crc kubenswrapper[5052]: I1203 12:39:15.233983 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2253cde7-04d1-41a2-82d3-9c22cc6099c4-kube-api-access-ll69p" (OuterVolumeSpecName: "kube-api-access-ll69p") pod "2253cde7-04d1-41a2-82d3-9c22cc6099c4" (UID: "2253cde7-04d1-41a2-82d3-9c22cc6099c4"). InnerVolumeSpecName "kube-api-access-ll69p". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:39:15 crc kubenswrapper[5052]: I1203 12:39:15.244811 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage11-crc" (OuterVolumeSpecName: "mysql-db") pod "2253cde7-04d1-41a2-82d3-9c22cc6099c4" (UID: "2253cde7-04d1-41a2-82d3-9c22cc6099c4"). InnerVolumeSpecName "local-storage11-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 12:39:15 crc kubenswrapper[5052]: I1203 12:39:15.254322 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:39:15 crc kubenswrapper[5052]: I1203 12:39:15.254433 5052 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Dec 03 12:39:15 crc kubenswrapper[5052]: I1203 12:39:15.260447 5052 generic.go:334] "Generic (PLEG): container finished" podID="2253cde7-04d1-41a2-82d3-9c22cc6099c4" containerID="422d4b6e0b64601b76bee8009b403a22fc44a6ee2de6d96f6fdfc586119b7cfe" exitCode=0 Dec 03 12:39:15 crc kubenswrapper[5052]: I1203 12:39:15.262606 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-cell1-galera-0" event={"ID":"2253cde7-04d1-41a2-82d3-9c22cc6099c4","Type":"ContainerDied","Data":"422d4b6e0b64601b76bee8009b403a22fc44a6ee2de6d96f6fdfc586119b7cfe"} Dec 03 12:39:15 crc kubenswrapper[5052]: I1203 12:39:15.262896 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-cell1-galera-0" event={"ID":"2253cde7-04d1-41a2-82d3-9c22cc6099c4","Type":"ContainerDied","Data":"3b64affb676a87f9e7e47b6b73e6e439f09cdb8f0e1aff76678a39a1611a3360"} Dec 03 12:39:15 crc kubenswrapper[5052]: I1203 12:39:15.263023 5052 scope.go:117] "RemoveContainer" containerID="422d4b6e0b64601b76bee8009b403a22fc44a6ee2de6d96f6fdfc586119b7cfe" Dec 03 12:39:15 crc kubenswrapper[5052]: I1203 12:39:15.262669 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:39:15 crc kubenswrapper[5052]: I1203 12:39:15.282682 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2253cde7-04d1-41a2-82d3-9c22cc6099c4-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "2253cde7-04d1-41a2-82d3-9c22cc6099c4" (UID: "2253cde7-04d1-41a2-82d3-9c22cc6099c4"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:39:15 crc kubenswrapper[5052]: I1203 12:39:15.317600 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/2253cde7-04d1-41a2-82d3-9c22cc6099c4-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:39:15 crc kubenswrapper[5052]: I1203 12:39:15.317640 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/2253cde7-04d1-41a2-82d3-9c22cc6099c4-config-data-default\") on node \"crc\" DevicePath \"\"" Dec 03 12:39:15 crc kubenswrapper[5052]: I1203 12:39:15.317655 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2253cde7-04d1-41a2-82d3-9c22cc6099c4-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:39:15 crc kubenswrapper[5052]: I1203 12:39:15.317687 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage11-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage11-crc\") on node \"crc\" " Dec 03 12:39:15 crc kubenswrapper[5052]: I1203 12:39:15.317702 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-ll69p\" (UniqueName: \"kubernetes.io/projected/2253cde7-04d1-41a2-82d3-9c22cc6099c4-kube-api-access-ll69p\") on node \"crc\" DevicePath \"\"" Dec 03 12:39:15 crc kubenswrapper[5052]: I1203 12:39:15.317715 5052 reconciler_common.go:293] "Volume detached for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/2253cde7-04d1-41a2-82d3-9c22cc6099c4-kolla-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:39:15 crc kubenswrapper[5052]: I1203 12:39:15.342918 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:39:15 crc kubenswrapper[5052]: I1203 12:39:15.352368 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage11-crc" (UniqueName: "kubernetes.io/local-volume/local-storage11-crc") on node "crc" Dec 03 12:39:15 crc kubenswrapper[5052]: I1203 12:39:15.357661 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2253cde7-04d1-41a2-82d3-9c22cc6099c4-galera-tls-certs" (OuterVolumeSpecName: "galera-tls-certs") pod "2253cde7-04d1-41a2-82d3-9c22cc6099c4" (UID: "2253cde7-04d1-41a2-82d3-9c22cc6099c4"). InnerVolumeSpecName "galera-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:39:15 crc kubenswrapper[5052]: I1203 12:39:15.363079 5052 scope.go:117] "RemoveContainer" containerID="cef98b05ac915fca7faa283789b624d495e993a12f3a6d014308f6a6c2b30343" Dec 03 12:39:15 crc kubenswrapper[5052]: I1203 12:39:15.411285 5052 scope.go:117] "RemoveContainer" containerID="422d4b6e0b64601b76bee8009b403a22fc44a6ee2de6d96f6fdfc586119b7cfe" Dec 03 12:39:15 crc kubenswrapper[5052]: E1203 12:39:15.416700 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"422d4b6e0b64601b76bee8009b403a22fc44a6ee2de6d96f6fdfc586119b7cfe\": container with ID starting with 422d4b6e0b64601b76bee8009b403a22fc44a6ee2de6d96f6fdfc586119b7cfe not found: ID does not exist" containerID="422d4b6e0b64601b76bee8009b403a22fc44a6ee2de6d96f6fdfc586119b7cfe" Dec 03 12:39:15 crc kubenswrapper[5052]: I1203 12:39:15.416763 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"422d4b6e0b64601b76bee8009b403a22fc44a6ee2de6d96f6fdfc586119b7cfe"} err="failed to get container status \"422d4b6e0b64601b76bee8009b403a22fc44a6ee2de6d96f6fdfc586119b7cfe\": rpc error: code = NotFound desc = could not find container \"422d4b6e0b64601b76bee8009b403a22fc44a6ee2de6d96f6fdfc586119b7cfe\": container with ID starting with 422d4b6e0b64601b76bee8009b403a22fc44a6ee2de6d96f6fdfc586119b7cfe not found: ID does not exist" Dec 03 12:39:15 crc kubenswrapper[5052]: I1203 12:39:15.416797 5052 scope.go:117] "RemoveContainer" containerID="cef98b05ac915fca7faa283789b624d495e993a12f3a6d014308f6a6c2b30343" Dec 03 12:39:15 crc kubenswrapper[5052]: E1203 12:39:15.419814 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cef98b05ac915fca7faa283789b624d495e993a12f3a6d014308f6a6c2b30343\": container with ID starting with cef98b05ac915fca7faa283789b624d495e993a12f3a6d014308f6a6c2b30343 not found: ID does not exist" containerID="cef98b05ac915fca7faa283789b624d495e993a12f3a6d014308f6a6c2b30343" Dec 03 12:39:15 crc kubenswrapper[5052]: I1203 12:39:15.419865 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cef98b05ac915fca7faa283789b624d495e993a12f3a6d014308f6a6c2b30343"} err="failed to get container status \"cef98b05ac915fca7faa283789b624d495e993a12f3a6d014308f6a6c2b30343\": rpc error: code = NotFound desc = could not find container \"cef98b05ac915fca7faa283789b624d495e993a12f3a6d014308f6a6c2b30343\": container with ID starting with cef98b05ac915fca7faa283789b624d495e993a12f3a6d014308f6a6c2b30343 not found: ID does not exist" Dec 03 12:39:15 crc kubenswrapper[5052]: I1203 12:39:15.424630 5052 reconciler_common.go:293] "Volume detached for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/2253cde7-04d1-41a2-82d3-9c22cc6099c4-galera-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:39:15 crc kubenswrapper[5052]: I1203 12:39:15.424814 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage11-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage11-crc\") on node \"crc\" DevicePath \"\"" Dec 03 12:39:15 crc kubenswrapper[5052]: I1203 12:39:15.429990 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/placement-76bf6b6758-lvtjf"] Dec 03 12:39:15 crc kubenswrapper[5052]: I1203 12:39:15.430237 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/placement-76bf6b6758-lvtjf" podUID="0519e3a0-a23c-45c6-a45b-b913f4983532" containerName="placement-log" containerID="cri-o://9378b56786328ddd0407e98056ad23855b875424e631516f969dc644a1efb7e4" gracePeriod=30 Dec 03 12:39:15 crc kubenswrapper[5052]: I1203 12:39:15.431624 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/placement-76bf6b6758-lvtjf" podUID="0519e3a0-a23c-45c6-a45b-b913f4983532" containerName="placement-api" containerID="cri-o://f388a5399432c5ccb31c169b3ad931f092c9dc3131b2e25657fd938dda7893db" gracePeriod=30 Dec 03 12:39:15 crc kubenswrapper[5052]: I1203 12:39:15.458385 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/placement-76bf6b6758-lvtjf" podUID="0519e3a0-a23c-45c6-a45b-b913f4983532" containerName="placement-log" probeResult="failure" output="Get \"https://10.217.1.108:8778/\": EOF" Dec 03 12:39:15 crc kubenswrapper[5052]: I1203 12:39:15.458793 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/placement-76bf6b6758-lvtjf" podUID="0519e3a0-a23c-45c6-a45b-b913f4983532" containerName="placement-api" probeResult="failure" output="Get \"https://10.217.1.108:8778/\": EOF" Dec 03 12:39:15 crc kubenswrapper[5052]: I1203 12:39:15.476344 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/placement-75785bcbbb-wlvdv"] Dec 03 12:39:15 crc kubenswrapper[5052]: E1203 12:39:15.476784 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="2253cde7-04d1-41a2-82d3-9c22cc6099c4" containerName="galera" Dec 03 12:39:15 crc kubenswrapper[5052]: I1203 12:39:15.476801 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="2253cde7-04d1-41a2-82d3-9c22cc6099c4" containerName="galera" Dec 03 12:39:15 crc kubenswrapper[5052]: E1203 12:39:15.476836 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="2253cde7-04d1-41a2-82d3-9c22cc6099c4" containerName="mysql-bootstrap" Dec 03 12:39:15 crc kubenswrapper[5052]: I1203 12:39:15.476843 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="2253cde7-04d1-41a2-82d3-9c22cc6099c4" containerName="mysql-bootstrap" Dec 03 12:39:15 crc kubenswrapper[5052]: I1203 12:39:15.477051 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="2253cde7-04d1-41a2-82d3-9c22cc6099c4" containerName="galera" Dec 03 12:39:15 crc kubenswrapper[5052]: I1203 12:39:15.478133 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-75785bcbbb-wlvdv" Dec 03 12:39:15 crc kubenswrapper[5052]: I1203 12:39:15.506755 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/placement-75785bcbbb-wlvdv"] Dec 03 12:39:15 crc kubenswrapper[5052]: I1203 12:39:15.623201 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/openstack-cell1-galera-0"] Dec 03 12:39:15 crc kubenswrapper[5052]: I1203 12:39:15.628763 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/068f200e-7f4b-4cc8-9757-615f13e58d7d-internal-tls-certs\") pod \"placement-75785bcbbb-wlvdv\" (UID: \"068f200e-7f4b-4cc8-9757-615f13e58d7d\") " pod="openstack-kuttl-tests/placement-75785bcbbb-wlvdv" Dec 03 12:39:15 crc kubenswrapper[5052]: I1203 12:39:15.628821 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/068f200e-7f4b-4cc8-9757-615f13e58d7d-combined-ca-bundle\") pod \"placement-75785bcbbb-wlvdv\" (UID: \"068f200e-7f4b-4cc8-9757-615f13e58d7d\") " pod="openstack-kuttl-tests/placement-75785bcbbb-wlvdv" Dec 03 12:39:15 crc kubenswrapper[5052]: I1203 12:39:15.628896 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/068f200e-7f4b-4cc8-9757-615f13e58d7d-config-data\") pod \"placement-75785bcbbb-wlvdv\" (UID: \"068f200e-7f4b-4cc8-9757-615f13e58d7d\") " pod="openstack-kuttl-tests/placement-75785bcbbb-wlvdv" Dec 03 12:39:15 crc kubenswrapper[5052]: I1203 12:39:15.628925 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/068f200e-7f4b-4cc8-9757-615f13e58d7d-logs\") pod \"placement-75785bcbbb-wlvdv\" (UID: \"068f200e-7f4b-4cc8-9757-615f13e58d7d\") " pod="openstack-kuttl-tests/placement-75785bcbbb-wlvdv" Dec 03 12:39:15 crc kubenswrapper[5052]: I1203 12:39:15.628974 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/068f200e-7f4b-4cc8-9757-615f13e58d7d-scripts\") pod \"placement-75785bcbbb-wlvdv\" (UID: \"068f200e-7f4b-4cc8-9757-615f13e58d7d\") " pod="openstack-kuttl-tests/placement-75785bcbbb-wlvdv" Dec 03 12:39:15 crc kubenswrapper[5052]: I1203 12:39:15.629006 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jx7t9\" (UniqueName: \"kubernetes.io/projected/068f200e-7f4b-4cc8-9757-615f13e58d7d-kube-api-access-jx7t9\") pod \"placement-75785bcbbb-wlvdv\" (UID: \"068f200e-7f4b-4cc8-9757-615f13e58d7d\") " pod="openstack-kuttl-tests/placement-75785bcbbb-wlvdv" Dec 03 12:39:15 crc kubenswrapper[5052]: I1203 12:39:15.629033 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/068f200e-7f4b-4cc8-9757-615f13e58d7d-public-tls-certs\") pod \"placement-75785bcbbb-wlvdv\" (UID: \"068f200e-7f4b-4cc8-9757-615f13e58d7d\") " pod="openstack-kuttl-tests/placement-75785bcbbb-wlvdv" Dec 03 12:39:15 crc kubenswrapper[5052]: I1203 12:39:15.639781 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/openstack-cell1-galera-0"] Dec 03 12:39:15 crc kubenswrapper[5052]: I1203 12:39:15.654729 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/openstack-cell1-galera-0"] Dec 03 12:39:15 crc kubenswrapper[5052]: I1203 12:39:15.656349 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:39:15 crc kubenswrapper[5052]: I1203 12:39:15.658891 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"galera-openstack-cell1-dockercfg-4wff5" Dec 03 12:39:15 crc kubenswrapper[5052]: I1203 12:39:15.660333 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-galera-openstack-cell1-svc" Dec 03 12:39:15 crc kubenswrapper[5052]: I1203 12:39:15.661383 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"openstack-cell1-scripts" Dec 03 12:39:15 crc kubenswrapper[5052]: I1203 12:39:15.661650 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"openstack-cell1-config-data" Dec 03 12:39:15 crc kubenswrapper[5052]: I1203 12:39:15.734826 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2253cde7-04d1-41a2-82d3-9c22cc6099c4" path="/var/lib/kubelet/pods/2253cde7-04d1-41a2-82d3-9c22cc6099c4/volumes" Dec 03 12:39:15 crc kubenswrapper[5052]: I1203 12:39:15.734956 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/068f200e-7f4b-4cc8-9757-615f13e58d7d-internal-tls-certs\") pod \"placement-75785bcbbb-wlvdv\" (UID: \"068f200e-7f4b-4cc8-9757-615f13e58d7d\") " pod="openstack-kuttl-tests/placement-75785bcbbb-wlvdv" Dec 03 12:39:15 crc kubenswrapper[5052]: I1203 12:39:15.735002 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/068f200e-7f4b-4cc8-9757-615f13e58d7d-combined-ca-bundle\") pod \"placement-75785bcbbb-wlvdv\" (UID: \"068f200e-7f4b-4cc8-9757-615f13e58d7d\") " pod="openstack-kuttl-tests/placement-75785bcbbb-wlvdv" Dec 03 12:39:15 crc kubenswrapper[5052]: I1203 12:39:15.735067 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/068f200e-7f4b-4cc8-9757-615f13e58d7d-config-data\") pod \"placement-75785bcbbb-wlvdv\" (UID: \"068f200e-7f4b-4cc8-9757-615f13e58d7d\") " pod="openstack-kuttl-tests/placement-75785bcbbb-wlvdv" Dec 03 12:39:15 crc kubenswrapper[5052]: I1203 12:39:15.735093 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/068f200e-7f4b-4cc8-9757-615f13e58d7d-logs\") pod \"placement-75785bcbbb-wlvdv\" (UID: \"068f200e-7f4b-4cc8-9757-615f13e58d7d\") " pod="openstack-kuttl-tests/placement-75785bcbbb-wlvdv" Dec 03 12:39:15 crc kubenswrapper[5052]: I1203 12:39:15.735151 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/068f200e-7f4b-4cc8-9757-615f13e58d7d-scripts\") pod \"placement-75785bcbbb-wlvdv\" (UID: \"068f200e-7f4b-4cc8-9757-615f13e58d7d\") " pod="openstack-kuttl-tests/placement-75785bcbbb-wlvdv" Dec 03 12:39:15 crc kubenswrapper[5052]: I1203 12:39:15.735190 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-jx7t9\" (UniqueName: \"kubernetes.io/projected/068f200e-7f4b-4cc8-9757-615f13e58d7d-kube-api-access-jx7t9\") pod \"placement-75785bcbbb-wlvdv\" (UID: \"068f200e-7f4b-4cc8-9757-615f13e58d7d\") " pod="openstack-kuttl-tests/placement-75785bcbbb-wlvdv" Dec 03 12:39:15 crc kubenswrapper[5052]: I1203 12:39:15.735211 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/068f200e-7f4b-4cc8-9757-615f13e58d7d-public-tls-certs\") pod \"placement-75785bcbbb-wlvdv\" (UID: \"068f200e-7f4b-4cc8-9757-615f13e58d7d\") " pod="openstack-kuttl-tests/placement-75785bcbbb-wlvdv" Dec 03 12:39:15 crc kubenswrapper[5052]: I1203 12:39:15.735852 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/openstack-cell1-galera-0"] Dec 03 12:39:15 crc kubenswrapper[5052]: I1203 12:39:15.736124 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/068f200e-7f4b-4cc8-9757-615f13e58d7d-logs\") pod \"placement-75785bcbbb-wlvdv\" (UID: \"068f200e-7f4b-4cc8-9757-615f13e58d7d\") " pod="openstack-kuttl-tests/placement-75785bcbbb-wlvdv" Dec 03 12:39:15 crc kubenswrapper[5052]: I1203 12:39:15.737181 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:39:15 crc kubenswrapper[5052]: I1203 12:39:15.740766 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/068f200e-7f4b-4cc8-9757-615f13e58d7d-public-tls-certs\") pod \"placement-75785bcbbb-wlvdv\" (UID: \"068f200e-7f4b-4cc8-9757-615f13e58d7d\") " pod="openstack-kuttl-tests/placement-75785bcbbb-wlvdv" Dec 03 12:39:15 crc kubenswrapper[5052]: I1203 12:39:15.740981 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/068f200e-7f4b-4cc8-9757-615f13e58d7d-scripts\") pod \"placement-75785bcbbb-wlvdv\" (UID: \"068f200e-7f4b-4cc8-9757-615f13e58d7d\") " pod="openstack-kuttl-tests/placement-75785bcbbb-wlvdv" Dec 03 12:39:15 crc kubenswrapper[5052]: I1203 12:39:15.740982 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/068f200e-7f4b-4cc8-9757-615f13e58d7d-combined-ca-bundle\") pod \"placement-75785bcbbb-wlvdv\" (UID: \"068f200e-7f4b-4cc8-9757-615f13e58d7d\") " pod="openstack-kuttl-tests/placement-75785bcbbb-wlvdv" Dec 03 12:39:15 crc kubenswrapper[5052]: I1203 12:39:15.744178 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/068f200e-7f4b-4cc8-9757-615f13e58d7d-config-data\") pod \"placement-75785bcbbb-wlvdv\" (UID: \"068f200e-7f4b-4cc8-9757-615f13e58d7d\") " pod="openstack-kuttl-tests/placement-75785bcbbb-wlvdv" Dec 03 12:39:15 crc kubenswrapper[5052]: I1203 12:39:15.746898 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/068f200e-7f4b-4cc8-9757-615f13e58d7d-internal-tls-certs\") pod \"placement-75785bcbbb-wlvdv\" (UID: \"068f200e-7f4b-4cc8-9757-615f13e58d7d\") " pod="openstack-kuttl-tests/placement-75785bcbbb-wlvdv" Dec 03 12:39:15 crc kubenswrapper[5052]: I1203 12:39:15.760108 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-jx7t9\" (UniqueName: \"kubernetes.io/projected/068f200e-7f4b-4cc8-9757-615f13e58d7d-kube-api-access-jx7t9\") pod \"placement-75785bcbbb-wlvdv\" (UID: \"068f200e-7f4b-4cc8-9757-615f13e58d7d\") " pod="openstack-kuttl-tests/placement-75785bcbbb-wlvdv" Dec 03 12:39:15 crc kubenswrapper[5052]: I1203 12:39:15.810737 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-75785bcbbb-wlvdv" Dec 03 12:39:15 crc kubenswrapper[5052]: I1203 12:39:15.838167 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/f89ce0f5-5a92-49eb-9951-f2b491ac2139-operator-scripts\") pod \"openstack-cell1-galera-0\" (UID: \"f89ce0f5-5a92-49eb-9951-f2b491ac2139\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:39:15 crc kubenswrapper[5052]: I1203 12:39:15.838276 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/f89ce0f5-5a92-49eb-9951-f2b491ac2139-config-data-generated\") pod \"openstack-cell1-galera-0\" (UID: \"f89ce0f5-5a92-49eb-9951-f2b491ac2139\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:39:15 crc kubenswrapper[5052]: I1203 12:39:15.838307 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/f89ce0f5-5a92-49eb-9951-f2b491ac2139-config-data-default\") pod \"openstack-cell1-galera-0\" (UID: \"f89ce0f5-5a92-49eb-9951-f2b491ac2139\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:39:15 crc kubenswrapper[5052]: I1203 12:39:15.838551 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage11-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage11-crc\") pod \"openstack-cell1-galera-0\" (UID: \"f89ce0f5-5a92-49eb-9951-f2b491ac2139\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:39:15 crc kubenswrapper[5052]: I1203 12:39:15.838767 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/f89ce0f5-5a92-49eb-9951-f2b491ac2139-kolla-config\") pod \"openstack-cell1-galera-0\" (UID: \"f89ce0f5-5a92-49eb-9951-f2b491ac2139\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:39:15 crc kubenswrapper[5052]: I1203 12:39:15.838886 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/f89ce0f5-5a92-49eb-9951-f2b491ac2139-galera-tls-certs\") pod \"openstack-cell1-galera-0\" (UID: \"f89ce0f5-5a92-49eb-9951-f2b491ac2139\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:39:15 crc kubenswrapper[5052]: I1203 12:39:15.838998 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kr5ht\" (UniqueName: \"kubernetes.io/projected/f89ce0f5-5a92-49eb-9951-f2b491ac2139-kube-api-access-kr5ht\") pod \"openstack-cell1-galera-0\" (UID: \"f89ce0f5-5a92-49eb-9951-f2b491ac2139\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:39:15 crc kubenswrapper[5052]: I1203 12:39:15.840298 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f89ce0f5-5a92-49eb-9951-f2b491ac2139-combined-ca-bundle\") pod \"openstack-cell1-galera-0\" (UID: \"f89ce0f5-5a92-49eb-9951-f2b491ac2139\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:39:15 crc kubenswrapper[5052]: I1203 12:39:15.942366 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-kr5ht\" (UniqueName: \"kubernetes.io/projected/f89ce0f5-5a92-49eb-9951-f2b491ac2139-kube-api-access-kr5ht\") pod \"openstack-cell1-galera-0\" (UID: \"f89ce0f5-5a92-49eb-9951-f2b491ac2139\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:39:15 crc kubenswrapper[5052]: I1203 12:39:15.942505 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f89ce0f5-5a92-49eb-9951-f2b491ac2139-combined-ca-bundle\") pod \"openstack-cell1-galera-0\" (UID: \"f89ce0f5-5a92-49eb-9951-f2b491ac2139\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:39:15 crc kubenswrapper[5052]: I1203 12:39:15.942616 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/f89ce0f5-5a92-49eb-9951-f2b491ac2139-operator-scripts\") pod \"openstack-cell1-galera-0\" (UID: \"f89ce0f5-5a92-49eb-9951-f2b491ac2139\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:39:15 crc kubenswrapper[5052]: I1203 12:39:15.942652 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/f89ce0f5-5a92-49eb-9951-f2b491ac2139-config-data-generated\") pod \"openstack-cell1-galera-0\" (UID: \"f89ce0f5-5a92-49eb-9951-f2b491ac2139\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:39:15 crc kubenswrapper[5052]: I1203 12:39:15.942675 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/f89ce0f5-5a92-49eb-9951-f2b491ac2139-config-data-default\") pod \"openstack-cell1-galera-0\" (UID: \"f89ce0f5-5a92-49eb-9951-f2b491ac2139\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:39:15 crc kubenswrapper[5052]: I1203 12:39:15.942709 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage11-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage11-crc\") pod \"openstack-cell1-galera-0\" (UID: \"f89ce0f5-5a92-49eb-9951-f2b491ac2139\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:39:15 crc kubenswrapper[5052]: I1203 12:39:15.942741 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/f89ce0f5-5a92-49eb-9951-f2b491ac2139-kolla-config\") pod \"openstack-cell1-galera-0\" (UID: \"f89ce0f5-5a92-49eb-9951-f2b491ac2139\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:39:15 crc kubenswrapper[5052]: I1203 12:39:15.942770 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/f89ce0f5-5a92-49eb-9951-f2b491ac2139-galera-tls-certs\") pod \"openstack-cell1-galera-0\" (UID: \"f89ce0f5-5a92-49eb-9951-f2b491ac2139\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:39:15 crc kubenswrapper[5052]: I1203 12:39:15.943697 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/f89ce0f5-5a92-49eb-9951-f2b491ac2139-config-data-generated\") pod \"openstack-cell1-galera-0\" (UID: \"f89ce0f5-5a92-49eb-9951-f2b491ac2139\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:39:15 crc kubenswrapper[5052]: I1203 12:39:15.944388 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage11-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage11-crc\") pod \"openstack-cell1-galera-0\" (UID: \"f89ce0f5-5a92-49eb-9951-f2b491ac2139\") device mount path \"/mnt/openstack/pv11\"" pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:39:15 crc kubenswrapper[5052]: I1203 12:39:15.945122 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/f89ce0f5-5a92-49eb-9951-f2b491ac2139-kolla-config\") pod \"openstack-cell1-galera-0\" (UID: \"f89ce0f5-5a92-49eb-9951-f2b491ac2139\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:39:15 crc kubenswrapper[5052]: I1203 12:39:15.945449 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/f89ce0f5-5a92-49eb-9951-f2b491ac2139-operator-scripts\") pod \"openstack-cell1-galera-0\" (UID: \"f89ce0f5-5a92-49eb-9951-f2b491ac2139\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:39:15 crc kubenswrapper[5052]: I1203 12:39:15.951096 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/f89ce0f5-5a92-49eb-9951-f2b491ac2139-config-data-default\") pod \"openstack-cell1-galera-0\" (UID: \"f89ce0f5-5a92-49eb-9951-f2b491ac2139\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:39:16 crc kubenswrapper[5052]: I1203 12:39:16.272265 5052 generic.go:334] "Generic (PLEG): container finished" podID="0519e3a0-a23c-45c6-a45b-b913f4983532" containerID="9378b56786328ddd0407e98056ad23855b875424e631516f969dc644a1efb7e4" exitCode=143 Dec 03 12:39:16 crc kubenswrapper[5052]: I1203 12:39:16.272330 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-76bf6b6758-lvtjf" event={"ID":"0519e3a0-a23c-45c6-a45b-b913f4983532","Type":"ContainerDied","Data":"9378b56786328ddd0407e98056ad23855b875424e631516f969dc644a1efb7e4"} Dec 03 12:39:16 crc kubenswrapper[5052]: I1203 12:39:16.368552 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/placement-75785bcbbb-wlvdv"] Dec 03 12:39:16 crc kubenswrapper[5052]: I1203 12:39:16.385800 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 12:39:16 crc kubenswrapper[5052]: I1203 12:39:16.386167 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/cinder-api-0" podUID="93664241-e156-4225-980c-4dbe8aff3353" containerName="cinder-api-log" containerID="cri-o://165fa18efd5297210936aec88fa2048a36107465b7ba46db18221643e103d851" gracePeriod=30 Dec 03 12:39:16 crc kubenswrapper[5052]: I1203 12:39:16.386182 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/cinder-api-0" podUID="93664241-e156-4225-980c-4dbe8aff3353" containerName="cinder-api" containerID="cri-o://305768045d12def6ad73ffe7ea66855fa5cc0e381b3a31f58d215ce49facba41" gracePeriod=30 Dec 03 12:39:16 crc kubenswrapper[5052]: I1203 12:39:16.949801 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-metadata-0" podUID="edea8e53-340d-4f1e-99e5-082268c64ceb" containerName="nova-metadata-metadata" probeResult="failure" output="Get \"https://10.217.1.100:8775/\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Dec 03 12:39:16 crc kubenswrapper[5052]: I1203 12:39:16.949882 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-metadata-0" podUID="edea8e53-340d-4f1e-99e5-082268c64ceb" containerName="nova-metadata-log" probeResult="failure" output="Get \"https://10.217.1.100:8775/\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Dec 03 12:39:17 crc kubenswrapper[5052]: I1203 12:39:17.327988 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f89ce0f5-5a92-49eb-9951-f2b491ac2139-combined-ca-bundle\") pod \"openstack-cell1-galera-0\" (UID: \"f89ce0f5-5a92-49eb-9951-f2b491ac2139\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:39:17 crc kubenswrapper[5052]: I1203 12:39:17.328198 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/f89ce0f5-5a92-49eb-9951-f2b491ac2139-galera-tls-certs\") pod \"openstack-cell1-galera-0\" (UID: \"f89ce0f5-5a92-49eb-9951-f2b491ac2139\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:39:17 crc kubenswrapper[5052]: I1203 12:39:17.335998 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-kr5ht\" (UniqueName: \"kubernetes.io/projected/f89ce0f5-5a92-49eb-9951-f2b491ac2139-kube-api-access-kr5ht\") pod \"openstack-cell1-galera-0\" (UID: \"f89ce0f5-5a92-49eb-9951-f2b491ac2139\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:39:17 crc kubenswrapper[5052]: I1203 12:39:17.398587 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage11-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage11-crc\") pod \"openstack-cell1-galera-0\" (UID: \"f89ce0f5-5a92-49eb-9951-f2b491ac2139\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:39:17 crc kubenswrapper[5052]: I1203 12:39:17.517797 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:39:18 crc kubenswrapper[5052]: I1203 12:39:18.052893 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/openstack-cell1-galera-0"] Dec 03 12:39:18 crc kubenswrapper[5052]: W1203 12:39:18.056323 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf89ce0f5_5a92_49eb_9951_f2b491ac2139.slice/crio-2d5424cae249b796c399d3d475049ca1e43197f2411c87542635bf83f57ad715 WatchSource:0}: Error finding container 2d5424cae249b796c399d3d475049ca1e43197f2411c87542635bf83f57ad715: Status 404 returned error can't find the container with id 2d5424cae249b796c399d3d475049ca1e43197f2411c87542635bf83f57ad715 Dec 03 12:39:18 crc kubenswrapper[5052]: I1203 12:39:18.311446 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-75785bcbbb-wlvdv" event={"ID":"068f200e-7f4b-4cc8-9757-615f13e58d7d","Type":"ContainerStarted","Data":"9acdd30c66dec3d20b4dce9dc97671fbb203abed74b4539b05e311efaa1ae060"} Dec 03 12:39:18 crc kubenswrapper[5052]: I1203 12:39:18.311495 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-75785bcbbb-wlvdv" event={"ID":"068f200e-7f4b-4cc8-9757-615f13e58d7d","Type":"ContainerStarted","Data":"1e204f6c3414f741bf890709ea5f0d5b7479276967cb8ad863ae2e8578aaadfd"} Dec 03 12:39:18 crc kubenswrapper[5052]: I1203 12:39:18.311506 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-75785bcbbb-wlvdv" event={"ID":"068f200e-7f4b-4cc8-9757-615f13e58d7d","Type":"ContainerStarted","Data":"c20c64037906971409cb88cfbe54b367be87e1e42c1ff1db2fb1e32968c68d5e"} Dec 03 12:39:18 crc kubenswrapper[5052]: I1203 12:39:18.311571 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/placement-75785bcbbb-wlvdv" Dec 03 12:39:18 crc kubenswrapper[5052]: I1203 12:39:18.311587 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/placement-75785bcbbb-wlvdv" Dec 03 12:39:18 crc kubenswrapper[5052]: I1203 12:39:18.313332 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-cell1-galera-0" event={"ID":"f89ce0f5-5a92-49eb-9951-f2b491ac2139","Type":"ContainerStarted","Data":"471a98ef0f6b12304a21a7203c5b57972e489a2e00b16c3bff98f0d169d8edf7"} Dec 03 12:39:18 crc kubenswrapper[5052]: I1203 12:39:18.313362 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-cell1-galera-0" event={"ID":"f89ce0f5-5a92-49eb-9951-f2b491ac2139","Type":"ContainerStarted","Data":"2d5424cae249b796c399d3d475049ca1e43197f2411c87542635bf83f57ad715"} Dec 03 12:39:18 crc kubenswrapper[5052]: I1203 12:39:18.316725 5052 generic.go:334] "Generic (PLEG): container finished" podID="93664241-e156-4225-980c-4dbe8aff3353" containerID="165fa18efd5297210936aec88fa2048a36107465b7ba46db18221643e103d851" exitCode=143 Dec 03 12:39:18 crc kubenswrapper[5052]: I1203 12:39:18.316771 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"93664241-e156-4225-980c-4dbe8aff3353","Type":"ContainerDied","Data":"165fa18efd5297210936aec88fa2048a36107465b7ba46db18221643e103d851"} Dec 03 12:39:18 crc kubenswrapper[5052]: I1203 12:39:18.338206 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/placement-75785bcbbb-wlvdv" podStartSLOduration=3.338190027 podStartE2EDuration="3.338190027s" podCreationTimestamp="2025-12-03 12:39:15 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:39:18.329366735 +0000 UTC m=+4748.917608499" watchObservedRunningTime="2025-12-03 12:39:18.338190027 +0000 UTC m=+4748.926431781" Dec 03 12:39:19 crc kubenswrapper[5052]: I1203 12:39:19.328289 5052 generic.go:334] "Generic (PLEG): container finished" podID="f89ce0f5-5a92-49eb-9951-f2b491ac2139" containerID="471a98ef0f6b12304a21a7203c5b57972e489a2e00b16c3bff98f0d169d8edf7" exitCode=0 Dec 03 12:39:19 crc kubenswrapper[5052]: I1203 12:39:19.328474 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-cell1-galera-0" event={"ID":"f89ce0f5-5a92-49eb-9951-f2b491ac2139","Type":"ContainerDied","Data":"471a98ef0f6b12304a21a7203c5b57972e489a2e00b16c3bff98f0d169d8edf7"} Dec 03 12:39:20 crc kubenswrapper[5052]: I1203 12:39:20.195097 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:39:20 crc kubenswrapper[5052]: I1203 12:39:20.211077 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:39:20 crc kubenswrapper[5052]: I1203 12:39:20.211210 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:39:20 crc kubenswrapper[5052]: I1203 12:39:20.239573 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:39:20 crc kubenswrapper[5052]: I1203 12:39:20.250633 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:39:20 crc kubenswrapper[5052]: I1203 12:39:20.339442 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/93664241-e156-4225-980c-4dbe8aff3353-combined-ca-bundle\") pod \"93664241-e156-4225-980c-4dbe8aff3353\" (UID: \"93664241-e156-4225-980c-4dbe8aff3353\") " Dec 03 12:39:20 crc kubenswrapper[5052]: I1203 12:39:20.339602 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/93664241-e156-4225-980c-4dbe8aff3353-scripts\") pod \"93664241-e156-4225-980c-4dbe8aff3353\" (UID: \"93664241-e156-4225-980c-4dbe8aff3353\") " Dec 03 12:39:20 crc kubenswrapper[5052]: I1203 12:39:20.339700 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/93664241-e156-4225-980c-4dbe8aff3353-etc-machine-id\") pod \"93664241-e156-4225-980c-4dbe8aff3353\" (UID: \"93664241-e156-4225-980c-4dbe8aff3353\") " Dec 03 12:39:20 crc kubenswrapper[5052]: I1203 12:39:20.339762 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/93664241-e156-4225-980c-4dbe8aff3353-internal-tls-certs\") pod \"93664241-e156-4225-980c-4dbe8aff3353\" (UID: \"93664241-e156-4225-980c-4dbe8aff3353\") " Dec 03 12:39:20 crc kubenswrapper[5052]: I1203 12:39:20.339821 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/93664241-e156-4225-980c-4dbe8aff3353-public-tls-certs\") pod \"93664241-e156-4225-980c-4dbe8aff3353\" (UID: \"93664241-e156-4225-980c-4dbe8aff3353\") " Dec 03 12:39:20 crc kubenswrapper[5052]: I1203 12:39:20.340047 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xsp7z\" (UniqueName: \"kubernetes.io/projected/93664241-e156-4225-980c-4dbe8aff3353-kube-api-access-xsp7z\") pod \"93664241-e156-4225-980c-4dbe8aff3353\" (UID: \"93664241-e156-4225-980c-4dbe8aff3353\") " Dec 03 12:39:20 crc kubenswrapper[5052]: I1203 12:39:20.340106 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/93664241-e156-4225-980c-4dbe8aff3353-config-data-custom\") pod \"93664241-e156-4225-980c-4dbe8aff3353\" (UID: \"93664241-e156-4225-980c-4dbe8aff3353\") " Dec 03 12:39:20 crc kubenswrapper[5052]: I1203 12:39:20.340139 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/93664241-e156-4225-980c-4dbe8aff3353-config-data\") pod \"93664241-e156-4225-980c-4dbe8aff3353\" (UID: \"93664241-e156-4225-980c-4dbe8aff3353\") " Dec 03 12:39:20 crc kubenswrapper[5052]: I1203 12:39:20.340172 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/93664241-e156-4225-980c-4dbe8aff3353-logs\") pod \"93664241-e156-4225-980c-4dbe8aff3353\" (UID: \"93664241-e156-4225-980c-4dbe8aff3353\") " Dec 03 12:39:20 crc kubenswrapper[5052]: I1203 12:39:20.343308 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/93664241-e156-4225-980c-4dbe8aff3353-etc-machine-id" (OuterVolumeSpecName: "etc-machine-id") pod "93664241-e156-4225-980c-4dbe8aff3353" (UID: "93664241-e156-4225-980c-4dbe8aff3353"). InnerVolumeSpecName "etc-machine-id". PluginName "kubernetes.io/host-path", VolumeGidValue "" Dec 03 12:39:20 crc kubenswrapper[5052]: I1203 12:39:20.343792 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93664241-e156-4225-980c-4dbe8aff3353-logs" (OuterVolumeSpecName: "logs") pod "93664241-e156-4225-980c-4dbe8aff3353" (UID: "93664241-e156-4225-980c-4dbe8aff3353"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:39:20 crc kubenswrapper[5052]: I1203 12:39:20.352753 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/93664241-e156-4225-980c-4dbe8aff3353-config-data-custom" (OuterVolumeSpecName: "config-data-custom") pod "93664241-e156-4225-980c-4dbe8aff3353" (UID: "93664241-e156-4225-980c-4dbe8aff3353"). InnerVolumeSpecName "config-data-custom". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:39:20 crc kubenswrapper[5052]: I1203 12:39:20.352857 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/93664241-e156-4225-980c-4dbe8aff3353-scripts" (OuterVolumeSpecName: "scripts") pod "93664241-e156-4225-980c-4dbe8aff3353" (UID: "93664241-e156-4225-980c-4dbe8aff3353"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:39:20 crc kubenswrapper[5052]: I1203 12:39:20.353135 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/93664241-e156-4225-980c-4dbe8aff3353-kube-api-access-xsp7z" (OuterVolumeSpecName: "kube-api-access-xsp7z") pod "93664241-e156-4225-980c-4dbe8aff3353" (UID: "93664241-e156-4225-980c-4dbe8aff3353"). InnerVolumeSpecName "kube-api-access-xsp7z". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:39:20 crc kubenswrapper[5052]: I1203 12:39:20.358773 5052 generic.go:334] "Generic (PLEG): container finished" podID="93664241-e156-4225-980c-4dbe8aff3353" containerID="305768045d12def6ad73ffe7ea66855fa5cc0e381b3a31f58d215ce49facba41" exitCode=0 Dec 03 12:39:20 crc kubenswrapper[5052]: I1203 12:39:20.358876 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"93664241-e156-4225-980c-4dbe8aff3353","Type":"ContainerDied","Data":"305768045d12def6ad73ffe7ea66855fa5cc0e381b3a31f58d215ce49facba41"} Dec 03 12:39:20 crc kubenswrapper[5052]: I1203 12:39:20.358924 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"93664241-e156-4225-980c-4dbe8aff3353","Type":"ContainerDied","Data":"e9aab40643fa964dffd0ed38e2df1e663e805d74053f0e1478502101a719137a"} Dec 03 12:39:20 crc kubenswrapper[5052]: I1203 12:39:20.358987 5052 scope.go:117] "RemoveContainer" containerID="305768045d12def6ad73ffe7ea66855fa5cc0e381b3a31f58d215ce49facba41" Dec 03 12:39:20 crc kubenswrapper[5052]: I1203 12:39:20.364112 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-cell1-galera-0" event={"ID":"f89ce0f5-5a92-49eb-9951-f2b491ac2139","Type":"ContainerStarted","Data":"0c3d8b3ca8a9d7d4611f59d9c0bcfa83d8ea56d3df8caa14069f56e2580808ef"} Dec 03 12:39:20 crc kubenswrapper[5052]: I1203 12:39:20.369185 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:39:20 crc kubenswrapper[5052]: I1203 12:39:20.359185 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:39:20 crc kubenswrapper[5052]: I1203 12:39:20.375341 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:39:20 crc kubenswrapper[5052]: I1203 12:39:20.398766 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/openstack-cell1-galera-0" podStartSLOduration=5.398742715 podStartE2EDuration="5.398742715s" podCreationTimestamp="2025-12-03 12:39:15 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:39:20.393137424 +0000 UTC m=+4750.981379188" watchObservedRunningTime="2025-12-03 12:39:20.398742715 +0000 UTC m=+4750.986984469" Dec 03 12:39:20 crc kubenswrapper[5052]: I1203 12:39:20.399875 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/93664241-e156-4225-980c-4dbe8aff3353-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "93664241-e156-4225-980c-4dbe8aff3353" (UID: "93664241-e156-4225-980c-4dbe8aff3353"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:39:20 crc kubenswrapper[5052]: I1203 12:39:20.443364 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-xsp7z\" (UniqueName: \"kubernetes.io/projected/93664241-e156-4225-980c-4dbe8aff3353-kube-api-access-xsp7z\") on node \"crc\" DevicePath \"\"" Dec 03 12:39:20 crc kubenswrapper[5052]: I1203 12:39:20.443402 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/93664241-e156-4225-980c-4dbe8aff3353-config-data-custom\") on node \"crc\" DevicePath \"\"" Dec 03 12:39:20 crc kubenswrapper[5052]: I1203 12:39:20.443414 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/93664241-e156-4225-980c-4dbe8aff3353-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:39:20 crc kubenswrapper[5052]: I1203 12:39:20.443427 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/93664241-e156-4225-980c-4dbe8aff3353-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:39:20 crc kubenswrapper[5052]: I1203 12:39:20.443438 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/93664241-e156-4225-980c-4dbe8aff3353-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:39:20 crc kubenswrapper[5052]: I1203 12:39:20.443448 5052 reconciler_common.go:293] "Volume detached for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/93664241-e156-4225-980c-4dbe8aff3353-etc-machine-id\") on node \"crc\" DevicePath \"\"" Dec 03 12:39:20 crc kubenswrapper[5052]: I1203 12:39:20.460702 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/93664241-e156-4225-980c-4dbe8aff3353-config-data" (OuterVolumeSpecName: "config-data") pod "93664241-e156-4225-980c-4dbe8aff3353" (UID: "93664241-e156-4225-980c-4dbe8aff3353"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:39:20 crc kubenswrapper[5052]: I1203 12:39:20.461699 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/93664241-e156-4225-980c-4dbe8aff3353-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "93664241-e156-4225-980c-4dbe8aff3353" (UID: "93664241-e156-4225-980c-4dbe8aff3353"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:39:20 crc kubenswrapper[5052]: I1203 12:39:20.477559 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/93664241-e156-4225-980c-4dbe8aff3353-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "93664241-e156-4225-980c-4dbe8aff3353" (UID: "93664241-e156-4225-980c-4dbe8aff3353"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:39:20 crc kubenswrapper[5052]: I1203 12:39:20.545360 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/93664241-e156-4225-980c-4dbe8aff3353-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:39:20 crc kubenswrapper[5052]: I1203 12:39:20.545891 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/93664241-e156-4225-980c-4dbe8aff3353-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:39:20 crc kubenswrapper[5052]: I1203 12:39:20.546008 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/93664241-e156-4225-980c-4dbe8aff3353-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:39:20 crc kubenswrapper[5052]: I1203 12:39:20.575325 5052 scope.go:117] "RemoveContainer" containerID="165fa18efd5297210936aec88fa2048a36107465b7ba46db18221643e103d851" Dec 03 12:39:20 crc kubenswrapper[5052]: I1203 12:39:20.593606 5052 scope.go:117] "RemoveContainer" containerID="305768045d12def6ad73ffe7ea66855fa5cc0e381b3a31f58d215ce49facba41" Dec 03 12:39:20 crc kubenswrapper[5052]: E1203 12:39:20.594110 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"305768045d12def6ad73ffe7ea66855fa5cc0e381b3a31f58d215ce49facba41\": container with ID starting with 305768045d12def6ad73ffe7ea66855fa5cc0e381b3a31f58d215ce49facba41 not found: ID does not exist" containerID="305768045d12def6ad73ffe7ea66855fa5cc0e381b3a31f58d215ce49facba41" Dec 03 12:39:20 crc kubenswrapper[5052]: I1203 12:39:20.594255 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"305768045d12def6ad73ffe7ea66855fa5cc0e381b3a31f58d215ce49facba41"} err="failed to get container status \"305768045d12def6ad73ffe7ea66855fa5cc0e381b3a31f58d215ce49facba41\": rpc error: code = NotFound desc = could not find container \"305768045d12def6ad73ffe7ea66855fa5cc0e381b3a31f58d215ce49facba41\": container with ID starting with 305768045d12def6ad73ffe7ea66855fa5cc0e381b3a31f58d215ce49facba41 not found: ID does not exist" Dec 03 12:39:20 crc kubenswrapper[5052]: I1203 12:39:20.594382 5052 scope.go:117] "RemoveContainer" containerID="165fa18efd5297210936aec88fa2048a36107465b7ba46db18221643e103d851" Dec 03 12:39:20 crc kubenswrapper[5052]: E1203 12:39:20.594797 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"165fa18efd5297210936aec88fa2048a36107465b7ba46db18221643e103d851\": container with ID starting with 165fa18efd5297210936aec88fa2048a36107465b7ba46db18221643e103d851 not found: ID does not exist" containerID="165fa18efd5297210936aec88fa2048a36107465b7ba46db18221643e103d851" Dec 03 12:39:20 crc kubenswrapper[5052]: I1203 12:39:20.594916 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"165fa18efd5297210936aec88fa2048a36107465b7ba46db18221643e103d851"} err="failed to get container status \"165fa18efd5297210936aec88fa2048a36107465b7ba46db18221643e103d851\": rpc error: code = NotFound desc = could not find container \"165fa18efd5297210936aec88fa2048a36107465b7ba46db18221643e103d851\": container with ID starting with 165fa18efd5297210936aec88fa2048a36107465b7ba46db18221643e103d851 not found: ID does not exist" Dec 03 12:39:20 crc kubenswrapper[5052]: I1203 12:39:20.707342 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 12:39:20 crc kubenswrapper[5052]: I1203 12:39:20.756573 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 12:39:20 crc kubenswrapper[5052]: I1203 12:39:20.781993 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 12:39:20 crc kubenswrapper[5052]: E1203 12:39:20.782595 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="93664241-e156-4225-980c-4dbe8aff3353" containerName="cinder-api-log" Dec 03 12:39:20 crc kubenswrapper[5052]: I1203 12:39:20.782625 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="93664241-e156-4225-980c-4dbe8aff3353" containerName="cinder-api-log" Dec 03 12:39:20 crc kubenswrapper[5052]: E1203 12:39:20.782646 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="93664241-e156-4225-980c-4dbe8aff3353" containerName="cinder-api" Dec 03 12:39:20 crc kubenswrapper[5052]: I1203 12:39:20.782654 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="93664241-e156-4225-980c-4dbe8aff3353" containerName="cinder-api" Dec 03 12:39:20 crc kubenswrapper[5052]: I1203 12:39:20.782919 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="93664241-e156-4225-980c-4dbe8aff3353" containerName="cinder-api" Dec 03 12:39:20 crc kubenswrapper[5052]: I1203 12:39:20.782945 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="93664241-e156-4225-980c-4dbe8aff3353" containerName="cinder-api-log" Dec 03 12:39:20 crc kubenswrapper[5052]: I1203 12:39:20.784327 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:39:20 crc kubenswrapper[5052]: I1203 12:39:20.786671 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-cinder-internal-svc" Dec 03 12:39:20 crc kubenswrapper[5052]: I1203 12:39:20.787048 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cinder-api-config-data" Dec 03 12:39:20 crc kubenswrapper[5052]: I1203 12:39:20.787246 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-cinder-public-svc" Dec 03 12:39:20 crc kubenswrapper[5052]: I1203 12:39:20.798013 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 12:39:20 crc kubenswrapper[5052]: I1203 12:39:20.826040 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:39:20 crc kubenswrapper[5052]: I1203 12:39:20.843038 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/placement-76bf6b6758-lvtjf" podUID="0519e3a0-a23c-45c6-a45b-b913f4983532" containerName="placement-api" probeResult="failure" output="Get \"https://10.217.1.108:8778/\": read tcp 10.217.0.2:57922->10.217.1.108:8778: read: connection reset by peer" Dec 03 12:39:20 crc kubenswrapper[5052]: I1203 12:39:20.843058 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/placement-76bf6b6758-lvtjf" podUID="0519e3a0-a23c-45c6-a45b-b913f4983532" containerName="placement-log" probeResult="failure" output="Get \"https://10.217.1.108:8778/\": read tcp 10.217.0.2:57930->10.217.1.108:8778: read: connection reset by peer" Dec 03 12:39:20 crc kubenswrapper[5052]: I1203 12:39:20.861242 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jkxtd\" (UniqueName: \"kubernetes.io/projected/5bca7a3b-316e-4357-8ae4-e47be319dbf1-kube-api-access-jkxtd\") pod \"cinder-api-0\" (UID: \"5bca7a3b-316e-4357-8ae4-e47be319dbf1\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:39:20 crc kubenswrapper[5052]: I1203 12:39:20.861304 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/5bca7a3b-316e-4357-8ae4-e47be319dbf1-config-data\") pod \"cinder-api-0\" (UID: \"5bca7a3b-316e-4357-8ae4-e47be319dbf1\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:39:20 crc kubenswrapper[5052]: I1203 12:39:20.861383 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/5bca7a3b-316e-4357-8ae4-e47be319dbf1-internal-tls-certs\") pod \"cinder-api-0\" (UID: \"5bca7a3b-316e-4357-8ae4-e47be319dbf1\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:39:20 crc kubenswrapper[5052]: I1203 12:39:20.861457 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/5bca7a3b-316e-4357-8ae4-e47be319dbf1-public-tls-certs\") pod \"cinder-api-0\" (UID: \"5bca7a3b-316e-4357-8ae4-e47be319dbf1\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:39:20 crc kubenswrapper[5052]: I1203 12:39:20.861480 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5bca7a3b-316e-4357-8ae4-e47be319dbf1-combined-ca-bundle\") pod \"cinder-api-0\" (UID: \"5bca7a3b-316e-4357-8ae4-e47be319dbf1\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:39:20 crc kubenswrapper[5052]: I1203 12:39:20.861498 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/5bca7a3b-316e-4357-8ae4-e47be319dbf1-config-data-custom\") pod \"cinder-api-0\" (UID: \"5bca7a3b-316e-4357-8ae4-e47be319dbf1\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:39:20 crc kubenswrapper[5052]: I1203 12:39:20.861533 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/5bca7a3b-316e-4357-8ae4-e47be319dbf1-logs\") pod \"cinder-api-0\" (UID: \"5bca7a3b-316e-4357-8ae4-e47be319dbf1\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:39:20 crc kubenswrapper[5052]: I1203 12:39:20.861571 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/5bca7a3b-316e-4357-8ae4-e47be319dbf1-scripts\") pod \"cinder-api-0\" (UID: \"5bca7a3b-316e-4357-8ae4-e47be319dbf1\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:39:20 crc kubenswrapper[5052]: I1203 12:39:20.861600 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/5bca7a3b-316e-4357-8ae4-e47be319dbf1-etc-machine-id\") pod \"cinder-api-0\" (UID: \"5bca7a3b-316e-4357-8ae4-e47be319dbf1\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:39:20 crc kubenswrapper[5052]: I1203 12:39:20.962913 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/5bca7a3b-316e-4357-8ae4-e47be319dbf1-public-tls-certs\") pod \"cinder-api-0\" (UID: \"5bca7a3b-316e-4357-8ae4-e47be319dbf1\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:39:20 crc kubenswrapper[5052]: I1203 12:39:20.962973 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5bca7a3b-316e-4357-8ae4-e47be319dbf1-combined-ca-bundle\") pod \"cinder-api-0\" (UID: \"5bca7a3b-316e-4357-8ae4-e47be319dbf1\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:39:20 crc kubenswrapper[5052]: I1203 12:39:20.963004 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/5bca7a3b-316e-4357-8ae4-e47be319dbf1-config-data-custom\") pod \"cinder-api-0\" (UID: \"5bca7a3b-316e-4357-8ae4-e47be319dbf1\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:39:20 crc kubenswrapper[5052]: I1203 12:39:20.963024 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/5bca7a3b-316e-4357-8ae4-e47be319dbf1-logs\") pod \"cinder-api-0\" (UID: \"5bca7a3b-316e-4357-8ae4-e47be319dbf1\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:39:20 crc kubenswrapper[5052]: I1203 12:39:20.963077 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/5bca7a3b-316e-4357-8ae4-e47be319dbf1-scripts\") pod \"cinder-api-0\" (UID: \"5bca7a3b-316e-4357-8ae4-e47be319dbf1\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:39:20 crc kubenswrapper[5052]: I1203 12:39:20.963116 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/5bca7a3b-316e-4357-8ae4-e47be319dbf1-etc-machine-id\") pod \"cinder-api-0\" (UID: \"5bca7a3b-316e-4357-8ae4-e47be319dbf1\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:39:20 crc kubenswrapper[5052]: I1203 12:39:20.963147 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-jkxtd\" (UniqueName: \"kubernetes.io/projected/5bca7a3b-316e-4357-8ae4-e47be319dbf1-kube-api-access-jkxtd\") pod \"cinder-api-0\" (UID: \"5bca7a3b-316e-4357-8ae4-e47be319dbf1\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:39:20 crc kubenswrapper[5052]: I1203 12:39:20.963195 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/5bca7a3b-316e-4357-8ae4-e47be319dbf1-config-data\") pod \"cinder-api-0\" (UID: \"5bca7a3b-316e-4357-8ae4-e47be319dbf1\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:39:20 crc kubenswrapper[5052]: I1203 12:39:20.963339 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/5bca7a3b-316e-4357-8ae4-e47be319dbf1-internal-tls-certs\") pod \"cinder-api-0\" (UID: \"5bca7a3b-316e-4357-8ae4-e47be319dbf1\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:39:20 crc kubenswrapper[5052]: I1203 12:39:20.963618 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/5bca7a3b-316e-4357-8ae4-e47be319dbf1-logs\") pod \"cinder-api-0\" (UID: \"5bca7a3b-316e-4357-8ae4-e47be319dbf1\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:39:20 crc kubenswrapper[5052]: I1203 12:39:20.963691 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/5bca7a3b-316e-4357-8ae4-e47be319dbf1-etc-machine-id\") pod \"cinder-api-0\" (UID: \"5bca7a3b-316e-4357-8ae4-e47be319dbf1\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:39:20 crc kubenswrapper[5052]: I1203 12:39:20.967743 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5bca7a3b-316e-4357-8ae4-e47be319dbf1-combined-ca-bundle\") pod \"cinder-api-0\" (UID: \"5bca7a3b-316e-4357-8ae4-e47be319dbf1\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:39:20 crc kubenswrapper[5052]: I1203 12:39:20.968081 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/5bca7a3b-316e-4357-8ae4-e47be319dbf1-scripts\") pod \"cinder-api-0\" (UID: \"5bca7a3b-316e-4357-8ae4-e47be319dbf1\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:39:20 crc kubenswrapper[5052]: I1203 12:39:20.968186 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/5bca7a3b-316e-4357-8ae4-e47be319dbf1-public-tls-certs\") pod \"cinder-api-0\" (UID: \"5bca7a3b-316e-4357-8ae4-e47be319dbf1\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:39:20 crc kubenswrapper[5052]: I1203 12:39:20.968868 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/5bca7a3b-316e-4357-8ae4-e47be319dbf1-config-data\") pod \"cinder-api-0\" (UID: \"5bca7a3b-316e-4357-8ae4-e47be319dbf1\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:39:20 crc kubenswrapper[5052]: I1203 12:39:20.969503 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/5bca7a3b-316e-4357-8ae4-e47be319dbf1-internal-tls-certs\") pod \"cinder-api-0\" (UID: \"5bca7a3b-316e-4357-8ae4-e47be319dbf1\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:39:20 crc kubenswrapper[5052]: I1203 12:39:20.969914 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/5bca7a3b-316e-4357-8ae4-e47be319dbf1-config-data-custom\") pod \"cinder-api-0\" (UID: \"5bca7a3b-316e-4357-8ae4-e47be319dbf1\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:39:20 crc kubenswrapper[5052]: I1203 12:39:20.982003 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-jkxtd\" (UniqueName: \"kubernetes.io/projected/5bca7a3b-316e-4357-8ae4-e47be319dbf1-kube-api-access-jkxtd\") pod \"cinder-api-0\" (UID: \"5bca7a3b-316e-4357-8ae4-e47be319dbf1\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:39:21 crc kubenswrapper[5052]: I1203 12:39:21.113700 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:39:21 crc kubenswrapper[5052]: I1203 12:39:21.396715 5052 generic.go:334] "Generic (PLEG): container finished" podID="0519e3a0-a23c-45c6-a45b-b913f4983532" containerID="f388a5399432c5ccb31c169b3ad931f092c9dc3131b2e25657fd938dda7893db" exitCode=0 Dec 03 12:39:21 crc kubenswrapper[5052]: I1203 12:39:21.396824 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-76bf6b6758-lvtjf" event={"ID":"0519e3a0-a23c-45c6-a45b-b913f4983532","Type":"ContainerDied","Data":"f388a5399432c5ccb31c169b3ad931f092c9dc3131b2e25657fd938dda7893db"} Dec 03 12:39:21 crc kubenswrapper[5052]: I1203 12:39:21.514547 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-76bf6b6758-lvtjf" Dec 03 12:39:21 crc kubenswrapper[5052]: I1203 12:39:21.579652 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0519e3a0-a23c-45c6-a45b-b913f4983532-combined-ca-bundle\") pod \"0519e3a0-a23c-45c6-a45b-b913f4983532\" (UID: \"0519e3a0-a23c-45c6-a45b-b913f4983532\") " Dec 03 12:39:21 crc kubenswrapper[5052]: I1203 12:39:21.579743 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/0519e3a0-a23c-45c6-a45b-b913f4983532-scripts\") pod \"0519e3a0-a23c-45c6-a45b-b913f4983532\" (UID: \"0519e3a0-a23c-45c6-a45b-b913f4983532\") " Dec 03 12:39:21 crc kubenswrapper[5052]: I1203 12:39:21.579770 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/0519e3a0-a23c-45c6-a45b-b913f4983532-public-tls-certs\") pod \"0519e3a0-a23c-45c6-a45b-b913f4983532\" (UID: \"0519e3a0-a23c-45c6-a45b-b913f4983532\") " Dec 03 12:39:21 crc kubenswrapper[5052]: I1203 12:39:21.579818 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/0519e3a0-a23c-45c6-a45b-b913f4983532-config-data\") pod \"0519e3a0-a23c-45c6-a45b-b913f4983532\" (UID: \"0519e3a0-a23c-45c6-a45b-b913f4983532\") " Dec 03 12:39:21 crc kubenswrapper[5052]: I1203 12:39:21.579903 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lt4sm\" (UniqueName: \"kubernetes.io/projected/0519e3a0-a23c-45c6-a45b-b913f4983532-kube-api-access-lt4sm\") pod \"0519e3a0-a23c-45c6-a45b-b913f4983532\" (UID: \"0519e3a0-a23c-45c6-a45b-b913f4983532\") " Dec 03 12:39:21 crc kubenswrapper[5052]: I1203 12:39:21.579930 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/0519e3a0-a23c-45c6-a45b-b913f4983532-internal-tls-certs\") pod \"0519e3a0-a23c-45c6-a45b-b913f4983532\" (UID: \"0519e3a0-a23c-45c6-a45b-b913f4983532\") " Dec 03 12:39:21 crc kubenswrapper[5052]: I1203 12:39:21.579971 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/0519e3a0-a23c-45c6-a45b-b913f4983532-logs\") pod \"0519e3a0-a23c-45c6-a45b-b913f4983532\" (UID: \"0519e3a0-a23c-45c6-a45b-b913f4983532\") " Dec 03 12:39:21 crc kubenswrapper[5052]: I1203 12:39:21.580758 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0519e3a0-a23c-45c6-a45b-b913f4983532-logs" (OuterVolumeSpecName: "logs") pod "0519e3a0-a23c-45c6-a45b-b913f4983532" (UID: "0519e3a0-a23c-45c6-a45b-b913f4983532"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:39:21 crc kubenswrapper[5052]: I1203 12:39:21.589533 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0519e3a0-a23c-45c6-a45b-b913f4983532-scripts" (OuterVolumeSpecName: "scripts") pod "0519e3a0-a23c-45c6-a45b-b913f4983532" (UID: "0519e3a0-a23c-45c6-a45b-b913f4983532"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:39:21 crc kubenswrapper[5052]: I1203 12:39:21.589770 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0519e3a0-a23c-45c6-a45b-b913f4983532-kube-api-access-lt4sm" (OuterVolumeSpecName: "kube-api-access-lt4sm") pod "0519e3a0-a23c-45c6-a45b-b913f4983532" (UID: "0519e3a0-a23c-45c6-a45b-b913f4983532"). InnerVolumeSpecName "kube-api-access-lt4sm". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:39:21 crc kubenswrapper[5052]: I1203 12:39:21.682442 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0519e3a0-a23c-45c6-a45b-b913f4983532-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "0519e3a0-a23c-45c6-a45b-b913f4983532" (UID: "0519e3a0-a23c-45c6-a45b-b913f4983532"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:39:21 crc kubenswrapper[5052]: I1203 12:39:21.682486 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/0519e3a0-a23c-45c6-a45b-b913f4983532-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:39:21 crc kubenswrapper[5052]: I1203 12:39:21.682539 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-lt4sm\" (UniqueName: \"kubernetes.io/projected/0519e3a0-a23c-45c6-a45b-b913f4983532-kube-api-access-lt4sm\") on node \"crc\" DevicePath \"\"" Dec 03 12:39:21 crc kubenswrapper[5052]: I1203 12:39:21.682553 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/0519e3a0-a23c-45c6-a45b-b913f4983532-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:39:21 crc kubenswrapper[5052]: I1203 12:39:21.686225 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0519e3a0-a23c-45c6-a45b-b913f4983532-config-data" (OuterVolumeSpecName: "config-data") pod "0519e3a0-a23c-45c6-a45b-b913f4983532" (UID: "0519e3a0-a23c-45c6-a45b-b913f4983532"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:39:21 crc kubenswrapper[5052]: I1203 12:39:21.697912 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="93664241-e156-4225-980c-4dbe8aff3353" path="/var/lib/kubelet/pods/93664241-e156-4225-980c-4dbe8aff3353/volumes" Dec 03 12:39:21 crc kubenswrapper[5052]: I1203 12:39:21.702589 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 12:39:21 crc kubenswrapper[5052]: I1203 12:39:21.744157 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0519e3a0-a23c-45c6-a45b-b913f4983532-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "0519e3a0-a23c-45c6-a45b-b913f4983532" (UID: "0519e3a0-a23c-45c6-a45b-b913f4983532"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:39:21 crc kubenswrapper[5052]: I1203 12:39:21.756821 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0519e3a0-a23c-45c6-a45b-b913f4983532-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "0519e3a0-a23c-45c6-a45b-b913f4983532" (UID: "0519e3a0-a23c-45c6-a45b-b913f4983532"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:39:21 crc kubenswrapper[5052]: I1203 12:39:21.785165 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0519e3a0-a23c-45c6-a45b-b913f4983532-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:39:21 crc kubenswrapper[5052]: I1203 12:39:21.785211 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/0519e3a0-a23c-45c6-a45b-b913f4983532-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:39:21 crc kubenswrapper[5052]: I1203 12:39:21.785225 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/0519e3a0-a23c-45c6-a45b-b913f4983532-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:39:21 crc kubenswrapper[5052]: I1203 12:39:21.785237 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/0519e3a0-a23c-45c6-a45b-b913f4983532-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:39:22 crc kubenswrapper[5052]: I1203 12:39:22.415441 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-76bf6b6758-lvtjf" event={"ID":"0519e3a0-a23c-45c6-a45b-b913f4983532","Type":"ContainerDied","Data":"47656fe9f6350bd895b8133e6f1e7fa7c0d77e1477a3062813d0d11f8a69fee9"} Dec 03 12:39:22 crc kubenswrapper[5052]: I1203 12:39:22.415503 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-76bf6b6758-lvtjf" Dec 03 12:39:22 crc kubenswrapper[5052]: I1203 12:39:22.416722 5052 scope.go:117] "RemoveContainer" containerID="f388a5399432c5ccb31c169b3ad931f092c9dc3131b2e25657fd938dda7893db" Dec 03 12:39:22 crc kubenswrapper[5052]: I1203 12:39:22.417391 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"5bca7a3b-316e-4357-8ae4-e47be319dbf1","Type":"ContainerStarted","Data":"f4509a279a5438f6fb012b7e3044b3b95b5feb8f15f16222f2e627c559da6ac9"} Dec 03 12:39:22 crc kubenswrapper[5052]: I1203 12:39:22.468863 5052 scope.go:117] "RemoveContainer" containerID="9378b56786328ddd0407e98056ad23855b875424e631516f969dc644a1efb7e4" Dec 03 12:39:22 crc kubenswrapper[5052]: I1203 12:39:22.492488 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/placement-76bf6b6758-lvtjf"] Dec 03 12:39:22 crc kubenswrapper[5052]: I1203 12:39:22.503848 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/placement-76bf6b6758-lvtjf"] Dec 03 12:39:22 crc kubenswrapper[5052]: I1203 12:39:22.618155 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/memcached-0" Dec 03 12:39:22 crc kubenswrapper[5052]: I1203 12:39:22.768977 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:39:22 crc kubenswrapper[5052]: I1203 12:39:22.769259 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-metadata-0" podUID="edea8e53-340d-4f1e-99e5-082268c64ceb" containerName="nova-metadata-log" containerID="cri-o://ebef114a01de4517efb29228cf3e1863ae4862f9108584e0350c29841b2315c3" gracePeriod=30 Dec 03 12:39:22 crc kubenswrapper[5052]: I1203 12:39:22.771065 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-metadata-0" podUID="edea8e53-340d-4f1e-99e5-082268c64ceb" containerName="nova-metadata-metadata" containerID="cri-o://e60c32d29dd74ca93adc4e272d8de99081bbf8e5961fe48ef0a2093869b9682e" gracePeriod=30 Dec 03 12:39:22 crc kubenswrapper[5052]: I1203 12:39:22.902846 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/neutron-544c66dd6d-75q8t"] Dec 03 12:39:22 crc kubenswrapper[5052]: I1203 12:39:22.903069 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/neutron-544c66dd6d-75q8t" podUID="c34523d4-d634-4d76-a238-36566ef6a682" containerName="neutron-httpd" containerID="cri-o://575b5e45ed0a1dfb51c75c86c63be7eabec2113683e893c6ee22d23b04e78576" gracePeriod=30 Dec 03 12:39:22 crc kubenswrapper[5052]: I1203 12:39:22.903156 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/neutron-544c66dd6d-75q8t" podUID="c34523d4-d634-4d76-a238-36566ef6a682" containerName="neutron-api" containerID="cri-o://a71490515349faed4ab3d5147860c7b66eb2e343b2b37fce8896d878d5c4382f" gracePeriod=30 Dec 03 12:39:22 crc kubenswrapper[5052]: I1203 12:39:22.924362 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/neutron-544c66dd6d-75q8t" podUID="c34523d4-d634-4d76-a238-36566ef6a682" containerName="neutron-httpd" probeResult="failure" output="Get \"https://10.217.1.107:9696/\": EOF" Dec 03 12:39:22 crc kubenswrapper[5052]: I1203 12:39:22.951097 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/keystone-7b8cddb768-5snf9"] Dec 03 12:39:22 crc kubenswrapper[5052]: I1203 12:39:22.951652 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/keystone-7b8cddb768-5snf9" podUID="2972cf0c-5378-4e71-b308-0235608d5bef" containerName="keystone-api" containerID="cri-o://1e9c082372288c89143c066dc4a3ffdcb4a743df94ac5b3327dcb6b74a2be2a7" gracePeriod=30 Dec 03 12:39:23 crc kubenswrapper[5052]: I1203 12:39:23.008706 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/keystone-66f85f5bfb-vln92"] Dec 03 12:39:23 crc kubenswrapper[5052]: E1203 12:39:23.009299 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="0519e3a0-a23c-45c6-a45b-b913f4983532" containerName="placement-api" Dec 03 12:39:23 crc kubenswrapper[5052]: I1203 12:39:23.009324 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="0519e3a0-a23c-45c6-a45b-b913f4983532" containerName="placement-api" Dec 03 12:39:23 crc kubenswrapper[5052]: E1203 12:39:23.009386 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="0519e3a0-a23c-45c6-a45b-b913f4983532" containerName="placement-log" Dec 03 12:39:23 crc kubenswrapper[5052]: I1203 12:39:23.009400 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="0519e3a0-a23c-45c6-a45b-b913f4983532" containerName="placement-log" Dec 03 12:39:23 crc kubenswrapper[5052]: I1203 12:39:23.009863 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="0519e3a0-a23c-45c6-a45b-b913f4983532" containerName="placement-api" Dec 03 12:39:23 crc kubenswrapper[5052]: I1203 12:39:23.009903 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="0519e3a0-a23c-45c6-a45b-b913f4983532" containerName="placement-log" Dec 03 12:39:23 crc kubenswrapper[5052]: I1203 12:39:23.010862 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-66f85f5bfb-vln92" Dec 03 12:39:23 crc kubenswrapper[5052]: I1203 12:39:23.026570 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/neutron-844859d58b-w4m57"] Dec 03 12:39:23 crc kubenswrapper[5052]: I1203 12:39:23.028703 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-844859d58b-w4m57" Dec 03 12:39:23 crc kubenswrapper[5052]: I1203 12:39:23.057867 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-844859d58b-w4m57"] Dec 03 12:39:23 crc kubenswrapper[5052]: I1203 12:39:23.088416 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-66f85f5bfb-vln92"] Dec 03 12:39:23 crc kubenswrapper[5052]: I1203 12:39:23.092355 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/keystone-7b8cddb768-5snf9" podUID="2972cf0c-5378-4e71-b308-0235608d5bef" containerName="keystone-api" probeResult="failure" output="Get \"https://10.217.1.115:5000/v3\": EOF" Dec 03 12:39:23 crc kubenswrapper[5052]: I1203 12:39:23.113668 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/fdeb105e-9fbc-44bb-ba15-bc81c7275db1-combined-ca-bundle\") pod \"keystone-66f85f5bfb-vln92\" (UID: \"fdeb105e-9fbc-44bb-ba15-bc81c7275db1\") " pod="openstack-kuttl-tests/keystone-66f85f5bfb-vln92" Dec 03 12:39:23 crc kubenswrapper[5052]: I1203 12:39:23.113743 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/061f3d17-a2e8-4002-ad1a-220ba7b476e3-ovndb-tls-certs\") pod \"neutron-844859d58b-w4m57\" (UID: \"061f3d17-a2e8-4002-ad1a-220ba7b476e3\") " pod="openstack-kuttl-tests/neutron-844859d58b-w4m57" Dec 03 12:39:23 crc kubenswrapper[5052]: I1203 12:39:23.113767 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lrdgs\" (UniqueName: \"kubernetes.io/projected/061f3d17-a2e8-4002-ad1a-220ba7b476e3-kube-api-access-lrdgs\") pod \"neutron-844859d58b-w4m57\" (UID: \"061f3d17-a2e8-4002-ad1a-220ba7b476e3\") " pod="openstack-kuttl-tests/neutron-844859d58b-w4m57" Dec 03 12:39:23 crc kubenswrapper[5052]: I1203 12:39:23.113800 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x4vkn\" (UniqueName: \"kubernetes.io/projected/fdeb105e-9fbc-44bb-ba15-bc81c7275db1-kube-api-access-x4vkn\") pod \"keystone-66f85f5bfb-vln92\" (UID: \"fdeb105e-9fbc-44bb-ba15-bc81c7275db1\") " pod="openstack-kuttl-tests/keystone-66f85f5bfb-vln92" Dec 03 12:39:23 crc kubenswrapper[5052]: I1203 12:39:23.113830 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/fdeb105e-9fbc-44bb-ba15-bc81c7275db1-scripts\") pod \"keystone-66f85f5bfb-vln92\" (UID: \"fdeb105e-9fbc-44bb-ba15-bc81c7275db1\") " pod="openstack-kuttl-tests/keystone-66f85f5bfb-vln92" Dec 03 12:39:23 crc kubenswrapper[5052]: I1203 12:39:23.113862 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/fdeb105e-9fbc-44bb-ba15-bc81c7275db1-internal-tls-certs\") pod \"keystone-66f85f5bfb-vln92\" (UID: \"fdeb105e-9fbc-44bb-ba15-bc81c7275db1\") " pod="openstack-kuttl-tests/keystone-66f85f5bfb-vln92" Dec 03 12:39:23 crc kubenswrapper[5052]: I1203 12:39:23.113929 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/fdeb105e-9fbc-44bb-ba15-bc81c7275db1-config-data\") pod \"keystone-66f85f5bfb-vln92\" (UID: \"fdeb105e-9fbc-44bb-ba15-bc81c7275db1\") " pod="openstack-kuttl-tests/keystone-66f85f5bfb-vln92" Dec 03 12:39:23 crc kubenswrapper[5052]: I1203 12:39:23.113945 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/fdeb105e-9fbc-44bb-ba15-bc81c7275db1-credential-keys\") pod \"keystone-66f85f5bfb-vln92\" (UID: \"fdeb105e-9fbc-44bb-ba15-bc81c7275db1\") " pod="openstack-kuttl-tests/keystone-66f85f5bfb-vln92" Dec 03 12:39:23 crc kubenswrapper[5052]: I1203 12:39:23.113970 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/fdeb105e-9fbc-44bb-ba15-bc81c7275db1-public-tls-certs\") pod \"keystone-66f85f5bfb-vln92\" (UID: \"fdeb105e-9fbc-44bb-ba15-bc81c7275db1\") " pod="openstack-kuttl-tests/keystone-66f85f5bfb-vln92" Dec 03 12:39:23 crc kubenswrapper[5052]: I1203 12:39:23.113999 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/061f3d17-a2e8-4002-ad1a-220ba7b476e3-combined-ca-bundle\") pod \"neutron-844859d58b-w4m57\" (UID: \"061f3d17-a2e8-4002-ad1a-220ba7b476e3\") " pod="openstack-kuttl-tests/neutron-844859d58b-w4m57" Dec 03 12:39:23 crc kubenswrapper[5052]: I1203 12:39:23.114023 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/fdeb105e-9fbc-44bb-ba15-bc81c7275db1-fernet-keys\") pod \"keystone-66f85f5bfb-vln92\" (UID: \"fdeb105e-9fbc-44bb-ba15-bc81c7275db1\") " pod="openstack-kuttl-tests/keystone-66f85f5bfb-vln92" Dec 03 12:39:23 crc kubenswrapper[5052]: I1203 12:39:23.114042 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/061f3d17-a2e8-4002-ad1a-220ba7b476e3-public-tls-certs\") pod \"neutron-844859d58b-w4m57\" (UID: \"061f3d17-a2e8-4002-ad1a-220ba7b476e3\") " pod="openstack-kuttl-tests/neutron-844859d58b-w4m57" Dec 03 12:39:23 crc kubenswrapper[5052]: I1203 12:39:23.114060 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/061f3d17-a2e8-4002-ad1a-220ba7b476e3-internal-tls-certs\") pod \"neutron-844859d58b-w4m57\" (UID: \"061f3d17-a2e8-4002-ad1a-220ba7b476e3\") " pod="openstack-kuttl-tests/neutron-844859d58b-w4m57" Dec 03 12:39:23 crc kubenswrapper[5052]: I1203 12:39:23.114080 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/061f3d17-a2e8-4002-ad1a-220ba7b476e3-httpd-config\") pod \"neutron-844859d58b-w4m57\" (UID: \"061f3d17-a2e8-4002-ad1a-220ba7b476e3\") " pod="openstack-kuttl-tests/neutron-844859d58b-w4m57" Dec 03 12:39:23 crc kubenswrapper[5052]: I1203 12:39:23.114095 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/061f3d17-a2e8-4002-ad1a-220ba7b476e3-config\") pod \"neutron-844859d58b-w4m57\" (UID: \"061f3d17-a2e8-4002-ad1a-220ba7b476e3\") " pod="openstack-kuttl-tests/neutron-844859d58b-w4m57" Dec 03 12:39:23 crc kubenswrapper[5052]: I1203 12:39:23.216019 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/fdeb105e-9fbc-44bb-ba15-bc81c7275db1-config-data\") pod \"keystone-66f85f5bfb-vln92\" (UID: \"fdeb105e-9fbc-44bb-ba15-bc81c7275db1\") " pod="openstack-kuttl-tests/keystone-66f85f5bfb-vln92" Dec 03 12:39:23 crc kubenswrapper[5052]: I1203 12:39:23.216075 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/fdeb105e-9fbc-44bb-ba15-bc81c7275db1-credential-keys\") pod \"keystone-66f85f5bfb-vln92\" (UID: \"fdeb105e-9fbc-44bb-ba15-bc81c7275db1\") " pod="openstack-kuttl-tests/keystone-66f85f5bfb-vln92" Dec 03 12:39:23 crc kubenswrapper[5052]: I1203 12:39:23.216108 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/fdeb105e-9fbc-44bb-ba15-bc81c7275db1-public-tls-certs\") pod \"keystone-66f85f5bfb-vln92\" (UID: \"fdeb105e-9fbc-44bb-ba15-bc81c7275db1\") " pod="openstack-kuttl-tests/keystone-66f85f5bfb-vln92" Dec 03 12:39:23 crc kubenswrapper[5052]: I1203 12:39:23.216145 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/061f3d17-a2e8-4002-ad1a-220ba7b476e3-combined-ca-bundle\") pod \"neutron-844859d58b-w4m57\" (UID: \"061f3d17-a2e8-4002-ad1a-220ba7b476e3\") " pod="openstack-kuttl-tests/neutron-844859d58b-w4m57" Dec 03 12:39:23 crc kubenswrapper[5052]: I1203 12:39:23.216185 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/fdeb105e-9fbc-44bb-ba15-bc81c7275db1-fernet-keys\") pod \"keystone-66f85f5bfb-vln92\" (UID: \"fdeb105e-9fbc-44bb-ba15-bc81c7275db1\") " pod="openstack-kuttl-tests/keystone-66f85f5bfb-vln92" Dec 03 12:39:23 crc kubenswrapper[5052]: I1203 12:39:23.216206 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/061f3d17-a2e8-4002-ad1a-220ba7b476e3-public-tls-certs\") pod \"neutron-844859d58b-w4m57\" (UID: \"061f3d17-a2e8-4002-ad1a-220ba7b476e3\") " pod="openstack-kuttl-tests/neutron-844859d58b-w4m57" Dec 03 12:39:23 crc kubenswrapper[5052]: I1203 12:39:23.216225 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/061f3d17-a2e8-4002-ad1a-220ba7b476e3-internal-tls-certs\") pod \"neutron-844859d58b-w4m57\" (UID: \"061f3d17-a2e8-4002-ad1a-220ba7b476e3\") " pod="openstack-kuttl-tests/neutron-844859d58b-w4m57" Dec 03 12:39:23 crc kubenswrapper[5052]: I1203 12:39:23.216248 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/061f3d17-a2e8-4002-ad1a-220ba7b476e3-httpd-config\") pod \"neutron-844859d58b-w4m57\" (UID: \"061f3d17-a2e8-4002-ad1a-220ba7b476e3\") " pod="openstack-kuttl-tests/neutron-844859d58b-w4m57" Dec 03 12:39:23 crc kubenswrapper[5052]: I1203 12:39:23.216265 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/061f3d17-a2e8-4002-ad1a-220ba7b476e3-config\") pod \"neutron-844859d58b-w4m57\" (UID: \"061f3d17-a2e8-4002-ad1a-220ba7b476e3\") " pod="openstack-kuttl-tests/neutron-844859d58b-w4m57" Dec 03 12:39:23 crc kubenswrapper[5052]: I1203 12:39:23.216304 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/fdeb105e-9fbc-44bb-ba15-bc81c7275db1-combined-ca-bundle\") pod \"keystone-66f85f5bfb-vln92\" (UID: \"fdeb105e-9fbc-44bb-ba15-bc81c7275db1\") " pod="openstack-kuttl-tests/keystone-66f85f5bfb-vln92" Dec 03 12:39:23 crc kubenswrapper[5052]: I1203 12:39:23.216329 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/061f3d17-a2e8-4002-ad1a-220ba7b476e3-ovndb-tls-certs\") pod \"neutron-844859d58b-w4m57\" (UID: \"061f3d17-a2e8-4002-ad1a-220ba7b476e3\") " pod="openstack-kuttl-tests/neutron-844859d58b-w4m57" Dec 03 12:39:23 crc kubenswrapper[5052]: I1203 12:39:23.216353 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-lrdgs\" (UniqueName: \"kubernetes.io/projected/061f3d17-a2e8-4002-ad1a-220ba7b476e3-kube-api-access-lrdgs\") pod \"neutron-844859d58b-w4m57\" (UID: \"061f3d17-a2e8-4002-ad1a-220ba7b476e3\") " pod="openstack-kuttl-tests/neutron-844859d58b-w4m57" Dec 03 12:39:23 crc kubenswrapper[5052]: I1203 12:39:23.216388 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-x4vkn\" (UniqueName: \"kubernetes.io/projected/fdeb105e-9fbc-44bb-ba15-bc81c7275db1-kube-api-access-x4vkn\") pod \"keystone-66f85f5bfb-vln92\" (UID: \"fdeb105e-9fbc-44bb-ba15-bc81c7275db1\") " pod="openstack-kuttl-tests/keystone-66f85f5bfb-vln92" Dec 03 12:39:23 crc kubenswrapper[5052]: I1203 12:39:23.216441 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/fdeb105e-9fbc-44bb-ba15-bc81c7275db1-scripts\") pod \"keystone-66f85f5bfb-vln92\" (UID: \"fdeb105e-9fbc-44bb-ba15-bc81c7275db1\") " pod="openstack-kuttl-tests/keystone-66f85f5bfb-vln92" Dec 03 12:39:23 crc kubenswrapper[5052]: I1203 12:39:23.216484 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/fdeb105e-9fbc-44bb-ba15-bc81c7275db1-internal-tls-certs\") pod \"keystone-66f85f5bfb-vln92\" (UID: \"fdeb105e-9fbc-44bb-ba15-bc81c7275db1\") " pod="openstack-kuttl-tests/keystone-66f85f5bfb-vln92" Dec 03 12:39:23 crc kubenswrapper[5052]: I1203 12:39:23.223423 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/061f3d17-a2e8-4002-ad1a-220ba7b476e3-config\") pod \"neutron-844859d58b-w4m57\" (UID: \"061f3d17-a2e8-4002-ad1a-220ba7b476e3\") " pod="openstack-kuttl-tests/neutron-844859d58b-w4m57" Dec 03 12:39:23 crc kubenswrapper[5052]: I1203 12:39:23.228622 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/061f3d17-a2e8-4002-ad1a-220ba7b476e3-ovndb-tls-certs\") pod \"neutron-844859d58b-w4m57\" (UID: \"061f3d17-a2e8-4002-ad1a-220ba7b476e3\") " pod="openstack-kuttl-tests/neutron-844859d58b-w4m57" Dec 03 12:39:23 crc kubenswrapper[5052]: I1203 12:39:23.228806 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/061f3d17-a2e8-4002-ad1a-220ba7b476e3-public-tls-certs\") pod \"neutron-844859d58b-w4m57\" (UID: \"061f3d17-a2e8-4002-ad1a-220ba7b476e3\") " pod="openstack-kuttl-tests/neutron-844859d58b-w4m57" Dec 03 12:39:23 crc kubenswrapper[5052]: I1203 12:39:23.231942 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/fdeb105e-9fbc-44bb-ba15-bc81c7275db1-credential-keys\") pod \"keystone-66f85f5bfb-vln92\" (UID: \"fdeb105e-9fbc-44bb-ba15-bc81c7275db1\") " pod="openstack-kuttl-tests/keystone-66f85f5bfb-vln92" Dec 03 12:39:23 crc kubenswrapper[5052]: I1203 12:39:23.235073 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/fdeb105e-9fbc-44bb-ba15-bc81c7275db1-public-tls-certs\") pod \"keystone-66f85f5bfb-vln92\" (UID: \"fdeb105e-9fbc-44bb-ba15-bc81c7275db1\") " pod="openstack-kuttl-tests/keystone-66f85f5bfb-vln92" Dec 03 12:39:23 crc kubenswrapper[5052]: I1203 12:39:23.235757 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/fdeb105e-9fbc-44bb-ba15-bc81c7275db1-combined-ca-bundle\") pod \"keystone-66f85f5bfb-vln92\" (UID: \"fdeb105e-9fbc-44bb-ba15-bc81c7275db1\") " pod="openstack-kuttl-tests/keystone-66f85f5bfb-vln92" Dec 03 12:39:23 crc kubenswrapper[5052]: I1203 12:39:23.236764 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/fdeb105e-9fbc-44bb-ba15-bc81c7275db1-config-data\") pod \"keystone-66f85f5bfb-vln92\" (UID: \"fdeb105e-9fbc-44bb-ba15-bc81c7275db1\") " pod="openstack-kuttl-tests/keystone-66f85f5bfb-vln92" Dec 03 12:39:23 crc kubenswrapper[5052]: I1203 12:39:23.242205 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/fdeb105e-9fbc-44bb-ba15-bc81c7275db1-fernet-keys\") pod \"keystone-66f85f5bfb-vln92\" (UID: \"fdeb105e-9fbc-44bb-ba15-bc81c7275db1\") " pod="openstack-kuttl-tests/keystone-66f85f5bfb-vln92" Dec 03 12:39:23 crc kubenswrapper[5052]: I1203 12:39:23.243392 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/fdeb105e-9fbc-44bb-ba15-bc81c7275db1-scripts\") pod \"keystone-66f85f5bfb-vln92\" (UID: \"fdeb105e-9fbc-44bb-ba15-bc81c7275db1\") " pod="openstack-kuttl-tests/keystone-66f85f5bfb-vln92" Dec 03 12:39:23 crc kubenswrapper[5052]: I1203 12:39:23.244048 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/061f3d17-a2e8-4002-ad1a-220ba7b476e3-internal-tls-certs\") pod \"neutron-844859d58b-w4m57\" (UID: \"061f3d17-a2e8-4002-ad1a-220ba7b476e3\") " pod="openstack-kuttl-tests/neutron-844859d58b-w4m57" Dec 03 12:39:23 crc kubenswrapper[5052]: I1203 12:39:23.246935 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/fdeb105e-9fbc-44bb-ba15-bc81c7275db1-internal-tls-certs\") pod \"keystone-66f85f5bfb-vln92\" (UID: \"fdeb105e-9fbc-44bb-ba15-bc81c7275db1\") " pod="openstack-kuttl-tests/keystone-66f85f5bfb-vln92" Dec 03 12:39:23 crc kubenswrapper[5052]: I1203 12:39:23.252916 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-lrdgs\" (UniqueName: \"kubernetes.io/projected/061f3d17-a2e8-4002-ad1a-220ba7b476e3-kube-api-access-lrdgs\") pod \"neutron-844859d58b-w4m57\" (UID: \"061f3d17-a2e8-4002-ad1a-220ba7b476e3\") " pod="openstack-kuttl-tests/neutron-844859d58b-w4m57" Dec 03 12:39:23 crc kubenswrapper[5052]: I1203 12:39:23.258723 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-x4vkn\" (UniqueName: \"kubernetes.io/projected/fdeb105e-9fbc-44bb-ba15-bc81c7275db1-kube-api-access-x4vkn\") pod \"keystone-66f85f5bfb-vln92\" (UID: \"fdeb105e-9fbc-44bb-ba15-bc81c7275db1\") " pod="openstack-kuttl-tests/keystone-66f85f5bfb-vln92" Dec 03 12:39:23 crc kubenswrapper[5052]: I1203 12:39:23.264174 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/061f3d17-a2e8-4002-ad1a-220ba7b476e3-httpd-config\") pod \"neutron-844859d58b-w4m57\" (UID: \"061f3d17-a2e8-4002-ad1a-220ba7b476e3\") " pod="openstack-kuttl-tests/neutron-844859d58b-w4m57" Dec 03 12:39:23 crc kubenswrapper[5052]: I1203 12:39:23.266420 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/061f3d17-a2e8-4002-ad1a-220ba7b476e3-combined-ca-bundle\") pod \"neutron-844859d58b-w4m57\" (UID: \"061f3d17-a2e8-4002-ad1a-220ba7b476e3\") " pod="openstack-kuttl-tests/neutron-844859d58b-w4m57" Dec 03 12:39:23 crc kubenswrapper[5052]: I1203 12:39:23.309985 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:39:23 crc kubenswrapper[5052]: I1203 12:39:23.310477 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="59dcc834-7abb-4dff-a10f-6127276698ee" containerName="ceilometer-central-agent" containerID="cri-o://1537061d8a4905c4818df97a435c7501964181424870fcee0c47aed528717497" gracePeriod=30 Dec 03 12:39:23 crc kubenswrapper[5052]: I1203 12:39:23.310754 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="59dcc834-7abb-4dff-a10f-6127276698ee" containerName="proxy-httpd" containerID="cri-o://be761ab2be8f4c31fcd16de111710f2bfb1713804672136a770634cfde4d0e6b" gracePeriod=30 Dec 03 12:39:23 crc kubenswrapper[5052]: I1203 12:39:23.310837 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="59dcc834-7abb-4dff-a10f-6127276698ee" containerName="sg-core" containerID="cri-o://8694cf282fe33e5849ce697fe875d63a8f6e6daaa6c0de9a9adcdfe4163c63d3" gracePeriod=30 Dec 03 12:39:23 crc kubenswrapper[5052]: I1203 12:39:23.310951 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="59dcc834-7abb-4dff-a10f-6127276698ee" containerName="ceilometer-notification-agent" containerID="cri-o://fab91a75f95875869bda13fb508f1578ec0f067822bbe9b4d97201b302a40035" gracePeriod=30 Dec 03 12:39:23 crc kubenswrapper[5052]: I1203 12:39:23.345238 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:39:23 crc kubenswrapper[5052]: I1203 12:39:23.347830 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-66f85f5bfb-vln92" Dec 03 12:39:23 crc kubenswrapper[5052]: I1203 12:39:23.376467 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-844859d58b-w4m57" Dec 03 12:39:23 crc kubenswrapper[5052]: I1203 12:39:23.480537 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"edea8e53-340d-4f1e-99e5-082268c64ceb","Type":"ContainerDied","Data":"ebef114a01de4517efb29228cf3e1863ae4862f9108584e0350c29841b2315c3"} Dec 03 12:39:23 crc kubenswrapper[5052]: I1203 12:39:23.480714 5052 generic.go:334] "Generic (PLEG): container finished" podID="edea8e53-340d-4f1e-99e5-082268c64ceb" containerID="ebef114a01de4517efb29228cf3e1863ae4862f9108584e0350c29841b2315c3" exitCode=143 Dec 03 12:39:23 crc kubenswrapper[5052]: I1203 12:39:23.515254 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"5bca7a3b-316e-4357-8ae4-e47be319dbf1","Type":"ContainerStarted","Data":"15342f245f3b466f27835c2c30a7036b0d06092684492805d84db0575d5b165a"} Dec 03 12:39:23 crc kubenswrapper[5052]: I1203 12:39:23.712028 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0519e3a0-a23c-45c6-a45b-b913f4983532" path="/var/lib/kubelet/pods/0519e3a0-a23c-45c6-a45b-b913f4983532/volumes" Dec 03 12:39:23 crc kubenswrapper[5052]: I1203 12:39:23.722805 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/barbican-api-764dd4c57d-54dvn"] Dec 03 12:39:23 crc kubenswrapper[5052]: I1203 12:39:23.725122 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-api-764dd4c57d-54dvn" Dec 03 12:39:23 crc kubenswrapper[5052]: I1203 12:39:23.751654 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-api-764dd4c57d-54dvn"] Dec 03 12:39:23 crc kubenswrapper[5052]: E1203 12:39:23.843225 5052 cadvisor_stats_provider.go:516] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod59dcc834_7abb_4dff_a10f_6127276698ee.slice/crio-8694cf282fe33e5849ce697fe875d63a8f6e6daaa6c0de9a9adcdfe4163c63d3.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc34523d4_d634_4d76_a238_36566ef6a682.slice/crio-575b5e45ed0a1dfb51c75c86c63be7eabec2113683e893c6ee22d23b04e78576.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc34523d4_d634_4d76_a238_36566ef6a682.slice/crio-conmon-575b5e45ed0a1dfb51c75c86c63be7eabec2113683e893c6ee22d23b04e78576.scope\": RecentStats: unable to find data in memory cache]" Dec 03 12:39:23 crc kubenswrapper[5052]: I1203 12:39:23.847222 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/5da4fd9f-78ea-4511-ba09-f00694058c69-config-data-custom\") pod \"barbican-api-764dd4c57d-54dvn\" (UID: \"5da4fd9f-78ea-4511-ba09-f00694058c69\") " pod="openstack-kuttl-tests/barbican-api-764dd4c57d-54dvn" Dec 03 12:39:23 crc kubenswrapper[5052]: I1203 12:39:23.847355 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/5da4fd9f-78ea-4511-ba09-f00694058c69-config-data\") pod \"barbican-api-764dd4c57d-54dvn\" (UID: \"5da4fd9f-78ea-4511-ba09-f00694058c69\") " pod="openstack-kuttl-tests/barbican-api-764dd4c57d-54dvn" Dec 03 12:39:23 crc kubenswrapper[5052]: I1203 12:39:23.847590 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/5da4fd9f-78ea-4511-ba09-f00694058c69-public-tls-certs\") pod \"barbican-api-764dd4c57d-54dvn\" (UID: \"5da4fd9f-78ea-4511-ba09-f00694058c69\") " pod="openstack-kuttl-tests/barbican-api-764dd4c57d-54dvn" Dec 03 12:39:23 crc kubenswrapper[5052]: I1203 12:39:23.847671 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/5da4fd9f-78ea-4511-ba09-f00694058c69-logs\") pod \"barbican-api-764dd4c57d-54dvn\" (UID: \"5da4fd9f-78ea-4511-ba09-f00694058c69\") " pod="openstack-kuttl-tests/barbican-api-764dd4c57d-54dvn" Dec 03 12:39:23 crc kubenswrapper[5052]: I1203 12:39:23.847735 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5da4fd9f-78ea-4511-ba09-f00694058c69-combined-ca-bundle\") pod \"barbican-api-764dd4c57d-54dvn\" (UID: \"5da4fd9f-78ea-4511-ba09-f00694058c69\") " pod="openstack-kuttl-tests/barbican-api-764dd4c57d-54dvn" Dec 03 12:39:23 crc kubenswrapper[5052]: I1203 12:39:23.847820 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/5da4fd9f-78ea-4511-ba09-f00694058c69-internal-tls-certs\") pod \"barbican-api-764dd4c57d-54dvn\" (UID: \"5da4fd9f-78ea-4511-ba09-f00694058c69\") " pod="openstack-kuttl-tests/barbican-api-764dd4c57d-54dvn" Dec 03 12:39:23 crc kubenswrapper[5052]: I1203 12:39:23.847958 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j7ngh\" (UniqueName: \"kubernetes.io/projected/5da4fd9f-78ea-4511-ba09-f00694058c69-kube-api-access-j7ngh\") pod \"barbican-api-764dd4c57d-54dvn\" (UID: \"5da4fd9f-78ea-4511-ba09-f00694058c69\") " pod="openstack-kuttl-tests/barbican-api-764dd4c57d-54dvn" Dec 03 12:39:23 crc kubenswrapper[5052]: I1203 12:39:23.951832 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/5da4fd9f-78ea-4511-ba09-f00694058c69-public-tls-certs\") pod \"barbican-api-764dd4c57d-54dvn\" (UID: \"5da4fd9f-78ea-4511-ba09-f00694058c69\") " pod="openstack-kuttl-tests/barbican-api-764dd4c57d-54dvn" Dec 03 12:39:23 crc kubenswrapper[5052]: I1203 12:39:23.951918 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/5da4fd9f-78ea-4511-ba09-f00694058c69-logs\") pod \"barbican-api-764dd4c57d-54dvn\" (UID: \"5da4fd9f-78ea-4511-ba09-f00694058c69\") " pod="openstack-kuttl-tests/barbican-api-764dd4c57d-54dvn" Dec 03 12:39:23 crc kubenswrapper[5052]: I1203 12:39:23.951959 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5da4fd9f-78ea-4511-ba09-f00694058c69-combined-ca-bundle\") pod \"barbican-api-764dd4c57d-54dvn\" (UID: \"5da4fd9f-78ea-4511-ba09-f00694058c69\") " pod="openstack-kuttl-tests/barbican-api-764dd4c57d-54dvn" Dec 03 12:39:23 crc kubenswrapper[5052]: I1203 12:39:23.952001 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/5da4fd9f-78ea-4511-ba09-f00694058c69-internal-tls-certs\") pod \"barbican-api-764dd4c57d-54dvn\" (UID: \"5da4fd9f-78ea-4511-ba09-f00694058c69\") " pod="openstack-kuttl-tests/barbican-api-764dd4c57d-54dvn" Dec 03 12:39:23 crc kubenswrapper[5052]: I1203 12:39:23.952066 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-j7ngh\" (UniqueName: \"kubernetes.io/projected/5da4fd9f-78ea-4511-ba09-f00694058c69-kube-api-access-j7ngh\") pod \"barbican-api-764dd4c57d-54dvn\" (UID: \"5da4fd9f-78ea-4511-ba09-f00694058c69\") " pod="openstack-kuttl-tests/barbican-api-764dd4c57d-54dvn" Dec 03 12:39:23 crc kubenswrapper[5052]: I1203 12:39:23.952097 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/5da4fd9f-78ea-4511-ba09-f00694058c69-config-data-custom\") pod \"barbican-api-764dd4c57d-54dvn\" (UID: \"5da4fd9f-78ea-4511-ba09-f00694058c69\") " pod="openstack-kuttl-tests/barbican-api-764dd4c57d-54dvn" Dec 03 12:39:23 crc kubenswrapper[5052]: I1203 12:39:23.952140 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/5da4fd9f-78ea-4511-ba09-f00694058c69-config-data\") pod \"barbican-api-764dd4c57d-54dvn\" (UID: \"5da4fd9f-78ea-4511-ba09-f00694058c69\") " pod="openstack-kuttl-tests/barbican-api-764dd4c57d-54dvn" Dec 03 12:39:23 crc kubenswrapper[5052]: I1203 12:39:23.957107 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/5da4fd9f-78ea-4511-ba09-f00694058c69-logs\") pod \"barbican-api-764dd4c57d-54dvn\" (UID: \"5da4fd9f-78ea-4511-ba09-f00694058c69\") " pod="openstack-kuttl-tests/barbican-api-764dd4c57d-54dvn" Dec 03 12:39:23 crc kubenswrapper[5052]: I1203 12:39:23.959579 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5da4fd9f-78ea-4511-ba09-f00694058c69-combined-ca-bundle\") pod \"barbican-api-764dd4c57d-54dvn\" (UID: \"5da4fd9f-78ea-4511-ba09-f00694058c69\") " pod="openstack-kuttl-tests/barbican-api-764dd4c57d-54dvn" Dec 03 12:39:23 crc kubenswrapper[5052]: I1203 12:39:23.960642 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/5da4fd9f-78ea-4511-ba09-f00694058c69-config-data-custom\") pod \"barbican-api-764dd4c57d-54dvn\" (UID: \"5da4fd9f-78ea-4511-ba09-f00694058c69\") " pod="openstack-kuttl-tests/barbican-api-764dd4c57d-54dvn" Dec 03 12:39:23 crc kubenswrapper[5052]: I1203 12:39:23.964891 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/5da4fd9f-78ea-4511-ba09-f00694058c69-config-data\") pod \"barbican-api-764dd4c57d-54dvn\" (UID: \"5da4fd9f-78ea-4511-ba09-f00694058c69\") " pod="openstack-kuttl-tests/barbican-api-764dd4c57d-54dvn" Dec 03 12:39:23 crc kubenswrapper[5052]: I1203 12:39:23.965301 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/5da4fd9f-78ea-4511-ba09-f00694058c69-public-tls-certs\") pod \"barbican-api-764dd4c57d-54dvn\" (UID: \"5da4fd9f-78ea-4511-ba09-f00694058c69\") " pod="openstack-kuttl-tests/barbican-api-764dd4c57d-54dvn" Dec 03 12:39:23 crc kubenswrapper[5052]: I1203 12:39:23.967103 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/5da4fd9f-78ea-4511-ba09-f00694058c69-internal-tls-certs\") pod \"barbican-api-764dd4c57d-54dvn\" (UID: \"5da4fd9f-78ea-4511-ba09-f00694058c69\") " pod="openstack-kuttl-tests/barbican-api-764dd4c57d-54dvn" Dec 03 12:39:23 crc kubenswrapper[5052]: I1203 12:39:23.975089 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-j7ngh\" (UniqueName: \"kubernetes.io/projected/5da4fd9f-78ea-4511-ba09-f00694058c69-kube-api-access-j7ngh\") pod \"barbican-api-764dd4c57d-54dvn\" (UID: \"5da4fd9f-78ea-4511-ba09-f00694058c69\") " pod="openstack-kuttl-tests/barbican-api-764dd4c57d-54dvn" Dec 03 12:39:24 crc kubenswrapper[5052]: I1203 12:39:24.017382 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-66f85f5bfb-vln92"] Dec 03 12:39:24 crc kubenswrapper[5052]: I1203 12:39:24.107950 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-api-764dd4c57d-54dvn" Dec 03 12:39:24 crc kubenswrapper[5052]: W1203 12:39:24.226029 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod061f3d17_a2e8_4002_ad1a_220ba7b476e3.slice/crio-6a00688cb6fb7a411e0d2866a1bf6c5532d21c44366d609d23981a2b288dc5d6 WatchSource:0}: Error finding container 6a00688cb6fb7a411e0d2866a1bf6c5532d21c44366d609d23981a2b288dc5d6: Status 404 returned error can't find the container with id 6a00688cb6fb7a411e0d2866a1bf6c5532d21c44366d609d23981a2b288dc5d6 Dec 03 12:39:24 crc kubenswrapper[5052]: I1203 12:39:24.239235 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-844859d58b-w4m57"] Dec 03 12:39:24 crc kubenswrapper[5052]: I1203 12:39:24.529153 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"5bca7a3b-316e-4357-8ae4-e47be319dbf1","Type":"ContainerStarted","Data":"87d55b80754e67e2615d7cea19e5d57d82b054e86c70bcb48fc87f5b7efa28d9"} Dec 03 12:39:24 crc kubenswrapper[5052]: I1203 12:39:24.529522 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:39:24 crc kubenswrapper[5052]: I1203 12:39:24.532094 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_neutron-544c66dd6d-75q8t_c34523d4-d634-4d76-a238-36566ef6a682/neutron-api/0.log" Dec 03 12:39:24 crc kubenswrapper[5052]: I1203 12:39:24.532132 5052 generic.go:334] "Generic (PLEG): container finished" podID="c34523d4-d634-4d76-a238-36566ef6a682" containerID="575b5e45ed0a1dfb51c75c86c63be7eabec2113683e893c6ee22d23b04e78576" exitCode=0 Dec 03 12:39:24 crc kubenswrapper[5052]: I1203 12:39:24.532178 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-544c66dd6d-75q8t" event={"ID":"c34523d4-d634-4d76-a238-36566ef6a682","Type":"ContainerDied","Data":"575b5e45ed0a1dfb51c75c86c63be7eabec2113683e893c6ee22d23b04e78576"} Dec 03 12:39:24 crc kubenswrapper[5052]: I1203 12:39:24.534901 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-66f85f5bfb-vln92" event={"ID":"fdeb105e-9fbc-44bb-ba15-bc81c7275db1","Type":"ContainerStarted","Data":"303e4e73500eec972e0823935bb4cf2cb918b5d2b159c46edd8b37894c6d37f1"} Dec 03 12:39:24 crc kubenswrapper[5052]: I1203 12:39:24.534937 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-66f85f5bfb-vln92" event={"ID":"fdeb105e-9fbc-44bb-ba15-bc81c7275db1","Type":"ContainerStarted","Data":"1c38b16a3c4c2e0c47a3b8e80727876a669b7b76c048e848f87a8d9ee3611ea9"} Dec 03 12:39:24 crc kubenswrapper[5052]: I1203 12:39:24.534994 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/keystone-66f85f5bfb-vln92" Dec 03 12:39:24 crc kubenswrapper[5052]: I1203 12:39:24.537109 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-844859d58b-w4m57" event={"ID":"061f3d17-a2e8-4002-ad1a-220ba7b476e3","Type":"ContainerStarted","Data":"6a00688cb6fb7a411e0d2866a1bf6c5532d21c44366d609d23981a2b288dc5d6"} Dec 03 12:39:24 crc kubenswrapper[5052]: I1203 12:39:24.540137 5052 generic.go:334] "Generic (PLEG): container finished" podID="59dcc834-7abb-4dff-a10f-6127276698ee" containerID="be761ab2be8f4c31fcd16de111710f2bfb1713804672136a770634cfde4d0e6b" exitCode=0 Dec 03 12:39:24 crc kubenswrapper[5052]: I1203 12:39:24.540287 5052 generic.go:334] "Generic (PLEG): container finished" podID="59dcc834-7abb-4dff-a10f-6127276698ee" containerID="8694cf282fe33e5849ce697fe875d63a8f6e6daaa6c0de9a9adcdfe4163c63d3" exitCode=2 Dec 03 12:39:24 crc kubenswrapper[5052]: I1203 12:39:24.540379 5052 generic.go:334] "Generic (PLEG): container finished" podID="59dcc834-7abb-4dff-a10f-6127276698ee" containerID="1537061d8a4905c4818df97a435c7501964181424870fcee0c47aed528717497" exitCode=0 Dec 03 12:39:24 crc kubenswrapper[5052]: I1203 12:39:24.540220 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"59dcc834-7abb-4dff-a10f-6127276698ee","Type":"ContainerDied","Data":"be761ab2be8f4c31fcd16de111710f2bfb1713804672136a770634cfde4d0e6b"} Dec 03 12:39:24 crc kubenswrapper[5052]: I1203 12:39:24.540554 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"59dcc834-7abb-4dff-a10f-6127276698ee","Type":"ContainerDied","Data":"8694cf282fe33e5849ce697fe875d63a8f6e6daaa6c0de9a9adcdfe4163c63d3"} Dec 03 12:39:24 crc kubenswrapper[5052]: I1203 12:39:24.540642 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"59dcc834-7abb-4dff-a10f-6127276698ee","Type":"ContainerDied","Data":"1537061d8a4905c4818df97a435c7501964181424870fcee0c47aed528717497"} Dec 03 12:39:24 crc kubenswrapper[5052]: I1203 12:39:24.554894 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:39:24 crc kubenswrapper[5052]: I1203 12:39:24.555211 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-external-api-0" podUID="02726407-aa1e-400f-aedb-78d896c83113" containerName="glance-log" containerID="cri-o://df1cbd0a5a0599784a42f96943876ae3db85b427e8a37874b50da6af3c45e953" gracePeriod=30 Dec 03 12:39:24 crc kubenswrapper[5052]: I1203 12:39:24.555265 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-external-api-0" podUID="02726407-aa1e-400f-aedb-78d896c83113" containerName="glance-httpd" containerID="cri-o://4407c34f059e21949a882b9b730e10be07f5e7b9fb9786cf9259647fe69f7a12" gracePeriod=30 Dec 03 12:39:24 crc kubenswrapper[5052]: I1203 12:39:24.560038 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/cinder-api-0" podStartSLOduration=4.560009988 podStartE2EDuration="4.560009988s" podCreationTimestamp="2025-12-03 12:39:20 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:39:24.549997482 +0000 UTC m=+4755.138239266" watchObservedRunningTime="2025-12-03 12:39:24.560009988 +0000 UTC m=+4755.148251752" Dec 03 12:39:24 crc kubenswrapper[5052]: I1203 12:39:24.583906 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/keystone-66f85f5bfb-vln92" podStartSLOduration=2.583886741 podStartE2EDuration="2.583886741s" podCreationTimestamp="2025-12-03 12:39:22 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:39:24.574717879 +0000 UTC m=+4755.162959643" watchObservedRunningTime="2025-12-03 12:39:24.583886741 +0000 UTC m=+4755.172128505" Dec 03 12:39:24 crc kubenswrapper[5052]: I1203 12:39:24.640620 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:39:24 crc kubenswrapper[5052]: I1203 12:39:24.640880 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-internal-api-0" podUID="5d4a61d7-2d2c-4186-a923-f59636bc0994" containerName="glance-log" containerID="cri-o://2981d01a1325fdf3537f8cf2d9ade2af4707530bbb2ce6666c3b3503c416127d" gracePeriod=30 Dec 03 12:39:24 crc kubenswrapper[5052]: I1203 12:39:24.641051 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-internal-api-0" podUID="5d4a61d7-2d2c-4186-a923-f59636bc0994" containerName="glance-httpd" containerID="cri-o://7ebd36be6c9083bc844d5b32e9c85265741962dbf955670e0381dd41fb13e4d3" gracePeriod=30 Dec 03 12:39:24 crc kubenswrapper[5052]: W1203 12:39:24.725627 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5da4fd9f_78ea_4511_ba09_f00694058c69.slice/crio-bb4ec90598abda1cb351aa0e21694297b64993fc4668ed9a7635f1e7e076ad07 WatchSource:0}: Error finding container bb4ec90598abda1cb351aa0e21694297b64993fc4668ed9a7635f1e7e076ad07: Status 404 returned error can't find the container with id bb4ec90598abda1cb351aa0e21694297b64993fc4668ed9a7635f1e7e076ad07 Dec 03 12:39:24 crc kubenswrapper[5052]: I1203 12:39:24.727910 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-api-764dd4c57d-54dvn"] Dec 03 12:39:24 crc kubenswrapper[5052]: I1203 12:39:24.766784 5052 scope.go:117] "RemoveContainer" containerID="7a510d263ec955ad2d5ebb00bcc48080e47a93e217cc757ad503fbe9cf666300" Dec 03 12:39:25 crc kubenswrapper[5052]: I1203 12:39:25.101307 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/cinder-api-0" podUID="93664241-e156-4225-980c-4dbe8aff3353" containerName="cinder-api" probeResult="failure" output="Get \"https://10.217.1.104:8776/healthcheck\": dial tcp 10.217.1.104:8776: i/o timeout" Dec 03 12:39:25 crc kubenswrapper[5052]: I1203 12:39:25.401691 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_neutron-544c66dd6d-75q8t_c34523d4-d634-4d76-a238-36566ef6a682/neutron-api/0.log" Dec 03 12:39:25 crc kubenswrapper[5052]: I1203 12:39:25.401773 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-544c66dd6d-75q8t" Dec 03 12:39:25 crc kubenswrapper[5052]: I1203 12:39:25.490798 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-api-764dd4c57d-54dvn"] Dec 03 12:39:25 crc kubenswrapper[5052]: I1203 12:39:25.500060 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/barbican-api-bcdbf47bb-m2ctz"] Dec 03 12:39:25 crc kubenswrapper[5052]: E1203 12:39:25.500475 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="c34523d4-d634-4d76-a238-36566ef6a682" containerName="neutron-httpd" Dec 03 12:39:25 crc kubenswrapper[5052]: I1203 12:39:25.500493 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="c34523d4-d634-4d76-a238-36566ef6a682" containerName="neutron-httpd" Dec 03 12:39:25 crc kubenswrapper[5052]: E1203 12:39:25.500524 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="c34523d4-d634-4d76-a238-36566ef6a682" containerName="neutron-api" Dec 03 12:39:25 crc kubenswrapper[5052]: I1203 12:39:25.500531 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="c34523d4-d634-4d76-a238-36566ef6a682" containerName="neutron-api" Dec 03 12:39:25 crc kubenswrapper[5052]: E1203 12:39:25.500551 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="c34523d4-d634-4d76-a238-36566ef6a682" containerName="neutron-api" Dec 03 12:39:25 crc kubenswrapper[5052]: I1203 12:39:25.500559 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="c34523d4-d634-4d76-a238-36566ef6a682" containerName="neutron-api" Dec 03 12:39:25 crc kubenswrapper[5052]: I1203 12:39:25.500644 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/c34523d4-d634-4d76-a238-36566ef6a682-internal-tls-certs\") pod \"c34523d4-d634-4d76-a238-36566ef6a682\" (UID: \"c34523d4-d634-4d76-a238-36566ef6a682\") " Dec 03 12:39:25 crc kubenswrapper[5052]: I1203 12:39:25.500704 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/c34523d4-d634-4d76-a238-36566ef6a682-httpd-config\") pod \"c34523d4-d634-4d76-a238-36566ef6a682\" (UID: \"c34523d4-d634-4d76-a238-36566ef6a682\") " Dec 03 12:39:25 crc kubenswrapper[5052]: I1203 12:39:25.500751 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8zvdv\" (UniqueName: \"kubernetes.io/projected/c34523d4-d634-4d76-a238-36566ef6a682-kube-api-access-8zvdv\") pod \"c34523d4-d634-4d76-a238-36566ef6a682\" (UID: \"c34523d4-d634-4d76-a238-36566ef6a682\") " Dec 03 12:39:25 crc kubenswrapper[5052]: I1203 12:39:25.500763 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="c34523d4-d634-4d76-a238-36566ef6a682" containerName="neutron-api" Dec 03 12:39:25 crc kubenswrapper[5052]: I1203 12:39:25.500778 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="c34523d4-d634-4d76-a238-36566ef6a682" containerName="neutron-api" Dec 03 12:39:25 crc kubenswrapper[5052]: I1203 12:39:25.500786 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="c34523d4-d634-4d76-a238-36566ef6a682" containerName="neutron-httpd" Dec 03 12:39:25 crc kubenswrapper[5052]: I1203 12:39:25.500863 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c34523d4-d634-4d76-a238-36566ef6a682-combined-ca-bundle\") pod \"c34523d4-d634-4d76-a238-36566ef6a682\" (UID: \"c34523d4-d634-4d76-a238-36566ef6a682\") " Dec 03 12:39:25 crc kubenswrapper[5052]: I1203 12:39:25.500907 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/c34523d4-d634-4d76-a238-36566ef6a682-config\") pod \"c34523d4-d634-4d76-a238-36566ef6a682\" (UID: \"c34523d4-d634-4d76-a238-36566ef6a682\") " Dec 03 12:39:25 crc kubenswrapper[5052]: I1203 12:39:25.500944 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/c34523d4-d634-4d76-a238-36566ef6a682-ovndb-tls-certs\") pod \"c34523d4-d634-4d76-a238-36566ef6a682\" (UID: \"c34523d4-d634-4d76-a238-36566ef6a682\") " Dec 03 12:39:25 crc kubenswrapper[5052]: I1203 12:39:25.500986 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/c34523d4-d634-4d76-a238-36566ef6a682-public-tls-certs\") pod \"c34523d4-d634-4d76-a238-36566ef6a682\" (UID: \"c34523d4-d634-4d76-a238-36566ef6a682\") " Dec 03 12:39:25 crc kubenswrapper[5052]: I1203 12:39:25.502444 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-api-bcdbf47bb-m2ctz" Dec 03 12:39:25 crc kubenswrapper[5052]: I1203 12:39:25.522069 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c34523d4-d634-4d76-a238-36566ef6a682-httpd-config" (OuterVolumeSpecName: "httpd-config") pod "c34523d4-d634-4d76-a238-36566ef6a682" (UID: "c34523d4-d634-4d76-a238-36566ef6a682"). InnerVolumeSpecName "httpd-config". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:39:25 crc kubenswrapper[5052]: I1203 12:39:25.537258 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-api-bcdbf47bb-m2ctz"] Dec 03 12:39:25 crc kubenswrapper[5052]: I1203 12:39:25.555862 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c34523d4-d634-4d76-a238-36566ef6a682-kube-api-access-8zvdv" (OuterVolumeSpecName: "kube-api-access-8zvdv") pod "c34523d4-d634-4d76-a238-36566ef6a682" (UID: "c34523d4-d634-4d76-a238-36566ef6a682"). InnerVolumeSpecName "kube-api-access-8zvdv". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:39:25 crc kubenswrapper[5052]: I1203 12:39:25.595958 5052 generic.go:334] "Generic (PLEG): container finished" podID="c34523d4-d634-4d76-a238-36566ef6a682" containerID="a71490515349faed4ab3d5147860c7b66eb2e343b2b37fce8896d878d5c4382f" exitCode=0 Dec 03 12:39:25 crc kubenswrapper[5052]: I1203 12:39:25.596292 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-544c66dd6d-75q8t" Dec 03 12:39:25 crc kubenswrapper[5052]: I1203 12:39:25.596131 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-544c66dd6d-75q8t" event={"ID":"c34523d4-d634-4d76-a238-36566ef6a682","Type":"ContainerDied","Data":"a71490515349faed4ab3d5147860c7b66eb2e343b2b37fce8896d878d5c4382f"} Dec 03 12:39:25 crc kubenswrapper[5052]: I1203 12:39:25.596533 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-544c66dd6d-75q8t" event={"ID":"c34523d4-d634-4d76-a238-36566ef6a682","Type":"ContainerDied","Data":"2da7df7c61cec0233bc5232f83d4c248600d2c0d8ce92aed8c21d58a826fea98"} Dec 03 12:39:25 crc kubenswrapper[5052]: I1203 12:39:25.596582 5052 scope.go:117] "RemoveContainer" containerID="a71490515349faed4ab3d5147860c7b66eb2e343b2b37fce8896d878d5c4382f" Dec 03 12:39:25 crc kubenswrapper[5052]: I1203 12:39:25.602869 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-844859d58b-w4m57" event={"ID":"061f3d17-a2e8-4002-ad1a-220ba7b476e3","Type":"ContainerStarted","Data":"5666f6ec09a1830c252db608dbb8ad35ec7dabe19b787b488fb6bf64296bffdf"} Dec 03 12:39:25 crc kubenswrapper[5052]: I1203 12:39:25.602925 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-844859d58b-w4m57" event={"ID":"061f3d17-a2e8-4002-ad1a-220ba7b476e3","Type":"ContainerStarted","Data":"acb084fda076785e51f840ee0c818b9e5cd206bb5a416282d7f91fb92bbb2890"} Dec 03 12:39:25 crc kubenswrapper[5052]: I1203 12:39:25.604127 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/neutron-844859d58b-w4m57" Dec 03 12:39:25 crc kubenswrapper[5052]: I1203 12:39:25.610414 5052 reconciler_common.go:293] "Volume detached for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/c34523d4-d634-4d76-a238-36566ef6a682-httpd-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:39:25 crc kubenswrapper[5052]: I1203 12:39:25.610452 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-8zvdv\" (UniqueName: \"kubernetes.io/projected/c34523d4-d634-4d76-a238-36566ef6a682-kube-api-access-8zvdv\") on node \"crc\" DevicePath \"\"" Dec 03 12:39:25 crc kubenswrapper[5052]: I1203 12:39:25.611142 5052 generic.go:334] "Generic (PLEG): container finished" podID="5d4a61d7-2d2c-4186-a923-f59636bc0994" containerID="2981d01a1325fdf3537f8cf2d9ade2af4707530bbb2ce6666c3b3503c416127d" exitCode=143 Dec 03 12:39:25 crc kubenswrapper[5052]: I1203 12:39:25.611239 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"5d4a61d7-2d2c-4186-a923-f59636bc0994","Type":"ContainerDied","Data":"2981d01a1325fdf3537f8cf2d9ade2af4707530bbb2ce6666c3b3503c416127d"} Dec 03 12:39:25 crc kubenswrapper[5052]: I1203 12:39:25.620862 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-764dd4c57d-54dvn" event={"ID":"5da4fd9f-78ea-4511-ba09-f00694058c69","Type":"ContainerStarted","Data":"149b09f0ffc6c75d49bcdf2891249bda90b68eaca953e917617a0f58c6463c58"} Dec 03 12:39:25 crc kubenswrapper[5052]: I1203 12:39:25.620929 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-764dd4c57d-54dvn" event={"ID":"5da4fd9f-78ea-4511-ba09-f00694058c69","Type":"ContainerStarted","Data":"276d7d9067803b5433ccd1312132cf50f0afd9d72333fbe834713799a5eb7360"} Dec 03 12:39:25 crc kubenswrapper[5052]: I1203 12:39:25.620966 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/barbican-api-764dd4c57d-54dvn" Dec 03 12:39:25 crc kubenswrapper[5052]: I1203 12:39:25.620982 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/barbican-api-764dd4c57d-54dvn" Dec 03 12:39:25 crc kubenswrapper[5052]: I1203 12:39:25.620992 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-764dd4c57d-54dvn" event={"ID":"5da4fd9f-78ea-4511-ba09-f00694058c69","Type":"ContainerStarted","Data":"bb4ec90598abda1cb351aa0e21694297b64993fc4668ed9a7635f1e7e076ad07"} Dec 03 12:39:25 crc kubenswrapper[5052]: I1203 12:39:25.628385 5052 generic.go:334] "Generic (PLEG): container finished" podID="02726407-aa1e-400f-aedb-78d896c83113" containerID="df1cbd0a5a0599784a42f96943876ae3db85b427e8a37874b50da6af3c45e953" exitCode=143 Dec 03 12:39:25 crc kubenswrapper[5052]: I1203 12:39:25.628600 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"02726407-aa1e-400f-aedb-78d896c83113","Type":"ContainerDied","Data":"df1cbd0a5a0599784a42f96943876ae3db85b427e8a37874b50da6af3c45e953"} Dec 03 12:39:25 crc kubenswrapper[5052]: I1203 12:39:25.681343 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/neutron-844859d58b-w4m57" podStartSLOduration=3.681324556 podStartE2EDuration="3.681324556s" podCreationTimestamp="2025-12-03 12:39:22 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:39:25.663142986 +0000 UTC m=+4756.251384750" watchObservedRunningTime="2025-12-03 12:39:25.681324556 +0000 UTC m=+4756.269566320" Dec 03 12:39:25 crc kubenswrapper[5052]: I1203 12:39:25.683534 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c34523d4-d634-4d76-a238-36566ef6a682-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "c34523d4-d634-4d76-a238-36566ef6a682" (UID: "c34523d4-d634-4d76-a238-36566ef6a682"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:39:25 crc kubenswrapper[5052]: I1203 12:39:25.697132 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c34523d4-d634-4d76-a238-36566ef6a682-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "c34523d4-d634-4d76-a238-36566ef6a682" (UID: "c34523d4-d634-4d76-a238-36566ef6a682"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:39:25 crc kubenswrapper[5052]: I1203 12:39:25.702234 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/barbican-api-764dd4c57d-54dvn" podStartSLOduration=2.702213124 podStartE2EDuration="2.702213124s" podCreationTimestamp="2025-12-03 12:39:23 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:39:25.689672305 +0000 UTC m=+4756.277914069" watchObservedRunningTime="2025-12-03 12:39:25.702213124 +0000 UTC m=+4756.290454898" Dec 03 12:39:25 crc kubenswrapper[5052]: I1203 12:39:25.713159 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a83592aa-a4c1-459e-9d07-f14b390a5e6b-config-data\") pod \"barbican-api-bcdbf47bb-m2ctz\" (UID: \"a83592aa-a4c1-459e-9d07-f14b390a5e6b\") " pod="openstack-kuttl-tests/barbican-api-bcdbf47bb-m2ctz" Dec 03 12:39:25 crc kubenswrapper[5052]: I1203 12:39:25.713280 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/a83592aa-a4c1-459e-9d07-f14b390a5e6b-internal-tls-certs\") pod \"barbican-api-bcdbf47bb-m2ctz\" (UID: \"a83592aa-a4c1-459e-9d07-f14b390a5e6b\") " pod="openstack-kuttl-tests/barbican-api-bcdbf47bb-m2ctz" Dec 03 12:39:25 crc kubenswrapper[5052]: I1203 12:39:25.714214 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/a83592aa-a4c1-459e-9d07-f14b390a5e6b-logs\") pod \"barbican-api-bcdbf47bb-m2ctz\" (UID: \"a83592aa-a4c1-459e-9d07-f14b390a5e6b\") " pod="openstack-kuttl-tests/barbican-api-bcdbf47bb-m2ctz" Dec 03 12:39:25 crc kubenswrapper[5052]: I1203 12:39:25.714565 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/a83592aa-a4c1-459e-9d07-f14b390a5e6b-config-data-custom\") pod \"barbican-api-bcdbf47bb-m2ctz\" (UID: \"a83592aa-a4c1-459e-9d07-f14b390a5e6b\") " pod="openstack-kuttl-tests/barbican-api-bcdbf47bb-m2ctz" Dec 03 12:39:25 crc kubenswrapper[5052]: I1203 12:39:25.714621 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tk6xf\" (UniqueName: \"kubernetes.io/projected/a83592aa-a4c1-459e-9d07-f14b390a5e6b-kube-api-access-tk6xf\") pod \"barbican-api-bcdbf47bb-m2ctz\" (UID: \"a83592aa-a4c1-459e-9d07-f14b390a5e6b\") " pod="openstack-kuttl-tests/barbican-api-bcdbf47bb-m2ctz" Dec 03 12:39:25 crc kubenswrapper[5052]: I1203 12:39:25.714716 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a83592aa-a4c1-459e-9d07-f14b390a5e6b-combined-ca-bundle\") pod \"barbican-api-bcdbf47bb-m2ctz\" (UID: \"a83592aa-a4c1-459e-9d07-f14b390a5e6b\") " pod="openstack-kuttl-tests/barbican-api-bcdbf47bb-m2ctz" Dec 03 12:39:25 crc kubenswrapper[5052]: I1203 12:39:25.714740 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/a83592aa-a4c1-459e-9d07-f14b390a5e6b-public-tls-certs\") pod \"barbican-api-bcdbf47bb-m2ctz\" (UID: \"a83592aa-a4c1-459e-9d07-f14b390a5e6b\") " pod="openstack-kuttl-tests/barbican-api-bcdbf47bb-m2ctz" Dec 03 12:39:25 crc kubenswrapper[5052]: I1203 12:39:25.714836 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/c34523d4-d634-4d76-a238-36566ef6a682-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:39:25 crc kubenswrapper[5052]: I1203 12:39:25.714851 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c34523d4-d634-4d76-a238-36566ef6a682-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:39:25 crc kubenswrapper[5052]: I1203 12:39:25.718485 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c34523d4-d634-4d76-a238-36566ef6a682-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "c34523d4-d634-4d76-a238-36566ef6a682" (UID: "c34523d4-d634-4d76-a238-36566ef6a682"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:39:25 crc kubenswrapper[5052]: I1203 12:39:25.749764 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c34523d4-d634-4d76-a238-36566ef6a682-config" (OuterVolumeSpecName: "config") pod "c34523d4-d634-4d76-a238-36566ef6a682" (UID: "c34523d4-d634-4d76-a238-36566ef6a682"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:39:25 crc kubenswrapper[5052]: I1203 12:39:25.749810 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c34523d4-d634-4d76-a238-36566ef6a682-ovndb-tls-certs" (OuterVolumeSpecName: "ovndb-tls-certs") pod "c34523d4-d634-4d76-a238-36566ef6a682" (UID: "c34523d4-d634-4d76-a238-36566ef6a682"). InnerVolumeSpecName "ovndb-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:39:25 crc kubenswrapper[5052]: I1203 12:39:25.819282 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/a83592aa-a4c1-459e-9d07-f14b390a5e6b-config-data-custom\") pod \"barbican-api-bcdbf47bb-m2ctz\" (UID: \"a83592aa-a4c1-459e-9d07-f14b390a5e6b\") " pod="openstack-kuttl-tests/barbican-api-bcdbf47bb-m2ctz" Dec 03 12:39:25 crc kubenswrapper[5052]: I1203 12:39:25.819363 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-tk6xf\" (UniqueName: \"kubernetes.io/projected/a83592aa-a4c1-459e-9d07-f14b390a5e6b-kube-api-access-tk6xf\") pod \"barbican-api-bcdbf47bb-m2ctz\" (UID: \"a83592aa-a4c1-459e-9d07-f14b390a5e6b\") " pod="openstack-kuttl-tests/barbican-api-bcdbf47bb-m2ctz" Dec 03 12:39:25 crc kubenswrapper[5052]: I1203 12:39:25.819462 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a83592aa-a4c1-459e-9d07-f14b390a5e6b-combined-ca-bundle\") pod \"barbican-api-bcdbf47bb-m2ctz\" (UID: \"a83592aa-a4c1-459e-9d07-f14b390a5e6b\") " pod="openstack-kuttl-tests/barbican-api-bcdbf47bb-m2ctz" Dec 03 12:39:25 crc kubenswrapper[5052]: I1203 12:39:25.819491 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/a83592aa-a4c1-459e-9d07-f14b390a5e6b-public-tls-certs\") pod \"barbican-api-bcdbf47bb-m2ctz\" (UID: \"a83592aa-a4c1-459e-9d07-f14b390a5e6b\") " pod="openstack-kuttl-tests/barbican-api-bcdbf47bb-m2ctz" Dec 03 12:39:25 crc kubenswrapper[5052]: I1203 12:39:25.819579 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a83592aa-a4c1-459e-9d07-f14b390a5e6b-config-data\") pod \"barbican-api-bcdbf47bb-m2ctz\" (UID: \"a83592aa-a4c1-459e-9d07-f14b390a5e6b\") " pod="openstack-kuttl-tests/barbican-api-bcdbf47bb-m2ctz" Dec 03 12:39:25 crc kubenswrapper[5052]: I1203 12:39:25.819621 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/a83592aa-a4c1-459e-9d07-f14b390a5e6b-internal-tls-certs\") pod \"barbican-api-bcdbf47bb-m2ctz\" (UID: \"a83592aa-a4c1-459e-9d07-f14b390a5e6b\") " pod="openstack-kuttl-tests/barbican-api-bcdbf47bb-m2ctz" Dec 03 12:39:25 crc kubenswrapper[5052]: I1203 12:39:25.819678 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/a83592aa-a4c1-459e-9d07-f14b390a5e6b-logs\") pod \"barbican-api-bcdbf47bb-m2ctz\" (UID: \"a83592aa-a4c1-459e-9d07-f14b390a5e6b\") " pod="openstack-kuttl-tests/barbican-api-bcdbf47bb-m2ctz" Dec 03 12:39:25 crc kubenswrapper[5052]: I1203 12:39:25.819967 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/c34523d4-d634-4d76-a238-36566ef6a682-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:39:25 crc kubenswrapper[5052]: I1203 12:39:25.819991 5052 reconciler_common.go:293] "Volume detached for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/c34523d4-d634-4d76-a238-36566ef6a682-ovndb-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:39:25 crc kubenswrapper[5052]: I1203 12:39:25.820005 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/c34523d4-d634-4d76-a238-36566ef6a682-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:39:25 crc kubenswrapper[5052]: I1203 12:39:25.821871 5052 scope.go:117] "RemoveContainer" containerID="575b5e45ed0a1dfb51c75c86c63be7eabec2113683e893c6ee22d23b04e78576" Dec 03 12:39:25 crc kubenswrapper[5052]: I1203 12:39:25.826396 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/a83592aa-a4c1-459e-9d07-f14b390a5e6b-config-data-custom\") pod \"barbican-api-bcdbf47bb-m2ctz\" (UID: \"a83592aa-a4c1-459e-9d07-f14b390a5e6b\") " pod="openstack-kuttl-tests/barbican-api-bcdbf47bb-m2ctz" Dec 03 12:39:25 crc kubenswrapper[5052]: I1203 12:39:25.826952 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/a83592aa-a4c1-459e-9d07-f14b390a5e6b-logs\") pod \"barbican-api-bcdbf47bb-m2ctz\" (UID: \"a83592aa-a4c1-459e-9d07-f14b390a5e6b\") " pod="openstack-kuttl-tests/barbican-api-bcdbf47bb-m2ctz" Dec 03 12:39:25 crc kubenswrapper[5052]: I1203 12:39:25.836774 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/a83592aa-a4c1-459e-9d07-f14b390a5e6b-internal-tls-certs\") pod \"barbican-api-bcdbf47bb-m2ctz\" (UID: \"a83592aa-a4c1-459e-9d07-f14b390a5e6b\") " pod="openstack-kuttl-tests/barbican-api-bcdbf47bb-m2ctz" Dec 03 12:39:25 crc kubenswrapper[5052]: I1203 12:39:25.837939 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/a83592aa-a4c1-459e-9d07-f14b390a5e6b-public-tls-certs\") pod \"barbican-api-bcdbf47bb-m2ctz\" (UID: \"a83592aa-a4c1-459e-9d07-f14b390a5e6b\") " pod="openstack-kuttl-tests/barbican-api-bcdbf47bb-m2ctz" Dec 03 12:39:25 crc kubenswrapper[5052]: I1203 12:39:25.838526 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a83592aa-a4c1-459e-9d07-f14b390a5e6b-combined-ca-bundle\") pod \"barbican-api-bcdbf47bb-m2ctz\" (UID: \"a83592aa-a4c1-459e-9d07-f14b390a5e6b\") " pod="openstack-kuttl-tests/barbican-api-bcdbf47bb-m2ctz" Dec 03 12:39:25 crc kubenswrapper[5052]: I1203 12:39:25.840269 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a83592aa-a4c1-459e-9d07-f14b390a5e6b-config-data\") pod \"barbican-api-bcdbf47bb-m2ctz\" (UID: \"a83592aa-a4c1-459e-9d07-f14b390a5e6b\") " pod="openstack-kuttl-tests/barbican-api-bcdbf47bb-m2ctz" Dec 03 12:39:25 crc kubenswrapper[5052]: I1203 12:39:25.843861 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-tk6xf\" (UniqueName: \"kubernetes.io/projected/a83592aa-a4c1-459e-9d07-f14b390a5e6b-kube-api-access-tk6xf\") pod \"barbican-api-bcdbf47bb-m2ctz\" (UID: \"a83592aa-a4c1-459e-9d07-f14b390a5e6b\") " pod="openstack-kuttl-tests/barbican-api-bcdbf47bb-m2ctz" Dec 03 12:39:25 crc kubenswrapper[5052]: I1203 12:39:25.859622 5052 scope.go:117] "RemoveContainer" containerID="2bddb69fe53e0c8a15b016cc5ddde5164332e92554c29e75c2ee6664a3dd67a7" Dec 03 12:39:25 crc kubenswrapper[5052]: I1203 12:39:25.909835 5052 scope.go:117] "RemoveContainer" containerID="a71490515349faed4ab3d5147860c7b66eb2e343b2b37fce8896d878d5c4382f" Dec 03 12:39:25 crc kubenswrapper[5052]: E1203 12:39:25.910429 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a71490515349faed4ab3d5147860c7b66eb2e343b2b37fce8896d878d5c4382f\": container with ID starting with a71490515349faed4ab3d5147860c7b66eb2e343b2b37fce8896d878d5c4382f not found: ID does not exist" containerID="a71490515349faed4ab3d5147860c7b66eb2e343b2b37fce8896d878d5c4382f" Dec 03 12:39:25 crc kubenswrapper[5052]: I1203 12:39:25.910481 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a71490515349faed4ab3d5147860c7b66eb2e343b2b37fce8896d878d5c4382f"} err="failed to get container status \"a71490515349faed4ab3d5147860c7b66eb2e343b2b37fce8896d878d5c4382f\": rpc error: code = NotFound desc = could not find container \"a71490515349faed4ab3d5147860c7b66eb2e343b2b37fce8896d878d5c4382f\": container with ID starting with a71490515349faed4ab3d5147860c7b66eb2e343b2b37fce8896d878d5c4382f not found: ID does not exist" Dec 03 12:39:25 crc kubenswrapper[5052]: I1203 12:39:25.910533 5052 scope.go:117] "RemoveContainer" containerID="575b5e45ed0a1dfb51c75c86c63be7eabec2113683e893c6ee22d23b04e78576" Dec 03 12:39:25 crc kubenswrapper[5052]: E1203 12:39:25.911033 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"575b5e45ed0a1dfb51c75c86c63be7eabec2113683e893c6ee22d23b04e78576\": container with ID starting with 575b5e45ed0a1dfb51c75c86c63be7eabec2113683e893c6ee22d23b04e78576 not found: ID does not exist" containerID="575b5e45ed0a1dfb51c75c86c63be7eabec2113683e893c6ee22d23b04e78576" Dec 03 12:39:25 crc kubenswrapper[5052]: I1203 12:39:25.911062 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"575b5e45ed0a1dfb51c75c86c63be7eabec2113683e893c6ee22d23b04e78576"} err="failed to get container status \"575b5e45ed0a1dfb51c75c86c63be7eabec2113683e893c6ee22d23b04e78576\": rpc error: code = NotFound desc = could not find container \"575b5e45ed0a1dfb51c75c86c63be7eabec2113683e893c6ee22d23b04e78576\": container with ID starting with 575b5e45ed0a1dfb51c75c86c63be7eabec2113683e893c6ee22d23b04e78576 not found: ID does not exist" Dec 03 12:39:25 crc kubenswrapper[5052]: I1203 12:39:25.911083 5052 scope.go:117] "RemoveContainer" containerID="2bddb69fe53e0c8a15b016cc5ddde5164332e92554c29e75c2ee6664a3dd67a7" Dec 03 12:39:25 crc kubenswrapper[5052]: E1203 12:39:25.911333 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2bddb69fe53e0c8a15b016cc5ddde5164332e92554c29e75c2ee6664a3dd67a7\": container with ID starting with 2bddb69fe53e0c8a15b016cc5ddde5164332e92554c29e75c2ee6664a3dd67a7 not found: ID does not exist" containerID="2bddb69fe53e0c8a15b016cc5ddde5164332e92554c29e75c2ee6664a3dd67a7" Dec 03 12:39:25 crc kubenswrapper[5052]: I1203 12:39:25.911359 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2bddb69fe53e0c8a15b016cc5ddde5164332e92554c29e75c2ee6664a3dd67a7"} err="failed to get container status \"2bddb69fe53e0c8a15b016cc5ddde5164332e92554c29e75c2ee6664a3dd67a7\": rpc error: code = NotFound desc = could not find container \"2bddb69fe53e0c8a15b016cc5ddde5164332e92554c29e75c2ee6664a3dd67a7\": container with ID starting with 2bddb69fe53e0c8a15b016cc5ddde5164332e92554c29e75c2ee6664a3dd67a7 not found: ID does not exist" Dec 03 12:39:25 crc kubenswrapper[5052]: I1203 12:39:25.941583 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/neutron-544c66dd6d-75q8t"] Dec 03 12:39:25 crc kubenswrapper[5052]: I1203 12:39:25.950727 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/neutron-544c66dd6d-75q8t"] Dec 03 12:39:26 crc kubenswrapper[5052]: I1203 12:39:26.121041 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-api-bcdbf47bb-m2ctz" Dec 03 12:39:26 crc kubenswrapper[5052]: I1203 12:39:26.535283 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:39:26 crc kubenswrapper[5052]: I1203 12:39:26.648823 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/edea8e53-340d-4f1e-99e5-082268c64ceb-nova-metadata-tls-certs\") pod \"edea8e53-340d-4f1e-99e5-082268c64ceb\" (UID: \"edea8e53-340d-4f1e-99e5-082268c64ceb\") " Dec 03 12:39:26 crc kubenswrapper[5052]: I1203 12:39:26.648956 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/edea8e53-340d-4f1e-99e5-082268c64ceb-combined-ca-bundle\") pod \"edea8e53-340d-4f1e-99e5-082268c64ceb\" (UID: \"edea8e53-340d-4f1e-99e5-082268c64ceb\") " Dec 03 12:39:26 crc kubenswrapper[5052]: I1203 12:39:26.649061 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dgnqg\" (UniqueName: \"kubernetes.io/projected/edea8e53-340d-4f1e-99e5-082268c64ceb-kube-api-access-dgnqg\") pod \"edea8e53-340d-4f1e-99e5-082268c64ceb\" (UID: \"edea8e53-340d-4f1e-99e5-082268c64ceb\") " Dec 03 12:39:26 crc kubenswrapper[5052]: I1203 12:39:26.649198 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/edea8e53-340d-4f1e-99e5-082268c64ceb-logs\") pod \"edea8e53-340d-4f1e-99e5-082268c64ceb\" (UID: \"edea8e53-340d-4f1e-99e5-082268c64ceb\") " Dec 03 12:39:26 crc kubenswrapper[5052]: I1203 12:39:26.649220 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/edea8e53-340d-4f1e-99e5-082268c64ceb-config-data\") pod \"edea8e53-340d-4f1e-99e5-082268c64ceb\" (UID: \"edea8e53-340d-4f1e-99e5-082268c64ceb\") " Dec 03 12:39:26 crc kubenswrapper[5052]: I1203 12:39:26.650616 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/edea8e53-340d-4f1e-99e5-082268c64ceb-logs" (OuterVolumeSpecName: "logs") pod "edea8e53-340d-4f1e-99e5-082268c64ceb" (UID: "edea8e53-340d-4f1e-99e5-082268c64ceb"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:39:26 crc kubenswrapper[5052]: I1203 12:39:26.660714 5052 generic.go:334] "Generic (PLEG): container finished" podID="edea8e53-340d-4f1e-99e5-082268c64ceb" containerID="e60c32d29dd74ca93adc4e272d8de99081bbf8e5961fe48ef0a2093869b9682e" exitCode=0 Dec 03 12:39:26 crc kubenswrapper[5052]: I1203 12:39:26.660929 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/barbican-api-764dd4c57d-54dvn" podUID="5da4fd9f-78ea-4511-ba09-f00694058c69" containerName="barbican-api-log" containerID="cri-o://276d7d9067803b5433ccd1312132cf50f0afd9d72333fbe834713799a5eb7360" gracePeriod=30 Dec 03 12:39:26 crc kubenswrapper[5052]: I1203 12:39:26.661050 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:39:26 crc kubenswrapper[5052]: I1203 12:39:26.661650 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"edea8e53-340d-4f1e-99e5-082268c64ceb","Type":"ContainerDied","Data":"e60c32d29dd74ca93adc4e272d8de99081bbf8e5961fe48ef0a2093869b9682e"} Dec 03 12:39:26 crc kubenswrapper[5052]: I1203 12:39:26.661687 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"edea8e53-340d-4f1e-99e5-082268c64ceb","Type":"ContainerDied","Data":"31fbf4be90f62cd778687f590b181dfa7ad9d21ee0c96fd40c93756d8d87ddbc"} Dec 03 12:39:26 crc kubenswrapper[5052]: I1203 12:39:26.661703 5052 scope.go:117] "RemoveContainer" containerID="e60c32d29dd74ca93adc4e272d8de99081bbf8e5961fe48ef0a2093869b9682e" Dec 03 12:39:26 crc kubenswrapper[5052]: I1203 12:39:26.662909 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/barbican-api-764dd4c57d-54dvn" podUID="5da4fd9f-78ea-4511-ba09-f00694058c69" containerName="barbican-api" containerID="cri-o://149b09f0ffc6c75d49bcdf2891249bda90b68eaca953e917617a0f58c6463c58" gracePeriod=30 Dec 03 12:39:26 crc kubenswrapper[5052]: I1203 12:39:26.688706 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/edea8e53-340d-4f1e-99e5-082268c64ceb-kube-api-access-dgnqg" (OuterVolumeSpecName: "kube-api-access-dgnqg") pod "edea8e53-340d-4f1e-99e5-082268c64ceb" (UID: "edea8e53-340d-4f1e-99e5-082268c64ceb"). InnerVolumeSpecName "kube-api-access-dgnqg". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:39:26 crc kubenswrapper[5052]: I1203 12:39:26.726382 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/edea8e53-340d-4f1e-99e5-082268c64ceb-config-data" (OuterVolumeSpecName: "config-data") pod "edea8e53-340d-4f1e-99e5-082268c64ceb" (UID: "edea8e53-340d-4f1e-99e5-082268c64ceb"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:39:26 crc kubenswrapper[5052]: I1203 12:39:26.731664 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/edea8e53-340d-4f1e-99e5-082268c64ceb-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "edea8e53-340d-4f1e-99e5-082268c64ceb" (UID: "edea8e53-340d-4f1e-99e5-082268c64ceb"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:39:26 crc kubenswrapper[5052]: I1203 12:39:26.756988 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/edea8e53-340d-4f1e-99e5-082268c64ceb-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:39:26 crc kubenswrapper[5052]: I1203 12:39:26.757027 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-dgnqg\" (UniqueName: \"kubernetes.io/projected/edea8e53-340d-4f1e-99e5-082268c64ceb-kube-api-access-dgnqg\") on node \"crc\" DevicePath \"\"" Dec 03 12:39:26 crc kubenswrapper[5052]: I1203 12:39:26.757041 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/edea8e53-340d-4f1e-99e5-082268c64ceb-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:39:26 crc kubenswrapper[5052]: I1203 12:39:26.757051 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/edea8e53-340d-4f1e-99e5-082268c64ceb-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:39:26 crc kubenswrapper[5052]: I1203 12:39:26.765372 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/edea8e53-340d-4f1e-99e5-082268c64ceb-nova-metadata-tls-certs" (OuterVolumeSpecName: "nova-metadata-tls-certs") pod "edea8e53-340d-4f1e-99e5-082268c64ceb" (UID: "edea8e53-340d-4f1e-99e5-082268c64ceb"). InnerVolumeSpecName "nova-metadata-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:39:26 crc kubenswrapper[5052]: I1203 12:39:26.859261 5052 reconciler_common.go:293] "Volume detached for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/edea8e53-340d-4f1e-99e5-082268c64ceb-nova-metadata-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:39:26 crc kubenswrapper[5052]: I1203 12:39:26.862351 5052 scope.go:117] "RemoveContainer" containerID="ebef114a01de4517efb29228cf3e1863ae4862f9108584e0350c29841b2315c3" Dec 03 12:39:26 crc kubenswrapper[5052]: I1203 12:39:26.873211 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-api-bcdbf47bb-m2ctz"] Dec 03 12:39:26 crc kubenswrapper[5052]: W1203 12:39:26.880615 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda83592aa_a4c1_459e_9d07_f14b390a5e6b.slice/crio-e8b8f7643190976f86014ef812f143b1cfef5ed2f44ceb63ca7fb421896c7b57 WatchSource:0}: Error finding container e8b8f7643190976f86014ef812f143b1cfef5ed2f44ceb63ca7fb421896c7b57: Status 404 returned error can't find the container with id e8b8f7643190976f86014ef812f143b1cfef5ed2f44ceb63ca7fb421896c7b57 Dec 03 12:39:26 crc kubenswrapper[5052]: I1203 12:39:26.885065 5052 scope.go:117] "RemoveContainer" containerID="e60c32d29dd74ca93adc4e272d8de99081bbf8e5961fe48ef0a2093869b9682e" Dec 03 12:39:26 crc kubenswrapper[5052]: E1203 12:39:26.885783 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e60c32d29dd74ca93adc4e272d8de99081bbf8e5961fe48ef0a2093869b9682e\": container with ID starting with e60c32d29dd74ca93adc4e272d8de99081bbf8e5961fe48ef0a2093869b9682e not found: ID does not exist" containerID="e60c32d29dd74ca93adc4e272d8de99081bbf8e5961fe48ef0a2093869b9682e" Dec 03 12:39:26 crc kubenswrapper[5052]: I1203 12:39:26.885836 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e60c32d29dd74ca93adc4e272d8de99081bbf8e5961fe48ef0a2093869b9682e"} err="failed to get container status \"e60c32d29dd74ca93adc4e272d8de99081bbf8e5961fe48ef0a2093869b9682e\": rpc error: code = NotFound desc = could not find container \"e60c32d29dd74ca93adc4e272d8de99081bbf8e5961fe48ef0a2093869b9682e\": container with ID starting with e60c32d29dd74ca93adc4e272d8de99081bbf8e5961fe48ef0a2093869b9682e not found: ID does not exist" Dec 03 12:39:26 crc kubenswrapper[5052]: I1203 12:39:26.885873 5052 scope.go:117] "RemoveContainer" containerID="ebef114a01de4517efb29228cf3e1863ae4862f9108584e0350c29841b2315c3" Dec 03 12:39:26 crc kubenswrapper[5052]: E1203 12:39:26.886126 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ebef114a01de4517efb29228cf3e1863ae4862f9108584e0350c29841b2315c3\": container with ID starting with ebef114a01de4517efb29228cf3e1863ae4862f9108584e0350c29841b2315c3 not found: ID does not exist" containerID="ebef114a01de4517efb29228cf3e1863ae4862f9108584e0350c29841b2315c3" Dec 03 12:39:26 crc kubenswrapper[5052]: I1203 12:39:26.886152 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ebef114a01de4517efb29228cf3e1863ae4862f9108584e0350c29841b2315c3"} err="failed to get container status \"ebef114a01de4517efb29228cf3e1863ae4862f9108584e0350c29841b2315c3\": rpc error: code = NotFound desc = could not find container \"ebef114a01de4517efb29228cf3e1863ae4862f9108584e0350c29841b2315c3\": container with ID starting with ebef114a01de4517efb29228cf3e1863ae4862f9108584e0350c29841b2315c3 not found: ID does not exist" Dec 03 12:39:27 crc kubenswrapper[5052]: I1203 12:39:27.001861 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:39:27 crc kubenswrapper[5052]: I1203 12:39:27.011977 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:39:27 crc kubenswrapper[5052]: I1203 12:39:27.028787 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:39:27 crc kubenswrapper[5052]: E1203 12:39:27.029472 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="edea8e53-340d-4f1e-99e5-082268c64ceb" containerName="nova-metadata-log" Dec 03 12:39:27 crc kubenswrapper[5052]: I1203 12:39:27.029624 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="edea8e53-340d-4f1e-99e5-082268c64ceb" containerName="nova-metadata-log" Dec 03 12:39:27 crc kubenswrapper[5052]: E1203 12:39:27.029715 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="edea8e53-340d-4f1e-99e5-082268c64ceb" containerName="nova-metadata-metadata" Dec 03 12:39:27 crc kubenswrapper[5052]: I1203 12:39:27.029768 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="edea8e53-340d-4f1e-99e5-082268c64ceb" containerName="nova-metadata-metadata" Dec 03 12:39:27 crc kubenswrapper[5052]: I1203 12:39:27.030059 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="edea8e53-340d-4f1e-99e5-082268c64ceb" containerName="nova-metadata-log" Dec 03 12:39:27 crc kubenswrapper[5052]: I1203 12:39:27.030184 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="edea8e53-340d-4f1e-99e5-082268c64ceb" containerName="nova-metadata-metadata" Dec 03 12:39:27 crc kubenswrapper[5052]: I1203 12:39:27.031483 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:39:27 crc kubenswrapper[5052]: I1203 12:39:27.035072 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-metadata-config-data" Dec 03 12:39:27 crc kubenswrapper[5052]: I1203 12:39:27.039665 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-nova-metadata-internal-svc" Dec 03 12:39:27 crc kubenswrapper[5052]: I1203 12:39:27.045557 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:39:27 crc kubenswrapper[5052]: I1203 12:39:27.169037 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/30045a80-9e2b-416b-b7bf-cdb9585ec67d-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"30045a80-9e2b-416b-b7bf-cdb9585ec67d\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:39:27 crc kubenswrapper[5052]: I1203 12:39:27.169139 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/30045a80-9e2b-416b-b7bf-cdb9585ec67d-nova-metadata-tls-certs\") pod \"nova-metadata-0\" (UID: \"30045a80-9e2b-416b-b7bf-cdb9585ec67d\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:39:27 crc kubenswrapper[5052]: I1203 12:39:27.169198 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/30045a80-9e2b-416b-b7bf-cdb9585ec67d-logs\") pod \"nova-metadata-0\" (UID: \"30045a80-9e2b-416b-b7bf-cdb9585ec67d\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:39:27 crc kubenswrapper[5052]: I1203 12:39:27.169237 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hpsn6\" (UniqueName: \"kubernetes.io/projected/30045a80-9e2b-416b-b7bf-cdb9585ec67d-kube-api-access-hpsn6\") pod \"nova-metadata-0\" (UID: \"30045a80-9e2b-416b-b7bf-cdb9585ec67d\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:39:27 crc kubenswrapper[5052]: I1203 12:39:27.169293 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/30045a80-9e2b-416b-b7bf-cdb9585ec67d-config-data\") pod \"nova-metadata-0\" (UID: \"30045a80-9e2b-416b-b7bf-cdb9585ec67d\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:39:27 crc kubenswrapper[5052]: I1203 12:39:27.271144 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/30045a80-9e2b-416b-b7bf-cdb9585ec67d-config-data\") pod \"nova-metadata-0\" (UID: \"30045a80-9e2b-416b-b7bf-cdb9585ec67d\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:39:27 crc kubenswrapper[5052]: I1203 12:39:27.271236 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/30045a80-9e2b-416b-b7bf-cdb9585ec67d-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"30045a80-9e2b-416b-b7bf-cdb9585ec67d\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:39:27 crc kubenswrapper[5052]: I1203 12:39:27.271359 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/30045a80-9e2b-416b-b7bf-cdb9585ec67d-nova-metadata-tls-certs\") pod \"nova-metadata-0\" (UID: \"30045a80-9e2b-416b-b7bf-cdb9585ec67d\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:39:27 crc kubenswrapper[5052]: I1203 12:39:27.271443 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/30045a80-9e2b-416b-b7bf-cdb9585ec67d-logs\") pod \"nova-metadata-0\" (UID: \"30045a80-9e2b-416b-b7bf-cdb9585ec67d\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:39:27 crc kubenswrapper[5052]: I1203 12:39:27.271506 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-hpsn6\" (UniqueName: \"kubernetes.io/projected/30045a80-9e2b-416b-b7bf-cdb9585ec67d-kube-api-access-hpsn6\") pod \"nova-metadata-0\" (UID: \"30045a80-9e2b-416b-b7bf-cdb9585ec67d\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:39:27 crc kubenswrapper[5052]: I1203 12:39:27.276961 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/30045a80-9e2b-416b-b7bf-cdb9585ec67d-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"30045a80-9e2b-416b-b7bf-cdb9585ec67d\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:39:27 crc kubenswrapper[5052]: I1203 12:39:27.277586 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/30045a80-9e2b-416b-b7bf-cdb9585ec67d-nova-metadata-tls-certs\") pod \"nova-metadata-0\" (UID: \"30045a80-9e2b-416b-b7bf-cdb9585ec67d\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:39:27 crc kubenswrapper[5052]: I1203 12:39:27.277881 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/30045a80-9e2b-416b-b7bf-cdb9585ec67d-logs\") pod \"nova-metadata-0\" (UID: \"30045a80-9e2b-416b-b7bf-cdb9585ec67d\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:39:27 crc kubenswrapper[5052]: I1203 12:39:27.296259 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/30045a80-9e2b-416b-b7bf-cdb9585ec67d-config-data\") pod \"nova-metadata-0\" (UID: \"30045a80-9e2b-416b-b7bf-cdb9585ec67d\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:39:27 crc kubenswrapper[5052]: I1203 12:39:27.302093 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-hpsn6\" (UniqueName: \"kubernetes.io/projected/30045a80-9e2b-416b-b7bf-cdb9585ec67d-kube-api-access-hpsn6\") pod \"nova-metadata-0\" (UID: \"30045a80-9e2b-416b-b7bf-cdb9585ec67d\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:39:27 crc kubenswrapper[5052]: I1203 12:39:27.350559 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:39:27 crc kubenswrapper[5052]: I1203 12:39:27.521784 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:39:27 crc kubenswrapper[5052]: I1203 12:39:27.523106 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:39:27 crc kubenswrapper[5052]: I1203 12:39:27.639922 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:39:27 crc kubenswrapper[5052]: I1203 12:39:27.709346 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c34523d4-d634-4d76-a238-36566ef6a682" path="/var/lib/kubelet/pods/c34523d4-d634-4d76-a238-36566ef6a682/volumes" Dec 03 12:39:27 crc kubenswrapper[5052]: I1203 12:39:27.711067 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="edea8e53-340d-4f1e-99e5-082268c64ceb" path="/var/lib/kubelet/pods/edea8e53-340d-4f1e-99e5-082268c64ceb/volumes" Dec 03 12:39:27 crc kubenswrapper[5052]: I1203 12:39:27.712207 5052 generic.go:334] "Generic (PLEG): container finished" podID="59dcc834-7abb-4dff-a10f-6127276698ee" containerID="fab91a75f95875869bda13fb508f1578ec0f067822bbe9b4d97201b302a40035" exitCode=0 Dec 03 12:39:27 crc kubenswrapper[5052]: I1203 12:39:27.712470 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:39:27 crc kubenswrapper[5052]: I1203 12:39:27.719266 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"59dcc834-7abb-4dff-a10f-6127276698ee","Type":"ContainerDied","Data":"fab91a75f95875869bda13fb508f1578ec0f067822bbe9b4d97201b302a40035"} Dec 03 12:39:27 crc kubenswrapper[5052]: I1203 12:39:27.719589 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"59dcc834-7abb-4dff-a10f-6127276698ee","Type":"ContainerDied","Data":"0ef98e7e01f4e7dd908ead07500cc88e4cfb923646967b80612c75f59e6f532d"} Dec 03 12:39:27 crc kubenswrapper[5052]: I1203 12:39:27.719702 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/barbican-api-bcdbf47bb-m2ctz" Dec 03 12:39:27 crc kubenswrapper[5052]: I1203 12:39:27.719782 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-bcdbf47bb-m2ctz" event={"ID":"a83592aa-a4c1-459e-9d07-f14b390a5e6b","Type":"ContainerStarted","Data":"be69f0c47ca79cbf547e094fd8c2d7c0c191112e45d357d7916c62c184aa277b"} Dec 03 12:39:27 crc kubenswrapper[5052]: I1203 12:39:27.719890 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/barbican-api-bcdbf47bb-m2ctz" Dec 03 12:39:27 crc kubenswrapper[5052]: I1203 12:39:27.719983 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-bcdbf47bb-m2ctz" event={"ID":"a83592aa-a4c1-459e-9d07-f14b390a5e6b","Type":"ContainerStarted","Data":"7af822baa4982fe79fa48809692cdf9c52486cac04ab62564e24484f37f0039e"} Dec 03 12:39:27 crc kubenswrapper[5052]: I1203 12:39:27.720060 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-bcdbf47bb-m2ctz" event={"ID":"a83592aa-a4c1-459e-9d07-f14b390a5e6b","Type":"ContainerStarted","Data":"e8b8f7643190976f86014ef812f143b1cfef5ed2f44ceb63ca7fb421896c7b57"} Dec 03 12:39:27 crc kubenswrapper[5052]: I1203 12:39:27.719826 5052 scope.go:117] "RemoveContainer" containerID="be761ab2be8f4c31fcd16de111710f2bfb1713804672136a770634cfde4d0e6b" Dec 03 12:39:27 crc kubenswrapper[5052]: I1203 12:39:27.738895 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/barbican-api-bcdbf47bb-m2ctz" podStartSLOduration=2.7388729080000003 podStartE2EDuration="2.738872908s" podCreationTimestamp="2025-12-03 12:39:25 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:39:27.736617254 +0000 UTC m=+4758.324859038" watchObservedRunningTime="2025-12-03 12:39:27.738872908 +0000 UTC m=+4758.327114672" Dec 03 12:39:27 crc kubenswrapper[5052]: I1203 12:39:27.740033 5052 generic.go:334] "Generic (PLEG): container finished" podID="5da4fd9f-78ea-4511-ba09-f00694058c69" containerID="149b09f0ffc6c75d49bcdf2891249bda90b68eaca953e917617a0f58c6463c58" exitCode=0 Dec 03 12:39:27 crc kubenswrapper[5052]: I1203 12:39:27.740056 5052 generic.go:334] "Generic (PLEG): container finished" podID="5da4fd9f-78ea-4511-ba09-f00694058c69" containerID="276d7d9067803b5433ccd1312132cf50f0afd9d72333fbe834713799a5eb7360" exitCode=143 Dec 03 12:39:27 crc kubenswrapper[5052]: I1203 12:39:27.741094 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-764dd4c57d-54dvn" event={"ID":"5da4fd9f-78ea-4511-ba09-f00694058c69","Type":"ContainerDied","Data":"149b09f0ffc6c75d49bcdf2891249bda90b68eaca953e917617a0f58c6463c58"} Dec 03 12:39:27 crc kubenswrapper[5052]: I1203 12:39:27.741127 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-764dd4c57d-54dvn" event={"ID":"5da4fd9f-78ea-4511-ba09-f00694058c69","Type":"ContainerDied","Data":"276d7d9067803b5433ccd1312132cf50f0afd9d72333fbe834713799a5eb7360"} Dec 03 12:39:27 crc kubenswrapper[5052]: I1203 12:39:27.755456 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:39:27 crc kubenswrapper[5052]: I1203 12:39:27.772456 5052 scope.go:117] "RemoveContainer" containerID="8694cf282fe33e5849ce697fe875d63a8f6e6daaa6c0de9a9adcdfe4163c63d3" Dec 03 12:39:27 crc kubenswrapper[5052]: I1203 12:39:27.790504 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/59dcc834-7abb-4dff-a10f-6127276698ee-sg-core-conf-yaml\") pod \"59dcc834-7abb-4dff-a10f-6127276698ee\" (UID: \"59dcc834-7abb-4dff-a10f-6127276698ee\") " Dec 03 12:39:27 crc kubenswrapper[5052]: I1203 12:39:27.790574 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/59dcc834-7abb-4dff-a10f-6127276698ee-combined-ca-bundle\") pod \"59dcc834-7abb-4dff-a10f-6127276698ee\" (UID: \"59dcc834-7abb-4dff-a10f-6127276698ee\") " Dec 03 12:39:27 crc kubenswrapper[5052]: I1203 12:39:27.790681 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/59dcc834-7abb-4dff-a10f-6127276698ee-scripts\") pod \"59dcc834-7abb-4dff-a10f-6127276698ee\" (UID: \"59dcc834-7abb-4dff-a10f-6127276698ee\") " Dec 03 12:39:27 crc kubenswrapper[5052]: I1203 12:39:27.790710 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/59dcc834-7abb-4dff-a10f-6127276698ee-config-data\") pod \"59dcc834-7abb-4dff-a10f-6127276698ee\" (UID: \"59dcc834-7abb-4dff-a10f-6127276698ee\") " Dec 03 12:39:27 crc kubenswrapper[5052]: I1203 12:39:27.790986 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2n9rs\" (UniqueName: \"kubernetes.io/projected/59dcc834-7abb-4dff-a10f-6127276698ee-kube-api-access-2n9rs\") pod \"59dcc834-7abb-4dff-a10f-6127276698ee\" (UID: \"59dcc834-7abb-4dff-a10f-6127276698ee\") " Dec 03 12:39:27 crc kubenswrapper[5052]: I1203 12:39:27.791067 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/59dcc834-7abb-4dff-a10f-6127276698ee-ceilometer-tls-certs\") pod \"59dcc834-7abb-4dff-a10f-6127276698ee\" (UID: \"59dcc834-7abb-4dff-a10f-6127276698ee\") " Dec 03 12:39:27 crc kubenswrapper[5052]: I1203 12:39:27.791088 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/59dcc834-7abb-4dff-a10f-6127276698ee-log-httpd\") pod \"59dcc834-7abb-4dff-a10f-6127276698ee\" (UID: \"59dcc834-7abb-4dff-a10f-6127276698ee\") " Dec 03 12:39:27 crc kubenswrapper[5052]: I1203 12:39:27.791107 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/59dcc834-7abb-4dff-a10f-6127276698ee-run-httpd\") pod \"59dcc834-7abb-4dff-a10f-6127276698ee\" (UID: \"59dcc834-7abb-4dff-a10f-6127276698ee\") " Dec 03 12:39:27 crc kubenswrapper[5052]: I1203 12:39:27.794748 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59dcc834-7abb-4dff-a10f-6127276698ee-run-httpd" (OuterVolumeSpecName: "run-httpd") pod "59dcc834-7abb-4dff-a10f-6127276698ee" (UID: "59dcc834-7abb-4dff-a10f-6127276698ee"). InnerVolumeSpecName "run-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:39:27 crc kubenswrapper[5052]: I1203 12:39:27.795004 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59dcc834-7abb-4dff-a10f-6127276698ee-log-httpd" (OuterVolumeSpecName: "log-httpd") pod "59dcc834-7abb-4dff-a10f-6127276698ee" (UID: "59dcc834-7abb-4dff-a10f-6127276698ee"). InnerVolumeSpecName "log-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:39:27 crc kubenswrapper[5052]: I1203 12:39:27.805813 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/59dcc834-7abb-4dff-a10f-6127276698ee-kube-api-access-2n9rs" (OuterVolumeSpecName: "kube-api-access-2n9rs") pod "59dcc834-7abb-4dff-a10f-6127276698ee" (UID: "59dcc834-7abb-4dff-a10f-6127276698ee"). InnerVolumeSpecName "kube-api-access-2n9rs". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:39:27 crc kubenswrapper[5052]: I1203 12:39:27.806224 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/59dcc834-7abb-4dff-a10f-6127276698ee-scripts" (OuterVolumeSpecName: "scripts") pod "59dcc834-7abb-4dff-a10f-6127276698ee" (UID: "59dcc834-7abb-4dff-a10f-6127276698ee"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:39:27 crc kubenswrapper[5052]: I1203 12:39:27.814899 5052 scope.go:117] "RemoveContainer" containerID="fab91a75f95875869bda13fb508f1578ec0f067822bbe9b4d97201b302a40035" Dec 03 12:39:27 crc kubenswrapper[5052]: I1203 12:39:27.894413 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-2n9rs\" (UniqueName: \"kubernetes.io/projected/59dcc834-7abb-4dff-a10f-6127276698ee-kube-api-access-2n9rs\") on node \"crc\" DevicePath \"\"" Dec 03 12:39:27 crc kubenswrapper[5052]: I1203 12:39:27.894441 5052 reconciler_common.go:293] "Volume detached for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/59dcc834-7abb-4dff-a10f-6127276698ee-log-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:39:27 crc kubenswrapper[5052]: I1203 12:39:27.894456 5052 reconciler_common.go:293] "Volume detached for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/59dcc834-7abb-4dff-a10f-6127276698ee-run-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:39:27 crc kubenswrapper[5052]: I1203 12:39:27.894468 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/59dcc834-7abb-4dff-a10f-6127276698ee-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:39:27 crc kubenswrapper[5052]: I1203 12:39:27.927714 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/59dcc834-7abb-4dff-a10f-6127276698ee-sg-core-conf-yaml" (OuterVolumeSpecName: "sg-core-conf-yaml") pod "59dcc834-7abb-4dff-a10f-6127276698ee" (UID: "59dcc834-7abb-4dff-a10f-6127276698ee"). InnerVolumeSpecName "sg-core-conf-yaml". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:39:27 crc kubenswrapper[5052]: I1203 12:39:27.971305 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:39:27 crc kubenswrapper[5052]: I1203 12:39:27.971970 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/59dcc834-7abb-4dff-a10f-6127276698ee-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "59dcc834-7abb-4dff-a10f-6127276698ee" (UID: "59dcc834-7abb-4dff-a10f-6127276698ee"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:39:27 crc kubenswrapper[5052]: I1203 12:39:27.972974 5052 scope.go:117] "RemoveContainer" containerID="1537061d8a4905c4818df97a435c7501964181424870fcee0c47aed528717497" Dec 03 12:39:27 crc kubenswrapper[5052]: I1203 12:39:27.997384 5052 reconciler_common.go:293] "Volume detached for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/59dcc834-7abb-4dff-a10f-6127276698ee-sg-core-conf-yaml\") on node \"crc\" DevicePath \"\"" Dec 03 12:39:27 crc kubenswrapper[5052]: I1203 12:39:27.997413 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/59dcc834-7abb-4dff-a10f-6127276698ee-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:39:28 crc kubenswrapper[5052]: I1203 12:39:28.002667 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/59dcc834-7abb-4dff-a10f-6127276698ee-ceilometer-tls-certs" (OuterVolumeSpecName: "ceilometer-tls-certs") pod "59dcc834-7abb-4dff-a10f-6127276698ee" (UID: "59dcc834-7abb-4dff-a10f-6127276698ee"). InnerVolumeSpecName "ceilometer-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:39:28 crc kubenswrapper[5052]: I1203 12:39:28.006339 5052 scope.go:117] "RemoveContainer" containerID="be761ab2be8f4c31fcd16de111710f2bfb1713804672136a770634cfde4d0e6b" Dec 03 12:39:28 crc kubenswrapper[5052]: E1203 12:39:28.006814 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"be761ab2be8f4c31fcd16de111710f2bfb1713804672136a770634cfde4d0e6b\": container with ID starting with be761ab2be8f4c31fcd16de111710f2bfb1713804672136a770634cfde4d0e6b not found: ID does not exist" containerID="be761ab2be8f4c31fcd16de111710f2bfb1713804672136a770634cfde4d0e6b" Dec 03 12:39:28 crc kubenswrapper[5052]: I1203 12:39:28.006846 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"be761ab2be8f4c31fcd16de111710f2bfb1713804672136a770634cfde4d0e6b"} err="failed to get container status \"be761ab2be8f4c31fcd16de111710f2bfb1713804672136a770634cfde4d0e6b\": rpc error: code = NotFound desc = could not find container \"be761ab2be8f4c31fcd16de111710f2bfb1713804672136a770634cfde4d0e6b\": container with ID starting with be761ab2be8f4c31fcd16de111710f2bfb1713804672136a770634cfde4d0e6b not found: ID does not exist" Dec 03 12:39:28 crc kubenswrapper[5052]: I1203 12:39:28.006867 5052 scope.go:117] "RemoveContainer" containerID="8694cf282fe33e5849ce697fe875d63a8f6e6daaa6c0de9a9adcdfe4163c63d3" Dec 03 12:39:28 crc kubenswrapper[5052]: I1203 12:39:28.006926 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/59dcc834-7abb-4dff-a10f-6127276698ee-config-data" (OuterVolumeSpecName: "config-data") pod "59dcc834-7abb-4dff-a10f-6127276698ee" (UID: "59dcc834-7abb-4dff-a10f-6127276698ee"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:39:28 crc kubenswrapper[5052]: E1203 12:39:28.011114 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8694cf282fe33e5849ce697fe875d63a8f6e6daaa6c0de9a9adcdfe4163c63d3\": container with ID starting with 8694cf282fe33e5849ce697fe875d63a8f6e6daaa6c0de9a9adcdfe4163c63d3 not found: ID does not exist" containerID="8694cf282fe33e5849ce697fe875d63a8f6e6daaa6c0de9a9adcdfe4163c63d3" Dec 03 12:39:28 crc kubenswrapper[5052]: I1203 12:39:28.011148 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8694cf282fe33e5849ce697fe875d63a8f6e6daaa6c0de9a9adcdfe4163c63d3"} err="failed to get container status \"8694cf282fe33e5849ce697fe875d63a8f6e6daaa6c0de9a9adcdfe4163c63d3\": rpc error: code = NotFound desc = could not find container \"8694cf282fe33e5849ce697fe875d63a8f6e6daaa6c0de9a9adcdfe4163c63d3\": container with ID starting with 8694cf282fe33e5849ce697fe875d63a8f6e6daaa6c0de9a9adcdfe4163c63d3 not found: ID does not exist" Dec 03 12:39:28 crc kubenswrapper[5052]: I1203 12:39:28.011170 5052 scope.go:117] "RemoveContainer" containerID="fab91a75f95875869bda13fb508f1578ec0f067822bbe9b4d97201b302a40035" Dec 03 12:39:28 crc kubenswrapper[5052]: E1203 12:39:28.012319 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fab91a75f95875869bda13fb508f1578ec0f067822bbe9b4d97201b302a40035\": container with ID starting with fab91a75f95875869bda13fb508f1578ec0f067822bbe9b4d97201b302a40035 not found: ID does not exist" containerID="fab91a75f95875869bda13fb508f1578ec0f067822bbe9b4d97201b302a40035" Dec 03 12:39:28 crc kubenswrapper[5052]: I1203 12:39:28.012370 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fab91a75f95875869bda13fb508f1578ec0f067822bbe9b4d97201b302a40035"} err="failed to get container status \"fab91a75f95875869bda13fb508f1578ec0f067822bbe9b4d97201b302a40035\": rpc error: code = NotFound desc = could not find container \"fab91a75f95875869bda13fb508f1578ec0f067822bbe9b4d97201b302a40035\": container with ID starting with fab91a75f95875869bda13fb508f1578ec0f067822bbe9b4d97201b302a40035 not found: ID does not exist" Dec 03 12:39:28 crc kubenswrapper[5052]: I1203 12:39:28.012400 5052 scope.go:117] "RemoveContainer" containerID="1537061d8a4905c4818df97a435c7501964181424870fcee0c47aed528717497" Dec 03 12:39:28 crc kubenswrapper[5052]: E1203 12:39:28.012885 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1537061d8a4905c4818df97a435c7501964181424870fcee0c47aed528717497\": container with ID starting with 1537061d8a4905c4818df97a435c7501964181424870fcee0c47aed528717497 not found: ID does not exist" containerID="1537061d8a4905c4818df97a435c7501964181424870fcee0c47aed528717497" Dec 03 12:39:28 crc kubenswrapper[5052]: I1203 12:39:28.012932 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1537061d8a4905c4818df97a435c7501964181424870fcee0c47aed528717497"} err="failed to get container status \"1537061d8a4905c4818df97a435c7501964181424870fcee0c47aed528717497\": rpc error: code = NotFound desc = could not find container \"1537061d8a4905c4818df97a435c7501964181424870fcee0c47aed528717497\": container with ID starting with 1537061d8a4905c4818df97a435c7501964181424870fcee0c47aed528717497 not found: ID does not exist" Dec 03 12:39:28 crc kubenswrapper[5052]: I1203 12:39:28.095003 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-api-764dd4c57d-54dvn" Dec 03 12:39:28 crc kubenswrapper[5052]: I1203 12:39:28.099363 5052 reconciler_common.go:293] "Volume detached for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/59dcc834-7abb-4dff-a10f-6127276698ee-ceilometer-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:39:28 crc kubenswrapper[5052]: I1203 12:39:28.099405 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/59dcc834-7abb-4dff-a10f-6127276698ee-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:39:28 crc kubenswrapper[5052]: I1203 12:39:28.122372 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:39:28 crc kubenswrapper[5052]: I1203 12:39:28.163657 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:39:28 crc kubenswrapper[5052]: I1203 12:39:28.188256 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:39:28 crc kubenswrapper[5052]: E1203 12:39:28.188748 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="59dcc834-7abb-4dff-a10f-6127276698ee" containerName="ceilometer-notification-agent" Dec 03 12:39:28 crc kubenswrapper[5052]: I1203 12:39:28.188768 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="59dcc834-7abb-4dff-a10f-6127276698ee" containerName="ceilometer-notification-agent" Dec 03 12:39:28 crc kubenswrapper[5052]: E1203 12:39:28.188785 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="5da4fd9f-78ea-4511-ba09-f00694058c69" containerName="barbican-api-log" Dec 03 12:39:28 crc kubenswrapper[5052]: I1203 12:39:28.188791 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="5da4fd9f-78ea-4511-ba09-f00694058c69" containerName="barbican-api-log" Dec 03 12:39:28 crc kubenswrapper[5052]: E1203 12:39:28.188804 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="59dcc834-7abb-4dff-a10f-6127276698ee" containerName="sg-core" Dec 03 12:39:28 crc kubenswrapper[5052]: I1203 12:39:28.188810 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="59dcc834-7abb-4dff-a10f-6127276698ee" containerName="sg-core" Dec 03 12:39:28 crc kubenswrapper[5052]: E1203 12:39:28.188838 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="5da4fd9f-78ea-4511-ba09-f00694058c69" containerName="barbican-api" Dec 03 12:39:28 crc kubenswrapper[5052]: I1203 12:39:28.188843 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="5da4fd9f-78ea-4511-ba09-f00694058c69" containerName="barbican-api" Dec 03 12:39:28 crc kubenswrapper[5052]: E1203 12:39:28.188851 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="59dcc834-7abb-4dff-a10f-6127276698ee" containerName="ceilometer-central-agent" Dec 03 12:39:28 crc kubenswrapper[5052]: I1203 12:39:28.188857 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="59dcc834-7abb-4dff-a10f-6127276698ee" containerName="ceilometer-central-agent" Dec 03 12:39:28 crc kubenswrapper[5052]: E1203 12:39:28.188869 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="59dcc834-7abb-4dff-a10f-6127276698ee" containerName="proxy-httpd" Dec 03 12:39:28 crc kubenswrapper[5052]: I1203 12:39:28.188875 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="59dcc834-7abb-4dff-a10f-6127276698ee" containerName="proxy-httpd" Dec 03 12:39:28 crc kubenswrapper[5052]: I1203 12:39:28.189042 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="59dcc834-7abb-4dff-a10f-6127276698ee" containerName="sg-core" Dec 03 12:39:28 crc kubenswrapper[5052]: I1203 12:39:28.189058 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="59dcc834-7abb-4dff-a10f-6127276698ee" containerName="ceilometer-notification-agent" Dec 03 12:39:28 crc kubenswrapper[5052]: I1203 12:39:28.189071 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="59dcc834-7abb-4dff-a10f-6127276698ee" containerName="ceilometer-central-agent" Dec 03 12:39:28 crc kubenswrapper[5052]: I1203 12:39:28.189087 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="59dcc834-7abb-4dff-a10f-6127276698ee" containerName="proxy-httpd" Dec 03 12:39:28 crc kubenswrapper[5052]: I1203 12:39:28.189106 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="5da4fd9f-78ea-4511-ba09-f00694058c69" containerName="barbican-api-log" Dec 03 12:39:28 crc kubenswrapper[5052]: I1203 12:39:28.189119 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="5da4fd9f-78ea-4511-ba09-f00694058c69" containerName="barbican-api" Dec 03 12:39:28 crc kubenswrapper[5052]: I1203 12:39:28.203835 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:39:28 crc kubenswrapper[5052]: I1203 12:39:28.205097 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/5da4fd9f-78ea-4511-ba09-f00694058c69-config-data-custom\") pod \"5da4fd9f-78ea-4511-ba09-f00694058c69\" (UID: \"5da4fd9f-78ea-4511-ba09-f00694058c69\") " Dec 03 12:39:28 crc kubenswrapper[5052]: I1203 12:39:28.205195 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/5da4fd9f-78ea-4511-ba09-f00694058c69-config-data\") pod \"5da4fd9f-78ea-4511-ba09-f00694058c69\" (UID: \"5da4fd9f-78ea-4511-ba09-f00694058c69\") " Dec 03 12:39:28 crc kubenswrapper[5052]: I1203 12:39:28.205334 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/5da4fd9f-78ea-4511-ba09-f00694058c69-logs\") pod \"5da4fd9f-78ea-4511-ba09-f00694058c69\" (UID: \"5da4fd9f-78ea-4511-ba09-f00694058c69\") " Dec 03 12:39:28 crc kubenswrapper[5052]: I1203 12:39:28.205371 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/5da4fd9f-78ea-4511-ba09-f00694058c69-internal-tls-certs\") pod \"5da4fd9f-78ea-4511-ba09-f00694058c69\" (UID: \"5da4fd9f-78ea-4511-ba09-f00694058c69\") " Dec 03 12:39:28 crc kubenswrapper[5052]: I1203 12:39:28.205481 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5da4fd9f-78ea-4511-ba09-f00694058c69-combined-ca-bundle\") pod \"5da4fd9f-78ea-4511-ba09-f00694058c69\" (UID: \"5da4fd9f-78ea-4511-ba09-f00694058c69\") " Dec 03 12:39:28 crc kubenswrapper[5052]: I1203 12:39:28.205550 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/5da4fd9f-78ea-4511-ba09-f00694058c69-public-tls-certs\") pod \"5da4fd9f-78ea-4511-ba09-f00694058c69\" (UID: \"5da4fd9f-78ea-4511-ba09-f00694058c69\") " Dec 03 12:39:28 crc kubenswrapper[5052]: I1203 12:39:28.206899 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j7ngh\" (UniqueName: \"kubernetes.io/projected/5da4fd9f-78ea-4511-ba09-f00694058c69-kube-api-access-j7ngh\") pod \"5da4fd9f-78ea-4511-ba09-f00694058c69\" (UID: \"5da4fd9f-78ea-4511-ba09-f00694058c69\") " Dec 03 12:39:28 crc kubenswrapper[5052]: I1203 12:39:28.207088 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-ceilometer-internal-svc" Dec 03 12:39:28 crc kubenswrapper[5052]: I1203 12:39:28.207248 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-scripts" Dec 03 12:39:28 crc kubenswrapper[5052]: I1203 12:39:28.207101 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-config-data" Dec 03 12:39:28 crc kubenswrapper[5052]: I1203 12:39:28.207875 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5da4fd9f-78ea-4511-ba09-f00694058c69-logs" (OuterVolumeSpecName: "logs") pod "5da4fd9f-78ea-4511-ba09-f00694058c69" (UID: "5da4fd9f-78ea-4511-ba09-f00694058c69"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:39:28 crc kubenswrapper[5052]: I1203 12:39:28.209423 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/5da4fd9f-78ea-4511-ba09-f00694058c69-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:39:28 crc kubenswrapper[5052]: I1203 12:39:28.212074 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5da4fd9f-78ea-4511-ba09-f00694058c69-kube-api-access-j7ngh" (OuterVolumeSpecName: "kube-api-access-j7ngh") pod "5da4fd9f-78ea-4511-ba09-f00694058c69" (UID: "5da4fd9f-78ea-4511-ba09-f00694058c69"). InnerVolumeSpecName "kube-api-access-j7ngh". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:39:28 crc kubenswrapper[5052]: I1203 12:39:28.213819 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5da4fd9f-78ea-4511-ba09-f00694058c69-config-data-custom" (OuterVolumeSpecName: "config-data-custom") pod "5da4fd9f-78ea-4511-ba09-f00694058c69" (UID: "5da4fd9f-78ea-4511-ba09-f00694058c69"). InnerVolumeSpecName "config-data-custom". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:39:28 crc kubenswrapper[5052]: I1203 12:39:28.235894 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:39:28 crc kubenswrapper[5052]: I1203 12:39:28.241326 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5da4fd9f-78ea-4511-ba09-f00694058c69-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "5da4fd9f-78ea-4511-ba09-f00694058c69" (UID: "5da4fd9f-78ea-4511-ba09-f00694058c69"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:39:28 crc kubenswrapper[5052]: I1203 12:39:28.273676 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5da4fd9f-78ea-4511-ba09-f00694058c69-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "5da4fd9f-78ea-4511-ba09-f00694058c69" (UID: "5da4fd9f-78ea-4511-ba09-f00694058c69"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:39:28 crc kubenswrapper[5052]: I1203 12:39:28.298100 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5da4fd9f-78ea-4511-ba09-f00694058c69-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "5da4fd9f-78ea-4511-ba09-f00694058c69" (UID: "5da4fd9f-78ea-4511-ba09-f00694058c69"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:39:28 crc kubenswrapper[5052]: I1203 12:39:28.302664 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5da4fd9f-78ea-4511-ba09-f00694058c69-config-data" (OuterVolumeSpecName: "config-data") pod "5da4fd9f-78ea-4511-ba09-f00694058c69" (UID: "5da4fd9f-78ea-4511-ba09-f00694058c69"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:39:28 crc kubenswrapper[5052]: I1203 12:39:28.310935 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/95a06af5-270d-4138-ab79-fcdd239b3997-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"95a06af5-270d-4138-ab79-fcdd239b3997\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:39:28 crc kubenswrapper[5052]: I1203 12:39:28.311001 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/95a06af5-270d-4138-ab79-fcdd239b3997-run-httpd\") pod \"ceilometer-0\" (UID: \"95a06af5-270d-4138-ab79-fcdd239b3997\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:39:28 crc kubenswrapper[5052]: I1203 12:39:28.311055 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/95a06af5-270d-4138-ab79-fcdd239b3997-scripts\") pod \"ceilometer-0\" (UID: \"95a06af5-270d-4138-ab79-fcdd239b3997\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:39:28 crc kubenswrapper[5052]: I1203 12:39:28.311077 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/95a06af5-270d-4138-ab79-fcdd239b3997-config-data\") pod \"ceilometer-0\" (UID: \"95a06af5-270d-4138-ab79-fcdd239b3997\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:39:28 crc kubenswrapper[5052]: I1203 12:39:28.311254 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7z7v5\" (UniqueName: \"kubernetes.io/projected/95a06af5-270d-4138-ab79-fcdd239b3997-kube-api-access-7z7v5\") pod \"ceilometer-0\" (UID: \"95a06af5-270d-4138-ab79-fcdd239b3997\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:39:28 crc kubenswrapper[5052]: I1203 12:39:28.311437 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/95a06af5-270d-4138-ab79-fcdd239b3997-log-httpd\") pod \"ceilometer-0\" (UID: \"95a06af5-270d-4138-ab79-fcdd239b3997\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:39:28 crc kubenswrapper[5052]: I1203 12:39:28.311627 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/95a06af5-270d-4138-ab79-fcdd239b3997-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"95a06af5-270d-4138-ab79-fcdd239b3997\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:39:28 crc kubenswrapper[5052]: I1203 12:39:28.311685 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/95a06af5-270d-4138-ab79-fcdd239b3997-ceilometer-tls-certs\") pod \"ceilometer-0\" (UID: \"95a06af5-270d-4138-ab79-fcdd239b3997\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:39:28 crc kubenswrapper[5052]: I1203 12:39:28.311871 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/5da4fd9f-78ea-4511-ba09-f00694058c69-config-data-custom\") on node \"crc\" DevicePath \"\"" Dec 03 12:39:28 crc kubenswrapper[5052]: I1203 12:39:28.311897 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/5da4fd9f-78ea-4511-ba09-f00694058c69-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:39:28 crc kubenswrapper[5052]: I1203 12:39:28.311909 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/5da4fd9f-78ea-4511-ba09-f00694058c69-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:39:28 crc kubenswrapper[5052]: I1203 12:39:28.311920 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5da4fd9f-78ea-4511-ba09-f00694058c69-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:39:28 crc kubenswrapper[5052]: I1203 12:39:28.311931 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/5da4fd9f-78ea-4511-ba09-f00694058c69-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:39:28 crc kubenswrapper[5052]: I1203 12:39:28.311943 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-j7ngh\" (UniqueName: \"kubernetes.io/projected/5da4fd9f-78ea-4511-ba09-f00694058c69-kube-api-access-j7ngh\") on node \"crc\" DevicePath \"\"" Dec 03 12:39:28 crc kubenswrapper[5052]: I1203 12:39:28.413843 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/95a06af5-270d-4138-ab79-fcdd239b3997-run-httpd\") pod \"ceilometer-0\" (UID: \"95a06af5-270d-4138-ab79-fcdd239b3997\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:39:28 crc kubenswrapper[5052]: I1203 12:39:28.413901 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/95a06af5-270d-4138-ab79-fcdd239b3997-scripts\") pod \"ceilometer-0\" (UID: \"95a06af5-270d-4138-ab79-fcdd239b3997\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:39:28 crc kubenswrapper[5052]: I1203 12:39:28.413916 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/95a06af5-270d-4138-ab79-fcdd239b3997-config-data\") pod \"ceilometer-0\" (UID: \"95a06af5-270d-4138-ab79-fcdd239b3997\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:39:28 crc kubenswrapper[5052]: I1203 12:39:28.413965 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-7z7v5\" (UniqueName: \"kubernetes.io/projected/95a06af5-270d-4138-ab79-fcdd239b3997-kube-api-access-7z7v5\") pod \"ceilometer-0\" (UID: \"95a06af5-270d-4138-ab79-fcdd239b3997\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:39:28 crc kubenswrapper[5052]: I1203 12:39:28.414010 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/95a06af5-270d-4138-ab79-fcdd239b3997-log-httpd\") pod \"ceilometer-0\" (UID: \"95a06af5-270d-4138-ab79-fcdd239b3997\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:39:28 crc kubenswrapper[5052]: I1203 12:39:28.414039 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/95a06af5-270d-4138-ab79-fcdd239b3997-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"95a06af5-270d-4138-ab79-fcdd239b3997\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:39:28 crc kubenswrapper[5052]: I1203 12:39:28.414057 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/95a06af5-270d-4138-ab79-fcdd239b3997-ceilometer-tls-certs\") pod \"ceilometer-0\" (UID: \"95a06af5-270d-4138-ab79-fcdd239b3997\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:39:28 crc kubenswrapper[5052]: I1203 12:39:28.414137 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/95a06af5-270d-4138-ab79-fcdd239b3997-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"95a06af5-270d-4138-ab79-fcdd239b3997\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:39:28 crc kubenswrapper[5052]: I1203 12:39:28.415197 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/95a06af5-270d-4138-ab79-fcdd239b3997-log-httpd\") pod \"ceilometer-0\" (UID: \"95a06af5-270d-4138-ab79-fcdd239b3997\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:39:28 crc kubenswrapper[5052]: I1203 12:39:28.415219 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/95a06af5-270d-4138-ab79-fcdd239b3997-run-httpd\") pod \"ceilometer-0\" (UID: \"95a06af5-270d-4138-ab79-fcdd239b3997\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:39:28 crc kubenswrapper[5052]: I1203 12:39:28.418781 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/95a06af5-270d-4138-ab79-fcdd239b3997-config-data\") pod \"ceilometer-0\" (UID: \"95a06af5-270d-4138-ab79-fcdd239b3997\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:39:28 crc kubenswrapper[5052]: I1203 12:39:28.419178 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/95a06af5-270d-4138-ab79-fcdd239b3997-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"95a06af5-270d-4138-ab79-fcdd239b3997\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:39:28 crc kubenswrapper[5052]: I1203 12:39:28.420426 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/95a06af5-270d-4138-ab79-fcdd239b3997-ceilometer-tls-certs\") pod \"ceilometer-0\" (UID: \"95a06af5-270d-4138-ab79-fcdd239b3997\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:39:28 crc kubenswrapper[5052]: I1203 12:39:28.422138 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/95a06af5-270d-4138-ab79-fcdd239b3997-scripts\") pod \"ceilometer-0\" (UID: \"95a06af5-270d-4138-ab79-fcdd239b3997\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:39:28 crc kubenswrapper[5052]: I1203 12:39:28.423447 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/95a06af5-270d-4138-ab79-fcdd239b3997-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"95a06af5-270d-4138-ab79-fcdd239b3997\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:39:28 crc kubenswrapper[5052]: I1203 12:39:28.480724 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-7z7v5\" (UniqueName: \"kubernetes.io/projected/95a06af5-270d-4138-ab79-fcdd239b3997-kube-api-access-7z7v5\") pod \"ceilometer-0\" (UID: \"95a06af5-270d-4138-ab79-fcdd239b3997\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:39:28 crc kubenswrapper[5052]: I1203 12:39:28.497594 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/neutron-844859d58b-w4m57"] Dec 03 12:39:28 crc kubenswrapper[5052]: I1203 12:39:28.497866 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/neutron-844859d58b-w4m57" podUID="061f3d17-a2e8-4002-ad1a-220ba7b476e3" containerName="neutron-api" containerID="cri-o://acb084fda076785e51f840ee0c818b9e5cd206bb5a416282d7f91fb92bbb2890" gracePeriod=30 Dec 03 12:39:28 crc kubenswrapper[5052]: I1203 12:39:28.498969 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/neutron-844859d58b-w4m57" podUID="061f3d17-a2e8-4002-ad1a-220ba7b476e3" containerName="neutron-httpd" containerID="cri-o://5666f6ec09a1830c252db608dbb8ad35ec7dabe19b787b488fb6bf64296bffdf" gracePeriod=30 Dec 03 12:39:28 crc kubenswrapper[5052]: I1203 12:39:28.529142 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:39:28 crc kubenswrapper[5052]: I1203 12:39:28.552995 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/neutron-7d74684fd-pqkf5"] Dec 03 12:39:28 crc kubenswrapper[5052]: I1203 12:39:28.559184 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-7d74684fd-pqkf5" Dec 03 12:39:28 crc kubenswrapper[5052]: I1203 12:39:28.574445 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-7d74684fd-pqkf5"] Dec 03 12:39:28 crc kubenswrapper[5052]: I1203 12:39:28.622796 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ddfa4a3f-b868-49b7-abec-62cb3ffde2bd-combined-ca-bundle\") pod \"neutron-7d74684fd-pqkf5\" (UID: \"ddfa4a3f-b868-49b7-abec-62cb3ffde2bd\") " pod="openstack-kuttl-tests/neutron-7d74684fd-pqkf5" Dec 03 12:39:28 crc kubenswrapper[5052]: I1203 12:39:28.623178 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/ddfa4a3f-b868-49b7-abec-62cb3ffde2bd-public-tls-certs\") pod \"neutron-7d74684fd-pqkf5\" (UID: \"ddfa4a3f-b868-49b7-abec-62cb3ffde2bd\") " pod="openstack-kuttl-tests/neutron-7d74684fd-pqkf5" Dec 03 12:39:28 crc kubenswrapper[5052]: I1203 12:39:28.623217 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/ddfa4a3f-b868-49b7-abec-62cb3ffde2bd-httpd-config\") pod \"neutron-7d74684fd-pqkf5\" (UID: \"ddfa4a3f-b868-49b7-abec-62cb3ffde2bd\") " pod="openstack-kuttl-tests/neutron-7d74684fd-pqkf5" Dec 03 12:39:28 crc kubenswrapper[5052]: I1203 12:39:28.623249 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sfvbz\" (UniqueName: \"kubernetes.io/projected/ddfa4a3f-b868-49b7-abec-62cb3ffde2bd-kube-api-access-sfvbz\") pod \"neutron-7d74684fd-pqkf5\" (UID: \"ddfa4a3f-b868-49b7-abec-62cb3ffde2bd\") " pod="openstack-kuttl-tests/neutron-7d74684fd-pqkf5" Dec 03 12:39:28 crc kubenswrapper[5052]: I1203 12:39:28.623304 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/ddfa4a3f-b868-49b7-abec-62cb3ffde2bd-config\") pod \"neutron-7d74684fd-pqkf5\" (UID: \"ddfa4a3f-b868-49b7-abec-62cb3ffde2bd\") " pod="openstack-kuttl-tests/neutron-7d74684fd-pqkf5" Dec 03 12:39:28 crc kubenswrapper[5052]: I1203 12:39:28.623371 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/ddfa4a3f-b868-49b7-abec-62cb3ffde2bd-internal-tls-certs\") pod \"neutron-7d74684fd-pqkf5\" (UID: \"ddfa4a3f-b868-49b7-abec-62cb3ffde2bd\") " pod="openstack-kuttl-tests/neutron-7d74684fd-pqkf5" Dec 03 12:39:28 crc kubenswrapper[5052]: I1203 12:39:28.623432 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/ddfa4a3f-b868-49b7-abec-62cb3ffde2bd-ovndb-tls-certs\") pod \"neutron-7d74684fd-pqkf5\" (UID: \"ddfa4a3f-b868-49b7-abec-62cb3ffde2bd\") " pod="openstack-kuttl-tests/neutron-7d74684fd-pqkf5" Dec 03 12:39:28 crc kubenswrapper[5052]: I1203 12:39:28.728052 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/ddfa4a3f-b868-49b7-abec-62cb3ffde2bd-public-tls-certs\") pod \"neutron-7d74684fd-pqkf5\" (UID: \"ddfa4a3f-b868-49b7-abec-62cb3ffde2bd\") " pod="openstack-kuttl-tests/neutron-7d74684fd-pqkf5" Dec 03 12:39:28 crc kubenswrapper[5052]: I1203 12:39:28.728126 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/ddfa4a3f-b868-49b7-abec-62cb3ffde2bd-httpd-config\") pod \"neutron-7d74684fd-pqkf5\" (UID: \"ddfa4a3f-b868-49b7-abec-62cb3ffde2bd\") " pod="openstack-kuttl-tests/neutron-7d74684fd-pqkf5" Dec 03 12:39:28 crc kubenswrapper[5052]: I1203 12:39:28.728161 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-sfvbz\" (UniqueName: \"kubernetes.io/projected/ddfa4a3f-b868-49b7-abec-62cb3ffde2bd-kube-api-access-sfvbz\") pod \"neutron-7d74684fd-pqkf5\" (UID: \"ddfa4a3f-b868-49b7-abec-62cb3ffde2bd\") " pod="openstack-kuttl-tests/neutron-7d74684fd-pqkf5" Dec 03 12:39:28 crc kubenswrapper[5052]: I1203 12:39:28.728253 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/ddfa4a3f-b868-49b7-abec-62cb3ffde2bd-config\") pod \"neutron-7d74684fd-pqkf5\" (UID: \"ddfa4a3f-b868-49b7-abec-62cb3ffde2bd\") " pod="openstack-kuttl-tests/neutron-7d74684fd-pqkf5" Dec 03 12:39:28 crc kubenswrapper[5052]: I1203 12:39:28.728659 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/ddfa4a3f-b868-49b7-abec-62cb3ffde2bd-internal-tls-certs\") pod \"neutron-7d74684fd-pqkf5\" (UID: \"ddfa4a3f-b868-49b7-abec-62cb3ffde2bd\") " pod="openstack-kuttl-tests/neutron-7d74684fd-pqkf5" Dec 03 12:39:28 crc kubenswrapper[5052]: I1203 12:39:28.728764 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/ddfa4a3f-b868-49b7-abec-62cb3ffde2bd-ovndb-tls-certs\") pod \"neutron-7d74684fd-pqkf5\" (UID: \"ddfa4a3f-b868-49b7-abec-62cb3ffde2bd\") " pod="openstack-kuttl-tests/neutron-7d74684fd-pqkf5" Dec 03 12:39:28 crc kubenswrapper[5052]: I1203 12:39:28.728946 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ddfa4a3f-b868-49b7-abec-62cb3ffde2bd-combined-ca-bundle\") pod \"neutron-7d74684fd-pqkf5\" (UID: \"ddfa4a3f-b868-49b7-abec-62cb3ffde2bd\") " pod="openstack-kuttl-tests/neutron-7d74684fd-pqkf5" Dec 03 12:39:28 crc kubenswrapper[5052]: I1203 12:39:28.737061 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/ddfa4a3f-b868-49b7-abec-62cb3ffde2bd-httpd-config\") pod \"neutron-7d74684fd-pqkf5\" (UID: \"ddfa4a3f-b868-49b7-abec-62cb3ffde2bd\") " pod="openstack-kuttl-tests/neutron-7d74684fd-pqkf5" Dec 03 12:39:28 crc kubenswrapper[5052]: I1203 12:39:28.738661 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ddfa4a3f-b868-49b7-abec-62cb3ffde2bd-combined-ca-bundle\") pod \"neutron-7d74684fd-pqkf5\" (UID: \"ddfa4a3f-b868-49b7-abec-62cb3ffde2bd\") " pod="openstack-kuttl-tests/neutron-7d74684fd-pqkf5" Dec 03 12:39:28 crc kubenswrapper[5052]: I1203 12:39:28.741503 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/ddfa4a3f-b868-49b7-abec-62cb3ffde2bd-config\") pod \"neutron-7d74684fd-pqkf5\" (UID: \"ddfa4a3f-b868-49b7-abec-62cb3ffde2bd\") " pod="openstack-kuttl-tests/neutron-7d74684fd-pqkf5" Dec 03 12:39:28 crc kubenswrapper[5052]: I1203 12:39:28.745200 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/ddfa4a3f-b868-49b7-abec-62cb3ffde2bd-ovndb-tls-certs\") pod \"neutron-7d74684fd-pqkf5\" (UID: \"ddfa4a3f-b868-49b7-abec-62cb3ffde2bd\") " pod="openstack-kuttl-tests/neutron-7d74684fd-pqkf5" Dec 03 12:39:28 crc kubenswrapper[5052]: I1203 12:39:28.747264 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/ddfa4a3f-b868-49b7-abec-62cb3ffde2bd-public-tls-certs\") pod \"neutron-7d74684fd-pqkf5\" (UID: \"ddfa4a3f-b868-49b7-abec-62cb3ffde2bd\") " pod="openstack-kuttl-tests/neutron-7d74684fd-pqkf5" Dec 03 12:39:28 crc kubenswrapper[5052]: I1203 12:39:28.758611 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-sfvbz\" (UniqueName: \"kubernetes.io/projected/ddfa4a3f-b868-49b7-abec-62cb3ffde2bd-kube-api-access-sfvbz\") pod \"neutron-7d74684fd-pqkf5\" (UID: \"ddfa4a3f-b868-49b7-abec-62cb3ffde2bd\") " pod="openstack-kuttl-tests/neutron-7d74684fd-pqkf5" Dec 03 12:39:28 crc kubenswrapper[5052]: I1203 12:39:28.770397 5052 generic.go:334] "Generic (PLEG): container finished" podID="2972cf0c-5378-4e71-b308-0235608d5bef" containerID="1e9c082372288c89143c066dc4a3ffdcb4a743df94ac5b3327dcb6b74a2be2a7" exitCode=0 Dec 03 12:39:28 crc kubenswrapper[5052]: I1203 12:39:28.770461 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-7b8cddb768-5snf9" event={"ID":"2972cf0c-5378-4e71-b308-0235608d5bef","Type":"ContainerDied","Data":"1e9c082372288c89143c066dc4a3ffdcb4a743df94ac5b3327dcb6b74a2be2a7"} Dec 03 12:39:28 crc kubenswrapper[5052]: I1203 12:39:28.774836 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/ddfa4a3f-b868-49b7-abec-62cb3ffde2bd-internal-tls-certs\") pod \"neutron-7d74684fd-pqkf5\" (UID: \"ddfa4a3f-b868-49b7-abec-62cb3ffde2bd\") " pod="openstack-kuttl-tests/neutron-7d74684fd-pqkf5" Dec 03 12:39:28 crc kubenswrapper[5052]: I1203 12:39:28.777103 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-7d74684fd-pqkf5" Dec 03 12:39:28 crc kubenswrapper[5052]: I1203 12:39:28.784731 5052 generic.go:334] "Generic (PLEG): container finished" podID="061f3d17-a2e8-4002-ad1a-220ba7b476e3" containerID="5666f6ec09a1830c252db608dbb8ad35ec7dabe19b787b488fb6bf64296bffdf" exitCode=0 Dec 03 12:39:28 crc kubenswrapper[5052]: I1203 12:39:28.784855 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-844859d58b-w4m57" event={"ID":"061f3d17-a2e8-4002-ad1a-220ba7b476e3","Type":"ContainerDied","Data":"5666f6ec09a1830c252db608dbb8ad35ec7dabe19b787b488fb6bf64296bffdf"} Dec 03 12:39:28 crc kubenswrapper[5052]: I1203 12:39:28.789541 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-764dd4c57d-54dvn" event={"ID":"5da4fd9f-78ea-4511-ba09-f00694058c69","Type":"ContainerDied","Data":"bb4ec90598abda1cb351aa0e21694297b64993fc4668ed9a7635f1e7e076ad07"} Dec 03 12:39:28 crc kubenswrapper[5052]: I1203 12:39:28.789603 5052 scope.go:117] "RemoveContainer" containerID="149b09f0ffc6c75d49bcdf2891249bda90b68eaca953e917617a0f58c6463c58" Dec 03 12:39:28 crc kubenswrapper[5052]: I1203 12:39:28.789776 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-api-764dd4c57d-54dvn" Dec 03 12:39:28 crc kubenswrapper[5052]: I1203 12:39:28.808600 5052 generic.go:334] "Generic (PLEG): container finished" podID="02726407-aa1e-400f-aedb-78d896c83113" containerID="4407c34f059e21949a882b9b730e10be07f5e7b9fb9786cf9259647fe69f7a12" exitCode=0 Dec 03 12:39:28 crc kubenswrapper[5052]: I1203 12:39:28.808831 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"02726407-aa1e-400f-aedb-78d896c83113","Type":"ContainerDied","Data":"4407c34f059e21949a882b9b730e10be07f5e7b9fb9786cf9259647fe69f7a12"} Dec 03 12:39:28 crc kubenswrapper[5052]: I1203 12:39:28.829121 5052 generic.go:334] "Generic (PLEG): container finished" podID="5d4a61d7-2d2c-4186-a923-f59636bc0994" containerID="7ebd36be6c9083bc844d5b32e9c85265741962dbf955670e0381dd41fb13e4d3" exitCode=0 Dec 03 12:39:28 crc kubenswrapper[5052]: I1203 12:39:28.831351 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"5d4a61d7-2d2c-4186-a923-f59636bc0994","Type":"ContainerDied","Data":"7ebd36be6c9083bc844d5b32e9c85265741962dbf955670e0381dd41fb13e4d3"} Dec 03 12:39:28 crc kubenswrapper[5052]: I1203 12:39:28.837230 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"30045a80-9e2b-416b-b7bf-cdb9585ec67d","Type":"ContainerStarted","Data":"c2aa910114395115673efcfd179c79ba17ccb8368f38abf250fd9ed6719330d7"} Dec 03 12:39:28 crc kubenswrapper[5052]: I1203 12:39:28.837278 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"30045a80-9e2b-416b-b7bf-cdb9585ec67d","Type":"ContainerStarted","Data":"22c4b7cc88f18fbfdf0a579df2019523c72c975d8a9b701a3adfaa710425c706"} Dec 03 12:39:28 crc kubenswrapper[5052]: I1203 12:39:28.869502 5052 scope.go:117] "RemoveContainer" containerID="276d7d9067803b5433ccd1312132cf50f0afd9d72333fbe834713799a5eb7360" Dec 03 12:39:28 crc kubenswrapper[5052]: I1203 12:39:28.895628 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:39:28 crc kubenswrapper[5052]: I1203 12:39:28.899144 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-api-764dd4c57d-54dvn"] Dec 03 12:39:28 crc kubenswrapper[5052]: I1203 12:39:28.910762 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/barbican-api-764dd4c57d-54dvn"] Dec 03 12:39:28 crc kubenswrapper[5052]: I1203 12:39:28.933210 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/02726407-aa1e-400f-aedb-78d896c83113-config-data\") pod \"02726407-aa1e-400f-aedb-78d896c83113\" (UID: \"02726407-aa1e-400f-aedb-78d896c83113\") " Dec 03 12:39:28 crc kubenswrapper[5052]: I1203 12:39:28.933253 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4mtft\" (UniqueName: \"kubernetes.io/projected/02726407-aa1e-400f-aedb-78d896c83113-kube-api-access-4mtft\") pod \"02726407-aa1e-400f-aedb-78d896c83113\" (UID: \"02726407-aa1e-400f-aedb-78d896c83113\") " Dec 03 12:39:28 crc kubenswrapper[5052]: I1203 12:39:28.933286 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/02726407-aa1e-400f-aedb-78d896c83113-combined-ca-bundle\") pod \"02726407-aa1e-400f-aedb-78d896c83113\" (UID: \"02726407-aa1e-400f-aedb-78d896c83113\") " Dec 03 12:39:28 crc kubenswrapper[5052]: I1203 12:39:28.933353 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/02726407-aa1e-400f-aedb-78d896c83113-scripts\") pod \"02726407-aa1e-400f-aedb-78d896c83113\" (UID: \"02726407-aa1e-400f-aedb-78d896c83113\") " Dec 03 12:39:28 crc kubenswrapper[5052]: I1203 12:39:28.933465 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/02726407-aa1e-400f-aedb-78d896c83113-logs\") pod \"02726407-aa1e-400f-aedb-78d896c83113\" (UID: \"02726407-aa1e-400f-aedb-78d896c83113\") " Dec 03 12:39:28 crc kubenswrapper[5052]: I1203 12:39:28.933550 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"glance\" (UniqueName: \"kubernetes.io/local-volume/local-storage12-crc\") pod \"02726407-aa1e-400f-aedb-78d896c83113\" (UID: \"02726407-aa1e-400f-aedb-78d896c83113\") " Dec 03 12:39:28 crc kubenswrapper[5052]: I1203 12:39:28.933591 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/02726407-aa1e-400f-aedb-78d896c83113-httpd-run\") pod \"02726407-aa1e-400f-aedb-78d896c83113\" (UID: \"02726407-aa1e-400f-aedb-78d896c83113\") " Dec 03 12:39:28 crc kubenswrapper[5052]: I1203 12:39:28.933619 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/02726407-aa1e-400f-aedb-78d896c83113-public-tls-certs\") pod \"02726407-aa1e-400f-aedb-78d896c83113\" (UID: \"02726407-aa1e-400f-aedb-78d896c83113\") " Dec 03 12:39:28 crc kubenswrapper[5052]: I1203 12:39:28.936230 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/02726407-aa1e-400f-aedb-78d896c83113-logs" (OuterVolumeSpecName: "logs") pod "02726407-aa1e-400f-aedb-78d896c83113" (UID: "02726407-aa1e-400f-aedb-78d896c83113"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:39:28 crc kubenswrapper[5052]: I1203 12:39:28.936696 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/02726407-aa1e-400f-aedb-78d896c83113-httpd-run" (OuterVolumeSpecName: "httpd-run") pod "02726407-aa1e-400f-aedb-78d896c83113" (UID: "02726407-aa1e-400f-aedb-78d896c83113"). InnerVolumeSpecName "httpd-run". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:39:28 crc kubenswrapper[5052]: I1203 12:39:28.944407 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/02726407-aa1e-400f-aedb-78d896c83113-scripts" (OuterVolumeSpecName: "scripts") pod "02726407-aa1e-400f-aedb-78d896c83113" (UID: "02726407-aa1e-400f-aedb-78d896c83113"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:39:28 crc kubenswrapper[5052]: I1203 12:39:28.958764 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage12-crc" (OuterVolumeSpecName: "glance") pod "02726407-aa1e-400f-aedb-78d896c83113" (UID: "02726407-aa1e-400f-aedb-78d896c83113"). InnerVolumeSpecName "local-storage12-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 12:39:28 crc kubenswrapper[5052]: I1203 12:39:28.964045 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/02726407-aa1e-400f-aedb-78d896c83113-kube-api-access-4mtft" (OuterVolumeSpecName: "kube-api-access-4mtft") pod "02726407-aa1e-400f-aedb-78d896c83113" (UID: "02726407-aa1e-400f-aedb-78d896c83113"). InnerVolumeSpecName "kube-api-access-4mtft". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:39:28 crc kubenswrapper[5052]: I1203 12:39:28.988004 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/02726407-aa1e-400f-aedb-78d896c83113-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "02726407-aa1e-400f-aedb-78d896c83113" (UID: "02726407-aa1e-400f-aedb-78d896c83113"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:39:28 crc kubenswrapper[5052]: I1203 12:39:28.991341 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:39:29 crc kubenswrapper[5052]: I1203 12:39:29.036155 5052 reconciler_common.go:293] "Volume detached for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/02726407-aa1e-400f-aedb-78d896c83113-httpd-run\") on node \"crc\" DevicePath \"\"" Dec 03 12:39:29 crc kubenswrapper[5052]: I1203 12:39:29.036726 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-4mtft\" (UniqueName: \"kubernetes.io/projected/02726407-aa1e-400f-aedb-78d896c83113-kube-api-access-4mtft\") on node \"crc\" DevicePath \"\"" Dec 03 12:39:29 crc kubenswrapper[5052]: I1203 12:39:29.036742 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/02726407-aa1e-400f-aedb-78d896c83113-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:39:29 crc kubenswrapper[5052]: I1203 12:39:29.036753 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/02726407-aa1e-400f-aedb-78d896c83113-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:39:29 crc kubenswrapper[5052]: I1203 12:39:29.036766 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/02726407-aa1e-400f-aedb-78d896c83113-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:39:29 crc kubenswrapper[5052]: I1203 12:39:29.036790 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage12-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage12-crc\") on node \"crc\" " Dec 03 12:39:29 crc kubenswrapper[5052]: I1203 12:39:29.045017 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/02726407-aa1e-400f-aedb-78d896c83113-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "02726407-aa1e-400f-aedb-78d896c83113" (UID: "02726407-aa1e-400f-aedb-78d896c83113"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:39:29 crc kubenswrapper[5052]: I1203 12:39:29.062625 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/02726407-aa1e-400f-aedb-78d896c83113-config-data" (OuterVolumeSpecName: "config-data") pod "02726407-aa1e-400f-aedb-78d896c83113" (UID: "02726407-aa1e-400f-aedb-78d896c83113"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:39:29 crc kubenswrapper[5052]: I1203 12:39:29.070680 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage12-crc" (UniqueName: "kubernetes.io/local-volume/local-storage12-crc") on node "crc" Dec 03 12:39:29 crc kubenswrapper[5052]: I1203 12:39:29.095395 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:39:29 crc kubenswrapper[5052]: I1203 12:39:29.095708 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-api-0" podUID="c74fd61a-bf75-4a68-b802-cdc7f4b48be9" containerName="nova-api-log" containerID="cri-o://f74dbe94a7303ed5a5e53c969e42792ff7d1835379c81468a7fb60a94a0d4124" gracePeriod=30 Dec 03 12:39:29 crc kubenswrapper[5052]: I1203 12:39:29.095822 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-api-0" podUID="c74fd61a-bf75-4a68-b802-cdc7f4b48be9" containerName="nova-api-api" containerID="cri-o://4233bebd5d63d6736a338fd2b32760cdc34fecfde4113939f42d9e6e5b01fa67" gracePeriod=30 Dec 03 12:39:29 crc kubenswrapper[5052]: I1203 12:39:29.138966 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage12-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage12-crc\") on node \"crc\" DevicePath \"\"" Dec 03 12:39:29 crc kubenswrapper[5052]: I1203 12:39:29.139008 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/02726407-aa1e-400f-aedb-78d896c83113-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:39:29 crc kubenswrapper[5052]: I1203 12:39:29.139018 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/02726407-aa1e-400f-aedb-78d896c83113-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:39:29 crc kubenswrapper[5052]: I1203 12:39:29.308974 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:39:29 crc kubenswrapper[5052]: I1203 12:39:29.346502 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/5d4a61d7-2d2c-4186-a923-f59636bc0994-httpd-run\") pod \"5d4a61d7-2d2c-4186-a923-f59636bc0994\" (UID: \"5d4a61d7-2d2c-4186-a923-f59636bc0994\") " Dec 03 12:39:29 crc kubenswrapper[5052]: I1203 12:39:29.346665 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/5d4a61d7-2d2c-4186-a923-f59636bc0994-internal-tls-certs\") pod \"5d4a61d7-2d2c-4186-a923-f59636bc0994\" (UID: \"5d4a61d7-2d2c-4186-a923-f59636bc0994\") " Dec 03 12:39:29 crc kubenswrapper[5052]: I1203 12:39:29.346716 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mqvdm\" (UniqueName: \"kubernetes.io/projected/5d4a61d7-2d2c-4186-a923-f59636bc0994-kube-api-access-mqvdm\") pod \"5d4a61d7-2d2c-4186-a923-f59636bc0994\" (UID: \"5d4a61d7-2d2c-4186-a923-f59636bc0994\") " Dec 03 12:39:29 crc kubenswrapper[5052]: I1203 12:39:29.346752 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5d4a61d7-2d2c-4186-a923-f59636bc0994-combined-ca-bundle\") pod \"5d4a61d7-2d2c-4186-a923-f59636bc0994\" (UID: \"5d4a61d7-2d2c-4186-a923-f59636bc0994\") " Dec 03 12:39:29 crc kubenswrapper[5052]: I1203 12:39:29.346795 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"glance\" (UniqueName: \"kubernetes.io/local-volume/local-storage18-crc\") pod \"5d4a61d7-2d2c-4186-a923-f59636bc0994\" (UID: \"5d4a61d7-2d2c-4186-a923-f59636bc0994\") " Dec 03 12:39:29 crc kubenswrapper[5052]: I1203 12:39:29.346821 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/5d4a61d7-2d2c-4186-a923-f59636bc0994-config-data\") pod \"5d4a61d7-2d2c-4186-a923-f59636bc0994\" (UID: \"5d4a61d7-2d2c-4186-a923-f59636bc0994\") " Dec 03 12:39:29 crc kubenswrapper[5052]: I1203 12:39:29.346982 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/5d4a61d7-2d2c-4186-a923-f59636bc0994-scripts\") pod \"5d4a61d7-2d2c-4186-a923-f59636bc0994\" (UID: \"5d4a61d7-2d2c-4186-a923-f59636bc0994\") " Dec 03 12:39:29 crc kubenswrapper[5052]: I1203 12:39:29.347018 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/5d4a61d7-2d2c-4186-a923-f59636bc0994-logs\") pod \"5d4a61d7-2d2c-4186-a923-f59636bc0994\" (UID: \"5d4a61d7-2d2c-4186-a923-f59636bc0994\") " Dec 03 12:39:29 crc kubenswrapper[5052]: I1203 12:39:29.348068 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5d4a61d7-2d2c-4186-a923-f59636bc0994-httpd-run" (OuterVolumeSpecName: "httpd-run") pod "5d4a61d7-2d2c-4186-a923-f59636bc0994" (UID: "5d4a61d7-2d2c-4186-a923-f59636bc0994"). InnerVolumeSpecName "httpd-run". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:39:29 crc kubenswrapper[5052]: I1203 12:39:29.348207 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5d4a61d7-2d2c-4186-a923-f59636bc0994-logs" (OuterVolumeSpecName: "logs") pod "5d4a61d7-2d2c-4186-a923-f59636bc0994" (UID: "5d4a61d7-2d2c-4186-a923-f59636bc0994"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:39:29 crc kubenswrapper[5052]: I1203 12:39:29.361875 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5d4a61d7-2d2c-4186-a923-f59636bc0994-scripts" (OuterVolumeSpecName: "scripts") pod "5d4a61d7-2d2c-4186-a923-f59636bc0994" (UID: "5d4a61d7-2d2c-4186-a923-f59636bc0994"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:39:29 crc kubenswrapper[5052]: I1203 12:39:29.362911 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage18-crc" (OuterVolumeSpecName: "glance") pod "5d4a61d7-2d2c-4186-a923-f59636bc0994" (UID: "5d4a61d7-2d2c-4186-a923-f59636bc0994"). InnerVolumeSpecName "local-storage18-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 12:39:29 crc kubenswrapper[5052]: I1203 12:39:29.363116 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5d4a61d7-2d2c-4186-a923-f59636bc0994-kube-api-access-mqvdm" (OuterVolumeSpecName: "kube-api-access-mqvdm") pod "5d4a61d7-2d2c-4186-a923-f59636bc0994" (UID: "5d4a61d7-2d2c-4186-a923-f59636bc0994"). InnerVolumeSpecName "kube-api-access-mqvdm". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:39:29 crc kubenswrapper[5052]: I1203 12:39:29.387697 5052 patch_prober.go:28] interesting pod/machine-config-daemon-74bj6 container/machine-config-daemon namespace/openshift-machine-config-operator: Liveness probe status=failure output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" start-of-body= Dec 03 12:39:29 crc kubenswrapper[5052]: I1203 12:39:29.387752 5052 prober.go:107] "Probe failed" probeType="Liveness" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" containerName="machine-config-daemon" probeResult="failure" output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" Dec 03 12:39:29 crc kubenswrapper[5052]: I1203 12:39:29.440121 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5d4a61d7-2d2c-4186-a923-f59636bc0994-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "5d4a61d7-2d2c-4186-a923-f59636bc0994" (UID: "5d4a61d7-2d2c-4186-a923-f59636bc0994"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:39:29 crc kubenswrapper[5052]: I1203 12:39:29.440863 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5d4a61d7-2d2c-4186-a923-f59636bc0994-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "5d4a61d7-2d2c-4186-a923-f59636bc0994" (UID: "5d4a61d7-2d2c-4186-a923-f59636bc0994"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:39:29 crc kubenswrapper[5052]: I1203 12:39:29.452101 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/5d4a61d7-2d2c-4186-a923-f59636bc0994-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:39:29 crc kubenswrapper[5052]: I1203 12:39:29.452129 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/5d4a61d7-2d2c-4186-a923-f59636bc0994-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:39:29 crc kubenswrapper[5052]: I1203 12:39:29.452141 5052 reconciler_common.go:293] "Volume detached for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/5d4a61d7-2d2c-4186-a923-f59636bc0994-httpd-run\") on node \"crc\" DevicePath \"\"" Dec 03 12:39:29 crc kubenswrapper[5052]: I1203 12:39:29.452152 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/5d4a61d7-2d2c-4186-a923-f59636bc0994-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:39:29 crc kubenswrapper[5052]: I1203 12:39:29.452167 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-mqvdm\" (UniqueName: \"kubernetes.io/projected/5d4a61d7-2d2c-4186-a923-f59636bc0994-kube-api-access-mqvdm\") on node \"crc\" DevicePath \"\"" Dec 03 12:39:29 crc kubenswrapper[5052]: I1203 12:39:29.452177 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5d4a61d7-2d2c-4186-a923-f59636bc0994-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:39:29 crc kubenswrapper[5052]: I1203 12:39:29.452199 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage18-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage18-crc\") on node \"crc\" " Dec 03 12:39:29 crc kubenswrapper[5052]: W1203 12:39:29.477785 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod95a06af5_270d_4138_ab79_fcdd239b3997.slice/crio-f5a24a550711d5cd372d9ce54cc9aec579be9f6116ef3c3e698d6afc2b1c85b4 WatchSource:0}: Error finding container f5a24a550711d5cd372d9ce54cc9aec579be9f6116ef3c3e698d6afc2b1c85b4: Status 404 returned error can't find the container with id f5a24a550711d5cd372d9ce54cc9aec579be9f6116ef3c3e698d6afc2b1c85b4 Dec 03 12:39:29 crc kubenswrapper[5052]: I1203 12:39:29.486914 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage18-crc" (UniqueName: "kubernetes.io/local-volume/local-storage18-crc") on node "crc" Dec 03 12:39:29 crc kubenswrapper[5052]: I1203 12:39:29.490228 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5d4a61d7-2d2c-4186-a923-f59636bc0994-config-data" (OuterVolumeSpecName: "config-data") pod "5d4a61d7-2d2c-4186-a923-f59636bc0994" (UID: "5d4a61d7-2d2c-4186-a923-f59636bc0994"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:39:29 crc kubenswrapper[5052]: I1203 12:39:29.511620 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:39:29 crc kubenswrapper[5052]: I1203 12:39:29.555200 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage18-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage18-crc\") on node \"crc\" DevicePath \"\"" Dec 03 12:39:29 crc kubenswrapper[5052]: I1203 12:39:29.555242 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/5d4a61d7-2d2c-4186-a923-f59636bc0994-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:39:29 crc kubenswrapper[5052]: I1203 12:39:29.588361 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-7d74684fd-pqkf5"] Dec 03 12:39:29 crc kubenswrapper[5052]: I1203 12:39:29.602081 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-7b8cddb768-5snf9" Dec 03 12:39:29 crc kubenswrapper[5052]: I1203 12:39:29.656288 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/2972cf0c-5378-4e71-b308-0235608d5bef-config-data\") pod \"2972cf0c-5378-4e71-b308-0235608d5bef\" (UID: \"2972cf0c-5378-4e71-b308-0235608d5bef\") " Dec 03 12:39:29 crc kubenswrapper[5052]: I1203 12:39:29.656549 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/2972cf0c-5378-4e71-b308-0235608d5bef-credential-keys\") pod \"2972cf0c-5378-4e71-b308-0235608d5bef\" (UID: \"2972cf0c-5378-4e71-b308-0235608d5bef\") " Dec 03 12:39:29 crc kubenswrapper[5052]: I1203 12:39:29.656666 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/2972cf0c-5378-4e71-b308-0235608d5bef-fernet-keys\") pod \"2972cf0c-5378-4e71-b308-0235608d5bef\" (UID: \"2972cf0c-5378-4e71-b308-0235608d5bef\") " Dec 03 12:39:29 crc kubenswrapper[5052]: I1203 12:39:29.656756 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/2972cf0c-5378-4e71-b308-0235608d5bef-scripts\") pod \"2972cf0c-5378-4e71-b308-0235608d5bef\" (UID: \"2972cf0c-5378-4e71-b308-0235608d5bef\") " Dec 03 12:39:29 crc kubenswrapper[5052]: I1203 12:39:29.656783 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/2972cf0c-5378-4e71-b308-0235608d5bef-internal-tls-certs\") pod \"2972cf0c-5378-4e71-b308-0235608d5bef\" (UID: \"2972cf0c-5378-4e71-b308-0235608d5bef\") " Dec 03 12:39:29 crc kubenswrapper[5052]: I1203 12:39:29.656861 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k799r\" (UniqueName: \"kubernetes.io/projected/2972cf0c-5378-4e71-b308-0235608d5bef-kube-api-access-k799r\") pod \"2972cf0c-5378-4e71-b308-0235608d5bef\" (UID: \"2972cf0c-5378-4e71-b308-0235608d5bef\") " Dec 03 12:39:29 crc kubenswrapper[5052]: I1203 12:39:29.656975 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/2972cf0c-5378-4e71-b308-0235608d5bef-public-tls-certs\") pod \"2972cf0c-5378-4e71-b308-0235608d5bef\" (UID: \"2972cf0c-5378-4e71-b308-0235608d5bef\") " Dec 03 12:39:29 crc kubenswrapper[5052]: I1203 12:39:29.657002 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2972cf0c-5378-4e71-b308-0235608d5bef-combined-ca-bundle\") pod \"2972cf0c-5378-4e71-b308-0235608d5bef\" (UID: \"2972cf0c-5378-4e71-b308-0235608d5bef\") " Dec 03 12:39:29 crc kubenswrapper[5052]: I1203 12:39:29.676713 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2972cf0c-5378-4e71-b308-0235608d5bef-scripts" (OuterVolumeSpecName: "scripts") pod "2972cf0c-5378-4e71-b308-0235608d5bef" (UID: "2972cf0c-5378-4e71-b308-0235608d5bef"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:39:29 crc kubenswrapper[5052]: I1203 12:39:29.690596 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2972cf0c-5378-4e71-b308-0235608d5bef-fernet-keys" (OuterVolumeSpecName: "fernet-keys") pod "2972cf0c-5378-4e71-b308-0235608d5bef" (UID: "2972cf0c-5378-4e71-b308-0235608d5bef"). InnerVolumeSpecName "fernet-keys". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:39:29 crc kubenswrapper[5052]: I1203 12:39:29.690627 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2972cf0c-5378-4e71-b308-0235608d5bef-credential-keys" (OuterVolumeSpecName: "credential-keys") pod "2972cf0c-5378-4e71-b308-0235608d5bef" (UID: "2972cf0c-5378-4e71-b308-0235608d5bef"). InnerVolumeSpecName "credential-keys". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:39:29 crc kubenswrapper[5052]: I1203 12:39:29.693506 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2972cf0c-5378-4e71-b308-0235608d5bef-kube-api-access-k799r" (OuterVolumeSpecName: "kube-api-access-k799r") pod "2972cf0c-5378-4e71-b308-0235608d5bef" (UID: "2972cf0c-5378-4e71-b308-0235608d5bef"). InnerVolumeSpecName "kube-api-access-k799r". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:39:29 crc kubenswrapper[5052]: I1203 12:39:29.712047 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="59dcc834-7abb-4dff-a10f-6127276698ee" path="/var/lib/kubelet/pods/59dcc834-7abb-4dff-a10f-6127276698ee/volumes" Dec 03 12:39:29 crc kubenswrapper[5052]: I1203 12:39:29.718890 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5da4fd9f-78ea-4511-ba09-f00694058c69" path="/var/lib/kubelet/pods/5da4fd9f-78ea-4511-ba09-f00694058c69/volumes" Dec 03 12:39:29 crc kubenswrapper[5052]: I1203 12:39:29.742375 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2972cf0c-5378-4e71-b308-0235608d5bef-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "2972cf0c-5378-4e71-b308-0235608d5bef" (UID: "2972cf0c-5378-4e71-b308-0235608d5bef"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:39:29 crc kubenswrapper[5052]: I1203 12:39:29.765274 5052 reconciler_common.go:293] "Volume detached for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/2972cf0c-5378-4e71-b308-0235608d5bef-fernet-keys\") on node \"crc\" DevicePath \"\"" Dec 03 12:39:29 crc kubenswrapper[5052]: I1203 12:39:29.765308 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/2972cf0c-5378-4e71-b308-0235608d5bef-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:39:29 crc kubenswrapper[5052]: I1203 12:39:29.765320 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-k799r\" (UniqueName: \"kubernetes.io/projected/2972cf0c-5378-4e71-b308-0235608d5bef-kube-api-access-k799r\") on node \"crc\" DevicePath \"\"" Dec 03 12:39:29 crc kubenswrapper[5052]: I1203 12:39:29.765333 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2972cf0c-5378-4e71-b308-0235608d5bef-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:39:29 crc kubenswrapper[5052]: I1203 12:39:29.765346 5052 reconciler_common.go:293] "Volume detached for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/2972cf0c-5378-4e71-b308-0235608d5bef-credential-keys\") on node \"crc\" DevicePath \"\"" Dec 03 12:39:29 crc kubenswrapper[5052]: I1203 12:39:29.770142 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2972cf0c-5378-4e71-b308-0235608d5bef-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "2972cf0c-5378-4e71-b308-0235608d5bef" (UID: "2972cf0c-5378-4e71-b308-0235608d5bef"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:39:29 crc kubenswrapper[5052]: I1203 12:39:29.840801 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2972cf0c-5378-4e71-b308-0235608d5bef-config-data" (OuterVolumeSpecName: "config-data") pod "2972cf0c-5378-4e71-b308-0235608d5bef" (UID: "2972cf0c-5378-4e71-b308-0235608d5bef"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:39:29 crc kubenswrapper[5052]: I1203 12:39:29.855190 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"02726407-aa1e-400f-aedb-78d896c83113","Type":"ContainerDied","Data":"b35d25d646491d5ba914899b4a1958f71c9b4b047d5f01932e519ac36f40ddcf"} Dec 03 12:39:29 crc kubenswrapper[5052]: I1203 12:39:29.855251 5052 scope.go:117] "RemoveContainer" containerID="4407c34f059e21949a882b9b730e10be07f5e7b9fb9786cf9259647fe69f7a12" Dec 03 12:39:29 crc kubenswrapper[5052]: I1203 12:39:29.855313 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:39:29 crc kubenswrapper[5052]: I1203 12:39:29.859250 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"5d4a61d7-2d2c-4186-a923-f59636bc0994","Type":"ContainerDied","Data":"fa4221771479fd3430e53e3dbc74221c74fd7652dcd22d0b3f64f1871afeca86"} Dec 03 12:39:29 crc kubenswrapper[5052]: I1203 12:39:29.859333 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:39:29 crc kubenswrapper[5052]: I1203 12:39:29.864209 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"30045a80-9e2b-416b-b7bf-cdb9585ec67d","Type":"ContainerStarted","Data":"0d3fa6f69a8171deaa21727301ede51856b6793ff9d13fde6cbe39442a2e4566"} Dec 03 12:39:29 crc kubenswrapper[5052]: I1203 12:39:29.867240 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/2972cf0c-5378-4e71-b308-0235608d5bef-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:39:29 crc kubenswrapper[5052]: I1203 12:39:29.867272 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/2972cf0c-5378-4e71-b308-0235608d5bef-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:39:29 crc kubenswrapper[5052]: I1203 12:39:29.867655 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-7d74684fd-pqkf5" event={"ID":"ddfa4a3f-b868-49b7-abec-62cb3ffde2bd","Type":"ContainerStarted","Data":"31851794851d731aa49dc5c7f2d98f48ac2eac1b939d702428bd58b2d363091c"} Dec 03 12:39:29 crc kubenswrapper[5052]: I1203 12:39:29.871042 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"95a06af5-270d-4138-ab79-fcdd239b3997","Type":"ContainerStarted","Data":"f5a24a550711d5cd372d9ce54cc9aec579be9f6116ef3c3e698d6afc2b1c85b4"} Dec 03 12:39:29 crc kubenswrapper[5052]: I1203 12:39:29.872745 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-7b8cddb768-5snf9" event={"ID":"2972cf0c-5378-4e71-b308-0235608d5bef","Type":"ContainerDied","Data":"9e8c8cfe7e17bc4dae78f6e83c14daf6847b88ce114876f007358139a98604c8"} Dec 03 12:39:29 crc kubenswrapper[5052]: I1203 12:39:29.872802 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-7b8cddb768-5snf9" Dec 03 12:39:29 crc kubenswrapper[5052]: I1203 12:39:29.875471 5052 generic.go:334] "Generic (PLEG): container finished" podID="c74fd61a-bf75-4a68-b802-cdc7f4b48be9" containerID="f74dbe94a7303ed5a5e53c969e42792ff7d1835379c81468a7fb60a94a0d4124" exitCode=143 Dec 03 12:39:29 crc kubenswrapper[5052]: I1203 12:39:29.875549 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"c74fd61a-bf75-4a68-b802-cdc7f4b48be9","Type":"ContainerDied","Data":"f74dbe94a7303ed5a5e53c969e42792ff7d1835379c81468a7fb60a94a0d4124"} Dec 03 12:39:29 crc kubenswrapper[5052]: I1203 12:39:29.888434 5052 scope.go:117] "RemoveContainer" containerID="df1cbd0a5a0599784a42f96943876ae3db85b427e8a37874b50da6af3c45e953" Dec 03 12:39:29 crc kubenswrapper[5052]: I1203 12:39:29.896764 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2972cf0c-5378-4e71-b308-0235608d5bef-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "2972cf0c-5378-4e71-b308-0235608d5bef" (UID: "2972cf0c-5378-4e71-b308-0235608d5bef"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:39:29 crc kubenswrapper[5052]: I1203 12:39:29.919772 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-metadata-0" podStartSLOduration=3.919750277 podStartE2EDuration="3.919750277s" podCreationTimestamp="2025-12-03 12:39:26 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:39:29.885080326 +0000 UTC m=+4760.473322090" watchObservedRunningTime="2025-12-03 12:39:29.919750277 +0000 UTC m=+4760.507992041" Dec 03 12:39:29 crc kubenswrapper[5052]: I1203 12:39:29.930920 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:39:29 crc kubenswrapper[5052]: I1203 12:39:29.950295 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:39:29 crc kubenswrapper[5052]: I1203 12:39:29.952740 5052 scope.go:117] "RemoveContainer" containerID="7ebd36be6c9083bc844d5b32e9c85265741962dbf955670e0381dd41fb13e4d3" Dec 03 12:39:29 crc kubenswrapper[5052]: I1203 12:39:29.966765 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:39:29 crc kubenswrapper[5052]: I1203 12:39:29.969003 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/2972cf0c-5378-4e71-b308-0235608d5bef-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:39:29 crc kubenswrapper[5052]: I1203 12:39:29.989324 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:39:29 crc kubenswrapper[5052]: I1203 12:39:29.990143 5052 scope.go:117] "RemoveContainer" containerID="2981d01a1325fdf3537f8cf2d9ade2af4707530bbb2ce6666c3b3503c416127d" Dec 03 12:39:30 crc kubenswrapper[5052]: I1203 12:39:30.004664 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:39:30 crc kubenswrapper[5052]: E1203 12:39:30.005138 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="02726407-aa1e-400f-aedb-78d896c83113" containerName="glance-log" Dec 03 12:39:30 crc kubenswrapper[5052]: I1203 12:39:30.005150 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="02726407-aa1e-400f-aedb-78d896c83113" containerName="glance-log" Dec 03 12:39:30 crc kubenswrapper[5052]: E1203 12:39:30.005161 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="5d4a61d7-2d2c-4186-a923-f59636bc0994" containerName="glance-httpd" Dec 03 12:39:30 crc kubenswrapper[5052]: I1203 12:39:30.005169 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="5d4a61d7-2d2c-4186-a923-f59636bc0994" containerName="glance-httpd" Dec 03 12:39:30 crc kubenswrapper[5052]: E1203 12:39:30.005194 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="5d4a61d7-2d2c-4186-a923-f59636bc0994" containerName="glance-log" Dec 03 12:39:30 crc kubenswrapper[5052]: I1203 12:39:30.005201 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="5d4a61d7-2d2c-4186-a923-f59636bc0994" containerName="glance-log" Dec 03 12:39:30 crc kubenswrapper[5052]: E1203 12:39:30.005216 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="02726407-aa1e-400f-aedb-78d896c83113" containerName="glance-httpd" Dec 03 12:39:30 crc kubenswrapper[5052]: I1203 12:39:30.005222 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="02726407-aa1e-400f-aedb-78d896c83113" containerName="glance-httpd" Dec 03 12:39:30 crc kubenswrapper[5052]: E1203 12:39:30.005230 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="2972cf0c-5378-4e71-b308-0235608d5bef" containerName="keystone-api" Dec 03 12:39:30 crc kubenswrapper[5052]: I1203 12:39:30.005235 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="2972cf0c-5378-4e71-b308-0235608d5bef" containerName="keystone-api" Dec 03 12:39:30 crc kubenswrapper[5052]: I1203 12:39:30.005418 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="5d4a61d7-2d2c-4186-a923-f59636bc0994" containerName="glance-log" Dec 03 12:39:30 crc kubenswrapper[5052]: I1203 12:39:30.005431 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="02726407-aa1e-400f-aedb-78d896c83113" containerName="glance-httpd" Dec 03 12:39:30 crc kubenswrapper[5052]: I1203 12:39:30.005443 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="5d4a61d7-2d2c-4186-a923-f59636bc0994" containerName="glance-httpd" Dec 03 12:39:30 crc kubenswrapper[5052]: I1203 12:39:30.005461 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="02726407-aa1e-400f-aedb-78d896c83113" containerName="glance-log" Dec 03 12:39:30 crc kubenswrapper[5052]: I1203 12:39:30.005474 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="2972cf0c-5378-4e71-b308-0235608d5bef" containerName="keystone-api" Dec 03 12:39:30 crc kubenswrapper[5052]: I1203 12:39:30.006607 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:39:30 crc kubenswrapper[5052]: I1203 12:39:30.009302 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-scripts" Dec 03 12:39:30 crc kubenswrapper[5052]: I1203 12:39:30.009634 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-glance-dockercfg-xkmzv" Dec 03 12:39:30 crc kubenswrapper[5052]: I1203 12:39:30.009928 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-default-external-config-data" Dec 03 12:39:30 crc kubenswrapper[5052]: I1203 12:39:30.011955 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-glance-default-public-svc" Dec 03 12:39:30 crc kubenswrapper[5052]: I1203 12:39:30.018253 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:39:30 crc kubenswrapper[5052]: I1203 12:39:30.034978 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:39:30 crc kubenswrapper[5052]: I1203 12:39:30.037795 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:39:30 crc kubenswrapper[5052]: I1203 12:39:30.040258 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-default-internal-config-data" Dec 03 12:39:30 crc kubenswrapper[5052]: I1203 12:39:30.042160 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-glance-default-internal-svc" Dec 03 12:39:30 crc kubenswrapper[5052]: I1203 12:39:30.053252 5052 scope.go:117] "RemoveContainer" containerID="1e9c082372288c89143c066dc4a3ffdcb4a743df94ac5b3327dcb6b74a2be2a7" Dec 03 12:39:30 crc kubenswrapper[5052]: I1203 12:39:30.056738 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:39:30 crc kubenswrapper[5052]: I1203 12:39:30.174844 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage18-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage18-crc\") pod \"glance-default-internal-api-0\" (UID: \"3039127c-0211-4430-9b2b-426412241ce4\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:39:30 crc kubenswrapper[5052]: I1203 12:39:30.174921 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/b0606156-f409-4cae-9be1-15d990f69955-public-tls-certs\") pod \"glance-default-external-api-0\" (UID: \"b0606156-f409-4cae-9be1-15d990f69955\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:39:30 crc kubenswrapper[5052]: I1203 12:39:30.174973 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b0606156-f409-4cae-9be1-15d990f69955-combined-ca-bundle\") pod \"glance-default-external-api-0\" (UID: \"b0606156-f409-4cae-9be1-15d990f69955\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:39:30 crc kubenswrapper[5052]: I1203 12:39:30.174995 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/b0606156-f409-4cae-9be1-15d990f69955-httpd-run\") pod \"glance-default-external-api-0\" (UID: \"b0606156-f409-4cae-9be1-15d990f69955\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:39:30 crc kubenswrapper[5052]: I1203 12:39:30.175045 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage12-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage12-crc\") pod \"glance-default-external-api-0\" (UID: \"b0606156-f409-4cae-9be1-15d990f69955\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:39:30 crc kubenswrapper[5052]: I1203 12:39:30.175068 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/3039127c-0211-4430-9b2b-426412241ce4-config-data\") pod \"glance-default-internal-api-0\" (UID: \"3039127c-0211-4430-9b2b-426412241ce4\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:39:30 crc kubenswrapper[5052]: I1203 12:39:30.175099 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/3039127c-0211-4430-9b2b-426412241ce4-httpd-run\") pod \"glance-default-internal-api-0\" (UID: \"3039127c-0211-4430-9b2b-426412241ce4\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:39:30 crc kubenswrapper[5052]: I1203 12:39:30.175121 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ssfhj\" (UniqueName: \"kubernetes.io/projected/b0606156-f409-4cae-9be1-15d990f69955-kube-api-access-ssfhj\") pod \"glance-default-external-api-0\" (UID: \"b0606156-f409-4cae-9be1-15d990f69955\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:39:30 crc kubenswrapper[5052]: I1203 12:39:30.175150 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3039127c-0211-4430-9b2b-426412241ce4-combined-ca-bundle\") pod \"glance-default-internal-api-0\" (UID: \"3039127c-0211-4430-9b2b-426412241ce4\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:39:30 crc kubenswrapper[5052]: I1203 12:39:30.175178 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mclw8\" (UniqueName: \"kubernetes.io/projected/3039127c-0211-4430-9b2b-426412241ce4-kube-api-access-mclw8\") pod \"glance-default-internal-api-0\" (UID: \"3039127c-0211-4430-9b2b-426412241ce4\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:39:30 crc kubenswrapper[5052]: I1203 12:39:30.175229 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/3039127c-0211-4430-9b2b-426412241ce4-logs\") pod \"glance-default-internal-api-0\" (UID: \"3039127c-0211-4430-9b2b-426412241ce4\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:39:30 crc kubenswrapper[5052]: I1203 12:39:30.175274 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/3039127c-0211-4430-9b2b-426412241ce4-internal-tls-certs\") pod \"glance-default-internal-api-0\" (UID: \"3039127c-0211-4430-9b2b-426412241ce4\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:39:30 crc kubenswrapper[5052]: I1203 12:39:30.175294 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/b0606156-f409-4cae-9be1-15d990f69955-logs\") pod \"glance-default-external-api-0\" (UID: \"b0606156-f409-4cae-9be1-15d990f69955\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:39:30 crc kubenswrapper[5052]: I1203 12:39:30.175317 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/b0606156-f409-4cae-9be1-15d990f69955-config-data\") pod \"glance-default-external-api-0\" (UID: \"b0606156-f409-4cae-9be1-15d990f69955\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:39:30 crc kubenswrapper[5052]: I1203 12:39:30.175333 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/3039127c-0211-4430-9b2b-426412241ce4-scripts\") pod \"glance-default-internal-api-0\" (UID: \"3039127c-0211-4430-9b2b-426412241ce4\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:39:30 crc kubenswrapper[5052]: I1203 12:39:30.175363 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/b0606156-f409-4cae-9be1-15d990f69955-scripts\") pod \"glance-default-external-api-0\" (UID: \"b0606156-f409-4cae-9be1-15d990f69955\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:39:30 crc kubenswrapper[5052]: I1203 12:39:30.259575 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/keystone-7b8cddb768-5snf9"] Dec 03 12:39:30 crc kubenswrapper[5052]: I1203 12:39:30.283990 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/keystone-7b8cddb768-5snf9"] Dec 03 12:39:30 crc kubenswrapper[5052]: I1203 12:39:30.289922 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/3039127c-0211-4430-9b2b-426412241ce4-internal-tls-certs\") pod \"glance-default-internal-api-0\" (UID: \"3039127c-0211-4430-9b2b-426412241ce4\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:39:30 crc kubenswrapper[5052]: I1203 12:39:30.289957 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/b0606156-f409-4cae-9be1-15d990f69955-logs\") pod \"glance-default-external-api-0\" (UID: \"b0606156-f409-4cae-9be1-15d990f69955\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:39:30 crc kubenswrapper[5052]: I1203 12:39:30.289990 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/b0606156-f409-4cae-9be1-15d990f69955-config-data\") pod \"glance-default-external-api-0\" (UID: \"b0606156-f409-4cae-9be1-15d990f69955\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:39:30 crc kubenswrapper[5052]: I1203 12:39:30.290010 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/3039127c-0211-4430-9b2b-426412241ce4-scripts\") pod \"glance-default-internal-api-0\" (UID: \"3039127c-0211-4430-9b2b-426412241ce4\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:39:30 crc kubenswrapper[5052]: I1203 12:39:30.290078 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/b0606156-f409-4cae-9be1-15d990f69955-scripts\") pod \"glance-default-external-api-0\" (UID: \"b0606156-f409-4cae-9be1-15d990f69955\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:39:30 crc kubenswrapper[5052]: I1203 12:39:30.290167 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage18-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage18-crc\") pod \"glance-default-internal-api-0\" (UID: \"3039127c-0211-4430-9b2b-426412241ce4\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:39:30 crc kubenswrapper[5052]: I1203 12:39:30.290208 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/b0606156-f409-4cae-9be1-15d990f69955-public-tls-certs\") pod \"glance-default-external-api-0\" (UID: \"b0606156-f409-4cae-9be1-15d990f69955\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:39:30 crc kubenswrapper[5052]: I1203 12:39:30.290242 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b0606156-f409-4cae-9be1-15d990f69955-combined-ca-bundle\") pod \"glance-default-external-api-0\" (UID: \"b0606156-f409-4cae-9be1-15d990f69955\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:39:30 crc kubenswrapper[5052]: I1203 12:39:30.290256 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/b0606156-f409-4cae-9be1-15d990f69955-httpd-run\") pod \"glance-default-external-api-0\" (UID: \"b0606156-f409-4cae-9be1-15d990f69955\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:39:30 crc kubenswrapper[5052]: I1203 12:39:30.290295 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage12-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage12-crc\") pod \"glance-default-external-api-0\" (UID: \"b0606156-f409-4cae-9be1-15d990f69955\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:39:30 crc kubenswrapper[5052]: I1203 12:39:30.290329 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/3039127c-0211-4430-9b2b-426412241ce4-config-data\") pod \"glance-default-internal-api-0\" (UID: \"3039127c-0211-4430-9b2b-426412241ce4\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:39:30 crc kubenswrapper[5052]: I1203 12:39:30.290362 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/3039127c-0211-4430-9b2b-426412241ce4-httpd-run\") pod \"glance-default-internal-api-0\" (UID: \"3039127c-0211-4430-9b2b-426412241ce4\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:39:30 crc kubenswrapper[5052]: I1203 12:39:30.290388 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-ssfhj\" (UniqueName: \"kubernetes.io/projected/b0606156-f409-4cae-9be1-15d990f69955-kube-api-access-ssfhj\") pod \"glance-default-external-api-0\" (UID: \"b0606156-f409-4cae-9be1-15d990f69955\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:39:30 crc kubenswrapper[5052]: I1203 12:39:30.290423 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3039127c-0211-4430-9b2b-426412241ce4-combined-ca-bundle\") pod \"glance-default-internal-api-0\" (UID: \"3039127c-0211-4430-9b2b-426412241ce4\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:39:30 crc kubenswrapper[5052]: I1203 12:39:30.290462 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-mclw8\" (UniqueName: \"kubernetes.io/projected/3039127c-0211-4430-9b2b-426412241ce4-kube-api-access-mclw8\") pod \"glance-default-internal-api-0\" (UID: \"3039127c-0211-4430-9b2b-426412241ce4\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:39:30 crc kubenswrapper[5052]: I1203 12:39:30.290552 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/3039127c-0211-4430-9b2b-426412241ce4-logs\") pod \"glance-default-internal-api-0\" (UID: \"3039127c-0211-4430-9b2b-426412241ce4\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:39:30 crc kubenswrapper[5052]: I1203 12:39:30.290968 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/3039127c-0211-4430-9b2b-426412241ce4-logs\") pod \"glance-default-internal-api-0\" (UID: \"3039127c-0211-4430-9b2b-426412241ce4\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:39:30 crc kubenswrapper[5052]: I1203 12:39:30.296603 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/3039127c-0211-4430-9b2b-426412241ce4-httpd-run\") pod \"glance-default-internal-api-0\" (UID: \"3039127c-0211-4430-9b2b-426412241ce4\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:39:30 crc kubenswrapper[5052]: I1203 12:39:30.297001 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/b0606156-f409-4cae-9be1-15d990f69955-httpd-run\") pod \"glance-default-external-api-0\" (UID: \"b0606156-f409-4cae-9be1-15d990f69955\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:39:30 crc kubenswrapper[5052]: I1203 12:39:30.297380 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage12-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage12-crc\") pod \"glance-default-external-api-0\" (UID: \"b0606156-f409-4cae-9be1-15d990f69955\") device mount path \"/mnt/openstack/pv12\"" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:39:30 crc kubenswrapper[5052]: I1203 12:39:30.302993 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage18-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage18-crc\") pod \"glance-default-internal-api-0\" (UID: \"3039127c-0211-4430-9b2b-426412241ce4\") device mount path \"/mnt/openstack/pv18\"" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:39:30 crc kubenswrapper[5052]: I1203 12:39:30.304626 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/3039127c-0211-4430-9b2b-426412241ce4-scripts\") pod \"glance-default-internal-api-0\" (UID: \"3039127c-0211-4430-9b2b-426412241ce4\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:39:30 crc kubenswrapper[5052]: I1203 12:39:30.304899 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/b0606156-f409-4cae-9be1-15d990f69955-logs\") pod \"glance-default-external-api-0\" (UID: \"b0606156-f409-4cae-9be1-15d990f69955\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:39:30 crc kubenswrapper[5052]: I1203 12:39:30.330352 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b0606156-f409-4cae-9be1-15d990f69955-combined-ca-bundle\") pod \"glance-default-external-api-0\" (UID: \"b0606156-f409-4cae-9be1-15d990f69955\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:39:30 crc kubenswrapper[5052]: I1203 12:39:30.330412 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/3039127c-0211-4430-9b2b-426412241ce4-internal-tls-certs\") pod \"glance-default-internal-api-0\" (UID: \"3039127c-0211-4430-9b2b-426412241ce4\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:39:30 crc kubenswrapper[5052]: I1203 12:39:30.331316 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3039127c-0211-4430-9b2b-426412241ce4-combined-ca-bundle\") pod \"glance-default-internal-api-0\" (UID: \"3039127c-0211-4430-9b2b-426412241ce4\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:39:30 crc kubenswrapper[5052]: I1203 12:39:30.359451 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/b0606156-f409-4cae-9be1-15d990f69955-scripts\") pod \"glance-default-external-api-0\" (UID: \"b0606156-f409-4cae-9be1-15d990f69955\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:39:30 crc kubenswrapper[5052]: I1203 12:39:30.360583 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/b0606156-f409-4cae-9be1-15d990f69955-config-data\") pod \"glance-default-external-api-0\" (UID: \"b0606156-f409-4cae-9be1-15d990f69955\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:39:30 crc kubenswrapper[5052]: I1203 12:39:30.361071 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/3039127c-0211-4430-9b2b-426412241ce4-config-data\") pod \"glance-default-internal-api-0\" (UID: \"3039127c-0211-4430-9b2b-426412241ce4\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:39:30 crc kubenswrapper[5052]: I1203 12:39:30.383400 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-ssfhj\" (UniqueName: \"kubernetes.io/projected/b0606156-f409-4cae-9be1-15d990f69955-kube-api-access-ssfhj\") pod \"glance-default-external-api-0\" (UID: \"b0606156-f409-4cae-9be1-15d990f69955\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:39:30 crc kubenswrapper[5052]: I1203 12:39:30.385233 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/b0606156-f409-4cae-9be1-15d990f69955-public-tls-certs\") pod \"glance-default-external-api-0\" (UID: \"b0606156-f409-4cae-9be1-15d990f69955\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:39:30 crc kubenswrapper[5052]: I1203 12:39:30.543003 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-mclw8\" (UniqueName: \"kubernetes.io/projected/3039127c-0211-4430-9b2b-426412241ce4-kube-api-access-mclw8\") pod \"glance-default-internal-api-0\" (UID: \"3039127c-0211-4430-9b2b-426412241ce4\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:39:30 crc kubenswrapper[5052]: I1203 12:39:30.583545 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage18-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage18-crc\") pod \"glance-default-internal-api-0\" (UID: \"3039127c-0211-4430-9b2b-426412241ce4\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:39:30 crc kubenswrapper[5052]: I1203 12:39:30.603094 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage12-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage12-crc\") pod \"glance-default-external-api-0\" (UID: \"b0606156-f409-4cae-9be1-15d990f69955\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:39:30 crc kubenswrapper[5052]: I1203 12:39:30.633041 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:39:30 crc kubenswrapper[5052]: I1203 12:39:30.676305 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:39:30 crc kubenswrapper[5052]: I1203 12:39:30.944472 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-7d74684fd-pqkf5" event={"ID":"ddfa4a3f-b868-49b7-abec-62cb3ffde2bd","Type":"ContainerStarted","Data":"8ed93a1c3257ab19479e68e87c18d74af55fe2f519944a6fbeee4da11170b261"} Dec 03 12:39:30 crc kubenswrapper[5052]: I1203 12:39:30.944928 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-7d74684fd-pqkf5" event={"ID":"ddfa4a3f-b868-49b7-abec-62cb3ffde2bd","Type":"ContainerStarted","Data":"ef78ab4daf6f2994bb54c1f8844ff1255eef71866c6e7be47ed5d4c7ef1d996d"} Dec 03 12:39:30 crc kubenswrapper[5052]: I1203 12:39:30.944952 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/neutron-7d74684fd-pqkf5" Dec 03 12:39:30 crc kubenswrapper[5052]: I1203 12:39:30.964794 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"95a06af5-270d-4138-ab79-fcdd239b3997","Type":"ContainerStarted","Data":"29357b3d97482fd8007dd5664e67838f357ddeab54a10504c300a6188e0d2e3d"} Dec 03 12:39:30 crc kubenswrapper[5052]: I1203 12:39:30.978452 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/neutron-7d74684fd-pqkf5" podStartSLOduration=2.978434514 podStartE2EDuration="2.978434514s" podCreationTimestamp="2025-12-03 12:39:28 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:39:30.974479431 +0000 UTC m=+4761.562721195" watchObservedRunningTime="2025-12-03 12:39:30.978434514 +0000 UTC m=+4761.566676278" Dec 03 12:39:31 crc kubenswrapper[5052]: W1203 12:39:31.262826 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3039127c_0211_4430_9b2b_426412241ce4.slice/crio-39c9c428d2f039ed7354568dafcb88fcc2e42dff18fcfb12022933e6e7c65a34 WatchSource:0}: Error finding container 39c9c428d2f039ed7354568dafcb88fcc2e42dff18fcfb12022933e6e7c65a34: Status 404 returned error can't find the container with id 39c9c428d2f039ed7354568dafcb88fcc2e42dff18fcfb12022933e6e7c65a34 Dec 03 12:39:31 crc kubenswrapper[5052]: I1203 12:39:31.263109 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:39:31 crc kubenswrapper[5052]: I1203 12:39:31.380611 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:39:31 crc kubenswrapper[5052]: W1203 12:39:31.397038 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb0606156_f409_4cae_9be1_15d990f69955.slice/crio-ecfd28d673e77d737ec3a816279736e50a6835ccc5c4a396f288e0cd654d3288 WatchSource:0}: Error finding container ecfd28d673e77d737ec3a816279736e50a6835ccc5c4a396f288e0cd654d3288: Status 404 returned error can't find the container with id ecfd28d673e77d737ec3a816279736e50a6835ccc5c4a396f288e0cd654d3288 Dec 03 12:39:31 crc kubenswrapper[5052]: I1203 12:39:31.730821 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="02726407-aa1e-400f-aedb-78d896c83113" path="/var/lib/kubelet/pods/02726407-aa1e-400f-aedb-78d896c83113/volumes" Dec 03 12:39:31 crc kubenswrapper[5052]: I1203 12:39:31.734716 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2972cf0c-5378-4e71-b308-0235608d5bef" path="/var/lib/kubelet/pods/2972cf0c-5378-4e71-b308-0235608d5bef/volumes" Dec 03 12:39:31 crc kubenswrapper[5052]: I1203 12:39:31.736073 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5d4a61d7-2d2c-4186-a923-f59636bc0994" path="/var/lib/kubelet/pods/5d4a61d7-2d2c-4186-a923-f59636bc0994/volumes" Dec 03 12:39:31 crc kubenswrapper[5052]: I1203 12:39:31.983789 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"b0606156-f409-4cae-9be1-15d990f69955","Type":"ContainerStarted","Data":"ecfd28d673e77d737ec3a816279736e50a6835ccc5c4a396f288e0cd654d3288"} Dec 03 12:39:31 crc kubenswrapper[5052]: I1203 12:39:31.988245 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"95a06af5-270d-4138-ab79-fcdd239b3997","Type":"ContainerStarted","Data":"0ef7cd09cfd4cf1e5792656dab2c9785fb2b5b3445c4f04f4ff27b9d93363509"} Dec 03 12:39:31 crc kubenswrapper[5052]: I1203 12:39:31.991997 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"3039127c-0211-4430-9b2b-426412241ce4","Type":"ContainerStarted","Data":"39c9c428d2f039ed7354568dafcb88fcc2e42dff18fcfb12022933e6e7c65a34"} Dec 03 12:39:32 crc kubenswrapper[5052]: I1203 12:39:32.351726 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:39:32 crc kubenswrapper[5052]: I1203 12:39:32.352338 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:39:33 crc kubenswrapper[5052]: I1203 12:39:33.011298 5052 generic.go:334] "Generic (PLEG): container finished" podID="c74fd61a-bf75-4a68-b802-cdc7f4b48be9" containerID="4233bebd5d63d6736a338fd2b32760cdc34fecfde4113939f42d9e6e5b01fa67" exitCode=0 Dec 03 12:39:33 crc kubenswrapper[5052]: I1203 12:39:33.011500 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"c74fd61a-bf75-4a68-b802-cdc7f4b48be9","Type":"ContainerDied","Data":"4233bebd5d63d6736a338fd2b32760cdc34fecfde4113939f42d9e6e5b01fa67"} Dec 03 12:39:33 crc kubenswrapper[5052]: I1203 12:39:33.020970 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"3039127c-0211-4430-9b2b-426412241ce4","Type":"ContainerStarted","Data":"1d3c75c1863b6df410b4948455d1e97a7c92fce681cbc217dafa1d03a2fae0c6"} Dec 03 12:39:33 crc kubenswrapper[5052]: I1203 12:39:33.021015 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"3039127c-0211-4430-9b2b-426412241ce4","Type":"ContainerStarted","Data":"101072c12291fb7e9e1ca717c6db562103892180f6281b3dd97442d2a61e51f8"} Dec 03 12:39:33 crc kubenswrapper[5052]: I1203 12:39:33.023756 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"b0606156-f409-4cae-9be1-15d990f69955","Type":"ContainerStarted","Data":"027b9a8307f751e8024d032edb748ef4b5da8f3d938c7c031bac5ed5e44f0c7d"} Dec 03 12:39:33 crc kubenswrapper[5052]: I1203 12:39:33.036540 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"95a06af5-270d-4138-ab79-fcdd239b3997","Type":"ContainerStarted","Data":"af9343e98edf473dab463b002e96eedd0ca172c95e67fa9e25832f1c73a3318a"} Dec 03 12:39:33 crc kubenswrapper[5052]: I1203 12:39:33.060426 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/glance-default-internal-api-0" podStartSLOduration=4.060404885 podStartE2EDuration="4.060404885s" podCreationTimestamp="2025-12-03 12:39:29 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:39:33.046074385 +0000 UTC m=+4763.634316149" watchObservedRunningTime="2025-12-03 12:39:33.060404885 +0000 UTC m=+4763.648646669" Dec 03 12:39:33 crc kubenswrapper[5052]: I1203 12:39:33.350851 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:39:33 crc kubenswrapper[5052]: I1203 12:39:33.376755 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/c74fd61a-bf75-4a68-b802-cdc7f4b48be9-config-data\") pod \"c74fd61a-bf75-4a68-b802-cdc7f4b48be9\" (UID: \"c74fd61a-bf75-4a68-b802-cdc7f4b48be9\") " Dec 03 12:39:33 crc kubenswrapper[5052]: I1203 12:39:33.376877 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c74fd61a-bf75-4a68-b802-cdc7f4b48be9-combined-ca-bundle\") pod \"c74fd61a-bf75-4a68-b802-cdc7f4b48be9\" (UID: \"c74fd61a-bf75-4a68-b802-cdc7f4b48be9\") " Dec 03 12:39:33 crc kubenswrapper[5052]: I1203 12:39:33.376995 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/c74fd61a-bf75-4a68-b802-cdc7f4b48be9-public-tls-certs\") pod \"c74fd61a-bf75-4a68-b802-cdc7f4b48be9\" (UID: \"c74fd61a-bf75-4a68-b802-cdc7f4b48be9\") " Dec 03 12:39:33 crc kubenswrapper[5052]: I1203 12:39:33.377052 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/c74fd61a-bf75-4a68-b802-cdc7f4b48be9-internal-tls-certs\") pod \"c74fd61a-bf75-4a68-b802-cdc7f4b48be9\" (UID: \"c74fd61a-bf75-4a68-b802-cdc7f4b48be9\") " Dec 03 12:39:33 crc kubenswrapper[5052]: I1203 12:39:33.378066 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/c74fd61a-bf75-4a68-b802-cdc7f4b48be9-logs\") pod \"c74fd61a-bf75-4a68-b802-cdc7f4b48be9\" (UID: \"c74fd61a-bf75-4a68-b802-cdc7f4b48be9\") " Dec 03 12:39:33 crc kubenswrapper[5052]: I1203 12:39:33.378133 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v9t9t\" (UniqueName: \"kubernetes.io/projected/c74fd61a-bf75-4a68-b802-cdc7f4b48be9-kube-api-access-v9t9t\") pod \"c74fd61a-bf75-4a68-b802-cdc7f4b48be9\" (UID: \"c74fd61a-bf75-4a68-b802-cdc7f4b48be9\") " Dec 03 12:39:33 crc kubenswrapper[5052]: I1203 12:39:33.378658 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c74fd61a-bf75-4a68-b802-cdc7f4b48be9-logs" (OuterVolumeSpecName: "logs") pod "c74fd61a-bf75-4a68-b802-cdc7f4b48be9" (UID: "c74fd61a-bf75-4a68-b802-cdc7f4b48be9"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:39:33 crc kubenswrapper[5052]: I1203 12:39:33.388417 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c74fd61a-bf75-4a68-b802-cdc7f4b48be9-kube-api-access-v9t9t" (OuterVolumeSpecName: "kube-api-access-v9t9t") pod "c74fd61a-bf75-4a68-b802-cdc7f4b48be9" (UID: "c74fd61a-bf75-4a68-b802-cdc7f4b48be9"). InnerVolumeSpecName "kube-api-access-v9t9t". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:39:33 crc kubenswrapper[5052]: I1203 12:39:33.478965 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c74fd61a-bf75-4a68-b802-cdc7f4b48be9-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "c74fd61a-bf75-4a68-b802-cdc7f4b48be9" (UID: "c74fd61a-bf75-4a68-b802-cdc7f4b48be9"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:39:33 crc kubenswrapper[5052]: I1203 12:39:33.480992 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/c74fd61a-bf75-4a68-b802-cdc7f4b48be9-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:39:33 crc kubenswrapper[5052]: I1203 12:39:33.481020 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-v9t9t\" (UniqueName: \"kubernetes.io/projected/c74fd61a-bf75-4a68-b802-cdc7f4b48be9-kube-api-access-v9t9t\") on node \"crc\" DevicePath \"\"" Dec 03 12:39:33 crc kubenswrapper[5052]: I1203 12:39:33.481034 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c74fd61a-bf75-4a68-b802-cdc7f4b48be9-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:39:33 crc kubenswrapper[5052]: I1203 12:39:33.512858 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 12:39:33 crc kubenswrapper[5052]: I1203 12:39:33.513079 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" podUID="20e672a5-6f07-415f-bf4d-51ef7bf20a0d" containerName="nova-cell1-novncproxy-novncproxy" containerID="cri-o://a2e11648a96c675347a628bf0acd6dbbc0ea6280cad79e45c8b977540157aa57" gracePeriod=30 Dec 03 12:39:33 crc kubenswrapper[5052]: I1203 12:39:33.537750 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c74fd61a-bf75-4a68-b802-cdc7f4b48be9-config-data" (OuterVolumeSpecName: "config-data") pod "c74fd61a-bf75-4a68-b802-cdc7f4b48be9" (UID: "c74fd61a-bf75-4a68-b802-cdc7f4b48be9"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:39:33 crc kubenswrapper[5052]: I1203 12:39:33.586169 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/c74fd61a-bf75-4a68-b802-cdc7f4b48be9-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:39:33 crc kubenswrapper[5052]: I1203 12:39:33.604702 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c74fd61a-bf75-4a68-b802-cdc7f4b48be9-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "c74fd61a-bf75-4a68-b802-cdc7f4b48be9" (UID: "c74fd61a-bf75-4a68-b802-cdc7f4b48be9"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:39:33 crc kubenswrapper[5052]: I1203 12:39:33.632778 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c74fd61a-bf75-4a68-b802-cdc7f4b48be9-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "c74fd61a-bf75-4a68-b802-cdc7f4b48be9" (UID: "c74fd61a-bf75-4a68-b802-cdc7f4b48be9"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:39:33 crc kubenswrapper[5052]: I1203 12:39:33.689127 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/c74fd61a-bf75-4a68-b802-cdc7f4b48be9-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:39:33 crc kubenswrapper[5052]: I1203 12:39:33.689445 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/c74fd61a-bf75-4a68-b802-cdc7f4b48be9-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:39:34 crc kubenswrapper[5052]: I1203 12:39:34.066593 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"c74fd61a-bf75-4a68-b802-cdc7f4b48be9","Type":"ContainerDied","Data":"4f43d5ebe24ab8b588b14aa860f7d67bd1fcbcefa5469616bcdd5d550ee70012"} Dec 03 12:39:34 crc kubenswrapper[5052]: I1203 12:39:34.066676 5052 scope.go:117] "RemoveContainer" containerID="4233bebd5d63d6736a338fd2b32760cdc34fecfde4113939f42d9e6e5b01fa67" Dec 03 12:39:34 crc kubenswrapper[5052]: I1203 12:39:34.066934 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:39:34 crc kubenswrapper[5052]: I1203 12:39:34.093729 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"b0606156-f409-4cae-9be1-15d990f69955","Type":"ContainerStarted","Data":"8ca42e62aac09b81598563bdb887c5ea6090758cf976adc8ad4b4e387ec44287"} Dec 03 12:39:34 crc kubenswrapper[5052]: I1203 12:39:34.106498 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:39:34 crc kubenswrapper[5052]: I1203 12:39:34.150832 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:39:34 crc kubenswrapper[5052]: I1203 12:39:34.169737 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:39:34 crc kubenswrapper[5052]: E1203 12:39:34.170277 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="c74fd61a-bf75-4a68-b802-cdc7f4b48be9" containerName="nova-api-log" Dec 03 12:39:34 crc kubenswrapper[5052]: I1203 12:39:34.170302 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="c74fd61a-bf75-4a68-b802-cdc7f4b48be9" containerName="nova-api-log" Dec 03 12:39:34 crc kubenswrapper[5052]: E1203 12:39:34.170331 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="c74fd61a-bf75-4a68-b802-cdc7f4b48be9" containerName="nova-api-api" Dec 03 12:39:34 crc kubenswrapper[5052]: I1203 12:39:34.170338 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="c74fd61a-bf75-4a68-b802-cdc7f4b48be9" containerName="nova-api-api" Dec 03 12:39:34 crc kubenswrapper[5052]: I1203 12:39:34.170655 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="c74fd61a-bf75-4a68-b802-cdc7f4b48be9" containerName="nova-api-log" Dec 03 12:39:34 crc kubenswrapper[5052]: I1203 12:39:34.170704 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="c74fd61a-bf75-4a68-b802-cdc7f4b48be9" containerName="nova-api-api" Dec 03 12:39:34 crc kubenswrapper[5052]: I1203 12:39:34.173098 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:39:34 crc kubenswrapper[5052]: I1203 12:39:34.177396 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-api-config-data" Dec 03 12:39:34 crc kubenswrapper[5052]: I1203 12:39:34.177873 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-nova-internal-svc" Dec 03 12:39:34 crc kubenswrapper[5052]: I1203 12:39:34.178693 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-nova-public-svc" Dec 03 12:39:34 crc kubenswrapper[5052]: I1203 12:39:34.197457 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/glance-default-external-api-0" podStartSLOduration=5.197432071 podStartE2EDuration="5.197432071s" podCreationTimestamp="2025-12-03 12:39:29 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:39:34.117845045 +0000 UTC m=+4764.706086809" watchObservedRunningTime="2025-12-03 12:39:34.197432071 +0000 UTC m=+4764.785673835" Dec 03 12:39:34 crc kubenswrapper[5052]: I1203 12:39:34.218876 5052 scope.go:117] "RemoveContainer" containerID="f74dbe94a7303ed5a5e53c969e42792ff7d1835379c81468a7fb60a94a0d4124" Dec 03 12:39:34 crc kubenswrapper[5052]: I1203 12:39:34.224383 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:39:34 crc kubenswrapper[5052]: E1203 12:39:34.272889 5052 cadvisor_stats_provider.go:516] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod20e672a5_6f07_415f_bf4d_51ef7bf20a0d.slice/crio-conmon-a2e11648a96c675347a628bf0acd6dbbc0ea6280cad79e45c8b977540157aa57.scope\": RecentStats: unable to find data in memory cache]" Dec 03 12:39:34 crc kubenswrapper[5052]: I1203 12:39:34.304795 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v8rm4\" (UniqueName: \"kubernetes.io/projected/50789f13-8194-4e24-b975-512591b816a5-kube-api-access-v8rm4\") pod \"nova-api-0\" (UID: \"50789f13-8194-4e24-b975-512591b816a5\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:39:34 crc kubenswrapper[5052]: I1203 12:39:34.305312 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/50789f13-8194-4e24-b975-512591b816a5-public-tls-certs\") pod \"nova-api-0\" (UID: \"50789f13-8194-4e24-b975-512591b816a5\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:39:34 crc kubenswrapper[5052]: I1203 12:39:34.313152 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/50789f13-8194-4e24-b975-512591b816a5-config-data\") pod \"nova-api-0\" (UID: \"50789f13-8194-4e24-b975-512591b816a5\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:39:34 crc kubenswrapper[5052]: I1203 12:39:34.313233 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/50789f13-8194-4e24-b975-512591b816a5-logs\") pod \"nova-api-0\" (UID: \"50789f13-8194-4e24-b975-512591b816a5\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:39:34 crc kubenswrapper[5052]: I1203 12:39:34.313383 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/50789f13-8194-4e24-b975-512591b816a5-internal-tls-certs\") pod \"nova-api-0\" (UID: \"50789f13-8194-4e24-b975-512591b816a5\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:39:34 crc kubenswrapper[5052]: I1203 12:39:34.313658 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/50789f13-8194-4e24-b975-512591b816a5-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"50789f13-8194-4e24-b975-512591b816a5\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:39:34 crc kubenswrapper[5052]: I1203 12:39:34.394882 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:39:34 crc kubenswrapper[5052]: I1203 12:39:34.416075 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/50789f13-8194-4e24-b975-512591b816a5-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"50789f13-8194-4e24-b975-512591b816a5\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:39:34 crc kubenswrapper[5052]: I1203 12:39:34.416299 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-v8rm4\" (UniqueName: \"kubernetes.io/projected/50789f13-8194-4e24-b975-512591b816a5-kube-api-access-v8rm4\") pod \"nova-api-0\" (UID: \"50789f13-8194-4e24-b975-512591b816a5\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:39:34 crc kubenswrapper[5052]: I1203 12:39:34.416401 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/50789f13-8194-4e24-b975-512591b816a5-public-tls-certs\") pod \"nova-api-0\" (UID: \"50789f13-8194-4e24-b975-512591b816a5\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:39:34 crc kubenswrapper[5052]: I1203 12:39:34.416446 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/50789f13-8194-4e24-b975-512591b816a5-config-data\") pod \"nova-api-0\" (UID: \"50789f13-8194-4e24-b975-512591b816a5\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:39:34 crc kubenswrapper[5052]: I1203 12:39:34.416489 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/50789f13-8194-4e24-b975-512591b816a5-logs\") pod \"nova-api-0\" (UID: \"50789f13-8194-4e24-b975-512591b816a5\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:39:34 crc kubenswrapper[5052]: I1203 12:39:34.416574 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/50789f13-8194-4e24-b975-512591b816a5-internal-tls-certs\") pod \"nova-api-0\" (UID: \"50789f13-8194-4e24-b975-512591b816a5\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:39:34 crc kubenswrapper[5052]: I1203 12:39:34.427580 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/50789f13-8194-4e24-b975-512591b816a5-logs\") pod \"nova-api-0\" (UID: \"50789f13-8194-4e24-b975-512591b816a5\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:39:34 crc kubenswrapper[5052]: I1203 12:39:34.430946 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/50789f13-8194-4e24-b975-512591b816a5-config-data\") pod \"nova-api-0\" (UID: \"50789f13-8194-4e24-b975-512591b816a5\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:39:34 crc kubenswrapper[5052]: I1203 12:39:34.434624 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/50789f13-8194-4e24-b975-512591b816a5-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"50789f13-8194-4e24-b975-512591b816a5\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:39:34 crc kubenswrapper[5052]: I1203 12:39:34.436002 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/50789f13-8194-4e24-b975-512591b816a5-internal-tls-certs\") pod \"nova-api-0\" (UID: \"50789f13-8194-4e24-b975-512591b816a5\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:39:34 crc kubenswrapper[5052]: I1203 12:39:34.439390 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/50789f13-8194-4e24-b975-512591b816a5-public-tls-certs\") pod \"nova-api-0\" (UID: \"50789f13-8194-4e24-b975-512591b816a5\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:39:34 crc kubenswrapper[5052]: I1203 12:39:34.448339 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-v8rm4\" (UniqueName: \"kubernetes.io/projected/50789f13-8194-4e24-b975-512591b816a5-kube-api-access-v8rm4\") pod \"nova-api-0\" (UID: \"50789f13-8194-4e24-b975-512591b816a5\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:39:34 crc kubenswrapper[5052]: I1203 12:39:34.509046 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:39:34 crc kubenswrapper[5052]: I1203 12:39:34.762217 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:39:34 crc kubenswrapper[5052]: I1203 12:39:34.835247 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"vencrypt-tls-certs\" (UniqueName: \"kubernetes.io/secret/20e672a5-6f07-415f-bf4d-51ef7bf20a0d-vencrypt-tls-certs\") pod \"20e672a5-6f07-415f-bf4d-51ef7bf20a0d\" (UID: \"20e672a5-6f07-415f-bf4d-51ef7bf20a0d\") " Dec 03 12:39:34 crc kubenswrapper[5052]: I1203 12:39:34.836233 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/20e672a5-6f07-415f-bf4d-51ef7bf20a0d-config-data\") pod \"20e672a5-6f07-415f-bf4d-51ef7bf20a0d\" (UID: \"20e672a5-6f07-415f-bf4d-51ef7bf20a0d\") " Dec 03 12:39:34 crc kubenswrapper[5052]: I1203 12:39:34.836351 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/20e672a5-6f07-415f-bf4d-51ef7bf20a0d-combined-ca-bundle\") pod \"20e672a5-6f07-415f-bf4d-51ef7bf20a0d\" (UID: \"20e672a5-6f07-415f-bf4d-51ef7bf20a0d\") " Dec 03 12:39:34 crc kubenswrapper[5052]: I1203 12:39:34.836422 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p7vnw\" (UniqueName: \"kubernetes.io/projected/20e672a5-6f07-415f-bf4d-51ef7bf20a0d-kube-api-access-p7vnw\") pod \"20e672a5-6f07-415f-bf4d-51ef7bf20a0d\" (UID: \"20e672a5-6f07-415f-bf4d-51ef7bf20a0d\") " Dec 03 12:39:34 crc kubenswrapper[5052]: I1203 12:39:34.836650 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"nova-novncproxy-tls-certs\" (UniqueName: \"kubernetes.io/secret/20e672a5-6f07-415f-bf4d-51ef7bf20a0d-nova-novncproxy-tls-certs\") pod \"20e672a5-6f07-415f-bf4d-51ef7bf20a0d\" (UID: \"20e672a5-6f07-415f-bf4d-51ef7bf20a0d\") " Dec 03 12:39:34 crc kubenswrapper[5052]: I1203 12:39:34.854840 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/20e672a5-6f07-415f-bf4d-51ef7bf20a0d-kube-api-access-p7vnw" (OuterVolumeSpecName: "kube-api-access-p7vnw") pod "20e672a5-6f07-415f-bf4d-51ef7bf20a0d" (UID: "20e672a5-6f07-415f-bf4d-51ef7bf20a0d"). InnerVolumeSpecName "kube-api-access-p7vnw". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:39:34 crc kubenswrapper[5052]: I1203 12:39:34.896716 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/20e672a5-6f07-415f-bf4d-51ef7bf20a0d-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "20e672a5-6f07-415f-bf4d-51ef7bf20a0d" (UID: "20e672a5-6f07-415f-bf4d-51ef7bf20a0d"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:39:34 crc kubenswrapper[5052]: I1203 12:39:34.907714 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/20e672a5-6f07-415f-bf4d-51ef7bf20a0d-config-data" (OuterVolumeSpecName: "config-data") pod "20e672a5-6f07-415f-bf4d-51ef7bf20a0d" (UID: "20e672a5-6f07-415f-bf4d-51ef7bf20a0d"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:39:34 crc kubenswrapper[5052]: I1203 12:39:34.927672 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/20e672a5-6f07-415f-bf4d-51ef7bf20a0d-vencrypt-tls-certs" (OuterVolumeSpecName: "vencrypt-tls-certs") pod "20e672a5-6f07-415f-bf4d-51ef7bf20a0d" (UID: "20e672a5-6f07-415f-bf4d-51ef7bf20a0d"). InnerVolumeSpecName "vencrypt-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:39:34 crc kubenswrapper[5052]: W1203 12:39:34.935733 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod50789f13_8194_4e24_b975_512591b816a5.slice/crio-eb3e43a555b8207795889d1049026acc685fad2f90f58455ca120d8b00561c89 WatchSource:0}: Error finding container eb3e43a555b8207795889d1049026acc685fad2f90f58455ca120d8b00561c89: Status 404 returned error can't find the container with id eb3e43a555b8207795889d1049026acc685fad2f90f58455ca120d8b00561c89 Dec 03 12:39:34 crc kubenswrapper[5052]: I1203 12:39:34.939190 5052 reconciler_common.go:293] "Volume detached for volume \"vencrypt-tls-certs\" (UniqueName: \"kubernetes.io/secret/20e672a5-6f07-415f-bf4d-51ef7bf20a0d-vencrypt-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:39:34 crc kubenswrapper[5052]: I1203 12:39:34.939359 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/20e672a5-6f07-415f-bf4d-51ef7bf20a0d-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:39:34 crc kubenswrapper[5052]: I1203 12:39:34.939464 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/20e672a5-6f07-415f-bf4d-51ef7bf20a0d-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:39:34 crc kubenswrapper[5052]: I1203 12:39:34.939673 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-p7vnw\" (UniqueName: \"kubernetes.io/projected/20e672a5-6f07-415f-bf4d-51ef7bf20a0d-kube-api-access-p7vnw\") on node \"crc\" DevicePath \"\"" Dec 03 12:39:34 crc kubenswrapper[5052]: I1203 12:39:34.948954 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:39:34 crc kubenswrapper[5052]: I1203 12:39:34.973209 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/20e672a5-6f07-415f-bf4d-51ef7bf20a0d-nova-novncproxy-tls-certs" (OuterVolumeSpecName: "nova-novncproxy-tls-certs") pod "20e672a5-6f07-415f-bf4d-51ef7bf20a0d" (UID: "20e672a5-6f07-415f-bf4d-51ef7bf20a0d"). InnerVolumeSpecName "nova-novncproxy-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:39:35 crc kubenswrapper[5052]: I1203 12:39:35.041415 5052 reconciler_common.go:293] "Volume detached for volume \"nova-novncproxy-tls-certs\" (UniqueName: \"kubernetes.io/secret/20e672a5-6f07-415f-bf4d-51ef7bf20a0d-nova-novncproxy-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:39:35 crc kubenswrapper[5052]: I1203 12:39:35.106163 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"50789f13-8194-4e24-b975-512591b816a5","Type":"ContainerStarted","Data":"eb3e43a555b8207795889d1049026acc685fad2f90f58455ca120d8b00561c89"} Dec 03 12:39:35 crc kubenswrapper[5052]: I1203 12:39:35.108128 5052 generic.go:334] "Generic (PLEG): container finished" podID="20e672a5-6f07-415f-bf4d-51ef7bf20a0d" containerID="a2e11648a96c675347a628bf0acd6dbbc0ea6280cad79e45c8b977540157aa57" exitCode=0 Dec 03 12:39:35 crc kubenswrapper[5052]: I1203 12:39:35.108223 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:39:35 crc kubenswrapper[5052]: I1203 12:39:35.108346 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" event={"ID":"20e672a5-6f07-415f-bf4d-51ef7bf20a0d","Type":"ContainerDied","Data":"a2e11648a96c675347a628bf0acd6dbbc0ea6280cad79e45c8b977540157aa57"} Dec 03 12:39:35 crc kubenswrapper[5052]: I1203 12:39:35.108378 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" event={"ID":"20e672a5-6f07-415f-bf4d-51ef7bf20a0d","Type":"ContainerDied","Data":"706d339f4fe6f91d8e28ba1d3aee0ca842a3b0d84d2564ef0e7af37ed0750be3"} Dec 03 12:39:35 crc kubenswrapper[5052]: I1203 12:39:35.108418 5052 scope.go:117] "RemoveContainer" containerID="a2e11648a96c675347a628bf0acd6dbbc0ea6280cad79e45c8b977540157aa57" Dec 03 12:39:35 crc kubenswrapper[5052]: I1203 12:39:35.118179 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"95a06af5-270d-4138-ab79-fcdd239b3997","Type":"ContainerStarted","Data":"9c894be2b5ea25d2e1dc93642f17b355a9f5eb4d38037b4b3d4de06025012260"} Dec 03 12:39:35 crc kubenswrapper[5052]: I1203 12:39:35.118240 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:39:35 crc kubenswrapper[5052]: I1203 12:39:35.162934 5052 scope.go:117] "RemoveContainer" containerID="a2e11648a96c675347a628bf0acd6dbbc0ea6280cad79e45c8b977540157aa57" Dec 03 12:39:35 crc kubenswrapper[5052]: E1203 12:39:35.163574 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a2e11648a96c675347a628bf0acd6dbbc0ea6280cad79e45c8b977540157aa57\": container with ID starting with a2e11648a96c675347a628bf0acd6dbbc0ea6280cad79e45c8b977540157aa57 not found: ID does not exist" containerID="a2e11648a96c675347a628bf0acd6dbbc0ea6280cad79e45c8b977540157aa57" Dec 03 12:39:35 crc kubenswrapper[5052]: I1203 12:39:35.163615 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a2e11648a96c675347a628bf0acd6dbbc0ea6280cad79e45c8b977540157aa57"} err="failed to get container status \"a2e11648a96c675347a628bf0acd6dbbc0ea6280cad79e45c8b977540157aa57\": rpc error: code = NotFound desc = could not find container \"a2e11648a96c675347a628bf0acd6dbbc0ea6280cad79e45c8b977540157aa57\": container with ID starting with a2e11648a96c675347a628bf0acd6dbbc0ea6280cad79e45c8b977540157aa57 not found: ID does not exist" Dec 03 12:39:35 crc kubenswrapper[5052]: I1203 12:39:35.172986 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/ceilometer-0" podStartSLOduration=2.231206125 podStartE2EDuration="7.17295954s" podCreationTimestamp="2025-12-03 12:39:28 +0000 UTC" firstStartedPulling="2025-12-03 12:39:29.488366391 +0000 UTC m=+4760.076608155" lastFinishedPulling="2025-12-03 12:39:34.430119806 +0000 UTC m=+4765.018361570" observedRunningTime="2025-12-03 12:39:35.13693777 +0000 UTC m=+4765.725179534" watchObservedRunningTime="2025-12-03 12:39:35.17295954 +0000 UTC m=+4765.761201314" Dec 03 12:39:35 crc kubenswrapper[5052]: I1203 12:39:35.207742 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 12:39:35 crc kubenswrapper[5052]: I1203 12:39:35.217637 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 12:39:35 crc kubenswrapper[5052]: I1203 12:39:35.228007 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 12:39:35 crc kubenswrapper[5052]: E1203 12:39:35.228647 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="20e672a5-6f07-415f-bf4d-51ef7bf20a0d" containerName="nova-cell1-novncproxy-novncproxy" Dec 03 12:39:35 crc kubenswrapper[5052]: I1203 12:39:35.228671 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="20e672a5-6f07-415f-bf4d-51ef7bf20a0d" containerName="nova-cell1-novncproxy-novncproxy" Dec 03 12:39:35 crc kubenswrapper[5052]: I1203 12:39:35.228912 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="20e672a5-6f07-415f-bf4d-51ef7bf20a0d" containerName="nova-cell1-novncproxy-novncproxy" Dec 03 12:39:35 crc kubenswrapper[5052]: I1203 12:39:35.229759 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:39:35 crc kubenswrapper[5052]: I1203 12:39:35.232921 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell1-novncproxy-config-data" Dec 03 12:39:35 crc kubenswrapper[5052]: I1203 12:39:35.233093 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-nova-novncproxy-cell1-vencrypt" Dec 03 12:39:35 crc kubenswrapper[5052]: I1203 12:39:35.235401 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-nova-novncproxy-cell1-public-svc" Dec 03 12:39:35 crc kubenswrapper[5052]: I1203 12:39:35.245377 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 12:39:35 crc kubenswrapper[5052]: I1203 12:39:35.351434 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nova-novncproxy-tls-certs\" (UniqueName: \"kubernetes.io/secret/0f16f521-a181-40e4-a1ed-38f4c8862ffb-nova-novncproxy-tls-certs\") pod \"nova-cell1-novncproxy-0\" (UID: \"0f16f521-a181-40e4-a1ed-38f4c8862ffb\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:39:35 crc kubenswrapper[5052]: I1203 12:39:35.351565 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"vencrypt-tls-certs\" (UniqueName: \"kubernetes.io/secret/0f16f521-a181-40e4-a1ed-38f4c8862ffb-vencrypt-tls-certs\") pod \"nova-cell1-novncproxy-0\" (UID: \"0f16f521-a181-40e4-a1ed-38f4c8862ffb\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:39:35 crc kubenswrapper[5052]: I1203 12:39:35.351628 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0f16f521-a181-40e4-a1ed-38f4c8862ffb-combined-ca-bundle\") pod \"nova-cell1-novncproxy-0\" (UID: \"0f16f521-a181-40e4-a1ed-38f4c8862ffb\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:39:35 crc kubenswrapper[5052]: I1203 12:39:35.351660 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-67l8z\" (UniqueName: \"kubernetes.io/projected/0f16f521-a181-40e4-a1ed-38f4c8862ffb-kube-api-access-67l8z\") pod \"nova-cell1-novncproxy-0\" (UID: \"0f16f521-a181-40e4-a1ed-38f4c8862ffb\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:39:35 crc kubenswrapper[5052]: I1203 12:39:35.351685 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/0f16f521-a181-40e4-a1ed-38f4c8862ffb-config-data\") pod \"nova-cell1-novncproxy-0\" (UID: \"0f16f521-a181-40e4-a1ed-38f4c8862ffb\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:39:35 crc kubenswrapper[5052]: I1203 12:39:35.454000 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0f16f521-a181-40e4-a1ed-38f4c8862ffb-combined-ca-bundle\") pod \"nova-cell1-novncproxy-0\" (UID: \"0f16f521-a181-40e4-a1ed-38f4c8862ffb\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:39:35 crc kubenswrapper[5052]: I1203 12:39:35.454070 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-67l8z\" (UniqueName: \"kubernetes.io/projected/0f16f521-a181-40e4-a1ed-38f4c8862ffb-kube-api-access-67l8z\") pod \"nova-cell1-novncproxy-0\" (UID: \"0f16f521-a181-40e4-a1ed-38f4c8862ffb\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:39:35 crc kubenswrapper[5052]: I1203 12:39:35.454110 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/0f16f521-a181-40e4-a1ed-38f4c8862ffb-config-data\") pod \"nova-cell1-novncproxy-0\" (UID: \"0f16f521-a181-40e4-a1ed-38f4c8862ffb\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:39:35 crc kubenswrapper[5052]: I1203 12:39:35.454280 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"nova-novncproxy-tls-certs\" (UniqueName: \"kubernetes.io/secret/0f16f521-a181-40e4-a1ed-38f4c8862ffb-nova-novncproxy-tls-certs\") pod \"nova-cell1-novncproxy-0\" (UID: \"0f16f521-a181-40e4-a1ed-38f4c8862ffb\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:39:35 crc kubenswrapper[5052]: I1203 12:39:35.454384 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"vencrypt-tls-certs\" (UniqueName: \"kubernetes.io/secret/0f16f521-a181-40e4-a1ed-38f4c8862ffb-vencrypt-tls-certs\") pod \"nova-cell1-novncproxy-0\" (UID: \"0f16f521-a181-40e4-a1ed-38f4c8862ffb\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:39:35 crc kubenswrapper[5052]: I1203 12:39:35.459654 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/0f16f521-a181-40e4-a1ed-38f4c8862ffb-config-data\") pod \"nova-cell1-novncproxy-0\" (UID: \"0f16f521-a181-40e4-a1ed-38f4c8862ffb\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:39:35 crc kubenswrapper[5052]: I1203 12:39:35.460219 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"nova-novncproxy-tls-certs\" (UniqueName: \"kubernetes.io/secret/0f16f521-a181-40e4-a1ed-38f4c8862ffb-nova-novncproxy-tls-certs\") pod \"nova-cell1-novncproxy-0\" (UID: \"0f16f521-a181-40e4-a1ed-38f4c8862ffb\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:39:35 crc kubenswrapper[5052]: I1203 12:39:35.462183 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0f16f521-a181-40e4-a1ed-38f4c8862ffb-combined-ca-bundle\") pod \"nova-cell1-novncproxy-0\" (UID: \"0f16f521-a181-40e4-a1ed-38f4c8862ffb\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:39:35 crc kubenswrapper[5052]: I1203 12:39:35.465035 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"vencrypt-tls-certs\" (UniqueName: \"kubernetes.io/secret/0f16f521-a181-40e4-a1ed-38f4c8862ffb-vencrypt-tls-certs\") pod \"nova-cell1-novncproxy-0\" (UID: \"0f16f521-a181-40e4-a1ed-38f4c8862ffb\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:39:35 crc kubenswrapper[5052]: I1203 12:39:35.477159 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-67l8z\" (UniqueName: \"kubernetes.io/projected/0f16f521-a181-40e4-a1ed-38f4c8862ffb-kube-api-access-67l8z\") pod \"nova-cell1-novncproxy-0\" (UID: \"0f16f521-a181-40e4-a1ed-38f4c8862ffb\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:39:35 crc kubenswrapper[5052]: I1203 12:39:35.560543 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:39:35 crc kubenswrapper[5052]: I1203 12:39:35.708115 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="20e672a5-6f07-415f-bf4d-51ef7bf20a0d" path="/var/lib/kubelet/pods/20e672a5-6f07-415f-bf4d-51ef7bf20a0d/volumes" Dec 03 12:39:35 crc kubenswrapper[5052]: I1203 12:39:35.708794 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c74fd61a-bf75-4a68-b802-cdc7f4b48be9" path="/var/lib/kubelet/pods/c74fd61a-bf75-4a68-b802-cdc7f4b48be9/volumes" Dec 03 12:39:36 crc kubenswrapper[5052]: I1203 12:39:36.105639 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 12:39:36 crc kubenswrapper[5052]: I1203 12:39:36.134476 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"50789f13-8194-4e24-b975-512591b816a5","Type":"ContainerStarted","Data":"256ac84e475433107bd853223013a808d99354e60928d4ce86ddc71170d1b9a7"} Dec 03 12:39:36 crc kubenswrapper[5052]: I1203 12:39:36.134603 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"50789f13-8194-4e24-b975-512591b816a5","Type":"ContainerStarted","Data":"f1298bb0944b7faf489bfe13489509679a09bf41f1f58deeacfdf2aae7d628e8"} Dec 03 12:39:36 crc kubenswrapper[5052]: I1203 12:39:36.149847 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" event={"ID":"0f16f521-a181-40e4-a1ed-38f4c8862ffb","Type":"ContainerStarted","Data":"10026819c94b63b87ca2799cb11896626abae2617817d72431026729a7b3c240"} Dec 03 12:39:36 crc kubenswrapper[5052]: I1203 12:39:36.157922 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-api-0" podStartSLOduration=2.157905888 podStartE2EDuration="2.157905888s" podCreationTimestamp="2025-12-03 12:39:34 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:39:36.1565934 +0000 UTC m=+4766.744835174" watchObservedRunningTime="2025-12-03 12:39:36.157905888 +0000 UTC m=+4766.746147652" Dec 03 12:39:36 crc kubenswrapper[5052]: I1203 12:39:36.343960 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/placement-75785bcbbb-wlvdv"] Dec 03 12:39:36 crc kubenswrapper[5052]: I1203 12:39:36.344674 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/placement-75785bcbbb-wlvdv" podUID="068f200e-7f4b-4cc8-9757-615f13e58d7d" containerName="placement-log" containerID="cri-o://1e204f6c3414f741bf890709ea5f0d5b7479276967cb8ad863ae2e8578aaadfd" gracePeriod=30 Dec 03 12:39:36 crc kubenswrapper[5052]: I1203 12:39:36.345489 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/placement-75785bcbbb-wlvdv" podUID="068f200e-7f4b-4cc8-9757-615f13e58d7d" containerName="placement-api" containerID="cri-o://9acdd30c66dec3d20b4dce9dc97671fbb203abed74b4539b05e311efaa1ae060" gracePeriod=30 Dec 03 12:39:36 crc kubenswrapper[5052]: I1203 12:39:36.430524 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/placement-cdf4456f4-v82rz"] Dec 03 12:39:36 crc kubenswrapper[5052]: I1203 12:39:36.432159 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-cdf4456f4-v82rz" Dec 03 12:39:36 crc kubenswrapper[5052]: I1203 12:39:36.446902 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/placement-cdf4456f4-v82rz"] Dec 03 12:39:36 crc kubenswrapper[5052]: I1203 12:39:36.478295 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/99a0961b-8ed8-49b7-abac-5025d7b351ee-combined-ca-bundle\") pod \"placement-cdf4456f4-v82rz\" (UID: \"99a0961b-8ed8-49b7-abac-5025d7b351ee\") " pod="openstack-kuttl-tests/placement-cdf4456f4-v82rz" Dec 03 12:39:36 crc kubenswrapper[5052]: I1203 12:39:36.478359 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/99a0961b-8ed8-49b7-abac-5025d7b351ee-logs\") pod \"placement-cdf4456f4-v82rz\" (UID: \"99a0961b-8ed8-49b7-abac-5025d7b351ee\") " pod="openstack-kuttl-tests/placement-cdf4456f4-v82rz" Dec 03 12:39:36 crc kubenswrapper[5052]: I1203 12:39:36.478433 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/99a0961b-8ed8-49b7-abac-5025d7b351ee-public-tls-certs\") pod \"placement-cdf4456f4-v82rz\" (UID: \"99a0961b-8ed8-49b7-abac-5025d7b351ee\") " pod="openstack-kuttl-tests/placement-cdf4456f4-v82rz" Dec 03 12:39:36 crc kubenswrapper[5052]: I1203 12:39:36.478482 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/99a0961b-8ed8-49b7-abac-5025d7b351ee-scripts\") pod \"placement-cdf4456f4-v82rz\" (UID: \"99a0961b-8ed8-49b7-abac-5025d7b351ee\") " pod="openstack-kuttl-tests/placement-cdf4456f4-v82rz" Dec 03 12:39:36 crc kubenswrapper[5052]: I1203 12:39:36.478553 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ljq7n\" (UniqueName: \"kubernetes.io/projected/99a0961b-8ed8-49b7-abac-5025d7b351ee-kube-api-access-ljq7n\") pod \"placement-cdf4456f4-v82rz\" (UID: \"99a0961b-8ed8-49b7-abac-5025d7b351ee\") " pod="openstack-kuttl-tests/placement-cdf4456f4-v82rz" Dec 03 12:39:36 crc kubenswrapper[5052]: I1203 12:39:36.478603 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/99a0961b-8ed8-49b7-abac-5025d7b351ee-internal-tls-certs\") pod \"placement-cdf4456f4-v82rz\" (UID: \"99a0961b-8ed8-49b7-abac-5025d7b351ee\") " pod="openstack-kuttl-tests/placement-cdf4456f4-v82rz" Dec 03 12:39:36 crc kubenswrapper[5052]: I1203 12:39:36.478681 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/99a0961b-8ed8-49b7-abac-5025d7b351ee-config-data\") pod \"placement-cdf4456f4-v82rz\" (UID: \"99a0961b-8ed8-49b7-abac-5025d7b351ee\") " pod="openstack-kuttl-tests/placement-cdf4456f4-v82rz" Dec 03 12:39:36 crc kubenswrapper[5052]: I1203 12:39:36.513104 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/placement-75785bcbbb-wlvdv" podUID="068f200e-7f4b-4cc8-9757-615f13e58d7d" containerName="placement-log" probeResult="failure" output="Get \"https://10.217.1.118:8778/\": read tcp 10.217.0.2:43322->10.217.1.118:8778: read: connection reset by peer" Dec 03 12:39:36 crc kubenswrapper[5052]: I1203 12:39:36.513616 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/placement-75785bcbbb-wlvdv" podUID="068f200e-7f4b-4cc8-9757-615f13e58d7d" containerName="placement-api" probeResult="failure" output="Get \"https://10.217.1.118:8778/\": read tcp 10.217.0.2:43316->10.217.1.118:8778: read: connection reset by peer" Dec 03 12:39:36 crc kubenswrapper[5052]: I1203 12:39:36.580083 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/99a0961b-8ed8-49b7-abac-5025d7b351ee-public-tls-certs\") pod \"placement-cdf4456f4-v82rz\" (UID: \"99a0961b-8ed8-49b7-abac-5025d7b351ee\") " pod="openstack-kuttl-tests/placement-cdf4456f4-v82rz" Dec 03 12:39:36 crc kubenswrapper[5052]: I1203 12:39:36.580174 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/99a0961b-8ed8-49b7-abac-5025d7b351ee-scripts\") pod \"placement-cdf4456f4-v82rz\" (UID: \"99a0961b-8ed8-49b7-abac-5025d7b351ee\") " pod="openstack-kuttl-tests/placement-cdf4456f4-v82rz" Dec 03 12:39:36 crc kubenswrapper[5052]: I1203 12:39:36.580210 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-ljq7n\" (UniqueName: \"kubernetes.io/projected/99a0961b-8ed8-49b7-abac-5025d7b351ee-kube-api-access-ljq7n\") pod \"placement-cdf4456f4-v82rz\" (UID: \"99a0961b-8ed8-49b7-abac-5025d7b351ee\") " pod="openstack-kuttl-tests/placement-cdf4456f4-v82rz" Dec 03 12:39:36 crc kubenswrapper[5052]: I1203 12:39:36.580233 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/99a0961b-8ed8-49b7-abac-5025d7b351ee-internal-tls-certs\") pod \"placement-cdf4456f4-v82rz\" (UID: \"99a0961b-8ed8-49b7-abac-5025d7b351ee\") " pod="openstack-kuttl-tests/placement-cdf4456f4-v82rz" Dec 03 12:39:36 crc kubenswrapper[5052]: I1203 12:39:36.580289 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/99a0961b-8ed8-49b7-abac-5025d7b351ee-config-data\") pod \"placement-cdf4456f4-v82rz\" (UID: \"99a0961b-8ed8-49b7-abac-5025d7b351ee\") " pod="openstack-kuttl-tests/placement-cdf4456f4-v82rz" Dec 03 12:39:36 crc kubenswrapper[5052]: I1203 12:39:36.580337 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/99a0961b-8ed8-49b7-abac-5025d7b351ee-combined-ca-bundle\") pod \"placement-cdf4456f4-v82rz\" (UID: \"99a0961b-8ed8-49b7-abac-5025d7b351ee\") " pod="openstack-kuttl-tests/placement-cdf4456f4-v82rz" Dec 03 12:39:36 crc kubenswrapper[5052]: I1203 12:39:36.580356 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/99a0961b-8ed8-49b7-abac-5025d7b351ee-logs\") pod \"placement-cdf4456f4-v82rz\" (UID: \"99a0961b-8ed8-49b7-abac-5025d7b351ee\") " pod="openstack-kuttl-tests/placement-cdf4456f4-v82rz" Dec 03 12:39:36 crc kubenswrapper[5052]: I1203 12:39:36.580878 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/99a0961b-8ed8-49b7-abac-5025d7b351ee-logs\") pod \"placement-cdf4456f4-v82rz\" (UID: \"99a0961b-8ed8-49b7-abac-5025d7b351ee\") " pod="openstack-kuttl-tests/placement-cdf4456f4-v82rz" Dec 03 12:39:36 crc kubenswrapper[5052]: I1203 12:39:36.930284 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/99a0961b-8ed8-49b7-abac-5025d7b351ee-scripts\") pod \"placement-cdf4456f4-v82rz\" (UID: \"99a0961b-8ed8-49b7-abac-5025d7b351ee\") " pod="openstack-kuttl-tests/placement-cdf4456f4-v82rz" Dec 03 12:39:36 crc kubenswrapper[5052]: I1203 12:39:36.930476 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/99a0961b-8ed8-49b7-abac-5025d7b351ee-internal-tls-certs\") pod \"placement-cdf4456f4-v82rz\" (UID: \"99a0961b-8ed8-49b7-abac-5025d7b351ee\") " pod="openstack-kuttl-tests/placement-cdf4456f4-v82rz" Dec 03 12:39:36 crc kubenswrapper[5052]: I1203 12:39:36.931035 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/99a0961b-8ed8-49b7-abac-5025d7b351ee-combined-ca-bundle\") pod \"placement-cdf4456f4-v82rz\" (UID: \"99a0961b-8ed8-49b7-abac-5025d7b351ee\") " pod="openstack-kuttl-tests/placement-cdf4456f4-v82rz" Dec 03 12:39:36 crc kubenswrapper[5052]: I1203 12:39:36.931391 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/99a0961b-8ed8-49b7-abac-5025d7b351ee-public-tls-certs\") pod \"placement-cdf4456f4-v82rz\" (UID: \"99a0961b-8ed8-49b7-abac-5025d7b351ee\") " pod="openstack-kuttl-tests/placement-cdf4456f4-v82rz" Dec 03 12:39:36 crc kubenswrapper[5052]: I1203 12:39:36.932235 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/99a0961b-8ed8-49b7-abac-5025d7b351ee-config-data\") pod \"placement-cdf4456f4-v82rz\" (UID: \"99a0961b-8ed8-49b7-abac-5025d7b351ee\") " pod="openstack-kuttl-tests/placement-cdf4456f4-v82rz" Dec 03 12:39:36 crc kubenswrapper[5052]: I1203 12:39:36.932821 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-ljq7n\" (UniqueName: \"kubernetes.io/projected/99a0961b-8ed8-49b7-abac-5025d7b351ee-kube-api-access-ljq7n\") pod \"placement-cdf4456f4-v82rz\" (UID: \"99a0961b-8ed8-49b7-abac-5025d7b351ee\") " pod="openstack-kuttl-tests/placement-cdf4456f4-v82rz" Dec 03 12:39:37 crc kubenswrapper[5052]: I1203 12:39:37.181856 5052 generic.go:334] "Generic (PLEG): container finished" podID="068f200e-7f4b-4cc8-9757-615f13e58d7d" containerID="9acdd30c66dec3d20b4dce9dc97671fbb203abed74b4539b05e311efaa1ae060" exitCode=0 Dec 03 12:39:37 crc kubenswrapper[5052]: I1203 12:39:37.182103 5052 generic.go:334] "Generic (PLEG): container finished" podID="068f200e-7f4b-4cc8-9757-615f13e58d7d" containerID="1e204f6c3414f741bf890709ea5f0d5b7479276967cb8ad863ae2e8578aaadfd" exitCode=143 Dec 03 12:39:37 crc kubenswrapper[5052]: I1203 12:39:37.181954 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-75785bcbbb-wlvdv" event={"ID":"068f200e-7f4b-4cc8-9757-615f13e58d7d","Type":"ContainerDied","Data":"9acdd30c66dec3d20b4dce9dc97671fbb203abed74b4539b05e311efaa1ae060"} Dec 03 12:39:37 crc kubenswrapper[5052]: I1203 12:39:37.182758 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-75785bcbbb-wlvdv" event={"ID":"068f200e-7f4b-4cc8-9757-615f13e58d7d","Type":"ContainerDied","Data":"1e204f6c3414f741bf890709ea5f0d5b7479276967cb8ad863ae2e8578aaadfd"} Dec 03 12:39:37 crc kubenswrapper[5052]: I1203 12:39:37.351102 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:39:37 crc kubenswrapper[5052]: I1203 12:39:37.351143 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:39:37 crc kubenswrapper[5052]: I1203 12:39:37.354783 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-cdf4456f4-v82rz" Dec 03 12:39:37 crc kubenswrapper[5052]: I1203 12:39:37.936452 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/placement-cdf4456f4-v82rz"] Dec 03 12:39:38 crc kubenswrapper[5052]: I1203 12:39:38.184609 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-75785bcbbb-wlvdv" Dec 03 12:39:38 crc kubenswrapper[5052]: I1203 12:39:38.199161 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-75785bcbbb-wlvdv" event={"ID":"068f200e-7f4b-4cc8-9757-615f13e58d7d","Type":"ContainerDied","Data":"c20c64037906971409cb88cfbe54b367be87e1e42c1ff1db2fb1e32968c68d5e"} Dec 03 12:39:38 crc kubenswrapper[5052]: I1203 12:39:38.199201 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-75785bcbbb-wlvdv" Dec 03 12:39:38 crc kubenswrapper[5052]: I1203 12:39:38.199221 5052 scope.go:117] "RemoveContainer" containerID="9acdd30c66dec3d20b4dce9dc97671fbb203abed74b4539b05e311efaa1ae060" Dec 03 12:39:38 crc kubenswrapper[5052]: I1203 12:39:38.200577 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-cdf4456f4-v82rz" event={"ID":"99a0961b-8ed8-49b7-abac-5025d7b351ee","Type":"ContainerStarted","Data":"2feb5eaeb8e07d9d55d3220880f69471cec3c64e41d5b551d1b87b699609793c"} Dec 03 12:39:38 crc kubenswrapper[5052]: I1203 12:39:38.207227 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" event={"ID":"0f16f521-a181-40e4-a1ed-38f4c8862ffb","Type":"ContainerStarted","Data":"49e65401462a62ad6b62529bdc792c44fb0d7a07381a58a73215f64add3d28c5"} Dec 03 12:39:38 crc kubenswrapper[5052]: I1203 12:39:38.235281 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/068f200e-7f4b-4cc8-9757-615f13e58d7d-logs\") pod \"068f200e-7f4b-4cc8-9757-615f13e58d7d\" (UID: \"068f200e-7f4b-4cc8-9757-615f13e58d7d\") " Dec 03 12:39:38 crc kubenswrapper[5052]: I1203 12:39:38.235431 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/068f200e-7f4b-4cc8-9757-615f13e58d7d-combined-ca-bundle\") pod \"068f200e-7f4b-4cc8-9757-615f13e58d7d\" (UID: \"068f200e-7f4b-4cc8-9757-615f13e58d7d\") " Dec 03 12:39:38 crc kubenswrapper[5052]: I1203 12:39:38.235479 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/068f200e-7f4b-4cc8-9757-615f13e58d7d-public-tls-certs\") pod \"068f200e-7f4b-4cc8-9757-615f13e58d7d\" (UID: \"068f200e-7f4b-4cc8-9757-615f13e58d7d\") " Dec 03 12:39:38 crc kubenswrapper[5052]: I1203 12:39:38.235531 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jx7t9\" (UniqueName: \"kubernetes.io/projected/068f200e-7f4b-4cc8-9757-615f13e58d7d-kube-api-access-jx7t9\") pod \"068f200e-7f4b-4cc8-9757-615f13e58d7d\" (UID: \"068f200e-7f4b-4cc8-9757-615f13e58d7d\") " Dec 03 12:39:38 crc kubenswrapper[5052]: I1203 12:39:38.235647 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/068f200e-7f4b-4cc8-9757-615f13e58d7d-internal-tls-certs\") pod \"068f200e-7f4b-4cc8-9757-615f13e58d7d\" (UID: \"068f200e-7f4b-4cc8-9757-615f13e58d7d\") " Dec 03 12:39:38 crc kubenswrapper[5052]: I1203 12:39:38.235722 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/068f200e-7f4b-4cc8-9757-615f13e58d7d-scripts\") pod \"068f200e-7f4b-4cc8-9757-615f13e58d7d\" (UID: \"068f200e-7f4b-4cc8-9757-615f13e58d7d\") " Dec 03 12:39:38 crc kubenswrapper[5052]: I1203 12:39:38.235744 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/068f200e-7f4b-4cc8-9757-615f13e58d7d-config-data\") pod \"068f200e-7f4b-4cc8-9757-615f13e58d7d\" (UID: \"068f200e-7f4b-4cc8-9757-615f13e58d7d\") " Dec 03 12:39:38 crc kubenswrapper[5052]: I1203 12:39:38.237870 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/barbican-api-bcdbf47bb-m2ctz" Dec 03 12:39:38 crc kubenswrapper[5052]: I1203 12:39:38.239310 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/068f200e-7f4b-4cc8-9757-615f13e58d7d-logs" (OuterVolumeSpecName: "logs") pod "068f200e-7f4b-4cc8-9757-615f13e58d7d" (UID: "068f200e-7f4b-4cc8-9757-615f13e58d7d"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:39:38 crc kubenswrapper[5052]: I1203 12:39:38.246796 5052 scope.go:117] "RemoveContainer" containerID="1e204f6c3414f741bf890709ea5f0d5b7479276967cb8ad863ae2e8578aaadfd" Dec 03 12:39:38 crc kubenswrapper[5052]: I1203 12:39:38.261576 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/068f200e-7f4b-4cc8-9757-615f13e58d7d-scripts" (OuterVolumeSpecName: "scripts") pod "068f200e-7f4b-4cc8-9757-615f13e58d7d" (UID: "068f200e-7f4b-4cc8-9757-615f13e58d7d"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:39:38 crc kubenswrapper[5052]: I1203 12:39:38.262023 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/068f200e-7f4b-4cc8-9757-615f13e58d7d-kube-api-access-jx7t9" (OuterVolumeSpecName: "kube-api-access-jx7t9") pod "068f200e-7f4b-4cc8-9757-615f13e58d7d" (UID: "068f200e-7f4b-4cc8-9757-615f13e58d7d"). InnerVolumeSpecName "kube-api-access-jx7t9". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:39:38 crc kubenswrapper[5052]: I1203 12:39:38.297896 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" podStartSLOduration=3.297878347 podStartE2EDuration="3.297878347s" podCreationTimestamp="2025-12-03 12:39:35 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:39:38.296414385 +0000 UTC m=+4768.884656149" watchObservedRunningTime="2025-12-03 12:39:38.297878347 +0000 UTC m=+4768.886120111" Dec 03 12:39:38 crc kubenswrapper[5052]: I1203 12:39:38.337914 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/068f200e-7f4b-4cc8-9757-615f13e58d7d-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:39:38 crc kubenswrapper[5052]: I1203 12:39:38.343124 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/068f200e-7f4b-4cc8-9757-615f13e58d7d-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:39:38 crc kubenswrapper[5052]: I1203 12:39:38.343724 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-jx7t9\" (UniqueName: \"kubernetes.io/projected/068f200e-7f4b-4cc8-9757-615f13e58d7d-kube-api-access-jx7t9\") on node \"crc\" DevicePath \"\"" Dec 03 12:39:38 crc kubenswrapper[5052]: I1203 12:39:38.362925 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-metadata-0" podUID="30045a80-9e2b-416b-b7bf-cdb9585ec67d" containerName="nova-metadata-metadata" probeResult="failure" output="Get \"https://10.217.1.125:8775/\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Dec 03 12:39:38 crc kubenswrapper[5052]: I1203 12:39:38.363470 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-metadata-0" podUID="30045a80-9e2b-416b-b7bf-cdb9585ec67d" containerName="nova-metadata-log" probeResult="failure" output="Get \"https://10.217.1.125:8775/\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Dec 03 12:39:38 crc kubenswrapper[5052]: I1203 12:39:38.379100 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/068f200e-7f4b-4cc8-9757-615f13e58d7d-config-data" (OuterVolumeSpecName: "config-data") pod "068f200e-7f4b-4cc8-9757-615f13e58d7d" (UID: "068f200e-7f4b-4cc8-9757-615f13e58d7d"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:39:38 crc kubenswrapper[5052]: I1203 12:39:38.421873 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/068f200e-7f4b-4cc8-9757-615f13e58d7d-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "068f200e-7f4b-4cc8-9757-615f13e58d7d" (UID: "068f200e-7f4b-4cc8-9757-615f13e58d7d"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:39:38 crc kubenswrapper[5052]: I1203 12:39:38.445567 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/068f200e-7f4b-4cc8-9757-615f13e58d7d-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:39:38 crc kubenswrapper[5052]: I1203 12:39:38.445616 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/068f200e-7f4b-4cc8-9757-615f13e58d7d-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:39:38 crc kubenswrapper[5052]: I1203 12:39:38.477118 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/068f200e-7f4b-4cc8-9757-615f13e58d7d-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "068f200e-7f4b-4cc8-9757-615f13e58d7d" (UID: "068f200e-7f4b-4cc8-9757-615f13e58d7d"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:39:38 crc kubenswrapper[5052]: I1203 12:39:38.477250 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/068f200e-7f4b-4cc8-9757-615f13e58d7d-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "068f200e-7f4b-4cc8-9757-615f13e58d7d" (UID: "068f200e-7f4b-4cc8-9757-615f13e58d7d"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:39:38 crc kubenswrapper[5052]: I1203 12:39:38.548149 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/068f200e-7f4b-4cc8-9757-615f13e58d7d-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:39:38 crc kubenswrapper[5052]: I1203 12:39:38.548362 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/068f200e-7f4b-4cc8-9757-615f13e58d7d-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:39:38 crc kubenswrapper[5052]: I1203 12:39:38.930270 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/barbican-api-bcdbf47bb-m2ctz" Dec 03 12:39:38 crc kubenswrapper[5052]: I1203 12:39:38.990211 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-api-9bbb48f5b-4tfjv"] Dec 03 12:39:38 crc kubenswrapper[5052]: I1203 12:39:38.990965 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/barbican-api-9bbb48f5b-4tfjv" podUID="a998e1b6-f576-4463-b5ba-9f539f6939ee" containerName="barbican-api-log" containerID="cri-o://2f2c98f514a16f9410da4d782bafec1bccda88bd4b1ae8222e7d00e250b0991c" gracePeriod=30 Dec 03 12:39:38 crc kubenswrapper[5052]: I1203 12:39:38.991062 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/barbican-api-9bbb48f5b-4tfjv" podUID="a998e1b6-f576-4463-b5ba-9f539f6939ee" containerName="barbican-api" containerID="cri-o://3ad84be36d758109b2b6d9b25c2c91696eb0fa36fbcff38476460061f16a372e" gracePeriod=30 Dec 03 12:39:39 crc kubenswrapper[5052]: I1203 12:39:39.141583 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/placement-75785bcbbb-wlvdv"] Dec 03 12:39:39 crc kubenswrapper[5052]: I1203 12:39:39.165916 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/placement-75785bcbbb-wlvdv"] Dec 03 12:39:39 crc kubenswrapper[5052]: I1203 12:39:39.224671 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-cdf4456f4-v82rz" event={"ID":"99a0961b-8ed8-49b7-abac-5025d7b351ee","Type":"ContainerStarted","Data":"995bc9f60d05257f8493e1dd6196163744598cee55f2e09b16470fc6e4d9962b"} Dec 03 12:39:39 crc kubenswrapper[5052]: I1203 12:39:39.227314 5052 generic.go:334] "Generic (PLEG): container finished" podID="a998e1b6-f576-4463-b5ba-9f539f6939ee" containerID="2f2c98f514a16f9410da4d782bafec1bccda88bd4b1ae8222e7d00e250b0991c" exitCode=143 Dec 03 12:39:39 crc kubenswrapper[5052]: I1203 12:39:39.227666 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-9bbb48f5b-4tfjv" event={"ID":"a998e1b6-f576-4463-b5ba-9f539f6939ee","Type":"ContainerDied","Data":"2f2c98f514a16f9410da4d782bafec1bccda88bd4b1ae8222e7d00e250b0991c"} Dec 03 12:39:39 crc kubenswrapper[5052]: I1203 12:39:39.684642 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="068f200e-7f4b-4cc8-9757-615f13e58d7d" path="/var/lib/kubelet/pods/068f200e-7f4b-4cc8-9757-615f13e58d7d/volumes" Dec 03 12:39:40 crc kubenswrapper[5052]: I1203 12:39:40.251705 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-cdf4456f4-v82rz" event={"ID":"99a0961b-8ed8-49b7-abac-5025d7b351ee","Type":"ContainerStarted","Data":"60eb1509e2cea82540bce951978d0b80c496f80fe40a218c77ecbe97a3e9c835"} Dec 03 12:39:40 crc kubenswrapper[5052]: I1203 12:39:40.252184 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/placement-cdf4456f4-v82rz" Dec 03 12:39:40 crc kubenswrapper[5052]: I1203 12:39:40.252199 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/placement-cdf4456f4-v82rz" Dec 03 12:39:40 crc kubenswrapper[5052]: I1203 12:39:40.284154 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/placement-cdf4456f4-v82rz" podStartSLOduration=4.284135281 podStartE2EDuration="4.284135281s" podCreationTimestamp="2025-12-03 12:39:36 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:39:40.280329882 +0000 UTC m=+4770.868571666" watchObservedRunningTime="2025-12-03 12:39:40.284135281 +0000 UTC m=+4770.872377045" Dec 03 12:39:40 crc kubenswrapper[5052]: I1203 12:39:40.562045 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:39:40 crc kubenswrapper[5052]: I1203 12:39:40.634154 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:39:40 crc kubenswrapper[5052]: I1203 12:39:40.634219 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:39:40 crc kubenswrapper[5052]: I1203 12:39:40.671116 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:39:40 crc kubenswrapper[5052]: I1203 12:39:40.677455 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:39:40 crc kubenswrapper[5052]: I1203 12:39:40.677516 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:39:40 crc kubenswrapper[5052]: I1203 12:39:40.683045 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:39:40 crc kubenswrapper[5052]: I1203 12:39:40.719379 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:39:40 crc kubenswrapper[5052]: I1203 12:39:40.729665 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:39:41 crc kubenswrapper[5052]: I1203 12:39:41.264674 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:39:41 crc kubenswrapper[5052]: I1203 12:39:41.265304 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:39:41 crc kubenswrapper[5052]: I1203 12:39:41.265324 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:39:41 crc kubenswrapper[5052]: I1203 12:39:41.265337 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:39:42 crc kubenswrapper[5052]: I1203 12:39:42.277133 5052 generic.go:334] "Generic (PLEG): container finished" podID="a998e1b6-f576-4463-b5ba-9f539f6939ee" containerID="3ad84be36d758109b2b6d9b25c2c91696eb0fa36fbcff38476460061f16a372e" exitCode=0 Dec 03 12:39:42 crc kubenswrapper[5052]: I1203 12:39:42.277242 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-9bbb48f5b-4tfjv" event={"ID":"a998e1b6-f576-4463-b5ba-9f539f6939ee","Type":"ContainerDied","Data":"3ad84be36d758109b2b6d9b25c2c91696eb0fa36fbcff38476460061f16a372e"} Dec 03 12:39:42 crc kubenswrapper[5052]: I1203 12:39:42.585118 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-api-9bbb48f5b-4tfjv" Dec 03 12:39:42 crc kubenswrapper[5052]: I1203 12:39:42.633011 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a998e1b6-f576-4463-b5ba-9f539f6939ee-config-data\") pod \"a998e1b6-f576-4463-b5ba-9f539f6939ee\" (UID: \"a998e1b6-f576-4463-b5ba-9f539f6939ee\") " Dec 03 12:39:42 crc kubenswrapper[5052]: I1203 12:39:42.633092 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/a998e1b6-f576-4463-b5ba-9f539f6939ee-internal-tls-certs\") pod \"a998e1b6-f576-4463-b5ba-9f539f6939ee\" (UID: \"a998e1b6-f576-4463-b5ba-9f539f6939ee\") " Dec 03 12:39:42 crc kubenswrapper[5052]: I1203 12:39:42.633167 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/a998e1b6-f576-4463-b5ba-9f539f6939ee-public-tls-certs\") pod \"a998e1b6-f576-4463-b5ba-9f539f6939ee\" (UID: \"a998e1b6-f576-4463-b5ba-9f539f6939ee\") " Dec 03 12:39:42 crc kubenswrapper[5052]: I1203 12:39:42.633203 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/a998e1b6-f576-4463-b5ba-9f539f6939ee-logs\") pod \"a998e1b6-f576-4463-b5ba-9f539f6939ee\" (UID: \"a998e1b6-f576-4463-b5ba-9f539f6939ee\") " Dec 03 12:39:42 crc kubenswrapper[5052]: I1203 12:39:42.633249 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a998e1b6-f576-4463-b5ba-9f539f6939ee-combined-ca-bundle\") pod \"a998e1b6-f576-4463-b5ba-9f539f6939ee\" (UID: \"a998e1b6-f576-4463-b5ba-9f539f6939ee\") " Dec 03 12:39:42 crc kubenswrapper[5052]: I1203 12:39:42.633279 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x5vhp\" (UniqueName: \"kubernetes.io/projected/a998e1b6-f576-4463-b5ba-9f539f6939ee-kube-api-access-x5vhp\") pod \"a998e1b6-f576-4463-b5ba-9f539f6939ee\" (UID: \"a998e1b6-f576-4463-b5ba-9f539f6939ee\") " Dec 03 12:39:42 crc kubenswrapper[5052]: I1203 12:39:42.633441 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/a998e1b6-f576-4463-b5ba-9f539f6939ee-config-data-custom\") pod \"a998e1b6-f576-4463-b5ba-9f539f6939ee\" (UID: \"a998e1b6-f576-4463-b5ba-9f539f6939ee\") " Dec 03 12:39:42 crc kubenswrapper[5052]: I1203 12:39:42.634391 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a998e1b6-f576-4463-b5ba-9f539f6939ee-logs" (OuterVolumeSpecName: "logs") pod "a998e1b6-f576-4463-b5ba-9f539f6939ee" (UID: "a998e1b6-f576-4463-b5ba-9f539f6939ee"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:39:42 crc kubenswrapper[5052]: I1203 12:39:42.634670 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/a998e1b6-f576-4463-b5ba-9f539f6939ee-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:39:42 crc kubenswrapper[5052]: I1203 12:39:42.639362 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a998e1b6-f576-4463-b5ba-9f539f6939ee-config-data-custom" (OuterVolumeSpecName: "config-data-custom") pod "a998e1b6-f576-4463-b5ba-9f539f6939ee" (UID: "a998e1b6-f576-4463-b5ba-9f539f6939ee"). InnerVolumeSpecName "config-data-custom". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:39:42 crc kubenswrapper[5052]: I1203 12:39:42.639645 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a998e1b6-f576-4463-b5ba-9f539f6939ee-kube-api-access-x5vhp" (OuterVolumeSpecName: "kube-api-access-x5vhp") pod "a998e1b6-f576-4463-b5ba-9f539f6939ee" (UID: "a998e1b6-f576-4463-b5ba-9f539f6939ee"). InnerVolumeSpecName "kube-api-access-x5vhp". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:39:42 crc kubenswrapper[5052]: I1203 12:39:42.691366 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a998e1b6-f576-4463-b5ba-9f539f6939ee-config-data" (OuterVolumeSpecName: "config-data") pod "a998e1b6-f576-4463-b5ba-9f539f6939ee" (UID: "a998e1b6-f576-4463-b5ba-9f539f6939ee"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:39:42 crc kubenswrapper[5052]: I1203 12:39:42.698580 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a998e1b6-f576-4463-b5ba-9f539f6939ee-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "a998e1b6-f576-4463-b5ba-9f539f6939ee" (UID: "a998e1b6-f576-4463-b5ba-9f539f6939ee"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:39:42 crc kubenswrapper[5052]: I1203 12:39:42.699704 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a998e1b6-f576-4463-b5ba-9f539f6939ee-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "a998e1b6-f576-4463-b5ba-9f539f6939ee" (UID: "a998e1b6-f576-4463-b5ba-9f539f6939ee"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:39:42 crc kubenswrapper[5052]: I1203 12:39:42.730100 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a998e1b6-f576-4463-b5ba-9f539f6939ee-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "a998e1b6-f576-4463-b5ba-9f539f6939ee" (UID: "a998e1b6-f576-4463-b5ba-9f539f6939ee"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:39:42 crc kubenswrapper[5052]: I1203 12:39:42.737349 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a998e1b6-f576-4463-b5ba-9f539f6939ee-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:39:42 crc kubenswrapper[5052]: I1203 12:39:42.737384 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/a998e1b6-f576-4463-b5ba-9f539f6939ee-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:39:42 crc kubenswrapper[5052]: I1203 12:39:42.737399 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/a998e1b6-f576-4463-b5ba-9f539f6939ee-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:39:42 crc kubenswrapper[5052]: I1203 12:39:42.737426 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a998e1b6-f576-4463-b5ba-9f539f6939ee-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:39:42 crc kubenswrapper[5052]: I1203 12:39:42.737436 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-x5vhp\" (UniqueName: \"kubernetes.io/projected/a998e1b6-f576-4463-b5ba-9f539f6939ee-kube-api-access-x5vhp\") on node \"crc\" DevicePath \"\"" Dec 03 12:39:42 crc kubenswrapper[5052]: I1203 12:39:42.737447 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/a998e1b6-f576-4463-b5ba-9f539f6939ee-config-data-custom\") on node \"crc\" DevicePath \"\"" Dec 03 12:39:43 crc kubenswrapper[5052]: I1203 12:39:43.261460 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:39:43 crc kubenswrapper[5052]: I1203 12:39:43.263851 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:39:43 crc kubenswrapper[5052]: I1203 12:39:43.276755 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:39:43 crc kubenswrapper[5052]: I1203 12:39:43.288345 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:39:43 crc kubenswrapper[5052]: I1203 12:39:43.291379 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-9bbb48f5b-4tfjv" event={"ID":"a998e1b6-f576-4463-b5ba-9f539f6939ee","Type":"ContainerDied","Data":"f12cbd23c70cc8b4cadb90311161d8d4682966a9d562eadf8036ddbae5f994ed"} Dec 03 12:39:43 crc kubenswrapper[5052]: I1203 12:39:43.291429 5052 scope.go:117] "RemoveContainer" containerID="3ad84be36d758109b2b6d9b25c2c91696eb0fa36fbcff38476460061f16a372e" Dec 03 12:39:43 crc kubenswrapper[5052]: I1203 12:39:43.291575 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-api-9bbb48f5b-4tfjv" Dec 03 12:39:43 crc kubenswrapper[5052]: I1203 12:39:43.324101 5052 scope.go:117] "RemoveContainer" containerID="2f2c98f514a16f9410da4d782bafec1bccda88bd4b1ae8222e7d00e250b0991c" Dec 03 12:39:43 crc kubenswrapper[5052]: I1203 12:39:43.367466 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-api-9bbb48f5b-4tfjv"] Dec 03 12:39:43 crc kubenswrapper[5052]: I1203 12:39:43.410823 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/barbican-api-9bbb48f5b-4tfjv"] Dec 03 12:39:43 crc kubenswrapper[5052]: I1203 12:39:43.689533 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a998e1b6-f576-4463-b5ba-9f539f6939ee" path="/var/lib/kubelet/pods/a998e1b6-f576-4463-b5ba-9f539f6939ee/volumes" Dec 03 12:39:44 crc kubenswrapper[5052]: I1203 12:39:44.509552 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:39:44 crc kubenswrapper[5052]: I1203 12:39:44.509865 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:39:45 crc kubenswrapper[5052]: I1203 12:39:45.515808 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-api-0" podUID="50789f13-8194-4e24-b975-512591b816a5" containerName="nova-api-api" probeResult="failure" output="Get \"https://10.217.1.130:8774/\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Dec 03 12:39:45 crc kubenswrapper[5052]: I1203 12:39:45.521868 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-api-0" podUID="50789f13-8194-4e24-b975-512591b816a5" containerName="nova-api-log" probeResult="failure" output="Get \"https://10.217.1.130:8774/\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Dec 03 12:39:45 crc kubenswrapper[5052]: I1203 12:39:45.561490 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:39:45 crc kubenswrapper[5052]: I1203 12:39:45.579969 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:39:46 crc kubenswrapper[5052]: I1203 12:39:46.338186 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:39:47 crc kubenswrapper[5052]: I1203 12:39:47.357777 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:39:47 crc kubenswrapper[5052]: I1203 12:39:47.358695 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:39:47 crc kubenswrapper[5052]: I1203 12:39:47.363589 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:39:47 crc kubenswrapper[5052]: I1203 12:39:47.367898 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:39:53 crc kubenswrapper[5052]: I1203 12:39:53.395252 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/neutron-844859d58b-w4m57" podUID="061f3d17-a2e8-4002-ad1a-220ba7b476e3" containerName="neutron-httpd" probeResult="failure" output="Get \"https://10.217.1.122:9696/\": dial tcp 10.217.1.122:9696: connect: connection refused" Dec 03 12:39:54 crc kubenswrapper[5052]: I1203 12:39:54.522090 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:39:54 crc kubenswrapper[5052]: I1203 12:39:54.522861 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:39:54 crc kubenswrapper[5052]: I1203 12:39:54.526314 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:39:54 crc kubenswrapper[5052]: I1203 12:39:54.530434 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:39:54 crc kubenswrapper[5052]: I1203 12:39:54.939276 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/keystone-66f85f5bfb-vln92" Dec 03 12:39:55 crc kubenswrapper[5052]: I1203 12:39:55.015782 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/keystone-7476b7fc95-6cxvn"] Dec 03 12:39:55 crc kubenswrapper[5052]: I1203 12:39:55.016368 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/keystone-7476b7fc95-6cxvn" podUID="3f5299f3-7867-4a8a-9d64-f5a60526053a" containerName="keystone-api" containerID="cri-o://531c4d1034757d8c13569682d86b9abe1a7d5520dc20a2e62198919870a9b33c" gracePeriod=30 Dec 03 12:39:55 crc kubenswrapper[5052]: I1203 12:39:55.421965 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:39:55 crc kubenswrapper[5052]: I1203 12:39:55.430130 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:39:57 crc kubenswrapper[5052]: I1203 12:39:57.873564 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/openstackclient"] Dec 03 12:39:57 crc kubenswrapper[5052]: I1203 12:39:57.874157 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/openstackclient" podUID="24444a47-099d-4057-a118-669a0b8a7367" containerName="openstackclient" containerID="cri-o://dbee67def34d5df95d62c1a89f2db75ed8290851a65f7a468f58b7a707d3ad20" gracePeriod=2 Dec 03 12:39:57 crc kubenswrapper[5052]: I1203 12:39:57.885707 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/openstackclient"] Dec 03 12:39:57 crc kubenswrapper[5052]: I1203 12:39:57.916585 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/openstackclient"] Dec 03 12:39:57 crc kubenswrapper[5052]: E1203 12:39:57.917286 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="068f200e-7f4b-4cc8-9757-615f13e58d7d" containerName="placement-log" Dec 03 12:39:57 crc kubenswrapper[5052]: I1203 12:39:57.917314 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="068f200e-7f4b-4cc8-9757-615f13e58d7d" containerName="placement-log" Dec 03 12:39:57 crc kubenswrapper[5052]: E1203 12:39:57.917349 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="a998e1b6-f576-4463-b5ba-9f539f6939ee" containerName="barbican-api-log" Dec 03 12:39:57 crc kubenswrapper[5052]: I1203 12:39:57.917361 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="a998e1b6-f576-4463-b5ba-9f539f6939ee" containerName="barbican-api-log" Dec 03 12:39:57 crc kubenswrapper[5052]: E1203 12:39:57.917383 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="24444a47-099d-4057-a118-669a0b8a7367" containerName="openstackclient" Dec 03 12:39:57 crc kubenswrapper[5052]: I1203 12:39:57.917392 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="24444a47-099d-4057-a118-669a0b8a7367" containerName="openstackclient" Dec 03 12:39:57 crc kubenswrapper[5052]: E1203 12:39:57.917404 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="a998e1b6-f576-4463-b5ba-9f539f6939ee" containerName="barbican-api" Dec 03 12:39:57 crc kubenswrapper[5052]: I1203 12:39:57.917411 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="a998e1b6-f576-4463-b5ba-9f539f6939ee" containerName="barbican-api" Dec 03 12:39:57 crc kubenswrapper[5052]: E1203 12:39:57.917446 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="068f200e-7f4b-4cc8-9757-615f13e58d7d" containerName="placement-api" Dec 03 12:39:57 crc kubenswrapper[5052]: I1203 12:39:57.917454 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="068f200e-7f4b-4cc8-9757-615f13e58d7d" containerName="placement-api" Dec 03 12:39:57 crc kubenswrapper[5052]: I1203 12:39:57.917737 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="068f200e-7f4b-4cc8-9757-615f13e58d7d" containerName="placement-api" Dec 03 12:39:57 crc kubenswrapper[5052]: I1203 12:39:57.917787 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="a998e1b6-f576-4463-b5ba-9f539f6939ee" containerName="barbican-api-log" Dec 03 12:39:57 crc kubenswrapper[5052]: I1203 12:39:57.917815 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="a998e1b6-f576-4463-b5ba-9f539f6939ee" containerName="barbican-api" Dec 03 12:39:57 crc kubenswrapper[5052]: I1203 12:39:57.917834 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="068f200e-7f4b-4cc8-9757-615f13e58d7d" containerName="placement-log" Dec 03 12:39:57 crc kubenswrapper[5052]: I1203 12:39:57.917863 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="24444a47-099d-4057-a118-669a0b8a7367" containerName="openstackclient" Dec 03 12:39:57 crc kubenswrapper[5052]: I1203 12:39:57.918948 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstackclient" Dec 03 12:39:57 crc kubenswrapper[5052]: I1203 12:39:57.932895 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/openstackclient"] Dec 03 12:39:57 crc kubenswrapper[5052]: I1203 12:39:57.946441 5052 status_manager.go:861] "Pod was deleted and then recreated, skipping status update" pod="openstack-kuttl-tests/openstackclient" oldPodUID="24444a47-099d-4057-a118-669a0b8a7367" podUID="7ce91d4a-389d-4b37-b582-8c886566aa74" Dec 03 12:39:58 crc kubenswrapper[5052]: I1203 12:39:58.034967 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/7ce91d4a-389d-4b37-b582-8c886566aa74-combined-ca-bundle\") pod \"openstackclient\" (UID: \"7ce91d4a-389d-4b37-b582-8c886566aa74\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 12:39:58 crc kubenswrapper[5052]: I1203 12:39:58.035048 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openstack-config-secret\" (UniqueName: \"kubernetes.io/secret/7ce91d4a-389d-4b37-b582-8c886566aa74-openstack-config-secret\") pod \"openstackclient\" (UID: \"7ce91d4a-389d-4b37-b582-8c886566aa74\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 12:39:58 crc kubenswrapper[5052]: I1203 12:39:58.035128 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openstack-config\" (UniqueName: \"kubernetes.io/configmap/7ce91d4a-389d-4b37-b582-8c886566aa74-openstack-config\") pod \"openstackclient\" (UID: \"7ce91d4a-389d-4b37-b582-8c886566aa74\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 12:39:58 crc kubenswrapper[5052]: I1203 12:39:58.035409 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gllgp\" (UniqueName: \"kubernetes.io/projected/7ce91d4a-389d-4b37-b582-8c886566aa74-kube-api-access-gllgp\") pod \"openstackclient\" (UID: \"7ce91d4a-389d-4b37-b582-8c886566aa74\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 12:39:58 crc kubenswrapper[5052]: I1203 12:39:58.137818 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/7ce91d4a-389d-4b37-b582-8c886566aa74-combined-ca-bundle\") pod \"openstackclient\" (UID: \"7ce91d4a-389d-4b37-b582-8c886566aa74\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 12:39:58 crc kubenswrapper[5052]: I1203 12:39:58.137907 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"openstack-config-secret\" (UniqueName: \"kubernetes.io/secret/7ce91d4a-389d-4b37-b582-8c886566aa74-openstack-config-secret\") pod \"openstackclient\" (UID: \"7ce91d4a-389d-4b37-b582-8c886566aa74\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 12:39:58 crc kubenswrapper[5052]: I1203 12:39:58.138046 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"openstack-config\" (UniqueName: \"kubernetes.io/configmap/7ce91d4a-389d-4b37-b582-8c886566aa74-openstack-config\") pod \"openstackclient\" (UID: \"7ce91d4a-389d-4b37-b582-8c886566aa74\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 12:39:58 crc kubenswrapper[5052]: I1203 12:39:58.138177 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-gllgp\" (UniqueName: \"kubernetes.io/projected/7ce91d4a-389d-4b37-b582-8c886566aa74-kube-api-access-gllgp\") pod \"openstackclient\" (UID: \"7ce91d4a-389d-4b37-b582-8c886566aa74\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 12:39:58 crc kubenswrapper[5052]: I1203 12:39:58.139216 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"openstack-config\" (UniqueName: \"kubernetes.io/configmap/7ce91d4a-389d-4b37-b582-8c886566aa74-openstack-config\") pod \"openstackclient\" (UID: \"7ce91d4a-389d-4b37-b582-8c886566aa74\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 12:39:58 crc kubenswrapper[5052]: I1203 12:39:58.543178 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:39:58 crc kubenswrapper[5052]: I1203 12:39:58.624418 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"openstack-config-secret\" (UniqueName: \"kubernetes.io/secret/7ce91d4a-389d-4b37-b582-8c886566aa74-openstack-config-secret\") pod \"openstackclient\" (UID: \"7ce91d4a-389d-4b37-b582-8c886566aa74\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 12:39:58 crc kubenswrapper[5052]: I1203 12:39:58.624676 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-gllgp\" (UniqueName: \"kubernetes.io/projected/7ce91d4a-389d-4b37-b582-8c886566aa74-kube-api-access-gllgp\") pod \"openstackclient\" (UID: \"7ce91d4a-389d-4b37-b582-8c886566aa74\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 12:39:58 crc kubenswrapper[5052]: I1203 12:39:58.625884 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/7ce91d4a-389d-4b37-b582-8c886566aa74-combined-ca-bundle\") pod \"openstackclient\" (UID: \"7ce91d4a-389d-4b37-b582-8c886566aa74\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 12:39:58 crc kubenswrapper[5052]: I1203 12:39:58.807127 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/neutron-7d74684fd-pqkf5" Dec 03 12:39:58 crc kubenswrapper[5052]: I1203 12:39:58.851240 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstackclient" Dec 03 12:39:58 crc kubenswrapper[5052]: I1203 12:39:58.896952 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/neutron-5d67d5bcdb-klgzj"] Dec 03 12:39:58 crc kubenswrapper[5052]: I1203 12:39:58.897215 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/neutron-5d67d5bcdb-klgzj" podUID="4426a4f1-76e6-4746-a88a-4f4efead3977" containerName="neutron-api" containerID="cri-o://dde274eb8e16ff85ae1b2ee940add419f52d4e53a71e9f14e7931dcb8897d9fe" gracePeriod=30 Dec 03 12:39:58 crc kubenswrapper[5052]: I1203 12:39:58.897918 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/neutron-5d67d5bcdb-klgzj" podUID="4426a4f1-76e6-4746-a88a-4f4efead3977" containerName="neutron-httpd" containerID="cri-o://1a7463968b45581a86dc3faca6452efdbcfd78fb18a9ccb7360af588a2bd68a5" gracePeriod=30 Dec 03 12:39:59 crc kubenswrapper[5052]: I1203 12:39:59.027110 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_neutron-844859d58b-w4m57_061f3d17-a2e8-4002-ad1a-220ba7b476e3/neutron-api/0.log" Dec 03 12:39:59 crc kubenswrapper[5052]: I1203 12:39:59.027492 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-844859d58b-w4m57" Dec 03 12:39:59 crc kubenswrapper[5052]: I1203 12:39:59.064581 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/061f3d17-a2e8-4002-ad1a-220ba7b476e3-config\") pod \"061f3d17-a2e8-4002-ad1a-220ba7b476e3\" (UID: \"061f3d17-a2e8-4002-ad1a-220ba7b476e3\") " Dec 03 12:39:59 crc kubenswrapper[5052]: I1203 12:39:59.064630 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/061f3d17-a2e8-4002-ad1a-220ba7b476e3-ovndb-tls-certs\") pod \"061f3d17-a2e8-4002-ad1a-220ba7b476e3\" (UID: \"061f3d17-a2e8-4002-ad1a-220ba7b476e3\") " Dec 03 12:39:59 crc kubenswrapper[5052]: I1203 12:39:59.064668 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/061f3d17-a2e8-4002-ad1a-220ba7b476e3-combined-ca-bundle\") pod \"061f3d17-a2e8-4002-ad1a-220ba7b476e3\" (UID: \"061f3d17-a2e8-4002-ad1a-220ba7b476e3\") " Dec 03 12:39:59 crc kubenswrapper[5052]: I1203 12:39:59.064746 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/061f3d17-a2e8-4002-ad1a-220ba7b476e3-internal-tls-certs\") pod \"061f3d17-a2e8-4002-ad1a-220ba7b476e3\" (UID: \"061f3d17-a2e8-4002-ad1a-220ba7b476e3\") " Dec 03 12:39:59 crc kubenswrapper[5052]: I1203 12:39:59.064838 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lrdgs\" (UniqueName: \"kubernetes.io/projected/061f3d17-a2e8-4002-ad1a-220ba7b476e3-kube-api-access-lrdgs\") pod \"061f3d17-a2e8-4002-ad1a-220ba7b476e3\" (UID: \"061f3d17-a2e8-4002-ad1a-220ba7b476e3\") " Dec 03 12:39:59 crc kubenswrapper[5052]: I1203 12:39:59.064893 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/061f3d17-a2e8-4002-ad1a-220ba7b476e3-public-tls-certs\") pod \"061f3d17-a2e8-4002-ad1a-220ba7b476e3\" (UID: \"061f3d17-a2e8-4002-ad1a-220ba7b476e3\") " Dec 03 12:39:59 crc kubenswrapper[5052]: I1203 12:39:59.064913 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/061f3d17-a2e8-4002-ad1a-220ba7b476e3-httpd-config\") pod \"061f3d17-a2e8-4002-ad1a-220ba7b476e3\" (UID: \"061f3d17-a2e8-4002-ad1a-220ba7b476e3\") " Dec 03 12:39:59 crc kubenswrapper[5052]: I1203 12:39:59.071291 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/061f3d17-a2e8-4002-ad1a-220ba7b476e3-httpd-config" (OuterVolumeSpecName: "httpd-config") pod "061f3d17-a2e8-4002-ad1a-220ba7b476e3" (UID: "061f3d17-a2e8-4002-ad1a-220ba7b476e3"). InnerVolumeSpecName "httpd-config". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:39:59 crc kubenswrapper[5052]: I1203 12:39:59.077597 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/061f3d17-a2e8-4002-ad1a-220ba7b476e3-kube-api-access-lrdgs" (OuterVolumeSpecName: "kube-api-access-lrdgs") pod "061f3d17-a2e8-4002-ad1a-220ba7b476e3" (UID: "061f3d17-a2e8-4002-ad1a-220ba7b476e3"). InnerVolumeSpecName "kube-api-access-lrdgs". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:39:59 crc kubenswrapper[5052]: I1203 12:39:59.174314 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-lrdgs\" (UniqueName: \"kubernetes.io/projected/061f3d17-a2e8-4002-ad1a-220ba7b476e3-kube-api-access-lrdgs\") on node \"crc\" DevicePath \"\"" Dec 03 12:39:59 crc kubenswrapper[5052]: I1203 12:39:59.174340 5052 reconciler_common.go:293] "Volume detached for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/061f3d17-a2e8-4002-ad1a-220ba7b476e3-httpd-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:39:59 crc kubenswrapper[5052]: I1203 12:39:59.230858 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/061f3d17-a2e8-4002-ad1a-220ba7b476e3-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "061f3d17-a2e8-4002-ad1a-220ba7b476e3" (UID: "061f3d17-a2e8-4002-ad1a-220ba7b476e3"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:39:59 crc kubenswrapper[5052]: I1203 12:39:59.247364 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-7476b7fc95-6cxvn" Dec 03 12:39:59 crc kubenswrapper[5052]: I1203 12:39:59.321814 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/3f5299f3-7867-4a8a-9d64-f5a60526053a-fernet-keys\") pod \"3f5299f3-7867-4a8a-9d64-f5a60526053a\" (UID: \"3f5299f3-7867-4a8a-9d64-f5a60526053a\") " Dec 03 12:39:59 crc kubenswrapper[5052]: I1203 12:39:59.322968 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/3f5299f3-7867-4a8a-9d64-f5a60526053a-credential-keys\") pod \"3f5299f3-7867-4a8a-9d64-f5a60526053a\" (UID: \"3f5299f3-7867-4a8a-9d64-f5a60526053a\") " Dec 03 12:39:59 crc kubenswrapper[5052]: I1203 12:39:59.323018 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/3f5299f3-7867-4a8a-9d64-f5a60526053a-config-data\") pod \"3f5299f3-7867-4a8a-9d64-f5a60526053a\" (UID: \"3f5299f3-7867-4a8a-9d64-f5a60526053a\") " Dec 03 12:39:59 crc kubenswrapper[5052]: I1203 12:39:59.324158 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/061f3d17-a2e8-4002-ad1a-220ba7b476e3-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:39:59 crc kubenswrapper[5052]: I1203 12:39:59.327247 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3f5299f3-7867-4a8a-9d64-f5a60526053a-fernet-keys" (OuterVolumeSpecName: "fernet-keys") pod "3f5299f3-7867-4a8a-9d64-f5a60526053a" (UID: "3f5299f3-7867-4a8a-9d64-f5a60526053a"). InnerVolumeSpecName "fernet-keys". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:39:59 crc kubenswrapper[5052]: I1203 12:39:59.328616 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/061f3d17-a2e8-4002-ad1a-220ba7b476e3-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "061f3d17-a2e8-4002-ad1a-220ba7b476e3" (UID: "061f3d17-a2e8-4002-ad1a-220ba7b476e3"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:39:59 crc kubenswrapper[5052]: I1203 12:39:59.342179 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3f5299f3-7867-4a8a-9d64-f5a60526053a-credential-keys" (OuterVolumeSpecName: "credential-keys") pod "3f5299f3-7867-4a8a-9d64-f5a60526053a" (UID: "3f5299f3-7867-4a8a-9d64-f5a60526053a"). InnerVolumeSpecName "credential-keys". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:39:59 crc kubenswrapper[5052]: I1203 12:39:59.342205 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/061f3d17-a2e8-4002-ad1a-220ba7b476e3-config" (OuterVolumeSpecName: "config") pod "061f3d17-a2e8-4002-ad1a-220ba7b476e3" (UID: "061f3d17-a2e8-4002-ad1a-220ba7b476e3"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:39:59 crc kubenswrapper[5052]: I1203 12:39:59.342310 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/061f3d17-a2e8-4002-ad1a-220ba7b476e3-ovndb-tls-certs" (OuterVolumeSpecName: "ovndb-tls-certs") pod "061f3d17-a2e8-4002-ad1a-220ba7b476e3" (UID: "061f3d17-a2e8-4002-ad1a-220ba7b476e3"). InnerVolumeSpecName "ovndb-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:39:59 crc kubenswrapper[5052]: I1203 12:39:59.344746 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/061f3d17-a2e8-4002-ad1a-220ba7b476e3-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "061f3d17-a2e8-4002-ad1a-220ba7b476e3" (UID: "061f3d17-a2e8-4002-ad1a-220ba7b476e3"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:39:59 crc kubenswrapper[5052]: I1203 12:39:59.381890 5052 patch_prober.go:28] interesting pod/machine-config-daemon-74bj6 container/machine-config-daemon namespace/openshift-machine-config-operator: Liveness probe status=failure output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" start-of-body= Dec 03 12:39:59 crc kubenswrapper[5052]: I1203 12:39:59.381983 5052 prober.go:107] "Probe failed" probeType="Liveness" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" containerName="machine-config-daemon" probeResult="failure" output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" Dec 03 12:39:59 crc kubenswrapper[5052]: I1203 12:39:59.385206 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3f5299f3-7867-4a8a-9d64-f5a60526053a-config-data" (OuterVolumeSpecName: "config-data") pod "3f5299f3-7867-4a8a-9d64-f5a60526053a" (UID: "3f5299f3-7867-4a8a-9d64-f5a60526053a"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:39:59 crc kubenswrapper[5052]: I1203 12:39:59.424759 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6g9s7\" (UniqueName: \"kubernetes.io/projected/3f5299f3-7867-4a8a-9d64-f5a60526053a-kube-api-access-6g9s7\") pod \"3f5299f3-7867-4a8a-9d64-f5a60526053a\" (UID: \"3f5299f3-7867-4a8a-9d64-f5a60526053a\") " Dec 03 12:39:59 crc kubenswrapper[5052]: I1203 12:39:59.424828 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/3f5299f3-7867-4a8a-9d64-f5a60526053a-internal-tls-certs\") pod \"3f5299f3-7867-4a8a-9d64-f5a60526053a\" (UID: \"3f5299f3-7867-4a8a-9d64-f5a60526053a\") " Dec 03 12:39:59 crc kubenswrapper[5052]: I1203 12:39:59.424870 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/3f5299f3-7867-4a8a-9d64-f5a60526053a-public-tls-certs\") pod \"3f5299f3-7867-4a8a-9d64-f5a60526053a\" (UID: \"3f5299f3-7867-4a8a-9d64-f5a60526053a\") " Dec 03 12:39:59 crc kubenswrapper[5052]: I1203 12:39:59.425052 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/3f5299f3-7867-4a8a-9d64-f5a60526053a-scripts\") pod \"3f5299f3-7867-4a8a-9d64-f5a60526053a\" (UID: \"3f5299f3-7867-4a8a-9d64-f5a60526053a\") " Dec 03 12:39:59 crc kubenswrapper[5052]: I1203 12:39:59.425088 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3f5299f3-7867-4a8a-9d64-f5a60526053a-combined-ca-bundle\") pod \"3f5299f3-7867-4a8a-9d64-f5a60526053a\" (UID: \"3f5299f3-7867-4a8a-9d64-f5a60526053a\") " Dec 03 12:39:59 crc kubenswrapper[5052]: I1203 12:39:59.425542 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/061f3d17-a2e8-4002-ad1a-220ba7b476e3-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:39:59 crc kubenswrapper[5052]: I1203 12:39:59.425565 5052 reconciler_common.go:293] "Volume detached for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/061f3d17-a2e8-4002-ad1a-220ba7b476e3-ovndb-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:39:59 crc kubenswrapper[5052]: I1203 12:39:59.425579 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/061f3d17-a2e8-4002-ad1a-220ba7b476e3-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:39:59 crc kubenswrapper[5052]: I1203 12:39:59.425591 5052 reconciler_common.go:293] "Volume detached for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/3f5299f3-7867-4a8a-9d64-f5a60526053a-fernet-keys\") on node \"crc\" DevicePath \"\"" Dec 03 12:39:59 crc kubenswrapper[5052]: I1203 12:39:59.425603 5052 reconciler_common.go:293] "Volume detached for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/3f5299f3-7867-4a8a-9d64-f5a60526053a-credential-keys\") on node \"crc\" DevicePath \"\"" Dec 03 12:39:59 crc kubenswrapper[5052]: I1203 12:39:59.425636 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/061f3d17-a2e8-4002-ad1a-220ba7b476e3-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:39:59 crc kubenswrapper[5052]: I1203 12:39:59.425649 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/3f5299f3-7867-4a8a-9d64-f5a60526053a-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:39:59 crc kubenswrapper[5052]: I1203 12:39:59.429139 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3f5299f3-7867-4a8a-9d64-f5a60526053a-kube-api-access-6g9s7" (OuterVolumeSpecName: "kube-api-access-6g9s7") pod "3f5299f3-7867-4a8a-9d64-f5a60526053a" (UID: "3f5299f3-7867-4a8a-9d64-f5a60526053a"). InnerVolumeSpecName "kube-api-access-6g9s7". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:39:59 crc kubenswrapper[5052]: I1203 12:39:59.434240 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3f5299f3-7867-4a8a-9d64-f5a60526053a-scripts" (OuterVolumeSpecName: "scripts") pod "3f5299f3-7867-4a8a-9d64-f5a60526053a" (UID: "3f5299f3-7867-4a8a-9d64-f5a60526053a"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:39:59 crc kubenswrapper[5052]: I1203 12:39:59.457393 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3f5299f3-7867-4a8a-9d64-f5a60526053a-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "3f5299f3-7867-4a8a-9d64-f5a60526053a" (UID: "3f5299f3-7867-4a8a-9d64-f5a60526053a"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:39:59 crc kubenswrapper[5052]: I1203 12:39:59.489328 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3f5299f3-7867-4a8a-9d64-f5a60526053a-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "3f5299f3-7867-4a8a-9d64-f5a60526053a" (UID: "3f5299f3-7867-4a8a-9d64-f5a60526053a"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:39:59 crc kubenswrapper[5052]: I1203 12:39:59.489939 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_neutron-844859d58b-w4m57_061f3d17-a2e8-4002-ad1a-220ba7b476e3/neutron-api/0.log" Dec 03 12:39:59 crc kubenswrapper[5052]: I1203 12:39:59.489989 5052 generic.go:334] "Generic (PLEG): container finished" podID="061f3d17-a2e8-4002-ad1a-220ba7b476e3" containerID="acb084fda076785e51f840ee0c818b9e5cd206bb5a416282d7f91fb92bbb2890" exitCode=137 Dec 03 12:39:59 crc kubenswrapper[5052]: I1203 12:39:59.490068 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-844859d58b-w4m57" event={"ID":"061f3d17-a2e8-4002-ad1a-220ba7b476e3","Type":"ContainerDied","Data":"acb084fda076785e51f840ee0c818b9e5cd206bb5a416282d7f91fb92bbb2890"} Dec 03 12:39:59 crc kubenswrapper[5052]: I1203 12:39:59.490105 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-844859d58b-w4m57" event={"ID":"061f3d17-a2e8-4002-ad1a-220ba7b476e3","Type":"ContainerDied","Data":"6a00688cb6fb7a411e0d2866a1bf6c5532d21c44366d609d23981a2b288dc5d6"} Dec 03 12:39:59 crc kubenswrapper[5052]: I1203 12:39:59.490125 5052 scope.go:117] "RemoveContainer" containerID="5666f6ec09a1830c252db608dbb8ad35ec7dabe19b787b488fb6bf64296bffdf" Dec 03 12:39:59 crc kubenswrapper[5052]: I1203 12:39:59.490295 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-844859d58b-w4m57" Dec 03 12:39:59 crc kubenswrapper[5052]: I1203 12:39:59.492477 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3f5299f3-7867-4a8a-9d64-f5a60526053a-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "3f5299f3-7867-4a8a-9d64-f5a60526053a" (UID: "3f5299f3-7867-4a8a-9d64-f5a60526053a"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:39:59 crc kubenswrapper[5052]: I1203 12:39:59.493130 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/openstackclient"] Dec 03 12:39:59 crc kubenswrapper[5052]: I1203 12:39:59.500492 5052 generic.go:334] "Generic (PLEG): container finished" podID="4426a4f1-76e6-4746-a88a-4f4efead3977" containerID="1a7463968b45581a86dc3faca6452efdbcfd78fb18a9ccb7360af588a2bd68a5" exitCode=0 Dec 03 12:39:59 crc kubenswrapper[5052]: I1203 12:39:59.500579 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-5d67d5bcdb-klgzj" event={"ID":"4426a4f1-76e6-4746-a88a-4f4efead3977","Type":"ContainerDied","Data":"1a7463968b45581a86dc3faca6452efdbcfd78fb18a9ccb7360af588a2bd68a5"} Dec 03 12:39:59 crc kubenswrapper[5052]: I1203 12:39:59.507821 5052 generic.go:334] "Generic (PLEG): container finished" podID="3f5299f3-7867-4a8a-9d64-f5a60526053a" containerID="531c4d1034757d8c13569682d86b9abe1a7d5520dc20a2e62198919870a9b33c" exitCode=0 Dec 03 12:39:59 crc kubenswrapper[5052]: I1203 12:39:59.507875 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-7476b7fc95-6cxvn" event={"ID":"3f5299f3-7867-4a8a-9d64-f5a60526053a","Type":"ContainerDied","Data":"531c4d1034757d8c13569682d86b9abe1a7d5520dc20a2e62198919870a9b33c"} Dec 03 12:39:59 crc kubenswrapper[5052]: I1203 12:39:59.507910 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-7476b7fc95-6cxvn" event={"ID":"3f5299f3-7867-4a8a-9d64-f5a60526053a","Type":"ContainerDied","Data":"edbcb5ee93daf0c7c41dbed35f926ee32d334282b63827a19e6be8eb3cb238ac"} Dec 03 12:39:59 crc kubenswrapper[5052]: I1203 12:39:59.507876 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-7476b7fc95-6cxvn" Dec 03 12:39:59 crc kubenswrapper[5052]: I1203 12:39:59.526904 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/3f5299f3-7867-4a8a-9d64-f5a60526053a-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:39:59 crc kubenswrapper[5052]: I1203 12:39:59.526936 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/3f5299f3-7867-4a8a-9d64-f5a60526053a-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:39:59 crc kubenswrapper[5052]: I1203 12:39:59.526948 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/3f5299f3-7867-4a8a-9d64-f5a60526053a-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:39:59 crc kubenswrapper[5052]: I1203 12:39:59.526956 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3f5299f3-7867-4a8a-9d64-f5a60526053a-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:39:59 crc kubenswrapper[5052]: I1203 12:39:59.526967 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-6g9s7\" (UniqueName: \"kubernetes.io/projected/3f5299f3-7867-4a8a-9d64-f5a60526053a-kube-api-access-6g9s7\") on node \"crc\" DevicePath \"\"" Dec 03 12:39:59 crc kubenswrapper[5052]: I1203 12:39:59.541962 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/neutron-844859d58b-w4m57"] Dec 03 12:39:59 crc kubenswrapper[5052]: I1203 12:39:59.545767 5052 scope.go:117] "RemoveContainer" containerID="acb084fda076785e51f840ee0c818b9e5cd206bb5a416282d7f91fb92bbb2890" Dec 03 12:39:59 crc kubenswrapper[5052]: I1203 12:39:59.557225 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/neutron-844859d58b-w4m57"] Dec 03 12:39:59 crc kubenswrapper[5052]: I1203 12:39:59.567821 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/keystone-7476b7fc95-6cxvn"] Dec 03 12:39:59 crc kubenswrapper[5052]: I1203 12:39:59.576940 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/keystone-7476b7fc95-6cxvn"] Dec 03 12:39:59 crc kubenswrapper[5052]: I1203 12:39:59.586386 5052 scope.go:117] "RemoveContainer" containerID="5666f6ec09a1830c252db608dbb8ad35ec7dabe19b787b488fb6bf64296bffdf" Dec 03 12:39:59 crc kubenswrapper[5052]: E1203 12:39:59.589173 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5666f6ec09a1830c252db608dbb8ad35ec7dabe19b787b488fb6bf64296bffdf\": container with ID starting with 5666f6ec09a1830c252db608dbb8ad35ec7dabe19b787b488fb6bf64296bffdf not found: ID does not exist" containerID="5666f6ec09a1830c252db608dbb8ad35ec7dabe19b787b488fb6bf64296bffdf" Dec 03 12:39:59 crc kubenswrapper[5052]: I1203 12:39:59.589224 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5666f6ec09a1830c252db608dbb8ad35ec7dabe19b787b488fb6bf64296bffdf"} err="failed to get container status \"5666f6ec09a1830c252db608dbb8ad35ec7dabe19b787b488fb6bf64296bffdf\": rpc error: code = NotFound desc = could not find container \"5666f6ec09a1830c252db608dbb8ad35ec7dabe19b787b488fb6bf64296bffdf\": container with ID starting with 5666f6ec09a1830c252db608dbb8ad35ec7dabe19b787b488fb6bf64296bffdf not found: ID does not exist" Dec 03 12:39:59 crc kubenswrapper[5052]: I1203 12:39:59.589258 5052 scope.go:117] "RemoveContainer" containerID="acb084fda076785e51f840ee0c818b9e5cd206bb5a416282d7f91fb92bbb2890" Dec 03 12:39:59 crc kubenswrapper[5052]: E1203 12:39:59.589791 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"acb084fda076785e51f840ee0c818b9e5cd206bb5a416282d7f91fb92bbb2890\": container with ID starting with acb084fda076785e51f840ee0c818b9e5cd206bb5a416282d7f91fb92bbb2890 not found: ID does not exist" containerID="acb084fda076785e51f840ee0c818b9e5cd206bb5a416282d7f91fb92bbb2890" Dec 03 12:39:59 crc kubenswrapper[5052]: I1203 12:39:59.589823 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"acb084fda076785e51f840ee0c818b9e5cd206bb5a416282d7f91fb92bbb2890"} err="failed to get container status \"acb084fda076785e51f840ee0c818b9e5cd206bb5a416282d7f91fb92bbb2890\": rpc error: code = NotFound desc = could not find container \"acb084fda076785e51f840ee0c818b9e5cd206bb5a416282d7f91fb92bbb2890\": container with ID starting with acb084fda076785e51f840ee0c818b9e5cd206bb5a416282d7f91fb92bbb2890 not found: ID does not exist" Dec 03 12:39:59 crc kubenswrapper[5052]: I1203 12:39:59.589842 5052 scope.go:117] "RemoveContainer" containerID="531c4d1034757d8c13569682d86b9abe1a7d5520dc20a2e62198919870a9b33c" Dec 03 12:39:59 crc kubenswrapper[5052]: I1203 12:39:59.623032 5052 scope.go:117] "RemoveContainer" containerID="531c4d1034757d8c13569682d86b9abe1a7d5520dc20a2e62198919870a9b33c" Dec 03 12:39:59 crc kubenswrapper[5052]: E1203 12:39:59.623763 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"531c4d1034757d8c13569682d86b9abe1a7d5520dc20a2e62198919870a9b33c\": container with ID starting with 531c4d1034757d8c13569682d86b9abe1a7d5520dc20a2e62198919870a9b33c not found: ID does not exist" containerID="531c4d1034757d8c13569682d86b9abe1a7d5520dc20a2e62198919870a9b33c" Dec 03 12:39:59 crc kubenswrapper[5052]: I1203 12:39:59.623880 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"531c4d1034757d8c13569682d86b9abe1a7d5520dc20a2e62198919870a9b33c"} err="failed to get container status \"531c4d1034757d8c13569682d86b9abe1a7d5520dc20a2e62198919870a9b33c\": rpc error: code = NotFound desc = could not find container \"531c4d1034757d8c13569682d86b9abe1a7d5520dc20a2e62198919870a9b33c\": container with ID starting with 531c4d1034757d8c13569682d86b9abe1a7d5520dc20a2e62198919870a9b33c not found: ID does not exist" Dec 03 12:39:59 crc kubenswrapper[5052]: I1203 12:39:59.686761 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="061f3d17-a2e8-4002-ad1a-220ba7b476e3" path="/var/lib/kubelet/pods/061f3d17-a2e8-4002-ad1a-220ba7b476e3/volumes" Dec 03 12:39:59 crc kubenswrapper[5052]: I1203 12:39:59.687687 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3f5299f3-7867-4a8a-9d64-f5a60526053a" path="/var/lib/kubelet/pods/3f5299f3-7867-4a8a-9d64-f5a60526053a/volumes" Dec 03 12:40:00 crc kubenswrapper[5052]: I1203 12:40:00.542897 5052 generic.go:334] "Generic (PLEG): container finished" podID="24444a47-099d-4057-a118-669a0b8a7367" containerID="dbee67def34d5df95d62c1a89f2db75ed8290851a65f7a468f58b7a707d3ad20" exitCode=137 Dec 03 12:40:00 crc kubenswrapper[5052]: I1203 12:40:00.543207 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5ad770201751b3d290beab02cfed3f0a09455c512e46f25c71ef148fa308869f" Dec 03 12:40:00 crc kubenswrapper[5052]: I1203 12:40:00.547076 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstackclient" event={"ID":"7ce91d4a-389d-4b37-b582-8c886566aa74","Type":"ContainerStarted","Data":"229df7ab571e9eb2bf196a218d66569b5405729090282a855310baf493b87409"} Dec 03 12:40:00 crc kubenswrapper[5052]: I1203 12:40:00.547115 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstackclient" event={"ID":"7ce91d4a-389d-4b37-b582-8c886566aa74","Type":"ContainerStarted","Data":"4ab86855c1c9a4b07f00dbffd2af9d9e18ba36b5823c06baeaeb2708e3bc2f03"} Dec 03 12:40:00 crc kubenswrapper[5052]: I1203 12:40:00.573768 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/openstackclient" podStartSLOduration=3.573743767 podStartE2EDuration="3.573743767s" podCreationTimestamp="2025-12-03 12:39:57 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:40:00.565290955 +0000 UTC m=+4791.153532719" watchObservedRunningTime="2025-12-03 12:40:00.573743767 +0000 UTC m=+4791.161985531" Dec 03 12:40:00 crc kubenswrapper[5052]: I1203 12:40:00.578230 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstackclient" Dec 03 12:40:00 crc kubenswrapper[5052]: I1203 12:40:00.583799 5052 status_manager.go:861] "Pod was deleted and then recreated, skipping status update" pod="openstack-kuttl-tests/openstackclient" oldPodUID="24444a47-099d-4057-a118-669a0b8a7367" podUID="7ce91d4a-389d-4b37-b582-8c886566aa74" Dec 03 12:40:00 crc kubenswrapper[5052]: I1203 12:40:00.749659 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/24444a47-099d-4057-a118-669a0b8a7367-combined-ca-bundle\") pod \"24444a47-099d-4057-a118-669a0b8a7367\" (UID: \"24444a47-099d-4057-a118-669a0b8a7367\") " Dec 03 12:40:00 crc kubenswrapper[5052]: I1203 12:40:00.749778 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sxvvg\" (UniqueName: \"kubernetes.io/projected/24444a47-099d-4057-a118-669a0b8a7367-kube-api-access-sxvvg\") pod \"24444a47-099d-4057-a118-669a0b8a7367\" (UID: \"24444a47-099d-4057-a118-669a0b8a7367\") " Dec 03 12:40:00 crc kubenswrapper[5052]: I1203 12:40:00.750300 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"openstack-config-secret\" (UniqueName: \"kubernetes.io/secret/24444a47-099d-4057-a118-669a0b8a7367-openstack-config-secret\") pod \"24444a47-099d-4057-a118-669a0b8a7367\" (UID: \"24444a47-099d-4057-a118-669a0b8a7367\") " Dec 03 12:40:00 crc kubenswrapper[5052]: I1203 12:40:00.750586 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"openstack-config\" (UniqueName: \"kubernetes.io/configmap/24444a47-099d-4057-a118-669a0b8a7367-openstack-config\") pod \"24444a47-099d-4057-a118-669a0b8a7367\" (UID: \"24444a47-099d-4057-a118-669a0b8a7367\") " Dec 03 12:40:00 crc kubenswrapper[5052]: I1203 12:40:00.760717 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/24444a47-099d-4057-a118-669a0b8a7367-kube-api-access-sxvvg" (OuterVolumeSpecName: "kube-api-access-sxvvg") pod "24444a47-099d-4057-a118-669a0b8a7367" (UID: "24444a47-099d-4057-a118-669a0b8a7367"). InnerVolumeSpecName "kube-api-access-sxvvg". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:40:00 crc kubenswrapper[5052]: I1203 12:40:00.788419 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/24444a47-099d-4057-a118-669a0b8a7367-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "24444a47-099d-4057-a118-669a0b8a7367" (UID: "24444a47-099d-4057-a118-669a0b8a7367"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:40:00 crc kubenswrapper[5052]: I1203 12:40:00.791338 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/24444a47-099d-4057-a118-669a0b8a7367-openstack-config" (OuterVolumeSpecName: "openstack-config") pod "24444a47-099d-4057-a118-669a0b8a7367" (UID: "24444a47-099d-4057-a118-669a0b8a7367"). InnerVolumeSpecName "openstack-config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:40:00 crc kubenswrapper[5052]: I1203 12:40:00.834608 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/24444a47-099d-4057-a118-669a0b8a7367-openstack-config-secret" (OuterVolumeSpecName: "openstack-config-secret") pod "24444a47-099d-4057-a118-669a0b8a7367" (UID: "24444a47-099d-4057-a118-669a0b8a7367"). InnerVolumeSpecName "openstack-config-secret". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:40:00 crc kubenswrapper[5052]: I1203 12:40:00.853288 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/24444a47-099d-4057-a118-669a0b8a7367-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:40:00 crc kubenswrapper[5052]: I1203 12:40:00.853343 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-sxvvg\" (UniqueName: \"kubernetes.io/projected/24444a47-099d-4057-a118-669a0b8a7367-kube-api-access-sxvvg\") on node \"crc\" DevicePath \"\"" Dec 03 12:40:00 crc kubenswrapper[5052]: I1203 12:40:00.853358 5052 reconciler_common.go:293] "Volume detached for volume \"openstack-config-secret\" (UniqueName: \"kubernetes.io/secret/24444a47-099d-4057-a118-669a0b8a7367-openstack-config-secret\") on node \"crc\" DevicePath \"\"" Dec 03 12:40:00 crc kubenswrapper[5052]: I1203 12:40:00.853372 5052 reconciler_common.go:293] "Volume detached for volume \"openstack-config\" (UniqueName: \"kubernetes.io/configmap/24444a47-099d-4057-a118-669a0b8a7367-openstack-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:40:01 crc kubenswrapper[5052]: I1203 12:40:01.557033 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstackclient" Dec 03 12:40:01 crc kubenswrapper[5052]: I1203 12:40:01.560859 5052 status_manager.go:861] "Pod was deleted and then recreated, skipping status update" pod="openstack-kuttl-tests/openstackclient" oldPodUID="24444a47-099d-4057-a118-669a0b8a7367" podUID="7ce91d4a-389d-4b37-b582-8c886566aa74" Dec 03 12:40:01 crc kubenswrapper[5052]: I1203 12:40:01.580812 5052 status_manager.go:861] "Pod was deleted and then recreated, skipping status update" pod="openstack-kuttl-tests/openstackclient" oldPodUID="24444a47-099d-4057-a118-669a0b8a7367" podUID="7ce91d4a-389d-4b37-b582-8c886566aa74" Dec 03 12:40:01 crc kubenswrapper[5052]: I1203 12:40:01.685474 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="24444a47-099d-4057-a118-669a0b8a7367" path="/var/lib/kubelet/pods/24444a47-099d-4057-a118-669a0b8a7367/volumes" Dec 03 12:40:03 crc kubenswrapper[5052]: I1203 12:40:03.073189 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:40:03 crc kubenswrapper[5052]: I1203 12:40:03.074266 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="95a06af5-270d-4138-ab79-fcdd239b3997" containerName="ceilometer-central-agent" containerID="cri-o://29357b3d97482fd8007dd5664e67838f357ddeab54a10504c300a6188e0d2e3d" gracePeriod=30 Dec 03 12:40:03 crc kubenswrapper[5052]: I1203 12:40:03.074349 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="95a06af5-270d-4138-ab79-fcdd239b3997" containerName="sg-core" containerID="cri-o://af9343e98edf473dab463b002e96eedd0ca172c95e67fa9e25832f1c73a3318a" gracePeriod=30 Dec 03 12:40:03 crc kubenswrapper[5052]: I1203 12:40:03.074403 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="95a06af5-270d-4138-ab79-fcdd239b3997" containerName="ceilometer-notification-agent" containerID="cri-o://0ef7cd09cfd4cf1e5792656dab2c9785fb2b5b3445c4f04f4ff27b9d93363509" gracePeriod=30 Dec 03 12:40:03 crc kubenswrapper[5052]: I1203 12:40:03.074609 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="95a06af5-270d-4138-ab79-fcdd239b3997" containerName="proxy-httpd" containerID="cri-o://9c894be2b5ea25d2e1dc93642f17b355a9f5eb4d38037b4b3d4de06025012260" gracePeriod=30 Dec 03 12:40:03 crc kubenswrapper[5052]: I1203 12:40:03.167993 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/swift-proxy-bf56d5474-cq7b2"] Dec 03 12:40:03 crc kubenswrapper[5052]: E1203 12:40:03.168676 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="3f5299f3-7867-4a8a-9d64-f5a60526053a" containerName="keystone-api" Dec 03 12:40:03 crc kubenswrapper[5052]: I1203 12:40:03.168694 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="3f5299f3-7867-4a8a-9d64-f5a60526053a" containerName="keystone-api" Dec 03 12:40:03 crc kubenswrapper[5052]: E1203 12:40:03.168706 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="061f3d17-a2e8-4002-ad1a-220ba7b476e3" containerName="neutron-api" Dec 03 12:40:03 crc kubenswrapper[5052]: I1203 12:40:03.168713 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="061f3d17-a2e8-4002-ad1a-220ba7b476e3" containerName="neutron-api" Dec 03 12:40:03 crc kubenswrapper[5052]: E1203 12:40:03.168734 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="061f3d17-a2e8-4002-ad1a-220ba7b476e3" containerName="neutron-httpd" Dec 03 12:40:03 crc kubenswrapper[5052]: I1203 12:40:03.168740 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="061f3d17-a2e8-4002-ad1a-220ba7b476e3" containerName="neutron-httpd" Dec 03 12:40:03 crc kubenswrapper[5052]: I1203 12:40:03.168949 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="3f5299f3-7867-4a8a-9d64-f5a60526053a" containerName="keystone-api" Dec 03 12:40:03 crc kubenswrapper[5052]: I1203 12:40:03.168963 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="061f3d17-a2e8-4002-ad1a-220ba7b476e3" containerName="neutron-api" Dec 03 12:40:03 crc kubenswrapper[5052]: I1203 12:40:03.168976 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="061f3d17-a2e8-4002-ad1a-220ba7b476e3" containerName="neutron-httpd" Dec 03 12:40:03 crc kubenswrapper[5052]: I1203 12:40:03.246714 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/swift-proxy-bf56d5474-cq7b2"] Dec 03 12:40:03 crc kubenswrapper[5052]: I1203 12:40:03.246847 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-proxy-bf56d5474-cq7b2" Dec 03 12:40:03 crc kubenswrapper[5052]: I1203 12:40:03.410487 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/621935c3-f8f9-43a1-9bc3-6d6ad9caf0ef-combined-ca-bundle\") pod \"swift-proxy-bf56d5474-cq7b2\" (UID: \"621935c3-f8f9-43a1-9bc3-6d6ad9caf0ef\") " pod="openstack-kuttl-tests/swift-proxy-bf56d5474-cq7b2" Dec 03 12:40:03 crc kubenswrapper[5052]: I1203 12:40:03.410576 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/621935c3-f8f9-43a1-9bc3-6d6ad9caf0ef-internal-tls-certs\") pod \"swift-proxy-bf56d5474-cq7b2\" (UID: \"621935c3-f8f9-43a1-9bc3-6d6ad9caf0ef\") " pod="openstack-kuttl-tests/swift-proxy-bf56d5474-cq7b2" Dec 03 12:40:03 crc kubenswrapper[5052]: I1203 12:40:03.410642 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/621935c3-f8f9-43a1-9bc3-6d6ad9caf0ef-log-httpd\") pod \"swift-proxy-bf56d5474-cq7b2\" (UID: \"621935c3-f8f9-43a1-9bc3-6d6ad9caf0ef\") " pod="openstack-kuttl-tests/swift-proxy-bf56d5474-cq7b2" Dec 03 12:40:03 crc kubenswrapper[5052]: I1203 12:40:03.410668 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/621935c3-f8f9-43a1-9bc3-6d6ad9caf0ef-etc-swift\") pod \"swift-proxy-bf56d5474-cq7b2\" (UID: \"621935c3-f8f9-43a1-9bc3-6d6ad9caf0ef\") " pod="openstack-kuttl-tests/swift-proxy-bf56d5474-cq7b2" Dec 03 12:40:03 crc kubenswrapper[5052]: I1203 12:40:03.410743 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4mnb7\" (UniqueName: \"kubernetes.io/projected/621935c3-f8f9-43a1-9bc3-6d6ad9caf0ef-kube-api-access-4mnb7\") pod \"swift-proxy-bf56d5474-cq7b2\" (UID: \"621935c3-f8f9-43a1-9bc3-6d6ad9caf0ef\") " pod="openstack-kuttl-tests/swift-proxy-bf56d5474-cq7b2" Dec 03 12:40:03 crc kubenswrapper[5052]: I1203 12:40:03.410799 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/621935c3-f8f9-43a1-9bc3-6d6ad9caf0ef-run-httpd\") pod \"swift-proxy-bf56d5474-cq7b2\" (UID: \"621935c3-f8f9-43a1-9bc3-6d6ad9caf0ef\") " pod="openstack-kuttl-tests/swift-proxy-bf56d5474-cq7b2" Dec 03 12:40:03 crc kubenswrapper[5052]: I1203 12:40:03.410873 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/621935c3-f8f9-43a1-9bc3-6d6ad9caf0ef-config-data\") pod \"swift-proxy-bf56d5474-cq7b2\" (UID: \"621935c3-f8f9-43a1-9bc3-6d6ad9caf0ef\") " pod="openstack-kuttl-tests/swift-proxy-bf56d5474-cq7b2" Dec 03 12:40:03 crc kubenswrapper[5052]: I1203 12:40:03.411019 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/621935c3-f8f9-43a1-9bc3-6d6ad9caf0ef-public-tls-certs\") pod \"swift-proxy-bf56d5474-cq7b2\" (UID: \"621935c3-f8f9-43a1-9bc3-6d6ad9caf0ef\") " pod="openstack-kuttl-tests/swift-proxy-bf56d5474-cq7b2" Dec 03 12:40:03 crc kubenswrapper[5052]: I1203 12:40:03.513654 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/621935c3-f8f9-43a1-9bc3-6d6ad9caf0ef-config-data\") pod \"swift-proxy-bf56d5474-cq7b2\" (UID: \"621935c3-f8f9-43a1-9bc3-6d6ad9caf0ef\") " pod="openstack-kuttl-tests/swift-proxy-bf56d5474-cq7b2" Dec 03 12:40:03 crc kubenswrapper[5052]: I1203 12:40:03.513733 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/621935c3-f8f9-43a1-9bc3-6d6ad9caf0ef-public-tls-certs\") pod \"swift-proxy-bf56d5474-cq7b2\" (UID: \"621935c3-f8f9-43a1-9bc3-6d6ad9caf0ef\") " pod="openstack-kuttl-tests/swift-proxy-bf56d5474-cq7b2" Dec 03 12:40:03 crc kubenswrapper[5052]: I1203 12:40:03.513853 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/621935c3-f8f9-43a1-9bc3-6d6ad9caf0ef-combined-ca-bundle\") pod \"swift-proxy-bf56d5474-cq7b2\" (UID: \"621935c3-f8f9-43a1-9bc3-6d6ad9caf0ef\") " pod="openstack-kuttl-tests/swift-proxy-bf56d5474-cq7b2" Dec 03 12:40:03 crc kubenswrapper[5052]: I1203 12:40:03.513887 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/621935c3-f8f9-43a1-9bc3-6d6ad9caf0ef-internal-tls-certs\") pod \"swift-proxy-bf56d5474-cq7b2\" (UID: \"621935c3-f8f9-43a1-9bc3-6d6ad9caf0ef\") " pod="openstack-kuttl-tests/swift-proxy-bf56d5474-cq7b2" Dec 03 12:40:03 crc kubenswrapper[5052]: I1203 12:40:03.513925 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/621935c3-f8f9-43a1-9bc3-6d6ad9caf0ef-log-httpd\") pod \"swift-proxy-bf56d5474-cq7b2\" (UID: \"621935c3-f8f9-43a1-9bc3-6d6ad9caf0ef\") " pod="openstack-kuttl-tests/swift-proxy-bf56d5474-cq7b2" Dec 03 12:40:03 crc kubenswrapper[5052]: I1203 12:40:03.513944 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/621935c3-f8f9-43a1-9bc3-6d6ad9caf0ef-etc-swift\") pod \"swift-proxy-bf56d5474-cq7b2\" (UID: \"621935c3-f8f9-43a1-9bc3-6d6ad9caf0ef\") " pod="openstack-kuttl-tests/swift-proxy-bf56d5474-cq7b2" Dec 03 12:40:03 crc kubenswrapper[5052]: I1203 12:40:03.513990 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-4mnb7\" (UniqueName: \"kubernetes.io/projected/621935c3-f8f9-43a1-9bc3-6d6ad9caf0ef-kube-api-access-4mnb7\") pod \"swift-proxy-bf56d5474-cq7b2\" (UID: \"621935c3-f8f9-43a1-9bc3-6d6ad9caf0ef\") " pod="openstack-kuttl-tests/swift-proxy-bf56d5474-cq7b2" Dec 03 12:40:03 crc kubenswrapper[5052]: I1203 12:40:03.514021 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/621935c3-f8f9-43a1-9bc3-6d6ad9caf0ef-run-httpd\") pod \"swift-proxy-bf56d5474-cq7b2\" (UID: \"621935c3-f8f9-43a1-9bc3-6d6ad9caf0ef\") " pod="openstack-kuttl-tests/swift-proxy-bf56d5474-cq7b2" Dec 03 12:40:03 crc kubenswrapper[5052]: I1203 12:40:03.514750 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/621935c3-f8f9-43a1-9bc3-6d6ad9caf0ef-run-httpd\") pod \"swift-proxy-bf56d5474-cq7b2\" (UID: \"621935c3-f8f9-43a1-9bc3-6d6ad9caf0ef\") " pod="openstack-kuttl-tests/swift-proxy-bf56d5474-cq7b2" Dec 03 12:40:03 crc kubenswrapper[5052]: I1203 12:40:03.514993 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/621935c3-f8f9-43a1-9bc3-6d6ad9caf0ef-log-httpd\") pod \"swift-proxy-bf56d5474-cq7b2\" (UID: \"621935c3-f8f9-43a1-9bc3-6d6ad9caf0ef\") " pod="openstack-kuttl-tests/swift-proxy-bf56d5474-cq7b2" Dec 03 12:40:03 crc kubenswrapper[5052]: I1203 12:40:03.523172 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/621935c3-f8f9-43a1-9bc3-6d6ad9caf0ef-internal-tls-certs\") pod \"swift-proxy-bf56d5474-cq7b2\" (UID: \"621935c3-f8f9-43a1-9bc3-6d6ad9caf0ef\") " pod="openstack-kuttl-tests/swift-proxy-bf56d5474-cq7b2" Dec 03 12:40:03 crc kubenswrapper[5052]: I1203 12:40:03.524194 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/621935c3-f8f9-43a1-9bc3-6d6ad9caf0ef-etc-swift\") pod \"swift-proxy-bf56d5474-cq7b2\" (UID: \"621935c3-f8f9-43a1-9bc3-6d6ad9caf0ef\") " pod="openstack-kuttl-tests/swift-proxy-bf56d5474-cq7b2" Dec 03 12:40:03 crc kubenswrapper[5052]: I1203 12:40:03.525260 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/621935c3-f8f9-43a1-9bc3-6d6ad9caf0ef-combined-ca-bundle\") pod \"swift-proxy-bf56d5474-cq7b2\" (UID: \"621935c3-f8f9-43a1-9bc3-6d6ad9caf0ef\") " pod="openstack-kuttl-tests/swift-proxy-bf56d5474-cq7b2" Dec 03 12:40:03 crc kubenswrapper[5052]: I1203 12:40:03.525980 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/621935c3-f8f9-43a1-9bc3-6d6ad9caf0ef-config-data\") pod \"swift-proxy-bf56d5474-cq7b2\" (UID: \"621935c3-f8f9-43a1-9bc3-6d6ad9caf0ef\") " pod="openstack-kuttl-tests/swift-proxy-bf56d5474-cq7b2" Dec 03 12:40:03 crc kubenswrapper[5052]: I1203 12:40:03.527942 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/621935c3-f8f9-43a1-9bc3-6d6ad9caf0ef-public-tls-certs\") pod \"swift-proxy-bf56d5474-cq7b2\" (UID: \"621935c3-f8f9-43a1-9bc3-6d6ad9caf0ef\") " pod="openstack-kuttl-tests/swift-proxy-bf56d5474-cq7b2" Dec 03 12:40:03 crc kubenswrapper[5052]: I1203 12:40:03.538110 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-4mnb7\" (UniqueName: \"kubernetes.io/projected/621935c3-f8f9-43a1-9bc3-6d6ad9caf0ef-kube-api-access-4mnb7\") pod \"swift-proxy-bf56d5474-cq7b2\" (UID: \"621935c3-f8f9-43a1-9bc3-6d6ad9caf0ef\") " pod="openstack-kuttl-tests/swift-proxy-bf56d5474-cq7b2" Dec 03 12:40:03 crc kubenswrapper[5052]: I1203 12:40:03.580986 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-proxy-bf56d5474-cq7b2" Dec 03 12:40:03 crc kubenswrapper[5052]: I1203 12:40:03.584912 5052 generic.go:334] "Generic (PLEG): container finished" podID="95a06af5-270d-4138-ab79-fcdd239b3997" containerID="9c894be2b5ea25d2e1dc93642f17b355a9f5eb4d38037b4b3d4de06025012260" exitCode=0 Dec 03 12:40:03 crc kubenswrapper[5052]: I1203 12:40:03.585130 5052 generic.go:334] "Generic (PLEG): container finished" podID="95a06af5-270d-4138-ab79-fcdd239b3997" containerID="af9343e98edf473dab463b002e96eedd0ca172c95e67fa9e25832f1c73a3318a" exitCode=2 Dec 03 12:40:03 crc kubenswrapper[5052]: I1203 12:40:03.585130 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"95a06af5-270d-4138-ab79-fcdd239b3997","Type":"ContainerDied","Data":"9c894be2b5ea25d2e1dc93642f17b355a9f5eb4d38037b4b3d4de06025012260"} Dec 03 12:40:03 crc kubenswrapper[5052]: I1203 12:40:03.585326 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"95a06af5-270d-4138-ab79-fcdd239b3997","Type":"ContainerDied","Data":"af9343e98edf473dab463b002e96eedd0ca172c95e67fa9e25832f1c73a3318a"} Dec 03 12:40:04 crc kubenswrapper[5052]: I1203 12:40:04.093376 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/swift-proxy-bf56d5474-cq7b2"] Dec 03 12:40:04 crc kubenswrapper[5052]: I1203 12:40:04.596579 5052 generic.go:334] "Generic (PLEG): container finished" podID="95a06af5-270d-4138-ab79-fcdd239b3997" containerID="29357b3d97482fd8007dd5664e67838f357ddeab54a10504c300a6188e0d2e3d" exitCode=0 Dec 03 12:40:04 crc kubenswrapper[5052]: I1203 12:40:04.596623 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"95a06af5-270d-4138-ab79-fcdd239b3997","Type":"ContainerDied","Data":"29357b3d97482fd8007dd5664e67838f357ddeab54a10504c300a6188e0d2e3d"} Dec 03 12:40:04 crc kubenswrapper[5052]: I1203 12:40:04.598957 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-proxy-bf56d5474-cq7b2" event={"ID":"621935c3-f8f9-43a1-9bc3-6d6ad9caf0ef","Type":"ContainerStarted","Data":"30dfc639d124cfef009d7624b44b757995f402edb14bbdfeb69e215f2a21b847"} Dec 03 12:40:04 crc kubenswrapper[5052]: I1203 12:40:04.598984 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-proxy-bf56d5474-cq7b2" event={"ID":"621935c3-f8f9-43a1-9bc3-6d6ad9caf0ef","Type":"ContainerStarted","Data":"b1c287082c6be0fc94453ba31598d9aa15f58f1efff238cadada8687a0b597d4"} Dec 03 12:40:05 crc kubenswrapper[5052]: I1203 12:40:05.610948 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-proxy-bf56d5474-cq7b2" event={"ID":"621935c3-f8f9-43a1-9bc3-6d6ad9caf0ef","Type":"ContainerStarted","Data":"53722c8da45292bfe9fdbd4b2d2a18b053dc900a020f10bc15c4643959d2c7c1"} Dec 03 12:40:05 crc kubenswrapper[5052]: I1203 12:40:05.611912 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/swift-proxy-bf56d5474-cq7b2" Dec 03 12:40:05 crc kubenswrapper[5052]: I1203 12:40:05.611928 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/swift-proxy-bf56d5474-cq7b2" Dec 03 12:40:05 crc kubenswrapper[5052]: I1203 12:40:05.616409 5052 generic.go:334] "Generic (PLEG): container finished" podID="95a06af5-270d-4138-ab79-fcdd239b3997" containerID="0ef7cd09cfd4cf1e5792656dab2c9785fb2b5b3445c4f04f4ff27b9d93363509" exitCode=0 Dec 03 12:40:05 crc kubenswrapper[5052]: I1203 12:40:05.616475 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"95a06af5-270d-4138-ab79-fcdd239b3997","Type":"ContainerDied","Data":"0ef7cd09cfd4cf1e5792656dab2c9785fb2b5b3445c4f04f4ff27b9d93363509"} Dec 03 12:40:05 crc kubenswrapper[5052]: I1203 12:40:05.616534 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"95a06af5-270d-4138-ab79-fcdd239b3997","Type":"ContainerDied","Data":"f5a24a550711d5cd372d9ce54cc9aec579be9f6116ef3c3e698d6afc2b1c85b4"} Dec 03 12:40:05 crc kubenswrapper[5052]: I1203 12:40:05.616553 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f5a24a550711d5cd372d9ce54cc9aec579be9f6116ef3c3e698d6afc2b1c85b4" Dec 03 12:40:05 crc kubenswrapper[5052]: I1203 12:40:05.643235 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/swift-proxy-bf56d5474-cq7b2" podStartSLOduration=2.643213444 podStartE2EDuration="2.643213444s" podCreationTimestamp="2025-12-03 12:40:03 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:40:05.628975926 +0000 UTC m=+4796.217217690" watchObservedRunningTime="2025-12-03 12:40:05.643213444 +0000 UTC m=+4796.231455208" Dec 03 12:40:05 crc kubenswrapper[5052]: I1203 12:40:05.673999 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:40:05 crc kubenswrapper[5052]: I1203 12:40:05.874194 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7z7v5\" (UniqueName: \"kubernetes.io/projected/95a06af5-270d-4138-ab79-fcdd239b3997-kube-api-access-7z7v5\") pod \"95a06af5-270d-4138-ab79-fcdd239b3997\" (UID: \"95a06af5-270d-4138-ab79-fcdd239b3997\") " Dec 03 12:40:05 crc kubenswrapper[5052]: I1203 12:40:05.874328 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/95a06af5-270d-4138-ab79-fcdd239b3997-log-httpd\") pod \"95a06af5-270d-4138-ab79-fcdd239b3997\" (UID: \"95a06af5-270d-4138-ab79-fcdd239b3997\") " Dec 03 12:40:05 crc kubenswrapper[5052]: I1203 12:40:05.874429 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/95a06af5-270d-4138-ab79-fcdd239b3997-sg-core-conf-yaml\") pod \"95a06af5-270d-4138-ab79-fcdd239b3997\" (UID: \"95a06af5-270d-4138-ab79-fcdd239b3997\") " Dec 03 12:40:05 crc kubenswrapper[5052]: I1203 12:40:05.874478 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/95a06af5-270d-4138-ab79-fcdd239b3997-run-httpd\") pod \"95a06af5-270d-4138-ab79-fcdd239b3997\" (UID: \"95a06af5-270d-4138-ab79-fcdd239b3997\") " Dec 03 12:40:05 crc kubenswrapper[5052]: I1203 12:40:05.874543 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/95a06af5-270d-4138-ab79-fcdd239b3997-combined-ca-bundle\") pod \"95a06af5-270d-4138-ab79-fcdd239b3997\" (UID: \"95a06af5-270d-4138-ab79-fcdd239b3997\") " Dec 03 12:40:05 crc kubenswrapper[5052]: I1203 12:40:05.874565 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/95a06af5-270d-4138-ab79-fcdd239b3997-ceilometer-tls-certs\") pod \"95a06af5-270d-4138-ab79-fcdd239b3997\" (UID: \"95a06af5-270d-4138-ab79-fcdd239b3997\") " Dec 03 12:40:05 crc kubenswrapper[5052]: I1203 12:40:05.874623 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/95a06af5-270d-4138-ab79-fcdd239b3997-scripts\") pod \"95a06af5-270d-4138-ab79-fcdd239b3997\" (UID: \"95a06af5-270d-4138-ab79-fcdd239b3997\") " Dec 03 12:40:05 crc kubenswrapper[5052]: I1203 12:40:05.874701 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/95a06af5-270d-4138-ab79-fcdd239b3997-config-data\") pod \"95a06af5-270d-4138-ab79-fcdd239b3997\" (UID: \"95a06af5-270d-4138-ab79-fcdd239b3997\") " Dec 03 12:40:05 crc kubenswrapper[5052]: I1203 12:40:05.874828 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/95a06af5-270d-4138-ab79-fcdd239b3997-log-httpd" (OuterVolumeSpecName: "log-httpd") pod "95a06af5-270d-4138-ab79-fcdd239b3997" (UID: "95a06af5-270d-4138-ab79-fcdd239b3997"). InnerVolumeSpecName "log-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:40:05 crc kubenswrapper[5052]: I1203 12:40:05.874848 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/95a06af5-270d-4138-ab79-fcdd239b3997-run-httpd" (OuterVolumeSpecName: "run-httpd") pod "95a06af5-270d-4138-ab79-fcdd239b3997" (UID: "95a06af5-270d-4138-ab79-fcdd239b3997"). InnerVolumeSpecName "run-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:40:05 crc kubenswrapper[5052]: I1203 12:40:05.875276 5052 reconciler_common.go:293] "Volume detached for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/95a06af5-270d-4138-ab79-fcdd239b3997-log-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:40:05 crc kubenswrapper[5052]: I1203 12:40:05.875303 5052 reconciler_common.go:293] "Volume detached for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/95a06af5-270d-4138-ab79-fcdd239b3997-run-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:40:05 crc kubenswrapper[5052]: I1203 12:40:05.880467 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/95a06af5-270d-4138-ab79-fcdd239b3997-kube-api-access-7z7v5" (OuterVolumeSpecName: "kube-api-access-7z7v5") pod "95a06af5-270d-4138-ab79-fcdd239b3997" (UID: "95a06af5-270d-4138-ab79-fcdd239b3997"). InnerVolumeSpecName "kube-api-access-7z7v5". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:40:05 crc kubenswrapper[5052]: I1203 12:40:05.903654 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/95a06af5-270d-4138-ab79-fcdd239b3997-scripts" (OuterVolumeSpecName: "scripts") pod "95a06af5-270d-4138-ab79-fcdd239b3997" (UID: "95a06af5-270d-4138-ab79-fcdd239b3997"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:40:05 crc kubenswrapper[5052]: I1203 12:40:05.933721 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/95a06af5-270d-4138-ab79-fcdd239b3997-sg-core-conf-yaml" (OuterVolumeSpecName: "sg-core-conf-yaml") pod "95a06af5-270d-4138-ab79-fcdd239b3997" (UID: "95a06af5-270d-4138-ab79-fcdd239b3997"). InnerVolumeSpecName "sg-core-conf-yaml". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:40:05 crc kubenswrapper[5052]: I1203 12:40:05.978444 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-7z7v5\" (UniqueName: \"kubernetes.io/projected/95a06af5-270d-4138-ab79-fcdd239b3997-kube-api-access-7z7v5\") on node \"crc\" DevicePath \"\"" Dec 03 12:40:05 crc kubenswrapper[5052]: I1203 12:40:05.978482 5052 reconciler_common.go:293] "Volume detached for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/95a06af5-270d-4138-ab79-fcdd239b3997-sg-core-conf-yaml\") on node \"crc\" DevicePath \"\"" Dec 03 12:40:05 crc kubenswrapper[5052]: I1203 12:40:05.978496 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/95a06af5-270d-4138-ab79-fcdd239b3997-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:40:06 crc kubenswrapper[5052]: I1203 12:40:06.037698 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/95a06af5-270d-4138-ab79-fcdd239b3997-ceilometer-tls-certs" (OuterVolumeSpecName: "ceilometer-tls-certs") pod "95a06af5-270d-4138-ab79-fcdd239b3997" (UID: "95a06af5-270d-4138-ab79-fcdd239b3997"). InnerVolumeSpecName "ceilometer-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:40:06 crc kubenswrapper[5052]: I1203 12:40:06.063879 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/95a06af5-270d-4138-ab79-fcdd239b3997-config-data" (OuterVolumeSpecName: "config-data") pod "95a06af5-270d-4138-ab79-fcdd239b3997" (UID: "95a06af5-270d-4138-ab79-fcdd239b3997"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:40:06 crc kubenswrapper[5052]: I1203 12:40:06.073724 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/95a06af5-270d-4138-ab79-fcdd239b3997-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "95a06af5-270d-4138-ab79-fcdd239b3997" (UID: "95a06af5-270d-4138-ab79-fcdd239b3997"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:40:06 crc kubenswrapper[5052]: I1203 12:40:06.080102 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/95a06af5-270d-4138-ab79-fcdd239b3997-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:40:06 crc kubenswrapper[5052]: I1203 12:40:06.080137 5052 reconciler_common.go:293] "Volume detached for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/95a06af5-270d-4138-ab79-fcdd239b3997-ceilometer-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:40:06 crc kubenswrapper[5052]: I1203 12:40:06.080147 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/95a06af5-270d-4138-ab79-fcdd239b3997-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:40:06 crc kubenswrapper[5052]: I1203 12:40:06.625116 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:40:06 crc kubenswrapper[5052]: I1203 12:40:06.673157 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:40:06 crc kubenswrapper[5052]: I1203 12:40:06.700422 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:40:06 crc kubenswrapper[5052]: I1203 12:40:06.716365 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:40:06 crc kubenswrapper[5052]: E1203 12:40:06.717043 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="95a06af5-270d-4138-ab79-fcdd239b3997" containerName="ceilometer-central-agent" Dec 03 12:40:06 crc kubenswrapper[5052]: I1203 12:40:06.717069 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="95a06af5-270d-4138-ab79-fcdd239b3997" containerName="ceilometer-central-agent" Dec 03 12:40:06 crc kubenswrapper[5052]: E1203 12:40:06.717102 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="95a06af5-270d-4138-ab79-fcdd239b3997" containerName="sg-core" Dec 03 12:40:06 crc kubenswrapper[5052]: I1203 12:40:06.717110 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="95a06af5-270d-4138-ab79-fcdd239b3997" containerName="sg-core" Dec 03 12:40:06 crc kubenswrapper[5052]: E1203 12:40:06.717135 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="95a06af5-270d-4138-ab79-fcdd239b3997" containerName="ceilometer-notification-agent" Dec 03 12:40:06 crc kubenswrapper[5052]: I1203 12:40:06.717143 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="95a06af5-270d-4138-ab79-fcdd239b3997" containerName="ceilometer-notification-agent" Dec 03 12:40:06 crc kubenswrapper[5052]: E1203 12:40:06.717158 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="95a06af5-270d-4138-ab79-fcdd239b3997" containerName="proxy-httpd" Dec 03 12:40:06 crc kubenswrapper[5052]: I1203 12:40:06.717165 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="95a06af5-270d-4138-ab79-fcdd239b3997" containerName="proxy-httpd" Dec 03 12:40:06 crc kubenswrapper[5052]: I1203 12:40:06.717427 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="95a06af5-270d-4138-ab79-fcdd239b3997" containerName="ceilometer-central-agent" Dec 03 12:40:06 crc kubenswrapper[5052]: I1203 12:40:06.717454 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="95a06af5-270d-4138-ab79-fcdd239b3997" containerName="proxy-httpd" Dec 03 12:40:06 crc kubenswrapper[5052]: I1203 12:40:06.717477 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="95a06af5-270d-4138-ab79-fcdd239b3997" containerName="sg-core" Dec 03 12:40:06 crc kubenswrapper[5052]: I1203 12:40:06.717496 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="95a06af5-270d-4138-ab79-fcdd239b3997" containerName="ceilometer-notification-agent" Dec 03 12:40:06 crc kubenswrapper[5052]: I1203 12:40:06.719894 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:40:06 crc kubenswrapper[5052]: I1203 12:40:06.724765 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-config-data" Dec 03 12:40:06 crc kubenswrapper[5052]: I1203 12:40:06.724895 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-scripts" Dec 03 12:40:06 crc kubenswrapper[5052]: I1203 12:40:06.725034 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-ceilometer-internal-svc" Dec 03 12:40:06 crc kubenswrapper[5052]: I1203 12:40:06.729867 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:40:06 crc kubenswrapper[5052]: I1203 12:40:06.824083 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/65cb22ea-3a9c-4f99-9128-d3c7681c3173-run-httpd\") pod \"ceilometer-0\" (UID: \"65cb22ea-3a9c-4f99-9128-d3c7681c3173\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:40:06 crc kubenswrapper[5052]: I1203 12:40:06.824364 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/65cb22ea-3a9c-4f99-9128-d3c7681c3173-ceilometer-tls-certs\") pod \"ceilometer-0\" (UID: \"65cb22ea-3a9c-4f99-9128-d3c7681c3173\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:40:06 crc kubenswrapper[5052]: I1203 12:40:06.824429 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/65cb22ea-3a9c-4f99-9128-d3c7681c3173-log-httpd\") pod \"ceilometer-0\" (UID: \"65cb22ea-3a9c-4f99-9128-d3c7681c3173\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:40:06 crc kubenswrapper[5052]: I1203 12:40:06.824496 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bshs9\" (UniqueName: \"kubernetes.io/projected/65cb22ea-3a9c-4f99-9128-d3c7681c3173-kube-api-access-bshs9\") pod \"ceilometer-0\" (UID: \"65cb22ea-3a9c-4f99-9128-d3c7681c3173\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:40:06 crc kubenswrapper[5052]: I1203 12:40:06.824593 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/65cb22ea-3a9c-4f99-9128-d3c7681c3173-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"65cb22ea-3a9c-4f99-9128-d3c7681c3173\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:40:06 crc kubenswrapper[5052]: I1203 12:40:06.825116 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/65cb22ea-3a9c-4f99-9128-d3c7681c3173-config-data\") pod \"ceilometer-0\" (UID: \"65cb22ea-3a9c-4f99-9128-d3c7681c3173\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:40:06 crc kubenswrapper[5052]: I1203 12:40:06.825555 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/65cb22ea-3a9c-4f99-9128-d3c7681c3173-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"65cb22ea-3a9c-4f99-9128-d3c7681c3173\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:40:06 crc kubenswrapper[5052]: I1203 12:40:06.825667 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/65cb22ea-3a9c-4f99-9128-d3c7681c3173-scripts\") pod \"ceilometer-0\" (UID: \"65cb22ea-3a9c-4f99-9128-d3c7681c3173\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:40:06 crc kubenswrapper[5052]: I1203 12:40:06.927281 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/65cb22ea-3a9c-4f99-9128-d3c7681c3173-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"65cb22ea-3a9c-4f99-9128-d3c7681c3173\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:40:06 crc kubenswrapper[5052]: I1203 12:40:06.927587 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/65cb22ea-3a9c-4f99-9128-d3c7681c3173-scripts\") pod \"ceilometer-0\" (UID: \"65cb22ea-3a9c-4f99-9128-d3c7681c3173\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:40:06 crc kubenswrapper[5052]: I1203 12:40:06.927622 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/65cb22ea-3a9c-4f99-9128-d3c7681c3173-run-httpd\") pod \"ceilometer-0\" (UID: \"65cb22ea-3a9c-4f99-9128-d3c7681c3173\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:40:06 crc kubenswrapper[5052]: I1203 12:40:06.927681 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/65cb22ea-3a9c-4f99-9128-d3c7681c3173-ceilometer-tls-certs\") pod \"ceilometer-0\" (UID: \"65cb22ea-3a9c-4f99-9128-d3c7681c3173\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:40:06 crc kubenswrapper[5052]: I1203 12:40:06.927705 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/65cb22ea-3a9c-4f99-9128-d3c7681c3173-log-httpd\") pod \"ceilometer-0\" (UID: \"65cb22ea-3a9c-4f99-9128-d3c7681c3173\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:40:06 crc kubenswrapper[5052]: I1203 12:40:06.927746 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-bshs9\" (UniqueName: \"kubernetes.io/projected/65cb22ea-3a9c-4f99-9128-d3c7681c3173-kube-api-access-bshs9\") pod \"ceilometer-0\" (UID: \"65cb22ea-3a9c-4f99-9128-d3c7681c3173\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:40:06 crc kubenswrapper[5052]: I1203 12:40:06.927781 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/65cb22ea-3a9c-4f99-9128-d3c7681c3173-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"65cb22ea-3a9c-4f99-9128-d3c7681c3173\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:40:06 crc kubenswrapper[5052]: I1203 12:40:06.927828 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/65cb22ea-3a9c-4f99-9128-d3c7681c3173-config-data\") pod \"ceilometer-0\" (UID: \"65cb22ea-3a9c-4f99-9128-d3c7681c3173\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:40:06 crc kubenswrapper[5052]: I1203 12:40:06.928136 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/65cb22ea-3a9c-4f99-9128-d3c7681c3173-run-httpd\") pod \"ceilometer-0\" (UID: \"65cb22ea-3a9c-4f99-9128-d3c7681c3173\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:40:06 crc kubenswrapper[5052]: I1203 12:40:06.928397 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/65cb22ea-3a9c-4f99-9128-d3c7681c3173-log-httpd\") pod \"ceilometer-0\" (UID: \"65cb22ea-3a9c-4f99-9128-d3c7681c3173\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:40:06 crc kubenswrapper[5052]: I1203 12:40:06.932234 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/65cb22ea-3a9c-4f99-9128-d3c7681c3173-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"65cb22ea-3a9c-4f99-9128-d3c7681c3173\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:40:06 crc kubenswrapper[5052]: I1203 12:40:06.932343 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/65cb22ea-3a9c-4f99-9128-d3c7681c3173-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"65cb22ea-3a9c-4f99-9128-d3c7681c3173\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:40:06 crc kubenswrapper[5052]: I1203 12:40:06.932470 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/65cb22ea-3a9c-4f99-9128-d3c7681c3173-scripts\") pod \"ceilometer-0\" (UID: \"65cb22ea-3a9c-4f99-9128-d3c7681c3173\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:40:06 crc kubenswrapper[5052]: I1203 12:40:06.933493 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/65cb22ea-3a9c-4f99-9128-d3c7681c3173-config-data\") pod \"ceilometer-0\" (UID: \"65cb22ea-3a9c-4f99-9128-d3c7681c3173\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:40:06 crc kubenswrapper[5052]: I1203 12:40:06.934984 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/65cb22ea-3a9c-4f99-9128-d3c7681c3173-ceilometer-tls-certs\") pod \"ceilometer-0\" (UID: \"65cb22ea-3a9c-4f99-9128-d3c7681c3173\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:40:06 crc kubenswrapper[5052]: I1203 12:40:06.953226 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-bshs9\" (UniqueName: \"kubernetes.io/projected/65cb22ea-3a9c-4f99-9128-d3c7681c3173-kube-api-access-bshs9\") pod \"ceilometer-0\" (UID: \"65cb22ea-3a9c-4f99-9128-d3c7681c3173\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:40:06 crc kubenswrapper[5052]: I1203 12:40:06.963871 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/neutron-5d67d5bcdb-klgzj" podUID="4426a4f1-76e6-4746-a88a-4f4efead3977" containerName="neutron-httpd" probeResult="failure" output="Get \"https://10.217.1.53:9696/\": dial tcp 10.217.1.53:9696: connect: connection refused" Dec 03 12:40:07 crc kubenswrapper[5052]: I1203 12:40:07.040878 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:40:07 crc kubenswrapper[5052]: I1203 12:40:07.483387 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:40:07 crc kubenswrapper[5052]: I1203 12:40:07.636569 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"65cb22ea-3a9c-4f99-9128-d3c7681c3173","Type":"ContainerStarted","Data":"180a719e6c23d31790bc4ff445ea488d36a3051bd6b6243751d58b4950a06083"} Dec 03 12:40:07 crc kubenswrapper[5052]: I1203 12:40:07.686551 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="95a06af5-270d-4138-ab79-fcdd239b3997" path="/var/lib/kubelet/pods/95a06af5-270d-4138-ab79-fcdd239b3997/volumes" Dec 03 12:40:08 crc kubenswrapper[5052]: I1203 12:40:08.428489 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/placement-cdf4456f4-v82rz" Dec 03 12:40:08 crc kubenswrapper[5052]: I1203 12:40:08.429592 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/placement-cdf4456f4-v82rz" Dec 03 12:40:08 crc kubenswrapper[5052]: I1203 12:40:08.651174 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"65cb22ea-3a9c-4f99-9128-d3c7681c3173","Type":"ContainerStarted","Data":"d970c5d7874c5afe76962513bf4d336686b29e37d400962b8518fcae8ce933fc"} Dec 03 12:40:08 crc kubenswrapper[5052]: I1203 12:40:08.740610 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/placement-6cd995dc74-hpvzm"] Dec 03 12:40:08 crc kubenswrapper[5052]: I1203 12:40:08.741211 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/placement-6cd995dc74-hpvzm" podUID="38bfd61a-669e-4d69-b667-69270b5ec99d" containerName="placement-log" containerID="cri-o://473883e07ac4b625b4c47f00b264f64fef22cf82efbcd9bdb6eb7bc82576beb7" gracePeriod=30 Dec 03 12:40:08 crc kubenswrapper[5052]: I1203 12:40:08.741763 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/placement-6cd995dc74-hpvzm" podUID="38bfd61a-669e-4d69-b667-69270b5ec99d" containerName="placement-api" containerID="cri-o://54bc9dba691e3f9bc3fd33b6f2ed864845a91b45a88e3b273a06f1bc0b2bce85" gracePeriod=30 Dec 03 12:40:09 crc kubenswrapper[5052]: I1203 12:40:09.671540 5052 generic.go:334] "Generic (PLEG): container finished" podID="38bfd61a-669e-4d69-b667-69270b5ec99d" containerID="473883e07ac4b625b4c47f00b264f64fef22cf82efbcd9bdb6eb7bc82576beb7" exitCode=143 Dec 03 12:40:09 crc kubenswrapper[5052]: I1203 12:40:09.671610 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-6cd995dc74-hpvzm" event={"ID":"38bfd61a-669e-4d69-b667-69270b5ec99d","Type":"ContainerDied","Data":"473883e07ac4b625b4c47f00b264f64fef22cf82efbcd9bdb6eb7bc82576beb7"} Dec 03 12:40:10 crc kubenswrapper[5052]: I1203 12:40:10.685588 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"65cb22ea-3a9c-4f99-9128-d3c7681c3173","Type":"ContainerStarted","Data":"b7bf0dcb2de110437bf240dda6407da49398f97022270b268197a8605493c273"} Dec 03 12:40:11 crc kubenswrapper[5052]: I1203 12:40:11.711367 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"65cb22ea-3a9c-4f99-9128-d3c7681c3173","Type":"ContainerStarted","Data":"1bc0a87b8c7078c46986fb762763f8730d5895f38f884a95a89a15f1527106de"} Dec 03 12:40:11 crc kubenswrapper[5052]: I1203 12:40:11.868898 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/placement-6cd995dc74-hpvzm" podUID="38bfd61a-669e-4d69-b667-69270b5ec99d" containerName="placement-log" probeResult="failure" output="Get \"https://10.217.1.42:8778/\": read tcp 10.217.0.2:34428->10.217.1.42:8778: read: connection reset by peer" Dec 03 12:40:11 crc kubenswrapper[5052]: I1203 12:40:11.869150 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/placement-6cd995dc74-hpvzm" podUID="38bfd61a-669e-4d69-b667-69270b5ec99d" containerName="placement-api" probeResult="failure" output="Get \"https://10.217.1.42:8778/\": read tcp 10.217.0.2:34430->10.217.1.42:8778: read: connection reset by peer" Dec 03 12:40:12 crc kubenswrapper[5052]: I1203 12:40:12.717158 5052 generic.go:334] "Generic (PLEG): container finished" podID="38bfd61a-669e-4d69-b667-69270b5ec99d" containerID="54bc9dba691e3f9bc3fd33b6f2ed864845a91b45a88e3b273a06f1bc0b2bce85" exitCode=0 Dec 03 12:40:12 crc kubenswrapper[5052]: I1203 12:40:12.717218 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-6cd995dc74-hpvzm" event={"ID":"38bfd61a-669e-4d69-b667-69270b5ec99d","Type":"ContainerDied","Data":"54bc9dba691e3f9bc3fd33b6f2ed864845a91b45a88e3b273a06f1bc0b2bce85"} Dec 03 12:40:12 crc kubenswrapper[5052]: I1203 12:40:12.813307 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-6cd995dc74-hpvzm" Dec 03 12:40:12 crc kubenswrapper[5052]: I1203 12:40:12.864859 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/38bfd61a-669e-4d69-b667-69270b5ec99d-config-data\") pod \"38bfd61a-669e-4d69-b667-69270b5ec99d\" (UID: \"38bfd61a-669e-4d69-b667-69270b5ec99d\") " Dec 03 12:40:12 crc kubenswrapper[5052]: I1203 12:40:12.864939 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/38bfd61a-669e-4d69-b667-69270b5ec99d-combined-ca-bundle\") pod \"38bfd61a-669e-4d69-b667-69270b5ec99d\" (UID: \"38bfd61a-669e-4d69-b667-69270b5ec99d\") " Dec 03 12:40:12 crc kubenswrapper[5052]: I1203 12:40:12.864975 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/38bfd61a-669e-4d69-b667-69270b5ec99d-internal-tls-certs\") pod \"38bfd61a-669e-4d69-b667-69270b5ec99d\" (UID: \"38bfd61a-669e-4d69-b667-69270b5ec99d\") " Dec 03 12:40:12 crc kubenswrapper[5052]: I1203 12:40:12.865006 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nvhjx\" (UniqueName: \"kubernetes.io/projected/38bfd61a-669e-4d69-b667-69270b5ec99d-kube-api-access-nvhjx\") pod \"38bfd61a-669e-4d69-b667-69270b5ec99d\" (UID: \"38bfd61a-669e-4d69-b667-69270b5ec99d\") " Dec 03 12:40:12 crc kubenswrapper[5052]: I1203 12:40:12.865098 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/38bfd61a-669e-4d69-b667-69270b5ec99d-scripts\") pod \"38bfd61a-669e-4d69-b667-69270b5ec99d\" (UID: \"38bfd61a-669e-4d69-b667-69270b5ec99d\") " Dec 03 12:40:12 crc kubenswrapper[5052]: I1203 12:40:12.865154 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/38bfd61a-669e-4d69-b667-69270b5ec99d-public-tls-certs\") pod \"38bfd61a-669e-4d69-b667-69270b5ec99d\" (UID: \"38bfd61a-669e-4d69-b667-69270b5ec99d\") " Dec 03 12:40:12 crc kubenswrapper[5052]: I1203 12:40:12.865186 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/38bfd61a-669e-4d69-b667-69270b5ec99d-logs\") pod \"38bfd61a-669e-4d69-b667-69270b5ec99d\" (UID: \"38bfd61a-669e-4d69-b667-69270b5ec99d\") " Dec 03 12:40:12 crc kubenswrapper[5052]: I1203 12:40:12.867536 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/38bfd61a-669e-4d69-b667-69270b5ec99d-logs" (OuterVolumeSpecName: "logs") pod "38bfd61a-669e-4d69-b667-69270b5ec99d" (UID: "38bfd61a-669e-4d69-b667-69270b5ec99d"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:40:12 crc kubenswrapper[5052]: I1203 12:40:12.870922 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/38bfd61a-669e-4d69-b667-69270b5ec99d-kube-api-access-nvhjx" (OuterVolumeSpecName: "kube-api-access-nvhjx") pod "38bfd61a-669e-4d69-b667-69270b5ec99d" (UID: "38bfd61a-669e-4d69-b667-69270b5ec99d"). InnerVolumeSpecName "kube-api-access-nvhjx". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:40:12 crc kubenswrapper[5052]: I1203 12:40:12.872357 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/38bfd61a-669e-4d69-b667-69270b5ec99d-scripts" (OuterVolumeSpecName: "scripts") pod "38bfd61a-669e-4d69-b667-69270b5ec99d" (UID: "38bfd61a-669e-4d69-b667-69270b5ec99d"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:40:12 crc kubenswrapper[5052]: I1203 12:40:12.926275 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/38bfd61a-669e-4d69-b667-69270b5ec99d-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "38bfd61a-669e-4d69-b667-69270b5ec99d" (UID: "38bfd61a-669e-4d69-b667-69270b5ec99d"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:40:12 crc kubenswrapper[5052]: I1203 12:40:12.929070 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/38bfd61a-669e-4d69-b667-69270b5ec99d-config-data" (OuterVolumeSpecName: "config-data") pod "38bfd61a-669e-4d69-b667-69270b5ec99d" (UID: "38bfd61a-669e-4d69-b667-69270b5ec99d"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:40:12 crc kubenswrapper[5052]: I1203 12:40:12.975678 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/38bfd61a-669e-4d69-b667-69270b5ec99d-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:40:12 crc kubenswrapper[5052]: I1203 12:40:12.979151 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/38bfd61a-669e-4d69-b667-69270b5ec99d-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:40:12 crc kubenswrapper[5052]: I1203 12:40:12.979172 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/38bfd61a-669e-4d69-b667-69270b5ec99d-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:40:12 crc kubenswrapper[5052]: I1203 12:40:12.979186 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/38bfd61a-669e-4d69-b667-69270b5ec99d-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:40:12 crc kubenswrapper[5052]: I1203 12:40:12.979200 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-nvhjx\" (UniqueName: \"kubernetes.io/projected/38bfd61a-669e-4d69-b667-69270b5ec99d-kube-api-access-nvhjx\") on node \"crc\" DevicePath \"\"" Dec 03 12:40:12 crc kubenswrapper[5052]: I1203 12:40:12.975995 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/38bfd61a-669e-4d69-b667-69270b5ec99d-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "38bfd61a-669e-4d69-b667-69270b5ec99d" (UID: "38bfd61a-669e-4d69-b667-69270b5ec99d"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:40:13 crc kubenswrapper[5052]: I1203 12:40:13.004412 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/38bfd61a-669e-4d69-b667-69270b5ec99d-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "38bfd61a-669e-4d69-b667-69270b5ec99d" (UID: "38bfd61a-669e-4d69-b667-69270b5ec99d"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:40:13 crc kubenswrapper[5052]: I1203 12:40:13.080893 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/38bfd61a-669e-4d69-b667-69270b5ec99d-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:40:13 crc kubenswrapper[5052]: I1203 12:40:13.080927 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/38bfd61a-669e-4d69-b667-69270b5ec99d-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:40:13 crc kubenswrapper[5052]: I1203 12:40:13.586620 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/swift-proxy-bf56d5474-cq7b2" Dec 03 12:40:13 crc kubenswrapper[5052]: I1203 12:40:13.589308 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/swift-proxy-bf56d5474-cq7b2" Dec 03 12:40:13 crc kubenswrapper[5052]: I1203 12:40:13.692470 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/swift-proxy-55f4df7cd-4gjcp"] Dec 03 12:40:13 crc kubenswrapper[5052]: I1203 12:40:13.692790 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-proxy-55f4df7cd-4gjcp" podUID="06b2a08b-6b91-44f2-b630-169938fe5771" containerName="proxy-httpd" containerID="cri-o://cf437da4e9616b7fcbb31740024db41704bd4de4a657e7d26a4bdbbb2a642eb7" gracePeriod=30 Dec 03 12:40:13 crc kubenswrapper[5052]: I1203 12:40:13.692843 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-proxy-55f4df7cd-4gjcp" podUID="06b2a08b-6b91-44f2-b630-169938fe5771" containerName="proxy-server" containerID="cri-o://d08bfd89e72c8e8b387d02b98de513fbe980b80423758f0f559173df742846d9" gracePeriod=30 Dec 03 12:40:13 crc kubenswrapper[5052]: I1203 12:40:13.731336 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-6cd995dc74-hpvzm" event={"ID":"38bfd61a-669e-4d69-b667-69270b5ec99d","Type":"ContainerDied","Data":"6035e3dec571cc358328664f1d0d08e0f5e176cb17551bba680b7f05828dd099"} Dec 03 12:40:13 crc kubenswrapper[5052]: I1203 12:40:13.731405 5052 scope.go:117] "RemoveContainer" containerID="54bc9dba691e3f9bc3fd33b6f2ed864845a91b45a88e3b273a06f1bc0b2bce85" Dec 03 12:40:13 crc kubenswrapper[5052]: I1203 12:40:13.731408 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-6cd995dc74-hpvzm" Dec 03 12:40:13 crc kubenswrapper[5052]: I1203 12:40:13.735999 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"65cb22ea-3a9c-4f99-9128-d3c7681c3173","Type":"ContainerStarted","Data":"f0b7b9f45f694a8501baf61dbded87068f05ef0463c5c28e5e6db8782cc546cc"} Dec 03 12:40:13 crc kubenswrapper[5052]: I1203 12:40:13.736133 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:40:13 crc kubenswrapper[5052]: I1203 12:40:13.781785 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/ceilometer-0" podStartSLOduration=2.6641661880000003 podStartE2EDuration="7.781764192s" podCreationTimestamp="2025-12-03 12:40:06 +0000 UTC" firstStartedPulling="2025-12-03 12:40:07.485794779 +0000 UTC m=+4798.074036543" lastFinishedPulling="2025-12-03 12:40:12.603392783 +0000 UTC m=+4803.191634547" observedRunningTime="2025-12-03 12:40:13.75792963 +0000 UTC m=+4804.346171394" watchObservedRunningTime="2025-12-03 12:40:13.781764192 +0000 UTC m=+4804.370005956" Dec 03 12:40:13 crc kubenswrapper[5052]: I1203 12:40:13.794567 5052 scope.go:117] "RemoveContainer" containerID="473883e07ac4b625b4c47f00b264f64fef22cf82efbcd9bdb6eb7bc82576beb7" Dec 03 12:40:13 crc kubenswrapper[5052]: I1203 12:40:13.832785 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/placement-6cd995dc74-hpvzm"] Dec 03 12:40:13 crc kubenswrapper[5052]: I1203 12:40:13.842298 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/placement-6cd995dc74-hpvzm"] Dec 03 12:40:14 crc kubenswrapper[5052]: I1203 12:40:14.752490 5052 generic.go:334] "Generic (PLEG): container finished" podID="06b2a08b-6b91-44f2-b630-169938fe5771" containerID="d08bfd89e72c8e8b387d02b98de513fbe980b80423758f0f559173df742846d9" exitCode=0 Dec 03 12:40:14 crc kubenswrapper[5052]: I1203 12:40:14.752827 5052 generic.go:334] "Generic (PLEG): container finished" podID="06b2a08b-6b91-44f2-b630-169938fe5771" containerID="cf437da4e9616b7fcbb31740024db41704bd4de4a657e7d26a4bdbbb2a642eb7" exitCode=0 Dec 03 12:40:14 crc kubenswrapper[5052]: I1203 12:40:14.752550 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-proxy-55f4df7cd-4gjcp" event={"ID":"06b2a08b-6b91-44f2-b630-169938fe5771","Type":"ContainerDied","Data":"d08bfd89e72c8e8b387d02b98de513fbe980b80423758f0f559173df742846d9"} Dec 03 12:40:14 crc kubenswrapper[5052]: I1203 12:40:14.752898 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-proxy-55f4df7cd-4gjcp" event={"ID":"06b2a08b-6b91-44f2-b630-169938fe5771","Type":"ContainerDied","Data":"cf437da4e9616b7fcbb31740024db41704bd4de4a657e7d26a4bdbbb2a642eb7"} Dec 03 12:40:14 crc kubenswrapper[5052]: I1203 12:40:14.931183 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-proxy-55f4df7cd-4gjcp" Dec 03 12:40:15 crc kubenswrapper[5052]: I1203 12:40:15.021240 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/06b2a08b-6b91-44f2-b630-169938fe5771-run-httpd\") pod \"06b2a08b-6b91-44f2-b630-169938fe5771\" (UID: \"06b2a08b-6b91-44f2-b630-169938fe5771\") " Dec 03 12:40:15 crc kubenswrapper[5052]: I1203 12:40:15.021330 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/06b2a08b-6b91-44f2-b630-169938fe5771-public-tls-certs\") pod \"06b2a08b-6b91-44f2-b630-169938fe5771\" (UID: \"06b2a08b-6b91-44f2-b630-169938fe5771\") " Dec 03 12:40:15 crc kubenswrapper[5052]: I1203 12:40:15.021395 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/06b2a08b-6b91-44f2-b630-169938fe5771-combined-ca-bundle\") pod \"06b2a08b-6b91-44f2-b630-169938fe5771\" (UID: \"06b2a08b-6b91-44f2-b630-169938fe5771\") " Dec 03 12:40:15 crc kubenswrapper[5052]: I1203 12:40:15.021568 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/06b2a08b-6b91-44f2-b630-169938fe5771-etc-swift\") pod \"06b2a08b-6b91-44f2-b630-169938fe5771\" (UID: \"06b2a08b-6b91-44f2-b630-169938fe5771\") " Dec 03 12:40:15 crc kubenswrapper[5052]: I1203 12:40:15.021609 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ndmt6\" (UniqueName: \"kubernetes.io/projected/06b2a08b-6b91-44f2-b630-169938fe5771-kube-api-access-ndmt6\") pod \"06b2a08b-6b91-44f2-b630-169938fe5771\" (UID: \"06b2a08b-6b91-44f2-b630-169938fe5771\") " Dec 03 12:40:15 crc kubenswrapper[5052]: I1203 12:40:15.021677 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/06b2a08b-6b91-44f2-b630-169938fe5771-internal-tls-certs\") pod \"06b2a08b-6b91-44f2-b630-169938fe5771\" (UID: \"06b2a08b-6b91-44f2-b630-169938fe5771\") " Dec 03 12:40:15 crc kubenswrapper[5052]: I1203 12:40:15.021708 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/06b2a08b-6b91-44f2-b630-169938fe5771-log-httpd\") pod \"06b2a08b-6b91-44f2-b630-169938fe5771\" (UID: \"06b2a08b-6b91-44f2-b630-169938fe5771\") " Dec 03 12:40:15 crc kubenswrapper[5052]: I1203 12:40:15.021743 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/06b2a08b-6b91-44f2-b630-169938fe5771-config-data\") pod \"06b2a08b-6b91-44f2-b630-169938fe5771\" (UID: \"06b2a08b-6b91-44f2-b630-169938fe5771\") " Dec 03 12:40:15 crc kubenswrapper[5052]: I1203 12:40:15.022473 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/06b2a08b-6b91-44f2-b630-169938fe5771-run-httpd" (OuterVolumeSpecName: "run-httpd") pod "06b2a08b-6b91-44f2-b630-169938fe5771" (UID: "06b2a08b-6b91-44f2-b630-169938fe5771"). InnerVolumeSpecName "run-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:40:15 crc kubenswrapper[5052]: I1203 12:40:15.022565 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/06b2a08b-6b91-44f2-b630-169938fe5771-log-httpd" (OuterVolumeSpecName: "log-httpd") pod "06b2a08b-6b91-44f2-b630-169938fe5771" (UID: "06b2a08b-6b91-44f2-b630-169938fe5771"). InnerVolumeSpecName "log-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:40:15 crc kubenswrapper[5052]: I1203 12:40:15.030574 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/06b2a08b-6b91-44f2-b630-169938fe5771-kube-api-access-ndmt6" (OuterVolumeSpecName: "kube-api-access-ndmt6") pod "06b2a08b-6b91-44f2-b630-169938fe5771" (UID: "06b2a08b-6b91-44f2-b630-169938fe5771"). InnerVolumeSpecName "kube-api-access-ndmt6". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:40:15 crc kubenswrapper[5052]: I1203 12:40:15.046530 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/06b2a08b-6b91-44f2-b630-169938fe5771-etc-swift" (OuterVolumeSpecName: "etc-swift") pod "06b2a08b-6b91-44f2-b630-169938fe5771" (UID: "06b2a08b-6b91-44f2-b630-169938fe5771"). InnerVolumeSpecName "etc-swift". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:40:15 crc kubenswrapper[5052]: I1203 12:40:15.093889 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/06b2a08b-6b91-44f2-b630-169938fe5771-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "06b2a08b-6b91-44f2-b630-169938fe5771" (UID: "06b2a08b-6b91-44f2-b630-169938fe5771"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:40:15 crc kubenswrapper[5052]: I1203 12:40:15.103903 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/06b2a08b-6b91-44f2-b630-169938fe5771-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "06b2a08b-6b91-44f2-b630-169938fe5771" (UID: "06b2a08b-6b91-44f2-b630-169938fe5771"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:40:15 crc kubenswrapper[5052]: I1203 12:40:15.112013 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/06b2a08b-6b91-44f2-b630-169938fe5771-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "06b2a08b-6b91-44f2-b630-169938fe5771" (UID: "06b2a08b-6b91-44f2-b630-169938fe5771"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:40:15 crc kubenswrapper[5052]: I1203 12:40:15.116220 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/06b2a08b-6b91-44f2-b630-169938fe5771-config-data" (OuterVolumeSpecName: "config-data") pod "06b2a08b-6b91-44f2-b630-169938fe5771" (UID: "06b2a08b-6b91-44f2-b630-169938fe5771"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:40:15 crc kubenswrapper[5052]: I1203 12:40:15.123688 5052 reconciler_common.go:293] "Volume detached for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/06b2a08b-6b91-44f2-b630-169938fe5771-run-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:40:15 crc kubenswrapper[5052]: I1203 12:40:15.123720 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/06b2a08b-6b91-44f2-b630-169938fe5771-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:40:15 crc kubenswrapper[5052]: I1203 12:40:15.123732 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/06b2a08b-6b91-44f2-b630-169938fe5771-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:40:15 crc kubenswrapper[5052]: I1203 12:40:15.123740 5052 reconciler_common.go:293] "Volume detached for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/06b2a08b-6b91-44f2-b630-169938fe5771-etc-swift\") on node \"crc\" DevicePath \"\"" Dec 03 12:40:15 crc kubenswrapper[5052]: I1203 12:40:15.123750 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-ndmt6\" (UniqueName: \"kubernetes.io/projected/06b2a08b-6b91-44f2-b630-169938fe5771-kube-api-access-ndmt6\") on node \"crc\" DevicePath \"\"" Dec 03 12:40:15 crc kubenswrapper[5052]: I1203 12:40:15.123761 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/06b2a08b-6b91-44f2-b630-169938fe5771-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:40:15 crc kubenswrapper[5052]: I1203 12:40:15.123770 5052 reconciler_common.go:293] "Volume detached for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/06b2a08b-6b91-44f2-b630-169938fe5771-log-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:40:15 crc kubenswrapper[5052]: I1203 12:40:15.123781 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/06b2a08b-6b91-44f2-b630-169938fe5771-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:40:15 crc kubenswrapper[5052]: I1203 12:40:15.687020 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="38bfd61a-669e-4d69-b667-69270b5ec99d" path="/var/lib/kubelet/pods/38bfd61a-669e-4d69-b667-69270b5ec99d/volumes" Dec 03 12:40:15 crc kubenswrapper[5052]: I1203 12:40:15.783405 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-proxy-55f4df7cd-4gjcp" event={"ID":"06b2a08b-6b91-44f2-b630-169938fe5771","Type":"ContainerDied","Data":"a3501750ee3c86ae7c02bc63bcbfc468c8155e083b2729cd978109b695842fe9"} Dec 03 12:40:15 crc kubenswrapper[5052]: I1203 12:40:15.783446 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-proxy-55f4df7cd-4gjcp" Dec 03 12:40:15 crc kubenswrapper[5052]: I1203 12:40:15.783467 5052 scope.go:117] "RemoveContainer" containerID="d08bfd89e72c8e8b387d02b98de513fbe980b80423758f0f559173df742846d9" Dec 03 12:40:15 crc kubenswrapper[5052]: I1203 12:40:15.815942 5052 scope.go:117] "RemoveContainer" containerID="cf437da4e9616b7fcbb31740024db41704bd4de4a657e7d26a4bdbbb2a642eb7" Dec 03 12:40:15 crc kubenswrapper[5052]: I1203 12:40:15.820503 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/swift-proxy-55f4df7cd-4gjcp"] Dec 03 12:40:15 crc kubenswrapper[5052]: I1203 12:40:15.829405 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/swift-proxy-55f4df7cd-4gjcp"] Dec 03 12:40:17 crc kubenswrapper[5052]: I1203 12:40:17.686475 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="06b2a08b-6b91-44f2-b630-169938fe5771" path="/var/lib/kubelet/pods/06b2a08b-6b91-44f2-b630-169938fe5771/volumes" Dec 03 12:40:25 crc kubenswrapper[5052]: I1203 12:40:25.607156 5052 scope.go:117] "RemoveContainer" containerID="34697f0f6773a06387fe7e2f667823c92555f8d2c3bcc6f8942a7d5f2769f8ed" Dec 03 12:40:25 crc kubenswrapper[5052]: I1203 12:40:25.635110 5052 scope.go:117] "RemoveContainer" containerID="940708945e467089d948362e1fc3b34404c0c0c4806d349291101b66943d6391" Dec 03 12:40:29 crc kubenswrapper[5052]: I1203 12:40:29.353614 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_neutron-5d67d5bcdb-klgzj_4426a4f1-76e6-4746-a88a-4f4efead3977/neutron-api/0.log" Dec 03 12:40:29 crc kubenswrapper[5052]: I1203 12:40:29.354139 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-5d67d5bcdb-klgzj" Dec 03 12:40:29 crc kubenswrapper[5052]: I1203 12:40:29.379596 5052 patch_prober.go:28] interesting pod/machine-config-daemon-74bj6 container/machine-config-daemon namespace/openshift-machine-config-operator: Liveness probe status=failure output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" start-of-body= Dec 03 12:40:29 crc kubenswrapper[5052]: I1203 12:40:29.379659 5052 prober.go:107] "Probe failed" probeType="Liveness" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" containerName="machine-config-daemon" probeResult="failure" output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" Dec 03 12:40:29 crc kubenswrapper[5052]: I1203 12:40:29.379705 5052 kubelet.go:2542] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" Dec 03 12:40:29 crc kubenswrapper[5052]: I1203 12:40:29.380456 5052 kuberuntime_manager.go:1027] "Message for Container of pod" containerName="machine-config-daemon" containerStatusID={"Type":"cri-o","ID":"bccf12735e965b0db2889952a34202c267bb425fd32c44f3ef490bba9c0424d8"} pod="openshift-machine-config-operator/machine-config-daemon-74bj6" containerMessage="Container machine-config-daemon failed liveness probe, will be restarted" Dec 03 12:40:29 crc kubenswrapper[5052]: I1203 12:40:29.380531 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" containerName="machine-config-daemon" containerID="cri-o://bccf12735e965b0db2889952a34202c267bb425fd32c44f3ef490bba9c0424d8" gracePeriod=600 Dec 03 12:40:29 crc kubenswrapper[5052]: I1203 12:40:29.502832 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/4426a4f1-76e6-4746-a88a-4f4efead3977-httpd-config\") pod \"4426a4f1-76e6-4746-a88a-4f4efead3977\" (UID: \"4426a4f1-76e6-4746-a88a-4f4efead3977\") " Dec 03 12:40:29 crc kubenswrapper[5052]: I1203 12:40:29.502881 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/4426a4f1-76e6-4746-a88a-4f4efead3977-config\") pod \"4426a4f1-76e6-4746-a88a-4f4efead3977\" (UID: \"4426a4f1-76e6-4746-a88a-4f4efead3977\") " Dec 03 12:40:29 crc kubenswrapper[5052]: I1203 12:40:29.502945 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/4426a4f1-76e6-4746-a88a-4f4efead3977-internal-tls-certs\") pod \"4426a4f1-76e6-4746-a88a-4f4efead3977\" (UID: \"4426a4f1-76e6-4746-a88a-4f4efead3977\") " Dec 03 12:40:29 crc kubenswrapper[5052]: I1203 12:40:29.502996 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-slhbg\" (UniqueName: \"kubernetes.io/projected/4426a4f1-76e6-4746-a88a-4f4efead3977-kube-api-access-slhbg\") pod \"4426a4f1-76e6-4746-a88a-4f4efead3977\" (UID: \"4426a4f1-76e6-4746-a88a-4f4efead3977\") " Dec 03 12:40:29 crc kubenswrapper[5052]: I1203 12:40:29.503112 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4426a4f1-76e6-4746-a88a-4f4efead3977-combined-ca-bundle\") pod \"4426a4f1-76e6-4746-a88a-4f4efead3977\" (UID: \"4426a4f1-76e6-4746-a88a-4f4efead3977\") " Dec 03 12:40:29 crc kubenswrapper[5052]: I1203 12:40:29.503193 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/4426a4f1-76e6-4746-a88a-4f4efead3977-ovndb-tls-certs\") pod \"4426a4f1-76e6-4746-a88a-4f4efead3977\" (UID: \"4426a4f1-76e6-4746-a88a-4f4efead3977\") " Dec 03 12:40:29 crc kubenswrapper[5052]: I1203 12:40:29.503315 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/4426a4f1-76e6-4746-a88a-4f4efead3977-public-tls-certs\") pod \"4426a4f1-76e6-4746-a88a-4f4efead3977\" (UID: \"4426a4f1-76e6-4746-a88a-4f4efead3977\") " Dec 03 12:40:29 crc kubenswrapper[5052]: I1203 12:40:29.510060 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4426a4f1-76e6-4746-a88a-4f4efead3977-kube-api-access-slhbg" (OuterVolumeSpecName: "kube-api-access-slhbg") pod "4426a4f1-76e6-4746-a88a-4f4efead3977" (UID: "4426a4f1-76e6-4746-a88a-4f4efead3977"). InnerVolumeSpecName "kube-api-access-slhbg". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:40:29 crc kubenswrapper[5052]: I1203 12:40:29.510297 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4426a4f1-76e6-4746-a88a-4f4efead3977-httpd-config" (OuterVolumeSpecName: "httpd-config") pod "4426a4f1-76e6-4746-a88a-4f4efead3977" (UID: "4426a4f1-76e6-4746-a88a-4f4efead3977"). InnerVolumeSpecName "httpd-config". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:40:29 crc kubenswrapper[5052]: I1203 12:40:29.564138 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4426a4f1-76e6-4746-a88a-4f4efead3977-config" (OuterVolumeSpecName: "config") pod "4426a4f1-76e6-4746-a88a-4f4efead3977" (UID: "4426a4f1-76e6-4746-a88a-4f4efead3977"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:40:29 crc kubenswrapper[5052]: I1203 12:40:29.573272 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4426a4f1-76e6-4746-a88a-4f4efead3977-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "4426a4f1-76e6-4746-a88a-4f4efead3977" (UID: "4426a4f1-76e6-4746-a88a-4f4efead3977"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:40:29 crc kubenswrapper[5052]: I1203 12:40:29.577753 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4426a4f1-76e6-4746-a88a-4f4efead3977-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "4426a4f1-76e6-4746-a88a-4f4efead3977" (UID: "4426a4f1-76e6-4746-a88a-4f4efead3977"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:40:29 crc kubenswrapper[5052]: I1203 12:40:29.585411 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4426a4f1-76e6-4746-a88a-4f4efead3977-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "4426a4f1-76e6-4746-a88a-4f4efead3977" (UID: "4426a4f1-76e6-4746-a88a-4f4efead3977"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:40:29 crc kubenswrapper[5052]: I1203 12:40:29.605344 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4426a4f1-76e6-4746-a88a-4f4efead3977-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:40:29 crc kubenswrapper[5052]: I1203 12:40:29.605380 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/4426a4f1-76e6-4746-a88a-4f4efead3977-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:40:29 crc kubenswrapper[5052]: I1203 12:40:29.605396 5052 reconciler_common.go:293] "Volume detached for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/4426a4f1-76e6-4746-a88a-4f4efead3977-httpd-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:40:29 crc kubenswrapper[5052]: I1203 12:40:29.605406 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/4426a4f1-76e6-4746-a88a-4f4efead3977-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:40:29 crc kubenswrapper[5052]: I1203 12:40:29.605416 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/4426a4f1-76e6-4746-a88a-4f4efead3977-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:40:29 crc kubenswrapper[5052]: I1203 12:40:29.605425 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-slhbg\" (UniqueName: \"kubernetes.io/projected/4426a4f1-76e6-4746-a88a-4f4efead3977-kube-api-access-slhbg\") on node \"crc\" DevicePath \"\"" Dec 03 12:40:29 crc kubenswrapper[5052]: I1203 12:40:29.619471 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4426a4f1-76e6-4746-a88a-4f4efead3977-ovndb-tls-certs" (OuterVolumeSpecName: "ovndb-tls-certs") pod "4426a4f1-76e6-4746-a88a-4f4efead3977" (UID: "4426a4f1-76e6-4746-a88a-4f4efead3977"). InnerVolumeSpecName "ovndb-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:40:29 crc kubenswrapper[5052]: I1203 12:40:29.707360 5052 reconciler_common.go:293] "Volume detached for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/4426a4f1-76e6-4746-a88a-4f4efead3977-ovndb-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:40:29 crc kubenswrapper[5052]: I1203 12:40:29.916158 5052 generic.go:334] "Generic (PLEG): container finished" podID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" containerID="bccf12735e965b0db2889952a34202c267bb425fd32c44f3ef490bba9c0424d8" exitCode=0 Dec 03 12:40:29 crc kubenswrapper[5052]: I1203 12:40:29.916224 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" event={"ID":"10c237bc-72e7-4da3-9f2c-b1688395bf6e","Type":"ContainerDied","Data":"bccf12735e965b0db2889952a34202c267bb425fd32c44f3ef490bba9c0424d8"} Dec 03 12:40:29 crc kubenswrapper[5052]: I1203 12:40:29.916571 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" event={"ID":"10c237bc-72e7-4da3-9f2c-b1688395bf6e","Type":"ContainerStarted","Data":"2aed1c760a326f3d008799e902b1093313df3764a57c879f0b3686410127d37d"} Dec 03 12:40:29 crc kubenswrapper[5052]: I1203 12:40:29.916690 5052 scope.go:117] "RemoveContainer" containerID="e6edaf188f85ea4f11f487f11cc489c700f25fb42b5ee269f4f496f1e127d772" Dec 03 12:40:29 crc kubenswrapper[5052]: I1203 12:40:29.918314 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_neutron-5d67d5bcdb-klgzj_4426a4f1-76e6-4746-a88a-4f4efead3977/neutron-api/0.log" Dec 03 12:40:29 crc kubenswrapper[5052]: I1203 12:40:29.918425 5052 generic.go:334] "Generic (PLEG): container finished" podID="4426a4f1-76e6-4746-a88a-4f4efead3977" containerID="dde274eb8e16ff85ae1b2ee940add419f52d4e53a71e9f14e7931dcb8897d9fe" exitCode=137 Dec 03 12:40:29 crc kubenswrapper[5052]: I1203 12:40:29.918551 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-5d67d5bcdb-klgzj" event={"ID":"4426a4f1-76e6-4746-a88a-4f4efead3977","Type":"ContainerDied","Data":"dde274eb8e16ff85ae1b2ee940add419f52d4e53a71e9f14e7931dcb8897d9fe"} Dec 03 12:40:29 crc kubenswrapper[5052]: I1203 12:40:29.918670 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-5d67d5bcdb-klgzj" Dec 03 12:40:29 crc kubenswrapper[5052]: I1203 12:40:29.918678 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-5d67d5bcdb-klgzj" event={"ID":"4426a4f1-76e6-4746-a88a-4f4efead3977","Type":"ContainerDied","Data":"de9dd63521e94c20c668e631d3ac331c247e96c7bb5868ebc0db0547180dc83c"} Dec 03 12:40:29 crc kubenswrapper[5052]: I1203 12:40:29.956170 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/neutron-5d67d5bcdb-klgzj"] Dec 03 12:40:29 crc kubenswrapper[5052]: I1203 12:40:29.956606 5052 scope.go:117] "RemoveContainer" containerID="1a7463968b45581a86dc3faca6452efdbcfd78fb18a9ccb7360af588a2bd68a5" Dec 03 12:40:29 crc kubenswrapper[5052]: I1203 12:40:29.967293 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/neutron-5d67d5bcdb-klgzj"] Dec 03 12:40:29 crc kubenswrapper[5052]: I1203 12:40:29.979562 5052 scope.go:117] "RemoveContainer" containerID="dde274eb8e16ff85ae1b2ee940add419f52d4e53a71e9f14e7931dcb8897d9fe" Dec 03 12:40:30 crc kubenswrapper[5052]: I1203 12:40:29.999935 5052 scope.go:117] "RemoveContainer" containerID="1a7463968b45581a86dc3faca6452efdbcfd78fb18a9ccb7360af588a2bd68a5" Dec 03 12:40:30 crc kubenswrapper[5052]: E1203 12:40:30.001266 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1a7463968b45581a86dc3faca6452efdbcfd78fb18a9ccb7360af588a2bd68a5\": container with ID starting with 1a7463968b45581a86dc3faca6452efdbcfd78fb18a9ccb7360af588a2bd68a5 not found: ID does not exist" containerID="1a7463968b45581a86dc3faca6452efdbcfd78fb18a9ccb7360af588a2bd68a5" Dec 03 12:40:30 crc kubenswrapper[5052]: I1203 12:40:30.001310 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1a7463968b45581a86dc3faca6452efdbcfd78fb18a9ccb7360af588a2bd68a5"} err="failed to get container status \"1a7463968b45581a86dc3faca6452efdbcfd78fb18a9ccb7360af588a2bd68a5\": rpc error: code = NotFound desc = could not find container \"1a7463968b45581a86dc3faca6452efdbcfd78fb18a9ccb7360af588a2bd68a5\": container with ID starting with 1a7463968b45581a86dc3faca6452efdbcfd78fb18a9ccb7360af588a2bd68a5 not found: ID does not exist" Dec 03 12:40:30 crc kubenswrapper[5052]: I1203 12:40:30.001342 5052 scope.go:117] "RemoveContainer" containerID="dde274eb8e16ff85ae1b2ee940add419f52d4e53a71e9f14e7931dcb8897d9fe" Dec 03 12:40:30 crc kubenswrapper[5052]: E1203 12:40:30.001764 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dde274eb8e16ff85ae1b2ee940add419f52d4e53a71e9f14e7931dcb8897d9fe\": container with ID starting with dde274eb8e16ff85ae1b2ee940add419f52d4e53a71e9f14e7931dcb8897d9fe not found: ID does not exist" containerID="dde274eb8e16ff85ae1b2ee940add419f52d4e53a71e9f14e7931dcb8897d9fe" Dec 03 12:40:30 crc kubenswrapper[5052]: I1203 12:40:30.001800 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dde274eb8e16ff85ae1b2ee940add419f52d4e53a71e9f14e7931dcb8897d9fe"} err="failed to get container status \"dde274eb8e16ff85ae1b2ee940add419f52d4e53a71e9f14e7931dcb8897d9fe\": rpc error: code = NotFound desc = could not find container \"dde274eb8e16ff85ae1b2ee940add419f52d4e53a71e9f14e7931dcb8897d9fe\": container with ID starting with dde274eb8e16ff85ae1b2ee940add419f52d4e53a71e9f14e7931dcb8897d9fe not found: ID does not exist" Dec 03 12:40:31 crc kubenswrapper[5052]: I1203 12:40:31.684668 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4426a4f1-76e6-4746-a88a-4f4efead3977" path="/var/lib/kubelet/pods/4426a4f1-76e6-4746-a88a-4f4efead3977/volumes" Dec 03 12:40:37 crc kubenswrapper[5052]: I1203 12:40:37.049978 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:40:56 crc kubenswrapper[5052]: I1203 12:40:56.704593 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/rabbitmq-cell1-server-0"] Dec 03 12:40:56 crc kubenswrapper[5052]: E1203 12:40:56.761388 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/combined-ca-bundle: secret "combined-ca-bundle" not found Dec 03 12:40:56 crc kubenswrapper[5052]: E1203 12:40:56.761829 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0556e9f0-e431-4193-9822-6079cb8a1e44-combined-ca-bundle podName:0556e9f0-e431-4193-9822-6079cb8a1e44 nodeName:}" failed. No retries permitted until 2025-12-03 12:40:57.261803915 +0000 UTC m=+4847.850045769 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "combined-ca-bundle" (UniqueName: "kubernetes.io/secret/0556e9f0-e431-4193-9822-6079cb8a1e44-combined-ca-bundle") pod "nova-cell1-conductor-0" (UID: "0556e9f0-e431-4193-9822-6079cb8a1e44") : secret "combined-ca-bundle" not found Dec 03 12:40:56 crc kubenswrapper[5052]: E1203 12:40:56.865483 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/rabbitmq-cell1-config-data: configmap "rabbitmq-cell1-config-data" not found Dec 03 12:40:56 crc kubenswrapper[5052]: E1203 12:40:56.865557 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f9bac64c-518e-4630-8889-20e09d5b7a39-config-data podName:f9bac64c-518e-4630-8889-20e09d5b7a39 nodeName:}" failed. No retries permitted until 2025-12-03 12:40:57.365541011 +0000 UTC m=+4847.953782775 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "config-data" (UniqueName: "kubernetes.io/configmap/f9bac64c-518e-4630-8889-20e09d5b7a39-config-data") pod "rabbitmq-cell1-server-0" (UID: "f9bac64c-518e-4630-8889-20e09d5b7a39") : configmap "rabbitmq-cell1-config-data" not found Dec 03 12:40:56 crc kubenswrapper[5052]: I1203 12:40:56.918418 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/openstackclient"] Dec 03 12:40:56 crc kubenswrapper[5052]: I1203 12:40:56.918664 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/openstackclient" podUID="7ce91d4a-389d-4b37-b582-8c886566aa74" containerName="openstackclient" containerID="cri-o://229df7ab571e9eb2bf196a218d66569b5405729090282a855310baf493b87409" gracePeriod=2 Dec 03 12:40:56 crc kubenswrapper[5052]: I1203 12:40:56.927468 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/openstackclient"] Dec 03 12:40:56 crc kubenswrapper[5052]: I1203 12:40:56.936871 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/glancec94f-account-delete-8t6wx"] Dec 03 12:40:56 crc kubenswrapper[5052]: E1203 12:40:56.937273 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="4426a4f1-76e6-4746-a88a-4f4efead3977" containerName="neutron-api" Dec 03 12:40:56 crc kubenswrapper[5052]: I1203 12:40:56.937291 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="4426a4f1-76e6-4746-a88a-4f4efead3977" containerName="neutron-api" Dec 03 12:40:56 crc kubenswrapper[5052]: E1203 12:40:56.937318 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="38bfd61a-669e-4d69-b667-69270b5ec99d" containerName="placement-api" Dec 03 12:40:56 crc kubenswrapper[5052]: I1203 12:40:56.937326 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="38bfd61a-669e-4d69-b667-69270b5ec99d" containerName="placement-api" Dec 03 12:40:56 crc kubenswrapper[5052]: E1203 12:40:56.937337 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="06b2a08b-6b91-44f2-b630-169938fe5771" containerName="proxy-server" Dec 03 12:40:56 crc kubenswrapper[5052]: I1203 12:40:56.937345 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="06b2a08b-6b91-44f2-b630-169938fe5771" containerName="proxy-server" Dec 03 12:40:56 crc kubenswrapper[5052]: E1203 12:40:56.937359 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="4426a4f1-76e6-4746-a88a-4f4efead3977" containerName="neutron-httpd" Dec 03 12:40:56 crc kubenswrapper[5052]: I1203 12:40:56.937366 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="4426a4f1-76e6-4746-a88a-4f4efead3977" containerName="neutron-httpd" Dec 03 12:40:56 crc kubenswrapper[5052]: E1203 12:40:56.937378 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="38bfd61a-669e-4d69-b667-69270b5ec99d" containerName="placement-log" Dec 03 12:40:56 crc kubenswrapper[5052]: I1203 12:40:56.937384 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="38bfd61a-669e-4d69-b667-69270b5ec99d" containerName="placement-log" Dec 03 12:40:56 crc kubenswrapper[5052]: E1203 12:40:56.937398 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="06b2a08b-6b91-44f2-b630-169938fe5771" containerName="proxy-httpd" Dec 03 12:40:56 crc kubenswrapper[5052]: I1203 12:40:56.937403 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="06b2a08b-6b91-44f2-b630-169938fe5771" containerName="proxy-httpd" Dec 03 12:40:56 crc kubenswrapper[5052]: E1203 12:40:56.937416 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="7ce91d4a-389d-4b37-b582-8c886566aa74" containerName="openstackclient" Dec 03 12:40:56 crc kubenswrapper[5052]: I1203 12:40:56.937422 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="7ce91d4a-389d-4b37-b582-8c886566aa74" containerName="openstackclient" Dec 03 12:40:56 crc kubenswrapper[5052]: I1203 12:40:56.937653 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="38bfd61a-669e-4d69-b667-69270b5ec99d" containerName="placement-log" Dec 03 12:40:56 crc kubenswrapper[5052]: I1203 12:40:56.937665 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="4426a4f1-76e6-4746-a88a-4f4efead3977" containerName="neutron-httpd" Dec 03 12:40:56 crc kubenswrapper[5052]: I1203 12:40:56.937674 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="38bfd61a-669e-4d69-b667-69270b5ec99d" containerName="placement-api" Dec 03 12:40:56 crc kubenswrapper[5052]: I1203 12:40:56.937685 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="4426a4f1-76e6-4746-a88a-4f4efead3977" containerName="neutron-api" Dec 03 12:40:56 crc kubenswrapper[5052]: I1203 12:40:56.937693 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="06b2a08b-6b91-44f2-b630-169938fe5771" containerName="proxy-httpd" Dec 03 12:40:56 crc kubenswrapper[5052]: I1203 12:40:56.937711 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="7ce91d4a-389d-4b37-b582-8c886566aa74" containerName="openstackclient" Dec 03 12:40:56 crc kubenswrapper[5052]: I1203 12:40:56.937725 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="06b2a08b-6b91-44f2-b630-169938fe5771" containerName="proxy-server" Dec 03 12:40:56 crc kubenswrapper[5052]: I1203 12:40:56.938316 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glancec94f-account-delete-8t6wx" Dec 03 12:40:56 crc kubenswrapper[5052]: I1203 12:40:56.973959 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glancec94f-account-delete-8t6wx"] Dec 03 12:40:57 crc kubenswrapper[5052]: I1203 12:40:57.073196 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nf27q\" (UniqueName: \"kubernetes.io/projected/b7c49d90-c0af-4d4b-a080-8c7dce3ee66a-kube-api-access-nf27q\") pod \"glancec94f-account-delete-8t6wx\" (UID: \"b7c49d90-c0af-4d4b-a080-8c7dce3ee66a\") " pod="openstack-kuttl-tests/glancec94f-account-delete-8t6wx" Dec 03 12:40:57 crc kubenswrapper[5052]: I1203 12:40:57.073274 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/b7c49d90-c0af-4d4b-a080-8c7dce3ee66a-operator-scripts\") pod \"glancec94f-account-delete-8t6wx\" (UID: \"b7c49d90-c0af-4d4b-a080-8c7dce3ee66a\") " pod="openstack-kuttl-tests/glancec94f-account-delete-8t6wx" Dec 03 12:40:57 crc kubenswrapper[5052]: I1203 12:40:57.095120 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/rabbitmq-server-0"] Dec 03 12:40:57 crc kubenswrapper[5052]: I1203 12:40:57.170857 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/neutrond24e-account-delete-p2mpp"] Dec 03 12:40:57 crc kubenswrapper[5052]: I1203 12:40:57.172062 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutrond24e-account-delete-p2mpp" Dec 03 12:40:57 crc kubenswrapper[5052]: I1203 12:40:57.174771 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-nf27q\" (UniqueName: \"kubernetes.io/projected/b7c49d90-c0af-4d4b-a080-8c7dce3ee66a-kube-api-access-nf27q\") pod \"glancec94f-account-delete-8t6wx\" (UID: \"b7c49d90-c0af-4d4b-a080-8c7dce3ee66a\") " pod="openstack-kuttl-tests/glancec94f-account-delete-8t6wx" Dec 03 12:40:57 crc kubenswrapper[5052]: I1203 12:40:57.174835 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/b7c49d90-c0af-4d4b-a080-8c7dce3ee66a-operator-scripts\") pod \"glancec94f-account-delete-8t6wx\" (UID: \"b7c49d90-c0af-4d4b-a080-8c7dce3ee66a\") " pod="openstack-kuttl-tests/glancec94f-account-delete-8t6wx" Dec 03 12:40:57 crc kubenswrapper[5052]: I1203 12:40:57.175687 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/b7c49d90-c0af-4d4b-a080-8c7dce3ee66a-operator-scripts\") pod \"glancec94f-account-delete-8t6wx\" (UID: \"b7c49d90-c0af-4d4b-a080-8c7dce3ee66a\") " pod="openstack-kuttl-tests/glancec94f-account-delete-8t6wx" Dec 03 12:40:57 crc kubenswrapper[5052]: I1203 12:40:57.272344 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-nf27q\" (UniqueName: \"kubernetes.io/projected/b7c49d90-c0af-4d4b-a080-8c7dce3ee66a-kube-api-access-nf27q\") pod \"glancec94f-account-delete-8t6wx\" (UID: \"b7c49d90-c0af-4d4b-a080-8c7dce3ee66a\") " pod="openstack-kuttl-tests/glancec94f-account-delete-8t6wx" Dec 03 12:40:57 crc kubenswrapper[5052]: I1203 12:40:57.349754 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-db-sync-rxhl2"] Dec 03 12:40:57 crc kubenswrapper[5052]: E1203 12:40:57.392954 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/rabbitmq-config-data: configmap "rabbitmq-config-data" not found Dec 03 12:40:57 crc kubenswrapper[5052]: E1203 12:40:57.393005 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d82ef8ea-0c3b-4d89-9c78-22095940f4da-config-data podName:d82ef8ea-0c3b-4d89-9c78-22095940f4da nodeName:}" failed. No retries permitted until 2025-12-03 12:40:57.892989136 +0000 UTC m=+4848.481230900 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "config-data" (UniqueName: "kubernetes.io/configmap/d82ef8ea-0c3b-4d89-9c78-22095940f4da-config-data") pod "rabbitmq-server-0" (UID: "d82ef8ea-0c3b-4d89-9c78-22095940f4da") : configmap "rabbitmq-config-data" not found Dec 03 12:40:57 crc kubenswrapper[5052]: E1203 12:40:57.393854 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/combined-ca-bundle: secret "combined-ca-bundle" not found Dec 03 12:40:57 crc kubenswrapper[5052]: E1203 12:40:57.393911 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0556e9f0-e431-4193-9822-6079cb8a1e44-combined-ca-bundle podName:0556e9f0-e431-4193-9822-6079cb8a1e44 nodeName:}" failed. No retries permitted until 2025-12-03 12:40:58.393896632 +0000 UTC m=+4848.982138396 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "combined-ca-bundle" (UniqueName: "kubernetes.io/secret/0556e9f0-e431-4193-9822-6079cb8a1e44-combined-ca-bundle") pod "nova-cell1-conductor-0" (UID: "0556e9f0-e431-4193-9822-6079cb8a1e44") : secret "combined-ca-bundle" not found Dec 03 12:40:57 crc kubenswrapper[5052]: I1203 12:40:57.423846 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-nb-0"] Dec 03 12:40:57 crc kubenswrapper[5052]: I1203 12:40:57.424397 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ovsdbserver-nb-0" podUID="46c590ef-2b7c-412d-8582-dfdc311e6778" containerName="openstack-network-exporter" containerID="cri-o://e918d411b918ed8ca93043d08217369893147e6405ff8425f54ff29adf217db7" gracePeriod=300 Dec 03 12:40:57 crc kubenswrapper[5052]: I1203 12:40:57.443147 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/neutron-db-sync-5kww4"] Dec 03 12:40:57 crc kubenswrapper[5052]: I1203 12:40:57.492439 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/glance-db-sync-rxhl2"] Dec 03 12:40:57 crc kubenswrapper[5052]: I1203 12:40:57.494999 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/12f52069-8b08-4e4f-a3c1-a5421efb5844-operator-scripts\") pod \"neutrond24e-account-delete-p2mpp\" (UID: \"12f52069-8b08-4e4f-a3c1-a5421efb5844\") " pod="openstack-kuttl-tests/neutrond24e-account-delete-p2mpp" Dec 03 12:40:57 crc kubenswrapper[5052]: I1203 12:40:57.495162 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tbm29\" (UniqueName: \"kubernetes.io/projected/12f52069-8b08-4e4f-a3c1-a5421efb5844-kube-api-access-tbm29\") pod \"neutrond24e-account-delete-p2mpp\" (UID: \"12f52069-8b08-4e4f-a3c1-a5421efb5844\") " pod="openstack-kuttl-tests/neutrond24e-account-delete-p2mpp" Dec 03 12:40:57 crc kubenswrapper[5052]: E1203 12:40:57.495338 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/rabbitmq-cell1-config-data: configmap "rabbitmq-cell1-config-data" not found Dec 03 12:40:57 crc kubenswrapper[5052]: E1203 12:40:57.495438 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f9bac64c-518e-4630-8889-20e09d5b7a39-config-data podName:f9bac64c-518e-4630-8889-20e09d5b7a39 nodeName:}" failed. No retries permitted until 2025-12-03 12:40:58.495423865 +0000 UTC m=+4849.083665629 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "config-data" (UniqueName: "kubernetes.io/configmap/f9bac64c-518e-4630-8889-20e09d5b7a39-config-data") pod "rabbitmq-cell1-server-0" (UID: "f9bac64c-518e-4630-8889-20e09d5b7a39") : configmap "rabbitmq-cell1-config-data" not found Dec 03 12:40:57 crc kubenswrapper[5052]: I1203 12:40:57.528650 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/neutron-db-sync-5kww4"] Dec 03 12:40:57 crc kubenswrapper[5052]: I1203 12:40:57.574576 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutrond24e-account-delete-p2mpp"] Dec 03 12:40:57 crc kubenswrapper[5052]: I1203 12:40:57.575091 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ovsdbserver-nb-0" podUID="46c590ef-2b7c-412d-8582-dfdc311e6778" containerName="ovsdbserver-nb" containerID="cri-o://db6fc581a8df4ec1884c71516e39b9792e494f950b83f3b0b693c01758dfa0f1" gracePeriod=300 Dec 03 12:40:57 crc kubenswrapper[5052]: I1203 12:40:57.582268 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glancec94f-account-delete-8t6wx" Dec 03 12:40:57 crc kubenswrapper[5052]: I1203 12:40:57.596326 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/12f52069-8b08-4e4f-a3c1-a5421efb5844-operator-scripts\") pod \"neutrond24e-account-delete-p2mpp\" (UID: \"12f52069-8b08-4e4f-a3c1-a5421efb5844\") " pod="openstack-kuttl-tests/neutrond24e-account-delete-p2mpp" Dec 03 12:40:57 crc kubenswrapper[5052]: I1203 12:40:57.596371 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-tbm29\" (UniqueName: \"kubernetes.io/projected/12f52069-8b08-4e4f-a3c1-a5421efb5844-kube-api-access-tbm29\") pod \"neutrond24e-account-delete-p2mpp\" (UID: \"12f52069-8b08-4e4f-a3c1-a5421efb5844\") " pod="openstack-kuttl-tests/neutrond24e-account-delete-p2mpp" Dec 03 12:40:57 crc kubenswrapper[5052]: I1203 12:40:57.597275 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/12f52069-8b08-4e4f-a3c1-a5421efb5844-operator-scripts\") pod \"neutrond24e-account-delete-p2mpp\" (UID: \"12f52069-8b08-4e4f-a3c1-a5421efb5844\") " pod="openstack-kuttl-tests/neutrond24e-account-delete-p2mpp" Dec 03 12:40:57 crc kubenswrapper[5052]: I1203 12:40:57.626693 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-sb-0"] Dec 03 12:40:57 crc kubenswrapper[5052]: I1203 12:40:57.630853 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ovsdbserver-sb-0" podUID="d6bedb3e-52e6-44a9-afd9-a164b0730f52" containerName="openstack-network-exporter" containerID="cri-o://3f73a35540704d1c4ac272256521dc836a1d429fbe1cee0a507bfe3859a50b06" gracePeriod=300 Dec 03 12:40:57 crc kubenswrapper[5052]: I1203 12:40:57.631504 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-tbm29\" (UniqueName: \"kubernetes.io/projected/12f52069-8b08-4e4f-a3c1-a5421efb5844-kube-api-access-tbm29\") pod \"neutrond24e-account-delete-p2mpp\" (UID: \"12f52069-8b08-4e4f-a3c1-a5421efb5844\") " pod="openstack-kuttl-tests/neutrond24e-account-delete-p2mpp" Dec 03 12:40:57 crc kubenswrapper[5052]: I1203 12:40:57.653093 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/cinder3a0d-account-delete-lxd6l"] Dec 03 12:40:57 crc kubenswrapper[5052]: I1203 12:40:57.654733 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder3a0d-account-delete-lxd6l" Dec 03 12:40:57 crc kubenswrapper[5052]: I1203 12:40:57.670099 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder-db-sync-s42b9"] Dec 03 12:40:57 crc kubenswrapper[5052]: I1203 12:40:57.698984 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/5fdc7b7d-f9ed-4d5c-a77b-6a6b0fd186f9-operator-scripts\") pod \"cinder3a0d-account-delete-lxd6l\" (UID: \"5fdc7b7d-f9ed-4d5c-a77b-6a6b0fd186f9\") " pod="openstack-kuttl-tests/cinder3a0d-account-delete-lxd6l" Dec 03 12:40:57 crc kubenswrapper[5052]: I1203 12:40:57.699087 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-85cc8\" (UniqueName: \"kubernetes.io/projected/5fdc7b7d-f9ed-4d5c-a77b-6a6b0fd186f9-kube-api-access-85cc8\") pod \"cinder3a0d-account-delete-lxd6l\" (UID: \"5fdc7b7d-f9ed-4d5c-a77b-6a6b0fd186f9\") " pod="openstack-kuttl-tests/cinder3a0d-account-delete-lxd6l" Dec 03 12:40:57 crc kubenswrapper[5052]: E1203 12:40:57.699476 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-nova-public-svc: secret "cert-nova-public-svc" not found Dec 03 12:40:57 crc kubenswrapper[5052]: E1203 12:40:57.699567 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/50789f13-8194-4e24-b975-512591b816a5-public-tls-certs podName:50789f13-8194-4e24-b975-512591b816a5 nodeName:}" failed. No retries permitted until 2025-12-03 12:40:58.199545053 +0000 UTC m=+4848.787786857 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "public-tls-certs" (UniqueName: "kubernetes.io/secret/50789f13-8194-4e24-b975-512591b816a5-public-tls-certs") pod "nova-api-0" (UID: "50789f13-8194-4e24-b975-512591b816a5") : secret "cert-nova-public-svc" not found Dec 03 12:40:57 crc kubenswrapper[5052]: E1203 12:40:57.700153 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/combined-ca-bundle: secret "combined-ca-bundle" not found Dec 03 12:40:57 crc kubenswrapper[5052]: E1203 12:40:57.700203 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/50789f13-8194-4e24-b975-512591b816a5-combined-ca-bundle podName:50789f13-8194-4e24-b975-512591b816a5 nodeName:}" failed. No retries permitted until 2025-12-03 12:40:58.200189381 +0000 UTC m=+4848.788431145 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "combined-ca-bundle" (UniqueName: "kubernetes.io/secret/50789f13-8194-4e24-b975-512591b816a5-combined-ca-bundle") pod "nova-api-0" (UID: "50789f13-8194-4e24-b975-512591b816a5") : secret "combined-ca-bundle" not found Dec 03 12:40:57 crc kubenswrapper[5052]: E1203 12:40:57.700297 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-nova-internal-svc: secret "cert-nova-internal-svc" not found Dec 03 12:40:57 crc kubenswrapper[5052]: E1203 12:40:57.700362 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/50789f13-8194-4e24-b975-512591b816a5-internal-tls-certs podName:50789f13-8194-4e24-b975-512591b816a5 nodeName:}" failed. No retries permitted until 2025-12-03 12:40:58.200347176 +0000 UTC m=+4848.788588940 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "internal-tls-certs" (UniqueName: "kubernetes.io/secret/50789f13-8194-4e24-b975-512591b816a5-internal-tls-certs") pod "nova-api-0" (UID: "50789f13-8194-4e24-b975-512591b816a5") : secret "cert-nova-internal-svc" not found Dec 03 12:40:57 crc kubenswrapper[5052]: I1203 12:40:57.733344 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="348487de-7480-462b-9f22-56a555321be8" path="/var/lib/kubelet/pods/348487de-7480-462b-9f22-56a555321be8/volumes" Dec 03 12:40:57 crc kubenswrapper[5052]: I1203 12:40:57.733928 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6cc1f6de-7848-429c-9f14-ea763a359583" path="/var/lib/kubelet/pods/6cc1f6de-7848-429c-9f14-ea763a359583/volumes" Dec 03 12:40:57 crc kubenswrapper[5052]: I1203 12:40:57.734729 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder3a0d-account-delete-lxd6l"] Dec 03 12:40:57 crc kubenswrapper[5052]: I1203 12:40:57.734758 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/cinder-db-sync-s42b9"] Dec 03 12:40:57 crc kubenswrapper[5052]: I1203 12:40:57.753675 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ovn-northd-0"] Dec 03 12:40:57 crc kubenswrapper[5052]: I1203 12:40:57.753941 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ovn-northd-0" podUID="ea4cd237-7ec6-49f7-9afd-1ea159b9b23b" containerName="ovn-northd" containerID="cri-o://b2f117c03e1adda1bc8b58e2d0870125a9b0fd3dffffabf7fb3480969ee908b3" gracePeriod=30 Dec 03 12:40:57 crc kubenswrapper[5052]: I1203 12:40:57.754430 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ovn-northd-0" podUID="ea4cd237-7ec6-49f7-9afd-1ea159b9b23b" containerName="openstack-network-exporter" containerID="cri-o://2e4992cc2e8905b7fe793a8b028dc60f77b2fc53b9e9aa87bd2e3fbe6b4f4d4c" gracePeriod=30 Dec 03 12:40:57 crc kubenswrapper[5052]: I1203 12:40:57.770800 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/placement7f30-account-delete-vld4v"] Dec 03 12:40:57 crc kubenswrapper[5052]: I1203 12:40:57.772085 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement7f30-account-delete-vld4v" Dec 03 12:40:57 crc kubenswrapper[5052]: I1203 12:40:57.787406 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/placement7f30-account-delete-vld4v"] Dec 03 12:40:57 crc kubenswrapper[5052]: I1203 12:40:57.802105 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/5fdc7b7d-f9ed-4d5c-a77b-6a6b0fd186f9-operator-scripts\") pod \"cinder3a0d-account-delete-lxd6l\" (UID: \"5fdc7b7d-f9ed-4d5c-a77b-6a6b0fd186f9\") " pod="openstack-kuttl-tests/cinder3a0d-account-delete-lxd6l" Dec 03 12:40:57 crc kubenswrapper[5052]: I1203 12:40:57.802180 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-85cc8\" (UniqueName: \"kubernetes.io/projected/5fdc7b7d-f9ed-4d5c-a77b-6a6b0fd186f9-kube-api-access-85cc8\") pod \"cinder3a0d-account-delete-lxd6l\" (UID: \"5fdc7b7d-f9ed-4d5c-a77b-6a6b0fd186f9\") " pod="openstack-kuttl-tests/cinder3a0d-account-delete-lxd6l" Dec 03 12:40:57 crc kubenswrapper[5052]: I1203 12:40:57.802251 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-448x2\" (UniqueName: \"kubernetes.io/projected/eb8b91d5-6c99-45ba-8d33-da99f1dd496c-kube-api-access-448x2\") pod \"placement7f30-account-delete-vld4v\" (UID: \"eb8b91d5-6c99-45ba-8d33-da99f1dd496c\") " pod="openstack-kuttl-tests/placement7f30-account-delete-vld4v" Dec 03 12:40:57 crc kubenswrapper[5052]: I1203 12:40:57.802356 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/eb8b91d5-6c99-45ba-8d33-da99f1dd496c-operator-scripts\") pod \"placement7f30-account-delete-vld4v\" (UID: \"eb8b91d5-6c99-45ba-8d33-da99f1dd496c\") " pod="openstack-kuttl-tests/placement7f30-account-delete-vld4v" Dec 03 12:40:57 crc kubenswrapper[5052]: I1203 12:40:57.809386 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/5fdc7b7d-f9ed-4d5c-a77b-6a6b0fd186f9-operator-scripts\") pod \"cinder3a0d-account-delete-lxd6l\" (UID: \"5fdc7b7d-f9ed-4d5c-a77b-6a6b0fd186f9\") " pod="openstack-kuttl-tests/cinder3a0d-account-delete-lxd6l" Dec 03 12:40:57 crc kubenswrapper[5052]: I1203 12:40:57.809457 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/placement-db-sync-fcpth"] Dec 03 12:40:57 crc kubenswrapper[5052]: I1203 12:40:57.833143 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/swift-storage-0"] Dec 03 12:40:57 crc kubenswrapper[5052]: I1203 12:40:57.833652 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="c7d598df-f336-4798-b381-9b813068f271" containerName="account-server" containerID="cri-o://7fd8d3e9755be0491e71f4771e79219dfa4803774a0535e6af7be76023acf022" gracePeriod=30 Dec 03 12:40:57 crc kubenswrapper[5052]: I1203 12:40:57.833775 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="c7d598df-f336-4798-b381-9b813068f271" containerName="object-server" containerID="cri-o://e82e8cd6cac0e3fe75cbcd377821970ca81cbeff69756a18c653617637c572e3" gracePeriod=30 Dec 03 12:40:57 crc kubenswrapper[5052]: I1203 12:40:57.833792 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="c7d598df-f336-4798-b381-9b813068f271" containerName="container-updater" containerID="cri-o://e1915bece2ea7e993dda7b1e15e670ab02f1ff041682e537f1d13ff72de48727" gracePeriod=30 Dec 03 12:40:57 crc kubenswrapper[5052]: I1203 12:40:57.833844 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="c7d598df-f336-4798-b381-9b813068f271" containerName="container-auditor" containerID="cri-o://bf315758fc67c50f371749f4f7bbec4a1f92fbcaefd1d1b439352a3050edde3f" gracePeriod=30 Dec 03 12:40:57 crc kubenswrapper[5052]: I1203 12:40:57.833887 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="c7d598df-f336-4798-b381-9b813068f271" containerName="container-replicator" containerID="cri-o://aa711f046bfc3e94d6ec5597e4da6fb83450c27a3396cf24f0175b9d7508c25a" gracePeriod=30 Dec 03 12:40:57 crc kubenswrapper[5052]: I1203 12:40:57.833924 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="c7d598df-f336-4798-b381-9b813068f271" containerName="container-server" containerID="cri-o://490483344d3233731e445298ec4b3a6f3b3d6ba0822ed43ceb5cdbc637a73f2c" gracePeriod=30 Dec 03 12:40:57 crc kubenswrapper[5052]: I1203 12:40:57.833975 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="c7d598df-f336-4798-b381-9b813068f271" containerName="account-reaper" containerID="cri-o://13466840574a6516f401dae001c130c3bbb09775106363deef64c61f44c71bac" gracePeriod=30 Dec 03 12:40:57 crc kubenswrapper[5052]: I1203 12:40:57.834015 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="c7d598df-f336-4798-b381-9b813068f271" containerName="account-auditor" containerID="cri-o://93f01c0cccc7bb298548365c64b66abc7b40d0c114c2d14e3eaed3c673c6da66" gracePeriod=30 Dec 03 12:40:57 crc kubenswrapper[5052]: I1203 12:40:57.834056 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="c7d598df-f336-4798-b381-9b813068f271" containerName="account-replicator" containerID="cri-o://cabbc82eac0ab602656d651e0b196849ff956b6dd323581aac8d857b23fae0e7" gracePeriod=30 Dec 03 12:40:57 crc kubenswrapper[5052]: I1203 12:40:57.834115 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="c7d598df-f336-4798-b381-9b813068f271" containerName="swift-recon-cron" containerID="cri-o://2c19477455e7e676ad01ea83d0e26999fb852f2f3e3c20bf2a784b76c5930dfc" gracePeriod=30 Dec 03 12:40:57 crc kubenswrapper[5052]: I1203 12:40:57.834215 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="c7d598df-f336-4798-b381-9b813068f271" containerName="rsync" containerID="cri-o://34d7b3c41a14d274d9f82dc6e97d783bfeea4af12baf996aefeb1b3be4f773dd" gracePeriod=30 Dec 03 12:40:57 crc kubenswrapper[5052]: I1203 12:40:57.834253 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="c7d598df-f336-4798-b381-9b813068f271" containerName="object-expirer" containerID="cri-o://3668ba2994a5322df4fc232407f1ab911e9742d99a5ce860e7891029d6b3262b" gracePeriod=30 Dec 03 12:40:57 crc kubenswrapper[5052]: I1203 12:40:57.834293 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="c7d598df-f336-4798-b381-9b813068f271" containerName="object-updater" containerID="cri-o://2bccd4a808d4dfbe6d20fb21dfd50899da53a3d24dee822e5b066fe75d15e745" gracePeriod=30 Dec 03 12:40:57 crc kubenswrapper[5052]: I1203 12:40:57.834329 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="c7d598df-f336-4798-b381-9b813068f271" containerName="object-auditor" containerID="cri-o://5e022d3b3c656463516700245609bcc85dbdb9a341c772fa6cfcb4273bc6c15d" gracePeriod=30 Dec 03 12:40:57 crc kubenswrapper[5052]: I1203 12:40:57.834376 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="c7d598df-f336-4798-b381-9b813068f271" containerName="object-replicator" containerID="cri-o://67aad15627897c77a5ca09bba14532fa30813bf015f5d0017d2d2538795a5666" gracePeriod=30 Dec 03 12:40:57 crc kubenswrapper[5052]: I1203 12:40:57.854741 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutrond24e-account-delete-p2mpp" Dec 03 12:40:57 crc kubenswrapper[5052]: I1203 12:40:57.892234 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-85cc8\" (UniqueName: \"kubernetes.io/projected/5fdc7b7d-f9ed-4d5c-a77b-6a6b0fd186f9-kube-api-access-85cc8\") pod \"cinder3a0d-account-delete-lxd6l\" (UID: \"5fdc7b7d-f9ed-4d5c-a77b-6a6b0fd186f9\") " pod="openstack-kuttl-tests/cinder3a0d-account-delete-lxd6l" Dec 03 12:40:57 crc kubenswrapper[5052]: I1203 12:40:57.903968 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ovsdbserver-sb-0" podUID="d6bedb3e-52e6-44a9-afd9-a164b0730f52" containerName="ovsdbserver-sb" containerID="cri-o://87e14300e8697a4e0c8594bb1dde5f610d618d2da77d5478a1bb33029c23a5e0" gracePeriod=300 Dec 03 12:40:57 crc kubenswrapper[5052]: I1203 12:40:57.908685 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/eb8b91d5-6c99-45ba-8d33-da99f1dd496c-operator-scripts\") pod \"placement7f30-account-delete-vld4v\" (UID: \"eb8b91d5-6c99-45ba-8d33-da99f1dd496c\") " pod="openstack-kuttl-tests/placement7f30-account-delete-vld4v" Dec 03 12:40:57 crc kubenswrapper[5052]: I1203 12:40:57.908863 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-448x2\" (UniqueName: \"kubernetes.io/projected/eb8b91d5-6c99-45ba-8d33-da99f1dd496c-kube-api-access-448x2\") pod \"placement7f30-account-delete-vld4v\" (UID: \"eb8b91d5-6c99-45ba-8d33-da99f1dd496c\") " pod="openstack-kuttl-tests/placement7f30-account-delete-vld4v" Dec 03 12:40:57 crc kubenswrapper[5052]: E1203 12:40:57.909160 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/rabbitmq-config-data: configmap "rabbitmq-config-data" not found Dec 03 12:40:57 crc kubenswrapper[5052]: E1203 12:40:57.909202 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d82ef8ea-0c3b-4d89-9c78-22095940f4da-config-data podName:d82ef8ea-0c3b-4d89-9c78-22095940f4da nodeName:}" failed. No retries permitted until 2025-12-03 12:40:58.909190008 +0000 UTC m=+4849.497431772 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "config-data" (UniqueName: "kubernetes.io/configmap/d82ef8ea-0c3b-4d89-9c78-22095940f4da-config-data") pod "rabbitmq-server-0" (UID: "d82ef8ea-0c3b-4d89-9c78-22095940f4da") : configmap "rabbitmq-config-data" not found Dec 03 12:40:57 crc kubenswrapper[5052]: I1203 12:40:57.915922 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/eb8b91d5-6c99-45ba-8d33-da99f1dd496c-operator-scripts\") pod \"placement7f30-account-delete-vld4v\" (UID: \"eb8b91d5-6c99-45ba-8d33-da99f1dd496c\") " pod="openstack-kuttl-tests/placement7f30-account-delete-vld4v" Dec 03 12:40:58 crc kubenswrapper[5052]: I1203 12:40:57.938230 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-448x2\" (UniqueName: \"kubernetes.io/projected/eb8b91d5-6c99-45ba-8d33-da99f1dd496c-kube-api-access-448x2\") pod \"placement7f30-account-delete-vld4v\" (UID: \"eb8b91d5-6c99-45ba-8d33-da99f1dd496c\") " pod="openstack-kuttl-tests/placement7f30-account-delete-vld4v" Dec 03 12:40:58 crc kubenswrapper[5052]: I1203 12:40:57.976734 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/placement-db-sync-fcpth"] Dec 03 12:40:58 crc kubenswrapper[5052]: I1203 12:40:58.008732 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:40:58 crc kubenswrapper[5052]: I1203 12:40:58.009081 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-internal-api-0" podUID="3039127c-0211-4430-9b2b-426412241ce4" containerName="glance-log" containerID="cri-o://101072c12291fb7e9e1ca717c6db562103892180f6281b3dd97442d2a61e51f8" gracePeriod=30 Dec 03 12:40:58 crc kubenswrapper[5052]: I1203 12:40:58.009726 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-internal-api-0" podUID="3039127c-0211-4430-9b2b-426412241ce4" containerName="glance-httpd" containerID="cri-o://1d3c75c1863b6df410b4948455d1e97a7c92fce681cbc217dafa1d03a2fae0c6" gracePeriod=30 Dec 03 12:40:58 crc kubenswrapper[5052]: I1203 12:40:58.026576 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/swift-ring-rebalance-jstcn"] Dec 03 12:40:58 crc kubenswrapper[5052]: I1203 12:40:58.063614 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/swift-ring-rebalance-jstcn"] Dec 03 12:40:58 crc kubenswrapper[5052]: I1203 12:40:58.078129 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-db-sync-nj49v"] Dec 03 12:40:58 crc kubenswrapper[5052]: I1203 12:40:58.096459 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/barbican-db-sync-nj49v"] Dec 03 12:40:58 crc kubenswrapper[5052]: I1203 12:40:58.121500 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/barbican1171-account-delete-9hmgl"] Dec 03 12:40:58 crc kubenswrapper[5052]: I1203 12:40:58.123016 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican1171-account-delete-9hmgl" Dec 03 12:40:58 crc kubenswrapper[5052]: I1203 12:40:58.148789 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican1171-account-delete-9hmgl"] Dec 03 12:40:58 crc kubenswrapper[5052]: I1203 12:40:58.192635 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/rabbitmq-cell1-server-0"] Dec 03 12:40:58 crc kubenswrapper[5052]: I1203 12:40:58.207107 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-8h8qz"] Dec 03 12:40:58 crc kubenswrapper[5052]: I1203 12:40:58.209448 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-8h8qz" Dec 03 12:40:58 crc kubenswrapper[5052]: I1203 12:40:58.230580 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:40:58 crc kubenswrapper[5052]: I1203 12:40:58.230920 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-external-api-0" podUID="b0606156-f409-4cae-9be1-15d990f69955" containerName="glance-log" containerID="cri-o://027b9a8307f751e8024d032edb748ef4b5da8f3d938c7c031bac5ed5e44f0c7d" gracePeriod=30 Dec 03 12:40:58 crc kubenswrapper[5052]: I1203 12:40:58.231094 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-external-api-0" podUID="b0606156-f409-4cae-9be1-15d990f69955" containerName="glance-httpd" containerID="cri-o://8ca42e62aac09b81598563bdb887c5ea6090758cf976adc8ad4b4e387ec44287" gracePeriod=30 Dec 03 12:40:58 crc kubenswrapper[5052]: I1203 12:40:58.233836 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/296f8cec-ad7b-4b51-a9a9-934301d43d2e-config\") pod \"dnsmasq-dnsmasq-84b9f45d47-8h8qz\" (UID: \"296f8cec-ad7b-4b51-a9a9-934301d43d2e\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-8h8qz" Dec 03 12:40:58 crc kubenswrapper[5052]: I1203 12:40:58.235277 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/296f8cec-ad7b-4b51-a9a9-934301d43d2e-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-84b9f45d47-8h8qz\" (UID: \"296f8cec-ad7b-4b51-a9a9-934301d43d2e\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-8h8qz" Dec 03 12:40:58 crc kubenswrapper[5052]: I1203 12:40:58.235444 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-59qch\" (UniqueName: \"kubernetes.io/projected/ab29f82e-0693-4001-9f2a-bb4cc2fa1d5a-kube-api-access-59qch\") pod \"barbican1171-account-delete-9hmgl\" (UID: \"ab29f82e-0693-4001-9f2a-bb4cc2fa1d5a\") " pod="openstack-kuttl-tests/barbican1171-account-delete-9hmgl" Dec 03 12:40:58 crc kubenswrapper[5052]: I1203 12:40:58.235529 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/ab29f82e-0693-4001-9f2a-bb4cc2fa1d5a-operator-scripts\") pod \"barbican1171-account-delete-9hmgl\" (UID: \"ab29f82e-0693-4001-9f2a-bb4cc2fa1d5a\") " pod="openstack-kuttl-tests/barbican1171-account-delete-9hmgl" Dec 03 12:40:58 crc kubenswrapper[5052]: E1203 12:40:58.235628 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-nova-public-svc: secret "cert-nova-public-svc" not found Dec 03 12:40:58 crc kubenswrapper[5052]: E1203 12:40:58.235699 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/50789f13-8194-4e24-b975-512591b816a5-public-tls-certs podName:50789f13-8194-4e24-b975-512591b816a5 nodeName:}" failed. No retries permitted until 2025-12-03 12:40:59.235676465 +0000 UTC m=+4849.823918229 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "public-tls-certs" (UniqueName: "kubernetes.io/secret/50789f13-8194-4e24-b975-512591b816a5-public-tls-certs") pod "nova-api-0" (UID: "50789f13-8194-4e24-b975-512591b816a5") : secret "cert-nova-public-svc" not found Dec 03 12:40:58 crc kubenswrapper[5052]: I1203 12:40:58.235879 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dwdpg\" (UniqueName: \"kubernetes.io/projected/296f8cec-ad7b-4b51-a9a9-934301d43d2e-kube-api-access-dwdpg\") pod \"dnsmasq-dnsmasq-84b9f45d47-8h8qz\" (UID: \"296f8cec-ad7b-4b51-a9a9-934301d43d2e\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-8h8qz" Dec 03 12:40:58 crc kubenswrapper[5052]: E1203 12:40:58.236189 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/combined-ca-bundle: secret "combined-ca-bundle" not found Dec 03 12:40:58 crc kubenswrapper[5052]: E1203 12:40:58.236241 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/50789f13-8194-4e24-b975-512591b816a5-combined-ca-bundle podName:50789f13-8194-4e24-b975-512591b816a5 nodeName:}" failed. No retries permitted until 2025-12-03 12:40:59.236219981 +0000 UTC m=+4849.824461745 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "combined-ca-bundle" (UniqueName: "kubernetes.io/secret/50789f13-8194-4e24-b975-512591b816a5-combined-ca-bundle") pod "nova-api-0" (UID: "50789f13-8194-4e24-b975-512591b816a5") : secret "combined-ca-bundle" not found Dec 03 12:40:58 crc kubenswrapper[5052]: E1203 12:40:58.236298 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-nova-internal-svc: secret "cert-nova-internal-svc" not found Dec 03 12:40:58 crc kubenswrapper[5052]: E1203 12:40:58.236328 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/50789f13-8194-4e24-b975-512591b816a5-internal-tls-certs podName:50789f13-8194-4e24-b975-512591b816a5 nodeName:}" failed. No retries permitted until 2025-12-03 12:40:59.236319204 +0000 UTC m=+4849.824560968 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "internal-tls-certs" (UniqueName: "kubernetes.io/secret/50789f13-8194-4e24-b975-512591b816a5-internal-tls-certs") pod "nova-api-0" (UID: "50789f13-8194-4e24-b975-512591b816a5") : secret "cert-nova-internal-svc" not found Dec 03 12:40:58 crc kubenswrapper[5052]: I1203 12:40:58.241087 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-8h8qz"] Dec 03 12:40:58 crc kubenswrapper[5052]: E1203 12:40:58.243835 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="b2f117c03e1adda1bc8b58e2d0870125a9b0fd3dffffabf7fb3480969ee908b3" cmd=["/usr/local/bin/container-scripts/status_check.sh"] Dec 03 12:40:58 crc kubenswrapper[5052]: E1203 12:40:58.268652 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="b2f117c03e1adda1bc8b58e2d0870125a9b0fd3dffffabf7fb3480969ee908b3" cmd=["/usr/local/bin/container-scripts/status_check.sh"] Dec 03 12:40:58 crc kubenswrapper[5052]: E1203 12:40:58.271318 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="b2f117c03e1adda1bc8b58e2d0870125a9b0fd3dffffabf7fb3480969ee908b3" cmd=["/usr/local/bin/container-scripts/status_check.sh"] Dec 03 12:40:58 crc kubenswrapper[5052]: E1203 12:40:58.271385 5052 prober.go:104] "Probe errored" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" probeType="Readiness" pod="openstack-kuttl-tests/ovn-northd-0" podUID="ea4cd237-7ec6-49f7-9afd-1ea159b9b23b" containerName="ovn-northd" Dec 03 12:40:58 crc kubenswrapper[5052]: I1203 12:40:58.289705 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/neutron-7d74684fd-pqkf5"] Dec 03 12:40:58 crc kubenswrapper[5052]: I1203 12:40:58.292172 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/neutron-7d74684fd-pqkf5" podUID="ddfa4a3f-b868-49b7-abec-62cb3ffde2bd" containerName="neutron-api" containerID="cri-o://ef78ab4daf6f2994bb54c1f8844ff1255eef71866c6e7be47ed5d4c7ef1d996d" gracePeriod=30 Dec 03 12:40:58 crc kubenswrapper[5052]: I1203 12:40:58.292646 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/neutron-7d74684fd-pqkf5" podUID="ddfa4a3f-b868-49b7-abec-62cb3ffde2bd" containerName="neutron-httpd" containerID="cri-o://8ed93a1c3257ab19479e68e87c18d74af55fe2f519944a6fbeee4da11170b261" gracePeriod=30 Dec 03 12:40:58 crc kubenswrapper[5052]: I1203 12:40:58.317236 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 12:40:58 crc kubenswrapper[5052]: I1203 12:40:58.317568 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/cinder-scheduler-0" podUID="d855c655-3b2f-4e4f-9b27-cf4e43a92045" containerName="probe" containerID="cri-o://323efac3746af7ff7702e46c55dedadf7e2d8d45f1a20d6b7d1fa2753913a05d" gracePeriod=30 Dec 03 12:40:58 crc kubenswrapper[5052]: I1203 12:40:58.318130 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/cinder-scheduler-0" podUID="d855c655-3b2f-4e4f-9b27-cf4e43a92045" containerName="cinder-scheduler" containerID="cri-o://84ca5b96b0f3458adcf7e677440d6f30818b02e536633792be662d5c24cf8b44" gracePeriod=30 Dec 03 12:40:58 crc kubenswrapper[5052]: I1203 12:40:58.338126 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/novacell07298-account-delete-726jv"] Dec 03 12:40:58 crc kubenswrapper[5052]: I1203 12:40:58.343632 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/296f8cec-ad7b-4b51-a9a9-934301d43d2e-config\") pod \"dnsmasq-dnsmasq-84b9f45d47-8h8qz\" (UID: \"296f8cec-ad7b-4b51-a9a9-934301d43d2e\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-8h8qz" Dec 03 12:40:58 crc kubenswrapper[5052]: I1203 12:40:58.343693 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/296f8cec-ad7b-4b51-a9a9-934301d43d2e-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-84b9f45d47-8h8qz\" (UID: \"296f8cec-ad7b-4b51-a9a9-934301d43d2e\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-8h8qz" Dec 03 12:40:58 crc kubenswrapper[5052]: I1203 12:40:58.343829 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-59qch\" (UniqueName: \"kubernetes.io/projected/ab29f82e-0693-4001-9f2a-bb4cc2fa1d5a-kube-api-access-59qch\") pod \"barbican1171-account-delete-9hmgl\" (UID: \"ab29f82e-0693-4001-9f2a-bb4cc2fa1d5a\") " pod="openstack-kuttl-tests/barbican1171-account-delete-9hmgl" Dec 03 12:40:58 crc kubenswrapper[5052]: I1203 12:40:58.343889 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/ab29f82e-0693-4001-9f2a-bb4cc2fa1d5a-operator-scripts\") pod \"barbican1171-account-delete-9hmgl\" (UID: \"ab29f82e-0693-4001-9f2a-bb4cc2fa1d5a\") " pod="openstack-kuttl-tests/barbican1171-account-delete-9hmgl" Dec 03 12:40:58 crc kubenswrapper[5052]: I1203 12:40:58.343963 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-dwdpg\" (UniqueName: \"kubernetes.io/projected/296f8cec-ad7b-4b51-a9a9-934301d43d2e-kube-api-access-dwdpg\") pod \"dnsmasq-dnsmasq-84b9f45d47-8h8qz\" (UID: \"296f8cec-ad7b-4b51-a9a9-934301d43d2e\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-8h8qz" Dec 03 12:40:58 crc kubenswrapper[5052]: I1203 12:40:58.346904 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/296f8cec-ad7b-4b51-a9a9-934301d43d2e-config\") pod \"dnsmasq-dnsmasq-84b9f45d47-8h8qz\" (UID: \"296f8cec-ad7b-4b51-a9a9-934301d43d2e\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-8h8qz" Dec 03 12:40:58 crc kubenswrapper[5052]: I1203 12:40:58.347615 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/296f8cec-ad7b-4b51-a9a9-934301d43d2e-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-84b9f45d47-8h8qz\" (UID: \"296f8cec-ad7b-4b51-a9a9-934301d43d2e\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-8h8qz" Dec 03 12:40:58 crc kubenswrapper[5052]: I1203 12:40:58.353499 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/ab29f82e-0693-4001-9f2a-bb4cc2fa1d5a-operator-scripts\") pod \"barbican1171-account-delete-9hmgl\" (UID: \"ab29f82e-0693-4001-9f2a-bb4cc2fa1d5a\") " pod="openstack-kuttl-tests/barbican1171-account-delete-9hmgl" Dec 03 12:40:58 crc kubenswrapper[5052]: I1203 12:40:58.353907 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/novacell07298-account-delete-726jv" Dec 03 12:40:58 crc kubenswrapper[5052]: I1203 12:40:58.393570 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/novacell07298-account-delete-726jv"] Dec 03 12:40:58 crc kubenswrapper[5052]: I1203 12:40:58.419437 5052 generic.go:334] "Generic (PLEG): container finished" podID="b0606156-f409-4cae-9be1-15d990f69955" containerID="027b9a8307f751e8024d032edb748ef4b5da8f3d938c7c031bac5ed5e44f0c7d" exitCode=143 Dec 03 12:40:58 crc kubenswrapper[5052]: I1203 12:40:58.419550 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"b0606156-f409-4cae-9be1-15d990f69955","Type":"ContainerDied","Data":"027b9a8307f751e8024d032edb748ef4b5da8f3d938c7c031bac5ed5e44f0c7d"} Dec 03 12:40:58 crc kubenswrapper[5052]: I1203 12:40:58.421344 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_ovsdbserver-nb-0_46c590ef-2b7c-412d-8582-dfdc311e6778/ovsdbserver-nb/0.log" Dec 03 12:40:58 crc kubenswrapper[5052]: I1203 12:40:58.421390 5052 generic.go:334] "Generic (PLEG): container finished" podID="46c590ef-2b7c-412d-8582-dfdc311e6778" containerID="e918d411b918ed8ca93043d08217369893147e6405ff8425f54ff29adf217db7" exitCode=2 Dec 03 12:40:58 crc kubenswrapper[5052]: I1203 12:40:58.421405 5052 generic.go:334] "Generic (PLEG): container finished" podID="46c590ef-2b7c-412d-8582-dfdc311e6778" containerID="db6fc581a8df4ec1884c71516e39b9792e494f950b83f3b0b693c01758dfa0f1" exitCode=143 Dec 03 12:40:58 crc kubenswrapper[5052]: I1203 12:40:58.421495 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-nb-0" event={"ID":"46c590ef-2b7c-412d-8582-dfdc311e6778","Type":"ContainerDied","Data":"e918d411b918ed8ca93043d08217369893147e6405ff8425f54ff29adf217db7"} Dec 03 12:40:58 crc kubenswrapper[5052]: I1203 12:40:58.432718 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-nb-0" event={"ID":"46c590ef-2b7c-412d-8582-dfdc311e6778","Type":"ContainerDied","Data":"db6fc581a8df4ec1884c71516e39b9792e494f950b83f3b0b693c01758dfa0f1"} Dec 03 12:40:58 crc kubenswrapper[5052]: I1203 12:40:58.425207 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_ovsdbserver-sb-0_d6bedb3e-52e6-44a9-afd9-a164b0730f52/ovsdbserver-sb/0.log" Dec 03 12:40:58 crc kubenswrapper[5052]: I1203 12:40:58.432802 5052 generic.go:334] "Generic (PLEG): container finished" podID="d6bedb3e-52e6-44a9-afd9-a164b0730f52" containerID="3f73a35540704d1c4ac272256521dc836a1d429fbe1cee0a507bfe3859a50b06" exitCode=2 Dec 03 12:40:58 crc kubenswrapper[5052]: I1203 12:40:58.432815 5052 generic.go:334] "Generic (PLEG): container finished" podID="d6bedb3e-52e6-44a9-afd9-a164b0730f52" containerID="87e14300e8697a4e0c8594bb1dde5f610d618d2da77d5478a1bb33029c23a5e0" exitCode=143 Dec 03 12:40:58 crc kubenswrapper[5052]: I1203 12:40:58.432943 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-sb-0" event={"ID":"d6bedb3e-52e6-44a9-afd9-a164b0730f52","Type":"ContainerDied","Data":"3f73a35540704d1c4ac272256521dc836a1d429fbe1cee0a507bfe3859a50b06"} Dec 03 12:40:58 crc kubenswrapper[5052]: I1203 12:40:58.432958 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-sb-0" event={"ID":"d6bedb3e-52e6-44a9-afd9-a164b0730f52","Type":"ContainerDied","Data":"87e14300e8697a4e0c8594bb1dde5f610d618d2da77d5478a1bb33029c23a5e0"} Dec 03 12:40:58 crc kubenswrapper[5052]: I1203 12:40:58.441593 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell1-cell-mapping-fj7tx"] Dec 03 12:40:58 crc kubenswrapper[5052]: I1203 12:40:58.447954 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4tjwn\" (UniqueName: \"kubernetes.io/projected/6dbc7d8f-34d4-4927-82fe-d049727808fd-kube-api-access-4tjwn\") pod \"novacell07298-account-delete-726jv\" (UID: \"6dbc7d8f-34d4-4927-82fe-d049727808fd\") " pod="openstack-kuttl-tests/novacell07298-account-delete-726jv" Dec 03 12:40:58 crc kubenswrapper[5052]: I1203 12:40:58.448088 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/6dbc7d8f-34d4-4927-82fe-d049727808fd-operator-scripts\") pod \"novacell07298-account-delete-726jv\" (UID: \"6dbc7d8f-34d4-4927-82fe-d049727808fd\") " pod="openstack-kuttl-tests/novacell07298-account-delete-726jv" Dec 03 12:40:58 crc kubenswrapper[5052]: E1203 12:40:58.448263 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/combined-ca-bundle: secret "combined-ca-bundle" not found Dec 03 12:40:58 crc kubenswrapper[5052]: E1203 12:40:58.448339 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0556e9f0-e431-4193-9822-6079cb8a1e44-combined-ca-bundle podName:0556e9f0-e431-4193-9822-6079cb8a1e44 nodeName:}" failed. No retries permitted until 2025-12-03 12:41:00.448319807 +0000 UTC m=+4851.036561571 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "combined-ca-bundle" (UniqueName: "kubernetes.io/secret/0556e9f0-e431-4193-9822-6079cb8a1e44-combined-ca-bundle") pod "nova-cell1-conductor-0" (UID: "0556e9f0-e431-4193-9822-6079cb8a1e44") : secret "combined-ca-bundle" not found Dec 03 12:40:58 crc kubenswrapper[5052]: I1203 12:40:58.448595 5052 generic.go:334] "Generic (PLEG): container finished" podID="c7d598df-f336-4798-b381-9b813068f271" containerID="3668ba2994a5322df4fc232407f1ab911e9742d99a5ce860e7891029d6b3262b" exitCode=0 Dec 03 12:40:58 crc kubenswrapper[5052]: I1203 12:40:58.448620 5052 generic.go:334] "Generic (PLEG): container finished" podID="c7d598df-f336-4798-b381-9b813068f271" containerID="2bccd4a808d4dfbe6d20fb21dfd50899da53a3d24dee822e5b066fe75d15e745" exitCode=0 Dec 03 12:40:58 crc kubenswrapper[5052]: I1203 12:40:58.448627 5052 generic.go:334] "Generic (PLEG): container finished" podID="c7d598df-f336-4798-b381-9b813068f271" containerID="5e022d3b3c656463516700245609bcc85dbdb9a341c772fa6cfcb4273bc6c15d" exitCode=0 Dec 03 12:40:58 crc kubenswrapper[5052]: I1203 12:40:58.448633 5052 generic.go:334] "Generic (PLEG): container finished" podID="c7d598df-f336-4798-b381-9b813068f271" containerID="e82e8cd6cac0e3fe75cbcd377821970ca81cbeff69756a18c653617637c572e3" exitCode=0 Dec 03 12:40:58 crc kubenswrapper[5052]: I1203 12:40:58.448640 5052 generic.go:334] "Generic (PLEG): container finished" podID="c7d598df-f336-4798-b381-9b813068f271" containerID="e1915bece2ea7e993dda7b1e15e670ab02f1ff041682e537f1d13ff72de48727" exitCode=0 Dec 03 12:40:58 crc kubenswrapper[5052]: I1203 12:40:58.448646 5052 generic.go:334] "Generic (PLEG): container finished" podID="c7d598df-f336-4798-b381-9b813068f271" containerID="bf315758fc67c50f371749f4f7bbec4a1f92fbcaefd1d1b439352a3050edde3f" exitCode=0 Dec 03 12:40:58 crc kubenswrapper[5052]: I1203 12:40:58.448652 5052 generic.go:334] "Generic (PLEG): container finished" podID="c7d598df-f336-4798-b381-9b813068f271" containerID="aa711f046bfc3e94d6ec5597e4da6fb83450c27a3396cf24f0175b9d7508c25a" exitCode=0 Dec 03 12:40:58 crc kubenswrapper[5052]: I1203 12:40:58.448661 5052 generic.go:334] "Generic (PLEG): container finished" podID="c7d598df-f336-4798-b381-9b813068f271" containerID="13466840574a6516f401dae001c130c3bbb09775106363deef64c61f44c71bac" exitCode=0 Dec 03 12:40:58 crc kubenswrapper[5052]: I1203 12:40:58.448671 5052 generic.go:334] "Generic (PLEG): container finished" podID="c7d598df-f336-4798-b381-9b813068f271" containerID="93f01c0cccc7bb298548365c64b66abc7b40d0c114c2d14e3eaed3c673c6da66" exitCode=0 Dec 03 12:40:58 crc kubenswrapper[5052]: I1203 12:40:58.448678 5052 generic.go:334] "Generic (PLEG): container finished" podID="c7d598df-f336-4798-b381-9b813068f271" containerID="cabbc82eac0ab602656d651e0b196849ff956b6dd323581aac8d857b23fae0e7" exitCode=0 Dec 03 12:40:58 crc kubenswrapper[5052]: I1203 12:40:58.448823 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"c7d598df-f336-4798-b381-9b813068f271","Type":"ContainerDied","Data":"3668ba2994a5322df4fc232407f1ab911e9742d99a5ce860e7891029d6b3262b"} Dec 03 12:40:58 crc kubenswrapper[5052]: I1203 12:40:58.448863 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"c7d598df-f336-4798-b381-9b813068f271","Type":"ContainerDied","Data":"2bccd4a808d4dfbe6d20fb21dfd50899da53a3d24dee822e5b066fe75d15e745"} Dec 03 12:40:58 crc kubenswrapper[5052]: I1203 12:40:58.448877 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"c7d598df-f336-4798-b381-9b813068f271","Type":"ContainerDied","Data":"5e022d3b3c656463516700245609bcc85dbdb9a341c772fa6cfcb4273bc6c15d"} Dec 03 12:40:58 crc kubenswrapper[5052]: I1203 12:40:58.448889 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"c7d598df-f336-4798-b381-9b813068f271","Type":"ContainerDied","Data":"e82e8cd6cac0e3fe75cbcd377821970ca81cbeff69756a18c653617637c572e3"} Dec 03 12:40:58 crc kubenswrapper[5052]: I1203 12:40:58.448912 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"c7d598df-f336-4798-b381-9b813068f271","Type":"ContainerDied","Data":"e1915bece2ea7e993dda7b1e15e670ab02f1ff041682e537f1d13ff72de48727"} Dec 03 12:40:58 crc kubenswrapper[5052]: I1203 12:40:58.448924 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"c7d598df-f336-4798-b381-9b813068f271","Type":"ContainerDied","Data":"bf315758fc67c50f371749f4f7bbec4a1f92fbcaefd1d1b439352a3050edde3f"} Dec 03 12:40:58 crc kubenswrapper[5052]: I1203 12:40:58.448937 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"c7d598df-f336-4798-b381-9b813068f271","Type":"ContainerDied","Data":"aa711f046bfc3e94d6ec5597e4da6fb83450c27a3396cf24f0175b9d7508c25a"} Dec 03 12:40:58 crc kubenswrapper[5052]: I1203 12:40:58.448948 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"c7d598df-f336-4798-b381-9b813068f271","Type":"ContainerDied","Data":"13466840574a6516f401dae001c130c3bbb09775106363deef64c61f44c71bac"} Dec 03 12:40:58 crc kubenswrapper[5052]: I1203 12:40:58.448961 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"c7d598df-f336-4798-b381-9b813068f271","Type":"ContainerDied","Data":"93f01c0cccc7bb298548365c64b66abc7b40d0c114c2d14e3eaed3c673c6da66"} Dec 03 12:40:58 crc kubenswrapper[5052]: I1203 12:40:58.448986 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"c7d598df-f336-4798-b381-9b813068f271","Type":"ContainerDied","Data":"cabbc82eac0ab602656d651e0b196849ff956b6dd323581aac8d857b23fae0e7"} Dec 03 12:40:58 crc kubenswrapper[5052]: I1203 12:40:58.457637 5052 generic.go:334] "Generic (PLEG): container finished" podID="3039127c-0211-4430-9b2b-426412241ce4" containerID="101072c12291fb7e9e1ca717c6db562103892180f6281b3dd97442d2a61e51f8" exitCode=143 Dec 03 12:40:58 crc kubenswrapper[5052]: I1203 12:40:58.457704 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"3039127c-0211-4430-9b2b-426412241ce4","Type":"ContainerDied","Data":"101072c12291fb7e9e1ca717c6db562103892180f6281b3dd97442d2a61e51f8"} Dec 03 12:40:58 crc kubenswrapper[5052]: I1203 12:40:58.462580 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/novaapib163-account-delete-8z4bt"] Dec 03 12:40:58 crc kubenswrapper[5052]: I1203 12:40:58.463984 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/novaapib163-account-delete-8z4bt" Dec 03 12:40:58 crc kubenswrapper[5052]: I1203 12:40:58.475553 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell1-cell-mapping-fj7tx"] Dec 03 12:40:58 crc kubenswrapper[5052]: I1203 12:40:58.486056 5052 generic.go:334] "Generic (PLEG): container finished" podID="ea4cd237-7ec6-49f7-9afd-1ea159b9b23b" containerID="2e4992cc2e8905b7fe793a8b028dc60f77b2fc53b9e9aa87bd2e3fbe6b4f4d4c" exitCode=2 Dec 03 12:40:58 crc kubenswrapper[5052]: I1203 12:40:58.486102 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovn-northd-0" event={"ID":"ea4cd237-7ec6-49f7-9afd-1ea159b9b23b","Type":"ContainerDied","Data":"2e4992cc2e8905b7fe793a8b028dc60f77b2fc53b9e9aa87bd2e3fbe6b4f4d4c"} Dec 03 12:40:58 crc kubenswrapper[5052]: E1203 12:40:58.492524 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of db6fc581a8df4ec1884c71516e39b9792e494f950b83f3b0b693c01758dfa0f1 is running failed: container process not found" containerID="db6fc581a8df4ec1884c71516e39b9792e494f950b83f3b0b693c01758dfa0f1" cmd=["/usr/bin/pidof","ovsdb-server"] Dec 03 12:40:58 crc kubenswrapper[5052]: E1203 12:40:58.494637 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of db6fc581a8df4ec1884c71516e39b9792e494f950b83f3b0b693c01758dfa0f1 is running failed: container process not found" containerID="db6fc581a8df4ec1884c71516e39b9792e494f950b83f3b0b693c01758dfa0f1" cmd=["/usr/bin/pidof","ovsdb-server"] Dec 03 12:40:58 crc kubenswrapper[5052]: E1203 12:40:58.495406 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of db6fc581a8df4ec1884c71516e39b9792e494f950b83f3b0b693c01758dfa0f1 is running failed: container process not found" containerID="db6fc581a8df4ec1884c71516e39b9792e494f950b83f3b0b693c01758dfa0f1" cmd=["/usr/bin/pidof","ovsdb-server"] Dec 03 12:40:58 crc kubenswrapper[5052]: E1203 12:40:58.495442 5052 prober.go:104] "Probe errored" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of db6fc581a8df4ec1884c71516e39b9792e494f950b83f3b0b693c01758dfa0f1 is running failed: container process not found" probeType="Readiness" pod="openstack-kuttl-tests/ovsdbserver-nb-0" podUID="46c590ef-2b7c-412d-8582-dfdc311e6778" containerName="ovsdbserver-nb" Dec 03 12:40:58 crc kubenswrapper[5052]: I1203 12:40:58.509625 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/novaapib163-account-delete-8z4bt"] Dec 03 12:40:58 crc kubenswrapper[5052]: I1203 12:40:58.518183 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell0-cell-mapping-x6pl4"] Dec 03 12:40:58 crc kubenswrapper[5052]: I1203 12:40:58.531588 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell0-cell-mapping-x6pl4"] Dec 03 12:40:58 crc kubenswrapper[5052]: I1203 12:40:58.541302 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/placement-cdf4456f4-v82rz"] Dec 03 12:40:58 crc kubenswrapper[5052]: I1203 12:40:58.541539 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/placement-cdf4456f4-v82rz" podUID="99a0961b-8ed8-49b7-abac-5025d7b351ee" containerName="placement-log" containerID="cri-o://995bc9f60d05257f8493e1dd6196163744598cee55f2e09b16470fc6e4d9962b" gracePeriod=30 Dec 03 12:40:58 crc kubenswrapper[5052]: I1203 12:40:58.541987 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/placement-cdf4456f4-v82rz" podUID="99a0961b-8ed8-49b7-abac-5025d7b351ee" containerName="placement-api" containerID="cri-o://60eb1509e2cea82540bce951978d0b80c496f80fe40a218c77ecbe97a3e9c835" gracePeriod=30 Dec 03 12:40:58 crc kubenswrapper[5052]: I1203 12:40:58.549634 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 12:40:58 crc kubenswrapper[5052]: I1203 12:40:58.549881 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/cinder-api-0" podUID="5bca7a3b-316e-4357-8ae4-e47be319dbf1" containerName="cinder-api-log" containerID="cri-o://15342f245f3b466f27835c2c30a7036b0d06092684492805d84db0575d5b165a" gracePeriod=30 Dec 03 12:40:58 crc kubenswrapper[5052]: I1203 12:40:58.551336 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/cinder-api-0" podUID="5bca7a3b-316e-4357-8ae4-e47be319dbf1" containerName="cinder-api" containerID="cri-o://87d55b80754e67e2615d7cea19e5d57d82b054e86c70bcb48fc87f5b7efa28d9" gracePeriod=30 Dec 03 12:40:58 crc kubenswrapper[5052]: I1203 12:40:58.555321 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/4b7b82d8-699c-43f2-8ab0-01936305623f-operator-scripts\") pod \"novaapib163-account-delete-8z4bt\" (UID: \"4b7b82d8-699c-43f2-8ab0-01936305623f\") " pod="openstack-kuttl-tests/novaapib163-account-delete-8z4bt" Dec 03 12:40:58 crc kubenswrapper[5052]: E1203 12:40:58.555772 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/rabbitmq-cell1-config-data: configmap "rabbitmq-cell1-config-data" not found Dec 03 12:40:58 crc kubenswrapper[5052]: E1203 12:40:58.555877 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f9bac64c-518e-4630-8889-20e09d5b7a39-config-data podName:f9bac64c-518e-4630-8889-20e09d5b7a39 nodeName:}" failed. No retries permitted until 2025-12-03 12:41:00.555859802 +0000 UTC m=+4851.144101556 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "config-data" (UniqueName: "kubernetes.io/configmap/f9bac64c-518e-4630-8889-20e09d5b7a39-config-data") pod "rabbitmq-cell1-server-0" (UID: "f9bac64c-518e-4630-8889-20e09d5b7a39") : configmap "rabbitmq-cell1-config-data" not found Dec 03 12:40:58 crc kubenswrapper[5052]: I1203 12:40:58.556700 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-4tjwn\" (UniqueName: \"kubernetes.io/projected/6dbc7d8f-34d4-4927-82fe-d049727808fd-kube-api-access-4tjwn\") pod \"novacell07298-account-delete-726jv\" (UID: \"6dbc7d8f-34d4-4927-82fe-d049727808fd\") " pod="openstack-kuttl-tests/novacell07298-account-delete-726jv" Dec 03 12:40:58 crc kubenswrapper[5052]: I1203 12:40:58.556849 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/6dbc7d8f-34d4-4927-82fe-d049727808fd-operator-scripts\") pod \"novacell07298-account-delete-726jv\" (UID: \"6dbc7d8f-34d4-4927-82fe-d049727808fd\") " pod="openstack-kuttl-tests/novacell07298-account-delete-726jv" Dec 03 12:40:58 crc kubenswrapper[5052]: I1203 12:40:58.557161 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8snr7\" (UniqueName: \"kubernetes.io/projected/4b7b82d8-699c-43f2-8ab0-01936305623f-kube-api-access-8snr7\") pod \"novaapib163-account-delete-8z4bt\" (UID: \"4b7b82d8-699c-43f2-8ab0-01936305623f\") " pod="openstack-kuttl-tests/novaapib163-account-delete-8z4bt" Dec 03 12:40:58 crc kubenswrapper[5052]: I1203 12:40:58.560748 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/6dbc7d8f-34d4-4927-82fe-d049727808fd-operator-scripts\") pod \"novacell07298-account-delete-726jv\" (UID: \"6dbc7d8f-34d4-4927-82fe-d049727808fd\") " pod="openstack-kuttl-tests/novacell07298-account-delete-726jv" Dec 03 12:40:58 crc kubenswrapper[5052]: I1203 12:40:58.603651 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:40:58 crc kubenswrapper[5052]: I1203 12:40:58.603892 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-metadata-0" podUID="30045a80-9e2b-416b-b7bf-cdb9585ec67d" containerName="nova-metadata-log" containerID="cri-o://c2aa910114395115673efcfd179c79ba17ccb8368f38abf250fd9ed6719330d7" gracePeriod=30 Dec 03 12:40:58 crc kubenswrapper[5052]: I1203 12:40:58.604359 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-metadata-0" podUID="30045a80-9e2b-416b-b7bf-cdb9585ec67d" containerName="nova-metadata-metadata" containerID="cri-o://0d3fa6f69a8171deaa21727301ede51856b6793ff9d13fde6cbe39442a2e4566" gracePeriod=30 Dec 03 12:40:58 crc kubenswrapper[5052]: I1203 12:40:58.615084 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/openstack-cell1-galera-0"] Dec 03 12:40:58 crc kubenswrapper[5052]: I1203 12:40:58.655377 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:40:58 crc kubenswrapper[5052]: I1203 12:40:58.655661 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-api-0" podUID="50789f13-8194-4e24-b975-512591b816a5" containerName="nova-api-log" containerID="cri-o://f1298bb0944b7faf489bfe13489509679a09bf41f1f58deeacfdf2aae7d628e8" gracePeriod=30 Dec 03 12:40:58 crc kubenswrapper[5052]: I1203 12:40:58.655962 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-api-0" podUID="50789f13-8194-4e24-b975-512591b816a5" containerName="nova-api-api" containerID="cri-o://256ac84e475433107bd853223013a808d99354e60928d4ce86ddc71170d1b9a7" gracePeriod=30 Dec 03 12:40:58 crc kubenswrapper[5052]: I1203 12:40:58.658762 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-8snr7\" (UniqueName: \"kubernetes.io/projected/4b7b82d8-699c-43f2-8ab0-01936305623f-kube-api-access-8snr7\") pod \"novaapib163-account-delete-8z4bt\" (UID: \"4b7b82d8-699c-43f2-8ab0-01936305623f\") " pod="openstack-kuttl-tests/novaapib163-account-delete-8z4bt" Dec 03 12:40:58 crc kubenswrapper[5052]: I1203 12:40:58.658925 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/4b7b82d8-699c-43f2-8ab0-01936305623f-operator-scripts\") pod \"novaapib163-account-delete-8z4bt\" (UID: \"4b7b82d8-699c-43f2-8ab0-01936305623f\") " pod="openstack-kuttl-tests/novaapib163-account-delete-8z4bt" Dec 03 12:40:58 crc kubenswrapper[5052]: I1203 12:40:58.660401 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/4b7b82d8-699c-43f2-8ab0-01936305623f-operator-scripts\") pod \"novaapib163-account-delete-8z4bt\" (UID: \"4b7b82d8-699c-43f2-8ab0-01936305623f\") " pod="openstack-kuttl-tests/novaapib163-account-delete-8z4bt" Dec 03 12:40:58 crc kubenswrapper[5052]: I1203 12:40:58.671892 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-keystone-listener-74b68b767b-8zpnm"] Dec 03 12:40:58 crc kubenswrapper[5052]: I1203 12:40:58.672804 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/barbican-keystone-listener-74b68b767b-8zpnm" podUID="bc0b4316-9c87-4c72-bc06-b4b526c2070e" containerName="barbican-keystone-listener-log" containerID="cri-o://f6309d5be7e4db13ae452596e6a4fcddfc94ab7011d5e22b0f3ce37272e5a6eb" gracePeriod=30 Dec 03 12:40:58 crc kubenswrapper[5052]: I1203 12:40:58.673321 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/barbican-keystone-listener-74b68b767b-8zpnm" podUID="bc0b4316-9c87-4c72-bc06-b4b526c2070e" containerName="barbican-keystone-listener" containerID="cri-o://a19585c85edfd20edc4392a9229786e6495d1e999b33f47c80f6aec8b04d2bf2" gracePeriod=30 Dec 03 12:40:58 crc kubenswrapper[5052]: I1203 12:40:58.690413 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-worker-7bd785845-99jqj"] Dec 03 12:40:58 crc kubenswrapper[5052]: I1203 12:40:58.690746 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/barbican-worker-7bd785845-99jqj" podUID="18a62c9b-4032-4639-be99-3520b515313b" containerName="barbican-worker-log" containerID="cri-o://c45f971177244262381100c68ce2c891e45f06d82ee0da8bc330ce4a83260099" gracePeriod=30 Dec 03 12:40:58 crc kubenswrapper[5052]: I1203 12:40:58.691160 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/barbican-worker-7bd785845-99jqj" podUID="18a62c9b-4032-4639-be99-3520b515313b" containerName="barbican-worker" containerID="cri-o://0aea18d8ca92744fcf1bb8068e5ff8a5eb48807d9cf65983d90eb513bcd33ddb" gracePeriod=30 Dec 03 12:40:58 crc kubenswrapper[5052]: I1203 12:40:58.702559 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/rabbitmq-server-0"] Dec 03 12:40:58 crc kubenswrapper[5052]: I1203 12:40:58.730145 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell1-db-create-22ctd"] Dec 03 12:40:58 crc kubenswrapper[5052]: I1203 12:40:58.739835 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell1-db-create-22ctd"] Dec 03 12:40:58 crc kubenswrapper[5052]: I1203 12:40:58.754739 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-dwdpg\" (UniqueName: \"kubernetes.io/projected/296f8cec-ad7b-4b51-a9a9-934301d43d2e-kube-api-access-dwdpg\") pod \"dnsmasq-dnsmasq-84b9f45d47-8h8qz\" (UID: \"296f8cec-ad7b-4b51-a9a9-934301d43d2e\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-8h8qz" Dec 03 12:40:58 crc kubenswrapper[5052]: I1203 12:40:58.756648 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-4tjwn\" (UniqueName: \"kubernetes.io/projected/6dbc7d8f-34d4-4927-82fe-d049727808fd-kube-api-access-4tjwn\") pod \"novacell07298-account-delete-726jv\" (UID: \"6dbc7d8f-34d4-4927-82fe-d049727808fd\") " pod="openstack-kuttl-tests/novacell07298-account-delete-726jv" Dec 03 12:40:58 crc kubenswrapper[5052]: I1203 12:40:58.759121 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell1-692e-account-create-update-44rrb"] Dec 03 12:40:58 crc kubenswrapper[5052]: I1203 12:40:58.759882 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-8snr7\" (UniqueName: \"kubernetes.io/projected/4b7b82d8-699c-43f2-8ab0-01936305623f-kube-api-access-8snr7\") pod \"novaapib163-account-delete-8z4bt\" (UID: \"4b7b82d8-699c-43f2-8ab0-01936305623f\") " pod="openstack-kuttl-tests/novaapib163-account-delete-8z4bt" Dec 03 12:40:58 crc kubenswrapper[5052]: I1203 12:40:58.760313 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-59qch\" (UniqueName: \"kubernetes.io/projected/ab29f82e-0693-4001-9f2a-bb4cc2fa1d5a-kube-api-access-59qch\") pod \"barbican1171-account-delete-9hmgl\" (UID: \"ab29f82e-0693-4001-9f2a-bb4cc2fa1d5a\") " pod="openstack-kuttl-tests/barbican1171-account-delete-9hmgl" Dec 03 12:40:58 crc kubenswrapper[5052]: I1203 12:40:58.792378 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell1-692e-account-create-update-44rrb"] Dec 03 12:40:58 crc kubenswrapper[5052]: I1203 12:40:58.804182 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/neutron-7d74684fd-pqkf5" podUID="ddfa4a3f-b868-49b7-abec-62cb3ffde2bd" containerName="neutron-httpd" probeResult="failure" output="Get \"https://10.217.1.127:9696/\": dial tcp 10.217.1.127:9696: connect: connection refused" Dec 03 12:40:58 crc kubenswrapper[5052]: I1203 12:40:58.804460 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-api-bcdbf47bb-m2ctz"] Dec 03 12:40:58 crc kubenswrapper[5052]: I1203 12:40:58.804674 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/barbican-api-bcdbf47bb-m2ctz" podUID="a83592aa-a4c1-459e-9d07-f14b390a5e6b" containerName="barbican-api-log" containerID="cri-o://7af822baa4982fe79fa48809692cdf9c52486cac04ab62564e24484f37f0039e" gracePeriod=30 Dec 03 12:40:58 crc kubenswrapper[5052]: I1203 12:40:58.805049 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/barbican-api-bcdbf47bb-m2ctz" podUID="a83592aa-a4c1-459e-9d07-f14b390a5e6b" containerName="barbican-api" containerID="cri-o://be69f0c47ca79cbf547e094fd8c2d7c0c191112e45d357d7916c62c184aa277b" gracePeriod=30 Dec 03 12:40:58 crc kubenswrapper[5052]: I1203 12:40:58.808770 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 12:40:58 crc kubenswrapper[5052]: I1203 12:40:58.808985 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" podUID="0f16f521-a181-40e4-a1ed-38f4c8862ffb" containerName="nova-cell1-novncproxy-novncproxy" containerID="cri-o://49e65401462a62ad6b62529bdc792c44fb0d7a07381a58a73215f64add3d28c5" gracePeriod=30 Dec 03 12:40:58 crc kubenswrapper[5052]: I1203 12:40:58.850626 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" podUID="f9bac64c-518e-4630-8889-20e09d5b7a39" containerName="rabbitmq" containerID="cri-o://ed967e0e50724d3854ca971b7a125ee6c019b59d594a733d91f837aeb86bc924" gracePeriod=604800 Dec 03 12:40:58 crc kubenswrapper[5052]: I1203 12:40:58.921506 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutrond24e-account-delete-p2mpp"] Dec 03 12:40:58 crc kubenswrapper[5052]: I1203 12:40:58.960165 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glancec94f-account-delete-8t6wx"] Dec 03 12:40:58 crc kubenswrapper[5052]: E1203 12:40:58.969572 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/rabbitmq-config-data: configmap "rabbitmq-config-data" not found Dec 03 12:40:58 crc kubenswrapper[5052]: E1203 12:40:58.969648 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d82ef8ea-0c3b-4d89-9c78-22095940f4da-config-data podName:d82ef8ea-0c3b-4d89-9c78-22095940f4da nodeName:}" failed. No retries permitted until 2025-12-03 12:41:00.969632285 +0000 UTC m=+4851.557874049 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "config-data" (UniqueName: "kubernetes.io/configmap/d82ef8ea-0c3b-4d89-9c78-22095940f4da-config-data") pod "rabbitmq-server-0" (UID: "d82ef8ea-0c3b-4d89-9c78-22095940f4da") : configmap "rabbitmq-config-data" not found Dec 03 12:40:58 crc kubenswrapper[5052]: I1203 12:40:58.970041 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/rabbitmq-server-0" podUID="d82ef8ea-0c3b-4d89-9c78-22095940f4da" containerName="rabbitmq" containerID="cri-o://ed184550d5a8d5611c9220d80018d6040a97d60ffbd768b55b87c5964020d439" gracePeriod=604800 Dec 03 12:40:59 crc kubenswrapper[5052]: E1203 12:40:59.087041 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of 87e14300e8697a4e0c8594bb1dde5f610d618d2da77d5478a1bb33029c23a5e0 is running failed: container process not found" containerID="87e14300e8697a4e0c8594bb1dde5f610d618d2da77d5478a1bb33029c23a5e0" cmd=["/usr/bin/pidof","ovsdb-server"] Dec 03 12:40:59 crc kubenswrapper[5052]: E1203 12:40:59.087724 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of 87e14300e8697a4e0c8594bb1dde5f610d618d2da77d5478a1bb33029c23a5e0 is running failed: container process not found" containerID="87e14300e8697a4e0c8594bb1dde5f610d618d2da77d5478a1bb33029c23a5e0" cmd=["/usr/bin/pidof","ovsdb-server"] Dec 03 12:40:59 crc kubenswrapper[5052]: E1203 12:40:59.087913 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of 87e14300e8697a4e0c8594bb1dde5f610d618d2da77d5478a1bb33029c23a5e0 is running failed: container process not found" containerID="87e14300e8697a4e0c8594bb1dde5f610d618d2da77d5478a1bb33029c23a5e0" cmd=["/usr/bin/pidof","ovsdb-server"] Dec 03 12:40:59 crc kubenswrapper[5052]: E1203 12:40:59.087944 5052 prober.go:104] "Probe errored" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of 87e14300e8697a4e0c8594bb1dde5f610d618d2da77d5478a1bb33029c23a5e0 is running failed: container process not found" probeType="Readiness" pod="openstack-kuttl-tests/ovsdbserver-sb-0" podUID="d6bedb3e-52e6-44a9-afd9-a164b0730f52" containerName="ovsdbserver-sb" Dec 03 12:40:59 crc kubenswrapper[5052]: I1203 12:40:59.120236 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/kube-state-metrics-0"] Dec 03 12:40:59 crc kubenswrapper[5052]: I1203 12:40:59.120441 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/kube-state-metrics-0" podUID="b7defe22-28b4-4fec-8fe5-ebde8e658c6b" containerName="kube-state-metrics" containerID="cri-o://26de7dd666c3dea50b83a35cff57a4ece7ec413a749af698ab13d88f7a800f76" gracePeriod=30 Dec 03 12:40:59 crc kubenswrapper[5052]: I1203 12:40:59.206278 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:40:59 crc kubenswrapper[5052]: I1203 12:40:59.207627 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="65cb22ea-3a9c-4f99-9128-d3c7681c3173" containerName="ceilometer-central-agent" containerID="cri-o://d970c5d7874c5afe76962513bf4d336686b29e37d400962b8518fcae8ce933fc" gracePeriod=30 Dec 03 12:40:59 crc kubenswrapper[5052]: I1203 12:40:59.208102 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="65cb22ea-3a9c-4f99-9128-d3c7681c3173" containerName="proxy-httpd" containerID="cri-o://f0b7b9f45f694a8501baf61dbded87068f05ef0463c5c28e5e6db8782cc546cc" gracePeriod=30 Dec 03 12:40:59 crc kubenswrapper[5052]: I1203 12:40:59.208293 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="65cb22ea-3a9c-4f99-9128-d3c7681c3173" containerName="ceilometer-notification-agent" containerID="cri-o://b7bf0dcb2de110437bf240dda6407da49398f97022270b268197a8605493c273" gracePeriod=30 Dec 03 12:40:59 crc kubenswrapper[5052]: I1203 12:40:59.208337 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="65cb22ea-3a9c-4f99-9128-d3c7681c3173" containerName="sg-core" containerID="cri-o://1bc0a87b8c7078c46986fb762763f8730d5895f38f884a95a89a15f1527106de" gracePeriod=30 Dec 03 12:40:59 crc kubenswrapper[5052]: I1203 12:40:59.269307 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-db-sync-2v6n2"] Dec 03 12:40:59 crc kubenswrapper[5052]: E1203 12:40:59.285672 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-nova-internal-svc: secret "cert-nova-internal-svc" not found Dec 03 12:40:59 crc kubenswrapper[5052]: E1203 12:40:59.285755 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/50789f13-8194-4e24-b975-512591b816a5-internal-tls-certs podName:50789f13-8194-4e24-b975-512591b816a5 nodeName:}" failed. No retries permitted until 2025-12-03 12:41:01.285737754 +0000 UTC m=+4851.873979518 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "internal-tls-certs" (UniqueName: "kubernetes.io/secret/50789f13-8194-4e24-b975-512591b816a5-internal-tls-certs") pod "nova-api-0" (UID: "50789f13-8194-4e24-b975-512591b816a5") : secret "cert-nova-internal-svc" not found Dec 03 12:40:59 crc kubenswrapper[5052]: E1203 12:40:59.285871 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-nova-public-svc: secret "cert-nova-public-svc" not found Dec 03 12:40:59 crc kubenswrapper[5052]: E1203 12:40:59.285952 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/50789f13-8194-4e24-b975-512591b816a5-public-tls-certs podName:50789f13-8194-4e24-b975-512591b816a5 nodeName:}" failed. No retries permitted until 2025-12-03 12:41:01.28593851 +0000 UTC m=+4851.874180274 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "public-tls-certs" (UniqueName: "kubernetes.io/secret/50789f13-8194-4e24-b975-512591b816a5-public-tls-certs") pod "nova-api-0" (UID: "50789f13-8194-4e24-b975-512591b816a5") : secret "cert-nova-public-svc" not found Dec 03 12:40:59 crc kubenswrapper[5052]: E1203 12:40:59.286014 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/combined-ca-bundle: secret "combined-ca-bundle" not found Dec 03 12:40:59 crc kubenswrapper[5052]: E1203 12:40:59.286035 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/50789f13-8194-4e24-b975-512591b816a5-combined-ca-bundle podName:50789f13-8194-4e24-b975-512591b816a5 nodeName:}" failed. No retries permitted until 2025-12-03 12:41:01.286028183 +0000 UTC m=+4851.874269947 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "combined-ca-bundle" (UniqueName: "kubernetes.io/secret/50789f13-8194-4e24-b975-512591b816a5-combined-ca-bundle") pod "nova-api-0" (UID: "50789f13-8194-4e24-b975-512591b816a5") : secret "combined-ca-bundle" not found Dec 03 12:40:59 crc kubenswrapper[5052]: I1203 12:40:59.294385 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-0"] Dec 03 12:40:59 crc kubenswrapper[5052]: I1203 12:40:59.294992 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-cell1-conductor-0" podUID="0556e9f0-e431-4193-9822-6079cb8a1e44" containerName="nova-cell1-conductor-conductor" containerID="cri-o://382891affaf25ff9b098ddaf284f3fa5667c14b93f3487974c23c2cdae354300" gracePeriod=30 Dec 03 12:40:59 crc kubenswrapper[5052]: I1203 12:40:59.323100 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-db-sync-2v6n2"] Dec 03 12:40:59 crc kubenswrapper[5052]: I1203 12:40:59.337828 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-0"] Dec 03 12:40:59 crc kubenswrapper[5052]: I1203 12:40:59.337927 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/openstack-cell1-galera-0" podUID="f89ce0f5-5a92-49eb-9951-f2b491ac2139" containerName="galera" containerID="cri-o://0c3d8b3ca8a9d7d4611f59d9c0bcfa83d8ea56d3df8caa14069f56e2580808ef" gracePeriod=30 Dec 03 12:40:59 crc kubenswrapper[5052]: I1203 12:40:59.338054 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-cell0-conductor-0" podUID="e9af392d-1940-4203-961e-39da1ac5f3da" containerName="nova-cell0-conductor-conductor" containerID="cri-o://88bb1e9d52b42c7f2f99966c163df2f71ff6d2b3bec16e67d60c4dbcaea76bb6" gracePeriod=30 Dec 03 12:40:59 crc kubenswrapper[5052]: I1203 12:40:59.356052 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/kube-state-metrics-0" podUID="b7defe22-28b4-4fec-8fe5-ebde8e658c6b" containerName="kube-state-metrics" probeResult="failure" output="Get \"https://10.217.1.85:8081/readyz\": dial tcp 10.217.1.85:8081: connect: connection refused" Dec 03 12:40:59 crc kubenswrapper[5052]: I1203 12:40:59.362630 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-db-sync-dpcff"] Dec 03 12:40:59 crc kubenswrapper[5052]: I1203 12:40:59.382915 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-db-sync-dpcff"] Dec 03 12:40:59 crc kubenswrapper[5052]: I1203 12:40:59.410316 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:40:59 crc kubenswrapper[5052]: I1203 12:40:59.410529 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-scheduler-0" podUID="9e3bfbf3-a39c-48d2-bb1d-062c90c0ef10" containerName="nova-scheduler-scheduler" containerID="cri-o://b1ff1f57b80e78690f7e46060a438671dfcfacd08f98a788c2bd60d1eb2bbdf1" gracePeriod=30 Dec 03 12:40:59 crc kubenswrapper[5052]: I1203 12:40:59.539482 5052 generic.go:334] "Generic (PLEG): container finished" podID="65cb22ea-3a9c-4f99-9128-d3c7681c3173" containerID="f0b7b9f45f694a8501baf61dbded87068f05ef0463c5c28e5e6db8782cc546cc" exitCode=0 Dec 03 12:40:59 crc kubenswrapper[5052]: I1203 12:40:59.539814 5052 generic.go:334] "Generic (PLEG): container finished" podID="65cb22ea-3a9c-4f99-9128-d3c7681c3173" containerID="1bc0a87b8c7078c46986fb762763f8730d5895f38f884a95a89a15f1527106de" exitCode=2 Dec 03 12:40:59 crc kubenswrapper[5052]: I1203 12:40:59.539915 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"65cb22ea-3a9c-4f99-9128-d3c7681c3173","Type":"ContainerDied","Data":"f0b7b9f45f694a8501baf61dbded87068f05ef0463c5c28e5e6db8782cc546cc"} Dec 03 12:40:59 crc kubenswrapper[5052]: I1203 12:40:59.539942 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"65cb22ea-3a9c-4f99-9128-d3c7681c3173","Type":"ContainerDied","Data":"1bc0a87b8c7078c46986fb762763f8730d5895f38f884a95a89a15f1527106de"} Dec 03 12:40:59 crc kubenswrapper[5052]: I1203 12:40:59.572709 5052 generic.go:334] "Generic (PLEG): container finished" podID="bc0b4316-9c87-4c72-bc06-b4b526c2070e" containerID="f6309d5be7e4db13ae452596e6a4fcddfc94ab7011d5e22b0f3ce37272e5a6eb" exitCode=143 Dec 03 12:40:59 crc kubenswrapper[5052]: I1203 12:40:59.572798 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-keystone-listener-74b68b767b-8zpnm" event={"ID":"bc0b4316-9c87-4c72-bc06-b4b526c2070e","Type":"ContainerDied","Data":"f6309d5be7e4db13ae452596e6a4fcddfc94ab7011d5e22b0f3ce37272e5a6eb"} Dec 03 12:40:59 crc kubenswrapper[5052]: I1203 12:40:59.586717 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutrond24e-account-delete-p2mpp" event={"ID":"12f52069-8b08-4e4f-a3c1-a5421efb5844","Type":"ContainerStarted","Data":"713211a2bad96f3f8994a25e2a54078414fa46ad3e231c79400c33e319e58013"} Dec 03 12:40:59 crc kubenswrapper[5052]: I1203 12:40:59.602136 5052 generic.go:334] "Generic (PLEG): container finished" podID="18a62c9b-4032-4639-be99-3520b515313b" containerID="c45f971177244262381100c68ce2c891e45f06d82ee0da8bc330ce4a83260099" exitCode=143 Dec 03 12:40:59 crc kubenswrapper[5052]: I1203 12:40:59.602190 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-worker-7bd785845-99jqj" event={"ID":"18a62c9b-4032-4639-be99-3520b515313b","Type":"ContainerDied","Data":"c45f971177244262381100c68ce2c891e45f06d82ee0da8bc330ce4a83260099"} Dec 03 12:40:59 crc kubenswrapper[5052]: I1203 12:40:59.612561 5052 generic.go:334] "Generic (PLEG): container finished" podID="b7defe22-28b4-4fec-8fe5-ebde8e658c6b" containerID="26de7dd666c3dea50b83a35cff57a4ece7ec413a749af698ab13d88f7a800f76" exitCode=2 Dec 03 12:40:59 crc kubenswrapper[5052]: I1203 12:40:59.612647 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/kube-state-metrics-0" event={"ID":"b7defe22-28b4-4fec-8fe5-ebde8e658c6b","Type":"ContainerDied","Data":"26de7dd666c3dea50b83a35cff57a4ece7ec413a749af698ab13d88f7a800f76"} Dec 03 12:40:59 crc kubenswrapper[5052]: I1203 12:40:59.619465 5052 generic.go:334] "Generic (PLEG): container finished" podID="5bca7a3b-316e-4357-8ae4-e47be319dbf1" containerID="15342f245f3b466f27835c2c30a7036b0d06092684492805d84db0575d5b165a" exitCode=143 Dec 03 12:40:59 crc kubenswrapper[5052]: I1203 12:40:59.619559 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"5bca7a3b-316e-4357-8ae4-e47be319dbf1","Type":"ContainerDied","Data":"15342f245f3b466f27835c2c30a7036b0d06092684492805d84db0575d5b165a"} Dec 03 12:40:59 crc kubenswrapper[5052]: I1203 12:40:59.631486 5052 generic.go:334] "Generic (PLEG): container finished" podID="7ce91d4a-389d-4b37-b582-8c886566aa74" containerID="229df7ab571e9eb2bf196a218d66569b5405729090282a855310baf493b87409" exitCode=137 Dec 03 12:40:59 crc kubenswrapper[5052]: I1203 12:40:59.631572 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4ab86855c1c9a4b07f00dbffd2af9d9e18ba36b5823c06baeaeb2708e3bc2f03" Dec 03 12:40:59 crc kubenswrapper[5052]: I1203 12:40:59.651614 5052 generic.go:334] "Generic (PLEG): container finished" podID="d855c655-3b2f-4e4f-9b27-cf4e43a92045" containerID="323efac3746af7ff7702e46c55dedadf7e2d8d45f1a20d6b7d1fa2753913a05d" exitCode=0 Dec 03 12:40:59 crc kubenswrapper[5052]: I1203 12:40:59.651671 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"d855c655-3b2f-4e4f-9b27-cf4e43a92045","Type":"ContainerDied","Data":"323efac3746af7ff7702e46c55dedadf7e2d8d45f1a20d6b7d1fa2753913a05d"} Dec 03 12:40:59 crc kubenswrapper[5052]: I1203 12:40:59.660225 5052 generic.go:334] "Generic (PLEG): container finished" podID="a83592aa-a4c1-459e-9d07-f14b390a5e6b" containerID="7af822baa4982fe79fa48809692cdf9c52486cac04ab62564e24484f37f0039e" exitCode=143 Dec 03 12:40:59 crc kubenswrapper[5052]: I1203 12:40:59.660319 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-bcdbf47bb-m2ctz" event={"ID":"a83592aa-a4c1-459e-9d07-f14b390a5e6b","Type":"ContainerDied","Data":"7af822baa4982fe79fa48809692cdf9c52486cac04ab62564e24484f37f0039e"} Dec 03 12:40:59 crc kubenswrapper[5052]: I1203 12:40:59.664264 5052 generic.go:334] "Generic (PLEG): container finished" podID="30045a80-9e2b-416b-b7bf-cdb9585ec67d" containerID="c2aa910114395115673efcfd179c79ba17ccb8368f38abf250fd9ed6719330d7" exitCode=143 Dec 03 12:40:59 crc kubenswrapper[5052]: I1203 12:40:59.664356 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"30045a80-9e2b-416b-b7bf-cdb9585ec67d","Type":"ContainerDied","Data":"c2aa910114395115673efcfd179c79ba17ccb8368f38abf250fd9ed6719330d7"} Dec 03 12:40:59 crc kubenswrapper[5052]: I1203 12:40:59.680224 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_ovsdbserver-nb-0_46c590ef-2b7c-412d-8582-dfdc311e6778/ovsdbserver-nb/0.log" Dec 03 12:40:59 crc kubenswrapper[5052]: I1203 12:40:59.680317 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-nb-0" event={"ID":"46c590ef-2b7c-412d-8582-dfdc311e6778","Type":"ContainerDied","Data":"7ff432b7958a2acdc47a781956d13dd5c2c28341d37af9335f8ed7019cdf4272"} Dec 03 12:40:59 crc kubenswrapper[5052]: I1203 12:40:59.680345 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7ff432b7958a2acdc47a781956d13dd5c2c28341d37af9335f8ed7019cdf4272" Dec 03 12:40:59 crc kubenswrapper[5052]: I1203 12:40:59.693436 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_ovsdbserver-sb-0_d6bedb3e-52e6-44a9-afd9-a164b0730f52/ovsdbserver-sb/0.log" Dec 03 12:40:59 crc kubenswrapper[5052]: I1203 12:40:59.702330 5052 generic.go:334] "Generic (PLEG): container finished" podID="99a0961b-8ed8-49b7-abac-5025d7b351ee" containerID="995bc9f60d05257f8493e1dd6196163744598cee55f2e09b16470fc6e4d9962b" exitCode=143 Dec 03 12:40:59 crc kubenswrapper[5052]: I1203 12:40:59.703975 5052 generic.go:334] "Generic (PLEG): container finished" podID="ddfa4a3f-b868-49b7-abec-62cb3ffde2bd" containerID="8ed93a1c3257ab19479e68e87c18d74af55fe2f519944a6fbeee4da11170b261" exitCode=0 Dec 03 12:40:59 crc kubenswrapper[5052]: I1203 12:40:59.707559 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="32d4c4ba-c6dd-49cf-80a3-4d7ed71cb6b5" path="/var/lib/kubelet/pods/32d4c4ba-c6dd-49cf-80a3-4d7ed71cb6b5/volumes" Dec 03 12:40:59 crc kubenswrapper[5052]: I1203 12:40:59.708197 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="47136a96-c6f5-428d-a11a-fa9eeb4910e7" path="/var/lib/kubelet/pods/47136a96-c6f5-428d-a11a-fa9eeb4910e7/volumes" Dec 03 12:40:59 crc kubenswrapper[5052]: I1203 12:40:59.708985 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="51f3388b-292f-40a9-9d41-ad60bb51ab68" path="/var/lib/kubelet/pods/51f3388b-292f-40a9-9d41-ad60bb51ab68/volumes" Dec 03 12:40:59 crc kubenswrapper[5052]: I1203 12:40:59.717313 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="55ccf323-53e4-47fe-bea8-a17273b8587c" path="/var/lib/kubelet/pods/55ccf323-53e4-47fe-bea8-a17273b8587c/volumes" Dec 03 12:40:59 crc kubenswrapper[5052]: I1203 12:40:59.718624 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="63821a5d-93bd-41a3-9c7d-979723bfc035" path="/var/lib/kubelet/pods/63821a5d-93bd-41a3-9c7d-979723bfc035/volumes" Dec 03 12:40:59 crc kubenswrapper[5052]: I1203 12:40:59.719274 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="772e7623-3c44-4375-9f34-5ba8683d5fe1" path="/var/lib/kubelet/pods/772e7623-3c44-4375-9f34-5ba8683d5fe1/volumes" Dec 03 12:40:59 crc kubenswrapper[5052]: I1203 12:40:59.725535 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a4de7484-329f-412e-b365-e373a4cab966" path="/var/lib/kubelet/pods/a4de7484-329f-412e-b365-e373a4cab966/volumes" Dec 03 12:40:59 crc kubenswrapper[5052]: I1203 12:40:59.726195 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c2dfe7a5-4ce2-4d6e-98e5-3ea6a4d3f3ca" path="/var/lib/kubelet/pods/c2dfe7a5-4ce2-4d6e-98e5-3ea6a4d3f3ca/volumes" Dec 03 12:40:59 crc kubenswrapper[5052]: I1203 12:40:59.727660 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d9523127-c2ea-4f5f-98ae-0288681fb93c" path="/var/lib/kubelet/pods/d9523127-c2ea-4f5f-98ae-0288681fb93c/volumes" Dec 03 12:40:59 crc kubenswrapper[5052]: I1203 12:40:59.729003 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f3af01db-f2de-4cd4-aeda-3cc200498cc7" path="/var/lib/kubelet/pods/f3af01db-f2de-4cd4-aeda-3cc200498cc7/volumes" Dec 03 12:40:59 crc kubenswrapper[5052]: I1203 12:40:59.729988 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-sb-0" event={"ID":"d6bedb3e-52e6-44a9-afd9-a164b0730f52","Type":"ContainerDied","Data":"ac810b46f2960a4c26ef9798138b1d1d954bd6def2c577bfa4e87659f07846f6"} Dec 03 12:40:59 crc kubenswrapper[5052]: I1203 12:40:59.730041 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ac810b46f2960a4c26ef9798138b1d1d954bd6def2c577bfa4e87659f07846f6" Dec 03 12:40:59 crc kubenswrapper[5052]: I1203 12:40:59.730056 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glancec94f-account-delete-8t6wx" event={"ID":"b7c49d90-c0af-4d4b-a080-8c7dce3ee66a","Type":"ContainerStarted","Data":"d7a710caf6f50b57a6b12dca0e90d790995d331e34888adedbd735d99df15c2d"} Dec 03 12:40:59 crc kubenswrapper[5052]: I1203 12:40:59.730069 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-cdf4456f4-v82rz" event={"ID":"99a0961b-8ed8-49b7-abac-5025d7b351ee","Type":"ContainerDied","Data":"995bc9f60d05257f8493e1dd6196163744598cee55f2e09b16470fc6e4d9962b"} Dec 03 12:40:59 crc kubenswrapper[5052]: I1203 12:40:59.730102 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-7d74684fd-pqkf5" event={"ID":"ddfa4a3f-b868-49b7-abec-62cb3ffde2bd","Type":"ContainerDied","Data":"8ed93a1c3257ab19479e68e87c18d74af55fe2f519944a6fbeee4da11170b261"} Dec 03 12:40:59 crc kubenswrapper[5052]: I1203 12:40:59.759744 5052 generic.go:334] "Generic (PLEG): container finished" podID="c7d598df-f336-4798-b381-9b813068f271" containerID="34d7b3c41a14d274d9f82dc6e97d783bfeea4af12baf996aefeb1b3be4f773dd" exitCode=0 Dec 03 12:40:59 crc kubenswrapper[5052]: I1203 12:40:59.759767 5052 generic.go:334] "Generic (PLEG): container finished" podID="c7d598df-f336-4798-b381-9b813068f271" containerID="67aad15627897c77a5ca09bba14532fa30813bf015f5d0017d2d2538795a5666" exitCode=0 Dec 03 12:40:59 crc kubenswrapper[5052]: I1203 12:40:59.759775 5052 generic.go:334] "Generic (PLEG): container finished" podID="c7d598df-f336-4798-b381-9b813068f271" containerID="490483344d3233731e445298ec4b3a6f3b3d6ba0822ed43ceb5cdbc637a73f2c" exitCode=0 Dec 03 12:40:59 crc kubenswrapper[5052]: I1203 12:40:59.759782 5052 generic.go:334] "Generic (PLEG): container finished" podID="c7d598df-f336-4798-b381-9b813068f271" containerID="7fd8d3e9755be0491e71f4771e79219dfa4803774a0535e6af7be76023acf022" exitCode=0 Dec 03 12:40:59 crc kubenswrapper[5052]: I1203 12:40:59.759831 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"c7d598df-f336-4798-b381-9b813068f271","Type":"ContainerDied","Data":"34d7b3c41a14d274d9f82dc6e97d783bfeea4af12baf996aefeb1b3be4f773dd"} Dec 03 12:40:59 crc kubenswrapper[5052]: I1203 12:40:59.759875 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"c7d598df-f336-4798-b381-9b813068f271","Type":"ContainerDied","Data":"67aad15627897c77a5ca09bba14532fa30813bf015f5d0017d2d2538795a5666"} Dec 03 12:40:59 crc kubenswrapper[5052]: I1203 12:40:59.759885 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"c7d598df-f336-4798-b381-9b813068f271","Type":"ContainerDied","Data":"490483344d3233731e445298ec4b3a6f3b3d6ba0822ed43ceb5cdbc637a73f2c"} Dec 03 12:40:59 crc kubenswrapper[5052]: I1203 12:40:59.759894 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"c7d598df-f336-4798-b381-9b813068f271","Type":"ContainerDied","Data":"7fd8d3e9755be0491e71f4771e79219dfa4803774a0535e6af7be76023acf022"} Dec 03 12:40:59 crc kubenswrapper[5052]: I1203 12:40:59.783119 5052 generic.go:334] "Generic (PLEG): container finished" podID="50789f13-8194-4e24-b975-512591b816a5" containerID="f1298bb0944b7faf489bfe13489509679a09bf41f1f58deeacfdf2aae7d628e8" exitCode=143 Dec 03 12:40:59 crc kubenswrapper[5052]: I1203 12:40:59.783161 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"50789f13-8194-4e24-b975-512591b816a5","Type":"ContainerDied","Data":"f1298bb0944b7faf489bfe13489509679a09bf41f1f58deeacfdf2aae7d628e8"} Dec 03 12:40:59 crc kubenswrapper[5052]: E1203 12:40:59.792995 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="88bb1e9d52b42c7f2f99966c163df2f71ff6d2b3bec16e67d60c4dbcaea76bb6" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:40:59 crc kubenswrapper[5052]: E1203 12:40:59.794869 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="88bb1e9d52b42c7f2f99966c163df2f71ff6d2b3bec16e67d60c4dbcaea76bb6" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:40:59 crc kubenswrapper[5052]: E1203 12:40:59.797886 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="88bb1e9d52b42c7f2f99966c163df2f71ff6d2b3bec16e67d60c4dbcaea76bb6" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:40:59 crc kubenswrapper[5052]: E1203 12:40:59.798103 5052 prober.go:104] "Probe errored" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" probeType="Readiness" pod="openstack-kuttl-tests/nova-cell0-conductor-0" podUID="e9af392d-1940-4203-961e-39da1ac5f3da" containerName="nova-cell0-conductor-conductor" Dec 03 12:40:59 crc kubenswrapper[5052]: I1203 12:40:59.899138 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder3a0d-account-delete-lxd6l" Dec 03 12:40:59 crc kubenswrapper[5052]: I1203 12:40:59.920566 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/swift-proxy-bf56d5474-cq7b2"] Dec 03 12:40:59 crc kubenswrapper[5052]: I1203 12:40:59.920852 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-proxy-bf56d5474-cq7b2" podUID="621935c3-f8f9-43a1-9bc3-6d6ad9caf0ef" containerName="proxy-httpd" containerID="cri-o://30dfc639d124cfef009d7624b44b757995f402edb14bbdfeb69e215f2a21b847" gracePeriod=30 Dec 03 12:40:59 crc kubenswrapper[5052]: I1203 12:40:59.921369 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-proxy-bf56d5474-cq7b2" podUID="621935c3-f8f9-43a1-9bc3-6d6ad9caf0ef" containerName="proxy-server" containerID="cri-o://53722c8da45292bfe9fdbd4b2d2a18b053dc900a020f10bc15c4643959d2c7c1" gracePeriod=30 Dec 03 12:40:59 crc kubenswrapper[5052]: I1203 12:40:59.990595 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement7f30-account-delete-vld4v" Dec 03 12:41:00 crc kubenswrapper[5052]: E1203 12:41:00.063338 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="b1ff1f57b80e78690f7e46060a438671dfcfacd08f98a788c2bd60d1eb2bbdf1" cmd=["/usr/bin/pgrep","-r","DRST","nova-scheduler"] Dec 03 12:41:00 crc kubenswrapper[5052]: E1203 12:41:00.073077 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="b1ff1f57b80e78690f7e46060a438671dfcfacd08f98a788c2bd60d1eb2bbdf1" cmd=["/usr/bin/pgrep","-r","DRST","nova-scheduler"] Dec 03 12:41:00 crc kubenswrapper[5052]: E1203 12:41:00.079125 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="b1ff1f57b80e78690f7e46060a438671dfcfacd08f98a788c2bd60d1eb2bbdf1" cmd=["/usr/bin/pgrep","-r","DRST","nova-scheduler"] Dec 03 12:41:00 crc kubenswrapper[5052]: E1203 12:41:00.079211 5052 prober.go:104] "Probe errored" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" probeType="Readiness" pod="openstack-kuttl-tests/nova-scheduler-0" podUID="9e3bfbf3-a39c-48d2-bb1d-062c90c0ef10" containerName="nova-scheduler-scheduler" Dec 03 12:41:00 crc kubenswrapper[5052]: I1203 12:41:00.080900 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican1171-account-delete-9hmgl" Dec 03 12:41:00 crc kubenswrapper[5052]: I1203 12:41:00.092909 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-8h8qz" Dec 03 12:41:00 crc kubenswrapper[5052]: I1203 12:41:00.110305 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/novacell07298-account-delete-726jv" Dec 03 12:41:00 crc kubenswrapper[5052]: I1203 12:41:00.124943 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/novaapib163-account-delete-8z4bt" Dec 03 12:41:00 crc kubenswrapper[5052]: I1203 12:41:00.166990 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_ovsdbserver-sb-0_d6bedb3e-52e6-44a9-afd9-a164b0730f52/ovsdbserver-sb/0.log" Dec 03 12:41:00 crc kubenswrapper[5052]: I1203 12:41:00.167263 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:41:00 crc kubenswrapper[5052]: I1203 12:41:00.171074 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstackclient" Dec 03 12:41:00 crc kubenswrapper[5052]: I1203 12:41:00.172144 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_ovsdbserver-nb-0_46c590ef-2b7c-412d-8582-dfdc311e6778/ovsdbserver-nb/0.log" Dec 03 12:41:00 crc kubenswrapper[5052]: I1203 12:41:00.172218 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:41:00 crc kubenswrapper[5052]: I1203 12:41:00.214227 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/d6bedb3e-52e6-44a9-afd9-a164b0730f52-metrics-certs-tls-certs\") pod \"d6bedb3e-52e6-44a9-afd9-a164b0730f52\" (UID: \"d6bedb3e-52e6-44a9-afd9-a164b0730f52\") " Dec 03 12:41:00 crc kubenswrapper[5052]: I1203 12:41:00.214286 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/46c590ef-2b7c-412d-8582-dfdc311e6778-ovsdb-rundir\") pod \"46c590ef-2b7c-412d-8582-dfdc311e6778\" (UID: \"46c590ef-2b7c-412d-8582-dfdc311e6778\") " Dec 03 12:41:00 crc kubenswrapper[5052]: I1203 12:41:00.214307 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gllgp\" (UniqueName: \"kubernetes.io/projected/7ce91d4a-389d-4b37-b582-8c886566aa74-kube-api-access-gllgp\") pod \"7ce91d4a-389d-4b37-b582-8c886566aa74\" (UID: \"7ce91d4a-389d-4b37-b582-8c886566aa74\") " Dec 03 12:41:00 crc kubenswrapper[5052]: I1203 12:41:00.214327 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/46c590ef-2b7c-412d-8582-dfdc311e6778-scripts\") pod \"46c590ef-2b7c-412d-8582-dfdc311e6778\" (UID: \"46c590ef-2b7c-412d-8582-dfdc311e6778\") " Dec 03 12:41:00 crc kubenswrapper[5052]: I1203 12:41:00.214344 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d6bedb3e-52e6-44a9-afd9-a164b0730f52-config\") pod \"d6bedb3e-52e6-44a9-afd9-a164b0730f52\" (UID: \"d6bedb3e-52e6-44a9-afd9-a164b0730f52\") " Dec 03 12:41:00 crc kubenswrapper[5052]: I1203 12:41:00.214365 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"openstack-config-secret\" (UniqueName: \"kubernetes.io/secret/7ce91d4a-389d-4b37-b582-8c886566aa74-openstack-config-secret\") pod \"7ce91d4a-389d-4b37-b582-8c886566aa74\" (UID: \"7ce91d4a-389d-4b37-b582-8c886566aa74\") " Dec 03 12:41:00 crc kubenswrapper[5052]: I1203 12:41:00.214383 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cbd64\" (UniqueName: \"kubernetes.io/projected/d6bedb3e-52e6-44a9-afd9-a164b0730f52-kube-api-access-cbd64\") pod \"d6bedb3e-52e6-44a9-afd9-a164b0730f52\" (UID: \"d6bedb3e-52e6-44a9-afd9-a164b0730f52\") " Dec 03 12:41:00 crc kubenswrapper[5052]: I1203 12:41:00.214398 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/d6bedb3e-52e6-44a9-afd9-a164b0730f52-scripts\") pod \"d6bedb3e-52e6-44a9-afd9-a164b0730f52\" (UID: \"d6bedb3e-52e6-44a9-afd9-a164b0730f52\") " Dec 03 12:41:00 crc kubenswrapper[5052]: I1203 12:41:00.214417 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/46c590ef-2b7c-412d-8582-dfdc311e6778-config\") pod \"46c590ef-2b7c-412d-8582-dfdc311e6778\" (UID: \"46c590ef-2b7c-412d-8582-dfdc311e6778\") " Dec 03 12:41:00 crc kubenswrapper[5052]: I1203 12:41:00.214496 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovndbcluster-sb-etc-ovn\" (UniqueName: \"kubernetes.io/local-volume/local-storage09-crc\") pod \"d6bedb3e-52e6-44a9-afd9-a164b0730f52\" (UID: \"d6bedb3e-52e6-44a9-afd9-a164b0730f52\") " Dec 03 12:41:00 crc kubenswrapper[5052]: I1203 12:41:00.214533 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/d6bedb3e-52e6-44a9-afd9-a164b0730f52-ovsdb-rundir\") pod \"d6bedb3e-52e6-44a9-afd9-a164b0730f52\" (UID: \"d6bedb3e-52e6-44a9-afd9-a164b0730f52\") " Dec 03 12:41:00 crc kubenswrapper[5052]: I1203 12:41:00.214557 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/46c590ef-2b7c-412d-8582-dfdc311e6778-metrics-certs-tls-certs\") pod \"46c590ef-2b7c-412d-8582-dfdc311e6778\" (UID: \"46c590ef-2b7c-412d-8582-dfdc311e6778\") " Dec 03 12:41:00 crc kubenswrapper[5052]: I1203 12:41:00.214584 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovsdbserver-nb-tls-certs\" (UniqueName: \"kubernetes.io/secret/46c590ef-2b7c-412d-8582-dfdc311e6778-ovsdbserver-nb-tls-certs\") pod \"46c590ef-2b7c-412d-8582-dfdc311e6778\" (UID: \"46c590ef-2b7c-412d-8582-dfdc311e6778\") " Dec 03 12:41:00 crc kubenswrapper[5052]: I1203 12:41:00.214604 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"openstack-config\" (UniqueName: \"kubernetes.io/configmap/7ce91d4a-389d-4b37-b582-8c886566aa74-openstack-config\") pod \"7ce91d4a-389d-4b37-b582-8c886566aa74\" (UID: \"7ce91d4a-389d-4b37-b582-8c886566aa74\") " Dec 03 12:41:00 crc kubenswrapper[5052]: I1203 12:41:00.214623 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-npwz9\" (UniqueName: \"kubernetes.io/projected/46c590ef-2b7c-412d-8582-dfdc311e6778-kube-api-access-npwz9\") pod \"46c590ef-2b7c-412d-8582-dfdc311e6778\" (UID: \"46c590ef-2b7c-412d-8582-dfdc311e6778\") " Dec 03 12:41:00 crc kubenswrapper[5052]: I1203 12:41:00.214644 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovsdbserver-sb-tls-certs\" (UniqueName: \"kubernetes.io/secret/d6bedb3e-52e6-44a9-afd9-a164b0730f52-ovsdbserver-sb-tls-certs\") pod \"d6bedb3e-52e6-44a9-afd9-a164b0730f52\" (UID: \"d6bedb3e-52e6-44a9-afd9-a164b0730f52\") " Dec 03 12:41:00 crc kubenswrapper[5052]: I1203 12:41:00.215561 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d6bedb3e-52e6-44a9-afd9-a164b0730f52-combined-ca-bundle\") pod \"d6bedb3e-52e6-44a9-afd9-a164b0730f52\" (UID: \"d6bedb3e-52e6-44a9-afd9-a164b0730f52\") " Dec 03 12:41:00 crc kubenswrapper[5052]: I1203 12:41:00.215656 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/46c590ef-2b7c-412d-8582-dfdc311e6778-combined-ca-bundle\") pod \"46c590ef-2b7c-412d-8582-dfdc311e6778\" (UID: \"46c590ef-2b7c-412d-8582-dfdc311e6778\") " Dec 03 12:41:00 crc kubenswrapper[5052]: I1203 12:41:00.215701 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovndbcluster-nb-etc-ovn\" (UniqueName: \"kubernetes.io/local-volume/local-storage01-crc\") pod \"46c590ef-2b7c-412d-8582-dfdc311e6778\" (UID: \"46c590ef-2b7c-412d-8582-dfdc311e6778\") " Dec 03 12:41:00 crc kubenswrapper[5052]: I1203 12:41:00.215728 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/7ce91d4a-389d-4b37-b582-8c886566aa74-combined-ca-bundle\") pod \"7ce91d4a-389d-4b37-b582-8c886566aa74\" (UID: \"7ce91d4a-389d-4b37-b582-8c886566aa74\") " Dec 03 12:41:00 crc kubenswrapper[5052]: I1203 12:41:00.218044 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:41:00 crc kubenswrapper[5052]: I1203 12:41:00.219635 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/46c590ef-2b7c-412d-8582-dfdc311e6778-scripts" (OuterVolumeSpecName: "scripts") pod "46c590ef-2b7c-412d-8582-dfdc311e6778" (UID: "46c590ef-2b7c-412d-8582-dfdc311e6778"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:41:00 crc kubenswrapper[5052]: I1203 12:41:00.228214 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7ce91d4a-389d-4b37-b582-8c886566aa74-kube-api-access-gllgp" (OuterVolumeSpecName: "kube-api-access-gllgp") pod "7ce91d4a-389d-4b37-b582-8c886566aa74" (UID: "7ce91d4a-389d-4b37-b582-8c886566aa74"). InnerVolumeSpecName "kube-api-access-gllgp". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:41:00 crc kubenswrapper[5052]: I1203 12:41:00.228692 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/46c590ef-2b7c-412d-8582-dfdc311e6778-ovsdb-rundir" (OuterVolumeSpecName: "ovsdb-rundir") pod "46c590ef-2b7c-412d-8582-dfdc311e6778" (UID: "46c590ef-2b7c-412d-8582-dfdc311e6778"). InnerVolumeSpecName "ovsdb-rundir". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:41:00 crc kubenswrapper[5052]: I1203 12:41:00.231631 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/46c590ef-2b7c-412d-8582-dfdc311e6778-kube-api-access-npwz9" (OuterVolumeSpecName: "kube-api-access-npwz9") pod "46c590ef-2b7c-412d-8582-dfdc311e6778" (UID: "46c590ef-2b7c-412d-8582-dfdc311e6778"). InnerVolumeSpecName "kube-api-access-npwz9". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:41:00 crc kubenswrapper[5052]: I1203 12:41:00.235134 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d6bedb3e-52e6-44a9-afd9-a164b0730f52-ovsdb-rundir" (OuterVolumeSpecName: "ovsdb-rundir") pod "d6bedb3e-52e6-44a9-afd9-a164b0730f52" (UID: "d6bedb3e-52e6-44a9-afd9-a164b0730f52"). InnerVolumeSpecName "ovsdb-rundir". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:41:00 crc kubenswrapper[5052]: I1203 12:41:00.235245 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d6bedb3e-52e6-44a9-afd9-a164b0730f52-config" (OuterVolumeSpecName: "config") pod "d6bedb3e-52e6-44a9-afd9-a164b0730f52" (UID: "d6bedb3e-52e6-44a9-afd9-a164b0730f52"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:41:00 crc kubenswrapper[5052]: I1203 12:41:00.235605 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/46c590ef-2b7c-412d-8582-dfdc311e6778-config" (OuterVolumeSpecName: "config") pod "46c590ef-2b7c-412d-8582-dfdc311e6778" (UID: "46c590ef-2b7c-412d-8582-dfdc311e6778"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:41:00 crc kubenswrapper[5052]: I1203 12:41:00.236442 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d6bedb3e-52e6-44a9-afd9-a164b0730f52-kube-api-access-cbd64" (OuterVolumeSpecName: "kube-api-access-cbd64") pod "d6bedb3e-52e6-44a9-afd9-a164b0730f52" (UID: "d6bedb3e-52e6-44a9-afd9-a164b0730f52"). InnerVolumeSpecName "kube-api-access-cbd64". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:41:00 crc kubenswrapper[5052]: I1203 12:41:00.241281 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d6bedb3e-52e6-44a9-afd9-a164b0730f52-scripts" (OuterVolumeSpecName: "scripts") pod "d6bedb3e-52e6-44a9-afd9-a164b0730f52" (UID: "d6bedb3e-52e6-44a9-afd9-a164b0730f52"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:41:00 crc kubenswrapper[5052]: I1203 12:41:00.250023 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:41:00 crc kubenswrapper[5052]: I1203 12:41:00.255155 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage09-crc" (OuterVolumeSpecName: "ovndbcluster-sb-etc-ovn") pod "d6bedb3e-52e6-44a9-afd9-a164b0730f52" (UID: "d6bedb3e-52e6-44a9-afd9-a164b0730f52"). InnerVolumeSpecName "local-storage09-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 12:41:00 crc kubenswrapper[5052]: I1203 12:41:00.300273 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage01-crc" (OuterVolumeSpecName: "ovndbcluster-nb-etc-ovn") pod "46c590ef-2b7c-412d-8582-dfdc311e6778" (UID: "46c590ef-2b7c-412d-8582-dfdc311e6778"). InnerVolumeSpecName "local-storage01-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 12:41:00 crc kubenswrapper[5052]: I1203 12:41:00.318214 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gvsj7\" (UniqueName: \"kubernetes.io/projected/b7defe22-28b4-4fec-8fe5-ebde8e658c6b-kube-api-access-gvsj7\") pod \"b7defe22-28b4-4fec-8fe5-ebde8e658c6b\" (UID: \"b7defe22-28b4-4fec-8fe5-ebde8e658c6b\") " Dec 03 12:41:00 crc kubenswrapper[5052]: I1203 12:41:00.318272 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/d855c655-3b2f-4e4f-9b27-cf4e43a92045-etc-machine-id\") pod \"d855c655-3b2f-4e4f-9b27-cf4e43a92045\" (UID: \"d855c655-3b2f-4e4f-9b27-cf4e43a92045\") " Dec 03 12:41:00 crc kubenswrapper[5052]: I1203 12:41:00.318311 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/d855c655-3b2f-4e4f-9b27-cf4e43a92045-config-data-custom\") pod \"d855c655-3b2f-4e4f-9b27-cf4e43a92045\" (UID: \"d855c655-3b2f-4e4f-9b27-cf4e43a92045\") " Dec 03 12:41:00 crc kubenswrapper[5052]: I1203 12:41:00.318342 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d855c655-3b2f-4e4f-9b27-cf4e43a92045-combined-ca-bundle\") pod \"d855c655-3b2f-4e4f-9b27-cf4e43a92045\" (UID: \"d855c655-3b2f-4e4f-9b27-cf4e43a92045\") " Dec 03 12:41:00 crc kubenswrapper[5052]: I1203 12:41:00.318425 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/d855c655-3b2f-4e4f-9b27-cf4e43a92045-config-data\") pod \"d855c655-3b2f-4e4f-9b27-cf4e43a92045\" (UID: \"d855c655-3b2f-4e4f-9b27-cf4e43a92045\") " Dec 03 12:41:00 crc kubenswrapper[5052]: I1203 12:41:00.318496 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/d855c655-3b2f-4e4f-9b27-cf4e43a92045-scripts\") pod \"d855c655-3b2f-4e4f-9b27-cf4e43a92045\" (UID: \"d855c655-3b2f-4e4f-9b27-cf4e43a92045\") " Dec 03 12:41:00 crc kubenswrapper[5052]: I1203 12:41:00.319741 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/d855c655-3b2f-4e4f-9b27-cf4e43a92045-etc-machine-id" (OuterVolumeSpecName: "etc-machine-id") pod "d855c655-3b2f-4e4f-9b27-cf4e43a92045" (UID: "d855c655-3b2f-4e4f-9b27-cf4e43a92045"). InnerVolumeSpecName "etc-machine-id". PluginName "kubernetes.io/host-path", VolumeGidValue "" Dec 03 12:41:00 crc kubenswrapper[5052]: I1203 12:41:00.328271 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b7defe22-28b4-4fec-8fe5-ebde8e658c6b-combined-ca-bundle\") pod \"b7defe22-28b4-4fec-8fe5-ebde8e658c6b\" (UID: \"b7defe22-28b4-4fec-8fe5-ebde8e658c6b\") " Dec 03 12:41:00 crc kubenswrapper[5052]: I1203 12:41:00.328404 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-state-metrics-tls-config\" (UniqueName: \"kubernetes.io/secret/b7defe22-28b4-4fec-8fe5-ebde8e658c6b-kube-state-metrics-tls-config\") pod \"b7defe22-28b4-4fec-8fe5-ebde8e658c6b\" (UID: \"b7defe22-28b4-4fec-8fe5-ebde8e658c6b\") " Dec 03 12:41:00 crc kubenswrapper[5052]: I1203 12:41:00.328444 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-state-metrics-tls-certs\" (UniqueName: \"kubernetes.io/secret/b7defe22-28b4-4fec-8fe5-ebde8e658c6b-kube-state-metrics-tls-certs\") pod \"b7defe22-28b4-4fec-8fe5-ebde8e658c6b\" (UID: \"b7defe22-28b4-4fec-8fe5-ebde8e658c6b\") " Dec 03 12:41:00 crc kubenswrapper[5052]: I1203 12:41:00.328475 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9qtk7\" (UniqueName: \"kubernetes.io/projected/d855c655-3b2f-4e4f-9b27-cf4e43a92045-kube-api-access-9qtk7\") pod \"d855c655-3b2f-4e4f-9b27-cf4e43a92045\" (UID: \"d855c655-3b2f-4e4f-9b27-cf4e43a92045\") " Dec 03 12:41:00 crc kubenswrapper[5052]: I1203 12:41:00.353738 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage09-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage09-crc\") on node \"crc\" " Dec 03 12:41:00 crc kubenswrapper[5052]: I1203 12:41:00.354072 5052 reconciler_common.go:293] "Volume detached for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/d6bedb3e-52e6-44a9-afd9-a164b0730f52-ovsdb-rundir\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:00 crc kubenswrapper[5052]: I1203 12:41:00.354087 5052 reconciler_common.go:293] "Volume detached for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/d855c655-3b2f-4e4f-9b27-cf4e43a92045-etc-machine-id\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:00 crc kubenswrapper[5052]: I1203 12:41:00.354104 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-npwz9\" (UniqueName: \"kubernetes.io/projected/46c590ef-2b7c-412d-8582-dfdc311e6778-kube-api-access-npwz9\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:00 crc kubenswrapper[5052]: I1203 12:41:00.354125 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage01-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage01-crc\") on node \"crc\" " Dec 03 12:41:00 crc kubenswrapper[5052]: I1203 12:41:00.354137 5052 reconciler_common.go:293] "Volume detached for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/46c590ef-2b7c-412d-8582-dfdc311e6778-ovsdb-rundir\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:00 crc kubenswrapper[5052]: I1203 12:41:00.354152 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-gllgp\" (UniqueName: \"kubernetes.io/projected/7ce91d4a-389d-4b37-b582-8c886566aa74-kube-api-access-gllgp\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:00 crc kubenswrapper[5052]: I1203 12:41:00.354165 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/46c590ef-2b7c-412d-8582-dfdc311e6778-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:00 crc kubenswrapper[5052]: I1203 12:41:00.354176 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d6bedb3e-52e6-44a9-afd9-a164b0730f52-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:00 crc kubenswrapper[5052]: I1203 12:41:00.354188 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-cbd64\" (UniqueName: \"kubernetes.io/projected/d6bedb3e-52e6-44a9-afd9-a164b0730f52-kube-api-access-cbd64\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:00 crc kubenswrapper[5052]: I1203 12:41:00.354200 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/d6bedb3e-52e6-44a9-afd9-a164b0730f52-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:00 crc kubenswrapper[5052]: I1203 12:41:00.354211 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/46c590ef-2b7c-412d-8582-dfdc311e6778-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:00 crc kubenswrapper[5052]: I1203 12:41:00.370094 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d855c655-3b2f-4e4f-9b27-cf4e43a92045-config-data-custom" (OuterVolumeSpecName: "config-data-custom") pod "d855c655-3b2f-4e4f-9b27-cf4e43a92045" (UID: "d855c655-3b2f-4e4f-9b27-cf4e43a92045"). InnerVolumeSpecName "config-data-custom". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:41:00 crc kubenswrapper[5052]: I1203 12:41:00.384063 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b7defe22-28b4-4fec-8fe5-ebde8e658c6b-kube-api-access-gvsj7" (OuterVolumeSpecName: "kube-api-access-gvsj7") pod "b7defe22-28b4-4fec-8fe5-ebde8e658c6b" (UID: "b7defe22-28b4-4fec-8fe5-ebde8e658c6b"). InnerVolumeSpecName "kube-api-access-gvsj7". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:41:00 crc kubenswrapper[5052]: I1203 12:41:00.384029 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7ce91d4a-389d-4b37-b582-8c886566aa74-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "7ce91d4a-389d-4b37-b582-8c886566aa74" (UID: "7ce91d4a-389d-4b37-b582-8c886566aa74"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:41:00 crc kubenswrapper[5052]: I1203 12:41:00.386143 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d855c655-3b2f-4e4f-9b27-cf4e43a92045-scripts" (OuterVolumeSpecName: "scripts") pod "d855c655-3b2f-4e4f-9b27-cf4e43a92045" (UID: "d855c655-3b2f-4e4f-9b27-cf4e43a92045"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:41:00 crc kubenswrapper[5052]: I1203 12:41:00.391082 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d855c655-3b2f-4e4f-9b27-cf4e43a92045-kube-api-access-9qtk7" (OuterVolumeSpecName: "kube-api-access-9qtk7") pod "d855c655-3b2f-4e4f-9b27-cf4e43a92045" (UID: "d855c655-3b2f-4e4f-9b27-cf4e43a92045"). InnerVolumeSpecName "kube-api-access-9qtk7". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:41:00 crc kubenswrapper[5052]: E1203 12:41:00.461141 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/combined-ca-bundle: secret "combined-ca-bundle" not found Dec 03 12:41:00 crc kubenswrapper[5052]: I1203 12:41:00.461170 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/d855c655-3b2f-4e4f-9b27-cf4e43a92045-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:00 crc kubenswrapper[5052]: E1203 12:41:00.461218 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0556e9f0-e431-4193-9822-6079cb8a1e44-combined-ca-bundle podName:0556e9f0-e431-4193-9822-6079cb8a1e44 nodeName:}" failed. No retries permitted until 2025-12-03 12:41:04.461196181 +0000 UTC m=+4855.049438035 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "combined-ca-bundle" (UniqueName: "kubernetes.io/secret/0556e9f0-e431-4193-9822-6079cb8a1e44-combined-ca-bundle") pod "nova-cell1-conductor-0" (UID: "0556e9f0-e431-4193-9822-6079cb8a1e44") : secret "combined-ca-bundle" not found Dec 03 12:41:00 crc kubenswrapper[5052]: I1203 12:41:00.461256 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/7ce91d4a-389d-4b37-b582-8c886566aa74-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:00 crc kubenswrapper[5052]: I1203 12:41:00.461277 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-9qtk7\" (UniqueName: \"kubernetes.io/projected/d855c655-3b2f-4e4f-9b27-cf4e43a92045-kube-api-access-9qtk7\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:00 crc kubenswrapper[5052]: I1203 12:41:00.461295 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-gvsj7\" (UniqueName: \"kubernetes.io/projected/b7defe22-28b4-4fec-8fe5-ebde8e658c6b-kube-api-access-gvsj7\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:00 crc kubenswrapper[5052]: I1203 12:41:00.461307 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/d855c655-3b2f-4e4f-9b27-cf4e43a92045-config-data-custom\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:00 crc kubenswrapper[5052]: I1203 12:41:00.489948 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7ce91d4a-389d-4b37-b582-8c886566aa74-openstack-config" (OuterVolumeSpecName: "openstack-config") pod "7ce91d4a-389d-4b37-b582-8c886566aa74" (UID: "7ce91d4a-389d-4b37-b582-8c886566aa74"). InnerVolumeSpecName "openstack-config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:41:00 crc kubenswrapper[5052]: I1203 12:41:00.520459 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage01-crc" (UniqueName: "kubernetes.io/local-volume/local-storage01-crc") on node "crc" Dec 03 12:41:00 crc kubenswrapper[5052]: I1203 12:41:00.524050 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d6bedb3e-52e6-44a9-afd9-a164b0730f52-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "d6bedb3e-52e6-44a9-afd9-a164b0730f52" (UID: "d6bedb3e-52e6-44a9-afd9-a164b0730f52"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:41:00 crc kubenswrapper[5052]: I1203 12:41:00.551735 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7ce91d4a-389d-4b37-b582-8c886566aa74-openstack-config-secret" (OuterVolumeSpecName: "openstack-config-secret") pod "7ce91d4a-389d-4b37-b582-8c886566aa74" (UID: "7ce91d4a-389d-4b37-b582-8c886566aa74"). InnerVolumeSpecName "openstack-config-secret". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:41:00 crc kubenswrapper[5052]: I1203 12:41:00.562072 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage09-crc" (UniqueName: "kubernetes.io/local-volume/local-storage09-crc") on node "crc" Dec 03 12:41:00 crc kubenswrapper[5052]: I1203 12:41:00.563108 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage09-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage09-crc\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:00 crc kubenswrapper[5052]: I1203 12:41:00.563149 5052 reconciler_common.go:293] "Volume detached for volume \"openstack-config\" (UniqueName: \"kubernetes.io/configmap/7ce91d4a-389d-4b37-b582-8c886566aa74-openstack-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:00 crc kubenswrapper[5052]: I1203 12:41:00.563166 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d6bedb3e-52e6-44a9-afd9-a164b0730f52-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:00 crc kubenswrapper[5052]: I1203 12:41:00.563177 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage01-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage01-crc\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:00 crc kubenswrapper[5052]: I1203 12:41:00.563189 5052 reconciler_common.go:293] "Volume detached for volume \"openstack-config-secret\" (UniqueName: \"kubernetes.io/secret/7ce91d4a-389d-4b37-b582-8c886566aa74-openstack-config-secret\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:00 crc kubenswrapper[5052]: E1203 12:41:00.563260 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/rabbitmq-cell1-config-data: configmap "rabbitmq-cell1-config-data" not found Dec 03 12:41:00 crc kubenswrapper[5052]: E1203 12:41:00.563321 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f9bac64c-518e-4630-8889-20e09d5b7a39-config-data podName:f9bac64c-518e-4630-8889-20e09d5b7a39 nodeName:}" failed. No retries permitted until 2025-12-03 12:41:04.563303541 +0000 UTC m=+4855.151545375 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "config-data" (UniqueName: "kubernetes.io/configmap/f9bac64c-518e-4630-8889-20e09d5b7a39-config-data") pod "rabbitmq-cell1-server-0" (UID: "f9bac64c-518e-4630-8889-20e09d5b7a39") : configmap "rabbitmq-cell1-config-data" not found Dec 03 12:41:00 crc kubenswrapper[5052]: I1203 12:41:00.577181 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/46c590ef-2b7c-412d-8582-dfdc311e6778-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "46c590ef-2b7c-412d-8582-dfdc311e6778" (UID: "46c590ef-2b7c-412d-8582-dfdc311e6778"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:41:00 crc kubenswrapper[5052]: I1203 12:41:00.581950 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b7defe22-28b4-4fec-8fe5-ebde8e658c6b-kube-state-metrics-tls-config" (OuterVolumeSpecName: "kube-state-metrics-tls-config") pod "b7defe22-28b4-4fec-8fe5-ebde8e658c6b" (UID: "b7defe22-28b4-4fec-8fe5-ebde8e658c6b"). InnerVolumeSpecName "kube-state-metrics-tls-config". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:41:00 crc kubenswrapper[5052]: I1203 12:41:00.593897 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b7defe22-28b4-4fec-8fe5-ebde8e658c6b-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "b7defe22-28b4-4fec-8fe5-ebde8e658c6b" (UID: "b7defe22-28b4-4fec-8fe5-ebde8e658c6b"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:41:00 crc kubenswrapper[5052]: I1203 12:41:00.615942 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b7defe22-28b4-4fec-8fe5-ebde8e658c6b-kube-state-metrics-tls-certs" (OuterVolumeSpecName: "kube-state-metrics-tls-certs") pod "b7defe22-28b4-4fec-8fe5-ebde8e658c6b" (UID: "b7defe22-28b4-4fec-8fe5-ebde8e658c6b"). InnerVolumeSpecName "kube-state-metrics-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:41:00 crc kubenswrapper[5052]: I1203 12:41:00.627278 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d6bedb3e-52e6-44a9-afd9-a164b0730f52-metrics-certs-tls-certs" (OuterVolumeSpecName: "metrics-certs-tls-certs") pod "d6bedb3e-52e6-44a9-afd9-a164b0730f52" (UID: "d6bedb3e-52e6-44a9-afd9-a164b0730f52"). InnerVolumeSpecName "metrics-certs-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:41:00 crc kubenswrapper[5052]: I1203 12:41:00.631148 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d855c655-3b2f-4e4f-9b27-cf4e43a92045-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "d855c655-3b2f-4e4f-9b27-cf4e43a92045" (UID: "d855c655-3b2f-4e4f-9b27-cf4e43a92045"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:41:00 crc kubenswrapper[5052]: I1203 12:41:00.632078 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/46c590ef-2b7c-412d-8582-dfdc311e6778-ovsdbserver-nb-tls-certs" (OuterVolumeSpecName: "ovsdbserver-nb-tls-certs") pod "46c590ef-2b7c-412d-8582-dfdc311e6778" (UID: "46c590ef-2b7c-412d-8582-dfdc311e6778"). InnerVolumeSpecName "ovsdbserver-nb-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:41:00 crc kubenswrapper[5052]: I1203 12:41:00.668127 5052 reconciler_common.go:293] "Volume detached for volume \"ovsdbserver-nb-tls-certs\" (UniqueName: \"kubernetes.io/secret/46c590ef-2b7c-412d-8582-dfdc311e6778-ovsdbserver-nb-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:00 crc kubenswrapper[5052]: I1203 12:41:00.668418 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d855c655-3b2f-4e4f-9b27-cf4e43a92045-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:00 crc kubenswrapper[5052]: I1203 12:41:00.668431 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/46c590ef-2b7c-412d-8582-dfdc311e6778-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:00 crc kubenswrapper[5052]: I1203 12:41:00.668452 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b7defe22-28b4-4fec-8fe5-ebde8e658c6b-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:00 crc kubenswrapper[5052]: I1203 12:41:00.668464 5052 reconciler_common.go:293] "Volume detached for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/d6bedb3e-52e6-44a9-afd9-a164b0730f52-metrics-certs-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:00 crc kubenswrapper[5052]: I1203 12:41:00.668476 5052 reconciler_common.go:293] "Volume detached for volume \"kube-state-metrics-tls-config\" (UniqueName: \"kubernetes.io/secret/b7defe22-28b4-4fec-8fe5-ebde8e658c6b-kube-state-metrics-tls-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:00 crc kubenswrapper[5052]: I1203 12:41:00.668490 5052 reconciler_common.go:293] "Volume detached for volume \"kube-state-metrics-tls-certs\" (UniqueName: \"kubernetes.io/secret/b7defe22-28b4-4fec-8fe5-ebde8e658c6b-kube-state-metrics-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:00 crc kubenswrapper[5052]: I1203 12:41:00.680363 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d6bedb3e-52e6-44a9-afd9-a164b0730f52-ovsdbserver-sb-tls-certs" (OuterVolumeSpecName: "ovsdbserver-sb-tls-certs") pod "d6bedb3e-52e6-44a9-afd9-a164b0730f52" (UID: "d6bedb3e-52e6-44a9-afd9-a164b0730f52"). InnerVolumeSpecName "ovsdbserver-sb-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:41:00 crc kubenswrapper[5052]: I1203 12:41:00.694704 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder3a0d-account-delete-lxd6l"] Dec 03 12:41:00 crc kubenswrapper[5052]: I1203 12:41:00.748532 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/46c590ef-2b7c-412d-8582-dfdc311e6778-metrics-certs-tls-certs" (OuterVolumeSpecName: "metrics-certs-tls-certs") pod "46c590ef-2b7c-412d-8582-dfdc311e6778" (UID: "46c590ef-2b7c-412d-8582-dfdc311e6778"). InnerVolumeSpecName "metrics-certs-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:41:00 crc kubenswrapper[5052]: I1203 12:41:00.770729 5052 reconciler_common.go:293] "Volume detached for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/46c590ef-2b7c-412d-8582-dfdc311e6778-metrics-certs-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:00 crc kubenswrapper[5052]: I1203 12:41:00.770769 5052 reconciler_common.go:293] "Volume detached for volume \"ovsdbserver-sb-tls-certs\" (UniqueName: \"kubernetes.io/secret/d6bedb3e-52e6-44a9-afd9-a164b0730f52-ovsdbserver-sb-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:00 crc kubenswrapper[5052]: I1203 12:41:00.770785 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d855c655-3b2f-4e4f-9b27-cf4e43a92045-config-data" (OuterVolumeSpecName: "config-data") pod "d855c655-3b2f-4e4f-9b27-cf4e43a92045" (UID: "d855c655-3b2f-4e4f-9b27-cf4e43a92045"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:41:00 crc kubenswrapper[5052]: I1203 12:41:00.786748 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/placement7f30-account-delete-vld4v"] Dec 03 12:41:00 crc kubenswrapper[5052]: I1203 12:41:00.813101 5052 generic.go:334] "Generic (PLEG): container finished" podID="b7c49d90-c0af-4d4b-a080-8c7dce3ee66a" containerID="919c4b64371a0cbf9bb20effc9d35fba8bb6ee8a56b9ff66f203f2d32d0ee97b" exitCode=0 Dec 03 12:41:00 crc kubenswrapper[5052]: I1203 12:41:00.813189 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glancec94f-account-delete-8t6wx" event={"ID":"b7c49d90-c0af-4d4b-a080-8c7dce3ee66a","Type":"ContainerDied","Data":"919c4b64371a0cbf9bb20effc9d35fba8bb6ee8a56b9ff66f203f2d32d0ee97b"} Dec 03 12:41:00 crc kubenswrapper[5052]: I1203 12:41:00.816083 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder3a0d-account-delete-lxd6l" event={"ID":"5fdc7b7d-f9ed-4d5c-a77b-6a6b0fd186f9","Type":"ContainerStarted","Data":"93ea83bcac826673f5de5369123cff8d55921f5215fcc30dac291272c62deb3d"} Dec 03 12:41:00 crc kubenswrapper[5052]: I1203 12:41:00.821276 5052 generic.go:334] "Generic (PLEG): container finished" podID="621935c3-f8f9-43a1-9bc3-6d6ad9caf0ef" containerID="53722c8da45292bfe9fdbd4b2d2a18b053dc900a020f10bc15c4643959d2c7c1" exitCode=0 Dec 03 12:41:00 crc kubenswrapper[5052]: I1203 12:41:00.821308 5052 generic.go:334] "Generic (PLEG): container finished" podID="621935c3-f8f9-43a1-9bc3-6d6ad9caf0ef" containerID="30dfc639d124cfef009d7624b44b757995f402edb14bbdfeb69e215f2a21b847" exitCode=0 Dec 03 12:41:00 crc kubenswrapper[5052]: I1203 12:41:00.821408 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-proxy-bf56d5474-cq7b2" event={"ID":"621935c3-f8f9-43a1-9bc3-6d6ad9caf0ef","Type":"ContainerDied","Data":"53722c8da45292bfe9fdbd4b2d2a18b053dc900a020f10bc15c4643959d2c7c1"} Dec 03 12:41:00 crc kubenswrapper[5052]: I1203 12:41:00.821456 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-proxy-bf56d5474-cq7b2" event={"ID":"621935c3-f8f9-43a1-9bc3-6d6ad9caf0ef","Type":"ContainerDied","Data":"30dfc639d124cfef009d7624b44b757995f402edb14bbdfeb69e215f2a21b847"} Dec 03 12:41:00 crc kubenswrapper[5052]: I1203 12:41:00.833979 5052 generic.go:334] "Generic (PLEG): container finished" podID="12f52069-8b08-4e4f-a3c1-a5421efb5844" containerID="c50569849ae16f9aa7c97dde8235fdd02d7e4703bdc46f99d859da6cf6d5b3c2" exitCode=0 Dec 03 12:41:00 crc kubenswrapper[5052]: I1203 12:41:00.834063 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutrond24e-account-delete-p2mpp" event={"ID":"12f52069-8b08-4e4f-a3c1-a5421efb5844","Type":"ContainerDied","Data":"c50569849ae16f9aa7c97dde8235fdd02d7e4703bdc46f99d859da6cf6d5b3c2"} Dec 03 12:41:00 crc kubenswrapper[5052]: I1203 12:41:00.838687 5052 generic.go:334] "Generic (PLEG): container finished" podID="d855c655-3b2f-4e4f-9b27-cf4e43a92045" containerID="84ca5b96b0f3458adcf7e677440d6f30818b02e536633792be662d5c24cf8b44" exitCode=0 Dec 03 12:41:00 crc kubenswrapper[5052]: I1203 12:41:00.838771 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:41:00 crc kubenswrapper[5052]: I1203 12:41:00.838828 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"d855c655-3b2f-4e4f-9b27-cf4e43a92045","Type":"ContainerDied","Data":"84ca5b96b0f3458adcf7e677440d6f30818b02e536633792be662d5c24cf8b44"} Dec 03 12:41:00 crc kubenswrapper[5052]: I1203 12:41:00.838872 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"d855c655-3b2f-4e4f-9b27-cf4e43a92045","Type":"ContainerDied","Data":"cbf4a90f447a46d3524bfa43c1ebb6ebb7b1a7f4d967436abf0fd6d4b0a6d875"} Dec 03 12:41:00 crc kubenswrapper[5052]: I1203 12:41:00.838894 5052 scope.go:117] "RemoveContainer" containerID="84ca5b96b0f3458adcf7e677440d6f30818b02e536633792be662d5c24cf8b44" Dec 03 12:41:00 crc kubenswrapper[5052]: I1203 12:41:00.841802 5052 generic.go:334] "Generic (PLEG): container finished" podID="f89ce0f5-5a92-49eb-9951-f2b491ac2139" containerID="0c3d8b3ca8a9d7d4611f59d9c0bcfa83d8ea56d3df8caa14069f56e2580808ef" exitCode=0 Dec 03 12:41:00 crc kubenswrapper[5052]: I1203 12:41:00.841869 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-cell1-galera-0" event={"ID":"f89ce0f5-5a92-49eb-9951-f2b491ac2139","Type":"ContainerDied","Data":"0c3d8b3ca8a9d7d4611f59d9c0bcfa83d8ea56d3df8caa14069f56e2580808ef"} Dec 03 12:41:00 crc kubenswrapper[5052]: I1203 12:41:00.841900 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-cell1-galera-0" event={"ID":"f89ce0f5-5a92-49eb-9951-f2b491ac2139","Type":"ContainerDied","Data":"2d5424cae249b796c399d3d475049ca1e43197f2411c87542635bf83f57ad715"} Dec 03 12:41:00 crc kubenswrapper[5052]: I1203 12:41:00.841915 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2d5424cae249b796c399d3d475049ca1e43197f2411c87542635bf83f57ad715" Dec 03 12:41:00 crc kubenswrapper[5052]: I1203 12:41:00.844138 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/kube-state-metrics-0" event={"ID":"b7defe22-28b4-4fec-8fe5-ebde8e658c6b","Type":"ContainerDied","Data":"d64051f0917a85a737352a0ca1c2523190331331b68cf6f466ba17250c4ba9ff"} Dec 03 12:41:00 crc kubenswrapper[5052]: I1203 12:41:00.844243 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:41:00 crc kubenswrapper[5052]: I1203 12:41:00.859343 5052 generic.go:334] "Generic (PLEG): container finished" podID="65cb22ea-3a9c-4f99-9128-d3c7681c3173" containerID="d970c5d7874c5afe76962513bf4d336686b29e37d400962b8518fcae8ce933fc" exitCode=0 Dec 03 12:41:00 crc kubenswrapper[5052]: I1203 12:41:00.859411 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"65cb22ea-3a9c-4f99-9128-d3c7681c3173","Type":"ContainerDied","Data":"d970c5d7874c5afe76962513bf4d336686b29e37d400962b8518fcae8ce933fc"} Dec 03 12:41:00 crc kubenswrapper[5052]: I1203 12:41:00.862425 5052 generic.go:334] "Generic (PLEG): container finished" podID="0f16f521-a181-40e4-a1ed-38f4c8862ffb" containerID="49e65401462a62ad6b62529bdc792c44fb0d7a07381a58a73215f64add3d28c5" exitCode=0 Dec 03 12:41:00 crc kubenswrapper[5052]: I1203 12:41:00.862491 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:41:00 crc kubenswrapper[5052]: I1203 12:41:00.863708 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" event={"ID":"0f16f521-a181-40e4-a1ed-38f4c8862ffb","Type":"ContainerDied","Data":"49e65401462a62ad6b62529bdc792c44fb0d7a07381a58a73215f64add3d28c5"} Dec 03 12:41:00 crc kubenswrapper[5052]: I1203 12:41:00.863768 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" event={"ID":"0f16f521-a181-40e4-a1ed-38f4c8862ffb","Type":"ContainerDied","Data":"10026819c94b63b87ca2799cb11896626abae2617817d72431026729a7b3c240"} Dec 03 12:41:00 crc kubenswrapper[5052]: I1203 12:41:00.863783 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="10026819c94b63b87ca2799cb11896626abae2617817d72431026729a7b3c240" Dec 03 12:41:00 crc kubenswrapper[5052]: I1203 12:41:00.863838 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:41:00 crc kubenswrapper[5052]: I1203 12:41:00.864559 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstackclient" Dec 03 12:41:00 crc kubenswrapper[5052]: I1203 12:41:00.872187 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/d855c655-3b2f-4e4f-9b27-cf4e43a92045-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:00 crc kubenswrapper[5052]: E1203 12:41:00.973854 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/rabbitmq-config-data: configmap "rabbitmq-config-data" not found Dec 03 12:41:00 crc kubenswrapper[5052]: E1203 12:41:00.974206 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d82ef8ea-0c3b-4d89-9c78-22095940f4da-config-data podName:d82ef8ea-0c3b-4d89-9c78-22095940f4da nodeName:}" failed. No retries permitted until 2025-12-03 12:41:04.974188112 +0000 UTC m=+4855.562429876 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "config-data" (UniqueName: "kubernetes.io/configmap/d82ef8ea-0c3b-4d89-9c78-22095940f4da-config-data") pod "rabbitmq-server-0" (UID: "d82ef8ea-0c3b-4d89-9c78-22095940f4da") : configmap "rabbitmq-config-data" not found Dec 03 12:41:01 crc kubenswrapper[5052]: I1203 12:41:01.010023 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/novacell07298-account-delete-726jv"] Dec 03 12:41:01 crc kubenswrapper[5052]: I1203 12:41:01.037039 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-8h8qz"] Dec 03 12:41:01 crc kubenswrapper[5052]: W1203 12:41:01.096764 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6dbc7d8f_34d4_4927_82fe_d049727808fd.slice/crio-89e35dabb00061c9ad442c01b1fc1047b590d4300a9f6a3f62d174b78e9185b4 WatchSource:0}: Error finding container 89e35dabb00061c9ad442c01b1fc1047b590d4300a9f6a3f62d174b78e9185b4: Status 404 returned error can't find the container with id 89e35dabb00061c9ad442c01b1fc1047b590d4300a9f6a3f62d174b78e9185b4 Dec 03 12:41:01 crc kubenswrapper[5052]: I1203 12:41:01.108855 5052 scope.go:117] "RemoveContainer" containerID="323efac3746af7ff7702e46c55dedadf7e2d8d45f1a20d6b7d1fa2753913a05d" Dec 03 12:41:01 crc kubenswrapper[5052]: I1203 12:41:01.223584 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:41:01 crc kubenswrapper[5052]: I1203 12:41:01.259229 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/rabbitmq-server-0" podUID="d82ef8ea-0c3b-4d89-9c78-22095940f4da" containerName="rabbitmq" probeResult="failure" output="dial tcp 10.217.1.4:5671: connect: connection refused" Dec 03 12:41:01 crc kubenswrapper[5052]: I1203 12:41:01.283993 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:41:01 crc kubenswrapper[5052]: I1203 12:41:01.284122 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/memcached-0"] Dec 03 12:41:01 crc kubenswrapper[5052]: I1203 12:41:01.284263 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/memcached-0" podUID="5d19811b-4e82-42b2-b6ec-f16e707484f2" containerName="memcached" containerID="cri-o://78246e0cecbf2891b1eda71870c4a06e4588bbbe3e1fc591176960ad3a92ef57" gracePeriod=30 Dec 03 12:41:01 crc kubenswrapper[5052]: I1203 12:41:01.285764 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/f89ce0f5-5a92-49eb-9951-f2b491ac2139-galera-tls-certs\") pod \"f89ce0f5-5a92-49eb-9951-f2b491ac2139\" (UID: \"f89ce0f5-5a92-49eb-9951-f2b491ac2139\") " Dec 03 12:41:01 crc kubenswrapper[5052]: I1203 12:41:01.285836 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/f89ce0f5-5a92-49eb-9951-f2b491ac2139-operator-scripts\") pod \"f89ce0f5-5a92-49eb-9951-f2b491ac2139\" (UID: \"f89ce0f5-5a92-49eb-9951-f2b491ac2139\") " Dec 03 12:41:01 crc kubenswrapper[5052]: I1203 12:41:01.285872 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/f89ce0f5-5a92-49eb-9951-f2b491ac2139-config-data-default\") pod \"f89ce0f5-5a92-49eb-9951-f2b491ac2139\" (UID: \"f89ce0f5-5a92-49eb-9951-f2b491ac2139\") " Dec 03 12:41:01 crc kubenswrapper[5052]: I1203 12:41:01.285962 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"mysql-db\" (UniqueName: \"kubernetes.io/local-volume/local-storage11-crc\") pod \"f89ce0f5-5a92-49eb-9951-f2b491ac2139\" (UID: \"f89ce0f5-5a92-49eb-9951-f2b491ac2139\") " Dec 03 12:41:01 crc kubenswrapper[5052]: I1203 12:41:01.286005 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f89ce0f5-5a92-49eb-9951-f2b491ac2139-combined-ca-bundle\") pod \"f89ce0f5-5a92-49eb-9951-f2b491ac2139\" (UID: \"f89ce0f5-5a92-49eb-9951-f2b491ac2139\") " Dec 03 12:41:01 crc kubenswrapper[5052]: I1203 12:41:01.286047 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/f89ce0f5-5a92-49eb-9951-f2b491ac2139-kolla-config\") pod \"f89ce0f5-5a92-49eb-9951-f2b491ac2139\" (UID: \"f89ce0f5-5a92-49eb-9951-f2b491ac2139\") " Dec 03 12:41:01 crc kubenswrapper[5052]: I1203 12:41:01.286171 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/f89ce0f5-5a92-49eb-9951-f2b491ac2139-config-data-generated\") pod \"f89ce0f5-5a92-49eb-9951-f2b491ac2139\" (UID: \"f89ce0f5-5a92-49eb-9951-f2b491ac2139\") " Dec 03 12:41:01 crc kubenswrapper[5052]: I1203 12:41:01.286196 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kr5ht\" (UniqueName: \"kubernetes.io/projected/f89ce0f5-5a92-49eb-9951-f2b491ac2139-kube-api-access-kr5ht\") pod \"f89ce0f5-5a92-49eb-9951-f2b491ac2139\" (UID: \"f89ce0f5-5a92-49eb-9951-f2b491ac2139\") " Dec 03 12:41:01 crc kubenswrapper[5052]: E1203 12:41:01.286700 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-nova-public-svc: secret "cert-nova-public-svc" not found Dec 03 12:41:01 crc kubenswrapper[5052]: E1203 12:41:01.286747 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/50789f13-8194-4e24-b975-512591b816a5-public-tls-certs podName:50789f13-8194-4e24-b975-512591b816a5 nodeName:}" failed. No retries permitted until 2025-12-03 12:41:05.28673344 +0000 UTC m=+4855.874975204 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "public-tls-certs" (UniqueName: "kubernetes.io/secret/50789f13-8194-4e24-b975-512591b816a5-public-tls-certs") pod "nova-api-0" (UID: "50789f13-8194-4e24-b975-512591b816a5") : secret "cert-nova-public-svc" not found Dec 03 12:41:01 crc kubenswrapper[5052]: I1203 12:41:01.290968 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f89ce0f5-5a92-49eb-9951-f2b491ac2139-kolla-config" (OuterVolumeSpecName: "kolla-config") pod "f89ce0f5-5a92-49eb-9951-f2b491ac2139" (UID: "f89ce0f5-5a92-49eb-9951-f2b491ac2139"). InnerVolumeSpecName "kolla-config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:41:01 crc kubenswrapper[5052]: I1203 12:41:01.291385 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f89ce0f5-5a92-49eb-9951-f2b491ac2139-config-data-default" (OuterVolumeSpecName: "config-data-default") pod "f89ce0f5-5a92-49eb-9951-f2b491ac2139" (UID: "f89ce0f5-5a92-49eb-9951-f2b491ac2139"). InnerVolumeSpecName "config-data-default". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:41:01 crc kubenswrapper[5052]: I1203 12:41:01.291893 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f89ce0f5-5a92-49eb-9951-f2b491ac2139-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "f89ce0f5-5a92-49eb-9951-f2b491ac2139" (UID: "f89ce0f5-5a92-49eb-9951-f2b491ac2139"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:41:01 crc kubenswrapper[5052]: E1203 12:41:01.295845 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-nova-internal-svc: secret "cert-nova-internal-svc" not found Dec 03 12:41:01 crc kubenswrapper[5052]: E1203 12:41:01.295919 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/50789f13-8194-4e24-b975-512591b816a5-internal-tls-certs podName:50789f13-8194-4e24-b975-512591b816a5 nodeName:}" failed. No retries permitted until 2025-12-03 12:41:05.295898712 +0000 UTC m=+4855.884140476 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "internal-tls-certs" (UniqueName: "kubernetes.io/secret/50789f13-8194-4e24-b975-512591b816a5-internal-tls-certs") pod "nova-api-0" (UID: "50789f13-8194-4e24-b975-512591b816a5") : secret "cert-nova-internal-svc" not found Dec 03 12:41:01 crc kubenswrapper[5052]: I1203 12:41:01.297628 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f89ce0f5-5a92-49eb-9951-f2b491ac2139-config-data-generated" (OuterVolumeSpecName: "config-data-generated") pod "f89ce0f5-5a92-49eb-9951-f2b491ac2139" (UID: "f89ce0f5-5a92-49eb-9951-f2b491ac2139"). InnerVolumeSpecName "config-data-generated". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:41:01 crc kubenswrapper[5052]: E1203 12:41:01.297669 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/combined-ca-bundle: secret "combined-ca-bundle" not found Dec 03 12:41:01 crc kubenswrapper[5052]: E1203 12:41:01.297746 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/50789f13-8194-4e24-b975-512591b816a5-combined-ca-bundle podName:50789f13-8194-4e24-b975-512591b816a5 nodeName:}" failed. No retries permitted until 2025-12-03 12:41:05.297729354 +0000 UTC m=+4855.885971118 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "combined-ca-bundle" (UniqueName: "kubernetes.io/secret/50789f13-8194-4e24-b975-512591b816a5-combined-ca-bundle") pod "nova-api-0" (UID: "50789f13-8194-4e24-b975-512591b816a5") : secret "combined-ca-bundle" not found Dec 03 12:41:01 crc kubenswrapper[5052]: I1203 12:41:01.326111 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican1171-account-delete-9hmgl"] Dec 03 12:41:01 crc kubenswrapper[5052]: I1203 12:41:01.338776 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f89ce0f5-5a92-49eb-9951-f2b491ac2139-kube-api-access-kr5ht" (OuterVolumeSpecName: "kube-api-access-kr5ht") pod "f89ce0f5-5a92-49eb-9951-f2b491ac2139" (UID: "f89ce0f5-5a92-49eb-9951-f2b491ac2139"). InnerVolumeSpecName "kube-api-access-kr5ht". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:41:01 crc kubenswrapper[5052]: I1203 12:41:01.356820 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-sb-0"] Dec 03 12:41:01 crc kubenswrapper[5052]: I1203 12:41:01.357686 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/glance-default-internal-api-0" podUID="3039127c-0211-4430-9b2b-426412241ce4" containerName="glance-httpd" probeResult="failure" output="Get \"https://10.217.1.129:9292/healthcheck\": read tcp 10.217.0.2:33730->10.217.1.129:9292: read: connection reset by peer" Dec 03 12:41:01 crc kubenswrapper[5052]: I1203 12:41:01.357879 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/glance-default-internal-api-0" podUID="3039127c-0211-4430-9b2b-426412241ce4" containerName="glance-log" probeResult="failure" output="Get \"https://10.217.1.129:9292/healthcheck\": read tcp 10.217.0.2:33720->10.217.1.129:9292: read: connection reset by peer" Dec 03 12:41:01 crc kubenswrapper[5052]: I1203 12:41:01.373156 5052 scope.go:117] "RemoveContainer" containerID="c25218175c330c36ee6dc83936ebcb6c9c017af62ee188f8b82f5fd416a29e9c" Dec 03 12:41:01 crc kubenswrapper[5052]: I1203 12:41:01.381321 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-sb-0"] Dec 03 12:41:01 crc kubenswrapper[5052]: I1203 12:41:01.400940 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"nova-novncproxy-tls-certs\" (UniqueName: \"kubernetes.io/secret/0f16f521-a181-40e4-a1ed-38f4c8862ffb-nova-novncproxy-tls-certs\") pod \"0f16f521-a181-40e4-a1ed-38f4c8862ffb\" (UID: \"0f16f521-a181-40e4-a1ed-38f4c8862ffb\") " Dec 03 12:41:01 crc kubenswrapper[5052]: I1203 12:41:01.401000 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-67l8z\" (UniqueName: \"kubernetes.io/projected/0f16f521-a181-40e4-a1ed-38f4c8862ffb-kube-api-access-67l8z\") pod \"0f16f521-a181-40e4-a1ed-38f4c8862ffb\" (UID: \"0f16f521-a181-40e4-a1ed-38f4c8862ffb\") " Dec 03 12:41:01 crc kubenswrapper[5052]: I1203 12:41:01.401039 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"vencrypt-tls-certs\" (UniqueName: \"kubernetes.io/secret/0f16f521-a181-40e4-a1ed-38f4c8862ffb-vencrypt-tls-certs\") pod \"0f16f521-a181-40e4-a1ed-38f4c8862ffb\" (UID: \"0f16f521-a181-40e4-a1ed-38f4c8862ffb\") " Dec 03 12:41:01 crc kubenswrapper[5052]: I1203 12:41:01.401706 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-proxy-bf56d5474-cq7b2" Dec 03 12:41:01 crc kubenswrapper[5052]: I1203 12:41:01.401776 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/0f16f521-a181-40e4-a1ed-38f4c8862ffb-config-data\") pod \"0f16f521-a181-40e4-a1ed-38f4c8862ffb\" (UID: \"0f16f521-a181-40e4-a1ed-38f4c8862ffb\") " Dec 03 12:41:01 crc kubenswrapper[5052]: I1203 12:41:01.401813 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0f16f521-a181-40e4-a1ed-38f4c8862ffb-combined-ca-bundle\") pod \"0f16f521-a181-40e4-a1ed-38f4c8862ffb\" (UID: \"0f16f521-a181-40e4-a1ed-38f4c8862ffb\") " Dec 03 12:41:01 crc kubenswrapper[5052]: I1203 12:41:01.403497 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/f89ce0f5-5a92-49eb-9951-f2b491ac2139-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:01 crc kubenswrapper[5052]: I1203 12:41:01.403570 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/f89ce0f5-5a92-49eb-9951-f2b491ac2139-config-data-default\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:01 crc kubenswrapper[5052]: I1203 12:41:01.403618 5052 reconciler_common.go:293] "Volume detached for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/f89ce0f5-5a92-49eb-9951-f2b491ac2139-kolla-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:01 crc kubenswrapper[5052]: I1203 12:41:01.403685 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/f89ce0f5-5a92-49eb-9951-f2b491ac2139-config-data-generated\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:01 crc kubenswrapper[5052]: I1203 12:41:01.403698 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-kr5ht\" (UniqueName: \"kubernetes.io/projected/f89ce0f5-5a92-49eb-9951-f2b491ac2139-kube-api-access-kr5ht\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:01 crc kubenswrapper[5052]: I1203 12:41:01.423456 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/keystone-db-sync-v5287"] Dec 03 12:41:01 crc kubenswrapper[5052]: I1203 12:41:01.423591 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage11-crc" (OuterVolumeSpecName: "mysql-db") pod "f89ce0f5-5a92-49eb-9951-f2b491ac2139" (UID: "f89ce0f5-5a92-49eb-9951-f2b491ac2139"). InnerVolumeSpecName "local-storage11-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 12:41:01 crc kubenswrapper[5052]: I1203 12:41:01.437392 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/keystone-bootstrap-9grd9"] Dec 03 12:41:01 crc kubenswrapper[5052]: I1203 12:41:01.446669 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/keystone-db-sync-v5287"] Dec 03 12:41:01 crc kubenswrapper[5052]: I1203 12:41:01.455431 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/keystone-bootstrap-9grd9"] Dec 03 12:41:01 crc kubenswrapper[5052]: I1203 12:41:01.456995 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/glance-default-external-api-0" podUID="b0606156-f409-4cae-9be1-15d990f69955" containerName="glance-log" probeResult="failure" output="Get \"https://10.217.1.128:9292/healthcheck\": read tcp 10.217.0.2:56266->10.217.1.128:9292: read: connection reset by peer" Dec 03 12:41:01 crc kubenswrapper[5052]: I1203 12:41:01.457452 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/glance-default-external-api-0" podUID="b0606156-f409-4cae-9be1-15d990f69955" containerName="glance-httpd" probeResult="failure" output="Get \"https://10.217.1.128:9292/healthcheck\": read tcp 10.217.0.2:56268->10.217.1.128:9292: read: connection reset by peer" Dec 03 12:41:01 crc kubenswrapper[5052]: I1203 12:41:01.472776 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/kube-state-metrics-0"] Dec 03 12:41:01 crc kubenswrapper[5052]: I1203 12:41:01.501746 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/kube-state-metrics-0"] Dec 03 12:41:01 crc kubenswrapper[5052]: I1203 12:41:01.504925 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/621935c3-f8f9-43a1-9bc3-6d6ad9caf0ef-log-httpd\") pod \"621935c3-f8f9-43a1-9bc3-6d6ad9caf0ef\" (UID: \"621935c3-f8f9-43a1-9bc3-6d6ad9caf0ef\") " Dec 03 12:41:01 crc kubenswrapper[5052]: I1203 12:41:01.505412 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/621935c3-f8f9-43a1-9bc3-6d6ad9caf0ef-log-httpd" (OuterVolumeSpecName: "log-httpd") pod "621935c3-f8f9-43a1-9bc3-6d6ad9caf0ef" (UID: "621935c3-f8f9-43a1-9bc3-6d6ad9caf0ef"). InnerVolumeSpecName "log-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:41:01 crc kubenswrapper[5052]: I1203 12:41:01.505473 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4mnb7\" (UniqueName: \"kubernetes.io/projected/621935c3-f8f9-43a1-9bc3-6d6ad9caf0ef-kube-api-access-4mnb7\") pod \"621935c3-f8f9-43a1-9bc3-6d6ad9caf0ef\" (UID: \"621935c3-f8f9-43a1-9bc3-6d6ad9caf0ef\") " Dec 03 12:41:01 crc kubenswrapper[5052]: I1203 12:41:01.505499 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/621935c3-f8f9-43a1-9bc3-6d6ad9caf0ef-config-data\") pod \"621935c3-f8f9-43a1-9bc3-6d6ad9caf0ef\" (UID: \"621935c3-f8f9-43a1-9bc3-6d6ad9caf0ef\") " Dec 03 12:41:01 crc kubenswrapper[5052]: I1203 12:41:01.505574 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/621935c3-f8f9-43a1-9bc3-6d6ad9caf0ef-run-httpd\") pod \"621935c3-f8f9-43a1-9bc3-6d6ad9caf0ef\" (UID: \"621935c3-f8f9-43a1-9bc3-6d6ad9caf0ef\") " Dec 03 12:41:01 crc kubenswrapper[5052]: I1203 12:41:01.505621 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/621935c3-f8f9-43a1-9bc3-6d6ad9caf0ef-combined-ca-bundle\") pod \"621935c3-f8f9-43a1-9bc3-6d6ad9caf0ef\" (UID: \"621935c3-f8f9-43a1-9bc3-6d6ad9caf0ef\") " Dec 03 12:41:01 crc kubenswrapper[5052]: I1203 12:41:01.505683 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/621935c3-f8f9-43a1-9bc3-6d6ad9caf0ef-public-tls-certs\") pod \"621935c3-f8f9-43a1-9bc3-6d6ad9caf0ef\" (UID: \"621935c3-f8f9-43a1-9bc3-6d6ad9caf0ef\") " Dec 03 12:41:01 crc kubenswrapper[5052]: I1203 12:41:01.505713 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/621935c3-f8f9-43a1-9bc3-6d6ad9caf0ef-etc-swift\") pod \"621935c3-f8f9-43a1-9bc3-6d6ad9caf0ef\" (UID: \"621935c3-f8f9-43a1-9bc3-6d6ad9caf0ef\") " Dec 03 12:41:01 crc kubenswrapper[5052]: I1203 12:41:01.505762 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/621935c3-f8f9-43a1-9bc3-6d6ad9caf0ef-internal-tls-certs\") pod \"621935c3-f8f9-43a1-9bc3-6d6ad9caf0ef\" (UID: \"621935c3-f8f9-43a1-9bc3-6d6ad9caf0ef\") " Dec 03 12:41:01 crc kubenswrapper[5052]: I1203 12:41:01.506359 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage11-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage11-crc\") on node \"crc\" " Dec 03 12:41:01 crc kubenswrapper[5052]: I1203 12:41:01.506383 5052 reconciler_common.go:293] "Volume detached for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/621935c3-f8f9-43a1-9bc3-6d6ad9caf0ef-log-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:01 crc kubenswrapper[5052]: I1203 12:41:01.508619 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/621935c3-f8f9-43a1-9bc3-6d6ad9caf0ef-run-httpd" (OuterVolumeSpecName: "run-httpd") pod "621935c3-f8f9-43a1-9bc3-6d6ad9caf0ef" (UID: "621935c3-f8f9-43a1-9bc3-6d6ad9caf0ef"). InnerVolumeSpecName "run-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:41:01 crc kubenswrapper[5052]: I1203 12:41:01.570002 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/keystone-66f85f5bfb-vln92"] Dec 03 12:41:01 crc kubenswrapper[5052]: I1203 12:41:01.570940 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0f16f521-a181-40e4-a1ed-38f4c8862ffb-kube-api-access-67l8z" (OuterVolumeSpecName: "kube-api-access-67l8z") pod "0f16f521-a181-40e4-a1ed-38f4c8862ffb" (UID: "0f16f521-a181-40e4-a1ed-38f4c8862ffb"). InnerVolumeSpecName "kube-api-access-67l8z". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:41:01 crc kubenswrapper[5052]: I1203 12:41:01.570348 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/keystone-66f85f5bfb-vln92" podUID="fdeb105e-9fbc-44bb-ba15-bc81c7275db1" containerName="keystone-api" containerID="cri-o://303e4e73500eec972e0823935bb4cf2cb918b5d2b159c46edd8b37894c6d37f1" gracePeriod=30 Dec 03 12:41:01 crc kubenswrapper[5052]: I1203 12:41:01.572295 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/621935c3-f8f9-43a1-9bc3-6d6ad9caf0ef-etc-swift" (OuterVolumeSpecName: "etc-swift") pod "621935c3-f8f9-43a1-9bc3-6d6ad9caf0ef" (UID: "621935c3-f8f9-43a1-9bc3-6d6ad9caf0ef"). InnerVolumeSpecName "etc-swift". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:41:01 crc kubenswrapper[5052]: I1203 12:41:01.579790 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/621935c3-f8f9-43a1-9bc3-6d6ad9caf0ef-kube-api-access-4mnb7" (OuterVolumeSpecName: "kube-api-access-4mnb7") pod "621935c3-f8f9-43a1-9bc3-6d6ad9caf0ef" (UID: "621935c3-f8f9-43a1-9bc3-6d6ad9caf0ef"). InnerVolumeSpecName "kube-api-access-4mnb7". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:41:01 crc kubenswrapper[5052]: I1203 12:41:01.585787 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" podUID="f9bac64c-518e-4630-8889-20e09d5b7a39" containerName="rabbitmq" probeResult="failure" output="dial tcp 10.217.1.5:5671: connect: connection refused" Dec 03 12:41:01 crc kubenswrapper[5052]: I1203 12:41:01.595571 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/openstack-galera-0"] Dec 03 12:41:01 crc kubenswrapper[5052]: I1203 12:41:01.617669 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-4mnb7\" (UniqueName: \"kubernetes.io/projected/621935c3-f8f9-43a1-9bc3-6d6ad9caf0ef-kube-api-access-4mnb7\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:01 crc kubenswrapper[5052]: I1203 12:41:01.617717 5052 reconciler_common.go:293] "Volume detached for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/621935c3-f8f9-43a1-9bc3-6d6ad9caf0ef-run-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:01 crc kubenswrapper[5052]: I1203 12:41:01.617729 5052 reconciler_common.go:293] "Volume detached for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/621935c3-f8f9-43a1-9bc3-6d6ad9caf0ef-etc-swift\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:01 crc kubenswrapper[5052]: I1203 12:41:01.617742 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-67l8z\" (UniqueName: \"kubernetes.io/projected/0f16f521-a181-40e4-a1ed-38f4c8862ffb-kube-api-access-67l8z\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:01 crc kubenswrapper[5052]: I1203 12:41:01.651605 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/novaapib163-account-delete-8z4bt"] Dec 03 12:41:01 crc kubenswrapper[5052]: I1203 12:41:01.692413 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0a3ff09e-7256-4090-9060-099706213333" path="/var/lib/kubelet/pods/0a3ff09e-7256-4090-9060-099706213333/volumes" Dec 03 12:41:01 crc kubenswrapper[5052]: I1203 12:41:01.693133 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7ce91d4a-389d-4b37-b582-8c886566aa74" path="/var/lib/kubelet/pods/7ce91d4a-389d-4b37-b582-8c886566aa74/volumes" Dec 03 12:41:01 crc kubenswrapper[5052]: I1203 12:41:01.693745 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="88758015-0ff4-4b36-bb8b-3b914824871c" path="/var/lib/kubelet/pods/88758015-0ff4-4b36-bb8b-3b914824871c/volumes" Dec 03 12:41:01 crc kubenswrapper[5052]: I1203 12:41:01.695236 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b7defe22-28b4-4fec-8fe5-ebde8e658c6b" path="/var/lib/kubelet/pods/b7defe22-28b4-4fec-8fe5-ebde8e658c6b/volumes" Dec 03 12:41:01 crc kubenswrapper[5052]: I1203 12:41:01.696333 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d6bedb3e-52e6-44a9-afd9-a164b0730f52" path="/var/lib/kubelet/pods/d6bedb3e-52e6-44a9-afd9-a164b0730f52/volumes" Dec 03 12:41:01 crc kubenswrapper[5052]: I1203 12:41:01.713700 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/cinder-api-0" podUID="5bca7a3b-316e-4357-8ae4-e47be319dbf1" containerName="cinder-api" probeResult="failure" output="Get \"https://10.217.1.120:8776/healthcheck\": read tcp 10.217.0.2:54116->10.217.1.120:8776: read: connection reset by peer" Dec 03 12:41:01 crc kubenswrapper[5052]: I1203 12:41:01.814781 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0f16f521-a181-40e4-a1ed-38f4c8862ffb-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "0f16f521-a181-40e4-a1ed-38f4c8862ffb" (UID: "0f16f521-a181-40e4-a1ed-38f4c8862ffb"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:41:01 crc kubenswrapper[5052]: I1203 12:41:01.821930 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0f16f521-a181-40e4-a1ed-38f4c8862ffb-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:01 crc kubenswrapper[5052]: I1203 12:41:01.849399 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage11-crc" (UniqueName: "kubernetes.io/local-volume/local-storage11-crc") on node "crc" Dec 03 12:41:01 crc kubenswrapper[5052]: I1203 12:41:01.906074 5052 generic.go:334] "Generic (PLEG): container finished" podID="5bca7a3b-316e-4357-8ae4-e47be319dbf1" containerID="87d55b80754e67e2615d7cea19e5d57d82b054e86c70bcb48fc87f5b7efa28d9" exitCode=0 Dec 03 12:41:01 crc kubenswrapper[5052]: I1203 12:41:01.921734 5052 generic.go:334] "Generic (PLEG): container finished" podID="b0606156-f409-4cae-9be1-15d990f69955" containerID="8ca42e62aac09b81598563bdb887c5ea6090758cf976adc8ad4b4e387ec44287" exitCode=0 Dec 03 12:41:01 crc kubenswrapper[5052]: I1203 12:41:01.925758 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage11-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage11-crc\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:01 crc kubenswrapper[5052]: I1203 12:41:01.934075 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f89ce0f5-5a92-49eb-9951-f2b491ac2139-galera-tls-certs" (OuterVolumeSpecName: "galera-tls-certs") pod "f89ce0f5-5a92-49eb-9951-f2b491ac2139" (UID: "f89ce0f5-5a92-49eb-9951-f2b491ac2139"). InnerVolumeSpecName "galera-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:41:01 crc kubenswrapper[5052]: I1203 12:41:01.950637 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0f16f521-a181-40e4-a1ed-38f4c8862ffb-config-data" (OuterVolumeSpecName: "config-data") pod "0f16f521-a181-40e4-a1ed-38f4c8862ffb" (UID: "0f16f521-a181-40e4-a1ed-38f4c8862ffb"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:41:01 crc kubenswrapper[5052]: I1203 12:41:01.950899 5052 generic.go:334] "Generic (PLEG): container finished" podID="99a0961b-8ed8-49b7-abac-5025d7b351ee" containerID="60eb1509e2cea82540bce951978d0b80c496f80fe40a218c77ecbe97a3e9c835" exitCode=0 Dec 03 12:41:02 crc kubenswrapper[5052]: I1203 12:41:02.008832 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f89ce0f5-5a92-49eb-9951-f2b491ac2139-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "f89ce0f5-5a92-49eb-9951-f2b491ac2139" (UID: "f89ce0f5-5a92-49eb-9951-f2b491ac2139"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:41:02 crc kubenswrapper[5052]: I1203 12:41:02.027345 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f89ce0f5-5a92-49eb-9951-f2b491ac2139-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:02 crc kubenswrapper[5052]: I1203 12:41:02.027436 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/0f16f521-a181-40e4-a1ed-38f4c8862ffb-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:02 crc kubenswrapper[5052]: I1203 12:41:02.027449 5052 reconciler_common.go:293] "Volume detached for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/f89ce0f5-5a92-49eb-9951-f2b491ac2139-galera-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:02 crc kubenswrapper[5052]: I1203 12:41:02.027855 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/621935c3-f8f9-43a1-9bc3-6d6ad9caf0ef-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "621935c3-f8f9-43a1-9bc3-6d6ad9caf0ef" (UID: "621935c3-f8f9-43a1-9bc3-6d6ad9caf0ef"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:41:02 crc kubenswrapper[5052]: I1203 12:41:02.033210 5052 kubelet_pods.go:1007] "Unable to retrieve pull secret, the image pull may not succeed." pod="openstack-kuttl-tests/cinder3a0d-account-delete-lxd6l" secret="" err="secret \"galera-openstack-dockercfg-hsg7j\" not found" Dec 03 12:41:02 crc kubenswrapper[5052]: I1203 12:41:02.043034 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/621935c3-f8f9-43a1-9bc3-6d6ad9caf0ef-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "621935c3-f8f9-43a1-9bc3-6d6ad9caf0ef" (UID: "621935c3-f8f9-43a1-9bc3-6d6ad9caf0ef"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:41:02 crc kubenswrapper[5052]: I1203 12:41:02.044930 5052 generic.go:334] "Generic (PLEG): container finished" podID="3039127c-0211-4430-9b2b-426412241ce4" containerID="1d3c75c1863b6df410b4948455d1e97a7c92fce681cbc217dafa1d03a2fae0c6" exitCode=0 Dec 03 12:41:02 crc kubenswrapper[5052]: I1203 12:41:02.048705 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/621935c3-f8f9-43a1-9bc3-6d6ad9caf0ef-config-data" (OuterVolumeSpecName: "config-data") pod "621935c3-f8f9-43a1-9bc3-6d6ad9caf0ef" (UID: "621935c3-f8f9-43a1-9bc3-6d6ad9caf0ef"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:41:02 crc kubenswrapper[5052]: I1203 12:41:02.049006 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-proxy-bf56d5474-cq7b2" Dec 03 12:41:02 crc kubenswrapper[5052]: I1203 12:41:02.060704 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/barbican-api-bcdbf47bb-m2ctz" podUID="a83592aa-a4c1-459e-9d07-f14b390a5e6b" containerName="barbican-api" probeResult="failure" output="Get \"https://10.217.1.124:9311/healthcheck\": read tcp 10.217.0.2:33046->10.217.1.124:9311: read: connection reset by peer" Dec 03 12:41:02 crc kubenswrapper[5052]: I1203 12:41:02.061863 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/barbican-api-bcdbf47bb-m2ctz" podUID="a83592aa-a4c1-459e-9d07-f14b390a5e6b" containerName="barbican-api-log" probeResult="failure" output="Get \"https://10.217.1.124:9311/healthcheck\": read tcp 10.217.0.2:33054->10.217.1.124:9311: read: connection reset by peer" Dec 03 12:41:02 crc kubenswrapper[5052]: I1203 12:41:02.065183 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:41:02 crc kubenswrapper[5052]: I1203 12:41:02.065699 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:41:02 crc kubenswrapper[5052]: I1203 12:41:02.129834 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/621935c3-f8f9-43a1-9bc3-6d6ad9caf0ef-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:02 crc kubenswrapper[5052]: I1203 12:41:02.129893 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/621935c3-f8f9-43a1-9bc3-6d6ad9caf0ef-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:02 crc kubenswrapper[5052]: I1203 12:41:02.129906 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/621935c3-f8f9-43a1-9bc3-6d6ad9caf0ef-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:02 crc kubenswrapper[5052]: E1203 12:41:02.130769 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:41:02 crc kubenswrapper[5052]: E1203 12:41:02.130830 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5fdc7b7d-f9ed-4d5c-a77b-6a6b0fd186f9-operator-scripts podName:5fdc7b7d-f9ed-4d5c-a77b-6a6b0fd186f9 nodeName:}" failed. No retries permitted until 2025-12-03 12:41:02.630811059 +0000 UTC m=+4853.219052833 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/5fdc7b7d-f9ed-4d5c-a77b-6a6b0fd186f9-operator-scripts") pod "cinder3a0d-account-delete-lxd6l" (UID: "5fdc7b7d-f9ed-4d5c-a77b-6a6b0fd186f9") : configmap "openstack-scripts" not found Dec 03 12:41:02 crc kubenswrapper[5052]: I1203 12:41:02.152986 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/openstack-galera-0" podUID="a7ae5e02-8418-40e2-b65d-7b74625fe5b3" containerName="galera" containerID="cri-o://fb98f0e6943bcf2cc568bb5bb0901b5c57179b59a118343bafcea6218d35b382" gracePeriod=30 Dec 03 12:41:02 crc kubenswrapper[5052]: I1203 12:41:02.176526 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0f16f521-a181-40e4-a1ed-38f4c8862ffb-nova-novncproxy-tls-certs" (OuterVolumeSpecName: "nova-novncproxy-tls-certs") pod "0f16f521-a181-40e4-a1ed-38f4c8862ffb" (UID: "0f16f521-a181-40e4-a1ed-38f4c8862ffb"). InnerVolumeSpecName "nova-novncproxy-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:41:02 crc kubenswrapper[5052]: I1203 12:41:02.191369 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/621935c3-f8f9-43a1-9bc3-6d6ad9caf0ef-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "621935c3-f8f9-43a1-9bc3-6d6ad9caf0ef" (UID: "621935c3-f8f9-43a1-9bc3-6d6ad9caf0ef"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:41:02 crc kubenswrapper[5052]: I1203 12:41:02.208696 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0f16f521-a181-40e4-a1ed-38f4c8862ffb-vencrypt-tls-certs" (OuterVolumeSpecName: "vencrypt-tls-certs") pod "0f16f521-a181-40e4-a1ed-38f4c8862ffb" (UID: "0f16f521-a181-40e4-a1ed-38f4c8862ffb"). InnerVolumeSpecName "vencrypt-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:41:02 crc kubenswrapper[5052]: I1203 12:41:02.232700 5052 reconciler_common.go:293] "Volume detached for volume \"nova-novncproxy-tls-certs\" (UniqueName: \"kubernetes.io/secret/0f16f521-a181-40e4-a1ed-38f4c8862ffb-nova-novncproxy-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:02 crc kubenswrapper[5052]: I1203 12:41:02.232732 5052 reconciler_common.go:293] "Volume detached for volume \"vencrypt-tls-certs\" (UniqueName: \"kubernetes.io/secret/0f16f521-a181-40e4-a1ed-38f4c8862ffb-vencrypt-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:02 crc kubenswrapper[5052]: I1203 12:41:02.232742 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/621935c3-f8f9-43a1-9bc3-6d6ad9caf0ef-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:02 crc kubenswrapper[5052]: I1203 12:41:02.236352 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/cinder3a0d-account-delete-lxd6l" podStartSLOduration=5.236327987 podStartE2EDuration="5.236327987s" podCreationTimestamp="2025-12-03 12:40:57 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:41:02.061989831 +0000 UTC m=+4852.650231605" watchObservedRunningTime="2025-12-03 12:41:02.236327987 +0000 UTC m=+4852.824569751" Dec 03 12:41:02 crc kubenswrapper[5052]: I1203 12:41:02.351502 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/nova-metadata-0" podUID="30045a80-9e2b-416b-b7bf-cdb9585ec67d" containerName="nova-metadata-metadata" probeResult="failure" output="Get \"https://10.217.1.125:8775/\": dial tcp 10.217.1.125:8775: connect: connection refused" Dec 03 12:41:02 crc kubenswrapper[5052]: I1203 12:41:02.352251 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/nova-metadata-0" podUID="30045a80-9e2b-416b-b7bf-cdb9585ec67d" containerName="nova-metadata-log" probeResult="failure" output="Get \"https://10.217.1.125:8775/\": dial tcp 10.217.1.125:8775: connect: connection refused" Dec 03 12:41:02 crc kubenswrapper[5052]: I1203 12:41:02.616955 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/memcached-0" podUID="5d19811b-4e82-42b2-b6ec-f16e707484f2" containerName="memcached" probeResult="failure" output="dial tcp 10.217.1.117:11211: connect: connection refused" Dec 03 12:41:02 crc kubenswrapper[5052]: E1203 12:41:02.651182 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:41:02 crc kubenswrapper[5052]: E1203 12:41:02.651270 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5fdc7b7d-f9ed-4d5c-a77b-6a6b0fd186f9-operator-scripts podName:5fdc7b7d-f9ed-4d5c-a77b-6a6b0fd186f9 nodeName:}" failed. No retries permitted until 2025-12-03 12:41:03.651244603 +0000 UTC m=+4854.239486367 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/5fdc7b7d-f9ed-4d5c-a77b-6a6b0fd186f9-operator-scripts") pod "cinder3a0d-account-delete-lxd6l" (UID: "5fdc7b7d-f9ed-4d5c-a77b-6a6b0fd186f9") : configmap "openstack-scripts" not found Dec 03 12:41:02 crc kubenswrapper[5052]: E1203 12:41:02.923708 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="382891affaf25ff9b098ddaf284f3fa5667c14b93f3487974c23c2cdae354300" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:41:02 crc kubenswrapper[5052]: E1203 12:41:02.927381 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="382891affaf25ff9b098ddaf284f3fa5667c14b93f3487974c23c2cdae354300" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:41:02 crc kubenswrapper[5052]: E1203 12:41:02.931676 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="382891affaf25ff9b098ddaf284f3fa5667c14b93f3487974c23c2cdae354300" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:41:02 crc kubenswrapper[5052]: E1203 12:41:02.931744 5052 prober.go:104] "Probe errored" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" probeType="Readiness" pod="openstack-kuttl-tests/nova-cell1-conductor-0" podUID="0556e9f0-e431-4193-9822-6079cb8a1e44" containerName="nova-cell1-conductor-conductor" Dec 03 12:41:03 crc kubenswrapper[5052]: E1203 12:41:03.016555 5052 kubelet.go:2526] "Housekeeping took longer than expected" err="housekeeping took too long" expected="1s" actual="1.344s" Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.016590 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.016992 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.017046 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/keystone-db-create-mvj42"] Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.017165 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/novacell07298-account-delete-726jv" event={"ID":"6dbc7d8f-34d4-4927-82fe-d049727808fd","Type":"ContainerStarted","Data":"89e35dabb00061c9ad442c01b1fc1047b590d4300a9f6a3f62d174b78e9185b4"} Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.017228 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/keystone-db-create-mvj42"] Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.017626 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-nb-0"] Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.017820 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-nb-0"] Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.018149 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/keystone-4ba6-account-create-update-nr8wd"] Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.018178 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/keystone-4ba6-account-create-update-nr8wd"] Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.018194 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"5bca7a3b-316e-4357-8ae4-e47be319dbf1","Type":"ContainerDied","Data":"87d55b80754e67e2615d7cea19e5d57d82b054e86c70bcb48fc87f5b7efa28d9"} Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.018215 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican1171-account-delete-9hmgl" event={"ID":"ab29f82e-0693-4001-9f2a-bb4cc2fa1d5a","Type":"ContainerStarted","Data":"8b84c689f4cc667ba6eb2f6916edf725df218b56118c0a6591c886c94674990a"} Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.018230 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-8h8qz" event={"ID":"296f8cec-ad7b-4b51-a9a9-934301d43d2e","Type":"ContainerStarted","Data":"cc15d46c7ef59e21f31e8fd200ff54e6e4037db8acc39ec7a4e3a93f9175feeb"} Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.018244 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement7f30-account-delete-vld4v" event={"ID":"eb8b91d5-6c99-45ba-8d33-da99f1dd496c","Type":"ContainerStarted","Data":"2797d43ce4aacb7212eabdda418f85dbb3bdf966af8318c082d3d3f611404fd5"} Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.018266 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-db-create-tsdrr"] Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.018283 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/glance-db-create-tsdrr"] Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.018296 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-c94f-account-create-update-jjfpq"] Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.018309 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/glance-c94f-account-create-update-jjfpq"] Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.018322 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"b0606156-f409-4cae-9be1-15d990f69955","Type":"ContainerDied","Data":"8ca42e62aac09b81598563bdb887c5ea6090758cf976adc8ad4b4e387ec44287"} Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.018340 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-cdf4456f4-v82rz" event={"ID":"99a0961b-8ed8-49b7-abac-5025d7b351ee","Type":"ContainerDied","Data":"60eb1509e2cea82540bce951978d0b80c496f80fe40a218c77ecbe97a3e9c835"} Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.018360 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder3a0d-account-delete-lxd6l" event={"ID":"5fdc7b7d-f9ed-4d5c-a77b-6a6b0fd186f9","Type":"ContainerStarted","Data":"660e428afdfd8a7e278a3dd2d6aa79748d2d7485dda4c8e8678801b9bcc2009e"} Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.018378 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glancec94f-account-delete-8t6wx"] Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.018394 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/novaapib163-account-delete-8z4bt" event={"ID":"4b7b82d8-699c-43f2-8ab0-01936305623f","Type":"ContainerStarted","Data":"28edaca94f750161a85e242825c026fc53591ffe57899bd5e56e3d735dde155e"} Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.018411 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/neutron-db-create-78h7j"] Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.018429 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/neutron-db-create-78h7j"] Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.018478 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"3039127c-0211-4430-9b2b-426412241ce4","Type":"ContainerDied","Data":"1d3c75c1863b6df410b4948455d1e97a7c92fce681cbc217dafa1d03a2fae0c6"} Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.018499 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/neutron-d24e-account-create-update-gskxw"] Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.018528 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-proxy-bf56d5474-cq7b2" event={"ID":"621935c3-f8f9-43a1-9bc3-6d6ad9caf0ef","Type":"ContainerDied","Data":"b1c287082c6be0fc94453ba31598d9aa15f58f1efff238cadada8687a0b597d4"} Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.018549 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/neutrond24e-account-delete-p2mpp"] Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.018567 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/neutron-d24e-account-create-update-gskxw"] Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.018580 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder-db-create-kc855"] Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.018593 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/cinder-db-create-kc855"] Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.018675 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder3a0d-account-delete-lxd6l"] Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.018690 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder-3a0d-account-create-update-wn6fm"] Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.018703 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/cinder-3a0d-account-create-update-wn6fm"] Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.018753 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/placement-db-create-mzssh"] Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.018766 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/placement-db-create-mzssh"] Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.018778 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/placement-7f30-account-create-update-zx4ld"] Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.018826 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/placement7f30-account-delete-vld4v"] Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.018843 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/placement-7f30-account-create-update-zx4ld"] Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.018855 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-db-create-j9nw5"] Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.018866 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/barbican-db-create-j9nw5"] Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.018877 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican1171-account-delete-9hmgl"] Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.018888 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-1171-account-create-update-gm2mn"] Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.018899 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/barbican-1171-account-create-update-gm2mn"] Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.018910 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell0-db-create-4cj5q"] Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.018962 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell0-db-create-4cj5q"] Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.018980 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell0-7298-account-create-update-5sx49"] Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.018990 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/novacell07298-account-delete-726jv"] Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.019004 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell0-7298-account-create-update-5sx49"] Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.019016 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-api-db-create-k2lkr"] Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.019026 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-api-db-create-k2lkr"] Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.019038 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/novaapib163-account-delete-8z4bt"] Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.025943 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-api-b163-account-create-update-d97hs"] Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.029933 5052 scope.go:117] "RemoveContainer" containerID="84ca5b96b0f3458adcf7e677440d6f30818b02e536633792be662d5c24cf8b44" Dec 03 12:41:03 crc kubenswrapper[5052]: E1203 12:41:03.031909 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"84ca5b96b0f3458adcf7e677440d6f30818b02e536633792be662d5c24cf8b44\": container with ID starting with 84ca5b96b0f3458adcf7e677440d6f30818b02e536633792be662d5c24cf8b44 not found: ID does not exist" containerID="84ca5b96b0f3458adcf7e677440d6f30818b02e536633792be662d5c24cf8b44" Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.031965 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"84ca5b96b0f3458adcf7e677440d6f30818b02e536633792be662d5c24cf8b44"} err="failed to get container status \"84ca5b96b0f3458adcf7e677440d6f30818b02e536633792be662d5c24cf8b44\": rpc error: code = NotFound desc = could not find container \"84ca5b96b0f3458adcf7e677440d6f30818b02e536633792be662d5c24cf8b44\": container with ID starting with 84ca5b96b0f3458adcf7e677440d6f30818b02e536633792be662d5c24cf8b44 not found: ID does not exist" Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.031998 5052 scope.go:117] "RemoveContainer" containerID="323efac3746af7ff7702e46c55dedadf7e2d8d45f1a20d6b7d1fa2753913a05d" Dec 03 12:41:03 crc kubenswrapper[5052]: E1203 12:41:03.032640 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"323efac3746af7ff7702e46c55dedadf7e2d8d45f1a20d6b7d1fa2753913a05d\": container with ID starting with 323efac3746af7ff7702e46c55dedadf7e2d8d45f1a20d6b7d1fa2753913a05d not found: ID does not exist" containerID="323efac3746af7ff7702e46c55dedadf7e2d8d45f1a20d6b7d1fa2753913a05d" Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.032684 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"323efac3746af7ff7702e46c55dedadf7e2d8d45f1a20d6b7d1fa2753913a05d"} err="failed to get container status \"323efac3746af7ff7702e46c55dedadf7e2d8d45f1a20d6b7d1fa2753913a05d\": rpc error: code = NotFound desc = could not find container \"323efac3746af7ff7702e46c55dedadf7e2d8d45f1a20d6b7d1fa2753913a05d\": container with ID starting with 323efac3746af7ff7702e46c55dedadf7e2d8d45f1a20d6b7d1fa2753913a05d not found: ID does not exist" Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.032697 5052 scope.go:117] "RemoveContainer" containerID="c25218175c330c36ee6dc83936ebcb6c9c017af62ee188f8b82f5fd416a29e9c" Dec 03 12:41:03 crc kubenswrapper[5052]: E1203 12:41:03.032958 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c25218175c330c36ee6dc83936ebcb6c9c017af62ee188f8b82f5fd416a29e9c\": container with ID starting with c25218175c330c36ee6dc83936ebcb6c9c017af62ee188f8b82f5fd416a29e9c not found: ID does not exist" containerID="c25218175c330c36ee6dc83936ebcb6c9c017af62ee188f8b82f5fd416a29e9c" Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.032994 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c25218175c330c36ee6dc83936ebcb6c9c017af62ee188f8b82f5fd416a29e9c"} err="failed to get container status \"c25218175c330c36ee6dc83936ebcb6c9c017af62ee188f8b82f5fd416a29e9c\": rpc error: code = NotFound desc = could not find container \"c25218175c330c36ee6dc83936ebcb6c9c017af62ee188f8b82f5fd416a29e9c\": container with ID starting with c25218175c330c36ee6dc83936ebcb6c9c017af62ee188f8b82f5fd416a29e9c not found: ID does not exist" Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.033007 5052 scope.go:117] "RemoveContainer" containerID="26de7dd666c3dea50b83a35cff57a4ece7ec413a749af698ab13d88f7a800f76" Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.037824 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-api-b163-account-create-update-d97hs"] Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.087684 5052 generic.go:334] "Generic (PLEG): container finished" podID="bc0b4316-9c87-4c72-bc06-b4b526c2070e" containerID="a19585c85edfd20edc4392a9229786e6495d1e999b33f47c80f6aec8b04d2bf2" exitCode=0 Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.087799 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-keystone-listener-74b68b767b-8zpnm" event={"ID":"bc0b4316-9c87-4c72-bc06-b4b526c2070e","Type":"ContainerDied","Data":"a19585c85edfd20edc4392a9229786e6495d1e999b33f47c80f6aec8b04d2bf2"} Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.090615 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"3039127c-0211-4430-9b2b-426412241ce4","Type":"ContainerDied","Data":"39c9c428d2f039ed7354568dafcb88fcc2e42dff18fcfb12022933e6e7c65a34"} Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.090652 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="39c9c428d2f039ed7354568dafcb88fcc2e42dff18fcfb12022933e6e7c65a34" Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.092718 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glancec94f-account-delete-8t6wx" event={"ID":"b7c49d90-c0af-4d4b-a080-8c7dce3ee66a","Type":"ContainerDied","Data":"d7a710caf6f50b57a6b12dca0e90d790995d331e34888adedbd735d99df15c2d"} Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.092733 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d7a710caf6f50b57a6b12dca0e90d790995d331e34888adedbd735d99df15c2d" Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.094601 5052 generic.go:334] "Generic (PLEG): container finished" podID="18a62c9b-4032-4639-be99-3520b515313b" containerID="0aea18d8ca92744fcf1bb8068e5ff8a5eb48807d9cf65983d90eb513bcd33ddb" exitCode=0 Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.094634 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-worker-7bd785845-99jqj" event={"ID":"18a62c9b-4032-4639-be99-3520b515313b","Type":"ContainerDied","Data":"0aea18d8ca92744fcf1bb8068e5ff8a5eb48807d9cf65983d90eb513bcd33ddb"} Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.096502 5052 generic.go:334] "Generic (PLEG): container finished" podID="50789f13-8194-4e24-b975-512591b816a5" containerID="256ac84e475433107bd853223013a808d99354e60928d4ce86ddc71170d1b9a7" exitCode=0 Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.096558 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"50789f13-8194-4e24-b975-512591b816a5","Type":"ContainerDied","Data":"256ac84e475433107bd853223013a808d99354e60928d4ce86ddc71170d1b9a7"} Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.099787 5052 scope.go:117] "RemoveContainer" containerID="53722c8da45292bfe9fdbd4b2d2a18b053dc900a020f10bc15c4643959d2c7c1" Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.112127 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"5bca7a3b-316e-4357-8ae4-e47be319dbf1","Type":"ContainerDied","Data":"f4509a279a5438f6fb012b7e3044b3b95b5feb8f15f16222f2e627c559da6ac9"} Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.112160 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f4509a279a5438f6fb012b7e3044b3b95b5feb8f15f16222f2e627c559da6ac9" Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.121149 5052 generic.go:334] "Generic (PLEG): container finished" podID="5d19811b-4e82-42b2-b6ec-f16e707484f2" containerID="78246e0cecbf2891b1eda71870c4a06e4588bbbe3e1fc591176960ad3a92ef57" exitCode=0 Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.121208 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/memcached-0" event={"ID":"5d19811b-4e82-42b2-b6ec-f16e707484f2","Type":"ContainerDied","Data":"78246e0cecbf2891b1eda71870c4a06e4588bbbe3e1fc591176960ad3a92ef57"} Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.129940 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-cdf4456f4-v82rz" event={"ID":"99a0961b-8ed8-49b7-abac-5025d7b351ee","Type":"ContainerDied","Data":"2feb5eaeb8e07d9d55d3220880f69471cec3c64e41d5b551d1b87b699609793c"} Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.129995 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2feb5eaeb8e07d9d55d3220880f69471cec3c64e41d5b551d1b87b699609793c" Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.134290 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement7f30-account-delete-vld4v" event={"ID":"eb8b91d5-6c99-45ba-8d33-da99f1dd496c","Type":"ContainerStarted","Data":"bb53f0acf7c936b9d6eebe4376be498cac662cc7e2d991df4516aca0dbea473e"} Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.134792 5052 kubelet_pods.go:1007] "Unable to retrieve pull secret, the image pull may not succeed." pod="openstack-kuttl-tests/placement7f30-account-delete-vld4v" secret="" err="secret \"galera-openstack-dockercfg-hsg7j\" not found" Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.139231 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.141181 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"b0606156-f409-4cae-9be1-15d990f69955","Type":"ContainerDied","Data":"ecfd28d673e77d737ec3a816279736e50a6835ccc5c4a396f288e0cd654d3288"} Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.141210 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ecfd28d673e77d737ec3a816279736e50a6835ccc5c4a396f288e0cd654d3288" Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.155972 5052 generic.go:334] "Generic (PLEG): container finished" podID="e9af392d-1940-4203-961e-39da1ac5f3da" containerID="88bb1e9d52b42c7f2f99966c163df2f71ff6d2b3bec16e67d60c4dbcaea76bb6" exitCode=0 Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.156064 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-conductor-0" event={"ID":"e9af392d-1940-4203-961e-39da1ac5f3da","Type":"ContainerDied","Data":"88bb1e9d52b42c7f2f99966c163df2f71ff6d2b3bec16e67d60c4dbcaea76bb6"} Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.158937 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/placement7f30-account-delete-vld4v" podStartSLOduration=6.158921911 podStartE2EDuration="6.158921911s" podCreationTimestamp="2025-12-03 12:40:57 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:41:03.152927939 +0000 UTC m=+4853.741169723" watchObservedRunningTime="2025-12-03 12:41:03.158921911 +0000 UTC m=+4853.747163675" Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.167077 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/3039127c-0211-4430-9b2b-426412241ce4-logs\") pod \"3039127c-0211-4430-9b2b-426412241ce4\" (UID: \"3039127c-0211-4430-9b2b-426412241ce4\") " Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.167128 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/3039127c-0211-4430-9b2b-426412241ce4-internal-tls-certs\") pod \"3039127c-0211-4430-9b2b-426412241ce4\" (UID: \"3039127c-0211-4430-9b2b-426412241ce4\") " Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.167211 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mclw8\" (UniqueName: \"kubernetes.io/projected/3039127c-0211-4430-9b2b-426412241ce4-kube-api-access-mclw8\") pod \"3039127c-0211-4430-9b2b-426412241ce4\" (UID: \"3039127c-0211-4430-9b2b-426412241ce4\") " Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.167236 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/3039127c-0211-4430-9b2b-426412241ce4-httpd-run\") pod \"3039127c-0211-4430-9b2b-426412241ce4\" (UID: \"3039127c-0211-4430-9b2b-426412241ce4\") " Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.167279 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/3039127c-0211-4430-9b2b-426412241ce4-scripts\") pod \"3039127c-0211-4430-9b2b-426412241ce4\" (UID: \"3039127c-0211-4430-9b2b-426412241ce4\") " Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.167305 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3039127c-0211-4430-9b2b-426412241ce4-combined-ca-bundle\") pod \"3039127c-0211-4430-9b2b-426412241ce4\" (UID: \"3039127c-0211-4430-9b2b-426412241ce4\") " Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.167357 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"glance\" (UniqueName: \"kubernetes.io/local-volume/local-storage18-crc\") pod \"3039127c-0211-4430-9b2b-426412241ce4\" (UID: \"3039127c-0211-4430-9b2b-426412241ce4\") " Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.167383 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/3039127c-0211-4430-9b2b-426412241ce4-config-data\") pod \"3039127c-0211-4430-9b2b-426412241ce4\" (UID: \"3039127c-0211-4430-9b2b-426412241ce4\") " Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.168071 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3039127c-0211-4430-9b2b-426412241ce4-httpd-run" (OuterVolumeSpecName: "httpd-run") pod "3039127c-0211-4430-9b2b-426412241ce4" (UID: "3039127c-0211-4430-9b2b-426412241ce4"). InnerVolumeSpecName "httpd-run". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:41:03 crc kubenswrapper[5052]: E1203 12:41:03.168091 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:41:03 crc kubenswrapper[5052]: E1203 12:41:03.168227 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/eb8b91d5-6c99-45ba-8d33-da99f1dd496c-operator-scripts podName:eb8b91d5-6c99-45ba-8d33-da99f1dd496c nodeName:}" failed. No retries permitted until 2025-12-03 12:41:03.668183636 +0000 UTC m=+4854.256425400 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/eb8b91d5-6c99-45ba-8d33-da99f1dd496c-operator-scripts") pod "placement7f30-account-delete-vld4v" (UID: "eb8b91d5-6c99-45ba-8d33-da99f1dd496c") : configmap "openstack-scripts" not found Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.169539 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3039127c-0211-4430-9b2b-426412241ce4-logs" (OuterVolumeSpecName: "logs") pod "3039127c-0211-4430-9b2b-426412241ce4" (UID: "3039127c-0211-4430-9b2b-426412241ce4"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.176930 5052 generic.go:334] "Generic (PLEG): container finished" podID="a83592aa-a4c1-459e-9d07-f14b390a5e6b" containerID="be69f0c47ca79cbf547e094fd8c2d7c0c191112e45d357d7916c62c184aa277b" exitCode=0 Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.176984 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-bcdbf47bb-m2ctz" event={"ID":"a83592aa-a4c1-459e-9d07-f14b390a5e6b","Type":"ContainerDied","Data":"be69f0c47ca79cbf547e094fd8c2d7c0c191112e45d357d7916c62c184aa277b"} Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.181273 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage18-crc" (OuterVolumeSpecName: "glance") pod "3039127c-0211-4430-9b2b-426412241ce4" (UID: "3039127c-0211-4430-9b2b-426412241ce4"). InnerVolumeSpecName "local-storage18-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.181444 5052 scope.go:117] "RemoveContainer" containerID="30dfc639d124cfef009d7624b44b757995f402edb14bbdfeb69e215f2a21b847" Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.183744 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.185025 5052 generic.go:334] "Generic (PLEG): container finished" podID="30045a80-9e2b-416b-b7bf-cdb9585ec67d" containerID="0d3fa6f69a8171deaa21727301ede51856b6793ff9d13fde6cbe39442a2e4566" exitCode=0 Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.185091 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"30045a80-9e2b-416b-b7bf-cdb9585ec67d","Type":"ContainerDied","Data":"0d3fa6f69a8171deaa21727301ede51856b6793ff9d13fde6cbe39442a2e4566"} Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.193878 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutrond24e-account-delete-p2mpp" event={"ID":"12f52069-8b08-4e4f-a3c1-a5421efb5844","Type":"ContainerDied","Data":"713211a2bad96f3f8994a25e2a54078414fa46ad3e231c79400c33e319e58013"} Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.193918 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="713211a2bad96f3f8994a25e2a54078414fa46ad3e231c79400c33e319e58013" Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.195807 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3039127c-0211-4430-9b2b-426412241ce4-kube-api-access-mclw8" (OuterVolumeSpecName: "kube-api-access-mclw8") pod "3039127c-0211-4430-9b2b-426412241ce4" (UID: "3039127c-0211-4430-9b2b-426412241ce4"). InnerVolumeSpecName "kube-api-access-mclw8". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.196837 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/cinder3a0d-account-delete-lxd6l" podUID="5fdc7b7d-f9ed-4d5c-a77b-6a6b0fd186f9" containerName="mariadb-account-delete" containerID="cri-o://660e428afdfd8a7e278a3dd2d6aa79748d2d7485dda4c8e8678801b9bcc2009e" gracePeriod=30 Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.218294 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3039127c-0211-4430-9b2b-426412241ce4-scripts" (OuterVolumeSpecName: "scripts") pod "3039127c-0211-4430-9b2b-426412241ce4" (UID: "3039127c-0211-4430-9b2b-426412241ce4"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:41:03 crc kubenswrapper[5052]: E1203 12:41:03.246497 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="b2f117c03e1adda1bc8b58e2d0870125a9b0fd3dffffabf7fb3480969ee908b3" cmd=["/usr/local/bin/container-scripts/status_check.sh"] Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.246580 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-cdf4456f4-v82rz" Dec 03 12:41:03 crc kubenswrapper[5052]: E1203 12:41:03.249971 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="b2f117c03e1adda1bc8b58e2d0870125a9b0fd3dffffabf7fb3480969ee908b3" cmd=["/usr/local/bin/container-scripts/status_check.sh"] Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.251856 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.254039 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/openstack-cell1-galera-0"] Dec 03 12:41:03 crc kubenswrapper[5052]: E1203 12:41:03.255704 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="b2f117c03e1adda1bc8b58e2d0870125a9b0fd3dffffabf7fb3480969ee908b3" cmd=["/usr/local/bin/container-scripts/status_check.sh"] Dec 03 12:41:03 crc kubenswrapper[5052]: E1203 12:41:03.255758 5052 prober.go:104] "Probe errored" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" probeType="Readiness" pod="openstack-kuttl-tests/ovn-northd-0" podUID="ea4cd237-7ec6-49f7-9afd-1ea159b9b23b" containerName="ovn-northd" Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.257620 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glancec94f-account-delete-8t6wx" Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.262188 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/openstack-cell1-galera-0"] Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.267946 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutrond24e-account-delete-p2mpp" Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.268577 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/5bca7a3b-316e-4357-8ae4-e47be319dbf1-public-tls-certs\") pod \"5bca7a3b-316e-4357-8ae4-e47be319dbf1\" (UID: \"5bca7a3b-316e-4357-8ae4-e47be319dbf1\") " Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.268644 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/5bca7a3b-316e-4357-8ae4-e47be319dbf1-config-data-custom\") pod \"5bca7a3b-316e-4357-8ae4-e47be319dbf1\" (UID: \"5bca7a3b-316e-4357-8ae4-e47be319dbf1\") " Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.268674 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/99a0961b-8ed8-49b7-abac-5025d7b351ee-combined-ca-bundle\") pod \"99a0961b-8ed8-49b7-abac-5025d7b351ee\" (UID: \"99a0961b-8ed8-49b7-abac-5025d7b351ee\") " Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.268708 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/5bca7a3b-316e-4357-8ae4-e47be319dbf1-internal-tls-certs\") pod \"5bca7a3b-316e-4357-8ae4-e47be319dbf1\" (UID: \"5bca7a3b-316e-4357-8ae4-e47be319dbf1\") " Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.268747 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/5bca7a3b-316e-4357-8ae4-e47be319dbf1-scripts\") pod \"5bca7a3b-316e-4357-8ae4-e47be319dbf1\" (UID: \"5bca7a3b-316e-4357-8ae4-e47be319dbf1\") " Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.268777 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/99a0961b-8ed8-49b7-abac-5025d7b351ee-internal-tls-certs\") pod \"99a0961b-8ed8-49b7-abac-5025d7b351ee\" (UID: \"99a0961b-8ed8-49b7-abac-5025d7b351ee\") " Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.268813 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jkxtd\" (UniqueName: \"kubernetes.io/projected/5bca7a3b-316e-4357-8ae4-e47be319dbf1-kube-api-access-jkxtd\") pod \"5bca7a3b-316e-4357-8ae4-e47be319dbf1\" (UID: \"5bca7a3b-316e-4357-8ae4-e47be319dbf1\") " Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.268831 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/5bca7a3b-316e-4357-8ae4-e47be319dbf1-config-data\") pod \"5bca7a3b-316e-4357-8ae4-e47be319dbf1\" (UID: \"5bca7a3b-316e-4357-8ae4-e47be319dbf1\") " Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.268847 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nf27q\" (UniqueName: \"kubernetes.io/projected/b7c49d90-c0af-4d4b-a080-8c7dce3ee66a-kube-api-access-nf27q\") pod \"b7c49d90-c0af-4d4b-a080-8c7dce3ee66a\" (UID: \"b7c49d90-c0af-4d4b-a080-8c7dce3ee66a\") " Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.268868 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/5bca7a3b-316e-4357-8ae4-e47be319dbf1-etc-machine-id\") pod \"5bca7a3b-316e-4357-8ae4-e47be319dbf1\" (UID: \"5bca7a3b-316e-4357-8ae4-e47be319dbf1\") " Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.268892 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ssfhj\" (UniqueName: \"kubernetes.io/projected/b0606156-f409-4cae-9be1-15d990f69955-kube-api-access-ssfhj\") pod \"b0606156-f409-4cae-9be1-15d990f69955\" (UID: \"b0606156-f409-4cae-9be1-15d990f69955\") " Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.268912 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/99a0961b-8ed8-49b7-abac-5025d7b351ee-config-data\") pod \"99a0961b-8ed8-49b7-abac-5025d7b351ee\" (UID: \"99a0961b-8ed8-49b7-abac-5025d7b351ee\") " Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.268931 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/99a0961b-8ed8-49b7-abac-5025d7b351ee-public-tls-certs\") pod \"99a0961b-8ed8-49b7-abac-5025d7b351ee\" (UID: \"99a0961b-8ed8-49b7-abac-5025d7b351ee\") " Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.268953 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/5bca7a3b-316e-4357-8ae4-e47be319dbf1-logs\") pod \"5bca7a3b-316e-4357-8ae4-e47be319dbf1\" (UID: \"5bca7a3b-316e-4357-8ae4-e47be319dbf1\") " Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.268979 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/b7c49d90-c0af-4d4b-a080-8c7dce3ee66a-operator-scripts\") pod \"b7c49d90-c0af-4d4b-a080-8c7dce3ee66a\" (UID: \"b7c49d90-c0af-4d4b-a080-8c7dce3ee66a\") " Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.269016 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ljq7n\" (UniqueName: \"kubernetes.io/projected/99a0961b-8ed8-49b7-abac-5025d7b351ee-kube-api-access-ljq7n\") pod \"99a0961b-8ed8-49b7-abac-5025d7b351ee\" (UID: \"99a0961b-8ed8-49b7-abac-5025d7b351ee\") " Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.269036 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/99a0961b-8ed8-49b7-abac-5025d7b351ee-logs\") pod \"99a0961b-8ed8-49b7-abac-5025d7b351ee\" (UID: \"99a0961b-8ed8-49b7-abac-5025d7b351ee\") " Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.269052 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b0606156-f409-4cae-9be1-15d990f69955-combined-ca-bundle\") pod \"b0606156-f409-4cae-9be1-15d990f69955\" (UID: \"b0606156-f409-4cae-9be1-15d990f69955\") " Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.269079 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/b0606156-f409-4cae-9be1-15d990f69955-httpd-run\") pod \"b0606156-f409-4cae-9be1-15d990f69955\" (UID: \"b0606156-f409-4cae-9be1-15d990f69955\") " Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.269095 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5bca7a3b-316e-4357-8ae4-e47be319dbf1-combined-ca-bundle\") pod \"5bca7a3b-316e-4357-8ae4-e47be319dbf1\" (UID: \"5bca7a3b-316e-4357-8ae4-e47be319dbf1\") " Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.269123 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/b0606156-f409-4cae-9be1-15d990f69955-scripts\") pod \"b0606156-f409-4cae-9be1-15d990f69955\" (UID: \"b0606156-f409-4cae-9be1-15d990f69955\") " Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.269140 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/b0606156-f409-4cae-9be1-15d990f69955-config-data\") pod \"b0606156-f409-4cae-9be1-15d990f69955\" (UID: \"b0606156-f409-4cae-9be1-15d990f69955\") " Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.269157 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"glance\" (UniqueName: \"kubernetes.io/local-volume/local-storage12-crc\") pod \"b0606156-f409-4cae-9be1-15d990f69955\" (UID: \"b0606156-f409-4cae-9be1-15d990f69955\") " Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.269179 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/99a0961b-8ed8-49b7-abac-5025d7b351ee-scripts\") pod \"99a0961b-8ed8-49b7-abac-5025d7b351ee\" (UID: \"99a0961b-8ed8-49b7-abac-5025d7b351ee\") " Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.269210 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/b0606156-f409-4cae-9be1-15d990f69955-logs\") pod \"b0606156-f409-4cae-9be1-15d990f69955\" (UID: \"b0606156-f409-4cae-9be1-15d990f69955\") " Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.269259 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/b0606156-f409-4cae-9be1-15d990f69955-public-tls-certs\") pod \"b0606156-f409-4cae-9be1-15d990f69955\" (UID: \"b0606156-f409-4cae-9be1-15d990f69955\") " Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.269633 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-mclw8\" (UniqueName: \"kubernetes.io/projected/3039127c-0211-4430-9b2b-426412241ce4-kube-api-access-mclw8\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.269647 5052 reconciler_common.go:293] "Volume detached for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/3039127c-0211-4430-9b2b-426412241ce4-httpd-run\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.269656 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/3039127c-0211-4430-9b2b-426412241ce4-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.269676 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage18-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage18-crc\") on node \"crc\" " Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.269685 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/3039127c-0211-4430-9b2b-426412241ce4-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.270821 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/swift-proxy-bf56d5474-cq7b2"] Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.277053 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b0606156-f409-4cae-9be1-15d990f69955-logs" (OuterVolumeSpecName: "logs") pod "b0606156-f409-4cae-9be1-15d990f69955" (UID: "b0606156-f409-4cae-9be1-15d990f69955"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.282186 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b0606156-f409-4cae-9be1-15d990f69955-httpd-run" (OuterVolumeSpecName: "httpd-run") pod "b0606156-f409-4cae-9be1-15d990f69955" (UID: "b0606156-f409-4cae-9be1-15d990f69955"). InnerVolumeSpecName "httpd-run". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.282262 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/swift-proxy-bf56d5474-cq7b2"] Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.282379 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/5bca7a3b-316e-4357-8ae4-e47be319dbf1-etc-machine-id" (OuterVolumeSpecName: "etc-machine-id") pod "5bca7a3b-316e-4357-8ae4-e47be319dbf1" (UID: "5bca7a3b-316e-4357-8ae4-e47be319dbf1"). InnerVolumeSpecName "etc-machine-id". PluginName "kubernetes.io/host-path", VolumeGidValue "" Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.282558 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5bca7a3b-316e-4357-8ae4-e47be319dbf1-logs" (OuterVolumeSpecName: "logs") pod "5bca7a3b-316e-4357-8ae4-e47be319dbf1" (UID: "5bca7a3b-316e-4357-8ae4-e47be319dbf1"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.283464 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b7c49d90-c0af-4d4b-a080-8c7dce3ee66a-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "b7c49d90-c0af-4d4b-a080-8c7dce3ee66a" (UID: "b7c49d90-c0af-4d4b-a080-8c7dce3ee66a"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.283794 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/99a0961b-8ed8-49b7-abac-5025d7b351ee-logs" (OuterVolumeSpecName: "logs") pod "99a0961b-8ed8-49b7-abac-5025d7b351ee" (UID: "99a0961b-8ed8-49b7-abac-5025d7b351ee"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.286548 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-api-bcdbf47bb-m2ctz" Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.287430 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage12-crc" (OuterVolumeSpecName: "glance") pod "b0606156-f409-4cae-9be1-15d990f69955" (UID: "b0606156-f409-4cae-9be1-15d990f69955"). InnerVolumeSpecName "local-storage12-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.296285 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b0606156-f409-4cae-9be1-15d990f69955-scripts" (OuterVolumeSpecName: "scripts") pod "b0606156-f409-4cae-9be1-15d990f69955" (UID: "b0606156-f409-4cae-9be1-15d990f69955"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.296605 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5bca7a3b-316e-4357-8ae4-e47be319dbf1-kube-api-access-jkxtd" (OuterVolumeSpecName: "kube-api-access-jkxtd") pod "5bca7a3b-316e-4357-8ae4-e47be319dbf1" (UID: "5bca7a3b-316e-4357-8ae4-e47be319dbf1"). InnerVolumeSpecName "kube-api-access-jkxtd". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.296758 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.300333 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.301075 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b0606156-f409-4cae-9be1-15d990f69955-kube-api-access-ssfhj" (OuterVolumeSpecName: "kube-api-access-ssfhj") pod "b0606156-f409-4cae-9be1-15d990f69955" (UID: "b0606156-f409-4cae-9be1-15d990f69955"). InnerVolumeSpecName "kube-api-access-ssfhj". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.303666 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/99a0961b-8ed8-49b7-abac-5025d7b351ee-kube-api-access-ljq7n" (OuterVolumeSpecName: "kube-api-access-ljq7n") pod "99a0961b-8ed8-49b7-abac-5025d7b351ee" (UID: "99a0961b-8ed8-49b7-abac-5025d7b351ee"). InnerVolumeSpecName "kube-api-access-ljq7n". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.304607 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.322274 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.328885 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/99a0961b-8ed8-49b7-abac-5025d7b351ee-scripts" (OuterVolumeSpecName: "scripts") pod "99a0961b-8ed8-49b7-abac-5025d7b351ee" (UID: "99a0961b-8ed8-49b7-abac-5025d7b351ee"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.338283 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b7c49d90-c0af-4d4b-a080-8c7dce3ee66a-kube-api-access-nf27q" (OuterVolumeSpecName: "kube-api-access-nf27q") pod "b7c49d90-c0af-4d4b-a080-8c7dce3ee66a" (UID: "b7c49d90-c0af-4d4b-a080-8c7dce3ee66a"). InnerVolumeSpecName "kube-api-access-nf27q". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.349248 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5bca7a3b-316e-4357-8ae4-e47be319dbf1-config-data-custom" (OuterVolumeSpecName: "config-data-custom") pod "5bca7a3b-316e-4357-8ae4-e47be319dbf1" (UID: "5bca7a3b-316e-4357-8ae4-e47be319dbf1"). InnerVolumeSpecName "config-data-custom". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.352613 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5bca7a3b-316e-4357-8ae4-e47be319dbf1-scripts" (OuterVolumeSpecName: "scripts") pod "5bca7a3b-316e-4357-8ae4-e47be319dbf1" (UID: "5bca7a3b-316e-4357-8ae4-e47be319dbf1"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.372250 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/a83592aa-a4c1-459e-9d07-f14b390a5e6b-config-data-custom\") pod \"a83592aa-a4c1-459e-9d07-f14b390a5e6b\" (UID: \"a83592aa-a4c1-459e-9d07-f14b390a5e6b\") " Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.372319 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/30045a80-9e2b-416b-b7bf-cdb9585ec67d-nova-metadata-tls-certs\") pod \"30045a80-9e2b-416b-b7bf-cdb9585ec67d\" (UID: \"30045a80-9e2b-416b-b7bf-cdb9585ec67d\") " Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.372366 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/50789f13-8194-4e24-b975-512591b816a5-combined-ca-bundle\") pod \"50789f13-8194-4e24-b975-512591b816a5\" (UID: \"50789f13-8194-4e24-b975-512591b816a5\") " Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.372411 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/30045a80-9e2b-416b-b7bf-cdb9585ec67d-combined-ca-bundle\") pod \"30045a80-9e2b-416b-b7bf-cdb9585ec67d\" (UID: \"30045a80-9e2b-416b-b7bf-cdb9585ec67d\") " Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.372447 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/30045a80-9e2b-416b-b7bf-cdb9585ec67d-config-data\") pod \"30045a80-9e2b-416b-b7bf-cdb9585ec67d\" (UID: \"30045a80-9e2b-416b-b7bf-cdb9585ec67d\") " Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.372486 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/50789f13-8194-4e24-b975-512591b816a5-internal-tls-certs\") pod \"50789f13-8194-4e24-b975-512591b816a5\" (UID: \"50789f13-8194-4e24-b975-512591b816a5\") " Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.372545 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/a83592aa-a4c1-459e-9d07-f14b390a5e6b-internal-tls-certs\") pod \"a83592aa-a4c1-459e-9d07-f14b390a5e6b\" (UID: \"a83592aa-a4c1-459e-9d07-f14b390a5e6b\") " Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.372603 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/a83592aa-a4c1-459e-9d07-f14b390a5e6b-public-tls-certs\") pod \"a83592aa-a4c1-459e-9d07-f14b390a5e6b\" (UID: \"a83592aa-a4c1-459e-9d07-f14b390a5e6b\") " Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.372632 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/50789f13-8194-4e24-b975-512591b816a5-public-tls-certs\") pod \"50789f13-8194-4e24-b975-512591b816a5\" (UID: \"50789f13-8194-4e24-b975-512591b816a5\") " Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.372680 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/a83592aa-a4c1-459e-9d07-f14b390a5e6b-logs\") pod \"a83592aa-a4c1-459e-9d07-f14b390a5e6b\" (UID: \"a83592aa-a4c1-459e-9d07-f14b390a5e6b\") " Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.372739 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/30045a80-9e2b-416b-b7bf-cdb9585ec67d-logs\") pod \"30045a80-9e2b-416b-b7bf-cdb9585ec67d\" (UID: \"30045a80-9e2b-416b-b7bf-cdb9585ec67d\") " Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.372816 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tbm29\" (UniqueName: \"kubernetes.io/projected/12f52069-8b08-4e4f-a3c1-a5421efb5844-kube-api-access-tbm29\") pod \"12f52069-8b08-4e4f-a3c1-a5421efb5844\" (UID: \"12f52069-8b08-4e4f-a3c1-a5421efb5844\") " Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.372842 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a83592aa-a4c1-459e-9d07-f14b390a5e6b-combined-ca-bundle\") pod \"a83592aa-a4c1-459e-9d07-f14b390a5e6b\" (UID: \"a83592aa-a4c1-459e-9d07-f14b390a5e6b\") " Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.372868 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hpsn6\" (UniqueName: \"kubernetes.io/projected/30045a80-9e2b-416b-b7bf-cdb9585ec67d-kube-api-access-hpsn6\") pod \"30045a80-9e2b-416b-b7bf-cdb9585ec67d\" (UID: \"30045a80-9e2b-416b-b7bf-cdb9585ec67d\") " Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.372918 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v8rm4\" (UniqueName: \"kubernetes.io/projected/50789f13-8194-4e24-b975-512591b816a5-kube-api-access-v8rm4\") pod \"50789f13-8194-4e24-b975-512591b816a5\" (UID: \"50789f13-8194-4e24-b975-512591b816a5\") " Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.372979 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tk6xf\" (UniqueName: \"kubernetes.io/projected/a83592aa-a4c1-459e-9d07-f14b390a5e6b-kube-api-access-tk6xf\") pod \"a83592aa-a4c1-459e-9d07-f14b390a5e6b\" (UID: \"a83592aa-a4c1-459e-9d07-f14b390a5e6b\") " Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.373031 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/50789f13-8194-4e24-b975-512591b816a5-config-data\") pod \"50789f13-8194-4e24-b975-512591b816a5\" (UID: \"50789f13-8194-4e24-b975-512591b816a5\") " Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.373056 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/12f52069-8b08-4e4f-a3c1-a5421efb5844-operator-scripts\") pod \"12f52069-8b08-4e4f-a3c1-a5421efb5844\" (UID: \"12f52069-8b08-4e4f-a3c1-a5421efb5844\") " Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.373083 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/50789f13-8194-4e24-b975-512591b816a5-logs\") pod \"50789f13-8194-4e24-b975-512591b816a5\" (UID: \"50789f13-8194-4e24-b975-512591b816a5\") " Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.373665 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-ssfhj\" (UniqueName: \"kubernetes.io/projected/b0606156-f409-4cae-9be1-15d990f69955-kube-api-access-ssfhj\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.373689 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/5bca7a3b-316e-4357-8ae4-e47be319dbf1-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.373704 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/b7c49d90-c0af-4d4b-a080-8c7dce3ee66a-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.373716 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-ljq7n\" (UniqueName: \"kubernetes.io/projected/99a0961b-8ed8-49b7-abac-5025d7b351ee-kube-api-access-ljq7n\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.373728 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/99a0961b-8ed8-49b7-abac-5025d7b351ee-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.373740 5052 reconciler_common.go:293] "Volume detached for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/b0606156-f409-4cae-9be1-15d990f69955-httpd-run\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.373751 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/b0606156-f409-4cae-9be1-15d990f69955-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.373775 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage12-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage12-crc\") on node \"crc\" " Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.373788 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/99a0961b-8ed8-49b7-abac-5025d7b351ee-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.373800 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/b0606156-f409-4cae-9be1-15d990f69955-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.373811 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/5bca7a3b-316e-4357-8ae4-e47be319dbf1-config-data-custom\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.373822 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/5bca7a3b-316e-4357-8ae4-e47be319dbf1-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.373835 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-jkxtd\" (UniqueName: \"kubernetes.io/projected/5bca7a3b-316e-4357-8ae4-e47be319dbf1-kube-api-access-jkxtd\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.373847 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-nf27q\" (UniqueName: \"kubernetes.io/projected/b7c49d90-c0af-4d4b-a080-8c7dce3ee66a-kube-api-access-nf27q\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.373859 5052 reconciler_common.go:293] "Volume detached for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/5bca7a3b-316e-4357-8ae4-e47be319dbf1-etc-machine-id\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.375007 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a83592aa-a4c1-459e-9d07-f14b390a5e6b-logs" (OuterVolumeSpecName: "logs") pod "a83592aa-a4c1-459e-9d07-f14b390a5e6b" (UID: "a83592aa-a4c1-459e-9d07-f14b390a5e6b"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.375134 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/30045a80-9e2b-416b-b7bf-cdb9585ec67d-logs" (OuterVolumeSpecName: "logs") pod "30045a80-9e2b-416b-b7bf-cdb9585ec67d" (UID: "30045a80-9e2b-416b-b7bf-cdb9585ec67d"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.377038 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/12f52069-8b08-4e4f-a3c1-a5421efb5844-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "12f52069-8b08-4e4f-a3c1-a5421efb5844" (UID: "12f52069-8b08-4e4f-a3c1-a5421efb5844"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.377961 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/50789f13-8194-4e24-b975-512591b816a5-logs" (OuterVolumeSpecName: "logs") pod "50789f13-8194-4e24-b975-512591b816a5" (UID: "50789f13-8194-4e24-b975-512591b816a5"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.380186 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-keystone-listener-74b68b767b-8zpnm" Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.388530 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/50789f13-8194-4e24-b975-512591b816a5-kube-api-access-v8rm4" (OuterVolumeSpecName: "kube-api-access-v8rm4") pod "50789f13-8194-4e24-b975-512591b816a5" (UID: "50789f13-8194-4e24-b975-512591b816a5"). InnerVolumeSpecName "kube-api-access-v8rm4". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.395291 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/30045a80-9e2b-416b-b7bf-cdb9585ec67d-kube-api-access-hpsn6" (OuterVolumeSpecName: "kube-api-access-hpsn6") pod "30045a80-9e2b-416b-b7bf-cdb9585ec67d" (UID: "30045a80-9e2b-416b-b7bf-cdb9585ec67d"). InnerVolumeSpecName "kube-api-access-hpsn6". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.411022 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a83592aa-a4c1-459e-9d07-f14b390a5e6b-config-data-custom" (OuterVolumeSpecName: "config-data-custom") pod "a83592aa-a4c1-459e-9d07-f14b390a5e6b" (UID: "a83592aa-a4c1-459e-9d07-f14b390a5e6b"). InnerVolumeSpecName "config-data-custom". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.431116 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/12f52069-8b08-4e4f-a3c1-a5421efb5844-kube-api-access-tbm29" (OuterVolumeSpecName: "kube-api-access-tbm29") pod "12f52069-8b08-4e4f-a3c1-a5421efb5844" (UID: "12f52069-8b08-4e4f-a3c1-a5421efb5844"). InnerVolumeSpecName "kube-api-access-tbm29". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.431260 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a83592aa-a4c1-459e-9d07-f14b390a5e6b-kube-api-access-tk6xf" (OuterVolumeSpecName: "kube-api-access-tk6xf") pod "a83592aa-a4c1-459e-9d07-f14b390a5e6b" (UID: "a83592aa-a4c1-459e-9d07-f14b390a5e6b"). InnerVolumeSpecName "kube-api-access-tk6xf". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.474788 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a83592aa-a4c1-459e-9d07-f14b390a5e6b-config-data\") pod \"a83592aa-a4c1-459e-9d07-f14b390a5e6b\" (UID: \"a83592aa-a4c1-459e-9d07-f14b390a5e6b\") " Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.474862 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lkrkl\" (UniqueName: \"kubernetes.io/projected/bc0b4316-9c87-4c72-bc06-b4b526c2070e-kube-api-access-lkrkl\") pod \"bc0b4316-9c87-4c72-bc06-b4b526c2070e\" (UID: \"bc0b4316-9c87-4c72-bc06-b4b526c2070e\") " Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.475003 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/bc0b4316-9c87-4c72-bc06-b4b526c2070e-config-data\") pod \"bc0b4316-9c87-4c72-bc06-b4b526c2070e\" (UID: \"bc0b4316-9c87-4c72-bc06-b4b526c2070e\") " Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.475055 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/bc0b4316-9c87-4c72-bc06-b4b526c2070e-config-data-custom\") pod \"bc0b4316-9c87-4c72-bc06-b4b526c2070e\" (UID: \"bc0b4316-9c87-4c72-bc06-b4b526c2070e\") " Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.475110 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/bc0b4316-9c87-4c72-bc06-b4b526c2070e-combined-ca-bundle\") pod \"bc0b4316-9c87-4c72-bc06-b4b526c2070e\" (UID: \"bc0b4316-9c87-4c72-bc06-b4b526c2070e\") " Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.475139 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/bc0b4316-9c87-4c72-bc06-b4b526c2070e-logs\") pod \"bc0b4316-9c87-4c72-bc06-b4b526c2070e\" (UID: \"bc0b4316-9c87-4c72-bc06-b4b526c2070e\") " Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.475836 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bc0b4316-9c87-4c72-bc06-b4b526c2070e-logs" (OuterVolumeSpecName: "logs") pod "bc0b4316-9c87-4c72-bc06-b4b526c2070e" (UID: "bc0b4316-9c87-4c72-bc06-b4b526c2070e"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.476534 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/a83592aa-a4c1-459e-9d07-f14b390a5e6b-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.476559 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/30045a80-9e2b-416b-b7bf-cdb9585ec67d-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.476574 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-tbm29\" (UniqueName: \"kubernetes.io/projected/12f52069-8b08-4e4f-a3c1-a5421efb5844-kube-api-access-tbm29\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.476589 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-hpsn6\" (UniqueName: \"kubernetes.io/projected/30045a80-9e2b-416b-b7bf-cdb9585ec67d-kube-api-access-hpsn6\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.476601 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-v8rm4\" (UniqueName: \"kubernetes.io/projected/50789f13-8194-4e24-b975-512591b816a5-kube-api-access-v8rm4\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.476612 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-tk6xf\" (UniqueName: \"kubernetes.io/projected/a83592aa-a4c1-459e-9d07-f14b390a5e6b-kube-api-access-tk6xf\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.476624 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/12f52069-8b08-4e4f-a3c1-a5421efb5844-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.476635 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/50789f13-8194-4e24-b975-512591b816a5-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.476646 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/a83592aa-a4c1-459e-9d07-f14b390a5e6b-config-data-custom\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.476658 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/bc0b4316-9c87-4c72-bc06-b4b526c2070e-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.484051 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bc0b4316-9c87-4c72-bc06-b4b526c2070e-kube-api-access-lkrkl" (OuterVolumeSpecName: "kube-api-access-lkrkl") pod "bc0b4316-9c87-4c72-bc06-b4b526c2070e" (UID: "bc0b4316-9c87-4c72-bc06-b4b526c2070e"). InnerVolumeSpecName "kube-api-access-lkrkl". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.484245 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bc0b4316-9c87-4c72-bc06-b4b526c2070e-config-data-custom" (OuterVolumeSpecName: "config-data-custom") pod "bc0b4316-9c87-4c72-bc06-b4b526c2070e" (UID: "bc0b4316-9c87-4c72-bc06-b4b526c2070e"). InnerVolumeSpecName "config-data-custom". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.586030 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-lkrkl\" (UniqueName: \"kubernetes.io/projected/bc0b4316-9c87-4c72-bc06-b4b526c2070e-kube-api-access-lkrkl\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.587020 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/bc0b4316-9c87-4c72-bc06-b4b526c2070e-config-data-custom\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.618371 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/50789f13-8194-4e24-b975-512591b816a5-config-data" (OuterVolumeSpecName: "config-data") pod "50789f13-8194-4e24-b975-512591b816a5" (UID: "50789f13-8194-4e24-b975-512591b816a5"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.626121 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/50789f13-8194-4e24-b975-512591b816a5-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "50789f13-8194-4e24-b975-512591b816a5" (UID: "50789f13-8194-4e24-b975-512591b816a5"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.677982 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3039127c-0211-4430-9b2b-426412241ce4-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "3039127c-0211-4430-9b2b-426412241ce4" (UID: "3039127c-0211-4430-9b2b-426412241ce4"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.705394 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/50789f13-8194-4e24-b975-512591b816a5-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.705445 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/50789f13-8194-4e24-b975-512591b816a5-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.705463 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3039127c-0211-4430-9b2b-426412241ce4-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:03 crc kubenswrapper[5052]: E1203 12:41:03.705577 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:41:03 crc kubenswrapper[5052]: E1203 12:41:03.705649 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/eb8b91d5-6c99-45ba-8d33-da99f1dd496c-operator-scripts podName:eb8b91d5-6c99-45ba-8d33-da99f1dd496c nodeName:}" failed. No retries permitted until 2025-12-03 12:41:04.705621795 +0000 UTC m=+4855.293863559 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/eb8b91d5-6c99-45ba-8d33-da99f1dd496c-operator-scripts") pod "placement7f30-account-delete-vld4v" (UID: "eb8b91d5-6c99-45ba-8d33-da99f1dd496c") : configmap "openstack-scripts" not found Dec 03 12:41:03 crc kubenswrapper[5052]: E1203 12:41:03.705954 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:41:03 crc kubenswrapper[5052]: E1203 12:41:03.706115 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5fdc7b7d-f9ed-4d5c-a77b-6a6b0fd186f9-operator-scripts podName:5fdc7b7d-f9ed-4d5c-a77b-6a6b0fd186f9 nodeName:}" failed. No retries permitted until 2025-12-03 12:41:05.706088639 +0000 UTC m=+4856.294330403 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/5fdc7b7d-f9ed-4d5c-a77b-6a6b0fd186f9-operator-scripts") pod "cinder3a0d-account-delete-lxd6l" (UID: "5fdc7b7d-f9ed-4d5c-a77b-6a6b0fd186f9") : configmap "openstack-scripts" not found Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.724771 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0f16f521-a181-40e4-a1ed-38f4c8862ffb" path="/var/lib/kubelet/pods/0f16f521-a181-40e4-a1ed-38f4c8862ffb/volumes" Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.729754 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="13e6f571-4161-442d-b45e-f30f6ea05843" path="/var/lib/kubelet/pods/13e6f571-4161-442d-b45e-f30f6ea05843/volumes" Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.739086 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2a13e864-04af-42a7-bfbb-9995231b567d" path="/var/lib/kubelet/pods/2a13e864-04af-42a7-bfbb-9995231b567d/volumes" Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.739893 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2a2e1ec2-a657-4a31-93fd-712d45fab72b" path="/var/lib/kubelet/pods/2a2e1ec2-a657-4a31-93fd-712d45fab72b/volumes" Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.741259 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2f122603-9186-43bc-b3c7-b21307f24c5a" path="/var/lib/kubelet/pods/2f122603-9186-43bc-b3c7-b21307f24c5a/volumes" Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.742124 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="41a48918-de1d-4463-94ed-3d19cc52ead8" path="/var/lib/kubelet/pods/41a48918-de1d-4463-94ed-3d19cc52ead8/volumes" Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.743016 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="44c9835e-4fd7-4713-8e42-2d3f0a82fbfe" path="/var/lib/kubelet/pods/44c9835e-4fd7-4713-8e42-2d3f0a82fbfe/volumes" Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.748015 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="46c590ef-2b7c-412d-8582-dfdc311e6778" path="/var/lib/kubelet/pods/46c590ef-2b7c-412d-8582-dfdc311e6778/volumes" Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.749311 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="621935c3-f8f9-43a1-9bc3-6d6ad9caf0ef" path="/var/lib/kubelet/pods/621935c3-f8f9-43a1-9bc3-6d6ad9caf0ef/volumes" Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.756910 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="65fef887-25c4-46ee-8a9b-cc82f32dcbe3" path="/var/lib/kubelet/pods/65fef887-25c4-46ee-8a9b-cc82f32dcbe3/volumes" Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.764308 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6daea253-ee08-4aa8-a900-39552cae7bd5" path="/var/lib/kubelet/pods/6daea253-ee08-4aa8-a900-39552cae7bd5/volumes" Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.765883 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8179fa49-7bb4-4c12-b863-a080cbabb765" path="/var/lib/kubelet/pods/8179fa49-7bb4-4c12-b863-a080cbabb765/volumes" Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.767917 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a6e548d7-7349-413c-a4f7-914f4df80c4b" path="/var/lib/kubelet/pods/a6e548d7-7349-413c-a4f7-914f4df80c4b/volumes" Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.771634 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cebde9f9-4c41-4628-b014-6b7771299e50" path="/var/lib/kubelet/pods/cebde9f9-4c41-4628-b014-6b7771299e50/volumes" Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.774268 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d855c655-3b2f-4e4f-9b27-cf4e43a92045" path="/var/lib/kubelet/pods/d855c655-3b2f-4e4f-9b27-cf4e43a92045/volumes" Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.775431 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dc792e82-3490-43dc-a435-8847a5d3940e" path="/var/lib/kubelet/pods/dc792e82-3490-43dc-a435-8847a5d3940e/volumes" Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.785837 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a83592aa-a4c1-459e-9d07-f14b390a5e6b-config-data" (OuterVolumeSpecName: "config-data") pod "a83592aa-a4c1-459e-9d07-f14b390a5e6b" (UID: "a83592aa-a4c1-459e-9d07-f14b390a5e6b"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.790652 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage12-crc" (UniqueName: "kubernetes.io/local-volume/local-storage12-crc") on node "crc" Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.790795 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dde85075-240d-43d9-a002-cc0206cfebf8" path="/var/lib/kubelet/pods/dde85075-240d-43d9-a002-cc0206cfebf8/volumes" Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.792846 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f5e0e5f2-c417-4ebc-b903-25e3b3803964" path="/var/lib/kubelet/pods/f5e0e5f2-c417-4ebc-b903-25e3b3803964/volumes" Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.793834 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f89ce0f5-5a92-49eb-9951-f2b491ac2139" path="/var/lib/kubelet/pods/f89ce0f5-5a92-49eb-9951-f2b491ac2139/volumes" Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.799920 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b0606156-f409-4cae-9be1-15d990f69955-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "b0606156-f409-4cae-9be1-15d990f69955" (UID: "b0606156-f409-4cae-9be1-15d990f69955"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.802304 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fc2e5ba2-92e9-4fbe-80f6-d977d0e1d11b" path="/var/lib/kubelet/pods/fc2e5ba2-92e9-4fbe-80f6-d977d0e1d11b/volumes" Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.803727 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ffa2902f-c174-44ec-98dc-de610f30b64f" path="/var/lib/kubelet/pods/ffa2902f-c174-44ec-98dc-de610f30b64f/volumes" Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.824600 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a83592aa-a4c1-459e-9d07-f14b390a5e6b-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.824628 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage12-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage12-crc\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.824639 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/b0606156-f409-4cae-9be1-15d990f69955-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.845336 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage18-crc" (UniqueName: "kubernetes.io/local-volume/local-storage18-crc") on node "crc" Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.855689 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/99a0961b-8ed8-49b7-abac-5025d7b351ee-config-data" (OuterVolumeSpecName: "config-data") pod "99a0961b-8ed8-49b7-abac-5025d7b351ee" (UID: "99a0961b-8ed8-49b7-abac-5025d7b351ee"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.871700 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/99a0961b-8ed8-49b7-abac-5025d7b351ee-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "99a0961b-8ed8-49b7-abac-5025d7b351ee" (UID: "99a0961b-8ed8-49b7-abac-5025d7b351ee"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.914761 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/30045a80-9e2b-416b-b7bf-cdb9585ec67d-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "30045a80-9e2b-416b-b7bf-cdb9585ec67d" (UID: "30045a80-9e2b-416b-b7bf-cdb9585ec67d"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.938179 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/99a0961b-8ed8-49b7-abac-5025d7b351ee-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.939292 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage18-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage18-crc\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.939379 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/99a0961b-8ed8-49b7-abac-5025d7b351ee-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.939461 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/30045a80-9e2b-416b-b7bf-cdb9585ec67d-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.949072 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/50789f13-8194-4e24-b975-512591b816a5-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "50789f13-8194-4e24-b975-512591b816a5" (UID: "50789f13-8194-4e24-b975-512591b816a5"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.949555 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/30045a80-9e2b-416b-b7bf-cdb9585ec67d-config-data" (OuterVolumeSpecName: "config-data") pod "30045a80-9e2b-416b-b7bf-cdb9585ec67d" (UID: "30045a80-9e2b-416b-b7bf-cdb9585ec67d"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:41:03 crc kubenswrapper[5052]: I1203 12:41:03.996894 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3039127c-0211-4430-9b2b-426412241ce4-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "3039127c-0211-4430-9b2b-426412241ce4" (UID: "3039127c-0211-4430-9b2b-426412241ce4"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.022339 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bc0b4316-9c87-4c72-bc06-b4b526c2070e-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "bc0b4316-9c87-4c72-bc06-b4b526c2070e" (UID: "bc0b4316-9c87-4c72-bc06-b4b526c2070e"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.022716 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/30045a80-9e2b-416b-b7bf-cdb9585ec67d-nova-metadata-tls-certs" (OuterVolumeSpecName: "nova-metadata-tls-certs") pod "30045a80-9e2b-416b-b7bf-cdb9585ec67d" (UID: "30045a80-9e2b-416b-b7bf-cdb9585ec67d"). InnerVolumeSpecName "nova-metadata-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.036958 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.037114 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5bca7a3b-316e-4357-8ae4-e47be319dbf1-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "5bca7a3b-316e-4357-8ae4-e47be319dbf1" (UID: "5bca7a3b-316e-4357-8ae4-e47be319dbf1"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.043233 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gphkk\" (UniqueName: \"kubernetes.io/projected/e9af392d-1940-4203-961e-39da1ac5f3da-kube-api-access-gphkk\") pod \"e9af392d-1940-4203-961e-39da1ac5f3da\" (UID: \"e9af392d-1940-4203-961e-39da1ac5f3da\") " Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.043283 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/e9af392d-1940-4203-961e-39da1ac5f3da-config-data\") pod \"e9af392d-1940-4203-961e-39da1ac5f3da\" (UID: \"e9af392d-1940-4203-961e-39da1ac5f3da\") " Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.043653 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e9af392d-1940-4203-961e-39da1ac5f3da-combined-ca-bundle\") pod \"e9af392d-1940-4203-961e-39da1ac5f3da\" (UID: \"e9af392d-1940-4203-961e-39da1ac5f3da\") " Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.044125 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/3039127c-0211-4430-9b2b-426412241ce4-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.044152 5052 reconciler_common.go:293] "Volume detached for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/30045a80-9e2b-416b-b7bf-cdb9585ec67d-nova-metadata-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.044166 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/30045a80-9e2b-416b-b7bf-cdb9585ec67d-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.044179 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/50789f13-8194-4e24-b975-512591b816a5-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.044193 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/bc0b4316-9c87-4c72-bc06-b4b526c2070e-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.044206 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/5bca7a3b-316e-4357-8ae4-e47be319dbf1-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.075054 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-worker-7bd785845-99jqj" Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.107613 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e9af392d-1940-4203-961e-39da1ac5f3da-kube-api-access-gphkk" (OuterVolumeSpecName: "kube-api-access-gphkk") pod "e9af392d-1940-4203-961e-39da1ac5f3da" (UID: "e9af392d-1940-4203-961e-39da1ac5f3da"). InnerVolumeSpecName "kube-api-access-gphkk". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.128360 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/memcached-0" Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.136368 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b0606156-f409-4cae-9be1-15d990f69955-config-data" (OuterVolumeSpecName: "config-data") pod "b0606156-f409-4cae-9be1-15d990f69955" (UID: "b0606156-f409-4cae-9be1-15d990f69955"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.151629 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hw2xp\" (UniqueName: \"kubernetes.io/projected/18a62c9b-4032-4639-be99-3520b515313b-kube-api-access-hw2xp\") pod \"18a62c9b-4032-4639-be99-3520b515313b\" (UID: \"18a62c9b-4032-4639-be99-3520b515313b\") " Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.151749 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/18a62c9b-4032-4639-be99-3520b515313b-logs\") pod \"18a62c9b-4032-4639-be99-3520b515313b\" (UID: \"18a62c9b-4032-4639-be99-3520b515313b\") " Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.152611 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5d19811b-4e82-42b2-b6ec-f16e707484f2-combined-ca-bundle\") pod \"5d19811b-4e82-42b2-b6ec-f16e707484f2\" (UID: \"5d19811b-4e82-42b2-b6ec-f16e707484f2\") " Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.152703 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/18a62c9b-4032-4639-be99-3520b515313b-config-data\") pod \"18a62c9b-4032-4639-be99-3520b515313b\" (UID: \"18a62c9b-4032-4639-be99-3520b515313b\") " Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.152777 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/5d19811b-4e82-42b2-b6ec-f16e707484f2-config-data\") pod \"5d19811b-4e82-42b2-b6ec-f16e707484f2\" (UID: \"5d19811b-4e82-42b2-b6ec-f16e707484f2\") " Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.152809 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"memcached-tls-certs\" (UniqueName: \"kubernetes.io/secret/5d19811b-4e82-42b2-b6ec-f16e707484f2-memcached-tls-certs\") pod \"5d19811b-4e82-42b2-b6ec-f16e707484f2\" (UID: \"5d19811b-4e82-42b2-b6ec-f16e707484f2\") " Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.152847 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/18a62c9b-4032-4639-be99-3520b515313b-config-data-custom\") pod \"18a62c9b-4032-4639-be99-3520b515313b\" (UID: \"18a62c9b-4032-4639-be99-3520b515313b\") " Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.152870 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/5d19811b-4e82-42b2-b6ec-f16e707484f2-kolla-config\") pod \"5d19811b-4e82-42b2-b6ec-f16e707484f2\" (UID: \"5d19811b-4e82-42b2-b6ec-f16e707484f2\") " Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.152983 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/18a62c9b-4032-4639-be99-3520b515313b-combined-ca-bundle\") pod \"18a62c9b-4032-4639-be99-3520b515313b\" (UID: \"18a62c9b-4032-4639-be99-3520b515313b\") " Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.153021 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/18a62c9b-4032-4639-be99-3520b515313b-logs" (OuterVolumeSpecName: "logs") pod "18a62c9b-4032-4639-be99-3520b515313b" (UID: "18a62c9b-4032-4639-be99-3520b515313b"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.153048 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ggq2r\" (UniqueName: \"kubernetes.io/projected/5d19811b-4e82-42b2-b6ec-f16e707484f2-kube-api-access-ggq2r\") pod \"5d19811b-4e82-42b2-b6ec-f16e707484f2\" (UID: \"5d19811b-4e82-42b2-b6ec-f16e707484f2\") " Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.154460 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/18a62c9b-4032-4639-be99-3520b515313b-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.154505 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-gphkk\" (UniqueName: \"kubernetes.io/projected/e9af392d-1940-4203-961e-39da1ac5f3da-kube-api-access-gphkk\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.154464 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5d19811b-4e82-42b2-b6ec-f16e707484f2-config-data" (OuterVolumeSpecName: "config-data") pod "5d19811b-4e82-42b2-b6ec-f16e707484f2" (UID: "5d19811b-4e82-42b2-b6ec-f16e707484f2"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.154538 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/b0606156-f409-4cae-9be1-15d990f69955-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.159913 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5d19811b-4e82-42b2-b6ec-f16e707484f2-kube-api-access-ggq2r" (OuterVolumeSpecName: "kube-api-access-ggq2r") pod "5d19811b-4e82-42b2-b6ec-f16e707484f2" (UID: "5d19811b-4e82-42b2-b6ec-f16e707484f2"). InnerVolumeSpecName "kube-api-access-ggq2r". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.160459 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/18a62c9b-4032-4639-be99-3520b515313b-kube-api-access-hw2xp" (OuterVolumeSpecName: "kube-api-access-hw2xp") pod "18a62c9b-4032-4639-be99-3520b515313b" (UID: "18a62c9b-4032-4639-be99-3520b515313b"). InnerVolumeSpecName "kube-api-access-hw2xp". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.163871 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5d19811b-4e82-42b2-b6ec-f16e707484f2-kolla-config" (OuterVolumeSpecName: "kolla-config") pod "5d19811b-4e82-42b2-b6ec-f16e707484f2" (UID: "5d19811b-4e82-42b2-b6ec-f16e707484f2"). InnerVolumeSpecName "kolla-config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.171978 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5bca7a3b-316e-4357-8ae4-e47be319dbf1-config-data" (OuterVolumeSpecName: "config-data") pod "5bca7a3b-316e-4357-8ae4-e47be319dbf1" (UID: "5bca7a3b-316e-4357-8ae4-e47be319dbf1"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.172148 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5bca7a3b-316e-4357-8ae4-e47be319dbf1-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "5bca7a3b-316e-4357-8ae4-e47be319dbf1" (UID: "5bca7a3b-316e-4357-8ae4-e47be319dbf1"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.199848 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/18a62c9b-4032-4639-be99-3520b515313b-config-data-custom" (OuterVolumeSpecName: "config-data-custom") pod "18a62c9b-4032-4639-be99-3520b515313b" (UID: "18a62c9b-4032-4639-be99-3520b515313b"). InnerVolumeSpecName "config-data-custom". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.207662 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b0606156-f409-4cae-9be1-15d990f69955-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "b0606156-f409-4cae-9be1-15d990f69955" (UID: "b0606156-f409-4cae-9be1-15d990f69955"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.215489 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"50789f13-8194-4e24-b975-512591b816a5","Type":"ContainerDied","Data":"eb3e43a555b8207795889d1049026acc685fad2f90f58455ca120d8b00561c89"} Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.215554 5052 scope.go:117] "RemoveContainer" containerID="256ac84e475433107bd853223013a808d99354e60928d4ce86ddc71170d1b9a7" Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.215694 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.219810 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5bca7a3b-316e-4357-8ae4-e47be319dbf1-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "5bca7a3b-316e-4357-8ae4-e47be319dbf1" (UID: "5bca7a3b-316e-4357-8ae4-e47be319dbf1"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.221003 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"30045a80-9e2b-416b-b7bf-cdb9585ec67d","Type":"ContainerDied","Data":"22c4b7cc88f18fbfdf0a579df2019523c72c975d8a9b701a3adfaa710425c706"} Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.221144 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.225895 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-keystone-listener-74b68b767b-8zpnm" event={"ID":"bc0b4316-9c87-4c72-bc06-b4b526c2070e","Type":"ContainerDied","Data":"845b3d6b4f24339abf3ed376dbf591ebb1a98cc2405345e7336eb05dd9f9d999"} Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.226007 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-keystone-listener-74b68b767b-8zpnm" Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.228462 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.230420 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-conductor-0" event={"ID":"e9af392d-1940-4203-961e-39da1ac5f3da","Type":"ContainerDied","Data":"9ef9370a5e9488feb6b7342fd954b62b84654eebee2633b913f8032d2a5c23f2"} Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.230482 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.244971 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e9af392d-1940-4203-961e-39da1ac5f3da-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "e9af392d-1940-4203-961e-39da1ac5f3da" (UID: "e9af392d-1940-4203-961e-39da1ac5f3da"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.245545 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-worker-7bd785845-99jqj" Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.246000 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-worker-7bd785845-99jqj" event={"ID":"18a62c9b-4032-4639-be99-3520b515313b","Type":"ContainerDied","Data":"acc2228edb501fe2e9a4a38f3f06480f14fabf6f9bde01adac460f118021a3d0"} Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.248049 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3039127c-0211-4430-9b2b-426412241ce4-config-data" (OuterVolumeSpecName: "config-data") pod "3039127c-0211-4430-9b2b-426412241ce4" (UID: "3039127c-0211-4430-9b2b-426412241ce4"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.252309 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a83592aa-a4c1-459e-9d07-f14b390a5e6b-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "a83592aa-a4c1-459e-9d07-f14b390a5e6b" (UID: "a83592aa-a4c1-459e-9d07-f14b390a5e6b"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.253285 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/50789f13-8194-4e24-b975-512591b816a5-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "50789f13-8194-4e24-b975-512591b816a5" (UID: "50789f13-8194-4e24-b975-512591b816a5"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.255965 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a83592aa-a4c1-459e-9d07-f14b390a5e6b-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "a83592aa-a4c1-459e-9d07-f14b390a5e6b" (UID: "a83592aa-a4c1-459e-9d07-f14b390a5e6b"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.257033 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/a7ae5e02-8418-40e2-b65d-7b74625fe5b3-kolla-config\") pod \"a7ae5e02-8418-40e2-b65d-7b74625fe5b3\" (UID: \"a7ae5e02-8418-40e2-b65d-7b74625fe5b3\") " Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.257173 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/a7ae5e02-8418-40e2-b65d-7b74625fe5b3-config-data-generated\") pod \"a7ae5e02-8418-40e2-b65d-7b74625fe5b3\" (UID: \"a7ae5e02-8418-40e2-b65d-7b74625fe5b3\") " Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.257276 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a7ae5e02-8418-40e2-b65d-7b74625fe5b3-combined-ca-bundle\") pod \"a7ae5e02-8418-40e2-b65d-7b74625fe5b3\" (UID: \"a7ae5e02-8418-40e2-b65d-7b74625fe5b3\") " Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.257429 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/a7ae5e02-8418-40e2-b65d-7b74625fe5b3-operator-scripts\") pod \"a7ae5e02-8418-40e2-b65d-7b74625fe5b3\" (UID: \"a7ae5e02-8418-40e2-b65d-7b74625fe5b3\") " Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.257639 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-84lr2\" (UniqueName: \"kubernetes.io/projected/a7ae5e02-8418-40e2-b65d-7b74625fe5b3-kube-api-access-84lr2\") pod \"a7ae5e02-8418-40e2-b65d-7b74625fe5b3\" (UID: \"a7ae5e02-8418-40e2-b65d-7b74625fe5b3\") " Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.258208 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/a7ae5e02-8418-40e2-b65d-7b74625fe5b3-config-data-default\") pod \"a7ae5e02-8418-40e2-b65d-7b74625fe5b3\" (UID: \"a7ae5e02-8418-40e2-b65d-7b74625fe5b3\") " Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.258378 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/a7ae5e02-8418-40e2-b65d-7b74625fe5b3-galera-tls-certs\") pod \"a7ae5e02-8418-40e2-b65d-7b74625fe5b3\" (UID: \"a7ae5e02-8418-40e2-b65d-7b74625fe5b3\") " Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.258570 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"mysql-db\" (UniqueName: \"kubernetes.io/local-volume/local-storage07-crc\") pod \"a7ae5e02-8418-40e2-b65d-7b74625fe5b3\" (UID: \"a7ae5e02-8418-40e2-b65d-7b74625fe5b3\") " Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.257923 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a7ae5e02-8418-40e2-b65d-7b74625fe5b3-config-data-generated" (OuterVolumeSpecName: "config-data-generated") pod "a7ae5e02-8418-40e2-b65d-7b74625fe5b3" (UID: "a7ae5e02-8418-40e2-b65d-7b74625fe5b3"). InnerVolumeSpecName "config-data-generated". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.258406 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a7ae5e02-8418-40e2-b65d-7b74625fe5b3-kolla-config" (OuterVolumeSpecName: "kolla-config") pod "a7ae5e02-8418-40e2-b65d-7b74625fe5b3" (UID: "a7ae5e02-8418-40e2-b65d-7b74625fe5b3"). InnerVolumeSpecName "kolla-config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.258644 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a7ae5e02-8418-40e2-b65d-7b74625fe5b3-config-data-default" (OuterVolumeSpecName: "config-data-default") pod "a7ae5e02-8418-40e2-b65d-7b74625fe5b3" (UID: "a7ae5e02-8418-40e2-b65d-7b74625fe5b3"). InnerVolumeSpecName "config-data-default". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.258761 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a7ae5e02-8418-40e2-b65d-7b74625fe5b3-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "a7ae5e02-8418-40e2-b65d-7b74625fe5b3" (UID: "a7ae5e02-8418-40e2-b65d-7b74625fe5b3"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.259846 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/3039127c-0211-4430-9b2b-426412241ce4-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.259966 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/5bca7a3b-316e-4357-8ae4-e47be319dbf1-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.260230 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/5d19811b-4e82-42b2-b6ec-f16e707484f2-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.260676 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/a7ae5e02-8418-40e2-b65d-7b74625fe5b3-config-data-default\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.260846 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/18a62c9b-4032-4639-be99-3520b515313b-config-data-custom\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.261014 5052 reconciler_common.go:293] "Volume detached for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/5d19811b-4e82-42b2-b6ec-f16e707484f2-kolla-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.261178 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b0606156-f409-4cae-9be1-15d990f69955-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.261299 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5bca7a3b-316e-4357-8ae4-e47be319dbf1-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.261680 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-ggq2r\" (UniqueName: \"kubernetes.io/projected/5d19811b-4e82-42b2-b6ec-f16e707484f2-kube-api-access-ggq2r\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.261835 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-hw2xp\" (UniqueName: \"kubernetes.io/projected/18a62c9b-4032-4639-be99-3520b515313b-kube-api-access-hw2xp\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.261937 5052 reconciler_common.go:293] "Volume detached for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/a7ae5e02-8418-40e2-b65d-7b74625fe5b3-kolla-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.262052 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/a7ae5e02-8418-40e2-b65d-7b74625fe5b3-config-data-generated\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.262134 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/a83592aa-a4c1-459e-9d07-f14b390a5e6b-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.263459 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/a7ae5e02-8418-40e2-b65d-7b74625fe5b3-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.263628 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/5bca7a3b-316e-4357-8ae4-e47be319dbf1-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.263765 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/a83592aa-a4c1-459e-9d07-f14b390a5e6b-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.263852 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/50789f13-8194-4e24-b975-512591b816a5-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.264272 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e9af392d-1940-4203-961e-39da1ac5f3da-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.265135 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-bcdbf47bb-m2ctz" event={"ID":"a83592aa-a4c1-459e-9d07-f14b390a5e6b","Type":"ContainerDied","Data":"e8b8f7643190976f86014ef812f143b1cfef5ed2f44ceb63ca7fb421896c7b57"} Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.265172 5052 scope.go:117] "RemoveContainer" containerID="f1298bb0944b7faf489bfe13489509679a09bf41f1f58deeacfdf2aae7d628e8" Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.265319 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-api-bcdbf47bb-m2ctz" Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.281811 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a7ae5e02-8418-40e2-b65d-7b74625fe5b3-kube-api-access-84lr2" (OuterVolumeSpecName: "kube-api-access-84lr2") pod "a7ae5e02-8418-40e2-b65d-7b74625fe5b3" (UID: "a7ae5e02-8418-40e2-b65d-7b74625fe5b3"). InnerVolumeSpecName "kube-api-access-84lr2". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.287938 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/novacell07298-account-delete-726jv" event={"ID":"6dbc7d8f-34d4-4927-82fe-d049727808fd","Type":"ContainerStarted","Data":"f2dea8c321129a2f70c2b4e2bb700afa4ed814580fa929f3613fc664d7afc648"} Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.288425 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/novacell07298-account-delete-726jv" podUID="6dbc7d8f-34d4-4927-82fe-d049727808fd" containerName="mariadb-account-delete" containerID="cri-o://f2dea8c321129a2f70c2b4e2bb700afa4ed814580fa929f3613fc664d7afc648" gracePeriod=30 Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.288956 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e9af392d-1940-4203-961e-39da1ac5f3da-config-data" (OuterVolumeSpecName: "config-data") pod "e9af392d-1940-4203-961e-39da1ac5f3da" (UID: "e9af392d-1940-4203-961e-39da1ac5f3da"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.292373 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican1171-account-delete-9hmgl" event={"ID":"ab29f82e-0693-4001-9f2a-bb4cc2fa1d5a","Type":"ContainerStarted","Data":"6b1c0593b31124275075bff6c57b21a145a59ea4fd8cdabc101e1eae90d84468"} Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.292698 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/barbican1171-account-delete-9hmgl" podUID="ab29f82e-0693-4001-9f2a-bb4cc2fa1d5a" containerName="mariadb-account-delete" containerID="cri-o://6b1c0593b31124275075bff6c57b21a145a59ea4fd8cdabc101e1eae90d84468" gracePeriod=30 Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.295585 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.299177 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/memcached-0" event={"ID":"5d19811b-4e82-42b2-b6ec-f16e707484f2","Type":"ContainerDied","Data":"3c4772e5ec1181020af33392aa008e4601d530e327e896e9a07400e27341f7d5"} Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.299327 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/memcached-0" Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.301470 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage07-crc" (OuterVolumeSpecName: "mysql-db") pod "a7ae5e02-8418-40e2-b65d-7b74625fe5b3" (UID: "a7ae5e02-8418-40e2-b65d-7b74625fe5b3"). InnerVolumeSpecName "local-storage07-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.303122 5052 scope.go:117] "RemoveContainer" containerID="0d3fa6f69a8171deaa21727301ede51856b6793ff9d13fde6cbe39442a2e4566" Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.303569 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bc0b4316-9c87-4c72-bc06-b4b526c2070e-config-data" (OuterVolumeSpecName: "config-data") pod "bc0b4316-9c87-4c72-bc06-b4b526c2070e" (UID: "bc0b4316-9c87-4c72-bc06-b4b526c2070e"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.304770 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a83592aa-a4c1-459e-9d07-f14b390a5e6b-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "a83592aa-a4c1-459e-9d07-f14b390a5e6b" (UID: "a83592aa-a4c1-459e-9d07-f14b390a5e6b"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.306788 5052 generic.go:334] "Generic (PLEG): container finished" podID="a7ae5e02-8418-40e2-b65d-7b74625fe5b3" containerID="fb98f0e6943bcf2cc568bb5bb0901b5c57179b59a118343bafcea6218d35b382" exitCode=0 Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.307006 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.307929 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-galera-0" event={"ID":"a7ae5e02-8418-40e2-b65d-7b74625fe5b3","Type":"ContainerDied","Data":"fb98f0e6943bcf2cc568bb5bb0901b5c57179b59a118343bafcea6218d35b382"} Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.307989 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.314308 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/novaapib163-account-delete-8z4bt" event={"ID":"4b7b82d8-699c-43f2-8ab0-01936305623f","Type":"ContainerStarted","Data":"d01c0d77268e5ca3389f96b721d0db61466ed205622f949655781949d35fbe36"} Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.314525 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/novaapib163-account-delete-8z4bt" podUID="4b7b82d8-699c-43f2-8ab0-01936305623f" containerName="mariadb-account-delete" containerID="cri-o://d01c0d77268e5ca3389f96b721d0db61466ed205622f949655781949d35fbe36" gracePeriod=30 Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.315681 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/novacell07298-account-delete-726jv" podStartSLOduration=7.315657632 podStartE2EDuration="7.315657632s" podCreationTimestamp="2025-12-03 12:40:57 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:41:04.310597747 +0000 UTC m=+4854.898839521" watchObservedRunningTime="2025-12-03 12:41:04.315657632 +0000 UTC m=+4854.903899396" Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.326761 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/18a62c9b-4032-4639-be99-3520b515313b-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "18a62c9b-4032-4639-be99-3520b515313b" (UID: "18a62c9b-4032-4639-be99-3520b515313b"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.338012 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/barbican1171-account-delete-9hmgl" podStartSLOduration=7.33798882 podStartE2EDuration="7.33798882s" podCreationTimestamp="2025-12-03 12:40:57 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:41:04.332929006 +0000 UTC m=+4854.921170770" watchObservedRunningTime="2025-12-03 12:41:04.33798882 +0000 UTC m=+4854.926230584" Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.338955 5052 generic.go:334] "Generic (PLEG): container finished" podID="296f8cec-ad7b-4b51-a9a9-934301d43d2e" containerID="32c6ff6dd047d38174cc0dc1926116d0446b65b6623a5d45be2ea5d7903f4b0c" exitCode=0 Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.339102 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.341437 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-8h8qz" event={"ID":"296f8cec-ad7b-4b51-a9a9-934301d43d2e","Type":"ContainerDied","Data":"32c6ff6dd047d38174cc0dc1926116d0446b65b6623a5d45be2ea5d7903f4b0c"} Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.342998 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.343089 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutrond24e-account-delete-p2mpp" Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.343117 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-cdf4456f4-v82rz" Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.343185 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.343302 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glancec94f-account-delete-8t6wx" Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.344732 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/placement7f30-account-delete-vld4v" podUID="eb8b91d5-6c99-45ba-8d33-da99f1dd496c" containerName="mariadb-account-delete" containerID="cri-o://bb53f0acf7c936b9d6eebe4376be498cac662cc7e2d991df4516aca0dbea473e" gracePeriod=30 Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.346032 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/18a62c9b-4032-4639-be99-3520b515313b-config-data" (OuterVolumeSpecName: "config-data") pod "18a62c9b-4032-4639-be99-3520b515313b" (UID: "18a62c9b-4032-4639-be99-3520b515313b"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.363879 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/novaapib163-account-delete-8z4bt" podStartSLOduration=7.36385823 podStartE2EDuration="7.36385823s" podCreationTimestamp="2025-12-03 12:40:57 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:41:04.356078208 +0000 UTC m=+4854.944319972" watchObservedRunningTime="2025-12-03 12:41:04.36385823 +0000 UTC m=+4854.952100014" Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.366221 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/bc0b4316-9c87-4c72-bc06-b4b526c2070e-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.366242 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-84lr2\" (UniqueName: \"kubernetes.io/projected/a7ae5e02-8418-40e2-b65d-7b74625fe5b3-kube-api-access-84lr2\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.366252 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a83592aa-a4c1-459e-9d07-f14b390a5e6b-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.366263 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/18a62c9b-4032-4639-be99-3520b515313b-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.366274 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/18a62c9b-4032-4639-be99-3520b515313b-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.366284 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/e9af392d-1940-4203-961e-39da1ac5f3da-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.366302 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage07-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage07-crc\") on node \"crc\" " Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.367145 5052 scope.go:117] "RemoveContainer" containerID="c2aa910114395115673efcfd179c79ba17ccb8368f38abf250fd9ed6719330d7" Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.385115 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5d19811b-4e82-42b2-b6ec-f16e707484f2-memcached-tls-certs" (OuterVolumeSpecName: "memcached-tls-certs") pod "5d19811b-4e82-42b2-b6ec-f16e707484f2" (UID: "5d19811b-4e82-42b2-b6ec-f16e707484f2"). InnerVolumeSpecName "memcached-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.390363 5052 scope.go:117] "RemoveContainer" containerID="a19585c85edfd20edc4392a9229786e6495d1e999b33f47c80f6aec8b04d2bf2" Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.392840 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glancec94f-account-delete-8t6wx"] Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.405380 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/99a0961b-8ed8-49b7-abac-5025d7b351ee-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "99a0961b-8ed8-49b7-abac-5025d7b351ee" (UID: "99a0961b-8ed8-49b7-abac-5025d7b351ee"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.409277 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/glancec94f-account-delete-8t6wx"] Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.413902 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a7ae5e02-8418-40e2-b65d-7b74625fe5b3-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "a7ae5e02-8418-40e2-b65d-7b74625fe5b3" (UID: "a7ae5e02-8418-40e2-b65d-7b74625fe5b3"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.422479 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage07-crc" (UniqueName: "kubernetes.io/local-volume/local-storage07-crc") on node "crc" Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.427552 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/99a0961b-8ed8-49b7-abac-5025d7b351ee-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "99a0961b-8ed8-49b7-abac-5025d7b351ee" (UID: "99a0961b-8ed8-49b7-abac-5025d7b351ee"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.431320 5052 scope.go:117] "RemoveContainer" containerID="f6309d5be7e4db13ae452596e6a4fcddfc94ab7011d5e22b0f3ce37272e5a6eb" Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.431882 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5d19811b-4e82-42b2-b6ec-f16e707484f2-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "5d19811b-4e82-42b2-b6ec-f16e707484f2" (UID: "5d19811b-4e82-42b2-b6ec-f16e707484f2"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.438637 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.442086 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a7ae5e02-8418-40e2-b65d-7b74625fe5b3-galera-tls-certs" (OuterVolumeSpecName: "galera-tls-certs") pod "a7ae5e02-8418-40e2-b65d-7b74625fe5b3" (UID: "a7ae5e02-8418-40e2-b65d-7b74625fe5b3"). InnerVolumeSpecName "galera-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.471886 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.471978 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a7ae5e02-8418-40e2-b65d-7b74625fe5b3-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.472010 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/99a0961b-8ed8-49b7-abac-5025d7b351ee-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.472025 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5d19811b-4e82-42b2-b6ec-f16e707484f2-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.472037 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/99a0961b-8ed8-49b7-abac-5025d7b351ee-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.472050 5052 reconciler_common.go:293] "Volume detached for volume \"memcached-tls-certs\" (UniqueName: \"kubernetes.io/secret/5d19811b-4e82-42b2-b6ec-f16e707484f2-memcached-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.472060 5052 reconciler_common.go:293] "Volume detached for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/a7ae5e02-8418-40e2-b65d-7b74625fe5b3-galera-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.472071 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage07-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage07-crc\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:04 crc kubenswrapper[5052]: E1203 12:41:04.472168 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/combined-ca-bundle: secret "combined-ca-bundle" not found Dec 03 12:41:04 crc kubenswrapper[5052]: E1203 12:41:04.472230 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0556e9f0-e431-4193-9822-6079cb8a1e44-combined-ca-bundle podName:0556e9f0-e431-4193-9822-6079cb8a1e44 nodeName:}" failed. No retries permitted until 2025-12-03 12:41:12.472210889 +0000 UTC m=+4863.060452653 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "combined-ca-bundle" (UniqueName: "kubernetes.io/secret/0556e9f0-e431-4193-9822-6079cb8a1e44-combined-ca-bundle") pod "nova-cell1-conductor-0" (UID: "0556e9f0-e431-4193-9822-6079cb8a1e44") : secret "combined-ca-bundle" not found Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.491779 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.506634 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.513894 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.520977 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.527417 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/neutrond24e-account-delete-p2mpp"] Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.534870 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/neutrond24e-account-delete-p2mpp"] Dec 03 12:41:04 crc kubenswrapper[5052]: E1203 12:41:04.574228 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/rabbitmq-cell1-config-data: configmap "rabbitmq-cell1-config-data" not found Dec 03 12:41:04 crc kubenswrapper[5052]: E1203 12:41:04.574319 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f9bac64c-518e-4630-8889-20e09d5b7a39-config-data podName:f9bac64c-518e-4630-8889-20e09d5b7a39 nodeName:}" failed. No retries permitted until 2025-12-03 12:41:12.574299768 +0000 UTC m=+4863.162541532 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "config-data" (UniqueName: "kubernetes.io/configmap/f9bac64c-518e-4630-8889-20e09d5b7a39-config-data") pod "rabbitmq-cell1-server-0" (UID: "f9bac64c-518e-4630-8889-20e09d5b7a39") : configmap "rabbitmq-cell1-config-data" not found Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.604751 5052 scope.go:117] "RemoveContainer" containerID="88bb1e9d52b42c7f2f99966c163df2f71ff6d2b3bec16e67d60c4dbcaea76bb6" Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.635586 5052 scope.go:117] "RemoveContainer" containerID="0aea18d8ca92744fcf1bb8068e5ff8a5eb48807d9cf65983d90eb513bcd33ddb" Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.648590 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.665189 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.674779 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-worker-7bd785845-99jqj"] Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.681348 5052 scope.go:117] "RemoveContainer" containerID="c45f971177244262381100c68ce2c891e45f06d82ee0da8bc330ce4a83260099" Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.686081 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/barbican-worker-7bd785845-99jqj"] Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.721620 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-api-bcdbf47bb-m2ctz"] Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.724645 5052 scope.go:117] "RemoveContainer" containerID="be69f0c47ca79cbf547e094fd8c2d7c0c191112e45d357d7916c62c184aa277b" Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.740209 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/barbican-api-bcdbf47bb-m2ctz"] Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.759768 5052 scope.go:117] "RemoveContainer" containerID="7af822baa4982fe79fa48809692cdf9c52486cac04ab62564e24484f37f0039e" Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.763390 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-0"] Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.772621 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-0"] Dec 03 12:41:04 crc kubenswrapper[5052]: E1203 12:41:04.778929 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:41:04 crc kubenswrapper[5052]: E1203 12:41:04.779254 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/eb8b91d5-6c99-45ba-8d33-da99f1dd496c-operator-scripts podName:eb8b91d5-6c99-45ba-8d33-da99f1dd496c nodeName:}" failed. No retries permitted until 2025-12-03 12:41:06.779232359 +0000 UTC m=+4857.367474143 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/eb8b91d5-6c99-45ba-8d33-da99f1dd496c-operator-scripts") pod "placement7f30-account-delete-vld4v" (UID: "eb8b91d5-6c99-45ba-8d33-da99f1dd496c") : configmap "openstack-scripts" not found Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.783286 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-keystone-listener-74b68b767b-8zpnm"] Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.792215 5052 scope.go:117] "RemoveContainer" containerID="78246e0cecbf2891b1eda71870c4a06e4588bbbe3e1fc591176960ad3a92ef57" Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.795150 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/barbican-keystone-listener-74b68b767b-8zpnm"] Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.823680 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/memcached-0"] Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.832339 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/memcached-0"] Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.837937 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/placement-cdf4456f4-v82rz"] Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.842266 5052 scope.go:117] "RemoveContainer" containerID="fb98f0e6943bcf2cc568bb5bb0901b5c57179b59a118343bafcea6218d35b382" Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.845532 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/placement-cdf4456f4-v82rz"] Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.853531 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/openstack-galera-0"] Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.859153 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/openstack-galera-0"] Dec 03 12:41:04 crc kubenswrapper[5052]: I1203 12:41:04.868227 5052 scope.go:117] "RemoveContainer" containerID="7010c3ab9f848b0d7bcc5baa0f7b66961b913b599dc72e2ae5037fc9fa66ce73" Dec 03 12:41:04 crc kubenswrapper[5052]: E1203 12:41:04.994412 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/rabbitmq-config-data: configmap "rabbitmq-config-data" not found Dec 03 12:41:04 crc kubenswrapper[5052]: E1203 12:41:04.994491 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d82ef8ea-0c3b-4d89-9c78-22095940f4da-config-data podName:d82ef8ea-0c3b-4d89-9c78-22095940f4da nodeName:}" failed. No retries permitted until 2025-12-03 12:41:12.994474745 +0000 UTC m=+4863.582716509 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "config-data" (UniqueName: "kubernetes.io/configmap/d82ef8ea-0c3b-4d89-9c78-22095940f4da-config-data") pod "rabbitmq-server-0" (UID: "d82ef8ea-0c3b-4d89-9c78-22095940f4da") : configmap "rabbitmq-config-data" not found Dec 03 12:41:05 crc kubenswrapper[5052]: E1203 12:41:05.063607 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="b1ff1f57b80e78690f7e46060a438671dfcfacd08f98a788c2bd60d1eb2bbdf1" cmd=["/usr/bin/pgrep","-r","DRST","nova-scheduler"] Dec 03 12:41:05 crc kubenswrapper[5052]: E1203 12:41:05.067459 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="b1ff1f57b80e78690f7e46060a438671dfcfacd08f98a788c2bd60d1eb2bbdf1" cmd=["/usr/bin/pgrep","-r","DRST","nova-scheduler"] Dec 03 12:41:05 crc kubenswrapper[5052]: E1203 12:41:05.068856 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="b1ff1f57b80e78690f7e46060a438671dfcfacd08f98a788c2bd60d1eb2bbdf1" cmd=["/usr/bin/pgrep","-r","DRST","nova-scheduler"] Dec 03 12:41:05 crc kubenswrapper[5052]: E1203 12:41:05.068898 5052 prober.go:104] "Probe errored" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" probeType="Readiness" pod="openstack-kuttl-tests/nova-scheduler-0" podUID="9e3bfbf3-a39c-48d2-bb1d-062c90c0ef10" containerName="nova-scheduler-scheduler" Dec 03 12:41:05 crc kubenswrapper[5052]: I1203 12:41:05.357279 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-8h8qz" event={"ID":"296f8cec-ad7b-4b51-a9a9-934301d43d2e","Type":"ContainerStarted","Data":"7931c8713f97dce1e306c981b449dd889f9caac3a2406f2863f294197a7435e2"} Dec 03 12:41:05 crc kubenswrapper[5052]: I1203 12:41:05.357399 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-8h8qz" Dec 03 12:41:05 crc kubenswrapper[5052]: I1203 12:41:05.366986 5052 generic.go:334] "Generic (PLEG): container finished" podID="d82ef8ea-0c3b-4d89-9c78-22095940f4da" containerID="ed184550d5a8d5611c9220d80018d6040a97d60ffbd768b55b87c5964020d439" exitCode=0 Dec 03 12:41:05 crc kubenswrapper[5052]: I1203 12:41:05.367045 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-server-0" event={"ID":"d82ef8ea-0c3b-4d89-9c78-22095940f4da","Type":"ContainerDied","Data":"ed184550d5a8d5611c9220d80018d6040a97d60ffbd768b55b87c5964020d439"} Dec 03 12:41:05 crc kubenswrapper[5052]: I1203 12:41:05.382950 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-8h8qz" podStartSLOduration=8.382928814 podStartE2EDuration="8.382928814s" podCreationTimestamp="2025-12-03 12:40:57 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:41:05.37718987 +0000 UTC m=+4855.965431634" watchObservedRunningTime="2025-12-03 12:41:05.382928814 +0000 UTC m=+4855.971170578" Dec 03 12:41:05 crc kubenswrapper[5052]: I1203 12:41:05.390211 5052 generic.go:334] "Generic (PLEG): container finished" podID="fdeb105e-9fbc-44bb-ba15-bc81c7275db1" containerID="303e4e73500eec972e0823935bb4cf2cb918b5d2b159c46edd8b37894c6d37f1" exitCode=0 Dec 03 12:41:05 crc kubenswrapper[5052]: I1203 12:41:05.390321 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-66f85f5bfb-vln92" event={"ID":"fdeb105e-9fbc-44bb-ba15-bc81c7275db1","Type":"ContainerDied","Data":"303e4e73500eec972e0823935bb4cf2cb918b5d2b159c46edd8b37894c6d37f1"} Dec 03 12:41:05 crc kubenswrapper[5052]: I1203 12:41:05.391858 5052 generic.go:334] "Generic (PLEG): container finished" podID="f9bac64c-518e-4630-8889-20e09d5b7a39" containerID="ed967e0e50724d3854ca971b7a125ee6c019b59d594a733d91f837aeb86bc924" exitCode=0 Dec 03 12:41:05 crc kubenswrapper[5052]: I1203 12:41:05.391915 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" event={"ID":"f9bac64c-518e-4630-8889-20e09d5b7a39","Type":"ContainerDied","Data":"ed967e0e50724d3854ca971b7a125ee6c019b59d594a733d91f837aeb86bc924"} Dec 03 12:41:05 crc kubenswrapper[5052]: I1203 12:41:05.573490 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" podUID="0f16f521-a181-40e4-a1ed-38f4c8862ffb" containerName="nova-cell1-novncproxy-novncproxy" probeResult="failure" output="Get \"https://10.217.1.131:6080/vnc_lite.html\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Dec 03 12:41:05 crc kubenswrapper[5052]: I1203 12:41:05.688846 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:41:05 crc kubenswrapper[5052]: I1203 12:41:05.693811 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="12f52069-8b08-4e4f-a3c1-a5421efb5844" path="/var/lib/kubelet/pods/12f52069-8b08-4e4f-a3c1-a5421efb5844/volumes" Dec 03 12:41:05 crc kubenswrapper[5052]: I1203 12:41:05.694585 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="18a62c9b-4032-4639-be99-3520b515313b" path="/var/lib/kubelet/pods/18a62c9b-4032-4639-be99-3520b515313b/volumes" Dec 03 12:41:05 crc kubenswrapper[5052]: I1203 12:41:05.695323 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="30045a80-9e2b-416b-b7bf-cdb9585ec67d" path="/var/lib/kubelet/pods/30045a80-9e2b-416b-b7bf-cdb9585ec67d/volumes" Dec 03 12:41:05 crc kubenswrapper[5052]: I1203 12:41:05.697939 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3039127c-0211-4430-9b2b-426412241ce4" path="/var/lib/kubelet/pods/3039127c-0211-4430-9b2b-426412241ce4/volumes" Dec 03 12:41:05 crc kubenswrapper[5052]: I1203 12:41:05.698961 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="50789f13-8194-4e24-b975-512591b816a5" path="/var/lib/kubelet/pods/50789f13-8194-4e24-b975-512591b816a5/volumes" Dec 03 12:41:05 crc kubenswrapper[5052]: I1203 12:41:05.699618 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5bca7a3b-316e-4357-8ae4-e47be319dbf1" path="/var/lib/kubelet/pods/5bca7a3b-316e-4357-8ae4-e47be319dbf1/volumes" Dec 03 12:41:05 crc kubenswrapper[5052]: I1203 12:41:05.700862 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5d19811b-4e82-42b2-b6ec-f16e707484f2" path="/var/lib/kubelet/pods/5d19811b-4e82-42b2-b6ec-f16e707484f2/volumes" Dec 03 12:41:05 crc kubenswrapper[5052]: I1203 12:41:05.701376 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="99a0961b-8ed8-49b7-abac-5025d7b351ee" path="/var/lib/kubelet/pods/99a0961b-8ed8-49b7-abac-5025d7b351ee/volumes" Dec 03 12:41:05 crc kubenswrapper[5052]: I1203 12:41:05.703006 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a7ae5e02-8418-40e2-b65d-7b74625fe5b3" path="/var/lib/kubelet/pods/a7ae5e02-8418-40e2-b65d-7b74625fe5b3/volumes" Dec 03 12:41:05 crc kubenswrapper[5052]: I1203 12:41:05.703697 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a83592aa-a4c1-459e-9d07-f14b390a5e6b" path="/var/lib/kubelet/pods/a83592aa-a4c1-459e-9d07-f14b390a5e6b/volumes" Dec 03 12:41:05 crc kubenswrapper[5052]: I1203 12:41:05.704713 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b0606156-f409-4cae-9be1-15d990f69955" path="/var/lib/kubelet/pods/b0606156-f409-4cae-9be1-15d990f69955/volumes" Dec 03 12:41:05 crc kubenswrapper[5052]: I1203 12:41:05.706193 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b7c49d90-c0af-4d4b-a080-8c7dce3ee66a" path="/var/lib/kubelet/pods/b7c49d90-c0af-4d4b-a080-8c7dce3ee66a/volumes" Dec 03 12:41:05 crc kubenswrapper[5052]: I1203 12:41:05.707133 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bc0b4316-9c87-4c72-bc06-b4b526c2070e" path="/var/lib/kubelet/pods/bc0b4316-9c87-4c72-bc06-b4b526c2070e/volumes" Dec 03 12:41:05 crc kubenswrapper[5052]: E1203 12:41:05.707837 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:41:05 crc kubenswrapper[5052]: E1203 12:41:05.707964 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5fdc7b7d-f9ed-4d5c-a77b-6a6b0fd186f9-operator-scripts podName:5fdc7b7d-f9ed-4d5c-a77b-6a6b0fd186f9 nodeName:}" failed. No retries permitted until 2025-12-03 12:41:09.707938629 +0000 UTC m=+4860.296180393 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/5fdc7b7d-f9ed-4d5c-a77b-6a6b0fd186f9-operator-scripts") pod "cinder3a0d-account-delete-lxd6l" (UID: "5fdc7b7d-f9ed-4d5c-a77b-6a6b0fd186f9") : configmap "openstack-scripts" not found Dec 03 12:41:05 crc kubenswrapper[5052]: I1203 12:41:05.708475 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e9af392d-1940-4203-961e-39da1ac5f3da" path="/var/lib/kubelet/pods/e9af392d-1940-4203-961e-39da1ac5f3da/volumes" Dec 03 12:41:05 crc kubenswrapper[5052]: I1203 12:41:05.801104 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-66f85f5bfb-vln92" Dec 03 12:41:05 crc kubenswrapper[5052]: I1203 12:41:05.804483 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:41:05 crc kubenswrapper[5052]: I1203 12:41:05.808297 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"server-conf\" (UniqueName: \"kubernetes.io/configmap/d82ef8ea-0c3b-4d89-9c78-22095940f4da-server-conf\") pod \"d82ef8ea-0c3b-4d89-9c78-22095940f4da\" (UID: \"d82ef8ea-0c3b-4d89-9c78-22095940f4da\") " Dec 03 12:41:05 crc kubenswrapper[5052]: I1203 12:41:05.808346 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"rabbitmq-confd\" (UniqueName: \"kubernetes.io/projected/d82ef8ea-0c3b-4d89-9c78-22095940f4da-rabbitmq-confd\") pod \"d82ef8ea-0c3b-4d89-9c78-22095940f4da\" (UID: \"d82ef8ea-0c3b-4d89-9c78-22095940f4da\") " Dec 03 12:41:05 crc kubenswrapper[5052]: I1203 12:41:05.808425 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"rabbitmq-erlang-cookie\" (UniqueName: \"kubernetes.io/empty-dir/d82ef8ea-0c3b-4d89-9c78-22095940f4da-rabbitmq-erlang-cookie\") pod \"d82ef8ea-0c3b-4d89-9c78-22095940f4da\" (UID: \"d82ef8ea-0c3b-4d89-9c78-22095940f4da\") " Dec 03 12:41:05 crc kubenswrapper[5052]: I1203 12:41:05.808488 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"persistence\" (UniqueName: \"kubernetes.io/local-volume/local-storage06-crc\") pod \"d82ef8ea-0c3b-4d89-9c78-22095940f4da\" (UID: \"d82ef8ea-0c3b-4d89-9c78-22095940f4da\") " Dec 03 12:41:05 crc kubenswrapper[5052]: I1203 12:41:05.808532 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"erlang-cookie-secret\" (UniqueName: \"kubernetes.io/secret/d82ef8ea-0c3b-4d89-9c78-22095940f4da-erlang-cookie-secret\") pod \"d82ef8ea-0c3b-4d89-9c78-22095940f4da\" (UID: \"d82ef8ea-0c3b-4d89-9c78-22095940f4da\") " Dec 03 12:41:05 crc kubenswrapper[5052]: I1203 12:41:05.808558 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"rabbitmq-plugins\" (UniqueName: \"kubernetes.io/empty-dir/d82ef8ea-0c3b-4d89-9c78-22095940f4da-rabbitmq-plugins\") pod \"d82ef8ea-0c3b-4d89-9c78-22095940f4da\" (UID: \"d82ef8ea-0c3b-4d89-9c78-22095940f4da\") " Dec 03 12:41:05 crc kubenswrapper[5052]: I1203 12:41:05.808607 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/d82ef8ea-0c3b-4d89-9c78-22095940f4da-config-data\") pod \"d82ef8ea-0c3b-4d89-9c78-22095940f4da\" (UID: \"d82ef8ea-0c3b-4d89-9c78-22095940f4da\") " Dec 03 12:41:05 crc kubenswrapper[5052]: I1203 12:41:05.808636 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"plugins-conf\" (UniqueName: \"kubernetes.io/configmap/d82ef8ea-0c3b-4d89-9c78-22095940f4da-plugins-conf\") pod \"d82ef8ea-0c3b-4d89-9c78-22095940f4da\" (UID: \"d82ef8ea-0c3b-4d89-9c78-22095940f4da\") " Dec 03 12:41:05 crc kubenswrapper[5052]: I1203 12:41:05.808677 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"rabbitmq-tls\" (UniqueName: \"kubernetes.io/projected/d82ef8ea-0c3b-4d89-9c78-22095940f4da-rabbitmq-tls\") pod \"d82ef8ea-0c3b-4d89-9c78-22095940f4da\" (UID: \"d82ef8ea-0c3b-4d89-9c78-22095940f4da\") " Dec 03 12:41:05 crc kubenswrapper[5052]: I1203 12:41:05.808717 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"pod-info\" (UniqueName: \"kubernetes.io/downward-api/d82ef8ea-0c3b-4d89-9c78-22095940f4da-pod-info\") pod \"d82ef8ea-0c3b-4d89-9c78-22095940f4da\" (UID: \"d82ef8ea-0c3b-4d89-9c78-22095940f4da\") " Dec 03 12:41:05 crc kubenswrapper[5052]: I1203 12:41:05.808759 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x46l5\" (UniqueName: \"kubernetes.io/projected/d82ef8ea-0c3b-4d89-9c78-22095940f4da-kube-api-access-x46l5\") pod \"d82ef8ea-0c3b-4d89-9c78-22095940f4da\" (UID: \"d82ef8ea-0c3b-4d89-9c78-22095940f4da\") " Dec 03 12:41:05 crc kubenswrapper[5052]: I1203 12:41:05.808919 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d82ef8ea-0c3b-4d89-9c78-22095940f4da-rabbitmq-erlang-cookie" (OuterVolumeSpecName: "rabbitmq-erlang-cookie") pod "d82ef8ea-0c3b-4d89-9c78-22095940f4da" (UID: "d82ef8ea-0c3b-4d89-9c78-22095940f4da"). InnerVolumeSpecName "rabbitmq-erlang-cookie". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:41:05 crc kubenswrapper[5052]: I1203 12:41:05.809138 5052 reconciler_common.go:293] "Volume detached for volume \"rabbitmq-erlang-cookie\" (UniqueName: \"kubernetes.io/empty-dir/d82ef8ea-0c3b-4d89-9c78-22095940f4da-rabbitmq-erlang-cookie\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:05 crc kubenswrapper[5052]: I1203 12:41:05.812726 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d82ef8ea-0c3b-4d89-9c78-22095940f4da-rabbitmq-plugins" (OuterVolumeSpecName: "rabbitmq-plugins") pod "d82ef8ea-0c3b-4d89-9c78-22095940f4da" (UID: "d82ef8ea-0c3b-4d89-9c78-22095940f4da"). InnerVolumeSpecName "rabbitmq-plugins". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:41:05 crc kubenswrapper[5052]: I1203 12:41:05.814152 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d82ef8ea-0c3b-4d89-9c78-22095940f4da-plugins-conf" (OuterVolumeSpecName: "plugins-conf") pod "d82ef8ea-0c3b-4d89-9c78-22095940f4da" (UID: "d82ef8ea-0c3b-4d89-9c78-22095940f4da"). InnerVolumeSpecName "plugins-conf". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:41:05 crc kubenswrapper[5052]: I1203 12:41:05.816177 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d82ef8ea-0c3b-4d89-9c78-22095940f4da-pod-info" (OuterVolumeSpecName: "pod-info") pod "d82ef8ea-0c3b-4d89-9c78-22095940f4da" (UID: "d82ef8ea-0c3b-4d89-9c78-22095940f4da"). InnerVolumeSpecName "pod-info". PluginName "kubernetes.io/downward-api", VolumeGidValue "" Dec 03 12:41:05 crc kubenswrapper[5052]: I1203 12:41:05.816271 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage06-crc" (OuterVolumeSpecName: "persistence") pod "d82ef8ea-0c3b-4d89-9c78-22095940f4da" (UID: "d82ef8ea-0c3b-4d89-9c78-22095940f4da"). InnerVolumeSpecName "local-storage06-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 12:41:05 crc kubenswrapper[5052]: I1203 12:41:05.816542 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d82ef8ea-0c3b-4d89-9c78-22095940f4da-kube-api-access-x46l5" (OuterVolumeSpecName: "kube-api-access-x46l5") pod "d82ef8ea-0c3b-4d89-9c78-22095940f4da" (UID: "d82ef8ea-0c3b-4d89-9c78-22095940f4da"). InnerVolumeSpecName "kube-api-access-x46l5". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:41:05 crc kubenswrapper[5052]: I1203 12:41:05.816583 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d82ef8ea-0c3b-4d89-9c78-22095940f4da-rabbitmq-tls" (OuterVolumeSpecName: "rabbitmq-tls") pod "d82ef8ea-0c3b-4d89-9c78-22095940f4da" (UID: "d82ef8ea-0c3b-4d89-9c78-22095940f4da"). InnerVolumeSpecName "rabbitmq-tls". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:41:05 crc kubenswrapper[5052]: I1203 12:41:05.816759 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d82ef8ea-0c3b-4d89-9c78-22095940f4da-erlang-cookie-secret" (OuterVolumeSpecName: "erlang-cookie-secret") pod "d82ef8ea-0c3b-4d89-9c78-22095940f4da" (UID: "d82ef8ea-0c3b-4d89-9c78-22095940f4da"). InnerVolumeSpecName "erlang-cookie-secret". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:41:05 crc kubenswrapper[5052]: I1203 12:41:05.845682 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d82ef8ea-0c3b-4d89-9c78-22095940f4da-config-data" (OuterVolumeSpecName: "config-data") pod "d82ef8ea-0c3b-4d89-9c78-22095940f4da" (UID: "d82ef8ea-0c3b-4d89-9c78-22095940f4da"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:41:05 crc kubenswrapper[5052]: I1203 12:41:05.890315 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d82ef8ea-0c3b-4d89-9c78-22095940f4da-server-conf" (OuterVolumeSpecName: "server-conf") pod "d82ef8ea-0c3b-4d89-9c78-22095940f4da" (UID: "d82ef8ea-0c3b-4d89-9c78-22095940f4da"). InnerVolumeSpecName "server-conf". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:41:05 crc kubenswrapper[5052]: I1203 12:41:05.910516 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/fdeb105e-9fbc-44bb-ba15-bc81c7275db1-scripts\") pod \"fdeb105e-9fbc-44bb-ba15-bc81c7275db1\" (UID: \"fdeb105e-9fbc-44bb-ba15-bc81c7275db1\") " Dec 03 12:41:05 crc kubenswrapper[5052]: I1203 12:41:05.910578 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"rabbitmq-confd\" (UniqueName: \"kubernetes.io/projected/f9bac64c-518e-4630-8889-20e09d5b7a39-rabbitmq-confd\") pod \"f9bac64c-518e-4630-8889-20e09d5b7a39\" (UID: \"f9bac64c-518e-4630-8889-20e09d5b7a39\") " Dec 03 12:41:05 crc kubenswrapper[5052]: I1203 12:41:05.910603 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/fdeb105e-9fbc-44bb-ba15-bc81c7275db1-internal-tls-certs\") pod \"fdeb105e-9fbc-44bb-ba15-bc81c7275db1\" (UID: \"fdeb105e-9fbc-44bb-ba15-bc81c7275db1\") " Dec 03 12:41:05 crc kubenswrapper[5052]: I1203 12:41:05.910634 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/fdeb105e-9fbc-44bb-ba15-bc81c7275db1-combined-ca-bundle\") pod \"fdeb105e-9fbc-44bb-ba15-bc81c7275db1\" (UID: \"fdeb105e-9fbc-44bb-ba15-bc81c7275db1\") " Dec 03 12:41:05 crc kubenswrapper[5052]: I1203 12:41:05.910667 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"rabbitmq-plugins\" (UniqueName: \"kubernetes.io/empty-dir/f9bac64c-518e-4630-8889-20e09d5b7a39-rabbitmq-plugins\") pod \"f9bac64c-518e-4630-8889-20e09d5b7a39\" (UID: \"f9bac64c-518e-4630-8889-20e09d5b7a39\") " Dec 03 12:41:05 crc kubenswrapper[5052]: I1203 12:41:05.910741 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x4vkn\" (UniqueName: \"kubernetes.io/projected/fdeb105e-9fbc-44bb-ba15-bc81c7275db1-kube-api-access-x4vkn\") pod \"fdeb105e-9fbc-44bb-ba15-bc81c7275db1\" (UID: \"fdeb105e-9fbc-44bb-ba15-bc81c7275db1\") " Dec 03 12:41:05 crc kubenswrapper[5052]: I1203 12:41:05.910763 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/fdeb105e-9fbc-44bb-ba15-bc81c7275db1-config-data\") pod \"fdeb105e-9fbc-44bb-ba15-bc81c7275db1\" (UID: \"fdeb105e-9fbc-44bb-ba15-bc81c7275db1\") " Dec 03 12:41:05 crc kubenswrapper[5052]: I1203 12:41:05.910797 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"server-conf\" (UniqueName: \"kubernetes.io/configmap/f9bac64c-518e-4630-8889-20e09d5b7a39-server-conf\") pod \"f9bac64c-518e-4630-8889-20e09d5b7a39\" (UID: \"f9bac64c-518e-4630-8889-20e09d5b7a39\") " Dec 03 12:41:05 crc kubenswrapper[5052]: I1203 12:41:05.910837 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"rabbitmq-tls\" (UniqueName: \"kubernetes.io/projected/f9bac64c-518e-4630-8889-20e09d5b7a39-rabbitmq-tls\") pod \"f9bac64c-518e-4630-8889-20e09d5b7a39\" (UID: \"f9bac64c-518e-4630-8889-20e09d5b7a39\") " Dec 03 12:41:05 crc kubenswrapper[5052]: I1203 12:41:05.910856 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/fdeb105e-9fbc-44bb-ba15-bc81c7275db1-fernet-keys\") pod \"fdeb105e-9fbc-44bb-ba15-bc81c7275db1\" (UID: \"fdeb105e-9fbc-44bb-ba15-bc81c7275db1\") " Dec 03 12:41:05 crc kubenswrapper[5052]: I1203 12:41:05.910889 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/fdeb105e-9fbc-44bb-ba15-bc81c7275db1-credential-keys\") pod \"fdeb105e-9fbc-44bb-ba15-bc81c7275db1\" (UID: \"fdeb105e-9fbc-44bb-ba15-bc81c7275db1\") " Dec 03 12:41:05 crc kubenswrapper[5052]: I1203 12:41:05.910913 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"persistence\" (UniqueName: \"kubernetes.io/local-volume/local-storage04-crc\") pod \"f9bac64c-518e-4630-8889-20e09d5b7a39\" (UID: \"f9bac64c-518e-4630-8889-20e09d5b7a39\") " Dec 03 12:41:05 crc kubenswrapper[5052]: I1203 12:41:05.910941 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"rabbitmq-erlang-cookie\" (UniqueName: \"kubernetes.io/empty-dir/f9bac64c-518e-4630-8889-20e09d5b7a39-rabbitmq-erlang-cookie\") pod \"f9bac64c-518e-4630-8889-20e09d5b7a39\" (UID: \"f9bac64c-518e-4630-8889-20e09d5b7a39\") " Dec 03 12:41:05 crc kubenswrapper[5052]: I1203 12:41:05.910962 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"erlang-cookie-secret\" (UniqueName: \"kubernetes.io/secret/f9bac64c-518e-4630-8889-20e09d5b7a39-erlang-cookie-secret\") pod \"f9bac64c-518e-4630-8889-20e09d5b7a39\" (UID: \"f9bac64c-518e-4630-8889-20e09d5b7a39\") " Dec 03 12:41:05 crc kubenswrapper[5052]: I1203 12:41:05.910981 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/fdeb105e-9fbc-44bb-ba15-bc81c7275db1-public-tls-certs\") pod \"fdeb105e-9fbc-44bb-ba15-bc81c7275db1\" (UID: \"fdeb105e-9fbc-44bb-ba15-bc81c7275db1\") " Dec 03 12:41:05 crc kubenswrapper[5052]: I1203 12:41:05.911006 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-69sgs\" (UniqueName: \"kubernetes.io/projected/f9bac64c-518e-4630-8889-20e09d5b7a39-kube-api-access-69sgs\") pod \"f9bac64c-518e-4630-8889-20e09d5b7a39\" (UID: \"f9bac64c-518e-4630-8889-20e09d5b7a39\") " Dec 03 12:41:05 crc kubenswrapper[5052]: I1203 12:41:05.911041 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"pod-info\" (UniqueName: \"kubernetes.io/downward-api/f9bac64c-518e-4630-8889-20e09d5b7a39-pod-info\") pod \"f9bac64c-518e-4630-8889-20e09d5b7a39\" (UID: \"f9bac64c-518e-4630-8889-20e09d5b7a39\") " Dec 03 12:41:05 crc kubenswrapper[5052]: I1203 12:41:05.911067 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"plugins-conf\" (UniqueName: \"kubernetes.io/configmap/f9bac64c-518e-4630-8889-20e09d5b7a39-plugins-conf\") pod \"f9bac64c-518e-4630-8889-20e09d5b7a39\" (UID: \"f9bac64c-518e-4630-8889-20e09d5b7a39\") " Dec 03 12:41:05 crc kubenswrapper[5052]: I1203 12:41:05.911089 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/f9bac64c-518e-4630-8889-20e09d5b7a39-config-data\") pod \"f9bac64c-518e-4630-8889-20e09d5b7a39\" (UID: \"f9bac64c-518e-4630-8889-20e09d5b7a39\") " Dec 03 12:41:05 crc kubenswrapper[5052]: I1203 12:41:05.911440 5052 reconciler_common.go:293] "Volume detached for volume \"pod-info\" (UniqueName: \"kubernetes.io/downward-api/d82ef8ea-0c3b-4d89-9c78-22095940f4da-pod-info\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:05 crc kubenswrapper[5052]: I1203 12:41:05.911459 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-x46l5\" (UniqueName: \"kubernetes.io/projected/d82ef8ea-0c3b-4d89-9c78-22095940f4da-kube-api-access-x46l5\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:05 crc kubenswrapper[5052]: I1203 12:41:05.911469 5052 reconciler_common.go:293] "Volume detached for volume \"server-conf\" (UniqueName: \"kubernetes.io/configmap/d82ef8ea-0c3b-4d89-9c78-22095940f4da-server-conf\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:05 crc kubenswrapper[5052]: I1203 12:41:05.911489 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage06-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage06-crc\") on node \"crc\" " Dec 03 12:41:05 crc kubenswrapper[5052]: I1203 12:41:05.911499 5052 reconciler_common.go:293] "Volume detached for volume \"erlang-cookie-secret\" (UniqueName: \"kubernetes.io/secret/d82ef8ea-0c3b-4d89-9c78-22095940f4da-erlang-cookie-secret\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:05 crc kubenswrapper[5052]: I1203 12:41:05.911521 5052 reconciler_common.go:293] "Volume detached for volume \"rabbitmq-plugins\" (UniqueName: \"kubernetes.io/empty-dir/d82ef8ea-0c3b-4d89-9c78-22095940f4da-rabbitmq-plugins\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:05 crc kubenswrapper[5052]: I1203 12:41:05.911532 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/d82ef8ea-0c3b-4d89-9c78-22095940f4da-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:05 crc kubenswrapper[5052]: I1203 12:41:05.911541 5052 reconciler_common.go:293] "Volume detached for volume \"plugins-conf\" (UniqueName: \"kubernetes.io/configmap/d82ef8ea-0c3b-4d89-9c78-22095940f4da-plugins-conf\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:05 crc kubenswrapper[5052]: I1203 12:41:05.911549 5052 reconciler_common.go:293] "Volume detached for volume \"rabbitmq-tls\" (UniqueName: \"kubernetes.io/projected/d82ef8ea-0c3b-4d89-9c78-22095940f4da-rabbitmq-tls\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:05 crc kubenswrapper[5052]: I1203 12:41:05.912268 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f9bac64c-518e-4630-8889-20e09d5b7a39-rabbitmq-plugins" (OuterVolumeSpecName: "rabbitmq-plugins") pod "f9bac64c-518e-4630-8889-20e09d5b7a39" (UID: "f9bac64c-518e-4630-8889-20e09d5b7a39"). InnerVolumeSpecName "rabbitmq-plugins". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:41:05 crc kubenswrapper[5052]: I1203 12:41:05.913143 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f9bac64c-518e-4630-8889-20e09d5b7a39-plugins-conf" (OuterVolumeSpecName: "plugins-conf") pod "f9bac64c-518e-4630-8889-20e09d5b7a39" (UID: "f9bac64c-518e-4630-8889-20e09d5b7a39"). InnerVolumeSpecName "plugins-conf". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:41:05 crc kubenswrapper[5052]: I1203 12:41:05.914338 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f9bac64c-518e-4630-8889-20e09d5b7a39-rabbitmq-erlang-cookie" (OuterVolumeSpecName: "rabbitmq-erlang-cookie") pod "f9bac64c-518e-4630-8889-20e09d5b7a39" (UID: "f9bac64c-518e-4630-8889-20e09d5b7a39"). InnerVolumeSpecName "rabbitmq-erlang-cookie". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:41:05 crc kubenswrapper[5052]: I1203 12:41:05.914777 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fdeb105e-9fbc-44bb-ba15-bc81c7275db1-kube-api-access-x4vkn" (OuterVolumeSpecName: "kube-api-access-x4vkn") pod "fdeb105e-9fbc-44bb-ba15-bc81c7275db1" (UID: "fdeb105e-9fbc-44bb-ba15-bc81c7275db1"). InnerVolumeSpecName "kube-api-access-x4vkn". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:41:05 crc kubenswrapper[5052]: I1203 12:41:05.915411 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f9bac64c-518e-4630-8889-20e09d5b7a39-rabbitmq-tls" (OuterVolumeSpecName: "rabbitmq-tls") pod "f9bac64c-518e-4630-8889-20e09d5b7a39" (UID: "f9bac64c-518e-4630-8889-20e09d5b7a39"). InnerVolumeSpecName "rabbitmq-tls". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:41:05 crc kubenswrapper[5052]: I1203 12:41:05.916119 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fdeb105e-9fbc-44bb-ba15-bc81c7275db1-scripts" (OuterVolumeSpecName: "scripts") pod "fdeb105e-9fbc-44bb-ba15-bc81c7275db1" (UID: "fdeb105e-9fbc-44bb-ba15-bc81c7275db1"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:41:05 crc kubenswrapper[5052]: I1203 12:41:05.916578 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f9bac64c-518e-4630-8889-20e09d5b7a39-kube-api-access-69sgs" (OuterVolumeSpecName: "kube-api-access-69sgs") pod "f9bac64c-518e-4630-8889-20e09d5b7a39" (UID: "f9bac64c-518e-4630-8889-20e09d5b7a39"). InnerVolumeSpecName "kube-api-access-69sgs". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:41:05 crc kubenswrapper[5052]: I1203 12:41:05.919013 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f9bac64c-518e-4630-8889-20e09d5b7a39-pod-info" (OuterVolumeSpecName: "pod-info") pod "f9bac64c-518e-4630-8889-20e09d5b7a39" (UID: "f9bac64c-518e-4630-8889-20e09d5b7a39"). InnerVolumeSpecName "pod-info". PluginName "kubernetes.io/downward-api", VolumeGidValue "" Dec 03 12:41:05 crc kubenswrapper[5052]: I1203 12:41:05.920354 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fdeb105e-9fbc-44bb-ba15-bc81c7275db1-credential-keys" (OuterVolumeSpecName: "credential-keys") pod "fdeb105e-9fbc-44bb-ba15-bc81c7275db1" (UID: "fdeb105e-9fbc-44bb-ba15-bc81c7275db1"). InnerVolumeSpecName "credential-keys". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:41:05 crc kubenswrapper[5052]: I1203 12:41:05.922034 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fdeb105e-9fbc-44bb-ba15-bc81c7275db1-fernet-keys" (OuterVolumeSpecName: "fernet-keys") pod "fdeb105e-9fbc-44bb-ba15-bc81c7275db1" (UID: "fdeb105e-9fbc-44bb-ba15-bc81c7275db1"). InnerVolumeSpecName "fernet-keys". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:41:05 crc kubenswrapper[5052]: I1203 12:41:05.922046 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f9bac64c-518e-4630-8889-20e09d5b7a39-erlang-cookie-secret" (OuterVolumeSpecName: "erlang-cookie-secret") pod "f9bac64c-518e-4630-8889-20e09d5b7a39" (UID: "f9bac64c-518e-4630-8889-20e09d5b7a39"). InnerVolumeSpecName "erlang-cookie-secret". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:41:05 crc kubenswrapper[5052]: I1203 12:41:05.925470 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d82ef8ea-0c3b-4d89-9c78-22095940f4da-rabbitmq-confd" (OuterVolumeSpecName: "rabbitmq-confd") pod "d82ef8ea-0c3b-4d89-9c78-22095940f4da" (UID: "d82ef8ea-0c3b-4d89-9c78-22095940f4da"). InnerVolumeSpecName "rabbitmq-confd". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:41:05 crc kubenswrapper[5052]: I1203 12:41:05.931190 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage04-crc" (OuterVolumeSpecName: "persistence") pod "f9bac64c-518e-4630-8889-20e09d5b7a39" (UID: "f9bac64c-518e-4630-8889-20e09d5b7a39"). InnerVolumeSpecName "local-storage04-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 12:41:05 crc kubenswrapper[5052]: I1203 12:41:05.931248 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage06-crc" (UniqueName: "kubernetes.io/local-volume/local-storage06-crc") on node "crc" Dec 03 12:41:05 crc kubenswrapper[5052]: I1203 12:41:05.944060 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fdeb105e-9fbc-44bb-ba15-bc81c7275db1-config-data" (OuterVolumeSpecName: "config-data") pod "fdeb105e-9fbc-44bb-ba15-bc81c7275db1" (UID: "fdeb105e-9fbc-44bb-ba15-bc81c7275db1"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:41:05 crc kubenswrapper[5052]: I1203 12:41:05.951769 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fdeb105e-9fbc-44bb-ba15-bc81c7275db1-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "fdeb105e-9fbc-44bb-ba15-bc81c7275db1" (UID: "fdeb105e-9fbc-44bb-ba15-bc81c7275db1"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:41:05 crc kubenswrapper[5052]: I1203 12:41:05.952948 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f9bac64c-518e-4630-8889-20e09d5b7a39-config-data" (OuterVolumeSpecName: "config-data") pod "f9bac64c-518e-4630-8889-20e09d5b7a39" (UID: "f9bac64c-518e-4630-8889-20e09d5b7a39"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:41:05 crc kubenswrapper[5052]: I1203 12:41:05.962920 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fdeb105e-9fbc-44bb-ba15-bc81c7275db1-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "fdeb105e-9fbc-44bb-ba15-bc81c7275db1" (UID: "fdeb105e-9fbc-44bb-ba15-bc81c7275db1"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:41:05 crc kubenswrapper[5052]: I1203 12:41:05.964898 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f9bac64c-518e-4630-8889-20e09d5b7a39-server-conf" (OuterVolumeSpecName: "server-conf") pod "f9bac64c-518e-4630-8889-20e09d5b7a39" (UID: "f9bac64c-518e-4630-8889-20e09d5b7a39"). InnerVolumeSpecName "server-conf". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:41:06 crc kubenswrapper[5052]: I1203 12:41:05.988140 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fdeb105e-9fbc-44bb-ba15-bc81c7275db1-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "fdeb105e-9fbc-44bb-ba15-bc81c7275db1" (UID: "fdeb105e-9fbc-44bb-ba15-bc81c7275db1"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:41:06 crc kubenswrapper[5052]: I1203 12:41:06.003303 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f9bac64c-518e-4630-8889-20e09d5b7a39-rabbitmq-confd" (OuterVolumeSpecName: "rabbitmq-confd") pod "f9bac64c-518e-4630-8889-20e09d5b7a39" (UID: "f9bac64c-518e-4630-8889-20e09d5b7a39"). InnerVolumeSpecName "rabbitmq-confd". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:41:06 crc kubenswrapper[5052]: I1203 12:41:06.013581 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage06-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage06-crc\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:06 crc kubenswrapper[5052]: I1203 12:41:06.013617 5052 reconciler_common.go:293] "Volume detached for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/fdeb105e-9fbc-44bb-ba15-bc81c7275db1-credential-keys\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:06 crc kubenswrapper[5052]: I1203 12:41:06.013663 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage04-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage04-crc\") on node \"crc\" " Dec 03 12:41:06 crc kubenswrapper[5052]: I1203 12:41:06.013680 5052 reconciler_common.go:293] "Volume detached for volume \"rabbitmq-erlang-cookie\" (UniqueName: \"kubernetes.io/empty-dir/f9bac64c-518e-4630-8889-20e09d5b7a39-rabbitmq-erlang-cookie\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:06 crc kubenswrapper[5052]: I1203 12:41:06.013694 5052 reconciler_common.go:293] "Volume detached for volume \"erlang-cookie-secret\" (UniqueName: \"kubernetes.io/secret/f9bac64c-518e-4630-8889-20e09d5b7a39-erlang-cookie-secret\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:06 crc kubenswrapper[5052]: I1203 12:41:06.013707 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/fdeb105e-9fbc-44bb-ba15-bc81c7275db1-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:06 crc kubenswrapper[5052]: I1203 12:41:06.013721 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-69sgs\" (UniqueName: \"kubernetes.io/projected/f9bac64c-518e-4630-8889-20e09d5b7a39-kube-api-access-69sgs\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:06 crc kubenswrapper[5052]: I1203 12:41:06.013734 5052 reconciler_common.go:293] "Volume detached for volume \"pod-info\" (UniqueName: \"kubernetes.io/downward-api/f9bac64c-518e-4630-8889-20e09d5b7a39-pod-info\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:06 crc kubenswrapper[5052]: I1203 12:41:06.013745 5052 reconciler_common.go:293] "Volume detached for volume \"plugins-conf\" (UniqueName: \"kubernetes.io/configmap/f9bac64c-518e-4630-8889-20e09d5b7a39-plugins-conf\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:06 crc kubenswrapper[5052]: I1203 12:41:06.013756 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/f9bac64c-518e-4630-8889-20e09d5b7a39-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:06 crc kubenswrapper[5052]: I1203 12:41:06.013770 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/fdeb105e-9fbc-44bb-ba15-bc81c7275db1-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:06 crc kubenswrapper[5052]: I1203 12:41:06.013783 5052 reconciler_common.go:293] "Volume detached for volume \"rabbitmq-confd\" (UniqueName: \"kubernetes.io/projected/f9bac64c-518e-4630-8889-20e09d5b7a39-rabbitmq-confd\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:06 crc kubenswrapper[5052]: I1203 12:41:06.013797 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/fdeb105e-9fbc-44bb-ba15-bc81c7275db1-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:06 crc kubenswrapper[5052]: I1203 12:41:06.013810 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/fdeb105e-9fbc-44bb-ba15-bc81c7275db1-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:06 crc kubenswrapper[5052]: I1203 12:41:06.013821 5052 reconciler_common.go:293] "Volume detached for volume \"rabbitmq-plugins\" (UniqueName: \"kubernetes.io/empty-dir/f9bac64c-518e-4630-8889-20e09d5b7a39-rabbitmq-plugins\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:06 crc kubenswrapper[5052]: I1203 12:41:06.013833 5052 reconciler_common.go:293] "Volume detached for volume \"rabbitmq-confd\" (UniqueName: \"kubernetes.io/projected/d82ef8ea-0c3b-4d89-9c78-22095940f4da-rabbitmq-confd\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:06 crc kubenswrapper[5052]: I1203 12:41:06.013846 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-x4vkn\" (UniqueName: \"kubernetes.io/projected/fdeb105e-9fbc-44bb-ba15-bc81c7275db1-kube-api-access-x4vkn\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:06 crc kubenswrapper[5052]: I1203 12:41:06.013857 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/fdeb105e-9fbc-44bb-ba15-bc81c7275db1-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:06 crc kubenswrapper[5052]: I1203 12:41:06.013869 5052 reconciler_common.go:293] "Volume detached for volume \"server-conf\" (UniqueName: \"kubernetes.io/configmap/f9bac64c-518e-4630-8889-20e09d5b7a39-server-conf\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:06 crc kubenswrapper[5052]: I1203 12:41:06.013929 5052 reconciler_common.go:293] "Volume detached for volume \"rabbitmq-tls\" (UniqueName: \"kubernetes.io/projected/f9bac64c-518e-4630-8889-20e09d5b7a39-rabbitmq-tls\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:06 crc kubenswrapper[5052]: I1203 12:41:06.013944 5052 reconciler_common.go:293] "Volume detached for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/fdeb105e-9fbc-44bb-ba15-bc81c7275db1-fernet-keys\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:06 crc kubenswrapper[5052]: I1203 12:41:06.032247 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage04-crc" (UniqueName: "kubernetes.io/local-volume/local-storage04-crc") on node "crc" Dec 03 12:41:06 crc kubenswrapper[5052]: I1203 12:41:06.116076 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage04-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage04-crc\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:06 crc kubenswrapper[5052]: I1203 12:41:06.405259 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-server-0" event={"ID":"d82ef8ea-0c3b-4d89-9c78-22095940f4da","Type":"ContainerDied","Data":"5c73705bdec2516967e4b29b1ee550293c51c115bca310b791b246711b0cea24"} Dec 03 12:41:06 crc kubenswrapper[5052]: I1203 12:41:06.405505 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:41:06 crc kubenswrapper[5052]: I1203 12:41:06.405557 5052 scope.go:117] "RemoveContainer" containerID="ed184550d5a8d5611c9220d80018d6040a97d60ffbd768b55b87c5964020d439" Dec 03 12:41:06 crc kubenswrapper[5052]: I1203 12:41:06.415123 5052 generic.go:334] "Generic (PLEG): container finished" podID="65cb22ea-3a9c-4f99-9128-d3c7681c3173" containerID="b7bf0dcb2de110437bf240dda6407da49398f97022270b268197a8605493c273" exitCode=0 Dec 03 12:41:06 crc kubenswrapper[5052]: I1203 12:41:06.415189 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"65cb22ea-3a9c-4f99-9128-d3c7681c3173","Type":"ContainerDied","Data":"b7bf0dcb2de110437bf240dda6407da49398f97022270b268197a8605493c273"} Dec 03 12:41:06 crc kubenswrapper[5052]: I1203 12:41:06.416851 5052 generic.go:334] "Generic (PLEG): container finished" podID="ddfa4a3f-b868-49b7-abec-62cb3ffde2bd" containerID="ef78ab4daf6f2994bb54c1f8844ff1255eef71866c6e7be47ed5d4c7ef1d996d" exitCode=0 Dec 03 12:41:06 crc kubenswrapper[5052]: I1203 12:41:06.416900 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-7d74684fd-pqkf5" event={"ID":"ddfa4a3f-b868-49b7-abec-62cb3ffde2bd","Type":"ContainerDied","Data":"ef78ab4daf6f2994bb54c1f8844ff1255eef71866c6e7be47ed5d4c7ef1d996d"} Dec 03 12:41:06 crc kubenswrapper[5052]: I1203 12:41:06.417963 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-66f85f5bfb-vln92" event={"ID":"fdeb105e-9fbc-44bb-ba15-bc81c7275db1","Type":"ContainerDied","Data":"1c38b16a3c4c2e0c47a3b8e80727876a669b7b76c048e848f87a8d9ee3611ea9"} Dec 03 12:41:06 crc kubenswrapper[5052]: I1203 12:41:06.418038 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-66f85f5bfb-vln92" Dec 03 12:41:06 crc kubenswrapper[5052]: I1203 12:41:06.426666 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" event={"ID":"f9bac64c-518e-4630-8889-20e09d5b7a39","Type":"ContainerDied","Data":"a2b1a35a346db4af01b43450f89d01305b46c2cd55396705ef6ff7ffa849bc0a"} Dec 03 12:41:06 crc kubenswrapper[5052]: I1203 12:41:06.426772 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:41:06 crc kubenswrapper[5052]: I1203 12:41:06.433721 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_ovn-northd-0_ea4cd237-7ec6-49f7-9afd-1ea159b9b23b/ovn-northd/0.log" Dec 03 12:41:06 crc kubenswrapper[5052]: I1203 12:41:06.433802 5052 generic.go:334] "Generic (PLEG): container finished" podID="ea4cd237-7ec6-49f7-9afd-1ea159b9b23b" containerID="b2f117c03e1adda1bc8b58e2d0870125a9b0fd3dffffabf7fb3480969ee908b3" exitCode=139 Dec 03 12:41:06 crc kubenswrapper[5052]: I1203 12:41:06.433902 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovn-northd-0" event={"ID":"ea4cd237-7ec6-49f7-9afd-1ea159b9b23b","Type":"ContainerDied","Data":"b2f117c03e1adda1bc8b58e2d0870125a9b0fd3dffffabf7fb3480969ee908b3"} Dec 03 12:41:06 crc kubenswrapper[5052]: I1203 12:41:06.646359 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/keystone-66f85f5bfb-vln92"] Dec 03 12:41:06 crc kubenswrapper[5052]: I1203 12:41:06.664535 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/keystone-66f85f5bfb-vln92"] Dec 03 12:41:06 crc kubenswrapper[5052]: I1203 12:41:06.665343 5052 scope.go:117] "RemoveContainer" containerID="69ee48f37ae71f152a64da3b3da0915d74d05a1eb1b6c9a0996fd2a43065ecba" Dec 03 12:41:06 crc kubenswrapper[5052]: I1203 12:41:06.670875 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/rabbitmq-server-0"] Dec 03 12:41:06 crc kubenswrapper[5052]: I1203 12:41:06.677252 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/rabbitmq-server-0"] Dec 03 12:41:06 crc kubenswrapper[5052]: I1203 12:41:06.712822 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/rabbitmq-cell1-server-0"] Dec 03 12:41:06 crc kubenswrapper[5052]: I1203 12:41:06.718205 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/rabbitmq-cell1-server-0"] Dec 03 12:41:06 crc kubenswrapper[5052]: E1203 12:41:06.791017 5052 cadvisor_stats_provider.go:516] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf9bac64c_518e_4630_8889_20e09d5b7a39.slice/crio-a2b1a35a346db4af01b43450f89d01305b46c2cd55396705ef6ff7ffa849bc0a\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd82ef8ea_0c3b_4d89_9c78_22095940f4da.slice\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf9bac64c_518e_4630_8889_20e09d5b7a39.slice\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfdeb105e_9fbc_44bb_ba15_bc81c7275db1.slice/crio-1c38b16a3c4c2e0c47a3b8e80727876a669b7b76c048e848f87a8d9ee3611ea9\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0556e9f0_e431_4193_9822_6079cb8a1e44.slice/crio-382891affaf25ff9b098ddaf284f3fa5667c14b93f3487974c23c2cdae354300.scope\": RecentStats: unable to find data in memory cache]" Dec 03 12:41:06 crc kubenswrapper[5052]: E1203 12:41:06.831027 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:41:06 crc kubenswrapper[5052]: E1203 12:41:06.831116 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/eb8b91d5-6c99-45ba-8d33-da99f1dd496c-operator-scripts podName:eb8b91d5-6c99-45ba-8d33-da99f1dd496c nodeName:}" failed. No retries permitted until 2025-12-03 12:41:10.831091308 +0000 UTC m=+4861.419333072 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/eb8b91d5-6c99-45ba-8d33-da99f1dd496c-operator-scripts") pod "placement7f30-account-delete-vld4v" (UID: "eb8b91d5-6c99-45ba-8d33-da99f1dd496c") : configmap "openstack-scripts" not found Dec 03 12:41:06 crc kubenswrapper[5052]: I1203 12:41:06.882216 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_ovn-northd-0_ea4cd237-7ec6-49f7-9afd-1ea159b9b23b/ovn-northd/0.log" Dec 03 12:41:06 crc kubenswrapper[5052]: I1203 12:41:06.882309 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:41:06 crc kubenswrapper[5052]: I1203 12:41:06.896429 5052 scope.go:117] "RemoveContainer" containerID="303e4e73500eec972e0823935bb4cf2cb918b5d2b159c46edd8b37894c6d37f1" Dec 03 12:41:06 crc kubenswrapper[5052]: I1203 12:41:06.896653 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:41:06 crc kubenswrapper[5052]: I1203 12:41:06.900800 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:41:06 crc kubenswrapper[5052]: I1203 12:41:06.935263 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5pfwg\" (UniqueName: \"kubernetes.io/projected/ea4cd237-7ec6-49f7-9afd-1ea159b9b23b-kube-api-access-5pfwg\") pod \"ea4cd237-7ec6-49f7-9afd-1ea159b9b23b\" (UID: \"ea4cd237-7ec6-49f7-9afd-1ea159b9b23b\") " Dec 03 12:41:06 crc kubenswrapper[5052]: I1203 12:41:06.935345 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bshs9\" (UniqueName: \"kubernetes.io/projected/65cb22ea-3a9c-4f99-9128-d3c7681c3173-kube-api-access-bshs9\") pod \"65cb22ea-3a9c-4f99-9128-d3c7681c3173\" (UID: \"65cb22ea-3a9c-4f99-9128-d3c7681c3173\") " Dec 03 12:41:06 crc kubenswrapper[5052]: I1203 12:41:06.935380 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/65cb22ea-3a9c-4f99-9128-d3c7681c3173-sg-core-conf-yaml\") pod \"65cb22ea-3a9c-4f99-9128-d3c7681c3173\" (UID: \"65cb22ea-3a9c-4f99-9128-d3c7681c3173\") " Dec 03 12:41:06 crc kubenswrapper[5052]: I1203 12:41:06.935412 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-prg4s\" (UniqueName: \"kubernetes.io/projected/0556e9f0-e431-4193-9822-6079cb8a1e44-kube-api-access-prg4s\") pod \"0556e9f0-e431-4193-9822-6079cb8a1e44\" (UID: \"0556e9f0-e431-4193-9822-6079cb8a1e44\") " Dec 03 12:41:06 crc kubenswrapper[5052]: I1203 12:41:06.935475 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0556e9f0-e431-4193-9822-6079cb8a1e44-combined-ca-bundle\") pod \"0556e9f0-e431-4193-9822-6079cb8a1e44\" (UID: \"0556e9f0-e431-4193-9822-6079cb8a1e44\") " Dec 03 12:41:06 crc kubenswrapper[5052]: I1203 12:41:06.935536 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/65cb22ea-3a9c-4f99-9128-d3c7681c3173-log-httpd\") pod \"65cb22ea-3a9c-4f99-9128-d3c7681c3173\" (UID: \"65cb22ea-3a9c-4f99-9128-d3c7681c3173\") " Dec 03 12:41:06 crc kubenswrapper[5052]: I1203 12:41:06.935565 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovn-northd-tls-certs\" (UniqueName: \"kubernetes.io/secret/ea4cd237-7ec6-49f7-9afd-1ea159b9b23b-ovn-northd-tls-certs\") pod \"ea4cd237-7ec6-49f7-9afd-1ea159b9b23b\" (UID: \"ea4cd237-7ec6-49f7-9afd-1ea159b9b23b\") " Dec 03 12:41:06 crc kubenswrapper[5052]: I1203 12:41:06.935591 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/65cb22ea-3a9c-4f99-9128-d3c7681c3173-scripts\") pod \"65cb22ea-3a9c-4f99-9128-d3c7681c3173\" (UID: \"65cb22ea-3a9c-4f99-9128-d3c7681c3173\") " Dec 03 12:41:06 crc kubenswrapper[5052]: I1203 12:41:06.935640 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/0556e9f0-e431-4193-9822-6079cb8a1e44-config-data\") pod \"0556e9f0-e431-4193-9822-6079cb8a1e44\" (UID: \"0556e9f0-e431-4193-9822-6079cb8a1e44\") " Dec 03 12:41:06 crc kubenswrapper[5052]: I1203 12:41:06.935674 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/65cb22ea-3a9c-4f99-9128-d3c7681c3173-ceilometer-tls-certs\") pod \"65cb22ea-3a9c-4f99-9128-d3c7681c3173\" (UID: \"65cb22ea-3a9c-4f99-9128-d3c7681c3173\") " Dec 03 12:41:06 crc kubenswrapper[5052]: I1203 12:41:06.935698 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/65cb22ea-3a9c-4f99-9128-d3c7681c3173-run-httpd\") pod \"65cb22ea-3a9c-4f99-9128-d3c7681c3173\" (UID: \"65cb22ea-3a9c-4f99-9128-d3c7681c3173\") " Dec 03 12:41:06 crc kubenswrapper[5052]: I1203 12:41:06.935791 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovn-rundir\" (UniqueName: \"kubernetes.io/empty-dir/ea4cd237-7ec6-49f7-9afd-1ea159b9b23b-ovn-rundir\") pod \"ea4cd237-7ec6-49f7-9afd-1ea159b9b23b\" (UID: \"ea4cd237-7ec6-49f7-9afd-1ea159b9b23b\") " Dec 03 12:41:06 crc kubenswrapper[5052]: I1203 12:41:06.935820 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/65cb22ea-3a9c-4f99-9128-d3c7681c3173-config-data\") pod \"65cb22ea-3a9c-4f99-9128-d3c7681c3173\" (UID: \"65cb22ea-3a9c-4f99-9128-d3c7681c3173\") " Dec 03 12:41:06 crc kubenswrapper[5052]: I1203 12:41:06.935849 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ea4cd237-7ec6-49f7-9afd-1ea159b9b23b-combined-ca-bundle\") pod \"ea4cd237-7ec6-49f7-9afd-1ea159b9b23b\" (UID: \"ea4cd237-7ec6-49f7-9afd-1ea159b9b23b\") " Dec 03 12:41:06 crc kubenswrapper[5052]: I1203 12:41:06.935897 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/ea4cd237-7ec6-49f7-9afd-1ea159b9b23b-metrics-certs-tls-certs\") pod \"ea4cd237-7ec6-49f7-9afd-1ea159b9b23b\" (UID: \"ea4cd237-7ec6-49f7-9afd-1ea159b9b23b\") " Dec 03 12:41:06 crc kubenswrapper[5052]: I1203 12:41:06.935961 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ea4cd237-7ec6-49f7-9afd-1ea159b9b23b-config\") pod \"ea4cd237-7ec6-49f7-9afd-1ea159b9b23b\" (UID: \"ea4cd237-7ec6-49f7-9afd-1ea159b9b23b\") " Dec 03 12:41:06 crc kubenswrapper[5052]: I1203 12:41:06.936031 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/65cb22ea-3a9c-4f99-9128-d3c7681c3173-combined-ca-bundle\") pod \"65cb22ea-3a9c-4f99-9128-d3c7681c3173\" (UID: \"65cb22ea-3a9c-4f99-9128-d3c7681c3173\") " Dec 03 12:41:06 crc kubenswrapper[5052]: I1203 12:41:06.936088 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/ea4cd237-7ec6-49f7-9afd-1ea159b9b23b-scripts\") pod \"ea4cd237-7ec6-49f7-9afd-1ea159b9b23b\" (UID: \"ea4cd237-7ec6-49f7-9afd-1ea159b9b23b\") " Dec 03 12:41:06 crc kubenswrapper[5052]: I1203 12:41:06.936576 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/65cb22ea-3a9c-4f99-9128-d3c7681c3173-run-httpd" (OuterVolumeSpecName: "run-httpd") pod "65cb22ea-3a9c-4f99-9128-d3c7681c3173" (UID: "65cb22ea-3a9c-4f99-9128-d3c7681c3173"). InnerVolumeSpecName "run-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:41:06 crc kubenswrapper[5052]: I1203 12:41:06.937096 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ea4cd237-7ec6-49f7-9afd-1ea159b9b23b-scripts" (OuterVolumeSpecName: "scripts") pod "ea4cd237-7ec6-49f7-9afd-1ea159b9b23b" (UID: "ea4cd237-7ec6-49f7-9afd-1ea159b9b23b"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:41:06 crc kubenswrapper[5052]: I1203 12:41:06.937460 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea4cd237-7ec6-49f7-9afd-1ea159b9b23b-ovn-rundir" (OuterVolumeSpecName: "ovn-rundir") pod "ea4cd237-7ec6-49f7-9afd-1ea159b9b23b" (UID: "ea4cd237-7ec6-49f7-9afd-1ea159b9b23b"). InnerVolumeSpecName "ovn-rundir". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:41:06 crc kubenswrapper[5052]: I1203 12:41:06.945117 5052 scope.go:117] "RemoveContainer" containerID="ed967e0e50724d3854ca971b7a125ee6c019b59d594a733d91f837aeb86bc924" Dec 03 12:41:06 crc kubenswrapper[5052]: I1203 12:41:06.946198 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ea4cd237-7ec6-49f7-9afd-1ea159b9b23b-config" (OuterVolumeSpecName: "config") pod "ea4cd237-7ec6-49f7-9afd-1ea159b9b23b" (UID: "ea4cd237-7ec6-49f7-9afd-1ea159b9b23b"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:41:06 crc kubenswrapper[5052]: I1203 12:41:06.954405 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0556e9f0-e431-4193-9822-6079cb8a1e44-kube-api-access-prg4s" (OuterVolumeSpecName: "kube-api-access-prg4s") pod "0556e9f0-e431-4193-9822-6079cb8a1e44" (UID: "0556e9f0-e431-4193-9822-6079cb8a1e44"). InnerVolumeSpecName "kube-api-access-prg4s". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:41:06 crc kubenswrapper[5052]: I1203 12:41:06.954554 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/65cb22ea-3a9c-4f99-9128-d3c7681c3173-kube-api-access-bshs9" (OuterVolumeSpecName: "kube-api-access-bshs9") pod "65cb22ea-3a9c-4f99-9128-d3c7681c3173" (UID: "65cb22ea-3a9c-4f99-9128-d3c7681c3173"). InnerVolumeSpecName "kube-api-access-bshs9". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:41:06 crc kubenswrapper[5052]: I1203 12:41:06.955084 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/65cb22ea-3a9c-4f99-9128-d3c7681c3173-log-httpd" (OuterVolumeSpecName: "log-httpd") pod "65cb22ea-3a9c-4f99-9128-d3c7681c3173" (UID: "65cb22ea-3a9c-4f99-9128-d3c7681c3173"). InnerVolumeSpecName "log-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:41:06 crc kubenswrapper[5052]: I1203 12:41:06.955984 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/65cb22ea-3a9c-4f99-9128-d3c7681c3173-scripts" (OuterVolumeSpecName: "scripts") pod "65cb22ea-3a9c-4f99-9128-d3c7681c3173" (UID: "65cb22ea-3a9c-4f99-9128-d3c7681c3173"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:41:06 crc kubenswrapper[5052]: I1203 12:41:06.978603 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0556e9f0-e431-4193-9822-6079cb8a1e44-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "0556e9f0-e431-4193-9822-6079cb8a1e44" (UID: "0556e9f0-e431-4193-9822-6079cb8a1e44"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:41:06 crc kubenswrapper[5052]: I1203 12:41:06.989011 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ea4cd237-7ec6-49f7-9afd-1ea159b9b23b-kube-api-access-5pfwg" (OuterVolumeSpecName: "kube-api-access-5pfwg") pod "ea4cd237-7ec6-49f7-9afd-1ea159b9b23b" (UID: "ea4cd237-7ec6-49f7-9afd-1ea159b9b23b"). InnerVolumeSpecName "kube-api-access-5pfwg". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:41:06 crc kubenswrapper[5052]: I1203 12:41:06.989118 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ea4cd237-7ec6-49f7-9afd-1ea159b9b23b-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "ea4cd237-7ec6-49f7-9afd-1ea159b9b23b" (UID: "ea4cd237-7ec6-49f7-9afd-1ea159b9b23b"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:41:06 crc kubenswrapper[5052]: I1203 12:41:06.994357 5052 scope.go:117] "RemoveContainer" containerID="806aad7eebb3067b147df9728513f4f5c522d884aded6480c84b0af0d1a1afd4" Dec 03 12:41:07 crc kubenswrapper[5052]: I1203 12:41:07.020774 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0556e9f0-e431-4193-9822-6079cb8a1e44-config-data" (OuterVolumeSpecName: "config-data") pod "0556e9f0-e431-4193-9822-6079cb8a1e44" (UID: "0556e9f0-e431-4193-9822-6079cb8a1e44"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:41:07 crc kubenswrapper[5052]: I1203 12:41:07.030120 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/65cb22ea-3a9c-4f99-9128-d3c7681c3173-ceilometer-tls-certs" (OuterVolumeSpecName: "ceilometer-tls-certs") pod "65cb22ea-3a9c-4f99-9128-d3c7681c3173" (UID: "65cb22ea-3a9c-4f99-9128-d3c7681c3173"). InnerVolumeSpecName "ceilometer-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:41:07 crc kubenswrapper[5052]: I1203 12:41:07.035618 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/65cb22ea-3a9c-4f99-9128-d3c7681c3173-sg-core-conf-yaml" (OuterVolumeSpecName: "sg-core-conf-yaml") pod "65cb22ea-3a9c-4f99-9128-d3c7681c3173" (UID: "65cb22ea-3a9c-4f99-9128-d3c7681c3173"). InnerVolumeSpecName "sg-core-conf-yaml". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:41:07 crc kubenswrapper[5052]: I1203 12:41:07.038649 5052 reconciler_common.go:293] "Volume detached for volume \"ovn-rundir\" (UniqueName: \"kubernetes.io/empty-dir/ea4cd237-7ec6-49f7-9afd-1ea159b9b23b-ovn-rundir\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:07 crc kubenswrapper[5052]: I1203 12:41:07.038688 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ea4cd237-7ec6-49f7-9afd-1ea159b9b23b-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:07 crc kubenswrapper[5052]: I1203 12:41:07.038701 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ea4cd237-7ec6-49f7-9afd-1ea159b9b23b-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:07 crc kubenswrapper[5052]: I1203 12:41:07.038747 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/ea4cd237-7ec6-49f7-9afd-1ea159b9b23b-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:07 crc kubenswrapper[5052]: I1203 12:41:07.038761 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-5pfwg\" (UniqueName: \"kubernetes.io/projected/ea4cd237-7ec6-49f7-9afd-1ea159b9b23b-kube-api-access-5pfwg\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:07 crc kubenswrapper[5052]: I1203 12:41:07.038774 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-bshs9\" (UniqueName: \"kubernetes.io/projected/65cb22ea-3a9c-4f99-9128-d3c7681c3173-kube-api-access-bshs9\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:07 crc kubenswrapper[5052]: I1203 12:41:07.038787 5052 reconciler_common.go:293] "Volume detached for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/65cb22ea-3a9c-4f99-9128-d3c7681c3173-sg-core-conf-yaml\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:07 crc kubenswrapper[5052]: I1203 12:41:07.038798 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-prg4s\" (UniqueName: \"kubernetes.io/projected/0556e9f0-e431-4193-9822-6079cb8a1e44-kube-api-access-prg4s\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:07 crc kubenswrapper[5052]: I1203 12:41:07.038809 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0556e9f0-e431-4193-9822-6079cb8a1e44-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:07 crc kubenswrapper[5052]: I1203 12:41:07.038819 5052 reconciler_common.go:293] "Volume detached for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/65cb22ea-3a9c-4f99-9128-d3c7681c3173-log-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:07 crc kubenswrapper[5052]: I1203 12:41:07.038830 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/65cb22ea-3a9c-4f99-9128-d3c7681c3173-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:07 crc kubenswrapper[5052]: I1203 12:41:07.038841 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/0556e9f0-e431-4193-9822-6079cb8a1e44-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:07 crc kubenswrapper[5052]: I1203 12:41:07.038851 5052 reconciler_common.go:293] "Volume detached for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/65cb22ea-3a9c-4f99-9128-d3c7681c3173-run-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:07 crc kubenswrapper[5052]: I1203 12:41:07.038861 5052 reconciler_common.go:293] "Volume detached for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/65cb22ea-3a9c-4f99-9128-d3c7681c3173-ceilometer-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:07 crc kubenswrapper[5052]: I1203 12:41:07.050340 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ea4cd237-7ec6-49f7-9afd-1ea159b9b23b-metrics-certs-tls-certs" (OuterVolumeSpecName: "metrics-certs-tls-certs") pod "ea4cd237-7ec6-49f7-9afd-1ea159b9b23b" (UID: "ea4cd237-7ec6-49f7-9afd-1ea159b9b23b"). InnerVolumeSpecName "metrics-certs-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:41:07 crc kubenswrapper[5052]: I1203 12:41:07.050560 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-7d74684fd-pqkf5" Dec 03 12:41:07 crc kubenswrapper[5052]: I1203 12:41:07.065392 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/65cb22ea-3a9c-4f99-9128-d3c7681c3173-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "65cb22ea-3a9c-4f99-9128-d3c7681c3173" (UID: "65cb22ea-3a9c-4f99-9128-d3c7681c3173"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:41:07 crc kubenswrapper[5052]: I1203 12:41:07.078431 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ea4cd237-7ec6-49f7-9afd-1ea159b9b23b-ovn-northd-tls-certs" (OuterVolumeSpecName: "ovn-northd-tls-certs") pod "ea4cd237-7ec6-49f7-9afd-1ea159b9b23b" (UID: "ea4cd237-7ec6-49f7-9afd-1ea159b9b23b"). InnerVolumeSpecName "ovn-northd-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:41:07 crc kubenswrapper[5052]: I1203 12:41:07.114903 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/65cb22ea-3a9c-4f99-9128-d3c7681c3173-config-data" (OuterVolumeSpecName: "config-data") pod "65cb22ea-3a9c-4f99-9128-d3c7681c3173" (UID: "65cb22ea-3a9c-4f99-9128-d3c7681c3173"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:41:07 crc kubenswrapper[5052]: I1203 12:41:07.139611 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ddfa4a3f-b868-49b7-abec-62cb3ffde2bd-combined-ca-bundle\") pod \"ddfa4a3f-b868-49b7-abec-62cb3ffde2bd\" (UID: \"ddfa4a3f-b868-49b7-abec-62cb3ffde2bd\") " Dec 03 12:41:07 crc kubenswrapper[5052]: I1203 12:41:07.139673 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/ddfa4a3f-b868-49b7-abec-62cb3ffde2bd-public-tls-certs\") pod \"ddfa4a3f-b868-49b7-abec-62cb3ffde2bd\" (UID: \"ddfa4a3f-b868-49b7-abec-62cb3ffde2bd\") " Dec 03 12:41:07 crc kubenswrapper[5052]: I1203 12:41:07.139696 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/ddfa4a3f-b868-49b7-abec-62cb3ffde2bd-config\") pod \"ddfa4a3f-b868-49b7-abec-62cb3ffde2bd\" (UID: \"ddfa4a3f-b868-49b7-abec-62cb3ffde2bd\") " Dec 03 12:41:07 crc kubenswrapper[5052]: I1203 12:41:07.139721 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/ddfa4a3f-b868-49b7-abec-62cb3ffde2bd-httpd-config\") pod \"ddfa4a3f-b868-49b7-abec-62cb3ffde2bd\" (UID: \"ddfa4a3f-b868-49b7-abec-62cb3ffde2bd\") " Dec 03 12:41:07 crc kubenswrapper[5052]: I1203 12:41:07.139739 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sfvbz\" (UniqueName: \"kubernetes.io/projected/ddfa4a3f-b868-49b7-abec-62cb3ffde2bd-kube-api-access-sfvbz\") pod \"ddfa4a3f-b868-49b7-abec-62cb3ffde2bd\" (UID: \"ddfa4a3f-b868-49b7-abec-62cb3ffde2bd\") " Dec 03 12:41:07 crc kubenswrapper[5052]: I1203 12:41:07.139759 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/ddfa4a3f-b868-49b7-abec-62cb3ffde2bd-ovndb-tls-certs\") pod \"ddfa4a3f-b868-49b7-abec-62cb3ffde2bd\" (UID: \"ddfa4a3f-b868-49b7-abec-62cb3ffde2bd\") " Dec 03 12:41:07 crc kubenswrapper[5052]: I1203 12:41:07.139774 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/ddfa4a3f-b868-49b7-abec-62cb3ffde2bd-internal-tls-certs\") pod \"ddfa4a3f-b868-49b7-abec-62cb3ffde2bd\" (UID: \"ddfa4a3f-b868-49b7-abec-62cb3ffde2bd\") " Dec 03 12:41:07 crc kubenswrapper[5052]: I1203 12:41:07.140116 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/65cb22ea-3a9c-4f99-9128-d3c7681c3173-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:07 crc kubenswrapper[5052]: I1203 12:41:07.140128 5052 reconciler_common.go:293] "Volume detached for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/ea4cd237-7ec6-49f7-9afd-1ea159b9b23b-metrics-certs-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:07 crc kubenswrapper[5052]: I1203 12:41:07.140139 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/65cb22ea-3a9c-4f99-9128-d3c7681c3173-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:07 crc kubenswrapper[5052]: I1203 12:41:07.140148 5052 reconciler_common.go:293] "Volume detached for volume \"ovn-northd-tls-certs\" (UniqueName: \"kubernetes.io/secret/ea4cd237-7ec6-49f7-9afd-1ea159b9b23b-ovn-northd-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:07 crc kubenswrapper[5052]: I1203 12:41:07.143477 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ddfa4a3f-b868-49b7-abec-62cb3ffde2bd-httpd-config" (OuterVolumeSpecName: "httpd-config") pod "ddfa4a3f-b868-49b7-abec-62cb3ffde2bd" (UID: "ddfa4a3f-b868-49b7-abec-62cb3ffde2bd"). InnerVolumeSpecName "httpd-config". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:41:07 crc kubenswrapper[5052]: I1203 12:41:07.143685 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ddfa4a3f-b868-49b7-abec-62cb3ffde2bd-kube-api-access-sfvbz" (OuterVolumeSpecName: "kube-api-access-sfvbz") pod "ddfa4a3f-b868-49b7-abec-62cb3ffde2bd" (UID: "ddfa4a3f-b868-49b7-abec-62cb3ffde2bd"). InnerVolumeSpecName "kube-api-access-sfvbz". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:41:07 crc kubenswrapper[5052]: I1203 12:41:07.177470 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ddfa4a3f-b868-49b7-abec-62cb3ffde2bd-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "ddfa4a3f-b868-49b7-abec-62cb3ffde2bd" (UID: "ddfa4a3f-b868-49b7-abec-62cb3ffde2bd"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:41:07 crc kubenswrapper[5052]: I1203 12:41:07.178379 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ddfa4a3f-b868-49b7-abec-62cb3ffde2bd-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "ddfa4a3f-b868-49b7-abec-62cb3ffde2bd" (UID: "ddfa4a3f-b868-49b7-abec-62cb3ffde2bd"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:41:07 crc kubenswrapper[5052]: I1203 12:41:07.180747 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ddfa4a3f-b868-49b7-abec-62cb3ffde2bd-config" (OuterVolumeSpecName: "config") pod "ddfa4a3f-b868-49b7-abec-62cb3ffde2bd" (UID: "ddfa4a3f-b868-49b7-abec-62cb3ffde2bd"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:41:07 crc kubenswrapper[5052]: I1203 12:41:07.185310 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ddfa4a3f-b868-49b7-abec-62cb3ffde2bd-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "ddfa4a3f-b868-49b7-abec-62cb3ffde2bd" (UID: "ddfa4a3f-b868-49b7-abec-62cb3ffde2bd"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:41:07 crc kubenswrapper[5052]: I1203 12:41:07.196534 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:41:07 crc kubenswrapper[5052]: I1203 12:41:07.200904 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ddfa4a3f-b868-49b7-abec-62cb3ffde2bd-ovndb-tls-certs" (OuterVolumeSpecName: "ovndb-tls-certs") pod "ddfa4a3f-b868-49b7-abec-62cb3ffde2bd" (UID: "ddfa4a3f-b868-49b7-abec-62cb3ffde2bd"). InnerVolumeSpecName "ovndb-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:41:07 crc kubenswrapper[5052]: I1203 12:41:07.241773 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/9e3bfbf3-a39c-48d2-bb1d-062c90c0ef10-config-data\") pod \"9e3bfbf3-a39c-48d2-bb1d-062c90c0ef10\" (UID: \"9e3bfbf3-a39c-48d2-bb1d-062c90c0ef10\") " Dec 03 12:41:07 crc kubenswrapper[5052]: I1203 12:41:07.241865 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9e3bfbf3-a39c-48d2-bb1d-062c90c0ef10-combined-ca-bundle\") pod \"9e3bfbf3-a39c-48d2-bb1d-062c90c0ef10\" (UID: \"9e3bfbf3-a39c-48d2-bb1d-062c90c0ef10\") " Dec 03 12:41:07 crc kubenswrapper[5052]: I1203 12:41:07.241899 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wj2jk\" (UniqueName: \"kubernetes.io/projected/9e3bfbf3-a39c-48d2-bb1d-062c90c0ef10-kube-api-access-wj2jk\") pod \"9e3bfbf3-a39c-48d2-bb1d-062c90c0ef10\" (UID: \"9e3bfbf3-a39c-48d2-bb1d-062c90c0ef10\") " Dec 03 12:41:07 crc kubenswrapper[5052]: I1203 12:41:07.242340 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ddfa4a3f-b868-49b7-abec-62cb3ffde2bd-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:07 crc kubenswrapper[5052]: I1203 12:41:07.242362 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/ddfa4a3f-b868-49b7-abec-62cb3ffde2bd-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:07 crc kubenswrapper[5052]: I1203 12:41:07.242371 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/ddfa4a3f-b868-49b7-abec-62cb3ffde2bd-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:07 crc kubenswrapper[5052]: I1203 12:41:07.242380 5052 reconciler_common.go:293] "Volume detached for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/ddfa4a3f-b868-49b7-abec-62cb3ffde2bd-httpd-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:07 crc kubenswrapper[5052]: I1203 12:41:07.242395 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-sfvbz\" (UniqueName: \"kubernetes.io/projected/ddfa4a3f-b868-49b7-abec-62cb3ffde2bd-kube-api-access-sfvbz\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:07 crc kubenswrapper[5052]: I1203 12:41:07.242412 5052 reconciler_common.go:293] "Volume detached for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/ddfa4a3f-b868-49b7-abec-62cb3ffde2bd-ovndb-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:07 crc kubenswrapper[5052]: I1203 12:41:07.242424 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/ddfa4a3f-b868-49b7-abec-62cb3ffde2bd-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:07 crc kubenswrapper[5052]: I1203 12:41:07.246301 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9e3bfbf3-a39c-48d2-bb1d-062c90c0ef10-kube-api-access-wj2jk" (OuterVolumeSpecName: "kube-api-access-wj2jk") pod "9e3bfbf3-a39c-48d2-bb1d-062c90c0ef10" (UID: "9e3bfbf3-a39c-48d2-bb1d-062c90c0ef10"). InnerVolumeSpecName "kube-api-access-wj2jk". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:41:07 crc kubenswrapper[5052]: I1203 12:41:07.262043 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9e3bfbf3-a39c-48d2-bb1d-062c90c0ef10-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "9e3bfbf3-a39c-48d2-bb1d-062c90c0ef10" (UID: "9e3bfbf3-a39c-48d2-bb1d-062c90c0ef10"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:41:07 crc kubenswrapper[5052]: I1203 12:41:07.264410 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9e3bfbf3-a39c-48d2-bb1d-062c90c0ef10-config-data" (OuterVolumeSpecName: "config-data") pod "9e3bfbf3-a39c-48d2-bb1d-062c90c0ef10" (UID: "9e3bfbf3-a39c-48d2-bb1d-062c90c0ef10"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:41:07 crc kubenswrapper[5052]: I1203 12:41:07.344323 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9e3bfbf3-a39c-48d2-bb1d-062c90c0ef10-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:07 crc kubenswrapper[5052]: I1203 12:41:07.345436 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-wj2jk\" (UniqueName: \"kubernetes.io/projected/9e3bfbf3-a39c-48d2-bb1d-062c90c0ef10-kube-api-access-wj2jk\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:07 crc kubenswrapper[5052]: I1203 12:41:07.345543 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/9e3bfbf3-a39c-48d2-bb1d-062c90c0ef10-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:07 crc kubenswrapper[5052]: I1203 12:41:07.459851 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_ovn-northd-0_ea4cd237-7ec6-49f7-9afd-1ea159b9b23b/ovn-northd/0.log" Dec 03 12:41:07 crc kubenswrapper[5052]: I1203 12:41:07.460026 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovn-northd-0" event={"ID":"ea4cd237-7ec6-49f7-9afd-1ea159b9b23b","Type":"ContainerDied","Data":"bb3afddf44a1f77efa151215ac24c463f8c74211df666d92c6489ae6bd78979b"} Dec 03 12:41:07 crc kubenswrapper[5052]: I1203 12:41:07.460079 5052 scope.go:117] "RemoveContainer" containerID="2e4992cc2e8905b7fe793a8b028dc60f77b2fc53b9e9aa87bd2e3fbe6b4f4d4c" Dec 03 12:41:07 crc kubenswrapper[5052]: I1203 12:41:07.460743 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:41:07 crc kubenswrapper[5052]: I1203 12:41:07.463176 5052 generic.go:334] "Generic (PLEG): container finished" podID="9e3bfbf3-a39c-48d2-bb1d-062c90c0ef10" containerID="b1ff1f57b80e78690f7e46060a438671dfcfacd08f98a788c2bd60d1eb2bbdf1" exitCode=0 Dec 03 12:41:07 crc kubenswrapper[5052]: I1203 12:41:07.463223 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"9e3bfbf3-a39c-48d2-bb1d-062c90c0ef10","Type":"ContainerDied","Data":"b1ff1f57b80e78690f7e46060a438671dfcfacd08f98a788c2bd60d1eb2bbdf1"} Dec 03 12:41:07 crc kubenswrapper[5052]: I1203 12:41:07.463247 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"9e3bfbf3-a39c-48d2-bb1d-062c90c0ef10","Type":"ContainerDied","Data":"e9d74f51a3c4d43463e19a86dda53575e91a595d1878cc45e8a213c710b19eab"} Dec 03 12:41:07 crc kubenswrapper[5052]: I1203 12:41:07.463279 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:41:07 crc kubenswrapper[5052]: I1203 12:41:07.472961 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"65cb22ea-3a9c-4f99-9128-d3c7681c3173","Type":"ContainerDied","Data":"180a719e6c23d31790bc4ff445ea488d36a3051bd6b6243751d58b4950a06083"} Dec 03 12:41:07 crc kubenswrapper[5052]: I1203 12:41:07.473018 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:41:07 crc kubenswrapper[5052]: I1203 12:41:07.475936 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-7d74684fd-pqkf5" event={"ID":"ddfa4a3f-b868-49b7-abec-62cb3ffde2bd","Type":"ContainerDied","Data":"31851794851d731aa49dc5c7f2d98f48ac2eac1b939d702428bd58b2d363091c"} Dec 03 12:41:07 crc kubenswrapper[5052]: I1203 12:41:07.475978 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-7d74684fd-pqkf5" Dec 03 12:41:07 crc kubenswrapper[5052]: I1203 12:41:07.478617 5052 generic.go:334] "Generic (PLEG): container finished" podID="0556e9f0-e431-4193-9822-6079cb8a1e44" containerID="382891affaf25ff9b098ddaf284f3fa5667c14b93f3487974c23c2cdae354300" exitCode=0 Dec 03 12:41:07 crc kubenswrapper[5052]: I1203 12:41:07.478686 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-conductor-0" event={"ID":"0556e9f0-e431-4193-9822-6079cb8a1e44","Type":"ContainerDied","Data":"382891affaf25ff9b098ddaf284f3fa5667c14b93f3487974c23c2cdae354300"} Dec 03 12:41:07 crc kubenswrapper[5052]: I1203 12:41:07.478695 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:41:07 crc kubenswrapper[5052]: I1203 12:41:07.478715 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-conductor-0" event={"ID":"0556e9f0-e431-4193-9822-6079cb8a1e44","Type":"ContainerDied","Data":"712de07ce5eeaee31029705ab3d33cfdf189affff7d973b2e31342976b5cb79c"} Dec 03 12:41:07 crc kubenswrapper[5052]: I1203 12:41:07.629990 5052 scope.go:117] "RemoveContainer" containerID="b2f117c03e1adda1bc8b58e2d0870125a9b0fd3dffffabf7fb3480969ee908b3" Dec 03 12:41:07 crc kubenswrapper[5052]: I1203 12:41:07.652567 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ovn-northd-0"] Dec 03 12:41:07 crc kubenswrapper[5052]: I1203 12:41:07.662027 5052 scope.go:117] "RemoveContainer" containerID="b1ff1f57b80e78690f7e46060a438671dfcfacd08f98a788c2bd60d1eb2bbdf1" Dec 03 12:41:07 crc kubenswrapper[5052]: I1203 12:41:07.662980 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ovn-northd-0"] Dec 03 12:41:07 crc kubenswrapper[5052]: I1203 12:41:07.699500 5052 scope.go:117] "RemoveContainer" containerID="b1ff1f57b80e78690f7e46060a438671dfcfacd08f98a788c2bd60d1eb2bbdf1" Dec 03 12:41:07 crc kubenswrapper[5052]: E1203 12:41:07.699929 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b1ff1f57b80e78690f7e46060a438671dfcfacd08f98a788c2bd60d1eb2bbdf1\": container with ID starting with b1ff1f57b80e78690f7e46060a438671dfcfacd08f98a788c2bd60d1eb2bbdf1 not found: ID does not exist" containerID="b1ff1f57b80e78690f7e46060a438671dfcfacd08f98a788c2bd60d1eb2bbdf1" Dec 03 12:41:07 crc kubenswrapper[5052]: I1203 12:41:07.699964 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b1ff1f57b80e78690f7e46060a438671dfcfacd08f98a788c2bd60d1eb2bbdf1"} err="failed to get container status \"b1ff1f57b80e78690f7e46060a438671dfcfacd08f98a788c2bd60d1eb2bbdf1\": rpc error: code = NotFound desc = could not find container \"b1ff1f57b80e78690f7e46060a438671dfcfacd08f98a788c2bd60d1eb2bbdf1\": container with ID starting with b1ff1f57b80e78690f7e46060a438671dfcfacd08f98a788c2bd60d1eb2bbdf1 not found: ID does not exist" Dec 03 12:41:07 crc kubenswrapper[5052]: I1203 12:41:07.699995 5052 scope.go:117] "RemoveContainer" containerID="f0b7b9f45f694a8501baf61dbded87068f05ef0463c5c28e5e6db8782cc546cc" Dec 03 12:41:07 crc kubenswrapper[5052]: I1203 12:41:07.703135 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d82ef8ea-0c3b-4d89-9c78-22095940f4da" path="/var/lib/kubelet/pods/d82ef8ea-0c3b-4d89-9c78-22095940f4da/volumes" Dec 03 12:41:07 crc kubenswrapper[5052]: I1203 12:41:07.704076 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ea4cd237-7ec6-49f7-9afd-1ea159b9b23b" path="/var/lib/kubelet/pods/ea4cd237-7ec6-49f7-9afd-1ea159b9b23b/volumes" Dec 03 12:41:07 crc kubenswrapper[5052]: I1203 12:41:07.705731 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f9bac64c-518e-4630-8889-20e09d5b7a39" path="/var/lib/kubelet/pods/f9bac64c-518e-4630-8889-20e09d5b7a39/volumes" Dec 03 12:41:07 crc kubenswrapper[5052]: I1203 12:41:07.706721 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fdeb105e-9fbc-44bb-ba15-bc81c7275db1" path="/var/lib/kubelet/pods/fdeb105e-9fbc-44bb-ba15-bc81c7275db1/volumes" Dec 03 12:41:07 crc kubenswrapper[5052]: I1203 12:41:07.707640 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:41:07 crc kubenswrapper[5052]: I1203 12:41:07.707808 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:41:07 crc kubenswrapper[5052]: I1203 12:41:07.707845 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:41:07 crc kubenswrapper[5052]: I1203 12:41:07.707861 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:41:07 crc kubenswrapper[5052]: I1203 12:41:07.707875 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/neutron-7d74684fd-pqkf5"] Dec 03 12:41:07 crc kubenswrapper[5052]: I1203 12:41:07.712903 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/neutron-7d74684fd-pqkf5"] Dec 03 12:41:07 crc kubenswrapper[5052]: I1203 12:41:07.719364 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-0"] Dec 03 12:41:07 crc kubenswrapper[5052]: I1203 12:41:07.720568 5052 scope.go:117] "RemoveContainer" containerID="1bc0a87b8c7078c46986fb762763f8730d5895f38f884a95a89a15f1527106de" Dec 03 12:41:07 crc kubenswrapper[5052]: I1203 12:41:07.725028 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-0"] Dec 03 12:41:07 crc kubenswrapper[5052]: I1203 12:41:07.739018 5052 scope.go:117] "RemoveContainer" containerID="b7bf0dcb2de110437bf240dda6407da49398f97022270b268197a8605493c273" Dec 03 12:41:07 crc kubenswrapper[5052]: I1203 12:41:07.760218 5052 scope.go:117] "RemoveContainer" containerID="d970c5d7874c5afe76962513bf4d336686b29e37d400962b8518fcae8ce933fc" Dec 03 12:41:07 crc kubenswrapper[5052]: I1203 12:41:07.780409 5052 scope.go:117] "RemoveContainer" containerID="8ed93a1c3257ab19479e68e87c18d74af55fe2f519944a6fbeee4da11170b261" Dec 03 12:41:07 crc kubenswrapper[5052]: I1203 12:41:07.804092 5052 scope.go:117] "RemoveContainer" containerID="ef78ab4daf6f2994bb54c1f8844ff1255eef71866c6e7be47ed5d4c7ef1d996d" Dec 03 12:41:07 crc kubenswrapper[5052]: I1203 12:41:07.828781 5052 scope.go:117] "RemoveContainer" containerID="382891affaf25ff9b098ddaf284f3fa5667c14b93f3487974c23c2cdae354300" Dec 03 12:41:07 crc kubenswrapper[5052]: I1203 12:41:07.858818 5052 scope.go:117] "RemoveContainer" containerID="382891affaf25ff9b098ddaf284f3fa5667c14b93f3487974c23c2cdae354300" Dec 03 12:41:07 crc kubenswrapper[5052]: E1203 12:41:07.859307 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"382891affaf25ff9b098ddaf284f3fa5667c14b93f3487974c23c2cdae354300\": container with ID starting with 382891affaf25ff9b098ddaf284f3fa5667c14b93f3487974c23c2cdae354300 not found: ID does not exist" containerID="382891affaf25ff9b098ddaf284f3fa5667c14b93f3487974c23c2cdae354300" Dec 03 12:41:07 crc kubenswrapper[5052]: I1203 12:41:07.859596 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"382891affaf25ff9b098ddaf284f3fa5667c14b93f3487974c23c2cdae354300"} err="failed to get container status \"382891affaf25ff9b098ddaf284f3fa5667c14b93f3487974c23c2cdae354300\": rpc error: code = NotFound desc = could not find container \"382891affaf25ff9b098ddaf284f3fa5667c14b93f3487974c23c2cdae354300\": container with ID starting with 382891affaf25ff9b098ddaf284f3fa5667c14b93f3487974c23c2cdae354300 not found: ID does not exist" Dec 03 12:41:09 crc kubenswrapper[5052]: I1203 12:41:09.683812 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0556e9f0-e431-4193-9822-6079cb8a1e44" path="/var/lib/kubelet/pods/0556e9f0-e431-4193-9822-6079cb8a1e44/volumes" Dec 03 12:41:09 crc kubenswrapper[5052]: I1203 12:41:09.686816 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="65cb22ea-3a9c-4f99-9128-d3c7681c3173" path="/var/lib/kubelet/pods/65cb22ea-3a9c-4f99-9128-d3c7681c3173/volumes" Dec 03 12:41:09 crc kubenswrapper[5052]: I1203 12:41:09.688269 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9e3bfbf3-a39c-48d2-bb1d-062c90c0ef10" path="/var/lib/kubelet/pods/9e3bfbf3-a39c-48d2-bb1d-062c90c0ef10/volumes" Dec 03 12:41:09 crc kubenswrapper[5052]: I1203 12:41:09.690232 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ddfa4a3f-b868-49b7-abec-62cb3ffde2bd" path="/var/lib/kubelet/pods/ddfa4a3f-b868-49b7-abec-62cb3ffde2bd/volumes" Dec 03 12:41:09 crc kubenswrapper[5052]: E1203 12:41:09.789216 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:41:09 crc kubenswrapper[5052]: E1203 12:41:09.789327 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5fdc7b7d-f9ed-4d5c-a77b-6a6b0fd186f9-operator-scripts podName:5fdc7b7d-f9ed-4d5c-a77b-6a6b0fd186f9 nodeName:}" failed. No retries permitted until 2025-12-03 12:41:17.789303648 +0000 UTC m=+4868.377545422 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/5fdc7b7d-f9ed-4d5c-a77b-6a6b0fd186f9-operator-scripts") pod "cinder3a0d-account-delete-lxd6l" (UID: "5fdc7b7d-f9ed-4d5c-a77b-6a6b0fd186f9") : configmap "openstack-scripts" not found Dec 03 12:41:10 crc kubenswrapper[5052]: I1203 12:41:10.094739 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-8h8qz" Dec 03 12:41:10 crc kubenswrapper[5052]: I1203 12:41:10.160143 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-57d8c988fc-78nns"] Dec 03 12:41:10 crc kubenswrapper[5052]: I1203 12:41:10.161110 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-57d8c988fc-78nns" podUID="1fcf641b-a032-4179-9dde-ad76e9d86925" containerName="dnsmasq-dns" containerID="cri-o://9e329fad0d27b1a8701714781d22f98230c08ebcc9a5d105a727094e58b0a0ee" gracePeriod=10 Dec 03 12:41:10 crc kubenswrapper[5052]: I1203 12:41:10.526870 5052 generic.go:334] "Generic (PLEG): container finished" podID="1fcf641b-a032-4179-9dde-ad76e9d86925" containerID="9e329fad0d27b1a8701714781d22f98230c08ebcc9a5d105a727094e58b0a0ee" exitCode=0 Dec 03 12:41:10 crc kubenswrapper[5052]: I1203 12:41:10.526924 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-57d8c988fc-78nns" event={"ID":"1fcf641b-a032-4179-9dde-ad76e9d86925","Type":"ContainerDied","Data":"9e329fad0d27b1a8701714781d22f98230c08ebcc9a5d105a727094e58b0a0ee"} Dec 03 12:41:10 crc kubenswrapper[5052]: E1203 12:41:10.908016 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:41:10 crc kubenswrapper[5052]: E1203 12:41:10.908287 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/eb8b91d5-6c99-45ba-8d33-da99f1dd496c-operator-scripts podName:eb8b91d5-6c99-45ba-8d33-da99f1dd496c nodeName:}" failed. No retries permitted until 2025-12-03 12:41:18.908268997 +0000 UTC m=+4869.496510761 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/eb8b91d5-6c99-45ba-8d33-da99f1dd496c-operator-scripts") pod "placement7f30-account-delete-vld4v" (UID: "eb8b91d5-6c99-45ba-8d33-da99f1dd496c") : configmap "openstack-scripts" not found Dec 03 12:41:11 crc kubenswrapper[5052]: I1203 12:41:11.164066 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-57d8c988fc-78nns" Dec 03 12:41:11 crc kubenswrapper[5052]: I1203 12:41:11.213588 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"dns-swift-storage-0\" (UniqueName: \"kubernetes.io/configmap/1fcf641b-a032-4179-9dde-ad76e9d86925-dns-swift-storage-0\") pod \"1fcf641b-a032-4179-9dde-ad76e9d86925\" (UID: \"1fcf641b-a032-4179-9dde-ad76e9d86925\") " Dec 03 12:41:11 crc kubenswrapper[5052]: I1203 12:41:11.213658 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mhfdp\" (UniqueName: \"kubernetes.io/projected/1fcf641b-a032-4179-9dde-ad76e9d86925-kube-api-access-mhfdp\") pod \"1fcf641b-a032-4179-9dde-ad76e9d86925\" (UID: \"1fcf641b-a032-4179-9dde-ad76e9d86925\") " Dec 03 12:41:11 crc kubenswrapper[5052]: I1203 12:41:11.213715 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1fcf641b-a032-4179-9dde-ad76e9d86925-config\") pod \"1fcf641b-a032-4179-9dde-ad76e9d86925\" (UID: \"1fcf641b-a032-4179-9dde-ad76e9d86925\") " Dec 03 12:41:11 crc kubenswrapper[5052]: I1203 12:41:11.213778 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/1fcf641b-a032-4179-9dde-ad76e9d86925-dnsmasq-svc\") pod \"1fcf641b-a032-4179-9dde-ad76e9d86925\" (UID: \"1fcf641b-a032-4179-9dde-ad76e9d86925\") " Dec 03 12:41:11 crc kubenswrapper[5052]: I1203 12:41:11.218939 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1fcf641b-a032-4179-9dde-ad76e9d86925-kube-api-access-mhfdp" (OuterVolumeSpecName: "kube-api-access-mhfdp") pod "1fcf641b-a032-4179-9dde-ad76e9d86925" (UID: "1fcf641b-a032-4179-9dde-ad76e9d86925"). InnerVolumeSpecName "kube-api-access-mhfdp". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:41:11 crc kubenswrapper[5052]: I1203 12:41:11.251334 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1fcf641b-a032-4179-9dde-ad76e9d86925-dnsmasq-svc" (OuterVolumeSpecName: "dnsmasq-svc") pod "1fcf641b-a032-4179-9dde-ad76e9d86925" (UID: "1fcf641b-a032-4179-9dde-ad76e9d86925"). InnerVolumeSpecName "dnsmasq-svc". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:41:11 crc kubenswrapper[5052]: I1203 12:41:11.252797 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1fcf641b-a032-4179-9dde-ad76e9d86925-config" (OuterVolumeSpecName: "config") pod "1fcf641b-a032-4179-9dde-ad76e9d86925" (UID: "1fcf641b-a032-4179-9dde-ad76e9d86925"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:41:11 crc kubenswrapper[5052]: I1203 12:41:11.265072 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1fcf641b-a032-4179-9dde-ad76e9d86925-dns-swift-storage-0" (OuterVolumeSpecName: "dns-swift-storage-0") pod "1fcf641b-a032-4179-9dde-ad76e9d86925" (UID: "1fcf641b-a032-4179-9dde-ad76e9d86925"). InnerVolumeSpecName "dns-swift-storage-0". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:41:11 crc kubenswrapper[5052]: I1203 12:41:11.315562 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-mhfdp\" (UniqueName: \"kubernetes.io/projected/1fcf641b-a032-4179-9dde-ad76e9d86925-kube-api-access-mhfdp\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:11 crc kubenswrapper[5052]: I1203 12:41:11.315591 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1fcf641b-a032-4179-9dde-ad76e9d86925-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:11 crc kubenswrapper[5052]: I1203 12:41:11.315600 5052 reconciler_common.go:293] "Volume detached for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/1fcf641b-a032-4179-9dde-ad76e9d86925-dnsmasq-svc\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:11 crc kubenswrapper[5052]: I1203 12:41:11.315609 5052 reconciler_common.go:293] "Volume detached for volume \"dns-swift-storage-0\" (UniqueName: \"kubernetes.io/configmap/1fcf641b-a032-4179-9dde-ad76e9d86925-dns-swift-storage-0\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:11 crc kubenswrapper[5052]: I1203 12:41:11.537700 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-57d8c988fc-78nns" event={"ID":"1fcf641b-a032-4179-9dde-ad76e9d86925","Type":"ContainerDied","Data":"df2a18a663b3a561fd9b9b69b618387618f19977fe89926626b99ddeb8b1fe35"} Dec 03 12:41:11 crc kubenswrapper[5052]: I1203 12:41:11.537744 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-57d8c988fc-78nns" Dec 03 12:41:11 crc kubenswrapper[5052]: I1203 12:41:11.537758 5052 scope.go:117] "RemoveContainer" containerID="9e329fad0d27b1a8701714781d22f98230c08ebcc9a5d105a727094e58b0a0ee" Dec 03 12:41:11 crc kubenswrapper[5052]: I1203 12:41:11.573726 5052 scope.go:117] "RemoveContainer" containerID="817dd5a079a81242fe031d9189c03112a82a5c61f2ee81b7e6a114983bb4f990" Dec 03 12:41:11 crc kubenswrapper[5052]: I1203 12:41:11.579997 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-57d8c988fc-78nns"] Dec 03 12:41:11 crc kubenswrapper[5052]: I1203 12:41:11.585761 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-57d8c988fc-78nns"] Dec 03 12:41:11 crc kubenswrapper[5052]: I1203 12:41:11.684040 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1fcf641b-a032-4179-9dde-ad76e9d86925" path="/var/lib/kubelet/pods/1fcf641b-a032-4179-9dde-ad76e9d86925/volumes" Dec 03 12:41:17 crc kubenswrapper[5052]: E1203 12:41:17.945244 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:41:17 crc kubenswrapper[5052]: E1203 12:41:17.945901 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5fdc7b7d-f9ed-4d5c-a77b-6a6b0fd186f9-operator-scripts podName:5fdc7b7d-f9ed-4d5c-a77b-6a6b0fd186f9 nodeName:}" failed. No retries permitted until 2025-12-03 12:41:33.94587673 +0000 UTC m=+4884.534118504 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/5fdc7b7d-f9ed-4d5c-a77b-6a6b0fd186f9-operator-scripts") pod "cinder3a0d-account-delete-lxd6l" (UID: "5fdc7b7d-f9ed-4d5c-a77b-6a6b0fd186f9") : configmap "openstack-scripts" not found Dec 03 12:41:18 crc kubenswrapper[5052]: E1203 12:41:18.963459 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:41:18 crc kubenswrapper[5052]: E1203 12:41:18.963566 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/eb8b91d5-6c99-45ba-8d33-da99f1dd496c-operator-scripts podName:eb8b91d5-6c99-45ba-8d33-da99f1dd496c nodeName:}" failed. No retries permitted until 2025-12-03 12:41:34.963549624 +0000 UTC m=+4885.551791388 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/eb8b91d5-6c99-45ba-8d33-da99f1dd496c-operator-scripts") pod "placement7f30-account-delete-vld4v" (UID: "eb8b91d5-6c99-45ba-8d33-da99f1dd496c") : configmap "openstack-scripts" not found Dec 03 12:41:25 crc kubenswrapper[5052]: I1203 12:41:25.910937 5052 scope.go:117] "RemoveContainer" containerID="adfc4f6fc4db6a74ee3543f31c610dc26c11391eefc7a3ac50cbb3115dbc02c5" Dec 03 12:41:25 crc kubenswrapper[5052]: I1203 12:41:25.938698 5052 scope.go:117] "RemoveContainer" containerID="37205114d12d1cc0882510cbd6d7f46eb0e500325b061ad47f151d3e67478480" Dec 03 12:41:26 crc kubenswrapper[5052]: I1203 12:41:26.028689 5052 scope.go:117] "RemoveContainer" containerID="a10bf847b707e0b7acccc05608dc70ede33326a069a0b09b022880875d8e19af" Dec 03 12:41:26 crc kubenswrapper[5052]: I1203 12:41:26.068361 5052 scope.go:117] "RemoveContainer" containerID="ae7fadfe5a8df990d9027b5c721db8c6366fb8ccd2d58034fa4422a3ec06e20a" Dec 03 12:41:26 crc kubenswrapper[5052]: I1203 12:41:26.101707 5052 scope.go:117] "RemoveContainer" containerID="0c4e092293ea6adf3158ffe86080aa757175ad4a21f6d1b968c71ccec2b3a499" Dec 03 12:41:26 crc kubenswrapper[5052]: I1203 12:41:26.124749 5052 scope.go:117] "RemoveContainer" containerID="ee2f15d5cf56ceed957dec77523e3537f6db3f8aa6f0663cfe5da91979d7d23c" Dec 03 12:41:26 crc kubenswrapper[5052]: I1203 12:41:26.149051 5052 scope.go:117] "RemoveContainer" containerID="2253171c4c8bd199a61de0bcd7cfe694d9c538afb521022af5f3e5fd80739aea" Dec 03 12:41:26 crc kubenswrapper[5052]: I1203 12:41:26.205249 5052 scope.go:117] "RemoveContainer" containerID="2276cac3a17897b8e09aaa253d140d85fe4fd6c6e7497729651aa093fde9b97f" Dec 03 12:41:26 crc kubenswrapper[5052]: I1203 12:41:26.228230 5052 scope.go:117] "RemoveContainer" containerID="bf727997773d9131797166f237393d3a676ff8ba1aaa81a1ab8bdb9c481d21ae" Dec 03 12:41:26 crc kubenswrapper[5052]: I1203 12:41:26.262736 5052 scope.go:117] "RemoveContainer" containerID="a8d5c79e35703d7180a5fe276802f60ef6ba0eabb63c93544208b5e293dcdf4e" Dec 03 12:41:26 crc kubenswrapper[5052]: I1203 12:41:26.303821 5052 scope.go:117] "RemoveContainer" containerID="e7b490b7c7957636f9cffce74b4da96fad7f27b7e1fc5e91d3a9fc13f1f970f9" Dec 03 12:41:26 crc kubenswrapper[5052]: I1203 12:41:26.338096 5052 scope.go:117] "RemoveContainer" containerID="5bb5bef178a578ef338d8e4c3a444c4fd52a248e069fcb9184e3dd53a4555372" Dec 03 12:41:26 crc kubenswrapper[5052]: I1203 12:41:26.358051 5052 scope.go:117] "RemoveContainer" containerID="1e61208c62ab46fd9d699374a5c8b4391a3017ecb2c3822df67c5b7f305c2abc" Dec 03 12:41:26 crc kubenswrapper[5052]: I1203 12:41:26.378936 5052 scope.go:117] "RemoveContainer" containerID="85c2b1d766a8ae5a49909c56750cc7a4bbef2172c6a1149c8104f3518834e4b6" Dec 03 12:41:26 crc kubenswrapper[5052]: I1203 12:41:26.399766 5052 scope.go:117] "RemoveContainer" containerID="b9fb898ae650a25991bcaf8b4202e50ad89d244003a528574792da3d36870f29" Dec 03 12:41:26 crc kubenswrapper[5052]: I1203 12:41:26.444063 5052 scope.go:117] "RemoveContainer" containerID="0f4cc658cedc4570aad8a710600353eb501f638bf059e857fdb8c388153955ff" Dec 03 12:41:26 crc kubenswrapper[5052]: I1203 12:41:26.475222 5052 scope.go:117] "RemoveContainer" containerID="95f57bf535752ee63cba070f80b83c1452bf047674a4d81bbcd2c9f86b68b156" Dec 03 12:41:26 crc kubenswrapper[5052]: I1203 12:41:26.494439 5052 scope.go:117] "RemoveContainer" containerID="c2385170deb299c29625c309b58a0737df77e5b8f042cdb7e41b0523b06cd3ed" Dec 03 12:41:26 crc kubenswrapper[5052]: I1203 12:41:26.515505 5052 scope.go:117] "RemoveContainer" containerID="f179d5351b61865d029b504ef4c3a0f25f02e76884020ed26f254aca9ad99586" Dec 03 12:41:26 crc kubenswrapper[5052]: I1203 12:41:26.539282 5052 scope.go:117] "RemoveContainer" containerID="affe246afdd79fb48a7eb807b7a9eee6026dac2baf6ba1d47fd16150e4137cd4" Dec 03 12:41:26 crc kubenswrapper[5052]: I1203 12:41:26.565936 5052 scope.go:117] "RemoveContainer" containerID="fe3736c7ba04bcc1974c0d7f89521fdb2f73dec260f35708f011fe12a31b1f16" Dec 03 12:41:26 crc kubenswrapper[5052]: I1203 12:41:26.599960 5052 scope.go:117] "RemoveContainer" containerID="de7ec0b74eb7ee449efa4ae5a8ddecacb3b2510555a213842ba699fc19a15a76" Dec 03 12:41:26 crc kubenswrapper[5052]: I1203 12:41:26.656039 5052 scope.go:117] "RemoveContainer" containerID="d8dc47bd79781784d95621d45bf044b65727ec3ede2bffae178e8e91d46cc7b7" Dec 03 12:41:28 crc kubenswrapper[5052]: I1203 12:41:28.240283 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:41:28 crc kubenswrapper[5052]: I1203 12:41:28.430963 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ghbrw\" (UniqueName: \"kubernetes.io/projected/c7d598df-f336-4798-b381-9b813068f271-kube-api-access-ghbrw\") pod \"c7d598df-f336-4798-b381-9b813068f271\" (UID: \"c7d598df-f336-4798-b381-9b813068f271\") " Dec 03 12:41:28 crc kubenswrapper[5052]: I1203 12:41:28.431018 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/c7d598df-f336-4798-b381-9b813068f271-etc-swift\") pod \"c7d598df-f336-4798-b381-9b813068f271\" (UID: \"c7d598df-f336-4798-b381-9b813068f271\") " Dec 03 12:41:28 crc kubenswrapper[5052]: I1203 12:41:28.431086 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"lock\" (UniqueName: \"kubernetes.io/empty-dir/c7d598df-f336-4798-b381-9b813068f271-lock\") pod \"c7d598df-f336-4798-b381-9b813068f271\" (UID: \"c7d598df-f336-4798-b381-9b813068f271\") " Dec 03 12:41:28 crc kubenswrapper[5052]: I1203 12:41:28.431110 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"swift\" (UniqueName: \"kubernetes.io/local-volume/local-storage13-crc\") pod \"c7d598df-f336-4798-b381-9b813068f271\" (UID: \"c7d598df-f336-4798-b381-9b813068f271\") " Dec 03 12:41:28 crc kubenswrapper[5052]: I1203 12:41:28.431169 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"cache\" (UniqueName: \"kubernetes.io/empty-dir/c7d598df-f336-4798-b381-9b813068f271-cache\") pod \"c7d598df-f336-4798-b381-9b813068f271\" (UID: \"c7d598df-f336-4798-b381-9b813068f271\") " Dec 03 12:41:28 crc kubenswrapper[5052]: I1203 12:41:28.432095 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c7d598df-f336-4798-b381-9b813068f271-lock" (OuterVolumeSpecName: "lock") pod "c7d598df-f336-4798-b381-9b813068f271" (UID: "c7d598df-f336-4798-b381-9b813068f271"). InnerVolumeSpecName "lock". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:41:28 crc kubenswrapper[5052]: I1203 12:41:28.432143 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c7d598df-f336-4798-b381-9b813068f271-cache" (OuterVolumeSpecName: "cache") pod "c7d598df-f336-4798-b381-9b813068f271" (UID: "c7d598df-f336-4798-b381-9b813068f271"). InnerVolumeSpecName "cache". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:41:28 crc kubenswrapper[5052]: I1203 12:41:28.436815 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c7d598df-f336-4798-b381-9b813068f271-kube-api-access-ghbrw" (OuterVolumeSpecName: "kube-api-access-ghbrw") pod "c7d598df-f336-4798-b381-9b813068f271" (UID: "c7d598df-f336-4798-b381-9b813068f271"). InnerVolumeSpecName "kube-api-access-ghbrw". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:41:28 crc kubenswrapper[5052]: I1203 12:41:28.437365 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c7d598df-f336-4798-b381-9b813068f271-etc-swift" (OuterVolumeSpecName: "etc-swift") pod "c7d598df-f336-4798-b381-9b813068f271" (UID: "c7d598df-f336-4798-b381-9b813068f271"). InnerVolumeSpecName "etc-swift". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:41:28 crc kubenswrapper[5052]: I1203 12:41:28.450904 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage13-crc" (OuterVolumeSpecName: "swift") pod "c7d598df-f336-4798-b381-9b813068f271" (UID: "c7d598df-f336-4798-b381-9b813068f271"). InnerVolumeSpecName "local-storage13-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 12:41:28 crc kubenswrapper[5052]: I1203 12:41:28.533062 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-ghbrw\" (UniqueName: \"kubernetes.io/projected/c7d598df-f336-4798-b381-9b813068f271-kube-api-access-ghbrw\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:28 crc kubenswrapper[5052]: I1203 12:41:28.533114 5052 reconciler_common.go:293] "Volume detached for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/c7d598df-f336-4798-b381-9b813068f271-etc-swift\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:28 crc kubenswrapper[5052]: I1203 12:41:28.533127 5052 reconciler_common.go:293] "Volume detached for volume \"lock\" (UniqueName: \"kubernetes.io/empty-dir/c7d598df-f336-4798-b381-9b813068f271-lock\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:28 crc kubenswrapper[5052]: I1203 12:41:28.533163 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage13-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage13-crc\") on node \"crc\" " Dec 03 12:41:28 crc kubenswrapper[5052]: I1203 12:41:28.533178 5052 reconciler_common.go:293] "Volume detached for volume \"cache\" (UniqueName: \"kubernetes.io/empty-dir/c7d598df-f336-4798-b381-9b813068f271-cache\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:28 crc kubenswrapper[5052]: I1203 12:41:28.550995 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage13-crc" (UniqueName: "kubernetes.io/local-volume/local-storage13-crc") on node "crc" Dec 03 12:41:28 crc kubenswrapper[5052]: I1203 12:41:28.634262 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage13-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage13-crc\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:28 crc kubenswrapper[5052]: I1203 12:41:28.758159 5052 generic.go:334] "Generic (PLEG): container finished" podID="c7d598df-f336-4798-b381-9b813068f271" containerID="2c19477455e7e676ad01ea83d0e26999fb852f2f3e3c20bf2a784b76c5930dfc" exitCode=137 Dec 03 12:41:28 crc kubenswrapper[5052]: I1203 12:41:28.758211 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"c7d598df-f336-4798-b381-9b813068f271","Type":"ContainerDied","Data":"2c19477455e7e676ad01ea83d0e26999fb852f2f3e3c20bf2a784b76c5930dfc"} Dec 03 12:41:28 crc kubenswrapper[5052]: I1203 12:41:28.758248 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"c7d598df-f336-4798-b381-9b813068f271","Type":"ContainerDied","Data":"f82206729c6907aa4b14611e3738346c1d95ff29c1214f31750b8be9aefd7692"} Dec 03 12:41:28 crc kubenswrapper[5052]: I1203 12:41:28.758275 5052 scope.go:117] "RemoveContainer" containerID="2c19477455e7e676ad01ea83d0e26999fb852f2f3e3c20bf2a784b76c5930dfc" Dec 03 12:41:28 crc kubenswrapper[5052]: I1203 12:41:28.758547 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:41:28 crc kubenswrapper[5052]: I1203 12:41:28.796019 5052 scope.go:117] "RemoveContainer" containerID="34d7b3c41a14d274d9f82dc6e97d783bfeea4af12baf996aefeb1b3be4f773dd" Dec 03 12:41:28 crc kubenswrapper[5052]: I1203 12:41:28.805049 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/swift-storage-0"] Dec 03 12:41:28 crc kubenswrapper[5052]: I1203 12:41:28.811587 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/swift-storage-0"] Dec 03 12:41:28 crc kubenswrapper[5052]: I1203 12:41:28.822968 5052 scope.go:117] "RemoveContainer" containerID="3668ba2994a5322df4fc232407f1ab911e9742d99a5ce860e7891029d6b3262b" Dec 03 12:41:28 crc kubenswrapper[5052]: I1203 12:41:28.846968 5052 scope.go:117] "RemoveContainer" containerID="2bccd4a808d4dfbe6d20fb21dfd50899da53a3d24dee822e5b066fe75d15e745" Dec 03 12:41:28 crc kubenswrapper[5052]: I1203 12:41:28.870358 5052 scope.go:117] "RemoveContainer" containerID="5e022d3b3c656463516700245609bcc85dbdb9a341c772fa6cfcb4273bc6c15d" Dec 03 12:41:28 crc kubenswrapper[5052]: I1203 12:41:28.890284 5052 scope.go:117] "RemoveContainer" containerID="67aad15627897c77a5ca09bba14532fa30813bf015f5d0017d2d2538795a5666" Dec 03 12:41:28 crc kubenswrapper[5052]: I1203 12:41:28.912308 5052 scope.go:117] "RemoveContainer" containerID="e82e8cd6cac0e3fe75cbcd377821970ca81cbeff69756a18c653617637c572e3" Dec 03 12:41:28 crc kubenswrapper[5052]: I1203 12:41:28.933573 5052 scope.go:117] "RemoveContainer" containerID="e1915bece2ea7e993dda7b1e15e670ab02f1ff041682e537f1d13ff72de48727" Dec 03 12:41:28 crc kubenswrapper[5052]: I1203 12:41:28.954294 5052 scope.go:117] "RemoveContainer" containerID="bf315758fc67c50f371749f4f7bbec4a1f92fbcaefd1d1b439352a3050edde3f" Dec 03 12:41:28 crc kubenswrapper[5052]: I1203 12:41:28.987008 5052 scope.go:117] "RemoveContainer" containerID="aa711f046bfc3e94d6ec5597e4da6fb83450c27a3396cf24f0175b9d7508c25a" Dec 03 12:41:29 crc kubenswrapper[5052]: I1203 12:41:29.010881 5052 scope.go:117] "RemoveContainer" containerID="490483344d3233731e445298ec4b3a6f3b3d6ba0822ed43ceb5cdbc637a73f2c" Dec 03 12:41:29 crc kubenswrapper[5052]: I1203 12:41:29.034898 5052 scope.go:117] "RemoveContainer" containerID="13466840574a6516f401dae001c130c3bbb09775106363deef64c61f44c71bac" Dec 03 12:41:29 crc kubenswrapper[5052]: I1203 12:41:29.066259 5052 scope.go:117] "RemoveContainer" containerID="93f01c0cccc7bb298548365c64b66abc7b40d0c114c2d14e3eaed3c673c6da66" Dec 03 12:41:29 crc kubenswrapper[5052]: I1203 12:41:29.086161 5052 scope.go:117] "RemoveContainer" containerID="cabbc82eac0ab602656d651e0b196849ff956b6dd323581aac8d857b23fae0e7" Dec 03 12:41:29 crc kubenswrapper[5052]: I1203 12:41:29.105254 5052 scope.go:117] "RemoveContainer" containerID="7fd8d3e9755be0491e71f4771e79219dfa4803774a0535e6af7be76023acf022" Dec 03 12:41:29 crc kubenswrapper[5052]: I1203 12:41:29.124728 5052 scope.go:117] "RemoveContainer" containerID="2c19477455e7e676ad01ea83d0e26999fb852f2f3e3c20bf2a784b76c5930dfc" Dec 03 12:41:29 crc kubenswrapper[5052]: E1203 12:41:29.125287 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2c19477455e7e676ad01ea83d0e26999fb852f2f3e3c20bf2a784b76c5930dfc\": container with ID starting with 2c19477455e7e676ad01ea83d0e26999fb852f2f3e3c20bf2a784b76c5930dfc not found: ID does not exist" containerID="2c19477455e7e676ad01ea83d0e26999fb852f2f3e3c20bf2a784b76c5930dfc" Dec 03 12:41:29 crc kubenswrapper[5052]: I1203 12:41:29.125343 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2c19477455e7e676ad01ea83d0e26999fb852f2f3e3c20bf2a784b76c5930dfc"} err="failed to get container status \"2c19477455e7e676ad01ea83d0e26999fb852f2f3e3c20bf2a784b76c5930dfc\": rpc error: code = NotFound desc = could not find container \"2c19477455e7e676ad01ea83d0e26999fb852f2f3e3c20bf2a784b76c5930dfc\": container with ID starting with 2c19477455e7e676ad01ea83d0e26999fb852f2f3e3c20bf2a784b76c5930dfc not found: ID does not exist" Dec 03 12:41:29 crc kubenswrapper[5052]: I1203 12:41:29.125372 5052 scope.go:117] "RemoveContainer" containerID="34d7b3c41a14d274d9f82dc6e97d783bfeea4af12baf996aefeb1b3be4f773dd" Dec 03 12:41:29 crc kubenswrapper[5052]: E1203 12:41:29.125960 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"34d7b3c41a14d274d9f82dc6e97d783bfeea4af12baf996aefeb1b3be4f773dd\": container with ID starting with 34d7b3c41a14d274d9f82dc6e97d783bfeea4af12baf996aefeb1b3be4f773dd not found: ID does not exist" containerID="34d7b3c41a14d274d9f82dc6e97d783bfeea4af12baf996aefeb1b3be4f773dd" Dec 03 12:41:29 crc kubenswrapper[5052]: I1203 12:41:29.125990 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"34d7b3c41a14d274d9f82dc6e97d783bfeea4af12baf996aefeb1b3be4f773dd"} err="failed to get container status \"34d7b3c41a14d274d9f82dc6e97d783bfeea4af12baf996aefeb1b3be4f773dd\": rpc error: code = NotFound desc = could not find container \"34d7b3c41a14d274d9f82dc6e97d783bfeea4af12baf996aefeb1b3be4f773dd\": container with ID starting with 34d7b3c41a14d274d9f82dc6e97d783bfeea4af12baf996aefeb1b3be4f773dd not found: ID does not exist" Dec 03 12:41:29 crc kubenswrapper[5052]: I1203 12:41:29.126010 5052 scope.go:117] "RemoveContainer" containerID="3668ba2994a5322df4fc232407f1ab911e9742d99a5ce860e7891029d6b3262b" Dec 03 12:41:29 crc kubenswrapper[5052]: E1203 12:41:29.126276 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3668ba2994a5322df4fc232407f1ab911e9742d99a5ce860e7891029d6b3262b\": container with ID starting with 3668ba2994a5322df4fc232407f1ab911e9742d99a5ce860e7891029d6b3262b not found: ID does not exist" containerID="3668ba2994a5322df4fc232407f1ab911e9742d99a5ce860e7891029d6b3262b" Dec 03 12:41:29 crc kubenswrapper[5052]: I1203 12:41:29.126295 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3668ba2994a5322df4fc232407f1ab911e9742d99a5ce860e7891029d6b3262b"} err="failed to get container status \"3668ba2994a5322df4fc232407f1ab911e9742d99a5ce860e7891029d6b3262b\": rpc error: code = NotFound desc = could not find container \"3668ba2994a5322df4fc232407f1ab911e9742d99a5ce860e7891029d6b3262b\": container with ID starting with 3668ba2994a5322df4fc232407f1ab911e9742d99a5ce860e7891029d6b3262b not found: ID does not exist" Dec 03 12:41:29 crc kubenswrapper[5052]: I1203 12:41:29.126310 5052 scope.go:117] "RemoveContainer" containerID="2bccd4a808d4dfbe6d20fb21dfd50899da53a3d24dee822e5b066fe75d15e745" Dec 03 12:41:29 crc kubenswrapper[5052]: E1203 12:41:29.126783 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2bccd4a808d4dfbe6d20fb21dfd50899da53a3d24dee822e5b066fe75d15e745\": container with ID starting with 2bccd4a808d4dfbe6d20fb21dfd50899da53a3d24dee822e5b066fe75d15e745 not found: ID does not exist" containerID="2bccd4a808d4dfbe6d20fb21dfd50899da53a3d24dee822e5b066fe75d15e745" Dec 03 12:41:29 crc kubenswrapper[5052]: I1203 12:41:29.126808 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2bccd4a808d4dfbe6d20fb21dfd50899da53a3d24dee822e5b066fe75d15e745"} err="failed to get container status \"2bccd4a808d4dfbe6d20fb21dfd50899da53a3d24dee822e5b066fe75d15e745\": rpc error: code = NotFound desc = could not find container \"2bccd4a808d4dfbe6d20fb21dfd50899da53a3d24dee822e5b066fe75d15e745\": container with ID starting with 2bccd4a808d4dfbe6d20fb21dfd50899da53a3d24dee822e5b066fe75d15e745 not found: ID does not exist" Dec 03 12:41:29 crc kubenswrapper[5052]: I1203 12:41:29.126824 5052 scope.go:117] "RemoveContainer" containerID="5e022d3b3c656463516700245609bcc85dbdb9a341c772fa6cfcb4273bc6c15d" Dec 03 12:41:29 crc kubenswrapper[5052]: E1203 12:41:29.127095 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5e022d3b3c656463516700245609bcc85dbdb9a341c772fa6cfcb4273bc6c15d\": container with ID starting with 5e022d3b3c656463516700245609bcc85dbdb9a341c772fa6cfcb4273bc6c15d not found: ID does not exist" containerID="5e022d3b3c656463516700245609bcc85dbdb9a341c772fa6cfcb4273bc6c15d" Dec 03 12:41:29 crc kubenswrapper[5052]: I1203 12:41:29.127125 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5e022d3b3c656463516700245609bcc85dbdb9a341c772fa6cfcb4273bc6c15d"} err="failed to get container status \"5e022d3b3c656463516700245609bcc85dbdb9a341c772fa6cfcb4273bc6c15d\": rpc error: code = NotFound desc = could not find container \"5e022d3b3c656463516700245609bcc85dbdb9a341c772fa6cfcb4273bc6c15d\": container with ID starting with 5e022d3b3c656463516700245609bcc85dbdb9a341c772fa6cfcb4273bc6c15d not found: ID does not exist" Dec 03 12:41:29 crc kubenswrapper[5052]: I1203 12:41:29.127140 5052 scope.go:117] "RemoveContainer" containerID="67aad15627897c77a5ca09bba14532fa30813bf015f5d0017d2d2538795a5666" Dec 03 12:41:29 crc kubenswrapper[5052]: E1203 12:41:29.127688 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"67aad15627897c77a5ca09bba14532fa30813bf015f5d0017d2d2538795a5666\": container with ID starting with 67aad15627897c77a5ca09bba14532fa30813bf015f5d0017d2d2538795a5666 not found: ID does not exist" containerID="67aad15627897c77a5ca09bba14532fa30813bf015f5d0017d2d2538795a5666" Dec 03 12:41:29 crc kubenswrapper[5052]: I1203 12:41:29.127708 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"67aad15627897c77a5ca09bba14532fa30813bf015f5d0017d2d2538795a5666"} err="failed to get container status \"67aad15627897c77a5ca09bba14532fa30813bf015f5d0017d2d2538795a5666\": rpc error: code = NotFound desc = could not find container \"67aad15627897c77a5ca09bba14532fa30813bf015f5d0017d2d2538795a5666\": container with ID starting with 67aad15627897c77a5ca09bba14532fa30813bf015f5d0017d2d2538795a5666 not found: ID does not exist" Dec 03 12:41:29 crc kubenswrapper[5052]: I1203 12:41:29.127720 5052 scope.go:117] "RemoveContainer" containerID="e82e8cd6cac0e3fe75cbcd377821970ca81cbeff69756a18c653617637c572e3" Dec 03 12:41:29 crc kubenswrapper[5052]: E1203 12:41:29.127958 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e82e8cd6cac0e3fe75cbcd377821970ca81cbeff69756a18c653617637c572e3\": container with ID starting with e82e8cd6cac0e3fe75cbcd377821970ca81cbeff69756a18c653617637c572e3 not found: ID does not exist" containerID="e82e8cd6cac0e3fe75cbcd377821970ca81cbeff69756a18c653617637c572e3" Dec 03 12:41:29 crc kubenswrapper[5052]: I1203 12:41:29.127979 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e82e8cd6cac0e3fe75cbcd377821970ca81cbeff69756a18c653617637c572e3"} err="failed to get container status \"e82e8cd6cac0e3fe75cbcd377821970ca81cbeff69756a18c653617637c572e3\": rpc error: code = NotFound desc = could not find container \"e82e8cd6cac0e3fe75cbcd377821970ca81cbeff69756a18c653617637c572e3\": container with ID starting with e82e8cd6cac0e3fe75cbcd377821970ca81cbeff69756a18c653617637c572e3 not found: ID does not exist" Dec 03 12:41:29 crc kubenswrapper[5052]: I1203 12:41:29.127991 5052 scope.go:117] "RemoveContainer" containerID="e1915bece2ea7e993dda7b1e15e670ab02f1ff041682e537f1d13ff72de48727" Dec 03 12:41:29 crc kubenswrapper[5052]: E1203 12:41:29.128293 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e1915bece2ea7e993dda7b1e15e670ab02f1ff041682e537f1d13ff72de48727\": container with ID starting with e1915bece2ea7e993dda7b1e15e670ab02f1ff041682e537f1d13ff72de48727 not found: ID does not exist" containerID="e1915bece2ea7e993dda7b1e15e670ab02f1ff041682e537f1d13ff72de48727" Dec 03 12:41:29 crc kubenswrapper[5052]: I1203 12:41:29.128315 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e1915bece2ea7e993dda7b1e15e670ab02f1ff041682e537f1d13ff72de48727"} err="failed to get container status \"e1915bece2ea7e993dda7b1e15e670ab02f1ff041682e537f1d13ff72de48727\": rpc error: code = NotFound desc = could not find container \"e1915bece2ea7e993dda7b1e15e670ab02f1ff041682e537f1d13ff72de48727\": container with ID starting with e1915bece2ea7e993dda7b1e15e670ab02f1ff041682e537f1d13ff72de48727 not found: ID does not exist" Dec 03 12:41:29 crc kubenswrapper[5052]: I1203 12:41:29.128327 5052 scope.go:117] "RemoveContainer" containerID="bf315758fc67c50f371749f4f7bbec4a1f92fbcaefd1d1b439352a3050edde3f" Dec 03 12:41:29 crc kubenswrapper[5052]: E1203 12:41:29.128562 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bf315758fc67c50f371749f4f7bbec4a1f92fbcaefd1d1b439352a3050edde3f\": container with ID starting with bf315758fc67c50f371749f4f7bbec4a1f92fbcaefd1d1b439352a3050edde3f not found: ID does not exist" containerID="bf315758fc67c50f371749f4f7bbec4a1f92fbcaefd1d1b439352a3050edde3f" Dec 03 12:41:29 crc kubenswrapper[5052]: I1203 12:41:29.128594 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bf315758fc67c50f371749f4f7bbec4a1f92fbcaefd1d1b439352a3050edde3f"} err="failed to get container status \"bf315758fc67c50f371749f4f7bbec4a1f92fbcaefd1d1b439352a3050edde3f\": rpc error: code = NotFound desc = could not find container \"bf315758fc67c50f371749f4f7bbec4a1f92fbcaefd1d1b439352a3050edde3f\": container with ID starting with bf315758fc67c50f371749f4f7bbec4a1f92fbcaefd1d1b439352a3050edde3f not found: ID does not exist" Dec 03 12:41:29 crc kubenswrapper[5052]: I1203 12:41:29.128609 5052 scope.go:117] "RemoveContainer" containerID="aa711f046bfc3e94d6ec5597e4da6fb83450c27a3396cf24f0175b9d7508c25a" Dec 03 12:41:29 crc kubenswrapper[5052]: E1203 12:41:29.128841 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aa711f046bfc3e94d6ec5597e4da6fb83450c27a3396cf24f0175b9d7508c25a\": container with ID starting with aa711f046bfc3e94d6ec5597e4da6fb83450c27a3396cf24f0175b9d7508c25a not found: ID does not exist" containerID="aa711f046bfc3e94d6ec5597e4da6fb83450c27a3396cf24f0175b9d7508c25a" Dec 03 12:41:29 crc kubenswrapper[5052]: I1203 12:41:29.128865 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aa711f046bfc3e94d6ec5597e4da6fb83450c27a3396cf24f0175b9d7508c25a"} err="failed to get container status \"aa711f046bfc3e94d6ec5597e4da6fb83450c27a3396cf24f0175b9d7508c25a\": rpc error: code = NotFound desc = could not find container \"aa711f046bfc3e94d6ec5597e4da6fb83450c27a3396cf24f0175b9d7508c25a\": container with ID starting with aa711f046bfc3e94d6ec5597e4da6fb83450c27a3396cf24f0175b9d7508c25a not found: ID does not exist" Dec 03 12:41:29 crc kubenswrapper[5052]: I1203 12:41:29.128880 5052 scope.go:117] "RemoveContainer" containerID="490483344d3233731e445298ec4b3a6f3b3d6ba0822ed43ceb5cdbc637a73f2c" Dec 03 12:41:29 crc kubenswrapper[5052]: E1203 12:41:29.129223 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"490483344d3233731e445298ec4b3a6f3b3d6ba0822ed43ceb5cdbc637a73f2c\": container with ID starting with 490483344d3233731e445298ec4b3a6f3b3d6ba0822ed43ceb5cdbc637a73f2c not found: ID does not exist" containerID="490483344d3233731e445298ec4b3a6f3b3d6ba0822ed43ceb5cdbc637a73f2c" Dec 03 12:41:29 crc kubenswrapper[5052]: I1203 12:41:29.129243 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"490483344d3233731e445298ec4b3a6f3b3d6ba0822ed43ceb5cdbc637a73f2c"} err="failed to get container status \"490483344d3233731e445298ec4b3a6f3b3d6ba0822ed43ceb5cdbc637a73f2c\": rpc error: code = NotFound desc = could not find container \"490483344d3233731e445298ec4b3a6f3b3d6ba0822ed43ceb5cdbc637a73f2c\": container with ID starting with 490483344d3233731e445298ec4b3a6f3b3d6ba0822ed43ceb5cdbc637a73f2c not found: ID does not exist" Dec 03 12:41:29 crc kubenswrapper[5052]: I1203 12:41:29.129258 5052 scope.go:117] "RemoveContainer" containerID="13466840574a6516f401dae001c130c3bbb09775106363deef64c61f44c71bac" Dec 03 12:41:29 crc kubenswrapper[5052]: E1203 12:41:29.129697 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"13466840574a6516f401dae001c130c3bbb09775106363deef64c61f44c71bac\": container with ID starting with 13466840574a6516f401dae001c130c3bbb09775106363deef64c61f44c71bac not found: ID does not exist" containerID="13466840574a6516f401dae001c130c3bbb09775106363deef64c61f44c71bac" Dec 03 12:41:29 crc kubenswrapper[5052]: I1203 12:41:29.129741 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"13466840574a6516f401dae001c130c3bbb09775106363deef64c61f44c71bac"} err="failed to get container status \"13466840574a6516f401dae001c130c3bbb09775106363deef64c61f44c71bac\": rpc error: code = NotFound desc = could not find container \"13466840574a6516f401dae001c130c3bbb09775106363deef64c61f44c71bac\": container with ID starting with 13466840574a6516f401dae001c130c3bbb09775106363deef64c61f44c71bac not found: ID does not exist" Dec 03 12:41:29 crc kubenswrapper[5052]: I1203 12:41:29.129756 5052 scope.go:117] "RemoveContainer" containerID="93f01c0cccc7bb298548365c64b66abc7b40d0c114c2d14e3eaed3c673c6da66" Dec 03 12:41:29 crc kubenswrapper[5052]: E1203 12:41:29.130047 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"93f01c0cccc7bb298548365c64b66abc7b40d0c114c2d14e3eaed3c673c6da66\": container with ID starting with 93f01c0cccc7bb298548365c64b66abc7b40d0c114c2d14e3eaed3c673c6da66 not found: ID does not exist" containerID="93f01c0cccc7bb298548365c64b66abc7b40d0c114c2d14e3eaed3c673c6da66" Dec 03 12:41:29 crc kubenswrapper[5052]: I1203 12:41:29.130067 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"93f01c0cccc7bb298548365c64b66abc7b40d0c114c2d14e3eaed3c673c6da66"} err="failed to get container status \"93f01c0cccc7bb298548365c64b66abc7b40d0c114c2d14e3eaed3c673c6da66\": rpc error: code = NotFound desc = could not find container \"93f01c0cccc7bb298548365c64b66abc7b40d0c114c2d14e3eaed3c673c6da66\": container with ID starting with 93f01c0cccc7bb298548365c64b66abc7b40d0c114c2d14e3eaed3c673c6da66 not found: ID does not exist" Dec 03 12:41:29 crc kubenswrapper[5052]: I1203 12:41:29.130084 5052 scope.go:117] "RemoveContainer" containerID="cabbc82eac0ab602656d651e0b196849ff956b6dd323581aac8d857b23fae0e7" Dec 03 12:41:29 crc kubenswrapper[5052]: E1203 12:41:29.130346 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cabbc82eac0ab602656d651e0b196849ff956b6dd323581aac8d857b23fae0e7\": container with ID starting with cabbc82eac0ab602656d651e0b196849ff956b6dd323581aac8d857b23fae0e7 not found: ID does not exist" containerID="cabbc82eac0ab602656d651e0b196849ff956b6dd323581aac8d857b23fae0e7" Dec 03 12:41:29 crc kubenswrapper[5052]: I1203 12:41:29.130366 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cabbc82eac0ab602656d651e0b196849ff956b6dd323581aac8d857b23fae0e7"} err="failed to get container status \"cabbc82eac0ab602656d651e0b196849ff956b6dd323581aac8d857b23fae0e7\": rpc error: code = NotFound desc = could not find container \"cabbc82eac0ab602656d651e0b196849ff956b6dd323581aac8d857b23fae0e7\": container with ID starting with cabbc82eac0ab602656d651e0b196849ff956b6dd323581aac8d857b23fae0e7 not found: ID does not exist" Dec 03 12:41:29 crc kubenswrapper[5052]: I1203 12:41:29.130378 5052 scope.go:117] "RemoveContainer" containerID="7fd8d3e9755be0491e71f4771e79219dfa4803774a0535e6af7be76023acf022" Dec 03 12:41:29 crc kubenswrapper[5052]: E1203 12:41:29.130751 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7fd8d3e9755be0491e71f4771e79219dfa4803774a0535e6af7be76023acf022\": container with ID starting with 7fd8d3e9755be0491e71f4771e79219dfa4803774a0535e6af7be76023acf022 not found: ID does not exist" containerID="7fd8d3e9755be0491e71f4771e79219dfa4803774a0535e6af7be76023acf022" Dec 03 12:41:29 crc kubenswrapper[5052]: I1203 12:41:29.130779 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7fd8d3e9755be0491e71f4771e79219dfa4803774a0535e6af7be76023acf022"} err="failed to get container status \"7fd8d3e9755be0491e71f4771e79219dfa4803774a0535e6af7be76023acf022\": rpc error: code = NotFound desc = could not find container \"7fd8d3e9755be0491e71f4771e79219dfa4803774a0535e6af7be76023acf022\": container with ID starting with 7fd8d3e9755be0491e71f4771e79219dfa4803774a0535e6af7be76023acf022 not found: ID does not exist" Dec 03 12:41:29 crc kubenswrapper[5052]: I1203 12:41:29.684292 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c7d598df-f336-4798-b381-9b813068f271" path="/var/lib/kubelet/pods/c7d598df-f336-4798-b381-9b813068f271/volumes" Dec 03 12:41:33 crc kubenswrapper[5052]: I1203 12:41:33.808122 5052 generic.go:334] "Generic (PLEG): container finished" podID="5fdc7b7d-f9ed-4d5c-a77b-6a6b0fd186f9" containerID="660e428afdfd8a7e278a3dd2d6aa79748d2d7485dda4c8e8678801b9bcc2009e" exitCode=137 Dec 03 12:41:33 crc kubenswrapper[5052]: I1203 12:41:33.808405 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder3a0d-account-delete-lxd6l" event={"ID":"5fdc7b7d-f9ed-4d5c-a77b-6a6b0fd186f9","Type":"ContainerDied","Data":"660e428afdfd8a7e278a3dd2d6aa79748d2d7485dda4c8e8678801b9bcc2009e"} Dec 03 12:41:34 crc kubenswrapper[5052]: I1203 12:41:34.005378 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder3a0d-account-delete-lxd6l" Dec 03 12:41:34 crc kubenswrapper[5052]: E1203 12:41:34.019551 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:41:34 crc kubenswrapper[5052]: E1203 12:41:34.019629 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5fdc7b7d-f9ed-4d5c-a77b-6a6b0fd186f9-operator-scripts podName:5fdc7b7d-f9ed-4d5c-a77b-6a6b0fd186f9 nodeName:}" failed. No retries permitted until 2025-12-03 12:42:06.019611111 +0000 UTC m=+4916.607852875 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/5fdc7b7d-f9ed-4d5c-a77b-6a6b0fd186f9-operator-scripts") pod "cinder3a0d-account-delete-lxd6l" (UID: "5fdc7b7d-f9ed-4d5c-a77b-6a6b0fd186f9") : configmap "openstack-scripts" not found Dec 03 12:41:34 crc kubenswrapper[5052]: I1203 12:41:34.120207 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/5fdc7b7d-f9ed-4d5c-a77b-6a6b0fd186f9-operator-scripts\") pod \"5fdc7b7d-f9ed-4d5c-a77b-6a6b0fd186f9\" (UID: \"5fdc7b7d-f9ed-4d5c-a77b-6a6b0fd186f9\") " Dec 03 12:41:34 crc kubenswrapper[5052]: I1203 12:41:34.120288 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-85cc8\" (UniqueName: \"kubernetes.io/projected/5fdc7b7d-f9ed-4d5c-a77b-6a6b0fd186f9-kube-api-access-85cc8\") pod \"5fdc7b7d-f9ed-4d5c-a77b-6a6b0fd186f9\" (UID: \"5fdc7b7d-f9ed-4d5c-a77b-6a6b0fd186f9\") " Dec 03 12:41:34 crc kubenswrapper[5052]: I1203 12:41:34.121361 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5fdc7b7d-f9ed-4d5c-a77b-6a6b0fd186f9-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "5fdc7b7d-f9ed-4d5c-a77b-6a6b0fd186f9" (UID: "5fdc7b7d-f9ed-4d5c-a77b-6a6b0fd186f9"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:41:34 crc kubenswrapper[5052]: I1203 12:41:34.129198 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5fdc7b7d-f9ed-4d5c-a77b-6a6b0fd186f9-kube-api-access-85cc8" (OuterVolumeSpecName: "kube-api-access-85cc8") pod "5fdc7b7d-f9ed-4d5c-a77b-6a6b0fd186f9" (UID: "5fdc7b7d-f9ed-4d5c-a77b-6a6b0fd186f9"). InnerVolumeSpecName "kube-api-access-85cc8". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:41:34 crc kubenswrapper[5052]: I1203 12:41:34.221965 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/5fdc7b7d-f9ed-4d5c-a77b-6a6b0fd186f9-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:34 crc kubenswrapper[5052]: I1203 12:41:34.222010 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-85cc8\" (UniqueName: \"kubernetes.io/projected/5fdc7b7d-f9ed-4d5c-a77b-6a6b0fd186f9-kube-api-access-85cc8\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:34 crc kubenswrapper[5052]: I1203 12:41:34.639010 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican1171-account-delete-9hmgl" Dec 03 12:41:34 crc kubenswrapper[5052]: I1203 12:41:34.731108 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-59qch\" (UniqueName: \"kubernetes.io/projected/ab29f82e-0693-4001-9f2a-bb4cc2fa1d5a-kube-api-access-59qch\") pod \"ab29f82e-0693-4001-9f2a-bb4cc2fa1d5a\" (UID: \"ab29f82e-0693-4001-9f2a-bb4cc2fa1d5a\") " Dec 03 12:41:34 crc kubenswrapper[5052]: I1203 12:41:34.731253 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/ab29f82e-0693-4001-9f2a-bb4cc2fa1d5a-operator-scripts\") pod \"ab29f82e-0693-4001-9f2a-bb4cc2fa1d5a\" (UID: \"ab29f82e-0693-4001-9f2a-bb4cc2fa1d5a\") " Dec 03 12:41:34 crc kubenswrapper[5052]: I1203 12:41:34.734589 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ab29f82e-0693-4001-9f2a-bb4cc2fa1d5a-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "ab29f82e-0693-4001-9f2a-bb4cc2fa1d5a" (UID: "ab29f82e-0693-4001-9f2a-bb4cc2fa1d5a"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:41:34 crc kubenswrapper[5052]: I1203 12:41:34.735088 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/ab29f82e-0693-4001-9f2a-bb4cc2fa1d5a-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:34 crc kubenswrapper[5052]: I1203 12:41:34.737533 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ab29f82e-0693-4001-9f2a-bb4cc2fa1d5a-kube-api-access-59qch" (OuterVolumeSpecName: "kube-api-access-59qch") pod "ab29f82e-0693-4001-9f2a-bb4cc2fa1d5a" (UID: "ab29f82e-0693-4001-9f2a-bb4cc2fa1d5a"). InnerVolumeSpecName "kube-api-access-59qch". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:41:34 crc kubenswrapper[5052]: I1203 12:41:34.775602 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/novaapib163-account-delete-8z4bt" Dec 03 12:41:34 crc kubenswrapper[5052]: I1203 12:41:34.802133 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement7f30-account-delete-vld4v" Dec 03 12:41:34 crc kubenswrapper[5052]: I1203 12:41:34.808695 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/novacell07298-account-delete-726jv" Dec 03 12:41:34 crc kubenswrapper[5052]: I1203 12:41:34.833357 5052 generic.go:334] "Generic (PLEG): container finished" podID="eb8b91d5-6c99-45ba-8d33-da99f1dd496c" containerID="bb53f0acf7c936b9d6eebe4376be498cac662cc7e2d991df4516aca0dbea473e" exitCode=137 Dec 03 12:41:34 crc kubenswrapper[5052]: I1203 12:41:34.833438 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement7f30-account-delete-vld4v" event={"ID":"eb8b91d5-6c99-45ba-8d33-da99f1dd496c","Type":"ContainerDied","Data":"bb53f0acf7c936b9d6eebe4376be498cac662cc7e2d991df4516aca0dbea473e"} Dec 03 12:41:34 crc kubenswrapper[5052]: I1203 12:41:34.833466 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement7f30-account-delete-vld4v" event={"ID":"eb8b91d5-6c99-45ba-8d33-da99f1dd496c","Type":"ContainerDied","Data":"2797d43ce4aacb7212eabdda418f85dbb3bdf966af8318c082d3d3f611404fd5"} Dec 03 12:41:34 crc kubenswrapper[5052]: I1203 12:41:34.833483 5052 scope.go:117] "RemoveContainer" containerID="bb53f0acf7c936b9d6eebe4376be498cac662cc7e2d991df4516aca0dbea473e" Dec 03 12:41:34 crc kubenswrapper[5052]: I1203 12:41:34.833626 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement7f30-account-delete-vld4v" Dec 03 12:41:34 crc kubenswrapper[5052]: I1203 12:41:34.836040 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-59qch\" (UniqueName: \"kubernetes.io/projected/ab29f82e-0693-4001-9f2a-bb4cc2fa1d5a-kube-api-access-59qch\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:34 crc kubenswrapper[5052]: I1203 12:41:34.837535 5052 generic.go:334] "Generic (PLEG): container finished" podID="6dbc7d8f-34d4-4927-82fe-d049727808fd" containerID="f2dea8c321129a2f70c2b4e2bb700afa4ed814580fa929f3613fc664d7afc648" exitCode=137 Dec 03 12:41:34 crc kubenswrapper[5052]: I1203 12:41:34.837581 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/novacell07298-account-delete-726jv" event={"ID":"6dbc7d8f-34d4-4927-82fe-d049727808fd","Type":"ContainerDied","Data":"f2dea8c321129a2f70c2b4e2bb700afa4ed814580fa929f3613fc664d7afc648"} Dec 03 12:41:34 crc kubenswrapper[5052]: I1203 12:41:34.837599 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/novacell07298-account-delete-726jv" event={"ID":"6dbc7d8f-34d4-4927-82fe-d049727808fd","Type":"ContainerDied","Data":"89e35dabb00061c9ad442c01b1fc1047b590d4300a9f6a3f62d174b78e9185b4"} Dec 03 12:41:34 crc kubenswrapper[5052]: I1203 12:41:34.837633 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/novacell07298-account-delete-726jv" Dec 03 12:41:34 crc kubenswrapper[5052]: I1203 12:41:34.839238 5052 generic.go:334] "Generic (PLEG): container finished" podID="ab29f82e-0693-4001-9f2a-bb4cc2fa1d5a" containerID="6b1c0593b31124275075bff6c57b21a145a59ea4fd8cdabc101e1eae90d84468" exitCode=137 Dec 03 12:41:34 crc kubenswrapper[5052]: I1203 12:41:34.839276 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican1171-account-delete-9hmgl" event={"ID":"ab29f82e-0693-4001-9f2a-bb4cc2fa1d5a","Type":"ContainerDied","Data":"6b1c0593b31124275075bff6c57b21a145a59ea4fd8cdabc101e1eae90d84468"} Dec 03 12:41:34 crc kubenswrapper[5052]: I1203 12:41:34.839291 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican1171-account-delete-9hmgl" event={"ID":"ab29f82e-0693-4001-9f2a-bb4cc2fa1d5a","Type":"ContainerDied","Data":"8b84c689f4cc667ba6eb2f6916edf725df218b56118c0a6591c886c94674990a"} Dec 03 12:41:34 crc kubenswrapper[5052]: I1203 12:41:34.839327 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican1171-account-delete-9hmgl" Dec 03 12:41:34 crc kubenswrapper[5052]: I1203 12:41:34.857951 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder3a0d-account-delete-lxd6l" event={"ID":"5fdc7b7d-f9ed-4d5c-a77b-6a6b0fd186f9","Type":"ContainerDied","Data":"93ea83bcac826673f5de5369123cff8d55921f5215fcc30dac291272c62deb3d"} Dec 03 12:41:34 crc kubenswrapper[5052]: I1203 12:41:34.858023 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder3a0d-account-delete-lxd6l" Dec 03 12:41:34 crc kubenswrapper[5052]: I1203 12:41:34.861296 5052 generic.go:334] "Generic (PLEG): container finished" podID="4b7b82d8-699c-43f2-8ab0-01936305623f" containerID="d01c0d77268e5ca3389f96b721d0db61466ed205622f949655781949d35fbe36" exitCode=137 Dec 03 12:41:34 crc kubenswrapper[5052]: I1203 12:41:34.861341 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/novaapib163-account-delete-8z4bt" event={"ID":"4b7b82d8-699c-43f2-8ab0-01936305623f","Type":"ContainerDied","Data":"d01c0d77268e5ca3389f96b721d0db61466ed205622f949655781949d35fbe36"} Dec 03 12:41:34 crc kubenswrapper[5052]: I1203 12:41:34.861366 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/novaapib163-account-delete-8z4bt" event={"ID":"4b7b82d8-699c-43f2-8ab0-01936305623f","Type":"ContainerDied","Data":"28edaca94f750161a85e242825c026fc53591ffe57899bd5e56e3d735dde155e"} Dec 03 12:41:34 crc kubenswrapper[5052]: I1203 12:41:34.861426 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/novaapib163-account-delete-8z4bt" Dec 03 12:41:34 crc kubenswrapper[5052]: I1203 12:41:34.873674 5052 scope.go:117] "RemoveContainer" containerID="bb53f0acf7c936b9d6eebe4376be498cac662cc7e2d991df4516aca0dbea473e" Dec 03 12:41:34 crc kubenswrapper[5052]: E1203 12:41:34.874161 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bb53f0acf7c936b9d6eebe4376be498cac662cc7e2d991df4516aca0dbea473e\": container with ID starting with bb53f0acf7c936b9d6eebe4376be498cac662cc7e2d991df4516aca0dbea473e not found: ID does not exist" containerID="bb53f0acf7c936b9d6eebe4376be498cac662cc7e2d991df4516aca0dbea473e" Dec 03 12:41:34 crc kubenswrapper[5052]: I1203 12:41:34.874221 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bb53f0acf7c936b9d6eebe4376be498cac662cc7e2d991df4516aca0dbea473e"} err="failed to get container status \"bb53f0acf7c936b9d6eebe4376be498cac662cc7e2d991df4516aca0dbea473e\": rpc error: code = NotFound desc = could not find container \"bb53f0acf7c936b9d6eebe4376be498cac662cc7e2d991df4516aca0dbea473e\": container with ID starting with bb53f0acf7c936b9d6eebe4376be498cac662cc7e2d991df4516aca0dbea473e not found: ID does not exist" Dec 03 12:41:34 crc kubenswrapper[5052]: I1203 12:41:34.874252 5052 scope.go:117] "RemoveContainer" containerID="f2dea8c321129a2f70c2b4e2bb700afa4ed814580fa929f3613fc664d7afc648" Dec 03 12:41:34 crc kubenswrapper[5052]: I1203 12:41:34.886165 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican1171-account-delete-9hmgl"] Dec 03 12:41:34 crc kubenswrapper[5052]: I1203 12:41:34.895369 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/barbican1171-account-delete-9hmgl"] Dec 03 12:41:34 crc kubenswrapper[5052]: I1203 12:41:34.899334 5052 scope.go:117] "RemoveContainer" containerID="f2dea8c321129a2f70c2b4e2bb700afa4ed814580fa929f3613fc664d7afc648" Dec 03 12:41:34 crc kubenswrapper[5052]: E1203 12:41:34.899827 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f2dea8c321129a2f70c2b4e2bb700afa4ed814580fa929f3613fc664d7afc648\": container with ID starting with f2dea8c321129a2f70c2b4e2bb700afa4ed814580fa929f3613fc664d7afc648 not found: ID does not exist" containerID="f2dea8c321129a2f70c2b4e2bb700afa4ed814580fa929f3613fc664d7afc648" Dec 03 12:41:34 crc kubenswrapper[5052]: I1203 12:41:34.899874 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f2dea8c321129a2f70c2b4e2bb700afa4ed814580fa929f3613fc664d7afc648"} err="failed to get container status \"f2dea8c321129a2f70c2b4e2bb700afa4ed814580fa929f3613fc664d7afc648\": rpc error: code = NotFound desc = could not find container \"f2dea8c321129a2f70c2b4e2bb700afa4ed814580fa929f3613fc664d7afc648\": container with ID starting with f2dea8c321129a2f70c2b4e2bb700afa4ed814580fa929f3613fc664d7afc648 not found: ID does not exist" Dec 03 12:41:34 crc kubenswrapper[5052]: I1203 12:41:34.899905 5052 scope.go:117] "RemoveContainer" containerID="6b1c0593b31124275075bff6c57b21a145a59ea4fd8cdabc101e1eae90d84468" Dec 03 12:41:34 crc kubenswrapper[5052]: I1203 12:41:34.902932 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder3a0d-account-delete-lxd6l"] Dec 03 12:41:34 crc kubenswrapper[5052]: I1203 12:41:34.913675 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/cinder3a0d-account-delete-lxd6l"] Dec 03 12:41:34 crc kubenswrapper[5052]: I1203 12:41:34.914354 5052 scope.go:117] "RemoveContainer" containerID="6b1c0593b31124275075bff6c57b21a145a59ea4fd8cdabc101e1eae90d84468" Dec 03 12:41:34 crc kubenswrapper[5052]: E1203 12:41:34.914803 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6b1c0593b31124275075bff6c57b21a145a59ea4fd8cdabc101e1eae90d84468\": container with ID starting with 6b1c0593b31124275075bff6c57b21a145a59ea4fd8cdabc101e1eae90d84468 not found: ID does not exist" containerID="6b1c0593b31124275075bff6c57b21a145a59ea4fd8cdabc101e1eae90d84468" Dec 03 12:41:34 crc kubenswrapper[5052]: I1203 12:41:34.914834 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6b1c0593b31124275075bff6c57b21a145a59ea4fd8cdabc101e1eae90d84468"} err="failed to get container status \"6b1c0593b31124275075bff6c57b21a145a59ea4fd8cdabc101e1eae90d84468\": rpc error: code = NotFound desc = could not find container \"6b1c0593b31124275075bff6c57b21a145a59ea4fd8cdabc101e1eae90d84468\": container with ID starting with 6b1c0593b31124275075bff6c57b21a145a59ea4fd8cdabc101e1eae90d84468 not found: ID does not exist" Dec 03 12:41:34 crc kubenswrapper[5052]: I1203 12:41:34.914874 5052 scope.go:117] "RemoveContainer" containerID="660e428afdfd8a7e278a3dd2d6aa79748d2d7485dda4c8e8678801b9bcc2009e" Dec 03 12:41:34 crc kubenswrapper[5052]: I1203 12:41:34.929867 5052 scope.go:117] "RemoveContainer" containerID="d01c0d77268e5ca3389f96b721d0db61466ed205622f949655781949d35fbe36" Dec 03 12:41:34 crc kubenswrapper[5052]: I1203 12:41:34.939228 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4b7b82d8-699c-43f2-8ab0-01936305623f-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "4b7b82d8-699c-43f2-8ab0-01936305623f" (UID: "4b7b82d8-699c-43f2-8ab0-01936305623f"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:41:34 crc kubenswrapper[5052]: I1203 12:41:34.939300 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/4b7b82d8-699c-43f2-8ab0-01936305623f-operator-scripts\") pod \"4b7b82d8-699c-43f2-8ab0-01936305623f\" (UID: \"4b7b82d8-699c-43f2-8ab0-01936305623f\") " Dec 03 12:41:34 crc kubenswrapper[5052]: I1203 12:41:34.939390 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8snr7\" (UniqueName: \"kubernetes.io/projected/4b7b82d8-699c-43f2-8ab0-01936305623f-kube-api-access-8snr7\") pod \"4b7b82d8-699c-43f2-8ab0-01936305623f\" (UID: \"4b7b82d8-699c-43f2-8ab0-01936305623f\") " Dec 03 12:41:34 crc kubenswrapper[5052]: I1203 12:41:34.939423 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-448x2\" (UniqueName: \"kubernetes.io/projected/eb8b91d5-6c99-45ba-8d33-da99f1dd496c-kube-api-access-448x2\") pod \"eb8b91d5-6c99-45ba-8d33-da99f1dd496c\" (UID: \"eb8b91d5-6c99-45ba-8d33-da99f1dd496c\") " Dec 03 12:41:34 crc kubenswrapper[5052]: I1203 12:41:34.939769 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4tjwn\" (UniqueName: \"kubernetes.io/projected/6dbc7d8f-34d4-4927-82fe-d049727808fd-kube-api-access-4tjwn\") pod \"6dbc7d8f-34d4-4927-82fe-d049727808fd\" (UID: \"6dbc7d8f-34d4-4927-82fe-d049727808fd\") " Dec 03 12:41:34 crc kubenswrapper[5052]: I1203 12:41:34.939825 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/eb8b91d5-6c99-45ba-8d33-da99f1dd496c-operator-scripts\") pod \"eb8b91d5-6c99-45ba-8d33-da99f1dd496c\" (UID: \"eb8b91d5-6c99-45ba-8d33-da99f1dd496c\") " Dec 03 12:41:34 crc kubenswrapper[5052]: I1203 12:41:34.940337 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eb8b91d5-6c99-45ba-8d33-da99f1dd496c-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "eb8b91d5-6c99-45ba-8d33-da99f1dd496c" (UID: "eb8b91d5-6c99-45ba-8d33-da99f1dd496c"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:41:34 crc kubenswrapper[5052]: I1203 12:41:34.940493 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/6dbc7d8f-34d4-4927-82fe-d049727808fd-operator-scripts\") pod \"6dbc7d8f-34d4-4927-82fe-d049727808fd\" (UID: \"6dbc7d8f-34d4-4927-82fe-d049727808fd\") " Dec 03 12:41:34 crc kubenswrapper[5052]: I1203 12:41:34.941076 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/eb8b91d5-6c99-45ba-8d33-da99f1dd496c-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:34 crc kubenswrapper[5052]: I1203 12:41:34.941084 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6dbc7d8f-34d4-4927-82fe-d049727808fd-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "6dbc7d8f-34d4-4927-82fe-d049727808fd" (UID: "6dbc7d8f-34d4-4927-82fe-d049727808fd"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:41:34 crc kubenswrapper[5052]: I1203 12:41:34.941106 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/4b7b82d8-699c-43f2-8ab0-01936305623f-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:34 crc kubenswrapper[5052]: I1203 12:41:34.942714 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eb8b91d5-6c99-45ba-8d33-da99f1dd496c-kube-api-access-448x2" (OuterVolumeSpecName: "kube-api-access-448x2") pod "eb8b91d5-6c99-45ba-8d33-da99f1dd496c" (UID: "eb8b91d5-6c99-45ba-8d33-da99f1dd496c"). InnerVolumeSpecName "kube-api-access-448x2". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:41:34 crc kubenswrapper[5052]: I1203 12:41:34.942821 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4b7b82d8-699c-43f2-8ab0-01936305623f-kube-api-access-8snr7" (OuterVolumeSpecName: "kube-api-access-8snr7") pod "4b7b82d8-699c-43f2-8ab0-01936305623f" (UID: "4b7b82d8-699c-43f2-8ab0-01936305623f"). InnerVolumeSpecName "kube-api-access-8snr7". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:41:34 crc kubenswrapper[5052]: I1203 12:41:34.946741 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6dbc7d8f-34d4-4927-82fe-d049727808fd-kube-api-access-4tjwn" (OuterVolumeSpecName: "kube-api-access-4tjwn") pod "6dbc7d8f-34d4-4927-82fe-d049727808fd" (UID: "6dbc7d8f-34d4-4927-82fe-d049727808fd"). InnerVolumeSpecName "kube-api-access-4tjwn". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:41:34 crc kubenswrapper[5052]: I1203 12:41:34.951772 5052 scope.go:117] "RemoveContainer" containerID="d01c0d77268e5ca3389f96b721d0db61466ed205622f949655781949d35fbe36" Dec 03 12:41:34 crc kubenswrapper[5052]: E1203 12:41:34.952190 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d01c0d77268e5ca3389f96b721d0db61466ed205622f949655781949d35fbe36\": container with ID starting with d01c0d77268e5ca3389f96b721d0db61466ed205622f949655781949d35fbe36 not found: ID does not exist" containerID="d01c0d77268e5ca3389f96b721d0db61466ed205622f949655781949d35fbe36" Dec 03 12:41:34 crc kubenswrapper[5052]: I1203 12:41:34.952216 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d01c0d77268e5ca3389f96b721d0db61466ed205622f949655781949d35fbe36"} err="failed to get container status \"d01c0d77268e5ca3389f96b721d0db61466ed205622f949655781949d35fbe36\": rpc error: code = NotFound desc = could not find container \"d01c0d77268e5ca3389f96b721d0db61466ed205622f949655781949d35fbe36\": container with ID starting with d01c0d77268e5ca3389f96b721d0db61466ed205622f949655781949d35fbe36 not found: ID does not exist" Dec 03 12:41:35 crc kubenswrapper[5052]: I1203 12:41:35.042177 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-8snr7\" (UniqueName: \"kubernetes.io/projected/4b7b82d8-699c-43f2-8ab0-01936305623f-kube-api-access-8snr7\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:35 crc kubenswrapper[5052]: I1203 12:41:35.042209 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-448x2\" (UniqueName: \"kubernetes.io/projected/eb8b91d5-6c99-45ba-8d33-da99f1dd496c-kube-api-access-448x2\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:35 crc kubenswrapper[5052]: I1203 12:41:35.042222 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-4tjwn\" (UniqueName: \"kubernetes.io/projected/6dbc7d8f-34d4-4927-82fe-d049727808fd-kube-api-access-4tjwn\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:35 crc kubenswrapper[5052]: I1203 12:41:35.042235 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/6dbc7d8f-34d4-4927-82fe-d049727808fd-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:35 crc kubenswrapper[5052]: I1203 12:41:35.164560 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/placement7f30-account-delete-vld4v"] Dec 03 12:41:35 crc kubenswrapper[5052]: I1203 12:41:35.173097 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/placement7f30-account-delete-vld4v"] Dec 03 12:41:35 crc kubenswrapper[5052]: I1203 12:41:35.180088 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/novacell07298-account-delete-726jv"] Dec 03 12:41:35 crc kubenswrapper[5052]: I1203 12:41:35.187550 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/novacell07298-account-delete-726jv"] Dec 03 12:41:35 crc kubenswrapper[5052]: I1203 12:41:35.195939 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/novaapib163-account-delete-8z4bt"] Dec 03 12:41:35 crc kubenswrapper[5052]: I1203 12:41:35.203320 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/novaapib163-account-delete-8z4bt"] Dec 03 12:41:35 crc kubenswrapper[5052]: I1203 12:41:35.682646 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4b7b82d8-699c-43f2-8ab0-01936305623f" path="/var/lib/kubelet/pods/4b7b82d8-699c-43f2-8ab0-01936305623f/volumes" Dec 03 12:41:35 crc kubenswrapper[5052]: I1203 12:41:35.683468 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5fdc7b7d-f9ed-4d5c-a77b-6a6b0fd186f9" path="/var/lib/kubelet/pods/5fdc7b7d-f9ed-4d5c-a77b-6a6b0fd186f9/volumes" Dec 03 12:41:35 crc kubenswrapper[5052]: I1203 12:41:35.684077 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6dbc7d8f-34d4-4927-82fe-d049727808fd" path="/var/lib/kubelet/pods/6dbc7d8f-34d4-4927-82fe-d049727808fd/volumes" Dec 03 12:41:35 crc kubenswrapper[5052]: I1203 12:41:35.684541 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ab29f82e-0693-4001-9f2a-bb4cc2fa1d5a" path="/var/lib/kubelet/pods/ab29f82e-0693-4001-9f2a-bb4cc2fa1d5a/volumes" Dec 03 12:41:35 crc kubenswrapper[5052]: I1203 12:41:35.685386 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eb8b91d5-6c99-45ba-8d33-da99f1dd496c" path="/var/lib/kubelet/pods/eb8b91d5-6c99-45ba-8d33-da99f1dd496c/volumes" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.612665 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/rabbitmq-server-0"] Dec 03 12:41:37 crc kubenswrapper[5052]: E1203 12:41:37.612964 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="621935c3-f8f9-43a1-9bc3-6d6ad9caf0ef" containerName="proxy-server" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.612984 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="621935c3-f8f9-43a1-9bc3-6d6ad9caf0ef" containerName="proxy-server" Dec 03 12:41:37 crc kubenswrapper[5052]: E1203 12:41:37.613008 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="d855c655-3b2f-4e4f-9b27-cf4e43a92045" containerName="probe" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.613014 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="d855c655-3b2f-4e4f-9b27-cf4e43a92045" containerName="probe" Dec 03 12:41:37 crc kubenswrapper[5052]: E1203 12:41:37.613027 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="65cb22ea-3a9c-4f99-9128-d3c7681c3173" containerName="ceilometer-central-agent" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.613034 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="65cb22ea-3a9c-4f99-9128-d3c7681c3173" containerName="ceilometer-central-agent" Dec 03 12:41:37 crc kubenswrapper[5052]: E1203 12:41:37.613043 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="30045a80-9e2b-416b-b7bf-cdb9585ec67d" containerName="nova-metadata-metadata" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.613049 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="30045a80-9e2b-416b-b7bf-cdb9585ec67d" containerName="nova-metadata-metadata" Dec 03 12:41:37 crc kubenswrapper[5052]: E1203 12:41:37.613056 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="c7d598df-f336-4798-b381-9b813068f271" containerName="object-server" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.613062 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="c7d598df-f336-4798-b381-9b813068f271" containerName="object-server" Dec 03 12:41:37 crc kubenswrapper[5052]: E1203 12:41:37.613072 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="b0606156-f409-4cae-9be1-15d990f69955" containerName="glance-log" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.613077 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="b0606156-f409-4cae-9be1-15d990f69955" containerName="glance-log" Dec 03 12:41:37 crc kubenswrapper[5052]: E1203 12:41:37.613085 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="f9bac64c-518e-4630-8889-20e09d5b7a39" containerName="setup-container" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.613090 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="f9bac64c-518e-4630-8889-20e09d5b7a39" containerName="setup-container" Dec 03 12:41:37 crc kubenswrapper[5052]: E1203 12:41:37.613097 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="18a62c9b-4032-4639-be99-3520b515313b" containerName="barbican-worker" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.613103 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="18a62c9b-4032-4639-be99-3520b515313b" containerName="barbican-worker" Dec 03 12:41:37 crc kubenswrapper[5052]: E1203 12:41:37.613111 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="c7d598df-f336-4798-b381-9b813068f271" containerName="container-updater" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.613116 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="c7d598df-f336-4798-b381-9b813068f271" containerName="container-updater" Dec 03 12:41:37 crc kubenswrapper[5052]: E1203 12:41:37.613127 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="18a62c9b-4032-4639-be99-3520b515313b" containerName="barbican-worker-log" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.613133 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="18a62c9b-4032-4639-be99-3520b515313b" containerName="barbican-worker-log" Dec 03 12:41:37 crc kubenswrapper[5052]: E1203 12:41:37.613142 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="f89ce0f5-5a92-49eb-9951-f2b491ac2139" containerName="mysql-bootstrap" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.613147 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="f89ce0f5-5a92-49eb-9951-f2b491ac2139" containerName="mysql-bootstrap" Dec 03 12:41:37 crc kubenswrapper[5052]: E1203 12:41:37.613157 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="46c590ef-2b7c-412d-8582-dfdc311e6778" containerName="ovsdbserver-nb" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.613163 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="46c590ef-2b7c-412d-8582-dfdc311e6778" containerName="ovsdbserver-nb" Dec 03 12:41:37 crc kubenswrapper[5052]: E1203 12:41:37.613170 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="c7d598df-f336-4798-b381-9b813068f271" containerName="swift-recon-cron" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.613176 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="c7d598df-f336-4798-b381-9b813068f271" containerName="swift-recon-cron" Dec 03 12:41:37 crc kubenswrapper[5052]: E1203 12:41:37.613182 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="d82ef8ea-0c3b-4d89-9c78-22095940f4da" containerName="setup-container" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.613190 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="d82ef8ea-0c3b-4d89-9c78-22095940f4da" containerName="setup-container" Dec 03 12:41:37 crc kubenswrapper[5052]: E1203 12:41:37.613199 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="4b7b82d8-699c-43f2-8ab0-01936305623f" containerName="mariadb-account-delete" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.613204 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="4b7b82d8-699c-43f2-8ab0-01936305623f" containerName="mariadb-account-delete" Dec 03 12:41:37 crc kubenswrapper[5052]: E1203 12:41:37.613211 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="c7d598df-f336-4798-b381-9b813068f271" containerName="container-replicator" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.613217 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="c7d598df-f336-4798-b381-9b813068f271" containerName="container-replicator" Dec 03 12:41:37 crc kubenswrapper[5052]: E1203 12:41:37.613225 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="99a0961b-8ed8-49b7-abac-5025d7b351ee" containerName="placement-log" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.613231 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="99a0961b-8ed8-49b7-abac-5025d7b351ee" containerName="placement-log" Dec 03 12:41:37 crc kubenswrapper[5052]: E1203 12:41:37.613237 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="c7d598df-f336-4798-b381-9b813068f271" containerName="account-server" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.613242 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="c7d598df-f336-4798-b381-9b813068f271" containerName="account-server" Dec 03 12:41:37 crc kubenswrapper[5052]: E1203 12:41:37.613254 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="bc0b4316-9c87-4c72-bc06-b4b526c2070e" containerName="barbican-keystone-listener-log" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.613260 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="bc0b4316-9c87-4c72-bc06-b4b526c2070e" containerName="barbican-keystone-listener-log" Dec 03 12:41:37 crc kubenswrapper[5052]: E1203 12:41:37.613268 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="c7d598df-f336-4798-b381-9b813068f271" containerName="container-auditor" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.613273 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="c7d598df-f336-4798-b381-9b813068f271" containerName="container-auditor" Dec 03 12:41:37 crc kubenswrapper[5052]: E1203 12:41:37.613280 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="5bca7a3b-316e-4357-8ae4-e47be319dbf1" containerName="cinder-api-log" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.613286 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="5bca7a3b-316e-4357-8ae4-e47be319dbf1" containerName="cinder-api-log" Dec 03 12:41:37 crc kubenswrapper[5052]: E1203 12:41:37.613295 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="bc0b4316-9c87-4c72-bc06-b4b526c2070e" containerName="barbican-keystone-listener" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.613301 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="bc0b4316-9c87-4c72-bc06-b4b526c2070e" containerName="barbican-keystone-listener" Dec 03 12:41:37 crc kubenswrapper[5052]: E1203 12:41:37.613311 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="99a0961b-8ed8-49b7-abac-5025d7b351ee" containerName="placement-api" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.613318 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="99a0961b-8ed8-49b7-abac-5025d7b351ee" containerName="placement-api" Dec 03 12:41:37 crc kubenswrapper[5052]: E1203 12:41:37.613329 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="5fdc7b7d-f9ed-4d5c-a77b-6a6b0fd186f9" containerName="mariadb-account-delete" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.613335 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="5fdc7b7d-f9ed-4d5c-a77b-6a6b0fd186f9" containerName="mariadb-account-delete" Dec 03 12:41:37 crc kubenswrapper[5052]: E1203 12:41:37.613341 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="a7ae5e02-8418-40e2-b65d-7b74625fe5b3" containerName="galera" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.613346 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="a7ae5e02-8418-40e2-b65d-7b74625fe5b3" containerName="galera" Dec 03 12:41:37 crc kubenswrapper[5052]: E1203 12:41:37.613355 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="fdeb105e-9fbc-44bb-ba15-bc81c7275db1" containerName="keystone-api" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.613363 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="fdeb105e-9fbc-44bb-ba15-bc81c7275db1" containerName="keystone-api" Dec 03 12:41:37 crc kubenswrapper[5052]: E1203 12:41:37.613370 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="ab29f82e-0693-4001-9f2a-bb4cc2fa1d5a" containerName="mariadb-account-delete" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.613376 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="ab29f82e-0693-4001-9f2a-bb4cc2fa1d5a" containerName="mariadb-account-delete" Dec 03 12:41:37 crc kubenswrapper[5052]: E1203 12:41:37.613387 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="d855c655-3b2f-4e4f-9b27-cf4e43a92045" containerName="cinder-scheduler" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.613392 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="d855c655-3b2f-4e4f-9b27-cf4e43a92045" containerName="cinder-scheduler" Dec 03 12:41:37 crc kubenswrapper[5052]: E1203 12:41:37.613400 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="d82ef8ea-0c3b-4d89-9c78-22095940f4da" containerName="rabbitmq" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.613406 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="d82ef8ea-0c3b-4d89-9c78-22095940f4da" containerName="rabbitmq" Dec 03 12:41:37 crc kubenswrapper[5052]: E1203 12:41:37.613414 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="0f16f521-a181-40e4-a1ed-38f4c8862ffb" containerName="nova-cell1-novncproxy-novncproxy" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.613420 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="0f16f521-a181-40e4-a1ed-38f4c8862ffb" containerName="nova-cell1-novncproxy-novncproxy" Dec 03 12:41:37 crc kubenswrapper[5052]: E1203 12:41:37.613431 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="9e3bfbf3-a39c-48d2-bb1d-062c90c0ef10" containerName="nova-scheduler-scheduler" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.613437 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="9e3bfbf3-a39c-48d2-bb1d-062c90c0ef10" containerName="nova-scheduler-scheduler" Dec 03 12:41:37 crc kubenswrapper[5052]: E1203 12:41:37.613445 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="ddfa4a3f-b868-49b7-abec-62cb3ffde2bd" containerName="neutron-api" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.613451 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="ddfa4a3f-b868-49b7-abec-62cb3ffde2bd" containerName="neutron-api" Dec 03 12:41:37 crc kubenswrapper[5052]: E1203 12:41:37.613458 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="e9af392d-1940-4203-961e-39da1ac5f3da" containerName="nova-cell0-conductor-conductor" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.613464 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="e9af392d-1940-4203-961e-39da1ac5f3da" containerName="nova-cell0-conductor-conductor" Dec 03 12:41:37 crc kubenswrapper[5052]: E1203 12:41:37.613471 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="46c590ef-2b7c-412d-8582-dfdc311e6778" containerName="openstack-network-exporter" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.613476 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="46c590ef-2b7c-412d-8582-dfdc311e6778" containerName="openstack-network-exporter" Dec 03 12:41:37 crc kubenswrapper[5052]: E1203 12:41:37.613484 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="6dbc7d8f-34d4-4927-82fe-d049727808fd" containerName="mariadb-account-delete" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.613489 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="6dbc7d8f-34d4-4927-82fe-d049727808fd" containerName="mariadb-account-delete" Dec 03 12:41:37 crc kubenswrapper[5052]: E1203 12:41:37.613496 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="1fcf641b-a032-4179-9dde-ad76e9d86925" containerName="dnsmasq-dns" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.613502 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="1fcf641b-a032-4179-9dde-ad76e9d86925" containerName="dnsmasq-dns" Dec 03 12:41:37 crc kubenswrapper[5052]: E1203 12:41:37.613574 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="65cb22ea-3a9c-4f99-9128-d3c7681c3173" containerName="ceilometer-notification-agent" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.613581 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="65cb22ea-3a9c-4f99-9128-d3c7681c3173" containerName="ceilometer-notification-agent" Dec 03 12:41:37 crc kubenswrapper[5052]: E1203 12:41:37.613591 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="ddfa4a3f-b868-49b7-abec-62cb3ffde2bd" containerName="neutron-httpd" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.613598 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="ddfa4a3f-b868-49b7-abec-62cb3ffde2bd" containerName="neutron-httpd" Dec 03 12:41:37 crc kubenswrapper[5052]: E1203 12:41:37.613608 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="a83592aa-a4c1-459e-9d07-f14b390a5e6b" containerName="barbican-api-log" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.613615 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="a83592aa-a4c1-459e-9d07-f14b390a5e6b" containerName="barbican-api-log" Dec 03 12:41:37 crc kubenswrapper[5052]: E1203 12:41:37.613626 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="d6bedb3e-52e6-44a9-afd9-a164b0730f52" containerName="openstack-network-exporter" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.613634 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="d6bedb3e-52e6-44a9-afd9-a164b0730f52" containerName="openstack-network-exporter" Dec 03 12:41:37 crc kubenswrapper[5052]: E1203 12:41:37.613641 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="3039127c-0211-4430-9b2b-426412241ce4" containerName="glance-httpd" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.613650 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="3039127c-0211-4430-9b2b-426412241ce4" containerName="glance-httpd" Dec 03 12:41:37 crc kubenswrapper[5052]: E1203 12:41:37.613663 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="a83592aa-a4c1-459e-9d07-f14b390a5e6b" containerName="barbican-api" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.613671 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="a83592aa-a4c1-459e-9d07-f14b390a5e6b" containerName="barbican-api" Dec 03 12:41:37 crc kubenswrapper[5052]: E1203 12:41:37.613680 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="621935c3-f8f9-43a1-9bc3-6d6ad9caf0ef" containerName="proxy-httpd" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.613687 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="621935c3-f8f9-43a1-9bc3-6d6ad9caf0ef" containerName="proxy-httpd" Dec 03 12:41:37 crc kubenswrapper[5052]: E1203 12:41:37.613698 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="b7defe22-28b4-4fec-8fe5-ebde8e658c6b" containerName="kube-state-metrics" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.613705 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="b7defe22-28b4-4fec-8fe5-ebde8e658c6b" containerName="kube-state-metrics" Dec 03 12:41:37 crc kubenswrapper[5052]: E1203 12:41:37.613718 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="b0606156-f409-4cae-9be1-15d990f69955" containerName="glance-httpd" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.613725 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="b0606156-f409-4cae-9be1-15d990f69955" containerName="glance-httpd" Dec 03 12:41:37 crc kubenswrapper[5052]: E1203 12:41:37.613736 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="c7d598df-f336-4798-b381-9b813068f271" containerName="account-reaper" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.613742 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="c7d598df-f336-4798-b381-9b813068f271" containerName="account-reaper" Dec 03 12:41:37 crc kubenswrapper[5052]: E1203 12:41:37.613752 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="eb8b91d5-6c99-45ba-8d33-da99f1dd496c" containerName="mariadb-account-delete" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.613759 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="eb8b91d5-6c99-45ba-8d33-da99f1dd496c" containerName="mariadb-account-delete" Dec 03 12:41:37 crc kubenswrapper[5052]: E1203 12:41:37.613770 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="5bca7a3b-316e-4357-8ae4-e47be319dbf1" containerName="cinder-api" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.613775 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="5bca7a3b-316e-4357-8ae4-e47be319dbf1" containerName="cinder-api" Dec 03 12:41:37 crc kubenswrapper[5052]: E1203 12:41:37.613785 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="1fcf641b-a032-4179-9dde-ad76e9d86925" containerName="init" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.613790 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="1fcf641b-a032-4179-9dde-ad76e9d86925" containerName="init" Dec 03 12:41:37 crc kubenswrapper[5052]: E1203 12:41:37.613797 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="50789f13-8194-4e24-b975-512591b816a5" containerName="nova-api-api" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.613803 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="50789f13-8194-4e24-b975-512591b816a5" containerName="nova-api-api" Dec 03 12:41:37 crc kubenswrapper[5052]: E1203 12:41:37.613812 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="c7d598df-f336-4798-b381-9b813068f271" containerName="account-auditor" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.613818 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="c7d598df-f336-4798-b381-9b813068f271" containerName="account-auditor" Dec 03 12:41:37 crc kubenswrapper[5052]: E1203 12:41:37.613827 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="d855c655-3b2f-4e4f-9b27-cf4e43a92045" containerName="cinder-scheduler" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.613833 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="d855c655-3b2f-4e4f-9b27-cf4e43a92045" containerName="cinder-scheduler" Dec 03 12:41:37 crc kubenswrapper[5052]: E1203 12:41:37.613840 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="c7d598df-f336-4798-b381-9b813068f271" containerName="container-server" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.613845 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="c7d598df-f336-4798-b381-9b813068f271" containerName="container-server" Dec 03 12:41:37 crc kubenswrapper[5052]: E1203 12:41:37.613853 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="c7d598df-f336-4798-b381-9b813068f271" containerName="account-replicator" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.613859 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="c7d598df-f336-4798-b381-9b813068f271" containerName="account-replicator" Dec 03 12:41:37 crc kubenswrapper[5052]: E1203 12:41:37.613870 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="c7d598df-f336-4798-b381-9b813068f271" containerName="rsync" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.613876 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="c7d598df-f336-4798-b381-9b813068f271" containerName="rsync" Dec 03 12:41:37 crc kubenswrapper[5052]: E1203 12:41:37.613882 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="65cb22ea-3a9c-4f99-9128-d3c7681c3173" containerName="sg-core" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.613888 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="65cb22ea-3a9c-4f99-9128-d3c7681c3173" containerName="sg-core" Dec 03 12:41:37 crc kubenswrapper[5052]: E1203 12:41:37.613897 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="0556e9f0-e431-4193-9822-6079cb8a1e44" containerName="nova-cell1-conductor-conductor" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.613904 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="0556e9f0-e431-4193-9822-6079cb8a1e44" containerName="nova-cell1-conductor-conductor" Dec 03 12:41:37 crc kubenswrapper[5052]: E1203 12:41:37.613914 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="50789f13-8194-4e24-b975-512591b816a5" containerName="nova-api-log" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.613922 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="50789f13-8194-4e24-b975-512591b816a5" containerName="nova-api-log" Dec 03 12:41:37 crc kubenswrapper[5052]: E1203 12:41:37.613933 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="b7c49d90-c0af-4d4b-a080-8c7dce3ee66a" containerName="mariadb-account-delete" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.613940 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="b7c49d90-c0af-4d4b-a080-8c7dce3ee66a" containerName="mariadb-account-delete" Dec 03 12:41:37 crc kubenswrapper[5052]: E1203 12:41:37.613949 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="d6bedb3e-52e6-44a9-afd9-a164b0730f52" containerName="ovsdbserver-sb" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.613957 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="d6bedb3e-52e6-44a9-afd9-a164b0730f52" containerName="ovsdbserver-sb" Dec 03 12:41:37 crc kubenswrapper[5052]: E1203 12:41:37.613964 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="5d19811b-4e82-42b2-b6ec-f16e707484f2" containerName="memcached" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.613971 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="5d19811b-4e82-42b2-b6ec-f16e707484f2" containerName="memcached" Dec 03 12:41:37 crc kubenswrapper[5052]: E1203 12:41:37.613978 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="a7ae5e02-8418-40e2-b65d-7b74625fe5b3" containerName="mysql-bootstrap" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.613984 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="a7ae5e02-8418-40e2-b65d-7b74625fe5b3" containerName="mysql-bootstrap" Dec 03 12:41:37 crc kubenswrapper[5052]: E1203 12:41:37.613991 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="c7d598df-f336-4798-b381-9b813068f271" containerName="object-replicator" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.613999 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="c7d598df-f336-4798-b381-9b813068f271" containerName="object-replicator" Dec 03 12:41:37 crc kubenswrapper[5052]: E1203 12:41:37.614009 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="c7d598df-f336-4798-b381-9b813068f271" containerName="object-auditor" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.614016 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="c7d598df-f336-4798-b381-9b813068f271" containerName="object-auditor" Dec 03 12:41:37 crc kubenswrapper[5052]: E1203 12:41:37.614023 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="12f52069-8b08-4e4f-a3c1-a5421efb5844" containerName="mariadb-account-delete" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.614030 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="12f52069-8b08-4e4f-a3c1-a5421efb5844" containerName="mariadb-account-delete" Dec 03 12:41:37 crc kubenswrapper[5052]: E1203 12:41:37.614040 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="3039127c-0211-4430-9b2b-426412241ce4" containerName="glance-log" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.614046 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="3039127c-0211-4430-9b2b-426412241ce4" containerName="glance-log" Dec 03 12:41:37 crc kubenswrapper[5052]: E1203 12:41:37.614054 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="ea4cd237-7ec6-49f7-9afd-1ea159b9b23b" containerName="openstack-network-exporter" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.614060 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="ea4cd237-7ec6-49f7-9afd-1ea159b9b23b" containerName="openstack-network-exporter" Dec 03 12:41:37 crc kubenswrapper[5052]: E1203 12:41:37.614070 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="c7d598df-f336-4798-b381-9b813068f271" containerName="object-updater" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.614077 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="c7d598df-f336-4798-b381-9b813068f271" containerName="object-updater" Dec 03 12:41:37 crc kubenswrapper[5052]: E1203 12:41:37.614088 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="c7d598df-f336-4798-b381-9b813068f271" containerName="object-expirer" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.614094 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="c7d598df-f336-4798-b381-9b813068f271" containerName="object-expirer" Dec 03 12:41:37 crc kubenswrapper[5052]: E1203 12:41:37.614102 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="30045a80-9e2b-416b-b7bf-cdb9585ec67d" containerName="nova-metadata-log" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.614108 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="30045a80-9e2b-416b-b7bf-cdb9585ec67d" containerName="nova-metadata-log" Dec 03 12:41:37 crc kubenswrapper[5052]: E1203 12:41:37.614116 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="65cb22ea-3a9c-4f99-9128-d3c7681c3173" containerName="proxy-httpd" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.614122 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="65cb22ea-3a9c-4f99-9128-d3c7681c3173" containerName="proxy-httpd" Dec 03 12:41:37 crc kubenswrapper[5052]: E1203 12:41:37.614130 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="f89ce0f5-5a92-49eb-9951-f2b491ac2139" containerName="galera" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.614136 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="f89ce0f5-5a92-49eb-9951-f2b491ac2139" containerName="galera" Dec 03 12:41:37 crc kubenswrapper[5052]: E1203 12:41:37.614142 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="ea4cd237-7ec6-49f7-9afd-1ea159b9b23b" containerName="ovn-northd" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.614147 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="ea4cd237-7ec6-49f7-9afd-1ea159b9b23b" containerName="ovn-northd" Dec 03 12:41:37 crc kubenswrapper[5052]: E1203 12:41:37.614156 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="f9bac64c-518e-4630-8889-20e09d5b7a39" containerName="rabbitmq" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.614163 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="f9bac64c-518e-4630-8889-20e09d5b7a39" containerName="rabbitmq" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.614289 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="d855c655-3b2f-4e4f-9b27-cf4e43a92045" containerName="cinder-scheduler" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.614303 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="c7d598df-f336-4798-b381-9b813068f271" containerName="object-updater" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.614312 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="ea4cd237-7ec6-49f7-9afd-1ea159b9b23b" containerName="ovn-northd" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.614323 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="6dbc7d8f-34d4-4927-82fe-d049727808fd" containerName="mariadb-account-delete" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.614332 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="3039127c-0211-4430-9b2b-426412241ce4" containerName="glance-httpd" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.614341 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="c7d598df-f336-4798-b381-9b813068f271" containerName="swift-recon-cron" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.614349 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="c7d598df-f336-4798-b381-9b813068f271" containerName="object-expirer" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.614355 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="c7d598df-f336-4798-b381-9b813068f271" containerName="account-reaper" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.614365 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="46c590ef-2b7c-412d-8582-dfdc311e6778" containerName="ovsdbserver-nb" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.614371 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="c7d598df-f336-4798-b381-9b813068f271" containerName="container-updater" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.614379 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="c7d598df-f336-4798-b381-9b813068f271" containerName="account-auditor" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.614388 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="9e3bfbf3-a39c-48d2-bb1d-062c90c0ef10" containerName="nova-scheduler-scheduler" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.614396 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="c7d598df-f336-4798-b381-9b813068f271" containerName="object-replicator" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.614405 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="621935c3-f8f9-43a1-9bc3-6d6ad9caf0ef" containerName="proxy-httpd" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.614416 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="30045a80-9e2b-416b-b7bf-cdb9585ec67d" containerName="nova-metadata-log" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.614425 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="b7c49d90-c0af-4d4b-a080-8c7dce3ee66a" containerName="mariadb-account-delete" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.614433 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="c7d598df-f336-4798-b381-9b813068f271" containerName="container-auditor" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.614440 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="1fcf641b-a032-4179-9dde-ad76e9d86925" containerName="dnsmasq-dns" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.614449 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="46c590ef-2b7c-412d-8582-dfdc311e6778" containerName="openstack-network-exporter" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.614458 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="12f52069-8b08-4e4f-a3c1-a5421efb5844" containerName="mariadb-account-delete" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.614464 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="c7d598df-f336-4798-b381-9b813068f271" containerName="object-server" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.614473 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="d855c655-3b2f-4e4f-9b27-cf4e43a92045" containerName="probe" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.614482 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="bc0b4316-9c87-4c72-bc06-b4b526c2070e" containerName="barbican-keystone-listener-log" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.614491 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="d855c655-3b2f-4e4f-9b27-cf4e43a92045" containerName="cinder-scheduler" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.614499 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="99a0961b-8ed8-49b7-abac-5025d7b351ee" containerName="placement-log" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.614541 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="621935c3-f8f9-43a1-9bc3-6d6ad9caf0ef" containerName="proxy-server" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.614551 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="c7d598df-f336-4798-b381-9b813068f271" containerName="account-server" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.614561 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="eb8b91d5-6c99-45ba-8d33-da99f1dd496c" containerName="mariadb-account-delete" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.614570 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="f89ce0f5-5a92-49eb-9951-f2b491ac2139" containerName="galera" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.614578 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="c7d598df-f336-4798-b381-9b813068f271" containerName="container-server" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.614586 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="3039127c-0211-4430-9b2b-426412241ce4" containerName="glance-log" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.614591 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="e9af392d-1940-4203-961e-39da1ac5f3da" containerName="nova-cell0-conductor-conductor" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.614601 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="5bca7a3b-316e-4357-8ae4-e47be319dbf1" containerName="cinder-api-log" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.614611 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="a7ae5e02-8418-40e2-b65d-7b74625fe5b3" containerName="galera" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.614620 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="18a62c9b-4032-4639-be99-3520b515313b" containerName="barbican-worker-log" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.614628 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="d82ef8ea-0c3b-4d89-9c78-22095940f4da" containerName="rabbitmq" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.614636 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="65cb22ea-3a9c-4f99-9128-d3c7681c3173" containerName="sg-core" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.614643 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="d6bedb3e-52e6-44a9-afd9-a164b0730f52" containerName="ovsdbserver-sb" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.614653 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="ddfa4a3f-b868-49b7-abec-62cb3ffde2bd" containerName="neutron-api" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.614659 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="30045a80-9e2b-416b-b7bf-cdb9585ec67d" containerName="nova-metadata-metadata" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.614666 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="c7d598df-f336-4798-b381-9b813068f271" containerName="rsync" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.614673 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="0556e9f0-e431-4193-9822-6079cb8a1e44" containerName="nova-cell1-conductor-conductor" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.614680 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="65cb22ea-3a9c-4f99-9128-d3c7681c3173" containerName="proxy-httpd" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.614688 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="a83592aa-a4c1-459e-9d07-f14b390a5e6b" containerName="barbican-api-log" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.614696 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="b0606156-f409-4cae-9be1-15d990f69955" containerName="glance-httpd" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.614705 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="50789f13-8194-4e24-b975-512591b816a5" containerName="nova-api-log" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.614713 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="c7d598df-f336-4798-b381-9b813068f271" containerName="container-replicator" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.614719 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="d6bedb3e-52e6-44a9-afd9-a164b0730f52" containerName="openstack-network-exporter" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.614728 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="5d19811b-4e82-42b2-b6ec-f16e707484f2" containerName="memcached" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.614735 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="c7d598df-f336-4798-b381-9b813068f271" containerName="account-replicator" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.614744 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="ddfa4a3f-b868-49b7-abec-62cb3ffde2bd" containerName="neutron-httpd" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.614753 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="65cb22ea-3a9c-4f99-9128-d3c7681c3173" containerName="ceilometer-central-agent" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.614760 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="c7d598df-f336-4798-b381-9b813068f271" containerName="object-auditor" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.614769 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="ab29f82e-0693-4001-9f2a-bb4cc2fa1d5a" containerName="mariadb-account-delete" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.614777 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="50789f13-8194-4e24-b975-512591b816a5" containerName="nova-api-api" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.614787 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="ea4cd237-7ec6-49f7-9afd-1ea159b9b23b" containerName="openstack-network-exporter" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.614792 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="65cb22ea-3a9c-4f99-9128-d3c7681c3173" containerName="ceilometer-notification-agent" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.614800 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="a83592aa-a4c1-459e-9d07-f14b390a5e6b" containerName="barbican-api" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.614808 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="99a0961b-8ed8-49b7-abac-5025d7b351ee" containerName="placement-api" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.614816 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="5fdc7b7d-f9ed-4d5c-a77b-6a6b0fd186f9" containerName="mariadb-account-delete" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.614825 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="4b7b82d8-699c-43f2-8ab0-01936305623f" containerName="mariadb-account-delete" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.614836 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="5bca7a3b-316e-4357-8ae4-e47be319dbf1" containerName="cinder-api" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.614842 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="b0606156-f409-4cae-9be1-15d990f69955" containerName="glance-log" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.614851 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="18a62c9b-4032-4639-be99-3520b515313b" containerName="barbican-worker" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.614861 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="f9bac64c-518e-4630-8889-20e09d5b7a39" containerName="rabbitmq" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.614868 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="fdeb105e-9fbc-44bb-ba15-bc81c7275db1" containerName="keystone-api" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.614876 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="bc0b4316-9c87-4c72-bc06-b4b526c2070e" containerName="barbican-keystone-listener" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.614884 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="b7defe22-28b4-4fec-8fe5-ebde8e658c6b" containerName="kube-state-metrics" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.614893 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="0f16f521-a181-40e4-a1ed-38f4c8862ffb" containerName="nova-cell1-novncproxy-novncproxy" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.615683 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.618317 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"rabbitmq-plugins-conf" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.618349 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"rabbitmq-config-data" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.618621 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"rabbitmq-server-conf" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.619113 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"rabbitmq-default-user" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.620340 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"rabbitmq-server-dockercfg-5v9sp" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.620345 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"rabbitmq-erlang-cookie" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.621600 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-rabbitmq-svc" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.626612 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/rabbitmq-server-0"] Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.738283 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"rabbitmq-confd\" (UniqueName: \"kubernetes.io/projected/0cb33a40-6f0a-4e89-a159-4edd3af3a2fa-rabbitmq-confd\") pod \"rabbitmq-server-0\" (UID: \"0cb33a40-6f0a-4e89-a159-4edd3af3a2fa\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.738343 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"rabbitmq-plugins\" (UniqueName: \"kubernetes.io/empty-dir/0cb33a40-6f0a-4e89-a159-4edd3af3a2fa-rabbitmq-plugins\") pod \"rabbitmq-server-0\" (UID: \"0cb33a40-6f0a-4e89-a159-4edd3af3a2fa\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.738399 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"server-conf\" (UniqueName: \"kubernetes.io/configmap/0cb33a40-6f0a-4e89-a159-4edd3af3a2fa-server-conf\") pod \"rabbitmq-server-0\" (UID: \"0cb33a40-6f0a-4e89-a159-4edd3af3a2fa\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.738437 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"rabbitmq-erlang-cookie\" (UniqueName: \"kubernetes.io/empty-dir/0cb33a40-6f0a-4e89-a159-4edd3af3a2fa-rabbitmq-erlang-cookie\") pod \"rabbitmq-server-0\" (UID: \"0cb33a40-6f0a-4e89-a159-4edd3af3a2fa\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.738481 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"erlang-cookie-secret\" (UniqueName: \"kubernetes.io/secret/0cb33a40-6f0a-4e89-a159-4edd3af3a2fa-erlang-cookie-secret\") pod \"rabbitmq-server-0\" (UID: \"0cb33a40-6f0a-4e89-a159-4edd3af3a2fa\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.738503 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage14-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage14-crc\") pod \"rabbitmq-server-0\" (UID: \"0cb33a40-6f0a-4e89-a159-4edd3af3a2fa\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.738542 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"pod-info\" (UniqueName: \"kubernetes.io/downward-api/0cb33a40-6f0a-4e89-a159-4edd3af3a2fa-pod-info\") pod \"rabbitmq-server-0\" (UID: \"0cb33a40-6f0a-4e89-a159-4edd3af3a2fa\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.738569 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-conf\" (UniqueName: \"kubernetes.io/configmap/0cb33a40-6f0a-4e89-a159-4edd3af3a2fa-plugins-conf\") pod \"rabbitmq-server-0\" (UID: \"0cb33a40-6f0a-4e89-a159-4edd3af3a2fa\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.738620 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"rabbitmq-tls\" (UniqueName: \"kubernetes.io/projected/0cb33a40-6f0a-4e89-a159-4edd3af3a2fa-rabbitmq-tls\") pod \"rabbitmq-server-0\" (UID: \"0cb33a40-6f0a-4e89-a159-4edd3af3a2fa\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.738642 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gblmn\" (UniqueName: \"kubernetes.io/projected/0cb33a40-6f0a-4e89-a159-4edd3af3a2fa-kube-api-access-gblmn\") pod \"rabbitmq-server-0\" (UID: \"0cb33a40-6f0a-4e89-a159-4edd3af3a2fa\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.738670 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/0cb33a40-6f0a-4e89-a159-4edd3af3a2fa-config-data\") pod \"rabbitmq-server-0\" (UID: \"0cb33a40-6f0a-4e89-a159-4edd3af3a2fa\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.840088 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"plugins-conf\" (UniqueName: \"kubernetes.io/configmap/0cb33a40-6f0a-4e89-a159-4edd3af3a2fa-plugins-conf\") pod \"rabbitmq-server-0\" (UID: \"0cb33a40-6f0a-4e89-a159-4edd3af3a2fa\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.840622 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"rabbitmq-tls\" (UniqueName: \"kubernetes.io/projected/0cb33a40-6f0a-4e89-a159-4edd3af3a2fa-rabbitmq-tls\") pod \"rabbitmq-server-0\" (UID: \"0cb33a40-6f0a-4e89-a159-4edd3af3a2fa\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.840695 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-gblmn\" (UniqueName: \"kubernetes.io/projected/0cb33a40-6f0a-4e89-a159-4edd3af3a2fa-kube-api-access-gblmn\") pod \"rabbitmq-server-0\" (UID: \"0cb33a40-6f0a-4e89-a159-4edd3af3a2fa\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.841163 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"plugins-conf\" (UniqueName: \"kubernetes.io/configmap/0cb33a40-6f0a-4e89-a159-4edd3af3a2fa-plugins-conf\") pod \"rabbitmq-server-0\" (UID: \"0cb33a40-6f0a-4e89-a159-4edd3af3a2fa\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.841443 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/0cb33a40-6f0a-4e89-a159-4edd3af3a2fa-config-data\") pod \"rabbitmq-server-0\" (UID: \"0cb33a40-6f0a-4e89-a159-4edd3af3a2fa\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.841624 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"rabbitmq-confd\" (UniqueName: \"kubernetes.io/projected/0cb33a40-6f0a-4e89-a159-4edd3af3a2fa-rabbitmq-confd\") pod \"rabbitmq-server-0\" (UID: \"0cb33a40-6f0a-4e89-a159-4edd3af3a2fa\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.841740 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"rabbitmq-plugins\" (UniqueName: \"kubernetes.io/empty-dir/0cb33a40-6f0a-4e89-a159-4edd3af3a2fa-rabbitmq-plugins\") pod \"rabbitmq-server-0\" (UID: \"0cb33a40-6f0a-4e89-a159-4edd3af3a2fa\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.841871 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"server-conf\" (UniqueName: \"kubernetes.io/configmap/0cb33a40-6f0a-4e89-a159-4edd3af3a2fa-server-conf\") pod \"rabbitmq-server-0\" (UID: \"0cb33a40-6f0a-4e89-a159-4edd3af3a2fa\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.841908 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"rabbitmq-erlang-cookie\" (UniqueName: \"kubernetes.io/empty-dir/0cb33a40-6f0a-4e89-a159-4edd3af3a2fa-rabbitmq-erlang-cookie\") pod \"rabbitmq-server-0\" (UID: \"0cb33a40-6f0a-4e89-a159-4edd3af3a2fa\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.841989 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"erlang-cookie-secret\" (UniqueName: \"kubernetes.io/secret/0cb33a40-6f0a-4e89-a159-4edd3af3a2fa-erlang-cookie-secret\") pod \"rabbitmq-server-0\" (UID: \"0cb33a40-6f0a-4e89-a159-4edd3af3a2fa\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.842026 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage14-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage14-crc\") pod \"rabbitmq-server-0\" (UID: \"0cb33a40-6f0a-4e89-a159-4edd3af3a2fa\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.842073 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"pod-info\" (UniqueName: \"kubernetes.io/downward-api/0cb33a40-6f0a-4e89-a159-4edd3af3a2fa-pod-info\") pod \"rabbitmq-server-0\" (UID: \"0cb33a40-6f0a-4e89-a159-4edd3af3a2fa\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.842338 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/0cb33a40-6f0a-4e89-a159-4edd3af3a2fa-config-data\") pod \"rabbitmq-server-0\" (UID: \"0cb33a40-6f0a-4e89-a159-4edd3af3a2fa\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.842703 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage14-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage14-crc\") pod \"rabbitmq-server-0\" (UID: \"0cb33a40-6f0a-4e89-a159-4edd3af3a2fa\") device mount path \"/mnt/openstack/pv14\"" pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.843049 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"server-conf\" (UniqueName: \"kubernetes.io/configmap/0cb33a40-6f0a-4e89-a159-4edd3af3a2fa-server-conf\") pod \"rabbitmq-server-0\" (UID: \"0cb33a40-6f0a-4e89-a159-4edd3af3a2fa\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.843313 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"rabbitmq-plugins\" (UniqueName: \"kubernetes.io/empty-dir/0cb33a40-6f0a-4e89-a159-4edd3af3a2fa-rabbitmq-plugins\") pod \"rabbitmq-server-0\" (UID: \"0cb33a40-6f0a-4e89-a159-4edd3af3a2fa\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.846435 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"rabbitmq-erlang-cookie\" (UniqueName: \"kubernetes.io/empty-dir/0cb33a40-6f0a-4e89-a159-4edd3af3a2fa-rabbitmq-erlang-cookie\") pod \"rabbitmq-server-0\" (UID: \"0cb33a40-6f0a-4e89-a159-4edd3af3a2fa\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.848073 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"pod-info\" (UniqueName: \"kubernetes.io/downward-api/0cb33a40-6f0a-4e89-a159-4edd3af3a2fa-pod-info\") pod \"rabbitmq-server-0\" (UID: \"0cb33a40-6f0a-4e89-a159-4edd3af3a2fa\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.848335 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"rabbitmq-confd\" (UniqueName: \"kubernetes.io/projected/0cb33a40-6f0a-4e89-a159-4edd3af3a2fa-rabbitmq-confd\") pod \"rabbitmq-server-0\" (UID: \"0cb33a40-6f0a-4e89-a159-4edd3af3a2fa\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.848678 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"rabbitmq-tls\" (UniqueName: \"kubernetes.io/projected/0cb33a40-6f0a-4e89-a159-4edd3af3a2fa-rabbitmq-tls\") pod \"rabbitmq-server-0\" (UID: \"0cb33a40-6f0a-4e89-a159-4edd3af3a2fa\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.853324 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"erlang-cookie-secret\" (UniqueName: \"kubernetes.io/secret/0cb33a40-6f0a-4e89-a159-4edd3af3a2fa-erlang-cookie-secret\") pod \"rabbitmq-server-0\" (UID: \"0cb33a40-6f0a-4e89-a159-4edd3af3a2fa\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.864259 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-gblmn\" (UniqueName: \"kubernetes.io/projected/0cb33a40-6f0a-4e89-a159-4edd3af3a2fa-kube-api-access-gblmn\") pod \"rabbitmq-server-0\" (UID: \"0cb33a40-6f0a-4e89-a159-4edd3af3a2fa\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.870044 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage14-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage14-crc\") pod \"rabbitmq-server-0\" (UID: \"0cb33a40-6f0a-4e89-a159-4edd3af3a2fa\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:41:37 crc kubenswrapper[5052]: I1203 12:41:37.936501 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:41:38 crc kubenswrapper[5052]: I1203 12:41:38.406333 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/rabbitmq-server-0"] Dec 03 12:41:38 crc kubenswrapper[5052]: W1203 12:41:38.422958 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0cb33a40_6f0a_4e89_a159_4edd3af3a2fa.slice/crio-649e9b73822db914ddce53841f5f090c49bb4102dea8d827fabb769084e11db2 WatchSource:0}: Error finding container 649e9b73822db914ddce53841f5f090c49bb4102dea8d827fabb769084e11db2: Status 404 returned error can't find the container with id 649e9b73822db914ddce53841f5f090c49bb4102dea8d827fabb769084e11db2 Dec 03 12:41:38 crc kubenswrapper[5052]: I1203 12:41:38.729333 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/rabbitmq-cell1-server-0"] Dec 03 12:41:38 crc kubenswrapper[5052]: I1203 12:41:38.730803 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:41:38 crc kubenswrapper[5052]: I1203 12:41:38.733521 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"rabbitmq-cell1-erlang-cookie" Dec 03 12:41:38 crc kubenswrapper[5052]: I1203 12:41:38.733593 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"rabbitmq-cell1-plugins-conf" Dec 03 12:41:38 crc kubenswrapper[5052]: I1203 12:41:38.734180 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"rabbitmq-cell1-config-data" Dec 03 12:41:38 crc kubenswrapper[5052]: I1203 12:41:38.734406 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"rabbitmq-cell1-server-dockercfg-fb87g" Dec 03 12:41:38 crc kubenswrapper[5052]: I1203 12:41:38.734843 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"rabbitmq-cell1-default-user" Dec 03 12:41:38 crc kubenswrapper[5052]: I1203 12:41:38.735011 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"rabbitmq-cell1-server-conf" Dec 03 12:41:38 crc kubenswrapper[5052]: I1203 12:41:38.737800 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-rabbitmq-cell1-svc" Dec 03 12:41:38 crc kubenswrapper[5052]: I1203 12:41:38.752868 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/rabbitmq-cell1-server-0"] Dec 03 12:41:38 crc kubenswrapper[5052]: I1203 12:41:38.860878 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"pod-info\" (UniqueName: \"kubernetes.io/downward-api/488c3529-4c2b-47c0-9dcf-05a064c34d0d-pod-info\") pod \"rabbitmq-cell1-server-0\" (UID: \"488c3529-4c2b-47c0-9dcf-05a064c34d0d\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:41:38 crc kubenswrapper[5052]: I1203 12:41:38.860952 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"rabbitmq-confd\" (UniqueName: \"kubernetes.io/projected/488c3529-4c2b-47c0-9dcf-05a064c34d0d-rabbitmq-confd\") pod \"rabbitmq-cell1-server-0\" (UID: \"488c3529-4c2b-47c0-9dcf-05a064c34d0d\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:41:38 crc kubenswrapper[5052]: I1203 12:41:38.860982 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cm686\" (UniqueName: \"kubernetes.io/projected/488c3529-4c2b-47c0-9dcf-05a064c34d0d-kube-api-access-cm686\") pod \"rabbitmq-cell1-server-0\" (UID: \"488c3529-4c2b-47c0-9dcf-05a064c34d0d\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:41:38 crc kubenswrapper[5052]: I1203 12:41:38.861013 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"erlang-cookie-secret\" (UniqueName: \"kubernetes.io/secret/488c3529-4c2b-47c0-9dcf-05a064c34d0d-erlang-cookie-secret\") pod \"rabbitmq-cell1-server-0\" (UID: \"488c3529-4c2b-47c0-9dcf-05a064c34d0d\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:41:38 crc kubenswrapper[5052]: I1203 12:41:38.861045 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-conf\" (UniqueName: \"kubernetes.io/configmap/488c3529-4c2b-47c0-9dcf-05a064c34d0d-plugins-conf\") pod \"rabbitmq-cell1-server-0\" (UID: \"488c3529-4c2b-47c0-9dcf-05a064c34d0d\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:41:38 crc kubenswrapper[5052]: I1203 12:41:38.861140 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/488c3529-4c2b-47c0-9dcf-05a064c34d0d-config-data\") pod \"rabbitmq-cell1-server-0\" (UID: \"488c3529-4c2b-47c0-9dcf-05a064c34d0d\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:41:38 crc kubenswrapper[5052]: I1203 12:41:38.861202 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"rabbitmq-plugins\" (UniqueName: \"kubernetes.io/empty-dir/488c3529-4c2b-47c0-9dcf-05a064c34d0d-rabbitmq-plugins\") pod \"rabbitmq-cell1-server-0\" (UID: \"488c3529-4c2b-47c0-9dcf-05a064c34d0d\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:41:38 crc kubenswrapper[5052]: I1203 12:41:38.861308 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage20-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage20-crc\") pod \"rabbitmq-cell1-server-0\" (UID: \"488c3529-4c2b-47c0-9dcf-05a064c34d0d\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:41:38 crc kubenswrapper[5052]: I1203 12:41:38.861338 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"rabbitmq-erlang-cookie\" (UniqueName: \"kubernetes.io/empty-dir/488c3529-4c2b-47c0-9dcf-05a064c34d0d-rabbitmq-erlang-cookie\") pod \"rabbitmq-cell1-server-0\" (UID: \"488c3529-4c2b-47c0-9dcf-05a064c34d0d\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:41:38 crc kubenswrapper[5052]: I1203 12:41:38.861367 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"server-conf\" (UniqueName: \"kubernetes.io/configmap/488c3529-4c2b-47c0-9dcf-05a064c34d0d-server-conf\") pod \"rabbitmq-cell1-server-0\" (UID: \"488c3529-4c2b-47c0-9dcf-05a064c34d0d\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:41:38 crc kubenswrapper[5052]: I1203 12:41:38.861394 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"rabbitmq-tls\" (UniqueName: \"kubernetes.io/projected/488c3529-4c2b-47c0-9dcf-05a064c34d0d-rabbitmq-tls\") pod \"rabbitmq-cell1-server-0\" (UID: \"488c3529-4c2b-47c0-9dcf-05a064c34d0d\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:41:38 crc kubenswrapper[5052]: I1203 12:41:38.900705 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-server-0" event={"ID":"0cb33a40-6f0a-4e89-a159-4edd3af3a2fa","Type":"ContainerStarted","Data":"649e9b73822db914ddce53841f5f090c49bb4102dea8d827fabb769084e11db2"} Dec 03 12:41:38 crc kubenswrapper[5052]: I1203 12:41:38.962861 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"erlang-cookie-secret\" (UniqueName: \"kubernetes.io/secret/488c3529-4c2b-47c0-9dcf-05a064c34d0d-erlang-cookie-secret\") pod \"rabbitmq-cell1-server-0\" (UID: \"488c3529-4c2b-47c0-9dcf-05a064c34d0d\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:41:38 crc kubenswrapper[5052]: I1203 12:41:38.963227 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"plugins-conf\" (UniqueName: \"kubernetes.io/configmap/488c3529-4c2b-47c0-9dcf-05a064c34d0d-plugins-conf\") pod \"rabbitmq-cell1-server-0\" (UID: \"488c3529-4c2b-47c0-9dcf-05a064c34d0d\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:41:38 crc kubenswrapper[5052]: I1203 12:41:38.963300 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/488c3529-4c2b-47c0-9dcf-05a064c34d0d-config-data\") pod \"rabbitmq-cell1-server-0\" (UID: \"488c3529-4c2b-47c0-9dcf-05a064c34d0d\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:41:38 crc kubenswrapper[5052]: I1203 12:41:38.963340 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"rabbitmq-plugins\" (UniqueName: \"kubernetes.io/empty-dir/488c3529-4c2b-47c0-9dcf-05a064c34d0d-rabbitmq-plugins\") pod \"rabbitmq-cell1-server-0\" (UID: \"488c3529-4c2b-47c0-9dcf-05a064c34d0d\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:41:38 crc kubenswrapper[5052]: I1203 12:41:38.963383 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage20-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage20-crc\") pod \"rabbitmq-cell1-server-0\" (UID: \"488c3529-4c2b-47c0-9dcf-05a064c34d0d\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:41:38 crc kubenswrapper[5052]: I1203 12:41:38.963412 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"rabbitmq-erlang-cookie\" (UniqueName: \"kubernetes.io/empty-dir/488c3529-4c2b-47c0-9dcf-05a064c34d0d-rabbitmq-erlang-cookie\") pod \"rabbitmq-cell1-server-0\" (UID: \"488c3529-4c2b-47c0-9dcf-05a064c34d0d\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:41:38 crc kubenswrapper[5052]: I1203 12:41:38.963439 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"server-conf\" (UniqueName: \"kubernetes.io/configmap/488c3529-4c2b-47c0-9dcf-05a064c34d0d-server-conf\") pod \"rabbitmq-cell1-server-0\" (UID: \"488c3529-4c2b-47c0-9dcf-05a064c34d0d\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:41:38 crc kubenswrapper[5052]: I1203 12:41:38.963470 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"rabbitmq-tls\" (UniqueName: \"kubernetes.io/projected/488c3529-4c2b-47c0-9dcf-05a064c34d0d-rabbitmq-tls\") pod \"rabbitmq-cell1-server-0\" (UID: \"488c3529-4c2b-47c0-9dcf-05a064c34d0d\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:41:38 crc kubenswrapper[5052]: I1203 12:41:38.963524 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"pod-info\" (UniqueName: \"kubernetes.io/downward-api/488c3529-4c2b-47c0-9dcf-05a064c34d0d-pod-info\") pod \"rabbitmq-cell1-server-0\" (UID: \"488c3529-4c2b-47c0-9dcf-05a064c34d0d\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:41:38 crc kubenswrapper[5052]: I1203 12:41:38.963558 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"rabbitmq-confd\" (UniqueName: \"kubernetes.io/projected/488c3529-4c2b-47c0-9dcf-05a064c34d0d-rabbitmq-confd\") pod \"rabbitmq-cell1-server-0\" (UID: \"488c3529-4c2b-47c0-9dcf-05a064c34d0d\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:41:38 crc kubenswrapper[5052]: I1203 12:41:38.963584 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-cm686\" (UniqueName: \"kubernetes.io/projected/488c3529-4c2b-47c0-9dcf-05a064c34d0d-kube-api-access-cm686\") pod \"rabbitmq-cell1-server-0\" (UID: \"488c3529-4c2b-47c0-9dcf-05a064c34d0d\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:41:38 crc kubenswrapper[5052]: I1203 12:41:38.964127 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"rabbitmq-erlang-cookie\" (UniqueName: \"kubernetes.io/empty-dir/488c3529-4c2b-47c0-9dcf-05a064c34d0d-rabbitmq-erlang-cookie\") pod \"rabbitmq-cell1-server-0\" (UID: \"488c3529-4c2b-47c0-9dcf-05a064c34d0d\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:41:38 crc kubenswrapper[5052]: I1203 12:41:38.964327 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"plugins-conf\" (UniqueName: \"kubernetes.io/configmap/488c3529-4c2b-47c0-9dcf-05a064c34d0d-plugins-conf\") pod \"rabbitmq-cell1-server-0\" (UID: \"488c3529-4c2b-47c0-9dcf-05a064c34d0d\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:41:38 crc kubenswrapper[5052]: I1203 12:41:38.964961 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/488c3529-4c2b-47c0-9dcf-05a064c34d0d-config-data\") pod \"rabbitmq-cell1-server-0\" (UID: \"488c3529-4c2b-47c0-9dcf-05a064c34d0d\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:41:38 crc kubenswrapper[5052]: I1203 12:41:38.965136 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"server-conf\" (UniqueName: \"kubernetes.io/configmap/488c3529-4c2b-47c0-9dcf-05a064c34d0d-server-conf\") pod \"rabbitmq-cell1-server-0\" (UID: \"488c3529-4c2b-47c0-9dcf-05a064c34d0d\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:41:38 crc kubenswrapper[5052]: I1203 12:41:38.965251 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"rabbitmq-plugins\" (UniqueName: \"kubernetes.io/empty-dir/488c3529-4c2b-47c0-9dcf-05a064c34d0d-rabbitmq-plugins\") pod \"rabbitmq-cell1-server-0\" (UID: \"488c3529-4c2b-47c0-9dcf-05a064c34d0d\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:41:38 crc kubenswrapper[5052]: I1203 12:41:38.965593 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage20-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage20-crc\") pod \"rabbitmq-cell1-server-0\" (UID: \"488c3529-4c2b-47c0-9dcf-05a064c34d0d\") device mount path \"/mnt/openstack/pv20\"" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:41:38 crc kubenswrapper[5052]: I1203 12:41:38.967662 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"pod-info\" (UniqueName: \"kubernetes.io/downward-api/488c3529-4c2b-47c0-9dcf-05a064c34d0d-pod-info\") pod \"rabbitmq-cell1-server-0\" (UID: \"488c3529-4c2b-47c0-9dcf-05a064c34d0d\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:41:38 crc kubenswrapper[5052]: I1203 12:41:38.968088 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"erlang-cookie-secret\" (UniqueName: \"kubernetes.io/secret/488c3529-4c2b-47c0-9dcf-05a064c34d0d-erlang-cookie-secret\") pod \"rabbitmq-cell1-server-0\" (UID: \"488c3529-4c2b-47c0-9dcf-05a064c34d0d\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:41:38 crc kubenswrapper[5052]: I1203 12:41:38.969767 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"rabbitmq-confd\" (UniqueName: \"kubernetes.io/projected/488c3529-4c2b-47c0-9dcf-05a064c34d0d-rabbitmq-confd\") pod \"rabbitmq-cell1-server-0\" (UID: \"488c3529-4c2b-47c0-9dcf-05a064c34d0d\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:41:38 crc kubenswrapper[5052]: I1203 12:41:38.978029 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"rabbitmq-tls\" (UniqueName: \"kubernetes.io/projected/488c3529-4c2b-47c0-9dcf-05a064c34d0d-rabbitmq-tls\") pod \"rabbitmq-cell1-server-0\" (UID: \"488c3529-4c2b-47c0-9dcf-05a064c34d0d\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:41:38 crc kubenswrapper[5052]: I1203 12:41:38.990615 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage20-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage20-crc\") pod \"rabbitmq-cell1-server-0\" (UID: \"488c3529-4c2b-47c0-9dcf-05a064c34d0d\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:41:38 crc kubenswrapper[5052]: I1203 12:41:38.992467 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-cm686\" (UniqueName: \"kubernetes.io/projected/488c3529-4c2b-47c0-9dcf-05a064c34d0d-kube-api-access-cm686\") pod \"rabbitmq-cell1-server-0\" (UID: \"488c3529-4c2b-47c0-9dcf-05a064c34d0d\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:41:39 crc kubenswrapper[5052]: I1203 12:41:39.082153 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:41:39 crc kubenswrapper[5052]: I1203 12:41:39.167062 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/openstack-galera-0"] Dec 03 12:41:39 crc kubenswrapper[5052]: I1203 12:41:39.175331 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:41:39 crc kubenswrapper[5052]: I1203 12:41:39.179597 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"galera-openstack-dockercfg-tsmcq" Dec 03 12:41:39 crc kubenswrapper[5052]: I1203 12:41:39.179692 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-galera-openstack-svc" Dec 03 12:41:39 crc kubenswrapper[5052]: I1203 12:41:39.181735 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"openstack-scripts" Dec 03 12:41:39 crc kubenswrapper[5052]: I1203 12:41:39.185527 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"combined-ca-bundle" Dec 03 12:41:39 crc kubenswrapper[5052]: I1203 12:41:39.185942 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"openstack-config-data" Dec 03 12:41:39 crc kubenswrapper[5052]: I1203 12:41:39.202606 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/openstack-galera-0"] Dec 03 12:41:39 crc kubenswrapper[5052]: I1203 12:41:39.269698 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/8fa85200-471e-4fba-9884-ef25328ebf64-config-data-generated\") pod \"openstack-galera-0\" (UID: \"8fa85200-471e-4fba-9884-ef25328ebf64\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:41:39 crc kubenswrapper[5052]: I1203 12:41:39.270005 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/8fa85200-471e-4fba-9884-ef25328ebf64-config-data-default\") pod \"openstack-galera-0\" (UID: \"8fa85200-471e-4fba-9884-ef25328ebf64\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:41:39 crc kubenswrapper[5052]: I1203 12:41:39.270072 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/8fa85200-471e-4fba-9884-ef25328ebf64-galera-tls-certs\") pod \"openstack-galera-0\" (UID: \"8fa85200-471e-4fba-9884-ef25328ebf64\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:41:39 crc kubenswrapper[5052]: I1203 12:41:39.270111 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/8fa85200-471e-4fba-9884-ef25328ebf64-kolla-config\") pod \"openstack-galera-0\" (UID: \"8fa85200-471e-4fba-9884-ef25328ebf64\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:41:39 crc kubenswrapper[5052]: I1203 12:41:39.270141 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/8fa85200-471e-4fba-9884-ef25328ebf64-operator-scripts\") pod \"openstack-galera-0\" (UID: \"8fa85200-471e-4fba-9884-ef25328ebf64\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:41:39 crc kubenswrapper[5052]: I1203 12:41:39.270183 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lplfm\" (UniqueName: \"kubernetes.io/projected/8fa85200-471e-4fba-9884-ef25328ebf64-kube-api-access-lplfm\") pod \"openstack-galera-0\" (UID: \"8fa85200-471e-4fba-9884-ef25328ebf64\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:41:39 crc kubenswrapper[5052]: I1203 12:41:39.270221 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage07-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage07-crc\") pod \"openstack-galera-0\" (UID: \"8fa85200-471e-4fba-9884-ef25328ebf64\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:41:39 crc kubenswrapper[5052]: I1203 12:41:39.270432 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8fa85200-471e-4fba-9884-ef25328ebf64-combined-ca-bundle\") pod \"openstack-galera-0\" (UID: \"8fa85200-471e-4fba-9884-ef25328ebf64\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:41:39 crc kubenswrapper[5052]: I1203 12:41:39.371415 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/8fa85200-471e-4fba-9884-ef25328ebf64-config-data-default\") pod \"openstack-galera-0\" (UID: \"8fa85200-471e-4fba-9884-ef25328ebf64\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:41:39 crc kubenswrapper[5052]: I1203 12:41:39.371554 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/8fa85200-471e-4fba-9884-ef25328ebf64-galera-tls-certs\") pod \"openstack-galera-0\" (UID: \"8fa85200-471e-4fba-9884-ef25328ebf64\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:41:39 crc kubenswrapper[5052]: I1203 12:41:39.371600 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/8fa85200-471e-4fba-9884-ef25328ebf64-kolla-config\") pod \"openstack-galera-0\" (UID: \"8fa85200-471e-4fba-9884-ef25328ebf64\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:41:39 crc kubenswrapper[5052]: I1203 12:41:39.371630 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/8fa85200-471e-4fba-9884-ef25328ebf64-operator-scripts\") pod \"openstack-galera-0\" (UID: \"8fa85200-471e-4fba-9884-ef25328ebf64\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:41:39 crc kubenswrapper[5052]: I1203 12:41:39.371674 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-lplfm\" (UniqueName: \"kubernetes.io/projected/8fa85200-471e-4fba-9884-ef25328ebf64-kube-api-access-lplfm\") pod \"openstack-galera-0\" (UID: \"8fa85200-471e-4fba-9884-ef25328ebf64\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:41:39 crc kubenswrapper[5052]: I1203 12:41:39.371724 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage07-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage07-crc\") pod \"openstack-galera-0\" (UID: \"8fa85200-471e-4fba-9884-ef25328ebf64\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:41:39 crc kubenswrapper[5052]: I1203 12:41:39.371777 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8fa85200-471e-4fba-9884-ef25328ebf64-combined-ca-bundle\") pod \"openstack-galera-0\" (UID: \"8fa85200-471e-4fba-9884-ef25328ebf64\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:41:39 crc kubenswrapper[5052]: I1203 12:41:39.371812 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/8fa85200-471e-4fba-9884-ef25328ebf64-config-data-generated\") pod \"openstack-galera-0\" (UID: \"8fa85200-471e-4fba-9884-ef25328ebf64\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:41:39 crc kubenswrapper[5052]: I1203 12:41:39.372099 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage07-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage07-crc\") pod \"openstack-galera-0\" (UID: \"8fa85200-471e-4fba-9884-ef25328ebf64\") device mount path \"/mnt/openstack/pv07\"" pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:41:39 crc kubenswrapper[5052]: I1203 12:41:39.372378 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/8fa85200-471e-4fba-9884-ef25328ebf64-config-data-generated\") pod \"openstack-galera-0\" (UID: \"8fa85200-471e-4fba-9884-ef25328ebf64\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:41:39 crc kubenswrapper[5052]: I1203 12:41:39.372912 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/8fa85200-471e-4fba-9884-ef25328ebf64-kolla-config\") pod \"openstack-galera-0\" (UID: \"8fa85200-471e-4fba-9884-ef25328ebf64\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:41:39 crc kubenswrapper[5052]: I1203 12:41:39.373828 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/8fa85200-471e-4fba-9884-ef25328ebf64-operator-scripts\") pod \"openstack-galera-0\" (UID: \"8fa85200-471e-4fba-9884-ef25328ebf64\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:41:39 crc kubenswrapper[5052]: I1203 12:41:39.375013 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/8fa85200-471e-4fba-9884-ef25328ebf64-config-data-default\") pod \"openstack-galera-0\" (UID: \"8fa85200-471e-4fba-9884-ef25328ebf64\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:41:39 crc kubenswrapper[5052]: I1203 12:41:39.382085 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8fa85200-471e-4fba-9884-ef25328ebf64-combined-ca-bundle\") pod \"openstack-galera-0\" (UID: \"8fa85200-471e-4fba-9884-ef25328ebf64\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:41:39 crc kubenswrapper[5052]: I1203 12:41:39.382215 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/8fa85200-471e-4fba-9884-ef25328ebf64-galera-tls-certs\") pod \"openstack-galera-0\" (UID: \"8fa85200-471e-4fba-9884-ef25328ebf64\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:41:39 crc kubenswrapper[5052]: I1203 12:41:39.391177 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-lplfm\" (UniqueName: \"kubernetes.io/projected/8fa85200-471e-4fba-9884-ef25328ebf64-kube-api-access-lplfm\") pod \"openstack-galera-0\" (UID: \"8fa85200-471e-4fba-9884-ef25328ebf64\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:41:39 crc kubenswrapper[5052]: I1203 12:41:39.398367 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage07-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage07-crc\") pod \"openstack-galera-0\" (UID: \"8fa85200-471e-4fba-9884-ef25328ebf64\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:41:39 crc kubenswrapper[5052]: I1203 12:41:39.522250 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:41:39 crc kubenswrapper[5052]: I1203 12:41:39.642817 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/rabbitmq-cell1-server-0"] Dec 03 12:41:39 crc kubenswrapper[5052]: I1203 12:41:39.912658 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" event={"ID":"488c3529-4c2b-47c0-9dcf-05a064c34d0d","Type":"ContainerStarted","Data":"9ba12cd8f00246c7ec1948393afaa34606091b41050f719916c5e78d1943d909"} Dec 03 12:41:39 crc kubenswrapper[5052]: I1203 12:41:39.921198 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-server-0" event={"ID":"0cb33a40-6f0a-4e89-a159-4edd3af3a2fa","Type":"ContainerStarted","Data":"40637a2936ebcaa4c4b798867fbd94260367dfcf9f69160a782dfc1f71402096"} Dec 03 12:41:40 crc kubenswrapper[5052]: I1203 12:41:40.002987 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/openstack-galera-0"] Dec 03 12:41:40 crc kubenswrapper[5052]: W1203 12:41:40.014447 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8fa85200_471e_4fba_9884_ef25328ebf64.slice/crio-3421f9030f680dfd5004e54202f04ffae977d213cdd3f225ef1cd6168dc03d0b WatchSource:0}: Error finding container 3421f9030f680dfd5004e54202f04ffae977d213cdd3f225ef1cd6168dc03d0b: Status 404 returned error can't find the container with id 3421f9030f680dfd5004e54202f04ffae977d213cdd3f225ef1cd6168dc03d0b Dec 03 12:41:40 crc kubenswrapper[5052]: I1203 12:41:40.459191 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/openstack-cell1-galera-0"] Dec 03 12:41:40 crc kubenswrapper[5052]: I1203 12:41:40.460784 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:41:40 crc kubenswrapper[5052]: I1203 12:41:40.462542 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"openstack-cell1-config-data" Dec 03 12:41:40 crc kubenswrapper[5052]: I1203 12:41:40.463473 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"galera-openstack-cell1-dockercfg-7lfhj" Dec 03 12:41:40 crc kubenswrapper[5052]: I1203 12:41:40.463482 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"openstack-cell1-scripts" Dec 03 12:41:40 crc kubenswrapper[5052]: I1203 12:41:40.463703 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-galera-openstack-cell1-svc" Dec 03 12:41:40 crc kubenswrapper[5052]: I1203 12:41:40.473814 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/openstack-cell1-galera-0"] Dec 03 12:41:40 crc kubenswrapper[5052]: I1203 12:41:40.595931 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2c53ea4e-56ba-4ba1-8cb6-8c24725b5856-combined-ca-bundle\") pod \"openstack-cell1-galera-0\" (UID: \"2c53ea4e-56ba-4ba1-8cb6-8c24725b5856\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:41:40 crc kubenswrapper[5052]: I1203 12:41:40.596023 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/2c53ea4e-56ba-4ba1-8cb6-8c24725b5856-galera-tls-certs\") pod \"openstack-cell1-galera-0\" (UID: \"2c53ea4e-56ba-4ba1-8cb6-8c24725b5856\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:41:40 crc kubenswrapper[5052]: I1203 12:41:40.596136 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/2c53ea4e-56ba-4ba1-8cb6-8c24725b5856-kolla-config\") pod \"openstack-cell1-galera-0\" (UID: \"2c53ea4e-56ba-4ba1-8cb6-8c24725b5856\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:41:40 crc kubenswrapper[5052]: I1203 12:41:40.596241 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage11-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage11-crc\") pod \"openstack-cell1-galera-0\" (UID: \"2c53ea4e-56ba-4ba1-8cb6-8c24725b5856\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:41:40 crc kubenswrapper[5052]: I1203 12:41:40.596319 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/2c53ea4e-56ba-4ba1-8cb6-8c24725b5856-operator-scripts\") pod \"openstack-cell1-galera-0\" (UID: \"2c53ea4e-56ba-4ba1-8cb6-8c24725b5856\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:41:40 crc kubenswrapper[5052]: I1203 12:41:40.596396 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/2c53ea4e-56ba-4ba1-8cb6-8c24725b5856-config-data-generated\") pod \"openstack-cell1-galera-0\" (UID: \"2c53ea4e-56ba-4ba1-8cb6-8c24725b5856\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:41:40 crc kubenswrapper[5052]: I1203 12:41:40.596440 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/2c53ea4e-56ba-4ba1-8cb6-8c24725b5856-config-data-default\") pod \"openstack-cell1-galera-0\" (UID: \"2c53ea4e-56ba-4ba1-8cb6-8c24725b5856\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:41:40 crc kubenswrapper[5052]: I1203 12:41:40.596562 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jf8gp\" (UniqueName: \"kubernetes.io/projected/2c53ea4e-56ba-4ba1-8cb6-8c24725b5856-kube-api-access-jf8gp\") pod \"openstack-cell1-galera-0\" (UID: \"2c53ea4e-56ba-4ba1-8cb6-8c24725b5856\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:41:40 crc kubenswrapper[5052]: I1203 12:41:40.701476 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/2c53ea4e-56ba-4ba1-8cb6-8c24725b5856-config-data-generated\") pod \"openstack-cell1-galera-0\" (UID: \"2c53ea4e-56ba-4ba1-8cb6-8c24725b5856\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:41:40 crc kubenswrapper[5052]: I1203 12:41:40.701543 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/2c53ea4e-56ba-4ba1-8cb6-8c24725b5856-config-data-default\") pod \"openstack-cell1-galera-0\" (UID: \"2c53ea4e-56ba-4ba1-8cb6-8c24725b5856\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:41:40 crc kubenswrapper[5052]: I1203 12:41:40.701572 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-jf8gp\" (UniqueName: \"kubernetes.io/projected/2c53ea4e-56ba-4ba1-8cb6-8c24725b5856-kube-api-access-jf8gp\") pod \"openstack-cell1-galera-0\" (UID: \"2c53ea4e-56ba-4ba1-8cb6-8c24725b5856\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:41:40 crc kubenswrapper[5052]: I1203 12:41:40.701640 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2c53ea4e-56ba-4ba1-8cb6-8c24725b5856-combined-ca-bundle\") pod \"openstack-cell1-galera-0\" (UID: \"2c53ea4e-56ba-4ba1-8cb6-8c24725b5856\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:41:40 crc kubenswrapper[5052]: I1203 12:41:40.701668 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/2c53ea4e-56ba-4ba1-8cb6-8c24725b5856-galera-tls-certs\") pod \"openstack-cell1-galera-0\" (UID: \"2c53ea4e-56ba-4ba1-8cb6-8c24725b5856\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:41:40 crc kubenswrapper[5052]: I1203 12:41:40.701704 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/2c53ea4e-56ba-4ba1-8cb6-8c24725b5856-kolla-config\") pod \"openstack-cell1-galera-0\" (UID: \"2c53ea4e-56ba-4ba1-8cb6-8c24725b5856\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:41:40 crc kubenswrapper[5052]: I1203 12:41:40.701728 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage11-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage11-crc\") pod \"openstack-cell1-galera-0\" (UID: \"2c53ea4e-56ba-4ba1-8cb6-8c24725b5856\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:41:40 crc kubenswrapper[5052]: I1203 12:41:40.701771 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/2c53ea4e-56ba-4ba1-8cb6-8c24725b5856-operator-scripts\") pod \"openstack-cell1-galera-0\" (UID: \"2c53ea4e-56ba-4ba1-8cb6-8c24725b5856\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:41:40 crc kubenswrapper[5052]: I1203 12:41:40.701981 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/2c53ea4e-56ba-4ba1-8cb6-8c24725b5856-config-data-generated\") pod \"openstack-cell1-galera-0\" (UID: \"2c53ea4e-56ba-4ba1-8cb6-8c24725b5856\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:41:40 crc kubenswrapper[5052]: I1203 12:41:40.702567 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage11-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage11-crc\") pod \"openstack-cell1-galera-0\" (UID: \"2c53ea4e-56ba-4ba1-8cb6-8c24725b5856\") device mount path \"/mnt/openstack/pv11\"" pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:41:40 crc kubenswrapper[5052]: I1203 12:41:40.702719 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/2c53ea4e-56ba-4ba1-8cb6-8c24725b5856-kolla-config\") pod \"openstack-cell1-galera-0\" (UID: \"2c53ea4e-56ba-4ba1-8cb6-8c24725b5856\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:41:40 crc kubenswrapper[5052]: I1203 12:41:40.702807 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/2c53ea4e-56ba-4ba1-8cb6-8c24725b5856-config-data-default\") pod \"openstack-cell1-galera-0\" (UID: \"2c53ea4e-56ba-4ba1-8cb6-8c24725b5856\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:41:40 crc kubenswrapper[5052]: I1203 12:41:40.703306 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/2c53ea4e-56ba-4ba1-8cb6-8c24725b5856-operator-scripts\") pod \"openstack-cell1-galera-0\" (UID: \"2c53ea4e-56ba-4ba1-8cb6-8c24725b5856\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:41:40 crc kubenswrapper[5052]: I1203 12:41:40.706918 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2c53ea4e-56ba-4ba1-8cb6-8c24725b5856-combined-ca-bundle\") pod \"openstack-cell1-galera-0\" (UID: \"2c53ea4e-56ba-4ba1-8cb6-8c24725b5856\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:41:40 crc kubenswrapper[5052]: I1203 12:41:40.707896 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/2c53ea4e-56ba-4ba1-8cb6-8c24725b5856-galera-tls-certs\") pod \"openstack-cell1-galera-0\" (UID: \"2c53ea4e-56ba-4ba1-8cb6-8c24725b5856\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:41:40 crc kubenswrapper[5052]: I1203 12:41:40.725843 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage11-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage11-crc\") pod \"openstack-cell1-galera-0\" (UID: \"2c53ea4e-56ba-4ba1-8cb6-8c24725b5856\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:41:40 crc kubenswrapper[5052]: I1203 12:41:40.727942 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-jf8gp\" (UniqueName: \"kubernetes.io/projected/2c53ea4e-56ba-4ba1-8cb6-8c24725b5856-kube-api-access-jf8gp\") pod \"openstack-cell1-galera-0\" (UID: \"2c53ea4e-56ba-4ba1-8cb6-8c24725b5856\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:41:40 crc kubenswrapper[5052]: I1203 12:41:40.777683 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:41:40 crc kubenswrapper[5052]: I1203 12:41:40.933410 5052 generic.go:334] "Generic (PLEG): container finished" podID="8fa85200-471e-4fba-9884-ef25328ebf64" containerID="e0c714dcc48f5f60b89824c4dfab71f3a51824e571a8af575bf534cb693276cc" exitCode=0 Dec 03 12:41:40 crc kubenswrapper[5052]: I1203 12:41:40.933492 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-galera-0" event={"ID":"8fa85200-471e-4fba-9884-ef25328ebf64","Type":"ContainerDied","Data":"e0c714dcc48f5f60b89824c4dfab71f3a51824e571a8af575bf534cb693276cc"} Dec 03 12:41:40 crc kubenswrapper[5052]: I1203 12:41:40.933540 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-galera-0" event={"ID":"8fa85200-471e-4fba-9884-ef25328ebf64","Type":"ContainerStarted","Data":"3421f9030f680dfd5004e54202f04ffae977d213cdd3f225ef1cd6168dc03d0b"} Dec 03 12:41:40 crc kubenswrapper[5052]: I1203 12:41:40.943482 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" event={"ID":"488c3529-4c2b-47c0-9dcf-05a064c34d0d","Type":"ContainerStarted","Data":"fd397a6821f16ad2b0e97b5bbbb387d00aed4ab2619cd9d9f06f9bc10900138f"} Dec 03 12:41:41 crc kubenswrapper[5052]: I1203 12:41:41.303394 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/openstack-cell1-galera-0"] Dec 03 12:41:41 crc kubenswrapper[5052]: W1203 12:41:41.305046 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2c53ea4e_56ba_4ba1_8cb6_8c24725b5856.slice/crio-493cca92a439545f5191bee251dbca32a69625aa12bc51f56bb98a8f07bfdee4 WatchSource:0}: Error finding container 493cca92a439545f5191bee251dbca32a69625aa12bc51f56bb98a8f07bfdee4: Status 404 returned error can't find the container with id 493cca92a439545f5191bee251dbca32a69625aa12bc51f56bb98a8f07bfdee4 Dec 03 12:41:41 crc kubenswrapper[5052]: I1203 12:41:41.688555 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/memcached-0"] Dec 03 12:41:41 crc kubenswrapper[5052]: I1203 12:41:41.690073 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/memcached-0" Dec 03 12:41:41 crc kubenswrapper[5052]: I1203 12:41:41.692115 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-memcached-svc" Dec 03 12:41:41 crc kubenswrapper[5052]: I1203 12:41:41.692333 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"memcached-config-data" Dec 03 12:41:41 crc kubenswrapper[5052]: I1203 12:41:41.692526 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"memcached-memcached-dockercfg-bshzz" Dec 03 12:41:41 crc kubenswrapper[5052]: I1203 12:41:41.706979 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/memcached-0"] Dec 03 12:41:41 crc kubenswrapper[5052]: I1203 12:41:41.718652 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/2f966eeb-76ef-4771-acdf-e0de2bb463a4-kolla-config\") pod \"memcached-0\" (UID: \"2f966eeb-76ef-4771-acdf-e0de2bb463a4\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:41:41 crc kubenswrapper[5052]: I1203 12:41:41.718796 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rbrwm\" (UniqueName: \"kubernetes.io/projected/2f966eeb-76ef-4771-acdf-e0de2bb463a4-kube-api-access-rbrwm\") pod \"memcached-0\" (UID: \"2f966eeb-76ef-4771-acdf-e0de2bb463a4\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:41:41 crc kubenswrapper[5052]: I1203 12:41:41.718953 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"memcached-tls-certs\" (UniqueName: \"kubernetes.io/secret/2f966eeb-76ef-4771-acdf-e0de2bb463a4-memcached-tls-certs\") pod \"memcached-0\" (UID: \"2f966eeb-76ef-4771-acdf-e0de2bb463a4\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:41:41 crc kubenswrapper[5052]: I1203 12:41:41.719215 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/2f966eeb-76ef-4771-acdf-e0de2bb463a4-config-data\") pod \"memcached-0\" (UID: \"2f966eeb-76ef-4771-acdf-e0de2bb463a4\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:41:41 crc kubenswrapper[5052]: I1203 12:41:41.719393 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2f966eeb-76ef-4771-acdf-e0de2bb463a4-combined-ca-bundle\") pod \"memcached-0\" (UID: \"2f966eeb-76ef-4771-acdf-e0de2bb463a4\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:41:41 crc kubenswrapper[5052]: I1203 12:41:41.821314 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-rbrwm\" (UniqueName: \"kubernetes.io/projected/2f966eeb-76ef-4771-acdf-e0de2bb463a4-kube-api-access-rbrwm\") pod \"memcached-0\" (UID: \"2f966eeb-76ef-4771-acdf-e0de2bb463a4\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:41:41 crc kubenswrapper[5052]: I1203 12:41:41.821458 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"memcached-tls-certs\" (UniqueName: \"kubernetes.io/secret/2f966eeb-76ef-4771-acdf-e0de2bb463a4-memcached-tls-certs\") pod \"memcached-0\" (UID: \"2f966eeb-76ef-4771-acdf-e0de2bb463a4\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:41:41 crc kubenswrapper[5052]: I1203 12:41:41.821563 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/2f966eeb-76ef-4771-acdf-e0de2bb463a4-config-data\") pod \"memcached-0\" (UID: \"2f966eeb-76ef-4771-acdf-e0de2bb463a4\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:41:41 crc kubenswrapper[5052]: I1203 12:41:41.821717 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2f966eeb-76ef-4771-acdf-e0de2bb463a4-combined-ca-bundle\") pod \"memcached-0\" (UID: \"2f966eeb-76ef-4771-acdf-e0de2bb463a4\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:41:41 crc kubenswrapper[5052]: I1203 12:41:41.821743 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/2f966eeb-76ef-4771-acdf-e0de2bb463a4-kolla-config\") pod \"memcached-0\" (UID: \"2f966eeb-76ef-4771-acdf-e0de2bb463a4\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:41:41 crc kubenswrapper[5052]: I1203 12:41:41.822503 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/2f966eeb-76ef-4771-acdf-e0de2bb463a4-config-data\") pod \"memcached-0\" (UID: \"2f966eeb-76ef-4771-acdf-e0de2bb463a4\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:41:41 crc kubenswrapper[5052]: I1203 12:41:41.822589 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/2f966eeb-76ef-4771-acdf-e0de2bb463a4-kolla-config\") pod \"memcached-0\" (UID: \"2f966eeb-76ef-4771-acdf-e0de2bb463a4\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:41:41 crc kubenswrapper[5052]: I1203 12:41:41.825544 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"memcached-tls-certs\" (UniqueName: \"kubernetes.io/secret/2f966eeb-76ef-4771-acdf-e0de2bb463a4-memcached-tls-certs\") pod \"memcached-0\" (UID: \"2f966eeb-76ef-4771-acdf-e0de2bb463a4\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:41:41 crc kubenswrapper[5052]: I1203 12:41:41.829601 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2f966eeb-76ef-4771-acdf-e0de2bb463a4-combined-ca-bundle\") pod \"memcached-0\" (UID: \"2f966eeb-76ef-4771-acdf-e0de2bb463a4\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:41:41 crc kubenswrapper[5052]: I1203 12:41:41.841084 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-rbrwm\" (UniqueName: \"kubernetes.io/projected/2f966eeb-76ef-4771-acdf-e0de2bb463a4-kube-api-access-rbrwm\") pod \"memcached-0\" (UID: \"2f966eeb-76ef-4771-acdf-e0de2bb463a4\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:41:41 crc kubenswrapper[5052]: I1203 12:41:41.953123 5052 generic.go:334] "Generic (PLEG): container finished" podID="2c53ea4e-56ba-4ba1-8cb6-8c24725b5856" containerID="f6a728d1305480f200dcdc3cfa0a4ead9021fc819a9828033afc74de07435201" exitCode=0 Dec 03 12:41:41 crc kubenswrapper[5052]: I1203 12:41:41.953196 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-cell1-galera-0" event={"ID":"2c53ea4e-56ba-4ba1-8cb6-8c24725b5856","Type":"ContainerDied","Data":"f6a728d1305480f200dcdc3cfa0a4ead9021fc819a9828033afc74de07435201"} Dec 03 12:41:41 crc kubenswrapper[5052]: I1203 12:41:41.953228 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-cell1-galera-0" event={"ID":"2c53ea4e-56ba-4ba1-8cb6-8c24725b5856","Type":"ContainerStarted","Data":"493cca92a439545f5191bee251dbca32a69625aa12bc51f56bb98a8f07bfdee4"} Dec 03 12:41:41 crc kubenswrapper[5052]: I1203 12:41:41.957048 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-galera-0" event={"ID":"8fa85200-471e-4fba-9884-ef25328ebf64","Type":"ContainerStarted","Data":"ea0e318bc1c2d00f8c07df735c18136c77c564a4ca1cc7bf9d3e0dba21c6f865"} Dec 03 12:41:42 crc kubenswrapper[5052]: I1203 12:41:42.019009 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/memcached-0" Dec 03 12:41:42 crc kubenswrapper[5052]: I1203 12:41:42.019194 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/openstack-galera-0" podStartSLOduration=3.019177434 podStartE2EDuration="3.019177434s" podCreationTimestamp="2025-12-03 12:41:39 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:41:42.016966231 +0000 UTC m=+4892.605207995" watchObservedRunningTime="2025-12-03 12:41:42.019177434 +0000 UTC m=+4892.607419188" Dec 03 12:41:42 crc kubenswrapper[5052]: E1203 12:41:42.452992 5052 upgradeaware.go:427] Error proxying data from client to backend: readfrom tcp 38.129.56.252:34046->38.129.56.252:34849: write tcp 38.129.56.252:34046->38.129.56.252:34849: write: broken pipe Dec 03 12:41:42 crc kubenswrapper[5052]: I1203 12:41:42.489693 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/memcached-0"] Dec 03 12:41:42 crc kubenswrapper[5052]: W1203 12:41:42.497090 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2f966eeb_76ef_4771_acdf_e0de2bb463a4.slice/crio-d6a981f05552d78cd6565fef69b3f65e0da68d983bdc3e996835365f9cdcc339 WatchSource:0}: Error finding container d6a981f05552d78cd6565fef69b3f65e0da68d983bdc3e996835365f9cdcc339: Status 404 returned error can't find the container with id d6a981f05552d78cd6565fef69b3f65e0da68d983bdc3e996835365f9cdcc339 Dec 03 12:41:42 crc kubenswrapper[5052]: I1203 12:41:42.968326 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/memcached-0" event={"ID":"2f966eeb-76ef-4771-acdf-e0de2bb463a4","Type":"ContainerStarted","Data":"db9cddd5e36fcb65f1558dec41404bab89ab5883ec640a193f5a47a8c712d924"} Dec 03 12:41:42 crc kubenswrapper[5052]: I1203 12:41:42.968886 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/memcached-0" event={"ID":"2f966eeb-76ef-4771-acdf-e0de2bb463a4","Type":"ContainerStarted","Data":"d6a981f05552d78cd6565fef69b3f65e0da68d983bdc3e996835365f9cdcc339"} Dec 03 12:41:42 crc kubenswrapper[5052]: I1203 12:41:42.968912 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/memcached-0" Dec 03 12:41:42 crc kubenswrapper[5052]: I1203 12:41:42.972703 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-cell1-galera-0" event={"ID":"2c53ea4e-56ba-4ba1-8cb6-8c24725b5856","Type":"ContainerStarted","Data":"fe1395b62b90460d04b6099206e52d20adacc07cc857b25d7b625ceac277317a"} Dec 03 12:41:42 crc kubenswrapper[5052]: I1203 12:41:42.986208 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/memcached-0" podStartSLOduration=1.986191869 podStartE2EDuration="1.986191869s" podCreationTimestamp="2025-12-03 12:41:41 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:41:42.985819008 +0000 UTC m=+4893.574060762" watchObservedRunningTime="2025-12-03 12:41:42.986191869 +0000 UTC m=+4893.574433633" Dec 03 12:41:43 crc kubenswrapper[5052]: I1203 12:41:43.011523 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/openstack-cell1-galera-0" podStartSLOduration=3.011486392 podStartE2EDuration="3.011486392s" podCreationTimestamp="2025-12-03 12:41:40 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:41:43.008200128 +0000 UTC m=+4893.596441902" watchObservedRunningTime="2025-12-03 12:41:43.011486392 +0000 UTC m=+4893.599728146" Dec 03 12:41:43 crc kubenswrapper[5052]: I1203 12:41:43.815439 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/kube-state-metrics-0"] Dec 03 12:41:43 crc kubenswrapper[5052]: I1203 12:41:43.817038 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:41:43 crc kubenswrapper[5052]: I1203 12:41:43.823600 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"telemetry-ceilometer-dockercfg-9lxff" Dec 03 12:41:43 crc kubenswrapper[5052]: I1203 12:41:43.833533 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/kube-state-metrics-0"] Dec 03 12:41:43 crc kubenswrapper[5052]: I1203 12:41:43.963364 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-khfjk\" (UniqueName: \"kubernetes.io/projected/555b864d-15ca-4c31-894e-1323f009c8b4-kube-api-access-khfjk\") pod \"kube-state-metrics-0\" (UID: \"555b864d-15ca-4c31-894e-1323f009c8b4\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:41:44 crc kubenswrapper[5052]: I1203 12:41:44.065384 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-khfjk\" (UniqueName: \"kubernetes.io/projected/555b864d-15ca-4c31-894e-1323f009c8b4-kube-api-access-khfjk\") pod \"kube-state-metrics-0\" (UID: \"555b864d-15ca-4c31-894e-1323f009c8b4\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:41:44 crc kubenswrapper[5052]: I1203 12:41:44.082825 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-khfjk\" (UniqueName: \"kubernetes.io/projected/555b864d-15ca-4c31-894e-1323f009c8b4-kube-api-access-khfjk\") pod \"kube-state-metrics-0\" (UID: \"555b864d-15ca-4c31-894e-1323f009c8b4\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:41:44 crc kubenswrapper[5052]: I1203 12:41:44.138635 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:41:44 crc kubenswrapper[5052]: I1203 12:41:44.572374 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/kube-state-metrics-0"] Dec 03 12:41:44 crc kubenswrapper[5052]: I1203 12:41:44.588895 5052 provider.go:102] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Dec 03 12:41:44 crc kubenswrapper[5052]: I1203 12:41:44.992258 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/kube-state-metrics-0" event={"ID":"555b864d-15ca-4c31-894e-1323f009c8b4","Type":"ContainerStarted","Data":"00e8703b524502ea1f1d66dc2e576a05f32a383d2b067241b9cb107ee89939e7"} Dec 03 12:41:46 crc kubenswrapper[5052]: I1203 12:41:46.001465 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/kube-state-metrics-0" event={"ID":"555b864d-15ca-4c31-894e-1323f009c8b4","Type":"ContainerStarted","Data":"277b668d318a45541794e33dacdc339c16a21f49597dcadafd93c6188d14b6c3"} Dec 03 12:41:46 crc kubenswrapper[5052]: I1203 12:41:46.001833 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:41:46 crc kubenswrapper[5052]: I1203 12:41:46.020448 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/kube-state-metrics-0" podStartSLOduration=2.6639043559999998 podStartE2EDuration="3.020425082s" podCreationTimestamp="2025-12-03 12:41:43 +0000 UTC" firstStartedPulling="2025-12-03 12:41:44.578045693 +0000 UTC m=+4895.166287457" lastFinishedPulling="2025-12-03 12:41:44.934566419 +0000 UTC m=+4895.522808183" observedRunningTime="2025-12-03 12:41:46.014197194 +0000 UTC m=+4896.602438968" watchObservedRunningTime="2025-12-03 12:41:46.020425082 +0000 UTC m=+4896.608666846" Dec 03 12:41:47 crc kubenswrapper[5052]: I1203 12:41:47.020785 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/memcached-0" Dec 03 12:41:48 crc kubenswrapper[5052]: I1203 12:41:48.512174 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ovsdbserver-nb-0"] Dec 03 12:41:48 crc kubenswrapper[5052]: I1203 12:41:48.514105 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:41:48 crc kubenswrapper[5052]: I1203 12:41:48.523012 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-ovndbcluster-nb-ovndbs" Dec 03 12:41:48 crc kubenswrapper[5052]: I1203 12:41:48.523117 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-ovn-metrics" Dec 03 12:41:48 crc kubenswrapper[5052]: I1203 12:41:48.524016 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"ovndbcluster-nb-config" Dec 03 12:41:48 crc kubenswrapper[5052]: I1203 12:41:48.526299 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"ovndbcluster-nb-scripts" Dec 03 12:41:48 crc kubenswrapper[5052]: I1203 12:41:48.533750 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ovncluster-ovndbcluster-nb-dockercfg-2jlnn" Dec 03 12:41:48 crc kubenswrapper[5052]: I1203 12:41:48.537599 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-nb-0"] Dec 03 12:41:48 crc kubenswrapper[5052]: I1203 12:41:48.638715 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ee05fe95-7cc0-4225-a86b-1682adc78b8b-combined-ca-bundle\") pod \"ovsdbserver-nb-0\" (UID: \"ee05fe95-7cc0-4225-a86b-1682adc78b8b\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:41:48 crc kubenswrapper[5052]: I1203 12:41:48.639021 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovsdbserver-nb-tls-certs\" (UniqueName: \"kubernetes.io/secret/ee05fe95-7cc0-4225-a86b-1682adc78b8b-ovsdbserver-nb-tls-certs\") pod \"ovsdbserver-nb-0\" (UID: \"ee05fe95-7cc0-4225-a86b-1682adc78b8b\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:41:48 crc kubenswrapper[5052]: I1203 12:41:48.639161 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9wz5q\" (UniqueName: \"kubernetes.io/projected/ee05fe95-7cc0-4225-a86b-1682adc78b8b-kube-api-access-9wz5q\") pod \"ovsdbserver-nb-0\" (UID: \"ee05fe95-7cc0-4225-a86b-1682adc78b8b\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:41:48 crc kubenswrapper[5052]: I1203 12:41:48.639351 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage01-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage01-crc\") pod \"ovsdbserver-nb-0\" (UID: \"ee05fe95-7cc0-4225-a86b-1682adc78b8b\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:41:48 crc kubenswrapper[5052]: I1203 12:41:48.639799 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/ee05fe95-7cc0-4225-a86b-1682adc78b8b-ovsdb-rundir\") pod \"ovsdbserver-nb-0\" (UID: \"ee05fe95-7cc0-4225-a86b-1682adc78b8b\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:41:48 crc kubenswrapper[5052]: I1203 12:41:48.640000 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ee05fe95-7cc0-4225-a86b-1682adc78b8b-config\") pod \"ovsdbserver-nb-0\" (UID: \"ee05fe95-7cc0-4225-a86b-1682adc78b8b\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:41:48 crc kubenswrapper[5052]: I1203 12:41:48.640131 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/ee05fe95-7cc0-4225-a86b-1682adc78b8b-metrics-certs-tls-certs\") pod \"ovsdbserver-nb-0\" (UID: \"ee05fe95-7cc0-4225-a86b-1682adc78b8b\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:41:48 crc kubenswrapper[5052]: I1203 12:41:48.640260 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/ee05fe95-7cc0-4225-a86b-1682adc78b8b-scripts\") pod \"ovsdbserver-nb-0\" (UID: \"ee05fe95-7cc0-4225-a86b-1682adc78b8b\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:41:48 crc kubenswrapper[5052]: I1203 12:41:48.742303 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/ee05fe95-7cc0-4225-a86b-1682adc78b8b-ovsdb-rundir\") pod \"ovsdbserver-nb-0\" (UID: \"ee05fe95-7cc0-4225-a86b-1682adc78b8b\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:41:48 crc kubenswrapper[5052]: I1203 12:41:48.742629 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ee05fe95-7cc0-4225-a86b-1682adc78b8b-config\") pod \"ovsdbserver-nb-0\" (UID: \"ee05fe95-7cc0-4225-a86b-1682adc78b8b\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:41:48 crc kubenswrapper[5052]: I1203 12:41:48.742760 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/ee05fe95-7cc0-4225-a86b-1682adc78b8b-metrics-certs-tls-certs\") pod \"ovsdbserver-nb-0\" (UID: \"ee05fe95-7cc0-4225-a86b-1682adc78b8b\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:41:48 crc kubenswrapper[5052]: I1203 12:41:48.742856 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/ee05fe95-7cc0-4225-a86b-1682adc78b8b-scripts\") pod \"ovsdbserver-nb-0\" (UID: \"ee05fe95-7cc0-4225-a86b-1682adc78b8b\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:41:48 crc kubenswrapper[5052]: I1203 12:41:48.742953 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/ee05fe95-7cc0-4225-a86b-1682adc78b8b-ovsdb-rundir\") pod \"ovsdbserver-nb-0\" (UID: \"ee05fe95-7cc0-4225-a86b-1682adc78b8b\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:41:48 crc kubenswrapper[5052]: I1203 12:41:48.743116 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ee05fe95-7cc0-4225-a86b-1682adc78b8b-combined-ca-bundle\") pod \"ovsdbserver-nb-0\" (UID: \"ee05fe95-7cc0-4225-a86b-1682adc78b8b\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:41:48 crc kubenswrapper[5052]: I1203 12:41:48.743221 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovsdbserver-nb-tls-certs\" (UniqueName: \"kubernetes.io/secret/ee05fe95-7cc0-4225-a86b-1682adc78b8b-ovsdbserver-nb-tls-certs\") pod \"ovsdbserver-nb-0\" (UID: \"ee05fe95-7cc0-4225-a86b-1682adc78b8b\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:41:48 crc kubenswrapper[5052]: I1203 12:41:48.743314 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-9wz5q\" (UniqueName: \"kubernetes.io/projected/ee05fe95-7cc0-4225-a86b-1682adc78b8b-kube-api-access-9wz5q\") pod \"ovsdbserver-nb-0\" (UID: \"ee05fe95-7cc0-4225-a86b-1682adc78b8b\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:41:48 crc kubenswrapper[5052]: I1203 12:41:48.743408 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage01-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage01-crc\") pod \"ovsdbserver-nb-0\" (UID: \"ee05fe95-7cc0-4225-a86b-1682adc78b8b\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:41:48 crc kubenswrapper[5052]: I1203 12:41:48.743878 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage01-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage01-crc\") pod \"ovsdbserver-nb-0\" (UID: \"ee05fe95-7cc0-4225-a86b-1682adc78b8b\") device mount path \"/mnt/openstack/pv01\"" pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:41:48 crc kubenswrapper[5052]: I1203 12:41:48.744824 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/ee05fe95-7cc0-4225-a86b-1682adc78b8b-scripts\") pod \"ovsdbserver-nb-0\" (UID: \"ee05fe95-7cc0-4225-a86b-1682adc78b8b\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:41:48 crc kubenswrapper[5052]: I1203 12:41:48.744861 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ee05fe95-7cc0-4225-a86b-1682adc78b8b-config\") pod \"ovsdbserver-nb-0\" (UID: \"ee05fe95-7cc0-4225-a86b-1682adc78b8b\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:41:48 crc kubenswrapper[5052]: I1203 12:41:48.785191 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ovsdbserver-sb-0"] Dec 03 12:41:48 crc kubenswrapper[5052]: I1203 12:41:48.786913 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:41:48 crc kubenswrapper[5052]: I1203 12:41:48.790984 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"ovndbcluster-sb-config" Dec 03 12:41:48 crc kubenswrapper[5052]: I1203 12:41:48.791196 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-ovndbcluster-sb-ovndbs" Dec 03 12:41:48 crc kubenswrapper[5052]: I1203 12:41:48.791419 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"ovndbcluster-sb-scripts" Dec 03 12:41:48 crc kubenswrapper[5052]: I1203 12:41:48.793958 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ovncluster-ovndbcluster-sb-dockercfg-f2jbx" Dec 03 12:41:48 crc kubenswrapper[5052]: I1203 12:41:48.800333 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-sb-0"] Dec 03 12:41:48 crc kubenswrapper[5052]: I1203 12:41:48.823936 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/ee05fe95-7cc0-4225-a86b-1682adc78b8b-metrics-certs-tls-certs\") pod \"ovsdbserver-nb-0\" (UID: \"ee05fe95-7cc0-4225-a86b-1682adc78b8b\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:41:48 crc kubenswrapper[5052]: I1203 12:41:48.824170 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovsdbserver-nb-tls-certs\" (UniqueName: \"kubernetes.io/secret/ee05fe95-7cc0-4225-a86b-1682adc78b8b-ovsdbserver-nb-tls-certs\") pod \"ovsdbserver-nb-0\" (UID: \"ee05fe95-7cc0-4225-a86b-1682adc78b8b\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:41:48 crc kubenswrapper[5052]: I1203 12:41:48.824356 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ee05fe95-7cc0-4225-a86b-1682adc78b8b-combined-ca-bundle\") pod \"ovsdbserver-nb-0\" (UID: \"ee05fe95-7cc0-4225-a86b-1682adc78b8b\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:41:48 crc kubenswrapper[5052]: I1203 12:41:48.825642 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-9wz5q\" (UniqueName: \"kubernetes.io/projected/ee05fe95-7cc0-4225-a86b-1682adc78b8b-kube-api-access-9wz5q\") pod \"ovsdbserver-nb-0\" (UID: \"ee05fe95-7cc0-4225-a86b-1682adc78b8b\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:41:48 crc kubenswrapper[5052]: I1203 12:41:48.946847 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/f2d7e3bb-2829-4166-90df-53f9a50272eb-scripts\") pod \"ovsdbserver-sb-0\" (UID: \"f2d7e3bb-2829-4166-90df-53f9a50272eb\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:41:48 crc kubenswrapper[5052]: I1203 12:41:48.946890 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage09-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage09-crc\") pod \"ovsdbserver-sb-0\" (UID: \"f2d7e3bb-2829-4166-90df-53f9a50272eb\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:41:48 crc kubenswrapper[5052]: I1203 12:41:48.946935 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/f2d7e3bb-2829-4166-90df-53f9a50272eb-metrics-certs-tls-certs\") pod \"ovsdbserver-sb-0\" (UID: \"f2d7e3bb-2829-4166-90df-53f9a50272eb\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:41:48 crc kubenswrapper[5052]: I1203 12:41:48.946970 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovsdbserver-sb-tls-certs\" (UniqueName: \"kubernetes.io/secret/f2d7e3bb-2829-4166-90df-53f9a50272eb-ovsdbserver-sb-tls-certs\") pod \"ovsdbserver-sb-0\" (UID: \"f2d7e3bb-2829-4166-90df-53f9a50272eb\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:41:48 crc kubenswrapper[5052]: I1203 12:41:48.946992 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/f2d7e3bb-2829-4166-90df-53f9a50272eb-ovsdb-rundir\") pod \"ovsdbserver-sb-0\" (UID: \"f2d7e3bb-2829-4166-90df-53f9a50272eb\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:41:48 crc kubenswrapper[5052]: I1203 12:41:48.947013 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f2d7e3bb-2829-4166-90df-53f9a50272eb-config\") pod \"ovsdbserver-sb-0\" (UID: \"f2d7e3bb-2829-4166-90df-53f9a50272eb\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:41:48 crc kubenswrapper[5052]: I1203 12:41:48.947123 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5jc57\" (UniqueName: \"kubernetes.io/projected/f2d7e3bb-2829-4166-90df-53f9a50272eb-kube-api-access-5jc57\") pod \"ovsdbserver-sb-0\" (UID: \"f2d7e3bb-2829-4166-90df-53f9a50272eb\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:41:48 crc kubenswrapper[5052]: I1203 12:41:48.947237 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f2d7e3bb-2829-4166-90df-53f9a50272eb-combined-ca-bundle\") pod \"ovsdbserver-sb-0\" (UID: \"f2d7e3bb-2829-4166-90df-53f9a50272eb\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:41:48 crc kubenswrapper[5052]: I1203 12:41:48.956792 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage01-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage01-crc\") pod \"ovsdbserver-nb-0\" (UID: \"ee05fe95-7cc0-4225-a86b-1682adc78b8b\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:41:49 crc kubenswrapper[5052]: I1203 12:41:49.048575 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/f2d7e3bb-2829-4166-90df-53f9a50272eb-metrics-certs-tls-certs\") pod \"ovsdbserver-sb-0\" (UID: \"f2d7e3bb-2829-4166-90df-53f9a50272eb\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:41:49 crc kubenswrapper[5052]: I1203 12:41:49.048639 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovsdbserver-sb-tls-certs\" (UniqueName: \"kubernetes.io/secret/f2d7e3bb-2829-4166-90df-53f9a50272eb-ovsdbserver-sb-tls-certs\") pod \"ovsdbserver-sb-0\" (UID: \"f2d7e3bb-2829-4166-90df-53f9a50272eb\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:41:49 crc kubenswrapper[5052]: I1203 12:41:49.048672 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/f2d7e3bb-2829-4166-90df-53f9a50272eb-ovsdb-rundir\") pod \"ovsdbserver-sb-0\" (UID: \"f2d7e3bb-2829-4166-90df-53f9a50272eb\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:41:49 crc kubenswrapper[5052]: I1203 12:41:49.048699 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f2d7e3bb-2829-4166-90df-53f9a50272eb-config\") pod \"ovsdbserver-sb-0\" (UID: \"f2d7e3bb-2829-4166-90df-53f9a50272eb\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:41:49 crc kubenswrapper[5052]: I1203 12:41:49.048726 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-5jc57\" (UniqueName: \"kubernetes.io/projected/f2d7e3bb-2829-4166-90df-53f9a50272eb-kube-api-access-5jc57\") pod \"ovsdbserver-sb-0\" (UID: \"f2d7e3bb-2829-4166-90df-53f9a50272eb\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:41:49 crc kubenswrapper[5052]: I1203 12:41:49.048791 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f2d7e3bb-2829-4166-90df-53f9a50272eb-combined-ca-bundle\") pod \"ovsdbserver-sb-0\" (UID: \"f2d7e3bb-2829-4166-90df-53f9a50272eb\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:41:49 crc kubenswrapper[5052]: I1203 12:41:49.048823 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/f2d7e3bb-2829-4166-90df-53f9a50272eb-scripts\") pod \"ovsdbserver-sb-0\" (UID: \"f2d7e3bb-2829-4166-90df-53f9a50272eb\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:41:49 crc kubenswrapper[5052]: I1203 12:41:49.048842 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage09-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage09-crc\") pod \"ovsdbserver-sb-0\" (UID: \"f2d7e3bb-2829-4166-90df-53f9a50272eb\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:41:49 crc kubenswrapper[5052]: I1203 12:41:49.049129 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage09-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage09-crc\") pod \"ovsdbserver-sb-0\" (UID: \"f2d7e3bb-2829-4166-90df-53f9a50272eb\") device mount path \"/mnt/openstack/pv09\"" pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:41:49 crc kubenswrapper[5052]: I1203 12:41:49.049198 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/f2d7e3bb-2829-4166-90df-53f9a50272eb-ovsdb-rundir\") pod \"ovsdbserver-sb-0\" (UID: \"f2d7e3bb-2829-4166-90df-53f9a50272eb\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:41:49 crc kubenswrapper[5052]: I1203 12:41:49.049838 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f2d7e3bb-2829-4166-90df-53f9a50272eb-config\") pod \"ovsdbserver-sb-0\" (UID: \"f2d7e3bb-2829-4166-90df-53f9a50272eb\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:41:49 crc kubenswrapper[5052]: I1203 12:41:49.050279 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/f2d7e3bb-2829-4166-90df-53f9a50272eb-scripts\") pod \"ovsdbserver-sb-0\" (UID: \"f2d7e3bb-2829-4166-90df-53f9a50272eb\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:41:49 crc kubenswrapper[5052]: I1203 12:41:49.052472 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f2d7e3bb-2829-4166-90df-53f9a50272eb-combined-ca-bundle\") pod \"ovsdbserver-sb-0\" (UID: \"f2d7e3bb-2829-4166-90df-53f9a50272eb\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:41:49 crc kubenswrapper[5052]: I1203 12:41:49.053469 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovsdbserver-sb-tls-certs\" (UniqueName: \"kubernetes.io/secret/f2d7e3bb-2829-4166-90df-53f9a50272eb-ovsdbserver-sb-tls-certs\") pod \"ovsdbserver-sb-0\" (UID: \"f2d7e3bb-2829-4166-90df-53f9a50272eb\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:41:49 crc kubenswrapper[5052]: I1203 12:41:49.054013 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/f2d7e3bb-2829-4166-90df-53f9a50272eb-metrics-certs-tls-certs\") pod \"ovsdbserver-sb-0\" (UID: \"f2d7e3bb-2829-4166-90df-53f9a50272eb\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:41:49 crc kubenswrapper[5052]: I1203 12:41:49.066096 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-5jc57\" (UniqueName: \"kubernetes.io/projected/f2d7e3bb-2829-4166-90df-53f9a50272eb-kube-api-access-5jc57\") pod \"ovsdbserver-sb-0\" (UID: \"f2d7e3bb-2829-4166-90df-53f9a50272eb\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:41:49 crc kubenswrapper[5052]: I1203 12:41:49.069528 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage09-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage09-crc\") pod \"ovsdbserver-sb-0\" (UID: \"f2d7e3bb-2829-4166-90df-53f9a50272eb\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:41:49 crc kubenswrapper[5052]: I1203 12:41:49.105757 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:41:49 crc kubenswrapper[5052]: I1203 12:41:49.135396 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:41:49 crc kubenswrapper[5052]: I1203 12:41:49.522537 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:41:49 crc kubenswrapper[5052]: I1203 12:41:49.522936 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:41:49 crc kubenswrapper[5052]: I1203 12:41:49.599311 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:41:49 crc kubenswrapper[5052]: I1203 12:41:49.640524 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-sb-0"] Dec 03 12:41:49 crc kubenswrapper[5052]: I1203 12:41:49.756212 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-nb-0"] Dec 03 12:41:49 crc kubenswrapper[5052]: W1203 12:41:49.762446 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podee05fe95_7cc0_4225_a86b_1682adc78b8b.slice/crio-027af82730bbe7317770cd1d55860ee4299bec5a78025826b71548604110db4b WatchSource:0}: Error finding container 027af82730bbe7317770cd1d55860ee4299bec5a78025826b71548604110db4b: Status 404 returned error can't find the container with id 027af82730bbe7317770cd1d55860ee4299bec5a78025826b71548604110db4b Dec 03 12:41:50 crc kubenswrapper[5052]: I1203 12:41:50.037563 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-sb-0" event={"ID":"f2d7e3bb-2829-4166-90df-53f9a50272eb","Type":"ContainerStarted","Data":"786e9ff2bff44817fc6c2946e61ae2defac118812bc4492e8c06e24dc823c83b"} Dec 03 12:41:50 crc kubenswrapper[5052]: I1203 12:41:50.040098 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-nb-0" event={"ID":"ee05fe95-7cc0-4225-a86b-1682adc78b8b","Type":"ContainerStarted","Data":"027af82730bbe7317770cd1d55860ee4299bec5a78025826b71548604110db4b"} Dec 03 12:41:50 crc kubenswrapper[5052]: I1203 12:41:50.107584 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:41:50 crc kubenswrapper[5052]: I1203 12:41:50.778648 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:41:50 crc kubenswrapper[5052]: I1203 12:41:50.778931 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:41:50 crc kubenswrapper[5052]: I1203 12:41:50.848083 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:41:51 crc kubenswrapper[5052]: I1203 12:41:51.047920 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-sb-0" event={"ID":"f2d7e3bb-2829-4166-90df-53f9a50272eb","Type":"ContainerStarted","Data":"1499c2af6931030cde94894361cbf72f702626308576bf5ee6e7760bbc5034df"} Dec 03 12:41:51 crc kubenswrapper[5052]: I1203 12:41:51.047963 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-sb-0" event={"ID":"f2d7e3bb-2829-4166-90df-53f9a50272eb","Type":"ContainerStarted","Data":"23cc31c39f29b7dd83db1992297f60565ae8b315ab364419ad6f8475dec1526c"} Dec 03 12:41:51 crc kubenswrapper[5052]: I1203 12:41:51.053498 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-nb-0" event={"ID":"ee05fe95-7cc0-4225-a86b-1682adc78b8b","Type":"ContainerStarted","Data":"61f7b1adc6b8d50af5982e338e056fb28d2ea6d8f6e0b8c7092747bfc78b185f"} Dec 03 12:41:51 crc kubenswrapper[5052]: I1203 12:41:51.053544 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-nb-0" event={"ID":"ee05fe95-7cc0-4225-a86b-1682adc78b8b","Type":"ContainerStarted","Data":"d12594c4d36cf7fcca02723f8cb056b7e65d76a4582dd775c6dc70d741593906"} Dec 03 12:41:51 crc kubenswrapper[5052]: I1203 12:41:51.081985 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/ovsdbserver-sb-0" podStartSLOduration=3.081943803 podStartE2EDuration="3.081943803s" podCreationTimestamp="2025-12-03 12:41:48 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:41:51.072484463 +0000 UTC m=+4901.660726237" watchObservedRunningTime="2025-12-03 12:41:51.081943803 +0000 UTC m=+4901.670185567" Dec 03 12:41:51 crc kubenswrapper[5052]: I1203 12:41:51.100984 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/ovsdbserver-nb-0" podStartSLOduration=3.100963997 podStartE2EDuration="3.100963997s" podCreationTimestamp="2025-12-03 12:41:48 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:41:51.094909064 +0000 UTC m=+4901.683150848" watchObservedRunningTime="2025-12-03 12:41:51.100963997 +0000 UTC m=+4901.689205761" Dec 03 12:41:51 crc kubenswrapper[5052]: I1203 12:41:51.137869 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:41:51 crc kubenswrapper[5052]: I1203 12:41:51.577929 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/keystone-db-create-zjnt8"] Dec 03 12:41:51 crc kubenswrapper[5052]: I1203 12:41:51.581210 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-db-create-zjnt8" Dec 03 12:41:51 crc kubenswrapper[5052]: I1203 12:41:51.588395 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-db-create-zjnt8"] Dec 03 12:41:51 crc kubenswrapper[5052]: I1203 12:41:51.621118 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/8b6e373e-2587-415b-8cbc-5fef0ba3427b-operator-scripts\") pod \"keystone-db-create-zjnt8\" (UID: \"8b6e373e-2587-415b-8cbc-5fef0ba3427b\") " pod="openstack-kuttl-tests/keystone-db-create-zjnt8" Dec 03 12:41:51 crc kubenswrapper[5052]: I1203 12:41:51.621480 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2ktnp\" (UniqueName: \"kubernetes.io/projected/8b6e373e-2587-415b-8cbc-5fef0ba3427b-kube-api-access-2ktnp\") pod \"keystone-db-create-zjnt8\" (UID: \"8b6e373e-2587-415b-8cbc-5fef0ba3427b\") " pod="openstack-kuttl-tests/keystone-db-create-zjnt8" Dec 03 12:41:51 crc kubenswrapper[5052]: I1203 12:41:51.712647 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/keystone-b51b-account-create-update-h6wlv"] Dec 03 12:41:51 crc kubenswrapper[5052]: I1203 12:41:51.713869 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-b51b-account-create-update-h6wlv" Dec 03 12:41:51 crc kubenswrapper[5052]: I1203 12:41:51.716833 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone-db-secret" Dec 03 12:41:51 crc kubenswrapper[5052]: I1203 12:41:51.722894 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/8b6e373e-2587-415b-8cbc-5fef0ba3427b-operator-scripts\") pod \"keystone-db-create-zjnt8\" (UID: \"8b6e373e-2587-415b-8cbc-5fef0ba3427b\") " pod="openstack-kuttl-tests/keystone-db-create-zjnt8" Dec 03 12:41:51 crc kubenswrapper[5052]: I1203 12:41:51.723243 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-2ktnp\" (UniqueName: \"kubernetes.io/projected/8b6e373e-2587-415b-8cbc-5fef0ba3427b-kube-api-access-2ktnp\") pod \"keystone-db-create-zjnt8\" (UID: \"8b6e373e-2587-415b-8cbc-5fef0ba3427b\") " pod="openstack-kuttl-tests/keystone-db-create-zjnt8" Dec 03 12:41:51 crc kubenswrapper[5052]: I1203 12:41:51.723788 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/8b6e373e-2587-415b-8cbc-5fef0ba3427b-operator-scripts\") pod \"keystone-db-create-zjnt8\" (UID: \"8b6e373e-2587-415b-8cbc-5fef0ba3427b\") " pod="openstack-kuttl-tests/keystone-db-create-zjnt8" Dec 03 12:41:51 crc kubenswrapper[5052]: I1203 12:41:51.727309 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-b51b-account-create-update-h6wlv"] Dec 03 12:41:51 crc kubenswrapper[5052]: I1203 12:41:51.752238 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-2ktnp\" (UniqueName: \"kubernetes.io/projected/8b6e373e-2587-415b-8cbc-5fef0ba3427b-kube-api-access-2ktnp\") pod \"keystone-db-create-zjnt8\" (UID: \"8b6e373e-2587-415b-8cbc-5fef0ba3427b\") " pod="openstack-kuttl-tests/keystone-db-create-zjnt8" Dec 03 12:41:51 crc kubenswrapper[5052]: I1203 12:41:51.825290 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hfhfn\" (UniqueName: \"kubernetes.io/projected/5ecf0384-e075-4985-98c9-017cf55e0f7d-kube-api-access-hfhfn\") pod \"keystone-b51b-account-create-update-h6wlv\" (UID: \"5ecf0384-e075-4985-98c9-017cf55e0f7d\") " pod="openstack-kuttl-tests/keystone-b51b-account-create-update-h6wlv" Dec 03 12:41:51 crc kubenswrapper[5052]: I1203 12:41:51.825394 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/5ecf0384-e075-4985-98c9-017cf55e0f7d-operator-scripts\") pod \"keystone-b51b-account-create-update-h6wlv\" (UID: \"5ecf0384-e075-4985-98c9-017cf55e0f7d\") " pod="openstack-kuttl-tests/keystone-b51b-account-create-update-h6wlv" Dec 03 12:41:51 crc kubenswrapper[5052]: I1203 12:41:51.908006 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-db-create-zjnt8" Dec 03 12:41:51 crc kubenswrapper[5052]: I1203 12:41:51.916942 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/placement-db-create-69hg4"] Dec 03 12:41:51 crc kubenswrapper[5052]: I1203 12:41:51.918651 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-db-create-69hg4" Dec 03 12:41:51 crc kubenswrapper[5052]: I1203 12:41:51.926619 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/5ecf0384-e075-4985-98c9-017cf55e0f7d-operator-scripts\") pod \"keystone-b51b-account-create-update-h6wlv\" (UID: \"5ecf0384-e075-4985-98c9-017cf55e0f7d\") " pod="openstack-kuttl-tests/keystone-b51b-account-create-update-h6wlv" Dec 03 12:41:51 crc kubenswrapper[5052]: I1203 12:41:51.926917 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-hfhfn\" (UniqueName: \"kubernetes.io/projected/5ecf0384-e075-4985-98c9-017cf55e0f7d-kube-api-access-hfhfn\") pod \"keystone-b51b-account-create-update-h6wlv\" (UID: \"5ecf0384-e075-4985-98c9-017cf55e0f7d\") " pod="openstack-kuttl-tests/keystone-b51b-account-create-update-h6wlv" Dec 03 12:41:51 crc kubenswrapper[5052]: I1203 12:41:51.927911 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/5ecf0384-e075-4985-98c9-017cf55e0f7d-operator-scripts\") pod \"keystone-b51b-account-create-update-h6wlv\" (UID: \"5ecf0384-e075-4985-98c9-017cf55e0f7d\") " pod="openstack-kuttl-tests/keystone-b51b-account-create-update-h6wlv" Dec 03 12:41:51 crc kubenswrapper[5052]: I1203 12:41:51.929886 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/placement-db-create-69hg4"] Dec 03 12:41:51 crc kubenswrapper[5052]: I1203 12:41:51.963410 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-hfhfn\" (UniqueName: \"kubernetes.io/projected/5ecf0384-e075-4985-98c9-017cf55e0f7d-kube-api-access-hfhfn\") pod \"keystone-b51b-account-create-update-h6wlv\" (UID: \"5ecf0384-e075-4985-98c9-017cf55e0f7d\") " pod="openstack-kuttl-tests/keystone-b51b-account-create-update-h6wlv" Dec 03 12:41:52 crc kubenswrapper[5052]: I1203 12:41:52.028129 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/2b4ed2db-d186-4bf9-8616-44f1249e7af1-operator-scripts\") pod \"placement-db-create-69hg4\" (UID: \"2b4ed2db-d186-4bf9-8616-44f1249e7af1\") " pod="openstack-kuttl-tests/placement-db-create-69hg4" Dec 03 12:41:52 crc kubenswrapper[5052]: I1203 12:41:52.028482 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qnntw\" (UniqueName: \"kubernetes.io/projected/2b4ed2db-d186-4bf9-8616-44f1249e7af1-kube-api-access-qnntw\") pod \"placement-db-create-69hg4\" (UID: \"2b4ed2db-d186-4bf9-8616-44f1249e7af1\") " pod="openstack-kuttl-tests/placement-db-create-69hg4" Dec 03 12:41:52 crc kubenswrapper[5052]: I1203 12:41:52.028885 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/placement-361c-account-create-update-4psp5"] Dec 03 12:41:52 crc kubenswrapper[5052]: I1203 12:41:52.030255 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-361c-account-create-update-4psp5" Dec 03 12:41:52 crc kubenswrapper[5052]: I1203 12:41:52.031186 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-b51b-account-create-update-h6wlv" Dec 03 12:41:52 crc kubenswrapper[5052]: I1203 12:41:52.034295 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"placement-db-secret" Dec 03 12:41:52 crc kubenswrapper[5052]: I1203 12:41:52.038605 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/placement-361c-account-create-update-4psp5"] Dec 03 12:41:52 crc kubenswrapper[5052]: I1203 12:41:52.107465 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:41:52 crc kubenswrapper[5052]: I1203 12:41:52.130009 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/2b4ed2db-d186-4bf9-8616-44f1249e7af1-operator-scripts\") pod \"placement-db-create-69hg4\" (UID: \"2b4ed2db-d186-4bf9-8616-44f1249e7af1\") " pod="openstack-kuttl-tests/placement-db-create-69hg4" Dec 03 12:41:52 crc kubenswrapper[5052]: I1203 12:41:52.130494 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-qnntw\" (UniqueName: \"kubernetes.io/projected/2b4ed2db-d186-4bf9-8616-44f1249e7af1-kube-api-access-qnntw\") pod \"placement-db-create-69hg4\" (UID: \"2b4ed2db-d186-4bf9-8616-44f1249e7af1\") " pod="openstack-kuttl-tests/placement-db-create-69hg4" Dec 03 12:41:52 crc kubenswrapper[5052]: I1203 12:41:52.130573 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/7f78d8a2-cb31-4277-a6f6-51bcb6d4ed15-operator-scripts\") pod \"placement-361c-account-create-update-4psp5\" (UID: \"7f78d8a2-cb31-4277-a6f6-51bcb6d4ed15\") " pod="openstack-kuttl-tests/placement-361c-account-create-update-4psp5" Dec 03 12:41:52 crc kubenswrapper[5052]: I1203 12:41:52.130875 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fxs5v\" (UniqueName: \"kubernetes.io/projected/7f78d8a2-cb31-4277-a6f6-51bcb6d4ed15-kube-api-access-fxs5v\") pod \"placement-361c-account-create-update-4psp5\" (UID: \"7f78d8a2-cb31-4277-a6f6-51bcb6d4ed15\") " pod="openstack-kuttl-tests/placement-361c-account-create-update-4psp5" Dec 03 12:41:52 crc kubenswrapper[5052]: I1203 12:41:52.131078 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/2b4ed2db-d186-4bf9-8616-44f1249e7af1-operator-scripts\") pod \"placement-db-create-69hg4\" (UID: \"2b4ed2db-d186-4bf9-8616-44f1249e7af1\") " pod="openstack-kuttl-tests/placement-db-create-69hg4" Dec 03 12:41:52 crc kubenswrapper[5052]: I1203 12:41:52.136241 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:41:52 crc kubenswrapper[5052]: I1203 12:41:52.153029 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-qnntw\" (UniqueName: \"kubernetes.io/projected/2b4ed2db-d186-4bf9-8616-44f1249e7af1-kube-api-access-qnntw\") pod \"placement-db-create-69hg4\" (UID: \"2b4ed2db-d186-4bf9-8616-44f1249e7af1\") " pod="openstack-kuttl-tests/placement-db-create-69hg4" Dec 03 12:41:52 crc kubenswrapper[5052]: I1203 12:41:52.234676 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-fxs5v\" (UniqueName: \"kubernetes.io/projected/7f78d8a2-cb31-4277-a6f6-51bcb6d4ed15-kube-api-access-fxs5v\") pod \"placement-361c-account-create-update-4psp5\" (UID: \"7f78d8a2-cb31-4277-a6f6-51bcb6d4ed15\") " pod="openstack-kuttl-tests/placement-361c-account-create-update-4psp5" Dec 03 12:41:52 crc kubenswrapper[5052]: I1203 12:41:52.234820 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/7f78d8a2-cb31-4277-a6f6-51bcb6d4ed15-operator-scripts\") pod \"placement-361c-account-create-update-4psp5\" (UID: \"7f78d8a2-cb31-4277-a6f6-51bcb6d4ed15\") " pod="openstack-kuttl-tests/placement-361c-account-create-update-4psp5" Dec 03 12:41:52 crc kubenswrapper[5052]: I1203 12:41:52.235865 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/7f78d8a2-cb31-4277-a6f6-51bcb6d4ed15-operator-scripts\") pod \"placement-361c-account-create-update-4psp5\" (UID: \"7f78d8a2-cb31-4277-a6f6-51bcb6d4ed15\") " pod="openstack-kuttl-tests/placement-361c-account-create-update-4psp5" Dec 03 12:41:52 crc kubenswrapper[5052]: I1203 12:41:52.252768 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-fxs5v\" (UniqueName: \"kubernetes.io/projected/7f78d8a2-cb31-4277-a6f6-51bcb6d4ed15-kube-api-access-fxs5v\") pod \"placement-361c-account-create-update-4psp5\" (UID: \"7f78d8a2-cb31-4277-a6f6-51bcb6d4ed15\") " pod="openstack-kuttl-tests/placement-361c-account-create-update-4psp5" Dec 03 12:41:52 crc kubenswrapper[5052]: I1203 12:41:52.316340 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-db-create-69hg4" Dec 03 12:41:52 crc kubenswrapper[5052]: I1203 12:41:52.357141 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-361c-account-create-update-4psp5" Dec 03 12:41:52 crc kubenswrapper[5052]: I1203 12:41:52.420647 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-db-create-zjnt8"] Dec 03 12:41:52 crc kubenswrapper[5052]: I1203 12:41:52.550070 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-b51b-account-create-update-h6wlv"] Dec 03 12:41:52 crc kubenswrapper[5052]: I1203 12:41:52.768806 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/placement-361c-account-create-update-4psp5"] Dec 03 12:41:52 crc kubenswrapper[5052]: I1203 12:41:52.788858 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/placement-db-create-69hg4"] Dec 03 12:41:53 crc kubenswrapper[5052]: I1203 12:41:53.075563 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-b51b-account-create-update-h6wlv" event={"ID":"5ecf0384-e075-4985-98c9-017cf55e0f7d","Type":"ContainerStarted","Data":"4fc9099c8e6b1d6bde19c5a50f204a4526a781a027bf3b6727e6310b27729d5f"} Dec 03 12:41:53 crc kubenswrapper[5052]: I1203 12:41:53.075922 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-b51b-account-create-update-h6wlv" event={"ID":"5ecf0384-e075-4985-98c9-017cf55e0f7d","Type":"ContainerStarted","Data":"f44ad0dad227a521dffa45530a330e188049d5547536b51296b46008b7f4a144"} Dec 03 12:41:53 crc kubenswrapper[5052]: I1203 12:41:53.077845 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-db-create-zjnt8" event={"ID":"8b6e373e-2587-415b-8cbc-5fef0ba3427b","Type":"ContainerStarted","Data":"2b66f8fd287d88b09b2bdfd5155d95c7605918d968768e443682215d572dbf83"} Dec 03 12:41:53 crc kubenswrapper[5052]: I1203 12:41:53.077886 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-db-create-zjnt8" event={"ID":"8b6e373e-2587-415b-8cbc-5fef0ba3427b","Type":"ContainerStarted","Data":"6509e966e980feccbfd01d7eb11af4fcd151e1080f714d65b4834ea2fe188ec2"} Dec 03 12:41:53 crc kubenswrapper[5052]: I1203 12:41:53.080445 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-361c-account-create-update-4psp5" event={"ID":"7f78d8a2-cb31-4277-a6f6-51bcb6d4ed15","Type":"ContainerStarted","Data":"ea03469d014c723497d4bfee22ce5b9ffd0adb13e988c34300527f50478aa8b1"} Dec 03 12:41:53 crc kubenswrapper[5052]: I1203 12:41:53.082951 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-db-create-69hg4" event={"ID":"2b4ed2db-d186-4bf9-8616-44f1249e7af1","Type":"ContainerStarted","Data":"4ba70b6506f8ca2f73348d1e8401ae23ffcfe18c72815e1a45df826e35e84397"} Dec 03 12:41:53 crc kubenswrapper[5052]: I1203 12:41:53.091348 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/keystone-b51b-account-create-update-h6wlv" podStartSLOduration=2.091334558 podStartE2EDuration="2.091334558s" podCreationTimestamp="2025-12-03 12:41:51 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:41:53.088581059 +0000 UTC m=+4903.676822823" watchObservedRunningTime="2025-12-03 12:41:53.091334558 +0000 UTC m=+4903.679576322" Dec 03 12:41:53 crc kubenswrapper[5052]: I1203 12:41:53.102721 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/keystone-db-create-zjnt8" podStartSLOduration=2.102696963 podStartE2EDuration="2.102696963s" podCreationTimestamp="2025-12-03 12:41:51 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:41:53.099984575 +0000 UTC m=+4903.688226339" watchObservedRunningTime="2025-12-03 12:41:53.102696963 +0000 UTC m=+4903.690938727" Dec 03 12:41:54 crc kubenswrapper[5052]: I1203 12:41:54.093461 5052 generic.go:334] "Generic (PLEG): container finished" podID="5ecf0384-e075-4985-98c9-017cf55e0f7d" containerID="4fc9099c8e6b1d6bde19c5a50f204a4526a781a027bf3b6727e6310b27729d5f" exitCode=0 Dec 03 12:41:54 crc kubenswrapper[5052]: I1203 12:41:54.093621 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-b51b-account-create-update-h6wlv" event={"ID":"5ecf0384-e075-4985-98c9-017cf55e0f7d","Type":"ContainerDied","Data":"4fc9099c8e6b1d6bde19c5a50f204a4526a781a027bf3b6727e6310b27729d5f"} Dec 03 12:41:54 crc kubenswrapper[5052]: I1203 12:41:54.095921 5052 generic.go:334] "Generic (PLEG): container finished" podID="8b6e373e-2587-415b-8cbc-5fef0ba3427b" containerID="2b66f8fd287d88b09b2bdfd5155d95c7605918d968768e443682215d572dbf83" exitCode=0 Dec 03 12:41:54 crc kubenswrapper[5052]: I1203 12:41:54.096020 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-db-create-zjnt8" event={"ID":"8b6e373e-2587-415b-8cbc-5fef0ba3427b","Type":"ContainerDied","Data":"2b66f8fd287d88b09b2bdfd5155d95c7605918d968768e443682215d572dbf83"} Dec 03 12:41:54 crc kubenswrapper[5052]: I1203 12:41:54.098346 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/swift-storage-0"] Dec 03 12:41:54 crc kubenswrapper[5052]: I1203 12:41:54.099240 5052 generic.go:334] "Generic (PLEG): container finished" podID="7f78d8a2-cb31-4277-a6f6-51bcb6d4ed15" containerID="851a4c98712831fdb1a18e01a6741b469ff7b63efd9396a566b09e32d17f05ba" exitCode=0 Dec 03 12:41:54 crc kubenswrapper[5052]: I1203 12:41:54.101454 5052 generic.go:334] "Generic (PLEG): container finished" podID="2b4ed2db-d186-4bf9-8616-44f1249e7af1" containerID="f9c7ff2956392c8515764b57f36f4a4495d905e7a71d9a81e290f937bf0e1dcd" exitCode=0 Dec 03 12:41:54 crc kubenswrapper[5052]: I1203 12:41:54.106882 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-361c-account-create-update-4psp5" event={"ID":"7f78d8a2-cb31-4277-a6f6-51bcb6d4ed15","Type":"ContainerDied","Data":"851a4c98712831fdb1a18e01a6741b469ff7b63efd9396a566b09e32d17f05ba"} Dec 03 12:41:54 crc kubenswrapper[5052]: I1203 12:41:54.106927 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-db-create-69hg4" event={"ID":"2b4ed2db-d186-4bf9-8616-44f1249e7af1","Type":"ContainerDied","Data":"f9c7ff2956392c8515764b57f36f4a4495d905e7a71d9a81e290f937bf0e1dcd"} Dec 03 12:41:54 crc kubenswrapper[5052]: I1203 12:41:54.106964 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:41:54 crc kubenswrapper[5052]: I1203 12:41:54.107064 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:41:54 crc kubenswrapper[5052]: I1203 12:41:54.109023 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"swift-conf" Dec 03 12:41:54 crc kubenswrapper[5052]: I1203 12:41:54.109150 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"swift-storage-config-data" Dec 03 12:41:54 crc kubenswrapper[5052]: I1203 12:41:54.110067 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"swift-ring-files" Dec 03 12:41:54 crc kubenswrapper[5052]: I1203 12:41:54.111500 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"swift-swift-dockercfg-nf95c" Dec 03 12:41:54 crc kubenswrapper[5052]: I1203 12:41:54.129954 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/swift-storage-0"] Dec 03 12:41:54 crc kubenswrapper[5052]: I1203 12:41:54.137818 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:41:54 crc kubenswrapper[5052]: I1203 12:41:54.182898 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:41:54 crc kubenswrapper[5052]: I1203 12:41:54.223805 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/swift-ring-rebalance-kvhc2"] Dec 03 12:41:54 crc kubenswrapper[5052]: I1203 12:41:54.224995 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-ring-rebalance-kvhc2" Dec 03 12:41:54 crc kubenswrapper[5052]: I1203 12:41:54.227132 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"swift-proxy-config-data" Dec 03 12:41:54 crc kubenswrapper[5052]: I1203 12:41:54.227142 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"swift-ring-scripts" Dec 03 12:41:54 crc kubenswrapper[5052]: I1203 12:41:54.227180 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"swift-ring-config-data" Dec 03 12:41:54 crc kubenswrapper[5052]: I1203 12:41:54.240489 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/swift-ring-rebalance-kvhc2"] Dec 03 12:41:54 crc kubenswrapper[5052]: I1203 12:41:54.278655 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/bd4392a2-9c70-4f3a-9050-ae6a6db01082-etc-swift\") pod \"swift-storage-0\" (UID: \"bd4392a2-9c70-4f3a-9050-ae6a6db01082\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:41:54 crc kubenswrapper[5052]: I1203 12:41:54.278812 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8t7k4\" (UniqueName: \"kubernetes.io/projected/bd4392a2-9c70-4f3a-9050-ae6a6db01082-kube-api-access-8t7k4\") pod \"swift-storage-0\" (UID: \"bd4392a2-9c70-4f3a-9050-ae6a6db01082\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:41:54 crc kubenswrapper[5052]: I1203 12:41:54.278913 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lock\" (UniqueName: \"kubernetes.io/empty-dir/bd4392a2-9c70-4f3a-9050-ae6a6db01082-lock\") pod \"swift-storage-0\" (UID: \"bd4392a2-9c70-4f3a-9050-ae6a6db01082\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:41:54 crc kubenswrapper[5052]: I1203 12:41:54.279524 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cache\" (UniqueName: \"kubernetes.io/empty-dir/bd4392a2-9c70-4f3a-9050-ae6a6db01082-cache\") pod \"swift-storage-0\" (UID: \"bd4392a2-9c70-4f3a-9050-ae6a6db01082\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:41:54 crc kubenswrapper[5052]: I1203 12:41:54.279625 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage13-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage13-crc\") pod \"swift-storage-0\" (UID: \"bd4392a2-9c70-4f3a-9050-ae6a6db01082\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:41:54 crc kubenswrapper[5052]: I1203 12:41:54.381287 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"swiftconf\" (UniqueName: \"kubernetes.io/secret/8d5c589d-50a2-4bdb-ac78-59c1592c03e8-swiftconf\") pod \"swift-ring-rebalance-kvhc2\" (UID: \"8d5c589d-50a2-4bdb-ac78-59c1592c03e8\") " pod="openstack-kuttl-tests/swift-ring-rebalance-kvhc2" Dec 03 12:41:54 crc kubenswrapper[5052]: I1203 12:41:54.381335 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wztzb\" (UniqueName: \"kubernetes.io/projected/8d5c589d-50a2-4bdb-ac78-59c1592c03e8-kube-api-access-wztzb\") pod \"swift-ring-rebalance-kvhc2\" (UID: \"8d5c589d-50a2-4bdb-ac78-59c1592c03e8\") " pod="openstack-kuttl-tests/swift-ring-rebalance-kvhc2" Dec 03 12:41:54 crc kubenswrapper[5052]: I1203 12:41:54.381359 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ring-data-devices\" (UniqueName: \"kubernetes.io/configmap/8d5c589d-50a2-4bdb-ac78-59c1592c03e8-ring-data-devices\") pod \"swift-ring-rebalance-kvhc2\" (UID: \"8d5c589d-50a2-4bdb-ac78-59c1592c03e8\") " pod="openstack-kuttl-tests/swift-ring-rebalance-kvhc2" Dec 03 12:41:54 crc kubenswrapper[5052]: I1203 12:41:54.381409 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage13-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage13-crc\") pod \"swift-storage-0\" (UID: \"bd4392a2-9c70-4f3a-9050-ae6a6db01082\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:41:54 crc kubenswrapper[5052]: I1203 12:41:54.381506 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/empty-dir/8d5c589d-50a2-4bdb-ac78-59c1592c03e8-etc-swift\") pod \"swift-ring-rebalance-kvhc2\" (UID: \"8d5c589d-50a2-4bdb-ac78-59c1592c03e8\") " pod="openstack-kuttl-tests/swift-ring-rebalance-kvhc2" Dec 03 12:41:54 crc kubenswrapper[5052]: I1203 12:41:54.381592 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8d5c589d-50a2-4bdb-ac78-59c1592c03e8-combined-ca-bundle\") pod \"swift-ring-rebalance-kvhc2\" (UID: \"8d5c589d-50a2-4bdb-ac78-59c1592c03e8\") " pod="openstack-kuttl-tests/swift-ring-rebalance-kvhc2" Dec 03 12:41:54 crc kubenswrapper[5052]: I1203 12:41:54.381636 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/bd4392a2-9c70-4f3a-9050-ae6a6db01082-etc-swift\") pod \"swift-storage-0\" (UID: \"bd4392a2-9c70-4f3a-9050-ae6a6db01082\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:41:54 crc kubenswrapper[5052]: I1203 12:41:54.381709 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-8t7k4\" (UniqueName: \"kubernetes.io/projected/bd4392a2-9c70-4f3a-9050-ae6a6db01082-kube-api-access-8t7k4\") pod \"swift-storage-0\" (UID: \"bd4392a2-9c70-4f3a-9050-ae6a6db01082\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:41:54 crc kubenswrapper[5052]: E1203 12:41:54.381863 5052 projected.go:288] Couldn't get configMap openstack-kuttl-tests/swift-ring-files: configmap "swift-ring-files" not found Dec 03 12:41:54 crc kubenswrapper[5052]: E1203 12:41:54.381880 5052 projected.go:194] Error preparing data for projected volume etc-swift for pod openstack-kuttl-tests/swift-storage-0: configmap "swift-ring-files" not found Dec 03 12:41:54 crc kubenswrapper[5052]: I1203 12:41:54.381878 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage13-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage13-crc\") pod \"swift-storage-0\" (UID: \"bd4392a2-9c70-4f3a-9050-ae6a6db01082\") device mount path \"/mnt/openstack/pv13\"" pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:41:54 crc kubenswrapper[5052]: E1203 12:41:54.381935 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bd4392a2-9c70-4f3a-9050-ae6a6db01082-etc-swift podName:bd4392a2-9c70-4f3a-9050-ae6a6db01082 nodeName:}" failed. No retries permitted until 2025-12-03 12:41:54.881915516 +0000 UTC m=+4905.470157280 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "etc-swift" (UniqueName: "kubernetes.io/projected/bd4392a2-9c70-4f3a-9050-ae6a6db01082-etc-swift") pod "swift-storage-0" (UID: "bd4392a2-9c70-4f3a-9050-ae6a6db01082") : configmap "swift-ring-files" not found Dec 03 12:41:54 crc kubenswrapper[5052]: I1203 12:41:54.382105 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/8d5c589d-50a2-4bdb-ac78-59c1592c03e8-scripts\") pod \"swift-ring-rebalance-kvhc2\" (UID: \"8d5c589d-50a2-4bdb-ac78-59c1592c03e8\") " pod="openstack-kuttl-tests/swift-ring-rebalance-kvhc2" Dec 03 12:41:54 crc kubenswrapper[5052]: I1203 12:41:54.382140 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"lock\" (UniqueName: \"kubernetes.io/empty-dir/bd4392a2-9c70-4f3a-9050-ae6a6db01082-lock\") pod \"swift-storage-0\" (UID: \"bd4392a2-9c70-4f3a-9050-ae6a6db01082\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:41:54 crc kubenswrapper[5052]: I1203 12:41:54.382255 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dispersionconf\" (UniqueName: \"kubernetes.io/secret/8d5c589d-50a2-4bdb-ac78-59c1592c03e8-dispersionconf\") pod \"swift-ring-rebalance-kvhc2\" (UID: \"8d5c589d-50a2-4bdb-ac78-59c1592c03e8\") " pod="openstack-kuttl-tests/swift-ring-rebalance-kvhc2" Dec 03 12:41:54 crc kubenswrapper[5052]: I1203 12:41:54.382282 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"cache\" (UniqueName: \"kubernetes.io/empty-dir/bd4392a2-9c70-4f3a-9050-ae6a6db01082-cache\") pod \"swift-storage-0\" (UID: \"bd4392a2-9c70-4f3a-9050-ae6a6db01082\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:41:54 crc kubenswrapper[5052]: I1203 12:41:54.382665 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"lock\" (UniqueName: \"kubernetes.io/empty-dir/bd4392a2-9c70-4f3a-9050-ae6a6db01082-lock\") pod \"swift-storage-0\" (UID: \"bd4392a2-9c70-4f3a-9050-ae6a6db01082\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:41:54 crc kubenswrapper[5052]: I1203 12:41:54.382706 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"cache\" (UniqueName: \"kubernetes.io/empty-dir/bd4392a2-9c70-4f3a-9050-ae6a6db01082-cache\") pod \"swift-storage-0\" (UID: \"bd4392a2-9c70-4f3a-9050-ae6a6db01082\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:41:54 crc kubenswrapper[5052]: I1203 12:41:54.400940 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-8t7k4\" (UniqueName: \"kubernetes.io/projected/bd4392a2-9c70-4f3a-9050-ae6a6db01082-kube-api-access-8t7k4\") pod \"swift-storage-0\" (UID: \"bd4392a2-9c70-4f3a-9050-ae6a6db01082\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:41:54 crc kubenswrapper[5052]: I1203 12:41:54.403993 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage13-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage13-crc\") pod \"swift-storage-0\" (UID: \"bd4392a2-9c70-4f3a-9050-ae6a6db01082\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:41:54 crc kubenswrapper[5052]: I1203 12:41:54.483588 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/empty-dir/8d5c589d-50a2-4bdb-ac78-59c1592c03e8-etc-swift\") pod \"swift-ring-rebalance-kvhc2\" (UID: \"8d5c589d-50a2-4bdb-ac78-59c1592c03e8\") " pod="openstack-kuttl-tests/swift-ring-rebalance-kvhc2" Dec 03 12:41:54 crc kubenswrapper[5052]: I1203 12:41:54.483645 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8d5c589d-50a2-4bdb-ac78-59c1592c03e8-combined-ca-bundle\") pod \"swift-ring-rebalance-kvhc2\" (UID: \"8d5c589d-50a2-4bdb-ac78-59c1592c03e8\") " pod="openstack-kuttl-tests/swift-ring-rebalance-kvhc2" Dec 03 12:41:54 crc kubenswrapper[5052]: I1203 12:41:54.483715 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/8d5c589d-50a2-4bdb-ac78-59c1592c03e8-scripts\") pod \"swift-ring-rebalance-kvhc2\" (UID: \"8d5c589d-50a2-4bdb-ac78-59c1592c03e8\") " pod="openstack-kuttl-tests/swift-ring-rebalance-kvhc2" Dec 03 12:41:54 crc kubenswrapper[5052]: I1203 12:41:54.483777 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"dispersionconf\" (UniqueName: \"kubernetes.io/secret/8d5c589d-50a2-4bdb-ac78-59c1592c03e8-dispersionconf\") pod \"swift-ring-rebalance-kvhc2\" (UID: \"8d5c589d-50a2-4bdb-ac78-59c1592c03e8\") " pod="openstack-kuttl-tests/swift-ring-rebalance-kvhc2" Dec 03 12:41:54 crc kubenswrapper[5052]: I1203 12:41:54.483803 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-wztzb\" (UniqueName: \"kubernetes.io/projected/8d5c589d-50a2-4bdb-ac78-59c1592c03e8-kube-api-access-wztzb\") pod \"swift-ring-rebalance-kvhc2\" (UID: \"8d5c589d-50a2-4bdb-ac78-59c1592c03e8\") " pod="openstack-kuttl-tests/swift-ring-rebalance-kvhc2" Dec 03 12:41:54 crc kubenswrapper[5052]: I1203 12:41:54.483854 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"swiftconf\" (UniqueName: \"kubernetes.io/secret/8d5c589d-50a2-4bdb-ac78-59c1592c03e8-swiftconf\") pod \"swift-ring-rebalance-kvhc2\" (UID: \"8d5c589d-50a2-4bdb-ac78-59c1592c03e8\") " pod="openstack-kuttl-tests/swift-ring-rebalance-kvhc2" Dec 03 12:41:54 crc kubenswrapper[5052]: I1203 12:41:54.483873 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ring-data-devices\" (UniqueName: \"kubernetes.io/configmap/8d5c589d-50a2-4bdb-ac78-59c1592c03e8-ring-data-devices\") pod \"swift-ring-rebalance-kvhc2\" (UID: \"8d5c589d-50a2-4bdb-ac78-59c1592c03e8\") " pod="openstack-kuttl-tests/swift-ring-rebalance-kvhc2" Dec 03 12:41:54 crc kubenswrapper[5052]: I1203 12:41:54.484177 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"etc-swift\" (UniqueName: \"kubernetes.io/empty-dir/8d5c589d-50a2-4bdb-ac78-59c1592c03e8-etc-swift\") pod \"swift-ring-rebalance-kvhc2\" (UID: \"8d5c589d-50a2-4bdb-ac78-59c1592c03e8\") " pod="openstack-kuttl-tests/swift-ring-rebalance-kvhc2" Dec 03 12:41:54 crc kubenswrapper[5052]: I1203 12:41:54.484827 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ring-data-devices\" (UniqueName: \"kubernetes.io/configmap/8d5c589d-50a2-4bdb-ac78-59c1592c03e8-ring-data-devices\") pod \"swift-ring-rebalance-kvhc2\" (UID: \"8d5c589d-50a2-4bdb-ac78-59c1592c03e8\") " pod="openstack-kuttl-tests/swift-ring-rebalance-kvhc2" Dec 03 12:41:54 crc kubenswrapper[5052]: I1203 12:41:54.485255 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/8d5c589d-50a2-4bdb-ac78-59c1592c03e8-scripts\") pod \"swift-ring-rebalance-kvhc2\" (UID: \"8d5c589d-50a2-4bdb-ac78-59c1592c03e8\") " pod="openstack-kuttl-tests/swift-ring-rebalance-kvhc2" Dec 03 12:41:54 crc kubenswrapper[5052]: I1203 12:41:54.487460 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"swiftconf\" (UniqueName: \"kubernetes.io/secret/8d5c589d-50a2-4bdb-ac78-59c1592c03e8-swiftconf\") pod \"swift-ring-rebalance-kvhc2\" (UID: \"8d5c589d-50a2-4bdb-ac78-59c1592c03e8\") " pod="openstack-kuttl-tests/swift-ring-rebalance-kvhc2" Dec 03 12:41:54 crc kubenswrapper[5052]: I1203 12:41:54.487699 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8d5c589d-50a2-4bdb-ac78-59c1592c03e8-combined-ca-bundle\") pod \"swift-ring-rebalance-kvhc2\" (UID: \"8d5c589d-50a2-4bdb-ac78-59c1592c03e8\") " pod="openstack-kuttl-tests/swift-ring-rebalance-kvhc2" Dec 03 12:41:54 crc kubenswrapper[5052]: I1203 12:41:54.488761 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"dispersionconf\" (UniqueName: \"kubernetes.io/secret/8d5c589d-50a2-4bdb-ac78-59c1592c03e8-dispersionconf\") pod \"swift-ring-rebalance-kvhc2\" (UID: \"8d5c589d-50a2-4bdb-ac78-59c1592c03e8\") " pod="openstack-kuttl-tests/swift-ring-rebalance-kvhc2" Dec 03 12:41:54 crc kubenswrapper[5052]: I1203 12:41:54.513243 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-wztzb\" (UniqueName: \"kubernetes.io/projected/8d5c589d-50a2-4bdb-ac78-59c1592c03e8-kube-api-access-wztzb\") pod \"swift-ring-rebalance-kvhc2\" (UID: \"8d5c589d-50a2-4bdb-ac78-59c1592c03e8\") " pod="openstack-kuttl-tests/swift-ring-rebalance-kvhc2" Dec 03 12:41:54 crc kubenswrapper[5052]: I1203 12:41:54.547395 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-ring-rebalance-kvhc2" Dec 03 12:41:54 crc kubenswrapper[5052]: I1203 12:41:54.892444 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/bd4392a2-9c70-4f3a-9050-ae6a6db01082-etc-swift\") pod \"swift-storage-0\" (UID: \"bd4392a2-9c70-4f3a-9050-ae6a6db01082\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:41:54 crc kubenswrapper[5052]: E1203 12:41:54.892604 5052 projected.go:288] Couldn't get configMap openstack-kuttl-tests/swift-ring-files: configmap "swift-ring-files" not found Dec 03 12:41:54 crc kubenswrapper[5052]: E1203 12:41:54.892914 5052 projected.go:194] Error preparing data for projected volume etc-swift for pod openstack-kuttl-tests/swift-storage-0: configmap "swift-ring-files" not found Dec 03 12:41:54 crc kubenswrapper[5052]: E1203 12:41:54.892984 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bd4392a2-9c70-4f3a-9050-ae6a6db01082-etc-swift podName:bd4392a2-9c70-4f3a-9050-ae6a6db01082 nodeName:}" failed. No retries permitted until 2025-12-03 12:41:55.892950601 +0000 UTC m=+4906.481192365 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "etc-swift" (UniqueName: "kubernetes.io/projected/bd4392a2-9c70-4f3a-9050-ae6a6db01082-etc-swift") pod "swift-storage-0" (UID: "bd4392a2-9c70-4f3a-9050-ae6a6db01082") : configmap "swift-ring-files" not found Dec 03 12:41:54 crc kubenswrapper[5052]: I1203 12:41:54.994155 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/swift-ring-rebalance-kvhc2"] Dec 03 12:41:54 crc kubenswrapper[5052]: W1203 12:41:54.997499 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8d5c589d_50a2_4bdb_ac78_59c1592c03e8.slice/crio-b3b5886b15c04a5016f83c6bcfa39a563e18111d10444620e7140a650a766205 WatchSource:0}: Error finding container b3b5886b15c04a5016f83c6bcfa39a563e18111d10444620e7140a650a766205: Status 404 returned error can't find the container with id b3b5886b15c04a5016f83c6bcfa39a563e18111d10444620e7140a650a766205 Dec 03 12:41:55 crc kubenswrapper[5052]: I1203 12:41:55.115326 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-ring-rebalance-kvhc2" event={"ID":"8d5c589d-50a2-4bdb-ac78-59c1592c03e8","Type":"ContainerStarted","Data":"b3b5886b15c04a5016f83c6bcfa39a563e18111d10444620e7140a650a766205"} Dec 03 12:41:55 crc kubenswrapper[5052]: I1203 12:41:55.156743 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:41:55 crc kubenswrapper[5052]: I1203 12:41:55.186300 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:41:55 crc kubenswrapper[5052]: I1203 12:41:55.197198 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:41:55 crc kubenswrapper[5052]: I1203 12:41:55.258967 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:41:55 crc kubenswrapper[5052]: I1203 12:41:55.428733 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-db-create-zjnt8" Dec 03 12:41:55 crc kubenswrapper[5052]: I1203 12:41:55.508422 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/8b6e373e-2587-415b-8cbc-5fef0ba3427b-operator-scripts\") pod \"8b6e373e-2587-415b-8cbc-5fef0ba3427b\" (UID: \"8b6e373e-2587-415b-8cbc-5fef0ba3427b\") " Dec 03 12:41:55 crc kubenswrapper[5052]: I1203 12:41:55.508482 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2ktnp\" (UniqueName: \"kubernetes.io/projected/8b6e373e-2587-415b-8cbc-5fef0ba3427b-kube-api-access-2ktnp\") pod \"8b6e373e-2587-415b-8cbc-5fef0ba3427b\" (UID: \"8b6e373e-2587-415b-8cbc-5fef0ba3427b\") " Dec 03 12:41:55 crc kubenswrapper[5052]: I1203 12:41:55.513068 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8b6e373e-2587-415b-8cbc-5fef0ba3427b-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "8b6e373e-2587-415b-8cbc-5fef0ba3427b" (UID: "8b6e373e-2587-415b-8cbc-5fef0ba3427b"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:41:55 crc kubenswrapper[5052]: I1203 12:41:55.521059 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ovn-northd-0"] Dec 03 12:41:55 crc kubenswrapper[5052]: E1203 12:41:55.522446 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="8b6e373e-2587-415b-8cbc-5fef0ba3427b" containerName="mariadb-database-create" Dec 03 12:41:55 crc kubenswrapper[5052]: I1203 12:41:55.522476 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="8b6e373e-2587-415b-8cbc-5fef0ba3427b" containerName="mariadb-database-create" Dec 03 12:41:55 crc kubenswrapper[5052]: I1203 12:41:55.522737 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="8b6e373e-2587-415b-8cbc-5fef0ba3427b" containerName="mariadb-database-create" Dec 03 12:41:55 crc kubenswrapper[5052]: I1203 12:41:55.523253 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8b6e373e-2587-415b-8cbc-5fef0ba3427b-kube-api-access-2ktnp" (OuterVolumeSpecName: "kube-api-access-2ktnp") pod "8b6e373e-2587-415b-8cbc-5fef0ba3427b" (UID: "8b6e373e-2587-415b-8cbc-5fef0ba3427b"). InnerVolumeSpecName "kube-api-access-2ktnp". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:41:55 crc kubenswrapper[5052]: I1203 12:41:55.523909 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:41:55 crc kubenswrapper[5052]: I1203 12:41:55.536434 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-ovnnorthd-ovndbs" Dec 03 12:41:55 crc kubenswrapper[5052]: I1203 12:41:55.536842 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ovnnorthd-ovnnorthd-dockercfg-lhpt7" Dec 03 12:41:55 crc kubenswrapper[5052]: I1203 12:41:55.537612 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"ovnnorthd-scripts" Dec 03 12:41:55 crc kubenswrapper[5052]: I1203 12:41:55.537812 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"ovnnorthd-config" Dec 03 12:41:55 crc kubenswrapper[5052]: I1203 12:41:55.544017 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ovn-northd-0"] Dec 03 12:41:55 crc kubenswrapper[5052]: I1203 12:41:55.609902 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d89l2\" (UniqueName: \"kubernetes.io/projected/6af0a301-2483-493d-bbc8-17138d79dfea-kube-api-access-d89l2\") pod \"ovn-northd-0\" (UID: \"6af0a301-2483-493d-bbc8-17138d79dfea\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:41:55 crc kubenswrapper[5052]: I1203 12:41:55.610140 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6af0a301-2483-493d-bbc8-17138d79dfea-combined-ca-bundle\") pod \"ovn-northd-0\" (UID: \"6af0a301-2483-493d-bbc8-17138d79dfea\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:41:55 crc kubenswrapper[5052]: I1203 12:41:55.610215 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-northd-tls-certs\" (UniqueName: \"kubernetes.io/secret/6af0a301-2483-493d-bbc8-17138d79dfea-ovn-northd-tls-certs\") pod \"ovn-northd-0\" (UID: \"6af0a301-2483-493d-bbc8-17138d79dfea\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:41:55 crc kubenswrapper[5052]: I1203 12:41:55.610291 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/6af0a301-2483-493d-bbc8-17138d79dfea-config\") pod \"ovn-northd-0\" (UID: \"6af0a301-2483-493d-bbc8-17138d79dfea\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:41:55 crc kubenswrapper[5052]: I1203 12:41:55.610386 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/6af0a301-2483-493d-bbc8-17138d79dfea-metrics-certs-tls-certs\") pod \"ovn-northd-0\" (UID: \"6af0a301-2483-493d-bbc8-17138d79dfea\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:41:55 crc kubenswrapper[5052]: I1203 12:41:55.610498 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/6af0a301-2483-493d-bbc8-17138d79dfea-scripts\") pod \"ovn-northd-0\" (UID: \"6af0a301-2483-493d-bbc8-17138d79dfea\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:41:55 crc kubenswrapper[5052]: I1203 12:41:55.610614 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-rundir\" (UniqueName: \"kubernetes.io/empty-dir/6af0a301-2483-493d-bbc8-17138d79dfea-ovn-rundir\") pod \"ovn-northd-0\" (UID: \"6af0a301-2483-493d-bbc8-17138d79dfea\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:41:55 crc kubenswrapper[5052]: I1203 12:41:55.610732 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/8b6e373e-2587-415b-8cbc-5fef0ba3427b-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:55 crc kubenswrapper[5052]: I1203 12:41:55.610797 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-2ktnp\" (UniqueName: \"kubernetes.io/projected/8b6e373e-2587-415b-8cbc-5fef0ba3427b-kube-api-access-2ktnp\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:55 crc kubenswrapper[5052]: I1203 12:41:55.711797 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-d89l2\" (UniqueName: \"kubernetes.io/projected/6af0a301-2483-493d-bbc8-17138d79dfea-kube-api-access-d89l2\") pod \"ovn-northd-0\" (UID: \"6af0a301-2483-493d-bbc8-17138d79dfea\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:41:55 crc kubenswrapper[5052]: I1203 12:41:55.711885 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6af0a301-2483-493d-bbc8-17138d79dfea-combined-ca-bundle\") pod \"ovn-northd-0\" (UID: \"6af0a301-2483-493d-bbc8-17138d79dfea\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:41:55 crc kubenswrapper[5052]: I1203 12:41:55.711914 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovn-northd-tls-certs\" (UniqueName: \"kubernetes.io/secret/6af0a301-2483-493d-bbc8-17138d79dfea-ovn-northd-tls-certs\") pod \"ovn-northd-0\" (UID: \"6af0a301-2483-493d-bbc8-17138d79dfea\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:41:55 crc kubenswrapper[5052]: I1203 12:41:55.711938 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/6af0a301-2483-493d-bbc8-17138d79dfea-config\") pod \"ovn-northd-0\" (UID: \"6af0a301-2483-493d-bbc8-17138d79dfea\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:41:55 crc kubenswrapper[5052]: I1203 12:41:55.711996 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/6af0a301-2483-493d-bbc8-17138d79dfea-metrics-certs-tls-certs\") pod \"ovn-northd-0\" (UID: \"6af0a301-2483-493d-bbc8-17138d79dfea\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:41:55 crc kubenswrapper[5052]: I1203 12:41:55.712081 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/6af0a301-2483-493d-bbc8-17138d79dfea-scripts\") pod \"ovn-northd-0\" (UID: \"6af0a301-2483-493d-bbc8-17138d79dfea\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:41:55 crc kubenswrapper[5052]: I1203 12:41:55.712135 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovn-rundir\" (UniqueName: \"kubernetes.io/empty-dir/6af0a301-2483-493d-bbc8-17138d79dfea-ovn-rundir\") pod \"ovn-northd-0\" (UID: \"6af0a301-2483-493d-bbc8-17138d79dfea\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:41:55 crc kubenswrapper[5052]: I1203 12:41:55.712788 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovn-rundir\" (UniqueName: \"kubernetes.io/empty-dir/6af0a301-2483-493d-bbc8-17138d79dfea-ovn-rundir\") pod \"ovn-northd-0\" (UID: \"6af0a301-2483-493d-bbc8-17138d79dfea\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:41:55 crc kubenswrapper[5052]: I1203 12:41:55.714021 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/6af0a301-2483-493d-bbc8-17138d79dfea-scripts\") pod \"ovn-northd-0\" (UID: \"6af0a301-2483-493d-bbc8-17138d79dfea\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:41:55 crc kubenswrapper[5052]: I1203 12:41:55.715087 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/6af0a301-2483-493d-bbc8-17138d79dfea-config\") pod \"ovn-northd-0\" (UID: \"6af0a301-2483-493d-bbc8-17138d79dfea\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:41:55 crc kubenswrapper[5052]: I1203 12:41:55.717663 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6af0a301-2483-493d-bbc8-17138d79dfea-combined-ca-bundle\") pod \"ovn-northd-0\" (UID: \"6af0a301-2483-493d-bbc8-17138d79dfea\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:41:55 crc kubenswrapper[5052]: I1203 12:41:55.718369 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovn-northd-tls-certs\" (UniqueName: \"kubernetes.io/secret/6af0a301-2483-493d-bbc8-17138d79dfea-ovn-northd-tls-certs\") pod \"ovn-northd-0\" (UID: \"6af0a301-2483-493d-bbc8-17138d79dfea\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:41:55 crc kubenswrapper[5052]: I1203 12:41:55.718913 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/6af0a301-2483-493d-bbc8-17138d79dfea-metrics-certs-tls-certs\") pod \"ovn-northd-0\" (UID: \"6af0a301-2483-493d-bbc8-17138d79dfea\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:41:55 crc kubenswrapper[5052]: I1203 12:41:55.734451 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-d89l2\" (UniqueName: \"kubernetes.io/projected/6af0a301-2483-493d-bbc8-17138d79dfea-kube-api-access-d89l2\") pod \"ovn-northd-0\" (UID: \"6af0a301-2483-493d-bbc8-17138d79dfea\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:41:55 crc kubenswrapper[5052]: I1203 12:41:55.771283 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-db-create-69hg4" Dec 03 12:41:55 crc kubenswrapper[5052]: I1203 12:41:55.785864 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-b51b-account-create-update-h6wlv" Dec 03 12:41:55 crc kubenswrapper[5052]: I1203 12:41:55.792073 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-361c-account-create-update-4psp5" Dec 03 12:41:55 crc kubenswrapper[5052]: I1203 12:41:55.812968 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/2b4ed2db-d186-4bf9-8616-44f1249e7af1-operator-scripts\") pod \"2b4ed2db-d186-4bf9-8616-44f1249e7af1\" (UID: \"2b4ed2db-d186-4bf9-8616-44f1249e7af1\") " Dec 03 12:41:55 crc kubenswrapper[5052]: I1203 12:41:55.813033 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qnntw\" (UniqueName: \"kubernetes.io/projected/2b4ed2db-d186-4bf9-8616-44f1249e7af1-kube-api-access-qnntw\") pod \"2b4ed2db-d186-4bf9-8616-44f1249e7af1\" (UID: \"2b4ed2db-d186-4bf9-8616-44f1249e7af1\") " Dec 03 12:41:55 crc kubenswrapper[5052]: I1203 12:41:55.818358 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2b4ed2db-d186-4bf9-8616-44f1249e7af1-kube-api-access-qnntw" (OuterVolumeSpecName: "kube-api-access-qnntw") pod "2b4ed2db-d186-4bf9-8616-44f1249e7af1" (UID: "2b4ed2db-d186-4bf9-8616-44f1249e7af1"). InnerVolumeSpecName "kube-api-access-qnntw". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:41:55 crc kubenswrapper[5052]: I1203 12:41:55.822037 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2b4ed2db-d186-4bf9-8616-44f1249e7af1-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "2b4ed2db-d186-4bf9-8616-44f1249e7af1" (UID: "2b4ed2db-d186-4bf9-8616-44f1249e7af1"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:41:55 crc kubenswrapper[5052]: I1203 12:41:55.891055 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:41:55 crc kubenswrapper[5052]: I1203 12:41:55.914782 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hfhfn\" (UniqueName: \"kubernetes.io/projected/5ecf0384-e075-4985-98c9-017cf55e0f7d-kube-api-access-hfhfn\") pod \"5ecf0384-e075-4985-98c9-017cf55e0f7d\" (UID: \"5ecf0384-e075-4985-98c9-017cf55e0f7d\") " Dec 03 12:41:55 crc kubenswrapper[5052]: I1203 12:41:55.914914 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/7f78d8a2-cb31-4277-a6f6-51bcb6d4ed15-operator-scripts\") pod \"7f78d8a2-cb31-4277-a6f6-51bcb6d4ed15\" (UID: \"7f78d8a2-cb31-4277-a6f6-51bcb6d4ed15\") " Dec 03 12:41:55 crc kubenswrapper[5052]: I1203 12:41:55.914996 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fxs5v\" (UniqueName: \"kubernetes.io/projected/7f78d8a2-cb31-4277-a6f6-51bcb6d4ed15-kube-api-access-fxs5v\") pod \"7f78d8a2-cb31-4277-a6f6-51bcb6d4ed15\" (UID: \"7f78d8a2-cb31-4277-a6f6-51bcb6d4ed15\") " Dec 03 12:41:55 crc kubenswrapper[5052]: I1203 12:41:55.915073 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/5ecf0384-e075-4985-98c9-017cf55e0f7d-operator-scripts\") pod \"5ecf0384-e075-4985-98c9-017cf55e0f7d\" (UID: \"5ecf0384-e075-4985-98c9-017cf55e0f7d\") " Dec 03 12:41:55 crc kubenswrapper[5052]: I1203 12:41:55.915334 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/bd4392a2-9c70-4f3a-9050-ae6a6db01082-etc-swift\") pod \"swift-storage-0\" (UID: \"bd4392a2-9c70-4f3a-9050-ae6a6db01082\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:41:55 crc kubenswrapper[5052]: I1203 12:41:55.915446 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/2b4ed2db-d186-4bf9-8616-44f1249e7af1-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:55 crc kubenswrapper[5052]: I1203 12:41:55.915462 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-qnntw\" (UniqueName: \"kubernetes.io/projected/2b4ed2db-d186-4bf9-8616-44f1249e7af1-kube-api-access-qnntw\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:55 crc kubenswrapper[5052]: E1203 12:41:55.915573 5052 projected.go:288] Couldn't get configMap openstack-kuttl-tests/swift-ring-files: configmap "swift-ring-files" not found Dec 03 12:41:55 crc kubenswrapper[5052]: E1203 12:41:55.915590 5052 projected.go:194] Error preparing data for projected volume etc-swift for pod openstack-kuttl-tests/swift-storage-0: configmap "swift-ring-files" not found Dec 03 12:41:55 crc kubenswrapper[5052]: E1203 12:41:55.915632 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bd4392a2-9c70-4f3a-9050-ae6a6db01082-etc-swift podName:bd4392a2-9c70-4f3a-9050-ae6a6db01082 nodeName:}" failed. No retries permitted until 2025-12-03 12:41:57.915617358 +0000 UTC m=+4908.503859122 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "etc-swift" (UniqueName: "kubernetes.io/projected/bd4392a2-9c70-4f3a-9050-ae6a6db01082-etc-swift") pod "swift-storage-0" (UID: "bd4392a2-9c70-4f3a-9050-ae6a6db01082") : configmap "swift-ring-files" not found Dec 03 12:41:55 crc kubenswrapper[5052]: I1203 12:41:55.915872 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7f78d8a2-cb31-4277-a6f6-51bcb6d4ed15-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "7f78d8a2-cb31-4277-a6f6-51bcb6d4ed15" (UID: "7f78d8a2-cb31-4277-a6f6-51bcb6d4ed15"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:41:55 crc kubenswrapper[5052]: I1203 12:41:55.916136 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5ecf0384-e075-4985-98c9-017cf55e0f7d-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "5ecf0384-e075-4985-98c9-017cf55e0f7d" (UID: "5ecf0384-e075-4985-98c9-017cf55e0f7d"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:41:55 crc kubenswrapper[5052]: I1203 12:41:55.919348 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7f78d8a2-cb31-4277-a6f6-51bcb6d4ed15-kube-api-access-fxs5v" (OuterVolumeSpecName: "kube-api-access-fxs5v") pod "7f78d8a2-cb31-4277-a6f6-51bcb6d4ed15" (UID: "7f78d8a2-cb31-4277-a6f6-51bcb6d4ed15"). InnerVolumeSpecName "kube-api-access-fxs5v". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:41:55 crc kubenswrapper[5052]: I1203 12:41:55.919530 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5ecf0384-e075-4985-98c9-017cf55e0f7d-kube-api-access-hfhfn" (OuterVolumeSpecName: "kube-api-access-hfhfn") pod "5ecf0384-e075-4985-98c9-017cf55e0f7d" (UID: "5ecf0384-e075-4985-98c9-017cf55e0f7d"). InnerVolumeSpecName "kube-api-access-hfhfn". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:41:56 crc kubenswrapper[5052]: I1203 12:41:56.018394 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/5ecf0384-e075-4985-98c9-017cf55e0f7d-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:56 crc kubenswrapper[5052]: I1203 12:41:56.018434 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-hfhfn\" (UniqueName: \"kubernetes.io/projected/5ecf0384-e075-4985-98c9-017cf55e0f7d-kube-api-access-hfhfn\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:56 crc kubenswrapper[5052]: I1203 12:41:56.018449 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/7f78d8a2-cb31-4277-a6f6-51bcb6d4ed15-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:56 crc kubenswrapper[5052]: I1203 12:41:56.018465 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-fxs5v\" (UniqueName: \"kubernetes.io/projected/7f78d8a2-cb31-4277-a6f6-51bcb6d4ed15-kube-api-access-fxs5v\") on node \"crc\" DevicePath \"\"" Dec 03 12:41:56 crc kubenswrapper[5052]: I1203 12:41:56.127119 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-b51b-account-create-update-h6wlv" Dec 03 12:41:56 crc kubenswrapper[5052]: I1203 12:41:56.127140 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-b51b-account-create-update-h6wlv" event={"ID":"5ecf0384-e075-4985-98c9-017cf55e0f7d","Type":"ContainerDied","Data":"f44ad0dad227a521dffa45530a330e188049d5547536b51296b46008b7f4a144"} Dec 03 12:41:56 crc kubenswrapper[5052]: I1203 12:41:56.127772 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f44ad0dad227a521dffa45530a330e188049d5547536b51296b46008b7f4a144" Dec 03 12:41:56 crc kubenswrapper[5052]: I1203 12:41:56.141876 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-ring-rebalance-kvhc2" event={"ID":"8d5c589d-50a2-4bdb-ac78-59c1592c03e8","Type":"ContainerStarted","Data":"5d97dda03f4bb7760eec9dbecdc78fddde19c90f53bee737bf5ab871d649c4ae"} Dec 03 12:41:56 crc kubenswrapper[5052]: I1203 12:41:56.151116 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-db-create-zjnt8" Dec 03 12:41:56 crc kubenswrapper[5052]: I1203 12:41:56.151389 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-db-create-zjnt8" event={"ID":"8b6e373e-2587-415b-8cbc-5fef0ba3427b","Type":"ContainerDied","Data":"6509e966e980feccbfd01d7eb11af4fcd151e1080f714d65b4834ea2fe188ec2"} Dec 03 12:41:56 crc kubenswrapper[5052]: I1203 12:41:56.151471 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6509e966e980feccbfd01d7eb11af4fcd151e1080f714d65b4834ea2fe188ec2" Dec 03 12:41:56 crc kubenswrapper[5052]: I1203 12:41:56.154671 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-361c-account-create-update-4psp5" event={"ID":"7f78d8a2-cb31-4277-a6f6-51bcb6d4ed15","Type":"ContainerDied","Data":"ea03469d014c723497d4bfee22ce5b9ffd0adb13e988c34300527f50478aa8b1"} Dec 03 12:41:56 crc kubenswrapper[5052]: I1203 12:41:56.154695 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ea03469d014c723497d4bfee22ce5b9ffd0adb13e988c34300527f50478aa8b1" Dec 03 12:41:56 crc kubenswrapper[5052]: I1203 12:41:56.154744 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-361c-account-create-update-4psp5" Dec 03 12:41:56 crc kubenswrapper[5052]: I1203 12:41:56.161057 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-db-create-69hg4" Dec 03 12:41:56 crc kubenswrapper[5052]: I1203 12:41:56.163050 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-db-create-69hg4" event={"ID":"2b4ed2db-d186-4bf9-8616-44f1249e7af1","Type":"ContainerDied","Data":"4ba70b6506f8ca2f73348d1e8401ae23ffcfe18c72815e1a45df826e35e84397"} Dec 03 12:41:56 crc kubenswrapper[5052]: I1203 12:41:56.163123 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4ba70b6506f8ca2f73348d1e8401ae23ffcfe18c72815e1a45df826e35e84397" Dec 03 12:41:56 crc kubenswrapper[5052]: I1203 12:41:56.205708 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/swift-ring-rebalance-kvhc2" podStartSLOduration=2.205688564 podStartE2EDuration="2.205688564s" podCreationTimestamp="2025-12-03 12:41:54 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:41:56.165661379 +0000 UTC m=+4906.753903143" watchObservedRunningTime="2025-12-03 12:41:56.205688564 +0000 UTC m=+4906.793930328" Dec 03 12:41:56 crc kubenswrapper[5052]: I1203 12:41:56.361799 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ovn-northd-0"] Dec 03 12:41:57 crc kubenswrapper[5052]: I1203 12:41:57.171336 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovn-northd-0" event={"ID":"6af0a301-2483-493d-bbc8-17138d79dfea","Type":"ContainerStarted","Data":"4307735eba8d16b00eb4ca5f1d746b675f8e81debcf3a1c8d9b709367fb6a8be"} Dec 03 12:41:57 crc kubenswrapper[5052]: I1203 12:41:57.171667 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovn-northd-0" event={"ID":"6af0a301-2483-493d-bbc8-17138d79dfea","Type":"ContainerStarted","Data":"adc4f6d0e36c4370a4bff93d0b0803fec09e3b34230d31ebbe4f500396923a29"} Dec 03 12:41:57 crc kubenswrapper[5052]: I1203 12:41:57.171683 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovn-northd-0" event={"ID":"6af0a301-2483-493d-bbc8-17138d79dfea","Type":"ContainerStarted","Data":"fc42c73e23176434d6c24f8df9babe1e324d8caa46125fa1de4d5f4f4ddcc90d"} Dec 03 12:41:57 crc kubenswrapper[5052]: I1203 12:41:57.171712 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:41:57 crc kubenswrapper[5052]: I1203 12:41:57.182988 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/glance-db-create-mfcd6"] Dec 03 12:41:57 crc kubenswrapper[5052]: E1203 12:41:57.183337 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="2b4ed2db-d186-4bf9-8616-44f1249e7af1" containerName="mariadb-database-create" Dec 03 12:41:57 crc kubenswrapper[5052]: I1203 12:41:57.183353 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="2b4ed2db-d186-4bf9-8616-44f1249e7af1" containerName="mariadb-database-create" Dec 03 12:41:57 crc kubenswrapper[5052]: E1203 12:41:57.183377 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="5ecf0384-e075-4985-98c9-017cf55e0f7d" containerName="mariadb-account-create-update" Dec 03 12:41:57 crc kubenswrapper[5052]: I1203 12:41:57.183384 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ecf0384-e075-4985-98c9-017cf55e0f7d" containerName="mariadb-account-create-update" Dec 03 12:41:57 crc kubenswrapper[5052]: E1203 12:41:57.183396 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="7f78d8a2-cb31-4277-a6f6-51bcb6d4ed15" containerName="mariadb-account-create-update" Dec 03 12:41:57 crc kubenswrapper[5052]: I1203 12:41:57.183401 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f78d8a2-cb31-4277-a6f6-51bcb6d4ed15" containerName="mariadb-account-create-update" Dec 03 12:41:57 crc kubenswrapper[5052]: I1203 12:41:57.184247 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="7f78d8a2-cb31-4277-a6f6-51bcb6d4ed15" containerName="mariadb-account-create-update" Dec 03 12:41:57 crc kubenswrapper[5052]: I1203 12:41:57.184271 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="2b4ed2db-d186-4bf9-8616-44f1249e7af1" containerName="mariadb-database-create" Dec 03 12:41:57 crc kubenswrapper[5052]: I1203 12:41:57.184285 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="5ecf0384-e075-4985-98c9-017cf55e0f7d" containerName="mariadb-account-create-update" Dec 03 12:41:57 crc kubenswrapper[5052]: I1203 12:41:57.184872 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-db-create-mfcd6" Dec 03 12:41:57 crc kubenswrapper[5052]: I1203 12:41:57.198381 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-db-create-mfcd6"] Dec 03 12:41:57 crc kubenswrapper[5052]: I1203 12:41:57.203246 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/ovn-northd-0" podStartSLOduration=2.203229961 podStartE2EDuration="2.203229961s" podCreationTimestamp="2025-12-03 12:41:55 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:41:57.201368178 +0000 UTC m=+4907.789609942" watchObservedRunningTime="2025-12-03 12:41:57.203229961 +0000 UTC m=+4907.791471725" Dec 03 12:41:57 crc kubenswrapper[5052]: I1203 12:41:57.243685 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/5f011b7b-96e7-4236-a0fc-b7e292e232fd-operator-scripts\") pod \"glance-db-create-mfcd6\" (UID: \"5f011b7b-96e7-4236-a0fc-b7e292e232fd\") " pod="openstack-kuttl-tests/glance-db-create-mfcd6" Dec 03 12:41:57 crc kubenswrapper[5052]: I1203 12:41:57.244125 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4z9cv\" (UniqueName: \"kubernetes.io/projected/5f011b7b-96e7-4236-a0fc-b7e292e232fd-kube-api-access-4z9cv\") pod \"glance-db-create-mfcd6\" (UID: \"5f011b7b-96e7-4236-a0fc-b7e292e232fd\") " pod="openstack-kuttl-tests/glance-db-create-mfcd6" Dec 03 12:41:57 crc kubenswrapper[5052]: I1203 12:41:57.293218 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/glance-d4a6-account-create-update-8qdvr"] Dec 03 12:41:57 crc kubenswrapper[5052]: I1203 12:41:57.294726 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-d4a6-account-create-update-8qdvr" Dec 03 12:41:57 crc kubenswrapper[5052]: I1203 12:41:57.297168 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-db-secret" Dec 03 12:41:57 crc kubenswrapper[5052]: I1203 12:41:57.309393 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-d4a6-account-create-update-8qdvr"] Dec 03 12:41:57 crc kubenswrapper[5052]: I1203 12:41:57.346231 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/5f011b7b-96e7-4236-a0fc-b7e292e232fd-operator-scripts\") pod \"glance-db-create-mfcd6\" (UID: \"5f011b7b-96e7-4236-a0fc-b7e292e232fd\") " pod="openstack-kuttl-tests/glance-db-create-mfcd6" Dec 03 12:41:57 crc kubenswrapper[5052]: I1203 12:41:57.346591 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-4z9cv\" (UniqueName: \"kubernetes.io/projected/5f011b7b-96e7-4236-a0fc-b7e292e232fd-kube-api-access-4z9cv\") pod \"glance-db-create-mfcd6\" (UID: \"5f011b7b-96e7-4236-a0fc-b7e292e232fd\") " pod="openstack-kuttl-tests/glance-db-create-mfcd6" Dec 03 12:41:57 crc kubenswrapper[5052]: I1203 12:41:57.346741 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/861301e3-bb06-498b-9225-b72962e175bc-operator-scripts\") pod \"glance-d4a6-account-create-update-8qdvr\" (UID: \"861301e3-bb06-498b-9225-b72962e175bc\") " pod="openstack-kuttl-tests/glance-d4a6-account-create-update-8qdvr" Dec 03 12:41:57 crc kubenswrapper[5052]: I1203 12:41:57.346973 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gdlxq\" (UniqueName: \"kubernetes.io/projected/861301e3-bb06-498b-9225-b72962e175bc-kube-api-access-gdlxq\") pod \"glance-d4a6-account-create-update-8qdvr\" (UID: \"861301e3-bb06-498b-9225-b72962e175bc\") " pod="openstack-kuttl-tests/glance-d4a6-account-create-update-8qdvr" Dec 03 12:41:57 crc kubenswrapper[5052]: I1203 12:41:57.348427 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/5f011b7b-96e7-4236-a0fc-b7e292e232fd-operator-scripts\") pod \"glance-db-create-mfcd6\" (UID: \"5f011b7b-96e7-4236-a0fc-b7e292e232fd\") " pod="openstack-kuttl-tests/glance-db-create-mfcd6" Dec 03 12:41:57 crc kubenswrapper[5052]: I1203 12:41:57.371344 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-4z9cv\" (UniqueName: \"kubernetes.io/projected/5f011b7b-96e7-4236-a0fc-b7e292e232fd-kube-api-access-4z9cv\") pod \"glance-db-create-mfcd6\" (UID: \"5f011b7b-96e7-4236-a0fc-b7e292e232fd\") " pod="openstack-kuttl-tests/glance-db-create-mfcd6" Dec 03 12:41:57 crc kubenswrapper[5052]: I1203 12:41:57.448069 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-gdlxq\" (UniqueName: \"kubernetes.io/projected/861301e3-bb06-498b-9225-b72962e175bc-kube-api-access-gdlxq\") pod \"glance-d4a6-account-create-update-8qdvr\" (UID: \"861301e3-bb06-498b-9225-b72962e175bc\") " pod="openstack-kuttl-tests/glance-d4a6-account-create-update-8qdvr" Dec 03 12:41:57 crc kubenswrapper[5052]: I1203 12:41:57.448421 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/861301e3-bb06-498b-9225-b72962e175bc-operator-scripts\") pod \"glance-d4a6-account-create-update-8qdvr\" (UID: \"861301e3-bb06-498b-9225-b72962e175bc\") " pod="openstack-kuttl-tests/glance-d4a6-account-create-update-8qdvr" Dec 03 12:41:57 crc kubenswrapper[5052]: I1203 12:41:57.449126 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/861301e3-bb06-498b-9225-b72962e175bc-operator-scripts\") pod \"glance-d4a6-account-create-update-8qdvr\" (UID: \"861301e3-bb06-498b-9225-b72962e175bc\") " pod="openstack-kuttl-tests/glance-d4a6-account-create-update-8qdvr" Dec 03 12:41:57 crc kubenswrapper[5052]: I1203 12:41:57.465925 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-gdlxq\" (UniqueName: \"kubernetes.io/projected/861301e3-bb06-498b-9225-b72962e175bc-kube-api-access-gdlxq\") pod \"glance-d4a6-account-create-update-8qdvr\" (UID: \"861301e3-bb06-498b-9225-b72962e175bc\") " pod="openstack-kuttl-tests/glance-d4a6-account-create-update-8qdvr" Dec 03 12:41:57 crc kubenswrapper[5052]: I1203 12:41:57.504280 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-db-create-mfcd6" Dec 03 12:41:57 crc kubenswrapper[5052]: I1203 12:41:57.614808 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-d4a6-account-create-update-8qdvr" Dec 03 12:41:57 crc kubenswrapper[5052]: I1203 12:41:57.964593 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/bd4392a2-9c70-4f3a-9050-ae6a6db01082-etc-swift\") pod \"swift-storage-0\" (UID: \"bd4392a2-9c70-4f3a-9050-ae6a6db01082\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:41:57 crc kubenswrapper[5052]: E1203 12:41:57.965144 5052 projected.go:288] Couldn't get configMap openstack-kuttl-tests/swift-ring-files: configmap "swift-ring-files" not found Dec 03 12:41:57 crc kubenswrapper[5052]: E1203 12:41:57.965163 5052 projected.go:194] Error preparing data for projected volume etc-swift for pod openstack-kuttl-tests/swift-storage-0: configmap "swift-ring-files" not found Dec 03 12:41:57 crc kubenswrapper[5052]: E1203 12:41:57.965212 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bd4392a2-9c70-4f3a-9050-ae6a6db01082-etc-swift podName:bd4392a2-9c70-4f3a-9050-ae6a6db01082 nodeName:}" failed. No retries permitted until 2025-12-03 12:42:01.965196282 +0000 UTC m=+4912.553438046 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "etc-swift" (UniqueName: "kubernetes.io/projected/bd4392a2-9c70-4f3a-9050-ae6a6db01082-etc-swift") pod "swift-storage-0" (UID: "bd4392a2-9c70-4f3a-9050-ae6a6db01082") : configmap "swift-ring-files" not found Dec 03 12:41:58 crc kubenswrapper[5052]: W1203 12:41:58.016271 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5f011b7b_96e7_4236_a0fc_b7e292e232fd.slice/crio-1bae80e92a7e98c1c2923da5ae1abb73cc845112fbc97a4fba384472d798d8a1 WatchSource:0}: Error finding container 1bae80e92a7e98c1c2923da5ae1abb73cc845112fbc97a4fba384472d798d8a1: Status 404 returned error can't find the container with id 1bae80e92a7e98c1c2923da5ae1abb73cc845112fbc97a4fba384472d798d8a1 Dec 03 12:41:58 crc kubenswrapper[5052]: I1203 12:41:58.021488 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-db-create-mfcd6"] Dec 03 12:41:58 crc kubenswrapper[5052]: I1203 12:41:58.205119 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-d4a6-account-create-update-8qdvr"] Dec 03 12:41:58 crc kubenswrapper[5052]: I1203 12:41:58.249897 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-db-create-mfcd6" event={"ID":"5f011b7b-96e7-4236-a0fc-b7e292e232fd","Type":"ContainerStarted","Data":"1bae80e92a7e98c1c2923da5ae1abb73cc845112fbc97a4fba384472d798d8a1"} Dec 03 12:41:59 crc kubenswrapper[5052]: I1203 12:41:59.260095 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-d4a6-account-create-update-8qdvr" event={"ID":"861301e3-bb06-498b-9225-b72962e175bc","Type":"ContainerStarted","Data":"e88eca179985f518ccd057e45f232aca0ad9b2db7b2c59e4e34612eeb6405a5a"} Dec 03 12:41:59 crc kubenswrapper[5052]: I1203 12:41:59.260441 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-d4a6-account-create-update-8qdvr" event={"ID":"861301e3-bb06-498b-9225-b72962e175bc","Type":"ContainerStarted","Data":"bb82d9fafc6af3d71b0b87815c2e5b6d108df18d4b25c25fde68f8187b990c82"} Dec 03 12:41:59 crc kubenswrapper[5052]: I1203 12:41:59.262041 5052 generic.go:334] "Generic (PLEG): container finished" podID="5f011b7b-96e7-4236-a0fc-b7e292e232fd" containerID="1e18bfb707a7b7ab93495e154e2c9ee1ca6a50570f41008061ee3a82fb2bf49f" exitCode=0 Dec 03 12:41:59 crc kubenswrapper[5052]: I1203 12:41:59.262349 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-db-create-mfcd6" event={"ID":"5f011b7b-96e7-4236-a0fc-b7e292e232fd","Type":"ContainerDied","Data":"1e18bfb707a7b7ab93495e154e2c9ee1ca6a50570f41008061ee3a82fb2bf49f"} Dec 03 12:41:59 crc kubenswrapper[5052]: I1203 12:41:59.293907 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/glance-d4a6-account-create-update-8qdvr" podStartSLOduration=2.29388898 podStartE2EDuration="2.29388898s" podCreationTimestamp="2025-12-03 12:41:57 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:41:59.276487633 +0000 UTC m=+4909.864729397" watchObservedRunningTime="2025-12-03 12:41:59.29388898 +0000 UTC m=+4909.882130744" Dec 03 12:42:00 crc kubenswrapper[5052]: I1203 12:42:00.272154 5052 generic.go:334] "Generic (PLEG): container finished" podID="861301e3-bb06-498b-9225-b72962e175bc" containerID="e88eca179985f518ccd057e45f232aca0ad9b2db7b2c59e4e34612eeb6405a5a" exitCode=0 Dec 03 12:42:00 crc kubenswrapper[5052]: I1203 12:42:00.272224 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-d4a6-account-create-update-8qdvr" event={"ID":"861301e3-bb06-498b-9225-b72962e175bc","Type":"ContainerDied","Data":"e88eca179985f518ccd057e45f232aca0ad9b2db7b2c59e4e34612eeb6405a5a"} Dec 03 12:42:00 crc kubenswrapper[5052]: I1203 12:42:00.637496 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-db-create-mfcd6" Dec 03 12:42:00 crc kubenswrapper[5052]: I1203 12:42:00.717584 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4z9cv\" (UniqueName: \"kubernetes.io/projected/5f011b7b-96e7-4236-a0fc-b7e292e232fd-kube-api-access-4z9cv\") pod \"5f011b7b-96e7-4236-a0fc-b7e292e232fd\" (UID: \"5f011b7b-96e7-4236-a0fc-b7e292e232fd\") " Dec 03 12:42:00 crc kubenswrapper[5052]: I1203 12:42:00.717732 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/5f011b7b-96e7-4236-a0fc-b7e292e232fd-operator-scripts\") pod \"5f011b7b-96e7-4236-a0fc-b7e292e232fd\" (UID: \"5f011b7b-96e7-4236-a0fc-b7e292e232fd\") " Dec 03 12:42:00 crc kubenswrapper[5052]: I1203 12:42:00.718614 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5f011b7b-96e7-4236-a0fc-b7e292e232fd-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "5f011b7b-96e7-4236-a0fc-b7e292e232fd" (UID: "5f011b7b-96e7-4236-a0fc-b7e292e232fd"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:42:00 crc kubenswrapper[5052]: I1203 12:42:00.723310 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5f011b7b-96e7-4236-a0fc-b7e292e232fd-kube-api-access-4z9cv" (OuterVolumeSpecName: "kube-api-access-4z9cv") pod "5f011b7b-96e7-4236-a0fc-b7e292e232fd" (UID: "5f011b7b-96e7-4236-a0fc-b7e292e232fd"). InnerVolumeSpecName "kube-api-access-4z9cv". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:42:00 crc kubenswrapper[5052]: I1203 12:42:00.820096 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/5f011b7b-96e7-4236-a0fc-b7e292e232fd-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:42:00 crc kubenswrapper[5052]: I1203 12:42:00.820131 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-4z9cv\" (UniqueName: \"kubernetes.io/projected/5f011b7b-96e7-4236-a0fc-b7e292e232fd-kube-api-access-4z9cv\") on node \"crc\" DevicePath \"\"" Dec 03 12:42:01 crc kubenswrapper[5052]: I1203 12:42:01.283360 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-db-create-mfcd6" Dec 03 12:42:01 crc kubenswrapper[5052]: I1203 12:42:01.283345 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-db-create-mfcd6" event={"ID":"5f011b7b-96e7-4236-a0fc-b7e292e232fd","Type":"ContainerDied","Data":"1bae80e92a7e98c1c2923da5ae1abb73cc845112fbc97a4fba384472d798d8a1"} Dec 03 12:42:01 crc kubenswrapper[5052]: I1203 12:42:01.283873 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1bae80e92a7e98c1c2923da5ae1abb73cc845112fbc97a4fba384472d798d8a1" Dec 03 12:42:01 crc kubenswrapper[5052]: I1203 12:42:01.667478 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-d4a6-account-create-update-8qdvr" Dec 03 12:42:01 crc kubenswrapper[5052]: I1203 12:42:01.735272 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/861301e3-bb06-498b-9225-b72962e175bc-operator-scripts\") pod \"861301e3-bb06-498b-9225-b72962e175bc\" (UID: \"861301e3-bb06-498b-9225-b72962e175bc\") " Dec 03 12:42:01 crc kubenswrapper[5052]: I1203 12:42:01.735534 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gdlxq\" (UniqueName: \"kubernetes.io/projected/861301e3-bb06-498b-9225-b72962e175bc-kube-api-access-gdlxq\") pod \"861301e3-bb06-498b-9225-b72962e175bc\" (UID: \"861301e3-bb06-498b-9225-b72962e175bc\") " Dec 03 12:42:01 crc kubenswrapper[5052]: I1203 12:42:01.735879 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/861301e3-bb06-498b-9225-b72962e175bc-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "861301e3-bb06-498b-9225-b72962e175bc" (UID: "861301e3-bb06-498b-9225-b72962e175bc"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:42:01 crc kubenswrapper[5052]: I1203 12:42:01.741698 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/861301e3-bb06-498b-9225-b72962e175bc-kube-api-access-gdlxq" (OuterVolumeSpecName: "kube-api-access-gdlxq") pod "861301e3-bb06-498b-9225-b72962e175bc" (UID: "861301e3-bb06-498b-9225-b72962e175bc"). InnerVolumeSpecName "kube-api-access-gdlxq". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:42:01 crc kubenswrapper[5052]: I1203 12:42:01.836962 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-gdlxq\" (UniqueName: \"kubernetes.io/projected/861301e3-bb06-498b-9225-b72962e175bc-kube-api-access-gdlxq\") on node \"crc\" DevicePath \"\"" Dec 03 12:42:01 crc kubenswrapper[5052]: I1203 12:42:01.837000 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/861301e3-bb06-498b-9225-b72962e175bc-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:42:02 crc kubenswrapper[5052]: I1203 12:42:02.040278 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/bd4392a2-9c70-4f3a-9050-ae6a6db01082-etc-swift\") pod \"swift-storage-0\" (UID: \"bd4392a2-9c70-4f3a-9050-ae6a6db01082\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:42:02 crc kubenswrapper[5052]: E1203 12:42:02.040441 5052 projected.go:288] Couldn't get configMap openstack-kuttl-tests/swift-ring-files: configmap "swift-ring-files" not found Dec 03 12:42:02 crc kubenswrapper[5052]: E1203 12:42:02.040468 5052 projected.go:194] Error preparing data for projected volume etc-swift for pod openstack-kuttl-tests/swift-storage-0: configmap "swift-ring-files" not found Dec 03 12:42:02 crc kubenswrapper[5052]: E1203 12:42:02.040604 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bd4392a2-9c70-4f3a-9050-ae6a6db01082-etc-swift podName:bd4392a2-9c70-4f3a-9050-ae6a6db01082 nodeName:}" failed. No retries permitted until 2025-12-03 12:42:10.040582151 +0000 UTC m=+4920.628823915 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "etc-swift" (UniqueName: "kubernetes.io/projected/bd4392a2-9c70-4f3a-9050-ae6a6db01082-etc-swift") pod "swift-storage-0" (UID: "bd4392a2-9c70-4f3a-9050-ae6a6db01082") : configmap "swift-ring-files" not found Dec 03 12:42:02 crc kubenswrapper[5052]: I1203 12:42:02.292985 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-d4a6-account-create-update-8qdvr" event={"ID":"861301e3-bb06-498b-9225-b72962e175bc","Type":"ContainerDied","Data":"bb82d9fafc6af3d71b0b87815c2e5b6d108df18d4b25c25fde68f8187b990c82"} Dec 03 12:42:02 crc kubenswrapper[5052]: I1203 12:42:02.293296 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bb82d9fafc6af3d71b0b87815c2e5b6d108df18d4b25c25fde68f8187b990c82" Dec 03 12:42:02 crc kubenswrapper[5052]: I1203 12:42:02.293036 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-d4a6-account-create-update-8qdvr" Dec 03 12:42:03 crc kubenswrapper[5052]: I1203 12:42:03.304191 5052 generic.go:334] "Generic (PLEG): container finished" podID="8d5c589d-50a2-4bdb-ac78-59c1592c03e8" containerID="5d97dda03f4bb7760eec9dbecdc78fddde19c90f53bee737bf5ab871d649c4ae" exitCode=0 Dec 03 12:42:03 crc kubenswrapper[5052]: I1203 12:42:03.304258 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-ring-rebalance-kvhc2" event={"ID":"8d5c589d-50a2-4bdb-ac78-59c1592c03e8","Type":"ContainerDied","Data":"5d97dda03f4bb7760eec9dbecdc78fddde19c90f53bee737bf5ab871d649c4ae"} Dec 03 12:42:04 crc kubenswrapper[5052]: I1203 12:42:04.690321 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-ring-rebalance-kvhc2" Dec 03 12:42:04 crc kubenswrapper[5052]: I1203 12:42:04.794133 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ring-data-devices\" (UniqueName: \"kubernetes.io/configmap/8d5c589d-50a2-4bdb-ac78-59c1592c03e8-ring-data-devices\") pod \"8d5c589d-50a2-4bdb-ac78-59c1592c03e8\" (UID: \"8d5c589d-50a2-4bdb-ac78-59c1592c03e8\") " Dec 03 12:42:04 crc kubenswrapper[5052]: I1203 12:42:04.794226 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wztzb\" (UniqueName: \"kubernetes.io/projected/8d5c589d-50a2-4bdb-ac78-59c1592c03e8-kube-api-access-wztzb\") pod \"8d5c589d-50a2-4bdb-ac78-59c1592c03e8\" (UID: \"8d5c589d-50a2-4bdb-ac78-59c1592c03e8\") " Dec 03 12:42:04 crc kubenswrapper[5052]: I1203 12:42:04.794250 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/8d5c589d-50a2-4bdb-ac78-59c1592c03e8-scripts\") pod \"8d5c589d-50a2-4bdb-ac78-59c1592c03e8\" (UID: \"8d5c589d-50a2-4bdb-ac78-59c1592c03e8\") " Dec 03 12:42:04 crc kubenswrapper[5052]: I1203 12:42:04.794322 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"dispersionconf\" (UniqueName: \"kubernetes.io/secret/8d5c589d-50a2-4bdb-ac78-59c1592c03e8-dispersionconf\") pod \"8d5c589d-50a2-4bdb-ac78-59c1592c03e8\" (UID: \"8d5c589d-50a2-4bdb-ac78-59c1592c03e8\") " Dec 03 12:42:04 crc kubenswrapper[5052]: I1203 12:42:04.794367 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"swiftconf\" (UniqueName: \"kubernetes.io/secret/8d5c589d-50a2-4bdb-ac78-59c1592c03e8-swiftconf\") pod \"8d5c589d-50a2-4bdb-ac78-59c1592c03e8\" (UID: \"8d5c589d-50a2-4bdb-ac78-59c1592c03e8\") " Dec 03 12:42:04 crc kubenswrapper[5052]: I1203 12:42:04.794441 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/empty-dir/8d5c589d-50a2-4bdb-ac78-59c1592c03e8-etc-swift\") pod \"8d5c589d-50a2-4bdb-ac78-59c1592c03e8\" (UID: \"8d5c589d-50a2-4bdb-ac78-59c1592c03e8\") " Dec 03 12:42:04 crc kubenswrapper[5052]: I1203 12:42:04.794529 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8d5c589d-50a2-4bdb-ac78-59c1592c03e8-combined-ca-bundle\") pod \"8d5c589d-50a2-4bdb-ac78-59c1592c03e8\" (UID: \"8d5c589d-50a2-4bdb-ac78-59c1592c03e8\") " Dec 03 12:42:04 crc kubenswrapper[5052]: I1203 12:42:04.794820 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8d5c589d-50a2-4bdb-ac78-59c1592c03e8-ring-data-devices" (OuterVolumeSpecName: "ring-data-devices") pod "8d5c589d-50a2-4bdb-ac78-59c1592c03e8" (UID: "8d5c589d-50a2-4bdb-ac78-59c1592c03e8"). InnerVolumeSpecName "ring-data-devices". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:42:04 crc kubenswrapper[5052]: I1203 12:42:04.794957 5052 reconciler_common.go:293] "Volume detached for volume \"ring-data-devices\" (UniqueName: \"kubernetes.io/configmap/8d5c589d-50a2-4bdb-ac78-59c1592c03e8-ring-data-devices\") on node \"crc\" DevicePath \"\"" Dec 03 12:42:04 crc kubenswrapper[5052]: I1203 12:42:04.795242 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8d5c589d-50a2-4bdb-ac78-59c1592c03e8-etc-swift" (OuterVolumeSpecName: "etc-swift") pod "8d5c589d-50a2-4bdb-ac78-59c1592c03e8" (UID: "8d5c589d-50a2-4bdb-ac78-59c1592c03e8"). InnerVolumeSpecName "etc-swift". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:42:04 crc kubenswrapper[5052]: I1203 12:42:04.802750 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8d5c589d-50a2-4bdb-ac78-59c1592c03e8-kube-api-access-wztzb" (OuterVolumeSpecName: "kube-api-access-wztzb") pod "8d5c589d-50a2-4bdb-ac78-59c1592c03e8" (UID: "8d5c589d-50a2-4bdb-ac78-59c1592c03e8"). InnerVolumeSpecName "kube-api-access-wztzb". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:42:04 crc kubenswrapper[5052]: I1203 12:42:04.811429 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8d5c589d-50a2-4bdb-ac78-59c1592c03e8-dispersionconf" (OuterVolumeSpecName: "dispersionconf") pod "8d5c589d-50a2-4bdb-ac78-59c1592c03e8" (UID: "8d5c589d-50a2-4bdb-ac78-59c1592c03e8"). InnerVolumeSpecName "dispersionconf". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:42:04 crc kubenswrapper[5052]: I1203 12:42:04.839637 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8d5c589d-50a2-4bdb-ac78-59c1592c03e8-scripts" (OuterVolumeSpecName: "scripts") pod "8d5c589d-50a2-4bdb-ac78-59c1592c03e8" (UID: "8d5c589d-50a2-4bdb-ac78-59c1592c03e8"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:42:04 crc kubenswrapper[5052]: I1203 12:42:04.843343 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8d5c589d-50a2-4bdb-ac78-59c1592c03e8-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "8d5c589d-50a2-4bdb-ac78-59c1592c03e8" (UID: "8d5c589d-50a2-4bdb-ac78-59c1592c03e8"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:42:04 crc kubenswrapper[5052]: I1203 12:42:04.847702 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8d5c589d-50a2-4bdb-ac78-59c1592c03e8-swiftconf" (OuterVolumeSpecName: "swiftconf") pod "8d5c589d-50a2-4bdb-ac78-59c1592c03e8" (UID: "8d5c589d-50a2-4bdb-ac78-59c1592c03e8"). InnerVolumeSpecName "swiftconf". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:42:04 crc kubenswrapper[5052]: I1203 12:42:04.896246 5052 reconciler_common.go:293] "Volume detached for volume \"etc-swift\" (UniqueName: \"kubernetes.io/empty-dir/8d5c589d-50a2-4bdb-ac78-59c1592c03e8-etc-swift\") on node \"crc\" DevicePath \"\"" Dec 03 12:42:04 crc kubenswrapper[5052]: I1203 12:42:04.896300 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8d5c589d-50a2-4bdb-ac78-59c1592c03e8-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:42:04 crc kubenswrapper[5052]: I1203 12:42:04.896319 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/8d5c589d-50a2-4bdb-ac78-59c1592c03e8-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:42:04 crc kubenswrapper[5052]: I1203 12:42:04.896339 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-wztzb\" (UniqueName: \"kubernetes.io/projected/8d5c589d-50a2-4bdb-ac78-59c1592c03e8-kube-api-access-wztzb\") on node \"crc\" DevicePath \"\"" Dec 03 12:42:04 crc kubenswrapper[5052]: I1203 12:42:04.896359 5052 reconciler_common.go:293] "Volume detached for volume \"dispersionconf\" (UniqueName: \"kubernetes.io/secret/8d5c589d-50a2-4bdb-ac78-59c1592c03e8-dispersionconf\") on node \"crc\" DevicePath \"\"" Dec 03 12:42:04 crc kubenswrapper[5052]: I1203 12:42:04.896375 5052 reconciler_common.go:293] "Volume detached for volume \"swiftconf\" (UniqueName: \"kubernetes.io/secret/8d5c589d-50a2-4bdb-ac78-59c1592c03e8-swiftconf\") on node \"crc\" DevicePath \"\"" Dec 03 12:42:05 crc kubenswrapper[5052]: I1203 12:42:05.331230 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-ring-rebalance-kvhc2" event={"ID":"8d5c589d-50a2-4bdb-ac78-59c1592c03e8","Type":"ContainerDied","Data":"b3b5886b15c04a5016f83c6bcfa39a563e18111d10444620e7140a650a766205"} Dec 03 12:42:05 crc kubenswrapper[5052]: I1203 12:42:05.331619 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b3b5886b15c04a5016f83c6bcfa39a563e18111d10444620e7140a650a766205" Dec 03 12:42:05 crc kubenswrapper[5052]: I1203 12:42:05.331470 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-ring-rebalance-kvhc2" Dec 03 12:42:07 crc kubenswrapper[5052]: I1203 12:42:07.495120 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/glance-db-sync-pp7qs"] Dec 03 12:42:07 crc kubenswrapper[5052]: E1203 12:42:07.495546 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="861301e3-bb06-498b-9225-b72962e175bc" containerName="mariadb-account-create-update" Dec 03 12:42:07 crc kubenswrapper[5052]: I1203 12:42:07.495562 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="861301e3-bb06-498b-9225-b72962e175bc" containerName="mariadb-account-create-update" Dec 03 12:42:07 crc kubenswrapper[5052]: E1203 12:42:07.495584 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="8d5c589d-50a2-4bdb-ac78-59c1592c03e8" containerName="swift-ring-rebalance" Dec 03 12:42:07 crc kubenswrapper[5052]: I1203 12:42:07.495593 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="8d5c589d-50a2-4bdb-ac78-59c1592c03e8" containerName="swift-ring-rebalance" Dec 03 12:42:07 crc kubenswrapper[5052]: E1203 12:42:07.495615 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="5f011b7b-96e7-4236-a0fc-b7e292e232fd" containerName="mariadb-database-create" Dec 03 12:42:07 crc kubenswrapper[5052]: I1203 12:42:07.495623 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f011b7b-96e7-4236-a0fc-b7e292e232fd" containerName="mariadb-database-create" Dec 03 12:42:07 crc kubenswrapper[5052]: I1203 12:42:07.495842 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="8d5c589d-50a2-4bdb-ac78-59c1592c03e8" containerName="swift-ring-rebalance" Dec 03 12:42:07 crc kubenswrapper[5052]: I1203 12:42:07.495867 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="5f011b7b-96e7-4236-a0fc-b7e292e232fd" containerName="mariadb-database-create" Dec 03 12:42:07 crc kubenswrapper[5052]: I1203 12:42:07.495880 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="861301e3-bb06-498b-9225-b72962e175bc" containerName="mariadb-account-create-update" Dec 03 12:42:07 crc kubenswrapper[5052]: I1203 12:42:07.496594 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-db-sync-pp7qs" Dec 03 12:42:07 crc kubenswrapper[5052]: I1203 12:42:07.500468 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-glance-dockercfg-ggt4k" Dec 03 12:42:07 crc kubenswrapper[5052]: I1203 12:42:07.501457 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-config-data" Dec 03 12:42:07 crc kubenswrapper[5052]: I1203 12:42:07.568616 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-db-sync-pp7qs"] Dec 03 12:42:07 crc kubenswrapper[5052]: I1203 12:42:07.575013 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6a6cd849-4cae-4f8f-8a79-1ebf7fb284ee-config-data\") pod \"glance-db-sync-pp7qs\" (UID: \"6a6cd849-4cae-4f8f-8a79-1ebf7fb284ee\") " pod="openstack-kuttl-tests/glance-db-sync-pp7qs" Dec 03 12:42:07 crc kubenswrapper[5052]: I1203 12:42:07.575109 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6a6cd849-4cae-4f8f-8a79-1ebf7fb284ee-combined-ca-bundle\") pod \"glance-db-sync-pp7qs\" (UID: \"6a6cd849-4cae-4f8f-8a79-1ebf7fb284ee\") " pod="openstack-kuttl-tests/glance-db-sync-pp7qs" Dec 03 12:42:07 crc kubenswrapper[5052]: I1203 12:42:07.575317 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/6a6cd849-4cae-4f8f-8a79-1ebf7fb284ee-db-sync-config-data\") pod \"glance-db-sync-pp7qs\" (UID: \"6a6cd849-4cae-4f8f-8a79-1ebf7fb284ee\") " pod="openstack-kuttl-tests/glance-db-sync-pp7qs" Dec 03 12:42:07 crc kubenswrapper[5052]: I1203 12:42:07.575430 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z9twq\" (UniqueName: \"kubernetes.io/projected/6a6cd849-4cae-4f8f-8a79-1ebf7fb284ee-kube-api-access-z9twq\") pod \"glance-db-sync-pp7qs\" (UID: \"6a6cd849-4cae-4f8f-8a79-1ebf7fb284ee\") " pod="openstack-kuttl-tests/glance-db-sync-pp7qs" Dec 03 12:42:07 crc kubenswrapper[5052]: I1203 12:42:07.676935 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6a6cd849-4cae-4f8f-8a79-1ebf7fb284ee-combined-ca-bundle\") pod \"glance-db-sync-pp7qs\" (UID: \"6a6cd849-4cae-4f8f-8a79-1ebf7fb284ee\") " pod="openstack-kuttl-tests/glance-db-sync-pp7qs" Dec 03 12:42:07 crc kubenswrapper[5052]: I1203 12:42:07.677095 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/6a6cd849-4cae-4f8f-8a79-1ebf7fb284ee-db-sync-config-data\") pod \"glance-db-sync-pp7qs\" (UID: \"6a6cd849-4cae-4f8f-8a79-1ebf7fb284ee\") " pod="openstack-kuttl-tests/glance-db-sync-pp7qs" Dec 03 12:42:07 crc kubenswrapper[5052]: I1203 12:42:07.677169 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-z9twq\" (UniqueName: \"kubernetes.io/projected/6a6cd849-4cae-4f8f-8a79-1ebf7fb284ee-kube-api-access-z9twq\") pod \"glance-db-sync-pp7qs\" (UID: \"6a6cd849-4cae-4f8f-8a79-1ebf7fb284ee\") " pod="openstack-kuttl-tests/glance-db-sync-pp7qs" Dec 03 12:42:07 crc kubenswrapper[5052]: I1203 12:42:07.677191 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6a6cd849-4cae-4f8f-8a79-1ebf7fb284ee-config-data\") pod \"glance-db-sync-pp7qs\" (UID: \"6a6cd849-4cae-4f8f-8a79-1ebf7fb284ee\") " pod="openstack-kuttl-tests/glance-db-sync-pp7qs" Dec 03 12:42:07 crc kubenswrapper[5052]: I1203 12:42:07.724237 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6a6cd849-4cae-4f8f-8a79-1ebf7fb284ee-combined-ca-bundle\") pod \"glance-db-sync-pp7qs\" (UID: \"6a6cd849-4cae-4f8f-8a79-1ebf7fb284ee\") " pod="openstack-kuttl-tests/glance-db-sync-pp7qs" Dec 03 12:42:07 crc kubenswrapper[5052]: I1203 12:42:07.726278 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6a6cd849-4cae-4f8f-8a79-1ebf7fb284ee-config-data\") pod \"glance-db-sync-pp7qs\" (UID: \"6a6cd849-4cae-4f8f-8a79-1ebf7fb284ee\") " pod="openstack-kuttl-tests/glance-db-sync-pp7qs" Dec 03 12:42:07 crc kubenswrapper[5052]: I1203 12:42:07.726576 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-z9twq\" (UniqueName: \"kubernetes.io/projected/6a6cd849-4cae-4f8f-8a79-1ebf7fb284ee-kube-api-access-z9twq\") pod \"glance-db-sync-pp7qs\" (UID: \"6a6cd849-4cae-4f8f-8a79-1ebf7fb284ee\") " pod="openstack-kuttl-tests/glance-db-sync-pp7qs" Dec 03 12:42:07 crc kubenswrapper[5052]: I1203 12:42:07.726935 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/6a6cd849-4cae-4f8f-8a79-1ebf7fb284ee-db-sync-config-data\") pod \"glance-db-sync-pp7qs\" (UID: \"6a6cd849-4cae-4f8f-8a79-1ebf7fb284ee\") " pod="openstack-kuttl-tests/glance-db-sync-pp7qs" Dec 03 12:42:07 crc kubenswrapper[5052]: I1203 12:42:07.996841 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-db-sync-pp7qs" Dec 03 12:42:08 crc kubenswrapper[5052]: I1203 12:42:08.458859 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-db-sync-pp7qs"] Dec 03 12:42:09 crc kubenswrapper[5052]: I1203 12:42:09.371207 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-db-sync-pp7qs" event={"ID":"6a6cd849-4cae-4f8f-8a79-1ebf7fb284ee","Type":"ContainerStarted","Data":"29577d728aa7f4df45f66cf1e60698a7460652a0971153a774ed4bab7555b7e7"} Dec 03 12:42:09 crc kubenswrapper[5052]: I1203 12:42:09.371617 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-db-sync-pp7qs" event={"ID":"6a6cd849-4cae-4f8f-8a79-1ebf7fb284ee","Type":"ContainerStarted","Data":"fbe3702cc724455a9b18458ebb2d58c05e3b579b4f71a9f1bed0309fa201b0ab"} Dec 03 12:42:09 crc kubenswrapper[5052]: I1203 12:42:09.389761 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/glance-db-sync-pp7qs" podStartSLOduration=2.389741264 podStartE2EDuration="2.389741264s" podCreationTimestamp="2025-12-03 12:42:07 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:42:09.385025109 +0000 UTC m=+4919.973266873" watchObservedRunningTime="2025-12-03 12:42:09.389741264 +0000 UTC m=+4919.977983028" Dec 03 12:42:10 crc kubenswrapper[5052]: I1203 12:42:10.115377 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/bd4392a2-9c70-4f3a-9050-ae6a6db01082-etc-swift\") pod \"swift-storage-0\" (UID: \"bd4392a2-9c70-4f3a-9050-ae6a6db01082\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:42:10 crc kubenswrapper[5052]: I1203 12:42:10.134990 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/bd4392a2-9c70-4f3a-9050-ae6a6db01082-etc-swift\") pod \"swift-storage-0\" (UID: \"bd4392a2-9c70-4f3a-9050-ae6a6db01082\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:42:10 crc kubenswrapper[5052]: I1203 12:42:10.326626 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:42:10 crc kubenswrapper[5052]: I1203 12:42:10.841955 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/swift-storage-0"] Dec 03 12:42:10 crc kubenswrapper[5052]: W1203 12:42:10.847758 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbd4392a2_9c70_4f3a_9050_ae6a6db01082.slice/crio-e445f81374349289566e935c5fcf09fc81e82d45facdd44b73e17b255cbb20c6 WatchSource:0}: Error finding container e445f81374349289566e935c5fcf09fc81e82d45facdd44b73e17b255cbb20c6: Status 404 returned error can't find the container with id e445f81374349289566e935c5fcf09fc81e82d45facdd44b73e17b255cbb20c6 Dec 03 12:42:10 crc kubenswrapper[5052]: I1203 12:42:10.941874 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:42:11 crc kubenswrapper[5052]: I1203 12:42:11.391991 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"bd4392a2-9c70-4f3a-9050-ae6a6db01082","Type":"ContainerStarted","Data":"8ea80229aee1ce207c0394efd61dfb519308fde6a2a25c517795af19f4c72da4"} Dec 03 12:42:11 crc kubenswrapper[5052]: I1203 12:42:11.392048 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"bd4392a2-9c70-4f3a-9050-ae6a6db01082","Type":"ContainerStarted","Data":"ee68253d5346b57e39fc0b0fed73d3c548e33dc2c36cd2673c5add72f235bb79"} Dec 03 12:42:11 crc kubenswrapper[5052]: I1203 12:42:11.392064 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"bd4392a2-9c70-4f3a-9050-ae6a6db01082","Type":"ContainerStarted","Data":"e445f81374349289566e935c5fcf09fc81e82d45facdd44b73e17b255cbb20c6"} Dec 03 12:42:12 crc kubenswrapper[5052]: I1203 12:42:12.403903 5052 generic.go:334] "Generic (PLEG): container finished" podID="0cb33a40-6f0a-4e89-a159-4edd3af3a2fa" containerID="40637a2936ebcaa4c4b798867fbd94260367dfcf9f69160a782dfc1f71402096" exitCode=0 Dec 03 12:42:12 crc kubenswrapper[5052]: I1203 12:42:12.404022 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-server-0" event={"ID":"0cb33a40-6f0a-4e89-a159-4edd3af3a2fa","Type":"ContainerDied","Data":"40637a2936ebcaa4c4b798867fbd94260367dfcf9f69160a782dfc1f71402096"} Dec 03 12:42:12 crc kubenswrapper[5052]: I1203 12:42:12.407373 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"bd4392a2-9c70-4f3a-9050-ae6a6db01082","Type":"ContainerStarted","Data":"2eaf7736f8bbe27fc3320dbe4500d1c1b7388bb8194a9d8a9bb7ef68f452b97f"} Dec 03 12:42:12 crc kubenswrapper[5052]: I1203 12:42:12.407421 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"bd4392a2-9c70-4f3a-9050-ae6a6db01082","Type":"ContainerStarted","Data":"409983b9eafc0681b072e5508199566f3fbeaf0cc55e055beaee8282045a003e"} Dec 03 12:42:12 crc kubenswrapper[5052]: I1203 12:42:12.407430 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"bd4392a2-9c70-4f3a-9050-ae6a6db01082","Type":"ContainerStarted","Data":"e4afa7a612ce1826fda8750703ccc15d8110e17d1d419c076a60957b33af8adb"} Dec 03 12:42:13 crc kubenswrapper[5052]: I1203 12:42:13.417788 5052 generic.go:334] "Generic (PLEG): container finished" podID="488c3529-4c2b-47c0-9dcf-05a064c34d0d" containerID="fd397a6821f16ad2b0e97b5bbbb387d00aed4ab2619cd9d9f06f9bc10900138f" exitCode=0 Dec 03 12:42:13 crc kubenswrapper[5052]: I1203 12:42:13.418015 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" event={"ID":"488c3529-4c2b-47c0-9dcf-05a064c34d0d","Type":"ContainerDied","Data":"fd397a6821f16ad2b0e97b5bbbb387d00aed4ab2619cd9d9f06f9bc10900138f"} Dec 03 12:42:13 crc kubenswrapper[5052]: I1203 12:42:13.425203 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-server-0" event={"ID":"0cb33a40-6f0a-4e89-a159-4edd3af3a2fa","Type":"ContainerStarted","Data":"e4371252da37cb77d701ae89da98daa7bb376b68ff8df323b2456a8b5876c5ba"} Dec 03 12:42:13 crc kubenswrapper[5052]: I1203 12:42:13.425647 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:42:13 crc kubenswrapper[5052]: I1203 12:42:13.429957 5052 generic.go:334] "Generic (PLEG): container finished" podID="6a6cd849-4cae-4f8f-8a79-1ebf7fb284ee" containerID="29577d728aa7f4df45f66cf1e60698a7460652a0971153a774ed4bab7555b7e7" exitCode=0 Dec 03 12:42:13 crc kubenswrapper[5052]: I1203 12:42:13.430033 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-db-sync-pp7qs" event={"ID":"6a6cd849-4cae-4f8f-8a79-1ebf7fb284ee","Type":"ContainerDied","Data":"29577d728aa7f4df45f66cf1e60698a7460652a0971153a774ed4bab7555b7e7"} Dec 03 12:42:13 crc kubenswrapper[5052]: I1203 12:42:13.443327 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"bd4392a2-9c70-4f3a-9050-ae6a6db01082","Type":"ContainerStarted","Data":"073f8906a127eaad4d2dcbdbaf2738671488e4b5da8001af8eb90b508703cf7a"} Dec 03 12:42:13 crc kubenswrapper[5052]: I1203 12:42:13.443629 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"bd4392a2-9c70-4f3a-9050-ae6a6db01082","Type":"ContainerStarted","Data":"0420f9807b79aa9a35f7e00db76c776194c37e61b2e6f2e9e3e83e3f355d270c"} Dec 03 12:42:13 crc kubenswrapper[5052]: I1203 12:42:13.443887 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"bd4392a2-9c70-4f3a-9050-ae6a6db01082","Type":"ContainerStarted","Data":"49448581e8cc46afbfef30edf584ba024f9fdfe5baf0de42a0abed3885f0ef23"} Dec 03 12:42:13 crc kubenswrapper[5052]: I1203 12:42:13.444727 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"bd4392a2-9c70-4f3a-9050-ae6a6db01082","Type":"ContainerStarted","Data":"4aa02cc6c01a66c5bcc1c0d61505c02dbf6374ff0a6550b9d548733640117138"} Dec 03 12:42:13 crc kubenswrapper[5052]: I1203 12:42:13.477188 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/rabbitmq-server-0" podStartSLOduration=37.477168277 podStartE2EDuration="37.477168277s" podCreationTimestamp="2025-12-03 12:41:36 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:42:13.475628453 +0000 UTC m=+4924.063870227" watchObservedRunningTime="2025-12-03 12:42:13.477168277 +0000 UTC m=+4924.065410041" Dec 03 12:42:14 crc kubenswrapper[5052]: I1203 12:42:14.458949 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"bd4392a2-9c70-4f3a-9050-ae6a6db01082","Type":"ContainerStarted","Data":"6eeb3694b4e9534ae61ec9fcf173cc77b70bcda76de435e9b1ede55531edf6ec"} Dec 03 12:42:14 crc kubenswrapper[5052]: I1203 12:42:14.459291 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"bd4392a2-9c70-4f3a-9050-ae6a6db01082","Type":"ContainerStarted","Data":"44d343c040a52a96a2507d69db579564483888cfc2ab0051211cae74e09f32f3"} Dec 03 12:42:14 crc kubenswrapper[5052]: I1203 12:42:14.459311 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"bd4392a2-9c70-4f3a-9050-ae6a6db01082","Type":"ContainerStarted","Data":"ef01830e5c368fdf7b4e0af6d27790eef08c24d4f1d55b39ccbab91a7ba97c10"} Dec 03 12:42:14 crc kubenswrapper[5052]: I1203 12:42:14.459324 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"bd4392a2-9c70-4f3a-9050-ae6a6db01082","Type":"ContainerStarted","Data":"4f5b00a3217544c70607eacf64edc7dd926767e810b84e309cbc071e09b652d5"} Dec 03 12:42:14 crc kubenswrapper[5052]: I1203 12:42:14.459336 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"bd4392a2-9c70-4f3a-9050-ae6a6db01082","Type":"ContainerStarted","Data":"d17825037af7c0e226677e493c307a512bb167e46667dd9c705ac03cff6758bb"} Dec 03 12:42:14 crc kubenswrapper[5052]: I1203 12:42:14.459346 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"bd4392a2-9c70-4f3a-9050-ae6a6db01082","Type":"ContainerStarted","Data":"257a3713d34f8297bc83998342be63e79d0964dc86ce021417f8cb3fc862ff41"} Dec 03 12:42:14 crc kubenswrapper[5052]: I1203 12:42:14.463172 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" event={"ID":"488c3529-4c2b-47c0-9dcf-05a064c34d0d","Type":"ContainerStarted","Data":"dfdff0317f61d3fa549a21a9c81b61a342d0bac7de867324e414cf592af4d97d"} Dec 03 12:42:14 crc kubenswrapper[5052]: I1203 12:42:14.463680 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:42:14 crc kubenswrapper[5052]: I1203 12:42:14.503771 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/swift-storage-0" podStartSLOduration=20.503753455000002 podStartE2EDuration="20.503753455s" podCreationTimestamp="2025-12-03 12:41:54 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:42:14.501668345 +0000 UTC m=+4925.089910109" watchObservedRunningTime="2025-12-03 12:42:14.503753455 +0000 UTC m=+4925.091995219" Dec 03 12:42:14 crc kubenswrapper[5052]: I1203 12:42:14.536013 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" podStartSLOduration=37.535992107 podStartE2EDuration="37.535992107s" podCreationTimestamp="2025-12-03 12:41:37 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:42:14.526904947 +0000 UTC m=+4925.115146711" watchObservedRunningTime="2025-12-03 12:42:14.535992107 +0000 UTC m=+4925.124233871" Dec 03 12:42:14 crc kubenswrapper[5052]: I1203 12:42:14.702726 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-8554887475-6pl7l"] Dec 03 12:42:14 crc kubenswrapper[5052]: I1203 12:42:14.706577 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-8554887475-6pl7l" Dec 03 12:42:14 crc kubenswrapper[5052]: I1203 12:42:14.712348 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"dns-swift-storage-0" Dec 03 12:42:14 crc kubenswrapper[5052]: I1203 12:42:14.737730 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-8554887475-6pl7l"] Dec 03 12:42:14 crc kubenswrapper[5052]: I1203 12:42:14.798186 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ee63b3ae-44c4-4225-8373-75f5950af9d5-config\") pod \"dnsmasq-dnsmasq-8554887475-6pl7l\" (UID: \"ee63b3ae-44c4-4225-8373-75f5950af9d5\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-8554887475-6pl7l" Dec 03 12:42:14 crc kubenswrapper[5052]: I1203 12:42:14.798271 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wwx7n\" (UniqueName: \"kubernetes.io/projected/ee63b3ae-44c4-4225-8373-75f5950af9d5-kube-api-access-wwx7n\") pod \"dnsmasq-dnsmasq-8554887475-6pl7l\" (UID: \"ee63b3ae-44c4-4225-8373-75f5950af9d5\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-8554887475-6pl7l" Dec 03 12:42:14 crc kubenswrapper[5052]: I1203 12:42:14.798323 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dns-swift-storage-0\" (UniqueName: \"kubernetes.io/configmap/ee63b3ae-44c4-4225-8373-75f5950af9d5-dns-swift-storage-0\") pod \"dnsmasq-dnsmasq-8554887475-6pl7l\" (UID: \"ee63b3ae-44c4-4225-8373-75f5950af9d5\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-8554887475-6pl7l" Dec 03 12:42:14 crc kubenswrapper[5052]: I1203 12:42:14.798374 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/ee63b3ae-44c4-4225-8373-75f5950af9d5-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-8554887475-6pl7l\" (UID: \"ee63b3ae-44c4-4225-8373-75f5950af9d5\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-8554887475-6pl7l" Dec 03 12:42:14 crc kubenswrapper[5052]: I1203 12:42:14.900183 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"dns-swift-storage-0\" (UniqueName: \"kubernetes.io/configmap/ee63b3ae-44c4-4225-8373-75f5950af9d5-dns-swift-storage-0\") pod \"dnsmasq-dnsmasq-8554887475-6pl7l\" (UID: \"ee63b3ae-44c4-4225-8373-75f5950af9d5\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-8554887475-6pl7l" Dec 03 12:42:14 crc kubenswrapper[5052]: I1203 12:42:14.900595 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/ee63b3ae-44c4-4225-8373-75f5950af9d5-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-8554887475-6pl7l\" (UID: \"ee63b3ae-44c4-4225-8373-75f5950af9d5\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-8554887475-6pl7l" Dec 03 12:42:14 crc kubenswrapper[5052]: I1203 12:42:14.900734 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ee63b3ae-44c4-4225-8373-75f5950af9d5-config\") pod \"dnsmasq-dnsmasq-8554887475-6pl7l\" (UID: \"ee63b3ae-44c4-4225-8373-75f5950af9d5\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-8554887475-6pl7l" Dec 03 12:42:14 crc kubenswrapper[5052]: I1203 12:42:14.900788 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-wwx7n\" (UniqueName: \"kubernetes.io/projected/ee63b3ae-44c4-4225-8373-75f5950af9d5-kube-api-access-wwx7n\") pod \"dnsmasq-dnsmasq-8554887475-6pl7l\" (UID: \"ee63b3ae-44c4-4225-8373-75f5950af9d5\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-8554887475-6pl7l" Dec 03 12:42:14 crc kubenswrapper[5052]: I1203 12:42:14.901090 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"dns-swift-storage-0\" (UniqueName: \"kubernetes.io/configmap/ee63b3ae-44c4-4225-8373-75f5950af9d5-dns-swift-storage-0\") pod \"dnsmasq-dnsmasq-8554887475-6pl7l\" (UID: \"ee63b3ae-44c4-4225-8373-75f5950af9d5\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-8554887475-6pl7l" Dec 03 12:42:14 crc kubenswrapper[5052]: I1203 12:42:14.901333 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/ee63b3ae-44c4-4225-8373-75f5950af9d5-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-8554887475-6pl7l\" (UID: \"ee63b3ae-44c4-4225-8373-75f5950af9d5\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-8554887475-6pl7l" Dec 03 12:42:14 crc kubenswrapper[5052]: I1203 12:42:14.901421 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ee63b3ae-44c4-4225-8373-75f5950af9d5-config\") pod \"dnsmasq-dnsmasq-8554887475-6pl7l\" (UID: \"ee63b3ae-44c4-4225-8373-75f5950af9d5\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-8554887475-6pl7l" Dec 03 12:42:14 crc kubenswrapper[5052]: I1203 12:42:14.922077 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-wwx7n\" (UniqueName: \"kubernetes.io/projected/ee63b3ae-44c4-4225-8373-75f5950af9d5-kube-api-access-wwx7n\") pod \"dnsmasq-dnsmasq-8554887475-6pl7l\" (UID: \"ee63b3ae-44c4-4225-8373-75f5950af9d5\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-8554887475-6pl7l" Dec 03 12:42:14 crc kubenswrapper[5052]: I1203 12:42:14.922697 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-db-sync-pp7qs" Dec 03 12:42:15 crc kubenswrapper[5052]: I1203 12:42:15.002321 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6a6cd849-4cae-4f8f-8a79-1ebf7fb284ee-config-data\") pod \"6a6cd849-4cae-4f8f-8a79-1ebf7fb284ee\" (UID: \"6a6cd849-4cae-4f8f-8a79-1ebf7fb284ee\") " Dec 03 12:42:15 crc kubenswrapper[5052]: I1203 12:42:15.002379 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z9twq\" (UniqueName: \"kubernetes.io/projected/6a6cd849-4cae-4f8f-8a79-1ebf7fb284ee-kube-api-access-z9twq\") pod \"6a6cd849-4cae-4f8f-8a79-1ebf7fb284ee\" (UID: \"6a6cd849-4cae-4f8f-8a79-1ebf7fb284ee\") " Dec 03 12:42:15 crc kubenswrapper[5052]: I1203 12:42:15.002542 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6a6cd849-4cae-4f8f-8a79-1ebf7fb284ee-combined-ca-bundle\") pod \"6a6cd849-4cae-4f8f-8a79-1ebf7fb284ee\" (UID: \"6a6cd849-4cae-4f8f-8a79-1ebf7fb284ee\") " Dec 03 12:42:15 crc kubenswrapper[5052]: I1203 12:42:15.002578 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/6a6cd849-4cae-4f8f-8a79-1ebf7fb284ee-db-sync-config-data\") pod \"6a6cd849-4cae-4f8f-8a79-1ebf7fb284ee\" (UID: \"6a6cd849-4cae-4f8f-8a79-1ebf7fb284ee\") " Dec 03 12:42:15 crc kubenswrapper[5052]: I1203 12:42:15.005947 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6a6cd849-4cae-4f8f-8a79-1ebf7fb284ee-db-sync-config-data" (OuterVolumeSpecName: "db-sync-config-data") pod "6a6cd849-4cae-4f8f-8a79-1ebf7fb284ee" (UID: "6a6cd849-4cae-4f8f-8a79-1ebf7fb284ee"). InnerVolumeSpecName "db-sync-config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:42:15 crc kubenswrapper[5052]: I1203 12:42:15.007054 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6a6cd849-4cae-4f8f-8a79-1ebf7fb284ee-kube-api-access-z9twq" (OuterVolumeSpecName: "kube-api-access-z9twq") pod "6a6cd849-4cae-4f8f-8a79-1ebf7fb284ee" (UID: "6a6cd849-4cae-4f8f-8a79-1ebf7fb284ee"). InnerVolumeSpecName "kube-api-access-z9twq". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:42:15 crc kubenswrapper[5052]: I1203 12:42:15.047406 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6a6cd849-4cae-4f8f-8a79-1ebf7fb284ee-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "6a6cd849-4cae-4f8f-8a79-1ebf7fb284ee" (UID: "6a6cd849-4cae-4f8f-8a79-1ebf7fb284ee"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:42:15 crc kubenswrapper[5052]: I1203 12:42:15.053148 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-8554887475-6pl7l" Dec 03 12:42:15 crc kubenswrapper[5052]: I1203 12:42:15.065779 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6a6cd849-4cae-4f8f-8a79-1ebf7fb284ee-config-data" (OuterVolumeSpecName: "config-data") pod "6a6cd849-4cae-4f8f-8a79-1ebf7fb284ee" (UID: "6a6cd849-4cae-4f8f-8a79-1ebf7fb284ee"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:42:15 crc kubenswrapper[5052]: I1203 12:42:15.104013 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6a6cd849-4cae-4f8f-8a79-1ebf7fb284ee-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:42:15 crc kubenswrapper[5052]: I1203 12:42:15.104348 5052 reconciler_common.go:293] "Volume detached for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/6a6cd849-4cae-4f8f-8a79-1ebf7fb284ee-db-sync-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:42:15 crc kubenswrapper[5052]: I1203 12:42:15.104445 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6a6cd849-4cae-4f8f-8a79-1ebf7fb284ee-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:42:15 crc kubenswrapper[5052]: I1203 12:42:15.104554 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-z9twq\" (UniqueName: \"kubernetes.io/projected/6a6cd849-4cae-4f8f-8a79-1ebf7fb284ee-kube-api-access-z9twq\") on node \"crc\" DevicePath \"\"" Dec 03 12:42:15 crc kubenswrapper[5052]: I1203 12:42:15.475623 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-db-sync-pp7qs" event={"ID":"6a6cd849-4cae-4f8f-8a79-1ebf7fb284ee","Type":"ContainerDied","Data":"fbe3702cc724455a9b18458ebb2d58c05e3b579b4f71a9f1bed0309fa201b0ab"} Dec 03 12:42:15 crc kubenswrapper[5052]: I1203 12:42:15.475966 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fbe3702cc724455a9b18458ebb2d58c05e3b579b4f71a9f1bed0309fa201b0ab" Dec 03 12:42:15 crc kubenswrapper[5052]: I1203 12:42:15.475669 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-db-sync-pp7qs" Dec 03 12:42:15 crc kubenswrapper[5052]: I1203 12:42:15.563917 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-8554887475-6pl7l"] Dec 03 12:42:16 crc kubenswrapper[5052]: I1203 12:42:16.486800 5052 generic.go:334] "Generic (PLEG): container finished" podID="ee63b3ae-44c4-4225-8373-75f5950af9d5" containerID="88f4f14061896d001d9cdb55644ba6c0aba013ab1758be59354f8602ded7f643" exitCode=0 Dec 03 12:42:16 crc kubenswrapper[5052]: I1203 12:42:16.486854 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-8554887475-6pl7l" event={"ID":"ee63b3ae-44c4-4225-8373-75f5950af9d5","Type":"ContainerDied","Data":"88f4f14061896d001d9cdb55644ba6c0aba013ab1758be59354f8602ded7f643"} Dec 03 12:42:16 crc kubenswrapper[5052]: I1203 12:42:16.486886 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-8554887475-6pl7l" event={"ID":"ee63b3ae-44c4-4225-8373-75f5950af9d5","Type":"ContainerStarted","Data":"d42b09e9571187e0cc5b0c0de500425a97201c0ae57f4351e072b6ecc809e171"} Dec 03 12:42:17 crc kubenswrapper[5052]: I1203 12:42:17.497948 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-8554887475-6pl7l" event={"ID":"ee63b3ae-44c4-4225-8373-75f5950af9d5","Type":"ContainerStarted","Data":"aad48c211b1075e99f24229e8da89586a407d1f73f3bdf7e13056ec73c5db24c"} Dec 03 12:42:17 crc kubenswrapper[5052]: I1203 12:42:17.499031 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-8554887475-6pl7l" Dec 03 12:42:25 crc kubenswrapper[5052]: I1203 12:42:25.054725 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-8554887475-6pl7l" Dec 03 12:42:25 crc kubenswrapper[5052]: I1203 12:42:25.080183 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-8554887475-6pl7l" podStartSLOduration=11.08012059 podStartE2EDuration="11.08012059s" podCreationTimestamp="2025-12-03 12:42:14 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:42:17.535868799 +0000 UTC m=+4928.124110563" watchObservedRunningTime="2025-12-03 12:42:25.08012059 +0000 UTC m=+4935.668362354" Dec 03 12:42:25 crc kubenswrapper[5052]: I1203 12:42:25.122940 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-8h8qz"] Dec 03 12:42:25 crc kubenswrapper[5052]: I1203 12:42:25.123169 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-8h8qz" podUID="296f8cec-ad7b-4b51-a9a9-934301d43d2e" containerName="dnsmasq-dns" containerID="cri-o://7931c8713f97dce1e306c981b449dd889f9caac3a2406f2863f294197a7435e2" gracePeriod=10 Dec 03 12:42:25 crc kubenswrapper[5052]: I1203 12:42:25.578716 5052 generic.go:334] "Generic (PLEG): container finished" podID="296f8cec-ad7b-4b51-a9a9-934301d43d2e" containerID="7931c8713f97dce1e306c981b449dd889f9caac3a2406f2863f294197a7435e2" exitCode=0 Dec 03 12:42:25 crc kubenswrapper[5052]: I1203 12:42:25.578803 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-8h8qz" event={"ID":"296f8cec-ad7b-4b51-a9a9-934301d43d2e","Type":"ContainerDied","Data":"7931c8713f97dce1e306c981b449dd889f9caac3a2406f2863f294197a7435e2"} Dec 03 12:42:25 crc kubenswrapper[5052]: I1203 12:42:25.578881 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-8h8qz" event={"ID":"296f8cec-ad7b-4b51-a9a9-934301d43d2e","Type":"ContainerDied","Data":"cc15d46c7ef59e21f31e8fd200ff54e6e4037db8acc39ec7a4e3a93f9175feeb"} Dec 03 12:42:25 crc kubenswrapper[5052]: I1203 12:42:25.578910 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cc15d46c7ef59e21f31e8fd200ff54e6e4037db8acc39ec7a4e3a93f9175feeb" Dec 03 12:42:25 crc kubenswrapper[5052]: I1203 12:42:25.585673 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-8h8qz" Dec 03 12:42:25 crc kubenswrapper[5052]: I1203 12:42:25.704596 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/296f8cec-ad7b-4b51-a9a9-934301d43d2e-config\") pod \"296f8cec-ad7b-4b51-a9a9-934301d43d2e\" (UID: \"296f8cec-ad7b-4b51-a9a9-934301d43d2e\") " Dec 03 12:42:25 crc kubenswrapper[5052]: I1203 12:42:25.704667 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dwdpg\" (UniqueName: \"kubernetes.io/projected/296f8cec-ad7b-4b51-a9a9-934301d43d2e-kube-api-access-dwdpg\") pod \"296f8cec-ad7b-4b51-a9a9-934301d43d2e\" (UID: \"296f8cec-ad7b-4b51-a9a9-934301d43d2e\") " Dec 03 12:42:25 crc kubenswrapper[5052]: I1203 12:42:25.704768 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/296f8cec-ad7b-4b51-a9a9-934301d43d2e-dnsmasq-svc\") pod \"296f8cec-ad7b-4b51-a9a9-934301d43d2e\" (UID: \"296f8cec-ad7b-4b51-a9a9-934301d43d2e\") " Dec 03 12:42:25 crc kubenswrapper[5052]: I1203 12:42:25.713194 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/296f8cec-ad7b-4b51-a9a9-934301d43d2e-kube-api-access-dwdpg" (OuterVolumeSpecName: "kube-api-access-dwdpg") pod "296f8cec-ad7b-4b51-a9a9-934301d43d2e" (UID: "296f8cec-ad7b-4b51-a9a9-934301d43d2e"). InnerVolumeSpecName "kube-api-access-dwdpg". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:42:25 crc kubenswrapper[5052]: I1203 12:42:25.751888 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/296f8cec-ad7b-4b51-a9a9-934301d43d2e-config" (OuterVolumeSpecName: "config") pod "296f8cec-ad7b-4b51-a9a9-934301d43d2e" (UID: "296f8cec-ad7b-4b51-a9a9-934301d43d2e"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:42:25 crc kubenswrapper[5052]: I1203 12:42:25.763257 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/296f8cec-ad7b-4b51-a9a9-934301d43d2e-dnsmasq-svc" (OuterVolumeSpecName: "dnsmasq-svc") pod "296f8cec-ad7b-4b51-a9a9-934301d43d2e" (UID: "296f8cec-ad7b-4b51-a9a9-934301d43d2e"). InnerVolumeSpecName "dnsmasq-svc". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:42:25 crc kubenswrapper[5052]: I1203 12:42:25.807220 5052 reconciler_common.go:293] "Volume detached for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/296f8cec-ad7b-4b51-a9a9-934301d43d2e-dnsmasq-svc\") on node \"crc\" DevicePath \"\"" Dec 03 12:42:25 crc kubenswrapper[5052]: I1203 12:42:25.807254 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/296f8cec-ad7b-4b51-a9a9-934301d43d2e-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:42:25 crc kubenswrapper[5052]: I1203 12:42:25.807268 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-dwdpg\" (UniqueName: \"kubernetes.io/projected/296f8cec-ad7b-4b51-a9a9-934301d43d2e-kube-api-access-dwdpg\") on node \"crc\" DevicePath \"\"" Dec 03 12:42:26 crc kubenswrapper[5052]: I1203 12:42:26.586458 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-8h8qz" Dec 03 12:42:26 crc kubenswrapper[5052]: I1203 12:42:26.619870 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-8h8qz"] Dec 03 12:42:26 crc kubenswrapper[5052]: I1203 12:42:26.626146 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-8h8qz"] Dec 03 12:42:27 crc kubenswrapper[5052]: I1203 12:42:27.444626 5052 scope.go:117] "RemoveContainer" containerID="1ba41230cf7215eee34d6d8ea8681f728e95087f0ac46229ec46ae822a73d0e3" Dec 03 12:42:27 crc kubenswrapper[5052]: I1203 12:42:27.468076 5052 scope.go:117] "RemoveContainer" containerID="c3d94fea1d9f1a0f6f228595edb362c6bd0e2d49784198212c905e4bc050a564" Dec 03 12:42:27 crc kubenswrapper[5052]: I1203 12:42:27.494566 5052 scope.go:117] "RemoveContainer" containerID="dbee67def34d5df95d62c1a89f2db75ed8290851a65f7a468f58b7a707d3ad20" Dec 03 12:42:27 crc kubenswrapper[5052]: I1203 12:42:27.523181 5052 scope.go:117] "RemoveContainer" containerID="ffc601b182d6f61d49cc3f4cc073d39e73625c22b23b4c46e4b1d092e911918c" Dec 03 12:42:27 crc kubenswrapper[5052]: I1203 12:42:27.561338 5052 scope.go:117] "RemoveContainer" containerID="999ebc2e5d21eb2c7377ec909cc77ed7c15f0032bb17311e3331c347b89b4ade" Dec 03 12:42:27 crc kubenswrapper[5052]: I1203 12:42:27.582658 5052 scope.go:117] "RemoveContainer" containerID="256155caf5dc190b0f04698e3cd7ad88f707a09926115e6b668bb753698f6ddf" Dec 03 12:42:27 crc kubenswrapper[5052]: I1203 12:42:27.605816 5052 scope.go:117] "RemoveContainer" containerID="00c3870e2cd22f1219f98f3fe1c8ae16c25295f882978dced379eafbfab9d78e" Dec 03 12:42:27 crc kubenswrapper[5052]: I1203 12:42:27.625777 5052 scope.go:117] "RemoveContainer" containerID="5debcb02a261adaa17b1df9a4bfe8eef2137d454c7652df878fe48809d10cad3" Dec 03 12:42:27 crc kubenswrapper[5052]: I1203 12:42:27.675715 5052 scope.go:117] "RemoveContainer" containerID="9330194596eda8836d7765112355fa440ad87fd5443eb2c1506c02b1767075eb" Dec 03 12:42:27 crc kubenswrapper[5052]: I1203 12:42:27.715778 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="296f8cec-ad7b-4b51-a9a9-934301d43d2e" path="/var/lib/kubelet/pods/296f8cec-ad7b-4b51-a9a9-934301d43d2e/volumes" Dec 03 12:42:27 crc kubenswrapper[5052]: I1203 12:42:27.797689 5052 scope.go:117] "RemoveContainer" containerID="24924091dd14c68a3a47c176dd1bea56c02b9f160a15ed0661a72c290a768b27" Dec 03 12:42:27 crc kubenswrapper[5052]: I1203 12:42:27.854699 5052 scope.go:117] "RemoveContainer" containerID="dd0ce7d71b0c277f456d230103929c6b85104381dfe61d5d51d3dbfbc10a484a" Dec 03 12:42:27 crc kubenswrapper[5052]: I1203 12:42:27.900073 5052 scope.go:117] "RemoveContainer" containerID="7cb1a81184710de145b19d3051c5761c627e9207bc129e10fab4d1acd4fc4174" Dec 03 12:42:27 crc kubenswrapper[5052]: I1203 12:42:27.940676 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:42:28 crc kubenswrapper[5052]: I1203 12:42:28.158563 5052 scope.go:117] "RemoveContainer" containerID="7f51aca2747cb14558df5b7a3e191c04da61ef6ed6f56e77fe911e095658488b" Dec 03 12:42:28 crc kubenswrapper[5052]: I1203 12:42:28.234949 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/cinder-db-create-jmgjk"] Dec 03 12:42:28 crc kubenswrapper[5052]: E1203 12:42:28.235703 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="6a6cd849-4cae-4f8f-8a79-1ebf7fb284ee" containerName="glance-db-sync" Dec 03 12:42:28 crc kubenswrapper[5052]: I1203 12:42:28.235727 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="6a6cd849-4cae-4f8f-8a79-1ebf7fb284ee" containerName="glance-db-sync" Dec 03 12:42:28 crc kubenswrapper[5052]: E1203 12:42:28.235749 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="296f8cec-ad7b-4b51-a9a9-934301d43d2e" containerName="init" Dec 03 12:42:28 crc kubenswrapper[5052]: I1203 12:42:28.235757 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="296f8cec-ad7b-4b51-a9a9-934301d43d2e" containerName="init" Dec 03 12:42:28 crc kubenswrapper[5052]: E1203 12:42:28.235772 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="296f8cec-ad7b-4b51-a9a9-934301d43d2e" containerName="dnsmasq-dns" Dec 03 12:42:28 crc kubenswrapper[5052]: I1203 12:42:28.235781 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="296f8cec-ad7b-4b51-a9a9-934301d43d2e" containerName="dnsmasq-dns" Dec 03 12:42:28 crc kubenswrapper[5052]: I1203 12:42:28.236008 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="6a6cd849-4cae-4f8f-8a79-1ebf7fb284ee" containerName="glance-db-sync" Dec 03 12:42:28 crc kubenswrapper[5052]: I1203 12:42:28.236036 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="296f8cec-ad7b-4b51-a9a9-934301d43d2e" containerName="dnsmasq-dns" Dec 03 12:42:28 crc kubenswrapper[5052]: I1203 12:42:28.236730 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-db-create-jmgjk" Dec 03 12:42:28 crc kubenswrapper[5052]: I1203 12:42:28.248786 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-db-create-jmgjk"] Dec 03 12:42:28 crc kubenswrapper[5052]: I1203 12:42:28.306449 5052 scope.go:117] "RemoveContainer" containerID="1729d01686e7da24ac8eadd64fe8d08a11e6f6f31264cea2c7dd0a578e797dfb" Dec 03 12:42:28 crc kubenswrapper[5052]: I1203 12:42:28.344688 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/barbican-db-create-5hb92"] Dec 03 12:42:28 crc kubenswrapper[5052]: I1203 12:42:28.346033 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-db-create-5hb92" Dec 03 12:42:28 crc kubenswrapper[5052]: I1203 12:42:28.347716 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/53834739-1c4b-4de1-a946-1a26d9036171-operator-scripts\") pod \"cinder-db-create-jmgjk\" (UID: \"53834739-1c4b-4de1-a946-1a26d9036171\") " pod="openstack-kuttl-tests/cinder-db-create-jmgjk" Dec 03 12:42:28 crc kubenswrapper[5052]: I1203 12:42:28.347875 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4bpmm\" (UniqueName: \"kubernetes.io/projected/53834739-1c4b-4de1-a946-1a26d9036171-kube-api-access-4bpmm\") pod \"cinder-db-create-jmgjk\" (UID: \"53834739-1c4b-4de1-a946-1a26d9036171\") " pod="openstack-kuttl-tests/cinder-db-create-jmgjk" Dec 03 12:42:28 crc kubenswrapper[5052]: I1203 12:42:28.353638 5052 scope.go:117] "RemoveContainer" containerID="e732642ec3d2e2c1b77a5c5cf4d43de4178312bd401dae5fa03708d2e3c9248f" Dec 03 12:42:28 crc kubenswrapper[5052]: I1203 12:42:28.364950 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-db-create-5hb92"] Dec 03 12:42:28 crc kubenswrapper[5052]: I1203 12:42:28.378534 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/cinder-4345-account-create-update-qqj8v"] Dec 03 12:42:28 crc kubenswrapper[5052]: I1203 12:42:28.379733 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-4345-account-create-update-qqj8v" Dec 03 12:42:28 crc kubenswrapper[5052]: I1203 12:42:28.386144 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cinder-db-secret" Dec 03 12:42:28 crc kubenswrapper[5052]: I1203 12:42:28.403062 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-4345-account-create-update-qqj8v"] Dec 03 12:42:28 crc kubenswrapper[5052]: I1203 12:42:28.503003 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/53834739-1c4b-4de1-a946-1a26d9036171-operator-scripts\") pod \"cinder-db-create-jmgjk\" (UID: \"53834739-1c4b-4de1-a946-1a26d9036171\") " pod="openstack-kuttl-tests/cinder-db-create-jmgjk" Dec 03 12:42:28 crc kubenswrapper[5052]: I1203 12:42:28.503083 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/984da95f-b2c9-422b-b09d-6b7f80bf56e1-operator-scripts\") pod \"barbican-db-create-5hb92\" (UID: \"984da95f-b2c9-422b-b09d-6b7f80bf56e1\") " pod="openstack-kuttl-tests/barbican-db-create-5hb92" Dec 03 12:42:28 crc kubenswrapper[5052]: I1203 12:42:28.503157 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-4bpmm\" (UniqueName: \"kubernetes.io/projected/53834739-1c4b-4de1-a946-1a26d9036171-kube-api-access-4bpmm\") pod \"cinder-db-create-jmgjk\" (UID: \"53834739-1c4b-4de1-a946-1a26d9036171\") " pod="openstack-kuttl-tests/cinder-db-create-jmgjk" Dec 03 12:42:28 crc kubenswrapper[5052]: I1203 12:42:28.503208 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/79f5c230-497d-4183-b174-e1a927ffc003-operator-scripts\") pod \"cinder-4345-account-create-update-qqj8v\" (UID: \"79f5c230-497d-4183-b174-e1a927ffc003\") " pod="openstack-kuttl-tests/cinder-4345-account-create-update-qqj8v" Dec 03 12:42:28 crc kubenswrapper[5052]: I1203 12:42:28.503277 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b6frr\" (UniqueName: \"kubernetes.io/projected/984da95f-b2c9-422b-b09d-6b7f80bf56e1-kube-api-access-b6frr\") pod \"barbican-db-create-5hb92\" (UID: \"984da95f-b2c9-422b-b09d-6b7f80bf56e1\") " pod="openstack-kuttl-tests/barbican-db-create-5hb92" Dec 03 12:42:28 crc kubenswrapper[5052]: I1203 12:42:28.503430 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vjsbg\" (UniqueName: \"kubernetes.io/projected/79f5c230-497d-4183-b174-e1a927ffc003-kube-api-access-vjsbg\") pod \"cinder-4345-account-create-update-qqj8v\" (UID: \"79f5c230-497d-4183-b174-e1a927ffc003\") " pod="openstack-kuttl-tests/cinder-4345-account-create-update-qqj8v" Dec 03 12:42:28 crc kubenswrapper[5052]: I1203 12:42:28.504381 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/53834739-1c4b-4de1-a946-1a26d9036171-operator-scripts\") pod \"cinder-db-create-jmgjk\" (UID: \"53834739-1c4b-4de1-a946-1a26d9036171\") " pod="openstack-kuttl-tests/cinder-db-create-jmgjk" Dec 03 12:42:28 crc kubenswrapper[5052]: I1203 12:42:28.516503 5052 scope.go:117] "RemoveContainer" containerID="8f30cdd770e803faba76b2be2f9c6ae0cba884a14e491e0151c3400a085e89d1" Dec 03 12:42:28 crc kubenswrapper[5052]: I1203 12:42:28.565808 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/barbican-02cc-account-create-update-zzv8d"] Dec 03 12:42:28 crc kubenswrapper[5052]: I1203 12:42:28.568367 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-02cc-account-create-update-zzv8d" Dec 03 12:42:28 crc kubenswrapper[5052]: I1203 12:42:28.571349 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"barbican-db-secret" Dec 03 12:42:28 crc kubenswrapper[5052]: I1203 12:42:28.576105 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-4bpmm\" (UniqueName: \"kubernetes.io/projected/53834739-1c4b-4de1-a946-1a26d9036171-kube-api-access-4bpmm\") pod \"cinder-db-create-jmgjk\" (UID: \"53834739-1c4b-4de1-a946-1a26d9036171\") " pod="openstack-kuttl-tests/cinder-db-create-jmgjk" Dec 03 12:42:28 crc kubenswrapper[5052]: I1203 12:42:28.588581 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-db-create-jmgjk" Dec 03 12:42:28 crc kubenswrapper[5052]: I1203 12:42:28.595711 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-02cc-account-create-update-zzv8d"] Dec 03 12:42:28 crc kubenswrapper[5052]: I1203 12:42:28.608524 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-vjsbg\" (UniqueName: \"kubernetes.io/projected/79f5c230-497d-4183-b174-e1a927ffc003-kube-api-access-vjsbg\") pod \"cinder-4345-account-create-update-qqj8v\" (UID: \"79f5c230-497d-4183-b174-e1a927ffc003\") " pod="openstack-kuttl-tests/cinder-4345-account-create-update-qqj8v" Dec 03 12:42:28 crc kubenswrapper[5052]: I1203 12:42:28.608636 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/984da95f-b2c9-422b-b09d-6b7f80bf56e1-operator-scripts\") pod \"barbican-db-create-5hb92\" (UID: \"984da95f-b2c9-422b-b09d-6b7f80bf56e1\") " pod="openstack-kuttl-tests/barbican-db-create-5hb92" Dec 03 12:42:28 crc kubenswrapper[5052]: I1203 12:42:28.608681 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/79f5c230-497d-4183-b174-e1a927ffc003-operator-scripts\") pod \"cinder-4345-account-create-update-qqj8v\" (UID: \"79f5c230-497d-4183-b174-e1a927ffc003\") " pod="openstack-kuttl-tests/cinder-4345-account-create-update-qqj8v" Dec 03 12:42:28 crc kubenswrapper[5052]: I1203 12:42:28.608728 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-b6frr\" (UniqueName: \"kubernetes.io/projected/984da95f-b2c9-422b-b09d-6b7f80bf56e1-kube-api-access-b6frr\") pod \"barbican-db-create-5hb92\" (UID: \"984da95f-b2c9-422b-b09d-6b7f80bf56e1\") " pod="openstack-kuttl-tests/barbican-db-create-5hb92" Dec 03 12:42:28 crc kubenswrapper[5052]: I1203 12:42:28.609497 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/984da95f-b2c9-422b-b09d-6b7f80bf56e1-operator-scripts\") pod \"barbican-db-create-5hb92\" (UID: \"984da95f-b2c9-422b-b09d-6b7f80bf56e1\") " pod="openstack-kuttl-tests/barbican-db-create-5hb92" Dec 03 12:42:28 crc kubenswrapper[5052]: I1203 12:42:28.609592 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/79f5c230-497d-4183-b174-e1a927ffc003-operator-scripts\") pod \"cinder-4345-account-create-update-qqj8v\" (UID: \"79f5c230-497d-4183-b174-e1a927ffc003\") " pod="openstack-kuttl-tests/cinder-4345-account-create-update-qqj8v" Dec 03 12:42:28 crc kubenswrapper[5052]: I1203 12:42:28.640573 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/neutron-db-create-jprpg"] Dec 03 12:42:28 crc kubenswrapper[5052]: I1203 12:42:28.641973 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-db-create-jprpg" Dec 03 12:42:28 crc kubenswrapper[5052]: I1203 12:42:28.654802 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-vjsbg\" (UniqueName: \"kubernetes.io/projected/79f5c230-497d-4183-b174-e1a927ffc003-kube-api-access-vjsbg\") pod \"cinder-4345-account-create-update-qqj8v\" (UID: \"79f5c230-497d-4183-b174-e1a927ffc003\") " pod="openstack-kuttl-tests/cinder-4345-account-create-update-qqj8v" Dec 03 12:42:28 crc kubenswrapper[5052]: I1203 12:42:28.660267 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-db-create-jprpg"] Dec 03 12:42:28 crc kubenswrapper[5052]: I1203 12:42:28.663673 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-b6frr\" (UniqueName: \"kubernetes.io/projected/984da95f-b2c9-422b-b09d-6b7f80bf56e1-kube-api-access-b6frr\") pod \"barbican-db-create-5hb92\" (UID: \"984da95f-b2c9-422b-b09d-6b7f80bf56e1\") " pod="openstack-kuttl-tests/barbican-db-create-5hb92" Dec 03 12:42:28 crc kubenswrapper[5052]: I1203 12:42:28.693193 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-db-create-5hb92" Dec 03 12:42:28 crc kubenswrapper[5052]: I1203 12:42:28.711731 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/0d36a272-0133-4e61-bb88-219ff124cc91-operator-scripts\") pod \"barbican-02cc-account-create-update-zzv8d\" (UID: \"0d36a272-0133-4e61-bb88-219ff124cc91\") " pod="openstack-kuttl-tests/barbican-02cc-account-create-update-zzv8d" Dec 03 12:42:28 crc kubenswrapper[5052]: I1203 12:42:28.711804 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/555223ae-4781-4c74-8ae0-053e1153ad9f-operator-scripts\") pod \"neutron-db-create-jprpg\" (UID: \"555223ae-4781-4c74-8ae0-053e1153ad9f\") " pod="openstack-kuttl-tests/neutron-db-create-jprpg" Dec 03 12:42:28 crc kubenswrapper[5052]: I1203 12:42:28.711858 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-69m79\" (UniqueName: \"kubernetes.io/projected/555223ae-4781-4c74-8ae0-053e1153ad9f-kube-api-access-69m79\") pod \"neutron-db-create-jprpg\" (UID: \"555223ae-4781-4c74-8ae0-053e1153ad9f\") " pod="openstack-kuttl-tests/neutron-db-create-jprpg" Dec 03 12:42:28 crc kubenswrapper[5052]: I1203 12:42:28.711884 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ddlmj\" (UniqueName: \"kubernetes.io/projected/0d36a272-0133-4e61-bb88-219ff124cc91-kube-api-access-ddlmj\") pod \"barbican-02cc-account-create-update-zzv8d\" (UID: \"0d36a272-0133-4e61-bb88-219ff124cc91\") " pod="openstack-kuttl-tests/barbican-02cc-account-create-update-zzv8d" Dec 03 12:42:28 crc kubenswrapper[5052]: I1203 12:42:28.722239 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-4345-account-create-update-qqj8v" Dec 03 12:42:28 crc kubenswrapper[5052]: I1203 12:42:28.771453 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/keystone-db-sync-rgdrt"] Dec 03 12:42:28 crc kubenswrapper[5052]: I1203 12:42:28.772845 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-db-sync-rgdrt" Dec 03 12:42:28 crc kubenswrapper[5052]: I1203 12:42:28.782191 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone-config-data" Dec 03 12:42:28 crc kubenswrapper[5052]: I1203 12:42:28.782608 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone-keystone-dockercfg-2f85l" Dec 03 12:42:28 crc kubenswrapper[5052]: I1203 12:42:28.782758 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone" Dec 03 12:42:28 crc kubenswrapper[5052]: I1203 12:42:28.782984 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone-scripts" Dec 03 12:42:28 crc kubenswrapper[5052]: I1203 12:42:28.783629 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/neutron-a25a-account-create-update-4bq8m"] Dec 03 12:42:28 crc kubenswrapper[5052]: I1203 12:42:28.785033 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-a25a-account-create-update-4bq8m" Dec 03 12:42:28 crc kubenswrapper[5052]: I1203 12:42:28.789342 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"neutron-db-secret" Dec 03 12:42:28 crc kubenswrapper[5052]: I1203 12:42:28.797270 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-db-sync-rgdrt"] Dec 03 12:42:28 crc kubenswrapper[5052]: I1203 12:42:28.814673 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/0d36a272-0133-4e61-bb88-219ff124cc91-operator-scripts\") pod \"barbican-02cc-account-create-update-zzv8d\" (UID: \"0d36a272-0133-4e61-bb88-219ff124cc91\") " pod="openstack-kuttl-tests/barbican-02cc-account-create-update-zzv8d" Dec 03 12:42:28 crc kubenswrapper[5052]: I1203 12:42:28.815091 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/555223ae-4781-4c74-8ae0-053e1153ad9f-operator-scripts\") pod \"neutron-db-create-jprpg\" (UID: \"555223ae-4781-4c74-8ae0-053e1153ad9f\") " pod="openstack-kuttl-tests/neutron-db-create-jprpg" Dec 03 12:42:28 crc kubenswrapper[5052]: I1203 12:42:28.815179 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-69m79\" (UniqueName: \"kubernetes.io/projected/555223ae-4781-4c74-8ae0-053e1153ad9f-kube-api-access-69m79\") pod \"neutron-db-create-jprpg\" (UID: \"555223ae-4781-4c74-8ae0-053e1153ad9f\") " pod="openstack-kuttl-tests/neutron-db-create-jprpg" Dec 03 12:42:28 crc kubenswrapper[5052]: I1203 12:42:28.815206 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-ddlmj\" (UniqueName: \"kubernetes.io/projected/0d36a272-0133-4e61-bb88-219ff124cc91-kube-api-access-ddlmj\") pod \"barbican-02cc-account-create-update-zzv8d\" (UID: \"0d36a272-0133-4e61-bb88-219ff124cc91\") " pod="openstack-kuttl-tests/barbican-02cc-account-create-update-zzv8d" Dec 03 12:42:28 crc kubenswrapper[5052]: I1203 12:42:28.815435 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/0d36a272-0133-4e61-bb88-219ff124cc91-operator-scripts\") pod \"barbican-02cc-account-create-update-zzv8d\" (UID: \"0d36a272-0133-4e61-bb88-219ff124cc91\") " pod="openstack-kuttl-tests/barbican-02cc-account-create-update-zzv8d" Dec 03 12:42:28 crc kubenswrapper[5052]: I1203 12:42:28.816252 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/555223ae-4781-4c74-8ae0-053e1153ad9f-operator-scripts\") pod \"neutron-db-create-jprpg\" (UID: \"555223ae-4781-4c74-8ae0-053e1153ad9f\") " pod="openstack-kuttl-tests/neutron-db-create-jprpg" Dec 03 12:42:28 crc kubenswrapper[5052]: I1203 12:42:28.829764 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-a25a-account-create-update-4bq8m"] Dec 03 12:42:28 crc kubenswrapper[5052]: I1203 12:42:28.842329 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-ddlmj\" (UniqueName: \"kubernetes.io/projected/0d36a272-0133-4e61-bb88-219ff124cc91-kube-api-access-ddlmj\") pod \"barbican-02cc-account-create-update-zzv8d\" (UID: \"0d36a272-0133-4e61-bb88-219ff124cc91\") " pod="openstack-kuttl-tests/barbican-02cc-account-create-update-zzv8d" Dec 03 12:42:28 crc kubenswrapper[5052]: I1203 12:42:28.849268 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-69m79\" (UniqueName: \"kubernetes.io/projected/555223ae-4781-4c74-8ae0-053e1153ad9f-kube-api-access-69m79\") pod \"neutron-db-create-jprpg\" (UID: \"555223ae-4781-4c74-8ae0-053e1153ad9f\") " pod="openstack-kuttl-tests/neutron-db-create-jprpg" Dec 03 12:42:28 crc kubenswrapper[5052]: I1203 12:42:28.916644 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vhq54\" (UniqueName: \"kubernetes.io/projected/2e4d7dee-0fd0-437e-a529-2dafd03c6492-kube-api-access-vhq54\") pod \"keystone-db-sync-rgdrt\" (UID: \"2e4d7dee-0fd0-437e-a529-2dafd03c6492\") " pod="openstack-kuttl-tests/keystone-db-sync-rgdrt" Dec 03 12:42:28 crc kubenswrapper[5052]: I1203 12:42:28.916978 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hhq54\" (UniqueName: \"kubernetes.io/projected/ec00af82-d7ae-44ad-9023-d88663706469-kube-api-access-hhq54\") pod \"neutron-a25a-account-create-update-4bq8m\" (UID: \"ec00af82-d7ae-44ad-9023-d88663706469\") " pod="openstack-kuttl-tests/neutron-a25a-account-create-update-4bq8m" Dec 03 12:42:28 crc kubenswrapper[5052]: I1203 12:42:28.917127 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2e4d7dee-0fd0-437e-a529-2dafd03c6492-combined-ca-bundle\") pod \"keystone-db-sync-rgdrt\" (UID: \"2e4d7dee-0fd0-437e-a529-2dafd03c6492\") " pod="openstack-kuttl-tests/keystone-db-sync-rgdrt" Dec 03 12:42:28 crc kubenswrapper[5052]: I1203 12:42:28.917229 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/2e4d7dee-0fd0-437e-a529-2dafd03c6492-config-data\") pod \"keystone-db-sync-rgdrt\" (UID: \"2e4d7dee-0fd0-437e-a529-2dafd03c6492\") " pod="openstack-kuttl-tests/keystone-db-sync-rgdrt" Dec 03 12:42:28 crc kubenswrapper[5052]: I1203 12:42:28.917271 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/ec00af82-d7ae-44ad-9023-d88663706469-operator-scripts\") pod \"neutron-a25a-account-create-update-4bq8m\" (UID: \"ec00af82-d7ae-44ad-9023-d88663706469\") " pod="openstack-kuttl-tests/neutron-a25a-account-create-update-4bq8m" Dec 03 12:42:29 crc kubenswrapper[5052]: I1203 12:42:29.019781 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-hhq54\" (UniqueName: \"kubernetes.io/projected/ec00af82-d7ae-44ad-9023-d88663706469-kube-api-access-hhq54\") pod \"neutron-a25a-account-create-update-4bq8m\" (UID: \"ec00af82-d7ae-44ad-9023-d88663706469\") " pod="openstack-kuttl-tests/neutron-a25a-account-create-update-4bq8m" Dec 03 12:42:29 crc kubenswrapper[5052]: I1203 12:42:29.019846 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2e4d7dee-0fd0-437e-a529-2dafd03c6492-combined-ca-bundle\") pod \"keystone-db-sync-rgdrt\" (UID: \"2e4d7dee-0fd0-437e-a529-2dafd03c6492\") " pod="openstack-kuttl-tests/keystone-db-sync-rgdrt" Dec 03 12:42:29 crc kubenswrapper[5052]: I1203 12:42:29.019889 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/2e4d7dee-0fd0-437e-a529-2dafd03c6492-config-data\") pod \"keystone-db-sync-rgdrt\" (UID: \"2e4d7dee-0fd0-437e-a529-2dafd03c6492\") " pod="openstack-kuttl-tests/keystone-db-sync-rgdrt" Dec 03 12:42:29 crc kubenswrapper[5052]: I1203 12:42:29.019917 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/ec00af82-d7ae-44ad-9023-d88663706469-operator-scripts\") pod \"neutron-a25a-account-create-update-4bq8m\" (UID: \"ec00af82-d7ae-44ad-9023-d88663706469\") " pod="openstack-kuttl-tests/neutron-a25a-account-create-update-4bq8m" Dec 03 12:42:29 crc kubenswrapper[5052]: I1203 12:42:29.019943 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-vhq54\" (UniqueName: \"kubernetes.io/projected/2e4d7dee-0fd0-437e-a529-2dafd03c6492-kube-api-access-vhq54\") pod \"keystone-db-sync-rgdrt\" (UID: \"2e4d7dee-0fd0-437e-a529-2dafd03c6492\") " pod="openstack-kuttl-tests/keystone-db-sync-rgdrt" Dec 03 12:42:29 crc kubenswrapper[5052]: I1203 12:42:29.021298 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/ec00af82-d7ae-44ad-9023-d88663706469-operator-scripts\") pod \"neutron-a25a-account-create-update-4bq8m\" (UID: \"ec00af82-d7ae-44ad-9023-d88663706469\") " pod="openstack-kuttl-tests/neutron-a25a-account-create-update-4bq8m" Dec 03 12:42:29 crc kubenswrapper[5052]: I1203 12:42:29.025449 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2e4d7dee-0fd0-437e-a529-2dafd03c6492-combined-ca-bundle\") pod \"keystone-db-sync-rgdrt\" (UID: \"2e4d7dee-0fd0-437e-a529-2dafd03c6492\") " pod="openstack-kuttl-tests/keystone-db-sync-rgdrt" Dec 03 12:42:29 crc kubenswrapper[5052]: I1203 12:42:29.028006 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/2e4d7dee-0fd0-437e-a529-2dafd03c6492-config-data\") pod \"keystone-db-sync-rgdrt\" (UID: \"2e4d7dee-0fd0-437e-a529-2dafd03c6492\") " pod="openstack-kuttl-tests/keystone-db-sync-rgdrt" Dec 03 12:42:29 crc kubenswrapper[5052]: I1203 12:42:29.040457 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-vhq54\" (UniqueName: \"kubernetes.io/projected/2e4d7dee-0fd0-437e-a529-2dafd03c6492-kube-api-access-vhq54\") pod \"keystone-db-sync-rgdrt\" (UID: \"2e4d7dee-0fd0-437e-a529-2dafd03c6492\") " pod="openstack-kuttl-tests/keystone-db-sync-rgdrt" Dec 03 12:42:29 crc kubenswrapper[5052]: I1203 12:42:29.042100 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-hhq54\" (UniqueName: \"kubernetes.io/projected/ec00af82-d7ae-44ad-9023-d88663706469-kube-api-access-hhq54\") pod \"neutron-a25a-account-create-update-4bq8m\" (UID: \"ec00af82-d7ae-44ad-9023-d88663706469\") " pod="openstack-kuttl-tests/neutron-a25a-account-create-update-4bq8m" Dec 03 12:42:29 crc kubenswrapper[5052]: I1203 12:42:29.060161 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-02cc-account-create-update-zzv8d" Dec 03 12:42:29 crc kubenswrapper[5052]: I1203 12:42:29.084662 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:42:29 crc kubenswrapper[5052]: I1203 12:42:29.127968 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-db-create-jprpg" Dec 03 12:42:29 crc kubenswrapper[5052]: I1203 12:42:29.148235 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-db-sync-rgdrt" Dec 03 12:42:29 crc kubenswrapper[5052]: I1203 12:42:29.154485 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-a25a-account-create-update-4bq8m" Dec 03 12:42:29 crc kubenswrapper[5052]: I1203 12:42:29.289385 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-db-create-5hb92"] Dec 03 12:42:29 crc kubenswrapper[5052]: W1203 12:42:29.295721 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod984da95f_b2c9_422b_b09d_6b7f80bf56e1.slice/crio-d4850330d25ba76d6230db34bf7331e6e2d5e9fd68adc3e02fcd4ceb0b6dc8a9 WatchSource:0}: Error finding container d4850330d25ba76d6230db34bf7331e6e2d5e9fd68adc3e02fcd4ceb0b6dc8a9: Status 404 returned error can't find the container with id d4850330d25ba76d6230db34bf7331e6e2d5e9fd68adc3e02fcd4ceb0b6dc8a9 Dec 03 12:42:29 crc kubenswrapper[5052]: I1203 12:42:29.343150 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-db-create-jmgjk"] Dec 03 12:42:29 crc kubenswrapper[5052]: I1203 12:42:29.361478 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-4345-account-create-update-qqj8v"] Dec 03 12:42:29 crc kubenswrapper[5052]: I1203 12:42:29.380974 5052 patch_prober.go:28] interesting pod/machine-config-daemon-74bj6 container/machine-config-daemon namespace/openshift-machine-config-operator: Liveness probe status=failure output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" start-of-body= Dec 03 12:42:29 crc kubenswrapper[5052]: I1203 12:42:29.381023 5052 prober.go:107] "Probe failed" probeType="Liveness" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" containerName="machine-config-daemon" probeResult="failure" output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" Dec 03 12:42:29 crc kubenswrapper[5052]: I1203 12:42:29.645157 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-db-create-jprpg"] Dec 03 12:42:29 crc kubenswrapper[5052]: I1203 12:42:29.704071 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-02cc-account-create-update-zzv8d"] Dec 03 12:42:29 crc kubenswrapper[5052]: I1203 12:42:29.801853 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-db-create-5hb92" event={"ID":"984da95f-b2c9-422b-b09d-6b7f80bf56e1","Type":"ContainerStarted","Data":"d4850330d25ba76d6230db34bf7331e6e2d5e9fd68adc3e02fcd4ceb0b6dc8a9"} Dec 03 12:42:29 crc kubenswrapper[5052]: I1203 12:42:29.913240 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-a25a-account-create-update-4bq8m"] Dec 03 12:42:29 crc kubenswrapper[5052]: I1203 12:42:29.939760 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-db-sync-rgdrt"] Dec 03 12:42:32 crc kubenswrapper[5052]: W1203 12:42:32.943885 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod79f5c230_497d_4183_b174_e1a927ffc003.slice/crio-b2bada8ac49e4cf0c410ad176fc86b54cb1cd5340421dd89dfd7178f071d5f28 WatchSource:0}: Error finding container b2bada8ac49e4cf0c410ad176fc86b54cb1cd5340421dd89dfd7178f071d5f28: Status 404 returned error can't find the container with id b2bada8ac49e4cf0c410ad176fc86b54cb1cd5340421dd89dfd7178f071d5f28 Dec 03 12:42:32 crc kubenswrapper[5052]: W1203 12:42:32.996104 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2e4d7dee_0fd0_437e_a529_2dafd03c6492.slice/crio-0e3084006b1bb94584f399d013bc851c7d7a22f7a8887d846dab273eb0603e4d WatchSource:0}: Error finding container 0e3084006b1bb94584f399d013bc851c7d7a22f7a8887d846dab273eb0603e4d: Status 404 returned error can't find the container with id 0e3084006b1bb94584f399d013bc851c7d7a22f7a8887d846dab273eb0603e4d Dec 03 12:42:33 crc kubenswrapper[5052]: I1203 12:42:33.843777 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-02cc-account-create-update-zzv8d" event={"ID":"0d36a272-0133-4e61-bb88-219ff124cc91","Type":"ContainerStarted","Data":"5574cf1b9637d9dc4df4d731d497a66be92d5061b8f70847d57c773861022281"} Dec 03 12:42:33 crc kubenswrapper[5052]: I1203 12:42:33.844131 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-02cc-account-create-update-zzv8d" event={"ID":"0d36a272-0133-4e61-bb88-219ff124cc91","Type":"ContainerStarted","Data":"3597aa00cf28180fa11515682cde30161b643eb32ba282d6d99091ebfde537f7"} Dec 03 12:42:33 crc kubenswrapper[5052]: I1203 12:42:33.846320 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-db-create-jprpg" event={"ID":"555223ae-4781-4c74-8ae0-053e1153ad9f","Type":"ContainerStarted","Data":"918f9e1c99cf9114714c32008148f123af54d1f2de2fdbb10e539c82c6d0d7a7"} Dec 03 12:42:33 crc kubenswrapper[5052]: I1203 12:42:33.846375 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-db-create-jprpg" event={"ID":"555223ae-4781-4c74-8ae0-053e1153ad9f","Type":"ContainerStarted","Data":"7e88e003b6d076fe95dc168978857f0c2b81b7f4636bffefe614188ef38f765b"} Dec 03 12:42:33 crc kubenswrapper[5052]: I1203 12:42:33.848182 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-4345-account-create-update-qqj8v" event={"ID":"79f5c230-497d-4183-b174-e1a927ffc003","Type":"ContainerStarted","Data":"28741b5100ab5f3a042ecef7948d376f4cfc37deb45f39e79dcf1bdea4115ee9"} Dec 03 12:42:33 crc kubenswrapper[5052]: I1203 12:42:33.848211 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-4345-account-create-update-qqj8v" event={"ID":"79f5c230-497d-4183-b174-e1a927ffc003","Type":"ContainerStarted","Data":"b2bada8ac49e4cf0c410ad176fc86b54cb1cd5340421dd89dfd7178f071d5f28"} Dec 03 12:42:33 crc kubenswrapper[5052]: I1203 12:42:33.850824 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-db-create-jmgjk" event={"ID":"53834739-1c4b-4de1-a946-1a26d9036171","Type":"ContainerStarted","Data":"fd0a7f48fe60812e7073999bafe14759c449ef5d471df89f6fa9c0b085b41b7c"} Dec 03 12:42:33 crc kubenswrapper[5052]: I1203 12:42:33.850846 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-db-create-jmgjk" event={"ID":"53834739-1c4b-4de1-a946-1a26d9036171","Type":"ContainerStarted","Data":"3d8a906ed5bae8a11a9ef46b6483ce7cc463976a76d7b64501e46573603dea97"} Dec 03 12:42:33 crc kubenswrapper[5052]: I1203 12:42:33.854187 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-db-sync-rgdrt" event={"ID":"2e4d7dee-0fd0-437e-a529-2dafd03c6492","Type":"ContainerStarted","Data":"0fcd271f98e04087a547c0155f1cd70b4b5b3a96bfb2a5c4a2f74fcf8ba00936"} Dec 03 12:42:33 crc kubenswrapper[5052]: I1203 12:42:33.854226 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-db-sync-rgdrt" event={"ID":"2e4d7dee-0fd0-437e-a529-2dafd03c6492","Type":"ContainerStarted","Data":"0e3084006b1bb94584f399d013bc851c7d7a22f7a8887d846dab273eb0603e4d"} Dec 03 12:42:33 crc kubenswrapper[5052]: I1203 12:42:33.857085 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-db-create-5hb92" event={"ID":"984da95f-b2c9-422b-b09d-6b7f80bf56e1","Type":"ContainerStarted","Data":"925b2497e4f642ed4fb1967329c7ab704e5ce009931ac3070b5ab19c386d16f2"} Dec 03 12:42:33 crc kubenswrapper[5052]: I1203 12:42:33.860084 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-a25a-account-create-update-4bq8m" event={"ID":"ec00af82-d7ae-44ad-9023-d88663706469","Type":"ContainerStarted","Data":"353555b1bb776ead8959c338289c90f24c0ffc2347769f7b77a175865eb773e9"} Dec 03 12:42:33 crc kubenswrapper[5052]: I1203 12:42:33.860234 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-a25a-account-create-update-4bq8m" event={"ID":"ec00af82-d7ae-44ad-9023-d88663706469","Type":"ContainerStarted","Data":"5acdda6d5345b00a50a6236b914225666bfa5af47fe78064d8fe43e81d4990eb"} Dec 03 12:42:33 crc kubenswrapper[5052]: I1203 12:42:33.866605 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/barbican-02cc-account-create-update-zzv8d" podStartSLOduration=5.866586808 podStartE2EDuration="5.866586808s" podCreationTimestamp="2025-12-03 12:42:28 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:42:33.864656893 +0000 UTC m=+4944.452898657" watchObservedRunningTime="2025-12-03 12:42:33.866586808 +0000 UTC m=+4944.454828572" Dec 03 12:42:33 crc kubenswrapper[5052]: I1203 12:42:33.889240 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/cinder-4345-account-create-update-qqj8v" podStartSLOduration=5.889215265 podStartE2EDuration="5.889215265s" podCreationTimestamp="2025-12-03 12:42:28 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:42:33.882660368 +0000 UTC m=+4944.470902132" watchObservedRunningTime="2025-12-03 12:42:33.889215265 +0000 UTC m=+4944.477457029" Dec 03 12:42:33 crc kubenswrapper[5052]: I1203 12:42:33.905099 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/neutron-a25a-account-create-update-4bq8m" podStartSLOduration=5.905078809 podStartE2EDuration="5.905078809s" podCreationTimestamp="2025-12-03 12:42:28 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:42:33.898493611 +0000 UTC m=+4944.486735385" watchObservedRunningTime="2025-12-03 12:42:33.905078809 +0000 UTC m=+4944.493320573" Dec 03 12:42:33 crc kubenswrapper[5052]: I1203 12:42:33.915641 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/keystone-db-sync-rgdrt" podStartSLOduration=5.91561658 podStartE2EDuration="5.91561658s" podCreationTimestamp="2025-12-03 12:42:28 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:42:33.913094898 +0000 UTC m=+4944.501336672" watchObservedRunningTime="2025-12-03 12:42:33.91561658 +0000 UTC m=+4944.503858344" Dec 03 12:42:33 crc kubenswrapper[5052]: I1203 12:42:33.931186 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/cinder-db-create-jmgjk" podStartSLOduration=5.9311639750000005 podStartE2EDuration="5.931163975s" podCreationTimestamp="2025-12-03 12:42:28 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:42:33.929058125 +0000 UTC m=+4944.517299889" watchObservedRunningTime="2025-12-03 12:42:33.931163975 +0000 UTC m=+4944.519405739" Dec 03 12:42:33 crc kubenswrapper[5052]: I1203 12:42:33.953005 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/barbican-db-create-5hb92" podStartSLOduration=5.952958028 podStartE2EDuration="5.952958028s" podCreationTimestamp="2025-12-03 12:42:28 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:42:33.946334889 +0000 UTC m=+4944.534576653" watchObservedRunningTime="2025-12-03 12:42:33.952958028 +0000 UTC m=+4944.541199792" Dec 03 12:42:33 crc kubenswrapper[5052]: I1203 12:42:33.967874 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/neutron-db-create-jprpg" podStartSLOduration=5.967852554 podStartE2EDuration="5.967852554s" podCreationTimestamp="2025-12-03 12:42:28 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:42:33.964181829 +0000 UTC m=+4944.552423593" watchObservedRunningTime="2025-12-03 12:42:33.967852554 +0000 UTC m=+4944.556094318" Dec 03 12:42:34 crc kubenswrapper[5052]: I1203 12:42:34.869802 5052 generic.go:334] "Generic (PLEG): container finished" podID="ec00af82-d7ae-44ad-9023-d88663706469" containerID="353555b1bb776ead8959c338289c90f24c0ffc2347769f7b77a175865eb773e9" exitCode=0 Dec 03 12:42:34 crc kubenswrapper[5052]: I1203 12:42:34.869891 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-a25a-account-create-update-4bq8m" event={"ID":"ec00af82-d7ae-44ad-9023-d88663706469","Type":"ContainerDied","Data":"353555b1bb776ead8959c338289c90f24c0ffc2347769f7b77a175865eb773e9"} Dec 03 12:42:34 crc kubenswrapper[5052]: I1203 12:42:34.872362 5052 generic.go:334] "Generic (PLEG): container finished" podID="0d36a272-0133-4e61-bb88-219ff124cc91" containerID="5574cf1b9637d9dc4df4d731d497a66be92d5061b8f70847d57c773861022281" exitCode=0 Dec 03 12:42:34 crc kubenswrapper[5052]: I1203 12:42:34.872426 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-02cc-account-create-update-zzv8d" event={"ID":"0d36a272-0133-4e61-bb88-219ff124cc91","Type":"ContainerDied","Data":"5574cf1b9637d9dc4df4d731d497a66be92d5061b8f70847d57c773861022281"} Dec 03 12:42:34 crc kubenswrapper[5052]: I1203 12:42:34.874561 5052 generic.go:334] "Generic (PLEG): container finished" podID="555223ae-4781-4c74-8ae0-053e1153ad9f" containerID="918f9e1c99cf9114714c32008148f123af54d1f2de2fdbb10e539c82c6d0d7a7" exitCode=0 Dec 03 12:42:34 crc kubenswrapper[5052]: I1203 12:42:34.874643 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-db-create-jprpg" event={"ID":"555223ae-4781-4c74-8ae0-053e1153ad9f","Type":"ContainerDied","Data":"918f9e1c99cf9114714c32008148f123af54d1f2de2fdbb10e539c82c6d0d7a7"} Dec 03 12:42:34 crc kubenswrapper[5052]: I1203 12:42:34.876415 5052 generic.go:334] "Generic (PLEG): container finished" podID="79f5c230-497d-4183-b174-e1a927ffc003" containerID="28741b5100ab5f3a042ecef7948d376f4cfc37deb45f39e79dcf1bdea4115ee9" exitCode=0 Dec 03 12:42:34 crc kubenswrapper[5052]: I1203 12:42:34.876496 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-4345-account-create-update-qqj8v" event={"ID":"79f5c230-497d-4183-b174-e1a927ffc003","Type":"ContainerDied","Data":"28741b5100ab5f3a042ecef7948d376f4cfc37deb45f39e79dcf1bdea4115ee9"} Dec 03 12:42:34 crc kubenswrapper[5052]: I1203 12:42:34.878051 5052 generic.go:334] "Generic (PLEG): container finished" podID="53834739-1c4b-4de1-a946-1a26d9036171" containerID="fd0a7f48fe60812e7073999bafe14759c449ef5d471df89f6fa9c0b085b41b7c" exitCode=0 Dec 03 12:42:34 crc kubenswrapper[5052]: I1203 12:42:34.878079 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-db-create-jmgjk" event={"ID":"53834739-1c4b-4de1-a946-1a26d9036171","Type":"ContainerDied","Data":"fd0a7f48fe60812e7073999bafe14759c449ef5d471df89f6fa9c0b085b41b7c"} Dec 03 12:42:34 crc kubenswrapper[5052]: I1203 12:42:34.879658 5052 generic.go:334] "Generic (PLEG): container finished" podID="984da95f-b2c9-422b-b09d-6b7f80bf56e1" containerID="925b2497e4f642ed4fb1967329c7ab704e5ce009931ac3070b5ab19c386d16f2" exitCode=0 Dec 03 12:42:34 crc kubenswrapper[5052]: I1203 12:42:34.879720 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-db-create-5hb92" event={"ID":"984da95f-b2c9-422b-b09d-6b7f80bf56e1","Type":"ContainerDied","Data":"925b2497e4f642ed4fb1967329c7ab704e5ce009931ac3070b5ab19c386d16f2"} Dec 03 12:42:35 crc kubenswrapper[5052]: I1203 12:42:35.890085 5052 generic.go:334] "Generic (PLEG): container finished" podID="2e4d7dee-0fd0-437e-a529-2dafd03c6492" containerID="0fcd271f98e04087a547c0155f1cd70b4b5b3a96bfb2a5c4a2f74fcf8ba00936" exitCode=0 Dec 03 12:42:35 crc kubenswrapper[5052]: I1203 12:42:35.890190 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-db-sync-rgdrt" event={"ID":"2e4d7dee-0fd0-437e-a529-2dafd03c6492","Type":"ContainerDied","Data":"0fcd271f98e04087a547c0155f1cd70b4b5b3a96bfb2a5c4a2f74fcf8ba00936"} Dec 03 12:42:36 crc kubenswrapper[5052]: I1203 12:42:36.271571 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-a25a-account-create-update-4bq8m" Dec 03 12:42:36 crc kubenswrapper[5052]: I1203 12:42:36.394607 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/ec00af82-d7ae-44ad-9023-d88663706469-operator-scripts\") pod \"ec00af82-d7ae-44ad-9023-d88663706469\" (UID: \"ec00af82-d7ae-44ad-9023-d88663706469\") " Dec 03 12:42:36 crc kubenswrapper[5052]: I1203 12:42:36.394650 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hhq54\" (UniqueName: \"kubernetes.io/projected/ec00af82-d7ae-44ad-9023-d88663706469-kube-api-access-hhq54\") pod \"ec00af82-d7ae-44ad-9023-d88663706469\" (UID: \"ec00af82-d7ae-44ad-9023-d88663706469\") " Dec 03 12:42:36 crc kubenswrapper[5052]: I1203 12:42:36.395470 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ec00af82-d7ae-44ad-9023-d88663706469-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "ec00af82-d7ae-44ad-9023-d88663706469" (UID: "ec00af82-d7ae-44ad-9023-d88663706469"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:42:36 crc kubenswrapper[5052]: I1203 12:42:36.396124 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/ec00af82-d7ae-44ad-9023-d88663706469-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:42:36 crc kubenswrapper[5052]: I1203 12:42:36.408009 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ec00af82-d7ae-44ad-9023-d88663706469-kube-api-access-hhq54" (OuterVolumeSpecName: "kube-api-access-hhq54") pod "ec00af82-d7ae-44ad-9023-d88663706469" (UID: "ec00af82-d7ae-44ad-9023-d88663706469"). InnerVolumeSpecName "kube-api-access-hhq54". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:42:36 crc kubenswrapper[5052]: I1203 12:42:36.472384 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-4345-account-create-update-qqj8v" Dec 03 12:42:36 crc kubenswrapper[5052]: I1203 12:42:36.480966 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-db-create-jprpg" Dec 03 12:42:36 crc kubenswrapper[5052]: I1203 12:42:36.493391 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-db-create-5hb92" Dec 03 12:42:36 crc kubenswrapper[5052]: I1203 12:42:36.496598 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b6frr\" (UniqueName: \"kubernetes.io/projected/984da95f-b2c9-422b-b09d-6b7f80bf56e1-kube-api-access-b6frr\") pod \"984da95f-b2c9-422b-b09d-6b7f80bf56e1\" (UID: \"984da95f-b2c9-422b-b09d-6b7f80bf56e1\") " Dec 03 12:42:36 crc kubenswrapper[5052]: I1203 12:42:36.496659 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/984da95f-b2c9-422b-b09d-6b7f80bf56e1-operator-scripts\") pod \"984da95f-b2c9-422b-b09d-6b7f80bf56e1\" (UID: \"984da95f-b2c9-422b-b09d-6b7f80bf56e1\") " Dec 03 12:42:36 crc kubenswrapper[5052]: I1203 12:42:36.496681 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-69m79\" (UniqueName: \"kubernetes.io/projected/555223ae-4781-4c74-8ae0-053e1153ad9f-kube-api-access-69m79\") pod \"555223ae-4781-4c74-8ae0-053e1153ad9f\" (UID: \"555223ae-4781-4c74-8ae0-053e1153ad9f\") " Dec 03 12:42:36 crc kubenswrapper[5052]: I1203 12:42:36.496708 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/79f5c230-497d-4183-b174-e1a927ffc003-operator-scripts\") pod \"79f5c230-497d-4183-b174-e1a927ffc003\" (UID: \"79f5c230-497d-4183-b174-e1a927ffc003\") " Dec 03 12:42:36 crc kubenswrapper[5052]: I1203 12:42:36.496791 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vjsbg\" (UniqueName: \"kubernetes.io/projected/79f5c230-497d-4183-b174-e1a927ffc003-kube-api-access-vjsbg\") pod \"79f5c230-497d-4183-b174-e1a927ffc003\" (UID: \"79f5c230-497d-4183-b174-e1a927ffc003\") " Dec 03 12:42:36 crc kubenswrapper[5052]: I1203 12:42:36.496814 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/555223ae-4781-4c74-8ae0-053e1153ad9f-operator-scripts\") pod \"555223ae-4781-4c74-8ae0-053e1153ad9f\" (UID: \"555223ae-4781-4c74-8ae0-053e1153ad9f\") " Dec 03 12:42:36 crc kubenswrapper[5052]: I1203 12:42:36.497068 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-hhq54\" (UniqueName: \"kubernetes.io/projected/ec00af82-d7ae-44ad-9023-d88663706469-kube-api-access-hhq54\") on node \"crc\" DevicePath \"\"" Dec 03 12:42:36 crc kubenswrapper[5052]: I1203 12:42:36.497413 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/555223ae-4781-4c74-8ae0-053e1153ad9f-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "555223ae-4781-4c74-8ae0-053e1153ad9f" (UID: "555223ae-4781-4c74-8ae0-053e1153ad9f"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:42:36 crc kubenswrapper[5052]: I1203 12:42:36.498371 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/79f5c230-497d-4183-b174-e1a927ffc003-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "79f5c230-497d-4183-b174-e1a927ffc003" (UID: "79f5c230-497d-4183-b174-e1a927ffc003"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:42:36 crc kubenswrapper[5052]: I1203 12:42:36.498463 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/984da95f-b2c9-422b-b09d-6b7f80bf56e1-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "984da95f-b2c9-422b-b09d-6b7f80bf56e1" (UID: "984da95f-b2c9-422b-b09d-6b7f80bf56e1"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:42:36 crc kubenswrapper[5052]: I1203 12:42:36.500689 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/79f5c230-497d-4183-b174-e1a927ffc003-kube-api-access-vjsbg" (OuterVolumeSpecName: "kube-api-access-vjsbg") pod "79f5c230-497d-4183-b174-e1a927ffc003" (UID: "79f5c230-497d-4183-b174-e1a927ffc003"). InnerVolumeSpecName "kube-api-access-vjsbg". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:42:36 crc kubenswrapper[5052]: I1203 12:42:36.500778 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/555223ae-4781-4c74-8ae0-053e1153ad9f-kube-api-access-69m79" (OuterVolumeSpecName: "kube-api-access-69m79") pod "555223ae-4781-4c74-8ae0-053e1153ad9f" (UID: "555223ae-4781-4c74-8ae0-053e1153ad9f"). InnerVolumeSpecName "kube-api-access-69m79". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:42:36 crc kubenswrapper[5052]: I1203 12:42:36.506867 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/984da95f-b2c9-422b-b09d-6b7f80bf56e1-kube-api-access-b6frr" (OuterVolumeSpecName: "kube-api-access-b6frr") pod "984da95f-b2c9-422b-b09d-6b7f80bf56e1" (UID: "984da95f-b2c9-422b-b09d-6b7f80bf56e1"). InnerVolumeSpecName "kube-api-access-b6frr". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:42:36 crc kubenswrapper[5052]: I1203 12:42:36.518237 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-db-create-jmgjk" Dec 03 12:42:36 crc kubenswrapper[5052]: I1203 12:42:36.551068 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-02cc-account-create-update-zzv8d" Dec 03 12:42:36 crc kubenswrapper[5052]: I1203 12:42:36.598146 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/0d36a272-0133-4e61-bb88-219ff124cc91-operator-scripts\") pod \"0d36a272-0133-4e61-bb88-219ff124cc91\" (UID: \"0d36a272-0133-4e61-bb88-219ff124cc91\") " Dec 03 12:42:36 crc kubenswrapper[5052]: I1203 12:42:36.598255 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ddlmj\" (UniqueName: \"kubernetes.io/projected/0d36a272-0133-4e61-bb88-219ff124cc91-kube-api-access-ddlmj\") pod \"0d36a272-0133-4e61-bb88-219ff124cc91\" (UID: \"0d36a272-0133-4e61-bb88-219ff124cc91\") " Dec 03 12:42:36 crc kubenswrapper[5052]: I1203 12:42:36.598289 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4bpmm\" (UniqueName: \"kubernetes.io/projected/53834739-1c4b-4de1-a946-1a26d9036171-kube-api-access-4bpmm\") pod \"53834739-1c4b-4de1-a946-1a26d9036171\" (UID: \"53834739-1c4b-4de1-a946-1a26d9036171\") " Dec 03 12:42:36 crc kubenswrapper[5052]: I1203 12:42:36.598330 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/53834739-1c4b-4de1-a946-1a26d9036171-operator-scripts\") pod \"53834739-1c4b-4de1-a946-1a26d9036171\" (UID: \"53834739-1c4b-4de1-a946-1a26d9036171\") " Dec 03 12:42:36 crc kubenswrapper[5052]: I1203 12:42:36.598651 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/555223ae-4781-4c74-8ae0-053e1153ad9f-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:42:36 crc kubenswrapper[5052]: I1203 12:42:36.598675 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-b6frr\" (UniqueName: \"kubernetes.io/projected/984da95f-b2c9-422b-b09d-6b7f80bf56e1-kube-api-access-b6frr\") on node \"crc\" DevicePath \"\"" Dec 03 12:42:36 crc kubenswrapper[5052]: I1203 12:42:36.598690 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/984da95f-b2c9-422b-b09d-6b7f80bf56e1-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:42:36 crc kubenswrapper[5052]: I1203 12:42:36.598702 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-69m79\" (UniqueName: \"kubernetes.io/projected/555223ae-4781-4c74-8ae0-053e1153ad9f-kube-api-access-69m79\") on node \"crc\" DevicePath \"\"" Dec 03 12:42:36 crc kubenswrapper[5052]: I1203 12:42:36.598691 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0d36a272-0133-4e61-bb88-219ff124cc91-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "0d36a272-0133-4e61-bb88-219ff124cc91" (UID: "0d36a272-0133-4e61-bb88-219ff124cc91"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:42:36 crc kubenswrapper[5052]: I1203 12:42:36.598714 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/79f5c230-497d-4183-b174-e1a927ffc003-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:42:36 crc kubenswrapper[5052]: I1203 12:42:36.598776 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-vjsbg\" (UniqueName: \"kubernetes.io/projected/79f5c230-497d-4183-b174-e1a927ffc003-kube-api-access-vjsbg\") on node \"crc\" DevicePath \"\"" Dec 03 12:42:36 crc kubenswrapper[5052]: I1203 12:42:36.599396 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/53834739-1c4b-4de1-a946-1a26d9036171-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "53834739-1c4b-4de1-a946-1a26d9036171" (UID: "53834739-1c4b-4de1-a946-1a26d9036171"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:42:36 crc kubenswrapper[5052]: I1203 12:42:36.601490 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0d36a272-0133-4e61-bb88-219ff124cc91-kube-api-access-ddlmj" (OuterVolumeSpecName: "kube-api-access-ddlmj") pod "0d36a272-0133-4e61-bb88-219ff124cc91" (UID: "0d36a272-0133-4e61-bb88-219ff124cc91"). InnerVolumeSpecName "kube-api-access-ddlmj". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:42:36 crc kubenswrapper[5052]: I1203 12:42:36.601686 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/53834739-1c4b-4de1-a946-1a26d9036171-kube-api-access-4bpmm" (OuterVolumeSpecName: "kube-api-access-4bpmm") pod "53834739-1c4b-4de1-a946-1a26d9036171" (UID: "53834739-1c4b-4de1-a946-1a26d9036171"). InnerVolumeSpecName "kube-api-access-4bpmm". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:42:36 crc kubenswrapper[5052]: I1203 12:42:36.704935 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-ddlmj\" (UniqueName: \"kubernetes.io/projected/0d36a272-0133-4e61-bb88-219ff124cc91-kube-api-access-ddlmj\") on node \"crc\" DevicePath \"\"" Dec 03 12:42:36 crc kubenswrapper[5052]: I1203 12:42:36.704967 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-4bpmm\" (UniqueName: \"kubernetes.io/projected/53834739-1c4b-4de1-a946-1a26d9036171-kube-api-access-4bpmm\") on node \"crc\" DevicePath \"\"" Dec 03 12:42:36 crc kubenswrapper[5052]: I1203 12:42:36.704979 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/53834739-1c4b-4de1-a946-1a26d9036171-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:42:36 crc kubenswrapper[5052]: I1203 12:42:36.704988 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/0d36a272-0133-4e61-bb88-219ff124cc91-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:42:36 crc kubenswrapper[5052]: I1203 12:42:36.900965 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-4345-account-create-update-qqj8v" event={"ID":"79f5c230-497d-4183-b174-e1a927ffc003","Type":"ContainerDied","Data":"b2bada8ac49e4cf0c410ad176fc86b54cb1cd5340421dd89dfd7178f071d5f28"} Dec 03 12:42:36 crc kubenswrapper[5052]: I1203 12:42:36.901293 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b2bada8ac49e4cf0c410ad176fc86b54cb1cd5340421dd89dfd7178f071d5f28" Dec 03 12:42:36 crc kubenswrapper[5052]: I1203 12:42:36.900989 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-4345-account-create-update-qqj8v" Dec 03 12:42:36 crc kubenswrapper[5052]: I1203 12:42:36.903740 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-db-create-jmgjk" Dec 03 12:42:36 crc kubenswrapper[5052]: I1203 12:42:36.906623 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-db-create-jmgjk" event={"ID":"53834739-1c4b-4de1-a946-1a26d9036171","Type":"ContainerDied","Data":"3d8a906ed5bae8a11a9ef46b6483ce7cc463976a76d7b64501e46573603dea97"} Dec 03 12:42:36 crc kubenswrapper[5052]: I1203 12:42:36.906684 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3d8a906ed5bae8a11a9ef46b6483ce7cc463976a76d7b64501e46573603dea97" Dec 03 12:42:36 crc kubenswrapper[5052]: I1203 12:42:36.908959 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-db-create-5hb92" event={"ID":"984da95f-b2c9-422b-b09d-6b7f80bf56e1","Type":"ContainerDied","Data":"d4850330d25ba76d6230db34bf7331e6e2d5e9fd68adc3e02fcd4ceb0b6dc8a9"} Dec 03 12:42:36 crc kubenswrapper[5052]: I1203 12:42:36.908984 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d4850330d25ba76d6230db34bf7331e6e2d5e9fd68adc3e02fcd4ceb0b6dc8a9" Dec 03 12:42:36 crc kubenswrapper[5052]: I1203 12:42:36.909033 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-db-create-5hb92" Dec 03 12:42:36 crc kubenswrapper[5052]: I1203 12:42:36.913152 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-a25a-account-create-update-4bq8m" Dec 03 12:42:36 crc kubenswrapper[5052]: I1203 12:42:36.913149 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-a25a-account-create-update-4bq8m" event={"ID":"ec00af82-d7ae-44ad-9023-d88663706469","Type":"ContainerDied","Data":"5acdda6d5345b00a50a6236b914225666bfa5af47fe78064d8fe43e81d4990eb"} Dec 03 12:42:36 crc kubenswrapper[5052]: I1203 12:42:36.913336 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5acdda6d5345b00a50a6236b914225666bfa5af47fe78064d8fe43e81d4990eb" Dec 03 12:42:36 crc kubenswrapper[5052]: I1203 12:42:36.918405 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-02cc-account-create-update-zzv8d" event={"ID":"0d36a272-0133-4e61-bb88-219ff124cc91","Type":"ContainerDied","Data":"3597aa00cf28180fa11515682cde30161b643eb32ba282d6d99091ebfde537f7"} Dec 03 12:42:36 crc kubenswrapper[5052]: I1203 12:42:36.918452 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3597aa00cf28180fa11515682cde30161b643eb32ba282d6d99091ebfde537f7" Dec 03 12:42:36 crc kubenswrapper[5052]: I1203 12:42:36.919646 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-02cc-account-create-update-zzv8d" Dec 03 12:42:36 crc kubenswrapper[5052]: I1203 12:42:36.920491 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-db-create-jprpg" event={"ID":"555223ae-4781-4c74-8ae0-053e1153ad9f","Type":"ContainerDied","Data":"7e88e003b6d076fe95dc168978857f0c2b81b7f4636bffefe614188ef38f765b"} Dec 03 12:42:36 crc kubenswrapper[5052]: I1203 12:42:36.920539 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7e88e003b6d076fe95dc168978857f0c2b81b7f4636bffefe614188ef38f765b" Dec 03 12:42:36 crc kubenswrapper[5052]: I1203 12:42:36.920596 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-db-create-jprpg" Dec 03 12:42:37 crc kubenswrapper[5052]: I1203 12:42:37.147430 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-db-sync-rgdrt" Dec 03 12:42:37 crc kubenswrapper[5052]: I1203 12:42:37.219721 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/2e4d7dee-0fd0-437e-a529-2dafd03c6492-config-data\") pod \"2e4d7dee-0fd0-437e-a529-2dafd03c6492\" (UID: \"2e4d7dee-0fd0-437e-a529-2dafd03c6492\") " Dec 03 12:42:37 crc kubenswrapper[5052]: I1203 12:42:37.219792 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2e4d7dee-0fd0-437e-a529-2dafd03c6492-combined-ca-bundle\") pod \"2e4d7dee-0fd0-437e-a529-2dafd03c6492\" (UID: \"2e4d7dee-0fd0-437e-a529-2dafd03c6492\") " Dec 03 12:42:37 crc kubenswrapper[5052]: I1203 12:42:37.219917 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vhq54\" (UniqueName: \"kubernetes.io/projected/2e4d7dee-0fd0-437e-a529-2dafd03c6492-kube-api-access-vhq54\") pod \"2e4d7dee-0fd0-437e-a529-2dafd03c6492\" (UID: \"2e4d7dee-0fd0-437e-a529-2dafd03c6492\") " Dec 03 12:42:37 crc kubenswrapper[5052]: I1203 12:42:37.227773 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2e4d7dee-0fd0-437e-a529-2dafd03c6492-kube-api-access-vhq54" (OuterVolumeSpecName: "kube-api-access-vhq54") pod "2e4d7dee-0fd0-437e-a529-2dafd03c6492" (UID: "2e4d7dee-0fd0-437e-a529-2dafd03c6492"). InnerVolumeSpecName "kube-api-access-vhq54". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:42:37 crc kubenswrapper[5052]: I1203 12:42:37.298690 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2e4d7dee-0fd0-437e-a529-2dafd03c6492-config-data" (OuterVolumeSpecName: "config-data") pod "2e4d7dee-0fd0-437e-a529-2dafd03c6492" (UID: "2e4d7dee-0fd0-437e-a529-2dafd03c6492"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:42:37 crc kubenswrapper[5052]: I1203 12:42:37.307745 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2e4d7dee-0fd0-437e-a529-2dafd03c6492-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "2e4d7dee-0fd0-437e-a529-2dafd03c6492" (UID: "2e4d7dee-0fd0-437e-a529-2dafd03c6492"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:42:37 crc kubenswrapper[5052]: I1203 12:42:37.323681 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-vhq54\" (UniqueName: \"kubernetes.io/projected/2e4d7dee-0fd0-437e-a529-2dafd03c6492-kube-api-access-vhq54\") on node \"crc\" DevicePath \"\"" Dec 03 12:42:37 crc kubenswrapper[5052]: I1203 12:42:37.323720 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/2e4d7dee-0fd0-437e-a529-2dafd03c6492-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:42:37 crc kubenswrapper[5052]: I1203 12:42:37.323733 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2e4d7dee-0fd0-437e-a529-2dafd03c6492-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:42:37 crc kubenswrapper[5052]: I1203 12:42:37.929466 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-db-sync-rgdrt" event={"ID":"2e4d7dee-0fd0-437e-a529-2dafd03c6492","Type":"ContainerDied","Data":"0e3084006b1bb94584f399d013bc851c7d7a22f7a8887d846dab273eb0603e4d"} Dec 03 12:42:37 crc kubenswrapper[5052]: I1203 12:42:37.929539 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0e3084006b1bb94584f399d013bc851c7d7a22f7a8887d846dab273eb0603e4d" Dec 03 12:42:37 crc kubenswrapper[5052]: I1203 12:42:37.930721 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-db-sync-rgdrt" Dec 03 12:42:38 crc kubenswrapper[5052]: I1203 12:42:38.307341 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/keystone-bootstrap-vbrn6"] Dec 03 12:42:38 crc kubenswrapper[5052]: E1203 12:42:38.307767 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="0d36a272-0133-4e61-bb88-219ff124cc91" containerName="mariadb-account-create-update" Dec 03 12:42:38 crc kubenswrapper[5052]: I1203 12:42:38.307784 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="0d36a272-0133-4e61-bb88-219ff124cc91" containerName="mariadb-account-create-update" Dec 03 12:42:38 crc kubenswrapper[5052]: E1203 12:42:38.307805 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="984da95f-b2c9-422b-b09d-6b7f80bf56e1" containerName="mariadb-database-create" Dec 03 12:42:38 crc kubenswrapper[5052]: I1203 12:42:38.307813 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="984da95f-b2c9-422b-b09d-6b7f80bf56e1" containerName="mariadb-database-create" Dec 03 12:42:38 crc kubenswrapper[5052]: E1203 12:42:38.307828 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="555223ae-4781-4c74-8ae0-053e1153ad9f" containerName="mariadb-database-create" Dec 03 12:42:38 crc kubenswrapper[5052]: I1203 12:42:38.307834 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="555223ae-4781-4c74-8ae0-053e1153ad9f" containerName="mariadb-database-create" Dec 03 12:42:38 crc kubenswrapper[5052]: E1203 12:42:38.307851 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="53834739-1c4b-4de1-a946-1a26d9036171" containerName="mariadb-database-create" Dec 03 12:42:38 crc kubenswrapper[5052]: I1203 12:42:38.307857 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="53834739-1c4b-4de1-a946-1a26d9036171" containerName="mariadb-database-create" Dec 03 12:42:38 crc kubenswrapper[5052]: E1203 12:42:38.307872 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="79f5c230-497d-4183-b174-e1a927ffc003" containerName="mariadb-account-create-update" Dec 03 12:42:38 crc kubenswrapper[5052]: I1203 12:42:38.307878 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="79f5c230-497d-4183-b174-e1a927ffc003" containerName="mariadb-account-create-update" Dec 03 12:42:38 crc kubenswrapper[5052]: E1203 12:42:38.307893 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="ec00af82-d7ae-44ad-9023-d88663706469" containerName="mariadb-account-create-update" Dec 03 12:42:38 crc kubenswrapper[5052]: I1203 12:42:38.307899 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="ec00af82-d7ae-44ad-9023-d88663706469" containerName="mariadb-account-create-update" Dec 03 12:42:38 crc kubenswrapper[5052]: E1203 12:42:38.307913 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="2e4d7dee-0fd0-437e-a529-2dafd03c6492" containerName="keystone-db-sync" Dec 03 12:42:38 crc kubenswrapper[5052]: I1203 12:42:38.307921 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="2e4d7dee-0fd0-437e-a529-2dafd03c6492" containerName="keystone-db-sync" Dec 03 12:42:38 crc kubenswrapper[5052]: I1203 12:42:38.308078 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="ec00af82-d7ae-44ad-9023-d88663706469" containerName="mariadb-account-create-update" Dec 03 12:42:38 crc kubenswrapper[5052]: I1203 12:42:38.308098 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="555223ae-4781-4c74-8ae0-053e1153ad9f" containerName="mariadb-database-create" Dec 03 12:42:38 crc kubenswrapper[5052]: I1203 12:42:38.308108 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="0d36a272-0133-4e61-bb88-219ff124cc91" containerName="mariadb-account-create-update" Dec 03 12:42:38 crc kubenswrapper[5052]: I1203 12:42:38.308126 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="984da95f-b2c9-422b-b09d-6b7f80bf56e1" containerName="mariadb-database-create" Dec 03 12:42:38 crc kubenswrapper[5052]: I1203 12:42:38.308137 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="79f5c230-497d-4183-b174-e1a927ffc003" containerName="mariadb-account-create-update" Dec 03 12:42:38 crc kubenswrapper[5052]: I1203 12:42:38.308152 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="53834739-1c4b-4de1-a946-1a26d9036171" containerName="mariadb-database-create" Dec 03 12:42:38 crc kubenswrapper[5052]: I1203 12:42:38.308166 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="2e4d7dee-0fd0-437e-a529-2dafd03c6492" containerName="keystone-db-sync" Dec 03 12:42:38 crc kubenswrapper[5052]: I1203 12:42:38.308791 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-bootstrap-vbrn6" Dec 03 12:42:38 crc kubenswrapper[5052]: I1203 12:42:38.312668 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"osp-secret" Dec 03 12:42:38 crc kubenswrapper[5052]: I1203 12:42:38.312891 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone-scripts" Dec 03 12:42:38 crc kubenswrapper[5052]: I1203 12:42:38.313043 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone" Dec 03 12:42:38 crc kubenswrapper[5052]: I1203 12:42:38.313737 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone-config-data" Dec 03 12:42:38 crc kubenswrapper[5052]: I1203 12:42:38.316314 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone-keystone-dockercfg-2f85l" Dec 03 12:42:38 crc kubenswrapper[5052]: I1203 12:42:38.337022 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-bootstrap-vbrn6"] Dec 03 12:42:38 crc kubenswrapper[5052]: I1203 12:42:38.343457 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/8ed7530c-9333-446a-8972-7eee88f76d82-fernet-keys\") pod \"keystone-bootstrap-vbrn6\" (UID: \"8ed7530c-9333-446a-8972-7eee88f76d82\") " pod="openstack-kuttl-tests/keystone-bootstrap-vbrn6" Dec 03 12:42:38 crc kubenswrapper[5052]: I1203 12:42:38.343553 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8ed7530c-9333-446a-8972-7eee88f76d82-combined-ca-bundle\") pod \"keystone-bootstrap-vbrn6\" (UID: \"8ed7530c-9333-446a-8972-7eee88f76d82\") " pod="openstack-kuttl-tests/keystone-bootstrap-vbrn6" Dec 03 12:42:38 crc kubenswrapper[5052]: I1203 12:42:38.343610 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h7sjd\" (UniqueName: \"kubernetes.io/projected/8ed7530c-9333-446a-8972-7eee88f76d82-kube-api-access-h7sjd\") pod \"keystone-bootstrap-vbrn6\" (UID: \"8ed7530c-9333-446a-8972-7eee88f76d82\") " pod="openstack-kuttl-tests/keystone-bootstrap-vbrn6" Dec 03 12:42:38 crc kubenswrapper[5052]: I1203 12:42:38.343647 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/8ed7530c-9333-446a-8972-7eee88f76d82-credential-keys\") pod \"keystone-bootstrap-vbrn6\" (UID: \"8ed7530c-9333-446a-8972-7eee88f76d82\") " pod="openstack-kuttl-tests/keystone-bootstrap-vbrn6" Dec 03 12:42:38 crc kubenswrapper[5052]: I1203 12:42:38.343693 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/8ed7530c-9333-446a-8972-7eee88f76d82-config-data\") pod \"keystone-bootstrap-vbrn6\" (UID: \"8ed7530c-9333-446a-8972-7eee88f76d82\") " pod="openstack-kuttl-tests/keystone-bootstrap-vbrn6" Dec 03 12:42:38 crc kubenswrapper[5052]: I1203 12:42:38.343730 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/8ed7530c-9333-446a-8972-7eee88f76d82-scripts\") pod \"keystone-bootstrap-vbrn6\" (UID: \"8ed7530c-9333-446a-8972-7eee88f76d82\") " pod="openstack-kuttl-tests/keystone-bootstrap-vbrn6" Dec 03 12:42:38 crc kubenswrapper[5052]: I1203 12:42:38.407827 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:42:38 crc kubenswrapper[5052]: I1203 12:42:38.409539 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:42:38 crc kubenswrapper[5052]: I1203 12:42:38.418861 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-scripts" Dec 03 12:42:38 crc kubenswrapper[5052]: I1203 12:42:38.418869 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-glance-dockercfg-ggt4k" Dec 03 12:42:38 crc kubenswrapper[5052]: I1203 12:42:38.418955 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-default-external-config-data" Dec 03 12:42:38 crc kubenswrapper[5052]: I1203 12:42:38.419011 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-glance-default-public-svc" Dec 03 12:42:38 crc kubenswrapper[5052]: I1203 12:42:38.421099 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:42:38 crc kubenswrapper[5052]: I1203 12:42:38.446775 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/8ed7530c-9333-446a-8972-7eee88f76d82-fernet-keys\") pod \"keystone-bootstrap-vbrn6\" (UID: \"8ed7530c-9333-446a-8972-7eee88f76d82\") " pod="openstack-kuttl-tests/keystone-bootstrap-vbrn6" Dec 03 12:42:38 crc kubenswrapper[5052]: I1203 12:42:38.446847 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8ed7530c-9333-446a-8972-7eee88f76d82-combined-ca-bundle\") pod \"keystone-bootstrap-vbrn6\" (UID: \"8ed7530c-9333-446a-8972-7eee88f76d82\") " pod="openstack-kuttl-tests/keystone-bootstrap-vbrn6" Dec 03 12:42:38 crc kubenswrapper[5052]: I1203 12:42:38.446899 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-h7sjd\" (UniqueName: \"kubernetes.io/projected/8ed7530c-9333-446a-8972-7eee88f76d82-kube-api-access-h7sjd\") pod \"keystone-bootstrap-vbrn6\" (UID: \"8ed7530c-9333-446a-8972-7eee88f76d82\") " pod="openstack-kuttl-tests/keystone-bootstrap-vbrn6" Dec 03 12:42:38 crc kubenswrapper[5052]: I1203 12:42:38.446941 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/8ed7530c-9333-446a-8972-7eee88f76d82-credential-keys\") pod \"keystone-bootstrap-vbrn6\" (UID: \"8ed7530c-9333-446a-8972-7eee88f76d82\") " pod="openstack-kuttl-tests/keystone-bootstrap-vbrn6" Dec 03 12:42:38 crc kubenswrapper[5052]: I1203 12:42:38.446994 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/8ed7530c-9333-446a-8972-7eee88f76d82-config-data\") pod \"keystone-bootstrap-vbrn6\" (UID: \"8ed7530c-9333-446a-8972-7eee88f76d82\") " pod="openstack-kuttl-tests/keystone-bootstrap-vbrn6" Dec 03 12:42:38 crc kubenswrapper[5052]: I1203 12:42:38.447034 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/8ed7530c-9333-446a-8972-7eee88f76d82-scripts\") pod \"keystone-bootstrap-vbrn6\" (UID: \"8ed7530c-9333-446a-8972-7eee88f76d82\") " pod="openstack-kuttl-tests/keystone-bootstrap-vbrn6" Dec 03 12:42:38 crc kubenswrapper[5052]: I1203 12:42:38.453942 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/8ed7530c-9333-446a-8972-7eee88f76d82-credential-keys\") pod \"keystone-bootstrap-vbrn6\" (UID: \"8ed7530c-9333-446a-8972-7eee88f76d82\") " pod="openstack-kuttl-tests/keystone-bootstrap-vbrn6" Dec 03 12:42:38 crc kubenswrapper[5052]: I1203 12:42:38.453968 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8ed7530c-9333-446a-8972-7eee88f76d82-combined-ca-bundle\") pod \"keystone-bootstrap-vbrn6\" (UID: \"8ed7530c-9333-446a-8972-7eee88f76d82\") " pod="openstack-kuttl-tests/keystone-bootstrap-vbrn6" Dec 03 12:42:38 crc kubenswrapper[5052]: I1203 12:42:38.454267 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/8ed7530c-9333-446a-8972-7eee88f76d82-fernet-keys\") pod \"keystone-bootstrap-vbrn6\" (UID: \"8ed7530c-9333-446a-8972-7eee88f76d82\") " pod="openstack-kuttl-tests/keystone-bootstrap-vbrn6" Dec 03 12:42:38 crc kubenswrapper[5052]: I1203 12:42:38.460017 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/8ed7530c-9333-446a-8972-7eee88f76d82-config-data\") pod \"keystone-bootstrap-vbrn6\" (UID: \"8ed7530c-9333-446a-8972-7eee88f76d82\") " pod="openstack-kuttl-tests/keystone-bootstrap-vbrn6" Dec 03 12:42:38 crc kubenswrapper[5052]: I1203 12:42:38.472304 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/8ed7530c-9333-446a-8972-7eee88f76d82-scripts\") pod \"keystone-bootstrap-vbrn6\" (UID: \"8ed7530c-9333-446a-8972-7eee88f76d82\") " pod="openstack-kuttl-tests/keystone-bootstrap-vbrn6" Dec 03 12:42:38 crc kubenswrapper[5052]: I1203 12:42:38.493256 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-h7sjd\" (UniqueName: \"kubernetes.io/projected/8ed7530c-9333-446a-8972-7eee88f76d82-kube-api-access-h7sjd\") pod \"keystone-bootstrap-vbrn6\" (UID: \"8ed7530c-9333-446a-8972-7eee88f76d82\") " pod="openstack-kuttl-tests/keystone-bootstrap-vbrn6" Dec 03 12:42:38 crc kubenswrapper[5052]: I1203 12:42:38.548454 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:42:38 crc kubenswrapper[5052]: I1203 12:42:38.549686 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/15420a4f-e75c-4ca6-940e-7f75f3eaab1d-scripts\") pod \"glance-default-external-api-0\" (UID: \"15420a4f-e75c-4ca6-940e-7f75f3eaab1d\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:42:38 crc kubenswrapper[5052]: I1203 12:42:38.549762 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jncsg\" (UniqueName: \"kubernetes.io/projected/15420a4f-e75c-4ca6-940e-7f75f3eaab1d-kube-api-access-jncsg\") pod \"glance-default-external-api-0\" (UID: \"15420a4f-e75c-4ca6-940e-7f75f3eaab1d\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:42:38 crc kubenswrapper[5052]: I1203 12:42:38.549792 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/15420a4f-e75c-4ca6-940e-7f75f3eaab1d-combined-ca-bundle\") pod \"glance-default-external-api-0\" (UID: \"15420a4f-e75c-4ca6-940e-7f75f3eaab1d\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:42:38 crc kubenswrapper[5052]: I1203 12:42:38.549823 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/15420a4f-e75c-4ca6-940e-7f75f3eaab1d-logs\") pod \"glance-default-external-api-0\" (UID: \"15420a4f-e75c-4ca6-940e-7f75f3eaab1d\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:42:38 crc kubenswrapper[5052]: I1203 12:42:38.549853 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/15420a4f-e75c-4ca6-940e-7f75f3eaab1d-public-tls-certs\") pod \"glance-default-external-api-0\" (UID: \"15420a4f-e75c-4ca6-940e-7f75f3eaab1d\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:42:38 crc kubenswrapper[5052]: I1203 12:42:38.550073 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/15420a4f-e75c-4ca6-940e-7f75f3eaab1d-config-data\") pod \"glance-default-external-api-0\" (UID: \"15420a4f-e75c-4ca6-940e-7f75f3eaab1d\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:42:38 crc kubenswrapper[5052]: I1203 12:42:38.550150 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage12-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage12-crc\") pod \"glance-default-external-api-0\" (UID: \"15420a4f-e75c-4ca6-940e-7f75f3eaab1d\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:42:38 crc kubenswrapper[5052]: I1203 12:42:38.550220 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/15420a4f-e75c-4ca6-940e-7f75f3eaab1d-httpd-run\") pod \"glance-default-external-api-0\" (UID: \"15420a4f-e75c-4ca6-940e-7f75f3eaab1d\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:42:38 crc kubenswrapper[5052]: I1203 12:42:38.550792 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:42:38 crc kubenswrapper[5052]: I1203 12:42:38.553180 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-scripts" Dec 03 12:42:38 crc kubenswrapper[5052]: I1203 12:42:38.553541 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-config-data" Dec 03 12:42:38 crc kubenswrapper[5052]: I1203 12:42:38.570392 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:42:38 crc kubenswrapper[5052]: I1203 12:42:38.572249 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:42:38 crc kubenswrapper[5052]: I1203 12:42:38.578663 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-default-internal-config-data" Dec 03 12:42:38 crc kubenswrapper[5052]: I1203 12:42:38.578972 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-glance-default-internal-svc" Dec 03 12:42:38 crc kubenswrapper[5052]: I1203 12:42:38.589566 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:42:38 crc kubenswrapper[5052]: I1203 12:42:38.613492 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:42:38 crc kubenswrapper[5052]: I1203 12:42:38.635944 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-bootstrap-vbrn6" Dec 03 12:42:38 crc kubenswrapper[5052]: I1203 12:42:38.652716 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/133758ce-c13c-4ac1-b672-49412b22ec08-log-httpd\") pod \"ceilometer-0\" (UID: \"133758ce-c13c-4ac1-b672-49412b22ec08\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:42:38 crc kubenswrapper[5052]: I1203 12:42:38.652778 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/15420a4f-e75c-4ca6-940e-7f75f3eaab1d-scripts\") pod \"glance-default-external-api-0\" (UID: \"15420a4f-e75c-4ca6-940e-7f75f3eaab1d\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:42:38 crc kubenswrapper[5052]: I1203 12:42:38.652827 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-jncsg\" (UniqueName: \"kubernetes.io/projected/15420a4f-e75c-4ca6-940e-7f75f3eaab1d-kube-api-access-jncsg\") pod \"glance-default-external-api-0\" (UID: \"15420a4f-e75c-4ca6-940e-7f75f3eaab1d\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:42:38 crc kubenswrapper[5052]: I1203 12:42:38.652856 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/133758ce-c13c-4ac1-b672-49412b22ec08-config-data\") pod \"ceilometer-0\" (UID: \"133758ce-c13c-4ac1-b672-49412b22ec08\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:42:38 crc kubenswrapper[5052]: I1203 12:42:38.652881 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/15420a4f-e75c-4ca6-940e-7f75f3eaab1d-combined-ca-bundle\") pod \"glance-default-external-api-0\" (UID: \"15420a4f-e75c-4ca6-940e-7f75f3eaab1d\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:42:38 crc kubenswrapper[5052]: I1203 12:42:38.652909 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/15420a4f-e75c-4ca6-940e-7f75f3eaab1d-logs\") pod \"glance-default-external-api-0\" (UID: \"15420a4f-e75c-4ca6-940e-7f75f3eaab1d\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:42:38 crc kubenswrapper[5052]: I1203 12:42:38.652944 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/15420a4f-e75c-4ca6-940e-7f75f3eaab1d-public-tls-certs\") pod \"glance-default-external-api-0\" (UID: \"15420a4f-e75c-4ca6-940e-7f75f3eaab1d\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:42:38 crc kubenswrapper[5052]: I1203 12:42:38.653002 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/133758ce-c13c-4ac1-b672-49412b22ec08-scripts\") pod \"ceilometer-0\" (UID: \"133758ce-c13c-4ac1-b672-49412b22ec08\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:42:38 crc kubenswrapper[5052]: I1203 12:42:38.653038 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/15420a4f-e75c-4ca6-940e-7f75f3eaab1d-config-data\") pod \"glance-default-external-api-0\" (UID: \"15420a4f-e75c-4ca6-940e-7f75f3eaab1d\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:42:38 crc kubenswrapper[5052]: I1203 12:42:38.653064 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage12-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage12-crc\") pod \"glance-default-external-api-0\" (UID: \"15420a4f-e75c-4ca6-940e-7f75f3eaab1d\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:42:38 crc kubenswrapper[5052]: I1203 12:42:38.653094 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/133758ce-c13c-4ac1-b672-49412b22ec08-run-httpd\") pod \"ceilometer-0\" (UID: \"133758ce-c13c-4ac1-b672-49412b22ec08\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:42:38 crc kubenswrapper[5052]: I1203 12:42:38.653115 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xrftx\" (UniqueName: \"kubernetes.io/projected/133758ce-c13c-4ac1-b672-49412b22ec08-kube-api-access-xrftx\") pod \"ceilometer-0\" (UID: \"133758ce-c13c-4ac1-b672-49412b22ec08\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:42:38 crc kubenswrapper[5052]: I1203 12:42:38.653133 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/15420a4f-e75c-4ca6-940e-7f75f3eaab1d-httpd-run\") pod \"glance-default-external-api-0\" (UID: \"15420a4f-e75c-4ca6-940e-7f75f3eaab1d\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:42:38 crc kubenswrapper[5052]: I1203 12:42:38.653158 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/133758ce-c13c-4ac1-b672-49412b22ec08-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"133758ce-c13c-4ac1-b672-49412b22ec08\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:42:38 crc kubenswrapper[5052]: I1203 12:42:38.653185 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/133758ce-c13c-4ac1-b672-49412b22ec08-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"133758ce-c13c-4ac1-b672-49412b22ec08\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:42:38 crc kubenswrapper[5052]: I1203 12:42:38.675151 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/15420a4f-e75c-4ca6-940e-7f75f3eaab1d-httpd-run\") pod \"glance-default-external-api-0\" (UID: \"15420a4f-e75c-4ca6-940e-7f75f3eaab1d\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:42:38 crc kubenswrapper[5052]: I1203 12:42:38.679526 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage12-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage12-crc\") pod \"glance-default-external-api-0\" (UID: \"15420a4f-e75c-4ca6-940e-7f75f3eaab1d\") device mount path \"/mnt/openstack/pv12\"" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:42:38 crc kubenswrapper[5052]: I1203 12:42:38.686317 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/15420a4f-e75c-4ca6-940e-7f75f3eaab1d-logs\") pod \"glance-default-external-api-0\" (UID: \"15420a4f-e75c-4ca6-940e-7f75f3eaab1d\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:42:38 crc kubenswrapper[5052]: I1203 12:42:38.693304 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/15420a4f-e75c-4ca6-940e-7f75f3eaab1d-config-data\") pod \"glance-default-external-api-0\" (UID: \"15420a4f-e75c-4ca6-940e-7f75f3eaab1d\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:42:38 crc kubenswrapper[5052]: I1203 12:42:38.703138 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/15420a4f-e75c-4ca6-940e-7f75f3eaab1d-combined-ca-bundle\") pod \"glance-default-external-api-0\" (UID: \"15420a4f-e75c-4ca6-940e-7f75f3eaab1d\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:42:38 crc kubenswrapper[5052]: I1203 12:42:38.703204 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/placement-db-sync-s7jr9"] Dec 03 12:42:38 crc kubenswrapper[5052]: I1203 12:42:38.704280 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-db-sync-s7jr9" Dec 03 12:42:38 crc kubenswrapper[5052]: I1203 12:42:38.705097 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/15420a4f-e75c-4ca6-940e-7f75f3eaab1d-scripts\") pod \"glance-default-external-api-0\" (UID: \"15420a4f-e75c-4ca6-940e-7f75f3eaab1d\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:42:38 crc kubenswrapper[5052]: I1203 12:42:38.707672 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"placement-scripts" Dec 03 12:42:38 crc kubenswrapper[5052]: I1203 12:42:38.709913 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"placement-config-data" Dec 03 12:42:38 crc kubenswrapper[5052]: I1203 12:42:38.710179 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"placement-placement-dockercfg-qt92n" Dec 03 12:42:38 crc kubenswrapper[5052]: I1203 12:42:38.712378 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-jncsg\" (UniqueName: \"kubernetes.io/projected/15420a4f-e75c-4ca6-940e-7f75f3eaab1d-kube-api-access-jncsg\") pod \"glance-default-external-api-0\" (UID: \"15420a4f-e75c-4ca6-940e-7f75f3eaab1d\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:42:38 crc kubenswrapper[5052]: I1203 12:42:38.716394 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/15420a4f-e75c-4ca6-940e-7f75f3eaab1d-public-tls-certs\") pod \"glance-default-external-api-0\" (UID: \"15420a4f-e75c-4ca6-940e-7f75f3eaab1d\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:42:38 crc kubenswrapper[5052]: I1203 12:42:38.733066 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/placement-db-sync-s7jr9"] Dec 03 12:42:38 crc kubenswrapper[5052]: I1203 12:42:38.758542 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/51d8ed61-508e-4dda-b7d6-7f386c7b357e-logs\") pod \"glance-default-internal-api-0\" (UID: \"51d8ed61-508e-4dda-b7d6-7f386c7b357e\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:42:38 crc kubenswrapper[5052]: I1203 12:42:38.758613 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage18-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage18-crc\") pod \"glance-default-internal-api-0\" (UID: \"51d8ed61-508e-4dda-b7d6-7f386c7b357e\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:42:38 crc kubenswrapper[5052]: I1203 12:42:38.758651 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/133758ce-c13c-4ac1-b672-49412b22ec08-scripts\") pod \"ceilometer-0\" (UID: \"133758ce-c13c-4ac1-b672-49412b22ec08\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:42:38 crc kubenswrapper[5052]: I1203 12:42:38.758708 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/51d8ed61-508e-4dda-b7d6-7f386c7b357e-httpd-run\") pod \"glance-default-internal-api-0\" (UID: \"51d8ed61-508e-4dda-b7d6-7f386c7b357e\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:42:38 crc kubenswrapper[5052]: I1203 12:42:38.759017 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/133758ce-c13c-4ac1-b672-49412b22ec08-run-httpd\") pod \"ceilometer-0\" (UID: \"133758ce-c13c-4ac1-b672-49412b22ec08\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:42:38 crc kubenswrapper[5052]: I1203 12:42:38.759039 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-xrftx\" (UniqueName: \"kubernetes.io/projected/133758ce-c13c-4ac1-b672-49412b22ec08-kube-api-access-xrftx\") pod \"ceilometer-0\" (UID: \"133758ce-c13c-4ac1-b672-49412b22ec08\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:42:38 crc kubenswrapper[5052]: I1203 12:42:38.759069 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/133758ce-c13c-4ac1-b672-49412b22ec08-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"133758ce-c13c-4ac1-b672-49412b22ec08\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:42:38 crc kubenswrapper[5052]: I1203 12:42:38.759086 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/51d8ed61-508e-4dda-b7d6-7f386c7b357e-scripts\") pod \"glance-default-internal-api-0\" (UID: \"51d8ed61-508e-4dda-b7d6-7f386c7b357e\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:42:38 crc kubenswrapper[5052]: I1203 12:42:38.759107 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/133758ce-c13c-4ac1-b672-49412b22ec08-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"133758ce-c13c-4ac1-b672-49412b22ec08\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:42:38 crc kubenswrapper[5052]: I1203 12:42:38.759168 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/133758ce-c13c-4ac1-b672-49412b22ec08-log-httpd\") pod \"ceilometer-0\" (UID: \"133758ce-c13c-4ac1-b672-49412b22ec08\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:42:38 crc kubenswrapper[5052]: I1203 12:42:38.759190 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/51d8ed61-508e-4dda-b7d6-7f386c7b357e-combined-ca-bundle\") pod \"glance-default-internal-api-0\" (UID: \"51d8ed61-508e-4dda-b7d6-7f386c7b357e\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:42:38 crc kubenswrapper[5052]: I1203 12:42:38.759212 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/51d8ed61-508e-4dda-b7d6-7f386c7b357e-internal-tls-certs\") pod \"glance-default-internal-api-0\" (UID: \"51d8ed61-508e-4dda-b7d6-7f386c7b357e\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:42:38 crc kubenswrapper[5052]: I1203 12:42:38.759253 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/133758ce-c13c-4ac1-b672-49412b22ec08-config-data\") pod \"ceilometer-0\" (UID: \"133758ce-c13c-4ac1-b672-49412b22ec08\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:42:38 crc kubenswrapper[5052]: I1203 12:42:38.759284 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s4ltz\" (UniqueName: \"kubernetes.io/projected/51d8ed61-508e-4dda-b7d6-7f386c7b357e-kube-api-access-s4ltz\") pod \"glance-default-internal-api-0\" (UID: \"51d8ed61-508e-4dda-b7d6-7f386c7b357e\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:42:38 crc kubenswrapper[5052]: I1203 12:42:38.759326 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/51d8ed61-508e-4dda-b7d6-7f386c7b357e-config-data\") pod \"glance-default-internal-api-0\" (UID: \"51d8ed61-508e-4dda-b7d6-7f386c7b357e\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:42:38 crc kubenswrapper[5052]: I1203 12:42:38.759811 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/133758ce-c13c-4ac1-b672-49412b22ec08-run-httpd\") pod \"ceilometer-0\" (UID: \"133758ce-c13c-4ac1-b672-49412b22ec08\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:42:38 crc kubenswrapper[5052]: I1203 12:42:38.759974 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/133758ce-c13c-4ac1-b672-49412b22ec08-log-httpd\") pod \"ceilometer-0\" (UID: \"133758ce-c13c-4ac1-b672-49412b22ec08\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:42:38 crc kubenswrapper[5052]: I1203 12:42:38.768336 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage12-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage12-crc\") pod \"glance-default-external-api-0\" (UID: \"15420a4f-e75c-4ca6-940e-7f75f3eaab1d\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:42:38 crc kubenswrapper[5052]: I1203 12:42:38.772307 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/133758ce-c13c-4ac1-b672-49412b22ec08-config-data\") pod \"ceilometer-0\" (UID: \"133758ce-c13c-4ac1-b672-49412b22ec08\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:42:38 crc kubenswrapper[5052]: I1203 12:42:38.778965 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/133758ce-c13c-4ac1-b672-49412b22ec08-scripts\") pod \"ceilometer-0\" (UID: \"133758ce-c13c-4ac1-b672-49412b22ec08\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:42:38 crc kubenswrapper[5052]: I1203 12:42:38.781371 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/133758ce-c13c-4ac1-b672-49412b22ec08-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"133758ce-c13c-4ac1-b672-49412b22ec08\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:42:38 crc kubenswrapper[5052]: I1203 12:42:38.786306 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/133758ce-c13c-4ac1-b672-49412b22ec08-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"133758ce-c13c-4ac1-b672-49412b22ec08\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:42:38 crc kubenswrapper[5052]: I1203 12:42:38.794963 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-xrftx\" (UniqueName: \"kubernetes.io/projected/133758ce-c13c-4ac1-b672-49412b22ec08-kube-api-access-xrftx\") pod \"ceilometer-0\" (UID: \"133758ce-c13c-4ac1-b672-49412b22ec08\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:42:38 crc kubenswrapper[5052]: I1203 12:42:38.847083 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/cinder-db-sync-2ttlg"] Dec 03 12:42:38 crc kubenswrapper[5052]: I1203 12:42:38.848709 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-db-sync-2ttlg" Dec 03 12:42:38 crc kubenswrapper[5052]: I1203 12:42:38.857073 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-db-sync-2ttlg"] Dec 03 12:42:38 crc kubenswrapper[5052]: I1203 12:42:38.886705 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:42:38 crc kubenswrapper[5052]: I1203 12:42:38.886742 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cinder-cinder-dockercfg-wgjw7" Dec 03 12:42:38 crc kubenswrapper[5052]: I1203 12:42:38.886974 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cinder-scripts" Dec 03 12:42:38 crc kubenswrapper[5052]: I1203 12:42:38.887137 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cinder-config-data" Dec 03 12:42:38 crc kubenswrapper[5052]: I1203 12:42:38.887358 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/51d8ed61-508e-4dda-b7d6-7f386c7b357e-combined-ca-bundle\") pod \"glance-default-internal-api-0\" (UID: \"51d8ed61-508e-4dda-b7d6-7f386c7b357e\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:42:38 crc kubenswrapper[5052]: I1203 12:42:38.887391 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/51d8ed61-508e-4dda-b7d6-7f386c7b357e-internal-tls-certs\") pod \"glance-default-internal-api-0\" (UID: \"51d8ed61-508e-4dda-b7d6-7f386c7b357e\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:42:38 crc kubenswrapper[5052]: I1203 12:42:38.887415 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/0063a179-2799-4532-9dd4-789e55b489be-scripts\") pod \"placement-db-sync-s7jr9\" (UID: \"0063a179-2799-4532-9dd4-789e55b489be\") " pod="openstack-kuttl-tests/placement-db-sync-s7jr9" Dec 03 12:42:38 crc kubenswrapper[5052]: I1203 12:42:38.887434 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/0063a179-2799-4532-9dd4-789e55b489be-config-data\") pod \"placement-db-sync-s7jr9\" (UID: \"0063a179-2799-4532-9dd4-789e55b489be\") " pod="openstack-kuttl-tests/placement-db-sync-s7jr9" Dec 03 12:42:38 crc kubenswrapper[5052]: I1203 12:42:38.887470 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-s4ltz\" (UniqueName: \"kubernetes.io/projected/51d8ed61-508e-4dda-b7d6-7f386c7b357e-kube-api-access-s4ltz\") pod \"glance-default-internal-api-0\" (UID: \"51d8ed61-508e-4dda-b7d6-7f386c7b357e\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:42:38 crc kubenswrapper[5052]: I1203 12:42:38.887490 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/0063a179-2799-4532-9dd4-789e55b489be-logs\") pod \"placement-db-sync-s7jr9\" (UID: \"0063a179-2799-4532-9dd4-789e55b489be\") " pod="openstack-kuttl-tests/placement-db-sync-s7jr9" Dec 03 12:42:38 crc kubenswrapper[5052]: I1203 12:42:38.887537 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/51d8ed61-508e-4dda-b7d6-7f386c7b357e-config-data\") pod \"glance-default-internal-api-0\" (UID: \"51d8ed61-508e-4dda-b7d6-7f386c7b357e\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:42:38 crc kubenswrapper[5052]: I1203 12:42:38.887556 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/51d8ed61-508e-4dda-b7d6-7f386c7b357e-logs\") pod \"glance-default-internal-api-0\" (UID: \"51d8ed61-508e-4dda-b7d6-7f386c7b357e\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:42:38 crc kubenswrapper[5052]: I1203 12:42:38.887577 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vsldg\" (UniqueName: \"kubernetes.io/projected/0063a179-2799-4532-9dd4-789e55b489be-kube-api-access-vsldg\") pod \"placement-db-sync-s7jr9\" (UID: \"0063a179-2799-4532-9dd4-789e55b489be\") " pod="openstack-kuttl-tests/placement-db-sync-s7jr9" Dec 03 12:42:38 crc kubenswrapper[5052]: I1203 12:42:38.887600 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage18-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage18-crc\") pod \"glance-default-internal-api-0\" (UID: \"51d8ed61-508e-4dda-b7d6-7f386c7b357e\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:42:38 crc kubenswrapper[5052]: I1203 12:42:38.887625 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0063a179-2799-4532-9dd4-789e55b489be-combined-ca-bundle\") pod \"placement-db-sync-s7jr9\" (UID: \"0063a179-2799-4532-9dd4-789e55b489be\") " pod="openstack-kuttl-tests/placement-db-sync-s7jr9" Dec 03 12:42:38 crc kubenswrapper[5052]: I1203 12:42:38.887671 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/51d8ed61-508e-4dda-b7d6-7f386c7b357e-httpd-run\") pod \"glance-default-internal-api-0\" (UID: \"51d8ed61-508e-4dda-b7d6-7f386c7b357e\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:42:38 crc kubenswrapper[5052]: I1203 12:42:38.894453 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/51d8ed61-508e-4dda-b7d6-7f386c7b357e-scripts\") pod \"glance-default-internal-api-0\" (UID: \"51d8ed61-508e-4dda-b7d6-7f386c7b357e\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:42:38 crc kubenswrapper[5052]: I1203 12:42:38.898685 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/51d8ed61-508e-4dda-b7d6-7f386c7b357e-config-data\") pod \"glance-default-internal-api-0\" (UID: \"51d8ed61-508e-4dda-b7d6-7f386c7b357e\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:42:38 crc kubenswrapper[5052]: I1203 12:42:38.899851 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/51d8ed61-508e-4dda-b7d6-7f386c7b357e-scripts\") pod \"glance-default-internal-api-0\" (UID: \"51d8ed61-508e-4dda-b7d6-7f386c7b357e\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:42:38 crc kubenswrapper[5052]: I1203 12:42:38.902945 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/51d8ed61-508e-4dda-b7d6-7f386c7b357e-logs\") pod \"glance-default-internal-api-0\" (UID: \"51d8ed61-508e-4dda-b7d6-7f386c7b357e\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:42:38 crc kubenswrapper[5052]: I1203 12:42:38.903007 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage18-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage18-crc\") pod \"glance-default-internal-api-0\" (UID: \"51d8ed61-508e-4dda-b7d6-7f386c7b357e\") device mount path \"/mnt/openstack/pv18\"" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:42:38 crc kubenswrapper[5052]: I1203 12:42:38.903487 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/51d8ed61-508e-4dda-b7d6-7f386c7b357e-httpd-run\") pod \"glance-default-internal-api-0\" (UID: \"51d8ed61-508e-4dda-b7d6-7f386c7b357e\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:42:38 crc kubenswrapper[5052]: I1203 12:42:38.907666 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/51d8ed61-508e-4dda-b7d6-7f386c7b357e-combined-ca-bundle\") pod \"glance-default-internal-api-0\" (UID: \"51d8ed61-508e-4dda-b7d6-7f386c7b357e\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:42:38 crc kubenswrapper[5052]: I1203 12:42:38.909834 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/51d8ed61-508e-4dda-b7d6-7f386c7b357e-internal-tls-certs\") pod \"glance-default-internal-api-0\" (UID: \"51d8ed61-508e-4dda-b7d6-7f386c7b357e\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:42:38 crc kubenswrapper[5052]: I1203 12:42:38.939393 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage18-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage18-crc\") pod \"glance-default-internal-api-0\" (UID: \"51d8ed61-508e-4dda-b7d6-7f386c7b357e\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:42:38 crc kubenswrapper[5052]: I1203 12:42:38.942232 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-s4ltz\" (UniqueName: \"kubernetes.io/projected/51d8ed61-508e-4dda-b7d6-7f386c7b357e-kube-api-access-s4ltz\") pod \"glance-default-internal-api-0\" (UID: \"51d8ed61-508e-4dda-b7d6-7f386c7b357e\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:42:39 crc kubenswrapper[5052]: I1203 12:42:39.006848 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1de481f1-0c54-48c3-b112-659497d86792-combined-ca-bundle\") pod \"cinder-db-sync-2ttlg\" (UID: \"1de481f1-0c54-48c3-b112-659497d86792\") " pod="openstack-kuttl-tests/cinder-db-sync-2ttlg" Dec 03 12:42:39 crc kubenswrapper[5052]: I1203 12:42:39.007212 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/1de481f1-0c54-48c3-b112-659497d86792-db-sync-config-data\") pod \"cinder-db-sync-2ttlg\" (UID: \"1de481f1-0c54-48c3-b112-659497d86792\") " pod="openstack-kuttl-tests/cinder-db-sync-2ttlg" Dec 03 12:42:39 crc kubenswrapper[5052]: I1203 12:42:39.007251 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/0063a179-2799-4532-9dd4-789e55b489be-scripts\") pod \"placement-db-sync-s7jr9\" (UID: \"0063a179-2799-4532-9dd4-789e55b489be\") " pod="openstack-kuttl-tests/placement-db-sync-s7jr9" Dec 03 12:42:39 crc kubenswrapper[5052]: I1203 12:42:39.007286 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/0063a179-2799-4532-9dd4-789e55b489be-config-data\") pod \"placement-db-sync-s7jr9\" (UID: \"0063a179-2799-4532-9dd4-789e55b489be\") " pod="openstack-kuttl-tests/placement-db-sync-s7jr9" Dec 03 12:42:39 crc kubenswrapper[5052]: I1203 12:42:39.009041 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/1de481f1-0c54-48c3-b112-659497d86792-scripts\") pod \"cinder-db-sync-2ttlg\" (UID: \"1de481f1-0c54-48c3-b112-659497d86792\") " pod="openstack-kuttl-tests/cinder-db-sync-2ttlg" Dec 03 12:42:39 crc kubenswrapper[5052]: I1203 12:42:39.009083 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4fx5r\" (UniqueName: \"kubernetes.io/projected/1de481f1-0c54-48c3-b112-659497d86792-kube-api-access-4fx5r\") pod \"cinder-db-sync-2ttlg\" (UID: \"1de481f1-0c54-48c3-b112-659497d86792\") " pod="openstack-kuttl-tests/cinder-db-sync-2ttlg" Dec 03 12:42:39 crc kubenswrapper[5052]: I1203 12:42:39.009115 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/1de481f1-0c54-48c3-b112-659497d86792-config-data\") pod \"cinder-db-sync-2ttlg\" (UID: \"1de481f1-0c54-48c3-b112-659497d86792\") " pod="openstack-kuttl-tests/cinder-db-sync-2ttlg" Dec 03 12:42:39 crc kubenswrapper[5052]: I1203 12:42:39.009139 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/0063a179-2799-4532-9dd4-789e55b489be-logs\") pod \"placement-db-sync-s7jr9\" (UID: \"0063a179-2799-4532-9dd4-789e55b489be\") " pod="openstack-kuttl-tests/placement-db-sync-s7jr9" Dec 03 12:42:39 crc kubenswrapper[5052]: I1203 12:42:39.009191 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/1de481f1-0c54-48c3-b112-659497d86792-etc-machine-id\") pod \"cinder-db-sync-2ttlg\" (UID: \"1de481f1-0c54-48c3-b112-659497d86792\") " pod="openstack-kuttl-tests/cinder-db-sync-2ttlg" Dec 03 12:42:39 crc kubenswrapper[5052]: I1203 12:42:39.009221 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-vsldg\" (UniqueName: \"kubernetes.io/projected/0063a179-2799-4532-9dd4-789e55b489be-kube-api-access-vsldg\") pod \"placement-db-sync-s7jr9\" (UID: \"0063a179-2799-4532-9dd4-789e55b489be\") " pod="openstack-kuttl-tests/placement-db-sync-s7jr9" Dec 03 12:42:39 crc kubenswrapper[5052]: I1203 12:42:39.009281 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0063a179-2799-4532-9dd4-789e55b489be-combined-ca-bundle\") pod \"placement-db-sync-s7jr9\" (UID: \"0063a179-2799-4532-9dd4-789e55b489be\") " pod="openstack-kuttl-tests/placement-db-sync-s7jr9" Dec 03 12:42:39 crc kubenswrapper[5052]: I1203 12:42:39.017429 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/0063a179-2799-4532-9dd4-789e55b489be-logs\") pod \"placement-db-sync-s7jr9\" (UID: \"0063a179-2799-4532-9dd4-789e55b489be\") " pod="openstack-kuttl-tests/placement-db-sync-s7jr9" Dec 03 12:42:39 crc kubenswrapper[5052]: I1203 12:42:39.017709 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/0063a179-2799-4532-9dd4-789e55b489be-scripts\") pod \"placement-db-sync-s7jr9\" (UID: \"0063a179-2799-4532-9dd4-789e55b489be\") " pod="openstack-kuttl-tests/placement-db-sync-s7jr9" Dec 03 12:42:39 crc kubenswrapper[5052]: I1203 12:42:39.027105 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/0063a179-2799-4532-9dd4-789e55b489be-config-data\") pod \"placement-db-sync-s7jr9\" (UID: \"0063a179-2799-4532-9dd4-789e55b489be\") " pod="openstack-kuttl-tests/placement-db-sync-s7jr9" Dec 03 12:42:39 crc kubenswrapper[5052]: I1203 12:42:39.042383 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:42:39 crc kubenswrapper[5052]: I1203 12:42:39.066123 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/barbican-db-sync-ccx7c"] Dec 03 12:42:39 crc kubenswrapper[5052]: I1203 12:42:39.084925 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-db-sync-ccx7c" Dec 03 12:42:39 crc kubenswrapper[5052]: I1203 12:42:39.085187 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-vsldg\" (UniqueName: \"kubernetes.io/projected/0063a179-2799-4532-9dd4-789e55b489be-kube-api-access-vsldg\") pod \"placement-db-sync-s7jr9\" (UID: \"0063a179-2799-4532-9dd4-789e55b489be\") " pod="openstack-kuttl-tests/placement-db-sync-s7jr9" Dec 03 12:42:39 crc kubenswrapper[5052]: I1203 12:42:39.090976 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"barbican-barbican-dockercfg-tjssv" Dec 03 12:42:39 crc kubenswrapper[5052]: I1203 12:42:39.091292 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"barbican-config-data" Dec 03 12:42:39 crc kubenswrapper[5052]: I1203 12:42:39.104880 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-db-sync-ccx7c"] Dec 03 12:42:39 crc kubenswrapper[5052]: I1203 12:42:39.111125 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1de481f1-0c54-48c3-b112-659497d86792-combined-ca-bundle\") pod \"cinder-db-sync-2ttlg\" (UID: \"1de481f1-0c54-48c3-b112-659497d86792\") " pod="openstack-kuttl-tests/cinder-db-sync-2ttlg" Dec 03 12:42:39 crc kubenswrapper[5052]: I1203 12:42:39.111171 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/1de481f1-0c54-48c3-b112-659497d86792-db-sync-config-data\") pod \"cinder-db-sync-2ttlg\" (UID: \"1de481f1-0c54-48c3-b112-659497d86792\") " pod="openstack-kuttl-tests/cinder-db-sync-2ttlg" Dec 03 12:42:39 crc kubenswrapper[5052]: I1203 12:42:39.111198 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/1de481f1-0c54-48c3-b112-659497d86792-scripts\") pod \"cinder-db-sync-2ttlg\" (UID: \"1de481f1-0c54-48c3-b112-659497d86792\") " pod="openstack-kuttl-tests/cinder-db-sync-2ttlg" Dec 03 12:42:39 crc kubenswrapper[5052]: I1203 12:42:39.111227 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-4fx5r\" (UniqueName: \"kubernetes.io/projected/1de481f1-0c54-48c3-b112-659497d86792-kube-api-access-4fx5r\") pod \"cinder-db-sync-2ttlg\" (UID: \"1de481f1-0c54-48c3-b112-659497d86792\") " pod="openstack-kuttl-tests/cinder-db-sync-2ttlg" Dec 03 12:42:39 crc kubenswrapper[5052]: I1203 12:42:39.111250 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/1de481f1-0c54-48c3-b112-659497d86792-config-data\") pod \"cinder-db-sync-2ttlg\" (UID: \"1de481f1-0c54-48c3-b112-659497d86792\") " pod="openstack-kuttl-tests/cinder-db-sync-2ttlg" Dec 03 12:42:39 crc kubenswrapper[5052]: I1203 12:42:39.111284 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/1de481f1-0c54-48c3-b112-659497d86792-etc-machine-id\") pod \"cinder-db-sync-2ttlg\" (UID: \"1de481f1-0c54-48c3-b112-659497d86792\") " pod="openstack-kuttl-tests/cinder-db-sync-2ttlg" Dec 03 12:42:39 crc kubenswrapper[5052]: I1203 12:42:39.111426 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/1de481f1-0c54-48c3-b112-659497d86792-etc-machine-id\") pod \"cinder-db-sync-2ttlg\" (UID: \"1de481f1-0c54-48c3-b112-659497d86792\") " pod="openstack-kuttl-tests/cinder-db-sync-2ttlg" Dec 03 12:42:39 crc kubenswrapper[5052]: I1203 12:42:39.113453 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0063a179-2799-4532-9dd4-789e55b489be-combined-ca-bundle\") pod \"placement-db-sync-s7jr9\" (UID: \"0063a179-2799-4532-9dd4-789e55b489be\") " pod="openstack-kuttl-tests/placement-db-sync-s7jr9" Dec 03 12:42:39 crc kubenswrapper[5052]: I1203 12:42:39.120245 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/1de481f1-0c54-48c3-b112-659497d86792-scripts\") pod \"cinder-db-sync-2ttlg\" (UID: \"1de481f1-0c54-48c3-b112-659497d86792\") " pod="openstack-kuttl-tests/cinder-db-sync-2ttlg" Dec 03 12:42:39 crc kubenswrapper[5052]: I1203 12:42:39.121033 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/1de481f1-0c54-48c3-b112-659497d86792-db-sync-config-data\") pod \"cinder-db-sync-2ttlg\" (UID: \"1de481f1-0c54-48c3-b112-659497d86792\") " pod="openstack-kuttl-tests/cinder-db-sync-2ttlg" Dec 03 12:42:39 crc kubenswrapper[5052]: I1203 12:42:39.124705 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/1de481f1-0c54-48c3-b112-659497d86792-config-data\") pod \"cinder-db-sync-2ttlg\" (UID: \"1de481f1-0c54-48c3-b112-659497d86792\") " pod="openstack-kuttl-tests/cinder-db-sync-2ttlg" Dec 03 12:42:39 crc kubenswrapper[5052]: I1203 12:42:39.126079 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1de481f1-0c54-48c3-b112-659497d86792-combined-ca-bundle\") pod \"cinder-db-sync-2ttlg\" (UID: \"1de481f1-0c54-48c3-b112-659497d86792\") " pod="openstack-kuttl-tests/cinder-db-sync-2ttlg" Dec 03 12:42:39 crc kubenswrapper[5052]: I1203 12:42:39.128841 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-db-sync-s7jr9" Dec 03 12:42:39 crc kubenswrapper[5052]: I1203 12:42:39.139623 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-4fx5r\" (UniqueName: \"kubernetes.io/projected/1de481f1-0c54-48c3-b112-659497d86792-kube-api-access-4fx5r\") pod \"cinder-db-sync-2ttlg\" (UID: \"1de481f1-0c54-48c3-b112-659497d86792\") " pod="openstack-kuttl-tests/cinder-db-sync-2ttlg" Dec 03 12:42:39 crc kubenswrapper[5052]: I1203 12:42:39.183626 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/neutron-db-sync-zf7km"] Dec 03 12:42:39 crc kubenswrapper[5052]: I1203 12:42:39.185949 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-db-sync-zf7km" Dec 03 12:42:39 crc kubenswrapper[5052]: I1203 12:42:39.191078 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:42:39 crc kubenswrapper[5052]: I1203 12:42:39.191326 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"neutron-neutron-dockercfg-4zn4q" Dec 03 12:42:39 crc kubenswrapper[5052]: I1203 12:42:39.192024 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"neutron-config" Dec 03 12:42:39 crc kubenswrapper[5052]: I1203 12:42:39.192283 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"neutron-httpd-config" Dec 03 12:42:39 crc kubenswrapper[5052]: I1203 12:42:39.209463 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-db-sync-2ttlg" Dec 03 12:42:39 crc kubenswrapper[5052]: I1203 12:42:39.213496 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/826d1642-afe5-4638-81e5-0d5b046e411e-db-sync-config-data\") pod \"barbican-db-sync-ccx7c\" (UID: \"826d1642-afe5-4638-81e5-0d5b046e411e\") " pod="openstack-kuttl-tests/barbican-db-sync-ccx7c" Dec 03 12:42:39 crc kubenswrapper[5052]: I1203 12:42:39.213645 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ffwjn\" (UniqueName: \"kubernetes.io/projected/826d1642-afe5-4638-81e5-0d5b046e411e-kube-api-access-ffwjn\") pod \"barbican-db-sync-ccx7c\" (UID: \"826d1642-afe5-4638-81e5-0d5b046e411e\") " pod="openstack-kuttl-tests/barbican-db-sync-ccx7c" Dec 03 12:42:39 crc kubenswrapper[5052]: I1203 12:42:39.213754 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/826d1642-afe5-4638-81e5-0d5b046e411e-combined-ca-bundle\") pod \"barbican-db-sync-ccx7c\" (UID: \"826d1642-afe5-4638-81e5-0d5b046e411e\") " pod="openstack-kuttl-tests/barbican-db-sync-ccx7c" Dec 03 12:42:39 crc kubenswrapper[5052]: I1203 12:42:39.213938 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-db-sync-zf7km"] Dec 03 12:42:40 crc kubenswrapper[5052]: I1203 12:42:39.315353 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/826d1642-afe5-4638-81e5-0d5b046e411e-db-sync-config-data\") pod \"barbican-db-sync-ccx7c\" (UID: \"826d1642-afe5-4638-81e5-0d5b046e411e\") " pod="openstack-kuttl-tests/barbican-db-sync-ccx7c" Dec 03 12:42:40 crc kubenswrapper[5052]: I1203 12:42:39.315389 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-ffwjn\" (UniqueName: \"kubernetes.io/projected/826d1642-afe5-4638-81e5-0d5b046e411e-kube-api-access-ffwjn\") pod \"barbican-db-sync-ccx7c\" (UID: \"826d1642-afe5-4638-81e5-0d5b046e411e\") " pod="openstack-kuttl-tests/barbican-db-sync-ccx7c" Dec 03 12:42:40 crc kubenswrapper[5052]: I1203 12:42:39.315422 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/e8677b10-c98b-4d0e-a5d7-7af9fd914f12-config\") pod \"neutron-db-sync-zf7km\" (UID: \"e8677b10-c98b-4d0e-a5d7-7af9fd914f12\") " pod="openstack-kuttl-tests/neutron-db-sync-zf7km" Dec 03 12:42:40 crc kubenswrapper[5052]: I1203 12:42:39.315459 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qs82p\" (UniqueName: \"kubernetes.io/projected/e8677b10-c98b-4d0e-a5d7-7af9fd914f12-kube-api-access-qs82p\") pod \"neutron-db-sync-zf7km\" (UID: \"e8677b10-c98b-4d0e-a5d7-7af9fd914f12\") " pod="openstack-kuttl-tests/neutron-db-sync-zf7km" Dec 03 12:42:40 crc kubenswrapper[5052]: I1203 12:42:39.315542 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/826d1642-afe5-4638-81e5-0d5b046e411e-combined-ca-bundle\") pod \"barbican-db-sync-ccx7c\" (UID: \"826d1642-afe5-4638-81e5-0d5b046e411e\") " pod="openstack-kuttl-tests/barbican-db-sync-ccx7c" Dec 03 12:42:40 crc kubenswrapper[5052]: I1203 12:42:39.315573 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e8677b10-c98b-4d0e-a5d7-7af9fd914f12-combined-ca-bundle\") pod \"neutron-db-sync-zf7km\" (UID: \"e8677b10-c98b-4d0e-a5d7-7af9fd914f12\") " pod="openstack-kuttl-tests/neutron-db-sync-zf7km" Dec 03 12:42:40 crc kubenswrapper[5052]: I1203 12:42:39.322923 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/826d1642-afe5-4638-81e5-0d5b046e411e-combined-ca-bundle\") pod \"barbican-db-sync-ccx7c\" (UID: \"826d1642-afe5-4638-81e5-0d5b046e411e\") " pod="openstack-kuttl-tests/barbican-db-sync-ccx7c" Dec 03 12:42:40 crc kubenswrapper[5052]: I1203 12:42:39.323255 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/826d1642-afe5-4638-81e5-0d5b046e411e-db-sync-config-data\") pod \"barbican-db-sync-ccx7c\" (UID: \"826d1642-afe5-4638-81e5-0d5b046e411e\") " pod="openstack-kuttl-tests/barbican-db-sync-ccx7c" Dec 03 12:42:40 crc kubenswrapper[5052]: I1203 12:42:39.348093 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-ffwjn\" (UniqueName: \"kubernetes.io/projected/826d1642-afe5-4638-81e5-0d5b046e411e-kube-api-access-ffwjn\") pod \"barbican-db-sync-ccx7c\" (UID: \"826d1642-afe5-4638-81e5-0d5b046e411e\") " pod="openstack-kuttl-tests/barbican-db-sync-ccx7c" Dec 03 12:42:40 crc kubenswrapper[5052]: I1203 12:42:39.416724 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/e8677b10-c98b-4d0e-a5d7-7af9fd914f12-config\") pod \"neutron-db-sync-zf7km\" (UID: \"e8677b10-c98b-4d0e-a5d7-7af9fd914f12\") " pod="openstack-kuttl-tests/neutron-db-sync-zf7km" Dec 03 12:42:40 crc kubenswrapper[5052]: I1203 12:42:39.416790 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-qs82p\" (UniqueName: \"kubernetes.io/projected/e8677b10-c98b-4d0e-a5d7-7af9fd914f12-kube-api-access-qs82p\") pod \"neutron-db-sync-zf7km\" (UID: \"e8677b10-c98b-4d0e-a5d7-7af9fd914f12\") " pod="openstack-kuttl-tests/neutron-db-sync-zf7km" Dec 03 12:42:40 crc kubenswrapper[5052]: I1203 12:42:39.416883 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e8677b10-c98b-4d0e-a5d7-7af9fd914f12-combined-ca-bundle\") pod \"neutron-db-sync-zf7km\" (UID: \"e8677b10-c98b-4d0e-a5d7-7af9fd914f12\") " pod="openstack-kuttl-tests/neutron-db-sync-zf7km" Dec 03 12:42:40 crc kubenswrapper[5052]: I1203 12:42:39.426417 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-db-sync-ccx7c" Dec 03 12:42:40 crc kubenswrapper[5052]: I1203 12:42:39.434834 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-qs82p\" (UniqueName: \"kubernetes.io/projected/e8677b10-c98b-4d0e-a5d7-7af9fd914f12-kube-api-access-qs82p\") pod \"neutron-db-sync-zf7km\" (UID: \"e8677b10-c98b-4d0e-a5d7-7af9fd914f12\") " pod="openstack-kuttl-tests/neutron-db-sync-zf7km" Dec 03 12:42:40 crc kubenswrapper[5052]: I1203 12:42:39.438394 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e8677b10-c98b-4d0e-a5d7-7af9fd914f12-combined-ca-bundle\") pod \"neutron-db-sync-zf7km\" (UID: \"e8677b10-c98b-4d0e-a5d7-7af9fd914f12\") " pod="openstack-kuttl-tests/neutron-db-sync-zf7km" Dec 03 12:42:40 crc kubenswrapper[5052]: I1203 12:42:39.445016 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/e8677b10-c98b-4d0e-a5d7-7af9fd914f12-config\") pod \"neutron-db-sync-zf7km\" (UID: \"e8677b10-c98b-4d0e-a5d7-7af9fd914f12\") " pod="openstack-kuttl-tests/neutron-db-sync-zf7km" Dec 03 12:42:40 crc kubenswrapper[5052]: I1203 12:42:39.476221 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-bootstrap-vbrn6"] Dec 03 12:42:40 crc kubenswrapper[5052]: I1203 12:42:39.507172 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-db-sync-zf7km" Dec 03 12:42:40 crc kubenswrapper[5052]: I1203 12:42:39.632488 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:42:40 crc kubenswrapper[5052]: W1203 12:42:39.656890 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod133758ce_c13c_4ac1_b672_49412b22ec08.slice/crio-05cb7f3a97655faff11f433f8381c8e973e0e6f53c1ea19005e152808d972518 WatchSource:0}: Error finding container 05cb7f3a97655faff11f433f8381c8e973e0e6f53c1ea19005e152808d972518: Status 404 returned error can't find the container with id 05cb7f3a97655faff11f433f8381c8e973e0e6f53c1ea19005e152808d972518 Dec 03 12:42:40 crc kubenswrapper[5052]: I1203 12:42:39.999670 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"133758ce-c13c-4ac1-b672-49412b22ec08","Type":"ContainerStarted","Data":"05cb7f3a97655faff11f433f8381c8e973e0e6f53c1ea19005e152808d972518"} Dec 03 12:42:40 crc kubenswrapper[5052]: I1203 12:42:40.001437 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-bootstrap-vbrn6" event={"ID":"8ed7530c-9333-446a-8972-7eee88f76d82","Type":"ContainerStarted","Data":"060ad9813ab6a37e3a1a8a5496ce0ed7a7fb29d84ac18787732f368231785872"} Dec 03 12:42:40 crc kubenswrapper[5052]: I1203 12:42:40.001457 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-bootstrap-vbrn6" event={"ID":"8ed7530c-9333-446a-8972-7eee88f76d82","Type":"ContainerStarted","Data":"1e150c21e0cb658125e78758440c5f44c367f35b7c580faa15b71005233fc368"} Dec 03 12:42:40 crc kubenswrapper[5052]: I1203 12:42:40.023771 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/keystone-bootstrap-vbrn6" podStartSLOduration=2.023753622 podStartE2EDuration="2.023753622s" podCreationTimestamp="2025-12-03 12:42:38 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:42:40.022815615 +0000 UTC m=+4950.611057369" watchObservedRunningTime="2025-12-03 12:42:40.023753622 +0000 UTC m=+4950.611995376" Dec 03 12:42:40 crc kubenswrapper[5052]: I1203 12:42:40.830217 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:42:41 crc kubenswrapper[5052]: I1203 12:42:41.016084 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:42:41 crc kubenswrapper[5052]: I1203 12:42:41.213795 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:42:41 crc kubenswrapper[5052]: I1203 12:42:41.299577 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-db-sync-2ttlg"] Dec 03 12:42:41 crc kubenswrapper[5052]: I1203 12:42:41.315545 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:42:41 crc kubenswrapper[5052]: I1203 12:42:41.331612 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-db-sync-ccx7c"] Dec 03 12:42:41 crc kubenswrapper[5052]: I1203 12:42:41.365811 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/placement-db-sync-s7jr9"] Dec 03 12:42:41 crc kubenswrapper[5052]: I1203 12:42:41.386575 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:42:41 crc kubenswrapper[5052]: I1203 12:42:41.429633 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-db-sync-zf7km"] Dec 03 12:42:42 crc kubenswrapper[5052]: I1203 12:42:42.082110 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-db-sync-ccx7c" event={"ID":"826d1642-afe5-4638-81e5-0d5b046e411e","Type":"ContainerStarted","Data":"265de199d1a82bc77793a76bb9691418b9980aa4b80f4223f554b36dc4b53abe"} Dec 03 12:42:42 crc kubenswrapper[5052]: I1203 12:42:42.082457 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-db-sync-ccx7c" event={"ID":"826d1642-afe5-4638-81e5-0d5b046e411e","Type":"ContainerStarted","Data":"8c3a6d0f976afc5dc9ac2d6fc3beed82dedcc8b47376d90126c785c1e4f60005"} Dec 03 12:42:42 crc kubenswrapper[5052]: I1203 12:42:42.086890 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-db-sync-zf7km" event={"ID":"e8677b10-c98b-4d0e-a5d7-7af9fd914f12","Type":"ContainerStarted","Data":"2268353a2de5ede55fc8582a63a842701e59a57d84ce16fa95b9377d33da7455"} Dec 03 12:42:42 crc kubenswrapper[5052]: I1203 12:42:42.086951 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-db-sync-zf7km" event={"ID":"e8677b10-c98b-4d0e-a5d7-7af9fd914f12","Type":"ContainerStarted","Data":"697fdba4e1d936cce1055e9177022b8dadce7e484cc927341bbbe9b8b8d43b41"} Dec 03 12:42:42 crc kubenswrapper[5052]: I1203 12:42:42.102360 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-db-sync-s7jr9" event={"ID":"0063a179-2799-4532-9dd4-789e55b489be","Type":"ContainerStarted","Data":"55b9e1923f83215ee935eb7dac25724eaea8aa946882abe2860d9d47a8b78111"} Dec 03 12:42:42 crc kubenswrapper[5052]: I1203 12:42:42.102494 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-db-sync-s7jr9" event={"ID":"0063a179-2799-4532-9dd4-789e55b489be","Type":"ContainerStarted","Data":"e9e356dce4c903a415ea6f2ee8aed74416204a0d727693aaefb1d62c2181dc99"} Dec 03 12:42:42 crc kubenswrapper[5052]: I1203 12:42:42.102562 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/barbican-db-sync-ccx7c" podStartSLOduration=4.102545032 podStartE2EDuration="4.102545032s" podCreationTimestamp="2025-12-03 12:42:38 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:42:42.099141175 +0000 UTC m=+4952.687382959" watchObservedRunningTime="2025-12-03 12:42:42.102545032 +0000 UTC m=+4952.690786796" Dec 03 12:42:42 crc kubenswrapper[5052]: I1203 12:42:42.129360 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"51d8ed61-508e-4dda-b7d6-7f386c7b357e","Type":"ContainerStarted","Data":"4a1016ff8c85c11711068892d20083fae13825ae7c2667fceae6ef9b51ce73c8"} Dec 03 12:42:42 crc kubenswrapper[5052]: I1203 12:42:42.139053 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/neutron-db-sync-zf7km" podStartSLOduration=3.139028406 podStartE2EDuration="3.139028406s" podCreationTimestamp="2025-12-03 12:42:39 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:42:42.127132735 +0000 UTC m=+4952.715374509" watchObservedRunningTime="2025-12-03 12:42:42.139028406 +0000 UTC m=+4952.727270180" Dec 03 12:42:42 crc kubenswrapper[5052]: I1203 12:42:42.142154 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-db-sync-2ttlg" event={"ID":"1de481f1-0c54-48c3-b112-659497d86792","Type":"ContainerStarted","Data":"08fac75c2b68b52925dc0dfecb563961552d86edff21aed10a901d020990fa3b"} Dec 03 12:42:42 crc kubenswrapper[5052]: I1203 12:42:42.146592 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"15420a4f-e75c-4ca6-940e-7f75f3eaab1d","Type":"ContainerStarted","Data":"44fbf70126c0eb48a5ca39121294b081deae704e0b67ee89d39d4bd70321e27d"} Dec 03 12:42:42 crc kubenswrapper[5052]: I1203 12:42:42.149670 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/placement-db-sync-s7jr9" podStartSLOduration=4.14965366 podStartE2EDuration="4.14965366s" podCreationTimestamp="2025-12-03 12:42:38 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:42:42.144812721 +0000 UTC m=+4952.733054495" watchObservedRunningTime="2025-12-03 12:42:42.14965366 +0000 UTC m=+4952.737895424" Dec 03 12:42:42 crc kubenswrapper[5052]: I1203 12:42:42.166931 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"133758ce-c13c-4ac1-b672-49412b22ec08","Type":"ContainerStarted","Data":"da6fb06ac7ebc0adff26fc1ccb94fc8314f3ea98f662feb777cf3e824d4d161b"} Dec 03 12:42:43 crc kubenswrapper[5052]: I1203 12:42:43.181542 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"51d8ed61-508e-4dda-b7d6-7f386c7b357e","Type":"ContainerStarted","Data":"b0ef06b9bca3d7fd9fe7560eb0fb2fee5193303a30f4fc5bd2eb8f2a5744ad4a"} Dec 03 12:42:43 crc kubenswrapper[5052]: I1203 12:42:43.189979 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-db-sync-2ttlg" event={"ID":"1de481f1-0c54-48c3-b112-659497d86792","Type":"ContainerStarted","Data":"3c9394ad9a076c5005649d998f0ffa542de55379118fcfb363d27cb846a45d9f"} Dec 03 12:42:43 crc kubenswrapper[5052]: I1203 12:42:43.194763 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"15420a4f-e75c-4ca6-940e-7f75f3eaab1d","Type":"ContainerStarted","Data":"25870e65f0fb50e8eee1e91f5b0915165d53fd9d07bf19e39a3c1360d7e618fd"} Dec 03 12:42:43 crc kubenswrapper[5052]: I1203 12:42:43.200873 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"133758ce-c13c-4ac1-b672-49412b22ec08","Type":"ContainerStarted","Data":"6d5877335791adc8136d132f76bce6689455c186a9abe604beb4d46b25a2646c"} Dec 03 12:42:43 crc kubenswrapper[5052]: I1203 12:42:43.220874 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/cinder-db-sync-2ttlg" podStartSLOduration=5.220855503 podStartE2EDuration="5.220855503s" podCreationTimestamp="2025-12-03 12:42:38 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:42:43.209974682 +0000 UTC m=+4953.798216456" watchObservedRunningTime="2025-12-03 12:42:43.220855503 +0000 UTC m=+4953.809097267" Dec 03 12:42:44 crc kubenswrapper[5052]: I1203 12:42:44.213581 5052 generic.go:334] "Generic (PLEG): container finished" podID="826d1642-afe5-4638-81e5-0d5b046e411e" containerID="265de199d1a82bc77793a76bb9691418b9980aa4b80f4223f554b36dc4b53abe" exitCode=0 Dec 03 12:42:44 crc kubenswrapper[5052]: I1203 12:42:44.213661 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-db-sync-ccx7c" event={"ID":"826d1642-afe5-4638-81e5-0d5b046e411e","Type":"ContainerDied","Data":"265de199d1a82bc77793a76bb9691418b9980aa4b80f4223f554b36dc4b53abe"} Dec 03 12:42:44 crc kubenswrapper[5052]: I1203 12:42:44.220950 5052 generic.go:334] "Generic (PLEG): container finished" podID="0063a179-2799-4532-9dd4-789e55b489be" containerID="55b9e1923f83215ee935eb7dac25724eaea8aa946882abe2860d9d47a8b78111" exitCode=0 Dec 03 12:42:44 crc kubenswrapper[5052]: I1203 12:42:44.221047 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-db-sync-s7jr9" event={"ID":"0063a179-2799-4532-9dd4-789e55b489be","Type":"ContainerDied","Data":"55b9e1923f83215ee935eb7dac25724eaea8aa946882abe2860d9d47a8b78111"} Dec 03 12:42:44 crc kubenswrapper[5052]: I1203 12:42:44.225666 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"51d8ed61-508e-4dda-b7d6-7f386c7b357e","Type":"ContainerStarted","Data":"5c3b9e640af4d71e82338eefd81058b8ac6d8f4d67a62aecc269d9d2bd52df8f"} Dec 03 12:42:44 crc kubenswrapper[5052]: I1203 12:42:44.225885 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-internal-api-0" podUID="51d8ed61-508e-4dda-b7d6-7f386c7b357e" containerName="glance-log" containerID="cri-o://b0ef06b9bca3d7fd9fe7560eb0fb2fee5193303a30f4fc5bd2eb8f2a5744ad4a" gracePeriod=30 Dec 03 12:42:44 crc kubenswrapper[5052]: I1203 12:42:44.225937 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-internal-api-0" podUID="51d8ed61-508e-4dda-b7d6-7f386c7b357e" containerName="glance-httpd" containerID="cri-o://5c3b9e640af4d71e82338eefd81058b8ac6d8f4d67a62aecc269d9d2bd52df8f" gracePeriod=30 Dec 03 12:42:44 crc kubenswrapper[5052]: I1203 12:42:44.231526 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"15420a4f-e75c-4ca6-940e-7f75f3eaab1d","Type":"ContainerStarted","Data":"b0b7fc75b0ed1fde4f2bc7deadd5c375c88355e1fd02f74e7d7be549eb0bc0e7"} Dec 03 12:42:44 crc kubenswrapper[5052]: I1203 12:42:44.231625 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-external-api-0" podUID="15420a4f-e75c-4ca6-940e-7f75f3eaab1d" containerName="glance-httpd" containerID="cri-o://b0b7fc75b0ed1fde4f2bc7deadd5c375c88355e1fd02f74e7d7be549eb0bc0e7" gracePeriod=30 Dec 03 12:42:44 crc kubenswrapper[5052]: I1203 12:42:44.231617 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-external-api-0" podUID="15420a4f-e75c-4ca6-940e-7f75f3eaab1d" containerName="glance-log" containerID="cri-o://25870e65f0fb50e8eee1e91f5b0915165d53fd9d07bf19e39a3c1360d7e618fd" gracePeriod=30 Dec 03 12:42:44 crc kubenswrapper[5052]: I1203 12:42:44.237874 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"133758ce-c13c-4ac1-b672-49412b22ec08","Type":"ContainerStarted","Data":"050e4f507ff648b4d9a847ec84654e743d8a6e6d3a90811c85901f7f8a74f34a"} Dec 03 12:42:44 crc kubenswrapper[5052]: I1203 12:42:44.327947 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/glance-default-internal-api-0" podStartSLOduration=6.327924634 podStartE2EDuration="6.327924634s" podCreationTimestamp="2025-12-03 12:42:38 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:42:44.304895195 +0000 UTC m=+4954.893136979" watchObservedRunningTime="2025-12-03 12:42:44.327924634 +0000 UTC m=+4954.916166398" Dec 03 12:42:44 crc kubenswrapper[5052]: I1203 12:42:44.328796 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/glance-default-external-api-0" podStartSLOduration=6.3287911789999995 podStartE2EDuration="6.328791179s" podCreationTimestamp="2025-12-03 12:42:38 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:42:44.325886706 +0000 UTC m=+4954.914128490" watchObservedRunningTime="2025-12-03 12:42:44.328791179 +0000 UTC m=+4954.917032943" Dec 03 12:42:44 crc kubenswrapper[5052]: I1203 12:42:44.841998 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:42:45 crc kubenswrapper[5052]: I1203 12:42:45.015497 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/51d8ed61-508e-4dda-b7d6-7f386c7b357e-combined-ca-bundle\") pod \"51d8ed61-508e-4dda-b7d6-7f386c7b357e\" (UID: \"51d8ed61-508e-4dda-b7d6-7f386c7b357e\") " Dec 03 12:42:45 crc kubenswrapper[5052]: I1203 12:42:45.015943 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/51d8ed61-508e-4dda-b7d6-7f386c7b357e-internal-tls-certs\") pod \"51d8ed61-508e-4dda-b7d6-7f386c7b357e\" (UID: \"51d8ed61-508e-4dda-b7d6-7f386c7b357e\") " Dec 03 12:42:45 crc kubenswrapper[5052]: I1203 12:42:45.016111 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/51d8ed61-508e-4dda-b7d6-7f386c7b357e-httpd-run\") pod \"51d8ed61-508e-4dda-b7d6-7f386c7b357e\" (UID: \"51d8ed61-508e-4dda-b7d6-7f386c7b357e\") " Dec 03 12:42:45 crc kubenswrapper[5052]: I1203 12:42:45.016234 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/51d8ed61-508e-4dda-b7d6-7f386c7b357e-logs\") pod \"51d8ed61-508e-4dda-b7d6-7f386c7b357e\" (UID: \"51d8ed61-508e-4dda-b7d6-7f386c7b357e\") " Dec 03 12:42:45 crc kubenswrapper[5052]: I1203 12:42:45.016341 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"glance\" (UniqueName: \"kubernetes.io/local-volume/local-storage18-crc\") pod \"51d8ed61-508e-4dda-b7d6-7f386c7b357e\" (UID: \"51d8ed61-508e-4dda-b7d6-7f386c7b357e\") " Dec 03 12:42:45 crc kubenswrapper[5052]: I1203 12:42:45.016433 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/51d8ed61-508e-4dda-b7d6-7f386c7b357e-config-data\") pod \"51d8ed61-508e-4dda-b7d6-7f386c7b357e\" (UID: \"51d8ed61-508e-4dda-b7d6-7f386c7b357e\") " Dec 03 12:42:45 crc kubenswrapper[5052]: I1203 12:42:45.016558 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/51d8ed61-508e-4dda-b7d6-7f386c7b357e-scripts\") pod \"51d8ed61-508e-4dda-b7d6-7f386c7b357e\" (UID: \"51d8ed61-508e-4dda-b7d6-7f386c7b357e\") " Dec 03 12:42:45 crc kubenswrapper[5052]: I1203 12:42:45.016682 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/51d8ed61-508e-4dda-b7d6-7f386c7b357e-httpd-run" (OuterVolumeSpecName: "httpd-run") pod "51d8ed61-508e-4dda-b7d6-7f386c7b357e" (UID: "51d8ed61-508e-4dda-b7d6-7f386c7b357e"). InnerVolumeSpecName "httpd-run". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:42:45 crc kubenswrapper[5052]: I1203 12:42:45.016724 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/51d8ed61-508e-4dda-b7d6-7f386c7b357e-logs" (OuterVolumeSpecName: "logs") pod "51d8ed61-508e-4dda-b7d6-7f386c7b357e" (UID: "51d8ed61-508e-4dda-b7d6-7f386c7b357e"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:42:45 crc kubenswrapper[5052]: I1203 12:42:45.016814 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s4ltz\" (UniqueName: \"kubernetes.io/projected/51d8ed61-508e-4dda-b7d6-7f386c7b357e-kube-api-access-s4ltz\") pod \"51d8ed61-508e-4dda-b7d6-7f386c7b357e\" (UID: \"51d8ed61-508e-4dda-b7d6-7f386c7b357e\") " Dec 03 12:42:45 crc kubenswrapper[5052]: I1203 12:42:45.017310 5052 reconciler_common.go:293] "Volume detached for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/51d8ed61-508e-4dda-b7d6-7f386c7b357e-httpd-run\") on node \"crc\" DevicePath \"\"" Dec 03 12:42:45 crc kubenswrapper[5052]: I1203 12:42:45.017398 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/51d8ed61-508e-4dda-b7d6-7f386c7b357e-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:42:45 crc kubenswrapper[5052]: I1203 12:42:45.022607 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/51d8ed61-508e-4dda-b7d6-7f386c7b357e-kube-api-access-s4ltz" (OuterVolumeSpecName: "kube-api-access-s4ltz") pod "51d8ed61-508e-4dda-b7d6-7f386c7b357e" (UID: "51d8ed61-508e-4dda-b7d6-7f386c7b357e"). InnerVolumeSpecName "kube-api-access-s4ltz". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:42:45 crc kubenswrapper[5052]: I1203 12:42:45.024034 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/51d8ed61-508e-4dda-b7d6-7f386c7b357e-scripts" (OuterVolumeSpecName: "scripts") pod "51d8ed61-508e-4dda-b7d6-7f386c7b357e" (UID: "51d8ed61-508e-4dda-b7d6-7f386c7b357e"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:42:45 crc kubenswrapper[5052]: I1203 12:42:45.054795 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage18-crc" (OuterVolumeSpecName: "glance") pod "51d8ed61-508e-4dda-b7d6-7f386c7b357e" (UID: "51d8ed61-508e-4dda-b7d6-7f386c7b357e"). InnerVolumeSpecName "local-storage18-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 12:42:45 crc kubenswrapper[5052]: I1203 12:42:45.068980 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/51d8ed61-508e-4dda-b7d6-7f386c7b357e-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "51d8ed61-508e-4dda-b7d6-7f386c7b357e" (UID: "51d8ed61-508e-4dda-b7d6-7f386c7b357e"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:42:45 crc kubenswrapper[5052]: I1203 12:42:45.107624 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/51d8ed61-508e-4dda-b7d6-7f386c7b357e-config-data" (OuterVolumeSpecName: "config-data") pod "51d8ed61-508e-4dda-b7d6-7f386c7b357e" (UID: "51d8ed61-508e-4dda-b7d6-7f386c7b357e"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:42:45 crc kubenswrapper[5052]: I1203 12:42:45.117342 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/51d8ed61-508e-4dda-b7d6-7f386c7b357e-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "51d8ed61-508e-4dda-b7d6-7f386c7b357e" (UID: "51d8ed61-508e-4dda-b7d6-7f386c7b357e"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:42:45 crc kubenswrapper[5052]: I1203 12:42:45.119266 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage18-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage18-crc\") on node \"crc\" " Dec 03 12:42:45 crc kubenswrapper[5052]: I1203 12:42:45.119293 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/51d8ed61-508e-4dda-b7d6-7f386c7b357e-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:42:45 crc kubenswrapper[5052]: I1203 12:42:45.119306 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/51d8ed61-508e-4dda-b7d6-7f386c7b357e-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:42:45 crc kubenswrapper[5052]: I1203 12:42:45.119322 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-s4ltz\" (UniqueName: \"kubernetes.io/projected/51d8ed61-508e-4dda-b7d6-7f386c7b357e-kube-api-access-s4ltz\") on node \"crc\" DevicePath \"\"" Dec 03 12:42:45 crc kubenswrapper[5052]: I1203 12:42:45.119334 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/51d8ed61-508e-4dda-b7d6-7f386c7b357e-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:42:45 crc kubenswrapper[5052]: I1203 12:42:45.119346 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/51d8ed61-508e-4dda-b7d6-7f386c7b357e-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:42:45 crc kubenswrapper[5052]: I1203 12:42:45.157080 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage18-crc" (UniqueName: "kubernetes.io/local-volume/local-storage18-crc") on node "crc" Dec 03 12:42:45 crc kubenswrapper[5052]: I1203 12:42:45.213941 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:42:45 crc kubenswrapper[5052]: I1203 12:42:45.221686 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage18-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage18-crc\") on node \"crc\" DevicePath \"\"" Dec 03 12:42:45 crc kubenswrapper[5052]: I1203 12:42:45.326879 5052 generic.go:334] "Generic (PLEG): container finished" podID="15420a4f-e75c-4ca6-940e-7f75f3eaab1d" containerID="b0b7fc75b0ed1fde4f2bc7deadd5c375c88355e1fd02f74e7d7be549eb0bc0e7" exitCode=0 Dec 03 12:42:45 crc kubenswrapper[5052]: I1203 12:42:45.326933 5052 generic.go:334] "Generic (PLEG): container finished" podID="15420a4f-e75c-4ca6-940e-7f75f3eaab1d" containerID="25870e65f0fb50e8eee1e91f5b0915165d53fd9d07bf19e39a3c1360d7e618fd" exitCode=143 Dec 03 12:42:45 crc kubenswrapper[5052]: I1203 12:42:45.327028 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"15420a4f-e75c-4ca6-940e-7f75f3eaab1d","Type":"ContainerDied","Data":"b0b7fc75b0ed1fde4f2bc7deadd5c375c88355e1fd02f74e7d7be549eb0bc0e7"} Dec 03 12:42:45 crc kubenswrapper[5052]: I1203 12:42:45.327063 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"15420a4f-e75c-4ca6-940e-7f75f3eaab1d","Type":"ContainerDied","Data":"25870e65f0fb50e8eee1e91f5b0915165d53fd9d07bf19e39a3c1360d7e618fd"} Dec 03 12:42:45 crc kubenswrapper[5052]: I1203 12:42:45.327075 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"15420a4f-e75c-4ca6-940e-7f75f3eaab1d","Type":"ContainerDied","Data":"44fbf70126c0eb48a5ca39121294b081deae704e0b67ee89d39d4bd70321e27d"} Dec 03 12:42:45 crc kubenswrapper[5052]: I1203 12:42:45.327108 5052 scope.go:117] "RemoveContainer" containerID="b0b7fc75b0ed1fde4f2bc7deadd5c375c88355e1fd02f74e7d7be549eb0bc0e7" Dec 03 12:42:45 crc kubenswrapper[5052]: I1203 12:42:45.327349 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:42:45 crc kubenswrapper[5052]: I1203 12:42:45.329275 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jncsg\" (UniqueName: \"kubernetes.io/projected/15420a4f-e75c-4ca6-940e-7f75f3eaab1d-kube-api-access-jncsg\") pod \"15420a4f-e75c-4ca6-940e-7f75f3eaab1d\" (UID: \"15420a4f-e75c-4ca6-940e-7f75f3eaab1d\") " Dec 03 12:42:45 crc kubenswrapper[5052]: I1203 12:42:45.329323 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"glance\" (UniqueName: \"kubernetes.io/local-volume/local-storage12-crc\") pod \"15420a4f-e75c-4ca6-940e-7f75f3eaab1d\" (UID: \"15420a4f-e75c-4ca6-940e-7f75f3eaab1d\") " Dec 03 12:42:45 crc kubenswrapper[5052]: I1203 12:42:45.329424 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/15420a4f-e75c-4ca6-940e-7f75f3eaab1d-config-data\") pod \"15420a4f-e75c-4ca6-940e-7f75f3eaab1d\" (UID: \"15420a4f-e75c-4ca6-940e-7f75f3eaab1d\") " Dec 03 12:42:45 crc kubenswrapper[5052]: I1203 12:42:45.329468 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/15420a4f-e75c-4ca6-940e-7f75f3eaab1d-logs\") pod \"15420a4f-e75c-4ca6-940e-7f75f3eaab1d\" (UID: \"15420a4f-e75c-4ca6-940e-7f75f3eaab1d\") " Dec 03 12:42:45 crc kubenswrapper[5052]: I1203 12:42:45.329496 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/15420a4f-e75c-4ca6-940e-7f75f3eaab1d-scripts\") pod \"15420a4f-e75c-4ca6-940e-7f75f3eaab1d\" (UID: \"15420a4f-e75c-4ca6-940e-7f75f3eaab1d\") " Dec 03 12:42:45 crc kubenswrapper[5052]: I1203 12:42:45.329568 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/15420a4f-e75c-4ca6-940e-7f75f3eaab1d-combined-ca-bundle\") pod \"15420a4f-e75c-4ca6-940e-7f75f3eaab1d\" (UID: \"15420a4f-e75c-4ca6-940e-7f75f3eaab1d\") " Dec 03 12:42:45 crc kubenswrapper[5052]: I1203 12:42:45.329619 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/15420a4f-e75c-4ca6-940e-7f75f3eaab1d-httpd-run\") pod \"15420a4f-e75c-4ca6-940e-7f75f3eaab1d\" (UID: \"15420a4f-e75c-4ca6-940e-7f75f3eaab1d\") " Dec 03 12:42:45 crc kubenswrapper[5052]: I1203 12:42:45.331302 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/15420a4f-e75c-4ca6-940e-7f75f3eaab1d-logs" (OuterVolumeSpecName: "logs") pod "15420a4f-e75c-4ca6-940e-7f75f3eaab1d" (UID: "15420a4f-e75c-4ca6-940e-7f75f3eaab1d"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:42:45 crc kubenswrapper[5052]: I1203 12:42:45.331441 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/15420a4f-e75c-4ca6-940e-7f75f3eaab1d-public-tls-certs\") pod \"15420a4f-e75c-4ca6-940e-7f75f3eaab1d\" (UID: \"15420a4f-e75c-4ca6-940e-7f75f3eaab1d\") " Dec 03 12:42:45 crc kubenswrapper[5052]: I1203 12:42:45.332111 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/15420a4f-e75c-4ca6-940e-7f75f3eaab1d-httpd-run" (OuterVolumeSpecName: "httpd-run") pod "15420a4f-e75c-4ca6-940e-7f75f3eaab1d" (UID: "15420a4f-e75c-4ca6-940e-7f75f3eaab1d"). InnerVolumeSpecName "httpd-run". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:42:45 crc kubenswrapper[5052]: I1203 12:42:45.385951 5052 generic.go:334] "Generic (PLEG): container finished" podID="8ed7530c-9333-446a-8972-7eee88f76d82" containerID="060ad9813ab6a37e3a1a8a5496ce0ed7a7fb29d84ac18787732f368231785872" exitCode=0 Dec 03 12:42:45 crc kubenswrapper[5052]: I1203 12:42:45.386107 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-bootstrap-vbrn6" event={"ID":"8ed7530c-9333-446a-8972-7eee88f76d82","Type":"ContainerDied","Data":"060ad9813ab6a37e3a1a8a5496ce0ed7a7fb29d84ac18787732f368231785872"} Dec 03 12:42:45 crc kubenswrapper[5052]: I1203 12:42:45.386628 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/15420a4f-e75c-4ca6-940e-7f75f3eaab1d-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:42:45 crc kubenswrapper[5052]: I1203 12:42:45.386647 5052 reconciler_common.go:293] "Volume detached for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/15420a4f-e75c-4ca6-940e-7f75f3eaab1d-httpd-run\") on node \"crc\" DevicePath \"\"" Dec 03 12:42:45 crc kubenswrapper[5052]: I1203 12:42:45.387350 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage12-crc" (OuterVolumeSpecName: "glance") pod "15420a4f-e75c-4ca6-940e-7f75f3eaab1d" (UID: "15420a4f-e75c-4ca6-940e-7f75f3eaab1d"). InnerVolumeSpecName "local-storage12-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 12:42:45 crc kubenswrapper[5052]: I1203 12:42:45.388034 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/15420a4f-e75c-4ca6-940e-7f75f3eaab1d-kube-api-access-jncsg" (OuterVolumeSpecName: "kube-api-access-jncsg") pod "15420a4f-e75c-4ca6-940e-7f75f3eaab1d" (UID: "15420a4f-e75c-4ca6-940e-7f75f3eaab1d"). InnerVolumeSpecName "kube-api-access-jncsg". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:42:45 crc kubenswrapper[5052]: I1203 12:42:45.388170 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/15420a4f-e75c-4ca6-940e-7f75f3eaab1d-scripts" (OuterVolumeSpecName: "scripts") pod "15420a4f-e75c-4ca6-940e-7f75f3eaab1d" (UID: "15420a4f-e75c-4ca6-940e-7f75f3eaab1d"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:42:45 crc kubenswrapper[5052]: I1203 12:42:45.429105 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/15420a4f-e75c-4ca6-940e-7f75f3eaab1d-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "15420a4f-e75c-4ca6-940e-7f75f3eaab1d" (UID: "15420a4f-e75c-4ca6-940e-7f75f3eaab1d"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:42:45 crc kubenswrapper[5052]: I1203 12:42:45.432604 5052 scope.go:117] "RemoveContainer" containerID="25870e65f0fb50e8eee1e91f5b0915165d53fd9d07bf19e39a3c1360d7e618fd" Dec 03 12:42:45 crc kubenswrapper[5052]: I1203 12:42:45.432869 5052 generic.go:334] "Generic (PLEG): container finished" podID="51d8ed61-508e-4dda-b7d6-7f386c7b357e" containerID="5c3b9e640af4d71e82338eefd81058b8ac6d8f4d67a62aecc269d9d2bd52df8f" exitCode=143 Dec 03 12:42:45 crc kubenswrapper[5052]: I1203 12:42:45.432888 5052 generic.go:334] "Generic (PLEG): container finished" podID="51d8ed61-508e-4dda-b7d6-7f386c7b357e" containerID="b0ef06b9bca3d7fd9fe7560eb0fb2fee5193303a30f4fc5bd2eb8f2a5744ad4a" exitCode=143 Dec 03 12:42:45 crc kubenswrapper[5052]: I1203 12:42:45.433151 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:42:45 crc kubenswrapper[5052]: I1203 12:42:45.434601 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"51d8ed61-508e-4dda-b7d6-7f386c7b357e","Type":"ContainerDied","Data":"5c3b9e640af4d71e82338eefd81058b8ac6d8f4d67a62aecc269d9d2bd52df8f"} Dec 03 12:42:45 crc kubenswrapper[5052]: I1203 12:42:45.434665 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"51d8ed61-508e-4dda-b7d6-7f386c7b357e","Type":"ContainerDied","Data":"b0ef06b9bca3d7fd9fe7560eb0fb2fee5193303a30f4fc5bd2eb8f2a5744ad4a"} Dec 03 12:42:45 crc kubenswrapper[5052]: I1203 12:42:45.434785 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"51d8ed61-508e-4dda-b7d6-7f386c7b357e","Type":"ContainerDied","Data":"4a1016ff8c85c11711068892d20083fae13825ae7c2667fceae6ef9b51ce73c8"} Dec 03 12:42:45 crc kubenswrapper[5052]: I1203 12:42:45.464264 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/15420a4f-e75c-4ca6-940e-7f75f3eaab1d-config-data" (OuterVolumeSpecName: "config-data") pod "15420a4f-e75c-4ca6-940e-7f75f3eaab1d" (UID: "15420a4f-e75c-4ca6-940e-7f75f3eaab1d"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:42:45 crc kubenswrapper[5052]: I1203 12:42:45.487931 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-jncsg\" (UniqueName: \"kubernetes.io/projected/15420a4f-e75c-4ca6-940e-7f75f3eaab1d-kube-api-access-jncsg\") on node \"crc\" DevicePath \"\"" Dec 03 12:42:45 crc kubenswrapper[5052]: I1203 12:42:45.488274 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage12-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage12-crc\") on node \"crc\" " Dec 03 12:42:45 crc kubenswrapper[5052]: I1203 12:42:45.488286 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/15420a4f-e75c-4ca6-940e-7f75f3eaab1d-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:42:45 crc kubenswrapper[5052]: I1203 12:42:45.488295 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/15420a4f-e75c-4ca6-940e-7f75f3eaab1d-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:42:45 crc kubenswrapper[5052]: I1203 12:42:45.488306 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/15420a4f-e75c-4ca6-940e-7f75f3eaab1d-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:42:45 crc kubenswrapper[5052]: I1203 12:42:45.503939 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/15420a4f-e75c-4ca6-940e-7f75f3eaab1d-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "15420a4f-e75c-4ca6-940e-7f75f3eaab1d" (UID: "15420a4f-e75c-4ca6-940e-7f75f3eaab1d"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:42:45 crc kubenswrapper[5052]: I1203 12:42:45.516800 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage12-crc" (UniqueName: "kubernetes.io/local-volume/local-storage12-crc") on node "crc" Dec 03 12:42:45 crc kubenswrapper[5052]: I1203 12:42:45.541733 5052 scope.go:117] "RemoveContainer" containerID="b0b7fc75b0ed1fde4f2bc7deadd5c375c88355e1fd02f74e7d7be549eb0bc0e7" Dec 03 12:42:45 crc kubenswrapper[5052]: E1203 12:42:45.548749 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b0b7fc75b0ed1fde4f2bc7deadd5c375c88355e1fd02f74e7d7be549eb0bc0e7\": container with ID starting with b0b7fc75b0ed1fde4f2bc7deadd5c375c88355e1fd02f74e7d7be549eb0bc0e7 not found: ID does not exist" containerID="b0b7fc75b0ed1fde4f2bc7deadd5c375c88355e1fd02f74e7d7be549eb0bc0e7" Dec 03 12:42:45 crc kubenswrapper[5052]: I1203 12:42:45.548795 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b0b7fc75b0ed1fde4f2bc7deadd5c375c88355e1fd02f74e7d7be549eb0bc0e7"} err="failed to get container status \"b0b7fc75b0ed1fde4f2bc7deadd5c375c88355e1fd02f74e7d7be549eb0bc0e7\": rpc error: code = NotFound desc = could not find container \"b0b7fc75b0ed1fde4f2bc7deadd5c375c88355e1fd02f74e7d7be549eb0bc0e7\": container with ID starting with b0b7fc75b0ed1fde4f2bc7deadd5c375c88355e1fd02f74e7d7be549eb0bc0e7 not found: ID does not exist" Dec 03 12:42:45 crc kubenswrapper[5052]: I1203 12:42:45.548824 5052 scope.go:117] "RemoveContainer" containerID="25870e65f0fb50e8eee1e91f5b0915165d53fd9d07bf19e39a3c1360d7e618fd" Dec 03 12:42:45 crc kubenswrapper[5052]: E1203 12:42:45.552932 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"25870e65f0fb50e8eee1e91f5b0915165d53fd9d07bf19e39a3c1360d7e618fd\": container with ID starting with 25870e65f0fb50e8eee1e91f5b0915165d53fd9d07bf19e39a3c1360d7e618fd not found: ID does not exist" containerID="25870e65f0fb50e8eee1e91f5b0915165d53fd9d07bf19e39a3c1360d7e618fd" Dec 03 12:42:45 crc kubenswrapper[5052]: I1203 12:42:45.552959 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"25870e65f0fb50e8eee1e91f5b0915165d53fd9d07bf19e39a3c1360d7e618fd"} err="failed to get container status \"25870e65f0fb50e8eee1e91f5b0915165d53fd9d07bf19e39a3c1360d7e618fd\": rpc error: code = NotFound desc = could not find container \"25870e65f0fb50e8eee1e91f5b0915165d53fd9d07bf19e39a3c1360d7e618fd\": container with ID starting with 25870e65f0fb50e8eee1e91f5b0915165d53fd9d07bf19e39a3c1360d7e618fd not found: ID does not exist" Dec 03 12:42:45 crc kubenswrapper[5052]: I1203 12:42:45.552987 5052 scope.go:117] "RemoveContainer" containerID="b0b7fc75b0ed1fde4f2bc7deadd5c375c88355e1fd02f74e7d7be549eb0bc0e7" Dec 03 12:42:45 crc kubenswrapper[5052]: I1203 12:42:45.554082 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b0b7fc75b0ed1fde4f2bc7deadd5c375c88355e1fd02f74e7d7be549eb0bc0e7"} err="failed to get container status \"b0b7fc75b0ed1fde4f2bc7deadd5c375c88355e1fd02f74e7d7be549eb0bc0e7\": rpc error: code = NotFound desc = could not find container \"b0b7fc75b0ed1fde4f2bc7deadd5c375c88355e1fd02f74e7d7be549eb0bc0e7\": container with ID starting with b0b7fc75b0ed1fde4f2bc7deadd5c375c88355e1fd02f74e7d7be549eb0bc0e7 not found: ID does not exist" Dec 03 12:42:45 crc kubenswrapper[5052]: I1203 12:42:45.554106 5052 scope.go:117] "RemoveContainer" containerID="25870e65f0fb50e8eee1e91f5b0915165d53fd9d07bf19e39a3c1360d7e618fd" Dec 03 12:42:45 crc kubenswrapper[5052]: I1203 12:42:45.554264 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"25870e65f0fb50e8eee1e91f5b0915165d53fd9d07bf19e39a3c1360d7e618fd"} err="failed to get container status \"25870e65f0fb50e8eee1e91f5b0915165d53fd9d07bf19e39a3c1360d7e618fd\": rpc error: code = NotFound desc = could not find container \"25870e65f0fb50e8eee1e91f5b0915165d53fd9d07bf19e39a3c1360d7e618fd\": container with ID starting with 25870e65f0fb50e8eee1e91f5b0915165d53fd9d07bf19e39a3c1360d7e618fd not found: ID does not exist" Dec 03 12:42:45 crc kubenswrapper[5052]: I1203 12:42:45.554286 5052 scope.go:117] "RemoveContainer" containerID="5c3b9e640af4d71e82338eefd81058b8ac6d8f4d67a62aecc269d9d2bd52df8f" Dec 03 12:42:45 crc kubenswrapper[5052]: I1203 12:42:45.568736 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:42:45 crc kubenswrapper[5052]: I1203 12:42:45.588574 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:42:45 crc kubenswrapper[5052]: I1203 12:42:45.589668 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/15420a4f-e75c-4ca6-940e-7f75f3eaab1d-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:42:45 crc kubenswrapper[5052]: I1203 12:42:45.589702 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage12-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage12-crc\") on node \"crc\" DevicePath \"\"" Dec 03 12:42:45 crc kubenswrapper[5052]: I1203 12:42:45.595282 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:42:45 crc kubenswrapper[5052]: E1203 12:42:45.595834 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="15420a4f-e75c-4ca6-940e-7f75f3eaab1d" containerName="glance-httpd" Dec 03 12:42:45 crc kubenswrapper[5052]: I1203 12:42:45.595858 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="15420a4f-e75c-4ca6-940e-7f75f3eaab1d" containerName="glance-httpd" Dec 03 12:42:45 crc kubenswrapper[5052]: E1203 12:42:45.595903 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="51d8ed61-508e-4dda-b7d6-7f386c7b357e" containerName="glance-log" Dec 03 12:42:45 crc kubenswrapper[5052]: I1203 12:42:45.595911 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="51d8ed61-508e-4dda-b7d6-7f386c7b357e" containerName="glance-log" Dec 03 12:42:45 crc kubenswrapper[5052]: E1203 12:42:45.595927 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="51d8ed61-508e-4dda-b7d6-7f386c7b357e" containerName="glance-httpd" Dec 03 12:42:45 crc kubenswrapper[5052]: I1203 12:42:45.595934 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="51d8ed61-508e-4dda-b7d6-7f386c7b357e" containerName="glance-httpd" Dec 03 12:42:45 crc kubenswrapper[5052]: E1203 12:42:45.595952 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="15420a4f-e75c-4ca6-940e-7f75f3eaab1d" containerName="glance-log" Dec 03 12:42:45 crc kubenswrapper[5052]: I1203 12:42:45.595961 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="15420a4f-e75c-4ca6-940e-7f75f3eaab1d" containerName="glance-log" Dec 03 12:42:45 crc kubenswrapper[5052]: I1203 12:42:45.596148 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="15420a4f-e75c-4ca6-940e-7f75f3eaab1d" containerName="glance-log" Dec 03 12:42:45 crc kubenswrapper[5052]: I1203 12:42:45.596172 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="51d8ed61-508e-4dda-b7d6-7f386c7b357e" containerName="glance-log" Dec 03 12:42:45 crc kubenswrapper[5052]: I1203 12:42:45.596189 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="15420a4f-e75c-4ca6-940e-7f75f3eaab1d" containerName="glance-httpd" Dec 03 12:42:45 crc kubenswrapper[5052]: I1203 12:42:45.596209 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="51d8ed61-508e-4dda-b7d6-7f386c7b357e" containerName="glance-httpd" Dec 03 12:42:45 crc kubenswrapper[5052]: I1203 12:42:45.597417 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:42:45 crc kubenswrapper[5052]: I1203 12:42:45.604647 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-glance-default-internal-svc" Dec 03 12:42:45 crc kubenswrapper[5052]: I1203 12:42:45.605082 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-default-internal-config-data" Dec 03 12:42:45 crc kubenswrapper[5052]: I1203 12:42:45.643787 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:42:45 crc kubenswrapper[5052]: I1203 12:42:45.670276 5052 scope.go:117] "RemoveContainer" containerID="b0ef06b9bca3d7fd9fe7560eb0fb2fee5193303a30f4fc5bd2eb8f2a5744ad4a" Dec 03 12:42:45 crc kubenswrapper[5052]: I1203 12:42:45.692672 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/8011917c-0ce8-4c83-b157-2d8538790e17-logs\") pod \"glance-default-internal-api-0\" (UID: \"8011917c-0ce8-4c83-b157-2d8538790e17\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:42:45 crc kubenswrapper[5052]: I1203 12:42:45.702843 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/8011917c-0ce8-4c83-b157-2d8538790e17-httpd-run\") pod \"glance-default-internal-api-0\" (UID: \"8011917c-0ce8-4c83-b157-2d8538790e17\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:42:45 crc kubenswrapper[5052]: I1203 12:42:45.703129 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/8011917c-0ce8-4c83-b157-2d8538790e17-config-data\") pod \"glance-default-internal-api-0\" (UID: \"8011917c-0ce8-4c83-b157-2d8538790e17\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:42:45 crc kubenswrapper[5052]: I1203 12:42:45.703276 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xkvtd\" (UniqueName: \"kubernetes.io/projected/8011917c-0ce8-4c83-b157-2d8538790e17-kube-api-access-xkvtd\") pod \"glance-default-internal-api-0\" (UID: \"8011917c-0ce8-4c83-b157-2d8538790e17\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:42:45 crc kubenswrapper[5052]: I1203 12:42:45.724742 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage18-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage18-crc\") pod \"glance-default-internal-api-0\" (UID: \"8011917c-0ce8-4c83-b157-2d8538790e17\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:42:45 crc kubenswrapper[5052]: I1203 12:42:45.724884 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/8011917c-0ce8-4c83-b157-2d8538790e17-internal-tls-certs\") pod \"glance-default-internal-api-0\" (UID: \"8011917c-0ce8-4c83-b157-2d8538790e17\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:42:45 crc kubenswrapper[5052]: I1203 12:42:45.725041 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/8011917c-0ce8-4c83-b157-2d8538790e17-scripts\") pod \"glance-default-internal-api-0\" (UID: \"8011917c-0ce8-4c83-b157-2d8538790e17\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:42:45 crc kubenswrapper[5052]: I1203 12:42:45.725192 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8011917c-0ce8-4c83-b157-2d8538790e17-combined-ca-bundle\") pod \"glance-default-internal-api-0\" (UID: \"8011917c-0ce8-4c83-b157-2d8538790e17\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:42:45 crc kubenswrapper[5052]: I1203 12:42:45.722223 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="51d8ed61-508e-4dda-b7d6-7f386c7b357e" path="/var/lib/kubelet/pods/51d8ed61-508e-4dda-b7d6-7f386c7b357e/volumes" Dec 03 12:42:45 crc kubenswrapper[5052]: I1203 12:42:45.742635 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:42:45 crc kubenswrapper[5052]: I1203 12:42:45.765604 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:42:45 crc kubenswrapper[5052]: I1203 12:42:45.780920 5052 scope.go:117] "RemoveContainer" containerID="5c3b9e640af4d71e82338eefd81058b8ac6d8f4d67a62aecc269d9d2bd52df8f" Dec 03 12:42:45 crc kubenswrapper[5052]: E1203 12:42:45.785674 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5c3b9e640af4d71e82338eefd81058b8ac6d8f4d67a62aecc269d9d2bd52df8f\": container with ID starting with 5c3b9e640af4d71e82338eefd81058b8ac6d8f4d67a62aecc269d9d2bd52df8f not found: ID does not exist" containerID="5c3b9e640af4d71e82338eefd81058b8ac6d8f4d67a62aecc269d9d2bd52df8f" Dec 03 12:42:45 crc kubenswrapper[5052]: I1203 12:42:45.785739 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5c3b9e640af4d71e82338eefd81058b8ac6d8f4d67a62aecc269d9d2bd52df8f"} err="failed to get container status \"5c3b9e640af4d71e82338eefd81058b8ac6d8f4d67a62aecc269d9d2bd52df8f\": rpc error: code = NotFound desc = could not find container \"5c3b9e640af4d71e82338eefd81058b8ac6d8f4d67a62aecc269d9d2bd52df8f\": container with ID starting with 5c3b9e640af4d71e82338eefd81058b8ac6d8f4d67a62aecc269d9d2bd52df8f not found: ID does not exist" Dec 03 12:42:45 crc kubenswrapper[5052]: I1203 12:42:45.785772 5052 scope.go:117] "RemoveContainer" containerID="b0ef06b9bca3d7fd9fe7560eb0fb2fee5193303a30f4fc5bd2eb8f2a5744ad4a" Dec 03 12:42:45 crc kubenswrapper[5052]: E1203 12:42:45.788987 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b0ef06b9bca3d7fd9fe7560eb0fb2fee5193303a30f4fc5bd2eb8f2a5744ad4a\": container with ID starting with b0ef06b9bca3d7fd9fe7560eb0fb2fee5193303a30f4fc5bd2eb8f2a5744ad4a not found: ID does not exist" containerID="b0ef06b9bca3d7fd9fe7560eb0fb2fee5193303a30f4fc5bd2eb8f2a5744ad4a" Dec 03 12:42:45 crc kubenswrapper[5052]: I1203 12:42:45.789030 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b0ef06b9bca3d7fd9fe7560eb0fb2fee5193303a30f4fc5bd2eb8f2a5744ad4a"} err="failed to get container status \"b0ef06b9bca3d7fd9fe7560eb0fb2fee5193303a30f4fc5bd2eb8f2a5744ad4a\": rpc error: code = NotFound desc = could not find container \"b0ef06b9bca3d7fd9fe7560eb0fb2fee5193303a30f4fc5bd2eb8f2a5744ad4a\": container with ID starting with b0ef06b9bca3d7fd9fe7560eb0fb2fee5193303a30f4fc5bd2eb8f2a5744ad4a not found: ID does not exist" Dec 03 12:42:45 crc kubenswrapper[5052]: I1203 12:42:45.789055 5052 scope.go:117] "RemoveContainer" containerID="5c3b9e640af4d71e82338eefd81058b8ac6d8f4d67a62aecc269d9d2bd52df8f" Dec 03 12:42:45 crc kubenswrapper[5052]: I1203 12:42:45.806332 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5c3b9e640af4d71e82338eefd81058b8ac6d8f4d67a62aecc269d9d2bd52df8f"} err="failed to get container status \"5c3b9e640af4d71e82338eefd81058b8ac6d8f4d67a62aecc269d9d2bd52df8f\": rpc error: code = NotFound desc = could not find container \"5c3b9e640af4d71e82338eefd81058b8ac6d8f4d67a62aecc269d9d2bd52df8f\": container with ID starting with 5c3b9e640af4d71e82338eefd81058b8ac6d8f4d67a62aecc269d9d2bd52df8f not found: ID does not exist" Dec 03 12:42:45 crc kubenswrapper[5052]: I1203 12:42:45.806372 5052 scope.go:117] "RemoveContainer" containerID="b0ef06b9bca3d7fd9fe7560eb0fb2fee5193303a30f4fc5bd2eb8f2a5744ad4a" Dec 03 12:42:45 crc kubenswrapper[5052]: I1203 12:42:45.809577 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:42:45 crc kubenswrapper[5052]: I1203 12:42:45.809969 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b0ef06b9bca3d7fd9fe7560eb0fb2fee5193303a30f4fc5bd2eb8f2a5744ad4a"} err="failed to get container status \"b0ef06b9bca3d7fd9fe7560eb0fb2fee5193303a30f4fc5bd2eb8f2a5744ad4a\": rpc error: code = NotFound desc = could not find container \"b0ef06b9bca3d7fd9fe7560eb0fb2fee5193303a30f4fc5bd2eb8f2a5744ad4a\": container with ID starting with b0ef06b9bca3d7fd9fe7560eb0fb2fee5193303a30f4fc5bd2eb8f2a5744ad4a not found: ID does not exist" Dec 03 12:42:45 crc kubenswrapper[5052]: I1203 12:42:45.811489 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:42:45 crc kubenswrapper[5052]: I1203 12:42:45.818033 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-glance-default-public-svc" Dec 03 12:42:45 crc kubenswrapper[5052]: I1203 12:42:45.823824 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-default-external-config-data" Dec 03 12:42:45 crc kubenswrapper[5052]: I1203 12:42:45.832720 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage18-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage18-crc\") pod \"glance-default-internal-api-0\" (UID: \"8011917c-0ce8-4c83-b157-2d8538790e17\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:42:45 crc kubenswrapper[5052]: I1203 12:42:45.832778 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/8011917c-0ce8-4c83-b157-2d8538790e17-internal-tls-certs\") pod \"glance-default-internal-api-0\" (UID: \"8011917c-0ce8-4c83-b157-2d8538790e17\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:42:45 crc kubenswrapper[5052]: I1203 12:42:45.832830 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/8011917c-0ce8-4c83-b157-2d8538790e17-scripts\") pod \"glance-default-internal-api-0\" (UID: \"8011917c-0ce8-4c83-b157-2d8538790e17\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:42:45 crc kubenswrapper[5052]: I1203 12:42:45.832905 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8011917c-0ce8-4c83-b157-2d8538790e17-combined-ca-bundle\") pod \"glance-default-internal-api-0\" (UID: \"8011917c-0ce8-4c83-b157-2d8538790e17\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:42:45 crc kubenswrapper[5052]: I1203 12:42:45.832920 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage18-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage18-crc\") pod \"glance-default-internal-api-0\" (UID: \"8011917c-0ce8-4c83-b157-2d8538790e17\") device mount path \"/mnt/openstack/pv18\"" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:42:45 crc kubenswrapper[5052]: I1203 12:42:45.834329 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/8011917c-0ce8-4c83-b157-2d8538790e17-logs\") pod \"glance-default-internal-api-0\" (UID: \"8011917c-0ce8-4c83-b157-2d8538790e17\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:42:45 crc kubenswrapper[5052]: I1203 12:42:45.834472 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/8011917c-0ce8-4c83-b157-2d8538790e17-httpd-run\") pod \"glance-default-internal-api-0\" (UID: \"8011917c-0ce8-4c83-b157-2d8538790e17\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:42:45 crc kubenswrapper[5052]: I1203 12:42:45.834642 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/8011917c-0ce8-4c83-b157-2d8538790e17-config-data\") pod \"glance-default-internal-api-0\" (UID: \"8011917c-0ce8-4c83-b157-2d8538790e17\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:42:45 crc kubenswrapper[5052]: I1203 12:42:45.834744 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-xkvtd\" (UniqueName: \"kubernetes.io/projected/8011917c-0ce8-4c83-b157-2d8538790e17-kube-api-access-xkvtd\") pod \"glance-default-internal-api-0\" (UID: \"8011917c-0ce8-4c83-b157-2d8538790e17\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:42:45 crc kubenswrapper[5052]: I1203 12:42:45.835032 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/8011917c-0ce8-4c83-b157-2d8538790e17-logs\") pod \"glance-default-internal-api-0\" (UID: \"8011917c-0ce8-4c83-b157-2d8538790e17\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:42:45 crc kubenswrapper[5052]: I1203 12:42:45.835070 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/8011917c-0ce8-4c83-b157-2d8538790e17-httpd-run\") pod \"glance-default-internal-api-0\" (UID: \"8011917c-0ce8-4c83-b157-2d8538790e17\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:42:45 crc kubenswrapper[5052]: I1203 12:42:45.847266 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/8011917c-0ce8-4c83-b157-2d8538790e17-config-data\") pod \"glance-default-internal-api-0\" (UID: \"8011917c-0ce8-4c83-b157-2d8538790e17\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:42:45 crc kubenswrapper[5052]: I1203 12:42:45.849043 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/8011917c-0ce8-4c83-b157-2d8538790e17-internal-tls-certs\") pod \"glance-default-internal-api-0\" (UID: \"8011917c-0ce8-4c83-b157-2d8538790e17\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:42:45 crc kubenswrapper[5052]: I1203 12:42:45.855303 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/8011917c-0ce8-4c83-b157-2d8538790e17-scripts\") pod \"glance-default-internal-api-0\" (UID: \"8011917c-0ce8-4c83-b157-2d8538790e17\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:42:45 crc kubenswrapper[5052]: I1203 12:42:45.856351 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8011917c-0ce8-4c83-b157-2d8538790e17-combined-ca-bundle\") pod \"glance-default-internal-api-0\" (UID: \"8011917c-0ce8-4c83-b157-2d8538790e17\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:42:45 crc kubenswrapper[5052]: I1203 12:42:45.867415 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-xkvtd\" (UniqueName: \"kubernetes.io/projected/8011917c-0ce8-4c83-b157-2d8538790e17-kube-api-access-xkvtd\") pod \"glance-default-internal-api-0\" (UID: \"8011917c-0ce8-4c83-b157-2d8538790e17\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:42:45 crc kubenswrapper[5052]: I1203 12:42:45.879483 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:42:45 crc kubenswrapper[5052]: I1203 12:42:45.882448 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage18-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage18-crc\") pod \"glance-default-internal-api-0\" (UID: \"8011917c-0ce8-4c83-b157-2d8538790e17\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:42:45 crc kubenswrapper[5052]: I1203 12:42:45.924980 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:42:45 crc kubenswrapper[5052]: I1203 12:42:45.936458 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage12-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage12-crc\") pod \"glance-default-external-api-0\" (UID: \"24ccc68c-26a9-485e-bf26-512dac006b61\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:42:45 crc kubenswrapper[5052]: I1203 12:42:45.940041 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/24ccc68c-26a9-485e-bf26-512dac006b61-scripts\") pod \"glance-default-external-api-0\" (UID: \"24ccc68c-26a9-485e-bf26-512dac006b61\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:42:45 crc kubenswrapper[5052]: I1203 12:42:45.940149 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/24ccc68c-26a9-485e-bf26-512dac006b61-httpd-run\") pod \"glance-default-external-api-0\" (UID: \"24ccc68c-26a9-485e-bf26-512dac006b61\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:42:45 crc kubenswrapper[5052]: I1203 12:42:45.940176 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/24ccc68c-26a9-485e-bf26-512dac006b61-public-tls-certs\") pod \"glance-default-external-api-0\" (UID: \"24ccc68c-26a9-485e-bf26-512dac006b61\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:42:45 crc kubenswrapper[5052]: I1203 12:42:45.940213 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/24ccc68c-26a9-485e-bf26-512dac006b61-config-data\") pod \"glance-default-external-api-0\" (UID: \"24ccc68c-26a9-485e-bf26-512dac006b61\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:42:45 crc kubenswrapper[5052]: I1203 12:42:45.940240 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/24ccc68c-26a9-485e-bf26-512dac006b61-combined-ca-bundle\") pod \"glance-default-external-api-0\" (UID: \"24ccc68c-26a9-485e-bf26-512dac006b61\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:42:45 crc kubenswrapper[5052]: I1203 12:42:45.940306 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/24ccc68c-26a9-485e-bf26-512dac006b61-logs\") pod \"glance-default-external-api-0\" (UID: \"24ccc68c-26a9-485e-bf26-512dac006b61\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:42:45 crc kubenswrapper[5052]: I1203 12:42:45.940356 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jgr7h\" (UniqueName: \"kubernetes.io/projected/24ccc68c-26a9-485e-bf26-512dac006b61-kube-api-access-jgr7h\") pod \"glance-default-external-api-0\" (UID: \"24ccc68c-26a9-485e-bf26-512dac006b61\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:42:46 crc kubenswrapper[5052]: I1203 12:42:46.041582 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/24ccc68c-26a9-485e-bf26-512dac006b61-scripts\") pod \"glance-default-external-api-0\" (UID: \"24ccc68c-26a9-485e-bf26-512dac006b61\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:42:46 crc kubenswrapper[5052]: I1203 12:42:46.041651 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/24ccc68c-26a9-485e-bf26-512dac006b61-httpd-run\") pod \"glance-default-external-api-0\" (UID: \"24ccc68c-26a9-485e-bf26-512dac006b61\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:42:46 crc kubenswrapper[5052]: I1203 12:42:46.041680 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/24ccc68c-26a9-485e-bf26-512dac006b61-public-tls-certs\") pod \"glance-default-external-api-0\" (UID: \"24ccc68c-26a9-485e-bf26-512dac006b61\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:42:46 crc kubenswrapper[5052]: I1203 12:42:46.041708 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/24ccc68c-26a9-485e-bf26-512dac006b61-config-data\") pod \"glance-default-external-api-0\" (UID: \"24ccc68c-26a9-485e-bf26-512dac006b61\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:42:46 crc kubenswrapper[5052]: I1203 12:42:46.041731 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/24ccc68c-26a9-485e-bf26-512dac006b61-combined-ca-bundle\") pod \"glance-default-external-api-0\" (UID: \"24ccc68c-26a9-485e-bf26-512dac006b61\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:42:46 crc kubenswrapper[5052]: I1203 12:42:46.041770 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/24ccc68c-26a9-485e-bf26-512dac006b61-logs\") pod \"glance-default-external-api-0\" (UID: \"24ccc68c-26a9-485e-bf26-512dac006b61\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:42:46 crc kubenswrapper[5052]: I1203 12:42:46.041805 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-jgr7h\" (UniqueName: \"kubernetes.io/projected/24ccc68c-26a9-485e-bf26-512dac006b61-kube-api-access-jgr7h\") pod \"glance-default-external-api-0\" (UID: \"24ccc68c-26a9-485e-bf26-512dac006b61\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:42:46 crc kubenswrapper[5052]: I1203 12:42:46.041893 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage12-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage12-crc\") pod \"glance-default-external-api-0\" (UID: \"24ccc68c-26a9-485e-bf26-512dac006b61\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:42:46 crc kubenswrapper[5052]: I1203 12:42:46.042236 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage12-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage12-crc\") pod \"glance-default-external-api-0\" (UID: \"24ccc68c-26a9-485e-bf26-512dac006b61\") device mount path \"/mnt/openstack/pv12\"" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:42:46 crc kubenswrapper[5052]: I1203 12:42:46.047619 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/24ccc68c-26a9-485e-bf26-512dac006b61-logs\") pod \"glance-default-external-api-0\" (UID: \"24ccc68c-26a9-485e-bf26-512dac006b61\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:42:46 crc kubenswrapper[5052]: I1203 12:42:46.047956 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/24ccc68c-26a9-485e-bf26-512dac006b61-httpd-run\") pod \"glance-default-external-api-0\" (UID: \"24ccc68c-26a9-485e-bf26-512dac006b61\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:42:46 crc kubenswrapper[5052]: I1203 12:42:46.057336 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/24ccc68c-26a9-485e-bf26-512dac006b61-scripts\") pod \"glance-default-external-api-0\" (UID: \"24ccc68c-26a9-485e-bf26-512dac006b61\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:42:46 crc kubenswrapper[5052]: I1203 12:42:46.060460 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/24ccc68c-26a9-485e-bf26-512dac006b61-public-tls-certs\") pod \"glance-default-external-api-0\" (UID: \"24ccc68c-26a9-485e-bf26-512dac006b61\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:42:46 crc kubenswrapper[5052]: I1203 12:42:46.064543 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/24ccc68c-26a9-485e-bf26-512dac006b61-combined-ca-bundle\") pod \"glance-default-external-api-0\" (UID: \"24ccc68c-26a9-485e-bf26-512dac006b61\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:42:46 crc kubenswrapper[5052]: I1203 12:42:46.064857 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/24ccc68c-26a9-485e-bf26-512dac006b61-config-data\") pod \"glance-default-external-api-0\" (UID: \"24ccc68c-26a9-485e-bf26-512dac006b61\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:42:46 crc kubenswrapper[5052]: I1203 12:42:46.069930 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage12-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage12-crc\") pod \"glance-default-external-api-0\" (UID: \"24ccc68c-26a9-485e-bf26-512dac006b61\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:42:46 crc kubenswrapper[5052]: I1203 12:42:46.097475 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-jgr7h\" (UniqueName: \"kubernetes.io/projected/24ccc68c-26a9-485e-bf26-512dac006b61-kube-api-access-jgr7h\") pod \"glance-default-external-api-0\" (UID: \"24ccc68c-26a9-485e-bf26-512dac006b61\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:42:46 crc kubenswrapper[5052]: I1203 12:42:46.350936 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-db-sync-s7jr9" Dec 03 12:42:46 crc kubenswrapper[5052]: I1203 12:42:46.396815 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-db-sync-ccx7c" Dec 03 12:42:46 crc kubenswrapper[5052]: I1203 12:42:46.397099 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:42:46 crc kubenswrapper[5052]: I1203 12:42:46.453574 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vsldg\" (UniqueName: \"kubernetes.io/projected/0063a179-2799-4532-9dd4-789e55b489be-kube-api-access-vsldg\") pod \"0063a179-2799-4532-9dd4-789e55b489be\" (UID: \"0063a179-2799-4532-9dd4-789e55b489be\") " Dec 03 12:42:46 crc kubenswrapper[5052]: I1203 12:42:46.453672 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/0063a179-2799-4532-9dd4-789e55b489be-logs\") pod \"0063a179-2799-4532-9dd4-789e55b489be\" (UID: \"0063a179-2799-4532-9dd4-789e55b489be\") " Dec 03 12:42:46 crc kubenswrapper[5052]: I1203 12:42:46.453712 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/0063a179-2799-4532-9dd4-789e55b489be-config-data\") pod \"0063a179-2799-4532-9dd4-789e55b489be\" (UID: \"0063a179-2799-4532-9dd4-789e55b489be\") " Dec 03 12:42:46 crc kubenswrapper[5052]: I1203 12:42:46.453816 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/826d1642-afe5-4638-81e5-0d5b046e411e-db-sync-config-data\") pod \"826d1642-afe5-4638-81e5-0d5b046e411e\" (UID: \"826d1642-afe5-4638-81e5-0d5b046e411e\") " Dec 03 12:42:46 crc kubenswrapper[5052]: I1203 12:42:46.453954 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ffwjn\" (UniqueName: \"kubernetes.io/projected/826d1642-afe5-4638-81e5-0d5b046e411e-kube-api-access-ffwjn\") pod \"826d1642-afe5-4638-81e5-0d5b046e411e\" (UID: \"826d1642-afe5-4638-81e5-0d5b046e411e\") " Dec 03 12:42:46 crc kubenswrapper[5052]: I1203 12:42:46.453984 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0063a179-2799-4532-9dd4-789e55b489be-combined-ca-bundle\") pod \"0063a179-2799-4532-9dd4-789e55b489be\" (UID: \"0063a179-2799-4532-9dd4-789e55b489be\") " Dec 03 12:42:46 crc kubenswrapper[5052]: I1203 12:42:46.454018 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/0063a179-2799-4532-9dd4-789e55b489be-scripts\") pod \"0063a179-2799-4532-9dd4-789e55b489be\" (UID: \"0063a179-2799-4532-9dd4-789e55b489be\") " Dec 03 12:42:46 crc kubenswrapper[5052]: I1203 12:42:46.454044 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/826d1642-afe5-4638-81e5-0d5b046e411e-combined-ca-bundle\") pod \"826d1642-afe5-4638-81e5-0d5b046e411e\" (UID: \"826d1642-afe5-4638-81e5-0d5b046e411e\") " Dec 03 12:42:46 crc kubenswrapper[5052]: I1203 12:42:46.455251 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0063a179-2799-4532-9dd4-789e55b489be-logs" (OuterVolumeSpecName: "logs") pod "0063a179-2799-4532-9dd4-789e55b489be" (UID: "0063a179-2799-4532-9dd4-789e55b489be"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:42:46 crc kubenswrapper[5052]: I1203 12:42:46.460799 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0063a179-2799-4532-9dd4-789e55b489be-kube-api-access-vsldg" (OuterVolumeSpecName: "kube-api-access-vsldg") pod "0063a179-2799-4532-9dd4-789e55b489be" (UID: "0063a179-2799-4532-9dd4-789e55b489be"). InnerVolumeSpecName "kube-api-access-vsldg". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:42:46 crc kubenswrapper[5052]: I1203 12:42:46.460884 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/826d1642-afe5-4638-81e5-0d5b046e411e-kube-api-access-ffwjn" (OuterVolumeSpecName: "kube-api-access-ffwjn") pod "826d1642-afe5-4638-81e5-0d5b046e411e" (UID: "826d1642-afe5-4638-81e5-0d5b046e411e"). InnerVolumeSpecName "kube-api-access-ffwjn". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:42:46 crc kubenswrapper[5052]: I1203 12:42:46.464201 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/826d1642-afe5-4638-81e5-0d5b046e411e-db-sync-config-data" (OuterVolumeSpecName: "db-sync-config-data") pod "826d1642-afe5-4638-81e5-0d5b046e411e" (UID: "826d1642-afe5-4638-81e5-0d5b046e411e"). InnerVolumeSpecName "db-sync-config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:42:46 crc kubenswrapper[5052]: I1203 12:42:46.486436 5052 generic.go:334] "Generic (PLEG): container finished" podID="e8677b10-c98b-4d0e-a5d7-7af9fd914f12" containerID="2268353a2de5ede55fc8582a63a842701e59a57d84ce16fa95b9377d33da7455" exitCode=0 Dec 03 12:42:46 crc kubenswrapper[5052]: I1203 12:42:46.486751 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-db-sync-zf7km" event={"ID":"e8677b10-c98b-4d0e-a5d7-7af9fd914f12","Type":"ContainerDied","Data":"2268353a2de5ede55fc8582a63a842701e59a57d84ce16fa95b9377d33da7455"} Dec 03 12:42:46 crc kubenswrapper[5052]: I1203 12:42:46.487075 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0063a179-2799-4532-9dd4-789e55b489be-scripts" (OuterVolumeSpecName: "scripts") pod "0063a179-2799-4532-9dd4-789e55b489be" (UID: "0063a179-2799-4532-9dd4-789e55b489be"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:42:46 crc kubenswrapper[5052]: I1203 12:42:46.489477 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-db-sync-s7jr9" event={"ID":"0063a179-2799-4532-9dd4-789e55b489be","Type":"ContainerDied","Data":"e9e356dce4c903a415ea6f2ee8aed74416204a0d727693aaefb1d62c2181dc99"} Dec 03 12:42:46 crc kubenswrapper[5052]: I1203 12:42:46.489517 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e9e356dce4c903a415ea6f2ee8aed74416204a0d727693aaefb1d62c2181dc99" Dec 03 12:42:46 crc kubenswrapper[5052]: I1203 12:42:46.489567 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-db-sync-s7jr9" Dec 03 12:42:46 crc kubenswrapper[5052]: I1203 12:42:46.491222 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/826d1642-afe5-4638-81e5-0d5b046e411e-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "826d1642-afe5-4638-81e5-0d5b046e411e" (UID: "826d1642-afe5-4638-81e5-0d5b046e411e"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:42:46 crc kubenswrapper[5052]: I1203 12:42:46.501688 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0063a179-2799-4532-9dd4-789e55b489be-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "0063a179-2799-4532-9dd4-789e55b489be" (UID: "0063a179-2799-4532-9dd4-789e55b489be"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:42:46 crc kubenswrapper[5052]: I1203 12:42:46.504777 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0063a179-2799-4532-9dd4-789e55b489be-config-data" (OuterVolumeSpecName: "config-data") pod "0063a179-2799-4532-9dd4-789e55b489be" (UID: "0063a179-2799-4532-9dd4-789e55b489be"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:42:46 crc kubenswrapper[5052]: I1203 12:42:46.531894 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"133758ce-c13c-4ac1-b672-49412b22ec08","Type":"ContainerStarted","Data":"8ecebefb263fcd4c52ad89050bdb294b17d15a1d16e9932448c14d8cd8f988b1"} Dec 03 12:42:46 crc kubenswrapper[5052]: I1203 12:42:46.532116 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="133758ce-c13c-4ac1-b672-49412b22ec08" containerName="ceilometer-central-agent" containerID="cri-o://da6fb06ac7ebc0adff26fc1ccb94fc8314f3ea98f662feb777cf3e824d4d161b" gracePeriod=30 Dec 03 12:42:46 crc kubenswrapper[5052]: I1203 12:42:46.532419 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:42:46 crc kubenswrapper[5052]: I1203 12:42:46.532705 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="133758ce-c13c-4ac1-b672-49412b22ec08" containerName="proxy-httpd" containerID="cri-o://8ecebefb263fcd4c52ad89050bdb294b17d15a1d16e9932448c14d8cd8f988b1" gracePeriod=30 Dec 03 12:42:46 crc kubenswrapper[5052]: I1203 12:42:46.532768 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="133758ce-c13c-4ac1-b672-49412b22ec08" containerName="sg-core" containerID="cri-o://050e4f507ff648b4d9a847ec84654e743d8a6e6d3a90811c85901f7f8a74f34a" gracePeriod=30 Dec 03 12:42:46 crc kubenswrapper[5052]: I1203 12:42:46.532823 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="133758ce-c13c-4ac1-b672-49412b22ec08" containerName="ceilometer-notification-agent" containerID="cri-o://6d5877335791adc8136d132f76bce6689455c186a9abe604beb4d46b25a2646c" gracePeriod=30 Dec 03 12:42:46 crc kubenswrapper[5052]: I1203 12:42:46.537846 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-db-sync-ccx7c" Dec 03 12:42:46 crc kubenswrapper[5052]: I1203 12:42:46.538671 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-db-sync-ccx7c" event={"ID":"826d1642-afe5-4638-81e5-0d5b046e411e","Type":"ContainerDied","Data":"8c3a6d0f976afc5dc9ac2d6fc3beed82dedcc8b47376d90126c785c1e4f60005"} Dec 03 12:42:46 crc kubenswrapper[5052]: I1203 12:42:46.538704 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8c3a6d0f976afc5dc9ac2d6fc3beed82dedcc8b47376d90126c785c1e4f60005" Dec 03 12:42:46 crc kubenswrapper[5052]: I1203 12:42:46.559044 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-ffwjn\" (UniqueName: \"kubernetes.io/projected/826d1642-afe5-4638-81e5-0d5b046e411e-kube-api-access-ffwjn\") on node \"crc\" DevicePath \"\"" Dec 03 12:42:46 crc kubenswrapper[5052]: I1203 12:42:46.559075 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0063a179-2799-4532-9dd4-789e55b489be-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:42:46 crc kubenswrapper[5052]: I1203 12:42:46.559087 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/0063a179-2799-4532-9dd4-789e55b489be-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:42:46 crc kubenswrapper[5052]: I1203 12:42:46.559099 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/826d1642-afe5-4638-81e5-0d5b046e411e-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:42:46 crc kubenswrapper[5052]: I1203 12:42:46.559110 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-vsldg\" (UniqueName: \"kubernetes.io/projected/0063a179-2799-4532-9dd4-789e55b489be-kube-api-access-vsldg\") on node \"crc\" DevicePath \"\"" Dec 03 12:42:46 crc kubenswrapper[5052]: I1203 12:42:46.559120 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/0063a179-2799-4532-9dd4-789e55b489be-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:42:46 crc kubenswrapper[5052]: I1203 12:42:46.559131 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/0063a179-2799-4532-9dd4-789e55b489be-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:42:46 crc kubenswrapper[5052]: I1203 12:42:46.559144 5052 reconciler_common.go:293] "Volume detached for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/826d1642-afe5-4638-81e5-0d5b046e411e-db-sync-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:42:46 crc kubenswrapper[5052]: I1203 12:42:46.577500 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/ceilometer-0" podStartSLOduration=2.742182988 podStartE2EDuration="8.577477677s" podCreationTimestamp="2025-12-03 12:42:38 +0000 UTC" firstStartedPulling="2025-12-03 12:42:39.659352301 +0000 UTC m=+4950.247594065" lastFinishedPulling="2025-12-03 12:42:45.49464699 +0000 UTC m=+4956.082888754" observedRunningTime="2025-12-03 12:42:46.564543127 +0000 UTC m=+4957.152784881" watchObservedRunningTime="2025-12-03 12:42:46.577477677 +0000 UTC m=+4957.165719441" Dec 03 12:42:46 crc kubenswrapper[5052]: I1203 12:42:46.673761 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:42:47 crc kubenswrapper[5052]: I1203 12:42:47.009949 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-bootstrap-vbrn6" Dec 03 12:42:47 crc kubenswrapper[5052]: I1203 12:42:47.075095 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:42:47 crc kubenswrapper[5052]: I1203 12:42:47.109012 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8ed7530c-9333-446a-8972-7eee88f76d82-combined-ca-bundle\") pod \"8ed7530c-9333-446a-8972-7eee88f76d82\" (UID: \"8ed7530c-9333-446a-8972-7eee88f76d82\") " Dec 03 12:42:47 crc kubenswrapper[5052]: I1203 12:42:47.109220 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/8ed7530c-9333-446a-8972-7eee88f76d82-credential-keys\") pod \"8ed7530c-9333-446a-8972-7eee88f76d82\" (UID: \"8ed7530c-9333-446a-8972-7eee88f76d82\") " Dec 03 12:42:47 crc kubenswrapper[5052]: I1203 12:42:47.109378 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/8ed7530c-9333-446a-8972-7eee88f76d82-fernet-keys\") pod \"8ed7530c-9333-446a-8972-7eee88f76d82\" (UID: \"8ed7530c-9333-446a-8972-7eee88f76d82\") " Dec 03 12:42:47 crc kubenswrapper[5052]: I1203 12:42:47.109416 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h7sjd\" (UniqueName: \"kubernetes.io/projected/8ed7530c-9333-446a-8972-7eee88f76d82-kube-api-access-h7sjd\") pod \"8ed7530c-9333-446a-8972-7eee88f76d82\" (UID: \"8ed7530c-9333-446a-8972-7eee88f76d82\") " Dec 03 12:42:47 crc kubenswrapper[5052]: I1203 12:42:47.109503 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/8ed7530c-9333-446a-8972-7eee88f76d82-scripts\") pod \"8ed7530c-9333-446a-8972-7eee88f76d82\" (UID: \"8ed7530c-9333-446a-8972-7eee88f76d82\") " Dec 03 12:42:47 crc kubenswrapper[5052]: I1203 12:42:47.109550 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/8ed7530c-9333-446a-8972-7eee88f76d82-config-data\") pod \"8ed7530c-9333-446a-8972-7eee88f76d82\" (UID: \"8ed7530c-9333-446a-8972-7eee88f76d82\") " Dec 03 12:42:47 crc kubenswrapper[5052]: I1203 12:42:47.118579 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8ed7530c-9333-446a-8972-7eee88f76d82-credential-keys" (OuterVolumeSpecName: "credential-keys") pod "8ed7530c-9333-446a-8972-7eee88f76d82" (UID: "8ed7530c-9333-446a-8972-7eee88f76d82"). InnerVolumeSpecName "credential-keys". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:42:47 crc kubenswrapper[5052]: I1203 12:42:47.119465 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8ed7530c-9333-446a-8972-7eee88f76d82-kube-api-access-h7sjd" (OuterVolumeSpecName: "kube-api-access-h7sjd") pod "8ed7530c-9333-446a-8972-7eee88f76d82" (UID: "8ed7530c-9333-446a-8972-7eee88f76d82"). InnerVolumeSpecName "kube-api-access-h7sjd". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:42:47 crc kubenswrapper[5052]: I1203 12:42:47.119596 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8ed7530c-9333-446a-8972-7eee88f76d82-fernet-keys" (OuterVolumeSpecName: "fernet-keys") pod "8ed7530c-9333-446a-8972-7eee88f76d82" (UID: "8ed7530c-9333-446a-8972-7eee88f76d82"). InnerVolumeSpecName "fernet-keys". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:42:47 crc kubenswrapper[5052]: I1203 12:42:47.120749 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8ed7530c-9333-446a-8972-7eee88f76d82-scripts" (OuterVolumeSpecName: "scripts") pod "8ed7530c-9333-446a-8972-7eee88f76d82" (UID: "8ed7530c-9333-446a-8972-7eee88f76d82"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:42:47 crc kubenswrapper[5052]: I1203 12:42:47.139374 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8ed7530c-9333-446a-8972-7eee88f76d82-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "8ed7530c-9333-446a-8972-7eee88f76d82" (UID: "8ed7530c-9333-446a-8972-7eee88f76d82"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:42:47 crc kubenswrapper[5052]: I1203 12:42:47.159694 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8ed7530c-9333-446a-8972-7eee88f76d82-config-data" (OuterVolumeSpecName: "config-data") pod "8ed7530c-9333-446a-8972-7eee88f76d82" (UID: "8ed7530c-9333-446a-8972-7eee88f76d82"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:42:47 crc kubenswrapper[5052]: I1203 12:42:47.212357 5052 reconciler_common.go:293] "Volume detached for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/8ed7530c-9333-446a-8972-7eee88f76d82-fernet-keys\") on node \"crc\" DevicePath \"\"" Dec 03 12:42:47 crc kubenswrapper[5052]: I1203 12:42:47.212397 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-h7sjd\" (UniqueName: \"kubernetes.io/projected/8ed7530c-9333-446a-8972-7eee88f76d82-kube-api-access-h7sjd\") on node \"crc\" DevicePath \"\"" Dec 03 12:42:47 crc kubenswrapper[5052]: I1203 12:42:47.212411 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/8ed7530c-9333-446a-8972-7eee88f76d82-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:42:47 crc kubenswrapper[5052]: I1203 12:42:47.212420 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/8ed7530c-9333-446a-8972-7eee88f76d82-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:42:47 crc kubenswrapper[5052]: I1203 12:42:47.212433 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8ed7530c-9333-446a-8972-7eee88f76d82-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:42:47 crc kubenswrapper[5052]: I1203 12:42:47.212445 5052 reconciler_common.go:293] "Volume detached for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/8ed7530c-9333-446a-8972-7eee88f76d82-credential-keys\") on node \"crc\" DevicePath \"\"" Dec 03 12:42:47 crc kubenswrapper[5052]: I1203 12:42:47.476563 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/placement-56b4b79f66-gfsxn"] Dec 03 12:42:47 crc kubenswrapper[5052]: E1203 12:42:47.476956 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="8ed7530c-9333-446a-8972-7eee88f76d82" containerName="keystone-bootstrap" Dec 03 12:42:47 crc kubenswrapper[5052]: I1203 12:42:47.476968 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="8ed7530c-9333-446a-8972-7eee88f76d82" containerName="keystone-bootstrap" Dec 03 12:42:47 crc kubenswrapper[5052]: E1203 12:42:47.476977 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="826d1642-afe5-4638-81e5-0d5b046e411e" containerName="barbican-db-sync" Dec 03 12:42:47 crc kubenswrapper[5052]: I1203 12:42:47.476982 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="826d1642-afe5-4638-81e5-0d5b046e411e" containerName="barbican-db-sync" Dec 03 12:42:47 crc kubenswrapper[5052]: E1203 12:42:47.477011 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="0063a179-2799-4532-9dd4-789e55b489be" containerName="placement-db-sync" Dec 03 12:42:47 crc kubenswrapper[5052]: I1203 12:42:47.477017 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="0063a179-2799-4532-9dd4-789e55b489be" containerName="placement-db-sync" Dec 03 12:42:47 crc kubenswrapper[5052]: I1203 12:42:47.477191 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="826d1642-afe5-4638-81e5-0d5b046e411e" containerName="barbican-db-sync" Dec 03 12:42:47 crc kubenswrapper[5052]: I1203 12:42:47.477204 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="0063a179-2799-4532-9dd4-789e55b489be" containerName="placement-db-sync" Dec 03 12:42:47 crc kubenswrapper[5052]: I1203 12:42:47.477222 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="8ed7530c-9333-446a-8972-7eee88f76d82" containerName="keystone-bootstrap" Dec 03 12:42:47 crc kubenswrapper[5052]: I1203 12:42:47.478148 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-56b4b79f66-gfsxn" Dec 03 12:42:47 crc kubenswrapper[5052]: I1203 12:42:47.480151 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"placement-scripts" Dec 03 12:42:47 crc kubenswrapper[5052]: I1203 12:42:47.480432 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"placement-config-data" Dec 03 12:42:47 crc kubenswrapper[5052]: I1203 12:42:47.480602 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-placement-public-svc" Dec 03 12:42:47 crc kubenswrapper[5052]: I1203 12:42:47.481007 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-placement-internal-svc" Dec 03 12:42:47 crc kubenswrapper[5052]: I1203 12:42:47.481611 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"placement-placement-dockercfg-qt92n" Dec 03 12:42:47 crc kubenswrapper[5052]: I1203 12:42:47.512307 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/placement-56b4b79f66-gfsxn"] Dec 03 12:42:47 crc kubenswrapper[5052]: I1203 12:42:47.623023 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/76534e0c-3b73-4ede-bc24-b12c7f8ea4db-internal-tls-certs\") pod \"placement-56b4b79f66-gfsxn\" (UID: \"76534e0c-3b73-4ede-bc24-b12c7f8ea4db\") " pod="openstack-kuttl-tests/placement-56b4b79f66-gfsxn" Dec 03 12:42:47 crc kubenswrapper[5052]: I1203 12:42:47.623104 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/76534e0c-3b73-4ede-bc24-b12c7f8ea4db-config-data\") pod \"placement-56b4b79f66-gfsxn\" (UID: \"76534e0c-3b73-4ede-bc24-b12c7f8ea4db\") " pod="openstack-kuttl-tests/placement-56b4b79f66-gfsxn" Dec 03 12:42:47 crc kubenswrapper[5052]: I1203 12:42:47.623150 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/76534e0c-3b73-4ede-bc24-b12c7f8ea4db-public-tls-certs\") pod \"placement-56b4b79f66-gfsxn\" (UID: \"76534e0c-3b73-4ede-bc24-b12c7f8ea4db\") " pod="openstack-kuttl-tests/placement-56b4b79f66-gfsxn" Dec 03 12:42:47 crc kubenswrapper[5052]: I1203 12:42:47.623193 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/76534e0c-3b73-4ede-bc24-b12c7f8ea4db-logs\") pod \"placement-56b4b79f66-gfsxn\" (UID: \"76534e0c-3b73-4ede-bc24-b12c7f8ea4db\") " pod="openstack-kuttl-tests/placement-56b4b79f66-gfsxn" Dec 03 12:42:47 crc kubenswrapper[5052]: I1203 12:42:47.623212 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/76534e0c-3b73-4ede-bc24-b12c7f8ea4db-scripts\") pod \"placement-56b4b79f66-gfsxn\" (UID: \"76534e0c-3b73-4ede-bc24-b12c7f8ea4db\") " pod="openstack-kuttl-tests/placement-56b4b79f66-gfsxn" Dec 03 12:42:47 crc kubenswrapper[5052]: I1203 12:42:47.623247 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lsljg\" (UniqueName: \"kubernetes.io/projected/76534e0c-3b73-4ede-bc24-b12c7f8ea4db-kube-api-access-lsljg\") pod \"placement-56b4b79f66-gfsxn\" (UID: \"76534e0c-3b73-4ede-bc24-b12c7f8ea4db\") " pod="openstack-kuttl-tests/placement-56b4b79f66-gfsxn" Dec 03 12:42:47 crc kubenswrapper[5052]: I1203 12:42:47.623263 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/76534e0c-3b73-4ede-bc24-b12c7f8ea4db-combined-ca-bundle\") pod \"placement-56b4b79f66-gfsxn\" (UID: \"76534e0c-3b73-4ede-bc24-b12c7f8ea4db\") " pod="openstack-kuttl-tests/placement-56b4b79f66-gfsxn" Dec 03 12:42:47 crc kubenswrapper[5052]: I1203 12:42:47.623484 5052 generic.go:334] "Generic (PLEG): container finished" podID="133758ce-c13c-4ac1-b672-49412b22ec08" containerID="8ecebefb263fcd4c52ad89050bdb294b17d15a1d16e9932448c14d8cd8f988b1" exitCode=0 Dec 03 12:42:47 crc kubenswrapper[5052]: I1203 12:42:47.623503 5052 generic.go:334] "Generic (PLEG): container finished" podID="133758ce-c13c-4ac1-b672-49412b22ec08" containerID="050e4f507ff648b4d9a847ec84654e743d8a6e6d3a90811c85901f7f8a74f34a" exitCode=2 Dec 03 12:42:47 crc kubenswrapper[5052]: I1203 12:42:47.623526 5052 generic.go:334] "Generic (PLEG): container finished" podID="133758ce-c13c-4ac1-b672-49412b22ec08" containerID="6d5877335791adc8136d132f76bce6689455c186a9abe604beb4d46b25a2646c" exitCode=0 Dec 03 12:42:47 crc kubenswrapper[5052]: I1203 12:42:47.623535 5052 generic.go:334] "Generic (PLEG): container finished" podID="133758ce-c13c-4ac1-b672-49412b22ec08" containerID="da6fb06ac7ebc0adff26fc1ccb94fc8314f3ea98f662feb777cf3e824d4d161b" exitCode=0 Dec 03 12:42:47 crc kubenswrapper[5052]: I1203 12:42:47.623623 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"133758ce-c13c-4ac1-b672-49412b22ec08","Type":"ContainerDied","Data":"8ecebefb263fcd4c52ad89050bdb294b17d15a1d16e9932448c14d8cd8f988b1"} Dec 03 12:42:47 crc kubenswrapper[5052]: I1203 12:42:47.623659 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"133758ce-c13c-4ac1-b672-49412b22ec08","Type":"ContainerDied","Data":"050e4f507ff648b4d9a847ec84654e743d8a6e6d3a90811c85901f7f8a74f34a"} Dec 03 12:42:47 crc kubenswrapper[5052]: I1203 12:42:47.623675 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"133758ce-c13c-4ac1-b672-49412b22ec08","Type":"ContainerDied","Data":"6d5877335791adc8136d132f76bce6689455c186a9abe604beb4d46b25a2646c"} Dec 03 12:42:47 crc kubenswrapper[5052]: I1203 12:42:47.623687 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"133758ce-c13c-4ac1-b672-49412b22ec08","Type":"ContainerDied","Data":"da6fb06ac7ebc0adff26fc1ccb94fc8314f3ea98f662feb777cf3e824d4d161b"} Dec 03 12:42:47 crc kubenswrapper[5052]: I1203 12:42:47.631280 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/barbican-worker-65fcd76cf7-xhs44"] Dec 03 12:42:47 crc kubenswrapper[5052]: I1203 12:42:47.634042 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-worker-65fcd76cf7-xhs44" Dec 03 12:42:47 crc kubenswrapper[5052]: I1203 12:42:47.634992 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-bootstrap-vbrn6" event={"ID":"8ed7530c-9333-446a-8972-7eee88f76d82","Type":"ContainerDied","Data":"1e150c21e0cb658125e78758440c5f44c367f35b7c580faa15b71005233fc368"} Dec 03 12:42:47 crc kubenswrapper[5052]: I1203 12:42:47.635069 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1e150c21e0cb658125e78758440c5f44c367f35b7c580faa15b71005233fc368" Dec 03 12:42:47 crc kubenswrapper[5052]: I1203 12:42:47.635165 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-bootstrap-vbrn6" Dec 03 12:42:47 crc kubenswrapper[5052]: I1203 12:42:47.648839 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"barbican-barbican-dockercfg-tjssv" Dec 03 12:42:47 crc kubenswrapper[5052]: I1203 12:42:47.648911 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"barbican-config-data" Dec 03 12:42:47 crc kubenswrapper[5052]: I1203 12:42:47.648962 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"barbican-worker-config-data" Dec 03 12:42:47 crc kubenswrapper[5052]: I1203 12:42:47.651881 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/barbican-keystone-listener-769bc98c4b-mnz6r"] Dec 03 12:42:47 crc kubenswrapper[5052]: I1203 12:42:47.654744 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-keystone-listener-769bc98c4b-mnz6r" Dec 03 12:42:47 crc kubenswrapper[5052]: I1203 12:42:47.657198 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"24ccc68c-26a9-485e-bf26-512dac006b61","Type":"ContainerStarted","Data":"62e7c426fcaf4df9986d8f33d839e1a4e2b3d6f13900f261856dbcb3908cbde1"} Dec 03 12:42:47 crc kubenswrapper[5052]: I1203 12:42:47.668381 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-worker-65fcd76cf7-xhs44"] Dec 03 12:42:47 crc kubenswrapper[5052]: I1203 12:42:47.679309 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"barbican-keystone-listener-config-data" Dec 03 12:42:47 crc kubenswrapper[5052]: I1203 12:42:47.721505 5052 generic.go:334] "Generic (PLEG): container finished" podID="1de481f1-0c54-48c3-b112-659497d86792" containerID="3c9394ad9a076c5005649d998f0ffa542de55379118fcfb363d27cb846a45d9f" exitCode=0 Dec 03 12:42:47 crc kubenswrapper[5052]: I1203 12:42:47.726096 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1d187c5a-7745-447f-a8e2-06c7528e5be0-combined-ca-bundle\") pod \"barbican-keystone-listener-769bc98c4b-mnz6r\" (UID: \"1d187c5a-7745-447f-a8e2-06c7528e5be0\") " pod="openstack-kuttl-tests/barbican-keystone-listener-769bc98c4b-mnz6r" Dec 03 12:42:47 crc kubenswrapper[5052]: I1203 12:42:47.726136 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/92290cff-ed52-4a47-80e9-720bfd9f3dd8-config-data-custom\") pod \"barbican-worker-65fcd76cf7-xhs44\" (UID: \"92290cff-ed52-4a47-80e9-720bfd9f3dd8\") " pod="openstack-kuttl-tests/barbican-worker-65fcd76cf7-xhs44" Dec 03 12:42:47 crc kubenswrapper[5052]: I1203 12:42:47.726159 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/92290cff-ed52-4a47-80e9-720bfd9f3dd8-combined-ca-bundle\") pod \"barbican-worker-65fcd76cf7-xhs44\" (UID: \"92290cff-ed52-4a47-80e9-720bfd9f3dd8\") " pod="openstack-kuttl-tests/barbican-worker-65fcd76cf7-xhs44" Dec 03 12:42:47 crc kubenswrapper[5052]: I1203 12:42:47.726181 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/1d187c5a-7745-447f-a8e2-06c7528e5be0-logs\") pod \"barbican-keystone-listener-769bc98c4b-mnz6r\" (UID: \"1d187c5a-7745-447f-a8e2-06c7528e5be0\") " pod="openstack-kuttl-tests/barbican-keystone-listener-769bc98c4b-mnz6r" Dec 03 12:42:47 crc kubenswrapper[5052]: I1203 12:42:47.726213 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/76534e0c-3b73-4ede-bc24-b12c7f8ea4db-config-data\") pod \"placement-56b4b79f66-gfsxn\" (UID: \"76534e0c-3b73-4ede-bc24-b12c7f8ea4db\") " pod="openstack-kuttl-tests/placement-56b4b79f66-gfsxn" Dec 03 12:42:47 crc kubenswrapper[5052]: I1203 12:42:47.726241 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mwn82\" (UniqueName: \"kubernetes.io/projected/92290cff-ed52-4a47-80e9-720bfd9f3dd8-kube-api-access-mwn82\") pod \"barbican-worker-65fcd76cf7-xhs44\" (UID: \"92290cff-ed52-4a47-80e9-720bfd9f3dd8\") " pod="openstack-kuttl-tests/barbican-worker-65fcd76cf7-xhs44" Dec 03 12:42:47 crc kubenswrapper[5052]: I1203 12:42:47.726269 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/1d187c5a-7745-447f-a8e2-06c7528e5be0-config-data-custom\") pod \"barbican-keystone-listener-769bc98c4b-mnz6r\" (UID: \"1d187c5a-7745-447f-a8e2-06c7528e5be0\") " pod="openstack-kuttl-tests/barbican-keystone-listener-769bc98c4b-mnz6r" Dec 03 12:42:47 crc kubenswrapper[5052]: I1203 12:42:47.726295 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/76534e0c-3b73-4ede-bc24-b12c7f8ea4db-public-tls-certs\") pod \"placement-56b4b79f66-gfsxn\" (UID: \"76534e0c-3b73-4ede-bc24-b12c7f8ea4db\") " pod="openstack-kuttl-tests/placement-56b4b79f66-gfsxn" Dec 03 12:42:47 crc kubenswrapper[5052]: I1203 12:42:47.726332 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9zt7x\" (UniqueName: \"kubernetes.io/projected/1d187c5a-7745-447f-a8e2-06c7528e5be0-kube-api-access-9zt7x\") pod \"barbican-keystone-listener-769bc98c4b-mnz6r\" (UID: \"1d187c5a-7745-447f-a8e2-06c7528e5be0\") " pod="openstack-kuttl-tests/barbican-keystone-listener-769bc98c4b-mnz6r" Dec 03 12:42:47 crc kubenswrapper[5052]: I1203 12:42:47.726356 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/1d187c5a-7745-447f-a8e2-06c7528e5be0-config-data\") pod \"barbican-keystone-listener-769bc98c4b-mnz6r\" (UID: \"1d187c5a-7745-447f-a8e2-06c7528e5be0\") " pod="openstack-kuttl-tests/barbican-keystone-listener-769bc98c4b-mnz6r" Dec 03 12:42:47 crc kubenswrapper[5052]: I1203 12:42:47.726376 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/76534e0c-3b73-4ede-bc24-b12c7f8ea4db-logs\") pod \"placement-56b4b79f66-gfsxn\" (UID: \"76534e0c-3b73-4ede-bc24-b12c7f8ea4db\") " pod="openstack-kuttl-tests/placement-56b4b79f66-gfsxn" Dec 03 12:42:47 crc kubenswrapper[5052]: I1203 12:42:47.726393 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/76534e0c-3b73-4ede-bc24-b12c7f8ea4db-scripts\") pod \"placement-56b4b79f66-gfsxn\" (UID: \"76534e0c-3b73-4ede-bc24-b12c7f8ea4db\") " pod="openstack-kuttl-tests/placement-56b4b79f66-gfsxn" Dec 03 12:42:47 crc kubenswrapper[5052]: I1203 12:42:47.726410 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/92290cff-ed52-4a47-80e9-720bfd9f3dd8-config-data\") pod \"barbican-worker-65fcd76cf7-xhs44\" (UID: \"92290cff-ed52-4a47-80e9-720bfd9f3dd8\") " pod="openstack-kuttl-tests/barbican-worker-65fcd76cf7-xhs44" Dec 03 12:42:47 crc kubenswrapper[5052]: I1203 12:42:47.726447 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-lsljg\" (UniqueName: \"kubernetes.io/projected/76534e0c-3b73-4ede-bc24-b12c7f8ea4db-kube-api-access-lsljg\") pod \"placement-56b4b79f66-gfsxn\" (UID: \"76534e0c-3b73-4ede-bc24-b12c7f8ea4db\") " pod="openstack-kuttl-tests/placement-56b4b79f66-gfsxn" Dec 03 12:42:47 crc kubenswrapper[5052]: I1203 12:42:47.726464 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/76534e0c-3b73-4ede-bc24-b12c7f8ea4db-combined-ca-bundle\") pod \"placement-56b4b79f66-gfsxn\" (UID: \"76534e0c-3b73-4ede-bc24-b12c7f8ea4db\") " pod="openstack-kuttl-tests/placement-56b4b79f66-gfsxn" Dec 03 12:42:47 crc kubenswrapper[5052]: I1203 12:42:47.726496 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/76534e0c-3b73-4ede-bc24-b12c7f8ea4db-internal-tls-certs\") pod \"placement-56b4b79f66-gfsxn\" (UID: \"76534e0c-3b73-4ede-bc24-b12c7f8ea4db\") " pod="openstack-kuttl-tests/placement-56b4b79f66-gfsxn" Dec 03 12:42:47 crc kubenswrapper[5052]: I1203 12:42:47.726532 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/92290cff-ed52-4a47-80e9-720bfd9f3dd8-logs\") pod \"barbican-worker-65fcd76cf7-xhs44\" (UID: \"92290cff-ed52-4a47-80e9-720bfd9f3dd8\") " pod="openstack-kuttl-tests/barbican-worker-65fcd76cf7-xhs44" Dec 03 12:42:47 crc kubenswrapper[5052]: I1203 12:42:47.726948 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/76534e0c-3b73-4ede-bc24-b12c7f8ea4db-logs\") pod \"placement-56b4b79f66-gfsxn\" (UID: \"76534e0c-3b73-4ede-bc24-b12c7f8ea4db\") " pod="openstack-kuttl-tests/placement-56b4b79f66-gfsxn" Dec 03 12:42:47 crc kubenswrapper[5052]: I1203 12:42:47.734242 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="15420a4f-e75c-4ca6-940e-7f75f3eaab1d" path="/var/lib/kubelet/pods/15420a4f-e75c-4ca6-940e-7f75f3eaab1d/volumes" Dec 03 12:42:47 crc kubenswrapper[5052]: I1203 12:42:47.760021 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-keystone-listener-769bc98c4b-mnz6r"] Dec 03 12:42:47 crc kubenswrapper[5052]: I1203 12:42:47.760073 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/keystone-bootstrap-vbrn6"] Dec 03 12:42:47 crc kubenswrapper[5052]: I1203 12:42:47.760090 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"8011917c-0ce8-4c83-b157-2d8538790e17","Type":"ContainerStarted","Data":"3d59485fd382731e310136b18cd55378ab24d2433daddc0d9279d4e9e6835ee3"} Dec 03 12:42:47 crc kubenswrapper[5052]: I1203 12:42:47.760116 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-db-sync-2ttlg" event={"ID":"1de481f1-0c54-48c3-b112-659497d86792","Type":"ContainerDied","Data":"3c9394ad9a076c5005649d998f0ffa542de55379118fcfb363d27cb846a45d9f"} Dec 03 12:42:47 crc kubenswrapper[5052]: I1203 12:42:47.760139 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/keystone-bootstrap-vbrn6"] Dec 03 12:42:47 crc kubenswrapper[5052]: I1203 12:42:47.770721 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/76534e0c-3b73-4ede-bc24-b12c7f8ea4db-config-data\") pod \"placement-56b4b79f66-gfsxn\" (UID: \"76534e0c-3b73-4ede-bc24-b12c7f8ea4db\") " pod="openstack-kuttl-tests/placement-56b4b79f66-gfsxn" Dec 03 12:42:47 crc kubenswrapper[5052]: I1203 12:42:47.771060 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/76534e0c-3b73-4ede-bc24-b12c7f8ea4db-public-tls-certs\") pod \"placement-56b4b79f66-gfsxn\" (UID: \"76534e0c-3b73-4ede-bc24-b12c7f8ea4db\") " pod="openstack-kuttl-tests/placement-56b4b79f66-gfsxn" Dec 03 12:42:47 crc kubenswrapper[5052]: I1203 12:42:47.771476 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-lsljg\" (UniqueName: \"kubernetes.io/projected/76534e0c-3b73-4ede-bc24-b12c7f8ea4db-kube-api-access-lsljg\") pod \"placement-56b4b79f66-gfsxn\" (UID: \"76534e0c-3b73-4ede-bc24-b12c7f8ea4db\") " pod="openstack-kuttl-tests/placement-56b4b79f66-gfsxn" Dec 03 12:42:47 crc kubenswrapper[5052]: I1203 12:42:47.771766 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/76534e0c-3b73-4ede-bc24-b12c7f8ea4db-scripts\") pod \"placement-56b4b79f66-gfsxn\" (UID: \"76534e0c-3b73-4ede-bc24-b12c7f8ea4db\") " pod="openstack-kuttl-tests/placement-56b4b79f66-gfsxn" Dec 03 12:42:47 crc kubenswrapper[5052]: I1203 12:42:47.773618 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/keystone-bootstrap-nnnzp"] Dec 03 12:42:47 crc kubenswrapper[5052]: I1203 12:42:47.774233 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/76534e0c-3b73-4ede-bc24-b12c7f8ea4db-combined-ca-bundle\") pod \"placement-56b4b79f66-gfsxn\" (UID: \"76534e0c-3b73-4ede-bc24-b12c7f8ea4db\") " pod="openstack-kuttl-tests/placement-56b4b79f66-gfsxn" Dec 03 12:42:47 crc kubenswrapper[5052]: I1203 12:42:47.774876 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-bootstrap-nnnzp" Dec 03 12:42:47 crc kubenswrapper[5052]: I1203 12:42:47.780383 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone-scripts" Dec 03 12:42:47 crc kubenswrapper[5052]: I1203 12:42:47.781693 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"osp-secret" Dec 03 12:42:47 crc kubenswrapper[5052]: I1203 12:42:47.781967 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone-config-data" Dec 03 12:42:47 crc kubenswrapper[5052]: I1203 12:42:47.782129 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone" Dec 03 12:42:47 crc kubenswrapper[5052]: I1203 12:42:47.782277 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone-keystone-dockercfg-2f85l" Dec 03 12:42:47 crc kubenswrapper[5052]: I1203 12:42:47.783575 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-bootstrap-nnnzp"] Dec 03 12:42:47 crc kubenswrapper[5052]: I1203 12:42:47.803833 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/76534e0c-3b73-4ede-bc24-b12c7f8ea4db-internal-tls-certs\") pod \"placement-56b4b79f66-gfsxn\" (UID: \"76534e0c-3b73-4ede-bc24-b12c7f8ea4db\") " pod="openstack-kuttl-tests/placement-56b4b79f66-gfsxn" Dec 03 12:42:48 crc kubenswrapper[5052]: I1203 12:42:47.827670 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-9zt7x\" (UniqueName: \"kubernetes.io/projected/1d187c5a-7745-447f-a8e2-06c7528e5be0-kube-api-access-9zt7x\") pod \"barbican-keystone-listener-769bc98c4b-mnz6r\" (UID: \"1d187c5a-7745-447f-a8e2-06c7528e5be0\") " pod="openstack-kuttl-tests/barbican-keystone-listener-769bc98c4b-mnz6r" Dec 03 12:42:48 crc kubenswrapper[5052]: I1203 12:42:47.827720 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/1d187c5a-7745-447f-a8e2-06c7528e5be0-config-data\") pod \"barbican-keystone-listener-769bc98c4b-mnz6r\" (UID: \"1d187c5a-7745-447f-a8e2-06c7528e5be0\") " pod="openstack-kuttl-tests/barbican-keystone-listener-769bc98c4b-mnz6r" Dec 03 12:42:48 crc kubenswrapper[5052]: I1203 12:42:47.827754 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/92290cff-ed52-4a47-80e9-720bfd9f3dd8-config-data\") pod \"barbican-worker-65fcd76cf7-xhs44\" (UID: \"92290cff-ed52-4a47-80e9-720bfd9f3dd8\") " pod="openstack-kuttl-tests/barbican-worker-65fcd76cf7-xhs44" Dec 03 12:42:48 crc kubenswrapper[5052]: I1203 12:42:47.827833 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/92290cff-ed52-4a47-80e9-720bfd9f3dd8-logs\") pod \"barbican-worker-65fcd76cf7-xhs44\" (UID: \"92290cff-ed52-4a47-80e9-720bfd9f3dd8\") " pod="openstack-kuttl-tests/barbican-worker-65fcd76cf7-xhs44" Dec 03 12:42:48 crc kubenswrapper[5052]: I1203 12:42:47.827867 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1d187c5a-7745-447f-a8e2-06c7528e5be0-combined-ca-bundle\") pod \"barbican-keystone-listener-769bc98c4b-mnz6r\" (UID: \"1d187c5a-7745-447f-a8e2-06c7528e5be0\") " pod="openstack-kuttl-tests/barbican-keystone-listener-769bc98c4b-mnz6r" Dec 03 12:42:48 crc kubenswrapper[5052]: I1203 12:42:47.827891 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/92290cff-ed52-4a47-80e9-720bfd9f3dd8-config-data-custom\") pod \"barbican-worker-65fcd76cf7-xhs44\" (UID: \"92290cff-ed52-4a47-80e9-720bfd9f3dd8\") " pod="openstack-kuttl-tests/barbican-worker-65fcd76cf7-xhs44" Dec 03 12:42:48 crc kubenswrapper[5052]: I1203 12:42:47.827908 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/92290cff-ed52-4a47-80e9-720bfd9f3dd8-combined-ca-bundle\") pod \"barbican-worker-65fcd76cf7-xhs44\" (UID: \"92290cff-ed52-4a47-80e9-720bfd9f3dd8\") " pod="openstack-kuttl-tests/barbican-worker-65fcd76cf7-xhs44" Dec 03 12:42:48 crc kubenswrapper[5052]: I1203 12:42:47.828196 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/1d187c5a-7745-447f-a8e2-06c7528e5be0-logs\") pod \"barbican-keystone-listener-769bc98c4b-mnz6r\" (UID: \"1d187c5a-7745-447f-a8e2-06c7528e5be0\") " pod="openstack-kuttl-tests/barbican-keystone-listener-769bc98c4b-mnz6r" Dec 03 12:42:48 crc kubenswrapper[5052]: I1203 12:42:47.828233 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-mwn82\" (UniqueName: \"kubernetes.io/projected/92290cff-ed52-4a47-80e9-720bfd9f3dd8-kube-api-access-mwn82\") pod \"barbican-worker-65fcd76cf7-xhs44\" (UID: \"92290cff-ed52-4a47-80e9-720bfd9f3dd8\") " pod="openstack-kuttl-tests/barbican-worker-65fcd76cf7-xhs44" Dec 03 12:42:48 crc kubenswrapper[5052]: I1203 12:42:47.828254 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/1d187c5a-7745-447f-a8e2-06c7528e5be0-config-data-custom\") pod \"barbican-keystone-listener-769bc98c4b-mnz6r\" (UID: \"1d187c5a-7745-447f-a8e2-06c7528e5be0\") " pod="openstack-kuttl-tests/barbican-keystone-listener-769bc98c4b-mnz6r" Dec 03 12:42:48 crc kubenswrapper[5052]: I1203 12:42:47.834209 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/92290cff-ed52-4a47-80e9-720bfd9f3dd8-logs\") pod \"barbican-worker-65fcd76cf7-xhs44\" (UID: \"92290cff-ed52-4a47-80e9-720bfd9f3dd8\") " pod="openstack-kuttl-tests/barbican-worker-65fcd76cf7-xhs44" Dec 03 12:42:48 crc kubenswrapper[5052]: I1203 12:42:47.851092 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/1d187c5a-7745-447f-a8e2-06c7528e5be0-logs\") pod \"barbican-keystone-listener-769bc98c4b-mnz6r\" (UID: \"1d187c5a-7745-447f-a8e2-06c7528e5be0\") " pod="openstack-kuttl-tests/barbican-keystone-listener-769bc98c4b-mnz6r" Dec 03 12:42:48 crc kubenswrapper[5052]: I1203 12:42:47.852479 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-56b4b79f66-gfsxn" Dec 03 12:42:48 crc kubenswrapper[5052]: I1203 12:42:47.868314 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/92290cff-ed52-4a47-80e9-720bfd9f3dd8-config-data-custom\") pod \"barbican-worker-65fcd76cf7-xhs44\" (UID: \"92290cff-ed52-4a47-80e9-720bfd9f3dd8\") " pod="openstack-kuttl-tests/barbican-worker-65fcd76cf7-xhs44" Dec 03 12:42:48 crc kubenswrapper[5052]: I1203 12:42:47.875269 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/92290cff-ed52-4a47-80e9-720bfd9f3dd8-combined-ca-bundle\") pod \"barbican-worker-65fcd76cf7-xhs44\" (UID: \"92290cff-ed52-4a47-80e9-720bfd9f3dd8\") " pod="openstack-kuttl-tests/barbican-worker-65fcd76cf7-xhs44" Dec 03 12:42:48 crc kubenswrapper[5052]: I1203 12:42:47.876139 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/1d187c5a-7745-447f-a8e2-06c7528e5be0-config-data-custom\") pod \"barbican-keystone-listener-769bc98c4b-mnz6r\" (UID: \"1d187c5a-7745-447f-a8e2-06c7528e5be0\") " pod="openstack-kuttl-tests/barbican-keystone-listener-769bc98c4b-mnz6r" Dec 03 12:42:48 crc kubenswrapper[5052]: I1203 12:42:47.877346 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/1d187c5a-7745-447f-a8e2-06c7528e5be0-config-data\") pod \"barbican-keystone-listener-769bc98c4b-mnz6r\" (UID: \"1d187c5a-7745-447f-a8e2-06c7528e5be0\") " pod="openstack-kuttl-tests/barbican-keystone-listener-769bc98c4b-mnz6r" Dec 03 12:42:48 crc kubenswrapper[5052]: I1203 12:42:47.878573 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1d187c5a-7745-447f-a8e2-06c7528e5be0-combined-ca-bundle\") pod \"barbican-keystone-listener-769bc98c4b-mnz6r\" (UID: \"1d187c5a-7745-447f-a8e2-06c7528e5be0\") " pod="openstack-kuttl-tests/barbican-keystone-listener-769bc98c4b-mnz6r" Dec 03 12:42:48 crc kubenswrapper[5052]: I1203 12:42:47.879414 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/92290cff-ed52-4a47-80e9-720bfd9f3dd8-config-data\") pod \"barbican-worker-65fcd76cf7-xhs44\" (UID: \"92290cff-ed52-4a47-80e9-720bfd9f3dd8\") " pod="openstack-kuttl-tests/barbican-worker-65fcd76cf7-xhs44" Dec 03 12:42:48 crc kubenswrapper[5052]: I1203 12:42:47.908220 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-mwn82\" (UniqueName: \"kubernetes.io/projected/92290cff-ed52-4a47-80e9-720bfd9f3dd8-kube-api-access-mwn82\") pod \"barbican-worker-65fcd76cf7-xhs44\" (UID: \"92290cff-ed52-4a47-80e9-720bfd9f3dd8\") " pod="openstack-kuttl-tests/barbican-worker-65fcd76cf7-xhs44" Dec 03 12:42:48 crc kubenswrapper[5052]: I1203 12:42:47.909336 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-9zt7x\" (UniqueName: \"kubernetes.io/projected/1d187c5a-7745-447f-a8e2-06c7528e5be0-kube-api-access-9zt7x\") pod \"barbican-keystone-listener-769bc98c4b-mnz6r\" (UID: \"1d187c5a-7745-447f-a8e2-06c7528e5be0\") " pod="openstack-kuttl-tests/barbican-keystone-listener-769bc98c4b-mnz6r" Dec 03 12:42:48 crc kubenswrapper[5052]: I1203 12:42:47.929622 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/b36ab165-21eb-4968-a755-2cef353ce195-fernet-keys\") pod \"keystone-bootstrap-nnnzp\" (UID: \"b36ab165-21eb-4968-a755-2cef353ce195\") " pod="openstack-kuttl-tests/keystone-bootstrap-nnnzp" Dec 03 12:42:48 crc kubenswrapper[5052]: I1203 12:42:47.929679 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/b36ab165-21eb-4968-a755-2cef353ce195-config-data\") pod \"keystone-bootstrap-nnnzp\" (UID: \"b36ab165-21eb-4968-a755-2cef353ce195\") " pod="openstack-kuttl-tests/keystone-bootstrap-nnnzp" Dec 03 12:42:48 crc kubenswrapper[5052]: I1203 12:42:47.929758 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/b36ab165-21eb-4968-a755-2cef353ce195-scripts\") pod \"keystone-bootstrap-nnnzp\" (UID: \"b36ab165-21eb-4968-a755-2cef353ce195\") " pod="openstack-kuttl-tests/keystone-bootstrap-nnnzp" Dec 03 12:42:48 crc kubenswrapper[5052]: I1203 12:42:47.929826 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/b36ab165-21eb-4968-a755-2cef353ce195-credential-keys\") pod \"keystone-bootstrap-nnnzp\" (UID: \"b36ab165-21eb-4968-a755-2cef353ce195\") " pod="openstack-kuttl-tests/keystone-bootstrap-nnnzp" Dec 03 12:42:48 crc kubenswrapper[5052]: I1203 12:42:47.929861 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b36ab165-21eb-4968-a755-2cef353ce195-combined-ca-bundle\") pod \"keystone-bootstrap-nnnzp\" (UID: \"b36ab165-21eb-4968-a755-2cef353ce195\") " pod="openstack-kuttl-tests/keystone-bootstrap-nnnzp" Dec 03 12:42:48 crc kubenswrapper[5052]: I1203 12:42:47.929901 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-phwjr\" (UniqueName: \"kubernetes.io/projected/b36ab165-21eb-4968-a755-2cef353ce195-kube-api-access-phwjr\") pod \"keystone-bootstrap-nnnzp\" (UID: \"b36ab165-21eb-4968-a755-2cef353ce195\") " pod="openstack-kuttl-tests/keystone-bootstrap-nnnzp" Dec 03 12:42:48 crc kubenswrapper[5052]: I1203 12:42:47.963646 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/barbican-api-866c868748-bdl5k"] Dec 03 12:42:48 crc kubenswrapper[5052]: I1203 12:42:47.965639 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-api-866c868748-bdl5k" Dec 03 12:42:48 crc kubenswrapper[5052]: I1203 12:42:47.979158 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"barbican-api-config-data" Dec 03 12:42:48 crc kubenswrapper[5052]: I1203 12:42:48.001579 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-api-866c868748-bdl5k"] Dec 03 12:42:48 crc kubenswrapper[5052]: I1203 12:42:48.038656 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/b36ab165-21eb-4968-a755-2cef353ce195-fernet-keys\") pod \"keystone-bootstrap-nnnzp\" (UID: \"b36ab165-21eb-4968-a755-2cef353ce195\") " pod="openstack-kuttl-tests/keystone-bootstrap-nnnzp" Dec 03 12:42:48 crc kubenswrapper[5052]: I1203 12:42:48.038712 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/b36ab165-21eb-4968-a755-2cef353ce195-config-data\") pod \"keystone-bootstrap-nnnzp\" (UID: \"b36ab165-21eb-4968-a755-2cef353ce195\") " pod="openstack-kuttl-tests/keystone-bootstrap-nnnzp" Dec 03 12:42:48 crc kubenswrapper[5052]: I1203 12:42:48.038762 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/4c57f5e8-fe35-4f32-b0b1-a3dc5d0ccf33-config-data-custom\") pod \"barbican-api-866c868748-bdl5k\" (UID: \"4c57f5e8-fe35-4f32-b0b1-a3dc5d0ccf33\") " pod="openstack-kuttl-tests/barbican-api-866c868748-bdl5k" Dec 03 12:42:48 crc kubenswrapper[5052]: I1203 12:42:48.038827 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/b36ab165-21eb-4968-a755-2cef353ce195-scripts\") pod \"keystone-bootstrap-nnnzp\" (UID: \"b36ab165-21eb-4968-a755-2cef353ce195\") " pod="openstack-kuttl-tests/keystone-bootstrap-nnnzp" Dec 03 12:42:48 crc kubenswrapper[5052]: I1203 12:42:48.038857 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-87bqn\" (UniqueName: \"kubernetes.io/projected/4c57f5e8-fe35-4f32-b0b1-a3dc5d0ccf33-kube-api-access-87bqn\") pod \"barbican-api-866c868748-bdl5k\" (UID: \"4c57f5e8-fe35-4f32-b0b1-a3dc5d0ccf33\") " pod="openstack-kuttl-tests/barbican-api-866c868748-bdl5k" Dec 03 12:42:48 crc kubenswrapper[5052]: I1203 12:42:48.038894 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4c57f5e8-fe35-4f32-b0b1-a3dc5d0ccf33-combined-ca-bundle\") pod \"barbican-api-866c868748-bdl5k\" (UID: \"4c57f5e8-fe35-4f32-b0b1-a3dc5d0ccf33\") " pod="openstack-kuttl-tests/barbican-api-866c868748-bdl5k" Dec 03 12:42:48 crc kubenswrapper[5052]: I1203 12:42:48.038922 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/4c57f5e8-fe35-4f32-b0b1-a3dc5d0ccf33-logs\") pod \"barbican-api-866c868748-bdl5k\" (UID: \"4c57f5e8-fe35-4f32-b0b1-a3dc5d0ccf33\") " pod="openstack-kuttl-tests/barbican-api-866c868748-bdl5k" Dec 03 12:42:48 crc kubenswrapper[5052]: I1203 12:42:48.038956 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/4c57f5e8-fe35-4f32-b0b1-a3dc5d0ccf33-config-data\") pod \"barbican-api-866c868748-bdl5k\" (UID: \"4c57f5e8-fe35-4f32-b0b1-a3dc5d0ccf33\") " pod="openstack-kuttl-tests/barbican-api-866c868748-bdl5k" Dec 03 12:42:48 crc kubenswrapper[5052]: I1203 12:42:48.038992 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/b36ab165-21eb-4968-a755-2cef353ce195-credential-keys\") pod \"keystone-bootstrap-nnnzp\" (UID: \"b36ab165-21eb-4968-a755-2cef353ce195\") " pod="openstack-kuttl-tests/keystone-bootstrap-nnnzp" Dec 03 12:42:48 crc kubenswrapper[5052]: I1203 12:42:48.039024 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b36ab165-21eb-4968-a755-2cef353ce195-combined-ca-bundle\") pod \"keystone-bootstrap-nnnzp\" (UID: \"b36ab165-21eb-4968-a755-2cef353ce195\") " pod="openstack-kuttl-tests/keystone-bootstrap-nnnzp" Dec 03 12:42:48 crc kubenswrapper[5052]: I1203 12:42:48.039060 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-phwjr\" (UniqueName: \"kubernetes.io/projected/b36ab165-21eb-4968-a755-2cef353ce195-kube-api-access-phwjr\") pod \"keystone-bootstrap-nnnzp\" (UID: \"b36ab165-21eb-4968-a755-2cef353ce195\") " pod="openstack-kuttl-tests/keystone-bootstrap-nnnzp" Dec 03 12:42:48 crc kubenswrapper[5052]: I1203 12:42:48.049442 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/b36ab165-21eb-4968-a755-2cef353ce195-fernet-keys\") pod \"keystone-bootstrap-nnnzp\" (UID: \"b36ab165-21eb-4968-a755-2cef353ce195\") " pod="openstack-kuttl-tests/keystone-bootstrap-nnnzp" Dec 03 12:42:48 crc kubenswrapper[5052]: I1203 12:42:48.050119 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b36ab165-21eb-4968-a755-2cef353ce195-combined-ca-bundle\") pod \"keystone-bootstrap-nnnzp\" (UID: \"b36ab165-21eb-4968-a755-2cef353ce195\") " pod="openstack-kuttl-tests/keystone-bootstrap-nnnzp" Dec 03 12:42:48 crc kubenswrapper[5052]: I1203 12:42:48.050418 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/b36ab165-21eb-4968-a755-2cef353ce195-scripts\") pod \"keystone-bootstrap-nnnzp\" (UID: \"b36ab165-21eb-4968-a755-2cef353ce195\") " pod="openstack-kuttl-tests/keystone-bootstrap-nnnzp" Dec 03 12:42:48 crc kubenswrapper[5052]: I1203 12:42:48.050754 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/b36ab165-21eb-4968-a755-2cef353ce195-credential-keys\") pod \"keystone-bootstrap-nnnzp\" (UID: \"b36ab165-21eb-4968-a755-2cef353ce195\") " pod="openstack-kuttl-tests/keystone-bootstrap-nnnzp" Dec 03 12:42:48 crc kubenswrapper[5052]: I1203 12:42:48.051147 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/b36ab165-21eb-4968-a755-2cef353ce195-config-data\") pod \"keystone-bootstrap-nnnzp\" (UID: \"b36ab165-21eb-4968-a755-2cef353ce195\") " pod="openstack-kuttl-tests/keystone-bootstrap-nnnzp" Dec 03 12:42:48 crc kubenswrapper[5052]: I1203 12:42:48.085275 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-phwjr\" (UniqueName: \"kubernetes.io/projected/b36ab165-21eb-4968-a755-2cef353ce195-kube-api-access-phwjr\") pod \"keystone-bootstrap-nnnzp\" (UID: \"b36ab165-21eb-4968-a755-2cef353ce195\") " pod="openstack-kuttl-tests/keystone-bootstrap-nnnzp" Dec 03 12:42:48 crc kubenswrapper[5052]: I1203 12:42:48.148559 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-87bqn\" (UniqueName: \"kubernetes.io/projected/4c57f5e8-fe35-4f32-b0b1-a3dc5d0ccf33-kube-api-access-87bqn\") pod \"barbican-api-866c868748-bdl5k\" (UID: \"4c57f5e8-fe35-4f32-b0b1-a3dc5d0ccf33\") " pod="openstack-kuttl-tests/barbican-api-866c868748-bdl5k" Dec 03 12:42:48 crc kubenswrapper[5052]: I1203 12:42:48.148626 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4c57f5e8-fe35-4f32-b0b1-a3dc5d0ccf33-combined-ca-bundle\") pod \"barbican-api-866c868748-bdl5k\" (UID: \"4c57f5e8-fe35-4f32-b0b1-a3dc5d0ccf33\") " pod="openstack-kuttl-tests/barbican-api-866c868748-bdl5k" Dec 03 12:42:48 crc kubenswrapper[5052]: I1203 12:42:48.148662 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/4c57f5e8-fe35-4f32-b0b1-a3dc5d0ccf33-logs\") pod \"barbican-api-866c868748-bdl5k\" (UID: \"4c57f5e8-fe35-4f32-b0b1-a3dc5d0ccf33\") " pod="openstack-kuttl-tests/barbican-api-866c868748-bdl5k" Dec 03 12:42:48 crc kubenswrapper[5052]: I1203 12:42:48.148700 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/4c57f5e8-fe35-4f32-b0b1-a3dc5d0ccf33-config-data\") pod \"barbican-api-866c868748-bdl5k\" (UID: \"4c57f5e8-fe35-4f32-b0b1-a3dc5d0ccf33\") " pod="openstack-kuttl-tests/barbican-api-866c868748-bdl5k" Dec 03 12:42:48 crc kubenswrapper[5052]: I1203 12:42:48.148819 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/4c57f5e8-fe35-4f32-b0b1-a3dc5d0ccf33-config-data-custom\") pod \"barbican-api-866c868748-bdl5k\" (UID: \"4c57f5e8-fe35-4f32-b0b1-a3dc5d0ccf33\") " pod="openstack-kuttl-tests/barbican-api-866c868748-bdl5k" Dec 03 12:42:48 crc kubenswrapper[5052]: I1203 12:42:48.158013 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/4c57f5e8-fe35-4f32-b0b1-a3dc5d0ccf33-logs\") pod \"barbican-api-866c868748-bdl5k\" (UID: \"4c57f5e8-fe35-4f32-b0b1-a3dc5d0ccf33\") " pod="openstack-kuttl-tests/barbican-api-866c868748-bdl5k" Dec 03 12:42:48 crc kubenswrapper[5052]: I1203 12:42:48.169145 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/4c57f5e8-fe35-4f32-b0b1-a3dc5d0ccf33-config-data-custom\") pod \"barbican-api-866c868748-bdl5k\" (UID: \"4c57f5e8-fe35-4f32-b0b1-a3dc5d0ccf33\") " pod="openstack-kuttl-tests/barbican-api-866c868748-bdl5k" Dec 03 12:42:48 crc kubenswrapper[5052]: I1203 12:42:48.181539 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4c57f5e8-fe35-4f32-b0b1-a3dc5d0ccf33-combined-ca-bundle\") pod \"barbican-api-866c868748-bdl5k\" (UID: \"4c57f5e8-fe35-4f32-b0b1-a3dc5d0ccf33\") " pod="openstack-kuttl-tests/barbican-api-866c868748-bdl5k" Dec 03 12:42:48 crc kubenswrapper[5052]: I1203 12:42:48.182850 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/4c57f5e8-fe35-4f32-b0b1-a3dc5d0ccf33-config-data\") pod \"barbican-api-866c868748-bdl5k\" (UID: \"4c57f5e8-fe35-4f32-b0b1-a3dc5d0ccf33\") " pod="openstack-kuttl-tests/barbican-api-866c868748-bdl5k" Dec 03 12:42:48 crc kubenswrapper[5052]: I1203 12:42:48.225008 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-87bqn\" (UniqueName: \"kubernetes.io/projected/4c57f5e8-fe35-4f32-b0b1-a3dc5d0ccf33-kube-api-access-87bqn\") pod \"barbican-api-866c868748-bdl5k\" (UID: \"4c57f5e8-fe35-4f32-b0b1-a3dc5d0ccf33\") " pod="openstack-kuttl-tests/barbican-api-866c868748-bdl5k" Dec 03 12:42:48 crc kubenswrapper[5052]: I1203 12:42:48.287249 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-worker-65fcd76cf7-xhs44" Dec 03 12:42:48 crc kubenswrapper[5052]: I1203 12:42:48.316247 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-keystone-listener-769bc98c4b-mnz6r" Dec 03 12:42:48 crc kubenswrapper[5052]: I1203 12:42:48.501555 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-bootstrap-nnnzp" Dec 03 12:42:48 crc kubenswrapper[5052]: I1203 12:42:48.544210 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-api-866c868748-bdl5k" Dec 03 12:42:48 crc kubenswrapper[5052]: I1203 12:42:48.578851 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-db-sync-zf7km" Dec 03 12:42:48 crc kubenswrapper[5052]: I1203 12:42:48.669776 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/e8677b10-c98b-4d0e-a5d7-7af9fd914f12-config\") pod \"e8677b10-c98b-4d0e-a5d7-7af9fd914f12\" (UID: \"e8677b10-c98b-4d0e-a5d7-7af9fd914f12\") " Dec 03 12:42:48 crc kubenswrapper[5052]: I1203 12:42:48.669865 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qs82p\" (UniqueName: \"kubernetes.io/projected/e8677b10-c98b-4d0e-a5d7-7af9fd914f12-kube-api-access-qs82p\") pod \"e8677b10-c98b-4d0e-a5d7-7af9fd914f12\" (UID: \"e8677b10-c98b-4d0e-a5d7-7af9fd914f12\") " Dec 03 12:42:48 crc kubenswrapper[5052]: I1203 12:42:48.670042 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e8677b10-c98b-4d0e-a5d7-7af9fd914f12-combined-ca-bundle\") pod \"e8677b10-c98b-4d0e-a5d7-7af9fd914f12\" (UID: \"e8677b10-c98b-4d0e-a5d7-7af9fd914f12\") " Dec 03 12:42:48 crc kubenswrapper[5052]: I1203 12:42:48.686990 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e8677b10-c98b-4d0e-a5d7-7af9fd914f12-kube-api-access-qs82p" (OuterVolumeSpecName: "kube-api-access-qs82p") pod "e8677b10-c98b-4d0e-a5d7-7af9fd914f12" (UID: "e8677b10-c98b-4d0e-a5d7-7af9fd914f12"). InnerVolumeSpecName "kube-api-access-qs82p". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:42:48 crc kubenswrapper[5052]: I1203 12:42:48.708447 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e8677b10-c98b-4d0e-a5d7-7af9fd914f12-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "e8677b10-c98b-4d0e-a5d7-7af9fd914f12" (UID: "e8677b10-c98b-4d0e-a5d7-7af9fd914f12"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:42:48 crc kubenswrapper[5052]: I1203 12:42:48.718574 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e8677b10-c98b-4d0e-a5d7-7af9fd914f12-config" (OuterVolumeSpecName: "config") pod "e8677b10-c98b-4d0e-a5d7-7af9fd914f12" (UID: "e8677b10-c98b-4d0e-a5d7-7af9fd914f12"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:42:48 crc kubenswrapper[5052]: I1203 12:42:48.718893 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:42:48 crc kubenswrapper[5052]: I1203 12:42:48.776646 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-qs82p\" (UniqueName: \"kubernetes.io/projected/e8677b10-c98b-4d0e-a5d7-7af9fd914f12-kube-api-access-qs82p\") on node \"crc\" DevicePath \"\"" Dec 03 12:42:48 crc kubenswrapper[5052]: I1203 12:42:48.778468 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e8677b10-c98b-4d0e-a5d7-7af9fd914f12-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:42:48 crc kubenswrapper[5052]: I1203 12:42:48.778495 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/e8677b10-c98b-4d0e-a5d7-7af9fd914f12-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:42:48 crc kubenswrapper[5052]: I1203 12:42:48.793155 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:42:48 crc kubenswrapper[5052]: I1203 12:42:48.793822 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"133758ce-c13c-4ac1-b672-49412b22ec08","Type":"ContainerDied","Data":"05cb7f3a97655faff11f433f8381c8e973e0e6f53c1ea19005e152808d972518"} Dec 03 12:42:48 crc kubenswrapper[5052]: I1203 12:42:48.793869 5052 scope.go:117] "RemoveContainer" containerID="8ecebefb263fcd4c52ad89050bdb294b17d15a1d16e9932448c14d8cd8f988b1" Dec 03 12:42:48 crc kubenswrapper[5052]: I1203 12:42:48.803961 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-db-sync-zf7km" event={"ID":"e8677b10-c98b-4d0e-a5d7-7af9fd914f12","Type":"ContainerDied","Data":"697fdba4e1d936cce1055e9177022b8dadce7e484cc927341bbbe9b8b8d43b41"} Dec 03 12:42:48 crc kubenswrapper[5052]: I1203 12:42:48.804015 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="697fdba4e1d936cce1055e9177022b8dadce7e484cc927341bbbe9b8b8d43b41" Dec 03 12:42:48 crc kubenswrapper[5052]: I1203 12:42:48.804140 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-db-sync-zf7km" Dec 03 12:42:48 crc kubenswrapper[5052]: I1203 12:42:48.820598 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"8011917c-0ce8-4c83-b157-2d8538790e17","Type":"ContainerStarted","Data":"c38db48afa4b344c32d00e4db4d452c8f75d832d0c682df5c1a48a83e977e312"} Dec 03 12:42:48 crc kubenswrapper[5052]: I1203 12:42:48.843807 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/placement-56b4b79f66-gfsxn"] Dec 03 12:42:48 crc kubenswrapper[5052]: I1203 12:42:48.854621 5052 scope.go:117] "RemoveContainer" containerID="050e4f507ff648b4d9a847ec84654e743d8a6e6d3a90811c85901f7f8a74f34a" Dec 03 12:42:48 crc kubenswrapper[5052]: I1203 12:42:48.879363 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/133758ce-c13c-4ac1-b672-49412b22ec08-scripts\") pod \"133758ce-c13c-4ac1-b672-49412b22ec08\" (UID: \"133758ce-c13c-4ac1-b672-49412b22ec08\") " Dec 03 12:42:48 crc kubenswrapper[5052]: I1203 12:42:48.879497 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/133758ce-c13c-4ac1-b672-49412b22ec08-run-httpd\") pod \"133758ce-c13c-4ac1-b672-49412b22ec08\" (UID: \"133758ce-c13c-4ac1-b672-49412b22ec08\") " Dec 03 12:42:48 crc kubenswrapper[5052]: I1203 12:42:48.879560 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/133758ce-c13c-4ac1-b672-49412b22ec08-sg-core-conf-yaml\") pod \"133758ce-c13c-4ac1-b672-49412b22ec08\" (UID: \"133758ce-c13c-4ac1-b672-49412b22ec08\") " Dec 03 12:42:48 crc kubenswrapper[5052]: I1203 12:42:48.879678 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/133758ce-c13c-4ac1-b672-49412b22ec08-config-data\") pod \"133758ce-c13c-4ac1-b672-49412b22ec08\" (UID: \"133758ce-c13c-4ac1-b672-49412b22ec08\") " Dec 03 12:42:48 crc kubenswrapper[5052]: I1203 12:42:48.879711 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xrftx\" (UniqueName: \"kubernetes.io/projected/133758ce-c13c-4ac1-b672-49412b22ec08-kube-api-access-xrftx\") pod \"133758ce-c13c-4ac1-b672-49412b22ec08\" (UID: \"133758ce-c13c-4ac1-b672-49412b22ec08\") " Dec 03 12:42:48 crc kubenswrapper[5052]: I1203 12:42:48.879735 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/133758ce-c13c-4ac1-b672-49412b22ec08-combined-ca-bundle\") pod \"133758ce-c13c-4ac1-b672-49412b22ec08\" (UID: \"133758ce-c13c-4ac1-b672-49412b22ec08\") " Dec 03 12:42:48 crc kubenswrapper[5052]: I1203 12:42:48.879782 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/133758ce-c13c-4ac1-b672-49412b22ec08-log-httpd\") pod \"133758ce-c13c-4ac1-b672-49412b22ec08\" (UID: \"133758ce-c13c-4ac1-b672-49412b22ec08\") " Dec 03 12:42:48 crc kubenswrapper[5052]: I1203 12:42:48.880262 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/133758ce-c13c-4ac1-b672-49412b22ec08-run-httpd" (OuterVolumeSpecName: "run-httpd") pod "133758ce-c13c-4ac1-b672-49412b22ec08" (UID: "133758ce-c13c-4ac1-b672-49412b22ec08"). InnerVolumeSpecName "run-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:42:48 crc kubenswrapper[5052]: I1203 12:42:48.881556 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/133758ce-c13c-4ac1-b672-49412b22ec08-log-httpd" (OuterVolumeSpecName: "log-httpd") pod "133758ce-c13c-4ac1-b672-49412b22ec08" (UID: "133758ce-c13c-4ac1-b672-49412b22ec08"). InnerVolumeSpecName "log-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:42:48 crc kubenswrapper[5052]: I1203 12:42:48.900418 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/133758ce-c13c-4ac1-b672-49412b22ec08-scripts" (OuterVolumeSpecName: "scripts") pod "133758ce-c13c-4ac1-b672-49412b22ec08" (UID: "133758ce-c13c-4ac1-b672-49412b22ec08"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:42:48 crc kubenswrapper[5052]: I1203 12:42:48.900855 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/133758ce-c13c-4ac1-b672-49412b22ec08-kube-api-access-xrftx" (OuterVolumeSpecName: "kube-api-access-xrftx") pod "133758ce-c13c-4ac1-b672-49412b22ec08" (UID: "133758ce-c13c-4ac1-b672-49412b22ec08"). InnerVolumeSpecName "kube-api-access-xrftx". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:42:48 crc kubenswrapper[5052]: I1203 12:42:48.921852 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/133758ce-c13c-4ac1-b672-49412b22ec08-sg-core-conf-yaml" (OuterVolumeSpecName: "sg-core-conf-yaml") pod "133758ce-c13c-4ac1-b672-49412b22ec08" (UID: "133758ce-c13c-4ac1-b672-49412b22ec08"). InnerVolumeSpecName "sg-core-conf-yaml". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:42:48 crc kubenswrapper[5052]: I1203 12:42:48.929863 5052 scope.go:117] "RemoveContainer" containerID="6d5877335791adc8136d132f76bce6689455c186a9abe604beb4d46b25a2646c" Dec 03 12:42:48 crc kubenswrapper[5052]: I1203 12:42:48.982318 5052 reconciler_common.go:293] "Volume detached for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/133758ce-c13c-4ac1-b672-49412b22ec08-run-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:42:48 crc kubenswrapper[5052]: I1203 12:42:48.982358 5052 reconciler_common.go:293] "Volume detached for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/133758ce-c13c-4ac1-b672-49412b22ec08-sg-core-conf-yaml\") on node \"crc\" DevicePath \"\"" Dec 03 12:42:48 crc kubenswrapper[5052]: I1203 12:42:48.982404 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-xrftx\" (UniqueName: \"kubernetes.io/projected/133758ce-c13c-4ac1-b672-49412b22ec08-kube-api-access-xrftx\") on node \"crc\" DevicePath \"\"" Dec 03 12:42:48 crc kubenswrapper[5052]: I1203 12:42:48.982418 5052 reconciler_common.go:293] "Volume detached for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/133758ce-c13c-4ac1-b672-49412b22ec08-log-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:42:48 crc kubenswrapper[5052]: I1203 12:42:48.982430 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/133758ce-c13c-4ac1-b672-49412b22ec08-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:42:48 crc kubenswrapper[5052]: I1203 12:42:48.994828 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-keystone-listener-769bc98c4b-mnz6r"] Dec 03 12:42:49 crc kubenswrapper[5052]: I1203 12:42:49.000962 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/133758ce-c13c-4ac1-b672-49412b22ec08-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "133758ce-c13c-4ac1-b672-49412b22ec08" (UID: "133758ce-c13c-4ac1-b672-49412b22ec08"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:42:49 crc kubenswrapper[5052]: I1203 12:42:49.007818 5052 scope.go:117] "RemoveContainer" containerID="da6fb06ac7ebc0adff26fc1ccb94fc8314f3ea98f662feb777cf3e824d4d161b" Dec 03 12:42:49 crc kubenswrapper[5052]: I1203 12:42:49.043821 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/133758ce-c13c-4ac1-b672-49412b22ec08-config-data" (OuterVolumeSpecName: "config-data") pod "133758ce-c13c-4ac1-b672-49412b22ec08" (UID: "133758ce-c13c-4ac1-b672-49412b22ec08"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:42:49 crc kubenswrapper[5052]: I1203 12:42:49.082300 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-worker-65fcd76cf7-xhs44"] Dec 03 12:42:49 crc kubenswrapper[5052]: I1203 12:42:49.084685 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/133758ce-c13c-4ac1-b672-49412b22ec08-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:42:49 crc kubenswrapper[5052]: I1203 12:42:49.084709 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/133758ce-c13c-4ac1-b672-49412b22ec08-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:42:49 crc kubenswrapper[5052]: W1203 12:42:49.112144 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod92290cff_ed52_4a47_80e9_720bfd9f3dd8.slice/crio-df24c1fdafb53577fadc51b0d23110c137497310f9a2ba7085895b957896da0a WatchSource:0}: Error finding container df24c1fdafb53577fadc51b0d23110c137497310f9a2ba7085895b957896da0a: Status 404 returned error can't find the container with id df24c1fdafb53577fadc51b0d23110c137497310f9a2ba7085895b957896da0a Dec 03 12:42:49 crc kubenswrapper[5052]: I1203 12:42:49.313486 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-bootstrap-nnnzp"] Dec 03 12:42:49 crc kubenswrapper[5052]: I1203 12:42:49.329583 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:42:49 crc kubenswrapper[5052]: W1203 12:42:49.334197 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb36ab165_21eb_4968_a755_2cef353ce195.slice/crio-6d979e63f07840ce2d3a84871b38104d458c9d29466a10ab20d131ed6f7d4445 WatchSource:0}: Error finding container 6d979e63f07840ce2d3a84871b38104d458c9d29466a10ab20d131ed6f7d4445: Status 404 returned error can't find the container with id 6d979e63f07840ce2d3a84871b38104d458c9d29466a10ab20d131ed6f7d4445 Dec 03 12:42:49 crc kubenswrapper[5052]: I1203 12:42:49.352653 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:42:49 crc kubenswrapper[5052]: I1203 12:42:49.361306 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:42:49 crc kubenswrapper[5052]: E1203 12:42:49.362365 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="133758ce-c13c-4ac1-b672-49412b22ec08" containerName="sg-core" Dec 03 12:42:49 crc kubenswrapper[5052]: I1203 12:42:49.362393 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="133758ce-c13c-4ac1-b672-49412b22ec08" containerName="sg-core" Dec 03 12:42:49 crc kubenswrapper[5052]: E1203 12:42:49.362409 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="e8677b10-c98b-4d0e-a5d7-7af9fd914f12" containerName="neutron-db-sync" Dec 03 12:42:49 crc kubenswrapper[5052]: I1203 12:42:49.362415 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="e8677b10-c98b-4d0e-a5d7-7af9fd914f12" containerName="neutron-db-sync" Dec 03 12:42:49 crc kubenswrapper[5052]: E1203 12:42:49.362426 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="133758ce-c13c-4ac1-b672-49412b22ec08" containerName="ceilometer-central-agent" Dec 03 12:42:49 crc kubenswrapper[5052]: I1203 12:42:49.362432 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="133758ce-c13c-4ac1-b672-49412b22ec08" containerName="ceilometer-central-agent" Dec 03 12:42:49 crc kubenswrapper[5052]: E1203 12:42:49.362441 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="133758ce-c13c-4ac1-b672-49412b22ec08" containerName="proxy-httpd" Dec 03 12:42:49 crc kubenswrapper[5052]: I1203 12:42:49.362446 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="133758ce-c13c-4ac1-b672-49412b22ec08" containerName="proxy-httpd" Dec 03 12:42:49 crc kubenswrapper[5052]: E1203 12:42:49.362469 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="133758ce-c13c-4ac1-b672-49412b22ec08" containerName="ceilometer-notification-agent" Dec 03 12:42:49 crc kubenswrapper[5052]: I1203 12:42:49.362475 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="133758ce-c13c-4ac1-b672-49412b22ec08" containerName="ceilometer-notification-agent" Dec 03 12:42:49 crc kubenswrapper[5052]: I1203 12:42:49.362661 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="133758ce-c13c-4ac1-b672-49412b22ec08" containerName="sg-core" Dec 03 12:42:49 crc kubenswrapper[5052]: I1203 12:42:49.362684 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="133758ce-c13c-4ac1-b672-49412b22ec08" containerName="ceilometer-notification-agent" Dec 03 12:42:49 crc kubenswrapper[5052]: I1203 12:42:49.362693 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="133758ce-c13c-4ac1-b672-49412b22ec08" containerName="ceilometer-central-agent" Dec 03 12:42:49 crc kubenswrapper[5052]: I1203 12:42:49.362702 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="e8677b10-c98b-4d0e-a5d7-7af9fd914f12" containerName="neutron-db-sync" Dec 03 12:42:49 crc kubenswrapper[5052]: I1203 12:42:49.362710 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="133758ce-c13c-4ac1-b672-49412b22ec08" containerName="proxy-httpd" Dec 03 12:42:49 crc kubenswrapper[5052]: I1203 12:42:49.369416 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:42:49 crc kubenswrapper[5052]: I1203 12:42:49.372180 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-scripts" Dec 03 12:42:49 crc kubenswrapper[5052]: I1203 12:42:49.372588 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-config-data" Dec 03 12:42:49 crc kubenswrapper[5052]: I1203 12:42:49.394746 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:42:49 crc kubenswrapper[5052]: I1203 12:42:49.430242 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-api-866c868748-bdl5k"] Dec 03 12:42:49 crc kubenswrapper[5052]: W1203 12:42:49.456325 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4c57f5e8_fe35_4f32_b0b1_a3dc5d0ccf33.slice/crio-8c7001d153f6c223ee493c5c05f995b394beefdef2772f671180aa50196e6178 WatchSource:0}: Error finding container 8c7001d153f6c223ee493c5c05f995b394beefdef2772f671180aa50196e6178: Status 404 returned error can't find the container with id 8c7001d153f6c223ee493c5c05f995b394beefdef2772f671180aa50196e6178 Dec 03 12:42:49 crc kubenswrapper[5052]: I1203 12:42:49.503601 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/50a91cd1-d2b0-48ad-954c-17c18007e891-config-data\") pod \"ceilometer-0\" (UID: \"50a91cd1-d2b0-48ad-954c-17c18007e891\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:42:49 crc kubenswrapper[5052]: I1203 12:42:49.503661 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/50a91cd1-d2b0-48ad-954c-17c18007e891-run-httpd\") pod \"ceilometer-0\" (UID: \"50a91cd1-d2b0-48ad-954c-17c18007e891\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:42:49 crc kubenswrapper[5052]: I1203 12:42:49.503846 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mbpg7\" (UniqueName: \"kubernetes.io/projected/50a91cd1-d2b0-48ad-954c-17c18007e891-kube-api-access-mbpg7\") pod \"ceilometer-0\" (UID: \"50a91cd1-d2b0-48ad-954c-17c18007e891\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:42:49 crc kubenswrapper[5052]: I1203 12:42:49.503899 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/50a91cd1-d2b0-48ad-954c-17c18007e891-scripts\") pod \"ceilometer-0\" (UID: \"50a91cd1-d2b0-48ad-954c-17c18007e891\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:42:49 crc kubenswrapper[5052]: I1203 12:42:49.503984 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/50a91cd1-d2b0-48ad-954c-17c18007e891-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"50a91cd1-d2b0-48ad-954c-17c18007e891\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:42:49 crc kubenswrapper[5052]: I1203 12:42:49.504033 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/50a91cd1-d2b0-48ad-954c-17c18007e891-log-httpd\") pod \"ceilometer-0\" (UID: \"50a91cd1-d2b0-48ad-954c-17c18007e891\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:42:49 crc kubenswrapper[5052]: I1203 12:42:49.504062 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/50a91cd1-d2b0-48ad-954c-17c18007e891-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"50a91cd1-d2b0-48ad-954c-17c18007e891\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:42:49 crc kubenswrapper[5052]: I1203 12:42:49.545063 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-db-sync-2ttlg" Dec 03 12:42:49 crc kubenswrapper[5052]: I1203 12:42:49.607193 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/50a91cd1-d2b0-48ad-954c-17c18007e891-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"50a91cd1-d2b0-48ad-954c-17c18007e891\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:42:49 crc kubenswrapper[5052]: I1203 12:42:49.607273 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/50a91cd1-d2b0-48ad-954c-17c18007e891-config-data\") pod \"ceilometer-0\" (UID: \"50a91cd1-d2b0-48ad-954c-17c18007e891\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:42:49 crc kubenswrapper[5052]: I1203 12:42:49.607303 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/50a91cd1-d2b0-48ad-954c-17c18007e891-run-httpd\") pod \"ceilometer-0\" (UID: \"50a91cd1-d2b0-48ad-954c-17c18007e891\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:42:49 crc kubenswrapper[5052]: I1203 12:42:49.607332 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-mbpg7\" (UniqueName: \"kubernetes.io/projected/50a91cd1-d2b0-48ad-954c-17c18007e891-kube-api-access-mbpg7\") pod \"ceilometer-0\" (UID: \"50a91cd1-d2b0-48ad-954c-17c18007e891\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:42:49 crc kubenswrapper[5052]: I1203 12:42:49.607354 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/50a91cd1-d2b0-48ad-954c-17c18007e891-scripts\") pod \"ceilometer-0\" (UID: \"50a91cd1-d2b0-48ad-954c-17c18007e891\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:42:49 crc kubenswrapper[5052]: I1203 12:42:49.607386 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/50a91cd1-d2b0-48ad-954c-17c18007e891-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"50a91cd1-d2b0-48ad-954c-17c18007e891\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:42:49 crc kubenswrapper[5052]: I1203 12:42:49.607412 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/50a91cd1-d2b0-48ad-954c-17c18007e891-log-httpd\") pod \"ceilometer-0\" (UID: \"50a91cd1-d2b0-48ad-954c-17c18007e891\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:42:49 crc kubenswrapper[5052]: I1203 12:42:49.607768 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/50a91cd1-d2b0-48ad-954c-17c18007e891-log-httpd\") pod \"ceilometer-0\" (UID: \"50a91cd1-d2b0-48ad-954c-17c18007e891\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:42:49 crc kubenswrapper[5052]: I1203 12:42:49.611071 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/50a91cd1-d2b0-48ad-954c-17c18007e891-run-httpd\") pod \"ceilometer-0\" (UID: \"50a91cd1-d2b0-48ad-954c-17c18007e891\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:42:49 crc kubenswrapper[5052]: I1203 12:42:49.616670 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/50a91cd1-d2b0-48ad-954c-17c18007e891-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"50a91cd1-d2b0-48ad-954c-17c18007e891\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:42:49 crc kubenswrapper[5052]: I1203 12:42:49.617565 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/50a91cd1-d2b0-48ad-954c-17c18007e891-scripts\") pod \"ceilometer-0\" (UID: \"50a91cd1-d2b0-48ad-954c-17c18007e891\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:42:49 crc kubenswrapper[5052]: I1203 12:42:49.620873 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/50a91cd1-d2b0-48ad-954c-17c18007e891-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"50a91cd1-d2b0-48ad-954c-17c18007e891\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:42:49 crc kubenswrapper[5052]: I1203 12:42:49.639253 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/50a91cd1-d2b0-48ad-954c-17c18007e891-config-data\") pod \"ceilometer-0\" (UID: \"50a91cd1-d2b0-48ad-954c-17c18007e891\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:42:49 crc kubenswrapper[5052]: I1203 12:42:49.644295 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-mbpg7\" (UniqueName: \"kubernetes.io/projected/50a91cd1-d2b0-48ad-954c-17c18007e891-kube-api-access-mbpg7\") pod \"ceilometer-0\" (UID: \"50a91cd1-d2b0-48ad-954c-17c18007e891\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:42:49 crc kubenswrapper[5052]: I1203 12:42:49.701231 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="133758ce-c13c-4ac1-b672-49412b22ec08" path="/var/lib/kubelet/pods/133758ce-c13c-4ac1-b672-49412b22ec08/volumes" Dec 03 12:42:49 crc kubenswrapper[5052]: I1203 12:42:49.702186 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8ed7530c-9333-446a-8972-7eee88f76d82" path="/var/lib/kubelet/pods/8ed7530c-9333-446a-8972-7eee88f76d82/volumes" Dec 03 12:42:49 crc kubenswrapper[5052]: I1203 12:42:49.709436 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/1de481f1-0c54-48c3-b112-659497d86792-etc-machine-id\") pod \"1de481f1-0c54-48c3-b112-659497d86792\" (UID: \"1de481f1-0c54-48c3-b112-659497d86792\") " Dec 03 12:42:49 crc kubenswrapper[5052]: I1203 12:42:49.709500 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1de481f1-0c54-48c3-b112-659497d86792-combined-ca-bundle\") pod \"1de481f1-0c54-48c3-b112-659497d86792\" (UID: \"1de481f1-0c54-48c3-b112-659497d86792\") " Dec 03 12:42:49 crc kubenswrapper[5052]: I1203 12:42:49.709625 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/1de481f1-0c54-48c3-b112-659497d86792-scripts\") pod \"1de481f1-0c54-48c3-b112-659497d86792\" (UID: \"1de481f1-0c54-48c3-b112-659497d86792\") " Dec 03 12:42:49 crc kubenswrapper[5052]: I1203 12:42:49.709656 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/1de481f1-0c54-48c3-b112-659497d86792-db-sync-config-data\") pod \"1de481f1-0c54-48c3-b112-659497d86792\" (UID: \"1de481f1-0c54-48c3-b112-659497d86792\") " Dec 03 12:42:49 crc kubenswrapper[5052]: I1203 12:42:49.709683 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/1de481f1-0c54-48c3-b112-659497d86792-config-data\") pod \"1de481f1-0c54-48c3-b112-659497d86792\" (UID: \"1de481f1-0c54-48c3-b112-659497d86792\") " Dec 03 12:42:49 crc kubenswrapper[5052]: I1203 12:42:49.709686 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/1de481f1-0c54-48c3-b112-659497d86792-etc-machine-id" (OuterVolumeSpecName: "etc-machine-id") pod "1de481f1-0c54-48c3-b112-659497d86792" (UID: "1de481f1-0c54-48c3-b112-659497d86792"). InnerVolumeSpecName "etc-machine-id". PluginName "kubernetes.io/host-path", VolumeGidValue "" Dec 03 12:42:49 crc kubenswrapper[5052]: I1203 12:42:49.709759 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4fx5r\" (UniqueName: \"kubernetes.io/projected/1de481f1-0c54-48c3-b112-659497d86792-kube-api-access-4fx5r\") pod \"1de481f1-0c54-48c3-b112-659497d86792\" (UID: \"1de481f1-0c54-48c3-b112-659497d86792\") " Dec 03 12:42:49 crc kubenswrapper[5052]: I1203 12:42:49.710111 5052 reconciler_common.go:293] "Volume detached for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/1de481f1-0c54-48c3-b112-659497d86792-etc-machine-id\") on node \"crc\" DevicePath \"\"" Dec 03 12:42:49 crc kubenswrapper[5052]: I1203 12:42:49.746859 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1de481f1-0c54-48c3-b112-659497d86792-kube-api-access-4fx5r" (OuterVolumeSpecName: "kube-api-access-4fx5r") pod "1de481f1-0c54-48c3-b112-659497d86792" (UID: "1de481f1-0c54-48c3-b112-659497d86792"). InnerVolumeSpecName "kube-api-access-4fx5r". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:42:49 crc kubenswrapper[5052]: I1203 12:42:49.747236 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1de481f1-0c54-48c3-b112-659497d86792-db-sync-config-data" (OuterVolumeSpecName: "db-sync-config-data") pod "1de481f1-0c54-48c3-b112-659497d86792" (UID: "1de481f1-0c54-48c3-b112-659497d86792"). InnerVolumeSpecName "db-sync-config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:42:49 crc kubenswrapper[5052]: I1203 12:42:49.753106 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1de481f1-0c54-48c3-b112-659497d86792-scripts" (OuterVolumeSpecName: "scripts") pod "1de481f1-0c54-48c3-b112-659497d86792" (UID: "1de481f1-0c54-48c3-b112-659497d86792"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:42:49 crc kubenswrapper[5052]: I1203 12:42:49.840451 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1de481f1-0c54-48c3-b112-659497d86792-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "1de481f1-0c54-48c3-b112-659497d86792" (UID: "1de481f1-0c54-48c3-b112-659497d86792"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:42:49 crc kubenswrapper[5052]: I1203 12:42:49.845870 5052 reconciler_common.go:293] "Volume detached for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/1de481f1-0c54-48c3-b112-659497d86792-db-sync-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:42:49 crc kubenswrapper[5052]: I1203 12:42:49.846300 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:42:49 crc kubenswrapper[5052]: I1203 12:42:49.846710 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-4fx5r\" (UniqueName: \"kubernetes.io/projected/1de481f1-0c54-48c3-b112-659497d86792-kube-api-access-4fx5r\") on node \"crc\" DevicePath \"\"" Dec 03 12:42:49 crc kubenswrapper[5052]: I1203 12:42:49.852197 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1de481f1-0c54-48c3-b112-659497d86792-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:42:49 crc kubenswrapper[5052]: I1203 12:42:49.856044 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/1de481f1-0c54-48c3-b112-659497d86792-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:42:49 crc kubenswrapper[5052]: I1203 12:42:49.913934 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1de481f1-0c54-48c3-b112-659497d86792-config-data" (OuterVolumeSpecName: "config-data") pod "1de481f1-0c54-48c3-b112-659497d86792" (UID: "1de481f1-0c54-48c3-b112-659497d86792"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:42:49 crc kubenswrapper[5052]: I1203 12:42:49.938655 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-56b4b79f66-gfsxn" event={"ID":"76534e0c-3b73-4ede-bc24-b12c7f8ea4db","Type":"ContainerStarted","Data":"8b321f78db963a5c8fd8a06d8d437e337dea72a0472cde308cbba093a65a6e60"} Dec 03 12:42:49 crc kubenswrapper[5052]: I1203 12:42:49.939023 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-56b4b79f66-gfsxn" event={"ID":"76534e0c-3b73-4ede-bc24-b12c7f8ea4db","Type":"ContainerStarted","Data":"96c237ac04f7109dc8403eb71f6ee5b59681e2a19e6ad3fa6f2004a8a43b803f"} Dec 03 12:42:49 crc kubenswrapper[5052]: I1203 12:42:49.943730 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"24ccc68c-26a9-485e-bf26-512dac006b61","Type":"ContainerStarted","Data":"23f9262d3a7c543b67221300c14860510bcdb1b097fab4de7283f6469298206b"} Dec 03 12:42:49 crc kubenswrapper[5052]: I1203 12:42:49.960761 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/1de481f1-0c54-48c3-b112-659497d86792-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:42:49 crc kubenswrapper[5052]: I1203 12:42:49.968135 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/neutron-64b7d8689c-p2dmd"] Dec 03 12:42:49 crc kubenswrapper[5052]: E1203 12:42:49.970479 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="1de481f1-0c54-48c3-b112-659497d86792" containerName="cinder-db-sync" Dec 03 12:42:49 crc kubenswrapper[5052]: I1203 12:42:49.972610 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="1de481f1-0c54-48c3-b112-659497d86792" containerName="cinder-db-sync" Dec 03 12:42:49 crc kubenswrapper[5052]: I1203 12:42:49.972875 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="1de481f1-0c54-48c3-b112-659497d86792" containerName="cinder-db-sync" Dec 03 12:42:49 crc kubenswrapper[5052]: I1203 12:42:49.979589 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-64b7d8689c-p2dmd"] Dec 03 12:42:49 crc kubenswrapper[5052]: I1203 12:42:49.994606 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-64b7d8689c-p2dmd" Dec 03 12:42:50 crc kubenswrapper[5052]: I1203 12:42:50.001369 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"neutron-neutron-dockercfg-4zn4q" Dec 03 12:42:50 crc kubenswrapper[5052]: I1203 12:42:50.001425 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-neutron-ovndbs" Dec 03 12:42:50 crc kubenswrapper[5052]: I1203 12:42:50.001602 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"neutron-httpd-config" Dec 03 12:42:50 crc kubenswrapper[5052]: I1203 12:42:50.001679 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"neutron-config" Dec 03 12:42:50 crc kubenswrapper[5052]: I1203 12:42:50.026715 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-keystone-listener-769bc98c4b-mnz6r" event={"ID":"1d187c5a-7745-447f-a8e2-06c7528e5be0","Type":"ContainerStarted","Data":"4208df0572c14c762c41a0bceeeb54f12d57e83a373e6e10caba8b5459e15ce7"} Dec 03 12:42:50 crc kubenswrapper[5052]: I1203 12:42:50.026765 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-keystone-listener-769bc98c4b-mnz6r" event={"ID":"1d187c5a-7745-447f-a8e2-06c7528e5be0","Type":"ContainerStarted","Data":"feac048847ae2836573d829abc28eeb03bc068950854ca5d2ef02e61ea8acdbb"} Dec 03 12:42:50 crc kubenswrapper[5052]: I1203 12:42:50.042951 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-worker-65fcd76cf7-xhs44" event={"ID":"92290cff-ed52-4a47-80e9-720bfd9f3dd8","Type":"ContainerStarted","Data":"6da49bc767ca3706e94a6d9f283174f99fca40c8d0d7ff4717a94a22b54ce36c"} Dec 03 12:42:50 crc kubenswrapper[5052]: I1203 12:42:50.043005 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-worker-65fcd76cf7-xhs44" event={"ID":"92290cff-ed52-4a47-80e9-720bfd9f3dd8","Type":"ContainerStarted","Data":"df24c1fdafb53577fadc51b0d23110c137497310f9a2ba7085895b957896da0a"} Dec 03 12:42:50 crc kubenswrapper[5052]: I1203 12:42:50.057825 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-db-sync-2ttlg" event={"ID":"1de481f1-0c54-48c3-b112-659497d86792","Type":"ContainerDied","Data":"08fac75c2b68b52925dc0dfecb563961552d86edff21aed10a901d020990fa3b"} Dec 03 12:42:50 crc kubenswrapper[5052]: I1203 12:42:50.057863 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="08fac75c2b68b52925dc0dfecb563961552d86edff21aed10a901d020990fa3b" Dec 03 12:42:50 crc kubenswrapper[5052]: I1203 12:42:50.057923 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-db-sync-2ttlg" Dec 03 12:42:50 crc kubenswrapper[5052]: I1203 12:42:50.064271 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/c0261f94-be97-4cce-9eac-362d42248260-httpd-config\") pod \"neutron-64b7d8689c-p2dmd\" (UID: \"c0261f94-be97-4cce-9eac-362d42248260\") " pod="openstack-kuttl-tests/neutron-64b7d8689c-p2dmd" Dec 03 12:42:50 crc kubenswrapper[5052]: I1203 12:42:50.064366 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/c0261f94-be97-4cce-9eac-362d42248260-config\") pod \"neutron-64b7d8689c-p2dmd\" (UID: \"c0261f94-be97-4cce-9eac-362d42248260\") " pod="openstack-kuttl-tests/neutron-64b7d8689c-p2dmd" Dec 03 12:42:50 crc kubenswrapper[5052]: I1203 12:42:50.064438 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q5qrs\" (UniqueName: \"kubernetes.io/projected/c0261f94-be97-4cce-9eac-362d42248260-kube-api-access-q5qrs\") pod \"neutron-64b7d8689c-p2dmd\" (UID: \"c0261f94-be97-4cce-9eac-362d42248260\") " pod="openstack-kuttl-tests/neutron-64b7d8689c-p2dmd" Dec 03 12:42:50 crc kubenswrapper[5052]: I1203 12:42:50.064467 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c0261f94-be97-4cce-9eac-362d42248260-combined-ca-bundle\") pod \"neutron-64b7d8689c-p2dmd\" (UID: \"c0261f94-be97-4cce-9eac-362d42248260\") " pod="openstack-kuttl-tests/neutron-64b7d8689c-p2dmd" Dec 03 12:42:50 crc kubenswrapper[5052]: I1203 12:42:50.064564 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/c0261f94-be97-4cce-9eac-362d42248260-ovndb-tls-certs\") pod \"neutron-64b7d8689c-p2dmd\" (UID: \"c0261f94-be97-4cce-9eac-362d42248260\") " pod="openstack-kuttl-tests/neutron-64b7d8689c-p2dmd" Dec 03 12:42:50 crc kubenswrapper[5052]: I1203 12:42:50.069137 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"8011917c-0ce8-4c83-b157-2d8538790e17","Type":"ContainerStarted","Data":"d871ecf0facb775d7ef4d9dacb2a2746b353a7fee55e6f036959cc141b2fcf0e"} Dec 03 12:42:50 crc kubenswrapper[5052]: I1203 12:42:50.080780 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 12:42:50 crc kubenswrapper[5052]: I1203 12:42:50.082844 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:42:50 crc kubenswrapper[5052]: I1203 12:42:50.085439 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cinder-config-data" Dec 03 12:42:50 crc kubenswrapper[5052]: I1203 12:42:50.085605 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cinder-scheduler-config-data" Dec 03 12:42:50 crc kubenswrapper[5052]: I1203 12:42:50.085816 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-bootstrap-nnnzp" event={"ID":"b36ab165-21eb-4968-a755-2cef353ce195","Type":"ContainerStarted","Data":"f48e73f815035e727aa3a398a4b9910157004cad319377fa765a601efb172b96"} Dec 03 12:42:50 crc kubenswrapper[5052]: I1203 12:42:50.085846 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-bootstrap-nnnzp" event={"ID":"b36ab165-21eb-4968-a755-2cef353ce195","Type":"ContainerStarted","Data":"6d979e63f07840ce2d3a84871b38104d458c9d29466a10ab20d131ed6f7d4445"} Dec 03 12:42:50 crc kubenswrapper[5052]: I1203 12:42:50.086238 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cinder-scripts" Dec 03 12:42:50 crc kubenswrapper[5052]: I1203 12:42:50.086452 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cinder-cinder-dockercfg-wgjw7" Dec 03 12:42:50 crc kubenswrapper[5052]: I1203 12:42:50.093064 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 12:42:50 crc kubenswrapper[5052]: I1203 12:42:50.098691 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-866c868748-bdl5k" event={"ID":"4c57f5e8-fe35-4f32-b0b1-a3dc5d0ccf33","Type":"ContainerStarted","Data":"bd1fd8ba880b814dd955b69768ac28222dd0eeff96580f6fa7d3c52c5b458a07"} Dec 03 12:42:50 crc kubenswrapper[5052]: I1203 12:42:50.098735 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-866c868748-bdl5k" event={"ID":"4c57f5e8-fe35-4f32-b0b1-a3dc5d0ccf33","Type":"ContainerStarted","Data":"8c7001d153f6c223ee493c5c05f995b394beefdef2772f671180aa50196e6178"} Dec 03 12:42:50 crc kubenswrapper[5052]: I1203 12:42:50.121550 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/glance-default-internal-api-0" podStartSLOduration=5.121522941 podStartE2EDuration="5.121522941s" podCreationTimestamp="2025-12-03 12:42:45 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:42:50.106334127 +0000 UTC m=+4960.694575911" watchObservedRunningTime="2025-12-03 12:42:50.121522941 +0000 UTC m=+4960.709764705" Dec 03 12:42:50 crc kubenswrapper[5052]: I1203 12:42:50.167031 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/8c8a4b1d-32cf-427d-a835-ff9237e5bb3f-scripts\") pod \"cinder-scheduler-0\" (UID: \"8c8a4b1d-32cf-427d-a835-ff9237e5bb3f\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:42:50 crc kubenswrapper[5052]: I1203 12:42:50.167111 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/c0261f94-be97-4cce-9eac-362d42248260-httpd-config\") pod \"neutron-64b7d8689c-p2dmd\" (UID: \"c0261f94-be97-4cce-9eac-362d42248260\") " pod="openstack-kuttl-tests/neutron-64b7d8689c-p2dmd" Dec 03 12:42:50 crc kubenswrapper[5052]: I1203 12:42:50.167167 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8c8a4b1d-32cf-427d-a835-ff9237e5bb3f-combined-ca-bundle\") pod \"cinder-scheduler-0\" (UID: \"8c8a4b1d-32cf-427d-a835-ff9237e5bb3f\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:42:50 crc kubenswrapper[5052]: I1203 12:42:50.167348 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/c0261f94-be97-4cce-9eac-362d42248260-config\") pod \"neutron-64b7d8689c-p2dmd\" (UID: \"c0261f94-be97-4cce-9eac-362d42248260\") " pod="openstack-kuttl-tests/neutron-64b7d8689c-p2dmd" Dec 03 12:42:50 crc kubenswrapper[5052]: I1203 12:42:50.167445 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-q5qrs\" (UniqueName: \"kubernetes.io/projected/c0261f94-be97-4cce-9eac-362d42248260-kube-api-access-q5qrs\") pod \"neutron-64b7d8689c-p2dmd\" (UID: \"c0261f94-be97-4cce-9eac-362d42248260\") " pod="openstack-kuttl-tests/neutron-64b7d8689c-p2dmd" Dec 03 12:42:50 crc kubenswrapper[5052]: I1203 12:42:50.167486 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c0261f94-be97-4cce-9eac-362d42248260-combined-ca-bundle\") pod \"neutron-64b7d8689c-p2dmd\" (UID: \"c0261f94-be97-4cce-9eac-362d42248260\") " pod="openstack-kuttl-tests/neutron-64b7d8689c-p2dmd" Dec 03 12:42:50 crc kubenswrapper[5052]: I1203 12:42:50.167573 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/c0261f94-be97-4cce-9eac-362d42248260-ovndb-tls-certs\") pod \"neutron-64b7d8689c-p2dmd\" (UID: \"c0261f94-be97-4cce-9eac-362d42248260\") " pod="openstack-kuttl-tests/neutron-64b7d8689c-p2dmd" Dec 03 12:42:50 crc kubenswrapper[5052]: I1203 12:42:50.167605 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/8c8a4b1d-32cf-427d-a835-ff9237e5bb3f-config-data\") pod \"cinder-scheduler-0\" (UID: \"8c8a4b1d-32cf-427d-a835-ff9237e5bb3f\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:42:50 crc kubenswrapper[5052]: I1203 12:42:50.167682 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b64d8\" (UniqueName: \"kubernetes.io/projected/8c8a4b1d-32cf-427d-a835-ff9237e5bb3f-kube-api-access-b64d8\") pod \"cinder-scheduler-0\" (UID: \"8c8a4b1d-32cf-427d-a835-ff9237e5bb3f\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:42:50 crc kubenswrapper[5052]: I1203 12:42:50.167765 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/8c8a4b1d-32cf-427d-a835-ff9237e5bb3f-config-data-custom\") pod \"cinder-scheduler-0\" (UID: \"8c8a4b1d-32cf-427d-a835-ff9237e5bb3f\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:42:50 crc kubenswrapper[5052]: I1203 12:42:50.167930 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/8c8a4b1d-32cf-427d-a835-ff9237e5bb3f-etc-machine-id\") pod \"cinder-scheduler-0\" (UID: \"8c8a4b1d-32cf-427d-a835-ff9237e5bb3f\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:42:50 crc kubenswrapper[5052]: I1203 12:42:50.171710 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/c0261f94-be97-4cce-9eac-362d42248260-ovndb-tls-certs\") pod \"neutron-64b7d8689c-p2dmd\" (UID: \"c0261f94-be97-4cce-9eac-362d42248260\") " pod="openstack-kuttl-tests/neutron-64b7d8689c-p2dmd" Dec 03 12:42:50 crc kubenswrapper[5052]: I1203 12:42:50.179352 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/c0261f94-be97-4cce-9eac-362d42248260-config\") pod \"neutron-64b7d8689c-p2dmd\" (UID: \"c0261f94-be97-4cce-9eac-362d42248260\") " pod="openstack-kuttl-tests/neutron-64b7d8689c-p2dmd" Dec 03 12:42:50 crc kubenswrapper[5052]: I1203 12:42:50.185794 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/c0261f94-be97-4cce-9eac-362d42248260-httpd-config\") pod \"neutron-64b7d8689c-p2dmd\" (UID: \"c0261f94-be97-4cce-9eac-362d42248260\") " pod="openstack-kuttl-tests/neutron-64b7d8689c-p2dmd" Dec 03 12:42:50 crc kubenswrapper[5052]: I1203 12:42:50.211673 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c0261f94-be97-4cce-9eac-362d42248260-combined-ca-bundle\") pod \"neutron-64b7d8689c-p2dmd\" (UID: \"c0261f94-be97-4cce-9eac-362d42248260\") " pod="openstack-kuttl-tests/neutron-64b7d8689c-p2dmd" Dec 03 12:42:50 crc kubenswrapper[5052]: I1203 12:42:50.276541 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/8c8a4b1d-32cf-427d-a835-ff9237e5bb3f-scripts\") pod \"cinder-scheduler-0\" (UID: \"8c8a4b1d-32cf-427d-a835-ff9237e5bb3f\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:42:50 crc kubenswrapper[5052]: I1203 12:42:50.276591 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8c8a4b1d-32cf-427d-a835-ff9237e5bb3f-combined-ca-bundle\") pod \"cinder-scheduler-0\" (UID: \"8c8a4b1d-32cf-427d-a835-ff9237e5bb3f\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:42:50 crc kubenswrapper[5052]: I1203 12:42:50.276952 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/8c8a4b1d-32cf-427d-a835-ff9237e5bb3f-config-data\") pod \"cinder-scheduler-0\" (UID: \"8c8a4b1d-32cf-427d-a835-ff9237e5bb3f\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:42:50 crc kubenswrapper[5052]: I1203 12:42:50.277025 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-b64d8\" (UniqueName: \"kubernetes.io/projected/8c8a4b1d-32cf-427d-a835-ff9237e5bb3f-kube-api-access-b64d8\") pod \"cinder-scheduler-0\" (UID: \"8c8a4b1d-32cf-427d-a835-ff9237e5bb3f\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:42:50 crc kubenswrapper[5052]: I1203 12:42:50.277068 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/8c8a4b1d-32cf-427d-a835-ff9237e5bb3f-config-data-custom\") pod \"cinder-scheduler-0\" (UID: \"8c8a4b1d-32cf-427d-a835-ff9237e5bb3f\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:42:50 crc kubenswrapper[5052]: I1203 12:42:50.277160 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/8c8a4b1d-32cf-427d-a835-ff9237e5bb3f-etc-machine-id\") pod \"cinder-scheduler-0\" (UID: \"8c8a4b1d-32cf-427d-a835-ff9237e5bb3f\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:42:50 crc kubenswrapper[5052]: I1203 12:42:50.277275 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/8c8a4b1d-32cf-427d-a835-ff9237e5bb3f-etc-machine-id\") pod \"cinder-scheduler-0\" (UID: \"8c8a4b1d-32cf-427d-a835-ff9237e5bb3f\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:42:50 crc kubenswrapper[5052]: I1203 12:42:50.280309 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-q5qrs\" (UniqueName: \"kubernetes.io/projected/c0261f94-be97-4cce-9eac-362d42248260-kube-api-access-q5qrs\") pod \"neutron-64b7d8689c-p2dmd\" (UID: \"c0261f94-be97-4cce-9eac-362d42248260\") " pod="openstack-kuttl-tests/neutron-64b7d8689c-p2dmd" Dec 03 12:42:50 crc kubenswrapper[5052]: I1203 12:42:50.289482 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/8c8a4b1d-32cf-427d-a835-ff9237e5bb3f-config-data\") pod \"cinder-scheduler-0\" (UID: \"8c8a4b1d-32cf-427d-a835-ff9237e5bb3f\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:42:50 crc kubenswrapper[5052]: I1203 12:42:50.298364 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/8c8a4b1d-32cf-427d-a835-ff9237e5bb3f-scripts\") pod \"cinder-scheduler-0\" (UID: \"8c8a4b1d-32cf-427d-a835-ff9237e5bb3f\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:42:50 crc kubenswrapper[5052]: I1203 12:42:50.321625 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 12:42:50 crc kubenswrapper[5052]: I1203 12:42:50.327612 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8c8a4b1d-32cf-427d-a835-ff9237e5bb3f-combined-ca-bundle\") pod \"cinder-scheduler-0\" (UID: \"8c8a4b1d-32cf-427d-a835-ff9237e5bb3f\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:42:50 crc kubenswrapper[5052]: I1203 12:42:50.327866 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-b64d8\" (UniqueName: \"kubernetes.io/projected/8c8a4b1d-32cf-427d-a835-ff9237e5bb3f-kube-api-access-b64d8\") pod \"cinder-scheduler-0\" (UID: \"8c8a4b1d-32cf-427d-a835-ff9237e5bb3f\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:42:50 crc kubenswrapper[5052]: I1203 12:42:50.337091 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/8c8a4b1d-32cf-427d-a835-ff9237e5bb3f-config-data-custom\") pod \"cinder-scheduler-0\" (UID: \"8c8a4b1d-32cf-427d-a835-ff9237e5bb3f\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:42:50 crc kubenswrapper[5052]: I1203 12:42:50.340068 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:42:50 crc kubenswrapper[5052]: I1203 12:42:50.345205 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cinder-api-config-data" Dec 03 12:42:50 crc kubenswrapper[5052]: I1203 12:42:50.351136 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 12:42:50 crc kubenswrapper[5052]: I1203 12:42:50.357253 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-64b7d8689c-p2dmd" Dec 03 12:42:50 crc kubenswrapper[5052]: I1203 12:42:50.369119 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/keystone-bootstrap-nnnzp" podStartSLOduration=3.36908636 podStartE2EDuration="3.36908636s" podCreationTimestamp="2025-12-03 12:42:47 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:42:50.286045805 +0000 UTC m=+4960.874287569" watchObservedRunningTime="2025-12-03 12:42:50.36908636 +0000 UTC m=+4960.957328124" Dec 03 12:42:50 crc kubenswrapper[5052]: I1203 12:42:50.491878 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/3f71c509-36f3-4e49-9c91-be36c2099411-scripts\") pod \"cinder-api-0\" (UID: \"3f71c509-36f3-4e49-9c91-be36c2099411\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:42:50 crc kubenswrapper[5052]: I1203 12:42:50.491934 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/3f71c509-36f3-4e49-9c91-be36c2099411-config-data\") pod \"cinder-api-0\" (UID: \"3f71c509-36f3-4e49-9c91-be36c2099411\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:42:50 crc kubenswrapper[5052]: I1203 12:42:50.492009 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/3f71c509-36f3-4e49-9c91-be36c2099411-logs\") pod \"cinder-api-0\" (UID: \"3f71c509-36f3-4e49-9c91-be36c2099411\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:42:50 crc kubenswrapper[5052]: I1203 12:42:50.492044 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/3f71c509-36f3-4e49-9c91-be36c2099411-etc-machine-id\") pod \"cinder-api-0\" (UID: \"3f71c509-36f3-4e49-9c91-be36c2099411\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:42:50 crc kubenswrapper[5052]: I1203 12:42:50.492075 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3f71c509-36f3-4e49-9c91-be36c2099411-combined-ca-bundle\") pod \"cinder-api-0\" (UID: \"3f71c509-36f3-4e49-9c91-be36c2099411\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:42:50 crc kubenswrapper[5052]: I1203 12:42:50.492094 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vvkhc\" (UniqueName: \"kubernetes.io/projected/3f71c509-36f3-4e49-9c91-be36c2099411-kube-api-access-vvkhc\") pod \"cinder-api-0\" (UID: \"3f71c509-36f3-4e49-9c91-be36c2099411\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:42:50 crc kubenswrapper[5052]: I1203 12:42:50.492128 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/3f71c509-36f3-4e49-9c91-be36c2099411-config-data-custom\") pod \"cinder-api-0\" (UID: \"3f71c509-36f3-4e49-9c91-be36c2099411\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:42:50 crc kubenswrapper[5052]: I1203 12:42:50.594067 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/3f71c509-36f3-4e49-9c91-be36c2099411-logs\") pod \"cinder-api-0\" (UID: \"3f71c509-36f3-4e49-9c91-be36c2099411\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:42:50 crc kubenswrapper[5052]: I1203 12:42:50.594423 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/3f71c509-36f3-4e49-9c91-be36c2099411-etc-machine-id\") pod \"cinder-api-0\" (UID: \"3f71c509-36f3-4e49-9c91-be36c2099411\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:42:50 crc kubenswrapper[5052]: I1203 12:42:50.594474 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3f71c509-36f3-4e49-9c91-be36c2099411-combined-ca-bundle\") pod \"cinder-api-0\" (UID: \"3f71c509-36f3-4e49-9c91-be36c2099411\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:42:50 crc kubenswrapper[5052]: I1203 12:42:50.594502 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-vvkhc\" (UniqueName: \"kubernetes.io/projected/3f71c509-36f3-4e49-9c91-be36c2099411-kube-api-access-vvkhc\") pod \"cinder-api-0\" (UID: \"3f71c509-36f3-4e49-9c91-be36c2099411\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:42:50 crc kubenswrapper[5052]: I1203 12:42:50.594581 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/3f71c509-36f3-4e49-9c91-be36c2099411-config-data-custom\") pod \"cinder-api-0\" (UID: \"3f71c509-36f3-4e49-9c91-be36c2099411\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:42:50 crc kubenswrapper[5052]: I1203 12:42:50.594593 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/3f71c509-36f3-4e49-9c91-be36c2099411-etc-machine-id\") pod \"cinder-api-0\" (UID: \"3f71c509-36f3-4e49-9c91-be36c2099411\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:42:50 crc kubenswrapper[5052]: I1203 12:42:50.594639 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/3f71c509-36f3-4e49-9c91-be36c2099411-scripts\") pod \"cinder-api-0\" (UID: \"3f71c509-36f3-4e49-9c91-be36c2099411\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:42:50 crc kubenswrapper[5052]: I1203 12:42:50.594664 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/3f71c509-36f3-4e49-9c91-be36c2099411-config-data\") pod \"cinder-api-0\" (UID: \"3f71c509-36f3-4e49-9c91-be36c2099411\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:42:50 crc kubenswrapper[5052]: I1203 12:42:50.594736 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/3f71c509-36f3-4e49-9c91-be36c2099411-logs\") pod \"cinder-api-0\" (UID: \"3f71c509-36f3-4e49-9c91-be36c2099411\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:42:50 crc kubenswrapper[5052]: I1203 12:42:50.673567 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:42:50 crc kubenswrapper[5052]: I1203 12:42:50.829126 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/3f71c509-36f3-4e49-9c91-be36c2099411-scripts\") pod \"cinder-api-0\" (UID: \"3f71c509-36f3-4e49-9c91-be36c2099411\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:42:50 crc kubenswrapper[5052]: I1203 12:42:50.830097 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3f71c509-36f3-4e49-9c91-be36c2099411-combined-ca-bundle\") pod \"cinder-api-0\" (UID: \"3f71c509-36f3-4e49-9c91-be36c2099411\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:42:50 crc kubenswrapper[5052]: I1203 12:42:50.830169 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/3f71c509-36f3-4e49-9c91-be36c2099411-config-data-custom\") pod \"cinder-api-0\" (UID: \"3f71c509-36f3-4e49-9c91-be36c2099411\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:42:50 crc kubenswrapper[5052]: I1203 12:42:50.830375 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-vvkhc\" (UniqueName: \"kubernetes.io/projected/3f71c509-36f3-4e49-9c91-be36c2099411-kube-api-access-vvkhc\") pod \"cinder-api-0\" (UID: \"3f71c509-36f3-4e49-9c91-be36c2099411\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:42:50 crc kubenswrapper[5052]: I1203 12:42:50.830795 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/3f71c509-36f3-4e49-9c91-be36c2099411-config-data\") pod \"cinder-api-0\" (UID: \"3f71c509-36f3-4e49-9c91-be36c2099411\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:42:50 crc kubenswrapper[5052]: W1203 12:42:50.933640 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod50a91cd1_d2b0_48ad_954c_17c18007e891.slice/crio-e30de4f6e400f75c400aa47f753b0a638068b280cd5702d95522c75bf45449ae WatchSource:0}: Error finding container e30de4f6e400f75c400aa47f753b0a638068b280cd5702d95522c75bf45449ae: Status 404 returned error can't find the container with id e30de4f6e400f75c400aa47f753b0a638068b280cd5702d95522c75bf45449ae Dec 03 12:42:51 crc kubenswrapper[5052]: I1203 12:42:51.019090 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:42:51 crc kubenswrapper[5052]: I1203 12:42:51.021187 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:42:51 crc kubenswrapper[5052]: I1203 12:42:51.133672 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"50a91cd1-d2b0-48ad-954c-17c18007e891","Type":"ContainerStarted","Data":"e30de4f6e400f75c400aa47f753b0a638068b280cd5702d95522c75bf45449ae"} Dec 03 12:42:51 crc kubenswrapper[5052]: I1203 12:42:51.139876 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-56b4b79f66-gfsxn" event={"ID":"76534e0c-3b73-4ede-bc24-b12c7f8ea4db","Type":"ContainerStarted","Data":"fc4f4321d2ac241b4a797a64c285f190f073d7f5f8f626ec4d2711dd1d73a837"} Dec 03 12:42:51 crc kubenswrapper[5052]: I1203 12:42:51.139990 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/placement-56b4b79f66-gfsxn" Dec 03 12:42:51 crc kubenswrapper[5052]: I1203 12:42:51.140023 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/placement-56b4b79f66-gfsxn" Dec 03 12:42:51 crc kubenswrapper[5052]: I1203 12:42:51.165805 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/placement-56b4b79f66-gfsxn" podStartSLOduration=4.165779124 podStartE2EDuration="4.165779124s" podCreationTimestamp="2025-12-03 12:42:47 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:42:51.159609407 +0000 UTC m=+4961.747851171" watchObservedRunningTime="2025-12-03 12:42:51.165779124 +0000 UTC m=+4961.754020888" Dec 03 12:42:51 crc kubenswrapper[5052]: I1203 12:42:51.583459 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-64b7d8689c-p2dmd"] Dec 03 12:42:51 crc kubenswrapper[5052]: I1203 12:42:51.771596 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 12:42:51 crc kubenswrapper[5052]: I1203 12:42:51.811329 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 12:42:52 crc kubenswrapper[5052]: I1203 12:42:52.158726 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-866c868748-bdl5k" event={"ID":"4c57f5e8-fe35-4f32-b0b1-a3dc5d0ccf33","Type":"ContainerStarted","Data":"572799af7b7f291e065f218508d292c61aa6a90d909f324e94d90ae2b89316c9"} Dec 03 12:42:52 crc kubenswrapper[5052]: I1203 12:42:52.160618 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/barbican-api-866c868748-bdl5k" Dec 03 12:42:52 crc kubenswrapper[5052]: I1203 12:42:52.160652 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/barbican-api-866c868748-bdl5k" Dec 03 12:42:52 crc kubenswrapper[5052]: I1203 12:42:52.172067 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"8c8a4b1d-32cf-427d-a835-ff9237e5bb3f","Type":"ContainerStarted","Data":"412239c9d4d21c82f8663f127316c7baf3e834ee9fdc0c72b5276000c9252edd"} Dec 03 12:42:52 crc kubenswrapper[5052]: I1203 12:42:52.173894 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"3f71c509-36f3-4e49-9c91-be36c2099411","Type":"ContainerStarted","Data":"b44729618a105fb764c5ac5d611e1b631cc48871e1c18178d51c15c07b5a5756"} Dec 03 12:42:52 crc kubenswrapper[5052]: I1203 12:42:52.203131 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/barbican-api-866c868748-bdl5k" podStartSLOduration=5.20311399 podStartE2EDuration="5.20311399s" podCreationTimestamp="2025-12-03 12:42:47 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:42:52.197345925 +0000 UTC m=+4962.785587689" watchObservedRunningTime="2025-12-03 12:42:52.20311399 +0000 UTC m=+4962.791355754" Dec 03 12:42:52 crc kubenswrapper[5052]: I1203 12:42:52.221093 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-64b7d8689c-p2dmd" event={"ID":"c0261f94-be97-4cce-9eac-362d42248260","Type":"ContainerStarted","Data":"c217b642a41725c40e7562f2d02446ddbab47d91f3bfeb0a9c3bb85a90aab8a8"} Dec 03 12:42:52 crc kubenswrapper[5052]: I1203 12:42:52.224608 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"24ccc68c-26a9-485e-bf26-512dac006b61","Type":"ContainerStarted","Data":"140826c39a57fa01ca2b271c735d5f9c464eb0c7c58b2f109153dcf2376fff98"} Dec 03 12:42:52 crc kubenswrapper[5052]: I1203 12:42:52.230343 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-keystone-listener-769bc98c4b-mnz6r" event={"ID":"1d187c5a-7745-447f-a8e2-06c7528e5be0","Type":"ContainerStarted","Data":"0b52d2ae1f61eeac5291c66875e41de2e8c84d29873caba7e0a0412afcac4845"} Dec 03 12:42:52 crc kubenswrapper[5052]: I1203 12:42:52.262926 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-worker-65fcd76cf7-xhs44" event={"ID":"92290cff-ed52-4a47-80e9-720bfd9f3dd8","Type":"ContainerStarted","Data":"f1a23446d7321c40e536bf2e9b6174acf23d15d1d963daca95ff134b7d58cd46"} Dec 03 12:42:52 crc kubenswrapper[5052]: I1203 12:42:52.293772 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/glance-default-external-api-0" podStartSLOduration=7.293749532 podStartE2EDuration="7.293749532s" podCreationTimestamp="2025-12-03 12:42:45 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:42:52.259317227 +0000 UTC m=+4962.847558991" watchObservedRunningTime="2025-12-03 12:42:52.293749532 +0000 UTC m=+4962.881991296" Dec 03 12:42:52 crc kubenswrapper[5052]: I1203 12:42:52.336622 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/barbican-keystone-listener-769bc98c4b-mnz6r" podStartSLOduration=5.336603778 podStartE2EDuration="5.336603778s" podCreationTimestamp="2025-12-03 12:42:47 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:42:52.297201991 +0000 UTC m=+4962.885443765" watchObservedRunningTime="2025-12-03 12:42:52.336603778 +0000 UTC m=+4962.924845542" Dec 03 12:42:52 crc kubenswrapper[5052]: I1203 12:42:52.360694 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/barbican-worker-65fcd76cf7-xhs44" podStartSLOduration=5.3606784659999995 podStartE2EDuration="5.360678466s" podCreationTimestamp="2025-12-03 12:42:47 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:42:52.320858977 +0000 UTC m=+4962.909100741" watchObservedRunningTime="2025-12-03 12:42:52.360678466 +0000 UTC m=+4962.948920230" Dec 03 12:42:53 crc kubenswrapper[5052]: I1203 12:42:53.326965 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"50a91cd1-d2b0-48ad-954c-17c18007e891","Type":"ContainerStarted","Data":"faac469fb234838cd16f1eedf3600b31b890b6868f17653562825912fa983149"} Dec 03 12:42:53 crc kubenswrapper[5052]: I1203 12:42:53.347988 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"8c8a4b1d-32cf-427d-a835-ff9237e5bb3f","Type":"ContainerStarted","Data":"1e0703a89dfa1271bf47184d8a7c32b3f5d67ea0d12324a1cdedb68a0888e8ae"} Dec 03 12:42:53 crc kubenswrapper[5052]: I1203 12:42:53.370239 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"3f71c509-36f3-4e49-9c91-be36c2099411","Type":"ContainerStarted","Data":"a9bb685e52420acd73e798bbfb09817a6a5b4d038d55ded35041658583ea9499"} Dec 03 12:42:53 crc kubenswrapper[5052]: I1203 12:42:53.394858 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-64b7d8689c-p2dmd" event={"ID":"c0261f94-be97-4cce-9eac-362d42248260","Type":"ContainerStarted","Data":"b623beb1d698577700860795ffcf72792b261e732bcbf58537ca90d6307df621"} Dec 03 12:42:53 crc kubenswrapper[5052]: I1203 12:42:53.394902 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-64b7d8689c-p2dmd" event={"ID":"c0261f94-be97-4cce-9eac-362d42248260","Type":"ContainerStarted","Data":"5b04eb43cca61516127ebbe98e362a70bd97994640daed64b640e02a6705f087"} Dec 03 12:42:53 crc kubenswrapper[5052]: I1203 12:42:53.415010 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/neutron-64b7d8689c-p2dmd" Dec 03 12:42:53 crc kubenswrapper[5052]: I1203 12:42:53.448191 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/neutron-64b7d8689c-p2dmd" podStartSLOduration=4.448165177 podStartE2EDuration="4.448165177s" podCreationTimestamp="2025-12-03 12:42:49 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:42:53.439413286 +0000 UTC m=+4964.027655060" watchObservedRunningTime="2025-12-03 12:42:53.448165177 +0000 UTC m=+4964.036406941" Dec 03 12:42:54 crc kubenswrapper[5052]: I1203 12:42:54.002747 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 12:42:54 crc kubenswrapper[5052]: I1203 12:42:54.437082 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"50a91cd1-d2b0-48ad-954c-17c18007e891","Type":"ContainerStarted","Data":"656e397b6d9cc32a2edd6ab337d8cc3f86f9dbda299c0fa90721c51fd6111e14"} Dec 03 12:42:54 crc kubenswrapper[5052]: I1203 12:42:54.439850 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"8c8a4b1d-32cf-427d-a835-ff9237e5bb3f","Type":"ContainerStarted","Data":"b98da4627823b0f5d22316d3fd8c916bb77b393dfaec11a5594d777700275156"} Dec 03 12:42:54 crc kubenswrapper[5052]: I1203 12:42:54.442060 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"3f71c509-36f3-4e49-9c91-be36c2099411","Type":"ContainerStarted","Data":"d9bf9f5a58dfd5707325e5b65fab65a13584b2a7c2cbbb70b9a7a301d75541d6"} Dec 03 12:42:54 crc kubenswrapper[5052]: I1203 12:42:54.469148 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/cinder-scheduler-0" podStartSLOduration=5.469129854 podStartE2EDuration="5.469129854s" podCreationTimestamp="2025-12-03 12:42:49 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:42:54.462383571 +0000 UTC m=+4965.050625335" watchObservedRunningTime="2025-12-03 12:42:54.469129854 +0000 UTC m=+4965.057371618" Dec 03 12:42:55 crc kubenswrapper[5052]: I1203 12:42:55.454049 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"50a91cd1-d2b0-48ad-954c-17c18007e891","Type":"ContainerStarted","Data":"fd43670029bd26793307cd8493e9456ce76cd5ee5b0b7b75dabac34375b7b698"} Dec 03 12:42:55 crc kubenswrapper[5052]: I1203 12:42:55.457668 5052 generic.go:334] "Generic (PLEG): container finished" podID="b36ab165-21eb-4968-a755-2cef353ce195" containerID="f48e73f815035e727aa3a398a4b9910157004cad319377fa765a601efb172b96" exitCode=0 Dec 03 12:42:55 crc kubenswrapper[5052]: I1203 12:42:55.457809 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-bootstrap-nnnzp" event={"ID":"b36ab165-21eb-4968-a755-2cef353ce195","Type":"ContainerDied","Data":"f48e73f815035e727aa3a398a4b9910157004cad319377fa765a601efb172b96"} Dec 03 12:42:55 crc kubenswrapper[5052]: I1203 12:42:55.457990 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:42:55 crc kubenswrapper[5052]: I1203 12:42:55.457992 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/cinder-api-0" podUID="3f71c509-36f3-4e49-9c91-be36c2099411" containerName="cinder-api-log" containerID="cri-o://a9bb685e52420acd73e798bbfb09817a6a5b4d038d55ded35041658583ea9499" gracePeriod=30 Dec 03 12:42:55 crc kubenswrapper[5052]: I1203 12:42:55.458065 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/cinder-api-0" podUID="3f71c509-36f3-4e49-9c91-be36c2099411" containerName="cinder-api" containerID="cri-o://d9bf9f5a58dfd5707325e5b65fab65a13584b2a7c2cbbb70b9a7a301d75541d6" gracePeriod=30 Dec 03 12:42:55 crc kubenswrapper[5052]: I1203 12:42:55.499088 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/cinder-api-0" podStartSLOduration=5.499069269 podStartE2EDuration="5.499069269s" podCreationTimestamp="2025-12-03 12:42:50 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:42:55.495566098 +0000 UTC m=+4966.083807872" watchObservedRunningTime="2025-12-03 12:42:55.499069269 +0000 UTC m=+4966.087311033" Dec 03 12:42:55 crc kubenswrapper[5052]: I1203 12:42:55.928608 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:42:55 crc kubenswrapper[5052]: I1203 12:42:55.928997 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:42:56 crc kubenswrapper[5052]: I1203 12:42:56.020636 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:42:56 crc kubenswrapper[5052]: I1203 12:42:56.046944 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:42:56 crc kubenswrapper[5052]: I1203 12:42:56.074202 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:42:56 crc kubenswrapper[5052]: I1203 12:42:56.398833 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:42:56 crc kubenswrapper[5052]: I1203 12:42:56.398900 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:42:56 crc kubenswrapper[5052]: I1203 12:42:56.459956 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:42:56 crc kubenswrapper[5052]: I1203 12:42:56.477387 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:42:56 crc kubenswrapper[5052]: I1203 12:42:56.488119 5052 generic.go:334] "Generic (PLEG): container finished" podID="3f71c509-36f3-4e49-9c91-be36c2099411" containerID="d9bf9f5a58dfd5707325e5b65fab65a13584b2a7c2cbbb70b9a7a301d75541d6" exitCode=0 Dec 03 12:42:56 crc kubenswrapper[5052]: I1203 12:42:56.488161 5052 generic.go:334] "Generic (PLEG): container finished" podID="3f71c509-36f3-4e49-9c91-be36c2099411" containerID="a9bb685e52420acd73e798bbfb09817a6a5b4d038d55ded35041658583ea9499" exitCode=143 Dec 03 12:42:56 crc kubenswrapper[5052]: I1203 12:42:56.488253 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"3f71c509-36f3-4e49-9c91-be36c2099411","Type":"ContainerDied","Data":"d9bf9f5a58dfd5707325e5b65fab65a13584b2a7c2cbbb70b9a7a301d75541d6"} Dec 03 12:42:56 crc kubenswrapper[5052]: I1203 12:42:56.488294 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"3f71c509-36f3-4e49-9c91-be36c2099411","Type":"ContainerDied","Data":"a9bb685e52420acd73e798bbfb09817a6a5b4d038d55ded35041658583ea9499"} Dec 03 12:42:56 crc kubenswrapper[5052]: I1203 12:42:56.496370 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"50a91cd1-d2b0-48ad-954c-17c18007e891","Type":"ContainerStarted","Data":"e880a804aafeeeb88e41aefe1189349e376a91868c5113e6db383cbf79761799"} Dec 03 12:42:56 crc kubenswrapper[5052]: I1203 12:42:56.496416 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:42:56 crc kubenswrapper[5052]: I1203 12:42:56.498023 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:42:56 crc kubenswrapper[5052]: I1203 12:42:56.498049 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:42:56 crc kubenswrapper[5052]: I1203 12:42:56.498209 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:42:56 crc kubenswrapper[5052]: I1203 12:42:56.498524 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:42:56 crc kubenswrapper[5052]: I1203 12:42:56.722529 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/ceilometer-0" podStartSLOduration=2.894454594 podStartE2EDuration="7.722497867s" podCreationTimestamp="2025-12-03 12:42:49 +0000 UTC" firstStartedPulling="2025-12-03 12:42:50.945716491 +0000 UTC m=+4961.533958265" lastFinishedPulling="2025-12-03 12:42:55.773759784 +0000 UTC m=+4966.362001538" observedRunningTime="2025-12-03 12:42:56.560264627 +0000 UTC m=+4967.148506401" watchObservedRunningTime="2025-12-03 12:42:56.722497867 +0000 UTC m=+4967.310739641" Dec 03 12:42:56 crc kubenswrapper[5052]: I1203 12:42:56.973710 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:42:57 crc kubenswrapper[5052]: I1203 12:42:57.009608 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/neutron-68f85bdc64-dc642"] Dec 03 12:42:57 crc kubenswrapper[5052]: E1203 12:42:57.010292 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="3f71c509-36f3-4e49-9c91-be36c2099411" containerName="cinder-api" Dec 03 12:42:57 crc kubenswrapper[5052]: I1203 12:42:57.010411 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="3f71c509-36f3-4e49-9c91-be36c2099411" containerName="cinder-api" Dec 03 12:42:57 crc kubenswrapper[5052]: E1203 12:42:57.010533 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="3f71c509-36f3-4e49-9c91-be36c2099411" containerName="cinder-api-log" Dec 03 12:42:57 crc kubenswrapper[5052]: I1203 12:42:57.010608 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="3f71c509-36f3-4e49-9c91-be36c2099411" containerName="cinder-api-log" Dec 03 12:42:57 crc kubenswrapper[5052]: I1203 12:42:57.010902 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="3f71c509-36f3-4e49-9c91-be36c2099411" containerName="cinder-api-log" Dec 03 12:42:57 crc kubenswrapper[5052]: I1203 12:42:57.010989 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="3f71c509-36f3-4e49-9c91-be36c2099411" containerName="cinder-api" Dec 03 12:42:57 crc kubenswrapper[5052]: I1203 12:42:57.020088 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-68f85bdc64-dc642" Dec 03 12:42:57 crc kubenswrapper[5052]: I1203 12:42:57.025056 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-neutron-internal-svc" Dec 03 12:42:57 crc kubenswrapper[5052]: I1203 12:42:57.027364 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-neutron-public-svc" Dec 03 12:42:57 crc kubenswrapper[5052]: I1203 12:42:57.031581 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-68f85bdc64-dc642"] Dec 03 12:42:57 crc kubenswrapper[5052]: I1203 12:42:57.087694 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vvkhc\" (UniqueName: \"kubernetes.io/projected/3f71c509-36f3-4e49-9c91-be36c2099411-kube-api-access-vvkhc\") pod \"3f71c509-36f3-4e49-9c91-be36c2099411\" (UID: \"3f71c509-36f3-4e49-9c91-be36c2099411\") " Dec 03 12:42:57 crc kubenswrapper[5052]: I1203 12:42:57.087876 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/3f71c509-36f3-4e49-9c91-be36c2099411-scripts\") pod \"3f71c509-36f3-4e49-9c91-be36c2099411\" (UID: \"3f71c509-36f3-4e49-9c91-be36c2099411\") " Dec 03 12:42:57 crc kubenswrapper[5052]: I1203 12:42:57.087944 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/3f71c509-36f3-4e49-9c91-be36c2099411-logs\") pod \"3f71c509-36f3-4e49-9c91-be36c2099411\" (UID: \"3f71c509-36f3-4e49-9c91-be36c2099411\") " Dec 03 12:42:57 crc kubenswrapper[5052]: I1203 12:42:57.088022 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3f71c509-36f3-4e49-9c91-be36c2099411-combined-ca-bundle\") pod \"3f71c509-36f3-4e49-9c91-be36c2099411\" (UID: \"3f71c509-36f3-4e49-9c91-be36c2099411\") " Dec 03 12:42:57 crc kubenswrapper[5052]: I1203 12:42:57.088069 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/3f71c509-36f3-4e49-9c91-be36c2099411-config-data-custom\") pod \"3f71c509-36f3-4e49-9c91-be36c2099411\" (UID: \"3f71c509-36f3-4e49-9c91-be36c2099411\") " Dec 03 12:42:57 crc kubenswrapper[5052]: I1203 12:42:57.088150 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/3f71c509-36f3-4e49-9c91-be36c2099411-config-data\") pod \"3f71c509-36f3-4e49-9c91-be36c2099411\" (UID: \"3f71c509-36f3-4e49-9c91-be36c2099411\") " Dec 03 12:42:57 crc kubenswrapper[5052]: I1203 12:42:57.088248 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/3f71c509-36f3-4e49-9c91-be36c2099411-etc-machine-id\") pod \"3f71c509-36f3-4e49-9c91-be36c2099411\" (UID: \"3f71c509-36f3-4e49-9c91-be36c2099411\") " Dec 03 12:42:57 crc kubenswrapper[5052]: I1203 12:42:57.088877 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/3f71c509-36f3-4e49-9c91-be36c2099411-etc-machine-id" (OuterVolumeSpecName: "etc-machine-id") pod "3f71c509-36f3-4e49-9c91-be36c2099411" (UID: "3f71c509-36f3-4e49-9c91-be36c2099411"). InnerVolumeSpecName "etc-machine-id". PluginName "kubernetes.io/host-path", VolumeGidValue "" Dec 03 12:42:57 crc kubenswrapper[5052]: I1203 12:42:57.093929 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/barbican-api-866c868748-bdl5k" Dec 03 12:42:57 crc kubenswrapper[5052]: I1203 12:42:57.095956 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3f71c509-36f3-4e49-9c91-be36c2099411-config-data-custom" (OuterVolumeSpecName: "config-data-custom") pod "3f71c509-36f3-4e49-9c91-be36c2099411" (UID: "3f71c509-36f3-4e49-9c91-be36c2099411"). InnerVolumeSpecName "config-data-custom". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:42:57 crc kubenswrapper[5052]: I1203 12:42:57.100148 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3f71c509-36f3-4e49-9c91-be36c2099411-logs" (OuterVolumeSpecName: "logs") pod "3f71c509-36f3-4e49-9c91-be36c2099411" (UID: "3f71c509-36f3-4e49-9c91-be36c2099411"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:42:57 crc kubenswrapper[5052]: I1203 12:42:57.116540 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3f71c509-36f3-4e49-9c91-be36c2099411-scripts" (OuterVolumeSpecName: "scripts") pod "3f71c509-36f3-4e49-9c91-be36c2099411" (UID: "3f71c509-36f3-4e49-9c91-be36c2099411"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:42:57 crc kubenswrapper[5052]: I1203 12:42:57.121933 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3f71c509-36f3-4e49-9c91-be36c2099411-kube-api-access-vvkhc" (OuterVolumeSpecName: "kube-api-access-vvkhc") pod "3f71c509-36f3-4e49-9c91-be36c2099411" (UID: "3f71c509-36f3-4e49-9c91-be36c2099411"). InnerVolumeSpecName "kube-api-access-vvkhc". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:42:57 crc kubenswrapper[5052]: I1203 12:42:57.173316 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3f71c509-36f3-4e49-9c91-be36c2099411-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "3f71c509-36f3-4e49-9c91-be36c2099411" (UID: "3f71c509-36f3-4e49-9c91-be36c2099411"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:42:57 crc kubenswrapper[5052]: I1203 12:42:57.190384 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/95997b63-313e-4f5f-adf6-b8ae63de9fb9-public-tls-certs\") pod \"neutron-68f85bdc64-dc642\" (UID: \"95997b63-313e-4f5f-adf6-b8ae63de9fb9\") " pod="openstack-kuttl-tests/neutron-68f85bdc64-dc642" Dec 03 12:42:57 crc kubenswrapper[5052]: I1203 12:42:57.190454 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cdwbz\" (UniqueName: \"kubernetes.io/projected/95997b63-313e-4f5f-adf6-b8ae63de9fb9-kube-api-access-cdwbz\") pod \"neutron-68f85bdc64-dc642\" (UID: \"95997b63-313e-4f5f-adf6-b8ae63de9fb9\") " pod="openstack-kuttl-tests/neutron-68f85bdc64-dc642" Dec 03 12:42:57 crc kubenswrapper[5052]: I1203 12:42:57.190497 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/95997b63-313e-4f5f-adf6-b8ae63de9fb9-combined-ca-bundle\") pod \"neutron-68f85bdc64-dc642\" (UID: \"95997b63-313e-4f5f-adf6-b8ae63de9fb9\") " pod="openstack-kuttl-tests/neutron-68f85bdc64-dc642" Dec 03 12:42:57 crc kubenswrapper[5052]: I1203 12:42:57.190547 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/95997b63-313e-4f5f-adf6-b8ae63de9fb9-config\") pod \"neutron-68f85bdc64-dc642\" (UID: \"95997b63-313e-4f5f-adf6-b8ae63de9fb9\") " pod="openstack-kuttl-tests/neutron-68f85bdc64-dc642" Dec 03 12:42:57 crc kubenswrapper[5052]: I1203 12:42:57.190563 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/95997b63-313e-4f5f-adf6-b8ae63de9fb9-ovndb-tls-certs\") pod \"neutron-68f85bdc64-dc642\" (UID: \"95997b63-313e-4f5f-adf6-b8ae63de9fb9\") " pod="openstack-kuttl-tests/neutron-68f85bdc64-dc642" Dec 03 12:42:57 crc kubenswrapper[5052]: I1203 12:42:57.190581 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/95997b63-313e-4f5f-adf6-b8ae63de9fb9-httpd-config\") pod \"neutron-68f85bdc64-dc642\" (UID: \"95997b63-313e-4f5f-adf6-b8ae63de9fb9\") " pod="openstack-kuttl-tests/neutron-68f85bdc64-dc642" Dec 03 12:42:57 crc kubenswrapper[5052]: I1203 12:42:57.190665 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/95997b63-313e-4f5f-adf6-b8ae63de9fb9-internal-tls-certs\") pod \"neutron-68f85bdc64-dc642\" (UID: \"95997b63-313e-4f5f-adf6-b8ae63de9fb9\") " pod="openstack-kuttl-tests/neutron-68f85bdc64-dc642" Dec 03 12:42:57 crc kubenswrapper[5052]: I1203 12:42:57.190870 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3f71c509-36f3-4e49-9c91-be36c2099411-config-data" (OuterVolumeSpecName: "config-data") pod "3f71c509-36f3-4e49-9c91-be36c2099411" (UID: "3f71c509-36f3-4e49-9c91-be36c2099411"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:42:57 crc kubenswrapper[5052]: I1203 12:42:57.190984 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-vvkhc\" (UniqueName: \"kubernetes.io/projected/3f71c509-36f3-4e49-9c91-be36c2099411-kube-api-access-vvkhc\") on node \"crc\" DevicePath \"\"" Dec 03 12:42:57 crc kubenswrapper[5052]: I1203 12:42:57.190997 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/3f71c509-36f3-4e49-9c91-be36c2099411-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:42:57 crc kubenswrapper[5052]: I1203 12:42:57.191006 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/3f71c509-36f3-4e49-9c91-be36c2099411-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:42:57 crc kubenswrapper[5052]: I1203 12:42:57.191041 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3f71c509-36f3-4e49-9c91-be36c2099411-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:42:57 crc kubenswrapper[5052]: I1203 12:42:57.191050 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/3f71c509-36f3-4e49-9c91-be36c2099411-config-data-custom\") on node \"crc\" DevicePath \"\"" Dec 03 12:42:57 crc kubenswrapper[5052]: I1203 12:42:57.191059 5052 reconciler_common.go:293] "Volume detached for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/3f71c509-36f3-4e49-9c91-be36c2099411-etc-machine-id\") on node \"crc\" DevicePath \"\"" Dec 03 12:42:57 crc kubenswrapper[5052]: I1203 12:42:57.201135 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-bootstrap-nnnzp" Dec 03 12:42:57 crc kubenswrapper[5052]: I1203 12:42:57.292373 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-phwjr\" (UniqueName: \"kubernetes.io/projected/b36ab165-21eb-4968-a755-2cef353ce195-kube-api-access-phwjr\") pod \"b36ab165-21eb-4968-a755-2cef353ce195\" (UID: \"b36ab165-21eb-4968-a755-2cef353ce195\") " Dec 03 12:42:57 crc kubenswrapper[5052]: I1203 12:42:57.292609 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/b36ab165-21eb-4968-a755-2cef353ce195-config-data\") pod \"b36ab165-21eb-4968-a755-2cef353ce195\" (UID: \"b36ab165-21eb-4968-a755-2cef353ce195\") " Dec 03 12:42:57 crc kubenswrapper[5052]: I1203 12:42:57.292637 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b36ab165-21eb-4968-a755-2cef353ce195-combined-ca-bundle\") pod \"b36ab165-21eb-4968-a755-2cef353ce195\" (UID: \"b36ab165-21eb-4968-a755-2cef353ce195\") " Dec 03 12:42:57 crc kubenswrapper[5052]: I1203 12:42:57.292662 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/b36ab165-21eb-4968-a755-2cef353ce195-fernet-keys\") pod \"b36ab165-21eb-4968-a755-2cef353ce195\" (UID: \"b36ab165-21eb-4968-a755-2cef353ce195\") " Dec 03 12:42:57 crc kubenswrapper[5052]: I1203 12:42:57.292705 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/b36ab165-21eb-4968-a755-2cef353ce195-scripts\") pod \"b36ab165-21eb-4968-a755-2cef353ce195\" (UID: \"b36ab165-21eb-4968-a755-2cef353ce195\") " Dec 03 12:42:57 crc kubenswrapper[5052]: I1203 12:42:57.292753 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/b36ab165-21eb-4968-a755-2cef353ce195-credential-keys\") pod \"b36ab165-21eb-4968-a755-2cef353ce195\" (UID: \"b36ab165-21eb-4968-a755-2cef353ce195\") " Dec 03 12:42:57 crc kubenswrapper[5052]: I1203 12:42:57.292931 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/95997b63-313e-4f5f-adf6-b8ae63de9fb9-internal-tls-certs\") pod \"neutron-68f85bdc64-dc642\" (UID: \"95997b63-313e-4f5f-adf6-b8ae63de9fb9\") " pod="openstack-kuttl-tests/neutron-68f85bdc64-dc642" Dec 03 12:42:57 crc kubenswrapper[5052]: I1203 12:42:57.293032 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/95997b63-313e-4f5f-adf6-b8ae63de9fb9-public-tls-certs\") pod \"neutron-68f85bdc64-dc642\" (UID: \"95997b63-313e-4f5f-adf6-b8ae63de9fb9\") " pod="openstack-kuttl-tests/neutron-68f85bdc64-dc642" Dec 03 12:42:57 crc kubenswrapper[5052]: I1203 12:42:57.293065 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-cdwbz\" (UniqueName: \"kubernetes.io/projected/95997b63-313e-4f5f-adf6-b8ae63de9fb9-kube-api-access-cdwbz\") pod \"neutron-68f85bdc64-dc642\" (UID: \"95997b63-313e-4f5f-adf6-b8ae63de9fb9\") " pod="openstack-kuttl-tests/neutron-68f85bdc64-dc642" Dec 03 12:42:57 crc kubenswrapper[5052]: I1203 12:42:57.293109 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/95997b63-313e-4f5f-adf6-b8ae63de9fb9-combined-ca-bundle\") pod \"neutron-68f85bdc64-dc642\" (UID: \"95997b63-313e-4f5f-adf6-b8ae63de9fb9\") " pod="openstack-kuttl-tests/neutron-68f85bdc64-dc642" Dec 03 12:42:57 crc kubenswrapper[5052]: I1203 12:42:57.293136 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/95997b63-313e-4f5f-adf6-b8ae63de9fb9-config\") pod \"neutron-68f85bdc64-dc642\" (UID: \"95997b63-313e-4f5f-adf6-b8ae63de9fb9\") " pod="openstack-kuttl-tests/neutron-68f85bdc64-dc642" Dec 03 12:42:57 crc kubenswrapper[5052]: I1203 12:42:57.293152 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/95997b63-313e-4f5f-adf6-b8ae63de9fb9-ovndb-tls-certs\") pod \"neutron-68f85bdc64-dc642\" (UID: \"95997b63-313e-4f5f-adf6-b8ae63de9fb9\") " pod="openstack-kuttl-tests/neutron-68f85bdc64-dc642" Dec 03 12:42:57 crc kubenswrapper[5052]: I1203 12:42:57.293171 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/95997b63-313e-4f5f-adf6-b8ae63de9fb9-httpd-config\") pod \"neutron-68f85bdc64-dc642\" (UID: \"95997b63-313e-4f5f-adf6-b8ae63de9fb9\") " pod="openstack-kuttl-tests/neutron-68f85bdc64-dc642" Dec 03 12:42:57 crc kubenswrapper[5052]: I1203 12:42:57.293219 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/3f71c509-36f3-4e49-9c91-be36c2099411-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:42:57 crc kubenswrapper[5052]: I1203 12:42:57.296453 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b36ab165-21eb-4968-a755-2cef353ce195-kube-api-access-phwjr" (OuterVolumeSpecName: "kube-api-access-phwjr") pod "b36ab165-21eb-4968-a755-2cef353ce195" (UID: "b36ab165-21eb-4968-a755-2cef353ce195"). InnerVolumeSpecName "kube-api-access-phwjr". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:42:57 crc kubenswrapper[5052]: I1203 12:42:57.301669 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b36ab165-21eb-4968-a755-2cef353ce195-fernet-keys" (OuterVolumeSpecName: "fernet-keys") pod "b36ab165-21eb-4968-a755-2cef353ce195" (UID: "b36ab165-21eb-4968-a755-2cef353ce195"). InnerVolumeSpecName "fernet-keys". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:42:57 crc kubenswrapper[5052]: I1203 12:42:57.302613 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/95997b63-313e-4f5f-adf6-b8ae63de9fb9-internal-tls-certs\") pod \"neutron-68f85bdc64-dc642\" (UID: \"95997b63-313e-4f5f-adf6-b8ae63de9fb9\") " pod="openstack-kuttl-tests/neutron-68f85bdc64-dc642" Dec 03 12:42:57 crc kubenswrapper[5052]: I1203 12:42:57.304119 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/95997b63-313e-4f5f-adf6-b8ae63de9fb9-httpd-config\") pod \"neutron-68f85bdc64-dc642\" (UID: \"95997b63-313e-4f5f-adf6-b8ae63de9fb9\") " pod="openstack-kuttl-tests/neutron-68f85bdc64-dc642" Dec 03 12:42:57 crc kubenswrapper[5052]: I1203 12:42:57.304367 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/95997b63-313e-4f5f-adf6-b8ae63de9fb9-public-tls-certs\") pod \"neutron-68f85bdc64-dc642\" (UID: \"95997b63-313e-4f5f-adf6-b8ae63de9fb9\") " pod="openstack-kuttl-tests/neutron-68f85bdc64-dc642" Dec 03 12:42:57 crc kubenswrapper[5052]: I1203 12:42:57.311030 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b36ab165-21eb-4968-a755-2cef353ce195-credential-keys" (OuterVolumeSpecName: "credential-keys") pod "b36ab165-21eb-4968-a755-2cef353ce195" (UID: "b36ab165-21eb-4968-a755-2cef353ce195"). InnerVolumeSpecName "credential-keys". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:42:57 crc kubenswrapper[5052]: I1203 12:42:57.313123 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/95997b63-313e-4f5f-adf6-b8ae63de9fb9-ovndb-tls-certs\") pod \"neutron-68f85bdc64-dc642\" (UID: \"95997b63-313e-4f5f-adf6-b8ae63de9fb9\") " pod="openstack-kuttl-tests/neutron-68f85bdc64-dc642" Dec 03 12:42:57 crc kubenswrapper[5052]: I1203 12:42:57.313490 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/95997b63-313e-4f5f-adf6-b8ae63de9fb9-combined-ca-bundle\") pod \"neutron-68f85bdc64-dc642\" (UID: \"95997b63-313e-4f5f-adf6-b8ae63de9fb9\") " pod="openstack-kuttl-tests/neutron-68f85bdc64-dc642" Dec 03 12:42:57 crc kubenswrapper[5052]: I1203 12:42:57.315420 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/95997b63-313e-4f5f-adf6-b8ae63de9fb9-config\") pod \"neutron-68f85bdc64-dc642\" (UID: \"95997b63-313e-4f5f-adf6-b8ae63de9fb9\") " pod="openstack-kuttl-tests/neutron-68f85bdc64-dc642" Dec 03 12:42:57 crc kubenswrapper[5052]: I1203 12:42:57.318749 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b36ab165-21eb-4968-a755-2cef353ce195-scripts" (OuterVolumeSpecName: "scripts") pod "b36ab165-21eb-4968-a755-2cef353ce195" (UID: "b36ab165-21eb-4968-a755-2cef353ce195"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:42:57 crc kubenswrapper[5052]: I1203 12:42:57.319374 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-cdwbz\" (UniqueName: \"kubernetes.io/projected/95997b63-313e-4f5f-adf6-b8ae63de9fb9-kube-api-access-cdwbz\") pod \"neutron-68f85bdc64-dc642\" (UID: \"95997b63-313e-4f5f-adf6-b8ae63de9fb9\") " pod="openstack-kuttl-tests/neutron-68f85bdc64-dc642" Dec 03 12:42:57 crc kubenswrapper[5052]: I1203 12:42:57.345204 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b36ab165-21eb-4968-a755-2cef353ce195-config-data" (OuterVolumeSpecName: "config-data") pod "b36ab165-21eb-4968-a755-2cef353ce195" (UID: "b36ab165-21eb-4968-a755-2cef353ce195"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:42:57 crc kubenswrapper[5052]: I1203 12:42:57.347735 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b36ab165-21eb-4968-a755-2cef353ce195-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "b36ab165-21eb-4968-a755-2cef353ce195" (UID: "b36ab165-21eb-4968-a755-2cef353ce195"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:42:57 crc kubenswrapper[5052]: I1203 12:42:57.395131 5052 reconciler_common.go:293] "Volume detached for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/b36ab165-21eb-4968-a755-2cef353ce195-credential-keys\") on node \"crc\" DevicePath \"\"" Dec 03 12:42:57 crc kubenswrapper[5052]: I1203 12:42:57.395176 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-phwjr\" (UniqueName: \"kubernetes.io/projected/b36ab165-21eb-4968-a755-2cef353ce195-kube-api-access-phwjr\") on node \"crc\" DevicePath \"\"" Dec 03 12:42:57 crc kubenswrapper[5052]: I1203 12:42:57.395193 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/b36ab165-21eb-4968-a755-2cef353ce195-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:42:57 crc kubenswrapper[5052]: I1203 12:42:57.395211 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b36ab165-21eb-4968-a755-2cef353ce195-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:42:57 crc kubenswrapper[5052]: I1203 12:42:57.395222 5052 reconciler_common.go:293] "Volume detached for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/b36ab165-21eb-4968-a755-2cef353ce195-fernet-keys\") on node \"crc\" DevicePath \"\"" Dec 03 12:42:57 crc kubenswrapper[5052]: I1203 12:42:57.395230 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/b36ab165-21eb-4968-a755-2cef353ce195-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:42:57 crc kubenswrapper[5052]: I1203 12:42:57.501559 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-68f85bdc64-dc642" Dec 03 12:42:57 crc kubenswrapper[5052]: I1203 12:42:57.538498 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-bootstrap-nnnzp" event={"ID":"b36ab165-21eb-4968-a755-2cef353ce195","Type":"ContainerDied","Data":"6d979e63f07840ce2d3a84871b38104d458c9d29466a10ab20d131ed6f7d4445"} Dec 03 12:42:57 crc kubenswrapper[5052]: I1203 12:42:57.538564 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6d979e63f07840ce2d3a84871b38104d458c9d29466a10ab20d131ed6f7d4445" Dec 03 12:42:57 crc kubenswrapper[5052]: I1203 12:42:57.538645 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-bootstrap-nnnzp" Dec 03 12:42:57 crc kubenswrapper[5052]: I1203 12:42:57.569970 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:42:57 crc kubenswrapper[5052]: I1203 12:42:57.570045 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"3f71c509-36f3-4e49-9c91-be36c2099411","Type":"ContainerDied","Data":"b44729618a105fb764c5ac5d611e1b631cc48871e1c18178d51c15c07b5a5756"} Dec 03 12:42:57 crc kubenswrapper[5052]: I1203 12:42:57.570100 5052 scope.go:117] "RemoveContainer" containerID="d9bf9f5a58dfd5707325e5b65fab65a13584b2a7c2cbbb70b9a7a301d75541d6" Dec 03 12:42:57 crc kubenswrapper[5052]: I1203 12:42:57.636327 5052 scope.go:117] "RemoveContainer" containerID="a9bb685e52420acd73e798bbfb09817a6a5b4d038d55ded35041658583ea9499" Dec 03 12:42:57 crc kubenswrapper[5052]: I1203 12:42:57.637707 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 12:42:57 crc kubenswrapper[5052]: I1203 12:42:57.671263 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 12:42:57 crc kubenswrapper[5052]: I1203 12:42:57.700999 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3f71c509-36f3-4e49-9c91-be36c2099411" path="/var/lib/kubelet/pods/3f71c509-36f3-4e49-9c91-be36c2099411/volumes" Dec 03 12:42:57 crc kubenswrapper[5052]: I1203 12:42:57.701610 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 12:42:57 crc kubenswrapper[5052]: E1203 12:42:57.701887 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="b36ab165-21eb-4968-a755-2cef353ce195" containerName="keystone-bootstrap" Dec 03 12:42:57 crc kubenswrapper[5052]: I1203 12:42:57.701898 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="b36ab165-21eb-4968-a755-2cef353ce195" containerName="keystone-bootstrap" Dec 03 12:42:57 crc kubenswrapper[5052]: I1203 12:42:57.702091 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="b36ab165-21eb-4968-a755-2cef353ce195" containerName="keystone-bootstrap" Dec 03 12:42:57 crc kubenswrapper[5052]: I1203 12:42:57.703370 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:42:57 crc kubenswrapper[5052]: I1203 12:42:57.704422 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/keystone-7f7d6d4c6c-sjnw5"] Dec 03 12:42:57 crc kubenswrapper[5052]: I1203 12:42:57.707577 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-cinder-internal-svc" Dec 03 12:42:57 crc kubenswrapper[5052]: I1203 12:42:57.707819 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-cinder-public-svc" Dec 03 12:42:57 crc kubenswrapper[5052]: I1203 12:42:57.708002 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cinder-api-config-data" Dec 03 12:42:57 crc kubenswrapper[5052]: I1203 12:42:57.716696 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-7f7d6d4c6c-sjnw5" Dec 03 12:42:57 crc kubenswrapper[5052]: I1203 12:42:57.725095 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone" Dec 03 12:42:57 crc kubenswrapper[5052]: I1203 12:42:57.725686 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone-config-data" Dec 03 12:42:57 crc kubenswrapper[5052]: I1203 12:42:57.725798 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-keystone-public-svc" Dec 03 12:42:57 crc kubenswrapper[5052]: I1203 12:42:57.725945 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-keystone-internal-svc" Dec 03 12:42:57 crc kubenswrapper[5052]: I1203 12:42:57.726081 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone-keystone-dockercfg-2f85l" Dec 03 12:42:57 crc kubenswrapper[5052]: I1203 12:42:57.726171 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone-scripts" Dec 03 12:42:57 crc kubenswrapper[5052]: I1203 12:42:57.738680 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 12:42:57 crc kubenswrapper[5052]: I1203 12:42:57.756540 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-7f7d6d4c6c-sjnw5"] Dec 03 12:42:57 crc kubenswrapper[5052]: I1203 12:42:57.814008 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/69bdf82f-4616-437b-a9aa-4a04978a3b63-fernet-keys\") pod \"keystone-7f7d6d4c6c-sjnw5\" (UID: \"69bdf82f-4616-437b-a9aa-4a04978a3b63\") " pod="openstack-kuttl-tests/keystone-7f7d6d4c6c-sjnw5" Dec 03 12:42:57 crc kubenswrapper[5052]: I1203 12:42:57.814111 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/68a80c17-9f7d-47de-b02c-5e7bb47656ab-combined-ca-bundle\") pod \"cinder-api-0\" (UID: \"68a80c17-9f7d-47de-b02c-5e7bb47656ab\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:42:57 crc kubenswrapper[5052]: I1203 12:42:57.814223 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/68a80c17-9f7d-47de-b02c-5e7bb47656ab-config-data\") pod \"cinder-api-0\" (UID: \"68a80c17-9f7d-47de-b02c-5e7bb47656ab\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:42:57 crc kubenswrapper[5052]: I1203 12:42:57.814258 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/69bdf82f-4616-437b-a9aa-4a04978a3b63-internal-tls-certs\") pod \"keystone-7f7d6d4c6c-sjnw5\" (UID: \"69bdf82f-4616-437b-a9aa-4a04978a3b63\") " pod="openstack-kuttl-tests/keystone-7f7d6d4c6c-sjnw5" Dec 03 12:42:57 crc kubenswrapper[5052]: I1203 12:42:57.814303 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/69bdf82f-4616-437b-a9aa-4a04978a3b63-credential-keys\") pod \"keystone-7f7d6d4c6c-sjnw5\" (UID: \"69bdf82f-4616-437b-a9aa-4a04978a3b63\") " pod="openstack-kuttl-tests/keystone-7f7d6d4c6c-sjnw5" Dec 03 12:42:57 crc kubenswrapper[5052]: I1203 12:42:57.814344 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/69bdf82f-4616-437b-a9aa-4a04978a3b63-config-data\") pod \"keystone-7f7d6d4c6c-sjnw5\" (UID: \"69bdf82f-4616-437b-a9aa-4a04978a3b63\") " pod="openstack-kuttl-tests/keystone-7f7d6d4c6c-sjnw5" Dec 03 12:42:57 crc kubenswrapper[5052]: I1203 12:42:57.814375 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nltzf\" (UniqueName: \"kubernetes.io/projected/68a80c17-9f7d-47de-b02c-5e7bb47656ab-kube-api-access-nltzf\") pod \"cinder-api-0\" (UID: \"68a80c17-9f7d-47de-b02c-5e7bb47656ab\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:42:57 crc kubenswrapper[5052]: I1203 12:42:57.814461 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/69bdf82f-4616-437b-a9aa-4a04978a3b63-combined-ca-bundle\") pod \"keystone-7f7d6d4c6c-sjnw5\" (UID: \"69bdf82f-4616-437b-a9aa-4a04978a3b63\") " pod="openstack-kuttl-tests/keystone-7f7d6d4c6c-sjnw5" Dec 03 12:42:57 crc kubenswrapper[5052]: I1203 12:42:57.814489 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/68a80c17-9f7d-47de-b02c-5e7bb47656ab-logs\") pod \"cinder-api-0\" (UID: \"68a80c17-9f7d-47de-b02c-5e7bb47656ab\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:42:57 crc kubenswrapper[5052]: I1203 12:42:57.814560 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/68a80c17-9f7d-47de-b02c-5e7bb47656ab-etc-machine-id\") pod \"cinder-api-0\" (UID: \"68a80c17-9f7d-47de-b02c-5e7bb47656ab\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:42:57 crc kubenswrapper[5052]: I1203 12:42:57.814601 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/68a80c17-9f7d-47de-b02c-5e7bb47656ab-internal-tls-certs\") pod \"cinder-api-0\" (UID: \"68a80c17-9f7d-47de-b02c-5e7bb47656ab\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:42:57 crc kubenswrapper[5052]: I1203 12:42:57.814672 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/68a80c17-9f7d-47de-b02c-5e7bb47656ab-config-data-custom\") pod \"cinder-api-0\" (UID: \"68a80c17-9f7d-47de-b02c-5e7bb47656ab\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:42:57 crc kubenswrapper[5052]: I1203 12:42:57.815781 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/69bdf82f-4616-437b-a9aa-4a04978a3b63-public-tls-certs\") pod \"keystone-7f7d6d4c6c-sjnw5\" (UID: \"69bdf82f-4616-437b-a9aa-4a04978a3b63\") " pod="openstack-kuttl-tests/keystone-7f7d6d4c6c-sjnw5" Dec 03 12:42:57 crc kubenswrapper[5052]: I1203 12:42:57.815820 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/68a80c17-9f7d-47de-b02c-5e7bb47656ab-scripts\") pod \"cinder-api-0\" (UID: \"68a80c17-9f7d-47de-b02c-5e7bb47656ab\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:42:57 crc kubenswrapper[5052]: I1203 12:42:57.815855 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/68a80c17-9f7d-47de-b02c-5e7bb47656ab-public-tls-certs\") pod \"cinder-api-0\" (UID: \"68a80c17-9f7d-47de-b02c-5e7bb47656ab\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:42:57 crc kubenswrapper[5052]: I1203 12:42:57.815883 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zhv82\" (UniqueName: \"kubernetes.io/projected/69bdf82f-4616-437b-a9aa-4a04978a3b63-kube-api-access-zhv82\") pod \"keystone-7f7d6d4c6c-sjnw5\" (UID: \"69bdf82f-4616-437b-a9aa-4a04978a3b63\") " pod="openstack-kuttl-tests/keystone-7f7d6d4c6c-sjnw5" Dec 03 12:42:57 crc kubenswrapper[5052]: I1203 12:42:57.815906 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/69bdf82f-4616-437b-a9aa-4a04978a3b63-scripts\") pod \"keystone-7f7d6d4c6c-sjnw5\" (UID: \"69bdf82f-4616-437b-a9aa-4a04978a3b63\") " pod="openstack-kuttl-tests/keystone-7f7d6d4c6c-sjnw5" Dec 03 12:42:57 crc kubenswrapper[5052]: I1203 12:42:57.917020 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/68a80c17-9f7d-47de-b02c-5e7bb47656ab-config-data\") pod \"cinder-api-0\" (UID: \"68a80c17-9f7d-47de-b02c-5e7bb47656ab\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:42:57 crc kubenswrapper[5052]: I1203 12:42:57.917071 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/69bdf82f-4616-437b-a9aa-4a04978a3b63-internal-tls-certs\") pod \"keystone-7f7d6d4c6c-sjnw5\" (UID: \"69bdf82f-4616-437b-a9aa-4a04978a3b63\") " pod="openstack-kuttl-tests/keystone-7f7d6d4c6c-sjnw5" Dec 03 12:42:57 crc kubenswrapper[5052]: I1203 12:42:57.917102 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/69bdf82f-4616-437b-a9aa-4a04978a3b63-credential-keys\") pod \"keystone-7f7d6d4c6c-sjnw5\" (UID: \"69bdf82f-4616-437b-a9aa-4a04978a3b63\") " pod="openstack-kuttl-tests/keystone-7f7d6d4c6c-sjnw5" Dec 03 12:42:57 crc kubenswrapper[5052]: I1203 12:42:57.917124 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/69bdf82f-4616-437b-a9aa-4a04978a3b63-config-data\") pod \"keystone-7f7d6d4c6c-sjnw5\" (UID: \"69bdf82f-4616-437b-a9aa-4a04978a3b63\") " pod="openstack-kuttl-tests/keystone-7f7d6d4c6c-sjnw5" Dec 03 12:42:57 crc kubenswrapper[5052]: I1203 12:42:57.917153 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-nltzf\" (UniqueName: \"kubernetes.io/projected/68a80c17-9f7d-47de-b02c-5e7bb47656ab-kube-api-access-nltzf\") pod \"cinder-api-0\" (UID: \"68a80c17-9f7d-47de-b02c-5e7bb47656ab\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:42:57 crc kubenswrapper[5052]: I1203 12:42:57.917194 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/69bdf82f-4616-437b-a9aa-4a04978a3b63-combined-ca-bundle\") pod \"keystone-7f7d6d4c6c-sjnw5\" (UID: \"69bdf82f-4616-437b-a9aa-4a04978a3b63\") " pod="openstack-kuttl-tests/keystone-7f7d6d4c6c-sjnw5" Dec 03 12:42:57 crc kubenswrapper[5052]: I1203 12:42:57.917211 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/68a80c17-9f7d-47de-b02c-5e7bb47656ab-logs\") pod \"cinder-api-0\" (UID: \"68a80c17-9f7d-47de-b02c-5e7bb47656ab\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:42:57 crc kubenswrapper[5052]: I1203 12:42:57.917230 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/68a80c17-9f7d-47de-b02c-5e7bb47656ab-etc-machine-id\") pod \"cinder-api-0\" (UID: \"68a80c17-9f7d-47de-b02c-5e7bb47656ab\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:42:57 crc kubenswrapper[5052]: I1203 12:42:57.917254 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/68a80c17-9f7d-47de-b02c-5e7bb47656ab-internal-tls-certs\") pod \"cinder-api-0\" (UID: \"68a80c17-9f7d-47de-b02c-5e7bb47656ab\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:42:57 crc kubenswrapper[5052]: I1203 12:42:57.917292 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/68a80c17-9f7d-47de-b02c-5e7bb47656ab-config-data-custom\") pod \"cinder-api-0\" (UID: \"68a80c17-9f7d-47de-b02c-5e7bb47656ab\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:42:57 crc kubenswrapper[5052]: I1203 12:42:57.917311 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/69bdf82f-4616-437b-a9aa-4a04978a3b63-public-tls-certs\") pod \"keystone-7f7d6d4c6c-sjnw5\" (UID: \"69bdf82f-4616-437b-a9aa-4a04978a3b63\") " pod="openstack-kuttl-tests/keystone-7f7d6d4c6c-sjnw5" Dec 03 12:42:57 crc kubenswrapper[5052]: I1203 12:42:57.917329 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/68a80c17-9f7d-47de-b02c-5e7bb47656ab-scripts\") pod \"cinder-api-0\" (UID: \"68a80c17-9f7d-47de-b02c-5e7bb47656ab\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:42:57 crc kubenswrapper[5052]: I1203 12:42:57.917349 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/68a80c17-9f7d-47de-b02c-5e7bb47656ab-public-tls-certs\") pod \"cinder-api-0\" (UID: \"68a80c17-9f7d-47de-b02c-5e7bb47656ab\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:42:57 crc kubenswrapper[5052]: I1203 12:42:57.917373 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-zhv82\" (UniqueName: \"kubernetes.io/projected/69bdf82f-4616-437b-a9aa-4a04978a3b63-kube-api-access-zhv82\") pod \"keystone-7f7d6d4c6c-sjnw5\" (UID: \"69bdf82f-4616-437b-a9aa-4a04978a3b63\") " pod="openstack-kuttl-tests/keystone-7f7d6d4c6c-sjnw5" Dec 03 12:42:57 crc kubenswrapper[5052]: I1203 12:42:57.917393 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/69bdf82f-4616-437b-a9aa-4a04978a3b63-scripts\") pod \"keystone-7f7d6d4c6c-sjnw5\" (UID: \"69bdf82f-4616-437b-a9aa-4a04978a3b63\") " pod="openstack-kuttl-tests/keystone-7f7d6d4c6c-sjnw5" Dec 03 12:42:57 crc kubenswrapper[5052]: I1203 12:42:57.917417 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/69bdf82f-4616-437b-a9aa-4a04978a3b63-fernet-keys\") pod \"keystone-7f7d6d4c6c-sjnw5\" (UID: \"69bdf82f-4616-437b-a9aa-4a04978a3b63\") " pod="openstack-kuttl-tests/keystone-7f7d6d4c6c-sjnw5" Dec 03 12:42:57 crc kubenswrapper[5052]: I1203 12:42:57.917442 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/68a80c17-9f7d-47de-b02c-5e7bb47656ab-combined-ca-bundle\") pod \"cinder-api-0\" (UID: \"68a80c17-9f7d-47de-b02c-5e7bb47656ab\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:42:57 crc kubenswrapper[5052]: I1203 12:42:57.917761 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/68a80c17-9f7d-47de-b02c-5e7bb47656ab-etc-machine-id\") pod \"cinder-api-0\" (UID: \"68a80c17-9f7d-47de-b02c-5e7bb47656ab\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:42:57 crc kubenswrapper[5052]: I1203 12:42:57.928003 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/69bdf82f-4616-437b-a9aa-4a04978a3b63-config-data\") pod \"keystone-7f7d6d4c6c-sjnw5\" (UID: \"69bdf82f-4616-437b-a9aa-4a04978a3b63\") " pod="openstack-kuttl-tests/keystone-7f7d6d4c6c-sjnw5" Dec 03 12:42:57 crc kubenswrapper[5052]: I1203 12:42:57.928733 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/69bdf82f-4616-437b-a9aa-4a04978a3b63-internal-tls-certs\") pod \"keystone-7f7d6d4c6c-sjnw5\" (UID: \"69bdf82f-4616-437b-a9aa-4a04978a3b63\") " pod="openstack-kuttl-tests/keystone-7f7d6d4c6c-sjnw5" Dec 03 12:42:57 crc kubenswrapper[5052]: I1203 12:42:57.930062 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/68a80c17-9f7d-47de-b02c-5e7bb47656ab-logs\") pod \"cinder-api-0\" (UID: \"68a80c17-9f7d-47de-b02c-5e7bb47656ab\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:42:57 crc kubenswrapper[5052]: I1203 12:42:57.931765 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/68a80c17-9f7d-47de-b02c-5e7bb47656ab-config-data-custom\") pod \"cinder-api-0\" (UID: \"68a80c17-9f7d-47de-b02c-5e7bb47656ab\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:42:57 crc kubenswrapper[5052]: I1203 12:42:57.931891 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/68a80c17-9f7d-47de-b02c-5e7bb47656ab-scripts\") pod \"cinder-api-0\" (UID: \"68a80c17-9f7d-47de-b02c-5e7bb47656ab\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:42:57 crc kubenswrapper[5052]: I1203 12:42:57.932200 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/68a80c17-9f7d-47de-b02c-5e7bb47656ab-internal-tls-certs\") pod \"cinder-api-0\" (UID: \"68a80c17-9f7d-47de-b02c-5e7bb47656ab\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:42:57 crc kubenswrapper[5052]: I1203 12:42:57.932896 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/69bdf82f-4616-437b-a9aa-4a04978a3b63-public-tls-certs\") pod \"keystone-7f7d6d4c6c-sjnw5\" (UID: \"69bdf82f-4616-437b-a9aa-4a04978a3b63\") " pod="openstack-kuttl-tests/keystone-7f7d6d4c6c-sjnw5" Dec 03 12:42:57 crc kubenswrapper[5052]: I1203 12:42:57.932936 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/69bdf82f-4616-437b-a9aa-4a04978a3b63-credential-keys\") pod \"keystone-7f7d6d4c6c-sjnw5\" (UID: \"69bdf82f-4616-437b-a9aa-4a04978a3b63\") " pod="openstack-kuttl-tests/keystone-7f7d6d4c6c-sjnw5" Dec 03 12:42:57 crc kubenswrapper[5052]: I1203 12:42:57.933961 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/68a80c17-9f7d-47de-b02c-5e7bb47656ab-public-tls-certs\") pod \"cinder-api-0\" (UID: \"68a80c17-9f7d-47de-b02c-5e7bb47656ab\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:42:57 crc kubenswrapper[5052]: I1203 12:42:57.935142 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/68a80c17-9f7d-47de-b02c-5e7bb47656ab-config-data\") pod \"cinder-api-0\" (UID: \"68a80c17-9f7d-47de-b02c-5e7bb47656ab\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:42:57 crc kubenswrapper[5052]: I1203 12:42:57.949963 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/69bdf82f-4616-437b-a9aa-4a04978a3b63-fernet-keys\") pod \"keystone-7f7d6d4c6c-sjnw5\" (UID: \"69bdf82f-4616-437b-a9aa-4a04978a3b63\") " pod="openstack-kuttl-tests/keystone-7f7d6d4c6c-sjnw5" Dec 03 12:42:57 crc kubenswrapper[5052]: I1203 12:42:57.950083 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/68a80c17-9f7d-47de-b02c-5e7bb47656ab-combined-ca-bundle\") pod \"cinder-api-0\" (UID: \"68a80c17-9f7d-47de-b02c-5e7bb47656ab\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:42:57 crc kubenswrapper[5052]: I1203 12:42:57.957100 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/69bdf82f-4616-437b-a9aa-4a04978a3b63-scripts\") pod \"keystone-7f7d6d4c6c-sjnw5\" (UID: \"69bdf82f-4616-437b-a9aa-4a04978a3b63\") " pod="openstack-kuttl-tests/keystone-7f7d6d4c6c-sjnw5" Dec 03 12:42:57 crc kubenswrapper[5052]: I1203 12:42:57.963427 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-nltzf\" (UniqueName: \"kubernetes.io/projected/68a80c17-9f7d-47de-b02c-5e7bb47656ab-kube-api-access-nltzf\") pod \"cinder-api-0\" (UID: \"68a80c17-9f7d-47de-b02c-5e7bb47656ab\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:42:57 crc kubenswrapper[5052]: I1203 12:42:57.964194 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/69bdf82f-4616-437b-a9aa-4a04978a3b63-combined-ca-bundle\") pod \"keystone-7f7d6d4c6c-sjnw5\" (UID: \"69bdf82f-4616-437b-a9aa-4a04978a3b63\") " pod="openstack-kuttl-tests/keystone-7f7d6d4c6c-sjnw5" Dec 03 12:42:57 crc kubenswrapper[5052]: I1203 12:42:57.983370 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-zhv82\" (UniqueName: \"kubernetes.io/projected/69bdf82f-4616-437b-a9aa-4a04978a3b63-kube-api-access-zhv82\") pod \"keystone-7f7d6d4c6c-sjnw5\" (UID: \"69bdf82f-4616-437b-a9aa-4a04978a3b63\") " pod="openstack-kuttl-tests/keystone-7f7d6d4c6c-sjnw5" Dec 03 12:42:58 crc kubenswrapper[5052]: I1203 12:42:58.061002 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:42:58 crc kubenswrapper[5052]: I1203 12:42:58.074477 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-7f7d6d4c6c-sjnw5" Dec 03 12:42:58 crc kubenswrapper[5052]: I1203 12:42:58.308614 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/barbican-api-6d86f7b884-lf4pd"] Dec 03 12:42:58 crc kubenswrapper[5052]: I1203 12:42:58.311064 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-api-6d86f7b884-lf4pd" Dec 03 12:42:58 crc kubenswrapper[5052]: I1203 12:42:58.314299 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-barbican-public-svc" Dec 03 12:42:58 crc kubenswrapper[5052]: I1203 12:42:58.315111 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-barbican-internal-svc" Dec 03 12:42:58 crc kubenswrapper[5052]: I1203 12:42:58.326555 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-api-6d86f7b884-lf4pd"] Dec 03 12:42:58 crc kubenswrapper[5052]: I1203 12:42:58.366681 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-68f85bdc64-dc642"] Dec 03 12:42:58 crc kubenswrapper[5052]: I1203 12:42:58.429656 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/5b63f869-c95a-4279-b542-3ebbec1a1fd5-public-tls-certs\") pod \"barbican-api-6d86f7b884-lf4pd\" (UID: \"5b63f869-c95a-4279-b542-3ebbec1a1fd5\") " pod="openstack-kuttl-tests/barbican-api-6d86f7b884-lf4pd" Dec 03 12:42:58 crc kubenswrapper[5052]: I1203 12:42:58.429755 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/5b63f869-c95a-4279-b542-3ebbec1a1fd5-internal-tls-certs\") pod \"barbican-api-6d86f7b884-lf4pd\" (UID: \"5b63f869-c95a-4279-b542-3ebbec1a1fd5\") " pod="openstack-kuttl-tests/barbican-api-6d86f7b884-lf4pd" Dec 03 12:42:58 crc kubenswrapper[5052]: I1203 12:42:58.429791 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bxhjw\" (UniqueName: \"kubernetes.io/projected/5b63f869-c95a-4279-b542-3ebbec1a1fd5-kube-api-access-bxhjw\") pod \"barbican-api-6d86f7b884-lf4pd\" (UID: \"5b63f869-c95a-4279-b542-3ebbec1a1fd5\") " pod="openstack-kuttl-tests/barbican-api-6d86f7b884-lf4pd" Dec 03 12:42:58 crc kubenswrapper[5052]: I1203 12:42:58.429822 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/5b63f869-c95a-4279-b542-3ebbec1a1fd5-logs\") pod \"barbican-api-6d86f7b884-lf4pd\" (UID: \"5b63f869-c95a-4279-b542-3ebbec1a1fd5\") " pod="openstack-kuttl-tests/barbican-api-6d86f7b884-lf4pd" Dec 03 12:42:58 crc kubenswrapper[5052]: I1203 12:42:58.429891 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/5b63f869-c95a-4279-b542-3ebbec1a1fd5-config-data\") pod \"barbican-api-6d86f7b884-lf4pd\" (UID: \"5b63f869-c95a-4279-b542-3ebbec1a1fd5\") " pod="openstack-kuttl-tests/barbican-api-6d86f7b884-lf4pd" Dec 03 12:42:58 crc kubenswrapper[5052]: I1203 12:42:58.429922 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5b63f869-c95a-4279-b542-3ebbec1a1fd5-combined-ca-bundle\") pod \"barbican-api-6d86f7b884-lf4pd\" (UID: \"5b63f869-c95a-4279-b542-3ebbec1a1fd5\") " pod="openstack-kuttl-tests/barbican-api-6d86f7b884-lf4pd" Dec 03 12:42:58 crc kubenswrapper[5052]: I1203 12:42:58.430597 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/5b63f869-c95a-4279-b542-3ebbec1a1fd5-config-data-custom\") pod \"barbican-api-6d86f7b884-lf4pd\" (UID: \"5b63f869-c95a-4279-b542-3ebbec1a1fd5\") " pod="openstack-kuttl-tests/barbican-api-6d86f7b884-lf4pd" Dec 03 12:42:58 crc kubenswrapper[5052]: I1203 12:42:58.588875 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5b63f869-c95a-4279-b542-3ebbec1a1fd5-combined-ca-bundle\") pod \"barbican-api-6d86f7b884-lf4pd\" (UID: \"5b63f869-c95a-4279-b542-3ebbec1a1fd5\") " pod="openstack-kuttl-tests/barbican-api-6d86f7b884-lf4pd" Dec 03 12:42:58 crc kubenswrapper[5052]: I1203 12:42:58.589380 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/5b63f869-c95a-4279-b542-3ebbec1a1fd5-config-data-custom\") pod \"barbican-api-6d86f7b884-lf4pd\" (UID: \"5b63f869-c95a-4279-b542-3ebbec1a1fd5\") " pod="openstack-kuttl-tests/barbican-api-6d86f7b884-lf4pd" Dec 03 12:42:58 crc kubenswrapper[5052]: I1203 12:42:58.589594 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/5b63f869-c95a-4279-b542-3ebbec1a1fd5-public-tls-certs\") pod \"barbican-api-6d86f7b884-lf4pd\" (UID: \"5b63f869-c95a-4279-b542-3ebbec1a1fd5\") " pod="openstack-kuttl-tests/barbican-api-6d86f7b884-lf4pd" Dec 03 12:42:58 crc kubenswrapper[5052]: I1203 12:42:58.589696 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/5b63f869-c95a-4279-b542-3ebbec1a1fd5-internal-tls-certs\") pod \"barbican-api-6d86f7b884-lf4pd\" (UID: \"5b63f869-c95a-4279-b542-3ebbec1a1fd5\") " pod="openstack-kuttl-tests/barbican-api-6d86f7b884-lf4pd" Dec 03 12:42:58 crc kubenswrapper[5052]: I1203 12:42:58.589739 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-bxhjw\" (UniqueName: \"kubernetes.io/projected/5b63f869-c95a-4279-b542-3ebbec1a1fd5-kube-api-access-bxhjw\") pod \"barbican-api-6d86f7b884-lf4pd\" (UID: \"5b63f869-c95a-4279-b542-3ebbec1a1fd5\") " pod="openstack-kuttl-tests/barbican-api-6d86f7b884-lf4pd" Dec 03 12:42:58 crc kubenswrapper[5052]: I1203 12:42:58.589780 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/5b63f869-c95a-4279-b542-3ebbec1a1fd5-logs\") pod \"barbican-api-6d86f7b884-lf4pd\" (UID: \"5b63f869-c95a-4279-b542-3ebbec1a1fd5\") " pod="openstack-kuttl-tests/barbican-api-6d86f7b884-lf4pd" Dec 03 12:42:58 crc kubenswrapper[5052]: I1203 12:42:58.589874 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/5b63f869-c95a-4279-b542-3ebbec1a1fd5-config-data\") pod \"barbican-api-6d86f7b884-lf4pd\" (UID: \"5b63f869-c95a-4279-b542-3ebbec1a1fd5\") " pod="openstack-kuttl-tests/barbican-api-6d86f7b884-lf4pd" Dec 03 12:42:58 crc kubenswrapper[5052]: I1203 12:42:58.591602 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/5b63f869-c95a-4279-b542-3ebbec1a1fd5-logs\") pod \"barbican-api-6d86f7b884-lf4pd\" (UID: \"5b63f869-c95a-4279-b542-3ebbec1a1fd5\") " pod="openstack-kuttl-tests/barbican-api-6d86f7b884-lf4pd" Dec 03 12:42:58 crc kubenswrapper[5052]: I1203 12:42:58.596104 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5b63f869-c95a-4279-b542-3ebbec1a1fd5-combined-ca-bundle\") pod \"barbican-api-6d86f7b884-lf4pd\" (UID: \"5b63f869-c95a-4279-b542-3ebbec1a1fd5\") " pod="openstack-kuttl-tests/barbican-api-6d86f7b884-lf4pd" Dec 03 12:42:58 crc kubenswrapper[5052]: I1203 12:42:58.596898 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/5b63f869-c95a-4279-b542-3ebbec1a1fd5-internal-tls-certs\") pod \"barbican-api-6d86f7b884-lf4pd\" (UID: \"5b63f869-c95a-4279-b542-3ebbec1a1fd5\") " pod="openstack-kuttl-tests/barbican-api-6d86f7b884-lf4pd" Dec 03 12:42:58 crc kubenswrapper[5052]: I1203 12:42:58.597258 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/5b63f869-c95a-4279-b542-3ebbec1a1fd5-config-data\") pod \"barbican-api-6d86f7b884-lf4pd\" (UID: \"5b63f869-c95a-4279-b542-3ebbec1a1fd5\") " pod="openstack-kuttl-tests/barbican-api-6d86f7b884-lf4pd" Dec 03 12:42:58 crc kubenswrapper[5052]: I1203 12:42:58.604500 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/5b63f869-c95a-4279-b542-3ebbec1a1fd5-public-tls-certs\") pod \"barbican-api-6d86f7b884-lf4pd\" (UID: \"5b63f869-c95a-4279-b542-3ebbec1a1fd5\") " pod="openstack-kuttl-tests/barbican-api-6d86f7b884-lf4pd" Dec 03 12:42:58 crc kubenswrapper[5052]: I1203 12:42:58.606178 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/5b63f869-c95a-4279-b542-3ebbec1a1fd5-config-data-custom\") pod \"barbican-api-6d86f7b884-lf4pd\" (UID: \"5b63f869-c95a-4279-b542-3ebbec1a1fd5\") " pod="openstack-kuttl-tests/barbican-api-6d86f7b884-lf4pd" Dec 03 12:42:58 crc kubenswrapper[5052]: I1203 12:42:58.638494 5052 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Dec 03 12:42:58 crc kubenswrapper[5052]: I1203 12:42:58.638534 5052 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Dec 03 12:42:58 crc kubenswrapper[5052]: I1203 12:42:58.639394 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-68f85bdc64-dc642" event={"ID":"95997b63-313e-4f5f-adf6-b8ae63de9fb9","Type":"ContainerStarted","Data":"7c6c7d14c1f8543dac5d6115d97af202f3a8137146128bbba9e6bdda2a168009"} Dec 03 12:42:58 crc kubenswrapper[5052]: I1203 12:42:58.640045 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-bxhjw\" (UniqueName: \"kubernetes.io/projected/5b63f869-c95a-4279-b542-3ebbec1a1fd5-kube-api-access-bxhjw\") pod \"barbican-api-6d86f7b884-lf4pd\" (UID: \"5b63f869-c95a-4279-b542-3ebbec1a1fd5\") " pod="openstack-kuttl-tests/barbican-api-6d86f7b884-lf4pd" Dec 03 12:42:58 crc kubenswrapper[5052]: I1203 12:42:58.738017 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-api-6d86f7b884-lf4pd" Dec 03 12:42:58 crc kubenswrapper[5052]: I1203 12:42:58.968388 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-7f7d6d4c6c-sjnw5"] Dec 03 12:42:59 crc kubenswrapper[5052]: I1203 12:42:59.085966 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 12:42:59 crc kubenswrapper[5052]: I1203 12:42:59.380251 5052 patch_prober.go:28] interesting pod/machine-config-daemon-74bj6 container/machine-config-daemon namespace/openshift-machine-config-operator: Liveness probe status=failure output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" start-of-body= Dec 03 12:42:59 crc kubenswrapper[5052]: I1203 12:42:59.380312 5052 prober.go:107] "Probe failed" probeType="Liveness" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" containerName="machine-config-daemon" probeResult="failure" output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" Dec 03 12:42:59 crc kubenswrapper[5052]: I1203 12:42:59.397488 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-api-6d86f7b884-lf4pd"] Dec 03 12:42:59 crc kubenswrapper[5052]: W1203 12:42:59.417723 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5b63f869_c95a_4279_b542_3ebbec1a1fd5.slice/crio-6bc37640eb704614061b31aa9d8d48b7b491d6bcceb5572feab011a7b4bc2dc4 WatchSource:0}: Error finding container 6bc37640eb704614061b31aa9d8d48b7b491d6bcceb5572feab011a7b4bc2dc4: Status 404 returned error can't find the container with id 6bc37640eb704614061b31aa9d8d48b7b491d6bcceb5572feab011a7b4bc2dc4 Dec 03 12:42:59 crc kubenswrapper[5052]: I1203 12:42:59.620367 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/barbican-api-866c868748-bdl5k" Dec 03 12:42:59 crc kubenswrapper[5052]: I1203 12:42:59.714467 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-68f85bdc64-dc642" event={"ID":"95997b63-313e-4f5f-adf6-b8ae63de9fb9","Type":"ContainerStarted","Data":"93ea570e117e1af074faa4804b0c9dd26bf11892c6a87661f85fc221a9464d09"} Dec 03 12:42:59 crc kubenswrapper[5052]: I1203 12:42:59.714551 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/neutron-68f85bdc64-dc642" Dec 03 12:42:59 crc kubenswrapper[5052]: I1203 12:42:59.714569 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-68f85bdc64-dc642" event={"ID":"95997b63-313e-4f5f-adf6-b8ae63de9fb9","Type":"ContainerStarted","Data":"65e1fdf0d5abf9142c0d5dd8fd15f412c134a90fb4aecb28999c1caba228326b"} Dec 03 12:42:59 crc kubenswrapper[5052]: I1203 12:42:59.714583 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"68a80c17-9f7d-47de-b02c-5e7bb47656ab","Type":"ContainerStarted","Data":"cf29619d34834e7d49a5eedc87579f786108ce54ec9ce704025e2fbebca41943"} Dec 03 12:42:59 crc kubenswrapper[5052]: I1203 12:42:59.714596 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-7f7d6d4c6c-sjnw5" event={"ID":"69bdf82f-4616-437b-a9aa-4a04978a3b63","Type":"ContainerStarted","Data":"86dfffcda0458cbe1fcc5f5c8662311799b8b237901bad0a12f858211ae70295"} Dec 03 12:42:59 crc kubenswrapper[5052]: I1203 12:42:59.714609 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-7f7d6d4c6c-sjnw5" event={"ID":"69bdf82f-4616-437b-a9aa-4a04978a3b63","Type":"ContainerStarted","Data":"92f7d32569328ac8ba5c2c899dd6d63b05ed17e11c98b3fe77fd40c06c4bdc81"} Dec 03 12:42:59 crc kubenswrapper[5052]: I1203 12:42:59.757083 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-6d86f7b884-lf4pd" event={"ID":"5b63f869-c95a-4279-b542-3ebbec1a1fd5","Type":"ContainerStarted","Data":"6bc37640eb704614061b31aa9d8d48b7b491d6bcceb5572feab011a7b4bc2dc4"} Dec 03 12:42:59 crc kubenswrapper[5052]: I1203 12:42:59.806288 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/neutron-68f85bdc64-dc642" podStartSLOduration=3.806267767 podStartE2EDuration="3.806267767s" podCreationTimestamp="2025-12-03 12:42:56 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:42:59.787034657 +0000 UTC m=+4970.375276451" watchObservedRunningTime="2025-12-03 12:42:59.806267767 +0000 UTC m=+4970.394509531" Dec 03 12:43:00 crc kubenswrapper[5052]: I1203 12:43:00.237461 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:43:00 crc kubenswrapper[5052]: I1203 12:43:00.237875 5052 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Dec 03 12:43:00 crc kubenswrapper[5052]: I1203 12:43:00.288370 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:43:00 crc kubenswrapper[5052]: I1203 12:43:00.288535 5052 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Dec 03 12:43:00 crc kubenswrapper[5052]: I1203 12:43:00.803429 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-6d86f7b884-lf4pd" event={"ID":"5b63f869-c95a-4279-b542-3ebbec1a1fd5","Type":"ContainerStarted","Data":"51b7f4e0f8d805d3cab7483cb57358f70f00e5c62c17634e4a78a7fc68d35bce"} Dec 03 12:43:00 crc kubenswrapper[5052]: I1203 12:43:00.803898 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/keystone-7f7d6d4c6c-sjnw5" Dec 03 12:43:00 crc kubenswrapper[5052]: I1203 12:43:00.829833 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/keystone-7f7d6d4c6c-sjnw5" podStartSLOduration=3.829813689 podStartE2EDuration="3.829813689s" podCreationTimestamp="2025-12-03 12:42:57 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:43:00.824674772 +0000 UTC m=+4971.412916546" watchObservedRunningTime="2025-12-03 12:43:00.829813689 +0000 UTC m=+4971.418055453" Dec 03 12:43:01 crc kubenswrapper[5052]: I1203 12:43:01.337329 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:43:01 crc kubenswrapper[5052]: I1203 12:43:01.489639 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:43:01 crc kubenswrapper[5052]: I1203 12:43:01.602725 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 12:43:01 crc kubenswrapper[5052]: I1203 12:43:01.711474 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:43:01 crc kubenswrapper[5052]: I1203 12:43:01.871824 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-6d86f7b884-lf4pd" event={"ID":"5b63f869-c95a-4279-b542-3ebbec1a1fd5","Type":"ContainerStarted","Data":"878de2f156357bd531b5738c30192dd837946cd61f68a045b6a3411f524181e4"} Dec 03 12:43:01 crc kubenswrapper[5052]: I1203 12:43:01.873998 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/barbican-api-6d86f7b884-lf4pd" Dec 03 12:43:01 crc kubenswrapper[5052]: I1203 12:43:01.874131 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/barbican-api-6d86f7b884-lf4pd" Dec 03 12:43:01 crc kubenswrapper[5052]: I1203 12:43:01.895396 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/cinder-scheduler-0" podUID="8c8a4b1d-32cf-427d-a835-ff9237e5bb3f" containerName="cinder-scheduler" containerID="cri-o://1e0703a89dfa1271bf47184d8a7c32b3f5d67ea0d12324a1cdedb68a0888e8ae" gracePeriod=30 Dec 03 12:43:01 crc kubenswrapper[5052]: I1203 12:43:01.895781 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"68a80c17-9f7d-47de-b02c-5e7bb47656ab","Type":"ContainerStarted","Data":"8562f7f2637fa32043b41c559e46c0266da41a1cab62e80ce7b44cb981784aae"} Dec 03 12:43:01 crc kubenswrapper[5052]: I1203 12:43:01.896811 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/cinder-scheduler-0" podUID="8c8a4b1d-32cf-427d-a835-ff9237e5bb3f" containerName="probe" containerID="cri-o://b98da4627823b0f5d22316d3fd8c916bb77b393dfaec11a5594d777700275156" gracePeriod=30 Dec 03 12:43:01 crc kubenswrapper[5052]: I1203 12:43:01.920012 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/barbican-api-6d86f7b884-lf4pd" podStartSLOduration=3.919986315 podStartE2EDuration="3.919986315s" podCreationTimestamp="2025-12-03 12:42:58 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:43:01.906190541 +0000 UTC m=+4972.494432315" watchObservedRunningTime="2025-12-03 12:43:01.919986315 +0000 UTC m=+4972.508228079" Dec 03 12:43:02 crc kubenswrapper[5052]: I1203 12:43:02.589406 5052 prober.go:107] "Probe failed" probeType="Liveness" pod="openstack-kuttl-tests/barbican-api-866c868748-bdl5k" podUID="4c57f5e8-fe35-4f32-b0b1-a3dc5d0ccf33" containerName="barbican-api-log" probeResult="failure" output="Get \"http://10.217.1.189:9311/healthcheck\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Dec 03 12:43:02 crc kubenswrapper[5052]: I1203 12:43:02.908346 5052 generic.go:334] "Generic (PLEG): container finished" podID="8c8a4b1d-32cf-427d-a835-ff9237e5bb3f" containerID="b98da4627823b0f5d22316d3fd8c916bb77b393dfaec11a5594d777700275156" exitCode=0 Dec 03 12:43:02 crc kubenswrapper[5052]: I1203 12:43:02.908421 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"8c8a4b1d-32cf-427d-a835-ff9237e5bb3f","Type":"ContainerDied","Data":"b98da4627823b0f5d22316d3fd8c916bb77b393dfaec11a5594d777700275156"} Dec 03 12:43:02 crc kubenswrapper[5052]: I1203 12:43:02.912603 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"68a80c17-9f7d-47de-b02c-5e7bb47656ab","Type":"ContainerStarted","Data":"36e0739e37891a21cddc05442e8ef21c0dc06d541fe56c5e1fce38f2ed446057"} Dec 03 12:43:02 crc kubenswrapper[5052]: I1203 12:43:02.912682 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:43:02 crc kubenswrapper[5052]: I1203 12:43:02.941772 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/cinder-api-0" podStartSLOduration=5.941748576 podStartE2EDuration="5.941748576s" podCreationTimestamp="2025-12-03 12:42:57 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:43:02.939284016 +0000 UTC m=+4973.527525800" watchObservedRunningTime="2025-12-03 12:43:02.941748576 +0000 UTC m=+4973.529990350" Dec 03 12:43:06 crc kubenswrapper[5052]: I1203 12:43:06.754791 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:43:06 crc kubenswrapper[5052]: I1203 12:43:06.795355 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8c8a4b1d-32cf-427d-a835-ff9237e5bb3f-combined-ca-bundle\") pod \"8c8a4b1d-32cf-427d-a835-ff9237e5bb3f\" (UID: \"8c8a4b1d-32cf-427d-a835-ff9237e5bb3f\") " Dec 03 12:43:06 crc kubenswrapper[5052]: I1203 12:43:06.795645 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/8c8a4b1d-32cf-427d-a835-ff9237e5bb3f-config-data\") pod \"8c8a4b1d-32cf-427d-a835-ff9237e5bb3f\" (UID: \"8c8a4b1d-32cf-427d-a835-ff9237e5bb3f\") " Dec 03 12:43:06 crc kubenswrapper[5052]: I1203 12:43:06.795804 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/8c8a4b1d-32cf-427d-a835-ff9237e5bb3f-etc-machine-id\") pod \"8c8a4b1d-32cf-427d-a835-ff9237e5bb3f\" (UID: \"8c8a4b1d-32cf-427d-a835-ff9237e5bb3f\") " Dec 03 12:43:06 crc kubenswrapper[5052]: I1203 12:43:06.795899 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/8c8a4b1d-32cf-427d-a835-ff9237e5bb3f-etc-machine-id" (OuterVolumeSpecName: "etc-machine-id") pod "8c8a4b1d-32cf-427d-a835-ff9237e5bb3f" (UID: "8c8a4b1d-32cf-427d-a835-ff9237e5bb3f"). InnerVolumeSpecName "etc-machine-id". PluginName "kubernetes.io/host-path", VolumeGidValue "" Dec 03 12:43:06 crc kubenswrapper[5052]: I1203 12:43:06.795935 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/8c8a4b1d-32cf-427d-a835-ff9237e5bb3f-scripts\") pod \"8c8a4b1d-32cf-427d-a835-ff9237e5bb3f\" (UID: \"8c8a4b1d-32cf-427d-a835-ff9237e5bb3f\") " Dec 03 12:43:06 crc kubenswrapper[5052]: I1203 12:43:06.796064 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b64d8\" (UniqueName: \"kubernetes.io/projected/8c8a4b1d-32cf-427d-a835-ff9237e5bb3f-kube-api-access-b64d8\") pod \"8c8a4b1d-32cf-427d-a835-ff9237e5bb3f\" (UID: \"8c8a4b1d-32cf-427d-a835-ff9237e5bb3f\") " Dec 03 12:43:06 crc kubenswrapper[5052]: I1203 12:43:06.796163 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/8c8a4b1d-32cf-427d-a835-ff9237e5bb3f-config-data-custom\") pod \"8c8a4b1d-32cf-427d-a835-ff9237e5bb3f\" (UID: \"8c8a4b1d-32cf-427d-a835-ff9237e5bb3f\") " Dec 03 12:43:06 crc kubenswrapper[5052]: I1203 12:43:06.796783 5052 reconciler_common.go:293] "Volume detached for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/8c8a4b1d-32cf-427d-a835-ff9237e5bb3f-etc-machine-id\") on node \"crc\" DevicePath \"\"" Dec 03 12:43:06 crc kubenswrapper[5052]: I1203 12:43:06.803809 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8c8a4b1d-32cf-427d-a835-ff9237e5bb3f-kube-api-access-b64d8" (OuterVolumeSpecName: "kube-api-access-b64d8") pod "8c8a4b1d-32cf-427d-a835-ff9237e5bb3f" (UID: "8c8a4b1d-32cf-427d-a835-ff9237e5bb3f"). InnerVolumeSpecName "kube-api-access-b64d8". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:43:06 crc kubenswrapper[5052]: I1203 12:43:06.805336 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8c8a4b1d-32cf-427d-a835-ff9237e5bb3f-config-data-custom" (OuterVolumeSpecName: "config-data-custom") pod "8c8a4b1d-32cf-427d-a835-ff9237e5bb3f" (UID: "8c8a4b1d-32cf-427d-a835-ff9237e5bb3f"). InnerVolumeSpecName "config-data-custom". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:43:06 crc kubenswrapper[5052]: I1203 12:43:06.807951 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8c8a4b1d-32cf-427d-a835-ff9237e5bb3f-scripts" (OuterVolumeSpecName: "scripts") pod "8c8a4b1d-32cf-427d-a835-ff9237e5bb3f" (UID: "8c8a4b1d-32cf-427d-a835-ff9237e5bb3f"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:43:06 crc kubenswrapper[5052]: I1203 12:43:06.860333 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8c8a4b1d-32cf-427d-a835-ff9237e5bb3f-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "8c8a4b1d-32cf-427d-a835-ff9237e5bb3f" (UID: "8c8a4b1d-32cf-427d-a835-ff9237e5bb3f"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:43:06 crc kubenswrapper[5052]: I1203 12:43:06.898409 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8c8a4b1d-32cf-427d-a835-ff9237e5bb3f-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:43:06 crc kubenswrapper[5052]: I1203 12:43:06.898453 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/8c8a4b1d-32cf-427d-a835-ff9237e5bb3f-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:43:06 crc kubenswrapper[5052]: I1203 12:43:06.898469 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-b64d8\" (UniqueName: \"kubernetes.io/projected/8c8a4b1d-32cf-427d-a835-ff9237e5bb3f-kube-api-access-b64d8\") on node \"crc\" DevicePath \"\"" Dec 03 12:43:06 crc kubenswrapper[5052]: I1203 12:43:06.898482 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/8c8a4b1d-32cf-427d-a835-ff9237e5bb3f-config-data-custom\") on node \"crc\" DevicePath \"\"" Dec 03 12:43:06 crc kubenswrapper[5052]: I1203 12:43:06.952233 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8c8a4b1d-32cf-427d-a835-ff9237e5bb3f-config-data" (OuterVolumeSpecName: "config-data") pod "8c8a4b1d-32cf-427d-a835-ff9237e5bb3f" (UID: "8c8a4b1d-32cf-427d-a835-ff9237e5bb3f"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:43:06 crc kubenswrapper[5052]: I1203 12:43:06.953985 5052 generic.go:334] "Generic (PLEG): container finished" podID="8c8a4b1d-32cf-427d-a835-ff9237e5bb3f" containerID="1e0703a89dfa1271bf47184d8a7c32b3f5d67ea0d12324a1cdedb68a0888e8ae" exitCode=0 Dec 03 12:43:06 crc kubenswrapper[5052]: I1203 12:43:06.954041 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:43:06 crc kubenswrapper[5052]: I1203 12:43:06.954041 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"8c8a4b1d-32cf-427d-a835-ff9237e5bb3f","Type":"ContainerDied","Data":"1e0703a89dfa1271bf47184d8a7c32b3f5d67ea0d12324a1cdedb68a0888e8ae"} Dec 03 12:43:06 crc kubenswrapper[5052]: I1203 12:43:06.954170 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"8c8a4b1d-32cf-427d-a835-ff9237e5bb3f","Type":"ContainerDied","Data":"412239c9d4d21c82f8663f127316c7baf3e834ee9fdc0c72b5276000c9252edd"} Dec 03 12:43:06 crc kubenswrapper[5052]: I1203 12:43:06.954266 5052 scope.go:117] "RemoveContainer" containerID="b98da4627823b0f5d22316d3fd8c916bb77b393dfaec11a5594d777700275156" Dec 03 12:43:06 crc kubenswrapper[5052]: I1203 12:43:06.990362 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 12:43:06 crc kubenswrapper[5052]: I1203 12:43:06.999792 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/8c8a4b1d-32cf-427d-a835-ff9237e5bb3f-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:43:06 crc kubenswrapper[5052]: I1203 12:43:06.999916 5052 scope.go:117] "RemoveContainer" containerID="1e0703a89dfa1271bf47184d8a7c32b3f5d67ea0d12324a1cdedb68a0888e8ae" Dec 03 12:43:07 crc kubenswrapper[5052]: I1203 12:43:07.010047 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 12:43:07 crc kubenswrapper[5052]: I1203 12:43:07.033467 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 12:43:07 crc kubenswrapper[5052]: E1203 12:43:07.034428 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="8c8a4b1d-32cf-427d-a835-ff9237e5bb3f" containerName="probe" Dec 03 12:43:07 crc kubenswrapper[5052]: I1203 12:43:07.034582 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="8c8a4b1d-32cf-427d-a835-ff9237e5bb3f" containerName="probe" Dec 03 12:43:07 crc kubenswrapper[5052]: E1203 12:43:07.034691 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="8c8a4b1d-32cf-427d-a835-ff9237e5bb3f" containerName="cinder-scheduler" Dec 03 12:43:07 crc kubenswrapper[5052]: I1203 12:43:07.034787 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="8c8a4b1d-32cf-427d-a835-ff9237e5bb3f" containerName="cinder-scheduler" Dec 03 12:43:07 crc kubenswrapper[5052]: I1203 12:43:07.034949 5052 scope.go:117] "RemoveContainer" containerID="b98da4627823b0f5d22316d3fd8c916bb77b393dfaec11a5594d777700275156" Dec 03 12:43:07 crc kubenswrapper[5052]: I1203 12:43:07.035342 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="8c8a4b1d-32cf-427d-a835-ff9237e5bb3f" containerName="cinder-scheduler" Dec 03 12:43:07 crc kubenswrapper[5052]: I1203 12:43:07.035744 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="8c8a4b1d-32cf-427d-a835-ff9237e5bb3f" containerName="probe" Dec 03 12:43:07 crc kubenswrapper[5052]: E1203 12:43:07.036224 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b98da4627823b0f5d22316d3fd8c916bb77b393dfaec11a5594d777700275156\": container with ID starting with b98da4627823b0f5d22316d3fd8c916bb77b393dfaec11a5594d777700275156 not found: ID does not exist" containerID="b98da4627823b0f5d22316d3fd8c916bb77b393dfaec11a5594d777700275156" Dec 03 12:43:07 crc kubenswrapper[5052]: I1203 12:43:07.036263 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b98da4627823b0f5d22316d3fd8c916bb77b393dfaec11a5594d777700275156"} err="failed to get container status \"b98da4627823b0f5d22316d3fd8c916bb77b393dfaec11a5594d777700275156\": rpc error: code = NotFound desc = could not find container \"b98da4627823b0f5d22316d3fd8c916bb77b393dfaec11a5594d777700275156\": container with ID starting with b98da4627823b0f5d22316d3fd8c916bb77b393dfaec11a5594d777700275156 not found: ID does not exist" Dec 03 12:43:07 crc kubenswrapper[5052]: I1203 12:43:07.036290 5052 scope.go:117] "RemoveContainer" containerID="1e0703a89dfa1271bf47184d8a7c32b3f5d67ea0d12324a1cdedb68a0888e8ae" Dec 03 12:43:07 crc kubenswrapper[5052]: I1203 12:43:07.037912 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:43:07 crc kubenswrapper[5052]: E1203 12:43:07.041915 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1e0703a89dfa1271bf47184d8a7c32b3f5d67ea0d12324a1cdedb68a0888e8ae\": container with ID starting with 1e0703a89dfa1271bf47184d8a7c32b3f5d67ea0d12324a1cdedb68a0888e8ae not found: ID does not exist" containerID="1e0703a89dfa1271bf47184d8a7c32b3f5d67ea0d12324a1cdedb68a0888e8ae" Dec 03 12:43:07 crc kubenswrapper[5052]: I1203 12:43:07.042410 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1e0703a89dfa1271bf47184d8a7c32b3f5d67ea0d12324a1cdedb68a0888e8ae"} err="failed to get container status \"1e0703a89dfa1271bf47184d8a7c32b3f5d67ea0d12324a1cdedb68a0888e8ae\": rpc error: code = NotFound desc = could not find container \"1e0703a89dfa1271bf47184d8a7c32b3f5d67ea0d12324a1cdedb68a0888e8ae\": container with ID starting with 1e0703a89dfa1271bf47184d8a7c32b3f5d67ea0d12324a1cdedb68a0888e8ae not found: ID does not exist" Dec 03 12:43:07 crc kubenswrapper[5052]: I1203 12:43:07.042204 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cinder-scheduler-config-data" Dec 03 12:43:07 crc kubenswrapper[5052]: I1203 12:43:07.054244 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 12:43:07 crc kubenswrapper[5052]: E1203 12:43:07.098240 5052 cadvisor_stats_provider.go:516] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8c8a4b1d_32cf_427d_a835_ff9237e5bb3f.slice\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8c8a4b1d_32cf_427d_a835_ff9237e5bb3f.slice/crio-412239c9d4d21c82f8663f127316c7baf3e834ee9fdc0c72b5276000c9252edd\": RecentStats: unable to find data in memory cache]" Dec 03 12:43:07 crc kubenswrapper[5052]: I1203 12:43:07.101339 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/3ae64378-aaa4-46bc-8c48-e1d90e490016-scripts\") pod \"cinder-scheduler-0\" (UID: \"3ae64378-aaa4-46bc-8c48-e1d90e490016\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:43:07 crc kubenswrapper[5052]: I1203 12:43:07.101399 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/3ae64378-aaa4-46bc-8c48-e1d90e490016-config-data\") pod \"cinder-scheduler-0\" (UID: \"3ae64378-aaa4-46bc-8c48-e1d90e490016\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:43:07 crc kubenswrapper[5052]: I1203 12:43:07.101429 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/3ae64378-aaa4-46bc-8c48-e1d90e490016-config-data-custom\") pod \"cinder-scheduler-0\" (UID: \"3ae64378-aaa4-46bc-8c48-e1d90e490016\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:43:07 crc kubenswrapper[5052]: I1203 12:43:07.101468 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3ae64378-aaa4-46bc-8c48-e1d90e490016-combined-ca-bundle\") pod \"cinder-scheduler-0\" (UID: \"3ae64378-aaa4-46bc-8c48-e1d90e490016\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:43:07 crc kubenswrapper[5052]: I1203 12:43:07.101484 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/3ae64378-aaa4-46bc-8c48-e1d90e490016-etc-machine-id\") pod \"cinder-scheduler-0\" (UID: \"3ae64378-aaa4-46bc-8c48-e1d90e490016\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:43:07 crc kubenswrapper[5052]: I1203 12:43:07.101601 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cb4rb\" (UniqueName: \"kubernetes.io/projected/3ae64378-aaa4-46bc-8c48-e1d90e490016-kube-api-access-cb4rb\") pod \"cinder-scheduler-0\" (UID: \"3ae64378-aaa4-46bc-8c48-e1d90e490016\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:43:07 crc kubenswrapper[5052]: I1203 12:43:07.204141 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/3ae64378-aaa4-46bc-8c48-e1d90e490016-config-data-custom\") pod \"cinder-scheduler-0\" (UID: \"3ae64378-aaa4-46bc-8c48-e1d90e490016\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:43:07 crc kubenswrapper[5052]: I1203 12:43:07.204302 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3ae64378-aaa4-46bc-8c48-e1d90e490016-combined-ca-bundle\") pod \"cinder-scheduler-0\" (UID: \"3ae64378-aaa4-46bc-8c48-e1d90e490016\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:43:07 crc kubenswrapper[5052]: I1203 12:43:07.204337 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/3ae64378-aaa4-46bc-8c48-e1d90e490016-etc-machine-id\") pod \"cinder-scheduler-0\" (UID: \"3ae64378-aaa4-46bc-8c48-e1d90e490016\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:43:07 crc kubenswrapper[5052]: I1203 12:43:07.204500 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/3ae64378-aaa4-46bc-8c48-e1d90e490016-etc-machine-id\") pod \"cinder-scheduler-0\" (UID: \"3ae64378-aaa4-46bc-8c48-e1d90e490016\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:43:07 crc kubenswrapper[5052]: I1203 12:43:07.204504 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-cb4rb\" (UniqueName: \"kubernetes.io/projected/3ae64378-aaa4-46bc-8c48-e1d90e490016-kube-api-access-cb4rb\") pod \"cinder-scheduler-0\" (UID: \"3ae64378-aaa4-46bc-8c48-e1d90e490016\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:43:07 crc kubenswrapper[5052]: I1203 12:43:07.204649 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/3ae64378-aaa4-46bc-8c48-e1d90e490016-scripts\") pod \"cinder-scheduler-0\" (UID: \"3ae64378-aaa4-46bc-8c48-e1d90e490016\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:43:07 crc kubenswrapper[5052]: I1203 12:43:07.204715 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/3ae64378-aaa4-46bc-8c48-e1d90e490016-config-data\") pod \"cinder-scheduler-0\" (UID: \"3ae64378-aaa4-46bc-8c48-e1d90e490016\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:43:07 crc kubenswrapper[5052]: I1203 12:43:07.208819 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/3ae64378-aaa4-46bc-8c48-e1d90e490016-config-data\") pod \"cinder-scheduler-0\" (UID: \"3ae64378-aaa4-46bc-8c48-e1d90e490016\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:43:07 crc kubenswrapper[5052]: I1203 12:43:07.209951 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3ae64378-aaa4-46bc-8c48-e1d90e490016-combined-ca-bundle\") pod \"cinder-scheduler-0\" (UID: \"3ae64378-aaa4-46bc-8c48-e1d90e490016\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:43:07 crc kubenswrapper[5052]: I1203 12:43:07.211015 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/3ae64378-aaa4-46bc-8c48-e1d90e490016-scripts\") pod \"cinder-scheduler-0\" (UID: \"3ae64378-aaa4-46bc-8c48-e1d90e490016\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:43:07 crc kubenswrapper[5052]: I1203 12:43:07.217564 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/3ae64378-aaa4-46bc-8c48-e1d90e490016-config-data-custom\") pod \"cinder-scheduler-0\" (UID: \"3ae64378-aaa4-46bc-8c48-e1d90e490016\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:43:07 crc kubenswrapper[5052]: I1203 12:43:07.221065 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-cb4rb\" (UniqueName: \"kubernetes.io/projected/3ae64378-aaa4-46bc-8c48-e1d90e490016-kube-api-access-cb4rb\") pod \"cinder-scheduler-0\" (UID: \"3ae64378-aaa4-46bc-8c48-e1d90e490016\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:43:07 crc kubenswrapper[5052]: I1203 12:43:07.369470 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:43:07 crc kubenswrapper[5052]: I1203 12:43:07.693300 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8c8a4b1d-32cf-427d-a835-ff9237e5bb3f" path="/var/lib/kubelet/pods/8c8a4b1d-32cf-427d-a835-ff9237e5bb3f/volumes" Dec 03 12:43:08 crc kubenswrapper[5052]: I1203 12:43:07.995426 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 12:43:08 crc kubenswrapper[5052]: I1203 12:43:08.985029 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"3ae64378-aaa4-46bc-8c48-e1d90e490016","Type":"ContainerStarted","Data":"8e0b376c29b7c2dd056afa36e69404e2be33493fa6b1a9ba35735a9c8becf167"} Dec 03 12:43:08 crc kubenswrapper[5052]: I1203 12:43:08.985311 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"3ae64378-aaa4-46bc-8c48-e1d90e490016","Type":"ContainerStarted","Data":"91363cf0c2a5494bdb58587815a54d4ffca02c51e4af6463e1c49693b97ab6a5"} Dec 03 12:43:10 crc kubenswrapper[5052]: I1203 12:43:10.000059 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"3ae64378-aaa4-46bc-8c48-e1d90e490016","Type":"ContainerStarted","Data":"3fa64d6a78f92e04f85cc78283bcecfe7e618fa805695f4e69f5237e051e4939"} Dec 03 12:43:10 crc kubenswrapper[5052]: I1203 12:43:10.133401 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:43:10 crc kubenswrapper[5052]: I1203 12:43:10.158757 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/cinder-scheduler-0" podStartSLOduration=4.158732769 podStartE2EDuration="4.158732769s" podCreationTimestamp="2025-12-03 12:43:06 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:43:10.0279846 +0000 UTC m=+4980.616226384" watchObservedRunningTime="2025-12-03 12:43:10.158732769 +0000 UTC m=+4980.746974533" Dec 03 12:43:10 crc kubenswrapper[5052]: I1203 12:43:10.284921 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/barbican-api-6d86f7b884-lf4pd" Dec 03 12:43:10 crc kubenswrapper[5052]: I1203 12:43:10.411278 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/barbican-api-6d86f7b884-lf4pd" Dec 03 12:43:10 crc kubenswrapper[5052]: I1203 12:43:10.498666 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-api-866c868748-bdl5k"] Dec 03 12:43:10 crc kubenswrapper[5052]: I1203 12:43:10.499013 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/barbican-api-866c868748-bdl5k" podUID="4c57f5e8-fe35-4f32-b0b1-a3dc5d0ccf33" containerName="barbican-api" containerID="cri-o://572799af7b7f291e065f218508d292c61aa6a90d909f324e94d90ae2b89316c9" gracePeriod=30 Dec 03 12:43:10 crc kubenswrapper[5052]: I1203 12:43:10.499373 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/barbican-api-866c868748-bdl5k" podUID="4c57f5e8-fe35-4f32-b0b1-a3dc5d0ccf33" containerName="barbican-api-log" containerID="cri-o://bd1fd8ba880b814dd955b69768ac28222dd0eeff96580f6fa7d3c52c5b458a07" gracePeriod=30 Dec 03 12:43:11 crc kubenswrapper[5052]: I1203 12:43:11.053443 5052 generic.go:334] "Generic (PLEG): container finished" podID="4c57f5e8-fe35-4f32-b0b1-a3dc5d0ccf33" containerID="bd1fd8ba880b814dd955b69768ac28222dd0eeff96580f6fa7d3c52c5b458a07" exitCode=143 Dec 03 12:43:11 crc kubenswrapper[5052]: I1203 12:43:11.056116 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-866c868748-bdl5k" event={"ID":"4c57f5e8-fe35-4f32-b0b1-a3dc5d0ccf33","Type":"ContainerDied","Data":"bd1fd8ba880b814dd955b69768ac28222dd0eeff96580f6fa7d3c52c5b458a07"} Dec 03 12:43:12 crc kubenswrapper[5052]: I1203 12:43:12.370543 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:43:13 crc kubenswrapper[5052]: I1203 12:43:13.663969 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/barbican-api-866c868748-bdl5k" podUID="4c57f5e8-fe35-4f32-b0b1-a3dc5d0ccf33" containerName="barbican-api" probeResult="failure" output="Get \"http://10.217.1.189:9311/healthcheck\": read tcp 10.217.0.2:42838->10.217.1.189:9311: read: connection reset by peer" Dec 03 12:43:13 crc kubenswrapper[5052]: I1203 12:43:13.663998 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/barbican-api-866c868748-bdl5k" podUID="4c57f5e8-fe35-4f32-b0b1-a3dc5d0ccf33" containerName="barbican-api-log" probeResult="failure" output="Get \"http://10.217.1.189:9311/healthcheck\": read tcp 10.217.0.2:42846->10.217.1.189:9311: read: connection reset by peer" Dec 03 12:43:14 crc kubenswrapper[5052]: I1203 12:43:14.155007 5052 generic.go:334] "Generic (PLEG): container finished" podID="4c57f5e8-fe35-4f32-b0b1-a3dc5d0ccf33" containerID="572799af7b7f291e065f218508d292c61aa6a90d909f324e94d90ae2b89316c9" exitCode=0 Dec 03 12:43:14 crc kubenswrapper[5052]: I1203 12:43:14.155248 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-866c868748-bdl5k" event={"ID":"4c57f5e8-fe35-4f32-b0b1-a3dc5d0ccf33","Type":"ContainerDied","Data":"572799af7b7f291e065f218508d292c61aa6a90d909f324e94d90ae2b89316c9"} Dec 03 12:43:14 crc kubenswrapper[5052]: I1203 12:43:14.613949 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-api-866c868748-bdl5k" Dec 03 12:43:14 crc kubenswrapper[5052]: I1203 12:43:14.671883 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/4c57f5e8-fe35-4f32-b0b1-a3dc5d0ccf33-logs\") pod \"4c57f5e8-fe35-4f32-b0b1-a3dc5d0ccf33\" (UID: \"4c57f5e8-fe35-4f32-b0b1-a3dc5d0ccf33\") " Dec 03 12:43:14 crc kubenswrapper[5052]: I1203 12:43:14.671959 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-87bqn\" (UniqueName: \"kubernetes.io/projected/4c57f5e8-fe35-4f32-b0b1-a3dc5d0ccf33-kube-api-access-87bqn\") pod \"4c57f5e8-fe35-4f32-b0b1-a3dc5d0ccf33\" (UID: \"4c57f5e8-fe35-4f32-b0b1-a3dc5d0ccf33\") " Dec 03 12:43:14 crc kubenswrapper[5052]: I1203 12:43:14.672112 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/4c57f5e8-fe35-4f32-b0b1-a3dc5d0ccf33-config-data\") pod \"4c57f5e8-fe35-4f32-b0b1-a3dc5d0ccf33\" (UID: \"4c57f5e8-fe35-4f32-b0b1-a3dc5d0ccf33\") " Dec 03 12:43:14 crc kubenswrapper[5052]: I1203 12:43:14.672192 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/4c57f5e8-fe35-4f32-b0b1-a3dc5d0ccf33-config-data-custom\") pod \"4c57f5e8-fe35-4f32-b0b1-a3dc5d0ccf33\" (UID: \"4c57f5e8-fe35-4f32-b0b1-a3dc5d0ccf33\") " Dec 03 12:43:14 crc kubenswrapper[5052]: I1203 12:43:14.672234 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4c57f5e8-fe35-4f32-b0b1-a3dc5d0ccf33-combined-ca-bundle\") pod \"4c57f5e8-fe35-4f32-b0b1-a3dc5d0ccf33\" (UID: \"4c57f5e8-fe35-4f32-b0b1-a3dc5d0ccf33\") " Dec 03 12:43:14 crc kubenswrapper[5052]: I1203 12:43:14.673253 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c57f5e8-fe35-4f32-b0b1-a3dc5d0ccf33-logs" (OuterVolumeSpecName: "logs") pod "4c57f5e8-fe35-4f32-b0b1-a3dc5d0ccf33" (UID: "4c57f5e8-fe35-4f32-b0b1-a3dc5d0ccf33"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:43:14 crc kubenswrapper[5052]: I1203 12:43:14.673571 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/4c57f5e8-fe35-4f32-b0b1-a3dc5d0ccf33-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:43:14 crc kubenswrapper[5052]: I1203 12:43:14.679765 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4c57f5e8-fe35-4f32-b0b1-a3dc5d0ccf33-config-data-custom" (OuterVolumeSpecName: "config-data-custom") pod "4c57f5e8-fe35-4f32-b0b1-a3dc5d0ccf33" (UID: "4c57f5e8-fe35-4f32-b0b1-a3dc5d0ccf33"). InnerVolumeSpecName "config-data-custom". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:43:14 crc kubenswrapper[5052]: I1203 12:43:14.682699 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4c57f5e8-fe35-4f32-b0b1-a3dc5d0ccf33-kube-api-access-87bqn" (OuterVolumeSpecName: "kube-api-access-87bqn") pod "4c57f5e8-fe35-4f32-b0b1-a3dc5d0ccf33" (UID: "4c57f5e8-fe35-4f32-b0b1-a3dc5d0ccf33"). InnerVolumeSpecName "kube-api-access-87bqn". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:43:14 crc kubenswrapper[5052]: I1203 12:43:14.725241 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4c57f5e8-fe35-4f32-b0b1-a3dc5d0ccf33-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "4c57f5e8-fe35-4f32-b0b1-a3dc5d0ccf33" (UID: "4c57f5e8-fe35-4f32-b0b1-a3dc5d0ccf33"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:43:14 crc kubenswrapper[5052]: I1203 12:43:14.734697 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4c57f5e8-fe35-4f32-b0b1-a3dc5d0ccf33-config-data" (OuterVolumeSpecName: "config-data") pod "4c57f5e8-fe35-4f32-b0b1-a3dc5d0ccf33" (UID: "4c57f5e8-fe35-4f32-b0b1-a3dc5d0ccf33"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:43:14 crc kubenswrapper[5052]: I1203 12:43:14.775983 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/4c57f5e8-fe35-4f32-b0b1-a3dc5d0ccf33-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:43:14 crc kubenswrapper[5052]: I1203 12:43:14.776037 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/4c57f5e8-fe35-4f32-b0b1-a3dc5d0ccf33-config-data-custom\") on node \"crc\" DevicePath \"\"" Dec 03 12:43:14 crc kubenswrapper[5052]: I1203 12:43:14.776053 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4c57f5e8-fe35-4f32-b0b1-a3dc5d0ccf33-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:43:14 crc kubenswrapper[5052]: I1203 12:43:14.776070 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-87bqn\" (UniqueName: \"kubernetes.io/projected/4c57f5e8-fe35-4f32-b0b1-a3dc5d0ccf33-kube-api-access-87bqn\") on node \"crc\" DevicePath \"\"" Dec 03 12:43:15 crc kubenswrapper[5052]: I1203 12:43:15.172436 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-866c868748-bdl5k" event={"ID":"4c57f5e8-fe35-4f32-b0b1-a3dc5d0ccf33","Type":"ContainerDied","Data":"8c7001d153f6c223ee493c5c05f995b394beefdef2772f671180aa50196e6178"} Dec 03 12:43:15 crc kubenswrapper[5052]: I1203 12:43:15.172478 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-api-866c868748-bdl5k" Dec 03 12:43:15 crc kubenswrapper[5052]: I1203 12:43:15.172542 5052 scope.go:117] "RemoveContainer" containerID="572799af7b7f291e065f218508d292c61aa6a90d909f324e94d90ae2b89316c9" Dec 03 12:43:15 crc kubenswrapper[5052]: I1203 12:43:15.205825 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-api-866c868748-bdl5k"] Dec 03 12:43:15 crc kubenswrapper[5052]: I1203 12:43:15.206258 5052 scope.go:117] "RemoveContainer" containerID="bd1fd8ba880b814dd955b69768ac28222dd0eeff96580f6fa7d3c52c5b458a07" Dec 03 12:43:15 crc kubenswrapper[5052]: I1203 12:43:15.213985 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/barbican-api-866c868748-bdl5k"] Dec 03 12:43:15 crc kubenswrapper[5052]: I1203 12:43:15.683555 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4c57f5e8-fe35-4f32-b0b1-a3dc5d0ccf33" path="/var/lib/kubelet/pods/4c57f5e8-fe35-4f32-b0b1-a3dc5d0ccf33/volumes" Dec 03 12:43:17 crc kubenswrapper[5052]: I1203 12:43:17.566377 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:43:19 crc kubenswrapper[5052]: I1203 12:43:19.016744 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/placement-56b4b79f66-gfsxn" Dec 03 12:43:19 crc kubenswrapper[5052]: I1203 12:43:19.018377 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/placement-56b4b79f66-gfsxn" Dec 03 12:43:19 crc kubenswrapper[5052]: I1203 12:43:19.853130 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:43:20 crc kubenswrapper[5052]: I1203 12:43:20.370940 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/neutron-64b7d8689c-p2dmd" Dec 03 12:43:27 crc kubenswrapper[5052]: I1203 12:43:27.514139 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/neutron-68f85bdc64-dc642" Dec 03 12:43:27 crc kubenswrapper[5052]: I1203 12:43:27.583791 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/neutron-64b7d8689c-p2dmd"] Dec 03 12:43:27 crc kubenswrapper[5052]: I1203 12:43:27.584066 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/neutron-64b7d8689c-p2dmd" podUID="c0261f94-be97-4cce-9eac-362d42248260" containerName="neutron-api" containerID="cri-o://5b04eb43cca61516127ebbe98e362a70bd97994640daed64b640e02a6705f087" gracePeriod=30 Dec 03 12:43:27 crc kubenswrapper[5052]: I1203 12:43:27.585489 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/neutron-64b7d8689c-p2dmd" podUID="c0261f94-be97-4cce-9eac-362d42248260" containerName="neutron-httpd" containerID="cri-o://b623beb1d698577700860795ffcf72792b261e732bcbf58537ca90d6307df621" gracePeriod=30 Dec 03 12:43:28 crc kubenswrapper[5052]: I1203 12:43:28.307962 5052 generic.go:334] "Generic (PLEG): container finished" podID="c0261f94-be97-4cce-9eac-362d42248260" containerID="b623beb1d698577700860795ffcf72792b261e732bcbf58537ca90d6307df621" exitCode=0 Dec 03 12:43:28 crc kubenswrapper[5052]: I1203 12:43:28.308026 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-64b7d8689c-p2dmd" event={"ID":"c0261f94-be97-4cce-9eac-362d42248260","Type":"ContainerDied","Data":"b623beb1d698577700860795ffcf72792b261e732bcbf58537ca90d6307df621"} Dec 03 12:43:29 crc kubenswrapper[5052]: I1203 12:43:29.201620 5052 scope.go:117] "RemoveContainer" containerID="89fab5932f39924d83d4ab819b2183e190ed6a7efae6f78ee3791c1efe360659" Dec 03 12:43:29 crc kubenswrapper[5052]: I1203 12:43:29.379406 5052 patch_prober.go:28] interesting pod/machine-config-daemon-74bj6 container/machine-config-daemon namespace/openshift-machine-config-operator: Liveness probe status=failure output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" start-of-body= Dec 03 12:43:29 crc kubenswrapper[5052]: I1203 12:43:29.379684 5052 prober.go:107] "Probe failed" probeType="Liveness" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" containerName="machine-config-daemon" probeResult="failure" output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" Dec 03 12:43:29 crc kubenswrapper[5052]: I1203 12:43:29.379715 5052 kubelet.go:2542] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" Dec 03 12:43:29 crc kubenswrapper[5052]: I1203 12:43:29.380334 5052 kuberuntime_manager.go:1027] "Message for Container of pod" containerName="machine-config-daemon" containerStatusID={"Type":"cri-o","ID":"2aed1c760a326f3d008799e902b1093313df3764a57c879f0b3686410127d37d"} pod="openshift-machine-config-operator/machine-config-daemon-74bj6" containerMessage="Container machine-config-daemon failed liveness probe, will be restarted" Dec 03 12:43:29 crc kubenswrapper[5052]: I1203 12:43:29.380391 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" containerName="machine-config-daemon" containerID="cri-o://2aed1c760a326f3d008799e902b1093313df3764a57c879f0b3686410127d37d" gracePeriod=600 Dec 03 12:43:29 crc kubenswrapper[5052]: E1203 12:43:29.586146 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 12:43:29 crc kubenswrapper[5052]: I1203 12:43:29.734704 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/keystone-7f7d6d4c6c-sjnw5" Dec 03 12:43:30 crc kubenswrapper[5052]: I1203 12:43:30.329242 5052 generic.go:334] "Generic (PLEG): container finished" podID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" containerID="2aed1c760a326f3d008799e902b1093313df3764a57c879f0b3686410127d37d" exitCode=0 Dec 03 12:43:30 crc kubenswrapper[5052]: I1203 12:43:30.329301 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" event={"ID":"10c237bc-72e7-4da3-9f2c-b1688395bf6e","Type":"ContainerDied","Data":"2aed1c760a326f3d008799e902b1093313df3764a57c879f0b3686410127d37d"} Dec 03 12:43:30 crc kubenswrapper[5052]: I1203 12:43:30.329341 5052 scope.go:117] "RemoveContainer" containerID="bccf12735e965b0db2889952a34202c267bb425fd32c44f3ef490bba9c0424d8" Dec 03 12:43:30 crc kubenswrapper[5052]: I1203 12:43:30.329957 5052 scope.go:117] "RemoveContainer" containerID="2aed1c760a326f3d008799e902b1093313df3764a57c879f0b3686410127d37d" Dec 03 12:43:30 crc kubenswrapper[5052]: E1203 12:43:30.330354 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 12:43:32 crc kubenswrapper[5052]: I1203 12:43:32.980391 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:43:32 crc kubenswrapper[5052]: I1203 12:43:32.981185 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="50a91cd1-d2b0-48ad-954c-17c18007e891" containerName="sg-core" containerID="cri-o://fd43670029bd26793307cd8493e9456ce76cd5ee5b0b7b75dabac34375b7b698" gracePeriod=30 Dec 03 12:43:32 crc kubenswrapper[5052]: I1203 12:43:32.981243 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="50a91cd1-d2b0-48ad-954c-17c18007e891" containerName="ceilometer-notification-agent" containerID="cri-o://656e397b6d9cc32a2edd6ab337d8cc3f86f9dbda299c0fa90721c51fd6111e14" gracePeriod=30 Dec 03 12:43:32 crc kubenswrapper[5052]: I1203 12:43:32.981264 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="50a91cd1-d2b0-48ad-954c-17c18007e891" containerName="proxy-httpd" containerID="cri-o://e880a804aafeeeb88e41aefe1189349e376a91868c5113e6db383cbf79761799" gracePeriod=30 Dec 03 12:43:32 crc kubenswrapper[5052]: I1203 12:43:32.981091 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="50a91cd1-d2b0-48ad-954c-17c18007e891" containerName="ceilometer-central-agent" containerID="cri-o://faac469fb234838cd16f1eedf3600b31b890b6868f17653562825912fa983149" gracePeriod=30 Dec 03 12:43:33 crc kubenswrapper[5052]: I1203 12:43:33.232685 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/openstackclient"] Dec 03 12:43:33 crc kubenswrapper[5052]: E1203 12:43:33.233087 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="4c57f5e8-fe35-4f32-b0b1-a3dc5d0ccf33" containerName="barbican-api-log" Dec 03 12:43:33 crc kubenswrapper[5052]: I1203 12:43:33.233098 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="4c57f5e8-fe35-4f32-b0b1-a3dc5d0ccf33" containerName="barbican-api-log" Dec 03 12:43:33 crc kubenswrapper[5052]: E1203 12:43:33.233114 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="4c57f5e8-fe35-4f32-b0b1-a3dc5d0ccf33" containerName="barbican-api" Dec 03 12:43:33 crc kubenswrapper[5052]: I1203 12:43:33.233119 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="4c57f5e8-fe35-4f32-b0b1-a3dc5d0ccf33" containerName="barbican-api" Dec 03 12:43:33 crc kubenswrapper[5052]: I1203 12:43:33.233325 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="4c57f5e8-fe35-4f32-b0b1-a3dc5d0ccf33" containerName="barbican-api" Dec 03 12:43:33 crc kubenswrapper[5052]: I1203 12:43:33.233345 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="4c57f5e8-fe35-4f32-b0b1-a3dc5d0ccf33" containerName="barbican-api-log" Dec 03 12:43:33 crc kubenswrapper[5052]: I1203 12:43:33.233997 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstackclient" Dec 03 12:43:33 crc kubenswrapper[5052]: I1203 12:43:33.237620 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"openstack-config" Dec 03 12:43:33 crc kubenswrapper[5052]: I1203 12:43:33.238004 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"openstackclient-openstackclient-dockercfg-n5mxr" Dec 03 12:43:33 crc kubenswrapper[5052]: I1203 12:43:33.238370 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"openstack-config-secret" Dec 03 12:43:33 crc kubenswrapper[5052]: I1203 12:43:33.238928 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/openstackclient"] Dec 03 12:43:33 crc kubenswrapper[5052]: I1203 12:43:33.325765 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openstack-config-secret\" (UniqueName: \"kubernetes.io/secret/6475384f-1965-4e8b-8806-af420d1381de-openstack-config-secret\") pod \"openstackclient\" (UID: \"6475384f-1965-4e8b-8806-af420d1381de\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 12:43:33 crc kubenswrapper[5052]: I1203 12:43:33.325839 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-chn6f\" (UniqueName: \"kubernetes.io/projected/6475384f-1965-4e8b-8806-af420d1381de-kube-api-access-chn6f\") pod \"openstackclient\" (UID: \"6475384f-1965-4e8b-8806-af420d1381de\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 12:43:33 crc kubenswrapper[5052]: I1203 12:43:33.325944 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openstack-config\" (UniqueName: \"kubernetes.io/configmap/6475384f-1965-4e8b-8806-af420d1381de-openstack-config\") pod \"openstackclient\" (UID: \"6475384f-1965-4e8b-8806-af420d1381de\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 12:43:33 crc kubenswrapper[5052]: I1203 12:43:33.326069 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6475384f-1965-4e8b-8806-af420d1381de-combined-ca-bundle\") pod \"openstackclient\" (UID: \"6475384f-1965-4e8b-8806-af420d1381de\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 12:43:33 crc kubenswrapper[5052]: I1203 12:43:33.363982 5052 generic.go:334] "Generic (PLEG): container finished" podID="50a91cd1-d2b0-48ad-954c-17c18007e891" containerID="e880a804aafeeeb88e41aefe1189349e376a91868c5113e6db383cbf79761799" exitCode=0 Dec 03 12:43:33 crc kubenswrapper[5052]: I1203 12:43:33.364011 5052 generic.go:334] "Generic (PLEG): container finished" podID="50a91cd1-d2b0-48ad-954c-17c18007e891" containerID="fd43670029bd26793307cd8493e9456ce76cd5ee5b0b7b75dabac34375b7b698" exitCode=2 Dec 03 12:43:33 crc kubenswrapper[5052]: I1203 12:43:33.364047 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"50a91cd1-d2b0-48ad-954c-17c18007e891","Type":"ContainerDied","Data":"e880a804aafeeeb88e41aefe1189349e376a91868c5113e6db383cbf79761799"} Dec 03 12:43:33 crc kubenswrapper[5052]: I1203 12:43:33.364072 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"50a91cd1-d2b0-48ad-954c-17c18007e891","Type":"ContainerDied","Data":"fd43670029bd26793307cd8493e9456ce76cd5ee5b0b7b75dabac34375b7b698"} Dec 03 12:43:33 crc kubenswrapper[5052]: I1203 12:43:33.366747 5052 generic.go:334] "Generic (PLEG): container finished" podID="c0261f94-be97-4cce-9eac-362d42248260" containerID="5b04eb43cca61516127ebbe98e362a70bd97994640daed64b640e02a6705f087" exitCode=0 Dec 03 12:43:33 crc kubenswrapper[5052]: I1203 12:43:33.366773 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-64b7d8689c-p2dmd" event={"ID":"c0261f94-be97-4cce-9eac-362d42248260","Type":"ContainerDied","Data":"5b04eb43cca61516127ebbe98e362a70bd97994640daed64b640e02a6705f087"} Dec 03 12:43:33 crc kubenswrapper[5052]: I1203 12:43:33.366791 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-64b7d8689c-p2dmd" event={"ID":"c0261f94-be97-4cce-9eac-362d42248260","Type":"ContainerDied","Data":"c217b642a41725c40e7562f2d02446ddbab47d91f3bfeb0a9c3bb85a90aab8a8"} Dec 03 12:43:33 crc kubenswrapper[5052]: I1203 12:43:33.366802 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c217b642a41725c40e7562f2d02446ddbab47d91f3bfeb0a9c3bb85a90aab8a8" Dec 03 12:43:33 crc kubenswrapper[5052]: I1203 12:43:33.381454 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-64b7d8689c-p2dmd" Dec 03 12:43:33 crc kubenswrapper[5052]: I1203 12:43:33.427759 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6475384f-1965-4e8b-8806-af420d1381de-combined-ca-bundle\") pod \"openstackclient\" (UID: \"6475384f-1965-4e8b-8806-af420d1381de\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 12:43:33 crc kubenswrapper[5052]: I1203 12:43:33.427879 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"openstack-config-secret\" (UniqueName: \"kubernetes.io/secret/6475384f-1965-4e8b-8806-af420d1381de-openstack-config-secret\") pod \"openstackclient\" (UID: \"6475384f-1965-4e8b-8806-af420d1381de\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 12:43:33 crc kubenswrapper[5052]: I1203 12:43:33.429412 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-chn6f\" (UniqueName: \"kubernetes.io/projected/6475384f-1965-4e8b-8806-af420d1381de-kube-api-access-chn6f\") pod \"openstackclient\" (UID: \"6475384f-1965-4e8b-8806-af420d1381de\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 12:43:33 crc kubenswrapper[5052]: I1203 12:43:33.429577 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"openstack-config\" (UniqueName: \"kubernetes.io/configmap/6475384f-1965-4e8b-8806-af420d1381de-openstack-config\") pod \"openstackclient\" (UID: \"6475384f-1965-4e8b-8806-af420d1381de\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 12:43:33 crc kubenswrapper[5052]: I1203 12:43:33.430488 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"openstack-config\" (UniqueName: \"kubernetes.io/configmap/6475384f-1965-4e8b-8806-af420d1381de-openstack-config\") pod \"openstackclient\" (UID: \"6475384f-1965-4e8b-8806-af420d1381de\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 12:43:33 crc kubenswrapper[5052]: I1203 12:43:33.438921 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"openstack-config-secret\" (UniqueName: \"kubernetes.io/secret/6475384f-1965-4e8b-8806-af420d1381de-openstack-config-secret\") pod \"openstackclient\" (UID: \"6475384f-1965-4e8b-8806-af420d1381de\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 12:43:33 crc kubenswrapper[5052]: I1203 12:43:33.440201 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6475384f-1965-4e8b-8806-af420d1381de-combined-ca-bundle\") pod \"openstackclient\" (UID: \"6475384f-1965-4e8b-8806-af420d1381de\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 12:43:33 crc kubenswrapper[5052]: I1203 12:43:33.458536 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-chn6f\" (UniqueName: \"kubernetes.io/projected/6475384f-1965-4e8b-8806-af420d1381de-kube-api-access-chn6f\") pod \"openstackclient\" (UID: \"6475384f-1965-4e8b-8806-af420d1381de\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 12:43:33 crc kubenswrapper[5052]: I1203 12:43:33.530457 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/c0261f94-be97-4cce-9eac-362d42248260-httpd-config\") pod \"c0261f94-be97-4cce-9eac-362d42248260\" (UID: \"c0261f94-be97-4cce-9eac-362d42248260\") " Dec 03 12:43:33 crc kubenswrapper[5052]: I1203 12:43:33.530606 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/c0261f94-be97-4cce-9eac-362d42248260-config\") pod \"c0261f94-be97-4cce-9eac-362d42248260\" (UID: \"c0261f94-be97-4cce-9eac-362d42248260\") " Dec 03 12:43:33 crc kubenswrapper[5052]: I1203 12:43:33.530673 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q5qrs\" (UniqueName: \"kubernetes.io/projected/c0261f94-be97-4cce-9eac-362d42248260-kube-api-access-q5qrs\") pod \"c0261f94-be97-4cce-9eac-362d42248260\" (UID: \"c0261f94-be97-4cce-9eac-362d42248260\") " Dec 03 12:43:33 crc kubenswrapper[5052]: I1203 12:43:33.530697 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/c0261f94-be97-4cce-9eac-362d42248260-ovndb-tls-certs\") pod \"c0261f94-be97-4cce-9eac-362d42248260\" (UID: \"c0261f94-be97-4cce-9eac-362d42248260\") " Dec 03 12:43:33 crc kubenswrapper[5052]: I1203 12:43:33.530765 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c0261f94-be97-4cce-9eac-362d42248260-combined-ca-bundle\") pod \"c0261f94-be97-4cce-9eac-362d42248260\" (UID: \"c0261f94-be97-4cce-9eac-362d42248260\") " Dec 03 12:43:33 crc kubenswrapper[5052]: I1203 12:43:33.533920 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c0261f94-be97-4cce-9eac-362d42248260-httpd-config" (OuterVolumeSpecName: "httpd-config") pod "c0261f94-be97-4cce-9eac-362d42248260" (UID: "c0261f94-be97-4cce-9eac-362d42248260"). InnerVolumeSpecName "httpd-config". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:43:33 crc kubenswrapper[5052]: I1203 12:43:33.539708 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c0261f94-be97-4cce-9eac-362d42248260-kube-api-access-q5qrs" (OuterVolumeSpecName: "kube-api-access-q5qrs") pod "c0261f94-be97-4cce-9eac-362d42248260" (UID: "c0261f94-be97-4cce-9eac-362d42248260"). InnerVolumeSpecName "kube-api-access-q5qrs". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:43:33 crc kubenswrapper[5052]: I1203 12:43:33.564819 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstackclient" Dec 03 12:43:33 crc kubenswrapper[5052]: I1203 12:43:33.585309 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c0261f94-be97-4cce-9eac-362d42248260-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "c0261f94-be97-4cce-9eac-362d42248260" (UID: "c0261f94-be97-4cce-9eac-362d42248260"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:43:33 crc kubenswrapper[5052]: I1203 12:43:33.613748 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c0261f94-be97-4cce-9eac-362d42248260-config" (OuterVolumeSpecName: "config") pod "c0261f94-be97-4cce-9eac-362d42248260" (UID: "c0261f94-be97-4cce-9eac-362d42248260"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:43:33 crc kubenswrapper[5052]: I1203 12:43:33.633685 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c0261f94-be97-4cce-9eac-362d42248260-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:43:33 crc kubenswrapper[5052]: I1203 12:43:33.633729 5052 reconciler_common.go:293] "Volume detached for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/c0261f94-be97-4cce-9eac-362d42248260-httpd-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:43:33 crc kubenswrapper[5052]: I1203 12:43:33.633744 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/c0261f94-be97-4cce-9eac-362d42248260-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:43:33 crc kubenswrapper[5052]: I1203 12:43:33.633758 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-q5qrs\" (UniqueName: \"kubernetes.io/projected/c0261f94-be97-4cce-9eac-362d42248260-kube-api-access-q5qrs\") on node \"crc\" DevicePath \"\"" Dec 03 12:43:33 crc kubenswrapper[5052]: I1203 12:43:33.657778 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c0261f94-be97-4cce-9eac-362d42248260-ovndb-tls-certs" (OuterVolumeSpecName: "ovndb-tls-certs") pod "c0261f94-be97-4cce-9eac-362d42248260" (UID: "c0261f94-be97-4cce-9eac-362d42248260"). InnerVolumeSpecName "ovndb-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:43:33 crc kubenswrapper[5052]: I1203 12:43:33.735809 5052 reconciler_common.go:293] "Volume detached for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/c0261f94-be97-4cce-9eac-362d42248260-ovndb-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:43:34 crc kubenswrapper[5052]: I1203 12:43:34.045146 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/openstackclient"] Dec 03 12:43:34 crc kubenswrapper[5052]: I1203 12:43:34.381650 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstackclient" event={"ID":"6475384f-1965-4e8b-8806-af420d1381de","Type":"ContainerStarted","Data":"06fb78df44fbba56afa104136ade5fc0d9d9f01772c05059cf7f5e2ec493e511"} Dec 03 12:43:34 crc kubenswrapper[5052]: I1203 12:43:34.382055 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstackclient" event={"ID":"6475384f-1965-4e8b-8806-af420d1381de","Type":"ContainerStarted","Data":"51bfbf9924d4614ecdc2ebbb74f253c7ff74716cfae76c233754dcb1b24cf749"} Dec 03 12:43:34 crc kubenswrapper[5052]: I1203 12:43:34.384805 5052 generic.go:334] "Generic (PLEG): container finished" podID="50a91cd1-d2b0-48ad-954c-17c18007e891" containerID="faac469fb234838cd16f1eedf3600b31b890b6868f17653562825912fa983149" exitCode=0 Dec 03 12:43:34 crc kubenswrapper[5052]: I1203 12:43:34.384887 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-64b7d8689c-p2dmd" Dec 03 12:43:34 crc kubenswrapper[5052]: I1203 12:43:34.384899 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"50a91cd1-d2b0-48ad-954c-17c18007e891","Type":"ContainerDied","Data":"faac469fb234838cd16f1eedf3600b31b890b6868f17653562825912fa983149"} Dec 03 12:43:34 crc kubenswrapper[5052]: I1203 12:43:34.410885 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/openstackclient" podStartSLOduration=1.410862217 podStartE2EDuration="1.410862217s" podCreationTimestamp="2025-12-03 12:43:33 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:43:34.397025402 +0000 UTC m=+5004.985267166" watchObservedRunningTime="2025-12-03 12:43:34.410862217 +0000 UTC m=+5004.999103981" Dec 03 12:43:34 crc kubenswrapper[5052]: I1203 12:43:34.428578 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/neutron-64b7d8689c-p2dmd"] Dec 03 12:43:34 crc kubenswrapper[5052]: I1203 12:43:34.437294 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/neutron-64b7d8689c-p2dmd"] Dec 03 12:43:35 crc kubenswrapper[5052]: I1203 12:43:35.405297 5052 generic.go:334] "Generic (PLEG): container finished" podID="50a91cd1-d2b0-48ad-954c-17c18007e891" containerID="656e397b6d9cc32a2edd6ab337d8cc3f86f9dbda299c0fa90721c51fd6111e14" exitCode=0 Dec 03 12:43:35 crc kubenswrapper[5052]: I1203 12:43:35.405433 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"50a91cd1-d2b0-48ad-954c-17c18007e891","Type":"ContainerDied","Data":"656e397b6d9cc32a2edd6ab337d8cc3f86f9dbda299c0fa90721c51fd6111e14"} Dec 03 12:43:35 crc kubenswrapper[5052]: I1203 12:43:35.582323 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:43:35 crc kubenswrapper[5052]: I1203 12:43:35.686134 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c0261f94-be97-4cce-9eac-362d42248260" path="/var/lib/kubelet/pods/c0261f94-be97-4cce-9eac-362d42248260/volumes" Dec 03 12:43:35 crc kubenswrapper[5052]: I1203 12:43:35.694615 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/50a91cd1-d2b0-48ad-954c-17c18007e891-combined-ca-bundle\") pod \"50a91cd1-d2b0-48ad-954c-17c18007e891\" (UID: \"50a91cd1-d2b0-48ad-954c-17c18007e891\") " Dec 03 12:43:35 crc kubenswrapper[5052]: I1203 12:43:35.694681 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/50a91cd1-d2b0-48ad-954c-17c18007e891-log-httpd\") pod \"50a91cd1-d2b0-48ad-954c-17c18007e891\" (UID: \"50a91cd1-d2b0-48ad-954c-17c18007e891\") " Dec 03 12:43:35 crc kubenswrapper[5052]: I1203 12:43:35.694727 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/50a91cd1-d2b0-48ad-954c-17c18007e891-sg-core-conf-yaml\") pod \"50a91cd1-d2b0-48ad-954c-17c18007e891\" (UID: \"50a91cd1-d2b0-48ad-954c-17c18007e891\") " Dec 03 12:43:35 crc kubenswrapper[5052]: I1203 12:43:35.694856 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/50a91cd1-d2b0-48ad-954c-17c18007e891-run-httpd\") pod \"50a91cd1-d2b0-48ad-954c-17c18007e891\" (UID: \"50a91cd1-d2b0-48ad-954c-17c18007e891\") " Dec 03 12:43:35 crc kubenswrapper[5052]: I1203 12:43:35.694905 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/50a91cd1-d2b0-48ad-954c-17c18007e891-config-data\") pod \"50a91cd1-d2b0-48ad-954c-17c18007e891\" (UID: \"50a91cd1-d2b0-48ad-954c-17c18007e891\") " Dec 03 12:43:35 crc kubenswrapper[5052]: I1203 12:43:35.694972 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/50a91cd1-d2b0-48ad-954c-17c18007e891-scripts\") pod \"50a91cd1-d2b0-48ad-954c-17c18007e891\" (UID: \"50a91cd1-d2b0-48ad-954c-17c18007e891\") " Dec 03 12:43:35 crc kubenswrapper[5052]: I1203 12:43:35.695036 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mbpg7\" (UniqueName: \"kubernetes.io/projected/50a91cd1-d2b0-48ad-954c-17c18007e891-kube-api-access-mbpg7\") pod \"50a91cd1-d2b0-48ad-954c-17c18007e891\" (UID: \"50a91cd1-d2b0-48ad-954c-17c18007e891\") " Dec 03 12:43:35 crc kubenswrapper[5052]: I1203 12:43:35.696547 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/50a91cd1-d2b0-48ad-954c-17c18007e891-log-httpd" (OuterVolumeSpecName: "log-httpd") pod "50a91cd1-d2b0-48ad-954c-17c18007e891" (UID: "50a91cd1-d2b0-48ad-954c-17c18007e891"). InnerVolumeSpecName "log-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:43:35 crc kubenswrapper[5052]: I1203 12:43:35.696813 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/50a91cd1-d2b0-48ad-954c-17c18007e891-run-httpd" (OuterVolumeSpecName: "run-httpd") pod "50a91cd1-d2b0-48ad-954c-17c18007e891" (UID: "50a91cd1-d2b0-48ad-954c-17c18007e891"). InnerVolumeSpecName "run-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:43:35 crc kubenswrapper[5052]: I1203 12:43:35.701222 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/50a91cd1-d2b0-48ad-954c-17c18007e891-kube-api-access-mbpg7" (OuterVolumeSpecName: "kube-api-access-mbpg7") pod "50a91cd1-d2b0-48ad-954c-17c18007e891" (UID: "50a91cd1-d2b0-48ad-954c-17c18007e891"). InnerVolumeSpecName "kube-api-access-mbpg7". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:43:35 crc kubenswrapper[5052]: I1203 12:43:35.702896 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/50a91cd1-d2b0-48ad-954c-17c18007e891-scripts" (OuterVolumeSpecName: "scripts") pod "50a91cd1-d2b0-48ad-954c-17c18007e891" (UID: "50a91cd1-d2b0-48ad-954c-17c18007e891"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:43:35 crc kubenswrapper[5052]: I1203 12:43:35.742725 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/50a91cd1-d2b0-48ad-954c-17c18007e891-sg-core-conf-yaml" (OuterVolumeSpecName: "sg-core-conf-yaml") pod "50a91cd1-d2b0-48ad-954c-17c18007e891" (UID: "50a91cd1-d2b0-48ad-954c-17c18007e891"). InnerVolumeSpecName "sg-core-conf-yaml". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:43:35 crc kubenswrapper[5052]: I1203 12:43:35.783632 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/50a91cd1-d2b0-48ad-954c-17c18007e891-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "50a91cd1-d2b0-48ad-954c-17c18007e891" (UID: "50a91cd1-d2b0-48ad-954c-17c18007e891"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:43:35 crc kubenswrapper[5052]: I1203 12:43:35.797035 5052 reconciler_common.go:293] "Volume detached for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/50a91cd1-d2b0-48ad-954c-17c18007e891-run-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:43:35 crc kubenswrapper[5052]: I1203 12:43:35.797272 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/50a91cd1-d2b0-48ad-954c-17c18007e891-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:43:35 crc kubenswrapper[5052]: I1203 12:43:35.797359 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-mbpg7\" (UniqueName: \"kubernetes.io/projected/50a91cd1-d2b0-48ad-954c-17c18007e891-kube-api-access-mbpg7\") on node \"crc\" DevicePath \"\"" Dec 03 12:43:35 crc kubenswrapper[5052]: I1203 12:43:35.797438 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/50a91cd1-d2b0-48ad-954c-17c18007e891-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:43:35 crc kubenswrapper[5052]: I1203 12:43:35.797615 5052 reconciler_common.go:293] "Volume detached for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/50a91cd1-d2b0-48ad-954c-17c18007e891-log-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:43:35 crc kubenswrapper[5052]: I1203 12:43:35.797701 5052 reconciler_common.go:293] "Volume detached for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/50a91cd1-d2b0-48ad-954c-17c18007e891-sg-core-conf-yaml\") on node \"crc\" DevicePath \"\"" Dec 03 12:43:35 crc kubenswrapper[5052]: I1203 12:43:35.825500 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/50a91cd1-d2b0-48ad-954c-17c18007e891-config-data" (OuterVolumeSpecName: "config-data") pod "50a91cd1-d2b0-48ad-954c-17c18007e891" (UID: "50a91cd1-d2b0-48ad-954c-17c18007e891"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:43:35 crc kubenswrapper[5052]: I1203 12:43:35.900529 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/50a91cd1-d2b0-48ad-954c-17c18007e891-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:43:36 crc kubenswrapper[5052]: I1203 12:43:36.157046 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/swift-proxy-dd4cf57c8-x4c45"] Dec 03 12:43:36 crc kubenswrapper[5052]: E1203 12:43:36.157439 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="50a91cd1-d2b0-48ad-954c-17c18007e891" containerName="ceilometer-notification-agent" Dec 03 12:43:36 crc kubenswrapper[5052]: I1203 12:43:36.157459 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="50a91cd1-d2b0-48ad-954c-17c18007e891" containerName="ceilometer-notification-agent" Dec 03 12:43:36 crc kubenswrapper[5052]: E1203 12:43:36.157474 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="50a91cd1-d2b0-48ad-954c-17c18007e891" containerName="proxy-httpd" Dec 03 12:43:36 crc kubenswrapper[5052]: I1203 12:43:36.157485 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="50a91cd1-d2b0-48ad-954c-17c18007e891" containerName="proxy-httpd" Dec 03 12:43:36 crc kubenswrapper[5052]: E1203 12:43:36.157496 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="50a91cd1-d2b0-48ad-954c-17c18007e891" containerName="sg-core" Dec 03 12:43:36 crc kubenswrapper[5052]: I1203 12:43:36.157503 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="50a91cd1-d2b0-48ad-954c-17c18007e891" containerName="sg-core" Dec 03 12:43:36 crc kubenswrapper[5052]: E1203 12:43:36.157543 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="c0261f94-be97-4cce-9eac-362d42248260" containerName="neutron-api" Dec 03 12:43:36 crc kubenswrapper[5052]: I1203 12:43:36.157551 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="c0261f94-be97-4cce-9eac-362d42248260" containerName="neutron-api" Dec 03 12:43:36 crc kubenswrapper[5052]: E1203 12:43:36.157563 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="50a91cd1-d2b0-48ad-954c-17c18007e891" containerName="ceilometer-central-agent" Dec 03 12:43:36 crc kubenswrapper[5052]: I1203 12:43:36.157570 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="50a91cd1-d2b0-48ad-954c-17c18007e891" containerName="ceilometer-central-agent" Dec 03 12:43:36 crc kubenswrapper[5052]: E1203 12:43:36.157590 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="c0261f94-be97-4cce-9eac-362d42248260" containerName="neutron-httpd" Dec 03 12:43:36 crc kubenswrapper[5052]: I1203 12:43:36.157596 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="c0261f94-be97-4cce-9eac-362d42248260" containerName="neutron-httpd" Dec 03 12:43:36 crc kubenswrapper[5052]: I1203 12:43:36.157776 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="50a91cd1-d2b0-48ad-954c-17c18007e891" containerName="ceilometer-notification-agent" Dec 03 12:43:36 crc kubenswrapper[5052]: I1203 12:43:36.157830 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="c0261f94-be97-4cce-9eac-362d42248260" containerName="neutron-httpd" Dec 03 12:43:36 crc kubenswrapper[5052]: I1203 12:43:36.157844 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="50a91cd1-d2b0-48ad-954c-17c18007e891" containerName="proxy-httpd" Dec 03 12:43:36 crc kubenswrapper[5052]: I1203 12:43:36.157866 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="50a91cd1-d2b0-48ad-954c-17c18007e891" containerName="ceilometer-central-agent" Dec 03 12:43:36 crc kubenswrapper[5052]: I1203 12:43:36.157884 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="50a91cd1-d2b0-48ad-954c-17c18007e891" containerName="sg-core" Dec 03 12:43:36 crc kubenswrapper[5052]: I1203 12:43:36.157900 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="c0261f94-be97-4cce-9eac-362d42248260" containerName="neutron-api" Dec 03 12:43:36 crc kubenswrapper[5052]: I1203 12:43:36.158962 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-proxy-dd4cf57c8-x4c45" Dec 03 12:43:36 crc kubenswrapper[5052]: I1203 12:43:36.161324 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"swift-proxy-config-data" Dec 03 12:43:36 crc kubenswrapper[5052]: I1203 12:43:36.161627 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-swift-public-svc" Dec 03 12:43:36 crc kubenswrapper[5052]: I1203 12:43:36.164183 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-swift-internal-svc" Dec 03 12:43:36 crc kubenswrapper[5052]: I1203 12:43:36.170859 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/swift-proxy-dd4cf57c8-x4c45"] Dec 03 12:43:36 crc kubenswrapper[5052]: I1203 12:43:36.221176 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openshift-marketplace/community-operators-gt2q8"] Dec 03 12:43:36 crc kubenswrapper[5052]: I1203 12:43:36.223553 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/community-operators-gt2q8" Dec 03 12:43:36 crc kubenswrapper[5052]: I1203 12:43:36.254115 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/community-operators-gt2q8"] Dec 03 12:43:36 crc kubenswrapper[5052]: I1203 12:43:36.308602 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gp8vs\" (UniqueName: \"kubernetes.io/projected/242be027-6461-4aea-ae7c-e52ca996b2ea-kube-api-access-gp8vs\") pod \"swift-proxy-dd4cf57c8-x4c45\" (UID: \"242be027-6461-4aea-ae7c-e52ca996b2ea\") " pod="openstack-kuttl-tests/swift-proxy-dd4cf57c8-x4c45" Dec 03 12:43:36 crc kubenswrapper[5052]: I1203 12:43:36.308653 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-98x49\" (UniqueName: \"kubernetes.io/projected/3f17a210-d54b-4753-91c1-2d9b97565b4e-kube-api-access-98x49\") pod \"community-operators-gt2q8\" (UID: \"3f17a210-d54b-4753-91c1-2d9b97565b4e\") " pod="openshift-marketplace/community-operators-gt2q8" Dec 03 12:43:36 crc kubenswrapper[5052]: I1203 12:43:36.308677 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/3f17a210-d54b-4753-91c1-2d9b97565b4e-utilities\") pod \"community-operators-gt2q8\" (UID: \"3f17a210-d54b-4753-91c1-2d9b97565b4e\") " pod="openshift-marketplace/community-operators-gt2q8" Dec 03 12:43:36 crc kubenswrapper[5052]: I1203 12:43:36.308705 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/242be027-6461-4aea-ae7c-e52ca996b2ea-etc-swift\") pod \"swift-proxy-dd4cf57c8-x4c45\" (UID: \"242be027-6461-4aea-ae7c-e52ca996b2ea\") " pod="openstack-kuttl-tests/swift-proxy-dd4cf57c8-x4c45" Dec 03 12:43:36 crc kubenswrapper[5052]: I1203 12:43:36.308721 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/3f17a210-d54b-4753-91c1-2d9b97565b4e-catalog-content\") pod \"community-operators-gt2q8\" (UID: \"3f17a210-d54b-4753-91c1-2d9b97565b4e\") " pod="openshift-marketplace/community-operators-gt2q8" Dec 03 12:43:36 crc kubenswrapper[5052]: I1203 12:43:36.308754 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/242be027-6461-4aea-ae7c-e52ca996b2ea-internal-tls-certs\") pod \"swift-proxy-dd4cf57c8-x4c45\" (UID: \"242be027-6461-4aea-ae7c-e52ca996b2ea\") " pod="openstack-kuttl-tests/swift-proxy-dd4cf57c8-x4c45" Dec 03 12:43:36 crc kubenswrapper[5052]: I1203 12:43:36.308774 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/242be027-6461-4aea-ae7c-e52ca996b2ea-combined-ca-bundle\") pod \"swift-proxy-dd4cf57c8-x4c45\" (UID: \"242be027-6461-4aea-ae7c-e52ca996b2ea\") " pod="openstack-kuttl-tests/swift-proxy-dd4cf57c8-x4c45" Dec 03 12:43:36 crc kubenswrapper[5052]: I1203 12:43:36.308802 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/242be027-6461-4aea-ae7c-e52ca996b2ea-log-httpd\") pod \"swift-proxy-dd4cf57c8-x4c45\" (UID: \"242be027-6461-4aea-ae7c-e52ca996b2ea\") " pod="openstack-kuttl-tests/swift-proxy-dd4cf57c8-x4c45" Dec 03 12:43:36 crc kubenswrapper[5052]: I1203 12:43:36.308826 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/242be027-6461-4aea-ae7c-e52ca996b2ea-config-data\") pod \"swift-proxy-dd4cf57c8-x4c45\" (UID: \"242be027-6461-4aea-ae7c-e52ca996b2ea\") " pod="openstack-kuttl-tests/swift-proxy-dd4cf57c8-x4c45" Dec 03 12:43:36 crc kubenswrapper[5052]: I1203 12:43:36.308842 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/242be027-6461-4aea-ae7c-e52ca996b2ea-public-tls-certs\") pod \"swift-proxy-dd4cf57c8-x4c45\" (UID: \"242be027-6461-4aea-ae7c-e52ca996b2ea\") " pod="openstack-kuttl-tests/swift-proxy-dd4cf57c8-x4c45" Dec 03 12:43:36 crc kubenswrapper[5052]: I1203 12:43:36.308900 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/242be027-6461-4aea-ae7c-e52ca996b2ea-run-httpd\") pod \"swift-proxy-dd4cf57c8-x4c45\" (UID: \"242be027-6461-4aea-ae7c-e52ca996b2ea\") " pod="openstack-kuttl-tests/swift-proxy-dd4cf57c8-x4c45" Dec 03 12:43:36 crc kubenswrapper[5052]: I1203 12:43:36.410035 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/242be027-6461-4aea-ae7c-e52ca996b2ea-run-httpd\") pod \"swift-proxy-dd4cf57c8-x4c45\" (UID: \"242be027-6461-4aea-ae7c-e52ca996b2ea\") " pod="openstack-kuttl-tests/swift-proxy-dd4cf57c8-x4c45" Dec 03 12:43:36 crc kubenswrapper[5052]: I1203 12:43:36.410756 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-gp8vs\" (UniqueName: \"kubernetes.io/projected/242be027-6461-4aea-ae7c-e52ca996b2ea-kube-api-access-gp8vs\") pod \"swift-proxy-dd4cf57c8-x4c45\" (UID: \"242be027-6461-4aea-ae7c-e52ca996b2ea\") " pod="openstack-kuttl-tests/swift-proxy-dd4cf57c8-x4c45" Dec 03 12:43:36 crc kubenswrapper[5052]: I1203 12:43:36.410863 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-98x49\" (UniqueName: \"kubernetes.io/projected/3f17a210-d54b-4753-91c1-2d9b97565b4e-kube-api-access-98x49\") pod \"community-operators-gt2q8\" (UID: \"3f17a210-d54b-4753-91c1-2d9b97565b4e\") " pod="openshift-marketplace/community-operators-gt2q8" Dec 03 12:43:36 crc kubenswrapper[5052]: I1203 12:43:36.410960 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/3f17a210-d54b-4753-91c1-2d9b97565b4e-utilities\") pod \"community-operators-gt2q8\" (UID: \"3f17a210-d54b-4753-91c1-2d9b97565b4e\") " pod="openshift-marketplace/community-operators-gt2q8" Dec 03 12:43:36 crc kubenswrapper[5052]: I1203 12:43:36.410753 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/242be027-6461-4aea-ae7c-e52ca996b2ea-run-httpd\") pod \"swift-proxy-dd4cf57c8-x4c45\" (UID: \"242be027-6461-4aea-ae7c-e52ca996b2ea\") " pod="openstack-kuttl-tests/swift-proxy-dd4cf57c8-x4c45" Dec 03 12:43:36 crc kubenswrapper[5052]: I1203 12:43:36.411133 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/3f17a210-d54b-4753-91c1-2d9b97565b4e-catalog-content\") pod \"community-operators-gt2q8\" (UID: \"3f17a210-d54b-4753-91c1-2d9b97565b4e\") " pod="openshift-marketplace/community-operators-gt2q8" Dec 03 12:43:36 crc kubenswrapper[5052]: I1203 12:43:36.411236 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/242be027-6461-4aea-ae7c-e52ca996b2ea-etc-swift\") pod \"swift-proxy-dd4cf57c8-x4c45\" (UID: \"242be027-6461-4aea-ae7c-e52ca996b2ea\") " pod="openstack-kuttl-tests/swift-proxy-dd4cf57c8-x4c45" Dec 03 12:43:36 crc kubenswrapper[5052]: I1203 12:43:36.411395 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/242be027-6461-4aea-ae7c-e52ca996b2ea-internal-tls-certs\") pod \"swift-proxy-dd4cf57c8-x4c45\" (UID: \"242be027-6461-4aea-ae7c-e52ca996b2ea\") " pod="openstack-kuttl-tests/swift-proxy-dd4cf57c8-x4c45" Dec 03 12:43:36 crc kubenswrapper[5052]: I1203 12:43:36.411544 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/242be027-6461-4aea-ae7c-e52ca996b2ea-combined-ca-bundle\") pod \"swift-proxy-dd4cf57c8-x4c45\" (UID: \"242be027-6461-4aea-ae7c-e52ca996b2ea\") " pod="openstack-kuttl-tests/swift-proxy-dd4cf57c8-x4c45" Dec 03 12:43:36 crc kubenswrapper[5052]: I1203 12:43:36.411695 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/242be027-6461-4aea-ae7c-e52ca996b2ea-log-httpd\") pod \"swift-proxy-dd4cf57c8-x4c45\" (UID: \"242be027-6461-4aea-ae7c-e52ca996b2ea\") " pod="openstack-kuttl-tests/swift-proxy-dd4cf57c8-x4c45" Dec 03 12:43:36 crc kubenswrapper[5052]: I1203 12:43:36.411765 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/3f17a210-d54b-4753-91c1-2d9b97565b4e-utilities\") pod \"community-operators-gt2q8\" (UID: \"3f17a210-d54b-4753-91c1-2d9b97565b4e\") " pod="openshift-marketplace/community-operators-gt2q8" Dec 03 12:43:36 crc kubenswrapper[5052]: I1203 12:43:36.411725 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/3f17a210-d54b-4753-91c1-2d9b97565b4e-catalog-content\") pod \"community-operators-gt2q8\" (UID: \"3f17a210-d54b-4753-91c1-2d9b97565b4e\") " pod="openshift-marketplace/community-operators-gt2q8" Dec 03 12:43:36 crc kubenswrapper[5052]: I1203 12:43:36.411932 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/242be027-6461-4aea-ae7c-e52ca996b2ea-config-data\") pod \"swift-proxy-dd4cf57c8-x4c45\" (UID: \"242be027-6461-4aea-ae7c-e52ca996b2ea\") " pod="openstack-kuttl-tests/swift-proxy-dd4cf57c8-x4c45" Dec 03 12:43:36 crc kubenswrapper[5052]: I1203 12:43:36.412036 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/242be027-6461-4aea-ae7c-e52ca996b2ea-public-tls-certs\") pod \"swift-proxy-dd4cf57c8-x4c45\" (UID: \"242be027-6461-4aea-ae7c-e52ca996b2ea\") " pod="openstack-kuttl-tests/swift-proxy-dd4cf57c8-x4c45" Dec 03 12:43:36 crc kubenswrapper[5052]: I1203 12:43:36.412045 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/242be027-6461-4aea-ae7c-e52ca996b2ea-log-httpd\") pod \"swift-proxy-dd4cf57c8-x4c45\" (UID: \"242be027-6461-4aea-ae7c-e52ca996b2ea\") " pod="openstack-kuttl-tests/swift-proxy-dd4cf57c8-x4c45" Dec 03 12:43:36 crc kubenswrapper[5052]: I1203 12:43:36.415956 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/242be027-6461-4aea-ae7c-e52ca996b2ea-combined-ca-bundle\") pod \"swift-proxy-dd4cf57c8-x4c45\" (UID: \"242be027-6461-4aea-ae7c-e52ca996b2ea\") " pod="openstack-kuttl-tests/swift-proxy-dd4cf57c8-x4c45" Dec 03 12:43:36 crc kubenswrapper[5052]: I1203 12:43:36.416473 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/242be027-6461-4aea-ae7c-e52ca996b2ea-public-tls-certs\") pod \"swift-proxy-dd4cf57c8-x4c45\" (UID: \"242be027-6461-4aea-ae7c-e52ca996b2ea\") " pod="openstack-kuttl-tests/swift-proxy-dd4cf57c8-x4c45" Dec 03 12:43:36 crc kubenswrapper[5052]: I1203 12:43:36.418818 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/242be027-6461-4aea-ae7c-e52ca996b2ea-config-data\") pod \"swift-proxy-dd4cf57c8-x4c45\" (UID: \"242be027-6461-4aea-ae7c-e52ca996b2ea\") " pod="openstack-kuttl-tests/swift-proxy-dd4cf57c8-x4c45" Dec 03 12:43:36 crc kubenswrapper[5052]: I1203 12:43:36.418981 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/242be027-6461-4aea-ae7c-e52ca996b2ea-internal-tls-certs\") pod \"swift-proxy-dd4cf57c8-x4c45\" (UID: \"242be027-6461-4aea-ae7c-e52ca996b2ea\") " pod="openstack-kuttl-tests/swift-proxy-dd4cf57c8-x4c45" Dec 03 12:43:36 crc kubenswrapper[5052]: I1203 12:43:36.420414 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/242be027-6461-4aea-ae7c-e52ca996b2ea-etc-swift\") pod \"swift-proxy-dd4cf57c8-x4c45\" (UID: \"242be027-6461-4aea-ae7c-e52ca996b2ea\") " pod="openstack-kuttl-tests/swift-proxy-dd4cf57c8-x4c45" Dec 03 12:43:36 crc kubenswrapper[5052]: I1203 12:43:36.430737 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"50a91cd1-d2b0-48ad-954c-17c18007e891","Type":"ContainerDied","Data":"e30de4f6e400f75c400aa47f753b0a638068b280cd5702d95522c75bf45449ae"} Dec 03 12:43:36 crc kubenswrapper[5052]: I1203 12:43:36.430818 5052 scope.go:117] "RemoveContainer" containerID="e880a804aafeeeb88e41aefe1189349e376a91868c5113e6db383cbf79761799" Dec 03 12:43:36 crc kubenswrapper[5052]: I1203 12:43:36.430825 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:43:36 crc kubenswrapper[5052]: I1203 12:43:36.434003 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-gp8vs\" (UniqueName: \"kubernetes.io/projected/242be027-6461-4aea-ae7c-e52ca996b2ea-kube-api-access-gp8vs\") pod \"swift-proxy-dd4cf57c8-x4c45\" (UID: \"242be027-6461-4aea-ae7c-e52ca996b2ea\") " pod="openstack-kuttl-tests/swift-proxy-dd4cf57c8-x4c45" Dec 03 12:43:36 crc kubenswrapper[5052]: I1203 12:43:36.440047 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-98x49\" (UniqueName: \"kubernetes.io/projected/3f17a210-d54b-4753-91c1-2d9b97565b4e-kube-api-access-98x49\") pod \"community-operators-gt2q8\" (UID: \"3f17a210-d54b-4753-91c1-2d9b97565b4e\") " pod="openshift-marketplace/community-operators-gt2q8" Dec 03 12:43:36 crc kubenswrapper[5052]: I1203 12:43:36.454258 5052 scope.go:117] "RemoveContainer" containerID="fd43670029bd26793307cd8493e9456ce76cd5ee5b0b7b75dabac34375b7b698" Dec 03 12:43:36 crc kubenswrapper[5052]: I1203 12:43:36.475404 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:43:36 crc kubenswrapper[5052]: I1203 12:43:36.479580 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-proxy-dd4cf57c8-x4c45" Dec 03 12:43:36 crc kubenswrapper[5052]: I1203 12:43:36.488752 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:43:36 crc kubenswrapper[5052]: I1203 12:43:36.488993 5052 scope.go:117] "RemoveContainer" containerID="656e397b6d9cc32a2edd6ab337d8cc3f86f9dbda299c0fa90721c51fd6111e14" Dec 03 12:43:36 crc kubenswrapper[5052]: I1203 12:43:36.504038 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:43:36 crc kubenswrapper[5052]: I1203 12:43:36.506377 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:43:36 crc kubenswrapper[5052]: I1203 12:43:36.510579 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-scripts" Dec 03 12:43:36 crc kubenswrapper[5052]: I1203 12:43:36.510828 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-config-data" Dec 03 12:43:36 crc kubenswrapper[5052]: I1203 12:43:36.514412 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:43:36 crc kubenswrapper[5052]: I1203 12:43:36.533777 5052 scope.go:117] "RemoveContainer" containerID="faac469fb234838cd16f1eedf3600b31b890b6868f17653562825912fa983149" Dec 03 12:43:36 crc kubenswrapper[5052]: I1203 12:43:36.575977 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/community-operators-gt2q8" Dec 03 12:43:36 crc kubenswrapper[5052]: I1203 12:43:36.615457 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mxg6m\" (UniqueName: \"kubernetes.io/projected/fa98aa64-e8ed-442e-90a0-6823aaa88edb-kube-api-access-mxg6m\") pod \"ceilometer-0\" (UID: \"fa98aa64-e8ed-442e-90a0-6823aaa88edb\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:43:36 crc kubenswrapper[5052]: I1203 12:43:36.618194 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/fa98aa64-e8ed-442e-90a0-6823aaa88edb-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"fa98aa64-e8ed-442e-90a0-6823aaa88edb\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:43:36 crc kubenswrapper[5052]: I1203 12:43:36.618420 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/fa98aa64-e8ed-442e-90a0-6823aaa88edb-run-httpd\") pod \"ceilometer-0\" (UID: \"fa98aa64-e8ed-442e-90a0-6823aaa88edb\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:43:36 crc kubenswrapper[5052]: I1203 12:43:36.618463 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/fa98aa64-e8ed-442e-90a0-6823aaa88edb-log-httpd\") pod \"ceilometer-0\" (UID: \"fa98aa64-e8ed-442e-90a0-6823aaa88edb\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:43:36 crc kubenswrapper[5052]: I1203 12:43:36.618549 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/fa98aa64-e8ed-442e-90a0-6823aaa88edb-config-data\") pod \"ceilometer-0\" (UID: \"fa98aa64-e8ed-442e-90a0-6823aaa88edb\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:43:36 crc kubenswrapper[5052]: I1203 12:43:36.618581 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/fa98aa64-e8ed-442e-90a0-6823aaa88edb-scripts\") pod \"ceilometer-0\" (UID: \"fa98aa64-e8ed-442e-90a0-6823aaa88edb\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:43:36 crc kubenswrapper[5052]: I1203 12:43:36.618620 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/fa98aa64-e8ed-442e-90a0-6823aaa88edb-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"fa98aa64-e8ed-442e-90a0-6823aaa88edb\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:43:36 crc kubenswrapper[5052]: I1203 12:43:36.722736 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/fa98aa64-e8ed-442e-90a0-6823aaa88edb-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"fa98aa64-e8ed-442e-90a0-6823aaa88edb\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:43:36 crc kubenswrapper[5052]: I1203 12:43:36.723095 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/fa98aa64-e8ed-442e-90a0-6823aaa88edb-run-httpd\") pod \"ceilometer-0\" (UID: \"fa98aa64-e8ed-442e-90a0-6823aaa88edb\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:43:36 crc kubenswrapper[5052]: I1203 12:43:36.723123 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/fa98aa64-e8ed-442e-90a0-6823aaa88edb-log-httpd\") pod \"ceilometer-0\" (UID: \"fa98aa64-e8ed-442e-90a0-6823aaa88edb\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:43:36 crc kubenswrapper[5052]: I1203 12:43:36.723149 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/fa98aa64-e8ed-442e-90a0-6823aaa88edb-config-data\") pod \"ceilometer-0\" (UID: \"fa98aa64-e8ed-442e-90a0-6823aaa88edb\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:43:36 crc kubenswrapper[5052]: I1203 12:43:36.723169 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/fa98aa64-e8ed-442e-90a0-6823aaa88edb-scripts\") pod \"ceilometer-0\" (UID: \"fa98aa64-e8ed-442e-90a0-6823aaa88edb\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:43:36 crc kubenswrapper[5052]: I1203 12:43:36.723189 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/fa98aa64-e8ed-442e-90a0-6823aaa88edb-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"fa98aa64-e8ed-442e-90a0-6823aaa88edb\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:43:36 crc kubenswrapper[5052]: I1203 12:43:36.723284 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-mxg6m\" (UniqueName: \"kubernetes.io/projected/fa98aa64-e8ed-442e-90a0-6823aaa88edb-kube-api-access-mxg6m\") pod \"ceilometer-0\" (UID: \"fa98aa64-e8ed-442e-90a0-6823aaa88edb\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:43:36 crc kubenswrapper[5052]: I1203 12:43:36.724212 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/fa98aa64-e8ed-442e-90a0-6823aaa88edb-log-httpd\") pod \"ceilometer-0\" (UID: \"fa98aa64-e8ed-442e-90a0-6823aaa88edb\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:43:36 crc kubenswrapper[5052]: I1203 12:43:36.726114 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/fa98aa64-e8ed-442e-90a0-6823aaa88edb-run-httpd\") pod \"ceilometer-0\" (UID: \"fa98aa64-e8ed-442e-90a0-6823aaa88edb\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:43:36 crc kubenswrapper[5052]: I1203 12:43:36.751208 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/fa98aa64-e8ed-442e-90a0-6823aaa88edb-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"fa98aa64-e8ed-442e-90a0-6823aaa88edb\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:43:36 crc kubenswrapper[5052]: I1203 12:43:36.752348 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/fa98aa64-e8ed-442e-90a0-6823aaa88edb-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"fa98aa64-e8ed-442e-90a0-6823aaa88edb\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:43:36 crc kubenswrapper[5052]: I1203 12:43:36.757360 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/fa98aa64-e8ed-442e-90a0-6823aaa88edb-scripts\") pod \"ceilometer-0\" (UID: \"fa98aa64-e8ed-442e-90a0-6823aaa88edb\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:43:36 crc kubenswrapper[5052]: I1203 12:43:36.760324 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/fa98aa64-e8ed-442e-90a0-6823aaa88edb-config-data\") pod \"ceilometer-0\" (UID: \"fa98aa64-e8ed-442e-90a0-6823aaa88edb\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:43:36 crc kubenswrapper[5052]: I1203 12:43:36.776589 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-mxg6m\" (UniqueName: \"kubernetes.io/projected/fa98aa64-e8ed-442e-90a0-6823aaa88edb-kube-api-access-mxg6m\") pod \"ceilometer-0\" (UID: \"fa98aa64-e8ed-442e-90a0-6823aaa88edb\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:43:36 crc kubenswrapper[5052]: I1203 12:43:36.885760 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/swift-proxy-dd4cf57c8-x4c45"] Dec 03 12:43:37 crc kubenswrapper[5052]: I1203 12:43:37.050652 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:43:37 crc kubenswrapper[5052]: I1203 12:43:37.348338 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/community-operators-gt2q8"] Dec 03 12:43:37 crc kubenswrapper[5052]: W1203 12:43:37.351102 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f17a210_d54b_4753_91c1_2d9b97565b4e.slice/crio-761a82ce4ed15956c0cb9b9d1cff360c5b1ad3d4ec66d725b90bc7d0b4c95d41 WatchSource:0}: Error finding container 761a82ce4ed15956c0cb9b9d1cff360c5b1ad3d4ec66d725b90bc7d0b4c95d41: Status 404 returned error can't find the container with id 761a82ce4ed15956c0cb9b9d1cff360c5b1ad3d4ec66d725b90bc7d0b4c95d41 Dec 03 12:43:37 crc kubenswrapper[5052]: I1203 12:43:37.461741 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/community-operators-gt2q8" event={"ID":"3f17a210-d54b-4753-91c1-2d9b97565b4e","Type":"ContainerStarted","Data":"761a82ce4ed15956c0cb9b9d1cff360c5b1ad3d4ec66d725b90bc7d0b4c95d41"} Dec 03 12:43:37 crc kubenswrapper[5052]: I1203 12:43:37.464279 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-proxy-dd4cf57c8-x4c45" event={"ID":"242be027-6461-4aea-ae7c-e52ca996b2ea","Type":"ContainerStarted","Data":"b0a1441f36be1742bcb32bdac564816d92f0341aece1f5d7b3bd6920cff1543f"} Dec 03 12:43:37 crc kubenswrapper[5052]: I1203 12:43:37.464325 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-proxy-dd4cf57c8-x4c45" event={"ID":"242be027-6461-4aea-ae7c-e52ca996b2ea","Type":"ContainerStarted","Data":"25209c72330b0e84607f594e854354f3b7d2de1eddba6ea5df0a4234371e4441"} Dec 03 12:43:37 crc kubenswrapper[5052]: W1203 12:43:37.616561 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfa98aa64_e8ed_442e_90a0_6823aaa88edb.slice/crio-d600424a7463762da5d2af4d43d66a46f63c6fdb6bce76c5527e43e1a9bf2f01 WatchSource:0}: Error finding container d600424a7463762da5d2af4d43d66a46f63c6fdb6bce76c5527e43e1a9bf2f01: Status 404 returned error can't find the container with id d600424a7463762da5d2af4d43d66a46f63c6fdb6bce76c5527e43e1a9bf2f01 Dec 03 12:43:37 crc kubenswrapper[5052]: I1203 12:43:37.626381 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:43:37 crc kubenswrapper[5052]: I1203 12:43:37.684672 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="50a91cd1-d2b0-48ad-954c-17c18007e891" path="/var/lib/kubelet/pods/50a91cd1-d2b0-48ad-954c-17c18007e891/volumes" Dec 03 12:43:37 crc kubenswrapper[5052]: I1203 12:43:37.778930 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-api-db-create-dx5bv"] Dec 03 12:43:37 crc kubenswrapper[5052]: I1203 12:43:37.780435 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-db-create-dx5bv" Dec 03 12:43:37 crc kubenswrapper[5052]: I1203 12:43:37.801131 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-db-create-dx5bv"] Dec 03 12:43:37 crc kubenswrapper[5052]: I1203 12:43:37.849663 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gvnxv\" (UniqueName: \"kubernetes.io/projected/b99b3896-035c-4491-8093-80f669e31f64-kube-api-access-gvnxv\") pod \"nova-api-db-create-dx5bv\" (UID: \"b99b3896-035c-4491-8093-80f669e31f64\") " pod="openstack-kuttl-tests/nova-api-db-create-dx5bv" Dec 03 12:43:37 crc kubenswrapper[5052]: I1203 12:43:37.850703 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/b99b3896-035c-4491-8093-80f669e31f64-operator-scripts\") pod \"nova-api-db-create-dx5bv\" (UID: \"b99b3896-035c-4491-8093-80f669e31f64\") " pod="openstack-kuttl-tests/nova-api-db-create-dx5bv" Dec 03 12:43:37 crc kubenswrapper[5052]: I1203 12:43:37.872814 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell0-db-create-ztpwm"] Dec 03 12:43:37 crc kubenswrapper[5052]: I1203 12:43:37.874430 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-db-create-ztpwm" Dec 03 12:43:37 crc kubenswrapper[5052]: I1203 12:43:37.902253 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell0-db-create-ztpwm"] Dec 03 12:43:37 crc kubenswrapper[5052]: I1203 12:43:37.915746 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-api-fd1c-account-create-update-vxzns"] Dec 03 12:43:37 crc kubenswrapper[5052]: I1203 12:43:37.917110 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-fd1c-account-create-update-vxzns" Dec 03 12:43:37 crc kubenswrapper[5052]: I1203 12:43:37.919200 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-api-db-secret" Dec 03 12:43:37 crc kubenswrapper[5052]: I1203 12:43:37.931762 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-fd1c-account-create-update-vxzns"] Dec 03 12:43:37 crc kubenswrapper[5052]: I1203 12:43:37.953361 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/b4e4f28c-dc12-4844-a34a-14241afb73db-operator-scripts\") pod \"nova-cell0-db-create-ztpwm\" (UID: \"b4e4f28c-dc12-4844-a34a-14241afb73db\") " pod="openstack-kuttl-tests/nova-cell0-db-create-ztpwm" Dec 03 12:43:37 crc kubenswrapper[5052]: I1203 12:43:37.953760 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/b99b3896-035c-4491-8093-80f669e31f64-operator-scripts\") pod \"nova-api-db-create-dx5bv\" (UID: \"b99b3896-035c-4491-8093-80f669e31f64\") " pod="openstack-kuttl-tests/nova-api-db-create-dx5bv" Dec 03 12:43:37 crc kubenswrapper[5052]: I1203 12:43:37.954008 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tkkkr\" (UniqueName: \"kubernetes.io/projected/b4e4f28c-dc12-4844-a34a-14241afb73db-kube-api-access-tkkkr\") pod \"nova-cell0-db-create-ztpwm\" (UID: \"b4e4f28c-dc12-4844-a34a-14241afb73db\") " pod="openstack-kuttl-tests/nova-cell0-db-create-ztpwm" Dec 03 12:43:37 crc kubenswrapper[5052]: I1203 12:43:37.954129 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-gvnxv\" (UniqueName: \"kubernetes.io/projected/b99b3896-035c-4491-8093-80f669e31f64-kube-api-access-gvnxv\") pod \"nova-api-db-create-dx5bv\" (UID: \"b99b3896-035c-4491-8093-80f669e31f64\") " pod="openstack-kuttl-tests/nova-api-db-create-dx5bv" Dec 03 12:43:37 crc kubenswrapper[5052]: I1203 12:43:37.954636 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/b99b3896-035c-4491-8093-80f669e31f64-operator-scripts\") pod \"nova-api-db-create-dx5bv\" (UID: \"b99b3896-035c-4491-8093-80f669e31f64\") " pod="openstack-kuttl-tests/nova-api-db-create-dx5bv" Dec 03 12:43:37 crc kubenswrapper[5052]: E1203 12:43:37.971317 5052 cadvisor_stats_provider.go:516] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f17a210_d54b_4753_91c1_2d9b97565b4e.slice/crio-conmon-2d5d078d4787f4a22f1ee4624f7bb3f93992795d0ecf33df45f1d2d6041f5a7d.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f17a210_d54b_4753_91c1_2d9b97565b4e.slice/crio-2d5d078d4787f4a22f1ee4624f7bb3f93992795d0ecf33df45f1d2d6041f5a7d.scope\": RecentStats: unable to find data in memory cache]" Dec 03 12:43:37 crc kubenswrapper[5052]: I1203 12:43:37.976217 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-gvnxv\" (UniqueName: \"kubernetes.io/projected/b99b3896-035c-4491-8093-80f669e31f64-kube-api-access-gvnxv\") pod \"nova-api-db-create-dx5bv\" (UID: \"b99b3896-035c-4491-8093-80f669e31f64\") " pod="openstack-kuttl-tests/nova-api-db-create-dx5bv" Dec 03 12:43:38 crc kubenswrapper[5052]: I1203 12:43:38.061092 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-tkkkr\" (UniqueName: \"kubernetes.io/projected/b4e4f28c-dc12-4844-a34a-14241afb73db-kube-api-access-tkkkr\") pod \"nova-cell0-db-create-ztpwm\" (UID: \"b4e4f28c-dc12-4844-a34a-14241afb73db\") " pod="openstack-kuttl-tests/nova-cell0-db-create-ztpwm" Dec 03 12:43:38 crc kubenswrapper[5052]: I1203 12:43:38.061161 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/795fe0cc-73d2-4647-ad2a-648f57e74fc3-operator-scripts\") pod \"nova-api-fd1c-account-create-update-vxzns\" (UID: \"795fe0cc-73d2-4647-ad2a-648f57e74fc3\") " pod="openstack-kuttl-tests/nova-api-fd1c-account-create-update-vxzns" Dec 03 12:43:38 crc kubenswrapper[5052]: I1203 12:43:38.061220 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/b4e4f28c-dc12-4844-a34a-14241afb73db-operator-scripts\") pod \"nova-cell0-db-create-ztpwm\" (UID: \"b4e4f28c-dc12-4844-a34a-14241afb73db\") " pod="openstack-kuttl-tests/nova-cell0-db-create-ztpwm" Dec 03 12:43:38 crc kubenswrapper[5052]: I1203 12:43:38.061258 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l622h\" (UniqueName: \"kubernetes.io/projected/795fe0cc-73d2-4647-ad2a-648f57e74fc3-kube-api-access-l622h\") pod \"nova-api-fd1c-account-create-update-vxzns\" (UID: \"795fe0cc-73d2-4647-ad2a-648f57e74fc3\") " pod="openstack-kuttl-tests/nova-api-fd1c-account-create-update-vxzns" Dec 03 12:43:38 crc kubenswrapper[5052]: I1203 12:43:38.062026 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/b4e4f28c-dc12-4844-a34a-14241afb73db-operator-scripts\") pod \"nova-cell0-db-create-ztpwm\" (UID: \"b4e4f28c-dc12-4844-a34a-14241afb73db\") " pod="openstack-kuttl-tests/nova-cell0-db-create-ztpwm" Dec 03 12:43:38 crc kubenswrapper[5052]: I1203 12:43:38.114925 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-tkkkr\" (UniqueName: \"kubernetes.io/projected/b4e4f28c-dc12-4844-a34a-14241afb73db-kube-api-access-tkkkr\") pod \"nova-cell0-db-create-ztpwm\" (UID: \"b4e4f28c-dc12-4844-a34a-14241afb73db\") " pod="openstack-kuttl-tests/nova-cell0-db-create-ztpwm" Dec 03 12:43:38 crc kubenswrapper[5052]: I1203 12:43:38.120085 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell1-db-create-tkzwz"] Dec 03 12:43:38 crc kubenswrapper[5052]: I1203 12:43:38.123997 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-db-create-tkzwz" Dec 03 12:43:38 crc kubenswrapper[5052]: I1203 12:43:38.125164 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-db-create-dx5bv" Dec 03 12:43:38 crc kubenswrapper[5052]: I1203 12:43:38.161704 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell0-5566-account-create-update-wh2vb"] Dec 03 12:43:38 crc kubenswrapper[5052]: I1203 12:43:38.162954 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-5566-account-create-update-wh2vb" Dec 03 12:43:38 crc kubenswrapper[5052]: I1203 12:43:38.164081 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/795fe0cc-73d2-4647-ad2a-648f57e74fc3-operator-scripts\") pod \"nova-api-fd1c-account-create-update-vxzns\" (UID: \"795fe0cc-73d2-4647-ad2a-648f57e74fc3\") " pod="openstack-kuttl-tests/nova-api-fd1c-account-create-update-vxzns" Dec 03 12:43:38 crc kubenswrapper[5052]: I1203 12:43:38.163295 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/795fe0cc-73d2-4647-ad2a-648f57e74fc3-operator-scripts\") pod \"nova-api-fd1c-account-create-update-vxzns\" (UID: \"795fe0cc-73d2-4647-ad2a-648f57e74fc3\") " pod="openstack-kuttl-tests/nova-api-fd1c-account-create-update-vxzns" Dec 03 12:43:38 crc kubenswrapper[5052]: I1203 12:43:38.168711 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-l622h\" (UniqueName: \"kubernetes.io/projected/795fe0cc-73d2-4647-ad2a-648f57e74fc3-kube-api-access-l622h\") pod \"nova-api-fd1c-account-create-update-vxzns\" (UID: \"795fe0cc-73d2-4647-ad2a-648f57e74fc3\") " pod="openstack-kuttl-tests/nova-api-fd1c-account-create-update-vxzns" Dec 03 12:43:38 crc kubenswrapper[5052]: I1203 12:43:38.168940 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell0-db-secret" Dec 03 12:43:38 crc kubenswrapper[5052]: I1203 12:43:38.183373 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-db-create-tkzwz"] Dec 03 12:43:38 crc kubenswrapper[5052]: I1203 12:43:38.190094 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-l622h\" (UniqueName: \"kubernetes.io/projected/795fe0cc-73d2-4647-ad2a-648f57e74fc3-kube-api-access-l622h\") pod \"nova-api-fd1c-account-create-update-vxzns\" (UID: \"795fe0cc-73d2-4647-ad2a-648f57e74fc3\") " pod="openstack-kuttl-tests/nova-api-fd1c-account-create-update-vxzns" Dec 03 12:43:38 crc kubenswrapper[5052]: I1203 12:43:38.193577 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell0-5566-account-create-update-wh2vb"] Dec 03 12:43:38 crc kubenswrapper[5052]: I1203 12:43:38.202965 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-db-create-ztpwm" Dec 03 12:43:38 crc kubenswrapper[5052]: I1203 12:43:38.264018 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-fd1c-account-create-update-vxzns" Dec 03 12:43:38 crc kubenswrapper[5052]: I1203 12:43:38.273140 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-889zs\" (UniqueName: \"kubernetes.io/projected/6e49273b-7e4d-4818-8f55-2db55a1250b9-kube-api-access-889zs\") pod \"nova-cell1-db-create-tkzwz\" (UID: \"6e49273b-7e4d-4818-8f55-2db55a1250b9\") " pod="openstack-kuttl-tests/nova-cell1-db-create-tkzwz" Dec 03 12:43:38 crc kubenswrapper[5052]: I1203 12:43:38.273223 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/d857bbf5-cd76-4d28-863c-1507de5c6cfd-operator-scripts\") pod \"nova-cell0-5566-account-create-update-wh2vb\" (UID: \"d857bbf5-cd76-4d28-863c-1507de5c6cfd\") " pod="openstack-kuttl-tests/nova-cell0-5566-account-create-update-wh2vb" Dec 03 12:43:38 crc kubenswrapper[5052]: I1203 12:43:38.273300 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/6e49273b-7e4d-4818-8f55-2db55a1250b9-operator-scripts\") pod \"nova-cell1-db-create-tkzwz\" (UID: \"6e49273b-7e4d-4818-8f55-2db55a1250b9\") " pod="openstack-kuttl-tests/nova-cell1-db-create-tkzwz" Dec 03 12:43:38 crc kubenswrapper[5052]: I1203 12:43:38.273373 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tthv4\" (UniqueName: \"kubernetes.io/projected/d857bbf5-cd76-4d28-863c-1507de5c6cfd-kube-api-access-tthv4\") pod \"nova-cell0-5566-account-create-update-wh2vb\" (UID: \"d857bbf5-cd76-4d28-863c-1507de5c6cfd\") " pod="openstack-kuttl-tests/nova-cell0-5566-account-create-update-wh2vb" Dec 03 12:43:38 crc kubenswrapper[5052]: I1203 12:43:38.303394 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell1-4975-account-create-update-77cc4"] Dec 03 12:43:38 crc kubenswrapper[5052]: I1203 12:43:38.304704 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-4975-account-create-update-77cc4" Dec 03 12:43:38 crc kubenswrapper[5052]: I1203 12:43:38.308782 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell1-db-secret" Dec 03 12:43:38 crc kubenswrapper[5052]: I1203 12:43:38.326566 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-4975-account-create-update-77cc4"] Dec 03 12:43:38 crc kubenswrapper[5052]: I1203 12:43:38.392982 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-889zs\" (UniqueName: \"kubernetes.io/projected/6e49273b-7e4d-4818-8f55-2db55a1250b9-kube-api-access-889zs\") pod \"nova-cell1-db-create-tkzwz\" (UID: \"6e49273b-7e4d-4818-8f55-2db55a1250b9\") " pod="openstack-kuttl-tests/nova-cell1-db-create-tkzwz" Dec 03 12:43:38 crc kubenswrapper[5052]: I1203 12:43:38.393055 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/d857bbf5-cd76-4d28-863c-1507de5c6cfd-operator-scripts\") pod \"nova-cell0-5566-account-create-update-wh2vb\" (UID: \"d857bbf5-cd76-4d28-863c-1507de5c6cfd\") " pod="openstack-kuttl-tests/nova-cell0-5566-account-create-update-wh2vb" Dec 03 12:43:38 crc kubenswrapper[5052]: I1203 12:43:38.393117 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/6e49273b-7e4d-4818-8f55-2db55a1250b9-operator-scripts\") pod \"nova-cell1-db-create-tkzwz\" (UID: \"6e49273b-7e4d-4818-8f55-2db55a1250b9\") " pod="openstack-kuttl-tests/nova-cell1-db-create-tkzwz" Dec 03 12:43:38 crc kubenswrapper[5052]: I1203 12:43:38.393167 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-tthv4\" (UniqueName: \"kubernetes.io/projected/d857bbf5-cd76-4d28-863c-1507de5c6cfd-kube-api-access-tthv4\") pod \"nova-cell0-5566-account-create-update-wh2vb\" (UID: \"d857bbf5-cd76-4d28-863c-1507de5c6cfd\") " pod="openstack-kuttl-tests/nova-cell0-5566-account-create-update-wh2vb" Dec 03 12:43:38 crc kubenswrapper[5052]: I1203 12:43:38.393222 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/7e89765b-8d00-48dc-9f2a-d7b6d7d7b5f0-operator-scripts\") pod \"nova-cell1-4975-account-create-update-77cc4\" (UID: \"7e89765b-8d00-48dc-9f2a-d7b6d7d7b5f0\") " pod="openstack-kuttl-tests/nova-cell1-4975-account-create-update-77cc4" Dec 03 12:43:38 crc kubenswrapper[5052]: I1203 12:43:38.393254 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c6q6w\" (UniqueName: \"kubernetes.io/projected/7e89765b-8d00-48dc-9f2a-d7b6d7d7b5f0-kube-api-access-c6q6w\") pod \"nova-cell1-4975-account-create-update-77cc4\" (UID: \"7e89765b-8d00-48dc-9f2a-d7b6d7d7b5f0\") " pod="openstack-kuttl-tests/nova-cell1-4975-account-create-update-77cc4" Dec 03 12:43:38 crc kubenswrapper[5052]: I1203 12:43:38.394262 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/6e49273b-7e4d-4818-8f55-2db55a1250b9-operator-scripts\") pod \"nova-cell1-db-create-tkzwz\" (UID: \"6e49273b-7e4d-4818-8f55-2db55a1250b9\") " pod="openstack-kuttl-tests/nova-cell1-db-create-tkzwz" Dec 03 12:43:38 crc kubenswrapper[5052]: I1203 12:43:38.403028 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/d857bbf5-cd76-4d28-863c-1507de5c6cfd-operator-scripts\") pod \"nova-cell0-5566-account-create-update-wh2vb\" (UID: \"d857bbf5-cd76-4d28-863c-1507de5c6cfd\") " pod="openstack-kuttl-tests/nova-cell0-5566-account-create-update-wh2vb" Dec 03 12:43:38 crc kubenswrapper[5052]: I1203 12:43:38.414842 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-tthv4\" (UniqueName: \"kubernetes.io/projected/d857bbf5-cd76-4d28-863c-1507de5c6cfd-kube-api-access-tthv4\") pod \"nova-cell0-5566-account-create-update-wh2vb\" (UID: \"d857bbf5-cd76-4d28-863c-1507de5c6cfd\") " pod="openstack-kuttl-tests/nova-cell0-5566-account-create-update-wh2vb" Dec 03 12:43:38 crc kubenswrapper[5052]: I1203 12:43:38.414916 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-889zs\" (UniqueName: \"kubernetes.io/projected/6e49273b-7e4d-4818-8f55-2db55a1250b9-kube-api-access-889zs\") pod \"nova-cell1-db-create-tkzwz\" (UID: \"6e49273b-7e4d-4818-8f55-2db55a1250b9\") " pod="openstack-kuttl-tests/nova-cell1-db-create-tkzwz" Dec 03 12:43:38 crc kubenswrapper[5052]: I1203 12:43:38.489024 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-proxy-dd4cf57c8-x4c45" event={"ID":"242be027-6461-4aea-ae7c-e52ca996b2ea","Type":"ContainerStarted","Data":"29b8aa0fef682b25addf6b174c213d6db9760bd388dc211dfc025f115b0ab6ed"} Dec 03 12:43:38 crc kubenswrapper[5052]: I1203 12:43:38.490170 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/swift-proxy-dd4cf57c8-x4c45" Dec 03 12:43:38 crc kubenswrapper[5052]: I1203 12:43:38.490195 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/swift-proxy-dd4cf57c8-x4c45" Dec 03 12:43:38 crc kubenswrapper[5052]: I1203 12:43:38.494302 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/7e89765b-8d00-48dc-9f2a-d7b6d7d7b5f0-operator-scripts\") pod \"nova-cell1-4975-account-create-update-77cc4\" (UID: \"7e89765b-8d00-48dc-9f2a-d7b6d7d7b5f0\") " pod="openstack-kuttl-tests/nova-cell1-4975-account-create-update-77cc4" Dec 03 12:43:38 crc kubenswrapper[5052]: I1203 12:43:38.494353 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-c6q6w\" (UniqueName: \"kubernetes.io/projected/7e89765b-8d00-48dc-9f2a-d7b6d7d7b5f0-kube-api-access-c6q6w\") pod \"nova-cell1-4975-account-create-update-77cc4\" (UID: \"7e89765b-8d00-48dc-9f2a-d7b6d7d7b5f0\") " pod="openstack-kuttl-tests/nova-cell1-4975-account-create-update-77cc4" Dec 03 12:43:38 crc kubenswrapper[5052]: I1203 12:43:38.495859 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/7e89765b-8d00-48dc-9f2a-d7b6d7d7b5f0-operator-scripts\") pod \"nova-cell1-4975-account-create-update-77cc4\" (UID: \"7e89765b-8d00-48dc-9f2a-d7b6d7d7b5f0\") " pod="openstack-kuttl-tests/nova-cell1-4975-account-create-update-77cc4" Dec 03 12:43:38 crc kubenswrapper[5052]: I1203 12:43:38.497902 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"fa98aa64-e8ed-442e-90a0-6823aaa88edb","Type":"ContainerStarted","Data":"d600424a7463762da5d2af4d43d66a46f63c6fdb6bce76c5527e43e1a9bf2f01"} Dec 03 12:43:38 crc kubenswrapper[5052]: I1203 12:43:38.501111 5052 generic.go:334] "Generic (PLEG): container finished" podID="3f17a210-d54b-4753-91c1-2d9b97565b4e" containerID="2d5d078d4787f4a22f1ee4624f7bb3f93992795d0ecf33df45f1d2d6041f5a7d" exitCode=0 Dec 03 12:43:38 crc kubenswrapper[5052]: I1203 12:43:38.501164 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/community-operators-gt2q8" event={"ID":"3f17a210-d54b-4753-91c1-2d9b97565b4e","Type":"ContainerDied","Data":"2d5d078d4787f4a22f1ee4624f7bb3f93992795d0ecf33df45f1d2d6041f5a7d"} Dec 03 12:43:38 crc kubenswrapper[5052]: I1203 12:43:38.517699 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-c6q6w\" (UniqueName: \"kubernetes.io/projected/7e89765b-8d00-48dc-9f2a-d7b6d7d7b5f0-kube-api-access-c6q6w\") pod \"nova-cell1-4975-account-create-update-77cc4\" (UID: \"7e89765b-8d00-48dc-9f2a-d7b6d7d7b5f0\") " pod="openstack-kuttl-tests/nova-cell1-4975-account-create-update-77cc4" Dec 03 12:43:38 crc kubenswrapper[5052]: I1203 12:43:38.532395 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-db-create-tkzwz" Dec 03 12:43:38 crc kubenswrapper[5052]: I1203 12:43:38.557817 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/swift-proxy-dd4cf57c8-x4c45" podStartSLOduration=2.557798613 podStartE2EDuration="2.557798613s" podCreationTimestamp="2025-12-03 12:43:36 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:43:38.524745497 +0000 UTC m=+5009.112987261" watchObservedRunningTime="2025-12-03 12:43:38.557798613 +0000 UTC m=+5009.146040377" Dec 03 12:43:38 crc kubenswrapper[5052]: I1203 12:43:38.575838 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-5566-account-create-update-wh2vb" Dec 03 12:43:38 crc kubenswrapper[5052]: I1203 12:43:38.627242 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-4975-account-create-update-77cc4" Dec 03 12:43:38 crc kubenswrapper[5052]: I1203 12:43:38.695627 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-db-create-dx5bv"] Dec 03 12:43:38 crc kubenswrapper[5052]: I1203 12:43:38.787076 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell0-db-create-ztpwm"] Dec 03 12:43:38 crc kubenswrapper[5052]: I1203 12:43:38.951803 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-fd1c-account-create-update-vxzns"] Dec 03 12:43:39 crc kubenswrapper[5052]: W1203 12:43:39.031668 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb99b3896_035c_4491_8093_80f669e31f64.slice/crio-e1fb09b8278d70bf5e696ccda2966e127747528fc4cbcabd258e63122bef45e3 WatchSource:0}: Error finding container e1fb09b8278d70bf5e696ccda2966e127747528fc4cbcabd258e63122bef45e3: Status 404 returned error can't find the container with id e1fb09b8278d70bf5e696ccda2966e127747528fc4cbcabd258e63122bef45e3 Dec 03 12:43:39 crc kubenswrapper[5052]: W1203 12:43:39.039310 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb4e4f28c_dc12_4844_a34a_14241afb73db.slice/crio-216d18fbf839f26fb17f5b534853900d0c6cc60fce25f14e16a03091fcbc4b63 WatchSource:0}: Error finding container 216d18fbf839f26fb17f5b534853900d0c6cc60fce25f14e16a03091fcbc4b63: Status 404 returned error can't find the container with id 216d18fbf839f26fb17f5b534853900d0c6cc60fce25f14e16a03091fcbc4b63 Dec 03 12:43:39 crc kubenswrapper[5052]: I1203 12:43:39.524520 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-db-create-dx5bv" event={"ID":"b99b3896-035c-4491-8093-80f669e31f64","Type":"ContainerStarted","Data":"e1fb09b8278d70bf5e696ccda2966e127747528fc4cbcabd258e63122bef45e3"} Dec 03 12:43:39 crc kubenswrapper[5052]: I1203 12:43:39.531738 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-db-create-ztpwm" event={"ID":"b4e4f28c-dc12-4844-a34a-14241afb73db","Type":"ContainerStarted","Data":"216d18fbf839f26fb17f5b534853900d0c6cc60fce25f14e16a03091fcbc4b63"} Dec 03 12:43:39 crc kubenswrapper[5052]: I1203 12:43:39.535291 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-fd1c-account-create-update-vxzns" event={"ID":"795fe0cc-73d2-4647-ad2a-648f57e74fc3","Type":"ContainerStarted","Data":"bbd583eb50d3dac6c9a9f9f4633fab53b7e4a1c4be7662a48caeb78c985b8b37"} Dec 03 12:43:39 crc kubenswrapper[5052]: I1203 12:43:39.796626 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell0-5566-account-create-update-wh2vb"] Dec 03 12:43:39 crc kubenswrapper[5052]: W1203 12:43:39.798885 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6e49273b_7e4d_4818_8f55_2db55a1250b9.slice/crio-3f56bb8a4ab6cac256920768504faa8651d931c90cc764ae9c80ab621bfad532 WatchSource:0}: Error finding container 3f56bb8a4ab6cac256920768504faa8651d931c90cc764ae9c80ab621bfad532: Status 404 returned error can't find the container with id 3f56bb8a4ab6cac256920768504faa8651d931c90cc764ae9c80ab621bfad532 Dec 03 12:43:39 crc kubenswrapper[5052]: I1203 12:43:39.826535 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-db-create-tkzwz"] Dec 03 12:43:39 crc kubenswrapper[5052]: I1203 12:43:39.867879 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-4975-account-create-update-77cc4"] Dec 03 12:43:40 crc kubenswrapper[5052]: I1203 12:43:40.549184 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-db-create-tkzwz" event={"ID":"6e49273b-7e4d-4818-8f55-2db55a1250b9","Type":"ContainerStarted","Data":"242b3296cda414e10917d8871e4c46cde20cc551a78ee9308847f958d3879db1"} Dec 03 12:43:40 crc kubenswrapper[5052]: I1203 12:43:40.549532 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-db-create-tkzwz" event={"ID":"6e49273b-7e4d-4818-8f55-2db55a1250b9","Type":"ContainerStarted","Data":"3f56bb8a4ab6cac256920768504faa8651d931c90cc764ae9c80ab621bfad532"} Dec 03 12:43:40 crc kubenswrapper[5052]: I1203 12:43:40.551400 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-5566-account-create-update-wh2vb" event={"ID":"d857bbf5-cd76-4d28-863c-1507de5c6cfd","Type":"ContainerStarted","Data":"abc3050de7da767805b268c2d3b5fa32accdabf063de3d1794a02867a3ae5a02"} Dec 03 12:43:40 crc kubenswrapper[5052]: I1203 12:43:40.551459 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-5566-account-create-update-wh2vb" event={"ID":"d857bbf5-cd76-4d28-863c-1507de5c6cfd","Type":"ContainerStarted","Data":"9e0bd52f613c4b9ca31d9340a69db4f6e1ce445db1e095a99a966db97fe951a0"} Dec 03 12:43:40 crc kubenswrapper[5052]: I1203 12:43:40.555217 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-4975-account-create-update-77cc4" event={"ID":"7e89765b-8d00-48dc-9f2a-d7b6d7d7b5f0","Type":"ContainerStarted","Data":"3ac4a2059b84064ce39da686d421b429257a9d750e421df9c7a06c8da2d6d181"} Dec 03 12:43:40 crc kubenswrapper[5052]: I1203 12:43:40.555271 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-4975-account-create-update-77cc4" event={"ID":"7e89765b-8d00-48dc-9f2a-d7b6d7d7b5f0","Type":"ContainerStarted","Data":"9e04c6807ba25e04f108eb49fbcd1e4e94c7afea0b64d139ede600b2bce0de31"} Dec 03 12:43:40 crc kubenswrapper[5052]: I1203 12:43:40.559385 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-db-create-dx5bv" event={"ID":"b99b3896-035c-4491-8093-80f669e31f64","Type":"ContainerStarted","Data":"588b5370b468d35103c4a171221a0e2f4784fb7d8d8468ab69a7e96a141588db"} Dec 03 12:43:40 crc kubenswrapper[5052]: I1203 12:43:40.561791 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-db-create-ztpwm" event={"ID":"b4e4f28c-dc12-4844-a34a-14241afb73db","Type":"ContainerStarted","Data":"945804751d5a870a849fdb2c2fced6c4170c19fc2099d177cfebd94e18f4ec26"} Dec 03 12:43:40 crc kubenswrapper[5052]: I1203 12:43:40.564716 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-fd1c-account-create-update-vxzns" event={"ID":"795fe0cc-73d2-4647-ad2a-648f57e74fc3","Type":"ContainerStarted","Data":"840307e68c72bbebca98f5317317ad1e1192eb2196b68e8a95a9c08f71d63e8b"} Dec 03 12:43:40 crc kubenswrapper[5052]: I1203 12:43:40.570224 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-cell1-db-create-tkzwz" podStartSLOduration=2.570206563 podStartE2EDuration="2.570206563s" podCreationTimestamp="2025-12-03 12:43:38 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:43:40.569175924 +0000 UTC m=+5011.157417708" watchObservedRunningTime="2025-12-03 12:43:40.570206563 +0000 UTC m=+5011.158448327" Dec 03 12:43:40 crc kubenswrapper[5052]: I1203 12:43:40.572327 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"fa98aa64-e8ed-442e-90a0-6823aaa88edb","Type":"ContainerStarted","Data":"c3413a78a793e89bd0b3f298b6e5e3d40cfcd493e84ae55bfbc2ea5d5d154265"} Dec 03 12:43:40 crc kubenswrapper[5052]: I1203 12:43:40.597442 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-api-db-create-dx5bv" podStartSLOduration=3.597423352 podStartE2EDuration="3.597423352s" podCreationTimestamp="2025-12-03 12:43:37 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:43:40.592538162 +0000 UTC m=+5011.180779926" watchObservedRunningTime="2025-12-03 12:43:40.597423352 +0000 UTC m=+5011.185665116" Dec 03 12:43:40 crc kubenswrapper[5052]: I1203 12:43:40.610723 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-cell1-4975-account-create-update-77cc4" podStartSLOduration=2.610703932 podStartE2EDuration="2.610703932s" podCreationTimestamp="2025-12-03 12:43:38 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:43:40.606039828 +0000 UTC m=+5011.194281612" watchObservedRunningTime="2025-12-03 12:43:40.610703932 +0000 UTC m=+5011.198945696" Dec 03 12:43:40 crc kubenswrapper[5052]: I1203 12:43:40.638250 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-cell0-db-create-ztpwm" podStartSLOduration=3.638226999 podStartE2EDuration="3.638226999s" podCreationTimestamp="2025-12-03 12:43:37 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:43:40.622405606 +0000 UTC m=+5011.210647390" watchObservedRunningTime="2025-12-03 12:43:40.638226999 +0000 UTC m=+5011.226468763" Dec 03 12:43:40 crc kubenswrapper[5052]: I1203 12:43:40.647936 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-cell0-5566-account-create-update-wh2vb" podStartSLOduration=2.647907576 podStartE2EDuration="2.647907576s" podCreationTimestamp="2025-12-03 12:43:38 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:43:40.636581982 +0000 UTC m=+5011.224823756" watchObservedRunningTime="2025-12-03 12:43:40.647907576 +0000 UTC m=+5011.236149340" Dec 03 12:43:40 crc kubenswrapper[5052]: I1203 12:43:40.655168 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-api-fd1c-account-create-update-vxzns" podStartSLOduration=3.655149553 podStartE2EDuration="3.655149553s" podCreationTimestamp="2025-12-03 12:43:37 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:43:40.652933069 +0000 UTC m=+5011.241174833" watchObservedRunningTime="2025-12-03 12:43:40.655149553 +0000 UTC m=+5011.243391317" Dec 03 12:43:41 crc kubenswrapper[5052]: I1203 12:43:41.589474 5052 generic.go:334] "Generic (PLEG): container finished" podID="b99b3896-035c-4491-8093-80f669e31f64" containerID="588b5370b468d35103c4a171221a0e2f4784fb7d8d8468ab69a7e96a141588db" exitCode=0 Dec 03 12:43:41 crc kubenswrapper[5052]: I1203 12:43:41.589558 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-db-create-dx5bv" event={"ID":"b99b3896-035c-4491-8093-80f669e31f64","Type":"ContainerDied","Data":"588b5370b468d35103c4a171221a0e2f4784fb7d8d8468ab69a7e96a141588db"} Dec 03 12:43:41 crc kubenswrapper[5052]: I1203 12:43:41.591787 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/community-operators-gt2q8" event={"ID":"3f17a210-d54b-4753-91c1-2d9b97565b4e","Type":"ContainerStarted","Data":"7a6e9317cc9c73ddf62181ea0be41ce3e631113e804f92ebef91af52e3c9a7be"} Dec 03 12:43:41 crc kubenswrapper[5052]: I1203 12:43:41.594139 5052 generic.go:334] "Generic (PLEG): container finished" podID="b4e4f28c-dc12-4844-a34a-14241afb73db" containerID="945804751d5a870a849fdb2c2fced6c4170c19fc2099d177cfebd94e18f4ec26" exitCode=0 Dec 03 12:43:41 crc kubenswrapper[5052]: I1203 12:43:41.594741 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-db-create-ztpwm" event={"ID":"b4e4f28c-dc12-4844-a34a-14241afb73db","Type":"ContainerDied","Data":"945804751d5a870a849fdb2c2fced6c4170c19fc2099d177cfebd94e18f4ec26"} Dec 03 12:43:41 crc kubenswrapper[5052]: I1203 12:43:41.673275 5052 scope.go:117] "RemoveContainer" containerID="2aed1c760a326f3d008799e902b1093313df3764a57c879f0b3686410127d37d" Dec 03 12:43:41 crc kubenswrapper[5052]: E1203 12:43:41.673559 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 12:43:42 crc kubenswrapper[5052]: I1203 12:43:42.609311 5052 generic.go:334] "Generic (PLEG): container finished" podID="795fe0cc-73d2-4647-ad2a-648f57e74fc3" containerID="840307e68c72bbebca98f5317317ad1e1192eb2196b68e8a95a9c08f71d63e8b" exitCode=0 Dec 03 12:43:42 crc kubenswrapper[5052]: I1203 12:43:42.609731 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-fd1c-account-create-update-vxzns" event={"ID":"795fe0cc-73d2-4647-ad2a-648f57e74fc3","Type":"ContainerDied","Data":"840307e68c72bbebca98f5317317ad1e1192eb2196b68e8a95a9c08f71d63e8b"} Dec 03 12:43:42 crc kubenswrapper[5052]: I1203 12:43:42.613851 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"fa98aa64-e8ed-442e-90a0-6823aaa88edb","Type":"ContainerStarted","Data":"2d10f32dbe1a375fa4b22fdd2415c34559e7c30be12f35f8ef9503c291f76702"} Dec 03 12:43:42 crc kubenswrapper[5052]: I1203 12:43:42.615874 5052 generic.go:334] "Generic (PLEG): container finished" podID="6e49273b-7e4d-4818-8f55-2db55a1250b9" containerID="242b3296cda414e10917d8871e4c46cde20cc551a78ee9308847f958d3879db1" exitCode=0 Dec 03 12:43:42 crc kubenswrapper[5052]: I1203 12:43:42.615943 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-db-create-tkzwz" event={"ID":"6e49273b-7e4d-4818-8f55-2db55a1250b9","Type":"ContainerDied","Data":"242b3296cda414e10917d8871e4c46cde20cc551a78ee9308847f958d3879db1"} Dec 03 12:43:42 crc kubenswrapper[5052]: I1203 12:43:42.618549 5052 generic.go:334] "Generic (PLEG): container finished" podID="d857bbf5-cd76-4d28-863c-1507de5c6cfd" containerID="abc3050de7da767805b268c2d3b5fa32accdabf063de3d1794a02867a3ae5a02" exitCode=0 Dec 03 12:43:42 crc kubenswrapper[5052]: I1203 12:43:42.618647 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-5566-account-create-update-wh2vb" event={"ID":"d857bbf5-cd76-4d28-863c-1507de5c6cfd","Type":"ContainerDied","Data":"abc3050de7da767805b268c2d3b5fa32accdabf063de3d1794a02867a3ae5a02"} Dec 03 12:43:42 crc kubenswrapper[5052]: I1203 12:43:42.621188 5052 generic.go:334] "Generic (PLEG): container finished" podID="7e89765b-8d00-48dc-9f2a-d7b6d7d7b5f0" containerID="3ac4a2059b84064ce39da686d421b429257a9d750e421df9c7a06c8da2d6d181" exitCode=0 Dec 03 12:43:42 crc kubenswrapper[5052]: I1203 12:43:42.621222 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-4975-account-create-update-77cc4" event={"ID":"7e89765b-8d00-48dc-9f2a-d7b6d7d7b5f0","Type":"ContainerDied","Data":"3ac4a2059b84064ce39da686d421b429257a9d750e421df9c7a06c8da2d6d181"} Dec 03 12:43:42 crc kubenswrapper[5052]: I1203 12:43:42.624246 5052 generic.go:334] "Generic (PLEG): container finished" podID="3f17a210-d54b-4753-91c1-2d9b97565b4e" containerID="7a6e9317cc9c73ddf62181ea0be41ce3e631113e804f92ebef91af52e3c9a7be" exitCode=0 Dec 03 12:43:42 crc kubenswrapper[5052]: I1203 12:43:42.625370 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/community-operators-gt2q8" event={"ID":"3f17a210-d54b-4753-91c1-2d9b97565b4e","Type":"ContainerDied","Data":"7a6e9317cc9c73ddf62181ea0be41ce3e631113e804f92ebef91af52e3c9a7be"} Dec 03 12:43:43 crc kubenswrapper[5052]: I1203 12:43:43.212046 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-db-create-dx5bv" Dec 03 12:43:43 crc kubenswrapper[5052]: I1203 12:43:43.240300 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-db-create-ztpwm" Dec 03 12:43:43 crc kubenswrapper[5052]: I1203 12:43:43.313073 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/b4e4f28c-dc12-4844-a34a-14241afb73db-operator-scripts\") pod \"b4e4f28c-dc12-4844-a34a-14241afb73db\" (UID: \"b4e4f28c-dc12-4844-a34a-14241afb73db\") " Dec 03 12:43:43 crc kubenswrapper[5052]: I1203 12:43:43.313183 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gvnxv\" (UniqueName: \"kubernetes.io/projected/b99b3896-035c-4491-8093-80f669e31f64-kube-api-access-gvnxv\") pod \"b99b3896-035c-4491-8093-80f669e31f64\" (UID: \"b99b3896-035c-4491-8093-80f669e31f64\") " Dec 03 12:43:43 crc kubenswrapper[5052]: I1203 12:43:43.313313 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tkkkr\" (UniqueName: \"kubernetes.io/projected/b4e4f28c-dc12-4844-a34a-14241afb73db-kube-api-access-tkkkr\") pod \"b4e4f28c-dc12-4844-a34a-14241afb73db\" (UID: \"b4e4f28c-dc12-4844-a34a-14241afb73db\") " Dec 03 12:43:43 crc kubenswrapper[5052]: I1203 12:43:43.313691 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b4e4f28c-dc12-4844-a34a-14241afb73db-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "b4e4f28c-dc12-4844-a34a-14241afb73db" (UID: "b4e4f28c-dc12-4844-a34a-14241afb73db"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:43:43 crc kubenswrapper[5052]: I1203 12:43:43.314237 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/b99b3896-035c-4491-8093-80f669e31f64-operator-scripts\") pod \"b99b3896-035c-4491-8093-80f669e31f64\" (UID: \"b99b3896-035c-4491-8093-80f669e31f64\") " Dec 03 12:43:43 crc kubenswrapper[5052]: I1203 12:43:43.314898 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b99b3896-035c-4491-8093-80f669e31f64-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "b99b3896-035c-4491-8093-80f669e31f64" (UID: "b99b3896-035c-4491-8093-80f669e31f64"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:43:43 crc kubenswrapper[5052]: I1203 12:43:43.315208 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/b99b3896-035c-4491-8093-80f669e31f64-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:43:43 crc kubenswrapper[5052]: I1203 12:43:43.315228 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/b4e4f28c-dc12-4844-a34a-14241afb73db-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:43:43 crc kubenswrapper[5052]: I1203 12:43:43.318702 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b99b3896-035c-4491-8093-80f669e31f64-kube-api-access-gvnxv" (OuterVolumeSpecName: "kube-api-access-gvnxv") pod "b99b3896-035c-4491-8093-80f669e31f64" (UID: "b99b3896-035c-4491-8093-80f669e31f64"). InnerVolumeSpecName "kube-api-access-gvnxv". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:43:43 crc kubenswrapper[5052]: I1203 12:43:43.318935 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b4e4f28c-dc12-4844-a34a-14241afb73db-kube-api-access-tkkkr" (OuterVolumeSpecName: "kube-api-access-tkkkr") pod "b4e4f28c-dc12-4844-a34a-14241afb73db" (UID: "b4e4f28c-dc12-4844-a34a-14241afb73db"). InnerVolumeSpecName "kube-api-access-tkkkr". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:43:43 crc kubenswrapper[5052]: I1203 12:43:43.417203 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-gvnxv\" (UniqueName: \"kubernetes.io/projected/b99b3896-035c-4491-8093-80f669e31f64-kube-api-access-gvnxv\") on node \"crc\" DevicePath \"\"" Dec 03 12:43:43 crc kubenswrapper[5052]: I1203 12:43:43.417238 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-tkkkr\" (UniqueName: \"kubernetes.io/projected/b4e4f28c-dc12-4844-a34a-14241afb73db-kube-api-access-tkkkr\") on node \"crc\" DevicePath \"\"" Dec 03 12:43:43 crc kubenswrapper[5052]: I1203 12:43:43.504876 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:43:43 crc kubenswrapper[5052]: I1203 12:43:43.586046 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:43:43 crc kubenswrapper[5052]: I1203 12:43:43.586387 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-internal-api-0" podUID="8011917c-0ce8-4c83-b157-2d8538790e17" containerName="glance-log" containerID="cri-o://c38db48afa4b344c32d00e4db4d452c8f75d832d0c682df5c1a48a83e977e312" gracePeriod=30 Dec 03 12:43:43 crc kubenswrapper[5052]: I1203 12:43:43.586552 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-internal-api-0" podUID="8011917c-0ce8-4c83-b157-2d8538790e17" containerName="glance-httpd" containerID="cri-o://d871ecf0facb775d7ef4d9dacb2a2746b353a7fee55e6f036959cc141b2fcf0e" gracePeriod=30 Dec 03 12:43:43 crc kubenswrapper[5052]: I1203 12:43:43.651234 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-db-create-dx5bv" event={"ID":"b99b3896-035c-4491-8093-80f669e31f64","Type":"ContainerDied","Data":"e1fb09b8278d70bf5e696ccda2966e127747528fc4cbcabd258e63122bef45e3"} Dec 03 12:43:43 crc kubenswrapper[5052]: I1203 12:43:43.651275 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e1fb09b8278d70bf5e696ccda2966e127747528fc4cbcabd258e63122bef45e3" Dec 03 12:43:43 crc kubenswrapper[5052]: I1203 12:43:43.651277 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-db-create-dx5bv" Dec 03 12:43:43 crc kubenswrapper[5052]: I1203 12:43:43.653980 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-db-create-ztpwm" event={"ID":"b4e4f28c-dc12-4844-a34a-14241afb73db","Type":"ContainerDied","Data":"216d18fbf839f26fb17f5b534853900d0c6cc60fce25f14e16a03091fcbc4b63"} Dec 03 12:43:43 crc kubenswrapper[5052]: I1203 12:43:43.654043 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="216d18fbf839f26fb17f5b534853900d0c6cc60fce25f14e16a03091fcbc4b63" Dec 03 12:43:43 crc kubenswrapper[5052]: I1203 12:43:43.654153 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-db-create-ztpwm" Dec 03 12:43:43 crc kubenswrapper[5052]: I1203 12:43:43.661873 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"fa98aa64-e8ed-442e-90a0-6823aaa88edb","Type":"ContainerStarted","Data":"3568fe20f05b7aa147cdfcb12cb7bb90f5c1e4b924211cc0ad6668d18a082ef5"} Dec 03 12:43:44 crc kubenswrapper[5052]: I1203 12:43:44.191939 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-db-create-tkzwz" Dec 03 12:43:44 crc kubenswrapper[5052]: I1203 12:43:44.232238 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/6e49273b-7e4d-4818-8f55-2db55a1250b9-operator-scripts\") pod \"6e49273b-7e4d-4818-8f55-2db55a1250b9\" (UID: \"6e49273b-7e4d-4818-8f55-2db55a1250b9\") " Dec 03 12:43:44 crc kubenswrapper[5052]: I1203 12:43:44.232299 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-889zs\" (UniqueName: \"kubernetes.io/projected/6e49273b-7e4d-4818-8f55-2db55a1250b9-kube-api-access-889zs\") pod \"6e49273b-7e4d-4818-8f55-2db55a1250b9\" (UID: \"6e49273b-7e4d-4818-8f55-2db55a1250b9\") " Dec 03 12:43:44 crc kubenswrapper[5052]: I1203 12:43:44.233434 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6e49273b-7e4d-4818-8f55-2db55a1250b9-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "6e49273b-7e4d-4818-8f55-2db55a1250b9" (UID: "6e49273b-7e4d-4818-8f55-2db55a1250b9"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:43:44 crc kubenswrapper[5052]: I1203 12:43:44.267752 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6e49273b-7e4d-4818-8f55-2db55a1250b9-kube-api-access-889zs" (OuterVolumeSpecName: "kube-api-access-889zs") pod "6e49273b-7e4d-4818-8f55-2db55a1250b9" (UID: "6e49273b-7e4d-4818-8f55-2db55a1250b9"). InnerVolumeSpecName "kube-api-access-889zs". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:43:44 crc kubenswrapper[5052]: I1203 12:43:44.334254 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/6e49273b-7e4d-4818-8f55-2db55a1250b9-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:43:44 crc kubenswrapper[5052]: I1203 12:43:44.334281 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-889zs\" (UniqueName: \"kubernetes.io/projected/6e49273b-7e4d-4818-8f55-2db55a1250b9-kube-api-access-889zs\") on node \"crc\" DevicePath \"\"" Dec 03 12:43:44 crc kubenswrapper[5052]: I1203 12:43:44.389405 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-4975-account-create-update-77cc4" Dec 03 12:43:44 crc kubenswrapper[5052]: I1203 12:43:44.395916 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-5566-account-create-update-wh2vb" Dec 03 12:43:44 crc kubenswrapper[5052]: I1203 12:43:44.415072 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-fd1c-account-create-update-vxzns" Dec 03 12:43:44 crc kubenswrapper[5052]: I1203 12:43:44.435995 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/795fe0cc-73d2-4647-ad2a-648f57e74fc3-operator-scripts\") pod \"795fe0cc-73d2-4647-ad2a-648f57e74fc3\" (UID: \"795fe0cc-73d2-4647-ad2a-648f57e74fc3\") " Dec 03 12:43:44 crc kubenswrapper[5052]: I1203 12:43:44.436181 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c6q6w\" (UniqueName: \"kubernetes.io/projected/7e89765b-8d00-48dc-9f2a-d7b6d7d7b5f0-kube-api-access-c6q6w\") pod \"7e89765b-8d00-48dc-9f2a-d7b6d7d7b5f0\" (UID: \"7e89765b-8d00-48dc-9f2a-d7b6d7d7b5f0\") " Dec 03 12:43:44 crc kubenswrapper[5052]: I1203 12:43:44.436241 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/d857bbf5-cd76-4d28-863c-1507de5c6cfd-operator-scripts\") pod \"d857bbf5-cd76-4d28-863c-1507de5c6cfd\" (UID: \"d857bbf5-cd76-4d28-863c-1507de5c6cfd\") " Dec 03 12:43:44 crc kubenswrapper[5052]: I1203 12:43:44.436313 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/7e89765b-8d00-48dc-9f2a-d7b6d7d7b5f0-operator-scripts\") pod \"7e89765b-8d00-48dc-9f2a-d7b6d7d7b5f0\" (UID: \"7e89765b-8d00-48dc-9f2a-d7b6d7d7b5f0\") " Dec 03 12:43:44 crc kubenswrapper[5052]: I1203 12:43:44.436369 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l622h\" (UniqueName: \"kubernetes.io/projected/795fe0cc-73d2-4647-ad2a-648f57e74fc3-kube-api-access-l622h\") pod \"795fe0cc-73d2-4647-ad2a-648f57e74fc3\" (UID: \"795fe0cc-73d2-4647-ad2a-648f57e74fc3\") " Dec 03 12:43:44 crc kubenswrapper[5052]: I1203 12:43:44.436406 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tthv4\" (UniqueName: \"kubernetes.io/projected/d857bbf5-cd76-4d28-863c-1507de5c6cfd-kube-api-access-tthv4\") pod \"d857bbf5-cd76-4d28-863c-1507de5c6cfd\" (UID: \"d857bbf5-cd76-4d28-863c-1507de5c6cfd\") " Dec 03 12:43:44 crc kubenswrapper[5052]: I1203 12:43:44.436943 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/795fe0cc-73d2-4647-ad2a-648f57e74fc3-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "795fe0cc-73d2-4647-ad2a-648f57e74fc3" (UID: "795fe0cc-73d2-4647-ad2a-648f57e74fc3"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:43:44 crc kubenswrapper[5052]: I1203 12:43:44.438372 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7e89765b-8d00-48dc-9f2a-d7b6d7d7b5f0-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "7e89765b-8d00-48dc-9f2a-d7b6d7d7b5f0" (UID: "7e89765b-8d00-48dc-9f2a-d7b6d7d7b5f0"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:43:44 crc kubenswrapper[5052]: I1203 12:43:44.438756 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d857bbf5-cd76-4d28-863c-1507de5c6cfd-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "d857bbf5-cd76-4d28-863c-1507de5c6cfd" (UID: "d857bbf5-cd76-4d28-863c-1507de5c6cfd"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:43:44 crc kubenswrapper[5052]: I1203 12:43:44.440602 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7e89765b-8d00-48dc-9f2a-d7b6d7d7b5f0-kube-api-access-c6q6w" (OuterVolumeSpecName: "kube-api-access-c6q6w") pod "7e89765b-8d00-48dc-9f2a-d7b6d7d7b5f0" (UID: "7e89765b-8d00-48dc-9f2a-d7b6d7d7b5f0"). InnerVolumeSpecName "kube-api-access-c6q6w". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:43:44 crc kubenswrapper[5052]: I1203 12:43:44.441119 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/795fe0cc-73d2-4647-ad2a-648f57e74fc3-kube-api-access-l622h" (OuterVolumeSpecName: "kube-api-access-l622h") pod "795fe0cc-73d2-4647-ad2a-648f57e74fc3" (UID: "795fe0cc-73d2-4647-ad2a-648f57e74fc3"). InnerVolumeSpecName "kube-api-access-l622h". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:43:44 crc kubenswrapper[5052]: I1203 12:43:44.443329 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d857bbf5-cd76-4d28-863c-1507de5c6cfd-kube-api-access-tthv4" (OuterVolumeSpecName: "kube-api-access-tthv4") pod "d857bbf5-cd76-4d28-863c-1507de5c6cfd" (UID: "d857bbf5-cd76-4d28-863c-1507de5c6cfd"). InnerVolumeSpecName "kube-api-access-tthv4". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:43:44 crc kubenswrapper[5052]: I1203 12:43:44.539062 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/d857bbf5-cd76-4d28-863c-1507de5c6cfd-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:43:44 crc kubenswrapper[5052]: I1203 12:43:44.539117 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/7e89765b-8d00-48dc-9f2a-d7b6d7d7b5f0-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:43:44 crc kubenswrapper[5052]: I1203 12:43:44.539134 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-l622h\" (UniqueName: \"kubernetes.io/projected/795fe0cc-73d2-4647-ad2a-648f57e74fc3-kube-api-access-l622h\") on node \"crc\" DevicePath \"\"" Dec 03 12:43:44 crc kubenswrapper[5052]: I1203 12:43:44.539148 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-tthv4\" (UniqueName: \"kubernetes.io/projected/d857bbf5-cd76-4d28-863c-1507de5c6cfd-kube-api-access-tthv4\") on node \"crc\" DevicePath \"\"" Dec 03 12:43:44 crc kubenswrapper[5052]: I1203 12:43:44.539160 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/795fe0cc-73d2-4647-ad2a-648f57e74fc3-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:43:44 crc kubenswrapper[5052]: I1203 12:43:44.539172 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-c6q6w\" (UniqueName: \"kubernetes.io/projected/7e89765b-8d00-48dc-9f2a-d7b6d7d7b5f0-kube-api-access-c6q6w\") on node \"crc\" DevicePath \"\"" Dec 03 12:43:44 crc kubenswrapper[5052]: I1203 12:43:44.673147 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-4975-account-create-update-77cc4" Dec 03 12:43:44 crc kubenswrapper[5052]: I1203 12:43:44.673141 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-4975-account-create-update-77cc4" event={"ID":"7e89765b-8d00-48dc-9f2a-d7b6d7d7b5f0","Type":"ContainerDied","Data":"9e04c6807ba25e04f108eb49fbcd1e4e94c7afea0b64d139ede600b2bce0de31"} Dec 03 12:43:44 crc kubenswrapper[5052]: I1203 12:43:44.673259 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9e04c6807ba25e04f108eb49fbcd1e4e94c7afea0b64d139ede600b2bce0de31" Dec 03 12:43:44 crc kubenswrapper[5052]: I1203 12:43:44.675614 5052 generic.go:334] "Generic (PLEG): container finished" podID="8011917c-0ce8-4c83-b157-2d8538790e17" containerID="c38db48afa4b344c32d00e4db4d452c8f75d832d0c682df5c1a48a83e977e312" exitCode=143 Dec 03 12:43:44 crc kubenswrapper[5052]: I1203 12:43:44.675640 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"8011917c-0ce8-4c83-b157-2d8538790e17","Type":"ContainerDied","Data":"c38db48afa4b344c32d00e4db4d452c8f75d832d0c682df5c1a48a83e977e312"} Dec 03 12:43:44 crc kubenswrapper[5052]: I1203 12:43:44.681394 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/community-operators-gt2q8" event={"ID":"3f17a210-d54b-4753-91c1-2d9b97565b4e","Type":"ContainerStarted","Data":"15082f95a5305cb2b4114d7cb325f148e1d42e449b1a4dd29fe625f2e9572260"} Dec 03 12:43:44 crc kubenswrapper[5052]: I1203 12:43:44.684986 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-fd1c-account-create-update-vxzns" event={"ID":"795fe0cc-73d2-4647-ad2a-648f57e74fc3","Type":"ContainerDied","Data":"bbd583eb50d3dac6c9a9f9f4633fab53b7e4a1c4be7662a48caeb78c985b8b37"} Dec 03 12:43:44 crc kubenswrapper[5052]: I1203 12:43:44.685028 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bbd583eb50d3dac6c9a9f9f4633fab53b7e4a1c4be7662a48caeb78c985b8b37" Dec 03 12:43:44 crc kubenswrapper[5052]: I1203 12:43:44.685087 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-fd1c-account-create-update-vxzns" Dec 03 12:43:44 crc kubenswrapper[5052]: I1203 12:43:44.690034 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-db-create-tkzwz" event={"ID":"6e49273b-7e4d-4818-8f55-2db55a1250b9","Type":"ContainerDied","Data":"3f56bb8a4ab6cac256920768504faa8651d931c90cc764ae9c80ab621bfad532"} Dec 03 12:43:44 crc kubenswrapper[5052]: I1203 12:43:44.690068 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3f56bb8a4ab6cac256920768504faa8651d931c90cc764ae9c80ab621bfad532" Dec 03 12:43:44 crc kubenswrapper[5052]: I1203 12:43:44.690125 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-db-create-tkzwz" Dec 03 12:43:44 crc kubenswrapper[5052]: I1203 12:43:44.697363 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-5566-account-create-update-wh2vb" event={"ID":"d857bbf5-cd76-4d28-863c-1507de5c6cfd","Type":"ContainerDied","Data":"9e0bd52f613c4b9ca31d9340a69db4f6e1ce445db1e095a99a966db97fe951a0"} Dec 03 12:43:44 crc kubenswrapper[5052]: I1203 12:43:44.697400 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9e0bd52f613c4b9ca31d9340a69db4f6e1ce445db1e095a99a966db97fe951a0" Dec 03 12:43:44 crc kubenswrapper[5052]: I1203 12:43:44.697458 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-5566-account-create-update-wh2vb" Dec 03 12:43:44 crc kubenswrapper[5052]: I1203 12:43:44.727295 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-marketplace/community-operators-gt2q8" podStartSLOduration=3.256424611 podStartE2EDuration="8.727274868s" podCreationTimestamp="2025-12-03 12:43:36 +0000 UTC" firstStartedPulling="2025-12-03 12:43:38.504781636 +0000 UTC m=+5009.093023400" lastFinishedPulling="2025-12-03 12:43:43.975631893 +0000 UTC m=+5014.563873657" observedRunningTime="2025-12-03 12:43:44.719328001 +0000 UTC m=+5015.307569775" watchObservedRunningTime="2025-12-03 12:43:44.727274868 +0000 UTC m=+5015.315516632" Dec 03 12:43:46 crc kubenswrapper[5052]: I1203 12:43:46.484695 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/swift-proxy-dd4cf57c8-x4c45" Dec 03 12:43:46 crc kubenswrapper[5052]: I1203 12:43:46.490768 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/swift-proxy-dd4cf57c8-x4c45" Dec 03 12:43:46 crc kubenswrapper[5052]: I1203 12:43:46.577650 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-marketplace/community-operators-gt2q8" Dec 03 12:43:46 crc kubenswrapper[5052]: I1203 12:43:46.577691 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openshift-marketplace/community-operators-gt2q8" Dec 03 12:43:46 crc kubenswrapper[5052]: I1203 12:43:46.773883 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/glance-default-internal-api-0" podUID="8011917c-0ce8-4c83-b157-2d8538790e17" containerName="glance-log" probeResult="failure" output="Get \"https://10.217.1.183:9292/healthcheck\": read tcp 10.217.0.2:56456->10.217.1.183:9292: read: connection reset by peer" Dec 03 12:43:46 crc kubenswrapper[5052]: I1203 12:43:46.774068 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/glance-default-internal-api-0" podUID="8011917c-0ce8-4c83-b157-2d8538790e17" containerName="glance-httpd" probeResult="failure" output="Get \"https://10.217.1.183:9292/healthcheck\": read tcp 10.217.0.2:56460->10.217.1.183:9292: read: connection reset by peer" Dec 03 12:43:47 crc kubenswrapper[5052]: I1203 12:43:47.631424 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openshift-marketplace/community-operators-gt2q8" podUID="3f17a210-d54b-4753-91c1-2d9b97565b4e" containerName="registry-server" probeResult="failure" output=< Dec 03 12:43:47 crc kubenswrapper[5052]: timeout: failed to connect service ":50051" within 1s Dec 03 12:43:47 crc kubenswrapper[5052]: > Dec 03 12:43:47 crc kubenswrapper[5052]: I1203 12:43:47.734533 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"fa98aa64-e8ed-442e-90a0-6823aaa88edb","Type":"ContainerStarted","Data":"c00bc3cdc6070763956a6f62a9aae570b12ea54c5b9b331777f9d6473f462dac"} Dec 03 12:43:48 crc kubenswrapper[5052]: I1203 12:43:48.327383 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-db-sync-ts4vh"] Dec 03 12:43:48 crc kubenswrapper[5052]: E1203 12:43:48.327950 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="7e89765b-8d00-48dc-9f2a-d7b6d7d7b5f0" containerName="mariadb-account-create-update" Dec 03 12:43:48 crc kubenswrapper[5052]: I1203 12:43:48.327969 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="7e89765b-8d00-48dc-9f2a-d7b6d7d7b5f0" containerName="mariadb-account-create-update" Dec 03 12:43:48 crc kubenswrapper[5052]: E1203 12:43:48.327985 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="6e49273b-7e4d-4818-8f55-2db55a1250b9" containerName="mariadb-database-create" Dec 03 12:43:48 crc kubenswrapper[5052]: I1203 12:43:48.327993 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="6e49273b-7e4d-4818-8f55-2db55a1250b9" containerName="mariadb-database-create" Dec 03 12:43:48 crc kubenswrapper[5052]: E1203 12:43:48.328008 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="795fe0cc-73d2-4647-ad2a-648f57e74fc3" containerName="mariadb-account-create-update" Dec 03 12:43:48 crc kubenswrapper[5052]: I1203 12:43:48.328015 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="795fe0cc-73d2-4647-ad2a-648f57e74fc3" containerName="mariadb-account-create-update" Dec 03 12:43:48 crc kubenswrapper[5052]: E1203 12:43:48.328029 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="b4e4f28c-dc12-4844-a34a-14241afb73db" containerName="mariadb-database-create" Dec 03 12:43:48 crc kubenswrapper[5052]: I1203 12:43:48.328035 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="b4e4f28c-dc12-4844-a34a-14241afb73db" containerName="mariadb-database-create" Dec 03 12:43:48 crc kubenswrapper[5052]: E1203 12:43:48.328048 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="b99b3896-035c-4491-8093-80f669e31f64" containerName="mariadb-database-create" Dec 03 12:43:48 crc kubenswrapper[5052]: I1203 12:43:48.328054 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="b99b3896-035c-4491-8093-80f669e31f64" containerName="mariadb-database-create" Dec 03 12:43:48 crc kubenswrapper[5052]: E1203 12:43:48.328091 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="d857bbf5-cd76-4d28-863c-1507de5c6cfd" containerName="mariadb-account-create-update" Dec 03 12:43:48 crc kubenswrapper[5052]: I1203 12:43:48.328097 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="d857bbf5-cd76-4d28-863c-1507de5c6cfd" containerName="mariadb-account-create-update" Dec 03 12:43:48 crc kubenswrapper[5052]: I1203 12:43:48.328324 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="7e89765b-8d00-48dc-9f2a-d7b6d7d7b5f0" containerName="mariadb-account-create-update" Dec 03 12:43:48 crc kubenswrapper[5052]: I1203 12:43:48.328340 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="d857bbf5-cd76-4d28-863c-1507de5c6cfd" containerName="mariadb-account-create-update" Dec 03 12:43:48 crc kubenswrapper[5052]: I1203 12:43:48.328354 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="6e49273b-7e4d-4818-8f55-2db55a1250b9" containerName="mariadb-database-create" Dec 03 12:43:48 crc kubenswrapper[5052]: I1203 12:43:48.328364 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="795fe0cc-73d2-4647-ad2a-648f57e74fc3" containerName="mariadb-account-create-update" Dec 03 12:43:48 crc kubenswrapper[5052]: I1203 12:43:48.328374 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="b99b3896-035c-4491-8093-80f669e31f64" containerName="mariadb-database-create" Dec 03 12:43:48 crc kubenswrapper[5052]: I1203 12:43:48.328387 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="b4e4f28c-dc12-4844-a34a-14241afb73db" containerName="mariadb-database-create" Dec 03 12:43:48 crc kubenswrapper[5052]: I1203 12:43:48.329092 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-ts4vh" Dec 03 12:43:48 crc kubenswrapper[5052]: I1203 12:43:48.338709 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-db-sync-ts4vh"] Dec 03 12:43:48 crc kubenswrapper[5052]: I1203 12:43:48.338867 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell0-conductor-scripts" Dec 03 12:43:48 crc kubenswrapper[5052]: I1203 12:43:48.339089 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell0-conductor-config-data" Dec 03 12:43:48 crc kubenswrapper[5052]: I1203 12:43:48.343603 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-nova-dockercfg-tj8z2" Dec 03 12:43:48 crc kubenswrapper[5052]: I1203 12:43:48.412351 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/9fd7aa9d-4ad5-4058-a784-6814fbd76852-scripts\") pod \"nova-cell0-conductor-db-sync-ts4vh\" (UID: \"9fd7aa9d-4ad5-4058-a784-6814fbd76852\") " pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-ts4vh" Dec 03 12:43:48 crc kubenswrapper[5052]: I1203 12:43:48.412672 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/9fd7aa9d-4ad5-4058-a784-6814fbd76852-config-data\") pod \"nova-cell0-conductor-db-sync-ts4vh\" (UID: \"9fd7aa9d-4ad5-4058-a784-6814fbd76852\") " pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-ts4vh" Dec 03 12:43:48 crc kubenswrapper[5052]: I1203 12:43:48.412753 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9fd7aa9d-4ad5-4058-a784-6814fbd76852-combined-ca-bundle\") pod \"nova-cell0-conductor-db-sync-ts4vh\" (UID: \"9fd7aa9d-4ad5-4058-a784-6814fbd76852\") " pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-ts4vh" Dec 03 12:43:48 crc kubenswrapper[5052]: I1203 12:43:48.412882 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vkmp6\" (UniqueName: \"kubernetes.io/projected/9fd7aa9d-4ad5-4058-a784-6814fbd76852-kube-api-access-vkmp6\") pod \"nova-cell0-conductor-db-sync-ts4vh\" (UID: \"9fd7aa9d-4ad5-4058-a784-6814fbd76852\") " pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-ts4vh" Dec 03 12:43:48 crc kubenswrapper[5052]: I1203 12:43:48.514517 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-vkmp6\" (UniqueName: \"kubernetes.io/projected/9fd7aa9d-4ad5-4058-a784-6814fbd76852-kube-api-access-vkmp6\") pod \"nova-cell0-conductor-db-sync-ts4vh\" (UID: \"9fd7aa9d-4ad5-4058-a784-6814fbd76852\") " pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-ts4vh" Dec 03 12:43:48 crc kubenswrapper[5052]: I1203 12:43:48.514696 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/9fd7aa9d-4ad5-4058-a784-6814fbd76852-scripts\") pod \"nova-cell0-conductor-db-sync-ts4vh\" (UID: \"9fd7aa9d-4ad5-4058-a784-6814fbd76852\") " pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-ts4vh" Dec 03 12:43:48 crc kubenswrapper[5052]: I1203 12:43:48.514765 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/9fd7aa9d-4ad5-4058-a784-6814fbd76852-config-data\") pod \"nova-cell0-conductor-db-sync-ts4vh\" (UID: \"9fd7aa9d-4ad5-4058-a784-6814fbd76852\") " pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-ts4vh" Dec 03 12:43:48 crc kubenswrapper[5052]: I1203 12:43:48.514798 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9fd7aa9d-4ad5-4058-a784-6814fbd76852-combined-ca-bundle\") pod \"nova-cell0-conductor-db-sync-ts4vh\" (UID: \"9fd7aa9d-4ad5-4058-a784-6814fbd76852\") " pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-ts4vh" Dec 03 12:43:48 crc kubenswrapper[5052]: I1203 12:43:48.520680 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/9fd7aa9d-4ad5-4058-a784-6814fbd76852-scripts\") pod \"nova-cell0-conductor-db-sync-ts4vh\" (UID: \"9fd7aa9d-4ad5-4058-a784-6814fbd76852\") " pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-ts4vh" Dec 03 12:43:48 crc kubenswrapper[5052]: I1203 12:43:48.520847 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/9fd7aa9d-4ad5-4058-a784-6814fbd76852-config-data\") pod \"nova-cell0-conductor-db-sync-ts4vh\" (UID: \"9fd7aa9d-4ad5-4058-a784-6814fbd76852\") " pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-ts4vh" Dec 03 12:43:48 crc kubenswrapper[5052]: I1203 12:43:48.521114 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9fd7aa9d-4ad5-4058-a784-6814fbd76852-combined-ca-bundle\") pod \"nova-cell0-conductor-db-sync-ts4vh\" (UID: \"9fd7aa9d-4ad5-4058-a784-6814fbd76852\") " pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-ts4vh" Dec 03 12:43:48 crc kubenswrapper[5052]: I1203 12:43:48.542694 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-vkmp6\" (UniqueName: \"kubernetes.io/projected/9fd7aa9d-4ad5-4058-a784-6814fbd76852-kube-api-access-vkmp6\") pod \"nova-cell0-conductor-db-sync-ts4vh\" (UID: \"9fd7aa9d-4ad5-4058-a784-6814fbd76852\") " pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-ts4vh" Dec 03 12:43:48 crc kubenswrapper[5052]: I1203 12:43:48.663165 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-ts4vh" Dec 03 12:43:48 crc kubenswrapper[5052]: I1203 12:43:48.790448 5052 generic.go:334] "Generic (PLEG): container finished" podID="8011917c-0ce8-4c83-b157-2d8538790e17" containerID="d871ecf0facb775d7ef4d9dacb2a2746b353a7fee55e6f036959cc141b2fcf0e" exitCode=0 Dec 03 12:43:48 crc kubenswrapper[5052]: I1203 12:43:48.791365 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="fa98aa64-e8ed-442e-90a0-6823aaa88edb" containerName="ceilometer-central-agent" containerID="cri-o://c3413a78a793e89bd0b3f298b6e5e3d40cfcd493e84ae55bfbc2ea5d5d154265" gracePeriod=30 Dec 03 12:43:48 crc kubenswrapper[5052]: I1203 12:43:48.791798 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"8011917c-0ce8-4c83-b157-2d8538790e17","Type":"ContainerDied","Data":"d871ecf0facb775d7ef4d9dacb2a2746b353a7fee55e6f036959cc141b2fcf0e"} Dec 03 12:43:48 crc kubenswrapper[5052]: I1203 12:43:48.791956 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:43:48 crc kubenswrapper[5052]: I1203 12:43:48.792438 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="fa98aa64-e8ed-442e-90a0-6823aaa88edb" containerName="proxy-httpd" containerID="cri-o://c00bc3cdc6070763956a6f62a9aae570b12ea54c5b9b331777f9d6473f462dac" gracePeriod=30 Dec 03 12:43:48 crc kubenswrapper[5052]: I1203 12:43:48.792628 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="fa98aa64-e8ed-442e-90a0-6823aaa88edb" containerName="sg-core" containerID="cri-o://3568fe20f05b7aa147cdfcb12cb7bb90f5c1e4b924211cc0ad6668d18a082ef5" gracePeriod=30 Dec 03 12:43:48 crc kubenswrapper[5052]: I1203 12:43:48.792769 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="fa98aa64-e8ed-442e-90a0-6823aaa88edb" containerName="ceilometer-notification-agent" containerID="cri-o://2d10f32dbe1a375fa4b22fdd2415c34559e7c30be12f35f8ef9503c291f76702" gracePeriod=30 Dec 03 12:43:48 crc kubenswrapper[5052]: I1203 12:43:48.828764 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/ceilometer-0" podStartSLOduration=4.171087929 podStartE2EDuration="12.828743463s" podCreationTimestamp="2025-12-03 12:43:36 +0000 UTC" firstStartedPulling="2025-12-03 12:43:37.619223551 +0000 UTC m=+5008.207465315" lastFinishedPulling="2025-12-03 12:43:46.276879085 +0000 UTC m=+5016.865120849" observedRunningTime="2025-12-03 12:43:48.815439113 +0000 UTC m=+5019.403680897" watchObservedRunningTime="2025-12-03 12:43:48.828743463 +0000 UTC m=+5019.416985227" Dec 03 12:43:49 crc kubenswrapper[5052]: I1203 12:43:49.254774 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-db-sync-ts4vh"] Dec 03 12:43:49 crc kubenswrapper[5052]: W1203 12:43:49.630308 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9fd7aa9d_4ad5_4058_a784_6814fbd76852.slice/crio-ff63c7d5be65035de7226b4a4eca9c62944cf4775fb258fb4031eaf6fb6e4005 WatchSource:0}: Error finding container ff63c7d5be65035de7226b4a4eca9c62944cf4775fb258fb4031eaf6fb6e4005: Status 404 returned error can't find the container with id ff63c7d5be65035de7226b4a4eca9c62944cf4775fb258fb4031eaf6fb6e4005 Dec 03 12:43:49 crc kubenswrapper[5052]: I1203 12:43:49.804125 5052 generic.go:334] "Generic (PLEG): container finished" podID="fa98aa64-e8ed-442e-90a0-6823aaa88edb" containerID="c00bc3cdc6070763956a6f62a9aae570b12ea54c5b9b331777f9d6473f462dac" exitCode=0 Dec 03 12:43:49 crc kubenswrapper[5052]: I1203 12:43:49.804465 5052 generic.go:334] "Generic (PLEG): container finished" podID="fa98aa64-e8ed-442e-90a0-6823aaa88edb" containerID="3568fe20f05b7aa147cdfcb12cb7bb90f5c1e4b924211cc0ad6668d18a082ef5" exitCode=2 Dec 03 12:43:49 crc kubenswrapper[5052]: I1203 12:43:49.804473 5052 generic.go:334] "Generic (PLEG): container finished" podID="fa98aa64-e8ed-442e-90a0-6823aaa88edb" containerID="2d10f32dbe1a375fa4b22fdd2415c34559e7c30be12f35f8ef9503c291f76702" exitCode=0 Dec 03 12:43:49 crc kubenswrapper[5052]: I1203 12:43:49.804584 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"fa98aa64-e8ed-442e-90a0-6823aaa88edb","Type":"ContainerDied","Data":"c00bc3cdc6070763956a6f62a9aae570b12ea54c5b9b331777f9d6473f462dac"} Dec 03 12:43:49 crc kubenswrapper[5052]: I1203 12:43:49.804612 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"fa98aa64-e8ed-442e-90a0-6823aaa88edb","Type":"ContainerDied","Data":"3568fe20f05b7aa147cdfcb12cb7bb90f5c1e4b924211cc0ad6668d18a082ef5"} Dec 03 12:43:49 crc kubenswrapper[5052]: I1203 12:43:49.804621 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"fa98aa64-e8ed-442e-90a0-6823aaa88edb","Type":"ContainerDied","Data":"2d10f32dbe1a375fa4b22fdd2415c34559e7c30be12f35f8ef9503c291f76702"} Dec 03 12:43:49 crc kubenswrapper[5052]: I1203 12:43:49.808104 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-ts4vh" event={"ID":"9fd7aa9d-4ad5-4058-a784-6814fbd76852","Type":"ContainerStarted","Data":"ff63c7d5be65035de7226b4a4eca9c62944cf4775fb258fb4031eaf6fb6e4005"} Dec 03 12:43:49 crc kubenswrapper[5052]: I1203 12:43:49.811929 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"8011917c-0ce8-4c83-b157-2d8538790e17","Type":"ContainerDied","Data":"3d59485fd382731e310136b18cd55378ab24d2433daddc0d9279d4e9e6835ee3"} Dec 03 12:43:49 crc kubenswrapper[5052]: I1203 12:43:49.811961 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3d59485fd382731e310136b18cd55378ab24d2433daddc0d9279d4e9e6835ee3" Dec 03 12:43:49 crc kubenswrapper[5052]: I1203 12:43:49.830143 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:43:50 crc kubenswrapper[5052]: I1203 12:43:50.011601 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/8011917c-0ce8-4c83-b157-2d8538790e17-scripts\") pod \"8011917c-0ce8-4c83-b157-2d8538790e17\" (UID: \"8011917c-0ce8-4c83-b157-2d8538790e17\") " Dec 03 12:43:50 crc kubenswrapper[5052]: I1203 12:43:50.011834 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/8011917c-0ce8-4c83-b157-2d8538790e17-internal-tls-certs\") pod \"8011917c-0ce8-4c83-b157-2d8538790e17\" (UID: \"8011917c-0ce8-4c83-b157-2d8538790e17\") " Dec 03 12:43:50 crc kubenswrapper[5052]: I1203 12:43:50.011990 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"glance\" (UniqueName: \"kubernetes.io/local-volume/local-storage18-crc\") pod \"8011917c-0ce8-4c83-b157-2d8538790e17\" (UID: \"8011917c-0ce8-4c83-b157-2d8538790e17\") " Dec 03 12:43:50 crc kubenswrapper[5052]: I1203 12:43:50.012079 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/8011917c-0ce8-4c83-b157-2d8538790e17-logs\") pod \"8011917c-0ce8-4c83-b157-2d8538790e17\" (UID: \"8011917c-0ce8-4c83-b157-2d8538790e17\") " Dec 03 12:43:50 crc kubenswrapper[5052]: I1203 12:43:50.012172 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8011917c-0ce8-4c83-b157-2d8538790e17-combined-ca-bundle\") pod \"8011917c-0ce8-4c83-b157-2d8538790e17\" (UID: \"8011917c-0ce8-4c83-b157-2d8538790e17\") " Dec 03 12:43:50 crc kubenswrapper[5052]: I1203 12:43:50.012303 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/8011917c-0ce8-4c83-b157-2d8538790e17-httpd-run\") pod \"8011917c-0ce8-4c83-b157-2d8538790e17\" (UID: \"8011917c-0ce8-4c83-b157-2d8538790e17\") " Dec 03 12:43:50 crc kubenswrapper[5052]: I1203 12:43:50.012470 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/8011917c-0ce8-4c83-b157-2d8538790e17-config-data\") pod \"8011917c-0ce8-4c83-b157-2d8538790e17\" (UID: \"8011917c-0ce8-4c83-b157-2d8538790e17\") " Dec 03 12:43:50 crc kubenswrapper[5052]: I1203 12:43:50.012619 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xkvtd\" (UniqueName: \"kubernetes.io/projected/8011917c-0ce8-4c83-b157-2d8538790e17-kube-api-access-xkvtd\") pod \"8011917c-0ce8-4c83-b157-2d8538790e17\" (UID: \"8011917c-0ce8-4c83-b157-2d8538790e17\") " Dec 03 12:43:50 crc kubenswrapper[5052]: I1203 12:43:50.013695 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8011917c-0ce8-4c83-b157-2d8538790e17-httpd-run" (OuterVolumeSpecName: "httpd-run") pod "8011917c-0ce8-4c83-b157-2d8538790e17" (UID: "8011917c-0ce8-4c83-b157-2d8538790e17"). InnerVolumeSpecName "httpd-run". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:43:50 crc kubenswrapper[5052]: I1203 12:43:50.013722 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8011917c-0ce8-4c83-b157-2d8538790e17-logs" (OuterVolumeSpecName: "logs") pod "8011917c-0ce8-4c83-b157-2d8538790e17" (UID: "8011917c-0ce8-4c83-b157-2d8538790e17"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:43:50 crc kubenswrapper[5052]: I1203 12:43:50.016584 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8011917c-0ce8-4c83-b157-2d8538790e17-scripts" (OuterVolumeSpecName: "scripts") pod "8011917c-0ce8-4c83-b157-2d8538790e17" (UID: "8011917c-0ce8-4c83-b157-2d8538790e17"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:43:50 crc kubenswrapper[5052]: I1203 12:43:50.016679 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage18-crc" (OuterVolumeSpecName: "glance") pod "8011917c-0ce8-4c83-b157-2d8538790e17" (UID: "8011917c-0ce8-4c83-b157-2d8538790e17"). InnerVolumeSpecName "local-storage18-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 12:43:50 crc kubenswrapper[5052]: I1203 12:43:50.020729 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8011917c-0ce8-4c83-b157-2d8538790e17-kube-api-access-xkvtd" (OuterVolumeSpecName: "kube-api-access-xkvtd") pod "8011917c-0ce8-4c83-b157-2d8538790e17" (UID: "8011917c-0ce8-4c83-b157-2d8538790e17"). InnerVolumeSpecName "kube-api-access-xkvtd". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:43:50 crc kubenswrapper[5052]: I1203 12:43:50.056101 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8011917c-0ce8-4c83-b157-2d8538790e17-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "8011917c-0ce8-4c83-b157-2d8538790e17" (UID: "8011917c-0ce8-4c83-b157-2d8538790e17"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:43:50 crc kubenswrapper[5052]: I1203 12:43:50.068679 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8011917c-0ce8-4c83-b157-2d8538790e17-config-data" (OuterVolumeSpecName: "config-data") pod "8011917c-0ce8-4c83-b157-2d8538790e17" (UID: "8011917c-0ce8-4c83-b157-2d8538790e17"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:43:50 crc kubenswrapper[5052]: I1203 12:43:50.078928 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8011917c-0ce8-4c83-b157-2d8538790e17-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "8011917c-0ce8-4c83-b157-2d8538790e17" (UID: "8011917c-0ce8-4c83-b157-2d8538790e17"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:43:50 crc kubenswrapper[5052]: I1203 12:43:50.115358 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage18-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage18-crc\") on node \"crc\" " Dec 03 12:43:50 crc kubenswrapper[5052]: I1203 12:43:50.115834 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/8011917c-0ce8-4c83-b157-2d8538790e17-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:43:50 crc kubenswrapper[5052]: I1203 12:43:50.115930 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8011917c-0ce8-4c83-b157-2d8538790e17-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:43:50 crc kubenswrapper[5052]: I1203 12:43:50.116019 5052 reconciler_common.go:293] "Volume detached for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/8011917c-0ce8-4c83-b157-2d8538790e17-httpd-run\") on node \"crc\" DevicePath \"\"" Dec 03 12:43:50 crc kubenswrapper[5052]: I1203 12:43:50.116098 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/8011917c-0ce8-4c83-b157-2d8538790e17-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:43:50 crc kubenswrapper[5052]: I1203 12:43:50.116182 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-xkvtd\" (UniqueName: \"kubernetes.io/projected/8011917c-0ce8-4c83-b157-2d8538790e17-kube-api-access-xkvtd\") on node \"crc\" DevicePath \"\"" Dec 03 12:43:50 crc kubenswrapper[5052]: I1203 12:43:50.116283 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/8011917c-0ce8-4c83-b157-2d8538790e17-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:43:50 crc kubenswrapper[5052]: I1203 12:43:50.116388 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/8011917c-0ce8-4c83-b157-2d8538790e17-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:43:50 crc kubenswrapper[5052]: I1203 12:43:50.138083 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage18-crc" (UniqueName: "kubernetes.io/local-volume/local-storage18-crc") on node "crc" Dec 03 12:43:50 crc kubenswrapper[5052]: I1203 12:43:50.218226 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage18-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage18-crc\") on node \"crc\" DevicePath \"\"" Dec 03 12:43:50 crc kubenswrapper[5052]: I1203 12:43:50.478947 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:43:50 crc kubenswrapper[5052]: I1203 12:43:50.479200 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-external-api-0" podUID="24ccc68c-26a9-485e-bf26-512dac006b61" containerName="glance-log" containerID="cri-o://23f9262d3a7c543b67221300c14860510bcdb1b097fab4de7283f6469298206b" gracePeriod=30 Dec 03 12:43:50 crc kubenswrapper[5052]: I1203 12:43:50.479338 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-external-api-0" podUID="24ccc68c-26a9-485e-bf26-512dac006b61" containerName="glance-httpd" containerID="cri-o://140826c39a57fa01ca2b271c735d5f9c464eb0c7c58b2f109153dcf2376fff98" gracePeriod=30 Dec 03 12:43:50 crc kubenswrapper[5052]: I1203 12:43:50.826570 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"24ccc68c-26a9-485e-bf26-512dac006b61","Type":"ContainerDied","Data":"23f9262d3a7c543b67221300c14860510bcdb1b097fab4de7283f6469298206b"} Dec 03 12:43:50 crc kubenswrapper[5052]: I1203 12:43:50.826506 5052 generic.go:334] "Generic (PLEG): container finished" podID="24ccc68c-26a9-485e-bf26-512dac006b61" containerID="23f9262d3a7c543b67221300c14860510bcdb1b097fab4de7283f6469298206b" exitCode=143 Dec 03 12:43:50 crc kubenswrapper[5052]: I1203 12:43:50.830451 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:43:50 crc kubenswrapper[5052]: I1203 12:43:50.831582 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-ts4vh" event={"ID":"9fd7aa9d-4ad5-4058-a784-6814fbd76852","Type":"ContainerStarted","Data":"8af40cfb10f5db16ec28ea25c0561fa79c3cc14b977ccb702d77cd50dcbe5974"} Dec 03 12:43:50 crc kubenswrapper[5052]: I1203 12:43:50.850914 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-ts4vh" podStartSLOduration=2.850899564 podStartE2EDuration="2.850899564s" podCreationTimestamp="2025-12-03 12:43:48 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:43:50.847744844 +0000 UTC m=+5021.435986608" watchObservedRunningTime="2025-12-03 12:43:50.850899564 +0000 UTC m=+5021.439141328" Dec 03 12:43:50 crc kubenswrapper[5052]: I1203 12:43:50.876938 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:43:50 crc kubenswrapper[5052]: I1203 12:43:50.892906 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:43:50 crc kubenswrapper[5052]: I1203 12:43:50.910410 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:43:50 crc kubenswrapper[5052]: E1203 12:43:50.911005 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="8011917c-0ce8-4c83-b157-2d8538790e17" containerName="glance-httpd" Dec 03 12:43:50 crc kubenswrapper[5052]: I1203 12:43:50.911025 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="8011917c-0ce8-4c83-b157-2d8538790e17" containerName="glance-httpd" Dec 03 12:43:50 crc kubenswrapper[5052]: E1203 12:43:50.911051 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="8011917c-0ce8-4c83-b157-2d8538790e17" containerName="glance-log" Dec 03 12:43:50 crc kubenswrapper[5052]: I1203 12:43:50.911059 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="8011917c-0ce8-4c83-b157-2d8538790e17" containerName="glance-log" Dec 03 12:43:50 crc kubenswrapper[5052]: I1203 12:43:50.911256 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="8011917c-0ce8-4c83-b157-2d8538790e17" containerName="glance-log" Dec 03 12:43:50 crc kubenswrapper[5052]: I1203 12:43:50.911284 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="8011917c-0ce8-4c83-b157-2d8538790e17" containerName="glance-httpd" Dec 03 12:43:50 crc kubenswrapper[5052]: I1203 12:43:50.912548 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:43:50 crc kubenswrapper[5052]: I1203 12:43:50.915916 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-glance-default-internal-svc" Dec 03 12:43:50 crc kubenswrapper[5052]: I1203 12:43:50.916094 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-default-internal-config-data" Dec 03 12:43:50 crc kubenswrapper[5052]: I1203 12:43:50.956838 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:43:51 crc kubenswrapper[5052]: I1203 12:43:51.038616 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/78361066-6add-4971-9a4b-9157f35ad494-scripts\") pod \"glance-default-internal-api-0\" (UID: \"78361066-6add-4971-9a4b-9157f35ad494\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:43:51 crc kubenswrapper[5052]: I1203 12:43:51.039007 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage18-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage18-crc\") pod \"glance-default-internal-api-0\" (UID: \"78361066-6add-4971-9a4b-9157f35ad494\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:43:51 crc kubenswrapper[5052]: I1203 12:43:51.039130 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/78361066-6add-4971-9a4b-9157f35ad494-internal-tls-certs\") pod \"glance-default-internal-api-0\" (UID: \"78361066-6add-4971-9a4b-9157f35ad494\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:43:51 crc kubenswrapper[5052]: I1203 12:43:51.039245 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/78361066-6add-4971-9a4b-9157f35ad494-httpd-run\") pod \"glance-default-internal-api-0\" (UID: \"78361066-6add-4971-9a4b-9157f35ad494\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:43:51 crc kubenswrapper[5052]: I1203 12:43:51.039502 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/78361066-6add-4971-9a4b-9157f35ad494-combined-ca-bundle\") pod \"glance-default-internal-api-0\" (UID: \"78361066-6add-4971-9a4b-9157f35ad494\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:43:51 crc kubenswrapper[5052]: I1203 12:43:51.039650 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/78361066-6add-4971-9a4b-9157f35ad494-config-data\") pod \"glance-default-internal-api-0\" (UID: \"78361066-6add-4971-9a4b-9157f35ad494\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:43:51 crc kubenswrapper[5052]: I1203 12:43:51.039694 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sp7f4\" (UniqueName: \"kubernetes.io/projected/78361066-6add-4971-9a4b-9157f35ad494-kube-api-access-sp7f4\") pod \"glance-default-internal-api-0\" (UID: \"78361066-6add-4971-9a4b-9157f35ad494\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:43:51 crc kubenswrapper[5052]: I1203 12:43:51.039729 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/78361066-6add-4971-9a4b-9157f35ad494-logs\") pod \"glance-default-internal-api-0\" (UID: \"78361066-6add-4971-9a4b-9157f35ad494\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:43:51 crc kubenswrapper[5052]: I1203 12:43:51.141951 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/78361066-6add-4971-9a4b-9157f35ad494-scripts\") pod \"glance-default-internal-api-0\" (UID: \"78361066-6add-4971-9a4b-9157f35ad494\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:43:51 crc kubenswrapper[5052]: I1203 12:43:51.142027 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage18-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage18-crc\") pod \"glance-default-internal-api-0\" (UID: \"78361066-6add-4971-9a4b-9157f35ad494\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:43:51 crc kubenswrapper[5052]: I1203 12:43:51.142054 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/78361066-6add-4971-9a4b-9157f35ad494-internal-tls-certs\") pod \"glance-default-internal-api-0\" (UID: \"78361066-6add-4971-9a4b-9157f35ad494\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:43:51 crc kubenswrapper[5052]: I1203 12:43:51.142079 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/78361066-6add-4971-9a4b-9157f35ad494-httpd-run\") pod \"glance-default-internal-api-0\" (UID: \"78361066-6add-4971-9a4b-9157f35ad494\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:43:51 crc kubenswrapper[5052]: I1203 12:43:51.142115 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/78361066-6add-4971-9a4b-9157f35ad494-combined-ca-bundle\") pod \"glance-default-internal-api-0\" (UID: \"78361066-6add-4971-9a4b-9157f35ad494\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:43:51 crc kubenswrapper[5052]: I1203 12:43:51.142153 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/78361066-6add-4971-9a4b-9157f35ad494-config-data\") pod \"glance-default-internal-api-0\" (UID: \"78361066-6add-4971-9a4b-9157f35ad494\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:43:51 crc kubenswrapper[5052]: I1203 12:43:51.142175 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-sp7f4\" (UniqueName: \"kubernetes.io/projected/78361066-6add-4971-9a4b-9157f35ad494-kube-api-access-sp7f4\") pod \"glance-default-internal-api-0\" (UID: \"78361066-6add-4971-9a4b-9157f35ad494\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:43:51 crc kubenswrapper[5052]: I1203 12:43:51.142199 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/78361066-6add-4971-9a4b-9157f35ad494-logs\") pod \"glance-default-internal-api-0\" (UID: \"78361066-6add-4971-9a4b-9157f35ad494\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:43:51 crc kubenswrapper[5052]: I1203 12:43:51.142784 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/78361066-6add-4971-9a4b-9157f35ad494-logs\") pod \"glance-default-internal-api-0\" (UID: \"78361066-6add-4971-9a4b-9157f35ad494\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:43:51 crc kubenswrapper[5052]: I1203 12:43:51.142938 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage18-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage18-crc\") pod \"glance-default-internal-api-0\" (UID: \"78361066-6add-4971-9a4b-9157f35ad494\") device mount path \"/mnt/openstack/pv18\"" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:43:51 crc kubenswrapper[5052]: I1203 12:43:51.143233 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/78361066-6add-4971-9a4b-9157f35ad494-httpd-run\") pod \"glance-default-internal-api-0\" (UID: \"78361066-6add-4971-9a4b-9157f35ad494\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:43:51 crc kubenswrapper[5052]: I1203 12:43:51.149690 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/78361066-6add-4971-9a4b-9157f35ad494-config-data\") pod \"glance-default-internal-api-0\" (UID: \"78361066-6add-4971-9a4b-9157f35ad494\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:43:51 crc kubenswrapper[5052]: I1203 12:43:51.150363 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/78361066-6add-4971-9a4b-9157f35ad494-combined-ca-bundle\") pod \"glance-default-internal-api-0\" (UID: \"78361066-6add-4971-9a4b-9157f35ad494\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:43:51 crc kubenswrapper[5052]: I1203 12:43:51.150901 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/78361066-6add-4971-9a4b-9157f35ad494-internal-tls-certs\") pod \"glance-default-internal-api-0\" (UID: \"78361066-6add-4971-9a4b-9157f35ad494\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:43:51 crc kubenswrapper[5052]: I1203 12:43:51.163293 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/78361066-6add-4971-9a4b-9157f35ad494-scripts\") pod \"glance-default-internal-api-0\" (UID: \"78361066-6add-4971-9a4b-9157f35ad494\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:43:51 crc kubenswrapper[5052]: I1203 12:43:51.168381 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-sp7f4\" (UniqueName: \"kubernetes.io/projected/78361066-6add-4971-9a4b-9157f35ad494-kube-api-access-sp7f4\") pod \"glance-default-internal-api-0\" (UID: \"78361066-6add-4971-9a4b-9157f35ad494\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:43:51 crc kubenswrapper[5052]: I1203 12:43:51.184712 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage18-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage18-crc\") pod \"glance-default-internal-api-0\" (UID: \"78361066-6add-4971-9a4b-9157f35ad494\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:43:51 crc kubenswrapper[5052]: I1203 12:43:51.250645 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:43:51 crc kubenswrapper[5052]: I1203 12:43:51.684920 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8011917c-0ce8-4c83-b157-2d8538790e17" path="/var/lib/kubelet/pods/8011917c-0ce8-4c83-b157-2d8538790e17/volumes" Dec 03 12:43:52 crc kubenswrapper[5052]: I1203 12:43:52.232233 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:43:52 crc kubenswrapper[5052]: I1203 12:43:52.855948 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"78361066-6add-4971-9a4b-9157f35ad494","Type":"ContainerStarted","Data":"8e72cb53ffa488be78204a96a13a28807e663bc5864239f1b6c619b61fcf83f4"} Dec 03 12:43:53 crc kubenswrapper[5052]: I1203 12:43:53.673923 5052 scope.go:117] "RemoveContainer" containerID="2aed1c760a326f3d008799e902b1093313df3764a57c879f0b3686410127d37d" Dec 03 12:43:53 crc kubenswrapper[5052]: E1203 12:43:53.674590 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 12:43:53 crc kubenswrapper[5052]: I1203 12:43:53.871487 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"78361066-6add-4971-9a4b-9157f35ad494","Type":"ContainerStarted","Data":"3310a6083ad0bd13b8590b4fa080eeb83719e19bfb1dddf441320e21f33c7afa"} Dec 03 12:43:53 crc kubenswrapper[5052]: I1203 12:43:53.871652 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"78361066-6add-4971-9a4b-9157f35ad494","Type":"ContainerStarted","Data":"6fe0649b6ce8be2bfd99aed0e4830c4017fd592947629aa86d78326653fa7af0"} Dec 03 12:43:53 crc kubenswrapper[5052]: I1203 12:43:53.880136 5052 generic.go:334] "Generic (PLEG): container finished" podID="24ccc68c-26a9-485e-bf26-512dac006b61" containerID="140826c39a57fa01ca2b271c735d5f9c464eb0c7c58b2f109153dcf2376fff98" exitCode=0 Dec 03 12:43:53 crc kubenswrapper[5052]: I1203 12:43:53.880183 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"24ccc68c-26a9-485e-bf26-512dac006b61","Type":"ContainerDied","Data":"140826c39a57fa01ca2b271c735d5f9c464eb0c7c58b2f109153dcf2376fff98"} Dec 03 12:43:53 crc kubenswrapper[5052]: I1203 12:43:53.895858 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/glance-default-internal-api-0" podStartSLOduration=3.895838294 podStartE2EDuration="3.895838294s" podCreationTimestamp="2025-12-03 12:43:50 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:43:53.892758096 +0000 UTC m=+5024.480999860" watchObservedRunningTime="2025-12-03 12:43:53.895838294 +0000 UTC m=+5024.484080058" Dec 03 12:43:54 crc kubenswrapper[5052]: I1203 12:43:54.119105 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:43:54 crc kubenswrapper[5052]: I1203 12:43:54.206697 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/24ccc68c-26a9-485e-bf26-512dac006b61-scripts\") pod \"24ccc68c-26a9-485e-bf26-512dac006b61\" (UID: \"24ccc68c-26a9-485e-bf26-512dac006b61\") " Dec 03 12:43:54 crc kubenswrapper[5052]: I1203 12:43:54.206747 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/24ccc68c-26a9-485e-bf26-512dac006b61-logs\") pod \"24ccc68c-26a9-485e-bf26-512dac006b61\" (UID: \"24ccc68c-26a9-485e-bf26-512dac006b61\") " Dec 03 12:43:54 crc kubenswrapper[5052]: I1203 12:43:54.206795 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/24ccc68c-26a9-485e-bf26-512dac006b61-httpd-run\") pod \"24ccc68c-26a9-485e-bf26-512dac006b61\" (UID: \"24ccc68c-26a9-485e-bf26-512dac006b61\") " Dec 03 12:43:54 crc kubenswrapper[5052]: I1203 12:43:54.206870 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/24ccc68c-26a9-485e-bf26-512dac006b61-config-data\") pod \"24ccc68c-26a9-485e-bf26-512dac006b61\" (UID: \"24ccc68c-26a9-485e-bf26-512dac006b61\") " Dec 03 12:43:54 crc kubenswrapper[5052]: I1203 12:43:54.206911 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jgr7h\" (UniqueName: \"kubernetes.io/projected/24ccc68c-26a9-485e-bf26-512dac006b61-kube-api-access-jgr7h\") pod \"24ccc68c-26a9-485e-bf26-512dac006b61\" (UID: \"24ccc68c-26a9-485e-bf26-512dac006b61\") " Dec 03 12:43:54 crc kubenswrapper[5052]: I1203 12:43:54.206955 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/24ccc68c-26a9-485e-bf26-512dac006b61-public-tls-certs\") pod \"24ccc68c-26a9-485e-bf26-512dac006b61\" (UID: \"24ccc68c-26a9-485e-bf26-512dac006b61\") " Dec 03 12:43:54 crc kubenswrapper[5052]: I1203 12:43:54.207009 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"glance\" (UniqueName: \"kubernetes.io/local-volume/local-storage12-crc\") pod \"24ccc68c-26a9-485e-bf26-512dac006b61\" (UID: \"24ccc68c-26a9-485e-bf26-512dac006b61\") " Dec 03 12:43:54 crc kubenswrapper[5052]: I1203 12:43:54.207066 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/24ccc68c-26a9-485e-bf26-512dac006b61-combined-ca-bundle\") pod \"24ccc68c-26a9-485e-bf26-512dac006b61\" (UID: \"24ccc68c-26a9-485e-bf26-512dac006b61\") " Dec 03 12:43:54 crc kubenswrapper[5052]: I1203 12:43:54.207750 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/24ccc68c-26a9-485e-bf26-512dac006b61-logs" (OuterVolumeSpecName: "logs") pod "24ccc68c-26a9-485e-bf26-512dac006b61" (UID: "24ccc68c-26a9-485e-bf26-512dac006b61"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:43:54 crc kubenswrapper[5052]: I1203 12:43:54.208022 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/24ccc68c-26a9-485e-bf26-512dac006b61-httpd-run" (OuterVolumeSpecName: "httpd-run") pod "24ccc68c-26a9-485e-bf26-512dac006b61" (UID: "24ccc68c-26a9-485e-bf26-512dac006b61"). InnerVolumeSpecName "httpd-run". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:43:54 crc kubenswrapper[5052]: I1203 12:43:54.215045 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/24ccc68c-26a9-485e-bf26-512dac006b61-kube-api-access-jgr7h" (OuterVolumeSpecName: "kube-api-access-jgr7h") pod "24ccc68c-26a9-485e-bf26-512dac006b61" (UID: "24ccc68c-26a9-485e-bf26-512dac006b61"). InnerVolumeSpecName "kube-api-access-jgr7h". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:43:54 crc kubenswrapper[5052]: I1203 12:43:54.215259 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/24ccc68c-26a9-485e-bf26-512dac006b61-scripts" (OuterVolumeSpecName: "scripts") pod "24ccc68c-26a9-485e-bf26-512dac006b61" (UID: "24ccc68c-26a9-485e-bf26-512dac006b61"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:43:54 crc kubenswrapper[5052]: I1203 12:43:54.225452 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage12-crc" (OuterVolumeSpecName: "glance") pod "24ccc68c-26a9-485e-bf26-512dac006b61" (UID: "24ccc68c-26a9-485e-bf26-512dac006b61"). InnerVolumeSpecName "local-storage12-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 12:43:54 crc kubenswrapper[5052]: I1203 12:43:54.244378 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/24ccc68c-26a9-485e-bf26-512dac006b61-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "24ccc68c-26a9-485e-bf26-512dac006b61" (UID: "24ccc68c-26a9-485e-bf26-512dac006b61"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:43:54 crc kubenswrapper[5052]: I1203 12:43:54.269173 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/24ccc68c-26a9-485e-bf26-512dac006b61-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "24ccc68c-26a9-485e-bf26-512dac006b61" (UID: "24ccc68c-26a9-485e-bf26-512dac006b61"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:43:54 crc kubenswrapper[5052]: I1203 12:43:54.276445 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/24ccc68c-26a9-485e-bf26-512dac006b61-config-data" (OuterVolumeSpecName: "config-data") pod "24ccc68c-26a9-485e-bf26-512dac006b61" (UID: "24ccc68c-26a9-485e-bf26-512dac006b61"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:43:54 crc kubenswrapper[5052]: I1203 12:43:54.310293 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage12-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage12-crc\") on node \"crc\" " Dec 03 12:43:54 crc kubenswrapper[5052]: I1203 12:43:54.310594 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/24ccc68c-26a9-485e-bf26-512dac006b61-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:43:54 crc kubenswrapper[5052]: I1203 12:43:54.310689 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/24ccc68c-26a9-485e-bf26-512dac006b61-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:43:54 crc kubenswrapper[5052]: I1203 12:43:54.310768 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/24ccc68c-26a9-485e-bf26-512dac006b61-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:43:54 crc kubenswrapper[5052]: I1203 12:43:54.310889 5052 reconciler_common.go:293] "Volume detached for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/24ccc68c-26a9-485e-bf26-512dac006b61-httpd-run\") on node \"crc\" DevicePath \"\"" Dec 03 12:43:54 crc kubenswrapper[5052]: I1203 12:43:54.310971 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/24ccc68c-26a9-485e-bf26-512dac006b61-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:43:54 crc kubenswrapper[5052]: I1203 12:43:54.311052 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-jgr7h\" (UniqueName: \"kubernetes.io/projected/24ccc68c-26a9-485e-bf26-512dac006b61-kube-api-access-jgr7h\") on node \"crc\" DevicePath \"\"" Dec 03 12:43:54 crc kubenswrapper[5052]: I1203 12:43:54.311128 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/24ccc68c-26a9-485e-bf26-512dac006b61-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:43:54 crc kubenswrapper[5052]: I1203 12:43:54.330286 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage12-crc" (UniqueName: "kubernetes.io/local-volume/local-storage12-crc") on node "crc" Dec 03 12:43:54 crc kubenswrapper[5052]: I1203 12:43:54.413089 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage12-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage12-crc\") on node \"crc\" DevicePath \"\"" Dec 03 12:43:54 crc kubenswrapper[5052]: I1203 12:43:54.890977 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"24ccc68c-26a9-485e-bf26-512dac006b61","Type":"ContainerDied","Data":"62e7c426fcaf4df9986d8f33d839e1a4e2b3d6f13900f261856dbcb3908cbde1"} Dec 03 12:43:54 crc kubenswrapper[5052]: I1203 12:43:54.891316 5052 scope.go:117] "RemoveContainer" containerID="140826c39a57fa01ca2b271c735d5f9c464eb0c7c58b2f109153dcf2376fff98" Dec 03 12:43:54 crc kubenswrapper[5052]: I1203 12:43:54.891421 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:43:54 crc kubenswrapper[5052]: I1203 12:43:54.895732 5052 generic.go:334] "Generic (PLEG): container finished" podID="fa98aa64-e8ed-442e-90a0-6823aaa88edb" containerID="c3413a78a793e89bd0b3f298b6e5e3d40cfcd493e84ae55bfbc2ea5d5d154265" exitCode=0 Dec 03 12:43:54 crc kubenswrapper[5052]: I1203 12:43:54.896540 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"fa98aa64-e8ed-442e-90a0-6823aaa88edb","Type":"ContainerDied","Data":"c3413a78a793e89bd0b3f298b6e5e3d40cfcd493e84ae55bfbc2ea5d5d154265"} Dec 03 12:43:54 crc kubenswrapper[5052]: I1203 12:43:54.896568 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"fa98aa64-e8ed-442e-90a0-6823aaa88edb","Type":"ContainerDied","Data":"d600424a7463762da5d2af4d43d66a46f63c6fdb6bce76c5527e43e1a9bf2f01"} Dec 03 12:43:54 crc kubenswrapper[5052]: I1203 12:43:54.896579 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d600424a7463762da5d2af4d43d66a46f63c6fdb6bce76c5527e43e1a9bf2f01" Dec 03 12:43:54 crc kubenswrapper[5052]: I1203 12:43:54.958601 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:43:54 crc kubenswrapper[5052]: I1203 12:43:54.976630 5052 scope.go:117] "RemoveContainer" containerID="23f9262d3a7c543b67221300c14860510bcdb1b097fab4de7283f6469298206b" Dec 03 12:43:54 crc kubenswrapper[5052]: I1203 12:43:54.996001 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:43:55 crc kubenswrapper[5052]: I1203 12:43:55.012430 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:43:55 crc kubenswrapper[5052]: I1203 12:43:55.045955 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:43:55 crc kubenswrapper[5052]: E1203 12:43:55.046535 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="24ccc68c-26a9-485e-bf26-512dac006b61" containerName="glance-log" Dec 03 12:43:55 crc kubenswrapper[5052]: I1203 12:43:55.046553 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="24ccc68c-26a9-485e-bf26-512dac006b61" containerName="glance-log" Dec 03 12:43:55 crc kubenswrapper[5052]: E1203 12:43:55.046581 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="fa98aa64-e8ed-442e-90a0-6823aaa88edb" containerName="sg-core" Dec 03 12:43:55 crc kubenswrapper[5052]: I1203 12:43:55.046589 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="fa98aa64-e8ed-442e-90a0-6823aaa88edb" containerName="sg-core" Dec 03 12:43:55 crc kubenswrapper[5052]: E1203 12:43:55.046610 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="fa98aa64-e8ed-442e-90a0-6823aaa88edb" containerName="ceilometer-central-agent" Dec 03 12:43:55 crc kubenswrapper[5052]: I1203 12:43:55.046618 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="fa98aa64-e8ed-442e-90a0-6823aaa88edb" containerName="ceilometer-central-agent" Dec 03 12:43:55 crc kubenswrapper[5052]: E1203 12:43:55.046628 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="fa98aa64-e8ed-442e-90a0-6823aaa88edb" containerName="ceilometer-notification-agent" Dec 03 12:43:55 crc kubenswrapper[5052]: I1203 12:43:55.046636 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="fa98aa64-e8ed-442e-90a0-6823aaa88edb" containerName="ceilometer-notification-agent" Dec 03 12:43:55 crc kubenswrapper[5052]: E1203 12:43:55.046660 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="24ccc68c-26a9-485e-bf26-512dac006b61" containerName="glance-httpd" Dec 03 12:43:55 crc kubenswrapper[5052]: I1203 12:43:55.046668 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="24ccc68c-26a9-485e-bf26-512dac006b61" containerName="glance-httpd" Dec 03 12:43:55 crc kubenswrapper[5052]: E1203 12:43:55.046676 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="fa98aa64-e8ed-442e-90a0-6823aaa88edb" containerName="proxy-httpd" Dec 03 12:43:55 crc kubenswrapper[5052]: I1203 12:43:55.046685 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="fa98aa64-e8ed-442e-90a0-6823aaa88edb" containerName="proxy-httpd" Dec 03 12:43:55 crc kubenswrapper[5052]: I1203 12:43:55.046910 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="fa98aa64-e8ed-442e-90a0-6823aaa88edb" containerName="ceilometer-central-agent" Dec 03 12:43:55 crc kubenswrapper[5052]: I1203 12:43:55.046929 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="24ccc68c-26a9-485e-bf26-512dac006b61" containerName="glance-log" Dec 03 12:43:55 crc kubenswrapper[5052]: I1203 12:43:55.046941 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="24ccc68c-26a9-485e-bf26-512dac006b61" containerName="glance-httpd" Dec 03 12:43:55 crc kubenswrapper[5052]: I1203 12:43:55.046956 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="fa98aa64-e8ed-442e-90a0-6823aaa88edb" containerName="proxy-httpd" Dec 03 12:43:55 crc kubenswrapper[5052]: I1203 12:43:55.046977 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="fa98aa64-e8ed-442e-90a0-6823aaa88edb" containerName="ceilometer-notification-agent" Dec 03 12:43:55 crc kubenswrapper[5052]: I1203 12:43:55.046996 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="fa98aa64-e8ed-442e-90a0-6823aaa88edb" containerName="sg-core" Dec 03 12:43:55 crc kubenswrapper[5052]: I1203 12:43:55.048799 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:43:55 crc kubenswrapper[5052]: I1203 12:43:55.053926 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-glance-default-public-svc" Dec 03 12:43:55 crc kubenswrapper[5052]: I1203 12:43:55.054389 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-default-external-config-data" Dec 03 12:43:55 crc kubenswrapper[5052]: I1203 12:43:55.066740 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:43:55 crc kubenswrapper[5052]: I1203 12:43:55.124302 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/fa98aa64-e8ed-442e-90a0-6823aaa88edb-run-httpd\") pod \"fa98aa64-e8ed-442e-90a0-6823aaa88edb\" (UID: \"fa98aa64-e8ed-442e-90a0-6823aaa88edb\") " Dec 03 12:43:55 crc kubenswrapper[5052]: I1203 12:43:55.124396 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mxg6m\" (UniqueName: \"kubernetes.io/projected/fa98aa64-e8ed-442e-90a0-6823aaa88edb-kube-api-access-mxg6m\") pod \"fa98aa64-e8ed-442e-90a0-6823aaa88edb\" (UID: \"fa98aa64-e8ed-442e-90a0-6823aaa88edb\") " Dec 03 12:43:55 crc kubenswrapper[5052]: I1203 12:43:55.124446 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/fa98aa64-e8ed-442e-90a0-6823aaa88edb-combined-ca-bundle\") pod \"fa98aa64-e8ed-442e-90a0-6823aaa88edb\" (UID: \"fa98aa64-e8ed-442e-90a0-6823aaa88edb\") " Dec 03 12:43:55 crc kubenswrapper[5052]: I1203 12:43:55.124464 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/fa98aa64-e8ed-442e-90a0-6823aaa88edb-sg-core-conf-yaml\") pod \"fa98aa64-e8ed-442e-90a0-6823aaa88edb\" (UID: \"fa98aa64-e8ed-442e-90a0-6823aaa88edb\") " Dec 03 12:43:55 crc kubenswrapper[5052]: I1203 12:43:55.124487 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/fa98aa64-e8ed-442e-90a0-6823aaa88edb-log-httpd\") pod \"fa98aa64-e8ed-442e-90a0-6823aaa88edb\" (UID: \"fa98aa64-e8ed-442e-90a0-6823aaa88edb\") " Dec 03 12:43:55 crc kubenswrapper[5052]: I1203 12:43:55.124620 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/fa98aa64-e8ed-442e-90a0-6823aaa88edb-config-data\") pod \"fa98aa64-e8ed-442e-90a0-6823aaa88edb\" (UID: \"fa98aa64-e8ed-442e-90a0-6823aaa88edb\") " Dec 03 12:43:55 crc kubenswrapper[5052]: I1203 12:43:55.124693 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/fa98aa64-e8ed-442e-90a0-6823aaa88edb-scripts\") pod \"fa98aa64-e8ed-442e-90a0-6823aaa88edb\" (UID: \"fa98aa64-e8ed-442e-90a0-6823aaa88edb\") " Dec 03 12:43:55 crc kubenswrapper[5052]: I1203 12:43:55.125363 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fa98aa64-e8ed-442e-90a0-6823aaa88edb-log-httpd" (OuterVolumeSpecName: "log-httpd") pod "fa98aa64-e8ed-442e-90a0-6823aaa88edb" (UID: "fa98aa64-e8ed-442e-90a0-6823aaa88edb"). InnerVolumeSpecName "log-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:43:55 crc kubenswrapper[5052]: I1203 12:43:55.125361 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fa98aa64-e8ed-442e-90a0-6823aaa88edb-run-httpd" (OuterVolumeSpecName: "run-httpd") pod "fa98aa64-e8ed-442e-90a0-6823aaa88edb" (UID: "fa98aa64-e8ed-442e-90a0-6823aaa88edb"). InnerVolumeSpecName "run-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:43:55 crc kubenswrapper[5052]: I1203 12:43:55.125840 5052 reconciler_common.go:293] "Volume detached for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/fa98aa64-e8ed-442e-90a0-6823aaa88edb-log-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:43:55 crc kubenswrapper[5052]: I1203 12:43:55.125865 5052 reconciler_common.go:293] "Volume detached for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/fa98aa64-e8ed-442e-90a0-6823aaa88edb-run-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:43:55 crc kubenswrapper[5052]: I1203 12:43:55.129892 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fa98aa64-e8ed-442e-90a0-6823aaa88edb-kube-api-access-mxg6m" (OuterVolumeSpecName: "kube-api-access-mxg6m") pod "fa98aa64-e8ed-442e-90a0-6823aaa88edb" (UID: "fa98aa64-e8ed-442e-90a0-6823aaa88edb"). InnerVolumeSpecName "kube-api-access-mxg6m". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:43:55 crc kubenswrapper[5052]: I1203 12:43:55.130455 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fa98aa64-e8ed-442e-90a0-6823aaa88edb-scripts" (OuterVolumeSpecName: "scripts") pod "fa98aa64-e8ed-442e-90a0-6823aaa88edb" (UID: "fa98aa64-e8ed-442e-90a0-6823aaa88edb"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:43:55 crc kubenswrapper[5052]: I1203 12:43:55.173820 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fa98aa64-e8ed-442e-90a0-6823aaa88edb-sg-core-conf-yaml" (OuterVolumeSpecName: "sg-core-conf-yaml") pod "fa98aa64-e8ed-442e-90a0-6823aaa88edb" (UID: "fa98aa64-e8ed-442e-90a0-6823aaa88edb"). InnerVolumeSpecName "sg-core-conf-yaml". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:43:55 crc kubenswrapper[5052]: I1203 12:43:55.217345 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fa98aa64-e8ed-442e-90a0-6823aaa88edb-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "fa98aa64-e8ed-442e-90a0-6823aaa88edb" (UID: "fa98aa64-e8ed-442e-90a0-6823aaa88edb"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:43:55 crc kubenswrapper[5052]: I1203 12:43:55.227683 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/9aea20a7-eaed-43a8-9b52-08c2c0212230-httpd-run\") pod \"glance-default-external-api-0\" (UID: \"9aea20a7-eaed-43a8-9b52-08c2c0212230\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:43:55 crc kubenswrapper[5052]: I1203 12:43:55.227744 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage12-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage12-crc\") pod \"glance-default-external-api-0\" (UID: \"9aea20a7-eaed-43a8-9b52-08c2c0212230\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:43:55 crc kubenswrapper[5052]: I1203 12:43:55.227811 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ql9lm\" (UniqueName: \"kubernetes.io/projected/9aea20a7-eaed-43a8-9b52-08c2c0212230-kube-api-access-ql9lm\") pod \"glance-default-external-api-0\" (UID: \"9aea20a7-eaed-43a8-9b52-08c2c0212230\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:43:55 crc kubenswrapper[5052]: I1203 12:43:55.227859 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/9aea20a7-eaed-43a8-9b52-08c2c0212230-logs\") pod \"glance-default-external-api-0\" (UID: \"9aea20a7-eaed-43a8-9b52-08c2c0212230\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:43:55 crc kubenswrapper[5052]: I1203 12:43:55.227905 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9aea20a7-eaed-43a8-9b52-08c2c0212230-combined-ca-bundle\") pod \"glance-default-external-api-0\" (UID: \"9aea20a7-eaed-43a8-9b52-08c2c0212230\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:43:55 crc kubenswrapper[5052]: I1203 12:43:55.227994 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/9aea20a7-eaed-43a8-9b52-08c2c0212230-public-tls-certs\") pod \"glance-default-external-api-0\" (UID: \"9aea20a7-eaed-43a8-9b52-08c2c0212230\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:43:55 crc kubenswrapper[5052]: I1203 12:43:55.228402 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/9aea20a7-eaed-43a8-9b52-08c2c0212230-config-data\") pod \"glance-default-external-api-0\" (UID: \"9aea20a7-eaed-43a8-9b52-08c2c0212230\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:43:55 crc kubenswrapper[5052]: I1203 12:43:55.228530 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/9aea20a7-eaed-43a8-9b52-08c2c0212230-scripts\") pod \"glance-default-external-api-0\" (UID: \"9aea20a7-eaed-43a8-9b52-08c2c0212230\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:43:55 crc kubenswrapper[5052]: I1203 12:43:55.228710 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-mxg6m\" (UniqueName: \"kubernetes.io/projected/fa98aa64-e8ed-442e-90a0-6823aaa88edb-kube-api-access-mxg6m\") on node \"crc\" DevicePath \"\"" Dec 03 12:43:55 crc kubenswrapper[5052]: I1203 12:43:55.228737 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/fa98aa64-e8ed-442e-90a0-6823aaa88edb-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:43:55 crc kubenswrapper[5052]: I1203 12:43:55.228751 5052 reconciler_common.go:293] "Volume detached for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/fa98aa64-e8ed-442e-90a0-6823aaa88edb-sg-core-conf-yaml\") on node \"crc\" DevicePath \"\"" Dec 03 12:43:55 crc kubenswrapper[5052]: I1203 12:43:55.228799 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/fa98aa64-e8ed-442e-90a0-6823aaa88edb-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:43:55 crc kubenswrapper[5052]: I1203 12:43:55.237667 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fa98aa64-e8ed-442e-90a0-6823aaa88edb-config-data" (OuterVolumeSpecName: "config-data") pod "fa98aa64-e8ed-442e-90a0-6823aaa88edb" (UID: "fa98aa64-e8ed-442e-90a0-6823aaa88edb"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:43:55 crc kubenswrapper[5052]: I1203 12:43:55.330872 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/9aea20a7-eaed-43a8-9b52-08c2c0212230-httpd-run\") pod \"glance-default-external-api-0\" (UID: \"9aea20a7-eaed-43a8-9b52-08c2c0212230\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:43:55 crc kubenswrapper[5052]: I1203 12:43:55.330945 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage12-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage12-crc\") pod \"glance-default-external-api-0\" (UID: \"9aea20a7-eaed-43a8-9b52-08c2c0212230\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:43:55 crc kubenswrapper[5052]: I1203 12:43:55.330982 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-ql9lm\" (UniqueName: \"kubernetes.io/projected/9aea20a7-eaed-43a8-9b52-08c2c0212230-kube-api-access-ql9lm\") pod \"glance-default-external-api-0\" (UID: \"9aea20a7-eaed-43a8-9b52-08c2c0212230\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:43:55 crc kubenswrapper[5052]: I1203 12:43:55.331008 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/9aea20a7-eaed-43a8-9b52-08c2c0212230-logs\") pod \"glance-default-external-api-0\" (UID: \"9aea20a7-eaed-43a8-9b52-08c2c0212230\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:43:55 crc kubenswrapper[5052]: I1203 12:43:55.331052 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9aea20a7-eaed-43a8-9b52-08c2c0212230-combined-ca-bundle\") pod \"glance-default-external-api-0\" (UID: \"9aea20a7-eaed-43a8-9b52-08c2c0212230\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:43:55 crc kubenswrapper[5052]: I1203 12:43:55.331115 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/9aea20a7-eaed-43a8-9b52-08c2c0212230-public-tls-certs\") pod \"glance-default-external-api-0\" (UID: \"9aea20a7-eaed-43a8-9b52-08c2c0212230\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:43:55 crc kubenswrapper[5052]: I1203 12:43:55.331196 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/9aea20a7-eaed-43a8-9b52-08c2c0212230-config-data\") pod \"glance-default-external-api-0\" (UID: \"9aea20a7-eaed-43a8-9b52-08c2c0212230\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:43:55 crc kubenswrapper[5052]: I1203 12:43:55.331242 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/9aea20a7-eaed-43a8-9b52-08c2c0212230-scripts\") pod \"glance-default-external-api-0\" (UID: \"9aea20a7-eaed-43a8-9b52-08c2c0212230\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:43:55 crc kubenswrapper[5052]: I1203 12:43:55.331333 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/fa98aa64-e8ed-442e-90a0-6823aaa88edb-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:43:55 crc kubenswrapper[5052]: I1203 12:43:55.332172 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/9aea20a7-eaed-43a8-9b52-08c2c0212230-logs\") pod \"glance-default-external-api-0\" (UID: \"9aea20a7-eaed-43a8-9b52-08c2c0212230\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:43:55 crc kubenswrapper[5052]: I1203 12:43:55.332435 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/9aea20a7-eaed-43a8-9b52-08c2c0212230-httpd-run\") pod \"glance-default-external-api-0\" (UID: \"9aea20a7-eaed-43a8-9b52-08c2c0212230\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:43:55 crc kubenswrapper[5052]: I1203 12:43:55.332895 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage12-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage12-crc\") pod \"glance-default-external-api-0\" (UID: \"9aea20a7-eaed-43a8-9b52-08c2c0212230\") device mount path \"/mnt/openstack/pv12\"" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:43:55 crc kubenswrapper[5052]: I1203 12:43:55.335945 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/9aea20a7-eaed-43a8-9b52-08c2c0212230-scripts\") pod \"glance-default-external-api-0\" (UID: \"9aea20a7-eaed-43a8-9b52-08c2c0212230\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:43:55 crc kubenswrapper[5052]: I1203 12:43:55.338911 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/9aea20a7-eaed-43a8-9b52-08c2c0212230-public-tls-certs\") pod \"glance-default-external-api-0\" (UID: \"9aea20a7-eaed-43a8-9b52-08c2c0212230\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:43:55 crc kubenswrapper[5052]: I1203 12:43:55.339008 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9aea20a7-eaed-43a8-9b52-08c2c0212230-combined-ca-bundle\") pod \"glance-default-external-api-0\" (UID: \"9aea20a7-eaed-43a8-9b52-08c2c0212230\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:43:55 crc kubenswrapper[5052]: I1203 12:43:55.342304 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/9aea20a7-eaed-43a8-9b52-08c2c0212230-config-data\") pod \"glance-default-external-api-0\" (UID: \"9aea20a7-eaed-43a8-9b52-08c2c0212230\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:43:55 crc kubenswrapper[5052]: I1203 12:43:55.355184 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-ql9lm\" (UniqueName: \"kubernetes.io/projected/9aea20a7-eaed-43a8-9b52-08c2c0212230-kube-api-access-ql9lm\") pod \"glance-default-external-api-0\" (UID: \"9aea20a7-eaed-43a8-9b52-08c2c0212230\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:43:55 crc kubenswrapper[5052]: I1203 12:43:55.364887 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage12-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage12-crc\") pod \"glance-default-external-api-0\" (UID: \"9aea20a7-eaed-43a8-9b52-08c2c0212230\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:43:55 crc kubenswrapper[5052]: I1203 12:43:55.378436 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:43:55 crc kubenswrapper[5052]: I1203 12:43:55.685013 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="24ccc68c-26a9-485e-bf26-512dac006b61" path="/var/lib/kubelet/pods/24ccc68c-26a9-485e-bf26-512dac006b61/volumes" Dec 03 12:43:55 crc kubenswrapper[5052]: I1203 12:43:55.837968 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:43:55 crc kubenswrapper[5052]: W1203 12:43:55.840521 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9aea20a7_eaed_43a8_9b52_08c2c0212230.slice/crio-19797ca6bcaa0d1b25af20fbf64d800512d2a5454279e6708f21c0ca08b2d7dc WatchSource:0}: Error finding container 19797ca6bcaa0d1b25af20fbf64d800512d2a5454279e6708f21c0ca08b2d7dc: Status 404 returned error can't find the container with id 19797ca6bcaa0d1b25af20fbf64d800512d2a5454279e6708f21c0ca08b2d7dc Dec 03 12:43:55 crc kubenswrapper[5052]: I1203 12:43:55.924181 5052 generic.go:334] "Generic (PLEG): container finished" podID="9fd7aa9d-4ad5-4058-a784-6814fbd76852" containerID="8af40cfb10f5db16ec28ea25c0561fa79c3cc14b977ccb702d77cd50dcbe5974" exitCode=0 Dec 03 12:43:55 crc kubenswrapper[5052]: I1203 12:43:55.924261 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-ts4vh" event={"ID":"9fd7aa9d-4ad5-4058-a784-6814fbd76852","Type":"ContainerDied","Data":"8af40cfb10f5db16ec28ea25c0561fa79c3cc14b977ccb702d77cd50dcbe5974"} Dec 03 12:43:55 crc kubenswrapper[5052]: I1203 12:43:55.927055 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"9aea20a7-eaed-43a8-9b52-08c2c0212230","Type":"ContainerStarted","Data":"19797ca6bcaa0d1b25af20fbf64d800512d2a5454279e6708f21c0ca08b2d7dc"} Dec 03 12:43:55 crc kubenswrapper[5052]: I1203 12:43:55.928474 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:43:55 crc kubenswrapper[5052]: I1203 12:43:55.968838 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:43:56 crc kubenswrapper[5052]: I1203 12:43:56.032757 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:43:56 crc kubenswrapper[5052]: I1203 12:43:56.048370 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:43:56 crc kubenswrapper[5052]: I1203 12:43:56.050786 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:43:56 crc kubenswrapper[5052]: I1203 12:43:56.057475 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-scripts" Dec 03 12:43:56 crc kubenswrapper[5052]: I1203 12:43:56.057476 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-config-data" Dec 03 12:43:56 crc kubenswrapper[5052]: I1203 12:43:56.061070 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:43:56 crc kubenswrapper[5052]: I1203 12:43:56.155274 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/68e23bb4-c34c-458a-a02e-dfc620082df6-config-data\") pod \"ceilometer-0\" (UID: \"68e23bb4-c34c-458a-a02e-dfc620082df6\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:43:56 crc kubenswrapper[5052]: I1203 12:43:56.155333 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/68e23bb4-c34c-458a-a02e-dfc620082df6-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"68e23bb4-c34c-458a-a02e-dfc620082df6\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:43:56 crc kubenswrapper[5052]: I1203 12:43:56.155368 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/68e23bb4-c34c-458a-a02e-dfc620082df6-log-httpd\") pod \"ceilometer-0\" (UID: \"68e23bb4-c34c-458a-a02e-dfc620082df6\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:43:56 crc kubenswrapper[5052]: I1203 12:43:56.155398 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/68e23bb4-c34c-458a-a02e-dfc620082df6-scripts\") pod \"ceilometer-0\" (UID: \"68e23bb4-c34c-458a-a02e-dfc620082df6\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:43:56 crc kubenswrapper[5052]: I1203 12:43:56.155431 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/68e23bb4-c34c-458a-a02e-dfc620082df6-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"68e23bb4-c34c-458a-a02e-dfc620082df6\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:43:56 crc kubenswrapper[5052]: I1203 12:43:56.155467 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wvkbt\" (UniqueName: \"kubernetes.io/projected/68e23bb4-c34c-458a-a02e-dfc620082df6-kube-api-access-wvkbt\") pod \"ceilometer-0\" (UID: \"68e23bb4-c34c-458a-a02e-dfc620082df6\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:43:56 crc kubenswrapper[5052]: I1203 12:43:56.155551 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/68e23bb4-c34c-458a-a02e-dfc620082df6-run-httpd\") pod \"ceilometer-0\" (UID: \"68e23bb4-c34c-458a-a02e-dfc620082df6\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:43:56 crc kubenswrapper[5052]: I1203 12:43:56.257431 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-wvkbt\" (UniqueName: \"kubernetes.io/projected/68e23bb4-c34c-458a-a02e-dfc620082df6-kube-api-access-wvkbt\") pod \"ceilometer-0\" (UID: \"68e23bb4-c34c-458a-a02e-dfc620082df6\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:43:56 crc kubenswrapper[5052]: I1203 12:43:56.257537 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/68e23bb4-c34c-458a-a02e-dfc620082df6-run-httpd\") pod \"ceilometer-0\" (UID: \"68e23bb4-c34c-458a-a02e-dfc620082df6\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:43:56 crc kubenswrapper[5052]: I1203 12:43:56.257611 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/68e23bb4-c34c-458a-a02e-dfc620082df6-config-data\") pod \"ceilometer-0\" (UID: \"68e23bb4-c34c-458a-a02e-dfc620082df6\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:43:56 crc kubenswrapper[5052]: I1203 12:43:56.257644 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/68e23bb4-c34c-458a-a02e-dfc620082df6-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"68e23bb4-c34c-458a-a02e-dfc620082df6\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:43:56 crc kubenswrapper[5052]: I1203 12:43:56.257673 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/68e23bb4-c34c-458a-a02e-dfc620082df6-log-httpd\") pod \"ceilometer-0\" (UID: \"68e23bb4-c34c-458a-a02e-dfc620082df6\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:43:56 crc kubenswrapper[5052]: I1203 12:43:56.257699 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/68e23bb4-c34c-458a-a02e-dfc620082df6-scripts\") pod \"ceilometer-0\" (UID: \"68e23bb4-c34c-458a-a02e-dfc620082df6\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:43:56 crc kubenswrapper[5052]: I1203 12:43:56.257736 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/68e23bb4-c34c-458a-a02e-dfc620082df6-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"68e23bb4-c34c-458a-a02e-dfc620082df6\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:43:56 crc kubenswrapper[5052]: I1203 12:43:56.258239 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/68e23bb4-c34c-458a-a02e-dfc620082df6-run-httpd\") pod \"ceilometer-0\" (UID: \"68e23bb4-c34c-458a-a02e-dfc620082df6\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:43:56 crc kubenswrapper[5052]: I1203 12:43:56.258617 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/68e23bb4-c34c-458a-a02e-dfc620082df6-log-httpd\") pod \"ceilometer-0\" (UID: \"68e23bb4-c34c-458a-a02e-dfc620082df6\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:43:56 crc kubenswrapper[5052]: I1203 12:43:56.265383 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/68e23bb4-c34c-458a-a02e-dfc620082df6-config-data\") pod \"ceilometer-0\" (UID: \"68e23bb4-c34c-458a-a02e-dfc620082df6\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:43:56 crc kubenswrapper[5052]: I1203 12:43:56.266507 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/68e23bb4-c34c-458a-a02e-dfc620082df6-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"68e23bb4-c34c-458a-a02e-dfc620082df6\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:43:56 crc kubenswrapper[5052]: I1203 12:43:56.275817 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/68e23bb4-c34c-458a-a02e-dfc620082df6-scripts\") pod \"ceilometer-0\" (UID: \"68e23bb4-c34c-458a-a02e-dfc620082df6\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:43:56 crc kubenswrapper[5052]: I1203 12:43:56.276954 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-wvkbt\" (UniqueName: \"kubernetes.io/projected/68e23bb4-c34c-458a-a02e-dfc620082df6-kube-api-access-wvkbt\") pod \"ceilometer-0\" (UID: \"68e23bb4-c34c-458a-a02e-dfc620082df6\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:43:56 crc kubenswrapper[5052]: I1203 12:43:56.284137 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/68e23bb4-c34c-458a-a02e-dfc620082df6-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"68e23bb4-c34c-458a-a02e-dfc620082df6\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:43:56 crc kubenswrapper[5052]: I1203 12:43:56.375795 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:43:56 crc kubenswrapper[5052]: I1203 12:43:56.634082 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-marketplace/community-operators-gt2q8" Dec 03 12:43:56 crc kubenswrapper[5052]: I1203 12:43:56.704761 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-marketplace/community-operators-gt2q8" Dec 03 12:43:56 crc kubenswrapper[5052]: I1203 12:43:56.827495 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:43:56 crc kubenswrapper[5052]: I1203 12:43:56.868331 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openshift-marketplace/community-operators-gt2q8"] Dec 03 12:43:56 crc kubenswrapper[5052]: I1203 12:43:56.946016 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"9aea20a7-eaed-43a8-9b52-08c2c0212230","Type":"ContainerStarted","Data":"c5ed1d5bcb2e814a4e6ff6889daaa85dec6c18fe852411af7cfacd20fd4d3320"} Dec 03 12:43:56 crc kubenswrapper[5052]: I1203 12:43:56.948664 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"68e23bb4-c34c-458a-a02e-dfc620082df6","Type":"ContainerStarted","Data":"1b2fa304d594b0349c950c74661c8bc88c5f959d923306417bf60c8f804c080f"} Dec 03 12:43:57 crc kubenswrapper[5052]: I1203 12:43:57.351949 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-ts4vh" Dec 03 12:43:57 crc kubenswrapper[5052]: I1203 12:43:57.484810 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vkmp6\" (UniqueName: \"kubernetes.io/projected/9fd7aa9d-4ad5-4058-a784-6814fbd76852-kube-api-access-vkmp6\") pod \"9fd7aa9d-4ad5-4058-a784-6814fbd76852\" (UID: \"9fd7aa9d-4ad5-4058-a784-6814fbd76852\") " Dec 03 12:43:57 crc kubenswrapper[5052]: I1203 12:43:57.484851 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/9fd7aa9d-4ad5-4058-a784-6814fbd76852-config-data\") pod \"9fd7aa9d-4ad5-4058-a784-6814fbd76852\" (UID: \"9fd7aa9d-4ad5-4058-a784-6814fbd76852\") " Dec 03 12:43:57 crc kubenswrapper[5052]: I1203 12:43:57.484948 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/9fd7aa9d-4ad5-4058-a784-6814fbd76852-scripts\") pod \"9fd7aa9d-4ad5-4058-a784-6814fbd76852\" (UID: \"9fd7aa9d-4ad5-4058-a784-6814fbd76852\") " Dec 03 12:43:57 crc kubenswrapper[5052]: I1203 12:43:57.485074 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9fd7aa9d-4ad5-4058-a784-6814fbd76852-combined-ca-bundle\") pod \"9fd7aa9d-4ad5-4058-a784-6814fbd76852\" (UID: \"9fd7aa9d-4ad5-4058-a784-6814fbd76852\") " Dec 03 12:43:57 crc kubenswrapper[5052]: I1203 12:43:57.490613 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9fd7aa9d-4ad5-4058-a784-6814fbd76852-scripts" (OuterVolumeSpecName: "scripts") pod "9fd7aa9d-4ad5-4058-a784-6814fbd76852" (UID: "9fd7aa9d-4ad5-4058-a784-6814fbd76852"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:43:57 crc kubenswrapper[5052]: I1203 12:43:57.491352 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9fd7aa9d-4ad5-4058-a784-6814fbd76852-kube-api-access-vkmp6" (OuterVolumeSpecName: "kube-api-access-vkmp6") pod "9fd7aa9d-4ad5-4058-a784-6814fbd76852" (UID: "9fd7aa9d-4ad5-4058-a784-6814fbd76852"). InnerVolumeSpecName "kube-api-access-vkmp6". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:43:57 crc kubenswrapper[5052]: I1203 12:43:57.515306 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9fd7aa9d-4ad5-4058-a784-6814fbd76852-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "9fd7aa9d-4ad5-4058-a784-6814fbd76852" (UID: "9fd7aa9d-4ad5-4058-a784-6814fbd76852"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:43:57 crc kubenswrapper[5052]: I1203 12:43:57.533766 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9fd7aa9d-4ad5-4058-a784-6814fbd76852-config-data" (OuterVolumeSpecName: "config-data") pod "9fd7aa9d-4ad5-4058-a784-6814fbd76852" (UID: "9fd7aa9d-4ad5-4058-a784-6814fbd76852"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:43:57 crc kubenswrapper[5052]: I1203 12:43:57.589138 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/9fd7aa9d-4ad5-4058-a784-6814fbd76852-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:43:57 crc kubenswrapper[5052]: I1203 12:43:57.589176 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9fd7aa9d-4ad5-4058-a784-6814fbd76852-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:43:57 crc kubenswrapper[5052]: I1203 12:43:57.589191 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-vkmp6\" (UniqueName: \"kubernetes.io/projected/9fd7aa9d-4ad5-4058-a784-6814fbd76852-kube-api-access-vkmp6\") on node \"crc\" DevicePath \"\"" Dec 03 12:43:57 crc kubenswrapper[5052]: I1203 12:43:57.589200 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/9fd7aa9d-4ad5-4058-a784-6814fbd76852-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:43:57 crc kubenswrapper[5052]: I1203 12:43:57.693586 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fa98aa64-e8ed-442e-90a0-6823aaa88edb" path="/var/lib/kubelet/pods/fa98aa64-e8ed-442e-90a0-6823aaa88edb/volumes" Dec 03 12:43:57 crc kubenswrapper[5052]: I1203 12:43:57.960458 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"9aea20a7-eaed-43a8-9b52-08c2c0212230","Type":"ContainerStarted","Data":"3242b025201bb8378bb8f5f363d45551adfc8da4587f92d211c75e0e901ef11a"} Dec 03 12:43:57 crc kubenswrapper[5052]: I1203 12:43:57.962609 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-ts4vh" event={"ID":"9fd7aa9d-4ad5-4058-a784-6814fbd76852","Type":"ContainerDied","Data":"ff63c7d5be65035de7226b4a4eca9c62944cf4775fb258fb4031eaf6fb6e4005"} Dec 03 12:43:57 crc kubenswrapper[5052]: I1203 12:43:57.962642 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ff63c7d5be65035de7226b4a4eca9c62944cf4775fb258fb4031eaf6fb6e4005" Dec 03 12:43:57 crc kubenswrapper[5052]: I1203 12:43:57.962639 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-ts4vh" Dec 03 12:43:57 crc kubenswrapper[5052]: I1203 12:43:57.964629 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"68e23bb4-c34c-458a-a02e-dfc620082df6","Type":"ContainerStarted","Data":"3a367bcc668e3d22b59106456e5a11faf700cbaf32b046598a5f08d9da45582a"} Dec 03 12:43:57 crc kubenswrapper[5052]: I1203 12:43:57.964755 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openshift-marketplace/community-operators-gt2q8" podUID="3f17a210-d54b-4753-91c1-2d9b97565b4e" containerName="registry-server" containerID="cri-o://15082f95a5305cb2b4114d7cb325f148e1d42e449b1a4dd29fe625f2e9572260" gracePeriod=2 Dec 03 12:43:57 crc kubenswrapper[5052]: I1203 12:43:57.991718 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/glance-default-external-api-0" podStartSLOduration=3.9917033589999997 podStartE2EDuration="3.991703359s" podCreationTimestamp="2025-12-03 12:43:54 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:43:57.991101841 +0000 UTC m=+5028.579343605" watchObservedRunningTime="2025-12-03 12:43:57.991703359 +0000 UTC m=+5028.579945123" Dec 03 12:43:58 crc kubenswrapper[5052]: I1203 12:43:58.026893 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-0"] Dec 03 12:43:58 crc kubenswrapper[5052]: E1203 12:43:58.027275 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="9fd7aa9d-4ad5-4058-a784-6814fbd76852" containerName="nova-cell0-conductor-db-sync" Dec 03 12:43:58 crc kubenswrapper[5052]: I1203 12:43:58.027290 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="9fd7aa9d-4ad5-4058-a784-6814fbd76852" containerName="nova-cell0-conductor-db-sync" Dec 03 12:43:58 crc kubenswrapper[5052]: I1203 12:43:58.027450 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="9fd7aa9d-4ad5-4058-a784-6814fbd76852" containerName="nova-cell0-conductor-db-sync" Dec 03 12:43:58 crc kubenswrapper[5052]: I1203 12:43:58.028036 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:43:58 crc kubenswrapper[5052]: I1203 12:43:58.030236 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-nova-dockercfg-tj8z2" Dec 03 12:43:58 crc kubenswrapper[5052]: I1203 12:43:58.043095 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell0-conductor-config-data" Dec 03 12:43:58 crc kubenswrapper[5052]: I1203 12:43:58.047983 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-0"] Dec 03 12:43:58 crc kubenswrapper[5052]: I1203 12:43:58.201112 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/307199d5-acf0-4ede-b755-ed6bc25eb978-combined-ca-bundle\") pod \"nova-cell0-conductor-0\" (UID: \"307199d5-acf0-4ede-b755-ed6bc25eb978\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:43:58 crc kubenswrapper[5052]: I1203 12:43:58.201284 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cqzpt\" (UniqueName: \"kubernetes.io/projected/307199d5-acf0-4ede-b755-ed6bc25eb978-kube-api-access-cqzpt\") pod \"nova-cell0-conductor-0\" (UID: \"307199d5-acf0-4ede-b755-ed6bc25eb978\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:43:58 crc kubenswrapper[5052]: I1203 12:43:58.201523 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/307199d5-acf0-4ede-b755-ed6bc25eb978-config-data\") pod \"nova-cell0-conductor-0\" (UID: \"307199d5-acf0-4ede-b755-ed6bc25eb978\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:43:58 crc kubenswrapper[5052]: I1203 12:43:58.302904 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/307199d5-acf0-4ede-b755-ed6bc25eb978-combined-ca-bundle\") pod \"nova-cell0-conductor-0\" (UID: \"307199d5-acf0-4ede-b755-ed6bc25eb978\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:43:58 crc kubenswrapper[5052]: I1203 12:43:58.302998 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-cqzpt\" (UniqueName: \"kubernetes.io/projected/307199d5-acf0-4ede-b755-ed6bc25eb978-kube-api-access-cqzpt\") pod \"nova-cell0-conductor-0\" (UID: \"307199d5-acf0-4ede-b755-ed6bc25eb978\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:43:58 crc kubenswrapper[5052]: I1203 12:43:58.303079 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/307199d5-acf0-4ede-b755-ed6bc25eb978-config-data\") pod \"nova-cell0-conductor-0\" (UID: \"307199d5-acf0-4ede-b755-ed6bc25eb978\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:43:58 crc kubenswrapper[5052]: I1203 12:43:58.307858 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/307199d5-acf0-4ede-b755-ed6bc25eb978-config-data\") pod \"nova-cell0-conductor-0\" (UID: \"307199d5-acf0-4ede-b755-ed6bc25eb978\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:43:58 crc kubenswrapper[5052]: I1203 12:43:58.307925 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/307199d5-acf0-4ede-b755-ed6bc25eb978-combined-ca-bundle\") pod \"nova-cell0-conductor-0\" (UID: \"307199d5-acf0-4ede-b755-ed6bc25eb978\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:43:58 crc kubenswrapper[5052]: I1203 12:43:58.327610 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-cqzpt\" (UniqueName: \"kubernetes.io/projected/307199d5-acf0-4ede-b755-ed6bc25eb978-kube-api-access-cqzpt\") pod \"nova-cell0-conductor-0\" (UID: \"307199d5-acf0-4ede-b755-ed6bc25eb978\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:43:58 crc kubenswrapper[5052]: I1203 12:43:58.346597 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:43:58 crc kubenswrapper[5052]: I1203 12:43:58.904897 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-0"] Dec 03 12:43:58 crc kubenswrapper[5052]: I1203 12:43:58.982801 5052 generic.go:334] "Generic (PLEG): container finished" podID="3f17a210-d54b-4753-91c1-2d9b97565b4e" containerID="15082f95a5305cb2b4114d7cb325f148e1d42e449b1a4dd29fe625f2e9572260" exitCode=0 Dec 03 12:43:58 crc kubenswrapper[5052]: I1203 12:43:58.982863 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/community-operators-gt2q8" event={"ID":"3f17a210-d54b-4753-91c1-2d9b97565b4e","Type":"ContainerDied","Data":"15082f95a5305cb2b4114d7cb325f148e1d42e449b1a4dd29fe625f2e9572260"} Dec 03 12:43:59 crc kubenswrapper[5052]: I1203 12:43:59.084461 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/community-operators-gt2q8" Dec 03 12:43:59 crc kubenswrapper[5052]: I1203 12:43:59.109042 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-0"] Dec 03 12:43:59 crc kubenswrapper[5052]: W1203 12:43:59.110944 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod307199d5_acf0_4ede_b755_ed6bc25eb978.slice/crio-23ca5c590a6929a4526ebf4c967682a631cdb1c6485699801b6bcd4e218e6b1f WatchSource:0}: Error finding container 23ca5c590a6929a4526ebf4c967682a631cdb1c6485699801b6bcd4e218e6b1f: Status 404 returned error can't find the container with id 23ca5c590a6929a4526ebf4c967682a631cdb1c6485699801b6bcd4e218e6b1f Dec 03 12:43:59 crc kubenswrapper[5052]: I1203 12:43:59.117950 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/3f17a210-d54b-4753-91c1-2d9b97565b4e-utilities\") pod \"3f17a210-d54b-4753-91c1-2d9b97565b4e\" (UID: \"3f17a210-d54b-4753-91c1-2d9b97565b4e\") " Dec 03 12:43:59 crc kubenswrapper[5052]: I1203 12:43:59.118216 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-98x49\" (UniqueName: \"kubernetes.io/projected/3f17a210-d54b-4753-91c1-2d9b97565b4e-kube-api-access-98x49\") pod \"3f17a210-d54b-4753-91c1-2d9b97565b4e\" (UID: \"3f17a210-d54b-4753-91c1-2d9b97565b4e\") " Dec 03 12:43:59 crc kubenswrapper[5052]: I1203 12:43:59.118252 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/3f17a210-d54b-4753-91c1-2d9b97565b4e-catalog-content\") pod \"3f17a210-d54b-4753-91c1-2d9b97565b4e\" (UID: \"3f17a210-d54b-4753-91c1-2d9b97565b4e\") " Dec 03 12:43:59 crc kubenswrapper[5052]: I1203 12:43:59.127388 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3f17a210-d54b-4753-91c1-2d9b97565b4e-utilities" (OuterVolumeSpecName: "utilities") pod "3f17a210-d54b-4753-91c1-2d9b97565b4e" (UID: "3f17a210-d54b-4753-91c1-2d9b97565b4e"). InnerVolumeSpecName "utilities". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:43:59 crc kubenswrapper[5052]: I1203 12:43:59.137034 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3f17a210-d54b-4753-91c1-2d9b97565b4e-kube-api-access-98x49" (OuterVolumeSpecName: "kube-api-access-98x49") pod "3f17a210-d54b-4753-91c1-2d9b97565b4e" (UID: "3f17a210-d54b-4753-91c1-2d9b97565b4e"). InnerVolumeSpecName "kube-api-access-98x49". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:43:59 crc kubenswrapper[5052]: I1203 12:43:59.225000 5052 reconciler_common.go:293] "Volume detached for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/3f17a210-d54b-4753-91c1-2d9b97565b4e-utilities\") on node \"crc\" DevicePath \"\"" Dec 03 12:43:59 crc kubenswrapper[5052]: I1203 12:43:59.225045 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-98x49\" (UniqueName: \"kubernetes.io/projected/3f17a210-d54b-4753-91c1-2d9b97565b4e-kube-api-access-98x49\") on node \"crc\" DevicePath \"\"" Dec 03 12:43:59 crc kubenswrapper[5052]: I1203 12:43:59.246315 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3f17a210-d54b-4753-91c1-2d9b97565b4e-catalog-content" (OuterVolumeSpecName: "catalog-content") pod "3f17a210-d54b-4753-91c1-2d9b97565b4e" (UID: "3f17a210-d54b-4753-91c1-2d9b97565b4e"). InnerVolumeSpecName "catalog-content". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:43:59 crc kubenswrapper[5052]: I1203 12:43:59.326386 5052 reconciler_common.go:293] "Volume detached for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/3f17a210-d54b-4753-91c1-2d9b97565b4e-catalog-content\") on node \"crc\" DevicePath \"\"" Dec 03 12:43:59 crc kubenswrapper[5052]: I1203 12:43:59.995918 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"68e23bb4-c34c-458a-a02e-dfc620082df6","Type":"ContainerStarted","Data":"12e601ccb583e45a27f9f873175b83c0b6728007392fa552bc07b64e7d4d0b16"} Dec 03 12:43:59 crc kubenswrapper[5052]: I1203 12:43:59.998810 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/community-operators-gt2q8" event={"ID":"3f17a210-d54b-4753-91c1-2d9b97565b4e","Type":"ContainerDied","Data":"761a82ce4ed15956c0cb9b9d1cff360c5b1ad3d4ec66d725b90bc7d0b4c95d41"} Dec 03 12:43:59 crc kubenswrapper[5052]: I1203 12:43:59.998865 5052 scope.go:117] "RemoveContainer" containerID="15082f95a5305cb2b4114d7cb325f148e1d42e449b1a4dd29fe625f2e9572260" Dec 03 12:43:59 crc kubenswrapper[5052]: I1203 12:43:59.999076 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/community-operators-gt2q8" Dec 03 12:44:00 crc kubenswrapper[5052]: I1203 12:44:00.004544 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-conductor-0" event={"ID":"307199d5-acf0-4ede-b755-ed6bc25eb978","Type":"ContainerStarted","Data":"23ca5c590a6929a4526ebf4c967682a631cdb1c6485699801b6bcd4e218e6b1f"} Dec 03 12:44:00 crc kubenswrapper[5052]: I1203 12:44:00.062643 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openshift-marketplace/community-operators-gt2q8"] Dec 03 12:44:00 crc kubenswrapper[5052]: I1203 12:44:00.069981 5052 scope.go:117] "RemoveContainer" containerID="7a6e9317cc9c73ddf62181ea0be41ce3e631113e804f92ebef91af52e3c9a7be" Dec 03 12:44:00 crc kubenswrapper[5052]: I1203 12:44:00.075653 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openshift-marketplace/community-operators-gt2q8"] Dec 03 12:44:00 crc kubenswrapper[5052]: I1203 12:44:00.343690 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:44:00 crc kubenswrapper[5052]: I1203 12:44:00.363609 5052 scope.go:117] "RemoveContainer" containerID="2d5d078d4787f4a22f1ee4624f7bb3f93992795d0ecf33df45f1d2d6041f5a7d" Dec 03 12:44:01 crc kubenswrapper[5052]: I1203 12:44:01.020930 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"68e23bb4-c34c-458a-a02e-dfc620082df6","Type":"ContainerStarted","Data":"8155928eaaa4da62211aa59eb0ec0eded75cda3bfa39b972216b9921042e0dc0"} Dec 03 12:44:01 crc kubenswrapper[5052]: I1203 12:44:01.026404 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-conductor-0" event={"ID":"307199d5-acf0-4ede-b755-ed6bc25eb978","Type":"ContainerStarted","Data":"8ff041a03ee1701a0052612991d867cafab4a1723240a7109f4b7132da5cae68"} Dec 03 12:44:01 crc kubenswrapper[5052]: I1203 12:44:01.026749 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-cell0-conductor-0" podUID="307199d5-acf0-4ede-b755-ed6bc25eb978" containerName="nova-cell0-conductor-conductor" containerID="cri-o://8ff041a03ee1701a0052612991d867cafab4a1723240a7109f4b7132da5cae68" gracePeriod=30 Dec 03 12:44:01 crc kubenswrapper[5052]: I1203 12:44:01.051301 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-cell0-conductor-0" podStartSLOduration=3.051279856 podStartE2EDuration="3.051279856s" podCreationTimestamp="2025-12-03 12:43:58 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:44:01.048916999 +0000 UTC m=+5031.637158783" watchObservedRunningTime="2025-12-03 12:44:01.051279856 +0000 UTC m=+5031.639521620" Dec 03 12:44:01 crc kubenswrapper[5052]: I1203 12:44:01.251578 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:44:01 crc kubenswrapper[5052]: I1203 12:44:01.251640 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:44:01 crc kubenswrapper[5052]: I1203 12:44:01.685164 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3f17a210-d54b-4753-91c1-2d9b97565b4e" path="/var/lib/kubelet/pods/3f17a210-d54b-4753-91c1-2d9b97565b4e/volumes" Dec 03 12:44:01 crc kubenswrapper[5052]: I1203 12:44:01.932288 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:44:01 crc kubenswrapper[5052]: I1203 12:44:01.932392 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:44:02 crc kubenswrapper[5052]: I1203 12:44:02.035937 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:44:02 crc kubenswrapper[5052]: I1203 12:44:02.035985 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:44:02 crc kubenswrapper[5052]: I1203 12:44:02.681025 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openshift-marketplace/certified-operators-k2f9v"] Dec 03 12:44:02 crc kubenswrapper[5052]: E1203 12:44:02.681656 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="3f17a210-d54b-4753-91c1-2d9b97565b4e" containerName="registry-server" Dec 03 12:44:02 crc kubenswrapper[5052]: I1203 12:44:02.681681 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="3f17a210-d54b-4753-91c1-2d9b97565b4e" containerName="registry-server" Dec 03 12:44:02 crc kubenswrapper[5052]: E1203 12:44:02.681705 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="3f17a210-d54b-4753-91c1-2d9b97565b4e" containerName="extract-utilities" Dec 03 12:44:02 crc kubenswrapper[5052]: I1203 12:44:02.681714 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="3f17a210-d54b-4753-91c1-2d9b97565b4e" containerName="extract-utilities" Dec 03 12:44:02 crc kubenswrapper[5052]: E1203 12:44:02.681746 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="3f17a210-d54b-4753-91c1-2d9b97565b4e" containerName="extract-content" Dec 03 12:44:02 crc kubenswrapper[5052]: I1203 12:44:02.681758 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="3f17a210-d54b-4753-91c1-2d9b97565b4e" containerName="extract-content" Dec 03 12:44:02 crc kubenswrapper[5052]: I1203 12:44:02.681972 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="3f17a210-d54b-4753-91c1-2d9b97565b4e" containerName="registry-server" Dec 03 12:44:02 crc kubenswrapper[5052]: I1203 12:44:02.685029 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/certified-operators-k2f9v" Dec 03 12:44:02 crc kubenswrapper[5052]: I1203 12:44:02.702017 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/certified-operators-k2f9v"] Dec 03 12:44:02 crc kubenswrapper[5052]: I1203 12:44:02.791608 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/21de3002-be4f-4497-805c-7ef8205f03ed-utilities\") pod \"certified-operators-k2f9v\" (UID: \"21de3002-be4f-4497-805c-7ef8205f03ed\") " pod="openshift-marketplace/certified-operators-k2f9v" Dec 03 12:44:02 crc kubenswrapper[5052]: I1203 12:44:02.791721 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v2p79\" (UniqueName: \"kubernetes.io/projected/21de3002-be4f-4497-805c-7ef8205f03ed-kube-api-access-v2p79\") pod \"certified-operators-k2f9v\" (UID: \"21de3002-be4f-4497-805c-7ef8205f03ed\") " pod="openshift-marketplace/certified-operators-k2f9v" Dec 03 12:44:02 crc kubenswrapper[5052]: I1203 12:44:02.791903 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/21de3002-be4f-4497-805c-7ef8205f03ed-catalog-content\") pod \"certified-operators-k2f9v\" (UID: \"21de3002-be4f-4497-805c-7ef8205f03ed\") " pod="openshift-marketplace/certified-operators-k2f9v" Dec 03 12:44:02 crc kubenswrapper[5052]: I1203 12:44:02.893635 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/21de3002-be4f-4497-805c-7ef8205f03ed-catalog-content\") pod \"certified-operators-k2f9v\" (UID: \"21de3002-be4f-4497-805c-7ef8205f03ed\") " pod="openshift-marketplace/certified-operators-k2f9v" Dec 03 12:44:02 crc kubenswrapper[5052]: I1203 12:44:02.893737 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/21de3002-be4f-4497-805c-7ef8205f03ed-utilities\") pod \"certified-operators-k2f9v\" (UID: \"21de3002-be4f-4497-805c-7ef8205f03ed\") " pod="openshift-marketplace/certified-operators-k2f9v" Dec 03 12:44:02 crc kubenswrapper[5052]: I1203 12:44:02.893802 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-v2p79\" (UniqueName: \"kubernetes.io/projected/21de3002-be4f-4497-805c-7ef8205f03ed-kube-api-access-v2p79\") pod \"certified-operators-k2f9v\" (UID: \"21de3002-be4f-4497-805c-7ef8205f03ed\") " pod="openshift-marketplace/certified-operators-k2f9v" Dec 03 12:44:02 crc kubenswrapper[5052]: I1203 12:44:02.894226 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/21de3002-be4f-4497-805c-7ef8205f03ed-catalog-content\") pod \"certified-operators-k2f9v\" (UID: \"21de3002-be4f-4497-805c-7ef8205f03ed\") " pod="openshift-marketplace/certified-operators-k2f9v" Dec 03 12:44:02 crc kubenswrapper[5052]: I1203 12:44:02.894382 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/21de3002-be4f-4497-805c-7ef8205f03ed-utilities\") pod \"certified-operators-k2f9v\" (UID: \"21de3002-be4f-4497-805c-7ef8205f03ed\") " pod="openshift-marketplace/certified-operators-k2f9v" Dec 03 12:44:02 crc kubenswrapper[5052]: I1203 12:44:02.923023 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-v2p79\" (UniqueName: \"kubernetes.io/projected/21de3002-be4f-4497-805c-7ef8205f03ed-kube-api-access-v2p79\") pod \"certified-operators-k2f9v\" (UID: \"21de3002-be4f-4497-805c-7ef8205f03ed\") " pod="openshift-marketplace/certified-operators-k2f9v" Dec 03 12:44:03 crc kubenswrapper[5052]: I1203 12:44:03.017376 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/certified-operators-k2f9v" Dec 03 12:44:03 crc kubenswrapper[5052]: I1203 12:44:03.054430 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="68e23bb4-c34c-458a-a02e-dfc620082df6" containerName="ceilometer-central-agent" containerID="cri-o://3a367bcc668e3d22b59106456e5a11faf700cbaf32b046598a5f08d9da45582a" gracePeriod=30 Dec 03 12:44:03 crc kubenswrapper[5052]: I1203 12:44:03.054731 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"68e23bb4-c34c-458a-a02e-dfc620082df6","Type":"ContainerStarted","Data":"45aaa2de3d0a762c69c4c671b66679d408169cdddc2b71ee9d42fa9b2783017f"} Dec 03 12:44:03 crc kubenswrapper[5052]: I1203 12:44:03.054768 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:44:03 crc kubenswrapper[5052]: I1203 12:44:03.055049 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="68e23bb4-c34c-458a-a02e-dfc620082df6" containerName="proxy-httpd" containerID="cri-o://45aaa2de3d0a762c69c4c671b66679d408169cdddc2b71ee9d42fa9b2783017f" gracePeriod=30 Dec 03 12:44:03 crc kubenswrapper[5052]: I1203 12:44:03.055100 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="68e23bb4-c34c-458a-a02e-dfc620082df6" containerName="sg-core" containerID="cri-o://8155928eaaa4da62211aa59eb0ec0eded75cda3bfa39b972216b9921042e0dc0" gracePeriod=30 Dec 03 12:44:03 crc kubenswrapper[5052]: I1203 12:44:03.055134 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="68e23bb4-c34c-458a-a02e-dfc620082df6" containerName="ceilometer-notification-agent" containerID="cri-o://12e601ccb583e45a27f9f873175b83c0b6728007392fa552bc07b64e7d4d0b16" gracePeriod=30 Dec 03 12:44:03 crc kubenswrapper[5052]: I1203 12:44:03.346842 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:44:03 crc kubenswrapper[5052]: I1203 12:44:03.600951 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/ceilometer-0" podStartSLOduration=2.987605961 podStartE2EDuration="8.600929721s" podCreationTimestamp="2025-12-03 12:43:55 +0000 UTC" firstStartedPulling="2025-12-03 12:43:56.846336073 +0000 UTC m=+5027.434577837" lastFinishedPulling="2025-12-03 12:44:02.459659833 +0000 UTC m=+5033.047901597" observedRunningTime="2025-12-03 12:44:03.081434435 +0000 UTC m=+5033.669676209" watchObservedRunningTime="2025-12-03 12:44:03.600929721 +0000 UTC m=+5034.189171485" Dec 03 12:44:03 crc kubenswrapper[5052]: I1203 12:44:03.606184 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/certified-operators-k2f9v"] Dec 03 12:44:04 crc kubenswrapper[5052]: I1203 12:44:04.083972 5052 generic.go:334] "Generic (PLEG): container finished" podID="21de3002-be4f-4497-805c-7ef8205f03ed" containerID="52e918cfb45febd349a692a34fb1794f56f4d89e4cc94e58a3a3235a8c6e8719" exitCode=0 Dec 03 12:44:04 crc kubenswrapper[5052]: I1203 12:44:04.084323 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/certified-operators-k2f9v" event={"ID":"21de3002-be4f-4497-805c-7ef8205f03ed","Type":"ContainerDied","Data":"52e918cfb45febd349a692a34fb1794f56f4d89e4cc94e58a3a3235a8c6e8719"} Dec 03 12:44:04 crc kubenswrapper[5052]: I1203 12:44:04.084360 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/certified-operators-k2f9v" event={"ID":"21de3002-be4f-4497-805c-7ef8205f03ed","Type":"ContainerStarted","Data":"d792929ff9e31d0b78d44c0d88af24694c0ac2effce71dde3090f7b0c846a4f1"} Dec 03 12:44:04 crc kubenswrapper[5052]: I1203 12:44:04.089267 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openshift-marketplace/redhat-marketplace-wmzlz"] Dec 03 12:44:04 crc kubenswrapper[5052]: I1203 12:44:04.091118 5052 generic.go:334] "Generic (PLEG): container finished" podID="68e23bb4-c34c-458a-a02e-dfc620082df6" containerID="45aaa2de3d0a762c69c4c671b66679d408169cdddc2b71ee9d42fa9b2783017f" exitCode=0 Dec 03 12:44:04 crc kubenswrapper[5052]: I1203 12:44:04.091147 5052 generic.go:334] "Generic (PLEG): container finished" podID="68e23bb4-c34c-458a-a02e-dfc620082df6" containerID="8155928eaaa4da62211aa59eb0ec0eded75cda3bfa39b972216b9921042e0dc0" exitCode=2 Dec 03 12:44:04 crc kubenswrapper[5052]: I1203 12:44:04.091156 5052 generic.go:334] "Generic (PLEG): container finished" podID="68e23bb4-c34c-458a-a02e-dfc620082df6" containerID="12e601ccb583e45a27f9f873175b83c0b6728007392fa552bc07b64e7d4d0b16" exitCode=0 Dec 03 12:44:04 crc kubenswrapper[5052]: I1203 12:44:04.091242 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"68e23bb4-c34c-458a-a02e-dfc620082df6","Type":"ContainerDied","Data":"45aaa2de3d0a762c69c4c671b66679d408169cdddc2b71ee9d42fa9b2783017f"} Dec 03 12:44:04 crc kubenswrapper[5052]: I1203 12:44:04.091270 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"68e23bb4-c34c-458a-a02e-dfc620082df6","Type":"ContainerDied","Data":"8155928eaaa4da62211aa59eb0ec0eded75cda3bfa39b972216b9921042e0dc0"} Dec 03 12:44:04 crc kubenswrapper[5052]: I1203 12:44:04.091284 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"68e23bb4-c34c-458a-a02e-dfc620082df6","Type":"ContainerDied","Data":"12e601ccb583e45a27f9f873175b83c0b6728007392fa552bc07b64e7d4d0b16"} Dec 03 12:44:04 crc kubenswrapper[5052]: I1203 12:44:04.091359 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/redhat-marketplace-wmzlz" Dec 03 12:44:04 crc kubenswrapper[5052]: I1203 12:44:04.115194 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-txhqv\" (UniqueName: \"kubernetes.io/projected/c95c6426-bfd6-45b8-b35d-969ff5a88e3a-kube-api-access-txhqv\") pod \"redhat-marketplace-wmzlz\" (UID: \"c95c6426-bfd6-45b8-b35d-969ff5a88e3a\") " pod="openshift-marketplace/redhat-marketplace-wmzlz" Dec 03 12:44:04 crc kubenswrapper[5052]: I1203 12:44:04.115254 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/c95c6426-bfd6-45b8-b35d-969ff5a88e3a-utilities\") pod \"redhat-marketplace-wmzlz\" (UID: \"c95c6426-bfd6-45b8-b35d-969ff5a88e3a\") " pod="openshift-marketplace/redhat-marketplace-wmzlz" Dec 03 12:44:04 crc kubenswrapper[5052]: I1203 12:44:04.115388 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/c95c6426-bfd6-45b8-b35d-969ff5a88e3a-catalog-content\") pod \"redhat-marketplace-wmzlz\" (UID: \"c95c6426-bfd6-45b8-b35d-969ff5a88e3a\") " pod="openshift-marketplace/redhat-marketplace-wmzlz" Dec 03 12:44:04 crc kubenswrapper[5052]: I1203 12:44:04.119638 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/redhat-marketplace-wmzlz"] Dec 03 12:44:04 crc kubenswrapper[5052]: I1203 12:44:04.217569 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/c95c6426-bfd6-45b8-b35d-969ff5a88e3a-catalog-content\") pod \"redhat-marketplace-wmzlz\" (UID: \"c95c6426-bfd6-45b8-b35d-969ff5a88e3a\") " pod="openshift-marketplace/redhat-marketplace-wmzlz" Dec 03 12:44:04 crc kubenswrapper[5052]: I1203 12:44:04.217768 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-txhqv\" (UniqueName: \"kubernetes.io/projected/c95c6426-bfd6-45b8-b35d-969ff5a88e3a-kube-api-access-txhqv\") pod \"redhat-marketplace-wmzlz\" (UID: \"c95c6426-bfd6-45b8-b35d-969ff5a88e3a\") " pod="openshift-marketplace/redhat-marketplace-wmzlz" Dec 03 12:44:04 crc kubenswrapper[5052]: I1203 12:44:04.217794 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/c95c6426-bfd6-45b8-b35d-969ff5a88e3a-utilities\") pod \"redhat-marketplace-wmzlz\" (UID: \"c95c6426-bfd6-45b8-b35d-969ff5a88e3a\") " pod="openshift-marketplace/redhat-marketplace-wmzlz" Dec 03 12:44:04 crc kubenswrapper[5052]: I1203 12:44:04.218278 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/c95c6426-bfd6-45b8-b35d-969ff5a88e3a-utilities\") pod \"redhat-marketplace-wmzlz\" (UID: \"c95c6426-bfd6-45b8-b35d-969ff5a88e3a\") " pod="openshift-marketplace/redhat-marketplace-wmzlz" Dec 03 12:44:04 crc kubenswrapper[5052]: I1203 12:44:04.218470 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/c95c6426-bfd6-45b8-b35d-969ff5a88e3a-catalog-content\") pod \"redhat-marketplace-wmzlz\" (UID: \"c95c6426-bfd6-45b8-b35d-969ff5a88e3a\") " pod="openshift-marketplace/redhat-marketplace-wmzlz" Dec 03 12:44:04 crc kubenswrapper[5052]: I1203 12:44:04.242938 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-txhqv\" (UniqueName: \"kubernetes.io/projected/c95c6426-bfd6-45b8-b35d-969ff5a88e3a-kube-api-access-txhqv\") pod \"redhat-marketplace-wmzlz\" (UID: \"c95c6426-bfd6-45b8-b35d-969ff5a88e3a\") " pod="openshift-marketplace/redhat-marketplace-wmzlz" Dec 03 12:44:04 crc kubenswrapper[5052]: I1203 12:44:04.424451 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/redhat-marketplace-wmzlz" Dec 03 12:44:04 crc kubenswrapper[5052]: I1203 12:44:04.850667 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:44:04 crc kubenswrapper[5052]: I1203 12:44:04.851047 5052 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Dec 03 12:44:04 crc kubenswrapper[5052]: I1203 12:44:04.932274 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/redhat-marketplace-wmzlz"] Dec 03 12:44:05 crc kubenswrapper[5052]: I1203 12:44:05.037150 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:44:05 crc kubenswrapper[5052]: I1203 12:44:05.117274 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-marketplace-wmzlz" event={"ID":"c95c6426-bfd6-45b8-b35d-969ff5a88e3a","Type":"ContainerStarted","Data":"bbb0b86e3a9923f1af3d51b0fdc7d014d8aa8ba265f1f0d2d0fb2cb881fc5539"} Dec 03 12:44:05 crc kubenswrapper[5052]: I1203 12:44:05.129359 5052 generic.go:334] "Generic (PLEG): container finished" podID="68e23bb4-c34c-458a-a02e-dfc620082df6" containerID="3a367bcc668e3d22b59106456e5a11faf700cbaf32b046598a5f08d9da45582a" exitCode=0 Dec 03 12:44:05 crc kubenswrapper[5052]: I1203 12:44:05.129429 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"68e23bb4-c34c-458a-a02e-dfc620082df6","Type":"ContainerDied","Data":"3a367bcc668e3d22b59106456e5a11faf700cbaf32b046598a5f08d9da45582a"} Dec 03 12:44:05 crc kubenswrapper[5052]: I1203 12:44:05.379604 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:44:05 crc kubenswrapper[5052]: I1203 12:44:05.380354 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:44:05 crc kubenswrapper[5052]: I1203 12:44:05.419397 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:44:05 crc kubenswrapper[5052]: I1203 12:44:05.455030 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:44:05 crc kubenswrapper[5052]: I1203 12:44:05.463274 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:44:05 crc kubenswrapper[5052]: I1203 12:44:05.560387 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/68e23bb4-c34c-458a-a02e-dfc620082df6-sg-core-conf-yaml\") pod \"68e23bb4-c34c-458a-a02e-dfc620082df6\" (UID: \"68e23bb4-c34c-458a-a02e-dfc620082df6\") " Dec 03 12:44:05 crc kubenswrapper[5052]: I1203 12:44:05.560459 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/68e23bb4-c34c-458a-a02e-dfc620082df6-combined-ca-bundle\") pod \"68e23bb4-c34c-458a-a02e-dfc620082df6\" (UID: \"68e23bb4-c34c-458a-a02e-dfc620082df6\") " Dec 03 12:44:05 crc kubenswrapper[5052]: I1203 12:44:05.560584 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/68e23bb4-c34c-458a-a02e-dfc620082df6-scripts\") pod \"68e23bb4-c34c-458a-a02e-dfc620082df6\" (UID: \"68e23bb4-c34c-458a-a02e-dfc620082df6\") " Dec 03 12:44:05 crc kubenswrapper[5052]: I1203 12:44:05.560615 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/68e23bb4-c34c-458a-a02e-dfc620082df6-run-httpd\") pod \"68e23bb4-c34c-458a-a02e-dfc620082df6\" (UID: \"68e23bb4-c34c-458a-a02e-dfc620082df6\") " Dec 03 12:44:05 crc kubenswrapper[5052]: I1203 12:44:05.560752 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wvkbt\" (UniqueName: \"kubernetes.io/projected/68e23bb4-c34c-458a-a02e-dfc620082df6-kube-api-access-wvkbt\") pod \"68e23bb4-c34c-458a-a02e-dfc620082df6\" (UID: \"68e23bb4-c34c-458a-a02e-dfc620082df6\") " Dec 03 12:44:05 crc kubenswrapper[5052]: I1203 12:44:05.560789 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/68e23bb4-c34c-458a-a02e-dfc620082df6-config-data\") pod \"68e23bb4-c34c-458a-a02e-dfc620082df6\" (UID: \"68e23bb4-c34c-458a-a02e-dfc620082df6\") " Dec 03 12:44:05 crc kubenswrapper[5052]: I1203 12:44:05.560816 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/68e23bb4-c34c-458a-a02e-dfc620082df6-log-httpd\") pod \"68e23bb4-c34c-458a-a02e-dfc620082df6\" (UID: \"68e23bb4-c34c-458a-a02e-dfc620082df6\") " Dec 03 12:44:05 crc kubenswrapper[5052]: I1203 12:44:05.562488 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/68e23bb4-c34c-458a-a02e-dfc620082df6-run-httpd" (OuterVolumeSpecName: "run-httpd") pod "68e23bb4-c34c-458a-a02e-dfc620082df6" (UID: "68e23bb4-c34c-458a-a02e-dfc620082df6"). InnerVolumeSpecName "run-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:44:05 crc kubenswrapper[5052]: I1203 12:44:05.562761 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/68e23bb4-c34c-458a-a02e-dfc620082df6-log-httpd" (OuterVolumeSpecName: "log-httpd") pod "68e23bb4-c34c-458a-a02e-dfc620082df6" (UID: "68e23bb4-c34c-458a-a02e-dfc620082df6"). InnerVolumeSpecName "log-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:44:05 crc kubenswrapper[5052]: I1203 12:44:05.576406 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/68e23bb4-c34c-458a-a02e-dfc620082df6-scripts" (OuterVolumeSpecName: "scripts") pod "68e23bb4-c34c-458a-a02e-dfc620082df6" (UID: "68e23bb4-c34c-458a-a02e-dfc620082df6"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:44:05 crc kubenswrapper[5052]: I1203 12:44:05.576441 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/68e23bb4-c34c-458a-a02e-dfc620082df6-kube-api-access-wvkbt" (OuterVolumeSpecName: "kube-api-access-wvkbt") pod "68e23bb4-c34c-458a-a02e-dfc620082df6" (UID: "68e23bb4-c34c-458a-a02e-dfc620082df6"). InnerVolumeSpecName "kube-api-access-wvkbt". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:44:05 crc kubenswrapper[5052]: I1203 12:44:05.600461 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/68e23bb4-c34c-458a-a02e-dfc620082df6-sg-core-conf-yaml" (OuterVolumeSpecName: "sg-core-conf-yaml") pod "68e23bb4-c34c-458a-a02e-dfc620082df6" (UID: "68e23bb4-c34c-458a-a02e-dfc620082df6"). InnerVolumeSpecName "sg-core-conf-yaml". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:44:05 crc kubenswrapper[5052]: I1203 12:44:05.648462 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/68e23bb4-c34c-458a-a02e-dfc620082df6-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "68e23bb4-c34c-458a-a02e-dfc620082df6" (UID: "68e23bb4-c34c-458a-a02e-dfc620082df6"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:44:05 crc kubenswrapper[5052]: I1203 12:44:05.663591 5052 reconciler_common.go:293] "Volume detached for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/68e23bb4-c34c-458a-a02e-dfc620082df6-sg-core-conf-yaml\") on node \"crc\" DevicePath \"\"" Dec 03 12:44:05 crc kubenswrapper[5052]: I1203 12:44:05.663636 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/68e23bb4-c34c-458a-a02e-dfc620082df6-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:44:05 crc kubenswrapper[5052]: I1203 12:44:05.663649 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/68e23bb4-c34c-458a-a02e-dfc620082df6-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:44:05 crc kubenswrapper[5052]: I1203 12:44:05.663661 5052 reconciler_common.go:293] "Volume detached for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/68e23bb4-c34c-458a-a02e-dfc620082df6-run-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:44:05 crc kubenswrapper[5052]: I1203 12:44:05.663675 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-wvkbt\" (UniqueName: \"kubernetes.io/projected/68e23bb4-c34c-458a-a02e-dfc620082df6-kube-api-access-wvkbt\") on node \"crc\" DevicePath \"\"" Dec 03 12:44:05 crc kubenswrapper[5052]: I1203 12:44:05.663688 5052 reconciler_common.go:293] "Volume detached for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/68e23bb4-c34c-458a-a02e-dfc620082df6-log-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:44:05 crc kubenswrapper[5052]: I1203 12:44:05.673870 5052 scope.go:117] "RemoveContainer" containerID="2aed1c760a326f3d008799e902b1093313df3764a57c879f0b3686410127d37d" Dec 03 12:44:05 crc kubenswrapper[5052]: E1203 12:44:05.674442 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 12:44:05 crc kubenswrapper[5052]: I1203 12:44:05.709687 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/68e23bb4-c34c-458a-a02e-dfc620082df6-config-data" (OuterVolumeSpecName: "config-data") pod "68e23bb4-c34c-458a-a02e-dfc620082df6" (UID: "68e23bb4-c34c-458a-a02e-dfc620082df6"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:44:05 crc kubenswrapper[5052]: I1203 12:44:05.767122 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/68e23bb4-c34c-458a-a02e-dfc620082df6-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:44:06 crc kubenswrapper[5052]: I1203 12:44:06.141966 5052 generic.go:334] "Generic (PLEG): container finished" podID="c95c6426-bfd6-45b8-b35d-969ff5a88e3a" containerID="3061b65a3ae17922aab71a0cb8765f84a54186703f7ce440b99b477a35e83b9d" exitCode=0 Dec 03 12:44:06 crc kubenswrapper[5052]: I1203 12:44:06.142268 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-marketplace-wmzlz" event={"ID":"c95c6426-bfd6-45b8-b35d-969ff5a88e3a","Type":"ContainerDied","Data":"3061b65a3ae17922aab71a0cb8765f84a54186703f7ce440b99b477a35e83b9d"} Dec 03 12:44:06 crc kubenswrapper[5052]: I1203 12:44:06.150899 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"68e23bb4-c34c-458a-a02e-dfc620082df6","Type":"ContainerDied","Data":"1b2fa304d594b0349c950c74661c8bc88c5f959d923306417bf60c8f804c080f"} Dec 03 12:44:06 crc kubenswrapper[5052]: I1203 12:44:06.150944 5052 scope.go:117] "RemoveContainer" containerID="45aaa2de3d0a762c69c4c671b66679d408169cdddc2b71ee9d42fa9b2783017f" Dec 03 12:44:06 crc kubenswrapper[5052]: I1203 12:44:06.150981 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:44:06 crc kubenswrapper[5052]: I1203 12:44:06.154064 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/certified-operators-k2f9v" event={"ID":"21de3002-be4f-4497-805c-7ef8205f03ed","Type":"ContainerStarted","Data":"8763dbc208c3681c490f71173ed527e579f4e7fb1019eb99754c623ac4ca2738"} Dec 03 12:44:06 crc kubenswrapper[5052]: I1203 12:44:06.154605 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:44:06 crc kubenswrapper[5052]: I1203 12:44:06.154641 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:44:06 crc kubenswrapper[5052]: I1203 12:44:06.210660 5052 scope.go:117] "RemoveContainer" containerID="8155928eaaa4da62211aa59eb0ec0eded75cda3bfa39b972216b9921042e0dc0" Dec 03 12:44:06 crc kubenswrapper[5052]: I1203 12:44:06.217406 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:44:06 crc kubenswrapper[5052]: I1203 12:44:06.228656 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:44:06 crc kubenswrapper[5052]: I1203 12:44:06.229026 5052 scope.go:117] "RemoveContainer" containerID="12e601ccb583e45a27f9f873175b83c0b6728007392fa552bc07b64e7d4d0b16" Dec 03 12:44:06 crc kubenswrapper[5052]: I1203 12:44:06.247664 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:44:06 crc kubenswrapper[5052]: E1203 12:44:06.248072 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="68e23bb4-c34c-458a-a02e-dfc620082df6" containerName="ceilometer-notification-agent" Dec 03 12:44:06 crc kubenswrapper[5052]: I1203 12:44:06.248084 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="68e23bb4-c34c-458a-a02e-dfc620082df6" containerName="ceilometer-notification-agent" Dec 03 12:44:06 crc kubenswrapper[5052]: E1203 12:44:06.248099 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="68e23bb4-c34c-458a-a02e-dfc620082df6" containerName="sg-core" Dec 03 12:44:06 crc kubenswrapper[5052]: I1203 12:44:06.248105 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="68e23bb4-c34c-458a-a02e-dfc620082df6" containerName="sg-core" Dec 03 12:44:06 crc kubenswrapper[5052]: E1203 12:44:06.248111 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="68e23bb4-c34c-458a-a02e-dfc620082df6" containerName="proxy-httpd" Dec 03 12:44:06 crc kubenswrapper[5052]: I1203 12:44:06.248116 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="68e23bb4-c34c-458a-a02e-dfc620082df6" containerName="proxy-httpd" Dec 03 12:44:06 crc kubenswrapper[5052]: E1203 12:44:06.248136 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="68e23bb4-c34c-458a-a02e-dfc620082df6" containerName="ceilometer-central-agent" Dec 03 12:44:06 crc kubenswrapper[5052]: I1203 12:44:06.248143 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="68e23bb4-c34c-458a-a02e-dfc620082df6" containerName="ceilometer-central-agent" Dec 03 12:44:06 crc kubenswrapper[5052]: I1203 12:44:06.248328 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="68e23bb4-c34c-458a-a02e-dfc620082df6" containerName="proxy-httpd" Dec 03 12:44:06 crc kubenswrapper[5052]: I1203 12:44:06.248342 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="68e23bb4-c34c-458a-a02e-dfc620082df6" containerName="ceilometer-central-agent" Dec 03 12:44:06 crc kubenswrapper[5052]: I1203 12:44:06.248351 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="68e23bb4-c34c-458a-a02e-dfc620082df6" containerName="ceilometer-notification-agent" Dec 03 12:44:06 crc kubenswrapper[5052]: I1203 12:44:06.248369 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="68e23bb4-c34c-458a-a02e-dfc620082df6" containerName="sg-core" Dec 03 12:44:06 crc kubenswrapper[5052]: I1203 12:44:06.250480 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:44:06 crc kubenswrapper[5052]: I1203 12:44:06.253161 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-scripts" Dec 03 12:44:06 crc kubenswrapper[5052]: I1203 12:44:06.254760 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-config-data" Dec 03 12:44:06 crc kubenswrapper[5052]: I1203 12:44:06.258957 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:44:06 crc kubenswrapper[5052]: I1203 12:44:06.261978 5052 scope.go:117] "RemoveContainer" containerID="3a367bcc668e3d22b59106456e5a11faf700cbaf32b046598a5f08d9da45582a" Dec 03 12:44:06 crc kubenswrapper[5052]: I1203 12:44:06.379353 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/3b455122-e3fd-44a8-baa3-2b10f856bfce-scripts\") pod \"ceilometer-0\" (UID: \"3b455122-e3fd-44a8-baa3-2b10f856bfce\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:44:06 crc kubenswrapper[5052]: I1203 12:44:06.379405 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/3b455122-e3fd-44a8-baa3-2b10f856bfce-run-httpd\") pod \"ceilometer-0\" (UID: \"3b455122-e3fd-44a8-baa3-2b10f856bfce\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:44:06 crc kubenswrapper[5052]: I1203 12:44:06.379429 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/3b455122-e3fd-44a8-baa3-2b10f856bfce-config-data\") pod \"ceilometer-0\" (UID: \"3b455122-e3fd-44a8-baa3-2b10f856bfce\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:44:06 crc kubenswrapper[5052]: I1203 12:44:06.379468 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/3b455122-e3fd-44a8-baa3-2b10f856bfce-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"3b455122-e3fd-44a8-baa3-2b10f856bfce\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:44:06 crc kubenswrapper[5052]: I1203 12:44:06.379497 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6lnv8\" (UniqueName: \"kubernetes.io/projected/3b455122-e3fd-44a8-baa3-2b10f856bfce-kube-api-access-6lnv8\") pod \"ceilometer-0\" (UID: \"3b455122-e3fd-44a8-baa3-2b10f856bfce\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:44:06 crc kubenswrapper[5052]: I1203 12:44:06.379518 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/3b455122-e3fd-44a8-baa3-2b10f856bfce-log-httpd\") pod \"ceilometer-0\" (UID: \"3b455122-e3fd-44a8-baa3-2b10f856bfce\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:44:06 crc kubenswrapper[5052]: I1203 12:44:06.379648 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3b455122-e3fd-44a8-baa3-2b10f856bfce-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"3b455122-e3fd-44a8-baa3-2b10f856bfce\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:44:06 crc kubenswrapper[5052]: I1203 12:44:06.481800 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3b455122-e3fd-44a8-baa3-2b10f856bfce-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"3b455122-e3fd-44a8-baa3-2b10f856bfce\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:44:06 crc kubenswrapper[5052]: I1203 12:44:06.481937 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/3b455122-e3fd-44a8-baa3-2b10f856bfce-scripts\") pod \"ceilometer-0\" (UID: \"3b455122-e3fd-44a8-baa3-2b10f856bfce\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:44:06 crc kubenswrapper[5052]: I1203 12:44:06.481963 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/3b455122-e3fd-44a8-baa3-2b10f856bfce-run-httpd\") pod \"ceilometer-0\" (UID: \"3b455122-e3fd-44a8-baa3-2b10f856bfce\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:44:06 crc kubenswrapper[5052]: I1203 12:44:06.481981 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/3b455122-e3fd-44a8-baa3-2b10f856bfce-config-data\") pod \"ceilometer-0\" (UID: \"3b455122-e3fd-44a8-baa3-2b10f856bfce\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:44:06 crc kubenswrapper[5052]: I1203 12:44:06.482958 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/3b455122-e3fd-44a8-baa3-2b10f856bfce-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"3b455122-e3fd-44a8-baa3-2b10f856bfce\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:44:06 crc kubenswrapper[5052]: I1203 12:44:06.483011 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-6lnv8\" (UniqueName: \"kubernetes.io/projected/3b455122-e3fd-44a8-baa3-2b10f856bfce-kube-api-access-6lnv8\") pod \"ceilometer-0\" (UID: \"3b455122-e3fd-44a8-baa3-2b10f856bfce\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:44:06 crc kubenswrapper[5052]: I1203 12:44:06.482583 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/3b455122-e3fd-44a8-baa3-2b10f856bfce-run-httpd\") pod \"ceilometer-0\" (UID: \"3b455122-e3fd-44a8-baa3-2b10f856bfce\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:44:06 crc kubenswrapper[5052]: I1203 12:44:06.483031 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/3b455122-e3fd-44a8-baa3-2b10f856bfce-log-httpd\") pod \"ceilometer-0\" (UID: \"3b455122-e3fd-44a8-baa3-2b10f856bfce\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:44:06 crc kubenswrapper[5052]: I1203 12:44:06.483594 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/3b455122-e3fd-44a8-baa3-2b10f856bfce-log-httpd\") pod \"ceilometer-0\" (UID: \"3b455122-e3fd-44a8-baa3-2b10f856bfce\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:44:06 crc kubenswrapper[5052]: I1203 12:44:06.489725 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/3b455122-e3fd-44a8-baa3-2b10f856bfce-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"3b455122-e3fd-44a8-baa3-2b10f856bfce\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:44:06 crc kubenswrapper[5052]: I1203 12:44:06.490100 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3b455122-e3fd-44a8-baa3-2b10f856bfce-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"3b455122-e3fd-44a8-baa3-2b10f856bfce\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:44:06 crc kubenswrapper[5052]: I1203 12:44:06.499671 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/3b455122-e3fd-44a8-baa3-2b10f856bfce-scripts\") pod \"ceilometer-0\" (UID: \"3b455122-e3fd-44a8-baa3-2b10f856bfce\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:44:06 crc kubenswrapper[5052]: I1203 12:44:06.505203 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-6lnv8\" (UniqueName: \"kubernetes.io/projected/3b455122-e3fd-44a8-baa3-2b10f856bfce-kube-api-access-6lnv8\") pod \"ceilometer-0\" (UID: \"3b455122-e3fd-44a8-baa3-2b10f856bfce\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:44:06 crc kubenswrapper[5052]: I1203 12:44:06.519145 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/3b455122-e3fd-44a8-baa3-2b10f856bfce-config-data\") pod \"ceilometer-0\" (UID: \"3b455122-e3fd-44a8-baa3-2b10f856bfce\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:44:06 crc kubenswrapper[5052]: I1203 12:44:06.576041 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:44:07 crc kubenswrapper[5052]: W1203 12:44:07.062431 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3b455122_e3fd_44a8_baa3_2b10f856bfce.slice/crio-8ac3459ca4d720b164d5bb90e6a490dd90e7b1a621c924db8632864b460a44e0 WatchSource:0}: Error finding container 8ac3459ca4d720b164d5bb90e6a490dd90e7b1a621c924db8632864b460a44e0: Status 404 returned error can't find the container with id 8ac3459ca4d720b164d5bb90e6a490dd90e7b1a621c924db8632864b460a44e0 Dec 03 12:44:07 crc kubenswrapper[5052]: I1203 12:44:07.070074 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:44:07 crc kubenswrapper[5052]: I1203 12:44:07.173767 5052 generic.go:334] "Generic (PLEG): container finished" podID="21de3002-be4f-4497-805c-7ef8205f03ed" containerID="8763dbc208c3681c490f71173ed527e579f4e7fb1019eb99754c623ac4ca2738" exitCode=0 Dec 03 12:44:07 crc kubenswrapper[5052]: I1203 12:44:07.174853 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/certified-operators-k2f9v" event={"ID":"21de3002-be4f-4497-805c-7ef8205f03ed","Type":"ContainerDied","Data":"8763dbc208c3681c490f71173ed527e579f4e7fb1019eb99754c623ac4ca2738"} Dec 03 12:44:07 crc kubenswrapper[5052]: I1203 12:44:07.174900 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/certified-operators-k2f9v" event={"ID":"21de3002-be4f-4497-805c-7ef8205f03ed","Type":"ContainerStarted","Data":"7df935dc5104efe7c030b20df59e99dc66141fc7489b85395c0c5c96399d83f9"} Dec 03 12:44:07 crc kubenswrapper[5052]: I1203 12:44:07.177428 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"3b455122-e3fd-44a8-baa3-2b10f856bfce","Type":"ContainerStarted","Data":"8ac3459ca4d720b164d5bb90e6a490dd90e7b1a621c924db8632864b460a44e0"} Dec 03 12:44:07 crc kubenswrapper[5052]: I1203 12:44:07.198333 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-marketplace/certified-operators-k2f9v" podStartSLOduration=2.739213384 podStartE2EDuration="5.19831617s" podCreationTimestamp="2025-12-03 12:44:02 +0000 UTC" firstStartedPulling="2025-12-03 12:44:04.087562658 +0000 UTC m=+5034.675804422" lastFinishedPulling="2025-12-03 12:44:06.546665444 +0000 UTC m=+5037.134907208" observedRunningTime="2025-12-03 12:44:07.190416324 +0000 UTC m=+5037.778658088" watchObservedRunningTime="2025-12-03 12:44:07.19831617 +0000 UTC m=+5037.786557934" Dec 03 12:44:07 crc kubenswrapper[5052]: I1203 12:44:07.685051 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="68e23bb4-c34c-458a-a02e-dfc620082df6" path="/var/lib/kubelet/pods/68e23bb4-c34c-458a-a02e-dfc620082df6/volumes" Dec 03 12:44:08 crc kubenswrapper[5052]: I1203 12:44:08.188496 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"3b455122-e3fd-44a8-baa3-2b10f856bfce","Type":"ContainerStarted","Data":"9b9f209894b73ff775c7636aa2252ab1c82d6c5abb431a90314a2d850e973805"} Dec 03 12:44:08 crc kubenswrapper[5052]: I1203 12:44:08.191248 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-marketplace-wmzlz" event={"ID":"c95c6426-bfd6-45b8-b35d-969ff5a88e3a","Type":"ContainerStarted","Data":"6805820d5106d3e06471a45a4f306835d9eeb0ee70dfb3c94bd018335f9d6c72"} Dec 03 12:44:08 crc kubenswrapper[5052]: I1203 12:44:08.191284 5052 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Dec 03 12:44:08 crc kubenswrapper[5052]: I1203 12:44:08.191306 5052 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Dec 03 12:44:08 crc kubenswrapper[5052]: I1203 12:44:08.224965 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:44:08 crc kubenswrapper[5052]: I1203 12:44:08.226750 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:44:08 crc kubenswrapper[5052]: E1203 12:44:08.349706 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="8ff041a03ee1701a0052612991d867cafab4a1723240a7109f4b7132da5cae68" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:44:08 crc kubenswrapper[5052]: E1203 12:44:08.351801 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="8ff041a03ee1701a0052612991d867cafab4a1723240a7109f4b7132da5cae68" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:44:08 crc kubenswrapper[5052]: E1203 12:44:08.357184 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="8ff041a03ee1701a0052612991d867cafab4a1723240a7109f4b7132da5cae68" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:44:08 crc kubenswrapper[5052]: E1203 12:44:08.357267 5052 prober.go:104] "Probe errored" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" probeType="Readiness" pod="openstack-kuttl-tests/nova-cell0-conductor-0" podUID="307199d5-acf0-4ede-b755-ed6bc25eb978" containerName="nova-cell0-conductor-conductor" Dec 03 12:44:09 crc kubenswrapper[5052]: I1203 12:44:09.203899 5052 generic.go:334] "Generic (PLEG): container finished" podID="c95c6426-bfd6-45b8-b35d-969ff5a88e3a" containerID="6805820d5106d3e06471a45a4f306835d9eeb0ee70dfb3c94bd018335f9d6c72" exitCode=0 Dec 03 12:44:09 crc kubenswrapper[5052]: I1203 12:44:09.204298 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-marketplace-wmzlz" event={"ID":"c95c6426-bfd6-45b8-b35d-969ff5a88e3a","Type":"ContainerDied","Data":"6805820d5106d3e06471a45a4f306835d9eeb0ee70dfb3c94bd018335f9d6c72"} Dec 03 12:44:09 crc kubenswrapper[5052]: I1203 12:44:09.211677 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"3b455122-e3fd-44a8-baa3-2b10f856bfce","Type":"ContainerStarted","Data":"1f02fbccdb7521208886acc6eb8caee6a28ba2f64b08fbbd8a5f9529383c29a3"} Dec 03 12:44:10 crc kubenswrapper[5052]: I1203 12:44:10.223732 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"3b455122-e3fd-44a8-baa3-2b10f856bfce","Type":"ContainerStarted","Data":"fe2a0268a0be69498dad63a9b2768a0651397fdd6a75908b40f05a1eb5621f57"} Dec 03 12:44:10 crc kubenswrapper[5052]: I1203 12:44:10.227206 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-marketplace-wmzlz" event={"ID":"c95c6426-bfd6-45b8-b35d-969ff5a88e3a","Type":"ContainerStarted","Data":"8ed469d4352febaa9d5ec81e6d03ac8f16b7bdb116048f36ed0cf7b38f27d5fa"} Dec 03 12:44:10 crc kubenswrapper[5052]: I1203 12:44:10.254703 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-marketplace/redhat-marketplace-wmzlz" podStartSLOduration=2.633802066 podStartE2EDuration="6.254682627s" podCreationTimestamp="2025-12-03 12:44:04 +0000 UTC" firstStartedPulling="2025-12-03 12:44:06.143990008 +0000 UTC m=+5036.732231772" lastFinishedPulling="2025-12-03 12:44:09.764870569 +0000 UTC m=+5040.353112333" observedRunningTime="2025-12-03 12:44:10.244356722 +0000 UTC m=+5040.832598486" watchObservedRunningTime="2025-12-03 12:44:10.254682627 +0000 UTC m=+5040.842924391" Dec 03 12:44:13 crc kubenswrapper[5052]: I1203 12:44:13.018026 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openshift-marketplace/certified-operators-k2f9v" Dec 03 12:44:13 crc kubenswrapper[5052]: I1203 12:44:13.018704 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-marketplace/certified-operators-k2f9v" Dec 03 12:44:13 crc kubenswrapper[5052]: I1203 12:44:13.068764 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-marketplace/certified-operators-k2f9v" Dec 03 12:44:13 crc kubenswrapper[5052]: I1203 12:44:13.264312 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"3b455122-e3fd-44a8-baa3-2b10f856bfce","Type":"ContainerStarted","Data":"d80485db5415a79a1bee865fce5b1dee64e47e17be467a9fdd9649b63865a438"} Dec 03 12:44:13 crc kubenswrapper[5052]: I1203 12:44:13.264371 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:44:13 crc kubenswrapper[5052]: I1203 12:44:13.317724 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-marketplace/certified-operators-k2f9v" Dec 03 12:44:13 crc kubenswrapper[5052]: I1203 12:44:13.338360 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/ceilometer-0" podStartSLOduration=1.772740639 podStartE2EDuration="7.338339974s" podCreationTimestamp="2025-12-03 12:44:06 +0000 UTC" firstStartedPulling="2025-12-03 12:44:07.069159317 +0000 UTC m=+5037.657401081" lastFinishedPulling="2025-12-03 12:44:12.634758652 +0000 UTC m=+5043.223000416" observedRunningTime="2025-12-03 12:44:13.308027877 +0000 UTC m=+5043.896269641" watchObservedRunningTime="2025-12-03 12:44:13.338339974 +0000 UTC m=+5043.926581738" Dec 03 12:44:13 crc kubenswrapper[5052]: E1203 12:44:13.350302 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="8ff041a03ee1701a0052612991d867cafab4a1723240a7109f4b7132da5cae68" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:44:13 crc kubenswrapper[5052]: E1203 12:44:13.351796 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="8ff041a03ee1701a0052612991d867cafab4a1723240a7109f4b7132da5cae68" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:44:13 crc kubenswrapper[5052]: E1203 12:44:13.353041 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="8ff041a03ee1701a0052612991d867cafab4a1723240a7109f4b7132da5cae68" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:44:13 crc kubenswrapper[5052]: E1203 12:44:13.353085 5052 prober.go:104] "Probe errored" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" probeType="Readiness" pod="openstack-kuttl-tests/nova-cell0-conductor-0" podUID="307199d5-acf0-4ede-b755-ed6bc25eb978" containerName="nova-cell0-conductor-conductor" Dec 03 12:44:14 crc kubenswrapper[5052]: I1203 12:44:14.425115 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openshift-marketplace/redhat-marketplace-wmzlz" Dec 03 12:44:14 crc kubenswrapper[5052]: I1203 12:44:14.425468 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-marketplace/redhat-marketplace-wmzlz" Dec 03 12:44:14 crc kubenswrapper[5052]: I1203 12:44:14.477086 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-marketplace/redhat-marketplace-wmzlz" Dec 03 12:44:15 crc kubenswrapper[5052]: I1203 12:44:15.338193 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-marketplace/redhat-marketplace-wmzlz" Dec 03 12:44:16 crc kubenswrapper[5052]: I1203 12:44:16.673632 5052 scope.go:117] "RemoveContainer" containerID="2aed1c760a326f3d008799e902b1093313df3764a57c879f0b3686410127d37d" Dec 03 12:44:16 crc kubenswrapper[5052]: E1203 12:44:16.674027 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 12:44:17 crc kubenswrapper[5052]: I1203 12:44:17.267851 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openshift-marketplace/redhat-marketplace-wmzlz"] Dec 03 12:44:17 crc kubenswrapper[5052]: I1203 12:44:17.306961 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openshift-marketplace/redhat-marketplace-wmzlz" podUID="c95c6426-bfd6-45b8-b35d-969ff5a88e3a" containerName="registry-server" containerID="cri-o://8ed469d4352febaa9d5ec81e6d03ac8f16b7bdb116048f36ed0cf7b38f27d5fa" gracePeriod=2 Dec 03 12:44:17 crc kubenswrapper[5052]: I1203 12:44:17.839321 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/redhat-marketplace-wmzlz" Dec 03 12:44:17 crc kubenswrapper[5052]: I1203 12:44:17.874382 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openshift-marketplace/certified-operators-k2f9v"] Dec 03 12:44:17 crc kubenswrapper[5052]: I1203 12:44:17.874651 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openshift-marketplace/certified-operators-k2f9v" podUID="21de3002-be4f-4497-805c-7ef8205f03ed" containerName="registry-server" containerID="cri-o://7df935dc5104efe7c030b20df59e99dc66141fc7489b85395c0c5c96399d83f9" gracePeriod=2 Dec 03 12:44:17 crc kubenswrapper[5052]: I1203 12:44:17.917705 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/c95c6426-bfd6-45b8-b35d-969ff5a88e3a-utilities\") pod \"c95c6426-bfd6-45b8-b35d-969ff5a88e3a\" (UID: \"c95c6426-bfd6-45b8-b35d-969ff5a88e3a\") " Dec 03 12:44:17 crc kubenswrapper[5052]: I1203 12:44:17.917789 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/c95c6426-bfd6-45b8-b35d-969ff5a88e3a-catalog-content\") pod \"c95c6426-bfd6-45b8-b35d-969ff5a88e3a\" (UID: \"c95c6426-bfd6-45b8-b35d-969ff5a88e3a\") " Dec 03 12:44:17 crc kubenswrapper[5052]: I1203 12:44:17.917901 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-txhqv\" (UniqueName: \"kubernetes.io/projected/c95c6426-bfd6-45b8-b35d-969ff5a88e3a-kube-api-access-txhqv\") pod \"c95c6426-bfd6-45b8-b35d-969ff5a88e3a\" (UID: \"c95c6426-bfd6-45b8-b35d-969ff5a88e3a\") " Dec 03 12:44:17 crc kubenswrapper[5052]: I1203 12:44:17.918965 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c95c6426-bfd6-45b8-b35d-969ff5a88e3a-utilities" (OuterVolumeSpecName: "utilities") pod "c95c6426-bfd6-45b8-b35d-969ff5a88e3a" (UID: "c95c6426-bfd6-45b8-b35d-969ff5a88e3a"). InnerVolumeSpecName "utilities". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:44:17 crc kubenswrapper[5052]: I1203 12:44:17.924657 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c95c6426-bfd6-45b8-b35d-969ff5a88e3a-kube-api-access-txhqv" (OuterVolumeSpecName: "kube-api-access-txhqv") pod "c95c6426-bfd6-45b8-b35d-969ff5a88e3a" (UID: "c95c6426-bfd6-45b8-b35d-969ff5a88e3a"). InnerVolumeSpecName "kube-api-access-txhqv". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:44:17 crc kubenswrapper[5052]: I1203 12:44:17.944586 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c95c6426-bfd6-45b8-b35d-969ff5a88e3a-catalog-content" (OuterVolumeSpecName: "catalog-content") pod "c95c6426-bfd6-45b8-b35d-969ff5a88e3a" (UID: "c95c6426-bfd6-45b8-b35d-969ff5a88e3a"). InnerVolumeSpecName "catalog-content". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:44:18 crc kubenswrapper[5052]: I1203 12:44:18.020557 5052 reconciler_common.go:293] "Volume detached for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/c95c6426-bfd6-45b8-b35d-969ff5a88e3a-catalog-content\") on node \"crc\" DevicePath \"\"" Dec 03 12:44:18 crc kubenswrapper[5052]: I1203 12:44:18.020606 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-txhqv\" (UniqueName: \"kubernetes.io/projected/c95c6426-bfd6-45b8-b35d-969ff5a88e3a-kube-api-access-txhqv\") on node \"crc\" DevicePath \"\"" Dec 03 12:44:18 crc kubenswrapper[5052]: I1203 12:44:18.020622 5052 reconciler_common.go:293] "Volume detached for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/c95c6426-bfd6-45b8-b35d-969ff5a88e3a-utilities\") on node \"crc\" DevicePath \"\"" Dec 03 12:44:18 crc kubenswrapper[5052]: I1203 12:44:18.276908 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/certified-operators-k2f9v" Dec 03 12:44:18 crc kubenswrapper[5052]: I1203 12:44:18.330202 5052 generic.go:334] "Generic (PLEG): container finished" podID="c95c6426-bfd6-45b8-b35d-969ff5a88e3a" containerID="8ed469d4352febaa9d5ec81e6d03ac8f16b7bdb116048f36ed0cf7b38f27d5fa" exitCode=0 Dec 03 12:44:18 crc kubenswrapper[5052]: I1203 12:44:18.330351 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-marketplace-wmzlz" event={"ID":"c95c6426-bfd6-45b8-b35d-969ff5a88e3a","Type":"ContainerDied","Data":"8ed469d4352febaa9d5ec81e6d03ac8f16b7bdb116048f36ed0cf7b38f27d5fa"} Dec 03 12:44:18 crc kubenswrapper[5052]: I1203 12:44:18.330382 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-marketplace-wmzlz" event={"ID":"c95c6426-bfd6-45b8-b35d-969ff5a88e3a","Type":"ContainerDied","Data":"bbb0b86e3a9923f1af3d51b0fdc7d014d8aa8ba265f1f0d2d0fb2cb881fc5539"} Dec 03 12:44:18 crc kubenswrapper[5052]: I1203 12:44:18.330400 5052 scope.go:117] "RemoveContainer" containerID="8ed469d4352febaa9d5ec81e6d03ac8f16b7bdb116048f36ed0cf7b38f27d5fa" Dec 03 12:44:18 crc kubenswrapper[5052]: I1203 12:44:18.330600 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/redhat-marketplace-wmzlz" Dec 03 12:44:18 crc kubenswrapper[5052]: I1203 12:44:18.341168 5052 generic.go:334] "Generic (PLEG): container finished" podID="21de3002-be4f-4497-805c-7ef8205f03ed" containerID="7df935dc5104efe7c030b20df59e99dc66141fc7489b85395c0c5c96399d83f9" exitCode=0 Dec 03 12:44:18 crc kubenswrapper[5052]: I1203 12:44:18.341220 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/certified-operators-k2f9v" event={"ID":"21de3002-be4f-4497-805c-7ef8205f03ed","Type":"ContainerDied","Data":"7df935dc5104efe7c030b20df59e99dc66141fc7489b85395c0c5c96399d83f9"} Dec 03 12:44:18 crc kubenswrapper[5052]: I1203 12:44:18.341243 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/certified-operators-k2f9v" Dec 03 12:44:18 crc kubenswrapper[5052]: I1203 12:44:18.341258 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/certified-operators-k2f9v" event={"ID":"21de3002-be4f-4497-805c-7ef8205f03ed","Type":"ContainerDied","Data":"d792929ff9e31d0b78d44c0d88af24694c0ac2effce71dde3090f7b0c846a4f1"} Dec 03 12:44:18 crc kubenswrapper[5052]: E1203 12:44:18.350853 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="8ff041a03ee1701a0052612991d867cafab4a1723240a7109f4b7132da5cae68" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:44:18 crc kubenswrapper[5052]: E1203 12:44:18.353022 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="8ff041a03ee1701a0052612991d867cafab4a1723240a7109f4b7132da5cae68" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:44:18 crc kubenswrapper[5052]: E1203 12:44:18.356883 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="8ff041a03ee1701a0052612991d867cafab4a1723240a7109f4b7132da5cae68" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:44:18 crc kubenswrapper[5052]: E1203 12:44:18.357130 5052 prober.go:104] "Probe errored" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" probeType="Readiness" pod="openstack-kuttl-tests/nova-cell0-conductor-0" podUID="307199d5-acf0-4ede-b755-ed6bc25eb978" containerName="nova-cell0-conductor-conductor" Dec 03 12:44:18 crc kubenswrapper[5052]: I1203 12:44:18.369124 5052 scope.go:117] "RemoveContainer" containerID="6805820d5106d3e06471a45a4f306835d9eeb0ee70dfb3c94bd018335f9d6c72" Dec 03 12:44:18 crc kubenswrapper[5052]: I1203 12:44:18.382355 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openshift-marketplace/redhat-marketplace-wmzlz"] Dec 03 12:44:18 crc kubenswrapper[5052]: I1203 12:44:18.392101 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openshift-marketplace/redhat-marketplace-wmzlz"] Dec 03 12:44:18 crc kubenswrapper[5052]: I1203 12:44:18.400701 5052 scope.go:117] "RemoveContainer" containerID="3061b65a3ae17922aab71a0cb8765f84a54186703f7ce440b99b477a35e83b9d" Dec 03 12:44:18 crc kubenswrapper[5052]: I1203 12:44:18.424894 5052 scope.go:117] "RemoveContainer" containerID="8ed469d4352febaa9d5ec81e6d03ac8f16b7bdb116048f36ed0cf7b38f27d5fa" Dec 03 12:44:18 crc kubenswrapper[5052]: I1203 12:44:18.427954 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/21de3002-be4f-4497-805c-7ef8205f03ed-catalog-content\") pod \"21de3002-be4f-4497-805c-7ef8205f03ed\" (UID: \"21de3002-be4f-4497-805c-7ef8205f03ed\") " Dec 03 12:44:18 crc kubenswrapper[5052]: I1203 12:44:18.428384 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v2p79\" (UniqueName: \"kubernetes.io/projected/21de3002-be4f-4497-805c-7ef8205f03ed-kube-api-access-v2p79\") pod \"21de3002-be4f-4497-805c-7ef8205f03ed\" (UID: \"21de3002-be4f-4497-805c-7ef8205f03ed\") " Dec 03 12:44:18 crc kubenswrapper[5052]: I1203 12:44:18.428484 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/21de3002-be4f-4497-805c-7ef8205f03ed-utilities\") pod \"21de3002-be4f-4497-805c-7ef8205f03ed\" (UID: \"21de3002-be4f-4497-805c-7ef8205f03ed\") " Dec 03 12:44:18 crc kubenswrapper[5052]: I1203 12:44:18.430045 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/21de3002-be4f-4497-805c-7ef8205f03ed-utilities" (OuterVolumeSpecName: "utilities") pod "21de3002-be4f-4497-805c-7ef8205f03ed" (UID: "21de3002-be4f-4497-805c-7ef8205f03ed"). InnerVolumeSpecName "utilities". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:44:18 crc kubenswrapper[5052]: E1203 12:44:18.431294 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8ed469d4352febaa9d5ec81e6d03ac8f16b7bdb116048f36ed0cf7b38f27d5fa\": container with ID starting with 8ed469d4352febaa9d5ec81e6d03ac8f16b7bdb116048f36ed0cf7b38f27d5fa not found: ID does not exist" containerID="8ed469d4352febaa9d5ec81e6d03ac8f16b7bdb116048f36ed0cf7b38f27d5fa" Dec 03 12:44:18 crc kubenswrapper[5052]: I1203 12:44:18.431353 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8ed469d4352febaa9d5ec81e6d03ac8f16b7bdb116048f36ed0cf7b38f27d5fa"} err="failed to get container status \"8ed469d4352febaa9d5ec81e6d03ac8f16b7bdb116048f36ed0cf7b38f27d5fa\": rpc error: code = NotFound desc = could not find container \"8ed469d4352febaa9d5ec81e6d03ac8f16b7bdb116048f36ed0cf7b38f27d5fa\": container with ID starting with 8ed469d4352febaa9d5ec81e6d03ac8f16b7bdb116048f36ed0cf7b38f27d5fa not found: ID does not exist" Dec 03 12:44:18 crc kubenswrapper[5052]: I1203 12:44:18.431396 5052 scope.go:117] "RemoveContainer" containerID="6805820d5106d3e06471a45a4f306835d9eeb0ee70dfb3c94bd018335f9d6c72" Dec 03 12:44:18 crc kubenswrapper[5052]: E1203 12:44:18.433112 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6805820d5106d3e06471a45a4f306835d9eeb0ee70dfb3c94bd018335f9d6c72\": container with ID starting with 6805820d5106d3e06471a45a4f306835d9eeb0ee70dfb3c94bd018335f9d6c72 not found: ID does not exist" containerID="6805820d5106d3e06471a45a4f306835d9eeb0ee70dfb3c94bd018335f9d6c72" Dec 03 12:44:18 crc kubenswrapper[5052]: I1203 12:44:18.433155 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6805820d5106d3e06471a45a4f306835d9eeb0ee70dfb3c94bd018335f9d6c72"} err="failed to get container status \"6805820d5106d3e06471a45a4f306835d9eeb0ee70dfb3c94bd018335f9d6c72\": rpc error: code = NotFound desc = could not find container \"6805820d5106d3e06471a45a4f306835d9eeb0ee70dfb3c94bd018335f9d6c72\": container with ID starting with 6805820d5106d3e06471a45a4f306835d9eeb0ee70dfb3c94bd018335f9d6c72 not found: ID does not exist" Dec 03 12:44:18 crc kubenswrapper[5052]: I1203 12:44:18.433202 5052 scope.go:117] "RemoveContainer" containerID="3061b65a3ae17922aab71a0cb8765f84a54186703f7ce440b99b477a35e83b9d" Dec 03 12:44:18 crc kubenswrapper[5052]: E1203 12:44:18.433736 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3061b65a3ae17922aab71a0cb8765f84a54186703f7ce440b99b477a35e83b9d\": container with ID starting with 3061b65a3ae17922aab71a0cb8765f84a54186703f7ce440b99b477a35e83b9d not found: ID does not exist" containerID="3061b65a3ae17922aab71a0cb8765f84a54186703f7ce440b99b477a35e83b9d" Dec 03 12:44:18 crc kubenswrapper[5052]: I1203 12:44:18.433758 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/21de3002-be4f-4497-805c-7ef8205f03ed-kube-api-access-v2p79" (OuterVolumeSpecName: "kube-api-access-v2p79") pod "21de3002-be4f-4497-805c-7ef8205f03ed" (UID: "21de3002-be4f-4497-805c-7ef8205f03ed"). InnerVolumeSpecName "kube-api-access-v2p79". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:44:18 crc kubenswrapper[5052]: I1203 12:44:18.433772 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3061b65a3ae17922aab71a0cb8765f84a54186703f7ce440b99b477a35e83b9d"} err="failed to get container status \"3061b65a3ae17922aab71a0cb8765f84a54186703f7ce440b99b477a35e83b9d\": rpc error: code = NotFound desc = could not find container \"3061b65a3ae17922aab71a0cb8765f84a54186703f7ce440b99b477a35e83b9d\": container with ID starting with 3061b65a3ae17922aab71a0cb8765f84a54186703f7ce440b99b477a35e83b9d not found: ID does not exist" Dec 03 12:44:18 crc kubenswrapper[5052]: I1203 12:44:18.433793 5052 scope.go:117] "RemoveContainer" containerID="7df935dc5104efe7c030b20df59e99dc66141fc7489b85395c0c5c96399d83f9" Dec 03 12:44:18 crc kubenswrapper[5052]: I1203 12:44:18.488791 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/21de3002-be4f-4497-805c-7ef8205f03ed-catalog-content" (OuterVolumeSpecName: "catalog-content") pod "21de3002-be4f-4497-805c-7ef8205f03ed" (UID: "21de3002-be4f-4497-805c-7ef8205f03ed"). InnerVolumeSpecName "catalog-content". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:44:18 crc kubenswrapper[5052]: I1203 12:44:18.523852 5052 scope.go:117] "RemoveContainer" containerID="8763dbc208c3681c490f71173ed527e579f4e7fb1019eb99754c623ac4ca2738" Dec 03 12:44:18 crc kubenswrapper[5052]: I1203 12:44:18.530409 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-v2p79\" (UniqueName: \"kubernetes.io/projected/21de3002-be4f-4497-805c-7ef8205f03ed-kube-api-access-v2p79\") on node \"crc\" DevicePath \"\"" Dec 03 12:44:18 crc kubenswrapper[5052]: I1203 12:44:18.530444 5052 reconciler_common.go:293] "Volume detached for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/21de3002-be4f-4497-805c-7ef8205f03ed-utilities\") on node \"crc\" DevicePath \"\"" Dec 03 12:44:18 crc kubenswrapper[5052]: I1203 12:44:18.530455 5052 reconciler_common.go:293] "Volume detached for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/21de3002-be4f-4497-805c-7ef8205f03ed-catalog-content\") on node \"crc\" DevicePath \"\"" Dec 03 12:44:18 crc kubenswrapper[5052]: I1203 12:44:18.558875 5052 scope.go:117] "RemoveContainer" containerID="52e918cfb45febd349a692a34fb1794f56f4d89e4cc94e58a3a3235a8c6e8719" Dec 03 12:44:18 crc kubenswrapper[5052]: I1203 12:44:18.618072 5052 scope.go:117] "RemoveContainer" containerID="7df935dc5104efe7c030b20df59e99dc66141fc7489b85395c0c5c96399d83f9" Dec 03 12:44:18 crc kubenswrapper[5052]: E1203 12:44:18.618643 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7df935dc5104efe7c030b20df59e99dc66141fc7489b85395c0c5c96399d83f9\": container with ID starting with 7df935dc5104efe7c030b20df59e99dc66141fc7489b85395c0c5c96399d83f9 not found: ID does not exist" containerID="7df935dc5104efe7c030b20df59e99dc66141fc7489b85395c0c5c96399d83f9" Dec 03 12:44:18 crc kubenswrapper[5052]: I1203 12:44:18.618758 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7df935dc5104efe7c030b20df59e99dc66141fc7489b85395c0c5c96399d83f9"} err="failed to get container status \"7df935dc5104efe7c030b20df59e99dc66141fc7489b85395c0c5c96399d83f9\": rpc error: code = NotFound desc = could not find container \"7df935dc5104efe7c030b20df59e99dc66141fc7489b85395c0c5c96399d83f9\": container with ID starting with 7df935dc5104efe7c030b20df59e99dc66141fc7489b85395c0c5c96399d83f9 not found: ID does not exist" Dec 03 12:44:18 crc kubenswrapper[5052]: I1203 12:44:18.618848 5052 scope.go:117] "RemoveContainer" containerID="8763dbc208c3681c490f71173ed527e579f4e7fb1019eb99754c623ac4ca2738" Dec 03 12:44:18 crc kubenswrapper[5052]: E1203 12:44:18.619252 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8763dbc208c3681c490f71173ed527e579f4e7fb1019eb99754c623ac4ca2738\": container with ID starting with 8763dbc208c3681c490f71173ed527e579f4e7fb1019eb99754c623ac4ca2738 not found: ID does not exist" containerID="8763dbc208c3681c490f71173ed527e579f4e7fb1019eb99754c623ac4ca2738" Dec 03 12:44:18 crc kubenswrapper[5052]: I1203 12:44:18.619298 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8763dbc208c3681c490f71173ed527e579f4e7fb1019eb99754c623ac4ca2738"} err="failed to get container status \"8763dbc208c3681c490f71173ed527e579f4e7fb1019eb99754c623ac4ca2738\": rpc error: code = NotFound desc = could not find container \"8763dbc208c3681c490f71173ed527e579f4e7fb1019eb99754c623ac4ca2738\": container with ID starting with 8763dbc208c3681c490f71173ed527e579f4e7fb1019eb99754c623ac4ca2738 not found: ID does not exist" Dec 03 12:44:18 crc kubenswrapper[5052]: I1203 12:44:18.619324 5052 scope.go:117] "RemoveContainer" containerID="52e918cfb45febd349a692a34fb1794f56f4d89e4cc94e58a3a3235a8c6e8719" Dec 03 12:44:18 crc kubenswrapper[5052]: E1203 12:44:18.619942 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"52e918cfb45febd349a692a34fb1794f56f4d89e4cc94e58a3a3235a8c6e8719\": container with ID starting with 52e918cfb45febd349a692a34fb1794f56f4d89e4cc94e58a3a3235a8c6e8719 not found: ID does not exist" containerID="52e918cfb45febd349a692a34fb1794f56f4d89e4cc94e58a3a3235a8c6e8719" Dec 03 12:44:18 crc kubenswrapper[5052]: I1203 12:44:18.620020 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"52e918cfb45febd349a692a34fb1794f56f4d89e4cc94e58a3a3235a8c6e8719"} err="failed to get container status \"52e918cfb45febd349a692a34fb1794f56f4d89e4cc94e58a3a3235a8c6e8719\": rpc error: code = NotFound desc = could not find container \"52e918cfb45febd349a692a34fb1794f56f4d89e4cc94e58a3a3235a8c6e8719\": container with ID starting with 52e918cfb45febd349a692a34fb1794f56f4d89e4cc94e58a3a3235a8c6e8719 not found: ID does not exist" Dec 03 12:44:18 crc kubenswrapper[5052]: I1203 12:44:18.686273 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openshift-marketplace/certified-operators-k2f9v"] Dec 03 12:44:18 crc kubenswrapper[5052]: I1203 12:44:18.699093 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openshift-marketplace/certified-operators-k2f9v"] Dec 03 12:44:19 crc kubenswrapper[5052]: I1203 12:44:19.684453 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="21de3002-be4f-4497-805c-7ef8205f03ed" path="/var/lib/kubelet/pods/21de3002-be4f-4497-805c-7ef8205f03ed/volumes" Dec 03 12:44:19 crc kubenswrapper[5052]: I1203 12:44:19.685827 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c95c6426-bfd6-45b8-b35d-969ff5a88e3a" path="/var/lib/kubelet/pods/c95c6426-bfd6-45b8-b35d-969ff5a88e3a/volumes" Dec 03 12:44:23 crc kubenswrapper[5052]: E1203 12:44:23.349496 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="8ff041a03ee1701a0052612991d867cafab4a1723240a7109f4b7132da5cae68" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:44:23 crc kubenswrapper[5052]: E1203 12:44:23.353288 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="8ff041a03ee1701a0052612991d867cafab4a1723240a7109f4b7132da5cae68" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:44:23 crc kubenswrapper[5052]: E1203 12:44:23.354889 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="8ff041a03ee1701a0052612991d867cafab4a1723240a7109f4b7132da5cae68" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:44:23 crc kubenswrapper[5052]: E1203 12:44:23.355024 5052 prober.go:104] "Probe errored" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" probeType="Readiness" pod="openstack-kuttl-tests/nova-cell0-conductor-0" podUID="307199d5-acf0-4ede-b755-ed6bc25eb978" containerName="nova-cell0-conductor-conductor" Dec 03 12:44:28 crc kubenswrapper[5052]: E1203 12:44:28.349247 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="8ff041a03ee1701a0052612991d867cafab4a1723240a7109f4b7132da5cae68" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:44:28 crc kubenswrapper[5052]: E1203 12:44:28.354988 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="8ff041a03ee1701a0052612991d867cafab4a1723240a7109f4b7132da5cae68" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:44:28 crc kubenswrapper[5052]: E1203 12:44:28.356780 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="8ff041a03ee1701a0052612991d867cafab4a1723240a7109f4b7132da5cae68" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:44:28 crc kubenswrapper[5052]: E1203 12:44:28.356827 5052 prober.go:104] "Probe errored" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" probeType="Readiness" pod="openstack-kuttl-tests/nova-cell0-conductor-0" podUID="307199d5-acf0-4ede-b755-ed6bc25eb978" containerName="nova-cell0-conductor-conductor" Dec 03 12:44:29 crc kubenswrapper[5052]: I1203 12:44:29.550709 5052 scope.go:117] "RemoveContainer" containerID="75aabd113fe3f8c31645077c72c336a2d111ebe7269663fa192668bdab05f74c" Dec 03 12:44:29 crc kubenswrapper[5052]: I1203 12:44:29.581227 5052 scope.go:117] "RemoveContainer" containerID="aece19546c52d53c88212e467860c80e8703242aed2c7ccbcbed127d5a4cd806" Dec 03 12:44:29 crc kubenswrapper[5052]: I1203 12:44:29.601675 5052 scope.go:117] "RemoveContainer" containerID="2c6d15a1288e67b74a8c91f85763a3f1a0710e092f088b7defedb0435024e3ee" Dec 03 12:44:29 crc kubenswrapper[5052]: I1203 12:44:29.631592 5052 scope.go:117] "RemoveContainer" containerID="a50f4e422eaeb77c3e3ddb88c72b4b83fabdf60cb1ad6b1d7fbca14f74ef6cc3" Dec 03 12:44:30 crc kubenswrapper[5052]: I1203 12:44:30.020007 5052 pod_container_manager_linux.go:210] "Failed to delete cgroup paths" cgroupName=["kubepods","burstable","pod3f17a210-d54b-4753-91c1-2d9b97565b4e"] err="unable to destroy cgroup paths for cgroup [kubepods burstable pod3f17a210-d54b-4753-91c1-2d9b97565b4e] : Timed out while waiting for systemd to remove kubepods-burstable-pod3f17a210_d54b_4753_91c1_2d9b97565b4e.slice" Dec 03 12:44:31 crc kubenswrapper[5052]: W1203 12:44:31.336262 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc95c6426_bfd6_45b8_b35d_969ff5a88e3a.slice/crio-8ed469d4352febaa9d5ec81e6d03ac8f16b7bdb116048f36ed0cf7b38f27d5fa.scope WatchSource:0}: Error finding container 8ed469d4352febaa9d5ec81e6d03ac8f16b7bdb116048f36ed0cf7b38f27d5fa: Status 404 returned error can't find the container with id 8ed469d4352febaa9d5ec81e6d03ac8f16b7bdb116048f36ed0cf7b38f27d5fa Dec 03 12:44:31 crc kubenswrapper[5052]: I1203 12:44:31.471334 5052 generic.go:334] "Generic (PLEG): container finished" podID="307199d5-acf0-4ede-b755-ed6bc25eb978" containerID="8ff041a03ee1701a0052612991d867cafab4a1723240a7109f4b7132da5cae68" exitCode=137 Dec 03 12:44:31 crc kubenswrapper[5052]: I1203 12:44:31.471416 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-conductor-0" event={"ID":"307199d5-acf0-4ede-b755-ed6bc25eb978","Type":"ContainerDied","Data":"8ff041a03ee1701a0052612991d867cafab4a1723240a7109f4b7132da5cae68"} Dec 03 12:44:31 crc kubenswrapper[5052]: I1203 12:44:31.673588 5052 scope.go:117] "RemoveContainer" containerID="2aed1c760a326f3d008799e902b1093313df3764a57c879f0b3686410127d37d" Dec 03 12:44:31 crc kubenswrapper[5052]: E1203 12:44:31.673980 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 12:44:31 crc kubenswrapper[5052]: I1203 12:44:31.725274 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:44:31 crc kubenswrapper[5052]: I1203 12:44:31.783077 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/307199d5-acf0-4ede-b755-ed6bc25eb978-combined-ca-bundle\") pod \"307199d5-acf0-4ede-b755-ed6bc25eb978\" (UID: \"307199d5-acf0-4ede-b755-ed6bc25eb978\") " Dec 03 12:44:31 crc kubenswrapper[5052]: I1203 12:44:31.783174 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/307199d5-acf0-4ede-b755-ed6bc25eb978-config-data\") pod \"307199d5-acf0-4ede-b755-ed6bc25eb978\" (UID: \"307199d5-acf0-4ede-b755-ed6bc25eb978\") " Dec 03 12:44:31 crc kubenswrapper[5052]: I1203 12:44:31.783369 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cqzpt\" (UniqueName: \"kubernetes.io/projected/307199d5-acf0-4ede-b755-ed6bc25eb978-kube-api-access-cqzpt\") pod \"307199d5-acf0-4ede-b755-ed6bc25eb978\" (UID: \"307199d5-acf0-4ede-b755-ed6bc25eb978\") " Dec 03 12:44:31 crc kubenswrapper[5052]: I1203 12:44:31.789786 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/307199d5-acf0-4ede-b755-ed6bc25eb978-kube-api-access-cqzpt" (OuterVolumeSpecName: "kube-api-access-cqzpt") pod "307199d5-acf0-4ede-b755-ed6bc25eb978" (UID: "307199d5-acf0-4ede-b755-ed6bc25eb978"). InnerVolumeSpecName "kube-api-access-cqzpt". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:44:31 crc kubenswrapper[5052]: I1203 12:44:31.809419 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/307199d5-acf0-4ede-b755-ed6bc25eb978-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "307199d5-acf0-4ede-b755-ed6bc25eb978" (UID: "307199d5-acf0-4ede-b755-ed6bc25eb978"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:44:31 crc kubenswrapper[5052]: I1203 12:44:31.814612 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/307199d5-acf0-4ede-b755-ed6bc25eb978-config-data" (OuterVolumeSpecName: "config-data") pod "307199d5-acf0-4ede-b755-ed6bc25eb978" (UID: "307199d5-acf0-4ede-b755-ed6bc25eb978"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:44:31 crc kubenswrapper[5052]: I1203 12:44:31.885890 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/307199d5-acf0-4ede-b755-ed6bc25eb978-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:44:31 crc kubenswrapper[5052]: I1203 12:44:31.886157 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-cqzpt\" (UniqueName: \"kubernetes.io/projected/307199d5-acf0-4ede-b755-ed6bc25eb978-kube-api-access-cqzpt\") on node \"crc\" DevicePath \"\"" Dec 03 12:44:31 crc kubenswrapper[5052]: I1203 12:44:31.886248 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/307199d5-acf0-4ede-b755-ed6bc25eb978-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:44:32 crc kubenswrapper[5052]: I1203 12:44:32.483045 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-conductor-0" event={"ID":"307199d5-acf0-4ede-b755-ed6bc25eb978","Type":"ContainerDied","Data":"23ca5c590a6929a4526ebf4c967682a631cdb1c6485699801b6bcd4e218e6b1f"} Dec 03 12:44:32 crc kubenswrapper[5052]: I1203 12:44:32.483131 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:44:32 crc kubenswrapper[5052]: I1203 12:44:32.484119 5052 scope.go:117] "RemoveContainer" containerID="8ff041a03ee1701a0052612991d867cafab4a1723240a7109f4b7132da5cae68" Dec 03 12:44:32 crc kubenswrapper[5052]: I1203 12:44:32.521752 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-0"] Dec 03 12:44:32 crc kubenswrapper[5052]: I1203 12:44:32.530780 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-0"] Dec 03 12:44:32 crc kubenswrapper[5052]: I1203 12:44:32.554210 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-0"] Dec 03 12:44:32 crc kubenswrapper[5052]: E1203 12:44:32.554684 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="21de3002-be4f-4497-805c-7ef8205f03ed" containerName="extract-content" Dec 03 12:44:32 crc kubenswrapper[5052]: I1203 12:44:32.554711 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="21de3002-be4f-4497-805c-7ef8205f03ed" containerName="extract-content" Dec 03 12:44:32 crc kubenswrapper[5052]: E1203 12:44:32.554724 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="307199d5-acf0-4ede-b755-ed6bc25eb978" containerName="nova-cell0-conductor-conductor" Dec 03 12:44:32 crc kubenswrapper[5052]: I1203 12:44:32.554733 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="307199d5-acf0-4ede-b755-ed6bc25eb978" containerName="nova-cell0-conductor-conductor" Dec 03 12:44:32 crc kubenswrapper[5052]: E1203 12:44:32.554748 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="c95c6426-bfd6-45b8-b35d-969ff5a88e3a" containerName="registry-server" Dec 03 12:44:32 crc kubenswrapper[5052]: I1203 12:44:32.554755 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="c95c6426-bfd6-45b8-b35d-969ff5a88e3a" containerName="registry-server" Dec 03 12:44:32 crc kubenswrapper[5052]: E1203 12:44:32.554772 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="21de3002-be4f-4497-805c-7ef8205f03ed" containerName="extract-utilities" Dec 03 12:44:32 crc kubenswrapper[5052]: I1203 12:44:32.554779 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="21de3002-be4f-4497-805c-7ef8205f03ed" containerName="extract-utilities" Dec 03 12:44:32 crc kubenswrapper[5052]: E1203 12:44:32.554808 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="c95c6426-bfd6-45b8-b35d-969ff5a88e3a" containerName="extract-utilities" Dec 03 12:44:32 crc kubenswrapper[5052]: I1203 12:44:32.554816 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="c95c6426-bfd6-45b8-b35d-969ff5a88e3a" containerName="extract-utilities" Dec 03 12:44:32 crc kubenswrapper[5052]: E1203 12:44:32.554837 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="21de3002-be4f-4497-805c-7ef8205f03ed" containerName="registry-server" Dec 03 12:44:32 crc kubenswrapper[5052]: I1203 12:44:32.554844 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="21de3002-be4f-4497-805c-7ef8205f03ed" containerName="registry-server" Dec 03 12:44:32 crc kubenswrapper[5052]: E1203 12:44:32.554860 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="c95c6426-bfd6-45b8-b35d-969ff5a88e3a" containerName="extract-content" Dec 03 12:44:32 crc kubenswrapper[5052]: I1203 12:44:32.554868 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="c95c6426-bfd6-45b8-b35d-969ff5a88e3a" containerName="extract-content" Dec 03 12:44:32 crc kubenswrapper[5052]: I1203 12:44:32.555075 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="21de3002-be4f-4497-805c-7ef8205f03ed" containerName="registry-server" Dec 03 12:44:32 crc kubenswrapper[5052]: I1203 12:44:32.555095 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="c95c6426-bfd6-45b8-b35d-969ff5a88e3a" containerName="registry-server" Dec 03 12:44:32 crc kubenswrapper[5052]: I1203 12:44:32.555107 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="307199d5-acf0-4ede-b755-ed6bc25eb978" containerName="nova-cell0-conductor-conductor" Dec 03 12:44:32 crc kubenswrapper[5052]: I1203 12:44:32.555914 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:44:32 crc kubenswrapper[5052]: I1203 12:44:32.557777 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-nova-dockercfg-tj8z2" Dec 03 12:44:32 crc kubenswrapper[5052]: I1203 12:44:32.559641 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell0-conductor-config-data" Dec 03 12:44:32 crc kubenswrapper[5052]: I1203 12:44:32.567722 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-0"] Dec 03 12:44:32 crc kubenswrapper[5052]: I1203 12:44:32.598702 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kvktx\" (UniqueName: \"kubernetes.io/projected/f7e26e55-1426-4cd4-b169-a63868acdfaa-kube-api-access-kvktx\") pod \"nova-cell0-conductor-0\" (UID: \"f7e26e55-1426-4cd4-b169-a63868acdfaa\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:44:32 crc kubenswrapper[5052]: I1203 12:44:32.599011 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f7e26e55-1426-4cd4-b169-a63868acdfaa-combined-ca-bundle\") pod \"nova-cell0-conductor-0\" (UID: \"f7e26e55-1426-4cd4-b169-a63868acdfaa\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:44:32 crc kubenswrapper[5052]: I1203 12:44:32.599110 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/f7e26e55-1426-4cd4-b169-a63868acdfaa-config-data\") pod \"nova-cell0-conductor-0\" (UID: \"f7e26e55-1426-4cd4-b169-a63868acdfaa\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:44:32 crc kubenswrapper[5052]: I1203 12:44:32.700273 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-kvktx\" (UniqueName: \"kubernetes.io/projected/f7e26e55-1426-4cd4-b169-a63868acdfaa-kube-api-access-kvktx\") pod \"nova-cell0-conductor-0\" (UID: \"f7e26e55-1426-4cd4-b169-a63868acdfaa\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:44:32 crc kubenswrapper[5052]: I1203 12:44:32.700633 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f7e26e55-1426-4cd4-b169-a63868acdfaa-combined-ca-bundle\") pod \"nova-cell0-conductor-0\" (UID: \"f7e26e55-1426-4cd4-b169-a63868acdfaa\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:44:32 crc kubenswrapper[5052]: I1203 12:44:32.700754 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/f7e26e55-1426-4cd4-b169-a63868acdfaa-config-data\") pod \"nova-cell0-conductor-0\" (UID: \"f7e26e55-1426-4cd4-b169-a63868acdfaa\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:44:32 crc kubenswrapper[5052]: I1203 12:44:32.923738 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/f7e26e55-1426-4cd4-b169-a63868acdfaa-config-data\") pod \"nova-cell0-conductor-0\" (UID: \"f7e26e55-1426-4cd4-b169-a63868acdfaa\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:44:32 crc kubenswrapper[5052]: I1203 12:44:32.925075 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f7e26e55-1426-4cd4-b169-a63868acdfaa-combined-ca-bundle\") pod \"nova-cell0-conductor-0\" (UID: \"f7e26e55-1426-4cd4-b169-a63868acdfaa\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:44:32 crc kubenswrapper[5052]: I1203 12:44:32.931283 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-kvktx\" (UniqueName: \"kubernetes.io/projected/f7e26e55-1426-4cd4-b169-a63868acdfaa-kube-api-access-kvktx\") pod \"nova-cell0-conductor-0\" (UID: \"f7e26e55-1426-4cd4-b169-a63868acdfaa\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:44:33 crc kubenswrapper[5052]: I1203 12:44:33.184214 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:44:33 crc kubenswrapper[5052]: I1203 12:44:33.672174 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-0"] Dec 03 12:44:33 crc kubenswrapper[5052]: W1203 12:44:33.675696 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf7e26e55_1426_4cd4_b169_a63868acdfaa.slice/crio-8dbbe64b3b63836daa942aa0826cc3c439c589221e596d583650e4f4faa19595 WatchSource:0}: Error finding container 8dbbe64b3b63836daa942aa0826cc3c439c589221e596d583650e4f4faa19595: Status 404 returned error can't find the container with id 8dbbe64b3b63836daa942aa0826cc3c439c589221e596d583650e4f4faa19595 Dec 03 12:44:33 crc kubenswrapper[5052]: I1203 12:44:33.684843 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="307199d5-acf0-4ede-b755-ed6bc25eb978" path="/var/lib/kubelet/pods/307199d5-acf0-4ede-b755-ed6bc25eb978/volumes" Dec 03 12:44:34 crc kubenswrapper[5052]: I1203 12:44:34.504786 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-conductor-0" event={"ID":"f7e26e55-1426-4cd4-b169-a63868acdfaa","Type":"ContainerStarted","Data":"bbc2755b5351566cbdd1dc7fd68849cb7ea314f303defbebf723032464036311"} Dec 03 12:44:34 crc kubenswrapper[5052]: I1203 12:44:34.505230 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:44:34 crc kubenswrapper[5052]: I1203 12:44:34.505248 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-conductor-0" event={"ID":"f7e26e55-1426-4cd4-b169-a63868acdfaa","Type":"ContainerStarted","Data":"8dbbe64b3b63836daa942aa0826cc3c439c589221e596d583650e4f4faa19595"} Dec 03 12:44:34 crc kubenswrapper[5052]: I1203 12:44:34.523674 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-cell0-conductor-0" podStartSLOduration=2.523654027 podStartE2EDuration="2.523654027s" podCreationTimestamp="2025-12-03 12:44:32 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:44:34.519406775 +0000 UTC m=+5065.107648539" watchObservedRunningTime="2025-12-03 12:44:34.523654027 +0000 UTC m=+5065.111895791" Dec 03 12:44:36 crc kubenswrapper[5052]: I1203 12:44:36.580531 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:44:38 crc kubenswrapper[5052]: I1203 12:44:38.214866 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:44:38 crc kubenswrapper[5052]: I1203 12:44:38.690345 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell0-cell-mapping-jq8w4"] Dec 03 12:44:38 crc kubenswrapper[5052]: I1203 12:44:38.692225 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-cell-mapping-jq8w4" Dec 03 12:44:38 crc kubenswrapper[5052]: I1203 12:44:38.693996 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell0-manage-scripts" Dec 03 12:44:38 crc kubenswrapper[5052]: I1203 12:44:38.694214 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell0-manage-config-data" Dec 03 12:44:38 crc kubenswrapper[5052]: I1203 12:44:38.700966 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell0-cell-mapping-jq8w4"] Dec 03 12:44:38 crc kubenswrapper[5052]: I1203 12:44:38.717474 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/8a321082-482c-4805-b252-88a304bb19ec-config-data\") pod \"nova-cell0-cell-mapping-jq8w4\" (UID: \"8a321082-482c-4805-b252-88a304bb19ec\") " pod="openstack-kuttl-tests/nova-cell0-cell-mapping-jq8w4" Dec 03 12:44:38 crc kubenswrapper[5052]: I1203 12:44:38.717779 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8a321082-482c-4805-b252-88a304bb19ec-combined-ca-bundle\") pod \"nova-cell0-cell-mapping-jq8w4\" (UID: \"8a321082-482c-4805-b252-88a304bb19ec\") " pod="openstack-kuttl-tests/nova-cell0-cell-mapping-jq8w4" Dec 03 12:44:38 crc kubenswrapper[5052]: I1203 12:44:38.717933 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9br48\" (UniqueName: \"kubernetes.io/projected/8a321082-482c-4805-b252-88a304bb19ec-kube-api-access-9br48\") pod \"nova-cell0-cell-mapping-jq8w4\" (UID: \"8a321082-482c-4805-b252-88a304bb19ec\") " pod="openstack-kuttl-tests/nova-cell0-cell-mapping-jq8w4" Dec 03 12:44:38 crc kubenswrapper[5052]: I1203 12:44:38.718165 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/8a321082-482c-4805-b252-88a304bb19ec-scripts\") pod \"nova-cell0-cell-mapping-jq8w4\" (UID: \"8a321082-482c-4805-b252-88a304bb19ec\") " pod="openstack-kuttl-tests/nova-cell0-cell-mapping-jq8w4" Dec 03 12:44:38 crc kubenswrapper[5052]: I1203 12:44:38.819776 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/8a321082-482c-4805-b252-88a304bb19ec-config-data\") pod \"nova-cell0-cell-mapping-jq8w4\" (UID: \"8a321082-482c-4805-b252-88a304bb19ec\") " pod="openstack-kuttl-tests/nova-cell0-cell-mapping-jq8w4" Dec 03 12:44:38 crc kubenswrapper[5052]: I1203 12:44:38.819840 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8a321082-482c-4805-b252-88a304bb19ec-combined-ca-bundle\") pod \"nova-cell0-cell-mapping-jq8w4\" (UID: \"8a321082-482c-4805-b252-88a304bb19ec\") " pod="openstack-kuttl-tests/nova-cell0-cell-mapping-jq8w4" Dec 03 12:44:38 crc kubenswrapper[5052]: I1203 12:44:38.819886 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-9br48\" (UniqueName: \"kubernetes.io/projected/8a321082-482c-4805-b252-88a304bb19ec-kube-api-access-9br48\") pod \"nova-cell0-cell-mapping-jq8w4\" (UID: \"8a321082-482c-4805-b252-88a304bb19ec\") " pod="openstack-kuttl-tests/nova-cell0-cell-mapping-jq8w4" Dec 03 12:44:38 crc kubenswrapper[5052]: I1203 12:44:38.820011 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/8a321082-482c-4805-b252-88a304bb19ec-scripts\") pod \"nova-cell0-cell-mapping-jq8w4\" (UID: \"8a321082-482c-4805-b252-88a304bb19ec\") " pod="openstack-kuttl-tests/nova-cell0-cell-mapping-jq8w4" Dec 03 12:44:38 crc kubenswrapper[5052]: I1203 12:44:38.829119 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8a321082-482c-4805-b252-88a304bb19ec-combined-ca-bundle\") pod \"nova-cell0-cell-mapping-jq8w4\" (UID: \"8a321082-482c-4805-b252-88a304bb19ec\") " pod="openstack-kuttl-tests/nova-cell0-cell-mapping-jq8w4" Dec 03 12:44:38 crc kubenswrapper[5052]: I1203 12:44:38.831029 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/8a321082-482c-4805-b252-88a304bb19ec-scripts\") pod \"nova-cell0-cell-mapping-jq8w4\" (UID: \"8a321082-482c-4805-b252-88a304bb19ec\") " pod="openstack-kuttl-tests/nova-cell0-cell-mapping-jq8w4" Dec 03 12:44:38 crc kubenswrapper[5052]: I1203 12:44:38.835327 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/8a321082-482c-4805-b252-88a304bb19ec-config-data\") pod \"nova-cell0-cell-mapping-jq8w4\" (UID: \"8a321082-482c-4805-b252-88a304bb19ec\") " pod="openstack-kuttl-tests/nova-cell0-cell-mapping-jq8w4" Dec 03 12:44:38 crc kubenswrapper[5052]: I1203 12:44:38.854823 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-9br48\" (UniqueName: \"kubernetes.io/projected/8a321082-482c-4805-b252-88a304bb19ec-kube-api-access-9br48\") pod \"nova-cell0-cell-mapping-jq8w4\" (UID: \"8a321082-482c-4805-b252-88a304bb19ec\") " pod="openstack-kuttl-tests/nova-cell0-cell-mapping-jq8w4" Dec 03 12:44:38 crc kubenswrapper[5052]: I1203 12:44:38.871523 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:44:38 crc kubenswrapper[5052]: I1203 12:44:38.873679 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:44:38 crc kubenswrapper[5052]: I1203 12:44:38.876020 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-api-config-data" Dec 03 12:44:38 crc kubenswrapper[5052]: I1203 12:44:38.887237 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:44:38 crc kubenswrapper[5052]: I1203 12:44:38.921278 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/a456264f-61b5-4399-b0ec-98034fbf6523-logs\") pod \"nova-api-0\" (UID: \"a456264f-61b5-4399-b0ec-98034fbf6523\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:44:38 crc kubenswrapper[5052]: I1203 12:44:38.921401 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a456264f-61b5-4399-b0ec-98034fbf6523-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"a456264f-61b5-4399-b0ec-98034fbf6523\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:44:38 crc kubenswrapper[5052]: I1203 12:44:38.921502 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bpmkf\" (UniqueName: \"kubernetes.io/projected/a456264f-61b5-4399-b0ec-98034fbf6523-kube-api-access-bpmkf\") pod \"nova-api-0\" (UID: \"a456264f-61b5-4399-b0ec-98034fbf6523\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:44:38 crc kubenswrapper[5052]: I1203 12:44:38.921541 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a456264f-61b5-4399-b0ec-98034fbf6523-config-data\") pod \"nova-api-0\" (UID: \"a456264f-61b5-4399-b0ec-98034fbf6523\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:44:38 crc kubenswrapper[5052]: I1203 12:44:38.969359 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:44:38 crc kubenswrapper[5052]: I1203 12:44:38.970996 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:44:38 crc kubenswrapper[5052]: I1203 12:44:38.977942 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-metadata-config-data" Dec 03 12:44:38 crc kubenswrapper[5052]: I1203 12:44:38.991418 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:44:39 crc kubenswrapper[5052]: I1203 12:44:39.007571 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:44:39 crc kubenswrapper[5052]: I1203 12:44:39.008844 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:44:39 crc kubenswrapper[5052]: I1203 12:44:39.013867 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-scheduler-config-data" Dec 03 12:44:39 crc kubenswrapper[5052]: I1203 12:44:39.023229 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-bpmkf\" (UniqueName: \"kubernetes.io/projected/a456264f-61b5-4399-b0ec-98034fbf6523-kube-api-access-bpmkf\") pod \"nova-api-0\" (UID: \"a456264f-61b5-4399-b0ec-98034fbf6523\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:44:39 crc kubenswrapper[5052]: I1203 12:44:39.023463 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a456264f-61b5-4399-b0ec-98034fbf6523-config-data\") pod \"nova-api-0\" (UID: \"a456264f-61b5-4399-b0ec-98034fbf6523\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:44:39 crc kubenswrapper[5052]: I1203 12:44:39.023701 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/a456264f-61b5-4399-b0ec-98034fbf6523-logs\") pod \"nova-api-0\" (UID: \"a456264f-61b5-4399-b0ec-98034fbf6523\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:44:39 crc kubenswrapper[5052]: I1203 12:44:39.023921 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/21af962a-04e3-4df3-9d30-53577d39619b-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"21af962a-04e3-4df3-9d30-53577d39619b\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:44:39 crc kubenswrapper[5052]: I1203 12:44:39.024050 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/21af962a-04e3-4df3-9d30-53577d39619b-logs\") pod \"nova-metadata-0\" (UID: \"21af962a-04e3-4df3-9d30-53577d39619b\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:44:39 crc kubenswrapper[5052]: I1203 12:44:39.024138 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:44:39 crc kubenswrapper[5052]: I1203 12:44:39.024260 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a456264f-61b5-4399-b0ec-98034fbf6523-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"a456264f-61b5-4399-b0ec-98034fbf6523\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:44:39 crc kubenswrapper[5052]: I1203 12:44:39.024400 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/21af962a-04e3-4df3-9d30-53577d39619b-config-data\") pod \"nova-metadata-0\" (UID: \"21af962a-04e3-4df3-9d30-53577d39619b\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:44:39 crc kubenswrapper[5052]: I1203 12:44:39.024540 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8wxvx\" (UniqueName: \"kubernetes.io/projected/21af962a-04e3-4df3-9d30-53577d39619b-kube-api-access-8wxvx\") pod \"nova-metadata-0\" (UID: \"21af962a-04e3-4df3-9d30-53577d39619b\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:44:39 crc kubenswrapper[5052]: I1203 12:44:39.025109 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/a456264f-61b5-4399-b0ec-98034fbf6523-logs\") pod \"nova-api-0\" (UID: \"a456264f-61b5-4399-b0ec-98034fbf6523\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:44:39 crc kubenswrapper[5052]: I1203 12:44:39.027141 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-cell-mapping-jq8w4" Dec 03 12:44:39 crc kubenswrapper[5052]: I1203 12:44:39.028876 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a456264f-61b5-4399-b0ec-98034fbf6523-config-data\") pod \"nova-api-0\" (UID: \"a456264f-61b5-4399-b0ec-98034fbf6523\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:44:39 crc kubenswrapper[5052]: I1203 12:44:39.054208 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-bpmkf\" (UniqueName: \"kubernetes.io/projected/a456264f-61b5-4399-b0ec-98034fbf6523-kube-api-access-bpmkf\") pod \"nova-api-0\" (UID: \"a456264f-61b5-4399-b0ec-98034fbf6523\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:44:39 crc kubenswrapper[5052]: I1203 12:44:39.054396 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a456264f-61b5-4399-b0ec-98034fbf6523-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"a456264f-61b5-4399-b0ec-98034fbf6523\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:44:39 crc kubenswrapper[5052]: I1203 12:44:39.087403 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 12:44:39 crc kubenswrapper[5052]: I1203 12:44:39.088868 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:44:39 crc kubenswrapper[5052]: I1203 12:44:39.091747 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell1-novncproxy-config-data" Dec 03 12:44:39 crc kubenswrapper[5052]: I1203 12:44:39.122808 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 12:44:39 crc kubenswrapper[5052]: I1203 12:44:39.126247 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/7f212f22-8e5b-41b1-9d4c-978a57ca90bd-config-data\") pod \"nova-cell1-novncproxy-0\" (UID: \"7f212f22-8e5b-41b1-9d4c-978a57ca90bd\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:44:39 crc kubenswrapper[5052]: I1203 12:44:39.126317 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/7f212f22-8e5b-41b1-9d4c-978a57ca90bd-combined-ca-bundle\") pod \"nova-cell1-novncproxy-0\" (UID: \"7f212f22-8e5b-41b1-9d4c-978a57ca90bd\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:44:39 crc kubenswrapper[5052]: I1203 12:44:39.126359 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/62c6a5e0-6e22-456b-bf69-3ca1d0357b65-config-data\") pod \"nova-scheduler-0\" (UID: \"62c6a5e0-6e22-456b-bf69-3ca1d0357b65\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:44:39 crc kubenswrapper[5052]: I1203 12:44:39.126404 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/21af962a-04e3-4df3-9d30-53577d39619b-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"21af962a-04e3-4df3-9d30-53577d39619b\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:44:39 crc kubenswrapper[5052]: I1203 12:44:39.126436 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/21af962a-04e3-4df3-9d30-53577d39619b-logs\") pod \"nova-metadata-0\" (UID: \"21af962a-04e3-4df3-9d30-53577d39619b\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:44:39 crc kubenswrapper[5052]: I1203 12:44:39.126467 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/62c6a5e0-6e22-456b-bf69-3ca1d0357b65-combined-ca-bundle\") pod \"nova-scheduler-0\" (UID: \"62c6a5e0-6e22-456b-bf69-3ca1d0357b65\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:44:39 crc kubenswrapper[5052]: I1203 12:44:39.126491 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xm2hw\" (UniqueName: \"kubernetes.io/projected/62c6a5e0-6e22-456b-bf69-3ca1d0357b65-kube-api-access-xm2hw\") pod \"nova-scheduler-0\" (UID: \"62c6a5e0-6e22-456b-bf69-3ca1d0357b65\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:44:39 crc kubenswrapper[5052]: I1203 12:44:39.130838 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zm86m\" (UniqueName: \"kubernetes.io/projected/7f212f22-8e5b-41b1-9d4c-978a57ca90bd-kube-api-access-zm86m\") pod \"nova-cell1-novncproxy-0\" (UID: \"7f212f22-8e5b-41b1-9d4c-978a57ca90bd\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:44:39 crc kubenswrapper[5052]: I1203 12:44:39.130911 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/21af962a-04e3-4df3-9d30-53577d39619b-config-data\") pod \"nova-metadata-0\" (UID: \"21af962a-04e3-4df3-9d30-53577d39619b\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:44:39 crc kubenswrapper[5052]: I1203 12:44:39.130936 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-8wxvx\" (UniqueName: \"kubernetes.io/projected/21af962a-04e3-4df3-9d30-53577d39619b-kube-api-access-8wxvx\") pod \"nova-metadata-0\" (UID: \"21af962a-04e3-4df3-9d30-53577d39619b\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:44:39 crc kubenswrapper[5052]: I1203 12:44:39.132292 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/21af962a-04e3-4df3-9d30-53577d39619b-logs\") pod \"nova-metadata-0\" (UID: \"21af962a-04e3-4df3-9d30-53577d39619b\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:44:39 crc kubenswrapper[5052]: I1203 12:44:39.143976 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/21af962a-04e3-4df3-9d30-53577d39619b-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"21af962a-04e3-4df3-9d30-53577d39619b\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:44:39 crc kubenswrapper[5052]: I1203 12:44:39.158652 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/21af962a-04e3-4df3-9d30-53577d39619b-config-data\") pod \"nova-metadata-0\" (UID: \"21af962a-04e3-4df3-9d30-53577d39619b\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:44:39 crc kubenswrapper[5052]: I1203 12:44:39.173083 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-8wxvx\" (UniqueName: \"kubernetes.io/projected/21af962a-04e3-4df3-9d30-53577d39619b-kube-api-access-8wxvx\") pod \"nova-metadata-0\" (UID: \"21af962a-04e3-4df3-9d30-53577d39619b\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:44:39 crc kubenswrapper[5052]: I1203 12:44:39.235575 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:44:39 crc kubenswrapper[5052]: I1203 12:44:39.271201 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/7f212f22-8e5b-41b1-9d4c-978a57ca90bd-config-data\") pod \"nova-cell1-novncproxy-0\" (UID: \"7f212f22-8e5b-41b1-9d4c-978a57ca90bd\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:44:39 crc kubenswrapper[5052]: I1203 12:44:39.271483 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/7f212f22-8e5b-41b1-9d4c-978a57ca90bd-combined-ca-bundle\") pod \"nova-cell1-novncproxy-0\" (UID: \"7f212f22-8e5b-41b1-9d4c-978a57ca90bd\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:44:39 crc kubenswrapper[5052]: I1203 12:44:39.271572 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/62c6a5e0-6e22-456b-bf69-3ca1d0357b65-config-data\") pod \"nova-scheduler-0\" (UID: \"62c6a5e0-6e22-456b-bf69-3ca1d0357b65\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:44:39 crc kubenswrapper[5052]: I1203 12:44:39.271677 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/62c6a5e0-6e22-456b-bf69-3ca1d0357b65-combined-ca-bundle\") pod \"nova-scheduler-0\" (UID: \"62c6a5e0-6e22-456b-bf69-3ca1d0357b65\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:44:39 crc kubenswrapper[5052]: I1203 12:44:39.271707 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-xm2hw\" (UniqueName: \"kubernetes.io/projected/62c6a5e0-6e22-456b-bf69-3ca1d0357b65-kube-api-access-xm2hw\") pod \"nova-scheduler-0\" (UID: \"62c6a5e0-6e22-456b-bf69-3ca1d0357b65\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:44:39 crc kubenswrapper[5052]: I1203 12:44:39.271801 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-zm86m\" (UniqueName: \"kubernetes.io/projected/7f212f22-8e5b-41b1-9d4c-978a57ca90bd-kube-api-access-zm86m\") pod \"nova-cell1-novncproxy-0\" (UID: \"7f212f22-8e5b-41b1-9d4c-978a57ca90bd\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:44:39 crc kubenswrapper[5052]: I1203 12:44:39.293360 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/7f212f22-8e5b-41b1-9d4c-978a57ca90bd-config-data\") pod \"nova-cell1-novncproxy-0\" (UID: \"7f212f22-8e5b-41b1-9d4c-978a57ca90bd\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:44:39 crc kubenswrapper[5052]: I1203 12:44:39.295070 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/7f212f22-8e5b-41b1-9d4c-978a57ca90bd-combined-ca-bundle\") pod \"nova-cell1-novncproxy-0\" (UID: \"7f212f22-8e5b-41b1-9d4c-978a57ca90bd\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:44:39 crc kubenswrapper[5052]: I1203 12:44:39.295451 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/62c6a5e0-6e22-456b-bf69-3ca1d0357b65-combined-ca-bundle\") pod \"nova-scheduler-0\" (UID: \"62c6a5e0-6e22-456b-bf69-3ca1d0357b65\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:44:39 crc kubenswrapper[5052]: I1203 12:44:39.295543 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:44:39 crc kubenswrapper[5052]: I1203 12:44:39.300176 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/62c6a5e0-6e22-456b-bf69-3ca1d0357b65-config-data\") pod \"nova-scheduler-0\" (UID: \"62c6a5e0-6e22-456b-bf69-3ca1d0357b65\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:44:39 crc kubenswrapper[5052]: I1203 12:44:39.302669 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-zm86m\" (UniqueName: \"kubernetes.io/projected/7f212f22-8e5b-41b1-9d4c-978a57ca90bd-kube-api-access-zm86m\") pod \"nova-cell1-novncproxy-0\" (UID: \"7f212f22-8e5b-41b1-9d4c-978a57ca90bd\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:44:39 crc kubenswrapper[5052]: I1203 12:44:39.315164 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-xm2hw\" (UniqueName: \"kubernetes.io/projected/62c6a5e0-6e22-456b-bf69-3ca1d0357b65-kube-api-access-xm2hw\") pod \"nova-scheduler-0\" (UID: \"62c6a5e0-6e22-456b-bf69-3ca1d0357b65\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:44:39 crc kubenswrapper[5052]: I1203 12:44:39.445097 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:44:39 crc kubenswrapper[5052]: I1203 12:44:39.523462 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:44:39 crc kubenswrapper[5052]: I1203 12:44:39.704776 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell0-cell-mapping-jq8w4"] Dec 03 12:44:39 crc kubenswrapper[5052]: I1203 12:44:39.842254 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:44:39 crc kubenswrapper[5052]: W1203 12:44:39.881046 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda456264f_61b5_4399_b0ec_98034fbf6523.slice/crio-d362b688f63a81bdd27907e4ed69571ef3149d8831d6e41961b15dd2bfce1391 WatchSource:0}: Error finding container d362b688f63a81bdd27907e4ed69571ef3149d8831d6e41961b15dd2bfce1391: Status 404 returned error can't find the container with id d362b688f63a81bdd27907e4ed69571ef3149d8831d6e41961b15dd2bfce1391 Dec 03 12:44:39 crc kubenswrapper[5052]: I1203 12:44:39.998903 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:44:40 crc kubenswrapper[5052]: I1203 12:44:40.123160 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-db-sync-pp5gn"] Dec 03 12:44:40 crc kubenswrapper[5052]: I1203 12:44:40.134385 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-pp5gn" Dec 03 12:44:40 crc kubenswrapper[5052]: I1203 12:44:40.137762 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell1-conductor-config-data" Dec 03 12:44:40 crc kubenswrapper[5052]: I1203 12:44:40.139956 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell1-conductor-scripts" Dec 03 12:44:40 crc kubenswrapper[5052]: I1203 12:44:40.143804 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-db-sync-pp5gn"] Dec 03 12:44:40 crc kubenswrapper[5052]: I1203 12:44:40.170494 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:44:40 crc kubenswrapper[5052]: I1203 12:44:40.194788 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/8996a798-f16d-47b8-98c0-7693ccee2f79-scripts\") pod \"nova-cell1-conductor-db-sync-pp5gn\" (UID: \"8996a798-f16d-47b8-98c0-7693ccee2f79\") " pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-pp5gn" Dec 03 12:44:40 crc kubenswrapper[5052]: I1203 12:44:40.194842 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j7wjk\" (UniqueName: \"kubernetes.io/projected/8996a798-f16d-47b8-98c0-7693ccee2f79-kube-api-access-j7wjk\") pod \"nova-cell1-conductor-db-sync-pp5gn\" (UID: \"8996a798-f16d-47b8-98c0-7693ccee2f79\") " pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-pp5gn" Dec 03 12:44:40 crc kubenswrapper[5052]: I1203 12:44:40.194895 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/8996a798-f16d-47b8-98c0-7693ccee2f79-config-data\") pod \"nova-cell1-conductor-db-sync-pp5gn\" (UID: \"8996a798-f16d-47b8-98c0-7693ccee2f79\") " pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-pp5gn" Dec 03 12:44:40 crc kubenswrapper[5052]: I1203 12:44:40.194959 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8996a798-f16d-47b8-98c0-7693ccee2f79-combined-ca-bundle\") pod \"nova-cell1-conductor-db-sync-pp5gn\" (UID: \"8996a798-f16d-47b8-98c0-7693ccee2f79\") " pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-pp5gn" Dec 03 12:44:40 crc kubenswrapper[5052]: I1203 12:44:40.222716 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 12:44:40 crc kubenswrapper[5052]: I1203 12:44:40.296877 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/8996a798-f16d-47b8-98c0-7693ccee2f79-scripts\") pod \"nova-cell1-conductor-db-sync-pp5gn\" (UID: \"8996a798-f16d-47b8-98c0-7693ccee2f79\") " pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-pp5gn" Dec 03 12:44:40 crc kubenswrapper[5052]: I1203 12:44:40.297916 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-j7wjk\" (UniqueName: \"kubernetes.io/projected/8996a798-f16d-47b8-98c0-7693ccee2f79-kube-api-access-j7wjk\") pod \"nova-cell1-conductor-db-sync-pp5gn\" (UID: \"8996a798-f16d-47b8-98c0-7693ccee2f79\") " pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-pp5gn" Dec 03 12:44:40 crc kubenswrapper[5052]: I1203 12:44:40.298145 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/8996a798-f16d-47b8-98c0-7693ccee2f79-config-data\") pod \"nova-cell1-conductor-db-sync-pp5gn\" (UID: \"8996a798-f16d-47b8-98c0-7693ccee2f79\") " pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-pp5gn" Dec 03 12:44:40 crc kubenswrapper[5052]: I1203 12:44:40.298379 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8996a798-f16d-47b8-98c0-7693ccee2f79-combined-ca-bundle\") pod \"nova-cell1-conductor-db-sync-pp5gn\" (UID: \"8996a798-f16d-47b8-98c0-7693ccee2f79\") " pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-pp5gn" Dec 03 12:44:40 crc kubenswrapper[5052]: I1203 12:44:40.562140 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"a456264f-61b5-4399-b0ec-98034fbf6523","Type":"ContainerStarted","Data":"d362b688f63a81bdd27907e4ed69571ef3149d8831d6e41961b15dd2bfce1391"} Dec 03 12:44:40 crc kubenswrapper[5052]: I1203 12:44:40.563998 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-cell-mapping-jq8w4" event={"ID":"8a321082-482c-4805-b252-88a304bb19ec","Type":"ContainerStarted","Data":"a9b87dccd0e7dcc3aa151cb7ead35d696cd2b5970c1fdb43b6dea2d3e0aee83d"} Dec 03 12:44:40 crc kubenswrapper[5052]: I1203 12:44:40.564451 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-cell-mapping-jq8w4" event={"ID":"8a321082-482c-4805-b252-88a304bb19ec","Type":"ContainerStarted","Data":"adbc3b5d2683bcbfc77ec7902bc3342730a422eab4b7e23b156c82ecf9f63883"} Dec 03 12:44:40 crc kubenswrapper[5052]: I1203 12:44:40.565098 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"62c6a5e0-6e22-456b-bf69-3ca1d0357b65","Type":"ContainerStarted","Data":"87db4bdda192df4693b10e926e0e336b8213e8eae657ef6d1333181a064f7c31"} Dec 03 12:44:40 crc kubenswrapper[5052]: I1203 12:44:40.566401 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"21af962a-04e3-4df3-9d30-53577d39619b","Type":"ContainerStarted","Data":"d2b0cde237802d5bc3fc424f9e7d217029867b22ef9eb16bc6893b182d160ab9"} Dec 03 12:44:40 crc kubenswrapper[5052]: I1203 12:44:40.582971 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-cell0-cell-mapping-jq8w4" podStartSLOduration=2.582955263 podStartE2EDuration="2.582955263s" podCreationTimestamp="2025-12-03 12:44:38 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:44:40.580532713 +0000 UTC m=+5071.168774497" watchObservedRunningTime="2025-12-03 12:44:40.582955263 +0000 UTC m=+5071.171197017" Dec 03 12:44:40 crc kubenswrapper[5052]: I1203 12:44:40.725071 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/8996a798-f16d-47b8-98c0-7693ccee2f79-scripts\") pod \"nova-cell1-conductor-db-sync-pp5gn\" (UID: \"8996a798-f16d-47b8-98c0-7693ccee2f79\") " pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-pp5gn" Dec 03 12:44:40 crc kubenswrapper[5052]: I1203 12:44:40.726180 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/8996a798-f16d-47b8-98c0-7693ccee2f79-config-data\") pod \"nova-cell1-conductor-db-sync-pp5gn\" (UID: \"8996a798-f16d-47b8-98c0-7693ccee2f79\") " pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-pp5gn" Dec 03 12:44:40 crc kubenswrapper[5052]: I1203 12:44:40.726713 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8996a798-f16d-47b8-98c0-7693ccee2f79-combined-ca-bundle\") pod \"nova-cell1-conductor-db-sync-pp5gn\" (UID: \"8996a798-f16d-47b8-98c0-7693ccee2f79\") " pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-pp5gn" Dec 03 12:44:40 crc kubenswrapper[5052]: I1203 12:44:40.727224 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-j7wjk\" (UniqueName: \"kubernetes.io/projected/8996a798-f16d-47b8-98c0-7693ccee2f79-kube-api-access-j7wjk\") pod \"nova-cell1-conductor-db-sync-pp5gn\" (UID: \"8996a798-f16d-47b8-98c0-7693ccee2f79\") " pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-pp5gn" Dec 03 12:44:40 crc kubenswrapper[5052]: W1203 12:44:40.729979 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7f212f22_8e5b_41b1_9d4c_978a57ca90bd.slice/crio-f9b614efde8eeaabf9fde672515f1dba3d0491570e13bb5ad690e51e2bda0047 WatchSource:0}: Error finding container f9b614efde8eeaabf9fde672515f1dba3d0491570e13bb5ad690e51e2bda0047: Status 404 returned error can't find the container with id f9b614efde8eeaabf9fde672515f1dba3d0491570e13bb5ad690e51e2bda0047 Dec 03 12:44:40 crc kubenswrapper[5052]: I1203 12:44:40.843748 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-pp5gn" Dec 03 12:44:41 crc kubenswrapper[5052]: I1203 12:44:41.385079 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-db-sync-pp5gn"] Dec 03 12:44:41 crc kubenswrapper[5052]: W1203 12:44:41.387174 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8996a798_f16d_47b8_98c0_7693ccee2f79.slice/crio-6f5a40c27b858f0f543e0e6e4f2944ff005725a9f60204b817ef7d013c090cc9 WatchSource:0}: Error finding container 6f5a40c27b858f0f543e0e6e4f2944ff005725a9f60204b817ef7d013c090cc9: Status 404 returned error can't find the container with id 6f5a40c27b858f0f543e0e6e4f2944ff005725a9f60204b817ef7d013c090cc9 Dec 03 12:44:41 crc kubenswrapper[5052]: I1203 12:44:41.578172 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"62c6a5e0-6e22-456b-bf69-3ca1d0357b65","Type":"ContainerStarted","Data":"1dc1d18beb6efe5606641ce856ba08888e10021eaa52fe8e30fa9b5ad38c7b63"} Dec 03 12:44:41 crc kubenswrapper[5052]: I1203 12:44:41.628954 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"21af962a-04e3-4df3-9d30-53577d39619b","Type":"ContainerStarted","Data":"2a3dbcd03631ccff246521d7a1e917b03fbdf35164224a5fb6f61068181202cb"} Dec 03 12:44:41 crc kubenswrapper[5052]: I1203 12:44:41.628997 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"21af962a-04e3-4df3-9d30-53577d39619b","Type":"ContainerStarted","Data":"2a1fda56a2d2bf61ed02ebd0851c9a69d3b418e9699b468112a03a686c7cab41"} Dec 03 12:44:41 crc kubenswrapper[5052]: I1203 12:44:41.645839 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-scheduler-0" podStartSLOduration=3.645821938 podStartE2EDuration="3.645821938s" podCreationTimestamp="2025-12-03 12:44:38 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:44:41.640974919 +0000 UTC m=+5072.229216683" watchObservedRunningTime="2025-12-03 12:44:41.645821938 +0000 UTC m=+5072.234063702" Dec 03 12:44:41 crc kubenswrapper[5052]: I1203 12:44:41.651173 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-pp5gn" event={"ID":"8996a798-f16d-47b8-98c0-7693ccee2f79","Type":"ContainerStarted","Data":"0746aff23d59af40796b6fe2a1bbd7fdff16310788905f24c7ce1f98b626c483"} Dec 03 12:44:41 crc kubenswrapper[5052]: I1203 12:44:41.651220 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-pp5gn" event={"ID":"8996a798-f16d-47b8-98c0-7693ccee2f79","Type":"ContainerStarted","Data":"6f5a40c27b858f0f543e0e6e4f2944ff005725a9f60204b817ef7d013c090cc9"} Dec 03 12:44:41 crc kubenswrapper[5052]: I1203 12:44:41.671380 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-metadata-0" podStartSLOduration=3.671356128 podStartE2EDuration="3.671356128s" podCreationTimestamp="2025-12-03 12:44:38 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:44:41.660960161 +0000 UTC m=+5072.249201925" watchObservedRunningTime="2025-12-03 12:44:41.671356128 +0000 UTC m=+5072.259597892" Dec 03 12:44:41 crc kubenswrapper[5052]: I1203 12:44:41.671825 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" event={"ID":"7f212f22-8e5b-41b1-9d4c-978a57ca90bd","Type":"ContainerStarted","Data":"24e365774e9eec3dcf02f1af8b6faa0b9b9fc103cd551a655ab70dc147a1a3dd"} Dec 03 12:44:41 crc kubenswrapper[5052]: I1203 12:44:41.671887 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" event={"ID":"7f212f22-8e5b-41b1-9d4c-978a57ca90bd","Type":"ContainerStarted","Data":"f9b614efde8eeaabf9fde672515f1dba3d0491570e13bb5ad690e51e2bda0047"} Dec 03 12:44:41 crc kubenswrapper[5052]: I1203 12:44:41.684658 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-pp5gn" podStartSLOduration=1.684635398 podStartE2EDuration="1.684635398s" podCreationTimestamp="2025-12-03 12:44:40 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:44:41.678582705 +0000 UTC m=+5072.266824469" watchObservedRunningTime="2025-12-03 12:44:41.684635398 +0000 UTC m=+5072.272877162" Dec 03 12:44:41 crc kubenswrapper[5052]: I1203 12:44:41.707225 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"a456264f-61b5-4399-b0ec-98034fbf6523","Type":"ContainerStarted","Data":"9f70b2fb3d27878416e4c3be09e384f6b0fde07ec909f079075137a457fbe877"} Dec 03 12:44:41 crc kubenswrapper[5052]: I1203 12:44:41.707268 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"a456264f-61b5-4399-b0ec-98034fbf6523","Type":"ContainerStarted","Data":"20314ac2879b72c5b778670025b0d747d87526b5a82e3632a17c8086995a5d1c"} Dec 03 12:44:41 crc kubenswrapper[5052]: I1203 12:44:41.716758 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" podStartSLOduration=2.716734716 podStartE2EDuration="2.716734716s" podCreationTimestamp="2025-12-03 12:44:39 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:44:41.707563294 +0000 UTC m=+5072.295805068" watchObservedRunningTime="2025-12-03 12:44:41.716734716 +0000 UTC m=+5072.304976480" Dec 03 12:44:41 crc kubenswrapper[5052]: I1203 12:44:41.738835 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-api-0" podStartSLOduration=3.738816598 podStartE2EDuration="3.738816598s" podCreationTimestamp="2025-12-03 12:44:38 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:44:41.727340609 +0000 UTC m=+5072.315582393" watchObservedRunningTime="2025-12-03 12:44:41.738816598 +0000 UTC m=+5072.327058362" Dec 03 12:44:42 crc kubenswrapper[5052]: I1203 12:44:42.674552 5052 scope.go:117] "RemoveContainer" containerID="2aed1c760a326f3d008799e902b1093313df3764a57c879f0b3686410127d37d" Dec 03 12:44:42 crc kubenswrapper[5052]: E1203 12:44:42.675142 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 12:44:42 crc kubenswrapper[5052]: I1203 12:44:42.871444 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:44:42 crc kubenswrapper[5052]: I1203 12:44:42.916471 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 12:44:43 crc kubenswrapper[5052]: I1203 12:44:43.707846 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" podUID="7f212f22-8e5b-41b1-9d4c-978a57ca90bd" containerName="nova-cell1-novncproxy-novncproxy" containerID="cri-o://24e365774e9eec3dcf02f1af8b6faa0b9b9fc103cd551a655ab70dc147a1a3dd" gracePeriod=30 Dec 03 12:44:43 crc kubenswrapper[5052]: I1203 12:44:43.707965 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-metadata-0" podUID="21af962a-04e3-4df3-9d30-53577d39619b" containerName="nova-metadata-log" containerID="cri-o://2a1fda56a2d2bf61ed02ebd0851c9a69d3b418e9699b468112a03a686c7cab41" gracePeriod=30 Dec 03 12:44:43 crc kubenswrapper[5052]: I1203 12:44:43.708076 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-metadata-0" podUID="21af962a-04e3-4df3-9d30-53577d39619b" containerName="nova-metadata-metadata" containerID="cri-o://2a3dbcd03631ccff246521d7a1e917b03fbdf35164224a5fb6f61068181202cb" gracePeriod=30 Dec 03 12:44:44 crc kubenswrapper[5052]: I1203 12:44:44.069056 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/kube-state-metrics-0"] Dec 03 12:44:44 crc kubenswrapper[5052]: I1203 12:44:44.069741 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/kube-state-metrics-0" podUID="555b864d-15ca-4c31-894e-1323f009c8b4" containerName="kube-state-metrics" containerID="cri-o://277b668d318a45541794e33dacdc339c16a21f49597dcadafd93c6188d14b6c3" gracePeriod=30 Dec 03 12:44:44 crc kubenswrapper[5052]: I1203 12:44:44.140627 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/kube-state-metrics-0" podUID="555b864d-15ca-4c31-894e-1323f009c8b4" containerName="kube-state-metrics" probeResult="failure" output="Get \"http://10.217.1.149:8081/readyz\": dial tcp 10.217.1.149:8081: connect: connection refused" Dec 03 12:44:44 crc kubenswrapper[5052]: I1203 12:44:44.295761 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:44:44 crc kubenswrapper[5052]: I1203 12:44:44.295829 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:44:44 crc kubenswrapper[5052]: I1203 12:44:44.446980 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:44:44 crc kubenswrapper[5052]: I1203 12:44:44.527929 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:44:44 crc kubenswrapper[5052]: I1203 12:44:44.581867 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:44:44 crc kubenswrapper[5052]: I1203 12:44:44.699635 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:44:44 crc kubenswrapper[5052]: I1203 12:44:44.699730 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/21af962a-04e3-4df3-9d30-53577d39619b-combined-ca-bundle\") pod \"21af962a-04e3-4df3-9d30-53577d39619b\" (UID: \"21af962a-04e3-4df3-9d30-53577d39619b\") " Dec 03 12:44:44 crc kubenswrapper[5052]: I1203 12:44:44.700853 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/21af962a-04e3-4df3-9d30-53577d39619b-config-data\") pod \"21af962a-04e3-4df3-9d30-53577d39619b\" (UID: \"21af962a-04e3-4df3-9d30-53577d39619b\") " Dec 03 12:44:44 crc kubenswrapper[5052]: I1203 12:44:44.700978 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8wxvx\" (UniqueName: \"kubernetes.io/projected/21af962a-04e3-4df3-9d30-53577d39619b-kube-api-access-8wxvx\") pod \"21af962a-04e3-4df3-9d30-53577d39619b\" (UID: \"21af962a-04e3-4df3-9d30-53577d39619b\") " Dec 03 12:44:44 crc kubenswrapper[5052]: I1203 12:44:44.701038 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/21af962a-04e3-4df3-9d30-53577d39619b-logs\") pod \"21af962a-04e3-4df3-9d30-53577d39619b\" (UID: \"21af962a-04e3-4df3-9d30-53577d39619b\") " Dec 03 12:44:44 crc kubenswrapper[5052]: I1203 12:44:44.702375 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/21af962a-04e3-4df3-9d30-53577d39619b-logs" (OuterVolumeSpecName: "logs") pod "21af962a-04e3-4df3-9d30-53577d39619b" (UID: "21af962a-04e3-4df3-9d30-53577d39619b"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:44:44 crc kubenswrapper[5052]: I1203 12:44:44.709428 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/21af962a-04e3-4df3-9d30-53577d39619b-kube-api-access-8wxvx" (OuterVolumeSpecName: "kube-api-access-8wxvx") pod "21af962a-04e3-4df3-9d30-53577d39619b" (UID: "21af962a-04e3-4df3-9d30-53577d39619b"). InnerVolumeSpecName "kube-api-access-8wxvx". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:44:44 crc kubenswrapper[5052]: I1203 12:44:44.729106 5052 generic.go:334] "Generic (PLEG): container finished" podID="555b864d-15ca-4c31-894e-1323f009c8b4" containerID="277b668d318a45541794e33dacdc339c16a21f49597dcadafd93c6188d14b6c3" exitCode=2 Dec 03 12:44:44 crc kubenswrapper[5052]: I1203 12:44:44.729210 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/kube-state-metrics-0" event={"ID":"555b864d-15ca-4c31-894e-1323f009c8b4","Type":"ContainerDied","Data":"277b668d318a45541794e33dacdc339c16a21f49597dcadafd93c6188d14b6c3"} Dec 03 12:44:44 crc kubenswrapper[5052]: I1203 12:44:44.731485 5052 generic.go:334] "Generic (PLEG): container finished" podID="21af962a-04e3-4df3-9d30-53577d39619b" containerID="2a3dbcd03631ccff246521d7a1e917b03fbdf35164224a5fb6f61068181202cb" exitCode=0 Dec 03 12:44:44 crc kubenswrapper[5052]: I1203 12:44:44.731504 5052 generic.go:334] "Generic (PLEG): container finished" podID="21af962a-04e3-4df3-9d30-53577d39619b" containerID="2a1fda56a2d2bf61ed02ebd0851c9a69d3b418e9699b468112a03a686c7cab41" exitCode=143 Dec 03 12:44:44 crc kubenswrapper[5052]: I1203 12:44:44.731632 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"21af962a-04e3-4df3-9d30-53577d39619b","Type":"ContainerDied","Data":"2a3dbcd03631ccff246521d7a1e917b03fbdf35164224a5fb6f61068181202cb"} Dec 03 12:44:44 crc kubenswrapper[5052]: I1203 12:44:44.731666 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"21af962a-04e3-4df3-9d30-53577d39619b","Type":"ContainerDied","Data":"2a1fda56a2d2bf61ed02ebd0851c9a69d3b418e9699b468112a03a686c7cab41"} Dec 03 12:44:44 crc kubenswrapper[5052]: I1203 12:44:44.731681 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"21af962a-04e3-4df3-9d30-53577d39619b","Type":"ContainerDied","Data":"d2b0cde237802d5bc3fc424f9e7d217029867b22ef9eb16bc6893b182d160ab9"} Dec 03 12:44:44 crc kubenswrapper[5052]: I1203 12:44:44.731707 5052 scope.go:117] "RemoveContainer" containerID="2a3dbcd03631ccff246521d7a1e917b03fbdf35164224a5fb6f61068181202cb" Dec 03 12:44:44 crc kubenswrapper[5052]: I1203 12:44:44.732020 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:44:44 crc kubenswrapper[5052]: I1203 12:44:44.733133 5052 generic.go:334] "Generic (PLEG): container finished" podID="8996a798-f16d-47b8-98c0-7693ccee2f79" containerID="0746aff23d59af40796b6fe2a1bbd7fdff16310788905f24c7ce1f98b626c483" exitCode=0 Dec 03 12:44:44 crc kubenswrapper[5052]: I1203 12:44:44.733188 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-pp5gn" event={"ID":"8996a798-f16d-47b8-98c0-7693ccee2f79","Type":"ContainerDied","Data":"0746aff23d59af40796b6fe2a1bbd7fdff16310788905f24c7ce1f98b626c483"} Dec 03 12:44:44 crc kubenswrapper[5052]: I1203 12:44:44.736168 5052 generic.go:334] "Generic (PLEG): container finished" podID="7f212f22-8e5b-41b1-9d4c-978a57ca90bd" containerID="24e365774e9eec3dcf02f1af8b6faa0b9b9fc103cd551a655ab70dc147a1a3dd" exitCode=0 Dec 03 12:44:44 crc kubenswrapper[5052]: I1203 12:44:44.736331 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" event={"ID":"7f212f22-8e5b-41b1-9d4c-978a57ca90bd","Type":"ContainerDied","Data":"24e365774e9eec3dcf02f1af8b6faa0b9b9fc103cd551a655ab70dc147a1a3dd"} Dec 03 12:44:44 crc kubenswrapper[5052]: I1203 12:44:44.736454 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" event={"ID":"7f212f22-8e5b-41b1-9d4c-978a57ca90bd","Type":"ContainerDied","Data":"f9b614efde8eeaabf9fde672515f1dba3d0491570e13bb5ad690e51e2bda0047"} Dec 03 12:44:44 crc kubenswrapper[5052]: I1203 12:44:44.736604 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:44:44 crc kubenswrapper[5052]: I1203 12:44:44.761570 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:44:44 crc kubenswrapper[5052]: I1203 12:44:44.765378 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/21af962a-04e3-4df3-9d30-53577d39619b-config-data" (OuterVolumeSpecName: "config-data") pod "21af962a-04e3-4df3-9d30-53577d39619b" (UID: "21af962a-04e3-4df3-9d30-53577d39619b"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:44:44 crc kubenswrapper[5052]: I1203 12:44:44.780023 5052 scope.go:117] "RemoveContainer" containerID="2a1fda56a2d2bf61ed02ebd0851c9a69d3b418e9699b468112a03a686c7cab41" Dec 03 12:44:44 crc kubenswrapper[5052]: I1203 12:44:44.782053 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/21af962a-04e3-4df3-9d30-53577d39619b-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "21af962a-04e3-4df3-9d30-53577d39619b" (UID: "21af962a-04e3-4df3-9d30-53577d39619b"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:44:44 crc kubenswrapper[5052]: I1203 12:44:44.802927 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-khfjk\" (UniqueName: \"kubernetes.io/projected/555b864d-15ca-4c31-894e-1323f009c8b4-kube-api-access-khfjk\") pod \"555b864d-15ca-4c31-894e-1323f009c8b4\" (UID: \"555b864d-15ca-4c31-894e-1323f009c8b4\") " Dec 03 12:44:44 crc kubenswrapper[5052]: I1203 12:44:44.803054 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zm86m\" (UniqueName: \"kubernetes.io/projected/7f212f22-8e5b-41b1-9d4c-978a57ca90bd-kube-api-access-zm86m\") pod \"7f212f22-8e5b-41b1-9d4c-978a57ca90bd\" (UID: \"7f212f22-8e5b-41b1-9d4c-978a57ca90bd\") " Dec 03 12:44:44 crc kubenswrapper[5052]: I1203 12:44:44.803087 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/7f212f22-8e5b-41b1-9d4c-978a57ca90bd-combined-ca-bundle\") pod \"7f212f22-8e5b-41b1-9d4c-978a57ca90bd\" (UID: \"7f212f22-8e5b-41b1-9d4c-978a57ca90bd\") " Dec 03 12:44:44 crc kubenswrapper[5052]: I1203 12:44:44.803308 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/7f212f22-8e5b-41b1-9d4c-978a57ca90bd-config-data\") pod \"7f212f22-8e5b-41b1-9d4c-978a57ca90bd\" (UID: \"7f212f22-8e5b-41b1-9d4c-978a57ca90bd\") " Dec 03 12:44:44 crc kubenswrapper[5052]: I1203 12:44:44.804319 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/21af962a-04e3-4df3-9d30-53577d39619b-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:44:44 crc kubenswrapper[5052]: I1203 12:44:44.804345 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/21af962a-04e3-4df3-9d30-53577d39619b-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:44:44 crc kubenswrapper[5052]: I1203 12:44:44.804359 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-8wxvx\" (UniqueName: \"kubernetes.io/projected/21af962a-04e3-4df3-9d30-53577d39619b-kube-api-access-8wxvx\") on node \"crc\" DevicePath \"\"" Dec 03 12:44:44 crc kubenswrapper[5052]: I1203 12:44:44.804371 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/21af962a-04e3-4df3-9d30-53577d39619b-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:44:44 crc kubenswrapper[5052]: I1203 12:44:44.812824 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/555b864d-15ca-4c31-894e-1323f009c8b4-kube-api-access-khfjk" (OuterVolumeSpecName: "kube-api-access-khfjk") pod "555b864d-15ca-4c31-894e-1323f009c8b4" (UID: "555b864d-15ca-4c31-894e-1323f009c8b4"). InnerVolumeSpecName "kube-api-access-khfjk". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:44:44 crc kubenswrapper[5052]: I1203 12:44:44.816505 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7f212f22-8e5b-41b1-9d4c-978a57ca90bd-kube-api-access-zm86m" (OuterVolumeSpecName: "kube-api-access-zm86m") pod "7f212f22-8e5b-41b1-9d4c-978a57ca90bd" (UID: "7f212f22-8e5b-41b1-9d4c-978a57ca90bd"). InnerVolumeSpecName "kube-api-access-zm86m". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:44:44 crc kubenswrapper[5052]: I1203 12:44:44.832284 5052 scope.go:117] "RemoveContainer" containerID="2a3dbcd03631ccff246521d7a1e917b03fbdf35164224a5fb6f61068181202cb" Dec 03 12:44:44 crc kubenswrapper[5052]: E1203 12:44:44.838796 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2a3dbcd03631ccff246521d7a1e917b03fbdf35164224a5fb6f61068181202cb\": container with ID starting with 2a3dbcd03631ccff246521d7a1e917b03fbdf35164224a5fb6f61068181202cb not found: ID does not exist" containerID="2a3dbcd03631ccff246521d7a1e917b03fbdf35164224a5fb6f61068181202cb" Dec 03 12:44:44 crc kubenswrapper[5052]: I1203 12:44:44.839052 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2a3dbcd03631ccff246521d7a1e917b03fbdf35164224a5fb6f61068181202cb"} err="failed to get container status \"2a3dbcd03631ccff246521d7a1e917b03fbdf35164224a5fb6f61068181202cb\": rpc error: code = NotFound desc = could not find container \"2a3dbcd03631ccff246521d7a1e917b03fbdf35164224a5fb6f61068181202cb\": container with ID starting with 2a3dbcd03631ccff246521d7a1e917b03fbdf35164224a5fb6f61068181202cb not found: ID does not exist" Dec 03 12:44:44 crc kubenswrapper[5052]: I1203 12:44:44.839186 5052 scope.go:117] "RemoveContainer" containerID="2a1fda56a2d2bf61ed02ebd0851c9a69d3b418e9699b468112a03a686c7cab41" Dec 03 12:44:44 crc kubenswrapper[5052]: E1203 12:44:44.841238 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2a1fda56a2d2bf61ed02ebd0851c9a69d3b418e9699b468112a03a686c7cab41\": container with ID starting with 2a1fda56a2d2bf61ed02ebd0851c9a69d3b418e9699b468112a03a686c7cab41 not found: ID does not exist" containerID="2a1fda56a2d2bf61ed02ebd0851c9a69d3b418e9699b468112a03a686c7cab41" Dec 03 12:44:44 crc kubenswrapper[5052]: I1203 12:44:44.841299 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2a1fda56a2d2bf61ed02ebd0851c9a69d3b418e9699b468112a03a686c7cab41"} err="failed to get container status \"2a1fda56a2d2bf61ed02ebd0851c9a69d3b418e9699b468112a03a686c7cab41\": rpc error: code = NotFound desc = could not find container \"2a1fda56a2d2bf61ed02ebd0851c9a69d3b418e9699b468112a03a686c7cab41\": container with ID starting with 2a1fda56a2d2bf61ed02ebd0851c9a69d3b418e9699b468112a03a686c7cab41 not found: ID does not exist" Dec 03 12:44:44 crc kubenswrapper[5052]: I1203 12:44:44.841324 5052 scope.go:117] "RemoveContainer" containerID="2a3dbcd03631ccff246521d7a1e917b03fbdf35164224a5fb6f61068181202cb" Dec 03 12:44:44 crc kubenswrapper[5052]: I1203 12:44:44.844877 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2a3dbcd03631ccff246521d7a1e917b03fbdf35164224a5fb6f61068181202cb"} err="failed to get container status \"2a3dbcd03631ccff246521d7a1e917b03fbdf35164224a5fb6f61068181202cb\": rpc error: code = NotFound desc = could not find container \"2a3dbcd03631ccff246521d7a1e917b03fbdf35164224a5fb6f61068181202cb\": container with ID starting with 2a3dbcd03631ccff246521d7a1e917b03fbdf35164224a5fb6f61068181202cb not found: ID does not exist" Dec 03 12:44:44 crc kubenswrapper[5052]: I1203 12:44:44.844983 5052 scope.go:117] "RemoveContainer" containerID="2a1fda56a2d2bf61ed02ebd0851c9a69d3b418e9699b468112a03a686c7cab41" Dec 03 12:44:44 crc kubenswrapper[5052]: I1203 12:44:44.850239 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2a1fda56a2d2bf61ed02ebd0851c9a69d3b418e9699b468112a03a686c7cab41"} err="failed to get container status \"2a1fda56a2d2bf61ed02ebd0851c9a69d3b418e9699b468112a03a686c7cab41\": rpc error: code = NotFound desc = could not find container \"2a1fda56a2d2bf61ed02ebd0851c9a69d3b418e9699b468112a03a686c7cab41\": container with ID starting with 2a1fda56a2d2bf61ed02ebd0851c9a69d3b418e9699b468112a03a686c7cab41 not found: ID does not exist" Dec 03 12:44:44 crc kubenswrapper[5052]: I1203 12:44:44.850473 5052 scope.go:117] "RemoveContainer" containerID="24e365774e9eec3dcf02f1af8b6faa0b9b9fc103cd551a655ab70dc147a1a3dd" Dec 03 12:44:44 crc kubenswrapper[5052]: I1203 12:44:44.861693 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7f212f22-8e5b-41b1-9d4c-978a57ca90bd-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "7f212f22-8e5b-41b1-9d4c-978a57ca90bd" (UID: "7f212f22-8e5b-41b1-9d4c-978a57ca90bd"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:44:44 crc kubenswrapper[5052]: I1203 12:44:44.871230 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7f212f22-8e5b-41b1-9d4c-978a57ca90bd-config-data" (OuterVolumeSpecName: "config-data") pod "7f212f22-8e5b-41b1-9d4c-978a57ca90bd" (UID: "7f212f22-8e5b-41b1-9d4c-978a57ca90bd"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:44:44 crc kubenswrapper[5052]: I1203 12:44:44.906647 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-zm86m\" (UniqueName: \"kubernetes.io/projected/7f212f22-8e5b-41b1-9d4c-978a57ca90bd-kube-api-access-zm86m\") on node \"crc\" DevicePath \"\"" Dec 03 12:44:44 crc kubenswrapper[5052]: I1203 12:44:44.906727 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/7f212f22-8e5b-41b1-9d4c-978a57ca90bd-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:44:44 crc kubenswrapper[5052]: I1203 12:44:44.906740 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/7f212f22-8e5b-41b1-9d4c-978a57ca90bd-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:44:44 crc kubenswrapper[5052]: I1203 12:44:44.906753 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-khfjk\" (UniqueName: \"kubernetes.io/projected/555b864d-15ca-4c31-894e-1323f009c8b4-kube-api-access-khfjk\") on node \"crc\" DevicePath \"\"" Dec 03 12:44:44 crc kubenswrapper[5052]: I1203 12:44:44.977770 5052 scope.go:117] "RemoveContainer" containerID="24e365774e9eec3dcf02f1af8b6faa0b9b9fc103cd551a655ab70dc147a1a3dd" Dec 03 12:44:44 crc kubenswrapper[5052]: E1203 12:44:44.978284 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"24e365774e9eec3dcf02f1af8b6faa0b9b9fc103cd551a655ab70dc147a1a3dd\": container with ID starting with 24e365774e9eec3dcf02f1af8b6faa0b9b9fc103cd551a655ab70dc147a1a3dd not found: ID does not exist" containerID="24e365774e9eec3dcf02f1af8b6faa0b9b9fc103cd551a655ab70dc147a1a3dd" Dec 03 12:44:44 crc kubenswrapper[5052]: I1203 12:44:44.978321 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"24e365774e9eec3dcf02f1af8b6faa0b9b9fc103cd551a655ab70dc147a1a3dd"} err="failed to get container status \"24e365774e9eec3dcf02f1af8b6faa0b9b9fc103cd551a655ab70dc147a1a3dd\": rpc error: code = NotFound desc = could not find container \"24e365774e9eec3dcf02f1af8b6faa0b9b9fc103cd551a655ab70dc147a1a3dd\": container with ID starting with 24e365774e9eec3dcf02f1af8b6faa0b9b9fc103cd551a655ab70dc147a1a3dd not found: ID does not exist" Dec 03 12:44:45 crc kubenswrapper[5052]: I1203 12:44:45.074023 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:44:45 crc kubenswrapper[5052]: I1203 12:44:45.083270 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:44:45 crc kubenswrapper[5052]: I1203 12:44:45.096362 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 12:44:45 crc kubenswrapper[5052]: I1203 12:44:45.106626 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 12:44:45 crc kubenswrapper[5052]: I1203 12:44:45.173958 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:44:45 crc kubenswrapper[5052]: E1203 12:44:45.179430 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="21af962a-04e3-4df3-9d30-53577d39619b" containerName="nova-metadata-metadata" Dec 03 12:44:45 crc kubenswrapper[5052]: I1203 12:44:45.179479 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="21af962a-04e3-4df3-9d30-53577d39619b" containerName="nova-metadata-metadata" Dec 03 12:44:45 crc kubenswrapper[5052]: E1203 12:44:45.179560 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="7f212f22-8e5b-41b1-9d4c-978a57ca90bd" containerName="nova-cell1-novncproxy-novncproxy" Dec 03 12:44:45 crc kubenswrapper[5052]: I1203 12:44:45.179571 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f212f22-8e5b-41b1-9d4c-978a57ca90bd" containerName="nova-cell1-novncproxy-novncproxy" Dec 03 12:44:45 crc kubenswrapper[5052]: E1203 12:44:45.179616 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="555b864d-15ca-4c31-894e-1323f009c8b4" containerName="kube-state-metrics" Dec 03 12:44:45 crc kubenswrapper[5052]: I1203 12:44:45.179624 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="555b864d-15ca-4c31-894e-1323f009c8b4" containerName="kube-state-metrics" Dec 03 12:44:45 crc kubenswrapper[5052]: E1203 12:44:45.179726 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="21af962a-04e3-4df3-9d30-53577d39619b" containerName="nova-metadata-log" Dec 03 12:44:45 crc kubenswrapper[5052]: I1203 12:44:45.179738 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="21af962a-04e3-4df3-9d30-53577d39619b" containerName="nova-metadata-log" Dec 03 12:44:45 crc kubenswrapper[5052]: I1203 12:44:45.180425 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="7f212f22-8e5b-41b1-9d4c-978a57ca90bd" containerName="nova-cell1-novncproxy-novncproxy" Dec 03 12:44:45 crc kubenswrapper[5052]: I1203 12:44:45.180458 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="21af962a-04e3-4df3-9d30-53577d39619b" containerName="nova-metadata-log" Dec 03 12:44:45 crc kubenswrapper[5052]: I1203 12:44:45.180479 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="21af962a-04e3-4df3-9d30-53577d39619b" containerName="nova-metadata-metadata" Dec 03 12:44:45 crc kubenswrapper[5052]: I1203 12:44:45.180526 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="555b864d-15ca-4c31-894e-1323f009c8b4" containerName="kube-state-metrics" Dec 03 12:44:45 crc kubenswrapper[5052]: I1203 12:44:45.183215 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:44:45 crc kubenswrapper[5052]: I1203 12:44:45.186415 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-metadata-config-data" Dec 03 12:44:45 crc kubenswrapper[5052]: I1203 12:44:45.186723 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-nova-metadata-internal-svc" Dec 03 12:44:45 crc kubenswrapper[5052]: I1203 12:44:45.214941 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:44:45 crc kubenswrapper[5052]: I1203 12:44:45.234921 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 12:44:45 crc kubenswrapper[5052]: I1203 12:44:45.237422 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:44:45 crc kubenswrapper[5052]: I1203 12:44:45.240119 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell1-novncproxy-config-data" Dec 03 12:44:45 crc kubenswrapper[5052]: I1203 12:44:45.244375 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-nova-novncproxy-cell1-vencrypt" Dec 03 12:44:45 crc kubenswrapper[5052]: I1203 12:44:45.244768 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-nova-novncproxy-cell1-public-svc" Dec 03 12:44:45 crc kubenswrapper[5052]: I1203 12:44:45.257068 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 12:44:45 crc kubenswrapper[5052]: I1203 12:44:45.314479 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e5f1bd78-3552-4d2d-8029-651f33c3918a-combined-ca-bundle\") pod \"nova-cell1-novncproxy-0\" (UID: \"e5f1bd78-3552-4d2d-8029-651f33c3918a\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:44:45 crc kubenswrapper[5052]: I1203 12:44:45.314567 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"vencrypt-tls-certs\" (UniqueName: \"kubernetes.io/secret/e5f1bd78-3552-4d2d-8029-651f33c3918a-vencrypt-tls-certs\") pod \"nova-cell1-novncproxy-0\" (UID: \"e5f1bd78-3552-4d2d-8029-651f33c3918a\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:44:45 crc kubenswrapper[5052]: I1203 12:44:45.314593 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/e5f1bd78-3552-4d2d-8029-651f33c3918a-config-data\") pod \"nova-cell1-novncproxy-0\" (UID: \"e5f1bd78-3552-4d2d-8029-651f33c3918a\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:44:45 crc kubenswrapper[5052]: I1203 12:44:45.314611 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/ad32b585-41d2-4a7e-a522-069824b84d97-logs\") pod \"nova-metadata-0\" (UID: \"ad32b585-41d2-4a7e-a522-069824b84d97\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:44:45 crc kubenswrapper[5052]: I1203 12:44:45.314647 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/ad32b585-41d2-4a7e-a522-069824b84d97-config-data\") pod \"nova-metadata-0\" (UID: \"ad32b585-41d2-4a7e-a522-069824b84d97\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:44:45 crc kubenswrapper[5052]: I1203 12:44:45.314877 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pjstv\" (UniqueName: \"kubernetes.io/projected/e5f1bd78-3552-4d2d-8029-651f33c3918a-kube-api-access-pjstv\") pod \"nova-cell1-novncproxy-0\" (UID: \"e5f1bd78-3552-4d2d-8029-651f33c3918a\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:44:45 crc kubenswrapper[5052]: I1203 12:44:45.314985 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-85kbf\" (UniqueName: \"kubernetes.io/projected/ad32b585-41d2-4a7e-a522-069824b84d97-kube-api-access-85kbf\") pod \"nova-metadata-0\" (UID: \"ad32b585-41d2-4a7e-a522-069824b84d97\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:44:45 crc kubenswrapper[5052]: I1203 12:44:45.315044 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/ad32b585-41d2-4a7e-a522-069824b84d97-nova-metadata-tls-certs\") pod \"nova-metadata-0\" (UID: \"ad32b585-41d2-4a7e-a522-069824b84d97\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:44:45 crc kubenswrapper[5052]: I1203 12:44:45.315164 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nova-novncproxy-tls-certs\" (UniqueName: \"kubernetes.io/secret/e5f1bd78-3552-4d2d-8029-651f33c3918a-nova-novncproxy-tls-certs\") pod \"nova-cell1-novncproxy-0\" (UID: \"e5f1bd78-3552-4d2d-8029-651f33c3918a\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:44:45 crc kubenswrapper[5052]: I1203 12:44:45.315292 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ad32b585-41d2-4a7e-a522-069824b84d97-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"ad32b585-41d2-4a7e-a522-069824b84d97\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:44:45 crc kubenswrapper[5052]: I1203 12:44:45.419148 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ad32b585-41d2-4a7e-a522-069824b84d97-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"ad32b585-41d2-4a7e-a522-069824b84d97\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:44:45 crc kubenswrapper[5052]: I1203 12:44:45.419246 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e5f1bd78-3552-4d2d-8029-651f33c3918a-combined-ca-bundle\") pod \"nova-cell1-novncproxy-0\" (UID: \"e5f1bd78-3552-4d2d-8029-651f33c3918a\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:44:45 crc kubenswrapper[5052]: I1203 12:44:45.419288 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"vencrypt-tls-certs\" (UniqueName: \"kubernetes.io/secret/e5f1bd78-3552-4d2d-8029-651f33c3918a-vencrypt-tls-certs\") pod \"nova-cell1-novncproxy-0\" (UID: \"e5f1bd78-3552-4d2d-8029-651f33c3918a\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:44:45 crc kubenswrapper[5052]: I1203 12:44:45.419317 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/e5f1bd78-3552-4d2d-8029-651f33c3918a-config-data\") pod \"nova-cell1-novncproxy-0\" (UID: \"e5f1bd78-3552-4d2d-8029-651f33c3918a\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:44:45 crc kubenswrapper[5052]: I1203 12:44:45.419340 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/ad32b585-41d2-4a7e-a522-069824b84d97-logs\") pod \"nova-metadata-0\" (UID: \"ad32b585-41d2-4a7e-a522-069824b84d97\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:44:45 crc kubenswrapper[5052]: I1203 12:44:45.419386 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/ad32b585-41d2-4a7e-a522-069824b84d97-config-data\") pod \"nova-metadata-0\" (UID: \"ad32b585-41d2-4a7e-a522-069824b84d97\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:44:45 crc kubenswrapper[5052]: I1203 12:44:45.419427 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-pjstv\" (UniqueName: \"kubernetes.io/projected/e5f1bd78-3552-4d2d-8029-651f33c3918a-kube-api-access-pjstv\") pod \"nova-cell1-novncproxy-0\" (UID: \"e5f1bd78-3552-4d2d-8029-651f33c3918a\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:44:45 crc kubenswrapper[5052]: I1203 12:44:45.419455 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-85kbf\" (UniqueName: \"kubernetes.io/projected/ad32b585-41d2-4a7e-a522-069824b84d97-kube-api-access-85kbf\") pod \"nova-metadata-0\" (UID: \"ad32b585-41d2-4a7e-a522-069824b84d97\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:44:45 crc kubenswrapper[5052]: I1203 12:44:45.419478 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/ad32b585-41d2-4a7e-a522-069824b84d97-nova-metadata-tls-certs\") pod \"nova-metadata-0\" (UID: \"ad32b585-41d2-4a7e-a522-069824b84d97\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:44:45 crc kubenswrapper[5052]: I1203 12:44:45.419523 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"nova-novncproxy-tls-certs\" (UniqueName: \"kubernetes.io/secret/e5f1bd78-3552-4d2d-8029-651f33c3918a-nova-novncproxy-tls-certs\") pod \"nova-cell1-novncproxy-0\" (UID: \"e5f1bd78-3552-4d2d-8029-651f33c3918a\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:44:45 crc kubenswrapper[5052]: I1203 12:44:45.419950 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/ad32b585-41d2-4a7e-a522-069824b84d97-logs\") pod \"nova-metadata-0\" (UID: \"ad32b585-41d2-4a7e-a522-069824b84d97\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:44:45 crc kubenswrapper[5052]: I1203 12:44:45.424896 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e5f1bd78-3552-4d2d-8029-651f33c3918a-combined-ca-bundle\") pod \"nova-cell1-novncproxy-0\" (UID: \"e5f1bd78-3552-4d2d-8029-651f33c3918a\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:44:45 crc kubenswrapper[5052]: I1203 12:44:45.426401 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/ad32b585-41d2-4a7e-a522-069824b84d97-nova-metadata-tls-certs\") pod \"nova-metadata-0\" (UID: \"ad32b585-41d2-4a7e-a522-069824b84d97\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:44:45 crc kubenswrapper[5052]: I1203 12:44:45.428135 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"vencrypt-tls-certs\" (UniqueName: \"kubernetes.io/secret/e5f1bd78-3552-4d2d-8029-651f33c3918a-vencrypt-tls-certs\") pod \"nova-cell1-novncproxy-0\" (UID: \"e5f1bd78-3552-4d2d-8029-651f33c3918a\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:44:45 crc kubenswrapper[5052]: I1203 12:44:45.430278 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/ad32b585-41d2-4a7e-a522-069824b84d97-config-data\") pod \"nova-metadata-0\" (UID: \"ad32b585-41d2-4a7e-a522-069824b84d97\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:44:45 crc kubenswrapper[5052]: I1203 12:44:45.435056 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"nova-novncproxy-tls-certs\" (UniqueName: \"kubernetes.io/secret/e5f1bd78-3552-4d2d-8029-651f33c3918a-nova-novncproxy-tls-certs\") pod \"nova-cell1-novncproxy-0\" (UID: \"e5f1bd78-3552-4d2d-8029-651f33c3918a\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:44:45 crc kubenswrapper[5052]: I1203 12:44:45.437115 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ad32b585-41d2-4a7e-a522-069824b84d97-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"ad32b585-41d2-4a7e-a522-069824b84d97\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:44:45 crc kubenswrapper[5052]: I1203 12:44:45.438128 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/e5f1bd78-3552-4d2d-8029-651f33c3918a-config-data\") pod \"nova-cell1-novncproxy-0\" (UID: \"e5f1bd78-3552-4d2d-8029-651f33c3918a\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:44:45 crc kubenswrapper[5052]: I1203 12:44:45.441044 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-pjstv\" (UniqueName: \"kubernetes.io/projected/e5f1bd78-3552-4d2d-8029-651f33c3918a-kube-api-access-pjstv\") pod \"nova-cell1-novncproxy-0\" (UID: \"e5f1bd78-3552-4d2d-8029-651f33c3918a\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:44:45 crc kubenswrapper[5052]: I1203 12:44:45.442971 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-85kbf\" (UniqueName: \"kubernetes.io/projected/ad32b585-41d2-4a7e-a522-069824b84d97-kube-api-access-85kbf\") pod \"nova-metadata-0\" (UID: \"ad32b585-41d2-4a7e-a522-069824b84d97\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:44:45 crc kubenswrapper[5052]: I1203 12:44:45.505675 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:44:45 crc kubenswrapper[5052]: I1203 12:44:45.556906 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:44:45 crc kubenswrapper[5052]: I1203 12:44:45.699108 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="21af962a-04e3-4df3-9d30-53577d39619b" path="/var/lib/kubelet/pods/21af962a-04e3-4df3-9d30-53577d39619b/volumes" Dec 03 12:44:45 crc kubenswrapper[5052]: I1203 12:44:45.703480 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7f212f22-8e5b-41b1-9d4c-978a57ca90bd" path="/var/lib/kubelet/pods/7f212f22-8e5b-41b1-9d4c-978a57ca90bd/volumes" Dec 03 12:44:45 crc kubenswrapper[5052]: I1203 12:44:45.755145 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/kube-state-metrics-0" event={"ID":"555b864d-15ca-4c31-894e-1323f009c8b4","Type":"ContainerDied","Data":"00e8703b524502ea1f1d66dc2e576a05f32a383d2b067241b9cb107ee89939e7"} Dec 03 12:44:45 crc kubenswrapper[5052]: I1203 12:44:45.755171 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:44:45 crc kubenswrapper[5052]: I1203 12:44:45.755200 5052 scope.go:117] "RemoveContainer" containerID="277b668d318a45541794e33dacdc339c16a21f49597dcadafd93c6188d14b6c3" Dec 03 12:44:45 crc kubenswrapper[5052]: I1203 12:44:45.819372 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/kube-state-metrics-0"] Dec 03 12:44:45 crc kubenswrapper[5052]: I1203 12:44:45.840622 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/kube-state-metrics-0"] Dec 03 12:44:45 crc kubenswrapper[5052]: I1203 12:44:45.851702 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/kube-state-metrics-0"] Dec 03 12:44:45 crc kubenswrapper[5052]: I1203 12:44:45.853992 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:44:45 crc kubenswrapper[5052]: I1203 12:44:45.858598 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"kube-state-metrics-tls-config" Dec 03 12:44:45 crc kubenswrapper[5052]: I1203 12:44:45.859112 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-kube-state-metrics-svc" Dec 03 12:44:45 crc kubenswrapper[5052]: I1203 12:44:45.862170 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/kube-state-metrics-0"] Dec 03 12:44:45 crc kubenswrapper[5052]: I1203 12:44:45.938041 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/876d06e6-0187-4e3d-9183-ed2083fe5189-combined-ca-bundle\") pod \"kube-state-metrics-0\" (UID: \"876d06e6-0187-4e3d-9183-ed2083fe5189\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:44:45 crc kubenswrapper[5052]: I1203 12:44:45.938120 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls-certs\" (UniqueName: \"kubernetes.io/secret/876d06e6-0187-4e3d-9183-ed2083fe5189-kube-state-metrics-tls-certs\") pod \"kube-state-metrics-0\" (UID: \"876d06e6-0187-4e3d-9183-ed2083fe5189\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:44:45 crc kubenswrapper[5052]: I1203 12:44:45.938256 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls-config\" (UniqueName: \"kubernetes.io/secret/876d06e6-0187-4e3d-9183-ed2083fe5189-kube-state-metrics-tls-config\") pod \"kube-state-metrics-0\" (UID: \"876d06e6-0187-4e3d-9183-ed2083fe5189\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:44:45 crc kubenswrapper[5052]: I1203 12:44:45.938291 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6vmzm\" (UniqueName: \"kubernetes.io/projected/876d06e6-0187-4e3d-9183-ed2083fe5189-kube-api-access-6vmzm\") pod \"kube-state-metrics-0\" (UID: \"876d06e6-0187-4e3d-9183-ed2083fe5189\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:44:45 crc kubenswrapper[5052]: I1203 12:44:45.995045 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:44:46 crc kubenswrapper[5052]: I1203 12:44:46.040408 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls-certs\" (UniqueName: \"kubernetes.io/secret/876d06e6-0187-4e3d-9183-ed2083fe5189-kube-state-metrics-tls-certs\") pod \"kube-state-metrics-0\" (UID: \"876d06e6-0187-4e3d-9183-ed2083fe5189\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:44:46 crc kubenswrapper[5052]: I1203 12:44:46.040684 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls-config\" (UniqueName: \"kubernetes.io/secret/876d06e6-0187-4e3d-9183-ed2083fe5189-kube-state-metrics-tls-config\") pod \"kube-state-metrics-0\" (UID: \"876d06e6-0187-4e3d-9183-ed2083fe5189\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:44:46 crc kubenswrapper[5052]: I1203 12:44:46.040801 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-6vmzm\" (UniqueName: \"kubernetes.io/projected/876d06e6-0187-4e3d-9183-ed2083fe5189-kube-api-access-6vmzm\") pod \"kube-state-metrics-0\" (UID: \"876d06e6-0187-4e3d-9183-ed2083fe5189\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:44:46 crc kubenswrapper[5052]: I1203 12:44:46.040878 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/876d06e6-0187-4e3d-9183-ed2083fe5189-combined-ca-bundle\") pod \"kube-state-metrics-0\" (UID: \"876d06e6-0187-4e3d-9183-ed2083fe5189\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:44:46 crc kubenswrapper[5052]: I1203 12:44:46.047062 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/876d06e6-0187-4e3d-9183-ed2083fe5189-combined-ca-bundle\") pod \"kube-state-metrics-0\" (UID: \"876d06e6-0187-4e3d-9183-ed2083fe5189\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:44:46 crc kubenswrapper[5052]: I1203 12:44:46.047089 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls-certs\" (UniqueName: \"kubernetes.io/secret/876d06e6-0187-4e3d-9183-ed2083fe5189-kube-state-metrics-tls-certs\") pod \"kube-state-metrics-0\" (UID: \"876d06e6-0187-4e3d-9183-ed2083fe5189\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:44:46 crc kubenswrapper[5052]: I1203 12:44:46.050306 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls-config\" (UniqueName: \"kubernetes.io/secret/876d06e6-0187-4e3d-9183-ed2083fe5189-kube-state-metrics-tls-config\") pod \"kube-state-metrics-0\" (UID: \"876d06e6-0187-4e3d-9183-ed2083fe5189\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:44:46 crc kubenswrapper[5052]: I1203 12:44:46.058850 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-6vmzm\" (UniqueName: \"kubernetes.io/projected/876d06e6-0187-4e3d-9183-ed2083fe5189-kube-api-access-6vmzm\") pod \"kube-state-metrics-0\" (UID: \"876d06e6-0187-4e3d-9183-ed2083fe5189\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:44:46 crc kubenswrapper[5052]: I1203 12:44:46.153358 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 12:44:46 crc kubenswrapper[5052]: I1203 12:44:46.224370 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:44:46 crc kubenswrapper[5052]: I1203 12:44:46.271967 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-pp5gn" Dec 03 12:44:46 crc kubenswrapper[5052]: I1203 12:44:46.352013 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/8996a798-f16d-47b8-98c0-7693ccee2f79-config-data\") pod \"8996a798-f16d-47b8-98c0-7693ccee2f79\" (UID: \"8996a798-f16d-47b8-98c0-7693ccee2f79\") " Dec 03 12:44:46 crc kubenswrapper[5052]: I1203 12:44:46.352210 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j7wjk\" (UniqueName: \"kubernetes.io/projected/8996a798-f16d-47b8-98c0-7693ccee2f79-kube-api-access-j7wjk\") pod \"8996a798-f16d-47b8-98c0-7693ccee2f79\" (UID: \"8996a798-f16d-47b8-98c0-7693ccee2f79\") " Dec 03 12:44:46 crc kubenswrapper[5052]: I1203 12:44:46.352352 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8996a798-f16d-47b8-98c0-7693ccee2f79-combined-ca-bundle\") pod \"8996a798-f16d-47b8-98c0-7693ccee2f79\" (UID: \"8996a798-f16d-47b8-98c0-7693ccee2f79\") " Dec 03 12:44:46 crc kubenswrapper[5052]: I1203 12:44:46.352380 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/8996a798-f16d-47b8-98c0-7693ccee2f79-scripts\") pod \"8996a798-f16d-47b8-98c0-7693ccee2f79\" (UID: \"8996a798-f16d-47b8-98c0-7693ccee2f79\") " Dec 03 12:44:46 crc kubenswrapper[5052]: I1203 12:44:46.357058 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8996a798-f16d-47b8-98c0-7693ccee2f79-kube-api-access-j7wjk" (OuterVolumeSpecName: "kube-api-access-j7wjk") pod "8996a798-f16d-47b8-98c0-7693ccee2f79" (UID: "8996a798-f16d-47b8-98c0-7693ccee2f79"). InnerVolumeSpecName "kube-api-access-j7wjk". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:44:46 crc kubenswrapper[5052]: I1203 12:44:46.362898 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8996a798-f16d-47b8-98c0-7693ccee2f79-scripts" (OuterVolumeSpecName: "scripts") pod "8996a798-f16d-47b8-98c0-7693ccee2f79" (UID: "8996a798-f16d-47b8-98c0-7693ccee2f79"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:44:46 crc kubenswrapper[5052]: I1203 12:44:46.410100 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8996a798-f16d-47b8-98c0-7693ccee2f79-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "8996a798-f16d-47b8-98c0-7693ccee2f79" (UID: "8996a798-f16d-47b8-98c0-7693ccee2f79"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:44:46 crc kubenswrapper[5052]: I1203 12:44:46.422086 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8996a798-f16d-47b8-98c0-7693ccee2f79-config-data" (OuterVolumeSpecName: "config-data") pod "8996a798-f16d-47b8-98c0-7693ccee2f79" (UID: "8996a798-f16d-47b8-98c0-7693ccee2f79"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:44:46 crc kubenswrapper[5052]: I1203 12:44:46.454846 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-j7wjk\" (UniqueName: \"kubernetes.io/projected/8996a798-f16d-47b8-98c0-7693ccee2f79-kube-api-access-j7wjk\") on node \"crc\" DevicePath \"\"" Dec 03 12:44:46 crc kubenswrapper[5052]: I1203 12:44:46.454900 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8996a798-f16d-47b8-98c0-7693ccee2f79-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:44:46 crc kubenswrapper[5052]: I1203 12:44:46.454915 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/8996a798-f16d-47b8-98c0-7693ccee2f79-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:44:46 crc kubenswrapper[5052]: I1203 12:44:46.454928 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/8996a798-f16d-47b8-98c0-7693ccee2f79-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:44:46 crc kubenswrapper[5052]: I1203 12:44:46.644884 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:44:46 crc kubenswrapper[5052]: I1203 12:44:46.645321 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="3b455122-e3fd-44a8-baa3-2b10f856bfce" containerName="ceilometer-central-agent" containerID="cri-o://9b9f209894b73ff775c7636aa2252ab1c82d6c5abb431a90314a2d850e973805" gracePeriod=30 Dec 03 12:44:46 crc kubenswrapper[5052]: I1203 12:44:46.645354 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="3b455122-e3fd-44a8-baa3-2b10f856bfce" containerName="sg-core" containerID="cri-o://fe2a0268a0be69498dad63a9b2768a0651397fdd6a75908b40f05a1eb5621f57" gracePeriod=30 Dec 03 12:44:46 crc kubenswrapper[5052]: I1203 12:44:46.645369 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="3b455122-e3fd-44a8-baa3-2b10f856bfce" containerName="ceilometer-notification-agent" containerID="cri-o://1f02fbccdb7521208886acc6eb8caee6a28ba2f64b08fbbd8a5f9529383c29a3" gracePeriod=30 Dec 03 12:44:46 crc kubenswrapper[5052]: I1203 12:44:46.645378 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="3b455122-e3fd-44a8-baa3-2b10f856bfce" containerName="proxy-httpd" containerID="cri-o://d80485db5415a79a1bee865fce5b1dee64e47e17be467a9fdd9649b63865a438" gracePeriod=30 Dec 03 12:44:46 crc kubenswrapper[5052]: I1203 12:44:46.776981 5052 generic.go:334] "Generic (PLEG): container finished" podID="3b455122-e3fd-44a8-baa3-2b10f856bfce" containerID="fe2a0268a0be69498dad63a9b2768a0651397fdd6a75908b40f05a1eb5621f57" exitCode=2 Dec 03 12:44:46 crc kubenswrapper[5052]: I1203 12:44:46.777050 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"3b455122-e3fd-44a8-baa3-2b10f856bfce","Type":"ContainerDied","Data":"fe2a0268a0be69498dad63a9b2768a0651397fdd6a75908b40f05a1eb5621f57"} Dec 03 12:44:46 crc kubenswrapper[5052]: I1203 12:44:46.779396 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" event={"ID":"e5f1bd78-3552-4d2d-8029-651f33c3918a","Type":"ContainerStarted","Data":"0c534558ed9f09feb4924d04acb90157e0fc3201f00732c7364a3bf82d62e52e"} Dec 03 12:44:46 crc kubenswrapper[5052]: I1203 12:44:46.779499 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" event={"ID":"e5f1bd78-3552-4d2d-8029-651f33c3918a","Type":"ContainerStarted","Data":"7f01f3df1fa966ec776f19821537dd47bd449c58615b2fa1bdcf12c7377f2e11"} Dec 03 12:44:46 crc kubenswrapper[5052]: I1203 12:44:46.782296 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"ad32b585-41d2-4a7e-a522-069824b84d97","Type":"ContainerStarted","Data":"9c3c946a30940eb77b460b12f157c0b4abe50db0e3fcce354df77b7a76ef9f02"} Dec 03 12:44:46 crc kubenswrapper[5052]: I1203 12:44:46.782352 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"ad32b585-41d2-4a7e-a522-069824b84d97","Type":"ContainerStarted","Data":"62d5405e200babb8c891fb3d45e5b63d190e284a98b443c5f799e10ad934a70d"} Dec 03 12:44:46 crc kubenswrapper[5052]: I1203 12:44:46.782369 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"ad32b585-41d2-4a7e-a522-069824b84d97","Type":"ContainerStarted","Data":"17c2f36f135b55cf108d15a9c4ba5322a0b7e88070295947a9b878f3abda8f03"} Dec 03 12:44:46 crc kubenswrapper[5052]: I1203 12:44:46.784296 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-pp5gn" event={"ID":"8996a798-f16d-47b8-98c0-7693ccee2f79","Type":"ContainerDied","Data":"6f5a40c27b858f0f543e0e6e4f2944ff005725a9f60204b817ef7d013c090cc9"} Dec 03 12:44:46 crc kubenswrapper[5052]: I1203 12:44:46.784323 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-pp5gn" Dec 03 12:44:46 crc kubenswrapper[5052]: I1203 12:44:46.784340 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6f5a40c27b858f0f543e0e6e4f2944ff005725a9f60204b817ef7d013c090cc9" Dec 03 12:44:46 crc kubenswrapper[5052]: I1203 12:44:46.807460 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/kube-state-metrics-0"] Dec 03 12:44:46 crc kubenswrapper[5052]: W1203 12:44:46.810337 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod876d06e6_0187_4e3d_9183_ed2083fe5189.slice/crio-be84dd2a321e8f5a869f52c256ca8452d3f9dad861bb220ba7e72a739992e6d4 WatchSource:0}: Error finding container be84dd2a321e8f5a869f52c256ca8452d3f9dad861bb220ba7e72a739992e6d4: Status 404 returned error can't find the container with id be84dd2a321e8f5a869f52c256ca8452d3f9dad861bb220ba7e72a739992e6d4 Dec 03 12:44:46 crc kubenswrapper[5052]: I1203 12:44:46.823220 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" podStartSLOduration=1.8232027020000001 podStartE2EDuration="1.823202702s" podCreationTimestamp="2025-12-03 12:44:45 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:44:46.809936373 +0000 UTC m=+5077.398178147" watchObservedRunningTime="2025-12-03 12:44:46.823202702 +0000 UTC m=+5077.411444466" Dec 03 12:44:46 crc kubenswrapper[5052]: I1203 12:44:46.840026 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-metadata-0" podStartSLOduration=1.840004193 podStartE2EDuration="1.840004193s" podCreationTimestamp="2025-12-03 12:44:45 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:44:46.836304947 +0000 UTC m=+5077.424546721" watchObservedRunningTime="2025-12-03 12:44:46.840004193 +0000 UTC m=+5077.428245957" Dec 03 12:44:46 crc kubenswrapper[5052]: I1203 12:44:46.899278 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-0"] Dec 03 12:44:46 crc kubenswrapper[5052]: E1203 12:44:46.899703 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="8996a798-f16d-47b8-98c0-7693ccee2f79" containerName="nova-cell1-conductor-db-sync" Dec 03 12:44:46 crc kubenswrapper[5052]: I1203 12:44:46.899727 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="8996a798-f16d-47b8-98c0-7693ccee2f79" containerName="nova-cell1-conductor-db-sync" Dec 03 12:44:46 crc kubenswrapper[5052]: I1203 12:44:46.899970 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="8996a798-f16d-47b8-98c0-7693ccee2f79" containerName="nova-cell1-conductor-db-sync" Dec 03 12:44:46 crc kubenswrapper[5052]: I1203 12:44:46.900798 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:44:46 crc kubenswrapper[5052]: I1203 12:44:46.904861 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell1-conductor-config-data" Dec 03 12:44:46 crc kubenswrapper[5052]: I1203 12:44:46.913253 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-0"] Dec 03 12:44:46 crc kubenswrapper[5052]: I1203 12:44:46.968980 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/320f928d-d6b0-48b4-b315-1a6c2ecd3380-combined-ca-bundle\") pod \"nova-cell1-conductor-0\" (UID: \"320f928d-d6b0-48b4-b315-1a6c2ecd3380\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:44:46 crc kubenswrapper[5052]: I1203 12:44:46.969200 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/320f928d-d6b0-48b4-b315-1a6c2ecd3380-config-data\") pod \"nova-cell1-conductor-0\" (UID: \"320f928d-d6b0-48b4-b315-1a6c2ecd3380\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:44:46 crc kubenswrapper[5052]: I1203 12:44:46.969231 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qc2cz\" (UniqueName: \"kubernetes.io/projected/320f928d-d6b0-48b4-b315-1a6c2ecd3380-kube-api-access-qc2cz\") pod \"nova-cell1-conductor-0\" (UID: \"320f928d-d6b0-48b4-b315-1a6c2ecd3380\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:44:47 crc kubenswrapper[5052]: I1203 12:44:47.071221 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/320f928d-d6b0-48b4-b315-1a6c2ecd3380-config-data\") pod \"nova-cell1-conductor-0\" (UID: \"320f928d-d6b0-48b4-b315-1a6c2ecd3380\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:44:47 crc kubenswrapper[5052]: I1203 12:44:47.071278 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-qc2cz\" (UniqueName: \"kubernetes.io/projected/320f928d-d6b0-48b4-b315-1a6c2ecd3380-kube-api-access-qc2cz\") pod \"nova-cell1-conductor-0\" (UID: \"320f928d-d6b0-48b4-b315-1a6c2ecd3380\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:44:47 crc kubenswrapper[5052]: I1203 12:44:47.071325 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/320f928d-d6b0-48b4-b315-1a6c2ecd3380-combined-ca-bundle\") pod \"nova-cell1-conductor-0\" (UID: \"320f928d-d6b0-48b4-b315-1a6c2ecd3380\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:44:47 crc kubenswrapper[5052]: I1203 12:44:47.078603 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/320f928d-d6b0-48b4-b315-1a6c2ecd3380-combined-ca-bundle\") pod \"nova-cell1-conductor-0\" (UID: \"320f928d-d6b0-48b4-b315-1a6c2ecd3380\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:44:47 crc kubenswrapper[5052]: I1203 12:44:47.080386 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/320f928d-d6b0-48b4-b315-1a6c2ecd3380-config-data\") pod \"nova-cell1-conductor-0\" (UID: \"320f928d-d6b0-48b4-b315-1a6c2ecd3380\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:44:47 crc kubenswrapper[5052]: I1203 12:44:47.090129 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-qc2cz\" (UniqueName: \"kubernetes.io/projected/320f928d-d6b0-48b4-b315-1a6c2ecd3380-kube-api-access-qc2cz\") pod \"nova-cell1-conductor-0\" (UID: \"320f928d-d6b0-48b4-b315-1a6c2ecd3380\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:44:47 crc kubenswrapper[5052]: I1203 12:44:47.271316 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:44:47 crc kubenswrapper[5052]: I1203 12:44:47.691445 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="555b864d-15ca-4c31-894e-1323f009c8b4" path="/var/lib/kubelet/pods/555b864d-15ca-4c31-894e-1323f009c8b4/volumes" Dec 03 12:44:47 crc kubenswrapper[5052]: W1203 12:44:47.730787 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod320f928d_d6b0_48b4_b315_1a6c2ecd3380.slice/crio-06ccbb7e691545be128b7411d4d7649f97da7c37d7109751a533219c6552676c WatchSource:0}: Error finding container 06ccbb7e691545be128b7411d4d7649f97da7c37d7109751a533219c6552676c: Status 404 returned error can't find the container with id 06ccbb7e691545be128b7411d4d7649f97da7c37d7109751a533219c6552676c Dec 03 12:44:47 crc kubenswrapper[5052]: I1203 12:44:47.731214 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-0"] Dec 03 12:44:47 crc kubenswrapper[5052]: I1203 12:44:47.799719 5052 generic.go:334] "Generic (PLEG): container finished" podID="8a321082-482c-4805-b252-88a304bb19ec" containerID="a9b87dccd0e7dcc3aa151cb7ead35d696cd2b5970c1fdb43b6dea2d3e0aee83d" exitCode=0 Dec 03 12:44:47 crc kubenswrapper[5052]: I1203 12:44:47.799800 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-cell-mapping-jq8w4" event={"ID":"8a321082-482c-4805-b252-88a304bb19ec","Type":"ContainerDied","Data":"a9b87dccd0e7dcc3aa151cb7ead35d696cd2b5970c1fdb43b6dea2d3e0aee83d"} Dec 03 12:44:47 crc kubenswrapper[5052]: I1203 12:44:47.803698 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-conductor-0" event={"ID":"320f928d-d6b0-48b4-b315-1a6c2ecd3380","Type":"ContainerStarted","Data":"06ccbb7e691545be128b7411d4d7649f97da7c37d7109751a533219c6552676c"} Dec 03 12:44:47 crc kubenswrapper[5052]: I1203 12:44:47.807824 5052 generic.go:334] "Generic (PLEG): container finished" podID="3b455122-e3fd-44a8-baa3-2b10f856bfce" containerID="d80485db5415a79a1bee865fce5b1dee64e47e17be467a9fdd9649b63865a438" exitCode=0 Dec 03 12:44:47 crc kubenswrapper[5052]: I1203 12:44:47.807862 5052 generic.go:334] "Generic (PLEG): container finished" podID="3b455122-e3fd-44a8-baa3-2b10f856bfce" containerID="9b9f209894b73ff775c7636aa2252ab1c82d6c5abb431a90314a2d850e973805" exitCode=0 Dec 03 12:44:47 crc kubenswrapper[5052]: I1203 12:44:47.807905 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"3b455122-e3fd-44a8-baa3-2b10f856bfce","Type":"ContainerDied","Data":"d80485db5415a79a1bee865fce5b1dee64e47e17be467a9fdd9649b63865a438"} Dec 03 12:44:47 crc kubenswrapper[5052]: I1203 12:44:47.807939 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"3b455122-e3fd-44a8-baa3-2b10f856bfce","Type":"ContainerDied","Data":"9b9f209894b73ff775c7636aa2252ab1c82d6c5abb431a90314a2d850e973805"} Dec 03 12:44:47 crc kubenswrapper[5052]: I1203 12:44:47.812336 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/kube-state-metrics-0" event={"ID":"876d06e6-0187-4e3d-9183-ed2083fe5189","Type":"ContainerStarted","Data":"3d1c3119a6baffe670b9369900b68bd9fa52d242fa306af736795fb6724df163"} Dec 03 12:44:47 crc kubenswrapper[5052]: I1203 12:44:47.812374 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/kube-state-metrics-0" event={"ID":"876d06e6-0187-4e3d-9183-ed2083fe5189","Type":"ContainerStarted","Data":"be84dd2a321e8f5a869f52c256ca8452d3f9dad861bb220ba7e72a739992e6d4"} Dec 03 12:44:47 crc kubenswrapper[5052]: I1203 12:44:47.812393 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:44:47 crc kubenswrapper[5052]: I1203 12:44:47.850179 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/kube-state-metrics-0" podStartSLOduration=2.470489754 podStartE2EDuration="2.850151712s" podCreationTimestamp="2025-12-03 12:44:45 +0000 UTC" firstStartedPulling="2025-12-03 12:44:46.819764414 +0000 UTC m=+5077.408006178" lastFinishedPulling="2025-12-03 12:44:47.199426372 +0000 UTC m=+5077.787668136" observedRunningTime="2025-12-03 12:44:47.83434564 +0000 UTC m=+5078.422587404" watchObservedRunningTime="2025-12-03 12:44:47.850151712 +0000 UTC m=+5078.438393476" Dec 03 12:44:48 crc kubenswrapper[5052]: I1203 12:44:48.823205 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-conductor-0" event={"ID":"320f928d-d6b0-48b4-b315-1a6c2ecd3380","Type":"ContainerStarted","Data":"2ff6796b6955202076b9ca76f2bd9d074d96d820f421983058345a0af1b54565"} Dec 03 12:44:48 crc kubenswrapper[5052]: I1203 12:44:48.823622 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:44:48 crc kubenswrapper[5052]: I1203 12:44:48.858495 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-cell1-conductor-0" podStartSLOduration=2.858468137 podStartE2EDuration="2.858468137s" podCreationTimestamp="2025-12-03 12:44:46 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:44:48.843737476 +0000 UTC m=+5079.431979250" watchObservedRunningTime="2025-12-03 12:44:48.858468137 +0000 UTC m=+5079.446709901" Dec 03 12:44:49 crc kubenswrapper[5052]: I1203 12:44:49.237145 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:44:49 crc kubenswrapper[5052]: I1203 12:44:49.237496 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:44:49 crc kubenswrapper[5052]: I1203 12:44:49.361701 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-cell-mapping-jq8w4" Dec 03 12:44:49 crc kubenswrapper[5052]: I1203 12:44:49.422451 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9br48\" (UniqueName: \"kubernetes.io/projected/8a321082-482c-4805-b252-88a304bb19ec-kube-api-access-9br48\") pod \"8a321082-482c-4805-b252-88a304bb19ec\" (UID: \"8a321082-482c-4805-b252-88a304bb19ec\") " Dec 03 12:44:49 crc kubenswrapper[5052]: I1203 12:44:49.422558 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8a321082-482c-4805-b252-88a304bb19ec-combined-ca-bundle\") pod \"8a321082-482c-4805-b252-88a304bb19ec\" (UID: \"8a321082-482c-4805-b252-88a304bb19ec\") " Dec 03 12:44:49 crc kubenswrapper[5052]: I1203 12:44:49.422679 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/8a321082-482c-4805-b252-88a304bb19ec-scripts\") pod \"8a321082-482c-4805-b252-88a304bb19ec\" (UID: \"8a321082-482c-4805-b252-88a304bb19ec\") " Dec 03 12:44:49 crc kubenswrapper[5052]: I1203 12:44:49.422836 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/8a321082-482c-4805-b252-88a304bb19ec-config-data\") pod \"8a321082-482c-4805-b252-88a304bb19ec\" (UID: \"8a321082-482c-4805-b252-88a304bb19ec\") " Dec 03 12:44:49 crc kubenswrapper[5052]: I1203 12:44:49.429716 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8a321082-482c-4805-b252-88a304bb19ec-scripts" (OuterVolumeSpecName: "scripts") pod "8a321082-482c-4805-b252-88a304bb19ec" (UID: "8a321082-482c-4805-b252-88a304bb19ec"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:44:49 crc kubenswrapper[5052]: I1203 12:44:49.441824 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8a321082-482c-4805-b252-88a304bb19ec-kube-api-access-9br48" (OuterVolumeSpecName: "kube-api-access-9br48") pod "8a321082-482c-4805-b252-88a304bb19ec" (UID: "8a321082-482c-4805-b252-88a304bb19ec"). InnerVolumeSpecName "kube-api-access-9br48". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:44:49 crc kubenswrapper[5052]: I1203 12:44:49.446033 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:44:49 crc kubenswrapper[5052]: I1203 12:44:49.467976 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8a321082-482c-4805-b252-88a304bb19ec-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "8a321082-482c-4805-b252-88a304bb19ec" (UID: "8a321082-482c-4805-b252-88a304bb19ec"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:44:49 crc kubenswrapper[5052]: I1203 12:44:49.486713 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8a321082-482c-4805-b252-88a304bb19ec-config-data" (OuterVolumeSpecName: "config-data") pod "8a321082-482c-4805-b252-88a304bb19ec" (UID: "8a321082-482c-4805-b252-88a304bb19ec"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:44:49 crc kubenswrapper[5052]: I1203 12:44:49.486805 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:44:49 crc kubenswrapper[5052]: I1203 12:44:49.526884 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-9br48\" (UniqueName: \"kubernetes.io/projected/8a321082-482c-4805-b252-88a304bb19ec-kube-api-access-9br48\") on node \"crc\" DevicePath \"\"" Dec 03 12:44:49 crc kubenswrapper[5052]: I1203 12:44:49.527145 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8a321082-482c-4805-b252-88a304bb19ec-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:44:49 crc kubenswrapper[5052]: I1203 12:44:49.527168 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/8a321082-482c-4805-b252-88a304bb19ec-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:44:49 crc kubenswrapper[5052]: I1203 12:44:49.527178 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/8a321082-482c-4805-b252-88a304bb19ec-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:44:49 crc kubenswrapper[5052]: I1203 12:44:49.834531 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-cell-mapping-jq8w4" event={"ID":"8a321082-482c-4805-b252-88a304bb19ec","Type":"ContainerDied","Data":"adbc3b5d2683bcbfc77ec7902bc3342730a422eab4b7e23b156c82ecf9f63883"} Dec 03 12:44:49 crc kubenswrapper[5052]: I1203 12:44:49.836281 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="adbc3b5d2683bcbfc77ec7902bc3342730a422eab4b7e23b156c82ecf9f63883" Dec 03 12:44:49 crc kubenswrapper[5052]: I1203 12:44:49.834948 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-cell-mapping-jq8w4" Dec 03 12:44:49 crc kubenswrapper[5052]: I1203 12:44:49.885386 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:44:50 crc kubenswrapper[5052]: I1203 12:44:50.059367 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:44:50 crc kubenswrapper[5052]: I1203 12:44:50.063463 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-api-0" podUID="a456264f-61b5-4399-b0ec-98034fbf6523" containerName="nova-api-log" containerID="cri-o://20314ac2879b72c5b778670025b0d747d87526b5a82e3632a17c8086995a5d1c" gracePeriod=30 Dec 03 12:44:50 crc kubenswrapper[5052]: I1203 12:44:50.063698 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-api-0" podUID="a456264f-61b5-4399-b0ec-98034fbf6523" containerName="nova-api-api" containerID="cri-o://9f70b2fb3d27878416e4c3be09e384f6b0fde07ec909f079075137a457fbe877" gracePeriod=30 Dec 03 12:44:50 crc kubenswrapper[5052]: I1203 12:44:50.094921 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-api-0" podUID="a456264f-61b5-4399-b0ec-98034fbf6523" containerName="nova-api-api" probeResult="failure" output="Get \"http://10.217.1.219:8774/\": EOF" Dec 03 12:44:50 crc kubenswrapper[5052]: I1203 12:44:50.095031 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-api-0" podUID="a456264f-61b5-4399-b0ec-98034fbf6523" containerName="nova-api-log" probeResult="failure" output="Get \"http://10.217.1.219:8774/\": EOF" Dec 03 12:44:50 crc kubenswrapper[5052]: I1203 12:44:50.100955 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:44:50 crc kubenswrapper[5052]: I1203 12:44:50.101227 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-metadata-0" podUID="ad32b585-41d2-4a7e-a522-069824b84d97" containerName="nova-metadata-log" containerID="cri-o://62d5405e200babb8c891fb3d45e5b63d190e284a98b443c5f799e10ad934a70d" gracePeriod=30 Dec 03 12:44:50 crc kubenswrapper[5052]: I1203 12:44:50.102827 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-metadata-0" podUID="ad32b585-41d2-4a7e-a522-069824b84d97" containerName="nova-metadata-metadata" containerID="cri-o://9c3c946a30940eb77b460b12f157c0b4abe50db0e3fcce354df77b7a76ef9f02" gracePeriod=30 Dec 03 12:44:50 crc kubenswrapper[5052]: I1203 12:44:50.383584 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:44:50 crc kubenswrapper[5052]: I1203 12:44:50.506830 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:44:50 crc kubenswrapper[5052]: I1203 12:44:50.506893 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:44:50 crc kubenswrapper[5052]: I1203 12:44:50.561458 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:44:50 crc kubenswrapper[5052]: I1203 12:44:50.794107 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:44:50 crc kubenswrapper[5052]: I1203 12:44:50.861532 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ad32b585-41d2-4a7e-a522-069824b84d97-combined-ca-bundle\") pod \"ad32b585-41d2-4a7e-a522-069824b84d97\" (UID: \"ad32b585-41d2-4a7e-a522-069824b84d97\") " Dec 03 12:44:50 crc kubenswrapper[5052]: I1203 12:44:50.861612 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-85kbf\" (UniqueName: \"kubernetes.io/projected/ad32b585-41d2-4a7e-a522-069824b84d97-kube-api-access-85kbf\") pod \"ad32b585-41d2-4a7e-a522-069824b84d97\" (UID: \"ad32b585-41d2-4a7e-a522-069824b84d97\") " Dec 03 12:44:50 crc kubenswrapper[5052]: I1203 12:44:50.861675 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/ad32b585-41d2-4a7e-a522-069824b84d97-logs\") pod \"ad32b585-41d2-4a7e-a522-069824b84d97\" (UID: \"ad32b585-41d2-4a7e-a522-069824b84d97\") " Dec 03 12:44:50 crc kubenswrapper[5052]: I1203 12:44:50.861709 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/ad32b585-41d2-4a7e-a522-069824b84d97-nova-metadata-tls-certs\") pod \"ad32b585-41d2-4a7e-a522-069824b84d97\" (UID: \"ad32b585-41d2-4a7e-a522-069824b84d97\") " Dec 03 12:44:50 crc kubenswrapper[5052]: I1203 12:44:50.861821 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/ad32b585-41d2-4a7e-a522-069824b84d97-config-data\") pod \"ad32b585-41d2-4a7e-a522-069824b84d97\" (UID: \"ad32b585-41d2-4a7e-a522-069824b84d97\") " Dec 03 12:44:50 crc kubenswrapper[5052]: I1203 12:44:50.862089 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ad32b585-41d2-4a7e-a522-069824b84d97-logs" (OuterVolumeSpecName: "logs") pod "ad32b585-41d2-4a7e-a522-069824b84d97" (UID: "ad32b585-41d2-4a7e-a522-069824b84d97"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:44:50 crc kubenswrapper[5052]: I1203 12:44:50.862534 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/ad32b585-41d2-4a7e-a522-069824b84d97-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:44:50 crc kubenswrapper[5052]: I1203 12:44:50.863683 5052 generic.go:334] "Generic (PLEG): container finished" podID="a456264f-61b5-4399-b0ec-98034fbf6523" containerID="20314ac2879b72c5b778670025b0d747d87526b5a82e3632a17c8086995a5d1c" exitCode=143 Dec 03 12:44:50 crc kubenswrapper[5052]: I1203 12:44:50.863776 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"a456264f-61b5-4399-b0ec-98034fbf6523","Type":"ContainerDied","Data":"20314ac2879b72c5b778670025b0d747d87526b5a82e3632a17c8086995a5d1c"} Dec 03 12:44:50 crc kubenswrapper[5052]: I1203 12:44:50.868747 5052 generic.go:334] "Generic (PLEG): container finished" podID="ad32b585-41d2-4a7e-a522-069824b84d97" containerID="9c3c946a30940eb77b460b12f157c0b4abe50db0e3fcce354df77b7a76ef9f02" exitCode=0 Dec 03 12:44:50 crc kubenswrapper[5052]: I1203 12:44:50.868786 5052 generic.go:334] "Generic (PLEG): container finished" podID="ad32b585-41d2-4a7e-a522-069824b84d97" containerID="62d5405e200babb8c891fb3d45e5b63d190e284a98b443c5f799e10ad934a70d" exitCode=143 Dec 03 12:44:50 crc kubenswrapper[5052]: I1203 12:44:50.869167 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"ad32b585-41d2-4a7e-a522-069824b84d97","Type":"ContainerDied","Data":"9c3c946a30940eb77b460b12f157c0b4abe50db0e3fcce354df77b7a76ef9f02"} Dec 03 12:44:50 crc kubenswrapper[5052]: I1203 12:44:50.869196 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:44:50 crc kubenswrapper[5052]: I1203 12:44:50.869246 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"ad32b585-41d2-4a7e-a522-069824b84d97","Type":"ContainerDied","Data":"62d5405e200babb8c891fb3d45e5b63d190e284a98b443c5f799e10ad934a70d"} Dec 03 12:44:50 crc kubenswrapper[5052]: I1203 12:44:50.869265 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"ad32b585-41d2-4a7e-a522-069824b84d97","Type":"ContainerDied","Data":"17c2f36f135b55cf108d15a9c4ba5322a0b7e88070295947a9b878f3abda8f03"} Dec 03 12:44:50 crc kubenswrapper[5052]: I1203 12:44:50.869286 5052 scope.go:117] "RemoveContainer" containerID="9c3c946a30940eb77b460b12f157c0b4abe50db0e3fcce354df77b7a76ef9f02" Dec 03 12:44:50 crc kubenswrapper[5052]: I1203 12:44:50.908647 5052 scope.go:117] "RemoveContainer" containerID="62d5405e200babb8c891fb3d45e5b63d190e284a98b443c5f799e10ad934a70d" Dec 03 12:44:50 crc kubenswrapper[5052]: I1203 12:44:50.941300 5052 scope.go:117] "RemoveContainer" containerID="9c3c946a30940eb77b460b12f157c0b4abe50db0e3fcce354df77b7a76ef9f02" Dec 03 12:44:50 crc kubenswrapper[5052]: E1203 12:44:50.942095 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9c3c946a30940eb77b460b12f157c0b4abe50db0e3fcce354df77b7a76ef9f02\": container with ID starting with 9c3c946a30940eb77b460b12f157c0b4abe50db0e3fcce354df77b7a76ef9f02 not found: ID does not exist" containerID="9c3c946a30940eb77b460b12f157c0b4abe50db0e3fcce354df77b7a76ef9f02" Dec 03 12:44:50 crc kubenswrapper[5052]: I1203 12:44:50.942137 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9c3c946a30940eb77b460b12f157c0b4abe50db0e3fcce354df77b7a76ef9f02"} err="failed to get container status \"9c3c946a30940eb77b460b12f157c0b4abe50db0e3fcce354df77b7a76ef9f02\": rpc error: code = NotFound desc = could not find container \"9c3c946a30940eb77b460b12f157c0b4abe50db0e3fcce354df77b7a76ef9f02\": container with ID starting with 9c3c946a30940eb77b460b12f157c0b4abe50db0e3fcce354df77b7a76ef9f02 not found: ID does not exist" Dec 03 12:44:50 crc kubenswrapper[5052]: I1203 12:44:50.942163 5052 scope.go:117] "RemoveContainer" containerID="62d5405e200babb8c891fb3d45e5b63d190e284a98b443c5f799e10ad934a70d" Dec 03 12:44:50 crc kubenswrapper[5052]: E1203 12:44:50.943531 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"62d5405e200babb8c891fb3d45e5b63d190e284a98b443c5f799e10ad934a70d\": container with ID starting with 62d5405e200babb8c891fb3d45e5b63d190e284a98b443c5f799e10ad934a70d not found: ID does not exist" containerID="62d5405e200babb8c891fb3d45e5b63d190e284a98b443c5f799e10ad934a70d" Dec 03 12:44:50 crc kubenswrapper[5052]: I1203 12:44:50.943597 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"62d5405e200babb8c891fb3d45e5b63d190e284a98b443c5f799e10ad934a70d"} err="failed to get container status \"62d5405e200babb8c891fb3d45e5b63d190e284a98b443c5f799e10ad934a70d\": rpc error: code = NotFound desc = could not find container \"62d5405e200babb8c891fb3d45e5b63d190e284a98b443c5f799e10ad934a70d\": container with ID starting with 62d5405e200babb8c891fb3d45e5b63d190e284a98b443c5f799e10ad934a70d not found: ID does not exist" Dec 03 12:44:50 crc kubenswrapper[5052]: I1203 12:44:50.943626 5052 scope.go:117] "RemoveContainer" containerID="9c3c946a30940eb77b460b12f157c0b4abe50db0e3fcce354df77b7a76ef9f02" Dec 03 12:44:50 crc kubenswrapper[5052]: I1203 12:44:50.943960 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9c3c946a30940eb77b460b12f157c0b4abe50db0e3fcce354df77b7a76ef9f02"} err="failed to get container status \"9c3c946a30940eb77b460b12f157c0b4abe50db0e3fcce354df77b7a76ef9f02\": rpc error: code = NotFound desc = could not find container \"9c3c946a30940eb77b460b12f157c0b4abe50db0e3fcce354df77b7a76ef9f02\": container with ID starting with 9c3c946a30940eb77b460b12f157c0b4abe50db0e3fcce354df77b7a76ef9f02 not found: ID does not exist" Dec 03 12:44:50 crc kubenswrapper[5052]: I1203 12:44:50.943994 5052 scope.go:117] "RemoveContainer" containerID="62d5405e200babb8c891fb3d45e5b63d190e284a98b443c5f799e10ad934a70d" Dec 03 12:44:50 crc kubenswrapper[5052]: I1203 12:44:50.944326 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"62d5405e200babb8c891fb3d45e5b63d190e284a98b443c5f799e10ad934a70d"} err="failed to get container status \"62d5405e200babb8c891fb3d45e5b63d190e284a98b443c5f799e10ad934a70d\": rpc error: code = NotFound desc = could not find container \"62d5405e200babb8c891fb3d45e5b63d190e284a98b443c5f799e10ad934a70d\": container with ID starting with 62d5405e200babb8c891fb3d45e5b63d190e284a98b443c5f799e10ad934a70d not found: ID does not exist" Dec 03 12:44:51 crc kubenswrapper[5052]: I1203 12:44:51.424407 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ad32b585-41d2-4a7e-a522-069824b84d97-kube-api-access-85kbf" (OuterVolumeSpecName: "kube-api-access-85kbf") pod "ad32b585-41d2-4a7e-a522-069824b84d97" (UID: "ad32b585-41d2-4a7e-a522-069824b84d97"). InnerVolumeSpecName "kube-api-access-85kbf". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:44:51 crc kubenswrapper[5052]: I1203 12:44:51.473862 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-85kbf\" (UniqueName: \"kubernetes.io/projected/ad32b585-41d2-4a7e-a522-069824b84d97-kube-api-access-85kbf\") on node \"crc\" DevicePath \"\"" Dec 03 12:44:51 crc kubenswrapper[5052]: I1203 12:44:51.728733 5052 prober.go:107] "Probe failed" probeType="Liveness" pod="openstack-kuttl-tests/openstack-cell1-galera-0" podUID="2c53ea4e-56ba-4ba1-8cb6-8c24725b5856" containerName="galera" probeResult="failure" output="command timed out" Dec 03 12:44:51 crc kubenswrapper[5052]: I1203 12:44:51.746892 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ad32b585-41d2-4a7e-a522-069824b84d97-config-data" (OuterVolumeSpecName: "config-data") pod "ad32b585-41d2-4a7e-a522-069824b84d97" (UID: "ad32b585-41d2-4a7e-a522-069824b84d97"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:44:51 crc kubenswrapper[5052]: I1203 12:44:51.761855 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ad32b585-41d2-4a7e-a522-069824b84d97-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "ad32b585-41d2-4a7e-a522-069824b84d97" (UID: "ad32b585-41d2-4a7e-a522-069824b84d97"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:44:51 crc kubenswrapper[5052]: I1203 12:44:51.781301 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/ad32b585-41d2-4a7e-a522-069824b84d97-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:44:51 crc kubenswrapper[5052]: I1203 12:44:51.781387 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ad32b585-41d2-4a7e-a522-069824b84d97-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:44:51 crc kubenswrapper[5052]: I1203 12:44:51.805817 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ad32b585-41d2-4a7e-a522-069824b84d97-nova-metadata-tls-certs" (OuterVolumeSpecName: "nova-metadata-tls-certs") pod "ad32b585-41d2-4a7e-a522-069824b84d97" (UID: "ad32b585-41d2-4a7e-a522-069824b84d97"). InnerVolumeSpecName "nova-metadata-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:44:51 crc kubenswrapper[5052]: I1203 12:44:51.882912 5052 reconciler_common.go:293] "Volume detached for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/ad32b585-41d2-4a7e-a522-069824b84d97-nova-metadata-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:44:51 crc kubenswrapper[5052]: I1203 12:44:51.884372 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-scheduler-0" podUID="62c6a5e0-6e22-456b-bf69-3ca1d0357b65" containerName="nova-scheduler-scheduler" containerID="cri-o://1dc1d18beb6efe5606641ce856ba08888e10021eaa52fe8e30fa9b5ad38c7b63" gracePeriod=30 Dec 03 12:44:52 crc kubenswrapper[5052]: I1203 12:44:52.165148 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:44:52 crc kubenswrapper[5052]: I1203 12:44:52.181929 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:44:52 crc kubenswrapper[5052]: I1203 12:44:52.193153 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:44:52 crc kubenswrapper[5052]: E1203 12:44:52.193894 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="ad32b585-41d2-4a7e-a522-069824b84d97" containerName="nova-metadata-log" Dec 03 12:44:52 crc kubenswrapper[5052]: I1203 12:44:52.193920 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="ad32b585-41d2-4a7e-a522-069824b84d97" containerName="nova-metadata-log" Dec 03 12:44:52 crc kubenswrapper[5052]: E1203 12:44:52.193976 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="ad32b585-41d2-4a7e-a522-069824b84d97" containerName="nova-metadata-metadata" Dec 03 12:44:52 crc kubenswrapper[5052]: I1203 12:44:52.193988 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="ad32b585-41d2-4a7e-a522-069824b84d97" containerName="nova-metadata-metadata" Dec 03 12:44:52 crc kubenswrapper[5052]: E1203 12:44:52.194020 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="8a321082-482c-4805-b252-88a304bb19ec" containerName="nova-manage" Dec 03 12:44:52 crc kubenswrapper[5052]: I1203 12:44:52.194075 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="8a321082-482c-4805-b252-88a304bb19ec" containerName="nova-manage" Dec 03 12:44:52 crc kubenswrapper[5052]: I1203 12:44:52.194414 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="ad32b585-41d2-4a7e-a522-069824b84d97" containerName="nova-metadata-metadata" Dec 03 12:44:52 crc kubenswrapper[5052]: I1203 12:44:52.194451 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="8a321082-482c-4805-b252-88a304bb19ec" containerName="nova-manage" Dec 03 12:44:52 crc kubenswrapper[5052]: I1203 12:44:52.194486 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="ad32b585-41d2-4a7e-a522-069824b84d97" containerName="nova-metadata-log" Dec 03 12:44:52 crc kubenswrapper[5052]: I1203 12:44:52.196460 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:44:52 crc kubenswrapper[5052]: I1203 12:44:52.199761 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-nova-metadata-internal-svc" Dec 03 12:44:52 crc kubenswrapper[5052]: I1203 12:44:52.200479 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-metadata-config-data" Dec 03 12:44:52 crc kubenswrapper[5052]: I1203 12:44:52.222618 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:44:52 crc kubenswrapper[5052]: E1203 12:44:52.239618 5052 cadvisor_stats_provider.go:516] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3b455122_e3fd_44a8_baa3_2b10f856bfce.slice/crio-1f02fbccdb7521208886acc6eb8caee6a28ba2f64b08fbbd8a5f9529383c29a3.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podad32b585_41d2_4a7e_a522_069824b84d97.slice\": RecentStats: unable to find data in memory cache]" Dec 03 12:44:52 crc kubenswrapper[5052]: I1203 12:44:52.292876 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/047b4ef7-6147-49da-b4c8-382319e7f83a-config-data\") pod \"nova-metadata-0\" (UID: \"047b4ef7-6147-49da-b4c8-382319e7f83a\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:44:52 crc kubenswrapper[5052]: I1203 12:44:52.293297 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/047b4ef7-6147-49da-b4c8-382319e7f83a-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"047b4ef7-6147-49da-b4c8-382319e7f83a\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:44:52 crc kubenswrapper[5052]: I1203 12:44:52.293356 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sx5px\" (UniqueName: \"kubernetes.io/projected/047b4ef7-6147-49da-b4c8-382319e7f83a-kube-api-access-sx5px\") pod \"nova-metadata-0\" (UID: \"047b4ef7-6147-49da-b4c8-382319e7f83a\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:44:52 crc kubenswrapper[5052]: I1203 12:44:52.293399 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/047b4ef7-6147-49da-b4c8-382319e7f83a-nova-metadata-tls-certs\") pod \"nova-metadata-0\" (UID: \"047b4ef7-6147-49da-b4c8-382319e7f83a\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:44:52 crc kubenswrapper[5052]: I1203 12:44:52.293422 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/047b4ef7-6147-49da-b4c8-382319e7f83a-logs\") pod \"nova-metadata-0\" (UID: \"047b4ef7-6147-49da-b4c8-382319e7f83a\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:44:52 crc kubenswrapper[5052]: I1203 12:44:52.313487 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:44:52 crc kubenswrapper[5052]: I1203 12:44:52.395148 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/047b4ef7-6147-49da-b4c8-382319e7f83a-logs\") pod \"nova-metadata-0\" (UID: \"047b4ef7-6147-49da-b4c8-382319e7f83a\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:44:52 crc kubenswrapper[5052]: I1203 12:44:52.395323 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/047b4ef7-6147-49da-b4c8-382319e7f83a-config-data\") pod \"nova-metadata-0\" (UID: \"047b4ef7-6147-49da-b4c8-382319e7f83a\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:44:52 crc kubenswrapper[5052]: I1203 12:44:52.395432 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/047b4ef7-6147-49da-b4c8-382319e7f83a-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"047b4ef7-6147-49da-b4c8-382319e7f83a\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:44:52 crc kubenswrapper[5052]: I1203 12:44:52.395530 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-sx5px\" (UniqueName: \"kubernetes.io/projected/047b4ef7-6147-49da-b4c8-382319e7f83a-kube-api-access-sx5px\") pod \"nova-metadata-0\" (UID: \"047b4ef7-6147-49da-b4c8-382319e7f83a\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:44:52 crc kubenswrapper[5052]: I1203 12:44:52.395583 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/047b4ef7-6147-49da-b4c8-382319e7f83a-nova-metadata-tls-certs\") pod \"nova-metadata-0\" (UID: \"047b4ef7-6147-49da-b4c8-382319e7f83a\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:44:52 crc kubenswrapper[5052]: I1203 12:44:52.398001 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/047b4ef7-6147-49da-b4c8-382319e7f83a-logs\") pod \"nova-metadata-0\" (UID: \"047b4ef7-6147-49da-b4c8-382319e7f83a\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:44:52 crc kubenswrapper[5052]: I1203 12:44:52.401549 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/047b4ef7-6147-49da-b4c8-382319e7f83a-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"047b4ef7-6147-49da-b4c8-382319e7f83a\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:44:52 crc kubenswrapper[5052]: I1203 12:44:52.401666 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/047b4ef7-6147-49da-b4c8-382319e7f83a-nova-metadata-tls-certs\") pod \"nova-metadata-0\" (UID: \"047b4ef7-6147-49da-b4c8-382319e7f83a\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:44:52 crc kubenswrapper[5052]: I1203 12:44:52.404226 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/047b4ef7-6147-49da-b4c8-382319e7f83a-config-data\") pod \"nova-metadata-0\" (UID: \"047b4ef7-6147-49da-b4c8-382319e7f83a\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:44:52 crc kubenswrapper[5052]: I1203 12:44:52.417873 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-sx5px\" (UniqueName: \"kubernetes.io/projected/047b4ef7-6147-49da-b4c8-382319e7f83a-kube-api-access-sx5px\") pod \"nova-metadata-0\" (UID: \"047b4ef7-6147-49da-b4c8-382319e7f83a\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:44:52 crc kubenswrapper[5052]: I1203 12:44:52.550124 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:44:52 crc kubenswrapper[5052]: I1203 12:44:52.564905 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:44:52 crc kubenswrapper[5052]: I1203 12:44:52.600970 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/3b455122-e3fd-44a8-baa3-2b10f856bfce-config-data\") pod \"3b455122-e3fd-44a8-baa3-2b10f856bfce\" (UID: \"3b455122-e3fd-44a8-baa3-2b10f856bfce\") " Dec 03 12:44:52 crc kubenswrapper[5052]: I1203 12:44:52.601338 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/3b455122-e3fd-44a8-baa3-2b10f856bfce-log-httpd\") pod \"3b455122-e3fd-44a8-baa3-2b10f856bfce\" (UID: \"3b455122-e3fd-44a8-baa3-2b10f856bfce\") " Dec 03 12:44:52 crc kubenswrapper[5052]: I1203 12:44:52.601385 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3b455122-e3fd-44a8-baa3-2b10f856bfce-combined-ca-bundle\") pod \"3b455122-e3fd-44a8-baa3-2b10f856bfce\" (UID: \"3b455122-e3fd-44a8-baa3-2b10f856bfce\") " Dec 03 12:44:52 crc kubenswrapper[5052]: I1203 12:44:52.602142 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b455122-e3fd-44a8-baa3-2b10f856bfce-log-httpd" (OuterVolumeSpecName: "log-httpd") pod "3b455122-e3fd-44a8-baa3-2b10f856bfce" (UID: "3b455122-e3fd-44a8-baa3-2b10f856bfce"). InnerVolumeSpecName "log-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:44:52 crc kubenswrapper[5052]: I1203 12:44:52.613835 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6lnv8\" (UniqueName: \"kubernetes.io/projected/3b455122-e3fd-44a8-baa3-2b10f856bfce-kube-api-access-6lnv8\") pod \"3b455122-e3fd-44a8-baa3-2b10f856bfce\" (UID: \"3b455122-e3fd-44a8-baa3-2b10f856bfce\") " Dec 03 12:44:52 crc kubenswrapper[5052]: I1203 12:44:52.614001 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/3b455122-e3fd-44a8-baa3-2b10f856bfce-run-httpd\") pod \"3b455122-e3fd-44a8-baa3-2b10f856bfce\" (UID: \"3b455122-e3fd-44a8-baa3-2b10f856bfce\") " Dec 03 12:44:52 crc kubenswrapper[5052]: I1203 12:44:52.614036 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/3b455122-e3fd-44a8-baa3-2b10f856bfce-sg-core-conf-yaml\") pod \"3b455122-e3fd-44a8-baa3-2b10f856bfce\" (UID: \"3b455122-e3fd-44a8-baa3-2b10f856bfce\") " Dec 03 12:44:52 crc kubenswrapper[5052]: I1203 12:44:52.614075 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/3b455122-e3fd-44a8-baa3-2b10f856bfce-scripts\") pod \"3b455122-e3fd-44a8-baa3-2b10f856bfce\" (UID: \"3b455122-e3fd-44a8-baa3-2b10f856bfce\") " Dec 03 12:44:52 crc kubenswrapper[5052]: I1203 12:44:52.614974 5052 reconciler_common.go:293] "Volume detached for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/3b455122-e3fd-44a8-baa3-2b10f856bfce-log-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:44:52 crc kubenswrapper[5052]: I1203 12:44:52.618486 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b455122-e3fd-44a8-baa3-2b10f856bfce-run-httpd" (OuterVolumeSpecName: "run-httpd") pod "3b455122-e3fd-44a8-baa3-2b10f856bfce" (UID: "3b455122-e3fd-44a8-baa3-2b10f856bfce"). InnerVolumeSpecName "run-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:44:52 crc kubenswrapper[5052]: I1203 12:44:52.621451 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3b455122-e3fd-44a8-baa3-2b10f856bfce-scripts" (OuterVolumeSpecName: "scripts") pod "3b455122-e3fd-44a8-baa3-2b10f856bfce" (UID: "3b455122-e3fd-44a8-baa3-2b10f856bfce"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:44:52 crc kubenswrapper[5052]: I1203 12:44:52.623576 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3b455122-e3fd-44a8-baa3-2b10f856bfce-kube-api-access-6lnv8" (OuterVolumeSpecName: "kube-api-access-6lnv8") pod "3b455122-e3fd-44a8-baa3-2b10f856bfce" (UID: "3b455122-e3fd-44a8-baa3-2b10f856bfce"). InnerVolumeSpecName "kube-api-access-6lnv8". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:44:52 crc kubenswrapper[5052]: I1203 12:44:52.653794 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3b455122-e3fd-44a8-baa3-2b10f856bfce-sg-core-conf-yaml" (OuterVolumeSpecName: "sg-core-conf-yaml") pod "3b455122-e3fd-44a8-baa3-2b10f856bfce" (UID: "3b455122-e3fd-44a8-baa3-2b10f856bfce"). InnerVolumeSpecName "sg-core-conf-yaml". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:44:52 crc kubenswrapper[5052]: I1203 12:44:52.719863 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-6lnv8\" (UniqueName: \"kubernetes.io/projected/3b455122-e3fd-44a8-baa3-2b10f856bfce-kube-api-access-6lnv8\") on node \"crc\" DevicePath \"\"" Dec 03 12:44:52 crc kubenswrapper[5052]: I1203 12:44:52.719893 5052 reconciler_common.go:293] "Volume detached for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/3b455122-e3fd-44a8-baa3-2b10f856bfce-run-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:44:52 crc kubenswrapper[5052]: I1203 12:44:52.719904 5052 reconciler_common.go:293] "Volume detached for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/3b455122-e3fd-44a8-baa3-2b10f856bfce-sg-core-conf-yaml\") on node \"crc\" DevicePath \"\"" Dec 03 12:44:52 crc kubenswrapper[5052]: I1203 12:44:52.719914 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/3b455122-e3fd-44a8-baa3-2b10f856bfce-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:44:52 crc kubenswrapper[5052]: I1203 12:44:52.732227 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3b455122-e3fd-44a8-baa3-2b10f856bfce-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "3b455122-e3fd-44a8-baa3-2b10f856bfce" (UID: "3b455122-e3fd-44a8-baa3-2b10f856bfce"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:44:52 crc kubenswrapper[5052]: I1203 12:44:52.795779 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3b455122-e3fd-44a8-baa3-2b10f856bfce-config-data" (OuterVolumeSpecName: "config-data") pod "3b455122-e3fd-44a8-baa3-2b10f856bfce" (UID: "3b455122-e3fd-44a8-baa3-2b10f856bfce"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:44:52 crc kubenswrapper[5052]: I1203 12:44:52.822756 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/3b455122-e3fd-44a8-baa3-2b10f856bfce-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:44:52 crc kubenswrapper[5052]: I1203 12:44:52.822802 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3b455122-e3fd-44a8-baa3-2b10f856bfce-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:44:52 crc kubenswrapper[5052]: I1203 12:44:52.900312 5052 generic.go:334] "Generic (PLEG): container finished" podID="3b455122-e3fd-44a8-baa3-2b10f856bfce" containerID="1f02fbccdb7521208886acc6eb8caee6a28ba2f64b08fbbd8a5f9529383c29a3" exitCode=0 Dec 03 12:44:52 crc kubenswrapper[5052]: I1203 12:44:52.900359 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"3b455122-e3fd-44a8-baa3-2b10f856bfce","Type":"ContainerDied","Data":"1f02fbccdb7521208886acc6eb8caee6a28ba2f64b08fbbd8a5f9529383c29a3"} Dec 03 12:44:52 crc kubenswrapper[5052]: I1203 12:44:52.900390 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"3b455122-e3fd-44a8-baa3-2b10f856bfce","Type":"ContainerDied","Data":"8ac3459ca4d720b164d5bb90e6a490dd90e7b1a621c924db8632864b460a44e0"} Dec 03 12:44:52 crc kubenswrapper[5052]: I1203 12:44:52.900409 5052 scope.go:117] "RemoveContainer" containerID="d80485db5415a79a1bee865fce5b1dee64e47e17be467a9fdd9649b63865a438" Dec 03 12:44:52 crc kubenswrapper[5052]: I1203 12:44:52.900490 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:44:52 crc kubenswrapper[5052]: I1203 12:44:52.930463 5052 scope.go:117] "RemoveContainer" containerID="fe2a0268a0be69498dad63a9b2768a0651397fdd6a75908b40f05a1eb5621f57" Dec 03 12:44:52 crc kubenswrapper[5052]: I1203 12:44:52.957797 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:44:52 crc kubenswrapper[5052]: I1203 12:44:52.967098 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:44:52 crc kubenswrapper[5052]: I1203 12:44:52.970901 5052 scope.go:117] "RemoveContainer" containerID="1f02fbccdb7521208886acc6eb8caee6a28ba2f64b08fbbd8a5f9529383c29a3" Dec 03 12:44:52 crc kubenswrapper[5052]: I1203 12:44:52.978422 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:44:52 crc kubenswrapper[5052]: E1203 12:44:52.979817 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="3b455122-e3fd-44a8-baa3-2b10f856bfce" containerName="proxy-httpd" Dec 03 12:44:52 crc kubenswrapper[5052]: I1203 12:44:52.979850 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b455122-e3fd-44a8-baa3-2b10f856bfce" containerName="proxy-httpd" Dec 03 12:44:52 crc kubenswrapper[5052]: E1203 12:44:52.979901 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="3b455122-e3fd-44a8-baa3-2b10f856bfce" containerName="sg-core" Dec 03 12:44:52 crc kubenswrapper[5052]: I1203 12:44:52.979911 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b455122-e3fd-44a8-baa3-2b10f856bfce" containerName="sg-core" Dec 03 12:44:52 crc kubenswrapper[5052]: E1203 12:44:52.979965 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="3b455122-e3fd-44a8-baa3-2b10f856bfce" containerName="ceilometer-notification-agent" Dec 03 12:44:52 crc kubenswrapper[5052]: I1203 12:44:52.979975 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b455122-e3fd-44a8-baa3-2b10f856bfce" containerName="ceilometer-notification-agent" Dec 03 12:44:52 crc kubenswrapper[5052]: E1203 12:44:52.980009 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="3b455122-e3fd-44a8-baa3-2b10f856bfce" containerName="ceilometer-central-agent" Dec 03 12:44:52 crc kubenswrapper[5052]: I1203 12:44:52.980018 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b455122-e3fd-44a8-baa3-2b10f856bfce" containerName="ceilometer-central-agent" Dec 03 12:44:52 crc kubenswrapper[5052]: I1203 12:44:52.980321 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="3b455122-e3fd-44a8-baa3-2b10f856bfce" containerName="sg-core" Dec 03 12:44:52 crc kubenswrapper[5052]: I1203 12:44:52.980383 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="3b455122-e3fd-44a8-baa3-2b10f856bfce" containerName="proxy-httpd" Dec 03 12:44:52 crc kubenswrapper[5052]: I1203 12:44:52.980408 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="3b455122-e3fd-44a8-baa3-2b10f856bfce" containerName="ceilometer-central-agent" Dec 03 12:44:52 crc kubenswrapper[5052]: I1203 12:44:52.980447 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="3b455122-e3fd-44a8-baa3-2b10f856bfce" containerName="ceilometer-notification-agent" Dec 03 12:44:52 crc kubenswrapper[5052]: I1203 12:44:52.983318 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:44:52 crc kubenswrapper[5052]: I1203 12:44:52.985953 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-config-data" Dec 03 12:44:52 crc kubenswrapper[5052]: I1203 12:44:52.986288 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-ceilometer-internal-svc" Dec 03 12:44:52 crc kubenswrapper[5052]: I1203 12:44:52.988373 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-scripts" Dec 03 12:44:53 crc kubenswrapper[5052]: I1203 12:44:53.016264 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:44:53 crc kubenswrapper[5052]: I1203 12:44:53.016741 5052 scope.go:117] "RemoveContainer" containerID="9b9f209894b73ff775c7636aa2252ab1c82d6c5abb431a90314a2d850e973805" Dec 03 12:44:53 crc kubenswrapper[5052]: I1203 12:44:53.026507 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e08a0c0d-36f0-49f1-9df8-f0e616d1d99d-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"e08a0c0d-36f0-49f1-9df8-f0e616d1d99d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:44:53 crc kubenswrapper[5052]: I1203 12:44:53.026570 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/e08a0c0d-36f0-49f1-9df8-f0e616d1d99d-run-httpd\") pod \"ceilometer-0\" (UID: \"e08a0c0d-36f0-49f1-9df8-f0e616d1d99d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:44:53 crc kubenswrapper[5052]: I1203 12:44:53.026645 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r65kf\" (UniqueName: \"kubernetes.io/projected/e08a0c0d-36f0-49f1-9df8-f0e616d1d99d-kube-api-access-r65kf\") pod \"ceilometer-0\" (UID: \"e08a0c0d-36f0-49f1-9df8-f0e616d1d99d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:44:53 crc kubenswrapper[5052]: I1203 12:44:53.026700 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/e08a0c0d-36f0-49f1-9df8-f0e616d1d99d-scripts\") pod \"ceilometer-0\" (UID: \"e08a0c0d-36f0-49f1-9df8-f0e616d1d99d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:44:53 crc kubenswrapper[5052]: I1203 12:44:53.026751 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/e08a0c0d-36f0-49f1-9df8-f0e616d1d99d-log-httpd\") pod \"ceilometer-0\" (UID: \"e08a0c0d-36f0-49f1-9df8-f0e616d1d99d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:44:53 crc kubenswrapper[5052]: I1203 12:44:53.026823 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/e08a0c0d-36f0-49f1-9df8-f0e616d1d99d-ceilometer-tls-certs\") pod \"ceilometer-0\" (UID: \"e08a0c0d-36f0-49f1-9df8-f0e616d1d99d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:44:53 crc kubenswrapper[5052]: I1203 12:44:53.026873 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/e08a0c0d-36f0-49f1-9df8-f0e616d1d99d-config-data\") pod \"ceilometer-0\" (UID: \"e08a0c0d-36f0-49f1-9df8-f0e616d1d99d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:44:53 crc kubenswrapper[5052]: I1203 12:44:53.026992 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/e08a0c0d-36f0-49f1-9df8-f0e616d1d99d-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"e08a0c0d-36f0-49f1-9df8-f0e616d1d99d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:44:53 crc kubenswrapper[5052]: I1203 12:44:53.048343 5052 scope.go:117] "RemoveContainer" containerID="d80485db5415a79a1bee865fce5b1dee64e47e17be467a9fdd9649b63865a438" Dec 03 12:44:53 crc kubenswrapper[5052]: E1203 12:44:53.049210 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d80485db5415a79a1bee865fce5b1dee64e47e17be467a9fdd9649b63865a438\": container with ID starting with d80485db5415a79a1bee865fce5b1dee64e47e17be467a9fdd9649b63865a438 not found: ID does not exist" containerID="d80485db5415a79a1bee865fce5b1dee64e47e17be467a9fdd9649b63865a438" Dec 03 12:44:53 crc kubenswrapper[5052]: I1203 12:44:53.049327 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d80485db5415a79a1bee865fce5b1dee64e47e17be467a9fdd9649b63865a438"} err="failed to get container status \"d80485db5415a79a1bee865fce5b1dee64e47e17be467a9fdd9649b63865a438\": rpc error: code = NotFound desc = could not find container \"d80485db5415a79a1bee865fce5b1dee64e47e17be467a9fdd9649b63865a438\": container with ID starting with d80485db5415a79a1bee865fce5b1dee64e47e17be467a9fdd9649b63865a438 not found: ID does not exist" Dec 03 12:44:53 crc kubenswrapper[5052]: I1203 12:44:53.049368 5052 scope.go:117] "RemoveContainer" containerID="fe2a0268a0be69498dad63a9b2768a0651397fdd6a75908b40f05a1eb5621f57" Dec 03 12:44:53 crc kubenswrapper[5052]: E1203 12:44:53.049843 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fe2a0268a0be69498dad63a9b2768a0651397fdd6a75908b40f05a1eb5621f57\": container with ID starting with fe2a0268a0be69498dad63a9b2768a0651397fdd6a75908b40f05a1eb5621f57 not found: ID does not exist" containerID="fe2a0268a0be69498dad63a9b2768a0651397fdd6a75908b40f05a1eb5621f57" Dec 03 12:44:53 crc kubenswrapper[5052]: I1203 12:44:53.049878 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fe2a0268a0be69498dad63a9b2768a0651397fdd6a75908b40f05a1eb5621f57"} err="failed to get container status \"fe2a0268a0be69498dad63a9b2768a0651397fdd6a75908b40f05a1eb5621f57\": rpc error: code = NotFound desc = could not find container \"fe2a0268a0be69498dad63a9b2768a0651397fdd6a75908b40f05a1eb5621f57\": container with ID starting with fe2a0268a0be69498dad63a9b2768a0651397fdd6a75908b40f05a1eb5621f57 not found: ID does not exist" Dec 03 12:44:53 crc kubenswrapper[5052]: I1203 12:44:53.049898 5052 scope.go:117] "RemoveContainer" containerID="1f02fbccdb7521208886acc6eb8caee6a28ba2f64b08fbbd8a5f9529383c29a3" Dec 03 12:44:53 crc kubenswrapper[5052]: E1203 12:44:53.050118 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1f02fbccdb7521208886acc6eb8caee6a28ba2f64b08fbbd8a5f9529383c29a3\": container with ID starting with 1f02fbccdb7521208886acc6eb8caee6a28ba2f64b08fbbd8a5f9529383c29a3 not found: ID does not exist" containerID="1f02fbccdb7521208886acc6eb8caee6a28ba2f64b08fbbd8a5f9529383c29a3" Dec 03 12:44:53 crc kubenswrapper[5052]: I1203 12:44:53.050150 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1f02fbccdb7521208886acc6eb8caee6a28ba2f64b08fbbd8a5f9529383c29a3"} err="failed to get container status \"1f02fbccdb7521208886acc6eb8caee6a28ba2f64b08fbbd8a5f9529383c29a3\": rpc error: code = NotFound desc = could not find container \"1f02fbccdb7521208886acc6eb8caee6a28ba2f64b08fbbd8a5f9529383c29a3\": container with ID starting with 1f02fbccdb7521208886acc6eb8caee6a28ba2f64b08fbbd8a5f9529383c29a3 not found: ID does not exist" Dec 03 12:44:53 crc kubenswrapper[5052]: I1203 12:44:53.050167 5052 scope.go:117] "RemoveContainer" containerID="9b9f209894b73ff775c7636aa2252ab1c82d6c5abb431a90314a2d850e973805" Dec 03 12:44:53 crc kubenswrapper[5052]: E1203 12:44:53.050368 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9b9f209894b73ff775c7636aa2252ab1c82d6c5abb431a90314a2d850e973805\": container with ID starting with 9b9f209894b73ff775c7636aa2252ab1c82d6c5abb431a90314a2d850e973805 not found: ID does not exist" containerID="9b9f209894b73ff775c7636aa2252ab1c82d6c5abb431a90314a2d850e973805" Dec 03 12:44:53 crc kubenswrapper[5052]: I1203 12:44:53.050395 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9b9f209894b73ff775c7636aa2252ab1c82d6c5abb431a90314a2d850e973805"} err="failed to get container status \"9b9f209894b73ff775c7636aa2252ab1c82d6c5abb431a90314a2d850e973805\": rpc error: code = NotFound desc = could not find container \"9b9f209894b73ff775c7636aa2252ab1c82d6c5abb431a90314a2d850e973805\": container with ID starting with 9b9f209894b73ff775c7636aa2252ab1c82d6c5abb431a90314a2d850e973805 not found: ID does not exist" Dec 03 12:44:53 crc kubenswrapper[5052]: I1203 12:44:53.082710 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:44:53 crc kubenswrapper[5052]: W1203 12:44:53.087665 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod047b4ef7_6147_49da_b4c8_382319e7f83a.slice/crio-87d968842d0f87896c822b6ced40469c04622652bd585174203619aac3818328 WatchSource:0}: Error finding container 87d968842d0f87896c822b6ced40469c04622652bd585174203619aac3818328: Status 404 returned error can't find the container with id 87d968842d0f87896c822b6ced40469c04622652bd585174203619aac3818328 Dec 03 12:44:53 crc kubenswrapper[5052]: I1203 12:44:53.129272 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e08a0c0d-36f0-49f1-9df8-f0e616d1d99d-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"e08a0c0d-36f0-49f1-9df8-f0e616d1d99d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:44:53 crc kubenswrapper[5052]: I1203 12:44:53.129332 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/e08a0c0d-36f0-49f1-9df8-f0e616d1d99d-run-httpd\") pod \"ceilometer-0\" (UID: \"e08a0c0d-36f0-49f1-9df8-f0e616d1d99d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:44:53 crc kubenswrapper[5052]: I1203 12:44:53.129380 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-r65kf\" (UniqueName: \"kubernetes.io/projected/e08a0c0d-36f0-49f1-9df8-f0e616d1d99d-kube-api-access-r65kf\") pod \"ceilometer-0\" (UID: \"e08a0c0d-36f0-49f1-9df8-f0e616d1d99d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:44:53 crc kubenswrapper[5052]: I1203 12:44:53.129430 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/e08a0c0d-36f0-49f1-9df8-f0e616d1d99d-scripts\") pod \"ceilometer-0\" (UID: \"e08a0c0d-36f0-49f1-9df8-f0e616d1d99d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:44:53 crc kubenswrapper[5052]: I1203 12:44:53.129478 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/e08a0c0d-36f0-49f1-9df8-f0e616d1d99d-log-httpd\") pod \"ceilometer-0\" (UID: \"e08a0c0d-36f0-49f1-9df8-f0e616d1d99d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:44:53 crc kubenswrapper[5052]: I1203 12:44:53.129546 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/e08a0c0d-36f0-49f1-9df8-f0e616d1d99d-ceilometer-tls-certs\") pod \"ceilometer-0\" (UID: \"e08a0c0d-36f0-49f1-9df8-f0e616d1d99d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:44:53 crc kubenswrapper[5052]: I1203 12:44:53.129615 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/e08a0c0d-36f0-49f1-9df8-f0e616d1d99d-config-data\") pod \"ceilometer-0\" (UID: \"e08a0c0d-36f0-49f1-9df8-f0e616d1d99d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:44:53 crc kubenswrapper[5052]: I1203 12:44:53.129739 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/e08a0c0d-36f0-49f1-9df8-f0e616d1d99d-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"e08a0c0d-36f0-49f1-9df8-f0e616d1d99d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:44:53 crc kubenswrapper[5052]: I1203 12:44:53.130171 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/e08a0c0d-36f0-49f1-9df8-f0e616d1d99d-log-httpd\") pod \"ceilometer-0\" (UID: \"e08a0c0d-36f0-49f1-9df8-f0e616d1d99d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:44:53 crc kubenswrapper[5052]: I1203 12:44:53.130255 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/e08a0c0d-36f0-49f1-9df8-f0e616d1d99d-run-httpd\") pod \"ceilometer-0\" (UID: \"e08a0c0d-36f0-49f1-9df8-f0e616d1d99d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:44:53 crc kubenswrapper[5052]: I1203 12:44:53.135646 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/e08a0c0d-36f0-49f1-9df8-f0e616d1d99d-ceilometer-tls-certs\") pod \"ceilometer-0\" (UID: \"e08a0c0d-36f0-49f1-9df8-f0e616d1d99d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:44:53 crc kubenswrapper[5052]: I1203 12:44:53.135754 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e08a0c0d-36f0-49f1-9df8-f0e616d1d99d-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"e08a0c0d-36f0-49f1-9df8-f0e616d1d99d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:44:53 crc kubenswrapper[5052]: I1203 12:44:53.136267 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/e08a0c0d-36f0-49f1-9df8-f0e616d1d99d-scripts\") pod \"ceilometer-0\" (UID: \"e08a0c0d-36f0-49f1-9df8-f0e616d1d99d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:44:53 crc kubenswrapper[5052]: I1203 12:44:53.136318 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/e08a0c0d-36f0-49f1-9df8-f0e616d1d99d-config-data\") pod \"ceilometer-0\" (UID: \"e08a0c0d-36f0-49f1-9df8-f0e616d1d99d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:44:53 crc kubenswrapper[5052]: I1203 12:44:53.136687 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/e08a0c0d-36f0-49f1-9df8-f0e616d1d99d-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"e08a0c0d-36f0-49f1-9df8-f0e616d1d99d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:44:53 crc kubenswrapper[5052]: I1203 12:44:53.149296 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-r65kf\" (UniqueName: \"kubernetes.io/projected/e08a0c0d-36f0-49f1-9df8-f0e616d1d99d-kube-api-access-r65kf\") pod \"ceilometer-0\" (UID: \"e08a0c0d-36f0-49f1-9df8-f0e616d1d99d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:44:53 crc kubenswrapper[5052]: I1203 12:44:53.313337 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:44:53 crc kubenswrapper[5052]: I1203 12:44:53.693248 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3b455122-e3fd-44a8-baa3-2b10f856bfce" path="/var/lib/kubelet/pods/3b455122-e3fd-44a8-baa3-2b10f856bfce/volumes" Dec 03 12:44:53 crc kubenswrapper[5052]: I1203 12:44:53.695315 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ad32b585-41d2-4a7e-a522-069824b84d97" path="/var/lib/kubelet/pods/ad32b585-41d2-4a7e-a522-069824b84d97/volumes" Dec 03 12:44:53 crc kubenswrapper[5052]: I1203 12:44:53.832654 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:44:53 crc kubenswrapper[5052]: I1203 12:44:53.912599 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"047b4ef7-6147-49da-b4c8-382319e7f83a","Type":"ContainerStarted","Data":"722c7be4df16aafcde67740e1e015e377d9008419180cd45892921e7e51a19ef"} Dec 03 12:44:53 crc kubenswrapper[5052]: I1203 12:44:53.912642 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"047b4ef7-6147-49da-b4c8-382319e7f83a","Type":"ContainerStarted","Data":"bd4fb5170e931a743cac2a0ebfb0fe722c35f2535a615030fafe0f1d840d184e"} Dec 03 12:44:53 crc kubenswrapper[5052]: I1203 12:44:53.912652 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"047b4ef7-6147-49da-b4c8-382319e7f83a","Type":"ContainerStarted","Data":"87d968842d0f87896c822b6ced40469c04622652bd585174203619aac3818328"} Dec 03 12:44:53 crc kubenswrapper[5052]: I1203 12:44:53.914946 5052 generic.go:334] "Generic (PLEG): container finished" podID="62c6a5e0-6e22-456b-bf69-3ca1d0357b65" containerID="1dc1d18beb6efe5606641ce856ba08888e10021eaa52fe8e30fa9b5ad38c7b63" exitCode=0 Dec 03 12:44:53 crc kubenswrapper[5052]: I1203 12:44:53.915019 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"62c6a5e0-6e22-456b-bf69-3ca1d0357b65","Type":"ContainerDied","Data":"1dc1d18beb6efe5606641ce856ba08888e10021eaa52fe8e30fa9b5ad38c7b63"} Dec 03 12:44:53 crc kubenswrapper[5052]: I1203 12:44:53.938537 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-metadata-0" podStartSLOduration=1.938500147 podStartE2EDuration="1.938500147s" podCreationTimestamp="2025-12-03 12:44:52 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:44:53.937868329 +0000 UTC m=+5084.526110093" watchObservedRunningTime="2025-12-03 12:44:53.938500147 +0000 UTC m=+5084.526741911" Dec 03 12:44:54 crc kubenswrapper[5052]: I1203 12:44:54.427318 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:44:54 crc kubenswrapper[5052]: I1203 12:44:54.455058 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/62c6a5e0-6e22-456b-bf69-3ca1d0357b65-combined-ca-bundle\") pod \"62c6a5e0-6e22-456b-bf69-3ca1d0357b65\" (UID: \"62c6a5e0-6e22-456b-bf69-3ca1d0357b65\") " Dec 03 12:44:54 crc kubenswrapper[5052]: I1203 12:44:54.455165 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xm2hw\" (UniqueName: \"kubernetes.io/projected/62c6a5e0-6e22-456b-bf69-3ca1d0357b65-kube-api-access-xm2hw\") pod \"62c6a5e0-6e22-456b-bf69-3ca1d0357b65\" (UID: \"62c6a5e0-6e22-456b-bf69-3ca1d0357b65\") " Dec 03 12:44:54 crc kubenswrapper[5052]: I1203 12:44:54.455358 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/62c6a5e0-6e22-456b-bf69-3ca1d0357b65-config-data\") pod \"62c6a5e0-6e22-456b-bf69-3ca1d0357b65\" (UID: \"62c6a5e0-6e22-456b-bf69-3ca1d0357b65\") " Dec 03 12:44:54 crc kubenswrapper[5052]: I1203 12:44:54.469742 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/62c6a5e0-6e22-456b-bf69-3ca1d0357b65-kube-api-access-xm2hw" (OuterVolumeSpecName: "kube-api-access-xm2hw") pod "62c6a5e0-6e22-456b-bf69-3ca1d0357b65" (UID: "62c6a5e0-6e22-456b-bf69-3ca1d0357b65"). InnerVolumeSpecName "kube-api-access-xm2hw". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:44:54 crc kubenswrapper[5052]: I1203 12:44:54.487859 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/62c6a5e0-6e22-456b-bf69-3ca1d0357b65-config-data" (OuterVolumeSpecName: "config-data") pod "62c6a5e0-6e22-456b-bf69-3ca1d0357b65" (UID: "62c6a5e0-6e22-456b-bf69-3ca1d0357b65"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:44:54 crc kubenswrapper[5052]: I1203 12:44:54.513804 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/62c6a5e0-6e22-456b-bf69-3ca1d0357b65-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "62c6a5e0-6e22-456b-bf69-3ca1d0357b65" (UID: "62c6a5e0-6e22-456b-bf69-3ca1d0357b65"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:44:54 crc kubenswrapper[5052]: I1203 12:44:54.560394 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/62c6a5e0-6e22-456b-bf69-3ca1d0357b65-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:44:54 crc kubenswrapper[5052]: I1203 12:44:54.560447 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/62c6a5e0-6e22-456b-bf69-3ca1d0357b65-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:44:54 crc kubenswrapper[5052]: I1203 12:44:54.560464 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-xm2hw\" (UniqueName: \"kubernetes.io/projected/62c6a5e0-6e22-456b-bf69-3ca1d0357b65-kube-api-access-xm2hw\") on node \"crc\" DevicePath \"\"" Dec 03 12:44:54 crc kubenswrapper[5052]: I1203 12:44:54.954490 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"62c6a5e0-6e22-456b-bf69-3ca1d0357b65","Type":"ContainerDied","Data":"87db4bdda192df4693b10e926e0e336b8213e8eae657ef6d1333181a064f7c31"} Dec 03 12:44:54 crc kubenswrapper[5052]: I1203 12:44:54.955152 5052 scope.go:117] "RemoveContainer" containerID="1dc1d18beb6efe5606641ce856ba08888e10021eaa52fe8e30fa9b5ad38c7b63" Dec 03 12:44:54 crc kubenswrapper[5052]: I1203 12:44:54.954764 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:44:54 crc kubenswrapper[5052]: I1203 12:44:54.958730 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"e08a0c0d-36f0-49f1-9df8-f0e616d1d99d","Type":"ContainerStarted","Data":"0af06a3fd779532e53d9dcc3c63361b2314d6b161c27df98ee0efb27b89b5b76"} Dec 03 12:44:55 crc kubenswrapper[5052]: I1203 12:44:55.021034 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:44:55 crc kubenswrapper[5052]: I1203 12:44:55.030315 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:44:55 crc kubenswrapper[5052]: I1203 12:44:55.040465 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:44:55 crc kubenswrapper[5052]: E1203 12:44:55.041077 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="62c6a5e0-6e22-456b-bf69-3ca1d0357b65" containerName="nova-scheduler-scheduler" Dec 03 12:44:55 crc kubenswrapper[5052]: I1203 12:44:55.041104 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="62c6a5e0-6e22-456b-bf69-3ca1d0357b65" containerName="nova-scheduler-scheduler" Dec 03 12:44:55 crc kubenswrapper[5052]: I1203 12:44:55.041346 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="62c6a5e0-6e22-456b-bf69-3ca1d0357b65" containerName="nova-scheduler-scheduler" Dec 03 12:44:55 crc kubenswrapper[5052]: I1203 12:44:55.042224 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:44:55 crc kubenswrapper[5052]: I1203 12:44:55.045308 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-scheduler-config-data" Dec 03 12:44:55 crc kubenswrapper[5052]: I1203 12:44:55.064274 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:44:55 crc kubenswrapper[5052]: I1203 12:44:55.071370 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/9a4cb938-bf89-4591-a358-3547074f4e97-config-data\") pod \"nova-scheduler-0\" (UID: \"9a4cb938-bf89-4591-a358-3547074f4e97\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:44:55 crc kubenswrapper[5052]: I1203 12:44:55.071460 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2vt44\" (UniqueName: \"kubernetes.io/projected/9a4cb938-bf89-4591-a358-3547074f4e97-kube-api-access-2vt44\") pod \"nova-scheduler-0\" (UID: \"9a4cb938-bf89-4591-a358-3547074f4e97\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:44:55 crc kubenswrapper[5052]: I1203 12:44:55.071483 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9a4cb938-bf89-4591-a358-3547074f4e97-combined-ca-bundle\") pod \"nova-scheduler-0\" (UID: \"9a4cb938-bf89-4591-a358-3547074f4e97\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:44:55 crc kubenswrapper[5052]: I1203 12:44:55.172503 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-2vt44\" (UniqueName: \"kubernetes.io/projected/9a4cb938-bf89-4591-a358-3547074f4e97-kube-api-access-2vt44\") pod \"nova-scheduler-0\" (UID: \"9a4cb938-bf89-4591-a358-3547074f4e97\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:44:55 crc kubenswrapper[5052]: I1203 12:44:55.172565 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9a4cb938-bf89-4591-a358-3547074f4e97-combined-ca-bundle\") pod \"nova-scheduler-0\" (UID: \"9a4cb938-bf89-4591-a358-3547074f4e97\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:44:55 crc kubenswrapper[5052]: I1203 12:44:55.172677 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/9a4cb938-bf89-4591-a358-3547074f4e97-config-data\") pod \"nova-scheduler-0\" (UID: \"9a4cb938-bf89-4591-a358-3547074f4e97\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:44:55 crc kubenswrapper[5052]: I1203 12:44:55.180576 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9a4cb938-bf89-4591-a358-3547074f4e97-combined-ca-bundle\") pod \"nova-scheduler-0\" (UID: \"9a4cb938-bf89-4591-a358-3547074f4e97\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:44:55 crc kubenswrapper[5052]: I1203 12:44:55.183225 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/9a4cb938-bf89-4591-a358-3547074f4e97-config-data\") pod \"nova-scheduler-0\" (UID: \"9a4cb938-bf89-4591-a358-3547074f4e97\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:44:55 crc kubenswrapper[5052]: I1203 12:44:55.191448 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-2vt44\" (UniqueName: \"kubernetes.io/projected/9a4cb938-bf89-4591-a358-3547074f4e97-kube-api-access-2vt44\") pod \"nova-scheduler-0\" (UID: \"9a4cb938-bf89-4591-a358-3547074f4e97\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:44:55 crc kubenswrapper[5052]: I1203 12:44:55.365351 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:44:55 crc kubenswrapper[5052]: I1203 12:44:55.559461 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:44:55 crc kubenswrapper[5052]: I1203 12:44:55.594986 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:44:55 crc kubenswrapper[5052]: I1203 12:44:55.688364 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="62c6a5e0-6e22-456b-bf69-3ca1d0357b65" path="/var/lib/kubelet/pods/62c6a5e0-6e22-456b-bf69-3ca1d0357b65/volumes" Dec 03 12:44:55 crc kubenswrapper[5052]: I1203 12:44:55.866977 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:44:55 crc kubenswrapper[5052]: I1203 12:44:55.970788 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"9a4cb938-bf89-4591-a358-3547074f4e97","Type":"ContainerStarted","Data":"5196b2768cacc0eb8223f8ca3c0a7b1894f90ef80461961ec4cb2b07221e51f9"} Dec 03 12:44:55 crc kubenswrapper[5052]: I1203 12:44:55.973748 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"e08a0c0d-36f0-49f1-9df8-f0e616d1d99d","Type":"ContainerStarted","Data":"6e4c007c3a17262d2fde8b64aa6bf056a29b57cd2ef4649732b01c4b83ed1119"} Dec 03 12:44:55 crc kubenswrapper[5052]: I1203 12:44:55.973797 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"e08a0c0d-36f0-49f1-9df8-f0e616d1d99d","Type":"ContainerStarted","Data":"d957f6d4ec3ecf5552e7abc256da98e11212433b834afa67d861c95a81528ec6"} Dec 03 12:44:56 crc kubenswrapper[5052]: I1203 12:44:56.000634 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:44:56 crc kubenswrapper[5052]: I1203 12:44:56.235349 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell1-cell-mapping-rztxs"] Dec 03 12:44:56 crc kubenswrapper[5052]: I1203 12:44:56.237117 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-cell-mapping-rztxs" Dec 03 12:44:56 crc kubenswrapper[5052]: I1203 12:44:56.237726 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:44:56 crc kubenswrapper[5052]: I1203 12:44:56.246304 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell1-manage-config-data" Dec 03 12:44:56 crc kubenswrapper[5052]: I1203 12:44:56.246549 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell1-manage-scripts" Dec 03 12:44:56 crc kubenswrapper[5052]: I1203 12:44:56.263007 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-cell-mapping-rztxs"] Dec 03 12:44:56 crc kubenswrapper[5052]: I1203 12:44:56.402400 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/15239b5b-4c53-4ab2-aadf-877baf8e7b0b-scripts\") pod \"nova-cell1-cell-mapping-rztxs\" (UID: \"15239b5b-4c53-4ab2-aadf-877baf8e7b0b\") " pod="openstack-kuttl-tests/nova-cell1-cell-mapping-rztxs" Dec 03 12:44:56 crc kubenswrapper[5052]: I1203 12:44:56.402560 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/15239b5b-4c53-4ab2-aadf-877baf8e7b0b-combined-ca-bundle\") pod \"nova-cell1-cell-mapping-rztxs\" (UID: \"15239b5b-4c53-4ab2-aadf-877baf8e7b0b\") " pod="openstack-kuttl-tests/nova-cell1-cell-mapping-rztxs" Dec 03 12:44:56 crc kubenswrapper[5052]: I1203 12:44:56.402739 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/15239b5b-4c53-4ab2-aadf-877baf8e7b0b-config-data\") pod \"nova-cell1-cell-mapping-rztxs\" (UID: \"15239b5b-4c53-4ab2-aadf-877baf8e7b0b\") " pod="openstack-kuttl-tests/nova-cell1-cell-mapping-rztxs" Dec 03 12:44:56 crc kubenswrapper[5052]: I1203 12:44:56.402837 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x49nn\" (UniqueName: \"kubernetes.io/projected/15239b5b-4c53-4ab2-aadf-877baf8e7b0b-kube-api-access-x49nn\") pod \"nova-cell1-cell-mapping-rztxs\" (UID: \"15239b5b-4c53-4ab2-aadf-877baf8e7b0b\") " pod="openstack-kuttl-tests/nova-cell1-cell-mapping-rztxs" Dec 03 12:44:56 crc kubenswrapper[5052]: I1203 12:44:56.505063 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/15239b5b-4c53-4ab2-aadf-877baf8e7b0b-combined-ca-bundle\") pod \"nova-cell1-cell-mapping-rztxs\" (UID: \"15239b5b-4c53-4ab2-aadf-877baf8e7b0b\") " pod="openstack-kuttl-tests/nova-cell1-cell-mapping-rztxs" Dec 03 12:44:56 crc kubenswrapper[5052]: I1203 12:44:56.505176 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/15239b5b-4c53-4ab2-aadf-877baf8e7b0b-config-data\") pod \"nova-cell1-cell-mapping-rztxs\" (UID: \"15239b5b-4c53-4ab2-aadf-877baf8e7b0b\") " pod="openstack-kuttl-tests/nova-cell1-cell-mapping-rztxs" Dec 03 12:44:56 crc kubenswrapper[5052]: I1203 12:44:56.505214 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-x49nn\" (UniqueName: \"kubernetes.io/projected/15239b5b-4c53-4ab2-aadf-877baf8e7b0b-kube-api-access-x49nn\") pod \"nova-cell1-cell-mapping-rztxs\" (UID: \"15239b5b-4c53-4ab2-aadf-877baf8e7b0b\") " pod="openstack-kuttl-tests/nova-cell1-cell-mapping-rztxs" Dec 03 12:44:56 crc kubenswrapper[5052]: I1203 12:44:56.505322 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/15239b5b-4c53-4ab2-aadf-877baf8e7b0b-scripts\") pod \"nova-cell1-cell-mapping-rztxs\" (UID: \"15239b5b-4c53-4ab2-aadf-877baf8e7b0b\") " pod="openstack-kuttl-tests/nova-cell1-cell-mapping-rztxs" Dec 03 12:44:56 crc kubenswrapper[5052]: I1203 12:44:56.509854 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/15239b5b-4c53-4ab2-aadf-877baf8e7b0b-config-data\") pod \"nova-cell1-cell-mapping-rztxs\" (UID: \"15239b5b-4c53-4ab2-aadf-877baf8e7b0b\") " pod="openstack-kuttl-tests/nova-cell1-cell-mapping-rztxs" Dec 03 12:44:56 crc kubenswrapper[5052]: I1203 12:44:56.510025 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/15239b5b-4c53-4ab2-aadf-877baf8e7b0b-combined-ca-bundle\") pod \"nova-cell1-cell-mapping-rztxs\" (UID: \"15239b5b-4c53-4ab2-aadf-877baf8e7b0b\") " pod="openstack-kuttl-tests/nova-cell1-cell-mapping-rztxs" Dec 03 12:44:56 crc kubenswrapper[5052]: I1203 12:44:56.510753 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/15239b5b-4c53-4ab2-aadf-877baf8e7b0b-scripts\") pod \"nova-cell1-cell-mapping-rztxs\" (UID: \"15239b5b-4c53-4ab2-aadf-877baf8e7b0b\") " pod="openstack-kuttl-tests/nova-cell1-cell-mapping-rztxs" Dec 03 12:44:56 crc kubenswrapper[5052]: I1203 12:44:56.526099 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-x49nn\" (UniqueName: \"kubernetes.io/projected/15239b5b-4c53-4ab2-aadf-877baf8e7b0b-kube-api-access-x49nn\") pod \"nova-cell1-cell-mapping-rztxs\" (UID: \"15239b5b-4c53-4ab2-aadf-877baf8e7b0b\") " pod="openstack-kuttl-tests/nova-cell1-cell-mapping-rztxs" Dec 03 12:44:56 crc kubenswrapper[5052]: I1203 12:44:56.555542 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-cell-mapping-rztxs" Dec 03 12:44:56 crc kubenswrapper[5052]: I1203 12:44:56.673716 5052 scope.go:117] "RemoveContainer" containerID="2aed1c760a326f3d008799e902b1093313df3764a57c879f0b3686410127d37d" Dec 03 12:44:56 crc kubenswrapper[5052]: E1203 12:44:56.674199 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 12:44:56 crc kubenswrapper[5052]: I1203 12:44:56.990312 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"9a4cb938-bf89-4591-a358-3547074f4e97","Type":"ContainerStarted","Data":"a76d27178065bb84ace8f9e4dd1a2c7b10629af1df1c9f257684f389d4aca5c3"} Dec 03 12:44:56 crc kubenswrapper[5052]: I1203 12:44:56.996522 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"e08a0c0d-36f0-49f1-9df8-f0e616d1d99d","Type":"ContainerStarted","Data":"6b9dc66f0245c06a5ff620f79133997e00f75f954583172251c9c6a8120066db"} Dec 03 12:44:57 crc kubenswrapper[5052]: I1203 12:44:57.011151 5052 generic.go:334] "Generic (PLEG): container finished" podID="a456264f-61b5-4399-b0ec-98034fbf6523" containerID="9f70b2fb3d27878416e4c3be09e384f6b0fde07ec909f079075137a457fbe877" exitCode=0 Dec 03 12:44:57 crc kubenswrapper[5052]: I1203 12:44:57.011421 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"a456264f-61b5-4399-b0ec-98034fbf6523","Type":"ContainerDied","Data":"9f70b2fb3d27878416e4c3be09e384f6b0fde07ec909f079075137a457fbe877"} Dec 03 12:44:57 crc kubenswrapper[5052]: I1203 12:44:57.017492 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-scheduler-0" podStartSLOduration=3.01747278 podStartE2EDuration="3.01747278s" podCreationTimestamp="2025-12-03 12:44:54 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:44:57.007432613 +0000 UTC m=+5087.595674377" watchObservedRunningTime="2025-12-03 12:44:57.01747278 +0000 UTC m=+5087.605714544" Dec 03 12:44:57 crc kubenswrapper[5052]: I1203 12:44:57.091582 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-cell-mapping-rztxs"] Dec 03 12:44:57 crc kubenswrapper[5052]: W1203 12:44:57.093608 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod15239b5b_4c53_4ab2_aadf_877baf8e7b0b.slice/crio-b27d5bbe56b01cfb524564ae4da2ac4182c4845cce26e362cc211eb1216593f0 WatchSource:0}: Error finding container b27d5bbe56b01cfb524564ae4da2ac4182c4845cce26e362cc211eb1216593f0: Status 404 returned error can't find the container with id b27d5bbe56b01cfb524564ae4da2ac4182c4845cce26e362cc211eb1216593f0 Dec 03 12:44:57 crc kubenswrapper[5052]: I1203 12:44:57.442731 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:44:57 crc kubenswrapper[5052]: I1203 12:44:57.550441 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:44:57 crc kubenswrapper[5052]: I1203 12:44:57.550501 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:44:57 crc kubenswrapper[5052]: I1203 12:44:57.639865 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bpmkf\" (UniqueName: \"kubernetes.io/projected/a456264f-61b5-4399-b0ec-98034fbf6523-kube-api-access-bpmkf\") pod \"a456264f-61b5-4399-b0ec-98034fbf6523\" (UID: \"a456264f-61b5-4399-b0ec-98034fbf6523\") " Dec 03 12:44:57 crc kubenswrapper[5052]: I1203 12:44:57.639985 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a456264f-61b5-4399-b0ec-98034fbf6523-combined-ca-bundle\") pod \"a456264f-61b5-4399-b0ec-98034fbf6523\" (UID: \"a456264f-61b5-4399-b0ec-98034fbf6523\") " Dec 03 12:44:57 crc kubenswrapper[5052]: I1203 12:44:57.640042 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a456264f-61b5-4399-b0ec-98034fbf6523-config-data\") pod \"a456264f-61b5-4399-b0ec-98034fbf6523\" (UID: \"a456264f-61b5-4399-b0ec-98034fbf6523\") " Dec 03 12:44:57 crc kubenswrapper[5052]: I1203 12:44:57.640157 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/a456264f-61b5-4399-b0ec-98034fbf6523-logs\") pod \"a456264f-61b5-4399-b0ec-98034fbf6523\" (UID: \"a456264f-61b5-4399-b0ec-98034fbf6523\") " Dec 03 12:44:57 crc kubenswrapper[5052]: I1203 12:44:57.641149 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a456264f-61b5-4399-b0ec-98034fbf6523-logs" (OuterVolumeSpecName: "logs") pod "a456264f-61b5-4399-b0ec-98034fbf6523" (UID: "a456264f-61b5-4399-b0ec-98034fbf6523"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:44:57 crc kubenswrapper[5052]: I1203 12:44:57.653850 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a456264f-61b5-4399-b0ec-98034fbf6523-kube-api-access-bpmkf" (OuterVolumeSpecName: "kube-api-access-bpmkf") pod "a456264f-61b5-4399-b0ec-98034fbf6523" (UID: "a456264f-61b5-4399-b0ec-98034fbf6523"). InnerVolumeSpecName "kube-api-access-bpmkf". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:44:57 crc kubenswrapper[5052]: I1203 12:44:57.668195 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a456264f-61b5-4399-b0ec-98034fbf6523-config-data" (OuterVolumeSpecName: "config-data") pod "a456264f-61b5-4399-b0ec-98034fbf6523" (UID: "a456264f-61b5-4399-b0ec-98034fbf6523"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:44:57 crc kubenswrapper[5052]: I1203 12:44:57.680923 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a456264f-61b5-4399-b0ec-98034fbf6523-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "a456264f-61b5-4399-b0ec-98034fbf6523" (UID: "a456264f-61b5-4399-b0ec-98034fbf6523"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:44:57 crc kubenswrapper[5052]: I1203 12:44:57.743547 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/a456264f-61b5-4399-b0ec-98034fbf6523-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:44:57 crc kubenswrapper[5052]: I1203 12:44:57.743864 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-bpmkf\" (UniqueName: \"kubernetes.io/projected/a456264f-61b5-4399-b0ec-98034fbf6523-kube-api-access-bpmkf\") on node \"crc\" DevicePath \"\"" Dec 03 12:44:57 crc kubenswrapper[5052]: I1203 12:44:57.743915 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a456264f-61b5-4399-b0ec-98034fbf6523-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:44:57 crc kubenswrapper[5052]: I1203 12:44:57.743933 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a456264f-61b5-4399-b0ec-98034fbf6523-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:44:58 crc kubenswrapper[5052]: I1203 12:44:58.023555 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:44:58 crc kubenswrapper[5052]: I1203 12:44:58.023551 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"a456264f-61b5-4399-b0ec-98034fbf6523","Type":"ContainerDied","Data":"d362b688f63a81bdd27907e4ed69571ef3149d8831d6e41961b15dd2bfce1391"} Dec 03 12:44:58 crc kubenswrapper[5052]: I1203 12:44:58.023958 5052 scope.go:117] "RemoveContainer" containerID="9f70b2fb3d27878416e4c3be09e384f6b0fde07ec909f079075137a457fbe877" Dec 03 12:44:58 crc kubenswrapper[5052]: I1203 12:44:58.029307 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-cell-mapping-rztxs" event={"ID":"15239b5b-4c53-4ab2-aadf-877baf8e7b0b","Type":"ContainerStarted","Data":"2027079a1fe5e803e78ab8612d4ce2d26d54f6fbbe2c40d7cccb7ba4a5c1f6f0"} Dec 03 12:44:58 crc kubenswrapper[5052]: I1203 12:44:58.029364 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-cell-mapping-rztxs" event={"ID":"15239b5b-4c53-4ab2-aadf-877baf8e7b0b","Type":"ContainerStarted","Data":"b27d5bbe56b01cfb524564ae4da2ac4182c4845cce26e362cc211eb1216593f0"} Dec 03 12:44:58 crc kubenswrapper[5052]: I1203 12:44:58.048287 5052 scope.go:117] "RemoveContainer" containerID="20314ac2879b72c5b778670025b0d747d87526b5a82e3632a17c8086995a5d1c" Dec 03 12:44:58 crc kubenswrapper[5052]: I1203 12:44:58.069173 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-cell1-cell-mapping-rztxs" podStartSLOduration=2.069147867 podStartE2EDuration="2.069147867s" podCreationTimestamp="2025-12-03 12:44:56 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:44:58.04931492 +0000 UTC m=+5088.637556684" watchObservedRunningTime="2025-12-03 12:44:58.069147867 +0000 UTC m=+5088.657389631" Dec 03 12:44:58 crc kubenswrapper[5052]: I1203 12:44:58.104569 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:44:58 crc kubenswrapper[5052]: I1203 12:44:58.120596 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:44:58 crc kubenswrapper[5052]: I1203 12:44:58.127809 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:44:58 crc kubenswrapper[5052]: E1203 12:44:58.128368 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="a456264f-61b5-4399-b0ec-98034fbf6523" containerName="nova-api-api" Dec 03 12:44:58 crc kubenswrapper[5052]: I1203 12:44:58.128397 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="a456264f-61b5-4399-b0ec-98034fbf6523" containerName="nova-api-api" Dec 03 12:44:58 crc kubenswrapper[5052]: E1203 12:44:58.128424 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="a456264f-61b5-4399-b0ec-98034fbf6523" containerName="nova-api-log" Dec 03 12:44:58 crc kubenswrapper[5052]: I1203 12:44:58.128433 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="a456264f-61b5-4399-b0ec-98034fbf6523" containerName="nova-api-log" Dec 03 12:44:58 crc kubenswrapper[5052]: I1203 12:44:58.128765 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="a456264f-61b5-4399-b0ec-98034fbf6523" containerName="nova-api-api" Dec 03 12:44:58 crc kubenswrapper[5052]: I1203 12:44:58.128801 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="a456264f-61b5-4399-b0ec-98034fbf6523" containerName="nova-api-log" Dec 03 12:44:58 crc kubenswrapper[5052]: I1203 12:44:58.130154 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:44:58 crc kubenswrapper[5052]: I1203 12:44:58.134791 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-api-config-data" Dec 03 12:44:58 crc kubenswrapper[5052]: I1203 12:44:58.140012 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:44:58 crc kubenswrapper[5052]: I1203 12:44:58.255714 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-brs7t\" (UniqueName: \"kubernetes.io/projected/1876ac95-3d75-4daf-a20d-7083b8db2726-kube-api-access-brs7t\") pod \"nova-api-0\" (UID: \"1876ac95-3d75-4daf-a20d-7083b8db2726\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:44:58 crc kubenswrapper[5052]: I1203 12:44:58.255808 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/1876ac95-3d75-4daf-a20d-7083b8db2726-logs\") pod \"nova-api-0\" (UID: \"1876ac95-3d75-4daf-a20d-7083b8db2726\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:44:58 crc kubenswrapper[5052]: I1203 12:44:58.255881 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/1876ac95-3d75-4daf-a20d-7083b8db2726-config-data\") pod \"nova-api-0\" (UID: \"1876ac95-3d75-4daf-a20d-7083b8db2726\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:44:58 crc kubenswrapper[5052]: I1203 12:44:58.255991 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1876ac95-3d75-4daf-a20d-7083b8db2726-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"1876ac95-3d75-4daf-a20d-7083b8db2726\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:44:58 crc kubenswrapper[5052]: I1203 12:44:58.358997 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-brs7t\" (UniqueName: \"kubernetes.io/projected/1876ac95-3d75-4daf-a20d-7083b8db2726-kube-api-access-brs7t\") pod \"nova-api-0\" (UID: \"1876ac95-3d75-4daf-a20d-7083b8db2726\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:44:58 crc kubenswrapper[5052]: I1203 12:44:58.359074 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/1876ac95-3d75-4daf-a20d-7083b8db2726-logs\") pod \"nova-api-0\" (UID: \"1876ac95-3d75-4daf-a20d-7083b8db2726\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:44:58 crc kubenswrapper[5052]: I1203 12:44:58.359106 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/1876ac95-3d75-4daf-a20d-7083b8db2726-config-data\") pod \"nova-api-0\" (UID: \"1876ac95-3d75-4daf-a20d-7083b8db2726\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:44:58 crc kubenswrapper[5052]: I1203 12:44:58.359143 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1876ac95-3d75-4daf-a20d-7083b8db2726-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"1876ac95-3d75-4daf-a20d-7083b8db2726\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:44:58 crc kubenswrapper[5052]: I1203 12:44:58.359910 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/1876ac95-3d75-4daf-a20d-7083b8db2726-logs\") pod \"nova-api-0\" (UID: \"1876ac95-3d75-4daf-a20d-7083b8db2726\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:44:58 crc kubenswrapper[5052]: I1203 12:44:58.365560 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1876ac95-3d75-4daf-a20d-7083b8db2726-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"1876ac95-3d75-4daf-a20d-7083b8db2726\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:44:58 crc kubenswrapper[5052]: I1203 12:44:58.365900 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/1876ac95-3d75-4daf-a20d-7083b8db2726-config-data\") pod \"nova-api-0\" (UID: \"1876ac95-3d75-4daf-a20d-7083b8db2726\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:44:58 crc kubenswrapper[5052]: I1203 12:44:58.377197 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-brs7t\" (UniqueName: \"kubernetes.io/projected/1876ac95-3d75-4daf-a20d-7083b8db2726-kube-api-access-brs7t\") pod \"nova-api-0\" (UID: \"1876ac95-3d75-4daf-a20d-7083b8db2726\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:44:58 crc kubenswrapper[5052]: I1203 12:44:58.457871 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:44:58 crc kubenswrapper[5052]: I1203 12:44:58.977849 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:44:59 crc kubenswrapper[5052]: I1203 12:44:59.044211 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"e08a0c0d-36f0-49f1-9df8-f0e616d1d99d","Type":"ContainerStarted","Data":"9a2e22eb61df2421257c5aa00a47a7949877fdb593a1d8c3d3099b33f82916e0"} Dec 03 12:44:59 crc kubenswrapper[5052]: I1203 12:44:59.045662 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:44:59 crc kubenswrapper[5052]: I1203 12:44:59.047529 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"1876ac95-3d75-4daf-a20d-7083b8db2726","Type":"ContainerStarted","Data":"93db5c46011640c2fe4a5fd400b5f6333d98439e2b5ec7ac804e45d5d7459e33"} Dec 03 12:44:59 crc kubenswrapper[5052]: I1203 12:44:59.081697 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/ceilometer-0" podStartSLOduration=2.884095839 podStartE2EDuration="7.081675003s" podCreationTimestamp="2025-12-03 12:44:52 +0000 UTC" firstStartedPulling="2025-12-03 12:44:54.332747052 +0000 UTC m=+5084.920988816" lastFinishedPulling="2025-12-03 12:44:58.530326226 +0000 UTC m=+5089.118567980" observedRunningTime="2025-12-03 12:44:59.07496042 +0000 UTC m=+5089.663202194" watchObservedRunningTime="2025-12-03 12:44:59.081675003 +0000 UTC m=+5089.669916767" Dec 03 12:44:59 crc kubenswrapper[5052]: I1203 12:44:59.695912 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a456264f-61b5-4399-b0ec-98034fbf6523" path="/var/lib/kubelet/pods/a456264f-61b5-4399-b0ec-98034fbf6523/volumes" Dec 03 12:45:00 crc kubenswrapper[5052]: I1203 12:45:00.063812 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"1876ac95-3d75-4daf-a20d-7083b8db2726","Type":"ContainerStarted","Data":"a779a3f0d0dc7495a8afb6b95c1aa3f7bdb4f7702e17e85e81939edcae2cefd6"} Dec 03 12:45:00 crc kubenswrapper[5052]: I1203 12:45:00.063855 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"1876ac95-3d75-4daf-a20d-7083b8db2726","Type":"ContainerStarted","Data":"46d60fc3bc984e06c7e882ddb5d331ec15cfd6a0bcbeb3a0e631ad4f1359a9a2"} Dec 03 12:45:00 crc kubenswrapper[5052]: I1203 12:45:00.087740 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-api-0" podStartSLOduration=2.0877219240000002 podStartE2EDuration="2.087721924s" podCreationTimestamp="2025-12-03 12:44:58 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:45:00.080394324 +0000 UTC m=+5090.668636098" watchObservedRunningTime="2025-12-03 12:45:00.087721924 +0000 UTC m=+5090.675963688" Dec 03 12:45:00 crc kubenswrapper[5052]: I1203 12:45:00.140910 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openshift-operator-lifecycle-manager/collect-profiles-29412765-wndkq"] Dec 03 12:45:00 crc kubenswrapper[5052]: I1203 12:45:00.142667 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-operator-lifecycle-manager/collect-profiles-29412765-wndkq" Dec 03 12:45:00 crc kubenswrapper[5052]: I1203 12:45:00.144946 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-operator-lifecycle-manager"/"collect-profiles-config" Dec 03 12:45:00 crc kubenswrapper[5052]: I1203 12:45:00.144998 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-operator-lifecycle-manager"/"collect-profiles-dockercfg-kzf4t" Dec 03 12:45:00 crc kubenswrapper[5052]: I1203 12:45:00.187808 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-operator-lifecycle-manager/collect-profiles-29412765-wndkq"] Dec 03 12:45:00 crc kubenswrapper[5052]: I1203 12:45:00.303596 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/64cc53c7-bf05-4064-98ee-ae5c931896cd-config-volume\") pod \"collect-profiles-29412765-wndkq\" (UID: \"64cc53c7-bf05-4064-98ee-ae5c931896cd\") " pod="openshift-operator-lifecycle-manager/collect-profiles-29412765-wndkq" Dec 03 12:45:00 crc kubenswrapper[5052]: I1203 12:45:00.303736 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-58s52\" (UniqueName: \"kubernetes.io/projected/64cc53c7-bf05-4064-98ee-ae5c931896cd-kube-api-access-58s52\") pod \"collect-profiles-29412765-wndkq\" (UID: \"64cc53c7-bf05-4064-98ee-ae5c931896cd\") " pod="openshift-operator-lifecycle-manager/collect-profiles-29412765-wndkq" Dec 03 12:45:00 crc kubenswrapper[5052]: I1203 12:45:00.303832 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-volume\" (UniqueName: \"kubernetes.io/secret/64cc53c7-bf05-4064-98ee-ae5c931896cd-secret-volume\") pod \"collect-profiles-29412765-wndkq\" (UID: \"64cc53c7-bf05-4064-98ee-ae5c931896cd\") " pod="openshift-operator-lifecycle-manager/collect-profiles-29412765-wndkq" Dec 03 12:45:00 crc kubenswrapper[5052]: I1203 12:45:00.366885 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:45:00 crc kubenswrapper[5052]: I1203 12:45:00.406186 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"secret-volume\" (UniqueName: \"kubernetes.io/secret/64cc53c7-bf05-4064-98ee-ae5c931896cd-secret-volume\") pod \"collect-profiles-29412765-wndkq\" (UID: \"64cc53c7-bf05-4064-98ee-ae5c931896cd\") " pod="openshift-operator-lifecycle-manager/collect-profiles-29412765-wndkq" Dec 03 12:45:00 crc kubenswrapper[5052]: I1203 12:45:00.406380 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/64cc53c7-bf05-4064-98ee-ae5c931896cd-config-volume\") pod \"collect-profiles-29412765-wndkq\" (UID: \"64cc53c7-bf05-4064-98ee-ae5c931896cd\") " pod="openshift-operator-lifecycle-manager/collect-profiles-29412765-wndkq" Dec 03 12:45:00 crc kubenswrapper[5052]: I1203 12:45:00.406466 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-58s52\" (UniqueName: \"kubernetes.io/projected/64cc53c7-bf05-4064-98ee-ae5c931896cd-kube-api-access-58s52\") pod \"collect-profiles-29412765-wndkq\" (UID: \"64cc53c7-bf05-4064-98ee-ae5c931896cd\") " pod="openshift-operator-lifecycle-manager/collect-profiles-29412765-wndkq" Dec 03 12:45:00 crc kubenswrapper[5052]: I1203 12:45:00.407534 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/64cc53c7-bf05-4064-98ee-ae5c931896cd-config-volume\") pod \"collect-profiles-29412765-wndkq\" (UID: \"64cc53c7-bf05-4064-98ee-ae5c931896cd\") " pod="openshift-operator-lifecycle-manager/collect-profiles-29412765-wndkq" Dec 03 12:45:00 crc kubenswrapper[5052]: I1203 12:45:00.418325 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"secret-volume\" (UniqueName: \"kubernetes.io/secret/64cc53c7-bf05-4064-98ee-ae5c931896cd-secret-volume\") pod \"collect-profiles-29412765-wndkq\" (UID: \"64cc53c7-bf05-4064-98ee-ae5c931896cd\") " pod="openshift-operator-lifecycle-manager/collect-profiles-29412765-wndkq" Dec 03 12:45:00 crc kubenswrapper[5052]: I1203 12:45:00.430525 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-58s52\" (UniqueName: \"kubernetes.io/projected/64cc53c7-bf05-4064-98ee-ae5c931896cd-kube-api-access-58s52\") pod \"collect-profiles-29412765-wndkq\" (UID: \"64cc53c7-bf05-4064-98ee-ae5c931896cd\") " pod="openshift-operator-lifecycle-manager/collect-profiles-29412765-wndkq" Dec 03 12:45:00 crc kubenswrapper[5052]: I1203 12:45:00.499000 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-operator-lifecycle-manager/collect-profiles-29412765-wndkq" Dec 03 12:45:00 crc kubenswrapper[5052]: I1203 12:45:00.972226 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-operator-lifecycle-manager/collect-profiles-29412765-wndkq"] Dec 03 12:45:00 crc kubenswrapper[5052]: W1203 12:45:00.978591 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod64cc53c7_bf05_4064_98ee_ae5c931896cd.slice/crio-f83e020bef9fab9e887b7e97c6743ba5533ce34bf5d8d1447aef92ebb6eb081d WatchSource:0}: Error finding container f83e020bef9fab9e887b7e97c6743ba5533ce34bf5d8d1447aef92ebb6eb081d: Status 404 returned error can't find the container with id f83e020bef9fab9e887b7e97c6743ba5533ce34bf5d8d1447aef92ebb6eb081d Dec 03 12:45:01 crc kubenswrapper[5052]: I1203 12:45:01.076800 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-operator-lifecycle-manager/collect-profiles-29412765-wndkq" event={"ID":"64cc53c7-bf05-4064-98ee-ae5c931896cd","Type":"ContainerStarted","Data":"f83e020bef9fab9e887b7e97c6743ba5533ce34bf5d8d1447aef92ebb6eb081d"} Dec 03 12:45:02 crc kubenswrapper[5052]: I1203 12:45:02.114762 5052 generic.go:334] "Generic (PLEG): container finished" podID="64cc53c7-bf05-4064-98ee-ae5c931896cd" containerID="cb8cc9c892038369c63d4b0785205187517f29920130257f2326f188cdfb66d9" exitCode=0 Dec 03 12:45:02 crc kubenswrapper[5052]: I1203 12:45:02.114850 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-operator-lifecycle-manager/collect-profiles-29412765-wndkq" event={"ID":"64cc53c7-bf05-4064-98ee-ae5c931896cd","Type":"ContainerDied","Data":"cb8cc9c892038369c63d4b0785205187517f29920130257f2326f188cdfb66d9"} Dec 03 12:45:02 crc kubenswrapper[5052]: I1203 12:45:02.551105 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:45:02 crc kubenswrapper[5052]: I1203 12:45:02.551994 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:45:03 crc kubenswrapper[5052]: I1203 12:45:03.130151 5052 generic.go:334] "Generic (PLEG): container finished" podID="15239b5b-4c53-4ab2-aadf-877baf8e7b0b" containerID="2027079a1fe5e803e78ab8612d4ce2d26d54f6fbbe2c40d7cccb7ba4a5c1f6f0" exitCode=0 Dec 03 12:45:03 crc kubenswrapper[5052]: I1203 12:45:03.130353 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-cell-mapping-rztxs" event={"ID":"15239b5b-4c53-4ab2-aadf-877baf8e7b0b","Type":"ContainerDied","Data":"2027079a1fe5e803e78ab8612d4ce2d26d54f6fbbe2c40d7cccb7ba4a5c1f6f0"} Dec 03 12:45:03 crc kubenswrapper[5052]: I1203 12:45:03.550031 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-operator-lifecycle-manager/collect-profiles-29412765-wndkq" Dec 03 12:45:03 crc kubenswrapper[5052]: I1203 12:45:03.560689 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-metadata-0" podUID="047b4ef7-6147-49da-b4c8-382319e7f83a" containerName="nova-metadata-metadata" probeResult="failure" output="Get \"https://10.217.1.228:8775/\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Dec 03 12:45:03 crc kubenswrapper[5052]: I1203 12:45:03.560932 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-metadata-0" podUID="047b4ef7-6147-49da-b4c8-382319e7f83a" containerName="nova-metadata-log" probeResult="failure" output="Get \"https://10.217.1.228:8775/\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Dec 03 12:45:03 crc kubenswrapper[5052]: I1203 12:45:03.714506 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/64cc53c7-bf05-4064-98ee-ae5c931896cd-config-volume\") pod \"64cc53c7-bf05-4064-98ee-ae5c931896cd\" (UID: \"64cc53c7-bf05-4064-98ee-ae5c931896cd\") " Dec 03 12:45:03 crc kubenswrapper[5052]: I1203 12:45:03.714837 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-58s52\" (UniqueName: \"kubernetes.io/projected/64cc53c7-bf05-4064-98ee-ae5c931896cd-kube-api-access-58s52\") pod \"64cc53c7-bf05-4064-98ee-ae5c931896cd\" (UID: \"64cc53c7-bf05-4064-98ee-ae5c931896cd\") " Dec 03 12:45:03 crc kubenswrapper[5052]: I1203 12:45:03.715385 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/64cc53c7-bf05-4064-98ee-ae5c931896cd-config-volume" (OuterVolumeSpecName: "config-volume") pod "64cc53c7-bf05-4064-98ee-ae5c931896cd" (UID: "64cc53c7-bf05-4064-98ee-ae5c931896cd"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:45:03 crc kubenswrapper[5052]: I1203 12:45:03.715814 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"secret-volume\" (UniqueName: \"kubernetes.io/secret/64cc53c7-bf05-4064-98ee-ae5c931896cd-secret-volume\") pod \"64cc53c7-bf05-4064-98ee-ae5c931896cd\" (UID: \"64cc53c7-bf05-4064-98ee-ae5c931896cd\") " Dec 03 12:45:03 crc kubenswrapper[5052]: I1203 12:45:03.716363 5052 reconciler_common.go:293] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/64cc53c7-bf05-4064-98ee-ae5c931896cd-config-volume\") on node \"crc\" DevicePath \"\"" Dec 03 12:45:03 crc kubenswrapper[5052]: I1203 12:45:03.721099 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/64cc53c7-bf05-4064-98ee-ae5c931896cd-secret-volume" (OuterVolumeSpecName: "secret-volume") pod "64cc53c7-bf05-4064-98ee-ae5c931896cd" (UID: "64cc53c7-bf05-4064-98ee-ae5c931896cd"). InnerVolumeSpecName "secret-volume". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:45:03 crc kubenswrapper[5052]: I1203 12:45:03.721347 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/64cc53c7-bf05-4064-98ee-ae5c931896cd-kube-api-access-58s52" (OuterVolumeSpecName: "kube-api-access-58s52") pod "64cc53c7-bf05-4064-98ee-ae5c931896cd" (UID: "64cc53c7-bf05-4064-98ee-ae5c931896cd"). InnerVolumeSpecName "kube-api-access-58s52". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:45:03 crc kubenswrapper[5052]: I1203 12:45:03.818110 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-58s52\" (UniqueName: \"kubernetes.io/projected/64cc53c7-bf05-4064-98ee-ae5c931896cd-kube-api-access-58s52\") on node \"crc\" DevicePath \"\"" Dec 03 12:45:03 crc kubenswrapper[5052]: I1203 12:45:03.818341 5052 reconciler_common.go:293] "Volume detached for volume \"secret-volume\" (UniqueName: \"kubernetes.io/secret/64cc53c7-bf05-4064-98ee-ae5c931896cd-secret-volume\") on node \"crc\" DevicePath \"\"" Dec 03 12:45:04 crc kubenswrapper[5052]: I1203 12:45:04.169614 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-operator-lifecycle-manager/collect-profiles-29412765-wndkq" Dec 03 12:45:04 crc kubenswrapper[5052]: I1203 12:45:04.171140 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-operator-lifecycle-manager/collect-profiles-29412765-wndkq" event={"ID":"64cc53c7-bf05-4064-98ee-ae5c931896cd","Type":"ContainerDied","Data":"f83e020bef9fab9e887b7e97c6743ba5533ce34bf5d8d1447aef92ebb6eb081d"} Dec 03 12:45:04 crc kubenswrapper[5052]: I1203 12:45:04.171187 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f83e020bef9fab9e887b7e97c6743ba5533ce34bf5d8d1447aef92ebb6eb081d" Dec 03 12:45:04 crc kubenswrapper[5052]: I1203 12:45:04.516208 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-cell-mapping-rztxs" Dec 03 12:45:04 crc kubenswrapper[5052]: I1203 12:45:04.636424 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x49nn\" (UniqueName: \"kubernetes.io/projected/15239b5b-4c53-4ab2-aadf-877baf8e7b0b-kube-api-access-x49nn\") pod \"15239b5b-4c53-4ab2-aadf-877baf8e7b0b\" (UID: \"15239b5b-4c53-4ab2-aadf-877baf8e7b0b\") " Dec 03 12:45:04 crc kubenswrapper[5052]: I1203 12:45:04.636488 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/15239b5b-4c53-4ab2-aadf-877baf8e7b0b-combined-ca-bundle\") pod \"15239b5b-4c53-4ab2-aadf-877baf8e7b0b\" (UID: \"15239b5b-4c53-4ab2-aadf-877baf8e7b0b\") " Dec 03 12:45:04 crc kubenswrapper[5052]: I1203 12:45:04.636644 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/15239b5b-4c53-4ab2-aadf-877baf8e7b0b-config-data\") pod \"15239b5b-4c53-4ab2-aadf-877baf8e7b0b\" (UID: \"15239b5b-4c53-4ab2-aadf-877baf8e7b0b\") " Dec 03 12:45:04 crc kubenswrapper[5052]: I1203 12:45:04.636723 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/15239b5b-4c53-4ab2-aadf-877baf8e7b0b-scripts\") pod \"15239b5b-4c53-4ab2-aadf-877baf8e7b0b\" (UID: \"15239b5b-4c53-4ab2-aadf-877baf8e7b0b\") " Dec 03 12:45:04 crc kubenswrapper[5052]: I1203 12:45:04.643621 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openshift-operator-lifecycle-manager/collect-profiles-29412720-28m24"] Dec 03 12:45:04 crc kubenswrapper[5052]: I1203 12:45:04.646700 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/15239b5b-4c53-4ab2-aadf-877baf8e7b0b-scripts" (OuterVolumeSpecName: "scripts") pod "15239b5b-4c53-4ab2-aadf-877baf8e7b0b" (UID: "15239b5b-4c53-4ab2-aadf-877baf8e7b0b"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:45:04 crc kubenswrapper[5052]: I1203 12:45:04.651755 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/15239b5b-4c53-4ab2-aadf-877baf8e7b0b-kube-api-access-x49nn" (OuterVolumeSpecName: "kube-api-access-x49nn") pod "15239b5b-4c53-4ab2-aadf-877baf8e7b0b" (UID: "15239b5b-4c53-4ab2-aadf-877baf8e7b0b"). InnerVolumeSpecName "kube-api-access-x49nn". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:45:04 crc kubenswrapper[5052]: I1203 12:45:04.656210 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openshift-operator-lifecycle-manager/collect-profiles-29412720-28m24"] Dec 03 12:45:04 crc kubenswrapper[5052]: I1203 12:45:04.669901 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/15239b5b-4c53-4ab2-aadf-877baf8e7b0b-config-data" (OuterVolumeSpecName: "config-data") pod "15239b5b-4c53-4ab2-aadf-877baf8e7b0b" (UID: "15239b5b-4c53-4ab2-aadf-877baf8e7b0b"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:45:04 crc kubenswrapper[5052]: I1203 12:45:04.675500 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/15239b5b-4c53-4ab2-aadf-877baf8e7b0b-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "15239b5b-4c53-4ab2-aadf-877baf8e7b0b" (UID: "15239b5b-4c53-4ab2-aadf-877baf8e7b0b"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:45:04 crc kubenswrapper[5052]: I1203 12:45:04.739039 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-x49nn\" (UniqueName: \"kubernetes.io/projected/15239b5b-4c53-4ab2-aadf-877baf8e7b0b-kube-api-access-x49nn\") on node \"crc\" DevicePath \"\"" Dec 03 12:45:04 crc kubenswrapper[5052]: I1203 12:45:04.739421 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/15239b5b-4c53-4ab2-aadf-877baf8e7b0b-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:45:04 crc kubenswrapper[5052]: I1203 12:45:04.739498 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/15239b5b-4c53-4ab2-aadf-877baf8e7b0b-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:45:04 crc kubenswrapper[5052]: I1203 12:45:04.739576 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/15239b5b-4c53-4ab2-aadf-877baf8e7b0b-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:45:05 crc kubenswrapper[5052]: I1203 12:45:05.181588 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-cell-mapping-rztxs" event={"ID":"15239b5b-4c53-4ab2-aadf-877baf8e7b0b","Type":"ContainerDied","Data":"b27d5bbe56b01cfb524564ae4da2ac4182c4845cce26e362cc211eb1216593f0"} Dec 03 12:45:05 crc kubenswrapper[5052]: I1203 12:45:05.181640 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b27d5bbe56b01cfb524564ae4da2ac4182c4845cce26e362cc211eb1216593f0" Dec 03 12:45:05 crc kubenswrapper[5052]: I1203 12:45:05.181711 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-cell-mapping-rztxs" Dec 03 12:45:05 crc kubenswrapper[5052]: I1203 12:45:05.353180 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:45:05 crc kubenswrapper[5052]: I1203 12:45:05.353417 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-scheduler-0" podUID="9a4cb938-bf89-4591-a358-3547074f4e97" containerName="nova-scheduler-scheduler" containerID="cri-o://a76d27178065bb84ace8f9e4dd1a2c7b10629af1df1c9f257684f389d4aca5c3" gracePeriod=30 Dec 03 12:45:05 crc kubenswrapper[5052]: I1203 12:45:05.366303 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:45:05 crc kubenswrapper[5052]: I1203 12:45:05.366600 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-api-0" podUID="1876ac95-3d75-4daf-a20d-7083b8db2726" containerName="nova-api-log" containerID="cri-o://46d60fc3bc984e06c7e882ddb5d331ec15cfd6a0bcbeb3a0e631ad4f1359a9a2" gracePeriod=30 Dec 03 12:45:05 crc kubenswrapper[5052]: I1203 12:45:05.367155 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-api-0" podUID="1876ac95-3d75-4daf-a20d-7083b8db2726" containerName="nova-api-api" containerID="cri-o://a779a3f0d0dc7495a8afb6b95c1aa3f7bdb4f7702e17e85e81939edcae2cefd6" gracePeriod=30 Dec 03 12:45:05 crc kubenswrapper[5052]: I1203 12:45:05.378747 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:45:05 crc kubenswrapper[5052]: I1203 12:45:05.378995 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-metadata-0" podUID="047b4ef7-6147-49da-b4c8-382319e7f83a" containerName="nova-metadata-log" containerID="cri-o://bd4fb5170e931a743cac2a0ebfb0fe722c35f2535a615030fafe0f1d840d184e" gracePeriod=30 Dec 03 12:45:05 crc kubenswrapper[5052]: I1203 12:45:05.379085 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-metadata-0" podUID="047b4ef7-6147-49da-b4c8-382319e7f83a" containerName="nova-metadata-metadata" containerID="cri-o://722c7be4df16aafcde67740e1e015e377d9008419180cd45892921e7e51a19ef" gracePeriod=30 Dec 03 12:45:05 crc kubenswrapper[5052]: I1203 12:45:05.692474 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f268de6e-4938-4114-b436-9515d64d3f60" path="/var/lib/kubelet/pods/f268de6e-4938-4114-b436-9515d64d3f60/volumes" Dec 03 12:45:05 crc kubenswrapper[5052]: I1203 12:45:05.975192 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:45:06 crc kubenswrapper[5052]: I1203 12:45:06.063908 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/1876ac95-3d75-4daf-a20d-7083b8db2726-config-data\") pod \"1876ac95-3d75-4daf-a20d-7083b8db2726\" (UID: \"1876ac95-3d75-4daf-a20d-7083b8db2726\") " Dec 03 12:45:06 crc kubenswrapper[5052]: I1203 12:45:06.063995 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/1876ac95-3d75-4daf-a20d-7083b8db2726-logs\") pod \"1876ac95-3d75-4daf-a20d-7083b8db2726\" (UID: \"1876ac95-3d75-4daf-a20d-7083b8db2726\") " Dec 03 12:45:06 crc kubenswrapper[5052]: I1203 12:45:06.064098 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1876ac95-3d75-4daf-a20d-7083b8db2726-combined-ca-bundle\") pod \"1876ac95-3d75-4daf-a20d-7083b8db2726\" (UID: \"1876ac95-3d75-4daf-a20d-7083b8db2726\") " Dec 03 12:45:06 crc kubenswrapper[5052]: I1203 12:45:06.064145 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-brs7t\" (UniqueName: \"kubernetes.io/projected/1876ac95-3d75-4daf-a20d-7083b8db2726-kube-api-access-brs7t\") pod \"1876ac95-3d75-4daf-a20d-7083b8db2726\" (UID: \"1876ac95-3d75-4daf-a20d-7083b8db2726\") " Dec 03 12:45:06 crc kubenswrapper[5052]: I1203 12:45:06.064342 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1876ac95-3d75-4daf-a20d-7083b8db2726-logs" (OuterVolumeSpecName: "logs") pod "1876ac95-3d75-4daf-a20d-7083b8db2726" (UID: "1876ac95-3d75-4daf-a20d-7083b8db2726"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:45:06 crc kubenswrapper[5052]: I1203 12:45:06.065035 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/1876ac95-3d75-4daf-a20d-7083b8db2726-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:45:06 crc kubenswrapper[5052]: I1203 12:45:06.068769 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1876ac95-3d75-4daf-a20d-7083b8db2726-kube-api-access-brs7t" (OuterVolumeSpecName: "kube-api-access-brs7t") pod "1876ac95-3d75-4daf-a20d-7083b8db2726" (UID: "1876ac95-3d75-4daf-a20d-7083b8db2726"). InnerVolumeSpecName "kube-api-access-brs7t". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:45:06 crc kubenswrapper[5052]: I1203 12:45:06.095501 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1876ac95-3d75-4daf-a20d-7083b8db2726-config-data" (OuterVolumeSpecName: "config-data") pod "1876ac95-3d75-4daf-a20d-7083b8db2726" (UID: "1876ac95-3d75-4daf-a20d-7083b8db2726"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:45:06 crc kubenswrapper[5052]: I1203 12:45:06.098047 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1876ac95-3d75-4daf-a20d-7083b8db2726-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "1876ac95-3d75-4daf-a20d-7083b8db2726" (UID: "1876ac95-3d75-4daf-a20d-7083b8db2726"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:45:06 crc kubenswrapper[5052]: I1203 12:45:06.167204 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1876ac95-3d75-4daf-a20d-7083b8db2726-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:45:06 crc kubenswrapper[5052]: I1203 12:45:06.167246 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-brs7t\" (UniqueName: \"kubernetes.io/projected/1876ac95-3d75-4daf-a20d-7083b8db2726-kube-api-access-brs7t\") on node \"crc\" DevicePath \"\"" Dec 03 12:45:06 crc kubenswrapper[5052]: I1203 12:45:06.167257 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/1876ac95-3d75-4daf-a20d-7083b8db2726-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:45:06 crc kubenswrapper[5052]: I1203 12:45:06.194098 5052 generic.go:334] "Generic (PLEG): container finished" podID="1876ac95-3d75-4daf-a20d-7083b8db2726" containerID="a779a3f0d0dc7495a8afb6b95c1aa3f7bdb4f7702e17e85e81939edcae2cefd6" exitCode=0 Dec 03 12:45:06 crc kubenswrapper[5052]: I1203 12:45:06.194137 5052 generic.go:334] "Generic (PLEG): container finished" podID="1876ac95-3d75-4daf-a20d-7083b8db2726" containerID="46d60fc3bc984e06c7e882ddb5d331ec15cfd6a0bcbeb3a0e631ad4f1359a9a2" exitCode=143 Dec 03 12:45:06 crc kubenswrapper[5052]: I1203 12:45:06.194149 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:45:06 crc kubenswrapper[5052]: I1203 12:45:06.194187 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"1876ac95-3d75-4daf-a20d-7083b8db2726","Type":"ContainerDied","Data":"a779a3f0d0dc7495a8afb6b95c1aa3f7bdb4f7702e17e85e81939edcae2cefd6"} Dec 03 12:45:06 crc kubenswrapper[5052]: I1203 12:45:06.194283 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"1876ac95-3d75-4daf-a20d-7083b8db2726","Type":"ContainerDied","Data":"46d60fc3bc984e06c7e882ddb5d331ec15cfd6a0bcbeb3a0e631ad4f1359a9a2"} Dec 03 12:45:06 crc kubenswrapper[5052]: I1203 12:45:06.194385 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"1876ac95-3d75-4daf-a20d-7083b8db2726","Type":"ContainerDied","Data":"93db5c46011640c2fe4a5fd400b5f6333d98439e2b5ec7ac804e45d5d7459e33"} Dec 03 12:45:06 crc kubenswrapper[5052]: I1203 12:45:06.194542 5052 scope.go:117] "RemoveContainer" containerID="a779a3f0d0dc7495a8afb6b95c1aa3f7bdb4f7702e17e85e81939edcae2cefd6" Dec 03 12:45:06 crc kubenswrapper[5052]: I1203 12:45:06.197041 5052 generic.go:334] "Generic (PLEG): container finished" podID="047b4ef7-6147-49da-b4c8-382319e7f83a" containerID="bd4fb5170e931a743cac2a0ebfb0fe722c35f2535a615030fafe0f1d840d184e" exitCode=143 Dec 03 12:45:06 crc kubenswrapper[5052]: I1203 12:45:06.197073 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"047b4ef7-6147-49da-b4c8-382319e7f83a","Type":"ContainerDied","Data":"bd4fb5170e931a743cac2a0ebfb0fe722c35f2535a615030fafe0f1d840d184e"} Dec 03 12:45:06 crc kubenswrapper[5052]: I1203 12:45:06.224198 5052 scope.go:117] "RemoveContainer" containerID="46d60fc3bc984e06c7e882ddb5d331ec15cfd6a0bcbeb3a0e631ad4f1359a9a2" Dec 03 12:45:06 crc kubenswrapper[5052]: I1203 12:45:06.243443 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:45:06 crc kubenswrapper[5052]: I1203 12:45:06.251267 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:45:06 crc kubenswrapper[5052]: I1203 12:45:06.255467 5052 scope.go:117] "RemoveContainer" containerID="a779a3f0d0dc7495a8afb6b95c1aa3f7bdb4f7702e17e85e81939edcae2cefd6" Dec 03 12:45:06 crc kubenswrapper[5052]: E1203 12:45:06.256016 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a779a3f0d0dc7495a8afb6b95c1aa3f7bdb4f7702e17e85e81939edcae2cefd6\": container with ID starting with a779a3f0d0dc7495a8afb6b95c1aa3f7bdb4f7702e17e85e81939edcae2cefd6 not found: ID does not exist" containerID="a779a3f0d0dc7495a8afb6b95c1aa3f7bdb4f7702e17e85e81939edcae2cefd6" Dec 03 12:45:06 crc kubenswrapper[5052]: I1203 12:45:06.256054 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a779a3f0d0dc7495a8afb6b95c1aa3f7bdb4f7702e17e85e81939edcae2cefd6"} err="failed to get container status \"a779a3f0d0dc7495a8afb6b95c1aa3f7bdb4f7702e17e85e81939edcae2cefd6\": rpc error: code = NotFound desc = could not find container \"a779a3f0d0dc7495a8afb6b95c1aa3f7bdb4f7702e17e85e81939edcae2cefd6\": container with ID starting with a779a3f0d0dc7495a8afb6b95c1aa3f7bdb4f7702e17e85e81939edcae2cefd6 not found: ID does not exist" Dec 03 12:45:06 crc kubenswrapper[5052]: I1203 12:45:06.256080 5052 scope.go:117] "RemoveContainer" containerID="46d60fc3bc984e06c7e882ddb5d331ec15cfd6a0bcbeb3a0e631ad4f1359a9a2" Dec 03 12:45:06 crc kubenswrapper[5052]: E1203 12:45:06.256568 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"46d60fc3bc984e06c7e882ddb5d331ec15cfd6a0bcbeb3a0e631ad4f1359a9a2\": container with ID starting with 46d60fc3bc984e06c7e882ddb5d331ec15cfd6a0bcbeb3a0e631ad4f1359a9a2 not found: ID does not exist" containerID="46d60fc3bc984e06c7e882ddb5d331ec15cfd6a0bcbeb3a0e631ad4f1359a9a2" Dec 03 12:45:06 crc kubenswrapper[5052]: I1203 12:45:06.256597 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"46d60fc3bc984e06c7e882ddb5d331ec15cfd6a0bcbeb3a0e631ad4f1359a9a2"} err="failed to get container status \"46d60fc3bc984e06c7e882ddb5d331ec15cfd6a0bcbeb3a0e631ad4f1359a9a2\": rpc error: code = NotFound desc = could not find container \"46d60fc3bc984e06c7e882ddb5d331ec15cfd6a0bcbeb3a0e631ad4f1359a9a2\": container with ID starting with 46d60fc3bc984e06c7e882ddb5d331ec15cfd6a0bcbeb3a0e631ad4f1359a9a2 not found: ID does not exist" Dec 03 12:45:06 crc kubenswrapper[5052]: I1203 12:45:06.256615 5052 scope.go:117] "RemoveContainer" containerID="a779a3f0d0dc7495a8afb6b95c1aa3f7bdb4f7702e17e85e81939edcae2cefd6" Dec 03 12:45:06 crc kubenswrapper[5052]: I1203 12:45:06.256924 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a779a3f0d0dc7495a8afb6b95c1aa3f7bdb4f7702e17e85e81939edcae2cefd6"} err="failed to get container status \"a779a3f0d0dc7495a8afb6b95c1aa3f7bdb4f7702e17e85e81939edcae2cefd6\": rpc error: code = NotFound desc = could not find container \"a779a3f0d0dc7495a8afb6b95c1aa3f7bdb4f7702e17e85e81939edcae2cefd6\": container with ID starting with a779a3f0d0dc7495a8afb6b95c1aa3f7bdb4f7702e17e85e81939edcae2cefd6 not found: ID does not exist" Dec 03 12:45:06 crc kubenswrapper[5052]: I1203 12:45:06.256982 5052 scope.go:117] "RemoveContainer" containerID="46d60fc3bc984e06c7e882ddb5d331ec15cfd6a0bcbeb3a0e631ad4f1359a9a2" Dec 03 12:45:06 crc kubenswrapper[5052]: I1203 12:45:06.257333 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"46d60fc3bc984e06c7e882ddb5d331ec15cfd6a0bcbeb3a0e631ad4f1359a9a2"} err="failed to get container status \"46d60fc3bc984e06c7e882ddb5d331ec15cfd6a0bcbeb3a0e631ad4f1359a9a2\": rpc error: code = NotFound desc = could not find container \"46d60fc3bc984e06c7e882ddb5d331ec15cfd6a0bcbeb3a0e631ad4f1359a9a2\": container with ID starting with 46d60fc3bc984e06c7e882ddb5d331ec15cfd6a0bcbeb3a0e631ad4f1359a9a2 not found: ID does not exist" Dec 03 12:45:06 crc kubenswrapper[5052]: I1203 12:45:06.270361 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:45:06 crc kubenswrapper[5052]: E1203 12:45:06.270820 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="15239b5b-4c53-4ab2-aadf-877baf8e7b0b" containerName="nova-manage" Dec 03 12:45:06 crc kubenswrapper[5052]: I1203 12:45:06.270842 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="15239b5b-4c53-4ab2-aadf-877baf8e7b0b" containerName="nova-manage" Dec 03 12:45:06 crc kubenswrapper[5052]: E1203 12:45:06.270859 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="64cc53c7-bf05-4064-98ee-ae5c931896cd" containerName="collect-profiles" Dec 03 12:45:06 crc kubenswrapper[5052]: I1203 12:45:06.270868 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="64cc53c7-bf05-4064-98ee-ae5c931896cd" containerName="collect-profiles" Dec 03 12:45:06 crc kubenswrapper[5052]: E1203 12:45:06.270891 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="1876ac95-3d75-4daf-a20d-7083b8db2726" containerName="nova-api-api" Dec 03 12:45:06 crc kubenswrapper[5052]: I1203 12:45:06.270900 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="1876ac95-3d75-4daf-a20d-7083b8db2726" containerName="nova-api-api" Dec 03 12:45:06 crc kubenswrapper[5052]: E1203 12:45:06.270916 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="1876ac95-3d75-4daf-a20d-7083b8db2726" containerName="nova-api-log" Dec 03 12:45:06 crc kubenswrapper[5052]: I1203 12:45:06.270921 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="1876ac95-3d75-4daf-a20d-7083b8db2726" containerName="nova-api-log" Dec 03 12:45:06 crc kubenswrapper[5052]: I1203 12:45:06.271132 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="1876ac95-3d75-4daf-a20d-7083b8db2726" containerName="nova-api-log" Dec 03 12:45:06 crc kubenswrapper[5052]: I1203 12:45:06.271162 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="15239b5b-4c53-4ab2-aadf-877baf8e7b0b" containerName="nova-manage" Dec 03 12:45:06 crc kubenswrapper[5052]: I1203 12:45:06.271175 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="64cc53c7-bf05-4064-98ee-ae5c931896cd" containerName="collect-profiles" Dec 03 12:45:06 crc kubenswrapper[5052]: I1203 12:45:06.271185 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="1876ac95-3d75-4daf-a20d-7083b8db2726" containerName="nova-api-api" Dec 03 12:45:06 crc kubenswrapper[5052]: I1203 12:45:06.272582 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:45:06 crc kubenswrapper[5052]: I1203 12:45:06.274861 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-api-config-data" Dec 03 12:45:06 crc kubenswrapper[5052]: I1203 12:45:06.280627 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:45:06 crc kubenswrapper[5052]: I1203 12:45:06.370807 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/a6e95405-86bc-4eeb-872b-2c4fdec53bfc-logs\") pod \"nova-api-0\" (UID: \"a6e95405-86bc-4eeb-872b-2c4fdec53bfc\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:45:06 crc kubenswrapper[5052]: I1203 12:45:06.370945 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a6e95405-86bc-4eeb-872b-2c4fdec53bfc-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"a6e95405-86bc-4eeb-872b-2c4fdec53bfc\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:45:06 crc kubenswrapper[5052]: I1203 12:45:06.371046 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ktpcp\" (UniqueName: \"kubernetes.io/projected/a6e95405-86bc-4eeb-872b-2c4fdec53bfc-kube-api-access-ktpcp\") pod \"nova-api-0\" (UID: \"a6e95405-86bc-4eeb-872b-2c4fdec53bfc\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:45:06 crc kubenswrapper[5052]: I1203 12:45:06.371088 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a6e95405-86bc-4eeb-872b-2c4fdec53bfc-config-data\") pod \"nova-api-0\" (UID: \"a6e95405-86bc-4eeb-872b-2c4fdec53bfc\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:45:06 crc kubenswrapper[5052]: I1203 12:45:06.473764 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/a6e95405-86bc-4eeb-872b-2c4fdec53bfc-logs\") pod \"nova-api-0\" (UID: \"a6e95405-86bc-4eeb-872b-2c4fdec53bfc\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:45:06 crc kubenswrapper[5052]: I1203 12:45:06.473939 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a6e95405-86bc-4eeb-872b-2c4fdec53bfc-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"a6e95405-86bc-4eeb-872b-2c4fdec53bfc\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:45:06 crc kubenswrapper[5052]: I1203 12:45:06.473992 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-ktpcp\" (UniqueName: \"kubernetes.io/projected/a6e95405-86bc-4eeb-872b-2c4fdec53bfc-kube-api-access-ktpcp\") pod \"nova-api-0\" (UID: \"a6e95405-86bc-4eeb-872b-2c4fdec53bfc\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:45:06 crc kubenswrapper[5052]: I1203 12:45:06.474025 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a6e95405-86bc-4eeb-872b-2c4fdec53bfc-config-data\") pod \"nova-api-0\" (UID: \"a6e95405-86bc-4eeb-872b-2c4fdec53bfc\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:45:06 crc kubenswrapper[5052]: I1203 12:45:06.474266 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/a6e95405-86bc-4eeb-872b-2c4fdec53bfc-logs\") pod \"nova-api-0\" (UID: \"a6e95405-86bc-4eeb-872b-2c4fdec53bfc\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:45:06 crc kubenswrapper[5052]: I1203 12:45:06.479619 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a6e95405-86bc-4eeb-872b-2c4fdec53bfc-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"a6e95405-86bc-4eeb-872b-2c4fdec53bfc\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:45:06 crc kubenswrapper[5052]: I1203 12:45:06.482393 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a6e95405-86bc-4eeb-872b-2c4fdec53bfc-config-data\") pod \"nova-api-0\" (UID: \"a6e95405-86bc-4eeb-872b-2c4fdec53bfc\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:45:06 crc kubenswrapper[5052]: I1203 12:45:06.493936 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-ktpcp\" (UniqueName: \"kubernetes.io/projected/a6e95405-86bc-4eeb-872b-2c4fdec53bfc-kube-api-access-ktpcp\") pod \"nova-api-0\" (UID: \"a6e95405-86bc-4eeb-872b-2c4fdec53bfc\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:45:06 crc kubenswrapper[5052]: I1203 12:45:06.603719 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:45:07 crc kubenswrapper[5052]: I1203 12:45:07.062260 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:45:07 crc kubenswrapper[5052]: I1203 12:45:07.216093 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"a6e95405-86bc-4eeb-872b-2c4fdec53bfc","Type":"ContainerStarted","Data":"9822fd49ef468126c30884ad0173c1959e4dea1e209bb3bf1736a25863b9cb7c"} Dec 03 12:45:07 crc kubenswrapper[5052]: I1203 12:45:07.684592 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1876ac95-3d75-4daf-a20d-7083b8db2726" path="/var/lib/kubelet/pods/1876ac95-3d75-4daf-a20d-7083b8db2726/volumes" Dec 03 12:45:08 crc kubenswrapper[5052]: I1203 12:45:08.232206 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"a6e95405-86bc-4eeb-872b-2c4fdec53bfc","Type":"ContainerStarted","Data":"d576322fd69e059064fcc3a5cf54fc65c5516c5825fd163d50c7cc75e230f368"} Dec 03 12:45:08 crc kubenswrapper[5052]: I1203 12:45:08.232280 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"a6e95405-86bc-4eeb-872b-2c4fdec53bfc","Type":"ContainerStarted","Data":"ad6be9bfabfc99a6b322850260fdffb57345a6b03420e364eb41589f6201bb82"} Dec 03 12:45:08 crc kubenswrapper[5052]: I1203 12:45:08.264345 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-api-0" podStartSLOduration=2.26432014 podStartE2EDuration="2.26432014s" podCreationTimestamp="2025-12-03 12:45:06 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:45:08.258616167 +0000 UTC m=+5098.846857931" watchObservedRunningTime="2025-12-03 12:45:08.26432014 +0000 UTC m=+5098.852561904" Dec 03 12:45:09 crc kubenswrapper[5052]: I1203 12:45:09.048318 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:45:09 crc kubenswrapper[5052]: I1203 12:45:09.136882 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2vt44\" (UniqueName: \"kubernetes.io/projected/9a4cb938-bf89-4591-a358-3547074f4e97-kube-api-access-2vt44\") pod \"9a4cb938-bf89-4591-a358-3547074f4e97\" (UID: \"9a4cb938-bf89-4591-a358-3547074f4e97\") " Dec 03 12:45:09 crc kubenswrapper[5052]: I1203 12:45:09.137181 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/9a4cb938-bf89-4591-a358-3547074f4e97-config-data\") pod \"9a4cb938-bf89-4591-a358-3547074f4e97\" (UID: \"9a4cb938-bf89-4591-a358-3547074f4e97\") " Dec 03 12:45:09 crc kubenswrapper[5052]: I1203 12:45:09.137230 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9a4cb938-bf89-4591-a358-3547074f4e97-combined-ca-bundle\") pod \"9a4cb938-bf89-4591-a358-3547074f4e97\" (UID: \"9a4cb938-bf89-4591-a358-3547074f4e97\") " Dec 03 12:45:09 crc kubenswrapper[5052]: I1203 12:45:09.145656 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9a4cb938-bf89-4591-a358-3547074f4e97-kube-api-access-2vt44" (OuterVolumeSpecName: "kube-api-access-2vt44") pod "9a4cb938-bf89-4591-a358-3547074f4e97" (UID: "9a4cb938-bf89-4591-a358-3547074f4e97"). InnerVolumeSpecName "kube-api-access-2vt44". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:45:09 crc kubenswrapper[5052]: I1203 12:45:09.170214 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9a4cb938-bf89-4591-a358-3547074f4e97-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "9a4cb938-bf89-4591-a358-3547074f4e97" (UID: "9a4cb938-bf89-4591-a358-3547074f4e97"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:45:09 crc kubenswrapper[5052]: I1203 12:45:09.175078 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9a4cb938-bf89-4591-a358-3547074f4e97-config-data" (OuterVolumeSpecName: "config-data") pod "9a4cb938-bf89-4591-a358-3547074f4e97" (UID: "9a4cb938-bf89-4591-a358-3547074f4e97"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:45:09 crc kubenswrapper[5052]: I1203 12:45:09.242349 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/9a4cb938-bf89-4591-a358-3547074f4e97-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:45:09 crc kubenswrapper[5052]: I1203 12:45:09.242737 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9a4cb938-bf89-4591-a358-3547074f4e97-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:45:09 crc kubenswrapper[5052]: I1203 12:45:09.242751 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-2vt44\" (UniqueName: \"kubernetes.io/projected/9a4cb938-bf89-4591-a358-3547074f4e97-kube-api-access-2vt44\") on node \"crc\" DevicePath \"\"" Dec 03 12:45:09 crc kubenswrapper[5052]: I1203 12:45:09.244863 5052 generic.go:334] "Generic (PLEG): container finished" podID="047b4ef7-6147-49da-b4c8-382319e7f83a" containerID="722c7be4df16aafcde67740e1e015e377d9008419180cd45892921e7e51a19ef" exitCode=0 Dec 03 12:45:09 crc kubenswrapper[5052]: I1203 12:45:09.244974 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"047b4ef7-6147-49da-b4c8-382319e7f83a","Type":"ContainerDied","Data":"722c7be4df16aafcde67740e1e015e377d9008419180cd45892921e7e51a19ef"} Dec 03 12:45:09 crc kubenswrapper[5052]: I1203 12:45:09.245021 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"047b4ef7-6147-49da-b4c8-382319e7f83a","Type":"ContainerDied","Data":"87d968842d0f87896c822b6ced40469c04622652bd585174203619aac3818328"} Dec 03 12:45:09 crc kubenswrapper[5052]: I1203 12:45:09.245036 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="87d968842d0f87896c822b6ced40469c04622652bd585174203619aac3818328" Dec 03 12:45:09 crc kubenswrapper[5052]: I1203 12:45:09.248348 5052 generic.go:334] "Generic (PLEG): container finished" podID="9a4cb938-bf89-4591-a358-3547074f4e97" containerID="a76d27178065bb84ace8f9e4dd1a2c7b10629af1df1c9f257684f389d4aca5c3" exitCode=0 Dec 03 12:45:09 crc kubenswrapper[5052]: I1203 12:45:09.248478 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:45:09 crc kubenswrapper[5052]: I1203 12:45:09.248502 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"9a4cb938-bf89-4591-a358-3547074f4e97","Type":"ContainerDied","Data":"a76d27178065bb84ace8f9e4dd1a2c7b10629af1df1c9f257684f389d4aca5c3"} Dec 03 12:45:09 crc kubenswrapper[5052]: I1203 12:45:09.248576 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"9a4cb938-bf89-4591-a358-3547074f4e97","Type":"ContainerDied","Data":"5196b2768cacc0eb8223f8ca3c0a7b1894f90ef80461961ec4cb2b07221e51f9"} Dec 03 12:45:09 crc kubenswrapper[5052]: I1203 12:45:09.248605 5052 scope.go:117] "RemoveContainer" containerID="a76d27178065bb84ace8f9e4dd1a2c7b10629af1df1c9f257684f389d4aca5c3" Dec 03 12:45:09 crc kubenswrapper[5052]: I1203 12:45:09.256351 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:45:09 crc kubenswrapper[5052]: I1203 12:45:09.290500 5052 scope.go:117] "RemoveContainer" containerID="a76d27178065bb84ace8f9e4dd1a2c7b10629af1df1c9f257684f389d4aca5c3" Dec 03 12:45:09 crc kubenswrapper[5052]: E1203 12:45:09.308577 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a76d27178065bb84ace8f9e4dd1a2c7b10629af1df1c9f257684f389d4aca5c3\": container with ID starting with a76d27178065bb84ace8f9e4dd1a2c7b10629af1df1c9f257684f389d4aca5c3 not found: ID does not exist" containerID="a76d27178065bb84ace8f9e4dd1a2c7b10629af1df1c9f257684f389d4aca5c3" Dec 03 12:45:09 crc kubenswrapper[5052]: I1203 12:45:09.308647 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a76d27178065bb84ace8f9e4dd1a2c7b10629af1df1c9f257684f389d4aca5c3"} err="failed to get container status \"a76d27178065bb84ace8f9e4dd1a2c7b10629af1df1c9f257684f389d4aca5c3\": rpc error: code = NotFound desc = could not find container \"a76d27178065bb84ace8f9e4dd1a2c7b10629af1df1c9f257684f389d4aca5c3\": container with ID starting with a76d27178065bb84ace8f9e4dd1a2c7b10629af1df1c9f257684f389d4aca5c3 not found: ID does not exist" Dec 03 12:45:09 crc kubenswrapper[5052]: I1203 12:45:09.314079 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:45:09 crc kubenswrapper[5052]: I1203 12:45:09.328936 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:45:09 crc kubenswrapper[5052]: I1203 12:45:09.341848 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:45:09 crc kubenswrapper[5052]: E1203 12:45:09.342537 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="047b4ef7-6147-49da-b4c8-382319e7f83a" containerName="nova-metadata-log" Dec 03 12:45:09 crc kubenswrapper[5052]: I1203 12:45:09.342557 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="047b4ef7-6147-49da-b4c8-382319e7f83a" containerName="nova-metadata-log" Dec 03 12:45:09 crc kubenswrapper[5052]: E1203 12:45:09.342577 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="9a4cb938-bf89-4591-a358-3547074f4e97" containerName="nova-scheduler-scheduler" Dec 03 12:45:09 crc kubenswrapper[5052]: I1203 12:45:09.342583 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="9a4cb938-bf89-4591-a358-3547074f4e97" containerName="nova-scheduler-scheduler" Dec 03 12:45:09 crc kubenswrapper[5052]: E1203 12:45:09.342604 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="047b4ef7-6147-49da-b4c8-382319e7f83a" containerName="nova-metadata-metadata" Dec 03 12:45:09 crc kubenswrapper[5052]: I1203 12:45:09.342610 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="047b4ef7-6147-49da-b4c8-382319e7f83a" containerName="nova-metadata-metadata" Dec 03 12:45:09 crc kubenswrapper[5052]: I1203 12:45:09.342800 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="047b4ef7-6147-49da-b4c8-382319e7f83a" containerName="nova-metadata-log" Dec 03 12:45:09 crc kubenswrapper[5052]: I1203 12:45:09.342812 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="9a4cb938-bf89-4591-a358-3547074f4e97" containerName="nova-scheduler-scheduler" Dec 03 12:45:09 crc kubenswrapper[5052]: I1203 12:45:09.342831 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="047b4ef7-6147-49da-b4c8-382319e7f83a" containerName="nova-metadata-metadata" Dec 03 12:45:09 crc kubenswrapper[5052]: I1203 12:45:09.343638 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:45:09 crc kubenswrapper[5052]: I1203 12:45:09.344561 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/047b4ef7-6147-49da-b4c8-382319e7f83a-nova-metadata-tls-certs\") pod \"047b4ef7-6147-49da-b4c8-382319e7f83a\" (UID: \"047b4ef7-6147-49da-b4c8-382319e7f83a\") " Dec 03 12:45:09 crc kubenswrapper[5052]: I1203 12:45:09.344711 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sx5px\" (UniqueName: \"kubernetes.io/projected/047b4ef7-6147-49da-b4c8-382319e7f83a-kube-api-access-sx5px\") pod \"047b4ef7-6147-49da-b4c8-382319e7f83a\" (UID: \"047b4ef7-6147-49da-b4c8-382319e7f83a\") " Dec 03 12:45:09 crc kubenswrapper[5052]: I1203 12:45:09.344774 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/047b4ef7-6147-49da-b4c8-382319e7f83a-config-data\") pod \"047b4ef7-6147-49da-b4c8-382319e7f83a\" (UID: \"047b4ef7-6147-49da-b4c8-382319e7f83a\") " Dec 03 12:45:09 crc kubenswrapper[5052]: I1203 12:45:09.344803 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/047b4ef7-6147-49da-b4c8-382319e7f83a-logs\") pod \"047b4ef7-6147-49da-b4c8-382319e7f83a\" (UID: \"047b4ef7-6147-49da-b4c8-382319e7f83a\") " Dec 03 12:45:09 crc kubenswrapper[5052]: I1203 12:45:09.344833 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/047b4ef7-6147-49da-b4c8-382319e7f83a-combined-ca-bundle\") pod \"047b4ef7-6147-49da-b4c8-382319e7f83a\" (UID: \"047b4ef7-6147-49da-b4c8-382319e7f83a\") " Dec 03 12:45:09 crc kubenswrapper[5052]: I1203 12:45:09.345702 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-scheduler-config-data" Dec 03 12:45:09 crc kubenswrapper[5052]: I1203 12:45:09.346040 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/047b4ef7-6147-49da-b4c8-382319e7f83a-logs" (OuterVolumeSpecName: "logs") pod "047b4ef7-6147-49da-b4c8-382319e7f83a" (UID: "047b4ef7-6147-49da-b4c8-382319e7f83a"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:45:09 crc kubenswrapper[5052]: I1203 12:45:09.351861 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/047b4ef7-6147-49da-b4c8-382319e7f83a-kube-api-access-sx5px" (OuterVolumeSpecName: "kube-api-access-sx5px") pod "047b4ef7-6147-49da-b4c8-382319e7f83a" (UID: "047b4ef7-6147-49da-b4c8-382319e7f83a"). InnerVolumeSpecName "kube-api-access-sx5px". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:45:09 crc kubenswrapper[5052]: I1203 12:45:09.360592 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:45:09 crc kubenswrapper[5052]: I1203 12:45:09.386276 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/047b4ef7-6147-49da-b4c8-382319e7f83a-config-data" (OuterVolumeSpecName: "config-data") pod "047b4ef7-6147-49da-b4c8-382319e7f83a" (UID: "047b4ef7-6147-49da-b4c8-382319e7f83a"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:45:09 crc kubenswrapper[5052]: I1203 12:45:09.386877 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/047b4ef7-6147-49da-b4c8-382319e7f83a-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "047b4ef7-6147-49da-b4c8-382319e7f83a" (UID: "047b4ef7-6147-49da-b4c8-382319e7f83a"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:45:09 crc kubenswrapper[5052]: I1203 12:45:09.397644 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/047b4ef7-6147-49da-b4c8-382319e7f83a-nova-metadata-tls-certs" (OuterVolumeSpecName: "nova-metadata-tls-certs") pod "047b4ef7-6147-49da-b4c8-382319e7f83a" (UID: "047b4ef7-6147-49da-b4c8-382319e7f83a"). InnerVolumeSpecName "nova-metadata-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:45:09 crc kubenswrapper[5052]: I1203 12:45:09.448261 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/dfda1fd2-f923-47bd-bf43-8a1be3f79f09-combined-ca-bundle\") pod \"nova-scheduler-0\" (UID: \"dfda1fd2-f923-47bd-bf43-8a1be3f79f09\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:45:09 crc kubenswrapper[5052]: I1203 12:45:09.448603 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ph884\" (UniqueName: \"kubernetes.io/projected/dfda1fd2-f923-47bd-bf43-8a1be3f79f09-kube-api-access-ph884\") pod \"nova-scheduler-0\" (UID: \"dfda1fd2-f923-47bd-bf43-8a1be3f79f09\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:45:09 crc kubenswrapper[5052]: I1203 12:45:09.448691 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/dfda1fd2-f923-47bd-bf43-8a1be3f79f09-config-data\") pod \"nova-scheduler-0\" (UID: \"dfda1fd2-f923-47bd-bf43-8a1be3f79f09\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:45:09 crc kubenswrapper[5052]: I1203 12:45:09.448799 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-sx5px\" (UniqueName: \"kubernetes.io/projected/047b4ef7-6147-49da-b4c8-382319e7f83a-kube-api-access-sx5px\") on node \"crc\" DevicePath \"\"" Dec 03 12:45:09 crc kubenswrapper[5052]: I1203 12:45:09.448821 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/047b4ef7-6147-49da-b4c8-382319e7f83a-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:45:09 crc kubenswrapper[5052]: I1203 12:45:09.448832 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/047b4ef7-6147-49da-b4c8-382319e7f83a-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:45:09 crc kubenswrapper[5052]: I1203 12:45:09.448853 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/047b4ef7-6147-49da-b4c8-382319e7f83a-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:45:09 crc kubenswrapper[5052]: I1203 12:45:09.448927 5052 reconciler_common.go:293] "Volume detached for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/047b4ef7-6147-49da-b4c8-382319e7f83a-nova-metadata-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:45:09 crc kubenswrapper[5052]: I1203 12:45:09.550691 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-ph884\" (UniqueName: \"kubernetes.io/projected/dfda1fd2-f923-47bd-bf43-8a1be3f79f09-kube-api-access-ph884\") pod \"nova-scheduler-0\" (UID: \"dfda1fd2-f923-47bd-bf43-8a1be3f79f09\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:45:09 crc kubenswrapper[5052]: I1203 12:45:09.550795 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/dfda1fd2-f923-47bd-bf43-8a1be3f79f09-config-data\") pod \"nova-scheduler-0\" (UID: \"dfda1fd2-f923-47bd-bf43-8a1be3f79f09\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:45:09 crc kubenswrapper[5052]: I1203 12:45:09.550922 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/dfda1fd2-f923-47bd-bf43-8a1be3f79f09-combined-ca-bundle\") pod \"nova-scheduler-0\" (UID: \"dfda1fd2-f923-47bd-bf43-8a1be3f79f09\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:45:09 crc kubenswrapper[5052]: I1203 12:45:09.555556 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/dfda1fd2-f923-47bd-bf43-8a1be3f79f09-combined-ca-bundle\") pod \"nova-scheduler-0\" (UID: \"dfda1fd2-f923-47bd-bf43-8a1be3f79f09\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:45:09 crc kubenswrapper[5052]: I1203 12:45:09.556166 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/dfda1fd2-f923-47bd-bf43-8a1be3f79f09-config-data\") pod \"nova-scheduler-0\" (UID: \"dfda1fd2-f923-47bd-bf43-8a1be3f79f09\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:45:09 crc kubenswrapper[5052]: I1203 12:45:09.568228 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-ph884\" (UniqueName: \"kubernetes.io/projected/dfda1fd2-f923-47bd-bf43-8a1be3f79f09-kube-api-access-ph884\") pod \"nova-scheduler-0\" (UID: \"dfda1fd2-f923-47bd-bf43-8a1be3f79f09\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:45:09 crc kubenswrapper[5052]: I1203 12:45:09.671618 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:45:09 crc kubenswrapper[5052]: I1203 12:45:09.679588 5052 scope.go:117] "RemoveContainer" containerID="2aed1c760a326f3d008799e902b1093313df3764a57c879f0b3686410127d37d" Dec 03 12:45:09 crc kubenswrapper[5052]: E1203 12:45:09.680112 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 12:45:09 crc kubenswrapper[5052]: I1203 12:45:09.685464 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9a4cb938-bf89-4591-a358-3547074f4e97" path="/var/lib/kubelet/pods/9a4cb938-bf89-4591-a358-3547074f4e97/volumes" Dec 03 12:45:10 crc kubenswrapper[5052]: I1203 12:45:10.147800 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:45:10 crc kubenswrapper[5052]: W1203 12:45:10.149871 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddfda1fd2_f923_47bd_bf43_8a1be3f79f09.slice/crio-cfdd01f2934613c9aad88bd35e6649ab3d537f03cad96217a4429a953ea0c318 WatchSource:0}: Error finding container cfdd01f2934613c9aad88bd35e6649ab3d537f03cad96217a4429a953ea0c318: Status 404 returned error can't find the container with id cfdd01f2934613c9aad88bd35e6649ab3d537f03cad96217a4429a953ea0c318 Dec 03 12:45:10 crc kubenswrapper[5052]: I1203 12:45:10.292467 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"dfda1fd2-f923-47bd-bf43-8a1be3f79f09","Type":"ContainerStarted","Data":"cfdd01f2934613c9aad88bd35e6649ab3d537f03cad96217a4429a953ea0c318"} Dec 03 12:45:10 crc kubenswrapper[5052]: I1203 12:45:10.292498 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:45:10 crc kubenswrapper[5052]: I1203 12:45:10.333649 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:45:10 crc kubenswrapper[5052]: I1203 12:45:10.348782 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:45:10 crc kubenswrapper[5052]: I1203 12:45:10.359692 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:45:10 crc kubenswrapper[5052]: I1203 12:45:10.362733 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:45:10 crc kubenswrapper[5052]: I1203 12:45:10.366424 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-nova-metadata-internal-svc" Dec 03 12:45:10 crc kubenswrapper[5052]: I1203 12:45:10.366764 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-metadata-config-data" Dec 03 12:45:10 crc kubenswrapper[5052]: I1203 12:45:10.368935 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:45:10 crc kubenswrapper[5052]: I1203 12:45:10.471036 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/9af09a5b-789c-4b83-b899-9d1b3f0d8603-nova-metadata-tls-certs\") pod \"nova-metadata-0\" (UID: \"9af09a5b-789c-4b83-b899-9d1b3f0d8603\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:45:10 crc kubenswrapper[5052]: I1203 12:45:10.471153 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dhpjf\" (UniqueName: \"kubernetes.io/projected/9af09a5b-789c-4b83-b899-9d1b3f0d8603-kube-api-access-dhpjf\") pod \"nova-metadata-0\" (UID: \"9af09a5b-789c-4b83-b899-9d1b3f0d8603\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:45:10 crc kubenswrapper[5052]: I1203 12:45:10.471187 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/9af09a5b-789c-4b83-b899-9d1b3f0d8603-config-data\") pod \"nova-metadata-0\" (UID: \"9af09a5b-789c-4b83-b899-9d1b3f0d8603\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:45:10 crc kubenswrapper[5052]: I1203 12:45:10.471211 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9af09a5b-789c-4b83-b899-9d1b3f0d8603-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"9af09a5b-789c-4b83-b899-9d1b3f0d8603\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:45:10 crc kubenswrapper[5052]: I1203 12:45:10.471340 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/9af09a5b-789c-4b83-b899-9d1b3f0d8603-logs\") pod \"nova-metadata-0\" (UID: \"9af09a5b-789c-4b83-b899-9d1b3f0d8603\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:45:10 crc kubenswrapper[5052]: I1203 12:45:10.573537 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/9af09a5b-789c-4b83-b899-9d1b3f0d8603-nova-metadata-tls-certs\") pod \"nova-metadata-0\" (UID: \"9af09a5b-789c-4b83-b899-9d1b3f0d8603\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:45:10 crc kubenswrapper[5052]: I1203 12:45:10.573652 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-dhpjf\" (UniqueName: \"kubernetes.io/projected/9af09a5b-789c-4b83-b899-9d1b3f0d8603-kube-api-access-dhpjf\") pod \"nova-metadata-0\" (UID: \"9af09a5b-789c-4b83-b899-9d1b3f0d8603\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:45:10 crc kubenswrapper[5052]: I1203 12:45:10.573688 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/9af09a5b-789c-4b83-b899-9d1b3f0d8603-config-data\") pod \"nova-metadata-0\" (UID: \"9af09a5b-789c-4b83-b899-9d1b3f0d8603\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:45:10 crc kubenswrapper[5052]: I1203 12:45:10.573712 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9af09a5b-789c-4b83-b899-9d1b3f0d8603-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"9af09a5b-789c-4b83-b899-9d1b3f0d8603\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:45:10 crc kubenswrapper[5052]: I1203 12:45:10.573751 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/9af09a5b-789c-4b83-b899-9d1b3f0d8603-logs\") pod \"nova-metadata-0\" (UID: \"9af09a5b-789c-4b83-b899-9d1b3f0d8603\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:45:10 crc kubenswrapper[5052]: I1203 12:45:10.574296 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/9af09a5b-789c-4b83-b899-9d1b3f0d8603-logs\") pod \"nova-metadata-0\" (UID: \"9af09a5b-789c-4b83-b899-9d1b3f0d8603\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:45:10 crc kubenswrapper[5052]: I1203 12:45:10.925262 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9af09a5b-789c-4b83-b899-9d1b3f0d8603-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"9af09a5b-789c-4b83-b899-9d1b3f0d8603\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:45:10 crc kubenswrapper[5052]: I1203 12:45:10.925617 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/9af09a5b-789c-4b83-b899-9d1b3f0d8603-config-data\") pod \"nova-metadata-0\" (UID: \"9af09a5b-789c-4b83-b899-9d1b3f0d8603\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:45:10 crc kubenswrapper[5052]: I1203 12:45:10.925865 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/9af09a5b-789c-4b83-b899-9d1b3f0d8603-nova-metadata-tls-certs\") pod \"nova-metadata-0\" (UID: \"9af09a5b-789c-4b83-b899-9d1b3f0d8603\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:45:10 crc kubenswrapper[5052]: I1203 12:45:10.927080 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-dhpjf\" (UniqueName: \"kubernetes.io/projected/9af09a5b-789c-4b83-b899-9d1b3f0d8603-kube-api-access-dhpjf\") pod \"nova-metadata-0\" (UID: \"9af09a5b-789c-4b83-b899-9d1b3f0d8603\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:45:10 crc kubenswrapper[5052]: I1203 12:45:10.995107 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:45:11 crc kubenswrapper[5052]: I1203 12:45:11.305626 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"dfda1fd2-f923-47bd-bf43-8a1be3f79f09","Type":"ContainerStarted","Data":"67e80f3b2185e807d12e6cb73fcdb3e6261d02a42c22e32ff2b8b1ff0732f99b"} Dec 03 12:45:11 crc kubenswrapper[5052]: I1203 12:45:11.460473 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-scheduler-0" podStartSLOduration=2.4604548250000002 podStartE2EDuration="2.460454825s" podCreationTimestamp="2025-12-03 12:45:09 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:45:11.333477593 +0000 UTC m=+5101.921719377" watchObservedRunningTime="2025-12-03 12:45:11.460454825 +0000 UTC m=+5102.048696589" Dec 03 12:45:11 crc kubenswrapper[5052]: I1203 12:45:11.461825 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:45:11 crc kubenswrapper[5052]: I1203 12:45:11.688576 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="047b4ef7-6147-49da-b4c8-382319e7f83a" path="/var/lib/kubelet/pods/047b4ef7-6147-49da-b4c8-382319e7f83a/volumes" Dec 03 12:45:12 crc kubenswrapper[5052]: I1203 12:45:12.318412 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"9af09a5b-789c-4b83-b899-9d1b3f0d8603","Type":"ContainerStarted","Data":"9ba6898618d6ecc3b57802d08d86da02201fb0d67fcd7288c3414ce61b3233ed"} Dec 03 12:45:12 crc kubenswrapper[5052]: I1203 12:45:12.318453 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"9af09a5b-789c-4b83-b899-9d1b3f0d8603","Type":"ContainerStarted","Data":"deb3641e5c7641ec59de2756335b3b8af86a98493025f1061ab9587775677d7d"} Dec 03 12:45:12 crc kubenswrapper[5052]: I1203 12:45:12.318464 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"9af09a5b-789c-4b83-b899-9d1b3f0d8603","Type":"ContainerStarted","Data":"7069ccbb6a2b1508b51ae009958cc0840dc2f497188ba2d99f0a2da359df1b40"} Dec 03 12:45:12 crc kubenswrapper[5052]: I1203 12:45:12.347140 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-metadata-0" podStartSLOduration=2.347122492 podStartE2EDuration="2.347122492s" podCreationTimestamp="2025-12-03 12:45:10 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:45:12.340856643 +0000 UTC m=+5102.929098417" watchObservedRunningTime="2025-12-03 12:45:12.347122492 +0000 UTC m=+5102.935364256" Dec 03 12:45:14 crc kubenswrapper[5052]: I1203 12:45:14.672353 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:45:15 crc kubenswrapper[5052]: I1203 12:45:15.995958 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:45:15 crc kubenswrapper[5052]: I1203 12:45:15.996348 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:45:16 crc kubenswrapper[5052]: I1203 12:45:16.605081 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:45:16 crc kubenswrapper[5052]: I1203 12:45:16.605152 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:45:17 crc kubenswrapper[5052]: I1203 12:45:17.686831 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-api-0" podUID="a6e95405-86bc-4eeb-872b-2c4fdec53bfc" containerName="nova-api-api" probeResult="failure" output="Get \"http://10.217.1.234:8774/\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Dec 03 12:45:17 crc kubenswrapper[5052]: I1203 12:45:17.686849 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-api-0" podUID="a6e95405-86bc-4eeb-872b-2c4fdec53bfc" containerName="nova-api-log" probeResult="failure" output="Get \"http://10.217.1.234:8774/\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Dec 03 12:45:19 crc kubenswrapper[5052]: I1203 12:45:19.684394 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:45:19 crc kubenswrapper[5052]: I1203 12:45:19.700141 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:45:20 crc kubenswrapper[5052]: I1203 12:45:20.435365 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:45:20 crc kubenswrapper[5052]: I1203 12:45:20.995719 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:45:20 crc kubenswrapper[5052]: I1203 12:45:20.996083 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:45:22 crc kubenswrapper[5052]: I1203 12:45:22.006674 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-metadata-0" podUID="9af09a5b-789c-4b83-b899-9d1b3f0d8603" containerName="nova-metadata-log" probeResult="failure" output="Get \"https://10.217.1.236:8775/\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Dec 03 12:45:22 crc kubenswrapper[5052]: I1203 12:45:22.006718 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-metadata-0" podUID="9af09a5b-789c-4b83-b899-9d1b3f0d8603" containerName="nova-metadata-metadata" probeResult="failure" output="Get \"https://10.217.1.236:8775/\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Dec 03 12:45:23 crc kubenswrapper[5052]: I1203 12:45:23.325151 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:45:23 crc kubenswrapper[5052]: I1203 12:45:23.673472 5052 scope.go:117] "RemoveContainer" containerID="2aed1c760a326f3d008799e902b1093313df3764a57c879f0b3686410127d37d" Dec 03 12:45:23 crc kubenswrapper[5052]: E1203 12:45:23.673799 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 12:45:26 crc kubenswrapper[5052]: I1203 12:45:26.611305 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:45:26 crc kubenswrapper[5052]: I1203 12:45:26.611716 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:45:26 crc kubenswrapper[5052]: I1203 12:45:26.612376 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:45:26 crc kubenswrapper[5052]: I1203 12:45:26.612431 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:45:26 crc kubenswrapper[5052]: I1203 12:45:26.617661 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:45:26 crc kubenswrapper[5052]: I1203 12:45:26.618236 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:45:29 crc kubenswrapper[5052]: I1203 12:45:29.244970 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:45:29 crc kubenswrapper[5052]: I1203 12:45:29.245752 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="e08a0c0d-36f0-49f1-9df8-f0e616d1d99d" containerName="ceilometer-central-agent" containerID="cri-o://d957f6d4ec3ecf5552e7abc256da98e11212433b834afa67d861c95a81528ec6" gracePeriod=30 Dec 03 12:45:29 crc kubenswrapper[5052]: I1203 12:45:29.245841 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="e08a0c0d-36f0-49f1-9df8-f0e616d1d99d" containerName="proxy-httpd" containerID="cri-o://9a2e22eb61df2421257c5aa00a47a7949877fdb593a1d8c3d3099b33f82916e0" gracePeriod=30 Dec 03 12:45:29 crc kubenswrapper[5052]: I1203 12:45:29.245841 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="e08a0c0d-36f0-49f1-9df8-f0e616d1d99d" containerName="sg-core" containerID="cri-o://6b9dc66f0245c06a5ff620f79133997e00f75f954583172251c9c6a8120066db" gracePeriod=30 Dec 03 12:45:29 crc kubenswrapper[5052]: I1203 12:45:29.245968 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="e08a0c0d-36f0-49f1-9df8-f0e616d1d99d" containerName="ceilometer-notification-agent" containerID="cri-o://6e4c007c3a17262d2fde8b64aa6bf056a29b57cd2ef4649732b01c4b83ed1119" gracePeriod=30 Dec 03 12:45:29 crc kubenswrapper[5052]: I1203 12:45:29.489573 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:45:29 crc kubenswrapper[5052]: I1203 12:45:29.511575 5052 generic.go:334] "Generic (PLEG): container finished" podID="e08a0c0d-36f0-49f1-9df8-f0e616d1d99d" containerID="9a2e22eb61df2421257c5aa00a47a7949877fdb593a1d8c3d3099b33f82916e0" exitCode=0 Dec 03 12:45:29 crc kubenswrapper[5052]: I1203 12:45:29.511607 5052 generic.go:334] "Generic (PLEG): container finished" podID="e08a0c0d-36f0-49f1-9df8-f0e616d1d99d" containerID="6b9dc66f0245c06a5ff620f79133997e00f75f954583172251c9c6a8120066db" exitCode=2 Dec 03 12:45:29 crc kubenswrapper[5052]: I1203 12:45:29.511657 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"e08a0c0d-36f0-49f1-9df8-f0e616d1d99d","Type":"ContainerDied","Data":"9a2e22eb61df2421257c5aa00a47a7949877fdb593a1d8c3d3099b33f82916e0"} Dec 03 12:45:29 crc kubenswrapper[5052]: I1203 12:45:29.511699 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"e08a0c0d-36f0-49f1-9df8-f0e616d1d99d","Type":"ContainerDied","Data":"6b9dc66f0245c06a5ff620f79133997e00f75f954583172251c9c6a8120066db"} Dec 03 12:45:29 crc kubenswrapper[5052]: I1203 12:45:29.511782 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-api-0" podUID="a6e95405-86bc-4eeb-872b-2c4fdec53bfc" containerName="nova-api-log" containerID="cri-o://ad6be9bfabfc99a6b322850260fdffb57345a6b03420e364eb41589f6201bb82" gracePeriod=30 Dec 03 12:45:29 crc kubenswrapper[5052]: I1203 12:45:29.511847 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-api-0" podUID="a6e95405-86bc-4eeb-872b-2c4fdec53bfc" containerName="nova-api-api" containerID="cri-o://d576322fd69e059064fcc3a5cf54fc65c5516c5825fd163d50c7cc75e230f368" gracePeriod=30 Dec 03 12:45:29 crc kubenswrapper[5052]: I1203 12:45:29.796639 5052 scope.go:117] "RemoveContainer" containerID="15342f245f3b466f27835c2c30a7036b0d06092684492805d84db0575d5b165a" Dec 03 12:45:29 crc kubenswrapper[5052]: I1203 12:45:29.831728 5052 scope.go:117] "RemoveContainer" containerID="3f73a35540704d1c4ac272256521dc836a1d429fbe1cee0a507bfe3859a50b06" Dec 03 12:45:29 crc kubenswrapper[5052]: I1203 12:45:29.856016 5052 scope.go:117] "RemoveContainer" containerID="471a98ef0f6b12304a21a7203c5b57972e489a2e00b16c3bff98f0d169d8edf7" Dec 03 12:45:29 crc kubenswrapper[5052]: I1203 12:45:29.894712 5052 scope.go:117] "RemoveContainer" containerID="e918d411b918ed8ca93043d08217369893147e6405ff8425f54ff29adf217db7" Dec 03 12:45:29 crc kubenswrapper[5052]: I1203 12:45:29.938943 5052 scope.go:117] "RemoveContainer" containerID="87d55b80754e67e2615d7cea19e5d57d82b054e86c70bcb48fc87f5b7efa28d9" Dec 03 12:45:29 crc kubenswrapper[5052]: I1203 12:45:29.961059 5052 scope.go:117] "RemoveContainer" containerID="87e14300e8697a4e0c8594bb1dde5f610d618d2da77d5478a1bb33029c23a5e0" Dec 03 12:45:29 crc kubenswrapper[5052]: I1203 12:45:29.980502 5052 scope.go:117] "RemoveContainer" containerID="d09c826d0d22b153c916799836e2f1426871c707d4364227e89e6fd55c6f616d" Dec 03 12:45:30 crc kubenswrapper[5052]: I1203 12:45:30.002940 5052 scope.go:117] "RemoveContainer" containerID="db6fc581a8df4ec1884c71516e39b9792e494f950b83f3b0b693c01758dfa0f1" Dec 03 12:45:30 crc kubenswrapper[5052]: I1203 12:45:30.024056 5052 scope.go:117] "RemoveContainer" containerID="0c3d8b3ca8a9d7d4611f59d9c0bcfa83d8ea56d3df8caa14069f56e2580808ef" Dec 03 12:45:30 crc kubenswrapper[5052]: I1203 12:45:30.527919 5052 generic.go:334] "Generic (PLEG): container finished" podID="e08a0c0d-36f0-49f1-9df8-f0e616d1d99d" containerID="d957f6d4ec3ecf5552e7abc256da98e11212433b834afa67d861c95a81528ec6" exitCode=0 Dec 03 12:45:30 crc kubenswrapper[5052]: I1203 12:45:30.527976 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"e08a0c0d-36f0-49f1-9df8-f0e616d1d99d","Type":"ContainerDied","Data":"d957f6d4ec3ecf5552e7abc256da98e11212433b834afa67d861c95a81528ec6"} Dec 03 12:45:30 crc kubenswrapper[5052]: I1203 12:45:30.530785 5052 generic.go:334] "Generic (PLEG): container finished" podID="a6e95405-86bc-4eeb-872b-2c4fdec53bfc" containerID="ad6be9bfabfc99a6b322850260fdffb57345a6b03420e364eb41589f6201bb82" exitCode=143 Dec 03 12:45:30 crc kubenswrapper[5052]: I1203 12:45:30.530826 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"a6e95405-86bc-4eeb-872b-2c4fdec53bfc","Type":"ContainerDied","Data":"ad6be9bfabfc99a6b322850260fdffb57345a6b03420e364eb41589f6201bb82"} Dec 03 12:45:31 crc kubenswrapper[5052]: I1203 12:45:31.003401 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:45:31 crc kubenswrapper[5052]: I1203 12:45:31.004884 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:45:31 crc kubenswrapper[5052]: I1203 12:45:31.009076 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:45:31 crc kubenswrapper[5052]: I1203 12:45:31.544180 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:45:33 crc kubenswrapper[5052]: I1203 12:45:33.103044 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:45:33 crc kubenswrapper[5052]: I1203 12:45:33.224596 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a6e95405-86bc-4eeb-872b-2c4fdec53bfc-config-data\") pod \"a6e95405-86bc-4eeb-872b-2c4fdec53bfc\" (UID: \"a6e95405-86bc-4eeb-872b-2c4fdec53bfc\") " Dec 03 12:45:33 crc kubenswrapper[5052]: I1203 12:45:33.224806 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ktpcp\" (UniqueName: \"kubernetes.io/projected/a6e95405-86bc-4eeb-872b-2c4fdec53bfc-kube-api-access-ktpcp\") pod \"a6e95405-86bc-4eeb-872b-2c4fdec53bfc\" (UID: \"a6e95405-86bc-4eeb-872b-2c4fdec53bfc\") " Dec 03 12:45:33 crc kubenswrapper[5052]: I1203 12:45:33.224857 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/a6e95405-86bc-4eeb-872b-2c4fdec53bfc-logs\") pod \"a6e95405-86bc-4eeb-872b-2c4fdec53bfc\" (UID: \"a6e95405-86bc-4eeb-872b-2c4fdec53bfc\") " Dec 03 12:45:33 crc kubenswrapper[5052]: I1203 12:45:33.224896 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a6e95405-86bc-4eeb-872b-2c4fdec53bfc-combined-ca-bundle\") pod \"a6e95405-86bc-4eeb-872b-2c4fdec53bfc\" (UID: \"a6e95405-86bc-4eeb-872b-2c4fdec53bfc\") " Dec 03 12:45:33 crc kubenswrapper[5052]: I1203 12:45:33.225157 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a6e95405-86bc-4eeb-872b-2c4fdec53bfc-logs" (OuterVolumeSpecName: "logs") pod "a6e95405-86bc-4eeb-872b-2c4fdec53bfc" (UID: "a6e95405-86bc-4eeb-872b-2c4fdec53bfc"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:45:33 crc kubenswrapper[5052]: I1203 12:45:33.225312 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/a6e95405-86bc-4eeb-872b-2c4fdec53bfc-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:45:33 crc kubenswrapper[5052]: I1203 12:45:33.234420 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a6e95405-86bc-4eeb-872b-2c4fdec53bfc-kube-api-access-ktpcp" (OuterVolumeSpecName: "kube-api-access-ktpcp") pod "a6e95405-86bc-4eeb-872b-2c4fdec53bfc" (UID: "a6e95405-86bc-4eeb-872b-2c4fdec53bfc"). InnerVolumeSpecName "kube-api-access-ktpcp". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:45:33 crc kubenswrapper[5052]: I1203 12:45:33.252300 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a6e95405-86bc-4eeb-872b-2c4fdec53bfc-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "a6e95405-86bc-4eeb-872b-2c4fdec53bfc" (UID: "a6e95405-86bc-4eeb-872b-2c4fdec53bfc"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:45:33 crc kubenswrapper[5052]: I1203 12:45:33.260228 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a6e95405-86bc-4eeb-872b-2c4fdec53bfc-config-data" (OuterVolumeSpecName: "config-data") pod "a6e95405-86bc-4eeb-872b-2c4fdec53bfc" (UID: "a6e95405-86bc-4eeb-872b-2c4fdec53bfc"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:45:33 crc kubenswrapper[5052]: I1203 12:45:33.327614 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a6e95405-86bc-4eeb-872b-2c4fdec53bfc-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:45:33 crc kubenswrapper[5052]: I1203 12:45:33.327663 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-ktpcp\" (UniqueName: \"kubernetes.io/projected/a6e95405-86bc-4eeb-872b-2c4fdec53bfc-kube-api-access-ktpcp\") on node \"crc\" DevicePath \"\"" Dec 03 12:45:33 crc kubenswrapper[5052]: I1203 12:45:33.327680 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a6e95405-86bc-4eeb-872b-2c4fdec53bfc-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:45:33 crc kubenswrapper[5052]: I1203 12:45:33.576619 5052 generic.go:334] "Generic (PLEG): container finished" podID="e08a0c0d-36f0-49f1-9df8-f0e616d1d99d" containerID="6e4c007c3a17262d2fde8b64aa6bf056a29b57cd2ef4649732b01c4b83ed1119" exitCode=0 Dec 03 12:45:33 crc kubenswrapper[5052]: I1203 12:45:33.576730 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"e08a0c0d-36f0-49f1-9df8-f0e616d1d99d","Type":"ContainerDied","Data":"6e4c007c3a17262d2fde8b64aa6bf056a29b57cd2ef4649732b01c4b83ed1119"} Dec 03 12:45:33 crc kubenswrapper[5052]: I1203 12:45:33.581239 5052 generic.go:334] "Generic (PLEG): container finished" podID="a6e95405-86bc-4eeb-872b-2c4fdec53bfc" containerID="d576322fd69e059064fcc3a5cf54fc65c5516c5825fd163d50c7cc75e230f368" exitCode=0 Dec 03 12:45:33 crc kubenswrapper[5052]: I1203 12:45:33.581564 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:45:33 crc kubenswrapper[5052]: I1203 12:45:33.582099 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"a6e95405-86bc-4eeb-872b-2c4fdec53bfc","Type":"ContainerDied","Data":"d576322fd69e059064fcc3a5cf54fc65c5516c5825fd163d50c7cc75e230f368"} Dec 03 12:45:33 crc kubenswrapper[5052]: I1203 12:45:33.582163 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"a6e95405-86bc-4eeb-872b-2c4fdec53bfc","Type":"ContainerDied","Data":"9822fd49ef468126c30884ad0173c1959e4dea1e209bb3bf1736a25863b9cb7c"} Dec 03 12:45:33 crc kubenswrapper[5052]: I1203 12:45:33.582183 5052 scope.go:117] "RemoveContainer" containerID="d576322fd69e059064fcc3a5cf54fc65c5516c5825fd163d50c7cc75e230f368" Dec 03 12:45:33 crc kubenswrapper[5052]: I1203 12:45:33.606877 5052 scope.go:117] "RemoveContainer" containerID="ad6be9bfabfc99a6b322850260fdffb57345a6b03420e364eb41589f6201bb82" Dec 03 12:45:33 crc kubenswrapper[5052]: I1203 12:45:33.624435 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:45:33 crc kubenswrapper[5052]: I1203 12:45:33.636714 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:45:33 crc kubenswrapper[5052]: I1203 12:45:33.664216 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:45:33 crc kubenswrapper[5052]: E1203 12:45:33.664702 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="a6e95405-86bc-4eeb-872b-2c4fdec53bfc" containerName="nova-api-log" Dec 03 12:45:33 crc kubenswrapper[5052]: I1203 12:45:33.664718 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="a6e95405-86bc-4eeb-872b-2c4fdec53bfc" containerName="nova-api-log" Dec 03 12:45:33 crc kubenswrapper[5052]: E1203 12:45:33.664752 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="a6e95405-86bc-4eeb-872b-2c4fdec53bfc" containerName="nova-api-api" Dec 03 12:45:33 crc kubenswrapper[5052]: I1203 12:45:33.664758 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="a6e95405-86bc-4eeb-872b-2c4fdec53bfc" containerName="nova-api-api" Dec 03 12:45:33 crc kubenswrapper[5052]: I1203 12:45:33.664937 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="a6e95405-86bc-4eeb-872b-2c4fdec53bfc" containerName="nova-api-log" Dec 03 12:45:33 crc kubenswrapper[5052]: I1203 12:45:33.664951 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="a6e95405-86bc-4eeb-872b-2c4fdec53bfc" containerName="nova-api-api" Dec 03 12:45:33 crc kubenswrapper[5052]: I1203 12:45:33.666024 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:45:33 crc kubenswrapper[5052]: I1203 12:45:33.672894 5052 scope.go:117] "RemoveContainer" containerID="d576322fd69e059064fcc3a5cf54fc65c5516c5825fd163d50c7cc75e230f368" Dec 03 12:45:33 crc kubenswrapper[5052]: I1203 12:45:33.673320 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-api-config-data" Dec 03 12:45:33 crc kubenswrapper[5052]: I1203 12:45:33.673958 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-nova-public-svc" Dec 03 12:45:33 crc kubenswrapper[5052]: I1203 12:45:33.674135 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-nova-internal-svc" Dec 03 12:45:33 crc kubenswrapper[5052]: E1203 12:45:33.675652 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d576322fd69e059064fcc3a5cf54fc65c5516c5825fd163d50c7cc75e230f368\": container with ID starting with d576322fd69e059064fcc3a5cf54fc65c5516c5825fd163d50c7cc75e230f368 not found: ID does not exist" containerID="d576322fd69e059064fcc3a5cf54fc65c5516c5825fd163d50c7cc75e230f368" Dec 03 12:45:33 crc kubenswrapper[5052]: I1203 12:45:33.675696 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d576322fd69e059064fcc3a5cf54fc65c5516c5825fd163d50c7cc75e230f368"} err="failed to get container status \"d576322fd69e059064fcc3a5cf54fc65c5516c5825fd163d50c7cc75e230f368\": rpc error: code = NotFound desc = could not find container \"d576322fd69e059064fcc3a5cf54fc65c5516c5825fd163d50c7cc75e230f368\": container with ID starting with d576322fd69e059064fcc3a5cf54fc65c5516c5825fd163d50c7cc75e230f368 not found: ID does not exist" Dec 03 12:45:33 crc kubenswrapper[5052]: I1203 12:45:33.675719 5052 scope.go:117] "RemoveContainer" containerID="ad6be9bfabfc99a6b322850260fdffb57345a6b03420e364eb41589f6201bb82" Dec 03 12:45:33 crc kubenswrapper[5052]: E1203 12:45:33.683580 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ad6be9bfabfc99a6b322850260fdffb57345a6b03420e364eb41589f6201bb82\": container with ID starting with ad6be9bfabfc99a6b322850260fdffb57345a6b03420e364eb41589f6201bb82 not found: ID does not exist" containerID="ad6be9bfabfc99a6b322850260fdffb57345a6b03420e364eb41589f6201bb82" Dec 03 12:45:33 crc kubenswrapper[5052]: I1203 12:45:33.683639 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ad6be9bfabfc99a6b322850260fdffb57345a6b03420e364eb41589f6201bb82"} err="failed to get container status \"ad6be9bfabfc99a6b322850260fdffb57345a6b03420e364eb41589f6201bb82\": rpc error: code = NotFound desc = could not find container \"ad6be9bfabfc99a6b322850260fdffb57345a6b03420e364eb41589f6201bb82\": container with ID starting with ad6be9bfabfc99a6b322850260fdffb57345a6b03420e364eb41589f6201bb82 not found: ID does not exist" Dec 03 12:45:33 crc kubenswrapper[5052]: I1203 12:45:33.698614 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a6e95405-86bc-4eeb-872b-2c4fdec53bfc" path="/var/lib/kubelet/pods/a6e95405-86bc-4eeb-872b-2c4fdec53bfc/volumes" Dec 03 12:45:33 crc kubenswrapper[5052]: I1203 12:45:33.699481 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:45:33 crc kubenswrapper[5052]: I1203 12:45:33.736173 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/fb6c11ac-1b0e-4e63-8988-256b2504ebc7-public-tls-certs\") pod \"nova-api-0\" (UID: \"fb6c11ac-1b0e-4e63-8988-256b2504ebc7\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:45:33 crc kubenswrapper[5052]: I1203 12:45:33.736227 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/fb6c11ac-1b0e-4e63-8988-256b2504ebc7-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"fb6c11ac-1b0e-4e63-8988-256b2504ebc7\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:45:33 crc kubenswrapper[5052]: I1203 12:45:33.736260 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mp72c\" (UniqueName: \"kubernetes.io/projected/fb6c11ac-1b0e-4e63-8988-256b2504ebc7-kube-api-access-mp72c\") pod \"nova-api-0\" (UID: \"fb6c11ac-1b0e-4e63-8988-256b2504ebc7\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:45:33 crc kubenswrapper[5052]: I1203 12:45:33.736289 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/fb6c11ac-1b0e-4e63-8988-256b2504ebc7-config-data\") pod \"nova-api-0\" (UID: \"fb6c11ac-1b0e-4e63-8988-256b2504ebc7\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:45:33 crc kubenswrapper[5052]: I1203 12:45:33.736372 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/fb6c11ac-1b0e-4e63-8988-256b2504ebc7-logs\") pod \"nova-api-0\" (UID: \"fb6c11ac-1b0e-4e63-8988-256b2504ebc7\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:45:33 crc kubenswrapper[5052]: I1203 12:45:33.736392 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/fb6c11ac-1b0e-4e63-8988-256b2504ebc7-internal-tls-certs\") pod \"nova-api-0\" (UID: \"fb6c11ac-1b0e-4e63-8988-256b2504ebc7\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:45:33 crc kubenswrapper[5052]: I1203 12:45:33.815090 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:45:33 crc kubenswrapper[5052]: I1203 12:45:33.837589 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/fb6c11ac-1b0e-4e63-8988-256b2504ebc7-logs\") pod \"nova-api-0\" (UID: \"fb6c11ac-1b0e-4e63-8988-256b2504ebc7\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:45:33 crc kubenswrapper[5052]: I1203 12:45:33.837634 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/fb6c11ac-1b0e-4e63-8988-256b2504ebc7-internal-tls-certs\") pod \"nova-api-0\" (UID: \"fb6c11ac-1b0e-4e63-8988-256b2504ebc7\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:45:33 crc kubenswrapper[5052]: I1203 12:45:33.837716 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/fb6c11ac-1b0e-4e63-8988-256b2504ebc7-public-tls-certs\") pod \"nova-api-0\" (UID: \"fb6c11ac-1b0e-4e63-8988-256b2504ebc7\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:45:33 crc kubenswrapper[5052]: I1203 12:45:33.837747 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/fb6c11ac-1b0e-4e63-8988-256b2504ebc7-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"fb6c11ac-1b0e-4e63-8988-256b2504ebc7\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:45:33 crc kubenswrapper[5052]: I1203 12:45:33.837966 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-mp72c\" (UniqueName: \"kubernetes.io/projected/fb6c11ac-1b0e-4e63-8988-256b2504ebc7-kube-api-access-mp72c\") pod \"nova-api-0\" (UID: \"fb6c11ac-1b0e-4e63-8988-256b2504ebc7\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:45:33 crc kubenswrapper[5052]: I1203 12:45:33.837990 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/fb6c11ac-1b0e-4e63-8988-256b2504ebc7-config-data\") pod \"nova-api-0\" (UID: \"fb6c11ac-1b0e-4e63-8988-256b2504ebc7\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:45:33 crc kubenswrapper[5052]: I1203 12:45:33.838264 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/fb6c11ac-1b0e-4e63-8988-256b2504ebc7-logs\") pod \"nova-api-0\" (UID: \"fb6c11ac-1b0e-4e63-8988-256b2504ebc7\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:45:33 crc kubenswrapper[5052]: I1203 12:45:33.843448 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/fb6c11ac-1b0e-4e63-8988-256b2504ebc7-internal-tls-certs\") pod \"nova-api-0\" (UID: \"fb6c11ac-1b0e-4e63-8988-256b2504ebc7\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:45:33 crc kubenswrapper[5052]: I1203 12:45:33.844074 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/fb6c11ac-1b0e-4e63-8988-256b2504ebc7-config-data\") pod \"nova-api-0\" (UID: \"fb6c11ac-1b0e-4e63-8988-256b2504ebc7\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:45:33 crc kubenswrapper[5052]: I1203 12:45:33.846057 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/fb6c11ac-1b0e-4e63-8988-256b2504ebc7-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"fb6c11ac-1b0e-4e63-8988-256b2504ebc7\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:45:33 crc kubenswrapper[5052]: I1203 12:45:33.856156 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/fb6c11ac-1b0e-4e63-8988-256b2504ebc7-public-tls-certs\") pod \"nova-api-0\" (UID: \"fb6c11ac-1b0e-4e63-8988-256b2504ebc7\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:45:33 crc kubenswrapper[5052]: I1203 12:45:33.865837 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-mp72c\" (UniqueName: \"kubernetes.io/projected/fb6c11ac-1b0e-4e63-8988-256b2504ebc7-kube-api-access-mp72c\") pod \"nova-api-0\" (UID: \"fb6c11ac-1b0e-4e63-8988-256b2504ebc7\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:45:33 crc kubenswrapper[5052]: I1203 12:45:33.938904 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/e08a0c0d-36f0-49f1-9df8-f0e616d1d99d-log-httpd\") pod \"e08a0c0d-36f0-49f1-9df8-f0e616d1d99d\" (UID: \"e08a0c0d-36f0-49f1-9df8-f0e616d1d99d\") " Dec 03 12:45:33 crc kubenswrapper[5052]: I1203 12:45:33.939007 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/e08a0c0d-36f0-49f1-9df8-f0e616d1d99d-scripts\") pod \"e08a0c0d-36f0-49f1-9df8-f0e616d1d99d\" (UID: \"e08a0c0d-36f0-49f1-9df8-f0e616d1d99d\") " Dec 03 12:45:33 crc kubenswrapper[5052]: I1203 12:45:33.939067 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/e08a0c0d-36f0-49f1-9df8-f0e616d1d99d-run-httpd\") pod \"e08a0c0d-36f0-49f1-9df8-f0e616d1d99d\" (UID: \"e08a0c0d-36f0-49f1-9df8-f0e616d1d99d\") " Dec 03 12:45:33 crc kubenswrapper[5052]: I1203 12:45:33.939103 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r65kf\" (UniqueName: \"kubernetes.io/projected/e08a0c0d-36f0-49f1-9df8-f0e616d1d99d-kube-api-access-r65kf\") pod \"e08a0c0d-36f0-49f1-9df8-f0e616d1d99d\" (UID: \"e08a0c0d-36f0-49f1-9df8-f0e616d1d99d\") " Dec 03 12:45:33 crc kubenswrapper[5052]: I1203 12:45:33.939136 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e08a0c0d-36f0-49f1-9df8-f0e616d1d99d-combined-ca-bundle\") pod \"e08a0c0d-36f0-49f1-9df8-f0e616d1d99d\" (UID: \"e08a0c0d-36f0-49f1-9df8-f0e616d1d99d\") " Dec 03 12:45:33 crc kubenswrapper[5052]: I1203 12:45:33.939157 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/e08a0c0d-36f0-49f1-9df8-f0e616d1d99d-ceilometer-tls-certs\") pod \"e08a0c0d-36f0-49f1-9df8-f0e616d1d99d\" (UID: \"e08a0c0d-36f0-49f1-9df8-f0e616d1d99d\") " Dec 03 12:45:33 crc kubenswrapper[5052]: I1203 12:45:33.939207 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/e08a0c0d-36f0-49f1-9df8-f0e616d1d99d-sg-core-conf-yaml\") pod \"e08a0c0d-36f0-49f1-9df8-f0e616d1d99d\" (UID: \"e08a0c0d-36f0-49f1-9df8-f0e616d1d99d\") " Dec 03 12:45:33 crc kubenswrapper[5052]: I1203 12:45:33.939277 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/e08a0c0d-36f0-49f1-9df8-f0e616d1d99d-config-data\") pod \"e08a0c0d-36f0-49f1-9df8-f0e616d1d99d\" (UID: \"e08a0c0d-36f0-49f1-9df8-f0e616d1d99d\") " Dec 03 12:45:33 crc kubenswrapper[5052]: I1203 12:45:33.941544 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e08a0c0d-36f0-49f1-9df8-f0e616d1d99d-run-httpd" (OuterVolumeSpecName: "run-httpd") pod "e08a0c0d-36f0-49f1-9df8-f0e616d1d99d" (UID: "e08a0c0d-36f0-49f1-9df8-f0e616d1d99d"). InnerVolumeSpecName "run-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:45:33 crc kubenswrapper[5052]: I1203 12:45:33.941555 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e08a0c0d-36f0-49f1-9df8-f0e616d1d99d-log-httpd" (OuterVolumeSpecName: "log-httpd") pod "e08a0c0d-36f0-49f1-9df8-f0e616d1d99d" (UID: "e08a0c0d-36f0-49f1-9df8-f0e616d1d99d"). InnerVolumeSpecName "log-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:45:33 crc kubenswrapper[5052]: I1203 12:45:33.946195 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e08a0c0d-36f0-49f1-9df8-f0e616d1d99d-kube-api-access-r65kf" (OuterVolumeSpecName: "kube-api-access-r65kf") pod "e08a0c0d-36f0-49f1-9df8-f0e616d1d99d" (UID: "e08a0c0d-36f0-49f1-9df8-f0e616d1d99d"). InnerVolumeSpecName "kube-api-access-r65kf". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:45:33 crc kubenswrapper[5052]: I1203 12:45:33.946954 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e08a0c0d-36f0-49f1-9df8-f0e616d1d99d-scripts" (OuterVolumeSpecName: "scripts") pod "e08a0c0d-36f0-49f1-9df8-f0e616d1d99d" (UID: "e08a0c0d-36f0-49f1-9df8-f0e616d1d99d"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:45:33 crc kubenswrapper[5052]: I1203 12:45:33.977823 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e08a0c0d-36f0-49f1-9df8-f0e616d1d99d-sg-core-conf-yaml" (OuterVolumeSpecName: "sg-core-conf-yaml") pod "e08a0c0d-36f0-49f1-9df8-f0e616d1d99d" (UID: "e08a0c0d-36f0-49f1-9df8-f0e616d1d99d"). InnerVolumeSpecName "sg-core-conf-yaml". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:45:33 crc kubenswrapper[5052]: I1203 12:45:33.999715 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e08a0c0d-36f0-49f1-9df8-f0e616d1d99d-ceilometer-tls-certs" (OuterVolumeSpecName: "ceilometer-tls-certs") pod "e08a0c0d-36f0-49f1-9df8-f0e616d1d99d" (UID: "e08a0c0d-36f0-49f1-9df8-f0e616d1d99d"). InnerVolumeSpecName "ceilometer-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:45:34 crc kubenswrapper[5052]: I1203 12:45:33.999965 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:45:34 crc kubenswrapper[5052]: I1203 12:45:34.029784 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e08a0c0d-36f0-49f1-9df8-f0e616d1d99d-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "e08a0c0d-36f0-49f1-9df8-f0e616d1d99d" (UID: "e08a0c0d-36f0-49f1-9df8-f0e616d1d99d"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:45:34 crc kubenswrapper[5052]: I1203 12:45:34.042569 5052 reconciler_common.go:293] "Volume detached for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/e08a0c0d-36f0-49f1-9df8-f0e616d1d99d-log-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:45:34 crc kubenswrapper[5052]: I1203 12:45:34.042608 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/e08a0c0d-36f0-49f1-9df8-f0e616d1d99d-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:45:34 crc kubenswrapper[5052]: I1203 12:45:34.042621 5052 reconciler_common.go:293] "Volume detached for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/e08a0c0d-36f0-49f1-9df8-f0e616d1d99d-run-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:45:34 crc kubenswrapper[5052]: I1203 12:45:34.042635 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-r65kf\" (UniqueName: \"kubernetes.io/projected/e08a0c0d-36f0-49f1-9df8-f0e616d1d99d-kube-api-access-r65kf\") on node \"crc\" DevicePath \"\"" Dec 03 12:45:34 crc kubenswrapper[5052]: I1203 12:45:34.042675 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e08a0c0d-36f0-49f1-9df8-f0e616d1d99d-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:45:34 crc kubenswrapper[5052]: I1203 12:45:34.042689 5052 reconciler_common.go:293] "Volume detached for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/e08a0c0d-36f0-49f1-9df8-f0e616d1d99d-ceilometer-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:45:34 crc kubenswrapper[5052]: I1203 12:45:34.042701 5052 reconciler_common.go:293] "Volume detached for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/e08a0c0d-36f0-49f1-9df8-f0e616d1d99d-sg-core-conf-yaml\") on node \"crc\" DevicePath \"\"" Dec 03 12:45:34 crc kubenswrapper[5052]: I1203 12:45:34.065752 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e08a0c0d-36f0-49f1-9df8-f0e616d1d99d-config-data" (OuterVolumeSpecName: "config-data") pod "e08a0c0d-36f0-49f1-9df8-f0e616d1d99d" (UID: "e08a0c0d-36f0-49f1-9df8-f0e616d1d99d"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:45:34 crc kubenswrapper[5052]: I1203 12:45:34.146969 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/e08a0c0d-36f0-49f1-9df8-f0e616d1d99d-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:45:34 crc kubenswrapper[5052]: I1203 12:45:34.456796 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:45:34 crc kubenswrapper[5052]: W1203 12:45:34.464924 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfb6c11ac_1b0e_4e63_8988_256b2504ebc7.slice/crio-a0169135c26c35f1fdcae98770c9290a702b10feb005661d47a023d4ebac16eb WatchSource:0}: Error finding container a0169135c26c35f1fdcae98770c9290a702b10feb005661d47a023d4ebac16eb: Status 404 returned error can't find the container with id a0169135c26c35f1fdcae98770c9290a702b10feb005661d47a023d4ebac16eb Dec 03 12:45:34 crc kubenswrapper[5052]: I1203 12:45:34.594995 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"e08a0c0d-36f0-49f1-9df8-f0e616d1d99d","Type":"ContainerDied","Data":"0af06a3fd779532e53d9dcc3c63361b2314d6b161c27df98ee0efb27b89b5b76"} Dec 03 12:45:34 crc kubenswrapper[5052]: I1203 12:45:34.595061 5052 scope.go:117] "RemoveContainer" containerID="9a2e22eb61df2421257c5aa00a47a7949877fdb593a1d8c3d3099b33f82916e0" Dec 03 12:45:34 crc kubenswrapper[5052]: I1203 12:45:34.594989 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:45:34 crc kubenswrapper[5052]: I1203 12:45:34.597770 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"fb6c11ac-1b0e-4e63-8988-256b2504ebc7","Type":"ContainerStarted","Data":"a0169135c26c35f1fdcae98770c9290a702b10feb005661d47a023d4ebac16eb"} Dec 03 12:45:34 crc kubenswrapper[5052]: I1203 12:45:34.622224 5052 scope.go:117] "RemoveContainer" containerID="6b9dc66f0245c06a5ff620f79133997e00f75f954583172251c9c6a8120066db" Dec 03 12:45:34 crc kubenswrapper[5052]: I1203 12:45:34.646490 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:45:34 crc kubenswrapper[5052]: I1203 12:45:34.661762 5052 scope.go:117] "RemoveContainer" containerID="6e4c007c3a17262d2fde8b64aa6bf056a29b57cd2ef4649732b01c4b83ed1119" Dec 03 12:45:34 crc kubenswrapper[5052]: I1203 12:45:34.664903 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:45:34 crc kubenswrapper[5052]: I1203 12:45:34.676428 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:45:34 crc kubenswrapper[5052]: E1203 12:45:34.677802 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="e08a0c0d-36f0-49f1-9df8-f0e616d1d99d" containerName="ceilometer-central-agent" Dec 03 12:45:34 crc kubenswrapper[5052]: I1203 12:45:34.677824 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="e08a0c0d-36f0-49f1-9df8-f0e616d1d99d" containerName="ceilometer-central-agent" Dec 03 12:45:34 crc kubenswrapper[5052]: E1203 12:45:34.677834 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="e08a0c0d-36f0-49f1-9df8-f0e616d1d99d" containerName="ceilometer-notification-agent" Dec 03 12:45:34 crc kubenswrapper[5052]: I1203 12:45:34.677840 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="e08a0c0d-36f0-49f1-9df8-f0e616d1d99d" containerName="ceilometer-notification-agent" Dec 03 12:45:34 crc kubenswrapper[5052]: E1203 12:45:34.677865 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="e08a0c0d-36f0-49f1-9df8-f0e616d1d99d" containerName="proxy-httpd" Dec 03 12:45:34 crc kubenswrapper[5052]: I1203 12:45:34.677871 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="e08a0c0d-36f0-49f1-9df8-f0e616d1d99d" containerName="proxy-httpd" Dec 03 12:45:34 crc kubenswrapper[5052]: E1203 12:45:34.677913 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="e08a0c0d-36f0-49f1-9df8-f0e616d1d99d" containerName="sg-core" Dec 03 12:45:34 crc kubenswrapper[5052]: I1203 12:45:34.677923 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="e08a0c0d-36f0-49f1-9df8-f0e616d1d99d" containerName="sg-core" Dec 03 12:45:34 crc kubenswrapper[5052]: I1203 12:45:34.678544 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="e08a0c0d-36f0-49f1-9df8-f0e616d1d99d" containerName="ceilometer-central-agent" Dec 03 12:45:34 crc kubenswrapper[5052]: I1203 12:45:34.678584 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="e08a0c0d-36f0-49f1-9df8-f0e616d1d99d" containerName="proxy-httpd" Dec 03 12:45:34 crc kubenswrapper[5052]: I1203 12:45:34.678625 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="e08a0c0d-36f0-49f1-9df8-f0e616d1d99d" containerName="ceilometer-notification-agent" Dec 03 12:45:34 crc kubenswrapper[5052]: I1203 12:45:34.678642 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="e08a0c0d-36f0-49f1-9df8-f0e616d1d99d" containerName="sg-core" Dec 03 12:45:34 crc kubenswrapper[5052]: I1203 12:45:34.680879 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:45:34 crc kubenswrapper[5052]: I1203 12:45:34.684407 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-ceilometer-internal-svc" Dec 03 12:45:34 crc kubenswrapper[5052]: I1203 12:45:34.684583 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-scripts" Dec 03 12:45:34 crc kubenswrapper[5052]: I1203 12:45:34.684684 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-config-data" Dec 03 12:45:34 crc kubenswrapper[5052]: I1203 12:45:34.704066 5052 scope.go:117] "RemoveContainer" containerID="d957f6d4ec3ecf5552e7abc256da98e11212433b834afa67d861c95a81528ec6" Dec 03 12:45:34 crc kubenswrapper[5052]: I1203 12:45:34.713564 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:45:34 crc kubenswrapper[5052]: I1203 12:45:34.760254 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/4eab5b48-f674-4100-9b42-f3ea257d47d4-scripts\") pod \"ceilometer-0\" (UID: \"4eab5b48-f674-4100-9b42-f3ea257d47d4\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:45:34 crc kubenswrapper[5052]: I1203 12:45:34.760315 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/4eab5b48-f674-4100-9b42-f3ea257d47d4-run-httpd\") pod \"ceilometer-0\" (UID: \"4eab5b48-f674-4100-9b42-f3ea257d47d4\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:45:34 crc kubenswrapper[5052]: I1203 12:45:34.760336 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4eab5b48-f674-4100-9b42-f3ea257d47d4-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"4eab5b48-f674-4100-9b42-f3ea257d47d4\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:45:34 crc kubenswrapper[5052]: I1203 12:45:34.760367 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/4eab5b48-f674-4100-9b42-f3ea257d47d4-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"4eab5b48-f674-4100-9b42-f3ea257d47d4\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:45:34 crc kubenswrapper[5052]: I1203 12:45:34.760404 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/4eab5b48-f674-4100-9b42-f3ea257d47d4-ceilometer-tls-certs\") pod \"ceilometer-0\" (UID: \"4eab5b48-f674-4100-9b42-f3ea257d47d4\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:45:34 crc kubenswrapper[5052]: I1203 12:45:34.760489 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/4eab5b48-f674-4100-9b42-f3ea257d47d4-log-httpd\") pod \"ceilometer-0\" (UID: \"4eab5b48-f674-4100-9b42-f3ea257d47d4\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:45:34 crc kubenswrapper[5052]: I1203 12:45:34.760798 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j2rns\" (UniqueName: \"kubernetes.io/projected/4eab5b48-f674-4100-9b42-f3ea257d47d4-kube-api-access-j2rns\") pod \"ceilometer-0\" (UID: \"4eab5b48-f674-4100-9b42-f3ea257d47d4\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:45:34 crc kubenswrapper[5052]: I1203 12:45:34.761440 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/4eab5b48-f674-4100-9b42-f3ea257d47d4-config-data\") pod \"ceilometer-0\" (UID: \"4eab5b48-f674-4100-9b42-f3ea257d47d4\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:45:34 crc kubenswrapper[5052]: I1203 12:45:34.862945 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-j2rns\" (UniqueName: \"kubernetes.io/projected/4eab5b48-f674-4100-9b42-f3ea257d47d4-kube-api-access-j2rns\") pod \"ceilometer-0\" (UID: \"4eab5b48-f674-4100-9b42-f3ea257d47d4\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:45:34 crc kubenswrapper[5052]: I1203 12:45:34.863003 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/4eab5b48-f674-4100-9b42-f3ea257d47d4-config-data\") pod \"ceilometer-0\" (UID: \"4eab5b48-f674-4100-9b42-f3ea257d47d4\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:45:34 crc kubenswrapper[5052]: I1203 12:45:34.863085 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/4eab5b48-f674-4100-9b42-f3ea257d47d4-scripts\") pod \"ceilometer-0\" (UID: \"4eab5b48-f674-4100-9b42-f3ea257d47d4\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:45:34 crc kubenswrapper[5052]: I1203 12:45:34.864001 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/4eab5b48-f674-4100-9b42-f3ea257d47d4-run-httpd\") pod \"ceilometer-0\" (UID: \"4eab5b48-f674-4100-9b42-f3ea257d47d4\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:45:34 crc kubenswrapper[5052]: I1203 12:45:34.864048 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4eab5b48-f674-4100-9b42-f3ea257d47d4-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"4eab5b48-f674-4100-9b42-f3ea257d47d4\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:45:34 crc kubenswrapper[5052]: I1203 12:45:34.864132 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/4eab5b48-f674-4100-9b42-f3ea257d47d4-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"4eab5b48-f674-4100-9b42-f3ea257d47d4\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:45:34 crc kubenswrapper[5052]: I1203 12:45:34.864188 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/4eab5b48-f674-4100-9b42-f3ea257d47d4-ceilometer-tls-certs\") pod \"ceilometer-0\" (UID: \"4eab5b48-f674-4100-9b42-f3ea257d47d4\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:45:34 crc kubenswrapper[5052]: I1203 12:45:34.864345 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/4eab5b48-f674-4100-9b42-f3ea257d47d4-log-httpd\") pod \"ceilometer-0\" (UID: \"4eab5b48-f674-4100-9b42-f3ea257d47d4\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:45:34 crc kubenswrapper[5052]: I1203 12:45:34.864718 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/4eab5b48-f674-4100-9b42-f3ea257d47d4-run-httpd\") pod \"ceilometer-0\" (UID: \"4eab5b48-f674-4100-9b42-f3ea257d47d4\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:45:34 crc kubenswrapper[5052]: I1203 12:45:34.864893 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/4eab5b48-f674-4100-9b42-f3ea257d47d4-log-httpd\") pod \"ceilometer-0\" (UID: \"4eab5b48-f674-4100-9b42-f3ea257d47d4\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:45:34 crc kubenswrapper[5052]: I1203 12:45:34.868111 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4eab5b48-f674-4100-9b42-f3ea257d47d4-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"4eab5b48-f674-4100-9b42-f3ea257d47d4\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:45:34 crc kubenswrapper[5052]: I1203 12:45:34.868188 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/4eab5b48-f674-4100-9b42-f3ea257d47d4-ceilometer-tls-certs\") pod \"ceilometer-0\" (UID: \"4eab5b48-f674-4100-9b42-f3ea257d47d4\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:45:34 crc kubenswrapper[5052]: I1203 12:45:34.868685 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/4eab5b48-f674-4100-9b42-f3ea257d47d4-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"4eab5b48-f674-4100-9b42-f3ea257d47d4\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:45:34 crc kubenswrapper[5052]: I1203 12:45:34.869277 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/4eab5b48-f674-4100-9b42-f3ea257d47d4-scripts\") pod \"ceilometer-0\" (UID: \"4eab5b48-f674-4100-9b42-f3ea257d47d4\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:45:34 crc kubenswrapper[5052]: I1203 12:45:34.871035 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/4eab5b48-f674-4100-9b42-f3ea257d47d4-config-data\") pod \"ceilometer-0\" (UID: \"4eab5b48-f674-4100-9b42-f3ea257d47d4\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:45:34 crc kubenswrapper[5052]: I1203 12:45:34.882923 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-j2rns\" (UniqueName: \"kubernetes.io/projected/4eab5b48-f674-4100-9b42-f3ea257d47d4-kube-api-access-j2rns\") pod \"ceilometer-0\" (UID: \"4eab5b48-f674-4100-9b42-f3ea257d47d4\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:45:35 crc kubenswrapper[5052]: I1203 12:45:35.002069 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:45:35 crc kubenswrapper[5052]: I1203 12:45:35.504411 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:45:35 crc kubenswrapper[5052]: W1203 12:45:35.506332 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4eab5b48_f674_4100_9b42_f3ea257d47d4.slice/crio-729c50e16fe2bc63c224552fab14a703ef50089d0da8fe68249c1879afea0e05 WatchSource:0}: Error finding container 729c50e16fe2bc63c224552fab14a703ef50089d0da8fe68249c1879afea0e05: Status 404 returned error can't find the container with id 729c50e16fe2bc63c224552fab14a703ef50089d0da8fe68249c1879afea0e05 Dec 03 12:45:35 crc kubenswrapper[5052]: I1203 12:45:35.611185 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"fb6c11ac-1b0e-4e63-8988-256b2504ebc7","Type":"ContainerStarted","Data":"c2c82ad1c4ab93ed36cb072abbe6d1eeb5753acbd507bfb1d2fbc3bacdb79bdf"} Dec 03 12:45:35 crc kubenswrapper[5052]: I1203 12:45:35.611226 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"fb6c11ac-1b0e-4e63-8988-256b2504ebc7","Type":"ContainerStarted","Data":"d57a2057a1ffffdae42cdf5eae9f0f819cc86143c1132d66146d4814f2b2355f"} Dec 03 12:45:35 crc kubenswrapper[5052]: I1203 12:45:35.613451 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"4eab5b48-f674-4100-9b42-f3ea257d47d4","Type":"ContainerStarted","Data":"729c50e16fe2bc63c224552fab14a703ef50089d0da8fe68249c1879afea0e05"} Dec 03 12:45:35 crc kubenswrapper[5052]: I1203 12:45:35.645531 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-api-0" podStartSLOduration=2.645490353 podStartE2EDuration="2.645490353s" podCreationTimestamp="2025-12-03 12:45:33 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:45:35.639152052 +0000 UTC m=+5126.227393836" watchObservedRunningTime="2025-12-03 12:45:35.645490353 +0000 UTC m=+5126.233732137" Dec 03 12:45:35 crc kubenswrapper[5052]: I1203 12:45:35.678009 5052 scope.go:117] "RemoveContainer" containerID="2aed1c760a326f3d008799e902b1093313df3764a57c879f0b3686410127d37d" Dec 03 12:45:35 crc kubenswrapper[5052]: E1203 12:45:35.678271 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 12:45:35 crc kubenswrapper[5052]: I1203 12:45:35.683447 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e08a0c0d-36f0-49f1-9df8-f0e616d1d99d" path="/var/lib/kubelet/pods/e08a0c0d-36f0-49f1-9df8-f0e616d1d99d/volumes" Dec 03 12:45:36 crc kubenswrapper[5052]: I1203 12:45:36.625680 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"4eab5b48-f674-4100-9b42-f3ea257d47d4","Type":"ContainerStarted","Data":"c9ba5984640e4b46f378e76e1ad9dbbc5b996fff2d3c392ba7de3ff6753a48fd"} Dec 03 12:45:38 crc kubenswrapper[5052]: I1203 12:45:38.647658 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"4eab5b48-f674-4100-9b42-f3ea257d47d4","Type":"ContainerStarted","Data":"d0d21a72bcbb3c194d8e900bc8615778059b26e70f1614cda87f183610dac51e"} Dec 03 12:45:40 crc kubenswrapper[5052]: I1203 12:45:40.671143 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"4eab5b48-f674-4100-9b42-f3ea257d47d4","Type":"ContainerStarted","Data":"7370547fbe4eb4455656b595065ce614afd49fa9ea95ccde635adb0b612d3c70"} Dec 03 12:45:42 crc kubenswrapper[5052]: I1203 12:45:42.700786 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"4eab5b48-f674-4100-9b42-f3ea257d47d4","Type":"ContainerStarted","Data":"01c4f4342abe4df4b3d520a97ffb3f4ba0abfe85cb81b459e64f94869c4a5f91"} Dec 03 12:45:42 crc kubenswrapper[5052]: I1203 12:45:42.701432 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:45:42 crc kubenswrapper[5052]: I1203 12:45:42.738229 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/ceilometer-0" podStartSLOduration=2.721995619 podStartE2EDuration="8.738204932s" podCreationTimestamp="2025-12-03 12:45:34 +0000 UTC" firstStartedPulling="2025-12-03 12:45:35.509175075 +0000 UTC m=+5126.097416829" lastFinishedPulling="2025-12-03 12:45:41.525384368 +0000 UTC m=+5132.113626142" observedRunningTime="2025-12-03 12:45:42.729444572 +0000 UTC m=+5133.317686336" watchObservedRunningTime="2025-12-03 12:45:42.738204932 +0000 UTC m=+5133.326446696" Dec 03 12:45:44 crc kubenswrapper[5052]: I1203 12:45:44.000972 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:45:44 crc kubenswrapper[5052]: I1203 12:45:44.001292 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:45:45 crc kubenswrapper[5052]: I1203 12:45:45.013702 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-api-0" podUID="fb6c11ac-1b0e-4e63-8988-256b2504ebc7" containerName="nova-api-api" probeResult="failure" output="Get \"https://10.217.1.237:8774/\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Dec 03 12:45:45 crc kubenswrapper[5052]: I1203 12:45:45.013741 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-api-0" podUID="fb6c11ac-1b0e-4e63-8988-256b2504ebc7" containerName="nova-api-log" probeResult="failure" output="Get \"https://10.217.1.237:8774/\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Dec 03 12:45:46 crc kubenswrapper[5052]: I1203 12:45:46.673087 5052 scope.go:117] "RemoveContainer" containerID="2aed1c760a326f3d008799e902b1093313df3764a57c879f0b3686410127d37d" Dec 03 12:45:46 crc kubenswrapper[5052]: E1203 12:45:46.674249 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 12:45:51 crc kubenswrapper[5052]: I1203 12:45:51.724598 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/openstack-cell1-galera-0" podUID="2c53ea4e-56ba-4ba1-8cb6-8c24725b5856" containerName="galera" probeResult="failure" output="command timed out" Dec 03 12:45:54 crc kubenswrapper[5052]: I1203 12:45:54.007681 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:45:54 crc kubenswrapper[5052]: I1203 12:45:54.008959 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:45:54 crc kubenswrapper[5052]: I1203 12:45:54.014299 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:45:54 crc kubenswrapper[5052]: I1203 12:45:54.016864 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:45:54 crc kubenswrapper[5052]: I1203 12:45:54.819934 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:45:54 crc kubenswrapper[5052]: I1203 12:45:54.827887 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:45:58 crc kubenswrapper[5052]: I1203 12:45:58.673744 5052 scope.go:117] "RemoveContainer" containerID="2aed1c760a326f3d008799e902b1093313df3764a57c879f0b3686410127d37d" Dec 03 12:45:58 crc kubenswrapper[5052]: E1203 12:45:58.674581 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 12:46:05 crc kubenswrapper[5052]: I1203 12:46:05.010846 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:46:11 crc kubenswrapper[5052]: I1203 12:46:11.356421 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:46:11 crc kubenswrapper[5052]: I1203 12:46:11.357287 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-metadata-0" podUID="9af09a5b-789c-4b83-b899-9d1b3f0d8603" containerName="nova-metadata-log" containerID="cri-o://deb3641e5c7641ec59de2756335b3b8af86a98493025f1061ab9587775677d7d" gracePeriod=30 Dec 03 12:46:11 crc kubenswrapper[5052]: I1203 12:46:11.357366 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-metadata-0" podUID="9af09a5b-789c-4b83-b899-9d1b3f0d8603" containerName="nova-metadata-metadata" containerID="cri-o://9ba6898618d6ecc3b57802d08d86da02201fb0d67fcd7288c3414ce61b3233ed" gracePeriod=30 Dec 03 12:46:11 crc kubenswrapper[5052]: I1203 12:46:11.373569 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:46:11 crc kubenswrapper[5052]: I1203 12:46:11.373826 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-scheduler-0" podUID="dfda1fd2-f923-47bd-bf43-8a1be3f79f09" containerName="nova-scheduler-scheduler" containerID="cri-o://67e80f3b2185e807d12e6cb73fcdb3e6261d02a42c22e32ff2b8b1ff0732f99b" gracePeriod=30 Dec 03 12:46:11 crc kubenswrapper[5052]: I1203 12:46:11.397775 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ovn-northd-0"] Dec 03 12:46:11 crc kubenswrapper[5052]: I1203 12:46:11.398031 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ovn-northd-0" podUID="6af0a301-2483-493d-bbc8-17138d79dfea" containerName="ovn-northd" containerID="cri-o://adc4f6d0e36c4370a4bff93d0b0803fec09e3b34230d31ebbe4f500396923a29" gracePeriod=30 Dec 03 12:46:11 crc kubenswrapper[5052]: I1203 12:46:11.398575 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ovn-northd-0" podUID="6af0a301-2483-493d-bbc8-17138d79dfea" containerName="openstack-network-exporter" containerID="cri-o://4307735eba8d16b00eb4ca5f1d746b675f8e81debcf3a1c8d9b709367fb6a8be" gracePeriod=30 Dec 03 12:46:11 crc kubenswrapper[5052]: I1203 12:46:11.410599 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-0"] Dec 03 12:46:11 crc kubenswrapper[5052]: I1203 12:46:11.410832 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-cell0-conductor-0" podUID="f7e26e55-1426-4cd4-b169-a63868acdfaa" containerName="nova-cell0-conductor-conductor" containerID="cri-o://bbc2755b5351566cbdd1dc7fd68849cb7ea314f303defbebf723032464036311" gracePeriod=30 Dec 03 12:46:11 crc kubenswrapper[5052]: I1203 12:46:11.428831 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/openstackclient"] Dec 03 12:46:11 crc kubenswrapper[5052]: I1203 12:46:11.429049 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/openstackclient" podUID="6475384f-1965-4e8b-8806-af420d1381de" containerName="openstackclient" containerID="cri-o://06fb78df44fbba56afa104136ade5fc0d9d9f01772c05059cf7f5e2ec493e511" gracePeriod=2 Dec 03 12:46:11 crc kubenswrapper[5052]: I1203 12:46:11.444693 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-sb-0"] Dec 03 12:46:11 crc kubenswrapper[5052]: I1203 12:46:11.445122 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ovsdbserver-sb-0" podUID="f2d7e3bb-2829-4166-90df-53f9a50272eb" containerName="openstack-network-exporter" containerID="cri-o://1499c2af6931030cde94894361cbf72f702626308576bf5ee6e7760bbc5034df" gracePeriod=300 Dec 03 12:46:11 crc kubenswrapper[5052]: I1203 12:46:11.461566 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/openstackclient"] Dec 03 12:46:11 crc kubenswrapper[5052]: I1203 12:46:11.478581 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/barbican-keystone-listener-68fd6d997d-7tjjw"] Dec 03 12:46:11 crc kubenswrapper[5052]: E1203 12:46:11.479094 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="6475384f-1965-4e8b-8806-af420d1381de" containerName="openstackclient" Dec 03 12:46:11 crc kubenswrapper[5052]: I1203 12:46:11.479116 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="6475384f-1965-4e8b-8806-af420d1381de" containerName="openstackclient" Dec 03 12:46:11 crc kubenswrapper[5052]: I1203 12:46:11.479295 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="6475384f-1965-4e8b-8806-af420d1381de" containerName="openstackclient" Dec 03 12:46:11 crc kubenswrapper[5052]: I1203 12:46:11.480495 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-keystone-listener-68fd6d997d-7tjjw" Dec 03 12:46:11 crc kubenswrapper[5052]: I1203 12:46:11.503958 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:46:11 crc kubenswrapper[5052]: I1203 12:46:11.504158 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-external-api-0" podUID="9aea20a7-eaed-43a8-9b52-08c2c0212230" containerName="glance-log" containerID="cri-o://c5ed1d5bcb2e814a4e6ff6889daaa85dec6c18fe852411af7cfacd20fd4d3320" gracePeriod=30 Dec 03 12:46:11 crc kubenswrapper[5052]: I1203 12:46:11.504301 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-external-api-0" podUID="9aea20a7-eaed-43a8-9b52-08c2c0212230" containerName="glance-httpd" containerID="cri-o://3242b025201bb8378bb8f5f363d45551adfc8da4587f92d211c75e0e901ef11a" gracePeriod=30 Dec 03 12:46:11 crc kubenswrapper[5052]: I1203 12:46:11.526650 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/barbican-worker-58f7cb7fd9-2k5tg"] Dec 03 12:46:11 crc kubenswrapper[5052]: I1203 12:46:11.528816 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-worker-58f7cb7fd9-2k5tg" Dec 03 12:46:11 crc kubenswrapper[5052]: I1203 12:46:11.552938 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-keystone-listener-68fd6d997d-7tjjw"] Dec 03 12:46:11 crc kubenswrapper[5052]: I1203 12:46:11.568566 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-worker-58f7cb7fd9-2k5tg"] Dec 03 12:46:11 crc kubenswrapper[5052]: I1203 12:46:11.569527 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rhgx2\" (UniqueName: \"kubernetes.io/projected/e14465ed-868a-49ba-b3ab-1e67196229e8-kube-api-access-rhgx2\") pod \"barbican-worker-58f7cb7fd9-2k5tg\" (UID: \"e14465ed-868a-49ba-b3ab-1e67196229e8\") " pod="openstack-kuttl-tests/barbican-worker-58f7cb7fd9-2k5tg" Dec 03 12:46:11 crc kubenswrapper[5052]: I1203 12:46:11.569620 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/2065f5bb-88eb-4e84-88b6-9982833d413d-config-data\") pod \"barbican-keystone-listener-68fd6d997d-7tjjw\" (UID: \"2065f5bb-88eb-4e84-88b6-9982833d413d\") " pod="openstack-kuttl-tests/barbican-keystone-listener-68fd6d997d-7tjjw" Dec 03 12:46:11 crc kubenswrapper[5052]: I1203 12:46:11.569690 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/2065f5bb-88eb-4e84-88b6-9982833d413d-config-data-custom\") pod \"barbican-keystone-listener-68fd6d997d-7tjjw\" (UID: \"2065f5bb-88eb-4e84-88b6-9982833d413d\") " pod="openstack-kuttl-tests/barbican-keystone-listener-68fd6d997d-7tjjw" Dec 03 12:46:11 crc kubenswrapper[5052]: I1203 12:46:11.569725 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bswmw\" (UniqueName: \"kubernetes.io/projected/2065f5bb-88eb-4e84-88b6-9982833d413d-kube-api-access-bswmw\") pod \"barbican-keystone-listener-68fd6d997d-7tjjw\" (UID: \"2065f5bb-88eb-4e84-88b6-9982833d413d\") " pod="openstack-kuttl-tests/barbican-keystone-listener-68fd6d997d-7tjjw" Dec 03 12:46:11 crc kubenswrapper[5052]: I1203 12:46:11.569792 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2065f5bb-88eb-4e84-88b6-9982833d413d-combined-ca-bundle\") pod \"barbican-keystone-listener-68fd6d997d-7tjjw\" (UID: \"2065f5bb-88eb-4e84-88b6-9982833d413d\") " pod="openstack-kuttl-tests/barbican-keystone-listener-68fd6d997d-7tjjw" Dec 03 12:46:11 crc kubenswrapper[5052]: I1203 12:46:11.569824 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/e14465ed-868a-49ba-b3ab-1e67196229e8-config-data-custom\") pod \"barbican-worker-58f7cb7fd9-2k5tg\" (UID: \"e14465ed-868a-49ba-b3ab-1e67196229e8\") " pod="openstack-kuttl-tests/barbican-worker-58f7cb7fd9-2k5tg" Dec 03 12:46:11 crc kubenswrapper[5052]: I1203 12:46:11.569856 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e14465ed-868a-49ba-b3ab-1e67196229e8-combined-ca-bundle\") pod \"barbican-worker-58f7cb7fd9-2k5tg\" (UID: \"e14465ed-868a-49ba-b3ab-1e67196229e8\") " pod="openstack-kuttl-tests/barbican-worker-58f7cb7fd9-2k5tg" Dec 03 12:46:11 crc kubenswrapper[5052]: I1203 12:46:11.569910 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/e14465ed-868a-49ba-b3ab-1e67196229e8-logs\") pod \"barbican-worker-58f7cb7fd9-2k5tg\" (UID: \"e14465ed-868a-49ba-b3ab-1e67196229e8\") " pod="openstack-kuttl-tests/barbican-worker-58f7cb7fd9-2k5tg" Dec 03 12:46:11 crc kubenswrapper[5052]: I1203 12:46:11.569962 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/e14465ed-868a-49ba-b3ab-1e67196229e8-config-data\") pod \"barbican-worker-58f7cb7fd9-2k5tg\" (UID: \"e14465ed-868a-49ba-b3ab-1e67196229e8\") " pod="openstack-kuttl-tests/barbican-worker-58f7cb7fd9-2k5tg" Dec 03 12:46:11 crc kubenswrapper[5052]: I1203 12:46:11.569989 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/2065f5bb-88eb-4e84-88b6-9982833d413d-logs\") pod \"barbican-keystone-listener-68fd6d997d-7tjjw\" (UID: \"2065f5bb-88eb-4e84-88b6-9982833d413d\") " pod="openstack-kuttl-tests/barbican-keystone-listener-68fd6d997d-7tjjw" Dec 03 12:46:11 crc kubenswrapper[5052]: I1203 12:46:11.586293 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/memcached-0"] Dec 03 12:46:11 crc kubenswrapper[5052]: I1203 12:46:11.586538 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/memcached-0" podUID="2f966eeb-76ef-4771-acdf-e0de2bb463a4" containerName="memcached" containerID="cri-o://db9cddd5e36fcb65f1558dec41404bab89ab5883ec640a193f5a47a8c712d924" gracePeriod=30 Dec 03 12:46:11 crc kubenswrapper[5052]: I1203 12:46:11.604488 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 12:46:11 crc kubenswrapper[5052]: I1203 12:46:11.604838 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/cinder-scheduler-0" podUID="3ae64378-aaa4-46bc-8c48-e1d90e490016" containerName="cinder-scheduler" containerID="cri-o://8e0b376c29b7c2dd056afa36e69404e2be33493fa6b1a9ba35735a9c8becf167" gracePeriod=30 Dec 03 12:46:11 crc kubenswrapper[5052]: I1203 12:46:11.605027 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/cinder-scheduler-0" podUID="3ae64378-aaa4-46bc-8c48-e1d90e490016" containerName="probe" containerID="cri-o://3fa64d6a78f92e04f85cc78283bcecfe7e618fa805695f4e69f5237e051e4939" gracePeriod=30 Dec 03 12:46:11 crc kubenswrapper[5052]: I1203 12:46:11.676591 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 12:46:11 crc kubenswrapper[5052]: I1203 12:46:11.676897 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" podUID="e5f1bd78-3552-4d2d-8029-651f33c3918a" containerName="nova-cell1-novncproxy-novncproxy" containerID="cri-o://0c534558ed9f09feb4924d04acb90157e0fc3201f00732c7364a3bf82d62e52e" gracePeriod=30 Dec 03 12:46:11 crc kubenswrapper[5052]: I1203 12:46:11.677710 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/2065f5bb-88eb-4e84-88b6-9982833d413d-config-data-custom\") pod \"barbican-keystone-listener-68fd6d997d-7tjjw\" (UID: \"2065f5bb-88eb-4e84-88b6-9982833d413d\") " pod="openstack-kuttl-tests/barbican-keystone-listener-68fd6d997d-7tjjw" Dec 03 12:46:11 crc kubenswrapper[5052]: I1203 12:46:11.677771 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-bswmw\" (UniqueName: \"kubernetes.io/projected/2065f5bb-88eb-4e84-88b6-9982833d413d-kube-api-access-bswmw\") pod \"barbican-keystone-listener-68fd6d997d-7tjjw\" (UID: \"2065f5bb-88eb-4e84-88b6-9982833d413d\") " pod="openstack-kuttl-tests/barbican-keystone-listener-68fd6d997d-7tjjw" Dec 03 12:46:11 crc kubenswrapper[5052]: I1203 12:46:11.677811 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2065f5bb-88eb-4e84-88b6-9982833d413d-combined-ca-bundle\") pod \"barbican-keystone-listener-68fd6d997d-7tjjw\" (UID: \"2065f5bb-88eb-4e84-88b6-9982833d413d\") " pod="openstack-kuttl-tests/barbican-keystone-listener-68fd6d997d-7tjjw" Dec 03 12:46:11 crc kubenswrapper[5052]: I1203 12:46:11.677844 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/e14465ed-868a-49ba-b3ab-1e67196229e8-config-data-custom\") pod \"barbican-worker-58f7cb7fd9-2k5tg\" (UID: \"e14465ed-868a-49ba-b3ab-1e67196229e8\") " pod="openstack-kuttl-tests/barbican-worker-58f7cb7fd9-2k5tg" Dec 03 12:46:11 crc kubenswrapper[5052]: I1203 12:46:11.677873 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e14465ed-868a-49ba-b3ab-1e67196229e8-combined-ca-bundle\") pod \"barbican-worker-58f7cb7fd9-2k5tg\" (UID: \"e14465ed-868a-49ba-b3ab-1e67196229e8\") " pod="openstack-kuttl-tests/barbican-worker-58f7cb7fd9-2k5tg" Dec 03 12:46:11 crc kubenswrapper[5052]: I1203 12:46:11.677918 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/e14465ed-868a-49ba-b3ab-1e67196229e8-logs\") pod \"barbican-worker-58f7cb7fd9-2k5tg\" (UID: \"e14465ed-868a-49ba-b3ab-1e67196229e8\") " pod="openstack-kuttl-tests/barbican-worker-58f7cb7fd9-2k5tg" Dec 03 12:46:11 crc kubenswrapper[5052]: I1203 12:46:11.677950 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/e14465ed-868a-49ba-b3ab-1e67196229e8-config-data\") pod \"barbican-worker-58f7cb7fd9-2k5tg\" (UID: \"e14465ed-868a-49ba-b3ab-1e67196229e8\") " pod="openstack-kuttl-tests/barbican-worker-58f7cb7fd9-2k5tg" Dec 03 12:46:11 crc kubenswrapper[5052]: I1203 12:46:11.677973 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/2065f5bb-88eb-4e84-88b6-9982833d413d-logs\") pod \"barbican-keystone-listener-68fd6d997d-7tjjw\" (UID: \"2065f5bb-88eb-4e84-88b6-9982833d413d\") " pod="openstack-kuttl-tests/barbican-keystone-listener-68fd6d997d-7tjjw" Dec 03 12:46:11 crc kubenswrapper[5052]: I1203 12:46:11.678062 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-rhgx2\" (UniqueName: \"kubernetes.io/projected/e14465ed-868a-49ba-b3ab-1e67196229e8-kube-api-access-rhgx2\") pod \"barbican-worker-58f7cb7fd9-2k5tg\" (UID: \"e14465ed-868a-49ba-b3ab-1e67196229e8\") " pod="openstack-kuttl-tests/barbican-worker-58f7cb7fd9-2k5tg" Dec 03 12:46:11 crc kubenswrapper[5052]: I1203 12:46:11.678100 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/2065f5bb-88eb-4e84-88b6-9982833d413d-config-data\") pod \"barbican-keystone-listener-68fd6d997d-7tjjw\" (UID: \"2065f5bb-88eb-4e84-88b6-9982833d413d\") " pod="openstack-kuttl-tests/barbican-keystone-listener-68fd6d997d-7tjjw" Dec 03 12:46:11 crc kubenswrapper[5052]: I1203 12:46:11.697132 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/2065f5bb-88eb-4e84-88b6-9982833d413d-logs\") pod \"barbican-keystone-listener-68fd6d997d-7tjjw\" (UID: \"2065f5bb-88eb-4e84-88b6-9982833d413d\") " pod="openstack-kuttl-tests/barbican-keystone-listener-68fd6d997d-7tjjw" Dec 03 12:46:11 crc kubenswrapper[5052]: I1203 12:46:11.698275 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/e14465ed-868a-49ba-b3ab-1e67196229e8-logs\") pod \"barbican-worker-58f7cb7fd9-2k5tg\" (UID: \"e14465ed-868a-49ba-b3ab-1e67196229e8\") " pod="openstack-kuttl-tests/barbican-worker-58f7cb7fd9-2k5tg" Dec 03 12:46:11 crc kubenswrapper[5052]: I1203 12:46:11.723990 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2065f5bb-88eb-4e84-88b6-9982833d413d-combined-ca-bundle\") pod \"barbican-keystone-listener-68fd6d997d-7tjjw\" (UID: \"2065f5bb-88eb-4e84-88b6-9982833d413d\") " pod="openstack-kuttl-tests/barbican-keystone-listener-68fd6d997d-7tjjw" Dec 03 12:46:11 crc kubenswrapper[5052]: I1203 12:46:11.736747 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/e14465ed-868a-49ba-b3ab-1e67196229e8-config-data\") pod \"barbican-worker-58f7cb7fd9-2k5tg\" (UID: \"e14465ed-868a-49ba-b3ab-1e67196229e8\") " pod="openstack-kuttl-tests/barbican-worker-58f7cb7fd9-2k5tg" Dec 03 12:46:11 crc kubenswrapper[5052]: I1203 12:46:11.737680 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/e14465ed-868a-49ba-b3ab-1e67196229e8-config-data-custom\") pod \"barbican-worker-58f7cb7fd9-2k5tg\" (UID: \"e14465ed-868a-49ba-b3ab-1e67196229e8\") " pod="openstack-kuttl-tests/barbican-worker-58f7cb7fd9-2k5tg" Dec 03 12:46:11 crc kubenswrapper[5052]: I1203 12:46:11.744909 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/2065f5bb-88eb-4e84-88b6-9982833d413d-config-data-custom\") pod \"barbican-keystone-listener-68fd6d997d-7tjjw\" (UID: \"2065f5bb-88eb-4e84-88b6-9982833d413d\") " pod="openstack-kuttl-tests/barbican-keystone-listener-68fd6d997d-7tjjw" Dec 03 12:46:11 crc kubenswrapper[5052]: I1203 12:46:11.779507 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/2065f5bb-88eb-4e84-88b6-9982833d413d-config-data\") pod \"barbican-keystone-listener-68fd6d997d-7tjjw\" (UID: \"2065f5bb-88eb-4e84-88b6-9982833d413d\") " pod="openstack-kuttl-tests/barbican-keystone-listener-68fd6d997d-7tjjw" Dec 03 12:46:11 crc kubenswrapper[5052]: I1203 12:46:11.801208 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-bswmw\" (UniqueName: \"kubernetes.io/projected/2065f5bb-88eb-4e84-88b6-9982833d413d-kube-api-access-bswmw\") pod \"barbican-keystone-listener-68fd6d997d-7tjjw\" (UID: \"2065f5bb-88eb-4e84-88b6-9982833d413d\") " pod="openstack-kuttl-tests/barbican-keystone-listener-68fd6d997d-7tjjw" Dec 03 12:46:11 crc kubenswrapper[5052]: I1203 12:46:11.804049 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/openstack-galera-0"] Dec 03 12:46:11 crc kubenswrapper[5052]: I1203 12:46:11.804104 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/placement-bbcfbbc9d-5q6gx"] Dec 03 12:46:11 crc kubenswrapper[5052]: I1203 12:46:11.806362 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-rhgx2\" (UniqueName: \"kubernetes.io/projected/e14465ed-868a-49ba-b3ab-1e67196229e8-kube-api-access-rhgx2\") pod \"barbican-worker-58f7cb7fd9-2k5tg\" (UID: \"e14465ed-868a-49ba-b3ab-1e67196229e8\") " pod="openstack-kuttl-tests/barbican-worker-58f7cb7fd9-2k5tg" Dec 03 12:46:11 crc kubenswrapper[5052]: I1203 12:46:11.810852 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-bbcfbbc9d-5q6gx" Dec 03 12:46:11 crc kubenswrapper[5052]: I1203 12:46:11.815716 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e14465ed-868a-49ba-b3ab-1e67196229e8-combined-ca-bundle\") pod \"barbican-worker-58f7cb7fd9-2k5tg\" (UID: \"e14465ed-868a-49ba-b3ab-1e67196229e8\") " pod="openstack-kuttl-tests/barbican-worker-58f7cb7fd9-2k5tg" Dec 03 12:46:11 crc kubenswrapper[5052]: I1203 12:46:11.900357 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/barbican-api-bcf49b784-jnkdb"] Dec 03 12:46:11 crc kubenswrapper[5052]: I1203 12:46:11.911181 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-keystone-listener-68fd6d997d-7tjjw" Dec 03 12:46:11 crc kubenswrapper[5052]: I1203 12:46:11.924591 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-api-bcf49b784-jnkdb" Dec 03 12:46:11 crc kubenswrapper[5052]: I1203 12:46:11.927356 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-worker-58f7cb7fd9-2k5tg" Dec 03 12:46:11 crc kubenswrapper[5052]: I1203 12:46:11.961572 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/placement-bbcfbbc9d-5q6gx"] Dec 03 12:46:11 crc kubenswrapper[5052]: I1203 12:46:11.972978 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/32f420c2-aa7e-44c8-a63a-7654aa3e6e34-config-data\") pod \"placement-bbcfbbc9d-5q6gx\" (UID: \"32f420c2-aa7e-44c8-a63a-7654aa3e6e34\") " pod="openstack-kuttl-tests/placement-bbcfbbc9d-5q6gx" Dec 03 12:46:11 crc kubenswrapper[5052]: I1203 12:46:11.973097 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/32f420c2-aa7e-44c8-a63a-7654aa3e6e34-combined-ca-bundle\") pod \"placement-bbcfbbc9d-5q6gx\" (UID: \"32f420c2-aa7e-44c8-a63a-7654aa3e6e34\") " pod="openstack-kuttl-tests/placement-bbcfbbc9d-5q6gx" Dec 03 12:46:11 crc kubenswrapper[5052]: I1203 12:46:11.973125 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/32f420c2-aa7e-44c8-a63a-7654aa3e6e34-internal-tls-certs\") pod \"placement-bbcfbbc9d-5q6gx\" (UID: \"32f420c2-aa7e-44c8-a63a-7654aa3e6e34\") " pod="openstack-kuttl-tests/placement-bbcfbbc9d-5q6gx" Dec 03 12:46:11 crc kubenswrapper[5052]: I1203 12:46:11.973160 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/32f420c2-aa7e-44c8-a63a-7654aa3e6e34-scripts\") pod \"placement-bbcfbbc9d-5q6gx\" (UID: \"32f420c2-aa7e-44c8-a63a-7654aa3e6e34\") " pod="openstack-kuttl-tests/placement-bbcfbbc9d-5q6gx" Dec 03 12:46:11 crc kubenswrapper[5052]: I1203 12:46:11.973180 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/32f420c2-aa7e-44c8-a63a-7654aa3e6e34-public-tls-certs\") pod \"placement-bbcfbbc9d-5q6gx\" (UID: \"32f420c2-aa7e-44c8-a63a-7654aa3e6e34\") " pod="openstack-kuttl-tests/placement-bbcfbbc9d-5q6gx" Dec 03 12:46:11 crc kubenswrapper[5052]: I1203 12:46:11.973235 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/32f420c2-aa7e-44c8-a63a-7654aa3e6e34-logs\") pod \"placement-bbcfbbc9d-5q6gx\" (UID: \"32f420c2-aa7e-44c8-a63a-7654aa3e6e34\") " pod="openstack-kuttl-tests/placement-bbcfbbc9d-5q6gx" Dec 03 12:46:11 crc kubenswrapper[5052]: I1203 12:46:11.973257 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xwlbj\" (UniqueName: \"kubernetes.io/projected/32f420c2-aa7e-44c8-a63a-7654aa3e6e34-kube-api-access-xwlbj\") pod \"placement-bbcfbbc9d-5q6gx\" (UID: \"32f420c2-aa7e-44c8-a63a-7654aa3e6e34\") " pod="openstack-kuttl-tests/placement-bbcfbbc9d-5q6gx" Dec 03 12:46:12 crc kubenswrapper[5052]: I1203 12:46:12.070743 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 12:46:12 crc kubenswrapper[5052]: I1203 12:46:12.071037 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/cinder-api-0" podUID="68a80c17-9f7d-47de-b02c-5e7bb47656ab" containerName="cinder-api-log" containerID="cri-o://8562f7f2637fa32043b41c559e46c0266da41a1cab62e80ce7b44cb981784aae" gracePeriod=30 Dec 03 12:46:12 crc kubenswrapper[5052]: I1203 12:46:12.071167 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/cinder-api-0" podUID="68a80c17-9f7d-47de-b02c-5e7bb47656ab" containerName="cinder-api" containerID="cri-o://36e0739e37891a21cddc05442e8ef21c0dc06d541fe56c5e1fce38f2ed446057" gracePeriod=30 Dec 03 12:46:12 crc kubenswrapper[5052]: I1203 12:46:12.077349 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/32f420c2-aa7e-44c8-a63a-7654aa3e6e34-scripts\") pod \"placement-bbcfbbc9d-5q6gx\" (UID: \"32f420c2-aa7e-44c8-a63a-7654aa3e6e34\") " pod="openstack-kuttl-tests/placement-bbcfbbc9d-5q6gx" Dec 03 12:46:12 crc kubenswrapper[5052]: I1203 12:46:12.077410 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/32f420c2-aa7e-44c8-a63a-7654aa3e6e34-public-tls-certs\") pod \"placement-bbcfbbc9d-5q6gx\" (UID: \"32f420c2-aa7e-44c8-a63a-7654aa3e6e34\") " pod="openstack-kuttl-tests/placement-bbcfbbc9d-5q6gx" Dec 03 12:46:12 crc kubenswrapper[5052]: I1203 12:46:12.077992 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/c069aee2-4cb6-4c8a-bbe5-949ab0b48331-public-tls-certs\") pod \"barbican-api-bcf49b784-jnkdb\" (UID: \"c069aee2-4cb6-4c8a-bbe5-949ab0b48331\") " pod="openstack-kuttl-tests/barbican-api-bcf49b784-jnkdb" Dec 03 12:46:12 crc kubenswrapper[5052]: I1203 12:46:12.078054 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/32f420c2-aa7e-44c8-a63a-7654aa3e6e34-logs\") pod \"placement-bbcfbbc9d-5q6gx\" (UID: \"32f420c2-aa7e-44c8-a63a-7654aa3e6e34\") " pod="openstack-kuttl-tests/placement-bbcfbbc9d-5q6gx" Dec 03 12:46:12 crc kubenswrapper[5052]: I1203 12:46:12.078083 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-xwlbj\" (UniqueName: \"kubernetes.io/projected/32f420c2-aa7e-44c8-a63a-7654aa3e6e34-kube-api-access-xwlbj\") pod \"placement-bbcfbbc9d-5q6gx\" (UID: \"32f420c2-aa7e-44c8-a63a-7654aa3e6e34\") " pod="openstack-kuttl-tests/placement-bbcfbbc9d-5q6gx" Dec 03 12:46:12 crc kubenswrapper[5052]: I1203 12:46:12.078147 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/c069aee2-4cb6-4c8a-bbe5-949ab0b48331-internal-tls-certs\") pod \"barbican-api-bcf49b784-jnkdb\" (UID: \"c069aee2-4cb6-4c8a-bbe5-949ab0b48331\") " pod="openstack-kuttl-tests/barbican-api-bcf49b784-jnkdb" Dec 03 12:46:12 crc kubenswrapper[5052]: I1203 12:46:12.078173 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/c069aee2-4cb6-4c8a-bbe5-949ab0b48331-logs\") pod \"barbican-api-bcf49b784-jnkdb\" (UID: \"c069aee2-4cb6-4c8a-bbe5-949ab0b48331\") " pod="openstack-kuttl-tests/barbican-api-bcf49b784-jnkdb" Dec 03 12:46:12 crc kubenswrapper[5052]: I1203 12:46:12.078196 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/32f420c2-aa7e-44c8-a63a-7654aa3e6e34-config-data\") pod \"placement-bbcfbbc9d-5q6gx\" (UID: \"32f420c2-aa7e-44c8-a63a-7654aa3e6e34\") " pod="openstack-kuttl-tests/placement-bbcfbbc9d-5q6gx" Dec 03 12:46:12 crc kubenswrapper[5052]: I1203 12:46:12.078230 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c069aee2-4cb6-4c8a-bbe5-949ab0b48331-combined-ca-bundle\") pod \"barbican-api-bcf49b784-jnkdb\" (UID: \"c069aee2-4cb6-4c8a-bbe5-949ab0b48331\") " pod="openstack-kuttl-tests/barbican-api-bcf49b784-jnkdb" Dec 03 12:46:12 crc kubenswrapper[5052]: I1203 12:46:12.078597 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/32f420c2-aa7e-44c8-a63a-7654aa3e6e34-combined-ca-bundle\") pod \"placement-bbcfbbc9d-5q6gx\" (UID: \"32f420c2-aa7e-44c8-a63a-7654aa3e6e34\") " pod="openstack-kuttl-tests/placement-bbcfbbc9d-5q6gx" Dec 03 12:46:12 crc kubenswrapper[5052]: I1203 12:46:12.078746 5052 generic.go:334] "Generic (PLEG): container finished" podID="6af0a301-2483-493d-bbc8-17138d79dfea" containerID="4307735eba8d16b00eb4ca5f1d746b675f8e81debcf3a1c8d9b709367fb6a8be" exitCode=2 Dec 03 12:46:12 crc kubenswrapper[5052]: I1203 12:46:12.078775 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovn-northd-0" event={"ID":"6af0a301-2483-493d-bbc8-17138d79dfea","Type":"ContainerDied","Data":"4307735eba8d16b00eb4ca5f1d746b675f8e81debcf3a1c8d9b709367fb6a8be"} Dec 03 12:46:12 crc kubenswrapper[5052]: I1203 12:46:12.078797 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/32f420c2-aa7e-44c8-a63a-7654aa3e6e34-logs\") pod \"placement-bbcfbbc9d-5q6gx\" (UID: \"32f420c2-aa7e-44c8-a63a-7654aa3e6e34\") " pod="openstack-kuttl-tests/placement-bbcfbbc9d-5q6gx" Dec 03 12:46:12 crc kubenswrapper[5052]: I1203 12:46:12.078923 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/32f420c2-aa7e-44c8-a63a-7654aa3e6e34-internal-tls-certs\") pod \"placement-bbcfbbc9d-5q6gx\" (UID: \"32f420c2-aa7e-44c8-a63a-7654aa3e6e34\") " pod="openstack-kuttl-tests/placement-bbcfbbc9d-5q6gx" Dec 03 12:46:12 crc kubenswrapper[5052]: I1203 12:46:12.078955 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/c069aee2-4cb6-4c8a-bbe5-949ab0b48331-config-data-custom\") pod \"barbican-api-bcf49b784-jnkdb\" (UID: \"c069aee2-4cb6-4c8a-bbe5-949ab0b48331\") " pod="openstack-kuttl-tests/barbican-api-bcf49b784-jnkdb" Dec 03 12:46:12 crc kubenswrapper[5052]: I1203 12:46:12.078997 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/c069aee2-4cb6-4c8a-bbe5-949ab0b48331-config-data\") pod \"barbican-api-bcf49b784-jnkdb\" (UID: \"c069aee2-4cb6-4c8a-bbe5-949ab0b48331\") " pod="openstack-kuttl-tests/barbican-api-bcf49b784-jnkdb" Dec 03 12:46:12 crc kubenswrapper[5052]: I1203 12:46:12.079024 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q9q5k\" (UniqueName: \"kubernetes.io/projected/c069aee2-4cb6-4c8a-bbe5-949ab0b48331-kube-api-access-q9q5k\") pod \"barbican-api-bcf49b784-jnkdb\" (UID: \"c069aee2-4cb6-4c8a-bbe5-949ab0b48331\") " pod="openstack-kuttl-tests/barbican-api-bcf49b784-jnkdb" Dec 03 12:46:12 crc kubenswrapper[5052]: I1203 12:46:12.108025 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-api-bcf49b784-jnkdb"] Dec 03 12:46:12 crc kubenswrapper[5052]: I1203 12:46:12.157949 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/keystone-687ff6fcfc-9vwb2"] Dec 03 12:46:12 crc kubenswrapper[5052]: I1203 12:46:12.211931 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/c069aee2-4cb6-4c8a-bbe5-949ab0b48331-config-data-custom\") pod \"barbican-api-bcf49b784-jnkdb\" (UID: \"c069aee2-4cb6-4c8a-bbe5-949ab0b48331\") " pod="openstack-kuttl-tests/barbican-api-bcf49b784-jnkdb" Dec 03 12:46:12 crc kubenswrapper[5052]: I1203 12:46:12.212020 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/c069aee2-4cb6-4c8a-bbe5-949ab0b48331-config-data\") pod \"barbican-api-bcf49b784-jnkdb\" (UID: \"c069aee2-4cb6-4c8a-bbe5-949ab0b48331\") " pod="openstack-kuttl-tests/barbican-api-bcf49b784-jnkdb" Dec 03 12:46:12 crc kubenswrapper[5052]: I1203 12:46:12.212062 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-q9q5k\" (UniqueName: \"kubernetes.io/projected/c069aee2-4cb6-4c8a-bbe5-949ab0b48331-kube-api-access-q9q5k\") pod \"barbican-api-bcf49b784-jnkdb\" (UID: \"c069aee2-4cb6-4c8a-bbe5-949ab0b48331\") " pod="openstack-kuttl-tests/barbican-api-bcf49b784-jnkdb" Dec 03 12:46:12 crc kubenswrapper[5052]: I1203 12:46:12.212230 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/c069aee2-4cb6-4c8a-bbe5-949ab0b48331-public-tls-certs\") pod \"barbican-api-bcf49b784-jnkdb\" (UID: \"c069aee2-4cb6-4c8a-bbe5-949ab0b48331\") " pod="openstack-kuttl-tests/barbican-api-bcf49b784-jnkdb" Dec 03 12:46:12 crc kubenswrapper[5052]: I1203 12:46:12.212380 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/c069aee2-4cb6-4c8a-bbe5-949ab0b48331-internal-tls-certs\") pod \"barbican-api-bcf49b784-jnkdb\" (UID: \"c069aee2-4cb6-4c8a-bbe5-949ab0b48331\") " pod="openstack-kuttl-tests/barbican-api-bcf49b784-jnkdb" Dec 03 12:46:12 crc kubenswrapper[5052]: I1203 12:46:12.212421 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/c069aee2-4cb6-4c8a-bbe5-949ab0b48331-logs\") pod \"barbican-api-bcf49b784-jnkdb\" (UID: \"c069aee2-4cb6-4c8a-bbe5-949ab0b48331\") " pod="openstack-kuttl-tests/barbican-api-bcf49b784-jnkdb" Dec 03 12:46:12 crc kubenswrapper[5052]: I1203 12:46:12.213409 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-687ff6fcfc-9vwb2" Dec 03 12:46:12 crc kubenswrapper[5052]: E1203 12:46:12.214770 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-barbican-public-svc: secret "cert-barbican-public-svc" not found Dec 03 12:46:12 crc kubenswrapper[5052]: E1203 12:46:12.214865 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c069aee2-4cb6-4c8a-bbe5-949ab0b48331-public-tls-certs podName:c069aee2-4cb6-4c8a-bbe5-949ab0b48331 nodeName:}" failed. No retries permitted until 2025-12-03 12:46:12.714848752 +0000 UTC m=+5163.303090516 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "public-tls-certs" (UniqueName: "kubernetes.io/secret/c069aee2-4cb6-4c8a-bbe5-949ab0b48331-public-tls-certs") pod "barbican-api-bcf49b784-jnkdb" (UID: "c069aee2-4cb6-4c8a-bbe5-949ab0b48331") : secret "cert-barbican-public-svc" not found Dec 03 12:46:12 crc kubenswrapper[5052]: E1203 12:46:12.216972 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-barbican-internal-svc: secret "cert-barbican-internal-svc" not found Dec 03 12:46:12 crc kubenswrapper[5052]: E1203 12:46:12.217026 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c069aee2-4cb6-4c8a-bbe5-949ab0b48331-internal-tls-certs podName:c069aee2-4cb6-4c8a-bbe5-949ab0b48331 nodeName:}" failed. No retries permitted until 2025-12-03 12:46:12.717012884 +0000 UTC m=+5163.305254648 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "internal-tls-certs" (UniqueName: "kubernetes.io/secret/c069aee2-4cb6-4c8a-bbe5-949ab0b48331-internal-tls-certs") pod "barbican-api-bcf49b784-jnkdb" (UID: "c069aee2-4cb6-4c8a-bbe5-949ab0b48331") : secret "cert-barbican-internal-svc" not found Dec 03 12:46:12 crc kubenswrapper[5052]: I1203 12:46:12.219441 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c069aee2-4cb6-4c8a-bbe5-949ab0b48331-combined-ca-bundle\") pod \"barbican-api-bcf49b784-jnkdb\" (UID: \"c069aee2-4cb6-4c8a-bbe5-949ab0b48331\") " pod="openstack-kuttl-tests/barbican-api-bcf49b784-jnkdb" Dec 03 12:46:12 crc kubenswrapper[5052]: I1203 12:46:12.283445 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/openstack-cell1-galera-0"] Dec 03 12:46:12 crc kubenswrapper[5052]: I1203 12:46:12.324483 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/fa1c223d-82fa-488c-ac48-e88bce36a839-scripts\") pod \"keystone-687ff6fcfc-9vwb2\" (UID: \"fa1c223d-82fa-488c-ac48-e88bce36a839\") " pod="openstack-kuttl-tests/keystone-687ff6fcfc-9vwb2" Dec 03 12:46:12 crc kubenswrapper[5052]: I1203 12:46:12.324863 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/fa1c223d-82fa-488c-ac48-e88bce36a839-public-tls-certs\") pod \"keystone-687ff6fcfc-9vwb2\" (UID: \"fa1c223d-82fa-488c-ac48-e88bce36a839\") " pod="openstack-kuttl-tests/keystone-687ff6fcfc-9vwb2" Dec 03 12:46:12 crc kubenswrapper[5052]: I1203 12:46:12.326116 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/fa1c223d-82fa-488c-ac48-e88bce36a839-internal-tls-certs\") pod \"keystone-687ff6fcfc-9vwb2\" (UID: \"fa1c223d-82fa-488c-ac48-e88bce36a839\") " pod="openstack-kuttl-tests/keystone-687ff6fcfc-9vwb2" Dec 03 12:46:12 crc kubenswrapper[5052]: I1203 12:46:12.326193 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/fa1c223d-82fa-488c-ac48-e88bce36a839-credential-keys\") pod \"keystone-687ff6fcfc-9vwb2\" (UID: \"fa1c223d-82fa-488c-ac48-e88bce36a839\") " pod="openstack-kuttl-tests/keystone-687ff6fcfc-9vwb2" Dec 03 12:46:12 crc kubenswrapper[5052]: I1203 12:46:12.326445 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/fa1c223d-82fa-488c-ac48-e88bce36a839-config-data\") pod \"keystone-687ff6fcfc-9vwb2\" (UID: \"fa1c223d-82fa-488c-ac48-e88bce36a839\") " pod="openstack-kuttl-tests/keystone-687ff6fcfc-9vwb2" Dec 03 12:46:12 crc kubenswrapper[5052]: I1203 12:46:12.339945 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/fa1c223d-82fa-488c-ac48-e88bce36a839-fernet-keys\") pod \"keystone-687ff6fcfc-9vwb2\" (UID: \"fa1c223d-82fa-488c-ac48-e88bce36a839\") " pod="openstack-kuttl-tests/keystone-687ff6fcfc-9vwb2" Dec 03 12:46:12 crc kubenswrapper[5052]: I1203 12:46:12.340017 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bsn6g\" (UniqueName: \"kubernetes.io/projected/fa1c223d-82fa-488c-ac48-e88bce36a839-kube-api-access-bsn6g\") pod \"keystone-687ff6fcfc-9vwb2\" (UID: \"fa1c223d-82fa-488c-ac48-e88bce36a839\") " pod="openstack-kuttl-tests/keystone-687ff6fcfc-9vwb2" Dec 03 12:46:12 crc kubenswrapper[5052]: I1203 12:46:12.340048 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/fa1c223d-82fa-488c-ac48-e88bce36a839-combined-ca-bundle\") pod \"keystone-687ff6fcfc-9vwb2\" (UID: \"fa1c223d-82fa-488c-ac48-e88bce36a839\") " pod="openstack-kuttl-tests/keystone-687ff6fcfc-9vwb2" Dec 03 12:46:12 crc kubenswrapper[5052]: I1203 12:46:12.348234 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/c069aee2-4cb6-4c8a-bbe5-949ab0b48331-logs\") pod \"barbican-api-bcf49b784-jnkdb\" (UID: \"c069aee2-4cb6-4c8a-bbe5-949ab0b48331\") " pod="openstack-kuttl-tests/barbican-api-bcf49b784-jnkdb" Dec 03 12:46:12 crc kubenswrapper[5052]: I1203 12:46:12.392583 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:46:12 crc kubenswrapper[5052]: I1203 12:46:12.392914 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-api-0" podUID="fb6c11ac-1b0e-4e63-8988-256b2504ebc7" containerName="nova-api-log" containerID="cri-o://d57a2057a1ffffdae42cdf5eae9f0f819cc86143c1132d66146d4814f2b2355f" gracePeriod=30 Dec 03 12:46:12 crc kubenswrapper[5052]: I1203 12:46:12.393499 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-api-0" podUID="fb6c11ac-1b0e-4e63-8988-256b2504ebc7" containerName="nova-api-api" containerID="cri-o://c2c82ad1c4ab93ed36cb072abbe6d1eeb5753acbd507bfb1d2fbc3bacdb79bdf" gracePeriod=30 Dec 03 12:46:12 crc kubenswrapper[5052]: I1203 12:46:12.439590 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-687ff6fcfc-9vwb2"] Dec 03 12:46:12 crc kubenswrapper[5052]: I1203 12:46:12.441745 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/fa1c223d-82fa-488c-ac48-e88bce36a839-fernet-keys\") pod \"keystone-687ff6fcfc-9vwb2\" (UID: \"fa1c223d-82fa-488c-ac48-e88bce36a839\") " pod="openstack-kuttl-tests/keystone-687ff6fcfc-9vwb2" Dec 03 12:46:12 crc kubenswrapper[5052]: I1203 12:46:12.441792 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-bsn6g\" (UniqueName: \"kubernetes.io/projected/fa1c223d-82fa-488c-ac48-e88bce36a839-kube-api-access-bsn6g\") pod \"keystone-687ff6fcfc-9vwb2\" (UID: \"fa1c223d-82fa-488c-ac48-e88bce36a839\") " pod="openstack-kuttl-tests/keystone-687ff6fcfc-9vwb2" Dec 03 12:46:12 crc kubenswrapper[5052]: I1203 12:46:12.441812 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/fa1c223d-82fa-488c-ac48-e88bce36a839-combined-ca-bundle\") pod \"keystone-687ff6fcfc-9vwb2\" (UID: \"fa1c223d-82fa-488c-ac48-e88bce36a839\") " pod="openstack-kuttl-tests/keystone-687ff6fcfc-9vwb2" Dec 03 12:46:12 crc kubenswrapper[5052]: I1203 12:46:12.441834 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/fa1c223d-82fa-488c-ac48-e88bce36a839-scripts\") pod \"keystone-687ff6fcfc-9vwb2\" (UID: \"fa1c223d-82fa-488c-ac48-e88bce36a839\") " pod="openstack-kuttl-tests/keystone-687ff6fcfc-9vwb2" Dec 03 12:46:12 crc kubenswrapper[5052]: I1203 12:46:12.441867 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/fa1c223d-82fa-488c-ac48-e88bce36a839-public-tls-certs\") pod \"keystone-687ff6fcfc-9vwb2\" (UID: \"fa1c223d-82fa-488c-ac48-e88bce36a839\") " pod="openstack-kuttl-tests/keystone-687ff6fcfc-9vwb2" Dec 03 12:46:12 crc kubenswrapper[5052]: I1203 12:46:12.441933 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/fa1c223d-82fa-488c-ac48-e88bce36a839-internal-tls-certs\") pod \"keystone-687ff6fcfc-9vwb2\" (UID: \"fa1c223d-82fa-488c-ac48-e88bce36a839\") " pod="openstack-kuttl-tests/keystone-687ff6fcfc-9vwb2" Dec 03 12:46:12 crc kubenswrapper[5052]: I1203 12:46:12.441952 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/fa1c223d-82fa-488c-ac48-e88bce36a839-credential-keys\") pod \"keystone-687ff6fcfc-9vwb2\" (UID: \"fa1c223d-82fa-488c-ac48-e88bce36a839\") " pod="openstack-kuttl-tests/keystone-687ff6fcfc-9vwb2" Dec 03 12:46:12 crc kubenswrapper[5052]: I1203 12:46:12.442054 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/fa1c223d-82fa-488c-ac48-e88bce36a839-config-data\") pod \"keystone-687ff6fcfc-9vwb2\" (UID: \"fa1c223d-82fa-488c-ac48-e88bce36a839\") " pod="openstack-kuttl-tests/keystone-687ff6fcfc-9vwb2" Dec 03 12:46:12 crc kubenswrapper[5052]: E1203 12:46:12.444692 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-keystone-internal-svc: secret "cert-keystone-internal-svc" not found Dec 03 12:46:12 crc kubenswrapper[5052]: E1203 12:46:12.444798 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fa1c223d-82fa-488c-ac48-e88bce36a839-internal-tls-certs podName:fa1c223d-82fa-488c-ac48-e88bce36a839 nodeName:}" failed. No retries permitted until 2025-12-03 12:46:12.944769207 +0000 UTC m=+5163.533011031 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "internal-tls-certs" (UniqueName: "kubernetes.io/secret/fa1c223d-82fa-488c-ac48-e88bce36a839-internal-tls-certs") pod "keystone-687ff6fcfc-9vwb2" (UID: "fa1c223d-82fa-488c-ac48-e88bce36a839") : secret "cert-keystone-internal-svc" not found Dec 03 12:46:12 crc kubenswrapper[5052]: E1203 12:46:12.445069 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-keystone-public-svc: secret "cert-keystone-public-svc" not found Dec 03 12:46:12 crc kubenswrapper[5052]: E1203 12:46:12.445110 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fa1c223d-82fa-488c-ac48-e88bce36a839-public-tls-certs podName:fa1c223d-82fa-488c-ac48-e88bce36a839 nodeName:}" failed. No retries permitted until 2025-12-03 12:46:12.945099726 +0000 UTC m=+5163.533341580 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "public-tls-certs" (UniqueName: "kubernetes.io/secret/fa1c223d-82fa-488c-ac48-e88bce36a839-public-tls-certs") pod "keystone-687ff6fcfc-9vwb2" (UID: "fa1c223d-82fa-488c-ac48-e88bce36a839") : secret "cert-keystone-public-svc" not found Dec 03 12:46:12 crc kubenswrapper[5052]: I1203 12:46:12.462798 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/neutron-6d64d7894b-xp9d5"] Dec 03 12:46:12 crc kubenswrapper[5052]: I1203 12:46:12.465243 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-6d64d7894b-xp9d5" Dec 03 12:46:12 crc kubenswrapper[5052]: I1203 12:46:12.492223 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:46:12 crc kubenswrapper[5052]: I1203 12:46:12.492450 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-internal-api-0" podUID="78361066-6add-4971-9a4b-9157f35ad494" containerName="glance-log" containerID="cri-o://6fe0649b6ce8be2bfd99aed0e4830c4017fd592947629aa86d78326653fa7af0" gracePeriod=30 Dec 03 12:46:12 crc kubenswrapper[5052]: I1203 12:46:12.492591 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-internal-api-0" podUID="78361066-6add-4971-9a4b-9157f35ad494" containerName="glance-httpd" containerID="cri-o://3310a6083ad0bd13b8590b4fa080eeb83719e19bfb1dddf441320e21f33c7afa" gracePeriod=30 Dec 03 12:46:12 crc kubenswrapper[5052]: I1203 12:46:12.520762 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-6d64d7894b-xp9d5"] Dec 03 12:46:12 crc kubenswrapper[5052]: I1203 12:46:12.544700 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/901a2f0b-fb8e-4c97-aea7-567d7ee42a23-ovndb-tls-certs\") pod \"neutron-6d64d7894b-xp9d5\" (UID: \"901a2f0b-fb8e-4c97-aea7-567d7ee42a23\") " pod="openstack-kuttl-tests/neutron-6d64d7894b-xp9d5" Dec 03 12:46:12 crc kubenswrapper[5052]: I1203 12:46:12.545018 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/901a2f0b-fb8e-4c97-aea7-567d7ee42a23-combined-ca-bundle\") pod \"neutron-6d64d7894b-xp9d5\" (UID: \"901a2f0b-fb8e-4c97-aea7-567d7ee42a23\") " pod="openstack-kuttl-tests/neutron-6d64d7894b-xp9d5" Dec 03 12:46:12 crc kubenswrapper[5052]: I1203 12:46:12.545171 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/901a2f0b-fb8e-4c97-aea7-567d7ee42a23-internal-tls-certs\") pod \"neutron-6d64d7894b-xp9d5\" (UID: \"901a2f0b-fb8e-4c97-aea7-567d7ee42a23\") " pod="openstack-kuttl-tests/neutron-6d64d7894b-xp9d5" Dec 03 12:46:12 crc kubenswrapper[5052]: I1203 12:46:12.545270 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mwcgt\" (UniqueName: \"kubernetes.io/projected/901a2f0b-fb8e-4c97-aea7-567d7ee42a23-kube-api-access-mwcgt\") pod \"neutron-6d64d7894b-xp9d5\" (UID: \"901a2f0b-fb8e-4c97-aea7-567d7ee42a23\") " pod="openstack-kuttl-tests/neutron-6d64d7894b-xp9d5" Dec 03 12:46:12 crc kubenswrapper[5052]: I1203 12:46:12.545462 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/901a2f0b-fb8e-4c97-aea7-567d7ee42a23-config\") pod \"neutron-6d64d7894b-xp9d5\" (UID: \"901a2f0b-fb8e-4c97-aea7-567d7ee42a23\") " pod="openstack-kuttl-tests/neutron-6d64d7894b-xp9d5" Dec 03 12:46:12 crc kubenswrapper[5052]: I1203 12:46:12.545671 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/901a2f0b-fb8e-4c97-aea7-567d7ee42a23-httpd-config\") pod \"neutron-6d64d7894b-xp9d5\" (UID: \"901a2f0b-fb8e-4c97-aea7-567d7ee42a23\") " pod="openstack-kuttl-tests/neutron-6d64d7894b-xp9d5" Dec 03 12:46:12 crc kubenswrapper[5052]: I1203 12:46:12.545789 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/901a2f0b-fb8e-4c97-aea7-567d7ee42a23-public-tls-certs\") pod \"neutron-6d64d7894b-xp9d5\" (UID: \"901a2f0b-fb8e-4c97-aea7-567d7ee42a23\") " pod="openstack-kuttl-tests/neutron-6d64d7894b-xp9d5" Dec 03 12:46:12 crc kubenswrapper[5052]: I1203 12:46:12.556614 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-nb-0"] Dec 03 12:46:12 crc kubenswrapper[5052]: I1203 12:46:12.556969 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ovsdbserver-nb-0" podUID="ee05fe95-7cc0-4225-a86b-1682adc78b8b" containerName="openstack-network-exporter" containerID="cri-o://61f7b1adc6b8d50af5982e338e056fb28d2ea6d8f6e0b8c7092747bfc78b185f" gracePeriod=300 Dec 03 12:46:12 crc kubenswrapper[5052]: I1203 12:46:12.647335 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/32f420c2-aa7e-44c8-a63a-7654aa3e6e34-internal-tls-certs\") pod \"placement-bbcfbbc9d-5q6gx\" (UID: \"32f420c2-aa7e-44c8-a63a-7654aa3e6e34\") " pod="openstack-kuttl-tests/placement-bbcfbbc9d-5q6gx" Dec 03 12:46:12 crc kubenswrapper[5052]: I1203 12:46:12.647349 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/901a2f0b-fb8e-4c97-aea7-567d7ee42a23-ovndb-tls-certs\") pod \"neutron-6d64d7894b-xp9d5\" (UID: \"901a2f0b-fb8e-4c97-aea7-567d7ee42a23\") " pod="openstack-kuttl-tests/neutron-6d64d7894b-xp9d5" Dec 03 12:46:12 crc kubenswrapper[5052]: I1203 12:46:12.647406 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/901a2f0b-fb8e-4c97-aea7-567d7ee42a23-combined-ca-bundle\") pod \"neutron-6d64d7894b-xp9d5\" (UID: \"901a2f0b-fb8e-4c97-aea7-567d7ee42a23\") " pod="openstack-kuttl-tests/neutron-6d64d7894b-xp9d5" Dec 03 12:46:12 crc kubenswrapper[5052]: I1203 12:46:12.647476 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/901a2f0b-fb8e-4c97-aea7-567d7ee42a23-internal-tls-certs\") pod \"neutron-6d64d7894b-xp9d5\" (UID: \"901a2f0b-fb8e-4c97-aea7-567d7ee42a23\") " pod="openstack-kuttl-tests/neutron-6d64d7894b-xp9d5" Dec 03 12:46:12 crc kubenswrapper[5052]: I1203 12:46:12.647501 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-mwcgt\" (UniqueName: \"kubernetes.io/projected/901a2f0b-fb8e-4c97-aea7-567d7ee42a23-kube-api-access-mwcgt\") pod \"neutron-6d64d7894b-xp9d5\" (UID: \"901a2f0b-fb8e-4c97-aea7-567d7ee42a23\") " pod="openstack-kuttl-tests/neutron-6d64d7894b-xp9d5" Dec 03 12:46:12 crc kubenswrapper[5052]: E1203 12:46:12.647558 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-neutron-ovndbs: secret "cert-neutron-ovndbs" not found Dec 03 12:46:12 crc kubenswrapper[5052]: E1203 12:46:12.647606 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/901a2f0b-fb8e-4c97-aea7-567d7ee42a23-ovndb-tls-certs podName:901a2f0b-fb8e-4c97-aea7-567d7ee42a23 nodeName:}" failed. No retries permitted until 2025-12-03 12:46:13.147592257 +0000 UTC m=+5163.735834021 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "ovndb-tls-certs" (UniqueName: "kubernetes.io/secret/901a2f0b-fb8e-4c97-aea7-567d7ee42a23-ovndb-tls-certs") pod "neutron-6d64d7894b-xp9d5" (UID: "901a2f0b-fb8e-4c97-aea7-567d7ee42a23") : secret "cert-neutron-ovndbs" not found Dec 03 12:46:12 crc kubenswrapper[5052]: I1203 12:46:12.647656 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/901a2f0b-fb8e-4c97-aea7-567d7ee42a23-config\") pod \"neutron-6d64d7894b-xp9d5\" (UID: \"901a2f0b-fb8e-4c97-aea7-567d7ee42a23\") " pod="openstack-kuttl-tests/neutron-6d64d7894b-xp9d5" Dec 03 12:46:12 crc kubenswrapper[5052]: I1203 12:46:12.647856 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/901a2f0b-fb8e-4c97-aea7-567d7ee42a23-httpd-config\") pod \"neutron-6d64d7894b-xp9d5\" (UID: \"901a2f0b-fb8e-4c97-aea7-567d7ee42a23\") " pod="openstack-kuttl-tests/neutron-6d64d7894b-xp9d5" Dec 03 12:46:12 crc kubenswrapper[5052]: I1203 12:46:12.647918 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/901a2f0b-fb8e-4c97-aea7-567d7ee42a23-public-tls-certs\") pod \"neutron-6d64d7894b-xp9d5\" (UID: \"901a2f0b-fb8e-4c97-aea7-567d7ee42a23\") " pod="openstack-kuttl-tests/neutron-6d64d7894b-xp9d5" Dec 03 12:46:12 crc kubenswrapper[5052]: E1203 12:46:12.648075 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-neutron-public-svc: secret "cert-neutron-public-svc" not found Dec 03 12:46:12 crc kubenswrapper[5052]: E1203 12:46:12.648097 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/901a2f0b-fb8e-4c97-aea7-567d7ee42a23-public-tls-certs podName:901a2f0b-fb8e-4c97-aea7-567d7ee42a23 nodeName:}" failed. No retries permitted until 2025-12-03 12:46:13.148089672 +0000 UTC m=+5163.736331446 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "public-tls-certs" (UniqueName: "kubernetes.io/secret/901a2f0b-fb8e-4c97-aea7-567d7ee42a23-public-tls-certs") pod "neutron-6d64d7894b-xp9d5" (UID: "901a2f0b-fb8e-4c97-aea7-567d7ee42a23") : secret "cert-neutron-public-svc" not found Dec 03 12:46:12 crc kubenswrapper[5052]: I1203 12:46:12.648168 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/32f420c2-aa7e-44c8-a63a-7654aa3e6e34-config-data\") pod \"placement-bbcfbbc9d-5q6gx\" (UID: \"32f420c2-aa7e-44c8-a63a-7654aa3e6e34\") " pod="openstack-kuttl-tests/placement-bbcfbbc9d-5q6gx" Dec 03 12:46:12 crc kubenswrapper[5052]: E1203 12:46:12.648180 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-neutron-internal-svc: secret "cert-neutron-internal-svc" not found Dec 03 12:46:12 crc kubenswrapper[5052]: E1203 12:46:12.648295 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/901a2f0b-fb8e-4c97-aea7-567d7ee42a23-internal-tls-certs podName:901a2f0b-fb8e-4c97-aea7-567d7ee42a23 nodeName:}" failed. No retries permitted until 2025-12-03 12:46:13.148273007 +0000 UTC m=+5163.736514841 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "internal-tls-certs" (UniqueName: "kubernetes.io/secret/901a2f0b-fb8e-4c97-aea7-567d7ee42a23-internal-tls-certs") pod "neutron-6d64d7894b-xp9d5" (UID: "901a2f0b-fb8e-4c97-aea7-567d7ee42a23") : secret "cert-neutron-internal-svc" not found Dec 03 12:46:12 crc kubenswrapper[5052]: I1203 12:46:12.651709 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/32f420c2-aa7e-44c8-a63a-7654aa3e6e34-public-tls-certs\") pod \"placement-bbcfbbc9d-5q6gx\" (UID: \"32f420c2-aa7e-44c8-a63a-7654aa3e6e34\") " pod="openstack-kuttl-tests/placement-bbcfbbc9d-5q6gx" Dec 03 12:46:12 crc kubenswrapper[5052]: I1203 12:46:12.652590 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/901a2f0b-fb8e-4c97-aea7-567d7ee42a23-httpd-config\") pod \"neutron-6d64d7894b-xp9d5\" (UID: \"901a2f0b-fb8e-4c97-aea7-567d7ee42a23\") " pod="openstack-kuttl-tests/neutron-6d64d7894b-xp9d5" Dec 03 12:46:12 crc kubenswrapper[5052]: I1203 12:46:12.652631 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/901a2f0b-fb8e-4c97-aea7-567d7ee42a23-combined-ca-bundle\") pod \"neutron-6d64d7894b-xp9d5\" (UID: \"901a2f0b-fb8e-4c97-aea7-567d7ee42a23\") " pod="openstack-kuttl-tests/neutron-6d64d7894b-xp9d5" Dec 03 12:46:12 crc kubenswrapper[5052]: I1203 12:46:12.655462 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/901a2f0b-fb8e-4c97-aea7-567d7ee42a23-config\") pod \"neutron-6d64d7894b-xp9d5\" (UID: \"901a2f0b-fb8e-4c97-aea7-567d7ee42a23\") " pod="openstack-kuttl-tests/neutron-6d64d7894b-xp9d5" Dec 03 12:46:12 crc kubenswrapper[5052]: I1203 12:46:12.656650 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/32f420c2-aa7e-44c8-a63a-7654aa3e6e34-scripts\") pod \"placement-bbcfbbc9d-5q6gx\" (UID: \"32f420c2-aa7e-44c8-a63a-7654aa3e6e34\") " pod="openstack-kuttl-tests/placement-bbcfbbc9d-5q6gx" Dec 03 12:46:12 crc kubenswrapper[5052]: I1203 12:46:12.658437 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/32f420c2-aa7e-44c8-a63a-7654aa3e6e34-combined-ca-bundle\") pod \"placement-bbcfbbc9d-5q6gx\" (UID: \"32f420c2-aa7e-44c8-a63a-7654aa3e6e34\") " pod="openstack-kuttl-tests/placement-bbcfbbc9d-5q6gx" Dec 03 12:46:12 crc kubenswrapper[5052]: I1203 12:46:12.661077 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/c069aee2-4cb6-4c8a-bbe5-949ab0b48331-config-data-custom\") pod \"barbican-api-bcf49b784-jnkdb\" (UID: \"c069aee2-4cb6-4c8a-bbe5-949ab0b48331\") " pod="openstack-kuttl-tests/barbican-api-bcf49b784-jnkdb" Dec 03 12:46:12 crc kubenswrapper[5052]: I1203 12:46:12.662276 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c069aee2-4cb6-4c8a-bbe5-949ab0b48331-combined-ca-bundle\") pod \"barbican-api-bcf49b784-jnkdb\" (UID: \"c069aee2-4cb6-4c8a-bbe5-949ab0b48331\") " pod="openstack-kuttl-tests/barbican-api-bcf49b784-jnkdb" Dec 03 12:46:12 crc kubenswrapper[5052]: I1203 12:46:12.665429 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-q9q5k\" (UniqueName: \"kubernetes.io/projected/c069aee2-4cb6-4c8a-bbe5-949ab0b48331-kube-api-access-q9q5k\") pod \"barbican-api-bcf49b784-jnkdb\" (UID: \"c069aee2-4cb6-4c8a-bbe5-949ab0b48331\") " pod="openstack-kuttl-tests/barbican-api-bcf49b784-jnkdb" Dec 03 12:46:12 crc kubenswrapper[5052]: I1203 12:46:12.666683 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/fa1c223d-82fa-488c-ac48-e88bce36a839-scripts\") pod \"keystone-687ff6fcfc-9vwb2\" (UID: \"fa1c223d-82fa-488c-ac48-e88bce36a839\") " pod="openstack-kuttl-tests/keystone-687ff6fcfc-9vwb2" Dec 03 12:46:12 crc kubenswrapper[5052]: I1203 12:46:12.671316 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/fa1c223d-82fa-488c-ac48-e88bce36a839-fernet-keys\") pod \"keystone-687ff6fcfc-9vwb2\" (UID: \"fa1c223d-82fa-488c-ac48-e88bce36a839\") " pod="openstack-kuttl-tests/keystone-687ff6fcfc-9vwb2" Dec 03 12:46:12 crc kubenswrapper[5052]: I1203 12:46:12.672173 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/fa1c223d-82fa-488c-ac48-e88bce36a839-combined-ca-bundle\") pod \"keystone-687ff6fcfc-9vwb2\" (UID: \"fa1c223d-82fa-488c-ac48-e88bce36a839\") " pod="openstack-kuttl-tests/keystone-687ff6fcfc-9vwb2" Dec 03 12:46:12 crc kubenswrapper[5052]: I1203 12:46:12.674098 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/fa1c223d-82fa-488c-ac48-e88bce36a839-credential-keys\") pod \"keystone-687ff6fcfc-9vwb2\" (UID: \"fa1c223d-82fa-488c-ac48-e88bce36a839\") " pod="openstack-kuttl-tests/keystone-687ff6fcfc-9vwb2" Dec 03 12:46:12 crc kubenswrapper[5052]: I1203 12:46:12.675119 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-xwlbj\" (UniqueName: \"kubernetes.io/projected/32f420c2-aa7e-44c8-a63a-7654aa3e6e34-kube-api-access-xwlbj\") pod \"placement-bbcfbbc9d-5q6gx\" (UID: \"32f420c2-aa7e-44c8-a63a-7654aa3e6e34\") " pod="openstack-kuttl-tests/placement-bbcfbbc9d-5q6gx" Dec 03 12:46:12 crc kubenswrapper[5052]: I1203 12:46:12.677596 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/c069aee2-4cb6-4c8a-bbe5-949ab0b48331-config-data\") pod \"barbican-api-bcf49b784-jnkdb\" (UID: \"c069aee2-4cb6-4c8a-bbe5-949ab0b48331\") " pod="openstack-kuttl-tests/barbican-api-bcf49b784-jnkdb" Dec 03 12:46:12 crc kubenswrapper[5052]: I1203 12:46:12.686177 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/fa1c223d-82fa-488c-ac48-e88bce36a839-config-data\") pod \"keystone-687ff6fcfc-9vwb2\" (UID: \"fa1c223d-82fa-488c-ac48-e88bce36a839\") " pod="openstack-kuttl-tests/keystone-687ff6fcfc-9vwb2" Dec 03 12:46:12 crc kubenswrapper[5052]: I1203 12:46:12.693539 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-bsn6g\" (UniqueName: \"kubernetes.io/projected/fa1c223d-82fa-488c-ac48-e88bce36a839-kube-api-access-bsn6g\") pod \"keystone-687ff6fcfc-9vwb2\" (UID: \"fa1c223d-82fa-488c-ac48-e88bce36a839\") " pod="openstack-kuttl-tests/keystone-687ff6fcfc-9vwb2" Dec 03 12:46:12 crc kubenswrapper[5052]: I1203 12:46:12.707985 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ovsdbserver-sb-0" podUID="f2d7e3bb-2829-4166-90df-53f9a50272eb" containerName="ovsdbserver-sb" containerID="cri-o://23cc31c39f29b7dd83db1992297f60565ae8b315ab364419ad6f8475dec1526c" gracePeriod=299 Dec 03 12:46:12 crc kubenswrapper[5052]: I1203 12:46:12.713315 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-mwcgt\" (UniqueName: \"kubernetes.io/projected/901a2f0b-fb8e-4c97-aea7-567d7ee42a23-kube-api-access-mwcgt\") pod \"neutron-6d64d7894b-xp9d5\" (UID: \"901a2f0b-fb8e-4c97-aea7-567d7ee42a23\") " pod="openstack-kuttl-tests/neutron-6d64d7894b-xp9d5" Dec 03 12:46:12 crc kubenswrapper[5052]: I1203 12:46:12.751651 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/c069aee2-4cb6-4c8a-bbe5-949ab0b48331-public-tls-certs\") pod \"barbican-api-bcf49b784-jnkdb\" (UID: \"c069aee2-4cb6-4c8a-bbe5-949ab0b48331\") " pod="openstack-kuttl-tests/barbican-api-bcf49b784-jnkdb" Dec 03 12:46:12 crc kubenswrapper[5052]: I1203 12:46:12.751864 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/c069aee2-4cb6-4c8a-bbe5-949ab0b48331-internal-tls-certs\") pod \"barbican-api-bcf49b784-jnkdb\" (UID: \"c069aee2-4cb6-4c8a-bbe5-949ab0b48331\") " pod="openstack-kuttl-tests/barbican-api-bcf49b784-jnkdb" Dec 03 12:46:12 crc kubenswrapper[5052]: E1203 12:46:12.753974 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-barbican-internal-svc: secret "cert-barbican-internal-svc" not found Dec 03 12:46:12 crc kubenswrapper[5052]: E1203 12:46:12.754031 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c069aee2-4cb6-4c8a-bbe5-949ab0b48331-internal-tls-certs podName:c069aee2-4cb6-4c8a-bbe5-949ab0b48331 nodeName:}" failed. No retries permitted until 2025-12-03 12:46:13.754015941 +0000 UTC m=+5164.342257705 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "internal-tls-certs" (UniqueName: "kubernetes.io/secret/c069aee2-4cb6-4c8a-bbe5-949ab0b48331-internal-tls-certs") pod "barbican-api-bcf49b784-jnkdb" (UID: "c069aee2-4cb6-4c8a-bbe5-949ab0b48331") : secret "cert-barbican-internal-svc" not found Dec 03 12:46:12 crc kubenswrapper[5052]: E1203 12:46:12.754483 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-barbican-public-svc: secret "cert-barbican-public-svc" not found Dec 03 12:46:12 crc kubenswrapper[5052]: E1203 12:46:12.754526 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c069aee2-4cb6-4c8a-bbe5-949ab0b48331-public-tls-certs podName:c069aee2-4cb6-4c8a-bbe5-949ab0b48331 nodeName:}" failed. No retries permitted until 2025-12-03 12:46:13.754503215 +0000 UTC m=+5164.342744979 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "public-tls-certs" (UniqueName: "kubernetes.io/secret/c069aee2-4cb6-4c8a-bbe5-949ab0b48331-public-tls-certs") pod "barbican-api-bcf49b784-jnkdb" (UID: "c069aee2-4cb6-4c8a-bbe5-949ab0b48331") : secret "cert-barbican-public-svc" not found Dec 03 12:46:12 crc kubenswrapper[5052]: I1203 12:46:12.863788 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ovsdbserver-nb-0" podUID="ee05fe95-7cc0-4225-a86b-1682adc78b8b" containerName="ovsdbserver-nb" containerID="cri-o://d12594c4d36cf7fcca02723f8cb056b7e65d76a4582dd775c6dc70d741593906" gracePeriod=300 Dec 03 12:46:12 crc kubenswrapper[5052]: I1203 12:46:12.914248 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-bbcfbbc9d-5q6gx" Dec 03 12:46:12 crc kubenswrapper[5052]: I1203 12:46:12.964841 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/fa1c223d-82fa-488c-ac48-e88bce36a839-public-tls-certs\") pod \"keystone-687ff6fcfc-9vwb2\" (UID: \"fa1c223d-82fa-488c-ac48-e88bce36a839\") " pod="openstack-kuttl-tests/keystone-687ff6fcfc-9vwb2" Dec 03 12:46:12 crc kubenswrapper[5052]: I1203 12:46:12.965244 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/fa1c223d-82fa-488c-ac48-e88bce36a839-internal-tls-certs\") pod \"keystone-687ff6fcfc-9vwb2\" (UID: \"fa1c223d-82fa-488c-ac48-e88bce36a839\") " pod="openstack-kuttl-tests/keystone-687ff6fcfc-9vwb2" Dec 03 12:46:12 crc kubenswrapper[5052]: E1203 12:46:12.965056 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-keystone-public-svc: secret "cert-keystone-public-svc" not found Dec 03 12:46:12 crc kubenswrapper[5052]: E1203 12:46:12.965624 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-keystone-internal-svc: secret "cert-keystone-internal-svc" not found Dec 03 12:46:12 crc kubenswrapper[5052]: E1203 12:46:12.965691 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fa1c223d-82fa-488c-ac48-e88bce36a839-public-tls-certs podName:fa1c223d-82fa-488c-ac48-e88bce36a839 nodeName:}" failed. No retries permitted until 2025-12-03 12:46:13.965658004 +0000 UTC m=+5164.553899768 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "public-tls-certs" (UniqueName: "kubernetes.io/secret/fa1c223d-82fa-488c-ac48-e88bce36a839-public-tls-certs") pod "keystone-687ff6fcfc-9vwb2" (UID: "fa1c223d-82fa-488c-ac48-e88bce36a839") : secret "cert-keystone-public-svc" not found Dec 03 12:46:12 crc kubenswrapper[5052]: E1203 12:46:12.965716 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fa1c223d-82fa-488c-ac48-e88bce36a839-internal-tls-certs podName:fa1c223d-82fa-488c-ac48-e88bce36a839 nodeName:}" failed. No retries permitted until 2025-12-03 12:46:13.965705255 +0000 UTC m=+5164.553947019 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "internal-tls-certs" (UniqueName: "kubernetes.io/secret/fa1c223d-82fa-488c-ac48-e88bce36a839-internal-tls-certs") pod "keystone-687ff6fcfc-9vwb2" (UID: "fa1c223d-82fa-488c-ac48-e88bce36a839") : secret "cert-keystone-internal-svc" not found Dec 03 12:46:13 crc kubenswrapper[5052]: I1203 12:46:13.053054 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/openstack-galera-0" podUID="8fa85200-471e-4fba-9884-ef25328ebf64" containerName="galera" containerID="cri-o://ea0e318bc1c2d00f8c07df735c18136c77c564a4ca1cc7bf9d3e0dba21c6f865" gracePeriod=29 Dec 03 12:46:13 crc kubenswrapper[5052]: I1203 12:46:13.118836 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/openstack-cell1-galera-0" podUID="2c53ea4e-56ba-4ba1-8cb6-8c24725b5856" containerName="galera" containerID="cri-o://fe1395b62b90460d04b6099206e52d20adacc07cc857b25d7b625ceac277317a" gracePeriod=30 Dec 03 12:46:13 crc kubenswrapper[5052]: I1203 12:46:13.163246 5052 generic.go:334] "Generic (PLEG): container finished" podID="9aea20a7-eaed-43a8-9b52-08c2c0212230" containerID="c5ed1d5bcb2e814a4e6ff6889daaa85dec6c18fe852411af7cfacd20fd4d3320" exitCode=143 Dec 03 12:46:13 crc kubenswrapper[5052]: I1203 12:46:13.163338 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"9aea20a7-eaed-43a8-9b52-08c2c0212230","Type":"ContainerDied","Data":"c5ed1d5bcb2e814a4e6ff6889daaa85dec6c18fe852411af7cfacd20fd4d3320"} Dec 03 12:46:13 crc kubenswrapper[5052]: I1203 12:46:13.166116 5052 generic.go:334] "Generic (PLEG): container finished" podID="78361066-6add-4971-9a4b-9157f35ad494" containerID="6fe0649b6ce8be2bfd99aed0e4830c4017fd592947629aa86d78326653fa7af0" exitCode=143 Dec 03 12:46:13 crc kubenswrapper[5052]: I1203 12:46:13.166161 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"78361066-6add-4971-9a4b-9157f35ad494","Type":"ContainerDied","Data":"6fe0649b6ce8be2bfd99aed0e4830c4017fd592947629aa86d78326653fa7af0"} Dec 03 12:46:13 crc kubenswrapper[5052]: I1203 12:46:13.177727 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/901a2f0b-fb8e-4c97-aea7-567d7ee42a23-internal-tls-certs\") pod \"neutron-6d64d7894b-xp9d5\" (UID: \"901a2f0b-fb8e-4c97-aea7-567d7ee42a23\") " pod="openstack-kuttl-tests/neutron-6d64d7894b-xp9d5" Dec 03 12:46:13 crc kubenswrapper[5052]: I1203 12:46:13.177906 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/901a2f0b-fb8e-4c97-aea7-567d7ee42a23-public-tls-certs\") pod \"neutron-6d64d7894b-xp9d5\" (UID: \"901a2f0b-fb8e-4c97-aea7-567d7ee42a23\") " pod="openstack-kuttl-tests/neutron-6d64d7894b-xp9d5" Dec 03 12:46:13 crc kubenswrapper[5052]: I1203 12:46:13.177970 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/901a2f0b-fb8e-4c97-aea7-567d7ee42a23-ovndb-tls-certs\") pod \"neutron-6d64d7894b-xp9d5\" (UID: \"901a2f0b-fb8e-4c97-aea7-567d7ee42a23\") " pod="openstack-kuttl-tests/neutron-6d64d7894b-xp9d5" Dec 03 12:46:13 crc kubenswrapper[5052]: E1203 12:46:13.178126 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-neutron-ovndbs: secret "cert-neutron-ovndbs" not found Dec 03 12:46:13 crc kubenswrapper[5052]: E1203 12:46:13.178187 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/901a2f0b-fb8e-4c97-aea7-567d7ee42a23-ovndb-tls-certs podName:901a2f0b-fb8e-4c97-aea7-567d7ee42a23 nodeName:}" failed. No retries permitted until 2025-12-03 12:46:14.178167811 +0000 UTC m=+5164.766409575 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "ovndb-tls-certs" (UniqueName: "kubernetes.io/secret/901a2f0b-fb8e-4c97-aea7-567d7ee42a23-ovndb-tls-certs") pod "neutron-6d64d7894b-xp9d5" (UID: "901a2f0b-fb8e-4c97-aea7-567d7ee42a23") : secret "cert-neutron-ovndbs" not found Dec 03 12:46:13 crc kubenswrapper[5052]: E1203 12:46:13.178598 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-neutron-public-svc: secret "cert-neutron-public-svc" not found Dec 03 12:46:13 crc kubenswrapper[5052]: E1203 12:46:13.178641 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/901a2f0b-fb8e-4c97-aea7-567d7ee42a23-public-tls-certs podName:901a2f0b-fb8e-4c97-aea7-567d7ee42a23 nodeName:}" failed. No retries permitted until 2025-12-03 12:46:14.178629444 +0000 UTC m=+5164.766871208 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "public-tls-certs" (UniqueName: "kubernetes.io/secret/901a2f0b-fb8e-4c97-aea7-567d7ee42a23-public-tls-certs") pod "neutron-6d64d7894b-xp9d5" (UID: "901a2f0b-fb8e-4c97-aea7-567d7ee42a23") : secret "cert-neutron-public-svc" not found Dec 03 12:46:13 crc kubenswrapper[5052]: E1203 12:46:13.178714 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-neutron-internal-svc: secret "cert-neutron-internal-svc" not found Dec 03 12:46:13 crc kubenswrapper[5052]: E1203 12:46:13.178746 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/901a2f0b-fb8e-4c97-aea7-567d7ee42a23-internal-tls-certs podName:901a2f0b-fb8e-4c97-aea7-567d7ee42a23 nodeName:}" failed. No retries permitted until 2025-12-03 12:46:14.178737987 +0000 UTC m=+5164.766979751 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "internal-tls-certs" (UniqueName: "kubernetes.io/secret/901a2f0b-fb8e-4c97-aea7-567d7ee42a23-internal-tls-certs") pod "neutron-6d64d7894b-xp9d5" (UID: "901a2f0b-fb8e-4c97-aea7-567d7ee42a23") : secret "cert-neutron-internal-svc" not found Dec 03 12:46:13 crc kubenswrapper[5052]: E1203 12:46:13.218182 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="bbc2755b5351566cbdd1dc7fd68849cb7ea314f303defbebf723032464036311" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:46:13 crc kubenswrapper[5052]: I1203 12:46:13.225220 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_ovsdbserver-sb-0_f2d7e3bb-2829-4166-90df-53f9a50272eb/ovsdbserver-sb/0.log" Dec 03 12:46:13 crc kubenswrapper[5052]: I1203 12:46:13.225274 5052 generic.go:334] "Generic (PLEG): container finished" podID="f2d7e3bb-2829-4166-90df-53f9a50272eb" containerID="1499c2af6931030cde94894361cbf72f702626308576bf5ee6e7760bbc5034df" exitCode=2 Dec 03 12:46:13 crc kubenswrapper[5052]: I1203 12:46:13.225289 5052 generic.go:334] "Generic (PLEG): container finished" podID="f2d7e3bb-2829-4166-90df-53f9a50272eb" containerID="23cc31c39f29b7dd83db1992297f60565ae8b315ab364419ad6f8475dec1526c" exitCode=143 Dec 03 12:46:13 crc kubenswrapper[5052]: I1203 12:46:13.225350 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-sb-0" event={"ID":"f2d7e3bb-2829-4166-90df-53f9a50272eb","Type":"ContainerDied","Data":"1499c2af6931030cde94894361cbf72f702626308576bf5ee6e7760bbc5034df"} Dec 03 12:46:13 crc kubenswrapper[5052]: I1203 12:46:13.225378 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-sb-0" event={"ID":"f2d7e3bb-2829-4166-90df-53f9a50272eb","Type":"ContainerDied","Data":"23cc31c39f29b7dd83db1992297f60565ae8b315ab364419ad6f8475dec1526c"} Dec 03 12:46:13 crc kubenswrapper[5052]: E1203 12:46:13.260290 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="bbc2755b5351566cbdd1dc7fd68849cb7ea314f303defbebf723032464036311" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:46:13 crc kubenswrapper[5052]: I1203 12:46:13.261392 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_ovsdbserver-nb-0_ee05fe95-7cc0-4225-a86b-1682adc78b8b/ovsdbserver-nb/0.log" Dec 03 12:46:13 crc kubenswrapper[5052]: I1203 12:46:13.261447 5052 generic.go:334] "Generic (PLEG): container finished" podID="ee05fe95-7cc0-4225-a86b-1682adc78b8b" containerID="61f7b1adc6b8d50af5982e338e056fb28d2ea6d8f6e0b8c7092747bfc78b185f" exitCode=2 Dec 03 12:46:13 crc kubenswrapper[5052]: I1203 12:46:13.261490 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-nb-0" event={"ID":"ee05fe95-7cc0-4225-a86b-1682adc78b8b","Type":"ContainerDied","Data":"61f7b1adc6b8d50af5982e338e056fb28d2ea6d8f6e0b8c7092747bfc78b185f"} Dec 03 12:46:13 crc kubenswrapper[5052]: E1203 12:46:13.262267 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="bbc2755b5351566cbdd1dc7fd68849cb7ea314f303defbebf723032464036311" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:46:13 crc kubenswrapper[5052]: E1203 12:46:13.262310 5052 prober.go:104] "Probe errored" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" probeType="Readiness" pod="openstack-kuttl-tests/nova-cell0-conductor-0" podUID="f7e26e55-1426-4cd4-b169-a63868acdfaa" containerName="nova-cell0-conductor-conductor" Dec 03 12:46:13 crc kubenswrapper[5052]: I1203 12:46:13.283145 5052 generic.go:334] "Generic (PLEG): container finished" podID="68a80c17-9f7d-47de-b02c-5e7bb47656ab" containerID="8562f7f2637fa32043b41c559e46c0266da41a1cab62e80ce7b44cb981784aae" exitCode=143 Dec 03 12:46:13 crc kubenswrapper[5052]: I1203 12:46:13.283206 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"68a80c17-9f7d-47de-b02c-5e7bb47656ab","Type":"ContainerDied","Data":"8562f7f2637fa32043b41c559e46c0266da41a1cab62e80ce7b44cb981784aae"} Dec 03 12:46:13 crc kubenswrapper[5052]: I1203 12:46:13.287106 5052 generic.go:334] "Generic (PLEG): container finished" podID="fb6c11ac-1b0e-4e63-8988-256b2504ebc7" containerID="d57a2057a1ffffdae42cdf5eae9f0f819cc86143c1132d66146d4814f2b2355f" exitCode=143 Dec 03 12:46:13 crc kubenswrapper[5052]: I1203 12:46:13.287155 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"fb6c11ac-1b0e-4e63-8988-256b2504ebc7","Type":"ContainerDied","Data":"d57a2057a1ffffdae42cdf5eae9f0f819cc86143c1132d66146d4814f2b2355f"} Dec 03 12:46:13 crc kubenswrapper[5052]: I1203 12:46:13.289527 5052 generic.go:334] "Generic (PLEG): container finished" podID="e5f1bd78-3552-4d2d-8029-651f33c3918a" containerID="0c534558ed9f09feb4924d04acb90157e0fc3201f00732c7364a3bf82d62e52e" exitCode=0 Dec 03 12:46:13 crc kubenswrapper[5052]: I1203 12:46:13.289577 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" event={"ID":"e5f1bd78-3552-4d2d-8029-651f33c3918a","Type":"ContainerDied","Data":"0c534558ed9f09feb4924d04acb90157e0fc3201f00732c7364a3bf82d62e52e"} Dec 03 12:46:13 crc kubenswrapper[5052]: I1203 12:46:13.291666 5052 generic.go:334] "Generic (PLEG): container finished" podID="9af09a5b-789c-4b83-b899-9d1b3f0d8603" containerID="deb3641e5c7641ec59de2756335b3b8af86a98493025f1061ab9587775677d7d" exitCode=143 Dec 03 12:46:13 crc kubenswrapper[5052]: I1203 12:46:13.291694 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"9af09a5b-789c-4b83-b899-9d1b3f0d8603","Type":"ContainerDied","Data":"deb3641e5c7641ec59de2756335b3b8af86a98493025f1061ab9587775677d7d"} Dec 03 12:46:13 crc kubenswrapper[5052]: I1203 12:46:13.421335 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-keystone-listener-68fd6d997d-7tjjw"] Dec 03 12:46:13 crc kubenswrapper[5052]: I1203 12:46:13.680411 5052 scope.go:117] "RemoveContainer" containerID="2aed1c760a326f3d008799e902b1093313df3764a57c879f0b3686410127d37d" Dec 03 12:46:13 crc kubenswrapper[5052]: E1203 12:46:13.681020 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 12:46:13 crc kubenswrapper[5052]: I1203 12:46:13.810256 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/c069aee2-4cb6-4c8a-bbe5-949ab0b48331-public-tls-certs\") pod \"barbican-api-bcf49b784-jnkdb\" (UID: \"c069aee2-4cb6-4c8a-bbe5-949ab0b48331\") " pod="openstack-kuttl-tests/barbican-api-bcf49b784-jnkdb" Dec 03 12:46:13 crc kubenswrapper[5052]: I1203 12:46:13.810400 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/c069aee2-4cb6-4c8a-bbe5-949ab0b48331-internal-tls-certs\") pod \"barbican-api-bcf49b784-jnkdb\" (UID: \"c069aee2-4cb6-4c8a-bbe5-949ab0b48331\") " pod="openstack-kuttl-tests/barbican-api-bcf49b784-jnkdb" Dec 03 12:46:13 crc kubenswrapper[5052]: E1203 12:46:13.810984 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-barbican-public-svc: secret "cert-barbican-public-svc" not found Dec 03 12:46:13 crc kubenswrapper[5052]: E1203 12:46:13.811053 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c069aee2-4cb6-4c8a-bbe5-949ab0b48331-public-tls-certs podName:c069aee2-4cb6-4c8a-bbe5-949ab0b48331 nodeName:}" failed. No retries permitted until 2025-12-03 12:46:15.81103242 +0000 UTC m=+5166.399274264 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "public-tls-certs" (UniqueName: "kubernetes.io/secret/c069aee2-4cb6-4c8a-bbe5-949ab0b48331-public-tls-certs") pod "barbican-api-bcf49b784-jnkdb" (UID: "c069aee2-4cb6-4c8a-bbe5-949ab0b48331") : secret "cert-barbican-public-svc" not found Dec 03 12:46:13 crc kubenswrapper[5052]: E1203 12:46:13.811764 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-barbican-internal-svc: secret "cert-barbican-internal-svc" not found Dec 03 12:46:13 crc kubenswrapper[5052]: E1203 12:46:13.811821 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c069aee2-4cb6-4c8a-bbe5-949ab0b48331-internal-tls-certs podName:c069aee2-4cb6-4c8a-bbe5-949ab0b48331 nodeName:}" failed. No retries permitted until 2025-12-03 12:46:15.811802442 +0000 UTC m=+5166.400044286 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "internal-tls-certs" (UniqueName: "kubernetes.io/secret/c069aee2-4cb6-4c8a-bbe5-949ab0b48331-internal-tls-certs") pod "barbican-api-bcf49b784-jnkdb" (UID: "c069aee2-4cb6-4c8a-bbe5-949ab0b48331") : secret "cert-barbican-internal-svc" not found Dec 03 12:46:13 crc kubenswrapper[5052]: I1203 12:46:13.982722 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:46:13 crc kubenswrapper[5052]: I1203 12:46:13.983179 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="4eab5b48-f674-4100-9b42-f3ea257d47d4" containerName="ceilometer-central-agent" containerID="cri-o://c9ba5984640e4b46f378e76e1ad9dbbc5b996fff2d3c392ba7de3ff6753a48fd" gracePeriod=30 Dec 03 12:46:13 crc kubenswrapper[5052]: I1203 12:46:13.983803 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="4eab5b48-f674-4100-9b42-f3ea257d47d4" containerName="proxy-httpd" containerID="cri-o://01c4f4342abe4df4b3d520a97ffb3f4ba0abfe85cb81b459e64f94869c4a5f91" gracePeriod=30 Dec 03 12:46:13 crc kubenswrapper[5052]: I1203 12:46:13.983868 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="4eab5b48-f674-4100-9b42-f3ea257d47d4" containerName="sg-core" containerID="cri-o://7370547fbe4eb4455656b595065ce614afd49fa9ea95ccde635adb0b612d3c70" gracePeriod=30 Dec 03 12:46:13 crc kubenswrapper[5052]: I1203 12:46:13.983945 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="4eab5b48-f674-4100-9b42-f3ea257d47d4" containerName="ceilometer-notification-agent" containerID="cri-o://d0d21a72bcbb3c194d8e900bc8615778059b26e70f1614cda87f183610dac51e" gracePeriod=30 Dec 03 12:46:13 crc kubenswrapper[5052]: I1203 12:46:13.994482 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_ovsdbserver-sb-0_f2d7e3bb-2829-4166-90df-53f9a50272eb/ovsdbserver-sb/0.log" Dec 03 12:46:13 crc kubenswrapper[5052]: I1203 12:46:13.994577 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:46:14 crc kubenswrapper[5052]: I1203 12:46:14.016536 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_ovsdbserver-nb-0_ee05fe95-7cc0-4225-a86b-1682adc78b8b/ovsdbserver-nb/0.log" Dec 03 12:46:14 crc kubenswrapper[5052]: I1203 12:46:14.016615 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:46:14 crc kubenswrapper[5052]: E1203 12:46:14.019759 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-keystone-public-svc: secret "cert-keystone-public-svc" not found Dec 03 12:46:14 crc kubenswrapper[5052]: E1203 12:46:14.019852 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fa1c223d-82fa-488c-ac48-e88bce36a839-public-tls-certs podName:fa1c223d-82fa-488c-ac48-e88bce36a839 nodeName:}" failed. No retries permitted until 2025-12-03 12:46:16.019828911 +0000 UTC m=+5166.608070735 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "public-tls-certs" (UniqueName: "kubernetes.io/secret/fa1c223d-82fa-488c-ac48-e88bce36a839-public-tls-certs") pod "keystone-687ff6fcfc-9vwb2" (UID: "fa1c223d-82fa-488c-ac48-e88bce36a839") : secret "cert-keystone-public-svc" not found Dec 03 12:46:14 crc kubenswrapper[5052]: I1203 12:46:14.021455 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:46:14 crc kubenswrapper[5052]: I1203 12:46:14.030629 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/fa1c223d-82fa-488c-ac48-e88bce36a839-public-tls-certs\") pod \"keystone-687ff6fcfc-9vwb2\" (UID: \"fa1c223d-82fa-488c-ac48-e88bce36a839\") " pod="openstack-kuttl-tests/keystone-687ff6fcfc-9vwb2" Dec 03 12:46:14 crc kubenswrapper[5052]: I1203 12:46:14.030877 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/fa1c223d-82fa-488c-ac48-e88bce36a839-internal-tls-certs\") pod \"keystone-687ff6fcfc-9vwb2\" (UID: \"fa1c223d-82fa-488c-ac48-e88bce36a839\") " pod="openstack-kuttl-tests/keystone-687ff6fcfc-9vwb2" Dec 03 12:46:14 crc kubenswrapper[5052]: E1203 12:46:14.031336 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-keystone-internal-svc: secret "cert-keystone-internal-svc" not found Dec 03 12:46:14 crc kubenswrapper[5052]: E1203 12:46:14.031396 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fa1c223d-82fa-488c-ac48-e88bce36a839-internal-tls-certs podName:fa1c223d-82fa-488c-ac48-e88bce36a839 nodeName:}" failed. No retries permitted until 2025-12-03 12:46:16.031379692 +0000 UTC m=+5166.619621456 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "internal-tls-certs" (UniqueName: "kubernetes.io/secret/fa1c223d-82fa-488c-ac48-e88bce36a839-internal-tls-certs") pod "keystone-687ff6fcfc-9vwb2" (UID: "fa1c223d-82fa-488c-ac48-e88bce36a839") : secret "cert-keystone-internal-svc" not found Dec 03 12:46:14 crc kubenswrapper[5052]: I1203 12:46:14.132365 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/e5f1bd78-3552-4d2d-8029-651f33c3918a-config-data\") pod \"e5f1bd78-3552-4d2d-8029-651f33c3918a\" (UID: \"e5f1bd78-3552-4d2d-8029-651f33c3918a\") " Dec 03 12:46:14 crc kubenswrapper[5052]: I1203 12:46:14.132727 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovndbcluster-nb-etc-ovn\" (UniqueName: \"kubernetes.io/local-volume/local-storage01-crc\") pod \"ee05fe95-7cc0-4225-a86b-1682adc78b8b\" (UID: \"ee05fe95-7cc0-4225-a86b-1682adc78b8b\") " Dec 03 12:46:14 crc kubenswrapper[5052]: I1203 12:46:14.132806 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ee05fe95-7cc0-4225-a86b-1682adc78b8b-config\") pod \"ee05fe95-7cc0-4225-a86b-1682adc78b8b\" (UID: \"ee05fe95-7cc0-4225-a86b-1682adc78b8b\") " Dec 03 12:46:14 crc kubenswrapper[5052]: I1203 12:46:14.132836 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/f2d7e3bb-2829-4166-90df-53f9a50272eb-ovsdb-rundir\") pod \"f2d7e3bb-2829-4166-90df-53f9a50272eb\" (UID: \"f2d7e3bb-2829-4166-90df-53f9a50272eb\") " Dec 03 12:46:14 crc kubenswrapper[5052]: I1203 12:46:14.132875 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"nova-novncproxy-tls-certs\" (UniqueName: \"kubernetes.io/secret/e5f1bd78-3552-4d2d-8029-651f33c3918a-nova-novncproxy-tls-certs\") pod \"e5f1bd78-3552-4d2d-8029-651f33c3918a\" (UID: \"e5f1bd78-3552-4d2d-8029-651f33c3918a\") " Dec 03 12:46:14 crc kubenswrapper[5052]: I1203 12:46:14.132905 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5jc57\" (UniqueName: \"kubernetes.io/projected/f2d7e3bb-2829-4166-90df-53f9a50272eb-kube-api-access-5jc57\") pod \"f2d7e3bb-2829-4166-90df-53f9a50272eb\" (UID: \"f2d7e3bb-2829-4166-90df-53f9a50272eb\") " Dec 03 12:46:14 crc kubenswrapper[5052]: I1203 12:46:14.132927 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovsdbserver-nb-tls-certs\" (UniqueName: \"kubernetes.io/secret/ee05fe95-7cc0-4225-a86b-1682adc78b8b-ovsdbserver-nb-tls-certs\") pod \"ee05fe95-7cc0-4225-a86b-1682adc78b8b\" (UID: \"ee05fe95-7cc0-4225-a86b-1682adc78b8b\") " Dec 03 12:46:14 crc kubenswrapper[5052]: I1203 12:46:14.132951 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9wz5q\" (UniqueName: \"kubernetes.io/projected/ee05fe95-7cc0-4225-a86b-1682adc78b8b-kube-api-access-9wz5q\") pod \"ee05fe95-7cc0-4225-a86b-1682adc78b8b\" (UID: \"ee05fe95-7cc0-4225-a86b-1682adc78b8b\") " Dec 03 12:46:14 crc kubenswrapper[5052]: I1203 12:46:14.132972 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/ee05fe95-7cc0-4225-a86b-1682adc78b8b-ovsdb-rundir\") pod \"ee05fe95-7cc0-4225-a86b-1682adc78b8b\" (UID: \"ee05fe95-7cc0-4225-a86b-1682adc78b8b\") " Dec 03 12:46:14 crc kubenswrapper[5052]: I1203 12:46:14.133017 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ee05fe95-7cc0-4225-a86b-1682adc78b8b-combined-ca-bundle\") pod \"ee05fe95-7cc0-4225-a86b-1682adc78b8b\" (UID: \"ee05fe95-7cc0-4225-a86b-1682adc78b8b\") " Dec 03 12:46:14 crc kubenswrapper[5052]: I1203 12:46:14.133058 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f2d7e3bb-2829-4166-90df-53f9a50272eb-config\") pod \"f2d7e3bb-2829-4166-90df-53f9a50272eb\" (UID: \"f2d7e3bb-2829-4166-90df-53f9a50272eb\") " Dec 03 12:46:14 crc kubenswrapper[5052]: I1203 12:46:14.133108 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/ee05fe95-7cc0-4225-a86b-1682adc78b8b-metrics-certs-tls-certs\") pod \"ee05fe95-7cc0-4225-a86b-1682adc78b8b\" (UID: \"ee05fe95-7cc0-4225-a86b-1682adc78b8b\") " Dec 03 12:46:14 crc kubenswrapper[5052]: I1203 12:46:14.133134 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovndbcluster-sb-etc-ovn\" (UniqueName: \"kubernetes.io/local-volume/local-storage09-crc\") pod \"f2d7e3bb-2829-4166-90df-53f9a50272eb\" (UID: \"f2d7e3bb-2829-4166-90df-53f9a50272eb\") " Dec 03 12:46:14 crc kubenswrapper[5052]: I1203 12:46:14.133159 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/f2d7e3bb-2829-4166-90df-53f9a50272eb-scripts\") pod \"f2d7e3bb-2829-4166-90df-53f9a50272eb\" (UID: \"f2d7e3bb-2829-4166-90df-53f9a50272eb\") " Dec 03 12:46:14 crc kubenswrapper[5052]: I1203 12:46:14.133184 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/ee05fe95-7cc0-4225-a86b-1682adc78b8b-scripts\") pod \"ee05fe95-7cc0-4225-a86b-1682adc78b8b\" (UID: \"ee05fe95-7cc0-4225-a86b-1682adc78b8b\") " Dec 03 12:46:14 crc kubenswrapper[5052]: I1203 12:46:14.133226 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovsdbserver-sb-tls-certs\" (UniqueName: \"kubernetes.io/secret/f2d7e3bb-2829-4166-90df-53f9a50272eb-ovsdbserver-sb-tls-certs\") pod \"f2d7e3bb-2829-4166-90df-53f9a50272eb\" (UID: \"f2d7e3bb-2829-4166-90df-53f9a50272eb\") " Dec 03 12:46:14 crc kubenswrapper[5052]: I1203 12:46:14.133251 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"vencrypt-tls-certs\" (UniqueName: \"kubernetes.io/secret/e5f1bd78-3552-4d2d-8029-651f33c3918a-vencrypt-tls-certs\") pod \"e5f1bd78-3552-4d2d-8029-651f33c3918a\" (UID: \"e5f1bd78-3552-4d2d-8029-651f33c3918a\") " Dec 03 12:46:14 crc kubenswrapper[5052]: I1203 12:46:14.133300 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pjstv\" (UniqueName: \"kubernetes.io/projected/e5f1bd78-3552-4d2d-8029-651f33c3918a-kube-api-access-pjstv\") pod \"e5f1bd78-3552-4d2d-8029-651f33c3918a\" (UID: \"e5f1bd78-3552-4d2d-8029-651f33c3918a\") " Dec 03 12:46:14 crc kubenswrapper[5052]: I1203 12:46:14.133324 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e5f1bd78-3552-4d2d-8029-651f33c3918a-combined-ca-bundle\") pod \"e5f1bd78-3552-4d2d-8029-651f33c3918a\" (UID: \"e5f1bd78-3552-4d2d-8029-651f33c3918a\") " Dec 03 12:46:14 crc kubenswrapper[5052]: I1203 12:46:14.133347 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f2d7e3bb-2829-4166-90df-53f9a50272eb-combined-ca-bundle\") pod \"f2d7e3bb-2829-4166-90df-53f9a50272eb\" (UID: \"f2d7e3bb-2829-4166-90df-53f9a50272eb\") " Dec 03 12:46:14 crc kubenswrapper[5052]: I1203 12:46:14.133376 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/f2d7e3bb-2829-4166-90df-53f9a50272eb-metrics-certs-tls-certs\") pod \"f2d7e3bb-2829-4166-90df-53f9a50272eb\" (UID: \"f2d7e3bb-2829-4166-90df-53f9a50272eb\") " Dec 03 12:46:14 crc kubenswrapper[5052]: I1203 12:46:14.140259 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f2d7e3bb-2829-4166-90df-53f9a50272eb-ovsdb-rundir" (OuterVolumeSpecName: "ovsdb-rundir") pod "f2d7e3bb-2829-4166-90df-53f9a50272eb" (UID: "f2d7e3bb-2829-4166-90df-53f9a50272eb"). InnerVolumeSpecName "ovsdb-rundir". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:46:14 crc kubenswrapper[5052]: I1203 12:46:14.140602 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ee05fe95-7cc0-4225-a86b-1682adc78b8b-ovsdb-rundir" (OuterVolumeSpecName: "ovsdb-rundir") pod "ee05fe95-7cc0-4225-a86b-1682adc78b8b" (UID: "ee05fe95-7cc0-4225-a86b-1682adc78b8b"). InnerVolumeSpecName "ovsdb-rundir". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:46:14 crc kubenswrapper[5052]: I1203 12:46:14.141088 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ee05fe95-7cc0-4225-a86b-1682adc78b8b-scripts" (OuterVolumeSpecName: "scripts") pod "ee05fe95-7cc0-4225-a86b-1682adc78b8b" (UID: "ee05fe95-7cc0-4225-a86b-1682adc78b8b"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:46:14 crc kubenswrapper[5052]: I1203 12:46:14.141616 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f2d7e3bb-2829-4166-90df-53f9a50272eb-config" (OuterVolumeSpecName: "config") pod "f2d7e3bb-2829-4166-90df-53f9a50272eb" (UID: "f2d7e3bb-2829-4166-90df-53f9a50272eb"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:46:14 crc kubenswrapper[5052]: I1203 12:46:14.143260 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ee05fe95-7cc0-4225-a86b-1682adc78b8b-config" (OuterVolumeSpecName: "config") pod "ee05fe95-7cc0-4225-a86b-1682adc78b8b" (UID: "ee05fe95-7cc0-4225-a86b-1682adc78b8b"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:46:14 crc kubenswrapper[5052]: I1203 12:46:14.144946 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f2d7e3bb-2829-4166-90df-53f9a50272eb-scripts" (OuterVolumeSpecName: "scripts") pod "f2d7e3bb-2829-4166-90df-53f9a50272eb" (UID: "f2d7e3bb-2829-4166-90df-53f9a50272eb"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:46:14 crc kubenswrapper[5052]: I1203 12:46:14.245869 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/901a2f0b-fb8e-4c97-aea7-567d7ee42a23-public-tls-certs\") pod \"neutron-6d64d7894b-xp9d5\" (UID: \"901a2f0b-fb8e-4c97-aea7-567d7ee42a23\") " pod="openstack-kuttl-tests/neutron-6d64d7894b-xp9d5" Dec 03 12:46:14 crc kubenswrapper[5052]: I1203 12:46:14.245962 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/901a2f0b-fb8e-4c97-aea7-567d7ee42a23-ovndb-tls-certs\") pod \"neutron-6d64d7894b-xp9d5\" (UID: \"901a2f0b-fb8e-4c97-aea7-567d7ee42a23\") " pod="openstack-kuttl-tests/neutron-6d64d7894b-xp9d5" Dec 03 12:46:14 crc kubenswrapper[5052]: I1203 12:46:14.246018 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/901a2f0b-fb8e-4c97-aea7-567d7ee42a23-internal-tls-certs\") pod \"neutron-6d64d7894b-xp9d5\" (UID: \"901a2f0b-fb8e-4c97-aea7-567d7ee42a23\") " pod="openstack-kuttl-tests/neutron-6d64d7894b-xp9d5" Dec 03 12:46:14 crc kubenswrapper[5052]: I1203 12:46:14.246328 5052 reconciler_common.go:293] "Volume detached for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/ee05fe95-7cc0-4225-a86b-1682adc78b8b-ovsdb-rundir\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:14 crc kubenswrapper[5052]: I1203 12:46:14.246346 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f2d7e3bb-2829-4166-90df-53f9a50272eb-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:14 crc kubenswrapper[5052]: I1203 12:46:14.246357 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/f2d7e3bb-2829-4166-90df-53f9a50272eb-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:14 crc kubenswrapper[5052]: I1203 12:46:14.246367 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/ee05fe95-7cc0-4225-a86b-1682adc78b8b-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:14 crc kubenswrapper[5052]: I1203 12:46:14.246395 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ee05fe95-7cc0-4225-a86b-1682adc78b8b-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:14 crc kubenswrapper[5052]: I1203 12:46:14.246409 5052 reconciler_common.go:293] "Volume detached for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/f2d7e3bb-2829-4166-90df-53f9a50272eb-ovsdb-rundir\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:14 crc kubenswrapper[5052]: E1203 12:46:14.246818 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-neutron-internal-svc: secret "cert-neutron-internal-svc" not found Dec 03 12:46:14 crc kubenswrapper[5052]: E1203 12:46:14.246887 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/901a2f0b-fb8e-4c97-aea7-567d7ee42a23-internal-tls-certs podName:901a2f0b-fb8e-4c97-aea7-567d7ee42a23 nodeName:}" failed. No retries permitted until 2025-12-03 12:46:16.246868183 +0000 UTC m=+5166.835109947 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "internal-tls-certs" (UniqueName: "kubernetes.io/secret/901a2f0b-fb8e-4c97-aea7-567d7ee42a23-internal-tls-certs") pod "neutron-6d64d7894b-xp9d5" (UID: "901a2f0b-fb8e-4c97-aea7-567d7ee42a23") : secret "cert-neutron-internal-svc" not found Dec 03 12:46:14 crc kubenswrapper[5052]: E1203 12:46:14.247864 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-neutron-public-svc: secret "cert-neutron-public-svc" not found Dec 03 12:46:14 crc kubenswrapper[5052]: E1203 12:46:14.247912 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/901a2f0b-fb8e-4c97-aea7-567d7ee42a23-public-tls-certs podName:901a2f0b-fb8e-4c97-aea7-567d7ee42a23 nodeName:}" failed. No retries permitted until 2025-12-03 12:46:16.247900103 +0000 UTC m=+5166.836141867 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "public-tls-certs" (UniqueName: "kubernetes.io/secret/901a2f0b-fb8e-4c97-aea7-567d7ee42a23-public-tls-certs") pod "neutron-6d64d7894b-xp9d5" (UID: "901a2f0b-fb8e-4c97-aea7-567d7ee42a23") : secret "cert-neutron-public-svc" not found Dec 03 12:46:14 crc kubenswrapper[5052]: E1203 12:46:14.247960 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-neutron-ovndbs: secret "cert-neutron-ovndbs" not found Dec 03 12:46:14 crc kubenswrapper[5052]: E1203 12:46:14.247984 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/901a2f0b-fb8e-4c97-aea7-567d7ee42a23-ovndb-tls-certs podName:901a2f0b-fb8e-4c97-aea7-567d7ee42a23 nodeName:}" failed. No retries permitted until 2025-12-03 12:46:16.247976565 +0000 UTC m=+5166.836218329 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "ovndb-tls-certs" (UniqueName: "kubernetes.io/secret/901a2f0b-fb8e-4c97-aea7-567d7ee42a23-ovndb-tls-certs") pod "neutron-6d64d7894b-xp9d5" (UID: "901a2f0b-fb8e-4c97-aea7-567d7ee42a23") : secret "cert-neutron-ovndbs" not found Dec 03 12:46:14 crc kubenswrapper[5052]: I1203 12:46:14.348469 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" event={"ID":"e5f1bd78-3552-4d2d-8029-651f33c3918a","Type":"ContainerDied","Data":"7f01f3df1fa966ec776f19821537dd47bd449c58615b2fa1bdcf12c7377f2e11"} Dec 03 12:46:14 crc kubenswrapper[5052]: I1203 12:46:14.348534 5052 scope.go:117] "RemoveContainer" containerID="0c534558ed9f09feb4924d04acb90157e0fc3201f00732c7364a3bf82d62e52e" Dec 03 12:46:14 crc kubenswrapper[5052]: I1203 12:46:14.348611 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:46:14 crc kubenswrapper[5052]: I1203 12:46:14.357890 5052 generic.go:334] "Generic (PLEG): container finished" podID="2f966eeb-76ef-4771-acdf-e0de2bb463a4" containerID="db9cddd5e36fcb65f1558dec41404bab89ab5883ec640a193f5a47a8c712d924" exitCode=0 Dec 03 12:46:14 crc kubenswrapper[5052]: I1203 12:46:14.358072 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/memcached-0" event={"ID":"2f966eeb-76ef-4771-acdf-e0de2bb463a4","Type":"ContainerDied","Data":"db9cddd5e36fcb65f1558dec41404bab89ab5883ec640a193f5a47a8c712d924"} Dec 03 12:46:14 crc kubenswrapper[5052]: I1203 12:46:14.370750 5052 generic.go:334] "Generic (PLEG): container finished" podID="2c53ea4e-56ba-4ba1-8cb6-8c24725b5856" containerID="fe1395b62b90460d04b6099206e52d20adacc07cc857b25d7b625ceac277317a" exitCode=0 Dec 03 12:46:14 crc kubenswrapper[5052]: I1203 12:46:14.370895 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-cell1-galera-0" event={"ID":"2c53ea4e-56ba-4ba1-8cb6-8c24725b5856","Type":"ContainerDied","Data":"fe1395b62b90460d04b6099206e52d20adacc07cc857b25d7b625ceac277317a"} Dec 03 12:46:14 crc kubenswrapper[5052]: I1203 12:46:14.374453 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-keystone-listener-68fd6d997d-7tjjw" event={"ID":"2065f5bb-88eb-4e84-88b6-9982833d413d","Type":"ContainerStarted","Data":"d9800cff12dbc4c6cb5054864b4a94efa21b1761b5e25554829174d9dbd0716f"} Dec 03 12:46:14 crc kubenswrapper[5052]: I1203 12:46:14.387031 5052 generic.go:334] "Generic (PLEG): container finished" podID="f7e26e55-1426-4cd4-b169-a63868acdfaa" containerID="bbc2755b5351566cbdd1dc7fd68849cb7ea314f303defbebf723032464036311" exitCode=0 Dec 03 12:46:14 crc kubenswrapper[5052]: I1203 12:46:14.387152 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-conductor-0" event={"ID":"f7e26e55-1426-4cd4-b169-a63868acdfaa","Type":"ContainerDied","Data":"bbc2755b5351566cbdd1dc7fd68849cb7ea314f303defbebf723032464036311"} Dec 03 12:46:14 crc kubenswrapper[5052]: I1203 12:46:14.397661 5052 generic.go:334] "Generic (PLEG): container finished" podID="8fa85200-471e-4fba-9884-ef25328ebf64" containerID="ea0e318bc1c2d00f8c07df735c18136c77c564a4ca1cc7bf9d3e0dba21c6f865" exitCode=0 Dec 03 12:46:14 crc kubenswrapper[5052]: I1203 12:46:14.397753 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-galera-0" event={"ID":"8fa85200-471e-4fba-9884-ef25328ebf64","Type":"ContainerDied","Data":"ea0e318bc1c2d00f8c07df735c18136c77c564a4ca1cc7bf9d3e0dba21c6f865"} Dec 03 12:46:14 crc kubenswrapper[5052]: I1203 12:46:14.408905 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_ovsdbserver-sb-0_f2d7e3bb-2829-4166-90df-53f9a50272eb/ovsdbserver-sb/0.log" Dec 03 12:46:14 crc kubenswrapper[5052]: I1203 12:46:14.409024 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-sb-0" event={"ID":"f2d7e3bb-2829-4166-90df-53f9a50272eb","Type":"ContainerDied","Data":"786e9ff2bff44817fc6c2946e61ae2defac118812bc4492e8c06e24dc823c83b"} Dec 03 12:46:14 crc kubenswrapper[5052]: I1203 12:46:14.409052 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:46:14 crc kubenswrapper[5052]: I1203 12:46:14.415321 5052 generic.go:334] "Generic (PLEG): container finished" podID="6475384f-1965-4e8b-8806-af420d1381de" containerID="06fb78df44fbba56afa104136ade5fc0d9d9f01772c05059cf7f5e2ec493e511" exitCode=137 Dec 03 12:46:14 crc kubenswrapper[5052]: I1203 12:46:14.438733 5052 generic.go:334] "Generic (PLEG): container finished" podID="3ae64378-aaa4-46bc-8c48-e1d90e490016" containerID="3fa64d6a78f92e04f85cc78283bcecfe7e618fa805695f4e69f5237e051e4939" exitCode=0 Dec 03 12:46:14 crc kubenswrapper[5052]: I1203 12:46:14.438839 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"3ae64378-aaa4-46bc-8c48-e1d90e490016","Type":"ContainerDied","Data":"3fa64d6a78f92e04f85cc78283bcecfe7e618fa805695f4e69f5237e051e4939"} Dec 03 12:46:14 crc kubenswrapper[5052]: I1203 12:46:14.444842 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_ovsdbserver-nb-0_ee05fe95-7cc0-4225-a86b-1682adc78b8b/ovsdbserver-nb/0.log" Dec 03 12:46:14 crc kubenswrapper[5052]: I1203 12:46:14.444889 5052 generic.go:334] "Generic (PLEG): container finished" podID="ee05fe95-7cc0-4225-a86b-1682adc78b8b" containerID="d12594c4d36cf7fcca02723f8cb056b7e65d76a4582dd775c6dc70d741593906" exitCode=143 Dec 03 12:46:14 crc kubenswrapper[5052]: I1203 12:46:14.444915 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-nb-0" event={"ID":"ee05fe95-7cc0-4225-a86b-1682adc78b8b","Type":"ContainerDied","Data":"d12594c4d36cf7fcca02723f8cb056b7e65d76a4582dd775c6dc70d741593906"} Dec 03 12:46:14 crc kubenswrapper[5052]: I1203 12:46:14.444939 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-nb-0" event={"ID":"ee05fe95-7cc0-4225-a86b-1682adc78b8b","Type":"ContainerDied","Data":"027af82730bbe7317770cd1d55860ee4299bec5a78025826b71548604110db4b"} Dec 03 12:46:14 crc kubenswrapper[5052]: I1203 12:46:14.444993 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:46:14 crc kubenswrapper[5052]: I1203 12:46:14.447866 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage01-crc" (OuterVolumeSpecName: "ovndbcluster-nb-etc-ovn") pod "ee05fe95-7cc0-4225-a86b-1682adc78b8b" (UID: "ee05fe95-7cc0-4225-a86b-1682adc78b8b"). InnerVolumeSpecName "local-storage01-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 12:46:14 crc kubenswrapper[5052]: I1203 12:46:14.448048 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f2d7e3bb-2829-4166-90df-53f9a50272eb-kube-api-access-5jc57" (OuterVolumeSpecName: "kube-api-access-5jc57") pod "f2d7e3bb-2829-4166-90df-53f9a50272eb" (UID: "f2d7e3bb-2829-4166-90df-53f9a50272eb"). InnerVolumeSpecName "kube-api-access-5jc57". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:46:14 crc kubenswrapper[5052]: I1203 12:46:14.449670 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage09-crc" (OuterVolumeSpecName: "ovndbcluster-sb-etc-ovn") pod "f2d7e3bb-2829-4166-90df-53f9a50272eb" (UID: "f2d7e3bb-2829-4166-90df-53f9a50272eb"). InnerVolumeSpecName "local-storage09-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 12:46:14 crc kubenswrapper[5052]: I1203 12:46:14.450437 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage01-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage01-crc\") on node \"crc\" " Dec 03 12:46:14 crc kubenswrapper[5052]: I1203 12:46:14.450461 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-5jc57\" (UniqueName: \"kubernetes.io/projected/f2d7e3bb-2829-4166-90df-53f9a50272eb-kube-api-access-5jc57\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:14 crc kubenswrapper[5052]: I1203 12:46:14.450480 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage09-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage09-crc\") on node \"crc\" " Dec 03 12:46:14 crc kubenswrapper[5052]: I1203 12:46:14.451261 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e5f1bd78-3552-4d2d-8029-651f33c3918a-kube-api-access-pjstv" (OuterVolumeSpecName: "kube-api-access-pjstv") pod "e5f1bd78-3552-4d2d-8029-651f33c3918a" (UID: "e5f1bd78-3552-4d2d-8029-651f33c3918a"). InnerVolumeSpecName "kube-api-access-pjstv". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:46:14 crc kubenswrapper[5052]: I1203 12:46:14.451764 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ee05fe95-7cc0-4225-a86b-1682adc78b8b-kube-api-access-9wz5q" (OuterVolumeSpecName: "kube-api-access-9wz5q") pod "ee05fe95-7cc0-4225-a86b-1682adc78b8b" (UID: "ee05fe95-7cc0-4225-a86b-1682adc78b8b"). InnerVolumeSpecName "kube-api-access-9wz5q". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:46:14 crc kubenswrapper[5052]: I1203 12:46:14.541220 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/placement-bbcfbbc9d-5q6gx"] Dec 03 12:46:14 crc kubenswrapper[5052]: I1203 12:46:14.550843 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-worker-58f7cb7fd9-2k5tg"] Dec 03 12:46:14 crc kubenswrapper[5052]: I1203 12:46:14.552159 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-9wz5q\" (UniqueName: \"kubernetes.io/projected/ee05fe95-7cc0-4225-a86b-1682adc78b8b-kube-api-access-9wz5q\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:14 crc kubenswrapper[5052]: I1203 12:46:14.552190 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-pjstv\" (UniqueName: \"kubernetes.io/projected/e5f1bd78-3552-4d2d-8029-651f33c3918a-kube-api-access-pjstv\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:14 crc kubenswrapper[5052]: I1203 12:46:14.566876 5052 scope.go:117] "RemoveContainer" containerID="1499c2af6931030cde94894361cbf72f702626308576bf5ee6e7760bbc5034df" Dec 03 12:46:14 crc kubenswrapper[5052]: E1203 12:46:14.683255 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of 67e80f3b2185e807d12e6cb73fcdb3e6261d02a42c22e32ff2b8b1ff0732f99b is running failed: container process not found" containerID="67e80f3b2185e807d12e6cb73fcdb3e6261d02a42c22e32ff2b8b1ff0732f99b" cmd=["/usr/bin/pgrep","-r","DRST","nova-scheduler"] Dec 03 12:46:14 crc kubenswrapper[5052]: E1203 12:46:14.684046 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of 67e80f3b2185e807d12e6cb73fcdb3e6261d02a42c22e32ff2b8b1ff0732f99b is running failed: container process not found" containerID="67e80f3b2185e807d12e6cb73fcdb3e6261d02a42c22e32ff2b8b1ff0732f99b" cmd=["/usr/bin/pgrep","-r","DRST","nova-scheduler"] Dec 03 12:46:14 crc kubenswrapper[5052]: E1203 12:46:14.684381 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of 67e80f3b2185e807d12e6cb73fcdb3e6261d02a42c22e32ff2b8b1ff0732f99b is running failed: container process not found" containerID="67e80f3b2185e807d12e6cb73fcdb3e6261d02a42c22e32ff2b8b1ff0732f99b" cmd=["/usr/bin/pgrep","-r","DRST","nova-scheduler"] Dec 03 12:46:14 crc kubenswrapper[5052]: E1203 12:46:14.684492 5052 prober.go:104] "Probe errored" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of 67e80f3b2185e807d12e6cb73fcdb3e6261d02a42c22e32ff2b8b1ff0732f99b is running failed: container process not found" probeType="Readiness" pod="openstack-kuttl-tests/nova-scheduler-0" podUID="dfda1fd2-f923-47bd-bf43-8a1be3f79f09" containerName="nova-scheduler-scheduler" Dec 03 12:46:14 crc kubenswrapper[5052]: I1203 12:46:14.835057 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e5f1bd78-3552-4d2d-8029-651f33c3918a-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "e5f1bd78-3552-4d2d-8029-651f33c3918a" (UID: "e5f1bd78-3552-4d2d-8029-651f33c3918a"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:46:14 crc kubenswrapper[5052]: I1203 12:46:14.845141 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e5f1bd78-3552-4d2d-8029-651f33c3918a-config-data" (OuterVolumeSpecName: "config-data") pod "e5f1bd78-3552-4d2d-8029-651f33c3918a" (UID: "e5f1bd78-3552-4d2d-8029-651f33c3918a"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:46:14 crc kubenswrapper[5052]: I1203 12:46:14.935271 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e5f1bd78-3552-4d2d-8029-651f33c3918a-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:14 crc kubenswrapper[5052]: I1203 12:46:14.935306 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/e5f1bd78-3552-4d2d-8029-651f33c3918a-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:14 crc kubenswrapper[5052]: I1203 12:46:14.951570 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage09-crc" (UniqueName: "kubernetes.io/local-volume/local-storage09-crc") on node "crc" Dec 03 12:46:14 crc kubenswrapper[5052]: I1203 12:46:14.983535 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage01-crc" (UniqueName: "kubernetes.io/local-volume/local-storage01-crc") on node "crc" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.011172 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e5f1bd78-3552-4d2d-8029-651f33c3918a-vencrypt-tls-certs" (OuterVolumeSpecName: "vencrypt-tls-certs") pod "e5f1bd78-3552-4d2d-8029-651f33c3918a" (UID: "e5f1bd78-3552-4d2d-8029-651f33c3918a"). InnerVolumeSpecName "vencrypt-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.014058 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f2d7e3bb-2829-4166-90df-53f9a50272eb-metrics-certs-tls-certs" (OuterVolumeSpecName: "metrics-certs-tls-certs") pod "f2d7e3bb-2829-4166-90df-53f9a50272eb" (UID: "f2d7e3bb-2829-4166-90df-53f9a50272eb"). InnerVolumeSpecName "metrics-certs-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.027540 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ee05fe95-7cc0-4225-a86b-1682adc78b8b-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "ee05fe95-7cc0-4225-a86b-1682adc78b8b" (UID: "ee05fe95-7cc0-4225-a86b-1682adc78b8b"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.028451 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f2d7e3bb-2829-4166-90df-53f9a50272eb-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "f2d7e3bb-2829-4166-90df-53f9a50272eb" (UID: "f2d7e3bb-2829-4166-90df-53f9a50272eb"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.038686 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage09-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage09-crc\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.038718 5052 reconciler_common.go:293] "Volume detached for volume \"vencrypt-tls-certs\" (UniqueName: \"kubernetes.io/secret/e5f1bd78-3552-4d2d-8029-651f33c3918a-vencrypt-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.038733 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f2d7e3bb-2829-4166-90df-53f9a50272eb-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.038745 5052 reconciler_common.go:293] "Volume detached for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/f2d7e3bb-2829-4166-90df-53f9a50272eb-metrics-certs-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.038757 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage01-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage01-crc\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.038769 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ee05fe95-7cc0-4225-a86b-1682adc78b8b-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.129268 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e5f1bd78-3552-4d2d-8029-651f33c3918a-nova-novncproxy-tls-certs" (OuterVolumeSpecName: "nova-novncproxy-tls-certs") pod "e5f1bd78-3552-4d2d-8029-651f33c3918a" (UID: "e5f1bd78-3552-4d2d-8029-651f33c3918a"). InnerVolumeSpecName "nova-novncproxy-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.136748 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ee05fe95-7cc0-4225-a86b-1682adc78b8b-ovsdbserver-nb-tls-certs" (OuterVolumeSpecName: "ovsdbserver-nb-tls-certs") pod "ee05fe95-7cc0-4225-a86b-1682adc78b8b" (UID: "ee05fe95-7cc0-4225-a86b-1682adc78b8b"). InnerVolumeSpecName "ovsdbserver-nb-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.141574 5052 reconciler_common.go:293] "Volume detached for volume \"nova-novncproxy-tls-certs\" (UniqueName: \"kubernetes.io/secret/e5f1bd78-3552-4d2d-8029-651f33c3918a-nova-novncproxy-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.141610 5052 reconciler_common.go:293] "Volume detached for volume \"ovsdbserver-nb-tls-certs\" (UniqueName: \"kubernetes.io/secret/ee05fe95-7cc0-4225-a86b-1682adc78b8b-ovsdbserver-nb-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.162149 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f2d7e3bb-2829-4166-90df-53f9a50272eb-ovsdbserver-sb-tls-certs" (OuterVolumeSpecName: "ovsdbserver-sb-tls-certs") pod "f2d7e3bb-2829-4166-90df-53f9a50272eb" (UID: "f2d7e3bb-2829-4166-90df-53f9a50272eb"). InnerVolumeSpecName "ovsdbserver-sb-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.162687 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ee05fe95-7cc0-4225-a86b-1682adc78b8b-metrics-certs-tls-certs" (OuterVolumeSpecName: "metrics-certs-tls-certs") pod "ee05fe95-7cc0-4225-a86b-1682adc78b8b" (UID: "ee05fe95-7cc0-4225-a86b-1682adc78b8b"). InnerVolumeSpecName "metrics-certs-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.243660 5052 reconciler_common.go:293] "Volume detached for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/ee05fe95-7cc0-4225-a86b-1682adc78b8b-metrics-certs-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.243978 5052 reconciler_common.go:293] "Volume detached for volume \"ovsdbserver-sb-tls-certs\" (UniqueName: \"kubernetes.io/secret/f2d7e3bb-2829-4166-90df-53f9a50272eb-ovsdbserver-sb-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.410793 5052 scope.go:117] "RemoveContainer" containerID="23cc31c39f29b7dd83db1992297f60565ae8b315ab364419ad6f8475dec1526c" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.465654 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-bbcfbbc9d-5q6gx" event={"ID":"32f420c2-aa7e-44c8-a63a-7654aa3e6e34","Type":"ContainerStarted","Data":"67cbac03f1a3302f889c26050cb5fda9a887534e786826a06099e9737b5f209c"} Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.470956 5052 generic.go:334] "Generic (PLEG): container finished" podID="4eab5b48-f674-4100-9b42-f3ea257d47d4" containerID="01c4f4342abe4df4b3d520a97ffb3f4ba0abfe85cb81b459e64f94869c4a5f91" exitCode=0 Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.470983 5052 generic.go:334] "Generic (PLEG): container finished" podID="4eab5b48-f674-4100-9b42-f3ea257d47d4" containerID="7370547fbe4eb4455656b595065ce614afd49fa9ea95ccde635adb0b612d3c70" exitCode=2 Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.470996 5052 generic.go:334] "Generic (PLEG): container finished" podID="4eab5b48-f674-4100-9b42-f3ea257d47d4" containerID="d0d21a72bcbb3c194d8e900bc8615778059b26e70f1614cda87f183610dac51e" exitCode=0 Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.471005 5052 generic.go:334] "Generic (PLEG): container finished" podID="4eab5b48-f674-4100-9b42-f3ea257d47d4" containerID="c9ba5984640e4b46f378e76e1ad9dbbc5b996fff2d3c392ba7de3ff6753a48fd" exitCode=0 Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.471056 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"4eab5b48-f674-4100-9b42-f3ea257d47d4","Type":"ContainerDied","Data":"01c4f4342abe4df4b3d520a97ffb3f4ba0abfe85cb81b459e64f94869c4a5f91"} Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.471127 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"4eab5b48-f674-4100-9b42-f3ea257d47d4","Type":"ContainerDied","Data":"7370547fbe4eb4455656b595065ce614afd49fa9ea95ccde635adb0b612d3c70"} Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.471139 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"4eab5b48-f674-4100-9b42-f3ea257d47d4","Type":"ContainerDied","Data":"d0d21a72bcbb3c194d8e900bc8615778059b26e70f1614cda87f183610dac51e"} Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.471149 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"4eab5b48-f674-4100-9b42-f3ea257d47d4","Type":"ContainerDied","Data":"c9ba5984640e4b46f378e76e1ad9dbbc5b996fff2d3c392ba7de3ff6753a48fd"} Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.474864 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-galera-0" event={"ID":"8fa85200-471e-4fba-9884-ef25328ebf64","Type":"ContainerDied","Data":"3421f9030f680dfd5004e54202f04ffae977d213cdd3f225ef1cd6168dc03d0b"} Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.474912 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3421f9030f680dfd5004e54202f04ffae977d213cdd3f225ef1cd6168dc03d0b" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.487037 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="51bfbf9924d4614ecdc2ebbb74f253c7ff74716cfae76c233754dcb1b24cf749" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.489838 5052 generic.go:334] "Generic (PLEG): container finished" podID="dfda1fd2-f923-47bd-bf43-8a1be3f79f09" containerID="67e80f3b2185e807d12e6cb73fcdb3e6261d02a42c22e32ff2b8b1ff0732f99b" exitCode=0 Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.489908 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"dfda1fd2-f923-47bd-bf43-8a1be3f79f09","Type":"ContainerDied","Data":"67e80f3b2185e807d12e6cb73fcdb3e6261d02a42c22e32ff2b8b1ff0732f99b"} Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.489930 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"dfda1fd2-f923-47bd-bf43-8a1be3f79f09","Type":"ContainerDied","Data":"cfdd01f2934613c9aad88bd35e6649ab3d537f03cad96217a4429a953ea0c318"} Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.489940 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cfdd01f2934613c9aad88bd35e6649ab3d537f03cad96217a4429a953ea0c318" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.491995 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-conductor-0" event={"ID":"f7e26e55-1426-4cd4-b169-a63868acdfaa","Type":"ContainerDied","Data":"8dbbe64b3b63836daa942aa0826cc3c439c589221e596d583650e4f4faa19595"} Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.492041 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8dbbe64b3b63836daa942aa0826cc3c439c589221e596d583650e4f4faa19595" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.493146 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-worker-58f7cb7fd9-2k5tg" event={"ID":"e14465ed-868a-49ba-b3ab-1e67196229e8","Type":"ContainerStarted","Data":"45485570e1f7f9e7f27f00dd5245dc00110787efec07233dcc8738241e8446fe"} Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.495476 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/memcached-0" event={"ID":"2f966eeb-76ef-4771-acdf-e0de2bb463a4","Type":"ContainerDied","Data":"d6a981f05552d78cd6565fef69b3f65e0da68d983bdc3e996835365f9cdcc339"} Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.495503 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d6a981f05552d78cd6565fef69b3f65e0da68d983bdc3e996835365f9cdcc339" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.498005 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-cell1-galera-0" event={"ID":"2c53ea4e-56ba-4ba1-8cb6-8c24725b5856","Type":"ContainerDied","Data":"493cca92a439545f5191bee251dbca32a69625aa12bc51f56bb98a8f07bfdee4"} Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.498033 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="493cca92a439545f5191bee251dbca32a69625aa12bc51f56bb98a8f07bfdee4" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.500545 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.511080 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-keystone-listener-68fd6d997d-7tjjw" event={"ID":"2065f5bb-88eb-4e84-88b6-9982833d413d","Type":"ContainerStarted","Data":"2e1db89e52b83cc4d03ffdc49b1a121ba636bb18c0424300ea015664b45cff9a"} Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.529015 5052 scope.go:117] "RemoveContainer" containerID="61f7b1adc6b8d50af5982e338e056fb28d2ea6d8f6e0b8c7092747bfc78b185f" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.549072 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kvktx\" (UniqueName: \"kubernetes.io/projected/f7e26e55-1426-4cd4-b169-a63868acdfaa-kube-api-access-kvktx\") pod \"f7e26e55-1426-4cd4-b169-a63868acdfaa\" (UID: \"f7e26e55-1426-4cd4-b169-a63868acdfaa\") " Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.549109 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f7e26e55-1426-4cd4-b169-a63868acdfaa-combined-ca-bundle\") pod \"f7e26e55-1426-4cd4-b169-a63868acdfaa\" (UID: \"f7e26e55-1426-4cd4-b169-a63868acdfaa\") " Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.549213 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/f7e26e55-1426-4cd4-b169-a63868acdfaa-config-data\") pod \"f7e26e55-1426-4cd4-b169-a63868acdfaa\" (UID: \"f7e26e55-1426-4cd4-b169-a63868acdfaa\") " Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.565336 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstackclient" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.583987 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-sb-0"] Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.602971 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-sb-0"] Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.610198 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/memcached-0" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.623702 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.650937 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.674443 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ovsdbserver-sb-0"] Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.661300 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6475384f-1965-4e8b-8806-af420d1381de-combined-ca-bundle\") pod \"6475384f-1965-4e8b-8806-af420d1381de\" (UID: \"6475384f-1965-4e8b-8806-af420d1381de\") " Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.674714 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/2f966eeb-76ef-4771-acdf-e0de2bb463a4-config-data\") pod \"2f966eeb-76ef-4771-acdf-e0de2bb463a4\" (UID: \"2f966eeb-76ef-4771-acdf-e0de2bb463a4\") " Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.674801 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rbrwm\" (UniqueName: \"kubernetes.io/projected/2f966eeb-76ef-4771-acdf-e0de2bb463a4-kube-api-access-rbrwm\") pod \"2f966eeb-76ef-4771-acdf-e0de2bb463a4\" (UID: \"2f966eeb-76ef-4771-acdf-e0de2bb463a4\") " Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.674912 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2f966eeb-76ef-4771-acdf-e0de2bb463a4-combined-ca-bundle\") pod \"2f966eeb-76ef-4771-acdf-e0de2bb463a4\" (UID: \"2f966eeb-76ef-4771-acdf-e0de2bb463a4\") " Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.674941 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"openstack-config\" (UniqueName: \"kubernetes.io/configmap/6475384f-1965-4e8b-8806-af420d1381de-openstack-config\") pod \"6475384f-1965-4e8b-8806-af420d1381de\" (UID: \"6475384f-1965-4e8b-8806-af420d1381de\") " Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.674968 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-chn6f\" (UniqueName: \"kubernetes.io/projected/6475384f-1965-4e8b-8806-af420d1381de-kube-api-access-chn6f\") pod \"6475384f-1965-4e8b-8806-af420d1381de\" (UID: \"6475384f-1965-4e8b-8806-af420d1381de\") " Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.675018 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"memcached-tls-certs\" (UniqueName: \"kubernetes.io/secret/2f966eeb-76ef-4771-acdf-e0de2bb463a4-memcached-tls-certs\") pod \"2f966eeb-76ef-4771-acdf-e0de2bb463a4\" (UID: \"2f966eeb-76ef-4771-acdf-e0de2bb463a4\") " Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.675043 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/2f966eeb-76ef-4771-acdf-e0de2bb463a4-kolla-config\") pod \"2f966eeb-76ef-4771-acdf-e0de2bb463a4\" (UID: \"2f966eeb-76ef-4771-acdf-e0de2bb463a4\") " Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.675114 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"openstack-config-secret\" (UniqueName: \"kubernetes.io/secret/6475384f-1965-4e8b-8806-af420d1381de-openstack-config-secret\") pod \"6475384f-1965-4e8b-8806-af420d1381de\" (UID: \"6475384f-1965-4e8b-8806-af420d1381de\") " Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.672623 5052 scope.go:117] "RemoveContainer" containerID="d12594c4d36cf7fcca02723f8cb056b7e65d76a4582dd775c6dc70d741593906" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.672202 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f7e26e55-1426-4cd4-b169-a63868acdfaa-kube-api-access-kvktx" (OuterVolumeSpecName: "kube-api-access-kvktx") pod "f7e26e55-1426-4cd4-b169-a63868acdfaa" (UID: "f7e26e55-1426-4cd4-b169-a63868acdfaa"). InnerVolumeSpecName "kube-api-access-kvktx". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.682433 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2f966eeb-76ef-4771-acdf-e0de2bb463a4-config-data" (OuterVolumeSpecName: "config-data") pod "2f966eeb-76ef-4771-acdf-e0de2bb463a4" (UID: "2f966eeb-76ef-4771-acdf-e0de2bb463a4"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:46:15 crc kubenswrapper[5052]: E1203 12:46:15.686013 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="2f966eeb-76ef-4771-acdf-e0de2bb463a4" containerName="memcached" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.686028 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f966eeb-76ef-4771-acdf-e0de2bb463a4" containerName="memcached" Dec 03 12:46:15 crc kubenswrapper[5052]: E1203 12:46:15.686057 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="f7e26e55-1426-4cd4-b169-a63868acdfaa" containerName="nova-cell0-conductor-conductor" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.686064 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7e26e55-1426-4cd4-b169-a63868acdfaa" containerName="nova-cell0-conductor-conductor" Dec 03 12:46:15 crc kubenswrapper[5052]: E1203 12:46:15.686079 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="f2d7e3bb-2829-4166-90df-53f9a50272eb" containerName="openstack-network-exporter" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.686086 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="f2d7e3bb-2829-4166-90df-53f9a50272eb" containerName="openstack-network-exporter" Dec 03 12:46:15 crc kubenswrapper[5052]: E1203 12:46:15.686096 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="f2d7e3bb-2829-4166-90df-53f9a50272eb" containerName="ovsdbserver-sb" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.686104 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="f2d7e3bb-2829-4166-90df-53f9a50272eb" containerName="ovsdbserver-sb" Dec 03 12:46:15 crc kubenswrapper[5052]: E1203 12:46:15.686122 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="ee05fe95-7cc0-4225-a86b-1682adc78b8b" containerName="ovsdbserver-nb" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.686128 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee05fe95-7cc0-4225-a86b-1682adc78b8b" containerName="ovsdbserver-nb" Dec 03 12:46:15 crc kubenswrapper[5052]: E1203 12:46:15.686140 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="ee05fe95-7cc0-4225-a86b-1682adc78b8b" containerName="openstack-network-exporter" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.686147 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee05fe95-7cc0-4225-a86b-1682adc78b8b" containerName="openstack-network-exporter" Dec 03 12:46:15 crc kubenswrapper[5052]: E1203 12:46:15.686155 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="e5f1bd78-3552-4d2d-8029-651f33c3918a" containerName="nova-cell1-novncproxy-novncproxy" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.686162 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="e5f1bd78-3552-4d2d-8029-651f33c3918a" containerName="nova-cell1-novncproxy-novncproxy" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.686327 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="f2d7e3bb-2829-4166-90df-53f9a50272eb" containerName="openstack-network-exporter" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.686342 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="2f966eeb-76ef-4771-acdf-e0de2bb463a4" containerName="memcached" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.686349 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="ee05fe95-7cc0-4225-a86b-1682adc78b8b" containerName="ovsdbserver-nb" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.686359 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="f2d7e3bb-2829-4166-90df-53f9a50272eb" containerName="ovsdbserver-sb" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.686350 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2f966eeb-76ef-4771-acdf-e0de2bb463a4-kolla-config" (OuterVolumeSpecName: "kolla-config") pod "2f966eeb-76ef-4771-acdf-e0de2bb463a4" (UID: "2f966eeb-76ef-4771-acdf-e0de2bb463a4"). InnerVolumeSpecName "kolla-config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.686366 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="e5f1bd78-3552-4d2d-8029-651f33c3918a" containerName="nova-cell1-novncproxy-novncproxy" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.686399 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="f7e26e55-1426-4cd4-b169-a63868acdfaa" containerName="nova-cell0-conductor-conductor" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.686420 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="ee05fe95-7cc0-4225-a86b-1682adc78b8b" containerName="openstack-network-exporter" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.689390 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.700074 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"ovndbcluster-sb-scripts" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.700187 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"ovndbcluster-sb-config" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.700202 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ovncluster-ovndbcluster-sb-dockercfg-f2jbx" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.700323 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.700634 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-ovndbcluster-sb-ovndbs" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.711481 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e5f1bd78-3552-4d2d-8029-651f33c3918a" path="/var/lib/kubelet/pods/e5f1bd78-3552-4d2d-8029-651f33c3918a/volumes" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.712151 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f2d7e3bb-2829-4166-90df-53f9a50272eb" path="/var/lib/kubelet/pods/f2d7e3bb-2829-4166-90df-53f9a50272eb/volumes" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.715039 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.720450 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.722923 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2f966eeb-76ef-4771-acdf-e0de2bb463a4-kube-api-access-rbrwm" (OuterVolumeSpecName: "kube-api-access-rbrwm") pod "2f966eeb-76ef-4771-acdf-e0de2bb463a4" (UID: "2f966eeb-76ef-4771-acdf-e0de2bb463a4"). InnerVolumeSpecName "kube-api-access-rbrwm". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.736292 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 12:46:15 crc kubenswrapper[5052]: E1203 12:46:15.738436 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="8fa85200-471e-4fba-9884-ef25328ebf64" containerName="galera" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.738461 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="8fa85200-471e-4fba-9884-ef25328ebf64" containerName="galera" Dec 03 12:46:15 crc kubenswrapper[5052]: E1203 12:46:15.738472 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="dfda1fd2-f923-47bd-bf43-8a1be3f79f09" containerName="nova-scheduler-scheduler" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.738480 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="dfda1fd2-f923-47bd-bf43-8a1be3f79f09" containerName="nova-scheduler-scheduler" Dec 03 12:46:15 crc kubenswrapper[5052]: E1203 12:46:15.738587 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="2c53ea4e-56ba-4ba1-8cb6-8c24725b5856" containerName="mysql-bootstrap" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.738594 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c53ea4e-56ba-4ba1-8cb6-8c24725b5856" containerName="mysql-bootstrap" Dec 03 12:46:15 crc kubenswrapper[5052]: E1203 12:46:15.738608 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="8fa85200-471e-4fba-9884-ef25328ebf64" containerName="mysql-bootstrap" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.738614 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="8fa85200-471e-4fba-9884-ef25328ebf64" containerName="mysql-bootstrap" Dec 03 12:46:15 crc kubenswrapper[5052]: E1203 12:46:15.738629 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="2c53ea4e-56ba-4ba1-8cb6-8c24725b5856" containerName="galera" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.738636 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c53ea4e-56ba-4ba1-8cb6-8c24725b5856" containerName="galera" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.738961 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="dfda1fd2-f923-47bd-bf43-8a1be3f79f09" containerName="nova-scheduler-scheduler" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.738982 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="2c53ea4e-56ba-4ba1-8cb6-8c24725b5856" containerName="galera" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.738995 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="8fa85200-471e-4fba-9884-ef25328ebf64" containerName="galera" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.739608 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-sb-0"] Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.739724 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.742331 5052 scope.go:117] "RemoveContainer" containerID="61f7b1adc6b8d50af5982e338e056fb28d2ea6d8f6e0b8c7092747bfc78b185f" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.742395 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-nova-novncproxy-cell1-vencrypt" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.743458 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell1-novncproxy-config-data" Dec 03 12:46:15 crc kubenswrapper[5052]: E1203 12:46:15.743627 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"61f7b1adc6b8d50af5982e338e056fb28d2ea6d8f6e0b8c7092747bfc78b185f\": container with ID starting with 61f7b1adc6b8d50af5982e338e056fb28d2ea6d8f6e0b8c7092747bfc78b185f not found: ID does not exist" containerID="61f7b1adc6b8d50af5982e338e056fb28d2ea6d8f6e0b8c7092747bfc78b185f" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.743650 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"61f7b1adc6b8d50af5982e338e056fb28d2ea6d8f6e0b8c7092747bfc78b185f"} err="failed to get container status \"61f7b1adc6b8d50af5982e338e056fb28d2ea6d8f6e0b8c7092747bfc78b185f\": rpc error: code = NotFound desc = could not find container \"61f7b1adc6b8d50af5982e338e056fb28d2ea6d8f6e0b8c7092747bfc78b185f\": container with ID starting with 61f7b1adc6b8d50af5982e338e056fb28d2ea6d8f6e0b8c7092747bfc78b185f not found: ID does not exist" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.743671 5052 scope.go:117] "RemoveContainer" containerID="d12594c4d36cf7fcca02723f8cb056b7e65d76a4582dd775c6dc70d741593906" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.743707 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-nova-novncproxy-cell1-public-svc" Dec 03 12:46:15 crc kubenswrapper[5052]: E1203 12:46:15.744880 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d12594c4d36cf7fcca02723f8cb056b7e65d76a4582dd775c6dc70d741593906\": container with ID starting with d12594c4d36cf7fcca02723f8cb056b7e65d76a4582dd775c6dc70d741593906 not found: ID does not exist" containerID="d12594c4d36cf7fcca02723f8cb056b7e65d76a4582dd775c6dc70d741593906" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.744907 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d12594c4d36cf7fcca02723f8cb056b7e65d76a4582dd775c6dc70d741593906"} err="failed to get container status \"d12594c4d36cf7fcca02723f8cb056b7e65d76a4582dd775c6dc70d741593906\": rpc error: code = NotFound desc = could not find container \"d12594c4d36cf7fcca02723f8cb056b7e65d76a4582dd775c6dc70d741593906\": container with ID starting with d12594c4d36cf7fcca02723f8cb056b7e65d76a4582dd775c6dc70d741593906 not found: ID does not exist" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.746768 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6475384f-1965-4e8b-8806-af420d1381de-kube-api-access-chn6f" (OuterVolumeSpecName: "kube-api-access-chn6f") pod "6475384f-1965-4e8b-8806-af420d1381de" (UID: "6475384f-1965-4e8b-8806-af420d1381de"). InnerVolumeSpecName "kube-api-access-chn6f". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.747529 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-nb-0"] Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.767673 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.778535 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/4eab5b48-f674-4100-9b42-f3ea257d47d4-config-data\") pod \"4eab5b48-f674-4100-9b42-f3ea257d47d4\" (UID: \"4eab5b48-f674-4100-9b42-f3ea257d47d4\") " Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.778585 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8fa85200-471e-4fba-9884-ef25328ebf64-combined-ca-bundle\") pod \"8fa85200-471e-4fba-9884-ef25328ebf64\" (UID: \"8fa85200-471e-4fba-9884-ef25328ebf64\") " Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.778621 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/2c53ea4e-56ba-4ba1-8cb6-8c24725b5856-kolla-config\") pod \"2c53ea4e-56ba-4ba1-8cb6-8c24725b5856\" (UID: \"2c53ea4e-56ba-4ba1-8cb6-8c24725b5856\") " Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.778665 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/2c53ea4e-56ba-4ba1-8cb6-8c24725b5856-galera-tls-certs\") pod \"2c53ea4e-56ba-4ba1-8cb6-8c24725b5856\" (UID: \"2c53ea4e-56ba-4ba1-8cb6-8c24725b5856\") " Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.778747 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jf8gp\" (UniqueName: \"kubernetes.io/projected/2c53ea4e-56ba-4ba1-8cb6-8c24725b5856-kube-api-access-jf8gp\") pod \"2c53ea4e-56ba-4ba1-8cb6-8c24725b5856\" (UID: \"2c53ea4e-56ba-4ba1-8cb6-8c24725b5856\") " Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.778778 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/2c53ea4e-56ba-4ba1-8cb6-8c24725b5856-config-data-default\") pod \"2c53ea4e-56ba-4ba1-8cb6-8c24725b5856\" (UID: \"2c53ea4e-56ba-4ba1-8cb6-8c24725b5856\") " Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.778802 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j2rns\" (UniqueName: \"kubernetes.io/projected/4eab5b48-f674-4100-9b42-f3ea257d47d4-kube-api-access-j2rns\") pod \"4eab5b48-f674-4100-9b42-f3ea257d47d4\" (UID: \"4eab5b48-f674-4100-9b42-f3ea257d47d4\") " Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.778829 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/8fa85200-471e-4fba-9884-ef25328ebf64-galera-tls-certs\") pod \"8fa85200-471e-4fba-9884-ef25328ebf64\" (UID: \"8fa85200-471e-4fba-9884-ef25328ebf64\") " Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.778851 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/8fa85200-471e-4fba-9884-ef25328ebf64-kolla-config\") pod \"8fa85200-471e-4fba-9884-ef25328ebf64\" (UID: \"8fa85200-471e-4fba-9884-ef25328ebf64\") " Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.778897 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/8fa85200-471e-4fba-9884-ef25328ebf64-config-data-generated\") pod \"8fa85200-471e-4fba-9884-ef25328ebf64\" (UID: \"8fa85200-471e-4fba-9884-ef25328ebf64\") " Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.778929 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/8fa85200-471e-4fba-9884-ef25328ebf64-operator-scripts\") pod \"8fa85200-471e-4fba-9884-ef25328ebf64\" (UID: \"8fa85200-471e-4fba-9884-ef25328ebf64\") " Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.778961 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/2c53ea4e-56ba-4ba1-8cb6-8c24725b5856-config-data-generated\") pod \"2c53ea4e-56ba-4ba1-8cb6-8c24725b5856\" (UID: \"2c53ea4e-56ba-4ba1-8cb6-8c24725b5856\") " Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.778983 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/4eab5b48-f674-4100-9b42-f3ea257d47d4-scripts\") pod \"4eab5b48-f674-4100-9b42-f3ea257d47d4\" (UID: \"4eab5b48-f674-4100-9b42-f3ea257d47d4\") " Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.779005 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/dfda1fd2-f923-47bd-bf43-8a1be3f79f09-combined-ca-bundle\") pod \"dfda1fd2-f923-47bd-bf43-8a1be3f79f09\" (UID: \"dfda1fd2-f923-47bd-bf43-8a1be3f79f09\") " Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.779045 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/4eab5b48-f674-4100-9b42-f3ea257d47d4-log-httpd\") pod \"4eab5b48-f674-4100-9b42-f3ea257d47d4\" (UID: \"4eab5b48-f674-4100-9b42-f3ea257d47d4\") " Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.779066 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/4eab5b48-f674-4100-9b42-f3ea257d47d4-sg-core-conf-yaml\") pod \"4eab5b48-f674-4100-9b42-f3ea257d47d4\" (UID: \"4eab5b48-f674-4100-9b42-f3ea257d47d4\") " Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.779098 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"mysql-db\" (UniqueName: \"kubernetes.io/local-volume/local-storage11-crc\") pod \"2c53ea4e-56ba-4ba1-8cb6-8c24725b5856\" (UID: \"2c53ea4e-56ba-4ba1-8cb6-8c24725b5856\") " Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.779124 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/2c53ea4e-56ba-4ba1-8cb6-8c24725b5856-operator-scripts\") pod \"2c53ea4e-56ba-4ba1-8cb6-8c24725b5856\" (UID: \"2c53ea4e-56ba-4ba1-8cb6-8c24725b5856\") " Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.779143 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"mysql-db\" (UniqueName: \"kubernetes.io/local-volume/local-storage07-crc\") pod \"8fa85200-471e-4fba-9884-ef25328ebf64\" (UID: \"8fa85200-471e-4fba-9884-ef25328ebf64\") " Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.779169 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lplfm\" (UniqueName: \"kubernetes.io/projected/8fa85200-471e-4fba-9884-ef25328ebf64-kube-api-access-lplfm\") pod \"8fa85200-471e-4fba-9884-ef25328ebf64\" (UID: \"8fa85200-471e-4fba-9884-ef25328ebf64\") " Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.779191 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2c53ea4e-56ba-4ba1-8cb6-8c24725b5856-combined-ca-bundle\") pod \"2c53ea4e-56ba-4ba1-8cb6-8c24725b5856\" (UID: \"2c53ea4e-56ba-4ba1-8cb6-8c24725b5856\") " Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.779210 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/4eab5b48-f674-4100-9b42-f3ea257d47d4-run-httpd\") pod \"4eab5b48-f674-4100-9b42-f3ea257d47d4\" (UID: \"4eab5b48-f674-4100-9b42-f3ea257d47d4\") " Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.779254 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/4eab5b48-f674-4100-9b42-f3ea257d47d4-ceilometer-tls-certs\") pod \"4eab5b48-f674-4100-9b42-f3ea257d47d4\" (UID: \"4eab5b48-f674-4100-9b42-f3ea257d47d4\") " Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.779283 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ph884\" (UniqueName: \"kubernetes.io/projected/dfda1fd2-f923-47bd-bf43-8a1be3f79f09-kube-api-access-ph884\") pod \"dfda1fd2-f923-47bd-bf43-8a1be3f79f09\" (UID: \"dfda1fd2-f923-47bd-bf43-8a1be3f79f09\") " Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.779313 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/dfda1fd2-f923-47bd-bf43-8a1be3f79f09-config-data\") pod \"dfda1fd2-f923-47bd-bf43-8a1be3f79f09\" (UID: \"dfda1fd2-f923-47bd-bf43-8a1be3f79f09\") " Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.779351 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4eab5b48-f674-4100-9b42-f3ea257d47d4-combined-ca-bundle\") pod \"4eab5b48-f674-4100-9b42-f3ea257d47d4\" (UID: \"4eab5b48-f674-4100-9b42-f3ea257d47d4\") " Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.779393 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/8fa85200-471e-4fba-9884-ef25328ebf64-config-data-default\") pod \"8fa85200-471e-4fba-9884-ef25328ebf64\" (UID: \"8fa85200-471e-4fba-9884-ef25328ebf64\") " Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.779592 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nova-novncproxy-tls-certs\" (UniqueName: \"kubernetes.io/secret/9c996590-4337-477e-a236-3e834d192346-nova-novncproxy-tls-certs\") pod \"nova-cell1-novncproxy-0\" (UID: \"9c996590-4337-477e-a236-3e834d192346\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.779664 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/c6c9b62e-bbbc-4557-92fd-da64bd8ee0cf-metrics-certs-tls-certs\") pod \"ovsdbserver-sb-0\" (UID: \"c6c9b62e-bbbc-4557-92fd-da64bd8ee0cf\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.779693 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage09-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage09-crc\") pod \"ovsdbserver-sb-0\" (UID: \"c6c9b62e-bbbc-4557-92fd-da64bd8ee0cf\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.779776 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qbgbz\" (UniqueName: \"kubernetes.io/projected/9c996590-4337-477e-a236-3e834d192346-kube-api-access-qbgbz\") pod \"nova-cell1-novncproxy-0\" (UID: \"9c996590-4337-477e-a236-3e834d192346\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.779846 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/c6c9b62e-bbbc-4557-92fd-da64bd8ee0cf-ovsdb-rundir\") pod \"ovsdbserver-sb-0\" (UID: \"c6c9b62e-bbbc-4557-92fd-da64bd8ee0cf\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.779878 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/9c996590-4337-477e-a236-3e834d192346-config-data\") pod \"nova-cell1-novncproxy-0\" (UID: \"9c996590-4337-477e-a236-3e834d192346\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.779909 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9c996590-4337-477e-a236-3e834d192346-combined-ca-bundle\") pod \"nova-cell1-novncproxy-0\" (UID: \"9c996590-4337-477e-a236-3e834d192346\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.779930 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c6c9b62e-bbbc-4557-92fd-da64bd8ee0cf-combined-ca-bundle\") pod \"ovsdbserver-sb-0\" (UID: \"c6c9b62e-bbbc-4557-92fd-da64bd8ee0cf\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.779974 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9bkcr\" (UniqueName: \"kubernetes.io/projected/c6c9b62e-bbbc-4557-92fd-da64bd8ee0cf-kube-api-access-9bkcr\") pod \"ovsdbserver-sb-0\" (UID: \"c6c9b62e-bbbc-4557-92fd-da64bd8ee0cf\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.780038 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/c6c9b62e-bbbc-4557-92fd-da64bd8ee0cf-scripts\") pod \"ovsdbserver-sb-0\" (UID: \"c6c9b62e-bbbc-4557-92fd-da64bd8ee0cf\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.780079 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovsdbserver-sb-tls-certs\" (UniqueName: \"kubernetes.io/secret/c6c9b62e-bbbc-4557-92fd-da64bd8ee0cf-ovsdbserver-sb-tls-certs\") pod \"ovsdbserver-sb-0\" (UID: \"c6c9b62e-bbbc-4557-92fd-da64bd8ee0cf\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.780183 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"vencrypt-tls-certs\" (UniqueName: \"kubernetes.io/secret/9c996590-4337-477e-a236-3e834d192346-vencrypt-tls-certs\") pod \"nova-cell1-novncproxy-0\" (UID: \"9c996590-4337-477e-a236-3e834d192346\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.780293 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c6c9b62e-bbbc-4557-92fd-da64bd8ee0cf-config\") pod \"ovsdbserver-sb-0\" (UID: \"c6c9b62e-bbbc-4557-92fd-da64bd8ee0cf\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.780460 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/2f966eeb-76ef-4771-acdf-e0de2bb463a4-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.780477 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-rbrwm\" (UniqueName: \"kubernetes.io/projected/2f966eeb-76ef-4771-acdf-e0de2bb463a4-kube-api-access-rbrwm\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.780491 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-chn6f\" (UniqueName: \"kubernetes.io/projected/6475384f-1965-4e8b-8806-af420d1381de-kube-api-access-chn6f\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.780528 5052 reconciler_common.go:293] "Volume detached for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/2f966eeb-76ef-4771-acdf-e0de2bb463a4-kolla-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.781673 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-kvktx\" (UniqueName: \"kubernetes.io/projected/f7e26e55-1426-4cd4-b169-a63868acdfaa-kube-api-access-kvktx\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.782095 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8fa85200-471e-4fba-9884-ef25328ebf64-config-data-generated" (OuterVolumeSpecName: "config-data-generated") pod "8fa85200-471e-4fba-9884-ef25328ebf64" (UID: "8fa85200-471e-4fba-9884-ef25328ebf64"). InnerVolumeSpecName "config-data-generated". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.782677 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2c53ea4e-56ba-4ba1-8cb6-8c24725b5856-config-data-default" (OuterVolumeSpecName: "config-data-default") pod "2c53ea4e-56ba-4ba1-8cb6-8c24725b5856" (UID: "2c53ea4e-56ba-4ba1-8cb6-8c24725b5856"). InnerVolumeSpecName "config-data-default". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.782993 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4eab5b48-f674-4100-9b42-f3ea257d47d4-run-httpd" (OuterVolumeSpecName: "run-httpd") pod "4eab5b48-f674-4100-9b42-f3ea257d47d4" (UID: "4eab5b48-f674-4100-9b42-f3ea257d47d4"). InnerVolumeSpecName "run-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.784149 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2c53ea4e-56ba-4ba1-8cb6-8c24725b5856-kolla-config" (OuterVolumeSpecName: "kolla-config") pod "2c53ea4e-56ba-4ba1-8cb6-8c24725b5856" (UID: "2c53ea4e-56ba-4ba1-8cb6-8c24725b5856"). InnerVolumeSpecName "kolla-config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.784179 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c53ea4e-56ba-4ba1-8cb6-8c24725b5856-config-data-generated" (OuterVolumeSpecName: "config-data-generated") pod "2c53ea4e-56ba-4ba1-8cb6-8c24725b5856" (UID: "2c53ea4e-56ba-4ba1-8cb6-8c24725b5856"). InnerVolumeSpecName "config-data-generated". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.784203 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4eab5b48-f674-4100-9b42-f3ea257d47d4-log-httpd" (OuterVolumeSpecName: "log-httpd") pod "4eab5b48-f674-4100-9b42-f3ea257d47d4" (UID: "4eab5b48-f674-4100-9b42-f3ea257d47d4"). InnerVolumeSpecName "log-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.784466 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8fa85200-471e-4fba-9884-ef25328ebf64-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "8fa85200-471e-4fba-9884-ef25328ebf64" (UID: "8fa85200-471e-4fba-9884-ef25328ebf64"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.784898 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8fa85200-471e-4fba-9884-ef25328ebf64-kolla-config" (OuterVolumeSpecName: "kolla-config") pod "8fa85200-471e-4fba-9884-ef25328ebf64" (UID: "8fa85200-471e-4fba-9884-ef25328ebf64"). InnerVolumeSpecName "kolla-config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.785015 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8fa85200-471e-4fba-9884-ef25328ebf64-config-data-default" (OuterVolumeSpecName: "config-data-default") pod "8fa85200-471e-4fba-9884-ef25328ebf64" (UID: "8fa85200-471e-4fba-9884-ef25328ebf64"). InnerVolumeSpecName "config-data-default". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.790745 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2c53ea4e-56ba-4ba1-8cb6-8c24725b5856-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "2c53ea4e-56ba-4ba1-8cb6-8c24725b5856" (UID: "2c53ea4e-56ba-4ba1-8cb6-8c24725b5856"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.790987 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-nb-0"] Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.808659 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.810679 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8fa85200-471e-4fba-9884-ef25328ebf64-kube-api-access-lplfm" (OuterVolumeSpecName: "kube-api-access-lplfm") pod "8fa85200-471e-4fba-9884-ef25328ebf64" (UID: "8fa85200-471e-4fba-9884-ef25328ebf64"). InnerVolumeSpecName "kube-api-access-lplfm". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.813211 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4eab5b48-f674-4100-9b42-f3ea257d47d4-kube-api-access-j2rns" (OuterVolumeSpecName: "kube-api-access-j2rns") pod "4eab5b48-f674-4100-9b42-f3ea257d47d4" (UID: "4eab5b48-f674-4100-9b42-f3ea257d47d4"). InnerVolumeSpecName "kube-api-access-j2rns". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.820081 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2c53ea4e-56ba-4ba1-8cb6-8c24725b5856-kube-api-access-jf8gp" (OuterVolumeSpecName: "kube-api-access-jf8gp") pod "2c53ea4e-56ba-4ba1-8cb6-8c24725b5856" (UID: "2c53ea4e-56ba-4ba1-8cb6-8c24725b5856"). InnerVolumeSpecName "kube-api-access-jf8gp". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.858305 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ovsdbserver-nb-0"] Dec 03 12:46:15 crc kubenswrapper[5052]: E1203 12:46:15.858955 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="4eab5b48-f674-4100-9b42-f3ea257d47d4" containerName="ceilometer-notification-agent" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.858984 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="4eab5b48-f674-4100-9b42-f3ea257d47d4" containerName="ceilometer-notification-agent" Dec 03 12:46:15 crc kubenswrapper[5052]: E1203 12:46:15.859012 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="4eab5b48-f674-4100-9b42-f3ea257d47d4" containerName="proxy-httpd" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.859018 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="4eab5b48-f674-4100-9b42-f3ea257d47d4" containerName="proxy-httpd" Dec 03 12:46:15 crc kubenswrapper[5052]: E1203 12:46:15.859033 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="4eab5b48-f674-4100-9b42-f3ea257d47d4" containerName="ceilometer-central-agent" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.859040 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="4eab5b48-f674-4100-9b42-f3ea257d47d4" containerName="ceilometer-central-agent" Dec 03 12:46:15 crc kubenswrapper[5052]: E1203 12:46:15.859051 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="4eab5b48-f674-4100-9b42-f3ea257d47d4" containerName="sg-core" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.859058 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="4eab5b48-f674-4100-9b42-f3ea257d47d4" containerName="sg-core" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.859309 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="4eab5b48-f674-4100-9b42-f3ea257d47d4" containerName="ceilometer-central-agent" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.859338 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="4eab5b48-f674-4100-9b42-f3ea257d47d4" containerName="sg-core" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.859458 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="4eab5b48-f674-4100-9b42-f3ea257d47d4" containerName="proxy-httpd" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.859494 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="4eab5b48-f674-4100-9b42-f3ea257d47d4" containerName="ceilometer-notification-agent" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.860648 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.866387 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-ovndbcluster-nb-ovndbs" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.866474 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dfda1fd2-f923-47bd-bf43-8a1be3f79f09-kube-api-access-ph884" (OuterVolumeSpecName: "kube-api-access-ph884") pod "dfda1fd2-f923-47bd-bf43-8a1be3f79f09" (UID: "dfda1fd2-f923-47bd-bf43-8a1be3f79f09"). InnerVolumeSpecName "kube-api-access-ph884". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.866398 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"ovndbcluster-nb-scripts" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.866600 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ovncluster-ovndbcluster-nb-dockercfg-2jlnn" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.866649 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"ovndbcluster-nb-config" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.870208 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4eab5b48-f674-4100-9b42-f3ea257d47d4-scripts" (OuterVolumeSpecName: "scripts") pod "4eab5b48-f674-4100-9b42-f3ea257d47d4" (UID: "4eab5b48-f674-4100-9b42-f3ea257d47d4"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.885558 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"nova-novncproxy-tls-certs\" (UniqueName: \"kubernetes.io/secret/9c996590-4337-477e-a236-3e834d192346-nova-novncproxy-tls-certs\") pod \"nova-cell1-novncproxy-0\" (UID: \"9c996590-4337-477e-a236-3e834d192346\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.885856 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/c6c9b62e-bbbc-4557-92fd-da64bd8ee0cf-metrics-certs-tls-certs\") pod \"ovsdbserver-sb-0\" (UID: \"c6c9b62e-bbbc-4557-92fd-da64bd8ee0cf\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.885943 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage09-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage09-crc\") pod \"ovsdbserver-sb-0\" (UID: \"c6c9b62e-bbbc-4557-92fd-da64bd8ee0cf\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.886040 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/c069aee2-4cb6-4c8a-bbe5-949ab0b48331-internal-tls-certs\") pod \"barbican-api-bcf49b784-jnkdb\" (UID: \"c069aee2-4cb6-4c8a-bbe5-949ab0b48331\") " pod="openstack-kuttl-tests/barbican-api-bcf49b784-jnkdb" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.886166 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-qbgbz\" (UniqueName: \"kubernetes.io/projected/9c996590-4337-477e-a236-3e834d192346-kube-api-access-qbgbz\") pod \"nova-cell1-novncproxy-0\" (UID: \"9c996590-4337-477e-a236-3e834d192346\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.890394 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/c6c9b62e-bbbc-4557-92fd-da64bd8ee0cf-ovsdb-rundir\") pod \"ovsdbserver-sb-0\" (UID: \"c6c9b62e-bbbc-4557-92fd-da64bd8ee0cf\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.890619 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/9c996590-4337-477e-a236-3e834d192346-config-data\") pod \"nova-cell1-novncproxy-0\" (UID: \"9c996590-4337-477e-a236-3e834d192346\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.890787 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c6c9b62e-bbbc-4557-92fd-da64bd8ee0cf-combined-ca-bundle\") pod \"ovsdbserver-sb-0\" (UID: \"c6c9b62e-bbbc-4557-92fd-da64bd8ee0cf\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.890904 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9c996590-4337-477e-a236-3e834d192346-combined-ca-bundle\") pod \"nova-cell1-novncproxy-0\" (UID: \"9c996590-4337-477e-a236-3e834d192346\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.891113 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/c6c9b62e-bbbc-4557-92fd-da64bd8ee0cf-ovsdb-rundir\") pod \"ovsdbserver-sb-0\" (UID: \"c6c9b62e-bbbc-4557-92fd-da64bd8ee0cf\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:46:15 crc kubenswrapper[5052]: E1203 12:46:15.889363 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-ovn-metrics: secret "cert-ovn-metrics" not found Dec 03 12:46:15 crc kubenswrapper[5052]: E1203 12:46:15.891210 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c6c9b62e-bbbc-4557-92fd-da64bd8ee0cf-metrics-certs-tls-certs podName:c6c9b62e-bbbc-4557-92fd-da64bd8ee0cf nodeName:}" failed. No retries permitted until 2025-12-03 12:46:16.391192159 +0000 UTC m=+5166.979433993 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs-tls-certs" (UniqueName: "kubernetes.io/secret/c6c9b62e-bbbc-4557-92fd-da64bd8ee0cf-metrics-certs-tls-certs") pod "ovsdbserver-sb-0" (UID: "c6c9b62e-bbbc-4557-92fd-da64bd8ee0cf") : secret "cert-ovn-metrics" not found Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.890501 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-nb-0"] Dec 03 12:46:15 crc kubenswrapper[5052]: E1203 12:46:15.889411 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-nova-novncproxy-cell1-public-svc: secret "cert-nova-novncproxy-cell1-public-svc" not found Dec 03 12:46:15 crc kubenswrapper[5052]: E1203 12:46:15.891414 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9c996590-4337-477e-a236-3e834d192346-nova-novncproxy-tls-certs podName:9c996590-4337-477e-a236-3e834d192346 nodeName:}" failed. No retries permitted until 2025-12-03 12:46:16.391404285 +0000 UTC m=+5166.979646049 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "nova-novncproxy-tls-certs" (UniqueName: "kubernetes.io/secret/9c996590-4337-477e-a236-3e834d192346-nova-novncproxy-tls-certs") pod "nova-cell1-novncproxy-0" (UID: "9c996590-4337-477e-a236-3e834d192346") : secret "cert-nova-novncproxy-cell1-public-svc" not found Dec 03 12:46:15 crc kubenswrapper[5052]: E1203 12:46:15.890478 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-barbican-internal-svc: secret "cert-barbican-internal-svc" not found Dec 03 12:46:15 crc kubenswrapper[5052]: E1203 12:46:15.891448 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c069aee2-4cb6-4c8a-bbe5-949ab0b48331-internal-tls-certs podName:c069aee2-4cb6-4c8a-bbe5-949ab0b48331 nodeName:}" failed. No retries permitted until 2025-12-03 12:46:19.891440056 +0000 UTC m=+5170.479681910 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "internal-tls-certs" (UniqueName: "kubernetes.io/secret/c069aee2-4cb6-4c8a-bbe5-949ab0b48331-internal-tls-certs") pod "barbican-api-bcf49b784-jnkdb" (UID: "c069aee2-4cb6-4c8a-bbe5-949ab0b48331") : secret "cert-barbican-internal-svc" not found Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.890430 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage09-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage09-crc\") pod \"ovsdbserver-sb-0\" (UID: \"c6c9b62e-bbbc-4557-92fd-da64bd8ee0cf\") device mount path \"/mnt/openstack/pv09\"" pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:46:15 crc kubenswrapper[5052]: E1203 12:46:15.896998 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="adc4f6d0e36c4370a4bff93d0b0803fec09e3b34230d31ebbe4f500396923a29" cmd=["/usr/local/bin/container-scripts/status_check.sh"] Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.897262 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-9bkcr\" (UniqueName: \"kubernetes.io/projected/c6c9b62e-bbbc-4557-92fd-da64bd8ee0cf-kube-api-access-9bkcr\") pod \"ovsdbserver-sb-0\" (UID: \"c6c9b62e-bbbc-4557-92fd-da64bd8ee0cf\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.897352 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/c6c9b62e-bbbc-4557-92fd-da64bd8ee0cf-scripts\") pod \"ovsdbserver-sb-0\" (UID: \"c6c9b62e-bbbc-4557-92fd-da64bd8ee0cf\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.897410 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovsdbserver-sb-tls-certs\" (UniqueName: \"kubernetes.io/secret/c6c9b62e-bbbc-4557-92fd-da64bd8ee0cf-ovsdbserver-sb-tls-certs\") pod \"ovsdbserver-sb-0\" (UID: \"c6c9b62e-bbbc-4557-92fd-da64bd8ee0cf\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.897451 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"vencrypt-tls-certs\" (UniqueName: \"kubernetes.io/secret/9c996590-4337-477e-a236-3e834d192346-vencrypt-tls-certs\") pod \"nova-cell1-novncproxy-0\" (UID: \"9c996590-4337-477e-a236-3e834d192346\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.897530 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c6c9b62e-bbbc-4557-92fd-da64bd8ee0cf-config\") pod \"ovsdbserver-sb-0\" (UID: \"c6c9b62e-bbbc-4557-92fd-da64bd8ee0cf\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.897571 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/c069aee2-4cb6-4c8a-bbe5-949ab0b48331-public-tls-certs\") pod \"barbican-api-bcf49b784-jnkdb\" (UID: \"c069aee2-4cb6-4c8a-bbe5-949ab0b48331\") " pod="openstack-kuttl-tests/barbican-api-bcf49b784-jnkdb" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.897713 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/2c53ea4e-56ba-4ba1-8cb6-8c24725b5856-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.897725 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-lplfm\" (UniqueName: \"kubernetes.io/projected/8fa85200-471e-4fba-9884-ef25328ebf64-kube-api-access-lplfm\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.897735 5052 reconciler_common.go:293] "Volume detached for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/4eab5b48-f674-4100-9b42-f3ea257d47d4-run-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.897747 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-ph884\" (UniqueName: \"kubernetes.io/projected/dfda1fd2-f923-47bd-bf43-8a1be3f79f09-kube-api-access-ph884\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.897757 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/8fa85200-471e-4fba-9884-ef25328ebf64-config-data-default\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.897770 5052 reconciler_common.go:293] "Volume detached for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/2c53ea4e-56ba-4ba1-8cb6-8c24725b5856-kolla-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.897782 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-jf8gp\" (UniqueName: \"kubernetes.io/projected/2c53ea4e-56ba-4ba1-8cb6-8c24725b5856-kube-api-access-jf8gp\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.897792 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/2c53ea4e-56ba-4ba1-8cb6-8c24725b5856-config-data-default\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.897802 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-j2rns\" (UniqueName: \"kubernetes.io/projected/4eab5b48-f674-4100-9b42-f3ea257d47d4-kube-api-access-j2rns\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.897811 5052 reconciler_common.go:293] "Volume detached for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/8fa85200-471e-4fba-9884-ef25328ebf64-kolla-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.897820 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/8fa85200-471e-4fba-9884-ef25328ebf64-config-data-generated\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.897829 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/8fa85200-471e-4fba-9884-ef25328ebf64-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.897841 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/2c53ea4e-56ba-4ba1-8cb6-8c24725b5856-config-data-generated\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.897850 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/4eab5b48-f674-4100-9b42-f3ea257d47d4-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.897859 5052 reconciler_common.go:293] "Volume detached for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/4eab5b48-f674-4100-9b42-f3ea257d47d4-log-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:15 crc kubenswrapper[5052]: E1203 12:46:15.897936 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-barbican-public-svc: secret "cert-barbican-public-svc" not found Dec 03 12:46:15 crc kubenswrapper[5052]: E1203 12:46:15.897976 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c069aee2-4cb6-4c8a-bbe5-949ab0b48331-public-tls-certs podName:c069aee2-4cb6-4c8a-bbe5-949ab0b48331 nodeName:}" failed. No retries permitted until 2025-12-03 12:46:19.897961932 +0000 UTC m=+5170.486203686 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "public-tls-certs" (UniqueName: "kubernetes.io/secret/c069aee2-4cb6-4c8a-bbe5-949ab0b48331-public-tls-certs") pod "barbican-api-bcf49b784-jnkdb" (UID: "c069aee2-4cb6-4c8a-bbe5-949ab0b48331") : secret "cert-barbican-public-svc" not found Dec 03 12:46:15 crc kubenswrapper[5052]: E1203 12:46:15.898698 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-nova-novncproxy-cell1-vencrypt: secret "cert-nova-novncproxy-cell1-vencrypt" not found Dec 03 12:46:15 crc kubenswrapper[5052]: E1203 12:46:15.898729 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9c996590-4337-477e-a236-3e834d192346-vencrypt-tls-certs podName:9c996590-4337-477e-a236-3e834d192346 nodeName:}" failed. No retries permitted until 2025-12-03 12:46:16.398719534 +0000 UTC m=+5166.986961298 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "vencrypt-tls-certs" (UniqueName: "kubernetes.io/secret/9c996590-4337-477e-a236-3e834d192346-vencrypt-tls-certs") pod "nova-cell1-novncproxy-0" (UID: "9c996590-4337-477e-a236-3e834d192346") : secret "cert-nova-novncproxy-cell1-vencrypt" not found Dec 03 12:46:15 crc kubenswrapper[5052]: E1203 12:46:15.898771 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-ovndbcluster-sb-ovndbs: secret "cert-ovndbcluster-sb-ovndbs" not found Dec 03 12:46:15 crc kubenswrapper[5052]: E1203 12:46:15.898791 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c6c9b62e-bbbc-4557-92fd-da64bd8ee0cf-ovsdbserver-sb-tls-certs podName:c6c9b62e-bbbc-4557-92fd-da64bd8ee0cf nodeName:}" failed. No retries permitted until 2025-12-03 12:46:16.398785216 +0000 UTC m=+5166.987026980 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "ovsdbserver-sb-tls-certs" (UniqueName: "kubernetes.io/secret/c6c9b62e-bbbc-4557-92fd-da64bd8ee0cf-ovsdbserver-sb-tls-certs") pod "ovsdbserver-sb-0" (UID: "c6c9b62e-bbbc-4557-92fd-da64bd8ee0cf") : secret "cert-ovndbcluster-sb-ovndbs" not found Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.899421 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c6c9b62e-bbbc-4557-92fd-da64bd8ee0cf-config\") pod \"ovsdbserver-sb-0\" (UID: \"c6c9b62e-bbbc-4557-92fd-da64bd8ee0cf\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.900534 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/c6c9b62e-bbbc-4557-92fd-da64bd8ee0cf-scripts\") pod \"ovsdbserver-sb-0\" (UID: \"c6c9b62e-bbbc-4557-92fd-da64bd8ee0cf\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:46:15 crc kubenswrapper[5052]: E1203 12:46:15.902099 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="adc4f6d0e36c4370a4bff93d0b0803fec09e3b34230d31ebbe4f500396923a29" cmd=["/usr/local/bin/container-scripts/status_check.sh"] Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.903543 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9c996590-4337-477e-a236-3e834d192346-combined-ca-bundle\") pod \"nova-cell1-novncproxy-0\" (UID: \"9c996590-4337-477e-a236-3e834d192346\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.903721 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/9c996590-4337-477e-a236-3e834d192346-config-data\") pod \"nova-cell1-novncproxy-0\" (UID: \"9c996590-4337-477e-a236-3e834d192346\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:46:15 crc kubenswrapper[5052]: E1203 12:46:15.903730 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="adc4f6d0e36c4370a4bff93d0b0803fec09e3b34230d31ebbe4f500396923a29" cmd=["/usr/local/bin/container-scripts/status_check.sh"] Dec 03 12:46:15 crc kubenswrapper[5052]: E1203 12:46:15.903760 5052 prober.go:104] "Probe errored" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" probeType="Readiness" pod="openstack-kuttl-tests/ovn-northd-0" podUID="6af0a301-2483-493d-bbc8-17138d79dfea" containerName="ovn-northd" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.904007 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c6c9b62e-bbbc-4557-92fd-da64bd8ee0cf-combined-ca-bundle\") pod \"ovsdbserver-sb-0\" (UID: \"c6c9b62e-bbbc-4557-92fd-da64bd8ee0cf\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.917230 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage11-crc" (OuterVolumeSpecName: "mysql-db") pod "2c53ea4e-56ba-4ba1-8cb6-8c24725b5856" (UID: "2c53ea4e-56ba-4ba1-8cb6-8c24725b5856"). InnerVolumeSpecName "local-storage11-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.923267 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-9bkcr\" (UniqueName: \"kubernetes.io/projected/c6c9b62e-bbbc-4557-92fd-da64bd8ee0cf-kube-api-access-9bkcr\") pod \"ovsdbserver-sb-0\" (UID: \"c6c9b62e-bbbc-4557-92fd-da64bd8ee0cf\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.924434 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-qbgbz\" (UniqueName: \"kubernetes.io/projected/9c996590-4337-477e-a236-3e834d192346-kube-api-access-qbgbz\") pod \"nova-cell1-novncproxy-0\" (UID: \"9c996590-4337-477e-a236-3e834d192346\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.936076 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage07-crc" (OuterVolumeSpecName: "mysql-db") pod "8fa85200-471e-4fba-9884-ef25328ebf64" (UID: "8fa85200-471e-4fba-9884-ef25328ebf64"). InnerVolumeSpecName "local-storage07-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 12:46:15 crc kubenswrapper[5052]: I1203 12:46:15.953684 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f7e26e55-1426-4cd4-b169-a63868acdfaa-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "f7e26e55-1426-4cd4-b169-a63868acdfaa" (UID: "f7e26e55-1426-4cd4-b169-a63868acdfaa"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:46:16 crc kubenswrapper[5052]: I1203 12:46:16.000033 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9zmq4\" (UniqueName: \"kubernetes.io/projected/29c7d055-4f37-45a4-a608-eb73a361a86c-kube-api-access-9zmq4\") pod \"ovsdbserver-nb-0\" (UID: \"29c7d055-4f37-45a4-a608-eb73a361a86c\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:46:16 crc kubenswrapper[5052]: I1203 12:46:16.000329 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovsdbserver-nb-tls-certs\" (UniqueName: \"kubernetes.io/secret/29c7d055-4f37-45a4-a608-eb73a361a86c-ovsdbserver-nb-tls-certs\") pod \"ovsdbserver-nb-0\" (UID: \"29c7d055-4f37-45a4-a608-eb73a361a86c\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:46:16 crc kubenswrapper[5052]: I1203 12:46:16.000486 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/29c7d055-4f37-45a4-a608-eb73a361a86c-ovsdb-rundir\") pod \"ovsdbserver-nb-0\" (UID: \"29c7d055-4f37-45a4-a608-eb73a361a86c\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:46:16 crc kubenswrapper[5052]: I1203 12:46:16.000691 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/29c7d055-4f37-45a4-a608-eb73a361a86c-config\") pod \"ovsdbserver-nb-0\" (UID: \"29c7d055-4f37-45a4-a608-eb73a361a86c\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:46:16 crc kubenswrapper[5052]: I1203 12:46:16.000867 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage01-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage01-crc\") pod \"ovsdbserver-nb-0\" (UID: \"29c7d055-4f37-45a4-a608-eb73a361a86c\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:46:16 crc kubenswrapper[5052]: I1203 12:46:16.000971 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/29c7d055-4f37-45a4-a608-eb73a361a86c-scripts\") pod \"ovsdbserver-nb-0\" (UID: \"29c7d055-4f37-45a4-a608-eb73a361a86c\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:46:16 crc kubenswrapper[5052]: I1203 12:46:16.001050 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/29c7d055-4f37-45a4-a608-eb73a361a86c-combined-ca-bundle\") pod \"ovsdbserver-nb-0\" (UID: \"29c7d055-4f37-45a4-a608-eb73a361a86c\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:46:16 crc kubenswrapper[5052]: I1203 12:46:16.001069 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/29c7d055-4f37-45a4-a608-eb73a361a86c-metrics-certs-tls-certs\") pod \"ovsdbserver-nb-0\" (UID: \"29c7d055-4f37-45a4-a608-eb73a361a86c\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:46:16 crc kubenswrapper[5052]: I1203 12:46:16.001234 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage11-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage11-crc\") on node \"crc\" " Dec 03 12:46:16 crc kubenswrapper[5052]: I1203 12:46:16.001277 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage07-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage07-crc\") on node \"crc\" " Dec 03 12:46:16 crc kubenswrapper[5052]: I1203 12:46:16.001414 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f7e26e55-1426-4cd4-b169-a63868acdfaa-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:16 crc kubenswrapper[5052]: I1203 12:46:16.041743 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage09-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage09-crc\") pod \"ovsdbserver-sb-0\" (UID: \"c6c9b62e-bbbc-4557-92fd-da64bd8ee0cf\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:46:16 crc kubenswrapper[5052]: I1203 12:46:16.052156 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/nova-metadata-0" podUID="9af09a5b-789c-4b83-b899-9d1b3f0d8603" containerName="nova-metadata-log" probeResult="failure" output="Get \"https://10.217.1.236:8775/\": read tcp 10.217.0.2:32972->10.217.1.236:8775: read: connection reset by peer" Dec 03 12:46:16 crc kubenswrapper[5052]: I1203 12:46:16.052174 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/nova-metadata-0" podUID="9af09a5b-789c-4b83-b899-9d1b3f0d8603" containerName="nova-metadata-metadata" probeResult="failure" output="Get \"https://10.217.1.236:8775/\": read tcp 10.217.0.2:32960->10.217.1.236:8775: read: connection reset by peer" Dec 03 12:46:16 crc kubenswrapper[5052]: I1203 12:46:16.103878 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/29c7d055-4f37-45a4-a608-eb73a361a86c-ovsdb-rundir\") pod \"ovsdbserver-nb-0\" (UID: \"29c7d055-4f37-45a4-a608-eb73a361a86c\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:46:16 crc kubenswrapper[5052]: I1203 12:46:16.103952 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/fa1c223d-82fa-488c-ac48-e88bce36a839-internal-tls-certs\") pod \"keystone-687ff6fcfc-9vwb2\" (UID: \"fa1c223d-82fa-488c-ac48-e88bce36a839\") " pod="openstack-kuttl-tests/keystone-687ff6fcfc-9vwb2" Dec 03 12:46:16 crc kubenswrapper[5052]: I1203 12:46:16.103976 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/29c7d055-4f37-45a4-a608-eb73a361a86c-config\") pod \"ovsdbserver-nb-0\" (UID: \"29c7d055-4f37-45a4-a608-eb73a361a86c\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:46:16 crc kubenswrapper[5052]: I1203 12:46:16.104058 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage01-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage01-crc\") pod \"ovsdbserver-nb-0\" (UID: \"29c7d055-4f37-45a4-a608-eb73a361a86c\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:46:16 crc kubenswrapper[5052]: I1203 12:46:16.104128 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/29c7d055-4f37-45a4-a608-eb73a361a86c-scripts\") pod \"ovsdbserver-nb-0\" (UID: \"29c7d055-4f37-45a4-a608-eb73a361a86c\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:46:16 crc kubenswrapper[5052]: I1203 12:46:16.104191 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/29c7d055-4f37-45a4-a608-eb73a361a86c-combined-ca-bundle\") pod \"ovsdbserver-nb-0\" (UID: \"29c7d055-4f37-45a4-a608-eb73a361a86c\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:46:16 crc kubenswrapper[5052]: I1203 12:46:16.104210 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/29c7d055-4f37-45a4-a608-eb73a361a86c-metrics-certs-tls-certs\") pod \"ovsdbserver-nb-0\" (UID: \"29c7d055-4f37-45a4-a608-eb73a361a86c\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:46:16 crc kubenswrapper[5052]: I1203 12:46:16.104332 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-9zmq4\" (UniqueName: \"kubernetes.io/projected/29c7d055-4f37-45a4-a608-eb73a361a86c-kube-api-access-9zmq4\") pod \"ovsdbserver-nb-0\" (UID: \"29c7d055-4f37-45a4-a608-eb73a361a86c\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:46:16 crc kubenswrapper[5052]: I1203 12:46:16.104370 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovsdbserver-nb-tls-certs\" (UniqueName: \"kubernetes.io/secret/29c7d055-4f37-45a4-a608-eb73a361a86c-ovsdbserver-nb-tls-certs\") pod \"ovsdbserver-nb-0\" (UID: \"29c7d055-4f37-45a4-a608-eb73a361a86c\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:46:16 crc kubenswrapper[5052]: I1203 12:46:16.104409 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/fa1c223d-82fa-488c-ac48-e88bce36a839-public-tls-certs\") pod \"keystone-687ff6fcfc-9vwb2\" (UID: \"fa1c223d-82fa-488c-ac48-e88bce36a839\") " pod="openstack-kuttl-tests/keystone-687ff6fcfc-9vwb2" Dec 03 12:46:16 crc kubenswrapper[5052]: E1203 12:46:16.104611 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-keystone-public-svc: secret "cert-keystone-public-svc" not found Dec 03 12:46:16 crc kubenswrapper[5052]: E1203 12:46:16.104699 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fa1c223d-82fa-488c-ac48-e88bce36a839-public-tls-certs podName:fa1c223d-82fa-488c-ac48-e88bce36a839 nodeName:}" failed. No retries permitted until 2025-12-03 12:46:20.104652213 +0000 UTC m=+5170.692893977 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "public-tls-certs" (UniqueName: "kubernetes.io/secret/fa1c223d-82fa-488c-ac48-e88bce36a839-public-tls-certs") pod "keystone-687ff6fcfc-9vwb2" (UID: "fa1c223d-82fa-488c-ac48-e88bce36a839") : secret "cert-keystone-public-svc" not found Dec 03 12:46:16 crc kubenswrapper[5052]: E1203 12:46:16.105051 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-ovndbcluster-nb-ovndbs: secret "cert-ovndbcluster-nb-ovndbs" not found Dec 03 12:46:16 crc kubenswrapper[5052]: E1203 12:46:16.105131 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/29c7d055-4f37-45a4-a608-eb73a361a86c-ovsdbserver-nb-tls-certs podName:29c7d055-4f37-45a4-a608-eb73a361a86c nodeName:}" failed. No retries permitted until 2025-12-03 12:46:16.605112116 +0000 UTC m=+5167.193353910 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "ovsdbserver-nb-tls-certs" (UniqueName: "kubernetes.io/secret/29c7d055-4f37-45a4-a608-eb73a361a86c-ovsdbserver-nb-tls-certs") pod "ovsdbserver-nb-0" (UID: "29c7d055-4f37-45a4-a608-eb73a361a86c") : secret "cert-ovndbcluster-nb-ovndbs" not found Dec 03 12:46:16 crc kubenswrapper[5052]: I1203 12:46:16.105418 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/29c7d055-4f37-45a4-a608-eb73a361a86c-ovsdb-rundir\") pod \"ovsdbserver-nb-0\" (UID: \"29c7d055-4f37-45a4-a608-eb73a361a86c\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:46:16 crc kubenswrapper[5052]: I1203 12:46:16.105473 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage01-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage01-crc\") pod \"ovsdbserver-nb-0\" (UID: \"29c7d055-4f37-45a4-a608-eb73a361a86c\") device mount path \"/mnt/openstack/pv01\"" pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:46:16 crc kubenswrapper[5052]: E1203 12:46:16.105847 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-ovn-metrics: secret "cert-ovn-metrics" not found Dec 03 12:46:16 crc kubenswrapper[5052]: E1203 12:46:16.105892 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/29c7d055-4f37-45a4-a608-eb73a361a86c-metrics-certs-tls-certs podName:29c7d055-4f37-45a4-a608-eb73a361a86c nodeName:}" failed. No retries permitted until 2025-12-03 12:46:16.605881858 +0000 UTC m=+5167.194123662 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs-tls-certs" (UniqueName: "kubernetes.io/secret/29c7d055-4f37-45a4-a608-eb73a361a86c-metrics-certs-tls-certs") pod "ovsdbserver-nb-0" (UID: "29c7d055-4f37-45a4-a608-eb73a361a86c") : secret "cert-ovn-metrics" not found Dec 03 12:46:16 crc kubenswrapper[5052]: E1203 12:46:16.107500 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-keystone-internal-svc: secret "cert-keystone-internal-svc" not found Dec 03 12:46:16 crc kubenswrapper[5052]: E1203 12:46:16.107571 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fa1c223d-82fa-488c-ac48-e88bce36a839-internal-tls-certs podName:fa1c223d-82fa-488c-ac48-e88bce36a839 nodeName:}" failed. No retries permitted until 2025-12-03 12:46:20.107556936 +0000 UTC m=+5170.695798750 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "internal-tls-certs" (UniqueName: "kubernetes.io/secret/fa1c223d-82fa-488c-ac48-e88bce36a839-internal-tls-certs") pod "keystone-687ff6fcfc-9vwb2" (UID: "fa1c223d-82fa-488c-ac48-e88bce36a839") : secret "cert-keystone-internal-svc" not found Dec 03 12:46:16 crc kubenswrapper[5052]: I1203 12:46:16.108763 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/29c7d055-4f37-45a4-a608-eb73a361a86c-scripts\") pod \"ovsdbserver-nb-0\" (UID: \"29c7d055-4f37-45a4-a608-eb73a361a86c\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:46:16 crc kubenswrapper[5052]: I1203 12:46:16.113060 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/29c7d055-4f37-45a4-a608-eb73a361a86c-config\") pod \"ovsdbserver-nb-0\" (UID: \"29c7d055-4f37-45a4-a608-eb73a361a86c\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:46:16 crc kubenswrapper[5052]: I1203 12:46:16.133403 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-9zmq4\" (UniqueName: \"kubernetes.io/projected/29c7d055-4f37-45a4-a608-eb73a361a86c-kube-api-access-9zmq4\") pod \"ovsdbserver-nb-0\" (UID: \"29c7d055-4f37-45a4-a608-eb73a361a86c\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:46:16 crc kubenswrapper[5052]: I1203 12:46:16.133668 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/29c7d055-4f37-45a4-a608-eb73a361a86c-combined-ca-bundle\") pod \"ovsdbserver-nb-0\" (UID: \"29c7d055-4f37-45a4-a608-eb73a361a86c\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:46:16 crc kubenswrapper[5052]: I1203 12:46:16.252419 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage07-crc" (UniqueName: "kubernetes.io/local-volume/local-storage07-crc") on node "crc" Dec 03 12:46:16 crc kubenswrapper[5052]: I1203 12:46:16.293054 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage11-crc" (UniqueName: "kubernetes.io/local-volume/local-storage11-crc") on node "crc" Dec 03 12:46:16 crc kubenswrapper[5052]: I1203 12:46:16.309949 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/901a2f0b-fb8e-4c97-aea7-567d7ee42a23-public-tls-certs\") pod \"neutron-6d64d7894b-xp9d5\" (UID: \"901a2f0b-fb8e-4c97-aea7-567d7ee42a23\") " pod="openstack-kuttl-tests/neutron-6d64d7894b-xp9d5" Dec 03 12:46:16 crc kubenswrapper[5052]: I1203 12:46:16.310078 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/901a2f0b-fb8e-4c97-aea7-567d7ee42a23-ovndb-tls-certs\") pod \"neutron-6d64d7894b-xp9d5\" (UID: \"901a2f0b-fb8e-4c97-aea7-567d7ee42a23\") " pod="openstack-kuttl-tests/neutron-6d64d7894b-xp9d5" Dec 03 12:46:16 crc kubenswrapper[5052]: I1203 12:46:16.310170 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/901a2f0b-fb8e-4c97-aea7-567d7ee42a23-internal-tls-certs\") pod \"neutron-6d64d7894b-xp9d5\" (UID: \"901a2f0b-fb8e-4c97-aea7-567d7ee42a23\") " pod="openstack-kuttl-tests/neutron-6d64d7894b-xp9d5" Dec 03 12:46:16 crc kubenswrapper[5052]: I1203 12:46:16.310471 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage11-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage11-crc\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:16 crc kubenswrapper[5052]: I1203 12:46:16.310484 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage07-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage07-crc\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:16 crc kubenswrapper[5052]: E1203 12:46:16.310597 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-neutron-internal-svc: secret "cert-neutron-internal-svc" not found Dec 03 12:46:16 crc kubenswrapper[5052]: E1203 12:46:16.310652 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/901a2f0b-fb8e-4c97-aea7-567d7ee42a23-internal-tls-certs podName:901a2f0b-fb8e-4c97-aea7-567d7ee42a23 nodeName:}" failed. No retries permitted until 2025-12-03 12:46:20.310632284 +0000 UTC m=+5170.898874048 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "internal-tls-certs" (UniqueName: "kubernetes.io/secret/901a2f0b-fb8e-4c97-aea7-567d7ee42a23-internal-tls-certs") pod "neutron-6d64d7894b-xp9d5" (UID: "901a2f0b-fb8e-4c97-aea7-567d7ee42a23") : secret "cert-neutron-internal-svc" not found Dec 03 12:46:16 crc kubenswrapper[5052]: E1203 12:46:16.311043 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-neutron-ovndbs: secret "cert-neutron-ovndbs" not found Dec 03 12:46:16 crc kubenswrapper[5052]: E1203 12:46:16.311138 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/901a2f0b-fb8e-4c97-aea7-567d7ee42a23-ovndb-tls-certs podName:901a2f0b-fb8e-4c97-aea7-567d7ee42a23 nodeName:}" failed. No retries permitted until 2025-12-03 12:46:20.311115388 +0000 UTC m=+5170.899357222 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "ovndb-tls-certs" (UniqueName: "kubernetes.io/secret/901a2f0b-fb8e-4c97-aea7-567d7ee42a23-ovndb-tls-certs") pod "neutron-6d64d7894b-xp9d5" (UID: "901a2f0b-fb8e-4c97-aea7-567d7ee42a23") : secret "cert-neutron-ovndbs" not found Dec 03 12:46:16 crc kubenswrapper[5052]: E1203 12:46:16.311197 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-neutron-public-svc: secret "cert-neutron-public-svc" not found Dec 03 12:46:16 crc kubenswrapper[5052]: E1203 12:46:16.311228 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/901a2f0b-fb8e-4c97-aea7-567d7ee42a23-public-tls-certs podName:901a2f0b-fb8e-4c97-aea7-567d7ee42a23 nodeName:}" failed. No retries permitted until 2025-12-03 12:46:20.311220101 +0000 UTC m=+5170.899461945 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "public-tls-certs" (UniqueName: "kubernetes.io/secret/901a2f0b-fb8e-4c97-aea7-567d7ee42a23-public-tls-certs") pod "neutron-6d64d7894b-xp9d5" (UID: "901a2f0b-fb8e-4c97-aea7-567d7ee42a23") : secret "cert-neutron-public-svc" not found Dec 03 12:46:16 crc kubenswrapper[5052]: I1203 12:46:16.331348 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f7e26e55-1426-4cd4-b169-a63868acdfaa-config-data" (OuterVolumeSpecName: "config-data") pod "f7e26e55-1426-4cd4-b169-a63868acdfaa" (UID: "f7e26e55-1426-4cd4-b169-a63868acdfaa"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:46:16 crc kubenswrapper[5052]: I1203 12:46:16.372951 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6475384f-1965-4e8b-8806-af420d1381de-openstack-config" (OuterVolumeSpecName: "openstack-config") pod "6475384f-1965-4e8b-8806-af420d1381de" (UID: "6475384f-1965-4e8b-8806-af420d1381de"). InnerVolumeSpecName "openstack-config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:46:16 crc kubenswrapper[5052]: I1203 12:46:16.373699 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8fa85200-471e-4fba-9884-ef25328ebf64-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "8fa85200-471e-4fba-9884-ef25328ebf64" (UID: "8fa85200-471e-4fba-9884-ef25328ebf64"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:46:16 crc kubenswrapper[5052]: I1203 12:46:16.391556 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dfda1fd2-f923-47bd-bf43-8a1be3f79f09-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "dfda1fd2-f923-47bd-bf43-8a1be3f79f09" (UID: "dfda1fd2-f923-47bd-bf43-8a1be3f79f09"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:46:16 crc kubenswrapper[5052]: I1203 12:46:16.395642 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dfda1fd2-f923-47bd-bf43-8a1be3f79f09-config-data" (OuterVolumeSpecName: "config-data") pod "dfda1fd2-f923-47bd-bf43-8a1be3f79f09" (UID: "dfda1fd2-f923-47bd-bf43-8a1be3f79f09"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:46:16 crc kubenswrapper[5052]: I1203 12:46:16.414551 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovsdbserver-sb-tls-certs\" (UniqueName: \"kubernetes.io/secret/c6c9b62e-bbbc-4557-92fd-da64bd8ee0cf-ovsdbserver-sb-tls-certs\") pod \"ovsdbserver-sb-0\" (UID: \"c6c9b62e-bbbc-4557-92fd-da64bd8ee0cf\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:46:16 crc kubenswrapper[5052]: I1203 12:46:16.415078 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"vencrypt-tls-certs\" (UniqueName: \"kubernetes.io/secret/9c996590-4337-477e-a236-3e834d192346-vencrypt-tls-certs\") pod \"nova-cell1-novncproxy-0\" (UID: \"9c996590-4337-477e-a236-3e834d192346\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:46:16 crc kubenswrapper[5052]: I1203 12:46:16.415390 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"nova-novncproxy-tls-certs\" (UniqueName: \"kubernetes.io/secret/9c996590-4337-477e-a236-3e834d192346-nova-novncproxy-tls-certs\") pod \"nova-cell1-novncproxy-0\" (UID: \"9c996590-4337-477e-a236-3e834d192346\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:46:16 crc kubenswrapper[5052]: I1203 12:46:16.415654 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/c6c9b62e-bbbc-4557-92fd-da64bd8ee0cf-metrics-certs-tls-certs\") pod \"ovsdbserver-sb-0\" (UID: \"c6c9b62e-bbbc-4557-92fd-da64bd8ee0cf\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:46:16 crc kubenswrapper[5052]: I1203 12:46:16.414682 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2c53ea4e-56ba-4ba1-8cb6-8c24725b5856-galera-tls-certs" (OuterVolumeSpecName: "galera-tls-certs") pod "2c53ea4e-56ba-4ba1-8cb6-8c24725b5856" (UID: "2c53ea4e-56ba-4ba1-8cb6-8c24725b5856"). InnerVolumeSpecName "galera-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:46:16 crc kubenswrapper[5052]: E1203 12:46:16.414718 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-ovndbcluster-sb-ovndbs: secret "cert-ovndbcluster-sb-ovndbs" not found Dec 03 12:46:16 crc kubenswrapper[5052]: E1203 12:46:16.416144 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-ovn-metrics: secret "cert-ovn-metrics" not found Dec 03 12:46:16 crc kubenswrapper[5052]: E1203 12:46:16.417258 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c6c9b62e-bbbc-4557-92fd-da64bd8ee0cf-ovsdbserver-sb-tls-certs podName:c6c9b62e-bbbc-4557-92fd-da64bd8ee0cf nodeName:}" failed. No retries permitted until 2025-12-03 12:46:17.417214782 +0000 UTC m=+5168.005456546 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "ovsdbserver-sb-tls-certs" (UniqueName: "kubernetes.io/secret/c6c9b62e-bbbc-4557-92fd-da64bd8ee0cf-ovsdbserver-sb-tls-certs") pod "ovsdbserver-sb-0" (UID: "c6c9b62e-bbbc-4557-92fd-da64bd8ee0cf") : secret "cert-ovndbcluster-sb-ovndbs" not found Dec 03 12:46:16 crc kubenswrapper[5052]: E1203 12:46:16.417822 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c6c9b62e-bbbc-4557-92fd-da64bd8ee0cf-metrics-certs-tls-certs podName:c6c9b62e-bbbc-4557-92fd-da64bd8ee0cf nodeName:}" failed. No retries permitted until 2025-12-03 12:46:17.417809939 +0000 UTC m=+5168.006051703 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs-tls-certs" (UniqueName: "kubernetes.io/secret/c6c9b62e-bbbc-4557-92fd-da64bd8ee0cf-metrics-certs-tls-certs") pod "ovsdbserver-sb-0" (UID: "c6c9b62e-bbbc-4557-92fd-da64bd8ee0cf") : secret "cert-ovn-metrics" not found Dec 03 12:46:16 crc kubenswrapper[5052]: E1203 12:46:16.419046 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-nova-novncproxy-cell1-public-svc: secret "cert-nova-novncproxy-cell1-public-svc" not found Dec 03 12:46:16 crc kubenswrapper[5052]: I1203 12:46:16.419095 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/dfda1fd2-f923-47bd-bf43-8a1be3f79f09-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:16 crc kubenswrapper[5052]: I1203 12:46:16.421485 5052 reconciler_common.go:293] "Volume detached for volume \"openstack-config\" (UniqueName: \"kubernetes.io/configmap/6475384f-1965-4e8b-8806-af420d1381de-openstack-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:16 crc kubenswrapper[5052]: I1203 12:46:16.421526 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8fa85200-471e-4fba-9884-ef25328ebf64-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:16 crc kubenswrapper[5052]: I1203 12:46:16.421540 5052 reconciler_common.go:293] "Volume detached for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/2c53ea4e-56ba-4ba1-8cb6-8c24725b5856-galera-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:16 crc kubenswrapper[5052]: I1203 12:46:16.421553 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/f7e26e55-1426-4cd4-b169-a63868acdfaa-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:16 crc kubenswrapper[5052]: I1203 12:46:16.421564 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/dfda1fd2-f923-47bd-bf43-8a1be3f79f09-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:16 crc kubenswrapper[5052]: E1203 12:46:16.419078 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-nova-novncproxy-cell1-vencrypt: secret "cert-nova-novncproxy-cell1-vencrypt" not found Dec 03 12:46:16 crc kubenswrapper[5052]: E1203 12:46:16.421701 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9c996590-4337-477e-a236-3e834d192346-vencrypt-tls-certs podName:9c996590-4337-477e-a236-3e834d192346 nodeName:}" failed. No retries permitted until 2025-12-03 12:46:17.42168315 +0000 UTC m=+5168.009924914 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "vencrypt-tls-certs" (UniqueName: "kubernetes.io/secret/9c996590-4337-477e-a236-3e834d192346-vencrypt-tls-certs") pod "nova-cell1-novncproxy-0" (UID: "9c996590-4337-477e-a236-3e834d192346") : secret "cert-nova-novncproxy-cell1-vencrypt" not found Dec 03 12:46:16 crc kubenswrapper[5052]: E1203 12:46:16.422120 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9c996590-4337-477e-a236-3e834d192346-nova-novncproxy-tls-certs podName:9c996590-4337-477e-a236-3e834d192346 nodeName:}" failed. No retries permitted until 2025-12-03 12:46:17.422054581 +0000 UTC m=+5168.010296425 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "nova-novncproxy-tls-certs" (UniqueName: "kubernetes.io/secret/9c996590-4337-477e-a236-3e834d192346-nova-novncproxy-tls-certs") pod "nova-cell1-novncproxy-0" (UID: "9c996590-4337-477e-a236-3e834d192346") : secret "cert-nova-novncproxy-cell1-public-svc" not found Dec 03 12:46:16 crc kubenswrapper[5052]: I1203 12:46:16.426726 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6475384f-1965-4e8b-8806-af420d1381de-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "6475384f-1965-4e8b-8806-af420d1381de" (UID: "6475384f-1965-4e8b-8806-af420d1381de"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:46:16 crc kubenswrapper[5052]: I1203 12:46:16.443995 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4eab5b48-f674-4100-9b42-f3ea257d47d4-sg-core-conf-yaml" (OuterVolumeSpecName: "sg-core-conf-yaml") pod "4eab5b48-f674-4100-9b42-f3ea257d47d4" (UID: "4eab5b48-f674-4100-9b42-f3ea257d47d4"). InnerVolumeSpecName "sg-core-conf-yaml". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:46:16 crc kubenswrapper[5052]: I1203 12:46:16.450937 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2c53ea4e-56ba-4ba1-8cb6-8c24725b5856-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "2c53ea4e-56ba-4ba1-8cb6-8c24725b5856" (UID: "2c53ea4e-56ba-4ba1-8cb6-8c24725b5856"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:46:16 crc kubenswrapper[5052]: I1203 12:46:16.453869 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage01-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage01-crc\") pod \"ovsdbserver-nb-0\" (UID: \"29c7d055-4f37-45a4-a608-eb73a361a86c\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:46:16 crc kubenswrapper[5052]: I1203 12:46:16.474739 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2f966eeb-76ef-4771-acdf-e0de2bb463a4-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "2f966eeb-76ef-4771-acdf-e0de2bb463a4" (UID: "2f966eeb-76ef-4771-acdf-e0de2bb463a4"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:46:16 crc kubenswrapper[5052]: I1203 12:46:16.526402 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6475384f-1965-4e8b-8806-af420d1381de-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:16 crc kubenswrapper[5052]: I1203 12:46:16.526444 5052 reconciler_common.go:293] "Volume detached for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/4eab5b48-f674-4100-9b42-f3ea257d47d4-sg-core-conf-yaml\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:16 crc kubenswrapper[5052]: I1203 12:46:16.526465 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2c53ea4e-56ba-4ba1-8cb6-8c24725b5856-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:16 crc kubenswrapper[5052]: I1203 12:46:16.526480 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2f966eeb-76ef-4771-acdf-e0de2bb463a4-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:16 crc kubenswrapper[5052]: I1203 12:46:16.531419 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4eab5b48-f674-4100-9b42-f3ea257d47d4-ceilometer-tls-certs" (OuterVolumeSpecName: "ceilometer-tls-certs") pod "4eab5b48-f674-4100-9b42-f3ea257d47d4" (UID: "4eab5b48-f674-4100-9b42-f3ea257d47d4"). InnerVolumeSpecName "ceilometer-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:46:16 crc kubenswrapper[5052]: I1203 12:46:16.548034 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:46:16 crc kubenswrapper[5052]: I1203 12:46:16.547214 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"4eab5b48-f674-4100-9b42-f3ea257d47d4","Type":"ContainerDied","Data":"729c50e16fe2bc63c224552fab14a703ef50089d0da8fe68249c1879afea0e05"} Dec 03 12:46:16 crc kubenswrapper[5052]: I1203 12:46:16.550173 5052 scope.go:117] "RemoveContainer" containerID="01c4f4342abe4df4b3d520a97ffb3f4ba0abfe85cb81b459e64f94869c4a5f91" Dec 03 12:46:16 crc kubenswrapper[5052]: I1203 12:46:16.567192 5052 generic.go:334] "Generic (PLEG): container finished" podID="9aea20a7-eaed-43a8-9b52-08c2c0212230" containerID="3242b025201bb8378bb8f5f363d45551adfc8da4587f92d211c75e0e901ef11a" exitCode=0 Dec 03 12:46:16 crc kubenswrapper[5052]: I1203 12:46:16.567279 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"9aea20a7-eaed-43a8-9b52-08c2c0212230","Type":"ContainerDied","Data":"3242b025201bb8378bb8f5f363d45551adfc8da4587f92d211c75e0e901ef11a"} Dec 03 12:46:16 crc kubenswrapper[5052]: I1203 12:46:16.583646 5052 generic.go:334] "Generic (PLEG): container finished" podID="78361066-6add-4971-9a4b-9157f35ad494" containerID="3310a6083ad0bd13b8590b4fa080eeb83719e19bfb1dddf441320e21f33c7afa" exitCode=0 Dec 03 12:46:16 crc kubenswrapper[5052]: I1203 12:46:16.583750 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"78361066-6add-4971-9a4b-9157f35ad494","Type":"ContainerDied","Data":"3310a6083ad0bd13b8590b4fa080eeb83719e19bfb1dddf441320e21f33c7afa"} Dec 03 12:46:16 crc kubenswrapper[5052]: I1203 12:46:16.592592 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-keystone-listener-68fd6d997d-7tjjw" event={"ID":"2065f5bb-88eb-4e84-88b6-9982833d413d","Type":"ContainerStarted","Data":"92eeaf97de94d65f0f332a8172dd776af0421be37c42d90e9dab8739d2cb83e3"} Dec 03 12:46:16 crc kubenswrapper[5052]: I1203 12:46:16.614590 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8fa85200-471e-4fba-9884-ef25328ebf64-galera-tls-certs" (OuterVolumeSpecName: "galera-tls-certs") pod "8fa85200-471e-4fba-9884-ef25328ebf64" (UID: "8fa85200-471e-4fba-9884-ef25328ebf64"). InnerVolumeSpecName "galera-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:46:16 crc kubenswrapper[5052]: I1203 12:46:16.620845 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2f966eeb-76ef-4771-acdf-e0de2bb463a4-memcached-tls-certs" (OuterVolumeSpecName: "memcached-tls-certs") pod "2f966eeb-76ef-4771-acdf-e0de2bb463a4" (UID: "2f966eeb-76ef-4771-acdf-e0de2bb463a4"). InnerVolumeSpecName "memcached-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:46:16 crc kubenswrapper[5052]: I1203 12:46:16.621472 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-worker-58f7cb7fd9-2k5tg" event={"ID":"e14465ed-868a-49ba-b3ab-1e67196229e8","Type":"ContainerStarted","Data":"805cd92e51a39e19c7bde5260dca999018b3ba92ebb95c79b560a3986eb6d94a"} Dec 03 12:46:16 crc kubenswrapper[5052]: I1203 12:46:16.624137 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-bbcfbbc9d-5q6gx" event={"ID":"32f420c2-aa7e-44c8-a63a-7654aa3e6e34","Type":"ContainerStarted","Data":"4c8ae2743fa5049ef6f420e9338425946c81202ba97ccaf63ece2ed399a93b82"} Dec 03 12:46:16 crc kubenswrapper[5052]: I1203 12:46:16.624454 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/barbican-keystone-listener-68fd6d997d-7tjjw" podStartSLOduration=5.624442589 podStartE2EDuration="5.624442589s" podCreationTimestamp="2025-12-03 12:46:11 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:46:16.615216925 +0000 UTC m=+5167.203458689" watchObservedRunningTime="2025-12-03 12:46:16.624442589 +0000 UTC m=+5167.212684353" Dec 03 12:46:16 crc kubenswrapper[5052]: I1203 12:46:16.637766 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6475384f-1965-4e8b-8806-af420d1381de-openstack-config-secret" (OuterVolumeSpecName: "openstack-config-secret") pod "6475384f-1965-4e8b-8806-af420d1381de" (UID: "6475384f-1965-4e8b-8806-af420d1381de"). InnerVolumeSpecName "openstack-config-secret". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:46:16 crc kubenswrapper[5052]: I1203 12:46:16.637844 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4eab5b48-f674-4100-9b42-f3ea257d47d4-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "4eab5b48-f674-4100-9b42-f3ea257d47d4" (UID: "4eab5b48-f674-4100-9b42-f3ea257d47d4"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:46:16 crc kubenswrapper[5052]: I1203 12:46:16.642797 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/29c7d055-4f37-45a4-a608-eb73a361a86c-metrics-certs-tls-certs\") pod \"ovsdbserver-nb-0\" (UID: \"29c7d055-4f37-45a4-a608-eb73a361a86c\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:46:16 crc kubenswrapper[5052]: I1203 12:46:16.643045 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovsdbserver-nb-tls-certs\" (UniqueName: \"kubernetes.io/secret/29c7d055-4f37-45a4-a608-eb73a361a86c-ovsdbserver-nb-tls-certs\") pod \"ovsdbserver-nb-0\" (UID: \"29c7d055-4f37-45a4-a608-eb73a361a86c\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:46:16 crc kubenswrapper[5052]: I1203 12:46:16.643244 5052 reconciler_common.go:293] "Volume detached for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/4eab5b48-f674-4100-9b42-f3ea257d47d4-ceilometer-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:16 crc kubenswrapper[5052]: I1203 12:46:16.643260 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4eab5b48-f674-4100-9b42-f3ea257d47d4-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:16 crc kubenswrapper[5052]: I1203 12:46:16.643274 5052 reconciler_common.go:293] "Volume detached for volume \"memcached-tls-certs\" (UniqueName: \"kubernetes.io/secret/2f966eeb-76ef-4771-acdf-e0de2bb463a4-memcached-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:16 crc kubenswrapper[5052]: I1203 12:46:16.643285 5052 reconciler_common.go:293] "Volume detached for volume \"openstack-config-secret\" (UniqueName: \"kubernetes.io/secret/6475384f-1965-4e8b-8806-af420d1381de-openstack-config-secret\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:16 crc kubenswrapper[5052]: I1203 12:46:16.643296 5052 reconciler_common.go:293] "Volume detached for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/8fa85200-471e-4fba-9884-ef25328ebf64-galera-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:16 crc kubenswrapper[5052]: E1203 12:46:16.643408 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-ovndbcluster-nb-ovndbs: secret "cert-ovndbcluster-nb-ovndbs" not found Dec 03 12:46:16 crc kubenswrapper[5052]: E1203 12:46:16.643457 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/29c7d055-4f37-45a4-a608-eb73a361a86c-ovsdbserver-nb-tls-certs podName:29c7d055-4f37-45a4-a608-eb73a361a86c nodeName:}" failed. No retries permitted until 2025-12-03 12:46:17.643443462 +0000 UTC m=+5168.231685226 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "ovsdbserver-nb-tls-certs" (UniqueName: "kubernetes.io/secret/29c7d055-4f37-45a4-a608-eb73a361a86c-ovsdbserver-nb-tls-certs") pod "ovsdbserver-nb-0" (UID: "29c7d055-4f37-45a4-a608-eb73a361a86c") : secret "cert-ovndbcluster-nb-ovndbs" not found Dec 03 12:46:16 crc kubenswrapper[5052]: E1203 12:46:16.644987 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-ovn-metrics: secret "cert-ovn-metrics" not found Dec 03 12:46:16 crc kubenswrapper[5052]: E1203 12:46:16.645034 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/29c7d055-4f37-45a4-a608-eb73a361a86c-metrics-certs-tls-certs podName:29c7d055-4f37-45a4-a608-eb73a361a86c nodeName:}" failed. No retries permitted until 2025-12-03 12:46:17.645021727 +0000 UTC m=+5168.233263491 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs-tls-certs" (UniqueName: "kubernetes.io/secret/29c7d055-4f37-45a4-a608-eb73a361a86c-metrics-certs-tls-certs") pod "ovsdbserver-nb-0" (UID: "29c7d055-4f37-45a4-a608-eb73a361a86c") : secret "cert-ovn-metrics" not found Dec 03 12:46:16 crc kubenswrapper[5052]: I1203 12:46:16.647137 5052 generic.go:334] "Generic (PLEG): container finished" podID="fb6c11ac-1b0e-4e63-8988-256b2504ebc7" containerID="c2c82ad1c4ab93ed36cb072abbe6d1eeb5753acbd507bfb1d2fbc3bacdb79bdf" exitCode=0 Dec 03 12:46:16 crc kubenswrapper[5052]: I1203 12:46:16.647205 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"fb6c11ac-1b0e-4e63-8988-256b2504ebc7","Type":"ContainerDied","Data":"c2c82ad1c4ab93ed36cb072abbe6d1eeb5753acbd507bfb1d2fbc3bacdb79bdf"} Dec 03 12:46:16 crc kubenswrapper[5052]: I1203 12:46:16.648795 5052 generic.go:334] "Generic (PLEG): container finished" podID="9af09a5b-789c-4b83-b899-9d1b3f0d8603" containerID="9ba6898618d6ecc3b57802d08d86da02201fb0d67fcd7288c3414ce61b3233ed" exitCode=0 Dec 03 12:46:16 crc kubenswrapper[5052]: I1203 12:46:16.648844 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"9af09a5b-789c-4b83-b899-9d1b3f0d8603","Type":"ContainerDied","Data":"9ba6898618d6ecc3b57802d08d86da02201fb0d67fcd7288c3414ce61b3233ed"} Dec 03 12:46:16 crc kubenswrapper[5052]: I1203 12:46:16.649800 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:46:16 crc kubenswrapper[5052]: I1203 12:46:16.652051 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/memcached-0" Dec 03 12:46:16 crc kubenswrapper[5052]: I1203 12:46:16.654565 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-keystone-listener-769bc98c4b-mnz6r"] Dec 03 12:46:16 crc kubenswrapper[5052]: I1203 12:46:16.654775 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/barbican-keystone-listener-769bc98c4b-mnz6r" podUID="1d187c5a-7745-447f-a8e2-06c7528e5be0" containerName="barbican-keystone-listener-log" containerID="cri-o://4208df0572c14c762c41a0bceeeb54f12d57e83a373e6e10caba8b5459e15ce7" gracePeriod=30 Dec 03 12:46:16 crc kubenswrapper[5052]: I1203 12:46:16.654882 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/barbican-keystone-listener-769bc98c4b-mnz6r" podUID="1d187c5a-7745-447f-a8e2-06c7528e5be0" containerName="barbican-keystone-listener" containerID="cri-o://0b52d2ae1f61eeac5291c66875e41de2e8c84d29873caba7e0a0412afcac4845" gracePeriod=30 Dec 03 12:46:16 crc kubenswrapper[5052]: I1203 12:46:16.655042 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:46:16 crc kubenswrapper[5052]: I1203 12:46:16.659374 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:46:16 crc kubenswrapper[5052]: I1203 12:46:16.666095 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:46:16 crc kubenswrapper[5052]: I1203 12:46:16.673857 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstackclient" Dec 03 12:46:16 crc kubenswrapper[5052]: I1203 12:46:16.730340 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/cinder-api-0" podUID="68a80c17-9f7d-47de-b02c-5e7bb47656ab" containerName="cinder-api" probeResult="failure" output="Get \"https://10.217.1.195:8776/healthcheck\": read tcp 10.217.0.2:35148->10.217.1.195:8776: read: connection reset by peer" Dec 03 12:46:16 crc kubenswrapper[5052]: I1203 12:46:16.799286 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4eab5b48-f674-4100-9b42-f3ea257d47d4-config-data" (OuterVolumeSpecName: "config-data") pod "4eab5b48-f674-4100-9b42-f3ea257d47d4" (UID: "4eab5b48-f674-4100-9b42-f3ea257d47d4"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:46:16 crc kubenswrapper[5052]: I1203 12:46:16.851405 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/4eab5b48-f674-4100-9b42-f3ea257d47d4-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:17 crc kubenswrapper[5052]: I1203 12:46:17.411633 5052 scope.go:117] "RemoveContainer" containerID="7370547fbe4eb4455656b595065ce614afd49fa9ea95ccde635adb0b612d3c70" Dec 03 12:46:17 crc kubenswrapper[5052]: I1203 12:46:17.431846 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:46:17 crc kubenswrapper[5052]: I1203 12:46:17.478111 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9af09a5b-789c-4b83-b899-9d1b3f0d8603-combined-ca-bundle\") pod \"9af09a5b-789c-4b83-b899-9d1b3f0d8603\" (UID: \"9af09a5b-789c-4b83-b899-9d1b3f0d8603\") " Dec 03 12:46:17 crc kubenswrapper[5052]: I1203 12:46:17.478255 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/9af09a5b-789c-4b83-b899-9d1b3f0d8603-nova-metadata-tls-certs\") pod \"9af09a5b-789c-4b83-b899-9d1b3f0d8603\" (UID: \"9af09a5b-789c-4b83-b899-9d1b3f0d8603\") " Dec 03 12:46:17 crc kubenswrapper[5052]: I1203 12:46:17.478278 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/9af09a5b-789c-4b83-b899-9d1b3f0d8603-logs\") pod \"9af09a5b-789c-4b83-b899-9d1b3f0d8603\" (UID: \"9af09a5b-789c-4b83-b899-9d1b3f0d8603\") " Dec 03 12:46:17 crc kubenswrapper[5052]: I1203 12:46:17.478392 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/9af09a5b-789c-4b83-b899-9d1b3f0d8603-config-data\") pod \"9af09a5b-789c-4b83-b899-9d1b3f0d8603\" (UID: \"9af09a5b-789c-4b83-b899-9d1b3f0d8603\") " Dec 03 12:46:17 crc kubenswrapper[5052]: I1203 12:46:17.478435 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dhpjf\" (UniqueName: \"kubernetes.io/projected/9af09a5b-789c-4b83-b899-9d1b3f0d8603-kube-api-access-dhpjf\") pod \"9af09a5b-789c-4b83-b899-9d1b3f0d8603\" (UID: \"9af09a5b-789c-4b83-b899-9d1b3f0d8603\") " Dec 03 12:46:17 crc kubenswrapper[5052]: I1203 12:46:17.478785 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovsdbserver-sb-tls-certs\" (UniqueName: \"kubernetes.io/secret/c6c9b62e-bbbc-4557-92fd-da64bd8ee0cf-ovsdbserver-sb-tls-certs\") pod \"ovsdbserver-sb-0\" (UID: \"c6c9b62e-bbbc-4557-92fd-da64bd8ee0cf\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:46:17 crc kubenswrapper[5052]: I1203 12:46:17.478814 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"vencrypt-tls-certs\" (UniqueName: \"kubernetes.io/secret/9c996590-4337-477e-a236-3e834d192346-vencrypt-tls-certs\") pod \"nova-cell1-novncproxy-0\" (UID: \"9c996590-4337-477e-a236-3e834d192346\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:46:17 crc kubenswrapper[5052]: I1203 12:46:17.478876 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"nova-novncproxy-tls-certs\" (UniqueName: \"kubernetes.io/secret/9c996590-4337-477e-a236-3e834d192346-nova-novncproxy-tls-certs\") pod \"nova-cell1-novncproxy-0\" (UID: \"9c996590-4337-477e-a236-3e834d192346\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:46:17 crc kubenswrapper[5052]: I1203 12:46:17.478919 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/c6c9b62e-bbbc-4557-92fd-da64bd8ee0cf-metrics-certs-tls-certs\") pod \"ovsdbserver-sb-0\" (UID: \"c6c9b62e-bbbc-4557-92fd-da64bd8ee0cf\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:46:17 crc kubenswrapper[5052]: E1203 12:46:17.479087 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-ovn-metrics: secret "cert-ovn-metrics" not found Dec 03 12:46:17 crc kubenswrapper[5052]: E1203 12:46:17.479132 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c6c9b62e-bbbc-4557-92fd-da64bd8ee0cf-metrics-certs-tls-certs podName:c6c9b62e-bbbc-4557-92fd-da64bd8ee0cf nodeName:}" failed. No retries permitted until 2025-12-03 12:46:19.479118381 +0000 UTC m=+5170.067360145 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs-tls-certs" (UniqueName: "kubernetes.io/secret/c6c9b62e-bbbc-4557-92fd-da64bd8ee0cf-metrics-certs-tls-certs") pod "ovsdbserver-sb-0" (UID: "c6c9b62e-bbbc-4557-92fd-da64bd8ee0cf") : secret "cert-ovn-metrics" not found Dec 03 12:46:17 crc kubenswrapper[5052]: E1203 12:46:17.503579 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-nova-novncproxy-cell1-vencrypt: secret "cert-nova-novncproxy-cell1-vencrypt" not found Dec 03 12:46:17 crc kubenswrapper[5052]: E1203 12:46:17.503672 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9c996590-4337-477e-a236-3e834d192346-vencrypt-tls-certs podName:9c996590-4337-477e-a236-3e834d192346 nodeName:}" failed. No retries permitted until 2025-12-03 12:46:19.503649743 +0000 UTC m=+5170.091891507 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "vencrypt-tls-certs" (UniqueName: "kubernetes.io/secret/9c996590-4337-477e-a236-3e834d192346-vencrypt-tls-certs") pod "nova-cell1-novncproxy-0" (UID: "9c996590-4337-477e-a236-3e834d192346") : secret "cert-nova-novncproxy-cell1-vencrypt" not found Dec 03 12:46:17 crc kubenswrapper[5052]: I1203 12:46:17.503900 5052 scope.go:117] "RemoveContainer" containerID="d0d21a72bcbb3c194d8e900bc8615778059b26e70f1614cda87f183610dac51e" Dec 03 12:46:17 crc kubenswrapper[5052]: E1203 12:46:17.504013 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-ovndbcluster-sb-ovndbs: secret "cert-ovndbcluster-sb-ovndbs" not found Dec 03 12:46:17 crc kubenswrapper[5052]: E1203 12:46:17.504052 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c6c9b62e-bbbc-4557-92fd-da64bd8ee0cf-ovsdbserver-sb-tls-certs podName:c6c9b62e-bbbc-4557-92fd-da64bd8ee0cf nodeName:}" failed. No retries permitted until 2025-12-03 12:46:19.504041114 +0000 UTC m=+5170.092282878 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "ovsdbserver-sb-tls-certs" (UniqueName: "kubernetes.io/secret/c6c9b62e-bbbc-4557-92fd-da64bd8ee0cf-ovsdbserver-sb-tls-certs") pod "ovsdbserver-sb-0" (UID: "c6c9b62e-bbbc-4557-92fd-da64bd8ee0cf") : secret "cert-ovndbcluster-sb-ovndbs" not found Dec 03 12:46:17 crc kubenswrapper[5052]: E1203 12:46:17.504173 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-nova-novncproxy-cell1-public-svc: secret "cert-nova-novncproxy-cell1-public-svc" not found Dec 03 12:46:17 crc kubenswrapper[5052]: E1203 12:46:17.504246 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9c996590-4337-477e-a236-3e834d192346-nova-novncproxy-tls-certs podName:9c996590-4337-477e-a236-3e834d192346 nodeName:}" failed. No retries permitted until 2025-12-03 12:46:19.504225609 +0000 UTC m=+5170.092467433 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "nova-novncproxy-tls-certs" (UniqueName: "kubernetes.io/secret/9c996590-4337-477e-a236-3e834d192346-nova-novncproxy-tls-certs") pod "nova-cell1-novncproxy-0" (UID: "9c996590-4337-477e-a236-3e834d192346") : secret "cert-nova-novncproxy-cell1-public-svc" not found Dec 03 12:46:17 crc kubenswrapper[5052]: I1203 12:46:17.504265 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:46:17 crc kubenswrapper[5052]: I1203 12:46:17.505462 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9af09a5b-789c-4b83-b899-9d1b3f0d8603-logs" (OuterVolumeSpecName: "logs") pod "9af09a5b-789c-4b83-b899-9d1b3f0d8603" (UID: "9af09a5b-789c-4b83-b899-9d1b3f0d8603"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:46:17 crc kubenswrapper[5052]: I1203 12:46:17.529747 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9af09a5b-789c-4b83-b899-9d1b3f0d8603-kube-api-access-dhpjf" (OuterVolumeSpecName: "kube-api-access-dhpjf") pod "9af09a5b-789c-4b83-b899-9d1b3f0d8603" (UID: "9af09a5b-789c-4b83-b899-9d1b3f0d8603"). InnerVolumeSpecName "kube-api-access-dhpjf". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:46:17 crc kubenswrapper[5052]: I1203 12:46:17.557787 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9af09a5b-789c-4b83-b899-9d1b3f0d8603-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "9af09a5b-789c-4b83-b899-9d1b3f0d8603" (UID: "9af09a5b-789c-4b83-b899-9d1b3f0d8603"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:46:17 crc kubenswrapper[5052]: I1203 12:46:17.565039 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:46:17 crc kubenswrapper[5052]: I1203 12:46:17.580316 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:46:17 crc kubenswrapper[5052]: I1203 12:46:17.580753 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/9af09a5b-789c-4b83-b899-9d1b3f0d8603-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:17 crc kubenswrapper[5052]: I1203 12:46:17.580784 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-dhpjf\" (UniqueName: \"kubernetes.io/projected/9af09a5b-789c-4b83-b899-9d1b3f0d8603-kube-api-access-dhpjf\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:17 crc kubenswrapper[5052]: I1203 12:46:17.580797 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9af09a5b-789c-4b83-b899-9d1b3f0d8603-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:17 crc kubenswrapper[5052]: I1203 12:46:17.581175 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:46:17 crc kubenswrapper[5052]: I1203 12:46:17.585788 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9af09a5b-789c-4b83-b899-9d1b3f0d8603-config-data" (OuterVolumeSpecName: "config-data") pod "9af09a5b-789c-4b83-b899-9d1b3f0d8603" (UID: "9af09a5b-789c-4b83-b899-9d1b3f0d8603"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:46:17 crc kubenswrapper[5052]: I1203 12:46:17.598396 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:46:17 crc kubenswrapper[5052]: I1203 12:46:17.647753 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:46:17 crc kubenswrapper[5052]: I1203 12:46:17.680079 5052 scope.go:117] "RemoveContainer" containerID="c9ba5984640e4b46f378e76e1ad9dbbc5b996fff2d3c392ba7de3ff6753a48fd" Dec 03 12:46:17 crc kubenswrapper[5052]: I1203 12:46:17.699205 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/68a80c17-9f7d-47de-b02c-5e7bb47656ab-etc-machine-id\") pod \"68a80c17-9f7d-47de-b02c-5e7bb47656ab\" (UID: \"68a80c17-9f7d-47de-b02c-5e7bb47656ab\") " Dec 03 12:46:17 crc kubenswrapper[5052]: I1203 12:46:17.699273 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nltzf\" (UniqueName: \"kubernetes.io/projected/68a80c17-9f7d-47de-b02c-5e7bb47656ab-kube-api-access-nltzf\") pod \"68a80c17-9f7d-47de-b02c-5e7bb47656ab\" (UID: \"68a80c17-9f7d-47de-b02c-5e7bb47656ab\") " Dec 03 12:46:17 crc kubenswrapper[5052]: I1203 12:46:17.699307 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/fb6c11ac-1b0e-4e63-8988-256b2504ebc7-internal-tls-certs\") pod \"fb6c11ac-1b0e-4e63-8988-256b2504ebc7\" (UID: \"fb6c11ac-1b0e-4e63-8988-256b2504ebc7\") " Dec 03 12:46:17 crc kubenswrapper[5052]: I1203 12:46:17.699327 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/68a80c17-9f7d-47de-b02c-5e7bb47656ab-combined-ca-bundle\") pod \"68a80c17-9f7d-47de-b02c-5e7bb47656ab\" (UID: \"68a80c17-9f7d-47de-b02c-5e7bb47656ab\") " Dec 03 12:46:17 crc kubenswrapper[5052]: I1203 12:46:17.699347 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/78361066-6add-4971-9a4b-9157f35ad494-config-data\") pod \"78361066-6add-4971-9a4b-9157f35ad494\" (UID: \"78361066-6add-4971-9a4b-9157f35ad494\") " Dec 03 12:46:17 crc kubenswrapper[5052]: I1203 12:46:17.699399 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sp7f4\" (UniqueName: \"kubernetes.io/projected/78361066-6add-4971-9a4b-9157f35ad494-kube-api-access-sp7f4\") pod \"78361066-6add-4971-9a4b-9157f35ad494\" (UID: \"78361066-6add-4971-9a4b-9157f35ad494\") " Dec 03 12:46:17 crc kubenswrapper[5052]: I1203 12:46:17.699431 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/fb6c11ac-1b0e-4e63-8988-256b2504ebc7-public-tls-certs\") pod \"fb6c11ac-1b0e-4e63-8988-256b2504ebc7\" (UID: \"fb6c11ac-1b0e-4e63-8988-256b2504ebc7\") " Dec 03 12:46:17 crc kubenswrapper[5052]: I1203 12:46:17.699453 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"glance\" (UniqueName: \"kubernetes.io/local-volume/local-storage18-crc\") pod \"78361066-6add-4971-9a4b-9157f35ad494\" (UID: \"78361066-6add-4971-9a4b-9157f35ad494\") " Dec 03 12:46:17 crc kubenswrapper[5052]: I1203 12:46:17.699479 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/9aea20a7-eaed-43a8-9b52-08c2c0212230-public-tls-certs\") pod \"9aea20a7-eaed-43a8-9b52-08c2c0212230\" (UID: \"9aea20a7-eaed-43a8-9b52-08c2c0212230\") " Dec 03 12:46:17 crc kubenswrapper[5052]: I1203 12:46:17.699535 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/78361066-6add-4971-9a4b-9157f35ad494-logs\") pod \"78361066-6add-4971-9a4b-9157f35ad494\" (UID: \"78361066-6add-4971-9a4b-9157f35ad494\") " Dec 03 12:46:17 crc kubenswrapper[5052]: I1203 12:46:17.699567 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/9aea20a7-eaed-43a8-9b52-08c2c0212230-httpd-run\") pod \"9aea20a7-eaed-43a8-9b52-08c2c0212230\" (UID: \"9aea20a7-eaed-43a8-9b52-08c2c0212230\") " Dec 03 12:46:17 crc kubenswrapper[5052]: I1203 12:46:17.699592 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/9aea20a7-eaed-43a8-9b52-08c2c0212230-logs\") pod \"9aea20a7-eaed-43a8-9b52-08c2c0212230\" (UID: \"9aea20a7-eaed-43a8-9b52-08c2c0212230\") " Dec 03 12:46:17 crc kubenswrapper[5052]: I1203 12:46:17.699617 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9aea20a7-eaed-43a8-9b52-08c2c0212230-combined-ca-bundle\") pod \"9aea20a7-eaed-43a8-9b52-08c2c0212230\" (UID: \"9aea20a7-eaed-43a8-9b52-08c2c0212230\") " Dec 03 12:46:17 crc kubenswrapper[5052]: I1203 12:46:17.699639 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/fb6c11ac-1b0e-4e63-8988-256b2504ebc7-logs\") pod \"fb6c11ac-1b0e-4e63-8988-256b2504ebc7\" (UID: \"fb6c11ac-1b0e-4e63-8988-256b2504ebc7\") " Dec 03 12:46:17 crc kubenswrapper[5052]: I1203 12:46:17.699653 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/78361066-6add-4971-9a4b-9157f35ad494-internal-tls-certs\") pod \"78361066-6add-4971-9a4b-9157f35ad494\" (UID: \"78361066-6add-4971-9a4b-9157f35ad494\") " Dec 03 12:46:17 crc kubenswrapper[5052]: I1203 12:46:17.699692 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/68a80c17-9f7d-47de-b02c-5e7bb47656ab-internal-tls-certs\") pod \"68a80c17-9f7d-47de-b02c-5e7bb47656ab\" (UID: \"68a80c17-9f7d-47de-b02c-5e7bb47656ab\") " Dec 03 12:46:17 crc kubenswrapper[5052]: I1203 12:46:17.699725 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ql9lm\" (UniqueName: \"kubernetes.io/projected/9aea20a7-eaed-43a8-9b52-08c2c0212230-kube-api-access-ql9lm\") pod \"9aea20a7-eaed-43a8-9b52-08c2c0212230\" (UID: \"9aea20a7-eaed-43a8-9b52-08c2c0212230\") " Dec 03 12:46:17 crc kubenswrapper[5052]: I1203 12:46:17.699747 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/78361066-6add-4971-9a4b-9157f35ad494-httpd-run\") pod \"78361066-6add-4971-9a4b-9157f35ad494\" (UID: \"78361066-6add-4971-9a4b-9157f35ad494\") " Dec 03 12:46:17 crc kubenswrapper[5052]: I1203 12:46:17.699772 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/68a80c17-9f7d-47de-b02c-5e7bb47656ab-config-data\") pod \"68a80c17-9f7d-47de-b02c-5e7bb47656ab\" (UID: \"68a80c17-9f7d-47de-b02c-5e7bb47656ab\") " Dec 03 12:46:17 crc kubenswrapper[5052]: I1203 12:46:17.699794 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/78361066-6add-4971-9a4b-9157f35ad494-scripts\") pod \"78361066-6add-4971-9a4b-9157f35ad494\" (UID: \"78361066-6add-4971-9a4b-9157f35ad494\") " Dec 03 12:46:17 crc kubenswrapper[5052]: I1203 12:46:17.699822 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mp72c\" (UniqueName: \"kubernetes.io/projected/fb6c11ac-1b0e-4e63-8988-256b2504ebc7-kube-api-access-mp72c\") pod \"fb6c11ac-1b0e-4e63-8988-256b2504ebc7\" (UID: \"fb6c11ac-1b0e-4e63-8988-256b2504ebc7\") " Dec 03 12:46:17 crc kubenswrapper[5052]: I1203 12:46:17.699854 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/68a80c17-9f7d-47de-b02c-5e7bb47656ab-public-tls-certs\") pod \"68a80c17-9f7d-47de-b02c-5e7bb47656ab\" (UID: \"68a80c17-9f7d-47de-b02c-5e7bb47656ab\") " Dec 03 12:46:17 crc kubenswrapper[5052]: I1203 12:46:17.699870 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"glance\" (UniqueName: \"kubernetes.io/local-volume/local-storage12-crc\") pod \"9aea20a7-eaed-43a8-9b52-08c2c0212230\" (UID: \"9aea20a7-eaed-43a8-9b52-08c2c0212230\") " Dec 03 12:46:17 crc kubenswrapper[5052]: I1203 12:46:17.699888 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/9aea20a7-eaed-43a8-9b52-08c2c0212230-config-data\") pod \"9aea20a7-eaed-43a8-9b52-08c2c0212230\" (UID: \"9aea20a7-eaed-43a8-9b52-08c2c0212230\") " Dec 03 12:46:17 crc kubenswrapper[5052]: I1203 12:46:17.699905 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/9aea20a7-eaed-43a8-9b52-08c2c0212230-scripts\") pod \"9aea20a7-eaed-43a8-9b52-08c2c0212230\" (UID: \"9aea20a7-eaed-43a8-9b52-08c2c0212230\") " Dec 03 12:46:17 crc kubenswrapper[5052]: I1203 12:46:17.699929 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/fb6c11ac-1b0e-4e63-8988-256b2504ebc7-combined-ca-bundle\") pod \"fb6c11ac-1b0e-4e63-8988-256b2504ebc7\" (UID: \"fb6c11ac-1b0e-4e63-8988-256b2504ebc7\") " Dec 03 12:46:17 crc kubenswrapper[5052]: I1203 12:46:17.699946 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/78361066-6add-4971-9a4b-9157f35ad494-combined-ca-bundle\") pod \"78361066-6add-4971-9a4b-9157f35ad494\" (UID: \"78361066-6add-4971-9a4b-9157f35ad494\") " Dec 03 12:46:17 crc kubenswrapper[5052]: I1203 12:46:17.699962 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/fb6c11ac-1b0e-4e63-8988-256b2504ebc7-config-data\") pod \"fb6c11ac-1b0e-4e63-8988-256b2504ebc7\" (UID: \"fb6c11ac-1b0e-4e63-8988-256b2504ebc7\") " Dec 03 12:46:17 crc kubenswrapper[5052]: I1203 12:46:17.699976 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/68a80c17-9f7d-47de-b02c-5e7bb47656ab-logs\") pod \"68a80c17-9f7d-47de-b02c-5e7bb47656ab\" (UID: \"68a80c17-9f7d-47de-b02c-5e7bb47656ab\") " Dec 03 12:46:17 crc kubenswrapper[5052]: I1203 12:46:17.700010 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/68a80c17-9f7d-47de-b02c-5e7bb47656ab-config-data-custom\") pod \"68a80c17-9f7d-47de-b02c-5e7bb47656ab\" (UID: \"68a80c17-9f7d-47de-b02c-5e7bb47656ab\") " Dec 03 12:46:17 crc kubenswrapper[5052]: I1203 12:46:17.700024 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/68a80c17-9f7d-47de-b02c-5e7bb47656ab-scripts\") pod \"68a80c17-9f7d-47de-b02c-5e7bb47656ab\" (UID: \"68a80c17-9f7d-47de-b02c-5e7bb47656ab\") " Dec 03 12:46:17 crc kubenswrapper[5052]: I1203 12:46:17.712544 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9aea20a7-eaed-43a8-9b52-08c2c0212230-httpd-run" (OuterVolumeSpecName: "httpd-run") pod "9aea20a7-eaed-43a8-9b52-08c2c0212230" (UID: "9aea20a7-eaed-43a8-9b52-08c2c0212230"). InnerVolumeSpecName "httpd-run". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:46:17 crc kubenswrapper[5052]: I1203 12:46:17.722775 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9aea20a7-eaed-43a8-9b52-08c2c0212230-logs" (OuterVolumeSpecName: "logs") pod "9aea20a7-eaed-43a8-9b52-08c2c0212230" (UID: "9aea20a7-eaed-43a8-9b52-08c2c0212230"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:46:17 crc kubenswrapper[5052]: I1203 12:46:17.727763 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/78361066-6add-4971-9a4b-9157f35ad494-logs" (OuterVolumeSpecName: "logs") pod "78361066-6add-4971-9a4b-9157f35ad494" (UID: "78361066-6add-4971-9a4b-9157f35ad494"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:46:17 crc kubenswrapper[5052]: I1203 12:46:17.729890 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb6c11ac-1b0e-4e63-8988-256b2504ebc7-logs" (OuterVolumeSpecName: "logs") pod "fb6c11ac-1b0e-4e63-8988-256b2504ebc7" (UID: "fb6c11ac-1b0e-4e63-8988-256b2504ebc7"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:46:17 crc kubenswrapper[5052]: I1203 12:46:17.734325 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/68a80c17-9f7d-47de-b02c-5e7bb47656ab-logs" (OuterVolumeSpecName: "logs") pod "68a80c17-9f7d-47de-b02c-5e7bb47656ab" (UID: "68a80c17-9f7d-47de-b02c-5e7bb47656ab"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:46:17 crc kubenswrapper[5052]: I1203 12:46:17.740112 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/68a80c17-9f7d-47de-b02c-5e7bb47656ab-etc-machine-id" (OuterVolumeSpecName: "etc-machine-id") pod "68a80c17-9f7d-47de-b02c-5e7bb47656ab" (UID: "68a80c17-9f7d-47de-b02c-5e7bb47656ab"). InnerVolumeSpecName "etc-machine-id". PluginName "kubernetes.io/host-path", VolumeGidValue "" Dec 03 12:46:17 crc kubenswrapper[5052]: I1203 12:46:17.741419 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4eab5b48-f674-4100-9b42-f3ea257d47d4" path="/var/lib/kubelet/pods/4eab5b48-f674-4100-9b42-f3ea257d47d4/volumes" Dec 03 12:46:17 crc kubenswrapper[5052]: I1203 12:46:17.745345 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/68a80c17-9f7d-47de-b02c-5e7bb47656ab-config-data-custom" (OuterVolumeSpecName: "config-data-custom") pod "68a80c17-9f7d-47de-b02c-5e7bb47656ab" (UID: "68a80c17-9f7d-47de-b02c-5e7bb47656ab"). InnerVolumeSpecName "config-data-custom". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:46:17 crc kubenswrapper[5052]: I1203 12:46:17.745925 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovsdbserver-nb-tls-certs\" (UniqueName: \"kubernetes.io/secret/29c7d055-4f37-45a4-a608-eb73a361a86c-ovsdbserver-nb-tls-certs\") pod \"ovsdbserver-nb-0\" (UID: \"29c7d055-4f37-45a4-a608-eb73a361a86c\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:46:17 crc kubenswrapper[5052]: I1203 12:46:17.750001 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6475384f-1965-4e8b-8806-af420d1381de" path="/var/lib/kubelet/pods/6475384f-1965-4e8b-8806-af420d1381de/volumes" Dec 03 12:46:17 crc kubenswrapper[5052]: I1203 12:46:17.756931 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage12-crc" (OuterVolumeSpecName: "glance") pod "9aea20a7-eaed-43a8-9b52-08c2c0212230" (UID: "9aea20a7-eaed-43a8-9b52-08c2c0212230"). InnerVolumeSpecName "local-storage12-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 12:46:17 crc kubenswrapper[5052]: I1203 12:46:17.759761 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/68a80c17-9f7d-47de-b02c-5e7bb47656ab-kube-api-access-nltzf" (OuterVolumeSpecName: "kube-api-access-nltzf") pod "68a80c17-9f7d-47de-b02c-5e7bb47656ab" (UID: "68a80c17-9f7d-47de-b02c-5e7bb47656ab"). InnerVolumeSpecName "kube-api-access-nltzf". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:46:17 crc kubenswrapper[5052]: I1203 12:46:17.760832 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage18-crc" (OuterVolumeSpecName: "glance") pod "78361066-6add-4971-9a4b-9157f35ad494" (UID: "78361066-6add-4971-9a4b-9157f35ad494"). InnerVolumeSpecName "local-storage18-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 12:46:17 crc kubenswrapper[5052]: I1203 12:46:17.761392 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:46:17 crc kubenswrapper[5052]: E1203 12:46:17.774241 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-ovndbcluster-nb-ovndbs: secret "cert-ovndbcluster-nb-ovndbs" not found Dec 03 12:46:17 crc kubenswrapper[5052]: I1203 12:46:17.775131 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ee05fe95-7cc0-4225-a86b-1682adc78b8b" path="/var/lib/kubelet/pods/ee05fe95-7cc0-4225-a86b-1682adc78b8b/volumes" Dec 03 12:46:17 crc kubenswrapper[5052]: I1203 12:46:17.781184 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/78361066-6add-4971-9a4b-9157f35ad494-httpd-run" (OuterVolumeSpecName: "httpd-run") pod "78361066-6add-4971-9a4b-9157f35ad494" (UID: "78361066-6add-4971-9a4b-9157f35ad494"). InnerVolumeSpecName "httpd-run". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:46:17 crc kubenswrapper[5052]: I1203 12:46:17.781231 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9aea20a7-eaed-43a8-9b52-08c2c0212230-kube-api-access-ql9lm" (OuterVolumeSpecName: "kube-api-access-ql9lm") pod "9aea20a7-eaed-43a8-9b52-08c2c0212230" (UID: "9aea20a7-eaed-43a8-9b52-08c2c0212230"). InnerVolumeSpecName "kube-api-access-ql9lm". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:46:17 crc kubenswrapper[5052]: E1203 12:46:17.781747 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/29c7d055-4f37-45a4-a608-eb73a361a86c-ovsdbserver-nb-tls-certs podName:29c7d055-4f37-45a4-a608-eb73a361a86c nodeName:}" failed. No retries permitted until 2025-12-03 12:46:19.781722664 +0000 UTC m=+5170.369964448 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "ovsdbserver-nb-tls-certs" (UniqueName: "kubernetes.io/secret/29c7d055-4f37-45a4-a608-eb73a361a86c-ovsdbserver-nb-tls-certs") pod "ovsdbserver-nb-0" (UID: "29c7d055-4f37-45a4-a608-eb73a361a86c") : secret "cert-ovndbcluster-nb-ovndbs" not found Dec 03 12:46:17 crc kubenswrapper[5052]: I1203 12:46:17.781963 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/29c7d055-4f37-45a4-a608-eb73a361a86c-metrics-certs-tls-certs\") pod \"ovsdbserver-nb-0\" (UID: \"29c7d055-4f37-45a4-a608-eb73a361a86c\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:46:17 crc kubenswrapper[5052]: I1203 12:46:17.782158 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/68a80c17-9f7d-47de-b02c-5e7bb47656ab-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:17 crc kubenswrapper[5052]: I1203 12:46:17.782240 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/68a80c17-9f7d-47de-b02c-5e7bb47656ab-config-data-custom\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:17 crc kubenswrapper[5052]: I1203 12:46:17.782302 5052 reconciler_common.go:293] "Volume detached for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/68a80c17-9f7d-47de-b02c-5e7bb47656ab-etc-machine-id\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:17 crc kubenswrapper[5052]: I1203 12:46:17.782404 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-nltzf\" (UniqueName: \"kubernetes.io/projected/68a80c17-9f7d-47de-b02c-5e7bb47656ab-kube-api-access-nltzf\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:17 crc kubenswrapper[5052]: I1203 12:46:17.782556 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage18-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage18-crc\") on node \"crc\" " Dec 03 12:46:17 crc kubenswrapper[5052]: I1203 12:46:17.782647 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/78361066-6add-4971-9a4b-9157f35ad494-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:17 crc kubenswrapper[5052]: I1203 12:46:17.782709 5052 reconciler_common.go:293] "Volume detached for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/9aea20a7-eaed-43a8-9b52-08c2c0212230-httpd-run\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:17 crc kubenswrapper[5052]: I1203 12:46:17.784320 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/9aea20a7-eaed-43a8-9b52-08c2c0212230-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:17 crc kubenswrapper[5052]: I1203 12:46:17.784353 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/fb6c11ac-1b0e-4e63-8988-256b2504ebc7-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:17 crc kubenswrapper[5052]: I1203 12:46:17.784371 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-ql9lm\" (UniqueName: \"kubernetes.io/projected/9aea20a7-eaed-43a8-9b52-08c2c0212230-kube-api-access-ql9lm\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:17 crc kubenswrapper[5052]: I1203 12:46:17.784388 5052 reconciler_common.go:293] "Volume detached for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/78361066-6add-4971-9a4b-9157f35ad494-httpd-run\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:17 crc kubenswrapper[5052]: I1203 12:46:17.784404 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/9af09a5b-789c-4b83-b899-9d1b3f0d8603-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:17 crc kubenswrapper[5052]: I1203 12:46:17.784433 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage12-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage12-crc\") on node \"crc\" " Dec 03 12:46:17 crc kubenswrapper[5052]: I1203 12:46:17.786064 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/68a80c17-9f7d-47de-b02c-5e7bb47656ab-scripts" (OuterVolumeSpecName: "scripts") pod "68a80c17-9f7d-47de-b02c-5e7bb47656ab" (UID: "68a80c17-9f7d-47de-b02c-5e7bb47656ab"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:46:17 crc kubenswrapper[5052]: E1203 12:46:17.786160 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-ovn-metrics: secret "cert-ovn-metrics" not found Dec 03 12:46:17 crc kubenswrapper[5052]: E1203 12:46:17.786202 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/29c7d055-4f37-45a4-a608-eb73a361a86c-metrics-certs-tls-certs podName:29c7d055-4f37-45a4-a608-eb73a361a86c nodeName:}" failed. No retries permitted until 2025-12-03 12:46:19.786187022 +0000 UTC m=+5170.374428786 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs-tls-certs" (UniqueName: "kubernetes.io/secret/29c7d055-4f37-45a4-a608-eb73a361a86c-metrics-certs-tls-certs") pod "ovsdbserver-nb-0" (UID: "29c7d055-4f37-45a4-a608-eb73a361a86c") : secret "cert-ovn-metrics" not found Dec 03 12:46:17 crc kubenswrapper[5052]: I1203 12:46:17.813060 5052 generic.go:334] "Generic (PLEG): container finished" podID="3ae64378-aaa4-46bc-8c48-e1d90e490016" containerID="8e0b376c29b7c2dd056afa36e69404e2be33493fa6b1a9ba35735a9c8becf167" exitCode=0 Dec 03 12:46:17 crc kubenswrapper[5052]: I1203 12:46:17.831293 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:46:17 crc kubenswrapper[5052]: E1203 12:46:17.831998 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="68a80c17-9f7d-47de-b02c-5e7bb47656ab" containerName="cinder-api" Dec 03 12:46:17 crc kubenswrapper[5052]: I1203 12:46:17.833315 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="68a80c17-9f7d-47de-b02c-5e7bb47656ab" containerName="cinder-api" Dec 03 12:46:17 crc kubenswrapper[5052]: E1203 12:46:17.833474 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="fb6c11ac-1b0e-4e63-8988-256b2504ebc7" containerName="nova-api-api" Dec 03 12:46:17 crc kubenswrapper[5052]: I1203 12:46:17.833597 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb6c11ac-1b0e-4e63-8988-256b2504ebc7" containerName="nova-api-api" Dec 03 12:46:17 crc kubenswrapper[5052]: E1203 12:46:17.833694 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="9aea20a7-eaed-43a8-9b52-08c2c0212230" containerName="glance-log" Dec 03 12:46:17 crc kubenswrapper[5052]: I1203 12:46:17.833779 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="9aea20a7-eaed-43a8-9b52-08c2c0212230" containerName="glance-log" Dec 03 12:46:17 crc kubenswrapper[5052]: E1203 12:46:17.833899 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="78361066-6add-4971-9a4b-9157f35ad494" containerName="glance-httpd" Dec 03 12:46:17 crc kubenswrapper[5052]: I1203 12:46:17.833979 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="78361066-6add-4971-9a4b-9157f35ad494" containerName="glance-httpd" Dec 03 12:46:17 crc kubenswrapper[5052]: E1203 12:46:17.834059 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="fb6c11ac-1b0e-4e63-8988-256b2504ebc7" containerName="nova-api-log" Dec 03 12:46:17 crc kubenswrapper[5052]: I1203 12:46:17.834126 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb6c11ac-1b0e-4e63-8988-256b2504ebc7" containerName="nova-api-log" Dec 03 12:46:17 crc kubenswrapper[5052]: E1203 12:46:17.834215 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="9af09a5b-789c-4b83-b899-9d1b3f0d8603" containerName="nova-metadata-metadata" Dec 03 12:46:17 crc kubenswrapper[5052]: I1203 12:46:17.834327 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="9af09a5b-789c-4b83-b899-9d1b3f0d8603" containerName="nova-metadata-metadata" Dec 03 12:46:17 crc kubenswrapper[5052]: E1203 12:46:17.834414 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="68a80c17-9f7d-47de-b02c-5e7bb47656ab" containerName="cinder-api-log" Dec 03 12:46:17 crc kubenswrapper[5052]: I1203 12:46:17.834483 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="68a80c17-9f7d-47de-b02c-5e7bb47656ab" containerName="cinder-api-log" Dec 03 12:46:17 crc kubenswrapper[5052]: E1203 12:46:17.849814 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="9af09a5b-789c-4b83-b899-9d1b3f0d8603" containerName="nova-metadata-log" Dec 03 12:46:17 crc kubenswrapper[5052]: I1203 12:46:17.850297 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="9af09a5b-789c-4b83-b899-9d1b3f0d8603" containerName="nova-metadata-log" Dec 03 12:46:17 crc kubenswrapper[5052]: E1203 12:46:17.850372 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="78361066-6add-4971-9a4b-9157f35ad494" containerName="glance-log" Dec 03 12:46:17 crc kubenswrapper[5052]: I1203 12:46:17.850463 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="78361066-6add-4971-9a4b-9157f35ad494" containerName="glance-log" Dec 03 12:46:17 crc kubenswrapper[5052]: E1203 12:46:17.850653 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="9aea20a7-eaed-43a8-9b52-08c2c0212230" containerName="glance-httpd" Dec 03 12:46:17 crc kubenswrapper[5052]: I1203 12:46:17.850719 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="9aea20a7-eaed-43a8-9b52-08c2c0212230" containerName="glance-httpd" Dec 03 12:46:17 crc kubenswrapper[5052]: I1203 12:46:17.833478 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9aea20a7-eaed-43a8-9b52-08c2c0212230-scripts" (OuterVolumeSpecName: "scripts") pod "9aea20a7-eaed-43a8-9b52-08c2c0212230" (UID: "9aea20a7-eaed-43a8-9b52-08c2c0212230"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:46:17 crc kubenswrapper[5052]: I1203 12:46:17.845069 5052 generic.go:334] "Generic (PLEG): container finished" podID="68a80c17-9f7d-47de-b02c-5e7bb47656ab" containerID="36e0739e37891a21cddc05442e8ef21c0dc06d541fe56c5e1fce38f2ed446057" exitCode=0 Dec 03 12:46:17 crc kubenswrapper[5052]: I1203 12:46:17.851185 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="78361066-6add-4971-9a4b-9157f35ad494" containerName="glance-httpd" Dec 03 12:46:17 crc kubenswrapper[5052]: I1203 12:46:17.851256 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="9aea20a7-eaed-43a8-9b52-08c2c0212230" containerName="glance-log" Dec 03 12:46:17 crc kubenswrapper[5052]: I1203 12:46:17.845121 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:46:17 crc kubenswrapper[5052]: I1203 12:46:17.851319 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="fb6c11ac-1b0e-4e63-8988-256b2504ebc7" containerName="nova-api-log" Dec 03 12:46:17 crc kubenswrapper[5052]: I1203 12:46:17.851426 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="68a80c17-9f7d-47de-b02c-5e7bb47656ab" containerName="cinder-api" Dec 03 12:46:17 crc kubenswrapper[5052]: I1203 12:46:17.853280 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="78361066-6add-4971-9a4b-9157f35ad494" containerName="glance-log" Dec 03 12:46:17 crc kubenswrapper[5052]: I1203 12:46:17.853389 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="68a80c17-9f7d-47de-b02c-5e7bb47656ab" containerName="cinder-api-log" Dec 03 12:46:17 crc kubenswrapper[5052]: I1203 12:46:17.853555 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="9af09a5b-789c-4b83-b899-9d1b3f0d8603" containerName="nova-metadata-log" Dec 03 12:46:17 crc kubenswrapper[5052]: I1203 12:46:17.853632 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="fb6c11ac-1b0e-4e63-8988-256b2504ebc7" containerName="nova-api-api" Dec 03 12:46:17 crc kubenswrapper[5052]: I1203 12:46:17.853704 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="9aea20a7-eaed-43a8-9b52-08c2c0212230" containerName="glance-httpd" Dec 03 12:46:17 crc kubenswrapper[5052]: I1203 12:46:17.853768 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="9af09a5b-789c-4b83-b899-9d1b3f0d8603" containerName="nova-metadata-metadata" Dec 03 12:46:17 crc kubenswrapper[5052]: I1203 12:46:17.855490 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"78361066-6add-4971-9a4b-9157f35ad494","Type":"ContainerDied","Data":"8e72cb53ffa488be78204a96a13a28807e663bc5864239f1b6c619b61fcf83f4"} Dec 03 12:46:17 crc kubenswrapper[5052]: I1203 12:46:17.856365 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:46:17 crc kubenswrapper[5052]: I1203 12:46:17.856471 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:46:17 crc kubenswrapper[5052]: I1203 12:46:17.856623 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"3ae64378-aaa4-46bc-8c48-e1d90e490016","Type":"ContainerDied","Data":"8e0b376c29b7c2dd056afa36e69404e2be33493fa6b1a9ba35735a9c8becf167"} Dec 03 12:46:17 crc kubenswrapper[5052]: I1203 12:46:17.856712 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"68a80c17-9f7d-47de-b02c-5e7bb47656ab","Type":"ContainerDied","Data":"36e0739e37891a21cddc05442e8ef21c0dc06d541fe56c5e1fce38f2ed446057"} Dec 03 12:46:17 crc kubenswrapper[5052]: I1203 12:46:17.856788 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"68a80c17-9f7d-47de-b02c-5e7bb47656ab","Type":"ContainerDied","Data":"cf29619d34834e7d49a5eedc87579f786108ce54ec9ce704025e2fbebca41943"} Dec 03 12:46:17 crc kubenswrapper[5052]: I1203 12:46:17.856857 5052 scope.go:117] "RemoveContainer" containerID="3310a6083ad0bd13b8590b4fa080eeb83719e19bfb1dddf441320e21f33c7afa" Dec 03 12:46:17 crc kubenswrapper[5052]: I1203 12:46:17.855661 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:46:17 crc kubenswrapper[5052]: I1203 12:46:17.859733 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-ceilometer-internal-svc" Dec 03 12:46:17 crc kubenswrapper[5052]: I1203 12:46:17.860001 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-config-data" Dec 03 12:46:17 crc kubenswrapper[5052]: I1203 12:46:17.863764 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-scripts" Dec 03 12:46:17 crc kubenswrapper[5052]: I1203 12:46:17.879602 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:46:17 crc kubenswrapper[5052]: I1203 12:46:17.886249 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/9aea20a7-eaed-43a8-9b52-08c2c0212230-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:17 crc kubenswrapper[5052]: I1203 12:46:17.886274 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/68a80c17-9f7d-47de-b02c-5e7bb47656ab-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:17 crc kubenswrapper[5052]: I1203 12:46:17.895325 5052 generic.go:334] "Generic (PLEG): container finished" podID="1d187c5a-7745-447f-a8e2-06c7528e5be0" containerID="4208df0572c14c762c41a0bceeeb54f12d57e83a373e6e10caba8b5459e15ce7" exitCode=143 Dec 03 12:46:17 crc kubenswrapper[5052]: I1203 12:46:17.895405 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-keystone-listener-769bc98c4b-mnz6r" event={"ID":"1d187c5a-7745-447f-a8e2-06c7528e5be0","Type":"ContainerDied","Data":"4208df0572c14c762c41a0bceeeb54f12d57e83a373e6e10caba8b5459e15ce7"} Dec 03 12:46:17 crc kubenswrapper[5052]: I1203 12:46:17.901987 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/openstack-galera-0"] Dec 03 12:46:17 crc kubenswrapper[5052]: I1203 12:46:17.903826 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fb6c11ac-1b0e-4e63-8988-256b2504ebc7-kube-api-access-mp72c" (OuterVolumeSpecName: "kube-api-access-mp72c") pod "fb6c11ac-1b0e-4e63-8988-256b2504ebc7" (UID: "fb6c11ac-1b0e-4e63-8988-256b2504ebc7"). InnerVolumeSpecName "kube-api-access-mp72c". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:46:17 crc kubenswrapper[5052]: I1203 12:46:17.903923 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"fb6c11ac-1b0e-4e63-8988-256b2504ebc7","Type":"ContainerDied","Data":"a0169135c26c35f1fdcae98770c9290a702b10feb005661d47a023d4ebac16eb"} Dec 03 12:46:17 crc kubenswrapper[5052]: I1203 12:46:17.903932 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:46:17 crc kubenswrapper[5052]: I1203 12:46:17.907766 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/78361066-6add-4971-9a4b-9157f35ad494-kube-api-access-sp7f4" (OuterVolumeSpecName: "kube-api-access-sp7f4") pod "78361066-6add-4971-9a4b-9157f35ad494" (UID: "78361066-6add-4971-9a4b-9157f35ad494"). InnerVolumeSpecName "kube-api-access-sp7f4". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:46:17 crc kubenswrapper[5052]: I1203 12:46:17.908251 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/78361066-6add-4971-9a4b-9157f35ad494-scripts" (OuterVolumeSpecName: "scripts") pod "78361066-6add-4971-9a4b-9157f35ad494" (UID: "78361066-6add-4971-9a4b-9157f35ad494"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:46:17 crc kubenswrapper[5052]: I1203 12:46:17.916819 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"9af09a5b-789c-4b83-b899-9d1b3f0d8603","Type":"ContainerDied","Data":"7069ccbb6a2b1508b51ae009958cc0840dc2f497188ba2d99f0a2da359df1b40"} Dec 03 12:46:17 crc kubenswrapper[5052]: I1203 12:46:17.916925 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:46:17 crc kubenswrapper[5052]: I1203 12:46:17.925536 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:46:17 crc kubenswrapper[5052]: I1203 12:46:17.925582 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"9aea20a7-eaed-43a8-9b52-08c2c0212230","Type":"ContainerDied","Data":"19797ca6bcaa0d1b25af20fbf64d800512d2a5454279e6708f21c0ca08b2d7dc"} Dec 03 12:46:17 crc kubenswrapper[5052]: I1203 12:46:17.952634 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/openstack-galera-0"] Dec 03 12:46:17 crc kubenswrapper[5052]: I1203 12:46:17.970160 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:46:17 crc kubenswrapper[5052]: I1203 12:46:17.972564 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:46:17 crc kubenswrapper[5052]: I1203 12:46:17.976816 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-scheduler-config-data" Dec 03 12:46:17 crc kubenswrapper[5052]: I1203 12:46:17.987576 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/memcached-0"] Dec 03 12:46:17 crc kubenswrapper[5052]: I1203 12:46:17.988423 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/9481ae66-152c-41e7-9cbc-90dcb5fa6171-config-data\") pod \"ceilometer-0\" (UID: \"9481ae66-152c-41e7-9cbc-90dcb5fa6171\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:46:17 crc kubenswrapper[5052]: I1203 12:46:17.988473 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9481ae66-152c-41e7-9cbc-90dcb5fa6171-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"9481ae66-152c-41e7-9cbc-90dcb5fa6171\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:46:17 crc kubenswrapper[5052]: I1203 12:46:17.988566 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/9481ae66-152c-41e7-9cbc-90dcb5fa6171-scripts\") pod \"ceilometer-0\" (UID: \"9481ae66-152c-41e7-9cbc-90dcb5fa6171\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:46:17 crc kubenswrapper[5052]: I1203 12:46:17.988637 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/9481ae66-152c-41e7-9cbc-90dcb5fa6171-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"9481ae66-152c-41e7-9cbc-90dcb5fa6171\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:46:17 crc kubenswrapper[5052]: I1203 12:46:17.988673 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/9481ae66-152c-41e7-9cbc-90dcb5fa6171-ceilometer-tls-certs\") pod \"ceilometer-0\" (UID: \"9481ae66-152c-41e7-9cbc-90dcb5fa6171\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:46:17 crc kubenswrapper[5052]: I1203 12:46:17.988732 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/9481ae66-152c-41e7-9cbc-90dcb5fa6171-log-httpd\") pod \"ceilometer-0\" (UID: \"9481ae66-152c-41e7-9cbc-90dcb5fa6171\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:46:17 crc kubenswrapper[5052]: I1203 12:46:17.988783 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/9481ae66-152c-41e7-9cbc-90dcb5fa6171-run-httpd\") pod \"ceilometer-0\" (UID: \"9481ae66-152c-41e7-9cbc-90dcb5fa6171\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:46:17 crc kubenswrapper[5052]: I1203 12:46:17.988849 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bwnpf\" (UniqueName: \"kubernetes.io/projected/9481ae66-152c-41e7-9cbc-90dcb5fa6171-kube-api-access-bwnpf\") pod \"ceilometer-0\" (UID: \"9481ae66-152c-41e7-9cbc-90dcb5fa6171\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:46:17 crc kubenswrapper[5052]: I1203 12:46:17.988942 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-sp7f4\" (UniqueName: \"kubernetes.io/projected/78361066-6add-4971-9a4b-9157f35ad494-kube-api-access-sp7f4\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:17 crc kubenswrapper[5052]: I1203 12:46:17.988959 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/78361066-6add-4971-9a4b-9157f35ad494-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:17 crc kubenswrapper[5052]: I1203 12:46:17.988977 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-mp72c\" (UniqueName: \"kubernetes.io/projected/fb6c11ac-1b0e-4e63-8988-256b2504ebc7-kube-api-access-mp72c\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.007623 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/memcached-0"] Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.062875 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.074792 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/openstack-galera-0"] Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.079934 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.091351 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"openstack-scripts" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.092307 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"galera-openstack-dockercfg-tsmcq" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.093535 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/9481ae66-152c-41e7-9cbc-90dcb5fa6171-config-data\") pod \"ceilometer-0\" (UID: \"9481ae66-152c-41e7-9cbc-90dcb5fa6171\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.093581 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9481ae66-152c-41e7-9cbc-90dcb5fa6171-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"9481ae66-152c-41e7-9cbc-90dcb5fa6171\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.093628 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"openstack-config-data" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.093637 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/9481ae66-152c-41e7-9cbc-90dcb5fa6171-scripts\") pod \"ceilometer-0\" (UID: \"9481ae66-152c-41e7-9cbc-90dcb5fa6171\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.093661 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nqqmx\" (UniqueName: \"kubernetes.io/projected/81709983-1fd0-4446-bbbf-fb94653bb3d2-kube-api-access-nqqmx\") pod \"nova-scheduler-0\" (UID: \"81709983-1fd0-4446-bbbf-fb94653bb3d2\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.093719 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/9481ae66-152c-41e7-9cbc-90dcb5fa6171-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"9481ae66-152c-41e7-9cbc-90dcb5fa6171\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.093751 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/81709983-1fd0-4446-bbbf-fb94653bb3d2-config-data\") pod \"nova-scheduler-0\" (UID: \"81709983-1fd0-4446-bbbf-fb94653bb3d2\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.093769 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/9481ae66-152c-41e7-9cbc-90dcb5fa6171-ceilometer-tls-certs\") pod \"ceilometer-0\" (UID: \"9481ae66-152c-41e7-9cbc-90dcb5fa6171\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.093814 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/9481ae66-152c-41e7-9cbc-90dcb5fa6171-log-httpd\") pod \"ceilometer-0\" (UID: \"9481ae66-152c-41e7-9cbc-90dcb5fa6171\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.093843 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/81709983-1fd0-4446-bbbf-fb94653bb3d2-combined-ca-bundle\") pod \"nova-scheduler-0\" (UID: \"81709983-1fd0-4446-bbbf-fb94653bb3d2\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.093867 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/9481ae66-152c-41e7-9cbc-90dcb5fa6171-run-httpd\") pod \"ceilometer-0\" (UID: \"9481ae66-152c-41e7-9cbc-90dcb5fa6171\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.093927 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-bwnpf\" (UniqueName: \"kubernetes.io/projected/9481ae66-152c-41e7-9cbc-90dcb5fa6171-kube-api-access-bwnpf\") pod \"ceilometer-0\" (UID: \"9481ae66-152c-41e7-9cbc-90dcb5fa6171\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.094862 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/memcached-0"] Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.096388 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/memcached-0" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.097829 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/9481ae66-152c-41e7-9cbc-90dcb5fa6171-log-httpd\") pod \"ceilometer-0\" (UID: \"9481ae66-152c-41e7-9cbc-90dcb5fa6171\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:46:18 crc kubenswrapper[5052]: E1203 12:46:18.097926 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-ceilometer-internal-svc: secret "cert-ceilometer-internal-svc" not found Dec 03 12:46:18 crc kubenswrapper[5052]: E1203 12:46:18.097976 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9481ae66-152c-41e7-9cbc-90dcb5fa6171-ceilometer-tls-certs podName:9481ae66-152c-41e7-9cbc-90dcb5fa6171 nodeName:}" failed. No retries permitted until 2025-12-03 12:46:18.597962288 +0000 UTC m=+5169.186204052 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "ceilometer-tls-certs" (UniqueName: "kubernetes.io/secret/9481ae66-152c-41e7-9cbc-90dcb5fa6171-ceilometer-tls-certs") pod "ceilometer-0" (UID: "9481ae66-152c-41e7-9cbc-90dcb5fa6171") : secret "cert-ceilometer-internal-svc" not found Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.098640 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/9481ae66-152c-41e7-9cbc-90dcb5fa6171-run-httpd\") pod \"ceilometer-0\" (UID: \"9481ae66-152c-41e7-9cbc-90dcb5fa6171\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.100714 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9481ae66-152c-41e7-9cbc-90dcb5fa6171-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"9481ae66-152c-41e7-9cbc-90dcb5fa6171\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.102351 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-galera-openstack-svc" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.104132 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/9481ae66-152c-41e7-9cbc-90dcb5fa6171-scripts\") pod \"ceilometer-0\" (UID: \"9481ae66-152c-41e7-9cbc-90dcb5fa6171\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.105438 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/9481ae66-152c-41e7-9cbc-90dcb5fa6171-config-data\") pod \"ceilometer-0\" (UID: \"9481ae66-152c-41e7-9cbc-90dcb5fa6171\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.111958 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"memcached-config-data" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.112157 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"memcached-memcached-dockercfg-bshzz" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.112269 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-memcached-svc" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.112503 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage12-crc" (UniqueName: "kubernetes.io/local-volume/local-storage12-crc") on node "crc" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.113106 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/9481ae66-152c-41e7-9cbc-90dcb5fa6171-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"9481ae66-152c-41e7-9cbc-90dcb5fa6171\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.127036 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/openstack-cell1-galera-0"] Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.127278 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-bwnpf\" (UniqueName: \"kubernetes.io/projected/9481ae66-152c-41e7-9cbc-90dcb5fa6171-kube-api-access-bwnpf\") pod \"ceilometer-0\" (UID: \"9481ae66-152c-41e7-9cbc-90dcb5fa6171\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.133075 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fb6c11ac-1b0e-4e63-8988-256b2504ebc7-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "fb6c11ac-1b0e-4e63-8988-256b2504ebc7" (UID: "fb6c11ac-1b0e-4e63-8988-256b2504ebc7"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.143257 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/openstack-cell1-galera-0"] Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.156550 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/openstack-galera-0"] Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.157763 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage18-crc" (UniqueName: "kubernetes.io/local-volume/local-storage18-crc") on node "crc" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.166351 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-0"] Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.176151 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/memcached-0"] Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.191384 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-0"] Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.195591 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h5m2d\" (UniqueName: \"kubernetes.io/projected/4d42b068-9b91-4df6-9a9c-81cad009be16-kube-api-access-h5m2d\") pod \"memcached-0\" (UID: \"4d42b068-9b91-4df6-9a9c-81cad009be16\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.195644 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/81709983-1fd0-4446-bbbf-fb94653bb3d2-combined-ca-bundle\") pod \"nova-scheduler-0\" (UID: \"81709983-1fd0-4446-bbbf-fb94653bb3d2\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.195667 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/818947c1-95f1-4235-8846-5886abeb57a1-config-data-generated\") pod \"openstack-galera-0\" (UID: \"818947c1-95f1-4235-8846-5886abeb57a1\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.195735 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/4d42b068-9b91-4df6-9a9c-81cad009be16-kolla-config\") pod \"memcached-0\" (UID: \"4d42b068-9b91-4df6-9a9c-81cad009be16\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.195782 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z2jcq\" (UniqueName: \"kubernetes.io/projected/818947c1-95f1-4235-8846-5886abeb57a1-kube-api-access-z2jcq\") pod \"openstack-galera-0\" (UID: \"818947c1-95f1-4235-8846-5886abeb57a1\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.195804 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/818947c1-95f1-4235-8846-5886abeb57a1-operator-scripts\") pod \"openstack-galera-0\" (UID: \"818947c1-95f1-4235-8846-5886abeb57a1\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.195834 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/818947c1-95f1-4235-8846-5886abeb57a1-config-data-default\") pod \"openstack-galera-0\" (UID: \"818947c1-95f1-4235-8846-5886abeb57a1\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.195887 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/4d42b068-9b91-4df6-9a9c-81cad009be16-config-data\") pod \"memcached-0\" (UID: \"4d42b068-9b91-4df6-9a9c-81cad009be16\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.195906 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-nqqmx\" (UniqueName: \"kubernetes.io/projected/81709983-1fd0-4446-bbbf-fb94653bb3d2-kube-api-access-nqqmx\") pod \"nova-scheduler-0\" (UID: \"81709983-1fd0-4446-bbbf-fb94653bb3d2\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.195931 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/818947c1-95f1-4235-8846-5886abeb57a1-combined-ca-bundle\") pod \"openstack-galera-0\" (UID: \"818947c1-95f1-4235-8846-5886abeb57a1\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.195953 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"memcached-tls-certs\" (UniqueName: \"kubernetes.io/secret/4d42b068-9b91-4df6-9a9c-81cad009be16-memcached-tls-certs\") pod \"memcached-0\" (UID: \"4d42b068-9b91-4df6-9a9c-81cad009be16\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.195970 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4d42b068-9b91-4df6-9a9c-81cad009be16-combined-ca-bundle\") pod \"memcached-0\" (UID: \"4d42b068-9b91-4df6-9a9c-81cad009be16\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.195985 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/818947c1-95f1-4235-8846-5886abeb57a1-kolla-config\") pod \"openstack-galera-0\" (UID: \"818947c1-95f1-4235-8846-5886abeb57a1\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.196036 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/81709983-1fd0-4446-bbbf-fb94653bb3d2-config-data\") pod \"nova-scheduler-0\" (UID: \"81709983-1fd0-4446-bbbf-fb94653bb3d2\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.196062 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage07-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage07-crc\") pod \"openstack-galera-0\" (UID: \"818947c1-95f1-4235-8846-5886abeb57a1\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.196105 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/818947c1-95f1-4235-8846-5886abeb57a1-galera-tls-certs\") pod \"openstack-galera-0\" (UID: \"818947c1-95f1-4235-8846-5886abeb57a1\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.196150 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage12-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage12-crc\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.196162 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/fb6c11ac-1b0e-4e63-8988-256b2504ebc7-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.196172 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage18-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage18-crc\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.204528 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/openstack-cell1-galera-0"] Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.207107 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.214852 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"galera-openstack-cell1-dockercfg-7lfhj" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.215073 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-galera-openstack-cell1-svc" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.215229 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"openstack-cell1-scripts" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.217248 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/81709983-1fd0-4446-bbbf-fb94653bb3d2-combined-ca-bundle\") pod \"nova-scheduler-0\" (UID: \"81709983-1fd0-4446-bbbf-fb94653bb3d2\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.218354 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/81709983-1fd0-4446-bbbf-fb94653bb3d2-config-data\") pod \"nova-scheduler-0\" (UID: \"81709983-1fd0-4446-bbbf-fb94653bb3d2\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.218601 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"openstack-cell1-config-data" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.231891 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/openstack-cell1-galera-0"] Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.239238 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-nqqmx\" (UniqueName: \"kubernetes.io/projected/81709983-1fd0-4446-bbbf-fb94653bb3d2-kube-api-access-nqqmx\") pod \"nova-scheduler-0\" (UID: \"81709983-1fd0-4446-bbbf-fb94653bb3d2\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.286001 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-0"] Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.287641 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.294106 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell0-conductor-config-data" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.297478 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage07-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage07-crc\") pod \"openstack-galera-0\" (UID: \"818947c1-95f1-4235-8846-5886abeb57a1\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.297582 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/818947c1-95f1-4235-8846-5886abeb57a1-galera-tls-certs\") pod \"openstack-galera-0\" (UID: \"818947c1-95f1-4235-8846-5886abeb57a1\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.297611 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-h5m2d\" (UniqueName: \"kubernetes.io/projected/4d42b068-9b91-4df6-9a9c-81cad009be16-kube-api-access-h5m2d\") pod \"memcached-0\" (UID: \"4d42b068-9b91-4df6-9a9c-81cad009be16\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.297656 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/818947c1-95f1-4235-8846-5886abeb57a1-config-data-generated\") pod \"openstack-galera-0\" (UID: \"818947c1-95f1-4235-8846-5886abeb57a1\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:46:18 crc kubenswrapper[5052]: E1203 12:46:18.298747 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-galera-openstack-svc: secret "cert-galera-openstack-svc" not found Dec 03 12:46:18 crc kubenswrapper[5052]: E1203 12:46:18.298807 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/818947c1-95f1-4235-8846-5886abeb57a1-galera-tls-certs podName:818947c1-95f1-4235-8846-5886abeb57a1 nodeName:}" failed. No retries permitted until 2025-12-03 12:46:18.798791681 +0000 UTC m=+5169.387033445 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "galera-tls-certs" (UniqueName: "kubernetes.io/secret/818947c1-95f1-4235-8846-5886abeb57a1-galera-tls-certs") pod "openstack-galera-0" (UID: "818947c1-95f1-4235-8846-5886abeb57a1") : secret "cert-galera-openstack-svc" not found Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.298944 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/4d42b068-9b91-4df6-9a9c-81cad009be16-kolla-config\") pod \"memcached-0\" (UID: \"4d42b068-9b91-4df6-9a9c-81cad009be16\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.299004 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-z2jcq\" (UniqueName: \"kubernetes.io/projected/818947c1-95f1-4235-8846-5886abeb57a1-kube-api-access-z2jcq\") pod \"openstack-galera-0\" (UID: \"818947c1-95f1-4235-8846-5886abeb57a1\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.299027 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/818947c1-95f1-4235-8846-5886abeb57a1-operator-scripts\") pod \"openstack-galera-0\" (UID: \"818947c1-95f1-4235-8846-5886abeb57a1\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.299064 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/818947c1-95f1-4235-8846-5886abeb57a1-config-data-default\") pod \"openstack-galera-0\" (UID: \"818947c1-95f1-4235-8846-5886abeb57a1\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.299137 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/4d42b068-9b91-4df6-9a9c-81cad009be16-config-data\") pod \"memcached-0\" (UID: \"4d42b068-9b91-4df6-9a9c-81cad009be16\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.299173 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/818947c1-95f1-4235-8846-5886abeb57a1-combined-ca-bundle\") pod \"openstack-galera-0\" (UID: \"818947c1-95f1-4235-8846-5886abeb57a1\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.299199 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"memcached-tls-certs\" (UniqueName: \"kubernetes.io/secret/4d42b068-9b91-4df6-9a9c-81cad009be16-memcached-tls-certs\") pod \"memcached-0\" (UID: \"4d42b068-9b91-4df6-9a9c-81cad009be16\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.299218 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4d42b068-9b91-4df6-9a9c-81cad009be16-combined-ca-bundle\") pod \"memcached-0\" (UID: \"4d42b068-9b91-4df6-9a9c-81cad009be16\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.299233 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/818947c1-95f1-4235-8846-5886abeb57a1-kolla-config\") pod \"openstack-galera-0\" (UID: \"818947c1-95f1-4235-8846-5886abeb57a1\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.299406 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/818947c1-95f1-4235-8846-5886abeb57a1-config-data-generated\") pod \"openstack-galera-0\" (UID: \"818947c1-95f1-4235-8846-5886abeb57a1\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.299958 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/818947c1-95f1-4235-8846-5886abeb57a1-kolla-config\") pod \"openstack-galera-0\" (UID: \"818947c1-95f1-4235-8846-5886abeb57a1\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.300337 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/4d42b068-9b91-4df6-9a9c-81cad009be16-config-data\") pod \"memcached-0\" (UID: \"4d42b068-9b91-4df6-9a9c-81cad009be16\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.300639 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/4d42b068-9b91-4df6-9a9c-81cad009be16-kolla-config\") pod \"memcached-0\" (UID: \"4d42b068-9b91-4df6-9a9c-81cad009be16\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.300689 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/78361066-6add-4971-9a4b-9157f35ad494-config-data" (OuterVolumeSpecName: "config-data") pod "78361066-6add-4971-9a4b-9157f35ad494" (UID: "78361066-6add-4971-9a4b-9157f35ad494"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:46:18 crc kubenswrapper[5052]: E1203 12:46:18.300764 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-memcached-svc: secret "cert-memcached-svc" not found Dec 03 12:46:18 crc kubenswrapper[5052]: E1203 12:46:18.300799 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4d42b068-9b91-4df6-9a9c-81cad009be16-memcached-tls-certs podName:4d42b068-9b91-4df6-9a9c-81cad009be16 nodeName:}" failed. No retries permitted until 2025-12-03 12:46:18.800788609 +0000 UTC m=+5169.389030373 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "memcached-tls-certs" (UniqueName: "kubernetes.io/secret/4d42b068-9b91-4df6-9a9c-81cad009be16-memcached-tls-certs") pod "memcached-0" (UID: "4d42b068-9b91-4df6-9a9c-81cad009be16") : secret "cert-memcached-svc" not found Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.301716 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/818947c1-95f1-4235-8846-5886abeb57a1-config-data-default\") pod \"openstack-galera-0\" (UID: \"818947c1-95f1-4235-8846-5886abeb57a1\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.302555 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/818947c1-95f1-4235-8846-5886abeb57a1-operator-scripts\") pod \"openstack-galera-0\" (UID: \"818947c1-95f1-4235-8846-5886abeb57a1\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.318249 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-0"] Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.328248 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/818947c1-95f1-4235-8846-5886abeb57a1-combined-ca-bundle\") pod \"openstack-galera-0\" (UID: \"818947c1-95f1-4235-8846-5886abeb57a1\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.329669 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/78361066-6add-4971-9a4b-9157f35ad494-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "78361066-6add-4971-9a4b-9157f35ad494" (UID: "78361066-6add-4971-9a4b-9157f35ad494"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.330468 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4d42b068-9b91-4df6-9a9c-81cad009be16-combined-ca-bundle\") pod \"memcached-0\" (UID: \"4d42b068-9b91-4df6-9a9c-81cad009be16\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.332012 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-h5m2d\" (UniqueName: \"kubernetes.io/projected/4d42b068-9b91-4df6-9a9c-81cad009be16-kube-api-access-h5m2d\") pod \"memcached-0\" (UID: \"4d42b068-9b91-4df6-9a9c-81cad009be16\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.336836 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage07-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage07-crc\") pod \"openstack-galera-0\" (UID: \"818947c1-95f1-4235-8846-5886abeb57a1\") device mount path \"/mnt/openstack/pv07\"" pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.351116 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-z2jcq\" (UniqueName: \"kubernetes.io/projected/818947c1-95f1-4235-8846-5886abeb57a1-kube-api-access-z2jcq\") pod \"openstack-galera-0\" (UID: \"818947c1-95f1-4235-8846-5886abeb57a1\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.380652 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9aea20a7-eaed-43a8-9b52-08c2c0212230-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "9aea20a7-eaed-43a8-9b52-08c2c0212230" (UID: "9aea20a7-eaed-43a8-9b52-08c2c0212230"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.400836 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x5ltc\" (UniqueName: \"kubernetes.io/projected/576b2af7-7647-4a45-9423-3aa752d84e75-kube-api-access-x5ltc\") pod \"openstack-cell1-galera-0\" (UID: \"576b2af7-7647-4a45-9423-3aa752d84e75\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.400891 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/576b2af7-7647-4a45-9423-3aa752d84e75-galera-tls-certs\") pod \"openstack-cell1-galera-0\" (UID: \"576b2af7-7647-4a45-9423-3aa752d84e75\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.400956 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ll4tx\" (UniqueName: \"kubernetes.io/projected/6d653682-52e2-4940-af66-633475bfe09a-kube-api-access-ll4tx\") pod \"nova-cell0-conductor-0\" (UID: \"6d653682-52e2-4940-af66-633475bfe09a\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.401008 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/576b2af7-7647-4a45-9423-3aa752d84e75-config-data-generated\") pod \"openstack-cell1-galera-0\" (UID: \"576b2af7-7647-4a45-9423-3aa752d84e75\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.401036 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/576b2af7-7647-4a45-9423-3aa752d84e75-combined-ca-bundle\") pod \"openstack-cell1-galera-0\" (UID: \"576b2af7-7647-4a45-9423-3aa752d84e75\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.401058 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6d653682-52e2-4940-af66-633475bfe09a-config-data\") pod \"nova-cell0-conductor-0\" (UID: \"6d653682-52e2-4940-af66-633475bfe09a\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.401130 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/576b2af7-7647-4a45-9423-3aa752d84e75-kolla-config\") pod \"openstack-cell1-galera-0\" (UID: \"576b2af7-7647-4a45-9423-3aa752d84e75\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.401196 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/576b2af7-7647-4a45-9423-3aa752d84e75-config-data-default\") pod \"openstack-cell1-galera-0\" (UID: \"576b2af7-7647-4a45-9423-3aa752d84e75\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.401272 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage11-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage11-crc\") pod \"openstack-cell1-galera-0\" (UID: \"576b2af7-7647-4a45-9423-3aa752d84e75\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.401356 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/576b2af7-7647-4a45-9423-3aa752d84e75-operator-scripts\") pod \"openstack-cell1-galera-0\" (UID: \"576b2af7-7647-4a45-9423-3aa752d84e75\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.401376 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6d653682-52e2-4940-af66-633475bfe09a-combined-ca-bundle\") pod \"nova-cell0-conductor-0\" (UID: \"6d653682-52e2-4940-af66-633475bfe09a\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.401442 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/78361066-6add-4971-9a4b-9157f35ad494-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.401464 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/78361066-6add-4971-9a4b-9157f35ad494-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.401481 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9aea20a7-eaed-43a8-9b52-08c2c0212230-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.404452 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fb6c11ac-1b0e-4e63-8988-256b2504ebc7-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "fb6c11ac-1b0e-4e63-8988-256b2504ebc7" (UID: "fb6c11ac-1b0e-4e63-8988-256b2504ebc7"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.454753 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9af09a5b-789c-4b83-b899-9d1b3f0d8603-nova-metadata-tls-certs" (OuterVolumeSpecName: "nova-metadata-tls-certs") pod "9af09a5b-789c-4b83-b899-9d1b3f0d8603" (UID: "9af09a5b-789c-4b83-b899-9d1b3f0d8603"). InnerVolumeSpecName "nova-metadata-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.459775 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/68a80c17-9f7d-47de-b02c-5e7bb47656ab-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "68a80c17-9f7d-47de-b02c-5e7bb47656ab" (UID: "68a80c17-9f7d-47de-b02c-5e7bb47656ab"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.462905 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9aea20a7-eaed-43a8-9b52-08c2c0212230-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "9aea20a7-eaed-43a8-9b52-08c2c0212230" (UID: "9aea20a7-eaed-43a8-9b52-08c2c0212230"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.464596 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fb6c11ac-1b0e-4e63-8988-256b2504ebc7-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "fb6c11ac-1b0e-4e63-8988-256b2504ebc7" (UID: "fb6c11ac-1b0e-4e63-8988-256b2504ebc7"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.466798 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9aea20a7-eaed-43a8-9b52-08c2c0212230-config-data" (OuterVolumeSpecName: "config-data") pod "9aea20a7-eaed-43a8-9b52-08c2c0212230" (UID: "9aea20a7-eaed-43a8-9b52-08c2c0212230"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.481010 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fb6c11ac-1b0e-4e63-8988-256b2504ebc7-config-data" (OuterVolumeSpecName: "config-data") pod "fb6c11ac-1b0e-4e63-8988-256b2504ebc7" (UID: "fb6c11ac-1b0e-4e63-8988-256b2504ebc7"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.483119 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/68a80c17-9f7d-47de-b02c-5e7bb47656ab-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "68a80c17-9f7d-47de-b02c-5e7bb47656ab" (UID: "68a80c17-9f7d-47de-b02c-5e7bb47656ab"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.488244 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/68a80c17-9f7d-47de-b02c-5e7bb47656ab-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "68a80c17-9f7d-47de-b02c-5e7bb47656ab" (UID: "68a80c17-9f7d-47de-b02c-5e7bb47656ab"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.489980 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage07-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage07-crc\") pod \"openstack-galera-0\" (UID: \"818947c1-95f1-4235-8846-5886abeb57a1\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.503783 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-x5ltc\" (UniqueName: \"kubernetes.io/projected/576b2af7-7647-4a45-9423-3aa752d84e75-kube-api-access-x5ltc\") pod \"openstack-cell1-galera-0\" (UID: \"576b2af7-7647-4a45-9423-3aa752d84e75\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.503821 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/576b2af7-7647-4a45-9423-3aa752d84e75-galera-tls-certs\") pod \"openstack-cell1-galera-0\" (UID: \"576b2af7-7647-4a45-9423-3aa752d84e75\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.503899 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-ll4tx\" (UniqueName: \"kubernetes.io/projected/6d653682-52e2-4940-af66-633475bfe09a-kube-api-access-ll4tx\") pod \"nova-cell0-conductor-0\" (UID: \"6d653682-52e2-4940-af66-633475bfe09a\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.503969 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/576b2af7-7647-4a45-9423-3aa752d84e75-config-data-generated\") pod \"openstack-cell1-galera-0\" (UID: \"576b2af7-7647-4a45-9423-3aa752d84e75\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.503992 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/576b2af7-7647-4a45-9423-3aa752d84e75-combined-ca-bundle\") pod \"openstack-cell1-galera-0\" (UID: \"576b2af7-7647-4a45-9423-3aa752d84e75\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.504010 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6d653682-52e2-4940-af66-633475bfe09a-config-data\") pod \"nova-cell0-conductor-0\" (UID: \"6d653682-52e2-4940-af66-633475bfe09a\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:46:18 crc kubenswrapper[5052]: E1203 12:46:18.504041 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-galera-openstack-cell1-svc: secret "cert-galera-openstack-cell1-svc" not found Dec 03 12:46:18 crc kubenswrapper[5052]: E1203 12:46:18.504180 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/576b2af7-7647-4a45-9423-3aa752d84e75-galera-tls-certs podName:576b2af7-7647-4a45-9423-3aa752d84e75 nodeName:}" failed. No retries permitted until 2025-12-03 12:46:19.004164115 +0000 UTC m=+5169.592405879 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "galera-tls-certs" (UniqueName: "kubernetes.io/secret/576b2af7-7647-4a45-9423-3aa752d84e75-galera-tls-certs") pod "openstack-cell1-galera-0" (UID: "576b2af7-7647-4a45-9423-3aa752d84e75") : secret "cert-galera-openstack-cell1-svc" not found Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.504079 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/576b2af7-7647-4a45-9423-3aa752d84e75-kolla-config\") pod \"openstack-cell1-galera-0\" (UID: \"576b2af7-7647-4a45-9423-3aa752d84e75\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.504742 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/576b2af7-7647-4a45-9423-3aa752d84e75-config-data-default\") pod \"openstack-cell1-galera-0\" (UID: \"576b2af7-7647-4a45-9423-3aa752d84e75\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.505198 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage11-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage11-crc\") pod \"openstack-cell1-galera-0\" (UID: \"576b2af7-7647-4a45-9423-3aa752d84e75\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.505317 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/576b2af7-7647-4a45-9423-3aa752d84e75-operator-scripts\") pod \"openstack-cell1-galera-0\" (UID: \"576b2af7-7647-4a45-9423-3aa752d84e75\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.505344 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6d653682-52e2-4940-af66-633475bfe09a-combined-ca-bundle\") pod \"nova-cell0-conductor-0\" (UID: \"6d653682-52e2-4940-af66-633475bfe09a\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.506998 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/68a80c17-9f7d-47de-b02c-5e7bb47656ab-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.507810 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/576b2af7-7647-4a45-9423-3aa752d84e75-kolla-config\") pod \"openstack-cell1-galera-0\" (UID: \"576b2af7-7647-4a45-9423-3aa752d84e75\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.507888 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage11-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage11-crc\") pod \"openstack-cell1-galera-0\" (UID: \"576b2af7-7647-4a45-9423-3aa752d84e75\") device mount path \"/mnt/openstack/pv11\"" pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.509020 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/576b2af7-7647-4a45-9423-3aa752d84e75-operator-scripts\") pod \"openstack-cell1-galera-0\" (UID: \"576b2af7-7647-4a45-9423-3aa752d84e75\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.509741 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/576b2af7-7647-4a45-9423-3aa752d84e75-config-data-generated\") pod \"openstack-cell1-galera-0\" (UID: \"576b2af7-7647-4a45-9423-3aa752d84e75\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.509819 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/576b2af7-7647-4a45-9423-3aa752d84e75-config-data-default\") pod \"openstack-cell1-galera-0\" (UID: \"576b2af7-7647-4a45-9423-3aa752d84e75\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.509901 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/9aea20a7-eaed-43a8-9b52-08c2c0212230-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.509917 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/fb6c11ac-1b0e-4e63-8988-256b2504ebc7-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.509929 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/fb6c11ac-1b0e-4e63-8988-256b2504ebc7-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.509941 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/68a80c17-9f7d-47de-b02c-5e7bb47656ab-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.509952 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/fb6c11ac-1b0e-4e63-8988-256b2504ebc7-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.509963 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/9aea20a7-eaed-43a8-9b52-08c2c0212230-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.509973 5052 reconciler_common.go:293] "Volume detached for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/9af09a5b-789c-4b83-b899-9d1b3f0d8603-nova-metadata-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.509984 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/68a80c17-9f7d-47de-b02c-5e7bb47656ab-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.511353 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6d653682-52e2-4940-af66-633475bfe09a-config-data\") pod \"nova-cell0-conductor-0\" (UID: \"6d653682-52e2-4940-af66-633475bfe09a\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.517276 5052 scope.go:117] "RemoveContainer" containerID="6fe0649b6ce8be2bfd99aed0e4830c4017fd592947629aa86d78326653fa7af0" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.517984 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/576b2af7-7647-4a45-9423-3aa752d84e75-combined-ca-bundle\") pod \"openstack-cell1-galera-0\" (UID: \"576b2af7-7647-4a45-9423-3aa752d84e75\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.524697 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/78361066-6add-4971-9a4b-9157f35ad494-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "78361066-6add-4971-9a4b-9157f35ad494" (UID: "78361066-6add-4971-9a4b-9157f35ad494"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.525235 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6d653682-52e2-4940-af66-633475bfe09a-combined-ca-bundle\") pod \"nova-cell0-conductor-0\" (UID: \"6d653682-52e2-4940-af66-633475bfe09a\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.526348 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.531738 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-ll4tx\" (UniqueName: \"kubernetes.io/projected/6d653682-52e2-4940-af66-633475bfe09a-kube-api-access-ll4tx\") pod \"nova-cell0-conductor-0\" (UID: \"6d653682-52e2-4940-af66-633475bfe09a\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.533999 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.534447 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-x5ltc\" (UniqueName: \"kubernetes.io/projected/576b2af7-7647-4a45-9423-3aa752d84e75-kube-api-access-x5ltc\") pod \"openstack-cell1-galera-0\" (UID: \"576b2af7-7647-4a45-9423-3aa752d84e75\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.536598 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage11-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage11-crc\") pod \"openstack-cell1-galera-0\" (UID: \"576b2af7-7647-4a45-9423-3aa752d84e75\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.574686 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/68a80c17-9f7d-47de-b02c-5e7bb47656ab-config-data" (OuterVolumeSpecName: "config-data") pod "68a80c17-9f7d-47de-b02c-5e7bb47656ab" (UID: "68a80c17-9f7d-47de-b02c-5e7bb47656ab"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.590592 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.593132 5052 scope.go:117] "RemoveContainer" containerID="36e0739e37891a21cddc05442e8ef21c0dc06d541fe56c5e1fce38f2ed446057" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.631687 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.632331 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/9481ae66-152c-41e7-9cbc-90dcb5fa6171-ceilometer-tls-certs\") pod \"ceilometer-0\" (UID: \"9481ae66-152c-41e7-9cbc-90dcb5fa6171\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.632610 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/78361066-6add-4971-9a4b-9157f35ad494-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.632632 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/68a80c17-9f7d-47de-b02c-5e7bb47656ab-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:18 crc kubenswrapper[5052]: E1203 12:46:18.634838 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-ceilometer-internal-svc: secret "cert-ceilometer-internal-svc" not found Dec 03 12:46:18 crc kubenswrapper[5052]: E1203 12:46:18.634906 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9481ae66-152c-41e7-9cbc-90dcb5fa6171-ceilometer-tls-certs podName:9481ae66-152c-41e7-9cbc-90dcb5fa6171 nodeName:}" failed. No retries permitted until 2025-12-03 12:46:19.634887103 +0000 UTC m=+5170.223128937 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "ceilometer-tls-certs" (UniqueName: "kubernetes.io/secret/9481ae66-152c-41e7-9cbc-90dcb5fa6171-ceilometer-tls-certs") pod "ceilometer-0" (UID: "9481ae66-152c-41e7-9cbc-90dcb5fa6171") : secret "cert-ceilometer-internal-svc" not found Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.652365 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:46:18 crc kubenswrapper[5052]: E1203 12:46:18.652908 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="3ae64378-aaa4-46bc-8c48-e1d90e490016" containerName="probe" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.652935 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="3ae64378-aaa4-46bc-8c48-e1d90e490016" containerName="probe" Dec 03 12:46:18 crc kubenswrapper[5052]: E1203 12:46:18.652967 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="3ae64378-aaa4-46bc-8c48-e1d90e490016" containerName="cinder-scheduler" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.652976 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="3ae64378-aaa4-46bc-8c48-e1d90e490016" containerName="cinder-scheduler" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.653230 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="3ae64378-aaa4-46bc-8c48-e1d90e490016" containerName="probe" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.653257 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="3ae64378-aaa4-46bc-8c48-e1d90e490016" containerName="cinder-scheduler" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.656196 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.660814 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-nova-public-svc" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.663143 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-nova-internal-svc" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.663436 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-api-config-data" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.677439 5052 scope.go:117] "RemoveContainer" containerID="8562f7f2637fa32043b41c559e46c0266da41a1cab62e80ce7b44cb981784aae" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.684595 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.686934 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.729732 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.733413 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/3ae64378-aaa4-46bc-8c48-e1d90e490016-scripts\") pod \"3ae64378-aaa4-46bc-8c48-e1d90e490016\" (UID: \"3ae64378-aaa4-46bc-8c48-e1d90e490016\") " Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.733564 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/3ae64378-aaa4-46bc-8c48-e1d90e490016-config-data\") pod \"3ae64378-aaa4-46bc-8c48-e1d90e490016\" (UID: \"3ae64378-aaa4-46bc-8c48-e1d90e490016\") " Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.733589 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/3ae64378-aaa4-46bc-8c48-e1d90e490016-etc-machine-id\") pod \"3ae64378-aaa4-46bc-8c48-e1d90e490016\" (UID: \"3ae64378-aaa4-46bc-8c48-e1d90e490016\") " Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.733652 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3ae64378-aaa4-46bc-8c48-e1d90e490016-combined-ca-bundle\") pod \"3ae64378-aaa4-46bc-8c48-e1d90e490016\" (UID: \"3ae64378-aaa4-46bc-8c48-e1d90e490016\") " Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.733679 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/3ae64378-aaa4-46bc-8c48-e1d90e490016-config-data-custom\") pod \"3ae64378-aaa4-46bc-8c48-e1d90e490016\" (UID: \"3ae64378-aaa4-46bc-8c48-e1d90e490016\") " Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.733755 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cb4rb\" (UniqueName: \"kubernetes.io/projected/3ae64378-aaa4-46bc-8c48-e1d90e490016-kube-api-access-cb4rb\") pod \"3ae64378-aaa4-46bc-8c48-e1d90e490016\" (UID: \"3ae64378-aaa4-46bc-8c48-e1d90e490016\") " Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.737434 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3ae64378-aaa4-46bc-8c48-e1d90e490016-kube-api-access-cb4rb" (OuterVolumeSpecName: "kube-api-access-cb4rb") pod "3ae64378-aaa4-46bc-8c48-e1d90e490016" (UID: "3ae64378-aaa4-46bc-8c48-e1d90e490016"). InnerVolumeSpecName "kube-api-access-cb4rb". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.740259 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3ae64378-aaa4-46bc-8c48-e1d90e490016-scripts" (OuterVolumeSpecName: "scripts") pod "3ae64378-aaa4-46bc-8c48-e1d90e490016" (UID: "3ae64378-aaa4-46bc-8c48-e1d90e490016"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.740345 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/3ae64378-aaa4-46bc-8c48-e1d90e490016-etc-machine-id" (OuterVolumeSpecName: "etc-machine-id") pod "3ae64378-aaa4-46bc-8c48-e1d90e490016" (UID: "3ae64378-aaa4-46bc-8c48-e1d90e490016"). InnerVolumeSpecName "etc-machine-id". PluginName "kubernetes.io/host-path", VolumeGidValue "" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.742239 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3ae64378-aaa4-46bc-8c48-e1d90e490016-config-data-custom" (OuterVolumeSpecName: "config-data-custom") pod "3ae64378-aaa4-46bc-8c48-e1d90e490016" (UID: "3ae64378-aaa4-46bc-8c48-e1d90e490016"). InnerVolumeSpecName "config-data-custom". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.757032 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.841871 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/ed7f9920-a1e4-437b-8074-a30d0fe4450c-public-tls-certs\") pod \"nova-api-0\" (UID: \"ed7f9920-a1e4-437b-8074-a30d0fe4450c\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.842248 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"memcached-tls-certs\" (UniqueName: \"kubernetes.io/secret/4d42b068-9b91-4df6-9a9c-81cad009be16-memcached-tls-certs\") pod \"memcached-0\" (UID: \"4d42b068-9b91-4df6-9a9c-81cad009be16\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:46:18 crc kubenswrapper[5052]: E1203 12:46:18.842396 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-memcached-svc: secret "cert-memcached-svc" not found Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.842405 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/ed7f9920-a1e4-437b-8074-a30d0fe4450c-internal-tls-certs\") pod \"nova-api-0\" (UID: \"ed7f9920-a1e4-437b-8074-a30d0fe4450c\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:46:18 crc kubenswrapper[5052]: E1203 12:46:18.842462 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4d42b068-9b91-4df6-9a9c-81cad009be16-memcached-tls-certs podName:4d42b068-9b91-4df6-9a9c-81cad009be16 nodeName:}" failed. No retries permitted until 2025-12-03 12:46:19.842436519 +0000 UTC m=+5170.430678283 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "memcached-tls-certs" (UniqueName: "kubernetes.io/secret/4d42b068-9b91-4df6-9a9c-81cad009be16-memcached-tls-certs") pod "memcached-0" (UID: "4d42b068-9b91-4df6-9a9c-81cad009be16") : secret "cert-memcached-svc" not found Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.842479 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/ed7f9920-a1e4-437b-8074-a30d0fe4450c-config-data\") pod \"nova-api-0\" (UID: \"ed7f9920-a1e4-437b-8074-a30d0fe4450c\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.842688 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/ed7f9920-a1e4-437b-8074-a30d0fe4450c-logs\") pod \"nova-api-0\" (UID: \"ed7f9920-a1e4-437b-8074-a30d0fe4450c\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.843414 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/818947c1-95f1-4235-8846-5886abeb57a1-galera-tls-certs\") pod \"openstack-galera-0\" (UID: \"818947c1-95f1-4235-8846-5886abeb57a1\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.844460 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r8cvt\" (UniqueName: \"kubernetes.io/projected/ed7f9920-a1e4-437b-8074-a30d0fe4450c-kube-api-access-r8cvt\") pod \"nova-api-0\" (UID: \"ed7f9920-a1e4-437b-8074-a30d0fe4450c\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.844659 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ed7f9920-a1e4-437b-8074-a30d0fe4450c-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"ed7f9920-a1e4-437b-8074-a30d0fe4450c\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.845076 5052 scope.go:117] "RemoveContainer" containerID="36e0739e37891a21cddc05442e8ef21c0dc06d541fe56c5e1fce38f2ed446057" Dec 03 12:46:18 crc kubenswrapper[5052]: E1203 12:46:18.845230 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-galera-openstack-svc: secret "cert-galera-openstack-svc" not found Dec 03 12:46:18 crc kubenswrapper[5052]: E1203 12:46:18.845292 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/818947c1-95f1-4235-8846-5886abeb57a1-galera-tls-certs podName:818947c1-95f1-4235-8846-5886abeb57a1 nodeName:}" failed. No retries permitted until 2025-12-03 12:46:19.84526385 +0000 UTC m=+5170.433505614 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "galera-tls-certs" (UniqueName: "kubernetes.io/secret/818947c1-95f1-4235-8846-5886abeb57a1-galera-tls-certs") pod "openstack-galera-0" (UID: "818947c1-95f1-4235-8846-5886abeb57a1") : secret "cert-galera-openstack-svc" not found Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.846095 5052 reconciler_common.go:293] "Volume detached for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/3ae64378-aaa4-46bc-8c48-e1d90e490016-etc-machine-id\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.846118 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/3ae64378-aaa4-46bc-8c48-e1d90e490016-config-data-custom\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.846141 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-cb4rb\" (UniqueName: \"kubernetes.io/projected/3ae64378-aaa4-46bc-8c48-e1d90e490016-kube-api-access-cb4rb\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.846159 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/3ae64378-aaa4-46bc-8c48-e1d90e490016-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:18 crc kubenswrapper[5052]: E1203 12:46:18.846434 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"36e0739e37891a21cddc05442e8ef21c0dc06d541fe56c5e1fce38f2ed446057\": container with ID starting with 36e0739e37891a21cddc05442e8ef21c0dc06d541fe56c5e1fce38f2ed446057 not found: ID does not exist" containerID="36e0739e37891a21cddc05442e8ef21c0dc06d541fe56c5e1fce38f2ed446057" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.846469 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"36e0739e37891a21cddc05442e8ef21c0dc06d541fe56c5e1fce38f2ed446057"} err="failed to get container status \"36e0739e37891a21cddc05442e8ef21c0dc06d541fe56c5e1fce38f2ed446057\": rpc error: code = NotFound desc = could not find container \"36e0739e37891a21cddc05442e8ef21c0dc06d541fe56c5e1fce38f2ed446057\": container with ID starting with 36e0739e37891a21cddc05442e8ef21c0dc06d541fe56c5e1fce38f2ed446057 not found: ID does not exist" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.846499 5052 scope.go:117] "RemoveContainer" containerID="8562f7f2637fa32043b41c559e46c0266da41a1cab62e80ce7b44cb981784aae" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.846689 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:46:18 crc kubenswrapper[5052]: E1203 12:46:18.848926 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-swift-internal-svc: secret "cert-swift-internal-svc" not found Dec 03 12:46:18 crc kubenswrapper[5052]: E1203 12:46:18.848994 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/242be027-6461-4aea-ae7c-e52ca996b2ea-internal-tls-certs podName:242be027-6461-4aea-ae7c-e52ca996b2ea nodeName:}" failed. No retries permitted until 2025-12-03 12:46:19.348976756 +0000 UTC m=+5169.937218520 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "internal-tls-certs" (UniqueName: "kubernetes.io/secret/242be027-6461-4aea-ae7c-e52ca996b2ea-internal-tls-certs") pod "swift-proxy-dd4cf57c8-x4c45" (UID: "242be027-6461-4aea-ae7c-e52ca996b2ea") : secret "cert-swift-internal-svc" not found Dec 03 12:46:18 crc kubenswrapper[5052]: E1203 12:46:18.849857 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-swift-public-svc: secret "cert-swift-public-svc" not found Dec 03 12:46:18 crc kubenswrapper[5052]: E1203 12:46:18.849949 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/242be027-6461-4aea-ae7c-e52ca996b2ea-public-tls-certs podName:242be027-6461-4aea-ae7c-e52ca996b2ea nodeName:}" failed. No retries permitted until 2025-12-03 12:46:19.349914963 +0000 UTC m=+5169.938156727 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "public-tls-certs" (UniqueName: "kubernetes.io/secret/242be027-6461-4aea-ae7c-e52ca996b2ea-public-tls-certs") pod "swift-proxy-dd4cf57c8-x4c45" (UID: "242be027-6461-4aea-ae7c-e52ca996b2ea") : secret "cert-swift-public-svc" not found Dec 03 12:46:18 crc kubenswrapper[5052]: E1203 12:46:18.851364 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8562f7f2637fa32043b41c559e46c0266da41a1cab62e80ce7b44cb981784aae\": container with ID starting with 8562f7f2637fa32043b41c559e46c0266da41a1cab62e80ce7b44cb981784aae not found: ID does not exist" containerID="8562f7f2637fa32043b41c559e46c0266da41a1cab62e80ce7b44cb981784aae" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.851411 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8562f7f2637fa32043b41c559e46c0266da41a1cab62e80ce7b44cb981784aae"} err="failed to get container status \"8562f7f2637fa32043b41c559e46c0266da41a1cab62e80ce7b44cb981784aae\": rpc error: code = NotFound desc = could not find container \"8562f7f2637fa32043b41c559e46c0266da41a1cab62e80ce7b44cb981784aae\": container with ID starting with 8562f7f2637fa32043b41c559e46c0266da41a1cab62e80ce7b44cb981784aae not found: ID does not exist" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.851439 5052 scope.go:117] "RemoveContainer" containerID="c2c82ad1c4ab93ed36cb072abbe6d1eeb5753acbd507bfb1d2fbc3bacdb79bdf" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.864350 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.868485 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-metadata-config-data" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.871146 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-nova-metadata-internal-svc" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.938818 5052 scope.go:117] "RemoveContainer" containerID="d57a2057a1ffffdae42cdf5eae9f0f819cc86143c1132d66146d4814f2b2355f" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.951205 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.954445 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3ae64378-aaa4-46bc-8c48-e1d90e490016-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "3ae64378-aaa4-46bc-8c48-e1d90e490016" (UID: "3ae64378-aaa4-46bc-8c48-e1d90e490016"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.955044 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3ae64378-aaa4-46bc-8c48-e1d90e490016-combined-ca-bundle\") pod \"3ae64378-aaa4-46bc-8c48-e1d90e490016\" (UID: \"3ae64378-aaa4-46bc-8c48-e1d90e490016\") " Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.955575 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/ed7f9920-a1e4-437b-8074-a30d0fe4450c-public-tls-certs\") pod \"nova-api-0\" (UID: \"ed7f9920-a1e4-437b-8074-a30d0fe4450c\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.957213 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/ed7f9920-a1e4-437b-8074-a30d0fe4450c-internal-tls-certs\") pod \"nova-api-0\" (UID: \"ed7f9920-a1e4-437b-8074-a30d0fe4450c\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.957320 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/ed7f9920-a1e4-437b-8074-a30d0fe4450c-config-data\") pod \"nova-api-0\" (UID: \"ed7f9920-a1e4-437b-8074-a30d0fe4450c\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.957758 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/ed7f9920-a1e4-437b-8074-a30d0fe4450c-logs\") pod \"nova-api-0\" (UID: \"ed7f9920-a1e4-437b-8074-a30d0fe4450c\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.958861 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-r8cvt\" (UniqueName: \"kubernetes.io/projected/ed7f9920-a1e4-437b-8074-a30d0fe4450c-kube-api-access-r8cvt\") pod \"nova-api-0\" (UID: \"ed7f9920-a1e4-437b-8074-a30d0fe4450c\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.959124 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ed7f9920-a1e4-437b-8074-a30d0fe4450c-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"ed7f9920-a1e4-437b-8074-a30d0fe4450c\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:46:18 crc kubenswrapper[5052]: W1203 12:46:18.956669 5052 empty_dir.go:500] Warning: Unmount skipped because path does not exist: /var/lib/kubelet/pods/3ae64378-aaa4-46bc-8c48-e1d90e490016/volumes/kubernetes.io~secret/combined-ca-bundle Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.959487 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3ae64378-aaa4-46bc-8c48-e1d90e490016-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "3ae64378-aaa4-46bc-8c48-e1d90e490016" (UID: "3ae64378-aaa4-46bc-8c48-e1d90e490016"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:46:18 crc kubenswrapper[5052]: E1203 12:46:18.956787 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-nova-public-svc: secret "cert-nova-public-svc" not found Dec 03 12:46:18 crc kubenswrapper[5052]: E1203 12:46:18.959681 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ed7f9920-a1e4-437b-8074-a30d0fe4450c-public-tls-certs podName:ed7f9920-a1e4-437b-8074-a30d0fe4450c nodeName:}" failed. No retries permitted until 2025-12-03 12:46:19.459665521 +0000 UTC m=+5170.047907285 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "public-tls-certs" (UniqueName: "kubernetes.io/secret/ed7f9920-a1e4-437b-8074-a30d0fe4450c-public-tls-certs") pod "nova-api-0" (UID: "ed7f9920-a1e4-437b-8074-a30d0fe4450c") : secret "cert-nova-public-svc" not found Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.958897 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/ed7f9920-a1e4-437b-8074-a30d0fe4450c-logs\") pod \"nova-api-0\" (UID: \"ed7f9920-a1e4-437b-8074-a30d0fe4450c\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:46:18 crc kubenswrapper[5052]: E1203 12:46:18.958059 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-nova-internal-svc: secret "cert-nova-internal-svc" not found Dec 03 12:46:18 crc kubenswrapper[5052]: E1203 12:46:18.962186 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ed7f9920-a1e4-437b-8074-a30d0fe4450c-internal-tls-certs podName:ed7f9920-a1e4-437b-8074-a30d0fe4450c nodeName:}" failed. No retries permitted until 2025-12-03 12:46:19.462172503 +0000 UTC m=+5170.050414267 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "internal-tls-certs" (UniqueName: "kubernetes.io/secret/ed7f9920-a1e4-437b-8074-a30d0fe4450c-internal-tls-certs") pod "nova-api-0" (UID: "ed7f9920-a1e4-437b-8074-a30d0fe4450c") : secret "cert-nova-internal-svc" not found Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.962976 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ed7f9920-a1e4-437b-8074-a30d0fe4450c-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"ed7f9920-a1e4-437b-8074-a30d0fe4450c\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.972155 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/ed7f9920-a1e4-437b-8074-a30d0fe4450c-config-data\") pod \"nova-api-0\" (UID: \"ed7f9920-a1e4-437b-8074-a30d0fe4450c\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.978568 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-r8cvt\" (UniqueName: \"kubernetes.io/projected/ed7f9920-a1e4-437b-8074-a30d0fe4450c-kube-api-access-r8cvt\") pod \"nova-api-0\" (UID: \"ed7f9920-a1e4-437b-8074-a30d0fe4450c\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:46:18 crc kubenswrapper[5052]: I1203 12:46:18.993150 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.011329 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"3ae64378-aaa4-46bc-8c48-e1d90e490016","Type":"ContainerDied","Data":"91363cf0c2a5494bdb58587815a54d4ffca02c51e4af6463e1c49693b97ab6a5"} Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.011424 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.016914 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-worker-58f7cb7fd9-2k5tg" event={"ID":"e14465ed-868a-49ba-b3ab-1e67196229e8","Type":"ContainerStarted","Data":"962dbb81a097bd1736b9740b6fea46c9644f48229fd42d2dc734698665523114"} Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.020110 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.044698 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.046900 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-bbcfbbc9d-5q6gx" event={"ID":"32f420c2-aa7e-44c8-a63a-7654aa3e6e34","Type":"ContainerStarted","Data":"353ad09a2eae7f22cab9c43ca6063becb9fb6d4010c1a66dcef0e6673635198d"} Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.047224 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.047322 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/placement-bbcfbbc9d-5q6gx" Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.047403 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/placement-bbcfbbc9d-5q6gx" Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.061173 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-scripts" Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.061926 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-default-external-config-data" Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.061901 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-glance-dockercfg-ggt4k" Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.062528 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/576b2af7-7647-4a45-9423-3aa752d84e75-galera-tls-certs\") pod \"openstack-cell1-galera-0\" (UID: \"576b2af7-7647-4a45-9423-3aa752d84e75\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:46:19 crc kubenswrapper[5052]: E1203 12:46:19.063759 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-galera-openstack-cell1-svc: secret "cert-galera-openstack-cell1-svc" not found Dec 03 12:46:19 crc kubenswrapper[5052]: E1203 12:46:19.063829 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/576b2af7-7647-4a45-9423-3aa752d84e75-galera-tls-certs podName:576b2af7-7647-4a45-9423-3aa752d84e75 nodeName:}" failed. No retries permitted until 2025-12-03 12:46:20.06380819 +0000 UTC m=+5170.652050014 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "galera-tls-certs" (UniqueName: "kubernetes.io/secret/576b2af7-7647-4a45-9423-3aa752d84e75-galera-tls-certs") pod "openstack-cell1-galera-0" (UID: "576b2af7-7647-4a45-9423-3aa752d84e75") : secret "cert-galera-openstack-cell1-svc" not found Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.065187 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-glance-default-public-svc" Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.065657 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.066649 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zw7qg\" (UniqueName: \"kubernetes.io/projected/75266130-5854-4acd-acc9-bc22accc19bc-kube-api-access-zw7qg\") pod \"nova-metadata-0\" (UID: \"75266130-5854-4acd-acc9-bc22accc19bc\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.066721 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/75266130-5854-4acd-acc9-bc22accc19bc-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"75266130-5854-4acd-acc9-bc22accc19bc\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.066745 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/75266130-5854-4acd-acc9-bc22accc19bc-logs\") pod \"nova-metadata-0\" (UID: \"75266130-5854-4acd-acc9-bc22accc19bc\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.066872 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/75266130-5854-4acd-acc9-bc22accc19bc-nova-metadata-tls-certs\") pod \"nova-metadata-0\" (UID: \"75266130-5854-4acd-acc9-bc22accc19bc\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.068816 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/75266130-5854-4acd-acc9-bc22accc19bc-config-data\") pod \"nova-metadata-0\" (UID: \"75266130-5854-4acd-acc9-bc22accc19bc\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.069337 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3ae64378-aaa4-46bc-8c48-e1d90e490016-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.079422 5052 scope.go:117] "RemoveContainer" containerID="9ba6898618d6ecc3b57802d08d86da02201fb0d67fcd7288c3414ce61b3233ed" Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.104870 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.110653 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3ae64378-aaa4-46bc-8c48-e1d90e490016-config-data" (OuterVolumeSpecName: "config-data") pod "3ae64378-aaa4-46bc-8c48-e1d90e490016" (UID: "3ae64378-aaa4-46bc-8c48-e1d90e490016"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.149805 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.165617 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.167786 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.170409 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/4c9f4da8-f1ae-4815-8ee2-0e873fb07d62-logs\") pod \"glance-default-external-api-0\" (UID: \"4c9f4da8-f1ae-4815-8ee2-0e873fb07d62\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.170459 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-zw7qg\" (UniqueName: \"kubernetes.io/projected/75266130-5854-4acd-acc9-bc22accc19bc-kube-api-access-zw7qg\") pod \"nova-metadata-0\" (UID: \"75266130-5854-4acd-acc9-bc22accc19bc\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.170499 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/4c9f4da8-f1ae-4815-8ee2-0e873fb07d62-config-data\") pod \"glance-default-external-api-0\" (UID: \"4c9f4da8-f1ae-4815-8ee2-0e873fb07d62\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.170558 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/75266130-5854-4acd-acc9-bc22accc19bc-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"75266130-5854-4acd-acc9-bc22accc19bc\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.170575 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/75266130-5854-4acd-acc9-bc22accc19bc-logs\") pod \"nova-metadata-0\" (UID: \"75266130-5854-4acd-acc9-bc22accc19bc\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.170666 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/75266130-5854-4acd-acc9-bc22accc19bc-nova-metadata-tls-certs\") pod \"nova-metadata-0\" (UID: \"75266130-5854-4acd-acc9-bc22accc19bc\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.170702 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4c9f4da8-f1ae-4815-8ee2-0e873fb07d62-combined-ca-bundle\") pod \"glance-default-external-api-0\" (UID: \"4c9f4da8-f1ae-4815-8ee2-0e873fb07d62\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.170724 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage12-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage12-crc\") pod \"glance-default-external-api-0\" (UID: \"4c9f4da8-f1ae-4815-8ee2-0e873fb07d62\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.170799 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/75266130-5854-4acd-acc9-bc22accc19bc-config-data\") pod \"nova-metadata-0\" (UID: \"75266130-5854-4acd-acc9-bc22accc19bc\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.170908 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/4c9f4da8-f1ae-4815-8ee2-0e873fb07d62-scripts\") pod \"glance-default-external-api-0\" (UID: \"4c9f4da8-f1ae-4815-8ee2-0e873fb07d62\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.170935 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/4c9f4da8-f1ae-4815-8ee2-0e873fb07d62-public-tls-certs\") pod \"glance-default-external-api-0\" (UID: \"4c9f4da8-f1ae-4815-8ee2-0e873fb07d62\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.170955 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7l7pm\" (UniqueName: \"kubernetes.io/projected/4c9f4da8-f1ae-4815-8ee2-0e873fb07d62-kube-api-access-7l7pm\") pod \"glance-default-external-api-0\" (UID: \"4c9f4da8-f1ae-4815-8ee2-0e873fb07d62\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.170983 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/4c9f4da8-f1ae-4815-8ee2-0e873fb07d62-httpd-run\") pod \"glance-default-external-api-0\" (UID: \"4c9f4da8-f1ae-4815-8ee2-0e873fb07d62\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.171096 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/3ae64378-aaa4-46bc-8c48-e1d90e490016-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:19 crc kubenswrapper[5052]: E1203 12:46:19.171182 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-placement-public-svc: secret "cert-placement-public-svc" not found Dec 03 12:46:19 crc kubenswrapper[5052]: E1203 12:46:19.171224 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/32f420c2-aa7e-44c8-a63a-7654aa3e6e34-public-tls-certs podName:32f420c2-aa7e-44c8-a63a-7654aa3e6e34 nodeName:}" failed. No retries permitted until 2025-12-03 12:46:19.671212451 +0000 UTC m=+5170.259454215 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "public-tls-certs" (UniqueName: "kubernetes.io/secret/32f420c2-aa7e-44c8-a63a-7654aa3e6e34-public-tls-certs") pod "placement-bbcfbbc9d-5q6gx" (UID: "32f420c2-aa7e-44c8-a63a-7654aa3e6e34") : secret "cert-placement-public-svc" not found Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.172081 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-glance-default-internal-svc" Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.172786 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-default-internal-config-data" Dec 03 12:46:19 crc kubenswrapper[5052]: E1203 12:46:19.173658 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-placement-internal-svc: secret "cert-placement-internal-svc" not found Dec 03 12:46:19 crc kubenswrapper[5052]: E1203 12:46:19.173694 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/32f420c2-aa7e-44c8-a63a-7654aa3e6e34-internal-tls-certs podName:32f420c2-aa7e-44c8-a63a-7654aa3e6e34 nodeName:}" failed. No retries permitted until 2025-12-03 12:46:19.673684842 +0000 UTC m=+5170.261926596 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "internal-tls-certs" (UniqueName: "kubernetes.io/secret/32f420c2-aa7e-44c8-a63a-7654aa3e6e34-internal-tls-certs") pod "placement-bbcfbbc9d-5q6gx" (UID: "32f420c2-aa7e-44c8-a63a-7654aa3e6e34") : secret "cert-placement-internal-svc" not found Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.173908 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/75266130-5854-4acd-acc9-bc22accc19bc-logs\") pod \"nova-metadata-0\" (UID: \"75266130-5854-4acd-acc9-bc22accc19bc\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.176830 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/75266130-5854-4acd-acc9-bc22accc19bc-nova-metadata-tls-certs\") pod \"nova-metadata-0\" (UID: \"75266130-5854-4acd-acc9-bc22accc19bc\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.176888 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/75266130-5854-4acd-acc9-bc22accc19bc-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"75266130-5854-4acd-acc9-bc22accc19bc\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.180814 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.193909 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.199902 5052 scope.go:117] "RemoveContainer" containerID="deb3641e5c7641ec59de2756335b3b8af86a98493025f1061ab9587775677d7d" Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.204188 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-zw7qg\" (UniqueName: \"kubernetes.io/projected/75266130-5854-4acd-acc9-bc22accc19bc-kube-api-access-zw7qg\") pod \"nova-metadata-0\" (UID: \"75266130-5854-4acd-acc9-bc22accc19bc\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.204199 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/75266130-5854-4acd-acc9-bc22accc19bc-config-data\") pod \"nova-metadata-0\" (UID: \"75266130-5854-4acd-acc9-bc22accc19bc\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.208493 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.215613 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.217843 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.223128 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-cinder-internal-svc" Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.223659 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cinder-api-config-data" Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.225340 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.230169 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-cinder-public-svc" Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.260261 5052 scope.go:117] "RemoveContainer" containerID="3242b025201bb8378bb8f5f363d45551adfc8da4587f92d211c75e0e901ef11a" Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.267139 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/barbican-worker-58f7cb7fd9-2k5tg" podStartSLOduration=8.267116864 podStartE2EDuration="8.267116864s" podCreationTimestamp="2025-12-03 12:46:11 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:46:19.03831002 +0000 UTC m=+5169.626551784" watchObservedRunningTime="2025-12-03 12:46:19.267116864 +0000 UTC m=+5169.855358638" Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.272352 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hzlsc\" (UniqueName: \"kubernetes.io/projected/bf8b9b9c-b8f9-4c93-8112-43913950fcdf-kube-api-access-hzlsc\") pod \"glance-default-internal-api-0\" (UID: \"bf8b9b9c-b8f9-4c93-8112-43913950fcdf\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.272467 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/4c9f4da8-f1ae-4815-8ee2-0e873fb07d62-logs\") pod \"glance-default-external-api-0\" (UID: \"4c9f4da8-f1ae-4815-8ee2-0e873fb07d62\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.272498 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/bf8b9b9c-b8f9-4c93-8112-43913950fcdf-logs\") pod \"glance-default-internal-api-0\" (UID: \"bf8b9b9c-b8f9-4c93-8112-43913950fcdf\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.272538 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/4c9f4da8-f1ae-4815-8ee2-0e873fb07d62-config-data\") pod \"glance-default-external-api-0\" (UID: \"4c9f4da8-f1ae-4815-8ee2-0e873fb07d62\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.272623 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage18-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage18-crc\") pod \"glance-default-internal-api-0\" (UID: \"bf8b9b9c-b8f9-4c93-8112-43913950fcdf\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.272711 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4c9f4da8-f1ae-4815-8ee2-0e873fb07d62-combined-ca-bundle\") pod \"glance-default-external-api-0\" (UID: \"4c9f4da8-f1ae-4815-8ee2-0e873fb07d62\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.272738 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage12-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage12-crc\") pod \"glance-default-external-api-0\" (UID: \"4c9f4da8-f1ae-4815-8ee2-0e873fb07d62\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.272919 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/bf8b9b9c-b8f9-4c93-8112-43913950fcdf-config-data\") pod \"glance-default-internal-api-0\" (UID: \"bf8b9b9c-b8f9-4c93-8112-43913950fcdf\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.272995 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/bf8b9b9c-b8f9-4c93-8112-43913950fcdf-httpd-run\") pod \"glance-default-internal-api-0\" (UID: \"bf8b9b9c-b8f9-4c93-8112-43913950fcdf\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.273084 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/bf8b9b9c-b8f9-4c93-8112-43913950fcdf-combined-ca-bundle\") pod \"glance-default-internal-api-0\" (UID: \"bf8b9b9c-b8f9-4c93-8112-43913950fcdf\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.273177 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/4c9f4da8-f1ae-4815-8ee2-0e873fb07d62-scripts\") pod \"glance-default-external-api-0\" (UID: \"4c9f4da8-f1ae-4815-8ee2-0e873fb07d62\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.273204 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/4c9f4da8-f1ae-4815-8ee2-0e873fb07d62-public-tls-certs\") pod \"glance-default-external-api-0\" (UID: \"4c9f4da8-f1ae-4815-8ee2-0e873fb07d62\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.273253 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-7l7pm\" (UniqueName: \"kubernetes.io/projected/4c9f4da8-f1ae-4815-8ee2-0e873fb07d62-kube-api-access-7l7pm\") pod \"glance-default-external-api-0\" (UID: \"4c9f4da8-f1ae-4815-8ee2-0e873fb07d62\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.273276 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/bf8b9b9c-b8f9-4c93-8112-43913950fcdf-scripts\") pod \"glance-default-internal-api-0\" (UID: \"bf8b9b9c-b8f9-4c93-8112-43913950fcdf\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.273338 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/4c9f4da8-f1ae-4815-8ee2-0e873fb07d62-httpd-run\") pod \"glance-default-external-api-0\" (UID: \"4c9f4da8-f1ae-4815-8ee2-0e873fb07d62\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.273431 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/bf8b9b9c-b8f9-4c93-8112-43913950fcdf-internal-tls-certs\") pod \"glance-default-internal-api-0\" (UID: \"bf8b9b9c-b8f9-4c93-8112-43913950fcdf\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.274152 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/4c9f4da8-f1ae-4815-8ee2-0e873fb07d62-logs\") pod \"glance-default-external-api-0\" (UID: \"4c9f4da8-f1ae-4815-8ee2-0e873fb07d62\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.274904 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage12-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage12-crc\") pod \"glance-default-external-api-0\" (UID: \"4c9f4da8-f1ae-4815-8ee2-0e873fb07d62\") device mount path \"/mnt/openstack/pv12\"" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.275863 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/4c9f4da8-f1ae-4815-8ee2-0e873fb07d62-httpd-run\") pod \"glance-default-external-api-0\" (UID: \"4c9f4da8-f1ae-4815-8ee2-0e873fb07d62\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:46:19 crc kubenswrapper[5052]: E1203 12:46:19.276364 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-glance-default-public-svc: secret "cert-glance-default-public-svc" not found Dec 03 12:46:19 crc kubenswrapper[5052]: E1203 12:46:19.276432 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4c9f4da8-f1ae-4815-8ee2-0e873fb07d62-public-tls-certs podName:4c9f4da8-f1ae-4815-8ee2-0e873fb07d62 nodeName:}" failed. No retries permitted until 2025-12-03 12:46:19.77641058 +0000 UTC m=+5170.364652424 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "public-tls-certs" (UniqueName: "kubernetes.io/secret/4c9f4da8-f1ae-4815-8ee2-0e873fb07d62-public-tls-certs") pod "glance-default-external-api-0" (UID: "4c9f4da8-f1ae-4815-8ee2-0e873fb07d62") : secret "cert-glance-default-public-svc" not found Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.284686 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/4c9f4da8-f1ae-4815-8ee2-0e873fb07d62-config-data\") pod \"glance-default-external-api-0\" (UID: \"4c9f4da8-f1ae-4815-8ee2-0e873fb07d62\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.285488 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4c9f4da8-f1ae-4815-8ee2-0e873fb07d62-combined-ca-bundle\") pod \"glance-default-external-api-0\" (UID: \"4c9f4da8-f1ae-4815-8ee2-0e873fb07d62\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.287626 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/4c9f4da8-f1ae-4815-8ee2-0e873fb07d62-scripts\") pod \"glance-default-external-api-0\" (UID: \"4c9f4da8-f1ae-4815-8ee2-0e873fb07d62\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.290682 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-worker-65fcd76cf7-xhs44"] Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.290938 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/barbican-worker-65fcd76cf7-xhs44" podUID="92290cff-ed52-4a47-80e9-720bfd9f3dd8" containerName="barbican-worker-log" containerID="cri-o://6da49bc767ca3706e94a6d9f283174f99fca40c8d0d7ff4717a94a22b54ce36c" gracePeriod=30 Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.291443 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/barbican-worker-65fcd76cf7-xhs44" podUID="92290cff-ed52-4a47-80e9-720bfd9f3dd8" containerName="barbican-worker" containerID="cri-o://f1a23446d7321c40e536bf2e9b6174acf23d15d1d963daca95ff134b7d58cd46" gracePeriod=30 Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.293064 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/placement-bbcfbbc9d-5q6gx" podStartSLOduration=8.293027845 podStartE2EDuration="8.293027845s" podCreationTimestamp="2025-12-03 12:46:11 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:46:19.081694011 +0000 UTC m=+5169.669935775" watchObservedRunningTime="2025-12-03 12:46:19.293027845 +0000 UTC m=+5169.881269609" Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.313027 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-7l7pm\" (UniqueName: \"kubernetes.io/projected/4c9f4da8-f1ae-4815-8ee2-0e873fb07d62-kube-api-access-7l7pm\") pod \"glance-default-external-api-0\" (UID: \"4c9f4da8-f1ae-4815-8ee2-0e873fb07d62\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.322716 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage12-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage12-crc\") pod \"glance-default-external-api-0\" (UID: \"4c9f4da8-f1ae-4815-8ee2-0e873fb07d62\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.349876 5052 scope.go:117] "RemoveContainer" containerID="c5ed1d5bcb2e814a4e6ff6889daaa85dec6c18fe852411af7cfacd20fd4d3320" Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.361712 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.373017 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.380241 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/bf8b9b9c-b8f9-4c93-8112-43913950fcdf-config-data\") pod \"glance-default-internal-api-0\" (UID: \"bf8b9b9c-b8f9-4c93-8112-43913950fcdf\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.380285 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/234278ab-b4a8-4fdf-abf9-c67cdbff1ac0-logs\") pod \"cinder-api-0\" (UID: \"234278ab-b4a8-4fdf-abf9-c67cdbff1ac0\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.380305 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/bf8b9b9c-b8f9-4c93-8112-43913950fcdf-httpd-run\") pod \"glance-default-internal-api-0\" (UID: \"bf8b9b9c-b8f9-4c93-8112-43913950fcdf\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.380338 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/bf8b9b9c-b8f9-4c93-8112-43913950fcdf-combined-ca-bundle\") pod \"glance-default-internal-api-0\" (UID: \"bf8b9b9c-b8f9-4c93-8112-43913950fcdf\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.380387 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/234278ab-b4a8-4fdf-abf9-c67cdbff1ac0-config-data\") pod \"cinder-api-0\" (UID: \"234278ab-b4a8-4fdf-abf9-c67cdbff1ac0\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.380408 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/bf8b9b9c-b8f9-4c93-8112-43913950fcdf-scripts\") pod \"glance-default-internal-api-0\" (UID: \"bf8b9b9c-b8f9-4c93-8112-43913950fcdf\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.380699 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/234278ab-b4a8-4fdf-abf9-c67cdbff1ac0-internal-tls-certs\") pod \"cinder-api-0\" (UID: \"234278ab-b4a8-4fdf-abf9-c67cdbff1ac0\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.380727 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/234278ab-b4a8-4fdf-abf9-c67cdbff1ac0-combined-ca-bundle\") pod \"cinder-api-0\" (UID: \"234278ab-b4a8-4fdf-abf9-c67cdbff1ac0\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.380770 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/bf8b9b9c-b8f9-4c93-8112-43913950fcdf-internal-tls-certs\") pod \"glance-default-internal-api-0\" (UID: \"bf8b9b9c-b8f9-4c93-8112-43913950fcdf\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.380817 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-hzlsc\" (UniqueName: \"kubernetes.io/projected/bf8b9b9c-b8f9-4c93-8112-43913950fcdf-kube-api-access-hzlsc\") pod \"glance-default-internal-api-0\" (UID: \"bf8b9b9c-b8f9-4c93-8112-43913950fcdf\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.380847 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/234278ab-b4a8-4fdf-abf9-c67cdbff1ac0-etc-machine-id\") pod \"cinder-api-0\" (UID: \"234278ab-b4a8-4fdf-abf9-c67cdbff1ac0\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.380918 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/234278ab-b4a8-4fdf-abf9-c67cdbff1ac0-config-data-custom\") pod \"cinder-api-0\" (UID: \"234278ab-b4a8-4fdf-abf9-c67cdbff1ac0\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.380941 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/bf8b9b9c-b8f9-4c93-8112-43913950fcdf-logs\") pod \"glance-default-internal-api-0\" (UID: \"bf8b9b9c-b8f9-4c93-8112-43913950fcdf\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.381029 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage18-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage18-crc\") pod \"glance-default-internal-api-0\" (UID: \"bf8b9b9c-b8f9-4c93-8112-43913950fcdf\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.381758 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/234278ab-b4a8-4fdf-abf9-c67cdbff1ac0-scripts\") pod \"cinder-api-0\" (UID: \"234278ab-b4a8-4fdf-abf9-c67cdbff1ac0\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.381817 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/234278ab-b4a8-4fdf-abf9-c67cdbff1ac0-public-tls-certs\") pod \"cinder-api-0\" (UID: \"234278ab-b4a8-4fdf-abf9-c67cdbff1ac0\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.381851 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j52n8\" (UniqueName: \"kubernetes.io/projected/234278ab-b4a8-4fdf-abf9-c67cdbff1ac0-kube-api-access-j52n8\") pod \"cinder-api-0\" (UID: \"234278ab-b4a8-4fdf-abf9-c67cdbff1ac0\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.382531 5052 scope.go:117] "RemoveContainer" containerID="3fa64d6a78f92e04f85cc78283bcecfe7e618fa805695f4e69f5237e051e4939" Dec 03 12:46:19 crc kubenswrapper[5052]: E1203 12:46:19.382917 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-swift-internal-svc: secret "cert-swift-internal-svc" not found Dec 03 12:46:19 crc kubenswrapper[5052]: E1203 12:46:19.383086 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/242be027-6461-4aea-ae7c-e52ca996b2ea-internal-tls-certs podName:242be027-6461-4aea-ae7c-e52ca996b2ea nodeName:}" failed. No retries permitted until 2025-12-03 12:46:20.38306851 +0000 UTC m=+5170.971310324 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "internal-tls-certs" (UniqueName: "kubernetes.io/secret/242be027-6461-4aea-ae7c-e52ca996b2ea-internal-tls-certs") pod "swift-proxy-dd4cf57c8-x4c45" (UID: "242be027-6461-4aea-ae7c-e52ca996b2ea") : secret "cert-swift-internal-svc" not found Dec 03 12:46:19 crc kubenswrapper[5052]: E1203 12:46:19.383239 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-swift-public-svc: secret "cert-swift-public-svc" not found Dec 03 12:46:19 crc kubenswrapper[5052]: E1203 12:46:19.383369 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/242be027-6461-4aea-ae7c-e52ca996b2ea-public-tls-certs podName:242be027-6461-4aea-ae7c-e52ca996b2ea nodeName:}" failed. No retries permitted until 2025-12-03 12:46:20.383356568 +0000 UTC m=+5170.971598332 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "public-tls-certs" (UniqueName: "kubernetes.io/secret/242be027-6461-4aea-ae7c-e52ca996b2ea-public-tls-certs") pod "swift-proxy-dd4cf57c8-x4c45" (UID: "242be027-6461-4aea-ae7c-e52ca996b2ea") : secret "cert-swift-public-svc" not found Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.383308 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/bf8b9b9c-b8f9-4c93-8112-43913950fcdf-logs\") pod \"glance-default-internal-api-0\" (UID: \"bf8b9b9c-b8f9-4c93-8112-43913950fcdf\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:46:19 crc kubenswrapper[5052]: E1203 12:46:19.383694 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-glance-default-internal-svc: secret "cert-glance-default-internal-svc" not found Dec 03 12:46:19 crc kubenswrapper[5052]: E1203 12:46:19.383791 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bf8b9b9c-b8f9-4c93-8112-43913950fcdf-internal-tls-certs podName:bf8b9b9c-b8f9-4c93-8112-43913950fcdf nodeName:}" failed. No retries permitted until 2025-12-03 12:46:19.883733629 +0000 UTC m=+5170.471975493 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "internal-tls-certs" (UniqueName: "kubernetes.io/secret/bf8b9b9c-b8f9-4c93-8112-43913950fcdf-internal-tls-certs") pod "glance-default-internal-api-0" (UID: "bf8b9b9c-b8f9-4c93-8112-43913950fcdf") : secret "cert-glance-default-internal-svc" not found Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.383182 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage18-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage18-crc\") pod \"glance-default-internal-api-0\" (UID: \"bf8b9b9c-b8f9-4c93-8112-43913950fcdf\") device mount path \"/mnt/openstack/pv18\"" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.384531 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/bf8b9b9c-b8f9-4c93-8112-43913950fcdf-httpd-run\") pod \"glance-default-internal-api-0\" (UID: \"bf8b9b9c-b8f9-4c93-8112-43913950fcdf\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.390773 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/bf8b9b9c-b8f9-4c93-8112-43913950fcdf-combined-ca-bundle\") pod \"glance-default-internal-api-0\" (UID: \"bf8b9b9c-b8f9-4c93-8112-43913950fcdf\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.393505 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/bf8b9b9c-b8f9-4c93-8112-43913950fcdf-scripts\") pod \"glance-default-internal-api-0\" (UID: \"bf8b9b9c-b8f9-4c93-8112-43913950fcdf\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.408543 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/bf8b9b9c-b8f9-4c93-8112-43913950fcdf-config-data\") pod \"glance-default-internal-api-0\" (UID: \"bf8b9b9c-b8f9-4c93-8112-43913950fcdf\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.409685 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-hzlsc\" (UniqueName: \"kubernetes.io/projected/bf8b9b9c-b8f9-4c93-8112-43913950fcdf-kube-api-access-hzlsc\") pod \"glance-default-internal-api-0\" (UID: \"bf8b9b9c-b8f9-4c93-8112-43913950fcdf\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.416365 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.426143 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage18-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage18-crc\") pod \"glance-default-internal-api-0\" (UID: \"bf8b9b9c-b8f9-4c93-8112-43913950fcdf\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.435758 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.472740 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.476818 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.478673 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cinder-scheduler-config-data" Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.483301 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/234278ab-b4a8-4fdf-abf9-c67cdbff1ac0-etc-machine-id\") pod \"cinder-api-0\" (UID: \"234278ab-b4a8-4fdf-abf9-c67cdbff1ac0\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.492167 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/234278ab-b4a8-4fdf-abf9-c67cdbff1ac0-config-data-custom\") pod \"cinder-api-0\" (UID: \"234278ab-b4a8-4fdf-abf9-c67cdbff1ac0\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.483676 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/234278ab-b4a8-4fdf-abf9-c67cdbff1ac0-etc-machine-id\") pod \"cinder-api-0\" (UID: \"234278ab-b4a8-4fdf-abf9-c67cdbff1ac0\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.492339 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/c6c9b62e-bbbc-4557-92fd-da64bd8ee0cf-metrics-certs-tls-certs\") pod \"ovsdbserver-sb-0\" (UID: \"c6c9b62e-bbbc-4557-92fd-da64bd8ee0cf\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.492389 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/ed7f9920-a1e4-437b-8074-a30d0fe4450c-public-tls-certs\") pod \"nova-api-0\" (UID: \"ed7f9920-a1e4-437b-8074-a30d0fe4450c\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.492486 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/234278ab-b4a8-4fdf-abf9-c67cdbff1ac0-scripts\") pod \"cinder-api-0\" (UID: \"234278ab-b4a8-4fdf-abf9-c67cdbff1ac0\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.492552 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/ed7f9920-a1e4-437b-8074-a30d0fe4450c-internal-tls-certs\") pod \"nova-api-0\" (UID: \"ed7f9920-a1e4-437b-8074-a30d0fe4450c\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.492583 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/234278ab-b4a8-4fdf-abf9-c67cdbff1ac0-public-tls-certs\") pod \"cinder-api-0\" (UID: \"234278ab-b4a8-4fdf-abf9-c67cdbff1ac0\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.492631 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-j52n8\" (UniqueName: \"kubernetes.io/projected/234278ab-b4a8-4fdf-abf9-c67cdbff1ac0-kube-api-access-j52n8\") pod \"cinder-api-0\" (UID: \"234278ab-b4a8-4fdf-abf9-c67cdbff1ac0\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.492803 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/234278ab-b4a8-4fdf-abf9-c67cdbff1ac0-logs\") pod \"cinder-api-0\" (UID: \"234278ab-b4a8-4fdf-abf9-c67cdbff1ac0\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.492903 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/234278ab-b4a8-4fdf-abf9-c67cdbff1ac0-config-data\") pod \"cinder-api-0\" (UID: \"234278ab-b4a8-4fdf-abf9-c67cdbff1ac0\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.496384 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/234278ab-b4a8-4fdf-abf9-c67cdbff1ac0-internal-tls-certs\") pod \"cinder-api-0\" (UID: \"234278ab-b4a8-4fdf-abf9-c67cdbff1ac0\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.496426 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/234278ab-b4a8-4fdf-abf9-c67cdbff1ac0-combined-ca-bundle\") pod \"cinder-api-0\" (UID: \"234278ab-b4a8-4fdf-abf9-c67cdbff1ac0\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:46:19 crc kubenswrapper[5052]: E1203 12:46:19.497193 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-ovn-metrics: secret "cert-ovn-metrics" not found Dec 03 12:46:19 crc kubenswrapper[5052]: E1203 12:46:19.497262 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c6c9b62e-bbbc-4557-92fd-da64bd8ee0cf-metrics-certs-tls-certs podName:c6c9b62e-bbbc-4557-92fd-da64bd8ee0cf nodeName:}" failed. No retries permitted until 2025-12-03 12:46:23.497240465 +0000 UTC m=+5174.085482309 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs-tls-certs" (UniqueName: "kubernetes.io/secret/c6c9b62e-bbbc-4557-92fd-da64bd8ee0cf-metrics-certs-tls-certs") pod "ovsdbserver-sb-0" (UID: "c6c9b62e-bbbc-4557-92fd-da64bd8ee0cf") : secret "cert-ovn-metrics" not found Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.497864 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/234278ab-b4a8-4fdf-abf9-c67cdbff1ac0-config-data-custom\") pod \"cinder-api-0\" (UID: \"234278ab-b4a8-4fdf-abf9-c67cdbff1ac0\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:46:19 crc kubenswrapper[5052]: E1203 12:46:19.500456 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-nova-public-svc: secret "cert-nova-public-svc" not found Dec 03 12:46:19 crc kubenswrapper[5052]: E1203 12:46:19.500553 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ed7f9920-a1e4-437b-8074-a30d0fe4450c-public-tls-certs podName:ed7f9920-a1e4-437b-8074-a30d0fe4450c nodeName:}" failed. No retries permitted until 2025-12-03 12:46:20.500502459 +0000 UTC m=+5171.088744283 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "public-tls-certs" (UniqueName: "kubernetes.io/secret/ed7f9920-a1e4-437b-8074-a30d0fe4450c-public-tls-certs") pod "nova-api-0" (UID: "ed7f9920-a1e4-437b-8074-a30d0fe4450c") : secret "cert-nova-public-svc" not found Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.500784 5052 scope.go:117] "RemoveContainer" containerID="8e0b376c29b7c2dd056afa36e69404e2be33493fa6b1a9ba35735a9c8becf167" Dec 03 12:46:19 crc kubenswrapper[5052]: E1203 12:46:19.501184 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-cinder-public-svc: secret "cert-cinder-public-svc" not found Dec 03 12:46:19 crc kubenswrapper[5052]: E1203 12:46:19.501218 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/234278ab-b4a8-4fdf-abf9-c67cdbff1ac0-public-tls-certs podName:234278ab-b4a8-4fdf-abf9-c67cdbff1ac0 nodeName:}" failed. No retries permitted until 2025-12-03 12:46:20.001208019 +0000 UTC m=+5170.589449783 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "public-tls-certs" (UniqueName: "kubernetes.io/secret/234278ab-b4a8-4fdf-abf9-c67cdbff1ac0-public-tls-certs") pod "cinder-api-0" (UID: "234278ab-b4a8-4fdf-abf9-c67cdbff1ac0") : secret "cert-cinder-public-svc" not found Dec 03 12:46:19 crc kubenswrapper[5052]: E1203 12:46:19.501252 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-nova-internal-svc: secret "cert-nova-internal-svc" not found Dec 03 12:46:19 crc kubenswrapper[5052]: E1203 12:46:19.501271 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ed7f9920-a1e4-437b-8074-a30d0fe4450c-internal-tls-certs podName:ed7f9920-a1e4-437b-8074-a30d0fe4450c nodeName:}" failed. No retries permitted until 2025-12-03 12:46:20.50126475 +0000 UTC m=+5171.089506514 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "internal-tls-certs" (UniqueName: "kubernetes.io/secret/ed7f9920-a1e4-437b-8074-a30d0fe4450c-internal-tls-certs") pod "nova-api-0" (UID: "ed7f9920-a1e4-437b-8074-a30d0fe4450c") : secret "cert-nova-internal-svc" not found Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.501372 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/234278ab-b4a8-4fdf-abf9-c67cdbff1ac0-logs\") pod \"cinder-api-0\" (UID: \"234278ab-b4a8-4fdf-abf9-c67cdbff1ac0\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:46:19 crc kubenswrapper[5052]: E1203 12:46:19.501451 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-cinder-internal-svc: secret "cert-cinder-internal-svc" not found Dec 03 12:46:19 crc kubenswrapper[5052]: E1203 12:46:19.501487 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/234278ab-b4a8-4fdf-abf9-c67cdbff1ac0-internal-tls-certs podName:234278ab-b4a8-4fdf-abf9-c67cdbff1ac0 nodeName:}" failed. No retries permitted until 2025-12-03 12:46:20.001475726 +0000 UTC m=+5170.589717490 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "internal-tls-certs" (UniqueName: "kubernetes.io/secret/234278ab-b4a8-4fdf-abf9-c67cdbff1ac0-internal-tls-certs") pod "cinder-api-0" (UID: "234278ab-b4a8-4fdf-abf9-c67cdbff1ac0") : secret "cert-cinder-internal-svc" not found Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.505572 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.511742 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/234278ab-b4a8-4fdf-abf9-c67cdbff1ac0-combined-ca-bundle\") pod \"cinder-api-0\" (UID: \"234278ab-b4a8-4fdf-abf9-c67cdbff1ac0\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.512039 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/234278ab-b4a8-4fdf-abf9-c67cdbff1ac0-config-data\") pod \"cinder-api-0\" (UID: \"234278ab-b4a8-4fdf-abf9-c67cdbff1ac0\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.511939 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/234278ab-b4a8-4fdf-abf9-c67cdbff1ac0-scripts\") pod \"cinder-api-0\" (UID: \"234278ab-b4a8-4fdf-abf9-c67cdbff1ac0\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.517494 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-0"] Dec 03 12:46:19 crc kubenswrapper[5052]: W1203 12:46:19.519769 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6d653682_52e2_4940_af66_633475bfe09a.slice/crio-1c60e1611f47b232078a4f996ef3cb8b4bd103c8fd84abb48567e8f4e445ef6f WatchSource:0}: Error finding container 1c60e1611f47b232078a4f996ef3cb8b4bd103c8fd84abb48567e8f4e445ef6f: Status 404 returned error can't find the container with id 1c60e1611f47b232078a4f996ef3cb8b4bd103c8fd84abb48567e8f4e445ef6f Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.523169 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-j52n8\" (UniqueName: \"kubernetes.io/projected/234278ab-b4a8-4fdf-abf9-c67cdbff1ac0-kube-api-access-j52n8\") pod \"cinder-api-0\" (UID: \"234278ab-b4a8-4fdf-abf9-c67cdbff1ac0\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.598885 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovsdbserver-sb-tls-certs\" (UniqueName: \"kubernetes.io/secret/c6c9b62e-bbbc-4557-92fd-da64bd8ee0cf-ovsdbserver-sb-tls-certs\") pod \"ovsdbserver-sb-0\" (UID: \"c6c9b62e-bbbc-4557-92fd-da64bd8ee0cf\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.599003 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"vencrypt-tls-certs\" (UniqueName: \"kubernetes.io/secret/9c996590-4337-477e-a236-3e834d192346-vencrypt-tls-certs\") pod \"nova-cell1-novncproxy-0\" (UID: \"9c996590-4337-477e-a236-3e834d192346\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.599111 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/aa557411-6b0a-4596-a5f5-0e23f8686dc4-config-data-custom\") pod \"cinder-scheduler-0\" (UID: \"aa557411-6b0a-4596-a5f5-0e23f8686dc4\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.599138 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/aa557411-6b0a-4596-a5f5-0e23f8686dc4-config-data\") pod \"cinder-scheduler-0\" (UID: \"aa557411-6b0a-4596-a5f5-0e23f8686dc4\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.599265 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/aa557411-6b0a-4596-a5f5-0e23f8686dc4-combined-ca-bundle\") pod \"cinder-scheduler-0\" (UID: \"aa557411-6b0a-4596-a5f5-0e23f8686dc4\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:46:19 crc kubenswrapper[5052]: E1203 12:46:19.599332 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-nova-novncproxy-cell1-vencrypt: secret "cert-nova-novncproxy-cell1-vencrypt" not found Dec 03 12:46:19 crc kubenswrapper[5052]: E1203 12:46:19.599417 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9c996590-4337-477e-a236-3e834d192346-vencrypt-tls-certs podName:9c996590-4337-477e-a236-3e834d192346 nodeName:}" failed. No retries permitted until 2025-12-03 12:46:23.599395127 +0000 UTC m=+5174.187636941 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "vencrypt-tls-certs" (UniqueName: "kubernetes.io/secret/9c996590-4337-477e-a236-3e834d192346-vencrypt-tls-certs") pod "nova-cell1-novncproxy-0" (UID: "9c996590-4337-477e-a236-3e834d192346") : secret "cert-nova-novncproxy-cell1-vencrypt" not found Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.599458 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"nova-novncproxy-tls-certs\" (UniqueName: \"kubernetes.io/secret/9c996590-4337-477e-a236-3e834d192346-nova-novncproxy-tls-certs\") pod \"nova-cell1-novncproxy-0\" (UID: \"9c996590-4337-477e-a236-3e834d192346\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.599714 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/aa557411-6b0a-4596-a5f5-0e23f8686dc4-etc-machine-id\") pod \"cinder-scheduler-0\" (UID: \"aa557411-6b0a-4596-a5f5-0e23f8686dc4\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:46:19 crc kubenswrapper[5052]: E1203 12:46:19.599978 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-ovndbcluster-sb-ovndbs: secret "cert-ovndbcluster-sb-ovndbs" not found Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.600002 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/aa557411-6b0a-4596-a5f5-0e23f8686dc4-scripts\") pod \"cinder-scheduler-0\" (UID: \"aa557411-6b0a-4596-a5f5-0e23f8686dc4\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:46:19 crc kubenswrapper[5052]: E1203 12:46:19.600041 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c6c9b62e-bbbc-4557-92fd-da64bd8ee0cf-ovsdbserver-sb-tls-certs podName:c6c9b62e-bbbc-4557-92fd-da64bd8ee0cf nodeName:}" failed. No retries permitted until 2025-12-03 12:46:23.600027415 +0000 UTC m=+5174.188269179 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "ovsdbserver-sb-tls-certs" (UniqueName: "kubernetes.io/secret/c6c9b62e-bbbc-4557-92fd-da64bd8ee0cf-ovsdbserver-sb-tls-certs") pod "ovsdbserver-sb-0" (UID: "c6c9b62e-bbbc-4557-92fd-da64bd8ee0cf") : secret "cert-ovndbcluster-sb-ovndbs" not found Dec 03 12:46:19 crc kubenswrapper[5052]: E1203 12:46:19.600090 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-nova-novncproxy-cell1-public-svc: secret "cert-nova-novncproxy-cell1-public-svc" not found Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.600105 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hfmkd\" (UniqueName: \"kubernetes.io/projected/aa557411-6b0a-4596-a5f5-0e23f8686dc4-kube-api-access-hfmkd\") pod \"cinder-scheduler-0\" (UID: \"aa557411-6b0a-4596-a5f5-0e23f8686dc4\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:46:19 crc kubenswrapper[5052]: E1203 12:46:19.600124 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9c996590-4337-477e-a236-3e834d192346-nova-novncproxy-tls-certs podName:9c996590-4337-477e-a236-3e834d192346 nodeName:}" failed. No retries permitted until 2025-12-03 12:46:23.600113357 +0000 UTC m=+5174.188355191 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "nova-novncproxy-tls-certs" (UniqueName: "kubernetes.io/secret/9c996590-4337-477e-a236-3e834d192346-nova-novncproxy-tls-certs") pod "nova-cell1-novncproxy-0" (UID: "9c996590-4337-477e-a236-3e834d192346") : secret "cert-nova-novncproxy-cell1-public-svc" not found Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.704461 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/aa557411-6b0a-4596-a5f5-0e23f8686dc4-scripts\") pod \"cinder-scheduler-0\" (UID: \"aa557411-6b0a-4596-a5f5-0e23f8686dc4\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.704574 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-hfmkd\" (UniqueName: \"kubernetes.io/projected/aa557411-6b0a-4596-a5f5-0e23f8686dc4-kube-api-access-hfmkd\") pod \"cinder-scheduler-0\" (UID: \"aa557411-6b0a-4596-a5f5-0e23f8686dc4\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.704740 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/aa557411-6b0a-4596-a5f5-0e23f8686dc4-config-data-custom\") pod \"cinder-scheduler-0\" (UID: \"aa557411-6b0a-4596-a5f5-0e23f8686dc4\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.704773 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/aa557411-6b0a-4596-a5f5-0e23f8686dc4-config-data\") pod \"cinder-scheduler-0\" (UID: \"aa557411-6b0a-4596-a5f5-0e23f8686dc4\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.704824 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/aa557411-6b0a-4596-a5f5-0e23f8686dc4-combined-ca-bundle\") pod \"cinder-scheduler-0\" (UID: \"aa557411-6b0a-4596-a5f5-0e23f8686dc4\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.704965 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/aa557411-6b0a-4596-a5f5-0e23f8686dc4-etc-machine-id\") pod \"cinder-scheduler-0\" (UID: \"aa557411-6b0a-4596-a5f5-0e23f8686dc4\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.705068 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/9481ae66-152c-41e7-9cbc-90dcb5fa6171-ceilometer-tls-certs\") pod \"ceilometer-0\" (UID: \"9481ae66-152c-41e7-9cbc-90dcb5fa6171\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:46:19 crc kubenswrapper[5052]: E1203 12:46:19.705246 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-ceilometer-internal-svc: secret "cert-ceilometer-internal-svc" not found Dec 03 12:46:19 crc kubenswrapper[5052]: E1203 12:46:19.705294 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9481ae66-152c-41e7-9cbc-90dcb5fa6171-ceilometer-tls-certs podName:9481ae66-152c-41e7-9cbc-90dcb5fa6171 nodeName:}" failed. No retries permitted until 2025-12-03 12:46:21.705281285 +0000 UTC m=+5172.293523049 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "ceilometer-tls-certs" (UniqueName: "kubernetes.io/secret/9481ae66-152c-41e7-9cbc-90dcb5fa6171-ceilometer-tls-certs") pod "ceilometer-0" (UID: "9481ae66-152c-41e7-9cbc-90dcb5fa6171") : secret "cert-ceilometer-internal-svc" not found Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.709639 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/aa557411-6b0a-4596-a5f5-0e23f8686dc4-etc-machine-id\") pod \"cinder-scheduler-0\" (UID: \"aa557411-6b0a-4596-a5f5-0e23f8686dc4\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:46:19 crc kubenswrapper[5052]: E1203 12:46:19.710230 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-placement-public-svc: secret "cert-placement-public-svc" not found Dec 03 12:46:19 crc kubenswrapper[5052]: E1203 12:46:19.710292 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/32f420c2-aa7e-44c8-a63a-7654aa3e6e34-public-tls-certs podName:32f420c2-aa7e-44c8-a63a-7654aa3e6e34 nodeName:}" failed. No retries permitted until 2025-12-03 12:46:20.710272218 +0000 UTC m=+5171.298513982 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "public-tls-certs" (UniqueName: "kubernetes.io/secret/32f420c2-aa7e-44c8-a63a-7654aa3e6e34-public-tls-certs") pod "placement-bbcfbbc9d-5q6gx" (UID: "32f420c2-aa7e-44c8-a63a-7654aa3e6e34") : secret "cert-placement-public-svc" not found Dec 03 12:46:19 crc kubenswrapper[5052]: E1203 12:46:19.715737 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-placement-internal-svc: secret "cert-placement-internal-svc" not found Dec 03 12:46:19 crc kubenswrapper[5052]: E1203 12:46:19.715830 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/32f420c2-aa7e-44c8-a63a-7654aa3e6e34-internal-tls-certs podName:32f420c2-aa7e-44c8-a63a-7654aa3e6e34 nodeName:}" failed. No retries permitted until 2025-12-03 12:46:20.715809276 +0000 UTC m=+5171.304051040 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "internal-tls-certs" (UniqueName: "kubernetes.io/secret/32f420c2-aa7e-44c8-a63a-7654aa3e6e34-internal-tls-certs") pod "placement-bbcfbbc9d-5q6gx" (UID: "32f420c2-aa7e-44c8-a63a-7654aa3e6e34") : secret "cert-placement-internal-svc" not found Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.716740 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/aa557411-6b0a-4596-a5f5-0e23f8686dc4-config-data\") pod \"cinder-scheduler-0\" (UID: \"aa557411-6b0a-4596-a5f5-0e23f8686dc4\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.721576 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/aa557411-6b0a-4596-a5f5-0e23f8686dc4-combined-ca-bundle\") pod \"cinder-scheduler-0\" (UID: \"aa557411-6b0a-4596-a5f5-0e23f8686dc4\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.726307 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/aa557411-6b0a-4596-a5f5-0e23f8686dc4-scripts\") pod \"cinder-scheduler-0\" (UID: \"aa557411-6b0a-4596-a5f5-0e23f8686dc4\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.728278 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/aa557411-6b0a-4596-a5f5-0e23f8686dc4-config-data-custom\") pod \"cinder-scheduler-0\" (UID: \"aa557411-6b0a-4596-a5f5-0e23f8686dc4\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.733713 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-hfmkd\" (UniqueName: \"kubernetes.io/projected/aa557411-6b0a-4596-a5f5-0e23f8686dc4-kube-api-access-hfmkd\") pod \"cinder-scheduler-0\" (UID: \"aa557411-6b0a-4596-a5f5-0e23f8686dc4\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.823073 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.828899 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovsdbserver-nb-tls-certs\" (UniqueName: \"kubernetes.io/secret/29c7d055-4f37-45a4-a608-eb73a361a86c-ovsdbserver-nb-tls-certs\") pod \"ovsdbserver-nb-0\" (UID: \"29c7d055-4f37-45a4-a608-eb73a361a86c\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:46:19 crc kubenswrapper[5052]: E1203 12:46:19.829023 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-ovndbcluster-nb-ovndbs: secret "cert-ovndbcluster-nb-ovndbs" not found Dec 03 12:46:19 crc kubenswrapper[5052]: E1203 12:46:19.829066 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/29c7d055-4f37-45a4-a608-eb73a361a86c-ovsdbserver-nb-tls-certs podName:29c7d055-4f37-45a4-a608-eb73a361a86c nodeName:}" failed. No retries permitted until 2025-12-03 12:46:23.829052735 +0000 UTC m=+5174.417294499 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "ovsdbserver-nb-tls-certs" (UniqueName: "kubernetes.io/secret/29c7d055-4f37-45a4-a608-eb73a361a86c-ovsdbserver-nb-tls-certs") pod "ovsdbserver-nb-0" (UID: "29c7d055-4f37-45a4-a608-eb73a361a86c") : secret "cert-ovndbcluster-nb-ovndbs" not found Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.830315 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/4c9f4da8-f1ae-4815-8ee2-0e873fb07d62-public-tls-certs\") pod \"glance-default-external-api-0\" (UID: \"4c9f4da8-f1ae-4815-8ee2-0e873fb07d62\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:46:19 crc kubenswrapper[5052]: E1203 12:46:19.830525 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-glance-default-public-svc: secret "cert-glance-default-public-svc" not found Dec 03 12:46:19 crc kubenswrapper[5052]: E1203 12:46:19.831045 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4c9f4da8-f1ae-4815-8ee2-0e873fb07d62-public-tls-certs podName:4c9f4da8-f1ae-4815-8ee2-0e873fb07d62 nodeName:}" failed. No retries permitted until 2025-12-03 12:46:20.831031051 +0000 UTC m=+5171.419272875 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "public-tls-certs" (UniqueName: "kubernetes.io/secret/4c9f4da8-f1ae-4815-8ee2-0e873fb07d62-public-tls-certs") pod "glance-default-external-api-0" (UID: "4c9f4da8-f1ae-4815-8ee2-0e873fb07d62") : secret "cert-glance-default-public-svc" not found Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.837913 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/29c7d055-4f37-45a4-a608-eb73a361a86c-metrics-certs-tls-certs\") pod \"ovsdbserver-nb-0\" (UID: \"29c7d055-4f37-45a4-a608-eb73a361a86c\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.845626 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-keystone-listener-769bc98c4b-mnz6r" Dec 03 12:46:19 crc kubenswrapper[5052]: E1203 12:46:19.847361 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-ovn-metrics: secret "cert-ovn-metrics" not found Dec 03 12:46:19 crc kubenswrapper[5052]: E1203 12:46:19.847418 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/29c7d055-4f37-45a4-a608-eb73a361a86c-metrics-certs-tls-certs podName:29c7d055-4f37-45a4-a608-eb73a361a86c nodeName:}" failed. No retries permitted until 2025-12-03 12:46:23.84740005 +0000 UTC m=+5174.435641814 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs-tls-certs" (UniqueName: "kubernetes.io/secret/29c7d055-4f37-45a4-a608-eb73a361a86c-metrics-certs-tls-certs") pod "ovsdbserver-nb-0" (UID: "29c7d055-4f37-45a4-a608-eb73a361a86c") : secret "cert-ovn-metrics" not found Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.849752 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2c53ea4e-56ba-4ba1-8cb6-8c24725b5856" path="/var/lib/kubelet/pods/2c53ea4e-56ba-4ba1-8cb6-8c24725b5856/volumes" Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.850419 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2f966eeb-76ef-4771-acdf-e0de2bb463a4" path="/var/lib/kubelet/pods/2f966eeb-76ef-4771-acdf-e0de2bb463a4/volumes" Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.850930 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3ae64378-aaa4-46bc-8c48-e1d90e490016" path="/var/lib/kubelet/pods/3ae64378-aaa4-46bc-8c48-e1d90e490016/volumes" Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.852153 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="68a80c17-9f7d-47de-b02c-5e7bb47656ab" path="/var/lib/kubelet/pods/68a80c17-9f7d-47de-b02c-5e7bb47656ab/volumes" Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.852857 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="78361066-6add-4971-9a4b-9157f35ad494" path="/var/lib/kubelet/pods/78361066-6add-4971-9a4b-9157f35ad494/volumes" Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.854312 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8fa85200-471e-4fba-9884-ef25328ebf64" path="/var/lib/kubelet/pods/8fa85200-471e-4fba-9884-ef25328ebf64/volumes" Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.854944 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9aea20a7-eaed-43a8-9b52-08c2c0212230" path="/var/lib/kubelet/pods/9aea20a7-eaed-43a8-9b52-08c2c0212230/volumes" Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.855630 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9af09a5b-789c-4b83-b899-9d1b3f0d8603" path="/var/lib/kubelet/pods/9af09a5b-789c-4b83-b899-9d1b3f0d8603/volumes" Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.861268 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dfda1fd2-f923-47bd-bf43-8a1be3f79f09" path="/var/lib/kubelet/pods/dfda1fd2-f923-47bd-bf43-8a1be3f79f09/volumes" Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.861930 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f7e26e55-1426-4cd4-b169-a63868acdfaa" path="/var/lib/kubelet/pods/f7e26e55-1426-4cd4-b169-a63868acdfaa/volumes" Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.862465 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fb6c11ac-1b0e-4e63-8988-256b2504ebc7" path="/var/lib/kubelet/pods/fb6c11ac-1b0e-4e63-8988-256b2504ebc7/volumes" Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.947144 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9zt7x\" (UniqueName: \"kubernetes.io/projected/1d187c5a-7745-447f-a8e2-06c7528e5be0-kube-api-access-9zt7x\") pod \"1d187c5a-7745-447f-a8e2-06c7528e5be0\" (UID: \"1d187c5a-7745-447f-a8e2-06c7528e5be0\") " Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.947216 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/1d187c5a-7745-447f-a8e2-06c7528e5be0-config-data-custom\") pod \"1d187c5a-7745-447f-a8e2-06c7528e5be0\" (UID: \"1d187c5a-7745-447f-a8e2-06c7528e5be0\") " Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.947317 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1d187c5a-7745-447f-a8e2-06c7528e5be0-combined-ca-bundle\") pod \"1d187c5a-7745-447f-a8e2-06c7528e5be0\" (UID: \"1d187c5a-7745-447f-a8e2-06c7528e5be0\") " Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.947433 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/1d187c5a-7745-447f-a8e2-06c7528e5be0-logs\") pod \"1d187c5a-7745-447f-a8e2-06c7528e5be0\" (UID: \"1d187c5a-7745-447f-a8e2-06c7528e5be0\") " Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.947501 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/1d187c5a-7745-447f-a8e2-06c7528e5be0-config-data\") pod \"1d187c5a-7745-447f-a8e2-06c7528e5be0\" (UID: \"1d187c5a-7745-447f-a8e2-06c7528e5be0\") " Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.947835 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/818947c1-95f1-4235-8846-5886abeb57a1-galera-tls-certs\") pod \"openstack-galera-0\" (UID: \"818947c1-95f1-4235-8846-5886abeb57a1\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.947984 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/bf8b9b9c-b8f9-4c93-8112-43913950fcdf-internal-tls-certs\") pod \"glance-default-internal-api-0\" (UID: \"bf8b9b9c-b8f9-4c93-8112-43913950fcdf\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.948059 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/c069aee2-4cb6-4c8a-bbe5-949ab0b48331-public-tls-certs\") pod \"barbican-api-bcf49b784-jnkdb\" (UID: \"c069aee2-4cb6-4c8a-bbe5-949ab0b48331\") " pod="openstack-kuttl-tests/barbican-api-bcf49b784-jnkdb" Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.948180 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"memcached-tls-certs\" (UniqueName: \"kubernetes.io/secret/4d42b068-9b91-4df6-9a9c-81cad009be16-memcached-tls-certs\") pod \"memcached-0\" (UID: \"4d42b068-9b91-4df6-9a9c-81cad009be16\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.948207 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/c069aee2-4cb6-4c8a-bbe5-949ab0b48331-internal-tls-certs\") pod \"barbican-api-bcf49b784-jnkdb\" (UID: \"c069aee2-4cb6-4c8a-bbe5-949ab0b48331\") " pod="openstack-kuttl-tests/barbican-api-bcf49b784-jnkdb" Dec 03 12:46:19 crc kubenswrapper[5052]: E1203 12:46:19.948401 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-barbican-internal-svc: secret "cert-barbican-internal-svc" not found Dec 03 12:46:19 crc kubenswrapper[5052]: E1203 12:46:19.948455 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c069aee2-4cb6-4c8a-bbe5-949ab0b48331-internal-tls-certs podName:c069aee2-4cb6-4c8a-bbe5-949ab0b48331 nodeName:}" failed. No retries permitted until 2025-12-03 12:46:27.948438419 +0000 UTC m=+5178.536680183 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "internal-tls-certs" (UniqueName: "kubernetes.io/secret/c069aee2-4cb6-4c8a-bbe5-949ab0b48331-internal-tls-certs") pod "barbican-api-bcf49b784-jnkdb" (UID: "c069aee2-4cb6-4c8a-bbe5-949ab0b48331") : secret "cert-barbican-internal-svc" not found Dec 03 12:46:19 crc kubenswrapper[5052]: E1203 12:46:19.948867 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-glance-default-internal-svc: secret "cert-glance-default-internal-svc" not found Dec 03 12:46:19 crc kubenswrapper[5052]: E1203 12:46:19.948901 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bf8b9b9c-b8f9-4c93-8112-43913950fcdf-internal-tls-certs podName:bf8b9b9c-b8f9-4c93-8112-43913950fcdf nodeName:}" failed. No retries permitted until 2025-12-03 12:46:20.948891422 +0000 UTC m=+5171.537133186 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "internal-tls-certs" (UniqueName: "kubernetes.io/secret/bf8b9b9c-b8f9-4c93-8112-43913950fcdf-internal-tls-certs") pod "glance-default-internal-api-0" (UID: "bf8b9b9c-b8f9-4c93-8112-43913950fcdf") : secret "cert-glance-default-internal-svc" not found Dec 03 12:46:19 crc kubenswrapper[5052]: E1203 12:46:19.948947 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-barbican-public-svc: secret "cert-barbican-public-svc" not found Dec 03 12:46:19 crc kubenswrapper[5052]: E1203 12:46:19.948973 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c069aee2-4cb6-4c8a-bbe5-949ab0b48331-public-tls-certs podName:c069aee2-4cb6-4c8a-bbe5-949ab0b48331 nodeName:}" failed. No retries permitted until 2025-12-03 12:46:27.948964824 +0000 UTC m=+5178.537206588 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "public-tls-certs" (UniqueName: "kubernetes.io/secret/c069aee2-4cb6-4c8a-bbe5-949ab0b48331-public-tls-certs") pod "barbican-api-bcf49b784-jnkdb" (UID: "c069aee2-4cb6-4c8a-bbe5-949ab0b48331") : secret "cert-barbican-public-svc" not found Dec 03 12:46:19 crc kubenswrapper[5052]: E1203 12:46:19.949131 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-galera-openstack-svc: secret "cert-galera-openstack-svc" not found Dec 03 12:46:19 crc kubenswrapper[5052]: E1203 12:46:19.949196 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/818947c1-95f1-4235-8846-5886abeb57a1-galera-tls-certs podName:818947c1-95f1-4235-8846-5886abeb57a1 nodeName:}" failed. No retries permitted until 2025-12-03 12:46:21.94917396 +0000 UTC m=+5172.537415804 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "galera-tls-certs" (UniqueName: "kubernetes.io/secret/818947c1-95f1-4235-8846-5886abeb57a1-galera-tls-certs") pod "openstack-galera-0" (UID: "818947c1-95f1-4235-8846-5886abeb57a1") : secret "cert-galera-openstack-svc" not found Dec 03 12:46:19 crc kubenswrapper[5052]: E1203 12:46:19.949257 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-memcached-svc: secret "cert-memcached-svc" not found Dec 03 12:46:19 crc kubenswrapper[5052]: E1203 12:46:19.949287 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4d42b068-9b91-4df6-9a9c-81cad009be16-memcached-tls-certs podName:4d42b068-9b91-4df6-9a9c-81cad009be16 nodeName:}" failed. No retries permitted until 2025-12-03 12:46:21.949277813 +0000 UTC m=+5172.537519677 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "memcached-tls-certs" (UniqueName: "kubernetes.io/secret/4d42b068-9b91-4df6-9a9c-81cad009be16-memcached-tls-certs") pod "memcached-0" (UID: "4d42b068-9b91-4df6-9a9c-81cad009be16") : secret "cert-memcached-svc" not found Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.950025 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1d187c5a-7745-447f-a8e2-06c7528e5be0-logs" (OuterVolumeSpecName: "logs") pod "1d187c5a-7745-447f-a8e2-06c7528e5be0" (UID: "1d187c5a-7745-447f-a8e2-06c7528e5be0"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.957057 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1d187c5a-7745-447f-a8e2-06c7528e5be0-kube-api-access-9zt7x" (OuterVolumeSpecName: "kube-api-access-9zt7x") pod "1d187c5a-7745-447f-a8e2-06c7528e5be0" (UID: "1d187c5a-7745-447f-a8e2-06c7528e5be0"). InnerVolumeSpecName "kube-api-access-9zt7x". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:46:19 crc kubenswrapper[5052]: I1203 12:46:19.960727 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1d187c5a-7745-447f-a8e2-06c7528e5be0-config-data-custom" (OuterVolumeSpecName: "config-data-custom") pod "1d187c5a-7745-447f-a8e2-06c7528e5be0" (UID: "1d187c5a-7745-447f-a8e2-06c7528e5be0"). InnerVolumeSpecName "config-data-custom". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:46:20 crc kubenswrapper[5052]: I1203 12:46:20.000879 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1d187c5a-7745-447f-a8e2-06c7528e5be0-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "1d187c5a-7745-447f-a8e2-06c7528e5be0" (UID: "1d187c5a-7745-447f-a8e2-06c7528e5be0"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:46:20 crc kubenswrapper[5052]: I1203 12:46:20.050974 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/234278ab-b4a8-4fdf-abf9-c67cdbff1ac0-public-tls-certs\") pod \"cinder-api-0\" (UID: \"234278ab-b4a8-4fdf-abf9-c67cdbff1ac0\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:46:20 crc kubenswrapper[5052]: I1203 12:46:20.051228 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/234278ab-b4a8-4fdf-abf9-c67cdbff1ac0-internal-tls-certs\") pod \"cinder-api-0\" (UID: \"234278ab-b4a8-4fdf-abf9-c67cdbff1ac0\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:46:20 crc kubenswrapper[5052]: I1203 12:46:20.055111 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/1d187c5a-7745-447f-a8e2-06c7528e5be0-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:20 crc kubenswrapper[5052]: I1203 12:46:20.055138 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-9zt7x\" (UniqueName: \"kubernetes.io/projected/1d187c5a-7745-447f-a8e2-06c7528e5be0-kube-api-access-9zt7x\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:20 crc kubenswrapper[5052]: I1203 12:46:20.055157 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/1d187c5a-7745-447f-a8e2-06c7528e5be0-config-data-custom\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:20 crc kubenswrapper[5052]: I1203 12:46:20.055173 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1d187c5a-7745-447f-a8e2-06c7528e5be0-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:20 crc kubenswrapper[5052]: E1203 12:46:20.051992 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-cinder-public-svc: secret "cert-cinder-public-svc" not found Dec 03 12:46:20 crc kubenswrapper[5052]: E1203 12:46:20.055253 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/234278ab-b4a8-4fdf-abf9-c67cdbff1ac0-public-tls-certs podName:234278ab-b4a8-4fdf-abf9-c67cdbff1ac0 nodeName:}" failed. No retries permitted until 2025-12-03 12:46:21.055231823 +0000 UTC m=+5171.643473597 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "public-tls-certs" (UniqueName: "kubernetes.io/secret/234278ab-b4a8-4fdf-abf9-c67cdbff1ac0-public-tls-certs") pod "cinder-api-0" (UID: "234278ab-b4a8-4fdf-abf9-c67cdbff1ac0") : secret "cert-cinder-public-svc" not found Dec 03 12:46:20 crc kubenswrapper[5052]: E1203 12:46:20.053457 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-cinder-internal-svc: secret "cert-cinder-internal-svc" not found Dec 03 12:46:20 crc kubenswrapper[5052]: E1203 12:46:20.055469 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/234278ab-b4a8-4fdf-abf9-c67cdbff1ac0-internal-tls-certs podName:234278ab-b4a8-4fdf-abf9-c67cdbff1ac0 nodeName:}" failed. No retries permitted until 2025-12-03 12:46:21.055443609 +0000 UTC m=+5171.643685423 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "internal-tls-certs" (UniqueName: "kubernetes.io/secret/234278ab-b4a8-4fdf-abf9-c67cdbff1ac0-internal-tls-certs") pod "cinder-api-0" (UID: "234278ab-b4a8-4fdf-abf9-c67cdbff1ac0") : secret "cert-cinder-internal-svc" not found Dec 03 12:46:20 crc kubenswrapper[5052]: I1203 12:46:20.121132 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1d187c5a-7745-447f-a8e2-06c7528e5be0-config-data" (OuterVolumeSpecName: "config-data") pod "1d187c5a-7745-447f-a8e2-06c7528e5be0" (UID: "1d187c5a-7745-447f-a8e2-06c7528e5be0"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:46:20 crc kubenswrapper[5052]: I1203 12:46:20.127353 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-conductor-0" event={"ID":"6d653682-52e2-4940-af66-633475bfe09a","Type":"ContainerStarted","Data":"6f8b18b5b30d4270384e6892b05ad3c64ed08fdfb147a6e3286cf16683c5b21e"} Dec 03 12:46:20 crc kubenswrapper[5052]: I1203 12:46:20.127392 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-conductor-0" event={"ID":"6d653682-52e2-4940-af66-633475bfe09a","Type":"ContainerStarted","Data":"1c60e1611f47b232078a4f996ef3cb8b4bd103c8fd84abb48567e8f4e445ef6f"} Dec 03 12:46:20 crc kubenswrapper[5052]: I1203 12:46:20.127647 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:46:20 crc kubenswrapper[5052]: I1203 12:46:20.141273 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"81709983-1fd0-4446-bbbf-fb94653bb3d2","Type":"ContainerStarted","Data":"26ed2e6897ff71a51086a7cc455488c21ad8adffd038be8de42d13bd1335a6fd"} Dec 03 12:46:20 crc kubenswrapper[5052]: I1203 12:46:20.141319 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"81709983-1fd0-4446-bbbf-fb94653bb3d2","Type":"ContainerStarted","Data":"2cbcded0899056536b72f9bbc9bb72c1f3ce02cd23f2749dd1c011532129b01c"} Dec 03 12:46:20 crc kubenswrapper[5052]: I1203 12:46:20.157640 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/fa1c223d-82fa-488c-ac48-e88bce36a839-internal-tls-certs\") pod \"keystone-687ff6fcfc-9vwb2\" (UID: \"fa1c223d-82fa-488c-ac48-e88bce36a839\") " pod="openstack-kuttl-tests/keystone-687ff6fcfc-9vwb2" Dec 03 12:46:20 crc kubenswrapper[5052]: I1203 12:46:20.157823 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/576b2af7-7647-4a45-9423-3aa752d84e75-galera-tls-certs\") pod \"openstack-cell1-galera-0\" (UID: \"576b2af7-7647-4a45-9423-3aa752d84e75\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:46:20 crc kubenswrapper[5052]: I1203 12:46:20.158543 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/fa1c223d-82fa-488c-ac48-e88bce36a839-public-tls-certs\") pod \"keystone-687ff6fcfc-9vwb2\" (UID: \"fa1c223d-82fa-488c-ac48-e88bce36a839\") " pod="openstack-kuttl-tests/keystone-687ff6fcfc-9vwb2" Dec 03 12:46:20 crc kubenswrapper[5052]: E1203 12:46:20.158631 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-keystone-internal-svc: secret "cert-keystone-internal-svc" not found Dec 03 12:46:20 crc kubenswrapper[5052]: E1203 12:46:20.158698 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fa1c223d-82fa-488c-ac48-e88bce36a839-internal-tls-certs podName:fa1c223d-82fa-488c-ac48-e88bce36a839 nodeName:}" failed. No retries permitted until 2025-12-03 12:46:28.158679742 +0000 UTC m=+5178.746921506 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "internal-tls-certs" (UniqueName: "kubernetes.io/secret/fa1c223d-82fa-488c-ac48-e88bce36a839-internal-tls-certs") pod "keystone-687ff6fcfc-9vwb2" (UID: "fa1c223d-82fa-488c-ac48-e88bce36a839") : secret "cert-keystone-internal-svc" not found Dec 03 12:46:20 crc kubenswrapper[5052]: E1203 12:46:20.158728 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-keystone-public-svc: secret "cert-keystone-public-svc" not found Dec 03 12:46:20 crc kubenswrapper[5052]: E1203 12:46:20.158767 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fa1c223d-82fa-488c-ac48-e88bce36a839-public-tls-certs podName:fa1c223d-82fa-488c-ac48-e88bce36a839 nodeName:}" failed. No retries permitted until 2025-12-03 12:46:28.158756024 +0000 UTC m=+5178.746997788 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "public-tls-certs" (UniqueName: "kubernetes.io/secret/fa1c223d-82fa-488c-ac48-e88bce36a839-public-tls-certs") pod "keystone-687ff6fcfc-9vwb2" (UID: "fa1c223d-82fa-488c-ac48-e88bce36a839") : secret "cert-keystone-public-svc" not found Dec 03 12:46:20 crc kubenswrapper[5052]: I1203 12:46:20.158918 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/1d187c5a-7745-447f-a8e2-06c7528e5be0-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:20 crc kubenswrapper[5052]: E1203 12:46:20.159256 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-galera-openstack-cell1-svc: secret "cert-galera-openstack-cell1-svc" not found Dec 03 12:46:20 crc kubenswrapper[5052]: E1203 12:46:20.159317 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/576b2af7-7647-4a45-9423-3aa752d84e75-galera-tls-certs podName:576b2af7-7647-4a45-9423-3aa752d84e75 nodeName:}" failed. No retries permitted until 2025-12-03 12:46:22.159300869 +0000 UTC m=+5172.747542633 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "galera-tls-certs" (UniqueName: "kubernetes.io/secret/576b2af7-7647-4a45-9423-3aa752d84e75-galera-tls-certs") pod "openstack-cell1-galera-0" (UID: "576b2af7-7647-4a45-9423-3aa752d84e75") : secret "cert-galera-openstack-cell1-svc" not found Dec 03 12:46:20 crc kubenswrapper[5052]: I1203 12:46:20.159421 5052 generic.go:334] "Generic (PLEG): container finished" podID="1d187c5a-7745-447f-a8e2-06c7528e5be0" containerID="0b52d2ae1f61eeac5291c66875e41de2e8c84d29873caba7e0a0412afcac4845" exitCode=0 Dec 03 12:46:20 crc kubenswrapper[5052]: I1203 12:46:20.159572 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-keystone-listener-769bc98c4b-mnz6r" event={"ID":"1d187c5a-7745-447f-a8e2-06c7528e5be0","Type":"ContainerDied","Data":"0b52d2ae1f61eeac5291c66875e41de2e8c84d29873caba7e0a0412afcac4845"} Dec 03 12:46:20 crc kubenswrapper[5052]: I1203 12:46:20.159609 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-keystone-listener-769bc98c4b-mnz6r" event={"ID":"1d187c5a-7745-447f-a8e2-06c7528e5be0","Type":"ContainerDied","Data":"feac048847ae2836573d829abc28eeb03bc068950854ca5d2ef02e61ea8acdbb"} Dec 03 12:46:20 crc kubenswrapper[5052]: I1203 12:46:20.159632 5052 scope.go:117] "RemoveContainer" containerID="0b52d2ae1f61eeac5291c66875e41de2e8c84d29873caba7e0a0412afcac4845" Dec 03 12:46:20 crc kubenswrapper[5052]: I1203 12:46:20.159760 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-keystone-listener-769bc98c4b-mnz6r" Dec 03 12:46:20 crc kubenswrapper[5052]: I1203 12:46:20.162310 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:46:20 crc kubenswrapper[5052]: I1203 12:46:20.165906 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-cell0-conductor-0" podStartSLOduration=3.165881468 podStartE2EDuration="3.165881468s" podCreationTimestamp="2025-12-03 12:46:17 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:46:20.155180662 +0000 UTC m=+5170.743422436" watchObservedRunningTime="2025-12-03 12:46:20.165881468 +0000 UTC m=+5170.754123232" Dec 03 12:46:20 crc kubenswrapper[5052]: I1203 12:46:20.175295 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-worker-65fcd76cf7-xhs44" Dec 03 12:46:20 crc kubenswrapper[5052]: I1203 12:46:20.175877 5052 generic.go:334] "Generic (PLEG): container finished" podID="92290cff-ed52-4a47-80e9-720bfd9f3dd8" containerID="f1a23446d7321c40e536bf2e9b6174acf23d15d1d963daca95ff134b7d58cd46" exitCode=0 Dec 03 12:46:20 crc kubenswrapper[5052]: I1203 12:46:20.176134 5052 generic.go:334] "Generic (PLEG): container finished" podID="92290cff-ed52-4a47-80e9-720bfd9f3dd8" containerID="6da49bc767ca3706e94a6d9f283174f99fca40c8d0d7ff4717a94a22b54ce36c" exitCode=143 Dec 03 12:46:20 crc kubenswrapper[5052]: I1203 12:46:20.176588 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-worker-65fcd76cf7-xhs44" event={"ID":"92290cff-ed52-4a47-80e9-720bfd9f3dd8","Type":"ContainerDied","Data":"f1a23446d7321c40e536bf2e9b6174acf23d15d1d963daca95ff134b7d58cd46"} Dec 03 12:46:20 crc kubenswrapper[5052]: I1203 12:46:20.176626 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-worker-65fcd76cf7-xhs44" event={"ID":"92290cff-ed52-4a47-80e9-720bfd9f3dd8","Type":"ContainerDied","Data":"6da49bc767ca3706e94a6d9f283174f99fca40c8d0d7ff4717a94a22b54ce36c"} Dec 03 12:46:20 crc kubenswrapper[5052]: I1203 12:46:20.195575 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-scheduler-0" podStartSLOduration=3.195554876 podStartE2EDuration="3.195554876s" podCreationTimestamp="2025-12-03 12:46:17 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:46:20.172815046 +0000 UTC m=+5170.761056820" watchObservedRunningTime="2025-12-03 12:46:20.195554876 +0000 UTC m=+5170.783796640" Dec 03 12:46:20 crc kubenswrapper[5052]: I1203 12:46:20.252269 5052 scope.go:117] "RemoveContainer" containerID="4208df0572c14c762c41a0bceeeb54f12d57e83a373e6e10caba8b5459e15ce7" Dec 03 12:46:20 crc kubenswrapper[5052]: I1203 12:46:20.255524 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-keystone-listener-769bc98c4b-mnz6r"] Dec 03 12:46:20 crc kubenswrapper[5052]: I1203 12:46:20.269331 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/barbican-keystone-listener-769bc98c4b-mnz6r"] Dec 03 12:46:20 crc kubenswrapper[5052]: I1203 12:46:20.311681 5052 scope.go:117] "RemoveContainer" containerID="0b52d2ae1f61eeac5291c66875e41de2e8c84d29873caba7e0a0412afcac4845" Dec 03 12:46:20 crc kubenswrapper[5052]: E1203 12:46:20.312372 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0b52d2ae1f61eeac5291c66875e41de2e8c84d29873caba7e0a0412afcac4845\": container with ID starting with 0b52d2ae1f61eeac5291c66875e41de2e8c84d29873caba7e0a0412afcac4845 not found: ID does not exist" containerID="0b52d2ae1f61eeac5291c66875e41de2e8c84d29873caba7e0a0412afcac4845" Dec 03 12:46:20 crc kubenswrapper[5052]: I1203 12:46:20.312416 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0b52d2ae1f61eeac5291c66875e41de2e8c84d29873caba7e0a0412afcac4845"} err="failed to get container status \"0b52d2ae1f61eeac5291c66875e41de2e8c84d29873caba7e0a0412afcac4845\": rpc error: code = NotFound desc = could not find container \"0b52d2ae1f61eeac5291c66875e41de2e8c84d29873caba7e0a0412afcac4845\": container with ID starting with 0b52d2ae1f61eeac5291c66875e41de2e8c84d29873caba7e0a0412afcac4845 not found: ID does not exist" Dec 03 12:46:20 crc kubenswrapper[5052]: I1203 12:46:20.312441 5052 scope.go:117] "RemoveContainer" containerID="4208df0572c14c762c41a0bceeeb54f12d57e83a373e6e10caba8b5459e15ce7" Dec 03 12:46:20 crc kubenswrapper[5052]: E1203 12:46:20.312792 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4208df0572c14c762c41a0bceeeb54f12d57e83a373e6e10caba8b5459e15ce7\": container with ID starting with 4208df0572c14c762c41a0bceeeb54f12d57e83a373e6e10caba8b5459e15ce7 not found: ID does not exist" containerID="4208df0572c14c762c41a0bceeeb54f12d57e83a373e6e10caba8b5459e15ce7" Dec 03 12:46:20 crc kubenswrapper[5052]: I1203 12:46:20.312816 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4208df0572c14c762c41a0bceeeb54f12d57e83a373e6e10caba8b5459e15ce7"} err="failed to get container status \"4208df0572c14c762c41a0bceeeb54f12d57e83a373e6e10caba8b5459e15ce7\": rpc error: code = NotFound desc = could not find container \"4208df0572c14c762c41a0bceeeb54f12d57e83a373e6e10caba8b5459e15ce7\": container with ID starting with 4208df0572c14c762c41a0bceeeb54f12d57e83a373e6e10caba8b5459e15ce7 not found: ID does not exist" Dec 03 12:46:20 crc kubenswrapper[5052]: I1203 12:46:20.312833 5052 scope.go:117] "RemoveContainer" containerID="f1a23446d7321c40e536bf2e9b6174acf23d15d1d963daca95ff134b7d58cd46" Dec 03 12:46:20 crc kubenswrapper[5052]: I1203 12:46:20.351699 5052 scope.go:117] "RemoveContainer" containerID="6da49bc767ca3706e94a6d9f283174f99fca40c8d0d7ff4717a94a22b54ce36c" Dec 03 12:46:20 crc kubenswrapper[5052]: I1203 12:46:20.366309 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/92290cff-ed52-4a47-80e9-720bfd9f3dd8-config-data\") pod \"92290cff-ed52-4a47-80e9-720bfd9f3dd8\" (UID: \"92290cff-ed52-4a47-80e9-720bfd9f3dd8\") " Dec 03 12:46:20 crc kubenswrapper[5052]: I1203 12:46:20.366406 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/92290cff-ed52-4a47-80e9-720bfd9f3dd8-combined-ca-bundle\") pod \"92290cff-ed52-4a47-80e9-720bfd9f3dd8\" (UID: \"92290cff-ed52-4a47-80e9-720bfd9f3dd8\") " Dec 03 12:46:20 crc kubenswrapper[5052]: I1203 12:46:20.366467 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mwn82\" (UniqueName: \"kubernetes.io/projected/92290cff-ed52-4a47-80e9-720bfd9f3dd8-kube-api-access-mwn82\") pod \"92290cff-ed52-4a47-80e9-720bfd9f3dd8\" (UID: \"92290cff-ed52-4a47-80e9-720bfd9f3dd8\") " Dec 03 12:46:20 crc kubenswrapper[5052]: I1203 12:46:20.366548 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/92290cff-ed52-4a47-80e9-720bfd9f3dd8-logs\") pod \"92290cff-ed52-4a47-80e9-720bfd9f3dd8\" (UID: \"92290cff-ed52-4a47-80e9-720bfd9f3dd8\") " Dec 03 12:46:20 crc kubenswrapper[5052]: I1203 12:46:20.366592 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/92290cff-ed52-4a47-80e9-720bfd9f3dd8-config-data-custom\") pod \"92290cff-ed52-4a47-80e9-720bfd9f3dd8\" (UID: \"92290cff-ed52-4a47-80e9-720bfd9f3dd8\") " Dec 03 12:46:20 crc kubenswrapper[5052]: I1203 12:46:20.366937 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/901a2f0b-fb8e-4c97-aea7-567d7ee42a23-ovndb-tls-certs\") pod \"neutron-6d64d7894b-xp9d5\" (UID: \"901a2f0b-fb8e-4c97-aea7-567d7ee42a23\") " pod="openstack-kuttl-tests/neutron-6d64d7894b-xp9d5" Dec 03 12:46:20 crc kubenswrapper[5052]: I1203 12:46:20.367016 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/901a2f0b-fb8e-4c97-aea7-567d7ee42a23-internal-tls-certs\") pod \"neutron-6d64d7894b-xp9d5\" (UID: \"901a2f0b-fb8e-4c97-aea7-567d7ee42a23\") " pod="openstack-kuttl-tests/neutron-6d64d7894b-xp9d5" Dec 03 12:46:20 crc kubenswrapper[5052]: I1203 12:46:20.367240 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/901a2f0b-fb8e-4c97-aea7-567d7ee42a23-public-tls-certs\") pod \"neutron-6d64d7894b-xp9d5\" (UID: \"901a2f0b-fb8e-4c97-aea7-567d7ee42a23\") " pod="openstack-kuttl-tests/neutron-6d64d7894b-xp9d5" Dec 03 12:46:20 crc kubenswrapper[5052]: E1203 12:46:20.367418 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-neutron-public-svc: secret "cert-neutron-public-svc" not found Dec 03 12:46:20 crc kubenswrapper[5052]: E1203 12:46:20.367464 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/901a2f0b-fb8e-4c97-aea7-567d7ee42a23-public-tls-certs podName:901a2f0b-fb8e-4c97-aea7-567d7ee42a23 nodeName:}" failed. No retries permitted until 2025-12-03 12:46:28.367452092 +0000 UTC m=+5178.955693856 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "public-tls-certs" (UniqueName: "kubernetes.io/secret/901a2f0b-fb8e-4c97-aea7-567d7ee42a23-public-tls-certs") pod "neutron-6d64d7894b-xp9d5" (UID: "901a2f0b-fb8e-4c97-aea7-567d7ee42a23") : secret "cert-neutron-public-svc" not found Dec 03 12:46:20 crc kubenswrapper[5052]: E1203 12:46:20.367847 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-neutron-ovndbs: secret "cert-neutron-ovndbs" not found Dec 03 12:46:20 crc kubenswrapper[5052]: E1203 12:46:20.367875 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/901a2f0b-fb8e-4c97-aea7-567d7ee42a23-ovndb-tls-certs podName:901a2f0b-fb8e-4c97-aea7-567d7ee42a23 nodeName:}" failed. No retries permitted until 2025-12-03 12:46:28.367866074 +0000 UTC m=+5178.956107838 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "ovndb-tls-certs" (UniqueName: "kubernetes.io/secret/901a2f0b-fb8e-4c97-aea7-567d7ee42a23-ovndb-tls-certs") pod "neutron-6d64d7894b-xp9d5" (UID: "901a2f0b-fb8e-4c97-aea7-567d7ee42a23") : secret "cert-neutron-ovndbs" not found Dec 03 12:46:20 crc kubenswrapper[5052]: I1203 12:46:20.368506 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/92290cff-ed52-4a47-80e9-720bfd9f3dd8-logs" (OuterVolumeSpecName: "logs") pod "92290cff-ed52-4a47-80e9-720bfd9f3dd8" (UID: "92290cff-ed52-4a47-80e9-720bfd9f3dd8"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:46:20 crc kubenswrapper[5052]: E1203 12:46:20.368996 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-neutron-internal-svc: secret "cert-neutron-internal-svc" not found Dec 03 12:46:20 crc kubenswrapper[5052]: E1203 12:46:20.369073 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/901a2f0b-fb8e-4c97-aea7-567d7ee42a23-internal-tls-certs podName:901a2f0b-fb8e-4c97-aea7-567d7ee42a23 nodeName:}" failed. No retries permitted until 2025-12-03 12:46:28.369056808 +0000 UTC m=+5178.957298572 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "internal-tls-certs" (UniqueName: "kubernetes.io/secret/901a2f0b-fb8e-4c97-aea7-567d7ee42a23-internal-tls-certs") pod "neutron-6d64d7894b-xp9d5" (UID: "901a2f0b-fb8e-4c97-aea7-567d7ee42a23") : secret "cert-neutron-internal-svc" not found Dec 03 12:46:20 crc kubenswrapper[5052]: I1203 12:46:20.374009 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/92290cff-ed52-4a47-80e9-720bfd9f3dd8-kube-api-access-mwn82" (OuterVolumeSpecName: "kube-api-access-mwn82") pod "92290cff-ed52-4a47-80e9-720bfd9f3dd8" (UID: "92290cff-ed52-4a47-80e9-720bfd9f3dd8"). InnerVolumeSpecName "kube-api-access-mwn82". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:46:20 crc kubenswrapper[5052]: I1203 12:46:20.374720 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/92290cff-ed52-4a47-80e9-720bfd9f3dd8-config-data-custom" (OuterVolumeSpecName: "config-data-custom") pod "92290cff-ed52-4a47-80e9-720bfd9f3dd8" (UID: "92290cff-ed52-4a47-80e9-720bfd9f3dd8"). InnerVolumeSpecName "config-data-custom". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:46:20 crc kubenswrapper[5052]: I1203 12:46:20.395372 5052 scope.go:117] "RemoveContainer" containerID="f1a23446d7321c40e536bf2e9b6174acf23d15d1d963daca95ff134b7d58cd46" Dec 03 12:46:20 crc kubenswrapper[5052]: E1203 12:46:20.396887 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f1a23446d7321c40e536bf2e9b6174acf23d15d1d963daca95ff134b7d58cd46\": container with ID starting with f1a23446d7321c40e536bf2e9b6174acf23d15d1d963daca95ff134b7d58cd46 not found: ID does not exist" containerID="f1a23446d7321c40e536bf2e9b6174acf23d15d1d963daca95ff134b7d58cd46" Dec 03 12:46:20 crc kubenswrapper[5052]: I1203 12:46:20.396913 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f1a23446d7321c40e536bf2e9b6174acf23d15d1d963daca95ff134b7d58cd46"} err="failed to get container status \"f1a23446d7321c40e536bf2e9b6174acf23d15d1d963daca95ff134b7d58cd46\": rpc error: code = NotFound desc = could not find container \"f1a23446d7321c40e536bf2e9b6174acf23d15d1d963daca95ff134b7d58cd46\": container with ID starting with f1a23446d7321c40e536bf2e9b6174acf23d15d1d963daca95ff134b7d58cd46 not found: ID does not exist" Dec 03 12:46:20 crc kubenswrapper[5052]: I1203 12:46:20.396935 5052 scope.go:117] "RemoveContainer" containerID="6da49bc767ca3706e94a6d9f283174f99fca40c8d0d7ff4717a94a22b54ce36c" Dec 03 12:46:20 crc kubenswrapper[5052]: E1203 12:46:20.403639 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6da49bc767ca3706e94a6d9f283174f99fca40c8d0d7ff4717a94a22b54ce36c\": container with ID starting with 6da49bc767ca3706e94a6d9f283174f99fca40c8d0d7ff4717a94a22b54ce36c not found: ID does not exist" containerID="6da49bc767ca3706e94a6d9f283174f99fca40c8d0d7ff4717a94a22b54ce36c" Dec 03 12:46:20 crc kubenswrapper[5052]: I1203 12:46:20.403679 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6da49bc767ca3706e94a6d9f283174f99fca40c8d0d7ff4717a94a22b54ce36c"} err="failed to get container status \"6da49bc767ca3706e94a6d9f283174f99fca40c8d0d7ff4717a94a22b54ce36c\": rpc error: code = NotFound desc = could not find container \"6da49bc767ca3706e94a6d9f283174f99fca40c8d0d7ff4717a94a22b54ce36c\": container with ID starting with 6da49bc767ca3706e94a6d9f283174f99fca40c8d0d7ff4717a94a22b54ce36c not found: ID does not exist" Dec 03 12:46:20 crc kubenswrapper[5052]: I1203 12:46:20.403756 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/92290cff-ed52-4a47-80e9-720bfd9f3dd8-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "92290cff-ed52-4a47-80e9-720bfd9f3dd8" (UID: "92290cff-ed52-4a47-80e9-720bfd9f3dd8"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:46:20 crc kubenswrapper[5052]: I1203 12:46:20.441528 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/92290cff-ed52-4a47-80e9-720bfd9f3dd8-config-data" (OuterVolumeSpecName: "config-data") pod "92290cff-ed52-4a47-80e9-720bfd9f3dd8" (UID: "92290cff-ed52-4a47-80e9-720bfd9f3dd8"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:46:20 crc kubenswrapper[5052]: I1203 12:46:20.470235 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/92290cff-ed52-4a47-80e9-720bfd9f3dd8-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:20 crc kubenswrapper[5052]: I1203 12:46:20.470267 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/92290cff-ed52-4a47-80e9-720bfd9f3dd8-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:20 crc kubenswrapper[5052]: I1203 12:46:20.470284 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-mwn82\" (UniqueName: \"kubernetes.io/projected/92290cff-ed52-4a47-80e9-720bfd9f3dd8-kube-api-access-mwn82\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:20 crc kubenswrapper[5052]: I1203 12:46:20.470297 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/92290cff-ed52-4a47-80e9-720bfd9f3dd8-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:20 crc kubenswrapper[5052]: I1203 12:46:20.470312 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/92290cff-ed52-4a47-80e9-720bfd9f3dd8-config-data-custom\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:20 crc kubenswrapper[5052]: E1203 12:46:20.470387 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-swift-internal-svc: secret "cert-swift-internal-svc" not found Dec 03 12:46:20 crc kubenswrapper[5052]: E1203 12:46:20.470409 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-swift-public-svc: secret "cert-swift-public-svc" not found Dec 03 12:46:20 crc kubenswrapper[5052]: E1203 12:46:20.470466 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/242be027-6461-4aea-ae7c-e52ca996b2ea-public-tls-certs podName:242be027-6461-4aea-ae7c-e52ca996b2ea nodeName:}" failed. No retries permitted until 2025-12-03 12:46:22.470450418 +0000 UTC m=+5173.058692172 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "public-tls-certs" (UniqueName: "kubernetes.io/secret/242be027-6461-4aea-ae7c-e52ca996b2ea-public-tls-certs") pod "swift-proxy-dd4cf57c8-x4c45" (UID: "242be027-6461-4aea-ae7c-e52ca996b2ea") : secret "cert-swift-public-svc" not found Dec 03 12:46:20 crc kubenswrapper[5052]: E1203 12:46:20.470484 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/242be027-6461-4aea-ae7c-e52ca996b2ea-internal-tls-certs podName:242be027-6461-4aea-ae7c-e52ca996b2ea nodeName:}" failed. No retries permitted until 2025-12-03 12:46:22.470476439 +0000 UTC m=+5173.058718203 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "internal-tls-certs" (UniqueName: "kubernetes.io/secret/242be027-6461-4aea-ae7c-e52ca996b2ea-internal-tls-certs") pod "swift-proxy-dd4cf57c8-x4c45" (UID: "242be027-6461-4aea-ae7c-e52ca996b2ea") : secret "cert-swift-internal-svc" not found Dec 03 12:46:20 crc kubenswrapper[5052]: I1203 12:46:20.485012 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 12:46:20 crc kubenswrapper[5052]: I1203 12:46:20.573570 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/ed7f9920-a1e4-437b-8074-a30d0fe4450c-public-tls-certs\") pod \"nova-api-0\" (UID: \"ed7f9920-a1e4-437b-8074-a30d0fe4450c\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:46:20 crc kubenswrapper[5052]: I1203 12:46:20.573657 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/ed7f9920-a1e4-437b-8074-a30d0fe4450c-internal-tls-certs\") pod \"nova-api-0\" (UID: \"ed7f9920-a1e4-437b-8074-a30d0fe4450c\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:46:20 crc kubenswrapper[5052]: E1203 12:46:20.573688 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-nova-public-svc: secret "cert-nova-public-svc" not found Dec 03 12:46:20 crc kubenswrapper[5052]: E1203 12:46:20.573747 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ed7f9920-a1e4-437b-8074-a30d0fe4450c-public-tls-certs podName:ed7f9920-a1e4-437b-8074-a30d0fe4450c nodeName:}" failed. No retries permitted until 2025-12-03 12:46:22.573729101 +0000 UTC m=+5173.161970865 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "public-tls-certs" (UniqueName: "kubernetes.io/secret/ed7f9920-a1e4-437b-8074-a30d0fe4450c-public-tls-certs") pod "nova-api-0" (UID: "ed7f9920-a1e4-437b-8074-a30d0fe4450c") : secret "cert-nova-public-svc" not found Dec 03 12:46:20 crc kubenswrapper[5052]: E1203 12:46:20.573784 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-nova-internal-svc: secret "cert-nova-internal-svc" not found Dec 03 12:46:20 crc kubenswrapper[5052]: E1203 12:46:20.573810 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ed7f9920-a1e4-437b-8074-a30d0fe4450c-internal-tls-certs podName:ed7f9920-a1e4-437b-8074-a30d0fe4450c nodeName:}" failed. No retries permitted until 2025-12-03 12:46:22.573802344 +0000 UTC m=+5173.162044108 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "internal-tls-certs" (UniqueName: "kubernetes.io/secret/ed7f9920-a1e4-437b-8074-a30d0fe4450c-internal-tls-certs") pod "nova-api-0" (UID: "ed7f9920-a1e4-437b-8074-a30d0fe4450c") : secret "cert-nova-internal-svc" not found Dec 03 12:46:20 crc kubenswrapper[5052]: E1203 12:46:20.677558 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-kube-state-metrics-svc: secret "cert-kube-state-metrics-svc" not found Dec 03 12:46:20 crc kubenswrapper[5052]: E1203 12:46:20.677657 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/876d06e6-0187-4e3d-9183-ed2083fe5189-kube-state-metrics-tls-certs podName:876d06e6-0187-4e3d-9183-ed2083fe5189 nodeName:}" failed. No retries permitted until 2025-12-03 12:46:21.177643183 +0000 UTC m=+5171.765884937 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls-certs" (UniqueName: "kubernetes.io/secret/876d06e6-0187-4e3d-9183-ed2083fe5189-kube-state-metrics-tls-certs") pod "kube-state-metrics-0" (UID: "876d06e6-0187-4e3d-9183-ed2083fe5189") : secret "cert-kube-state-metrics-svc" not found Dec 03 12:46:20 crc kubenswrapper[5052]: I1203 12:46:20.705405 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/openstack-galera-0"] Dec 03 12:46:20 crc kubenswrapper[5052]: E1203 12:46:20.706232 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[galera-tls-certs], unattached volumes=[], failed to process volumes=[]: context canceled" pod="openstack-kuttl-tests/openstack-galera-0" podUID="818947c1-95f1-4235-8846-5886abeb57a1" Dec 03 12:46:20 crc kubenswrapper[5052]: E1203 12:46:20.779291 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-placement-internal-svc: secret "cert-placement-internal-svc" not found Dec 03 12:46:20 crc kubenswrapper[5052]: E1203 12:46:20.779309 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-placement-public-svc: secret "cert-placement-public-svc" not found Dec 03 12:46:20 crc kubenswrapper[5052]: E1203 12:46:20.779378 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/32f420c2-aa7e-44c8-a63a-7654aa3e6e34-internal-tls-certs podName:32f420c2-aa7e-44c8-a63a-7654aa3e6e34 nodeName:}" failed. No retries permitted until 2025-12-03 12:46:22.779360152 +0000 UTC m=+5173.367601916 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "internal-tls-certs" (UniqueName: "kubernetes.io/secret/32f420c2-aa7e-44c8-a63a-7654aa3e6e34-internal-tls-certs") pod "placement-bbcfbbc9d-5q6gx" (UID: "32f420c2-aa7e-44c8-a63a-7654aa3e6e34") : secret "cert-placement-internal-svc" not found Dec 03 12:46:20 crc kubenswrapper[5052]: E1203 12:46:20.779395 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/32f420c2-aa7e-44c8-a63a-7654aa3e6e34-public-tls-certs podName:32f420c2-aa7e-44c8-a63a-7654aa3e6e34 nodeName:}" failed. No retries permitted until 2025-12-03 12:46:22.779388853 +0000 UTC m=+5173.367630617 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "public-tls-certs" (UniqueName: "kubernetes.io/secret/32f420c2-aa7e-44c8-a63a-7654aa3e6e34-public-tls-certs") pod "placement-bbcfbbc9d-5q6gx" (UID: "32f420c2-aa7e-44c8-a63a-7654aa3e6e34") : secret "cert-placement-public-svc" not found Dec 03 12:46:20 crc kubenswrapper[5052]: I1203 12:46:20.881272 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/4c9f4da8-f1ae-4815-8ee2-0e873fb07d62-public-tls-certs\") pod \"glance-default-external-api-0\" (UID: \"4c9f4da8-f1ae-4815-8ee2-0e873fb07d62\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:46:20 crc kubenswrapper[5052]: E1203 12:46:20.881603 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-glance-default-public-svc: secret "cert-glance-default-public-svc" not found Dec 03 12:46:20 crc kubenswrapper[5052]: E1203 12:46:20.881976 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4c9f4da8-f1ae-4815-8ee2-0e873fb07d62-public-tls-certs podName:4c9f4da8-f1ae-4815-8ee2-0e873fb07d62 nodeName:}" failed. No retries permitted until 2025-12-03 12:46:22.881957306 +0000 UTC m=+5173.470199070 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "public-tls-certs" (UniqueName: "kubernetes.io/secret/4c9f4da8-f1ae-4815-8ee2-0e873fb07d62-public-tls-certs") pod "glance-default-external-api-0" (UID: "4c9f4da8-f1ae-4815-8ee2-0e873fb07d62") : secret "cert-glance-default-public-svc" not found Dec 03 12:46:20 crc kubenswrapper[5052]: E1203 12:46:20.896002 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="adc4f6d0e36c4370a4bff93d0b0803fec09e3b34230d31ebbe4f500396923a29" cmd=["/usr/local/bin/container-scripts/status_check.sh"] Dec 03 12:46:20 crc kubenswrapper[5052]: E1203 12:46:20.898303 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="adc4f6d0e36c4370a4bff93d0b0803fec09e3b34230d31ebbe4f500396923a29" cmd=["/usr/local/bin/container-scripts/status_check.sh"] Dec 03 12:46:20 crc kubenswrapper[5052]: E1203 12:46:20.900225 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="adc4f6d0e36c4370a4bff93d0b0803fec09e3b34230d31ebbe4f500396923a29" cmd=["/usr/local/bin/container-scripts/status_check.sh"] Dec 03 12:46:20 crc kubenswrapper[5052]: E1203 12:46:20.900298 5052 prober.go:104] "Probe errored" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" probeType="Readiness" pod="openstack-kuttl-tests/ovn-northd-0" podUID="6af0a301-2483-493d-bbc8-17138d79dfea" containerName="ovn-northd" Dec 03 12:46:20 crc kubenswrapper[5052]: I1203 12:46:20.983916 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/bf8b9b9c-b8f9-4c93-8112-43913950fcdf-internal-tls-certs\") pod \"glance-default-internal-api-0\" (UID: \"bf8b9b9c-b8f9-4c93-8112-43913950fcdf\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:46:20 crc kubenswrapper[5052]: E1203 12:46:20.984089 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-glance-default-internal-svc: secret "cert-glance-default-internal-svc" not found Dec 03 12:46:20 crc kubenswrapper[5052]: E1203 12:46:20.984145 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bf8b9b9c-b8f9-4c93-8112-43913950fcdf-internal-tls-certs podName:bf8b9b9c-b8f9-4c93-8112-43913950fcdf nodeName:}" failed. No retries permitted until 2025-12-03 12:46:22.984128988 +0000 UTC m=+5173.572370752 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "internal-tls-certs" (UniqueName: "kubernetes.io/secret/bf8b9b9c-b8f9-4c93-8112-43913950fcdf-internal-tls-certs") pod "glance-default-internal-api-0" (UID: "bf8b9b9c-b8f9-4c93-8112-43913950fcdf") : secret "cert-glance-default-internal-svc" not found Dec 03 12:46:21 crc kubenswrapper[5052]: I1203 12:46:21.085760 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/234278ab-b4a8-4fdf-abf9-c67cdbff1ac0-public-tls-certs\") pod \"cinder-api-0\" (UID: \"234278ab-b4a8-4fdf-abf9-c67cdbff1ac0\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:46:21 crc kubenswrapper[5052]: E1203 12:46:21.085959 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-cinder-public-svc: secret "cert-cinder-public-svc" not found Dec 03 12:46:21 crc kubenswrapper[5052]: I1203 12:46:21.085999 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/234278ab-b4a8-4fdf-abf9-c67cdbff1ac0-internal-tls-certs\") pod \"cinder-api-0\" (UID: \"234278ab-b4a8-4fdf-abf9-c67cdbff1ac0\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:46:21 crc kubenswrapper[5052]: E1203 12:46:21.086099 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/234278ab-b4a8-4fdf-abf9-c67cdbff1ac0-public-tls-certs podName:234278ab-b4a8-4fdf-abf9-c67cdbff1ac0 nodeName:}" failed. No retries permitted until 2025-12-03 12:46:23.086055893 +0000 UTC m=+5173.674297657 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "public-tls-certs" (UniqueName: "kubernetes.io/secret/234278ab-b4a8-4fdf-abf9-c67cdbff1ac0-public-tls-certs") pod "cinder-api-0" (UID: "234278ab-b4a8-4fdf-abf9-c67cdbff1ac0") : secret "cert-cinder-public-svc" not found Dec 03 12:46:21 crc kubenswrapper[5052]: E1203 12:46:21.086154 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-cinder-internal-svc: secret "cert-cinder-internal-svc" not found Dec 03 12:46:21 crc kubenswrapper[5052]: E1203 12:46:21.086220 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/234278ab-b4a8-4fdf-abf9-c67cdbff1ac0-internal-tls-certs podName:234278ab-b4a8-4fdf-abf9-c67cdbff1ac0 nodeName:}" failed. No retries permitted until 2025-12-03 12:46:23.086194827 +0000 UTC m=+5173.674436681 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "internal-tls-certs" (UniqueName: "kubernetes.io/secret/234278ab-b4a8-4fdf-abf9-c67cdbff1ac0-internal-tls-certs") pod "cinder-api-0" (UID: "234278ab-b4a8-4fdf-abf9-c67cdbff1ac0") : secret "cert-cinder-internal-svc" not found Dec 03 12:46:21 crc kubenswrapper[5052]: E1203 12:46:21.187736 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-kube-state-metrics-svc: secret "cert-kube-state-metrics-svc" not found Dec 03 12:46:21 crc kubenswrapper[5052]: E1203 12:46:21.187796 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/876d06e6-0187-4e3d-9183-ed2083fe5189-kube-state-metrics-tls-certs podName:876d06e6-0187-4e3d-9183-ed2083fe5189 nodeName:}" failed. No retries permitted until 2025-12-03 12:46:22.187783062 +0000 UTC m=+5172.776024826 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls-certs" (UniqueName: "kubernetes.io/secret/876d06e6-0187-4e3d-9183-ed2083fe5189-kube-state-metrics-tls-certs") pod "kube-state-metrics-0" (UID: "876d06e6-0187-4e3d-9183-ed2083fe5189") : secret "cert-kube-state-metrics-svc" not found Dec 03 12:46:21 crc kubenswrapper[5052]: I1203 12:46:21.219802 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"75266130-5854-4acd-acc9-bc22accc19bc","Type":"ContainerStarted","Data":"f494000fc74b8d824265c717da4442151f3a4e76a4efbba51ec6695872f15bb0"} Dec 03 12:46:21 crc kubenswrapper[5052]: I1203 12:46:21.219848 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"75266130-5854-4acd-acc9-bc22accc19bc","Type":"ContainerStarted","Data":"3e1d7a5d05726614ff20caaaa6d897aa3cbb28e2e859ab554810ad90e4b4a72c"} Dec 03 12:46:21 crc kubenswrapper[5052]: I1203 12:46:21.219858 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"75266130-5854-4acd-acc9-bc22accc19bc","Type":"ContainerStarted","Data":"0ef35cf8ef0a727c1b5075e4b54d1d87e062ce8c5d2c4f60eb3f0c1b8e2b6917"} Dec 03 12:46:21 crc kubenswrapper[5052]: I1203 12:46:21.232314 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"aa557411-6b0a-4596-a5f5-0e23f8686dc4","Type":"ContainerStarted","Data":"f9e0a759d0693033821fbf62611d284dded228a4c298c654482ae59247092610"} Dec 03 12:46:21 crc kubenswrapper[5052]: I1203 12:46:21.238927 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-worker-65fcd76cf7-xhs44" Dec 03 12:46:21 crc kubenswrapper[5052]: I1203 12:46:21.239308 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-worker-65fcd76cf7-xhs44" event={"ID":"92290cff-ed52-4a47-80e9-720bfd9f3dd8","Type":"ContainerDied","Data":"df24c1fdafb53577fadc51b0d23110c137497310f9a2ba7085895b957896da0a"} Dec 03 12:46:21 crc kubenswrapper[5052]: I1203 12:46:21.239590 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:46:21 crc kubenswrapper[5052]: I1203 12:46:21.273198 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-metadata-0" podStartSLOduration=3.273173084 podStartE2EDuration="3.273173084s" podCreationTimestamp="2025-12-03 12:46:18 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:46:21.250100035 +0000 UTC m=+5171.838341799" watchObservedRunningTime="2025-12-03 12:46:21.273173084 +0000 UTC m=+5171.861414848" Dec 03 12:46:21 crc kubenswrapper[5052]: I1203 12:46:21.349008 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:46:21 crc kubenswrapper[5052]: I1203 12:46:21.376136 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-worker-65fcd76cf7-xhs44"] Dec 03 12:46:21 crc kubenswrapper[5052]: I1203 12:46:21.389347 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/barbican-worker-65fcd76cf7-xhs44"] Dec 03 12:46:21 crc kubenswrapper[5052]: I1203 12:46:21.495739 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/818947c1-95f1-4235-8846-5886abeb57a1-config-data-default\") pod \"818947c1-95f1-4235-8846-5886abeb57a1\" (UID: \"818947c1-95f1-4235-8846-5886abeb57a1\") " Dec 03 12:46:21 crc kubenswrapper[5052]: I1203 12:46:21.495940 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/818947c1-95f1-4235-8846-5886abeb57a1-config-data-generated\") pod \"818947c1-95f1-4235-8846-5886abeb57a1\" (UID: \"818947c1-95f1-4235-8846-5886abeb57a1\") " Dec 03 12:46:21 crc kubenswrapper[5052]: I1203 12:46:21.495969 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"mysql-db\" (UniqueName: \"kubernetes.io/local-volume/local-storage07-crc\") pod \"818947c1-95f1-4235-8846-5886abeb57a1\" (UID: \"818947c1-95f1-4235-8846-5886abeb57a1\") " Dec 03 12:46:21 crc kubenswrapper[5052]: I1203 12:46:21.496000 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z2jcq\" (UniqueName: \"kubernetes.io/projected/818947c1-95f1-4235-8846-5886abeb57a1-kube-api-access-z2jcq\") pod \"818947c1-95f1-4235-8846-5886abeb57a1\" (UID: \"818947c1-95f1-4235-8846-5886abeb57a1\") " Dec 03 12:46:21 crc kubenswrapper[5052]: I1203 12:46:21.496028 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/818947c1-95f1-4235-8846-5886abeb57a1-kolla-config\") pod \"818947c1-95f1-4235-8846-5886abeb57a1\" (UID: \"818947c1-95f1-4235-8846-5886abeb57a1\") " Dec 03 12:46:21 crc kubenswrapper[5052]: I1203 12:46:21.496091 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/818947c1-95f1-4235-8846-5886abeb57a1-combined-ca-bundle\") pod \"818947c1-95f1-4235-8846-5886abeb57a1\" (UID: \"818947c1-95f1-4235-8846-5886abeb57a1\") " Dec 03 12:46:21 crc kubenswrapper[5052]: I1203 12:46:21.496126 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/818947c1-95f1-4235-8846-5886abeb57a1-operator-scripts\") pod \"818947c1-95f1-4235-8846-5886abeb57a1\" (UID: \"818947c1-95f1-4235-8846-5886abeb57a1\") " Dec 03 12:46:21 crc kubenswrapper[5052]: I1203 12:46:21.498814 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/818947c1-95f1-4235-8846-5886abeb57a1-kolla-config" (OuterVolumeSpecName: "kolla-config") pod "818947c1-95f1-4235-8846-5886abeb57a1" (UID: "818947c1-95f1-4235-8846-5886abeb57a1"). InnerVolumeSpecName "kolla-config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:46:21 crc kubenswrapper[5052]: I1203 12:46:21.500416 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/818947c1-95f1-4235-8846-5886abeb57a1-config-data-default" (OuterVolumeSpecName: "config-data-default") pod "818947c1-95f1-4235-8846-5886abeb57a1" (UID: "818947c1-95f1-4235-8846-5886abeb57a1"). InnerVolumeSpecName "config-data-default". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:46:21 crc kubenswrapper[5052]: I1203 12:46:21.501048 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/818947c1-95f1-4235-8846-5886abeb57a1-config-data-generated" (OuterVolumeSpecName: "config-data-generated") pod "818947c1-95f1-4235-8846-5886abeb57a1" (UID: "818947c1-95f1-4235-8846-5886abeb57a1"). InnerVolumeSpecName "config-data-generated". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:46:21 crc kubenswrapper[5052]: I1203 12:46:21.501475 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage07-crc" (OuterVolumeSpecName: "mysql-db") pod "818947c1-95f1-4235-8846-5886abeb57a1" (UID: "818947c1-95f1-4235-8846-5886abeb57a1"). InnerVolumeSpecName "local-storage07-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 12:46:21 crc kubenswrapper[5052]: I1203 12:46:21.501701 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/818947c1-95f1-4235-8846-5886abeb57a1-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "818947c1-95f1-4235-8846-5886abeb57a1" (UID: "818947c1-95f1-4235-8846-5886abeb57a1"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:46:21 crc kubenswrapper[5052]: I1203 12:46:21.502414 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/818947c1-95f1-4235-8846-5886abeb57a1-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "818947c1-95f1-4235-8846-5886abeb57a1" (UID: "818947c1-95f1-4235-8846-5886abeb57a1"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:46:21 crc kubenswrapper[5052]: I1203 12:46:21.503319 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/818947c1-95f1-4235-8846-5886abeb57a1-kube-api-access-z2jcq" (OuterVolumeSpecName: "kube-api-access-z2jcq") pod "818947c1-95f1-4235-8846-5886abeb57a1" (UID: "818947c1-95f1-4235-8846-5886abeb57a1"). InnerVolumeSpecName "kube-api-access-z2jcq". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:46:21 crc kubenswrapper[5052]: I1203 12:46:21.601704 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/818947c1-95f1-4235-8846-5886abeb57a1-config-data-generated\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:21 crc kubenswrapper[5052]: I1203 12:46:21.601770 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage07-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage07-crc\") on node \"crc\" " Dec 03 12:46:21 crc kubenswrapper[5052]: I1203 12:46:21.601787 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-z2jcq\" (UniqueName: \"kubernetes.io/projected/818947c1-95f1-4235-8846-5886abeb57a1-kube-api-access-z2jcq\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:21 crc kubenswrapper[5052]: I1203 12:46:21.601804 5052 reconciler_common.go:293] "Volume detached for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/818947c1-95f1-4235-8846-5886abeb57a1-kolla-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:21 crc kubenswrapper[5052]: I1203 12:46:21.601818 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/818947c1-95f1-4235-8846-5886abeb57a1-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:21 crc kubenswrapper[5052]: I1203 12:46:21.601831 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/818947c1-95f1-4235-8846-5886abeb57a1-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:21 crc kubenswrapper[5052]: I1203 12:46:21.601843 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/818947c1-95f1-4235-8846-5886abeb57a1-config-data-default\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:21 crc kubenswrapper[5052]: I1203 12:46:21.631211 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage07-crc" (UniqueName: "kubernetes.io/local-volume/local-storage07-crc") on node "crc" Dec 03 12:46:21 crc kubenswrapper[5052]: I1203 12:46:21.711637 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/9481ae66-152c-41e7-9cbc-90dcb5fa6171-ceilometer-tls-certs\") pod \"ceilometer-0\" (UID: \"9481ae66-152c-41e7-9cbc-90dcb5fa6171\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:46:21 crc kubenswrapper[5052]: I1203 12:46:21.714841 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage07-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage07-crc\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:21 crc kubenswrapper[5052]: I1203 12:46:21.716077 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1d187c5a-7745-447f-a8e2-06c7528e5be0" path="/var/lib/kubelet/pods/1d187c5a-7745-447f-a8e2-06c7528e5be0/volumes" Dec 03 12:46:21 crc kubenswrapper[5052]: I1203 12:46:21.717260 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="92290cff-ed52-4a47-80e9-720bfd9f3dd8" path="/var/lib/kubelet/pods/92290cff-ed52-4a47-80e9-720bfd9f3dd8/volumes" Dec 03 12:46:21 crc kubenswrapper[5052]: I1203 12:46:21.742036 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/9481ae66-152c-41e7-9cbc-90dcb5fa6171-ceilometer-tls-certs\") pod \"ceilometer-0\" (UID: \"9481ae66-152c-41e7-9cbc-90dcb5fa6171\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:46:21 crc kubenswrapper[5052]: I1203 12:46:21.803147 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:46:22 crc kubenswrapper[5052]: I1203 12:46:22.022213 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"memcached-tls-certs\" (UniqueName: \"kubernetes.io/secret/4d42b068-9b91-4df6-9a9c-81cad009be16-memcached-tls-certs\") pod \"memcached-0\" (UID: \"4d42b068-9b91-4df6-9a9c-81cad009be16\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:46:22 crc kubenswrapper[5052]: I1203 12:46:22.022411 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/818947c1-95f1-4235-8846-5886abeb57a1-galera-tls-certs\") pod \"openstack-galera-0\" (UID: \"818947c1-95f1-4235-8846-5886abeb57a1\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:46:22 crc kubenswrapper[5052]: E1203 12:46:22.022702 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-memcached-svc: secret "cert-memcached-svc" not found Dec 03 12:46:22 crc kubenswrapper[5052]: E1203 12:46:22.022808 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4d42b068-9b91-4df6-9a9c-81cad009be16-memcached-tls-certs podName:4d42b068-9b91-4df6-9a9c-81cad009be16 nodeName:}" failed. No retries permitted until 2025-12-03 12:46:26.022765921 +0000 UTC m=+5176.611007685 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "memcached-tls-certs" (UniqueName: "kubernetes.io/secret/4d42b068-9b91-4df6-9a9c-81cad009be16-memcached-tls-certs") pod "memcached-0" (UID: "4d42b068-9b91-4df6-9a9c-81cad009be16") : secret "cert-memcached-svc" not found Dec 03 12:46:22 crc kubenswrapper[5052]: I1203 12:46:22.230421 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/576b2af7-7647-4a45-9423-3aa752d84e75-galera-tls-certs\") pod \"openstack-cell1-galera-0\" (UID: \"576b2af7-7647-4a45-9423-3aa752d84e75\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:46:22 crc kubenswrapper[5052]: E1203 12:46:22.230465 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-kube-state-metrics-svc: secret "cert-kube-state-metrics-svc" not found Dec 03 12:46:22 crc kubenswrapper[5052]: E1203 12:46:22.230962 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/876d06e6-0187-4e3d-9183-ed2083fe5189-kube-state-metrics-tls-certs podName:876d06e6-0187-4e3d-9183-ed2083fe5189 nodeName:}" failed. No retries permitted until 2025-12-03 12:46:24.230949055 +0000 UTC m=+5174.819190819 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls-certs" (UniqueName: "kubernetes.io/secret/876d06e6-0187-4e3d-9183-ed2083fe5189-kube-state-metrics-tls-certs") pod "kube-state-metrics-0" (UID: "876d06e6-0187-4e3d-9183-ed2083fe5189") : secret "cert-kube-state-metrics-svc" not found Dec 03 12:46:22 crc kubenswrapper[5052]: E1203 12:46:22.230541 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-galera-openstack-cell1-svc: secret "cert-galera-openstack-cell1-svc" not found Dec 03 12:46:22 crc kubenswrapper[5052]: E1203 12:46:22.230995 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/576b2af7-7647-4a45-9423-3aa752d84e75-galera-tls-certs podName:576b2af7-7647-4a45-9423-3aa752d84e75 nodeName:}" failed. No retries permitted until 2025-12-03 12:46:26.230987026 +0000 UTC m=+5176.819228790 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "galera-tls-certs" (UniqueName: "kubernetes.io/secret/576b2af7-7647-4a45-9423-3aa752d84e75-galera-tls-certs") pod "openstack-cell1-galera-0" (UID: "576b2af7-7647-4a45-9423-3aa752d84e75") : secret "cert-galera-openstack-cell1-svc" not found Dec 03 12:46:22 crc kubenswrapper[5052]: I1203 12:46:22.251553 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"aa557411-6b0a-4596-a5f5-0e23f8686dc4","Type":"ContainerStarted","Data":"28dad3a0f48d004bf9433a47f2e9bb435f9945510d0f67da8d4246236a90195d"} Dec 03 12:46:22 crc kubenswrapper[5052]: I1203 12:46:22.253575 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_ovn-northd-0_6af0a301-2483-493d-bbc8-17138d79dfea/ovn-northd/0.log" Dec 03 12:46:22 crc kubenswrapper[5052]: I1203 12:46:22.253617 5052 generic.go:334] "Generic (PLEG): container finished" podID="6af0a301-2483-493d-bbc8-17138d79dfea" containerID="adc4f6d0e36c4370a4bff93d0b0803fec09e3b34230d31ebbe4f500396923a29" exitCode=139 Dec 03 12:46:22 crc kubenswrapper[5052]: I1203 12:46:22.253693 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:46:22 crc kubenswrapper[5052]: I1203 12:46:22.253716 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovn-northd-0" event={"ID":"6af0a301-2483-493d-bbc8-17138d79dfea","Type":"ContainerDied","Data":"adc4f6d0e36c4370a4bff93d0b0803fec09e3b34230d31ebbe4f500396923a29"} Dec 03 12:46:22 crc kubenswrapper[5052]: I1203 12:46:22.309847 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/openstack-galera-0"] Dec 03 12:46:22 crc kubenswrapper[5052]: I1203 12:46:22.316284 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/openstack-galera-0"] Dec 03 12:46:22 crc kubenswrapper[5052]: I1203 12:46:22.358319 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/openstack-galera-0"] Dec 03 12:46:22 crc kubenswrapper[5052]: E1203 12:46:22.358826 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="92290cff-ed52-4a47-80e9-720bfd9f3dd8" containerName="barbican-worker" Dec 03 12:46:22 crc kubenswrapper[5052]: I1203 12:46:22.358848 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="92290cff-ed52-4a47-80e9-720bfd9f3dd8" containerName="barbican-worker" Dec 03 12:46:22 crc kubenswrapper[5052]: E1203 12:46:22.358871 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="1d187c5a-7745-447f-a8e2-06c7528e5be0" containerName="barbican-keystone-listener" Dec 03 12:46:22 crc kubenswrapper[5052]: I1203 12:46:22.358878 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="1d187c5a-7745-447f-a8e2-06c7528e5be0" containerName="barbican-keystone-listener" Dec 03 12:46:22 crc kubenswrapper[5052]: E1203 12:46:22.358886 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="92290cff-ed52-4a47-80e9-720bfd9f3dd8" containerName="barbican-worker-log" Dec 03 12:46:22 crc kubenswrapper[5052]: I1203 12:46:22.358892 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="92290cff-ed52-4a47-80e9-720bfd9f3dd8" containerName="barbican-worker-log" Dec 03 12:46:22 crc kubenswrapper[5052]: E1203 12:46:22.358900 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="1d187c5a-7745-447f-a8e2-06c7528e5be0" containerName="barbican-keystone-listener-log" Dec 03 12:46:22 crc kubenswrapper[5052]: I1203 12:46:22.358906 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="1d187c5a-7745-447f-a8e2-06c7528e5be0" containerName="barbican-keystone-listener-log" Dec 03 12:46:22 crc kubenswrapper[5052]: I1203 12:46:22.359132 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="1d187c5a-7745-447f-a8e2-06c7528e5be0" containerName="barbican-keystone-listener" Dec 03 12:46:22 crc kubenswrapper[5052]: I1203 12:46:22.359155 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="1d187c5a-7745-447f-a8e2-06c7528e5be0" containerName="barbican-keystone-listener-log" Dec 03 12:46:22 crc kubenswrapper[5052]: I1203 12:46:22.359177 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="92290cff-ed52-4a47-80e9-720bfd9f3dd8" containerName="barbican-worker-log" Dec 03 12:46:22 crc kubenswrapper[5052]: I1203 12:46:22.359197 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="92290cff-ed52-4a47-80e9-720bfd9f3dd8" containerName="barbican-worker" Dec 03 12:46:22 crc kubenswrapper[5052]: I1203 12:46:22.360145 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/openstack-galera-0"] Dec 03 12:46:22 crc kubenswrapper[5052]: I1203 12:46:22.360237 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:46:22 crc kubenswrapper[5052]: I1203 12:46:22.363425 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-galera-openstack-svc" Dec 03 12:46:22 crc kubenswrapper[5052]: I1203 12:46:22.364666 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"openstack-config-data" Dec 03 12:46:22 crc kubenswrapper[5052]: I1203 12:46:22.364842 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"openstack-scripts" Dec 03 12:46:22 crc kubenswrapper[5052]: I1203 12:46:22.366008 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"galera-openstack-dockercfg-tsmcq" Dec 03 12:46:22 crc kubenswrapper[5052]: I1203 12:46:22.768830 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/ed7f9920-a1e4-437b-8074-a30d0fe4450c-internal-tls-certs\") pod \"nova-api-0\" (UID: \"ed7f9920-a1e4-437b-8074-a30d0fe4450c\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:46:22 crc kubenswrapper[5052]: I1203 12:46:22.769536 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/ed7f9920-a1e4-437b-8074-a30d0fe4450c-public-tls-certs\") pod \"nova-api-0\" (UID: \"ed7f9920-a1e4-437b-8074-a30d0fe4450c\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:46:22 crc kubenswrapper[5052]: E1203 12:46:22.769745 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-swift-public-svc: secret "cert-swift-public-svc" not found Dec 03 12:46:22 crc kubenswrapper[5052]: E1203 12:46:22.769802 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/242be027-6461-4aea-ae7c-e52ca996b2ea-public-tls-certs podName:242be027-6461-4aea-ae7c-e52ca996b2ea nodeName:}" failed. No retries permitted until 2025-12-03 12:46:26.769786454 +0000 UTC m=+5177.358028218 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "public-tls-certs" (UniqueName: "kubernetes.io/secret/242be027-6461-4aea-ae7c-e52ca996b2ea-public-tls-certs") pod "swift-proxy-dd4cf57c8-x4c45" (UID: "242be027-6461-4aea-ae7c-e52ca996b2ea") : secret "cert-swift-public-svc" not found Dec 03 12:46:22 crc kubenswrapper[5052]: E1203 12:46:22.770397 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-swift-internal-svc: secret "cert-swift-internal-svc" not found Dec 03 12:46:22 crc kubenswrapper[5052]: E1203 12:46:22.770398 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-nova-internal-svc: secret "cert-nova-internal-svc" not found Dec 03 12:46:22 crc kubenswrapper[5052]: E1203 12:46:22.770911 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-nova-public-svc: secret "cert-nova-public-svc" not found Dec 03 12:46:22 crc kubenswrapper[5052]: E1203 12:46:22.771026 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ed7f9920-a1e4-437b-8074-a30d0fe4450c-public-tls-certs podName:ed7f9920-a1e4-437b-8074-a30d0fe4450c nodeName:}" failed. No retries permitted until 2025-12-03 12:46:26.771003469 +0000 UTC m=+5177.359245293 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "public-tls-certs" (UniqueName: "kubernetes.io/secret/ed7f9920-a1e4-437b-8074-a30d0fe4450c-public-tls-certs") pod "nova-api-0" (UID: "ed7f9920-a1e4-437b-8074-a30d0fe4450c") : secret "cert-nova-public-svc" not found Dec 03 12:46:22 crc kubenswrapper[5052]: E1203 12:46:22.771162 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ed7f9920-a1e4-437b-8074-a30d0fe4450c-internal-tls-certs podName:ed7f9920-a1e4-437b-8074-a30d0fe4450c nodeName:}" failed. No retries permitted until 2025-12-03 12:46:26.771136483 +0000 UTC m=+5177.359378307 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "internal-tls-certs" (UniqueName: "kubernetes.io/secret/ed7f9920-a1e4-437b-8074-a30d0fe4450c-internal-tls-certs") pod "nova-api-0" (UID: "ed7f9920-a1e4-437b-8074-a30d0fe4450c") : secret "cert-nova-internal-svc" not found Dec 03 12:46:22 crc kubenswrapper[5052]: E1203 12:46:22.771246 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/242be027-6461-4aea-ae7c-e52ca996b2ea-internal-tls-certs podName:242be027-6461-4aea-ae7c-e52ca996b2ea nodeName:}" failed. No retries permitted until 2025-12-03 12:46:26.771235246 +0000 UTC m=+5177.359477120 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "internal-tls-certs" (UniqueName: "kubernetes.io/secret/242be027-6461-4aea-ae7c-e52ca996b2ea-internal-tls-certs") pod "swift-proxy-dd4cf57c8-x4c45" (UID: "242be027-6461-4aea-ae7c-e52ca996b2ea") : secret "cert-swift-internal-svc" not found Dec 03 12:46:22 crc kubenswrapper[5052]: I1203 12:46:22.771231 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/818947c1-95f1-4235-8846-5886abeb57a1-galera-tls-certs\") pod \"openstack-galera-0\" (UID: \"818947c1-95f1-4235-8846-5886abeb57a1\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:46:22 crc kubenswrapper[5052]: I1203 12:46:22.837732 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_ovn-northd-0_6af0a301-2483-493d-bbc8-17138d79dfea/ovn-northd/0.log" Dec 03 12:46:22 crc kubenswrapper[5052]: I1203 12:46:22.838083 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:46:22 crc kubenswrapper[5052]: W1203 12:46:22.847630 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9481ae66_152c_41e7_9cbc_90dcb5fa6171.slice/crio-6ac650b6651daa123e83a2c4f419a2e53e6d72716603c465e8592bd1223209b1 WatchSource:0}: Error finding container 6ac650b6651daa123e83a2c4f419a2e53e6d72716603c465e8592bd1223209b1: Status 404 returned error can't find the container with id 6ac650b6651daa123e83a2c4f419a2e53e6d72716603c465e8592bd1223209b1 Dec 03 12:46:22 crc kubenswrapper[5052]: I1203 12:46:22.849654 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:46:22 crc kubenswrapper[5052]: I1203 12:46:22.871327 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/818947c1-95f1-4235-8846-5886abeb57a1-galera-tls-certs\") pod \"818947c1-95f1-4235-8846-5886abeb57a1\" (UID: \"818947c1-95f1-4235-8846-5886abeb57a1\") " Dec 03 12:46:22 crc kubenswrapper[5052]: I1203 12:46:22.871919 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/3c8e4e3f-f6c1-4362-b64c-b10a973723dd-kolla-config\") pod \"openstack-galera-0\" (UID: \"3c8e4e3f-f6c1-4362-b64c-b10a973723dd\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:46:22 crc kubenswrapper[5052]: I1203 12:46:22.872024 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/3c8e4e3f-f6c1-4362-b64c-b10a973723dd-operator-scripts\") pod \"openstack-galera-0\" (UID: \"3c8e4e3f-f6c1-4362-b64c-b10a973723dd\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:46:22 crc kubenswrapper[5052]: I1203 12:46:22.872056 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/3c8e4e3f-f6c1-4362-b64c-b10a973723dd-galera-tls-certs\") pod \"openstack-galera-0\" (UID: \"3c8e4e3f-f6c1-4362-b64c-b10a973723dd\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:46:22 crc kubenswrapper[5052]: I1203 12:46:22.872139 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage07-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage07-crc\") pod \"openstack-galera-0\" (UID: \"3c8e4e3f-f6c1-4362-b64c-b10a973723dd\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:46:22 crc kubenswrapper[5052]: I1203 12:46:22.872267 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/3c8e4e3f-f6c1-4362-b64c-b10a973723dd-config-data-generated\") pod \"openstack-galera-0\" (UID: \"3c8e4e3f-f6c1-4362-b64c-b10a973723dd\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:46:22 crc kubenswrapper[5052]: I1203 12:46:22.872337 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/3c8e4e3f-f6c1-4362-b64c-b10a973723dd-config-data-default\") pod \"openstack-galera-0\" (UID: \"3c8e4e3f-f6c1-4362-b64c-b10a973723dd\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:46:22 crc kubenswrapper[5052]: I1203 12:46:22.872387 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ftscq\" (UniqueName: \"kubernetes.io/projected/3c8e4e3f-f6c1-4362-b64c-b10a973723dd-kube-api-access-ftscq\") pod \"openstack-galera-0\" (UID: \"3c8e4e3f-f6c1-4362-b64c-b10a973723dd\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:46:22 crc kubenswrapper[5052]: I1203 12:46:22.872484 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3c8e4e3f-f6c1-4362-b64c-b10a973723dd-combined-ca-bundle\") pod \"openstack-galera-0\" (UID: \"3c8e4e3f-f6c1-4362-b64c-b10a973723dd\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:46:22 crc kubenswrapper[5052]: E1203 12:46:22.872914 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-placement-internal-svc: secret "cert-placement-internal-svc" not found Dec 03 12:46:22 crc kubenswrapper[5052]: E1203 12:46:22.872970 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/32f420c2-aa7e-44c8-a63a-7654aa3e6e34-internal-tls-certs podName:32f420c2-aa7e-44c8-a63a-7654aa3e6e34 nodeName:}" failed. No retries permitted until 2025-12-03 12:46:26.872953915 +0000 UTC m=+5177.461195679 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "internal-tls-certs" (UniqueName: "kubernetes.io/secret/32f420c2-aa7e-44c8-a63a-7654aa3e6e34-internal-tls-certs") pod "placement-bbcfbbc9d-5q6gx" (UID: "32f420c2-aa7e-44c8-a63a-7654aa3e6e34") : secret "cert-placement-internal-svc" not found Dec 03 12:46:22 crc kubenswrapper[5052]: E1203 12:46:22.873302 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-placement-public-svc: secret "cert-placement-public-svc" not found Dec 03 12:46:22 crc kubenswrapper[5052]: E1203 12:46:22.873342 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/32f420c2-aa7e-44c8-a63a-7654aa3e6e34-public-tls-certs podName:32f420c2-aa7e-44c8-a63a-7654aa3e6e34 nodeName:}" failed. No retries permitted until 2025-12-03 12:46:26.873331996 +0000 UTC m=+5177.461573760 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "public-tls-certs" (UniqueName: "kubernetes.io/secret/32f420c2-aa7e-44c8-a63a-7654aa3e6e34-public-tls-certs") pod "placement-bbcfbbc9d-5q6gx" (UID: "32f420c2-aa7e-44c8-a63a-7654aa3e6e34") : secret "cert-placement-public-svc" not found Dec 03 12:46:22 crc kubenswrapper[5052]: I1203 12:46:22.882832 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/818947c1-95f1-4235-8846-5886abeb57a1-galera-tls-certs" (OuterVolumeSpecName: "galera-tls-certs") pod "818947c1-95f1-4235-8846-5886abeb57a1" (UID: "818947c1-95f1-4235-8846-5886abeb57a1"). InnerVolumeSpecName "galera-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:46:22 crc kubenswrapper[5052]: I1203 12:46:22.973274 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d89l2\" (UniqueName: \"kubernetes.io/projected/6af0a301-2483-493d-bbc8-17138d79dfea-kube-api-access-d89l2\") pod \"6af0a301-2483-493d-bbc8-17138d79dfea\" (UID: \"6af0a301-2483-493d-bbc8-17138d79dfea\") " Dec 03 12:46:22 crc kubenswrapper[5052]: I1203 12:46:22.973383 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/6af0a301-2483-493d-bbc8-17138d79dfea-config\") pod \"6af0a301-2483-493d-bbc8-17138d79dfea\" (UID: \"6af0a301-2483-493d-bbc8-17138d79dfea\") " Dec 03 12:46:22 crc kubenswrapper[5052]: I1203 12:46:22.973411 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/6af0a301-2483-493d-bbc8-17138d79dfea-scripts\") pod \"6af0a301-2483-493d-bbc8-17138d79dfea\" (UID: \"6af0a301-2483-493d-bbc8-17138d79dfea\") " Dec 03 12:46:22 crc kubenswrapper[5052]: I1203 12:46:22.973445 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovn-northd-tls-certs\" (UniqueName: \"kubernetes.io/secret/6af0a301-2483-493d-bbc8-17138d79dfea-ovn-northd-tls-certs\") pod \"6af0a301-2483-493d-bbc8-17138d79dfea\" (UID: \"6af0a301-2483-493d-bbc8-17138d79dfea\") " Dec 03 12:46:22 crc kubenswrapper[5052]: I1203 12:46:22.973543 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/6af0a301-2483-493d-bbc8-17138d79dfea-metrics-certs-tls-certs\") pod \"6af0a301-2483-493d-bbc8-17138d79dfea\" (UID: \"6af0a301-2483-493d-bbc8-17138d79dfea\") " Dec 03 12:46:22 crc kubenswrapper[5052]: I1203 12:46:22.973635 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovn-rundir\" (UniqueName: \"kubernetes.io/empty-dir/6af0a301-2483-493d-bbc8-17138d79dfea-ovn-rundir\") pod \"6af0a301-2483-493d-bbc8-17138d79dfea\" (UID: \"6af0a301-2483-493d-bbc8-17138d79dfea\") " Dec 03 12:46:22 crc kubenswrapper[5052]: I1203 12:46:22.973699 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6af0a301-2483-493d-bbc8-17138d79dfea-combined-ca-bundle\") pod \"6af0a301-2483-493d-bbc8-17138d79dfea\" (UID: \"6af0a301-2483-493d-bbc8-17138d79dfea\") " Dec 03 12:46:22 crc kubenswrapper[5052]: I1203 12:46:22.974006 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/3c8e4e3f-f6c1-4362-b64c-b10a973723dd-config-data-generated\") pod \"openstack-galera-0\" (UID: \"3c8e4e3f-f6c1-4362-b64c-b10a973723dd\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:46:22 crc kubenswrapper[5052]: I1203 12:46:22.974045 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/4c9f4da8-f1ae-4815-8ee2-0e873fb07d62-public-tls-certs\") pod \"glance-default-external-api-0\" (UID: \"4c9f4da8-f1ae-4815-8ee2-0e873fb07d62\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:46:22 crc kubenswrapper[5052]: I1203 12:46:22.974072 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/3c8e4e3f-f6c1-4362-b64c-b10a973723dd-config-data-default\") pod \"openstack-galera-0\" (UID: \"3c8e4e3f-f6c1-4362-b64c-b10a973723dd\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:46:22 crc kubenswrapper[5052]: I1203 12:46:22.974101 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-ftscq\" (UniqueName: \"kubernetes.io/projected/3c8e4e3f-f6c1-4362-b64c-b10a973723dd-kube-api-access-ftscq\") pod \"openstack-galera-0\" (UID: \"3c8e4e3f-f6c1-4362-b64c-b10a973723dd\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:46:22 crc kubenswrapper[5052]: I1203 12:46:22.974201 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3c8e4e3f-f6c1-4362-b64c-b10a973723dd-combined-ca-bundle\") pod \"openstack-galera-0\" (UID: \"3c8e4e3f-f6c1-4362-b64c-b10a973723dd\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:46:22 crc kubenswrapper[5052]: I1203 12:46:22.974302 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/3c8e4e3f-f6c1-4362-b64c-b10a973723dd-kolla-config\") pod \"openstack-galera-0\" (UID: \"3c8e4e3f-f6c1-4362-b64c-b10a973723dd\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:46:22 crc kubenswrapper[5052]: I1203 12:46:22.974377 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/3c8e4e3f-f6c1-4362-b64c-b10a973723dd-operator-scripts\") pod \"openstack-galera-0\" (UID: \"3c8e4e3f-f6c1-4362-b64c-b10a973723dd\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:46:22 crc kubenswrapper[5052]: I1203 12:46:22.974404 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/3c8e4e3f-f6c1-4362-b64c-b10a973723dd-galera-tls-certs\") pod \"openstack-galera-0\" (UID: \"3c8e4e3f-f6c1-4362-b64c-b10a973723dd\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:46:22 crc kubenswrapper[5052]: I1203 12:46:22.974466 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage07-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage07-crc\") pod \"openstack-galera-0\" (UID: \"3c8e4e3f-f6c1-4362-b64c-b10a973723dd\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:46:22 crc kubenswrapper[5052]: I1203 12:46:22.974554 5052 reconciler_common.go:293] "Volume detached for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/818947c1-95f1-4235-8846-5886abeb57a1-galera-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:22 crc kubenswrapper[5052]: I1203 12:46:22.974885 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage07-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage07-crc\") pod \"openstack-galera-0\" (UID: \"3c8e4e3f-f6c1-4362-b64c-b10a973723dd\") device mount path \"/mnt/openstack/pv07\"" pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:46:22 crc kubenswrapper[5052]: I1203 12:46:22.978066 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/3c8e4e3f-f6c1-4362-b64c-b10a973723dd-config-data-generated\") pod \"openstack-galera-0\" (UID: \"3c8e4e3f-f6c1-4362-b64c-b10a973723dd\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:46:22 crc kubenswrapper[5052]: E1203 12:46:22.985065 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-glance-default-public-svc: secret "cert-glance-default-public-svc" not found Dec 03 12:46:22 crc kubenswrapper[5052]: E1203 12:46:22.985171 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4c9f4da8-f1ae-4815-8ee2-0e873fb07d62-public-tls-certs podName:4c9f4da8-f1ae-4815-8ee2-0e873fb07d62 nodeName:}" failed. No retries permitted until 2025-12-03 12:46:26.985147573 +0000 UTC m=+5177.573389337 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "public-tls-certs" (UniqueName: "kubernetes.io/secret/4c9f4da8-f1ae-4815-8ee2-0e873fb07d62-public-tls-certs") pod "glance-default-external-api-0" (UID: "4c9f4da8-f1ae-4815-8ee2-0e873fb07d62") : secret "cert-glance-default-public-svc" not found Dec 03 12:46:22 crc kubenswrapper[5052]: I1203 12:46:22.985602 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/3c8e4e3f-f6c1-4362-b64c-b10a973723dd-config-data-default\") pod \"openstack-galera-0\" (UID: \"3c8e4e3f-f6c1-4362-b64c-b10a973723dd\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:46:22 crc kubenswrapper[5052]: I1203 12:46:22.986622 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/3c8e4e3f-f6c1-4362-b64c-b10a973723dd-kolla-config\") pod \"openstack-galera-0\" (UID: \"3c8e4e3f-f6c1-4362-b64c-b10a973723dd\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:46:22 crc kubenswrapper[5052]: I1203 12:46:22.987230 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/3c8e4e3f-f6c1-4362-b64c-b10a973723dd-operator-scripts\") pod \"openstack-galera-0\" (UID: \"3c8e4e3f-f6c1-4362-b64c-b10a973723dd\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:46:22 crc kubenswrapper[5052]: I1203 12:46:22.987783 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6af0a301-2483-493d-bbc8-17138d79dfea-kube-api-access-d89l2" (OuterVolumeSpecName: "kube-api-access-d89l2") pod "6af0a301-2483-493d-bbc8-17138d79dfea" (UID: "6af0a301-2483-493d-bbc8-17138d79dfea"). InnerVolumeSpecName "kube-api-access-d89l2". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:46:22 crc kubenswrapper[5052]: I1203 12:46:22.990054 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6af0a301-2483-493d-bbc8-17138d79dfea-ovn-rundir" (OuterVolumeSpecName: "ovn-rundir") pod "6af0a301-2483-493d-bbc8-17138d79dfea" (UID: "6af0a301-2483-493d-bbc8-17138d79dfea"). InnerVolumeSpecName "ovn-rundir". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:46:22 crc kubenswrapper[5052]: I1203 12:46:22.990179 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3c8e4e3f-f6c1-4362-b64c-b10a973723dd-combined-ca-bundle\") pod \"openstack-galera-0\" (UID: \"3c8e4e3f-f6c1-4362-b64c-b10a973723dd\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:46:22 crc kubenswrapper[5052]: I1203 12:46:22.990840 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6af0a301-2483-493d-bbc8-17138d79dfea-scripts" (OuterVolumeSpecName: "scripts") pod "6af0a301-2483-493d-bbc8-17138d79dfea" (UID: "6af0a301-2483-493d-bbc8-17138d79dfea"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:46:23 crc kubenswrapper[5052]: I1203 12:46:23.013778 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6af0a301-2483-493d-bbc8-17138d79dfea-config" (OuterVolumeSpecName: "config") pod "6af0a301-2483-493d-bbc8-17138d79dfea" (UID: "6af0a301-2483-493d-bbc8-17138d79dfea"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:46:23 crc kubenswrapper[5052]: I1203 12:46:23.020480 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/3c8e4e3f-f6c1-4362-b64c-b10a973723dd-galera-tls-certs\") pod \"openstack-galera-0\" (UID: \"3c8e4e3f-f6c1-4362-b64c-b10a973723dd\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:46:23 crc kubenswrapper[5052]: I1203 12:46:23.020619 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage07-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage07-crc\") pod \"openstack-galera-0\" (UID: \"3c8e4e3f-f6c1-4362-b64c-b10a973723dd\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:46:23 crc kubenswrapper[5052]: I1203 12:46:23.035674 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6af0a301-2483-493d-bbc8-17138d79dfea-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "6af0a301-2483-493d-bbc8-17138d79dfea" (UID: "6af0a301-2483-493d-bbc8-17138d79dfea"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:46:23 crc kubenswrapper[5052]: I1203 12:46:23.040320 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-ftscq\" (UniqueName: \"kubernetes.io/projected/3c8e4e3f-f6c1-4362-b64c-b10a973723dd-kube-api-access-ftscq\") pod \"openstack-galera-0\" (UID: \"3c8e4e3f-f6c1-4362-b64c-b10a973723dd\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:46:23 crc kubenswrapper[5052]: I1203 12:46:23.076916 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/bf8b9b9c-b8f9-4c93-8112-43913950fcdf-internal-tls-certs\") pod \"glance-default-internal-api-0\" (UID: \"bf8b9b9c-b8f9-4c93-8112-43913950fcdf\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:46:23 crc kubenswrapper[5052]: E1203 12:46:23.078420 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-glance-default-internal-svc: secret "cert-glance-default-internal-svc" not found Dec 03 12:46:23 crc kubenswrapper[5052]: E1203 12:46:23.078521 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bf8b9b9c-b8f9-4c93-8112-43913950fcdf-internal-tls-certs podName:bf8b9b9c-b8f9-4c93-8112-43913950fcdf nodeName:}" failed. No retries permitted until 2025-12-03 12:46:27.078485203 +0000 UTC m=+5177.666726967 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "internal-tls-certs" (UniqueName: "kubernetes.io/secret/bf8b9b9c-b8f9-4c93-8112-43913950fcdf-internal-tls-certs") pod "glance-default-internal-api-0" (UID: "bf8b9b9c-b8f9-4c93-8112-43913950fcdf") : secret "cert-glance-default-internal-svc" not found Dec 03 12:46:23 crc kubenswrapper[5052]: I1203 12:46:23.078701 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6af0a301-2483-493d-bbc8-17138d79dfea-metrics-certs-tls-certs" (OuterVolumeSpecName: "metrics-certs-tls-certs") pod "6af0a301-2483-493d-bbc8-17138d79dfea" (UID: "6af0a301-2483-493d-bbc8-17138d79dfea"). InnerVolumeSpecName "metrics-certs-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:46:23 crc kubenswrapper[5052]: I1203 12:46:23.080439 5052 reconciler_common.go:293] "Volume detached for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/6af0a301-2483-493d-bbc8-17138d79dfea-metrics-certs-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:23 crc kubenswrapper[5052]: I1203 12:46:23.080466 5052 reconciler_common.go:293] "Volume detached for volume \"ovn-rundir\" (UniqueName: \"kubernetes.io/empty-dir/6af0a301-2483-493d-bbc8-17138d79dfea-ovn-rundir\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:23 crc kubenswrapper[5052]: I1203 12:46:23.080479 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6af0a301-2483-493d-bbc8-17138d79dfea-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:23 crc kubenswrapper[5052]: I1203 12:46:23.080493 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-d89l2\" (UniqueName: \"kubernetes.io/projected/6af0a301-2483-493d-bbc8-17138d79dfea-kube-api-access-d89l2\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:23 crc kubenswrapper[5052]: I1203 12:46:23.080517 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/6af0a301-2483-493d-bbc8-17138d79dfea-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:23 crc kubenswrapper[5052]: I1203 12:46:23.080528 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/6af0a301-2483-493d-bbc8-17138d79dfea-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:23 crc kubenswrapper[5052]: I1203 12:46:23.090858 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6af0a301-2483-493d-bbc8-17138d79dfea-ovn-northd-tls-certs" (OuterVolumeSpecName: "ovn-northd-tls-certs") pod "6af0a301-2483-493d-bbc8-17138d79dfea" (UID: "6af0a301-2483-493d-bbc8-17138d79dfea"). InnerVolumeSpecName "ovn-northd-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:46:23 crc kubenswrapper[5052]: I1203 12:46:23.182677 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/234278ab-b4a8-4fdf-abf9-c67cdbff1ac0-internal-tls-certs\") pod \"cinder-api-0\" (UID: \"234278ab-b4a8-4fdf-abf9-c67cdbff1ac0\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:46:23 crc kubenswrapper[5052]: I1203 12:46:23.183036 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/234278ab-b4a8-4fdf-abf9-c67cdbff1ac0-public-tls-certs\") pod \"cinder-api-0\" (UID: \"234278ab-b4a8-4fdf-abf9-c67cdbff1ac0\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:46:23 crc kubenswrapper[5052]: I1203 12:46:23.183202 5052 reconciler_common.go:293] "Volume detached for volume \"ovn-northd-tls-certs\" (UniqueName: \"kubernetes.io/secret/6af0a301-2483-493d-bbc8-17138d79dfea-ovn-northd-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:23 crc kubenswrapper[5052]: E1203 12:46:23.183297 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-cinder-public-svc: secret "cert-cinder-public-svc" not found Dec 03 12:46:23 crc kubenswrapper[5052]: E1203 12:46:23.183346 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/234278ab-b4a8-4fdf-abf9-c67cdbff1ac0-public-tls-certs podName:234278ab-b4a8-4fdf-abf9-c67cdbff1ac0 nodeName:}" failed. No retries permitted until 2025-12-03 12:46:27.183333691 +0000 UTC m=+5177.771575455 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "public-tls-certs" (UniqueName: "kubernetes.io/secret/234278ab-b4a8-4fdf-abf9-c67cdbff1ac0-public-tls-certs") pod "cinder-api-0" (UID: "234278ab-b4a8-4fdf-abf9-c67cdbff1ac0") : secret "cert-cinder-public-svc" not found Dec 03 12:46:23 crc kubenswrapper[5052]: E1203 12:46:23.183825 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-cinder-internal-svc: secret "cert-cinder-internal-svc" not found Dec 03 12:46:23 crc kubenswrapper[5052]: E1203 12:46:23.183934 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/234278ab-b4a8-4fdf-abf9-c67cdbff1ac0-internal-tls-certs podName:234278ab-b4a8-4fdf-abf9-c67cdbff1ac0 nodeName:}" failed. No retries permitted until 2025-12-03 12:46:27.183912278 +0000 UTC m=+5177.772154082 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "internal-tls-certs" (UniqueName: "kubernetes.io/secret/234278ab-b4a8-4fdf-abf9-c67cdbff1ac0-internal-tls-certs") pod "cinder-api-0" (UID: "234278ab-b4a8-4fdf-abf9-c67cdbff1ac0") : secret "cert-cinder-internal-svc" not found Dec 03 12:46:23 crc kubenswrapper[5052]: I1203 12:46:23.265989 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"9481ae66-152c-41e7-9cbc-90dcb5fa6171","Type":"ContainerStarted","Data":"6ac650b6651daa123e83a2c4f419a2e53e6d72716603c465e8592bd1223209b1"} Dec 03 12:46:23 crc kubenswrapper[5052]: I1203 12:46:23.268383 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_ovn-northd-0_6af0a301-2483-493d-bbc8-17138d79dfea/ovn-northd/0.log" Dec 03 12:46:23 crc kubenswrapper[5052]: I1203 12:46:23.268464 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovn-northd-0" event={"ID":"6af0a301-2483-493d-bbc8-17138d79dfea","Type":"ContainerDied","Data":"fc42c73e23176434d6c24f8df9babe1e324d8caa46125fa1de4d5f4f4ddcc90d"} Dec 03 12:46:23 crc kubenswrapper[5052]: I1203 12:46:23.268495 5052 scope.go:117] "RemoveContainer" containerID="4307735eba8d16b00eb4ca5f1d746b675f8e81debcf3a1c8d9b709367fb6a8be" Dec 03 12:46:23 crc kubenswrapper[5052]: I1203 12:46:23.268530 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:46:23 crc kubenswrapper[5052]: I1203 12:46:23.272115 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"aa557411-6b0a-4596-a5f5-0e23f8686dc4","Type":"ContainerStarted","Data":"1d9679ea8e92979b2a265175ba31597cc56e80e13b2610ba8cc4b4e015500253"} Dec 03 12:46:23 crc kubenswrapper[5052]: I1203 12:46:23.280473 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:46:23 crc kubenswrapper[5052]: I1203 12:46:23.304736 5052 scope.go:117] "RemoveContainer" containerID="adc4f6d0e36c4370a4bff93d0b0803fec09e3b34230d31ebbe4f500396923a29" Dec 03 12:46:23 crc kubenswrapper[5052]: I1203 12:46:23.313232 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/cinder-scheduler-0" podStartSLOduration=4.313213866 podStartE2EDuration="4.313213866s" podCreationTimestamp="2025-12-03 12:46:19 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:46:23.291271218 +0000 UTC m=+5173.879512982" watchObservedRunningTime="2025-12-03 12:46:23.313213866 +0000 UTC m=+5173.901455620" Dec 03 12:46:23 crc kubenswrapper[5052]: I1203 12:46:23.355657 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ovn-northd-0"] Dec 03 12:46:23 crc kubenswrapper[5052]: I1203 12:46:23.369600 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ovn-northd-0"] Dec 03 12:46:23 crc kubenswrapper[5052]: I1203 12:46:23.380176 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ovn-northd-0"] Dec 03 12:46:23 crc kubenswrapper[5052]: E1203 12:46:23.380919 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="6af0a301-2483-493d-bbc8-17138d79dfea" containerName="openstack-network-exporter" Dec 03 12:46:23 crc kubenswrapper[5052]: I1203 12:46:23.380941 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="6af0a301-2483-493d-bbc8-17138d79dfea" containerName="openstack-network-exporter" Dec 03 12:46:23 crc kubenswrapper[5052]: E1203 12:46:23.380998 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="6af0a301-2483-493d-bbc8-17138d79dfea" containerName="ovn-northd" Dec 03 12:46:23 crc kubenswrapper[5052]: I1203 12:46:23.381006 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="6af0a301-2483-493d-bbc8-17138d79dfea" containerName="ovn-northd" Dec 03 12:46:23 crc kubenswrapper[5052]: I1203 12:46:23.381326 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="6af0a301-2483-493d-bbc8-17138d79dfea" containerName="ovn-northd" Dec 03 12:46:23 crc kubenswrapper[5052]: I1203 12:46:23.381383 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="6af0a301-2483-493d-bbc8-17138d79dfea" containerName="openstack-network-exporter" Dec 03 12:46:23 crc kubenswrapper[5052]: I1203 12:46:23.383408 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:46:23 crc kubenswrapper[5052]: I1203 12:46:23.386396 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"ovnnorthd-scripts" Dec 03 12:46:23 crc kubenswrapper[5052]: I1203 12:46:23.386774 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-ovnnorthd-ovndbs" Dec 03 12:46:23 crc kubenswrapper[5052]: I1203 12:46:23.387019 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"ovnnorthd-config" Dec 03 12:46:23 crc kubenswrapper[5052]: I1203 12:46:23.387254 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ovnnorthd-ovnnorthd-dockercfg-lhpt7" Dec 03 12:46:23 crc kubenswrapper[5052]: I1203 12:46:23.395352 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ovn-northd-0"] Dec 03 12:46:24 crc kubenswrapper[5052]: I1203 12:46:23.492068 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f8ab3054-5998-40c1-852f-be0dbaf022e5-combined-ca-bundle\") pod \"ovn-northd-0\" (UID: \"f8ab3054-5998-40c1-852f-be0dbaf022e5\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:46:24 crc kubenswrapper[5052]: I1203 12:46:23.492134 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-rundir\" (UniqueName: \"kubernetes.io/empty-dir/f8ab3054-5998-40c1-852f-be0dbaf022e5-ovn-rundir\") pod \"ovn-northd-0\" (UID: \"f8ab3054-5998-40c1-852f-be0dbaf022e5\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:46:24 crc kubenswrapper[5052]: I1203 12:46:23.492226 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/f8ab3054-5998-40c1-852f-be0dbaf022e5-metrics-certs-tls-certs\") pod \"ovn-northd-0\" (UID: \"f8ab3054-5998-40c1-852f-be0dbaf022e5\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:46:24 crc kubenswrapper[5052]: I1203 12:46:23.492334 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-northd-tls-certs\" (UniqueName: \"kubernetes.io/secret/f8ab3054-5998-40c1-852f-be0dbaf022e5-ovn-northd-tls-certs\") pod \"ovn-northd-0\" (UID: \"f8ab3054-5998-40c1-852f-be0dbaf022e5\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:46:24 crc kubenswrapper[5052]: I1203 12:46:23.492485 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/f8ab3054-5998-40c1-852f-be0dbaf022e5-scripts\") pod \"ovn-northd-0\" (UID: \"f8ab3054-5998-40c1-852f-be0dbaf022e5\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:46:24 crc kubenswrapper[5052]: I1203 12:46:23.492563 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p6qch\" (UniqueName: \"kubernetes.io/projected/f8ab3054-5998-40c1-852f-be0dbaf022e5-kube-api-access-p6qch\") pod \"ovn-northd-0\" (UID: \"f8ab3054-5998-40c1-852f-be0dbaf022e5\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:46:24 crc kubenswrapper[5052]: I1203 12:46:23.492589 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f8ab3054-5998-40c1-852f-be0dbaf022e5-config\") pod \"ovn-northd-0\" (UID: \"f8ab3054-5998-40c1-852f-be0dbaf022e5\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:46:24 crc kubenswrapper[5052]: I1203 12:46:23.527332 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:46:24 crc kubenswrapper[5052]: I1203 12:46:23.594393 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovn-northd-tls-certs\" (UniqueName: \"kubernetes.io/secret/f8ab3054-5998-40c1-852f-be0dbaf022e5-ovn-northd-tls-certs\") pod \"ovn-northd-0\" (UID: \"f8ab3054-5998-40c1-852f-be0dbaf022e5\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:46:24 crc kubenswrapper[5052]: I1203 12:46:23.594899 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/c6c9b62e-bbbc-4557-92fd-da64bd8ee0cf-metrics-certs-tls-certs\") pod \"ovsdbserver-sb-0\" (UID: \"c6c9b62e-bbbc-4557-92fd-da64bd8ee0cf\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:46:24 crc kubenswrapper[5052]: E1203 12:46:23.594639 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-ovnnorthd-ovndbs: secret "cert-ovnnorthd-ovndbs" not found Dec 03 12:46:24 crc kubenswrapper[5052]: I1203 12:46:23.594956 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/f8ab3054-5998-40c1-852f-be0dbaf022e5-scripts\") pod \"ovn-northd-0\" (UID: \"f8ab3054-5998-40c1-852f-be0dbaf022e5\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:46:24 crc kubenswrapper[5052]: E1203 12:46:23.594996 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f8ab3054-5998-40c1-852f-be0dbaf022e5-ovn-northd-tls-certs podName:f8ab3054-5998-40c1-852f-be0dbaf022e5 nodeName:}" failed. No retries permitted until 2025-12-03 12:46:24.094973704 +0000 UTC m=+5174.683215518 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "ovn-northd-tls-certs" (UniqueName: "kubernetes.io/secret/f8ab3054-5998-40c1-852f-be0dbaf022e5-ovn-northd-tls-certs") pod "ovn-northd-0" (UID: "f8ab3054-5998-40c1-852f-be0dbaf022e5") : secret "cert-ovnnorthd-ovndbs" not found Dec 03 12:46:24 crc kubenswrapper[5052]: I1203 12:46:23.595111 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-p6qch\" (UniqueName: \"kubernetes.io/projected/f8ab3054-5998-40c1-852f-be0dbaf022e5-kube-api-access-p6qch\") pod \"ovn-northd-0\" (UID: \"f8ab3054-5998-40c1-852f-be0dbaf022e5\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:46:24 crc kubenswrapper[5052]: I1203 12:46:23.595148 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f8ab3054-5998-40c1-852f-be0dbaf022e5-config\") pod \"ovn-northd-0\" (UID: \"f8ab3054-5998-40c1-852f-be0dbaf022e5\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:46:24 crc kubenswrapper[5052]: I1203 12:46:23.596188 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/f8ab3054-5998-40c1-852f-be0dbaf022e5-scripts\") pod \"ovn-northd-0\" (UID: \"f8ab3054-5998-40c1-852f-be0dbaf022e5\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:46:24 crc kubenswrapper[5052]: E1203 12:46:23.596759 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-ovn-metrics: secret "cert-ovn-metrics" not found Dec 03 12:46:24 crc kubenswrapper[5052]: E1203 12:46:23.596854 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c6c9b62e-bbbc-4557-92fd-da64bd8ee0cf-metrics-certs-tls-certs podName:c6c9b62e-bbbc-4557-92fd-da64bd8ee0cf nodeName:}" failed. No retries permitted until 2025-12-03 12:46:31.596810426 +0000 UTC m=+5182.185052260 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs-tls-certs" (UniqueName: "kubernetes.io/secret/c6c9b62e-bbbc-4557-92fd-da64bd8ee0cf-metrics-certs-tls-certs") pod "ovsdbserver-sb-0" (UID: "c6c9b62e-bbbc-4557-92fd-da64bd8ee0cf") : secret "cert-ovn-metrics" not found Dec 03 12:46:24 crc kubenswrapper[5052]: I1203 12:46:23.597982 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f8ab3054-5998-40c1-852f-be0dbaf022e5-config\") pod \"ovn-northd-0\" (UID: \"f8ab3054-5998-40c1-852f-be0dbaf022e5\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:46:24 crc kubenswrapper[5052]: I1203 12:46:23.598111 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f8ab3054-5998-40c1-852f-be0dbaf022e5-combined-ca-bundle\") pod \"ovn-northd-0\" (UID: \"f8ab3054-5998-40c1-852f-be0dbaf022e5\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:46:24 crc kubenswrapper[5052]: I1203 12:46:23.598176 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovn-rundir\" (UniqueName: \"kubernetes.io/empty-dir/f8ab3054-5998-40c1-852f-be0dbaf022e5-ovn-rundir\") pod \"ovn-northd-0\" (UID: \"f8ab3054-5998-40c1-852f-be0dbaf022e5\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:46:24 crc kubenswrapper[5052]: I1203 12:46:23.600161 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovn-rundir\" (UniqueName: \"kubernetes.io/empty-dir/f8ab3054-5998-40c1-852f-be0dbaf022e5-ovn-rundir\") pod \"ovn-northd-0\" (UID: \"f8ab3054-5998-40c1-852f-be0dbaf022e5\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:46:24 crc kubenswrapper[5052]: I1203 12:46:23.601181 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"vencrypt-tls-certs\" (UniqueName: \"kubernetes.io/secret/9c996590-4337-477e-a236-3e834d192346-vencrypt-tls-certs\") pod \"nova-cell1-novncproxy-0\" (UID: \"9c996590-4337-477e-a236-3e834d192346\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:46:24 crc kubenswrapper[5052]: I1203 12:46:23.601243 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/f8ab3054-5998-40c1-852f-be0dbaf022e5-metrics-certs-tls-certs\") pod \"ovn-northd-0\" (UID: \"f8ab3054-5998-40c1-852f-be0dbaf022e5\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:46:24 crc kubenswrapper[5052]: E1203 12:46:23.601559 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-nova-novncproxy-cell1-vencrypt: secret "cert-nova-novncproxy-cell1-vencrypt" not found Dec 03 12:46:24 crc kubenswrapper[5052]: E1203 12:46:23.601625 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-ovn-metrics: secret "cert-ovn-metrics" not found Dec 03 12:46:24 crc kubenswrapper[5052]: E1203 12:46:23.603211 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9c996590-4337-477e-a236-3e834d192346-vencrypt-tls-certs podName:9c996590-4337-477e-a236-3e834d192346 nodeName:}" failed. No retries permitted until 2025-12-03 12:46:31.603186658 +0000 UTC m=+5182.191428422 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "vencrypt-tls-certs" (UniqueName: "kubernetes.io/secret/9c996590-4337-477e-a236-3e834d192346-vencrypt-tls-certs") pod "nova-cell1-novncproxy-0" (UID: "9c996590-4337-477e-a236-3e834d192346") : secret "cert-nova-novncproxy-cell1-vencrypt" not found Dec 03 12:46:24 crc kubenswrapper[5052]: E1203 12:46:23.603245 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f8ab3054-5998-40c1-852f-be0dbaf022e5-metrics-certs-tls-certs podName:f8ab3054-5998-40c1-852f-be0dbaf022e5 nodeName:}" failed. No retries permitted until 2025-12-03 12:46:24.10323309 +0000 UTC m=+5174.691474944 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs-tls-certs" (UniqueName: "kubernetes.io/secret/f8ab3054-5998-40c1-852f-be0dbaf022e5-metrics-certs-tls-certs") pod "ovn-northd-0" (UID: "f8ab3054-5998-40c1-852f-be0dbaf022e5") : secret "cert-ovn-metrics" not found Dec 03 12:46:24 crc kubenswrapper[5052]: I1203 12:46:23.610125 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f8ab3054-5998-40c1-852f-be0dbaf022e5-combined-ca-bundle\") pod \"ovn-northd-0\" (UID: \"f8ab3054-5998-40c1-852f-be0dbaf022e5\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:46:24 crc kubenswrapper[5052]: I1203 12:46:23.613028 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-p6qch\" (UniqueName: \"kubernetes.io/projected/f8ab3054-5998-40c1-852f-be0dbaf022e5-kube-api-access-p6qch\") pod \"ovn-northd-0\" (UID: \"f8ab3054-5998-40c1-852f-be0dbaf022e5\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:46:24 crc kubenswrapper[5052]: I1203 12:46:23.684607 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6af0a301-2483-493d-bbc8-17138d79dfea" path="/var/lib/kubelet/pods/6af0a301-2483-493d-bbc8-17138d79dfea/volumes" Dec 03 12:46:24 crc kubenswrapper[5052]: I1203 12:46:23.685555 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="818947c1-95f1-4235-8846-5886abeb57a1" path="/var/lib/kubelet/pods/818947c1-95f1-4235-8846-5886abeb57a1/volumes" Dec 03 12:46:24 crc kubenswrapper[5052]: I1203 12:46:23.705871 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovsdbserver-sb-tls-certs\" (UniqueName: \"kubernetes.io/secret/c6c9b62e-bbbc-4557-92fd-da64bd8ee0cf-ovsdbserver-sb-tls-certs\") pod \"ovsdbserver-sb-0\" (UID: \"c6c9b62e-bbbc-4557-92fd-da64bd8ee0cf\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:46:24 crc kubenswrapper[5052]: E1203 12:46:23.706029 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-ovndbcluster-sb-ovndbs: secret "cert-ovndbcluster-sb-ovndbs" not found Dec 03 12:46:24 crc kubenswrapper[5052]: E1203 12:46:23.706085 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c6c9b62e-bbbc-4557-92fd-da64bd8ee0cf-ovsdbserver-sb-tls-certs podName:c6c9b62e-bbbc-4557-92fd-da64bd8ee0cf nodeName:}" failed. No retries permitted until 2025-12-03 12:46:31.706071791 +0000 UTC m=+5182.294313555 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "ovsdbserver-sb-tls-certs" (UniqueName: "kubernetes.io/secret/c6c9b62e-bbbc-4557-92fd-da64bd8ee0cf-ovsdbserver-sb-tls-certs") pod "ovsdbserver-sb-0" (UID: "c6c9b62e-bbbc-4557-92fd-da64bd8ee0cf") : secret "cert-ovndbcluster-sb-ovndbs" not found Dec 03 12:46:24 crc kubenswrapper[5052]: I1203 12:46:23.709319 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"nova-novncproxy-tls-certs\" (UniqueName: \"kubernetes.io/secret/9c996590-4337-477e-a236-3e834d192346-nova-novncproxy-tls-certs\") pod \"nova-cell1-novncproxy-0\" (UID: \"9c996590-4337-477e-a236-3e834d192346\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:46:24 crc kubenswrapper[5052]: E1203 12:46:23.709855 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-nova-novncproxy-cell1-public-svc: secret "cert-nova-novncproxy-cell1-public-svc" not found Dec 03 12:46:24 crc kubenswrapper[5052]: E1203 12:46:23.709915 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9c996590-4337-477e-a236-3e834d192346-nova-novncproxy-tls-certs podName:9c996590-4337-477e-a236-3e834d192346 nodeName:}" failed. No retries permitted until 2025-12-03 12:46:31.70989519 +0000 UTC m=+5182.298137024 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "nova-novncproxy-tls-certs" (UniqueName: "kubernetes.io/secret/9c996590-4337-477e-a236-3e834d192346-nova-novncproxy-tls-certs") pod "nova-cell1-novncproxy-0" (UID: "9c996590-4337-477e-a236-3e834d192346") : secret "cert-nova-novncproxy-cell1-public-svc" not found Dec 03 12:46:24 crc kubenswrapper[5052]: I1203 12:46:23.916460 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovsdbserver-nb-tls-certs\" (UniqueName: \"kubernetes.io/secret/29c7d055-4f37-45a4-a608-eb73a361a86c-ovsdbserver-nb-tls-certs\") pod \"ovsdbserver-nb-0\" (UID: \"29c7d055-4f37-45a4-a608-eb73a361a86c\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:46:24 crc kubenswrapper[5052]: E1203 12:46:23.916651 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-ovndbcluster-nb-ovndbs: secret "cert-ovndbcluster-nb-ovndbs" not found Dec 03 12:46:24 crc kubenswrapper[5052]: E1203 12:46:23.916787 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/29c7d055-4f37-45a4-a608-eb73a361a86c-ovsdbserver-nb-tls-certs podName:29c7d055-4f37-45a4-a608-eb73a361a86c nodeName:}" failed. No retries permitted until 2025-12-03 12:46:31.916771257 +0000 UTC m=+5182.505013021 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "ovsdbserver-nb-tls-certs" (UniqueName: "kubernetes.io/secret/29c7d055-4f37-45a4-a608-eb73a361a86c-ovsdbserver-nb-tls-certs") pod "ovsdbserver-nb-0" (UID: "29c7d055-4f37-45a4-a608-eb73a361a86c") : secret "cert-ovndbcluster-nb-ovndbs" not found Dec 03 12:46:24 crc kubenswrapper[5052]: I1203 12:46:23.917338 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/29c7d055-4f37-45a4-a608-eb73a361a86c-metrics-certs-tls-certs\") pod \"ovsdbserver-nb-0\" (UID: \"29c7d055-4f37-45a4-a608-eb73a361a86c\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:46:24 crc kubenswrapper[5052]: E1203 12:46:23.917434 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-ovn-metrics: secret "cert-ovn-metrics" not found Dec 03 12:46:24 crc kubenswrapper[5052]: E1203 12:46:23.917470 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/29c7d055-4f37-45a4-a608-eb73a361a86c-metrics-certs-tls-certs podName:29c7d055-4f37-45a4-a608-eb73a361a86c nodeName:}" failed. No retries permitted until 2025-12-03 12:46:31.917461426 +0000 UTC m=+5182.505703190 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs-tls-certs" (UniqueName: "kubernetes.io/secret/29c7d055-4f37-45a4-a608-eb73a361a86c-metrics-certs-tls-certs") pod "ovsdbserver-nb-0" (UID: "29c7d055-4f37-45a4-a608-eb73a361a86c") : secret "cert-ovn-metrics" not found Dec 03 12:46:24 crc kubenswrapper[5052]: I1203 12:46:24.122943 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/f8ab3054-5998-40c1-852f-be0dbaf022e5-metrics-certs-tls-certs\") pod \"ovn-northd-0\" (UID: \"f8ab3054-5998-40c1-852f-be0dbaf022e5\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:46:24 crc kubenswrapper[5052]: E1203 12:46:24.123146 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-ovn-metrics: secret "cert-ovn-metrics" not found Dec 03 12:46:24 crc kubenswrapper[5052]: I1203 12:46:24.123426 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovn-northd-tls-certs\" (UniqueName: \"kubernetes.io/secret/f8ab3054-5998-40c1-852f-be0dbaf022e5-ovn-northd-tls-certs\") pod \"ovn-northd-0\" (UID: \"f8ab3054-5998-40c1-852f-be0dbaf022e5\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:46:24 crc kubenswrapper[5052]: E1203 12:46:24.123487 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f8ab3054-5998-40c1-852f-be0dbaf022e5-metrics-certs-tls-certs podName:f8ab3054-5998-40c1-852f-be0dbaf022e5 nodeName:}" failed. No retries permitted until 2025-12-03 12:46:25.123465248 +0000 UTC m=+5175.711707012 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs-tls-certs" (UniqueName: "kubernetes.io/secret/f8ab3054-5998-40c1-852f-be0dbaf022e5-metrics-certs-tls-certs") pod "ovn-northd-0" (UID: "f8ab3054-5998-40c1-852f-be0dbaf022e5") : secret "cert-ovn-metrics" not found Dec 03 12:46:24 crc kubenswrapper[5052]: E1203 12:46:24.123622 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-ovnnorthd-ovndbs: secret "cert-ovnnorthd-ovndbs" not found Dec 03 12:46:24 crc kubenswrapper[5052]: E1203 12:46:24.123687 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f8ab3054-5998-40c1-852f-be0dbaf022e5-ovn-northd-tls-certs podName:f8ab3054-5998-40c1-852f-be0dbaf022e5 nodeName:}" failed. No retries permitted until 2025-12-03 12:46:25.123670494 +0000 UTC m=+5175.711912308 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "ovn-northd-tls-certs" (UniqueName: "kubernetes.io/secret/f8ab3054-5998-40c1-852f-be0dbaf022e5-ovn-northd-tls-certs") pod "ovn-northd-0" (UID: "f8ab3054-5998-40c1-852f-be0dbaf022e5") : secret "cert-ovnnorthd-ovndbs" not found Dec 03 12:46:24 crc kubenswrapper[5052]: I1203 12:46:24.287831 5052 generic.go:334] "Generic (PLEG): container finished" podID="6d653682-52e2-4940-af66-633475bfe09a" containerID="6f8b18b5b30d4270384e6892b05ad3c64ed08fdfb147a6e3286cf16683c5b21e" exitCode=1 Dec 03 12:46:24 crc kubenswrapper[5052]: I1203 12:46:24.287903 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-conductor-0" event={"ID":"6d653682-52e2-4940-af66-633475bfe09a","Type":"ContainerDied","Data":"6f8b18b5b30d4270384e6892b05ad3c64ed08fdfb147a6e3286cf16683c5b21e"} Dec 03 12:46:24 crc kubenswrapper[5052]: I1203 12:46:24.288831 5052 scope.go:117] "RemoveContainer" containerID="6f8b18b5b30d4270384e6892b05ad3c64ed08fdfb147a6e3286cf16683c5b21e" Dec 03 12:46:24 crc kubenswrapper[5052]: I1203 12:46:24.293473 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"9481ae66-152c-41e7-9cbc-90dcb5fa6171","Type":"ContainerStarted","Data":"8d6b67218da38e98daea9b518ff99007b1a85805172d6e47d9084b2c93ac9e4e"} Dec 03 12:46:24 crc kubenswrapper[5052]: E1203 12:46:24.331607 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-kube-state-metrics-svc: secret "cert-kube-state-metrics-svc" not found Dec 03 12:46:24 crc kubenswrapper[5052]: E1203 12:46:24.331763 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/876d06e6-0187-4e3d-9183-ed2083fe5189-kube-state-metrics-tls-certs podName:876d06e6-0187-4e3d-9183-ed2083fe5189 nodeName:}" failed. No retries permitted until 2025-12-03 12:46:28.331739864 +0000 UTC m=+5178.919981708 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls-certs" (UniqueName: "kubernetes.io/secret/876d06e6-0187-4e3d-9183-ed2083fe5189-kube-state-metrics-tls-certs") pod "kube-state-metrics-0" (UID: "876d06e6-0187-4e3d-9183-ed2083fe5189") : secret "cert-kube-state-metrics-svc" not found Dec 03 12:46:24 crc kubenswrapper[5052]: I1203 12:46:24.375835 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:46:24 crc kubenswrapper[5052]: I1203 12:46:24.375955 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:46:24 crc kubenswrapper[5052]: I1203 12:46:24.470973 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/openstack-galera-0"] Dec 03 12:46:24 crc kubenswrapper[5052]: I1203 12:46:24.685179 5052 kubelet.go:2542] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:46:24 crc kubenswrapper[5052]: I1203 12:46:24.824757 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:46:25 crc kubenswrapper[5052]: I1203 12:46:25.188400 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/f8ab3054-5998-40c1-852f-be0dbaf022e5-metrics-certs-tls-certs\") pod \"ovn-northd-0\" (UID: \"f8ab3054-5998-40c1-852f-be0dbaf022e5\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:46:25 crc kubenswrapper[5052]: E1203 12:46:25.188542 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-ovn-metrics: secret "cert-ovn-metrics" not found Dec 03 12:46:25 crc kubenswrapper[5052]: E1203 12:46:25.188630 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f8ab3054-5998-40c1-852f-be0dbaf022e5-metrics-certs-tls-certs podName:f8ab3054-5998-40c1-852f-be0dbaf022e5 nodeName:}" failed. No retries permitted until 2025-12-03 12:46:27.188609498 +0000 UTC m=+5177.776851302 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs-tls-certs" (UniqueName: "kubernetes.io/secret/f8ab3054-5998-40c1-852f-be0dbaf022e5-metrics-certs-tls-certs") pod "ovn-northd-0" (UID: "f8ab3054-5998-40c1-852f-be0dbaf022e5") : secret "cert-ovn-metrics" not found Dec 03 12:46:25 crc kubenswrapper[5052]: I1203 12:46:25.188778 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovn-northd-tls-certs\" (UniqueName: \"kubernetes.io/secret/f8ab3054-5998-40c1-852f-be0dbaf022e5-ovn-northd-tls-certs\") pod \"ovn-northd-0\" (UID: \"f8ab3054-5998-40c1-852f-be0dbaf022e5\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:46:25 crc kubenswrapper[5052]: E1203 12:46:25.188982 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-ovnnorthd-ovndbs: secret "cert-ovnnorthd-ovndbs" not found Dec 03 12:46:25 crc kubenswrapper[5052]: E1203 12:46:25.189122 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f8ab3054-5998-40c1-852f-be0dbaf022e5-ovn-northd-tls-certs podName:f8ab3054-5998-40c1-852f-be0dbaf022e5 nodeName:}" failed. No retries permitted until 2025-12-03 12:46:27.189108972 +0000 UTC m=+5177.777350736 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "ovn-northd-tls-certs" (UniqueName: "kubernetes.io/secret/f8ab3054-5998-40c1-852f-be0dbaf022e5-ovn-northd-tls-certs") pod "ovn-northd-0" (UID: "f8ab3054-5998-40c1-852f-be0dbaf022e5") : secret "cert-ovnnorthd-ovndbs" not found Dec 03 12:46:25 crc kubenswrapper[5052]: I1203 12:46:25.304291 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-galera-0" event={"ID":"3c8e4e3f-f6c1-4362-b64c-b10a973723dd","Type":"ContainerStarted","Data":"18d688987807d911f1579610334ad2972251071bd9402179512a647c3ef36c10"} Dec 03 12:46:26 crc kubenswrapper[5052]: I1203 12:46:26.110845 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"memcached-tls-certs\" (UniqueName: \"kubernetes.io/secret/4d42b068-9b91-4df6-9a9c-81cad009be16-memcached-tls-certs\") pod \"memcached-0\" (UID: \"4d42b068-9b91-4df6-9a9c-81cad009be16\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:46:26 crc kubenswrapper[5052]: E1203 12:46:26.111037 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-memcached-svc: secret "cert-memcached-svc" not found Dec 03 12:46:26 crc kubenswrapper[5052]: E1203 12:46:26.111267 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4d42b068-9b91-4df6-9a9c-81cad009be16-memcached-tls-certs podName:4d42b068-9b91-4df6-9a9c-81cad009be16 nodeName:}" failed. No retries permitted until 2025-12-03 12:46:34.111243954 +0000 UTC m=+5184.699485778 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "memcached-tls-certs" (UniqueName: "kubernetes.io/secret/4d42b068-9b91-4df6-9a9c-81cad009be16-memcached-tls-certs") pod "memcached-0" (UID: "4d42b068-9b91-4df6-9a9c-81cad009be16") : secret "cert-memcached-svc" not found Dec 03 12:46:26 crc kubenswrapper[5052]: I1203 12:46:26.117968 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 12:46:26 crc kubenswrapper[5052]: E1203 12:46:26.118877 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[internal-tls-certs public-tls-certs], unattached volumes=[], failed to process volumes=[]: context canceled" pod="openstack-kuttl-tests/cinder-api-0" podUID="234278ab-b4a8-4fdf-abf9-c67cdbff1ac0" Dec 03 12:46:26 crc kubenswrapper[5052]: I1203 12:46:26.314277 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-conductor-0" event={"ID":"6d653682-52e2-4940-af66-633475bfe09a","Type":"ContainerStarted","Data":"965af26b70386d0042f76a35140b047b193a7f695cf03cf43e91184f27f9aaa1"} Dec 03 12:46:26 crc kubenswrapper[5052]: I1203 12:46:26.314423 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:46:26 crc kubenswrapper[5052]: I1203 12:46:26.315838 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/576b2af7-7647-4a45-9423-3aa752d84e75-galera-tls-certs\") pod \"openstack-cell1-galera-0\" (UID: \"576b2af7-7647-4a45-9423-3aa752d84e75\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:46:26 crc kubenswrapper[5052]: E1203 12:46:26.316025 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-galera-openstack-cell1-svc: secret "cert-galera-openstack-cell1-svc" not found Dec 03 12:46:26 crc kubenswrapper[5052]: I1203 12:46:26.316600 5052 generic.go:334] "Generic (PLEG): container finished" podID="3c8e4e3f-f6c1-4362-b64c-b10a973723dd" containerID="4ba4f75a72e1d824adaefd96f9adda9fa4cebaac7aadd0b439e99a73bbb29f1e" exitCode=0 Dec 03 12:46:26 crc kubenswrapper[5052]: I1203 12:46:26.316651 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-galera-0" event={"ID":"3c8e4e3f-f6c1-4362-b64c-b10a973723dd","Type":"ContainerDied","Data":"4ba4f75a72e1d824adaefd96f9adda9fa4cebaac7aadd0b439e99a73bbb29f1e"} Dec 03 12:46:26 crc kubenswrapper[5052]: E1203 12:46:26.316785 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/576b2af7-7647-4a45-9423-3aa752d84e75-galera-tls-certs podName:576b2af7-7647-4a45-9423-3aa752d84e75 nodeName:}" failed. No retries permitted until 2025-12-03 12:46:34.316063952 +0000 UTC m=+5184.904305776 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "galera-tls-certs" (UniqueName: "kubernetes.io/secret/576b2af7-7647-4a45-9423-3aa752d84e75-galera-tls-certs") pod "openstack-cell1-galera-0" (UID: "576b2af7-7647-4a45-9423-3aa752d84e75") : secret "cert-galera-openstack-cell1-svc" not found Dec 03 12:46:26 crc kubenswrapper[5052]: I1203 12:46:26.320083 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:46:26 crc kubenswrapper[5052]: I1203 12:46:26.320341 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"9481ae66-152c-41e7-9cbc-90dcb5fa6171","Type":"ContainerStarted","Data":"dc1f7e238c595e37880087caf4c48af2813c1fa9c81659fabdd99d813fa482a6"} Dec 03 12:46:26 crc kubenswrapper[5052]: I1203 12:46:26.320417 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"9481ae66-152c-41e7-9cbc-90dcb5fa6171","Type":"ContainerStarted","Data":"1466275e6c25f9b46a0ec21cf1af15f3c40b914e854dc9a544b1dba27f630296"} Dec 03 12:46:26 crc kubenswrapper[5052]: I1203 12:46:26.443844 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:46:26 crc kubenswrapper[5052]: I1203 12:46:26.622165 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/234278ab-b4a8-4fdf-abf9-c67cdbff1ac0-scripts\") pod \"234278ab-b4a8-4fdf-abf9-c67cdbff1ac0\" (UID: \"234278ab-b4a8-4fdf-abf9-c67cdbff1ac0\") " Dec 03 12:46:26 crc kubenswrapper[5052]: I1203 12:46:26.622379 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/234278ab-b4a8-4fdf-abf9-c67cdbff1ac0-config-data-custom\") pod \"234278ab-b4a8-4fdf-abf9-c67cdbff1ac0\" (UID: \"234278ab-b4a8-4fdf-abf9-c67cdbff1ac0\") " Dec 03 12:46:26 crc kubenswrapper[5052]: I1203 12:46:26.622429 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/234278ab-b4a8-4fdf-abf9-c67cdbff1ac0-config-data\") pod \"234278ab-b4a8-4fdf-abf9-c67cdbff1ac0\" (UID: \"234278ab-b4a8-4fdf-abf9-c67cdbff1ac0\") " Dec 03 12:46:26 crc kubenswrapper[5052]: I1203 12:46:26.622498 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j52n8\" (UniqueName: \"kubernetes.io/projected/234278ab-b4a8-4fdf-abf9-c67cdbff1ac0-kube-api-access-j52n8\") pod \"234278ab-b4a8-4fdf-abf9-c67cdbff1ac0\" (UID: \"234278ab-b4a8-4fdf-abf9-c67cdbff1ac0\") " Dec 03 12:46:26 crc kubenswrapper[5052]: I1203 12:46:26.622557 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/234278ab-b4a8-4fdf-abf9-c67cdbff1ac0-etc-machine-id\") pod \"234278ab-b4a8-4fdf-abf9-c67cdbff1ac0\" (UID: \"234278ab-b4a8-4fdf-abf9-c67cdbff1ac0\") " Dec 03 12:46:26 crc kubenswrapper[5052]: I1203 12:46:26.622620 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/234278ab-b4a8-4fdf-abf9-c67cdbff1ac0-combined-ca-bundle\") pod \"234278ab-b4a8-4fdf-abf9-c67cdbff1ac0\" (UID: \"234278ab-b4a8-4fdf-abf9-c67cdbff1ac0\") " Dec 03 12:46:26 crc kubenswrapper[5052]: I1203 12:46:26.622680 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/234278ab-b4a8-4fdf-abf9-c67cdbff1ac0-logs\") pod \"234278ab-b4a8-4fdf-abf9-c67cdbff1ac0\" (UID: \"234278ab-b4a8-4fdf-abf9-c67cdbff1ac0\") " Dec 03 12:46:26 crc kubenswrapper[5052]: I1203 12:46:26.623674 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/234278ab-b4a8-4fdf-abf9-c67cdbff1ac0-logs" (OuterVolumeSpecName: "logs") pod "234278ab-b4a8-4fdf-abf9-c67cdbff1ac0" (UID: "234278ab-b4a8-4fdf-abf9-c67cdbff1ac0"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:46:26 crc kubenswrapper[5052]: I1203 12:46:26.623713 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/234278ab-b4a8-4fdf-abf9-c67cdbff1ac0-etc-machine-id" (OuterVolumeSpecName: "etc-machine-id") pod "234278ab-b4a8-4fdf-abf9-c67cdbff1ac0" (UID: "234278ab-b4a8-4fdf-abf9-c67cdbff1ac0"). InnerVolumeSpecName "etc-machine-id". PluginName "kubernetes.io/host-path", VolumeGidValue "" Dec 03 12:46:26 crc kubenswrapper[5052]: I1203 12:46:26.626737 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/234278ab-b4a8-4fdf-abf9-c67cdbff1ac0-kube-api-access-j52n8" (OuterVolumeSpecName: "kube-api-access-j52n8") pod "234278ab-b4a8-4fdf-abf9-c67cdbff1ac0" (UID: "234278ab-b4a8-4fdf-abf9-c67cdbff1ac0"). InnerVolumeSpecName "kube-api-access-j52n8". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:46:26 crc kubenswrapper[5052]: I1203 12:46:26.626791 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/234278ab-b4a8-4fdf-abf9-c67cdbff1ac0-scripts" (OuterVolumeSpecName: "scripts") pod "234278ab-b4a8-4fdf-abf9-c67cdbff1ac0" (UID: "234278ab-b4a8-4fdf-abf9-c67cdbff1ac0"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:46:26 crc kubenswrapper[5052]: I1203 12:46:26.630646 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/234278ab-b4a8-4fdf-abf9-c67cdbff1ac0-config-data-custom" (OuterVolumeSpecName: "config-data-custom") pod "234278ab-b4a8-4fdf-abf9-c67cdbff1ac0" (UID: "234278ab-b4a8-4fdf-abf9-c67cdbff1ac0"). InnerVolumeSpecName "config-data-custom". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:46:26 crc kubenswrapper[5052]: I1203 12:46:26.641645 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/234278ab-b4a8-4fdf-abf9-c67cdbff1ac0-config-data" (OuterVolumeSpecName: "config-data") pod "234278ab-b4a8-4fdf-abf9-c67cdbff1ac0" (UID: "234278ab-b4a8-4fdf-abf9-c67cdbff1ac0"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:46:26 crc kubenswrapper[5052]: I1203 12:46:26.641748 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/234278ab-b4a8-4fdf-abf9-c67cdbff1ac0-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "234278ab-b4a8-4fdf-abf9-c67cdbff1ac0" (UID: "234278ab-b4a8-4fdf-abf9-c67cdbff1ac0"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:46:26 crc kubenswrapper[5052]: I1203 12:46:26.673651 5052 scope.go:117] "RemoveContainer" containerID="2aed1c760a326f3d008799e902b1093313df3764a57c879f0b3686410127d37d" Dec 03 12:46:26 crc kubenswrapper[5052]: E1203 12:46:26.673933 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 12:46:26 crc kubenswrapper[5052]: I1203 12:46:26.725652 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/234278ab-b4a8-4fdf-abf9-c67cdbff1ac0-config-data-custom\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:26 crc kubenswrapper[5052]: I1203 12:46:26.725712 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/234278ab-b4a8-4fdf-abf9-c67cdbff1ac0-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:26 crc kubenswrapper[5052]: I1203 12:46:26.725727 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-j52n8\" (UniqueName: \"kubernetes.io/projected/234278ab-b4a8-4fdf-abf9-c67cdbff1ac0-kube-api-access-j52n8\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:26 crc kubenswrapper[5052]: I1203 12:46:26.725742 5052 reconciler_common.go:293] "Volume detached for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/234278ab-b4a8-4fdf-abf9-c67cdbff1ac0-etc-machine-id\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:26 crc kubenswrapper[5052]: I1203 12:46:26.725754 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/234278ab-b4a8-4fdf-abf9-c67cdbff1ac0-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:26 crc kubenswrapper[5052]: I1203 12:46:26.725768 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/234278ab-b4a8-4fdf-abf9-c67cdbff1ac0-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:26 crc kubenswrapper[5052]: I1203 12:46:26.725872 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/234278ab-b4a8-4fdf-abf9-c67cdbff1ac0-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:26 crc kubenswrapper[5052]: I1203 12:46:26.827075 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/ed7f9920-a1e4-437b-8074-a30d0fe4450c-internal-tls-certs\") pod \"nova-api-0\" (UID: \"ed7f9920-a1e4-437b-8074-a30d0fe4450c\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:46:26 crc kubenswrapper[5052]: E1203 12:46:26.827286 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-nova-internal-svc: secret "cert-nova-internal-svc" not found Dec 03 12:46:26 crc kubenswrapper[5052]: I1203 12:46:26.827456 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/ed7f9920-a1e4-437b-8074-a30d0fe4450c-public-tls-certs\") pod \"nova-api-0\" (UID: \"ed7f9920-a1e4-437b-8074-a30d0fe4450c\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:46:26 crc kubenswrapper[5052]: E1203 12:46:26.827683 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-swift-internal-svc: secret "cert-swift-internal-svc" not found Dec 03 12:46:26 crc kubenswrapper[5052]: E1203 12:46:26.827730 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-swift-public-svc: secret "cert-swift-public-svc" not found Dec 03 12:46:26 crc kubenswrapper[5052]: E1203 12:46:26.827742 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/242be027-6461-4aea-ae7c-e52ca996b2ea-internal-tls-certs podName:242be027-6461-4aea-ae7c-e52ca996b2ea nodeName:}" failed. No retries permitted until 2025-12-03 12:46:34.827721504 +0000 UTC m=+5185.415963308 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "internal-tls-certs" (UniqueName: "kubernetes.io/secret/242be027-6461-4aea-ae7c-e52ca996b2ea-internal-tls-certs") pod "swift-proxy-dd4cf57c8-x4c45" (UID: "242be027-6461-4aea-ae7c-e52ca996b2ea") : secret "cert-swift-internal-svc" not found Dec 03 12:46:26 crc kubenswrapper[5052]: E1203 12:46:26.827794 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-nova-public-svc: secret "cert-nova-public-svc" not found Dec 03 12:46:26 crc kubenswrapper[5052]: E1203 12:46:26.827822 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ed7f9920-a1e4-437b-8074-a30d0fe4450c-public-tls-certs podName:ed7f9920-a1e4-437b-8074-a30d0fe4450c nodeName:}" failed. No retries permitted until 2025-12-03 12:46:34.827812017 +0000 UTC m=+5185.416053871 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "public-tls-certs" (UniqueName: "kubernetes.io/secret/ed7f9920-a1e4-437b-8074-a30d0fe4450c-public-tls-certs") pod "nova-api-0" (UID: "ed7f9920-a1e4-437b-8074-a30d0fe4450c") : secret "cert-nova-public-svc" not found Dec 03 12:46:26 crc kubenswrapper[5052]: E1203 12:46:26.827847 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/242be027-6461-4aea-ae7c-e52ca996b2ea-public-tls-certs podName:242be027-6461-4aea-ae7c-e52ca996b2ea nodeName:}" failed. No retries permitted until 2025-12-03 12:46:34.827839958 +0000 UTC m=+5185.416081832 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "public-tls-certs" (UniqueName: "kubernetes.io/secret/242be027-6461-4aea-ae7c-e52ca996b2ea-public-tls-certs") pod "swift-proxy-dd4cf57c8-x4c45" (UID: "242be027-6461-4aea-ae7c-e52ca996b2ea") : secret "cert-swift-public-svc" not found Dec 03 12:46:26 crc kubenswrapper[5052]: E1203 12:46:26.827876 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ed7f9920-a1e4-437b-8074-a30d0fe4450c-internal-tls-certs podName:ed7f9920-a1e4-437b-8074-a30d0fe4450c nodeName:}" failed. No retries permitted until 2025-12-03 12:46:34.827867019 +0000 UTC m=+5185.416108853 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "internal-tls-certs" (UniqueName: "kubernetes.io/secret/ed7f9920-a1e4-437b-8074-a30d0fe4450c-internal-tls-certs") pod "nova-api-0" (UID: "ed7f9920-a1e4-437b-8074-a30d0fe4450c") : secret "cert-nova-internal-svc" not found Dec 03 12:46:26 crc kubenswrapper[5052]: E1203 12:46:26.930192 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-placement-public-svc: secret "cert-placement-public-svc" not found Dec 03 12:46:26 crc kubenswrapper[5052]: E1203 12:46:26.930280 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/32f420c2-aa7e-44c8-a63a-7654aa3e6e34-public-tls-certs podName:32f420c2-aa7e-44c8-a63a-7654aa3e6e34 nodeName:}" failed. No retries permitted until 2025-12-03 12:46:34.930262517 +0000 UTC m=+5185.518504281 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "public-tls-certs" (UniqueName: "kubernetes.io/secret/32f420c2-aa7e-44c8-a63a-7654aa3e6e34-public-tls-certs") pod "placement-bbcfbbc9d-5q6gx" (UID: "32f420c2-aa7e-44c8-a63a-7654aa3e6e34") : secret "cert-placement-public-svc" not found Dec 03 12:46:26 crc kubenswrapper[5052]: E1203 12:46:26.930687 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-placement-internal-svc: secret "cert-placement-internal-svc" not found Dec 03 12:46:26 crc kubenswrapper[5052]: E1203 12:46:26.930730 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/32f420c2-aa7e-44c8-a63a-7654aa3e6e34-internal-tls-certs podName:32f420c2-aa7e-44c8-a63a-7654aa3e6e34 nodeName:}" failed. No retries permitted until 2025-12-03 12:46:34.9307197 +0000 UTC m=+5185.518961474 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "internal-tls-certs" (UniqueName: "kubernetes.io/secret/32f420c2-aa7e-44c8-a63a-7654aa3e6e34-internal-tls-certs") pod "placement-bbcfbbc9d-5q6gx" (UID: "32f420c2-aa7e-44c8-a63a-7654aa3e6e34") : secret "cert-placement-internal-svc" not found Dec 03 12:46:27 crc kubenswrapper[5052]: I1203 12:46:27.031931 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/4c9f4da8-f1ae-4815-8ee2-0e873fb07d62-public-tls-certs\") pod \"glance-default-external-api-0\" (UID: \"4c9f4da8-f1ae-4815-8ee2-0e873fb07d62\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:46:27 crc kubenswrapper[5052]: I1203 12:46:27.036859 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/4c9f4da8-f1ae-4815-8ee2-0e873fb07d62-public-tls-certs\") pod \"glance-default-external-api-0\" (UID: \"4c9f4da8-f1ae-4815-8ee2-0e873fb07d62\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:46:27 crc kubenswrapper[5052]: I1203 12:46:27.133742 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/bf8b9b9c-b8f9-4c93-8112-43913950fcdf-internal-tls-certs\") pod \"glance-default-internal-api-0\" (UID: \"bf8b9b9c-b8f9-4c93-8112-43913950fcdf\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:46:27 crc kubenswrapper[5052]: E1203 12:46:27.134158 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-glance-default-internal-svc: secret "cert-glance-default-internal-svc" not found Dec 03 12:46:27 crc kubenswrapper[5052]: E1203 12:46:27.134224 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bf8b9b9c-b8f9-4c93-8112-43913950fcdf-internal-tls-certs podName:bf8b9b9c-b8f9-4c93-8112-43913950fcdf nodeName:}" failed. No retries permitted until 2025-12-03 12:46:35.134205999 +0000 UTC m=+5185.722447763 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "internal-tls-certs" (UniqueName: "kubernetes.io/secret/bf8b9b9c-b8f9-4c93-8112-43913950fcdf-internal-tls-certs") pod "glance-default-internal-api-0" (UID: "bf8b9b9c-b8f9-4c93-8112-43913950fcdf") : secret "cert-glance-default-internal-svc" not found Dec 03 12:46:27 crc kubenswrapper[5052]: I1203 12:46:27.185931 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:46:27 crc kubenswrapper[5052]: I1203 12:46:27.237541 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/234278ab-b4a8-4fdf-abf9-c67cdbff1ac0-internal-tls-certs\") pod \"cinder-api-0\" (UID: \"234278ab-b4a8-4fdf-abf9-c67cdbff1ac0\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:46:27 crc kubenswrapper[5052]: I1203 12:46:27.237626 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/f8ab3054-5998-40c1-852f-be0dbaf022e5-metrics-certs-tls-certs\") pod \"ovn-northd-0\" (UID: \"f8ab3054-5998-40c1-852f-be0dbaf022e5\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:46:27 crc kubenswrapper[5052]: I1203 12:46:27.237738 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovn-northd-tls-certs\" (UniqueName: \"kubernetes.io/secret/f8ab3054-5998-40c1-852f-be0dbaf022e5-ovn-northd-tls-certs\") pod \"ovn-northd-0\" (UID: \"f8ab3054-5998-40c1-852f-be0dbaf022e5\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:46:27 crc kubenswrapper[5052]: E1203 12:46:27.237781 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-ovn-metrics: secret "cert-ovn-metrics" not found Dec 03 12:46:27 crc kubenswrapper[5052]: E1203 12:46:27.237876 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f8ab3054-5998-40c1-852f-be0dbaf022e5-metrics-certs-tls-certs podName:f8ab3054-5998-40c1-852f-be0dbaf022e5 nodeName:}" failed. No retries permitted until 2025-12-03 12:46:31.237852574 +0000 UTC m=+5181.826094378 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs-tls-certs" (UniqueName: "kubernetes.io/secret/f8ab3054-5998-40c1-852f-be0dbaf022e5-metrics-certs-tls-certs") pod "ovn-northd-0" (UID: "f8ab3054-5998-40c1-852f-be0dbaf022e5") : secret "cert-ovn-metrics" not found Dec 03 12:46:27 crc kubenswrapper[5052]: E1203 12:46:27.237955 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-ovnnorthd-ovndbs: secret "cert-ovnnorthd-ovndbs" not found Dec 03 12:46:27 crc kubenswrapper[5052]: E1203 12:46:27.238040 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f8ab3054-5998-40c1-852f-be0dbaf022e5-ovn-northd-tls-certs podName:f8ab3054-5998-40c1-852f-be0dbaf022e5 nodeName:}" failed. No retries permitted until 2025-12-03 12:46:31.238020098 +0000 UTC m=+5181.826261852 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "ovn-northd-tls-certs" (UniqueName: "kubernetes.io/secret/f8ab3054-5998-40c1-852f-be0dbaf022e5-ovn-northd-tls-certs") pod "ovn-northd-0" (UID: "f8ab3054-5998-40c1-852f-be0dbaf022e5") : secret "cert-ovnnorthd-ovndbs" not found Dec 03 12:46:27 crc kubenswrapper[5052]: I1203 12:46:27.237962 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/234278ab-b4a8-4fdf-abf9-c67cdbff1ac0-public-tls-certs\") pod \"cinder-api-0\" (UID: \"234278ab-b4a8-4fdf-abf9-c67cdbff1ac0\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:46:27 crc kubenswrapper[5052]: I1203 12:46:27.241352 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/234278ab-b4a8-4fdf-abf9-c67cdbff1ac0-public-tls-certs\") pod \"cinder-api-0\" (UID: \"234278ab-b4a8-4fdf-abf9-c67cdbff1ac0\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:46:27 crc kubenswrapper[5052]: I1203 12:46:27.242106 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/234278ab-b4a8-4fdf-abf9-c67cdbff1ac0-internal-tls-certs\") pod \"cinder-api-0\" (UID: \"234278ab-b4a8-4fdf-abf9-c67cdbff1ac0\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:46:27 crc kubenswrapper[5052]: I1203 12:46:27.345779 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-galera-0" event={"ID":"3c8e4e3f-f6c1-4362-b64c-b10a973723dd","Type":"ContainerStarted","Data":"86f66b1381c35478a52f56b7c8ce2c98cdc37bae9cd163bbea7b6f45c6c705b9"} Dec 03 12:46:27 crc kubenswrapper[5052]: I1203 12:46:27.345967 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:46:27 crc kubenswrapper[5052]: I1203 12:46:27.380432 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/openstack-galera-0" podStartSLOduration=5.38041107 podStartE2EDuration="5.38041107s" podCreationTimestamp="2025-12-03 12:46:22 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:46:27.374562923 +0000 UTC m=+5177.962804687" watchObservedRunningTime="2025-12-03 12:46:27.38041107 +0000 UTC m=+5177.968652834" Dec 03 12:46:27 crc kubenswrapper[5052]: I1203 12:46:27.442483 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/234278ab-b4a8-4fdf-abf9-c67cdbff1ac0-internal-tls-certs\") pod \"234278ab-b4a8-4fdf-abf9-c67cdbff1ac0\" (UID: \"234278ab-b4a8-4fdf-abf9-c67cdbff1ac0\") " Dec 03 12:46:27 crc kubenswrapper[5052]: I1203 12:46:27.442745 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/234278ab-b4a8-4fdf-abf9-c67cdbff1ac0-public-tls-certs\") pod \"234278ab-b4a8-4fdf-abf9-c67cdbff1ac0\" (UID: \"234278ab-b4a8-4fdf-abf9-c67cdbff1ac0\") " Dec 03 12:46:27 crc kubenswrapper[5052]: I1203 12:46:27.446812 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/234278ab-b4a8-4fdf-abf9-c67cdbff1ac0-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "234278ab-b4a8-4fdf-abf9-c67cdbff1ac0" (UID: "234278ab-b4a8-4fdf-abf9-c67cdbff1ac0"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:46:27 crc kubenswrapper[5052]: I1203 12:46:27.452790 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/234278ab-b4a8-4fdf-abf9-c67cdbff1ac0-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "234278ab-b4a8-4fdf-abf9-c67cdbff1ac0" (UID: "234278ab-b4a8-4fdf-abf9-c67cdbff1ac0"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:46:27 crc kubenswrapper[5052]: I1203 12:46:27.546296 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/234278ab-b4a8-4fdf-abf9-c67cdbff1ac0-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:27 crc kubenswrapper[5052]: I1203 12:46:27.546334 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/234278ab-b4a8-4fdf-abf9-c67cdbff1ac0-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:27 crc kubenswrapper[5052]: W1203 12:46:27.637598 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4c9f4da8_f1ae_4815_8ee2_0e873fb07d62.slice/crio-5508cf0583ceb9a8b712a6f4c574d921b80de874086d91a5e31bb2abd237c9b1 WatchSource:0}: Error finding container 5508cf0583ceb9a8b712a6f4c574d921b80de874086d91a5e31bb2abd237c9b1: Status 404 returned error can't find the container with id 5508cf0583ceb9a8b712a6f4c574d921b80de874086d91a5e31bb2abd237c9b1 Dec 03 12:46:27 crc kubenswrapper[5052]: I1203 12:46:27.646567 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:46:27 crc kubenswrapper[5052]: I1203 12:46:27.723403 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 12:46:27 crc kubenswrapper[5052]: I1203 12:46:27.735670 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 12:46:27 crc kubenswrapper[5052]: I1203 12:46:27.748928 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 12:46:27 crc kubenswrapper[5052]: I1203 12:46:27.751072 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:46:27 crc kubenswrapper[5052]: I1203 12:46:27.755319 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cinder-api-config-data" Dec 03 12:46:27 crc kubenswrapper[5052]: I1203 12:46:27.755597 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-cinder-public-svc" Dec 03 12:46:27 crc kubenswrapper[5052]: I1203 12:46:27.757782 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-cinder-internal-svc" Dec 03 12:46:27 crc kubenswrapper[5052]: I1203 12:46:27.772204 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 12:46:27 crc kubenswrapper[5052]: I1203 12:46:27.852272 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/9bb8adcd-621e-489c-8cec-c8965062b1c9-logs\") pod \"cinder-api-0\" (UID: \"9bb8adcd-621e-489c-8cec-c8965062b1c9\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:46:27 crc kubenswrapper[5052]: I1203 12:46:27.852570 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/9bb8adcd-621e-489c-8cec-c8965062b1c9-scripts\") pod \"cinder-api-0\" (UID: \"9bb8adcd-621e-489c-8cec-c8965062b1c9\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:46:27 crc kubenswrapper[5052]: I1203 12:46:27.852660 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/9bb8adcd-621e-489c-8cec-c8965062b1c9-config-data\") pod \"cinder-api-0\" (UID: \"9bb8adcd-621e-489c-8cec-c8965062b1c9\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:46:27 crc kubenswrapper[5052]: I1203 12:46:27.852874 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r68tw\" (UniqueName: \"kubernetes.io/projected/9bb8adcd-621e-489c-8cec-c8965062b1c9-kube-api-access-r68tw\") pod \"cinder-api-0\" (UID: \"9bb8adcd-621e-489c-8cec-c8965062b1c9\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:46:27 crc kubenswrapper[5052]: I1203 12:46:27.853170 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/9bb8adcd-621e-489c-8cec-c8965062b1c9-etc-machine-id\") pod \"cinder-api-0\" (UID: \"9bb8adcd-621e-489c-8cec-c8965062b1c9\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:46:27 crc kubenswrapper[5052]: I1203 12:46:27.853297 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/9bb8adcd-621e-489c-8cec-c8965062b1c9-config-data-custom\") pod \"cinder-api-0\" (UID: \"9bb8adcd-621e-489c-8cec-c8965062b1c9\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:46:27 crc kubenswrapper[5052]: I1203 12:46:27.853439 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/9bb8adcd-621e-489c-8cec-c8965062b1c9-internal-tls-certs\") pod \"cinder-api-0\" (UID: \"9bb8adcd-621e-489c-8cec-c8965062b1c9\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:46:27 crc kubenswrapper[5052]: I1203 12:46:27.853605 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/9bb8adcd-621e-489c-8cec-c8965062b1c9-public-tls-certs\") pod \"cinder-api-0\" (UID: \"9bb8adcd-621e-489c-8cec-c8965062b1c9\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:46:27 crc kubenswrapper[5052]: E1203 12:46:27.853660 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-barbican-internal-svc: secret "cert-barbican-internal-svc" not found Dec 03 12:46:27 crc kubenswrapper[5052]: E1203 12:46:27.854046 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5b63f869-c95a-4279-b542-3ebbec1a1fd5-internal-tls-certs podName:5b63f869-c95a-4279-b542-3ebbec1a1fd5 nodeName:}" failed. No retries permitted until 2025-12-03 12:46:28.354031355 +0000 UTC m=+5178.942273119 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "internal-tls-certs" (UniqueName: "kubernetes.io/secret/5b63f869-c95a-4279-b542-3ebbec1a1fd5-internal-tls-certs") pod "barbican-api-6d86f7b884-lf4pd" (UID: "5b63f869-c95a-4279-b542-3ebbec1a1fd5") : secret "cert-barbican-internal-svc" not found Dec 03 12:46:27 crc kubenswrapper[5052]: I1203 12:46:27.854398 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9bb8adcd-621e-489c-8cec-c8965062b1c9-combined-ca-bundle\") pod \"cinder-api-0\" (UID: \"9bb8adcd-621e-489c-8cec-c8965062b1c9\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:46:27 crc kubenswrapper[5052]: I1203 12:46:27.956214 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/c069aee2-4cb6-4c8a-bbe5-949ab0b48331-public-tls-certs\") pod \"barbican-api-bcf49b784-jnkdb\" (UID: \"c069aee2-4cb6-4c8a-bbe5-949ab0b48331\") " pod="openstack-kuttl-tests/barbican-api-bcf49b784-jnkdb" Dec 03 12:46:27 crc kubenswrapper[5052]: I1203 12:46:27.956317 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-r68tw\" (UniqueName: \"kubernetes.io/projected/9bb8adcd-621e-489c-8cec-c8965062b1c9-kube-api-access-r68tw\") pod \"cinder-api-0\" (UID: \"9bb8adcd-621e-489c-8cec-c8965062b1c9\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:46:27 crc kubenswrapper[5052]: I1203 12:46:27.956420 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/9bb8adcd-621e-489c-8cec-c8965062b1c9-etc-machine-id\") pod \"cinder-api-0\" (UID: \"9bb8adcd-621e-489c-8cec-c8965062b1c9\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:46:27 crc kubenswrapper[5052]: I1203 12:46:27.956538 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/9bb8adcd-621e-489c-8cec-c8965062b1c9-config-data-custom\") pod \"cinder-api-0\" (UID: \"9bb8adcd-621e-489c-8cec-c8965062b1c9\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:46:27 crc kubenswrapper[5052]: I1203 12:46:27.956567 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/c069aee2-4cb6-4c8a-bbe5-949ab0b48331-internal-tls-certs\") pod \"barbican-api-bcf49b784-jnkdb\" (UID: \"c069aee2-4cb6-4c8a-bbe5-949ab0b48331\") " pod="openstack-kuttl-tests/barbican-api-bcf49b784-jnkdb" Dec 03 12:46:27 crc kubenswrapper[5052]: I1203 12:46:27.956616 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/9bb8adcd-621e-489c-8cec-c8965062b1c9-internal-tls-certs\") pod \"cinder-api-0\" (UID: \"9bb8adcd-621e-489c-8cec-c8965062b1c9\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:46:27 crc kubenswrapper[5052]: I1203 12:46:27.956693 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/9bb8adcd-621e-489c-8cec-c8965062b1c9-public-tls-certs\") pod \"cinder-api-0\" (UID: \"9bb8adcd-621e-489c-8cec-c8965062b1c9\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:46:27 crc kubenswrapper[5052]: I1203 12:46:27.956747 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9bb8adcd-621e-489c-8cec-c8965062b1c9-combined-ca-bundle\") pod \"cinder-api-0\" (UID: \"9bb8adcd-621e-489c-8cec-c8965062b1c9\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:46:27 crc kubenswrapper[5052]: I1203 12:46:27.956804 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/9bb8adcd-621e-489c-8cec-c8965062b1c9-logs\") pod \"cinder-api-0\" (UID: \"9bb8adcd-621e-489c-8cec-c8965062b1c9\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:46:27 crc kubenswrapper[5052]: I1203 12:46:27.956851 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/9bb8adcd-621e-489c-8cec-c8965062b1c9-scripts\") pod \"cinder-api-0\" (UID: \"9bb8adcd-621e-489c-8cec-c8965062b1c9\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:46:27 crc kubenswrapper[5052]: I1203 12:46:27.956886 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/9bb8adcd-621e-489c-8cec-c8965062b1c9-config-data\") pod \"cinder-api-0\" (UID: \"9bb8adcd-621e-489c-8cec-c8965062b1c9\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:46:27 crc kubenswrapper[5052]: I1203 12:46:27.957140 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/9bb8adcd-621e-489c-8cec-c8965062b1c9-etc-machine-id\") pod \"cinder-api-0\" (UID: \"9bb8adcd-621e-489c-8cec-c8965062b1c9\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:46:27 crc kubenswrapper[5052]: E1203 12:46:27.957261 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-barbican-internal-svc: secret "cert-barbican-internal-svc" not found Dec 03 12:46:27 crc kubenswrapper[5052]: E1203 12:46:27.957320 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c069aee2-4cb6-4c8a-bbe5-949ab0b48331-internal-tls-certs podName:c069aee2-4cb6-4c8a-bbe5-949ab0b48331 nodeName:}" failed. No retries permitted until 2025-12-03 12:46:43.957302819 +0000 UTC m=+5194.545544583 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "internal-tls-certs" (UniqueName: "kubernetes.io/secret/c069aee2-4cb6-4c8a-bbe5-949ab0b48331-internal-tls-certs") pod "barbican-api-bcf49b784-jnkdb" (UID: "c069aee2-4cb6-4c8a-bbe5-949ab0b48331") : secret "cert-barbican-internal-svc" not found Dec 03 12:46:27 crc kubenswrapper[5052]: I1203 12:46:27.957830 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/9bb8adcd-621e-489c-8cec-c8965062b1c9-logs\") pod \"cinder-api-0\" (UID: \"9bb8adcd-621e-489c-8cec-c8965062b1c9\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:46:27 crc kubenswrapper[5052]: I1203 12:46:27.961556 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/9bb8adcd-621e-489c-8cec-c8965062b1c9-scripts\") pod \"cinder-api-0\" (UID: \"9bb8adcd-621e-489c-8cec-c8965062b1c9\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:46:27 crc kubenswrapper[5052]: I1203 12:46:27.961673 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/c069aee2-4cb6-4c8a-bbe5-949ab0b48331-public-tls-certs\") pod \"barbican-api-bcf49b784-jnkdb\" (UID: \"c069aee2-4cb6-4c8a-bbe5-949ab0b48331\") " pod="openstack-kuttl-tests/barbican-api-bcf49b784-jnkdb" Dec 03 12:46:27 crc kubenswrapper[5052]: I1203 12:46:27.962013 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/9bb8adcd-621e-489c-8cec-c8965062b1c9-internal-tls-certs\") pod \"cinder-api-0\" (UID: \"9bb8adcd-621e-489c-8cec-c8965062b1c9\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:46:27 crc kubenswrapper[5052]: I1203 12:46:27.962228 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/9bb8adcd-621e-489c-8cec-c8965062b1c9-config-data-custom\") pod \"cinder-api-0\" (UID: \"9bb8adcd-621e-489c-8cec-c8965062b1c9\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:46:27 crc kubenswrapper[5052]: I1203 12:46:27.963230 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/9bb8adcd-621e-489c-8cec-c8965062b1c9-public-tls-certs\") pod \"cinder-api-0\" (UID: \"9bb8adcd-621e-489c-8cec-c8965062b1c9\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:46:27 crc kubenswrapper[5052]: I1203 12:46:27.963803 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/9bb8adcd-621e-489c-8cec-c8965062b1c9-config-data\") pod \"cinder-api-0\" (UID: \"9bb8adcd-621e-489c-8cec-c8965062b1c9\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:46:27 crc kubenswrapper[5052]: I1203 12:46:27.964533 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9bb8adcd-621e-489c-8cec-c8965062b1c9-combined-ca-bundle\") pod \"cinder-api-0\" (UID: \"9bb8adcd-621e-489c-8cec-c8965062b1c9\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:46:27 crc kubenswrapper[5052]: I1203 12:46:27.973809 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-r68tw\" (UniqueName: \"kubernetes.io/projected/9bb8adcd-621e-489c-8cec-c8965062b1c9-kube-api-access-r68tw\") pod \"cinder-api-0\" (UID: \"9bb8adcd-621e-489c-8cec-c8965062b1c9\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:46:28 crc kubenswrapper[5052]: I1203 12:46:28.085852 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:46:28 crc kubenswrapper[5052]: I1203 12:46:28.161168 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/fa1c223d-82fa-488c-ac48-e88bce36a839-public-tls-certs\") pod \"keystone-687ff6fcfc-9vwb2\" (UID: \"fa1c223d-82fa-488c-ac48-e88bce36a839\") " pod="openstack-kuttl-tests/keystone-687ff6fcfc-9vwb2" Dec 03 12:46:28 crc kubenswrapper[5052]: I1203 12:46:28.161278 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/fa1c223d-82fa-488c-ac48-e88bce36a839-internal-tls-certs\") pod \"keystone-687ff6fcfc-9vwb2\" (UID: \"fa1c223d-82fa-488c-ac48-e88bce36a839\") " pod="openstack-kuttl-tests/keystone-687ff6fcfc-9vwb2" Dec 03 12:46:28 crc kubenswrapper[5052]: E1203 12:46:28.161753 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-keystone-public-svc: secret "cert-keystone-public-svc" not found Dec 03 12:46:28 crc kubenswrapper[5052]: E1203 12:46:28.161843 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fa1c223d-82fa-488c-ac48-e88bce36a839-public-tls-certs podName:fa1c223d-82fa-488c-ac48-e88bce36a839 nodeName:}" failed. No retries permitted until 2025-12-03 12:46:44.161822508 +0000 UTC m=+5194.750064322 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "public-tls-certs" (UniqueName: "kubernetes.io/secret/fa1c223d-82fa-488c-ac48-e88bce36a839-public-tls-certs") pod "keystone-687ff6fcfc-9vwb2" (UID: "fa1c223d-82fa-488c-ac48-e88bce36a839") : secret "cert-keystone-public-svc" not found Dec 03 12:46:28 crc kubenswrapper[5052]: E1203 12:46:28.161766 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-keystone-internal-svc: secret "cert-keystone-internal-svc" not found Dec 03 12:46:28 crc kubenswrapper[5052]: E1203 12:46:28.161967 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fa1c223d-82fa-488c-ac48-e88bce36a839-internal-tls-certs podName:fa1c223d-82fa-488c-ac48-e88bce36a839 nodeName:}" failed. No retries permitted until 2025-12-03 12:46:44.161948301 +0000 UTC m=+5194.750190105 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "internal-tls-certs" (UniqueName: "kubernetes.io/secret/fa1c223d-82fa-488c-ac48-e88bce36a839-internal-tls-certs") pod "keystone-687ff6fcfc-9vwb2" (UID: "fa1c223d-82fa-488c-ac48-e88bce36a839") : secret "cert-keystone-internal-svc" not found Dec 03 12:46:28 crc kubenswrapper[5052]: E1203 12:46:28.365815 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-barbican-internal-svc: secret "cert-barbican-internal-svc" not found Dec 03 12:46:28 crc kubenswrapper[5052]: E1203 12:46:28.366134 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5b63f869-c95a-4279-b542-3ebbec1a1fd5-internal-tls-certs podName:5b63f869-c95a-4279-b542-3ebbec1a1fd5 nodeName:}" failed. No retries permitted until 2025-12-03 12:46:29.36612079 +0000 UTC m=+5179.954362554 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "internal-tls-certs" (UniqueName: "kubernetes.io/secret/5b63f869-c95a-4279-b542-3ebbec1a1fd5-internal-tls-certs") pod "barbican-api-6d86f7b884-lf4pd" (UID: "5b63f869-c95a-4279-b542-3ebbec1a1fd5") : secret "cert-barbican-internal-svc" not found Dec 03 12:46:28 crc kubenswrapper[5052]: E1203 12:46:28.366088 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-kube-state-metrics-svc: secret "cert-kube-state-metrics-svc" not found Dec 03 12:46:28 crc kubenswrapper[5052]: E1203 12:46:28.366162 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/876d06e6-0187-4e3d-9183-ed2083fe5189-kube-state-metrics-tls-certs podName:876d06e6-0187-4e3d-9183-ed2083fe5189 nodeName:}" failed. No retries permitted until 2025-12-03 12:46:36.366153481 +0000 UTC m=+5186.954395245 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls-certs" (UniqueName: "kubernetes.io/secret/876d06e6-0187-4e3d-9183-ed2083fe5189-kube-state-metrics-tls-certs") pod "kube-state-metrics-0" (UID: "876d06e6-0187-4e3d-9183-ed2083fe5189") : secret "cert-kube-state-metrics-svc" not found Dec 03 12:46:28 crc kubenswrapper[5052]: I1203 12:46:28.369630 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"4c9f4da8-f1ae-4815-8ee2-0e873fb07d62","Type":"ContainerStarted","Data":"5508cf0583ceb9a8b712a6f4c574d921b80de874086d91a5e31bb2abd237c9b1"} Dec 03 12:46:28 crc kubenswrapper[5052]: I1203 12:46:28.372428 5052 generic.go:334] "Generic (PLEG): container finished" podID="6d653682-52e2-4940-af66-633475bfe09a" containerID="965af26b70386d0042f76a35140b047b193a7f695cf03cf43e91184f27f9aaa1" exitCode=1 Dec 03 12:46:28 crc kubenswrapper[5052]: I1203 12:46:28.372489 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-conductor-0" event={"ID":"6d653682-52e2-4940-af66-633475bfe09a","Type":"ContainerDied","Data":"965af26b70386d0042f76a35140b047b193a7f695cf03cf43e91184f27f9aaa1"} Dec 03 12:46:28 crc kubenswrapper[5052]: I1203 12:46:28.372577 5052 scope.go:117] "RemoveContainer" containerID="6f8b18b5b30d4270384e6892b05ad3c64ed08fdfb147a6e3286cf16683c5b21e" Dec 03 12:46:28 crc kubenswrapper[5052]: I1203 12:46:28.373434 5052 scope.go:117] "RemoveContainer" containerID="965af26b70386d0042f76a35140b047b193a7f695cf03cf43e91184f27f9aaa1" Dec 03 12:46:28 crc kubenswrapper[5052]: E1203 12:46:28.375726 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"nova-cell0-conductor-conductor\" with CrashLoopBackOff: \"back-off 10s restarting failed container=nova-cell0-conductor-conductor pod=nova-cell0-conductor-0_openstack-kuttl-tests(6d653682-52e2-4940-af66-633475bfe09a)\"" pod="openstack-kuttl-tests/nova-cell0-conductor-0" podUID="6d653682-52e2-4940-af66-633475bfe09a" Dec 03 12:46:28 crc kubenswrapper[5052]: I1203 12:46:28.392880 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"9481ae66-152c-41e7-9cbc-90dcb5fa6171","Type":"ContainerStarted","Data":"f933eb5223bf5c6f62c2c9ab3d18b10809735490714d791df109aa3328a1e091"} Dec 03 12:46:28 crc kubenswrapper[5052]: I1203 12:46:28.392949 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:46:28 crc kubenswrapper[5052]: I1203 12:46:28.424236 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/ceilometer-0" podStartSLOduration=6.887665985 podStartE2EDuration="11.424218422s" podCreationTimestamp="2025-12-03 12:46:17 +0000 UTC" firstStartedPulling="2025-12-03 12:46:22.85249652 +0000 UTC m=+5173.440738284" lastFinishedPulling="2025-12-03 12:46:27.389048957 +0000 UTC m=+5177.977290721" observedRunningTime="2025-12-03 12:46:28.417912091 +0000 UTC m=+5179.006153855" watchObservedRunningTime="2025-12-03 12:46:28.424218422 +0000 UTC m=+5179.012460186" Dec 03 12:46:28 crc kubenswrapper[5052]: I1203 12:46:28.466953 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/901a2f0b-fb8e-4c97-aea7-567d7ee42a23-public-tls-certs\") pod \"neutron-6d64d7894b-xp9d5\" (UID: \"901a2f0b-fb8e-4c97-aea7-567d7ee42a23\") " pod="openstack-kuttl-tests/neutron-6d64d7894b-xp9d5" Dec 03 12:46:28 crc kubenswrapper[5052]: I1203 12:46:28.467103 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/901a2f0b-fb8e-4c97-aea7-567d7ee42a23-ovndb-tls-certs\") pod \"neutron-6d64d7894b-xp9d5\" (UID: \"901a2f0b-fb8e-4c97-aea7-567d7ee42a23\") " pod="openstack-kuttl-tests/neutron-6d64d7894b-xp9d5" Dec 03 12:46:28 crc kubenswrapper[5052]: I1203 12:46:28.467198 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/901a2f0b-fb8e-4c97-aea7-567d7ee42a23-internal-tls-certs\") pod \"neutron-6d64d7894b-xp9d5\" (UID: \"901a2f0b-fb8e-4c97-aea7-567d7ee42a23\") " pod="openstack-kuttl-tests/neutron-6d64d7894b-xp9d5" Dec 03 12:46:28 crc kubenswrapper[5052]: E1203 12:46:28.468764 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-neutron-public-svc: secret "cert-neutron-public-svc" not found Dec 03 12:46:28 crc kubenswrapper[5052]: E1203 12:46:28.468811 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/901a2f0b-fb8e-4c97-aea7-567d7ee42a23-public-tls-certs podName:901a2f0b-fb8e-4c97-aea7-567d7ee42a23 nodeName:}" failed. No retries permitted until 2025-12-03 12:46:44.468798076 +0000 UTC m=+5195.057039840 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "public-tls-certs" (UniqueName: "kubernetes.io/secret/901a2f0b-fb8e-4c97-aea7-567d7ee42a23-public-tls-certs") pod "neutron-6d64d7894b-xp9d5" (UID: "901a2f0b-fb8e-4c97-aea7-567d7ee42a23") : secret "cert-neutron-public-svc" not found Dec 03 12:46:28 crc kubenswrapper[5052]: E1203 12:46:28.468934 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-neutron-ovndbs: secret "cert-neutron-ovndbs" not found Dec 03 12:46:28 crc kubenswrapper[5052]: E1203 12:46:28.469875 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/901a2f0b-fb8e-4c97-aea7-567d7ee42a23-ovndb-tls-certs podName:901a2f0b-fb8e-4c97-aea7-567d7ee42a23 nodeName:}" failed. No retries permitted until 2025-12-03 12:46:44.469847706 +0000 UTC m=+5195.058089530 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "ovndb-tls-certs" (UniqueName: "kubernetes.io/secret/901a2f0b-fb8e-4c97-aea7-567d7ee42a23-ovndb-tls-certs") pod "neutron-6d64d7894b-xp9d5" (UID: "901a2f0b-fb8e-4c97-aea7-567d7ee42a23") : secret "cert-neutron-ovndbs" not found Dec 03 12:46:28 crc kubenswrapper[5052]: I1203 12:46:28.480787 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/901a2f0b-fb8e-4c97-aea7-567d7ee42a23-internal-tls-certs\") pod \"neutron-6d64d7894b-xp9d5\" (UID: \"901a2f0b-fb8e-4c97-aea7-567d7ee42a23\") " pod="openstack-kuttl-tests/neutron-6d64d7894b-xp9d5" Dec 03 12:46:28 crc kubenswrapper[5052]: I1203 12:46:28.533997 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:46:28 crc kubenswrapper[5052]: I1203 12:46:28.559534 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 12:46:28 crc kubenswrapper[5052]: I1203 12:46:28.572770 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:46:28 crc kubenswrapper[5052]: I1203 12:46:28.702877 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:46:29 crc kubenswrapper[5052]: I1203 12:46:29.375544 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:46:29 crc kubenswrapper[5052]: I1203 12:46:29.375933 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:46:29 crc kubenswrapper[5052]: E1203 12:46:29.397263 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-barbican-internal-svc: secret "cert-barbican-internal-svc" not found Dec 03 12:46:29 crc kubenswrapper[5052]: E1203 12:46:29.397327 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5b63f869-c95a-4279-b542-3ebbec1a1fd5-internal-tls-certs podName:5b63f869-c95a-4279-b542-3ebbec1a1fd5 nodeName:}" failed. No retries permitted until 2025-12-03 12:46:31.3973115 +0000 UTC m=+5181.985553264 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "internal-tls-certs" (UniqueName: "kubernetes.io/secret/5b63f869-c95a-4279-b542-3ebbec1a1fd5-internal-tls-certs") pod "barbican-api-6d86f7b884-lf4pd" (UID: "5b63f869-c95a-4279-b542-3ebbec1a1fd5") : secret "cert-barbican-internal-svc" not found Dec 03 12:46:29 crc kubenswrapper[5052]: I1203 12:46:29.417411 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"4c9f4da8-f1ae-4815-8ee2-0e873fb07d62","Type":"ContainerStarted","Data":"1974352f744e6b1132798362ce57f0fa14a8df4a45523231261a216257d4872d"} Dec 03 12:46:29 crc kubenswrapper[5052]: I1203 12:46:29.417465 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"4c9f4da8-f1ae-4815-8ee2-0e873fb07d62","Type":"ContainerStarted","Data":"c04c0411733e4ad72fe617e9a0abb2297cf6a19a36fdb5ef4c3061887c382c9a"} Dec 03 12:46:29 crc kubenswrapper[5052]: I1203 12:46:29.422291 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"9bb8adcd-621e-489c-8cec-c8965062b1c9","Type":"ContainerStarted","Data":"19e947f84460012632efd07d8c682a8795d68eec61822aabc8b926ea68c6ecb2"} Dec 03 12:46:29 crc kubenswrapper[5052]: I1203 12:46:29.422329 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"9bb8adcd-621e-489c-8cec-c8965062b1c9","Type":"ContainerStarted","Data":"452fba7a09b72cd3c94bf2ba9022e1cf82113ba97bf06a1114ba92b1610e53cd"} Dec 03 12:46:29 crc kubenswrapper[5052]: I1203 12:46:29.452837 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/glance-default-external-api-0" podStartSLOduration=11.452816537 podStartE2EDuration="11.452816537s" podCreationTimestamp="2025-12-03 12:46:18 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:46:29.439708262 +0000 UTC m=+5180.027950026" watchObservedRunningTime="2025-12-03 12:46:29.452816537 +0000 UTC m=+5180.041058301" Dec 03 12:46:29 crc kubenswrapper[5052]: I1203 12:46:29.460375 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:46:29 crc kubenswrapper[5052]: I1203 12:46:29.691881 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="234278ab-b4a8-4fdf-abf9-c67cdbff1ac0" path="/var/lib/kubelet/pods/234278ab-b4a8-4fdf-abf9-c67cdbff1ac0/volumes" Dec 03 12:46:30 crc kubenswrapper[5052]: I1203 12:46:30.322559 5052 scope.go:117] "RemoveContainer" containerID="49e65401462a62ad6b62529bdc792c44fb0d7a07381a58a73215f64add3d28c5" Dec 03 12:46:30 crc kubenswrapper[5052]: I1203 12:46:30.350239 5052 scope.go:117] "RemoveContainer" containerID="1d3c75c1863b6df410b4948455d1e97a7c92fce681cbc217dafa1d03a2fae0c6" Dec 03 12:46:30 crc kubenswrapper[5052]: I1203 12:46:30.381770 5052 scope.go:117] "RemoveContainer" containerID="af9343e98edf473dab463b002e96eedd0ca172c95e67fa9e25832f1c73a3318a" Dec 03 12:46:30 crc kubenswrapper[5052]: I1203 12:46:30.389743 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-metadata-0" podUID="75266130-5854-4acd-acc9-bc22accc19bc" containerName="nova-metadata-metadata" probeResult="failure" output="Get \"https://10.217.0.12:8775/\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Dec 03 12:46:30 crc kubenswrapper[5052]: I1203 12:46:30.389816 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-metadata-0" podUID="75266130-5854-4acd-acc9-bc22accc19bc" containerName="nova-metadata-log" probeResult="failure" output="Get \"https://10.217.0.12:8775/\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Dec 03 12:46:30 crc kubenswrapper[5052]: I1203 12:46:30.400437 5052 scope.go:117] "RemoveContainer" containerID="29357b3d97482fd8007dd5664e67838f357ddeab54a10504c300a6188e0d2e3d" Dec 03 12:46:30 crc kubenswrapper[5052]: I1203 12:46:30.435099 5052 scope.go:117] "RemoveContainer" containerID="8ca42e62aac09b81598563bdb887c5ea6090758cf976adc8ad4b4e387ec44287" Dec 03 12:46:30 crc kubenswrapper[5052]: I1203 12:46:30.442402 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"9bb8adcd-621e-489c-8cec-c8965062b1c9","Type":"ContainerStarted","Data":"4dfc52960ffcbb4fcd5ce878bcf6783f646724cbd4fb7ed867685b8bc3a9bff4"} Dec 03 12:46:30 crc kubenswrapper[5052]: I1203 12:46:30.442570 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:46:30 crc kubenswrapper[5052]: I1203 12:46:30.445033 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="9481ae66-152c-41e7-9cbc-90dcb5fa6171" containerName="ceilometer-central-agent" containerID="cri-o://8d6b67218da38e98daea9b518ff99007b1a85805172d6e47d9084b2c93ac9e4e" gracePeriod=30 Dec 03 12:46:30 crc kubenswrapper[5052]: I1203 12:46:30.445060 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="9481ae66-152c-41e7-9cbc-90dcb5fa6171" containerName="proxy-httpd" containerID="cri-o://f933eb5223bf5c6f62c2c9ab3d18b10809735490714d791df109aa3328a1e091" gracePeriod=30 Dec 03 12:46:30 crc kubenswrapper[5052]: I1203 12:46:30.445064 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="9481ae66-152c-41e7-9cbc-90dcb5fa6171" containerName="sg-core" containerID="cri-o://dc1f7e238c595e37880087caf4c48af2813c1fa9c81659fabdd99d813fa482a6" gracePeriod=30 Dec 03 12:46:30 crc kubenswrapper[5052]: I1203 12:46:30.445127 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="9481ae66-152c-41e7-9cbc-90dcb5fa6171" containerName="ceilometer-notification-agent" containerID="cri-o://1466275e6c25f9b46a0ec21cf1af15f3c40b914e854dc9a544b1dba27f630296" gracePeriod=30 Dec 03 12:46:30 crc kubenswrapper[5052]: I1203 12:46:30.474602 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/cinder-api-0" podStartSLOduration=3.474580068 podStartE2EDuration="3.474580068s" podCreationTimestamp="2025-12-03 12:46:27 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:46:30.46134129 +0000 UTC m=+5181.049583064" watchObservedRunningTime="2025-12-03 12:46:30.474580068 +0000 UTC m=+5181.062821852" Dec 03 12:46:30 crc kubenswrapper[5052]: I1203 12:46:30.483057 5052 scope.go:117] "RemoveContainer" containerID="0ef7cd09cfd4cf1e5792656dab2c9785fb2b5b3445c4f04f4ff27b9d93363509" Dec 03 12:46:30 crc kubenswrapper[5052]: I1203 12:46:30.510651 5052 scope.go:117] "RemoveContainer" containerID="9c894be2b5ea25d2e1dc93642f17b355a9f5eb4d38037b4b3d4de06025012260" Dec 03 12:46:30 crc kubenswrapper[5052]: I1203 12:46:30.537123 5052 scope.go:117] "RemoveContainer" containerID="027b9a8307f751e8024d032edb748ef4b5da8f3d938c7c031bac5ed5e44f0c7d" Dec 03 12:46:30 crc kubenswrapper[5052]: I1203 12:46:30.561413 5052 scope.go:117] "RemoveContainer" containerID="995bc9f60d05257f8493e1dd6196163744598cee55f2e09b16470fc6e4d9962b" Dec 03 12:46:30 crc kubenswrapper[5052]: I1203 12:46:30.666909 5052 scope.go:117] "RemoveContainer" containerID="60eb1509e2cea82540bce951978d0b80c496f80fe40a218c77ecbe97a3e9c835" Dec 03 12:46:30 crc kubenswrapper[5052]: I1203 12:46:30.688980 5052 kubelet.go:2542] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:46:30 crc kubenswrapper[5052]: I1203 12:46:30.689667 5052 scope.go:117] "RemoveContainer" containerID="965af26b70386d0042f76a35140b047b193a7f695cf03cf43e91184f27f9aaa1" Dec 03 12:46:30 crc kubenswrapper[5052]: E1203 12:46:30.689893 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"nova-cell0-conductor-conductor\" with CrashLoopBackOff: \"back-off 10s restarting failed container=nova-cell0-conductor-conductor pod=nova-cell0-conductor-0_openstack-kuttl-tests(6d653682-52e2-4940-af66-633475bfe09a)\"" pod="openstack-kuttl-tests/nova-cell0-conductor-0" podUID="6d653682-52e2-4940-af66-633475bfe09a" Dec 03 12:46:30 crc kubenswrapper[5052]: I1203 12:46:30.702527 5052 scope.go:117] "RemoveContainer" containerID="229df7ab571e9eb2bf196a218d66569b5405729090282a855310baf493b87409" Dec 03 12:46:30 crc kubenswrapper[5052]: I1203 12:46:30.737211 5052 scope.go:117] "RemoveContainer" containerID="101072c12291fb7e9e1ca717c6db562103892180f6281b3dd97442d2a61e51f8" Dec 03 12:46:31 crc kubenswrapper[5052]: I1203 12:46:31.252169 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/f8ab3054-5998-40c1-852f-be0dbaf022e5-metrics-certs-tls-certs\") pod \"ovn-northd-0\" (UID: \"f8ab3054-5998-40c1-852f-be0dbaf022e5\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:46:31 crc kubenswrapper[5052]: I1203 12:46:31.252809 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovn-northd-tls-certs\" (UniqueName: \"kubernetes.io/secret/f8ab3054-5998-40c1-852f-be0dbaf022e5-ovn-northd-tls-certs\") pod \"ovn-northd-0\" (UID: \"f8ab3054-5998-40c1-852f-be0dbaf022e5\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:46:31 crc kubenswrapper[5052]: E1203 12:46:31.253034 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-ovnnorthd-ovndbs: secret "cert-ovnnorthd-ovndbs" not found Dec 03 12:46:31 crc kubenswrapper[5052]: E1203 12:46:31.253107 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f8ab3054-5998-40c1-852f-be0dbaf022e5-ovn-northd-tls-certs podName:f8ab3054-5998-40c1-852f-be0dbaf022e5 nodeName:}" failed. No retries permitted until 2025-12-03 12:46:39.253086592 +0000 UTC m=+5189.841328356 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "ovn-northd-tls-certs" (UniqueName: "kubernetes.io/secret/f8ab3054-5998-40c1-852f-be0dbaf022e5-ovn-northd-tls-certs") pod "ovn-northd-0" (UID: "f8ab3054-5998-40c1-852f-be0dbaf022e5") : secret "cert-ovnnorthd-ovndbs" not found Dec 03 12:46:31 crc kubenswrapper[5052]: E1203 12:46:31.253442 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-ovn-metrics: secret "cert-ovn-metrics" not found Dec 03 12:46:31 crc kubenswrapper[5052]: E1203 12:46:31.253565 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f8ab3054-5998-40c1-852f-be0dbaf022e5-metrics-certs-tls-certs podName:f8ab3054-5998-40c1-852f-be0dbaf022e5 nodeName:}" failed. No retries permitted until 2025-12-03 12:46:39.253537505 +0000 UTC m=+5189.841779279 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs-tls-certs" (UniqueName: "kubernetes.io/secret/f8ab3054-5998-40c1-852f-be0dbaf022e5-metrics-certs-tls-certs") pod "ovn-northd-0" (UID: "f8ab3054-5998-40c1-852f-be0dbaf022e5") : secret "cert-ovn-metrics" not found Dec 03 12:46:31 crc kubenswrapper[5052]: I1203 12:46:31.324872 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:46:31 crc kubenswrapper[5052]: I1203 12:46:31.458625 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9481ae66-152c-41e7-9cbc-90dcb5fa6171-combined-ca-bundle\") pod \"9481ae66-152c-41e7-9cbc-90dcb5fa6171\" (UID: \"9481ae66-152c-41e7-9cbc-90dcb5fa6171\") " Dec 03 12:46:31 crc kubenswrapper[5052]: I1203 12:46:31.458780 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bwnpf\" (UniqueName: \"kubernetes.io/projected/9481ae66-152c-41e7-9cbc-90dcb5fa6171-kube-api-access-bwnpf\") pod \"9481ae66-152c-41e7-9cbc-90dcb5fa6171\" (UID: \"9481ae66-152c-41e7-9cbc-90dcb5fa6171\") " Dec 03 12:46:31 crc kubenswrapper[5052]: I1203 12:46:31.458815 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/9481ae66-152c-41e7-9cbc-90dcb5fa6171-run-httpd\") pod \"9481ae66-152c-41e7-9cbc-90dcb5fa6171\" (UID: \"9481ae66-152c-41e7-9cbc-90dcb5fa6171\") " Dec 03 12:46:31 crc kubenswrapper[5052]: I1203 12:46:31.458888 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/9481ae66-152c-41e7-9cbc-90dcb5fa6171-sg-core-conf-yaml\") pod \"9481ae66-152c-41e7-9cbc-90dcb5fa6171\" (UID: \"9481ae66-152c-41e7-9cbc-90dcb5fa6171\") " Dec 03 12:46:31 crc kubenswrapper[5052]: I1203 12:46:31.458940 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/9481ae66-152c-41e7-9cbc-90dcb5fa6171-ceilometer-tls-certs\") pod \"9481ae66-152c-41e7-9cbc-90dcb5fa6171\" (UID: \"9481ae66-152c-41e7-9cbc-90dcb5fa6171\") " Dec 03 12:46:31 crc kubenswrapper[5052]: I1203 12:46:31.459014 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/9481ae66-152c-41e7-9cbc-90dcb5fa6171-log-httpd\") pod \"9481ae66-152c-41e7-9cbc-90dcb5fa6171\" (UID: \"9481ae66-152c-41e7-9cbc-90dcb5fa6171\") " Dec 03 12:46:31 crc kubenswrapper[5052]: I1203 12:46:31.459041 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/9481ae66-152c-41e7-9cbc-90dcb5fa6171-scripts\") pod \"9481ae66-152c-41e7-9cbc-90dcb5fa6171\" (UID: \"9481ae66-152c-41e7-9cbc-90dcb5fa6171\") " Dec 03 12:46:31 crc kubenswrapper[5052]: I1203 12:46:31.459068 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/9481ae66-152c-41e7-9cbc-90dcb5fa6171-config-data\") pod \"9481ae66-152c-41e7-9cbc-90dcb5fa6171\" (UID: \"9481ae66-152c-41e7-9cbc-90dcb5fa6171\") " Dec 03 12:46:31 crc kubenswrapper[5052]: E1203 12:46:31.460689 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-barbican-internal-svc: secret "cert-barbican-internal-svc" not found Dec 03 12:46:31 crc kubenswrapper[5052]: E1203 12:46:31.460763 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5b63f869-c95a-4279-b542-3ebbec1a1fd5-internal-tls-certs podName:5b63f869-c95a-4279-b542-3ebbec1a1fd5 nodeName:}" failed. No retries permitted until 2025-12-03 12:46:35.460743551 +0000 UTC m=+5186.048985315 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "internal-tls-certs" (UniqueName: "kubernetes.io/secret/5b63f869-c95a-4279-b542-3ebbec1a1fd5-internal-tls-certs") pod "barbican-api-6d86f7b884-lf4pd" (UID: "5b63f869-c95a-4279-b542-3ebbec1a1fd5") : secret "cert-barbican-internal-svc" not found Dec 03 12:46:31 crc kubenswrapper[5052]: I1203 12:46:31.465234 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9481ae66-152c-41e7-9cbc-90dcb5fa6171-run-httpd" (OuterVolumeSpecName: "run-httpd") pod "9481ae66-152c-41e7-9cbc-90dcb5fa6171" (UID: "9481ae66-152c-41e7-9cbc-90dcb5fa6171"). InnerVolumeSpecName "run-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:46:31 crc kubenswrapper[5052]: I1203 12:46:31.465358 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9481ae66-152c-41e7-9cbc-90dcb5fa6171-log-httpd" (OuterVolumeSpecName: "log-httpd") pod "9481ae66-152c-41e7-9cbc-90dcb5fa6171" (UID: "9481ae66-152c-41e7-9cbc-90dcb5fa6171"). InnerVolumeSpecName "log-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:46:31 crc kubenswrapper[5052]: I1203 12:46:31.468704 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9481ae66-152c-41e7-9cbc-90dcb5fa6171-kube-api-access-bwnpf" (OuterVolumeSpecName: "kube-api-access-bwnpf") pod "9481ae66-152c-41e7-9cbc-90dcb5fa6171" (UID: "9481ae66-152c-41e7-9cbc-90dcb5fa6171"). InnerVolumeSpecName "kube-api-access-bwnpf". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:46:31 crc kubenswrapper[5052]: I1203 12:46:31.473438 5052 generic.go:334] "Generic (PLEG): container finished" podID="9481ae66-152c-41e7-9cbc-90dcb5fa6171" containerID="f933eb5223bf5c6f62c2c9ab3d18b10809735490714d791df109aa3328a1e091" exitCode=0 Dec 03 12:46:31 crc kubenswrapper[5052]: I1203 12:46:31.473475 5052 generic.go:334] "Generic (PLEG): container finished" podID="9481ae66-152c-41e7-9cbc-90dcb5fa6171" containerID="dc1f7e238c595e37880087caf4c48af2813c1fa9c81659fabdd99d813fa482a6" exitCode=2 Dec 03 12:46:31 crc kubenswrapper[5052]: I1203 12:46:31.473484 5052 generic.go:334] "Generic (PLEG): container finished" podID="9481ae66-152c-41e7-9cbc-90dcb5fa6171" containerID="1466275e6c25f9b46a0ec21cf1af15f3c40b914e854dc9a544b1dba27f630296" exitCode=0 Dec 03 12:46:31 crc kubenswrapper[5052]: I1203 12:46:31.473492 5052 generic.go:334] "Generic (PLEG): container finished" podID="9481ae66-152c-41e7-9cbc-90dcb5fa6171" containerID="8d6b67218da38e98daea9b518ff99007b1a85805172d6e47d9084b2c93ac9e4e" exitCode=0 Dec 03 12:46:31 crc kubenswrapper[5052]: I1203 12:46:31.473697 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:46:31 crc kubenswrapper[5052]: I1203 12:46:31.473736 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"9481ae66-152c-41e7-9cbc-90dcb5fa6171","Type":"ContainerDied","Data":"f933eb5223bf5c6f62c2c9ab3d18b10809735490714d791df109aa3328a1e091"} Dec 03 12:46:31 crc kubenswrapper[5052]: I1203 12:46:31.473770 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"9481ae66-152c-41e7-9cbc-90dcb5fa6171","Type":"ContainerDied","Data":"dc1f7e238c595e37880087caf4c48af2813c1fa9c81659fabdd99d813fa482a6"} Dec 03 12:46:31 crc kubenswrapper[5052]: I1203 12:46:31.473793 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"9481ae66-152c-41e7-9cbc-90dcb5fa6171","Type":"ContainerDied","Data":"1466275e6c25f9b46a0ec21cf1af15f3c40b914e854dc9a544b1dba27f630296"} Dec 03 12:46:31 crc kubenswrapper[5052]: I1203 12:46:31.473806 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"9481ae66-152c-41e7-9cbc-90dcb5fa6171","Type":"ContainerDied","Data":"8d6b67218da38e98daea9b518ff99007b1a85805172d6e47d9084b2c93ac9e4e"} Dec 03 12:46:31 crc kubenswrapper[5052]: I1203 12:46:31.473817 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"9481ae66-152c-41e7-9cbc-90dcb5fa6171","Type":"ContainerDied","Data":"6ac650b6651daa123e83a2c4f419a2e53e6d72716603c465e8592bd1223209b1"} Dec 03 12:46:31 crc kubenswrapper[5052]: I1203 12:46:31.473873 5052 scope.go:117] "RemoveContainer" containerID="f933eb5223bf5c6f62c2c9ab3d18b10809735490714d791df109aa3328a1e091" Dec 03 12:46:31 crc kubenswrapper[5052]: I1203 12:46:31.495673 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9481ae66-152c-41e7-9cbc-90dcb5fa6171-scripts" (OuterVolumeSpecName: "scripts") pod "9481ae66-152c-41e7-9cbc-90dcb5fa6171" (UID: "9481ae66-152c-41e7-9cbc-90dcb5fa6171"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:46:31 crc kubenswrapper[5052]: I1203 12:46:31.508604 5052 scope.go:117] "RemoveContainer" containerID="dc1f7e238c595e37880087caf4c48af2813c1fa9c81659fabdd99d813fa482a6" Dec 03 12:46:31 crc kubenswrapper[5052]: I1203 12:46:31.519145 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9481ae66-152c-41e7-9cbc-90dcb5fa6171-sg-core-conf-yaml" (OuterVolumeSpecName: "sg-core-conf-yaml") pod "9481ae66-152c-41e7-9cbc-90dcb5fa6171" (UID: "9481ae66-152c-41e7-9cbc-90dcb5fa6171"). InnerVolumeSpecName "sg-core-conf-yaml". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:46:31 crc kubenswrapper[5052]: I1203 12:46:31.522878 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9481ae66-152c-41e7-9cbc-90dcb5fa6171-ceilometer-tls-certs" (OuterVolumeSpecName: "ceilometer-tls-certs") pod "9481ae66-152c-41e7-9cbc-90dcb5fa6171" (UID: "9481ae66-152c-41e7-9cbc-90dcb5fa6171"). InnerVolumeSpecName "ceilometer-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:46:31 crc kubenswrapper[5052]: I1203 12:46:31.535206 5052 scope.go:117] "RemoveContainer" containerID="1466275e6c25f9b46a0ec21cf1af15f3c40b914e854dc9a544b1dba27f630296" Dec 03 12:46:31 crc kubenswrapper[5052]: I1203 12:46:31.555817 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9481ae66-152c-41e7-9cbc-90dcb5fa6171-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "9481ae66-152c-41e7-9cbc-90dcb5fa6171" (UID: "9481ae66-152c-41e7-9cbc-90dcb5fa6171"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:46:31 crc kubenswrapper[5052]: I1203 12:46:31.558287 5052 scope.go:117] "RemoveContainer" containerID="8d6b67218da38e98daea9b518ff99007b1a85805172d6e47d9084b2c93ac9e4e" Dec 03 12:46:31 crc kubenswrapper[5052]: I1203 12:46:31.562791 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9481ae66-152c-41e7-9cbc-90dcb5fa6171-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:31 crc kubenswrapper[5052]: I1203 12:46:31.562824 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-bwnpf\" (UniqueName: \"kubernetes.io/projected/9481ae66-152c-41e7-9cbc-90dcb5fa6171-kube-api-access-bwnpf\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:31 crc kubenswrapper[5052]: I1203 12:46:31.562837 5052 reconciler_common.go:293] "Volume detached for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/9481ae66-152c-41e7-9cbc-90dcb5fa6171-run-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:31 crc kubenswrapper[5052]: I1203 12:46:31.562849 5052 reconciler_common.go:293] "Volume detached for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/9481ae66-152c-41e7-9cbc-90dcb5fa6171-sg-core-conf-yaml\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:31 crc kubenswrapper[5052]: I1203 12:46:31.562859 5052 reconciler_common.go:293] "Volume detached for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/9481ae66-152c-41e7-9cbc-90dcb5fa6171-ceilometer-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:31 crc kubenswrapper[5052]: I1203 12:46:31.562875 5052 reconciler_common.go:293] "Volume detached for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/9481ae66-152c-41e7-9cbc-90dcb5fa6171-log-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:31 crc kubenswrapper[5052]: I1203 12:46:31.562886 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/9481ae66-152c-41e7-9cbc-90dcb5fa6171-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:31 crc kubenswrapper[5052]: I1203 12:46:31.598557 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9481ae66-152c-41e7-9cbc-90dcb5fa6171-config-data" (OuterVolumeSpecName: "config-data") pod "9481ae66-152c-41e7-9cbc-90dcb5fa6171" (UID: "9481ae66-152c-41e7-9cbc-90dcb5fa6171"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:46:31 crc kubenswrapper[5052]: I1203 12:46:31.665263 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"vencrypt-tls-certs\" (UniqueName: \"kubernetes.io/secret/9c996590-4337-477e-a236-3e834d192346-vencrypt-tls-certs\") pod \"nova-cell1-novncproxy-0\" (UID: \"9c996590-4337-477e-a236-3e834d192346\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:46:31 crc kubenswrapper[5052]: E1203 12:46:31.665461 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-nova-novncproxy-cell1-vencrypt: secret "cert-nova-novncproxy-cell1-vencrypt" not found Dec 03 12:46:31 crc kubenswrapper[5052]: E1203 12:46:31.665570 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9c996590-4337-477e-a236-3e834d192346-vencrypt-tls-certs podName:9c996590-4337-477e-a236-3e834d192346 nodeName:}" failed. No retries permitted until 2025-12-03 12:46:47.665537788 +0000 UTC m=+5198.253779552 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "vencrypt-tls-certs" (UniqueName: "kubernetes.io/secret/9c996590-4337-477e-a236-3e834d192346-vencrypt-tls-certs") pod "nova-cell1-novncproxy-0" (UID: "9c996590-4337-477e-a236-3e834d192346") : secret "cert-nova-novncproxy-cell1-vencrypt" not found Dec 03 12:46:31 crc kubenswrapper[5052]: I1203 12:46:31.665643 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/c6c9b62e-bbbc-4557-92fd-da64bd8ee0cf-metrics-certs-tls-certs\") pod \"ovsdbserver-sb-0\" (UID: \"c6c9b62e-bbbc-4557-92fd-da64bd8ee0cf\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:46:31 crc kubenswrapper[5052]: I1203 12:46:31.665827 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/9481ae66-152c-41e7-9cbc-90dcb5fa6171-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:31 crc kubenswrapper[5052]: E1203 12:46:31.665877 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-ovn-metrics: secret "cert-ovn-metrics" not found Dec 03 12:46:31 crc kubenswrapper[5052]: E1203 12:46:31.665989 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c6c9b62e-bbbc-4557-92fd-da64bd8ee0cf-metrics-certs-tls-certs podName:c6c9b62e-bbbc-4557-92fd-da64bd8ee0cf nodeName:}" failed. No retries permitted until 2025-12-03 12:46:47.66596487 +0000 UTC m=+5198.254206724 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs-tls-certs" (UniqueName: "kubernetes.io/secret/c6c9b62e-bbbc-4557-92fd-da64bd8ee0cf-metrics-certs-tls-certs") pod "ovsdbserver-sb-0" (UID: "c6c9b62e-bbbc-4557-92fd-da64bd8ee0cf") : secret "cert-ovn-metrics" not found Dec 03 12:46:31 crc kubenswrapper[5052]: I1203 12:46:31.678687 5052 scope.go:117] "RemoveContainer" containerID="f933eb5223bf5c6f62c2c9ab3d18b10809735490714d791df109aa3328a1e091" Dec 03 12:46:31 crc kubenswrapper[5052]: E1203 12:46:31.679212 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f933eb5223bf5c6f62c2c9ab3d18b10809735490714d791df109aa3328a1e091\": container with ID starting with f933eb5223bf5c6f62c2c9ab3d18b10809735490714d791df109aa3328a1e091 not found: ID does not exist" containerID="f933eb5223bf5c6f62c2c9ab3d18b10809735490714d791df109aa3328a1e091" Dec 03 12:46:31 crc kubenswrapper[5052]: I1203 12:46:31.679246 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f933eb5223bf5c6f62c2c9ab3d18b10809735490714d791df109aa3328a1e091"} err="failed to get container status \"f933eb5223bf5c6f62c2c9ab3d18b10809735490714d791df109aa3328a1e091\": rpc error: code = NotFound desc = could not find container \"f933eb5223bf5c6f62c2c9ab3d18b10809735490714d791df109aa3328a1e091\": container with ID starting with f933eb5223bf5c6f62c2c9ab3d18b10809735490714d791df109aa3328a1e091 not found: ID does not exist" Dec 03 12:46:31 crc kubenswrapper[5052]: I1203 12:46:31.679274 5052 scope.go:117] "RemoveContainer" containerID="dc1f7e238c595e37880087caf4c48af2813c1fa9c81659fabdd99d813fa482a6" Dec 03 12:46:31 crc kubenswrapper[5052]: E1203 12:46:31.679727 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dc1f7e238c595e37880087caf4c48af2813c1fa9c81659fabdd99d813fa482a6\": container with ID starting with dc1f7e238c595e37880087caf4c48af2813c1fa9c81659fabdd99d813fa482a6 not found: ID does not exist" containerID="dc1f7e238c595e37880087caf4c48af2813c1fa9c81659fabdd99d813fa482a6" Dec 03 12:46:31 crc kubenswrapper[5052]: I1203 12:46:31.679770 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dc1f7e238c595e37880087caf4c48af2813c1fa9c81659fabdd99d813fa482a6"} err="failed to get container status \"dc1f7e238c595e37880087caf4c48af2813c1fa9c81659fabdd99d813fa482a6\": rpc error: code = NotFound desc = could not find container \"dc1f7e238c595e37880087caf4c48af2813c1fa9c81659fabdd99d813fa482a6\": container with ID starting with dc1f7e238c595e37880087caf4c48af2813c1fa9c81659fabdd99d813fa482a6 not found: ID does not exist" Dec 03 12:46:31 crc kubenswrapper[5052]: I1203 12:46:31.679802 5052 scope.go:117] "RemoveContainer" containerID="1466275e6c25f9b46a0ec21cf1af15f3c40b914e854dc9a544b1dba27f630296" Dec 03 12:46:31 crc kubenswrapper[5052]: E1203 12:46:31.680121 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1466275e6c25f9b46a0ec21cf1af15f3c40b914e854dc9a544b1dba27f630296\": container with ID starting with 1466275e6c25f9b46a0ec21cf1af15f3c40b914e854dc9a544b1dba27f630296 not found: ID does not exist" containerID="1466275e6c25f9b46a0ec21cf1af15f3c40b914e854dc9a544b1dba27f630296" Dec 03 12:46:31 crc kubenswrapper[5052]: I1203 12:46:31.680151 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1466275e6c25f9b46a0ec21cf1af15f3c40b914e854dc9a544b1dba27f630296"} err="failed to get container status \"1466275e6c25f9b46a0ec21cf1af15f3c40b914e854dc9a544b1dba27f630296\": rpc error: code = NotFound desc = could not find container \"1466275e6c25f9b46a0ec21cf1af15f3c40b914e854dc9a544b1dba27f630296\": container with ID starting with 1466275e6c25f9b46a0ec21cf1af15f3c40b914e854dc9a544b1dba27f630296 not found: ID does not exist" Dec 03 12:46:31 crc kubenswrapper[5052]: I1203 12:46:31.680169 5052 scope.go:117] "RemoveContainer" containerID="8d6b67218da38e98daea9b518ff99007b1a85805172d6e47d9084b2c93ac9e4e" Dec 03 12:46:31 crc kubenswrapper[5052]: E1203 12:46:31.680397 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8d6b67218da38e98daea9b518ff99007b1a85805172d6e47d9084b2c93ac9e4e\": container with ID starting with 8d6b67218da38e98daea9b518ff99007b1a85805172d6e47d9084b2c93ac9e4e not found: ID does not exist" containerID="8d6b67218da38e98daea9b518ff99007b1a85805172d6e47d9084b2c93ac9e4e" Dec 03 12:46:31 crc kubenswrapper[5052]: I1203 12:46:31.680421 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8d6b67218da38e98daea9b518ff99007b1a85805172d6e47d9084b2c93ac9e4e"} err="failed to get container status \"8d6b67218da38e98daea9b518ff99007b1a85805172d6e47d9084b2c93ac9e4e\": rpc error: code = NotFound desc = could not find container \"8d6b67218da38e98daea9b518ff99007b1a85805172d6e47d9084b2c93ac9e4e\": container with ID starting with 8d6b67218da38e98daea9b518ff99007b1a85805172d6e47d9084b2c93ac9e4e not found: ID does not exist" Dec 03 12:46:31 crc kubenswrapper[5052]: I1203 12:46:31.680439 5052 scope.go:117] "RemoveContainer" containerID="f933eb5223bf5c6f62c2c9ab3d18b10809735490714d791df109aa3328a1e091" Dec 03 12:46:31 crc kubenswrapper[5052]: I1203 12:46:31.680741 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f933eb5223bf5c6f62c2c9ab3d18b10809735490714d791df109aa3328a1e091"} err="failed to get container status \"f933eb5223bf5c6f62c2c9ab3d18b10809735490714d791df109aa3328a1e091\": rpc error: code = NotFound desc = could not find container \"f933eb5223bf5c6f62c2c9ab3d18b10809735490714d791df109aa3328a1e091\": container with ID starting with f933eb5223bf5c6f62c2c9ab3d18b10809735490714d791df109aa3328a1e091 not found: ID does not exist" Dec 03 12:46:31 crc kubenswrapper[5052]: I1203 12:46:31.680765 5052 scope.go:117] "RemoveContainer" containerID="dc1f7e238c595e37880087caf4c48af2813c1fa9c81659fabdd99d813fa482a6" Dec 03 12:46:31 crc kubenswrapper[5052]: I1203 12:46:31.681216 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dc1f7e238c595e37880087caf4c48af2813c1fa9c81659fabdd99d813fa482a6"} err="failed to get container status \"dc1f7e238c595e37880087caf4c48af2813c1fa9c81659fabdd99d813fa482a6\": rpc error: code = NotFound desc = could not find container \"dc1f7e238c595e37880087caf4c48af2813c1fa9c81659fabdd99d813fa482a6\": container with ID starting with dc1f7e238c595e37880087caf4c48af2813c1fa9c81659fabdd99d813fa482a6 not found: ID does not exist" Dec 03 12:46:31 crc kubenswrapper[5052]: I1203 12:46:31.681243 5052 scope.go:117] "RemoveContainer" containerID="1466275e6c25f9b46a0ec21cf1af15f3c40b914e854dc9a544b1dba27f630296" Dec 03 12:46:31 crc kubenswrapper[5052]: I1203 12:46:31.681439 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1466275e6c25f9b46a0ec21cf1af15f3c40b914e854dc9a544b1dba27f630296"} err="failed to get container status \"1466275e6c25f9b46a0ec21cf1af15f3c40b914e854dc9a544b1dba27f630296\": rpc error: code = NotFound desc = could not find container \"1466275e6c25f9b46a0ec21cf1af15f3c40b914e854dc9a544b1dba27f630296\": container with ID starting with 1466275e6c25f9b46a0ec21cf1af15f3c40b914e854dc9a544b1dba27f630296 not found: ID does not exist" Dec 03 12:46:31 crc kubenswrapper[5052]: I1203 12:46:31.681456 5052 scope.go:117] "RemoveContainer" containerID="8d6b67218da38e98daea9b518ff99007b1a85805172d6e47d9084b2c93ac9e4e" Dec 03 12:46:31 crc kubenswrapper[5052]: I1203 12:46:31.681666 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8d6b67218da38e98daea9b518ff99007b1a85805172d6e47d9084b2c93ac9e4e"} err="failed to get container status \"8d6b67218da38e98daea9b518ff99007b1a85805172d6e47d9084b2c93ac9e4e\": rpc error: code = NotFound desc = could not find container \"8d6b67218da38e98daea9b518ff99007b1a85805172d6e47d9084b2c93ac9e4e\": container with ID starting with 8d6b67218da38e98daea9b518ff99007b1a85805172d6e47d9084b2c93ac9e4e not found: ID does not exist" Dec 03 12:46:31 crc kubenswrapper[5052]: I1203 12:46:31.681683 5052 scope.go:117] "RemoveContainer" containerID="f933eb5223bf5c6f62c2c9ab3d18b10809735490714d791df109aa3328a1e091" Dec 03 12:46:31 crc kubenswrapper[5052]: I1203 12:46:31.681981 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f933eb5223bf5c6f62c2c9ab3d18b10809735490714d791df109aa3328a1e091"} err="failed to get container status \"f933eb5223bf5c6f62c2c9ab3d18b10809735490714d791df109aa3328a1e091\": rpc error: code = NotFound desc = could not find container \"f933eb5223bf5c6f62c2c9ab3d18b10809735490714d791df109aa3328a1e091\": container with ID starting with f933eb5223bf5c6f62c2c9ab3d18b10809735490714d791df109aa3328a1e091 not found: ID does not exist" Dec 03 12:46:31 crc kubenswrapper[5052]: I1203 12:46:31.682027 5052 scope.go:117] "RemoveContainer" containerID="dc1f7e238c595e37880087caf4c48af2813c1fa9c81659fabdd99d813fa482a6" Dec 03 12:46:31 crc kubenswrapper[5052]: I1203 12:46:31.682362 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dc1f7e238c595e37880087caf4c48af2813c1fa9c81659fabdd99d813fa482a6"} err="failed to get container status \"dc1f7e238c595e37880087caf4c48af2813c1fa9c81659fabdd99d813fa482a6\": rpc error: code = NotFound desc = could not find container \"dc1f7e238c595e37880087caf4c48af2813c1fa9c81659fabdd99d813fa482a6\": container with ID starting with dc1f7e238c595e37880087caf4c48af2813c1fa9c81659fabdd99d813fa482a6 not found: ID does not exist" Dec 03 12:46:31 crc kubenswrapper[5052]: I1203 12:46:31.682389 5052 scope.go:117] "RemoveContainer" containerID="1466275e6c25f9b46a0ec21cf1af15f3c40b914e854dc9a544b1dba27f630296" Dec 03 12:46:31 crc kubenswrapper[5052]: I1203 12:46:31.682763 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1466275e6c25f9b46a0ec21cf1af15f3c40b914e854dc9a544b1dba27f630296"} err="failed to get container status \"1466275e6c25f9b46a0ec21cf1af15f3c40b914e854dc9a544b1dba27f630296\": rpc error: code = NotFound desc = could not find container \"1466275e6c25f9b46a0ec21cf1af15f3c40b914e854dc9a544b1dba27f630296\": container with ID starting with 1466275e6c25f9b46a0ec21cf1af15f3c40b914e854dc9a544b1dba27f630296 not found: ID does not exist" Dec 03 12:46:31 crc kubenswrapper[5052]: I1203 12:46:31.682790 5052 scope.go:117] "RemoveContainer" containerID="8d6b67218da38e98daea9b518ff99007b1a85805172d6e47d9084b2c93ac9e4e" Dec 03 12:46:31 crc kubenswrapper[5052]: I1203 12:46:31.683003 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8d6b67218da38e98daea9b518ff99007b1a85805172d6e47d9084b2c93ac9e4e"} err="failed to get container status \"8d6b67218da38e98daea9b518ff99007b1a85805172d6e47d9084b2c93ac9e4e\": rpc error: code = NotFound desc = could not find container \"8d6b67218da38e98daea9b518ff99007b1a85805172d6e47d9084b2c93ac9e4e\": container with ID starting with 8d6b67218da38e98daea9b518ff99007b1a85805172d6e47d9084b2c93ac9e4e not found: ID does not exist" Dec 03 12:46:31 crc kubenswrapper[5052]: I1203 12:46:31.683030 5052 scope.go:117] "RemoveContainer" containerID="f933eb5223bf5c6f62c2c9ab3d18b10809735490714d791df109aa3328a1e091" Dec 03 12:46:31 crc kubenswrapper[5052]: I1203 12:46:31.683184 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f933eb5223bf5c6f62c2c9ab3d18b10809735490714d791df109aa3328a1e091"} err="failed to get container status \"f933eb5223bf5c6f62c2c9ab3d18b10809735490714d791df109aa3328a1e091\": rpc error: code = NotFound desc = could not find container \"f933eb5223bf5c6f62c2c9ab3d18b10809735490714d791df109aa3328a1e091\": container with ID starting with f933eb5223bf5c6f62c2c9ab3d18b10809735490714d791df109aa3328a1e091 not found: ID does not exist" Dec 03 12:46:31 crc kubenswrapper[5052]: I1203 12:46:31.683209 5052 scope.go:117] "RemoveContainer" containerID="dc1f7e238c595e37880087caf4c48af2813c1fa9c81659fabdd99d813fa482a6" Dec 03 12:46:31 crc kubenswrapper[5052]: I1203 12:46:31.683367 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dc1f7e238c595e37880087caf4c48af2813c1fa9c81659fabdd99d813fa482a6"} err="failed to get container status \"dc1f7e238c595e37880087caf4c48af2813c1fa9c81659fabdd99d813fa482a6\": rpc error: code = NotFound desc = could not find container \"dc1f7e238c595e37880087caf4c48af2813c1fa9c81659fabdd99d813fa482a6\": container with ID starting with dc1f7e238c595e37880087caf4c48af2813c1fa9c81659fabdd99d813fa482a6 not found: ID does not exist" Dec 03 12:46:31 crc kubenswrapper[5052]: I1203 12:46:31.683392 5052 scope.go:117] "RemoveContainer" containerID="1466275e6c25f9b46a0ec21cf1af15f3c40b914e854dc9a544b1dba27f630296" Dec 03 12:46:31 crc kubenswrapper[5052]: I1203 12:46:31.683631 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1466275e6c25f9b46a0ec21cf1af15f3c40b914e854dc9a544b1dba27f630296"} err="failed to get container status \"1466275e6c25f9b46a0ec21cf1af15f3c40b914e854dc9a544b1dba27f630296\": rpc error: code = NotFound desc = could not find container \"1466275e6c25f9b46a0ec21cf1af15f3c40b914e854dc9a544b1dba27f630296\": container with ID starting with 1466275e6c25f9b46a0ec21cf1af15f3c40b914e854dc9a544b1dba27f630296 not found: ID does not exist" Dec 03 12:46:31 crc kubenswrapper[5052]: I1203 12:46:31.683655 5052 scope.go:117] "RemoveContainer" containerID="8d6b67218da38e98daea9b518ff99007b1a85805172d6e47d9084b2c93ac9e4e" Dec 03 12:46:31 crc kubenswrapper[5052]: I1203 12:46:31.683814 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8d6b67218da38e98daea9b518ff99007b1a85805172d6e47d9084b2c93ac9e4e"} err="failed to get container status \"8d6b67218da38e98daea9b518ff99007b1a85805172d6e47d9084b2c93ac9e4e\": rpc error: code = NotFound desc = could not find container \"8d6b67218da38e98daea9b518ff99007b1a85805172d6e47d9084b2c93ac9e4e\": container with ID starting with 8d6b67218da38e98daea9b518ff99007b1a85805172d6e47d9084b2c93ac9e4e not found: ID does not exist" Dec 03 12:46:31 crc kubenswrapper[5052]: I1203 12:46:31.767061 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovsdbserver-sb-tls-certs\" (UniqueName: \"kubernetes.io/secret/c6c9b62e-bbbc-4557-92fd-da64bd8ee0cf-ovsdbserver-sb-tls-certs\") pod \"ovsdbserver-sb-0\" (UID: \"c6c9b62e-bbbc-4557-92fd-da64bd8ee0cf\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:46:31 crc kubenswrapper[5052]: E1203 12:46:31.767221 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-ovndbcluster-sb-ovndbs: secret "cert-ovndbcluster-sb-ovndbs" not found Dec 03 12:46:31 crc kubenswrapper[5052]: I1203 12:46:31.767280 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"nova-novncproxy-tls-certs\" (UniqueName: \"kubernetes.io/secret/9c996590-4337-477e-a236-3e834d192346-nova-novncproxy-tls-certs\") pod \"nova-cell1-novncproxy-0\" (UID: \"9c996590-4337-477e-a236-3e834d192346\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:46:31 crc kubenswrapper[5052]: E1203 12:46:31.767303 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c6c9b62e-bbbc-4557-92fd-da64bd8ee0cf-ovsdbserver-sb-tls-certs podName:c6c9b62e-bbbc-4557-92fd-da64bd8ee0cf nodeName:}" failed. No retries permitted until 2025-12-03 12:46:47.767280987 +0000 UTC m=+5198.355522751 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "ovsdbserver-sb-tls-certs" (UniqueName: "kubernetes.io/secret/c6c9b62e-bbbc-4557-92fd-da64bd8ee0cf-ovsdbserver-sb-tls-certs") pod "ovsdbserver-sb-0" (UID: "c6c9b62e-bbbc-4557-92fd-da64bd8ee0cf") : secret "cert-ovndbcluster-sb-ovndbs" not found Dec 03 12:46:31 crc kubenswrapper[5052]: E1203 12:46:31.767385 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-nova-novncproxy-cell1-public-svc: secret "cert-nova-novncproxy-cell1-public-svc" not found Dec 03 12:46:31 crc kubenswrapper[5052]: E1203 12:46:31.767433 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9c996590-4337-477e-a236-3e834d192346-nova-novncproxy-tls-certs podName:9c996590-4337-477e-a236-3e834d192346 nodeName:}" failed. No retries permitted until 2025-12-03 12:46:47.767419391 +0000 UTC m=+5198.355661155 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "nova-novncproxy-tls-certs" (UniqueName: "kubernetes.io/secret/9c996590-4337-477e-a236-3e834d192346-nova-novncproxy-tls-certs") pod "nova-cell1-novncproxy-0" (UID: "9c996590-4337-477e-a236-3e834d192346") : secret "cert-nova-novncproxy-cell1-public-svc" not found Dec 03 12:46:31 crc kubenswrapper[5052]: I1203 12:46:31.802197 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:46:31 crc kubenswrapper[5052]: I1203 12:46:31.816527 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:46:31 crc kubenswrapper[5052]: I1203 12:46:31.833597 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:46:31 crc kubenswrapper[5052]: E1203 12:46:31.834131 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="9481ae66-152c-41e7-9cbc-90dcb5fa6171" containerName="proxy-httpd" Dec 03 12:46:31 crc kubenswrapper[5052]: I1203 12:46:31.834158 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="9481ae66-152c-41e7-9cbc-90dcb5fa6171" containerName="proxy-httpd" Dec 03 12:46:31 crc kubenswrapper[5052]: E1203 12:46:31.834180 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="9481ae66-152c-41e7-9cbc-90dcb5fa6171" containerName="ceilometer-notification-agent" Dec 03 12:46:31 crc kubenswrapper[5052]: I1203 12:46:31.834189 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="9481ae66-152c-41e7-9cbc-90dcb5fa6171" containerName="ceilometer-notification-agent" Dec 03 12:46:31 crc kubenswrapper[5052]: E1203 12:46:31.834213 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="9481ae66-152c-41e7-9cbc-90dcb5fa6171" containerName="ceilometer-central-agent" Dec 03 12:46:31 crc kubenswrapper[5052]: I1203 12:46:31.834225 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="9481ae66-152c-41e7-9cbc-90dcb5fa6171" containerName="ceilometer-central-agent" Dec 03 12:46:31 crc kubenswrapper[5052]: E1203 12:46:31.834256 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="9481ae66-152c-41e7-9cbc-90dcb5fa6171" containerName="sg-core" Dec 03 12:46:31 crc kubenswrapper[5052]: I1203 12:46:31.834264 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="9481ae66-152c-41e7-9cbc-90dcb5fa6171" containerName="sg-core" Dec 03 12:46:31 crc kubenswrapper[5052]: I1203 12:46:31.834498 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="9481ae66-152c-41e7-9cbc-90dcb5fa6171" containerName="proxy-httpd" Dec 03 12:46:31 crc kubenswrapper[5052]: I1203 12:46:31.834542 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="9481ae66-152c-41e7-9cbc-90dcb5fa6171" containerName="sg-core" Dec 03 12:46:31 crc kubenswrapper[5052]: I1203 12:46:31.834562 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="9481ae66-152c-41e7-9cbc-90dcb5fa6171" containerName="ceilometer-central-agent" Dec 03 12:46:31 crc kubenswrapper[5052]: I1203 12:46:31.834577 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="9481ae66-152c-41e7-9cbc-90dcb5fa6171" containerName="ceilometer-notification-agent" Dec 03 12:46:31 crc kubenswrapper[5052]: I1203 12:46:31.836905 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:46:31 crc kubenswrapper[5052]: I1203 12:46:31.840983 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-scripts" Dec 03 12:46:31 crc kubenswrapper[5052]: I1203 12:46:31.841291 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-ceilometer-internal-svc" Dec 03 12:46:31 crc kubenswrapper[5052]: I1203 12:46:31.841457 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-config-data" Dec 03 12:46:31 crc kubenswrapper[5052]: I1203 12:46:31.854845 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:46:31 crc kubenswrapper[5052]: I1203 12:46:31.972235 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/1b7ff11c-2d16-40e6-98f5-5e4a9a78fed3-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"1b7ff11c-2d16-40e6-98f5-5e4a9a78fed3\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:46:31 crc kubenswrapper[5052]: I1203 12:46:31.972285 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j7g6z\" (UniqueName: \"kubernetes.io/projected/1b7ff11c-2d16-40e6-98f5-5e4a9a78fed3-kube-api-access-j7g6z\") pod \"ceilometer-0\" (UID: \"1b7ff11c-2d16-40e6-98f5-5e4a9a78fed3\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:46:31 crc kubenswrapper[5052]: I1203 12:46:31.972344 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/29c7d055-4f37-45a4-a608-eb73a361a86c-metrics-certs-tls-certs\") pod \"ovsdbserver-nb-0\" (UID: \"29c7d055-4f37-45a4-a608-eb73a361a86c\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:46:31 crc kubenswrapper[5052]: I1203 12:46:31.972463 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1b7ff11c-2d16-40e6-98f5-5e4a9a78fed3-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"1b7ff11c-2d16-40e6-98f5-5e4a9a78fed3\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:46:31 crc kubenswrapper[5052]: I1203 12:46:31.972497 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/1b7ff11c-2d16-40e6-98f5-5e4a9a78fed3-log-httpd\") pod \"ceilometer-0\" (UID: \"1b7ff11c-2d16-40e6-98f5-5e4a9a78fed3\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:46:31 crc kubenswrapper[5052]: I1203 12:46:31.972549 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/1b7ff11c-2d16-40e6-98f5-5e4a9a78fed3-run-httpd\") pod \"ceilometer-0\" (UID: \"1b7ff11c-2d16-40e6-98f5-5e4a9a78fed3\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:46:31 crc kubenswrapper[5052]: I1203 12:46:31.972595 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/1b7ff11c-2d16-40e6-98f5-5e4a9a78fed3-ceilometer-tls-certs\") pod \"ceilometer-0\" (UID: \"1b7ff11c-2d16-40e6-98f5-5e4a9a78fed3\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:46:31 crc kubenswrapper[5052]: I1203 12:46:31.972615 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovsdbserver-nb-tls-certs\" (UniqueName: \"kubernetes.io/secret/29c7d055-4f37-45a4-a608-eb73a361a86c-ovsdbserver-nb-tls-certs\") pod \"ovsdbserver-nb-0\" (UID: \"29c7d055-4f37-45a4-a608-eb73a361a86c\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:46:31 crc kubenswrapper[5052]: I1203 12:46:31.972648 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/1b7ff11c-2d16-40e6-98f5-5e4a9a78fed3-config-data\") pod \"ceilometer-0\" (UID: \"1b7ff11c-2d16-40e6-98f5-5e4a9a78fed3\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:46:31 crc kubenswrapper[5052]: I1203 12:46:31.972674 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/1b7ff11c-2d16-40e6-98f5-5e4a9a78fed3-scripts\") pod \"ceilometer-0\" (UID: \"1b7ff11c-2d16-40e6-98f5-5e4a9a78fed3\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:46:31 crc kubenswrapper[5052]: E1203 12:46:31.972645 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-ovn-metrics: secret "cert-ovn-metrics" not found Dec 03 12:46:31 crc kubenswrapper[5052]: E1203 12:46:31.972820 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/29c7d055-4f37-45a4-a608-eb73a361a86c-metrics-certs-tls-certs podName:29c7d055-4f37-45a4-a608-eb73a361a86c nodeName:}" failed. No retries permitted until 2025-12-03 12:46:47.972804665 +0000 UTC m=+5198.561046539 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs-tls-certs" (UniqueName: "kubernetes.io/secret/29c7d055-4f37-45a4-a608-eb73a361a86c-metrics-certs-tls-certs") pod "ovsdbserver-nb-0" (UID: "29c7d055-4f37-45a4-a608-eb73a361a86c") : secret "cert-ovn-metrics" not found Dec 03 12:46:31 crc kubenswrapper[5052]: E1203 12:46:31.972773 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-ovndbcluster-nb-ovndbs: secret "cert-ovndbcluster-nb-ovndbs" not found Dec 03 12:46:31 crc kubenswrapper[5052]: E1203 12:46:31.972960 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/29c7d055-4f37-45a4-a608-eb73a361a86c-ovsdbserver-nb-tls-certs podName:29c7d055-4f37-45a4-a608-eb73a361a86c nodeName:}" failed. No retries permitted until 2025-12-03 12:46:47.972936169 +0000 UTC m=+5198.561178013 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "ovsdbserver-nb-tls-certs" (UniqueName: "kubernetes.io/secret/29c7d055-4f37-45a4-a608-eb73a361a86c-ovsdbserver-nb-tls-certs") pod "ovsdbserver-nb-0" (UID: "29c7d055-4f37-45a4-a608-eb73a361a86c") : secret "cert-ovndbcluster-nb-ovndbs" not found Dec 03 12:46:32 crc kubenswrapper[5052]: I1203 12:46:32.075008 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/1b7ff11c-2d16-40e6-98f5-5e4a9a78fed3-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"1b7ff11c-2d16-40e6-98f5-5e4a9a78fed3\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:46:32 crc kubenswrapper[5052]: I1203 12:46:32.075321 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-j7g6z\" (UniqueName: \"kubernetes.io/projected/1b7ff11c-2d16-40e6-98f5-5e4a9a78fed3-kube-api-access-j7g6z\") pod \"ceilometer-0\" (UID: \"1b7ff11c-2d16-40e6-98f5-5e4a9a78fed3\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:46:32 crc kubenswrapper[5052]: I1203 12:46:32.075596 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1b7ff11c-2d16-40e6-98f5-5e4a9a78fed3-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"1b7ff11c-2d16-40e6-98f5-5e4a9a78fed3\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:46:32 crc kubenswrapper[5052]: I1203 12:46:32.075711 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/1b7ff11c-2d16-40e6-98f5-5e4a9a78fed3-log-httpd\") pod \"ceilometer-0\" (UID: \"1b7ff11c-2d16-40e6-98f5-5e4a9a78fed3\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:46:32 crc kubenswrapper[5052]: I1203 12:46:32.075832 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/1b7ff11c-2d16-40e6-98f5-5e4a9a78fed3-run-httpd\") pod \"ceilometer-0\" (UID: \"1b7ff11c-2d16-40e6-98f5-5e4a9a78fed3\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:46:32 crc kubenswrapper[5052]: I1203 12:46:32.075956 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/1b7ff11c-2d16-40e6-98f5-5e4a9a78fed3-ceilometer-tls-certs\") pod \"ceilometer-0\" (UID: \"1b7ff11c-2d16-40e6-98f5-5e4a9a78fed3\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:46:32 crc kubenswrapper[5052]: I1203 12:46:32.076088 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/1b7ff11c-2d16-40e6-98f5-5e4a9a78fed3-config-data\") pod \"ceilometer-0\" (UID: \"1b7ff11c-2d16-40e6-98f5-5e4a9a78fed3\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:46:32 crc kubenswrapper[5052]: I1203 12:46:32.076190 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/1b7ff11c-2d16-40e6-98f5-5e4a9a78fed3-scripts\") pod \"ceilometer-0\" (UID: \"1b7ff11c-2d16-40e6-98f5-5e4a9a78fed3\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:46:32 crc kubenswrapper[5052]: I1203 12:46:32.076556 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/1b7ff11c-2d16-40e6-98f5-5e4a9a78fed3-log-httpd\") pod \"ceilometer-0\" (UID: \"1b7ff11c-2d16-40e6-98f5-5e4a9a78fed3\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:46:32 crc kubenswrapper[5052]: I1203 12:46:32.077870 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/1b7ff11c-2d16-40e6-98f5-5e4a9a78fed3-run-httpd\") pod \"ceilometer-0\" (UID: \"1b7ff11c-2d16-40e6-98f5-5e4a9a78fed3\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:46:32 crc kubenswrapper[5052]: I1203 12:46:32.079152 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/1b7ff11c-2d16-40e6-98f5-5e4a9a78fed3-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"1b7ff11c-2d16-40e6-98f5-5e4a9a78fed3\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:46:32 crc kubenswrapper[5052]: I1203 12:46:32.080689 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1b7ff11c-2d16-40e6-98f5-5e4a9a78fed3-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"1b7ff11c-2d16-40e6-98f5-5e4a9a78fed3\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:46:32 crc kubenswrapper[5052]: I1203 12:46:32.080788 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/1b7ff11c-2d16-40e6-98f5-5e4a9a78fed3-ceilometer-tls-certs\") pod \"ceilometer-0\" (UID: \"1b7ff11c-2d16-40e6-98f5-5e4a9a78fed3\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:46:32 crc kubenswrapper[5052]: I1203 12:46:32.081769 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/1b7ff11c-2d16-40e6-98f5-5e4a9a78fed3-config-data\") pod \"ceilometer-0\" (UID: \"1b7ff11c-2d16-40e6-98f5-5e4a9a78fed3\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:46:32 crc kubenswrapper[5052]: I1203 12:46:32.087688 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/1b7ff11c-2d16-40e6-98f5-5e4a9a78fed3-scripts\") pod \"ceilometer-0\" (UID: \"1b7ff11c-2d16-40e6-98f5-5e4a9a78fed3\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:46:32 crc kubenswrapper[5052]: I1203 12:46:32.092738 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-j7g6z\" (UniqueName: \"kubernetes.io/projected/1b7ff11c-2d16-40e6-98f5-5e4a9a78fed3-kube-api-access-j7g6z\") pod \"ceilometer-0\" (UID: \"1b7ff11c-2d16-40e6-98f5-5e4a9a78fed3\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:46:32 crc kubenswrapper[5052]: I1203 12:46:32.159248 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:46:32 crc kubenswrapper[5052]: W1203 12:46:32.591488 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1b7ff11c_2d16_40e6_98f5_5e4a9a78fed3.slice/crio-1156d1f8f662a93773bdc584e55d1dba9a5eed86642ad79df3e23367662ed118 WatchSource:0}: Error finding container 1156d1f8f662a93773bdc584e55d1dba9a5eed86642ad79df3e23367662ed118: Status 404 returned error can't find the container with id 1156d1f8f662a93773bdc584e55d1dba9a5eed86642ad79df3e23367662ed118 Dec 03 12:46:32 crc kubenswrapper[5052]: I1203 12:46:32.595174 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:46:33 crc kubenswrapper[5052]: I1203 12:46:33.281543 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:46:33 crc kubenswrapper[5052]: I1203 12:46:33.281908 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:46:33 crc kubenswrapper[5052]: I1203 12:46:33.359096 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:46:33 crc kubenswrapper[5052]: I1203 12:46:33.494891 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"1b7ff11c-2d16-40e6-98f5-5e4a9a78fed3","Type":"ContainerStarted","Data":"feafdcdf212617a7513bd5d21b3d0f1577e3d60a91b5b9c44988b01a3e8d821f"} Dec 03 12:46:33 crc kubenswrapper[5052]: I1203 12:46:33.494955 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"1b7ff11c-2d16-40e6-98f5-5e4a9a78fed3","Type":"ContainerStarted","Data":"1156d1f8f662a93773bdc584e55d1dba9a5eed86642ad79df3e23367662ed118"} Dec 03 12:46:33 crc kubenswrapper[5052]: I1203 12:46:33.572983 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:46:33 crc kubenswrapper[5052]: I1203 12:46:33.691589 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9481ae66-152c-41e7-9cbc-90dcb5fa6171" path="/var/lib/kubelet/pods/9481ae66-152c-41e7-9cbc-90dcb5fa6171/volumes" Dec 03 12:46:34 crc kubenswrapper[5052]: I1203 12:46:34.121921 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"memcached-tls-certs\" (UniqueName: \"kubernetes.io/secret/4d42b068-9b91-4df6-9a9c-81cad009be16-memcached-tls-certs\") pod \"memcached-0\" (UID: \"4d42b068-9b91-4df6-9a9c-81cad009be16\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:46:34 crc kubenswrapper[5052]: I1203 12:46:34.128099 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"memcached-tls-certs\" (UniqueName: \"kubernetes.io/secret/4d42b068-9b91-4df6-9a9c-81cad009be16-memcached-tls-certs\") pod \"memcached-0\" (UID: \"4d42b068-9b91-4df6-9a9c-81cad009be16\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:46:34 crc kubenswrapper[5052]: I1203 12:46:34.209355 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/memcached-0" Dec 03 12:46:34 crc kubenswrapper[5052]: I1203 12:46:34.329854 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/576b2af7-7647-4a45-9423-3aa752d84e75-galera-tls-certs\") pod \"openstack-cell1-galera-0\" (UID: \"576b2af7-7647-4a45-9423-3aa752d84e75\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:46:34 crc kubenswrapper[5052]: E1203 12:46:34.330000 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-galera-openstack-cell1-svc: secret "cert-galera-openstack-cell1-svc" not found Dec 03 12:46:34 crc kubenswrapper[5052]: E1203 12:46:34.330329 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/576b2af7-7647-4a45-9423-3aa752d84e75-galera-tls-certs podName:576b2af7-7647-4a45-9423-3aa752d84e75 nodeName:}" failed. No retries permitted until 2025-12-03 12:46:50.330308715 +0000 UTC m=+5200.918550479 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "galera-tls-certs" (UniqueName: "kubernetes.io/secret/576b2af7-7647-4a45-9423-3aa752d84e75-galera-tls-certs") pod "openstack-cell1-galera-0" (UID: "576b2af7-7647-4a45-9423-3aa752d84e75") : secret "cert-galera-openstack-cell1-svc" not found Dec 03 12:46:34 crc kubenswrapper[5052]: I1203 12:46:34.510118 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"1b7ff11c-2d16-40e6-98f5-5e4a9a78fed3","Type":"ContainerStarted","Data":"c28f7df12f8d79aa250110ffaaddfd53cd8cd67650e84fc06d1f0a2c416a20e9"} Dec 03 12:46:34 crc kubenswrapper[5052]: I1203 12:46:34.678107 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/memcached-0"] Dec 03 12:46:34 crc kubenswrapper[5052]: I1203 12:46:34.861362 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/ed7f9920-a1e4-437b-8074-a30d0fe4450c-public-tls-certs\") pod \"nova-api-0\" (UID: \"ed7f9920-a1e4-437b-8074-a30d0fe4450c\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:46:34 crc kubenswrapper[5052]: I1203 12:46:34.861492 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/ed7f9920-a1e4-437b-8074-a30d0fe4450c-internal-tls-certs\") pod \"nova-api-0\" (UID: \"ed7f9920-a1e4-437b-8074-a30d0fe4450c\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:46:34 crc kubenswrapper[5052]: E1203 12:46:34.861539 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-nova-public-svc: secret "cert-nova-public-svc" not found Dec 03 12:46:34 crc kubenswrapper[5052]: E1203 12:46:34.861593 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ed7f9920-a1e4-437b-8074-a30d0fe4450c-public-tls-certs podName:ed7f9920-a1e4-437b-8074-a30d0fe4450c nodeName:}" failed. No retries permitted until 2025-12-03 12:46:50.861578069 +0000 UTC m=+5201.449819833 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "public-tls-certs" (UniqueName: "kubernetes.io/secret/ed7f9920-a1e4-437b-8074-a30d0fe4450c-public-tls-certs") pod "nova-api-0" (UID: "ed7f9920-a1e4-437b-8074-a30d0fe4450c") : secret "cert-nova-public-svc" not found Dec 03 12:46:34 crc kubenswrapper[5052]: E1203 12:46:34.861849 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-swift-public-svc: secret "cert-swift-public-svc" not found Dec 03 12:46:34 crc kubenswrapper[5052]: E1203 12:46:34.861870 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/242be027-6461-4aea-ae7c-e52ca996b2ea-public-tls-certs podName:242be027-6461-4aea-ae7c-e52ca996b2ea nodeName:}" failed. No retries permitted until 2025-12-03 12:46:50.861864097 +0000 UTC m=+5201.450105861 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "public-tls-certs" (UniqueName: "kubernetes.io/secret/242be027-6461-4aea-ae7c-e52ca996b2ea-public-tls-certs") pod "swift-proxy-dd4cf57c8-x4c45" (UID: "242be027-6461-4aea-ae7c-e52ca996b2ea") : secret "cert-swift-public-svc" not found Dec 03 12:46:34 crc kubenswrapper[5052]: E1203 12:46:34.861901 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-swift-internal-svc: secret "cert-swift-internal-svc" not found Dec 03 12:46:34 crc kubenswrapper[5052]: E1203 12:46:34.861917 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/242be027-6461-4aea-ae7c-e52ca996b2ea-internal-tls-certs podName:242be027-6461-4aea-ae7c-e52ca996b2ea nodeName:}" failed. No retries permitted until 2025-12-03 12:46:50.861912578 +0000 UTC m=+5201.450154342 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "internal-tls-certs" (UniqueName: "kubernetes.io/secret/242be027-6461-4aea-ae7c-e52ca996b2ea-internal-tls-certs") pod "swift-proxy-dd4cf57c8-x4c45" (UID: "242be027-6461-4aea-ae7c-e52ca996b2ea") : secret "cert-swift-internal-svc" not found Dec 03 12:46:34 crc kubenswrapper[5052]: I1203 12:46:34.866586 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/ed7f9920-a1e4-437b-8074-a30d0fe4450c-internal-tls-certs\") pod \"nova-api-0\" (UID: \"ed7f9920-a1e4-437b-8074-a30d0fe4450c\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:46:34 crc kubenswrapper[5052]: I1203 12:46:34.868750 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/cinder-scheduler-0" podUID="aa557411-6b0a-4596-a5f5-0e23f8686dc4" containerName="cinder-scheduler" probeResult="failure" output="Get \"http://10.217.0.50:8080/\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Dec 03 12:46:34 crc kubenswrapper[5052]: E1203 12:46:34.964245 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-placement-internal-svc: secret "cert-placement-internal-svc" not found Dec 03 12:46:34 crc kubenswrapper[5052]: E1203 12:46:34.964306 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-placement-public-svc: secret "cert-placement-public-svc" not found Dec 03 12:46:34 crc kubenswrapper[5052]: E1203 12:46:34.964326 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/32f420c2-aa7e-44c8-a63a-7654aa3e6e34-internal-tls-certs podName:32f420c2-aa7e-44c8-a63a-7654aa3e6e34 nodeName:}" failed. No retries permitted until 2025-12-03 12:46:50.964307467 +0000 UTC m=+5201.552549231 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "internal-tls-certs" (UniqueName: "kubernetes.io/secret/32f420c2-aa7e-44c8-a63a-7654aa3e6e34-internal-tls-certs") pod "placement-bbcfbbc9d-5q6gx" (UID: "32f420c2-aa7e-44c8-a63a-7654aa3e6e34") : secret "cert-placement-internal-svc" not found Dec 03 12:46:34 crc kubenswrapper[5052]: E1203 12:46:34.964349 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/32f420c2-aa7e-44c8-a63a-7654aa3e6e34-public-tls-certs podName:32f420c2-aa7e-44c8-a63a-7654aa3e6e34 nodeName:}" failed. No retries permitted until 2025-12-03 12:46:50.964338108 +0000 UTC m=+5201.552579872 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "public-tls-certs" (UniqueName: "kubernetes.io/secret/32f420c2-aa7e-44c8-a63a-7654aa3e6e34-public-tls-certs") pod "placement-bbcfbbc9d-5q6gx" (UID: "32f420c2-aa7e-44c8-a63a-7654aa3e6e34") : secret "cert-placement-public-svc" not found Dec 03 12:46:35 crc kubenswrapper[5052]: I1203 12:46:35.168875 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/bf8b9b9c-b8f9-4c93-8112-43913950fcdf-internal-tls-certs\") pod \"glance-default-internal-api-0\" (UID: \"bf8b9b9c-b8f9-4c93-8112-43913950fcdf\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:46:35 crc kubenswrapper[5052]: E1203 12:46:35.169098 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-glance-default-internal-svc: secret "cert-glance-default-internal-svc" not found Dec 03 12:46:35 crc kubenswrapper[5052]: E1203 12:46:35.169257 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bf8b9b9c-b8f9-4c93-8112-43913950fcdf-internal-tls-certs podName:bf8b9b9c-b8f9-4c93-8112-43913950fcdf nodeName:}" failed. No retries permitted until 2025-12-03 12:46:51.169238567 +0000 UTC m=+5201.757480331 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "internal-tls-certs" (UniqueName: "kubernetes.io/secret/bf8b9b9c-b8f9-4c93-8112-43913950fcdf-internal-tls-certs") pod "glance-default-internal-api-0" (UID: "bf8b9b9c-b8f9-4c93-8112-43913950fcdf") : secret "cert-glance-default-internal-svc" not found Dec 03 12:46:35 crc kubenswrapper[5052]: E1203 12:46:35.475912 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-barbican-internal-svc: secret "cert-barbican-internal-svc" not found Dec 03 12:46:35 crc kubenswrapper[5052]: E1203 12:46:35.476101 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5b63f869-c95a-4279-b542-3ebbec1a1fd5-internal-tls-certs podName:5b63f869-c95a-4279-b542-3ebbec1a1fd5 nodeName:}" failed. No retries permitted until 2025-12-03 12:46:43.476072742 +0000 UTC m=+5194.064314506 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "internal-tls-certs" (UniqueName: "kubernetes.io/secret/5b63f869-c95a-4279-b542-3ebbec1a1fd5-internal-tls-certs") pod "barbican-api-6d86f7b884-lf4pd" (UID: "5b63f869-c95a-4279-b542-3ebbec1a1fd5") : secret "cert-barbican-internal-svc" not found Dec 03 12:46:35 crc kubenswrapper[5052]: I1203 12:46:35.520848 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"1b7ff11c-2d16-40e6-98f5-5e4a9a78fed3","Type":"ContainerStarted","Data":"9a89f3d83f92ed5f8d9b084b364bbf81c57e64d6a73001585da1afa98e2cb7cb"} Dec 03 12:46:35 crc kubenswrapper[5052]: I1203 12:46:35.522455 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/memcached-0" event={"ID":"4d42b068-9b91-4df6-9a9c-81cad009be16","Type":"ContainerStarted","Data":"c222a3a51a9f475798e3bef53ce522fe0044f6c1d5020c5fa20ba4e43d10b518"} Dec 03 12:46:35 crc kubenswrapper[5052]: I1203 12:46:35.522479 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/memcached-0" event={"ID":"4d42b068-9b91-4df6-9a9c-81cad009be16","Type":"ContainerStarted","Data":"7587e638e7960d1cbb26b56db77c21d6d610e0e260a612a862c6e8601775a5ff"} Dec 03 12:46:35 crc kubenswrapper[5052]: I1203 12:46:35.522669 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/memcached-0" Dec 03 12:46:35 crc kubenswrapper[5052]: I1203 12:46:35.543299 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/memcached-0" podStartSLOduration=18.543279104 podStartE2EDuration="18.543279104s" podCreationTimestamp="2025-12-03 12:46:17 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:46:35.54031632 +0000 UTC m=+5186.128558084" watchObservedRunningTime="2025-12-03 12:46:35.543279104 +0000 UTC m=+5186.131520868" Dec 03 12:46:36 crc kubenswrapper[5052]: I1203 12:46:36.533097 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"1b7ff11c-2d16-40e6-98f5-5e4a9a78fed3","Type":"ContainerStarted","Data":"91120088947d8c381c8640229b02472087109db20e7308332ea507f1841d751b"} Dec 03 12:46:36 crc kubenswrapper[5052]: I1203 12:46:36.534473 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:46:36 crc kubenswrapper[5052]: I1203 12:46:36.564218 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/ceilometer-0" podStartSLOduration=2.297968922 podStartE2EDuration="5.56419608s" podCreationTimestamp="2025-12-03 12:46:31 +0000 UTC" firstStartedPulling="2025-12-03 12:46:32.594548825 +0000 UTC m=+5183.182790589" lastFinishedPulling="2025-12-03 12:46:35.860775983 +0000 UTC m=+5186.449017747" observedRunningTime="2025-12-03 12:46:36.554281507 +0000 UTC m=+5187.142523271" watchObservedRunningTime="2025-12-03 12:46:36.56419608 +0000 UTC m=+5187.152437844" Dec 03 12:46:37 crc kubenswrapper[5052]: I1203 12:46:37.186422 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:46:37 crc kubenswrapper[5052]: I1203 12:46:37.186815 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:46:37 crc kubenswrapper[5052]: I1203 12:46:37.221807 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:46:37 crc kubenswrapper[5052]: I1203 12:46:37.236347 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:46:37 crc kubenswrapper[5052]: I1203 12:46:37.544207 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:46:37 crc kubenswrapper[5052]: I1203 12:46:37.545503 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:46:38 crc kubenswrapper[5052]: I1203 12:46:38.819794 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-nb-0"] Dec 03 12:46:38 crc kubenswrapper[5052]: E1203 12:46:38.820927 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs-tls-certs ovsdbserver-nb-tls-certs], unattached volumes=[], failed to process volumes=[]: context canceled" pod="openstack-kuttl-tests/ovsdbserver-nb-0" podUID="29c7d055-4f37-45a4-a608-eb73a361a86c" Dec 03 12:46:39 crc kubenswrapper[5052]: I1203 12:46:39.210754 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/memcached-0" Dec 03 12:46:39 crc kubenswrapper[5052]: I1203 12:46:39.263553 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/memcached-0"] Dec 03 12:46:39 crc kubenswrapper[5052]: I1203 12:46:39.308357 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/f8ab3054-5998-40c1-852f-be0dbaf022e5-metrics-certs-tls-certs\") pod \"ovn-northd-0\" (UID: \"f8ab3054-5998-40c1-852f-be0dbaf022e5\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:46:39 crc kubenswrapper[5052]: I1203 12:46:39.310257 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovn-northd-tls-certs\" (UniqueName: \"kubernetes.io/secret/f8ab3054-5998-40c1-852f-be0dbaf022e5-ovn-northd-tls-certs\") pod \"ovn-northd-0\" (UID: \"f8ab3054-5998-40c1-852f-be0dbaf022e5\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:46:39 crc kubenswrapper[5052]: E1203 12:46:39.310645 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-ovnnorthd-ovndbs: secret "cert-ovnnorthd-ovndbs" not found Dec 03 12:46:39 crc kubenswrapper[5052]: E1203 12:46:39.310735 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f8ab3054-5998-40c1-852f-be0dbaf022e5-ovn-northd-tls-certs podName:f8ab3054-5998-40c1-852f-be0dbaf022e5 nodeName:}" failed. No retries permitted until 2025-12-03 12:46:55.310713616 +0000 UTC m=+5205.898955410 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "ovn-northd-tls-certs" (UniqueName: "kubernetes.io/secret/f8ab3054-5998-40c1-852f-be0dbaf022e5-ovn-northd-tls-certs") pod "ovn-northd-0" (UID: "f8ab3054-5998-40c1-852f-be0dbaf022e5") : secret "cert-ovnnorthd-ovndbs" not found Dec 03 12:46:39 crc kubenswrapper[5052]: I1203 12:46:39.334162 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/f8ab3054-5998-40c1-852f-be0dbaf022e5-metrics-certs-tls-certs\") pod \"ovn-northd-0\" (UID: \"f8ab3054-5998-40c1-852f-be0dbaf022e5\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:46:39 crc kubenswrapper[5052]: I1203 12:46:39.377499 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 12:46:39 crc kubenswrapper[5052]: E1203 12:46:39.378658 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[nova-novncproxy-tls-certs vencrypt-tls-certs], unattached volumes=[], failed to process volumes=[]: context canceled" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" podUID="9c996590-4337-477e-a236-3e834d192346" Dec 03 12:46:39 crc kubenswrapper[5052]: I1203 12:46:39.394643 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:46:39 crc kubenswrapper[5052]: E1203 12:46:39.395447 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[public-tls-certs], unattached volumes=[], failed to process volumes=[]: context canceled" pod="openstack-kuttl-tests/nova-api-0" podUID="ed7f9920-a1e4-437b-8074-a30d0fe4450c" Dec 03 12:46:39 crc kubenswrapper[5052]: I1203 12:46:39.436872 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-sb-0"] Dec 03 12:46:39 crc kubenswrapper[5052]: E1203 12:46:39.438031 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs-tls-certs ovsdbserver-sb-tls-certs], unattached volumes=[], failed to process volumes=[]: context canceled" pod="openstack-kuttl-tests/ovsdbserver-sb-0" podUID="c6c9b62e-bbbc-4557-92fd-da64bd8ee0cf" Dec 03 12:46:39 crc kubenswrapper[5052]: I1203 12:46:39.489494 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-0"] Dec 03 12:46:39 crc kubenswrapper[5052]: I1203 12:46:39.490035 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-cell1-conductor-0" podUID="320f928d-d6b0-48b4-b315-1a6c2ecd3380" containerName="nova-cell1-conductor-conductor" containerID="cri-o://2ff6796b6955202076b9ca76f2bd9d074d96d820f421983058345a0af1b54565" gracePeriod=30 Dec 03 12:46:39 crc kubenswrapper[5052]: I1203 12:46:39.561643 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:46:39 crc kubenswrapper[5052]: I1203 12:46:39.561761 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/memcached-0" podUID="4d42b068-9b91-4df6-9a9c-81cad009be16" containerName="memcached" containerID="cri-o://c222a3a51a9f475798e3bef53ce522fe0044f6c1d5020c5fa20ba4e43d10b518" gracePeriod=30 Dec 03 12:46:39 crc kubenswrapper[5052]: I1203 12:46:39.561793 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:46:39 crc kubenswrapper[5052]: I1203 12:46:39.561666 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:46:39 crc kubenswrapper[5052]: I1203 12:46:39.562194 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:46:39 crc kubenswrapper[5052]: I1203 12:46:39.576203 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:46:39 crc kubenswrapper[5052]: I1203 12:46:39.584472 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:46:39 crc kubenswrapper[5052]: I1203 12:46:39.594805 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:46:39 crc kubenswrapper[5052]: I1203 12:46:39.601340 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:46:39 crc kubenswrapper[5052]: I1203 12:46:39.622067 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/neutron-6d64d7894b-xp9d5"] Dec 03 12:46:39 crc kubenswrapper[5052]: E1203 12:46:39.623234 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[ovndb-tls-certs public-tls-certs], unattached volumes=[], failed to process volumes=[]: context canceled" pod="openstack-kuttl-tests/neutron-6d64d7894b-xp9d5" podUID="901a2f0b-fb8e-4c97-aea7-567d7ee42a23" Dec 03 12:46:39 crc kubenswrapper[5052]: I1203 12:46:39.657736 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/neutron-bcd9f4f4-djpt8"] Dec 03 12:46:39 crc kubenswrapper[5052]: I1203 12:46:39.659853 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-bcd9f4f4-djpt8" Dec 03 12:46:39 crc kubenswrapper[5052]: I1203 12:46:39.709702 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-bcd9f4f4-djpt8"] Dec 03 12:46:39 crc kubenswrapper[5052]: I1203 12:46:39.718626 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c6c9b62e-bbbc-4557-92fd-da64bd8ee0cf-config\") pod \"c6c9b62e-bbbc-4557-92fd-da64bd8ee0cf\" (UID: \"c6c9b62e-bbbc-4557-92fd-da64bd8ee0cf\") " Dec 03 12:46:39 crc kubenswrapper[5052]: I1203 12:46:39.718690 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9c996590-4337-477e-a236-3e834d192346-combined-ca-bundle\") pod \"9c996590-4337-477e-a236-3e834d192346\" (UID: \"9c996590-4337-477e-a236-3e834d192346\") " Dec 03 12:46:39 crc kubenswrapper[5052]: I1203 12:46:39.718748 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/9c996590-4337-477e-a236-3e834d192346-config-data\") pod \"9c996590-4337-477e-a236-3e834d192346\" (UID: \"9c996590-4337-477e-a236-3e834d192346\") " Dec 03 12:46:39 crc kubenswrapper[5052]: I1203 12:46:39.718782 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/29c7d055-4f37-45a4-a608-eb73a361a86c-config\") pod \"29c7d055-4f37-45a4-a608-eb73a361a86c\" (UID: \"29c7d055-4f37-45a4-a608-eb73a361a86c\") " Dec 03 12:46:39 crc kubenswrapper[5052]: I1203 12:46:39.718821 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/ed7f9920-a1e4-437b-8074-a30d0fe4450c-config-data\") pod \"ed7f9920-a1e4-437b-8074-a30d0fe4450c\" (UID: \"ed7f9920-a1e4-437b-8074-a30d0fe4450c\") " Dec 03 12:46:39 crc kubenswrapper[5052]: I1203 12:46:39.718866 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/29c7d055-4f37-45a4-a608-eb73a361a86c-combined-ca-bundle\") pod \"29c7d055-4f37-45a4-a608-eb73a361a86c\" (UID: \"29c7d055-4f37-45a4-a608-eb73a361a86c\") " Dec 03 12:46:39 crc kubenswrapper[5052]: I1203 12:46:39.718901 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9zmq4\" (UniqueName: \"kubernetes.io/projected/29c7d055-4f37-45a4-a608-eb73a361a86c-kube-api-access-9zmq4\") pod \"29c7d055-4f37-45a4-a608-eb73a361a86c\" (UID: \"29c7d055-4f37-45a4-a608-eb73a361a86c\") " Dec 03 12:46:39 crc kubenswrapper[5052]: I1203 12:46:39.718946 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r8cvt\" (UniqueName: \"kubernetes.io/projected/ed7f9920-a1e4-437b-8074-a30d0fe4450c-kube-api-access-r8cvt\") pod \"ed7f9920-a1e4-437b-8074-a30d0fe4450c\" (UID: \"ed7f9920-a1e4-437b-8074-a30d0fe4450c\") " Dec 03 12:46:39 crc kubenswrapper[5052]: I1203 12:46:39.718976 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ed7f9920-a1e4-437b-8074-a30d0fe4450c-combined-ca-bundle\") pod \"ed7f9920-a1e4-437b-8074-a30d0fe4450c\" (UID: \"ed7f9920-a1e4-437b-8074-a30d0fe4450c\") " Dec 03 12:46:39 crc kubenswrapper[5052]: I1203 12:46:39.719005 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovndbcluster-nb-etc-ovn\" (UniqueName: \"kubernetes.io/local-volume/local-storage01-crc\") pod \"29c7d055-4f37-45a4-a608-eb73a361a86c\" (UID: \"29c7d055-4f37-45a4-a608-eb73a361a86c\") " Dec 03 12:46:39 crc kubenswrapper[5052]: I1203 12:46:39.719115 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/29c7d055-4f37-45a4-a608-eb73a361a86c-scripts\") pod \"29c7d055-4f37-45a4-a608-eb73a361a86c\" (UID: \"29c7d055-4f37-45a4-a608-eb73a361a86c\") " Dec 03 12:46:39 crc kubenswrapper[5052]: I1203 12:46:39.719158 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/ed7f9920-a1e4-437b-8074-a30d0fe4450c-logs\") pod \"ed7f9920-a1e4-437b-8074-a30d0fe4450c\" (UID: \"ed7f9920-a1e4-437b-8074-a30d0fe4450c\") " Dec 03 12:46:39 crc kubenswrapper[5052]: I1203 12:46:39.719196 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/c6c9b62e-bbbc-4557-92fd-da64bd8ee0cf-ovsdb-rundir\") pod \"c6c9b62e-bbbc-4557-92fd-da64bd8ee0cf\" (UID: \"c6c9b62e-bbbc-4557-92fd-da64bd8ee0cf\") " Dec 03 12:46:39 crc kubenswrapper[5052]: I1203 12:46:39.719241 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/ed7f9920-a1e4-437b-8074-a30d0fe4450c-internal-tls-certs\") pod \"ed7f9920-a1e4-437b-8074-a30d0fe4450c\" (UID: \"ed7f9920-a1e4-437b-8074-a30d0fe4450c\") " Dec 03 12:46:39 crc kubenswrapper[5052]: I1203 12:46:39.719286 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9bkcr\" (UniqueName: \"kubernetes.io/projected/c6c9b62e-bbbc-4557-92fd-da64bd8ee0cf-kube-api-access-9bkcr\") pod \"c6c9b62e-bbbc-4557-92fd-da64bd8ee0cf\" (UID: \"c6c9b62e-bbbc-4557-92fd-da64bd8ee0cf\") " Dec 03 12:46:39 crc kubenswrapper[5052]: I1203 12:46:39.719332 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c6c9b62e-bbbc-4557-92fd-da64bd8ee0cf-combined-ca-bundle\") pod \"c6c9b62e-bbbc-4557-92fd-da64bd8ee0cf\" (UID: \"c6c9b62e-bbbc-4557-92fd-da64bd8ee0cf\") " Dec 03 12:46:39 crc kubenswrapper[5052]: I1203 12:46:39.719364 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/29c7d055-4f37-45a4-a608-eb73a361a86c-ovsdb-rundir\") pod \"29c7d055-4f37-45a4-a608-eb73a361a86c\" (UID: \"29c7d055-4f37-45a4-a608-eb73a361a86c\") " Dec 03 12:46:39 crc kubenswrapper[5052]: I1203 12:46:39.719388 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovndbcluster-sb-etc-ovn\" (UniqueName: \"kubernetes.io/local-volume/local-storage09-crc\") pod \"c6c9b62e-bbbc-4557-92fd-da64bd8ee0cf\" (UID: \"c6c9b62e-bbbc-4557-92fd-da64bd8ee0cf\") " Dec 03 12:46:39 crc kubenswrapper[5052]: I1203 12:46:39.719427 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qbgbz\" (UniqueName: \"kubernetes.io/projected/9c996590-4337-477e-a236-3e834d192346-kube-api-access-qbgbz\") pod \"9c996590-4337-477e-a236-3e834d192346\" (UID: \"9c996590-4337-477e-a236-3e834d192346\") " Dec 03 12:46:39 crc kubenswrapper[5052]: I1203 12:46:39.719471 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/c6c9b62e-bbbc-4557-92fd-da64bd8ee0cf-scripts\") pod \"c6c9b62e-bbbc-4557-92fd-da64bd8ee0cf\" (UID: \"c6c9b62e-bbbc-4557-92fd-da64bd8ee0cf\") " Dec 03 12:46:39 crc kubenswrapper[5052]: I1203 12:46:39.721235 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c6c9b62e-bbbc-4557-92fd-da64bd8ee0cf-ovsdb-rundir" (OuterVolumeSpecName: "ovsdb-rundir") pod "c6c9b62e-bbbc-4557-92fd-da64bd8ee0cf" (UID: "c6c9b62e-bbbc-4557-92fd-da64bd8ee0cf"). InnerVolumeSpecName "ovsdb-rundir". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:46:39 crc kubenswrapper[5052]: I1203 12:46:39.721887 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/29c7d055-4f37-45a4-a608-eb73a361a86c-config" (OuterVolumeSpecName: "config") pod "29c7d055-4f37-45a4-a608-eb73a361a86c" (UID: "29c7d055-4f37-45a4-a608-eb73a361a86c"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:46:39 crc kubenswrapper[5052]: I1203 12:46:39.721891 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29c7d055-4f37-45a4-a608-eb73a361a86c-ovsdb-rundir" (OuterVolumeSpecName: "ovsdb-rundir") pod "29c7d055-4f37-45a4-a608-eb73a361a86c" (UID: "29c7d055-4f37-45a4-a608-eb73a361a86c"). InnerVolumeSpecName "ovsdb-rundir". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:46:39 crc kubenswrapper[5052]: I1203 12:46:39.722079 5052 reconciler_common.go:293] "Volume detached for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/c6c9b62e-bbbc-4557-92fd-da64bd8ee0cf-ovsdb-rundir\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:39 crc kubenswrapper[5052]: I1203 12:46:39.722492 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c6c9b62e-bbbc-4557-92fd-da64bd8ee0cf-scripts" (OuterVolumeSpecName: "scripts") pod "c6c9b62e-bbbc-4557-92fd-da64bd8ee0cf" (UID: "c6c9b62e-bbbc-4557-92fd-da64bd8ee0cf"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:46:39 crc kubenswrapper[5052]: I1203 12:46:39.723681 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed7f9920-a1e4-437b-8074-a30d0fe4450c-logs" (OuterVolumeSpecName: "logs") pod "ed7f9920-a1e4-437b-8074-a30d0fe4450c" (UID: "ed7f9920-a1e4-437b-8074-a30d0fe4450c"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:46:39 crc kubenswrapper[5052]: I1203 12:46:39.740910 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c6c9b62e-bbbc-4557-92fd-da64bd8ee0cf-config" (OuterVolumeSpecName: "config") pod "c6c9b62e-bbbc-4557-92fd-da64bd8ee0cf" (UID: "c6c9b62e-bbbc-4557-92fd-da64bd8ee0cf"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:46:39 crc kubenswrapper[5052]: I1203 12:46:39.741273 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/29c7d055-4f37-45a4-a608-eb73a361a86c-scripts" (OuterVolumeSpecName: "scripts") pod "29c7d055-4f37-45a4-a608-eb73a361a86c" (UID: "29c7d055-4f37-45a4-a608-eb73a361a86c"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:46:39 crc kubenswrapper[5052]: I1203 12:46:39.772602 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ovn-northd-0"] Dec 03 12:46:39 crc kubenswrapper[5052]: E1203 12:46:39.773550 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[ovn-northd-tls-certs], unattached volumes=[], failed to process volumes=[]: context canceled" pod="openstack-kuttl-tests/ovn-northd-0" podUID="f8ab3054-5998-40c1-852f-be0dbaf022e5" Dec 03 12:46:39 crc kubenswrapper[5052]: I1203 12:46:39.824612 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/049a231b-3a12-4710-9b7a-117b1bb82e72-public-tls-certs\") pod \"neutron-bcd9f4f4-djpt8\" (UID: \"049a231b-3a12-4710-9b7a-117b1bb82e72\") " pod="openstack-kuttl-tests/neutron-bcd9f4f4-djpt8" Dec 03 12:46:39 crc kubenswrapper[5052]: I1203 12:46:39.824692 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zrb86\" (UniqueName: \"kubernetes.io/projected/049a231b-3a12-4710-9b7a-117b1bb82e72-kube-api-access-zrb86\") pod \"neutron-bcd9f4f4-djpt8\" (UID: \"049a231b-3a12-4710-9b7a-117b1bb82e72\") " pod="openstack-kuttl-tests/neutron-bcd9f4f4-djpt8" Dec 03 12:46:39 crc kubenswrapper[5052]: I1203 12:46:39.824726 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/049a231b-3a12-4710-9b7a-117b1bb82e72-internal-tls-certs\") pod \"neutron-bcd9f4f4-djpt8\" (UID: \"049a231b-3a12-4710-9b7a-117b1bb82e72\") " pod="openstack-kuttl-tests/neutron-bcd9f4f4-djpt8" Dec 03 12:46:39 crc kubenswrapper[5052]: I1203 12:46:39.824761 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/049a231b-3a12-4710-9b7a-117b1bb82e72-httpd-config\") pod \"neutron-bcd9f4f4-djpt8\" (UID: \"049a231b-3a12-4710-9b7a-117b1bb82e72\") " pod="openstack-kuttl-tests/neutron-bcd9f4f4-djpt8" Dec 03 12:46:39 crc kubenswrapper[5052]: I1203 12:46:39.824791 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/049a231b-3a12-4710-9b7a-117b1bb82e72-combined-ca-bundle\") pod \"neutron-bcd9f4f4-djpt8\" (UID: \"049a231b-3a12-4710-9b7a-117b1bb82e72\") " pod="openstack-kuttl-tests/neutron-bcd9f4f4-djpt8" Dec 03 12:46:39 crc kubenswrapper[5052]: I1203 12:46:39.824920 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/049a231b-3a12-4710-9b7a-117b1bb82e72-config\") pod \"neutron-bcd9f4f4-djpt8\" (UID: \"049a231b-3a12-4710-9b7a-117b1bb82e72\") " pod="openstack-kuttl-tests/neutron-bcd9f4f4-djpt8" Dec 03 12:46:39 crc kubenswrapper[5052]: I1203 12:46:39.824958 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/049a231b-3a12-4710-9b7a-117b1bb82e72-ovndb-tls-certs\") pod \"neutron-bcd9f4f4-djpt8\" (UID: \"049a231b-3a12-4710-9b7a-117b1bb82e72\") " pod="openstack-kuttl-tests/neutron-bcd9f4f4-djpt8" Dec 03 12:46:39 crc kubenswrapper[5052]: I1203 12:46:39.825040 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/29c7d055-4f37-45a4-a608-eb73a361a86c-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:39 crc kubenswrapper[5052]: I1203 12:46:39.825052 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/29c7d055-4f37-45a4-a608-eb73a361a86c-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:39 crc kubenswrapper[5052]: I1203 12:46:39.825061 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/ed7f9920-a1e4-437b-8074-a30d0fe4450c-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:39 crc kubenswrapper[5052]: I1203 12:46:39.825070 5052 reconciler_common.go:293] "Volume detached for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/29c7d055-4f37-45a4-a608-eb73a361a86c-ovsdb-rundir\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:39 crc kubenswrapper[5052]: I1203 12:46:39.825081 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/c6c9b62e-bbbc-4557-92fd-da64bd8ee0cf-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:39 crc kubenswrapper[5052]: I1203 12:46:39.825091 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c6c9b62e-bbbc-4557-92fd-da64bd8ee0cf-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:39 crc kubenswrapper[5052]: I1203 12:46:39.911703 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/cinder-scheduler-0" podUID="aa557411-6b0a-4596-a5f5-0e23f8686dc4" containerName="cinder-scheduler" probeResult="failure" output="Get \"http://10.217.0.50:8080/\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Dec 03 12:46:39 crc kubenswrapper[5052]: I1203 12:46:39.926445 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/049a231b-3a12-4710-9b7a-117b1bb82e72-public-tls-certs\") pod \"neutron-bcd9f4f4-djpt8\" (UID: \"049a231b-3a12-4710-9b7a-117b1bb82e72\") " pod="openstack-kuttl-tests/neutron-bcd9f4f4-djpt8" Dec 03 12:46:39 crc kubenswrapper[5052]: I1203 12:46:39.926554 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-zrb86\" (UniqueName: \"kubernetes.io/projected/049a231b-3a12-4710-9b7a-117b1bb82e72-kube-api-access-zrb86\") pod \"neutron-bcd9f4f4-djpt8\" (UID: \"049a231b-3a12-4710-9b7a-117b1bb82e72\") " pod="openstack-kuttl-tests/neutron-bcd9f4f4-djpt8" Dec 03 12:46:39 crc kubenswrapper[5052]: I1203 12:46:39.926602 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/049a231b-3a12-4710-9b7a-117b1bb82e72-internal-tls-certs\") pod \"neutron-bcd9f4f4-djpt8\" (UID: \"049a231b-3a12-4710-9b7a-117b1bb82e72\") " pod="openstack-kuttl-tests/neutron-bcd9f4f4-djpt8" Dec 03 12:46:39 crc kubenswrapper[5052]: I1203 12:46:39.926646 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/049a231b-3a12-4710-9b7a-117b1bb82e72-httpd-config\") pod \"neutron-bcd9f4f4-djpt8\" (UID: \"049a231b-3a12-4710-9b7a-117b1bb82e72\") " pod="openstack-kuttl-tests/neutron-bcd9f4f4-djpt8" Dec 03 12:46:39 crc kubenswrapper[5052]: I1203 12:46:39.926681 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/049a231b-3a12-4710-9b7a-117b1bb82e72-combined-ca-bundle\") pod \"neutron-bcd9f4f4-djpt8\" (UID: \"049a231b-3a12-4710-9b7a-117b1bb82e72\") " pod="openstack-kuttl-tests/neutron-bcd9f4f4-djpt8" Dec 03 12:46:39 crc kubenswrapper[5052]: I1203 12:46:39.926810 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/049a231b-3a12-4710-9b7a-117b1bb82e72-config\") pod \"neutron-bcd9f4f4-djpt8\" (UID: \"049a231b-3a12-4710-9b7a-117b1bb82e72\") " pod="openstack-kuttl-tests/neutron-bcd9f4f4-djpt8" Dec 03 12:46:39 crc kubenswrapper[5052]: I1203 12:46:39.926848 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/049a231b-3a12-4710-9b7a-117b1bb82e72-ovndb-tls-certs\") pod \"neutron-bcd9f4f4-djpt8\" (UID: \"049a231b-3a12-4710-9b7a-117b1bb82e72\") " pod="openstack-kuttl-tests/neutron-bcd9f4f4-djpt8" Dec 03 12:46:39 crc kubenswrapper[5052]: I1203 12:46:39.960288 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:46:39 crc kubenswrapper[5052]: I1203 12:46:39.960370 5052 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Dec 03 12:46:39 crc kubenswrapper[5052]: I1203 12:46:39.962330 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:46:40 crc kubenswrapper[5052]: I1203 12:46:40.124466 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage09-crc" (OuterVolumeSpecName: "ovndbcluster-sb-etc-ovn") pod "c6c9b62e-bbbc-4557-92fd-da64bd8ee0cf" (UID: "c6c9b62e-bbbc-4557-92fd-da64bd8ee0cf"). InnerVolumeSpecName "local-storage09-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 12:46:40 crc kubenswrapper[5052]: I1203 12:46:40.124497 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ed7f9920-a1e4-437b-8074-a30d0fe4450c-kube-api-access-r8cvt" (OuterVolumeSpecName: "kube-api-access-r8cvt") pod "ed7f9920-a1e4-437b-8074-a30d0fe4450c" (UID: "ed7f9920-a1e4-437b-8074-a30d0fe4450c"). InnerVolumeSpecName "kube-api-access-r8cvt". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:46:40 crc kubenswrapper[5052]: I1203 12:46:40.124542 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ed7f9920-a1e4-437b-8074-a30d0fe4450c-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "ed7f9920-a1e4-437b-8074-a30d0fe4450c" (UID: "ed7f9920-a1e4-437b-8074-a30d0fe4450c"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:46:40 crc kubenswrapper[5052]: I1203 12:46:40.125927 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:46:40 crc kubenswrapper[5052]: I1203 12:46:40.136010 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c6c9b62e-bbbc-4557-92fd-da64bd8ee0cf-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "c6c9b62e-bbbc-4557-92fd-da64bd8ee0cf" (UID: "c6c9b62e-bbbc-4557-92fd-da64bd8ee0cf"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:46:40 crc kubenswrapper[5052]: I1203 12:46:40.136231 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c6c9b62e-bbbc-4557-92fd-da64bd8ee0cf-combined-ca-bundle\") pod \"c6c9b62e-bbbc-4557-92fd-da64bd8ee0cf\" (UID: \"c6c9b62e-bbbc-4557-92fd-da64bd8ee0cf\") " Dec 03 12:46:40 crc kubenswrapper[5052]: W1203 12:46:40.136332 5052 empty_dir.go:500] Warning: Unmount skipped because path does not exist: /var/lib/kubelet/pods/c6c9b62e-bbbc-4557-92fd-da64bd8ee0cf/volumes/kubernetes.io~secret/combined-ca-bundle Dec 03 12:46:40 crc kubenswrapper[5052]: I1203 12:46:40.136355 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c6c9b62e-bbbc-4557-92fd-da64bd8ee0cf-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "c6c9b62e-bbbc-4557-92fd-da64bd8ee0cf" (UID: "c6c9b62e-bbbc-4557-92fd-da64bd8ee0cf"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:46:40 crc kubenswrapper[5052]: I1203 12:46:40.137018 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c6c9b62e-bbbc-4557-92fd-da64bd8ee0cf-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:40 crc kubenswrapper[5052]: I1203 12:46:40.137050 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage09-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage09-crc\") on node \"crc\" " Dec 03 12:46:40 crc kubenswrapper[5052]: I1203 12:46:40.137062 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-r8cvt\" (UniqueName: \"kubernetes.io/projected/ed7f9920-a1e4-437b-8074-a30d0fe4450c-kube-api-access-r8cvt\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:40 crc kubenswrapper[5052]: I1203 12:46:40.137073 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ed7f9920-a1e4-437b-8074-a30d0fe4450c-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:40 crc kubenswrapper[5052]: I1203 12:46:40.137142 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ed7f9920-a1e4-437b-8074-a30d0fe4450c-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "ed7f9920-a1e4-437b-8074-a30d0fe4450c" (UID: "ed7f9920-a1e4-437b-8074-a30d0fe4450c"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:46:40 crc kubenswrapper[5052]: I1203 12:46:40.137173 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9c996590-4337-477e-a236-3e834d192346-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "9c996590-4337-477e-a236-3e834d192346" (UID: "9c996590-4337-477e-a236-3e834d192346"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:46:40 crc kubenswrapper[5052]: I1203 12:46:40.138323 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ed7f9920-a1e4-437b-8074-a30d0fe4450c-config-data" (OuterVolumeSpecName: "config-data") pod "ed7f9920-a1e4-437b-8074-a30d0fe4450c" (UID: "ed7f9920-a1e4-437b-8074-a30d0fe4450c"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:46:40 crc kubenswrapper[5052]: I1203 12:46:40.138888 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage01-crc" (OuterVolumeSpecName: "ovndbcluster-nb-etc-ovn") pod "29c7d055-4f37-45a4-a608-eb73a361a86c" (UID: "29c7d055-4f37-45a4-a608-eb73a361a86c"). InnerVolumeSpecName "local-storage01-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 12:46:40 crc kubenswrapper[5052]: I1203 12:46:40.139251 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9c996590-4337-477e-a236-3e834d192346-kube-api-access-qbgbz" (OuterVolumeSpecName: "kube-api-access-qbgbz") pod "9c996590-4337-477e-a236-3e834d192346" (UID: "9c996590-4337-477e-a236-3e834d192346"). InnerVolumeSpecName "kube-api-access-qbgbz". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:46:40 crc kubenswrapper[5052]: I1203 12:46:40.140947 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/29c7d055-4f37-45a4-a608-eb73a361a86c-kube-api-access-9zmq4" (OuterVolumeSpecName: "kube-api-access-9zmq4") pod "29c7d055-4f37-45a4-a608-eb73a361a86c" (UID: "29c7d055-4f37-45a4-a608-eb73a361a86c"). InnerVolumeSpecName "kube-api-access-9zmq4". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:46:40 crc kubenswrapper[5052]: I1203 12:46:40.141756 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/29c7d055-4f37-45a4-a608-eb73a361a86c-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "29c7d055-4f37-45a4-a608-eb73a361a86c" (UID: "29c7d055-4f37-45a4-a608-eb73a361a86c"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:46:40 crc kubenswrapper[5052]: I1203 12:46:40.141855 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9c996590-4337-477e-a236-3e834d192346-config-data" (OuterVolumeSpecName: "config-data") pod "9c996590-4337-477e-a236-3e834d192346" (UID: "9c996590-4337-477e-a236-3e834d192346"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:46:40 crc kubenswrapper[5052]: I1203 12:46:40.142274 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c6c9b62e-bbbc-4557-92fd-da64bd8ee0cf-kube-api-access-9bkcr" (OuterVolumeSpecName: "kube-api-access-9bkcr") pod "c6c9b62e-bbbc-4557-92fd-da64bd8ee0cf" (UID: "c6c9b62e-bbbc-4557-92fd-da64bd8ee0cf"). InnerVolumeSpecName "kube-api-access-9bkcr". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:46:40 crc kubenswrapper[5052]: I1203 12:46:40.143572 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/049a231b-3a12-4710-9b7a-117b1bb82e72-internal-tls-certs\") pod \"neutron-bcd9f4f4-djpt8\" (UID: \"049a231b-3a12-4710-9b7a-117b1bb82e72\") " pod="openstack-kuttl-tests/neutron-bcd9f4f4-djpt8" Dec 03 12:46:40 crc kubenswrapper[5052]: I1203 12:46:40.143770 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/049a231b-3a12-4710-9b7a-117b1bb82e72-combined-ca-bundle\") pod \"neutron-bcd9f4f4-djpt8\" (UID: \"049a231b-3a12-4710-9b7a-117b1bb82e72\") " pod="openstack-kuttl-tests/neutron-bcd9f4f4-djpt8" Dec 03 12:46:40 crc kubenswrapper[5052]: I1203 12:46:40.144170 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/049a231b-3a12-4710-9b7a-117b1bb82e72-httpd-config\") pod \"neutron-bcd9f4f4-djpt8\" (UID: \"049a231b-3a12-4710-9b7a-117b1bb82e72\") " pod="openstack-kuttl-tests/neutron-bcd9f4f4-djpt8" Dec 03 12:46:40 crc kubenswrapper[5052]: I1203 12:46:40.144411 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/049a231b-3a12-4710-9b7a-117b1bb82e72-ovndb-tls-certs\") pod \"neutron-bcd9f4f4-djpt8\" (UID: \"049a231b-3a12-4710-9b7a-117b1bb82e72\") " pod="openstack-kuttl-tests/neutron-bcd9f4f4-djpt8" Dec 03 12:46:40 crc kubenswrapper[5052]: I1203 12:46:40.152432 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/049a231b-3a12-4710-9b7a-117b1bb82e72-config\") pod \"neutron-bcd9f4f4-djpt8\" (UID: \"049a231b-3a12-4710-9b7a-117b1bb82e72\") " pod="openstack-kuttl-tests/neutron-bcd9f4f4-djpt8" Dec 03 12:46:40 crc kubenswrapper[5052]: I1203 12:46:40.152461 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/049a231b-3a12-4710-9b7a-117b1bb82e72-public-tls-certs\") pod \"neutron-bcd9f4f4-djpt8\" (UID: \"049a231b-3a12-4710-9b7a-117b1bb82e72\") " pod="openstack-kuttl-tests/neutron-bcd9f4f4-djpt8" Dec 03 12:46:40 crc kubenswrapper[5052]: I1203 12:46:40.152616 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-zrb86\" (UniqueName: \"kubernetes.io/projected/049a231b-3a12-4710-9b7a-117b1bb82e72-kube-api-access-zrb86\") pod \"neutron-bcd9f4f4-djpt8\" (UID: \"049a231b-3a12-4710-9b7a-117b1bb82e72\") " pod="openstack-kuttl-tests/neutron-bcd9f4f4-djpt8" Dec 03 12:46:40 crc kubenswrapper[5052]: I1203 12:46:40.171133 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage09-crc" (UniqueName: "kubernetes.io/local-volume/local-storage09-crc") on node "crc" Dec 03 12:46:40 crc kubenswrapper[5052]: I1203 12:46:40.239559 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage09-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage09-crc\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:40 crc kubenswrapper[5052]: I1203 12:46:40.239602 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-qbgbz\" (UniqueName: \"kubernetes.io/projected/9c996590-4337-477e-a236-3e834d192346-kube-api-access-qbgbz\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:40 crc kubenswrapper[5052]: I1203 12:46:40.239618 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9c996590-4337-477e-a236-3e834d192346-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:40 crc kubenswrapper[5052]: I1203 12:46:40.239632 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/9c996590-4337-477e-a236-3e834d192346-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:40 crc kubenswrapper[5052]: I1203 12:46:40.239645 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/ed7f9920-a1e4-437b-8074-a30d0fe4450c-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:40 crc kubenswrapper[5052]: I1203 12:46:40.239660 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/29c7d055-4f37-45a4-a608-eb73a361a86c-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:40 crc kubenswrapper[5052]: I1203 12:46:40.239674 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-9zmq4\" (UniqueName: \"kubernetes.io/projected/29c7d055-4f37-45a4-a608-eb73a361a86c-kube-api-access-9zmq4\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:40 crc kubenswrapper[5052]: I1203 12:46:40.239707 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage01-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage01-crc\") on node \"crc\" " Dec 03 12:46:40 crc kubenswrapper[5052]: I1203 12:46:40.239724 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/ed7f9920-a1e4-437b-8074-a30d0fe4450c-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:40 crc kubenswrapper[5052]: I1203 12:46:40.239738 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-9bkcr\" (UniqueName: \"kubernetes.io/projected/c6c9b62e-bbbc-4557-92fd-da64bd8ee0cf-kube-api-access-9bkcr\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:40 crc kubenswrapper[5052]: I1203 12:46:40.269092 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage01-crc" (UniqueName: "kubernetes.io/local-volume/local-storage01-crc") on node "crc" Dec 03 12:46:40 crc kubenswrapper[5052]: I1203 12:46:40.277759 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-bcd9f4f4-djpt8" Dec 03 12:46:40 crc kubenswrapper[5052]: I1203 12:46:40.316470 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:46:40 crc kubenswrapper[5052]: I1203 12:46:40.343549 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage01-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage01-crc\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:40 crc kubenswrapper[5052]: I1203 12:46:40.389683 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-metadata-0" podUID="75266130-5854-4acd-acc9-bc22accc19bc" containerName="nova-metadata-metadata" probeResult="failure" output="Get \"https://10.217.0.12:8775/\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Dec 03 12:46:40 crc kubenswrapper[5052]: I1203 12:46:40.389699 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-metadata-0" podUID="75266130-5854-4acd-acc9-bc22accc19bc" containerName="nova-metadata-log" probeResult="failure" output="Get \"https://10.217.0.12:8775/\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Dec 03 12:46:40 crc kubenswrapper[5052]: I1203 12:46:40.570632 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:46:40 crc kubenswrapper[5052]: I1203 12:46:40.570655 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:46:40 crc kubenswrapper[5052]: I1203 12:46:40.570708 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-6d64d7894b-xp9d5" Dec 03 12:46:40 crc kubenswrapper[5052]: I1203 12:46:40.570845 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:46:40 crc kubenswrapper[5052]: I1203 12:46:40.570852 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:46:40 crc kubenswrapper[5052]: I1203 12:46:40.570873 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:46:40 crc kubenswrapper[5052]: I1203 12:46:40.596858 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-6d64d7894b-xp9d5" Dec 03 12:46:40 crc kubenswrapper[5052]: I1203 12:46:40.616698 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:46:40 crc kubenswrapper[5052]: I1203 12:46:40.660037 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-nb-0"] Dec 03 12:46:40 crc kubenswrapper[5052]: I1203 12:46:40.668199 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-nb-0"] Dec 03 12:46:40 crc kubenswrapper[5052]: I1203 12:46:40.674394 5052 scope.go:117] "RemoveContainer" containerID="2aed1c760a326f3d008799e902b1093313df3764a57c879f0b3686410127d37d" Dec 03 12:46:40 crc kubenswrapper[5052]: E1203 12:46:40.674827 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 12:46:40 crc kubenswrapper[5052]: I1203 12:46:40.704031 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ovsdbserver-nb-0"] Dec 03 12:46:40 crc kubenswrapper[5052]: I1203 12:46:40.706124 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:46:40 crc kubenswrapper[5052]: I1203 12:46:40.716366 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"ovndbcluster-nb-config" Dec 03 12:46:40 crc kubenswrapper[5052]: I1203 12:46:40.716733 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ovncluster-ovndbcluster-nb-dockercfg-2jlnn" Dec 03 12:46:40 crc kubenswrapper[5052]: I1203 12:46:40.716940 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"ovndbcluster-nb-scripts" Dec 03 12:46:40 crc kubenswrapper[5052]: I1203 12:46:40.717309 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-ovndbcluster-nb-ovndbs" Dec 03 12:46:40 crc kubenswrapper[5052]: I1203 12:46:40.722246 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 12:46:40 crc kubenswrapper[5052]: I1203 12:46:40.737838 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 12:46:40 crc kubenswrapper[5052]: I1203 12:46:40.795237 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/f8ab3054-5998-40c1-852f-be0dbaf022e5-scripts\") pod \"f8ab3054-5998-40c1-852f-be0dbaf022e5\" (UID: \"f8ab3054-5998-40c1-852f-be0dbaf022e5\") " Dec 03 12:46:40 crc kubenswrapper[5052]: I1203 12:46:40.795544 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/901a2f0b-fb8e-4c97-aea7-567d7ee42a23-internal-tls-certs\") pod \"901a2f0b-fb8e-4c97-aea7-567d7ee42a23\" (UID: \"901a2f0b-fb8e-4c97-aea7-567d7ee42a23\") " Dec 03 12:46:40 crc kubenswrapper[5052]: I1203 12:46:40.795692 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p6qch\" (UniqueName: \"kubernetes.io/projected/f8ab3054-5998-40c1-852f-be0dbaf022e5-kube-api-access-p6qch\") pod \"f8ab3054-5998-40c1-852f-be0dbaf022e5\" (UID: \"f8ab3054-5998-40c1-852f-be0dbaf022e5\") " Dec 03 12:46:40 crc kubenswrapper[5052]: I1203 12:46:40.795832 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f8ab3054-5998-40c1-852f-be0dbaf022e5-config\") pod \"f8ab3054-5998-40c1-852f-be0dbaf022e5\" (UID: \"f8ab3054-5998-40c1-852f-be0dbaf022e5\") " Dec 03 12:46:40 crc kubenswrapper[5052]: I1203 12:46:40.795979 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/901a2f0b-fb8e-4c97-aea7-567d7ee42a23-combined-ca-bundle\") pod \"901a2f0b-fb8e-4c97-aea7-567d7ee42a23\" (UID: \"901a2f0b-fb8e-4c97-aea7-567d7ee42a23\") " Dec 03 12:46:40 crc kubenswrapper[5052]: I1203 12:46:40.796121 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/f8ab3054-5998-40c1-852f-be0dbaf022e5-metrics-certs-tls-certs\") pod \"f8ab3054-5998-40c1-852f-be0dbaf022e5\" (UID: \"f8ab3054-5998-40c1-852f-be0dbaf022e5\") " Dec 03 12:46:40 crc kubenswrapper[5052]: I1203 12:46:40.796252 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f8ab3054-5998-40c1-852f-be0dbaf022e5-combined-ca-bundle\") pod \"f8ab3054-5998-40c1-852f-be0dbaf022e5\" (UID: \"f8ab3054-5998-40c1-852f-be0dbaf022e5\") " Dec 03 12:46:40 crc kubenswrapper[5052]: I1203 12:46:40.796380 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mwcgt\" (UniqueName: \"kubernetes.io/projected/901a2f0b-fb8e-4c97-aea7-567d7ee42a23-kube-api-access-mwcgt\") pod \"901a2f0b-fb8e-4c97-aea7-567d7ee42a23\" (UID: \"901a2f0b-fb8e-4c97-aea7-567d7ee42a23\") " Dec 03 12:46:40 crc kubenswrapper[5052]: I1203 12:46:40.796583 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/901a2f0b-fb8e-4c97-aea7-567d7ee42a23-httpd-config\") pod \"901a2f0b-fb8e-4c97-aea7-567d7ee42a23\" (UID: \"901a2f0b-fb8e-4c97-aea7-567d7ee42a23\") " Dec 03 12:46:40 crc kubenswrapper[5052]: I1203 12:46:40.796717 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovn-rundir\" (UniqueName: \"kubernetes.io/empty-dir/f8ab3054-5998-40c1-852f-be0dbaf022e5-ovn-rundir\") pod \"f8ab3054-5998-40c1-852f-be0dbaf022e5\" (UID: \"f8ab3054-5998-40c1-852f-be0dbaf022e5\") " Dec 03 12:46:40 crc kubenswrapper[5052]: I1203 12:46:40.796812 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/901a2f0b-fb8e-4c97-aea7-567d7ee42a23-config\") pod \"901a2f0b-fb8e-4c97-aea7-567d7ee42a23\" (UID: \"901a2f0b-fb8e-4c97-aea7-567d7ee42a23\") " Dec 03 12:46:40 crc kubenswrapper[5052]: I1203 12:46:40.800772 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-nb-0"] Dec 03 12:46:40 crc kubenswrapper[5052]: I1203 12:46:40.801706 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f8ab3054-5998-40c1-852f-be0dbaf022e5-ovn-rundir" (OuterVolumeSpecName: "ovn-rundir") pod "f8ab3054-5998-40c1-852f-be0dbaf022e5" (UID: "f8ab3054-5998-40c1-852f-be0dbaf022e5"). InnerVolumeSpecName "ovn-rundir". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:46:40 crc kubenswrapper[5052]: I1203 12:46:40.809492 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f8ab3054-5998-40c1-852f-be0dbaf022e5-config" (OuterVolumeSpecName: "config") pod "f8ab3054-5998-40c1-852f-be0dbaf022e5" (UID: "f8ab3054-5998-40c1-852f-be0dbaf022e5"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:46:40 crc kubenswrapper[5052]: I1203 12:46:40.809992 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f8ab3054-5998-40c1-852f-be0dbaf022e5-scripts" (OuterVolumeSpecName: "scripts") pod "f8ab3054-5998-40c1-852f-be0dbaf022e5" (UID: "f8ab3054-5998-40c1-852f-be0dbaf022e5"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:46:40 crc kubenswrapper[5052]: I1203 12:46:40.816295 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/901a2f0b-fb8e-4c97-aea7-567d7ee42a23-kube-api-access-mwcgt" (OuterVolumeSpecName: "kube-api-access-mwcgt") pod "901a2f0b-fb8e-4c97-aea7-567d7ee42a23" (UID: "901a2f0b-fb8e-4c97-aea7-567d7ee42a23"). InnerVolumeSpecName "kube-api-access-mwcgt". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:46:40 crc kubenswrapper[5052]: I1203 12:46:40.816462 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f8ab3054-5998-40c1-852f-be0dbaf022e5-kube-api-access-p6qch" (OuterVolumeSpecName: "kube-api-access-p6qch") pod "f8ab3054-5998-40c1-852f-be0dbaf022e5" (UID: "f8ab3054-5998-40c1-852f-be0dbaf022e5"). InnerVolumeSpecName "kube-api-access-p6qch". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:46:40 crc kubenswrapper[5052]: I1203 12:46:40.817377 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f8ab3054-5998-40c1-852f-be0dbaf022e5-metrics-certs-tls-certs" (OuterVolumeSpecName: "metrics-certs-tls-certs") pod "f8ab3054-5998-40c1-852f-be0dbaf022e5" (UID: "f8ab3054-5998-40c1-852f-be0dbaf022e5"). InnerVolumeSpecName "metrics-certs-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:46:40 crc kubenswrapper[5052]: I1203 12:46:40.824759 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/901a2f0b-fb8e-4c97-aea7-567d7ee42a23-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "901a2f0b-fb8e-4c97-aea7-567d7ee42a23" (UID: "901a2f0b-fb8e-4c97-aea7-567d7ee42a23"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:46:40 crc kubenswrapper[5052]: I1203 12:46:40.828895 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/901a2f0b-fb8e-4c97-aea7-567d7ee42a23-httpd-config" (OuterVolumeSpecName: "httpd-config") pod "901a2f0b-fb8e-4c97-aea7-567d7ee42a23" (UID: "901a2f0b-fb8e-4c97-aea7-567d7ee42a23"). InnerVolumeSpecName "httpd-config". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:46:40 crc kubenswrapper[5052]: I1203 12:46:40.829540 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/901a2f0b-fb8e-4c97-aea7-567d7ee42a23-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "901a2f0b-fb8e-4c97-aea7-567d7ee42a23" (UID: "901a2f0b-fb8e-4c97-aea7-567d7ee42a23"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:46:40 crc kubenswrapper[5052]: I1203 12:46:40.833799 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 12:46:40 crc kubenswrapper[5052]: I1203 12:46:40.835887 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:46:40 crc kubenswrapper[5052]: I1203 12:46:40.839926 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-nova-novncproxy-cell1-public-svc" Dec 03 12:46:40 crc kubenswrapper[5052]: I1203 12:46:40.840353 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell1-novncproxy-config-data" Dec 03 12:46:40 crc kubenswrapper[5052]: I1203 12:46:40.842707 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-sb-0"] Dec 03 12:46:40 crc kubenswrapper[5052]: I1203 12:46:40.851801 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/901a2f0b-fb8e-4c97-aea7-567d7ee42a23-config" (OuterVolumeSpecName: "config") pod "901a2f0b-fb8e-4c97-aea7-567d7ee42a23" (UID: "901a2f0b-fb8e-4c97-aea7-567d7ee42a23"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:46:40 crc kubenswrapper[5052]: I1203 12:46:40.852473 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-nova-novncproxy-cell1-vencrypt" Dec 03 12:46:40 crc kubenswrapper[5052]: I1203 12:46:40.854642 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f8ab3054-5998-40c1-852f-be0dbaf022e5-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "f8ab3054-5998-40c1-852f-be0dbaf022e5" (UID: "f8ab3054-5998-40c1-852f-be0dbaf022e5"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:46:40 crc kubenswrapper[5052]: I1203 12:46:40.854693 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-sb-0"] Dec 03 12:46:40 crc kubenswrapper[5052]: I1203 12:46:40.873797 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 12:46:40 crc kubenswrapper[5052]: I1203 12:46:40.882458 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ovsdbserver-sb-0"] Dec 03 12:46:40 crc kubenswrapper[5052]: I1203 12:46:40.894095 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:46:40 crc kubenswrapper[5052]: I1203 12:46:40.896720 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"ovndbcluster-sb-scripts" Dec 03 12:46:40 crc kubenswrapper[5052]: I1203 12:46:40.897922 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-ovndbcluster-sb-ovndbs" Dec 03 12:46:40 crc kubenswrapper[5052]: I1203 12:46:40.898131 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"ovndbcluster-sb-config" Dec 03 12:46:40 crc kubenswrapper[5052]: I1203 12:46:40.898326 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ovncluster-ovndbcluster-sb-dockercfg-f2jbx" Dec 03 12:46:40 crc kubenswrapper[5052]: I1203 12:46:40.899831 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage01-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage01-crc\") pod \"ovsdbserver-nb-0\" (UID: \"2005f89c-8a19-403c-acc9-c13ede717372\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:46:40 crc kubenswrapper[5052]: I1203 12:46:40.899943 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/2005f89c-8a19-403c-acc9-c13ede717372-config\") pod \"ovsdbserver-nb-0\" (UID: \"2005f89c-8a19-403c-acc9-c13ede717372\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:46:40 crc kubenswrapper[5052]: I1203 12:46:40.900076 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2005f89c-8a19-403c-acc9-c13ede717372-combined-ca-bundle\") pod \"ovsdbserver-nb-0\" (UID: \"2005f89c-8a19-403c-acc9-c13ede717372\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:46:40 crc kubenswrapper[5052]: I1203 12:46:40.900209 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/2005f89c-8a19-403c-acc9-c13ede717372-metrics-certs-tls-certs\") pod \"ovsdbserver-nb-0\" (UID: \"2005f89c-8a19-403c-acc9-c13ede717372\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:46:40 crc kubenswrapper[5052]: I1203 12:46:40.900330 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/2005f89c-8a19-403c-acc9-c13ede717372-ovsdb-rundir\") pod \"ovsdbserver-nb-0\" (UID: \"2005f89c-8a19-403c-acc9-c13ede717372\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:46:40 crc kubenswrapper[5052]: I1203 12:46:40.900502 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovsdbserver-nb-tls-certs\" (UniqueName: \"kubernetes.io/secret/2005f89c-8a19-403c-acc9-c13ede717372-ovsdbserver-nb-tls-certs\") pod \"ovsdbserver-nb-0\" (UID: \"2005f89c-8a19-403c-acc9-c13ede717372\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:46:40 crc kubenswrapper[5052]: I1203 12:46:40.900663 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wrxdr\" (UniqueName: \"kubernetes.io/projected/2005f89c-8a19-403c-acc9-c13ede717372-kube-api-access-wrxdr\") pod \"ovsdbserver-nb-0\" (UID: \"2005f89c-8a19-403c-acc9-c13ede717372\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:46:40 crc kubenswrapper[5052]: I1203 12:46:40.900801 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/2005f89c-8a19-403c-acc9-c13ede717372-scripts\") pod \"ovsdbserver-nb-0\" (UID: \"2005f89c-8a19-403c-acc9-c13ede717372\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:46:40 crc kubenswrapper[5052]: I1203 12:46:40.900995 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/f8ab3054-5998-40c1-852f-be0dbaf022e5-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:40 crc kubenswrapper[5052]: I1203 12:46:40.901055 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/901a2f0b-fb8e-4c97-aea7-567d7ee42a23-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:40 crc kubenswrapper[5052]: I1203 12:46:40.903160 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-p6qch\" (UniqueName: \"kubernetes.io/projected/f8ab3054-5998-40c1-852f-be0dbaf022e5-kube-api-access-p6qch\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:40 crc kubenswrapper[5052]: I1203 12:46:40.903264 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f8ab3054-5998-40c1-852f-be0dbaf022e5-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:40 crc kubenswrapper[5052]: I1203 12:46:40.903338 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/901a2f0b-fb8e-4c97-aea7-567d7ee42a23-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:40 crc kubenswrapper[5052]: I1203 12:46:40.903397 5052 reconciler_common.go:293] "Volume detached for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/f8ab3054-5998-40c1-852f-be0dbaf022e5-metrics-certs-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:40 crc kubenswrapper[5052]: I1203 12:46:40.903448 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f8ab3054-5998-40c1-852f-be0dbaf022e5-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:40 crc kubenswrapper[5052]: I1203 12:46:40.903556 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-mwcgt\" (UniqueName: \"kubernetes.io/projected/901a2f0b-fb8e-4c97-aea7-567d7ee42a23-kube-api-access-mwcgt\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:40 crc kubenswrapper[5052]: I1203 12:46:40.903687 5052 reconciler_common.go:293] "Volume detached for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/29c7d055-4f37-45a4-a608-eb73a361a86c-metrics-certs-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:40 crc kubenswrapper[5052]: I1203 12:46:40.903740 5052 reconciler_common.go:293] "Volume detached for volume \"ovsdbserver-nb-tls-certs\" (UniqueName: \"kubernetes.io/secret/29c7d055-4f37-45a4-a608-eb73a361a86c-ovsdbserver-nb-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:40 crc kubenswrapper[5052]: I1203 12:46:40.903792 5052 reconciler_common.go:293] "Volume detached for volume \"vencrypt-tls-certs\" (UniqueName: \"kubernetes.io/secret/9c996590-4337-477e-a236-3e834d192346-vencrypt-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:40 crc kubenswrapper[5052]: I1203 12:46:40.903904 5052 reconciler_common.go:293] "Volume detached for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/901a2f0b-fb8e-4c97-aea7-567d7ee42a23-httpd-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:40 crc kubenswrapper[5052]: I1203 12:46:40.903960 5052 reconciler_common.go:293] "Volume detached for volume \"nova-novncproxy-tls-certs\" (UniqueName: \"kubernetes.io/secret/9c996590-4337-477e-a236-3e834d192346-nova-novncproxy-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:40 crc kubenswrapper[5052]: I1203 12:46:40.904009 5052 reconciler_common.go:293] "Volume detached for volume \"ovn-rundir\" (UniqueName: \"kubernetes.io/empty-dir/f8ab3054-5998-40c1-852f-be0dbaf022e5-ovn-rundir\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:40 crc kubenswrapper[5052]: I1203 12:46:40.904058 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/901a2f0b-fb8e-4c97-aea7-567d7ee42a23-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:40 crc kubenswrapper[5052]: I1203 12:46:40.973014 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-sb-0"] Dec 03 12:46:41 crc kubenswrapper[5052]: I1203 12:46:41.002909 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:46:41 crc kubenswrapper[5052]: I1203 12:46:41.005193 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2005f89c-8a19-403c-acc9-c13ede717372-combined-ca-bundle\") pod \"ovsdbserver-nb-0\" (UID: \"2005f89c-8a19-403c-acc9-c13ede717372\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:46:41 crc kubenswrapper[5052]: I1203 12:46:41.005271 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/84f26afe-5b0c-43bb-a4c8-a9ac21d24330-combined-ca-bundle\") pod \"ovsdbserver-sb-0\" (UID: \"84f26afe-5b0c-43bb-a4c8-a9ac21d24330\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:46:41 crc kubenswrapper[5052]: I1203 12:46:41.005301 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8xfbb\" (UniqueName: \"kubernetes.io/projected/84f26afe-5b0c-43bb-a4c8-a9ac21d24330-kube-api-access-8xfbb\") pod \"ovsdbserver-sb-0\" (UID: \"84f26afe-5b0c-43bb-a4c8-a9ac21d24330\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:46:41 crc kubenswrapper[5052]: I1203 12:46:41.005319 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage09-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage09-crc\") pod \"ovsdbserver-sb-0\" (UID: \"84f26afe-5b0c-43bb-a4c8-a9ac21d24330\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:46:41 crc kubenswrapper[5052]: I1203 12:46:41.005353 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/2005f89c-8a19-403c-acc9-c13ede717372-metrics-certs-tls-certs\") pod \"ovsdbserver-nb-0\" (UID: \"2005f89c-8a19-403c-acc9-c13ede717372\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:46:41 crc kubenswrapper[5052]: I1203 12:46:41.005416 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t5kzv\" (UniqueName: \"kubernetes.io/projected/5cafb759-6926-4fd6-a77e-7e18a9becd03-kube-api-access-t5kzv\") pod \"nova-cell1-novncproxy-0\" (UID: \"5cafb759-6926-4fd6-a77e-7e18a9becd03\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:46:41 crc kubenswrapper[5052]: I1203 12:46:41.005437 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/2005f89c-8a19-403c-acc9-c13ede717372-ovsdb-rundir\") pod \"ovsdbserver-nb-0\" (UID: \"2005f89c-8a19-403c-acc9-c13ede717372\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:46:41 crc kubenswrapper[5052]: I1203 12:46:41.005461 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/5cafb759-6926-4fd6-a77e-7e18a9becd03-config-data\") pod \"nova-cell1-novncproxy-0\" (UID: \"5cafb759-6926-4fd6-a77e-7e18a9becd03\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:46:41 crc kubenswrapper[5052]: I1203 12:46:41.005505 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovsdbserver-sb-tls-certs\" (UniqueName: \"kubernetes.io/secret/84f26afe-5b0c-43bb-a4c8-a9ac21d24330-ovsdbserver-sb-tls-certs\") pod \"ovsdbserver-sb-0\" (UID: \"84f26afe-5b0c-43bb-a4c8-a9ac21d24330\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:46:41 crc kubenswrapper[5052]: I1203 12:46:41.005612 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/84f26afe-5b0c-43bb-a4c8-a9ac21d24330-metrics-certs-tls-certs\") pod \"ovsdbserver-sb-0\" (UID: \"84f26afe-5b0c-43bb-a4c8-a9ac21d24330\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:46:41 crc kubenswrapper[5052]: I1203 12:46:41.005643 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovsdbserver-nb-tls-certs\" (UniqueName: \"kubernetes.io/secret/2005f89c-8a19-403c-acc9-c13ede717372-ovsdbserver-nb-tls-certs\") pod \"ovsdbserver-nb-0\" (UID: \"2005f89c-8a19-403c-acc9-c13ede717372\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:46:41 crc kubenswrapper[5052]: I1203 12:46:41.005675 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/84f26afe-5b0c-43bb-a4c8-a9ac21d24330-scripts\") pod \"ovsdbserver-sb-0\" (UID: \"84f26afe-5b0c-43bb-a4c8-a9ac21d24330\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:46:41 crc kubenswrapper[5052]: I1203 12:46:41.005692 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/84f26afe-5b0c-43bb-a4c8-a9ac21d24330-ovsdb-rundir\") pod \"ovsdbserver-sb-0\" (UID: \"84f26afe-5b0c-43bb-a4c8-a9ac21d24330\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:46:41 crc kubenswrapper[5052]: I1203 12:46:41.005735 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-wrxdr\" (UniqueName: \"kubernetes.io/projected/2005f89c-8a19-403c-acc9-c13ede717372-kube-api-access-wrxdr\") pod \"ovsdbserver-nb-0\" (UID: \"2005f89c-8a19-403c-acc9-c13ede717372\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:46:41 crc kubenswrapper[5052]: I1203 12:46:41.005780 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5cafb759-6926-4fd6-a77e-7e18a9becd03-combined-ca-bundle\") pod \"nova-cell1-novncproxy-0\" (UID: \"5cafb759-6926-4fd6-a77e-7e18a9becd03\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:46:41 crc kubenswrapper[5052]: I1203 12:46:41.005801 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nova-novncproxy-tls-certs\" (UniqueName: \"kubernetes.io/secret/5cafb759-6926-4fd6-a77e-7e18a9becd03-nova-novncproxy-tls-certs\") pod \"nova-cell1-novncproxy-0\" (UID: \"5cafb759-6926-4fd6-a77e-7e18a9becd03\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:46:41 crc kubenswrapper[5052]: I1203 12:46:41.005831 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/2005f89c-8a19-403c-acc9-c13ede717372-scripts\") pod \"ovsdbserver-nb-0\" (UID: \"2005f89c-8a19-403c-acc9-c13ede717372\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:46:41 crc kubenswrapper[5052]: I1203 12:46:41.005874 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage01-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage01-crc\") pod \"ovsdbserver-nb-0\" (UID: \"2005f89c-8a19-403c-acc9-c13ede717372\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:46:41 crc kubenswrapper[5052]: I1203 12:46:41.005904 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/2005f89c-8a19-403c-acc9-c13ede717372-config\") pod \"ovsdbserver-nb-0\" (UID: \"2005f89c-8a19-403c-acc9-c13ede717372\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:46:41 crc kubenswrapper[5052]: I1203 12:46:41.005936 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"vencrypt-tls-certs\" (UniqueName: \"kubernetes.io/secret/5cafb759-6926-4fd6-a77e-7e18a9becd03-vencrypt-tls-certs\") pod \"nova-cell1-novncproxy-0\" (UID: \"5cafb759-6926-4fd6-a77e-7e18a9becd03\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:46:41 crc kubenswrapper[5052]: I1203 12:46:41.005956 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/84f26afe-5b0c-43bb-a4c8-a9ac21d24330-config\") pod \"ovsdbserver-sb-0\" (UID: \"84f26afe-5b0c-43bb-a4c8-a9ac21d24330\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:46:41 crc kubenswrapper[5052]: I1203 12:46:41.006010 5052 reconciler_common.go:293] "Volume detached for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/c6c9b62e-bbbc-4557-92fd-da64bd8ee0cf-metrics-certs-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:41 crc kubenswrapper[5052]: I1203 12:46:41.006022 5052 reconciler_common.go:293] "Volume detached for volume \"ovsdbserver-sb-tls-certs\" (UniqueName: \"kubernetes.io/secret/c6c9b62e-bbbc-4557-92fd-da64bd8ee0cf-ovsdbserver-sb-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:41 crc kubenswrapper[5052]: I1203 12:46:41.006029 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/2005f89c-8a19-403c-acc9-c13ede717372-ovsdb-rundir\") pod \"ovsdbserver-nb-0\" (UID: \"2005f89c-8a19-403c-acc9-c13ede717372\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:46:41 crc kubenswrapper[5052]: I1203 12:46:41.006495 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage01-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage01-crc\") pod \"ovsdbserver-nb-0\" (UID: \"2005f89c-8a19-403c-acc9-c13ede717372\") device mount path \"/mnt/openstack/pv01\"" pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:46:41 crc kubenswrapper[5052]: I1203 12:46:41.008661 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/2005f89c-8a19-403c-acc9-c13ede717372-config\") pod \"ovsdbserver-nb-0\" (UID: \"2005f89c-8a19-403c-acc9-c13ede717372\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:46:41 crc kubenswrapper[5052]: I1203 12:46:41.008807 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/2005f89c-8a19-403c-acc9-c13ede717372-scripts\") pod \"ovsdbserver-nb-0\" (UID: \"2005f89c-8a19-403c-acc9-c13ede717372\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:46:41 crc kubenswrapper[5052]: I1203 12:46:41.017469 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2005f89c-8a19-403c-acc9-c13ede717372-combined-ca-bundle\") pod \"ovsdbserver-nb-0\" (UID: \"2005f89c-8a19-403c-acc9-c13ede717372\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:46:41 crc kubenswrapper[5052]: I1203 12:46:41.018747 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/2005f89c-8a19-403c-acc9-c13ede717372-metrics-certs-tls-certs\") pod \"ovsdbserver-nb-0\" (UID: \"2005f89c-8a19-403c-acc9-c13ede717372\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:46:41 crc kubenswrapper[5052]: I1203 12:46:41.026377 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovsdbserver-nb-tls-certs\" (UniqueName: \"kubernetes.io/secret/2005f89c-8a19-403c-acc9-c13ede717372-ovsdbserver-nb-tls-certs\") pod \"ovsdbserver-nb-0\" (UID: \"2005f89c-8a19-403c-acc9-c13ede717372\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:46:41 crc kubenswrapper[5052]: I1203 12:46:41.026442 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:46:41 crc kubenswrapper[5052]: I1203 12:46:41.029947 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-wrxdr\" (UniqueName: \"kubernetes.io/projected/2005f89c-8a19-403c-acc9-c13ede717372-kube-api-access-wrxdr\") pod \"ovsdbserver-nb-0\" (UID: \"2005f89c-8a19-403c-acc9-c13ede717372\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:46:41 crc kubenswrapper[5052]: I1203 12:46:41.039138 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage01-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage01-crc\") pod \"ovsdbserver-nb-0\" (UID: \"2005f89c-8a19-403c-acc9-c13ede717372\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:46:41 crc kubenswrapper[5052]: I1203 12:46:41.051125 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:46:41 crc kubenswrapper[5052]: I1203 12:46:41.053168 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:46:41 crc kubenswrapper[5052]: I1203 12:46:41.055249 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-nova-internal-svc" Dec 03 12:46:41 crc kubenswrapper[5052]: I1203 12:46:41.055749 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-api-config-data" Dec 03 12:46:41 crc kubenswrapper[5052]: I1203 12:46:41.057053 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-nova-public-svc" Dec 03 12:46:41 crc kubenswrapper[5052]: I1203 12:46:41.061536 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-bcd9f4f4-djpt8"] Dec 03 12:46:41 crc kubenswrapper[5052]: I1203 12:46:41.109759 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/84f26afe-5b0c-43bb-a4c8-a9ac21d24330-config\") pod \"ovsdbserver-sb-0\" (UID: \"84f26afe-5b0c-43bb-a4c8-a9ac21d24330\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:46:41 crc kubenswrapper[5052]: I1203 12:46:41.109879 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xnpkp\" (UniqueName: \"kubernetes.io/projected/d5719b07-205b-4e6a-9fe2-b6531b779b0a-kube-api-access-xnpkp\") pod \"nova-api-0\" (UID: \"d5719b07-205b-4e6a-9fe2-b6531b779b0a\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:46:41 crc kubenswrapper[5052]: I1203 12:46:41.109914 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/84f26afe-5b0c-43bb-a4c8-a9ac21d24330-combined-ca-bundle\") pod \"ovsdbserver-sb-0\" (UID: \"84f26afe-5b0c-43bb-a4c8-a9ac21d24330\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:46:41 crc kubenswrapper[5052]: I1203 12:46:41.109957 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-8xfbb\" (UniqueName: \"kubernetes.io/projected/84f26afe-5b0c-43bb-a4c8-a9ac21d24330-kube-api-access-8xfbb\") pod \"ovsdbserver-sb-0\" (UID: \"84f26afe-5b0c-43bb-a4c8-a9ac21d24330\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:46:41 crc kubenswrapper[5052]: I1203 12:46:41.109984 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage09-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage09-crc\") pod \"ovsdbserver-sb-0\" (UID: \"84f26afe-5b0c-43bb-a4c8-a9ac21d24330\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:46:41 crc kubenswrapper[5052]: I1203 12:46:41.110011 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d5719b07-205b-4e6a-9fe2-b6531b779b0a-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"d5719b07-205b-4e6a-9fe2-b6531b779b0a\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:46:41 crc kubenswrapper[5052]: I1203 12:46:41.110106 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/d5719b07-205b-4e6a-9fe2-b6531b779b0a-public-tls-certs\") pod \"nova-api-0\" (UID: \"d5719b07-205b-4e6a-9fe2-b6531b779b0a\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:46:41 crc kubenswrapper[5052]: I1203 12:46:41.110146 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-t5kzv\" (UniqueName: \"kubernetes.io/projected/5cafb759-6926-4fd6-a77e-7e18a9becd03-kube-api-access-t5kzv\") pod \"nova-cell1-novncproxy-0\" (UID: \"5cafb759-6926-4fd6-a77e-7e18a9becd03\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:46:41 crc kubenswrapper[5052]: I1203 12:46:41.110184 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/5cafb759-6926-4fd6-a77e-7e18a9becd03-config-data\") pod \"nova-cell1-novncproxy-0\" (UID: \"5cafb759-6926-4fd6-a77e-7e18a9becd03\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:46:41 crc kubenswrapper[5052]: I1203 12:46:41.110222 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/d5719b07-205b-4e6a-9fe2-b6531b779b0a-internal-tls-certs\") pod \"nova-api-0\" (UID: \"d5719b07-205b-4e6a-9fe2-b6531b779b0a\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:46:41 crc kubenswrapper[5052]: I1203 12:46:41.110257 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage09-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage09-crc\") pod \"ovsdbserver-sb-0\" (UID: \"84f26afe-5b0c-43bb-a4c8-a9ac21d24330\") device mount path \"/mnt/openstack/pv09\"" pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:46:41 crc kubenswrapper[5052]: I1203 12:46:41.110272 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovsdbserver-sb-tls-certs\" (UniqueName: \"kubernetes.io/secret/84f26afe-5b0c-43bb-a4c8-a9ac21d24330-ovsdbserver-sb-tls-certs\") pod \"ovsdbserver-sb-0\" (UID: \"84f26afe-5b0c-43bb-a4c8-a9ac21d24330\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:46:41 crc kubenswrapper[5052]: I1203 12:46:41.110792 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/84f26afe-5b0c-43bb-a4c8-a9ac21d24330-metrics-certs-tls-certs\") pod \"ovsdbserver-sb-0\" (UID: \"84f26afe-5b0c-43bb-a4c8-a9ac21d24330\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:46:41 crc kubenswrapper[5052]: I1203 12:46:41.112073 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/84f26afe-5b0c-43bb-a4c8-a9ac21d24330-scripts\") pod \"ovsdbserver-sb-0\" (UID: \"84f26afe-5b0c-43bb-a4c8-a9ac21d24330\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:46:41 crc kubenswrapper[5052]: I1203 12:46:41.112107 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/84f26afe-5b0c-43bb-a4c8-a9ac21d24330-ovsdb-rundir\") pod \"ovsdbserver-sb-0\" (UID: \"84f26afe-5b0c-43bb-a4c8-a9ac21d24330\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:46:41 crc kubenswrapper[5052]: I1203 12:46:41.112266 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/d5719b07-205b-4e6a-9fe2-b6531b779b0a-logs\") pod \"nova-api-0\" (UID: \"d5719b07-205b-4e6a-9fe2-b6531b779b0a\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:46:41 crc kubenswrapper[5052]: I1203 12:46:41.112390 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5cafb759-6926-4fd6-a77e-7e18a9becd03-combined-ca-bundle\") pod \"nova-cell1-novncproxy-0\" (UID: \"5cafb759-6926-4fd6-a77e-7e18a9becd03\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:46:41 crc kubenswrapper[5052]: I1203 12:46:41.112434 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"nova-novncproxy-tls-certs\" (UniqueName: \"kubernetes.io/secret/5cafb759-6926-4fd6-a77e-7e18a9becd03-nova-novncproxy-tls-certs\") pod \"nova-cell1-novncproxy-0\" (UID: \"5cafb759-6926-4fd6-a77e-7e18a9becd03\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:46:41 crc kubenswrapper[5052]: I1203 12:46:41.112492 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/d5719b07-205b-4e6a-9fe2-b6531b779b0a-config-data\") pod \"nova-api-0\" (UID: \"d5719b07-205b-4e6a-9fe2-b6531b779b0a\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:46:41 crc kubenswrapper[5052]: I1203 12:46:41.116913 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"vencrypt-tls-certs\" (UniqueName: \"kubernetes.io/secret/5cafb759-6926-4fd6-a77e-7e18a9becd03-vencrypt-tls-certs\") pod \"nova-cell1-novncproxy-0\" (UID: \"5cafb759-6926-4fd6-a77e-7e18a9becd03\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:46:41 crc kubenswrapper[5052]: I1203 12:46:41.117067 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/ed7f9920-a1e4-437b-8074-a30d0fe4450c-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:41 crc kubenswrapper[5052]: I1203 12:46:41.118410 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/84f26afe-5b0c-43bb-a4c8-a9ac21d24330-config\") pod \"ovsdbserver-sb-0\" (UID: \"84f26afe-5b0c-43bb-a4c8-a9ac21d24330\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:46:41 crc kubenswrapper[5052]: I1203 12:46:41.118907 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/5cafb759-6926-4fd6-a77e-7e18a9becd03-config-data\") pod \"nova-cell1-novncproxy-0\" (UID: \"5cafb759-6926-4fd6-a77e-7e18a9becd03\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:46:41 crc kubenswrapper[5052]: I1203 12:46:41.120392 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/84f26afe-5b0c-43bb-a4c8-a9ac21d24330-scripts\") pod \"ovsdbserver-sb-0\" (UID: \"84f26afe-5b0c-43bb-a4c8-a9ac21d24330\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:46:41 crc kubenswrapper[5052]: I1203 12:46:41.121037 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/84f26afe-5b0c-43bb-a4c8-a9ac21d24330-ovsdb-rundir\") pod \"ovsdbserver-sb-0\" (UID: \"84f26afe-5b0c-43bb-a4c8-a9ac21d24330\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:46:41 crc kubenswrapper[5052]: I1203 12:46:41.123041 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovsdbserver-sb-tls-certs\" (UniqueName: \"kubernetes.io/secret/84f26afe-5b0c-43bb-a4c8-a9ac21d24330-ovsdbserver-sb-tls-certs\") pod \"ovsdbserver-sb-0\" (UID: \"84f26afe-5b0c-43bb-a4c8-a9ac21d24330\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:46:41 crc kubenswrapper[5052]: I1203 12:46:41.124718 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:46:41 crc kubenswrapper[5052]: I1203 12:46:41.125951 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/84f26afe-5b0c-43bb-a4c8-a9ac21d24330-combined-ca-bundle\") pod \"ovsdbserver-sb-0\" (UID: \"84f26afe-5b0c-43bb-a4c8-a9ac21d24330\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:46:41 crc kubenswrapper[5052]: I1203 12:46:41.127642 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"nova-novncproxy-tls-certs\" (UniqueName: \"kubernetes.io/secret/5cafb759-6926-4fd6-a77e-7e18a9becd03-nova-novncproxy-tls-certs\") pod \"nova-cell1-novncproxy-0\" (UID: \"5cafb759-6926-4fd6-a77e-7e18a9becd03\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:46:41 crc kubenswrapper[5052]: I1203 12:46:41.129145 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/84f26afe-5b0c-43bb-a4c8-a9ac21d24330-metrics-certs-tls-certs\") pod \"ovsdbserver-sb-0\" (UID: \"84f26afe-5b0c-43bb-a4c8-a9ac21d24330\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:46:41 crc kubenswrapper[5052]: I1203 12:46:41.130668 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5cafb759-6926-4fd6-a77e-7e18a9becd03-combined-ca-bundle\") pod \"nova-cell1-novncproxy-0\" (UID: \"5cafb759-6926-4fd6-a77e-7e18a9becd03\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:46:41 crc kubenswrapper[5052]: I1203 12:46:41.149726 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"vencrypt-tls-certs\" (UniqueName: \"kubernetes.io/secret/5cafb759-6926-4fd6-a77e-7e18a9becd03-vencrypt-tls-certs\") pod \"nova-cell1-novncproxy-0\" (UID: \"5cafb759-6926-4fd6-a77e-7e18a9becd03\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:46:41 crc kubenswrapper[5052]: I1203 12:46:41.150118 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-8xfbb\" (UniqueName: \"kubernetes.io/projected/84f26afe-5b0c-43bb-a4c8-a9ac21d24330-kube-api-access-8xfbb\") pod \"ovsdbserver-sb-0\" (UID: \"84f26afe-5b0c-43bb-a4c8-a9ac21d24330\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:46:41 crc kubenswrapper[5052]: I1203 12:46:41.150439 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-t5kzv\" (UniqueName: \"kubernetes.io/projected/5cafb759-6926-4fd6-a77e-7e18a9becd03-kube-api-access-t5kzv\") pod \"nova-cell1-novncproxy-0\" (UID: \"5cafb759-6926-4fd6-a77e-7e18a9becd03\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:46:41 crc kubenswrapper[5052]: I1203 12:46:41.166761 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage09-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage09-crc\") pod \"ovsdbserver-sb-0\" (UID: \"84f26afe-5b0c-43bb-a4c8-a9ac21d24330\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:46:41 crc kubenswrapper[5052]: I1203 12:46:41.220131 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/d5719b07-205b-4e6a-9fe2-b6531b779b0a-config-data\") pod \"nova-api-0\" (UID: \"d5719b07-205b-4e6a-9fe2-b6531b779b0a\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:46:41 crc kubenswrapper[5052]: I1203 12:46:41.220295 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-xnpkp\" (UniqueName: \"kubernetes.io/projected/d5719b07-205b-4e6a-9fe2-b6531b779b0a-kube-api-access-xnpkp\") pod \"nova-api-0\" (UID: \"d5719b07-205b-4e6a-9fe2-b6531b779b0a\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:46:41 crc kubenswrapper[5052]: I1203 12:46:41.220374 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d5719b07-205b-4e6a-9fe2-b6531b779b0a-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"d5719b07-205b-4e6a-9fe2-b6531b779b0a\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:46:41 crc kubenswrapper[5052]: I1203 12:46:41.220455 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/d5719b07-205b-4e6a-9fe2-b6531b779b0a-public-tls-certs\") pod \"nova-api-0\" (UID: \"d5719b07-205b-4e6a-9fe2-b6531b779b0a\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:46:41 crc kubenswrapper[5052]: I1203 12:46:41.220522 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/d5719b07-205b-4e6a-9fe2-b6531b779b0a-internal-tls-certs\") pod \"nova-api-0\" (UID: \"d5719b07-205b-4e6a-9fe2-b6531b779b0a\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:46:41 crc kubenswrapper[5052]: I1203 12:46:41.220656 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/d5719b07-205b-4e6a-9fe2-b6531b779b0a-logs\") pod \"nova-api-0\" (UID: \"d5719b07-205b-4e6a-9fe2-b6531b779b0a\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:46:41 crc kubenswrapper[5052]: I1203 12:46:41.221101 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/d5719b07-205b-4e6a-9fe2-b6531b779b0a-logs\") pod \"nova-api-0\" (UID: \"d5719b07-205b-4e6a-9fe2-b6531b779b0a\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:46:41 crc kubenswrapper[5052]: I1203 12:46:41.228212 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/d5719b07-205b-4e6a-9fe2-b6531b779b0a-config-data\") pod \"nova-api-0\" (UID: \"d5719b07-205b-4e6a-9fe2-b6531b779b0a\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:46:41 crc kubenswrapper[5052]: I1203 12:46:41.228415 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/d5719b07-205b-4e6a-9fe2-b6531b779b0a-public-tls-certs\") pod \"nova-api-0\" (UID: \"d5719b07-205b-4e6a-9fe2-b6531b779b0a\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:46:41 crc kubenswrapper[5052]: I1203 12:46:41.228753 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/d5719b07-205b-4e6a-9fe2-b6531b779b0a-internal-tls-certs\") pod \"nova-api-0\" (UID: \"d5719b07-205b-4e6a-9fe2-b6531b779b0a\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:46:41 crc kubenswrapper[5052]: I1203 12:46:41.229292 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d5719b07-205b-4e6a-9fe2-b6531b779b0a-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"d5719b07-205b-4e6a-9fe2-b6531b779b0a\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:46:41 crc kubenswrapper[5052]: I1203 12:46:41.246821 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-xnpkp\" (UniqueName: \"kubernetes.io/projected/d5719b07-205b-4e6a-9fe2-b6531b779b0a-kube-api-access-xnpkp\") pod \"nova-api-0\" (UID: \"d5719b07-205b-4e6a-9fe2-b6531b779b0a\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:46:41 crc kubenswrapper[5052]: I1203 12:46:41.282676 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:46:41 crc kubenswrapper[5052]: I1203 12:46:41.336910 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:46:41 crc kubenswrapper[5052]: I1203 12:46:41.471781 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:46:41 crc kubenswrapper[5052]: I1203 12:46:41.471890 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:46:41 crc kubenswrapper[5052]: I1203 12:46:41.590006 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-bcd9f4f4-djpt8" event={"ID":"049a231b-3a12-4710-9b7a-117b1bb82e72","Type":"ContainerStarted","Data":"4b6385cf4545e25cb919fcac9e45a38d9979e76dac85ed14f6bd36d5c92843a5"} Dec 03 12:46:41 crc kubenswrapper[5052]: I1203 12:46:41.593160 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-bcd9f4f4-djpt8" event={"ID":"049a231b-3a12-4710-9b7a-117b1bb82e72","Type":"ContainerStarted","Data":"9cf710603d4318f916c0abb14b25ef6c5d623e0b2a0ba1023eb1c306bb24b6f0"} Dec 03 12:46:41 crc kubenswrapper[5052]: I1203 12:46:41.604985 5052 generic.go:334] "Generic (PLEG): container finished" podID="4d42b068-9b91-4df6-9a9c-81cad009be16" containerID="c222a3a51a9f475798e3bef53ce522fe0044f6c1d5020c5fa20ba4e43d10b518" exitCode=0 Dec 03 12:46:41 crc kubenswrapper[5052]: I1203 12:46:41.605088 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-6d64d7894b-xp9d5" Dec 03 12:46:41 crc kubenswrapper[5052]: I1203 12:46:41.608577 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/memcached-0" event={"ID":"4d42b068-9b91-4df6-9a9c-81cad009be16","Type":"ContainerDied","Data":"c222a3a51a9f475798e3bef53ce522fe0044f6c1d5020c5fa20ba4e43d10b518"} Dec 03 12:46:41 crc kubenswrapper[5052]: I1203 12:46:41.608650 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/memcached-0" event={"ID":"4d42b068-9b91-4df6-9a9c-81cad009be16","Type":"ContainerDied","Data":"7587e638e7960d1cbb26b56db77c21d6d610e0e260a612a862c6e8601775a5ff"} Dec 03 12:46:41 crc kubenswrapper[5052]: I1203 12:46:41.608661 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7587e638e7960d1cbb26b56db77c21d6d610e0e260a612a862c6e8601775a5ff" Dec 03 12:46:41 crc kubenswrapper[5052]: I1203 12:46:41.608752 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:46:41 crc kubenswrapper[5052]: I1203 12:46:41.690928 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="29c7d055-4f37-45a4-a608-eb73a361a86c" path="/var/lib/kubelet/pods/29c7d055-4f37-45a4-a608-eb73a361a86c/volumes" Dec 03 12:46:41 crc kubenswrapper[5052]: I1203 12:46:41.691472 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9c996590-4337-477e-a236-3e834d192346" path="/var/lib/kubelet/pods/9c996590-4337-477e-a236-3e834d192346/volumes" Dec 03 12:46:41 crc kubenswrapper[5052]: I1203 12:46:41.692347 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c6c9b62e-bbbc-4557-92fd-da64bd8ee0cf" path="/var/lib/kubelet/pods/c6c9b62e-bbbc-4557-92fd-da64bd8ee0cf/volumes" Dec 03 12:46:41 crc kubenswrapper[5052]: I1203 12:46:41.692839 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ed7f9920-a1e4-437b-8074-a30d0fe4450c" path="/var/lib/kubelet/pods/ed7f9920-a1e4-437b-8074-a30d0fe4450c/volumes" Dec 03 12:46:41 crc kubenswrapper[5052]: I1203 12:46:41.946770 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 12:46:42 crc kubenswrapper[5052]: I1203 12:46:42.019862 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-nb-0"] Dec 03 12:46:42 crc kubenswrapper[5052]: E1203 12:46:42.281832 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of 2ff6796b6955202076b9ca76f2bd9d074d96d820f421983058345a0af1b54565 is running failed: container process not found" containerID="2ff6796b6955202076b9ca76f2bd9d074d96d820f421983058345a0af1b54565" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:46:42 crc kubenswrapper[5052]: E1203 12:46:42.288649 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of 2ff6796b6955202076b9ca76f2bd9d074d96d820f421983058345a0af1b54565 is running failed: container process not found" containerID="2ff6796b6955202076b9ca76f2bd9d074d96d820f421983058345a0af1b54565" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:46:42 crc kubenswrapper[5052]: E1203 12:46:42.292630 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of 2ff6796b6955202076b9ca76f2bd9d074d96d820f421983058345a0af1b54565 is running failed: container process not found" containerID="2ff6796b6955202076b9ca76f2bd9d074d96d820f421983058345a0af1b54565" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:46:42 crc kubenswrapper[5052]: E1203 12:46:42.292693 5052 prober.go:104] "Probe errored" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of 2ff6796b6955202076b9ca76f2bd9d074d96d820f421983058345a0af1b54565 is running failed: container process not found" probeType="Readiness" pod="openstack-kuttl-tests/nova-cell1-conductor-0" podUID="320f928d-d6b0-48b4-b315-1a6c2ecd3380" containerName="nova-cell1-conductor-conductor" Dec 03 12:46:42 crc kubenswrapper[5052]: I1203 12:46:42.424521 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/memcached-0" Dec 03 12:46:42 crc kubenswrapper[5052]: I1203 12:46:42.502926 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:46:42 crc kubenswrapper[5052]: I1203 12:46:42.522078 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"memcached-tls-certs\" (UniqueName: \"kubernetes.io/secret/4d42b068-9b91-4df6-9a9c-81cad009be16-memcached-tls-certs\") pod \"4d42b068-9b91-4df6-9a9c-81cad009be16\" (UID: \"4d42b068-9b91-4df6-9a9c-81cad009be16\") " Dec 03 12:46:42 crc kubenswrapper[5052]: I1203 12:46:42.522476 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/4d42b068-9b91-4df6-9a9c-81cad009be16-config-data\") pod \"4d42b068-9b91-4df6-9a9c-81cad009be16\" (UID: \"4d42b068-9b91-4df6-9a9c-81cad009be16\") " Dec 03 12:46:42 crc kubenswrapper[5052]: I1203 12:46:42.523183 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4d42b068-9b91-4df6-9a9c-81cad009be16-combined-ca-bundle\") pod \"4d42b068-9b91-4df6-9a9c-81cad009be16\" (UID: \"4d42b068-9b91-4df6-9a9c-81cad009be16\") " Dec 03 12:46:42 crc kubenswrapper[5052]: I1203 12:46:42.523226 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h5m2d\" (UniqueName: \"kubernetes.io/projected/4d42b068-9b91-4df6-9a9c-81cad009be16-kube-api-access-h5m2d\") pod \"4d42b068-9b91-4df6-9a9c-81cad009be16\" (UID: \"4d42b068-9b91-4df6-9a9c-81cad009be16\") " Dec 03 12:46:42 crc kubenswrapper[5052]: I1203 12:46:42.523257 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/4d42b068-9b91-4df6-9a9c-81cad009be16-kolla-config\") pod \"4d42b068-9b91-4df6-9a9c-81cad009be16\" (UID: \"4d42b068-9b91-4df6-9a9c-81cad009be16\") " Dec 03 12:46:42 crc kubenswrapper[5052]: I1203 12:46:42.525892 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4d42b068-9b91-4df6-9a9c-81cad009be16-kolla-config" (OuterVolumeSpecName: "kolla-config") pod "4d42b068-9b91-4df6-9a9c-81cad009be16" (UID: "4d42b068-9b91-4df6-9a9c-81cad009be16"). InnerVolumeSpecName "kolla-config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:46:42 crc kubenswrapper[5052]: I1203 12:46:42.526311 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4d42b068-9b91-4df6-9a9c-81cad009be16-config-data" (OuterVolumeSpecName: "config-data") pod "4d42b068-9b91-4df6-9a9c-81cad009be16" (UID: "4d42b068-9b91-4df6-9a9c-81cad009be16"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:46:42 crc kubenswrapper[5052]: I1203 12:46:42.558123 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4d42b068-9b91-4df6-9a9c-81cad009be16-kube-api-access-h5m2d" (OuterVolumeSpecName: "kube-api-access-h5m2d") pod "4d42b068-9b91-4df6-9a9c-81cad009be16" (UID: "4d42b068-9b91-4df6-9a9c-81cad009be16"). InnerVolumeSpecName "kube-api-access-h5m2d". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:46:42 crc kubenswrapper[5052]: I1203 12:46:42.595883 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/neutron-6d64d7894b-xp9d5"] Dec 03 12:46:42 crc kubenswrapper[5052]: I1203 12:46:42.626341 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/4d42b068-9b91-4df6-9a9c-81cad009be16-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:42 crc kubenswrapper[5052]: I1203 12:46:42.626451 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-h5m2d\" (UniqueName: \"kubernetes.io/projected/4d42b068-9b91-4df6-9a9c-81cad009be16-kube-api-access-h5m2d\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:42 crc kubenswrapper[5052]: I1203 12:46:42.626525 5052 reconciler_common.go:293] "Volume detached for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/4d42b068-9b91-4df6-9a9c-81cad009be16-kolla-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:42 crc kubenswrapper[5052]: I1203 12:46:42.631348 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4d42b068-9b91-4df6-9a9c-81cad009be16-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "4d42b068-9b91-4df6-9a9c-81cad009be16" (UID: "4d42b068-9b91-4df6-9a9c-81cad009be16"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:46:42 crc kubenswrapper[5052]: I1203 12:46:42.637098 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/neutron-6d64d7894b-xp9d5"] Dec 03 12:46:42 crc kubenswrapper[5052]: I1203 12:46:42.648956 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-nb-0" event={"ID":"2005f89c-8a19-403c-acc9-c13ede717372","Type":"ContainerStarted","Data":"97e7375adc02fed58699749dcd3cac71fe870cf4ea77be0d3889c888ce45f78b"} Dec 03 12:46:42 crc kubenswrapper[5052]: I1203 12:46:42.655070 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" event={"ID":"5cafb759-6926-4fd6-a77e-7e18a9becd03","Type":"ContainerStarted","Data":"e14839f4541e00060453ac3f79a70d8dbe437da0b37ce8e08926308c30ecc583"} Dec 03 12:46:42 crc kubenswrapper[5052]: I1203 12:46:42.657837 5052 generic.go:334] "Generic (PLEG): container finished" podID="320f928d-d6b0-48b4-b315-1a6c2ecd3380" containerID="2ff6796b6955202076b9ca76f2bd9d074d96d820f421983058345a0af1b54565" exitCode=0 Dec 03 12:46:42 crc kubenswrapper[5052]: I1203 12:46:42.657931 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-conductor-0" event={"ID":"320f928d-d6b0-48b4-b315-1a6c2ecd3380","Type":"ContainerDied","Data":"2ff6796b6955202076b9ca76f2bd9d074d96d820f421983058345a0af1b54565"} Dec 03 12:46:42 crc kubenswrapper[5052]: I1203 12:46:42.658723 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-sb-0"] Dec 03 12:46:42 crc kubenswrapper[5052]: I1203 12:46:42.670390 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/memcached-0" Dec 03 12:46:42 crc kubenswrapper[5052]: W1203 12:46:42.671809 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod84f26afe_5b0c_43bb_a4c8_a9ac21d24330.slice/crio-2bf9da39b82d602ec1c4f37d5f315087e20eb503a371d87ae4e4bada773f70c7 WatchSource:0}: Error finding container 2bf9da39b82d602ec1c4f37d5f315087e20eb503a371d87ae4e4bada773f70c7: Status 404 returned error can't find the container with id 2bf9da39b82d602ec1c4f37d5f315087e20eb503a371d87ae4e4bada773f70c7 Dec 03 12:46:42 crc kubenswrapper[5052]: I1203 12:46:42.671879 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"d5719b07-205b-4e6a-9fe2-b6531b779b0a","Type":"ContainerStarted","Data":"5fd11a286ce87d05f1bf202e0033bf3bdd9f521e33004bcc3c2218a19adfc2af"} Dec 03 12:46:42 crc kubenswrapper[5052]: I1203 12:46:42.730307 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4d42b068-9b91-4df6-9a9c-81cad009be16-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:42 crc kubenswrapper[5052]: I1203 12:46:42.730339 5052 reconciler_common.go:293] "Volume detached for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/901a2f0b-fb8e-4c97-aea7-567d7ee42a23-ovndb-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:42 crc kubenswrapper[5052]: I1203 12:46:42.730347 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/901a2f0b-fb8e-4c97-aea7-567d7ee42a23-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:42 crc kubenswrapper[5052]: I1203 12:46:42.783671 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4d42b068-9b91-4df6-9a9c-81cad009be16-memcached-tls-certs" (OuterVolumeSpecName: "memcached-tls-certs") pod "4d42b068-9b91-4df6-9a9c-81cad009be16" (UID: "4d42b068-9b91-4df6-9a9c-81cad009be16"). InnerVolumeSpecName "memcached-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:46:42 crc kubenswrapper[5052]: I1203 12:46:42.834582 5052 reconciler_common.go:293] "Volume detached for volume \"memcached-tls-certs\" (UniqueName: \"kubernetes.io/secret/4d42b068-9b91-4df6-9a9c-81cad009be16-memcached-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:42 crc kubenswrapper[5052]: I1203 12:46:42.896029 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:46:42 crc kubenswrapper[5052]: I1203 12:46:42.935410 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/320f928d-d6b0-48b4-b315-1a6c2ecd3380-combined-ca-bundle\") pod \"320f928d-d6b0-48b4-b315-1a6c2ecd3380\" (UID: \"320f928d-d6b0-48b4-b315-1a6c2ecd3380\") " Dec 03 12:46:42 crc kubenswrapper[5052]: I1203 12:46:42.935546 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/320f928d-d6b0-48b4-b315-1a6c2ecd3380-config-data\") pod \"320f928d-d6b0-48b4-b315-1a6c2ecd3380\" (UID: \"320f928d-d6b0-48b4-b315-1a6c2ecd3380\") " Dec 03 12:46:42 crc kubenswrapper[5052]: I1203 12:46:42.935730 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qc2cz\" (UniqueName: \"kubernetes.io/projected/320f928d-d6b0-48b4-b315-1a6c2ecd3380-kube-api-access-qc2cz\") pod \"320f928d-d6b0-48b4-b315-1a6c2ecd3380\" (UID: \"320f928d-d6b0-48b4-b315-1a6c2ecd3380\") " Dec 03 12:46:42 crc kubenswrapper[5052]: I1203 12:46:42.937847 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/placement-bbcfbbc9d-5q6gx" Dec 03 12:46:42 crc kubenswrapper[5052]: I1203 12:46:42.940953 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/320f928d-d6b0-48b4-b315-1a6c2ecd3380-kube-api-access-qc2cz" (OuterVolumeSpecName: "kube-api-access-qc2cz") pod "320f928d-d6b0-48b4-b315-1a6c2ecd3380" (UID: "320f928d-d6b0-48b4-b315-1a6c2ecd3380"). InnerVolumeSpecName "kube-api-access-qc2cz". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:46:42 crc kubenswrapper[5052]: I1203 12:46:42.989676 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/320f928d-d6b0-48b4-b315-1a6c2ecd3380-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "320f928d-d6b0-48b4-b315-1a6c2ecd3380" (UID: "320f928d-d6b0-48b4-b315-1a6c2ecd3380"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:46:42 crc kubenswrapper[5052]: I1203 12:46:42.995269 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/320f928d-d6b0-48b4-b315-1a6c2ecd3380-config-data" (OuterVolumeSpecName: "config-data") pod "320f928d-d6b0-48b4-b315-1a6c2ecd3380" (UID: "320f928d-d6b0-48b4-b315-1a6c2ecd3380"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:46:43 crc kubenswrapper[5052]: I1203 12:46:43.032166 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/memcached-0"] Dec 03 12:46:43 crc kubenswrapper[5052]: I1203 12:46:43.041428 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/320f928d-d6b0-48b4-b315-1a6c2ecd3380-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:43 crc kubenswrapper[5052]: I1203 12:46:43.041491 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/320f928d-d6b0-48b4-b315-1a6c2ecd3380-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:43 crc kubenswrapper[5052]: I1203 12:46:43.041522 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-qc2cz\" (UniqueName: \"kubernetes.io/projected/320f928d-d6b0-48b4-b315-1a6c2ecd3380-kube-api-access-qc2cz\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:43 crc kubenswrapper[5052]: I1203 12:46:43.087326 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/memcached-0"] Dec 03 12:46:43 crc kubenswrapper[5052]: I1203 12:46:43.098367 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/memcached-0"] Dec 03 12:46:43 crc kubenswrapper[5052]: E1203 12:46:43.100782 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="320f928d-d6b0-48b4-b315-1a6c2ecd3380" containerName="nova-cell1-conductor-conductor" Dec 03 12:46:43 crc kubenswrapper[5052]: I1203 12:46:43.100869 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="320f928d-d6b0-48b4-b315-1a6c2ecd3380" containerName="nova-cell1-conductor-conductor" Dec 03 12:46:43 crc kubenswrapper[5052]: E1203 12:46:43.100945 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="4d42b068-9b91-4df6-9a9c-81cad009be16" containerName="memcached" Dec 03 12:46:43 crc kubenswrapper[5052]: I1203 12:46:43.101029 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="4d42b068-9b91-4df6-9a9c-81cad009be16" containerName="memcached" Dec 03 12:46:43 crc kubenswrapper[5052]: I1203 12:46:43.101885 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="4d42b068-9b91-4df6-9a9c-81cad009be16" containerName="memcached" Dec 03 12:46:43 crc kubenswrapper[5052]: I1203 12:46:43.101994 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="320f928d-d6b0-48b4-b315-1a6c2ecd3380" containerName="nova-cell1-conductor-conductor" Dec 03 12:46:43 crc kubenswrapper[5052]: I1203 12:46:43.103101 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/memcached-0" Dec 03 12:46:43 crc kubenswrapper[5052]: I1203 12:46:43.109312 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/memcached-0"] Dec 03 12:46:43 crc kubenswrapper[5052]: I1203 12:46:43.169323 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"memcached-memcached-dockercfg-bshzz" Dec 03 12:46:43 crc kubenswrapper[5052]: I1203 12:46:43.169590 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-memcached-svc" Dec 03 12:46:43 crc kubenswrapper[5052]: I1203 12:46:43.156870 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"memcached-config-data" Dec 03 12:46:43 crc kubenswrapper[5052]: I1203 12:46:43.273871 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"memcached-tls-certs\" (UniqueName: \"kubernetes.io/secret/eef9c304-b598-4a31-9c47-09be0087c25e-memcached-tls-certs\") pod \"memcached-0\" (UID: \"eef9c304-b598-4a31-9c47-09be0087c25e\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:46:43 crc kubenswrapper[5052]: I1203 12:46:43.274000 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/eef9c304-b598-4a31-9c47-09be0087c25e-kolla-config\") pod \"memcached-0\" (UID: \"eef9c304-b598-4a31-9c47-09be0087c25e\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:46:43 crc kubenswrapper[5052]: I1203 12:46:43.274038 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/eef9c304-b598-4a31-9c47-09be0087c25e-combined-ca-bundle\") pod \"memcached-0\" (UID: \"eef9c304-b598-4a31-9c47-09be0087c25e\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:46:43 crc kubenswrapper[5052]: I1203 12:46:43.274358 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/eef9c304-b598-4a31-9c47-09be0087c25e-config-data\") pod \"memcached-0\" (UID: \"eef9c304-b598-4a31-9c47-09be0087c25e\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:46:43 crc kubenswrapper[5052]: I1203 12:46:43.274408 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m4lg2\" (UniqueName: \"kubernetes.io/projected/eef9c304-b598-4a31-9c47-09be0087c25e-kube-api-access-m4lg2\") pod \"memcached-0\" (UID: \"eef9c304-b598-4a31-9c47-09be0087c25e\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:46:43 crc kubenswrapper[5052]: I1203 12:46:43.377669 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"memcached-tls-certs\" (UniqueName: \"kubernetes.io/secret/eef9c304-b598-4a31-9c47-09be0087c25e-memcached-tls-certs\") pod \"memcached-0\" (UID: \"eef9c304-b598-4a31-9c47-09be0087c25e\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:46:43 crc kubenswrapper[5052]: I1203 12:46:43.377754 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/eef9c304-b598-4a31-9c47-09be0087c25e-kolla-config\") pod \"memcached-0\" (UID: \"eef9c304-b598-4a31-9c47-09be0087c25e\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:46:43 crc kubenswrapper[5052]: I1203 12:46:43.377780 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/eef9c304-b598-4a31-9c47-09be0087c25e-combined-ca-bundle\") pod \"memcached-0\" (UID: \"eef9c304-b598-4a31-9c47-09be0087c25e\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:46:43 crc kubenswrapper[5052]: I1203 12:46:43.377923 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/eef9c304-b598-4a31-9c47-09be0087c25e-config-data\") pod \"memcached-0\" (UID: \"eef9c304-b598-4a31-9c47-09be0087c25e\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:46:43 crc kubenswrapper[5052]: I1203 12:46:43.377953 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-m4lg2\" (UniqueName: \"kubernetes.io/projected/eef9c304-b598-4a31-9c47-09be0087c25e-kube-api-access-m4lg2\") pod \"memcached-0\" (UID: \"eef9c304-b598-4a31-9c47-09be0087c25e\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:46:43 crc kubenswrapper[5052]: I1203 12:46:43.379059 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/eef9c304-b598-4a31-9c47-09be0087c25e-kolla-config\") pod \"memcached-0\" (UID: \"eef9c304-b598-4a31-9c47-09be0087c25e\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:46:43 crc kubenswrapper[5052]: I1203 12:46:43.380901 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/eef9c304-b598-4a31-9c47-09be0087c25e-config-data\") pod \"memcached-0\" (UID: \"eef9c304-b598-4a31-9c47-09be0087c25e\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:46:43 crc kubenswrapper[5052]: I1203 12:46:43.384282 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"memcached-tls-certs\" (UniqueName: \"kubernetes.io/secret/eef9c304-b598-4a31-9c47-09be0087c25e-memcached-tls-certs\") pod \"memcached-0\" (UID: \"eef9c304-b598-4a31-9c47-09be0087c25e\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:46:43 crc kubenswrapper[5052]: I1203 12:46:43.387781 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/eef9c304-b598-4a31-9c47-09be0087c25e-combined-ca-bundle\") pod \"memcached-0\" (UID: \"eef9c304-b598-4a31-9c47-09be0087c25e\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:46:43 crc kubenswrapper[5052]: I1203 12:46:43.401753 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-m4lg2\" (UniqueName: \"kubernetes.io/projected/eef9c304-b598-4a31-9c47-09be0087c25e-kube-api-access-m4lg2\") pod \"memcached-0\" (UID: \"eef9c304-b598-4a31-9c47-09be0087c25e\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:46:43 crc kubenswrapper[5052]: E1203 12:46:43.480998 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-barbican-internal-svc: secret "cert-barbican-internal-svc" not found Dec 03 12:46:43 crc kubenswrapper[5052]: E1203 12:46:43.481078 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5b63f869-c95a-4279-b542-3ebbec1a1fd5-internal-tls-certs podName:5b63f869-c95a-4279-b542-3ebbec1a1fd5 nodeName:}" failed. No retries permitted until 2025-12-03 12:46:59.4810609 +0000 UTC m=+5210.069302654 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "internal-tls-certs" (UniqueName: "kubernetes.io/secret/5b63f869-c95a-4279-b542-3ebbec1a1fd5-internal-tls-certs") pod "barbican-api-6d86f7b884-lf4pd" (UID: "5b63f869-c95a-4279-b542-3ebbec1a1fd5") : secret "cert-barbican-internal-svc" not found Dec 03 12:46:43 crc kubenswrapper[5052]: I1203 12:46:43.503661 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/memcached-0" Dec 03 12:46:43 crc kubenswrapper[5052]: E1203 12:46:43.527776 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of 26ed2e6897ff71a51086a7cc455488c21ad8adffd038be8de42d13bd1335a6fd is running failed: container process not found" containerID="26ed2e6897ff71a51086a7cc455488c21ad8adffd038be8de42d13bd1335a6fd" cmd=["/usr/bin/pgrep","-r","DRST","nova-scheduler"] Dec 03 12:46:43 crc kubenswrapper[5052]: E1203 12:46:43.529442 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of 26ed2e6897ff71a51086a7cc455488c21ad8adffd038be8de42d13bd1335a6fd is running failed: container process not found" containerID="26ed2e6897ff71a51086a7cc455488c21ad8adffd038be8de42d13bd1335a6fd" cmd=["/usr/bin/pgrep","-r","DRST","nova-scheduler"] Dec 03 12:46:43 crc kubenswrapper[5052]: E1203 12:46:43.529719 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of 26ed2e6897ff71a51086a7cc455488c21ad8adffd038be8de42d13bd1335a6fd is running failed: container process not found" containerID="26ed2e6897ff71a51086a7cc455488c21ad8adffd038be8de42d13bd1335a6fd" cmd=["/usr/bin/pgrep","-r","DRST","nova-scheduler"] Dec 03 12:46:43 crc kubenswrapper[5052]: E1203 12:46:43.529761 5052 prober.go:104] "Probe errored" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of 26ed2e6897ff71a51086a7cc455488c21ad8adffd038be8de42d13bd1335a6fd is running failed: container process not found" probeType="Readiness" pod="openstack-kuttl-tests/nova-scheduler-0" podUID="81709983-1fd0-4446-bbbf-fb94653bb3d2" containerName="nova-scheduler-scheduler" Dec 03 12:46:43 crc kubenswrapper[5052]: I1203 12:46:43.686934 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4d42b068-9b91-4df6-9a9c-81cad009be16" path="/var/lib/kubelet/pods/4d42b068-9b91-4df6-9a9c-81cad009be16/volumes" Dec 03 12:46:43 crc kubenswrapper[5052]: I1203 12:46:43.687693 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="901a2f0b-fb8e-4c97-aea7-567d7ee42a23" path="/var/lib/kubelet/pods/901a2f0b-fb8e-4c97-aea7-567d7ee42a23/volumes" Dec 03 12:46:43 crc kubenswrapper[5052]: I1203 12:46:43.688730 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-bcd9f4f4-djpt8" event={"ID":"049a231b-3a12-4710-9b7a-117b1bb82e72","Type":"ContainerStarted","Data":"3349ccb1c242563324485e03dbd9f9320ac9a751d16ea4130a94da5db86659ab"} Dec 03 12:46:43 crc kubenswrapper[5052]: I1203 12:46:43.688761 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/neutron-bcd9f4f4-djpt8" Dec 03 12:46:43 crc kubenswrapper[5052]: I1203 12:46:43.689814 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-sb-0" event={"ID":"84f26afe-5b0c-43bb-a4c8-a9ac21d24330","Type":"ContainerStarted","Data":"721e21ea9214ab0b7f428a1744d9bf1b20c7493291c82e15a89dd5d11891df97"} Dec 03 12:46:43 crc kubenswrapper[5052]: I1203 12:46:43.689845 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-sb-0" event={"ID":"84f26afe-5b0c-43bb-a4c8-a9ac21d24330","Type":"ContainerStarted","Data":"b74065369c814d7538cce05dc1a2188d258f8a0970dbde7fa52adce059389dc5"} Dec 03 12:46:43 crc kubenswrapper[5052]: I1203 12:46:43.689854 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-sb-0" event={"ID":"84f26afe-5b0c-43bb-a4c8-a9ac21d24330","Type":"ContainerStarted","Data":"2bf9da39b82d602ec1c4f37d5f315087e20eb503a371d87ae4e4bada773f70c7"} Dec 03 12:46:43 crc kubenswrapper[5052]: I1203 12:46:43.704222 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:46:43 crc kubenswrapper[5052]: I1203 12:46:43.704851 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-conductor-0" event={"ID":"320f928d-d6b0-48b4-b315-1a6c2ecd3380","Type":"ContainerDied","Data":"06ccbb7e691545be128b7411d4d7649f97da7c37d7109751a533219c6552676c"} Dec 03 12:46:43 crc kubenswrapper[5052]: I1203 12:46:43.704936 5052 scope.go:117] "RemoveContainer" containerID="2ff6796b6955202076b9ca76f2bd9d074d96d820f421983058345a0af1b54565" Dec 03 12:46:43 crc kubenswrapper[5052]: I1203 12:46:43.740070 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"d5719b07-205b-4e6a-9fe2-b6531b779b0a","Type":"ContainerStarted","Data":"033df125bf601aa712f091f9637bf143ce864965096aeccefa080e46608d8437"} Dec 03 12:46:43 crc kubenswrapper[5052]: I1203 12:46:43.740118 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"d5719b07-205b-4e6a-9fe2-b6531b779b0a","Type":"ContainerStarted","Data":"32f0eb2434b003178dbbca6a7f974c5933898aa66a832336cc4f079190148796"} Dec 03 12:46:43 crc kubenswrapper[5052]: I1203 12:46:43.743124 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/ovsdbserver-sb-0" podStartSLOduration=3.7431010540000003 podStartE2EDuration="3.743101054s" podCreationTimestamp="2025-12-03 12:46:40 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:46:43.740347085 +0000 UTC m=+5194.328588859" watchObservedRunningTime="2025-12-03 12:46:43.743101054 +0000 UTC m=+5194.331342818" Dec 03 12:46:43 crc kubenswrapper[5052]: I1203 12:46:43.747469 5052 generic.go:334] "Generic (PLEG): container finished" podID="81709983-1fd0-4446-bbbf-fb94653bb3d2" containerID="26ed2e6897ff71a51086a7cc455488c21ad8adffd038be8de42d13bd1335a6fd" exitCode=1 Dec 03 12:46:43 crc kubenswrapper[5052]: I1203 12:46:43.747545 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"81709983-1fd0-4446-bbbf-fb94653bb3d2","Type":"ContainerDied","Data":"26ed2e6897ff71a51086a7cc455488c21ad8adffd038be8de42d13bd1335a6fd"} Dec 03 12:46:43 crc kubenswrapper[5052]: I1203 12:46:43.748341 5052 scope.go:117] "RemoveContainer" containerID="26ed2e6897ff71a51086a7cc455488c21ad8adffd038be8de42d13bd1335a6fd" Dec 03 12:46:43 crc kubenswrapper[5052]: I1203 12:46:43.751411 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/neutron-bcd9f4f4-djpt8" podStartSLOduration=4.751389491 podStartE2EDuration="4.751389491s" podCreationTimestamp="2025-12-03 12:46:39 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:46:43.710911034 +0000 UTC m=+5194.299152798" watchObservedRunningTime="2025-12-03 12:46:43.751389491 +0000 UTC m=+5194.339631255" Dec 03 12:46:43 crc kubenswrapper[5052]: I1203 12:46:43.773069 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-nb-0" event={"ID":"2005f89c-8a19-403c-acc9-c13ede717372","Type":"ContainerStarted","Data":"c8c8497d5896f2a84e4979c47d2430cbec8075e3536d1d41039d14854df9a795"} Dec 03 12:46:43 crc kubenswrapper[5052]: I1203 12:46:43.773122 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-nb-0" event={"ID":"2005f89c-8a19-403c-acc9-c13ede717372","Type":"ContainerStarted","Data":"14ec8a29b2f04a213b48a3edd706ccb3eed3920a11add7c7ed0658a51c533ce3"} Dec 03 12:46:43 crc kubenswrapper[5052]: I1203 12:46:43.786767 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" event={"ID":"5cafb759-6926-4fd6-a77e-7e18a9becd03","Type":"ContainerStarted","Data":"a1eac786eb973740a08ecdd82bac9b30836804efc46b172c9ef6ef222f8a88c5"} Dec 03 12:46:43 crc kubenswrapper[5052]: I1203 12:46:43.794642 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-api-0" podStartSLOduration=3.794625258 podStartE2EDuration="3.794625258s" podCreationTimestamp="2025-12-03 12:46:40 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:46:43.783267033 +0000 UTC m=+5194.371508787" watchObservedRunningTime="2025-12-03 12:46:43.794625258 +0000 UTC m=+5194.382867022" Dec 03 12:46:43 crc kubenswrapper[5052]: I1203 12:46:43.812564 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-0"] Dec 03 12:46:43 crc kubenswrapper[5052]: I1203 12:46:43.853578 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-0"] Dec 03 12:46:43 crc kubenswrapper[5052]: I1203 12:46:43.873065 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-0"] Dec 03 12:46:43 crc kubenswrapper[5052]: I1203 12:46:43.877955 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" podStartSLOduration=3.877903919 podStartE2EDuration="3.877903919s" podCreationTimestamp="2025-12-03 12:46:40 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:46:43.821935729 +0000 UTC m=+5194.410177503" watchObservedRunningTime="2025-12-03 12:46:43.877903919 +0000 UTC m=+5194.466145683" Dec 03 12:46:43 crc kubenswrapper[5052]: I1203 12:46:43.880053 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:46:43 crc kubenswrapper[5052]: I1203 12:46:43.883862 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell1-conductor-config-data" Dec 03 12:46:43 crc kubenswrapper[5052]: I1203 12:46:43.910373 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/ovsdbserver-nb-0" podStartSLOduration=3.910357767 podStartE2EDuration="3.910357767s" podCreationTimestamp="2025-12-03 12:46:40 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:46:43.849040134 +0000 UTC m=+5194.437281898" watchObservedRunningTime="2025-12-03 12:46:43.910357767 +0000 UTC m=+5194.498599531" Dec 03 12:46:43 crc kubenswrapper[5052]: I1203 12:46:43.910521 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-0"] Dec 03 12:46:43 crc kubenswrapper[5052]: I1203 12:46:43.995908 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/c069aee2-4cb6-4c8a-bbe5-949ab0b48331-internal-tls-certs\") pod \"barbican-api-bcf49b784-jnkdb\" (UID: \"c069aee2-4cb6-4c8a-bbe5-949ab0b48331\") " pod="openstack-kuttl-tests/barbican-api-bcf49b784-jnkdb" Dec 03 12:46:43 crc kubenswrapper[5052]: I1203 12:46:43.995998 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8669832d-c47a-49bd-9491-dc7141ac6281-combined-ca-bundle\") pod \"nova-cell1-conductor-0\" (UID: \"8669832d-c47a-49bd-9491-dc7141ac6281\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:46:43 crc kubenswrapper[5052]: I1203 12:46:43.996151 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/8669832d-c47a-49bd-9491-dc7141ac6281-config-data\") pod \"nova-cell1-conductor-0\" (UID: \"8669832d-c47a-49bd-9491-dc7141ac6281\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:46:43 crc kubenswrapper[5052]: I1203 12:46:43.996225 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-66gvc\" (UniqueName: \"kubernetes.io/projected/8669832d-c47a-49bd-9491-dc7141ac6281-kube-api-access-66gvc\") pod \"nova-cell1-conductor-0\" (UID: \"8669832d-c47a-49bd-9491-dc7141ac6281\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:46:43 crc kubenswrapper[5052]: E1203 12:46:43.996431 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-barbican-internal-svc: secret "cert-barbican-internal-svc" not found Dec 03 12:46:43 crc kubenswrapper[5052]: E1203 12:46:43.996488 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c069aee2-4cb6-4c8a-bbe5-949ab0b48331-internal-tls-certs podName:c069aee2-4cb6-4c8a-bbe5-949ab0b48331 nodeName:}" failed. No retries permitted until 2025-12-03 12:47:15.99646797 +0000 UTC m=+5226.584709734 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "internal-tls-certs" (UniqueName: "kubernetes.io/secret/c069aee2-4cb6-4c8a-bbe5-949ab0b48331-internal-tls-certs") pod "barbican-api-bcf49b784-jnkdb" (UID: "c069aee2-4cb6-4c8a-bbe5-949ab0b48331") : secret "cert-barbican-internal-svc" not found Dec 03 12:46:44 crc kubenswrapper[5052]: I1203 12:46:44.077777 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/memcached-0"] Dec 03 12:46:44 crc kubenswrapper[5052]: I1203 12:46:44.098150 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/8669832d-c47a-49bd-9491-dc7141ac6281-config-data\") pod \"nova-cell1-conductor-0\" (UID: \"8669832d-c47a-49bd-9491-dc7141ac6281\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:46:44 crc kubenswrapper[5052]: I1203 12:46:44.098250 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-66gvc\" (UniqueName: \"kubernetes.io/projected/8669832d-c47a-49bd-9491-dc7141ac6281-kube-api-access-66gvc\") pod \"nova-cell1-conductor-0\" (UID: \"8669832d-c47a-49bd-9491-dc7141ac6281\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:46:44 crc kubenswrapper[5052]: I1203 12:46:44.098362 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8669832d-c47a-49bd-9491-dc7141ac6281-combined-ca-bundle\") pod \"nova-cell1-conductor-0\" (UID: \"8669832d-c47a-49bd-9491-dc7141ac6281\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:46:44 crc kubenswrapper[5052]: I1203 12:46:44.106903 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8669832d-c47a-49bd-9491-dc7141ac6281-combined-ca-bundle\") pod \"nova-cell1-conductor-0\" (UID: \"8669832d-c47a-49bd-9491-dc7141ac6281\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:46:44 crc kubenswrapper[5052]: I1203 12:46:44.106926 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/8669832d-c47a-49bd-9491-dc7141ac6281-config-data\") pod \"nova-cell1-conductor-0\" (UID: \"8669832d-c47a-49bd-9491-dc7141ac6281\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:46:44 crc kubenswrapper[5052]: I1203 12:46:44.118082 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-66gvc\" (UniqueName: \"kubernetes.io/projected/8669832d-c47a-49bd-9491-dc7141ac6281-kube-api-access-66gvc\") pod \"nova-cell1-conductor-0\" (UID: \"8669832d-c47a-49bd-9491-dc7141ac6281\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:46:44 crc kubenswrapper[5052]: I1203 12:46:44.199784 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/fa1c223d-82fa-488c-ac48-e88bce36a839-public-tls-certs\") pod \"keystone-687ff6fcfc-9vwb2\" (UID: \"fa1c223d-82fa-488c-ac48-e88bce36a839\") " pod="openstack-kuttl-tests/keystone-687ff6fcfc-9vwb2" Dec 03 12:46:44 crc kubenswrapper[5052]: I1203 12:46:44.200230 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/fa1c223d-82fa-488c-ac48-e88bce36a839-internal-tls-certs\") pod \"keystone-687ff6fcfc-9vwb2\" (UID: \"fa1c223d-82fa-488c-ac48-e88bce36a839\") " pod="openstack-kuttl-tests/keystone-687ff6fcfc-9vwb2" Dec 03 12:46:44 crc kubenswrapper[5052]: E1203 12:46:44.200502 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-keystone-internal-svc: secret "cert-keystone-internal-svc" not found Dec 03 12:46:44 crc kubenswrapper[5052]: E1203 12:46:44.200586 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fa1c223d-82fa-488c-ac48-e88bce36a839-internal-tls-certs podName:fa1c223d-82fa-488c-ac48-e88bce36a839 nodeName:}" failed. No retries permitted until 2025-12-03 12:47:16.200560407 +0000 UTC m=+5226.788802171 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "internal-tls-certs" (UniqueName: "kubernetes.io/secret/fa1c223d-82fa-488c-ac48-e88bce36a839-internal-tls-certs") pod "keystone-687ff6fcfc-9vwb2" (UID: "fa1c223d-82fa-488c-ac48-e88bce36a839") : secret "cert-keystone-internal-svc" not found Dec 03 12:46:44 crc kubenswrapper[5052]: I1203 12:46:44.209481 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/fa1c223d-82fa-488c-ac48-e88bce36a839-public-tls-certs\") pod \"keystone-687ff6fcfc-9vwb2\" (UID: \"fa1c223d-82fa-488c-ac48-e88bce36a839\") " pod="openstack-kuttl-tests/keystone-687ff6fcfc-9vwb2" Dec 03 12:46:44 crc kubenswrapper[5052]: I1203 12:46:44.282871 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:46:44 crc kubenswrapper[5052]: I1203 12:46:44.339673 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:46:44 crc kubenswrapper[5052]: I1203 12:46:44.341618 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/placement-bbcfbbc9d-5q6gx"] Dec 03 12:46:44 crc kubenswrapper[5052]: I1203 12:46:44.341740 5052 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Dec 03 12:46:44 crc kubenswrapper[5052]: I1203 12:46:44.341858 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/placement-bbcfbbc9d-5q6gx" podUID="32f420c2-aa7e-44c8-a63a-7654aa3e6e34" containerName="placement-log" containerID="cri-o://4c8ae2743fa5049ef6f420e9338425946c81202ba97ccaf63ece2ed399a93b82" gracePeriod=30 Dec 03 12:46:44 crc kubenswrapper[5052]: I1203 12:46:44.342432 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/placement-bbcfbbc9d-5q6gx" podUID="32f420c2-aa7e-44c8-a63a-7654aa3e6e34" containerName="placement-api" containerID="cri-o://353ad09a2eae7f22cab9c43ca6063becb9fb6d4010c1a66dcef0e6673635198d" gracePeriod=30 Dec 03 12:46:44 crc kubenswrapper[5052]: I1203 12:46:44.410136 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/placement-6d46dc454-rpjwz"] Dec 03 12:46:44 crc kubenswrapper[5052]: I1203 12:46:44.412591 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-6d46dc454-rpjwz" Dec 03 12:46:44 crc kubenswrapper[5052]: I1203 12:46:44.454943 5052 prober.go:107] "Probe failed" probeType="Liveness" pod="openstack-kuttl-tests/placement-bbcfbbc9d-5q6gx" podUID="32f420c2-aa7e-44c8-a63a-7654aa3e6e34" containerName="placement-api" probeResult="failure" output="Get \"https://10.217.1.241:8778/\": EOF" Dec 03 12:46:44 crc kubenswrapper[5052]: I1203 12:46:44.455368 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/placement-bbcfbbc9d-5q6gx" podUID="32f420c2-aa7e-44c8-a63a-7654aa3e6e34" containerName="placement-api" probeResult="failure" output="Get \"https://10.217.1.241:8778/\": EOF" Dec 03 12:46:44 crc kubenswrapper[5052]: I1203 12:46:44.464591 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/placement-6d46dc454-rpjwz"] Dec 03 12:46:44 crc kubenswrapper[5052]: I1203 12:46:44.472500 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/placement-bbcfbbc9d-5q6gx" Dec 03 12:46:44 crc kubenswrapper[5052]: I1203 12:46:44.472853 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:46:44 crc kubenswrapper[5052]: I1203 12:46:44.513749 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/8fbc67c1-6c51-4c53-85c0-1cf3fa2d5fc2-scripts\") pod \"placement-6d46dc454-rpjwz\" (UID: \"8fbc67c1-6c51-4c53-85c0-1cf3fa2d5fc2\") " pod="openstack-kuttl-tests/placement-6d46dc454-rpjwz" Dec 03 12:46:44 crc kubenswrapper[5052]: I1203 12:46:44.513791 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/8fbc67c1-6c51-4c53-85c0-1cf3fa2d5fc2-logs\") pod \"placement-6d46dc454-rpjwz\" (UID: \"8fbc67c1-6c51-4c53-85c0-1cf3fa2d5fc2\") " pod="openstack-kuttl-tests/placement-6d46dc454-rpjwz" Dec 03 12:46:44 crc kubenswrapper[5052]: I1203 12:46:44.513812 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/8fbc67c1-6c51-4c53-85c0-1cf3fa2d5fc2-public-tls-certs\") pod \"placement-6d46dc454-rpjwz\" (UID: \"8fbc67c1-6c51-4c53-85c0-1cf3fa2d5fc2\") " pod="openstack-kuttl-tests/placement-6d46dc454-rpjwz" Dec 03 12:46:44 crc kubenswrapper[5052]: I1203 12:46:44.513862 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8fbc67c1-6c51-4c53-85c0-1cf3fa2d5fc2-combined-ca-bundle\") pod \"placement-6d46dc454-rpjwz\" (UID: \"8fbc67c1-6c51-4c53-85c0-1cf3fa2d5fc2\") " pod="openstack-kuttl-tests/placement-6d46dc454-rpjwz" Dec 03 12:46:44 crc kubenswrapper[5052]: I1203 12:46:44.513880 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jndtz\" (UniqueName: \"kubernetes.io/projected/8fbc67c1-6c51-4c53-85c0-1cf3fa2d5fc2-kube-api-access-jndtz\") pod \"placement-6d46dc454-rpjwz\" (UID: \"8fbc67c1-6c51-4c53-85c0-1cf3fa2d5fc2\") " pod="openstack-kuttl-tests/placement-6d46dc454-rpjwz" Dec 03 12:46:44 crc kubenswrapper[5052]: I1203 12:46:44.513930 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/8fbc67c1-6c51-4c53-85c0-1cf3fa2d5fc2-internal-tls-certs\") pod \"placement-6d46dc454-rpjwz\" (UID: \"8fbc67c1-6c51-4c53-85c0-1cf3fa2d5fc2\") " pod="openstack-kuttl-tests/placement-6d46dc454-rpjwz" Dec 03 12:46:44 crc kubenswrapper[5052]: I1203 12:46:44.513956 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/8fbc67c1-6c51-4c53-85c0-1cf3fa2d5fc2-config-data\") pod \"placement-6d46dc454-rpjwz\" (UID: \"8fbc67c1-6c51-4c53-85c0-1cf3fa2d5fc2\") " pod="openstack-kuttl-tests/placement-6d46dc454-rpjwz" Dec 03 12:46:44 crc kubenswrapper[5052]: I1203 12:46:44.627734 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/8fbc67c1-6c51-4c53-85c0-1cf3fa2d5fc2-scripts\") pod \"placement-6d46dc454-rpjwz\" (UID: \"8fbc67c1-6c51-4c53-85c0-1cf3fa2d5fc2\") " pod="openstack-kuttl-tests/placement-6d46dc454-rpjwz" Dec 03 12:46:44 crc kubenswrapper[5052]: I1203 12:46:44.628097 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/8fbc67c1-6c51-4c53-85c0-1cf3fa2d5fc2-logs\") pod \"placement-6d46dc454-rpjwz\" (UID: \"8fbc67c1-6c51-4c53-85c0-1cf3fa2d5fc2\") " pod="openstack-kuttl-tests/placement-6d46dc454-rpjwz" Dec 03 12:46:44 crc kubenswrapper[5052]: I1203 12:46:44.628121 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/8fbc67c1-6c51-4c53-85c0-1cf3fa2d5fc2-public-tls-certs\") pod \"placement-6d46dc454-rpjwz\" (UID: \"8fbc67c1-6c51-4c53-85c0-1cf3fa2d5fc2\") " pod="openstack-kuttl-tests/placement-6d46dc454-rpjwz" Dec 03 12:46:44 crc kubenswrapper[5052]: I1203 12:46:44.628209 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8fbc67c1-6c51-4c53-85c0-1cf3fa2d5fc2-combined-ca-bundle\") pod \"placement-6d46dc454-rpjwz\" (UID: \"8fbc67c1-6c51-4c53-85c0-1cf3fa2d5fc2\") " pod="openstack-kuttl-tests/placement-6d46dc454-rpjwz" Dec 03 12:46:44 crc kubenswrapper[5052]: I1203 12:46:44.628229 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-jndtz\" (UniqueName: \"kubernetes.io/projected/8fbc67c1-6c51-4c53-85c0-1cf3fa2d5fc2-kube-api-access-jndtz\") pod \"placement-6d46dc454-rpjwz\" (UID: \"8fbc67c1-6c51-4c53-85c0-1cf3fa2d5fc2\") " pod="openstack-kuttl-tests/placement-6d46dc454-rpjwz" Dec 03 12:46:44 crc kubenswrapper[5052]: I1203 12:46:44.628773 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/8fbc67c1-6c51-4c53-85c0-1cf3fa2d5fc2-internal-tls-certs\") pod \"placement-6d46dc454-rpjwz\" (UID: \"8fbc67c1-6c51-4c53-85c0-1cf3fa2d5fc2\") " pod="openstack-kuttl-tests/placement-6d46dc454-rpjwz" Dec 03 12:46:44 crc kubenswrapper[5052]: I1203 12:46:44.628855 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/8fbc67c1-6c51-4c53-85c0-1cf3fa2d5fc2-config-data\") pod \"placement-6d46dc454-rpjwz\" (UID: \"8fbc67c1-6c51-4c53-85c0-1cf3fa2d5fc2\") " pod="openstack-kuttl-tests/placement-6d46dc454-rpjwz" Dec 03 12:46:44 crc kubenswrapper[5052]: I1203 12:46:44.630428 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/8fbc67c1-6c51-4c53-85c0-1cf3fa2d5fc2-logs\") pod \"placement-6d46dc454-rpjwz\" (UID: \"8fbc67c1-6c51-4c53-85c0-1cf3fa2d5fc2\") " pod="openstack-kuttl-tests/placement-6d46dc454-rpjwz" Dec 03 12:46:44 crc kubenswrapper[5052]: I1203 12:46:44.636698 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/8fbc67c1-6c51-4c53-85c0-1cf3fa2d5fc2-scripts\") pod \"placement-6d46dc454-rpjwz\" (UID: \"8fbc67c1-6c51-4c53-85c0-1cf3fa2d5fc2\") " pod="openstack-kuttl-tests/placement-6d46dc454-rpjwz" Dec 03 12:46:44 crc kubenswrapper[5052]: I1203 12:46:44.637616 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/8fbc67c1-6c51-4c53-85c0-1cf3fa2d5fc2-config-data\") pod \"placement-6d46dc454-rpjwz\" (UID: \"8fbc67c1-6c51-4c53-85c0-1cf3fa2d5fc2\") " pod="openstack-kuttl-tests/placement-6d46dc454-rpjwz" Dec 03 12:46:44 crc kubenswrapper[5052]: I1203 12:46:44.638028 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/8fbc67c1-6c51-4c53-85c0-1cf3fa2d5fc2-public-tls-certs\") pod \"placement-6d46dc454-rpjwz\" (UID: \"8fbc67c1-6c51-4c53-85c0-1cf3fa2d5fc2\") " pod="openstack-kuttl-tests/placement-6d46dc454-rpjwz" Dec 03 12:46:44 crc kubenswrapper[5052]: I1203 12:46:44.643769 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/8fbc67c1-6c51-4c53-85c0-1cf3fa2d5fc2-internal-tls-certs\") pod \"placement-6d46dc454-rpjwz\" (UID: \"8fbc67c1-6c51-4c53-85c0-1cf3fa2d5fc2\") " pod="openstack-kuttl-tests/placement-6d46dc454-rpjwz" Dec 03 12:46:44 crc kubenswrapper[5052]: I1203 12:46:44.644412 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8fbc67c1-6c51-4c53-85c0-1cf3fa2d5fc2-combined-ca-bundle\") pod \"placement-6d46dc454-rpjwz\" (UID: \"8fbc67c1-6c51-4c53-85c0-1cf3fa2d5fc2\") " pod="openstack-kuttl-tests/placement-6d46dc454-rpjwz" Dec 03 12:46:44 crc kubenswrapper[5052]: I1203 12:46:44.657441 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-jndtz\" (UniqueName: \"kubernetes.io/projected/8fbc67c1-6c51-4c53-85c0-1cf3fa2d5fc2-kube-api-access-jndtz\") pod \"placement-6d46dc454-rpjwz\" (UID: \"8fbc67c1-6c51-4c53-85c0-1cf3fa2d5fc2\") " pod="openstack-kuttl-tests/placement-6d46dc454-rpjwz" Dec 03 12:46:44 crc kubenswrapper[5052]: I1203 12:46:44.672815 5052 scope.go:117] "RemoveContainer" containerID="965af26b70386d0042f76a35140b047b193a7f695cf03cf43e91184f27f9aaa1" Dec 03 12:46:44 crc kubenswrapper[5052]: I1203 12:46:44.766146 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-6d46dc454-rpjwz" Dec 03 12:46:44 crc kubenswrapper[5052]: I1203 12:46:44.799403 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"81709983-1fd0-4446-bbbf-fb94653bb3d2","Type":"ContainerStarted","Data":"c397988625a9ab9075d72d6f8c51dd95b36e8b2a4214beb4c4dcf9e2ea912582"} Dec 03 12:46:44 crc kubenswrapper[5052]: I1203 12:46:44.802791 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/memcached-0" event={"ID":"eef9c304-b598-4a31-9c47-09be0087c25e","Type":"ContainerStarted","Data":"6423f3d02f84e310e7c409b2c1c1e2b55e3efb427edea3e0e3f82312b7c58d5c"} Dec 03 12:46:44 crc kubenswrapper[5052]: I1203 12:46:44.806802 5052 generic.go:334] "Generic (PLEG): container finished" podID="32f420c2-aa7e-44c8-a63a-7654aa3e6e34" containerID="4c8ae2743fa5049ef6f420e9338425946c81202ba97ccaf63ece2ed399a93b82" exitCode=143 Dec 03 12:46:44 crc kubenswrapper[5052]: I1203 12:46:44.806930 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-bbcfbbc9d-5q6gx" event={"ID":"32f420c2-aa7e-44c8-a63a-7654aa3e6e34","Type":"ContainerDied","Data":"4c8ae2743fa5049ef6f420e9338425946c81202ba97ccaf63ece2ed399a93b82"} Dec 03 12:46:45 crc kubenswrapper[5052]: I1203 12:46:45.017945 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-0"] Dec 03 12:46:45 crc kubenswrapper[5052]: W1203 12:46:45.043794 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8669832d_c47a_49bd_9491_dc7141ac6281.slice/crio-71aa97adfbba84a05616bbe88826b6bf5072c85c532f501d3d903dd85d92b0bb WatchSource:0}: Error finding container 71aa97adfbba84a05616bbe88826b6bf5072c85c532f501d3d903dd85d92b0bb: Status 404 returned error can't find the container with id 71aa97adfbba84a05616bbe88826b6bf5072c85c532f501d3d903dd85d92b0bb Dec 03 12:46:45 crc kubenswrapper[5052]: I1203 12:46:45.170413 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/placement-6d46dc454-rpjwz"] Dec 03 12:46:45 crc kubenswrapper[5052]: I1203 12:46:45.239962 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/openstack-cell1-galera-0"] Dec 03 12:46:45 crc kubenswrapper[5052]: E1203 12:46:45.240810 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[galera-tls-certs], unattached volumes=[], failed to process volumes=[]: context canceled" pod="openstack-kuttl-tests/openstack-cell1-galera-0" podUID="576b2af7-7647-4a45-9423-3aa752d84e75" Dec 03 12:46:45 crc kubenswrapper[5052]: I1203 12:46:45.685215 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="320f928d-d6b0-48b4-b315-1a6c2ecd3380" path="/var/lib/kubelet/pods/320f928d-d6b0-48b4-b315-1a6c2ecd3380/volumes" Dec 03 12:46:45 crc kubenswrapper[5052]: I1203 12:46:45.827566 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-conductor-0" event={"ID":"8669832d-c47a-49bd-9491-dc7141ac6281","Type":"ContainerStarted","Data":"d845f1b1ea8802fd6fd4c7f6098f521746fa875f2f1e1c94973d2bf864333d42"} Dec 03 12:46:45 crc kubenswrapper[5052]: I1203 12:46:45.827990 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:46:45 crc kubenswrapper[5052]: I1203 12:46:45.828003 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-conductor-0" event={"ID":"8669832d-c47a-49bd-9491-dc7141ac6281","Type":"ContainerStarted","Data":"71aa97adfbba84a05616bbe88826b6bf5072c85c532f501d3d903dd85d92b0bb"} Dec 03 12:46:45 crc kubenswrapper[5052]: I1203 12:46:45.829826 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-conductor-0" event={"ID":"6d653682-52e2-4940-af66-633475bfe09a","Type":"ContainerStarted","Data":"4c1d25fd5e7f80502770dbe44896eb5a3327b1bebde080de0e66dead94625111"} Dec 03 12:46:45 crc kubenswrapper[5052]: I1203 12:46:45.831023 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:46:45 crc kubenswrapper[5052]: I1203 12:46:45.841689 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-6d46dc454-rpjwz" event={"ID":"8fbc67c1-6c51-4c53-85c0-1cf3fa2d5fc2","Type":"ContainerStarted","Data":"4c37d110de42cbaa1349c97632b6d688bda0d1d1a7cefd193e59a9fbe693828d"} Dec 03 12:46:45 crc kubenswrapper[5052]: I1203 12:46:45.841836 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-6d46dc454-rpjwz" event={"ID":"8fbc67c1-6c51-4c53-85c0-1cf3fa2d5fc2","Type":"ContainerStarted","Data":"7a4502b4df7e176325fc9d9d7127106b358c7a727cbfa2e138f865de18f5629a"} Dec 03 12:46:45 crc kubenswrapper[5052]: I1203 12:46:45.845911 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/memcached-0" event={"ID":"eef9c304-b598-4a31-9c47-09be0087c25e","Type":"ContainerStarted","Data":"3cec53cfcf32bebda98bd1f984b0bb1ae1c9eaa0f2a096f12724a271e1103ca8"} Dec 03 12:46:45 crc kubenswrapper[5052]: I1203 12:46:45.845951 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/memcached-0" Dec 03 12:46:45 crc kubenswrapper[5052]: I1203 12:46:45.847602 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:46:45 crc kubenswrapper[5052]: I1203 12:46:45.868634 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-cell1-conductor-0" podStartSLOduration=2.868613711 podStartE2EDuration="2.868613711s" podCreationTimestamp="2025-12-03 12:46:43 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:46:45.847649251 +0000 UTC m=+5196.435891015" watchObservedRunningTime="2025-12-03 12:46:45.868613711 +0000 UTC m=+5196.456855475" Dec 03 12:46:45 crc kubenswrapper[5052]: I1203 12:46:45.878393 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:46:45 crc kubenswrapper[5052]: I1203 12:46:45.901197 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/memcached-0" podStartSLOduration=2.901174672 podStartE2EDuration="2.901174672s" podCreationTimestamp="2025-12-03 12:46:43 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:46:45.885090252 +0000 UTC m=+5196.473332016" watchObservedRunningTime="2025-12-03 12:46:45.901174672 +0000 UTC m=+5196.489416446" Dec 03 12:46:46 crc kubenswrapper[5052]: I1203 12:46:46.038332 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x5ltc\" (UniqueName: \"kubernetes.io/projected/576b2af7-7647-4a45-9423-3aa752d84e75-kube-api-access-x5ltc\") pod \"576b2af7-7647-4a45-9423-3aa752d84e75\" (UID: \"576b2af7-7647-4a45-9423-3aa752d84e75\") " Dec 03 12:46:46 crc kubenswrapper[5052]: I1203 12:46:46.038792 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"mysql-db\" (UniqueName: \"kubernetes.io/local-volume/local-storage11-crc\") pod \"576b2af7-7647-4a45-9423-3aa752d84e75\" (UID: \"576b2af7-7647-4a45-9423-3aa752d84e75\") " Dec 03 12:46:46 crc kubenswrapper[5052]: I1203 12:46:46.038867 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/576b2af7-7647-4a45-9423-3aa752d84e75-config-data-generated\") pod \"576b2af7-7647-4a45-9423-3aa752d84e75\" (UID: \"576b2af7-7647-4a45-9423-3aa752d84e75\") " Dec 03 12:46:46 crc kubenswrapper[5052]: I1203 12:46:46.038991 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/576b2af7-7647-4a45-9423-3aa752d84e75-operator-scripts\") pod \"576b2af7-7647-4a45-9423-3aa752d84e75\" (UID: \"576b2af7-7647-4a45-9423-3aa752d84e75\") " Dec 03 12:46:46 crc kubenswrapper[5052]: I1203 12:46:46.039021 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/576b2af7-7647-4a45-9423-3aa752d84e75-kolla-config\") pod \"576b2af7-7647-4a45-9423-3aa752d84e75\" (UID: \"576b2af7-7647-4a45-9423-3aa752d84e75\") " Dec 03 12:46:46 crc kubenswrapper[5052]: I1203 12:46:46.039044 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/576b2af7-7647-4a45-9423-3aa752d84e75-combined-ca-bundle\") pod \"576b2af7-7647-4a45-9423-3aa752d84e75\" (UID: \"576b2af7-7647-4a45-9423-3aa752d84e75\") " Dec 03 12:46:46 crc kubenswrapper[5052]: I1203 12:46:46.039073 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/576b2af7-7647-4a45-9423-3aa752d84e75-config-data-default\") pod \"576b2af7-7647-4a45-9423-3aa752d84e75\" (UID: \"576b2af7-7647-4a45-9423-3aa752d84e75\") " Dec 03 12:46:46 crc kubenswrapper[5052]: I1203 12:46:46.044093 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/576b2af7-7647-4a45-9423-3aa752d84e75-config-data-default" (OuterVolumeSpecName: "config-data-default") pod "576b2af7-7647-4a45-9423-3aa752d84e75" (UID: "576b2af7-7647-4a45-9423-3aa752d84e75"). InnerVolumeSpecName "config-data-default". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:46:46 crc kubenswrapper[5052]: I1203 12:46:46.044228 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/576b2af7-7647-4a45-9423-3aa752d84e75-config-data-generated" (OuterVolumeSpecName: "config-data-generated") pod "576b2af7-7647-4a45-9423-3aa752d84e75" (UID: "576b2af7-7647-4a45-9423-3aa752d84e75"). InnerVolumeSpecName "config-data-generated". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:46:46 crc kubenswrapper[5052]: I1203 12:46:46.044693 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/576b2af7-7647-4a45-9423-3aa752d84e75-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "576b2af7-7647-4a45-9423-3aa752d84e75" (UID: "576b2af7-7647-4a45-9423-3aa752d84e75"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:46:46 crc kubenswrapper[5052]: I1203 12:46:46.045178 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/576b2af7-7647-4a45-9423-3aa752d84e75-kolla-config" (OuterVolumeSpecName: "kolla-config") pod "576b2af7-7647-4a45-9423-3aa752d84e75" (UID: "576b2af7-7647-4a45-9423-3aa752d84e75"). InnerVolumeSpecName "kolla-config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:46:46 crc kubenswrapper[5052]: I1203 12:46:46.045886 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/576b2af7-7647-4a45-9423-3aa752d84e75-kube-api-access-x5ltc" (OuterVolumeSpecName: "kube-api-access-x5ltc") pod "576b2af7-7647-4a45-9423-3aa752d84e75" (UID: "576b2af7-7647-4a45-9423-3aa752d84e75"). InnerVolumeSpecName "kube-api-access-x5ltc". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:46:46 crc kubenswrapper[5052]: I1203 12:46:46.048358 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/576b2af7-7647-4a45-9423-3aa752d84e75-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "576b2af7-7647-4a45-9423-3aa752d84e75" (UID: "576b2af7-7647-4a45-9423-3aa752d84e75"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:46:46 crc kubenswrapper[5052]: I1203 12:46:46.048607 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage11-crc" (OuterVolumeSpecName: "mysql-db") pod "576b2af7-7647-4a45-9423-3aa752d84e75" (UID: "576b2af7-7647-4a45-9423-3aa752d84e75"). InnerVolumeSpecName "local-storage11-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 12:46:46 crc kubenswrapper[5052]: I1203 12:46:46.141537 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-x5ltc\" (UniqueName: \"kubernetes.io/projected/576b2af7-7647-4a45-9423-3aa752d84e75-kube-api-access-x5ltc\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:46 crc kubenswrapper[5052]: I1203 12:46:46.141587 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage11-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage11-crc\") on node \"crc\" " Dec 03 12:46:46 crc kubenswrapper[5052]: I1203 12:46:46.141599 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/576b2af7-7647-4a45-9423-3aa752d84e75-config-data-generated\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:46 crc kubenswrapper[5052]: I1203 12:46:46.141610 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/576b2af7-7647-4a45-9423-3aa752d84e75-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:46 crc kubenswrapper[5052]: I1203 12:46:46.141621 5052 reconciler_common.go:293] "Volume detached for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/576b2af7-7647-4a45-9423-3aa752d84e75-kolla-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:46 crc kubenswrapper[5052]: I1203 12:46:46.141631 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/576b2af7-7647-4a45-9423-3aa752d84e75-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:46 crc kubenswrapper[5052]: I1203 12:46:46.141640 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/576b2af7-7647-4a45-9423-3aa752d84e75-config-data-default\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:46 crc kubenswrapper[5052]: I1203 12:46:46.163703 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage11-crc" (UniqueName: "kubernetes.io/local-volume/local-storage11-crc") on node "crc" Dec 03 12:46:46 crc kubenswrapper[5052]: I1203 12:46:46.243745 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage11-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage11-crc\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:46 crc kubenswrapper[5052]: I1203 12:46:46.284043 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:46:46 crc kubenswrapper[5052]: I1203 12:46:46.337850 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:46:46 crc kubenswrapper[5052]: I1203 12:46:46.472608 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:46:46 crc kubenswrapper[5052]: I1203 12:46:46.857398 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-6d46dc454-rpjwz" event={"ID":"8fbc67c1-6c51-4c53-85c0-1cf3fa2d5fc2","Type":"ContainerStarted","Data":"5a171d88072795e803a5d56decdf74b34dfde4756856ab4c457fb64a1101b16f"} Dec 03 12:46:46 crc kubenswrapper[5052]: I1203 12:46:46.858375 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:46:46 crc kubenswrapper[5052]: I1203 12:46:46.858713 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/placement-6d46dc454-rpjwz" Dec 03 12:46:46 crc kubenswrapper[5052]: I1203 12:46:46.858774 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/placement-6d46dc454-rpjwz" Dec 03 12:46:46 crc kubenswrapper[5052]: I1203 12:46:46.889320 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/placement-6d46dc454-rpjwz" podStartSLOduration=2.88929581 podStartE2EDuration="2.88929581s" podCreationTimestamp="2025-12-03 12:46:44 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:46:46.886014236 +0000 UTC m=+5197.474256020" watchObservedRunningTime="2025-12-03 12:46:46.88929581 +0000 UTC m=+5197.477537574" Dec 03 12:46:46 crc kubenswrapper[5052]: I1203 12:46:46.929189 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/openstack-cell1-galera-0"] Dec 03 12:46:46 crc kubenswrapper[5052]: I1203 12:46:46.941204 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/openstack-cell1-galera-0"] Dec 03 12:46:46 crc kubenswrapper[5052]: I1203 12:46:46.965221 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/openstack-cell1-galera-0"] Dec 03 12:46:46 crc kubenswrapper[5052]: I1203 12:46:46.967243 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:46:46 crc kubenswrapper[5052]: I1203 12:46:46.974814 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"openstack-cell1-scripts" Dec 03 12:46:46 crc kubenswrapper[5052]: I1203 12:46:46.975182 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-galera-openstack-cell1-svc" Dec 03 12:46:46 crc kubenswrapper[5052]: I1203 12:46:46.991330 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/openstack-cell1-galera-0"] Dec 03 12:46:46 crc kubenswrapper[5052]: I1203 12:46:46.992096 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"openstack-cell1-config-data" Dec 03 12:46:46 crc kubenswrapper[5052]: I1203 12:46:46.992343 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"galera-openstack-cell1-dockercfg-7lfhj" Dec 03 12:46:47 crc kubenswrapper[5052]: I1203 12:46:47.063275 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/3479f9ad-871f-463e-96ce-50f4069088b3-kolla-config\") pod \"openstack-cell1-galera-0\" (UID: \"3479f9ad-871f-463e-96ce-50f4069088b3\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:46:47 crc kubenswrapper[5052]: I1203 12:46:47.063337 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cbqj6\" (UniqueName: \"kubernetes.io/projected/3479f9ad-871f-463e-96ce-50f4069088b3-kube-api-access-cbqj6\") pod \"openstack-cell1-galera-0\" (UID: \"3479f9ad-871f-463e-96ce-50f4069088b3\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:46:47 crc kubenswrapper[5052]: I1203 12:46:47.063403 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/3479f9ad-871f-463e-96ce-50f4069088b3-config-data-default\") pod \"openstack-cell1-galera-0\" (UID: \"3479f9ad-871f-463e-96ce-50f4069088b3\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:46:47 crc kubenswrapper[5052]: I1203 12:46:47.063432 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/3479f9ad-871f-463e-96ce-50f4069088b3-galera-tls-certs\") pod \"openstack-cell1-galera-0\" (UID: \"3479f9ad-871f-463e-96ce-50f4069088b3\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:46:47 crc kubenswrapper[5052]: I1203 12:46:47.063449 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage11-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage11-crc\") pod \"openstack-cell1-galera-0\" (UID: \"3479f9ad-871f-463e-96ce-50f4069088b3\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:46:47 crc kubenswrapper[5052]: I1203 12:46:47.063487 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3479f9ad-871f-463e-96ce-50f4069088b3-combined-ca-bundle\") pod \"openstack-cell1-galera-0\" (UID: \"3479f9ad-871f-463e-96ce-50f4069088b3\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:46:47 crc kubenswrapper[5052]: I1203 12:46:47.063521 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/3479f9ad-871f-463e-96ce-50f4069088b3-config-data-generated\") pod \"openstack-cell1-galera-0\" (UID: \"3479f9ad-871f-463e-96ce-50f4069088b3\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:46:47 crc kubenswrapper[5052]: I1203 12:46:47.064105 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/3479f9ad-871f-463e-96ce-50f4069088b3-operator-scripts\") pod \"openstack-cell1-galera-0\" (UID: \"3479f9ad-871f-463e-96ce-50f4069088b3\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:46:47 crc kubenswrapper[5052]: I1203 12:46:47.064196 5052 reconciler_common.go:293] "Volume detached for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/576b2af7-7647-4a45-9423-3aa752d84e75-galera-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:47 crc kubenswrapper[5052]: I1203 12:46:47.165689 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/3479f9ad-871f-463e-96ce-50f4069088b3-kolla-config\") pod \"openstack-cell1-galera-0\" (UID: \"3479f9ad-871f-463e-96ce-50f4069088b3\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:46:47 crc kubenswrapper[5052]: I1203 12:46:47.165787 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-cbqj6\" (UniqueName: \"kubernetes.io/projected/3479f9ad-871f-463e-96ce-50f4069088b3-kube-api-access-cbqj6\") pod \"openstack-cell1-galera-0\" (UID: \"3479f9ad-871f-463e-96ce-50f4069088b3\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:46:47 crc kubenswrapper[5052]: I1203 12:46:47.165863 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/3479f9ad-871f-463e-96ce-50f4069088b3-config-data-default\") pod \"openstack-cell1-galera-0\" (UID: \"3479f9ad-871f-463e-96ce-50f4069088b3\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:46:47 crc kubenswrapper[5052]: I1203 12:46:47.165901 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/3479f9ad-871f-463e-96ce-50f4069088b3-galera-tls-certs\") pod \"openstack-cell1-galera-0\" (UID: \"3479f9ad-871f-463e-96ce-50f4069088b3\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:46:47 crc kubenswrapper[5052]: I1203 12:46:47.165930 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage11-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage11-crc\") pod \"openstack-cell1-galera-0\" (UID: \"3479f9ad-871f-463e-96ce-50f4069088b3\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:46:47 crc kubenswrapper[5052]: I1203 12:46:47.165978 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3479f9ad-871f-463e-96ce-50f4069088b3-combined-ca-bundle\") pod \"openstack-cell1-galera-0\" (UID: \"3479f9ad-871f-463e-96ce-50f4069088b3\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:46:47 crc kubenswrapper[5052]: I1203 12:46:47.166005 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/3479f9ad-871f-463e-96ce-50f4069088b3-config-data-generated\") pod \"openstack-cell1-galera-0\" (UID: \"3479f9ad-871f-463e-96ce-50f4069088b3\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:46:47 crc kubenswrapper[5052]: I1203 12:46:47.166041 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/3479f9ad-871f-463e-96ce-50f4069088b3-operator-scripts\") pod \"openstack-cell1-galera-0\" (UID: \"3479f9ad-871f-463e-96ce-50f4069088b3\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:46:47 crc kubenswrapper[5052]: I1203 12:46:47.167503 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage11-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage11-crc\") pod \"openstack-cell1-galera-0\" (UID: \"3479f9ad-871f-463e-96ce-50f4069088b3\") device mount path \"/mnt/openstack/pv11\"" pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:46:47 crc kubenswrapper[5052]: I1203 12:46:47.168334 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/3479f9ad-871f-463e-96ce-50f4069088b3-kolla-config\") pod \"openstack-cell1-galera-0\" (UID: \"3479f9ad-871f-463e-96ce-50f4069088b3\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:46:47 crc kubenswrapper[5052]: I1203 12:46:47.168535 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/3479f9ad-871f-463e-96ce-50f4069088b3-config-data-generated\") pod \"openstack-cell1-galera-0\" (UID: \"3479f9ad-871f-463e-96ce-50f4069088b3\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:46:47 crc kubenswrapper[5052]: I1203 12:46:47.169129 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/3479f9ad-871f-463e-96ce-50f4069088b3-config-data-default\") pod \"openstack-cell1-galera-0\" (UID: \"3479f9ad-871f-463e-96ce-50f4069088b3\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:46:47 crc kubenswrapper[5052]: I1203 12:46:47.169375 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/3479f9ad-871f-463e-96ce-50f4069088b3-operator-scripts\") pod \"openstack-cell1-galera-0\" (UID: \"3479f9ad-871f-463e-96ce-50f4069088b3\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:46:47 crc kubenswrapper[5052]: I1203 12:46:47.172257 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/3479f9ad-871f-463e-96ce-50f4069088b3-galera-tls-certs\") pod \"openstack-cell1-galera-0\" (UID: \"3479f9ad-871f-463e-96ce-50f4069088b3\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:46:47 crc kubenswrapper[5052]: I1203 12:46:47.187313 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3479f9ad-871f-463e-96ce-50f4069088b3-combined-ca-bundle\") pod \"openstack-cell1-galera-0\" (UID: \"3479f9ad-871f-463e-96ce-50f4069088b3\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:46:47 crc kubenswrapper[5052]: I1203 12:46:47.195097 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-cbqj6\" (UniqueName: \"kubernetes.io/projected/3479f9ad-871f-463e-96ce-50f4069088b3-kube-api-access-cbqj6\") pod \"openstack-cell1-galera-0\" (UID: \"3479f9ad-871f-463e-96ce-50f4069088b3\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:46:47 crc kubenswrapper[5052]: I1203 12:46:47.200658 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage11-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage11-crc\") pod \"openstack-cell1-galera-0\" (UID: \"3479f9ad-871f-463e-96ce-50f4069088b3\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:46:47 crc kubenswrapper[5052]: I1203 12:46:47.290448 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:46:47 crc kubenswrapper[5052]: I1203 12:46:47.389099 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:46:47 crc kubenswrapper[5052]: I1203 12:46:47.521792 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:46:47 crc kubenswrapper[5052]: I1203 12:46:47.687126 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="576b2af7-7647-4a45-9423-3aa752d84e75" path="/var/lib/kubelet/pods/576b2af7-7647-4a45-9423-3aa752d84e75/volumes" Dec 03 12:46:47 crc kubenswrapper[5052]: I1203 12:46:47.782477 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/openstack-cell1-galera-0"] Dec 03 12:46:47 crc kubenswrapper[5052]: I1203 12:46:47.875820 5052 generic.go:334] "Generic (PLEG): container finished" podID="81709983-1fd0-4446-bbbf-fb94653bb3d2" containerID="c397988625a9ab9075d72d6f8c51dd95b36e8b2a4214beb4c4dcf9e2ea912582" exitCode=1 Dec 03 12:46:47 crc kubenswrapper[5052]: I1203 12:46:47.876159 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"81709983-1fd0-4446-bbbf-fb94653bb3d2","Type":"ContainerDied","Data":"c397988625a9ab9075d72d6f8c51dd95b36e8b2a4214beb4c4dcf9e2ea912582"} Dec 03 12:46:47 crc kubenswrapper[5052]: I1203 12:46:47.876551 5052 scope.go:117] "RemoveContainer" containerID="26ed2e6897ff71a51086a7cc455488c21ad8adffd038be8de42d13bd1335a6fd" Dec 03 12:46:47 crc kubenswrapper[5052]: I1203 12:46:47.877236 5052 scope.go:117] "RemoveContainer" containerID="c397988625a9ab9075d72d6f8c51dd95b36e8b2a4214beb4c4dcf9e2ea912582" Dec 03 12:46:47 crc kubenswrapper[5052]: E1203 12:46:47.877500 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"nova-scheduler-scheduler\" with CrashLoopBackOff: \"back-off 10s restarting failed container=nova-scheduler-scheduler pod=nova-scheduler-0_openstack-kuttl-tests(81709983-1fd0-4446-bbbf-fb94653bb3d2)\"" pod="openstack-kuttl-tests/nova-scheduler-0" podUID="81709983-1fd0-4446-bbbf-fb94653bb3d2" Dec 03 12:46:47 crc kubenswrapper[5052]: I1203 12:46:47.879482 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-cell1-galera-0" event={"ID":"3479f9ad-871f-463e-96ce-50f4069088b3","Type":"ContainerStarted","Data":"e463002f432d54290e18bcb85de7ca23165a1437ec67a5d132c0921026ce9ec1"} Dec 03 12:46:47 crc kubenswrapper[5052]: I1203 12:46:47.904891 5052 generic.go:334] "Generic (PLEG): container finished" podID="6d653682-52e2-4940-af66-633475bfe09a" containerID="4c1d25fd5e7f80502770dbe44896eb5a3327b1bebde080de0e66dead94625111" exitCode=1 Dec 03 12:46:47 crc kubenswrapper[5052]: I1203 12:46:47.906205 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-conductor-0" event={"ID":"6d653682-52e2-4940-af66-633475bfe09a","Type":"ContainerDied","Data":"4c1d25fd5e7f80502770dbe44896eb5a3327b1bebde080de0e66dead94625111"} Dec 03 12:46:47 crc kubenswrapper[5052]: I1203 12:46:47.906717 5052 scope.go:117] "RemoveContainer" containerID="4c1d25fd5e7f80502770dbe44896eb5a3327b1bebde080de0e66dead94625111" Dec 03 12:46:47 crc kubenswrapper[5052]: E1203 12:46:47.907037 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"nova-cell0-conductor-conductor\" with CrashLoopBackOff: \"back-off 20s restarting failed container=nova-cell0-conductor-conductor pod=nova-cell0-conductor-0_openstack-kuttl-tests(6d653682-52e2-4940-af66-633475bfe09a)\"" pod="openstack-kuttl-tests/nova-cell0-conductor-0" podUID="6d653682-52e2-4940-af66-633475bfe09a" Dec 03 12:46:48 crc kubenswrapper[5052]: I1203 12:46:48.029581 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:46:48 crc kubenswrapper[5052]: I1203 12:46:48.131946 5052 scope.go:117] "RemoveContainer" containerID="965af26b70386d0042f76a35140b047b193a7f695cf03cf43e91184f27f9aaa1" Dec 03 12:46:48 crc kubenswrapper[5052]: I1203 12:46:48.527149 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:46:48 crc kubenswrapper[5052]: I1203 12:46:48.527474 5052 kubelet.go:2542] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:46:48 crc kubenswrapper[5052]: I1203 12:46:48.527488 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:46:48 crc kubenswrapper[5052]: I1203 12:46:48.527497 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:46:48 crc kubenswrapper[5052]: I1203 12:46:48.685400 5052 kubelet.go:2542] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:46:48 crc kubenswrapper[5052]: I1203 12:46:48.956590 5052 generic.go:334] "Generic (PLEG): container finished" podID="3479f9ad-871f-463e-96ce-50f4069088b3" containerID="f8b9d65f292776aa251d63e395ac7e5031feb16b30ea525e3b1072e08b062178" exitCode=0 Dec 03 12:46:48 crc kubenswrapper[5052]: I1203 12:46:48.956943 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-cell1-galera-0" event={"ID":"3479f9ad-871f-463e-96ce-50f4069088b3","Type":"ContainerDied","Data":"f8b9d65f292776aa251d63e395ac7e5031feb16b30ea525e3b1072e08b062178"} Dec 03 12:46:48 crc kubenswrapper[5052]: I1203 12:46:48.961443 5052 scope.go:117] "RemoveContainer" containerID="c397988625a9ab9075d72d6f8c51dd95b36e8b2a4214beb4c4dcf9e2ea912582" Dec 03 12:46:48 crc kubenswrapper[5052]: E1203 12:46:48.961756 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"nova-scheduler-scheduler\" with CrashLoopBackOff: \"back-off 10s restarting failed container=nova-scheduler-scheduler pod=nova-scheduler-0_openstack-kuttl-tests(81709983-1fd0-4446-bbbf-fb94653bb3d2)\"" pod="openstack-kuttl-tests/nova-scheduler-0" podUID="81709983-1fd0-4446-bbbf-fb94653bb3d2" Dec 03 12:46:48 crc kubenswrapper[5052]: I1203 12:46:48.968991 5052 generic.go:334] "Generic (PLEG): container finished" podID="8669832d-c47a-49bd-9491-dc7141ac6281" containerID="d845f1b1ea8802fd6fd4c7f6098f521746fa875f2f1e1c94973d2bf864333d42" exitCode=1 Dec 03 12:46:48 crc kubenswrapper[5052]: I1203 12:46:48.969285 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-conductor-0" event={"ID":"8669832d-c47a-49bd-9491-dc7141ac6281","Type":"ContainerDied","Data":"d845f1b1ea8802fd6fd4c7f6098f521746fa875f2f1e1c94973d2bf864333d42"} Dec 03 12:46:48 crc kubenswrapper[5052]: I1203 12:46:48.974339 5052 scope.go:117] "RemoveContainer" containerID="d845f1b1ea8802fd6fd4c7f6098f521746fa875f2f1e1c94973d2bf864333d42" Dec 03 12:46:48 crc kubenswrapper[5052]: I1203 12:46:48.978200 5052 scope.go:117] "RemoveContainer" containerID="4c1d25fd5e7f80502770dbe44896eb5a3327b1bebde080de0e66dead94625111" Dec 03 12:46:48 crc kubenswrapper[5052]: E1203 12:46:48.978375 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"nova-cell0-conductor-conductor\" with CrashLoopBackOff: \"back-off 20s restarting failed container=nova-cell0-conductor-conductor pod=nova-cell0-conductor-0_openstack-kuttl-tests(6d653682-52e2-4940-af66-633475bfe09a)\"" pod="openstack-kuttl-tests/nova-cell0-conductor-0" podUID="6d653682-52e2-4940-af66-633475bfe09a" Dec 03 12:46:49 crc kubenswrapper[5052]: I1203 12:46:49.014579 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-api-bcf49b784-jnkdb"] Dec 03 12:46:49 crc kubenswrapper[5052]: E1203 12:46:49.015485 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[internal-tls-certs], unattached volumes=[], failed to process volumes=[]: context canceled" pod="openstack-kuttl-tests/barbican-api-bcf49b784-jnkdb" podUID="c069aee2-4cb6-4c8a-bbe5-949ab0b48331" Dec 03 12:46:49 crc kubenswrapper[5052]: I1203 12:46:49.072902 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/barbican-api-6cd888f578-nv6fh"] Dec 03 12:46:49 crc kubenswrapper[5052]: I1203 12:46:49.074637 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-api-6cd888f578-nv6fh" Dec 03 12:46:49 crc kubenswrapper[5052]: I1203 12:46:49.084845 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-api-6cd888f578-nv6fh"] Dec 03 12:46:49 crc kubenswrapper[5052]: I1203 12:46:49.246953 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/1f026d39-713b-4611-80fd-9577e697d0c4-config-data\") pod \"barbican-api-6cd888f578-nv6fh\" (UID: \"1f026d39-713b-4611-80fd-9577e697d0c4\") " pod="openstack-kuttl-tests/barbican-api-6cd888f578-nv6fh" Dec 03 12:46:49 crc kubenswrapper[5052]: I1203 12:46:49.246991 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6hl9q\" (UniqueName: \"kubernetes.io/projected/1f026d39-713b-4611-80fd-9577e697d0c4-kube-api-access-6hl9q\") pod \"barbican-api-6cd888f578-nv6fh\" (UID: \"1f026d39-713b-4611-80fd-9577e697d0c4\") " pod="openstack-kuttl-tests/barbican-api-6cd888f578-nv6fh" Dec 03 12:46:49 crc kubenswrapper[5052]: I1203 12:46:49.247072 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1f026d39-713b-4611-80fd-9577e697d0c4-combined-ca-bundle\") pod \"barbican-api-6cd888f578-nv6fh\" (UID: \"1f026d39-713b-4611-80fd-9577e697d0c4\") " pod="openstack-kuttl-tests/barbican-api-6cd888f578-nv6fh" Dec 03 12:46:49 crc kubenswrapper[5052]: I1203 12:46:49.247094 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/1f026d39-713b-4611-80fd-9577e697d0c4-logs\") pod \"barbican-api-6cd888f578-nv6fh\" (UID: \"1f026d39-713b-4611-80fd-9577e697d0c4\") " pod="openstack-kuttl-tests/barbican-api-6cd888f578-nv6fh" Dec 03 12:46:49 crc kubenswrapper[5052]: I1203 12:46:49.247238 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/1f026d39-713b-4611-80fd-9577e697d0c4-config-data-custom\") pod \"barbican-api-6cd888f578-nv6fh\" (UID: \"1f026d39-713b-4611-80fd-9577e697d0c4\") " pod="openstack-kuttl-tests/barbican-api-6cd888f578-nv6fh" Dec 03 12:46:49 crc kubenswrapper[5052]: I1203 12:46:49.247297 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/1f026d39-713b-4611-80fd-9577e697d0c4-public-tls-certs\") pod \"barbican-api-6cd888f578-nv6fh\" (UID: \"1f026d39-713b-4611-80fd-9577e697d0c4\") " pod="openstack-kuttl-tests/barbican-api-6cd888f578-nv6fh" Dec 03 12:46:49 crc kubenswrapper[5052]: I1203 12:46:49.247315 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/1f026d39-713b-4611-80fd-9577e697d0c4-internal-tls-certs\") pod \"barbican-api-6cd888f578-nv6fh\" (UID: \"1f026d39-713b-4611-80fd-9577e697d0c4\") " pod="openstack-kuttl-tests/barbican-api-6cd888f578-nv6fh" Dec 03 12:46:49 crc kubenswrapper[5052]: I1203 12:46:49.349582 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/1f026d39-713b-4611-80fd-9577e697d0c4-config-data-custom\") pod \"barbican-api-6cd888f578-nv6fh\" (UID: \"1f026d39-713b-4611-80fd-9577e697d0c4\") " pod="openstack-kuttl-tests/barbican-api-6cd888f578-nv6fh" Dec 03 12:46:49 crc kubenswrapper[5052]: I1203 12:46:49.350028 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/1f026d39-713b-4611-80fd-9577e697d0c4-public-tls-certs\") pod \"barbican-api-6cd888f578-nv6fh\" (UID: \"1f026d39-713b-4611-80fd-9577e697d0c4\") " pod="openstack-kuttl-tests/barbican-api-6cd888f578-nv6fh" Dec 03 12:46:49 crc kubenswrapper[5052]: I1203 12:46:49.350061 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/1f026d39-713b-4611-80fd-9577e697d0c4-internal-tls-certs\") pod \"barbican-api-6cd888f578-nv6fh\" (UID: \"1f026d39-713b-4611-80fd-9577e697d0c4\") " pod="openstack-kuttl-tests/barbican-api-6cd888f578-nv6fh" Dec 03 12:46:49 crc kubenswrapper[5052]: I1203 12:46:49.350178 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/1f026d39-713b-4611-80fd-9577e697d0c4-config-data\") pod \"barbican-api-6cd888f578-nv6fh\" (UID: \"1f026d39-713b-4611-80fd-9577e697d0c4\") " pod="openstack-kuttl-tests/barbican-api-6cd888f578-nv6fh" Dec 03 12:46:49 crc kubenswrapper[5052]: I1203 12:46:49.350206 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-6hl9q\" (UniqueName: \"kubernetes.io/projected/1f026d39-713b-4611-80fd-9577e697d0c4-kube-api-access-6hl9q\") pod \"barbican-api-6cd888f578-nv6fh\" (UID: \"1f026d39-713b-4611-80fd-9577e697d0c4\") " pod="openstack-kuttl-tests/barbican-api-6cd888f578-nv6fh" Dec 03 12:46:49 crc kubenswrapper[5052]: I1203 12:46:49.350268 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1f026d39-713b-4611-80fd-9577e697d0c4-combined-ca-bundle\") pod \"barbican-api-6cd888f578-nv6fh\" (UID: \"1f026d39-713b-4611-80fd-9577e697d0c4\") " pod="openstack-kuttl-tests/barbican-api-6cd888f578-nv6fh" Dec 03 12:46:49 crc kubenswrapper[5052]: I1203 12:46:49.350301 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/1f026d39-713b-4611-80fd-9577e697d0c4-logs\") pod \"barbican-api-6cd888f578-nv6fh\" (UID: \"1f026d39-713b-4611-80fd-9577e697d0c4\") " pod="openstack-kuttl-tests/barbican-api-6cd888f578-nv6fh" Dec 03 12:46:49 crc kubenswrapper[5052]: I1203 12:46:49.350993 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/1f026d39-713b-4611-80fd-9577e697d0c4-logs\") pod \"barbican-api-6cd888f578-nv6fh\" (UID: \"1f026d39-713b-4611-80fd-9577e697d0c4\") " pod="openstack-kuttl-tests/barbican-api-6cd888f578-nv6fh" Dec 03 12:46:49 crc kubenswrapper[5052]: I1203 12:46:49.356250 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/1f026d39-713b-4611-80fd-9577e697d0c4-public-tls-certs\") pod \"barbican-api-6cd888f578-nv6fh\" (UID: \"1f026d39-713b-4611-80fd-9577e697d0c4\") " pod="openstack-kuttl-tests/barbican-api-6cd888f578-nv6fh" Dec 03 12:46:49 crc kubenswrapper[5052]: I1203 12:46:49.356581 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1f026d39-713b-4611-80fd-9577e697d0c4-combined-ca-bundle\") pod \"barbican-api-6cd888f578-nv6fh\" (UID: \"1f026d39-713b-4611-80fd-9577e697d0c4\") " pod="openstack-kuttl-tests/barbican-api-6cd888f578-nv6fh" Dec 03 12:46:49 crc kubenswrapper[5052]: I1203 12:46:49.357159 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/1f026d39-713b-4611-80fd-9577e697d0c4-config-data-custom\") pod \"barbican-api-6cd888f578-nv6fh\" (UID: \"1f026d39-713b-4611-80fd-9577e697d0c4\") " pod="openstack-kuttl-tests/barbican-api-6cd888f578-nv6fh" Dec 03 12:46:49 crc kubenswrapper[5052]: I1203 12:46:49.357914 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/1f026d39-713b-4611-80fd-9577e697d0c4-internal-tls-certs\") pod \"barbican-api-6cd888f578-nv6fh\" (UID: \"1f026d39-713b-4611-80fd-9577e697d0c4\") " pod="openstack-kuttl-tests/barbican-api-6cd888f578-nv6fh" Dec 03 12:46:49 crc kubenswrapper[5052]: I1203 12:46:49.361405 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/1f026d39-713b-4611-80fd-9577e697d0c4-config-data\") pod \"barbican-api-6cd888f578-nv6fh\" (UID: \"1f026d39-713b-4611-80fd-9577e697d0c4\") " pod="openstack-kuttl-tests/barbican-api-6cd888f578-nv6fh" Dec 03 12:46:49 crc kubenswrapper[5052]: I1203 12:46:49.388466 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-6hl9q\" (UniqueName: \"kubernetes.io/projected/1f026d39-713b-4611-80fd-9577e697d0c4-kube-api-access-6hl9q\") pod \"barbican-api-6cd888f578-nv6fh\" (UID: \"1f026d39-713b-4611-80fd-9577e697d0c4\") " pod="openstack-kuttl-tests/barbican-api-6cd888f578-nv6fh" Dec 03 12:46:49 crc kubenswrapper[5052]: I1203 12:46:49.472589 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-bbcfbbc9d-5q6gx" Dec 03 12:46:49 crc kubenswrapper[5052]: I1203 12:46:49.554951 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xwlbj\" (UniqueName: \"kubernetes.io/projected/32f420c2-aa7e-44c8-a63a-7654aa3e6e34-kube-api-access-xwlbj\") pod \"32f420c2-aa7e-44c8-a63a-7654aa3e6e34\" (UID: \"32f420c2-aa7e-44c8-a63a-7654aa3e6e34\") " Dec 03 12:46:49 crc kubenswrapper[5052]: I1203 12:46:49.555342 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/32f420c2-aa7e-44c8-a63a-7654aa3e6e34-config-data\") pod \"32f420c2-aa7e-44c8-a63a-7654aa3e6e34\" (UID: \"32f420c2-aa7e-44c8-a63a-7654aa3e6e34\") " Dec 03 12:46:49 crc kubenswrapper[5052]: I1203 12:46:49.555400 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/32f420c2-aa7e-44c8-a63a-7654aa3e6e34-internal-tls-certs\") pod \"32f420c2-aa7e-44c8-a63a-7654aa3e6e34\" (UID: \"32f420c2-aa7e-44c8-a63a-7654aa3e6e34\") " Dec 03 12:46:49 crc kubenswrapper[5052]: I1203 12:46:49.555441 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/32f420c2-aa7e-44c8-a63a-7654aa3e6e34-public-tls-certs\") pod \"32f420c2-aa7e-44c8-a63a-7654aa3e6e34\" (UID: \"32f420c2-aa7e-44c8-a63a-7654aa3e6e34\") " Dec 03 12:46:49 crc kubenswrapper[5052]: I1203 12:46:49.555571 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/32f420c2-aa7e-44c8-a63a-7654aa3e6e34-combined-ca-bundle\") pod \"32f420c2-aa7e-44c8-a63a-7654aa3e6e34\" (UID: \"32f420c2-aa7e-44c8-a63a-7654aa3e6e34\") " Dec 03 12:46:49 crc kubenswrapper[5052]: I1203 12:46:49.555657 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/32f420c2-aa7e-44c8-a63a-7654aa3e6e34-scripts\") pod \"32f420c2-aa7e-44c8-a63a-7654aa3e6e34\" (UID: \"32f420c2-aa7e-44c8-a63a-7654aa3e6e34\") " Dec 03 12:46:49 crc kubenswrapper[5052]: I1203 12:46:49.555709 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/32f420c2-aa7e-44c8-a63a-7654aa3e6e34-logs\") pod \"32f420c2-aa7e-44c8-a63a-7654aa3e6e34\" (UID: \"32f420c2-aa7e-44c8-a63a-7654aa3e6e34\") " Dec 03 12:46:49 crc kubenswrapper[5052]: I1203 12:46:49.556824 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/32f420c2-aa7e-44c8-a63a-7654aa3e6e34-logs" (OuterVolumeSpecName: "logs") pod "32f420c2-aa7e-44c8-a63a-7654aa3e6e34" (UID: "32f420c2-aa7e-44c8-a63a-7654aa3e6e34"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:46:49 crc kubenswrapper[5052]: I1203 12:46:49.560908 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/32f420c2-aa7e-44c8-a63a-7654aa3e6e34-kube-api-access-xwlbj" (OuterVolumeSpecName: "kube-api-access-xwlbj") pod "32f420c2-aa7e-44c8-a63a-7654aa3e6e34" (UID: "32f420c2-aa7e-44c8-a63a-7654aa3e6e34"). InnerVolumeSpecName "kube-api-access-xwlbj". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:46:49 crc kubenswrapper[5052]: I1203 12:46:49.565638 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-api-6cd888f578-nv6fh" Dec 03 12:46:49 crc kubenswrapper[5052]: I1203 12:46:49.569202 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/32f420c2-aa7e-44c8-a63a-7654aa3e6e34-scripts" (OuterVolumeSpecName: "scripts") pod "32f420c2-aa7e-44c8-a63a-7654aa3e6e34" (UID: "32f420c2-aa7e-44c8-a63a-7654aa3e6e34"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:46:49 crc kubenswrapper[5052]: I1203 12:46:49.623472 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/32f420c2-aa7e-44c8-a63a-7654aa3e6e34-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "32f420c2-aa7e-44c8-a63a-7654aa3e6e34" (UID: "32f420c2-aa7e-44c8-a63a-7654aa3e6e34"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:46:49 crc kubenswrapper[5052]: I1203 12:46:49.625703 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/32f420c2-aa7e-44c8-a63a-7654aa3e6e34-config-data" (OuterVolumeSpecName: "config-data") pod "32f420c2-aa7e-44c8-a63a-7654aa3e6e34" (UID: "32f420c2-aa7e-44c8-a63a-7654aa3e6e34"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:46:49 crc kubenswrapper[5052]: I1203 12:46:49.658257 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/32f420c2-aa7e-44c8-a63a-7654aa3e6e34-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:49 crc kubenswrapper[5052]: I1203 12:46:49.658300 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/32f420c2-aa7e-44c8-a63a-7654aa3e6e34-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:49 crc kubenswrapper[5052]: I1203 12:46:49.658315 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-xwlbj\" (UniqueName: \"kubernetes.io/projected/32f420c2-aa7e-44c8-a63a-7654aa3e6e34-kube-api-access-xwlbj\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:49 crc kubenswrapper[5052]: I1203 12:46:49.658329 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/32f420c2-aa7e-44c8-a63a-7654aa3e6e34-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:49 crc kubenswrapper[5052]: I1203 12:46:49.658342 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/32f420c2-aa7e-44c8-a63a-7654aa3e6e34-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:49 crc kubenswrapper[5052]: I1203 12:46:49.694623 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/32f420c2-aa7e-44c8-a63a-7654aa3e6e34-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "32f420c2-aa7e-44c8-a63a-7654aa3e6e34" (UID: "32f420c2-aa7e-44c8-a63a-7654aa3e6e34"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:46:49 crc kubenswrapper[5052]: I1203 12:46:49.706407 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/32f420c2-aa7e-44c8-a63a-7654aa3e6e34-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "32f420c2-aa7e-44c8-a63a-7654aa3e6e34" (UID: "32f420c2-aa7e-44c8-a63a-7654aa3e6e34"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:46:49 crc kubenswrapper[5052]: I1203 12:46:49.762043 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/32f420c2-aa7e-44c8-a63a-7654aa3e6e34-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:49 crc kubenswrapper[5052]: I1203 12:46:49.762081 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/32f420c2-aa7e-44c8-a63a-7654aa3e6e34-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:49 crc kubenswrapper[5052]: I1203 12:46:49.991238 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_neutron-bcd9f4f4-djpt8_049a231b-3a12-4710-9b7a-117b1bb82e72/neutron-api/0.log" Dec 03 12:46:49 crc kubenswrapper[5052]: I1203 12:46:49.991578 5052 generic.go:334] "Generic (PLEG): container finished" podID="049a231b-3a12-4710-9b7a-117b1bb82e72" containerID="4b6385cf4545e25cb919fcac9e45a38d9979e76dac85ed14f6bd36d5c92843a5" exitCode=1 Dec 03 12:46:49 crc kubenswrapper[5052]: I1203 12:46:49.991616 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-bcd9f4f4-djpt8" event={"ID":"049a231b-3a12-4710-9b7a-117b1bb82e72","Type":"ContainerDied","Data":"4b6385cf4545e25cb919fcac9e45a38d9979e76dac85ed14f6bd36d5c92843a5"} Dec 03 12:46:49 crc kubenswrapper[5052]: I1203 12:46:49.992612 5052 scope.go:117] "RemoveContainer" containerID="4b6385cf4545e25cb919fcac9e45a38d9979e76dac85ed14f6bd36d5c92843a5" Dec 03 12:46:49 crc kubenswrapper[5052]: I1203 12:46:49.993610 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-conductor-0" event={"ID":"8669832d-c47a-49bd-9491-dc7141ac6281","Type":"ContainerStarted","Data":"e102925eb6bfd868ed0b7dff92ded1a29e84b2579d3092b133af1f0bf476d04c"} Dec 03 12:46:49 crc kubenswrapper[5052]: I1203 12:46:49.993823 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:46:49 crc kubenswrapper[5052]: I1203 12:46:49.996282 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-cell1-galera-0" event={"ID":"3479f9ad-871f-463e-96ce-50f4069088b3","Type":"ContainerStarted","Data":"bf37583e842ac2986eebf906db0e42704630e142c5ffc66b3b257042bb9c2bf5"} Dec 03 12:46:49 crc kubenswrapper[5052]: I1203 12:46:49.998421 5052 generic.go:334] "Generic (PLEG): container finished" podID="32f420c2-aa7e-44c8-a63a-7654aa3e6e34" containerID="353ad09a2eae7f22cab9c43ca6063becb9fb6d4010c1a66dcef0e6673635198d" exitCode=0 Dec 03 12:46:49 crc kubenswrapper[5052]: I1203 12:46:49.998500 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-api-bcf49b784-jnkdb" Dec 03 12:46:49 crc kubenswrapper[5052]: I1203 12:46:49.998537 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-bbcfbbc9d-5q6gx" Dec 03 12:46:49 crc kubenswrapper[5052]: I1203 12:46:49.998567 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-bbcfbbc9d-5q6gx" event={"ID":"32f420c2-aa7e-44c8-a63a-7654aa3e6e34","Type":"ContainerDied","Data":"353ad09a2eae7f22cab9c43ca6063becb9fb6d4010c1a66dcef0e6673635198d"} Dec 03 12:46:49 crc kubenswrapper[5052]: I1203 12:46:49.998585 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-bbcfbbc9d-5q6gx" event={"ID":"32f420c2-aa7e-44c8-a63a-7654aa3e6e34","Type":"ContainerDied","Data":"67cbac03f1a3302f889c26050cb5fda9a887534e786826a06099e9737b5f209c"} Dec 03 12:46:49 crc kubenswrapper[5052]: I1203 12:46:49.998600 5052 scope.go:117] "RemoveContainer" containerID="353ad09a2eae7f22cab9c43ca6063becb9fb6d4010c1a66dcef0e6673635198d" Dec 03 12:46:50 crc kubenswrapper[5052]: I1203 12:46:50.011014 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/neutron-bcd9f4f4-djpt8" podUID="049a231b-3a12-4710-9b7a-117b1bb82e72" containerName="neutron-httpd" probeResult="failure" output="HTTP probe failed with statuscode: 503" Dec 03 12:46:50 crc kubenswrapper[5052]: I1203 12:46:50.012548 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-api-bcf49b784-jnkdb" Dec 03 12:46:50 crc kubenswrapper[5052]: I1203 12:46:50.055372 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/openstack-cell1-galera-0" podStartSLOduration=4.055353483 podStartE2EDuration="4.055353483s" podCreationTimestamp="2025-12-03 12:46:46 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:46:50.053274634 +0000 UTC m=+5200.641516398" watchObservedRunningTime="2025-12-03 12:46:50.055353483 +0000 UTC m=+5200.643595247" Dec 03 12:46:50 crc kubenswrapper[5052]: I1203 12:46:50.069452 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c069aee2-4cb6-4c8a-bbe5-949ab0b48331-combined-ca-bundle\") pod \"c069aee2-4cb6-4c8a-bbe5-949ab0b48331\" (UID: \"c069aee2-4cb6-4c8a-bbe5-949ab0b48331\") " Dec 03 12:46:50 crc kubenswrapper[5052]: I1203 12:46:50.069609 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/c069aee2-4cb6-4c8a-bbe5-949ab0b48331-public-tls-certs\") pod \"c069aee2-4cb6-4c8a-bbe5-949ab0b48331\" (UID: \"c069aee2-4cb6-4c8a-bbe5-949ab0b48331\") " Dec 03 12:46:50 crc kubenswrapper[5052]: I1203 12:46:50.069716 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/c069aee2-4cb6-4c8a-bbe5-949ab0b48331-config-data-custom\") pod \"c069aee2-4cb6-4c8a-bbe5-949ab0b48331\" (UID: \"c069aee2-4cb6-4c8a-bbe5-949ab0b48331\") " Dec 03 12:46:50 crc kubenswrapper[5052]: I1203 12:46:50.069805 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/c069aee2-4cb6-4c8a-bbe5-949ab0b48331-logs\") pod \"c069aee2-4cb6-4c8a-bbe5-949ab0b48331\" (UID: \"c069aee2-4cb6-4c8a-bbe5-949ab0b48331\") " Dec 03 12:46:50 crc kubenswrapper[5052]: I1203 12:46:50.069912 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/c069aee2-4cb6-4c8a-bbe5-949ab0b48331-config-data\") pod \"c069aee2-4cb6-4c8a-bbe5-949ab0b48331\" (UID: \"c069aee2-4cb6-4c8a-bbe5-949ab0b48331\") " Dec 03 12:46:50 crc kubenswrapper[5052]: I1203 12:46:50.069947 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q9q5k\" (UniqueName: \"kubernetes.io/projected/c069aee2-4cb6-4c8a-bbe5-949ab0b48331-kube-api-access-q9q5k\") pod \"c069aee2-4cb6-4c8a-bbe5-949ab0b48331\" (UID: \"c069aee2-4cb6-4c8a-bbe5-949ab0b48331\") " Dec 03 12:46:50 crc kubenswrapper[5052]: I1203 12:46:50.072283 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c069aee2-4cb6-4c8a-bbe5-949ab0b48331-logs" (OuterVolumeSpecName: "logs") pod "c069aee2-4cb6-4c8a-bbe5-949ab0b48331" (UID: "c069aee2-4cb6-4c8a-bbe5-949ab0b48331"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:46:50 crc kubenswrapper[5052]: I1203 12:46:50.081223 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c069aee2-4cb6-4c8a-bbe5-949ab0b48331-config-data-custom" (OuterVolumeSpecName: "config-data-custom") pod "c069aee2-4cb6-4c8a-bbe5-949ab0b48331" (UID: "c069aee2-4cb6-4c8a-bbe5-949ab0b48331"). InnerVolumeSpecName "config-data-custom". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:46:50 crc kubenswrapper[5052]: I1203 12:46:50.081394 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c069aee2-4cb6-4c8a-bbe5-949ab0b48331-kube-api-access-q9q5k" (OuterVolumeSpecName: "kube-api-access-q9q5k") pod "c069aee2-4cb6-4c8a-bbe5-949ab0b48331" (UID: "c069aee2-4cb6-4c8a-bbe5-949ab0b48331"). InnerVolumeSpecName "kube-api-access-q9q5k". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:46:50 crc kubenswrapper[5052]: I1203 12:46:50.082869 5052 scope.go:117] "RemoveContainer" containerID="4c8ae2743fa5049ef6f420e9338425946c81202ba97ccaf63ece2ed399a93b82" Dec 03 12:46:50 crc kubenswrapper[5052]: I1203 12:46:50.084048 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-api-6cd888f578-nv6fh"] Dec 03 12:46:50 crc kubenswrapper[5052]: I1203 12:46:50.088477 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c069aee2-4cb6-4c8a-bbe5-949ab0b48331-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "c069aee2-4cb6-4c8a-bbe5-949ab0b48331" (UID: "c069aee2-4cb6-4c8a-bbe5-949ab0b48331"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:46:50 crc kubenswrapper[5052]: I1203 12:46:50.088832 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c069aee2-4cb6-4c8a-bbe5-949ab0b48331-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "c069aee2-4cb6-4c8a-bbe5-949ab0b48331" (UID: "c069aee2-4cb6-4c8a-bbe5-949ab0b48331"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:46:50 crc kubenswrapper[5052]: I1203 12:46:50.097819 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c069aee2-4cb6-4c8a-bbe5-949ab0b48331-config-data" (OuterVolumeSpecName: "config-data") pod "c069aee2-4cb6-4c8a-bbe5-949ab0b48331" (UID: "c069aee2-4cb6-4c8a-bbe5-949ab0b48331"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:46:50 crc kubenswrapper[5052]: I1203 12:46:50.097923 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/placement-bbcfbbc9d-5q6gx"] Dec 03 12:46:50 crc kubenswrapper[5052]: W1203 12:46:50.098424 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1f026d39_713b_4611_80fd_9577e697d0c4.slice/crio-5ed7fdb90750b8af1a853a0981c8a82939452ba3c9225b2cb4998ae709d8cb14 WatchSource:0}: Error finding container 5ed7fdb90750b8af1a853a0981c8a82939452ba3c9225b2cb4998ae709d8cb14: Status 404 returned error can't find the container with id 5ed7fdb90750b8af1a853a0981c8a82939452ba3c9225b2cb4998ae709d8cb14 Dec 03 12:46:50 crc kubenswrapper[5052]: I1203 12:46:50.119056 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/placement-bbcfbbc9d-5q6gx"] Dec 03 12:46:50 crc kubenswrapper[5052]: I1203 12:46:50.135478 5052 scope.go:117] "RemoveContainer" containerID="353ad09a2eae7f22cab9c43ca6063becb9fb6d4010c1a66dcef0e6673635198d" Dec 03 12:46:50 crc kubenswrapper[5052]: E1203 12:46:50.137686 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"353ad09a2eae7f22cab9c43ca6063becb9fb6d4010c1a66dcef0e6673635198d\": container with ID starting with 353ad09a2eae7f22cab9c43ca6063becb9fb6d4010c1a66dcef0e6673635198d not found: ID does not exist" containerID="353ad09a2eae7f22cab9c43ca6063becb9fb6d4010c1a66dcef0e6673635198d" Dec 03 12:46:50 crc kubenswrapper[5052]: I1203 12:46:50.137723 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"353ad09a2eae7f22cab9c43ca6063becb9fb6d4010c1a66dcef0e6673635198d"} err="failed to get container status \"353ad09a2eae7f22cab9c43ca6063becb9fb6d4010c1a66dcef0e6673635198d\": rpc error: code = NotFound desc = could not find container \"353ad09a2eae7f22cab9c43ca6063becb9fb6d4010c1a66dcef0e6673635198d\": container with ID starting with 353ad09a2eae7f22cab9c43ca6063becb9fb6d4010c1a66dcef0e6673635198d not found: ID does not exist" Dec 03 12:46:50 crc kubenswrapper[5052]: I1203 12:46:50.137746 5052 scope.go:117] "RemoveContainer" containerID="4c8ae2743fa5049ef6f420e9338425946c81202ba97ccaf63ece2ed399a93b82" Dec 03 12:46:50 crc kubenswrapper[5052]: E1203 12:46:50.138065 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4c8ae2743fa5049ef6f420e9338425946c81202ba97ccaf63ece2ed399a93b82\": container with ID starting with 4c8ae2743fa5049ef6f420e9338425946c81202ba97ccaf63ece2ed399a93b82 not found: ID does not exist" containerID="4c8ae2743fa5049ef6f420e9338425946c81202ba97ccaf63ece2ed399a93b82" Dec 03 12:46:50 crc kubenswrapper[5052]: I1203 12:46:50.138442 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4c8ae2743fa5049ef6f420e9338425946c81202ba97ccaf63ece2ed399a93b82"} err="failed to get container status \"4c8ae2743fa5049ef6f420e9338425946c81202ba97ccaf63ece2ed399a93b82\": rpc error: code = NotFound desc = could not find container \"4c8ae2743fa5049ef6f420e9338425946c81202ba97ccaf63ece2ed399a93b82\": container with ID starting with 4c8ae2743fa5049ef6f420e9338425946c81202ba97ccaf63ece2ed399a93b82 not found: ID does not exist" Dec 03 12:46:50 crc kubenswrapper[5052]: I1203 12:46:50.173051 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/c069aee2-4cb6-4c8a-bbe5-949ab0b48331-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:50 crc kubenswrapper[5052]: I1203 12:46:50.173088 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-q9q5k\" (UniqueName: \"kubernetes.io/projected/c069aee2-4cb6-4c8a-bbe5-949ab0b48331-kube-api-access-q9q5k\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:50 crc kubenswrapper[5052]: I1203 12:46:50.173102 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c069aee2-4cb6-4c8a-bbe5-949ab0b48331-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:50 crc kubenswrapper[5052]: I1203 12:46:50.173114 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/c069aee2-4cb6-4c8a-bbe5-949ab0b48331-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:50 crc kubenswrapper[5052]: I1203 12:46:50.173125 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/c069aee2-4cb6-4c8a-bbe5-949ab0b48331-config-data-custom\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:50 crc kubenswrapper[5052]: I1203 12:46:50.173135 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/c069aee2-4cb6-4c8a-bbe5-949ab0b48331-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:50 crc kubenswrapper[5052]: I1203 12:46:50.384693 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-metadata-0" podUID="75266130-5854-4acd-acc9-bc22accc19bc" containerName="nova-metadata-log" probeResult="failure" output="Get \"https://10.217.0.12:8775/\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Dec 03 12:46:50 crc kubenswrapper[5052]: I1203 12:46:50.384736 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-metadata-0" podUID="75266130-5854-4acd-acc9-bc22accc19bc" containerName="nova-metadata-metadata" probeResult="failure" output="Get \"https://10.217.0.12:8775/\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Dec 03 12:46:51 crc kubenswrapper[5052]: I1203 12:46:51.009816 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_neutron-bcd9f4f4-djpt8_049a231b-3a12-4710-9b7a-117b1bb82e72/neutron-api/0.log" Dec 03 12:46:51 crc kubenswrapper[5052]: I1203 12:46:51.009883 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-bcd9f4f4-djpt8" event={"ID":"049a231b-3a12-4710-9b7a-117b1bb82e72","Type":"ContainerStarted","Data":"25be0ef1cecaae7aecb0b6c706591b1e2a075e2221623da71a4fac6ef2da0c06"} Dec 03 12:46:51 crc kubenswrapper[5052]: I1203 12:46:51.014579 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-6cd888f578-nv6fh" event={"ID":"1f026d39-713b-4611-80fd-9577e697d0c4","Type":"ContainerStarted","Data":"56d9b1a087e1cc1a1e32bb0b52dc44b84176e308fe03f914bdbda9333ae12466"} Dec 03 12:46:51 crc kubenswrapper[5052]: I1203 12:46:51.014622 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-6cd888f578-nv6fh" event={"ID":"1f026d39-713b-4611-80fd-9577e697d0c4","Type":"ContainerStarted","Data":"795d64dcbb859f297be1ee826fd9ec0faf4da92063117b21eb577b822e4b7657"} Dec 03 12:46:51 crc kubenswrapper[5052]: I1203 12:46:51.014632 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-6cd888f578-nv6fh" event={"ID":"1f026d39-713b-4611-80fd-9577e697d0c4","Type":"ContainerStarted","Data":"5ed7fdb90750b8af1a853a0981c8a82939452ba3c9225b2cb4998ae709d8cb14"} Dec 03 12:46:51 crc kubenswrapper[5052]: I1203 12:46:51.015569 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/barbican-api-6cd888f578-nv6fh" Dec 03 12:46:51 crc kubenswrapper[5052]: I1203 12:46:51.015621 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/barbican-api-6cd888f578-nv6fh" Dec 03 12:46:51 crc kubenswrapper[5052]: I1203 12:46:51.017292 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-api-bcf49b784-jnkdb" Dec 03 12:46:51 crc kubenswrapper[5052]: I1203 12:46:51.028678 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/neutron-bcd9f4f4-djpt8" podUID="049a231b-3a12-4710-9b7a-117b1bb82e72" containerName="neutron-httpd" probeResult="failure" output="HTTP probe failed with statuscode: 503" Dec 03 12:46:51 crc kubenswrapper[5052]: I1203 12:46:51.069002 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/barbican-api-6cd888f578-nv6fh" podStartSLOduration=2.068985442 podStartE2EDuration="2.068985442s" podCreationTimestamp="2025-12-03 12:46:49 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:46:51.067060437 +0000 UTC m=+5201.655302201" watchObservedRunningTime="2025-12-03 12:46:51.068985442 +0000 UTC m=+5201.657227196" Dec 03 12:46:51 crc kubenswrapper[5052]: I1203 12:46:51.113342 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-api-bcf49b784-jnkdb"] Dec 03 12:46:51 crc kubenswrapper[5052]: I1203 12:46:51.123154 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/barbican-api-bcf49b784-jnkdb"] Dec 03 12:46:51 crc kubenswrapper[5052]: I1203 12:46:51.195243 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/bf8b9b9c-b8f9-4c93-8112-43913950fcdf-internal-tls-certs\") pod \"glance-default-internal-api-0\" (UID: \"bf8b9b9c-b8f9-4c93-8112-43913950fcdf\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:46:51 crc kubenswrapper[5052]: I1203 12:46:51.195334 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/c069aee2-4cb6-4c8a-bbe5-949ab0b48331-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:51 crc kubenswrapper[5052]: I1203 12:46:51.200465 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/bf8b9b9c-b8f9-4c93-8112-43913950fcdf-internal-tls-certs\") pod \"glance-default-internal-api-0\" (UID: \"bf8b9b9c-b8f9-4c93-8112-43913950fcdf\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:46:51 crc kubenswrapper[5052]: I1203 12:46:51.283604 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:46:51 crc kubenswrapper[5052]: I1203 12:46:51.303260 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:46:51 crc kubenswrapper[5052]: I1203 12:46:51.350100 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:46:51 crc kubenswrapper[5052]: I1203 12:46:51.472436 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:46:51 crc kubenswrapper[5052]: I1203 12:46:51.472493 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:46:51 crc kubenswrapper[5052]: I1203 12:46:51.538810 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:46:51 crc kubenswrapper[5052]: I1203 12:46:51.695426 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="32f420c2-aa7e-44c8-a63a-7654aa3e6e34" path="/var/lib/kubelet/pods/32f420c2-aa7e-44c8-a63a-7654aa3e6e34/volumes" Dec 03 12:46:51 crc kubenswrapper[5052]: I1203 12:46:51.696072 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c069aee2-4cb6-4c8a-bbe5-949ab0b48331" path="/var/lib/kubelet/pods/c069aee2-4cb6-4c8a-bbe5-949ab0b48331/volumes" Dec 03 12:46:51 crc kubenswrapper[5052]: I1203 12:46:51.939210 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:46:52 crc kubenswrapper[5052]: I1203 12:46:52.029891 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"bf8b9b9c-b8f9-4c93-8112-43913950fcdf","Type":"ContainerStarted","Data":"1bd9dac09969a248dc0fa8abc9681c81993311f2ce27d7a6014d719e9e4dbd38"} Dec 03 12:46:52 crc kubenswrapper[5052]: I1203 12:46:52.054896 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:46:52 crc kubenswrapper[5052]: I1203 12:46:52.518771 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-api-0" podUID="d5719b07-205b-4e6a-9fe2-b6531b779b0a" containerName="nova-api-api" probeResult="failure" output="Get \"https://10.217.0.98:8774/\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Dec 03 12:46:52 crc kubenswrapper[5052]: I1203 12:46:52.519336 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-api-0" podUID="d5719b07-205b-4e6a-9fe2-b6531b779b0a" containerName="nova-api-log" probeResult="failure" output="Get \"https://10.217.0.98:8774/\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Dec 03 12:46:53 crc kubenswrapper[5052]: I1203 12:46:53.055289 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"bf8b9b9c-b8f9-4c93-8112-43913950fcdf","Type":"ContainerStarted","Data":"acb5b85ad6f2dd6eeace8cadc076149f4f2d58d20643d30b4d92c04ad6dcd41d"} Dec 03 12:46:53 crc kubenswrapper[5052]: I1203 12:46:53.069300 5052 generic.go:334] "Generic (PLEG): container finished" podID="8669832d-c47a-49bd-9491-dc7141ac6281" containerID="e102925eb6bfd868ed0b7dff92ded1a29e84b2579d3092b133af1f0bf476d04c" exitCode=1 Dec 03 12:46:53 crc kubenswrapper[5052]: I1203 12:46:53.069688 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-conductor-0" event={"ID":"8669832d-c47a-49bd-9491-dc7141ac6281","Type":"ContainerDied","Data":"e102925eb6bfd868ed0b7dff92ded1a29e84b2579d3092b133af1f0bf476d04c"} Dec 03 12:46:53 crc kubenswrapper[5052]: I1203 12:46:53.069764 5052 scope.go:117] "RemoveContainer" containerID="d845f1b1ea8802fd6fd4c7f6098f521746fa875f2f1e1c94973d2bf864333d42" Dec 03 12:46:53 crc kubenswrapper[5052]: I1203 12:46:53.072486 5052 scope.go:117] "RemoveContainer" containerID="e102925eb6bfd868ed0b7dff92ded1a29e84b2579d3092b133af1f0bf476d04c" Dec 03 12:46:53 crc kubenswrapper[5052]: E1203 12:46:53.075794 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"nova-cell1-conductor-conductor\" with CrashLoopBackOff: \"back-off 10s restarting failed container=nova-cell1-conductor-conductor pod=nova-cell1-conductor-0_openstack-kuttl-tests(8669832d-c47a-49bd-9491-dc7141ac6281)\"" pod="openstack-kuttl-tests/nova-cell1-conductor-0" podUID="8669832d-c47a-49bd-9491-dc7141ac6281" Dec 03 12:46:53 crc kubenswrapper[5052]: I1203 12:46:53.284413 5052 kubelet.go:2542] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:46:53 crc kubenswrapper[5052]: I1203 12:46:53.504698 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/memcached-0" Dec 03 12:46:53 crc kubenswrapper[5052]: I1203 12:46:53.634469 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:46:53 crc kubenswrapper[5052]: I1203 12:46:53.639218 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-external-api-0" podUID="4c9f4da8-f1ae-4815-8ee2-0e873fb07d62" containerName="glance-log" containerID="cri-o://1974352f744e6b1132798362ce57f0fa14a8df4a45523231261a216257d4872d" gracePeriod=30 Dec 03 12:46:53 crc kubenswrapper[5052]: I1203 12:46:53.639383 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-external-api-0" podUID="4c9f4da8-f1ae-4815-8ee2-0e873fb07d62" containerName="glance-httpd" containerID="cri-o://c04c0411733e4ad72fe617e9a0abb2297cf6a19a36fdb5ef4c3061887c382c9a" gracePeriod=30 Dec 03 12:46:53 crc kubenswrapper[5052]: I1203 12:46:53.675417 5052 scope.go:117] "RemoveContainer" containerID="2aed1c760a326f3d008799e902b1093313df3764a57c879f0b3686410127d37d" Dec 03 12:46:53 crc kubenswrapper[5052]: E1203 12:46:53.675929 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 12:46:53 crc kubenswrapper[5052]: I1203 12:46:53.723152 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:46:53 crc kubenswrapper[5052]: I1203 12:46:53.761803 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/keystone-687ff6fcfc-9vwb2"] Dec 03 12:46:53 crc kubenswrapper[5052]: E1203 12:46:53.762948 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[internal-tls-certs], unattached volumes=[], failed to process volumes=[]: context canceled" pod="openstack-kuttl-tests/keystone-687ff6fcfc-9vwb2" podUID="fa1c223d-82fa-488c-ac48-e88bce36a839" Dec 03 12:46:53 crc kubenswrapper[5052]: I1203 12:46:53.809565 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/keystone-6c4f97778c-d7wtk"] Dec 03 12:46:53 crc kubenswrapper[5052]: E1203 12:46:53.810109 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="32f420c2-aa7e-44c8-a63a-7654aa3e6e34" containerName="placement-log" Dec 03 12:46:53 crc kubenswrapper[5052]: I1203 12:46:53.810132 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="32f420c2-aa7e-44c8-a63a-7654aa3e6e34" containerName="placement-log" Dec 03 12:46:53 crc kubenswrapper[5052]: E1203 12:46:53.810169 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="32f420c2-aa7e-44c8-a63a-7654aa3e6e34" containerName="placement-api" Dec 03 12:46:53 crc kubenswrapper[5052]: I1203 12:46:53.810177 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="32f420c2-aa7e-44c8-a63a-7654aa3e6e34" containerName="placement-api" Dec 03 12:46:53 crc kubenswrapper[5052]: I1203 12:46:53.810456 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="32f420c2-aa7e-44c8-a63a-7654aa3e6e34" containerName="placement-api" Dec 03 12:46:53 crc kubenswrapper[5052]: I1203 12:46:53.810478 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="32f420c2-aa7e-44c8-a63a-7654aa3e6e34" containerName="placement-log" Dec 03 12:46:53 crc kubenswrapper[5052]: I1203 12:46:53.811330 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-6c4f97778c-d7wtk" Dec 03 12:46:53 crc kubenswrapper[5052]: I1203 12:46:53.829447 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-6c4f97778c-d7wtk"] Dec 03 12:46:53 crc kubenswrapper[5052]: I1203 12:46:53.880281 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/cb2b8abb-bf7a-4e95-87dc-cd980d8bce31-combined-ca-bundle\") pod \"keystone-6c4f97778c-d7wtk\" (UID: \"cb2b8abb-bf7a-4e95-87dc-cd980d8bce31\") " pod="openstack-kuttl-tests/keystone-6c4f97778c-d7wtk" Dec 03 12:46:53 crc kubenswrapper[5052]: I1203 12:46:53.880349 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/cb2b8abb-bf7a-4e95-87dc-cd980d8bce31-fernet-keys\") pod \"keystone-6c4f97778c-d7wtk\" (UID: \"cb2b8abb-bf7a-4e95-87dc-cd980d8bce31\") " pod="openstack-kuttl-tests/keystone-6c4f97778c-d7wtk" Dec 03 12:46:53 crc kubenswrapper[5052]: I1203 12:46:53.880393 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/cb2b8abb-bf7a-4e95-87dc-cd980d8bce31-config-data\") pod \"keystone-6c4f97778c-d7wtk\" (UID: \"cb2b8abb-bf7a-4e95-87dc-cd980d8bce31\") " pod="openstack-kuttl-tests/keystone-6c4f97778c-d7wtk" Dec 03 12:46:53 crc kubenswrapper[5052]: I1203 12:46:53.880443 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jb4nv\" (UniqueName: \"kubernetes.io/projected/cb2b8abb-bf7a-4e95-87dc-cd980d8bce31-kube-api-access-jb4nv\") pod \"keystone-6c4f97778c-d7wtk\" (UID: \"cb2b8abb-bf7a-4e95-87dc-cd980d8bce31\") " pod="openstack-kuttl-tests/keystone-6c4f97778c-d7wtk" Dec 03 12:46:53 crc kubenswrapper[5052]: I1203 12:46:53.880702 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/cb2b8abb-bf7a-4e95-87dc-cd980d8bce31-public-tls-certs\") pod \"keystone-6c4f97778c-d7wtk\" (UID: \"cb2b8abb-bf7a-4e95-87dc-cd980d8bce31\") " pod="openstack-kuttl-tests/keystone-6c4f97778c-d7wtk" Dec 03 12:46:53 crc kubenswrapper[5052]: I1203 12:46:53.880804 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/cb2b8abb-bf7a-4e95-87dc-cd980d8bce31-credential-keys\") pod \"keystone-6c4f97778c-d7wtk\" (UID: \"cb2b8abb-bf7a-4e95-87dc-cd980d8bce31\") " pod="openstack-kuttl-tests/keystone-6c4f97778c-d7wtk" Dec 03 12:46:53 crc kubenswrapper[5052]: I1203 12:46:53.880946 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/cb2b8abb-bf7a-4e95-87dc-cd980d8bce31-internal-tls-certs\") pod \"keystone-6c4f97778c-d7wtk\" (UID: \"cb2b8abb-bf7a-4e95-87dc-cd980d8bce31\") " pod="openstack-kuttl-tests/keystone-6c4f97778c-d7wtk" Dec 03 12:46:53 crc kubenswrapper[5052]: I1203 12:46:53.881036 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/cb2b8abb-bf7a-4e95-87dc-cd980d8bce31-scripts\") pod \"keystone-6c4f97778c-d7wtk\" (UID: \"cb2b8abb-bf7a-4e95-87dc-cd980d8bce31\") " pod="openstack-kuttl-tests/keystone-6c4f97778c-d7wtk" Dec 03 12:46:53 crc kubenswrapper[5052]: I1203 12:46:53.982400 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/cb2b8abb-bf7a-4e95-87dc-cd980d8bce31-credential-keys\") pod \"keystone-6c4f97778c-d7wtk\" (UID: \"cb2b8abb-bf7a-4e95-87dc-cd980d8bce31\") " pod="openstack-kuttl-tests/keystone-6c4f97778c-d7wtk" Dec 03 12:46:53 crc kubenswrapper[5052]: I1203 12:46:53.982529 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/cb2b8abb-bf7a-4e95-87dc-cd980d8bce31-internal-tls-certs\") pod \"keystone-6c4f97778c-d7wtk\" (UID: \"cb2b8abb-bf7a-4e95-87dc-cd980d8bce31\") " pod="openstack-kuttl-tests/keystone-6c4f97778c-d7wtk" Dec 03 12:46:53 crc kubenswrapper[5052]: I1203 12:46:53.982580 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/cb2b8abb-bf7a-4e95-87dc-cd980d8bce31-scripts\") pod \"keystone-6c4f97778c-d7wtk\" (UID: \"cb2b8abb-bf7a-4e95-87dc-cd980d8bce31\") " pod="openstack-kuttl-tests/keystone-6c4f97778c-d7wtk" Dec 03 12:46:53 crc kubenswrapper[5052]: I1203 12:46:53.982643 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/cb2b8abb-bf7a-4e95-87dc-cd980d8bce31-combined-ca-bundle\") pod \"keystone-6c4f97778c-d7wtk\" (UID: \"cb2b8abb-bf7a-4e95-87dc-cd980d8bce31\") " pod="openstack-kuttl-tests/keystone-6c4f97778c-d7wtk" Dec 03 12:46:53 crc kubenswrapper[5052]: I1203 12:46:53.982658 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/cb2b8abb-bf7a-4e95-87dc-cd980d8bce31-fernet-keys\") pod \"keystone-6c4f97778c-d7wtk\" (UID: \"cb2b8abb-bf7a-4e95-87dc-cd980d8bce31\") " pod="openstack-kuttl-tests/keystone-6c4f97778c-d7wtk" Dec 03 12:46:53 crc kubenswrapper[5052]: I1203 12:46:53.982677 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/cb2b8abb-bf7a-4e95-87dc-cd980d8bce31-config-data\") pod \"keystone-6c4f97778c-d7wtk\" (UID: \"cb2b8abb-bf7a-4e95-87dc-cd980d8bce31\") " pod="openstack-kuttl-tests/keystone-6c4f97778c-d7wtk" Dec 03 12:46:53 crc kubenswrapper[5052]: I1203 12:46:53.982698 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-jb4nv\" (UniqueName: \"kubernetes.io/projected/cb2b8abb-bf7a-4e95-87dc-cd980d8bce31-kube-api-access-jb4nv\") pod \"keystone-6c4f97778c-d7wtk\" (UID: \"cb2b8abb-bf7a-4e95-87dc-cd980d8bce31\") " pod="openstack-kuttl-tests/keystone-6c4f97778c-d7wtk" Dec 03 12:46:53 crc kubenswrapper[5052]: I1203 12:46:53.982742 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/cb2b8abb-bf7a-4e95-87dc-cd980d8bce31-public-tls-certs\") pod \"keystone-6c4f97778c-d7wtk\" (UID: \"cb2b8abb-bf7a-4e95-87dc-cd980d8bce31\") " pod="openstack-kuttl-tests/keystone-6c4f97778c-d7wtk" Dec 03 12:46:53 crc kubenswrapper[5052]: I1203 12:46:53.989540 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/cb2b8abb-bf7a-4e95-87dc-cd980d8bce31-scripts\") pod \"keystone-6c4f97778c-d7wtk\" (UID: \"cb2b8abb-bf7a-4e95-87dc-cd980d8bce31\") " pod="openstack-kuttl-tests/keystone-6c4f97778c-d7wtk" Dec 03 12:46:53 crc kubenswrapper[5052]: I1203 12:46:53.990502 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/cb2b8abb-bf7a-4e95-87dc-cd980d8bce31-config-data\") pod \"keystone-6c4f97778c-d7wtk\" (UID: \"cb2b8abb-bf7a-4e95-87dc-cd980d8bce31\") " pod="openstack-kuttl-tests/keystone-6c4f97778c-d7wtk" Dec 03 12:46:53 crc kubenswrapper[5052]: I1203 12:46:53.991075 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/cb2b8abb-bf7a-4e95-87dc-cd980d8bce31-credential-keys\") pod \"keystone-6c4f97778c-d7wtk\" (UID: \"cb2b8abb-bf7a-4e95-87dc-cd980d8bce31\") " pod="openstack-kuttl-tests/keystone-6c4f97778c-d7wtk" Dec 03 12:46:53 crc kubenswrapper[5052]: I1203 12:46:53.991766 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/cb2b8abb-bf7a-4e95-87dc-cd980d8bce31-public-tls-certs\") pod \"keystone-6c4f97778c-d7wtk\" (UID: \"cb2b8abb-bf7a-4e95-87dc-cd980d8bce31\") " pod="openstack-kuttl-tests/keystone-6c4f97778c-d7wtk" Dec 03 12:46:53 crc kubenswrapper[5052]: I1203 12:46:53.994477 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/cb2b8abb-bf7a-4e95-87dc-cd980d8bce31-combined-ca-bundle\") pod \"keystone-6c4f97778c-d7wtk\" (UID: \"cb2b8abb-bf7a-4e95-87dc-cd980d8bce31\") " pod="openstack-kuttl-tests/keystone-6c4f97778c-d7wtk" Dec 03 12:46:53 crc kubenswrapper[5052]: I1203 12:46:53.998249 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/cb2b8abb-bf7a-4e95-87dc-cd980d8bce31-fernet-keys\") pod \"keystone-6c4f97778c-d7wtk\" (UID: \"cb2b8abb-bf7a-4e95-87dc-cd980d8bce31\") " pod="openstack-kuttl-tests/keystone-6c4f97778c-d7wtk" Dec 03 12:46:53 crc kubenswrapper[5052]: I1203 12:46:53.999154 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/cb2b8abb-bf7a-4e95-87dc-cd980d8bce31-internal-tls-certs\") pod \"keystone-6c4f97778c-d7wtk\" (UID: \"cb2b8abb-bf7a-4e95-87dc-cd980d8bce31\") " pod="openstack-kuttl-tests/keystone-6c4f97778c-d7wtk" Dec 03 12:46:54 crc kubenswrapper[5052]: I1203 12:46:54.018991 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-jb4nv\" (UniqueName: \"kubernetes.io/projected/cb2b8abb-bf7a-4e95-87dc-cd980d8bce31-kube-api-access-jb4nv\") pod \"keystone-6c4f97778c-d7wtk\" (UID: \"cb2b8abb-bf7a-4e95-87dc-cd980d8bce31\") " pod="openstack-kuttl-tests/keystone-6c4f97778c-d7wtk" Dec 03 12:46:54 crc kubenswrapper[5052]: I1203 12:46:54.081563 5052 scope.go:117] "RemoveContainer" containerID="e102925eb6bfd868ed0b7dff92ded1a29e84b2579d3092b133af1f0bf476d04c" Dec 03 12:46:54 crc kubenswrapper[5052]: E1203 12:46:54.082048 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"nova-cell1-conductor-conductor\" with CrashLoopBackOff: \"back-off 10s restarting failed container=nova-cell1-conductor-conductor pod=nova-cell1-conductor-0_openstack-kuttl-tests(8669832d-c47a-49bd-9491-dc7141ac6281)\"" pod="openstack-kuttl-tests/nova-cell1-conductor-0" podUID="8669832d-c47a-49bd-9491-dc7141ac6281" Dec 03 12:46:54 crc kubenswrapper[5052]: I1203 12:46:54.083065 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"bf8b9b9c-b8f9-4c93-8112-43913950fcdf","Type":"ContainerStarted","Data":"47769437fc8e2335ee64f20b3f593a902f20b71b2bb53ebf37e7accf0cc2c14e"} Dec 03 12:46:54 crc kubenswrapper[5052]: I1203 12:46:54.092277 5052 generic.go:334] "Generic (PLEG): container finished" podID="4c9f4da8-f1ae-4815-8ee2-0e873fb07d62" containerID="1974352f744e6b1132798362ce57f0fa14a8df4a45523231261a216257d4872d" exitCode=143 Dec 03 12:46:54 crc kubenswrapper[5052]: I1203 12:46:54.092343 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-687ff6fcfc-9vwb2" Dec 03 12:46:54 crc kubenswrapper[5052]: I1203 12:46:54.093112 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"4c9f4da8-f1ae-4815-8ee2-0e873fb07d62","Type":"ContainerDied","Data":"1974352f744e6b1132798362ce57f0fa14a8df4a45523231261a216257d4872d"} Dec 03 12:46:54 crc kubenswrapper[5052]: I1203 12:46:54.113259 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-687ff6fcfc-9vwb2" Dec 03 12:46:54 crc kubenswrapper[5052]: I1203 12:46:54.131343 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-6c4f97778c-d7wtk" Dec 03 12:46:54 crc kubenswrapper[5052]: I1203 12:46:54.286703 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/fa1c223d-82fa-488c-ac48-e88bce36a839-combined-ca-bundle\") pod \"fa1c223d-82fa-488c-ac48-e88bce36a839\" (UID: \"fa1c223d-82fa-488c-ac48-e88bce36a839\") " Dec 03 12:46:54 crc kubenswrapper[5052]: I1203 12:46:54.286845 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/fa1c223d-82fa-488c-ac48-e88bce36a839-public-tls-certs\") pod \"fa1c223d-82fa-488c-ac48-e88bce36a839\" (UID: \"fa1c223d-82fa-488c-ac48-e88bce36a839\") " Dec 03 12:46:54 crc kubenswrapper[5052]: I1203 12:46:54.286881 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/fa1c223d-82fa-488c-ac48-e88bce36a839-credential-keys\") pod \"fa1c223d-82fa-488c-ac48-e88bce36a839\" (UID: \"fa1c223d-82fa-488c-ac48-e88bce36a839\") " Dec 03 12:46:54 crc kubenswrapper[5052]: I1203 12:46:54.286919 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/fa1c223d-82fa-488c-ac48-e88bce36a839-fernet-keys\") pod \"fa1c223d-82fa-488c-ac48-e88bce36a839\" (UID: \"fa1c223d-82fa-488c-ac48-e88bce36a839\") " Dec 03 12:46:54 crc kubenswrapper[5052]: I1203 12:46:54.286986 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bsn6g\" (UniqueName: \"kubernetes.io/projected/fa1c223d-82fa-488c-ac48-e88bce36a839-kube-api-access-bsn6g\") pod \"fa1c223d-82fa-488c-ac48-e88bce36a839\" (UID: \"fa1c223d-82fa-488c-ac48-e88bce36a839\") " Dec 03 12:46:54 crc kubenswrapper[5052]: I1203 12:46:54.287082 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/fa1c223d-82fa-488c-ac48-e88bce36a839-scripts\") pod \"fa1c223d-82fa-488c-ac48-e88bce36a839\" (UID: \"fa1c223d-82fa-488c-ac48-e88bce36a839\") " Dec 03 12:46:54 crc kubenswrapper[5052]: I1203 12:46:54.287147 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/fa1c223d-82fa-488c-ac48-e88bce36a839-config-data\") pod \"fa1c223d-82fa-488c-ac48-e88bce36a839\" (UID: \"fa1c223d-82fa-488c-ac48-e88bce36a839\") " Dec 03 12:46:54 crc kubenswrapper[5052]: I1203 12:46:54.292915 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fa1c223d-82fa-488c-ac48-e88bce36a839-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "fa1c223d-82fa-488c-ac48-e88bce36a839" (UID: "fa1c223d-82fa-488c-ac48-e88bce36a839"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:46:54 crc kubenswrapper[5052]: I1203 12:46:54.292966 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fa1c223d-82fa-488c-ac48-e88bce36a839-config-data" (OuterVolumeSpecName: "config-data") pod "fa1c223d-82fa-488c-ac48-e88bce36a839" (UID: "fa1c223d-82fa-488c-ac48-e88bce36a839"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:46:54 crc kubenswrapper[5052]: I1203 12:46:54.292979 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fa1c223d-82fa-488c-ac48-e88bce36a839-scripts" (OuterVolumeSpecName: "scripts") pod "fa1c223d-82fa-488c-ac48-e88bce36a839" (UID: "fa1c223d-82fa-488c-ac48-e88bce36a839"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:46:54 crc kubenswrapper[5052]: I1203 12:46:54.292997 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fa1c223d-82fa-488c-ac48-e88bce36a839-credential-keys" (OuterVolumeSpecName: "credential-keys") pod "fa1c223d-82fa-488c-ac48-e88bce36a839" (UID: "fa1c223d-82fa-488c-ac48-e88bce36a839"). InnerVolumeSpecName "credential-keys". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:46:54 crc kubenswrapper[5052]: I1203 12:46:54.293041 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fa1c223d-82fa-488c-ac48-e88bce36a839-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "fa1c223d-82fa-488c-ac48-e88bce36a839" (UID: "fa1c223d-82fa-488c-ac48-e88bce36a839"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:46:54 crc kubenswrapper[5052]: I1203 12:46:54.294779 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fa1c223d-82fa-488c-ac48-e88bce36a839-fernet-keys" (OuterVolumeSpecName: "fernet-keys") pod "fa1c223d-82fa-488c-ac48-e88bce36a839" (UID: "fa1c223d-82fa-488c-ac48-e88bce36a839"). InnerVolumeSpecName "fernet-keys". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:46:54 crc kubenswrapper[5052]: I1203 12:46:54.294823 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fa1c223d-82fa-488c-ac48-e88bce36a839-kube-api-access-bsn6g" (OuterVolumeSpecName: "kube-api-access-bsn6g") pod "fa1c223d-82fa-488c-ac48-e88bce36a839" (UID: "fa1c223d-82fa-488c-ac48-e88bce36a839"). InnerVolumeSpecName "kube-api-access-bsn6g". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:46:54 crc kubenswrapper[5052]: I1203 12:46:54.390052 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/fa1c223d-82fa-488c-ac48-e88bce36a839-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:54 crc kubenswrapper[5052]: I1203 12:46:54.390093 5052 reconciler_common.go:293] "Volume detached for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/fa1c223d-82fa-488c-ac48-e88bce36a839-credential-keys\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:54 crc kubenswrapper[5052]: I1203 12:46:54.390105 5052 reconciler_common.go:293] "Volume detached for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/fa1c223d-82fa-488c-ac48-e88bce36a839-fernet-keys\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:54 crc kubenswrapper[5052]: I1203 12:46:54.390119 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-bsn6g\" (UniqueName: \"kubernetes.io/projected/fa1c223d-82fa-488c-ac48-e88bce36a839-kube-api-access-bsn6g\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:54 crc kubenswrapper[5052]: I1203 12:46:54.390134 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/fa1c223d-82fa-488c-ac48-e88bce36a839-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:54 crc kubenswrapper[5052]: I1203 12:46:54.390147 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/fa1c223d-82fa-488c-ac48-e88bce36a839-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:54 crc kubenswrapper[5052]: I1203 12:46:54.390159 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/fa1c223d-82fa-488c-ac48-e88bce36a839-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:54 crc kubenswrapper[5052]: I1203 12:46:54.590925 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/glance-default-internal-api-0" podStartSLOduration=36.590908192 podStartE2EDuration="36.590908192s" podCreationTimestamp="2025-12-03 12:46:18 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:46:54.133709027 +0000 UTC m=+5204.721950821" watchObservedRunningTime="2025-12-03 12:46:54.590908192 +0000 UTC m=+5205.179149956" Dec 03 12:46:54 crc kubenswrapper[5052]: I1203 12:46:54.598632 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-6c4f97778c-d7wtk"] Dec 03 12:46:55 crc kubenswrapper[5052]: I1203 12:46:55.103139 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-6c4f97778c-d7wtk" event={"ID":"cb2b8abb-bf7a-4e95-87dc-cd980d8bce31","Type":"ContainerStarted","Data":"50ac031a1fe86efac0c23bcb4c777a2da13c93e90ce099262825387ad869bc18"} Dec 03 12:46:55 crc kubenswrapper[5052]: I1203 12:46:55.103465 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-6c4f97778c-d7wtk" event={"ID":"cb2b8abb-bf7a-4e95-87dc-cd980d8bce31","Type":"ContainerStarted","Data":"74d6a732063e9dc0e9610fcaae0057097ded837585a892d81f869b30a04dd0db"} Dec 03 12:46:55 crc kubenswrapper[5052]: I1203 12:46:55.103307 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-internal-api-0" podUID="bf8b9b9c-b8f9-4c93-8112-43913950fcdf" containerName="glance-log" containerID="cri-o://acb5b85ad6f2dd6eeace8cadc076149f4f2d58d20643d30b4d92c04ad6dcd41d" gracePeriod=30 Dec 03 12:46:55 crc kubenswrapper[5052]: I1203 12:46:55.103440 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-internal-api-0" podUID="bf8b9b9c-b8f9-4c93-8112-43913950fcdf" containerName="glance-httpd" containerID="cri-o://47769437fc8e2335ee64f20b3f593a902f20b71b2bb53ebf37e7accf0cc2c14e" gracePeriod=30 Dec 03 12:46:55 crc kubenswrapper[5052]: I1203 12:46:55.103181 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-687ff6fcfc-9vwb2" Dec 03 12:46:55 crc kubenswrapper[5052]: I1203 12:46:55.129452 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/keystone-6c4f97778c-d7wtk" podStartSLOduration=2.129435693 podStartE2EDuration="2.129435693s" podCreationTimestamp="2025-12-03 12:46:53 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:46:55.129237918 +0000 UTC m=+5205.717479682" watchObservedRunningTime="2025-12-03 12:46:55.129435693 +0000 UTC m=+5205.717677457" Dec 03 12:46:55 crc kubenswrapper[5052]: I1203 12:46:55.173719 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/keystone-687ff6fcfc-9vwb2"] Dec 03 12:46:55 crc kubenswrapper[5052]: I1203 12:46:55.183450 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/keystone-687ff6fcfc-9vwb2"] Dec 03 12:46:55 crc kubenswrapper[5052]: I1203 12:46:55.308445 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/fa1c223d-82fa-488c-ac48-e88bce36a839-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:55 crc kubenswrapper[5052]: I1203 12:46:55.410048 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovn-northd-tls-certs\" (UniqueName: \"kubernetes.io/secret/f8ab3054-5998-40c1-852f-be0dbaf022e5-ovn-northd-tls-certs\") pod \"ovn-northd-0\" (UID: \"f8ab3054-5998-40c1-852f-be0dbaf022e5\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:46:55 crc kubenswrapper[5052]: E1203 12:46:55.410232 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/cert-ovnnorthd-ovndbs: object "openstack-kuttl-tests"/"cert-ovnnorthd-ovndbs" not registered Dec 03 12:46:55 crc kubenswrapper[5052]: E1203 12:46:55.410580 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f8ab3054-5998-40c1-852f-be0dbaf022e5-ovn-northd-tls-certs podName:f8ab3054-5998-40c1-852f-be0dbaf022e5 nodeName:}" failed. No retries permitted until 2025-12-03 12:47:27.410560763 +0000 UTC m=+5237.998802527 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "ovn-northd-tls-certs" (UniqueName: "kubernetes.io/secret/f8ab3054-5998-40c1-852f-be0dbaf022e5-ovn-northd-tls-certs") pod "ovn-northd-0" (UID: "f8ab3054-5998-40c1-852f-be0dbaf022e5") : object "openstack-kuttl-tests"/"cert-ovnnorthd-ovndbs" not registered Dec 03 12:46:55 crc kubenswrapper[5052]: I1203 12:46:55.651160 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/placement-6d46dc454-rpjwz"] Dec 03 12:46:55 crc kubenswrapper[5052]: I1203 12:46:55.651466 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/placement-6d46dc454-rpjwz" podUID="8fbc67c1-6c51-4c53-85c0-1cf3fa2d5fc2" containerName="placement-log" containerID="cri-o://4c37d110de42cbaa1349c97632b6d688bda0d1d1a7cefd193e59a9fbe693828d" gracePeriod=30 Dec 03 12:46:55 crc kubenswrapper[5052]: I1203 12:46:55.651642 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/placement-6d46dc454-rpjwz" podUID="8fbc67c1-6c51-4c53-85c0-1cf3fa2d5fc2" containerName="placement-api" containerID="cri-o://5a171d88072795e803a5d56decdf74b34dfde4756856ab4c457fb64a1101b16f" gracePeriod=30 Dec 03 12:46:55 crc kubenswrapper[5052]: I1203 12:46:55.688855 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fa1c223d-82fa-488c-ac48-e88bce36a839" path="/var/lib/kubelet/pods/fa1c223d-82fa-488c-ac48-e88bce36a839/volumes" Dec 03 12:46:55 crc kubenswrapper[5052]: I1203 12:46:55.729737 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/placement-74746c7f4-h6sx8"] Dec 03 12:46:55 crc kubenswrapper[5052]: I1203 12:46:55.733143 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-74746c7f4-h6sx8" Dec 03 12:46:55 crc kubenswrapper[5052]: I1203 12:46:55.734324 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:46:55 crc kubenswrapper[5052]: E1203 12:46:55.788237 5052 cadvisor_stats_provider.go:516] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8fbc67c1_6c51_4c53_85c0_1cf3fa2d5fc2.slice/crio-4c37d110de42cbaa1349c97632b6d688bda0d1d1a7cefd193e59a9fbe693828d.scope\": RecentStats: unable to find data in memory cache]" Dec 03 12:46:55 crc kubenswrapper[5052]: I1203 12:46:55.798341 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/placement-74746c7f4-h6sx8"] Dec 03 12:46:55 crc kubenswrapper[5052]: I1203 12:46:55.926195 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/bf8b9b9c-b8f9-4c93-8112-43913950fcdf-combined-ca-bundle\") pod \"bf8b9b9c-b8f9-4c93-8112-43913950fcdf\" (UID: \"bf8b9b9c-b8f9-4c93-8112-43913950fcdf\") " Dec 03 12:46:55 crc kubenswrapper[5052]: I1203 12:46:55.926807 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/bf8b9b9c-b8f9-4c93-8112-43913950fcdf-scripts\") pod \"bf8b9b9c-b8f9-4c93-8112-43913950fcdf\" (UID: \"bf8b9b9c-b8f9-4c93-8112-43913950fcdf\") " Dec 03 12:46:55 crc kubenswrapper[5052]: I1203 12:46:55.926884 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/bf8b9b9c-b8f9-4c93-8112-43913950fcdf-logs\") pod \"bf8b9b9c-b8f9-4c93-8112-43913950fcdf\" (UID: \"bf8b9b9c-b8f9-4c93-8112-43913950fcdf\") " Dec 03 12:46:55 crc kubenswrapper[5052]: I1203 12:46:55.926998 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"glance\" (UniqueName: \"kubernetes.io/local-volume/local-storage18-crc\") pod \"bf8b9b9c-b8f9-4c93-8112-43913950fcdf\" (UID: \"bf8b9b9c-b8f9-4c93-8112-43913950fcdf\") " Dec 03 12:46:55 crc kubenswrapper[5052]: I1203 12:46:55.927147 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hzlsc\" (UniqueName: \"kubernetes.io/projected/bf8b9b9c-b8f9-4c93-8112-43913950fcdf-kube-api-access-hzlsc\") pod \"bf8b9b9c-b8f9-4c93-8112-43913950fcdf\" (UID: \"bf8b9b9c-b8f9-4c93-8112-43913950fcdf\") " Dec 03 12:46:55 crc kubenswrapper[5052]: I1203 12:46:55.927269 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/bf8b9b9c-b8f9-4c93-8112-43913950fcdf-config-data\") pod \"bf8b9b9c-b8f9-4c93-8112-43913950fcdf\" (UID: \"bf8b9b9c-b8f9-4c93-8112-43913950fcdf\") " Dec 03 12:46:55 crc kubenswrapper[5052]: I1203 12:46:55.927342 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/bf8b9b9c-b8f9-4c93-8112-43913950fcdf-httpd-run\") pod \"bf8b9b9c-b8f9-4c93-8112-43913950fcdf\" (UID: \"bf8b9b9c-b8f9-4c93-8112-43913950fcdf\") " Dec 03 12:46:55 crc kubenswrapper[5052]: I1203 12:46:55.927421 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/bf8b9b9c-b8f9-4c93-8112-43913950fcdf-internal-tls-certs\") pod \"bf8b9b9c-b8f9-4c93-8112-43913950fcdf\" (UID: \"bf8b9b9c-b8f9-4c93-8112-43913950fcdf\") " Dec 03 12:46:55 crc kubenswrapper[5052]: I1203 12:46:55.927842 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/5201872c-fde1-49d6-9784-6a7f54497990-config-data\") pod \"placement-74746c7f4-h6sx8\" (UID: \"5201872c-fde1-49d6-9784-6a7f54497990\") " pod="openstack-kuttl-tests/placement-74746c7f4-h6sx8" Dec 03 12:46:55 crc kubenswrapper[5052]: I1203 12:46:55.927912 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/5201872c-fde1-49d6-9784-6a7f54497990-public-tls-certs\") pod \"placement-74746c7f4-h6sx8\" (UID: \"5201872c-fde1-49d6-9784-6a7f54497990\") " pod="openstack-kuttl-tests/placement-74746c7f4-h6sx8" Dec 03 12:46:55 crc kubenswrapper[5052]: I1203 12:46:55.927979 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/5201872c-fde1-49d6-9784-6a7f54497990-scripts\") pod \"placement-74746c7f4-h6sx8\" (UID: \"5201872c-fde1-49d6-9784-6a7f54497990\") " pod="openstack-kuttl-tests/placement-74746c7f4-h6sx8" Dec 03 12:46:55 crc kubenswrapper[5052]: I1203 12:46:55.928070 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5201872c-fde1-49d6-9784-6a7f54497990-combined-ca-bundle\") pod \"placement-74746c7f4-h6sx8\" (UID: \"5201872c-fde1-49d6-9784-6a7f54497990\") " pod="openstack-kuttl-tests/placement-74746c7f4-h6sx8" Dec 03 12:46:55 crc kubenswrapper[5052]: I1203 12:46:55.928145 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/5201872c-fde1-49d6-9784-6a7f54497990-internal-tls-certs\") pod \"placement-74746c7f4-h6sx8\" (UID: \"5201872c-fde1-49d6-9784-6a7f54497990\") " pod="openstack-kuttl-tests/placement-74746c7f4-h6sx8" Dec 03 12:46:55 crc kubenswrapper[5052]: I1203 12:46:55.928218 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6r529\" (UniqueName: \"kubernetes.io/projected/5201872c-fde1-49d6-9784-6a7f54497990-kube-api-access-6r529\") pod \"placement-74746c7f4-h6sx8\" (UID: \"5201872c-fde1-49d6-9784-6a7f54497990\") " pod="openstack-kuttl-tests/placement-74746c7f4-h6sx8" Dec 03 12:46:55 crc kubenswrapper[5052]: I1203 12:46:55.928380 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/5201872c-fde1-49d6-9784-6a7f54497990-logs\") pod \"placement-74746c7f4-h6sx8\" (UID: \"5201872c-fde1-49d6-9784-6a7f54497990\") " pod="openstack-kuttl-tests/placement-74746c7f4-h6sx8" Dec 03 12:46:55 crc kubenswrapper[5052]: I1203 12:46:55.948952 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bf8b9b9c-b8f9-4c93-8112-43913950fcdf-httpd-run" (OuterVolumeSpecName: "httpd-run") pod "bf8b9b9c-b8f9-4c93-8112-43913950fcdf" (UID: "bf8b9b9c-b8f9-4c93-8112-43913950fcdf"). InnerVolumeSpecName "httpd-run". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:46:55 crc kubenswrapper[5052]: I1203 12:46:55.949222 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bf8b9b9c-b8f9-4c93-8112-43913950fcdf-logs" (OuterVolumeSpecName: "logs") pod "bf8b9b9c-b8f9-4c93-8112-43913950fcdf" (UID: "bf8b9b9c-b8f9-4c93-8112-43913950fcdf"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:46:56 crc kubenswrapper[5052]: I1203 12:46:56.044800 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/5201872c-fde1-49d6-9784-6a7f54497990-config-data\") pod \"placement-74746c7f4-h6sx8\" (UID: \"5201872c-fde1-49d6-9784-6a7f54497990\") " pod="openstack-kuttl-tests/placement-74746c7f4-h6sx8" Dec 03 12:46:56 crc kubenswrapper[5052]: I1203 12:46:56.044868 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/5201872c-fde1-49d6-9784-6a7f54497990-public-tls-certs\") pod \"placement-74746c7f4-h6sx8\" (UID: \"5201872c-fde1-49d6-9784-6a7f54497990\") " pod="openstack-kuttl-tests/placement-74746c7f4-h6sx8" Dec 03 12:46:56 crc kubenswrapper[5052]: I1203 12:46:56.044896 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/5201872c-fde1-49d6-9784-6a7f54497990-scripts\") pod \"placement-74746c7f4-h6sx8\" (UID: \"5201872c-fde1-49d6-9784-6a7f54497990\") " pod="openstack-kuttl-tests/placement-74746c7f4-h6sx8" Dec 03 12:46:56 crc kubenswrapper[5052]: I1203 12:46:56.044959 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5201872c-fde1-49d6-9784-6a7f54497990-combined-ca-bundle\") pod \"placement-74746c7f4-h6sx8\" (UID: \"5201872c-fde1-49d6-9784-6a7f54497990\") " pod="openstack-kuttl-tests/placement-74746c7f4-h6sx8" Dec 03 12:46:56 crc kubenswrapper[5052]: I1203 12:46:56.045002 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/5201872c-fde1-49d6-9784-6a7f54497990-internal-tls-certs\") pod \"placement-74746c7f4-h6sx8\" (UID: \"5201872c-fde1-49d6-9784-6a7f54497990\") " pod="openstack-kuttl-tests/placement-74746c7f4-h6sx8" Dec 03 12:46:56 crc kubenswrapper[5052]: I1203 12:46:56.045028 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-6r529\" (UniqueName: \"kubernetes.io/projected/5201872c-fde1-49d6-9784-6a7f54497990-kube-api-access-6r529\") pod \"placement-74746c7f4-h6sx8\" (UID: \"5201872c-fde1-49d6-9784-6a7f54497990\") " pod="openstack-kuttl-tests/placement-74746c7f4-h6sx8" Dec 03 12:46:56 crc kubenswrapper[5052]: I1203 12:46:56.045138 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/5201872c-fde1-49d6-9784-6a7f54497990-logs\") pod \"placement-74746c7f4-h6sx8\" (UID: \"5201872c-fde1-49d6-9784-6a7f54497990\") " pod="openstack-kuttl-tests/placement-74746c7f4-h6sx8" Dec 03 12:46:56 crc kubenswrapper[5052]: I1203 12:46:56.045366 5052 reconciler_common.go:293] "Volume detached for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/bf8b9b9c-b8f9-4c93-8112-43913950fcdf-httpd-run\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:56 crc kubenswrapper[5052]: I1203 12:46:56.045390 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/bf8b9b9c-b8f9-4c93-8112-43913950fcdf-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:56 crc kubenswrapper[5052]: I1203 12:46:56.045882 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/5201872c-fde1-49d6-9784-6a7f54497990-logs\") pod \"placement-74746c7f4-h6sx8\" (UID: \"5201872c-fde1-49d6-9784-6a7f54497990\") " pod="openstack-kuttl-tests/placement-74746c7f4-h6sx8" Dec 03 12:46:56 crc kubenswrapper[5052]: I1203 12:46:56.130573 5052 generic.go:334] "Generic (PLEG): container finished" podID="8fbc67c1-6c51-4c53-85c0-1cf3fa2d5fc2" containerID="4c37d110de42cbaa1349c97632b6d688bda0d1d1a7cefd193e59a9fbe693828d" exitCode=143 Dec 03 12:46:56 crc kubenswrapper[5052]: I1203 12:46:56.130665 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-6d46dc454-rpjwz" event={"ID":"8fbc67c1-6c51-4c53-85c0-1cf3fa2d5fc2","Type":"ContainerDied","Data":"4c37d110de42cbaa1349c97632b6d688bda0d1d1a7cefd193e59a9fbe693828d"} Dec 03 12:46:56 crc kubenswrapper[5052]: I1203 12:46:56.135690 5052 generic.go:334] "Generic (PLEG): container finished" podID="bf8b9b9c-b8f9-4c93-8112-43913950fcdf" containerID="47769437fc8e2335ee64f20b3f593a902f20b71b2bb53ebf37e7accf0cc2c14e" exitCode=0 Dec 03 12:46:56 crc kubenswrapper[5052]: I1203 12:46:56.135725 5052 generic.go:334] "Generic (PLEG): container finished" podID="bf8b9b9c-b8f9-4c93-8112-43913950fcdf" containerID="acb5b85ad6f2dd6eeace8cadc076149f4f2d58d20643d30b4d92c04ad6dcd41d" exitCode=143 Dec 03 12:46:56 crc kubenswrapper[5052]: I1203 12:46:56.135838 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:46:56 crc kubenswrapper[5052]: I1203 12:46:56.136039 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"bf8b9b9c-b8f9-4c93-8112-43913950fcdf","Type":"ContainerDied","Data":"47769437fc8e2335ee64f20b3f593a902f20b71b2bb53ebf37e7accf0cc2c14e"} Dec 03 12:46:56 crc kubenswrapper[5052]: I1203 12:46:56.136089 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"bf8b9b9c-b8f9-4c93-8112-43913950fcdf","Type":"ContainerDied","Data":"acb5b85ad6f2dd6eeace8cadc076149f4f2d58d20643d30b4d92c04ad6dcd41d"} Dec 03 12:46:56 crc kubenswrapper[5052]: I1203 12:46:56.136105 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"bf8b9b9c-b8f9-4c93-8112-43913950fcdf","Type":"ContainerDied","Data":"1bd9dac09969a248dc0fa8abc9681c81993311f2ce27d7a6014d719e9e4dbd38"} Dec 03 12:46:56 crc kubenswrapper[5052]: I1203 12:46:56.136124 5052 scope.go:117] "RemoveContainer" containerID="47769437fc8e2335ee64f20b3f593a902f20b71b2bb53ebf37e7accf0cc2c14e" Dec 03 12:46:56 crc kubenswrapper[5052]: I1203 12:46:56.136679 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/keystone-6c4f97778c-d7wtk" Dec 03 12:46:56 crc kubenswrapper[5052]: I1203 12:46:56.290296 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/placement-6d46dc454-rpjwz" podUID="8fbc67c1-6c51-4c53-85c0-1cf3fa2d5fc2" containerName="placement-log" probeResult="failure" output="Get \"https://10.217.0.101:8778/\": read tcp 10.217.0.2:39744->10.217.0.101:8778: read: connection reset by peer" Dec 03 12:46:56 crc kubenswrapper[5052]: I1203 12:46:56.290304 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/placement-6d46dc454-rpjwz" podUID="8fbc67c1-6c51-4c53-85c0-1cf3fa2d5fc2" containerName="placement-api" probeResult="failure" output="Get \"https://10.217.0.101:8778/\": read tcp 10.217.0.2:39738->10.217.0.101:8778: read: connection reset by peer" Dec 03 12:46:56 crc kubenswrapper[5052]: I1203 12:46:56.325181 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bf8b9b9c-b8f9-4c93-8112-43913950fcdf-scripts" (OuterVolumeSpecName: "scripts") pod "bf8b9b9c-b8f9-4c93-8112-43913950fcdf" (UID: "bf8b9b9c-b8f9-4c93-8112-43913950fcdf"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:46:56 crc kubenswrapper[5052]: I1203 12:46:56.337084 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage18-crc" (OuterVolumeSpecName: "glance") pod "bf8b9b9c-b8f9-4c93-8112-43913950fcdf" (UID: "bf8b9b9c-b8f9-4c93-8112-43913950fcdf"). InnerVolumeSpecName "local-storage18-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 12:46:56 crc kubenswrapper[5052]: I1203 12:46:56.337604 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bf8b9b9c-b8f9-4c93-8112-43913950fcdf-kube-api-access-hzlsc" (OuterVolumeSpecName: "kube-api-access-hzlsc") pod "bf8b9b9c-b8f9-4c93-8112-43913950fcdf" (UID: "bf8b9b9c-b8f9-4c93-8112-43913950fcdf"). InnerVolumeSpecName "kube-api-access-hzlsc". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:46:56 crc kubenswrapper[5052]: I1203 12:46:56.337754 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/5201872c-fde1-49d6-9784-6a7f54497990-scripts\") pod \"placement-74746c7f4-h6sx8\" (UID: \"5201872c-fde1-49d6-9784-6a7f54497990\") " pod="openstack-kuttl-tests/placement-74746c7f4-h6sx8" Dec 03 12:46:56 crc kubenswrapper[5052]: I1203 12:46:56.339100 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/5201872c-fde1-49d6-9784-6a7f54497990-config-data\") pod \"placement-74746c7f4-h6sx8\" (UID: \"5201872c-fde1-49d6-9784-6a7f54497990\") " pod="openstack-kuttl-tests/placement-74746c7f4-h6sx8" Dec 03 12:46:56 crc kubenswrapper[5052]: I1203 12:46:56.339248 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/5201872c-fde1-49d6-9784-6a7f54497990-public-tls-certs\") pod \"placement-74746c7f4-h6sx8\" (UID: \"5201872c-fde1-49d6-9784-6a7f54497990\") " pod="openstack-kuttl-tests/placement-74746c7f4-h6sx8" Dec 03 12:46:56 crc kubenswrapper[5052]: I1203 12:46:56.339754 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-6r529\" (UniqueName: \"kubernetes.io/projected/5201872c-fde1-49d6-9784-6a7f54497990-kube-api-access-6r529\") pod \"placement-74746c7f4-h6sx8\" (UID: \"5201872c-fde1-49d6-9784-6a7f54497990\") " pod="openstack-kuttl-tests/placement-74746c7f4-h6sx8" Dec 03 12:46:56 crc kubenswrapper[5052]: I1203 12:46:56.340168 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5201872c-fde1-49d6-9784-6a7f54497990-combined-ca-bundle\") pod \"placement-74746c7f4-h6sx8\" (UID: \"5201872c-fde1-49d6-9784-6a7f54497990\") " pod="openstack-kuttl-tests/placement-74746c7f4-h6sx8" Dec 03 12:46:56 crc kubenswrapper[5052]: I1203 12:46:56.340311 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/5201872c-fde1-49d6-9784-6a7f54497990-internal-tls-certs\") pod \"placement-74746c7f4-h6sx8\" (UID: \"5201872c-fde1-49d6-9784-6a7f54497990\") " pod="openstack-kuttl-tests/placement-74746c7f4-h6sx8" Dec 03 12:46:56 crc kubenswrapper[5052]: I1203 12:46:56.345349 5052 scope.go:117] "RemoveContainer" containerID="acb5b85ad6f2dd6eeace8cadc076149f4f2d58d20643d30b4d92c04ad6dcd41d" Dec 03 12:46:56 crc kubenswrapper[5052]: I1203 12:46:56.352445 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-hzlsc\" (UniqueName: \"kubernetes.io/projected/bf8b9b9c-b8f9-4c93-8112-43913950fcdf-kube-api-access-hzlsc\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:56 crc kubenswrapper[5052]: I1203 12:46:56.352480 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/bf8b9b9c-b8f9-4c93-8112-43913950fcdf-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:56 crc kubenswrapper[5052]: I1203 12:46:56.352520 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage18-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage18-crc\") on node \"crc\" " Dec 03 12:46:56 crc kubenswrapper[5052]: I1203 12:46:56.353979 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-74746c7f4-h6sx8" Dec 03 12:46:56 crc kubenswrapper[5052]: I1203 12:46:56.374113 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bf8b9b9c-b8f9-4c93-8112-43913950fcdf-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "bf8b9b9c-b8f9-4c93-8112-43913950fcdf" (UID: "bf8b9b9c-b8f9-4c93-8112-43913950fcdf"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:46:56 crc kubenswrapper[5052]: I1203 12:46:56.382078 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage18-crc" (UniqueName: "kubernetes.io/local-volume/local-storage18-crc") on node "crc" Dec 03 12:46:56 crc kubenswrapper[5052]: I1203 12:46:56.400682 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bf8b9b9c-b8f9-4c93-8112-43913950fcdf-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "bf8b9b9c-b8f9-4c93-8112-43913950fcdf" (UID: "bf8b9b9c-b8f9-4c93-8112-43913950fcdf"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:46:56 crc kubenswrapper[5052]: I1203 12:46:56.415843 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bf8b9b9c-b8f9-4c93-8112-43913950fcdf-config-data" (OuterVolumeSpecName: "config-data") pod "bf8b9b9c-b8f9-4c93-8112-43913950fcdf" (UID: "bf8b9b9c-b8f9-4c93-8112-43913950fcdf"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:46:56 crc kubenswrapper[5052]: I1203 12:46:56.454595 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/bf8b9b9c-b8f9-4c93-8112-43913950fcdf-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:56 crc kubenswrapper[5052]: I1203 12:46:56.454635 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/bf8b9b9c-b8f9-4c93-8112-43913950fcdf-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:56 crc kubenswrapper[5052]: I1203 12:46:56.454650 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/bf8b9b9c-b8f9-4c93-8112-43913950fcdf-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:56 crc kubenswrapper[5052]: I1203 12:46:56.454661 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage18-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage18-crc\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:56 crc kubenswrapper[5052]: I1203 12:46:56.668640 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:46:56 crc kubenswrapper[5052]: I1203 12:46:56.673671 5052 scope.go:117] "RemoveContainer" containerID="47769437fc8e2335ee64f20b3f593a902f20b71b2bb53ebf37e7accf0cc2c14e" Dec 03 12:46:56 crc kubenswrapper[5052]: E1203 12:46:56.677418 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"47769437fc8e2335ee64f20b3f593a902f20b71b2bb53ebf37e7accf0cc2c14e\": container with ID starting with 47769437fc8e2335ee64f20b3f593a902f20b71b2bb53ebf37e7accf0cc2c14e not found: ID does not exist" containerID="47769437fc8e2335ee64f20b3f593a902f20b71b2bb53ebf37e7accf0cc2c14e" Dec 03 12:46:56 crc kubenswrapper[5052]: I1203 12:46:56.677490 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"47769437fc8e2335ee64f20b3f593a902f20b71b2bb53ebf37e7accf0cc2c14e"} err="failed to get container status \"47769437fc8e2335ee64f20b3f593a902f20b71b2bb53ebf37e7accf0cc2c14e\": rpc error: code = NotFound desc = could not find container \"47769437fc8e2335ee64f20b3f593a902f20b71b2bb53ebf37e7accf0cc2c14e\": container with ID starting with 47769437fc8e2335ee64f20b3f593a902f20b71b2bb53ebf37e7accf0cc2c14e not found: ID does not exist" Dec 03 12:46:56 crc kubenswrapper[5052]: I1203 12:46:56.677643 5052 scope.go:117] "RemoveContainer" containerID="acb5b85ad6f2dd6eeace8cadc076149f4f2d58d20643d30b4d92c04ad6dcd41d" Dec 03 12:46:56 crc kubenswrapper[5052]: E1203 12:46:56.679717 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"acb5b85ad6f2dd6eeace8cadc076149f4f2d58d20643d30b4d92c04ad6dcd41d\": container with ID starting with acb5b85ad6f2dd6eeace8cadc076149f4f2d58d20643d30b4d92c04ad6dcd41d not found: ID does not exist" containerID="acb5b85ad6f2dd6eeace8cadc076149f4f2d58d20643d30b4d92c04ad6dcd41d" Dec 03 12:46:56 crc kubenswrapper[5052]: I1203 12:46:56.679760 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"acb5b85ad6f2dd6eeace8cadc076149f4f2d58d20643d30b4d92c04ad6dcd41d"} err="failed to get container status \"acb5b85ad6f2dd6eeace8cadc076149f4f2d58d20643d30b4d92c04ad6dcd41d\": rpc error: code = NotFound desc = could not find container \"acb5b85ad6f2dd6eeace8cadc076149f4f2d58d20643d30b4d92c04ad6dcd41d\": container with ID starting with acb5b85ad6f2dd6eeace8cadc076149f4f2d58d20643d30b4d92c04ad6dcd41d not found: ID does not exist" Dec 03 12:46:56 crc kubenswrapper[5052]: I1203 12:46:56.679787 5052 scope.go:117] "RemoveContainer" containerID="47769437fc8e2335ee64f20b3f593a902f20b71b2bb53ebf37e7accf0cc2c14e" Dec 03 12:46:56 crc kubenswrapper[5052]: I1203 12:46:56.691743 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"47769437fc8e2335ee64f20b3f593a902f20b71b2bb53ebf37e7accf0cc2c14e"} err="failed to get container status \"47769437fc8e2335ee64f20b3f593a902f20b71b2bb53ebf37e7accf0cc2c14e\": rpc error: code = NotFound desc = could not find container \"47769437fc8e2335ee64f20b3f593a902f20b71b2bb53ebf37e7accf0cc2c14e\": container with ID starting with 47769437fc8e2335ee64f20b3f593a902f20b71b2bb53ebf37e7accf0cc2c14e not found: ID does not exist" Dec 03 12:46:56 crc kubenswrapper[5052]: I1203 12:46:56.691785 5052 scope.go:117] "RemoveContainer" containerID="acb5b85ad6f2dd6eeace8cadc076149f4f2d58d20643d30b4d92c04ad6dcd41d" Dec 03 12:46:56 crc kubenswrapper[5052]: I1203 12:46:56.692303 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"acb5b85ad6f2dd6eeace8cadc076149f4f2d58d20643d30b4d92c04ad6dcd41d"} err="failed to get container status \"acb5b85ad6f2dd6eeace8cadc076149f4f2d58d20643d30b4d92c04ad6dcd41d\": rpc error: code = NotFound desc = could not find container \"acb5b85ad6f2dd6eeace8cadc076149f4f2d58d20643d30b4d92c04ad6dcd41d\": container with ID starting with acb5b85ad6f2dd6eeace8cadc076149f4f2d58d20643d30b4d92c04ad6dcd41d not found: ID does not exist" Dec 03 12:46:56 crc kubenswrapper[5052]: I1203 12:46:56.746792 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:46:56 crc kubenswrapper[5052]: I1203 12:46:56.760390 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:46:56 crc kubenswrapper[5052]: E1203 12:46:56.761423 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="bf8b9b9c-b8f9-4c93-8112-43913950fcdf" containerName="glance-httpd" Dec 03 12:46:56 crc kubenswrapper[5052]: I1203 12:46:56.761457 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="bf8b9b9c-b8f9-4c93-8112-43913950fcdf" containerName="glance-httpd" Dec 03 12:46:56 crc kubenswrapper[5052]: E1203 12:46:56.761472 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="bf8b9b9c-b8f9-4c93-8112-43913950fcdf" containerName="glance-log" Dec 03 12:46:56 crc kubenswrapper[5052]: I1203 12:46:56.761479 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="bf8b9b9c-b8f9-4c93-8112-43913950fcdf" containerName="glance-log" Dec 03 12:46:56 crc kubenswrapper[5052]: I1203 12:46:56.761776 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="bf8b9b9c-b8f9-4c93-8112-43913950fcdf" containerName="glance-httpd" Dec 03 12:46:56 crc kubenswrapper[5052]: I1203 12:46:56.761809 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="bf8b9b9c-b8f9-4c93-8112-43913950fcdf" containerName="glance-log" Dec 03 12:46:56 crc kubenswrapper[5052]: I1203 12:46:56.763305 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:46:56 crc kubenswrapper[5052]: I1203 12:46:56.766832 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-default-internal-config-data" Dec 03 12:46:56 crc kubenswrapper[5052]: I1203 12:46:56.766863 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-glance-default-internal-svc" Dec 03 12:46:56 crc kubenswrapper[5052]: I1203 12:46:56.770919 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:46:56 crc kubenswrapper[5052]: I1203 12:46:56.865290 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/ffe02c07-7dca-4dc6-b9a2-71096980da67-scripts\") pod \"glance-default-internal-api-0\" (UID: \"ffe02c07-7dca-4dc6-b9a2-71096980da67\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:46:56 crc kubenswrapper[5052]: I1203 12:46:56.865367 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ffe02c07-7dca-4dc6-b9a2-71096980da67-combined-ca-bundle\") pod \"glance-default-internal-api-0\" (UID: \"ffe02c07-7dca-4dc6-b9a2-71096980da67\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:46:56 crc kubenswrapper[5052]: I1203 12:46:56.865422 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/ffe02c07-7dca-4dc6-b9a2-71096980da67-config-data\") pod \"glance-default-internal-api-0\" (UID: \"ffe02c07-7dca-4dc6-b9a2-71096980da67\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:46:56 crc kubenswrapper[5052]: I1203 12:46:56.865461 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage18-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage18-crc\") pod \"glance-default-internal-api-0\" (UID: \"ffe02c07-7dca-4dc6-b9a2-71096980da67\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:46:56 crc kubenswrapper[5052]: I1203 12:46:56.865555 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/ffe02c07-7dca-4dc6-b9a2-71096980da67-internal-tls-certs\") pod \"glance-default-internal-api-0\" (UID: \"ffe02c07-7dca-4dc6-b9a2-71096980da67\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:46:56 crc kubenswrapper[5052]: I1203 12:46:56.865626 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/ffe02c07-7dca-4dc6-b9a2-71096980da67-logs\") pod \"glance-default-internal-api-0\" (UID: \"ffe02c07-7dca-4dc6-b9a2-71096980da67\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:46:56 crc kubenswrapper[5052]: I1203 12:46:56.865727 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lnjlp\" (UniqueName: \"kubernetes.io/projected/ffe02c07-7dca-4dc6-b9a2-71096980da67-kube-api-access-lnjlp\") pod \"glance-default-internal-api-0\" (UID: \"ffe02c07-7dca-4dc6-b9a2-71096980da67\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:46:56 crc kubenswrapper[5052]: I1203 12:46:56.865839 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/ffe02c07-7dca-4dc6-b9a2-71096980da67-httpd-run\") pod \"glance-default-internal-api-0\" (UID: \"ffe02c07-7dca-4dc6-b9a2-71096980da67\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:46:56 crc kubenswrapper[5052]: I1203 12:46:56.872761 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-6d46dc454-rpjwz" Dec 03 12:46:56 crc kubenswrapper[5052]: I1203 12:46:56.969238 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/8fbc67c1-6c51-4c53-85c0-1cf3fa2d5fc2-internal-tls-certs\") pod \"8fbc67c1-6c51-4c53-85c0-1cf3fa2d5fc2\" (UID: \"8fbc67c1-6c51-4c53-85c0-1cf3fa2d5fc2\") " Dec 03 12:46:56 crc kubenswrapper[5052]: I1203 12:46:56.969312 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/8fbc67c1-6c51-4c53-85c0-1cf3fa2d5fc2-config-data\") pod \"8fbc67c1-6c51-4c53-85c0-1cf3fa2d5fc2\" (UID: \"8fbc67c1-6c51-4c53-85c0-1cf3fa2d5fc2\") " Dec 03 12:46:56 crc kubenswrapper[5052]: I1203 12:46:56.969357 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/8fbc67c1-6c51-4c53-85c0-1cf3fa2d5fc2-logs\") pod \"8fbc67c1-6c51-4c53-85c0-1cf3fa2d5fc2\" (UID: \"8fbc67c1-6c51-4c53-85c0-1cf3fa2d5fc2\") " Dec 03 12:46:56 crc kubenswrapper[5052]: I1203 12:46:56.969394 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jndtz\" (UniqueName: \"kubernetes.io/projected/8fbc67c1-6c51-4c53-85c0-1cf3fa2d5fc2-kube-api-access-jndtz\") pod \"8fbc67c1-6c51-4c53-85c0-1cf3fa2d5fc2\" (UID: \"8fbc67c1-6c51-4c53-85c0-1cf3fa2d5fc2\") " Dec 03 12:46:56 crc kubenswrapper[5052]: I1203 12:46:56.969437 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/8fbc67c1-6c51-4c53-85c0-1cf3fa2d5fc2-public-tls-certs\") pod \"8fbc67c1-6c51-4c53-85c0-1cf3fa2d5fc2\" (UID: \"8fbc67c1-6c51-4c53-85c0-1cf3fa2d5fc2\") " Dec 03 12:46:56 crc kubenswrapper[5052]: I1203 12:46:56.969578 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/8fbc67c1-6c51-4c53-85c0-1cf3fa2d5fc2-scripts\") pod \"8fbc67c1-6c51-4c53-85c0-1cf3fa2d5fc2\" (UID: \"8fbc67c1-6c51-4c53-85c0-1cf3fa2d5fc2\") " Dec 03 12:46:56 crc kubenswrapper[5052]: I1203 12:46:56.969618 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8fbc67c1-6c51-4c53-85c0-1cf3fa2d5fc2-combined-ca-bundle\") pod \"8fbc67c1-6c51-4c53-85c0-1cf3fa2d5fc2\" (UID: \"8fbc67c1-6c51-4c53-85c0-1cf3fa2d5fc2\") " Dec 03 12:46:56 crc kubenswrapper[5052]: I1203 12:46:56.969875 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/ffe02c07-7dca-4dc6-b9a2-71096980da67-internal-tls-certs\") pod \"glance-default-internal-api-0\" (UID: \"ffe02c07-7dca-4dc6-b9a2-71096980da67\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:46:56 crc kubenswrapper[5052]: I1203 12:46:56.969930 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/ffe02c07-7dca-4dc6-b9a2-71096980da67-logs\") pod \"glance-default-internal-api-0\" (UID: \"ffe02c07-7dca-4dc6-b9a2-71096980da67\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:46:56 crc kubenswrapper[5052]: I1203 12:46:56.969982 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-lnjlp\" (UniqueName: \"kubernetes.io/projected/ffe02c07-7dca-4dc6-b9a2-71096980da67-kube-api-access-lnjlp\") pod \"glance-default-internal-api-0\" (UID: \"ffe02c07-7dca-4dc6-b9a2-71096980da67\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:46:56 crc kubenswrapper[5052]: I1203 12:46:56.970031 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/ffe02c07-7dca-4dc6-b9a2-71096980da67-httpd-run\") pod \"glance-default-internal-api-0\" (UID: \"ffe02c07-7dca-4dc6-b9a2-71096980da67\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:46:56 crc kubenswrapper[5052]: I1203 12:46:56.970125 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/ffe02c07-7dca-4dc6-b9a2-71096980da67-scripts\") pod \"glance-default-internal-api-0\" (UID: \"ffe02c07-7dca-4dc6-b9a2-71096980da67\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:46:56 crc kubenswrapper[5052]: I1203 12:46:56.970150 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ffe02c07-7dca-4dc6-b9a2-71096980da67-combined-ca-bundle\") pod \"glance-default-internal-api-0\" (UID: \"ffe02c07-7dca-4dc6-b9a2-71096980da67\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:46:56 crc kubenswrapper[5052]: I1203 12:46:56.970201 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/ffe02c07-7dca-4dc6-b9a2-71096980da67-config-data\") pod \"glance-default-internal-api-0\" (UID: \"ffe02c07-7dca-4dc6-b9a2-71096980da67\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:46:56 crc kubenswrapper[5052]: I1203 12:46:56.970226 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage18-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage18-crc\") pod \"glance-default-internal-api-0\" (UID: \"ffe02c07-7dca-4dc6-b9a2-71096980da67\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:46:56 crc kubenswrapper[5052]: I1203 12:46:56.970425 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage18-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage18-crc\") pod \"glance-default-internal-api-0\" (UID: \"ffe02c07-7dca-4dc6-b9a2-71096980da67\") device mount path \"/mnt/openstack/pv18\"" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:46:56 crc kubenswrapper[5052]: I1203 12:46:56.975793 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8fbc67c1-6c51-4c53-85c0-1cf3fa2d5fc2-logs" (OuterVolumeSpecName: "logs") pod "8fbc67c1-6c51-4c53-85c0-1cf3fa2d5fc2" (UID: "8fbc67c1-6c51-4c53-85c0-1cf3fa2d5fc2"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:46:56 crc kubenswrapper[5052]: I1203 12:46:56.976998 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/ffe02c07-7dca-4dc6-b9a2-71096980da67-httpd-run\") pod \"glance-default-internal-api-0\" (UID: \"ffe02c07-7dca-4dc6-b9a2-71096980da67\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:46:56 crc kubenswrapper[5052]: I1203 12:46:56.977426 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/ffe02c07-7dca-4dc6-b9a2-71096980da67-logs\") pod \"glance-default-internal-api-0\" (UID: \"ffe02c07-7dca-4dc6-b9a2-71096980da67\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:46:56 crc kubenswrapper[5052]: I1203 12:46:56.979034 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8fbc67c1-6c51-4c53-85c0-1cf3fa2d5fc2-kube-api-access-jndtz" (OuterVolumeSpecName: "kube-api-access-jndtz") pod "8fbc67c1-6c51-4c53-85c0-1cf3fa2d5fc2" (UID: "8fbc67c1-6c51-4c53-85c0-1cf3fa2d5fc2"). InnerVolumeSpecName "kube-api-access-jndtz". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:46:56 crc kubenswrapper[5052]: W1203 12:46:56.987290 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5201872c_fde1_49d6_9784_6a7f54497990.slice/crio-eafe73590de0755ec90eef2cf9741c8d072e54f8dcd15b0fdadbd3a1a6023c72 WatchSource:0}: Error finding container eafe73590de0755ec90eef2cf9741c8d072e54f8dcd15b0fdadbd3a1a6023c72: Status 404 returned error can't find the container with id eafe73590de0755ec90eef2cf9741c8d072e54f8dcd15b0fdadbd3a1a6023c72 Dec 03 12:46:56 crc kubenswrapper[5052]: I1203 12:46:56.988783 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8fbc67c1-6c51-4c53-85c0-1cf3fa2d5fc2-scripts" (OuterVolumeSpecName: "scripts") pod "8fbc67c1-6c51-4c53-85c0-1cf3fa2d5fc2" (UID: "8fbc67c1-6c51-4c53-85c0-1cf3fa2d5fc2"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:46:57 crc kubenswrapper[5052]: I1203 12:46:56.991270 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/placement-74746c7f4-h6sx8"] Dec 03 12:46:57 crc kubenswrapper[5052]: I1203 12:46:56.996136 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/ffe02c07-7dca-4dc6-b9a2-71096980da67-internal-tls-certs\") pod \"glance-default-internal-api-0\" (UID: \"ffe02c07-7dca-4dc6-b9a2-71096980da67\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:46:57 crc kubenswrapper[5052]: I1203 12:46:56.996800 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/ffe02c07-7dca-4dc6-b9a2-71096980da67-config-data\") pod \"glance-default-internal-api-0\" (UID: \"ffe02c07-7dca-4dc6-b9a2-71096980da67\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:46:57 crc kubenswrapper[5052]: I1203 12:46:56.999465 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ffe02c07-7dca-4dc6-b9a2-71096980da67-combined-ca-bundle\") pod \"glance-default-internal-api-0\" (UID: \"ffe02c07-7dca-4dc6-b9a2-71096980da67\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:46:57 crc kubenswrapper[5052]: I1203 12:46:57.003014 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-lnjlp\" (UniqueName: \"kubernetes.io/projected/ffe02c07-7dca-4dc6-b9a2-71096980da67-kube-api-access-lnjlp\") pod \"glance-default-internal-api-0\" (UID: \"ffe02c07-7dca-4dc6-b9a2-71096980da67\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:46:57 crc kubenswrapper[5052]: I1203 12:46:57.003328 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/ffe02c07-7dca-4dc6-b9a2-71096980da67-scripts\") pod \"glance-default-internal-api-0\" (UID: \"ffe02c07-7dca-4dc6-b9a2-71096980da67\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:46:57 crc kubenswrapper[5052]: I1203 12:46:57.026040 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage18-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage18-crc\") pod \"glance-default-internal-api-0\" (UID: \"ffe02c07-7dca-4dc6-b9a2-71096980da67\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:46:57 crc kubenswrapper[5052]: I1203 12:46:57.091968 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8fbc67c1-6c51-4c53-85c0-1cf3fa2d5fc2-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "8fbc67c1-6c51-4c53-85c0-1cf3fa2d5fc2" (UID: "8fbc67c1-6c51-4c53-85c0-1cf3fa2d5fc2"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:46:57 crc kubenswrapper[5052]: I1203 12:46:57.095878 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8fbc67c1-6c51-4c53-85c0-1cf3fa2d5fc2-combined-ca-bundle\") pod \"8fbc67c1-6c51-4c53-85c0-1cf3fa2d5fc2\" (UID: \"8fbc67c1-6c51-4c53-85c0-1cf3fa2d5fc2\") " Dec 03 12:46:57 crc kubenswrapper[5052]: W1203 12:46:57.096719 5052 empty_dir.go:500] Warning: Unmount skipped because path does not exist: /var/lib/kubelet/pods/8fbc67c1-6c51-4c53-85c0-1cf3fa2d5fc2/volumes/kubernetes.io~secret/combined-ca-bundle Dec 03 12:46:57 crc kubenswrapper[5052]: I1203 12:46:57.096752 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8fbc67c1-6c51-4c53-85c0-1cf3fa2d5fc2-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "8fbc67c1-6c51-4c53-85c0-1cf3fa2d5fc2" (UID: "8fbc67c1-6c51-4c53-85c0-1cf3fa2d5fc2"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:46:57 crc kubenswrapper[5052]: I1203 12:46:57.097142 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/8fbc67c1-6c51-4c53-85c0-1cf3fa2d5fc2-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:57 crc kubenswrapper[5052]: I1203 12:46:57.097169 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-jndtz\" (UniqueName: \"kubernetes.io/projected/8fbc67c1-6c51-4c53-85c0-1cf3fa2d5fc2-kube-api-access-jndtz\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:57 crc kubenswrapper[5052]: I1203 12:46:57.097203 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/8fbc67c1-6c51-4c53-85c0-1cf3fa2d5fc2-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:57 crc kubenswrapper[5052]: I1203 12:46:57.097214 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8fbc67c1-6c51-4c53-85c0-1cf3fa2d5fc2-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:57 crc kubenswrapper[5052]: I1203 12:46:57.117862 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8fbc67c1-6c51-4c53-85c0-1cf3fa2d5fc2-config-data" (OuterVolumeSpecName: "config-data") pod "8fbc67c1-6c51-4c53-85c0-1cf3fa2d5fc2" (UID: "8fbc67c1-6c51-4c53-85c0-1cf3fa2d5fc2"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:46:57 crc kubenswrapper[5052]: I1203 12:46:57.155380 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8fbc67c1-6c51-4c53-85c0-1cf3fa2d5fc2-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "8fbc67c1-6c51-4c53-85c0-1cf3fa2d5fc2" (UID: "8fbc67c1-6c51-4c53-85c0-1cf3fa2d5fc2"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:46:57 crc kubenswrapper[5052]: I1203 12:46:57.156863 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-74746c7f4-h6sx8" event={"ID":"5201872c-fde1-49d6-9784-6a7f54497990","Type":"ContainerStarted","Data":"eafe73590de0755ec90eef2cf9741c8d072e54f8dcd15b0fdadbd3a1a6023c72"} Dec 03 12:46:57 crc kubenswrapper[5052]: I1203 12:46:57.159061 5052 generic.go:334] "Generic (PLEG): container finished" podID="8fbc67c1-6c51-4c53-85c0-1cf3fa2d5fc2" containerID="5a171d88072795e803a5d56decdf74b34dfde4756856ab4c457fb64a1101b16f" exitCode=0 Dec 03 12:46:57 crc kubenswrapper[5052]: I1203 12:46:57.159093 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-6d46dc454-rpjwz" Dec 03 12:46:57 crc kubenswrapper[5052]: I1203 12:46:57.159130 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-6d46dc454-rpjwz" event={"ID":"8fbc67c1-6c51-4c53-85c0-1cf3fa2d5fc2","Type":"ContainerDied","Data":"5a171d88072795e803a5d56decdf74b34dfde4756856ab4c457fb64a1101b16f"} Dec 03 12:46:57 crc kubenswrapper[5052]: I1203 12:46:57.159172 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-6d46dc454-rpjwz" event={"ID":"8fbc67c1-6c51-4c53-85c0-1cf3fa2d5fc2","Type":"ContainerDied","Data":"7a4502b4df7e176325fc9d9d7127106b358c7a727cbfa2e138f865de18f5629a"} Dec 03 12:46:57 crc kubenswrapper[5052]: I1203 12:46:57.159195 5052 scope.go:117] "RemoveContainer" containerID="5a171d88072795e803a5d56decdf74b34dfde4756856ab4c457fb64a1101b16f" Dec 03 12:46:57 crc kubenswrapper[5052]: I1203 12:46:57.166745 5052 generic.go:334] "Generic (PLEG): container finished" podID="4c9f4da8-f1ae-4815-8ee2-0e873fb07d62" containerID="c04c0411733e4ad72fe617e9a0abb2297cf6a19a36fdb5ef4c3061887c382c9a" exitCode=0 Dec 03 12:46:57 crc kubenswrapper[5052]: I1203 12:46:57.169128 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"4c9f4da8-f1ae-4815-8ee2-0e873fb07d62","Type":"ContainerDied","Data":"c04c0411733e4ad72fe617e9a0abb2297cf6a19a36fdb5ef4c3061887c382c9a"} Dec 03 12:46:57 crc kubenswrapper[5052]: I1203 12:46:57.184620 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:46:57 crc kubenswrapper[5052]: I1203 12:46:57.193484 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8fbc67c1-6c51-4c53-85c0-1cf3fa2d5fc2-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "8fbc67c1-6c51-4c53-85c0-1cf3fa2d5fc2" (UID: "8fbc67c1-6c51-4c53-85c0-1cf3fa2d5fc2"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:46:57 crc kubenswrapper[5052]: I1203 12:46:57.199566 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/8fbc67c1-6c51-4c53-85c0-1cf3fa2d5fc2-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:57 crc kubenswrapper[5052]: I1203 12:46:57.199600 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/8fbc67c1-6c51-4c53-85c0-1cf3fa2d5fc2-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:57 crc kubenswrapper[5052]: I1203 12:46:57.199615 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/8fbc67c1-6c51-4c53-85c0-1cf3fa2d5fc2-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:57 crc kubenswrapper[5052]: I1203 12:46:57.205403 5052 scope.go:117] "RemoveContainer" containerID="4c37d110de42cbaa1349c97632b6d688bda0d1d1a7cefd193e59a9fbe693828d" Dec 03 12:46:57 crc kubenswrapper[5052]: I1203 12:46:57.264854 5052 scope.go:117] "RemoveContainer" containerID="5a171d88072795e803a5d56decdf74b34dfde4756856ab4c457fb64a1101b16f" Dec 03 12:46:57 crc kubenswrapper[5052]: E1203 12:46:57.273141 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5a171d88072795e803a5d56decdf74b34dfde4756856ab4c457fb64a1101b16f\": container with ID starting with 5a171d88072795e803a5d56decdf74b34dfde4756856ab4c457fb64a1101b16f not found: ID does not exist" containerID="5a171d88072795e803a5d56decdf74b34dfde4756856ab4c457fb64a1101b16f" Dec 03 12:46:57 crc kubenswrapper[5052]: I1203 12:46:57.273186 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5a171d88072795e803a5d56decdf74b34dfde4756856ab4c457fb64a1101b16f"} err="failed to get container status \"5a171d88072795e803a5d56decdf74b34dfde4756856ab4c457fb64a1101b16f\": rpc error: code = NotFound desc = could not find container \"5a171d88072795e803a5d56decdf74b34dfde4756856ab4c457fb64a1101b16f\": container with ID starting with 5a171d88072795e803a5d56decdf74b34dfde4756856ab4c457fb64a1101b16f not found: ID does not exist" Dec 03 12:46:57 crc kubenswrapper[5052]: I1203 12:46:57.273215 5052 scope.go:117] "RemoveContainer" containerID="4c37d110de42cbaa1349c97632b6d688bda0d1d1a7cefd193e59a9fbe693828d" Dec 03 12:46:57 crc kubenswrapper[5052]: E1203 12:46:57.273584 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4c37d110de42cbaa1349c97632b6d688bda0d1d1a7cefd193e59a9fbe693828d\": container with ID starting with 4c37d110de42cbaa1349c97632b6d688bda0d1d1a7cefd193e59a9fbe693828d not found: ID does not exist" containerID="4c37d110de42cbaa1349c97632b6d688bda0d1d1a7cefd193e59a9fbe693828d" Dec 03 12:46:57 crc kubenswrapper[5052]: I1203 12:46:57.273605 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4c37d110de42cbaa1349c97632b6d688bda0d1d1a7cefd193e59a9fbe693828d"} err="failed to get container status \"4c37d110de42cbaa1349c97632b6d688bda0d1d1a7cefd193e59a9fbe693828d\": rpc error: code = NotFound desc = could not find container \"4c37d110de42cbaa1349c97632b6d688bda0d1d1a7cefd193e59a9fbe693828d\": container with ID starting with 4c37d110de42cbaa1349c97632b6d688bda0d1d1a7cefd193e59a9fbe693828d not found: ID does not exist" Dec 03 12:46:57 crc kubenswrapper[5052]: I1203 12:46:57.286847 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:46:57 crc kubenswrapper[5052]: I1203 12:46:57.291221 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:46:57 crc kubenswrapper[5052]: I1203 12:46:57.291335 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:46:57 crc kubenswrapper[5052]: I1203 12:46:57.404493 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4c9f4da8-f1ae-4815-8ee2-0e873fb07d62-combined-ca-bundle\") pod \"4c9f4da8-f1ae-4815-8ee2-0e873fb07d62\" (UID: \"4c9f4da8-f1ae-4815-8ee2-0e873fb07d62\") " Dec 03 12:46:57 crc kubenswrapper[5052]: I1203 12:46:57.404948 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/4c9f4da8-f1ae-4815-8ee2-0e873fb07d62-logs\") pod \"4c9f4da8-f1ae-4815-8ee2-0e873fb07d62\" (UID: \"4c9f4da8-f1ae-4815-8ee2-0e873fb07d62\") " Dec 03 12:46:57 crc kubenswrapper[5052]: I1203 12:46:57.404973 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/4c9f4da8-f1ae-4815-8ee2-0e873fb07d62-httpd-run\") pod \"4c9f4da8-f1ae-4815-8ee2-0e873fb07d62\" (UID: \"4c9f4da8-f1ae-4815-8ee2-0e873fb07d62\") " Dec 03 12:46:57 crc kubenswrapper[5052]: I1203 12:46:57.405001 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/4c9f4da8-f1ae-4815-8ee2-0e873fb07d62-config-data\") pod \"4c9f4da8-f1ae-4815-8ee2-0e873fb07d62\" (UID: \"4c9f4da8-f1ae-4815-8ee2-0e873fb07d62\") " Dec 03 12:46:57 crc kubenswrapper[5052]: I1203 12:46:57.405033 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"glance\" (UniqueName: \"kubernetes.io/local-volume/local-storage12-crc\") pod \"4c9f4da8-f1ae-4815-8ee2-0e873fb07d62\" (UID: \"4c9f4da8-f1ae-4815-8ee2-0e873fb07d62\") " Dec 03 12:46:57 crc kubenswrapper[5052]: I1203 12:46:57.405126 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/4c9f4da8-f1ae-4815-8ee2-0e873fb07d62-scripts\") pod \"4c9f4da8-f1ae-4815-8ee2-0e873fb07d62\" (UID: \"4c9f4da8-f1ae-4815-8ee2-0e873fb07d62\") " Dec 03 12:46:57 crc kubenswrapper[5052]: I1203 12:46:57.405222 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7l7pm\" (UniqueName: \"kubernetes.io/projected/4c9f4da8-f1ae-4815-8ee2-0e873fb07d62-kube-api-access-7l7pm\") pod \"4c9f4da8-f1ae-4815-8ee2-0e873fb07d62\" (UID: \"4c9f4da8-f1ae-4815-8ee2-0e873fb07d62\") " Dec 03 12:46:57 crc kubenswrapper[5052]: I1203 12:46:57.405261 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/4c9f4da8-f1ae-4815-8ee2-0e873fb07d62-public-tls-certs\") pod \"4c9f4da8-f1ae-4815-8ee2-0e873fb07d62\" (UID: \"4c9f4da8-f1ae-4815-8ee2-0e873fb07d62\") " Dec 03 12:46:57 crc kubenswrapper[5052]: I1203 12:46:57.407212 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c9f4da8-f1ae-4815-8ee2-0e873fb07d62-logs" (OuterVolumeSpecName: "logs") pod "4c9f4da8-f1ae-4815-8ee2-0e873fb07d62" (UID: "4c9f4da8-f1ae-4815-8ee2-0e873fb07d62"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:46:57 crc kubenswrapper[5052]: I1203 12:46:57.407583 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c9f4da8-f1ae-4815-8ee2-0e873fb07d62-httpd-run" (OuterVolumeSpecName: "httpd-run") pod "4c9f4da8-f1ae-4815-8ee2-0e873fb07d62" (UID: "4c9f4da8-f1ae-4815-8ee2-0e873fb07d62"). InnerVolumeSpecName "httpd-run". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:46:57 crc kubenswrapper[5052]: I1203 12:46:57.414544 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage12-crc" (OuterVolumeSpecName: "glance") pod "4c9f4da8-f1ae-4815-8ee2-0e873fb07d62" (UID: "4c9f4da8-f1ae-4815-8ee2-0e873fb07d62"). InnerVolumeSpecName "local-storage12-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 12:46:57 crc kubenswrapper[5052]: I1203 12:46:57.423165 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4c9f4da8-f1ae-4815-8ee2-0e873fb07d62-scripts" (OuterVolumeSpecName: "scripts") pod "4c9f4da8-f1ae-4815-8ee2-0e873fb07d62" (UID: "4c9f4da8-f1ae-4815-8ee2-0e873fb07d62"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:46:57 crc kubenswrapper[5052]: I1203 12:46:57.425822 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4c9f4da8-f1ae-4815-8ee2-0e873fb07d62-kube-api-access-7l7pm" (OuterVolumeSpecName: "kube-api-access-7l7pm") pod "4c9f4da8-f1ae-4815-8ee2-0e873fb07d62" (UID: "4c9f4da8-f1ae-4815-8ee2-0e873fb07d62"). InnerVolumeSpecName "kube-api-access-7l7pm". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:46:57 crc kubenswrapper[5052]: I1203 12:46:57.450699 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4c9f4da8-f1ae-4815-8ee2-0e873fb07d62-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "4c9f4da8-f1ae-4815-8ee2-0e873fb07d62" (UID: "4c9f4da8-f1ae-4815-8ee2-0e873fb07d62"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:46:57 crc kubenswrapper[5052]: I1203 12:46:57.507641 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4c9f4da8-f1ae-4815-8ee2-0e873fb07d62-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:57 crc kubenswrapper[5052]: I1203 12:46:57.507677 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/4c9f4da8-f1ae-4815-8ee2-0e873fb07d62-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:57 crc kubenswrapper[5052]: I1203 12:46:57.507692 5052 reconciler_common.go:293] "Volume detached for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/4c9f4da8-f1ae-4815-8ee2-0e873fb07d62-httpd-run\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:57 crc kubenswrapper[5052]: I1203 12:46:57.507718 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage12-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage12-crc\") on node \"crc\" " Dec 03 12:46:57 crc kubenswrapper[5052]: I1203 12:46:57.507732 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/4c9f4da8-f1ae-4815-8ee2-0e873fb07d62-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:57 crc kubenswrapper[5052]: I1203 12:46:57.507746 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-7l7pm\" (UniqueName: \"kubernetes.io/projected/4c9f4da8-f1ae-4815-8ee2-0e873fb07d62-kube-api-access-7l7pm\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:57 crc kubenswrapper[5052]: I1203 12:46:57.508970 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:46:57 crc kubenswrapper[5052]: I1203 12:46:57.510480 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4c9f4da8-f1ae-4815-8ee2-0e873fb07d62-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "4c9f4da8-f1ae-4815-8ee2-0e873fb07d62" (UID: "4c9f4da8-f1ae-4815-8ee2-0e873fb07d62"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:46:57 crc kubenswrapper[5052]: I1203 12:46:57.549807 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4c9f4da8-f1ae-4815-8ee2-0e873fb07d62-config-data" (OuterVolumeSpecName: "config-data") pod "4c9f4da8-f1ae-4815-8ee2-0e873fb07d62" (UID: "4c9f4da8-f1ae-4815-8ee2-0e873fb07d62"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:46:57 crc kubenswrapper[5052]: I1203 12:46:57.564684 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage12-crc" (UniqueName: "kubernetes.io/local-volume/local-storage12-crc") on node "crc" Dec 03 12:46:57 crc kubenswrapper[5052]: I1203 12:46:57.610339 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/4c9f4da8-f1ae-4815-8ee2-0e873fb07d62-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:57 crc kubenswrapper[5052]: I1203 12:46:57.610389 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage12-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage12-crc\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:57 crc kubenswrapper[5052]: I1203 12:46:57.610403 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/4c9f4da8-f1ae-4815-8ee2-0e873fb07d62-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:46:57 crc kubenswrapper[5052]: I1203 12:46:57.686727 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bf8b9b9c-b8f9-4c93-8112-43913950fcdf" path="/var/lib/kubelet/pods/bf8b9b9c-b8f9-4c93-8112-43913950fcdf/volumes" Dec 03 12:46:57 crc kubenswrapper[5052]: I1203 12:46:57.688099 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/placement-6d46dc454-rpjwz"] Dec 03 12:46:57 crc kubenswrapper[5052]: I1203 12:46:57.691332 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/placement-6d46dc454-rpjwz"] Dec 03 12:46:57 crc kubenswrapper[5052]: W1203 12:46:57.755080 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podffe02c07_7dca_4dc6_b9a2_71096980da67.slice/crio-deea0040fd270142f385fe61e06a3fcca607700162c942c18e1035a3ae17faec WatchSource:0}: Error finding container deea0040fd270142f385fe61e06a3fcca607700162c942c18e1035a3ae17faec: Status 404 returned error can't find the container with id deea0040fd270142f385fe61e06a3fcca607700162c942c18e1035a3ae17faec Dec 03 12:46:57 crc kubenswrapper[5052]: I1203 12:46:57.760125 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:46:58 crc kubenswrapper[5052]: I1203 12:46:58.187409 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:46:58 crc kubenswrapper[5052]: I1203 12:46:58.187409 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"4c9f4da8-f1ae-4815-8ee2-0e873fb07d62","Type":"ContainerDied","Data":"5508cf0583ceb9a8b712a6f4c574d921b80de874086d91a5e31bb2abd237c9b1"} Dec 03 12:46:58 crc kubenswrapper[5052]: I1203 12:46:58.187809 5052 scope.go:117] "RemoveContainer" containerID="c04c0411733e4ad72fe617e9a0abb2297cf6a19a36fdb5ef4c3061887c382c9a" Dec 03 12:46:58 crc kubenswrapper[5052]: I1203 12:46:58.191066 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"ffe02c07-7dca-4dc6-b9a2-71096980da67","Type":"ContainerStarted","Data":"deea0040fd270142f385fe61e06a3fcca607700162c942c18e1035a3ae17faec"} Dec 03 12:46:58 crc kubenswrapper[5052]: I1203 12:46:58.194287 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-74746c7f4-h6sx8" event={"ID":"5201872c-fde1-49d6-9784-6a7f54497990","Type":"ContainerStarted","Data":"d07d82d1f66b2f1adb94be8826e8db31e731aedc69314cb16576303298684ad0"} Dec 03 12:46:58 crc kubenswrapper[5052]: I1203 12:46:58.194327 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-74746c7f4-h6sx8" event={"ID":"5201872c-fde1-49d6-9784-6a7f54497990","Type":"ContainerStarted","Data":"775232fb24de37720556752c4b1bc2dd4ea9fd1233582be2d16b27194e0a1798"} Dec 03 12:46:58 crc kubenswrapper[5052]: I1203 12:46:58.194433 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/placement-74746c7f4-h6sx8" Dec 03 12:46:58 crc kubenswrapper[5052]: I1203 12:46:58.194458 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/placement-74746c7f4-h6sx8" Dec 03 12:46:58 crc kubenswrapper[5052]: I1203 12:46:58.239655 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/placement-74746c7f4-h6sx8" podStartSLOduration=3.239633969 podStartE2EDuration="3.239633969s" podCreationTimestamp="2025-12-03 12:46:55 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:46:58.221375657 +0000 UTC m=+5208.809617421" watchObservedRunningTime="2025-12-03 12:46:58.239633969 +0000 UTC m=+5208.827875733" Dec 03 12:46:58 crc kubenswrapper[5052]: I1203 12:46:58.240086 5052 scope.go:117] "RemoveContainer" containerID="1974352f744e6b1132798362ce57f0fa14a8df4a45523231261a216257d4872d" Dec 03 12:46:58 crc kubenswrapper[5052]: I1203 12:46:58.254975 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:46:58 crc kubenswrapper[5052]: I1203 12:46:58.278403 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:46:58 crc kubenswrapper[5052]: I1203 12:46:58.301196 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:46:58 crc kubenswrapper[5052]: E1203 12:46:58.301759 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="4c9f4da8-f1ae-4815-8ee2-0e873fb07d62" containerName="glance-httpd" Dec 03 12:46:58 crc kubenswrapper[5052]: I1203 12:46:58.301775 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="4c9f4da8-f1ae-4815-8ee2-0e873fb07d62" containerName="glance-httpd" Dec 03 12:46:58 crc kubenswrapper[5052]: E1203 12:46:58.301811 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="4c9f4da8-f1ae-4815-8ee2-0e873fb07d62" containerName="glance-log" Dec 03 12:46:58 crc kubenswrapper[5052]: I1203 12:46:58.301817 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="4c9f4da8-f1ae-4815-8ee2-0e873fb07d62" containerName="glance-log" Dec 03 12:46:58 crc kubenswrapper[5052]: E1203 12:46:58.301830 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="8fbc67c1-6c51-4c53-85c0-1cf3fa2d5fc2" containerName="placement-log" Dec 03 12:46:58 crc kubenswrapper[5052]: I1203 12:46:58.301836 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="8fbc67c1-6c51-4c53-85c0-1cf3fa2d5fc2" containerName="placement-log" Dec 03 12:46:58 crc kubenswrapper[5052]: E1203 12:46:58.301855 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="8fbc67c1-6c51-4c53-85c0-1cf3fa2d5fc2" containerName="placement-api" Dec 03 12:46:58 crc kubenswrapper[5052]: I1203 12:46:58.301862 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="8fbc67c1-6c51-4c53-85c0-1cf3fa2d5fc2" containerName="placement-api" Dec 03 12:46:58 crc kubenswrapper[5052]: I1203 12:46:58.302034 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="8fbc67c1-6c51-4c53-85c0-1cf3fa2d5fc2" containerName="placement-log" Dec 03 12:46:58 crc kubenswrapper[5052]: I1203 12:46:58.302059 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="8fbc67c1-6c51-4c53-85c0-1cf3fa2d5fc2" containerName="placement-api" Dec 03 12:46:58 crc kubenswrapper[5052]: I1203 12:46:58.302076 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="4c9f4da8-f1ae-4815-8ee2-0e873fb07d62" containerName="glance-httpd" Dec 03 12:46:58 crc kubenswrapper[5052]: I1203 12:46:58.302088 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="4c9f4da8-f1ae-4815-8ee2-0e873fb07d62" containerName="glance-log" Dec 03 12:46:58 crc kubenswrapper[5052]: I1203 12:46:58.303176 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:46:58 crc kubenswrapper[5052]: I1203 12:46:58.310655 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-glance-default-public-svc" Dec 03 12:46:58 crc kubenswrapper[5052]: I1203 12:46:58.310934 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-default-external-config-data" Dec 03 12:46:58 crc kubenswrapper[5052]: I1203 12:46:58.319602 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:46:58 crc kubenswrapper[5052]: I1203 12:46:58.404304 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:46:58 crc kubenswrapper[5052]: I1203 12:46:58.468298 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-api-6cd888f578-nv6fh"] Dec 03 12:46:58 crc kubenswrapper[5052]: I1203 12:46:58.468617 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/barbican-api-6cd888f578-nv6fh" podUID="1f026d39-713b-4611-80fd-9577e697d0c4" containerName="barbican-api-log" containerID="cri-o://795d64dcbb859f297be1ee826fd9ec0faf4da92063117b21eb577b822e4b7657" gracePeriod=30 Dec 03 12:46:58 crc kubenswrapper[5052]: I1203 12:46:58.468756 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/barbican-api-6cd888f578-nv6fh" podUID="1f026d39-713b-4611-80fd-9577e697d0c4" containerName="barbican-api" containerID="cri-o://56d9b1a087e1cc1a1e32bb0b52dc44b84176e308fe03f914bdbda9333ae12466" gracePeriod=30 Dec 03 12:46:58 crc kubenswrapper[5052]: I1203 12:46:58.506906 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/barbican-api-6cd888f578-nv6fh" podUID="1f026d39-713b-4611-80fd-9577e697d0c4" containerName="barbican-api-log" probeResult="failure" output="Get \"https://10.217.0.103:9311/healthcheck\": EOF" Dec 03 12:46:58 crc kubenswrapper[5052]: I1203 12:46:58.506991 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/barbican-api-6cd888f578-nv6fh" podUID="1f026d39-713b-4611-80fd-9577e697d0c4" containerName="barbican-api" probeResult="failure" output="Get \"https://10.217.0.103:9311/healthcheck\": EOF" Dec 03 12:46:58 crc kubenswrapper[5052]: I1203 12:46:58.515699 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/barbican-api-7764f4c764-tsnp4"] Dec 03 12:46:58 crc kubenswrapper[5052]: I1203 12:46:58.517490 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-api-7764f4c764-tsnp4" Dec 03 12:46:58 crc kubenswrapper[5052]: I1203 12:46:58.532428 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-api-7764f4c764-tsnp4"] Dec 03 12:46:58 crc kubenswrapper[5052]: I1203 12:46:58.533889 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wl9wh\" (UniqueName: \"kubernetes.io/projected/907ed515-37e6-4cf2-b5a8-1513b8d6b10a-kube-api-access-wl9wh\") pod \"glance-default-external-api-0\" (UID: \"907ed515-37e6-4cf2-b5a8-1513b8d6b10a\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:46:58 crc kubenswrapper[5052]: I1203 12:46:58.533981 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/907ed515-37e6-4cf2-b5a8-1513b8d6b10a-combined-ca-bundle\") pod \"glance-default-external-api-0\" (UID: \"907ed515-37e6-4cf2-b5a8-1513b8d6b10a\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:46:58 crc kubenswrapper[5052]: I1203 12:46:58.534028 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage12-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage12-crc\") pod \"glance-default-external-api-0\" (UID: \"907ed515-37e6-4cf2-b5a8-1513b8d6b10a\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:46:58 crc kubenswrapper[5052]: I1203 12:46:58.534059 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/907ed515-37e6-4cf2-b5a8-1513b8d6b10a-config-data\") pod \"glance-default-external-api-0\" (UID: \"907ed515-37e6-4cf2-b5a8-1513b8d6b10a\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:46:58 crc kubenswrapper[5052]: I1203 12:46:58.534182 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/907ed515-37e6-4cf2-b5a8-1513b8d6b10a-httpd-run\") pod \"glance-default-external-api-0\" (UID: \"907ed515-37e6-4cf2-b5a8-1513b8d6b10a\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:46:58 crc kubenswrapper[5052]: I1203 12:46:58.534288 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/907ed515-37e6-4cf2-b5a8-1513b8d6b10a-public-tls-certs\") pod \"glance-default-external-api-0\" (UID: \"907ed515-37e6-4cf2-b5a8-1513b8d6b10a\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:46:58 crc kubenswrapper[5052]: I1203 12:46:58.534358 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/907ed515-37e6-4cf2-b5a8-1513b8d6b10a-logs\") pod \"glance-default-external-api-0\" (UID: \"907ed515-37e6-4cf2-b5a8-1513b8d6b10a\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:46:58 crc kubenswrapper[5052]: I1203 12:46:58.534449 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/907ed515-37e6-4cf2-b5a8-1513b8d6b10a-scripts\") pod \"glance-default-external-api-0\" (UID: \"907ed515-37e6-4cf2-b5a8-1513b8d6b10a\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:46:58 crc kubenswrapper[5052]: I1203 12:46:58.638058 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/74a94443-98ef-44b2-ae1b-43644b7c393b-internal-tls-certs\") pod \"barbican-api-7764f4c764-tsnp4\" (UID: \"74a94443-98ef-44b2-ae1b-43644b7c393b\") " pod="openstack-kuttl-tests/barbican-api-7764f4c764-tsnp4" Dec 03 12:46:58 crc kubenswrapper[5052]: I1203 12:46:58.638561 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/907ed515-37e6-4cf2-b5a8-1513b8d6b10a-httpd-run\") pod \"glance-default-external-api-0\" (UID: \"907ed515-37e6-4cf2-b5a8-1513b8d6b10a\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:46:58 crc kubenswrapper[5052]: I1203 12:46:58.638629 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/74a94443-98ef-44b2-ae1b-43644b7c393b-config-data\") pod \"barbican-api-7764f4c764-tsnp4\" (UID: \"74a94443-98ef-44b2-ae1b-43644b7c393b\") " pod="openstack-kuttl-tests/barbican-api-7764f4c764-tsnp4" Dec 03 12:46:58 crc kubenswrapper[5052]: I1203 12:46:58.638663 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/74a94443-98ef-44b2-ae1b-43644b7c393b-logs\") pod \"barbican-api-7764f4c764-tsnp4\" (UID: \"74a94443-98ef-44b2-ae1b-43644b7c393b\") " pod="openstack-kuttl-tests/barbican-api-7764f4c764-tsnp4" Dec 03 12:46:58 crc kubenswrapper[5052]: I1203 12:46:58.638698 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/907ed515-37e6-4cf2-b5a8-1513b8d6b10a-public-tls-certs\") pod \"glance-default-external-api-0\" (UID: \"907ed515-37e6-4cf2-b5a8-1513b8d6b10a\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:46:58 crc kubenswrapper[5052]: I1203 12:46:58.638740 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/907ed515-37e6-4cf2-b5a8-1513b8d6b10a-logs\") pod \"glance-default-external-api-0\" (UID: \"907ed515-37e6-4cf2-b5a8-1513b8d6b10a\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:46:58 crc kubenswrapper[5052]: I1203 12:46:58.638767 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/74a94443-98ef-44b2-ae1b-43644b7c393b-combined-ca-bundle\") pod \"barbican-api-7764f4c764-tsnp4\" (UID: \"74a94443-98ef-44b2-ae1b-43644b7c393b\") " pod="openstack-kuttl-tests/barbican-api-7764f4c764-tsnp4" Dec 03 12:46:58 crc kubenswrapper[5052]: I1203 12:46:58.638804 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/74a94443-98ef-44b2-ae1b-43644b7c393b-config-data-custom\") pod \"barbican-api-7764f4c764-tsnp4\" (UID: \"74a94443-98ef-44b2-ae1b-43644b7c393b\") " pod="openstack-kuttl-tests/barbican-api-7764f4c764-tsnp4" Dec 03 12:46:58 crc kubenswrapper[5052]: I1203 12:46:58.638827 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/74a94443-98ef-44b2-ae1b-43644b7c393b-public-tls-certs\") pod \"barbican-api-7764f4c764-tsnp4\" (UID: \"74a94443-98ef-44b2-ae1b-43644b7c393b\") " pod="openstack-kuttl-tests/barbican-api-7764f4c764-tsnp4" Dec 03 12:46:58 crc kubenswrapper[5052]: I1203 12:46:58.638871 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gbsb2\" (UniqueName: \"kubernetes.io/projected/74a94443-98ef-44b2-ae1b-43644b7c393b-kube-api-access-gbsb2\") pod \"barbican-api-7764f4c764-tsnp4\" (UID: \"74a94443-98ef-44b2-ae1b-43644b7c393b\") " pod="openstack-kuttl-tests/barbican-api-7764f4c764-tsnp4" Dec 03 12:46:58 crc kubenswrapper[5052]: I1203 12:46:58.638899 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/907ed515-37e6-4cf2-b5a8-1513b8d6b10a-scripts\") pod \"glance-default-external-api-0\" (UID: \"907ed515-37e6-4cf2-b5a8-1513b8d6b10a\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:46:58 crc kubenswrapper[5052]: I1203 12:46:58.638992 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-wl9wh\" (UniqueName: \"kubernetes.io/projected/907ed515-37e6-4cf2-b5a8-1513b8d6b10a-kube-api-access-wl9wh\") pod \"glance-default-external-api-0\" (UID: \"907ed515-37e6-4cf2-b5a8-1513b8d6b10a\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:46:58 crc kubenswrapper[5052]: I1203 12:46:58.639045 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/907ed515-37e6-4cf2-b5a8-1513b8d6b10a-combined-ca-bundle\") pod \"glance-default-external-api-0\" (UID: \"907ed515-37e6-4cf2-b5a8-1513b8d6b10a\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:46:58 crc kubenswrapper[5052]: I1203 12:46:58.639083 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage12-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage12-crc\") pod \"glance-default-external-api-0\" (UID: \"907ed515-37e6-4cf2-b5a8-1513b8d6b10a\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:46:58 crc kubenswrapper[5052]: I1203 12:46:58.639113 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/907ed515-37e6-4cf2-b5a8-1513b8d6b10a-config-data\") pod \"glance-default-external-api-0\" (UID: \"907ed515-37e6-4cf2-b5a8-1513b8d6b10a\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:46:58 crc kubenswrapper[5052]: I1203 12:46:58.640980 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/907ed515-37e6-4cf2-b5a8-1513b8d6b10a-httpd-run\") pod \"glance-default-external-api-0\" (UID: \"907ed515-37e6-4cf2-b5a8-1513b8d6b10a\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:46:58 crc kubenswrapper[5052]: I1203 12:46:58.642020 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/907ed515-37e6-4cf2-b5a8-1513b8d6b10a-logs\") pod \"glance-default-external-api-0\" (UID: \"907ed515-37e6-4cf2-b5a8-1513b8d6b10a\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:46:58 crc kubenswrapper[5052]: I1203 12:46:58.643256 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage12-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage12-crc\") pod \"glance-default-external-api-0\" (UID: \"907ed515-37e6-4cf2-b5a8-1513b8d6b10a\") device mount path \"/mnt/openstack/pv12\"" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:46:58 crc kubenswrapper[5052]: I1203 12:46:58.646730 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/907ed515-37e6-4cf2-b5a8-1513b8d6b10a-config-data\") pod \"glance-default-external-api-0\" (UID: \"907ed515-37e6-4cf2-b5a8-1513b8d6b10a\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:46:58 crc kubenswrapper[5052]: I1203 12:46:58.651487 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/907ed515-37e6-4cf2-b5a8-1513b8d6b10a-combined-ca-bundle\") pod \"glance-default-external-api-0\" (UID: \"907ed515-37e6-4cf2-b5a8-1513b8d6b10a\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:46:58 crc kubenswrapper[5052]: I1203 12:46:58.654263 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/907ed515-37e6-4cf2-b5a8-1513b8d6b10a-scripts\") pod \"glance-default-external-api-0\" (UID: \"907ed515-37e6-4cf2-b5a8-1513b8d6b10a\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:46:58 crc kubenswrapper[5052]: I1203 12:46:58.662273 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/907ed515-37e6-4cf2-b5a8-1513b8d6b10a-public-tls-certs\") pod \"glance-default-external-api-0\" (UID: \"907ed515-37e6-4cf2-b5a8-1513b8d6b10a\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:46:58 crc kubenswrapper[5052]: I1203 12:46:58.665691 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-wl9wh\" (UniqueName: \"kubernetes.io/projected/907ed515-37e6-4cf2-b5a8-1513b8d6b10a-kube-api-access-wl9wh\") pod \"glance-default-external-api-0\" (UID: \"907ed515-37e6-4cf2-b5a8-1513b8d6b10a\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:46:58 crc kubenswrapper[5052]: I1203 12:46:58.697009 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage12-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage12-crc\") pod \"glance-default-external-api-0\" (UID: \"907ed515-37e6-4cf2-b5a8-1513b8d6b10a\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:46:58 crc kubenswrapper[5052]: I1203 12:46:58.740619 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/74a94443-98ef-44b2-ae1b-43644b7c393b-combined-ca-bundle\") pod \"barbican-api-7764f4c764-tsnp4\" (UID: \"74a94443-98ef-44b2-ae1b-43644b7c393b\") " pod="openstack-kuttl-tests/barbican-api-7764f4c764-tsnp4" Dec 03 12:46:58 crc kubenswrapper[5052]: I1203 12:46:58.740705 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/74a94443-98ef-44b2-ae1b-43644b7c393b-config-data-custom\") pod \"barbican-api-7764f4c764-tsnp4\" (UID: \"74a94443-98ef-44b2-ae1b-43644b7c393b\") " pod="openstack-kuttl-tests/barbican-api-7764f4c764-tsnp4" Dec 03 12:46:58 crc kubenswrapper[5052]: I1203 12:46:58.740730 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/74a94443-98ef-44b2-ae1b-43644b7c393b-public-tls-certs\") pod \"barbican-api-7764f4c764-tsnp4\" (UID: \"74a94443-98ef-44b2-ae1b-43644b7c393b\") " pod="openstack-kuttl-tests/barbican-api-7764f4c764-tsnp4" Dec 03 12:46:58 crc kubenswrapper[5052]: I1203 12:46:58.740778 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-gbsb2\" (UniqueName: \"kubernetes.io/projected/74a94443-98ef-44b2-ae1b-43644b7c393b-kube-api-access-gbsb2\") pod \"barbican-api-7764f4c764-tsnp4\" (UID: \"74a94443-98ef-44b2-ae1b-43644b7c393b\") " pod="openstack-kuttl-tests/barbican-api-7764f4c764-tsnp4" Dec 03 12:46:58 crc kubenswrapper[5052]: I1203 12:46:58.740971 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/74a94443-98ef-44b2-ae1b-43644b7c393b-internal-tls-certs\") pod \"barbican-api-7764f4c764-tsnp4\" (UID: \"74a94443-98ef-44b2-ae1b-43644b7c393b\") " pod="openstack-kuttl-tests/barbican-api-7764f4c764-tsnp4" Dec 03 12:46:58 crc kubenswrapper[5052]: I1203 12:46:58.741040 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/74a94443-98ef-44b2-ae1b-43644b7c393b-config-data\") pod \"barbican-api-7764f4c764-tsnp4\" (UID: \"74a94443-98ef-44b2-ae1b-43644b7c393b\") " pod="openstack-kuttl-tests/barbican-api-7764f4c764-tsnp4" Dec 03 12:46:58 crc kubenswrapper[5052]: I1203 12:46:58.741073 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/74a94443-98ef-44b2-ae1b-43644b7c393b-logs\") pod \"barbican-api-7764f4c764-tsnp4\" (UID: \"74a94443-98ef-44b2-ae1b-43644b7c393b\") " pod="openstack-kuttl-tests/barbican-api-7764f4c764-tsnp4" Dec 03 12:46:58 crc kubenswrapper[5052]: I1203 12:46:58.741627 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/74a94443-98ef-44b2-ae1b-43644b7c393b-logs\") pod \"barbican-api-7764f4c764-tsnp4\" (UID: \"74a94443-98ef-44b2-ae1b-43644b7c393b\") " pod="openstack-kuttl-tests/barbican-api-7764f4c764-tsnp4" Dec 03 12:46:58 crc kubenswrapper[5052]: I1203 12:46:58.745501 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/74a94443-98ef-44b2-ae1b-43644b7c393b-combined-ca-bundle\") pod \"barbican-api-7764f4c764-tsnp4\" (UID: \"74a94443-98ef-44b2-ae1b-43644b7c393b\") " pod="openstack-kuttl-tests/barbican-api-7764f4c764-tsnp4" Dec 03 12:46:58 crc kubenswrapper[5052]: I1203 12:46:58.748313 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/74a94443-98ef-44b2-ae1b-43644b7c393b-internal-tls-certs\") pod \"barbican-api-7764f4c764-tsnp4\" (UID: \"74a94443-98ef-44b2-ae1b-43644b7c393b\") " pod="openstack-kuttl-tests/barbican-api-7764f4c764-tsnp4" Dec 03 12:46:58 crc kubenswrapper[5052]: I1203 12:46:58.749310 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/74a94443-98ef-44b2-ae1b-43644b7c393b-public-tls-certs\") pod \"barbican-api-7764f4c764-tsnp4\" (UID: \"74a94443-98ef-44b2-ae1b-43644b7c393b\") " pod="openstack-kuttl-tests/barbican-api-7764f4c764-tsnp4" Dec 03 12:46:58 crc kubenswrapper[5052]: I1203 12:46:58.750237 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/74a94443-98ef-44b2-ae1b-43644b7c393b-config-data-custom\") pod \"barbican-api-7764f4c764-tsnp4\" (UID: \"74a94443-98ef-44b2-ae1b-43644b7c393b\") " pod="openstack-kuttl-tests/barbican-api-7764f4c764-tsnp4" Dec 03 12:46:58 crc kubenswrapper[5052]: I1203 12:46:58.755856 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/74a94443-98ef-44b2-ae1b-43644b7c393b-config-data\") pod \"barbican-api-7764f4c764-tsnp4\" (UID: \"74a94443-98ef-44b2-ae1b-43644b7c393b\") " pod="openstack-kuttl-tests/barbican-api-7764f4c764-tsnp4" Dec 03 12:46:58 crc kubenswrapper[5052]: I1203 12:46:58.770222 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-gbsb2\" (UniqueName: \"kubernetes.io/projected/74a94443-98ef-44b2-ae1b-43644b7c393b-kube-api-access-gbsb2\") pod \"barbican-api-7764f4c764-tsnp4\" (UID: \"74a94443-98ef-44b2-ae1b-43644b7c393b\") " pod="openstack-kuttl-tests/barbican-api-7764f4c764-tsnp4" Dec 03 12:46:58 crc kubenswrapper[5052]: I1203 12:46:58.843597 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-api-7764f4c764-tsnp4" Dec 03 12:46:58 crc kubenswrapper[5052]: I1203 12:46:58.988252 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:46:59 crc kubenswrapper[5052]: I1203 12:46:59.212068 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"ffe02c07-7dca-4dc6-b9a2-71096980da67","Type":"ContainerStarted","Data":"9f987a2975c4c708322ab3333444993f64841bd455d7d47ea4fd315d851e5758"} Dec 03 12:46:59 crc kubenswrapper[5052]: I1203 12:46:59.225617 5052 generic.go:334] "Generic (PLEG): container finished" podID="1f026d39-713b-4611-80fd-9577e697d0c4" containerID="795d64dcbb859f297be1ee826fd9ec0faf4da92063117b21eb577b822e4b7657" exitCode=143 Dec 03 12:46:59 crc kubenswrapper[5052]: I1203 12:46:59.225929 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-6cd888f578-nv6fh" event={"ID":"1f026d39-713b-4611-80fd-9577e697d0c4","Type":"ContainerDied","Data":"795d64dcbb859f297be1ee826fd9ec0faf4da92063117b21eb577b822e4b7657"} Dec 03 12:46:59 crc kubenswrapper[5052]: I1203 12:46:59.386888 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-api-7764f4c764-tsnp4"] Dec 03 12:46:59 crc kubenswrapper[5052]: I1203 12:46:59.568827 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:46:59 crc kubenswrapper[5052]: W1203 12:46:59.572720 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod907ed515_37e6_4cf2_b5a8_1513b8d6b10a.slice/crio-e07915e77c0024ee3055ed4746eafdf4bdb0fa6f96d475a321adf37d3d3835b6 WatchSource:0}: Error finding container e07915e77c0024ee3055ed4746eafdf4bdb0fa6f96d475a321adf37d3d3835b6: Status 404 returned error can't find the container with id e07915e77c0024ee3055ed4746eafdf4bdb0fa6f96d475a321adf37d3d3835b6 Dec 03 12:46:59 crc kubenswrapper[5052]: I1203 12:46:59.691466 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4c9f4da8-f1ae-4815-8ee2-0e873fb07d62" path="/var/lib/kubelet/pods/4c9f4da8-f1ae-4815-8ee2-0e873fb07d62/volumes" Dec 03 12:46:59 crc kubenswrapper[5052]: I1203 12:46:59.692786 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8fbc67c1-6c51-4c53-85c0-1cf3fa2d5fc2" path="/var/lib/kubelet/pods/8fbc67c1-6c51-4c53-85c0-1cf3fa2d5fc2/volumes" Dec 03 12:47:00 crc kubenswrapper[5052]: I1203 12:47:00.266611 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"ffe02c07-7dca-4dc6-b9a2-71096980da67","Type":"ContainerStarted","Data":"cf227852e5ace90c69093b6805395659785574fe0027d072330aed46598ce53e"} Dec 03 12:47:00 crc kubenswrapper[5052]: I1203 12:47:00.280885 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"907ed515-37e6-4cf2-b5a8-1513b8d6b10a","Type":"ContainerStarted","Data":"ad61278101ca96b627cc3c2e0d5d4d7a0579b5dec7926e0f7cf122ebe2243d4d"} Dec 03 12:47:00 crc kubenswrapper[5052]: I1203 12:47:00.280931 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"907ed515-37e6-4cf2-b5a8-1513b8d6b10a","Type":"ContainerStarted","Data":"e07915e77c0024ee3055ed4746eafdf4bdb0fa6f96d475a321adf37d3d3835b6"} Dec 03 12:47:00 crc kubenswrapper[5052]: I1203 12:47:00.284552 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-7764f4c764-tsnp4" event={"ID":"74a94443-98ef-44b2-ae1b-43644b7c393b","Type":"ContainerStarted","Data":"6e751423b8df4113d67dc20a6fb9fa3c1e998f1985a9bb1d69a0dc6459218bec"} Dec 03 12:47:00 crc kubenswrapper[5052]: I1203 12:47:00.284621 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-7764f4c764-tsnp4" event={"ID":"74a94443-98ef-44b2-ae1b-43644b7c393b","Type":"ContainerStarted","Data":"5010ae0ecbbec159c8035c9663c6ff54ee13ce879a982154a989755d6696637c"} Dec 03 12:47:00 crc kubenswrapper[5052]: I1203 12:47:00.284638 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-7764f4c764-tsnp4" event={"ID":"74a94443-98ef-44b2-ae1b-43644b7c393b","Type":"ContainerStarted","Data":"1d085744642a41502dc368e41a82a9d82c36ae3bc40b6401638cfa3cdcdca60e"} Dec 03 12:47:00 crc kubenswrapper[5052]: I1203 12:47:00.284669 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/barbican-api-7764f4c764-tsnp4" Dec 03 12:47:00 crc kubenswrapper[5052]: I1203 12:47:00.284858 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/barbican-api-7764f4c764-tsnp4" Dec 03 12:47:00 crc kubenswrapper[5052]: I1203 12:47:00.306604 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/glance-default-internal-api-0" podStartSLOduration=4.306575271 podStartE2EDuration="4.306575271s" podCreationTimestamp="2025-12-03 12:46:56 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:47:00.29048177 +0000 UTC m=+5210.878723544" watchObservedRunningTime="2025-12-03 12:47:00.306575271 +0000 UTC m=+5210.894817045" Dec 03 12:47:00 crc kubenswrapper[5052]: I1203 12:47:00.326715 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/barbican-api-7764f4c764-tsnp4" podStartSLOduration=2.326695576 podStartE2EDuration="2.326695576s" podCreationTimestamp="2025-12-03 12:46:58 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:47:00.322572888 +0000 UTC m=+5210.910814652" watchObservedRunningTime="2025-12-03 12:47:00.326695576 +0000 UTC m=+5210.914937340" Dec 03 12:47:00 crc kubenswrapper[5052]: I1203 12:47:00.384693 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-metadata-0" podUID="75266130-5854-4acd-acc9-bc22accc19bc" containerName="nova-metadata-log" probeResult="failure" output="Get \"https://10.217.0.12:8775/\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Dec 03 12:47:00 crc kubenswrapper[5052]: I1203 12:47:00.384826 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-metadata-0" podUID="75266130-5854-4acd-acc9-bc22accc19bc" containerName="nova-metadata-metadata" probeResult="failure" output="Get \"https://10.217.0.12:8775/\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Dec 03 12:47:00 crc kubenswrapper[5052]: I1203 12:47:00.673718 5052 scope.go:117] "RemoveContainer" containerID="4c1d25fd5e7f80502770dbe44896eb5a3327b1bebde080de0e66dead94625111" Dec 03 12:47:00 crc kubenswrapper[5052]: E1203 12:47:00.674282 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"nova-cell0-conductor-conductor\" with CrashLoopBackOff: \"back-off 20s restarting failed container=nova-cell0-conductor-conductor pod=nova-cell0-conductor-0_openstack-kuttl-tests(6d653682-52e2-4940-af66-633475bfe09a)\"" pod="openstack-kuttl-tests/nova-cell0-conductor-0" podUID="6d653682-52e2-4940-af66-633475bfe09a" Dec 03 12:47:01 crc kubenswrapper[5052]: I1203 12:47:01.295942 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"907ed515-37e6-4cf2-b5a8-1513b8d6b10a","Type":"ContainerStarted","Data":"5a7d6ee55e4c5a722aa4bdb52015036d562245e1ebcdb5aa1403b17740fdfb59"} Dec 03 12:47:01 crc kubenswrapper[5052]: I1203 12:47:01.326223 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/glance-default-external-api-0" podStartSLOduration=3.326202289 podStartE2EDuration="3.326202289s" podCreationTimestamp="2025-12-03 12:46:58 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:47:01.321443533 +0000 UTC m=+5211.909685307" watchObservedRunningTime="2025-12-03 12:47:01.326202289 +0000 UTC m=+5211.914444053" Dec 03 12:47:02 crc kubenswrapper[5052]: I1203 12:47:02.167292 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:47:02 crc kubenswrapper[5052]: I1203 12:47:02.478724 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-api-0" podUID="d5719b07-205b-4e6a-9fe2-b6531b779b0a" containerName="nova-api-log" probeResult="failure" output="Get \"https://10.217.0.98:8774/\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Dec 03 12:47:02 crc kubenswrapper[5052]: I1203 12:47:02.482914 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-api-0" podUID="d5719b07-205b-4e6a-9fe2-b6531b779b0a" containerName="nova-api-api" probeResult="failure" output="Get \"https://10.217.0.98:8774/\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Dec 03 12:47:02 crc kubenswrapper[5052]: I1203 12:47:02.673811 5052 scope.go:117] "RemoveContainer" containerID="c397988625a9ab9075d72d6f8c51dd95b36e8b2a4214beb4c4dcf9e2ea912582" Dec 03 12:47:03 crc kubenswrapper[5052]: I1203 12:47:03.320338 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"81709983-1fd0-4446-bbbf-fb94653bb3d2","Type":"ContainerStarted","Data":"d1e48098c512a28625a35e3a019f961f22ad8e764f883b4f8931a03d713b8c01"} Dec 03 12:47:03 crc kubenswrapper[5052]: I1203 12:47:03.527088 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:47:03 crc kubenswrapper[5052]: I1203 12:47:03.931187 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/barbican-api-6cd888f578-nv6fh" podUID="1f026d39-713b-4611-80fd-9577e697d0c4" containerName="barbican-api-log" probeResult="failure" output="Get \"https://10.217.0.103:9311/healthcheck\": read tcp 10.217.0.2:48288->10.217.0.103:9311: read: connection reset by peer" Dec 03 12:47:03 crc kubenswrapper[5052]: I1203 12:47:03.931192 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/barbican-api-6cd888f578-nv6fh" podUID="1f026d39-713b-4611-80fd-9577e697d0c4" containerName="barbican-api" probeResult="failure" output="Get \"https://10.217.0.103:9311/healthcheck\": read tcp 10.217.0.2:48282->10.217.0.103:9311: read: connection reset by peer" Dec 03 12:47:04 crc kubenswrapper[5052]: I1203 12:47:04.339836 5052 generic.go:334] "Generic (PLEG): container finished" podID="1f026d39-713b-4611-80fd-9577e697d0c4" containerID="56d9b1a087e1cc1a1e32bb0b52dc44b84176e308fe03f914bdbda9333ae12466" exitCode=0 Dec 03 12:47:04 crc kubenswrapper[5052]: I1203 12:47:04.339923 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-6cd888f578-nv6fh" event={"ID":"1f026d39-713b-4611-80fd-9577e697d0c4","Type":"ContainerDied","Data":"56d9b1a087e1cc1a1e32bb0b52dc44b84176e308fe03f914bdbda9333ae12466"} Dec 03 12:47:04 crc kubenswrapper[5052]: I1203 12:47:04.340430 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-6cd888f578-nv6fh" event={"ID":"1f026d39-713b-4611-80fd-9577e697d0c4","Type":"ContainerDied","Data":"5ed7fdb90750b8af1a853a0981c8a82939452ba3c9225b2cb4998ae709d8cb14"} Dec 03 12:47:04 crc kubenswrapper[5052]: I1203 12:47:04.340447 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5ed7fdb90750b8af1a853a0981c8a82939452ba3c9225b2cb4998ae709d8cb14" Dec 03 12:47:04 crc kubenswrapper[5052]: I1203 12:47:04.350047 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-api-6cd888f578-nv6fh" Dec 03 12:47:04 crc kubenswrapper[5052]: I1203 12:47:04.471253 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/1f026d39-713b-4611-80fd-9577e697d0c4-internal-tls-certs\") pod \"1f026d39-713b-4611-80fd-9577e697d0c4\" (UID: \"1f026d39-713b-4611-80fd-9577e697d0c4\") " Dec 03 12:47:04 crc kubenswrapper[5052]: I1203 12:47:04.471532 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/1f026d39-713b-4611-80fd-9577e697d0c4-config-data-custom\") pod \"1f026d39-713b-4611-80fd-9577e697d0c4\" (UID: \"1f026d39-713b-4611-80fd-9577e697d0c4\") " Dec 03 12:47:04 crc kubenswrapper[5052]: I1203 12:47:04.471623 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/1f026d39-713b-4611-80fd-9577e697d0c4-config-data\") pod \"1f026d39-713b-4611-80fd-9577e697d0c4\" (UID: \"1f026d39-713b-4611-80fd-9577e697d0c4\") " Dec 03 12:47:04 crc kubenswrapper[5052]: I1203 12:47:04.471663 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6hl9q\" (UniqueName: \"kubernetes.io/projected/1f026d39-713b-4611-80fd-9577e697d0c4-kube-api-access-6hl9q\") pod \"1f026d39-713b-4611-80fd-9577e697d0c4\" (UID: \"1f026d39-713b-4611-80fd-9577e697d0c4\") " Dec 03 12:47:04 crc kubenswrapper[5052]: I1203 12:47:04.471692 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/1f026d39-713b-4611-80fd-9577e697d0c4-public-tls-certs\") pod \"1f026d39-713b-4611-80fd-9577e697d0c4\" (UID: \"1f026d39-713b-4611-80fd-9577e697d0c4\") " Dec 03 12:47:04 crc kubenswrapper[5052]: I1203 12:47:04.471719 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/1f026d39-713b-4611-80fd-9577e697d0c4-logs\") pod \"1f026d39-713b-4611-80fd-9577e697d0c4\" (UID: \"1f026d39-713b-4611-80fd-9577e697d0c4\") " Dec 03 12:47:04 crc kubenswrapper[5052]: I1203 12:47:04.471782 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1f026d39-713b-4611-80fd-9577e697d0c4-combined-ca-bundle\") pod \"1f026d39-713b-4611-80fd-9577e697d0c4\" (UID: \"1f026d39-713b-4611-80fd-9577e697d0c4\") " Dec 03 12:47:04 crc kubenswrapper[5052]: I1203 12:47:04.473315 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f026d39-713b-4611-80fd-9577e697d0c4-logs" (OuterVolumeSpecName: "logs") pod "1f026d39-713b-4611-80fd-9577e697d0c4" (UID: "1f026d39-713b-4611-80fd-9577e697d0c4"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:47:04 crc kubenswrapper[5052]: I1203 12:47:04.483716 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1f026d39-713b-4611-80fd-9577e697d0c4-kube-api-access-6hl9q" (OuterVolumeSpecName: "kube-api-access-6hl9q") pod "1f026d39-713b-4611-80fd-9577e697d0c4" (UID: "1f026d39-713b-4611-80fd-9577e697d0c4"). InnerVolumeSpecName "kube-api-access-6hl9q". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:47:04 crc kubenswrapper[5052]: I1203 12:47:04.487776 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1f026d39-713b-4611-80fd-9577e697d0c4-config-data-custom" (OuterVolumeSpecName: "config-data-custom") pod "1f026d39-713b-4611-80fd-9577e697d0c4" (UID: "1f026d39-713b-4611-80fd-9577e697d0c4"). InnerVolumeSpecName "config-data-custom". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:47:04 crc kubenswrapper[5052]: I1203 12:47:04.515776 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1f026d39-713b-4611-80fd-9577e697d0c4-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "1f026d39-713b-4611-80fd-9577e697d0c4" (UID: "1f026d39-713b-4611-80fd-9577e697d0c4"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:47:04 crc kubenswrapper[5052]: I1203 12:47:04.532733 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1f026d39-713b-4611-80fd-9577e697d0c4-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "1f026d39-713b-4611-80fd-9577e697d0c4" (UID: "1f026d39-713b-4611-80fd-9577e697d0c4"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:47:04 crc kubenswrapper[5052]: I1203 12:47:04.541944 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1f026d39-713b-4611-80fd-9577e697d0c4-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "1f026d39-713b-4611-80fd-9577e697d0c4" (UID: "1f026d39-713b-4611-80fd-9577e697d0c4"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:47:04 crc kubenswrapper[5052]: I1203 12:47:04.563653 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1f026d39-713b-4611-80fd-9577e697d0c4-config-data" (OuterVolumeSpecName: "config-data") pod "1f026d39-713b-4611-80fd-9577e697d0c4" (UID: "1f026d39-713b-4611-80fd-9577e697d0c4"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:47:04 crc kubenswrapper[5052]: I1203 12:47:04.575215 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/1f026d39-713b-4611-80fd-9577e697d0c4-config-data-custom\") on node \"crc\" DevicePath \"\"" Dec 03 12:47:04 crc kubenswrapper[5052]: I1203 12:47:04.575526 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/1f026d39-713b-4611-80fd-9577e697d0c4-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:47:04 crc kubenswrapper[5052]: I1203 12:47:04.575601 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-6hl9q\" (UniqueName: \"kubernetes.io/projected/1f026d39-713b-4611-80fd-9577e697d0c4-kube-api-access-6hl9q\") on node \"crc\" DevicePath \"\"" Dec 03 12:47:04 crc kubenswrapper[5052]: I1203 12:47:04.575659 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/1f026d39-713b-4611-80fd-9577e697d0c4-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:47:04 crc kubenswrapper[5052]: I1203 12:47:04.575760 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/1f026d39-713b-4611-80fd-9577e697d0c4-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:47:04 crc kubenswrapper[5052]: I1203 12:47:04.575828 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1f026d39-713b-4611-80fd-9577e697d0c4-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:47:04 crc kubenswrapper[5052]: I1203 12:47:04.575897 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/1f026d39-713b-4611-80fd-9577e697d0c4-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:47:05 crc kubenswrapper[5052]: I1203 12:47:05.353747 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-api-6cd888f578-nv6fh" Dec 03 12:47:05 crc kubenswrapper[5052]: I1203 12:47:05.407452 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-api-6cd888f578-nv6fh"] Dec 03 12:47:05 crc kubenswrapper[5052]: I1203 12:47:05.416682 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/barbican-api-6cd888f578-nv6fh"] Dec 03 12:47:05 crc kubenswrapper[5052]: I1203 12:47:05.673594 5052 scope.go:117] "RemoveContainer" containerID="e102925eb6bfd868ed0b7dff92ded1a29e84b2579d3092b133af1f0bf476d04c" Dec 03 12:47:05 crc kubenswrapper[5052]: I1203 12:47:05.687311 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1f026d39-713b-4611-80fd-9577e697d0c4" path="/var/lib/kubelet/pods/1f026d39-713b-4611-80fd-9577e697d0c4/volumes" Dec 03 12:47:06 crc kubenswrapper[5052]: I1203 12:47:06.020551 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:47:06 crc kubenswrapper[5052]: I1203 12:47:06.021270 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-external-api-0" podUID="907ed515-37e6-4cf2-b5a8-1513b8d6b10a" containerName="glance-log" containerID="cri-o://ad61278101ca96b627cc3c2e0d5d4d7a0579b5dec7926e0f7cf122ebe2243d4d" gracePeriod=30 Dec 03 12:47:06 crc kubenswrapper[5052]: I1203 12:47:06.021488 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-external-api-0" podUID="907ed515-37e6-4cf2-b5a8-1513b8d6b10a" containerName="glance-httpd" containerID="cri-o://5a7d6ee55e4c5a722aa4bdb52015036d562245e1ebcdb5aa1403b17740fdfb59" gracePeriod=30 Dec 03 12:47:06 crc kubenswrapper[5052]: I1203 12:47:06.090875 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:47:06 crc kubenswrapper[5052]: I1203 12:47:06.091175 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-internal-api-0" podUID="ffe02c07-7dca-4dc6-b9a2-71096980da67" containerName="glance-log" containerID="cri-o://9f987a2975c4c708322ab3333444993f64841bd455d7d47ea4fd315d851e5758" gracePeriod=30 Dec 03 12:47:06 crc kubenswrapper[5052]: I1203 12:47:06.091829 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-internal-api-0" podUID="ffe02c07-7dca-4dc6-b9a2-71096980da67" containerName="glance-httpd" containerID="cri-o://cf227852e5ace90c69093b6805395659785574fe0027d072330aed46598ce53e" gracePeriod=30 Dec 03 12:47:06 crc kubenswrapper[5052]: E1203 12:47:06.159472 5052 cadvisor_stats_provider.go:516] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod907ed515_37e6_4cf2_b5a8_1513b8d6b10a.slice/crio-ad61278101ca96b627cc3c2e0d5d4d7a0579b5dec7926e0f7cf122ebe2243d4d.scope\": RecentStats: unable to find data in memory cache]" Dec 03 12:47:06 crc kubenswrapper[5052]: I1203 12:47:06.399657 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-conductor-0" event={"ID":"8669832d-c47a-49bd-9491-dc7141ac6281","Type":"ContainerStarted","Data":"13a4cd2432b46ba6f3a0ba15e8d71c8570f57e964a36ff41f1150e41215a4cc8"} Dec 03 12:47:06 crc kubenswrapper[5052]: I1203 12:47:06.400521 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:47:06 crc kubenswrapper[5052]: I1203 12:47:06.410716 5052 generic.go:334] "Generic (PLEG): container finished" podID="907ed515-37e6-4cf2-b5a8-1513b8d6b10a" containerID="5a7d6ee55e4c5a722aa4bdb52015036d562245e1ebcdb5aa1403b17740fdfb59" exitCode=0 Dec 03 12:47:06 crc kubenswrapper[5052]: I1203 12:47:06.410773 5052 generic.go:334] "Generic (PLEG): container finished" podID="907ed515-37e6-4cf2-b5a8-1513b8d6b10a" containerID="ad61278101ca96b627cc3c2e0d5d4d7a0579b5dec7926e0f7cf122ebe2243d4d" exitCode=143 Dec 03 12:47:06 crc kubenswrapper[5052]: I1203 12:47:06.410912 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"907ed515-37e6-4cf2-b5a8-1513b8d6b10a","Type":"ContainerDied","Data":"5a7d6ee55e4c5a722aa4bdb52015036d562245e1ebcdb5aa1403b17740fdfb59"} Dec 03 12:47:06 crc kubenswrapper[5052]: I1203 12:47:06.410961 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"907ed515-37e6-4cf2-b5a8-1513b8d6b10a","Type":"ContainerDied","Data":"ad61278101ca96b627cc3c2e0d5d4d7a0579b5dec7926e0f7cf122ebe2243d4d"} Dec 03 12:47:06 crc kubenswrapper[5052]: I1203 12:47:06.430196 5052 generic.go:334] "Generic (PLEG): container finished" podID="81709983-1fd0-4446-bbbf-fb94653bb3d2" containerID="d1e48098c512a28625a35e3a019f961f22ad8e764f883b4f8931a03d713b8c01" exitCode=1 Dec 03 12:47:06 crc kubenswrapper[5052]: I1203 12:47:06.430266 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"81709983-1fd0-4446-bbbf-fb94653bb3d2","Type":"ContainerDied","Data":"d1e48098c512a28625a35e3a019f961f22ad8e764f883b4f8931a03d713b8c01"} Dec 03 12:47:06 crc kubenswrapper[5052]: I1203 12:47:06.430758 5052 scope.go:117] "RemoveContainer" containerID="c397988625a9ab9075d72d6f8c51dd95b36e8b2a4214beb4c4dcf9e2ea912582" Dec 03 12:47:06 crc kubenswrapper[5052]: I1203 12:47:06.440295 5052 scope.go:117] "RemoveContainer" containerID="d1e48098c512a28625a35e3a019f961f22ad8e764f883b4f8931a03d713b8c01" Dec 03 12:47:06 crc kubenswrapper[5052]: E1203 12:47:06.440844 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"nova-scheduler-scheduler\" with CrashLoopBackOff: \"back-off 20s restarting failed container=nova-scheduler-scheduler pod=nova-scheduler-0_openstack-kuttl-tests(81709983-1fd0-4446-bbbf-fb94653bb3d2)\"" pod="openstack-kuttl-tests/nova-scheduler-0" podUID="81709983-1fd0-4446-bbbf-fb94653bb3d2" Dec 03 12:47:06 crc kubenswrapper[5052]: I1203 12:47:06.450191 5052 generic.go:334] "Generic (PLEG): container finished" podID="ffe02c07-7dca-4dc6-b9a2-71096980da67" containerID="cf227852e5ace90c69093b6805395659785574fe0027d072330aed46598ce53e" exitCode=0 Dec 03 12:47:06 crc kubenswrapper[5052]: I1203 12:47:06.450227 5052 generic.go:334] "Generic (PLEG): container finished" podID="ffe02c07-7dca-4dc6-b9a2-71096980da67" containerID="9f987a2975c4c708322ab3333444993f64841bd455d7d47ea4fd315d851e5758" exitCode=143 Dec 03 12:47:06 crc kubenswrapper[5052]: I1203 12:47:06.450251 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"ffe02c07-7dca-4dc6-b9a2-71096980da67","Type":"ContainerDied","Data":"cf227852e5ace90c69093b6805395659785574fe0027d072330aed46598ce53e"} Dec 03 12:47:06 crc kubenswrapper[5052]: I1203 12:47:06.450280 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"ffe02c07-7dca-4dc6-b9a2-71096980da67","Type":"ContainerDied","Data":"9f987a2975c4c708322ab3333444993f64841bd455d7d47ea4fd315d851e5758"} Dec 03 12:47:06 crc kubenswrapper[5052]: I1203 12:47:06.636963 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 12:47:06 crc kubenswrapper[5052]: I1203 12:47:06.637653 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/cinder-api-0" podUID="9bb8adcd-621e-489c-8cec-c8965062b1c9" containerName="cinder-api" containerID="cri-o://4dfc52960ffcbb4fcd5ce878bcf6783f646724cbd4fb7ed867685b8bc3a9bff4" gracePeriod=30 Dec 03 12:47:06 crc kubenswrapper[5052]: I1203 12:47:06.639456 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/cinder-api-0" podUID="9bb8adcd-621e-489c-8cec-c8965062b1c9" containerName="cinder-api-log" containerID="cri-o://19e947f84460012632efd07d8c682a8795d68eec61822aabc8b926ea68c6ecb2" gracePeriod=30 Dec 03 12:47:06 crc kubenswrapper[5052]: I1203 12:47:06.659786 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:47:06 crc kubenswrapper[5052]: I1203 12:47:06.728580 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/907ed515-37e6-4cf2-b5a8-1513b8d6b10a-config-data\") pod \"907ed515-37e6-4cf2-b5a8-1513b8d6b10a\" (UID: \"907ed515-37e6-4cf2-b5a8-1513b8d6b10a\") " Dec 03 12:47:06 crc kubenswrapper[5052]: I1203 12:47:06.728751 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"glance\" (UniqueName: \"kubernetes.io/local-volume/local-storage12-crc\") pod \"907ed515-37e6-4cf2-b5a8-1513b8d6b10a\" (UID: \"907ed515-37e6-4cf2-b5a8-1513b8d6b10a\") " Dec 03 12:47:06 crc kubenswrapper[5052]: I1203 12:47:06.728842 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/907ed515-37e6-4cf2-b5a8-1513b8d6b10a-logs\") pod \"907ed515-37e6-4cf2-b5a8-1513b8d6b10a\" (UID: \"907ed515-37e6-4cf2-b5a8-1513b8d6b10a\") " Dec 03 12:47:06 crc kubenswrapper[5052]: I1203 12:47:06.728875 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wl9wh\" (UniqueName: \"kubernetes.io/projected/907ed515-37e6-4cf2-b5a8-1513b8d6b10a-kube-api-access-wl9wh\") pod \"907ed515-37e6-4cf2-b5a8-1513b8d6b10a\" (UID: \"907ed515-37e6-4cf2-b5a8-1513b8d6b10a\") " Dec 03 12:47:06 crc kubenswrapper[5052]: I1203 12:47:06.729766 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/907ed515-37e6-4cf2-b5a8-1513b8d6b10a-scripts\") pod \"907ed515-37e6-4cf2-b5a8-1513b8d6b10a\" (UID: \"907ed515-37e6-4cf2-b5a8-1513b8d6b10a\") " Dec 03 12:47:06 crc kubenswrapper[5052]: I1203 12:47:06.729840 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/907ed515-37e6-4cf2-b5a8-1513b8d6b10a-public-tls-certs\") pod \"907ed515-37e6-4cf2-b5a8-1513b8d6b10a\" (UID: \"907ed515-37e6-4cf2-b5a8-1513b8d6b10a\") " Dec 03 12:47:06 crc kubenswrapper[5052]: I1203 12:47:06.729899 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/907ed515-37e6-4cf2-b5a8-1513b8d6b10a-combined-ca-bundle\") pod \"907ed515-37e6-4cf2-b5a8-1513b8d6b10a\" (UID: \"907ed515-37e6-4cf2-b5a8-1513b8d6b10a\") " Dec 03 12:47:06 crc kubenswrapper[5052]: I1203 12:47:06.729935 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/907ed515-37e6-4cf2-b5a8-1513b8d6b10a-httpd-run\") pod \"907ed515-37e6-4cf2-b5a8-1513b8d6b10a\" (UID: \"907ed515-37e6-4cf2-b5a8-1513b8d6b10a\") " Dec 03 12:47:06 crc kubenswrapper[5052]: I1203 12:47:06.731968 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:47:06 crc kubenswrapper[5052]: I1203 12:47:06.732093 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/907ed515-37e6-4cf2-b5a8-1513b8d6b10a-logs" (OuterVolumeSpecName: "logs") pod "907ed515-37e6-4cf2-b5a8-1513b8d6b10a" (UID: "907ed515-37e6-4cf2-b5a8-1513b8d6b10a"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:47:06 crc kubenswrapper[5052]: I1203 12:47:06.732733 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/907ed515-37e6-4cf2-b5a8-1513b8d6b10a-httpd-run" (OuterVolumeSpecName: "httpd-run") pod "907ed515-37e6-4cf2-b5a8-1513b8d6b10a" (UID: "907ed515-37e6-4cf2-b5a8-1513b8d6b10a"). InnerVolumeSpecName "httpd-run". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:47:06 crc kubenswrapper[5052]: I1203 12:47:06.740647 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/907ed515-37e6-4cf2-b5a8-1513b8d6b10a-scripts" (OuterVolumeSpecName: "scripts") pod "907ed515-37e6-4cf2-b5a8-1513b8d6b10a" (UID: "907ed515-37e6-4cf2-b5a8-1513b8d6b10a"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:47:06 crc kubenswrapper[5052]: I1203 12:47:06.779322 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage12-crc" (OuterVolumeSpecName: "glance") pod "907ed515-37e6-4cf2-b5a8-1513b8d6b10a" (UID: "907ed515-37e6-4cf2-b5a8-1513b8d6b10a"). InnerVolumeSpecName "local-storage12-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 12:47:06 crc kubenswrapper[5052]: I1203 12:47:06.786146 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/907ed515-37e6-4cf2-b5a8-1513b8d6b10a-kube-api-access-wl9wh" (OuterVolumeSpecName: "kube-api-access-wl9wh") pod "907ed515-37e6-4cf2-b5a8-1513b8d6b10a" (UID: "907ed515-37e6-4cf2-b5a8-1513b8d6b10a"). InnerVolumeSpecName "kube-api-access-wl9wh". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:47:06 crc kubenswrapper[5052]: I1203 12:47:06.832857 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/ffe02c07-7dca-4dc6-b9a2-71096980da67-internal-tls-certs\") pod \"ffe02c07-7dca-4dc6-b9a2-71096980da67\" (UID: \"ffe02c07-7dca-4dc6-b9a2-71096980da67\") " Dec 03 12:47:06 crc kubenswrapper[5052]: I1203 12:47:06.832939 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/ffe02c07-7dca-4dc6-b9a2-71096980da67-logs\") pod \"ffe02c07-7dca-4dc6-b9a2-71096980da67\" (UID: \"ffe02c07-7dca-4dc6-b9a2-71096980da67\") " Dec 03 12:47:06 crc kubenswrapper[5052]: I1203 12:47:06.833099 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ffe02c07-7dca-4dc6-b9a2-71096980da67-combined-ca-bundle\") pod \"ffe02c07-7dca-4dc6-b9a2-71096980da67\" (UID: \"ffe02c07-7dca-4dc6-b9a2-71096980da67\") " Dec 03 12:47:06 crc kubenswrapper[5052]: I1203 12:47:06.833132 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"glance\" (UniqueName: \"kubernetes.io/local-volume/local-storage18-crc\") pod \"ffe02c07-7dca-4dc6-b9a2-71096980da67\" (UID: \"ffe02c07-7dca-4dc6-b9a2-71096980da67\") " Dec 03 12:47:06 crc kubenswrapper[5052]: I1203 12:47:06.833166 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/ffe02c07-7dca-4dc6-b9a2-71096980da67-scripts\") pod \"ffe02c07-7dca-4dc6-b9a2-71096980da67\" (UID: \"ffe02c07-7dca-4dc6-b9a2-71096980da67\") " Dec 03 12:47:06 crc kubenswrapper[5052]: I1203 12:47:06.833210 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/ffe02c07-7dca-4dc6-b9a2-71096980da67-config-data\") pod \"ffe02c07-7dca-4dc6-b9a2-71096980da67\" (UID: \"ffe02c07-7dca-4dc6-b9a2-71096980da67\") " Dec 03 12:47:06 crc kubenswrapper[5052]: I1203 12:47:06.833264 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lnjlp\" (UniqueName: \"kubernetes.io/projected/ffe02c07-7dca-4dc6-b9a2-71096980da67-kube-api-access-lnjlp\") pod \"ffe02c07-7dca-4dc6-b9a2-71096980da67\" (UID: \"ffe02c07-7dca-4dc6-b9a2-71096980da67\") " Dec 03 12:47:06 crc kubenswrapper[5052]: I1203 12:47:06.833317 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/ffe02c07-7dca-4dc6-b9a2-71096980da67-httpd-run\") pod \"ffe02c07-7dca-4dc6-b9a2-71096980da67\" (UID: \"ffe02c07-7dca-4dc6-b9a2-71096980da67\") " Dec 03 12:47:06 crc kubenswrapper[5052]: I1203 12:47:06.834572 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ffe02c07-7dca-4dc6-b9a2-71096980da67-logs" (OuterVolumeSpecName: "logs") pod "ffe02c07-7dca-4dc6-b9a2-71096980da67" (UID: "ffe02c07-7dca-4dc6-b9a2-71096980da67"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:47:06 crc kubenswrapper[5052]: I1203 12:47:06.834973 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage12-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage12-crc\") on node \"crc\" " Dec 03 12:47:06 crc kubenswrapper[5052]: I1203 12:47:06.834994 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/ffe02c07-7dca-4dc6-b9a2-71096980da67-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:47:06 crc kubenswrapper[5052]: I1203 12:47:06.835004 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/907ed515-37e6-4cf2-b5a8-1513b8d6b10a-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:47:06 crc kubenswrapper[5052]: I1203 12:47:06.835072 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-wl9wh\" (UniqueName: \"kubernetes.io/projected/907ed515-37e6-4cf2-b5a8-1513b8d6b10a-kube-api-access-wl9wh\") on node \"crc\" DevicePath \"\"" Dec 03 12:47:06 crc kubenswrapper[5052]: I1203 12:47:06.835085 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/907ed515-37e6-4cf2-b5a8-1513b8d6b10a-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:47:06 crc kubenswrapper[5052]: I1203 12:47:06.835097 5052 reconciler_common.go:293] "Volume detached for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/907ed515-37e6-4cf2-b5a8-1513b8d6b10a-httpd-run\") on node \"crc\" DevicePath \"\"" Dec 03 12:47:06 crc kubenswrapper[5052]: I1203 12:47:06.835122 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ffe02c07-7dca-4dc6-b9a2-71096980da67-httpd-run" (OuterVolumeSpecName: "httpd-run") pod "ffe02c07-7dca-4dc6-b9a2-71096980da67" (UID: "ffe02c07-7dca-4dc6-b9a2-71096980da67"). InnerVolumeSpecName "httpd-run". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:47:06 crc kubenswrapper[5052]: I1203 12:47:06.840769 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/907ed515-37e6-4cf2-b5a8-1513b8d6b10a-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "907ed515-37e6-4cf2-b5a8-1513b8d6b10a" (UID: "907ed515-37e6-4cf2-b5a8-1513b8d6b10a"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:47:06 crc kubenswrapper[5052]: I1203 12:47:06.841628 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage18-crc" (OuterVolumeSpecName: "glance") pod "ffe02c07-7dca-4dc6-b9a2-71096980da67" (UID: "ffe02c07-7dca-4dc6-b9a2-71096980da67"). InnerVolumeSpecName "local-storage18-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 12:47:06 crc kubenswrapper[5052]: I1203 12:47:06.844740 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ffe02c07-7dca-4dc6-b9a2-71096980da67-kube-api-access-lnjlp" (OuterVolumeSpecName: "kube-api-access-lnjlp") pod "ffe02c07-7dca-4dc6-b9a2-71096980da67" (UID: "ffe02c07-7dca-4dc6-b9a2-71096980da67"). InnerVolumeSpecName "kube-api-access-lnjlp". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:47:06 crc kubenswrapper[5052]: I1203 12:47:06.858710 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ffe02c07-7dca-4dc6-b9a2-71096980da67-scripts" (OuterVolumeSpecName: "scripts") pod "ffe02c07-7dca-4dc6-b9a2-71096980da67" (UID: "ffe02c07-7dca-4dc6-b9a2-71096980da67"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:47:06 crc kubenswrapper[5052]: I1203 12:47:06.862698 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage12-crc" (UniqueName: "kubernetes.io/local-volume/local-storage12-crc") on node "crc" Dec 03 12:47:06 crc kubenswrapper[5052]: I1203 12:47:06.863869 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/907ed515-37e6-4cf2-b5a8-1513b8d6b10a-config-data" (OuterVolumeSpecName: "config-data") pod "907ed515-37e6-4cf2-b5a8-1513b8d6b10a" (UID: "907ed515-37e6-4cf2-b5a8-1513b8d6b10a"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:47:06 crc kubenswrapper[5052]: I1203 12:47:06.879467 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/907ed515-37e6-4cf2-b5a8-1513b8d6b10a-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "907ed515-37e6-4cf2-b5a8-1513b8d6b10a" (UID: "907ed515-37e6-4cf2-b5a8-1513b8d6b10a"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:47:06 crc kubenswrapper[5052]: I1203 12:47:06.890692 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ffe02c07-7dca-4dc6-b9a2-71096980da67-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "ffe02c07-7dca-4dc6-b9a2-71096980da67" (UID: "ffe02c07-7dca-4dc6-b9a2-71096980da67"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:47:06 crc kubenswrapper[5052]: I1203 12:47:06.912706 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ffe02c07-7dca-4dc6-b9a2-71096980da67-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "ffe02c07-7dca-4dc6-b9a2-71096980da67" (UID: "ffe02c07-7dca-4dc6-b9a2-71096980da67"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:47:06 crc kubenswrapper[5052]: I1203 12:47:06.917744 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ffe02c07-7dca-4dc6-b9a2-71096980da67-config-data" (OuterVolumeSpecName: "config-data") pod "ffe02c07-7dca-4dc6-b9a2-71096980da67" (UID: "ffe02c07-7dca-4dc6-b9a2-71096980da67"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:47:06 crc kubenswrapper[5052]: I1203 12:47:06.937273 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/907ed515-37e6-4cf2-b5a8-1513b8d6b10a-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:47:06 crc kubenswrapper[5052]: I1203 12:47:06.937313 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ffe02c07-7dca-4dc6-b9a2-71096980da67-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:47:06 crc kubenswrapper[5052]: I1203 12:47:06.937347 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage18-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage18-crc\") on node \"crc\" " Dec 03 12:47:06 crc kubenswrapper[5052]: I1203 12:47:06.937363 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/907ed515-37e6-4cf2-b5a8-1513b8d6b10a-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:47:06 crc kubenswrapper[5052]: I1203 12:47:06.937374 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/ffe02c07-7dca-4dc6-b9a2-71096980da67-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:47:06 crc kubenswrapper[5052]: I1203 12:47:06.937385 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/ffe02c07-7dca-4dc6-b9a2-71096980da67-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:47:06 crc kubenswrapper[5052]: I1203 12:47:06.937398 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-lnjlp\" (UniqueName: \"kubernetes.io/projected/ffe02c07-7dca-4dc6-b9a2-71096980da67-kube-api-access-lnjlp\") on node \"crc\" DevicePath \"\"" Dec 03 12:47:06 crc kubenswrapper[5052]: I1203 12:47:06.937409 5052 reconciler_common.go:293] "Volume detached for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/ffe02c07-7dca-4dc6-b9a2-71096980da67-httpd-run\") on node \"crc\" DevicePath \"\"" Dec 03 12:47:06 crc kubenswrapper[5052]: I1203 12:47:06.937420 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/907ed515-37e6-4cf2-b5a8-1513b8d6b10a-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:47:06 crc kubenswrapper[5052]: I1203 12:47:06.937432 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/ffe02c07-7dca-4dc6-b9a2-71096980da67-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:47:06 crc kubenswrapper[5052]: I1203 12:47:06.937443 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage12-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage12-crc\") on node \"crc\" DevicePath \"\"" Dec 03 12:47:06 crc kubenswrapper[5052]: I1203 12:47:06.959310 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage18-crc" (UniqueName: "kubernetes.io/local-volume/local-storage18-crc") on node "crc" Dec 03 12:47:07 crc kubenswrapper[5052]: I1203 12:47:07.039470 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage18-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage18-crc\") on node \"crc\" DevicePath \"\"" Dec 03 12:47:07 crc kubenswrapper[5052]: I1203 12:47:07.483062 5052 generic.go:334] "Generic (PLEG): container finished" podID="9bb8adcd-621e-489c-8cec-c8965062b1c9" containerID="19e947f84460012632efd07d8c682a8795d68eec61822aabc8b926ea68c6ecb2" exitCode=143 Dec 03 12:47:07 crc kubenswrapper[5052]: I1203 12:47:07.483161 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"9bb8adcd-621e-489c-8cec-c8965062b1c9","Type":"ContainerDied","Data":"19e947f84460012632efd07d8c682a8795d68eec61822aabc8b926ea68c6ecb2"} Dec 03 12:47:07 crc kubenswrapper[5052]: I1203 12:47:07.487803 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"907ed515-37e6-4cf2-b5a8-1513b8d6b10a","Type":"ContainerDied","Data":"e07915e77c0024ee3055ed4746eafdf4bdb0fa6f96d475a321adf37d3d3835b6"} Dec 03 12:47:07 crc kubenswrapper[5052]: I1203 12:47:07.487847 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:47:07 crc kubenswrapper[5052]: I1203 12:47:07.487871 5052 scope.go:117] "RemoveContainer" containerID="5a7d6ee55e4c5a722aa4bdb52015036d562245e1ebcdb5aa1403b17740fdfb59" Dec 03 12:47:07 crc kubenswrapper[5052]: I1203 12:47:07.493807 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"ffe02c07-7dca-4dc6-b9a2-71096980da67","Type":"ContainerDied","Data":"deea0040fd270142f385fe61e06a3fcca607700162c942c18e1035a3ae17faec"} Dec 03 12:47:07 crc kubenswrapper[5052]: I1203 12:47:07.493847 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:47:07 crc kubenswrapper[5052]: I1203 12:47:07.530366 5052 scope.go:117] "RemoveContainer" containerID="ad61278101ca96b627cc3c2e0d5d4d7a0579b5dec7926e0f7cf122ebe2243d4d" Dec 03 12:47:07 crc kubenswrapper[5052]: I1203 12:47:07.531597 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:47:07 crc kubenswrapper[5052]: I1203 12:47:07.545404 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:47:07 crc kubenswrapper[5052]: I1203 12:47:07.558303 5052 scope.go:117] "RemoveContainer" containerID="cf227852e5ace90c69093b6805395659785574fe0027d072330aed46598ce53e" Dec 03 12:47:07 crc kubenswrapper[5052]: I1203 12:47:07.560110 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:47:07 crc kubenswrapper[5052]: I1203 12:47:07.571446 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:47:07 crc kubenswrapper[5052]: I1203 12:47:07.584624 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:47:07 crc kubenswrapper[5052]: E1203 12:47:07.585039 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="ffe02c07-7dca-4dc6-b9a2-71096980da67" containerName="glance-log" Dec 03 12:47:07 crc kubenswrapper[5052]: I1203 12:47:07.585057 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="ffe02c07-7dca-4dc6-b9a2-71096980da67" containerName="glance-log" Dec 03 12:47:07 crc kubenswrapper[5052]: E1203 12:47:07.585089 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="907ed515-37e6-4cf2-b5a8-1513b8d6b10a" containerName="glance-httpd" Dec 03 12:47:07 crc kubenswrapper[5052]: I1203 12:47:07.585096 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="907ed515-37e6-4cf2-b5a8-1513b8d6b10a" containerName="glance-httpd" Dec 03 12:47:07 crc kubenswrapper[5052]: E1203 12:47:07.585113 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="1f026d39-713b-4611-80fd-9577e697d0c4" containerName="barbican-api" Dec 03 12:47:07 crc kubenswrapper[5052]: I1203 12:47:07.585120 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f026d39-713b-4611-80fd-9577e697d0c4" containerName="barbican-api" Dec 03 12:47:07 crc kubenswrapper[5052]: E1203 12:47:07.585140 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="907ed515-37e6-4cf2-b5a8-1513b8d6b10a" containerName="glance-log" Dec 03 12:47:07 crc kubenswrapper[5052]: I1203 12:47:07.585147 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="907ed515-37e6-4cf2-b5a8-1513b8d6b10a" containerName="glance-log" Dec 03 12:47:07 crc kubenswrapper[5052]: E1203 12:47:07.585155 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="1f026d39-713b-4611-80fd-9577e697d0c4" containerName="barbican-api-log" Dec 03 12:47:07 crc kubenswrapper[5052]: I1203 12:47:07.585161 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f026d39-713b-4611-80fd-9577e697d0c4" containerName="barbican-api-log" Dec 03 12:47:07 crc kubenswrapper[5052]: E1203 12:47:07.585177 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="ffe02c07-7dca-4dc6-b9a2-71096980da67" containerName="glance-httpd" Dec 03 12:47:07 crc kubenswrapper[5052]: I1203 12:47:07.585187 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="ffe02c07-7dca-4dc6-b9a2-71096980da67" containerName="glance-httpd" Dec 03 12:47:07 crc kubenswrapper[5052]: I1203 12:47:07.585369 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="1f026d39-713b-4611-80fd-9577e697d0c4" containerName="barbican-api" Dec 03 12:47:07 crc kubenswrapper[5052]: I1203 12:47:07.585386 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="ffe02c07-7dca-4dc6-b9a2-71096980da67" containerName="glance-log" Dec 03 12:47:07 crc kubenswrapper[5052]: I1203 12:47:07.585410 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="907ed515-37e6-4cf2-b5a8-1513b8d6b10a" containerName="glance-httpd" Dec 03 12:47:07 crc kubenswrapper[5052]: I1203 12:47:07.585422 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="ffe02c07-7dca-4dc6-b9a2-71096980da67" containerName="glance-httpd" Dec 03 12:47:07 crc kubenswrapper[5052]: I1203 12:47:07.585445 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="907ed515-37e6-4cf2-b5a8-1513b8d6b10a" containerName="glance-log" Dec 03 12:47:07 crc kubenswrapper[5052]: I1203 12:47:07.585457 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="1f026d39-713b-4611-80fd-9577e697d0c4" containerName="barbican-api-log" Dec 03 12:47:07 crc kubenswrapper[5052]: I1203 12:47:07.587329 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:47:07 crc kubenswrapper[5052]: I1203 12:47:07.591832 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-default-external-config-data" Dec 03 12:47:07 crc kubenswrapper[5052]: I1203 12:47:07.596216 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-glance-dockercfg-ggt4k" Dec 03 12:47:07 crc kubenswrapper[5052]: I1203 12:47:07.596485 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-glance-default-public-svc" Dec 03 12:47:07 crc kubenswrapper[5052]: I1203 12:47:07.597428 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-scripts" Dec 03 12:47:07 crc kubenswrapper[5052]: I1203 12:47:07.602666 5052 scope.go:117] "RemoveContainer" containerID="9f987a2975c4c708322ab3333444993f64841bd455d7d47ea4fd315d851e5758" Dec 03 12:47:07 crc kubenswrapper[5052]: I1203 12:47:07.603566 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:47:07 crc kubenswrapper[5052]: I1203 12:47:07.643071 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:47:07 crc kubenswrapper[5052]: I1203 12:47:07.645602 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:47:07 crc kubenswrapper[5052]: I1203 12:47:07.649561 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-default-internal-config-data" Dec 03 12:47:07 crc kubenswrapper[5052]: I1203 12:47:07.650044 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-glance-default-internal-svc" Dec 03 12:47:07 crc kubenswrapper[5052]: I1203 12:47:07.657444 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b1a74998-b009-4bb6-a0dc-d94ffae8b641-combined-ca-bundle\") pod \"glance-default-external-api-0\" (UID: \"b1a74998-b009-4bb6-a0dc-d94ffae8b641\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:47:07 crc kubenswrapper[5052]: I1203 12:47:07.657538 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vhbfz\" (UniqueName: \"kubernetes.io/projected/b1a74998-b009-4bb6-a0dc-d94ffae8b641-kube-api-access-vhbfz\") pod \"glance-default-external-api-0\" (UID: \"b1a74998-b009-4bb6-a0dc-d94ffae8b641\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:47:07 crc kubenswrapper[5052]: I1203 12:47:07.657593 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/b1a74998-b009-4bb6-a0dc-d94ffae8b641-config-data\") pod \"glance-default-external-api-0\" (UID: \"b1a74998-b009-4bb6-a0dc-d94ffae8b641\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:47:07 crc kubenswrapper[5052]: I1203 12:47:07.657640 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/b1a74998-b009-4bb6-a0dc-d94ffae8b641-logs\") pod \"glance-default-external-api-0\" (UID: \"b1a74998-b009-4bb6-a0dc-d94ffae8b641\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:47:07 crc kubenswrapper[5052]: I1203 12:47:07.657801 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/b1a74998-b009-4bb6-a0dc-d94ffae8b641-httpd-run\") pod \"glance-default-external-api-0\" (UID: \"b1a74998-b009-4bb6-a0dc-d94ffae8b641\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:47:07 crc kubenswrapper[5052]: I1203 12:47:07.657848 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage12-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage12-crc\") pod \"glance-default-external-api-0\" (UID: \"b1a74998-b009-4bb6-a0dc-d94ffae8b641\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:47:07 crc kubenswrapper[5052]: I1203 12:47:07.657985 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/b1a74998-b009-4bb6-a0dc-d94ffae8b641-public-tls-certs\") pod \"glance-default-external-api-0\" (UID: \"b1a74998-b009-4bb6-a0dc-d94ffae8b641\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:47:07 crc kubenswrapper[5052]: I1203 12:47:07.658048 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/b1a74998-b009-4bb6-a0dc-d94ffae8b641-scripts\") pod \"glance-default-external-api-0\" (UID: \"b1a74998-b009-4bb6-a0dc-d94ffae8b641\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:47:07 crc kubenswrapper[5052]: I1203 12:47:07.660585 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:47:07 crc kubenswrapper[5052]: I1203 12:47:07.676127 5052 scope.go:117] "RemoveContainer" containerID="2aed1c760a326f3d008799e902b1093313df3764a57c879f0b3686410127d37d" Dec 03 12:47:07 crc kubenswrapper[5052]: E1203 12:47:07.676438 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 12:47:07 crc kubenswrapper[5052]: I1203 12:47:07.693098 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="907ed515-37e6-4cf2-b5a8-1513b8d6b10a" path="/var/lib/kubelet/pods/907ed515-37e6-4cf2-b5a8-1513b8d6b10a/volumes" Dec 03 12:47:07 crc kubenswrapper[5052]: I1203 12:47:07.694110 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ffe02c07-7dca-4dc6-b9a2-71096980da67" path="/var/lib/kubelet/pods/ffe02c07-7dca-4dc6-b9a2-71096980da67/volumes" Dec 03 12:47:07 crc kubenswrapper[5052]: I1203 12:47:07.760417 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/098a46f8-2d49-4fea-a95c-5678601aa164-logs\") pod \"glance-default-internal-api-0\" (UID: \"098a46f8-2d49-4fea-a95c-5678601aa164\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:47:07 crc kubenswrapper[5052]: I1203 12:47:07.760544 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-vhbfz\" (UniqueName: \"kubernetes.io/projected/b1a74998-b009-4bb6-a0dc-d94ffae8b641-kube-api-access-vhbfz\") pod \"glance-default-external-api-0\" (UID: \"b1a74998-b009-4bb6-a0dc-d94ffae8b641\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:47:07 crc kubenswrapper[5052]: I1203 12:47:07.760612 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/b1a74998-b009-4bb6-a0dc-d94ffae8b641-config-data\") pod \"glance-default-external-api-0\" (UID: \"b1a74998-b009-4bb6-a0dc-d94ffae8b641\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:47:07 crc kubenswrapper[5052]: I1203 12:47:07.760648 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/098a46f8-2d49-4fea-a95c-5678601aa164-combined-ca-bundle\") pod \"glance-default-internal-api-0\" (UID: \"098a46f8-2d49-4fea-a95c-5678601aa164\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:47:07 crc kubenswrapper[5052]: I1203 12:47:07.760707 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/b1a74998-b009-4bb6-a0dc-d94ffae8b641-logs\") pod \"glance-default-external-api-0\" (UID: \"b1a74998-b009-4bb6-a0dc-d94ffae8b641\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:47:07 crc kubenswrapper[5052]: I1203 12:47:07.760762 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/098a46f8-2d49-4fea-a95c-5678601aa164-internal-tls-certs\") pod \"glance-default-internal-api-0\" (UID: \"098a46f8-2d49-4fea-a95c-5678601aa164\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:47:07 crc kubenswrapper[5052]: I1203 12:47:07.760810 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jlhbj\" (UniqueName: \"kubernetes.io/projected/098a46f8-2d49-4fea-a95c-5678601aa164-kube-api-access-jlhbj\") pod \"glance-default-internal-api-0\" (UID: \"098a46f8-2d49-4fea-a95c-5678601aa164\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:47:07 crc kubenswrapper[5052]: I1203 12:47:07.760836 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/098a46f8-2d49-4fea-a95c-5678601aa164-scripts\") pod \"glance-default-internal-api-0\" (UID: \"098a46f8-2d49-4fea-a95c-5678601aa164\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:47:07 crc kubenswrapper[5052]: I1203 12:47:07.760876 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage18-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage18-crc\") pod \"glance-default-internal-api-0\" (UID: \"098a46f8-2d49-4fea-a95c-5678601aa164\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:47:07 crc kubenswrapper[5052]: I1203 12:47:07.760924 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/b1a74998-b009-4bb6-a0dc-d94ffae8b641-httpd-run\") pod \"glance-default-external-api-0\" (UID: \"b1a74998-b009-4bb6-a0dc-d94ffae8b641\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:47:07 crc kubenswrapper[5052]: I1203 12:47:07.760982 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage12-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage12-crc\") pod \"glance-default-external-api-0\" (UID: \"b1a74998-b009-4bb6-a0dc-d94ffae8b641\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:47:07 crc kubenswrapper[5052]: I1203 12:47:07.761036 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/098a46f8-2d49-4fea-a95c-5678601aa164-config-data\") pod \"glance-default-internal-api-0\" (UID: \"098a46f8-2d49-4fea-a95c-5678601aa164\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:47:07 crc kubenswrapper[5052]: I1203 12:47:07.761084 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/098a46f8-2d49-4fea-a95c-5678601aa164-httpd-run\") pod \"glance-default-internal-api-0\" (UID: \"098a46f8-2d49-4fea-a95c-5678601aa164\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:47:07 crc kubenswrapper[5052]: I1203 12:47:07.761260 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/b1a74998-b009-4bb6-a0dc-d94ffae8b641-public-tls-certs\") pod \"glance-default-external-api-0\" (UID: \"b1a74998-b009-4bb6-a0dc-d94ffae8b641\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:47:07 crc kubenswrapper[5052]: I1203 12:47:07.761772 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage12-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage12-crc\") pod \"glance-default-external-api-0\" (UID: \"b1a74998-b009-4bb6-a0dc-d94ffae8b641\") device mount path \"/mnt/openstack/pv12\"" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:47:07 crc kubenswrapper[5052]: I1203 12:47:07.761975 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/b1a74998-b009-4bb6-a0dc-d94ffae8b641-logs\") pod \"glance-default-external-api-0\" (UID: \"b1a74998-b009-4bb6-a0dc-d94ffae8b641\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:47:07 crc kubenswrapper[5052]: I1203 12:47:07.762161 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/b1a74998-b009-4bb6-a0dc-d94ffae8b641-scripts\") pod \"glance-default-external-api-0\" (UID: \"b1a74998-b009-4bb6-a0dc-d94ffae8b641\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:47:07 crc kubenswrapper[5052]: I1203 12:47:07.762225 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b1a74998-b009-4bb6-a0dc-d94ffae8b641-combined-ca-bundle\") pod \"glance-default-external-api-0\" (UID: \"b1a74998-b009-4bb6-a0dc-d94ffae8b641\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:47:07 crc kubenswrapper[5052]: I1203 12:47:07.764755 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/b1a74998-b009-4bb6-a0dc-d94ffae8b641-httpd-run\") pod \"glance-default-external-api-0\" (UID: \"b1a74998-b009-4bb6-a0dc-d94ffae8b641\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:47:07 crc kubenswrapper[5052]: I1203 12:47:07.768493 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/b1a74998-b009-4bb6-a0dc-d94ffae8b641-public-tls-certs\") pod \"glance-default-external-api-0\" (UID: \"b1a74998-b009-4bb6-a0dc-d94ffae8b641\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:47:07 crc kubenswrapper[5052]: I1203 12:47:07.768586 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b1a74998-b009-4bb6-a0dc-d94ffae8b641-combined-ca-bundle\") pod \"glance-default-external-api-0\" (UID: \"b1a74998-b009-4bb6-a0dc-d94ffae8b641\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:47:07 crc kubenswrapper[5052]: I1203 12:47:07.768650 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/b1a74998-b009-4bb6-a0dc-d94ffae8b641-scripts\") pod \"glance-default-external-api-0\" (UID: \"b1a74998-b009-4bb6-a0dc-d94ffae8b641\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:47:07 crc kubenswrapper[5052]: I1203 12:47:07.769465 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/b1a74998-b009-4bb6-a0dc-d94ffae8b641-config-data\") pod \"glance-default-external-api-0\" (UID: \"b1a74998-b009-4bb6-a0dc-d94ffae8b641\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:47:07 crc kubenswrapper[5052]: I1203 12:47:07.784625 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-vhbfz\" (UniqueName: \"kubernetes.io/projected/b1a74998-b009-4bb6-a0dc-d94ffae8b641-kube-api-access-vhbfz\") pod \"glance-default-external-api-0\" (UID: \"b1a74998-b009-4bb6-a0dc-d94ffae8b641\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:47:07 crc kubenswrapper[5052]: I1203 12:47:07.838219 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage12-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage12-crc\") pod \"glance-default-external-api-0\" (UID: \"b1a74998-b009-4bb6-a0dc-d94ffae8b641\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:47:07 crc kubenswrapper[5052]: I1203 12:47:07.868966 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/098a46f8-2d49-4fea-a95c-5678601aa164-combined-ca-bundle\") pod \"glance-default-internal-api-0\" (UID: \"098a46f8-2d49-4fea-a95c-5678601aa164\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:47:07 crc kubenswrapper[5052]: I1203 12:47:07.869167 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/098a46f8-2d49-4fea-a95c-5678601aa164-internal-tls-certs\") pod \"glance-default-internal-api-0\" (UID: \"098a46f8-2d49-4fea-a95c-5678601aa164\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:47:07 crc kubenswrapper[5052]: I1203 12:47:07.869207 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-jlhbj\" (UniqueName: \"kubernetes.io/projected/098a46f8-2d49-4fea-a95c-5678601aa164-kube-api-access-jlhbj\") pod \"glance-default-internal-api-0\" (UID: \"098a46f8-2d49-4fea-a95c-5678601aa164\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:47:07 crc kubenswrapper[5052]: I1203 12:47:07.869227 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/098a46f8-2d49-4fea-a95c-5678601aa164-scripts\") pod \"glance-default-internal-api-0\" (UID: \"098a46f8-2d49-4fea-a95c-5678601aa164\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:47:07 crc kubenswrapper[5052]: I1203 12:47:07.869273 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage18-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage18-crc\") pod \"glance-default-internal-api-0\" (UID: \"098a46f8-2d49-4fea-a95c-5678601aa164\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:47:07 crc kubenswrapper[5052]: I1203 12:47:07.869461 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/098a46f8-2d49-4fea-a95c-5678601aa164-config-data\") pod \"glance-default-internal-api-0\" (UID: \"098a46f8-2d49-4fea-a95c-5678601aa164\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:47:07 crc kubenswrapper[5052]: I1203 12:47:07.869523 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/098a46f8-2d49-4fea-a95c-5678601aa164-httpd-run\") pod \"glance-default-internal-api-0\" (UID: \"098a46f8-2d49-4fea-a95c-5678601aa164\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:47:07 crc kubenswrapper[5052]: I1203 12:47:07.869629 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/098a46f8-2d49-4fea-a95c-5678601aa164-logs\") pod \"glance-default-internal-api-0\" (UID: \"098a46f8-2d49-4fea-a95c-5678601aa164\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:47:07 crc kubenswrapper[5052]: I1203 12:47:07.870066 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/098a46f8-2d49-4fea-a95c-5678601aa164-logs\") pod \"glance-default-internal-api-0\" (UID: \"098a46f8-2d49-4fea-a95c-5678601aa164\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:47:07 crc kubenswrapper[5052]: I1203 12:47:07.870241 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage18-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage18-crc\") pod \"glance-default-internal-api-0\" (UID: \"098a46f8-2d49-4fea-a95c-5678601aa164\") device mount path \"/mnt/openstack/pv18\"" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:47:07 crc kubenswrapper[5052]: I1203 12:47:07.874145 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/098a46f8-2d49-4fea-a95c-5678601aa164-internal-tls-certs\") pod \"glance-default-internal-api-0\" (UID: \"098a46f8-2d49-4fea-a95c-5678601aa164\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:47:07 crc kubenswrapper[5052]: I1203 12:47:07.877114 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/098a46f8-2d49-4fea-a95c-5678601aa164-scripts\") pod \"glance-default-internal-api-0\" (UID: \"098a46f8-2d49-4fea-a95c-5678601aa164\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:47:07 crc kubenswrapper[5052]: I1203 12:47:07.877179 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/098a46f8-2d49-4fea-a95c-5678601aa164-config-data\") pod \"glance-default-internal-api-0\" (UID: \"098a46f8-2d49-4fea-a95c-5678601aa164\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:47:07 crc kubenswrapper[5052]: I1203 12:47:07.877552 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/098a46f8-2d49-4fea-a95c-5678601aa164-httpd-run\") pod \"glance-default-internal-api-0\" (UID: \"098a46f8-2d49-4fea-a95c-5678601aa164\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:47:07 crc kubenswrapper[5052]: I1203 12:47:07.881140 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/098a46f8-2d49-4fea-a95c-5678601aa164-combined-ca-bundle\") pod \"glance-default-internal-api-0\" (UID: \"098a46f8-2d49-4fea-a95c-5678601aa164\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:47:07 crc kubenswrapper[5052]: I1203 12:47:07.891590 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-jlhbj\" (UniqueName: \"kubernetes.io/projected/098a46f8-2d49-4fea-a95c-5678601aa164-kube-api-access-jlhbj\") pod \"glance-default-internal-api-0\" (UID: \"098a46f8-2d49-4fea-a95c-5678601aa164\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:47:07 crc kubenswrapper[5052]: I1203 12:47:07.908769 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage18-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage18-crc\") pod \"glance-default-internal-api-0\" (UID: \"098a46f8-2d49-4fea-a95c-5678601aa164\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:47:07 crc kubenswrapper[5052]: I1203 12:47:07.920810 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:47:07 crc kubenswrapper[5052]: I1203 12:47:07.979960 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:47:08 crc kubenswrapper[5052]: I1203 12:47:08.523698 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:47:08 crc kubenswrapper[5052]: I1203 12:47:08.527604 5052 kubelet.go:2542] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:47:08 crc kubenswrapper[5052]: I1203 12:47:08.527663 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:47:08 crc kubenswrapper[5052]: I1203 12:47:08.527678 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:47:08 crc kubenswrapper[5052]: I1203 12:47:08.528636 5052 scope.go:117] "RemoveContainer" containerID="d1e48098c512a28625a35e3a019f961f22ad8e764f883b4f8931a03d713b8c01" Dec 03 12:47:08 crc kubenswrapper[5052]: E1203 12:47:08.529110 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"nova-scheduler-scheduler\" with CrashLoopBackOff: \"back-off 20s restarting failed container=nova-scheduler-scheduler pod=nova-scheduler-0_openstack-kuttl-tests(81709983-1fd0-4446-bbbf-fb94653bb3d2)\"" pod="openstack-kuttl-tests/nova-scheduler-0" podUID="81709983-1fd0-4446-bbbf-fb94653bb3d2" Dec 03 12:47:08 crc kubenswrapper[5052]: I1203 12:47:08.646689 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:47:09 crc kubenswrapper[5052]: I1203 12:47:09.437956 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:47:09 crc kubenswrapper[5052]: I1203 12:47:09.529489 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:47:09 crc kubenswrapper[5052]: I1203 12:47:09.544900 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:47:09 crc kubenswrapper[5052]: I1203 12:47:09.589591 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"b1a74998-b009-4bb6-a0dc-d94ffae8b641","Type":"ContainerStarted","Data":"85ce7bf592dfcc3912055f83c1ee322990e24127bd6cf39edab03198beaa6c50"} Dec 03 12:47:09 crc kubenswrapper[5052]: I1203 12:47:09.606071 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"098a46f8-2d49-4fea-a95c-5678601aa164","Type":"ContainerStarted","Data":"b5f14849b9c0500c19252fb36f88804a02027b9b135c74780ffb794c962f7301"} Dec 03 12:47:09 crc kubenswrapper[5052]: I1203 12:47:09.696828 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:47:09 crc kubenswrapper[5052]: I1203 12:47:09.883111 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/cinder-api-0" podUID="9bb8adcd-621e-489c-8cec-c8965062b1c9" containerName="cinder-api" probeResult="failure" output="Get \"https://10.217.0.54:8776/healthcheck\": read tcp 10.217.0.2:53604->10.217.0.54:8776: read: connection reset by peer" Dec 03 12:47:10 crc kubenswrapper[5052]: I1203 12:47:10.164592 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/keystone-6c4f97778c-d7wtk"] Dec 03 12:47:10 crc kubenswrapper[5052]: I1203 12:47:10.165346 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/keystone-6c4f97778c-d7wtk" podUID="cb2b8abb-bf7a-4e95-87dc-cd980d8bce31" containerName="keystone-api" containerID="cri-o://50ac031a1fe86efac0c23bcb4c777a2da13c93e90ce099262825387ad869bc18" gracePeriod=30 Dec 03 12:47:10 crc kubenswrapper[5052]: I1203 12:47:10.177383 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/keystone-6c4f97778c-d7wtk" podUID="cb2b8abb-bf7a-4e95-87dc-cd980d8bce31" containerName="keystone-api" probeResult="failure" output="Get \"https://10.217.0.104:5000/v3\": read tcp 10.217.0.2:41672->10.217.0.104:5000: read: connection reset by peer" Dec 03 12:47:10 crc kubenswrapper[5052]: I1203 12:47:10.211829 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/keystone-5f884bdbfd-2xzz4"] Dec 03 12:47:10 crc kubenswrapper[5052]: I1203 12:47:10.213643 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-5f884bdbfd-2xzz4" Dec 03 12:47:10 crc kubenswrapper[5052]: I1203 12:47:10.245326 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-5f884bdbfd-2xzz4"] Dec 03 12:47:10 crc kubenswrapper[5052]: I1203 12:47:10.324105 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/neutron-bcd9f4f4-djpt8" Dec 03 12:47:10 crc kubenswrapper[5052]: I1203 12:47:10.377427 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/bff27172-5e30-474d-a49c-389a35dd3052-credential-keys\") pod \"keystone-5f884bdbfd-2xzz4\" (UID: \"bff27172-5e30-474d-a49c-389a35dd3052\") " pod="openstack-kuttl-tests/keystone-5f884bdbfd-2xzz4" Dec 03 12:47:10 crc kubenswrapper[5052]: I1203 12:47:10.377485 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/bff27172-5e30-474d-a49c-389a35dd3052-public-tls-certs\") pod \"keystone-5f884bdbfd-2xzz4\" (UID: \"bff27172-5e30-474d-a49c-389a35dd3052\") " pod="openstack-kuttl-tests/keystone-5f884bdbfd-2xzz4" Dec 03 12:47:10 crc kubenswrapper[5052]: I1203 12:47:10.377589 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/bff27172-5e30-474d-a49c-389a35dd3052-scripts\") pod \"keystone-5f884bdbfd-2xzz4\" (UID: \"bff27172-5e30-474d-a49c-389a35dd3052\") " pod="openstack-kuttl-tests/keystone-5f884bdbfd-2xzz4" Dec 03 12:47:10 crc kubenswrapper[5052]: I1203 12:47:10.377643 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/bff27172-5e30-474d-a49c-389a35dd3052-fernet-keys\") pod \"keystone-5f884bdbfd-2xzz4\" (UID: \"bff27172-5e30-474d-a49c-389a35dd3052\") " pod="openstack-kuttl-tests/keystone-5f884bdbfd-2xzz4" Dec 03 12:47:10 crc kubenswrapper[5052]: I1203 12:47:10.377726 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/bff27172-5e30-474d-a49c-389a35dd3052-combined-ca-bundle\") pod \"keystone-5f884bdbfd-2xzz4\" (UID: \"bff27172-5e30-474d-a49c-389a35dd3052\") " pod="openstack-kuttl-tests/keystone-5f884bdbfd-2xzz4" Dec 03 12:47:10 crc kubenswrapper[5052]: I1203 12:47:10.377750 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/bff27172-5e30-474d-a49c-389a35dd3052-internal-tls-certs\") pod \"keystone-5f884bdbfd-2xzz4\" (UID: \"bff27172-5e30-474d-a49c-389a35dd3052\") " pod="openstack-kuttl-tests/keystone-5f884bdbfd-2xzz4" Dec 03 12:47:10 crc kubenswrapper[5052]: I1203 12:47:10.377862 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ncvgs\" (UniqueName: \"kubernetes.io/projected/bff27172-5e30-474d-a49c-389a35dd3052-kube-api-access-ncvgs\") pod \"keystone-5f884bdbfd-2xzz4\" (UID: \"bff27172-5e30-474d-a49c-389a35dd3052\") " pod="openstack-kuttl-tests/keystone-5f884bdbfd-2xzz4" Dec 03 12:47:10 crc kubenswrapper[5052]: I1203 12:47:10.377928 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/bff27172-5e30-474d-a49c-389a35dd3052-config-data\") pod \"keystone-5f884bdbfd-2xzz4\" (UID: \"bff27172-5e30-474d-a49c-389a35dd3052\") " pod="openstack-kuttl-tests/keystone-5f884bdbfd-2xzz4" Dec 03 12:47:10 crc kubenswrapper[5052]: I1203 12:47:10.449649 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/neutron-bcd9f4f4-djpt8"] Dec 03 12:47:10 crc kubenswrapper[5052]: I1203 12:47:10.479441 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/bff27172-5e30-474d-a49c-389a35dd3052-fernet-keys\") pod \"keystone-5f884bdbfd-2xzz4\" (UID: \"bff27172-5e30-474d-a49c-389a35dd3052\") " pod="openstack-kuttl-tests/keystone-5f884bdbfd-2xzz4" Dec 03 12:47:10 crc kubenswrapper[5052]: I1203 12:47:10.480009 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/bff27172-5e30-474d-a49c-389a35dd3052-combined-ca-bundle\") pod \"keystone-5f884bdbfd-2xzz4\" (UID: \"bff27172-5e30-474d-a49c-389a35dd3052\") " pod="openstack-kuttl-tests/keystone-5f884bdbfd-2xzz4" Dec 03 12:47:10 crc kubenswrapper[5052]: I1203 12:47:10.480111 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/bff27172-5e30-474d-a49c-389a35dd3052-internal-tls-certs\") pod \"keystone-5f884bdbfd-2xzz4\" (UID: \"bff27172-5e30-474d-a49c-389a35dd3052\") " pod="openstack-kuttl-tests/keystone-5f884bdbfd-2xzz4" Dec 03 12:47:10 crc kubenswrapper[5052]: I1203 12:47:10.480287 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-ncvgs\" (UniqueName: \"kubernetes.io/projected/bff27172-5e30-474d-a49c-389a35dd3052-kube-api-access-ncvgs\") pod \"keystone-5f884bdbfd-2xzz4\" (UID: \"bff27172-5e30-474d-a49c-389a35dd3052\") " pod="openstack-kuttl-tests/keystone-5f884bdbfd-2xzz4" Dec 03 12:47:10 crc kubenswrapper[5052]: I1203 12:47:10.480405 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/bff27172-5e30-474d-a49c-389a35dd3052-config-data\") pod \"keystone-5f884bdbfd-2xzz4\" (UID: \"bff27172-5e30-474d-a49c-389a35dd3052\") " pod="openstack-kuttl-tests/keystone-5f884bdbfd-2xzz4" Dec 03 12:47:10 crc kubenswrapper[5052]: I1203 12:47:10.480548 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/bff27172-5e30-474d-a49c-389a35dd3052-credential-keys\") pod \"keystone-5f884bdbfd-2xzz4\" (UID: \"bff27172-5e30-474d-a49c-389a35dd3052\") " pod="openstack-kuttl-tests/keystone-5f884bdbfd-2xzz4" Dec 03 12:47:10 crc kubenswrapper[5052]: I1203 12:47:10.480663 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/bff27172-5e30-474d-a49c-389a35dd3052-public-tls-certs\") pod \"keystone-5f884bdbfd-2xzz4\" (UID: \"bff27172-5e30-474d-a49c-389a35dd3052\") " pod="openstack-kuttl-tests/keystone-5f884bdbfd-2xzz4" Dec 03 12:47:10 crc kubenswrapper[5052]: I1203 12:47:10.480833 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/bff27172-5e30-474d-a49c-389a35dd3052-scripts\") pod \"keystone-5f884bdbfd-2xzz4\" (UID: \"bff27172-5e30-474d-a49c-389a35dd3052\") " pod="openstack-kuttl-tests/keystone-5f884bdbfd-2xzz4" Dec 03 12:47:10 crc kubenswrapper[5052]: I1203 12:47:10.483291 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/bff27172-5e30-474d-a49c-389a35dd3052-scripts\") pod \"keystone-5f884bdbfd-2xzz4\" (UID: \"bff27172-5e30-474d-a49c-389a35dd3052\") " pod="openstack-kuttl-tests/keystone-5f884bdbfd-2xzz4" Dec 03 12:47:10 crc kubenswrapper[5052]: I1203 12:47:10.484041 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/bff27172-5e30-474d-a49c-389a35dd3052-internal-tls-certs\") pod \"keystone-5f884bdbfd-2xzz4\" (UID: \"bff27172-5e30-474d-a49c-389a35dd3052\") " pod="openstack-kuttl-tests/keystone-5f884bdbfd-2xzz4" Dec 03 12:47:10 crc kubenswrapper[5052]: I1203 12:47:10.485086 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/bff27172-5e30-474d-a49c-389a35dd3052-credential-keys\") pod \"keystone-5f884bdbfd-2xzz4\" (UID: \"bff27172-5e30-474d-a49c-389a35dd3052\") " pod="openstack-kuttl-tests/keystone-5f884bdbfd-2xzz4" Dec 03 12:47:10 crc kubenswrapper[5052]: I1203 12:47:10.486254 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/bff27172-5e30-474d-a49c-389a35dd3052-public-tls-certs\") pod \"keystone-5f884bdbfd-2xzz4\" (UID: \"bff27172-5e30-474d-a49c-389a35dd3052\") " pod="openstack-kuttl-tests/keystone-5f884bdbfd-2xzz4" Dec 03 12:47:10 crc kubenswrapper[5052]: I1203 12:47:10.486296 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/neutron-8b544ff76-6hjc7"] Dec 03 12:47:10 crc kubenswrapper[5052]: I1203 12:47:10.489048 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/bff27172-5e30-474d-a49c-389a35dd3052-fernet-keys\") pod \"keystone-5f884bdbfd-2xzz4\" (UID: \"bff27172-5e30-474d-a49c-389a35dd3052\") " pod="openstack-kuttl-tests/keystone-5f884bdbfd-2xzz4" Dec 03 12:47:10 crc kubenswrapper[5052]: I1203 12:47:10.489392 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/bff27172-5e30-474d-a49c-389a35dd3052-config-data\") pod \"keystone-5f884bdbfd-2xzz4\" (UID: \"bff27172-5e30-474d-a49c-389a35dd3052\") " pod="openstack-kuttl-tests/keystone-5f884bdbfd-2xzz4" Dec 03 12:47:10 crc kubenswrapper[5052]: I1203 12:47:10.491047 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-8b544ff76-6hjc7" Dec 03 12:47:10 crc kubenswrapper[5052]: I1203 12:47:10.491227 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/bff27172-5e30-474d-a49c-389a35dd3052-combined-ca-bundle\") pod \"keystone-5f884bdbfd-2xzz4\" (UID: \"bff27172-5e30-474d-a49c-389a35dd3052\") " pod="openstack-kuttl-tests/keystone-5f884bdbfd-2xzz4" Dec 03 12:47:10 crc kubenswrapper[5052]: I1203 12:47:10.508660 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-8b544ff76-6hjc7"] Dec 03 12:47:10 crc kubenswrapper[5052]: I1203 12:47:10.514170 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-ncvgs\" (UniqueName: \"kubernetes.io/projected/bff27172-5e30-474d-a49c-389a35dd3052-kube-api-access-ncvgs\") pod \"keystone-5f884bdbfd-2xzz4\" (UID: \"bff27172-5e30-474d-a49c-389a35dd3052\") " pod="openstack-kuttl-tests/keystone-5f884bdbfd-2xzz4" Dec 03 12:47:10 crc kubenswrapper[5052]: I1203 12:47:10.527651 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:47:10 crc kubenswrapper[5052]: I1203 12:47:10.527993 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-api-0" podUID="d5719b07-205b-4e6a-9fe2-b6531b779b0a" containerName="nova-api-log" containerID="cri-o://32f0eb2434b003178dbbca6a7f974c5933898aa66a832336cc4f079190148796" gracePeriod=30 Dec 03 12:47:10 crc kubenswrapper[5052]: I1203 12:47:10.528039 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-api-0" podUID="d5719b07-205b-4e6a-9fe2-b6531b779b0a" containerName="nova-api-api" containerID="cri-o://033df125bf601aa712f091f9637bf143ce864965096aeccefa080e46608d8437" gracePeriod=30 Dec 03 12:47:10 crc kubenswrapper[5052]: I1203 12:47:10.583590 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/718fd921-0756-4a3e-b383-ed0d0ddf5b72-httpd-config\") pod \"neutron-8b544ff76-6hjc7\" (UID: \"718fd921-0756-4a3e-b383-ed0d0ddf5b72\") " pod="openstack-kuttl-tests/neutron-8b544ff76-6hjc7" Dec 03 12:47:10 crc kubenswrapper[5052]: I1203 12:47:10.583647 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/718fd921-0756-4a3e-b383-ed0d0ddf5b72-public-tls-certs\") pod \"neutron-8b544ff76-6hjc7\" (UID: \"718fd921-0756-4a3e-b383-ed0d0ddf5b72\") " pod="openstack-kuttl-tests/neutron-8b544ff76-6hjc7" Dec 03 12:47:10 crc kubenswrapper[5052]: I1203 12:47:10.583685 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/718fd921-0756-4a3e-b383-ed0d0ddf5b72-config\") pod \"neutron-8b544ff76-6hjc7\" (UID: \"718fd921-0756-4a3e-b383-ed0d0ddf5b72\") " pod="openstack-kuttl-tests/neutron-8b544ff76-6hjc7" Dec 03 12:47:10 crc kubenswrapper[5052]: I1203 12:47:10.583711 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/718fd921-0756-4a3e-b383-ed0d0ddf5b72-combined-ca-bundle\") pod \"neutron-8b544ff76-6hjc7\" (UID: \"718fd921-0756-4a3e-b383-ed0d0ddf5b72\") " pod="openstack-kuttl-tests/neutron-8b544ff76-6hjc7" Dec 03 12:47:10 crc kubenswrapper[5052]: I1203 12:47:10.583755 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tp9bv\" (UniqueName: \"kubernetes.io/projected/718fd921-0756-4a3e-b383-ed0d0ddf5b72-kube-api-access-tp9bv\") pod \"neutron-8b544ff76-6hjc7\" (UID: \"718fd921-0756-4a3e-b383-ed0d0ddf5b72\") " pod="openstack-kuttl-tests/neutron-8b544ff76-6hjc7" Dec 03 12:47:10 crc kubenswrapper[5052]: I1203 12:47:10.583832 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/718fd921-0756-4a3e-b383-ed0d0ddf5b72-internal-tls-certs\") pod \"neutron-8b544ff76-6hjc7\" (UID: \"718fd921-0756-4a3e-b383-ed0d0ddf5b72\") " pod="openstack-kuttl-tests/neutron-8b544ff76-6hjc7" Dec 03 12:47:10 crc kubenswrapper[5052]: I1203 12:47:10.583921 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/718fd921-0756-4a3e-b383-ed0d0ddf5b72-ovndb-tls-certs\") pod \"neutron-8b544ff76-6hjc7\" (UID: \"718fd921-0756-4a3e-b383-ed0d0ddf5b72\") " pod="openstack-kuttl-tests/neutron-8b544ff76-6hjc7" Dec 03 12:47:10 crc kubenswrapper[5052]: I1203 12:47:10.626038 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"098a46f8-2d49-4fea-a95c-5678601aa164","Type":"ContainerStarted","Data":"786d9178f57d4b0d285a0e72f6cd93f8939028d39964463dca3a233c79a6225d"} Dec 03 12:47:10 crc kubenswrapper[5052]: I1203 12:47:10.651177 5052 generic.go:334] "Generic (PLEG): container finished" podID="8669832d-c47a-49bd-9491-dc7141ac6281" containerID="13a4cd2432b46ba6f3a0ba15e8d71c8570f57e964a36ff41f1150e41215a4cc8" exitCode=1 Dec 03 12:47:10 crc kubenswrapper[5052]: I1203 12:47:10.651251 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-conductor-0" event={"ID":"8669832d-c47a-49bd-9491-dc7141ac6281","Type":"ContainerDied","Data":"13a4cd2432b46ba6f3a0ba15e8d71c8570f57e964a36ff41f1150e41215a4cc8"} Dec 03 12:47:10 crc kubenswrapper[5052]: I1203 12:47:10.651286 5052 scope.go:117] "RemoveContainer" containerID="e102925eb6bfd868ed0b7dff92ded1a29e84b2579d3092b133af1f0bf476d04c" Dec 03 12:47:10 crc kubenswrapper[5052]: I1203 12:47:10.654715 5052 scope.go:117] "RemoveContainer" containerID="13a4cd2432b46ba6f3a0ba15e8d71c8570f57e964a36ff41f1150e41215a4cc8" Dec 03 12:47:10 crc kubenswrapper[5052]: E1203 12:47:10.655116 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"nova-cell1-conductor-conductor\" with CrashLoopBackOff: \"back-off 20s restarting failed container=nova-cell1-conductor-conductor pod=nova-cell1-conductor-0_openstack-kuttl-tests(8669832d-c47a-49bd-9491-dc7141ac6281)\"" pod="openstack-kuttl-tests/nova-cell1-conductor-0" podUID="8669832d-c47a-49bd-9491-dc7141ac6281" Dec 03 12:47:10 crc kubenswrapper[5052]: I1203 12:47:10.659494 5052 generic.go:334] "Generic (PLEG): container finished" podID="9bb8adcd-621e-489c-8cec-c8965062b1c9" containerID="4dfc52960ffcbb4fcd5ce878bcf6783f646724cbd4fb7ed867685b8bc3a9bff4" exitCode=0 Dec 03 12:47:10 crc kubenswrapper[5052]: I1203 12:47:10.659572 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"9bb8adcd-621e-489c-8cec-c8965062b1c9","Type":"ContainerDied","Data":"4dfc52960ffcbb4fcd5ce878bcf6783f646724cbd4fb7ed867685b8bc3a9bff4"} Dec 03 12:47:10 crc kubenswrapper[5052]: I1203 12:47:10.661929 5052 generic.go:334] "Generic (PLEG): container finished" podID="cb2b8abb-bf7a-4e95-87dc-cd980d8bce31" containerID="50ac031a1fe86efac0c23bcb4c777a2da13c93e90ce099262825387ad869bc18" exitCode=0 Dec 03 12:47:10 crc kubenswrapper[5052]: I1203 12:47:10.661975 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-6c4f97778c-d7wtk" event={"ID":"cb2b8abb-bf7a-4e95-87dc-cd980d8bce31","Type":"ContainerDied","Data":"50ac031a1fe86efac0c23bcb4c777a2da13c93e90ce099262825387ad869bc18"} Dec 03 12:47:10 crc kubenswrapper[5052]: I1203 12:47:10.663722 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/neutron-bcd9f4f4-djpt8" podUID="049a231b-3a12-4710-9b7a-117b1bb82e72" containerName="neutron-httpd" containerID="cri-o://3349ccb1c242563324485e03dbd9f9320ac9a751d16ea4130a94da5db86659ab" gracePeriod=30 Dec 03 12:47:10 crc kubenswrapper[5052]: I1203 12:47:10.663954 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"b1a74998-b009-4bb6-a0dc-d94ffae8b641","Type":"ContainerStarted","Data":"9a055bdf916ddbec20efe694430ce069882c0ee2f3b4626ae5ce60fc3fd5c826"} Dec 03 12:47:10 crc kubenswrapper[5052]: I1203 12:47:10.664957 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/neutron-bcd9f4f4-djpt8" podUID="049a231b-3a12-4710-9b7a-117b1bb82e72" containerName="neutron-api" containerID="cri-o://25be0ef1cecaae7aecb0b6c706591b1e2a075e2221623da71a4fac6ef2da0c06" gracePeriod=30 Dec 03 12:47:10 crc kubenswrapper[5052]: I1203 12:47:10.682037 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-5f884bdbfd-2xzz4" Dec 03 12:47:10 crc kubenswrapper[5052]: I1203 12:47:10.685790 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/718fd921-0756-4a3e-b383-ed0d0ddf5b72-internal-tls-certs\") pod \"neutron-8b544ff76-6hjc7\" (UID: \"718fd921-0756-4a3e-b383-ed0d0ddf5b72\") " pod="openstack-kuttl-tests/neutron-8b544ff76-6hjc7" Dec 03 12:47:10 crc kubenswrapper[5052]: I1203 12:47:10.685868 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/718fd921-0756-4a3e-b383-ed0d0ddf5b72-ovndb-tls-certs\") pod \"neutron-8b544ff76-6hjc7\" (UID: \"718fd921-0756-4a3e-b383-ed0d0ddf5b72\") " pod="openstack-kuttl-tests/neutron-8b544ff76-6hjc7" Dec 03 12:47:10 crc kubenswrapper[5052]: I1203 12:47:10.685978 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/718fd921-0756-4a3e-b383-ed0d0ddf5b72-httpd-config\") pod \"neutron-8b544ff76-6hjc7\" (UID: \"718fd921-0756-4a3e-b383-ed0d0ddf5b72\") " pod="openstack-kuttl-tests/neutron-8b544ff76-6hjc7" Dec 03 12:47:10 crc kubenswrapper[5052]: I1203 12:47:10.685999 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/718fd921-0756-4a3e-b383-ed0d0ddf5b72-public-tls-certs\") pod \"neutron-8b544ff76-6hjc7\" (UID: \"718fd921-0756-4a3e-b383-ed0d0ddf5b72\") " pod="openstack-kuttl-tests/neutron-8b544ff76-6hjc7" Dec 03 12:47:10 crc kubenswrapper[5052]: I1203 12:47:10.686017 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/718fd921-0756-4a3e-b383-ed0d0ddf5b72-config\") pod \"neutron-8b544ff76-6hjc7\" (UID: \"718fd921-0756-4a3e-b383-ed0d0ddf5b72\") " pod="openstack-kuttl-tests/neutron-8b544ff76-6hjc7" Dec 03 12:47:10 crc kubenswrapper[5052]: I1203 12:47:10.686034 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/718fd921-0756-4a3e-b383-ed0d0ddf5b72-combined-ca-bundle\") pod \"neutron-8b544ff76-6hjc7\" (UID: \"718fd921-0756-4a3e-b383-ed0d0ddf5b72\") " pod="openstack-kuttl-tests/neutron-8b544ff76-6hjc7" Dec 03 12:47:10 crc kubenswrapper[5052]: I1203 12:47:10.686089 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-tp9bv\" (UniqueName: \"kubernetes.io/projected/718fd921-0756-4a3e-b383-ed0d0ddf5b72-kube-api-access-tp9bv\") pod \"neutron-8b544ff76-6hjc7\" (UID: \"718fd921-0756-4a3e-b383-ed0d0ddf5b72\") " pod="openstack-kuttl-tests/neutron-8b544ff76-6hjc7" Dec 03 12:47:10 crc kubenswrapper[5052]: I1203 12:47:10.692914 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/718fd921-0756-4a3e-b383-ed0d0ddf5b72-internal-tls-certs\") pod \"neutron-8b544ff76-6hjc7\" (UID: \"718fd921-0756-4a3e-b383-ed0d0ddf5b72\") " pod="openstack-kuttl-tests/neutron-8b544ff76-6hjc7" Dec 03 12:47:10 crc kubenswrapper[5052]: I1203 12:47:10.693059 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/718fd921-0756-4a3e-b383-ed0d0ddf5b72-config\") pod \"neutron-8b544ff76-6hjc7\" (UID: \"718fd921-0756-4a3e-b383-ed0d0ddf5b72\") " pod="openstack-kuttl-tests/neutron-8b544ff76-6hjc7" Dec 03 12:47:10 crc kubenswrapper[5052]: I1203 12:47:10.693604 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/718fd921-0756-4a3e-b383-ed0d0ddf5b72-public-tls-certs\") pod \"neutron-8b544ff76-6hjc7\" (UID: \"718fd921-0756-4a3e-b383-ed0d0ddf5b72\") " pod="openstack-kuttl-tests/neutron-8b544ff76-6hjc7" Dec 03 12:47:10 crc kubenswrapper[5052]: I1203 12:47:10.695718 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/718fd921-0756-4a3e-b383-ed0d0ddf5b72-httpd-config\") pod \"neutron-8b544ff76-6hjc7\" (UID: \"718fd921-0756-4a3e-b383-ed0d0ddf5b72\") " pod="openstack-kuttl-tests/neutron-8b544ff76-6hjc7" Dec 03 12:47:10 crc kubenswrapper[5052]: I1203 12:47:10.696895 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/718fd921-0756-4a3e-b383-ed0d0ddf5b72-ovndb-tls-certs\") pod \"neutron-8b544ff76-6hjc7\" (UID: \"718fd921-0756-4a3e-b383-ed0d0ddf5b72\") " pod="openstack-kuttl-tests/neutron-8b544ff76-6hjc7" Dec 03 12:47:10 crc kubenswrapper[5052]: I1203 12:47:10.701180 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/718fd921-0756-4a3e-b383-ed0d0ddf5b72-combined-ca-bundle\") pod \"neutron-8b544ff76-6hjc7\" (UID: \"718fd921-0756-4a3e-b383-ed0d0ddf5b72\") " pod="openstack-kuttl-tests/neutron-8b544ff76-6hjc7" Dec 03 12:47:10 crc kubenswrapper[5052]: I1203 12:47:10.706433 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-tp9bv\" (UniqueName: \"kubernetes.io/projected/718fd921-0756-4a3e-b383-ed0d0ddf5b72-kube-api-access-tp9bv\") pod \"neutron-8b544ff76-6hjc7\" (UID: \"718fd921-0756-4a3e-b383-ed0d0ddf5b72\") " pod="openstack-kuttl-tests/neutron-8b544ff76-6hjc7" Dec 03 12:47:10 crc kubenswrapper[5052]: I1203 12:47:10.864581 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-8b544ff76-6hjc7" Dec 03 12:47:11 crc kubenswrapper[5052]: I1203 12:47:11.284456 5052 kubelet.go:2542] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:47:11 crc kubenswrapper[5052]: I1203 12:47:11.325608 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-5f884bdbfd-2xzz4"] Dec 03 12:47:11 crc kubenswrapper[5052]: I1203 12:47:11.330261 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/barbican-api-7764f4c764-tsnp4" Dec 03 12:47:11 crc kubenswrapper[5052]: W1203 12:47:11.344682 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbff27172_5e30_474d_a49c_389a35dd3052.slice/crio-01bee3660667320f46e63fda825172057bae6fd00afdd0ff3d3609b7f3093174 WatchSource:0}: Error finding container 01bee3660667320f46e63fda825172057bae6fd00afdd0ff3d3609b7f3093174: Status 404 returned error can't find the container with id 01bee3660667320f46e63fda825172057bae6fd00afdd0ff3d3609b7f3093174 Dec 03 12:47:11 crc kubenswrapper[5052]: I1203 12:47:11.459819 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/barbican-api-7764f4c764-tsnp4" Dec 03 12:47:11 crc kubenswrapper[5052]: I1203 12:47:11.482775 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:47:11 crc kubenswrapper[5052]: I1203 12:47:11.482817 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:47:11 crc kubenswrapper[5052]: I1203 12:47:11.559870 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-api-6d86f7b884-lf4pd"] Dec 03 12:47:11 crc kubenswrapper[5052]: I1203 12:47:11.561058 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/barbican-api-6d86f7b884-lf4pd" podUID="5b63f869-c95a-4279-b542-3ebbec1a1fd5" containerName="barbican-api" containerID="cri-o://878de2f156357bd531b5738c30192dd837946cd61f68a045b6a3411f524181e4" gracePeriod=30 Dec 03 12:47:11 crc kubenswrapper[5052]: I1203 12:47:11.566626 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/barbican-api-6d86f7b884-lf4pd" podUID="5b63f869-c95a-4279-b542-3ebbec1a1fd5" containerName="barbican-api-log" containerID="cri-o://51b7f4e0f8d805d3cab7483cb57358f70f00e5c62c17634e4a78a7fc68d35bce" gracePeriod=30 Dec 03 12:47:11 crc kubenswrapper[5052]: I1203 12:47:11.617551 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-6c4f97778c-d7wtk" Dec 03 12:47:11 crc kubenswrapper[5052]: I1203 12:47:11.621172 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:47:11 crc kubenswrapper[5052]: I1203 12:47:11.699813 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_neutron-bcd9f4f4-djpt8_049a231b-3a12-4710-9b7a-117b1bb82e72/neutron-api/0.log" Dec 03 12:47:11 crc kubenswrapper[5052]: I1203 12:47:11.699880 5052 generic.go:334] "Generic (PLEG): container finished" podID="049a231b-3a12-4710-9b7a-117b1bb82e72" containerID="3349ccb1c242563324485e03dbd9f9320ac9a751d16ea4130a94da5db86659ab" exitCode=0 Dec 03 12:47:11 crc kubenswrapper[5052]: I1203 12:47:11.708462 5052 scope.go:117] "RemoveContainer" containerID="13a4cd2432b46ba6f3a0ba15e8d71c8570f57e964a36ff41f1150e41215a4cc8" Dec 03 12:47:11 crc kubenswrapper[5052]: E1203 12:47:11.708816 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"nova-cell1-conductor-conductor\" with CrashLoopBackOff: \"back-off 20s restarting failed container=nova-cell1-conductor-conductor pod=nova-cell1-conductor-0_openstack-kuttl-tests(8669832d-c47a-49bd-9491-dc7141ac6281)\"" pod="openstack-kuttl-tests/nova-cell1-conductor-0" podUID="8669832d-c47a-49bd-9491-dc7141ac6281" Dec 03 12:47:11 crc kubenswrapper[5052]: I1203 12:47:11.713078 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-bcd9f4f4-djpt8" event={"ID":"049a231b-3a12-4710-9b7a-117b1bb82e72","Type":"ContainerDied","Data":"3349ccb1c242563324485e03dbd9f9320ac9a751d16ea4130a94da5db86659ab"} Dec 03 12:47:11 crc kubenswrapper[5052]: I1203 12:47:11.714790 5052 generic.go:334] "Generic (PLEG): container finished" podID="d5719b07-205b-4e6a-9fe2-b6531b779b0a" containerID="32f0eb2434b003178dbbca6a7f974c5933898aa66a832336cc4f079190148796" exitCode=143 Dec 03 12:47:11 crc kubenswrapper[5052]: I1203 12:47:11.714856 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"d5719b07-205b-4e6a-9fe2-b6531b779b0a","Type":"ContainerDied","Data":"32f0eb2434b003178dbbca6a7f974c5933898aa66a832336cc4f079190148796"} Dec 03 12:47:11 crc kubenswrapper[5052]: I1203 12:47:11.719390 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"9bb8adcd-621e-489c-8cec-c8965062b1c9","Type":"ContainerDied","Data":"452fba7a09b72cd3c94bf2ba9022e1cf82113ba97bf06a1114ba92b1610e53cd"} Dec 03 12:47:11 crc kubenswrapper[5052]: I1203 12:47:11.719443 5052 scope.go:117] "RemoveContainer" containerID="4dfc52960ffcbb4fcd5ce878bcf6783f646724cbd4fb7ed867685b8bc3a9bff4" Dec 03 12:47:11 crc kubenswrapper[5052]: I1203 12:47:11.719573 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:47:11 crc kubenswrapper[5052]: I1203 12:47:11.722625 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/9bb8adcd-621e-489c-8cec-c8965062b1c9-etc-machine-id\") pod \"9bb8adcd-621e-489c-8cec-c8965062b1c9\" (UID: \"9bb8adcd-621e-489c-8cec-c8965062b1c9\") " Dec 03 12:47:11 crc kubenswrapper[5052]: I1203 12:47:11.722691 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/cb2b8abb-bf7a-4e95-87dc-cd980d8bce31-fernet-keys\") pod \"cb2b8abb-bf7a-4e95-87dc-cd980d8bce31\" (UID: \"cb2b8abb-bf7a-4e95-87dc-cd980d8bce31\") " Dec 03 12:47:11 crc kubenswrapper[5052]: I1203 12:47:11.722754 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/cb2b8abb-bf7a-4e95-87dc-cd980d8bce31-combined-ca-bundle\") pod \"cb2b8abb-bf7a-4e95-87dc-cd980d8bce31\" (UID: \"cb2b8abb-bf7a-4e95-87dc-cd980d8bce31\") " Dec 03 12:47:11 crc kubenswrapper[5052]: I1203 12:47:11.722809 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/9bb8adcd-621e-489c-8cec-c8965062b1c9-config-data-custom\") pod \"9bb8adcd-621e-489c-8cec-c8965062b1c9\" (UID: \"9bb8adcd-621e-489c-8cec-c8965062b1c9\") " Dec 03 12:47:11 crc kubenswrapper[5052]: I1203 12:47:11.722841 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/9bb8adcd-621e-489c-8cec-c8965062b1c9-public-tls-certs\") pod \"9bb8adcd-621e-489c-8cec-c8965062b1c9\" (UID: \"9bb8adcd-621e-489c-8cec-c8965062b1c9\") " Dec 03 12:47:11 crc kubenswrapper[5052]: I1203 12:47:11.722885 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/9bb8adcd-621e-489c-8cec-c8965062b1c9-logs\") pod \"9bb8adcd-621e-489c-8cec-c8965062b1c9\" (UID: \"9bb8adcd-621e-489c-8cec-c8965062b1c9\") " Dec 03 12:47:11 crc kubenswrapper[5052]: I1203 12:47:11.722922 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/cb2b8abb-bf7a-4e95-87dc-cd980d8bce31-internal-tls-certs\") pod \"cb2b8abb-bf7a-4e95-87dc-cd980d8bce31\" (UID: \"cb2b8abb-bf7a-4e95-87dc-cd980d8bce31\") " Dec 03 12:47:11 crc kubenswrapper[5052]: I1203 12:47:11.722962 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/9bb8adcd-621e-489c-8cec-c8965062b1c9-config-data\") pod \"9bb8adcd-621e-489c-8cec-c8965062b1c9\" (UID: \"9bb8adcd-621e-489c-8cec-c8965062b1c9\") " Dec 03 12:47:11 crc kubenswrapper[5052]: I1203 12:47:11.722985 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r68tw\" (UniqueName: \"kubernetes.io/projected/9bb8adcd-621e-489c-8cec-c8965062b1c9-kube-api-access-r68tw\") pod \"9bb8adcd-621e-489c-8cec-c8965062b1c9\" (UID: \"9bb8adcd-621e-489c-8cec-c8965062b1c9\") " Dec 03 12:47:11 crc kubenswrapper[5052]: I1203 12:47:11.723027 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9bb8adcd-621e-489c-8cec-c8965062b1c9-combined-ca-bundle\") pod \"9bb8adcd-621e-489c-8cec-c8965062b1c9\" (UID: \"9bb8adcd-621e-489c-8cec-c8965062b1c9\") " Dec 03 12:47:11 crc kubenswrapper[5052]: I1203 12:47:11.723068 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/cb2b8abb-bf7a-4e95-87dc-cd980d8bce31-public-tls-certs\") pod \"cb2b8abb-bf7a-4e95-87dc-cd980d8bce31\" (UID: \"cb2b8abb-bf7a-4e95-87dc-cd980d8bce31\") " Dec 03 12:47:11 crc kubenswrapper[5052]: I1203 12:47:11.723101 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/9bb8adcd-621e-489c-8cec-c8965062b1c9-internal-tls-certs\") pod \"9bb8adcd-621e-489c-8cec-c8965062b1c9\" (UID: \"9bb8adcd-621e-489c-8cec-c8965062b1c9\") " Dec 03 12:47:11 crc kubenswrapper[5052]: I1203 12:47:11.723119 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jb4nv\" (UniqueName: \"kubernetes.io/projected/cb2b8abb-bf7a-4e95-87dc-cd980d8bce31-kube-api-access-jb4nv\") pod \"cb2b8abb-bf7a-4e95-87dc-cd980d8bce31\" (UID: \"cb2b8abb-bf7a-4e95-87dc-cd980d8bce31\") " Dec 03 12:47:11 crc kubenswrapper[5052]: I1203 12:47:11.723135 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/cb2b8abb-bf7a-4e95-87dc-cd980d8bce31-scripts\") pod \"cb2b8abb-bf7a-4e95-87dc-cd980d8bce31\" (UID: \"cb2b8abb-bf7a-4e95-87dc-cd980d8bce31\") " Dec 03 12:47:11 crc kubenswrapper[5052]: I1203 12:47:11.723151 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/cb2b8abb-bf7a-4e95-87dc-cd980d8bce31-credential-keys\") pod \"cb2b8abb-bf7a-4e95-87dc-cd980d8bce31\" (UID: \"cb2b8abb-bf7a-4e95-87dc-cd980d8bce31\") " Dec 03 12:47:11 crc kubenswrapper[5052]: I1203 12:47:11.723195 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/9bb8adcd-621e-489c-8cec-c8965062b1c9-scripts\") pod \"9bb8adcd-621e-489c-8cec-c8965062b1c9\" (UID: \"9bb8adcd-621e-489c-8cec-c8965062b1c9\") " Dec 03 12:47:11 crc kubenswrapper[5052]: I1203 12:47:11.723260 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/cb2b8abb-bf7a-4e95-87dc-cd980d8bce31-config-data\") pod \"cb2b8abb-bf7a-4e95-87dc-cd980d8bce31\" (UID: \"cb2b8abb-bf7a-4e95-87dc-cd980d8bce31\") " Dec 03 12:47:11 crc kubenswrapper[5052]: I1203 12:47:11.724092 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/9bb8adcd-621e-489c-8cec-c8965062b1c9-etc-machine-id" (OuterVolumeSpecName: "etc-machine-id") pod "9bb8adcd-621e-489c-8cec-c8965062b1c9" (UID: "9bb8adcd-621e-489c-8cec-c8965062b1c9"). InnerVolumeSpecName "etc-machine-id". PluginName "kubernetes.io/host-path", VolumeGidValue "" Dec 03 12:47:11 crc kubenswrapper[5052]: I1203 12:47:11.735373 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cb2b8abb-bf7a-4e95-87dc-cd980d8bce31-credential-keys" (OuterVolumeSpecName: "credential-keys") pod "cb2b8abb-bf7a-4e95-87dc-cd980d8bce31" (UID: "cb2b8abb-bf7a-4e95-87dc-cd980d8bce31"). InnerVolumeSpecName "credential-keys". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:47:11 crc kubenswrapper[5052]: I1203 12:47:11.742373 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9bb8adcd-621e-489c-8cec-c8965062b1c9-logs" (OuterVolumeSpecName: "logs") pod "9bb8adcd-621e-489c-8cec-c8965062b1c9" (UID: "9bb8adcd-621e-489c-8cec-c8965062b1c9"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:47:11 crc kubenswrapper[5052]: I1203 12:47:11.748557 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-5f884bdbfd-2xzz4" event={"ID":"bff27172-5e30-474d-a49c-389a35dd3052","Type":"ContainerStarted","Data":"01bee3660667320f46e63fda825172057bae6fd00afdd0ff3d3609b7f3093174"} Dec 03 12:47:11 crc kubenswrapper[5052]: I1203 12:47:11.773073 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-6c4f97778c-d7wtk" Dec 03 12:47:11 crc kubenswrapper[5052]: I1203 12:47:11.773949 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-6c4f97778c-d7wtk" event={"ID":"cb2b8abb-bf7a-4e95-87dc-cd980d8bce31","Type":"ContainerDied","Data":"74d6a732063e9dc0e9610fcaae0057097ded837585a892d81f869b30a04dd0db"} Dec 03 12:47:11 crc kubenswrapper[5052]: I1203 12:47:11.774190 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cb2b8abb-bf7a-4e95-87dc-cd980d8bce31-scripts" (OuterVolumeSpecName: "scripts") pod "cb2b8abb-bf7a-4e95-87dc-cd980d8bce31" (UID: "cb2b8abb-bf7a-4e95-87dc-cd980d8bce31"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:47:11 crc kubenswrapper[5052]: I1203 12:47:11.777107 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9bb8adcd-621e-489c-8cec-c8965062b1c9-kube-api-access-r68tw" (OuterVolumeSpecName: "kube-api-access-r68tw") pod "9bb8adcd-621e-489c-8cec-c8965062b1c9" (UID: "9bb8adcd-621e-489c-8cec-c8965062b1c9"). InnerVolumeSpecName "kube-api-access-r68tw". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:47:11 crc kubenswrapper[5052]: I1203 12:47:11.798120 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cb2b8abb-bf7a-4e95-87dc-cd980d8bce31-kube-api-access-jb4nv" (OuterVolumeSpecName: "kube-api-access-jb4nv") pod "cb2b8abb-bf7a-4e95-87dc-cd980d8bce31" (UID: "cb2b8abb-bf7a-4e95-87dc-cd980d8bce31"). InnerVolumeSpecName "kube-api-access-jb4nv". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:47:11 crc kubenswrapper[5052]: I1203 12:47:11.802260 5052 scope.go:117] "RemoveContainer" containerID="19e947f84460012632efd07d8c682a8795d68eec61822aabc8b926ea68c6ecb2" Dec 03 12:47:11 crc kubenswrapper[5052]: I1203 12:47:11.807378 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-8b544ff76-6hjc7"] Dec 03 12:47:11 crc kubenswrapper[5052]: I1203 12:47:11.811034 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9bb8adcd-621e-489c-8cec-c8965062b1c9-scripts" (OuterVolumeSpecName: "scripts") pod "9bb8adcd-621e-489c-8cec-c8965062b1c9" (UID: "9bb8adcd-621e-489c-8cec-c8965062b1c9"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:47:11 crc kubenswrapper[5052]: I1203 12:47:11.814106 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cb2b8abb-bf7a-4e95-87dc-cd980d8bce31-fernet-keys" (OuterVolumeSpecName: "fernet-keys") pod "cb2b8abb-bf7a-4e95-87dc-cd980d8bce31" (UID: "cb2b8abb-bf7a-4e95-87dc-cd980d8bce31"). InnerVolumeSpecName "fernet-keys". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:47:11 crc kubenswrapper[5052]: I1203 12:47:11.820117 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9bb8adcd-621e-489c-8cec-c8965062b1c9-config-data-custom" (OuterVolumeSpecName: "config-data-custom") pod "9bb8adcd-621e-489c-8cec-c8965062b1c9" (UID: "9bb8adcd-621e-489c-8cec-c8965062b1c9"). InnerVolumeSpecName "config-data-custom". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:47:11 crc kubenswrapper[5052]: I1203 12:47:11.828142 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-jb4nv\" (UniqueName: \"kubernetes.io/projected/cb2b8abb-bf7a-4e95-87dc-cd980d8bce31-kube-api-access-jb4nv\") on node \"crc\" DevicePath \"\"" Dec 03 12:47:11 crc kubenswrapper[5052]: I1203 12:47:11.828180 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/cb2b8abb-bf7a-4e95-87dc-cd980d8bce31-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:47:11 crc kubenswrapper[5052]: I1203 12:47:11.828194 5052 reconciler_common.go:293] "Volume detached for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/cb2b8abb-bf7a-4e95-87dc-cd980d8bce31-credential-keys\") on node \"crc\" DevicePath \"\"" Dec 03 12:47:11 crc kubenswrapper[5052]: I1203 12:47:11.828206 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/9bb8adcd-621e-489c-8cec-c8965062b1c9-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:47:11 crc kubenswrapper[5052]: I1203 12:47:11.828218 5052 reconciler_common.go:293] "Volume detached for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/9bb8adcd-621e-489c-8cec-c8965062b1c9-etc-machine-id\") on node \"crc\" DevicePath \"\"" Dec 03 12:47:11 crc kubenswrapper[5052]: I1203 12:47:11.828228 5052 reconciler_common.go:293] "Volume detached for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/cb2b8abb-bf7a-4e95-87dc-cd980d8bce31-fernet-keys\") on node \"crc\" DevicePath \"\"" Dec 03 12:47:11 crc kubenswrapper[5052]: I1203 12:47:11.828240 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/9bb8adcd-621e-489c-8cec-c8965062b1c9-config-data-custom\") on node \"crc\" DevicePath \"\"" Dec 03 12:47:11 crc kubenswrapper[5052]: I1203 12:47:11.828253 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/9bb8adcd-621e-489c-8cec-c8965062b1c9-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:47:11 crc kubenswrapper[5052]: I1203 12:47:11.828266 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-r68tw\" (UniqueName: \"kubernetes.io/projected/9bb8adcd-621e-489c-8cec-c8965062b1c9-kube-api-access-r68tw\") on node \"crc\" DevicePath \"\"" Dec 03 12:47:11 crc kubenswrapper[5052]: I1203 12:47:11.867796 5052 scope.go:117] "RemoveContainer" containerID="50ac031a1fe86efac0c23bcb4c777a2da13c93e90ce099262825387ad869bc18" Dec 03 12:47:12 crc kubenswrapper[5052]: I1203 12:47:12.039114 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/neutron-8b544ff76-6hjc7"] Dec 03 12:47:12 crc kubenswrapper[5052]: I1203 12:47:12.090591 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9bb8adcd-621e-489c-8cec-c8965062b1c9-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "9bb8adcd-621e-489c-8cec-c8965062b1c9" (UID: "9bb8adcd-621e-489c-8cec-c8965062b1c9"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:47:12 crc kubenswrapper[5052]: I1203 12:47:12.091725 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/neutron-fdd8f875d-xvq79"] Dec 03 12:47:12 crc kubenswrapper[5052]: E1203 12:47:12.092472 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="9bb8adcd-621e-489c-8cec-c8965062b1c9" containerName="cinder-api-log" Dec 03 12:47:12 crc kubenswrapper[5052]: I1203 12:47:12.092497 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="9bb8adcd-621e-489c-8cec-c8965062b1c9" containerName="cinder-api-log" Dec 03 12:47:12 crc kubenswrapper[5052]: E1203 12:47:12.093657 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="9bb8adcd-621e-489c-8cec-c8965062b1c9" containerName="cinder-api" Dec 03 12:47:12 crc kubenswrapper[5052]: I1203 12:47:12.093678 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="9bb8adcd-621e-489c-8cec-c8965062b1c9" containerName="cinder-api" Dec 03 12:47:12 crc kubenswrapper[5052]: E1203 12:47:12.093722 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="cb2b8abb-bf7a-4e95-87dc-cd980d8bce31" containerName="keystone-api" Dec 03 12:47:12 crc kubenswrapper[5052]: I1203 12:47:12.093730 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="cb2b8abb-bf7a-4e95-87dc-cd980d8bce31" containerName="keystone-api" Dec 03 12:47:12 crc kubenswrapper[5052]: I1203 12:47:12.094159 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="9bb8adcd-621e-489c-8cec-c8965062b1c9" containerName="cinder-api-log" Dec 03 12:47:12 crc kubenswrapper[5052]: I1203 12:47:12.094179 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="cb2b8abb-bf7a-4e95-87dc-cd980d8bce31" containerName="keystone-api" Dec 03 12:47:12 crc kubenswrapper[5052]: I1203 12:47:12.094192 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="9bb8adcd-621e-489c-8cec-c8965062b1c9" containerName="cinder-api" Dec 03 12:47:12 crc kubenswrapper[5052]: I1203 12:47:12.095753 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-fdd8f875d-xvq79" Dec 03 12:47:12 crc kubenswrapper[5052]: I1203 12:47:12.104715 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-fdd8f875d-xvq79"] Dec 03 12:47:12 crc kubenswrapper[5052]: I1203 12:47:12.112893 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cb2b8abb-bf7a-4e95-87dc-cd980d8bce31-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "cb2b8abb-bf7a-4e95-87dc-cd980d8bce31" (UID: "cb2b8abb-bf7a-4e95-87dc-cd980d8bce31"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:47:12 crc kubenswrapper[5052]: I1203 12:47:12.142577 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9bb8adcd-621e-489c-8cec-c8965062b1c9-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:47:12 crc kubenswrapper[5052]: I1203 12:47:12.142620 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/cb2b8abb-bf7a-4e95-87dc-cd980d8bce31-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:47:12 crc kubenswrapper[5052]: I1203 12:47:12.242932 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9bb8adcd-621e-489c-8cec-c8965062b1c9-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "9bb8adcd-621e-489c-8cec-c8965062b1c9" (UID: "9bb8adcd-621e-489c-8cec-c8965062b1c9"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:47:12 crc kubenswrapper[5052]: I1203 12:47:12.243726 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cb2b8abb-bf7a-4e95-87dc-cd980d8bce31-config-data" (OuterVolumeSpecName: "config-data") pod "cb2b8abb-bf7a-4e95-87dc-cd980d8bce31" (UID: "cb2b8abb-bf7a-4e95-87dc-cd980d8bce31"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:47:12 crc kubenswrapper[5052]: I1203 12:47:12.245649 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9bb8adcd-621e-489c-8cec-c8965062b1c9-config-data" (OuterVolumeSpecName: "config-data") pod "9bb8adcd-621e-489c-8cec-c8965062b1c9" (UID: "9bb8adcd-621e-489c-8cec-c8965062b1c9"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:47:12 crc kubenswrapper[5052]: I1203 12:47:12.247793 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/cb2b8abb-bf7a-4e95-87dc-cd980d8bce31-config-data\") pod \"cb2b8abb-bf7a-4e95-87dc-cd980d8bce31\" (UID: \"cb2b8abb-bf7a-4e95-87dc-cd980d8bce31\") " Dec 03 12:47:12 crc kubenswrapper[5052]: W1203 12:47:12.247981 5052 empty_dir.go:500] Warning: Unmount skipped because path does not exist: /var/lib/kubelet/pods/cb2b8abb-bf7a-4e95-87dc-cd980d8bce31/volumes/kubernetes.io~secret/config-data Dec 03 12:47:12 crc kubenswrapper[5052]: I1203 12:47:12.248224 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cb2b8abb-bf7a-4e95-87dc-cd980d8bce31-config-data" (OuterVolumeSpecName: "config-data") pod "cb2b8abb-bf7a-4e95-87dc-cd980d8bce31" (UID: "cb2b8abb-bf7a-4e95-87dc-cd980d8bce31"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:47:12 crc kubenswrapper[5052]: I1203 12:47:12.248326 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cb2b8abb-bf7a-4e95-87dc-cd980d8bce31-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "cb2b8abb-bf7a-4e95-87dc-cd980d8bce31" (UID: "cb2b8abb-bf7a-4e95-87dc-cd980d8bce31"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:47:12 crc kubenswrapper[5052]: I1203 12:47:12.248529 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/9bb8adcd-621e-489c-8cec-c8965062b1c9-config-data\") pod \"9bb8adcd-621e-489c-8cec-c8965062b1c9\" (UID: \"9bb8adcd-621e-489c-8cec-c8965062b1c9\") " Dec 03 12:47:12 crc kubenswrapper[5052]: I1203 12:47:12.248737 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/cb2b8abb-bf7a-4e95-87dc-cd980d8bce31-public-tls-certs\") pod \"cb2b8abb-bf7a-4e95-87dc-cd980d8bce31\" (UID: \"cb2b8abb-bf7a-4e95-87dc-cd980d8bce31\") " Dec 03 12:47:12 crc kubenswrapper[5052]: I1203 12:47:12.248897 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/9bb8adcd-621e-489c-8cec-c8965062b1c9-internal-tls-certs\") pod \"9bb8adcd-621e-489c-8cec-c8965062b1c9\" (UID: \"9bb8adcd-621e-489c-8cec-c8965062b1c9\") " Dec 03 12:47:12 crc kubenswrapper[5052]: I1203 12:47:12.249282 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/77825998-48a8-496e-a523-8e0c371fe532-httpd-config\") pod \"neutron-fdd8f875d-xvq79\" (UID: \"77825998-48a8-496e-a523-8e0c371fe532\") " pod="openstack-kuttl-tests/neutron-fdd8f875d-xvq79" Dec 03 12:47:12 crc kubenswrapper[5052]: I1203 12:47:12.249368 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/77825998-48a8-496e-a523-8e0c371fe532-ovndb-tls-certs\") pod \"neutron-fdd8f875d-xvq79\" (UID: \"77825998-48a8-496e-a523-8e0c371fe532\") " pod="openstack-kuttl-tests/neutron-fdd8f875d-xvq79" Dec 03 12:47:12 crc kubenswrapper[5052]: I1203 12:47:12.249476 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/77825998-48a8-496e-a523-8e0c371fe532-config\") pod \"neutron-fdd8f875d-xvq79\" (UID: \"77825998-48a8-496e-a523-8e0c371fe532\") " pod="openstack-kuttl-tests/neutron-fdd8f875d-xvq79" Dec 03 12:47:12 crc kubenswrapper[5052]: I1203 12:47:12.249605 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/77825998-48a8-496e-a523-8e0c371fe532-combined-ca-bundle\") pod \"neutron-fdd8f875d-xvq79\" (UID: \"77825998-48a8-496e-a523-8e0c371fe532\") " pod="openstack-kuttl-tests/neutron-fdd8f875d-xvq79" Dec 03 12:47:12 crc kubenswrapper[5052]: I1203 12:47:12.249726 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/77825998-48a8-496e-a523-8e0c371fe532-public-tls-certs\") pod \"neutron-fdd8f875d-xvq79\" (UID: \"77825998-48a8-496e-a523-8e0c371fe532\") " pod="openstack-kuttl-tests/neutron-fdd8f875d-xvq79" Dec 03 12:47:12 crc kubenswrapper[5052]: W1203 12:47:12.248662 5052 empty_dir.go:500] Warning: Unmount skipped because path does not exist: /var/lib/kubelet/pods/9bb8adcd-621e-489c-8cec-c8965062b1c9/volumes/kubernetes.io~secret/config-data Dec 03 12:47:12 crc kubenswrapper[5052]: I1203 12:47:12.249885 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9bb8adcd-621e-489c-8cec-c8965062b1c9-config-data" (OuterVolumeSpecName: "config-data") pod "9bb8adcd-621e-489c-8cec-c8965062b1c9" (UID: "9bb8adcd-621e-489c-8cec-c8965062b1c9"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:47:12 crc kubenswrapper[5052]: W1203 12:47:12.248844 5052 empty_dir.go:500] Warning: Unmount skipped because path does not exist: /var/lib/kubelet/pods/cb2b8abb-bf7a-4e95-87dc-cd980d8bce31/volumes/kubernetes.io~secret/public-tls-certs Dec 03 12:47:12 crc kubenswrapper[5052]: I1203 12:47:12.249908 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cb2b8abb-bf7a-4e95-87dc-cd980d8bce31-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "cb2b8abb-bf7a-4e95-87dc-cd980d8bce31" (UID: "cb2b8abb-bf7a-4e95-87dc-cd980d8bce31"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:47:12 crc kubenswrapper[5052]: W1203 12:47:12.249009 5052 empty_dir.go:500] Warning: Unmount skipped because path does not exist: /var/lib/kubelet/pods/9bb8adcd-621e-489c-8cec-c8965062b1c9/volumes/kubernetes.io~secret/internal-tls-certs Dec 03 12:47:12 crc kubenswrapper[5052]: I1203 12:47:12.249923 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9bb8adcd-621e-489c-8cec-c8965062b1c9-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "9bb8adcd-621e-489c-8cec-c8965062b1c9" (UID: "9bb8adcd-621e-489c-8cec-c8965062b1c9"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:47:12 crc kubenswrapper[5052]: I1203 12:47:12.250301 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/77825998-48a8-496e-a523-8e0c371fe532-internal-tls-certs\") pod \"neutron-fdd8f875d-xvq79\" (UID: \"77825998-48a8-496e-a523-8e0c371fe532\") " pod="openstack-kuttl-tests/neutron-fdd8f875d-xvq79" Dec 03 12:47:12 crc kubenswrapper[5052]: I1203 12:47:12.250383 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5mblw\" (UniqueName: \"kubernetes.io/projected/77825998-48a8-496e-a523-8e0c371fe532-kube-api-access-5mblw\") pod \"neutron-fdd8f875d-xvq79\" (UID: \"77825998-48a8-496e-a523-8e0c371fe532\") " pod="openstack-kuttl-tests/neutron-fdd8f875d-xvq79" Dec 03 12:47:12 crc kubenswrapper[5052]: I1203 12:47:12.250731 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/cb2b8abb-bf7a-4e95-87dc-cd980d8bce31-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:47:12 crc kubenswrapper[5052]: I1203 12:47:12.250822 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/9bb8adcd-621e-489c-8cec-c8965062b1c9-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:47:12 crc kubenswrapper[5052]: I1203 12:47:12.250847 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/cb2b8abb-bf7a-4e95-87dc-cd980d8bce31-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:47:12 crc kubenswrapper[5052]: I1203 12:47:12.250868 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/9bb8adcd-621e-489c-8cec-c8965062b1c9-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:47:12 crc kubenswrapper[5052]: I1203 12:47:12.252621 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9bb8adcd-621e-489c-8cec-c8965062b1c9-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "9bb8adcd-621e-489c-8cec-c8965062b1c9" (UID: "9bb8adcd-621e-489c-8cec-c8965062b1c9"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:47:12 crc kubenswrapper[5052]: I1203 12:47:12.281642 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cb2b8abb-bf7a-4e95-87dc-cd980d8bce31-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "cb2b8abb-bf7a-4e95-87dc-cd980d8bce31" (UID: "cb2b8abb-bf7a-4e95-87dc-cd980d8bce31"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:47:12 crc kubenswrapper[5052]: I1203 12:47:12.323938 5052 pod_container_manager_linux.go:210] "Failed to delete cgroup paths" cgroupName=["kubepods","besteffort","podf8ab3054-5998-40c1-852f-be0dbaf022e5"] err="unable to destroy cgroup paths for cgroup [kubepods besteffort podf8ab3054-5998-40c1-852f-be0dbaf022e5] : Timed out while waiting for systemd to remove kubepods-besteffort-podf8ab3054_5998_40c1_852f_be0dbaf022e5.slice" Dec 03 12:47:12 crc kubenswrapper[5052]: E1203 12:47:12.324310 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to delete cgroup paths for [kubepods besteffort podf8ab3054-5998-40c1-852f-be0dbaf022e5] : unable to destroy cgroup paths for cgroup [kubepods besteffort podf8ab3054-5998-40c1-852f-be0dbaf022e5] : Timed out while waiting for systemd to remove kubepods-besteffort-podf8ab3054_5998_40c1_852f_be0dbaf022e5.slice" pod="openstack-kuttl-tests/ovn-northd-0" podUID="f8ab3054-5998-40c1-852f-be0dbaf022e5" Dec 03 12:47:12 crc kubenswrapper[5052]: I1203 12:47:12.353542 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/77825998-48a8-496e-a523-8e0c371fe532-combined-ca-bundle\") pod \"neutron-fdd8f875d-xvq79\" (UID: \"77825998-48a8-496e-a523-8e0c371fe532\") " pod="openstack-kuttl-tests/neutron-fdd8f875d-xvq79" Dec 03 12:47:12 crc kubenswrapper[5052]: I1203 12:47:12.353897 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/77825998-48a8-496e-a523-8e0c371fe532-public-tls-certs\") pod \"neutron-fdd8f875d-xvq79\" (UID: \"77825998-48a8-496e-a523-8e0c371fe532\") " pod="openstack-kuttl-tests/neutron-fdd8f875d-xvq79" Dec 03 12:47:12 crc kubenswrapper[5052]: I1203 12:47:12.354074 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/77825998-48a8-496e-a523-8e0c371fe532-internal-tls-certs\") pod \"neutron-fdd8f875d-xvq79\" (UID: \"77825998-48a8-496e-a523-8e0c371fe532\") " pod="openstack-kuttl-tests/neutron-fdd8f875d-xvq79" Dec 03 12:47:12 crc kubenswrapper[5052]: I1203 12:47:12.354176 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-5mblw\" (UniqueName: \"kubernetes.io/projected/77825998-48a8-496e-a523-8e0c371fe532-kube-api-access-5mblw\") pod \"neutron-fdd8f875d-xvq79\" (UID: \"77825998-48a8-496e-a523-8e0c371fe532\") " pod="openstack-kuttl-tests/neutron-fdd8f875d-xvq79" Dec 03 12:47:12 crc kubenswrapper[5052]: I1203 12:47:12.354296 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/77825998-48a8-496e-a523-8e0c371fe532-httpd-config\") pod \"neutron-fdd8f875d-xvq79\" (UID: \"77825998-48a8-496e-a523-8e0c371fe532\") " pod="openstack-kuttl-tests/neutron-fdd8f875d-xvq79" Dec 03 12:47:12 crc kubenswrapper[5052]: I1203 12:47:12.354321 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/77825998-48a8-496e-a523-8e0c371fe532-ovndb-tls-certs\") pod \"neutron-fdd8f875d-xvq79\" (UID: \"77825998-48a8-496e-a523-8e0c371fe532\") " pod="openstack-kuttl-tests/neutron-fdd8f875d-xvq79" Dec 03 12:47:12 crc kubenswrapper[5052]: I1203 12:47:12.354353 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/77825998-48a8-496e-a523-8e0c371fe532-config\") pod \"neutron-fdd8f875d-xvq79\" (UID: \"77825998-48a8-496e-a523-8e0c371fe532\") " pod="openstack-kuttl-tests/neutron-fdd8f875d-xvq79" Dec 03 12:47:12 crc kubenswrapper[5052]: I1203 12:47:12.354424 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/9bb8adcd-621e-489c-8cec-c8965062b1c9-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:47:12 crc kubenswrapper[5052]: I1203 12:47:12.354466 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/cb2b8abb-bf7a-4e95-87dc-cd980d8bce31-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:47:12 crc kubenswrapper[5052]: I1203 12:47:12.362093 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/77825998-48a8-496e-a523-8e0c371fe532-config\") pod \"neutron-fdd8f875d-xvq79\" (UID: \"77825998-48a8-496e-a523-8e0c371fe532\") " pod="openstack-kuttl-tests/neutron-fdd8f875d-xvq79" Dec 03 12:47:12 crc kubenswrapper[5052]: I1203 12:47:12.363853 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/77825998-48a8-496e-a523-8e0c371fe532-internal-tls-certs\") pod \"neutron-fdd8f875d-xvq79\" (UID: \"77825998-48a8-496e-a523-8e0c371fe532\") " pod="openstack-kuttl-tests/neutron-fdd8f875d-xvq79" Dec 03 12:47:12 crc kubenswrapper[5052]: I1203 12:47:12.368724 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/77825998-48a8-496e-a523-8e0c371fe532-combined-ca-bundle\") pod \"neutron-fdd8f875d-xvq79\" (UID: \"77825998-48a8-496e-a523-8e0c371fe532\") " pod="openstack-kuttl-tests/neutron-fdd8f875d-xvq79" Dec 03 12:47:12 crc kubenswrapper[5052]: I1203 12:47:12.369912 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/77825998-48a8-496e-a523-8e0c371fe532-httpd-config\") pod \"neutron-fdd8f875d-xvq79\" (UID: \"77825998-48a8-496e-a523-8e0c371fe532\") " pod="openstack-kuttl-tests/neutron-fdd8f875d-xvq79" Dec 03 12:47:12 crc kubenswrapper[5052]: I1203 12:47:12.371678 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/77825998-48a8-496e-a523-8e0c371fe532-public-tls-certs\") pod \"neutron-fdd8f875d-xvq79\" (UID: \"77825998-48a8-496e-a523-8e0c371fe532\") " pod="openstack-kuttl-tests/neutron-fdd8f875d-xvq79" Dec 03 12:47:12 crc kubenswrapper[5052]: I1203 12:47:12.389925 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-5mblw\" (UniqueName: \"kubernetes.io/projected/77825998-48a8-496e-a523-8e0c371fe532-kube-api-access-5mblw\") pod \"neutron-fdd8f875d-xvq79\" (UID: \"77825998-48a8-496e-a523-8e0c371fe532\") " pod="openstack-kuttl-tests/neutron-fdd8f875d-xvq79" Dec 03 12:47:12 crc kubenswrapper[5052]: I1203 12:47:12.402204 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/77825998-48a8-496e-a523-8e0c371fe532-ovndb-tls-certs\") pod \"neutron-fdd8f875d-xvq79\" (UID: \"77825998-48a8-496e-a523-8e0c371fe532\") " pod="openstack-kuttl-tests/neutron-fdd8f875d-xvq79" Dec 03 12:47:12 crc kubenswrapper[5052]: I1203 12:47:12.677966 5052 scope.go:117] "RemoveContainer" containerID="4c1d25fd5e7f80502770dbe44896eb5a3327b1bebde080de0e66dead94625111" Dec 03 12:47:12 crc kubenswrapper[5052]: I1203 12:47:12.697477 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-fdd8f875d-xvq79" Dec 03 12:47:12 crc kubenswrapper[5052]: I1203 12:47:12.712800 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 12:47:12 crc kubenswrapper[5052]: I1203 12:47:12.736066 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_neutron-bcd9f4f4-djpt8_049a231b-3a12-4710-9b7a-117b1bb82e72/neutron-api/0.log" Dec 03 12:47:12 crc kubenswrapper[5052]: I1203 12:47:12.736162 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-bcd9f4f4-djpt8" Dec 03 12:47:12 crc kubenswrapper[5052]: I1203 12:47:12.794026 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 12:47:12 crc kubenswrapper[5052]: I1203 12:47:12.810949 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/keystone-6c4f97778c-d7wtk"] Dec 03 12:47:12 crc kubenswrapper[5052]: I1203 12:47:12.830643 5052 generic.go:334] "Generic (PLEG): container finished" podID="5b63f869-c95a-4279-b542-3ebbec1a1fd5" containerID="51b7f4e0f8d805d3cab7483cb57358f70f00e5c62c17634e4a78a7fc68d35bce" exitCode=143 Dec 03 12:47:12 crc kubenswrapper[5052]: I1203 12:47:12.830747 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-6d86f7b884-lf4pd" event={"ID":"5b63f869-c95a-4279-b542-3ebbec1a1fd5","Type":"ContainerDied","Data":"51b7f4e0f8d805d3cab7483cb57358f70f00e5c62c17634e4a78a7fc68d35bce"} Dec 03 12:47:12 crc kubenswrapper[5052]: I1203 12:47:12.833159 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/keystone-6c4f97778c-d7wtk"] Dec 03 12:47:12 crc kubenswrapper[5052]: I1203 12:47:12.838355 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"b1a74998-b009-4bb6-a0dc-d94ffae8b641","Type":"ContainerStarted","Data":"f28ffd80db46452a6cf33fcc0825d49c5f14be470338b023f15a46aa3b004932"} Dec 03 12:47:12 crc kubenswrapper[5052]: I1203 12:47:12.841399 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-8b544ff76-6hjc7" event={"ID":"718fd921-0756-4a3e-b383-ed0d0ddf5b72","Type":"ContainerStarted","Data":"f4563a004dfcdb6bfd12fed5548968c63fc1f68175e452a2c9649613e0d8f5c6"} Dec 03 12:47:12 crc kubenswrapper[5052]: I1203 12:47:12.843008 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"098a46f8-2d49-4fea-a95c-5678601aa164","Type":"ContainerStarted","Data":"3a02ca07996460062957faa0b466285e3ee2c66925c520449ab5df8d346825d8"} Dec 03 12:47:12 crc kubenswrapper[5052]: I1203 12:47:12.846389 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 12:47:12 crc kubenswrapper[5052]: E1203 12:47:12.848030 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="049a231b-3a12-4710-9b7a-117b1bb82e72" containerName="neutron-api" Dec 03 12:47:12 crc kubenswrapper[5052]: I1203 12:47:12.848054 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="049a231b-3a12-4710-9b7a-117b1bb82e72" containerName="neutron-api" Dec 03 12:47:12 crc kubenswrapper[5052]: E1203 12:47:12.848319 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="049a231b-3a12-4710-9b7a-117b1bb82e72" containerName="neutron-httpd" Dec 03 12:47:12 crc kubenswrapper[5052]: I1203 12:47:12.848343 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="049a231b-3a12-4710-9b7a-117b1bb82e72" containerName="neutron-httpd" Dec 03 12:47:12 crc kubenswrapper[5052]: E1203 12:47:12.848368 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="049a231b-3a12-4710-9b7a-117b1bb82e72" containerName="neutron-api" Dec 03 12:47:12 crc kubenswrapper[5052]: I1203 12:47:12.848376 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="049a231b-3a12-4710-9b7a-117b1bb82e72" containerName="neutron-api" Dec 03 12:47:12 crc kubenswrapper[5052]: I1203 12:47:12.848837 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="049a231b-3a12-4710-9b7a-117b1bb82e72" containerName="neutron-httpd" Dec 03 12:47:12 crc kubenswrapper[5052]: I1203 12:47:12.848874 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="049a231b-3a12-4710-9b7a-117b1bb82e72" containerName="neutron-api" Dec 03 12:47:12 crc kubenswrapper[5052]: I1203 12:47:12.850346 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="049a231b-3a12-4710-9b7a-117b1bb82e72" containerName="neutron-api" Dec 03 12:47:12 crc kubenswrapper[5052]: I1203 12:47:12.851138 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_neutron-bcd9f4f4-djpt8_049a231b-3a12-4710-9b7a-117b1bb82e72/neutron-api/0.log" Dec 03 12:47:12 crc kubenswrapper[5052]: I1203 12:47:12.851184 5052 generic.go:334] "Generic (PLEG): container finished" podID="049a231b-3a12-4710-9b7a-117b1bb82e72" containerID="25be0ef1cecaae7aecb0b6c706591b1e2a075e2221623da71a4fac6ef2da0c06" exitCode=0 Dec 03 12:47:12 crc kubenswrapper[5052]: I1203 12:47:12.851269 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-bcd9f4f4-djpt8" Dec 03 12:47:12 crc kubenswrapper[5052]: I1203 12:47:12.851397 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-bcd9f4f4-djpt8" event={"ID":"049a231b-3a12-4710-9b7a-117b1bb82e72","Type":"ContainerDied","Data":"25be0ef1cecaae7aecb0b6c706591b1e2a075e2221623da71a4fac6ef2da0c06"} Dec 03 12:47:12 crc kubenswrapper[5052]: I1203 12:47:12.851451 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-bcd9f4f4-djpt8" event={"ID":"049a231b-3a12-4710-9b7a-117b1bb82e72","Type":"ContainerDied","Data":"9cf710603d4318f916c0abb14b25ef6c5d623e0b2a0ba1023eb1c306bb24b6f0"} Dec 03 12:47:12 crc kubenswrapper[5052]: I1203 12:47:12.851479 5052 scope.go:117] "RemoveContainer" containerID="25be0ef1cecaae7aecb0b6c706591b1e2a075e2221623da71a4fac6ef2da0c06" Dec 03 12:47:12 crc kubenswrapper[5052]: I1203 12:47:12.851679 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:47:12 crc kubenswrapper[5052]: I1203 12:47:12.858386 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-cinder-public-svc" Dec 03 12:47:12 crc kubenswrapper[5052]: I1203 12:47:12.858611 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-cinder-internal-svc" Dec 03 12:47:12 crc kubenswrapper[5052]: I1203 12:47:12.859533 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cinder-api-config-data" Dec 03 12:47:12 crc kubenswrapper[5052]: I1203 12:47:12.862398 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 12:47:12 crc kubenswrapper[5052]: I1203 12:47:12.868441 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zrb86\" (UniqueName: \"kubernetes.io/projected/049a231b-3a12-4710-9b7a-117b1bb82e72-kube-api-access-zrb86\") pod \"049a231b-3a12-4710-9b7a-117b1bb82e72\" (UID: \"049a231b-3a12-4710-9b7a-117b1bb82e72\") " Dec 03 12:47:12 crc kubenswrapper[5052]: I1203 12:47:12.868623 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/049a231b-3a12-4710-9b7a-117b1bb82e72-public-tls-certs\") pod \"049a231b-3a12-4710-9b7a-117b1bb82e72\" (UID: \"049a231b-3a12-4710-9b7a-117b1bb82e72\") " Dec 03 12:47:12 crc kubenswrapper[5052]: I1203 12:47:12.868710 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/049a231b-3a12-4710-9b7a-117b1bb82e72-config\") pod \"049a231b-3a12-4710-9b7a-117b1bb82e72\" (UID: \"049a231b-3a12-4710-9b7a-117b1bb82e72\") " Dec 03 12:47:12 crc kubenswrapper[5052]: I1203 12:47:12.868876 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/049a231b-3a12-4710-9b7a-117b1bb82e72-ovndb-tls-certs\") pod \"049a231b-3a12-4710-9b7a-117b1bb82e72\" (UID: \"049a231b-3a12-4710-9b7a-117b1bb82e72\") " Dec 03 12:47:12 crc kubenswrapper[5052]: I1203 12:47:12.868923 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/049a231b-3a12-4710-9b7a-117b1bb82e72-internal-tls-certs\") pod \"049a231b-3a12-4710-9b7a-117b1bb82e72\" (UID: \"049a231b-3a12-4710-9b7a-117b1bb82e72\") " Dec 03 12:47:12 crc kubenswrapper[5052]: I1203 12:47:12.868946 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/049a231b-3a12-4710-9b7a-117b1bb82e72-httpd-config\") pod \"049a231b-3a12-4710-9b7a-117b1bb82e72\" (UID: \"049a231b-3a12-4710-9b7a-117b1bb82e72\") " Dec 03 12:47:12 crc kubenswrapper[5052]: I1203 12:47:12.868967 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/049a231b-3a12-4710-9b7a-117b1bb82e72-combined-ca-bundle\") pod \"049a231b-3a12-4710-9b7a-117b1bb82e72\" (UID: \"049a231b-3a12-4710-9b7a-117b1bb82e72\") " Dec 03 12:47:12 crc kubenswrapper[5052]: I1203 12:47:12.872794 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/049a231b-3a12-4710-9b7a-117b1bb82e72-kube-api-access-zrb86" (OuterVolumeSpecName: "kube-api-access-zrb86") pod "049a231b-3a12-4710-9b7a-117b1bb82e72" (UID: "049a231b-3a12-4710-9b7a-117b1bb82e72"). InnerVolumeSpecName "kube-api-access-zrb86". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:47:12 crc kubenswrapper[5052]: I1203 12:47:12.876419 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:47:12 crc kubenswrapper[5052]: I1203 12:47:12.877283 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-5f884bdbfd-2xzz4" event={"ID":"bff27172-5e30-474d-a49c-389a35dd3052","Type":"ContainerStarted","Data":"363ded64823f176012eadd8f4efa965cf7bb21ad177ce1c3b4cc513c83f5ebab"} Dec 03 12:47:12 crc kubenswrapper[5052]: I1203 12:47:12.877327 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/keystone-5f884bdbfd-2xzz4" Dec 03 12:47:12 crc kubenswrapper[5052]: I1203 12:47:12.938902 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/glance-default-external-api-0" podStartSLOduration=5.9388770619999995 podStartE2EDuration="5.938877062s" podCreationTimestamp="2025-12-03 12:47:07 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:47:12.864126464 +0000 UTC m=+5223.452368238" watchObservedRunningTime="2025-12-03 12:47:12.938877062 +0000 UTC m=+5223.527118826" Dec 03 12:47:12 crc kubenswrapper[5052]: I1203 12:47:12.961642 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/049a231b-3a12-4710-9b7a-117b1bb82e72-httpd-config" (OuterVolumeSpecName: "httpd-config") pod "049a231b-3a12-4710-9b7a-117b1bb82e72" (UID: "049a231b-3a12-4710-9b7a-117b1bb82e72"). InnerVolumeSpecName "httpd-config". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:47:12 crc kubenswrapper[5052]: I1203 12:47:12.971250 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6fgll\" (UniqueName: \"kubernetes.io/projected/557175a9-81e5-4605-bd67-f1dfa575fb1b-kube-api-access-6fgll\") pod \"cinder-api-0\" (UID: \"557175a9-81e5-4605-bd67-f1dfa575fb1b\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:47:12 crc kubenswrapper[5052]: I1203 12:47:12.971378 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/557175a9-81e5-4605-bd67-f1dfa575fb1b-internal-tls-certs\") pod \"cinder-api-0\" (UID: \"557175a9-81e5-4605-bd67-f1dfa575fb1b\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:47:12 crc kubenswrapper[5052]: I1203 12:47:12.971409 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/557175a9-81e5-4605-bd67-f1dfa575fb1b-config-data\") pod \"cinder-api-0\" (UID: \"557175a9-81e5-4605-bd67-f1dfa575fb1b\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:47:12 crc kubenswrapper[5052]: I1203 12:47:12.971461 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/557175a9-81e5-4605-bd67-f1dfa575fb1b-scripts\") pod \"cinder-api-0\" (UID: \"557175a9-81e5-4605-bd67-f1dfa575fb1b\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:47:12 crc kubenswrapper[5052]: I1203 12:47:12.971540 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/557175a9-81e5-4605-bd67-f1dfa575fb1b-etc-machine-id\") pod \"cinder-api-0\" (UID: \"557175a9-81e5-4605-bd67-f1dfa575fb1b\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:47:12 crc kubenswrapper[5052]: I1203 12:47:12.971564 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/557175a9-81e5-4605-bd67-f1dfa575fb1b-public-tls-certs\") pod \"cinder-api-0\" (UID: \"557175a9-81e5-4605-bd67-f1dfa575fb1b\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:47:12 crc kubenswrapper[5052]: I1203 12:47:12.971782 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/557175a9-81e5-4605-bd67-f1dfa575fb1b-config-data-custom\") pod \"cinder-api-0\" (UID: \"557175a9-81e5-4605-bd67-f1dfa575fb1b\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:47:12 crc kubenswrapper[5052]: I1203 12:47:12.971813 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/557175a9-81e5-4605-bd67-f1dfa575fb1b-combined-ca-bundle\") pod \"cinder-api-0\" (UID: \"557175a9-81e5-4605-bd67-f1dfa575fb1b\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:47:12 crc kubenswrapper[5052]: I1203 12:47:12.971880 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/557175a9-81e5-4605-bd67-f1dfa575fb1b-logs\") pod \"cinder-api-0\" (UID: \"557175a9-81e5-4605-bd67-f1dfa575fb1b\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:47:12 crc kubenswrapper[5052]: I1203 12:47:12.971973 5052 reconciler_common.go:293] "Volume detached for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/049a231b-3a12-4710-9b7a-117b1bb82e72-httpd-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:47:12 crc kubenswrapper[5052]: I1203 12:47:12.971992 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-zrb86\" (UniqueName: \"kubernetes.io/projected/049a231b-3a12-4710-9b7a-117b1bb82e72-kube-api-access-zrb86\") on node \"crc\" DevicePath \"\"" Dec 03 12:47:12 crc kubenswrapper[5052]: I1203 12:47:12.992147 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/glance-default-internal-api-0" podStartSLOduration=5.992125005 podStartE2EDuration="5.992125005s" podCreationTimestamp="2025-12-03 12:47:07 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:47:12.89159833 +0000 UTC m=+5223.479840104" watchObservedRunningTime="2025-12-03 12:47:12.992125005 +0000 UTC m=+5223.580366769" Dec 03 12:47:12 crc kubenswrapper[5052]: I1203 12:47:12.995475 5052 scope.go:117] "RemoveContainer" containerID="3349ccb1c242563324485e03dbd9f9320ac9a751d16ea4130a94da5db86659ab" Dec 03 12:47:13 crc kubenswrapper[5052]: I1203 12:47:13.075219 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-6fgll\" (UniqueName: \"kubernetes.io/projected/557175a9-81e5-4605-bd67-f1dfa575fb1b-kube-api-access-6fgll\") pod \"cinder-api-0\" (UID: \"557175a9-81e5-4605-bd67-f1dfa575fb1b\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:47:13 crc kubenswrapper[5052]: I1203 12:47:13.075306 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/557175a9-81e5-4605-bd67-f1dfa575fb1b-internal-tls-certs\") pod \"cinder-api-0\" (UID: \"557175a9-81e5-4605-bd67-f1dfa575fb1b\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:47:13 crc kubenswrapper[5052]: I1203 12:47:13.075342 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/557175a9-81e5-4605-bd67-f1dfa575fb1b-config-data\") pod \"cinder-api-0\" (UID: \"557175a9-81e5-4605-bd67-f1dfa575fb1b\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:47:13 crc kubenswrapper[5052]: I1203 12:47:13.075400 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/557175a9-81e5-4605-bd67-f1dfa575fb1b-scripts\") pod \"cinder-api-0\" (UID: \"557175a9-81e5-4605-bd67-f1dfa575fb1b\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:47:13 crc kubenswrapper[5052]: I1203 12:47:13.075439 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/557175a9-81e5-4605-bd67-f1dfa575fb1b-etc-machine-id\") pod \"cinder-api-0\" (UID: \"557175a9-81e5-4605-bd67-f1dfa575fb1b\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:47:13 crc kubenswrapper[5052]: I1203 12:47:13.075478 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/557175a9-81e5-4605-bd67-f1dfa575fb1b-public-tls-certs\") pod \"cinder-api-0\" (UID: \"557175a9-81e5-4605-bd67-f1dfa575fb1b\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:47:13 crc kubenswrapper[5052]: I1203 12:47:13.075806 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/557175a9-81e5-4605-bd67-f1dfa575fb1b-config-data-custom\") pod \"cinder-api-0\" (UID: \"557175a9-81e5-4605-bd67-f1dfa575fb1b\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:47:13 crc kubenswrapper[5052]: I1203 12:47:13.075888 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/557175a9-81e5-4605-bd67-f1dfa575fb1b-combined-ca-bundle\") pod \"cinder-api-0\" (UID: \"557175a9-81e5-4605-bd67-f1dfa575fb1b\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:47:13 crc kubenswrapper[5052]: I1203 12:47:13.078542 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/557175a9-81e5-4605-bd67-f1dfa575fb1b-logs\") pod \"cinder-api-0\" (UID: \"557175a9-81e5-4605-bd67-f1dfa575fb1b\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:47:13 crc kubenswrapper[5052]: I1203 12:47:13.079024 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/557175a9-81e5-4605-bd67-f1dfa575fb1b-logs\") pod \"cinder-api-0\" (UID: \"557175a9-81e5-4605-bd67-f1dfa575fb1b\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:47:13 crc kubenswrapper[5052]: I1203 12:47:13.079080 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/557175a9-81e5-4605-bd67-f1dfa575fb1b-etc-machine-id\") pod \"cinder-api-0\" (UID: \"557175a9-81e5-4605-bd67-f1dfa575fb1b\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:47:13 crc kubenswrapper[5052]: I1203 12:47:13.081898 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/557175a9-81e5-4605-bd67-f1dfa575fb1b-scripts\") pod \"cinder-api-0\" (UID: \"557175a9-81e5-4605-bd67-f1dfa575fb1b\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:47:13 crc kubenswrapper[5052]: I1203 12:47:13.087673 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/557175a9-81e5-4605-bd67-f1dfa575fb1b-public-tls-certs\") pod \"cinder-api-0\" (UID: \"557175a9-81e5-4605-bd67-f1dfa575fb1b\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:47:13 crc kubenswrapper[5052]: I1203 12:47:13.096924 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/557175a9-81e5-4605-bd67-f1dfa575fb1b-internal-tls-certs\") pod \"cinder-api-0\" (UID: \"557175a9-81e5-4605-bd67-f1dfa575fb1b\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:47:13 crc kubenswrapper[5052]: I1203 12:47:13.106479 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ovn-northd-0"] Dec 03 12:47:13 crc kubenswrapper[5052]: I1203 12:47:13.107593 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/557175a9-81e5-4605-bd67-f1dfa575fb1b-combined-ca-bundle\") pod \"cinder-api-0\" (UID: \"557175a9-81e5-4605-bd67-f1dfa575fb1b\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:47:13 crc kubenswrapper[5052]: I1203 12:47:13.108688 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/557175a9-81e5-4605-bd67-f1dfa575fb1b-config-data\") pod \"cinder-api-0\" (UID: \"557175a9-81e5-4605-bd67-f1dfa575fb1b\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:47:13 crc kubenswrapper[5052]: I1203 12:47:13.108855 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/049a231b-3a12-4710-9b7a-117b1bb82e72-config" (OuterVolumeSpecName: "config") pod "049a231b-3a12-4710-9b7a-117b1bb82e72" (UID: "049a231b-3a12-4710-9b7a-117b1bb82e72"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:47:13 crc kubenswrapper[5052]: I1203 12:47:13.109116 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/557175a9-81e5-4605-bd67-f1dfa575fb1b-config-data-custom\") pod \"cinder-api-0\" (UID: \"557175a9-81e5-4605-bd67-f1dfa575fb1b\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:47:13 crc kubenswrapper[5052]: I1203 12:47:13.117103 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-6fgll\" (UniqueName: \"kubernetes.io/projected/557175a9-81e5-4605-bd67-f1dfa575fb1b-kube-api-access-6fgll\") pod \"cinder-api-0\" (UID: \"557175a9-81e5-4605-bd67-f1dfa575fb1b\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:47:13 crc kubenswrapper[5052]: I1203 12:47:13.137313 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/049a231b-3a12-4710-9b7a-117b1bb82e72-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "049a231b-3a12-4710-9b7a-117b1bb82e72" (UID: "049a231b-3a12-4710-9b7a-117b1bb82e72"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:47:13 crc kubenswrapper[5052]: I1203 12:47:13.156240 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ovn-northd-0"] Dec 03 12:47:13 crc kubenswrapper[5052]: I1203 12:47:13.178175 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ovn-northd-0"] Dec 03 12:47:13 crc kubenswrapper[5052]: I1203 12:47:13.180733 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:47:13 crc kubenswrapper[5052]: I1203 12:47:13.181024 5052 reconciler_common.go:293] "Volume detached for volume \"ovn-northd-tls-certs\" (UniqueName: \"kubernetes.io/secret/f8ab3054-5998-40c1-852f-be0dbaf022e5-ovn-northd-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:47:13 crc kubenswrapper[5052]: I1203 12:47:13.181119 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/049a231b-3a12-4710-9b7a-117b1bb82e72-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:47:13 crc kubenswrapper[5052]: I1203 12:47:13.181191 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/049a231b-3a12-4710-9b7a-117b1bb82e72-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:47:13 crc kubenswrapper[5052]: I1203 12:47:13.186398 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/keystone-5f884bdbfd-2xzz4" podStartSLOduration=3.186353599 podStartE2EDuration="3.186353599s" podCreationTimestamp="2025-12-03 12:47:10 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:47:13.017232503 +0000 UTC m=+5223.605474267" watchObservedRunningTime="2025-12-03 12:47:13.186353599 +0000 UTC m=+5223.774595373" Dec 03 12:47:13 crc kubenswrapper[5052]: I1203 12:47:13.191167 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-ovnnorthd-ovndbs" Dec 03 12:47:13 crc kubenswrapper[5052]: I1203 12:47:13.191401 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"ovnnorthd-config" Dec 03 12:47:13 crc kubenswrapper[5052]: I1203 12:47:13.191566 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ovnnorthd-ovnnorthd-dockercfg-lhpt7" Dec 03 12:47:13 crc kubenswrapper[5052]: I1203 12:47:13.191745 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"ovnnorthd-scripts" Dec 03 12:47:13 crc kubenswrapper[5052]: I1203 12:47:13.197133 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/049a231b-3a12-4710-9b7a-117b1bb82e72-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "049a231b-3a12-4710-9b7a-117b1bb82e72" (UID: "049a231b-3a12-4710-9b7a-117b1bb82e72"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:47:13 crc kubenswrapper[5052]: I1203 12:47:13.214937 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/049a231b-3a12-4710-9b7a-117b1bb82e72-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "049a231b-3a12-4710-9b7a-117b1bb82e72" (UID: "049a231b-3a12-4710-9b7a-117b1bb82e72"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:47:13 crc kubenswrapper[5052]: I1203 12:47:13.220214 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:47:13 crc kubenswrapper[5052]: I1203 12:47:13.225667 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ovn-northd-0"] Dec 03 12:47:13 crc kubenswrapper[5052]: I1203 12:47:13.246127 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/049a231b-3a12-4710-9b7a-117b1bb82e72-ovndb-tls-certs" (OuterVolumeSpecName: "ovndb-tls-certs") pod "049a231b-3a12-4710-9b7a-117b1bb82e72" (UID: "049a231b-3a12-4710-9b7a-117b1bb82e72"). InnerVolumeSpecName "ovndb-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:47:13 crc kubenswrapper[5052]: I1203 12:47:13.284455 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/342cba80-22e1-488a-bbfb-75b67ca536ac-scripts\") pod \"ovn-northd-0\" (UID: \"342cba80-22e1-488a-bbfb-75b67ca536ac\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:47:13 crc kubenswrapper[5052]: I1203 12:47:13.284614 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/342cba80-22e1-488a-bbfb-75b67ca536ac-combined-ca-bundle\") pod \"ovn-northd-0\" (UID: \"342cba80-22e1-488a-bbfb-75b67ca536ac\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:47:13 crc kubenswrapper[5052]: I1203 12:47:13.284718 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2hxgh\" (UniqueName: \"kubernetes.io/projected/342cba80-22e1-488a-bbfb-75b67ca536ac-kube-api-access-2hxgh\") pod \"ovn-northd-0\" (UID: \"342cba80-22e1-488a-bbfb-75b67ca536ac\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:47:13 crc kubenswrapper[5052]: I1203 12:47:13.284752 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/342cba80-22e1-488a-bbfb-75b67ca536ac-config\") pod \"ovn-northd-0\" (UID: \"342cba80-22e1-488a-bbfb-75b67ca536ac\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:47:13 crc kubenswrapper[5052]: I1203 12:47:13.284826 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-northd-tls-certs\" (UniqueName: \"kubernetes.io/secret/342cba80-22e1-488a-bbfb-75b67ca536ac-ovn-northd-tls-certs\") pod \"ovn-northd-0\" (UID: \"342cba80-22e1-488a-bbfb-75b67ca536ac\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:47:13 crc kubenswrapper[5052]: I1203 12:47:13.284905 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-rundir\" (UniqueName: \"kubernetes.io/empty-dir/342cba80-22e1-488a-bbfb-75b67ca536ac-ovn-rundir\") pod \"ovn-northd-0\" (UID: \"342cba80-22e1-488a-bbfb-75b67ca536ac\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:47:13 crc kubenswrapper[5052]: I1203 12:47:13.284931 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/342cba80-22e1-488a-bbfb-75b67ca536ac-metrics-certs-tls-certs\") pod \"ovn-northd-0\" (UID: \"342cba80-22e1-488a-bbfb-75b67ca536ac\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:47:13 crc kubenswrapper[5052]: I1203 12:47:13.285009 5052 reconciler_common.go:293] "Volume detached for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/049a231b-3a12-4710-9b7a-117b1bb82e72-ovndb-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:47:13 crc kubenswrapper[5052]: I1203 12:47:13.285034 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/049a231b-3a12-4710-9b7a-117b1bb82e72-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:47:13 crc kubenswrapper[5052]: I1203 12:47:13.285046 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/049a231b-3a12-4710-9b7a-117b1bb82e72-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:47:13 crc kubenswrapper[5052]: I1203 12:47:13.388485 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovn-rundir\" (UniqueName: \"kubernetes.io/empty-dir/342cba80-22e1-488a-bbfb-75b67ca536ac-ovn-rundir\") pod \"ovn-northd-0\" (UID: \"342cba80-22e1-488a-bbfb-75b67ca536ac\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:47:13 crc kubenswrapper[5052]: I1203 12:47:13.388566 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/342cba80-22e1-488a-bbfb-75b67ca536ac-metrics-certs-tls-certs\") pod \"ovn-northd-0\" (UID: \"342cba80-22e1-488a-bbfb-75b67ca536ac\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:47:13 crc kubenswrapper[5052]: I1203 12:47:13.388692 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/342cba80-22e1-488a-bbfb-75b67ca536ac-scripts\") pod \"ovn-northd-0\" (UID: \"342cba80-22e1-488a-bbfb-75b67ca536ac\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:47:13 crc kubenswrapper[5052]: I1203 12:47:13.388798 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/342cba80-22e1-488a-bbfb-75b67ca536ac-combined-ca-bundle\") pod \"ovn-northd-0\" (UID: \"342cba80-22e1-488a-bbfb-75b67ca536ac\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:47:13 crc kubenswrapper[5052]: I1203 12:47:13.388896 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-2hxgh\" (UniqueName: \"kubernetes.io/projected/342cba80-22e1-488a-bbfb-75b67ca536ac-kube-api-access-2hxgh\") pod \"ovn-northd-0\" (UID: \"342cba80-22e1-488a-bbfb-75b67ca536ac\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:47:13 crc kubenswrapper[5052]: I1203 12:47:13.388948 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/342cba80-22e1-488a-bbfb-75b67ca536ac-config\") pod \"ovn-northd-0\" (UID: \"342cba80-22e1-488a-bbfb-75b67ca536ac\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:47:13 crc kubenswrapper[5052]: I1203 12:47:13.389037 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovn-northd-tls-certs\" (UniqueName: \"kubernetes.io/secret/342cba80-22e1-488a-bbfb-75b67ca536ac-ovn-northd-tls-certs\") pod \"ovn-northd-0\" (UID: \"342cba80-22e1-488a-bbfb-75b67ca536ac\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:47:13 crc kubenswrapper[5052]: I1203 12:47:13.389356 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovn-rundir\" (UniqueName: \"kubernetes.io/empty-dir/342cba80-22e1-488a-bbfb-75b67ca536ac-ovn-rundir\") pod \"ovn-northd-0\" (UID: \"342cba80-22e1-488a-bbfb-75b67ca536ac\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:47:13 crc kubenswrapper[5052]: I1203 12:47:13.391559 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/342cba80-22e1-488a-bbfb-75b67ca536ac-config\") pod \"ovn-northd-0\" (UID: \"342cba80-22e1-488a-bbfb-75b67ca536ac\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:47:13 crc kubenswrapper[5052]: I1203 12:47:13.394359 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/342cba80-22e1-488a-bbfb-75b67ca536ac-combined-ca-bundle\") pod \"ovn-northd-0\" (UID: \"342cba80-22e1-488a-bbfb-75b67ca536ac\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:47:13 crc kubenswrapper[5052]: I1203 12:47:13.395127 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/342cba80-22e1-488a-bbfb-75b67ca536ac-metrics-certs-tls-certs\") pod \"ovn-northd-0\" (UID: \"342cba80-22e1-488a-bbfb-75b67ca536ac\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:47:13 crc kubenswrapper[5052]: I1203 12:47:13.395230 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/342cba80-22e1-488a-bbfb-75b67ca536ac-scripts\") pod \"ovn-northd-0\" (UID: \"342cba80-22e1-488a-bbfb-75b67ca536ac\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:47:13 crc kubenswrapper[5052]: I1203 12:47:13.396972 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovn-northd-tls-certs\" (UniqueName: \"kubernetes.io/secret/342cba80-22e1-488a-bbfb-75b67ca536ac-ovn-northd-tls-certs\") pod \"ovn-northd-0\" (UID: \"342cba80-22e1-488a-bbfb-75b67ca536ac\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:47:13 crc kubenswrapper[5052]: I1203 12:47:13.403893 5052 scope.go:117] "RemoveContainer" containerID="4b6385cf4545e25cb919fcac9e45a38d9979e76dac85ed14f6bd36d5c92843a5" Dec 03 12:47:13 crc kubenswrapper[5052]: I1203 12:47:13.424439 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-2hxgh\" (UniqueName: \"kubernetes.io/projected/342cba80-22e1-488a-bbfb-75b67ca536ac-kube-api-access-2hxgh\") pod \"ovn-northd-0\" (UID: \"342cba80-22e1-488a-bbfb-75b67ca536ac\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:47:13 crc kubenswrapper[5052]: I1203 12:47:13.476159 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-fdd8f875d-xvq79"] Dec 03 12:47:13 crc kubenswrapper[5052]: I1203 12:47:13.540712 5052 scope.go:117] "RemoveContainer" containerID="25be0ef1cecaae7aecb0b6c706591b1e2a075e2221623da71a4fac6ef2da0c06" Dec 03 12:47:13 crc kubenswrapper[5052]: I1203 12:47:13.544888 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/neutron-bcd9f4f4-djpt8"] Dec 03 12:47:13 crc kubenswrapper[5052]: E1203 12:47:13.546434 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"25be0ef1cecaae7aecb0b6c706591b1e2a075e2221623da71a4fac6ef2da0c06\": container with ID starting with 25be0ef1cecaae7aecb0b6c706591b1e2a075e2221623da71a4fac6ef2da0c06 not found: ID does not exist" containerID="25be0ef1cecaae7aecb0b6c706591b1e2a075e2221623da71a4fac6ef2da0c06" Dec 03 12:47:13 crc kubenswrapper[5052]: I1203 12:47:13.546673 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"25be0ef1cecaae7aecb0b6c706591b1e2a075e2221623da71a4fac6ef2da0c06"} err="failed to get container status \"25be0ef1cecaae7aecb0b6c706591b1e2a075e2221623da71a4fac6ef2da0c06\": rpc error: code = NotFound desc = could not find container \"25be0ef1cecaae7aecb0b6c706591b1e2a075e2221623da71a4fac6ef2da0c06\": container with ID starting with 25be0ef1cecaae7aecb0b6c706591b1e2a075e2221623da71a4fac6ef2da0c06 not found: ID does not exist" Dec 03 12:47:13 crc kubenswrapper[5052]: I1203 12:47:13.546722 5052 scope.go:117] "RemoveContainer" containerID="3349ccb1c242563324485e03dbd9f9320ac9a751d16ea4130a94da5db86659ab" Dec 03 12:47:13 crc kubenswrapper[5052]: E1203 12:47:13.548760 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3349ccb1c242563324485e03dbd9f9320ac9a751d16ea4130a94da5db86659ab\": container with ID starting with 3349ccb1c242563324485e03dbd9f9320ac9a751d16ea4130a94da5db86659ab not found: ID does not exist" containerID="3349ccb1c242563324485e03dbd9f9320ac9a751d16ea4130a94da5db86659ab" Dec 03 12:47:13 crc kubenswrapper[5052]: I1203 12:47:13.548804 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3349ccb1c242563324485e03dbd9f9320ac9a751d16ea4130a94da5db86659ab"} err="failed to get container status \"3349ccb1c242563324485e03dbd9f9320ac9a751d16ea4130a94da5db86659ab\": rpc error: code = NotFound desc = could not find container \"3349ccb1c242563324485e03dbd9f9320ac9a751d16ea4130a94da5db86659ab\": container with ID starting with 3349ccb1c242563324485e03dbd9f9320ac9a751d16ea4130a94da5db86659ab not found: ID does not exist" Dec 03 12:47:13 crc kubenswrapper[5052]: I1203 12:47:13.548839 5052 scope.go:117] "RemoveContainer" containerID="4b6385cf4545e25cb919fcac9e45a38d9979e76dac85ed14f6bd36d5c92843a5" Dec 03 12:47:13 crc kubenswrapper[5052]: E1203 12:47:13.549364 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4b6385cf4545e25cb919fcac9e45a38d9979e76dac85ed14f6bd36d5c92843a5\": container with ID starting with 4b6385cf4545e25cb919fcac9e45a38d9979e76dac85ed14f6bd36d5c92843a5 not found: ID does not exist" containerID="4b6385cf4545e25cb919fcac9e45a38d9979e76dac85ed14f6bd36d5c92843a5" Dec 03 12:47:13 crc kubenswrapper[5052]: I1203 12:47:13.549486 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4b6385cf4545e25cb919fcac9e45a38d9979e76dac85ed14f6bd36d5c92843a5"} err="failed to get container status \"4b6385cf4545e25cb919fcac9e45a38d9979e76dac85ed14f6bd36d5c92843a5\": rpc error: code = NotFound desc = could not find container \"4b6385cf4545e25cb919fcac9e45a38d9979e76dac85ed14f6bd36d5c92843a5\": container with ID starting with 4b6385cf4545e25cb919fcac9e45a38d9979e76dac85ed14f6bd36d5c92843a5 not found: ID does not exist" Dec 03 12:47:13 crc kubenswrapper[5052]: I1203 12:47:13.560378 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/neutron-bcd9f4f4-djpt8"] Dec 03 12:47:13 crc kubenswrapper[5052]: I1203 12:47:13.609572 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 12:47:13 crc kubenswrapper[5052]: I1203 12:47:13.612139 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" podUID="5cafb759-6926-4fd6-a77e-7e18a9becd03" containerName="nova-cell1-novncproxy-novncproxy" containerID="cri-o://a1eac786eb973740a08ecdd82bac9b30836804efc46b172c9ef6ef222f8a88c5" gracePeriod=30 Dec 03 12:47:13 crc kubenswrapper[5052]: I1203 12:47:13.688492 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="049a231b-3a12-4710-9b7a-117b1bb82e72" path="/var/lib/kubelet/pods/049a231b-3a12-4710-9b7a-117b1bb82e72/volumes" Dec 03 12:47:13 crc kubenswrapper[5052]: I1203 12:47:13.689381 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9bb8adcd-621e-489c-8cec-c8965062b1c9" path="/var/lib/kubelet/pods/9bb8adcd-621e-489c-8cec-c8965062b1c9/volumes" Dec 03 12:47:13 crc kubenswrapper[5052]: I1203 12:47:13.690195 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cb2b8abb-bf7a-4e95-87dc-cd980d8bce31" path="/var/lib/kubelet/pods/cb2b8abb-bf7a-4e95-87dc-cd980d8bce31/volumes" Dec 03 12:47:13 crc kubenswrapper[5052]: I1203 12:47:13.691432 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f8ab3054-5998-40c1-852f-be0dbaf022e5" path="/var/lib/kubelet/pods/f8ab3054-5998-40c1-852f-be0dbaf022e5/volumes" Dec 03 12:47:13 crc kubenswrapper[5052]: I1203 12:47:13.716166 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:47:13 crc kubenswrapper[5052]: I1203 12:47:13.796790 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 12:47:13 crc kubenswrapper[5052]: W1203 12:47:13.810904 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod557175a9_81e5_4605_bd67_f1dfa575fb1b.slice/crio-ee4a6851ea72f93c6b9c40f241d2d2bbb83602fe5628f3ddba531a171ed30864 WatchSource:0}: Error finding container ee4a6851ea72f93c6b9c40f241d2d2bbb83602fe5628f3ddba531a171ed30864: Status 404 returned error can't find the container with id ee4a6851ea72f93c6b9c40f241d2d2bbb83602fe5628f3ddba531a171ed30864 Dec 03 12:47:13 crc kubenswrapper[5052]: I1203 12:47:13.901271 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-8b544ff76-6hjc7" event={"ID":"718fd921-0756-4a3e-b383-ed0d0ddf5b72","Type":"ContainerStarted","Data":"454384af2bc2744e1c70e4a67dc88563309e7147062c1b5e0c424094b4fea86d"} Dec 03 12:47:13 crc kubenswrapper[5052]: I1203 12:47:13.901638 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-8b544ff76-6hjc7" event={"ID":"718fd921-0756-4a3e-b383-ed0d0ddf5b72","Type":"ContainerStarted","Data":"8bd50216472e00f32d9e5e953f59c3394da6a32b9f692dcbcfb78a4e404beee3"} Dec 03 12:47:13 crc kubenswrapper[5052]: I1203 12:47:13.901761 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/neutron-8b544ff76-6hjc7" podUID="718fd921-0756-4a3e-b383-ed0d0ddf5b72" containerName="neutron-api" containerID="cri-o://8bd50216472e00f32d9e5e953f59c3394da6a32b9f692dcbcfb78a4e404beee3" gracePeriod=30 Dec 03 12:47:13 crc kubenswrapper[5052]: I1203 12:47:13.902226 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/neutron-8b544ff76-6hjc7" Dec 03 12:47:13 crc kubenswrapper[5052]: I1203 12:47:13.902538 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/neutron-8b544ff76-6hjc7" podUID="718fd921-0756-4a3e-b383-ed0d0ddf5b72" containerName="neutron-httpd" containerID="cri-o://454384af2bc2744e1c70e4a67dc88563309e7147062c1b5e0c424094b4fea86d" gracePeriod=30 Dec 03 12:47:13 crc kubenswrapper[5052]: I1203 12:47:13.914216 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"557175a9-81e5-4605-bd67-f1dfa575fb1b","Type":"ContainerStarted","Data":"ee4a6851ea72f93c6b9c40f241d2d2bbb83602fe5628f3ddba531a171ed30864"} Dec 03 12:47:13 crc kubenswrapper[5052]: I1203 12:47:13.921998 5052 generic.go:334] "Generic (PLEG): container finished" podID="d5719b07-205b-4e6a-9fe2-b6531b779b0a" containerID="033df125bf601aa712f091f9637bf143ce864965096aeccefa080e46608d8437" exitCode=0 Dec 03 12:47:13 crc kubenswrapper[5052]: I1203 12:47:13.922101 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"d5719b07-205b-4e6a-9fe2-b6531b779b0a","Type":"ContainerDied","Data":"033df125bf601aa712f091f9637bf143ce864965096aeccefa080e46608d8437"} Dec 03 12:47:13 crc kubenswrapper[5052]: I1203 12:47:13.942203 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-conductor-0" event={"ID":"6d653682-52e2-4940-af66-633475bfe09a","Type":"ContainerStarted","Data":"1fdbade92a13691b593283a5b263382dfec190075f1201c77fff78a7be754487"} Dec 03 12:47:13 crc kubenswrapper[5052]: I1203 12:47:13.943645 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:47:13 crc kubenswrapper[5052]: I1203 12:47:13.954433 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/neutron-8b544ff76-6hjc7" podStartSLOduration=3.954409405 podStartE2EDuration="3.954409405s" podCreationTimestamp="2025-12-03 12:47:10 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:47:13.920300539 +0000 UTC m=+5224.508542303" watchObservedRunningTime="2025-12-03 12:47:13.954409405 +0000 UTC m=+5224.542651169" Dec 03 12:47:13 crc kubenswrapper[5052]: I1203 12:47:13.993172 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-fdd8f875d-xvq79" event={"ID":"77825998-48a8-496e-a523-8e0c371fe532","Type":"ContainerStarted","Data":"6185634e3cf243590ca0c31348d7b9fa332bb5339856c6af05dece7a198d886f"} Dec 03 12:47:14 crc kubenswrapper[5052]: I1203 12:47:14.367245 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:47:14 crc kubenswrapper[5052]: I1203 12:47:14.412171 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ovn-northd-0"] Dec 03 12:47:14 crc kubenswrapper[5052]: I1203 12:47:14.521053 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d5719b07-205b-4e6a-9fe2-b6531b779b0a-combined-ca-bundle\") pod \"d5719b07-205b-4e6a-9fe2-b6531b779b0a\" (UID: \"d5719b07-205b-4e6a-9fe2-b6531b779b0a\") " Dec 03 12:47:14 crc kubenswrapper[5052]: I1203 12:47:14.521852 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/d5719b07-205b-4e6a-9fe2-b6531b779b0a-logs\") pod \"d5719b07-205b-4e6a-9fe2-b6531b779b0a\" (UID: \"d5719b07-205b-4e6a-9fe2-b6531b779b0a\") " Dec 03 12:47:14 crc kubenswrapper[5052]: I1203 12:47:14.521960 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/d5719b07-205b-4e6a-9fe2-b6531b779b0a-internal-tls-certs\") pod \"d5719b07-205b-4e6a-9fe2-b6531b779b0a\" (UID: \"d5719b07-205b-4e6a-9fe2-b6531b779b0a\") " Dec 03 12:47:14 crc kubenswrapper[5052]: I1203 12:47:14.521995 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/d5719b07-205b-4e6a-9fe2-b6531b779b0a-public-tls-certs\") pod \"d5719b07-205b-4e6a-9fe2-b6531b779b0a\" (UID: \"d5719b07-205b-4e6a-9fe2-b6531b779b0a\") " Dec 03 12:47:14 crc kubenswrapper[5052]: I1203 12:47:14.522107 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xnpkp\" (UniqueName: \"kubernetes.io/projected/d5719b07-205b-4e6a-9fe2-b6531b779b0a-kube-api-access-xnpkp\") pod \"d5719b07-205b-4e6a-9fe2-b6531b779b0a\" (UID: \"d5719b07-205b-4e6a-9fe2-b6531b779b0a\") " Dec 03 12:47:14 crc kubenswrapper[5052]: I1203 12:47:14.522152 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/d5719b07-205b-4e6a-9fe2-b6531b779b0a-config-data\") pod \"d5719b07-205b-4e6a-9fe2-b6531b779b0a\" (UID: \"d5719b07-205b-4e6a-9fe2-b6531b779b0a\") " Dec 03 12:47:14 crc kubenswrapper[5052]: I1203 12:47:14.522273 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d5719b07-205b-4e6a-9fe2-b6531b779b0a-logs" (OuterVolumeSpecName: "logs") pod "d5719b07-205b-4e6a-9fe2-b6531b779b0a" (UID: "d5719b07-205b-4e6a-9fe2-b6531b779b0a"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:47:14 crc kubenswrapper[5052]: I1203 12:47:14.522852 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/d5719b07-205b-4e6a-9fe2-b6531b779b0a-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:47:14 crc kubenswrapper[5052]: I1203 12:47:14.529637 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d5719b07-205b-4e6a-9fe2-b6531b779b0a-kube-api-access-xnpkp" (OuterVolumeSpecName: "kube-api-access-xnpkp") pod "d5719b07-205b-4e6a-9fe2-b6531b779b0a" (UID: "d5719b07-205b-4e6a-9fe2-b6531b779b0a"). InnerVolumeSpecName "kube-api-access-xnpkp". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:47:14 crc kubenswrapper[5052]: I1203 12:47:14.567698 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d5719b07-205b-4e6a-9fe2-b6531b779b0a-config-data" (OuterVolumeSpecName: "config-data") pod "d5719b07-205b-4e6a-9fe2-b6531b779b0a" (UID: "d5719b07-205b-4e6a-9fe2-b6531b779b0a"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:47:14 crc kubenswrapper[5052]: I1203 12:47:14.603504 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d5719b07-205b-4e6a-9fe2-b6531b779b0a-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "d5719b07-205b-4e6a-9fe2-b6531b779b0a" (UID: "d5719b07-205b-4e6a-9fe2-b6531b779b0a"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:47:14 crc kubenswrapper[5052]: I1203 12:47:14.619827 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d5719b07-205b-4e6a-9fe2-b6531b779b0a-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "d5719b07-205b-4e6a-9fe2-b6531b779b0a" (UID: "d5719b07-205b-4e6a-9fe2-b6531b779b0a"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:47:14 crc kubenswrapper[5052]: I1203 12:47:14.626997 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-xnpkp\" (UniqueName: \"kubernetes.io/projected/d5719b07-205b-4e6a-9fe2-b6531b779b0a-kube-api-access-xnpkp\") on node \"crc\" DevicePath \"\"" Dec 03 12:47:14 crc kubenswrapper[5052]: I1203 12:47:14.627044 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/d5719b07-205b-4e6a-9fe2-b6531b779b0a-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:47:14 crc kubenswrapper[5052]: I1203 12:47:14.627057 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d5719b07-205b-4e6a-9fe2-b6531b779b0a-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:47:14 crc kubenswrapper[5052]: I1203 12:47:14.627068 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/d5719b07-205b-4e6a-9fe2-b6531b779b0a-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:47:14 crc kubenswrapper[5052]: I1203 12:47:14.661501 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d5719b07-205b-4e6a-9fe2-b6531b779b0a-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "d5719b07-205b-4e6a-9fe2-b6531b779b0a" (UID: "d5719b07-205b-4e6a-9fe2-b6531b779b0a"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:47:14 crc kubenswrapper[5052]: I1203 12:47:14.710209 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:47:14 crc kubenswrapper[5052]: I1203 12:47:14.712627 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-metadata-0" podUID="75266130-5854-4acd-acc9-bc22accc19bc" containerName="nova-metadata-log" containerID="cri-o://3e1d7a5d05726614ff20caaaa6d897aa3cbb28e2e859ab554810ad90e4b4a72c" gracePeriod=30 Dec 03 12:47:14 crc kubenswrapper[5052]: I1203 12:47:14.712704 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-metadata-0" podUID="75266130-5854-4acd-acc9-bc22accc19bc" containerName="nova-metadata-metadata" containerID="cri-o://f494000fc74b8d824265c717da4442151f3a4e76a4efbba51ec6695872f15bb0" gracePeriod=30 Dec 03 12:47:14 crc kubenswrapper[5052]: I1203 12:47:14.745311 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/d5719b07-205b-4e6a-9fe2-b6531b779b0a-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:47:14 crc kubenswrapper[5052]: I1203 12:47:14.841896 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:47:14 crc kubenswrapper[5052]: I1203 12:47:14.849464 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t5kzv\" (UniqueName: \"kubernetes.io/projected/5cafb759-6926-4fd6-a77e-7e18a9becd03-kube-api-access-t5kzv\") pod \"5cafb759-6926-4fd6-a77e-7e18a9becd03\" (UID: \"5cafb759-6926-4fd6-a77e-7e18a9becd03\") " Dec 03 12:47:14 crc kubenswrapper[5052]: I1203 12:47:14.849646 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"vencrypt-tls-certs\" (UniqueName: \"kubernetes.io/secret/5cafb759-6926-4fd6-a77e-7e18a9becd03-vencrypt-tls-certs\") pod \"5cafb759-6926-4fd6-a77e-7e18a9becd03\" (UID: \"5cafb759-6926-4fd6-a77e-7e18a9becd03\") " Dec 03 12:47:14 crc kubenswrapper[5052]: I1203 12:47:14.849746 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/5cafb759-6926-4fd6-a77e-7e18a9becd03-config-data\") pod \"5cafb759-6926-4fd6-a77e-7e18a9becd03\" (UID: \"5cafb759-6926-4fd6-a77e-7e18a9becd03\") " Dec 03 12:47:14 crc kubenswrapper[5052]: I1203 12:47:14.849909 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5cafb759-6926-4fd6-a77e-7e18a9becd03-combined-ca-bundle\") pod \"5cafb759-6926-4fd6-a77e-7e18a9becd03\" (UID: \"5cafb759-6926-4fd6-a77e-7e18a9becd03\") " Dec 03 12:47:14 crc kubenswrapper[5052]: I1203 12:47:14.850011 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"nova-novncproxy-tls-certs\" (UniqueName: \"kubernetes.io/secret/5cafb759-6926-4fd6-a77e-7e18a9becd03-nova-novncproxy-tls-certs\") pod \"5cafb759-6926-4fd6-a77e-7e18a9becd03\" (UID: \"5cafb759-6926-4fd6-a77e-7e18a9becd03\") " Dec 03 12:47:14 crc kubenswrapper[5052]: I1203 12:47:14.857866 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5cafb759-6926-4fd6-a77e-7e18a9becd03-kube-api-access-t5kzv" (OuterVolumeSpecName: "kube-api-access-t5kzv") pod "5cafb759-6926-4fd6-a77e-7e18a9becd03" (UID: "5cafb759-6926-4fd6-a77e-7e18a9becd03"). InnerVolumeSpecName "kube-api-access-t5kzv". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:47:14 crc kubenswrapper[5052]: I1203 12:47:14.916429 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5cafb759-6926-4fd6-a77e-7e18a9becd03-config-data" (OuterVolumeSpecName: "config-data") pod "5cafb759-6926-4fd6-a77e-7e18a9becd03" (UID: "5cafb759-6926-4fd6-a77e-7e18a9becd03"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:47:14 crc kubenswrapper[5052]: I1203 12:47:14.925272 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5cafb759-6926-4fd6-a77e-7e18a9becd03-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "5cafb759-6926-4fd6-a77e-7e18a9becd03" (UID: "5cafb759-6926-4fd6-a77e-7e18a9becd03"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:47:14 crc kubenswrapper[5052]: I1203 12:47:14.957985 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-t5kzv\" (UniqueName: \"kubernetes.io/projected/5cafb759-6926-4fd6-a77e-7e18a9becd03-kube-api-access-t5kzv\") on node \"crc\" DevicePath \"\"" Dec 03 12:47:14 crc kubenswrapper[5052]: I1203 12:47:14.958037 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/5cafb759-6926-4fd6-a77e-7e18a9becd03-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:47:14 crc kubenswrapper[5052]: I1203 12:47:14.958054 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5cafb759-6926-4fd6-a77e-7e18a9becd03-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:47:14 crc kubenswrapper[5052]: I1203 12:47:14.993978 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5cafb759-6926-4fd6-a77e-7e18a9becd03-vencrypt-tls-certs" (OuterVolumeSpecName: "vencrypt-tls-certs") pod "5cafb759-6926-4fd6-a77e-7e18a9becd03" (UID: "5cafb759-6926-4fd6-a77e-7e18a9becd03"). InnerVolumeSpecName "vencrypt-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:47:15 crc kubenswrapper[5052]: I1203 12:47:15.021541 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-fdd8f875d-xvq79" event={"ID":"77825998-48a8-496e-a523-8e0c371fe532","Type":"ContainerStarted","Data":"15655b574d1d35cf1c932536528e9b95711b3ebc8dd150537384f5080772cd5e"} Dec 03 12:47:15 crc kubenswrapper[5052]: I1203 12:47:15.026119 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-fdd8f875d-xvq79" event={"ID":"77825998-48a8-496e-a523-8e0c371fe532","Type":"ContainerStarted","Data":"c79f26b11c1bba88e99d71d696bb0decec843bf868e7a4fdf02ceb21feddb964"} Dec 03 12:47:15 crc kubenswrapper[5052]: I1203 12:47:15.026220 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/neutron-fdd8f875d-xvq79" Dec 03 12:47:15 crc kubenswrapper[5052]: I1203 12:47:15.026001 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5cafb759-6926-4fd6-a77e-7e18a9becd03-nova-novncproxy-tls-certs" (OuterVolumeSpecName: "nova-novncproxy-tls-certs") pod "5cafb759-6926-4fd6-a77e-7e18a9becd03" (UID: "5cafb759-6926-4fd6-a77e-7e18a9becd03"). InnerVolumeSpecName "nova-novncproxy-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:47:15 crc kubenswrapper[5052]: I1203 12:47:15.031536 5052 generic.go:334] "Generic (PLEG): container finished" podID="75266130-5854-4acd-acc9-bc22accc19bc" containerID="3e1d7a5d05726614ff20caaaa6d897aa3cbb28e2e859ab554810ad90e4b4a72c" exitCode=143 Dec 03 12:47:15 crc kubenswrapper[5052]: I1203 12:47:15.031622 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"75266130-5854-4acd-acc9-bc22accc19bc","Type":"ContainerDied","Data":"3e1d7a5d05726614ff20caaaa6d897aa3cbb28e2e859ab554810ad90e4b4a72c"} Dec 03 12:47:15 crc kubenswrapper[5052]: I1203 12:47:15.033657 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovn-northd-0" event={"ID":"342cba80-22e1-488a-bbfb-75b67ca536ac","Type":"ContainerStarted","Data":"6b272b57a393df881b0806ef34c9c3a75d64d13721c0eab27f38fa3374aed7ed"} Dec 03 12:47:15 crc kubenswrapper[5052]: I1203 12:47:15.033698 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovn-northd-0" event={"ID":"342cba80-22e1-488a-bbfb-75b67ca536ac","Type":"ContainerStarted","Data":"ce49b1f529433537dacac11dc47b93030dd9670b6a41b713c883a09a1fe19146"} Dec 03 12:47:15 crc kubenswrapper[5052]: I1203 12:47:15.060465 5052 generic.go:334] "Generic (PLEG): container finished" podID="5cafb759-6926-4fd6-a77e-7e18a9becd03" containerID="a1eac786eb973740a08ecdd82bac9b30836804efc46b172c9ef6ef222f8a88c5" exitCode=0 Dec 03 12:47:15 crc kubenswrapper[5052]: I1203 12:47:15.060605 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" event={"ID":"5cafb759-6926-4fd6-a77e-7e18a9becd03","Type":"ContainerDied","Data":"a1eac786eb973740a08ecdd82bac9b30836804efc46b172c9ef6ef222f8a88c5"} Dec 03 12:47:15 crc kubenswrapper[5052]: I1203 12:47:15.060640 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" event={"ID":"5cafb759-6926-4fd6-a77e-7e18a9becd03","Type":"ContainerDied","Data":"e14839f4541e00060453ac3f79a70d8dbe437da0b37ce8e08926308c30ecc583"} Dec 03 12:47:15 crc kubenswrapper[5052]: I1203 12:47:15.060660 5052 scope.go:117] "RemoveContainer" containerID="a1eac786eb973740a08ecdd82bac9b30836804efc46b172c9ef6ef222f8a88c5" Dec 03 12:47:15 crc kubenswrapper[5052]: I1203 12:47:15.060821 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:47:15 crc kubenswrapper[5052]: I1203 12:47:15.065381 5052 reconciler_common.go:293] "Volume detached for volume \"vencrypt-tls-certs\" (UniqueName: \"kubernetes.io/secret/5cafb759-6926-4fd6-a77e-7e18a9becd03-vencrypt-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:47:15 crc kubenswrapper[5052]: I1203 12:47:15.066652 5052 reconciler_common.go:293] "Volume detached for volume \"nova-novncproxy-tls-certs\" (UniqueName: \"kubernetes.io/secret/5cafb759-6926-4fd6-a77e-7e18a9becd03-nova-novncproxy-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:47:15 crc kubenswrapper[5052]: I1203 12:47:15.077409 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/barbican-api-6d86f7b884-lf4pd" podUID="5b63f869-c95a-4279-b542-3ebbec1a1fd5" containerName="barbican-api" probeResult="failure" output="Get \"https://10.217.1.197:9311/healthcheck\": read tcp 10.217.0.2:52430->10.217.1.197:9311: read: connection reset by peer" Dec 03 12:47:15 crc kubenswrapper[5052]: I1203 12:47:15.077409 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/barbican-api-6d86f7b884-lf4pd" podUID="5b63f869-c95a-4279-b542-3ebbec1a1fd5" containerName="barbican-api-log" probeResult="failure" output="Get \"https://10.217.1.197:9311/healthcheck\": read tcp 10.217.0.2:52418->10.217.1.197:9311: read: connection reset by peer" Dec 03 12:47:15 crc kubenswrapper[5052]: I1203 12:47:15.084358 5052 generic.go:334] "Generic (PLEG): container finished" podID="718fd921-0756-4a3e-b383-ed0d0ddf5b72" containerID="454384af2bc2744e1c70e4a67dc88563309e7147062c1b5e0c424094b4fea86d" exitCode=0 Dec 03 12:47:15 crc kubenswrapper[5052]: I1203 12:47:15.084476 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-8b544ff76-6hjc7" event={"ID":"718fd921-0756-4a3e-b383-ed0d0ddf5b72","Type":"ContainerDied","Data":"454384af2bc2744e1c70e4a67dc88563309e7147062c1b5e0c424094b4fea86d"} Dec 03 12:47:15 crc kubenswrapper[5052]: I1203 12:47:15.089300 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/neutron-fdd8f875d-xvq79" podStartSLOduration=3.089277159 podStartE2EDuration="3.089277159s" podCreationTimestamp="2025-12-03 12:47:12 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:47:15.047862095 +0000 UTC m=+5225.636103859" watchObservedRunningTime="2025-12-03 12:47:15.089277159 +0000 UTC m=+5225.677518933" Dec 03 12:47:15 crc kubenswrapper[5052]: I1203 12:47:15.100541 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"557175a9-81e5-4605-bd67-f1dfa575fb1b","Type":"ContainerStarted","Data":"60ff2fa7c14572046a3621f0f09cee6b8670305019e8df8b2b52c2d7c1c39683"} Dec 03 12:47:15 crc kubenswrapper[5052]: I1203 12:47:15.103653 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:47:15 crc kubenswrapper[5052]: I1203 12:47:15.103693 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"d5719b07-205b-4e6a-9fe2-b6531b779b0a","Type":"ContainerDied","Data":"5fd11a286ce87d05f1bf202e0033bf3bdd9f521e33004bcc3c2218a19adfc2af"} Dec 03 12:47:15 crc kubenswrapper[5052]: I1203 12:47:15.338116 5052 scope.go:117] "RemoveContainer" containerID="a1eac786eb973740a08ecdd82bac9b30836804efc46b172c9ef6ef222f8a88c5" Dec 03 12:47:15 crc kubenswrapper[5052]: E1203 12:47:15.339488 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a1eac786eb973740a08ecdd82bac9b30836804efc46b172c9ef6ef222f8a88c5\": container with ID starting with a1eac786eb973740a08ecdd82bac9b30836804efc46b172c9ef6ef222f8a88c5 not found: ID does not exist" containerID="a1eac786eb973740a08ecdd82bac9b30836804efc46b172c9ef6ef222f8a88c5" Dec 03 12:47:15 crc kubenswrapper[5052]: I1203 12:47:15.339651 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a1eac786eb973740a08ecdd82bac9b30836804efc46b172c9ef6ef222f8a88c5"} err="failed to get container status \"a1eac786eb973740a08ecdd82bac9b30836804efc46b172c9ef6ef222f8a88c5\": rpc error: code = NotFound desc = could not find container \"a1eac786eb973740a08ecdd82bac9b30836804efc46b172c9ef6ef222f8a88c5\": container with ID starting with a1eac786eb973740a08ecdd82bac9b30836804efc46b172c9ef6ef222f8a88c5 not found: ID does not exist" Dec 03 12:47:15 crc kubenswrapper[5052]: I1203 12:47:15.339717 5052 scope.go:117] "RemoveContainer" containerID="033df125bf601aa712f091f9637bf143ce864965096aeccefa080e46608d8437" Dec 03 12:47:15 crc kubenswrapper[5052]: I1203 12:47:15.376611 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:47:15 crc kubenswrapper[5052]: I1203 12:47:15.405359 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:47:15 crc kubenswrapper[5052]: I1203 12:47:15.442355 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 12:47:15 crc kubenswrapper[5052]: I1203 12:47:15.459599 5052 scope.go:117] "RemoveContainer" containerID="32f0eb2434b003178dbbca6a7f974c5933898aa66a832336cc4f079190148796" Dec 03 12:47:15 crc kubenswrapper[5052]: I1203 12:47:15.475714 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:47:15 crc kubenswrapper[5052]: E1203 12:47:15.476241 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="d5719b07-205b-4e6a-9fe2-b6531b779b0a" containerName="nova-api-log" Dec 03 12:47:15 crc kubenswrapper[5052]: I1203 12:47:15.476267 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="d5719b07-205b-4e6a-9fe2-b6531b779b0a" containerName="nova-api-log" Dec 03 12:47:15 crc kubenswrapper[5052]: E1203 12:47:15.476287 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="5cafb759-6926-4fd6-a77e-7e18a9becd03" containerName="nova-cell1-novncproxy-novncproxy" Dec 03 12:47:15 crc kubenswrapper[5052]: I1203 12:47:15.476296 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="5cafb759-6926-4fd6-a77e-7e18a9becd03" containerName="nova-cell1-novncproxy-novncproxy" Dec 03 12:47:15 crc kubenswrapper[5052]: E1203 12:47:15.476341 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="d5719b07-205b-4e6a-9fe2-b6531b779b0a" containerName="nova-api-api" Dec 03 12:47:15 crc kubenswrapper[5052]: I1203 12:47:15.476347 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="d5719b07-205b-4e6a-9fe2-b6531b779b0a" containerName="nova-api-api" Dec 03 12:47:15 crc kubenswrapper[5052]: I1203 12:47:15.476728 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="d5719b07-205b-4e6a-9fe2-b6531b779b0a" containerName="nova-api-log" Dec 03 12:47:15 crc kubenswrapper[5052]: I1203 12:47:15.476752 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="d5719b07-205b-4e6a-9fe2-b6531b779b0a" containerName="nova-api-api" Dec 03 12:47:15 crc kubenswrapper[5052]: I1203 12:47:15.476766 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="5cafb759-6926-4fd6-a77e-7e18a9becd03" containerName="nova-cell1-novncproxy-novncproxy" Dec 03 12:47:15 crc kubenswrapper[5052]: I1203 12:47:15.477865 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:47:15 crc kubenswrapper[5052]: I1203 12:47:15.484803 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-api-config-data" Dec 03 12:47:15 crc kubenswrapper[5052]: I1203 12:47:15.504332 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 12:47:15 crc kubenswrapper[5052]: I1203 12:47:15.498825 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-nova-public-svc" Dec 03 12:47:15 crc kubenswrapper[5052]: I1203 12:47:15.499487 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-nova-internal-svc" Dec 03 12:47:15 crc kubenswrapper[5052]: I1203 12:47:15.536321 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:47:15 crc kubenswrapper[5052]: I1203 12:47:15.550461 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 12:47:15 crc kubenswrapper[5052]: I1203 12:47:15.553374 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:47:15 crc kubenswrapper[5052]: I1203 12:47:15.556563 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 12:47:15 crc kubenswrapper[5052]: I1203 12:47:15.565237 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-nova-novncproxy-cell1-vencrypt" Dec 03 12:47:15 crc kubenswrapper[5052]: I1203 12:47:15.565490 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell1-novncproxy-config-data" Dec 03 12:47:15 crc kubenswrapper[5052]: I1203 12:47:15.565241 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-nova-novncproxy-cell1-public-svc" Dec 03 12:47:15 crc kubenswrapper[5052]: I1203 12:47:15.600844 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/1e70c60d-8835-4b4f-80dc-7190e8f6facd-public-tls-certs\") pod \"nova-api-0\" (UID: \"1e70c60d-8835-4b4f-80dc-7190e8f6facd\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:47:15 crc kubenswrapper[5052]: I1203 12:47:15.600899 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/1e70c60d-8835-4b4f-80dc-7190e8f6facd-config-data\") pod \"nova-api-0\" (UID: \"1e70c60d-8835-4b4f-80dc-7190e8f6facd\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:47:15 crc kubenswrapper[5052]: I1203 12:47:15.600950 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/edae7848-51ff-40d6-83f2-ffff085f3f3e-config-data\") pod \"nova-cell1-novncproxy-0\" (UID: \"edae7848-51ff-40d6-83f2-ffff085f3f3e\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:47:15 crc kubenswrapper[5052]: I1203 12:47:15.601016 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nova-novncproxy-tls-certs\" (UniqueName: \"kubernetes.io/secret/edae7848-51ff-40d6-83f2-ffff085f3f3e-nova-novncproxy-tls-certs\") pod \"nova-cell1-novncproxy-0\" (UID: \"edae7848-51ff-40d6-83f2-ffff085f3f3e\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:47:15 crc kubenswrapper[5052]: I1203 12:47:15.601082 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"vencrypt-tls-certs\" (UniqueName: \"kubernetes.io/secret/edae7848-51ff-40d6-83f2-ffff085f3f3e-vencrypt-tls-certs\") pod \"nova-cell1-novncproxy-0\" (UID: \"edae7848-51ff-40d6-83f2-ffff085f3f3e\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:47:15 crc kubenswrapper[5052]: I1203 12:47:15.601124 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kk6gr\" (UniqueName: \"kubernetes.io/projected/edae7848-51ff-40d6-83f2-ffff085f3f3e-kube-api-access-kk6gr\") pod \"nova-cell1-novncproxy-0\" (UID: \"edae7848-51ff-40d6-83f2-ffff085f3f3e\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:47:15 crc kubenswrapper[5052]: I1203 12:47:15.601161 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/1e70c60d-8835-4b4f-80dc-7190e8f6facd-internal-tls-certs\") pod \"nova-api-0\" (UID: \"1e70c60d-8835-4b4f-80dc-7190e8f6facd\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:47:15 crc kubenswrapper[5052]: I1203 12:47:15.601194 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bbtns\" (UniqueName: \"kubernetes.io/projected/1e70c60d-8835-4b4f-80dc-7190e8f6facd-kube-api-access-bbtns\") pod \"nova-api-0\" (UID: \"1e70c60d-8835-4b4f-80dc-7190e8f6facd\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:47:15 crc kubenswrapper[5052]: I1203 12:47:15.601225 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/edae7848-51ff-40d6-83f2-ffff085f3f3e-combined-ca-bundle\") pod \"nova-cell1-novncproxy-0\" (UID: \"edae7848-51ff-40d6-83f2-ffff085f3f3e\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:47:15 crc kubenswrapper[5052]: I1203 12:47:15.601338 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1e70c60d-8835-4b4f-80dc-7190e8f6facd-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"1e70c60d-8835-4b4f-80dc-7190e8f6facd\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:47:15 crc kubenswrapper[5052]: I1203 12:47:15.601363 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/1e70c60d-8835-4b4f-80dc-7190e8f6facd-logs\") pod \"nova-api-0\" (UID: \"1e70c60d-8835-4b4f-80dc-7190e8f6facd\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:47:15 crc kubenswrapper[5052]: I1203 12:47:15.696612 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5cafb759-6926-4fd6-a77e-7e18a9becd03" path="/var/lib/kubelet/pods/5cafb759-6926-4fd6-a77e-7e18a9becd03/volumes" Dec 03 12:47:15 crc kubenswrapper[5052]: I1203 12:47:15.697777 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d5719b07-205b-4e6a-9fe2-b6531b779b0a" path="/var/lib/kubelet/pods/d5719b07-205b-4e6a-9fe2-b6531b779b0a/volumes" Dec 03 12:47:15 crc kubenswrapper[5052]: I1203 12:47:15.703500 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/1e70c60d-8835-4b4f-80dc-7190e8f6facd-config-data\") pod \"nova-api-0\" (UID: \"1e70c60d-8835-4b4f-80dc-7190e8f6facd\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:47:15 crc kubenswrapper[5052]: I1203 12:47:15.703592 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/edae7848-51ff-40d6-83f2-ffff085f3f3e-config-data\") pod \"nova-cell1-novncproxy-0\" (UID: \"edae7848-51ff-40d6-83f2-ffff085f3f3e\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:47:15 crc kubenswrapper[5052]: I1203 12:47:15.703663 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"nova-novncproxy-tls-certs\" (UniqueName: \"kubernetes.io/secret/edae7848-51ff-40d6-83f2-ffff085f3f3e-nova-novncproxy-tls-certs\") pod \"nova-cell1-novncproxy-0\" (UID: \"edae7848-51ff-40d6-83f2-ffff085f3f3e\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:47:15 crc kubenswrapper[5052]: I1203 12:47:15.703736 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"vencrypt-tls-certs\" (UniqueName: \"kubernetes.io/secret/edae7848-51ff-40d6-83f2-ffff085f3f3e-vencrypt-tls-certs\") pod \"nova-cell1-novncproxy-0\" (UID: \"edae7848-51ff-40d6-83f2-ffff085f3f3e\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:47:15 crc kubenswrapper[5052]: I1203 12:47:15.703819 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-kk6gr\" (UniqueName: \"kubernetes.io/projected/edae7848-51ff-40d6-83f2-ffff085f3f3e-kube-api-access-kk6gr\") pod \"nova-cell1-novncproxy-0\" (UID: \"edae7848-51ff-40d6-83f2-ffff085f3f3e\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:47:15 crc kubenswrapper[5052]: I1203 12:47:15.703863 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/1e70c60d-8835-4b4f-80dc-7190e8f6facd-internal-tls-certs\") pod \"nova-api-0\" (UID: \"1e70c60d-8835-4b4f-80dc-7190e8f6facd\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:47:15 crc kubenswrapper[5052]: I1203 12:47:15.703898 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-bbtns\" (UniqueName: \"kubernetes.io/projected/1e70c60d-8835-4b4f-80dc-7190e8f6facd-kube-api-access-bbtns\") pod \"nova-api-0\" (UID: \"1e70c60d-8835-4b4f-80dc-7190e8f6facd\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:47:15 crc kubenswrapper[5052]: I1203 12:47:15.703930 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/edae7848-51ff-40d6-83f2-ffff085f3f3e-combined-ca-bundle\") pod \"nova-cell1-novncproxy-0\" (UID: \"edae7848-51ff-40d6-83f2-ffff085f3f3e\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:47:15 crc kubenswrapper[5052]: I1203 12:47:15.704053 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1e70c60d-8835-4b4f-80dc-7190e8f6facd-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"1e70c60d-8835-4b4f-80dc-7190e8f6facd\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:47:15 crc kubenswrapper[5052]: I1203 12:47:15.704083 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/1e70c60d-8835-4b4f-80dc-7190e8f6facd-logs\") pod \"nova-api-0\" (UID: \"1e70c60d-8835-4b4f-80dc-7190e8f6facd\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:47:15 crc kubenswrapper[5052]: I1203 12:47:15.704111 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/1e70c60d-8835-4b4f-80dc-7190e8f6facd-public-tls-certs\") pod \"nova-api-0\" (UID: \"1e70c60d-8835-4b4f-80dc-7190e8f6facd\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:47:15 crc kubenswrapper[5052]: I1203 12:47:15.705629 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/1e70c60d-8835-4b4f-80dc-7190e8f6facd-logs\") pod \"nova-api-0\" (UID: \"1e70c60d-8835-4b4f-80dc-7190e8f6facd\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:47:15 crc kubenswrapper[5052]: I1203 12:47:15.715145 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"nova-novncproxy-tls-certs\" (UniqueName: \"kubernetes.io/secret/edae7848-51ff-40d6-83f2-ffff085f3f3e-nova-novncproxy-tls-certs\") pod \"nova-cell1-novncproxy-0\" (UID: \"edae7848-51ff-40d6-83f2-ffff085f3f3e\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:47:15 crc kubenswrapper[5052]: I1203 12:47:15.721339 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/edae7848-51ff-40d6-83f2-ffff085f3f3e-combined-ca-bundle\") pod \"nova-cell1-novncproxy-0\" (UID: \"edae7848-51ff-40d6-83f2-ffff085f3f3e\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:47:15 crc kubenswrapper[5052]: I1203 12:47:15.722897 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/1e70c60d-8835-4b4f-80dc-7190e8f6facd-public-tls-certs\") pod \"nova-api-0\" (UID: \"1e70c60d-8835-4b4f-80dc-7190e8f6facd\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:47:15 crc kubenswrapper[5052]: I1203 12:47:15.723194 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/edae7848-51ff-40d6-83f2-ffff085f3f3e-config-data\") pod \"nova-cell1-novncproxy-0\" (UID: \"edae7848-51ff-40d6-83f2-ffff085f3f3e\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:47:15 crc kubenswrapper[5052]: I1203 12:47:15.723769 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/1e70c60d-8835-4b4f-80dc-7190e8f6facd-internal-tls-certs\") pod \"nova-api-0\" (UID: \"1e70c60d-8835-4b4f-80dc-7190e8f6facd\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:47:15 crc kubenswrapper[5052]: I1203 12:47:15.724483 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/1e70c60d-8835-4b4f-80dc-7190e8f6facd-config-data\") pod \"nova-api-0\" (UID: \"1e70c60d-8835-4b4f-80dc-7190e8f6facd\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:47:15 crc kubenswrapper[5052]: I1203 12:47:15.724632 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1e70c60d-8835-4b4f-80dc-7190e8f6facd-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"1e70c60d-8835-4b4f-80dc-7190e8f6facd\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:47:15 crc kubenswrapper[5052]: I1203 12:47:15.730755 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-kk6gr\" (UniqueName: \"kubernetes.io/projected/edae7848-51ff-40d6-83f2-ffff085f3f3e-kube-api-access-kk6gr\") pod \"nova-cell1-novncproxy-0\" (UID: \"edae7848-51ff-40d6-83f2-ffff085f3f3e\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:47:15 crc kubenswrapper[5052]: I1203 12:47:15.737581 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-bbtns\" (UniqueName: \"kubernetes.io/projected/1e70c60d-8835-4b4f-80dc-7190e8f6facd-kube-api-access-bbtns\") pod \"nova-api-0\" (UID: \"1e70c60d-8835-4b4f-80dc-7190e8f6facd\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:47:15 crc kubenswrapper[5052]: I1203 12:47:15.739045 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"vencrypt-tls-certs\" (UniqueName: \"kubernetes.io/secret/edae7848-51ff-40d6-83f2-ffff085f3f3e-vencrypt-tls-certs\") pod \"nova-cell1-novncproxy-0\" (UID: \"edae7848-51ff-40d6-83f2-ffff085f3f3e\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:47:15 crc kubenswrapper[5052]: I1203 12:47:15.831317 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-api-6d86f7b884-lf4pd" Dec 03 12:47:15 crc kubenswrapper[5052]: I1203 12:47:15.839777 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:47:15 crc kubenswrapper[5052]: I1203 12:47:15.845687 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:47:15 crc kubenswrapper[5052]: I1203 12:47:15.913534 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:47:16 crc kubenswrapper[5052]: I1203 12:47:16.016390 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/5b63f869-c95a-4279-b542-3ebbec1a1fd5-public-tls-certs\") pod \"5b63f869-c95a-4279-b542-3ebbec1a1fd5\" (UID: \"5b63f869-c95a-4279-b542-3ebbec1a1fd5\") " Dec 03 12:47:16 crc kubenswrapper[5052]: I1203 12:47:16.016603 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5b63f869-c95a-4279-b542-3ebbec1a1fd5-combined-ca-bundle\") pod \"5b63f869-c95a-4279-b542-3ebbec1a1fd5\" (UID: \"5b63f869-c95a-4279-b542-3ebbec1a1fd5\") " Dec 03 12:47:16 crc kubenswrapper[5052]: I1203 12:47:16.016661 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/5b63f869-c95a-4279-b542-3ebbec1a1fd5-config-data-custom\") pod \"5b63f869-c95a-4279-b542-3ebbec1a1fd5\" (UID: \"5b63f869-c95a-4279-b542-3ebbec1a1fd5\") " Dec 03 12:47:16 crc kubenswrapper[5052]: I1203 12:47:16.016702 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/5b63f869-c95a-4279-b542-3ebbec1a1fd5-config-data\") pod \"5b63f869-c95a-4279-b542-3ebbec1a1fd5\" (UID: \"5b63f869-c95a-4279-b542-3ebbec1a1fd5\") " Dec 03 12:47:16 crc kubenswrapper[5052]: I1203 12:47:16.016809 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/5b63f869-c95a-4279-b542-3ebbec1a1fd5-logs\") pod \"5b63f869-c95a-4279-b542-3ebbec1a1fd5\" (UID: \"5b63f869-c95a-4279-b542-3ebbec1a1fd5\") " Dec 03 12:47:16 crc kubenswrapper[5052]: I1203 12:47:16.016885 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bxhjw\" (UniqueName: \"kubernetes.io/projected/5b63f869-c95a-4279-b542-3ebbec1a1fd5-kube-api-access-bxhjw\") pod \"5b63f869-c95a-4279-b542-3ebbec1a1fd5\" (UID: \"5b63f869-c95a-4279-b542-3ebbec1a1fd5\") " Dec 03 12:47:16 crc kubenswrapper[5052]: I1203 12:47:16.016922 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/5b63f869-c95a-4279-b542-3ebbec1a1fd5-internal-tls-certs\") pod \"5b63f869-c95a-4279-b542-3ebbec1a1fd5\" (UID: \"5b63f869-c95a-4279-b542-3ebbec1a1fd5\") " Dec 03 12:47:16 crc kubenswrapper[5052]: I1203 12:47:16.028609 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5b63f869-c95a-4279-b542-3ebbec1a1fd5-logs" (OuterVolumeSpecName: "logs") pod "5b63f869-c95a-4279-b542-3ebbec1a1fd5" (UID: "5b63f869-c95a-4279-b542-3ebbec1a1fd5"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:47:16 crc kubenswrapper[5052]: I1203 12:47:16.039847 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5b63f869-c95a-4279-b542-3ebbec1a1fd5-kube-api-access-bxhjw" (OuterVolumeSpecName: "kube-api-access-bxhjw") pod "5b63f869-c95a-4279-b542-3ebbec1a1fd5" (UID: "5b63f869-c95a-4279-b542-3ebbec1a1fd5"). InnerVolumeSpecName "kube-api-access-bxhjw". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:47:16 crc kubenswrapper[5052]: I1203 12:47:16.044805 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5b63f869-c95a-4279-b542-3ebbec1a1fd5-config-data-custom" (OuterVolumeSpecName: "config-data-custom") pod "5b63f869-c95a-4279-b542-3ebbec1a1fd5" (UID: "5b63f869-c95a-4279-b542-3ebbec1a1fd5"). InnerVolumeSpecName "config-data-custom". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:47:16 crc kubenswrapper[5052]: I1203 12:47:16.095739 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5b63f869-c95a-4279-b542-3ebbec1a1fd5-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "5b63f869-c95a-4279-b542-3ebbec1a1fd5" (UID: "5b63f869-c95a-4279-b542-3ebbec1a1fd5"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:47:16 crc kubenswrapper[5052]: I1203 12:47:16.104899 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5b63f869-c95a-4279-b542-3ebbec1a1fd5-config-data" (OuterVolumeSpecName: "config-data") pod "5b63f869-c95a-4279-b542-3ebbec1a1fd5" (UID: "5b63f869-c95a-4279-b542-3ebbec1a1fd5"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:47:16 crc kubenswrapper[5052]: I1203 12:47:16.114520 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5b63f869-c95a-4279-b542-3ebbec1a1fd5-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "5b63f869-c95a-4279-b542-3ebbec1a1fd5" (UID: "5b63f869-c95a-4279-b542-3ebbec1a1fd5"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:47:16 crc kubenswrapper[5052]: I1203 12:47:16.119101 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/5b63f869-c95a-4279-b542-3ebbec1a1fd5-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:47:16 crc kubenswrapper[5052]: I1203 12:47:16.119133 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-bxhjw\" (UniqueName: \"kubernetes.io/projected/5b63f869-c95a-4279-b542-3ebbec1a1fd5-kube-api-access-bxhjw\") on node \"crc\" DevicePath \"\"" Dec 03 12:47:16 crc kubenswrapper[5052]: I1203 12:47:16.119146 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/5b63f869-c95a-4279-b542-3ebbec1a1fd5-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:47:16 crc kubenswrapper[5052]: I1203 12:47:16.119155 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5b63f869-c95a-4279-b542-3ebbec1a1fd5-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:47:16 crc kubenswrapper[5052]: I1203 12:47:16.119166 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/5b63f869-c95a-4279-b542-3ebbec1a1fd5-config-data-custom\") on node \"crc\" DevicePath \"\"" Dec 03 12:47:16 crc kubenswrapper[5052]: I1203 12:47:16.119177 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/5b63f869-c95a-4279-b542-3ebbec1a1fd5-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:47:16 crc kubenswrapper[5052]: I1203 12:47:16.124572 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovn-northd-0" event={"ID":"342cba80-22e1-488a-bbfb-75b67ca536ac","Type":"ContainerStarted","Data":"8df9cc82a4e4e7032e7155d5c8c0fda7507b5b3d7b99882037f3bf0a8995f1d7"} Dec 03 12:47:16 crc kubenswrapper[5052]: I1203 12:47:16.126321 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:47:16 crc kubenswrapper[5052]: I1203 12:47:16.142228 5052 generic.go:334] "Generic (PLEG): container finished" podID="5b63f869-c95a-4279-b542-3ebbec1a1fd5" containerID="878de2f156357bd531b5738c30192dd837946cd61f68a045b6a3411f524181e4" exitCode=0 Dec 03 12:47:16 crc kubenswrapper[5052]: I1203 12:47:16.142359 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-6d86f7b884-lf4pd" event={"ID":"5b63f869-c95a-4279-b542-3ebbec1a1fd5","Type":"ContainerDied","Data":"878de2f156357bd531b5738c30192dd837946cd61f68a045b6a3411f524181e4"} Dec 03 12:47:16 crc kubenswrapper[5052]: I1203 12:47:16.142413 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-6d86f7b884-lf4pd" event={"ID":"5b63f869-c95a-4279-b542-3ebbec1a1fd5","Type":"ContainerDied","Data":"6bc37640eb704614061b31aa9d8d48b7b491d6bcceb5572feab011a7b4bc2dc4"} Dec 03 12:47:16 crc kubenswrapper[5052]: I1203 12:47:16.142434 5052 scope.go:117] "RemoveContainer" containerID="878de2f156357bd531b5738c30192dd837946cd61f68a045b6a3411f524181e4" Dec 03 12:47:16 crc kubenswrapper[5052]: I1203 12:47:16.142554 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-api-6d86f7b884-lf4pd" Dec 03 12:47:16 crc kubenswrapper[5052]: I1203 12:47:16.155079 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5b63f869-c95a-4279-b542-3ebbec1a1fd5-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "5b63f869-c95a-4279-b542-3ebbec1a1fd5" (UID: "5b63f869-c95a-4279-b542-3ebbec1a1fd5"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:47:16 crc kubenswrapper[5052]: I1203 12:47:16.155158 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/ovn-northd-0" podStartSLOduration=4.155140411 podStartE2EDuration="4.155140411s" podCreationTimestamp="2025-12-03 12:47:12 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:47:16.147731929 +0000 UTC m=+5226.735973703" watchObservedRunningTime="2025-12-03 12:47:16.155140411 +0000 UTC m=+5226.743382175" Dec 03 12:47:16 crc kubenswrapper[5052]: I1203 12:47:16.220555 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/5b63f869-c95a-4279-b542-3ebbec1a1fd5-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:47:16 crc kubenswrapper[5052]: I1203 12:47:16.248442 5052 scope.go:117] "RemoveContainer" containerID="51b7f4e0f8d805d3cab7483cb57358f70f00e5c62c17634e4a78a7fc68d35bce" Dec 03 12:47:16 crc kubenswrapper[5052]: I1203 12:47:16.288308 5052 scope.go:117] "RemoveContainer" containerID="878de2f156357bd531b5738c30192dd837946cd61f68a045b6a3411f524181e4" Dec 03 12:47:16 crc kubenswrapper[5052]: E1203 12:47:16.288676 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"878de2f156357bd531b5738c30192dd837946cd61f68a045b6a3411f524181e4\": container with ID starting with 878de2f156357bd531b5738c30192dd837946cd61f68a045b6a3411f524181e4 not found: ID does not exist" containerID="878de2f156357bd531b5738c30192dd837946cd61f68a045b6a3411f524181e4" Dec 03 12:47:16 crc kubenswrapper[5052]: I1203 12:47:16.288716 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"878de2f156357bd531b5738c30192dd837946cd61f68a045b6a3411f524181e4"} err="failed to get container status \"878de2f156357bd531b5738c30192dd837946cd61f68a045b6a3411f524181e4\": rpc error: code = NotFound desc = could not find container \"878de2f156357bd531b5738c30192dd837946cd61f68a045b6a3411f524181e4\": container with ID starting with 878de2f156357bd531b5738c30192dd837946cd61f68a045b6a3411f524181e4 not found: ID does not exist" Dec 03 12:47:16 crc kubenswrapper[5052]: I1203 12:47:16.288746 5052 scope.go:117] "RemoveContainer" containerID="51b7f4e0f8d805d3cab7483cb57358f70f00e5c62c17634e4a78a7fc68d35bce" Dec 03 12:47:16 crc kubenswrapper[5052]: E1203 12:47:16.288976 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"51b7f4e0f8d805d3cab7483cb57358f70f00e5c62c17634e4a78a7fc68d35bce\": container with ID starting with 51b7f4e0f8d805d3cab7483cb57358f70f00e5c62c17634e4a78a7fc68d35bce not found: ID does not exist" containerID="51b7f4e0f8d805d3cab7483cb57358f70f00e5c62c17634e4a78a7fc68d35bce" Dec 03 12:47:16 crc kubenswrapper[5052]: I1203 12:47:16.289013 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"51b7f4e0f8d805d3cab7483cb57358f70f00e5c62c17634e4a78a7fc68d35bce"} err="failed to get container status \"51b7f4e0f8d805d3cab7483cb57358f70f00e5c62c17634e4a78a7fc68d35bce\": rpc error: code = NotFound desc = could not find container \"51b7f4e0f8d805d3cab7483cb57358f70f00e5c62c17634e4a78a7fc68d35bce\": container with ID starting with 51b7f4e0f8d805d3cab7483cb57358f70f00e5c62c17634e4a78a7fc68d35bce not found: ID does not exist" Dec 03 12:47:16 crc kubenswrapper[5052]: I1203 12:47:16.476350 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:47:16 crc kubenswrapper[5052]: I1203 12:47:16.510575 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-api-6d86f7b884-lf4pd"] Dec 03 12:47:16 crc kubenswrapper[5052]: I1203 12:47:16.516432 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/barbican-api-6d86f7b884-lf4pd"] Dec 03 12:47:16 crc kubenswrapper[5052]: I1203 12:47:16.590902 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 12:47:17 crc kubenswrapper[5052]: I1203 12:47:17.154669 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" event={"ID":"edae7848-51ff-40d6-83f2-ffff085f3f3e","Type":"ContainerStarted","Data":"d0169811e45510e60a9be05d74b820c1652bf1608236beac08ab52eb9b8b22b5"} Dec 03 12:47:17 crc kubenswrapper[5052]: I1203 12:47:17.157597 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"557175a9-81e5-4605-bd67-f1dfa575fb1b","Type":"ContainerStarted","Data":"4f6a6de545ed43fd89fd3c33b250e3e6c4f74774fb05bed0727171451864fd53"} Dec 03 12:47:17 crc kubenswrapper[5052]: I1203 12:47:17.158727 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:47:17 crc kubenswrapper[5052]: I1203 12:47:17.166147 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"1e70c60d-8835-4b4f-80dc-7190e8f6facd","Type":"ContainerStarted","Data":"ab41ba3190a253c4fde3b178de0a2fcbce57b7c5a7424a3d906631e122b2a3e0"} Dec 03 12:47:17 crc kubenswrapper[5052]: I1203 12:47:17.188625 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/cinder-api-0" podStartSLOduration=5.188603787 podStartE2EDuration="5.188603787s" podCreationTimestamp="2025-12-03 12:47:12 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:47:17.181541765 +0000 UTC m=+5227.769783529" watchObservedRunningTime="2025-12-03 12:47:17.188603787 +0000 UTC m=+5227.776845551" Dec 03 12:47:17 crc kubenswrapper[5052]: I1203 12:47:17.686968 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5b63f869-c95a-4279-b542-3ebbec1a1fd5" path="/var/lib/kubelet/pods/5b63f869-c95a-4279-b542-3ebbec1a1fd5/volumes" Dec 03 12:47:17 crc kubenswrapper[5052]: I1203 12:47:17.921698 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:47:17 crc kubenswrapper[5052]: I1203 12:47:17.922125 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:47:17 crc kubenswrapper[5052]: I1203 12:47:17.956280 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:47:17 crc kubenswrapper[5052]: I1203 12:47:17.964584 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:47:17 crc kubenswrapper[5052]: I1203 12:47:17.991052 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:47:17 crc kubenswrapper[5052]: I1203 12:47:17.991103 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:47:18 crc kubenswrapper[5052]: I1203 12:47:18.029475 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:47:18 crc kubenswrapper[5052]: I1203 12:47:18.050465 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:47:18 crc kubenswrapper[5052]: I1203 12:47:18.179375 5052 generic.go:334] "Generic (PLEG): container finished" podID="6d653682-52e2-4940-af66-633475bfe09a" containerID="1fdbade92a13691b593283a5b263382dfec190075f1201c77fff78a7be754487" exitCode=1 Dec 03 12:47:18 crc kubenswrapper[5052]: I1203 12:47:18.179455 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-conductor-0" event={"ID":"6d653682-52e2-4940-af66-633475bfe09a","Type":"ContainerDied","Data":"1fdbade92a13691b593283a5b263382dfec190075f1201c77fff78a7be754487"} Dec 03 12:47:18 crc kubenswrapper[5052]: I1203 12:47:18.179489 5052 scope.go:117] "RemoveContainer" containerID="4c1d25fd5e7f80502770dbe44896eb5a3327b1bebde080de0e66dead94625111" Dec 03 12:47:18 crc kubenswrapper[5052]: I1203 12:47:18.180326 5052 scope.go:117] "RemoveContainer" containerID="1fdbade92a13691b593283a5b263382dfec190075f1201c77fff78a7be754487" Dec 03 12:47:18 crc kubenswrapper[5052]: E1203 12:47:18.180635 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"nova-cell0-conductor-conductor\" with CrashLoopBackOff: \"back-off 40s restarting failed container=nova-cell0-conductor-conductor pod=nova-cell0-conductor-0_openstack-kuttl-tests(6d653682-52e2-4940-af66-633475bfe09a)\"" pod="openstack-kuttl-tests/nova-cell0-conductor-0" podUID="6d653682-52e2-4940-af66-633475bfe09a" Dec 03 12:47:18 crc kubenswrapper[5052]: I1203 12:47:18.182291 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"1e70c60d-8835-4b4f-80dc-7190e8f6facd","Type":"ContainerStarted","Data":"338686a6402baec72b3a148a936c96c0893afe29ea77e3b16918de13c7a7ba42"} Dec 03 12:47:18 crc kubenswrapper[5052]: I1203 12:47:18.182384 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"1e70c60d-8835-4b4f-80dc-7190e8f6facd","Type":"ContainerStarted","Data":"21ef156febcb78fdcdacf2b041efeec032d5dae34190cb2b7c5dbc63df6f93ab"} Dec 03 12:47:18 crc kubenswrapper[5052]: I1203 12:47:18.182398 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-api-0" podUID="1e70c60d-8835-4b4f-80dc-7190e8f6facd" containerName="nova-api-log" containerID="cri-o://21ef156febcb78fdcdacf2b041efeec032d5dae34190cb2b7c5dbc63df6f93ab" gracePeriod=30 Dec 03 12:47:18 crc kubenswrapper[5052]: I1203 12:47:18.182438 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-api-0" podUID="1e70c60d-8835-4b4f-80dc-7190e8f6facd" containerName="nova-api-api" containerID="cri-o://338686a6402baec72b3a148a936c96c0893afe29ea77e3b16918de13c7a7ba42" gracePeriod=30 Dec 03 12:47:18 crc kubenswrapper[5052]: I1203 12:47:18.184525 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" event={"ID":"edae7848-51ff-40d6-83f2-ffff085f3f3e","Type":"ContainerStarted","Data":"1470a22e3c4e2dd79a87f38bb53dbab1acc3ac13debd607ce9ac04a443f55380"} Dec 03 12:47:18 crc kubenswrapper[5052]: I1203 12:47:18.185620 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:47:18 crc kubenswrapper[5052]: I1203 12:47:18.185785 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:47:18 crc kubenswrapper[5052]: I1203 12:47:18.185874 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:47:18 crc kubenswrapper[5052]: I1203 12:47:18.185951 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:47:18 crc kubenswrapper[5052]: I1203 12:47:18.236591 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" podStartSLOduration=3.236572267 podStartE2EDuration="3.236572267s" podCreationTimestamp="2025-12-03 12:47:15 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:47:18.231970755 +0000 UTC m=+5228.820212539" watchObservedRunningTime="2025-12-03 12:47:18.236572267 +0000 UTC m=+5228.824814031" Dec 03 12:47:18 crc kubenswrapper[5052]: I1203 12:47:18.264641 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-api-0" podStartSLOduration=3.264622979 podStartE2EDuration="3.264622979s" podCreationTimestamp="2025-12-03 12:47:15 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:47:18.24994882 +0000 UTC m=+5228.838190594" watchObservedRunningTime="2025-12-03 12:47:18.264622979 +0000 UTC m=+5228.852864743" Dec 03 12:47:18 crc kubenswrapper[5052]: I1203 12:47:18.663809 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:47:18 crc kubenswrapper[5052]: I1203 12:47:18.688295 5052 kubelet.go:2542] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:47:18 crc kubenswrapper[5052]: I1203 12:47:18.782162 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/75266130-5854-4acd-acc9-bc22accc19bc-config-data\") pod \"75266130-5854-4acd-acc9-bc22accc19bc\" (UID: \"75266130-5854-4acd-acc9-bc22accc19bc\") " Dec 03 12:47:18 crc kubenswrapper[5052]: I1203 12:47:18.782226 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/75266130-5854-4acd-acc9-bc22accc19bc-combined-ca-bundle\") pod \"75266130-5854-4acd-acc9-bc22accc19bc\" (UID: \"75266130-5854-4acd-acc9-bc22accc19bc\") " Dec 03 12:47:18 crc kubenswrapper[5052]: I1203 12:47:18.782364 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zw7qg\" (UniqueName: \"kubernetes.io/projected/75266130-5854-4acd-acc9-bc22accc19bc-kube-api-access-zw7qg\") pod \"75266130-5854-4acd-acc9-bc22accc19bc\" (UID: \"75266130-5854-4acd-acc9-bc22accc19bc\") " Dec 03 12:47:18 crc kubenswrapper[5052]: I1203 12:47:18.782388 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/75266130-5854-4acd-acc9-bc22accc19bc-nova-metadata-tls-certs\") pod \"75266130-5854-4acd-acc9-bc22accc19bc\" (UID: \"75266130-5854-4acd-acc9-bc22accc19bc\") " Dec 03 12:47:18 crc kubenswrapper[5052]: I1203 12:47:18.782473 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/75266130-5854-4acd-acc9-bc22accc19bc-logs\") pod \"75266130-5854-4acd-acc9-bc22accc19bc\" (UID: \"75266130-5854-4acd-acc9-bc22accc19bc\") " Dec 03 12:47:18 crc kubenswrapper[5052]: I1203 12:47:18.784690 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/75266130-5854-4acd-acc9-bc22accc19bc-logs" (OuterVolumeSpecName: "logs") pod "75266130-5854-4acd-acc9-bc22accc19bc" (UID: "75266130-5854-4acd-acc9-bc22accc19bc"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:47:18 crc kubenswrapper[5052]: I1203 12:47:18.791096 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:47:18 crc kubenswrapper[5052]: I1203 12:47:18.797716 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/75266130-5854-4acd-acc9-bc22accc19bc-kube-api-access-zw7qg" (OuterVolumeSpecName: "kube-api-access-zw7qg") pod "75266130-5854-4acd-acc9-bc22accc19bc" (UID: "75266130-5854-4acd-acc9-bc22accc19bc"). InnerVolumeSpecName "kube-api-access-zw7qg". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:47:18 crc kubenswrapper[5052]: I1203 12:47:18.840719 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/75266130-5854-4acd-acc9-bc22accc19bc-config-data" (OuterVolumeSpecName: "config-data") pod "75266130-5854-4acd-acc9-bc22accc19bc" (UID: "75266130-5854-4acd-acc9-bc22accc19bc"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:47:18 crc kubenswrapper[5052]: I1203 12:47:18.844119 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/75266130-5854-4acd-acc9-bc22accc19bc-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "75266130-5854-4acd-acc9-bc22accc19bc" (UID: "75266130-5854-4acd-acc9-bc22accc19bc"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:47:18 crc kubenswrapper[5052]: I1203 12:47:18.877713 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/75266130-5854-4acd-acc9-bc22accc19bc-nova-metadata-tls-certs" (OuterVolumeSpecName: "nova-metadata-tls-certs") pod "75266130-5854-4acd-acc9-bc22accc19bc" (UID: "75266130-5854-4acd-acc9-bc22accc19bc"). InnerVolumeSpecName "nova-metadata-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:47:18 crc kubenswrapper[5052]: I1203 12:47:18.887272 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/1e70c60d-8835-4b4f-80dc-7190e8f6facd-logs\") pod \"1e70c60d-8835-4b4f-80dc-7190e8f6facd\" (UID: \"1e70c60d-8835-4b4f-80dc-7190e8f6facd\") " Dec 03 12:47:18 crc kubenswrapper[5052]: I1203 12:47:18.887369 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1e70c60d-8835-4b4f-80dc-7190e8f6facd-combined-ca-bundle\") pod \"1e70c60d-8835-4b4f-80dc-7190e8f6facd\" (UID: \"1e70c60d-8835-4b4f-80dc-7190e8f6facd\") " Dec 03 12:47:18 crc kubenswrapper[5052]: I1203 12:47:18.887502 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/1e70c60d-8835-4b4f-80dc-7190e8f6facd-public-tls-certs\") pod \"1e70c60d-8835-4b4f-80dc-7190e8f6facd\" (UID: \"1e70c60d-8835-4b4f-80dc-7190e8f6facd\") " Dec 03 12:47:18 crc kubenswrapper[5052]: I1203 12:47:18.887542 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/1e70c60d-8835-4b4f-80dc-7190e8f6facd-internal-tls-certs\") pod \"1e70c60d-8835-4b4f-80dc-7190e8f6facd\" (UID: \"1e70c60d-8835-4b4f-80dc-7190e8f6facd\") " Dec 03 12:47:18 crc kubenswrapper[5052]: I1203 12:47:18.887606 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bbtns\" (UniqueName: \"kubernetes.io/projected/1e70c60d-8835-4b4f-80dc-7190e8f6facd-kube-api-access-bbtns\") pod \"1e70c60d-8835-4b4f-80dc-7190e8f6facd\" (UID: \"1e70c60d-8835-4b4f-80dc-7190e8f6facd\") " Dec 03 12:47:18 crc kubenswrapper[5052]: I1203 12:47:18.887638 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/1e70c60d-8835-4b4f-80dc-7190e8f6facd-config-data\") pod \"1e70c60d-8835-4b4f-80dc-7190e8f6facd\" (UID: \"1e70c60d-8835-4b4f-80dc-7190e8f6facd\") " Dec 03 12:47:18 crc kubenswrapper[5052]: I1203 12:47:18.888086 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/75266130-5854-4acd-acc9-bc22accc19bc-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:47:18 crc kubenswrapper[5052]: I1203 12:47:18.888109 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/75266130-5854-4acd-acc9-bc22accc19bc-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:47:18 crc kubenswrapper[5052]: I1203 12:47:18.888121 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-zw7qg\" (UniqueName: \"kubernetes.io/projected/75266130-5854-4acd-acc9-bc22accc19bc-kube-api-access-zw7qg\") on node \"crc\" DevicePath \"\"" Dec 03 12:47:18 crc kubenswrapper[5052]: I1203 12:47:18.888131 5052 reconciler_common.go:293] "Volume detached for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/75266130-5854-4acd-acc9-bc22accc19bc-nova-metadata-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:47:18 crc kubenswrapper[5052]: I1203 12:47:18.888141 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/75266130-5854-4acd-acc9-bc22accc19bc-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:47:18 crc kubenswrapper[5052]: I1203 12:47:18.888865 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1e70c60d-8835-4b4f-80dc-7190e8f6facd-logs" (OuterVolumeSpecName: "logs") pod "1e70c60d-8835-4b4f-80dc-7190e8f6facd" (UID: "1e70c60d-8835-4b4f-80dc-7190e8f6facd"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:47:18 crc kubenswrapper[5052]: I1203 12:47:18.894332 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1e70c60d-8835-4b4f-80dc-7190e8f6facd-kube-api-access-bbtns" (OuterVolumeSpecName: "kube-api-access-bbtns") pod "1e70c60d-8835-4b4f-80dc-7190e8f6facd" (UID: "1e70c60d-8835-4b4f-80dc-7190e8f6facd"). InnerVolumeSpecName "kube-api-access-bbtns". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:47:18 crc kubenswrapper[5052]: I1203 12:47:18.917832 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1e70c60d-8835-4b4f-80dc-7190e8f6facd-config-data" (OuterVolumeSpecName: "config-data") pod "1e70c60d-8835-4b4f-80dc-7190e8f6facd" (UID: "1e70c60d-8835-4b4f-80dc-7190e8f6facd"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:47:18 crc kubenswrapper[5052]: I1203 12:47:18.939387 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1e70c60d-8835-4b4f-80dc-7190e8f6facd-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "1e70c60d-8835-4b4f-80dc-7190e8f6facd" (UID: "1e70c60d-8835-4b4f-80dc-7190e8f6facd"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:47:18 crc kubenswrapper[5052]: I1203 12:47:18.968184 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1e70c60d-8835-4b4f-80dc-7190e8f6facd-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "1e70c60d-8835-4b4f-80dc-7190e8f6facd" (UID: "1e70c60d-8835-4b4f-80dc-7190e8f6facd"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:47:18 crc kubenswrapper[5052]: I1203 12:47:18.989814 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1e70c60d-8835-4b4f-80dc-7190e8f6facd-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:47:18 crc kubenswrapper[5052]: I1203 12:47:18.990128 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/1e70c60d-8835-4b4f-80dc-7190e8f6facd-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:47:18 crc kubenswrapper[5052]: I1203 12:47:18.990219 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-bbtns\" (UniqueName: \"kubernetes.io/projected/1e70c60d-8835-4b4f-80dc-7190e8f6facd-kube-api-access-bbtns\") on node \"crc\" DevicePath \"\"" Dec 03 12:47:18 crc kubenswrapper[5052]: I1203 12:47:18.990307 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/1e70c60d-8835-4b4f-80dc-7190e8f6facd-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:47:18 crc kubenswrapper[5052]: I1203 12:47:18.990392 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/1e70c60d-8835-4b4f-80dc-7190e8f6facd-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:47:18 crc kubenswrapper[5052]: I1203 12:47:18.992861 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1e70c60d-8835-4b4f-80dc-7190e8f6facd-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "1e70c60d-8835-4b4f-80dc-7190e8f6facd" (UID: "1e70c60d-8835-4b4f-80dc-7190e8f6facd"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:47:19 crc kubenswrapper[5052]: I1203 12:47:19.038040 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/barbican-api-75bdf7fcbb-mlkgz"] Dec 03 12:47:19 crc kubenswrapper[5052]: E1203 12:47:19.038508 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="1e70c60d-8835-4b4f-80dc-7190e8f6facd" containerName="nova-api-api" Dec 03 12:47:19 crc kubenswrapper[5052]: I1203 12:47:19.038545 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="1e70c60d-8835-4b4f-80dc-7190e8f6facd" containerName="nova-api-api" Dec 03 12:47:19 crc kubenswrapper[5052]: E1203 12:47:19.038558 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="5b63f869-c95a-4279-b542-3ebbec1a1fd5" containerName="barbican-api" Dec 03 12:47:19 crc kubenswrapper[5052]: I1203 12:47:19.038565 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="5b63f869-c95a-4279-b542-3ebbec1a1fd5" containerName="barbican-api" Dec 03 12:47:19 crc kubenswrapper[5052]: E1203 12:47:19.038579 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="75266130-5854-4acd-acc9-bc22accc19bc" containerName="nova-metadata-metadata" Dec 03 12:47:19 crc kubenswrapper[5052]: I1203 12:47:19.038586 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="75266130-5854-4acd-acc9-bc22accc19bc" containerName="nova-metadata-metadata" Dec 03 12:47:19 crc kubenswrapper[5052]: E1203 12:47:19.038609 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="1e70c60d-8835-4b4f-80dc-7190e8f6facd" containerName="nova-api-log" Dec 03 12:47:19 crc kubenswrapper[5052]: I1203 12:47:19.038615 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="1e70c60d-8835-4b4f-80dc-7190e8f6facd" containerName="nova-api-log" Dec 03 12:47:19 crc kubenswrapper[5052]: E1203 12:47:19.038631 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="5b63f869-c95a-4279-b542-3ebbec1a1fd5" containerName="barbican-api-log" Dec 03 12:47:19 crc kubenswrapper[5052]: I1203 12:47:19.038637 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="5b63f869-c95a-4279-b542-3ebbec1a1fd5" containerName="barbican-api-log" Dec 03 12:47:19 crc kubenswrapper[5052]: E1203 12:47:19.038648 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="75266130-5854-4acd-acc9-bc22accc19bc" containerName="nova-metadata-log" Dec 03 12:47:19 crc kubenswrapper[5052]: I1203 12:47:19.038654 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="75266130-5854-4acd-acc9-bc22accc19bc" containerName="nova-metadata-log" Dec 03 12:47:19 crc kubenswrapper[5052]: I1203 12:47:19.038868 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="5b63f869-c95a-4279-b542-3ebbec1a1fd5" containerName="barbican-api-log" Dec 03 12:47:19 crc kubenswrapper[5052]: I1203 12:47:19.038892 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="1e70c60d-8835-4b4f-80dc-7190e8f6facd" containerName="nova-api-api" Dec 03 12:47:19 crc kubenswrapper[5052]: I1203 12:47:19.038907 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="75266130-5854-4acd-acc9-bc22accc19bc" containerName="nova-metadata-log" Dec 03 12:47:19 crc kubenswrapper[5052]: I1203 12:47:19.038917 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="5b63f869-c95a-4279-b542-3ebbec1a1fd5" containerName="barbican-api" Dec 03 12:47:19 crc kubenswrapper[5052]: I1203 12:47:19.038944 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="1e70c60d-8835-4b4f-80dc-7190e8f6facd" containerName="nova-api-log" Dec 03 12:47:19 crc kubenswrapper[5052]: I1203 12:47:19.038962 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="75266130-5854-4acd-acc9-bc22accc19bc" containerName="nova-metadata-metadata" Dec 03 12:47:19 crc kubenswrapper[5052]: I1203 12:47:19.041356 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-api-75bdf7fcbb-mlkgz" Dec 03 12:47:19 crc kubenswrapper[5052]: I1203 12:47:19.071376 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-api-75bdf7fcbb-mlkgz"] Dec 03 12:47:19 crc kubenswrapper[5052]: I1203 12:47:19.092266 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/b8eb690c-165a-4906-a750-bec0178daf6c-config-data-custom\") pod \"barbican-api-75bdf7fcbb-mlkgz\" (UID: \"b8eb690c-165a-4906-a750-bec0178daf6c\") " pod="openstack-kuttl-tests/barbican-api-75bdf7fcbb-mlkgz" Dec 03 12:47:19 crc kubenswrapper[5052]: I1203 12:47:19.092321 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/b8eb690c-165a-4906-a750-bec0178daf6c-public-tls-certs\") pod \"barbican-api-75bdf7fcbb-mlkgz\" (UID: \"b8eb690c-165a-4906-a750-bec0178daf6c\") " pod="openstack-kuttl-tests/barbican-api-75bdf7fcbb-mlkgz" Dec 03 12:47:19 crc kubenswrapper[5052]: I1203 12:47:19.092356 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/b8eb690c-165a-4906-a750-bec0178daf6c-logs\") pod \"barbican-api-75bdf7fcbb-mlkgz\" (UID: \"b8eb690c-165a-4906-a750-bec0178daf6c\") " pod="openstack-kuttl-tests/barbican-api-75bdf7fcbb-mlkgz" Dec 03 12:47:19 crc kubenswrapper[5052]: I1203 12:47:19.092381 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b8eb690c-165a-4906-a750-bec0178daf6c-combined-ca-bundle\") pod \"barbican-api-75bdf7fcbb-mlkgz\" (UID: \"b8eb690c-165a-4906-a750-bec0178daf6c\") " pod="openstack-kuttl-tests/barbican-api-75bdf7fcbb-mlkgz" Dec 03 12:47:19 crc kubenswrapper[5052]: I1203 12:47:19.092404 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/b8eb690c-165a-4906-a750-bec0178daf6c-internal-tls-certs\") pod \"barbican-api-75bdf7fcbb-mlkgz\" (UID: \"b8eb690c-165a-4906-a750-bec0178daf6c\") " pod="openstack-kuttl-tests/barbican-api-75bdf7fcbb-mlkgz" Dec 03 12:47:19 crc kubenswrapper[5052]: I1203 12:47:19.092457 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-znpn4\" (UniqueName: \"kubernetes.io/projected/b8eb690c-165a-4906-a750-bec0178daf6c-kube-api-access-znpn4\") pod \"barbican-api-75bdf7fcbb-mlkgz\" (UID: \"b8eb690c-165a-4906-a750-bec0178daf6c\") " pod="openstack-kuttl-tests/barbican-api-75bdf7fcbb-mlkgz" Dec 03 12:47:19 crc kubenswrapper[5052]: I1203 12:47:19.092541 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/b8eb690c-165a-4906-a750-bec0178daf6c-config-data\") pod \"barbican-api-75bdf7fcbb-mlkgz\" (UID: \"b8eb690c-165a-4906-a750-bec0178daf6c\") " pod="openstack-kuttl-tests/barbican-api-75bdf7fcbb-mlkgz" Dec 03 12:47:19 crc kubenswrapper[5052]: I1203 12:47:19.092638 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/1e70c60d-8835-4b4f-80dc-7190e8f6facd-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:47:19 crc kubenswrapper[5052]: I1203 12:47:19.195201 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-znpn4\" (UniqueName: \"kubernetes.io/projected/b8eb690c-165a-4906-a750-bec0178daf6c-kube-api-access-znpn4\") pod \"barbican-api-75bdf7fcbb-mlkgz\" (UID: \"b8eb690c-165a-4906-a750-bec0178daf6c\") " pod="openstack-kuttl-tests/barbican-api-75bdf7fcbb-mlkgz" Dec 03 12:47:19 crc kubenswrapper[5052]: I1203 12:47:19.195777 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/b8eb690c-165a-4906-a750-bec0178daf6c-config-data\") pod \"barbican-api-75bdf7fcbb-mlkgz\" (UID: \"b8eb690c-165a-4906-a750-bec0178daf6c\") " pod="openstack-kuttl-tests/barbican-api-75bdf7fcbb-mlkgz" Dec 03 12:47:19 crc kubenswrapper[5052]: I1203 12:47:19.195940 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/b8eb690c-165a-4906-a750-bec0178daf6c-config-data-custom\") pod \"barbican-api-75bdf7fcbb-mlkgz\" (UID: \"b8eb690c-165a-4906-a750-bec0178daf6c\") " pod="openstack-kuttl-tests/barbican-api-75bdf7fcbb-mlkgz" Dec 03 12:47:19 crc kubenswrapper[5052]: I1203 12:47:19.195974 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/b8eb690c-165a-4906-a750-bec0178daf6c-public-tls-certs\") pod \"barbican-api-75bdf7fcbb-mlkgz\" (UID: \"b8eb690c-165a-4906-a750-bec0178daf6c\") " pod="openstack-kuttl-tests/barbican-api-75bdf7fcbb-mlkgz" Dec 03 12:47:19 crc kubenswrapper[5052]: I1203 12:47:19.196003 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/b8eb690c-165a-4906-a750-bec0178daf6c-logs\") pod \"barbican-api-75bdf7fcbb-mlkgz\" (UID: \"b8eb690c-165a-4906-a750-bec0178daf6c\") " pod="openstack-kuttl-tests/barbican-api-75bdf7fcbb-mlkgz" Dec 03 12:47:19 crc kubenswrapper[5052]: I1203 12:47:19.196024 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b8eb690c-165a-4906-a750-bec0178daf6c-combined-ca-bundle\") pod \"barbican-api-75bdf7fcbb-mlkgz\" (UID: \"b8eb690c-165a-4906-a750-bec0178daf6c\") " pod="openstack-kuttl-tests/barbican-api-75bdf7fcbb-mlkgz" Dec 03 12:47:19 crc kubenswrapper[5052]: I1203 12:47:19.196044 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/b8eb690c-165a-4906-a750-bec0178daf6c-internal-tls-certs\") pod \"barbican-api-75bdf7fcbb-mlkgz\" (UID: \"b8eb690c-165a-4906-a750-bec0178daf6c\") " pod="openstack-kuttl-tests/barbican-api-75bdf7fcbb-mlkgz" Dec 03 12:47:19 crc kubenswrapper[5052]: I1203 12:47:19.197375 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/b8eb690c-165a-4906-a750-bec0178daf6c-logs\") pod \"barbican-api-75bdf7fcbb-mlkgz\" (UID: \"b8eb690c-165a-4906-a750-bec0178daf6c\") " pod="openstack-kuttl-tests/barbican-api-75bdf7fcbb-mlkgz" Dec 03 12:47:19 crc kubenswrapper[5052]: I1203 12:47:19.211988 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/b8eb690c-165a-4906-a750-bec0178daf6c-public-tls-certs\") pod \"barbican-api-75bdf7fcbb-mlkgz\" (UID: \"b8eb690c-165a-4906-a750-bec0178daf6c\") " pod="openstack-kuttl-tests/barbican-api-75bdf7fcbb-mlkgz" Dec 03 12:47:19 crc kubenswrapper[5052]: I1203 12:47:19.212012 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/b8eb690c-165a-4906-a750-bec0178daf6c-config-data-custom\") pod \"barbican-api-75bdf7fcbb-mlkgz\" (UID: \"b8eb690c-165a-4906-a750-bec0178daf6c\") " pod="openstack-kuttl-tests/barbican-api-75bdf7fcbb-mlkgz" Dec 03 12:47:19 crc kubenswrapper[5052]: I1203 12:47:19.212245 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/b8eb690c-165a-4906-a750-bec0178daf6c-config-data\") pod \"barbican-api-75bdf7fcbb-mlkgz\" (UID: \"b8eb690c-165a-4906-a750-bec0178daf6c\") " pod="openstack-kuttl-tests/barbican-api-75bdf7fcbb-mlkgz" Dec 03 12:47:19 crc kubenswrapper[5052]: I1203 12:47:19.212858 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/b8eb690c-165a-4906-a750-bec0178daf6c-internal-tls-certs\") pod \"barbican-api-75bdf7fcbb-mlkgz\" (UID: \"b8eb690c-165a-4906-a750-bec0178daf6c\") " pod="openstack-kuttl-tests/barbican-api-75bdf7fcbb-mlkgz" Dec 03 12:47:19 crc kubenswrapper[5052]: I1203 12:47:19.214705 5052 scope.go:117] "RemoveContainer" containerID="1fdbade92a13691b593283a5b263382dfec190075f1201c77fff78a7be754487" Dec 03 12:47:19 crc kubenswrapper[5052]: E1203 12:47:19.215305 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"nova-cell0-conductor-conductor\" with CrashLoopBackOff: \"back-off 40s restarting failed container=nova-cell0-conductor-conductor pod=nova-cell0-conductor-0_openstack-kuttl-tests(6d653682-52e2-4940-af66-633475bfe09a)\"" pod="openstack-kuttl-tests/nova-cell0-conductor-0" podUID="6d653682-52e2-4940-af66-633475bfe09a" Dec 03 12:47:19 crc kubenswrapper[5052]: I1203 12:47:19.216933 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b8eb690c-165a-4906-a750-bec0178daf6c-combined-ca-bundle\") pod \"barbican-api-75bdf7fcbb-mlkgz\" (UID: \"b8eb690c-165a-4906-a750-bec0178daf6c\") " pod="openstack-kuttl-tests/barbican-api-75bdf7fcbb-mlkgz" Dec 03 12:47:19 crc kubenswrapper[5052]: I1203 12:47:19.234944 5052 generic.go:334] "Generic (PLEG): container finished" podID="1e70c60d-8835-4b4f-80dc-7190e8f6facd" containerID="338686a6402baec72b3a148a936c96c0893afe29ea77e3b16918de13c7a7ba42" exitCode=0 Dec 03 12:47:19 crc kubenswrapper[5052]: I1203 12:47:19.235039 5052 generic.go:334] "Generic (PLEG): container finished" podID="1e70c60d-8835-4b4f-80dc-7190e8f6facd" containerID="21ef156febcb78fdcdacf2b041efeec032d5dae34190cb2b7c5dbc63df6f93ab" exitCode=143 Dec 03 12:47:19 crc kubenswrapper[5052]: I1203 12:47:19.235168 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"1e70c60d-8835-4b4f-80dc-7190e8f6facd","Type":"ContainerDied","Data":"338686a6402baec72b3a148a936c96c0893afe29ea77e3b16918de13c7a7ba42"} Dec 03 12:47:19 crc kubenswrapper[5052]: I1203 12:47:19.235217 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"1e70c60d-8835-4b4f-80dc-7190e8f6facd","Type":"ContainerDied","Data":"21ef156febcb78fdcdacf2b041efeec032d5dae34190cb2b7c5dbc63df6f93ab"} Dec 03 12:47:19 crc kubenswrapper[5052]: I1203 12:47:19.235233 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"1e70c60d-8835-4b4f-80dc-7190e8f6facd","Type":"ContainerDied","Data":"ab41ba3190a253c4fde3b178de0a2fcbce57b7c5a7424a3d906631e122b2a3e0"} Dec 03 12:47:19 crc kubenswrapper[5052]: I1203 12:47:19.235263 5052 scope.go:117] "RemoveContainer" containerID="338686a6402baec72b3a148a936c96c0893afe29ea77e3b16918de13c7a7ba42" Dec 03 12:47:19 crc kubenswrapper[5052]: I1203 12:47:19.235540 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:47:19 crc kubenswrapper[5052]: I1203 12:47:19.241849 5052 generic.go:334] "Generic (PLEG): container finished" podID="75266130-5854-4acd-acc9-bc22accc19bc" containerID="f494000fc74b8d824265c717da4442151f3a4e76a4efbba51ec6695872f15bb0" exitCode=0 Dec 03 12:47:19 crc kubenswrapper[5052]: I1203 12:47:19.242055 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"75266130-5854-4acd-acc9-bc22accc19bc","Type":"ContainerDied","Data":"f494000fc74b8d824265c717da4442151f3a4e76a4efbba51ec6695872f15bb0"} Dec 03 12:47:19 crc kubenswrapper[5052]: I1203 12:47:19.242176 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"75266130-5854-4acd-acc9-bc22accc19bc","Type":"ContainerDied","Data":"0ef35cf8ef0a727c1b5075e4b54d1d87e062ce8c5d2c4f60eb3f0c1b8e2b6917"} Dec 03 12:47:19 crc kubenswrapper[5052]: I1203 12:47:19.242224 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-znpn4\" (UniqueName: \"kubernetes.io/projected/b8eb690c-165a-4906-a750-bec0178daf6c-kube-api-access-znpn4\") pod \"barbican-api-75bdf7fcbb-mlkgz\" (UID: \"b8eb690c-165a-4906-a750-bec0178daf6c\") " pod="openstack-kuttl-tests/barbican-api-75bdf7fcbb-mlkgz" Dec 03 12:47:19 crc kubenswrapper[5052]: I1203 12:47:19.242421 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:47:19 crc kubenswrapper[5052]: I1203 12:47:19.363008 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-api-75bdf7fcbb-mlkgz" Dec 03 12:47:19 crc kubenswrapper[5052]: I1203 12:47:19.409508 5052 scope.go:117] "RemoveContainer" containerID="21ef156febcb78fdcdacf2b041efeec032d5dae34190cb2b7c5dbc63df6f93ab" Dec 03 12:47:19 crc kubenswrapper[5052]: I1203 12:47:19.891709 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-api-75bdf7fcbb-mlkgz"] Dec 03 12:47:20 crc kubenswrapper[5052]: I1203 12:47:20.162850 5052 scope.go:117] "RemoveContainer" containerID="338686a6402baec72b3a148a936c96c0893afe29ea77e3b16918de13c7a7ba42" Dec 03 12:47:20 crc kubenswrapper[5052]: E1203 12:47:20.163882 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"338686a6402baec72b3a148a936c96c0893afe29ea77e3b16918de13c7a7ba42\": container with ID starting with 338686a6402baec72b3a148a936c96c0893afe29ea77e3b16918de13c7a7ba42 not found: ID does not exist" containerID="338686a6402baec72b3a148a936c96c0893afe29ea77e3b16918de13c7a7ba42" Dec 03 12:47:20 crc kubenswrapper[5052]: I1203 12:47:20.163918 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"338686a6402baec72b3a148a936c96c0893afe29ea77e3b16918de13c7a7ba42"} err="failed to get container status \"338686a6402baec72b3a148a936c96c0893afe29ea77e3b16918de13c7a7ba42\": rpc error: code = NotFound desc = could not find container \"338686a6402baec72b3a148a936c96c0893afe29ea77e3b16918de13c7a7ba42\": container with ID starting with 338686a6402baec72b3a148a936c96c0893afe29ea77e3b16918de13c7a7ba42 not found: ID does not exist" Dec 03 12:47:20 crc kubenswrapper[5052]: I1203 12:47:20.163947 5052 scope.go:117] "RemoveContainer" containerID="21ef156febcb78fdcdacf2b041efeec032d5dae34190cb2b7c5dbc63df6f93ab" Dec 03 12:47:20 crc kubenswrapper[5052]: E1203 12:47:20.164417 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"21ef156febcb78fdcdacf2b041efeec032d5dae34190cb2b7c5dbc63df6f93ab\": container with ID starting with 21ef156febcb78fdcdacf2b041efeec032d5dae34190cb2b7c5dbc63df6f93ab not found: ID does not exist" containerID="21ef156febcb78fdcdacf2b041efeec032d5dae34190cb2b7c5dbc63df6f93ab" Dec 03 12:47:20 crc kubenswrapper[5052]: I1203 12:47:20.164497 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"21ef156febcb78fdcdacf2b041efeec032d5dae34190cb2b7c5dbc63df6f93ab"} err="failed to get container status \"21ef156febcb78fdcdacf2b041efeec032d5dae34190cb2b7c5dbc63df6f93ab\": rpc error: code = NotFound desc = could not find container \"21ef156febcb78fdcdacf2b041efeec032d5dae34190cb2b7c5dbc63df6f93ab\": container with ID starting with 21ef156febcb78fdcdacf2b041efeec032d5dae34190cb2b7c5dbc63df6f93ab not found: ID does not exist" Dec 03 12:47:20 crc kubenswrapper[5052]: I1203 12:47:20.164562 5052 scope.go:117] "RemoveContainer" containerID="338686a6402baec72b3a148a936c96c0893afe29ea77e3b16918de13c7a7ba42" Dec 03 12:47:20 crc kubenswrapper[5052]: I1203 12:47:20.167471 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"338686a6402baec72b3a148a936c96c0893afe29ea77e3b16918de13c7a7ba42"} err="failed to get container status \"338686a6402baec72b3a148a936c96c0893afe29ea77e3b16918de13c7a7ba42\": rpc error: code = NotFound desc = could not find container \"338686a6402baec72b3a148a936c96c0893afe29ea77e3b16918de13c7a7ba42\": container with ID starting with 338686a6402baec72b3a148a936c96c0893afe29ea77e3b16918de13c7a7ba42 not found: ID does not exist" Dec 03 12:47:20 crc kubenswrapper[5052]: I1203 12:47:20.167638 5052 scope.go:117] "RemoveContainer" containerID="21ef156febcb78fdcdacf2b041efeec032d5dae34190cb2b7c5dbc63df6f93ab" Dec 03 12:47:20 crc kubenswrapper[5052]: I1203 12:47:20.171631 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"21ef156febcb78fdcdacf2b041efeec032d5dae34190cb2b7c5dbc63df6f93ab"} err="failed to get container status \"21ef156febcb78fdcdacf2b041efeec032d5dae34190cb2b7c5dbc63df6f93ab\": rpc error: code = NotFound desc = could not find container \"21ef156febcb78fdcdacf2b041efeec032d5dae34190cb2b7c5dbc63df6f93ab\": container with ID starting with 21ef156febcb78fdcdacf2b041efeec032d5dae34190cb2b7c5dbc63df6f93ab not found: ID does not exist" Dec 03 12:47:20 crc kubenswrapper[5052]: I1203 12:47:20.171673 5052 scope.go:117] "RemoveContainer" containerID="f494000fc74b8d824265c717da4442151f3a4e76a4efbba51ec6695872f15bb0" Dec 03 12:47:20 crc kubenswrapper[5052]: I1203 12:47:20.273263 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-75bdf7fcbb-mlkgz" event={"ID":"b8eb690c-165a-4906-a750-bec0178daf6c","Type":"ContainerStarted","Data":"23a7cc9313c35930de4dadf709c0ffaca2b9ae4bdea1ed8ac27170c5acee546f"} Dec 03 12:47:20 crc kubenswrapper[5052]: I1203 12:47:20.275572 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:47:20 crc kubenswrapper[5052]: I1203 12:47:20.278148 5052 scope.go:117] "RemoveContainer" containerID="3e1d7a5d05726614ff20caaaa6d897aa3cbb28e2e859ab554810ad90e4b4a72c" Dec 03 12:47:20 crc kubenswrapper[5052]: I1203 12:47:20.279321 5052 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Dec 03 12:47:20 crc kubenswrapper[5052]: I1203 12:47:20.279339 5052 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Dec 03 12:47:20 crc kubenswrapper[5052]: I1203 12:47:20.280035 5052 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Dec 03 12:47:20 crc kubenswrapper[5052]: I1203 12:47:20.280054 5052 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Dec 03 12:47:20 crc kubenswrapper[5052]: I1203 12:47:20.303873 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:47:20 crc kubenswrapper[5052]: I1203 12:47:20.316695 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:47:20 crc kubenswrapper[5052]: I1203 12:47:20.319308 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:47:20 crc kubenswrapper[5052]: I1203 12:47:20.321774 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-metadata-config-data" Dec 03 12:47:20 crc kubenswrapper[5052]: I1203 12:47:20.322148 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-nova-metadata-internal-svc" Dec 03 12:47:20 crc kubenswrapper[5052]: I1203 12:47:20.327892 5052 scope.go:117] "RemoveContainer" containerID="f494000fc74b8d824265c717da4442151f3a4e76a4efbba51ec6695872f15bb0" Dec 03 12:47:20 crc kubenswrapper[5052]: E1203 12:47:20.329652 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f494000fc74b8d824265c717da4442151f3a4e76a4efbba51ec6695872f15bb0\": container with ID starting with f494000fc74b8d824265c717da4442151f3a4e76a4efbba51ec6695872f15bb0 not found: ID does not exist" containerID="f494000fc74b8d824265c717da4442151f3a4e76a4efbba51ec6695872f15bb0" Dec 03 12:47:20 crc kubenswrapper[5052]: I1203 12:47:20.329695 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f494000fc74b8d824265c717da4442151f3a4e76a4efbba51ec6695872f15bb0"} err="failed to get container status \"f494000fc74b8d824265c717da4442151f3a4e76a4efbba51ec6695872f15bb0\": rpc error: code = NotFound desc = could not find container \"f494000fc74b8d824265c717da4442151f3a4e76a4efbba51ec6695872f15bb0\": container with ID starting with f494000fc74b8d824265c717da4442151f3a4e76a4efbba51ec6695872f15bb0 not found: ID does not exist" Dec 03 12:47:20 crc kubenswrapper[5052]: I1203 12:47:20.329724 5052 scope.go:117] "RemoveContainer" containerID="3e1d7a5d05726614ff20caaaa6d897aa3cbb28e2e859ab554810ad90e4b4a72c" Dec 03 12:47:20 crc kubenswrapper[5052]: E1203 12:47:20.330015 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3e1d7a5d05726614ff20caaaa6d897aa3cbb28e2e859ab554810ad90e4b4a72c\": container with ID starting with 3e1d7a5d05726614ff20caaaa6d897aa3cbb28e2e859ab554810ad90e4b4a72c not found: ID does not exist" containerID="3e1d7a5d05726614ff20caaaa6d897aa3cbb28e2e859ab554810ad90e4b4a72c" Dec 03 12:47:20 crc kubenswrapper[5052]: I1203 12:47:20.330033 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3e1d7a5d05726614ff20caaaa6d897aa3cbb28e2e859ab554810ad90e4b4a72c"} err="failed to get container status \"3e1d7a5d05726614ff20caaaa6d897aa3cbb28e2e859ab554810ad90e4b4a72c\": rpc error: code = NotFound desc = could not find container \"3e1d7a5d05726614ff20caaaa6d897aa3cbb28e2e859ab554810ad90e4b4a72c\": container with ID starting with 3e1d7a5d05726614ff20caaaa6d897aa3cbb28e2e859ab554810ad90e4b4a72c not found: ID does not exist" Dec 03 12:47:20 crc kubenswrapper[5052]: I1203 12:47:20.331921 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:47:20 crc kubenswrapper[5052]: I1203 12:47:20.434773 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/efb0e60c-2a71-4641-87b2-0f5532b28678-config-data\") pod \"nova-metadata-0\" (UID: \"efb0e60c-2a71-4641-87b2-0f5532b28678\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:47:20 crc kubenswrapper[5052]: I1203 12:47:20.435148 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/efb0e60c-2a71-4641-87b2-0f5532b28678-nova-metadata-tls-certs\") pod \"nova-metadata-0\" (UID: \"efb0e60c-2a71-4641-87b2-0f5532b28678\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:47:20 crc kubenswrapper[5052]: I1203 12:47:20.435200 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/efb0e60c-2a71-4641-87b2-0f5532b28678-logs\") pod \"nova-metadata-0\" (UID: \"efb0e60c-2a71-4641-87b2-0f5532b28678\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:47:20 crc kubenswrapper[5052]: I1203 12:47:20.435266 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zg457\" (UniqueName: \"kubernetes.io/projected/efb0e60c-2a71-4641-87b2-0f5532b28678-kube-api-access-zg457\") pod \"nova-metadata-0\" (UID: \"efb0e60c-2a71-4641-87b2-0f5532b28678\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:47:20 crc kubenswrapper[5052]: I1203 12:47:20.435325 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/efb0e60c-2a71-4641-87b2-0f5532b28678-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"efb0e60c-2a71-4641-87b2-0f5532b28678\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:47:20 crc kubenswrapper[5052]: I1203 12:47:20.536869 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/efb0e60c-2a71-4641-87b2-0f5532b28678-config-data\") pod \"nova-metadata-0\" (UID: \"efb0e60c-2a71-4641-87b2-0f5532b28678\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:47:20 crc kubenswrapper[5052]: I1203 12:47:20.536969 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/efb0e60c-2a71-4641-87b2-0f5532b28678-nova-metadata-tls-certs\") pod \"nova-metadata-0\" (UID: \"efb0e60c-2a71-4641-87b2-0f5532b28678\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:47:20 crc kubenswrapper[5052]: I1203 12:47:20.537013 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/efb0e60c-2a71-4641-87b2-0f5532b28678-logs\") pod \"nova-metadata-0\" (UID: \"efb0e60c-2a71-4641-87b2-0f5532b28678\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:47:20 crc kubenswrapper[5052]: I1203 12:47:20.537070 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-zg457\" (UniqueName: \"kubernetes.io/projected/efb0e60c-2a71-4641-87b2-0f5532b28678-kube-api-access-zg457\") pod \"nova-metadata-0\" (UID: \"efb0e60c-2a71-4641-87b2-0f5532b28678\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:47:20 crc kubenswrapper[5052]: I1203 12:47:20.537114 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/efb0e60c-2a71-4641-87b2-0f5532b28678-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"efb0e60c-2a71-4641-87b2-0f5532b28678\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:47:20 crc kubenswrapper[5052]: I1203 12:47:20.537974 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/efb0e60c-2a71-4641-87b2-0f5532b28678-logs\") pod \"nova-metadata-0\" (UID: \"efb0e60c-2a71-4641-87b2-0f5532b28678\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:47:20 crc kubenswrapper[5052]: I1203 12:47:20.545163 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/efb0e60c-2a71-4641-87b2-0f5532b28678-config-data\") pod \"nova-metadata-0\" (UID: \"efb0e60c-2a71-4641-87b2-0f5532b28678\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:47:20 crc kubenswrapper[5052]: I1203 12:47:20.554548 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/efb0e60c-2a71-4641-87b2-0f5532b28678-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"efb0e60c-2a71-4641-87b2-0f5532b28678\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:47:20 crc kubenswrapper[5052]: I1203 12:47:20.562607 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/efb0e60c-2a71-4641-87b2-0f5532b28678-nova-metadata-tls-certs\") pod \"nova-metadata-0\" (UID: \"efb0e60c-2a71-4641-87b2-0f5532b28678\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:47:20 crc kubenswrapper[5052]: I1203 12:47:20.562800 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-zg457\" (UniqueName: \"kubernetes.io/projected/efb0e60c-2a71-4641-87b2-0f5532b28678-kube-api-access-zg457\") pod \"nova-metadata-0\" (UID: \"efb0e60c-2a71-4641-87b2-0f5532b28678\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:47:20 crc kubenswrapper[5052]: I1203 12:47:20.626425 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:47:20 crc kubenswrapper[5052]: I1203 12:47:20.647382 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:47:20 crc kubenswrapper[5052]: I1203 12:47:20.674287 5052 scope.go:117] "RemoveContainer" containerID="2aed1c760a326f3d008799e902b1093313df3764a57c879f0b3686410127d37d" Dec 03 12:47:20 crc kubenswrapper[5052]: E1203 12:47:20.674603 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 12:47:20 crc kubenswrapper[5052]: I1203 12:47:20.908634 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:47:20 crc kubenswrapper[5052]: I1203 12:47:20.914537 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:47:20 crc kubenswrapper[5052]: I1203 12:47:20.951642 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:47:21 crc kubenswrapper[5052]: I1203 12:47:21.058188 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:47:21 crc kubenswrapper[5052]: W1203 12:47:21.184048 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podefb0e60c_2a71_4641_87b2_0f5532b28678.slice/crio-001d3190cef4c1889c7c8aab967ae547a13994c1a2d79e762912f956f402dd93 WatchSource:0}: Error finding container 001d3190cef4c1889c7c8aab967ae547a13994c1a2d79e762912f956f402dd93: Status 404 returned error can't find the container with id 001d3190cef4c1889c7c8aab967ae547a13994c1a2d79e762912f956f402dd93 Dec 03 12:47:21 crc kubenswrapper[5052]: I1203 12:47:21.185852 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:47:21 crc kubenswrapper[5052]: I1203 12:47:21.296243 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-75bdf7fcbb-mlkgz" event={"ID":"b8eb690c-165a-4906-a750-bec0178daf6c","Type":"ContainerStarted","Data":"275c7f7b179720a472f88be78dd4725e7d963350bb431c4a491cf0a780e6019a"} Dec 03 12:47:21 crc kubenswrapper[5052]: I1203 12:47:21.296288 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-75bdf7fcbb-mlkgz" event={"ID":"b8eb690c-165a-4906-a750-bec0178daf6c","Type":"ContainerStarted","Data":"1986f1fabd06880b71b76b4c94e01ab46c4e8d1cff3eae60d3b1cd7579b2c785"} Dec 03 12:47:21 crc kubenswrapper[5052]: I1203 12:47:21.304423 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"efb0e60c-2a71-4641-87b2-0f5532b28678","Type":"ContainerStarted","Data":"001d3190cef4c1889c7c8aab967ae547a13994c1a2d79e762912f956f402dd93"} Dec 03 12:47:21 crc kubenswrapper[5052]: I1203 12:47:21.673861 5052 scope.go:117] "RemoveContainer" containerID="d1e48098c512a28625a35e3a019f961f22ad8e764f883b4f8931a03d713b8c01" Dec 03 12:47:21 crc kubenswrapper[5052]: E1203 12:47:21.674334 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"nova-scheduler-scheduler\" with CrashLoopBackOff: \"back-off 20s restarting failed container=nova-scheduler-scheduler pod=nova-scheduler-0_openstack-kuttl-tests(81709983-1fd0-4446-bbbf-fb94653bb3d2)\"" pod="openstack-kuttl-tests/nova-scheduler-0" podUID="81709983-1fd0-4446-bbbf-fb94653bb3d2" Dec 03 12:47:21 crc kubenswrapper[5052]: I1203 12:47:21.687068 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="75266130-5854-4acd-acc9-bc22accc19bc" path="/var/lib/kubelet/pods/75266130-5854-4acd-acc9-bc22accc19bc/volumes" Dec 03 12:47:21 crc kubenswrapper[5052]: I1203 12:47:21.731555 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:47:21 crc kubenswrapper[5052]: I1203 12:47:21.731888 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="1b7ff11c-2d16-40e6-98f5-5e4a9a78fed3" containerName="ceilometer-central-agent" containerID="cri-o://feafdcdf212617a7513bd5d21b3d0f1577e3d60a91b5b9c44988b01a3e8d821f" gracePeriod=30 Dec 03 12:47:21 crc kubenswrapper[5052]: I1203 12:47:21.731963 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="1b7ff11c-2d16-40e6-98f5-5e4a9a78fed3" containerName="proxy-httpd" containerID="cri-o://91120088947d8c381c8640229b02472087109db20e7308332ea507f1841d751b" gracePeriod=30 Dec 03 12:47:21 crc kubenswrapper[5052]: I1203 12:47:21.732014 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="1b7ff11c-2d16-40e6-98f5-5e4a9a78fed3" containerName="sg-core" containerID="cri-o://9a89f3d83f92ed5f8d9b084b364bbf81c57e64d6a73001585da1afa98e2cb7cb" gracePeriod=30 Dec 03 12:47:21 crc kubenswrapper[5052]: I1203 12:47:21.732049 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="1b7ff11c-2d16-40e6-98f5-5e4a9a78fed3" containerName="ceilometer-notification-agent" containerID="cri-o://c28f7df12f8d79aa250110ffaaddfd53cd8cd67650e84fc06d1f0a2c416a20e9" gracePeriod=30 Dec 03 12:47:22 crc kubenswrapper[5052]: I1203 12:47:22.316450 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"efb0e60c-2a71-4641-87b2-0f5532b28678","Type":"ContainerStarted","Data":"fde954c2e50f163494a08fec446c2659dbced2f64031d8d4fb5557a44bc92f99"} Dec 03 12:47:22 crc kubenswrapper[5052]: I1203 12:47:22.316712 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"efb0e60c-2a71-4641-87b2-0f5532b28678","Type":"ContainerStarted","Data":"d42c23e387ad7773d4b53a13fb7267499e89a43ce6fa0b570fa164ddcc0ed712"} Dec 03 12:47:22 crc kubenswrapper[5052]: I1203 12:47:22.319439 5052 generic.go:334] "Generic (PLEG): container finished" podID="1b7ff11c-2d16-40e6-98f5-5e4a9a78fed3" containerID="91120088947d8c381c8640229b02472087109db20e7308332ea507f1841d751b" exitCode=0 Dec 03 12:47:22 crc kubenswrapper[5052]: I1203 12:47:22.319461 5052 generic.go:334] "Generic (PLEG): container finished" podID="1b7ff11c-2d16-40e6-98f5-5e4a9a78fed3" containerID="9a89f3d83f92ed5f8d9b084b364bbf81c57e64d6a73001585da1afa98e2cb7cb" exitCode=2 Dec 03 12:47:22 crc kubenswrapper[5052]: I1203 12:47:22.319467 5052 generic.go:334] "Generic (PLEG): container finished" podID="1b7ff11c-2d16-40e6-98f5-5e4a9a78fed3" containerID="feafdcdf212617a7513bd5d21b3d0f1577e3d60a91b5b9c44988b01a3e8d821f" exitCode=0 Dec 03 12:47:22 crc kubenswrapper[5052]: I1203 12:47:22.319599 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"1b7ff11c-2d16-40e6-98f5-5e4a9a78fed3","Type":"ContainerDied","Data":"91120088947d8c381c8640229b02472087109db20e7308332ea507f1841d751b"} Dec 03 12:47:22 crc kubenswrapper[5052]: I1203 12:47:22.319644 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"1b7ff11c-2d16-40e6-98f5-5e4a9a78fed3","Type":"ContainerDied","Data":"9a89f3d83f92ed5f8d9b084b364bbf81c57e64d6a73001585da1afa98e2cb7cb"} Dec 03 12:47:22 crc kubenswrapper[5052]: I1203 12:47:22.319657 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"1b7ff11c-2d16-40e6-98f5-5e4a9a78fed3","Type":"ContainerDied","Data":"feafdcdf212617a7513bd5d21b3d0f1577e3d60a91b5b9c44988b01a3e8d821f"} Dec 03 12:47:22 crc kubenswrapper[5052]: I1203 12:47:22.341581 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-metadata-0" podStartSLOduration=2.341561832 podStartE2EDuration="2.341561832s" podCreationTimestamp="2025-12-03 12:47:20 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:47:22.33344903 +0000 UTC m=+5232.921690794" watchObservedRunningTime="2025-12-03 12:47:22.341561832 +0000 UTC m=+5232.929803586" Dec 03 12:47:23 crc kubenswrapper[5052]: I1203 12:47:23.075089 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:47:23 crc kubenswrapper[5052]: I1203 12:47:23.103410 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/barbican-api-75bdf7fcbb-mlkgz" podStartSLOduration=4.103392709 podStartE2EDuration="4.103392709s" podCreationTimestamp="2025-12-03 12:47:19 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:47:22.355249073 +0000 UTC m=+5232.943490847" watchObservedRunningTime="2025-12-03 12:47:23.103392709 +0000 UTC m=+5233.691634473" Dec 03 12:47:23 crc kubenswrapper[5052]: I1203 12:47:23.215002 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1b7ff11c-2d16-40e6-98f5-5e4a9a78fed3-combined-ca-bundle\") pod \"1b7ff11c-2d16-40e6-98f5-5e4a9a78fed3\" (UID: \"1b7ff11c-2d16-40e6-98f5-5e4a9a78fed3\") " Dec 03 12:47:23 crc kubenswrapper[5052]: I1203 12:47:23.215085 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/1b7ff11c-2d16-40e6-98f5-5e4a9a78fed3-scripts\") pod \"1b7ff11c-2d16-40e6-98f5-5e4a9a78fed3\" (UID: \"1b7ff11c-2d16-40e6-98f5-5e4a9a78fed3\") " Dec 03 12:47:23 crc kubenswrapper[5052]: I1203 12:47:23.215137 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/1b7ff11c-2d16-40e6-98f5-5e4a9a78fed3-log-httpd\") pod \"1b7ff11c-2d16-40e6-98f5-5e4a9a78fed3\" (UID: \"1b7ff11c-2d16-40e6-98f5-5e4a9a78fed3\") " Dec 03 12:47:23 crc kubenswrapper[5052]: I1203 12:47:23.215174 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/1b7ff11c-2d16-40e6-98f5-5e4a9a78fed3-config-data\") pod \"1b7ff11c-2d16-40e6-98f5-5e4a9a78fed3\" (UID: \"1b7ff11c-2d16-40e6-98f5-5e4a9a78fed3\") " Dec 03 12:47:23 crc kubenswrapper[5052]: I1203 12:47:23.215918 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1b7ff11c-2d16-40e6-98f5-5e4a9a78fed3-log-httpd" (OuterVolumeSpecName: "log-httpd") pod "1b7ff11c-2d16-40e6-98f5-5e4a9a78fed3" (UID: "1b7ff11c-2d16-40e6-98f5-5e4a9a78fed3"). InnerVolumeSpecName "log-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:47:23 crc kubenswrapper[5052]: I1203 12:47:23.216077 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j7g6z\" (UniqueName: \"kubernetes.io/projected/1b7ff11c-2d16-40e6-98f5-5e4a9a78fed3-kube-api-access-j7g6z\") pod \"1b7ff11c-2d16-40e6-98f5-5e4a9a78fed3\" (UID: \"1b7ff11c-2d16-40e6-98f5-5e4a9a78fed3\") " Dec 03 12:47:23 crc kubenswrapper[5052]: I1203 12:47:23.216378 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/1b7ff11c-2d16-40e6-98f5-5e4a9a78fed3-sg-core-conf-yaml\") pod \"1b7ff11c-2d16-40e6-98f5-5e4a9a78fed3\" (UID: \"1b7ff11c-2d16-40e6-98f5-5e4a9a78fed3\") " Dec 03 12:47:23 crc kubenswrapper[5052]: I1203 12:47:23.216476 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/1b7ff11c-2d16-40e6-98f5-5e4a9a78fed3-ceilometer-tls-certs\") pod \"1b7ff11c-2d16-40e6-98f5-5e4a9a78fed3\" (UID: \"1b7ff11c-2d16-40e6-98f5-5e4a9a78fed3\") " Dec 03 12:47:23 crc kubenswrapper[5052]: I1203 12:47:23.216543 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/1b7ff11c-2d16-40e6-98f5-5e4a9a78fed3-run-httpd\") pod \"1b7ff11c-2d16-40e6-98f5-5e4a9a78fed3\" (UID: \"1b7ff11c-2d16-40e6-98f5-5e4a9a78fed3\") " Dec 03 12:47:23 crc kubenswrapper[5052]: I1203 12:47:23.217098 5052 reconciler_common.go:293] "Volume detached for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/1b7ff11c-2d16-40e6-98f5-5e4a9a78fed3-log-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:47:23 crc kubenswrapper[5052]: I1203 12:47:23.217257 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1b7ff11c-2d16-40e6-98f5-5e4a9a78fed3-run-httpd" (OuterVolumeSpecName: "run-httpd") pod "1b7ff11c-2d16-40e6-98f5-5e4a9a78fed3" (UID: "1b7ff11c-2d16-40e6-98f5-5e4a9a78fed3"). InnerVolumeSpecName "run-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:47:23 crc kubenswrapper[5052]: I1203 12:47:23.233738 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1b7ff11c-2d16-40e6-98f5-5e4a9a78fed3-kube-api-access-j7g6z" (OuterVolumeSpecName: "kube-api-access-j7g6z") pod "1b7ff11c-2d16-40e6-98f5-5e4a9a78fed3" (UID: "1b7ff11c-2d16-40e6-98f5-5e4a9a78fed3"). InnerVolumeSpecName "kube-api-access-j7g6z". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:47:23 crc kubenswrapper[5052]: I1203 12:47:23.285691 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1b7ff11c-2d16-40e6-98f5-5e4a9a78fed3-scripts" (OuterVolumeSpecName: "scripts") pod "1b7ff11c-2d16-40e6-98f5-5e4a9a78fed3" (UID: "1b7ff11c-2d16-40e6-98f5-5e4a9a78fed3"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:47:23 crc kubenswrapper[5052]: I1203 12:47:23.291660 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1b7ff11c-2d16-40e6-98f5-5e4a9a78fed3-sg-core-conf-yaml" (OuterVolumeSpecName: "sg-core-conf-yaml") pod "1b7ff11c-2d16-40e6-98f5-5e4a9a78fed3" (UID: "1b7ff11c-2d16-40e6-98f5-5e4a9a78fed3"). InnerVolumeSpecName "sg-core-conf-yaml". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:47:23 crc kubenswrapper[5052]: I1203 12:47:23.318790 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-j7g6z\" (UniqueName: \"kubernetes.io/projected/1b7ff11c-2d16-40e6-98f5-5e4a9a78fed3-kube-api-access-j7g6z\") on node \"crc\" DevicePath \"\"" Dec 03 12:47:23 crc kubenswrapper[5052]: I1203 12:47:23.318825 5052 reconciler_common.go:293] "Volume detached for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/1b7ff11c-2d16-40e6-98f5-5e4a9a78fed3-sg-core-conf-yaml\") on node \"crc\" DevicePath \"\"" Dec 03 12:47:23 crc kubenswrapper[5052]: I1203 12:47:23.318836 5052 reconciler_common.go:293] "Volume detached for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/1b7ff11c-2d16-40e6-98f5-5e4a9a78fed3-run-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:47:23 crc kubenswrapper[5052]: I1203 12:47:23.318846 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/1b7ff11c-2d16-40e6-98f5-5e4a9a78fed3-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:47:23 crc kubenswrapper[5052]: I1203 12:47:23.398235 5052 generic.go:334] "Generic (PLEG): container finished" podID="1b7ff11c-2d16-40e6-98f5-5e4a9a78fed3" containerID="c28f7df12f8d79aa250110ffaaddfd53cd8cd67650e84fc06d1f0a2c416a20e9" exitCode=0 Dec 03 12:47:23 crc kubenswrapper[5052]: I1203 12:47:23.398639 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:47:23 crc kubenswrapper[5052]: I1203 12:47:23.398699 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"1b7ff11c-2d16-40e6-98f5-5e4a9a78fed3","Type":"ContainerDied","Data":"c28f7df12f8d79aa250110ffaaddfd53cd8cd67650e84fc06d1f0a2c416a20e9"} Dec 03 12:47:23 crc kubenswrapper[5052]: I1203 12:47:23.398740 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"1b7ff11c-2d16-40e6-98f5-5e4a9a78fed3","Type":"ContainerDied","Data":"1156d1f8f662a93773bdc584e55d1dba9a5eed86642ad79df3e23367662ed118"} Dec 03 12:47:23 crc kubenswrapper[5052]: I1203 12:47:23.398757 5052 scope.go:117] "RemoveContainer" containerID="91120088947d8c381c8640229b02472087109db20e7308332ea507f1841d751b" Dec 03 12:47:23 crc kubenswrapper[5052]: I1203 12:47:23.679935 5052 scope.go:117] "RemoveContainer" containerID="13a4cd2432b46ba6f3a0ba15e8d71c8570f57e964a36ff41f1150e41215a4cc8" Dec 03 12:47:23 crc kubenswrapper[5052]: E1203 12:47:23.680379 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"nova-cell1-conductor-conductor\" with CrashLoopBackOff: \"back-off 20s restarting failed container=nova-cell1-conductor-conductor pod=nova-cell1-conductor-0_openstack-kuttl-tests(8669832d-c47a-49bd-9491-dc7141ac6281)\"" pod="openstack-kuttl-tests/nova-cell1-conductor-0" podUID="8669832d-c47a-49bd-9491-dc7141ac6281" Dec 03 12:47:23 crc kubenswrapper[5052]: I1203 12:47:23.703496 5052 scope.go:117] "RemoveContainer" containerID="9a89f3d83f92ed5f8d9b084b364bbf81c57e64d6a73001585da1afa98e2cb7cb" Dec 03 12:47:23 crc kubenswrapper[5052]: I1203 12:47:23.768660 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1b7ff11c-2d16-40e6-98f5-5e4a9a78fed3-ceilometer-tls-certs" (OuterVolumeSpecName: "ceilometer-tls-certs") pod "1b7ff11c-2d16-40e6-98f5-5e4a9a78fed3" (UID: "1b7ff11c-2d16-40e6-98f5-5e4a9a78fed3"). InnerVolumeSpecName "ceilometer-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:47:23 crc kubenswrapper[5052]: I1203 12:47:23.779835 5052 scope.go:117] "RemoveContainer" containerID="c28f7df12f8d79aa250110ffaaddfd53cd8cd67650e84fc06d1f0a2c416a20e9" Dec 03 12:47:23 crc kubenswrapper[5052]: I1203 12:47:23.809640 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1b7ff11c-2d16-40e6-98f5-5e4a9a78fed3-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "1b7ff11c-2d16-40e6-98f5-5e4a9a78fed3" (UID: "1b7ff11c-2d16-40e6-98f5-5e4a9a78fed3"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:47:23 crc kubenswrapper[5052]: I1203 12:47:23.836679 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1b7ff11c-2d16-40e6-98f5-5e4a9a78fed3-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:47:23 crc kubenswrapper[5052]: I1203 12:47:23.836710 5052 reconciler_common.go:293] "Volume detached for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/1b7ff11c-2d16-40e6-98f5-5e4a9a78fed3-ceilometer-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:47:23 crc kubenswrapper[5052]: I1203 12:47:23.871649 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1b7ff11c-2d16-40e6-98f5-5e4a9a78fed3-config-data" (OuterVolumeSpecName: "config-data") pod "1b7ff11c-2d16-40e6-98f5-5e4a9a78fed3" (UID: "1b7ff11c-2d16-40e6-98f5-5e4a9a78fed3"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:47:23 crc kubenswrapper[5052]: I1203 12:47:23.938907 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/1b7ff11c-2d16-40e6-98f5-5e4a9a78fed3-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:47:23 crc kubenswrapper[5052]: I1203 12:47:23.961494 5052 scope.go:117] "RemoveContainer" containerID="feafdcdf212617a7513bd5d21b3d0f1577e3d60a91b5b9c44988b01a3e8d821f" Dec 03 12:47:23 crc kubenswrapper[5052]: I1203 12:47:23.989885 5052 scope.go:117] "RemoveContainer" containerID="91120088947d8c381c8640229b02472087109db20e7308332ea507f1841d751b" Dec 03 12:47:23 crc kubenswrapper[5052]: E1203 12:47:23.990298 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"91120088947d8c381c8640229b02472087109db20e7308332ea507f1841d751b\": container with ID starting with 91120088947d8c381c8640229b02472087109db20e7308332ea507f1841d751b not found: ID does not exist" containerID="91120088947d8c381c8640229b02472087109db20e7308332ea507f1841d751b" Dec 03 12:47:23 crc kubenswrapper[5052]: I1203 12:47:23.990340 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"91120088947d8c381c8640229b02472087109db20e7308332ea507f1841d751b"} err="failed to get container status \"91120088947d8c381c8640229b02472087109db20e7308332ea507f1841d751b\": rpc error: code = NotFound desc = could not find container \"91120088947d8c381c8640229b02472087109db20e7308332ea507f1841d751b\": container with ID starting with 91120088947d8c381c8640229b02472087109db20e7308332ea507f1841d751b not found: ID does not exist" Dec 03 12:47:23 crc kubenswrapper[5052]: I1203 12:47:23.990368 5052 scope.go:117] "RemoveContainer" containerID="9a89f3d83f92ed5f8d9b084b364bbf81c57e64d6a73001585da1afa98e2cb7cb" Dec 03 12:47:23 crc kubenswrapper[5052]: E1203 12:47:23.990700 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9a89f3d83f92ed5f8d9b084b364bbf81c57e64d6a73001585da1afa98e2cb7cb\": container with ID starting with 9a89f3d83f92ed5f8d9b084b364bbf81c57e64d6a73001585da1afa98e2cb7cb not found: ID does not exist" containerID="9a89f3d83f92ed5f8d9b084b364bbf81c57e64d6a73001585da1afa98e2cb7cb" Dec 03 12:47:23 crc kubenswrapper[5052]: I1203 12:47:23.990736 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9a89f3d83f92ed5f8d9b084b364bbf81c57e64d6a73001585da1afa98e2cb7cb"} err="failed to get container status \"9a89f3d83f92ed5f8d9b084b364bbf81c57e64d6a73001585da1afa98e2cb7cb\": rpc error: code = NotFound desc = could not find container \"9a89f3d83f92ed5f8d9b084b364bbf81c57e64d6a73001585da1afa98e2cb7cb\": container with ID starting with 9a89f3d83f92ed5f8d9b084b364bbf81c57e64d6a73001585da1afa98e2cb7cb not found: ID does not exist" Dec 03 12:47:23 crc kubenswrapper[5052]: I1203 12:47:23.990755 5052 scope.go:117] "RemoveContainer" containerID="c28f7df12f8d79aa250110ffaaddfd53cd8cd67650e84fc06d1f0a2c416a20e9" Dec 03 12:47:23 crc kubenswrapper[5052]: E1203 12:47:23.990996 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c28f7df12f8d79aa250110ffaaddfd53cd8cd67650e84fc06d1f0a2c416a20e9\": container with ID starting with c28f7df12f8d79aa250110ffaaddfd53cd8cd67650e84fc06d1f0a2c416a20e9 not found: ID does not exist" containerID="c28f7df12f8d79aa250110ffaaddfd53cd8cd67650e84fc06d1f0a2c416a20e9" Dec 03 12:47:23 crc kubenswrapper[5052]: I1203 12:47:23.991026 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c28f7df12f8d79aa250110ffaaddfd53cd8cd67650e84fc06d1f0a2c416a20e9"} err="failed to get container status \"c28f7df12f8d79aa250110ffaaddfd53cd8cd67650e84fc06d1f0a2c416a20e9\": rpc error: code = NotFound desc = could not find container \"c28f7df12f8d79aa250110ffaaddfd53cd8cd67650e84fc06d1f0a2c416a20e9\": container with ID starting with c28f7df12f8d79aa250110ffaaddfd53cd8cd67650e84fc06d1f0a2c416a20e9 not found: ID does not exist" Dec 03 12:47:23 crc kubenswrapper[5052]: I1203 12:47:23.991046 5052 scope.go:117] "RemoveContainer" containerID="feafdcdf212617a7513bd5d21b3d0f1577e3d60a91b5b9c44988b01a3e8d821f" Dec 03 12:47:23 crc kubenswrapper[5052]: E1203 12:47:23.991259 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"feafdcdf212617a7513bd5d21b3d0f1577e3d60a91b5b9c44988b01a3e8d821f\": container with ID starting with feafdcdf212617a7513bd5d21b3d0f1577e3d60a91b5b9c44988b01a3e8d821f not found: ID does not exist" containerID="feafdcdf212617a7513bd5d21b3d0f1577e3d60a91b5b9c44988b01a3e8d821f" Dec 03 12:47:23 crc kubenswrapper[5052]: I1203 12:47:23.991281 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"feafdcdf212617a7513bd5d21b3d0f1577e3d60a91b5b9c44988b01a3e8d821f"} err="failed to get container status \"feafdcdf212617a7513bd5d21b3d0f1577e3d60a91b5b9c44988b01a3e8d821f\": rpc error: code = NotFound desc = could not find container \"feafdcdf212617a7513bd5d21b3d0f1577e3d60a91b5b9c44988b01a3e8d821f\": container with ID starting with feafdcdf212617a7513bd5d21b3d0f1577e3d60a91b5b9c44988b01a3e8d821f not found: ID does not exist" Dec 03 12:47:24 crc kubenswrapper[5052]: I1203 12:47:24.050637 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:47:24 crc kubenswrapper[5052]: I1203 12:47:24.063833 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:47:24 crc kubenswrapper[5052]: I1203 12:47:24.075801 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:47:24 crc kubenswrapper[5052]: E1203 12:47:24.076343 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="1b7ff11c-2d16-40e6-98f5-5e4a9a78fed3" containerName="ceilometer-central-agent" Dec 03 12:47:24 crc kubenswrapper[5052]: I1203 12:47:24.076366 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="1b7ff11c-2d16-40e6-98f5-5e4a9a78fed3" containerName="ceilometer-central-agent" Dec 03 12:47:24 crc kubenswrapper[5052]: E1203 12:47:24.076391 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="1b7ff11c-2d16-40e6-98f5-5e4a9a78fed3" containerName="proxy-httpd" Dec 03 12:47:24 crc kubenswrapper[5052]: I1203 12:47:24.076400 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="1b7ff11c-2d16-40e6-98f5-5e4a9a78fed3" containerName="proxy-httpd" Dec 03 12:47:24 crc kubenswrapper[5052]: E1203 12:47:24.076418 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="1b7ff11c-2d16-40e6-98f5-5e4a9a78fed3" containerName="sg-core" Dec 03 12:47:24 crc kubenswrapper[5052]: I1203 12:47:24.076426 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="1b7ff11c-2d16-40e6-98f5-5e4a9a78fed3" containerName="sg-core" Dec 03 12:47:24 crc kubenswrapper[5052]: E1203 12:47:24.076451 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="1b7ff11c-2d16-40e6-98f5-5e4a9a78fed3" containerName="ceilometer-notification-agent" Dec 03 12:47:24 crc kubenswrapper[5052]: I1203 12:47:24.076459 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="1b7ff11c-2d16-40e6-98f5-5e4a9a78fed3" containerName="ceilometer-notification-agent" Dec 03 12:47:24 crc kubenswrapper[5052]: I1203 12:47:24.076706 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="1b7ff11c-2d16-40e6-98f5-5e4a9a78fed3" containerName="ceilometer-notification-agent" Dec 03 12:47:24 crc kubenswrapper[5052]: I1203 12:47:24.076725 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="1b7ff11c-2d16-40e6-98f5-5e4a9a78fed3" containerName="proxy-httpd" Dec 03 12:47:24 crc kubenswrapper[5052]: I1203 12:47:24.076750 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="1b7ff11c-2d16-40e6-98f5-5e4a9a78fed3" containerName="ceilometer-central-agent" Dec 03 12:47:24 crc kubenswrapper[5052]: I1203 12:47:24.076771 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="1b7ff11c-2d16-40e6-98f5-5e4a9a78fed3" containerName="sg-core" Dec 03 12:47:24 crc kubenswrapper[5052]: I1203 12:47:24.079010 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:47:24 crc kubenswrapper[5052]: I1203 12:47:24.081255 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-scripts" Dec 03 12:47:24 crc kubenswrapper[5052]: I1203 12:47:24.088173 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-ceilometer-internal-svc" Dec 03 12:47:24 crc kubenswrapper[5052]: I1203 12:47:24.092123 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-config-data" Dec 03 12:47:24 crc kubenswrapper[5052]: I1203 12:47:24.098553 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:47:24 crc kubenswrapper[5052]: I1203 12:47:24.246250 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/0068c169-1560-4afb-918b-952444f11921-run-httpd\") pod \"ceilometer-0\" (UID: \"0068c169-1560-4afb-918b-952444f11921\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:47:24 crc kubenswrapper[5052]: I1203 12:47:24.246685 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0068c169-1560-4afb-918b-952444f11921-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"0068c169-1560-4afb-918b-952444f11921\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:47:24 crc kubenswrapper[5052]: I1203 12:47:24.246732 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x72h8\" (UniqueName: \"kubernetes.io/projected/0068c169-1560-4afb-918b-952444f11921-kube-api-access-x72h8\") pod \"ceilometer-0\" (UID: \"0068c169-1560-4afb-918b-952444f11921\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:47:24 crc kubenswrapper[5052]: I1203 12:47:24.246755 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/0068c169-1560-4afb-918b-952444f11921-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"0068c169-1560-4afb-918b-952444f11921\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:47:24 crc kubenswrapper[5052]: I1203 12:47:24.246901 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/0068c169-1560-4afb-918b-952444f11921-scripts\") pod \"ceilometer-0\" (UID: \"0068c169-1560-4afb-918b-952444f11921\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:47:24 crc kubenswrapper[5052]: I1203 12:47:24.246952 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/0068c169-1560-4afb-918b-952444f11921-ceilometer-tls-certs\") pod \"ceilometer-0\" (UID: \"0068c169-1560-4afb-918b-952444f11921\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:47:24 crc kubenswrapper[5052]: I1203 12:47:24.247206 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/0068c169-1560-4afb-918b-952444f11921-log-httpd\") pod \"ceilometer-0\" (UID: \"0068c169-1560-4afb-918b-952444f11921\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:47:24 crc kubenswrapper[5052]: I1203 12:47:24.247231 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/0068c169-1560-4afb-918b-952444f11921-config-data\") pod \"ceilometer-0\" (UID: \"0068c169-1560-4afb-918b-952444f11921\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:47:24 crc kubenswrapper[5052]: I1203 12:47:24.356932 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/0068c169-1560-4afb-918b-952444f11921-run-httpd\") pod \"ceilometer-0\" (UID: \"0068c169-1560-4afb-918b-952444f11921\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:47:24 crc kubenswrapper[5052]: I1203 12:47:24.357008 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0068c169-1560-4afb-918b-952444f11921-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"0068c169-1560-4afb-918b-952444f11921\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:47:24 crc kubenswrapper[5052]: I1203 12:47:24.357055 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-x72h8\" (UniqueName: \"kubernetes.io/projected/0068c169-1560-4afb-918b-952444f11921-kube-api-access-x72h8\") pod \"ceilometer-0\" (UID: \"0068c169-1560-4afb-918b-952444f11921\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:47:24 crc kubenswrapper[5052]: I1203 12:47:24.357075 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/0068c169-1560-4afb-918b-952444f11921-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"0068c169-1560-4afb-918b-952444f11921\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:47:24 crc kubenswrapper[5052]: I1203 12:47:24.357104 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/0068c169-1560-4afb-918b-952444f11921-scripts\") pod \"ceilometer-0\" (UID: \"0068c169-1560-4afb-918b-952444f11921\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:47:24 crc kubenswrapper[5052]: I1203 12:47:24.357119 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/0068c169-1560-4afb-918b-952444f11921-ceilometer-tls-certs\") pod \"ceilometer-0\" (UID: \"0068c169-1560-4afb-918b-952444f11921\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:47:24 crc kubenswrapper[5052]: I1203 12:47:24.357200 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/0068c169-1560-4afb-918b-952444f11921-log-httpd\") pod \"ceilometer-0\" (UID: \"0068c169-1560-4afb-918b-952444f11921\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:47:24 crc kubenswrapper[5052]: I1203 12:47:24.357220 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/0068c169-1560-4afb-918b-952444f11921-config-data\") pod \"ceilometer-0\" (UID: \"0068c169-1560-4afb-918b-952444f11921\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:47:24 crc kubenswrapper[5052]: I1203 12:47:24.358536 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/0068c169-1560-4afb-918b-952444f11921-log-httpd\") pod \"ceilometer-0\" (UID: \"0068c169-1560-4afb-918b-952444f11921\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:47:24 crc kubenswrapper[5052]: I1203 12:47:24.358585 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/0068c169-1560-4afb-918b-952444f11921-run-httpd\") pod \"ceilometer-0\" (UID: \"0068c169-1560-4afb-918b-952444f11921\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:47:24 crc kubenswrapper[5052]: I1203 12:47:24.361860 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/0068c169-1560-4afb-918b-952444f11921-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"0068c169-1560-4afb-918b-952444f11921\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:47:24 crc kubenswrapper[5052]: I1203 12:47:24.362428 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/0068c169-1560-4afb-918b-952444f11921-ceilometer-tls-certs\") pod \"ceilometer-0\" (UID: \"0068c169-1560-4afb-918b-952444f11921\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:47:24 crc kubenswrapper[5052]: I1203 12:47:24.362711 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0068c169-1560-4afb-918b-952444f11921-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"0068c169-1560-4afb-918b-952444f11921\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:47:24 crc kubenswrapper[5052]: I1203 12:47:24.364946 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/0068c169-1560-4afb-918b-952444f11921-config-data\") pod \"ceilometer-0\" (UID: \"0068c169-1560-4afb-918b-952444f11921\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:47:24 crc kubenswrapper[5052]: I1203 12:47:24.366261 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/barbican-api-75bdf7fcbb-mlkgz" Dec 03 12:47:24 crc kubenswrapper[5052]: I1203 12:47:24.366316 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/0068c169-1560-4afb-918b-952444f11921-scripts\") pod \"ceilometer-0\" (UID: \"0068c169-1560-4afb-918b-952444f11921\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:47:24 crc kubenswrapper[5052]: I1203 12:47:24.367172 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/barbican-api-75bdf7fcbb-mlkgz" Dec 03 12:47:24 crc kubenswrapper[5052]: I1203 12:47:24.381183 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-x72h8\" (UniqueName: \"kubernetes.io/projected/0068c169-1560-4afb-918b-952444f11921-kube-api-access-x72h8\") pod \"ceilometer-0\" (UID: \"0068c169-1560-4afb-918b-952444f11921\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:47:24 crc kubenswrapper[5052]: I1203 12:47:24.396181 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:47:25 crc kubenswrapper[5052]: I1203 12:47:25.017945 5052 provider.go:102] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Dec 03 12:47:25 crc kubenswrapper[5052]: I1203 12:47:25.032956 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:47:25 crc kubenswrapper[5052]: I1203 12:47:25.435339 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"0068c169-1560-4afb-918b-952444f11921","Type":"ContainerStarted","Data":"645f90bba79889f7aee955f4c90f199e56090fac12aa7016a046614c3ca497fe"} Dec 03 12:47:25 crc kubenswrapper[5052]: I1203 12:47:25.648428 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:47:25 crc kubenswrapper[5052]: I1203 12:47:25.648740 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:47:25 crc kubenswrapper[5052]: I1203 12:47:25.683352 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1b7ff11c-2d16-40e6-98f5-5e4a9a78fed3" path="/var/lib/kubelet/pods/1b7ff11c-2d16-40e6-98f5-5e4a9a78fed3/volumes" Dec 03 12:47:25 crc kubenswrapper[5052]: I1203 12:47:25.821810 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:47:25 crc kubenswrapper[5052]: I1203 12:47:25.916799 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:47:25 crc kubenswrapper[5052]: I1203 12:47:25.943686 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:47:26 crc kubenswrapper[5052]: I1203 12:47:26.453800 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"0068c169-1560-4afb-918b-952444f11921","Type":"ContainerStarted","Data":"8640584ba053d7aa82893be1329fb1cb047434c76621e6cdd0181fc1cc14fcee"} Dec 03 12:47:26 crc kubenswrapper[5052]: I1203 12:47:26.494368 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:47:27 crc kubenswrapper[5052]: I1203 12:47:27.186819 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/glance-default-external-api-0" podUID="4c9f4da8-f1ae-4815-8ee2-0e873fb07d62" containerName="glance-httpd" probeResult="failure" output="Get \"https://10.217.0.16:9292/healthcheck\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Dec 03 12:47:27 crc kubenswrapper[5052]: I1203 12:47:27.187020 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/glance-default-external-api-0" podUID="4c9f4da8-f1ae-4815-8ee2-0e873fb07d62" containerName="glance-log" probeResult="failure" output="Get \"https://10.217.0.16:9292/healthcheck\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Dec 03 12:47:27 crc kubenswrapper[5052]: I1203 12:47:27.466368 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"0068c169-1560-4afb-918b-952444f11921","Type":"ContainerStarted","Data":"ee710ca8fa0188ffa62dbf27204f55233993aeb05b3f79709c5ef0748fbb7339"} Dec 03 12:47:27 crc kubenswrapper[5052]: I1203 12:47:27.618705 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/placement-74746c7f4-h6sx8" Dec 03 12:47:27 crc kubenswrapper[5052]: I1203 12:47:27.648231 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/placement-74746c7f4-h6sx8" Dec 03 12:47:27 crc kubenswrapper[5052]: I1203 12:47:27.746323 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/placement-56b4b79f66-gfsxn"] Dec 03 12:47:27 crc kubenswrapper[5052]: I1203 12:47:27.747761 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/placement-56b4b79f66-gfsxn" podUID="76534e0c-3b73-4ede-bc24-b12c7f8ea4db" containerName="placement-api" containerID="cri-o://fc4f4321d2ac241b4a797a64c285f190f073d7f5f8f626ec4d2711dd1d73a837" gracePeriod=30 Dec 03 12:47:27 crc kubenswrapper[5052]: I1203 12:47:27.748270 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/placement-56b4b79f66-gfsxn" podUID="76534e0c-3b73-4ede-bc24-b12c7f8ea4db" containerName="placement-log" containerID="cri-o://8b321f78db963a5c8fd8a06d8d437e337dea72a0472cde308cbba093a65a6e60" gracePeriod=30 Dec 03 12:47:28 crc kubenswrapper[5052]: I1203 12:47:28.480623 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"0068c169-1560-4afb-918b-952444f11921","Type":"ContainerStarted","Data":"cc4689adb2d0f67123301989da14c8be70ee0b36ea8974223f0120e1d748cc86"} Dec 03 12:47:28 crc kubenswrapper[5052]: I1203 12:47:28.483791 5052 generic.go:334] "Generic (PLEG): container finished" podID="76534e0c-3b73-4ede-bc24-b12c7f8ea4db" containerID="8b321f78db963a5c8fd8a06d8d437e337dea72a0472cde308cbba093a65a6e60" exitCode=143 Dec 03 12:47:28 crc kubenswrapper[5052]: I1203 12:47:28.484913 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-56b4b79f66-gfsxn" event={"ID":"76534e0c-3b73-4ede-bc24-b12c7f8ea4db","Type":"ContainerDied","Data":"8b321f78db963a5c8fd8a06d8d437e337dea72a0472cde308cbba093a65a6e60"} Dec 03 12:47:28 crc kubenswrapper[5052]: I1203 12:47:28.803649 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:47:30 crc kubenswrapper[5052]: I1203 12:47:30.512261 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"0068c169-1560-4afb-918b-952444f11921","Type":"ContainerStarted","Data":"a258638afb610e911ce997d085715008b91cd8f4631d20d5dbcd5ab4b26cdb22"} Dec 03 12:47:30 crc kubenswrapper[5052]: I1203 12:47:30.512769 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:47:30 crc kubenswrapper[5052]: I1203 12:47:30.545879 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/ceilometer-0" podStartSLOduration=1.948284368 podStartE2EDuration="6.54586162s" podCreationTimestamp="2025-12-03 12:47:24 +0000 UTC" firstStartedPulling="2025-12-03 12:47:25.017748826 +0000 UTC m=+5235.605990590" lastFinishedPulling="2025-12-03 12:47:29.615326078 +0000 UTC m=+5240.203567842" observedRunningTime="2025-12-03 12:47:30.538620663 +0000 UTC m=+5241.126862427" watchObservedRunningTime="2025-12-03 12:47:30.54586162 +0000 UTC m=+5241.134103384" Dec 03 12:47:30 crc kubenswrapper[5052]: I1203 12:47:30.648701 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:47:30 crc kubenswrapper[5052]: I1203 12:47:30.648787 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:47:30 crc kubenswrapper[5052]: I1203 12:47:30.997801 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/barbican-api-75bdf7fcbb-mlkgz" Dec 03 12:47:31 crc kubenswrapper[5052]: I1203 12:47:31.040327 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/barbican-api-75bdf7fcbb-mlkgz" Dec 03 12:47:31 crc kubenswrapper[5052]: I1203 12:47:31.129950 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-api-7764f4c764-tsnp4"] Dec 03 12:47:31 crc kubenswrapper[5052]: I1203 12:47:31.130249 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/barbican-api-7764f4c764-tsnp4" podUID="74a94443-98ef-44b2-ae1b-43644b7c393b" containerName="barbican-api-log" containerID="cri-o://5010ae0ecbbec159c8035c9663c6ff54ee13ce879a982154a989755d6696637c" gracePeriod=30 Dec 03 12:47:31 crc kubenswrapper[5052]: I1203 12:47:31.130297 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/barbican-api-7764f4c764-tsnp4" podUID="74a94443-98ef-44b2-ae1b-43644b7c393b" containerName="barbican-api" containerID="cri-o://6e751423b8df4113d67dc20a6fb9fa3c1e998f1985a9bb1d69a0dc6459218bec" gracePeriod=30 Dec 03 12:47:31 crc kubenswrapper[5052]: I1203 12:47:31.274541 5052 scope.go:117] "RemoveContainer" containerID="7931c8713f97dce1e306c981b449dd889f9caac3a2406f2863f294197a7435e2" Dec 03 12:47:31 crc kubenswrapper[5052]: I1203 12:47:31.300702 5052 scope.go:117] "RemoveContainer" containerID="32c6ff6dd047d38174cc0dc1926116d0446b65b6623a5d45be2ea5d7903f4b0c" Dec 03 12:47:31 crc kubenswrapper[5052]: I1203 12:47:31.559100 5052 generic.go:334] "Generic (PLEG): container finished" podID="76534e0c-3b73-4ede-bc24-b12c7f8ea4db" containerID="fc4f4321d2ac241b4a797a64c285f190f073d7f5f8f626ec4d2711dd1d73a837" exitCode=0 Dec 03 12:47:31 crc kubenswrapper[5052]: I1203 12:47:31.559197 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-56b4b79f66-gfsxn" event={"ID":"76534e0c-3b73-4ede-bc24-b12c7f8ea4db","Type":"ContainerDied","Data":"fc4f4321d2ac241b4a797a64c285f190f073d7f5f8f626ec4d2711dd1d73a837"} Dec 03 12:47:31 crc kubenswrapper[5052]: I1203 12:47:31.559246 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-56b4b79f66-gfsxn" event={"ID":"76534e0c-3b73-4ede-bc24-b12c7f8ea4db","Type":"ContainerDied","Data":"96c237ac04f7109dc8403eb71f6ee5b59681e2a19e6ad3fa6f2004a8a43b803f"} Dec 03 12:47:31 crc kubenswrapper[5052]: I1203 12:47:31.559261 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="96c237ac04f7109dc8403eb71f6ee5b59681e2a19e6ad3fa6f2004a8a43b803f" Dec 03 12:47:31 crc kubenswrapper[5052]: I1203 12:47:31.563069 5052 generic.go:334] "Generic (PLEG): container finished" podID="74a94443-98ef-44b2-ae1b-43644b7c393b" containerID="5010ae0ecbbec159c8035c9663c6ff54ee13ce879a982154a989755d6696637c" exitCode=143 Dec 03 12:47:31 crc kubenswrapper[5052]: I1203 12:47:31.563141 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-7764f4c764-tsnp4" event={"ID":"74a94443-98ef-44b2-ae1b-43644b7c393b","Type":"ContainerDied","Data":"5010ae0ecbbec159c8035c9663c6ff54ee13ce879a982154a989755d6696637c"} Dec 03 12:47:31 crc kubenswrapper[5052]: I1203 12:47:31.661135 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-metadata-0" podUID="efb0e60c-2a71-4641-87b2-0f5532b28678" containerName="nova-metadata-metadata" probeResult="failure" output="Get \"https://10.217.0.119:8775/\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Dec 03 12:47:31 crc kubenswrapper[5052]: I1203 12:47:31.672727 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-metadata-0" podUID="efb0e60c-2a71-4641-87b2-0f5532b28678" containerName="nova-metadata-log" probeResult="failure" output="Get \"https://10.217.0.119:8775/\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Dec 03 12:47:31 crc kubenswrapper[5052]: I1203 12:47:31.802010 5052 scope.go:117] "RemoveContainer" containerID="919c4b64371a0cbf9bb20effc9d35fba8bb6ee8a56b9ff66f203f2d32d0ee97b" Dec 03 12:47:31 crc kubenswrapper[5052]: I1203 12:47:31.840987 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-56b4b79f66-gfsxn" Dec 03 12:47:31 crc kubenswrapper[5052]: I1203 12:47:31.940159 5052 scope.go:117] "RemoveContainer" containerID="c50569849ae16f9aa7c97dde8235fdd02d7e4703bdc46f99d859da6cf6d5b3c2" Dec 03 12:47:31 crc kubenswrapper[5052]: I1203 12:47:31.955498 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/76534e0c-3b73-4ede-bc24-b12c7f8ea4db-public-tls-certs\") pod \"76534e0c-3b73-4ede-bc24-b12c7f8ea4db\" (UID: \"76534e0c-3b73-4ede-bc24-b12c7f8ea4db\") " Dec 03 12:47:31 crc kubenswrapper[5052]: I1203 12:47:31.955630 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lsljg\" (UniqueName: \"kubernetes.io/projected/76534e0c-3b73-4ede-bc24-b12c7f8ea4db-kube-api-access-lsljg\") pod \"76534e0c-3b73-4ede-bc24-b12c7f8ea4db\" (UID: \"76534e0c-3b73-4ede-bc24-b12c7f8ea4db\") " Dec 03 12:47:31 crc kubenswrapper[5052]: I1203 12:47:31.955688 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/76534e0c-3b73-4ede-bc24-b12c7f8ea4db-scripts\") pod \"76534e0c-3b73-4ede-bc24-b12c7f8ea4db\" (UID: \"76534e0c-3b73-4ede-bc24-b12c7f8ea4db\") " Dec 03 12:47:31 crc kubenswrapper[5052]: I1203 12:47:31.955720 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/76534e0c-3b73-4ede-bc24-b12c7f8ea4db-config-data\") pod \"76534e0c-3b73-4ede-bc24-b12c7f8ea4db\" (UID: \"76534e0c-3b73-4ede-bc24-b12c7f8ea4db\") " Dec 03 12:47:31 crc kubenswrapper[5052]: I1203 12:47:31.955796 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/76534e0c-3b73-4ede-bc24-b12c7f8ea4db-internal-tls-certs\") pod \"76534e0c-3b73-4ede-bc24-b12c7f8ea4db\" (UID: \"76534e0c-3b73-4ede-bc24-b12c7f8ea4db\") " Dec 03 12:47:31 crc kubenswrapper[5052]: I1203 12:47:31.955821 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/76534e0c-3b73-4ede-bc24-b12c7f8ea4db-combined-ca-bundle\") pod \"76534e0c-3b73-4ede-bc24-b12c7f8ea4db\" (UID: \"76534e0c-3b73-4ede-bc24-b12c7f8ea4db\") " Dec 03 12:47:31 crc kubenswrapper[5052]: I1203 12:47:31.955857 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/76534e0c-3b73-4ede-bc24-b12c7f8ea4db-logs\") pod \"76534e0c-3b73-4ede-bc24-b12c7f8ea4db\" (UID: \"76534e0c-3b73-4ede-bc24-b12c7f8ea4db\") " Dec 03 12:47:31 crc kubenswrapper[5052]: I1203 12:47:31.956752 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/76534e0c-3b73-4ede-bc24-b12c7f8ea4db-logs" (OuterVolumeSpecName: "logs") pod "76534e0c-3b73-4ede-bc24-b12c7f8ea4db" (UID: "76534e0c-3b73-4ede-bc24-b12c7f8ea4db"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:47:31 crc kubenswrapper[5052]: I1203 12:47:31.972708 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/76534e0c-3b73-4ede-bc24-b12c7f8ea4db-kube-api-access-lsljg" (OuterVolumeSpecName: "kube-api-access-lsljg") pod "76534e0c-3b73-4ede-bc24-b12c7f8ea4db" (UID: "76534e0c-3b73-4ede-bc24-b12c7f8ea4db"). InnerVolumeSpecName "kube-api-access-lsljg". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:47:31 crc kubenswrapper[5052]: I1203 12:47:31.979688 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/76534e0c-3b73-4ede-bc24-b12c7f8ea4db-scripts" (OuterVolumeSpecName: "scripts") pod "76534e0c-3b73-4ede-bc24-b12c7f8ea4db" (UID: "76534e0c-3b73-4ede-bc24-b12c7f8ea4db"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:47:32 crc kubenswrapper[5052]: I1203 12:47:32.060307 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/76534e0c-3b73-4ede-bc24-b12c7f8ea4db-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:47:32 crc kubenswrapper[5052]: I1203 12:47:32.060595 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-lsljg\" (UniqueName: \"kubernetes.io/projected/76534e0c-3b73-4ede-bc24-b12c7f8ea4db-kube-api-access-lsljg\") on node \"crc\" DevicePath \"\"" Dec 03 12:47:32 crc kubenswrapper[5052]: I1203 12:47:32.060697 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/76534e0c-3b73-4ede-bc24-b12c7f8ea4db-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:47:32 crc kubenswrapper[5052]: I1203 12:47:32.079875 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/76534e0c-3b73-4ede-bc24-b12c7f8ea4db-config-data" (OuterVolumeSpecName: "config-data") pod "76534e0c-3b73-4ede-bc24-b12c7f8ea4db" (UID: "76534e0c-3b73-4ede-bc24-b12c7f8ea4db"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:47:32 crc kubenswrapper[5052]: I1203 12:47:32.080686 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/76534e0c-3b73-4ede-bc24-b12c7f8ea4db-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "76534e0c-3b73-4ede-bc24-b12c7f8ea4db" (UID: "76534e0c-3b73-4ede-bc24-b12c7f8ea4db"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:47:32 crc kubenswrapper[5052]: I1203 12:47:32.154157 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/76534e0c-3b73-4ede-bc24-b12c7f8ea4db-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "76534e0c-3b73-4ede-bc24-b12c7f8ea4db" (UID: "76534e0c-3b73-4ede-bc24-b12c7f8ea4db"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:47:32 crc kubenswrapper[5052]: I1203 12:47:32.159626 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/76534e0c-3b73-4ede-bc24-b12c7f8ea4db-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "76534e0c-3b73-4ede-bc24-b12c7f8ea4db" (UID: "76534e0c-3b73-4ede-bc24-b12c7f8ea4db"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:47:32 crc kubenswrapper[5052]: I1203 12:47:32.164652 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/76534e0c-3b73-4ede-bc24-b12c7f8ea4db-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:47:32 crc kubenswrapper[5052]: I1203 12:47:32.164691 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/76534e0c-3b73-4ede-bc24-b12c7f8ea4db-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:47:32 crc kubenswrapper[5052]: I1203 12:47:32.164707 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/76534e0c-3b73-4ede-bc24-b12c7f8ea4db-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:47:32 crc kubenswrapper[5052]: I1203 12:47:32.164721 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/76534e0c-3b73-4ede-bc24-b12c7f8ea4db-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:47:32 crc kubenswrapper[5052]: I1203 12:47:32.577735 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-56b4b79f66-gfsxn" Dec 03 12:47:32 crc kubenswrapper[5052]: I1203 12:47:32.614375 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/placement-56b4b79f66-gfsxn"] Dec 03 12:47:32 crc kubenswrapper[5052]: I1203 12:47:32.626134 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/placement-56b4b79f66-gfsxn"] Dec 03 12:47:32 crc kubenswrapper[5052]: I1203 12:47:32.673319 5052 scope.go:117] "RemoveContainer" containerID="2aed1c760a326f3d008799e902b1093313df3764a57c879f0b3686410127d37d" Dec 03 12:47:32 crc kubenswrapper[5052]: E1203 12:47:32.673936 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 12:47:33 crc kubenswrapper[5052]: I1203 12:47:33.673672 5052 scope.go:117] "RemoveContainer" containerID="d1e48098c512a28625a35e3a019f961f22ad8e764f883b4f8931a03d713b8c01" Dec 03 12:47:33 crc kubenswrapper[5052]: I1203 12:47:33.704683 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="76534e0c-3b73-4ede-bc24-b12c7f8ea4db" path="/var/lib/kubelet/pods/76534e0c-3b73-4ede-bc24-b12c7f8ea4db/volumes" Dec 03 12:47:34 crc kubenswrapper[5052]: I1203 12:47:34.301979 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/barbican-api-7764f4c764-tsnp4" podUID="74a94443-98ef-44b2-ae1b-43644b7c393b" containerName="barbican-api" probeResult="failure" output="Get \"https://10.217.0.108:9311/healthcheck\": read tcp 10.217.0.2:58098->10.217.0.108:9311: read: connection reset by peer" Dec 03 12:47:34 crc kubenswrapper[5052]: I1203 12:47:34.302138 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/barbican-api-7764f4c764-tsnp4" podUID="74a94443-98ef-44b2-ae1b-43644b7c393b" containerName="barbican-api-log" probeResult="failure" output="Get \"https://10.217.0.108:9311/healthcheck\": read tcp 10.217.0.2:58102->10.217.0.108:9311: read: connection reset by peer" Dec 03 12:47:34 crc kubenswrapper[5052]: I1203 12:47:34.601434 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"81709983-1fd0-4446-bbbf-fb94653bb3d2","Type":"ContainerStarted","Data":"b1f27aab2d1cce51772c0ce99bd8dfc4afd0620b38b776e6b968d9114105f165"} Dec 03 12:47:34 crc kubenswrapper[5052]: I1203 12:47:34.604406 5052 generic.go:334] "Generic (PLEG): container finished" podID="74a94443-98ef-44b2-ae1b-43644b7c393b" containerID="6e751423b8df4113d67dc20a6fb9fa3c1e998f1985a9bb1d69a0dc6459218bec" exitCode=0 Dec 03 12:47:34 crc kubenswrapper[5052]: I1203 12:47:34.604444 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-7764f4c764-tsnp4" event={"ID":"74a94443-98ef-44b2-ae1b-43644b7c393b","Type":"ContainerDied","Data":"6e751423b8df4113d67dc20a6fb9fa3c1e998f1985a9bb1d69a0dc6459218bec"} Dec 03 12:47:34 crc kubenswrapper[5052]: I1203 12:47:34.673385 5052 scope.go:117] "RemoveContainer" containerID="1fdbade92a13691b593283a5b263382dfec190075f1201c77fff78a7be754487" Dec 03 12:47:34 crc kubenswrapper[5052]: E1203 12:47:34.673679 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"nova-cell0-conductor-conductor\" with CrashLoopBackOff: \"back-off 40s restarting failed container=nova-cell0-conductor-conductor pod=nova-cell0-conductor-0_openstack-kuttl-tests(6d653682-52e2-4940-af66-633475bfe09a)\"" pod="openstack-kuttl-tests/nova-cell0-conductor-0" podUID="6d653682-52e2-4940-af66-633475bfe09a" Dec 03 12:47:34 crc kubenswrapper[5052]: I1203 12:47:34.718462 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-api-7764f4c764-tsnp4" Dec 03 12:47:34 crc kubenswrapper[5052]: I1203 12:47:34.821556 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/74a94443-98ef-44b2-ae1b-43644b7c393b-config-data-custom\") pod \"74a94443-98ef-44b2-ae1b-43644b7c393b\" (UID: \"74a94443-98ef-44b2-ae1b-43644b7c393b\") " Dec 03 12:47:34 crc kubenswrapper[5052]: I1203 12:47:34.821629 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/74a94443-98ef-44b2-ae1b-43644b7c393b-internal-tls-certs\") pod \"74a94443-98ef-44b2-ae1b-43644b7c393b\" (UID: \"74a94443-98ef-44b2-ae1b-43644b7c393b\") " Dec 03 12:47:34 crc kubenswrapper[5052]: I1203 12:47:34.821669 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/74a94443-98ef-44b2-ae1b-43644b7c393b-logs\") pod \"74a94443-98ef-44b2-ae1b-43644b7c393b\" (UID: \"74a94443-98ef-44b2-ae1b-43644b7c393b\") " Dec 03 12:47:34 crc kubenswrapper[5052]: I1203 12:47:34.821745 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/74a94443-98ef-44b2-ae1b-43644b7c393b-public-tls-certs\") pod \"74a94443-98ef-44b2-ae1b-43644b7c393b\" (UID: \"74a94443-98ef-44b2-ae1b-43644b7c393b\") " Dec 03 12:47:34 crc kubenswrapper[5052]: I1203 12:47:34.821800 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gbsb2\" (UniqueName: \"kubernetes.io/projected/74a94443-98ef-44b2-ae1b-43644b7c393b-kube-api-access-gbsb2\") pod \"74a94443-98ef-44b2-ae1b-43644b7c393b\" (UID: \"74a94443-98ef-44b2-ae1b-43644b7c393b\") " Dec 03 12:47:34 crc kubenswrapper[5052]: I1203 12:47:34.821892 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/74a94443-98ef-44b2-ae1b-43644b7c393b-combined-ca-bundle\") pod \"74a94443-98ef-44b2-ae1b-43644b7c393b\" (UID: \"74a94443-98ef-44b2-ae1b-43644b7c393b\") " Dec 03 12:47:34 crc kubenswrapper[5052]: I1203 12:47:34.821909 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/74a94443-98ef-44b2-ae1b-43644b7c393b-config-data\") pod \"74a94443-98ef-44b2-ae1b-43644b7c393b\" (UID: \"74a94443-98ef-44b2-ae1b-43644b7c393b\") " Dec 03 12:47:34 crc kubenswrapper[5052]: I1203 12:47:34.824607 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/74a94443-98ef-44b2-ae1b-43644b7c393b-logs" (OuterVolumeSpecName: "logs") pod "74a94443-98ef-44b2-ae1b-43644b7c393b" (UID: "74a94443-98ef-44b2-ae1b-43644b7c393b"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:47:34 crc kubenswrapper[5052]: I1203 12:47:34.835814 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/74a94443-98ef-44b2-ae1b-43644b7c393b-config-data-custom" (OuterVolumeSpecName: "config-data-custom") pod "74a94443-98ef-44b2-ae1b-43644b7c393b" (UID: "74a94443-98ef-44b2-ae1b-43644b7c393b"). InnerVolumeSpecName "config-data-custom". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:47:34 crc kubenswrapper[5052]: I1203 12:47:34.835944 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/74a94443-98ef-44b2-ae1b-43644b7c393b-kube-api-access-gbsb2" (OuterVolumeSpecName: "kube-api-access-gbsb2") pod "74a94443-98ef-44b2-ae1b-43644b7c393b" (UID: "74a94443-98ef-44b2-ae1b-43644b7c393b"). InnerVolumeSpecName "kube-api-access-gbsb2". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:47:34 crc kubenswrapper[5052]: I1203 12:47:34.877216 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/74a94443-98ef-44b2-ae1b-43644b7c393b-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "74a94443-98ef-44b2-ae1b-43644b7c393b" (UID: "74a94443-98ef-44b2-ae1b-43644b7c393b"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:47:34 crc kubenswrapper[5052]: I1203 12:47:34.887626 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/74a94443-98ef-44b2-ae1b-43644b7c393b-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "74a94443-98ef-44b2-ae1b-43644b7c393b" (UID: "74a94443-98ef-44b2-ae1b-43644b7c393b"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:47:34 crc kubenswrapper[5052]: I1203 12:47:34.896871 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/74a94443-98ef-44b2-ae1b-43644b7c393b-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "74a94443-98ef-44b2-ae1b-43644b7c393b" (UID: "74a94443-98ef-44b2-ae1b-43644b7c393b"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:47:34 crc kubenswrapper[5052]: I1203 12:47:34.911471 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/74a94443-98ef-44b2-ae1b-43644b7c393b-config-data" (OuterVolumeSpecName: "config-data") pod "74a94443-98ef-44b2-ae1b-43644b7c393b" (UID: "74a94443-98ef-44b2-ae1b-43644b7c393b"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:47:34 crc kubenswrapper[5052]: I1203 12:47:34.924742 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/74a94443-98ef-44b2-ae1b-43644b7c393b-config-data-custom\") on node \"crc\" DevicePath \"\"" Dec 03 12:47:34 crc kubenswrapper[5052]: I1203 12:47:34.924785 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/74a94443-98ef-44b2-ae1b-43644b7c393b-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:47:34 crc kubenswrapper[5052]: I1203 12:47:34.924800 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/74a94443-98ef-44b2-ae1b-43644b7c393b-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:47:34 crc kubenswrapper[5052]: I1203 12:47:34.924817 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/74a94443-98ef-44b2-ae1b-43644b7c393b-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:47:34 crc kubenswrapper[5052]: I1203 12:47:34.924832 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-gbsb2\" (UniqueName: \"kubernetes.io/projected/74a94443-98ef-44b2-ae1b-43644b7c393b-kube-api-access-gbsb2\") on node \"crc\" DevicePath \"\"" Dec 03 12:47:34 crc kubenswrapper[5052]: I1203 12:47:34.924845 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/74a94443-98ef-44b2-ae1b-43644b7c393b-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:47:34 crc kubenswrapper[5052]: I1203 12:47:34.924857 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/74a94443-98ef-44b2-ae1b-43644b7c393b-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:47:35 crc kubenswrapper[5052]: I1203 12:47:35.615604 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-7764f4c764-tsnp4" event={"ID":"74a94443-98ef-44b2-ae1b-43644b7c393b","Type":"ContainerDied","Data":"1d085744642a41502dc368e41a82a9d82c36ae3bc40b6401638cfa3cdcdca60e"} Dec 03 12:47:35 crc kubenswrapper[5052]: I1203 12:47:35.615668 5052 scope.go:117] "RemoveContainer" containerID="6e751423b8df4113d67dc20a6fb9fa3c1e998f1985a9bb1d69a0dc6459218bec" Dec 03 12:47:35 crc kubenswrapper[5052]: I1203 12:47:35.615823 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-api-7764f4c764-tsnp4" Dec 03 12:47:35 crc kubenswrapper[5052]: I1203 12:47:35.648052 5052 scope.go:117] "RemoveContainer" containerID="5010ae0ecbbec159c8035c9663c6ff54ee13ce879a982154a989755d6696637c" Dec 03 12:47:35 crc kubenswrapper[5052]: I1203 12:47:35.656841 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-api-7764f4c764-tsnp4"] Dec 03 12:47:35 crc kubenswrapper[5052]: I1203 12:47:35.672880 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/barbican-api-7764f4c764-tsnp4"] Dec 03 12:47:35 crc kubenswrapper[5052]: I1203 12:47:35.684634 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="74a94443-98ef-44b2-ae1b-43644b7c393b" path="/var/lib/kubelet/pods/74a94443-98ef-44b2-ae1b-43644b7c393b/volumes" Dec 03 12:47:36 crc kubenswrapper[5052]: I1203 12:47:36.672765 5052 scope.go:117] "RemoveContainer" containerID="13a4cd2432b46ba6f3a0ba15e8d71c8570f57e964a36ff41f1150e41215a4cc8" Dec 03 12:47:37 crc kubenswrapper[5052]: I1203 12:47:37.639959 5052 generic.go:334] "Generic (PLEG): container finished" podID="81709983-1fd0-4446-bbbf-fb94653bb3d2" containerID="b1f27aab2d1cce51772c0ce99bd8dfc4afd0620b38b776e6b968d9114105f165" exitCode=1 Dec 03 12:47:37 crc kubenswrapper[5052]: I1203 12:47:37.640025 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"81709983-1fd0-4446-bbbf-fb94653bb3d2","Type":"ContainerDied","Data":"b1f27aab2d1cce51772c0ce99bd8dfc4afd0620b38b776e6b968d9114105f165"} Dec 03 12:47:37 crc kubenswrapper[5052]: I1203 12:47:37.640678 5052 scope.go:117] "RemoveContainer" containerID="d1e48098c512a28625a35e3a019f961f22ad8e764f883b4f8931a03d713b8c01" Dec 03 12:47:37 crc kubenswrapper[5052]: I1203 12:47:37.641486 5052 scope.go:117] "RemoveContainer" containerID="b1f27aab2d1cce51772c0ce99bd8dfc4afd0620b38b776e6b968d9114105f165" Dec 03 12:47:37 crc kubenswrapper[5052]: E1203 12:47:37.642961 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"nova-scheduler-scheduler\" with CrashLoopBackOff: \"back-off 40s restarting failed container=nova-scheduler-scheduler pod=nova-scheduler-0_openstack-kuttl-tests(81709983-1fd0-4446-bbbf-fb94653bb3d2)\"" pod="openstack-kuttl-tests/nova-scheduler-0" podUID="81709983-1fd0-4446-bbbf-fb94653bb3d2" Dec 03 12:47:37 crc kubenswrapper[5052]: I1203 12:47:37.644723 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-conductor-0" event={"ID":"8669832d-c47a-49bd-9491-dc7141ac6281","Type":"ContainerStarted","Data":"8e3764a05c784a75a82ccff0fcb9f4aaac35f5d6845cca81b14deee31318466f"} Dec 03 12:47:37 crc kubenswrapper[5052]: I1203 12:47:37.645465 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:47:38 crc kubenswrapper[5052]: I1203 12:47:38.527037 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:47:38 crc kubenswrapper[5052]: I1203 12:47:38.527079 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:47:38 crc kubenswrapper[5052]: I1203 12:47:38.527088 5052 kubelet.go:2542] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:47:38 crc kubenswrapper[5052]: I1203 12:47:38.527099 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:47:38 crc kubenswrapper[5052]: I1203 12:47:38.660628 5052 scope.go:117] "RemoveContainer" containerID="b1f27aab2d1cce51772c0ce99bd8dfc4afd0620b38b776e6b968d9114105f165" Dec 03 12:47:38 crc kubenswrapper[5052]: E1203 12:47:38.660846 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"nova-scheduler-scheduler\" with CrashLoopBackOff: \"back-off 40s restarting failed container=nova-scheduler-scheduler pod=nova-scheduler-0_openstack-kuttl-tests(81709983-1fd0-4446-bbbf-fb94653bb3d2)\"" pod="openstack-kuttl-tests/nova-scheduler-0" podUID="81709983-1fd0-4446-bbbf-fb94653bb3d2" Dec 03 12:47:39 crc kubenswrapper[5052]: I1203 12:47:39.709711 5052 generic.go:334] "Generic (PLEG): container finished" podID="8669832d-c47a-49bd-9491-dc7141ac6281" containerID="8e3764a05c784a75a82ccff0fcb9f4aaac35f5d6845cca81b14deee31318466f" exitCode=1 Dec 03 12:47:39 crc kubenswrapper[5052]: I1203 12:47:39.709829 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-conductor-0" event={"ID":"8669832d-c47a-49bd-9491-dc7141ac6281","Type":"ContainerDied","Data":"8e3764a05c784a75a82ccff0fcb9f4aaac35f5d6845cca81b14deee31318466f"} Dec 03 12:47:39 crc kubenswrapper[5052]: I1203 12:47:39.711151 5052 scope.go:117] "RemoveContainer" containerID="13a4cd2432b46ba6f3a0ba15e8d71c8570f57e964a36ff41f1150e41215a4cc8" Dec 03 12:47:39 crc kubenswrapper[5052]: I1203 12:47:39.711266 5052 scope.go:117] "RemoveContainer" containerID="8e3764a05c784a75a82ccff0fcb9f4aaac35f5d6845cca81b14deee31318466f" Dec 03 12:47:39 crc kubenswrapper[5052]: E1203 12:47:39.711646 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"nova-cell1-conductor-conductor\" with CrashLoopBackOff: \"back-off 40s restarting failed container=nova-cell1-conductor-conductor pod=nova-cell1-conductor-0_openstack-kuttl-tests(8669832d-c47a-49bd-9491-dc7141ac6281)\"" pod="openstack-kuttl-tests/nova-cell1-conductor-0" podUID="8669832d-c47a-49bd-9491-dc7141ac6281" Dec 03 12:47:40 crc kubenswrapper[5052]: I1203 12:47:40.658013 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:47:40 crc kubenswrapper[5052]: I1203 12:47:40.659150 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:47:40 crc kubenswrapper[5052]: I1203 12:47:40.665612 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:47:40 crc kubenswrapper[5052]: I1203 12:47:40.741494 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:47:40 crc kubenswrapper[5052]: I1203 12:47:40.872060 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/neutron-8b544ff76-6hjc7" podUID="718fd921-0756-4a3e-b383-ed0d0ddf5b72" containerName="neutron-httpd" probeResult="failure" output="Get \"https://10.217.0.112:9696/\": dial tcp 10.217.0.112:9696: connect: connection refused" Dec 03 12:47:41 crc kubenswrapper[5052]: I1203 12:47:41.284874 5052 kubelet.go:2542] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:47:41 crc kubenswrapper[5052]: I1203 12:47:41.285557 5052 scope.go:117] "RemoveContainer" containerID="8e3764a05c784a75a82ccff0fcb9f4aaac35f5d6845cca81b14deee31318466f" Dec 03 12:47:41 crc kubenswrapper[5052]: E1203 12:47:41.285839 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"nova-cell1-conductor-conductor\" with CrashLoopBackOff: \"back-off 40s restarting failed container=nova-cell1-conductor-conductor pod=nova-cell1-conductor-0_openstack-kuttl-tests(8669832d-c47a-49bd-9491-dc7141ac6281)\"" pod="openstack-kuttl-tests/nova-cell1-conductor-0" podUID="8669832d-c47a-49bd-9491-dc7141ac6281" Dec 03 12:47:42 crc kubenswrapper[5052]: I1203 12:47:42.449340 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/keystone-5f884bdbfd-2xzz4" Dec 03 12:47:42 crc kubenswrapper[5052]: I1203 12:47:42.525926 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/keystone-7f7d6d4c6c-sjnw5"] Dec 03 12:47:42 crc kubenswrapper[5052]: I1203 12:47:42.526226 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/keystone-7f7d6d4c6c-sjnw5" podUID="69bdf82f-4616-437b-a9aa-4a04978a3b63" containerName="keystone-api" containerID="cri-o://86dfffcda0458cbe1fcc5f5c8662311799b8b237901bad0a12f858211ae70295" gracePeriod=30 Dec 03 12:47:42 crc kubenswrapper[5052]: I1203 12:47:42.730058 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/neutron-fdd8f875d-xvq79" Dec 03 12:47:42 crc kubenswrapper[5052]: I1203 12:47:42.807628 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/neutron-68f85bdc64-dc642"] Dec 03 12:47:42 crc kubenswrapper[5052]: I1203 12:47:42.807910 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/neutron-68f85bdc64-dc642" podUID="95997b63-313e-4f5f-adf6-b8ae63de9fb9" containerName="neutron-api" containerID="cri-o://65e1fdf0d5abf9142c0d5dd8fd15f412c134a90fb4aecb28999c1caba228326b" gracePeriod=30 Dec 03 12:47:42 crc kubenswrapper[5052]: I1203 12:47:42.808042 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/neutron-68f85bdc64-dc642" podUID="95997b63-313e-4f5f-adf6-b8ae63de9fb9" containerName="neutron-httpd" containerID="cri-o://93ea570e117e1af074faa4804b0c9dd26bf11892c6a87661f85fc221a9464d09" gracePeriod=30 Dec 03 12:47:43 crc kubenswrapper[5052]: I1203 12:47:43.768507 5052 generic.go:334] "Generic (PLEG): container finished" podID="95997b63-313e-4f5f-adf6-b8ae63de9fb9" containerID="93ea570e117e1af074faa4804b0c9dd26bf11892c6a87661f85fc221a9464d09" exitCode=0 Dec 03 12:47:43 crc kubenswrapper[5052]: I1203 12:47:43.768559 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-68f85bdc64-dc642" event={"ID":"95997b63-313e-4f5f-adf6-b8ae63de9fb9","Type":"ContainerDied","Data":"93ea570e117e1af074faa4804b0c9dd26bf11892c6a87661f85fc221a9464d09"} Dec 03 12:47:44 crc kubenswrapper[5052]: I1203 12:47:44.754983 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_neutron-8b544ff76-6hjc7_718fd921-0756-4a3e-b383-ed0d0ddf5b72/neutron-api/0.log" Dec 03 12:47:44 crc kubenswrapper[5052]: I1203 12:47:44.755558 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-8b544ff76-6hjc7" Dec 03 12:47:44 crc kubenswrapper[5052]: I1203 12:47:44.780794 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_neutron-8b544ff76-6hjc7_718fd921-0756-4a3e-b383-ed0d0ddf5b72/neutron-api/0.log" Dec 03 12:47:44 crc kubenswrapper[5052]: I1203 12:47:44.780853 5052 generic.go:334] "Generic (PLEG): container finished" podID="718fd921-0756-4a3e-b383-ed0d0ddf5b72" containerID="8bd50216472e00f32d9e5e953f59c3394da6a32b9f692dcbcfb78a4e404beee3" exitCode=137 Dec 03 12:47:44 crc kubenswrapper[5052]: I1203 12:47:44.780881 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-8b544ff76-6hjc7" event={"ID":"718fd921-0756-4a3e-b383-ed0d0ddf5b72","Type":"ContainerDied","Data":"8bd50216472e00f32d9e5e953f59c3394da6a32b9f692dcbcfb78a4e404beee3"} Dec 03 12:47:44 crc kubenswrapper[5052]: I1203 12:47:44.780906 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-8b544ff76-6hjc7" event={"ID":"718fd921-0756-4a3e-b383-ed0d0ddf5b72","Type":"ContainerDied","Data":"f4563a004dfcdb6bfd12fed5548968c63fc1f68175e452a2c9649613e0d8f5c6"} Dec 03 12:47:44 crc kubenswrapper[5052]: I1203 12:47:44.780937 5052 scope.go:117] "RemoveContainer" containerID="454384af2bc2744e1c70e4a67dc88563309e7147062c1b5e0c424094b4fea86d" Dec 03 12:47:44 crc kubenswrapper[5052]: I1203 12:47:44.781090 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-8b544ff76-6hjc7" Dec 03 12:47:44 crc kubenswrapper[5052]: I1203 12:47:44.820681 5052 scope.go:117] "RemoveContainer" containerID="8bd50216472e00f32d9e5e953f59c3394da6a32b9f692dcbcfb78a4e404beee3" Dec 03 12:47:44 crc kubenswrapper[5052]: I1203 12:47:44.852707 5052 scope.go:117] "RemoveContainer" containerID="454384af2bc2744e1c70e4a67dc88563309e7147062c1b5e0c424094b4fea86d" Dec 03 12:47:44 crc kubenswrapper[5052]: E1203 12:47:44.853698 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"454384af2bc2744e1c70e4a67dc88563309e7147062c1b5e0c424094b4fea86d\": container with ID starting with 454384af2bc2744e1c70e4a67dc88563309e7147062c1b5e0c424094b4fea86d not found: ID does not exist" containerID="454384af2bc2744e1c70e4a67dc88563309e7147062c1b5e0c424094b4fea86d" Dec 03 12:47:44 crc kubenswrapper[5052]: I1203 12:47:44.853776 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"454384af2bc2744e1c70e4a67dc88563309e7147062c1b5e0c424094b4fea86d"} err="failed to get container status \"454384af2bc2744e1c70e4a67dc88563309e7147062c1b5e0c424094b4fea86d\": rpc error: code = NotFound desc = could not find container \"454384af2bc2744e1c70e4a67dc88563309e7147062c1b5e0c424094b4fea86d\": container with ID starting with 454384af2bc2744e1c70e4a67dc88563309e7147062c1b5e0c424094b4fea86d not found: ID does not exist" Dec 03 12:47:44 crc kubenswrapper[5052]: I1203 12:47:44.853809 5052 scope.go:117] "RemoveContainer" containerID="8bd50216472e00f32d9e5e953f59c3394da6a32b9f692dcbcfb78a4e404beee3" Dec 03 12:47:44 crc kubenswrapper[5052]: E1203 12:47:44.860108 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8bd50216472e00f32d9e5e953f59c3394da6a32b9f692dcbcfb78a4e404beee3\": container with ID starting with 8bd50216472e00f32d9e5e953f59c3394da6a32b9f692dcbcfb78a4e404beee3 not found: ID does not exist" containerID="8bd50216472e00f32d9e5e953f59c3394da6a32b9f692dcbcfb78a4e404beee3" Dec 03 12:47:44 crc kubenswrapper[5052]: I1203 12:47:44.860154 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8bd50216472e00f32d9e5e953f59c3394da6a32b9f692dcbcfb78a4e404beee3"} err="failed to get container status \"8bd50216472e00f32d9e5e953f59c3394da6a32b9f692dcbcfb78a4e404beee3\": rpc error: code = NotFound desc = could not find container \"8bd50216472e00f32d9e5e953f59c3394da6a32b9f692dcbcfb78a4e404beee3\": container with ID starting with 8bd50216472e00f32d9e5e953f59c3394da6a32b9f692dcbcfb78a4e404beee3 not found: ID does not exist" Dec 03 12:47:44 crc kubenswrapper[5052]: I1203 12:47:44.865383 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/718fd921-0756-4a3e-b383-ed0d0ddf5b72-httpd-config\") pod \"718fd921-0756-4a3e-b383-ed0d0ddf5b72\" (UID: \"718fd921-0756-4a3e-b383-ed0d0ddf5b72\") " Dec 03 12:47:44 crc kubenswrapper[5052]: I1203 12:47:44.865466 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/718fd921-0756-4a3e-b383-ed0d0ddf5b72-internal-tls-certs\") pod \"718fd921-0756-4a3e-b383-ed0d0ddf5b72\" (UID: \"718fd921-0756-4a3e-b383-ed0d0ddf5b72\") " Dec 03 12:47:44 crc kubenswrapper[5052]: I1203 12:47:44.865594 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/718fd921-0756-4a3e-b383-ed0d0ddf5b72-combined-ca-bundle\") pod \"718fd921-0756-4a3e-b383-ed0d0ddf5b72\" (UID: \"718fd921-0756-4a3e-b383-ed0d0ddf5b72\") " Dec 03 12:47:44 crc kubenswrapper[5052]: I1203 12:47:44.865667 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/718fd921-0756-4a3e-b383-ed0d0ddf5b72-ovndb-tls-certs\") pod \"718fd921-0756-4a3e-b383-ed0d0ddf5b72\" (UID: \"718fd921-0756-4a3e-b383-ed0d0ddf5b72\") " Dec 03 12:47:44 crc kubenswrapper[5052]: I1203 12:47:44.865709 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tp9bv\" (UniqueName: \"kubernetes.io/projected/718fd921-0756-4a3e-b383-ed0d0ddf5b72-kube-api-access-tp9bv\") pod \"718fd921-0756-4a3e-b383-ed0d0ddf5b72\" (UID: \"718fd921-0756-4a3e-b383-ed0d0ddf5b72\") " Dec 03 12:47:44 crc kubenswrapper[5052]: I1203 12:47:44.865809 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/718fd921-0756-4a3e-b383-ed0d0ddf5b72-public-tls-certs\") pod \"718fd921-0756-4a3e-b383-ed0d0ddf5b72\" (UID: \"718fd921-0756-4a3e-b383-ed0d0ddf5b72\") " Dec 03 12:47:44 crc kubenswrapper[5052]: I1203 12:47:44.865862 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/718fd921-0756-4a3e-b383-ed0d0ddf5b72-config\") pod \"718fd921-0756-4a3e-b383-ed0d0ddf5b72\" (UID: \"718fd921-0756-4a3e-b383-ed0d0ddf5b72\") " Dec 03 12:47:44 crc kubenswrapper[5052]: I1203 12:47:44.872753 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/718fd921-0756-4a3e-b383-ed0d0ddf5b72-kube-api-access-tp9bv" (OuterVolumeSpecName: "kube-api-access-tp9bv") pod "718fd921-0756-4a3e-b383-ed0d0ddf5b72" (UID: "718fd921-0756-4a3e-b383-ed0d0ddf5b72"). InnerVolumeSpecName "kube-api-access-tp9bv". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:47:44 crc kubenswrapper[5052]: I1203 12:47:44.872995 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/718fd921-0756-4a3e-b383-ed0d0ddf5b72-httpd-config" (OuterVolumeSpecName: "httpd-config") pod "718fd921-0756-4a3e-b383-ed0d0ddf5b72" (UID: "718fd921-0756-4a3e-b383-ed0d0ddf5b72"). InnerVolumeSpecName "httpd-config". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:47:44 crc kubenswrapper[5052]: I1203 12:47:44.936031 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/718fd921-0756-4a3e-b383-ed0d0ddf5b72-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "718fd921-0756-4a3e-b383-ed0d0ddf5b72" (UID: "718fd921-0756-4a3e-b383-ed0d0ddf5b72"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:47:44 crc kubenswrapper[5052]: I1203 12:47:44.946178 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/718fd921-0756-4a3e-b383-ed0d0ddf5b72-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "718fd921-0756-4a3e-b383-ed0d0ddf5b72" (UID: "718fd921-0756-4a3e-b383-ed0d0ddf5b72"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:47:44 crc kubenswrapper[5052]: I1203 12:47:44.956928 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/718fd921-0756-4a3e-b383-ed0d0ddf5b72-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "718fd921-0756-4a3e-b383-ed0d0ddf5b72" (UID: "718fd921-0756-4a3e-b383-ed0d0ddf5b72"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:47:44 crc kubenswrapper[5052]: I1203 12:47:44.962123 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/718fd921-0756-4a3e-b383-ed0d0ddf5b72-ovndb-tls-certs" (OuterVolumeSpecName: "ovndb-tls-certs") pod "718fd921-0756-4a3e-b383-ed0d0ddf5b72" (UID: "718fd921-0756-4a3e-b383-ed0d0ddf5b72"). InnerVolumeSpecName "ovndb-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:47:44 crc kubenswrapper[5052]: I1203 12:47:44.967731 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/718fd921-0756-4a3e-b383-ed0d0ddf5b72-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:47:44 crc kubenswrapper[5052]: I1203 12:47:44.967761 5052 reconciler_common.go:293] "Volume detached for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/718fd921-0756-4a3e-b383-ed0d0ddf5b72-ovndb-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:47:44 crc kubenswrapper[5052]: I1203 12:47:44.967773 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-tp9bv\" (UniqueName: \"kubernetes.io/projected/718fd921-0756-4a3e-b383-ed0d0ddf5b72-kube-api-access-tp9bv\") on node \"crc\" DevicePath \"\"" Dec 03 12:47:44 crc kubenswrapper[5052]: I1203 12:47:44.967784 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/718fd921-0756-4a3e-b383-ed0d0ddf5b72-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:47:44 crc kubenswrapper[5052]: I1203 12:47:44.967793 5052 reconciler_common.go:293] "Volume detached for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/718fd921-0756-4a3e-b383-ed0d0ddf5b72-httpd-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:47:44 crc kubenswrapper[5052]: I1203 12:47:44.967801 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/718fd921-0756-4a3e-b383-ed0d0ddf5b72-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:47:44 crc kubenswrapper[5052]: I1203 12:47:44.969054 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/718fd921-0756-4a3e-b383-ed0d0ddf5b72-config" (OuterVolumeSpecName: "config") pod "718fd921-0756-4a3e-b383-ed0d0ddf5b72" (UID: "718fd921-0756-4a3e-b383-ed0d0ddf5b72"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:47:45 crc kubenswrapper[5052]: I1203 12:47:45.069427 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/718fd921-0756-4a3e-b383-ed0d0ddf5b72-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:47:45 crc kubenswrapper[5052]: I1203 12:47:45.109152 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/neutron-8b544ff76-6hjc7"] Dec 03 12:47:45 crc kubenswrapper[5052]: I1203 12:47:45.121529 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/neutron-8b544ff76-6hjc7"] Dec 03 12:47:45 crc kubenswrapper[5052]: I1203 12:47:45.683396 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="718fd921-0756-4a3e-b383-ed0d0ddf5b72" path="/var/lib/kubelet/pods/718fd921-0756-4a3e-b383-ed0d0ddf5b72/volumes" Dec 03 12:47:46 crc kubenswrapper[5052]: I1203 12:47:46.592679 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/openstackclient"] Dec 03 12:47:46 crc kubenswrapper[5052]: E1203 12:47:46.593181 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="76534e0c-3b73-4ede-bc24-b12c7f8ea4db" containerName="placement-log" Dec 03 12:47:46 crc kubenswrapper[5052]: I1203 12:47:46.593196 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="76534e0c-3b73-4ede-bc24-b12c7f8ea4db" containerName="placement-log" Dec 03 12:47:46 crc kubenswrapper[5052]: E1203 12:47:46.593226 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="74a94443-98ef-44b2-ae1b-43644b7c393b" containerName="barbican-api" Dec 03 12:47:46 crc kubenswrapper[5052]: I1203 12:47:46.593234 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="74a94443-98ef-44b2-ae1b-43644b7c393b" containerName="barbican-api" Dec 03 12:47:46 crc kubenswrapper[5052]: E1203 12:47:46.593261 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="74a94443-98ef-44b2-ae1b-43644b7c393b" containerName="barbican-api-log" Dec 03 12:47:46 crc kubenswrapper[5052]: I1203 12:47:46.593269 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="74a94443-98ef-44b2-ae1b-43644b7c393b" containerName="barbican-api-log" Dec 03 12:47:46 crc kubenswrapper[5052]: E1203 12:47:46.593283 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="76534e0c-3b73-4ede-bc24-b12c7f8ea4db" containerName="placement-api" Dec 03 12:47:46 crc kubenswrapper[5052]: I1203 12:47:46.593290 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="76534e0c-3b73-4ede-bc24-b12c7f8ea4db" containerName="placement-api" Dec 03 12:47:46 crc kubenswrapper[5052]: E1203 12:47:46.593315 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="718fd921-0756-4a3e-b383-ed0d0ddf5b72" containerName="neutron-httpd" Dec 03 12:47:46 crc kubenswrapper[5052]: I1203 12:47:46.593323 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="718fd921-0756-4a3e-b383-ed0d0ddf5b72" containerName="neutron-httpd" Dec 03 12:47:46 crc kubenswrapper[5052]: E1203 12:47:46.593339 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="718fd921-0756-4a3e-b383-ed0d0ddf5b72" containerName="neutron-api" Dec 03 12:47:46 crc kubenswrapper[5052]: I1203 12:47:46.593351 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="718fd921-0756-4a3e-b383-ed0d0ddf5b72" containerName="neutron-api" Dec 03 12:47:46 crc kubenswrapper[5052]: I1203 12:47:46.593583 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="74a94443-98ef-44b2-ae1b-43644b7c393b" containerName="barbican-api-log" Dec 03 12:47:46 crc kubenswrapper[5052]: I1203 12:47:46.593613 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="74a94443-98ef-44b2-ae1b-43644b7c393b" containerName="barbican-api" Dec 03 12:47:46 crc kubenswrapper[5052]: I1203 12:47:46.593630 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="76534e0c-3b73-4ede-bc24-b12c7f8ea4db" containerName="placement-api" Dec 03 12:47:46 crc kubenswrapper[5052]: I1203 12:47:46.593640 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="718fd921-0756-4a3e-b383-ed0d0ddf5b72" containerName="neutron-api" Dec 03 12:47:46 crc kubenswrapper[5052]: I1203 12:47:46.593656 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="718fd921-0756-4a3e-b383-ed0d0ddf5b72" containerName="neutron-httpd" Dec 03 12:47:46 crc kubenswrapper[5052]: I1203 12:47:46.593669 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="76534e0c-3b73-4ede-bc24-b12c7f8ea4db" containerName="placement-log" Dec 03 12:47:46 crc kubenswrapper[5052]: I1203 12:47:46.594459 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstackclient" Dec 03 12:47:46 crc kubenswrapper[5052]: I1203 12:47:46.597862 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"openstack-config" Dec 03 12:47:46 crc kubenswrapper[5052]: I1203 12:47:46.598047 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"openstack-config-secret" Dec 03 12:47:46 crc kubenswrapper[5052]: I1203 12:47:46.599106 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"openstackclient-openstackclient-dockercfg-n5mxr" Dec 03 12:47:46 crc kubenswrapper[5052]: I1203 12:47:46.612477 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/openstackclient"] Dec 03 12:47:46 crc kubenswrapper[5052]: I1203 12:47:46.700269 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openstack-config-secret\" (UniqueName: \"kubernetes.io/secret/8a94fda3-63d8-4517-8063-f6c06688b430-openstack-config-secret\") pod \"openstackclient\" (UID: \"8a94fda3-63d8-4517-8063-f6c06688b430\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 12:47:46 crc kubenswrapper[5052]: I1203 12:47:46.700377 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openstack-config\" (UniqueName: \"kubernetes.io/configmap/8a94fda3-63d8-4517-8063-f6c06688b430-openstack-config\") pod \"openstackclient\" (UID: \"8a94fda3-63d8-4517-8063-f6c06688b430\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 12:47:46 crc kubenswrapper[5052]: I1203 12:47:46.700448 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hhpzk\" (UniqueName: \"kubernetes.io/projected/8a94fda3-63d8-4517-8063-f6c06688b430-kube-api-access-hhpzk\") pod \"openstackclient\" (UID: \"8a94fda3-63d8-4517-8063-f6c06688b430\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 12:47:46 crc kubenswrapper[5052]: I1203 12:47:46.700547 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8a94fda3-63d8-4517-8063-f6c06688b430-combined-ca-bundle\") pod \"openstackclient\" (UID: \"8a94fda3-63d8-4517-8063-f6c06688b430\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 12:47:46 crc kubenswrapper[5052]: I1203 12:47:46.801743 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"openstack-config-secret\" (UniqueName: \"kubernetes.io/secret/8a94fda3-63d8-4517-8063-f6c06688b430-openstack-config-secret\") pod \"openstackclient\" (UID: \"8a94fda3-63d8-4517-8063-f6c06688b430\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 12:47:46 crc kubenswrapper[5052]: I1203 12:47:46.801799 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"openstack-config\" (UniqueName: \"kubernetes.io/configmap/8a94fda3-63d8-4517-8063-f6c06688b430-openstack-config\") pod \"openstackclient\" (UID: \"8a94fda3-63d8-4517-8063-f6c06688b430\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 12:47:46 crc kubenswrapper[5052]: I1203 12:47:46.801862 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-hhpzk\" (UniqueName: \"kubernetes.io/projected/8a94fda3-63d8-4517-8063-f6c06688b430-kube-api-access-hhpzk\") pod \"openstackclient\" (UID: \"8a94fda3-63d8-4517-8063-f6c06688b430\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 12:47:46 crc kubenswrapper[5052]: I1203 12:47:46.801912 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8a94fda3-63d8-4517-8063-f6c06688b430-combined-ca-bundle\") pod \"openstackclient\" (UID: \"8a94fda3-63d8-4517-8063-f6c06688b430\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 12:47:46 crc kubenswrapper[5052]: I1203 12:47:46.802825 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"openstack-config\" (UniqueName: \"kubernetes.io/configmap/8a94fda3-63d8-4517-8063-f6c06688b430-openstack-config\") pod \"openstackclient\" (UID: \"8a94fda3-63d8-4517-8063-f6c06688b430\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 12:47:46 crc kubenswrapper[5052]: I1203 12:47:46.803103 5052 generic.go:334] "Generic (PLEG): container finished" podID="69bdf82f-4616-437b-a9aa-4a04978a3b63" containerID="86dfffcda0458cbe1fcc5f5c8662311799b8b237901bad0a12f858211ae70295" exitCode=0 Dec 03 12:47:46 crc kubenswrapper[5052]: I1203 12:47:46.803200 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-7f7d6d4c6c-sjnw5" event={"ID":"69bdf82f-4616-437b-a9aa-4a04978a3b63","Type":"ContainerDied","Data":"86dfffcda0458cbe1fcc5f5c8662311799b8b237901bad0a12f858211ae70295"} Dec 03 12:47:46 crc kubenswrapper[5052]: I1203 12:47:46.807217 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"openstack-config-secret\" (UniqueName: \"kubernetes.io/secret/8a94fda3-63d8-4517-8063-f6c06688b430-openstack-config-secret\") pod \"openstackclient\" (UID: \"8a94fda3-63d8-4517-8063-f6c06688b430\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 12:47:46 crc kubenswrapper[5052]: I1203 12:47:46.808298 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8a94fda3-63d8-4517-8063-f6c06688b430-combined-ca-bundle\") pod \"openstackclient\" (UID: \"8a94fda3-63d8-4517-8063-f6c06688b430\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 12:47:46 crc kubenswrapper[5052]: I1203 12:47:46.828355 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-hhpzk\" (UniqueName: \"kubernetes.io/projected/8a94fda3-63d8-4517-8063-f6c06688b430-kube-api-access-hhpzk\") pod \"openstackclient\" (UID: \"8a94fda3-63d8-4517-8063-f6c06688b430\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 12:47:46 crc kubenswrapper[5052]: I1203 12:47:46.919566 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstackclient" Dec 03 12:47:47 crc kubenswrapper[5052]: I1203 12:47:47.211164 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/openstackclient"] Dec 03 12:47:47 crc kubenswrapper[5052]: I1203 12:47:47.474774 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-7f7d6d4c6c-sjnw5" Dec 03 12:47:47 crc kubenswrapper[5052]: I1203 12:47:47.540309 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/69bdf82f-4616-437b-a9aa-4a04978a3b63-scripts\") pod \"69bdf82f-4616-437b-a9aa-4a04978a3b63\" (UID: \"69bdf82f-4616-437b-a9aa-4a04978a3b63\") " Dec 03 12:47:47 crc kubenswrapper[5052]: I1203 12:47:47.541010 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/69bdf82f-4616-437b-a9aa-4a04978a3b63-credential-keys\") pod \"69bdf82f-4616-437b-a9aa-4a04978a3b63\" (UID: \"69bdf82f-4616-437b-a9aa-4a04978a3b63\") " Dec 03 12:47:47 crc kubenswrapper[5052]: I1203 12:47:47.541067 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/69bdf82f-4616-437b-a9aa-4a04978a3b63-fernet-keys\") pod \"69bdf82f-4616-437b-a9aa-4a04978a3b63\" (UID: \"69bdf82f-4616-437b-a9aa-4a04978a3b63\") " Dec 03 12:47:47 crc kubenswrapper[5052]: I1203 12:47:47.541106 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/69bdf82f-4616-437b-a9aa-4a04978a3b63-combined-ca-bundle\") pod \"69bdf82f-4616-437b-a9aa-4a04978a3b63\" (UID: \"69bdf82f-4616-437b-a9aa-4a04978a3b63\") " Dec 03 12:47:47 crc kubenswrapper[5052]: I1203 12:47:47.541153 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/69bdf82f-4616-437b-a9aa-4a04978a3b63-internal-tls-certs\") pod \"69bdf82f-4616-437b-a9aa-4a04978a3b63\" (UID: \"69bdf82f-4616-437b-a9aa-4a04978a3b63\") " Dec 03 12:47:47 crc kubenswrapper[5052]: I1203 12:47:47.541253 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/69bdf82f-4616-437b-a9aa-4a04978a3b63-config-data\") pod \"69bdf82f-4616-437b-a9aa-4a04978a3b63\" (UID: \"69bdf82f-4616-437b-a9aa-4a04978a3b63\") " Dec 03 12:47:47 crc kubenswrapper[5052]: I1203 12:47:47.541326 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zhv82\" (UniqueName: \"kubernetes.io/projected/69bdf82f-4616-437b-a9aa-4a04978a3b63-kube-api-access-zhv82\") pod \"69bdf82f-4616-437b-a9aa-4a04978a3b63\" (UID: \"69bdf82f-4616-437b-a9aa-4a04978a3b63\") " Dec 03 12:47:47 crc kubenswrapper[5052]: I1203 12:47:47.541416 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/69bdf82f-4616-437b-a9aa-4a04978a3b63-public-tls-certs\") pod \"69bdf82f-4616-437b-a9aa-4a04978a3b63\" (UID: \"69bdf82f-4616-437b-a9aa-4a04978a3b63\") " Dec 03 12:47:47 crc kubenswrapper[5052]: I1203 12:47:47.554281 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/69bdf82f-4616-437b-a9aa-4a04978a3b63-credential-keys" (OuterVolumeSpecName: "credential-keys") pod "69bdf82f-4616-437b-a9aa-4a04978a3b63" (UID: "69bdf82f-4616-437b-a9aa-4a04978a3b63"). InnerVolumeSpecName "credential-keys". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:47:47 crc kubenswrapper[5052]: I1203 12:47:47.555223 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/69bdf82f-4616-437b-a9aa-4a04978a3b63-fernet-keys" (OuterVolumeSpecName: "fernet-keys") pod "69bdf82f-4616-437b-a9aa-4a04978a3b63" (UID: "69bdf82f-4616-437b-a9aa-4a04978a3b63"). InnerVolumeSpecName "fernet-keys". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:47:47 crc kubenswrapper[5052]: I1203 12:47:47.558152 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/69bdf82f-4616-437b-a9aa-4a04978a3b63-scripts" (OuterVolumeSpecName: "scripts") pod "69bdf82f-4616-437b-a9aa-4a04978a3b63" (UID: "69bdf82f-4616-437b-a9aa-4a04978a3b63"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:47:47 crc kubenswrapper[5052]: I1203 12:47:47.570506 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/69bdf82f-4616-437b-a9aa-4a04978a3b63-kube-api-access-zhv82" (OuterVolumeSpecName: "kube-api-access-zhv82") pod "69bdf82f-4616-437b-a9aa-4a04978a3b63" (UID: "69bdf82f-4616-437b-a9aa-4a04978a3b63"). InnerVolumeSpecName "kube-api-access-zhv82". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:47:47 crc kubenswrapper[5052]: I1203 12:47:47.602422 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/69bdf82f-4616-437b-a9aa-4a04978a3b63-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "69bdf82f-4616-437b-a9aa-4a04978a3b63" (UID: "69bdf82f-4616-437b-a9aa-4a04978a3b63"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:47:47 crc kubenswrapper[5052]: I1203 12:47:47.627998 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/69bdf82f-4616-437b-a9aa-4a04978a3b63-config-data" (OuterVolumeSpecName: "config-data") pod "69bdf82f-4616-437b-a9aa-4a04978a3b63" (UID: "69bdf82f-4616-437b-a9aa-4a04978a3b63"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:47:47 crc kubenswrapper[5052]: I1203 12:47:47.630406 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/69bdf82f-4616-437b-a9aa-4a04978a3b63-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "69bdf82f-4616-437b-a9aa-4a04978a3b63" (UID: "69bdf82f-4616-437b-a9aa-4a04978a3b63"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:47:47 crc kubenswrapper[5052]: I1203 12:47:47.640318 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/69bdf82f-4616-437b-a9aa-4a04978a3b63-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "69bdf82f-4616-437b-a9aa-4a04978a3b63" (UID: "69bdf82f-4616-437b-a9aa-4a04978a3b63"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:47:47 crc kubenswrapper[5052]: I1203 12:47:47.645156 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/69bdf82f-4616-437b-a9aa-4a04978a3b63-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:47:47 crc kubenswrapper[5052]: I1203 12:47:47.645190 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/69bdf82f-4616-437b-a9aa-4a04978a3b63-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:47:47 crc kubenswrapper[5052]: I1203 12:47:47.645200 5052 reconciler_common.go:293] "Volume detached for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/69bdf82f-4616-437b-a9aa-4a04978a3b63-credential-keys\") on node \"crc\" DevicePath \"\"" Dec 03 12:47:47 crc kubenswrapper[5052]: I1203 12:47:47.645209 5052 reconciler_common.go:293] "Volume detached for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/69bdf82f-4616-437b-a9aa-4a04978a3b63-fernet-keys\") on node \"crc\" DevicePath \"\"" Dec 03 12:47:47 crc kubenswrapper[5052]: I1203 12:47:47.645219 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/69bdf82f-4616-437b-a9aa-4a04978a3b63-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:47:47 crc kubenswrapper[5052]: I1203 12:47:47.645229 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/69bdf82f-4616-437b-a9aa-4a04978a3b63-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:47:47 crc kubenswrapper[5052]: I1203 12:47:47.645239 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/69bdf82f-4616-437b-a9aa-4a04978a3b63-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:47:47 crc kubenswrapper[5052]: I1203 12:47:47.645249 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-zhv82\" (UniqueName: \"kubernetes.io/projected/69bdf82f-4616-437b-a9aa-4a04978a3b63-kube-api-access-zhv82\") on node \"crc\" DevicePath \"\"" Dec 03 12:47:47 crc kubenswrapper[5052]: I1203 12:47:47.673469 5052 scope.go:117] "RemoveContainer" containerID="2aed1c760a326f3d008799e902b1093313df3764a57c879f0b3686410127d37d" Dec 03 12:47:47 crc kubenswrapper[5052]: E1203 12:47:47.673820 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 12:47:47 crc kubenswrapper[5052]: I1203 12:47:47.815683 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstackclient" event={"ID":"8a94fda3-63d8-4517-8063-f6c06688b430","Type":"ContainerStarted","Data":"72865419cd302812d0e8a3d9aaf5db18884e1b028e954ec10b2f5bb12f983c9c"} Dec 03 12:47:47 crc kubenswrapper[5052]: I1203 12:47:47.821961 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-7f7d6d4c6c-sjnw5" event={"ID":"69bdf82f-4616-437b-a9aa-4a04978a3b63","Type":"ContainerDied","Data":"92f7d32569328ac8ba5c2c899dd6d63b05ed17e11c98b3fe77fd40c06c4bdc81"} Dec 03 12:47:47 crc kubenswrapper[5052]: I1203 12:47:47.822015 5052 scope.go:117] "RemoveContainer" containerID="86dfffcda0458cbe1fcc5f5c8662311799b8b237901bad0a12f858211ae70295" Dec 03 12:47:47 crc kubenswrapper[5052]: I1203 12:47:47.822213 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-7f7d6d4c6c-sjnw5" Dec 03 12:47:47 crc kubenswrapper[5052]: I1203 12:47:47.924113 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/keystone-7f7d6d4c6c-sjnw5"] Dec 03 12:47:47 crc kubenswrapper[5052]: I1203 12:47:47.948262 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/keystone-7f7d6d4c6c-sjnw5"] Dec 03 12:47:48 crc kubenswrapper[5052]: I1203 12:47:48.839127 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstackclient" event={"ID":"8a94fda3-63d8-4517-8063-f6c06688b430","Type":"ContainerStarted","Data":"e3c10016f8e333ebc2b5eef7809847e5f7ea2904a0f92aa3ddc61a80ff2d9403"} Dec 03 12:47:48 crc kubenswrapper[5052]: I1203 12:47:48.864313 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/openstackclient" podStartSLOduration=2.864284994 podStartE2EDuration="2.864284994s" podCreationTimestamp="2025-12-03 12:47:46 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:47:48.853058293 +0000 UTC m=+5259.441300057" watchObservedRunningTime="2025-12-03 12:47:48.864284994 +0000 UTC m=+5259.452526758" Dec 03 12:47:49 crc kubenswrapper[5052]: I1203 12:47:49.683455 5052 scope.go:117] "RemoveContainer" containerID="1fdbade92a13691b593283a5b263382dfec190075f1201c77fff78a7be754487" Dec 03 12:47:49 crc kubenswrapper[5052]: E1203 12:47:49.683711 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"nova-cell0-conductor-conductor\" with CrashLoopBackOff: \"back-off 40s restarting failed container=nova-cell0-conductor-conductor pod=nova-cell0-conductor-0_openstack-kuttl-tests(6d653682-52e2-4940-af66-633475bfe09a)\"" pod="openstack-kuttl-tests/nova-cell0-conductor-0" podUID="6d653682-52e2-4940-af66-633475bfe09a" Dec 03 12:47:49 crc kubenswrapper[5052]: I1203 12:47:49.688157 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="69bdf82f-4616-437b-a9aa-4a04978a3b63" path="/var/lib/kubelet/pods/69bdf82f-4616-437b-a9aa-4a04978a3b63/volumes" Dec 03 12:47:50 crc kubenswrapper[5052]: I1203 12:47:50.128468 5052 pod_container_manager_linux.go:210] "Failed to delete cgroup paths" cgroupName=["kubepods","besteffort","pod1e70c60d-8835-4b4f-80dc-7190e8f6facd"] err="unable to destroy cgroup paths for cgroup [kubepods besteffort pod1e70c60d-8835-4b4f-80dc-7190e8f6facd] : Timed out while waiting for systemd to remove kubepods-besteffort-pod1e70c60d_8835_4b4f_80dc_7190e8f6facd.slice" Dec 03 12:47:50 crc kubenswrapper[5052]: E1203 12:47:50.128810 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to delete cgroup paths for [kubepods besteffort pod1e70c60d-8835-4b4f-80dc-7190e8f6facd] : unable to destroy cgroup paths for cgroup [kubepods besteffort pod1e70c60d-8835-4b4f-80dc-7190e8f6facd] : Timed out while waiting for systemd to remove kubepods-besteffort-pod1e70c60d_8835_4b4f_80dc_7190e8f6facd.slice" pod="openstack-kuttl-tests/nova-api-0" podUID="1e70c60d-8835-4b4f-80dc-7190e8f6facd" Dec 03 12:47:50 crc kubenswrapper[5052]: I1203 12:47:50.860342 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:47:50 crc kubenswrapper[5052]: I1203 12:47:50.901435 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:47:50 crc kubenswrapper[5052]: I1203 12:47:50.915762 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:47:50 crc kubenswrapper[5052]: I1203 12:47:50.929429 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:47:50 crc kubenswrapper[5052]: E1203 12:47:50.929943 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="69bdf82f-4616-437b-a9aa-4a04978a3b63" containerName="keystone-api" Dec 03 12:47:50 crc kubenswrapper[5052]: I1203 12:47:50.929971 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="69bdf82f-4616-437b-a9aa-4a04978a3b63" containerName="keystone-api" Dec 03 12:47:50 crc kubenswrapper[5052]: I1203 12:47:50.930259 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="69bdf82f-4616-437b-a9aa-4a04978a3b63" containerName="keystone-api" Dec 03 12:47:50 crc kubenswrapper[5052]: I1203 12:47:50.931277 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:47:50 crc kubenswrapper[5052]: I1203 12:47:50.934570 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-api-config-data" Dec 03 12:47:50 crc kubenswrapper[5052]: I1203 12:47:50.935022 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-nova-internal-svc" Dec 03 12:47:50 crc kubenswrapper[5052]: I1203 12:47:50.935461 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-nova-public-svc" Dec 03 12:47:50 crc kubenswrapper[5052]: I1203 12:47:50.941759 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:47:51 crc kubenswrapper[5052]: I1203 12:47:51.013102 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6c8ad676-e7dd-479c-8d22-f546dc5e50af-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"6c8ad676-e7dd-479c-8d22-f546dc5e50af\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:47:51 crc kubenswrapper[5052]: I1203 12:47:51.013165 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/6c8ad676-e7dd-479c-8d22-f546dc5e50af-public-tls-certs\") pod \"nova-api-0\" (UID: \"6c8ad676-e7dd-479c-8d22-f546dc5e50af\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:47:51 crc kubenswrapper[5052]: I1203 12:47:51.013205 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/6c8ad676-e7dd-479c-8d22-f546dc5e50af-logs\") pod \"nova-api-0\" (UID: \"6c8ad676-e7dd-479c-8d22-f546dc5e50af\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:47:51 crc kubenswrapper[5052]: I1203 12:47:51.013299 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n55nw\" (UniqueName: \"kubernetes.io/projected/6c8ad676-e7dd-479c-8d22-f546dc5e50af-kube-api-access-n55nw\") pod \"nova-api-0\" (UID: \"6c8ad676-e7dd-479c-8d22-f546dc5e50af\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:47:51 crc kubenswrapper[5052]: I1203 12:47:51.013396 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/6c8ad676-e7dd-479c-8d22-f546dc5e50af-internal-tls-certs\") pod \"nova-api-0\" (UID: \"6c8ad676-e7dd-479c-8d22-f546dc5e50af\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:47:51 crc kubenswrapper[5052]: I1203 12:47:51.013494 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6c8ad676-e7dd-479c-8d22-f546dc5e50af-config-data\") pod \"nova-api-0\" (UID: \"6c8ad676-e7dd-479c-8d22-f546dc5e50af\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:47:51 crc kubenswrapper[5052]: I1203 12:47:51.115499 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6c8ad676-e7dd-479c-8d22-f546dc5e50af-config-data\") pod \"nova-api-0\" (UID: \"6c8ad676-e7dd-479c-8d22-f546dc5e50af\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:47:51 crc kubenswrapper[5052]: I1203 12:47:51.115657 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6c8ad676-e7dd-479c-8d22-f546dc5e50af-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"6c8ad676-e7dd-479c-8d22-f546dc5e50af\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:47:51 crc kubenswrapper[5052]: I1203 12:47:51.115694 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/6c8ad676-e7dd-479c-8d22-f546dc5e50af-public-tls-certs\") pod \"nova-api-0\" (UID: \"6c8ad676-e7dd-479c-8d22-f546dc5e50af\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:47:51 crc kubenswrapper[5052]: I1203 12:47:51.115715 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/6c8ad676-e7dd-479c-8d22-f546dc5e50af-logs\") pod \"nova-api-0\" (UID: \"6c8ad676-e7dd-479c-8d22-f546dc5e50af\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:47:51 crc kubenswrapper[5052]: I1203 12:47:51.115761 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-n55nw\" (UniqueName: \"kubernetes.io/projected/6c8ad676-e7dd-479c-8d22-f546dc5e50af-kube-api-access-n55nw\") pod \"nova-api-0\" (UID: \"6c8ad676-e7dd-479c-8d22-f546dc5e50af\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:47:51 crc kubenswrapper[5052]: I1203 12:47:51.115845 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/6c8ad676-e7dd-479c-8d22-f546dc5e50af-internal-tls-certs\") pod \"nova-api-0\" (UID: \"6c8ad676-e7dd-479c-8d22-f546dc5e50af\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:47:51 crc kubenswrapper[5052]: I1203 12:47:51.118033 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/6c8ad676-e7dd-479c-8d22-f546dc5e50af-logs\") pod \"nova-api-0\" (UID: \"6c8ad676-e7dd-479c-8d22-f546dc5e50af\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:47:51 crc kubenswrapper[5052]: I1203 12:47:51.123364 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/6c8ad676-e7dd-479c-8d22-f546dc5e50af-internal-tls-certs\") pod \"nova-api-0\" (UID: \"6c8ad676-e7dd-479c-8d22-f546dc5e50af\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:47:51 crc kubenswrapper[5052]: I1203 12:47:51.123764 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6c8ad676-e7dd-479c-8d22-f546dc5e50af-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"6c8ad676-e7dd-479c-8d22-f546dc5e50af\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:47:51 crc kubenswrapper[5052]: I1203 12:47:51.126416 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6c8ad676-e7dd-479c-8d22-f546dc5e50af-config-data\") pod \"nova-api-0\" (UID: \"6c8ad676-e7dd-479c-8d22-f546dc5e50af\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:47:51 crc kubenswrapper[5052]: I1203 12:47:51.126862 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/6c8ad676-e7dd-479c-8d22-f546dc5e50af-public-tls-certs\") pod \"nova-api-0\" (UID: \"6c8ad676-e7dd-479c-8d22-f546dc5e50af\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:47:51 crc kubenswrapper[5052]: I1203 12:47:51.149960 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-n55nw\" (UniqueName: \"kubernetes.io/projected/6c8ad676-e7dd-479c-8d22-f546dc5e50af-kube-api-access-n55nw\") pod \"nova-api-0\" (UID: \"6c8ad676-e7dd-479c-8d22-f546dc5e50af\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:47:51 crc kubenswrapper[5052]: I1203 12:47:51.253618 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:47:51 crc kubenswrapper[5052]: I1203 12:47:51.669445 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:47:51 crc kubenswrapper[5052]: I1203 12:47:51.673008 5052 scope.go:117] "RemoveContainer" containerID="b1f27aab2d1cce51772c0ce99bd8dfc4afd0620b38b776e6b968d9114105f165" Dec 03 12:47:51 crc kubenswrapper[5052]: E1203 12:47:51.673345 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"nova-scheduler-scheduler\" with CrashLoopBackOff: \"back-off 40s restarting failed container=nova-scheduler-scheduler pod=nova-scheduler-0_openstack-kuttl-tests(81709983-1fd0-4446-bbbf-fb94653bb3d2)\"" pod="openstack-kuttl-tests/nova-scheduler-0" podUID="81709983-1fd0-4446-bbbf-fb94653bb3d2" Dec 03 12:47:51 crc kubenswrapper[5052]: I1203 12:47:51.701225 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1e70c60d-8835-4b4f-80dc-7190e8f6facd" path="/var/lib/kubelet/pods/1e70c60d-8835-4b4f-80dc-7190e8f6facd/volumes" Dec 03 12:47:51 crc kubenswrapper[5052]: I1203 12:47:51.880970 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"6c8ad676-e7dd-479c-8d22-f546dc5e50af","Type":"ContainerStarted","Data":"a3f4f93f3787a5482025f013961d9f934ba51e800bf3b24730bce1b1e2696ee6"} Dec 03 12:47:51 crc kubenswrapper[5052]: I1203 12:47:51.881352 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"6c8ad676-e7dd-479c-8d22-f546dc5e50af","Type":"ContainerStarted","Data":"f1927cb0d3f87c444e2fca5cdd8c0bdb808bdc516086e07afbe7cd05db09f571"} Dec 03 12:47:52 crc kubenswrapper[5052]: I1203 12:47:52.353473 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/swift-proxy-78f78b88b6-fmjvp"] Dec 03 12:47:52 crc kubenswrapper[5052]: I1203 12:47:52.355879 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-proxy-78f78b88b6-fmjvp" Dec 03 12:47:52 crc kubenswrapper[5052]: I1203 12:47:52.385999 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/swift-proxy-78f78b88b6-fmjvp"] Dec 03 12:47:52 crc kubenswrapper[5052]: I1203 12:47:52.443588 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/dd8df313-3680-4dd1-a57d-ff85b0b2c6f1-internal-tls-certs\") pod \"swift-proxy-78f78b88b6-fmjvp\" (UID: \"dd8df313-3680-4dd1-a57d-ff85b0b2c6f1\") " pod="openstack-kuttl-tests/swift-proxy-78f78b88b6-fmjvp" Dec 03 12:47:52 crc kubenswrapper[5052]: I1203 12:47:52.443935 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/dd8df313-3680-4dd1-a57d-ff85b0b2c6f1-log-httpd\") pod \"swift-proxy-78f78b88b6-fmjvp\" (UID: \"dd8df313-3680-4dd1-a57d-ff85b0b2c6f1\") " pod="openstack-kuttl-tests/swift-proxy-78f78b88b6-fmjvp" Dec 03 12:47:52 crc kubenswrapper[5052]: I1203 12:47:52.443987 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/dd8df313-3680-4dd1-a57d-ff85b0b2c6f1-run-httpd\") pod \"swift-proxy-78f78b88b6-fmjvp\" (UID: \"dd8df313-3680-4dd1-a57d-ff85b0b2c6f1\") " pod="openstack-kuttl-tests/swift-proxy-78f78b88b6-fmjvp" Dec 03 12:47:52 crc kubenswrapper[5052]: I1203 12:47:52.444242 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qcfj5\" (UniqueName: \"kubernetes.io/projected/dd8df313-3680-4dd1-a57d-ff85b0b2c6f1-kube-api-access-qcfj5\") pod \"swift-proxy-78f78b88b6-fmjvp\" (UID: \"dd8df313-3680-4dd1-a57d-ff85b0b2c6f1\") " pod="openstack-kuttl-tests/swift-proxy-78f78b88b6-fmjvp" Dec 03 12:47:52 crc kubenswrapper[5052]: I1203 12:47:52.444300 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/dd8df313-3680-4dd1-a57d-ff85b0b2c6f1-public-tls-certs\") pod \"swift-proxy-78f78b88b6-fmjvp\" (UID: \"dd8df313-3680-4dd1-a57d-ff85b0b2c6f1\") " pod="openstack-kuttl-tests/swift-proxy-78f78b88b6-fmjvp" Dec 03 12:47:52 crc kubenswrapper[5052]: I1203 12:47:52.444523 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/dd8df313-3680-4dd1-a57d-ff85b0b2c6f1-etc-swift\") pod \"swift-proxy-78f78b88b6-fmjvp\" (UID: \"dd8df313-3680-4dd1-a57d-ff85b0b2c6f1\") " pod="openstack-kuttl-tests/swift-proxy-78f78b88b6-fmjvp" Dec 03 12:47:52 crc kubenswrapper[5052]: I1203 12:47:52.444598 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/dd8df313-3680-4dd1-a57d-ff85b0b2c6f1-config-data\") pod \"swift-proxy-78f78b88b6-fmjvp\" (UID: \"dd8df313-3680-4dd1-a57d-ff85b0b2c6f1\") " pod="openstack-kuttl-tests/swift-proxy-78f78b88b6-fmjvp" Dec 03 12:47:52 crc kubenswrapper[5052]: I1203 12:47:52.444767 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/dd8df313-3680-4dd1-a57d-ff85b0b2c6f1-combined-ca-bundle\") pod \"swift-proxy-78f78b88b6-fmjvp\" (UID: \"dd8df313-3680-4dd1-a57d-ff85b0b2c6f1\") " pod="openstack-kuttl-tests/swift-proxy-78f78b88b6-fmjvp" Dec 03 12:47:52 crc kubenswrapper[5052]: I1203 12:47:52.546858 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/dd8df313-3680-4dd1-a57d-ff85b0b2c6f1-internal-tls-certs\") pod \"swift-proxy-78f78b88b6-fmjvp\" (UID: \"dd8df313-3680-4dd1-a57d-ff85b0b2c6f1\") " pod="openstack-kuttl-tests/swift-proxy-78f78b88b6-fmjvp" Dec 03 12:47:52 crc kubenswrapper[5052]: I1203 12:47:52.546971 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/dd8df313-3680-4dd1-a57d-ff85b0b2c6f1-run-httpd\") pod \"swift-proxy-78f78b88b6-fmjvp\" (UID: \"dd8df313-3680-4dd1-a57d-ff85b0b2c6f1\") " pod="openstack-kuttl-tests/swift-proxy-78f78b88b6-fmjvp" Dec 03 12:47:52 crc kubenswrapper[5052]: I1203 12:47:52.546995 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/dd8df313-3680-4dd1-a57d-ff85b0b2c6f1-log-httpd\") pod \"swift-proxy-78f78b88b6-fmjvp\" (UID: \"dd8df313-3680-4dd1-a57d-ff85b0b2c6f1\") " pod="openstack-kuttl-tests/swift-proxy-78f78b88b6-fmjvp" Dec 03 12:47:52 crc kubenswrapper[5052]: I1203 12:47:52.547092 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-qcfj5\" (UniqueName: \"kubernetes.io/projected/dd8df313-3680-4dd1-a57d-ff85b0b2c6f1-kube-api-access-qcfj5\") pod \"swift-proxy-78f78b88b6-fmjvp\" (UID: \"dd8df313-3680-4dd1-a57d-ff85b0b2c6f1\") " pod="openstack-kuttl-tests/swift-proxy-78f78b88b6-fmjvp" Dec 03 12:47:52 crc kubenswrapper[5052]: I1203 12:47:52.547121 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/dd8df313-3680-4dd1-a57d-ff85b0b2c6f1-public-tls-certs\") pod \"swift-proxy-78f78b88b6-fmjvp\" (UID: \"dd8df313-3680-4dd1-a57d-ff85b0b2c6f1\") " pod="openstack-kuttl-tests/swift-proxy-78f78b88b6-fmjvp" Dec 03 12:47:52 crc kubenswrapper[5052]: I1203 12:47:52.547160 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/dd8df313-3680-4dd1-a57d-ff85b0b2c6f1-etc-swift\") pod \"swift-proxy-78f78b88b6-fmjvp\" (UID: \"dd8df313-3680-4dd1-a57d-ff85b0b2c6f1\") " pod="openstack-kuttl-tests/swift-proxy-78f78b88b6-fmjvp" Dec 03 12:47:52 crc kubenswrapper[5052]: I1203 12:47:52.547190 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/dd8df313-3680-4dd1-a57d-ff85b0b2c6f1-config-data\") pod \"swift-proxy-78f78b88b6-fmjvp\" (UID: \"dd8df313-3680-4dd1-a57d-ff85b0b2c6f1\") " pod="openstack-kuttl-tests/swift-proxy-78f78b88b6-fmjvp" Dec 03 12:47:52 crc kubenswrapper[5052]: I1203 12:47:52.547237 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/dd8df313-3680-4dd1-a57d-ff85b0b2c6f1-combined-ca-bundle\") pod \"swift-proxy-78f78b88b6-fmjvp\" (UID: \"dd8df313-3680-4dd1-a57d-ff85b0b2c6f1\") " pod="openstack-kuttl-tests/swift-proxy-78f78b88b6-fmjvp" Dec 03 12:47:52 crc kubenswrapper[5052]: I1203 12:47:52.547639 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/dd8df313-3680-4dd1-a57d-ff85b0b2c6f1-log-httpd\") pod \"swift-proxy-78f78b88b6-fmjvp\" (UID: \"dd8df313-3680-4dd1-a57d-ff85b0b2c6f1\") " pod="openstack-kuttl-tests/swift-proxy-78f78b88b6-fmjvp" Dec 03 12:47:52 crc kubenswrapper[5052]: I1203 12:47:52.547929 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/dd8df313-3680-4dd1-a57d-ff85b0b2c6f1-run-httpd\") pod \"swift-proxy-78f78b88b6-fmjvp\" (UID: \"dd8df313-3680-4dd1-a57d-ff85b0b2c6f1\") " pod="openstack-kuttl-tests/swift-proxy-78f78b88b6-fmjvp" Dec 03 12:47:52 crc kubenswrapper[5052]: I1203 12:47:52.553169 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/dd8df313-3680-4dd1-a57d-ff85b0b2c6f1-config-data\") pod \"swift-proxy-78f78b88b6-fmjvp\" (UID: \"dd8df313-3680-4dd1-a57d-ff85b0b2c6f1\") " pod="openstack-kuttl-tests/swift-proxy-78f78b88b6-fmjvp" Dec 03 12:47:52 crc kubenswrapper[5052]: I1203 12:47:52.554985 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/dd8df313-3680-4dd1-a57d-ff85b0b2c6f1-public-tls-certs\") pod \"swift-proxy-78f78b88b6-fmjvp\" (UID: \"dd8df313-3680-4dd1-a57d-ff85b0b2c6f1\") " pod="openstack-kuttl-tests/swift-proxy-78f78b88b6-fmjvp" Dec 03 12:47:52 crc kubenswrapper[5052]: I1203 12:47:52.557461 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/dd8df313-3680-4dd1-a57d-ff85b0b2c6f1-internal-tls-certs\") pod \"swift-proxy-78f78b88b6-fmjvp\" (UID: \"dd8df313-3680-4dd1-a57d-ff85b0b2c6f1\") " pod="openstack-kuttl-tests/swift-proxy-78f78b88b6-fmjvp" Dec 03 12:47:52 crc kubenswrapper[5052]: I1203 12:47:52.572654 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/dd8df313-3680-4dd1-a57d-ff85b0b2c6f1-etc-swift\") pod \"swift-proxy-78f78b88b6-fmjvp\" (UID: \"dd8df313-3680-4dd1-a57d-ff85b0b2c6f1\") " pod="openstack-kuttl-tests/swift-proxy-78f78b88b6-fmjvp" Dec 03 12:47:52 crc kubenswrapper[5052]: I1203 12:47:52.578695 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/dd8df313-3680-4dd1-a57d-ff85b0b2c6f1-combined-ca-bundle\") pod \"swift-proxy-78f78b88b6-fmjvp\" (UID: \"dd8df313-3680-4dd1-a57d-ff85b0b2c6f1\") " pod="openstack-kuttl-tests/swift-proxy-78f78b88b6-fmjvp" Dec 03 12:47:52 crc kubenswrapper[5052]: I1203 12:47:52.580330 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-qcfj5\" (UniqueName: \"kubernetes.io/projected/dd8df313-3680-4dd1-a57d-ff85b0b2c6f1-kube-api-access-qcfj5\") pod \"swift-proxy-78f78b88b6-fmjvp\" (UID: \"dd8df313-3680-4dd1-a57d-ff85b0b2c6f1\") " pod="openstack-kuttl-tests/swift-proxy-78f78b88b6-fmjvp" Dec 03 12:47:52 crc kubenswrapper[5052]: I1203 12:47:52.681412 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-proxy-78f78b88b6-fmjvp" Dec 03 12:47:52 crc kubenswrapper[5052]: I1203 12:47:52.901368 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"6c8ad676-e7dd-479c-8d22-f546dc5e50af","Type":"ContainerStarted","Data":"f38a12088ebf63cf61a94810998aa693db18e09b88bf0391d892ee960b99e9ce"} Dec 03 12:47:52 crc kubenswrapper[5052]: I1203 12:47:52.944730 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-api-0" podStartSLOduration=2.944695637 podStartE2EDuration="2.944695637s" podCreationTimestamp="2025-12-03 12:47:50 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:47:52.934602618 +0000 UTC m=+5263.522844392" watchObservedRunningTime="2025-12-03 12:47:52.944695637 +0000 UTC m=+5263.532937401" Dec 03 12:47:53 crc kubenswrapper[5052]: I1203 12:47:53.166820 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/swift-proxy-78f78b88b6-fmjvp"] Dec 03 12:47:53 crc kubenswrapper[5052]: I1203 12:47:53.915184 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-proxy-78f78b88b6-fmjvp" event={"ID":"dd8df313-3680-4dd1-a57d-ff85b0b2c6f1","Type":"ContainerStarted","Data":"45b85b0267828ad53c05c095096047353cbe686d4edc60d6dabed09ff937a326"} Dec 03 12:47:53 crc kubenswrapper[5052]: I1203 12:47:53.915728 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/swift-proxy-78f78b88b6-fmjvp" Dec 03 12:47:53 crc kubenswrapper[5052]: I1203 12:47:53.915753 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-proxy-78f78b88b6-fmjvp" event={"ID":"dd8df313-3680-4dd1-a57d-ff85b0b2c6f1","Type":"ContainerStarted","Data":"acb0be96c14d669ff7b80a9476845494757b206ec5aaddb2a34ac9ff0fe87bd7"} Dec 03 12:47:53 crc kubenswrapper[5052]: I1203 12:47:53.915767 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-proxy-78f78b88b6-fmjvp" event={"ID":"dd8df313-3680-4dd1-a57d-ff85b0b2c6f1","Type":"ContainerStarted","Data":"dad1f7c6237c7c3f6729d53c0f9f7960cf9320a5a2f22fe4bc78d3fc12ff914e"} Dec 03 12:47:53 crc kubenswrapper[5052]: I1203 12:47:53.939337 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/swift-proxy-78f78b88b6-fmjvp" podStartSLOduration=1.939313351 podStartE2EDuration="1.939313351s" podCreationTimestamp="2025-12-03 12:47:52 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:47:53.936775379 +0000 UTC m=+5264.525017143" watchObservedRunningTime="2025-12-03 12:47:53.939313351 +0000 UTC m=+5264.527555115" Dec 03 12:47:54 crc kubenswrapper[5052]: I1203 12:47:54.403310 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:47:54 crc kubenswrapper[5052]: I1203 12:47:54.673239 5052 scope.go:117] "RemoveContainer" containerID="8e3764a05c784a75a82ccff0fcb9f4aaac35f5d6845cca81b14deee31318466f" Dec 03 12:47:54 crc kubenswrapper[5052]: E1203 12:47:54.673852 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"nova-cell1-conductor-conductor\" with CrashLoopBackOff: \"back-off 40s restarting failed container=nova-cell1-conductor-conductor pod=nova-cell1-conductor-0_openstack-kuttl-tests(8669832d-c47a-49bd-9491-dc7141ac6281)\"" pod="openstack-kuttl-tests/nova-cell1-conductor-0" podUID="8669832d-c47a-49bd-9491-dc7141ac6281" Dec 03 12:47:54 crc kubenswrapper[5052]: I1203 12:47:54.924785 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/swift-proxy-78f78b88b6-fmjvp" Dec 03 12:47:56 crc kubenswrapper[5052]: I1203 12:47:56.396847 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:47:56 crc kubenswrapper[5052]: I1203 12:47:56.397426 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="0068c169-1560-4afb-918b-952444f11921" containerName="ceilometer-central-agent" containerID="cri-o://8640584ba053d7aa82893be1329fb1cb047434c76621e6cdd0181fc1cc14fcee" gracePeriod=30 Dec 03 12:47:56 crc kubenswrapper[5052]: I1203 12:47:56.398024 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="0068c169-1560-4afb-918b-952444f11921" containerName="proxy-httpd" containerID="cri-o://a258638afb610e911ce997d085715008b91cd8f4631d20d5dbcd5ab4b26cdb22" gracePeriod=30 Dec 03 12:47:56 crc kubenswrapper[5052]: I1203 12:47:56.398073 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="0068c169-1560-4afb-918b-952444f11921" containerName="sg-core" containerID="cri-o://cc4689adb2d0f67123301989da14c8be70ee0b36ea8974223f0120e1d748cc86" gracePeriod=30 Dec 03 12:47:56 crc kubenswrapper[5052]: I1203 12:47:56.398110 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="0068c169-1560-4afb-918b-952444f11921" containerName="ceilometer-notification-agent" containerID="cri-o://ee710ca8fa0188ffa62dbf27204f55233993aeb05b3f79709c5ef0748fbb7339" gracePeriod=30 Dec 03 12:47:56 crc kubenswrapper[5052]: I1203 12:47:56.947582 5052 generic.go:334] "Generic (PLEG): container finished" podID="0068c169-1560-4afb-918b-952444f11921" containerID="a258638afb610e911ce997d085715008b91cd8f4631d20d5dbcd5ab4b26cdb22" exitCode=0 Dec 03 12:47:56 crc kubenswrapper[5052]: I1203 12:47:56.947616 5052 generic.go:334] "Generic (PLEG): container finished" podID="0068c169-1560-4afb-918b-952444f11921" containerID="cc4689adb2d0f67123301989da14c8be70ee0b36ea8974223f0120e1d748cc86" exitCode=2 Dec 03 12:47:56 crc kubenswrapper[5052]: I1203 12:47:56.947640 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"0068c169-1560-4afb-918b-952444f11921","Type":"ContainerDied","Data":"a258638afb610e911ce997d085715008b91cd8f4631d20d5dbcd5ab4b26cdb22"} Dec 03 12:47:56 crc kubenswrapper[5052]: I1203 12:47:56.947679 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"0068c169-1560-4afb-918b-952444f11921","Type":"ContainerDied","Data":"cc4689adb2d0f67123301989da14c8be70ee0b36ea8974223f0120e1d748cc86"} Dec 03 12:47:57 crc kubenswrapper[5052]: I1203 12:47:57.505146 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/neutron-68f85bdc64-dc642" podUID="95997b63-313e-4f5f-adf6-b8ae63de9fb9" containerName="neutron-httpd" probeResult="failure" output="Get \"https://10.217.1.194:9696/\": dial tcp 10.217.1.194:9696: connect: connection refused" Dec 03 12:47:57 crc kubenswrapper[5052]: I1203 12:47:57.959098 5052 generic.go:334] "Generic (PLEG): container finished" podID="0068c169-1560-4afb-918b-952444f11921" containerID="8640584ba053d7aa82893be1329fb1cb047434c76621e6cdd0181fc1cc14fcee" exitCode=0 Dec 03 12:47:57 crc kubenswrapper[5052]: I1203 12:47:57.959145 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"0068c169-1560-4afb-918b-952444f11921","Type":"ContainerDied","Data":"8640584ba053d7aa82893be1329fb1cb047434c76621e6cdd0181fc1cc14fcee"} Dec 03 12:47:59 crc kubenswrapper[5052]: I1203 12:47:59.428006 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:47:59 crc kubenswrapper[5052]: I1203 12:47:59.493239 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/0068c169-1560-4afb-918b-952444f11921-scripts\") pod \"0068c169-1560-4afb-918b-952444f11921\" (UID: \"0068c169-1560-4afb-918b-952444f11921\") " Dec 03 12:47:59 crc kubenswrapper[5052]: I1203 12:47:59.493290 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/0068c169-1560-4afb-918b-952444f11921-log-httpd\") pod \"0068c169-1560-4afb-918b-952444f11921\" (UID: \"0068c169-1560-4afb-918b-952444f11921\") " Dec 03 12:47:59 crc kubenswrapper[5052]: I1203 12:47:59.493318 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0068c169-1560-4afb-918b-952444f11921-combined-ca-bundle\") pod \"0068c169-1560-4afb-918b-952444f11921\" (UID: \"0068c169-1560-4afb-918b-952444f11921\") " Dec 03 12:47:59 crc kubenswrapper[5052]: I1203 12:47:59.493407 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/0068c169-1560-4afb-918b-952444f11921-config-data\") pod \"0068c169-1560-4afb-918b-952444f11921\" (UID: \"0068c169-1560-4afb-918b-952444f11921\") " Dec 03 12:47:59 crc kubenswrapper[5052]: I1203 12:47:59.493439 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/0068c169-1560-4afb-918b-952444f11921-run-httpd\") pod \"0068c169-1560-4afb-918b-952444f11921\" (UID: \"0068c169-1560-4afb-918b-952444f11921\") " Dec 03 12:47:59 crc kubenswrapper[5052]: I1203 12:47:59.493520 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/0068c169-1560-4afb-918b-952444f11921-ceilometer-tls-certs\") pod \"0068c169-1560-4afb-918b-952444f11921\" (UID: \"0068c169-1560-4afb-918b-952444f11921\") " Dec 03 12:47:59 crc kubenswrapper[5052]: I1203 12:47:59.493558 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x72h8\" (UniqueName: \"kubernetes.io/projected/0068c169-1560-4afb-918b-952444f11921-kube-api-access-x72h8\") pod \"0068c169-1560-4afb-918b-952444f11921\" (UID: \"0068c169-1560-4afb-918b-952444f11921\") " Dec 03 12:47:59 crc kubenswrapper[5052]: I1203 12:47:59.493581 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/0068c169-1560-4afb-918b-952444f11921-sg-core-conf-yaml\") pod \"0068c169-1560-4afb-918b-952444f11921\" (UID: \"0068c169-1560-4afb-918b-952444f11921\") " Dec 03 12:47:59 crc kubenswrapper[5052]: I1203 12:47:59.498302 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0068c169-1560-4afb-918b-952444f11921-run-httpd" (OuterVolumeSpecName: "run-httpd") pod "0068c169-1560-4afb-918b-952444f11921" (UID: "0068c169-1560-4afb-918b-952444f11921"). InnerVolumeSpecName "run-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:47:59 crc kubenswrapper[5052]: I1203 12:47:59.498849 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0068c169-1560-4afb-918b-952444f11921-log-httpd" (OuterVolumeSpecName: "log-httpd") pod "0068c169-1560-4afb-918b-952444f11921" (UID: "0068c169-1560-4afb-918b-952444f11921"). InnerVolumeSpecName "log-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:47:59 crc kubenswrapper[5052]: I1203 12:47:59.503078 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0068c169-1560-4afb-918b-952444f11921-kube-api-access-x72h8" (OuterVolumeSpecName: "kube-api-access-x72h8") pod "0068c169-1560-4afb-918b-952444f11921" (UID: "0068c169-1560-4afb-918b-952444f11921"). InnerVolumeSpecName "kube-api-access-x72h8". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:47:59 crc kubenswrapper[5052]: I1203 12:47:59.514900 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0068c169-1560-4afb-918b-952444f11921-scripts" (OuterVolumeSpecName: "scripts") pod "0068c169-1560-4afb-918b-952444f11921" (UID: "0068c169-1560-4afb-918b-952444f11921"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:47:59 crc kubenswrapper[5052]: I1203 12:47:59.568734 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0068c169-1560-4afb-918b-952444f11921-sg-core-conf-yaml" (OuterVolumeSpecName: "sg-core-conf-yaml") pod "0068c169-1560-4afb-918b-952444f11921" (UID: "0068c169-1560-4afb-918b-952444f11921"). InnerVolumeSpecName "sg-core-conf-yaml". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:47:59 crc kubenswrapper[5052]: I1203 12:47:59.596001 5052 reconciler_common.go:293] "Volume detached for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/0068c169-1560-4afb-918b-952444f11921-run-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:47:59 crc kubenswrapper[5052]: I1203 12:47:59.596782 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-x72h8\" (UniqueName: \"kubernetes.io/projected/0068c169-1560-4afb-918b-952444f11921-kube-api-access-x72h8\") on node \"crc\" DevicePath \"\"" Dec 03 12:47:59 crc kubenswrapper[5052]: I1203 12:47:59.596858 5052 reconciler_common.go:293] "Volume detached for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/0068c169-1560-4afb-918b-952444f11921-sg-core-conf-yaml\") on node \"crc\" DevicePath \"\"" Dec 03 12:47:59 crc kubenswrapper[5052]: I1203 12:47:59.596937 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/0068c169-1560-4afb-918b-952444f11921-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:47:59 crc kubenswrapper[5052]: I1203 12:47:59.597002 5052 reconciler_common.go:293] "Volume detached for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/0068c169-1560-4afb-918b-952444f11921-log-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:47:59 crc kubenswrapper[5052]: I1203 12:47:59.599809 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0068c169-1560-4afb-918b-952444f11921-ceilometer-tls-certs" (OuterVolumeSpecName: "ceilometer-tls-certs") pod "0068c169-1560-4afb-918b-952444f11921" (UID: "0068c169-1560-4afb-918b-952444f11921"). InnerVolumeSpecName "ceilometer-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:47:59 crc kubenswrapper[5052]: I1203 12:47:59.611747 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0068c169-1560-4afb-918b-952444f11921-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "0068c169-1560-4afb-918b-952444f11921" (UID: "0068c169-1560-4afb-918b-952444f11921"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:47:59 crc kubenswrapper[5052]: I1203 12:47:59.660635 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0068c169-1560-4afb-918b-952444f11921-config-data" (OuterVolumeSpecName: "config-data") pod "0068c169-1560-4afb-918b-952444f11921" (UID: "0068c169-1560-4afb-918b-952444f11921"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:47:59 crc kubenswrapper[5052]: I1203 12:47:59.699151 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0068c169-1560-4afb-918b-952444f11921-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:47:59 crc kubenswrapper[5052]: I1203 12:47:59.699179 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/0068c169-1560-4afb-918b-952444f11921-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:47:59 crc kubenswrapper[5052]: I1203 12:47:59.699190 5052 reconciler_common.go:293] "Volume detached for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/0068c169-1560-4afb-918b-952444f11921-ceilometer-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:47:59 crc kubenswrapper[5052]: I1203 12:47:59.985331 5052 generic.go:334] "Generic (PLEG): container finished" podID="0068c169-1560-4afb-918b-952444f11921" containerID="ee710ca8fa0188ffa62dbf27204f55233993aeb05b3f79709c5ef0748fbb7339" exitCode=0 Dec 03 12:47:59 crc kubenswrapper[5052]: I1203 12:47:59.985391 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:47:59 crc kubenswrapper[5052]: I1203 12:47:59.985381 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"0068c169-1560-4afb-918b-952444f11921","Type":"ContainerDied","Data":"ee710ca8fa0188ffa62dbf27204f55233993aeb05b3f79709c5ef0748fbb7339"} Dec 03 12:47:59 crc kubenswrapper[5052]: I1203 12:47:59.986093 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"0068c169-1560-4afb-918b-952444f11921","Type":"ContainerDied","Data":"645f90bba79889f7aee955f4c90f199e56090fac12aa7016a046614c3ca497fe"} Dec 03 12:47:59 crc kubenswrapper[5052]: I1203 12:47:59.986183 5052 scope.go:117] "RemoveContainer" containerID="a258638afb610e911ce997d085715008b91cd8f4631d20d5dbcd5ab4b26cdb22" Dec 03 12:48:00 crc kubenswrapper[5052]: I1203 12:48:00.025444 5052 scope.go:117] "RemoveContainer" containerID="cc4689adb2d0f67123301989da14c8be70ee0b36ea8974223f0120e1d748cc86" Dec 03 12:48:00 crc kubenswrapper[5052]: I1203 12:48:00.039605 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:48:00 crc kubenswrapper[5052]: I1203 12:48:00.056108 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:48:00 crc kubenswrapper[5052]: I1203 12:48:00.089417 5052 scope.go:117] "RemoveContainer" containerID="ee710ca8fa0188ffa62dbf27204f55233993aeb05b3f79709c5ef0748fbb7339" Dec 03 12:48:00 crc kubenswrapper[5052]: I1203 12:48:00.099824 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:48:00 crc kubenswrapper[5052]: E1203 12:48:00.105155 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="0068c169-1560-4afb-918b-952444f11921" containerName="ceilometer-notification-agent" Dec 03 12:48:00 crc kubenswrapper[5052]: I1203 12:48:00.105199 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="0068c169-1560-4afb-918b-952444f11921" containerName="ceilometer-notification-agent" Dec 03 12:48:00 crc kubenswrapper[5052]: E1203 12:48:00.105234 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="0068c169-1560-4afb-918b-952444f11921" containerName="ceilometer-central-agent" Dec 03 12:48:00 crc kubenswrapper[5052]: I1203 12:48:00.105247 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="0068c169-1560-4afb-918b-952444f11921" containerName="ceilometer-central-agent" Dec 03 12:48:00 crc kubenswrapper[5052]: E1203 12:48:00.105282 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="0068c169-1560-4afb-918b-952444f11921" containerName="sg-core" Dec 03 12:48:00 crc kubenswrapper[5052]: I1203 12:48:00.105290 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="0068c169-1560-4afb-918b-952444f11921" containerName="sg-core" Dec 03 12:48:00 crc kubenswrapper[5052]: E1203 12:48:00.105324 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="0068c169-1560-4afb-918b-952444f11921" containerName="proxy-httpd" Dec 03 12:48:00 crc kubenswrapper[5052]: I1203 12:48:00.105331 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="0068c169-1560-4afb-918b-952444f11921" containerName="proxy-httpd" Dec 03 12:48:00 crc kubenswrapper[5052]: I1203 12:48:00.106215 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="0068c169-1560-4afb-918b-952444f11921" containerName="proxy-httpd" Dec 03 12:48:00 crc kubenswrapper[5052]: I1203 12:48:00.106257 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="0068c169-1560-4afb-918b-952444f11921" containerName="ceilometer-central-agent" Dec 03 12:48:00 crc kubenswrapper[5052]: I1203 12:48:00.106283 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="0068c169-1560-4afb-918b-952444f11921" containerName="ceilometer-notification-agent" Dec 03 12:48:00 crc kubenswrapper[5052]: I1203 12:48:00.106301 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="0068c169-1560-4afb-918b-952444f11921" containerName="sg-core" Dec 03 12:48:00 crc kubenswrapper[5052]: I1203 12:48:00.110836 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:48:00 crc kubenswrapper[5052]: I1203 12:48:00.114747 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:48:00 crc kubenswrapper[5052]: I1203 12:48:00.121826 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-config-data" Dec 03 12:48:00 crc kubenswrapper[5052]: I1203 12:48:00.123748 5052 scope.go:117] "RemoveContainer" containerID="8640584ba053d7aa82893be1329fb1cb047434c76621e6cdd0181fc1cc14fcee" Dec 03 12:48:00 crc kubenswrapper[5052]: I1203 12:48:00.124174 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-ceilometer-internal-svc" Dec 03 12:48:00 crc kubenswrapper[5052]: I1203 12:48:00.124434 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-scripts" Dec 03 12:48:00 crc kubenswrapper[5052]: I1203 12:48:00.146063 5052 scope.go:117] "RemoveContainer" containerID="a258638afb610e911ce997d085715008b91cd8f4631d20d5dbcd5ab4b26cdb22" Dec 03 12:48:00 crc kubenswrapper[5052]: E1203 12:48:00.146547 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a258638afb610e911ce997d085715008b91cd8f4631d20d5dbcd5ab4b26cdb22\": container with ID starting with a258638afb610e911ce997d085715008b91cd8f4631d20d5dbcd5ab4b26cdb22 not found: ID does not exist" containerID="a258638afb610e911ce997d085715008b91cd8f4631d20d5dbcd5ab4b26cdb22" Dec 03 12:48:00 crc kubenswrapper[5052]: I1203 12:48:00.146583 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a258638afb610e911ce997d085715008b91cd8f4631d20d5dbcd5ab4b26cdb22"} err="failed to get container status \"a258638afb610e911ce997d085715008b91cd8f4631d20d5dbcd5ab4b26cdb22\": rpc error: code = NotFound desc = could not find container \"a258638afb610e911ce997d085715008b91cd8f4631d20d5dbcd5ab4b26cdb22\": container with ID starting with a258638afb610e911ce997d085715008b91cd8f4631d20d5dbcd5ab4b26cdb22 not found: ID does not exist" Dec 03 12:48:00 crc kubenswrapper[5052]: I1203 12:48:00.146608 5052 scope.go:117] "RemoveContainer" containerID="cc4689adb2d0f67123301989da14c8be70ee0b36ea8974223f0120e1d748cc86" Dec 03 12:48:00 crc kubenswrapper[5052]: E1203 12:48:00.146841 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cc4689adb2d0f67123301989da14c8be70ee0b36ea8974223f0120e1d748cc86\": container with ID starting with cc4689adb2d0f67123301989da14c8be70ee0b36ea8974223f0120e1d748cc86 not found: ID does not exist" containerID="cc4689adb2d0f67123301989da14c8be70ee0b36ea8974223f0120e1d748cc86" Dec 03 12:48:00 crc kubenswrapper[5052]: I1203 12:48:00.146865 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cc4689adb2d0f67123301989da14c8be70ee0b36ea8974223f0120e1d748cc86"} err="failed to get container status \"cc4689adb2d0f67123301989da14c8be70ee0b36ea8974223f0120e1d748cc86\": rpc error: code = NotFound desc = could not find container \"cc4689adb2d0f67123301989da14c8be70ee0b36ea8974223f0120e1d748cc86\": container with ID starting with cc4689adb2d0f67123301989da14c8be70ee0b36ea8974223f0120e1d748cc86 not found: ID does not exist" Dec 03 12:48:00 crc kubenswrapper[5052]: I1203 12:48:00.146880 5052 scope.go:117] "RemoveContainer" containerID="ee710ca8fa0188ffa62dbf27204f55233993aeb05b3f79709c5ef0748fbb7339" Dec 03 12:48:00 crc kubenswrapper[5052]: E1203 12:48:00.147183 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ee710ca8fa0188ffa62dbf27204f55233993aeb05b3f79709c5ef0748fbb7339\": container with ID starting with ee710ca8fa0188ffa62dbf27204f55233993aeb05b3f79709c5ef0748fbb7339 not found: ID does not exist" containerID="ee710ca8fa0188ffa62dbf27204f55233993aeb05b3f79709c5ef0748fbb7339" Dec 03 12:48:00 crc kubenswrapper[5052]: I1203 12:48:00.147207 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ee710ca8fa0188ffa62dbf27204f55233993aeb05b3f79709c5ef0748fbb7339"} err="failed to get container status \"ee710ca8fa0188ffa62dbf27204f55233993aeb05b3f79709c5ef0748fbb7339\": rpc error: code = NotFound desc = could not find container \"ee710ca8fa0188ffa62dbf27204f55233993aeb05b3f79709c5ef0748fbb7339\": container with ID starting with ee710ca8fa0188ffa62dbf27204f55233993aeb05b3f79709c5ef0748fbb7339 not found: ID does not exist" Dec 03 12:48:00 crc kubenswrapper[5052]: I1203 12:48:00.147222 5052 scope.go:117] "RemoveContainer" containerID="8640584ba053d7aa82893be1329fb1cb047434c76621e6cdd0181fc1cc14fcee" Dec 03 12:48:00 crc kubenswrapper[5052]: E1203 12:48:00.147629 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8640584ba053d7aa82893be1329fb1cb047434c76621e6cdd0181fc1cc14fcee\": container with ID starting with 8640584ba053d7aa82893be1329fb1cb047434c76621e6cdd0181fc1cc14fcee not found: ID does not exist" containerID="8640584ba053d7aa82893be1329fb1cb047434c76621e6cdd0181fc1cc14fcee" Dec 03 12:48:00 crc kubenswrapper[5052]: I1203 12:48:00.147655 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8640584ba053d7aa82893be1329fb1cb047434c76621e6cdd0181fc1cc14fcee"} err="failed to get container status \"8640584ba053d7aa82893be1329fb1cb047434c76621e6cdd0181fc1cc14fcee\": rpc error: code = NotFound desc = could not find container \"8640584ba053d7aa82893be1329fb1cb047434c76621e6cdd0181fc1cc14fcee\": container with ID starting with 8640584ba053d7aa82893be1329fb1cb047434c76621e6cdd0181fc1cc14fcee not found: ID does not exist" Dec 03 12:48:00 crc kubenswrapper[5052]: I1203 12:48:00.209034 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a69bfb18-1267-4383-b51a-5e0eba70d1bc-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"a69bfb18-1267-4383-b51a-5e0eba70d1bc\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:48:00 crc kubenswrapper[5052]: I1203 12:48:00.209119 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b6f8l\" (UniqueName: \"kubernetes.io/projected/a69bfb18-1267-4383-b51a-5e0eba70d1bc-kube-api-access-b6f8l\") pod \"ceilometer-0\" (UID: \"a69bfb18-1267-4383-b51a-5e0eba70d1bc\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:48:00 crc kubenswrapper[5052]: I1203 12:48:00.209146 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a69bfb18-1267-4383-b51a-5e0eba70d1bc-config-data\") pod \"ceilometer-0\" (UID: \"a69bfb18-1267-4383-b51a-5e0eba70d1bc\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:48:00 crc kubenswrapper[5052]: I1203 12:48:00.209207 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/a69bfb18-1267-4383-b51a-5e0eba70d1bc-ceilometer-tls-certs\") pod \"ceilometer-0\" (UID: \"a69bfb18-1267-4383-b51a-5e0eba70d1bc\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:48:00 crc kubenswrapper[5052]: I1203 12:48:00.209227 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/a69bfb18-1267-4383-b51a-5e0eba70d1bc-scripts\") pod \"ceilometer-0\" (UID: \"a69bfb18-1267-4383-b51a-5e0eba70d1bc\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:48:00 crc kubenswrapper[5052]: I1203 12:48:00.209258 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/a69bfb18-1267-4383-b51a-5e0eba70d1bc-run-httpd\") pod \"ceilometer-0\" (UID: \"a69bfb18-1267-4383-b51a-5e0eba70d1bc\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:48:00 crc kubenswrapper[5052]: I1203 12:48:00.209327 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/a69bfb18-1267-4383-b51a-5e0eba70d1bc-log-httpd\") pod \"ceilometer-0\" (UID: \"a69bfb18-1267-4383-b51a-5e0eba70d1bc\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:48:00 crc kubenswrapper[5052]: I1203 12:48:00.209350 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/a69bfb18-1267-4383-b51a-5e0eba70d1bc-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"a69bfb18-1267-4383-b51a-5e0eba70d1bc\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:48:00 crc kubenswrapper[5052]: I1203 12:48:00.310502 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/a69bfb18-1267-4383-b51a-5e0eba70d1bc-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"a69bfb18-1267-4383-b51a-5e0eba70d1bc\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:48:00 crc kubenswrapper[5052]: I1203 12:48:00.310616 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a69bfb18-1267-4383-b51a-5e0eba70d1bc-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"a69bfb18-1267-4383-b51a-5e0eba70d1bc\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:48:00 crc kubenswrapper[5052]: I1203 12:48:00.310689 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-b6f8l\" (UniqueName: \"kubernetes.io/projected/a69bfb18-1267-4383-b51a-5e0eba70d1bc-kube-api-access-b6f8l\") pod \"ceilometer-0\" (UID: \"a69bfb18-1267-4383-b51a-5e0eba70d1bc\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:48:00 crc kubenswrapper[5052]: I1203 12:48:00.310729 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a69bfb18-1267-4383-b51a-5e0eba70d1bc-config-data\") pod \"ceilometer-0\" (UID: \"a69bfb18-1267-4383-b51a-5e0eba70d1bc\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:48:00 crc kubenswrapper[5052]: I1203 12:48:00.310797 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/a69bfb18-1267-4383-b51a-5e0eba70d1bc-scripts\") pod \"ceilometer-0\" (UID: \"a69bfb18-1267-4383-b51a-5e0eba70d1bc\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:48:00 crc kubenswrapper[5052]: I1203 12:48:00.310813 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/a69bfb18-1267-4383-b51a-5e0eba70d1bc-ceilometer-tls-certs\") pod \"ceilometer-0\" (UID: \"a69bfb18-1267-4383-b51a-5e0eba70d1bc\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:48:00 crc kubenswrapper[5052]: I1203 12:48:00.310836 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/a69bfb18-1267-4383-b51a-5e0eba70d1bc-run-httpd\") pod \"ceilometer-0\" (UID: \"a69bfb18-1267-4383-b51a-5e0eba70d1bc\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:48:00 crc kubenswrapper[5052]: I1203 12:48:00.310890 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/a69bfb18-1267-4383-b51a-5e0eba70d1bc-log-httpd\") pod \"ceilometer-0\" (UID: \"a69bfb18-1267-4383-b51a-5e0eba70d1bc\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:48:00 crc kubenswrapper[5052]: I1203 12:48:00.311358 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/a69bfb18-1267-4383-b51a-5e0eba70d1bc-log-httpd\") pod \"ceilometer-0\" (UID: \"a69bfb18-1267-4383-b51a-5e0eba70d1bc\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:48:00 crc kubenswrapper[5052]: I1203 12:48:00.312819 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/a69bfb18-1267-4383-b51a-5e0eba70d1bc-run-httpd\") pod \"ceilometer-0\" (UID: \"a69bfb18-1267-4383-b51a-5e0eba70d1bc\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:48:00 crc kubenswrapper[5052]: I1203 12:48:00.315763 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a69bfb18-1267-4383-b51a-5e0eba70d1bc-config-data\") pod \"ceilometer-0\" (UID: \"a69bfb18-1267-4383-b51a-5e0eba70d1bc\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:48:00 crc kubenswrapper[5052]: I1203 12:48:00.315873 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/a69bfb18-1267-4383-b51a-5e0eba70d1bc-ceilometer-tls-certs\") pod \"ceilometer-0\" (UID: \"a69bfb18-1267-4383-b51a-5e0eba70d1bc\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:48:00 crc kubenswrapper[5052]: I1203 12:48:00.316326 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/a69bfb18-1267-4383-b51a-5e0eba70d1bc-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"a69bfb18-1267-4383-b51a-5e0eba70d1bc\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:48:00 crc kubenswrapper[5052]: I1203 12:48:00.323939 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a69bfb18-1267-4383-b51a-5e0eba70d1bc-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"a69bfb18-1267-4383-b51a-5e0eba70d1bc\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:48:00 crc kubenswrapper[5052]: I1203 12:48:00.331584 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/a69bfb18-1267-4383-b51a-5e0eba70d1bc-scripts\") pod \"ceilometer-0\" (UID: \"a69bfb18-1267-4383-b51a-5e0eba70d1bc\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:48:00 crc kubenswrapper[5052]: I1203 12:48:00.332087 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-b6f8l\" (UniqueName: \"kubernetes.io/projected/a69bfb18-1267-4383-b51a-5e0eba70d1bc-kube-api-access-b6f8l\") pod \"ceilometer-0\" (UID: \"a69bfb18-1267-4383-b51a-5e0eba70d1bc\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:48:00 crc kubenswrapper[5052]: I1203 12:48:00.431315 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:48:00 crc kubenswrapper[5052]: I1203 12:48:00.673244 5052 scope.go:117] "RemoveContainer" containerID="2aed1c760a326f3d008799e902b1093313df3764a57c879f0b3686410127d37d" Dec 03 12:48:00 crc kubenswrapper[5052]: E1203 12:48:00.674243 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 12:48:00 crc kubenswrapper[5052]: I1203 12:48:00.926833 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:48:00 crc kubenswrapper[5052]: W1203 12:48:00.927224 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda69bfb18_1267_4383_b51a_5e0eba70d1bc.slice/crio-41ead924f537c723d851acf33ed51e22a3dc127f908a763b554398d819505739 WatchSource:0}: Error finding container 41ead924f537c723d851acf33ed51e22a3dc127f908a763b554398d819505739: Status 404 returned error can't find the container with id 41ead924f537c723d851acf33ed51e22a3dc127f908a763b554398d819505739 Dec 03 12:48:00 crc kubenswrapper[5052]: I1203 12:48:00.997271 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"a69bfb18-1267-4383-b51a-5e0eba70d1bc","Type":"ContainerStarted","Data":"41ead924f537c723d851acf33ed51e22a3dc127f908a763b554398d819505739"} Dec 03 12:48:01 crc kubenswrapper[5052]: I1203 12:48:01.255963 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:48:01 crc kubenswrapper[5052]: I1203 12:48:01.257977 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:48:01 crc kubenswrapper[5052]: I1203 12:48:01.719538 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0068c169-1560-4afb-918b-952444f11921" path="/var/lib/kubelet/pods/0068c169-1560-4afb-918b-952444f11921/volumes" Dec 03 12:48:02 crc kubenswrapper[5052]: I1203 12:48:02.020905 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"a69bfb18-1267-4383-b51a-5e0eba70d1bc","Type":"ContainerStarted","Data":"4f74304b5716b547aed360588b0a9be626bd9def7e57532f38549b982366ac6f"} Dec 03 12:48:02 crc kubenswrapper[5052]: I1203 12:48:02.273722 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-api-0" podUID="6c8ad676-e7dd-479c-8d22-f546dc5e50af" containerName="nova-api-log" probeResult="failure" output="Get \"https://10.217.0.122:8774/\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Dec 03 12:48:02 crc kubenswrapper[5052]: I1203 12:48:02.273759 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-api-0" podUID="6c8ad676-e7dd-479c-8d22-f546dc5e50af" containerName="nova-api-api" probeResult="failure" output="Get \"https://10.217.0.122:8774/\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Dec 03 12:48:02 crc kubenswrapper[5052]: I1203 12:48:02.673733 5052 scope.go:117] "RemoveContainer" containerID="b1f27aab2d1cce51772c0ce99bd8dfc4afd0620b38b776e6b968d9114105f165" Dec 03 12:48:02 crc kubenswrapper[5052]: I1203 12:48:02.673853 5052 scope.go:117] "RemoveContainer" containerID="1fdbade92a13691b593283a5b263382dfec190075f1201c77fff78a7be754487" Dec 03 12:48:02 crc kubenswrapper[5052]: E1203 12:48:02.674040 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"nova-scheduler-scheduler\" with CrashLoopBackOff: \"back-off 40s restarting failed container=nova-scheduler-scheduler pod=nova-scheduler-0_openstack-kuttl-tests(81709983-1fd0-4446-bbbf-fb94653bb3d2)\"" pod="openstack-kuttl-tests/nova-scheduler-0" podUID="81709983-1fd0-4446-bbbf-fb94653bb3d2" Dec 03 12:48:02 crc kubenswrapper[5052]: I1203 12:48:02.690831 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/swift-proxy-78f78b88b6-fmjvp" Dec 03 12:48:02 crc kubenswrapper[5052]: I1203 12:48:02.700686 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/swift-proxy-78f78b88b6-fmjvp" Dec 03 12:48:02 crc kubenswrapper[5052]: I1203 12:48:02.825548 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/swift-proxy-dd4cf57c8-x4c45"] Dec 03 12:48:02 crc kubenswrapper[5052]: I1203 12:48:02.826206 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-proxy-dd4cf57c8-x4c45" podUID="242be027-6461-4aea-ae7c-e52ca996b2ea" containerName="proxy-httpd" containerID="cri-o://b0a1441f36be1742bcb32bdac564816d92f0341aece1f5d7b3bd6920cff1543f" gracePeriod=30 Dec 03 12:48:02 crc kubenswrapper[5052]: I1203 12:48:02.826321 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-proxy-dd4cf57c8-x4c45" podUID="242be027-6461-4aea-ae7c-e52ca996b2ea" containerName="proxy-server" containerID="cri-o://29b8aa0fef682b25addf6b174c213d6db9760bd388dc211dfc025f115b0ab6ed" gracePeriod=30 Dec 03 12:48:03 crc kubenswrapper[5052]: I1203 12:48:03.031926 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-conductor-0" event={"ID":"6d653682-52e2-4940-af66-633475bfe09a","Type":"ContainerStarted","Data":"2b5c5249476f9fa6344558a0eabe96f98d744f43f1cd4faf1d66ffa9c49d4941"} Dec 03 12:48:03 crc kubenswrapper[5052]: I1203 12:48:03.032183 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:48:03 crc kubenswrapper[5052]: I1203 12:48:03.044090 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"a69bfb18-1267-4383-b51a-5e0eba70d1bc","Type":"ContainerStarted","Data":"b1a655b5c3d0ddb68e291790261b40d699c5471d8a606da7a88e93dbced9a7f8"} Dec 03 12:48:04 crc kubenswrapper[5052]: I1203 12:48:04.099837 5052 generic.go:334] "Generic (PLEG): container finished" podID="242be027-6461-4aea-ae7c-e52ca996b2ea" containerID="29b8aa0fef682b25addf6b174c213d6db9760bd388dc211dfc025f115b0ab6ed" exitCode=0 Dec 03 12:48:04 crc kubenswrapper[5052]: I1203 12:48:04.100322 5052 generic.go:334] "Generic (PLEG): container finished" podID="242be027-6461-4aea-ae7c-e52ca996b2ea" containerID="b0a1441f36be1742bcb32bdac564816d92f0341aece1f5d7b3bd6920cff1543f" exitCode=0 Dec 03 12:48:04 crc kubenswrapper[5052]: I1203 12:48:04.101287 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-proxy-dd4cf57c8-x4c45" event={"ID":"242be027-6461-4aea-ae7c-e52ca996b2ea","Type":"ContainerDied","Data":"29b8aa0fef682b25addf6b174c213d6db9760bd388dc211dfc025f115b0ab6ed"} Dec 03 12:48:04 crc kubenswrapper[5052]: I1203 12:48:04.101312 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-proxy-dd4cf57c8-x4c45" event={"ID":"242be027-6461-4aea-ae7c-e52ca996b2ea","Type":"ContainerDied","Data":"b0a1441f36be1742bcb32bdac564816d92f0341aece1f5d7b3bd6920cff1543f"} Dec 03 12:48:04 crc kubenswrapper[5052]: I1203 12:48:04.939200 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-proxy-dd4cf57c8-x4c45" Dec 03 12:48:05 crc kubenswrapper[5052]: I1203 12:48:05.037416 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/242be027-6461-4aea-ae7c-e52ca996b2ea-etc-swift\") pod \"242be027-6461-4aea-ae7c-e52ca996b2ea\" (UID: \"242be027-6461-4aea-ae7c-e52ca996b2ea\") " Dec 03 12:48:05 crc kubenswrapper[5052]: I1203 12:48:05.037955 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/242be027-6461-4aea-ae7c-e52ca996b2ea-combined-ca-bundle\") pod \"242be027-6461-4aea-ae7c-e52ca996b2ea\" (UID: \"242be027-6461-4aea-ae7c-e52ca996b2ea\") " Dec 03 12:48:05 crc kubenswrapper[5052]: I1203 12:48:05.038597 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/242be027-6461-4aea-ae7c-e52ca996b2ea-config-data\") pod \"242be027-6461-4aea-ae7c-e52ca996b2ea\" (UID: \"242be027-6461-4aea-ae7c-e52ca996b2ea\") " Dec 03 12:48:05 crc kubenswrapper[5052]: I1203 12:48:05.038757 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/242be027-6461-4aea-ae7c-e52ca996b2ea-run-httpd\") pod \"242be027-6461-4aea-ae7c-e52ca996b2ea\" (UID: \"242be027-6461-4aea-ae7c-e52ca996b2ea\") " Dec 03 12:48:05 crc kubenswrapper[5052]: I1203 12:48:05.038974 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/242be027-6461-4aea-ae7c-e52ca996b2ea-internal-tls-certs\") pod \"242be027-6461-4aea-ae7c-e52ca996b2ea\" (UID: \"242be027-6461-4aea-ae7c-e52ca996b2ea\") " Dec 03 12:48:05 crc kubenswrapper[5052]: I1203 12:48:05.039083 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/242be027-6461-4aea-ae7c-e52ca996b2ea-run-httpd" (OuterVolumeSpecName: "run-httpd") pod "242be027-6461-4aea-ae7c-e52ca996b2ea" (UID: "242be027-6461-4aea-ae7c-e52ca996b2ea"). InnerVolumeSpecName "run-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:48:05 crc kubenswrapper[5052]: I1203 12:48:05.039214 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gp8vs\" (UniqueName: \"kubernetes.io/projected/242be027-6461-4aea-ae7c-e52ca996b2ea-kube-api-access-gp8vs\") pod \"242be027-6461-4aea-ae7c-e52ca996b2ea\" (UID: \"242be027-6461-4aea-ae7c-e52ca996b2ea\") " Dec 03 12:48:05 crc kubenswrapper[5052]: I1203 12:48:05.039347 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/242be027-6461-4aea-ae7c-e52ca996b2ea-public-tls-certs\") pod \"242be027-6461-4aea-ae7c-e52ca996b2ea\" (UID: \"242be027-6461-4aea-ae7c-e52ca996b2ea\") " Dec 03 12:48:05 crc kubenswrapper[5052]: I1203 12:48:05.039450 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/242be027-6461-4aea-ae7c-e52ca996b2ea-log-httpd\") pod \"242be027-6461-4aea-ae7c-e52ca996b2ea\" (UID: \"242be027-6461-4aea-ae7c-e52ca996b2ea\") " Dec 03 12:48:05 crc kubenswrapper[5052]: I1203 12:48:05.040226 5052 reconciler_common.go:293] "Volume detached for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/242be027-6461-4aea-ae7c-e52ca996b2ea-run-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:48:05 crc kubenswrapper[5052]: I1203 12:48:05.040652 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/242be027-6461-4aea-ae7c-e52ca996b2ea-log-httpd" (OuterVolumeSpecName: "log-httpd") pod "242be027-6461-4aea-ae7c-e52ca996b2ea" (UID: "242be027-6461-4aea-ae7c-e52ca996b2ea"). InnerVolumeSpecName "log-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:48:05 crc kubenswrapper[5052]: I1203 12:48:05.045756 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/242be027-6461-4aea-ae7c-e52ca996b2ea-kube-api-access-gp8vs" (OuterVolumeSpecName: "kube-api-access-gp8vs") pod "242be027-6461-4aea-ae7c-e52ca996b2ea" (UID: "242be027-6461-4aea-ae7c-e52ca996b2ea"). InnerVolumeSpecName "kube-api-access-gp8vs". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:48:05 crc kubenswrapper[5052]: I1203 12:48:05.060503 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/242be027-6461-4aea-ae7c-e52ca996b2ea-etc-swift" (OuterVolumeSpecName: "etc-swift") pod "242be027-6461-4aea-ae7c-e52ca996b2ea" (UID: "242be027-6461-4aea-ae7c-e52ca996b2ea"). InnerVolumeSpecName "etc-swift". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:48:05 crc kubenswrapper[5052]: I1203 12:48:05.121314 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/242be027-6461-4aea-ae7c-e52ca996b2ea-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "242be027-6461-4aea-ae7c-e52ca996b2ea" (UID: "242be027-6461-4aea-ae7c-e52ca996b2ea"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:48:05 crc kubenswrapper[5052]: I1203 12:48:05.121492 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/242be027-6461-4aea-ae7c-e52ca996b2ea-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "242be027-6461-4aea-ae7c-e52ca996b2ea" (UID: "242be027-6461-4aea-ae7c-e52ca996b2ea"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:48:05 crc kubenswrapper[5052]: I1203 12:48:05.126045 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-proxy-dd4cf57c8-x4c45" event={"ID":"242be027-6461-4aea-ae7c-e52ca996b2ea","Type":"ContainerDied","Data":"25209c72330b0e84607f594e854354f3b7d2de1eddba6ea5df0a4234371e4441"} Dec 03 12:48:05 crc kubenswrapper[5052]: I1203 12:48:05.126103 5052 scope.go:117] "RemoveContainer" containerID="29b8aa0fef682b25addf6b174c213d6db9760bd388dc211dfc025f115b0ab6ed" Dec 03 12:48:05 crc kubenswrapper[5052]: I1203 12:48:05.126266 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-proxy-dd4cf57c8-x4c45" Dec 03 12:48:05 crc kubenswrapper[5052]: I1203 12:48:05.128998 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/242be027-6461-4aea-ae7c-e52ca996b2ea-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "242be027-6461-4aea-ae7c-e52ca996b2ea" (UID: "242be027-6461-4aea-ae7c-e52ca996b2ea"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:48:05 crc kubenswrapper[5052]: I1203 12:48:05.142760 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/242be027-6461-4aea-ae7c-e52ca996b2ea-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:48:05 crc kubenswrapper[5052]: I1203 12:48:05.142800 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/242be027-6461-4aea-ae7c-e52ca996b2ea-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:48:05 crc kubenswrapper[5052]: I1203 12:48:05.142811 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-gp8vs\" (UniqueName: \"kubernetes.io/projected/242be027-6461-4aea-ae7c-e52ca996b2ea-kube-api-access-gp8vs\") on node \"crc\" DevicePath \"\"" Dec 03 12:48:05 crc kubenswrapper[5052]: I1203 12:48:05.142839 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/242be027-6461-4aea-ae7c-e52ca996b2ea-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:48:05 crc kubenswrapper[5052]: I1203 12:48:05.142853 5052 reconciler_common.go:293] "Volume detached for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/242be027-6461-4aea-ae7c-e52ca996b2ea-log-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:48:05 crc kubenswrapper[5052]: I1203 12:48:05.142878 5052 reconciler_common.go:293] "Volume detached for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/242be027-6461-4aea-ae7c-e52ca996b2ea-etc-swift\") on node \"crc\" DevicePath \"\"" Dec 03 12:48:05 crc kubenswrapper[5052]: I1203 12:48:05.186679 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/242be027-6461-4aea-ae7c-e52ca996b2ea-config-data" (OuterVolumeSpecName: "config-data") pod "242be027-6461-4aea-ae7c-e52ca996b2ea" (UID: "242be027-6461-4aea-ae7c-e52ca996b2ea"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:48:05 crc kubenswrapper[5052]: I1203 12:48:05.244186 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/242be027-6461-4aea-ae7c-e52ca996b2ea-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:48:05 crc kubenswrapper[5052]: I1203 12:48:05.247433 5052 scope.go:117] "RemoveContainer" containerID="b0a1441f36be1742bcb32bdac564816d92f0341aece1f5d7b3bd6920cff1543f" Dec 03 12:48:05 crc kubenswrapper[5052]: I1203 12:48:05.462579 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/swift-proxy-dd4cf57c8-x4c45"] Dec 03 12:48:05 crc kubenswrapper[5052]: I1203 12:48:05.474709 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/swift-proxy-dd4cf57c8-x4c45"] Dec 03 12:48:05 crc kubenswrapper[5052]: I1203 12:48:05.687833 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="242be027-6461-4aea-ae7c-e52ca996b2ea" path="/var/lib/kubelet/pods/242be027-6461-4aea-ae7c-e52ca996b2ea/volumes" Dec 03 12:48:06 crc kubenswrapper[5052]: I1203 12:48:06.668620 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openshift-marketplace/redhat-operators-9nx4s"] Dec 03 12:48:06 crc kubenswrapper[5052]: E1203 12:48:06.669426 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="242be027-6461-4aea-ae7c-e52ca996b2ea" containerName="proxy-server" Dec 03 12:48:06 crc kubenswrapper[5052]: I1203 12:48:06.669447 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="242be027-6461-4aea-ae7c-e52ca996b2ea" containerName="proxy-server" Dec 03 12:48:06 crc kubenswrapper[5052]: E1203 12:48:06.669468 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="242be027-6461-4aea-ae7c-e52ca996b2ea" containerName="proxy-httpd" Dec 03 12:48:06 crc kubenswrapper[5052]: I1203 12:48:06.669477 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="242be027-6461-4aea-ae7c-e52ca996b2ea" containerName="proxy-httpd" Dec 03 12:48:06 crc kubenswrapper[5052]: I1203 12:48:06.669746 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="242be027-6461-4aea-ae7c-e52ca996b2ea" containerName="proxy-server" Dec 03 12:48:06 crc kubenswrapper[5052]: I1203 12:48:06.669763 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="242be027-6461-4aea-ae7c-e52ca996b2ea" containerName="proxy-httpd" Dec 03 12:48:06 crc kubenswrapper[5052]: I1203 12:48:06.671287 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/redhat-operators-9nx4s" Dec 03 12:48:06 crc kubenswrapper[5052]: I1203 12:48:06.682926 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/redhat-operators-9nx4s"] Dec 03 12:48:06 crc kubenswrapper[5052]: I1203 12:48:06.773541 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qsdcz\" (UniqueName: \"kubernetes.io/projected/b7cd9737-53b4-414c-a51b-85e21e6f4aa7-kube-api-access-qsdcz\") pod \"redhat-operators-9nx4s\" (UID: \"b7cd9737-53b4-414c-a51b-85e21e6f4aa7\") " pod="openshift-marketplace/redhat-operators-9nx4s" Dec 03 12:48:06 crc kubenswrapper[5052]: I1203 12:48:06.773656 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/b7cd9737-53b4-414c-a51b-85e21e6f4aa7-utilities\") pod \"redhat-operators-9nx4s\" (UID: \"b7cd9737-53b4-414c-a51b-85e21e6f4aa7\") " pod="openshift-marketplace/redhat-operators-9nx4s" Dec 03 12:48:06 crc kubenswrapper[5052]: I1203 12:48:06.773742 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/b7cd9737-53b4-414c-a51b-85e21e6f4aa7-catalog-content\") pod \"redhat-operators-9nx4s\" (UID: \"b7cd9737-53b4-414c-a51b-85e21e6f4aa7\") " pod="openshift-marketplace/redhat-operators-9nx4s" Dec 03 12:48:06 crc kubenswrapper[5052]: I1203 12:48:06.875678 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-qsdcz\" (UniqueName: \"kubernetes.io/projected/b7cd9737-53b4-414c-a51b-85e21e6f4aa7-kube-api-access-qsdcz\") pod \"redhat-operators-9nx4s\" (UID: \"b7cd9737-53b4-414c-a51b-85e21e6f4aa7\") " pod="openshift-marketplace/redhat-operators-9nx4s" Dec 03 12:48:06 crc kubenswrapper[5052]: I1203 12:48:06.875753 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/b7cd9737-53b4-414c-a51b-85e21e6f4aa7-utilities\") pod \"redhat-operators-9nx4s\" (UID: \"b7cd9737-53b4-414c-a51b-85e21e6f4aa7\") " pod="openshift-marketplace/redhat-operators-9nx4s" Dec 03 12:48:06 crc kubenswrapper[5052]: I1203 12:48:06.875816 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/b7cd9737-53b4-414c-a51b-85e21e6f4aa7-catalog-content\") pod \"redhat-operators-9nx4s\" (UID: \"b7cd9737-53b4-414c-a51b-85e21e6f4aa7\") " pod="openshift-marketplace/redhat-operators-9nx4s" Dec 03 12:48:06 crc kubenswrapper[5052]: I1203 12:48:06.876284 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/b7cd9737-53b4-414c-a51b-85e21e6f4aa7-utilities\") pod \"redhat-operators-9nx4s\" (UID: \"b7cd9737-53b4-414c-a51b-85e21e6f4aa7\") " pod="openshift-marketplace/redhat-operators-9nx4s" Dec 03 12:48:06 crc kubenswrapper[5052]: I1203 12:48:06.876328 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/b7cd9737-53b4-414c-a51b-85e21e6f4aa7-catalog-content\") pod \"redhat-operators-9nx4s\" (UID: \"b7cd9737-53b4-414c-a51b-85e21e6f4aa7\") " pod="openshift-marketplace/redhat-operators-9nx4s" Dec 03 12:48:06 crc kubenswrapper[5052]: I1203 12:48:06.900682 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-qsdcz\" (UniqueName: \"kubernetes.io/projected/b7cd9737-53b4-414c-a51b-85e21e6f4aa7-kube-api-access-qsdcz\") pod \"redhat-operators-9nx4s\" (UID: \"b7cd9737-53b4-414c-a51b-85e21e6f4aa7\") " pod="openshift-marketplace/redhat-operators-9nx4s" Dec 03 12:48:06 crc kubenswrapper[5052]: I1203 12:48:06.998051 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/redhat-operators-9nx4s" Dec 03 12:48:07 crc kubenswrapper[5052]: W1203 12:48:07.585888 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7cd9737_53b4_414c_a51b_85e21e6f4aa7.slice/crio-03a8be39a06733e6dedc4c99e67f09d0a920a5c87c75a639fc37b533b033c2e9 WatchSource:0}: Error finding container 03a8be39a06733e6dedc4c99e67f09d0a920a5c87c75a639fc37b533b033c2e9: Status 404 returned error can't find the container with id 03a8be39a06733e6dedc4c99e67f09d0a920a5c87c75a639fc37b533b033c2e9 Dec 03 12:48:07 crc kubenswrapper[5052]: I1203 12:48:07.593166 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/redhat-operators-9nx4s"] Dec 03 12:48:07 crc kubenswrapper[5052]: I1203 12:48:07.674941 5052 scope.go:117] "RemoveContainer" containerID="8e3764a05c784a75a82ccff0fcb9f4aaac35f5d6845cca81b14deee31318466f" Dec 03 12:48:07 crc kubenswrapper[5052]: E1203 12:48:07.678254 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"nova-cell1-conductor-conductor\" with CrashLoopBackOff: \"back-off 40s restarting failed container=nova-cell1-conductor-conductor pod=nova-cell1-conductor-0_openstack-kuttl-tests(8669832d-c47a-49bd-9491-dc7141ac6281)\"" pod="openstack-kuttl-tests/nova-cell1-conductor-0" podUID="8669832d-c47a-49bd-9491-dc7141ac6281" Dec 03 12:48:08 crc kubenswrapper[5052]: I1203 12:48:08.161711 5052 generic.go:334] "Generic (PLEG): container finished" podID="b7cd9737-53b4-414c-a51b-85e21e6f4aa7" containerID="c97e72423d1d83658766dc2e82aa17152e2bd056591379659920ae4ef1d9a268" exitCode=0 Dec 03 12:48:08 crc kubenswrapper[5052]: I1203 12:48:08.161827 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-operators-9nx4s" event={"ID":"b7cd9737-53b4-414c-a51b-85e21e6f4aa7","Type":"ContainerDied","Data":"c97e72423d1d83658766dc2e82aa17152e2bd056591379659920ae4ef1d9a268"} Dec 03 12:48:08 crc kubenswrapper[5052]: I1203 12:48:08.162055 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-operators-9nx4s" event={"ID":"b7cd9737-53b4-414c-a51b-85e21e6f4aa7","Type":"ContainerStarted","Data":"03a8be39a06733e6dedc4c99e67f09d0a920a5c87c75a639fc37b533b033c2e9"} Dec 03 12:48:08 crc kubenswrapper[5052]: I1203 12:48:08.165489 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"a69bfb18-1267-4383-b51a-5e0eba70d1bc","Type":"ContainerStarted","Data":"18022901c9e16d4c4eb5c5f284061eb245203a106525b3fb6b0192a33f4b392e"} Dec 03 12:48:08 crc kubenswrapper[5052]: I1203 12:48:08.712145 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:48:11 crc kubenswrapper[5052]: I1203 12:48:11.427095 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:48:11 crc kubenswrapper[5052]: I1203 12:48:11.427926 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:48:11 crc kubenswrapper[5052]: I1203 12:48:11.517774 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:48:11 crc kubenswrapper[5052]: I1203 12:48:11.519606 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:48:11 crc kubenswrapper[5052]: I1203 12:48:11.673769 5052 scope.go:117] "RemoveContainer" containerID="2aed1c760a326f3d008799e902b1093313df3764a57c879f0b3686410127d37d" Dec 03 12:48:11 crc kubenswrapper[5052]: E1203 12:48:11.674324 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 12:48:12 crc kubenswrapper[5052]: I1203 12:48:12.211170 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:48:12 crc kubenswrapper[5052]: I1203 12:48:12.217299 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:48:13 crc kubenswrapper[5052]: I1203 12:48:13.224113 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_neutron-68f85bdc64-dc642_95997b63-313e-4f5f-adf6-b8ae63de9fb9/neutron-api/0.log" Dec 03 12:48:13 crc kubenswrapper[5052]: I1203 12:48:13.224543 5052 generic.go:334] "Generic (PLEG): container finished" podID="95997b63-313e-4f5f-adf6-b8ae63de9fb9" containerID="65e1fdf0d5abf9142c0d5dd8fd15f412c134a90fb4aecb28999c1caba228326b" exitCode=137 Dec 03 12:48:13 crc kubenswrapper[5052]: I1203 12:48:13.224586 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-68f85bdc64-dc642" event={"ID":"95997b63-313e-4f5f-adf6-b8ae63de9fb9","Type":"ContainerDied","Data":"65e1fdf0d5abf9142c0d5dd8fd15f412c134a90fb4aecb28999c1caba228326b"} Dec 03 12:48:14 crc kubenswrapper[5052]: I1203 12:48:14.238042 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_neutron-68f85bdc64-dc642_95997b63-313e-4f5f-adf6-b8ae63de9fb9/neutron-api/0.log" Dec 03 12:48:14 crc kubenswrapper[5052]: I1203 12:48:14.238700 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-68f85bdc64-dc642" event={"ID":"95997b63-313e-4f5f-adf6-b8ae63de9fb9","Type":"ContainerDied","Data":"7c6c7d14c1f8543dac5d6115d97af202f3a8137146128bbba9e6bdda2a168009"} Dec 03 12:48:14 crc kubenswrapper[5052]: I1203 12:48:14.238727 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7c6c7d14c1f8543dac5d6115d97af202f3a8137146128bbba9e6bdda2a168009" Dec 03 12:48:14 crc kubenswrapper[5052]: I1203 12:48:14.242692 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-operators-9nx4s" event={"ID":"b7cd9737-53b4-414c-a51b-85e21e6f4aa7","Type":"ContainerStarted","Data":"1d02d8acfd7135f1ee2b76bfcdb23259e05bd05d52a412e5809afd7e88dcf52b"} Dec 03 12:48:14 crc kubenswrapper[5052]: I1203 12:48:14.257014 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_neutron-68f85bdc64-dc642_95997b63-313e-4f5f-adf6-b8ae63de9fb9/neutron-api/0.log" Dec 03 12:48:14 crc kubenswrapper[5052]: I1203 12:48:14.257100 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-68f85bdc64-dc642" Dec 03 12:48:14 crc kubenswrapper[5052]: I1203 12:48:14.362913 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/95997b63-313e-4f5f-adf6-b8ae63de9fb9-combined-ca-bundle\") pod \"95997b63-313e-4f5f-adf6-b8ae63de9fb9\" (UID: \"95997b63-313e-4f5f-adf6-b8ae63de9fb9\") " Dec 03 12:48:14 crc kubenswrapper[5052]: I1203 12:48:14.363003 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/95997b63-313e-4f5f-adf6-b8ae63de9fb9-config\") pod \"95997b63-313e-4f5f-adf6-b8ae63de9fb9\" (UID: \"95997b63-313e-4f5f-adf6-b8ae63de9fb9\") " Dec 03 12:48:14 crc kubenswrapper[5052]: I1203 12:48:14.363069 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/95997b63-313e-4f5f-adf6-b8ae63de9fb9-httpd-config\") pod \"95997b63-313e-4f5f-adf6-b8ae63de9fb9\" (UID: \"95997b63-313e-4f5f-adf6-b8ae63de9fb9\") " Dec 03 12:48:14 crc kubenswrapper[5052]: I1203 12:48:14.363130 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/95997b63-313e-4f5f-adf6-b8ae63de9fb9-ovndb-tls-certs\") pod \"95997b63-313e-4f5f-adf6-b8ae63de9fb9\" (UID: \"95997b63-313e-4f5f-adf6-b8ae63de9fb9\") " Dec 03 12:48:14 crc kubenswrapper[5052]: I1203 12:48:14.363178 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/95997b63-313e-4f5f-adf6-b8ae63de9fb9-public-tls-certs\") pod \"95997b63-313e-4f5f-adf6-b8ae63de9fb9\" (UID: \"95997b63-313e-4f5f-adf6-b8ae63de9fb9\") " Dec 03 12:48:14 crc kubenswrapper[5052]: I1203 12:48:14.363247 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cdwbz\" (UniqueName: \"kubernetes.io/projected/95997b63-313e-4f5f-adf6-b8ae63de9fb9-kube-api-access-cdwbz\") pod \"95997b63-313e-4f5f-adf6-b8ae63de9fb9\" (UID: \"95997b63-313e-4f5f-adf6-b8ae63de9fb9\") " Dec 03 12:48:14 crc kubenswrapper[5052]: I1203 12:48:14.363292 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/95997b63-313e-4f5f-adf6-b8ae63de9fb9-internal-tls-certs\") pod \"95997b63-313e-4f5f-adf6-b8ae63de9fb9\" (UID: \"95997b63-313e-4f5f-adf6-b8ae63de9fb9\") " Dec 03 12:48:14 crc kubenswrapper[5052]: I1203 12:48:14.370736 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/95997b63-313e-4f5f-adf6-b8ae63de9fb9-httpd-config" (OuterVolumeSpecName: "httpd-config") pod "95997b63-313e-4f5f-adf6-b8ae63de9fb9" (UID: "95997b63-313e-4f5f-adf6-b8ae63de9fb9"). InnerVolumeSpecName "httpd-config". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:48:14 crc kubenswrapper[5052]: I1203 12:48:14.375861 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/95997b63-313e-4f5f-adf6-b8ae63de9fb9-kube-api-access-cdwbz" (OuterVolumeSpecName: "kube-api-access-cdwbz") pod "95997b63-313e-4f5f-adf6-b8ae63de9fb9" (UID: "95997b63-313e-4f5f-adf6-b8ae63de9fb9"). InnerVolumeSpecName "kube-api-access-cdwbz". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:48:14 crc kubenswrapper[5052]: I1203 12:48:14.428061 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/95997b63-313e-4f5f-adf6-b8ae63de9fb9-config" (OuterVolumeSpecName: "config") pod "95997b63-313e-4f5f-adf6-b8ae63de9fb9" (UID: "95997b63-313e-4f5f-adf6-b8ae63de9fb9"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:48:14 crc kubenswrapper[5052]: I1203 12:48:14.435174 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/95997b63-313e-4f5f-adf6-b8ae63de9fb9-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "95997b63-313e-4f5f-adf6-b8ae63de9fb9" (UID: "95997b63-313e-4f5f-adf6-b8ae63de9fb9"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:48:14 crc kubenswrapper[5052]: I1203 12:48:14.450727 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/95997b63-313e-4f5f-adf6-b8ae63de9fb9-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "95997b63-313e-4f5f-adf6-b8ae63de9fb9" (UID: "95997b63-313e-4f5f-adf6-b8ae63de9fb9"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:48:14 crc kubenswrapper[5052]: I1203 12:48:14.459026 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/95997b63-313e-4f5f-adf6-b8ae63de9fb9-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "95997b63-313e-4f5f-adf6-b8ae63de9fb9" (UID: "95997b63-313e-4f5f-adf6-b8ae63de9fb9"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:48:14 crc kubenswrapper[5052]: I1203 12:48:14.465435 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/95997b63-313e-4f5f-adf6-b8ae63de9fb9-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:48:14 crc kubenswrapper[5052]: I1203 12:48:14.465700 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-cdwbz\" (UniqueName: \"kubernetes.io/projected/95997b63-313e-4f5f-adf6-b8ae63de9fb9-kube-api-access-cdwbz\") on node \"crc\" DevicePath \"\"" Dec 03 12:48:14 crc kubenswrapper[5052]: I1203 12:48:14.465805 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/95997b63-313e-4f5f-adf6-b8ae63de9fb9-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:48:14 crc kubenswrapper[5052]: I1203 12:48:14.465882 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/95997b63-313e-4f5f-adf6-b8ae63de9fb9-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:48:14 crc kubenswrapper[5052]: I1203 12:48:14.465958 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/95997b63-313e-4f5f-adf6-b8ae63de9fb9-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:48:14 crc kubenswrapper[5052]: I1203 12:48:14.466031 5052 reconciler_common.go:293] "Volume detached for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/95997b63-313e-4f5f-adf6-b8ae63de9fb9-httpd-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:48:14 crc kubenswrapper[5052]: I1203 12:48:14.475884 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/95997b63-313e-4f5f-adf6-b8ae63de9fb9-ovndb-tls-certs" (OuterVolumeSpecName: "ovndb-tls-certs") pod "95997b63-313e-4f5f-adf6-b8ae63de9fb9" (UID: "95997b63-313e-4f5f-adf6-b8ae63de9fb9"). InnerVolumeSpecName "ovndb-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:48:14 crc kubenswrapper[5052]: I1203 12:48:14.568485 5052 reconciler_common.go:293] "Volume detached for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/95997b63-313e-4f5f-adf6-b8ae63de9fb9-ovndb-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:48:15 crc kubenswrapper[5052]: I1203 12:48:15.254538 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"a69bfb18-1267-4383-b51a-5e0eba70d1bc","Type":"ContainerStarted","Data":"74e52eb4f69076f917a1b3ab5d18438ec4c2d708d4bc29581185ab517d52919e"} Dec 03 12:48:15 crc kubenswrapper[5052]: I1203 12:48:15.254607 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-68f85bdc64-dc642" Dec 03 12:48:15 crc kubenswrapper[5052]: I1203 12:48:15.299278 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/ceilometer-0" podStartSLOduration=2.420270482 podStartE2EDuration="15.299254728s" podCreationTimestamp="2025-12-03 12:48:00 +0000 UTC" firstStartedPulling="2025-12-03 12:48:00.930081315 +0000 UTC m=+5271.518323069" lastFinishedPulling="2025-12-03 12:48:13.809065551 +0000 UTC m=+5284.397307315" observedRunningTime="2025-12-03 12:48:15.28568429 +0000 UTC m=+5285.873926064" watchObservedRunningTime="2025-12-03 12:48:15.299254728 +0000 UTC m=+5285.887496492" Dec 03 12:48:15 crc kubenswrapper[5052]: I1203 12:48:15.320502 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/neutron-68f85bdc64-dc642"] Dec 03 12:48:15 crc kubenswrapper[5052]: I1203 12:48:15.339606 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/neutron-68f85bdc64-dc642"] Dec 03 12:48:15 crc kubenswrapper[5052]: I1203 12:48:15.685274 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="95997b63-313e-4f5f-adf6-b8ae63de9fb9" path="/var/lib/kubelet/pods/95997b63-313e-4f5f-adf6-b8ae63de9fb9/volumes" Dec 03 12:48:16 crc kubenswrapper[5052]: I1203 12:48:16.267066 5052 generic.go:334] "Generic (PLEG): container finished" podID="b7cd9737-53b4-414c-a51b-85e21e6f4aa7" containerID="1d02d8acfd7135f1ee2b76bfcdb23259e05bd05d52a412e5809afd7e88dcf52b" exitCode=0 Dec 03 12:48:16 crc kubenswrapper[5052]: I1203 12:48:16.267170 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-operators-9nx4s" event={"ID":"b7cd9737-53b4-414c-a51b-85e21e6f4aa7","Type":"ContainerDied","Data":"1d02d8acfd7135f1ee2b76bfcdb23259e05bd05d52a412e5809afd7e88dcf52b"} Dec 03 12:48:16 crc kubenswrapper[5052]: I1203 12:48:16.267367 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:48:16 crc kubenswrapper[5052]: I1203 12:48:16.673395 5052 scope.go:117] "RemoveContainer" containerID="b1f27aab2d1cce51772c0ce99bd8dfc4afd0620b38b776e6b968d9114105f165" Dec 03 12:48:16 crc kubenswrapper[5052]: E1203 12:48:16.673679 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"nova-scheduler-scheduler\" with CrashLoopBackOff: \"back-off 40s restarting failed container=nova-scheduler-scheduler pod=nova-scheduler-0_openstack-kuttl-tests(81709983-1fd0-4446-bbbf-fb94653bb3d2)\"" pod="openstack-kuttl-tests/nova-scheduler-0" podUID="81709983-1fd0-4446-bbbf-fb94653bb3d2" Dec 03 12:48:17 crc kubenswrapper[5052]: I1203 12:48:17.279146 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-operators-9nx4s" event={"ID":"b7cd9737-53b4-414c-a51b-85e21e6f4aa7","Type":"ContainerStarted","Data":"36278016e7f1acc45cc98e13ca70cbede708ca3f836f4836fe1a85096a93bdcf"} Dec 03 12:48:17 crc kubenswrapper[5052]: I1203 12:48:17.299242 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-marketplace/redhat-operators-9nx4s" podStartSLOduration=2.8402015609999998 podStartE2EDuration="11.299220663s" podCreationTimestamp="2025-12-03 12:48:06 +0000 UTC" firstStartedPulling="2025-12-03 12:48:08.165414703 +0000 UTC m=+5278.753656467" lastFinishedPulling="2025-12-03 12:48:16.624433805 +0000 UTC m=+5287.212675569" observedRunningTime="2025-12-03 12:48:17.294950831 +0000 UTC m=+5287.883192595" watchObservedRunningTime="2025-12-03 12:48:17.299220663 +0000 UTC m=+5287.887462427" Dec 03 12:48:19 crc kubenswrapper[5052]: I1203 12:48:19.681051 5052 scope.go:117] "RemoveContainer" containerID="8e3764a05c784a75a82ccff0fcb9f4aaac35f5d6845cca81b14deee31318466f" Dec 03 12:48:21 crc kubenswrapper[5052]: I1203 12:48:21.343605 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-conductor-0" event={"ID":"8669832d-c47a-49bd-9491-dc7141ac6281","Type":"ContainerStarted","Data":"caf28df7281c196a4c4d326b47a9f4c24851d6defc41fd8fa20f0e5325c7cdac"} Dec 03 12:48:22 crc kubenswrapper[5052]: I1203 12:48:22.352827 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:48:22 crc kubenswrapper[5052]: I1203 12:48:22.673478 5052 scope.go:117] "RemoveContainer" containerID="2aed1c760a326f3d008799e902b1093313df3764a57c879f0b3686410127d37d" Dec 03 12:48:22 crc kubenswrapper[5052]: E1203 12:48:22.673746 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 12:48:26 crc kubenswrapper[5052]: I1203 12:48:26.998626 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openshift-marketplace/redhat-operators-9nx4s" Dec 03 12:48:26 crc kubenswrapper[5052]: I1203 12:48:26.999275 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-marketplace/redhat-operators-9nx4s" Dec 03 12:48:27 crc kubenswrapper[5052]: I1203 12:48:27.053535 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-marketplace/redhat-operators-9nx4s" Dec 03 12:48:27 crc kubenswrapper[5052]: I1203 12:48:27.443047 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-marketplace/redhat-operators-9nx4s" Dec 03 12:48:27 crc kubenswrapper[5052]: I1203 12:48:27.501345 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openshift-marketplace/redhat-operators-9nx4s"] Dec 03 12:48:29 crc kubenswrapper[5052]: I1203 12:48:29.311415 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:48:29 crc kubenswrapper[5052]: I1203 12:48:29.420396 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openshift-marketplace/redhat-operators-9nx4s" podUID="b7cd9737-53b4-414c-a51b-85e21e6f4aa7" containerName="registry-server" containerID="cri-o://36278016e7f1acc45cc98e13ca70cbede708ca3f836f4836fe1a85096a93bdcf" gracePeriod=2 Dec 03 12:48:30 crc kubenswrapper[5052]: I1203 12:48:30.435891 5052 generic.go:334] "Generic (PLEG): container finished" podID="b7cd9737-53b4-414c-a51b-85e21e6f4aa7" containerID="36278016e7f1acc45cc98e13ca70cbede708ca3f836f4836fe1a85096a93bdcf" exitCode=0 Dec 03 12:48:30 crc kubenswrapper[5052]: I1203 12:48:30.435955 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-operators-9nx4s" event={"ID":"b7cd9737-53b4-414c-a51b-85e21e6f4aa7","Type":"ContainerDied","Data":"36278016e7f1acc45cc98e13ca70cbede708ca3f836f4836fe1a85096a93bdcf"} Dec 03 12:48:30 crc kubenswrapper[5052]: I1203 12:48:30.436250 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-operators-9nx4s" event={"ID":"b7cd9737-53b4-414c-a51b-85e21e6f4aa7","Type":"ContainerDied","Data":"03a8be39a06733e6dedc4c99e67f09d0a920a5c87c75a639fc37b533b033c2e9"} Dec 03 12:48:30 crc kubenswrapper[5052]: I1203 12:48:30.436264 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="03a8be39a06733e6dedc4c99e67f09d0a920a5c87c75a639fc37b533b033c2e9" Dec 03 12:48:30 crc kubenswrapper[5052]: I1203 12:48:30.444649 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:48:30 crc kubenswrapper[5052]: I1203 12:48:30.445043 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/redhat-operators-9nx4s" Dec 03 12:48:30 crc kubenswrapper[5052]: I1203 12:48:30.479394 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/b7cd9737-53b4-414c-a51b-85e21e6f4aa7-catalog-content\") pod \"b7cd9737-53b4-414c-a51b-85e21e6f4aa7\" (UID: \"b7cd9737-53b4-414c-a51b-85e21e6f4aa7\") " Dec 03 12:48:30 crc kubenswrapper[5052]: I1203 12:48:30.479777 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qsdcz\" (UniqueName: \"kubernetes.io/projected/b7cd9737-53b4-414c-a51b-85e21e6f4aa7-kube-api-access-qsdcz\") pod \"b7cd9737-53b4-414c-a51b-85e21e6f4aa7\" (UID: \"b7cd9737-53b4-414c-a51b-85e21e6f4aa7\") " Dec 03 12:48:30 crc kubenswrapper[5052]: I1203 12:48:30.479994 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/b7cd9737-53b4-414c-a51b-85e21e6f4aa7-utilities\") pod \"b7cd9737-53b4-414c-a51b-85e21e6f4aa7\" (UID: \"b7cd9737-53b4-414c-a51b-85e21e6f4aa7\") " Dec 03 12:48:30 crc kubenswrapper[5052]: I1203 12:48:30.480792 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b7cd9737-53b4-414c-a51b-85e21e6f4aa7-utilities" (OuterVolumeSpecName: "utilities") pod "b7cd9737-53b4-414c-a51b-85e21e6f4aa7" (UID: "b7cd9737-53b4-414c-a51b-85e21e6f4aa7"). InnerVolumeSpecName "utilities". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:48:30 crc kubenswrapper[5052]: I1203 12:48:30.488133 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b7cd9737-53b4-414c-a51b-85e21e6f4aa7-kube-api-access-qsdcz" (OuterVolumeSpecName: "kube-api-access-qsdcz") pod "b7cd9737-53b4-414c-a51b-85e21e6f4aa7" (UID: "b7cd9737-53b4-414c-a51b-85e21e6f4aa7"). InnerVolumeSpecName "kube-api-access-qsdcz". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:48:30 crc kubenswrapper[5052]: I1203 12:48:30.582135 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-qsdcz\" (UniqueName: \"kubernetes.io/projected/b7cd9737-53b4-414c-a51b-85e21e6f4aa7-kube-api-access-qsdcz\") on node \"crc\" DevicePath \"\"" Dec 03 12:48:30 crc kubenswrapper[5052]: I1203 12:48:30.582166 5052 reconciler_common.go:293] "Volume detached for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/b7cd9737-53b4-414c-a51b-85e21e6f4aa7-utilities\") on node \"crc\" DevicePath \"\"" Dec 03 12:48:30 crc kubenswrapper[5052]: I1203 12:48:30.606946 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b7cd9737-53b4-414c-a51b-85e21e6f4aa7-catalog-content" (OuterVolumeSpecName: "catalog-content") pod "b7cd9737-53b4-414c-a51b-85e21e6f4aa7" (UID: "b7cd9737-53b4-414c-a51b-85e21e6f4aa7"). InnerVolumeSpecName "catalog-content". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:48:30 crc kubenswrapper[5052]: I1203 12:48:30.673419 5052 scope.go:117] "RemoveContainer" containerID="b1f27aab2d1cce51772c0ce99bd8dfc4afd0620b38b776e6b968d9114105f165" Dec 03 12:48:30 crc kubenswrapper[5052]: I1203 12:48:30.684199 5052 reconciler_common.go:293] "Volume detached for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/b7cd9737-53b4-414c-a51b-85e21e6f4aa7-catalog-content\") on node \"crc\" DevicePath \"\"" Dec 03 12:48:31 crc kubenswrapper[5052]: I1203 12:48:31.447067 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/redhat-operators-9nx4s" Dec 03 12:48:31 crc kubenswrapper[5052]: I1203 12:48:31.447236 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"81709983-1fd0-4446-bbbf-fb94653bb3d2","Type":"ContainerStarted","Data":"6964abaffe4177de7695a956baa51adeedd8fc7ae4b4885ca8e38046eae0369a"} Dec 03 12:48:31 crc kubenswrapper[5052]: I1203 12:48:31.493530 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openshift-marketplace/redhat-operators-9nx4s"] Dec 03 12:48:31 crc kubenswrapper[5052]: I1203 12:48:31.507841 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openshift-marketplace/redhat-operators-9nx4s"] Dec 03 12:48:31 crc kubenswrapper[5052]: I1203 12:48:31.685066 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b7cd9737-53b4-414c-a51b-85e21e6f4aa7" path="/var/lib/kubelet/pods/b7cd9737-53b4-414c-a51b-85e21e6f4aa7/volumes" Dec 03 12:48:32 crc kubenswrapper[5052]: I1203 12:48:32.428154 5052 scope.go:117] "RemoveContainer" containerID="f6a728d1305480f200dcdc3cfa0a4ead9021fc819a9828033afc74de07435201" Dec 03 12:48:32 crc kubenswrapper[5052]: I1203 12:48:32.474767 5052 scope.go:117] "RemoveContainer" containerID="db9cddd5e36fcb65f1558dec41404bab89ab5883ec640a193f5a47a8c712d924" Dec 03 12:48:32 crc kubenswrapper[5052]: I1203 12:48:32.522170 5052 scope.go:117] "RemoveContainer" containerID="fe1395b62b90460d04b6099206e52d20adacc07cc857b25d7b625ceac277317a" Dec 03 12:48:32 crc kubenswrapper[5052]: I1203 12:48:32.547629 5052 scope.go:117] "RemoveContainer" containerID="e0c714dcc48f5f60b89824c4dfab71f3a51824e571a8af575bf534cb693276cc" Dec 03 12:48:32 crc kubenswrapper[5052]: I1203 12:48:32.567801 5052 scope.go:117] "RemoveContainer" containerID="ea0e318bc1c2d00f8c07df735c18136c77c564a4ca1cc7bf9d3e0dba21c6f865" Dec 03 12:48:33 crc kubenswrapper[5052]: I1203 12:48:33.526932 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:48:36 crc kubenswrapper[5052]: I1203 12:48:36.673256 5052 scope.go:117] "RemoveContainer" containerID="2aed1c760a326f3d008799e902b1093313df3764a57c879f0b3686410127d37d" Dec 03 12:48:37 crc kubenswrapper[5052]: I1203 12:48:37.511963 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" event={"ID":"10c237bc-72e7-4da3-9f2c-b1688395bf6e","Type":"ContainerStarted","Data":"75d5669ef68596efd94ee7584a4e5e570e7db6cc9f424db1fa9cd7f60acbab7b"} Dec 03 12:48:38 crc kubenswrapper[5052]: I1203 12:48:38.527386 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:48:38 crc kubenswrapper[5052]: I1203 12:48:38.573616 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:48:39 crc kubenswrapper[5052]: I1203 12:48:39.561843 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:48:56 crc kubenswrapper[5052]: I1203 12:48:56.885583 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/rabbitmq-cell1-server-0"] Dec 03 12:48:57 crc kubenswrapper[5052]: E1203 12:48:57.074408 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/rabbitmq-cell1-config-data: configmap "rabbitmq-cell1-config-data" not found Dec 03 12:48:57 crc kubenswrapper[5052]: E1203 12:48:57.075254 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/488c3529-4c2b-47c0-9dcf-05a064c34d0d-config-data podName:488c3529-4c2b-47c0-9dcf-05a064c34d0d nodeName:}" failed. No retries permitted until 2025-12-03 12:48:57.575235777 +0000 UTC m=+5328.163477541 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "config-data" (UniqueName: "kubernetes.io/configmap/488c3529-4c2b-47c0-9dcf-05a064c34d0d-config-data") pod "rabbitmq-cell1-server-0" (UID: "488c3529-4c2b-47c0-9dcf-05a064c34d0d") : configmap "rabbitmq-cell1-config-data" not found Dec 03 12:48:57 crc kubenswrapper[5052]: I1203 12:48:57.235986 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/rabbitmq-server-0"] Dec 03 12:48:57 crc kubenswrapper[5052]: I1203 12:48:57.263314 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/openstackclient"] Dec 03 12:48:57 crc kubenswrapper[5052]: I1203 12:48:57.263684 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/openstackclient" podUID="8a94fda3-63d8-4517-8063-f6c06688b430" containerName="openstackclient" containerID="cri-o://e3c10016f8e333ebc2b5eef7809847e5f7ea2904a0f92aa3ddc61a80ff2d9403" gracePeriod=2 Dec 03 12:48:57 crc kubenswrapper[5052]: I1203 12:48:57.277411 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/openstackclient"] Dec 03 12:48:57 crc kubenswrapper[5052]: E1203 12:48:57.382604 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/rabbitmq-config-data: configmap "rabbitmq-config-data" not found Dec 03 12:48:57 crc kubenswrapper[5052]: E1203 12:48:57.382658 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0cb33a40-6f0a-4e89-a159-4edd3af3a2fa-config-data podName:0cb33a40-6f0a-4e89-a159-4edd3af3a2fa nodeName:}" failed. No retries permitted until 2025-12-03 12:48:57.882645538 +0000 UTC m=+5328.470887302 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "config-data" (UniqueName: "kubernetes.io/configmap/0cb33a40-6f0a-4e89-a159-4edd3af3a2fa-config-data") pod "rabbitmq-server-0" (UID: "0cb33a40-6f0a-4e89-a159-4edd3af3a2fa") : configmap "rabbitmq-config-data" not found Dec 03 12:48:57 crc kubenswrapper[5052]: E1203 12:48:57.588861 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/rabbitmq-cell1-config-data: configmap "rabbitmq-cell1-config-data" not found Dec 03 12:48:57 crc kubenswrapper[5052]: E1203 12:48:57.588923 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/488c3529-4c2b-47c0-9dcf-05a064c34d0d-config-data podName:488c3529-4c2b-47c0-9dcf-05a064c34d0d nodeName:}" failed. No retries permitted until 2025-12-03 12:48:58.588909447 +0000 UTC m=+5329.177151221 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "config-data" (UniqueName: "kubernetes.io/configmap/488c3529-4c2b-47c0-9dcf-05a064c34d0d-config-data") pod "rabbitmq-cell1-server-0" (UID: "488c3529-4c2b-47c0-9dcf-05a064c34d0d") : configmap "rabbitmq-cell1-config-data" not found Dec 03 12:48:57 crc kubenswrapper[5052]: I1203 12:48:57.656633 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-nb-0"] Dec 03 12:48:57 crc kubenswrapper[5052]: I1203 12:48:57.657047 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ovsdbserver-nb-0" podUID="2005f89c-8a19-403c-acc9-c13ede717372" containerName="openstack-network-exporter" containerID="cri-o://c8c8497d5896f2a84e4979c47d2430cbec8075e3536d1d41039d14854df9a795" gracePeriod=300 Dec 03 12:48:57 crc kubenswrapper[5052]: I1203 12:48:57.753826 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/placement361c-account-delete-67ttd"] Dec 03 12:48:57 crc kubenswrapper[5052]: E1203 12:48:57.754679 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="b7cd9737-53b4-414c-a51b-85e21e6f4aa7" containerName="extract-utilities" Dec 03 12:48:57 crc kubenswrapper[5052]: I1203 12:48:57.754703 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="b7cd9737-53b4-414c-a51b-85e21e6f4aa7" containerName="extract-utilities" Dec 03 12:48:57 crc kubenswrapper[5052]: E1203 12:48:57.754718 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="8a94fda3-63d8-4517-8063-f6c06688b430" containerName="openstackclient" Dec 03 12:48:57 crc kubenswrapper[5052]: I1203 12:48:57.754726 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="8a94fda3-63d8-4517-8063-f6c06688b430" containerName="openstackclient" Dec 03 12:48:57 crc kubenswrapper[5052]: E1203 12:48:57.754747 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="95997b63-313e-4f5f-adf6-b8ae63de9fb9" containerName="neutron-api" Dec 03 12:48:57 crc kubenswrapper[5052]: I1203 12:48:57.754756 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="95997b63-313e-4f5f-adf6-b8ae63de9fb9" containerName="neutron-api" Dec 03 12:48:57 crc kubenswrapper[5052]: E1203 12:48:57.754784 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="b7cd9737-53b4-414c-a51b-85e21e6f4aa7" containerName="registry-server" Dec 03 12:48:57 crc kubenswrapper[5052]: I1203 12:48:57.754791 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="b7cd9737-53b4-414c-a51b-85e21e6f4aa7" containerName="registry-server" Dec 03 12:48:57 crc kubenswrapper[5052]: E1203 12:48:57.754805 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="95997b63-313e-4f5f-adf6-b8ae63de9fb9" containerName="neutron-httpd" Dec 03 12:48:57 crc kubenswrapper[5052]: I1203 12:48:57.754812 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="95997b63-313e-4f5f-adf6-b8ae63de9fb9" containerName="neutron-httpd" Dec 03 12:48:57 crc kubenswrapper[5052]: E1203 12:48:57.754837 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="b7cd9737-53b4-414c-a51b-85e21e6f4aa7" containerName="extract-content" Dec 03 12:48:57 crc kubenswrapper[5052]: I1203 12:48:57.754845 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="b7cd9737-53b4-414c-a51b-85e21e6f4aa7" containerName="extract-content" Dec 03 12:48:57 crc kubenswrapper[5052]: I1203 12:48:57.755083 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="95997b63-313e-4f5f-adf6-b8ae63de9fb9" containerName="neutron-httpd" Dec 03 12:48:57 crc kubenswrapper[5052]: I1203 12:48:57.755097 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="b7cd9737-53b4-414c-a51b-85e21e6f4aa7" containerName="registry-server" Dec 03 12:48:57 crc kubenswrapper[5052]: I1203 12:48:57.755120 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="8a94fda3-63d8-4517-8063-f6c06688b430" containerName="openstackclient" Dec 03 12:48:57 crc kubenswrapper[5052]: I1203 12:48:57.755140 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="95997b63-313e-4f5f-adf6-b8ae63de9fb9" containerName="neutron-api" Dec 03 12:48:57 crc kubenswrapper[5052]: I1203 12:48:57.763240 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement361c-account-delete-67ttd" Dec 03 12:48:57 crc kubenswrapper[5052]: I1203 12:48:57.789236 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/placement361c-account-delete-67ttd"] Dec 03 12:48:57 crc kubenswrapper[5052]: I1203 12:48:57.807780 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ovn-northd-0"] Dec 03 12:48:57 crc kubenswrapper[5052]: I1203 12:48:57.808006 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ovn-northd-0" podUID="342cba80-22e1-488a-bbfb-75b67ca536ac" containerName="ovn-northd" containerID="cri-o://6b272b57a393df881b0806ef34c9c3a75d64d13721c0eab27f38fa3374aed7ed" gracePeriod=30 Dec 03 12:48:57 crc kubenswrapper[5052]: I1203 12:48:57.808125 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ovn-northd-0" podUID="342cba80-22e1-488a-bbfb-75b67ca536ac" containerName="openstack-network-exporter" containerID="cri-o://8df9cc82a4e4e7032e7155d5c8c0fda7507b5b3d7b99882037f3bf0a8995f1d7" gracePeriod=30 Dec 03 12:48:57 crc kubenswrapper[5052]: I1203 12:48:57.900990 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/456c3fb1-90c0-4d17-863e-414380759f97-operator-scripts\") pod \"placement361c-account-delete-67ttd\" (UID: \"456c3fb1-90c0-4d17-863e-414380759f97\") " pod="openstack-kuttl-tests/placement361c-account-delete-67ttd" Dec 03 12:48:57 crc kubenswrapper[5052]: I1203 12:48:57.901141 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7t96z\" (UniqueName: \"kubernetes.io/projected/456c3fb1-90c0-4d17-863e-414380759f97-kube-api-access-7t96z\") pod \"placement361c-account-delete-67ttd\" (UID: \"456c3fb1-90c0-4d17-863e-414380759f97\") " pod="openstack-kuttl-tests/placement361c-account-delete-67ttd" Dec 03 12:48:57 crc kubenswrapper[5052]: E1203 12:48:57.901347 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/rabbitmq-config-data: configmap "rabbitmq-config-data" not found Dec 03 12:48:57 crc kubenswrapper[5052]: E1203 12:48:57.901401 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0cb33a40-6f0a-4e89-a159-4edd3af3a2fa-config-data podName:0cb33a40-6f0a-4e89-a159-4edd3af3a2fa nodeName:}" failed. No retries permitted until 2025-12-03 12:48:58.901387793 +0000 UTC m=+5329.489629557 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "config-data" (UniqueName: "kubernetes.io/configmap/0cb33a40-6f0a-4e89-a159-4edd3af3a2fa-config-data") pod "rabbitmq-server-0" (UID: "0cb33a40-6f0a-4e89-a159-4edd3af3a2fa") : configmap "rabbitmq-config-data" not found Dec 03 12:48:57 crc kubenswrapper[5052]: I1203 12:48:57.968934 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ovsdbserver-nb-0" podUID="2005f89c-8a19-403c-acc9-c13ede717372" containerName="ovsdbserver-nb" containerID="cri-o://14ec8a29b2f04a213b48a3edd706ccb3eed3920a11add7c7ed0658a51c533ce3" gracePeriod=300 Dec 03 12:48:58 crc kubenswrapper[5052]: I1203 12:48:58.014433 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/456c3fb1-90c0-4d17-863e-414380759f97-operator-scripts\") pod \"placement361c-account-delete-67ttd\" (UID: \"456c3fb1-90c0-4d17-863e-414380759f97\") " pod="openstack-kuttl-tests/placement361c-account-delete-67ttd" Dec 03 12:48:58 crc kubenswrapper[5052]: I1203 12:48:58.016775 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/456c3fb1-90c0-4d17-863e-414380759f97-operator-scripts\") pod \"placement361c-account-delete-67ttd\" (UID: \"456c3fb1-90c0-4d17-863e-414380759f97\") " pod="openstack-kuttl-tests/placement361c-account-delete-67ttd" Dec 03 12:48:58 crc kubenswrapper[5052]: I1203 12:48:58.021992 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-7t96z\" (UniqueName: \"kubernetes.io/projected/456c3fb1-90c0-4d17-863e-414380759f97-kube-api-access-7t96z\") pod \"placement361c-account-delete-67ttd\" (UID: \"456c3fb1-90c0-4d17-863e-414380759f97\") " pod="openstack-kuttl-tests/placement361c-account-delete-67ttd" Dec 03 12:48:58 crc kubenswrapper[5052]: I1203 12:48:58.103495 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-7t96z\" (UniqueName: \"kubernetes.io/projected/456c3fb1-90c0-4d17-863e-414380759f97-kube-api-access-7t96z\") pod \"placement361c-account-delete-67ttd\" (UID: \"456c3fb1-90c0-4d17-863e-414380759f97\") " pod="openstack-kuttl-tests/placement361c-account-delete-67ttd" Dec 03 12:48:58 crc kubenswrapper[5052]: I1203 12:48:58.120724 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/glanced4a6-account-delete-69llz"] Dec 03 12:48:58 crc kubenswrapper[5052]: I1203 12:48:58.151378 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glanced4a6-account-delete-69llz"] Dec 03 12:48:58 crc kubenswrapper[5052]: I1203 12:48:58.151493 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glanced4a6-account-delete-69llz" Dec 03 12:48:58 crc kubenswrapper[5052]: I1203 12:48:58.173164 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/placement-db-sync-s7jr9"] Dec 03 12:48:58 crc kubenswrapper[5052]: I1203 12:48:58.188666 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/placement-db-sync-s7jr9"] Dec 03 12:48:58 crc kubenswrapper[5052]: I1203 12:48:58.195767 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/cinder4345-account-delete-vmdzh"] Dec 03 12:48:58 crc kubenswrapper[5052]: I1203 12:48:58.201082 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder4345-account-delete-vmdzh" Dec 03 12:48:58 crc kubenswrapper[5052]: I1203 12:48:58.251989 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/6a5de7fb-b482-4849-a957-e679616aabc0-operator-scripts\") pod \"glanced4a6-account-delete-69llz\" (UID: \"6a5de7fb-b482-4849-a957-e679616aabc0\") " pod="openstack-kuttl-tests/glanced4a6-account-delete-69llz" Dec 03 12:48:58 crc kubenswrapper[5052]: I1203 12:48:58.252119 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d722f\" (UniqueName: \"kubernetes.io/projected/6a5de7fb-b482-4849-a957-e679616aabc0-kube-api-access-d722f\") pod \"glanced4a6-account-delete-69llz\" (UID: \"6a5de7fb-b482-4849-a957-e679616aabc0\") " pod="openstack-kuttl-tests/glanced4a6-account-delete-69llz" Dec 03 12:48:58 crc kubenswrapper[5052]: I1203 12:48:58.252172 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/5d11b02e-c14a-4d1a-819a-4c3b66aedd33-operator-scripts\") pod \"cinder4345-account-delete-vmdzh\" (UID: \"5d11b02e-c14a-4d1a-819a-4c3b66aedd33\") " pod="openstack-kuttl-tests/cinder4345-account-delete-vmdzh" Dec 03 12:48:58 crc kubenswrapper[5052]: I1203 12:48:58.252219 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8clcg\" (UniqueName: \"kubernetes.io/projected/5d11b02e-c14a-4d1a-819a-4c3b66aedd33-kube-api-access-8clcg\") pod \"cinder4345-account-delete-vmdzh\" (UID: \"5d11b02e-c14a-4d1a-819a-4c3b66aedd33\") " pod="openstack-kuttl-tests/cinder4345-account-delete-vmdzh" Dec 03 12:48:58 crc kubenswrapper[5052]: I1203 12:48:58.254900 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-db-sync-pp7qs"] Dec 03 12:48:58 crc kubenswrapper[5052]: I1203 12:48:58.256756 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement361c-account-delete-67ttd" Dec 03 12:48:58 crc kubenswrapper[5052]: I1203 12:48:58.318197 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/glance-db-sync-pp7qs"] Dec 03 12:48:58 crc kubenswrapper[5052]: I1203 12:48:58.353937 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-d722f\" (UniqueName: \"kubernetes.io/projected/6a5de7fb-b482-4849-a957-e679616aabc0-kube-api-access-d722f\") pod \"glanced4a6-account-delete-69llz\" (UID: \"6a5de7fb-b482-4849-a957-e679616aabc0\") " pod="openstack-kuttl-tests/glanced4a6-account-delete-69llz" Dec 03 12:48:58 crc kubenswrapper[5052]: I1203 12:48:58.354017 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/5d11b02e-c14a-4d1a-819a-4c3b66aedd33-operator-scripts\") pod \"cinder4345-account-delete-vmdzh\" (UID: \"5d11b02e-c14a-4d1a-819a-4c3b66aedd33\") " pod="openstack-kuttl-tests/cinder4345-account-delete-vmdzh" Dec 03 12:48:58 crc kubenswrapper[5052]: I1203 12:48:58.354063 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-8clcg\" (UniqueName: \"kubernetes.io/projected/5d11b02e-c14a-4d1a-819a-4c3b66aedd33-kube-api-access-8clcg\") pod \"cinder4345-account-delete-vmdzh\" (UID: \"5d11b02e-c14a-4d1a-819a-4c3b66aedd33\") " pod="openstack-kuttl-tests/cinder4345-account-delete-vmdzh" Dec 03 12:48:58 crc kubenswrapper[5052]: I1203 12:48:58.354117 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/6a5de7fb-b482-4849-a957-e679616aabc0-operator-scripts\") pod \"glanced4a6-account-delete-69llz\" (UID: \"6a5de7fb-b482-4849-a957-e679616aabc0\") " pod="openstack-kuttl-tests/glanced4a6-account-delete-69llz" Dec 03 12:48:58 crc kubenswrapper[5052]: I1203 12:48:58.354924 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/6a5de7fb-b482-4849-a957-e679616aabc0-operator-scripts\") pod \"glanced4a6-account-delete-69llz\" (UID: \"6a5de7fb-b482-4849-a957-e679616aabc0\") " pod="openstack-kuttl-tests/glanced4a6-account-delete-69llz" Dec 03 12:48:58 crc kubenswrapper[5052]: I1203 12:48:58.355166 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/5d11b02e-c14a-4d1a-819a-4c3b66aedd33-operator-scripts\") pod \"cinder4345-account-delete-vmdzh\" (UID: \"5d11b02e-c14a-4d1a-819a-4c3b66aedd33\") " pod="openstack-kuttl-tests/cinder4345-account-delete-vmdzh" Dec 03 12:48:58 crc kubenswrapper[5052]: I1203 12:48:58.359955 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-sb-0"] Dec 03 12:48:58 crc kubenswrapper[5052]: I1203 12:48:58.360856 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ovsdbserver-sb-0" podUID="84f26afe-5b0c-43bb-a4c8-a9ac21d24330" containerName="openstack-network-exporter" containerID="cri-o://721e21ea9214ab0b7f428a1744d9bf1b20c7493291c82e15a89dd5d11891df97" gracePeriod=300 Dec 03 12:48:58 crc kubenswrapper[5052]: I1203 12:48:58.414115 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder-db-sync-2ttlg"] Dec 03 12:48:58 crc kubenswrapper[5052]: I1203 12:48:58.428059 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/cinder-db-sync-2ttlg"] Dec 03 12:48:58 crc kubenswrapper[5052]: I1203 12:48:58.461865 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder4345-account-delete-vmdzh"] Dec 03 12:48:58 crc kubenswrapper[5052]: I1203 12:48:58.466796 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-8clcg\" (UniqueName: \"kubernetes.io/projected/5d11b02e-c14a-4d1a-819a-4c3b66aedd33-kube-api-access-8clcg\") pod \"cinder4345-account-delete-vmdzh\" (UID: \"5d11b02e-c14a-4d1a-819a-4c3b66aedd33\") " pod="openstack-kuttl-tests/cinder4345-account-delete-vmdzh" Dec 03 12:48:58 crc kubenswrapper[5052]: I1203 12:48:58.475643 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/neutrona25a-account-delete-vbr24"] Dec 03 12:48:58 crc kubenswrapper[5052]: I1203 12:48:58.477006 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutrona25a-account-delete-vbr24" Dec 03 12:48:58 crc kubenswrapper[5052]: I1203 12:48:58.482590 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ovsdbserver-sb-0" podUID="84f26afe-5b0c-43bb-a4c8-a9ac21d24330" containerName="ovsdbserver-sb" containerID="cri-o://b74065369c814d7538cce05dc1a2188d258f8a0970dbde7fa52adce059389dc5" gracePeriod=300 Dec 03 12:48:58 crc kubenswrapper[5052]: I1203 12:48:58.518618 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutrona25a-account-delete-vbr24"] Dec 03 12:48:58 crc kubenswrapper[5052]: I1203 12:48:58.534791 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-d722f\" (UniqueName: \"kubernetes.io/projected/6a5de7fb-b482-4849-a957-e679616aabc0-kube-api-access-d722f\") pod \"glanced4a6-account-delete-69llz\" (UID: \"6a5de7fb-b482-4849-a957-e679616aabc0\") " pod="openstack-kuttl-tests/glanced4a6-account-delete-69llz" Dec 03 12:48:58 crc kubenswrapper[5052]: I1203 12:48:58.563858 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glanced4a6-account-delete-69llz" Dec 03 12:48:58 crc kubenswrapper[5052]: I1203 12:48:58.590106 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/barbican02cc-account-delete-gf94b"] Dec 03 12:48:58 crc kubenswrapper[5052]: I1203 12:48:58.592154 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican02cc-account-delete-gf94b" Dec 03 12:48:58 crc kubenswrapper[5052]: I1203 12:48:58.603722 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder4345-account-delete-vmdzh" Dec 03 12:48:58 crc kubenswrapper[5052]: I1203 12:48:58.669474 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tpc6k\" (UniqueName: \"kubernetes.io/projected/6ceba96c-7e86-4eb2-afa7-444f0fcc27ce-kube-api-access-tpc6k\") pod \"neutrona25a-account-delete-vbr24\" (UID: \"6ceba96c-7e86-4eb2-afa7-444f0fcc27ce\") " pod="openstack-kuttl-tests/neutrona25a-account-delete-vbr24" Dec 03 12:48:58 crc kubenswrapper[5052]: I1203 12:48:58.670266 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/6ceba96c-7e86-4eb2-afa7-444f0fcc27ce-operator-scripts\") pod \"neutrona25a-account-delete-vbr24\" (UID: \"6ceba96c-7e86-4eb2-afa7-444f0fcc27ce\") " pod="openstack-kuttl-tests/neutrona25a-account-delete-vbr24" Dec 03 12:48:58 crc kubenswrapper[5052]: E1203 12:48:58.670468 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/rabbitmq-cell1-config-data: configmap "rabbitmq-cell1-config-data" not found Dec 03 12:48:58 crc kubenswrapper[5052]: E1203 12:48:58.670515 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/488c3529-4c2b-47c0-9dcf-05a064c34d0d-config-data podName:488c3529-4c2b-47c0-9dcf-05a064c34d0d nodeName:}" failed. No retries permitted until 2025-12-03 12:49:00.670497339 +0000 UTC m=+5331.258739103 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "config-data" (UniqueName: "kubernetes.io/configmap/488c3529-4c2b-47c0-9dcf-05a064c34d0d-config-data") pod "rabbitmq-cell1-server-0" (UID: "488c3529-4c2b-47c0-9dcf-05a064c34d0d") : configmap "rabbitmq-cell1-config-data" not found Dec 03 12:48:58 crc kubenswrapper[5052]: I1203 12:48:58.673227 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican02cc-account-delete-gf94b"] Dec 03 12:48:58 crc kubenswrapper[5052]: I1203 12:48:58.708002 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/neutron-db-sync-zf7km"] Dec 03 12:48:58 crc kubenswrapper[5052]: I1203 12:48:58.722214 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/neutron-db-sync-zf7km"] Dec 03 12:48:58 crc kubenswrapper[5052]: E1203 12:48:58.729039 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="6b272b57a393df881b0806ef34c9c3a75d64d13721c0eab27f38fa3374aed7ed" cmd=["/usr/local/bin/container-scripts/status_check.sh"] Dec 03 12:48:58 crc kubenswrapper[5052]: E1203 12:48:58.732273 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="6b272b57a393df881b0806ef34c9c3a75d64d13721c0eab27f38fa3374aed7ed" cmd=["/usr/local/bin/container-scripts/status_check.sh"] Dec 03 12:48:58 crc kubenswrapper[5052]: I1203 12:48:58.733731 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-db-sync-ccx7c"] Dec 03 12:48:58 crc kubenswrapper[5052]: E1203 12:48:58.742765 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="6b272b57a393df881b0806ef34c9c3a75d64d13721c0eab27f38fa3374aed7ed" cmd=["/usr/local/bin/container-scripts/status_check.sh"] Dec 03 12:48:58 crc kubenswrapper[5052]: E1203 12:48:58.742831 5052 prober.go:104] "Probe errored" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" probeType="Readiness" pod="openstack-kuttl-tests/ovn-northd-0" podUID="342cba80-22e1-488a-bbfb-75b67ca536ac" containerName="ovn-northd" Dec 03 12:48:58 crc kubenswrapper[5052]: I1203 12:48:58.744908 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/swift-storage-0"] Dec 03 12:48:58 crc kubenswrapper[5052]: I1203 12:48:58.745487 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="bd4392a2-9c70-4f3a-9050-ae6a6db01082" containerName="account-server" containerID="cri-o://ee68253d5346b57e39fc0b0fed73d3c548e33dc2c36cd2673c5add72f235bb79" gracePeriod=30 Dec 03 12:48:58 crc kubenswrapper[5052]: I1203 12:48:58.745956 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="bd4392a2-9c70-4f3a-9050-ae6a6db01082" containerName="swift-recon-cron" containerID="cri-o://6eeb3694b4e9534ae61ec9fcf173cc77b70bcda76de435e9b1ede55531edf6ec" gracePeriod=30 Dec 03 12:48:58 crc kubenswrapper[5052]: I1203 12:48:58.746020 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="bd4392a2-9c70-4f3a-9050-ae6a6db01082" containerName="rsync" containerID="cri-o://44d343c040a52a96a2507d69db579564483888cfc2ab0051211cae74e09f32f3" gracePeriod=30 Dec 03 12:48:58 crc kubenswrapper[5052]: I1203 12:48:58.746083 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="bd4392a2-9c70-4f3a-9050-ae6a6db01082" containerName="object-expirer" containerID="cri-o://ef01830e5c368fdf7b4e0af6d27790eef08c24d4f1d55b39ccbab91a7ba97c10" gracePeriod=30 Dec 03 12:48:58 crc kubenswrapper[5052]: I1203 12:48:58.746116 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="bd4392a2-9c70-4f3a-9050-ae6a6db01082" containerName="object-updater" containerID="cri-o://4f5b00a3217544c70607eacf64edc7dd926767e810b84e309cbc071e09b652d5" gracePeriod=30 Dec 03 12:48:58 crc kubenswrapper[5052]: I1203 12:48:58.746146 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="bd4392a2-9c70-4f3a-9050-ae6a6db01082" containerName="object-auditor" containerID="cri-o://d17825037af7c0e226677e493c307a512bb167e46667dd9c705ac03cff6758bb" gracePeriod=30 Dec 03 12:48:58 crc kubenswrapper[5052]: I1203 12:48:58.746175 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="bd4392a2-9c70-4f3a-9050-ae6a6db01082" containerName="object-replicator" containerID="cri-o://257a3713d34f8297bc83998342be63e79d0964dc86ce021417f8cb3fc862ff41" gracePeriod=30 Dec 03 12:48:58 crc kubenswrapper[5052]: I1203 12:48:58.746205 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="bd4392a2-9c70-4f3a-9050-ae6a6db01082" containerName="object-server" containerID="cri-o://073f8906a127eaad4d2dcbdbaf2738671488e4b5da8001af8eb90b508703cf7a" gracePeriod=30 Dec 03 12:48:58 crc kubenswrapper[5052]: I1203 12:48:58.746237 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="bd4392a2-9c70-4f3a-9050-ae6a6db01082" containerName="container-updater" containerID="cri-o://0420f9807b79aa9a35f7e00db76c776194c37e61b2e6f2e9e3e83e3f355d270c" gracePeriod=30 Dec 03 12:48:58 crc kubenswrapper[5052]: I1203 12:48:58.746269 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="bd4392a2-9c70-4f3a-9050-ae6a6db01082" containerName="container-auditor" containerID="cri-o://49448581e8cc46afbfef30edf584ba024f9fdfe5baf0de42a0abed3885f0ef23" gracePeriod=30 Dec 03 12:48:58 crc kubenswrapper[5052]: I1203 12:48:58.746297 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="bd4392a2-9c70-4f3a-9050-ae6a6db01082" containerName="container-replicator" containerID="cri-o://4aa02cc6c01a66c5bcc1c0d61505c02dbf6374ff0a6550b9d548733640117138" gracePeriod=30 Dec 03 12:48:58 crc kubenswrapper[5052]: I1203 12:48:58.746328 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="bd4392a2-9c70-4f3a-9050-ae6a6db01082" containerName="container-server" containerID="cri-o://2eaf7736f8bbe27fc3320dbe4500d1c1b7388bb8194a9d8a9bb7ef68f452b97f" gracePeriod=30 Dec 03 12:48:58 crc kubenswrapper[5052]: I1203 12:48:58.746359 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="bd4392a2-9c70-4f3a-9050-ae6a6db01082" containerName="account-reaper" containerID="cri-o://409983b9eafc0681b072e5508199566f3fbeaf0cc55e055beaee8282045a003e" gracePeriod=30 Dec 03 12:48:58 crc kubenswrapper[5052]: I1203 12:48:58.749296 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="bd4392a2-9c70-4f3a-9050-ae6a6db01082" containerName="account-replicator" containerID="cri-o://8ea80229aee1ce207c0394efd61dfb519308fde6a2a25c517795af19f4c72da4" gracePeriod=30 Dec 03 12:48:58 crc kubenswrapper[5052]: I1203 12:48:58.749998 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="bd4392a2-9c70-4f3a-9050-ae6a6db01082" containerName="account-auditor" containerID="cri-o://e4afa7a612ce1826fda8750703ccc15d8110e17d1d419c076a60957b33af8adb" gracePeriod=30 Dec 03 12:48:58 crc kubenswrapper[5052]: I1203 12:48:58.758875 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/barbican-db-sync-ccx7c"] Dec 03 12:48:58 crc kubenswrapper[5052]: I1203 12:48:58.776322 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/857b8a8a-d730-4832-a71c-d45a2f3ce406-operator-scripts\") pod \"barbican02cc-account-delete-gf94b\" (UID: \"857b8a8a-d730-4832-a71c-d45a2f3ce406\") " pod="openstack-kuttl-tests/barbican02cc-account-delete-gf94b" Dec 03 12:48:58 crc kubenswrapper[5052]: I1203 12:48:58.776468 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/6ceba96c-7e86-4eb2-afa7-444f0fcc27ce-operator-scripts\") pod \"neutrona25a-account-delete-vbr24\" (UID: \"6ceba96c-7e86-4eb2-afa7-444f0fcc27ce\") " pod="openstack-kuttl-tests/neutrona25a-account-delete-vbr24" Dec 03 12:48:58 crc kubenswrapper[5052]: I1203 12:48:58.776495 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5mh5h\" (UniqueName: \"kubernetes.io/projected/857b8a8a-d730-4832-a71c-d45a2f3ce406-kube-api-access-5mh5h\") pod \"barbican02cc-account-delete-gf94b\" (UID: \"857b8a8a-d730-4832-a71c-d45a2f3ce406\") " pod="openstack-kuttl-tests/barbican02cc-account-delete-gf94b" Dec 03 12:48:58 crc kubenswrapper[5052]: I1203 12:48:58.776597 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-tpc6k\" (UniqueName: \"kubernetes.io/projected/6ceba96c-7e86-4eb2-afa7-444f0fcc27ce-kube-api-access-tpc6k\") pod \"neutrona25a-account-delete-vbr24\" (UID: \"6ceba96c-7e86-4eb2-afa7-444f0fcc27ce\") " pod="openstack-kuttl-tests/neutrona25a-account-delete-vbr24" Dec 03 12:48:58 crc kubenswrapper[5052]: I1203 12:48:58.778199 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/6ceba96c-7e86-4eb2-afa7-444f0fcc27ce-operator-scripts\") pod \"neutrona25a-account-delete-vbr24\" (UID: \"6ceba96c-7e86-4eb2-afa7-444f0fcc27ce\") " pod="openstack-kuttl-tests/neutrona25a-account-delete-vbr24" Dec 03 12:48:58 crc kubenswrapper[5052]: I1203 12:48:58.810705 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/swift-ring-rebalance-kvhc2"] Dec 03 12:48:58 crc kubenswrapper[5052]: I1203 12:48:58.813360 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/rabbitmq-cell1-server-0"] Dec 03 12:48:58 crc kubenswrapper[5052]: I1203 12:48:58.822072 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-tpc6k\" (UniqueName: \"kubernetes.io/projected/6ceba96c-7e86-4eb2-afa7-444f0fcc27ce-kube-api-access-tpc6k\") pod \"neutrona25a-account-delete-vbr24\" (UID: \"6ceba96c-7e86-4eb2-afa7-444f0fcc27ce\") " pod="openstack-kuttl-tests/neutrona25a-account-delete-vbr24" Dec 03 12:48:58 crc kubenswrapper[5052]: I1203 12:48:58.847140 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_ovsdbserver-nb-0_2005f89c-8a19-403c-acc9-c13ede717372/ovsdbserver-nb/0.log" Dec 03 12:48:58 crc kubenswrapper[5052]: I1203 12:48:58.847199 5052 generic.go:334] "Generic (PLEG): container finished" podID="2005f89c-8a19-403c-acc9-c13ede717372" containerID="c8c8497d5896f2a84e4979c47d2430cbec8075e3536d1d41039d14854df9a795" exitCode=2 Dec 03 12:48:58 crc kubenswrapper[5052]: I1203 12:48:58.847217 5052 generic.go:334] "Generic (PLEG): container finished" podID="2005f89c-8a19-403c-acc9-c13ede717372" containerID="14ec8a29b2f04a213b48a3edd706ccb3eed3920a11add7c7ed0658a51c533ce3" exitCode=143 Dec 03 12:48:58 crc kubenswrapper[5052]: I1203 12:48:58.851705 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-nb-0" event={"ID":"2005f89c-8a19-403c-acc9-c13ede717372","Type":"ContainerDied","Data":"c8c8497d5896f2a84e4979c47d2430cbec8075e3536d1d41039d14854df9a795"} Dec 03 12:48:58 crc kubenswrapper[5052]: I1203 12:48:58.851785 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-nb-0" event={"ID":"2005f89c-8a19-403c-acc9-c13ede717372","Type":"ContainerDied","Data":"14ec8a29b2f04a213b48a3edd706ccb3eed3920a11add7c7ed0658a51c533ce3"} Dec 03 12:48:58 crc kubenswrapper[5052]: I1203 12:48:58.867991 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutrona25a-account-delete-vbr24" Dec 03 12:48:58 crc kubenswrapper[5052]: I1203 12:48:58.876738 5052 generic.go:334] "Generic (PLEG): container finished" podID="342cba80-22e1-488a-bbfb-75b67ca536ac" containerID="8df9cc82a4e4e7032e7155d5c8c0fda7507b5b3d7b99882037f3bf0a8995f1d7" exitCode=2 Dec 03 12:48:58 crc kubenswrapper[5052]: I1203 12:48:58.876879 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/placement-74746c7f4-h6sx8"] Dec 03 12:48:58 crc kubenswrapper[5052]: I1203 12:48:58.876914 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovn-northd-0" event={"ID":"342cba80-22e1-488a-bbfb-75b67ca536ac","Type":"ContainerDied","Data":"8df9cc82a4e4e7032e7155d5c8c0fda7507b5b3d7b99882037f3bf0a8995f1d7"} Dec 03 12:48:58 crc kubenswrapper[5052]: I1203 12:48:58.877149 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/placement-74746c7f4-h6sx8" podUID="5201872c-fde1-49d6-9784-6a7f54497990" containerName="placement-log" containerID="cri-o://775232fb24de37720556752c4b1bc2dd4ea9fd1233582be2d16b27194e0a1798" gracePeriod=30 Dec 03 12:48:58 crc kubenswrapper[5052]: I1203 12:48:58.877644 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/placement-74746c7f4-h6sx8" podUID="5201872c-fde1-49d6-9784-6a7f54497990" containerName="placement-api" containerID="cri-o://d07d82d1f66b2f1adb94be8826e8db31e731aedc69314cb16576303298684ad0" gracePeriod=30 Dec 03 12:48:58 crc kubenswrapper[5052]: I1203 12:48:58.879072 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-5mh5h\" (UniqueName: \"kubernetes.io/projected/857b8a8a-d730-4832-a71c-d45a2f3ce406-kube-api-access-5mh5h\") pod \"barbican02cc-account-delete-gf94b\" (UID: \"857b8a8a-d730-4832-a71c-d45a2f3ce406\") " pod="openstack-kuttl-tests/barbican02cc-account-delete-gf94b" Dec 03 12:48:58 crc kubenswrapper[5052]: I1203 12:48:58.879190 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/857b8a8a-d730-4832-a71c-d45a2f3ce406-operator-scripts\") pod \"barbican02cc-account-delete-gf94b\" (UID: \"857b8a8a-d730-4832-a71c-d45a2f3ce406\") " pod="openstack-kuttl-tests/barbican02cc-account-delete-gf94b" Dec 03 12:48:58 crc kubenswrapper[5052]: I1203 12:48:58.880243 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/857b8a8a-d730-4832-a71c-d45a2f3ce406-operator-scripts\") pod \"barbican02cc-account-delete-gf94b\" (UID: \"857b8a8a-d730-4832-a71c-d45a2f3ce406\") " pod="openstack-kuttl-tests/barbican02cc-account-delete-gf94b" Dec 03 12:48:58 crc kubenswrapper[5052]: I1203 12:48:58.893058 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_ovsdbserver-sb-0_84f26afe-5b0c-43bb-a4c8-a9ac21d24330/ovsdbserver-sb/0.log" Dec 03 12:48:58 crc kubenswrapper[5052]: I1203 12:48:58.893109 5052 generic.go:334] "Generic (PLEG): container finished" podID="84f26afe-5b0c-43bb-a4c8-a9ac21d24330" containerID="721e21ea9214ab0b7f428a1744d9bf1b20c7493291c82e15a89dd5d11891df97" exitCode=2 Dec 03 12:48:58 crc kubenswrapper[5052]: I1203 12:48:58.893128 5052 generic.go:334] "Generic (PLEG): container finished" podID="84f26afe-5b0c-43bb-a4c8-a9ac21d24330" containerID="b74065369c814d7538cce05dc1a2188d258f8a0970dbde7fa52adce059389dc5" exitCode=143 Dec 03 12:48:58 crc kubenswrapper[5052]: I1203 12:48:58.893152 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-sb-0" event={"ID":"84f26afe-5b0c-43bb-a4c8-a9ac21d24330","Type":"ContainerDied","Data":"721e21ea9214ab0b7f428a1744d9bf1b20c7493291c82e15a89dd5d11891df97"} Dec 03 12:48:58 crc kubenswrapper[5052]: I1203 12:48:58.893182 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-sb-0" event={"ID":"84f26afe-5b0c-43bb-a4c8-a9ac21d24330","Type":"ContainerDied","Data":"b74065369c814d7538cce05dc1a2188d258f8a0970dbde7fa52adce059389dc5"} Dec 03 12:48:58 crc kubenswrapper[5052]: I1203 12:48:58.908990 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell0-cell-mapping-jq8w4"] Dec 03 12:48:58 crc kubenswrapper[5052]: I1203 12:48:58.929055 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:48:58 crc kubenswrapper[5052]: I1203 12:48:58.929633 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-external-api-0" podUID="b1a74998-b009-4bb6-a0dc-d94ffae8b641" containerName="glance-log" containerID="cri-o://9a055bdf916ddbec20efe694430ce069882c0ee2f3b4626ae5ce60fc3fd5c826" gracePeriod=30 Dec 03 12:48:58 crc kubenswrapper[5052]: I1203 12:48:58.933656 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-external-api-0" podUID="b1a74998-b009-4bb6-a0dc-d94ffae8b641" containerName="glance-httpd" containerID="cri-o://f28ffd80db46452a6cf33fcc0825d49c5f14be470338b023f15a46aa3b004932" gracePeriod=30 Dec 03 12:48:58 crc kubenswrapper[5052]: I1203 12:48:58.947778 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-5mh5h\" (UniqueName: \"kubernetes.io/projected/857b8a8a-d730-4832-a71c-d45a2f3ce406-kube-api-access-5mh5h\") pod \"barbican02cc-account-delete-gf94b\" (UID: \"857b8a8a-d730-4832-a71c-d45a2f3ce406\") " pod="openstack-kuttl-tests/barbican02cc-account-delete-gf94b" Dec 03 12:48:58 crc kubenswrapper[5052]: I1203 12:48:58.967787 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell0-cell-mapping-jq8w4"] Dec 03 12:48:58 crc kubenswrapper[5052]: E1203 12:48:58.981642 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/rabbitmq-config-data: configmap "rabbitmq-config-data" not found Dec 03 12:48:58 crc kubenswrapper[5052]: E1203 12:48:58.981706 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0cb33a40-6f0a-4e89-a159-4edd3af3a2fa-config-data podName:0cb33a40-6f0a-4e89-a159-4edd3af3a2fa nodeName:}" failed. No retries permitted until 2025-12-03 12:49:00.981692167 +0000 UTC m=+5331.569933931 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "config-data" (UniqueName: "kubernetes.io/configmap/0cb33a40-6f0a-4e89-a159-4edd3af3a2fa-config-data") pod "rabbitmq-server-0" (UID: "0cb33a40-6f0a-4e89-a159-4edd3af3a2fa") : configmap "rabbitmq-config-data" not found Dec 03 12:48:59 crc kubenswrapper[5052]: I1203 12:48:59.012596 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/swift-ring-rebalance-kvhc2"] Dec 03 12:48:59 crc kubenswrapper[5052]: I1203 12:48:59.036836 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/novaapifd1c-account-delete-vhjzv"] Dec 03 12:48:59 crc kubenswrapper[5052]: I1203 12:48:59.038675 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/novaapifd1c-account-delete-vhjzv" Dec 03 12:48:59 crc kubenswrapper[5052]: I1203 12:48:59.052925 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell1-cell-mapping-rztxs"] Dec 03 12:48:59 crc kubenswrapper[5052]: I1203 12:48:59.072017 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell1-cell-mapping-rztxs"] Dec 03 12:48:59 crc kubenswrapper[5052]: I1203 12:48:59.091559 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" podUID="488c3529-4c2b-47c0-9dcf-05a064c34d0d" containerName="rabbitmq" containerID="cri-o://dfdff0317f61d3fa549a21a9c81b61a342d0bac7de867324e414cf592af4d97d" gracePeriod=604800 Dec 03 12:48:59 crc kubenswrapper[5052]: I1203 12:48:59.101003 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/novaapifd1c-account-delete-vhjzv"] Dec 03 12:48:59 crc kubenswrapper[5052]: I1203 12:48:59.139927 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 12:48:59 crc kubenswrapper[5052]: I1203 12:48:59.140270 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/cinder-scheduler-0" podUID="aa557411-6b0a-4596-a5f5-0e23f8686dc4" containerName="cinder-scheduler" containerID="cri-o://28dad3a0f48d004bf9433a47f2e9bb435f9945510d0f67da8d4246236a90195d" gracePeriod=30 Dec 03 12:48:59 crc kubenswrapper[5052]: I1203 12:48:59.140530 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/cinder-scheduler-0" podUID="aa557411-6b0a-4596-a5f5-0e23f8686dc4" containerName="probe" containerID="cri-o://1d9679ea8e92979b2a265175ba31597cc56e80e13b2610ba8cc4b4e015500253" gracePeriod=30 Dec 03 12:48:59 crc kubenswrapper[5052]: I1203 12:48:59.184742 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/neutron-fdd8f875d-xvq79"] Dec 03 12:48:59 crc kubenswrapper[5052]: I1203 12:48:59.185008 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/neutron-fdd8f875d-xvq79" podUID="77825998-48a8-496e-a523-8e0c371fe532" containerName="neutron-api" containerID="cri-o://15655b574d1d35cf1c932536528e9b95711b3ebc8dd150537384f5080772cd5e" gracePeriod=30 Dec 03 12:48:59 crc kubenswrapper[5052]: I1203 12:48:59.185578 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/neutron-fdd8f875d-xvq79" podUID="77825998-48a8-496e-a523-8e0c371fe532" containerName="neutron-httpd" containerID="cri-o://c79f26b11c1bba88e99d71d696bb0decec843bf868e7a4fdf02ceb21feddb964" gracePeriod=30 Dec 03 12:48:59 crc kubenswrapper[5052]: I1203 12:48:59.194728 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8jp9d\" (UniqueName: \"kubernetes.io/projected/2846c397-2869-49f3-aeca-cf4b56bc8834-kube-api-access-8jp9d\") pod \"novaapifd1c-account-delete-vhjzv\" (UID: \"2846c397-2869-49f3-aeca-cf4b56bc8834\") " pod="openstack-kuttl-tests/novaapifd1c-account-delete-vhjzv" Dec 03 12:48:59 crc kubenswrapper[5052]: I1203 12:48:59.195217 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/2846c397-2869-49f3-aeca-cf4b56bc8834-operator-scripts\") pod \"novaapifd1c-account-delete-vhjzv\" (UID: \"2846c397-2869-49f3-aeca-cf4b56bc8834\") " pod="openstack-kuttl-tests/novaapifd1c-account-delete-vhjzv" Dec 03 12:48:59 crc kubenswrapper[5052]: I1203 12:48:59.224677 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-9t4zp"] Dec 03 12:48:59 crc kubenswrapper[5052]: I1203 12:48:59.226344 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-9t4zp" Dec 03 12:48:59 crc kubenswrapper[5052]: I1203 12:48:59.238143 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican02cc-account-delete-gf94b" Dec 03 12:48:59 crc kubenswrapper[5052]: I1203 12:48:59.299977 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-8jp9d\" (UniqueName: \"kubernetes.io/projected/2846c397-2869-49f3-aeca-cf4b56bc8834-kube-api-access-8jp9d\") pod \"novaapifd1c-account-delete-vhjzv\" (UID: \"2846c397-2869-49f3-aeca-cf4b56bc8834\") " pod="openstack-kuttl-tests/novaapifd1c-account-delete-vhjzv" Dec 03 12:48:59 crc kubenswrapper[5052]: I1203 12:48:59.300121 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/2846c397-2869-49f3-aeca-cf4b56bc8834-operator-scripts\") pod \"novaapifd1c-account-delete-vhjzv\" (UID: \"2846c397-2869-49f3-aeca-cf4b56bc8834\") " pod="openstack-kuttl-tests/novaapifd1c-account-delete-vhjzv" Dec 03 12:48:59 crc kubenswrapper[5052]: I1203 12:48:59.306560 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/2846c397-2869-49f3-aeca-cf4b56bc8834-operator-scripts\") pod \"novaapifd1c-account-delete-vhjzv\" (UID: \"2846c397-2869-49f3-aeca-cf4b56bc8834\") " pod="openstack-kuttl-tests/novaapifd1c-account-delete-vhjzv" Dec 03 12:48:59 crc kubenswrapper[5052]: I1203 12:48:59.332403 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-8jp9d\" (UniqueName: \"kubernetes.io/projected/2846c397-2869-49f3-aeca-cf4b56bc8834-kube-api-access-8jp9d\") pod \"novaapifd1c-account-delete-vhjzv\" (UID: \"2846c397-2869-49f3-aeca-cf4b56bc8834\") " pod="openstack-kuttl-tests/novaapifd1c-account-delete-vhjzv" Dec 03 12:48:59 crc kubenswrapper[5052]: I1203 12:48:59.351266 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-9t4zp"] Dec 03 12:48:59 crc kubenswrapper[5052]: I1203 12:48:59.394050 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/novacell05566-account-delete-2f4dm"] Dec 03 12:48:59 crc kubenswrapper[5052]: I1203 12:48:59.396133 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/novacell05566-account-delete-2f4dm" Dec 03 12:48:59 crc kubenswrapper[5052]: I1203 12:48:59.402050 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/59ec248d-84c7-4d23-b9f6-29ea8c2f36fe-config\") pod \"dnsmasq-dnsmasq-84b9f45d47-9t4zp\" (UID: \"59ec248d-84c7-4d23-b9f6-29ea8c2f36fe\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-9t4zp" Dec 03 12:48:59 crc kubenswrapper[5052]: I1203 12:48:59.402116 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/59ec248d-84c7-4d23-b9f6-29ea8c2f36fe-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-84b9f45d47-9t4zp\" (UID: \"59ec248d-84c7-4d23-b9f6-29ea8c2f36fe\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-9t4zp" Dec 03 12:48:59 crc kubenswrapper[5052]: I1203 12:48:59.402681 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lj2nq\" (UniqueName: \"kubernetes.io/projected/59ec248d-84c7-4d23-b9f6-29ea8c2f36fe-kube-api-access-lj2nq\") pod \"dnsmasq-dnsmasq-84b9f45d47-9t4zp\" (UID: \"59ec248d-84c7-4d23-b9f6-29ea8c2f36fe\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-9t4zp" Dec 03 12:48:59 crc kubenswrapper[5052]: I1203 12:48:59.406008 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 12:48:59 crc kubenswrapper[5052]: I1203 12:48:59.406258 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/cinder-api-0" podUID="557175a9-81e5-4605-bd67-f1dfa575fb1b" containerName="cinder-api-log" containerID="cri-o://60ff2fa7c14572046a3621f0f09cee6b8670305019e8df8b2b52c2d7c1c39683" gracePeriod=30 Dec 03 12:48:59 crc kubenswrapper[5052]: I1203 12:48:59.406376 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/cinder-api-0" podUID="557175a9-81e5-4605-bd67-f1dfa575fb1b" containerName="cinder-api" containerID="cri-o://4f6a6de545ed43fd89fd3c33b250e3e6c4f74774fb05bed0727171451864fd53" gracePeriod=30 Dec 03 12:48:59 crc kubenswrapper[5052]: I1203 12:48:59.427988 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/novacell05566-account-delete-2f4dm"] Dec 03 12:48:59 crc kubenswrapper[5052]: I1203 12:48:59.470827 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/rabbitmq-server-0"] Dec 03 12:48:59 crc kubenswrapper[5052]: I1203 12:48:59.501361 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:48:59 crc kubenswrapper[5052]: I1203 12:48:59.501712 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-internal-api-0" podUID="098a46f8-2d49-4fea-a95c-5678601aa164" containerName="glance-log" containerID="cri-o://786d9178f57d4b0d285a0e72f6cd93f8939028d39964463dca3a233c79a6225d" gracePeriod=30 Dec 03 12:48:59 crc kubenswrapper[5052]: I1203 12:48:59.501965 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-internal-api-0" podUID="098a46f8-2d49-4fea-a95c-5678601aa164" containerName="glance-httpd" containerID="cri-o://3a02ca07996460062957faa0b466285e3ee2c66925c520449ab5df8d346825d8" gracePeriod=30 Dec 03 12:48:59 crc kubenswrapper[5052]: I1203 12:48:59.508147 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z7sjx\" (UniqueName: \"kubernetes.io/projected/a2bc501c-ebef-4e68-8ae2-e8c4217cadff-kube-api-access-z7sjx\") pod \"novacell05566-account-delete-2f4dm\" (UID: \"a2bc501c-ebef-4e68-8ae2-e8c4217cadff\") " pod="openstack-kuttl-tests/novacell05566-account-delete-2f4dm" Dec 03 12:48:59 crc kubenswrapper[5052]: I1203 12:48:59.508281 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/a2bc501c-ebef-4e68-8ae2-e8c4217cadff-operator-scripts\") pod \"novacell05566-account-delete-2f4dm\" (UID: \"a2bc501c-ebef-4e68-8ae2-e8c4217cadff\") " pod="openstack-kuttl-tests/novacell05566-account-delete-2f4dm" Dec 03 12:48:59 crc kubenswrapper[5052]: I1203 12:48:59.508337 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-lj2nq\" (UniqueName: \"kubernetes.io/projected/59ec248d-84c7-4d23-b9f6-29ea8c2f36fe-kube-api-access-lj2nq\") pod \"dnsmasq-dnsmasq-84b9f45d47-9t4zp\" (UID: \"59ec248d-84c7-4d23-b9f6-29ea8c2f36fe\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-9t4zp" Dec 03 12:48:59 crc kubenswrapper[5052]: I1203 12:48:59.508512 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/59ec248d-84c7-4d23-b9f6-29ea8c2f36fe-config\") pod \"dnsmasq-dnsmasq-84b9f45d47-9t4zp\" (UID: \"59ec248d-84c7-4d23-b9f6-29ea8c2f36fe\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-9t4zp" Dec 03 12:48:59 crc kubenswrapper[5052]: I1203 12:48:59.508581 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/59ec248d-84c7-4d23-b9f6-29ea8c2f36fe-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-84b9f45d47-9t4zp\" (UID: \"59ec248d-84c7-4d23-b9f6-29ea8c2f36fe\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-9t4zp" Dec 03 12:48:59 crc kubenswrapper[5052]: I1203 12:48:59.513620 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/59ec248d-84c7-4d23-b9f6-29ea8c2f36fe-config\") pod \"dnsmasq-dnsmasq-84b9f45d47-9t4zp\" (UID: \"59ec248d-84c7-4d23-b9f6-29ea8c2f36fe\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-9t4zp" Dec 03 12:48:59 crc kubenswrapper[5052]: I1203 12:48:59.514471 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/59ec248d-84c7-4d23-b9f6-29ea8c2f36fe-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-84b9f45d47-9t4zp\" (UID: \"59ec248d-84c7-4d23-b9f6-29ea8c2f36fe\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-9t4zp" Dec 03 12:48:59 crc kubenswrapper[5052]: I1203 12:48:59.552171 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-lj2nq\" (UniqueName: \"kubernetes.io/projected/59ec248d-84c7-4d23-b9f6-29ea8c2f36fe-kube-api-access-lj2nq\") pod \"dnsmasq-dnsmasq-84b9f45d47-9t4zp\" (UID: \"59ec248d-84c7-4d23-b9f6-29ea8c2f36fe\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-9t4zp" Dec 03 12:48:59 crc kubenswrapper[5052]: I1203 12:48:59.557316 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/swift-proxy-78f78b88b6-fmjvp"] Dec 03 12:48:59 crc kubenswrapper[5052]: I1203 12:48:59.557649 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-proxy-78f78b88b6-fmjvp" podUID="dd8df313-3680-4dd1-a57d-ff85b0b2c6f1" containerName="proxy-httpd" containerID="cri-o://acb0be96c14d669ff7b80a9476845494757b206ec5aaddb2a34ac9ff0fe87bd7" gracePeriod=30 Dec 03 12:48:59 crc kubenswrapper[5052]: I1203 12:48:59.558234 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-proxy-78f78b88b6-fmjvp" podUID="dd8df313-3680-4dd1-a57d-ff85b0b2c6f1" containerName="proxy-server" containerID="cri-o://45b85b0267828ad53c05c095096047353cbe686d4edc60d6dabed09ff937a326" gracePeriod=30 Dec 03 12:48:59 crc kubenswrapper[5052]: I1203 12:48:59.583378 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:48:59 crc kubenswrapper[5052]: I1203 12:48:59.583625 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-api-0" podUID="6c8ad676-e7dd-479c-8d22-f546dc5e50af" containerName="nova-api-log" containerID="cri-o://a3f4f93f3787a5482025f013961d9f934ba51e800bf3b24730bce1b1e2696ee6" gracePeriod=30 Dec 03 12:48:59 crc kubenswrapper[5052]: I1203 12:48:59.584057 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-api-0" podUID="6c8ad676-e7dd-479c-8d22-f546dc5e50af" containerName="nova-api-api" containerID="cri-o://f38a12088ebf63cf61a94810998aa693db18e09b88bf0391d892ee960b99e9ce" gracePeriod=30 Dec 03 12:48:59 crc kubenswrapper[5052]: I1203 12:48:59.613208 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-z7sjx\" (UniqueName: \"kubernetes.io/projected/a2bc501c-ebef-4e68-8ae2-e8c4217cadff-kube-api-access-z7sjx\") pod \"novacell05566-account-delete-2f4dm\" (UID: \"a2bc501c-ebef-4e68-8ae2-e8c4217cadff\") " pod="openstack-kuttl-tests/novacell05566-account-delete-2f4dm" Dec 03 12:48:59 crc kubenswrapper[5052]: I1203 12:48:59.613280 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/a2bc501c-ebef-4e68-8ae2-e8c4217cadff-operator-scripts\") pod \"novacell05566-account-delete-2f4dm\" (UID: \"a2bc501c-ebef-4e68-8ae2-e8c4217cadff\") " pod="openstack-kuttl-tests/novacell05566-account-delete-2f4dm" Dec 03 12:48:59 crc kubenswrapper[5052]: I1203 12:48:59.622574 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/openstack-cell1-galera-0"] Dec 03 12:48:59 crc kubenswrapper[5052]: I1203 12:48:59.632392 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/a2bc501c-ebef-4e68-8ae2-e8c4217cadff-operator-scripts\") pod \"novacell05566-account-delete-2f4dm\" (UID: \"a2bc501c-ebef-4e68-8ae2-e8c4217cadff\") " pod="openstack-kuttl-tests/novacell05566-account-delete-2f4dm" Dec 03 12:48:59 crc kubenswrapper[5052]: I1203 12:48:59.686602 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-z7sjx\" (UniqueName: \"kubernetes.io/projected/a2bc501c-ebef-4e68-8ae2-e8c4217cadff-kube-api-access-z7sjx\") pod \"novacell05566-account-delete-2f4dm\" (UID: \"a2bc501c-ebef-4e68-8ae2-e8c4217cadff\") " pod="openstack-kuttl-tests/novacell05566-account-delete-2f4dm" Dec 03 12:48:59 crc kubenswrapper[5052]: I1203 12:48:59.937864 5052 generic.go:334] "Generic (PLEG): container finished" podID="6c8ad676-e7dd-479c-8d22-f546dc5e50af" containerID="a3f4f93f3787a5482025f013961d9f934ba51e800bf3b24730bce1b1e2696ee6" exitCode=143 Dec 03 12:48:59 crc kubenswrapper[5052]: I1203 12:48:59.939374 5052 generic.go:334] "Generic (PLEG): container finished" podID="8a94fda3-63d8-4517-8063-f6c06688b430" containerID="e3c10016f8e333ebc2b5eef7809847e5f7ea2904a0f92aa3ddc61a80ff2d9403" exitCode=137 Dec 03 12:48:59 crc kubenswrapper[5052]: I1203 12:48:59.940869 5052 generic.go:334] "Generic (PLEG): container finished" podID="098a46f8-2d49-4fea-a95c-5678601aa164" containerID="786d9178f57d4b0d285a0e72f6cd93f8939028d39964463dca3a233c79a6225d" exitCode=143 Dec 03 12:48:59 crc kubenswrapper[5052]: I1203 12:48:59.955273 5052 generic.go:334] "Generic (PLEG): container finished" podID="557175a9-81e5-4605-bd67-f1dfa575fb1b" containerID="60ff2fa7c14572046a3621f0f09cee6b8670305019e8df8b2b52c2d7c1c39683" exitCode=143 Dec 03 12:48:59 crc kubenswrapper[5052]: I1203 12:48:59.959386 5052 generic.go:334] "Generic (PLEG): container finished" podID="5201872c-fde1-49d6-9784-6a7f54497990" containerID="775232fb24de37720556752c4b1bc2dd4ea9fd1233582be2d16b27194e0a1798" exitCode=143 Dec 03 12:48:59 crc kubenswrapper[5052]: I1203 12:48:59.965251 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0063a179-2799-4532-9dd4-789e55b489be" path="/var/lib/kubelet/pods/0063a179-2799-4532-9dd4-789e55b489be/volumes" Dec 03 12:48:59 crc kubenswrapper[5052]: I1203 12:48:59.965973 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="15239b5b-4c53-4ab2-aadf-877baf8e7b0b" path="/var/lib/kubelet/pods/15239b5b-4c53-4ab2-aadf-877baf8e7b0b/volumes" Dec 03 12:48:59 crc kubenswrapper[5052]: I1203 12:48:59.966271 5052 generic.go:334] "Generic (PLEG): container finished" podID="77825998-48a8-496e-a523-8e0c371fe532" containerID="c79f26b11c1bba88e99d71d696bb0decec843bf868e7a4fdf02ceb21feddb964" exitCode=0 Dec 03 12:48:59 crc kubenswrapper[5052]: I1203 12:48:59.966717 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1de481f1-0c54-48c3-b112-659497d86792" path="/var/lib/kubelet/pods/1de481f1-0c54-48c3-b112-659497d86792/volumes" Dec 03 12:48:59 crc kubenswrapper[5052]: I1203 12:48:59.968128 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6a6cd849-4cae-4f8f-8a79-1ebf7fb284ee" path="/var/lib/kubelet/pods/6a6cd849-4cae-4f8f-8a79-1ebf7fb284ee/volumes" Dec 03 12:48:59 crc kubenswrapper[5052]: I1203 12:48:59.968969 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="826d1642-afe5-4638-81e5-0d5b046e411e" path="/var/lib/kubelet/pods/826d1642-afe5-4638-81e5-0d5b046e411e/volumes" Dec 03 12:48:59 crc kubenswrapper[5052]: I1203 12:48:59.969665 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8a321082-482c-4805-b252-88a304bb19ec" path="/var/lib/kubelet/pods/8a321082-482c-4805-b252-88a304bb19ec/volumes" Dec 03 12:48:59 crc kubenswrapper[5052]: I1203 12:48:59.970385 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8d5c589d-50a2-4bdb-ac78-59c1592c03e8" path="/var/lib/kubelet/pods/8d5c589d-50a2-4bdb-ac78-59c1592c03e8/volumes" Dec 03 12:48:59 crc kubenswrapper[5052]: I1203 12:48:59.975548 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e8677b10-c98b-4d0e-a5d7-7af9fd914f12" path="/var/lib/kubelet/pods/e8677b10-c98b-4d0e-a5d7-7af9fd914f12/volumes" Dec 03 12:48:59 crc kubenswrapper[5052]: I1203 12:48:59.976263 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"6c8ad676-e7dd-479c-8d22-f546dc5e50af","Type":"ContainerDied","Data":"a3f4f93f3787a5482025f013961d9f934ba51e800bf3b24730bce1b1e2696ee6"} Dec 03 12:48:59 crc kubenswrapper[5052]: I1203 12:48:59.976305 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-worker-58f7cb7fd9-2k5tg"] Dec 03 12:48:59 crc kubenswrapper[5052]: I1203 12:48:59.976336 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"098a46f8-2d49-4fea-a95c-5678601aa164","Type":"ContainerDied","Data":"786d9178f57d4b0d285a0e72f6cd93f8939028d39964463dca3a233c79a6225d"} Dec 03 12:48:59 crc kubenswrapper[5052]: I1203 12:48:59.976350 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:48:59 crc kubenswrapper[5052]: I1203 12:48:59.976368 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-keystone-listener-68fd6d997d-7tjjw"] Dec 03 12:48:59 crc kubenswrapper[5052]: I1203 12:48:59.976382 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-api-75bdf7fcbb-mlkgz"] Dec 03 12:48:59 crc kubenswrapper[5052]: I1203 12:48:59.976397 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"557175a9-81e5-4605-bd67-f1dfa575fb1b","Type":"ContainerDied","Data":"60ff2fa7c14572046a3621f0f09cee6b8670305019e8df8b2b52c2d7c1c39683"} Dec 03 12:48:59 crc kubenswrapper[5052]: I1203 12:48:59.976414 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-74746c7f4-h6sx8" event={"ID":"5201872c-fde1-49d6-9784-6a7f54497990","Type":"ContainerDied","Data":"775232fb24de37720556752c4b1bc2dd4ea9fd1233582be2d16b27194e0a1798"} Dec 03 12:48:59 crc kubenswrapper[5052]: I1203 12:48:59.976428 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-fdd8f875d-xvq79" event={"ID":"77825998-48a8-496e-a523-8e0c371fe532","Type":"ContainerDied","Data":"c79f26b11c1bba88e99d71d696bb0decec843bf868e7a4fdf02ceb21feddb964"} Dec 03 12:48:59 crc kubenswrapper[5052]: I1203 12:48:59.976443 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell1-db-create-tkzwz"] Dec 03 12:48:59 crc kubenswrapper[5052]: I1203 12:48:59.976453 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell1-4975-account-create-update-77cc4"] Dec 03 12:48:59 crc kubenswrapper[5052]: I1203 12:48:59.979165 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/barbican-api-75bdf7fcbb-mlkgz" podUID="b8eb690c-165a-4906-a750-bec0178daf6c" containerName="barbican-api-log" containerID="cri-o://1986f1fabd06880b71b76b4c94e01ab46c4e8d1cff3eae60d3b1cd7579b2c785" gracePeriod=30 Dec 03 12:48:59 crc kubenswrapper[5052]: I1203 12:48:59.979467 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/barbican-worker-58f7cb7fd9-2k5tg" podUID="e14465ed-868a-49ba-b3ab-1e67196229e8" containerName="barbican-worker-log" containerID="cri-o://805cd92e51a39e19c7bde5260dca999018b3ba92ebb95c79b560a3986eb6d94a" gracePeriod=30 Dec 03 12:48:59 crc kubenswrapper[5052]: I1203 12:48:59.979662 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-metadata-0" podUID="efb0e60c-2a71-4641-87b2-0f5532b28678" containerName="nova-metadata-log" containerID="cri-o://d42c23e387ad7773d4b53a13fb7267499e89a43ce6fa0b570fa164ddcc0ed712" gracePeriod=30 Dec 03 12:48:59 crc kubenswrapper[5052]: I1203 12:48:59.979816 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/barbican-keystone-listener-68fd6d997d-7tjjw" podUID="2065f5bb-88eb-4e84-88b6-9982833d413d" containerName="barbican-keystone-listener-log" containerID="cri-o://2e1db89e52b83cc4d03ffdc49b1a121ba636bb18c0424300ea015664b45cff9a" gracePeriod=30 Dec 03 12:48:59 crc kubenswrapper[5052]: I1203 12:48:59.980294 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/barbican-api-75bdf7fcbb-mlkgz" podUID="b8eb690c-165a-4906-a750-bec0178daf6c" containerName="barbican-api" containerID="cri-o://275c7f7b179720a472f88be78dd4725e7d963350bb431c4a491cf0a780e6019a" gracePeriod=30 Dec 03 12:48:59 crc kubenswrapper[5052]: I1203 12:48:59.980482 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/barbican-worker-58f7cb7fd9-2k5tg" podUID="e14465ed-868a-49ba-b3ab-1e67196229e8" containerName="barbican-worker" containerID="cri-o://962dbb81a097bd1736b9740b6fea46c9644f48229fd42d2dc734698665523114" gracePeriod=30 Dec 03 12:48:59 crc kubenswrapper[5052]: I1203 12:48:59.980590 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-metadata-0" podUID="efb0e60c-2a71-4641-87b2-0f5532b28678" containerName="nova-metadata-metadata" containerID="cri-o://fde954c2e50f163494a08fec446c2659dbced2f64031d8d4fb5557a44bc92f99" gracePeriod=30 Dec 03 12:48:59 crc kubenswrapper[5052]: I1203 12:48:59.980639 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/barbican-keystone-listener-68fd6d997d-7tjjw" podUID="2065f5bb-88eb-4e84-88b6-9982833d413d" containerName="barbican-keystone-listener" containerID="cri-o://92eeaf97de94d65f0f332a8172dd776af0421be37c42d90e9dab8739d2cb83e3" gracePeriod=30 Dec 03 12:48:59 crc kubenswrapper[5052]: I1203 12:48:59.983684 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell1-4975-account-create-update-77cc4"] Dec 03 12:48:59 crc kubenswrapper[5052]: I1203 12:48:59.990443 5052 generic.go:334] "Generic (PLEG): container finished" podID="b1a74998-b009-4bb6-a0dc-d94ffae8b641" containerID="9a055bdf916ddbec20efe694430ce069882c0ee2f3b4626ae5ce60fc3fd5c826" exitCode=143 Dec 03 12:48:59 crc kubenswrapper[5052]: I1203 12:48:59.990507 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"b1a74998-b009-4bb6-a0dc-d94ffae8b641","Type":"ContainerDied","Data":"9a055bdf916ddbec20efe694430ce069882c0ee2f3b4626ae5ce60fc3fd5c826"} Dec 03 12:49:00 crc kubenswrapper[5052]: I1203 12:49:00.000967 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell1-db-create-tkzwz"] Dec 03 12:49:00 crc kubenswrapper[5052]: I1203 12:49:00.007290 5052 generic.go:334] "Generic (PLEG): container finished" podID="bd4392a2-9c70-4f3a-9050-ae6a6db01082" containerID="44d343c040a52a96a2507d69db579564483888cfc2ab0051211cae74e09f32f3" exitCode=0 Dec 03 12:49:00 crc kubenswrapper[5052]: I1203 12:49:00.007316 5052 generic.go:334] "Generic (PLEG): container finished" podID="bd4392a2-9c70-4f3a-9050-ae6a6db01082" containerID="ef01830e5c368fdf7b4e0af6d27790eef08c24d4f1d55b39ccbab91a7ba97c10" exitCode=0 Dec 03 12:49:00 crc kubenswrapper[5052]: I1203 12:49:00.007329 5052 generic.go:334] "Generic (PLEG): container finished" podID="bd4392a2-9c70-4f3a-9050-ae6a6db01082" containerID="4f5b00a3217544c70607eacf64edc7dd926767e810b84e309cbc071e09b652d5" exitCode=0 Dec 03 12:49:00 crc kubenswrapper[5052]: I1203 12:49:00.007339 5052 generic.go:334] "Generic (PLEG): container finished" podID="bd4392a2-9c70-4f3a-9050-ae6a6db01082" containerID="d17825037af7c0e226677e493c307a512bb167e46667dd9c705ac03cff6758bb" exitCode=0 Dec 03 12:49:00 crc kubenswrapper[5052]: I1203 12:49:00.007351 5052 generic.go:334] "Generic (PLEG): container finished" podID="bd4392a2-9c70-4f3a-9050-ae6a6db01082" containerID="257a3713d34f8297bc83998342be63e79d0964dc86ce021417f8cb3fc862ff41" exitCode=0 Dec 03 12:49:00 crc kubenswrapper[5052]: I1203 12:49:00.007359 5052 generic.go:334] "Generic (PLEG): container finished" podID="bd4392a2-9c70-4f3a-9050-ae6a6db01082" containerID="073f8906a127eaad4d2dcbdbaf2738671488e4b5da8001af8eb90b508703cf7a" exitCode=0 Dec 03 12:49:00 crc kubenswrapper[5052]: I1203 12:49:00.007369 5052 generic.go:334] "Generic (PLEG): container finished" podID="bd4392a2-9c70-4f3a-9050-ae6a6db01082" containerID="0420f9807b79aa9a35f7e00db76c776194c37e61b2e6f2e9e3e83e3f355d270c" exitCode=0 Dec 03 12:49:00 crc kubenswrapper[5052]: I1203 12:49:00.007377 5052 generic.go:334] "Generic (PLEG): container finished" podID="bd4392a2-9c70-4f3a-9050-ae6a6db01082" containerID="49448581e8cc46afbfef30edf584ba024f9fdfe5baf0de42a0abed3885f0ef23" exitCode=0 Dec 03 12:49:00 crc kubenswrapper[5052]: I1203 12:49:00.007387 5052 generic.go:334] "Generic (PLEG): container finished" podID="bd4392a2-9c70-4f3a-9050-ae6a6db01082" containerID="4aa02cc6c01a66c5bcc1c0d61505c02dbf6374ff0a6550b9d548733640117138" exitCode=0 Dec 03 12:49:00 crc kubenswrapper[5052]: I1203 12:49:00.007395 5052 generic.go:334] "Generic (PLEG): container finished" podID="bd4392a2-9c70-4f3a-9050-ae6a6db01082" containerID="2eaf7736f8bbe27fc3320dbe4500d1c1b7388bb8194a9d8a9bb7ef68f452b97f" exitCode=0 Dec 03 12:49:00 crc kubenswrapper[5052]: I1203 12:49:00.007405 5052 generic.go:334] "Generic (PLEG): container finished" podID="bd4392a2-9c70-4f3a-9050-ae6a6db01082" containerID="409983b9eafc0681b072e5508199566f3fbeaf0cc55e055beaee8282045a003e" exitCode=0 Dec 03 12:49:00 crc kubenswrapper[5052]: I1203 12:49:00.007413 5052 generic.go:334] "Generic (PLEG): container finished" podID="bd4392a2-9c70-4f3a-9050-ae6a6db01082" containerID="e4afa7a612ce1826fda8750703ccc15d8110e17d1d419c076a60957b33af8adb" exitCode=0 Dec 03 12:49:00 crc kubenswrapper[5052]: I1203 12:49:00.007423 5052 generic.go:334] "Generic (PLEG): container finished" podID="bd4392a2-9c70-4f3a-9050-ae6a6db01082" containerID="8ea80229aee1ce207c0394efd61dfb519308fde6a2a25c517795af19f4c72da4" exitCode=0 Dec 03 12:49:00 crc kubenswrapper[5052]: I1203 12:49:00.007432 5052 generic.go:334] "Generic (PLEG): container finished" podID="bd4392a2-9c70-4f3a-9050-ae6a6db01082" containerID="ee68253d5346b57e39fc0b0fed73d3c548e33dc2c36cd2673c5add72f235bb79" exitCode=0 Dec 03 12:49:00 crc kubenswrapper[5052]: I1203 12:49:00.007454 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"bd4392a2-9c70-4f3a-9050-ae6a6db01082","Type":"ContainerDied","Data":"44d343c040a52a96a2507d69db579564483888cfc2ab0051211cae74e09f32f3"} Dec 03 12:49:00 crc kubenswrapper[5052]: I1203 12:49:00.007483 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"bd4392a2-9c70-4f3a-9050-ae6a6db01082","Type":"ContainerDied","Data":"ef01830e5c368fdf7b4e0af6d27790eef08c24d4f1d55b39ccbab91a7ba97c10"} Dec 03 12:49:00 crc kubenswrapper[5052]: I1203 12:49:00.007499 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"bd4392a2-9c70-4f3a-9050-ae6a6db01082","Type":"ContainerDied","Data":"4f5b00a3217544c70607eacf64edc7dd926767e810b84e309cbc071e09b652d5"} Dec 03 12:49:00 crc kubenswrapper[5052]: I1203 12:49:00.007515 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"bd4392a2-9c70-4f3a-9050-ae6a6db01082","Type":"ContainerDied","Data":"d17825037af7c0e226677e493c307a512bb167e46667dd9c705ac03cff6758bb"} Dec 03 12:49:00 crc kubenswrapper[5052]: I1203 12:49:00.007731 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"bd4392a2-9c70-4f3a-9050-ae6a6db01082","Type":"ContainerDied","Data":"257a3713d34f8297bc83998342be63e79d0964dc86ce021417f8cb3fc862ff41"} Dec 03 12:49:00 crc kubenswrapper[5052]: I1203 12:49:00.007755 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"bd4392a2-9c70-4f3a-9050-ae6a6db01082","Type":"ContainerDied","Data":"073f8906a127eaad4d2dcbdbaf2738671488e4b5da8001af8eb90b508703cf7a"} Dec 03 12:49:00 crc kubenswrapper[5052]: I1203 12:49:00.007768 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"bd4392a2-9c70-4f3a-9050-ae6a6db01082","Type":"ContainerDied","Data":"0420f9807b79aa9a35f7e00db76c776194c37e61b2e6f2e9e3e83e3f355d270c"} Dec 03 12:49:00 crc kubenswrapper[5052]: I1203 12:49:00.007780 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"bd4392a2-9c70-4f3a-9050-ae6a6db01082","Type":"ContainerDied","Data":"49448581e8cc46afbfef30edf584ba024f9fdfe5baf0de42a0abed3885f0ef23"} Dec 03 12:49:00 crc kubenswrapper[5052]: I1203 12:49:00.007791 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"bd4392a2-9c70-4f3a-9050-ae6a6db01082","Type":"ContainerDied","Data":"4aa02cc6c01a66c5bcc1c0d61505c02dbf6374ff0a6550b9d548733640117138"} Dec 03 12:49:00 crc kubenswrapper[5052]: I1203 12:49:00.007806 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"bd4392a2-9c70-4f3a-9050-ae6a6db01082","Type":"ContainerDied","Data":"2eaf7736f8bbe27fc3320dbe4500d1c1b7388bb8194a9d8a9bb7ef68f452b97f"} Dec 03 12:49:00 crc kubenswrapper[5052]: I1203 12:49:00.007817 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"bd4392a2-9c70-4f3a-9050-ae6a6db01082","Type":"ContainerDied","Data":"409983b9eafc0681b072e5508199566f3fbeaf0cc55e055beaee8282045a003e"} Dec 03 12:49:00 crc kubenswrapper[5052]: I1203 12:49:00.007828 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"bd4392a2-9c70-4f3a-9050-ae6a6db01082","Type":"ContainerDied","Data":"e4afa7a612ce1826fda8750703ccc15d8110e17d1d419c076a60957b33af8adb"} Dec 03 12:49:00 crc kubenswrapper[5052]: I1203 12:49:00.007839 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"bd4392a2-9c70-4f3a-9050-ae6a6db01082","Type":"ContainerDied","Data":"8ea80229aee1ce207c0394efd61dfb519308fde6a2a25c517795af19f4c72da4"} Dec 03 12:49:00 crc kubenswrapper[5052]: I1203 12:49:00.007848 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"bd4392a2-9c70-4f3a-9050-ae6a6db01082","Type":"ContainerDied","Data":"ee68253d5346b57e39fc0b0fed73d3c548e33dc2c36cd2673c5add72f235bb79"} Dec 03 12:49:00 crc kubenswrapper[5052]: I1203 12:49:00.026795 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 12:49:00 crc kubenswrapper[5052]: I1203 12:49:00.027029 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" podUID="edae7848-51ff-40d6-83f2-ffff085f3f3e" containerName="nova-cell1-novncproxy-novncproxy" containerID="cri-o://1470a22e3c4e2dd79a87f38bb53dbab1acc3ac13debd607ce9ac04a443f55380" gracePeriod=30 Dec 03 12:49:00 crc kubenswrapper[5052]: I1203 12:49:00.036077 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-db-sync-pp5gn"] Dec 03 12:49:00 crc kubenswrapper[5052]: I1203 12:49:00.037779 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/rabbitmq-server-0" podUID="0cb33a40-6f0a-4e89-a159-4edd3af3a2fa" containerName="rabbitmq" containerID="cri-o://e4371252da37cb77d701ae89da98daa7bb376b68ff8df323b2456a8b5876c5ba" gracePeriod=604800 Dec 03 12:49:00 crc kubenswrapper[5052]: I1203 12:49:00.066420 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-0"] Dec 03 12:49:00 crc kubenswrapper[5052]: I1203 12:49:00.066721 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-cell1-conductor-0" podUID="8669832d-c47a-49bd-9491-dc7141ac6281" containerName="nova-cell1-conductor-conductor" containerID="cri-o://caf28df7281c196a4c4d326b47a9f4c24851d6defc41fd8fa20f0e5325c7cdac" gracePeriod=30 Dec 03 12:49:00 crc kubenswrapper[5052]: I1203 12:49:00.111963 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-db-sync-pp5gn"] Dec 03 12:49:00 crc kubenswrapper[5052]: I1203 12:49:00.131576 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-0"] Dec 03 12:49:00 crc kubenswrapper[5052]: I1203 12:49:00.131848 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-cell0-conductor-0" podUID="6d653682-52e2-4940-af66-633475bfe09a" containerName="nova-cell0-conductor-conductor" containerID="cri-o://2b5c5249476f9fa6344558a0eabe96f98d744f43f1cd4faf1d66ffa9c49d4941" gracePeriod=30 Dec 03 12:49:00 crc kubenswrapper[5052]: I1203 12:49:00.142147 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-db-sync-ts4vh"] Dec 03 12:49:00 crc kubenswrapper[5052]: I1203 12:49:00.181688 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-db-sync-ts4vh"] Dec 03 12:49:00 crc kubenswrapper[5052]: E1203 12:49:00.212055 5052 cadvisor_stats_provider.go:516] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbd4392a2_9c70_4f3a_9050_ae6a6db01082.slice/crio-2eaf7736f8bbe27fc3320dbe4500d1c1b7388bb8194a9d8a9bb7ef68f452b97f.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8a94fda3_63d8_4517_8063_f6c06688b430.slice/crio-conmon-e3c10016f8e333ebc2b5eef7809847e5f7ea2904a0f92aa3ddc61a80ff2d9403.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbd4392a2_9c70_4f3a_9050_ae6a6db01082.slice/crio-conmon-44d343c040a52a96a2507d69db579564483888cfc2ab0051211cae74e09f32f3.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbd4392a2_9c70_4f3a_9050_ae6a6db01082.slice/crio-conmon-d17825037af7c0e226677e493c307a512bb167e46667dd9c705ac03cff6758bb.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbd4392a2_9c70_4f3a_9050_ae6a6db01082.slice/crio-conmon-073f8906a127eaad4d2dcbdbaf2738671488e4b5da8001af8eb90b508703cf7a.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbd4392a2_9c70_4f3a_9050_ae6a6db01082.slice/crio-conmon-4aa02cc6c01a66c5bcc1c0d61505c02dbf6374ff0a6550b9d548733640117138.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb1a74998_b009_4bb6_a0dc_d94ffae8b641.slice/crio-9a055bdf916ddbec20efe694430ce069882c0ee2f3b4626ae5ce60fc3fd5c826.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5201872c_fde1_49d6_9784_6a7f54497990.slice/crio-775232fb24de37720556752c4b1bc2dd4ea9fd1233582be2d16b27194e0a1798.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod557175a9_81e5_4605_bd67_f1dfa575fb1b.slice/crio-conmon-60ff2fa7c14572046a3621f0f09cee6b8670305019e8df8b2b52c2d7c1c39683.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbd4392a2_9c70_4f3a_9050_ae6a6db01082.slice/crio-44d343c040a52a96a2507d69db579564483888cfc2ab0051211cae74e09f32f3.scope\": RecentStats: unable to find data in memory cache]" Dec 03 12:49:00 crc kubenswrapper[5052]: I1203 12:49:00.222272 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:49:00 crc kubenswrapper[5052]: I1203 12:49:00.222713 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-scheduler-0" podUID="81709983-1fd0-4446-bbbf-fb94653bb3d2" containerName="nova-scheduler-scheduler" containerID="cri-o://6964abaffe4177de7695a956baa51adeedd8fc7ae4b4885ca8e38046eae0369a" gracePeriod=30 Dec 03 12:49:00 crc kubenswrapper[5052]: I1203 12:49:00.255104 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:49:00 crc kubenswrapper[5052]: I1203 12:49:00.256204 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="a69bfb18-1267-4383-b51a-5e0eba70d1bc" containerName="proxy-httpd" containerID="cri-o://74e52eb4f69076f917a1b3ab5d18438ec4c2d708d4bc29581185ab517d52919e" gracePeriod=30 Dec 03 12:49:00 crc kubenswrapper[5052]: I1203 12:49:00.256436 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="a69bfb18-1267-4383-b51a-5e0eba70d1bc" containerName="sg-core" containerID="cri-o://18022901c9e16d4c4eb5c5f284061eb245203a106525b3fb6b0192a33f4b392e" gracePeriod=30 Dec 03 12:49:00 crc kubenswrapper[5052]: I1203 12:49:00.256585 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="a69bfb18-1267-4383-b51a-5e0eba70d1bc" containerName="ceilometer-notification-agent" containerID="cri-o://b1a655b5c3d0ddb68e291790261b40d699c5471d8a606da7a88e93dbced9a7f8" gracePeriod=30 Dec 03 12:49:00 crc kubenswrapper[5052]: I1203 12:49:00.256729 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="a69bfb18-1267-4383-b51a-5e0eba70d1bc" containerName="ceilometer-central-agent" containerID="cri-o://4f74304b5716b547aed360588b0a9be626bd9def7e57532f38549b982366ac6f" gracePeriod=30 Dec 03 12:49:00 crc kubenswrapper[5052]: I1203 12:49:00.286680 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/kube-state-metrics-0"] Dec 03 12:49:00 crc kubenswrapper[5052]: I1203 12:49:00.286943 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/kube-state-metrics-0" podUID="876d06e6-0187-4e3d-9183-ed2083fe5189" containerName="kube-state-metrics" containerID="cri-o://3d1c3119a6baffe670b9369900b68bd9fa52d242fa306af736795fb6724df163" gracePeriod=30 Dec 03 12:49:00 crc kubenswrapper[5052]: I1203 12:49:00.341902 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/memcached-0"] Dec 03 12:49:00 crc kubenswrapper[5052]: I1203 12:49:00.342104 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/memcached-0" podUID="eef9c304-b598-4a31-9c47-09be0087c25e" containerName="memcached" containerID="cri-o://3cec53cfcf32bebda98bd1f984b0bb1ae1c9eaa0f2a096f12724a271e1103ca8" gracePeriod=30 Dec 03 12:49:00 crc kubenswrapper[5052]: I1203 12:49:00.387834 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/openstack-cell1-galera-0" podUID="3479f9ad-871f-463e-96ce-50f4069088b3" containerName="galera" containerID="cri-o://bf37583e842ac2986eebf906db0e42704630e142c5ffc66b3b257042bb9c2bf5" gracePeriod=30 Dec 03 12:49:00 crc kubenswrapper[5052]: I1203 12:49:00.414648 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/keystone-bootstrap-nnnzp"] Dec 03 12:49:00 crc kubenswrapper[5052]: I1203 12:49:00.433577 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/ceilometer-0" podUID="a69bfb18-1267-4383-b51a-5e0eba70d1bc" containerName="proxy-httpd" probeResult="failure" output="Get \"https://10.217.0.124:3000/\": dial tcp 10.217.0.124:3000: connect: connection refused" Dec 03 12:49:00 crc kubenswrapper[5052]: I1203 12:49:00.448664 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/keystone-bootstrap-nnnzp"] Dec 03 12:49:00 crc kubenswrapper[5052]: I1203 12:49:00.483518 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/keystone-db-sync-rgdrt"] Dec 03 12:49:00 crc kubenswrapper[5052]: I1203 12:49:00.544640 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/keystone-db-sync-rgdrt"] Dec 03 12:49:00 crc kubenswrapper[5052]: I1203 12:49:00.579781 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/keystoneb51b-account-delete-ds4mz"] Dec 03 12:49:00 crc kubenswrapper[5052]: I1203 12:49:00.584275 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystoneb51b-account-delete-ds4mz" Dec 03 12:49:00 crc kubenswrapper[5052]: I1203 12:49:00.618665 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/keystone-5f884bdbfd-2xzz4"] Dec 03 12:49:00 crc kubenswrapper[5052]: I1203 12:49:00.618932 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/keystone-5f884bdbfd-2xzz4" podUID="bff27172-5e30-474d-a49c-389a35dd3052" containerName="keystone-api" containerID="cri-o://363ded64823f176012eadd8f4efa965cf7bb21ad177ce1c3b4cc513c83f5ebab" gracePeriod=30 Dec 03 12:49:00 crc kubenswrapper[5052]: I1203 12:49:00.643116 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystoneb51b-account-delete-ds4mz"] Dec 03 12:49:00 crc kubenswrapper[5052]: I1203 12:49:00.677613 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/openstack-galera-0"] Dec 03 12:49:00 crc kubenswrapper[5052]: I1203 12:49:00.687873 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g7ffc\" (UniqueName: \"kubernetes.io/projected/6854341b-41e8-48b0-87f7-aca589364aaf-kube-api-access-g7ffc\") pod \"keystoneb51b-account-delete-ds4mz\" (UID: \"6854341b-41e8-48b0-87f7-aca589364aaf\") " pod="openstack-kuttl-tests/keystoneb51b-account-delete-ds4mz" Dec 03 12:49:00 crc kubenswrapper[5052]: I1203 12:49:00.688095 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/6854341b-41e8-48b0-87f7-aca589364aaf-operator-scripts\") pod \"keystoneb51b-account-delete-ds4mz\" (UID: \"6854341b-41e8-48b0-87f7-aca589364aaf\") " pod="openstack-kuttl-tests/keystoneb51b-account-delete-ds4mz" Dec 03 12:49:00 crc kubenswrapper[5052]: E1203 12:49:00.687980 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/rabbitmq-cell1-config-data: configmap "rabbitmq-cell1-config-data" not found Dec 03 12:49:00 crc kubenswrapper[5052]: E1203 12:49:00.688455 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/488c3529-4c2b-47c0-9dcf-05a064c34d0d-config-data podName:488c3529-4c2b-47c0-9dcf-05a064c34d0d nodeName:}" failed. No retries permitted until 2025-12-03 12:49:04.688437258 +0000 UTC m=+5335.276679022 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "config-data" (UniqueName: "kubernetes.io/configmap/488c3529-4c2b-47c0-9dcf-05a064c34d0d-config-data") pod "rabbitmq-cell1-server-0" (UID: "488c3529-4c2b-47c0-9dcf-05a064c34d0d") : configmap "rabbitmq-cell1-config-data" not found Dec 03 12:49:00 crc kubenswrapper[5052]: I1203 12:49:00.689058 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/placement361c-account-delete-67ttd"] Dec 03 12:49:00 crc kubenswrapper[5052]: I1203 12:49:00.697756 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/keystone-db-create-zjnt8"] Dec 03 12:49:00 crc kubenswrapper[5052]: I1203 12:49:00.717278 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/keystone-db-create-zjnt8"] Dec 03 12:49:00 crc kubenswrapper[5052]: I1203 12:49:00.723008 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/keystone-b51b-account-create-update-h6wlv"] Dec 03 12:49:00 crc kubenswrapper[5052]: I1203 12:49:00.728599 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/keystoneb51b-account-delete-ds4mz"] Dec 03 12:49:00 crc kubenswrapper[5052]: I1203 12:49:00.736656 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/keystone-b51b-account-create-update-h6wlv"] Dec 03 12:49:00 crc kubenswrapper[5052]: I1203 12:49:00.744238 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glanced4a6-account-delete-69llz"] Dec 03 12:49:00 crc kubenswrapper[5052]: I1203 12:49:00.751844 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder4345-account-delete-vmdzh"] Dec 03 12:49:00 crc kubenswrapper[5052]: I1203 12:49:00.759657 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutrona25a-account-delete-vbr24"] Dec 03 12:49:00 crc kubenswrapper[5052]: I1203 12:49:00.767378 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican02cc-account-delete-gf94b"] Dec 03 12:49:00 crc kubenswrapper[5052]: I1203 12:49:00.794395 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-g7ffc\" (UniqueName: \"kubernetes.io/projected/6854341b-41e8-48b0-87f7-aca589364aaf-kube-api-access-g7ffc\") pod \"keystoneb51b-account-delete-ds4mz\" (UID: \"6854341b-41e8-48b0-87f7-aca589364aaf\") " pod="openstack-kuttl-tests/keystoneb51b-account-delete-ds4mz" Dec 03 12:49:00 crc kubenswrapper[5052]: I1203 12:49:00.794447 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/6854341b-41e8-48b0-87f7-aca589364aaf-operator-scripts\") pod \"keystoneb51b-account-delete-ds4mz\" (UID: \"6854341b-41e8-48b0-87f7-aca589364aaf\") " pod="openstack-kuttl-tests/keystoneb51b-account-delete-ds4mz" Dec 03 12:49:00 crc kubenswrapper[5052]: E1203 12:49:00.794577 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:49:00 crc kubenswrapper[5052]: E1203 12:49:00.794640 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6854341b-41e8-48b0-87f7-aca589364aaf-operator-scripts podName:6854341b-41e8-48b0-87f7-aca589364aaf nodeName:}" failed. No retries permitted until 2025-12-03 12:49:01.294620414 +0000 UTC m=+5331.882862188 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/6854341b-41e8-48b0-87f7-aca589364aaf-operator-scripts") pod "keystoneb51b-account-delete-ds4mz" (UID: "6854341b-41e8-48b0-87f7-aca589364aaf") : configmap "openstack-scripts" not found Dec 03 12:49:00 crc kubenswrapper[5052]: E1203 12:49:00.799052 5052 projected.go:194] Error preparing data for projected volume kube-api-access-g7ffc for pod openstack-kuttl-tests/keystoneb51b-account-delete-ds4mz: failed to fetch token: serviceaccounts "galera-openstack" not found Dec 03 12:49:00 crc kubenswrapper[5052]: E1203 12:49:00.799114 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6854341b-41e8-48b0-87f7-aca589364aaf-kube-api-access-g7ffc podName:6854341b-41e8-48b0-87f7-aca589364aaf nodeName:}" failed. No retries permitted until 2025-12-03 12:49:01.299094842 +0000 UTC m=+5331.887336606 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-g7ffc" (UniqueName: "kubernetes.io/projected/6854341b-41e8-48b0-87f7-aca589364aaf-kube-api-access-g7ffc") pod "keystoneb51b-account-delete-ds4mz" (UID: "6854341b-41e8-48b0-87f7-aca589364aaf") : failed to fetch token: serviceaccounts "galera-openstack" not found Dec 03 12:49:00 crc kubenswrapper[5052]: I1203 12:49:00.805637 5052 kubelet_pods.go:1007] "Unable to retrieve pull secret, the image pull may not succeed." pod="openstack-kuttl-tests/novaapifd1c-account-delete-vhjzv" secret="" err="secret \"galera-openstack-dockercfg-tsmcq\" not found" Dec 03 12:49:00 crc kubenswrapper[5052]: I1203 12:49:00.805688 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/novaapifd1c-account-delete-vhjzv" Dec 03 12:49:00 crc kubenswrapper[5052]: I1203 12:49:00.830866 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-9t4zp" Dec 03 12:49:00 crc kubenswrapper[5052]: I1203 12:49:00.869093 5052 kubelet_pods.go:1007] "Unable to retrieve pull secret, the image pull may not succeed." pod="openstack-kuttl-tests/novacell05566-account-delete-2f4dm" secret="" err="secret \"galera-openstack-dockercfg-tsmcq\" not found" Dec 03 12:49:00 crc kubenswrapper[5052]: I1203 12:49:00.869141 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/novacell05566-account-delete-2f4dm" Dec 03 12:49:00 crc kubenswrapper[5052]: I1203 12:49:00.879172 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_ovsdbserver-nb-0_2005f89c-8a19-403c-acc9-c13ede717372/ovsdbserver-nb/0.log" Dec 03 12:49:00 crc kubenswrapper[5052]: I1203 12:49:00.879254 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:49:00 crc kubenswrapper[5052]: E1203 12:49:00.897280 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:49:00 crc kubenswrapper[5052]: E1203 12:49:00.897349 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2846c397-2869-49f3-aeca-cf4b56bc8834-operator-scripts podName:2846c397-2869-49f3-aeca-cf4b56bc8834 nodeName:}" failed. No retries permitted until 2025-12-03 12:49:01.397329882 +0000 UTC m=+5331.985571646 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/2846c397-2869-49f3-aeca-cf4b56bc8834-operator-scripts") pod "novaapifd1c-account-delete-vhjzv" (UID: "2846c397-2869-49f3-aeca-cf4b56bc8834") : configmap "openstack-scripts" not found Dec 03 12:49:00 crc kubenswrapper[5052]: I1203 12:49:00.920683 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" podUID="edae7848-51ff-40d6-83f2-ffff085f3f3e" containerName="nova-cell1-novncproxy-novncproxy" probeResult="failure" output="Get \"https://10.217.0.117:6080/vnc_lite.html\": dial tcp 10.217.0.117:6080: connect: connection refused" Dec 03 12:49:00 crc kubenswrapper[5052]: I1203 12:49:00.924173 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_ovsdbserver-sb-0_84f26afe-5b0c-43bb-a4c8-a9ac21d24330/ovsdbserver-sb/0.log" Dec 03 12:49:00 crc kubenswrapper[5052]: I1203 12:49:00.924263 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:49:00 crc kubenswrapper[5052]: I1203 12:49:00.946013 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstackclient" Dec 03 12:49:00 crc kubenswrapper[5052]: I1203 12:49:00.972361 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/openstack-galera-0" podUID="3c8e4e3f-f6c1-4362-b64c-b10a973723dd" containerName="galera" containerID="cri-o://86f66b1381c35478a52f56b7c8ce2c98cdc37bae9cd163bbea7b6f45c6c705b9" gracePeriod=30 Dec 03 12:49:00 crc kubenswrapper[5052]: I1203 12:49:00.998070 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/84f26afe-5b0c-43bb-a4c8-a9ac21d24330-ovsdb-rundir\") pod \"84f26afe-5b0c-43bb-a4c8-a9ac21d24330\" (UID: \"84f26afe-5b0c-43bb-a4c8-a9ac21d24330\") " Dec 03 12:49:00 crc kubenswrapper[5052]: I1203 12:49:00.998441 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hhpzk\" (UniqueName: \"kubernetes.io/projected/8a94fda3-63d8-4517-8063-f6c06688b430-kube-api-access-hhpzk\") pod \"8a94fda3-63d8-4517-8063-f6c06688b430\" (UID: \"8a94fda3-63d8-4517-8063-f6c06688b430\") " Dec 03 12:49:00 crc kubenswrapper[5052]: I1203 12:49:00.998471 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/2005f89c-8a19-403c-acc9-c13ede717372-ovsdb-rundir\") pod \"2005f89c-8a19-403c-acc9-c13ede717372\" (UID: \"2005f89c-8a19-403c-acc9-c13ede717372\") " Dec 03 12:49:00 crc kubenswrapper[5052]: I1203 12:49:00.998512 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/2005f89c-8a19-403c-acc9-c13ede717372-config\") pod \"2005f89c-8a19-403c-acc9-c13ede717372\" (UID: \"2005f89c-8a19-403c-acc9-c13ede717372\") " Dec 03 12:49:00 crc kubenswrapper[5052]: I1203 12:49:00.998546 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovsdbserver-nb-tls-certs\" (UniqueName: \"kubernetes.io/secret/2005f89c-8a19-403c-acc9-c13ede717372-ovsdbserver-nb-tls-certs\") pod \"2005f89c-8a19-403c-acc9-c13ede717372\" (UID: \"2005f89c-8a19-403c-acc9-c13ede717372\") " Dec 03 12:49:00 crc kubenswrapper[5052]: I1203 12:49:00.998564 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/84f26afe-5b0c-43bb-a4c8-a9ac21d24330-scripts\") pod \"84f26afe-5b0c-43bb-a4c8-a9ac21d24330\" (UID: \"84f26afe-5b0c-43bb-a4c8-a9ac21d24330\") " Dec 03 12:49:00 crc kubenswrapper[5052]: I1203 12:49:00.998595 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/2005f89c-8a19-403c-acc9-c13ede717372-metrics-certs-tls-certs\") pod \"2005f89c-8a19-403c-acc9-c13ede717372\" (UID: \"2005f89c-8a19-403c-acc9-c13ede717372\") " Dec 03 12:49:00 crc kubenswrapper[5052]: I1203 12:49:00.998612 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8a94fda3-63d8-4517-8063-f6c06688b430-combined-ca-bundle\") pod \"8a94fda3-63d8-4517-8063-f6c06688b430\" (UID: \"8a94fda3-63d8-4517-8063-f6c06688b430\") " Dec 03 12:49:00 crc kubenswrapper[5052]: I1203 12:49:00.998661 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovndbcluster-sb-etc-ovn\" (UniqueName: \"kubernetes.io/local-volume/local-storage09-crc\") pod \"84f26afe-5b0c-43bb-a4c8-a9ac21d24330\" (UID: \"84f26afe-5b0c-43bb-a4c8-a9ac21d24330\") " Dec 03 12:49:00 crc kubenswrapper[5052]: I1203 12:49:00.998684 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/84f26afe-5b0c-43bb-a4c8-a9ac21d24330-metrics-certs-tls-certs\") pod \"84f26afe-5b0c-43bb-a4c8-a9ac21d24330\" (UID: \"84f26afe-5b0c-43bb-a4c8-a9ac21d24330\") " Dec 03 12:49:00 crc kubenswrapper[5052]: I1203 12:49:00.998708 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"openstack-config-secret\" (UniqueName: \"kubernetes.io/secret/8a94fda3-63d8-4517-8063-f6c06688b430-openstack-config-secret\") pod \"8a94fda3-63d8-4517-8063-f6c06688b430\" (UID: \"8a94fda3-63d8-4517-8063-f6c06688b430\") " Dec 03 12:49:00 crc kubenswrapper[5052]: I1203 12:49:00.998745 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wrxdr\" (UniqueName: \"kubernetes.io/projected/2005f89c-8a19-403c-acc9-c13ede717372-kube-api-access-wrxdr\") pod \"2005f89c-8a19-403c-acc9-c13ede717372\" (UID: \"2005f89c-8a19-403c-acc9-c13ede717372\") " Dec 03 12:49:00 crc kubenswrapper[5052]: I1203 12:49:00.998741 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/84f26afe-5b0c-43bb-a4c8-a9ac21d24330-ovsdb-rundir" (OuterVolumeSpecName: "ovsdb-rundir") pod "84f26afe-5b0c-43bb-a4c8-a9ac21d24330" (UID: "84f26afe-5b0c-43bb-a4c8-a9ac21d24330"). InnerVolumeSpecName "ovsdb-rundir". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:49:00 crc kubenswrapper[5052]: I1203 12:49:00.998760 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/2005f89c-8a19-403c-acc9-c13ede717372-scripts\") pod \"2005f89c-8a19-403c-acc9-c13ede717372\" (UID: \"2005f89c-8a19-403c-acc9-c13ede717372\") " Dec 03 12:49:00 crc kubenswrapper[5052]: I1203 12:49:00.998885 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/84f26afe-5b0c-43bb-a4c8-a9ac21d24330-config\") pod \"84f26afe-5b0c-43bb-a4c8-a9ac21d24330\" (UID: \"84f26afe-5b0c-43bb-a4c8-a9ac21d24330\") " Dec 03 12:49:00 crc kubenswrapper[5052]: I1203 12:49:00.998923 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8xfbb\" (UniqueName: \"kubernetes.io/projected/84f26afe-5b0c-43bb-a4c8-a9ac21d24330-kube-api-access-8xfbb\") pod \"84f26afe-5b0c-43bb-a4c8-a9ac21d24330\" (UID: \"84f26afe-5b0c-43bb-a4c8-a9ac21d24330\") " Dec 03 12:49:00 crc kubenswrapper[5052]: I1203 12:49:00.998953 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/84f26afe-5b0c-43bb-a4c8-a9ac21d24330-combined-ca-bundle\") pod \"84f26afe-5b0c-43bb-a4c8-a9ac21d24330\" (UID: \"84f26afe-5b0c-43bb-a4c8-a9ac21d24330\") " Dec 03 12:49:00 crc kubenswrapper[5052]: I1203 12:49:00.999047 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovsdbserver-sb-tls-certs\" (UniqueName: \"kubernetes.io/secret/84f26afe-5b0c-43bb-a4c8-a9ac21d24330-ovsdbserver-sb-tls-certs\") pod \"84f26afe-5b0c-43bb-a4c8-a9ac21d24330\" (UID: \"84f26afe-5b0c-43bb-a4c8-a9ac21d24330\") " Dec 03 12:49:00 crc kubenswrapper[5052]: I1203 12:49:00.999141 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovndbcluster-nb-etc-ovn\" (UniqueName: \"kubernetes.io/local-volume/local-storage01-crc\") pod \"2005f89c-8a19-403c-acc9-c13ede717372\" (UID: \"2005f89c-8a19-403c-acc9-c13ede717372\") " Dec 03 12:49:00 crc kubenswrapper[5052]: I1203 12:49:00.999169 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2005f89c-8a19-403c-acc9-c13ede717372-combined-ca-bundle\") pod \"2005f89c-8a19-403c-acc9-c13ede717372\" (UID: \"2005f89c-8a19-403c-acc9-c13ede717372\") " Dec 03 12:49:00 crc kubenswrapper[5052]: I1203 12:49:00.999262 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"openstack-config\" (UniqueName: \"kubernetes.io/configmap/8a94fda3-63d8-4517-8063-f6c06688b430-openstack-config\") pod \"8a94fda3-63d8-4517-8063-f6c06688b430\" (UID: \"8a94fda3-63d8-4517-8063-f6c06688b430\") " Dec 03 12:49:00 crc kubenswrapper[5052]: I1203 12:49:00.999310 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2005f89c-8a19-403c-acc9-c13ede717372-scripts" (OuterVolumeSpecName: "scripts") pod "2005f89c-8a19-403c-acc9-c13ede717372" (UID: "2005f89c-8a19-403c-acc9-c13ede717372"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:49:01 crc kubenswrapper[5052]: I1203 12:49:00.999930 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2005f89c-8a19-403c-acc9-c13ede717372-ovsdb-rundir" (OuterVolumeSpecName: "ovsdb-rundir") pod "2005f89c-8a19-403c-acc9-c13ede717372" (UID: "2005f89c-8a19-403c-acc9-c13ede717372"). InnerVolumeSpecName "ovsdb-rundir". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:49:01 crc kubenswrapper[5052]: I1203 12:49:01.000174 5052 reconciler_common.go:293] "Volume detached for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/84f26afe-5b0c-43bb-a4c8-a9ac21d24330-ovsdb-rundir\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:01 crc kubenswrapper[5052]: I1203 12:49:01.000474 5052 reconciler_common.go:293] "Volume detached for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/2005f89c-8a19-403c-acc9-c13ede717372-ovsdb-rundir\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:01 crc kubenswrapper[5052]: I1203 12:49:01.000492 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/2005f89c-8a19-403c-acc9-c13ede717372-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:01 crc kubenswrapper[5052]: E1203 12:49:01.000503 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/rabbitmq-config-data: configmap "rabbitmq-config-data" not found Dec 03 12:49:01 crc kubenswrapper[5052]: E1203 12:49:01.000572 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:49:01 crc kubenswrapper[5052]: E1203 12:49:01.000584 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0cb33a40-6f0a-4e89-a159-4edd3af3a2fa-config-data podName:0cb33a40-6f0a-4e89-a159-4edd3af3a2fa nodeName:}" failed. No retries permitted until 2025-12-03 12:49:05.000563904 +0000 UTC m=+5335.588805668 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "config-data" (UniqueName: "kubernetes.io/configmap/0cb33a40-6f0a-4e89-a159-4edd3af3a2fa-config-data") pod "rabbitmq-server-0" (UID: "0cb33a40-6f0a-4e89-a159-4edd3af3a2fa") : configmap "rabbitmq-config-data" not found Dec 03 12:49:01 crc kubenswrapper[5052]: E1203 12:49:01.000628 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a2bc501c-ebef-4e68-8ae2-e8c4217cadff-operator-scripts podName:a2bc501c-ebef-4e68-8ae2-e8c4217cadff nodeName:}" failed. No retries permitted until 2025-12-03 12:49:01.500608635 +0000 UTC m=+5332.088850409 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/a2bc501c-ebef-4e68-8ae2-e8c4217cadff-operator-scripts") pod "novacell05566-account-delete-2f4dm" (UID: "a2bc501c-ebef-4e68-8ae2-e8c4217cadff") : configmap "openstack-scripts" not found Dec 03 12:49:01 crc kubenswrapper[5052]: I1203 12:49:01.005237 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/84f26afe-5b0c-43bb-a4c8-a9ac21d24330-config" (OuterVolumeSpecName: "config") pod "84f26afe-5b0c-43bb-a4c8-a9ac21d24330" (UID: "84f26afe-5b0c-43bb-a4c8-a9ac21d24330"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:49:01 crc kubenswrapper[5052]: I1203 12:49:01.006152 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2005f89c-8a19-403c-acc9-c13ede717372-config" (OuterVolumeSpecName: "config") pod "2005f89c-8a19-403c-acc9-c13ede717372" (UID: "2005f89c-8a19-403c-acc9-c13ede717372"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:49:01 crc kubenswrapper[5052]: I1203 12:49:01.006190 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage09-crc" (OuterVolumeSpecName: "ovndbcluster-sb-etc-ovn") pod "84f26afe-5b0c-43bb-a4c8-a9ac21d24330" (UID: "84f26afe-5b0c-43bb-a4c8-a9ac21d24330"). InnerVolumeSpecName "local-storage09-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 12:49:01 crc kubenswrapper[5052]: I1203 12:49:01.006807 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/84f26afe-5b0c-43bb-a4c8-a9ac21d24330-scripts" (OuterVolumeSpecName: "scripts") pod "84f26afe-5b0c-43bb-a4c8-a9ac21d24330" (UID: "84f26afe-5b0c-43bb-a4c8-a9ac21d24330"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:49:01 crc kubenswrapper[5052]: I1203 12:49:01.010205 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8a94fda3-63d8-4517-8063-f6c06688b430-kube-api-access-hhpzk" (OuterVolumeSpecName: "kube-api-access-hhpzk") pod "8a94fda3-63d8-4517-8063-f6c06688b430" (UID: "8a94fda3-63d8-4517-8063-f6c06688b430"). InnerVolumeSpecName "kube-api-access-hhpzk". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:49:01 crc kubenswrapper[5052]: I1203 12:49:01.013654 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage01-crc" (OuterVolumeSpecName: "ovndbcluster-nb-etc-ovn") pod "2005f89c-8a19-403c-acc9-c13ede717372" (UID: "2005f89c-8a19-403c-acc9-c13ede717372"). InnerVolumeSpecName "local-storage01-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 12:49:01 crc kubenswrapper[5052]: I1203 12:49:01.013799 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/84f26afe-5b0c-43bb-a4c8-a9ac21d24330-kube-api-access-8xfbb" (OuterVolumeSpecName: "kube-api-access-8xfbb") pod "84f26afe-5b0c-43bb-a4c8-a9ac21d24330" (UID: "84f26afe-5b0c-43bb-a4c8-a9ac21d24330"). InnerVolumeSpecName "kube-api-access-8xfbb". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:49:01 crc kubenswrapper[5052]: I1203 12:49:01.032871 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2005f89c-8a19-403c-acc9-c13ede717372-kube-api-access-wrxdr" (OuterVolumeSpecName: "kube-api-access-wrxdr") pod "2005f89c-8a19-403c-acc9-c13ede717372" (UID: "2005f89c-8a19-403c-acc9-c13ede717372"). InnerVolumeSpecName "kube-api-access-wrxdr". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:49:01 crc kubenswrapper[5052]: I1203 12:49:01.038195 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8a94fda3-63d8-4517-8063-f6c06688b430-openstack-config" (OuterVolumeSpecName: "openstack-config") pod "8a94fda3-63d8-4517-8063-f6c06688b430" (UID: "8a94fda3-63d8-4517-8063-f6c06688b430"). InnerVolumeSpecName "openstack-config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:49:01 crc kubenswrapper[5052]: I1203 12:49:01.054424 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8a94fda3-63d8-4517-8063-f6c06688b430-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "8a94fda3-63d8-4517-8063-f6c06688b430" (UID: "8a94fda3-63d8-4517-8063-f6c06688b430"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:49:01 crc kubenswrapper[5052]: I1203 12:49:01.055288 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glanced4a6-account-delete-69llz" event={"ID":"6a5de7fb-b482-4849-a957-e679616aabc0","Type":"ContainerStarted","Data":"b4607ef57b739c3dcf2c9020dc61004b4be30f5f17eab10a2de4e5156869e7dd"} Dec 03 12:49:01 crc kubenswrapper[5052]: I1203 12:49:01.062956 5052 generic.go:334] "Generic (PLEG): container finished" podID="2065f5bb-88eb-4e84-88b6-9982833d413d" containerID="2e1db89e52b83cc4d03ffdc49b1a121ba636bb18c0424300ea015664b45cff9a" exitCode=143 Dec 03 12:49:01 crc kubenswrapper[5052]: I1203 12:49:01.063024 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-keystone-listener-68fd6d997d-7tjjw" event={"ID":"2065f5bb-88eb-4e84-88b6-9982833d413d","Type":"ContainerDied","Data":"2e1db89e52b83cc4d03ffdc49b1a121ba636bb18c0424300ea015664b45cff9a"} Dec 03 12:49:01 crc kubenswrapper[5052]: I1203 12:49:01.074932 5052 generic.go:334] "Generic (PLEG): container finished" podID="a69bfb18-1267-4383-b51a-5e0eba70d1bc" containerID="74e52eb4f69076f917a1b3ab5d18438ec4c2d708d4bc29581185ab517d52919e" exitCode=0 Dec 03 12:49:01 crc kubenswrapper[5052]: I1203 12:49:01.074962 5052 generic.go:334] "Generic (PLEG): container finished" podID="a69bfb18-1267-4383-b51a-5e0eba70d1bc" containerID="18022901c9e16d4c4eb5c5f284061eb245203a106525b3fb6b0192a33f4b392e" exitCode=2 Dec 03 12:49:01 crc kubenswrapper[5052]: I1203 12:49:01.074972 5052 generic.go:334] "Generic (PLEG): container finished" podID="a69bfb18-1267-4383-b51a-5e0eba70d1bc" containerID="4f74304b5716b547aed360588b0a9be626bd9def7e57532f38549b982366ac6f" exitCode=0 Dec 03 12:49:01 crc kubenswrapper[5052]: I1203 12:49:01.075072 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"a69bfb18-1267-4383-b51a-5e0eba70d1bc","Type":"ContainerDied","Data":"74e52eb4f69076f917a1b3ab5d18438ec4c2d708d4bc29581185ab517d52919e"} Dec 03 12:49:01 crc kubenswrapper[5052]: I1203 12:49:01.075101 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"a69bfb18-1267-4383-b51a-5e0eba70d1bc","Type":"ContainerDied","Data":"18022901c9e16d4c4eb5c5f284061eb245203a106525b3fb6b0192a33f4b392e"} Dec 03 12:49:01 crc kubenswrapper[5052]: I1203 12:49:01.075111 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"a69bfb18-1267-4383-b51a-5e0eba70d1bc","Type":"ContainerDied","Data":"4f74304b5716b547aed360588b0a9be626bd9def7e57532f38549b982366ac6f"} Dec 03 12:49:01 crc kubenswrapper[5052]: I1203 12:49:01.082650 5052 generic.go:334] "Generic (PLEG): container finished" podID="dd8df313-3680-4dd1-a57d-ff85b0b2c6f1" containerID="45b85b0267828ad53c05c095096047353cbe686d4edc60d6dabed09ff937a326" exitCode=0 Dec 03 12:49:01 crc kubenswrapper[5052]: I1203 12:49:01.082685 5052 generic.go:334] "Generic (PLEG): container finished" podID="dd8df313-3680-4dd1-a57d-ff85b0b2c6f1" containerID="acb0be96c14d669ff7b80a9476845494757b206ec5aaddb2a34ac9ff0fe87bd7" exitCode=0 Dec 03 12:49:01 crc kubenswrapper[5052]: I1203 12:49:01.082755 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-proxy-78f78b88b6-fmjvp" event={"ID":"dd8df313-3680-4dd1-a57d-ff85b0b2c6f1","Type":"ContainerDied","Data":"45b85b0267828ad53c05c095096047353cbe686d4edc60d6dabed09ff937a326"} Dec 03 12:49:01 crc kubenswrapper[5052]: I1203 12:49:01.082786 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-proxy-78f78b88b6-fmjvp" event={"ID":"dd8df313-3680-4dd1-a57d-ff85b0b2c6f1","Type":"ContainerDied","Data":"acb0be96c14d669ff7b80a9476845494757b206ec5aaddb2a34ac9ff0fe87bd7"} Dec 03 12:49:01 crc kubenswrapper[5052]: I1203 12:49:01.104127 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-hhpzk\" (UniqueName: \"kubernetes.io/projected/8a94fda3-63d8-4517-8063-f6c06688b430-kube-api-access-hhpzk\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:01 crc kubenswrapper[5052]: I1203 12:49:01.104157 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/2005f89c-8a19-403c-acc9-c13ede717372-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:01 crc kubenswrapper[5052]: I1203 12:49:01.104170 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/84f26afe-5b0c-43bb-a4c8-a9ac21d24330-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:01 crc kubenswrapper[5052]: I1203 12:49:01.104181 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8a94fda3-63d8-4517-8063-f6c06688b430-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:01 crc kubenswrapper[5052]: I1203 12:49:01.104209 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage09-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage09-crc\") on node \"crc\" " Dec 03 12:49:01 crc kubenswrapper[5052]: I1203 12:49:01.104224 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-wrxdr\" (UniqueName: \"kubernetes.io/projected/2005f89c-8a19-403c-acc9-c13ede717372-kube-api-access-wrxdr\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:01 crc kubenswrapper[5052]: I1203 12:49:01.104237 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/84f26afe-5b0c-43bb-a4c8-a9ac21d24330-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:01 crc kubenswrapper[5052]: I1203 12:49:01.104249 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-8xfbb\" (UniqueName: \"kubernetes.io/projected/84f26afe-5b0c-43bb-a4c8-a9ac21d24330-kube-api-access-8xfbb\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:01 crc kubenswrapper[5052]: I1203 12:49:01.104266 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage01-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage01-crc\") on node \"crc\" " Dec 03 12:49:01 crc kubenswrapper[5052]: I1203 12:49:01.104277 5052 reconciler_common.go:293] "Volume detached for volume \"openstack-config\" (UniqueName: \"kubernetes.io/configmap/8a94fda3-63d8-4517-8063-f6c06688b430-openstack-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:01 crc kubenswrapper[5052]: I1203 12:49:01.110804 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder4345-account-delete-vmdzh" event={"ID":"5d11b02e-c14a-4d1a-819a-4c3b66aedd33","Type":"ContainerStarted","Data":"edd916e2f697136555c97bf7187eabf8e7e3c3a04c9c08fec4f86f0bc3ca9a5c"} Dec 03 12:49:01 crc kubenswrapper[5052]: I1203 12:49:01.129115 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8a94fda3-63d8-4517-8063-f6c06688b430-openstack-config-secret" (OuterVolumeSpecName: "openstack-config-secret") pod "8a94fda3-63d8-4517-8063-f6c06688b430" (UID: "8a94fda3-63d8-4517-8063-f6c06688b430"). InnerVolumeSpecName "openstack-config-secret". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:49:01 crc kubenswrapper[5052]: I1203 12:49:01.133310 5052 generic.go:334] "Generic (PLEG): container finished" podID="b8eb690c-165a-4906-a750-bec0178daf6c" containerID="1986f1fabd06880b71b76b4c94e01ab46c4e8d1cff3eae60d3b1cd7579b2c785" exitCode=143 Dec 03 12:49:01 crc kubenswrapper[5052]: I1203 12:49:01.133712 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-75bdf7fcbb-mlkgz" event={"ID":"b8eb690c-165a-4906-a750-bec0178daf6c","Type":"ContainerDied","Data":"1986f1fabd06880b71b76b4c94e01ab46c4e8d1cff3eae60d3b1cd7579b2c785"} Dec 03 12:49:01 crc kubenswrapper[5052]: I1203 12:49:01.150497 5052 generic.go:334] "Generic (PLEG): container finished" podID="aa557411-6b0a-4596-a5f5-0e23f8686dc4" containerID="1d9679ea8e92979b2a265175ba31597cc56e80e13b2610ba8cc4b4e015500253" exitCode=0 Dec 03 12:49:01 crc kubenswrapper[5052]: I1203 12:49:01.150564 5052 generic.go:334] "Generic (PLEG): container finished" podID="aa557411-6b0a-4596-a5f5-0e23f8686dc4" containerID="28dad3a0f48d004bf9433a47f2e9bb435f9945510d0f67da8d4246236a90195d" exitCode=0 Dec 03 12:49:01 crc kubenswrapper[5052]: I1203 12:49:01.150672 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"aa557411-6b0a-4596-a5f5-0e23f8686dc4","Type":"ContainerDied","Data":"1d9679ea8e92979b2a265175ba31597cc56e80e13b2610ba8cc4b4e015500253"} Dec 03 12:49:01 crc kubenswrapper[5052]: I1203 12:49:01.150728 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"aa557411-6b0a-4596-a5f5-0e23f8686dc4","Type":"ContainerDied","Data":"28dad3a0f48d004bf9433a47f2e9bb435f9945510d0f67da8d4246236a90195d"} Dec 03 12:49:01 crc kubenswrapper[5052]: I1203 12:49:01.159026 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_ovsdbserver-sb-0_84f26afe-5b0c-43bb-a4c8-a9ac21d24330/ovsdbserver-sb/0.log" Dec 03 12:49:01 crc kubenswrapper[5052]: I1203 12:49:01.159194 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-sb-0" event={"ID":"84f26afe-5b0c-43bb-a4c8-a9ac21d24330","Type":"ContainerDied","Data":"2bf9da39b82d602ec1c4f37d5f315087e20eb503a371d87ae4e4bada773f70c7"} Dec 03 12:49:01 crc kubenswrapper[5052]: I1203 12:49:01.159237 5052 scope.go:117] "RemoveContainer" containerID="721e21ea9214ab0b7f428a1744d9bf1b20c7493291c82e15a89dd5d11891df97" Dec 03 12:49:01 crc kubenswrapper[5052]: I1203 12:49:01.159395 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:49:01 crc kubenswrapper[5052]: I1203 12:49:01.170492 5052 generic.go:334] "Generic (PLEG): container finished" podID="e14465ed-868a-49ba-b3ab-1e67196229e8" containerID="805cd92e51a39e19c7bde5260dca999018b3ba92ebb95c79b560a3986eb6d94a" exitCode=143 Dec 03 12:49:01 crc kubenswrapper[5052]: I1203 12:49:01.170578 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-worker-58f7cb7fd9-2k5tg" event={"ID":"e14465ed-868a-49ba-b3ab-1e67196229e8","Type":"ContainerDied","Data":"805cd92e51a39e19c7bde5260dca999018b3ba92ebb95c79b560a3986eb6d94a"} Dec 03 12:49:01 crc kubenswrapper[5052]: I1203 12:49:01.174079 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican02cc-account-delete-gf94b" event={"ID":"857b8a8a-d730-4832-a71c-d45a2f3ce406","Type":"ContainerStarted","Data":"823c9ad272bf683331ba7b76dc06ffdad3a303ada7a89dddc6c6456b71e08dd0"} Dec 03 12:49:01 crc kubenswrapper[5052]: I1203 12:49:01.183405 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2005f89c-8a19-403c-acc9-c13ede717372-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "2005f89c-8a19-403c-acc9-c13ede717372" (UID: "2005f89c-8a19-403c-acc9-c13ede717372"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:49:01 crc kubenswrapper[5052]: I1203 12:49:01.189654 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/84f26afe-5b0c-43bb-a4c8-a9ac21d24330-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "84f26afe-5b0c-43bb-a4c8-a9ac21d24330" (UID: "84f26afe-5b0c-43bb-a4c8-a9ac21d24330"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:49:01 crc kubenswrapper[5052]: I1203 12:49:01.205559 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage01-crc" (UniqueName: "kubernetes.io/local-volume/local-storage01-crc") on node "crc" Dec 03 12:49:01 crc kubenswrapper[5052]: I1203 12:49:01.208771 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage01-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage01-crc\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:01 crc kubenswrapper[5052]: I1203 12:49:01.208828 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2005f89c-8a19-403c-acc9-c13ede717372-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:01 crc kubenswrapper[5052]: I1203 12:49:01.208847 5052 reconciler_common.go:293] "Volume detached for volume \"openstack-config-secret\" (UniqueName: \"kubernetes.io/secret/8a94fda3-63d8-4517-8063-f6c06688b430-openstack-config-secret\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:01 crc kubenswrapper[5052]: I1203 12:49:01.208860 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/84f26afe-5b0c-43bb-a4c8-a9ac21d24330-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:01 crc kubenswrapper[5052]: I1203 12:49:01.210670 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage09-crc" (UniqueName: "kubernetes.io/local-volume/local-storage09-crc") on node "crc" Dec 03 12:49:01 crc kubenswrapper[5052]: I1203 12:49:01.212042 5052 generic.go:334] "Generic (PLEG): container finished" podID="edae7848-51ff-40d6-83f2-ffff085f3f3e" containerID="1470a22e3c4e2dd79a87f38bb53dbab1acc3ac13debd607ce9ac04a443f55380" exitCode=0 Dec 03 12:49:01 crc kubenswrapper[5052]: I1203 12:49:01.212133 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" event={"ID":"edae7848-51ff-40d6-83f2-ffff085f3f3e","Type":"ContainerDied","Data":"1470a22e3c4e2dd79a87f38bb53dbab1acc3ac13debd607ce9ac04a443f55380"} Dec 03 12:49:01 crc kubenswrapper[5052]: I1203 12:49:01.221502 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement361c-account-delete-67ttd" event={"ID":"456c3fb1-90c0-4d17-863e-414380759f97","Type":"ContainerStarted","Data":"b9d37c789843877b9b3d38fbc02a09b3df53f9dbff1743cb3ac630230e61dd79"} Dec 03 12:49:01 crc kubenswrapper[5052]: I1203 12:49:01.223835 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutrona25a-account-delete-vbr24" event={"ID":"6ceba96c-7e86-4eb2-afa7-444f0fcc27ce","Type":"ContainerStarted","Data":"4484c628b57167ddc8097fb24cae67ddfbe52a0c2419f2ef5e9d75c47d729dc6"} Dec 03 12:49:01 crc kubenswrapper[5052]: I1203 12:49:01.229672 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstackclient" Dec 03 12:49:01 crc kubenswrapper[5052]: I1203 12:49:01.241876 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_ovsdbserver-nb-0_2005f89c-8a19-403c-acc9-c13ede717372/ovsdbserver-nb/0.log" Dec 03 12:49:01 crc kubenswrapper[5052]: I1203 12:49:01.242025 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:49:01 crc kubenswrapper[5052]: I1203 12:49:01.242813 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-nb-0" event={"ID":"2005f89c-8a19-403c-acc9-c13ede717372","Type":"ContainerDied","Data":"97e7375adc02fed58699749dcd3cac71fe870cf4ea77be0d3889c888ce45f78b"} Dec 03 12:49:01 crc kubenswrapper[5052]: I1203 12:49:01.251875 5052 generic.go:334] "Generic (PLEG): container finished" podID="efb0e60c-2a71-4641-87b2-0f5532b28678" containerID="d42c23e387ad7773d4b53a13fb7267499e89a43ce6fa0b570fa164ddcc0ed712" exitCode=143 Dec 03 12:49:01 crc kubenswrapper[5052]: I1203 12:49:01.252027 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"efb0e60c-2a71-4641-87b2-0f5532b28678","Type":"ContainerDied","Data":"d42c23e387ad7773d4b53a13fb7267499e89a43ce6fa0b570fa164ddcc0ed712"} Dec 03 12:49:01 crc kubenswrapper[5052]: I1203 12:49:01.259550 5052 generic.go:334] "Generic (PLEG): container finished" podID="876d06e6-0187-4e3d-9183-ed2083fe5189" containerID="3d1c3119a6baffe670b9369900b68bd9fa52d242fa306af736795fb6724df163" exitCode=2 Dec 03 12:49:01 crc kubenswrapper[5052]: I1203 12:49:01.259709 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/kube-state-metrics-0" event={"ID":"876d06e6-0187-4e3d-9183-ed2083fe5189","Type":"ContainerDied","Data":"3d1c3119a6baffe670b9369900b68bd9fa52d242fa306af736795fb6724df163"} Dec 03 12:49:01 crc kubenswrapper[5052]: I1203 12:49:01.259822 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/kube-state-metrics-0" event={"ID":"876d06e6-0187-4e3d-9183-ed2083fe5189","Type":"ContainerDied","Data":"be84dd2a321e8f5a869f52c256ca8452d3f9dad861bb220ba7e72a739992e6d4"} Dec 03 12:49:01 crc kubenswrapper[5052]: I1203 12:49:01.259841 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="be84dd2a321e8f5a869f52c256ca8452d3f9dad861bb220ba7e72a739992e6d4" Dec 03 12:49:01 crc kubenswrapper[5052]: I1203 12:49:01.280759 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/84f26afe-5b0c-43bb-a4c8-a9ac21d24330-metrics-certs-tls-certs" (OuterVolumeSpecName: "metrics-certs-tls-certs") pod "84f26afe-5b0c-43bb-a4c8-a9ac21d24330" (UID: "84f26afe-5b0c-43bb-a4c8-a9ac21d24330"). InnerVolumeSpecName "metrics-certs-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:49:01 crc kubenswrapper[5052]: I1203 12:49:01.281553 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2005f89c-8a19-403c-acc9-c13ede717372-ovsdbserver-nb-tls-certs" (OuterVolumeSpecName: "ovsdbserver-nb-tls-certs") pod "2005f89c-8a19-403c-acc9-c13ede717372" (UID: "2005f89c-8a19-403c-acc9-c13ede717372"). InnerVolumeSpecName "ovsdbserver-nb-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:49:01 crc kubenswrapper[5052]: I1203 12:49:01.311480 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-g7ffc\" (UniqueName: \"kubernetes.io/projected/6854341b-41e8-48b0-87f7-aca589364aaf-kube-api-access-g7ffc\") pod \"keystoneb51b-account-delete-ds4mz\" (UID: \"6854341b-41e8-48b0-87f7-aca589364aaf\") " pod="openstack-kuttl-tests/keystoneb51b-account-delete-ds4mz" Dec 03 12:49:01 crc kubenswrapper[5052]: I1203 12:49:01.311542 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/6854341b-41e8-48b0-87f7-aca589364aaf-operator-scripts\") pod \"keystoneb51b-account-delete-ds4mz\" (UID: \"6854341b-41e8-48b0-87f7-aca589364aaf\") " pod="openstack-kuttl-tests/keystoneb51b-account-delete-ds4mz" Dec 03 12:49:01 crc kubenswrapper[5052]: I1203 12:49:01.311753 5052 reconciler_common.go:293] "Volume detached for volume \"ovsdbserver-nb-tls-certs\" (UniqueName: \"kubernetes.io/secret/2005f89c-8a19-403c-acc9-c13ede717372-ovsdbserver-nb-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:01 crc kubenswrapper[5052]: I1203 12:49:01.311771 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage09-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage09-crc\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:01 crc kubenswrapper[5052]: I1203 12:49:01.311781 5052 reconciler_common.go:293] "Volume detached for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/84f26afe-5b0c-43bb-a4c8-a9ac21d24330-metrics-certs-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:01 crc kubenswrapper[5052]: E1203 12:49:01.311843 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:49:01 crc kubenswrapper[5052]: E1203 12:49:01.311886 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6854341b-41e8-48b0-87f7-aca589364aaf-operator-scripts podName:6854341b-41e8-48b0-87f7-aca589364aaf nodeName:}" failed. No retries permitted until 2025-12-03 12:49:02.311872957 +0000 UTC m=+5332.900114721 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/6854341b-41e8-48b0-87f7-aca589364aaf-operator-scripts") pod "keystoneb51b-account-delete-ds4mz" (UID: "6854341b-41e8-48b0-87f7-aca589364aaf") : configmap "openstack-scripts" not found Dec 03 12:49:01 crc kubenswrapper[5052]: I1203 12:49:01.314674 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/84f26afe-5b0c-43bb-a4c8-a9ac21d24330-ovsdbserver-sb-tls-certs" (OuterVolumeSpecName: "ovsdbserver-sb-tls-certs") pod "84f26afe-5b0c-43bb-a4c8-a9ac21d24330" (UID: "84f26afe-5b0c-43bb-a4c8-a9ac21d24330"). InnerVolumeSpecName "ovsdbserver-sb-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:49:01 crc kubenswrapper[5052]: E1203 12:49:01.314687 5052 projected.go:194] Error preparing data for projected volume kube-api-access-g7ffc for pod openstack-kuttl-tests/keystoneb51b-account-delete-ds4mz: failed to fetch token: serviceaccounts "galera-openstack" not found Dec 03 12:49:01 crc kubenswrapper[5052]: E1203 12:49:01.314775 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6854341b-41e8-48b0-87f7-aca589364aaf-kube-api-access-g7ffc podName:6854341b-41e8-48b0-87f7-aca589364aaf nodeName:}" failed. No retries permitted until 2025-12-03 12:49:02.314751939 +0000 UTC m=+5332.902993763 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-g7ffc" (UniqueName: "kubernetes.io/projected/6854341b-41e8-48b0-87f7-aca589364aaf-kube-api-access-g7ffc") pod "keystoneb51b-account-delete-ds4mz" (UID: "6854341b-41e8-48b0-87f7-aca589364aaf") : failed to fetch token: serviceaccounts "galera-openstack" not found Dec 03 12:49:01 crc kubenswrapper[5052]: E1203 12:49:01.349494 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-g7ffc operator-scripts], unattached volumes=[], failed to process volumes=[]: context canceled" pod="openstack-kuttl-tests/keystoneb51b-account-delete-ds4mz" podUID="6854341b-41e8-48b0-87f7-aca589364aaf" Dec 03 12:49:01 crc kubenswrapper[5052]: I1203 12:49:01.383103 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2005f89c-8a19-403c-acc9-c13ede717372-metrics-certs-tls-certs" (OuterVolumeSpecName: "metrics-certs-tls-certs") pod "2005f89c-8a19-403c-acc9-c13ede717372" (UID: "2005f89c-8a19-403c-acc9-c13ede717372"). InnerVolumeSpecName "metrics-certs-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:49:01 crc kubenswrapper[5052]: I1203 12:49:01.385916 5052 scope.go:117] "RemoveContainer" containerID="b74065369c814d7538cce05dc1a2188d258f8a0970dbde7fa52adce059389dc5" Dec 03 12:49:01 crc kubenswrapper[5052]: I1203 12:49:01.410841 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:49:01 crc kubenswrapper[5052]: I1203 12:49:01.415435 5052 reconciler_common.go:293] "Volume detached for volume \"ovsdbserver-sb-tls-certs\" (UniqueName: \"kubernetes.io/secret/84f26afe-5b0c-43bb-a4c8-a9ac21d24330-ovsdbserver-sb-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:01 crc kubenswrapper[5052]: I1203 12:49:01.415471 5052 reconciler_common.go:293] "Volume detached for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/2005f89c-8a19-403c-acc9-c13ede717372-metrics-certs-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:01 crc kubenswrapper[5052]: E1203 12:49:01.415484 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:49:01 crc kubenswrapper[5052]: E1203 12:49:01.415575 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2846c397-2869-49f3-aeca-cf4b56bc8834-operator-scripts podName:2846c397-2869-49f3-aeca-cf4b56bc8834 nodeName:}" failed. No retries permitted until 2025-12-03 12:49:02.415555162 +0000 UTC m=+5333.003796996 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/2846c397-2869-49f3-aeca-cf4b56bc8834-operator-scripts") pod "novaapifd1c-account-delete-vhjzv" (UID: "2846c397-2869-49f3-aeca-cf4b56bc8834") : configmap "openstack-scripts" not found Dec 03 12:49:01 crc kubenswrapper[5052]: I1203 12:49:01.417803 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-proxy-78f78b88b6-fmjvp" Dec 03 12:49:01 crc kubenswrapper[5052]: I1203 12:49:01.419360 5052 scope.go:117] "RemoveContainer" containerID="e3c10016f8e333ebc2b5eef7809847e5f7ea2904a0f92aa3ddc61a80ff2d9403" Dec 03 12:49:01 crc kubenswrapper[5052]: I1203 12:49:01.470345 5052 scope.go:117] "RemoveContainer" containerID="c8c8497d5896f2a84e4979c47d2430cbec8075e3536d1d41039d14854df9a795" Dec 03 12:49:01 crc kubenswrapper[5052]: I1203 12:49:01.500038 5052 scope.go:117] "RemoveContainer" containerID="14ec8a29b2f04a213b48a3edd706ccb3eed3920a11add7c7ed0658a51c533ce3" Dec 03 12:49:01 crc kubenswrapper[5052]: I1203 12:49:01.505846 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-sb-0"] Dec 03 12:49:01 crc kubenswrapper[5052]: I1203 12:49:01.514215 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-sb-0"] Dec 03 12:49:01 crc kubenswrapper[5052]: I1203 12:49:01.516093 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/dd8df313-3680-4dd1-a57d-ff85b0b2c6f1-public-tls-certs\") pod \"dd8df313-3680-4dd1-a57d-ff85b0b2c6f1\" (UID: \"dd8df313-3680-4dd1-a57d-ff85b0b2c6f1\") " Dec 03 12:49:01 crc kubenswrapper[5052]: I1203 12:49:01.516194 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/dd8df313-3680-4dd1-a57d-ff85b0b2c6f1-config-data\") pod \"dd8df313-3680-4dd1-a57d-ff85b0b2c6f1\" (UID: \"dd8df313-3680-4dd1-a57d-ff85b0b2c6f1\") " Dec 03 12:49:01 crc kubenswrapper[5052]: I1203 12:49:01.516226 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/dd8df313-3680-4dd1-a57d-ff85b0b2c6f1-internal-tls-certs\") pod \"dd8df313-3680-4dd1-a57d-ff85b0b2c6f1\" (UID: \"dd8df313-3680-4dd1-a57d-ff85b0b2c6f1\") " Dec 03 12:49:01 crc kubenswrapper[5052]: I1203 12:49:01.516290 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-state-metrics-tls-config\" (UniqueName: \"kubernetes.io/secret/876d06e6-0187-4e3d-9183-ed2083fe5189-kube-state-metrics-tls-config\") pod \"876d06e6-0187-4e3d-9183-ed2083fe5189\" (UID: \"876d06e6-0187-4e3d-9183-ed2083fe5189\") " Dec 03 12:49:01 crc kubenswrapper[5052]: I1203 12:49:01.516335 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/876d06e6-0187-4e3d-9183-ed2083fe5189-combined-ca-bundle\") pod \"876d06e6-0187-4e3d-9183-ed2083fe5189\" (UID: \"876d06e6-0187-4e3d-9183-ed2083fe5189\") " Dec 03 12:49:01 crc kubenswrapper[5052]: I1203 12:49:01.516384 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/dd8df313-3680-4dd1-a57d-ff85b0b2c6f1-log-httpd\") pod \"dd8df313-3680-4dd1-a57d-ff85b0b2c6f1\" (UID: \"dd8df313-3680-4dd1-a57d-ff85b0b2c6f1\") " Dec 03 12:49:01 crc kubenswrapper[5052]: I1203 12:49:01.516409 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-state-metrics-tls-certs\" (UniqueName: \"kubernetes.io/secret/876d06e6-0187-4e3d-9183-ed2083fe5189-kube-state-metrics-tls-certs\") pod \"876d06e6-0187-4e3d-9183-ed2083fe5189\" (UID: \"876d06e6-0187-4e3d-9183-ed2083fe5189\") " Dec 03 12:49:01 crc kubenswrapper[5052]: I1203 12:49:01.516459 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/dd8df313-3680-4dd1-a57d-ff85b0b2c6f1-combined-ca-bundle\") pod \"dd8df313-3680-4dd1-a57d-ff85b0b2c6f1\" (UID: \"dd8df313-3680-4dd1-a57d-ff85b0b2c6f1\") " Dec 03 12:49:01 crc kubenswrapper[5052]: I1203 12:49:01.516481 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/dd8df313-3680-4dd1-a57d-ff85b0b2c6f1-run-httpd\") pod \"dd8df313-3680-4dd1-a57d-ff85b0b2c6f1\" (UID: \"dd8df313-3680-4dd1-a57d-ff85b0b2c6f1\") " Dec 03 12:49:01 crc kubenswrapper[5052]: I1203 12:49:01.516525 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qcfj5\" (UniqueName: \"kubernetes.io/projected/dd8df313-3680-4dd1-a57d-ff85b0b2c6f1-kube-api-access-qcfj5\") pod \"dd8df313-3680-4dd1-a57d-ff85b0b2c6f1\" (UID: \"dd8df313-3680-4dd1-a57d-ff85b0b2c6f1\") " Dec 03 12:49:01 crc kubenswrapper[5052]: I1203 12:49:01.516561 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6vmzm\" (UniqueName: \"kubernetes.io/projected/876d06e6-0187-4e3d-9183-ed2083fe5189-kube-api-access-6vmzm\") pod \"876d06e6-0187-4e3d-9183-ed2083fe5189\" (UID: \"876d06e6-0187-4e3d-9183-ed2083fe5189\") " Dec 03 12:49:01 crc kubenswrapper[5052]: I1203 12:49:01.516594 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/dd8df313-3680-4dd1-a57d-ff85b0b2c6f1-etc-swift\") pod \"dd8df313-3680-4dd1-a57d-ff85b0b2c6f1\" (UID: \"dd8df313-3680-4dd1-a57d-ff85b0b2c6f1\") " Dec 03 12:49:01 crc kubenswrapper[5052]: I1203 12:49:01.517336 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd8df313-3680-4dd1-a57d-ff85b0b2c6f1-log-httpd" (OuterVolumeSpecName: "log-httpd") pod "dd8df313-3680-4dd1-a57d-ff85b0b2c6f1" (UID: "dd8df313-3680-4dd1-a57d-ff85b0b2c6f1"). InnerVolumeSpecName "log-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:49:01 crc kubenswrapper[5052]: E1203 12:49:01.520593 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:49:01 crc kubenswrapper[5052]: E1203 12:49:01.520716 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a2bc501c-ebef-4e68-8ae2-e8c4217cadff-operator-scripts podName:a2bc501c-ebef-4e68-8ae2-e8c4217cadff nodeName:}" failed. No retries permitted until 2025-12-03 12:49:02.520694789 +0000 UTC m=+5333.108936553 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/a2bc501c-ebef-4e68-8ae2-e8c4217cadff-operator-scripts") pod "novacell05566-account-delete-2f4dm" (UID: "a2bc501c-ebef-4e68-8ae2-e8c4217cadff") : configmap "openstack-scripts" not found Dec 03 12:49:01 crc kubenswrapper[5052]: I1203 12:49:01.526446 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd8df313-3680-4dd1-a57d-ff85b0b2c6f1-run-httpd" (OuterVolumeSpecName: "run-httpd") pod "dd8df313-3680-4dd1-a57d-ff85b0b2c6f1" (UID: "dd8df313-3680-4dd1-a57d-ff85b0b2c6f1"). InnerVolumeSpecName "run-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:49:01 crc kubenswrapper[5052]: I1203 12:49:01.540815 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/876d06e6-0187-4e3d-9183-ed2083fe5189-kube-api-access-6vmzm" (OuterVolumeSpecName: "kube-api-access-6vmzm") pod "876d06e6-0187-4e3d-9183-ed2083fe5189" (UID: "876d06e6-0187-4e3d-9183-ed2083fe5189"). InnerVolumeSpecName "kube-api-access-6vmzm". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:49:01 crc kubenswrapper[5052]: I1203 12:49:01.553616 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dd8df313-3680-4dd1-a57d-ff85b0b2c6f1-etc-swift" (OuterVolumeSpecName: "etc-swift") pod "dd8df313-3680-4dd1-a57d-ff85b0b2c6f1" (UID: "dd8df313-3680-4dd1-a57d-ff85b0b2c6f1"). InnerVolumeSpecName "etc-swift". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:49:01 crc kubenswrapper[5052]: I1203 12:49:01.554889 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dd8df313-3680-4dd1-a57d-ff85b0b2c6f1-kube-api-access-qcfj5" (OuterVolumeSpecName: "kube-api-access-qcfj5") pod "dd8df313-3680-4dd1-a57d-ff85b0b2c6f1" (UID: "dd8df313-3680-4dd1-a57d-ff85b0b2c6f1"). InnerVolumeSpecName "kube-api-access-qcfj5". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:49:01 crc kubenswrapper[5052]: I1203 12:49:01.557198 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/novaapifd1c-account-delete-vhjzv"] Dec 03 12:49:01 crc kubenswrapper[5052]: I1203 12:49:01.575593 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/876d06e6-0187-4e3d-9183-ed2083fe5189-kube-state-metrics-tls-config" (OuterVolumeSpecName: "kube-state-metrics-tls-config") pod "876d06e6-0187-4e3d-9183-ed2083fe5189" (UID: "876d06e6-0187-4e3d-9183-ed2083fe5189"). InnerVolumeSpecName "kube-state-metrics-tls-config". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:49:01 crc kubenswrapper[5052]: I1203 12:49:01.584734 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-nb-0"] Dec 03 12:49:01 crc kubenswrapper[5052]: I1203 12:49:01.593825 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/876d06e6-0187-4e3d-9183-ed2083fe5189-kube-state-metrics-tls-certs" (OuterVolumeSpecName: "kube-state-metrics-tls-certs") pod "876d06e6-0187-4e3d-9183-ed2083fe5189" (UID: "876d06e6-0187-4e3d-9183-ed2083fe5189"). InnerVolumeSpecName "kube-state-metrics-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:49:01 crc kubenswrapper[5052]: I1203 12:49:01.596355 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-nb-0"] Dec 03 12:49:01 crc kubenswrapper[5052]: I1203 12:49:01.603625 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dd8df313-3680-4dd1-a57d-ff85b0b2c6f1-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "dd8df313-3680-4dd1-a57d-ff85b0b2c6f1" (UID: "dd8df313-3680-4dd1-a57d-ff85b0b2c6f1"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:49:01 crc kubenswrapper[5052]: I1203 12:49:01.603625 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/876d06e6-0187-4e3d-9183-ed2083fe5189-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "876d06e6-0187-4e3d-9183-ed2083fe5189" (UID: "876d06e6-0187-4e3d-9183-ed2083fe5189"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:49:01 crc kubenswrapper[5052]: I1203 12:49:01.620040 5052 reconciler_common.go:293] "Volume detached for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/dd8df313-3680-4dd1-a57d-ff85b0b2c6f1-run-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:01 crc kubenswrapper[5052]: I1203 12:49:01.620078 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-qcfj5\" (UniqueName: \"kubernetes.io/projected/dd8df313-3680-4dd1-a57d-ff85b0b2c6f1-kube-api-access-qcfj5\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:01 crc kubenswrapper[5052]: I1203 12:49:01.620088 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-6vmzm\" (UniqueName: \"kubernetes.io/projected/876d06e6-0187-4e3d-9183-ed2083fe5189-kube-api-access-6vmzm\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:01 crc kubenswrapper[5052]: I1203 12:49:01.620096 5052 reconciler_common.go:293] "Volume detached for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/dd8df313-3680-4dd1-a57d-ff85b0b2c6f1-etc-swift\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:01 crc kubenswrapper[5052]: I1203 12:49:01.620107 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/dd8df313-3680-4dd1-a57d-ff85b0b2c6f1-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:01 crc kubenswrapper[5052]: I1203 12:49:01.620121 5052 reconciler_common.go:293] "Volume detached for volume \"kube-state-metrics-tls-config\" (UniqueName: \"kubernetes.io/secret/876d06e6-0187-4e3d-9183-ed2083fe5189-kube-state-metrics-tls-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:01 crc kubenswrapper[5052]: I1203 12:49:01.620132 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/876d06e6-0187-4e3d-9183-ed2083fe5189-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:01 crc kubenswrapper[5052]: I1203 12:49:01.620143 5052 reconciler_common.go:293] "Volume detached for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/dd8df313-3680-4dd1-a57d-ff85b0b2c6f1-log-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:01 crc kubenswrapper[5052]: I1203 12:49:01.620153 5052 reconciler_common.go:293] "Volume detached for volume \"kube-state-metrics-tls-certs\" (UniqueName: \"kubernetes.io/secret/876d06e6-0187-4e3d-9183-ed2083fe5189-kube-state-metrics-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:01 crc kubenswrapper[5052]: I1203 12:49:01.640535 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dd8df313-3680-4dd1-a57d-ff85b0b2c6f1-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "dd8df313-3680-4dd1-a57d-ff85b0b2c6f1" (UID: "dd8df313-3680-4dd1-a57d-ff85b0b2c6f1"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:49:01 crc kubenswrapper[5052]: I1203 12:49:01.647811 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dd8df313-3680-4dd1-a57d-ff85b0b2c6f1-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "dd8df313-3680-4dd1-a57d-ff85b0b2c6f1" (UID: "dd8df313-3680-4dd1-a57d-ff85b0b2c6f1"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:49:01 crc kubenswrapper[5052]: I1203 12:49:01.670373 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dd8df313-3680-4dd1-a57d-ff85b0b2c6f1-config-data" (OuterVolumeSpecName: "config-data") pod "dd8df313-3680-4dd1-a57d-ff85b0b2c6f1" (UID: "dd8df313-3680-4dd1-a57d-ff85b0b2c6f1"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:49:01 crc kubenswrapper[5052]: I1203 12:49:01.688010 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2005f89c-8a19-403c-acc9-c13ede717372" path="/var/lib/kubelet/pods/2005f89c-8a19-403c-acc9-c13ede717372/volumes" Dec 03 12:49:01 crc kubenswrapper[5052]: I1203 12:49:01.688854 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2e4d7dee-0fd0-437e-a529-2dafd03c6492" path="/var/lib/kubelet/pods/2e4d7dee-0fd0-437e-a529-2dafd03c6492/volumes" Dec 03 12:49:01 crc kubenswrapper[5052]: I1203 12:49:01.690070 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5ecf0384-e075-4985-98c9-017cf55e0f7d" path="/var/lib/kubelet/pods/5ecf0384-e075-4985-98c9-017cf55e0f7d/volumes" Dec 03 12:49:01 crc kubenswrapper[5052]: I1203 12:49:01.691763 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6e49273b-7e4d-4818-8f55-2db55a1250b9" path="/var/lib/kubelet/pods/6e49273b-7e4d-4818-8f55-2db55a1250b9/volumes" Dec 03 12:49:01 crc kubenswrapper[5052]: I1203 12:49:01.692457 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7e89765b-8d00-48dc-9f2a-d7b6d7d7b5f0" path="/var/lib/kubelet/pods/7e89765b-8d00-48dc-9f2a-d7b6d7d7b5f0/volumes" Dec 03 12:49:01 crc kubenswrapper[5052]: I1203 12:49:01.693258 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="84f26afe-5b0c-43bb-a4c8-a9ac21d24330" path="/var/lib/kubelet/pods/84f26afe-5b0c-43bb-a4c8-a9ac21d24330/volumes" Dec 03 12:49:01 crc kubenswrapper[5052]: I1203 12:49:01.694349 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8996a798-f16d-47b8-98c0-7693ccee2f79" path="/var/lib/kubelet/pods/8996a798-f16d-47b8-98c0-7693ccee2f79/volumes" Dec 03 12:49:01 crc kubenswrapper[5052]: I1203 12:49:01.694926 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8a94fda3-63d8-4517-8063-f6c06688b430" path="/var/lib/kubelet/pods/8a94fda3-63d8-4517-8063-f6c06688b430/volumes" Dec 03 12:49:01 crc kubenswrapper[5052]: I1203 12:49:01.695625 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8b6e373e-2587-415b-8cbc-5fef0ba3427b" path="/var/lib/kubelet/pods/8b6e373e-2587-415b-8cbc-5fef0ba3427b/volumes" Dec 03 12:49:01 crc kubenswrapper[5052]: I1203 12:49:01.696788 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9fd7aa9d-4ad5-4058-a784-6814fbd76852" path="/var/lib/kubelet/pods/9fd7aa9d-4ad5-4058-a784-6814fbd76852/volumes" Dec 03 12:49:01 crc kubenswrapper[5052]: I1203 12:49:01.697461 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b36ab165-21eb-4968-a755-2cef353ce195" path="/var/lib/kubelet/pods/b36ab165-21eb-4968-a755-2cef353ce195/volumes" Dec 03 12:49:01 crc kubenswrapper[5052]: I1203 12:49:01.709667 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/novacell05566-account-delete-2f4dm"] Dec 03 12:49:01 crc kubenswrapper[5052]: W1203 12:49:01.710773 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda2bc501c_ebef_4e68_8ae2_e8c4217cadff.slice/crio-a06ff1694d0794e101db19e3b416b77efe12f42ffdd886b18c15f4ce2305b6f0 WatchSource:0}: Error finding container a06ff1694d0794e101db19e3b416b77efe12f42ffdd886b18c15f4ce2305b6f0: Status 404 returned error can't find the container with id a06ff1694d0794e101db19e3b416b77efe12f42ffdd886b18c15f4ce2305b6f0 Dec 03 12:49:01 crc kubenswrapper[5052]: I1203 12:49:01.721851 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/dd8df313-3680-4dd1-a57d-ff85b0b2c6f1-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:01 crc kubenswrapper[5052]: I1203 12:49:01.721887 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/dd8df313-3680-4dd1-a57d-ff85b0b2c6f1-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:01 crc kubenswrapper[5052]: I1203 12:49:01.721901 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/dd8df313-3680-4dd1-a57d-ff85b0b2c6f1-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:01 crc kubenswrapper[5052]: I1203 12:49:01.832733 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-9t4zp"] Dec 03 12:49:01 crc kubenswrapper[5052]: W1203 12:49:01.841180 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod59ec248d_84c7_4d23_b9f6_29ea8c2f36fe.slice/crio-4a1dccd1b6083cf5d0f44566001da9de050803d7e6b9a2639d43f390c4686981 WatchSource:0}: Error finding container 4a1dccd1b6083cf5d0f44566001da9de050803d7e6b9a2639d43f390c4686981: Status 404 returned error can't find the container with id 4a1dccd1b6083cf5d0f44566001da9de050803d7e6b9a2639d43f390c4686981 Dec 03 12:49:01 crc kubenswrapper[5052]: I1203 12:49:01.928425 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:49:02 crc kubenswrapper[5052]: I1203 12:49:02.027295 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/aa557411-6b0a-4596-a5f5-0e23f8686dc4-config-data\") pod \"aa557411-6b0a-4596-a5f5-0e23f8686dc4\" (UID: \"aa557411-6b0a-4596-a5f5-0e23f8686dc4\") " Dec 03 12:49:02 crc kubenswrapper[5052]: I1203 12:49:02.027342 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/aa557411-6b0a-4596-a5f5-0e23f8686dc4-combined-ca-bundle\") pod \"aa557411-6b0a-4596-a5f5-0e23f8686dc4\" (UID: \"aa557411-6b0a-4596-a5f5-0e23f8686dc4\") " Dec 03 12:49:02 crc kubenswrapper[5052]: I1203 12:49:02.027429 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/aa557411-6b0a-4596-a5f5-0e23f8686dc4-config-data-custom\") pod \"aa557411-6b0a-4596-a5f5-0e23f8686dc4\" (UID: \"aa557411-6b0a-4596-a5f5-0e23f8686dc4\") " Dec 03 12:49:02 crc kubenswrapper[5052]: I1203 12:49:02.027498 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hfmkd\" (UniqueName: \"kubernetes.io/projected/aa557411-6b0a-4596-a5f5-0e23f8686dc4-kube-api-access-hfmkd\") pod \"aa557411-6b0a-4596-a5f5-0e23f8686dc4\" (UID: \"aa557411-6b0a-4596-a5f5-0e23f8686dc4\") " Dec 03 12:49:02 crc kubenswrapper[5052]: I1203 12:49:02.027576 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/aa557411-6b0a-4596-a5f5-0e23f8686dc4-etc-machine-id\") pod \"aa557411-6b0a-4596-a5f5-0e23f8686dc4\" (UID: \"aa557411-6b0a-4596-a5f5-0e23f8686dc4\") " Dec 03 12:49:02 crc kubenswrapper[5052]: I1203 12:49:02.027648 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/aa557411-6b0a-4596-a5f5-0e23f8686dc4-scripts\") pod \"aa557411-6b0a-4596-a5f5-0e23f8686dc4\" (UID: \"aa557411-6b0a-4596-a5f5-0e23f8686dc4\") " Dec 03 12:49:02 crc kubenswrapper[5052]: I1203 12:49:02.029685 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/aa557411-6b0a-4596-a5f5-0e23f8686dc4-etc-machine-id" (OuterVolumeSpecName: "etc-machine-id") pod "aa557411-6b0a-4596-a5f5-0e23f8686dc4" (UID: "aa557411-6b0a-4596-a5f5-0e23f8686dc4"). InnerVolumeSpecName "etc-machine-id". PluginName "kubernetes.io/host-path", VolumeGidValue "" Dec 03 12:49:02 crc kubenswrapper[5052]: I1203 12:49:02.031760 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aa557411-6b0a-4596-a5f5-0e23f8686dc4-config-data-custom" (OuterVolumeSpecName: "config-data-custom") pod "aa557411-6b0a-4596-a5f5-0e23f8686dc4" (UID: "aa557411-6b0a-4596-a5f5-0e23f8686dc4"). InnerVolumeSpecName "config-data-custom". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:49:02 crc kubenswrapper[5052]: I1203 12:49:02.031958 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aa557411-6b0a-4596-a5f5-0e23f8686dc4-scripts" (OuterVolumeSpecName: "scripts") pod "aa557411-6b0a-4596-a5f5-0e23f8686dc4" (UID: "aa557411-6b0a-4596-a5f5-0e23f8686dc4"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:49:02 crc kubenswrapper[5052]: I1203 12:49:02.032407 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aa557411-6b0a-4596-a5f5-0e23f8686dc4-kube-api-access-hfmkd" (OuterVolumeSpecName: "kube-api-access-hfmkd") pod "aa557411-6b0a-4596-a5f5-0e23f8686dc4" (UID: "aa557411-6b0a-4596-a5f5-0e23f8686dc4"). InnerVolumeSpecName "kube-api-access-hfmkd". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:49:02 crc kubenswrapper[5052]: I1203 12:49:02.085908 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aa557411-6b0a-4596-a5f5-0e23f8686dc4-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "aa557411-6b0a-4596-a5f5-0e23f8686dc4" (UID: "aa557411-6b0a-4596-a5f5-0e23f8686dc4"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:49:02 crc kubenswrapper[5052]: I1203 12:49:02.130240 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/aa557411-6b0a-4596-a5f5-0e23f8686dc4-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:02 crc kubenswrapper[5052]: I1203 12:49:02.130273 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/aa557411-6b0a-4596-a5f5-0e23f8686dc4-config-data-custom\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:02 crc kubenswrapper[5052]: I1203 12:49:02.130282 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-hfmkd\" (UniqueName: \"kubernetes.io/projected/aa557411-6b0a-4596-a5f5-0e23f8686dc4-kube-api-access-hfmkd\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:02 crc kubenswrapper[5052]: I1203 12:49:02.130292 5052 reconciler_common.go:293] "Volume detached for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/aa557411-6b0a-4596-a5f5-0e23f8686dc4-etc-machine-id\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:02 crc kubenswrapper[5052]: I1203 12:49:02.130301 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/aa557411-6b0a-4596-a5f5-0e23f8686dc4-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:02 crc kubenswrapper[5052]: I1203 12:49:02.151673 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aa557411-6b0a-4596-a5f5-0e23f8686dc4-config-data" (OuterVolumeSpecName: "config-data") pod "aa557411-6b0a-4596-a5f5-0e23f8686dc4" (UID: "aa557411-6b0a-4596-a5f5-0e23f8686dc4"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:49:02 crc kubenswrapper[5052]: I1203 12:49:02.231676 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/aa557411-6b0a-4596-a5f5-0e23f8686dc4-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:02 crc kubenswrapper[5052]: I1203 12:49:02.281799 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement361c-account-delete-67ttd" event={"ID":"456c3fb1-90c0-4d17-863e-414380759f97","Type":"ContainerStarted","Data":"7427500c7abb1caced6a34336c8c514e1bf6bd1b958e728d97933c47c78d9675"} Dec 03 12:49:02 crc kubenswrapper[5052]: I1203 12:49:02.285105 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/novaapifd1c-account-delete-vhjzv" event={"ID":"2846c397-2869-49f3-aeca-cf4b56bc8834","Type":"ContainerStarted","Data":"fd19211cd437571e300e0af34e890cfd3bcac9bd1d280e66811c743e020d56b1"} Dec 03 12:49:02 crc kubenswrapper[5052]: I1203 12:49:02.287782 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-9t4zp" event={"ID":"59ec248d-84c7-4d23-b9f6-29ea8c2f36fe","Type":"ContainerStarted","Data":"4a1dccd1b6083cf5d0f44566001da9de050803d7e6b9a2639d43f390c4686981"} Dec 03 12:49:02 crc kubenswrapper[5052]: I1203 12:49:02.291018 5052 generic.go:334] "Generic (PLEG): container finished" podID="e14465ed-868a-49ba-b3ab-1e67196229e8" containerID="962dbb81a097bd1736b9740b6fea46c9644f48229fd42d2dc734698665523114" exitCode=0 Dec 03 12:49:02 crc kubenswrapper[5052]: I1203 12:49:02.291076 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-worker-58f7cb7fd9-2k5tg" event={"ID":"e14465ed-868a-49ba-b3ab-1e67196229e8","Type":"ContainerDied","Data":"962dbb81a097bd1736b9740b6fea46c9644f48229fd42d2dc734698665523114"} Dec 03 12:49:02 crc kubenswrapper[5052]: I1203 12:49:02.293894 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-proxy-78f78b88b6-fmjvp" event={"ID":"dd8df313-3680-4dd1-a57d-ff85b0b2c6f1","Type":"ContainerDied","Data":"dad1f7c6237c7c3f6729d53c0f9f7960cf9320a5a2f22fe4bc78d3fc12ff914e"} Dec 03 12:49:02 crc kubenswrapper[5052]: I1203 12:49:02.293961 5052 scope.go:117] "RemoveContainer" containerID="45b85b0267828ad53c05c095096047353cbe686d4edc60d6dabed09ff937a326" Dec 03 12:49:02 crc kubenswrapper[5052]: I1203 12:49:02.294028 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-proxy-78f78b88b6-fmjvp" Dec 03 12:49:02 crc kubenswrapper[5052]: I1203 12:49:02.295961 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutrona25a-account-delete-vbr24" event={"ID":"6ceba96c-7e86-4eb2-afa7-444f0fcc27ce","Type":"ContainerStarted","Data":"3b37742fa0a9e71beb41d34a9311dc58c88f7bd963804764ab51a685adf84ccf"} Dec 03 12:49:02 crc kubenswrapper[5052]: I1203 12:49:02.297868 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/novacell05566-account-delete-2f4dm" event={"ID":"a2bc501c-ebef-4e68-8ae2-e8c4217cadff","Type":"ContainerStarted","Data":"a06ff1694d0794e101db19e3b416b77efe12f42ffdd886b18c15f4ce2305b6f0"} Dec 03 12:49:02 crc kubenswrapper[5052]: I1203 12:49:02.304161 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glanced4a6-account-delete-69llz" event={"ID":"6a5de7fb-b482-4849-a957-e679616aabc0","Type":"ContainerStarted","Data":"7f4d1f5e1e1d3b41756391a25a136984b2f8629b0050f20cbf09bc7b030ba18f"} Dec 03 12:49:02 crc kubenswrapper[5052]: I1203 12:49:02.306882 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"aa557411-6b0a-4596-a5f5-0e23f8686dc4","Type":"ContainerDied","Data":"f9e0a759d0693033821fbf62611d284dded228a4c298c654482ae59247092610"} Dec 03 12:49:02 crc kubenswrapper[5052]: I1203 12:49:02.306922 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:49:02 crc kubenswrapper[5052]: I1203 12:49:02.306957 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystoneb51b-account-delete-ds4mz" Dec 03 12:49:02 crc kubenswrapper[5052]: I1203 12:49:02.306958 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:49:02 crc kubenswrapper[5052]: I1203 12:49:02.324832 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystoneb51b-account-delete-ds4mz" Dec 03 12:49:02 crc kubenswrapper[5052]: I1203 12:49:02.332894 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-g7ffc\" (UniqueName: \"kubernetes.io/projected/6854341b-41e8-48b0-87f7-aca589364aaf-kube-api-access-g7ffc\") pod \"keystoneb51b-account-delete-ds4mz\" (UID: \"6854341b-41e8-48b0-87f7-aca589364aaf\") " pod="openstack-kuttl-tests/keystoneb51b-account-delete-ds4mz" Dec 03 12:49:02 crc kubenswrapper[5052]: I1203 12:49:02.332945 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/6854341b-41e8-48b0-87f7-aca589364aaf-operator-scripts\") pod \"keystoneb51b-account-delete-ds4mz\" (UID: \"6854341b-41e8-48b0-87f7-aca589364aaf\") " pod="openstack-kuttl-tests/keystoneb51b-account-delete-ds4mz" Dec 03 12:49:02 crc kubenswrapper[5052]: E1203 12:49:02.333270 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:49:02 crc kubenswrapper[5052]: E1203 12:49:02.333325 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6854341b-41e8-48b0-87f7-aca589364aaf-operator-scripts podName:6854341b-41e8-48b0-87f7-aca589364aaf nodeName:}" failed. No retries permitted until 2025-12-03 12:49:04.333309479 +0000 UTC m=+5334.921551243 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/6854341b-41e8-48b0-87f7-aca589364aaf-operator-scripts") pod "keystoneb51b-account-delete-ds4mz" (UID: "6854341b-41e8-48b0-87f7-aca589364aaf") : configmap "openstack-scripts" not found Dec 03 12:49:02 crc kubenswrapper[5052]: E1203 12:49:02.339376 5052 projected.go:194] Error preparing data for projected volume kube-api-access-g7ffc for pod openstack-kuttl-tests/keystoneb51b-account-delete-ds4mz: failed to fetch token: serviceaccounts "galera-openstack" not found Dec 03 12:49:02 crc kubenswrapper[5052]: E1203 12:49:02.339697 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6854341b-41e8-48b0-87f7-aca589364aaf-kube-api-access-g7ffc podName:6854341b-41e8-48b0-87f7-aca589364aaf nodeName:}" failed. No retries permitted until 2025-12-03 12:49:04.339497096 +0000 UTC m=+5334.927738860 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-g7ffc" (UniqueName: "kubernetes.io/projected/6854341b-41e8-48b0-87f7-aca589364aaf-kube-api-access-g7ffc") pod "keystoneb51b-account-delete-ds4mz" (UID: "6854341b-41e8-48b0-87f7-aca589364aaf") : failed to fetch token: serviceaccounts "galera-openstack" not found Dec 03 12:49:02 crc kubenswrapper[5052]: I1203 12:49:02.340838 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/swift-proxy-78f78b88b6-fmjvp"] Dec 03 12:49:02 crc kubenswrapper[5052]: I1203 12:49:02.348399 5052 scope.go:117] "RemoveContainer" containerID="acb0be96c14d669ff7b80a9476845494757b206ec5aaddb2a34ac9ff0fe87bd7" Dec 03 12:49:02 crc kubenswrapper[5052]: I1203 12:49:02.353276 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/swift-proxy-78f78b88b6-fmjvp"] Dec 03 12:49:02 crc kubenswrapper[5052]: I1203 12:49:02.362645 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/kube-state-metrics-0"] Dec 03 12:49:02 crc kubenswrapper[5052]: I1203 12:49:02.373629 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/kube-state-metrics-0"] Dec 03 12:49:02 crc kubenswrapper[5052]: I1203 12:49:02.373847 5052 scope.go:117] "RemoveContainer" containerID="1d9679ea8e92979b2a265175ba31597cc56e80e13b2610ba8cc4b4e015500253" Dec 03 12:49:02 crc kubenswrapper[5052]: I1203 12:49:02.380934 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 12:49:02 crc kubenswrapper[5052]: I1203 12:49:02.387076 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 12:49:02 crc kubenswrapper[5052]: I1203 12:49:02.393776 5052 scope.go:117] "RemoveContainer" containerID="28dad3a0f48d004bf9433a47f2e9bb435f9945510d0f67da8d4246236a90195d" Dec 03 12:49:02 crc kubenswrapper[5052]: E1203 12:49:02.435496 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:49:02 crc kubenswrapper[5052]: E1203 12:49:02.435580 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2846c397-2869-49f3-aeca-cf4b56bc8834-operator-scripts podName:2846c397-2869-49f3-aeca-cf4b56bc8834 nodeName:}" failed. No retries permitted until 2025-12-03 12:49:04.435565822 +0000 UTC m=+5335.023807586 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/2846c397-2869-49f3-aeca-cf4b56bc8834-operator-scripts") pod "novaapifd1c-account-delete-vhjzv" (UID: "2846c397-2869-49f3-aeca-cf4b56bc8834") : configmap "openstack-scripts" not found Dec 03 12:49:02 crc kubenswrapper[5052]: E1203 12:49:02.539718 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:49:02 crc kubenswrapper[5052]: E1203 12:49:02.539777 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a2bc501c-ebef-4e68-8ae2-e8c4217cadff-operator-scripts podName:a2bc501c-ebef-4e68-8ae2-e8c4217cadff nodeName:}" failed. No retries permitted until 2025-12-03 12:49:04.539759302 +0000 UTC m=+5335.128001066 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/a2bc501c-ebef-4e68-8ae2-e8c4217cadff-operator-scripts") pod "novacell05566-account-delete-2f4dm" (UID: "a2bc501c-ebef-4e68-8ae2-e8c4217cadff") : configmap "openstack-scripts" not found Dec 03 12:49:02 crc kubenswrapper[5052]: I1203 12:49:02.713138 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/placement-db-create-69hg4"] Dec 03 12:49:02 crc kubenswrapper[5052]: I1203 12:49:02.744207 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/placement-db-create-69hg4"] Dec 03 12:49:02 crc kubenswrapper[5052]: I1203 12:49:02.763562 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/placement-361c-account-create-update-4psp5"] Dec 03 12:49:02 crc kubenswrapper[5052]: I1203 12:49:02.772822 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/placement361c-account-delete-67ttd"] Dec 03 12:49:02 crc kubenswrapper[5052]: I1203 12:49:02.775433 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/placement-361c-account-create-update-4psp5"] Dec 03 12:49:02 crc kubenswrapper[5052]: I1203 12:49:02.895757 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:49:02 crc kubenswrapper[5052]: I1203 12:49:02.913024 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-db-create-mfcd6"] Dec 03 12:49:02 crc kubenswrapper[5052]: I1203 12:49:02.935534 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/glance-db-create-mfcd6"] Dec 03 12:49:02 crc kubenswrapper[5052]: I1203 12:49:02.952056 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"vencrypt-tls-certs\" (UniqueName: \"kubernetes.io/secret/edae7848-51ff-40d6-83f2-ffff085f3f3e-vencrypt-tls-certs\") pod \"edae7848-51ff-40d6-83f2-ffff085f3f3e\" (UID: \"edae7848-51ff-40d6-83f2-ffff085f3f3e\") " Dec 03 12:49:02 crc kubenswrapper[5052]: I1203 12:49:02.952147 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/edae7848-51ff-40d6-83f2-ffff085f3f3e-combined-ca-bundle\") pod \"edae7848-51ff-40d6-83f2-ffff085f3f3e\" (UID: \"edae7848-51ff-40d6-83f2-ffff085f3f3e\") " Dec 03 12:49:02 crc kubenswrapper[5052]: I1203 12:49:02.952179 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kk6gr\" (UniqueName: \"kubernetes.io/projected/edae7848-51ff-40d6-83f2-ffff085f3f3e-kube-api-access-kk6gr\") pod \"edae7848-51ff-40d6-83f2-ffff085f3f3e\" (UID: \"edae7848-51ff-40d6-83f2-ffff085f3f3e\") " Dec 03 12:49:02 crc kubenswrapper[5052]: I1203 12:49:02.952208 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"nova-novncproxy-tls-certs\" (UniqueName: \"kubernetes.io/secret/edae7848-51ff-40d6-83f2-ffff085f3f3e-nova-novncproxy-tls-certs\") pod \"edae7848-51ff-40d6-83f2-ffff085f3f3e\" (UID: \"edae7848-51ff-40d6-83f2-ffff085f3f3e\") " Dec 03 12:49:02 crc kubenswrapper[5052]: I1203 12:49:02.952232 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/edae7848-51ff-40d6-83f2-ffff085f3f3e-config-data\") pod \"edae7848-51ff-40d6-83f2-ffff085f3f3e\" (UID: \"edae7848-51ff-40d6-83f2-ffff085f3f3e\") " Dec 03 12:49:02 crc kubenswrapper[5052]: I1203 12:49:02.968114 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-d4a6-account-create-update-8qdvr"] Dec 03 12:49:02 crc kubenswrapper[5052]: I1203 12:49:02.975474 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/edae7848-51ff-40d6-83f2-ffff085f3f3e-kube-api-access-kk6gr" (OuterVolumeSpecName: "kube-api-access-kk6gr") pod "edae7848-51ff-40d6-83f2-ffff085f3f3e" (UID: "edae7848-51ff-40d6-83f2-ffff085f3f3e"). InnerVolumeSpecName "kube-api-access-kk6gr". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:49:02 crc kubenswrapper[5052]: I1203 12:49:02.978129 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/glance-d4a6-account-create-update-8qdvr"] Dec 03 12:49:02 crc kubenswrapper[5052]: I1203 12:49:02.992895 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glanced4a6-account-delete-69llz"] Dec 03 12:49:03 crc kubenswrapper[5052]: I1203 12:49:03.009950 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder-db-create-jmgjk"] Dec 03 12:49:03 crc kubenswrapper[5052]: I1203 12:49:03.018041 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/cinder-db-create-jmgjk"] Dec 03 12:49:03 crc kubenswrapper[5052]: I1203 12:49:03.032690 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder4345-account-delete-vmdzh"] Dec 03 12:49:03 crc kubenswrapper[5052]: I1203 12:49:03.033585 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/edae7848-51ff-40d6-83f2-ffff085f3f3e-config-data" (OuterVolumeSpecName: "config-data") pod "edae7848-51ff-40d6-83f2-ffff085f3f3e" (UID: "edae7848-51ff-40d6-83f2-ffff085f3f3e"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:49:03 crc kubenswrapper[5052]: I1203 12:49:03.040369 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder-4345-account-create-update-qqj8v"] Dec 03 12:49:03 crc kubenswrapper[5052]: I1203 12:49:03.049036 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/cinder-4345-account-create-update-qqj8v"] Dec 03 12:49:03 crc kubenswrapper[5052]: I1203 12:49:03.057806 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-kk6gr\" (UniqueName: \"kubernetes.io/projected/edae7848-51ff-40d6-83f2-ffff085f3f3e-kube-api-access-kk6gr\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:03 crc kubenswrapper[5052]: I1203 12:49:03.057839 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/edae7848-51ff-40d6-83f2-ffff085f3f3e-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:03 crc kubenswrapper[5052]: I1203 12:49:03.102685 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/neutron-db-create-jprpg"] Dec 03 12:49:03 crc kubenswrapper[5052]: I1203 12:49:03.119491 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/edae7848-51ff-40d6-83f2-ffff085f3f3e-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "edae7848-51ff-40d6-83f2-ffff085f3f3e" (UID: "edae7848-51ff-40d6-83f2-ffff085f3f3e"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:49:03 crc kubenswrapper[5052]: I1203 12:49:03.130629 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/neutron-db-create-jprpg"] Dec 03 12:49:03 crc kubenswrapper[5052]: I1203 12:49:03.149251 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/edae7848-51ff-40d6-83f2-ffff085f3f3e-vencrypt-tls-certs" (OuterVolumeSpecName: "vencrypt-tls-certs") pod "edae7848-51ff-40d6-83f2-ffff085f3f3e" (UID: "edae7848-51ff-40d6-83f2-ffff085f3f3e"). InnerVolumeSpecName "vencrypt-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:49:03 crc kubenswrapper[5052]: I1203 12:49:03.160980 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/edae7848-51ff-40d6-83f2-ffff085f3f3e-nova-novncproxy-tls-certs" (OuterVolumeSpecName: "nova-novncproxy-tls-certs") pod "edae7848-51ff-40d6-83f2-ffff085f3f3e" (UID: "edae7848-51ff-40d6-83f2-ffff085f3f3e"). InnerVolumeSpecName "nova-novncproxy-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:49:03 crc kubenswrapper[5052]: I1203 12:49:03.168238 5052 reconciler_common.go:293] "Volume detached for volume \"vencrypt-tls-certs\" (UniqueName: \"kubernetes.io/secret/edae7848-51ff-40d6-83f2-ffff085f3f3e-vencrypt-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:03 crc kubenswrapper[5052]: I1203 12:49:03.168287 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/edae7848-51ff-40d6-83f2-ffff085f3f3e-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:03 crc kubenswrapper[5052]: I1203 12:49:03.168301 5052 reconciler_common.go:293] "Volume detached for volume \"nova-novncproxy-tls-certs\" (UniqueName: \"kubernetes.io/secret/edae7848-51ff-40d6-83f2-ffff085f3f3e-nova-novncproxy-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:03 crc kubenswrapper[5052]: I1203 12:49:03.180167 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/neutrona25a-account-delete-vbr24"] Dec 03 12:49:03 crc kubenswrapper[5052]: I1203 12:49:03.180398 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/nova-metadata-0" podUID="efb0e60c-2a71-4641-87b2-0f5532b28678" containerName="nova-metadata-log" probeResult="failure" output="Get \"https://10.217.0.119:8775/\": read tcp 10.217.0.2:42658->10.217.0.119:8775: read: connection reset by peer" Dec 03 12:49:03 crc kubenswrapper[5052]: I1203 12:49:03.180568 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/nova-metadata-0" podUID="efb0e60c-2a71-4641-87b2-0f5532b28678" containerName="nova-metadata-metadata" probeResult="failure" output="Get \"https://10.217.0.119:8775/\": read tcp 10.217.0.2:42660->10.217.0.119:8775: read: connection reset by peer" Dec 03 12:49:03 crc kubenswrapper[5052]: I1203 12:49:03.222580 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/neutron-a25a-account-create-update-4bq8m"] Dec 03 12:49:03 crc kubenswrapper[5052]: I1203 12:49:03.224125 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/cinder-api-0" podUID="557175a9-81e5-4605-bd67-f1dfa575fb1b" containerName="cinder-api" probeResult="failure" output="Get \"https://10.217.0.114:8776/healthcheck\": dial tcp 10.217.0.114:8776: connect: connection refused" Dec 03 12:49:03 crc kubenswrapper[5052]: I1203 12:49:03.251558 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/neutron-a25a-account-create-update-4bq8m"] Dec 03 12:49:03 crc kubenswrapper[5052]: I1203 12:49:03.275483 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-db-create-5hb92"] Dec 03 12:49:03 crc kubenswrapper[5052]: I1203 12:49:03.280283 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/barbican-db-create-5hb92"] Dec 03 12:49:03 crc kubenswrapper[5052]: E1203 12:49:03.287159 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="86f66b1381c35478a52f56b7c8ce2c98cdc37bae9cd163bbea7b6f45c6c705b9" cmd=["/bin/bash","/var/lib/operator-scripts/mysql_probe.sh","readiness"] Dec 03 12:49:03 crc kubenswrapper[5052]: E1203 12:49:03.298899 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="86f66b1381c35478a52f56b7c8ce2c98cdc37bae9cd163bbea7b6f45c6c705b9" cmd=["/bin/bash","/var/lib/operator-scripts/mysql_probe.sh","readiness"] Dec 03 12:49:03 crc kubenswrapper[5052]: I1203 12:49:03.302635 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican02cc-account-delete-gf94b"] Dec 03 12:49:03 crc kubenswrapper[5052]: E1203 12:49:03.302674 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="86f66b1381c35478a52f56b7c8ce2c98cdc37bae9cd163bbea7b6f45c6c705b9" cmd=["/bin/bash","/var/lib/operator-scripts/mysql_probe.sh","readiness"] Dec 03 12:49:03 crc kubenswrapper[5052]: E1203 12:49:03.302741 5052 prober.go:104] "Probe errored" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" probeType="Readiness" pod="openstack-kuttl-tests/openstack-galera-0" podUID="3c8e4e3f-f6c1-4362-b64c-b10a973723dd" containerName="galera" Dec 03 12:49:03 crc kubenswrapper[5052]: I1203 12:49:03.311367 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-02cc-account-create-update-zzv8d"] Dec 03 12:49:03 crc kubenswrapper[5052]: I1203 12:49:03.322261 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/barbican-02cc-account-create-update-zzv8d"] Dec 03 12:49:03 crc kubenswrapper[5052]: I1203 12:49:03.328096 5052 generic.go:334] "Generic (PLEG): container finished" podID="59ec248d-84c7-4d23-b9f6-29ea8c2f36fe" containerID="1e281e7872fa27702c42c19852a987a152988cec19cb89cd3614fa3d4e9a5e56" exitCode=0 Dec 03 12:49:03 crc kubenswrapper[5052]: I1203 12:49:03.328164 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-9t4zp" event={"ID":"59ec248d-84c7-4d23-b9f6-29ea8c2f36fe","Type":"ContainerDied","Data":"1e281e7872fa27702c42c19852a987a152988cec19cb89cd3614fa3d4e9a5e56"} Dec 03 12:49:03 crc kubenswrapper[5052]: I1203 12:49:03.330212 5052 generic.go:334] "Generic (PLEG): container finished" podID="456c3fb1-90c0-4d17-863e-414380759f97" containerID="7427500c7abb1caced6a34336c8c514e1bf6bd1b958e728d97933c47c78d9675" exitCode=1 Dec 03 12:49:03 crc kubenswrapper[5052]: I1203 12:49:03.330262 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement361c-account-delete-67ttd" event={"ID":"456c3fb1-90c0-4d17-863e-414380759f97","Type":"ContainerDied","Data":"7427500c7abb1caced6a34336c8c514e1bf6bd1b958e728d97933c47c78d9675"} Dec 03 12:49:03 crc kubenswrapper[5052]: I1203 12:49:03.330818 5052 kubelet_pods.go:1007] "Unable to retrieve pull secret, the image pull may not succeed." pod="openstack-kuttl-tests/placement361c-account-delete-67ttd" secret="" err="secret \"galera-openstack-dockercfg-tsmcq\" not found" Dec 03 12:49:03 crc kubenswrapper[5052]: I1203 12:49:03.330856 5052 scope.go:117] "RemoveContainer" containerID="7427500c7abb1caced6a34336c8c514e1bf6bd1b958e728d97933c47c78d9675" Dec 03 12:49:03 crc kubenswrapper[5052]: I1203 12:49:03.356581 5052 generic.go:334] "Generic (PLEG): container finished" podID="5201872c-fde1-49d6-9784-6a7f54497990" containerID="d07d82d1f66b2f1adb94be8826e8db31e731aedc69314cb16576303298684ad0" exitCode=0 Dec 03 12:49:03 crc kubenswrapper[5052]: I1203 12:49:03.356690 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-74746c7f4-h6sx8" event={"ID":"5201872c-fde1-49d6-9784-6a7f54497990","Type":"ContainerDied","Data":"d07d82d1f66b2f1adb94be8826e8db31e731aedc69314cb16576303298684ad0"} Dec 03 12:49:03 crc kubenswrapper[5052]: I1203 12:49:03.358897 5052 generic.go:334] "Generic (PLEG): container finished" podID="a2bc501c-ebef-4e68-8ae2-e8c4217cadff" containerID="a2b104f1b7080afce6644e6daf4c349c7a2bb8f2c68da0f5a404e6190c2e33ad" exitCode=1 Dec 03 12:49:03 crc kubenswrapper[5052]: I1203 12:49:03.358946 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/novacell05566-account-delete-2f4dm" event={"ID":"a2bc501c-ebef-4e68-8ae2-e8c4217cadff","Type":"ContainerDied","Data":"a2b104f1b7080afce6644e6daf4c349c7a2bb8f2c68da0f5a404e6190c2e33ad"} Dec 03 12:49:03 crc kubenswrapper[5052]: I1203 12:49:03.361356 5052 kubelet_pods.go:1007] "Unable to retrieve pull secret, the image pull may not succeed." pod="openstack-kuttl-tests/novacell05566-account-delete-2f4dm" secret="" err="secret \"galera-openstack-dockercfg-tsmcq\" not found" Dec 03 12:49:03 crc kubenswrapper[5052]: I1203 12:49:03.361396 5052 scope.go:117] "RemoveContainer" containerID="a2b104f1b7080afce6644e6daf4c349c7a2bb8f2c68da0f5a404e6190c2e33ad" Dec 03 12:49:03 crc kubenswrapper[5052]: I1203 12:49:03.366401 5052 generic.go:334] "Generic (PLEG): container finished" podID="857b8a8a-d730-4832-a71c-d45a2f3ce406" containerID="626ebae6286b0199eb107048b8e04a3c2f172a614197d9cbb4a4d9047544b8e0" exitCode=1 Dec 03 12:49:03 crc kubenswrapper[5052]: I1203 12:49:03.366472 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican02cc-account-delete-gf94b" event={"ID":"857b8a8a-d730-4832-a71c-d45a2f3ce406","Type":"ContainerDied","Data":"626ebae6286b0199eb107048b8e04a3c2f172a614197d9cbb4a4d9047544b8e0"} Dec 03 12:49:03 crc kubenswrapper[5052]: I1203 12:49:03.366962 5052 kubelet_pods.go:1007] "Unable to retrieve pull secret, the image pull may not succeed." pod="openstack-kuttl-tests/barbican02cc-account-delete-gf94b" secret="" err="secret \"galera-openstack-dockercfg-tsmcq\" not found" Dec 03 12:49:03 crc kubenswrapper[5052]: I1203 12:49:03.367000 5052 scope.go:117] "RemoveContainer" containerID="626ebae6286b0199eb107048b8e04a3c2f172a614197d9cbb4a4d9047544b8e0" Dec 03 12:49:03 crc kubenswrapper[5052]: I1203 12:49:03.370979 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" event={"ID":"edae7848-51ff-40d6-83f2-ffff085f3f3e","Type":"ContainerDied","Data":"d0169811e45510e60a9be05d74b820c1652bf1608236beac08ab52eb9b8b22b5"} Dec 03 12:49:03 crc kubenswrapper[5052]: I1203 12:49:03.371027 5052 scope.go:117] "RemoveContainer" containerID="1470a22e3c4e2dd79a87f38bb53dbab1acc3ac13debd607ce9ac04a443f55380" Dec 03 12:49:03 crc kubenswrapper[5052]: I1203 12:49:03.371117 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:49:03 crc kubenswrapper[5052]: I1203 12:49:03.393796 5052 generic.go:334] "Generic (PLEG): container finished" podID="557175a9-81e5-4605-bd67-f1dfa575fb1b" containerID="4f6a6de545ed43fd89fd3c33b250e3e6c4f74774fb05bed0727171451864fd53" exitCode=0 Dec 03 12:49:03 crc kubenswrapper[5052]: I1203 12:49:03.393855 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"557175a9-81e5-4605-bd67-f1dfa575fb1b","Type":"ContainerDied","Data":"4f6a6de545ed43fd89fd3c33b250e3e6c4f74774fb05bed0727171451864fd53"} Dec 03 12:49:03 crc kubenswrapper[5052]: I1203 12:49:03.397339 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/novaapifd1c-account-delete-vhjzv" event={"ID":"2846c397-2869-49f3-aeca-cf4b56bc8834","Type":"ContainerStarted","Data":"747dff2aa2b4006f1eacd1565f1e19dcab8b056e4b9a569dfe7735eef649cbe8"} Dec 03 12:49:03 crc kubenswrapper[5052]: I1203 12:49:03.397923 5052 kubelet_pods.go:1007] "Unable to retrieve pull secret, the image pull may not succeed." pod="openstack-kuttl-tests/novaapifd1c-account-delete-vhjzv" secret="" err="secret \"galera-openstack-dockercfg-tsmcq\" not found" Dec 03 12:49:03 crc kubenswrapper[5052]: I1203 12:49:03.397957 5052 scope.go:117] "RemoveContainer" containerID="747dff2aa2b4006f1eacd1565f1e19dcab8b056e4b9a569dfe7735eef649cbe8" Dec 03 12:49:03 crc kubenswrapper[5052]: I1203 12:49:03.402362 5052 generic.go:334] "Generic (PLEG): container finished" podID="eef9c304-b598-4a31-9c47-09be0087c25e" containerID="3cec53cfcf32bebda98bd1f984b0bb1ae1c9eaa0f2a096f12724a271e1103ca8" exitCode=0 Dec 03 12:49:03 crc kubenswrapper[5052]: I1203 12:49:03.402425 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/memcached-0" event={"ID":"eef9c304-b598-4a31-9c47-09be0087c25e","Type":"ContainerDied","Data":"3cec53cfcf32bebda98bd1f984b0bb1ae1c9eaa0f2a096f12724a271e1103ca8"} Dec 03 12:49:03 crc kubenswrapper[5052]: I1203 12:49:03.409977 5052 generic.go:334] "Generic (PLEG): container finished" podID="3479f9ad-871f-463e-96ce-50f4069088b3" containerID="bf37583e842ac2986eebf906db0e42704630e142c5ffc66b3b257042bb9c2bf5" exitCode=0 Dec 03 12:49:03 crc kubenswrapper[5052]: I1203 12:49:03.410022 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-cell1-galera-0" event={"ID":"3479f9ad-871f-463e-96ce-50f4069088b3","Type":"ContainerDied","Data":"bf37583e842ac2986eebf906db0e42704630e142c5ffc66b3b257042bb9c2bf5"} Dec 03 12:49:03 crc kubenswrapper[5052]: I1203 12:49:03.410077 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-cell1-galera-0" event={"ID":"3479f9ad-871f-463e-96ce-50f4069088b3","Type":"ContainerDied","Data":"e463002f432d54290e18bcb85de7ca23165a1437ec67a5d132c0921026ce9ec1"} Dec 03 12:49:03 crc kubenswrapper[5052]: I1203 12:49:03.410089 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e463002f432d54290e18bcb85de7ca23165a1437ec67a5d132c0921026ce9ec1" Dec 03 12:49:03 crc kubenswrapper[5052]: I1203 12:49:03.459044 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-worker-58f7cb7fd9-2k5tg" event={"ID":"e14465ed-868a-49ba-b3ab-1e67196229e8","Type":"ContainerDied","Data":"45485570e1f7f9e7f27f00dd5245dc00110787efec07233dcc8738241e8446fe"} Dec 03 12:49:03 crc kubenswrapper[5052]: I1203 12:49:03.459101 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="45485570e1f7f9e7f27f00dd5245dc00110787efec07233dcc8738241e8446fe" Dec 03 12:49:03 crc kubenswrapper[5052]: I1203 12:49:03.505134 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-api-db-create-dx5bv"] Dec 03 12:49:03 crc kubenswrapper[5052]: I1203 12:49:03.505475 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/memcached-0" podUID="eef9c304-b598-4a31-9c47-09be0087c25e" containerName="memcached" probeResult="failure" output="dial tcp 10.217.0.99:11211: connect: connection refused" Dec 03 12:49:03 crc kubenswrapper[5052]: E1203 12:49:03.507788 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:49:03 crc kubenswrapper[5052]: E1203 12:49:03.507863 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/456c3fb1-90c0-4d17-863e-414380759f97-operator-scripts podName:456c3fb1-90c0-4d17-863e-414380759f97 nodeName:}" failed. No retries permitted until 2025-12-03 12:49:04.007845478 +0000 UTC m=+5334.596087242 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/456c3fb1-90c0-4d17-863e-414380759f97-operator-scripts") pod "placement361c-account-delete-67ttd" (UID: "456c3fb1-90c0-4d17-863e-414380759f97") : configmap "openstack-scripts" not found Dec 03 12:49:03 crc kubenswrapper[5052]: E1203 12:49:03.508966 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:49:03 crc kubenswrapper[5052]: E1203 12:49:03.509022 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/857b8a8a-d730-4832-a71c-d45a2f3ce406-operator-scripts podName:857b8a8a-d730-4832-a71c-d45a2f3ce406 nodeName:}" failed. No retries permitted until 2025-12-03 12:49:04.009001071 +0000 UTC m=+5334.597242835 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/857b8a8a-d730-4832-a71c-d45a2f3ce406-operator-scripts") pod "barbican02cc-account-delete-gf94b" (UID: "857b8a8a-d730-4832-a71c-d45a2f3ce406") : configmap "openstack-scripts" not found Dec 03 12:49:03 crc kubenswrapper[5052]: I1203 12:49:03.515439 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder4345-account-delete-vmdzh" event={"ID":"5d11b02e-c14a-4d1a-819a-4c3b66aedd33","Type":"ContainerStarted","Data":"b58ebff57a6b5eca68fc9558830a4aee95150c885bec5d3131628c10c8d77c4e"} Dec 03 12:49:03 crc kubenswrapper[5052]: I1203 12:49:03.516235 5052 kubelet_pods.go:1007] "Unable to retrieve pull secret, the image pull may not succeed." pod="openstack-kuttl-tests/cinder4345-account-delete-vmdzh" secret="" err="secret \"galera-openstack-dockercfg-tsmcq\" not found" Dec 03 12:49:03 crc kubenswrapper[5052]: I1203 12:49:03.516281 5052 scope.go:117] "RemoveContainer" containerID="b58ebff57a6b5eca68fc9558830a4aee95150c885bec5d3131628c10c8d77c4e" Dec 03 12:49:03 crc kubenswrapper[5052]: I1203 12:49:03.517853 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-api-db-create-dx5bv"] Dec 03 12:49:03 crc kubenswrapper[5052]: I1203 12:49:03.548840 5052 generic.go:334] "Generic (PLEG): container finished" podID="6ceba96c-7e86-4eb2-afa7-444f0fcc27ce" containerID="3b37742fa0a9e71beb41d34a9311dc58c88f7bd963804764ab51a685adf84ccf" exitCode=1 Dec 03 12:49:03 crc kubenswrapper[5052]: I1203 12:49:03.549370 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutrona25a-account-delete-vbr24" event={"ID":"6ceba96c-7e86-4eb2-afa7-444f0fcc27ce","Type":"ContainerDied","Data":"3b37742fa0a9e71beb41d34a9311dc58c88f7bd963804764ab51a685adf84ccf"} Dec 03 12:49:03 crc kubenswrapper[5052]: E1203 12:49:03.549855 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="6964abaffe4177de7695a956baa51adeedd8fc7ae4b4885ca8e38046eae0369a" cmd=["/usr/bin/pgrep","-r","DRST","nova-scheduler"] Dec 03 12:49:03 crc kubenswrapper[5052]: I1203 12:49:03.550727 5052 kubelet_pods.go:1007] "Unable to retrieve pull secret, the image pull may not succeed." pod="openstack-kuttl-tests/neutrona25a-account-delete-vbr24" secret="" err="secret \"galera-openstack-dockercfg-tsmcq\" not found" Dec 03 12:49:03 crc kubenswrapper[5052]: I1203 12:49:03.550764 5052 scope.go:117] "RemoveContainer" containerID="3b37742fa0a9e71beb41d34a9311dc58c88f7bd963804764ab51a685adf84ccf" Dec 03 12:49:03 crc kubenswrapper[5052]: E1203 12:49:03.562107 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="6964abaffe4177de7695a956baa51adeedd8fc7ae4b4885ca8e38046eae0369a" cmd=["/usr/bin/pgrep","-r","DRST","nova-scheduler"] Dec 03 12:49:03 crc kubenswrapper[5052]: I1203 12:49:03.567241 5052 generic.go:334] "Generic (PLEG): container finished" podID="b1a74998-b009-4bb6-a0dc-d94ffae8b641" containerID="f28ffd80db46452a6cf33fcc0825d49c5f14be470338b023f15a46aa3b004932" exitCode=0 Dec 03 12:49:03 crc kubenswrapper[5052]: I1203 12:49:03.567560 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"b1a74998-b009-4bb6-a0dc-d94ffae8b641","Type":"ContainerDied","Data":"f28ffd80db46452a6cf33fcc0825d49c5f14be470338b023f15a46aa3b004932"} Dec 03 12:49:03 crc kubenswrapper[5052]: I1203 12:49:03.569660 5052 generic.go:334] "Generic (PLEG): container finished" podID="6a5de7fb-b482-4849-a957-e679616aabc0" containerID="7f4d1f5e1e1d3b41756391a25a136984b2f8629b0050f20cbf09bc7b030ba18f" exitCode=1 Dec 03 12:49:03 crc kubenswrapper[5052]: I1203 12:49:03.569730 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystoneb51b-account-delete-ds4mz" Dec 03 12:49:03 crc kubenswrapper[5052]: I1203 12:49:03.571042 5052 kubelet_pods.go:1007] "Unable to retrieve pull secret, the image pull may not succeed." pod="openstack-kuttl-tests/glanced4a6-account-delete-69llz" secret="" err="secret \"galera-openstack-dockercfg-tsmcq\" not found" Dec 03 12:49:03 crc kubenswrapper[5052]: I1203 12:49:03.571094 5052 scope.go:117] "RemoveContainer" containerID="7f4d1f5e1e1d3b41756391a25a136984b2f8629b0050f20cbf09bc7b030ba18f" Dec 03 12:49:03 crc kubenswrapper[5052]: I1203 12:49:03.575774 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glanced4a6-account-delete-69llz" event={"ID":"6a5de7fb-b482-4849-a957-e679616aabc0","Type":"ContainerDied","Data":"7f4d1f5e1e1d3b41756391a25a136984b2f8629b0050f20cbf09bc7b030ba18f"} Dec 03 12:49:03 crc kubenswrapper[5052]: I1203 12:49:03.575826 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/novaapifd1c-account-delete-vhjzv"] Dec 03 12:49:03 crc kubenswrapper[5052]: I1203 12:49:03.586147 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-api-fd1c-account-create-update-vxzns"] Dec 03 12:49:03 crc kubenswrapper[5052]: E1203 12:49:03.595277 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="6964abaffe4177de7695a956baa51adeedd8fc7ae4b4885ca8e38046eae0369a" cmd=["/usr/bin/pgrep","-r","DRST","nova-scheduler"] Dec 03 12:49:03 crc kubenswrapper[5052]: E1203 12:49:03.595372 5052 prober.go:104] "Probe errored" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" probeType="Readiness" pod="openstack-kuttl-tests/nova-scheduler-0" podUID="81709983-1fd0-4446-bbbf-fb94653bb3d2" containerName="nova-scheduler-scheduler" Dec 03 12:49:03 crc kubenswrapper[5052]: I1203 12:49:03.598102 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-api-fd1c-account-create-update-vxzns"] Dec 03 12:49:03 crc kubenswrapper[5052]: E1203 12:49:03.611216 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:49:03 crc kubenswrapper[5052]: E1203 12:49:03.611272 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5d11b02e-c14a-4d1a-819a-4c3b66aedd33-operator-scripts podName:5d11b02e-c14a-4d1a-819a-4c3b66aedd33 nodeName:}" failed. No retries permitted until 2025-12-03 12:49:04.111255625 +0000 UTC m=+5334.699497389 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/5d11b02e-c14a-4d1a-819a-4c3b66aedd33-operator-scripts") pod "cinder4345-account-delete-vmdzh" (UID: "5d11b02e-c14a-4d1a-819a-4c3b66aedd33") : configmap "openstack-scripts" not found Dec 03 12:49:03 crc kubenswrapper[5052]: I1203 12:49:03.652230 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell0-db-create-ztpwm"] Dec 03 12:49:03 crc kubenswrapper[5052]: I1203 12:49:03.653224 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-worker-58f7cb7fd9-2k5tg" Dec 03 12:49:03 crc kubenswrapper[5052]: I1203 12:49:03.659569 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell0-db-create-ztpwm"] Dec 03 12:49:03 crc kubenswrapper[5052]: E1203 12:49:03.712578 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:49:03 crc kubenswrapper[5052]: E1203 12:49:03.712638 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6ceba96c-7e86-4eb2-afa7-444f0fcc27ce-operator-scripts podName:6ceba96c-7e86-4eb2-afa7-444f0fcc27ce nodeName:}" failed. No retries permitted until 2025-12-03 12:49:04.212621034 +0000 UTC m=+5334.800862798 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/6ceba96c-7e86-4eb2-afa7-444f0fcc27ce-operator-scripts") pod "neutrona25a-account-delete-vbr24" (UID: "6ceba96c-7e86-4eb2-afa7-444f0fcc27ce") : configmap "openstack-scripts" not found Dec 03 12:49:03 crc kubenswrapper[5052]: E1203 12:49:03.712648 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:49:03 crc kubenswrapper[5052]: E1203 12:49:03.712729 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6a5de7fb-b482-4849-a957-e679616aabc0-operator-scripts podName:6a5de7fb-b482-4849-a957-e679616aabc0 nodeName:}" failed. No retries permitted until 2025-12-03 12:49:04.212709376 +0000 UTC m=+5334.800951190 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/6a5de7fb-b482-4849-a957-e679616aabc0-operator-scripts") pod "glanced4a6-account-delete-69llz" (UID: "6a5de7fb-b482-4849-a957-e679616aabc0") : configmap "openstack-scripts" not found Dec 03 12:49:03 crc kubenswrapper[5052]: E1203 12:49:03.726487 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="6b272b57a393df881b0806ef34c9c3a75d64d13721c0eab27f38fa3374aed7ed" cmd=["/usr/local/bin/container-scripts/status_check.sh"] Dec 03 12:49:03 crc kubenswrapper[5052]: I1203 12:49:03.726751 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:49:03 crc kubenswrapper[5052]: E1203 12:49:03.734796 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="6b272b57a393df881b0806ef34c9c3a75d64d13721c0eab27f38fa3374aed7ed" cmd=["/usr/local/bin/container-scripts/status_check.sh"] Dec 03 12:49:03 crc kubenswrapper[5052]: E1203 12:49:03.737418 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="2b5c5249476f9fa6344558a0eabe96f98d744f43f1cd4faf1d66ffa9c49d4941" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:49:03 crc kubenswrapper[5052]: E1203 12:49:03.742087 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="6b272b57a393df881b0806ef34c9c3a75d64d13721c0eab27f38fa3374aed7ed" cmd=["/usr/local/bin/container-scripts/status_check.sh"] Dec 03 12:49:03 crc kubenswrapper[5052]: E1203 12:49:03.742122 5052 prober.go:104] "Probe errored" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" probeType="Readiness" pod="openstack-kuttl-tests/ovn-northd-0" podUID="342cba80-22e1-488a-bbfb-75b67ca536ac" containerName="ovn-northd" Dec 03 12:49:03 crc kubenswrapper[5052]: I1203 12:49:03.744162 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0d36a272-0133-4e61-bb88-219ff124cc91" path="/var/lib/kubelet/pods/0d36a272-0133-4e61-bb88-219ff124cc91/volumes" Dec 03 12:49:03 crc kubenswrapper[5052]: I1203 12:49:03.744814 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2b4ed2db-d186-4bf9-8616-44f1249e7af1" path="/var/lib/kubelet/pods/2b4ed2db-d186-4bf9-8616-44f1249e7af1/volumes" Dec 03 12:49:03 crc kubenswrapper[5052]: I1203 12:49:03.745342 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="53834739-1c4b-4de1-a946-1a26d9036171" path="/var/lib/kubelet/pods/53834739-1c4b-4de1-a946-1a26d9036171/volumes" Dec 03 12:49:03 crc kubenswrapper[5052]: I1203 12:49:03.746113 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="555223ae-4781-4c74-8ae0-053e1153ad9f" path="/var/lib/kubelet/pods/555223ae-4781-4c74-8ae0-053e1153ad9f/volumes" Dec 03 12:49:03 crc kubenswrapper[5052]: I1203 12:49:03.747381 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5f011b7b-96e7-4236-a0fc-b7e292e232fd" path="/var/lib/kubelet/pods/5f011b7b-96e7-4236-a0fc-b7e292e232fd/volumes" Dec 03 12:49:03 crc kubenswrapper[5052]: I1203 12:49:03.748069 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="795fe0cc-73d2-4647-ad2a-648f57e74fc3" path="/var/lib/kubelet/pods/795fe0cc-73d2-4647-ad2a-648f57e74fc3/volumes" Dec 03 12:49:03 crc kubenswrapper[5052]: E1203 12:49:03.748085 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="2b5c5249476f9fa6344558a0eabe96f98d744f43f1cd4faf1d66ffa9c49d4941" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:49:03 crc kubenswrapper[5052]: I1203 12:49:03.748757 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="79f5c230-497d-4183-b174-e1a927ffc003" path="/var/lib/kubelet/pods/79f5c230-497d-4183-b174-e1a927ffc003/volumes" Dec 03 12:49:03 crc kubenswrapper[5052]: I1203 12:49:03.750582 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7f78d8a2-cb31-4277-a6f6-51bcb6d4ed15" path="/var/lib/kubelet/pods/7f78d8a2-cb31-4277-a6f6-51bcb6d4ed15/volumes" Dec 03 12:49:03 crc kubenswrapper[5052]: I1203 12:49:03.751132 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="861301e3-bb06-498b-9225-b72962e175bc" path="/var/lib/kubelet/pods/861301e3-bb06-498b-9225-b72962e175bc/volumes" Dec 03 12:49:03 crc kubenswrapper[5052]: I1203 12:49:03.751672 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="876d06e6-0187-4e3d-9183-ed2083fe5189" path="/var/lib/kubelet/pods/876d06e6-0187-4e3d-9183-ed2083fe5189/volumes" Dec 03 12:49:03 crc kubenswrapper[5052]: I1203 12:49:03.752794 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="984da95f-b2c9-422b-b09d-6b7f80bf56e1" path="/var/lib/kubelet/pods/984da95f-b2c9-422b-b09d-6b7f80bf56e1/volumes" Dec 03 12:49:03 crc kubenswrapper[5052]: I1203 12:49:03.753296 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aa557411-6b0a-4596-a5f5-0e23f8686dc4" path="/var/lib/kubelet/pods/aa557411-6b0a-4596-a5f5-0e23f8686dc4/volumes" Dec 03 12:49:03 crc kubenswrapper[5052]: E1203 12:49:03.753562 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="2b5c5249476f9fa6344558a0eabe96f98d744f43f1cd4faf1d66ffa9c49d4941" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:49:03 crc kubenswrapper[5052]: E1203 12:49:03.753627 5052 prober.go:104] "Probe errored" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" probeType="Readiness" pod="openstack-kuttl-tests/nova-cell0-conductor-0" podUID="6d653682-52e2-4940-af66-633475bfe09a" containerName="nova-cell0-conductor-conductor" Dec 03 12:49:03 crc kubenswrapper[5052]: I1203 12:49:03.753887 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b4e4f28c-dc12-4844-a34a-14241afb73db" path="/var/lib/kubelet/pods/b4e4f28c-dc12-4844-a34a-14241afb73db/volumes" Dec 03 12:49:03 crc kubenswrapper[5052]: I1203 12:49:03.758836 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b99b3896-035c-4491-8093-80f669e31f64" path="/var/lib/kubelet/pods/b99b3896-035c-4491-8093-80f669e31f64/volumes" Dec 03 12:49:03 crc kubenswrapper[5052]: I1203 12:49:03.760451 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dd8df313-3680-4dd1-a57d-ff85b0b2c6f1" path="/var/lib/kubelet/pods/dd8df313-3680-4dd1-a57d-ff85b0b2c6f1/volumes" Dec 03 12:49:03 crc kubenswrapper[5052]: I1203 12:49:03.761137 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ec00af82-d7ae-44ad-9023-d88663706469" path="/var/lib/kubelet/pods/ec00af82-d7ae-44ad-9023-d88663706469/volumes" Dec 03 12:49:03 crc kubenswrapper[5052]: I1203 12:49:03.767248 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/novacell05566-account-delete-2f4dm"] Dec 03 12:49:03 crc kubenswrapper[5052]: I1203 12:49:03.767286 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell0-5566-account-create-update-wh2vb"] Dec 03 12:49:03 crc kubenswrapper[5052]: I1203 12:49:03.767298 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell0-5566-account-create-update-wh2vb"] Dec 03 12:49:03 crc kubenswrapper[5052]: I1203 12:49:03.777801 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/keystoneb51b-account-delete-ds4mz"] Dec 03 12:49:03 crc kubenswrapper[5052]: I1203 12:49:03.787714 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/keystoneb51b-account-delete-ds4mz"] Dec 03 12:49:03 crc kubenswrapper[5052]: I1203 12:49:03.797590 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 12:49:03 crc kubenswrapper[5052]: I1203 12:49:03.803106 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 12:49:03 crc kubenswrapper[5052]: I1203 12:49:03.812852 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/3479f9ad-871f-463e-96ce-50f4069088b3-config-data-generated\") pod \"3479f9ad-871f-463e-96ce-50f4069088b3\" (UID: \"3479f9ad-871f-463e-96ce-50f4069088b3\") " Dec 03 12:49:03 crc kubenswrapper[5052]: I1203 12:49:03.812897 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/e14465ed-868a-49ba-b3ab-1e67196229e8-config-data-custom\") pod \"e14465ed-868a-49ba-b3ab-1e67196229e8\" (UID: \"e14465ed-868a-49ba-b3ab-1e67196229e8\") " Dec 03 12:49:03 crc kubenswrapper[5052]: I1203 12:49:03.812971 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e14465ed-868a-49ba-b3ab-1e67196229e8-combined-ca-bundle\") pod \"e14465ed-868a-49ba-b3ab-1e67196229e8\" (UID: \"e14465ed-868a-49ba-b3ab-1e67196229e8\") " Dec 03 12:49:03 crc kubenswrapper[5052]: I1203 12:49:03.813011 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rhgx2\" (UniqueName: \"kubernetes.io/projected/e14465ed-868a-49ba-b3ab-1e67196229e8-kube-api-access-rhgx2\") pod \"e14465ed-868a-49ba-b3ab-1e67196229e8\" (UID: \"e14465ed-868a-49ba-b3ab-1e67196229e8\") " Dec 03 12:49:03 crc kubenswrapper[5052]: I1203 12:49:03.813058 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/3479f9ad-871f-463e-96ce-50f4069088b3-galera-tls-certs\") pod \"3479f9ad-871f-463e-96ce-50f4069088b3\" (UID: \"3479f9ad-871f-463e-96ce-50f4069088b3\") " Dec 03 12:49:03 crc kubenswrapper[5052]: I1203 12:49:03.813086 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3479f9ad-871f-463e-96ce-50f4069088b3-combined-ca-bundle\") pod \"3479f9ad-871f-463e-96ce-50f4069088b3\" (UID: \"3479f9ad-871f-463e-96ce-50f4069088b3\") " Dec 03 12:49:03 crc kubenswrapper[5052]: I1203 12:49:03.813135 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cbqj6\" (UniqueName: \"kubernetes.io/projected/3479f9ad-871f-463e-96ce-50f4069088b3-kube-api-access-cbqj6\") pod \"3479f9ad-871f-463e-96ce-50f4069088b3\" (UID: \"3479f9ad-871f-463e-96ce-50f4069088b3\") " Dec 03 12:49:03 crc kubenswrapper[5052]: I1203 12:49:03.813166 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/3479f9ad-871f-463e-96ce-50f4069088b3-operator-scripts\") pod \"3479f9ad-871f-463e-96ce-50f4069088b3\" (UID: \"3479f9ad-871f-463e-96ce-50f4069088b3\") " Dec 03 12:49:03 crc kubenswrapper[5052]: I1203 12:49:03.813194 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/3479f9ad-871f-463e-96ce-50f4069088b3-config-data-default\") pod \"3479f9ad-871f-463e-96ce-50f4069088b3\" (UID: \"3479f9ad-871f-463e-96ce-50f4069088b3\") " Dec 03 12:49:03 crc kubenswrapper[5052]: I1203 12:49:03.813222 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/e14465ed-868a-49ba-b3ab-1e67196229e8-config-data\") pod \"e14465ed-868a-49ba-b3ab-1e67196229e8\" (UID: \"e14465ed-868a-49ba-b3ab-1e67196229e8\") " Dec 03 12:49:03 crc kubenswrapper[5052]: I1203 12:49:03.813245 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/e14465ed-868a-49ba-b3ab-1e67196229e8-logs\") pod \"e14465ed-868a-49ba-b3ab-1e67196229e8\" (UID: \"e14465ed-868a-49ba-b3ab-1e67196229e8\") " Dec 03 12:49:03 crc kubenswrapper[5052]: I1203 12:49:03.813283 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/3479f9ad-871f-463e-96ce-50f4069088b3-kolla-config\") pod \"3479f9ad-871f-463e-96ce-50f4069088b3\" (UID: \"3479f9ad-871f-463e-96ce-50f4069088b3\") " Dec 03 12:49:03 crc kubenswrapper[5052]: I1203 12:49:03.813324 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"mysql-db\" (UniqueName: \"kubernetes.io/local-volume/local-storage11-crc\") pod \"3479f9ad-871f-463e-96ce-50f4069088b3\" (UID: \"3479f9ad-871f-463e-96ce-50f4069088b3\") " Dec 03 12:49:03 crc kubenswrapper[5052]: I1203 12:49:03.815018 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3479f9ad-871f-463e-96ce-50f4069088b3-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "3479f9ad-871f-463e-96ce-50f4069088b3" (UID: "3479f9ad-871f-463e-96ce-50f4069088b3"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:49:03 crc kubenswrapper[5052]: I1203 12:49:03.815040 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3479f9ad-871f-463e-96ce-50f4069088b3-config-data-default" (OuterVolumeSpecName: "config-data-default") pod "3479f9ad-871f-463e-96ce-50f4069088b3" (UID: "3479f9ad-871f-463e-96ce-50f4069088b3"). InnerVolumeSpecName "config-data-default". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:49:03 crc kubenswrapper[5052]: I1203 12:49:03.815785 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3479f9ad-871f-463e-96ce-50f4069088b3-config-data-generated" (OuterVolumeSpecName: "config-data-generated") pod "3479f9ad-871f-463e-96ce-50f4069088b3" (UID: "3479f9ad-871f-463e-96ce-50f4069088b3"). InnerVolumeSpecName "config-data-generated". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:49:03 crc kubenswrapper[5052]: I1203 12:49:03.819217 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e14465ed-868a-49ba-b3ab-1e67196229e8-logs" (OuterVolumeSpecName: "logs") pod "e14465ed-868a-49ba-b3ab-1e67196229e8" (UID: "e14465ed-868a-49ba-b3ab-1e67196229e8"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:49:03 crc kubenswrapper[5052]: I1203 12:49:03.825054 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e14465ed-868a-49ba-b3ab-1e67196229e8-config-data-custom" (OuterVolumeSpecName: "config-data-custom") pod "e14465ed-868a-49ba-b3ab-1e67196229e8" (UID: "e14465ed-868a-49ba-b3ab-1e67196229e8"). InnerVolumeSpecName "config-data-custom". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:49:03 crc kubenswrapper[5052]: I1203 12:49:03.827315 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3479f9ad-871f-463e-96ce-50f4069088b3-kolla-config" (OuterVolumeSpecName: "kolla-config") pod "3479f9ad-871f-463e-96ce-50f4069088b3" (UID: "3479f9ad-871f-463e-96ce-50f4069088b3"). InnerVolumeSpecName "kolla-config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:49:03 crc kubenswrapper[5052]: I1203 12:49:03.841458 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e14465ed-868a-49ba-b3ab-1e67196229e8-kube-api-access-rhgx2" (OuterVolumeSpecName: "kube-api-access-rhgx2") pod "e14465ed-868a-49ba-b3ab-1e67196229e8" (UID: "e14465ed-868a-49ba-b3ab-1e67196229e8"). InnerVolumeSpecName "kube-api-access-rhgx2". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:49:03 crc kubenswrapper[5052]: I1203 12:49:03.842117 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3479f9ad-871f-463e-96ce-50f4069088b3-kube-api-access-cbqj6" (OuterVolumeSpecName: "kube-api-access-cbqj6") pod "3479f9ad-871f-463e-96ce-50f4069088b3" (UID: "3479f9ad-871f-463e-96ce-50f4069088b3"). InnerVolumeSpecName "kube-api-access-cbqj6". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:49:03 crc kubenswrapper[5052]: I1203 12:49:03.882107 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage11-crc" (OuterVolumeSpecName: "mysql-db") pod "3479f9ad-871f-463e-96ce-50f4069088b3" (UID: "3479f9ad-871f-463e-96ce-50f4069088b3"). InnerVolumeSpecName "local-storage11-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 12:49:03 crc kubenswrapper[5052]: I1203 12:49:03.915233 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/3479f9ad-871f-463e-96ce-50f4069088b3-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:03 crc kubenswrapper[5052]: I1203 12:49:03.915413 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/3479f9ad-871f-463e-96ce-50f4069088b3-config-data-default\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:03 crc kubenswrapper[5052]: I1203 12:49:03.915489 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/e14465ed-868a-49ba-b3ab-1e67196229e8-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:03 crc kubenswrapper[5052]: I1203 12:49:03.915614 5052 reconciler_common.go:293] "Volume detached for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/3479f9ad-871f-463e-96ce-50f4069088b3-kolla-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:03 crc kubenswrapper[5052]: I1203 12:49:03.915761 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage11-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage11-crc\") on node \"crc\" " Dec 03 12:49:03 crc kubenswrapper[5052]: I1203 12:49:03.915842 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/3479f9ad-871f-463e-96ce-50f4069088b3-config-data-generated\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:03 crc kubenswrapper[5052]: I1203 12:49:03.915923 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/e14465ed-868a-49ba-b3ab-1e67196229e8-config-data-custom\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:03 crc kubenswrapper[5052]: I1203 12:49:03.916010 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-rhgx2\" (UniqueName: \"kubernetes.io/projected/e14465ed-868a-49ba-b3ab-1e67196229e8-kube-api-access-rhgx2\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:03 crc kubenswrapper[5052]: I1203 12:49:03.916090 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-g7ffc\" (UniqueName: \"kubernetes.io/projected/6854341b-41e8-48b0-87f7-aca589364aaf-kube-api-access-g7ffc\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:03 crc kubenswrapper[5052]: I1203 12:49:03.916156 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/6854341b-41e8-48b0-87f7-aca589364aaf-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:03 crc kubenswrapper[5052]: I1203 12:49:03.916221 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-cbqj6\" (UniqueName: \"kubernetes.io/projected/3479f9ad-871f-463e-96ce-50f4069088b3-kube-api-access-cbqj6\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:03 crc kubenswrapper[5052]: I1203 12:49:03.921673 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/memcached-0" Dec 03 12:49:04 crc kubenswrapper[5052]: I1203 12:49:04.017554 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/eef9c304-b598-4a31-9c47-09be0087c25e-config-data\") pod \"eef9c304-b598-4a31-9c47-09be0087c25e\" (UID: \"eef9c304-b598-4a31-9c47-09be0087c25e\") " Dec 03 12:49:04 crc kubenswrapper[5052]: I1203 12:49:04.017685 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"memcached-tls-certs\" (UniqueName: \"kubernetes.io/secret/eef9c304-b598-4a31-9c47-09be0087c25e-memcached-tls-certs\") pod \"eef9c304-b598-4a31-9c47-09be0087c25e\" (UID: \"eef9c304-b598-4a31-9c47-09be0087c25e\") " Dec 03 12:49:04 crc kubenswrapper[5052]: I1203 12:49:04.017730 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/eef9c304-b598-4a31-9c47-09be0087c25e-kolla-config\") pod \"eef9c304-b598-4a31-9c47-09be0087c25e\" (UID: \"eef9c304-b598-4a31-9c47-09be0087c25e\") " Dec 03 12:49:04 crc kubenswrapper[5052]: I1203 12:49:04.017785 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/eef9c304-b598-4a31-9c47-09be0087c25e-combined-ca-bundle\") pod \"eef9c304-b598-4a31-9c47-09be0087c25e\" (UID: \"eef9c304-b598-4a31-9c47-09be0087c25e\") " Dec 03 12:49:04 crc kubenswrapper[5052]: I1203 12:49:04.017885 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m4lg2\" (UniqueName: \"kubernetes.io/projected/eef9c304-b598-4a31-9c47-09be0087c25e-kube-api-access-m4lg2\") pod \"eef9c304-b598-4a31-9c47-09be0087c25e\" (UID: \"eef9c304-b598-4a31-9c47-09be0087c25e\") " Dec 03 12:49:04 crc kubenswrapper[5052]: I1203 12:49:04.018246 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eef9c304-b598-4a31-9c47-09be0087c25e-config-data" (OuterVolumeSpecName: "config-data") pod "eef9c304-b598-4a31-9c47-09be0087c25e" (UID: "eef9c304-b598-4a31-9c47-09be0087c25e"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:49:04 crc kubenswrapper[5052]: E1203 12:49:04.018334 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:49:04 crc kubenswrapper[5052]: I1203 12:49:04.018347 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eef9c304-b598-4a31-9c47-09be0087c25e-kolla-config" (OuterVolumeSpecName: "kolla-config") pod "eef9c304-b598-4a31-9c47-09be0087c25e" (UID: "eef9c304-b598-4a31-9c47-09be0087c25e"). InnerVolumeSpecName "kolla-config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:49:04 crc kubenswrapper[5052]: E1203 12:49:04.018376 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/857b8a8a-d730-4832-a71c-d45a2f3ce406-operator-scripts podName:857b8a8a-d730-4832-a71c-d45a2f3ce406 nodeName:}" failed. No retries permitted until 2025-12-03 12:49:05.018362808 +0000 UTC m=+5335.606604572 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/857b8a8a-d730-4832-a71c-d45a2f3ce406-operator-scripts") pod "barbican02cc-account-delete-gf94b" (UID: "857b8a8a-d730-4832-a71c-d45a2f3ce406") : configmap "openstack-scripts" not found Dec 03 12:49:04 crc kubenswrapper[5052]: E1203 12:49:04.018334 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:49:04 crc kubenswrapper[5052]: E1203 12:49:04.018458 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/456c3fb1-90c0-4d17-863e-414380759f97-operator-scripts podName:456c3fb1-90c0-4d17-863e-414380759f97 nodeName:}" failed. No retries permitted until 2025-12-03 12:49:05.01843499 +0000 UTC m=+5335.606676834 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/456c3fb1-90c0-4d17-863e-414380759f97-operator-scripts") pod "placement361c-account-delete-67ttd" (UID: "456c3fb1-90c0-4d17-863e-414380759f97") : configmap "openstack-scripts" not found Dec 03 12:49:04 crc kubenswrapper[5052]: I1203 12:49:04.035386 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eef9c304-b598-4a31-9c47-09be0087c25e-kube-api-access-m4lg2" (OuterVolumeSpecName: "kube-api-access-m4lg2") pod "eef9c304-b598-4a31-9c47-09be0087c25e" (UID: "eef9c304-b598-4a31-9c47-09be0087c25e"). InnerVolumeSpecName "kube-api-access-m4lg2". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:49:04 crc kubenswrapper[5052]: I1203 12:49:04.097076 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage11-crc" (UniqueName: "kubernetes.io/local-volume/local-storage11-crc") on node "crc" Dec 03 12:49:04 crc kubenswrapper[5052]: I1203 12:49:04.123552 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-m4lg2\" (UniqueName: \"kubernetes.io/projected/eef9c304-b598-4a31-9c47-09be0087c25e-kube-api-access-m4lg2\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:04 crc kubenswrapper[5052]: I1203 12:49:04.123586 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage11-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage11-crc\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:04 crc kubenswrapper[5052]: I1203 12:49:04.123601 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/eef9c304-b598-4a31-9c47-09be0087c25e-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:04 crc kubenswrapper[5052]: I1203 12:49:04.123613 5052 reconciler_common.go:293] "Volume detached for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/eef9c304-b598-4a31-9c47-09be0087c25e-kolla-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:04 crc kubenswrapper[5052]: E1203 12:49:04.123686 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:49:04 crc kubenswrapper[5052]: E1203 12:49:04.123772 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5d11b02e-c14a-4d1a-819a-4c3b66aedd33-operator-scripts podName:5d11b02e-c14a-4d1a-819a-4c3b66aedd33 nodeName:}" failed. No retries permitted until 2025-12-03 12:49:05.123755932 +0000 UTC m=+5335.711997706 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/5d11b02e-c14a-4d1a-819a-4c3b66aedd33-operator-scripts") pod "cinder4345-account-delete-vmdzh" (UID: "5d11b02e-c14a-4d1a-819a-4c3b66aedd33") : configmap "openstack-scripts" not found Dec 03 12:49:04 crc kubenswrapper[5052]: I1203 12:49:04.166765 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e14465ed-868a-49ba-b3ab-1e67196229e8-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "e14465ed-868a-49ba-b3ab-1e67196229e8" (UID: "e14465ed-868a-49ba-b3ab-1e67196229e8"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:49:04 crc kubenswrapper[5052]: I1203 12:49:04.167700 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3479f9ad-871f-463e-96ce-50f4069088b3-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "3479f9ad-871f-463e-96ce-50f4069088b3" (UID: "3479f9ad-871f-463e-96ce-50f4069088b3"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:49:04 crc kubenswrapper[5052]: I1203 12:49:04.169622 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eef9c304-b598-4a31-9c47-09be0087c25e-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "eef9c304-b598-4a31-9c47-09be0087c25e" (UID: "eef9c304-b598-4a31-9c47-09be0087c25e"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:49:04 crc kubenswrapper[5052]: I1203 12:49:04.186290 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eef9c304-b598-4a31-9c47-09be0087c25e-memcached-tls-certs" (OuterVolumeSpecName: "memcached-tls-certs") pod "eef9c304-b598-4a31-9c47-09be0087c25e" (UID: "eef9c304-b598-4a31-9c47-09be0087c25e"). InnerVolumeSpecName "memcached-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:49:04 crc kubenswrapper[5052]: I1203 12:49:04.186787 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e14465ed-868a-49ba-b3ab-1e67196229e8-config-data" (OuterVolumeSpecName: "config-data") pod "e14465ed-868a-49ba-b3ab-1e67196229e8" (UID: "e14465ed-868a-49ba-b3ab-1e67196229e8"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:49:04 crc kubenswrapper[5052]: I1203 12:49:04.197535 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3479f9ad-871f-463e-96ce-50f4069088b3-galera-tls-certs" (OuterVolumeSpecName: "galera-tls-certs") pod "3479f9ad-871f-463e-96ce-50f4069088b3" (UID: "3479f9ad-871f-463e-96ce-50f4069088b3"). InnerVolumeSpecName "galera-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:49:04 crc kubenswrapper[5052]: I1203 12:49:04.225669 5052 reconciler_common.go:293] "Volume detached for volume \"memcached-tls-certs\" (UniqueName: \"kubernetes.io/secret/eef9c304-b598-4a31-9c47-09be0087c25e-memcached-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:04 crc kubenswrapper[5052]: I1203 12:49:04.225698 5052 reconciler_common.go:293] "Volume detached for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/3479f9ad-871f-463e-96ce-50f4069088b3-galera-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:04 crc kubenswrapper[5052]: I1203 12:49:04.225716 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3479f9ad-871f-463e-96ce-50f4069088b3-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:04 crc kubenswrapper[5052]: I1203 12:49:04.225724 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/eef9c304-b598-4a31-9c47-09be0087c25e-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:04 crc kubenswrapper[5052]: I1203 12:49:04.225819 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/e14465ed-868a-49ba-b3ab-1e67196229e8-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:04 crc kubenswrapper[5052]: I1203 12:49:04.225831 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e14465ed-868a-49ba-b3ab-1e67196229e8-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:04 crc kubenswrapper[5052]: E1203 12:49:04.225906 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:49:04 crc kubenswrapper[5052]: E1203 12:49:04.225855 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:49:04 crc kubenswrapper[5052]: E1203 12:49:04.226078 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6a5de7fb-b482-4849-a957-e679616aabc0-operator-scripts podName:6a5de7fb-b482-4849-a957-e679616aabc0 nodeName:}" failed. No retries permitted until 2025-12-03 12:49:05.226017526 +0000 UTC m=+5335.814259380 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/6a5de7fb-b482-4849-a957-e679616aabc0-operator-scripts") pod "glanced4a6-account-delete-69llz" (UID: "6a5de7fb-b482-4849-a957-e679616aabc0") : configmap "openstack-scripts" not found Dec 03 12:49:04 crc kubenswrapper[5052]: E1203 12:49:04.226099 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6ceba96c-7e86-4eb2-afa7-444f0fcc27ce-operator-scripts podName:6ceba96c-7e86-4eb2-afa7-444f0fcc27ce nodeName:}" failed. No retries permitted until 2025-12-03 12:49:05.226090878 +0000 UTC m=+5335.814332752 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/6ceba96c-7e86-4eb2-afa7-444f0fcc27ce-operator-scripts") pod "neutrona25a-account-delete-vbr24" (UID: "6ceba96c-7e86-4eb2-afa7-444f0fcc27ce") : configmap "openstack-scripts" not found Dec 03 12:49:04 crc kubenswrapper[5052]: E1203 12:49:04.285396 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="caf28df7281c196a4c4d326b47a9f4c24851d6defc41fd8fa20f0e5325c7cdac" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:49:04 crc kubenswrapper[5052]: E1203 12:49:04.286652 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="caf28df7281c196a4c4d326b47a9f4c24851d6defc41fd8fa20f0e5325c7cdac" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:49:04 crc kubenswrapper[5052]: E1203 12:49:04.288393 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="caf28df7281c196a4c4d326b47a9f4c24851d6defc41fd8fa20f0e5325c7cdac" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:49:04 crc kubenswrapper[5052]: E1203 12:49:04.288504 5052 prober.go:104] "Probe errored" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" probeType="Readiness" pod="openstack-kuttl-tests/nova-cell1-conductor-0" podUID="8669832d-c47a-49bd-9491-dc7141ac6281" containerName="nova-cell1-conductor-conductor" Dec 03 12:49:04 crc kubenswrapper[5052]: I1203 12:49:04.365125 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/barbican-api-75bdf7fcbb-mlkgz" podUID="b8eb690c-165a-4906-a750-bec0178daf6c" containerName="barbican-api" probeResult="failure" output="Get \"https://10.217.0.118:9311/healthcheck\": dial tcp 10.217.0.118:9311: connect: connection refused" Dec 03 12:49:04 crc kubenswrapper[5052]: I1203 12:49:04.368262 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/barbican-api-75bdf7fcbb-mlkgz" podUID="b8eb690c-165a-4906-a750-bec0178daf6c" containerName="barbican-api-log" probeResult="failure" output="Get \"https://10.217.0.118:9311/healthcheck\": dial tcp 10.217.0.118:9311: connect: connection refused" Dec 03 12:49:04 crc kubenswrapper[5052]: I1203 12:49:04.484832 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:49:04 crc kubenswrapper[5052]: I1203 12:49:04.529162 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-74746c7f4-h6sx8" Dec 03 12:49:04 crc kubenswrapper[5052]: E1203 12:49:04.531888 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:49:04 crc kubenswrapper[5052]: E1203 12:49:04.532000 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2846c397-2869-49f3-aeca-cf4b56bc8834-operator-scripts podName:2846c397-2869-49f3-aeca-cf4b56bc8834 nodeName:}" failed. No retries permitted until 2025-12-03 12:49:08.531970686 +0000 UTC m=+5339.120212500 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/2846c397-2869-49f3-aeca-cf4b56bc8834-operator-scripts") pod "novaapifd1c-account-delete-vhjzv" (UID: "2846c397-2869-49f3-aeca-cf4b56bc8834") : configmap "openstack-scripts" not found Dec 03 12:49:04 crc kubenswrapper[5052]: I1203 12:49:04.572886 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:49:04 crc kubenswrapper[5052]: I1203 12:49:04.584436 5052 generic.go:334] "Generic (PLEG): container finished" podID="77825998-48a8-496e-a523-8e0c371fe532" containerID="15655b574d1d35cf1c932536528e9b95711b3ebc8dd150537384f5080772cd5e" exitCode=0 Dec 03 12:49:04 crc kubenswrapper[5052]: I1203 12:49:04.584500 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-fdd8f875d-xvq79" event={"ID":"77825998-48a8-496e-a523-8e0c371fe532","Type":"ContainerDied","Data":"15655b574d1d35cf1c932536528e9b95711b3ebc8dd150537384f5080772cd5e"} Dec 03 12:49:04 crc kubenswrapper[5052]: I1203 12:49:04.589368 5052 generic.go:334] "Generic (PLEG): container finished" podID="098a46f8-2d49-4fea-a95c-5678601aa164" containerID="3a02ca07996460062957faa0b466285e3ee2c66925c520449ab5df8d346825d8" exitCode=0 Dec 03 12:49:04 crc kubenswrapper[5052]: I1203 12:49:04.589428 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"098a46f8-2d49-4fea-a95c-5678601aa164","Type":"ContainerDied","Data":"3a02ca07996460062957faa0b466285e3ee2c66925c520449ab5df8d346825d8"} Dec 03 12:49:04 crc kubenswrapper[5052]: I1203 12:49:04.595032 5052 generic.go:334] "Generic (PLEG): container finished" podID="efb0e60c-2a71-4641-87b2-0f5532b28678" containerID="fde954c2e50f163494a08fec446c2659dbced2f64031d8d4fb5557a44bc92f99" exitCode=0 Dec 03 12:49:04 crc kubenswrapper[5052]: I1203 12:49:04.595075 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"efb0e60c-2a71-4641-87b2-0f5532b28678","Type":"ContainerDied","Data":"fde954c2e50f163494a08fec446c2659dbced2f64031d8d4fb5557a44bc92f99"} Dec 03 12:49:04 crc kubenswrapper[5052]: I1203 12:49:04.597450 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-74746c7f4-h6sx8" Dec 03 12:49:04 crc kubenswrapper[5052]: I1203 12:49:04.598182 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-74746c7f4-h6sx8" event={"ID":"5201872c-fde1-49d6-9784-6a7f54497990","Type":"ContainerDied","Data":"eafe73590de0755ec90eef2cf9741c8d072e54f8dcd15b0fdadbd3a1a6023c72"} Dec 03 12:49:04 crc kubenswrapper[5052]: I1203 12:49:04.598224 5052 scope.go:117] "RemoveContainer" containerID="d07d82d1f66b2f1adb94be8826e8db31e731aedc69314cb16576303298684ad0" Dec 03 12:49:04 crc kubenswrapper[5052]: I1203 12:49:04.613923 5052 generic.go:334] "Generic (PLEG): container finished" podID="b8eb690c-165a-4906-a750-bec0178daf6c" containerID="275c7f7b179720a472f88be78dd4725e7d963350bb431c4a491cf0a780e6019a" exitCode=0 Dec 03 12:49:04 crc kubenswrapper[5052]: I1203 12:49:04.613998 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-75bdf7fcbb-mlkgz" event={"ID":"b8eb690c-165a-4906-a750-bec0178daf6c","Type":"ContainerDied","Data":"275c7f7b179720a472f88be78dd4725e7d963350bb431c4a491cf0a780e6019a"} Dec 03 12:49:04 crc kubenswrapper[5052]: I1203 12:49:04.617997 5052 generic.go:334] "Generic (PLEG): container finished" podID="bff27172-5e30-474d-a49c-389a35dd3052" containerID="363ded64823f176012eadd8f4efa965cf7bb21ad177ce1c3b4cc513c83f5ebab" exitCode=0 Dec 03 12:49:04 crc kubenswrapper[5052]: I1203 12:49:04.618041 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-5f884bdbfd-2xzz4" event={"ID":"bff27172-5e30-474d-a49c-389a35dd3052","Type":"ContainerDied","Data":"363ded64823f176012eadd8f4efa965cf7bb21ad177ce1c3b4cc513c83f5ebab"} Dec 03 12:49:04 crc kubenswrapper[5052]: I1203 12:49:04.619838 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"b1a74998-b009-4bb6-a0dc-d94ffae8b641","Type":"ContainerDied","Data":"85ce7bf592dfcc3912055f83c1ee322990e24127bd6cf39edab03198beaa6c50"} Dec 03 12:49:04 crc kubenswrapper[5052]: I1203 12:49:04.619908 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:49:04 crc kubenswrapper[5052]: I1203 12:49:04.623331 5052 generic.go:334] "Generic (PLEG): container finished" podID="6c8ad676-e7dd-479c-8d22-f546dc5e50af" containerID="f38a12088ebf63cf61a94810998aa693db18e09b88bf0391d892ee960b99e9ce" exitCode=0 Dec 03 12:49:04 crc kubenswrapper[5052]: I1203 12:49:04.623378 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"6c8ad676-e7dd-479c-8d22-f546dc5e50af","Type":"ContainerDied","Data":"f38a12088ebf63cf61a94810998aa693db18e09b88bf0391d892ee960b99e9ce"} Dec 03 12:49:04 crc kubenswrapper[5052]: I1203 12:49:04.626279 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/memcached-0" event={"ID":"eef9c304-b598-4a31-9c47-09be0087c25e","Type":"ContainerDied","Data":"6423f3d02f84e310e7c409b2c1c1e2b55e3efb427edea3e0e3f82312b7c58d5c"} Dec 03 12:49:04 crc kubenswrapper[5052]: I1203 12:49:04.626295 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/memcached-0" Dec 03 12:49:04 crc kubenswrapper[5052]: I1203 12:49:04.632541 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/5201872c-fde1-49d6-9784-6a7f54497990-scripts\") pod \"5201872c-fde1-49d6-9784-6a7f54497990\" (UID: \"5201872c-fde1-49d6-9784-6a7f54497990\") " Dec 03 12:49:04 crc kubenswrapper[5052]: I1203 12:49:04.633334 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5201872c-fde1-49d6-9784-6a7f54497990-combined-ca-bundle\") pod \"5201872c-fde1-49d6-9784-6a7f54497990\" (UID: \"5201872c-fde1-49d6-9784-6a7f54497990\") " Dec 03 12:49:04 crc kubenswrapper[5052]: I1203 12:49:04.634047 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/b1a74998-b009-4bb6-a0dc-d94ffae8b641-scripts\") pod \"b1a74998-b009-4bb6-a0dc-d94ffae8b641\" (UID: \"b1a74998-b009-4bb6-a0dc-d94ffae8b641\") " Dec 03 12:49:04 crc kubenswrapper[5052]: I1203 12:49:04.634133 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vhbfz\" (UniqueName: \"kubernetes.io/projected/b1a74998-b009-4bb6-a0dc-d94ffae8b641-kube-api-access-vhbfz\") pod \"b1a74998-b009-4bb6-a0dc-d94ffae8b641\" (UID: \"b1a74998-b009-4bb6-a0dc-d94ffae8b641\") " Dec 03 12:49:04 crc kubenswrapper[5052]: I1203 12:49:04.634237 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/5201872c-fde1-49d6-9784-6a7f54497990-public-tls-certs\") pod \"5201872c-fde1-49d6-9784-6a7f54497990\" (UID: \"5201872c-fde1-49d6-9784-6a7f54497990\") " Dec 03 12:49:04 crc kubenswrapper[5052]: I1203 12:49:04.634318 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/5201872c-fde1-49d6-9784-6a7f54497990-logs\") pod \"5201872c-fde1-49d6-9784-6a7f54497990\" (UID: \"5201872c-fde1-49d6-9784-6a7f54497990\") " Dec 03 12:49:04 crc kubenswrapper[5052]: I1203 12:49:04.634490 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6r529\" (UniqueName: \"kubernetes.io/projected/5201872c-fde1-49d6-9784-6a7f54497990-kube-api-access-6r529\") pod \"5201872c-fde1-49d6-9784-6a7f54497990\" (UID: \"5201872c-fde1-49d6-9784-6a7f54497990\") " Dec 03 12:49:04 crc kubenswrapper[5052]: I1203 12:49:04.633118 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:49:04 crc kubenswrapper[5052]: I1203 12:49:04.634729 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/b1a74998-b009-4bb6-a0dc-d94ffae8b641-httpd-run\") pod \"b1a74998-b009-4bb6-a0dc-d94ffae8b641\" (UID: \"b1a74998-b009-4bb6-a0dc-d94ffae8b641\") " Dec 03 12:49:04 crc kubenswrapper[5052]: I1203 12:49:04.634831 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/5201872c-fde1-49d6-9784-6a7f54497990-config-data\") pod \"5201872c-fde1-49d6-9784-6a7f54497990\" (UID: \"5201872c-fde1-49d6-9784-6a7f54497990\") " Dec 03 12:49:04 crc kubenswrapper[5052]: I1203 12:49:04.634919 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/b1a74998-b009-4bb6-a0dc-d94ffae8b641-public-tls-certs\") pod \"b1a74998-b009-4bb6-a0dc-d94ffae8b641\" (UID: \"b1a74998-b009-4bb6-a0dc-d94ffae8b641\") " Dec 03 12:49:04 crc kubenswrapper[5052]: I1203 12:49:04.634988 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/b1a74998-b009-4bb6-a0dc-d94ffae8b641-logs\") pod \"b1a74998-b009-4bb6-a0dc-d94ffae8b641\" (UID: \"b1a74998-b009-4bb6-a0dc-d94ffae8b641\") " Dec 03 12:49:04 crc kubenswrapper[5052]: I1203 12:49:04.635068 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/b1a74998-b009-4bb6-a0dc-d94ffae8b641-config-data\") pod \"b1a74998-b009-4bb6-a0dc-d94ffae8b641\" (UID: \"b1a74998-b009-4bb6-a0dc-d94ffae8b641\") " Dec 03 12:49:04 crc kubenswrapper[5052]: I1203 12:49:04.633110 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"557175a9-81e5-4605-bd67-f1dfa575fb1b","Type":"ContainerDied","Data":"ee4a6851ea72f93c6b9c40f241d2d2bbb83602fe5628f3ddba531a171ed30864"} Dec 03 12:49:04 crc kubenswrapper[5052]: I1203 12:49:04.635265 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"glance\" (UniqueName: \"kubernetes.io/local-volume/local-storage12-crc\") pod \"b1a74998-b009-4bb6-a0dc-d94ffae8b641\" (UID: \"b1a74998-b009-4bb6-a0dc-d94ffae8b641\") " Dec 03 12:49:04 crc kubenswrapper[5052]: I1203 12:49:04.635437 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/5201872c-fde1-49d6-9784-6a7f54497990-internal-tls-certs\") pod \"5201872c-fde1-49d6-9784-6a7f54497990\" (UID: \"5201872c-fde1-49d6-9784-6a7f54497990\") " Dec 03 12:49:04 crc kubenswrapper[5052]: I1203 12:49:04.635569 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b1a74998-b009-4bb6-a0dc-d94ffae8b641-combined-ca-bundle\") pod \"b1a74998-b009-4bb6-a0dc-d94ffae8b641\" (UID: \"b1a74998-b009-4bb6-a0dc-d94ffae8b641\") " Dec 03 12:49:04 crc kubenswrapper[5052]: I1203 12:49:04.635755 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b1a74998-b009-4bb6-a0dc-d94ffae8b641-httpd-run" (OuterVolumeSpecName: "httpd-run") pod "b1a74998-b009-4bb6-a0dc-d94ffae8b641" (UID: "b1a74998-b009-4bb6-a0dc-d94ffae8b641"). InnerVolumeSpecName "httpd-run". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:49:04 crc kubenswrapper[5052]: I1203 12:49:04.636162 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5201872c-fde1-49d6-9784-6a7f54497990-logs" (OuterVolumeSpecName: "logs") pod "5201872c-fde1-49d6-9784-6a7f54497990" (UID: "5201872c-fde1-49d6-9784-6a7f54497990"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:49:04 crc kubenswrapper[5052]: I1203 12:49:04.636955 5052 reconciler_common.go:293] "Volume detached for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/b1a74998-b009-4bb6-a0dc-d94ffae8b641-httpd-run\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:04 crc kubenswrapper[5052]: I1203 12:49:04.638711 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/5201872c-fde1-49d6-9784-6a7f54497990-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:04 crc kubenswrapper[5052]: I1203 12:49:04.637414 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5201872c-fde1-49d6-9784-6a7f54497990-scripts" (OuterVolumeSpecName: "scripts") pod "5201872c-fde1-49d6-9784-6a7f54497990" (UID: "5201872c-fde1-49d6-9784-6a7f54497990"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:49:04 crc kubenswrapper[5052]: E1203 12:49:04.639001 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:49:04 crc kubenswrapper[5052]: E1203 12:49:04.639118 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a2bc501c-ebef-4e68-8ae2-e8c4217cadff-operator-scripts podName:a2bc501c-ebef-4e68-8ae2-e8c4217cadff nodeName:}" failed. No retries permitted until 2025-12-03 12:49:08.63910393 +0000 UTC m=+5339.227345684 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/a2bc501c-ebef-4e68-8ae2-e8c4217cadff-operator-scripts") pod "novacell05566-account-delete-2f4dm" (UID: "a2bc501c-ebef-4e68-8ae2-e8c4217cadff") : configmap "openstack-scripts" not found Dec 03 12:49:04 crc kubenswrapper[5052]: I1203 12:49:04.639762 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5201872c-fde1-49d6-9784-6a7f54497990-kube-api-access-6r529" (OuterVolumeSpecName: "kube-api-access-6r529") pod "5201872c-fde1-49d6-9784-6a7f54497990" (UID: "5201872c-fde1-49d6-9784-6a7f54497990"). InnerVolumeSpecName "kube-api-access-6r529". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:49:04 crc kubenswrapper[5052]: I1203 12:49:04.639858 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b1a74998-b009-4bb6-a0dc-d94ffae8b641-logs" (OuterVolumeSpecName: "logs") pod "b1a74998-b009-4bb6-a0dc-d94ffae8b641" (UID: "b1a74998-b009-4bb6-a0dc-d94ffae8b641"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:49:04 crc kubenswrapper[5052]: I1203 12:49:04.642832 5052 generic.go:334] "Generic (PLEG): container finished" podID="5d11b02e-c14a-4d1a-819a-4c3b66aedd33" containerID="b58ebff57a6b5eca68fc9558830a4aee95150c885bec5d3131628c10c8d77c4e" exitCode=1 Dec 03 12:49:04 crc kubenswrapper[5052]: I1203 12:49:04.643173 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder4345-account-delete-vmdzh" event={"ID":"5d11b02e-c14a-4d1a-819a-4c3b66aedd33","Type":"ContainerDied","Data":"b58ebff57a6b5eca68fc9558830a4aee95150c885bec5d3131628c10c8d77c4e"} Dec 03 12:49:04 crc kubenswrapper[5052]: I1203 12:49:04.648654 5052 generic.go:334] "Generic (PLEG): container finished" podID="2846c397-2869-49f3-aeca-cf4b56bc8834" containerID="747dff2aa2b4006f1eacd1565f1e19dcab8b056e4b9a569dfe7735eef649cbe8" exitCode=1 Dec 03 12:49:04 crc kubenswrapper[5052]: I1203 12:49:04.649994 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/novaapifd1c-account-delete-vhjzv" event={"ID":"2846c397-2869-49f3-aeca-cf4b56bc8834","Type":"ContainerDied","Data":"747dff2aa2b4006f1eacd1565f1e19dcab8b056e4b9a569dfe7735eef649cbe8"} Dec 03 12:49:04 crc kubenswrapper[5052]: I1203 12:49:04.650168 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-worker-58f7cb7fd9-2k5tg" Dec 03 12:49:04 crc kubenswrapper[5052]: I1203 12:49:04.650777 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:49:04 crc kubenswrapper[5052]: I1203 12:49:04.659596 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b1a74998-b009-4bb6-a0dc-d94ffae8b641-scripts" (OuterVolumeSpecName: "scripts") pod "b1a74998-b009-4bb6-a0dc-d94ffae8b641" (UID: "b1a74998-b009-4bb6-a0dc-d94ffae8b641"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:49:04 crc kubenswrapper[5052]: I1203 12:49:04.660376 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b1a74998-b009-4bb6-a0dc-d94ffae8b641-kube-api-access-vhbfz" (OuterVolumeSpecName: "kube-api-access-vhbfz") pod "b1a74998-b009-4bb6-a0dc-d94ffae8b641" (UID: "b1a74998-b009-4bb6-a0dc-d94ffae8b641"). InnerVolumeSpecName "kube-api-access-vhbfz". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:49:04 crc kubenswrapper[5052]: I1203 12:49:04.670030 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage12-crc" (OuterVolumeSpecName: "glance") pod "b1a74998-b009-4bb6-a0dc-d94ffae8b641" (UID: "b1a74998-b009-4bb6-a0dc-d94ffae8b641"). InnerVolumeSpecName "local-storage12-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 12:49:04 crc kubenswrapper[5052]: I1203 12:49:04.700054 5052 scope.go:117] "RemoveContainer" containerID="775232fb24de37720556752c4b1bc2dd4ea9fd1233582be2d16b27194e0a1798" Dec 03 12:49:04 crc kubenswrapper[5052]: I1203 12:49:04.752061 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/557175a9-81e5-4605-bd67-f1dfa575fb1b-public-tls-certs\") pod \"557175a9-81e5-4605-bd67-f1dfa575fb1b\" (UID: \"557175a9-81e5-4605-bd67-f1dfa575fb1b\") " Dec 03 12:49:04 crc kubenswrapper[5052]: I1203 12:49:04.752105 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/557175a9-81e5-4605-bd67-f1dfa575fb1b-etc-machine-id\") pod \"557175a9-81e5-4605-bd67-f1dfa575fb1b\" (UID: \"557175a9-81e5-4605-bd67-f1dfa575fb1b\") " Dec 03 12:49:04 crc kubenswrapper[5052]: I1203 12:49:04.752131 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/557175a9-81e5-4605-bd67-f1dfa575fb1b-scripts\") pod \"557175a9-81e5-4605-bd67-f1dfa575fb1b\" (UID: \"557175a9-81e5-4605-bd67-f1dfa575fb1b\") " Dec 03 12:49:04 crc kubenswrapper[5052]: I1203 12:49:04.752180 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/557175a9-81e5-4605-bd67-f1dfa575fb1b-internal-tls-certs\") pod \"557175a9-81e5-4605-bd67-f1dfa575fb1b\" (UID: \"557175a9-81e5-4605-bd67-f1dfa575fb1b\") " Dec 03 12:49:04 crc kubenswrapper[5052]: I1203 12:49:04.752233 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/557175a9-81e5-4605-bd67-f1dfa575fb1b-logs\") pod \"557175a9-81e5-4605-bd67-f1dfa575fb1b\" (UID: \"557175a9-81e5-4605-bd67-f1dfa575fb1b\") " Dec 03 12:49:04 crc kubenswrapper[5052]: I1203 12:49:04.752302 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/557175a9-81e5-4605-bd67-f1dfa575fb1b-combined-ca-bundle\") pod \"557175a9-81e5-4605-bd67-f1dfa575fb1b\" (UID: \"557175a9-81e5-4605-bd67-f1dfa575fb1b\") " Dec 03 12:49:04 crc kubenswrapper[5052]: I1203 12:49:04.752340 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/557175a9-81e5-4605-bd67-f1dfa575fb1b-config-data-custom\") pod \"557175a9-81e5-4605-bd67-f1dfa575fb1b\" (UID: \"557175a9-81e5-4605-bd67-f1dfa575fb1b\") " Dec 03 12:49:04 crc kubenswrapper[5052]: I1203 12:49:04.752361 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6fgll\" (UniqueName: \"kubernetes.io/projected/557175a9-81e5-4605-bd67-f1dfa575fb1b-kube-api-access-6fgll\") pod \"557175a9-81e5-4605-bd67-f1dfa575fb1b\" (UID: \"557175a9-81e5-4605-bd67-f1dfa575fb1b\") " Dec 03 12:49:04 crc kubenswrapper[5052]: I1203 12:49:04.752436 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/557175a9-81e5-4605-bd67-f1dfa575fb1b-config-data\") pod \"557175a9-81e5-4605-bd67-f1dfa575fb1b\" (UID: \"557175a9-81e5-4605-bd67-f1dfa575fb1b\") " Dec 03 12:49:04 crc kubenswrapper[5052]: I1203 12:49:04.752942 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-6r529\" (UniqueName: \"kubernetes.io/projected/5201872c-fde1-49d6-9784-6a7f54497990-kube-api-access-6r529\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:04 crc kubenswrapper[5052]: I1203 12:49:04.752960 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/b1a74998-b009-4bb6-a0dc-d94ffae8b641-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:04 crc kubenswrapper[5052]: I1203 12:49:04.752979 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage12-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage12-crc\") on node \"crc\" " Dec 03 12:49:04 crc kubenswrapper[5052]: I1203 12:49:04.752991 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/5201872c-fde1-49d6-9784-6a7f54497990-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:04 crc kubenswrapper[5052]: I1203 12:49:04.753001 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/b1a74998-b009-4bb6-a0dc-d94ffae8b641-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:04 crc kubenswrapper[5052]: I1203 12:49:04.753012 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-vhbfz\" (UniqueName: \"kubernetes.io/projected/b1a74998-b009-4bb6-a0dc-d94ffae8b641-kube-api-access-vhbfz\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:04 crc kubenswrapper[5052]: I1203 12:49:04.761754 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/557175a9-81e5-4605-bd67-f1dfa575fb1b-logs" (OuterVolumeSpecName: "logs") pod "557175a9-81e5-4605-bd67-f1dfa575fb1b" (UID: "557175a9-81e5-4605-bd67-f1dfa575fb1b"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:49:04 crc kubenswrapper[5052]: I1203 12:49:04.764927 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/557175a9-81e5-4605-bd67-f1dfa575fb1b-etc-machine-id" (OuterVolumeSpecName: "etc-machine-id") pod "557175a9-81e5-4605-bd67-f1dfa575fb1b" (UID: "557175a9-81e5-4605-bd67-f1dfa575fb1b"). InnerVolumeSpecName "etc-machine-id". PluginName "kubernetes.io/host-path", VolumeGidValue "" Dec 03 12:49:04 crc kubenswrapper[5052]: E1203 12:49:04.764982 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/rabbitmq-cell1-config-data: configmap "rabbitmq-cell1-config-data" not found Dec 03 12:49:04 crc kubenswrapper[5052]: E1203 12:49:04.765024 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/488c3529-4c2b-47c0-9dcf-05a064c34d0d-config-data podName:488c3529-4c2b-47c0-9dcf-05a064c34d0d nodeName:}" failed. No retries permitted until 2025-12-03 12:49:12.765009691 +0000 UTC m=+5343.353251455 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "config-data" (UniqueName: "kubernetes.io/configmap/488c3529-4c2b-47c0-9dcf-05a064c34d0d-config-data") pod "rabbitmq-cell1-server-0" (UID: "488c3529-4c2b-47c0-9dcf-05a064c34d0d") : configmap "rabbitmq-cell1-config-data" not found Dec 03 12:49:04 crc kubenswrapper[5052]: I1203 12:49:04.810095 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/557175a9-81e5-4605-bd67-f1dfa575fb1b-scripts" (OuterVolumeSpecName: "scripts") pod "557175a9-81e5-4605-bd67-f1dfa575fb1b" (UID: "557175a9-81e5-4605-bd67-f1dfa575fb1b"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:49:04 crc kubenswrapper[5052]: I1203 12:49:04.832792 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/557175a9-81e5-4605-bd67-f1dfa575fb1b-config-data-custom" (OuterVolumeSpecName: "config-data-custom") pod "557175a9-81e5-4605-bd67-f1dfa575fb1b" (UID: "557175a9-81e5-4605-bd67-f1dfa575fb1b"). InnerVolumeSpecName "config-data-custom". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:49:04 crc kubenswrapper[5052]: I1203 12:49:04.833014 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/557175a9-81e5-4605-bd67-f1dfa575fb1b-kube-api-access-6fgll" (OuterVolumeSpecName: "kube-api-access-6fgll") pod "557175a9-81e5-4605-bd67-f1dfa575fb1b" (UID: "557175a9-81e5-4605-bd67-f1dfa575fb1b"). InnerVolumeSpecName "kube-api-access-6fgll". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:49:04 crc kubenswrapper[5052]: I1203 12:49:04.858321 5052 reconciler_common.go:293] "Volume detached for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/557175a9-81e5-4605-bd67-f1dfa575fb1b-etc-machine-id\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:04 crc kubenswrapper[5052]: I1203 12:49:04.858354 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/557175a9-81e5-4605-bd67-f1dfa575fb1b-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:04 crc kubenswrapper[5052]: I1203 12:49:04.858367 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/557175a9-81e5-4605-bd67-f1dfa575fb1b-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:04 crc kubenswrapper[5052]: I1203 12:49:04.858379 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/557175a9-81e5-4605-bd67-f1dfa575fb1b-config-data-custom\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:04 crc kubenswrapper[5052]: I1203 12:49:04.858392 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-6fgll\" (UniqueName: \"kubernetes.io/projected/557175a9-81e5-4605-bd67-f1dfa575fb1b-kube-api-access-6fgll\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:04 crc kubenswrapper[5052]: I1203 12:49:04.868075 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5201872c-fde1-49d6-9784-6a7f54497990-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "5201872c-fde1-49d6-9784-6a7f54497990" (UID: "5201872c-fde1-49d6-9784-6a7f54497990"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:49:04 crc kubenswrapper[5052]: I1203 12:49:04.994850 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5201872c-fde1-49d6-9784-6a7f54497990-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:05 crc kubenswrapper[5052]: I1203 12:49:05.013905 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b1a74998-b009-4bb6-a0dc-d94ffae8b641-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "b1a74998-b009-4bb6-a0dc-d94ffae8b641" (UID: "b1a74998-b009-4bb6-a0dc-d94ffae8b641"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:49:05 crc kubenswrapper[5052]: I1203 12:49:05.014364 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5201872c-fde1-49d6-9784-6a7f54497990-config-data" (OuterVolumeSpecName: "config-data") pod "5201872c-fde1-49d6-9784-6a7f54497990" (UID: "5201872c-fde1-49d6-9784-6a7f54497990"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:49:05 crc kubenswrapper[5052]: I1203 12:49:05.031380 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b1a74998-b009-4bb6-a0dc-d94ffae8b641-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "b1a74998-b009-4bb6-a0dc-d94ffae8b641" (UID: "b1a74998-b009-4bb6-a0dc-d94ffae8b641"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:49:05 crc kubenswrapper[5052]: I1203 12:49:05.038543 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b1a74998-b009-4bb6-a0dc-d94ffae8b641-config-data" (OuterVolumeSpecName: "config-data") pod "b1a74998-b009-4bb6-a0dc-d94ffae8b641" (UID: "b1a74998-b009-4bb6-a0dc-d94ffae8b641"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:49:05 crc kubenswrapper[5052]: I1203 12:49:05.041530 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage12-crc" (UniqueName: "kubernetes.io/local-volume/local-storage12-crc") on node "crc" Dec 03 12:49:05 crc kubenswrapper[5052]: I1203 12:49:05.078115 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/557175a9-81e5-4605-bd67-f1dfa575fb1b-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "557175a9-81e5-4605-bd67-f1dfa575fb1b" (UID: "557175a9-81e5-4605-bd67-f1dfa575fb1b"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:49:05 crc kubenswrapper[5052]: E1203 12:49:05.107740 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:49:05 crc kubenswrapper[5052]: E1203 12:49:05.107927 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/456c3fb1-90c0-4d17-863e-414380759f97-operator-scripts podName:456c3fb1-90c0-4d17-863e-414380759f97 nodeName:}" failed. No retries permitted until 2025-12-03 12:49:07.107900777 +0000 UTC m=+5337.696142541 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/456c3fb1-90c0-4d17-863e-414380759f97-operator-scripts") pod "placement361c-account-delete-67ttd" (UID: "456c3fb1-90c0-4d17-863e-414380759f97") : configmap "openstack-scripts" not found Dec 03 12:49:05 crc kubenswrapper[5052]: E1203 12:49:05.108389 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:49:05 crc kubenswrapper[5052]: E1203 12:49:05.108542 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/857b8a8a-d730-4832-a71c-d45a2f3ce406-operator-scripts podName:857b8a8a-d730-4832-a71c-d45a2f3ce406 nodeName:}" failed. No retries permitted until 2025-12-03 12:49:07.108515744 +0000 UTC m=+5337.696757558 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/857b8a8a-d730-4832-a71c-d45a2f3ce406-operator-scripts") pod "barbican02cc-account-delete-gf94b" (UID: "857b8a8a-d730-4832-a71c-d45a2f3ce406") : configmap "openstack-scripts" not found Dec 03 12:49:05 crc kubenswrapper[5052]: I1203 12:49:05.108955 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/5201872c-fde1-49d6-9784-6a7f54497990-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:05 crc kubenswrapper[5052]: I1203 12:49:05.109063 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/b1a74998-b009-4bb6-a0dc-d94ffae8b641-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:05 crc kubenswrapper[5052]: I1203 12:49:05.109219 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/b1a74998-b009-4bb6-a0dc-d94ffae8b641-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:05 crc kubenswrapper[5052]: I1203 12:49:05.109295 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage12-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage12-crc\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:05 crc kubenswrapper[5052]: I1203 12:49:05.109375 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b1a74998-b009-4bb6-a0dc-d94ffae8b641-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:05 crc kubenswrapper[5052]: I1203 12:49:05.109536 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/557175a9-81e5-4605-bd67-f1dfa575fb1b-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:05 crc kubenswrapper[5052]: E1203 12:49:05.109672 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/rabbitmq-config-data: configmap "rabbitmq-config-data" not found Dec 03 12:49:05 crc kubenswrapper[5052]: E1203 12:49:05.109814 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0cb33a40-6f0a-4e89-a159-4edd3af3a2fa-config-data podName:0cb33a40-6f0a-4e89-a159-4edd3af3a2fa nodeName:}" failed. No retries permitted until 2025-12-03 12:49:13.109800981 +0000 UTC m=+5343.698042785 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "config-data" (UniqueName: "kubernetes.io/configmap/0cb33a40-6f0a-4e89-a159-4edd3af3a2fa-config-data") pod "rabbitmq-server-0" (UID: "0cb33a40-6f0a-4e89-a159-4edd3af3a2fa") : configmap "rabbitmq-config-data" not found Dec 03 12:49:05 crc kubenswrapper[5052]: I1203 12:49:05.142876 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/557175a9-81e5-4605-bd67-f1dfa575fb1b-config-data" (OuterVolumeSpecName: "config-data") pod "557175a9-81e5-4605-bd67-f1dfa575fb1b" (UID: "557175a9-81e5-4605-bd67-f1dfa575fb1b"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:49:05 crc kubenswrapper[5052]: I1203 12:49:05.155778 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/557175a9-81e5-4605-bd67-f1dfa575fb1b-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "557175a9-81e5-4605-bd67-f1dfa575fb1b" (UID: "557175a9-81e5-4605-bd67-f1dfa575fb1b"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:49:05 crc kubenswrapper[5052]: I1203 12:49:05.171724 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5201872c-fde1-49d6-9784-6a7f54497990-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "5201872c-fde1-49d6-9784-6a7f54497990" (UID: "5201872c-fde1-49d6-9784-6a7f54497990"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:49:05 crc kubenswrapper[5052]: I1203 12:49:05.171797 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/557175a9-81e5-4605-bd67-f1dfa575fb1b-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "557175a9-81e5-4605-bd67-f1dfa575fb1b" (UID: "557175a9-81e5-4605-bd67-f1dfa575fb1b"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:49:05 crc kubenswrapper[5052]: I1203 12:49:05.173760 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5201872c-fde1-49d6-9784-6a7f54497990-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "5201872c-fde1-49d6-9784-6a7f54497990" (UID: "5201872c-fde1-49d6-9784-6a7f54497990"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:49:05 crc kubenswrapper[5052]: I1203 12:49:05.212492 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/557175a9-81e5-4605-bd67-f1dfa575fb1b-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:05 crc kubenswrapper[5052]: I1203 12:49:05.212550 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/557175a9-81e5-4605-bd67-f1dfa575fb1b-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:05 crc kubenswrapper[5052]: I1203 12:49:05.212564 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/5201872c-fde1-49d6-9784-6a7f54497990-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:05 crc kubenswrapper[5052]: I1203 12:49:05.212574 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/5201872c-fde1-49d6-9784-6a7f54497990-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:05 crc kubenswrapper[5052]: I1203 12:49:05.212587 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/557175a9-81e5-4605-bd67-f1dfa575fb1b-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:05 crc kubenswrapper[5052]: E1203 12:49:05.212743 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:49:05 crc kubenswrapper[5052]: E1203 12:49:05.212831 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5d11b02e-c14a-4d1a-819a-4c3b66aedd33-operator-scripts podName:5d11b02e-c14a-4d1a-819a-4c3b66aedd33 nodeName:}" failed. No retries permitted until 2025-12-03 12:49:07.212809687 +0000 UTC m=+5337.801051511 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/5d11b02e-c14a-4d1a-819a-4c3b66aedd33-operator-scripts") pod "cinder4345-account-delete-vmdzh" (UID: "5d11b02e-c14a-4d1a-819a-4c3b66aedd33") : configmap "openstack-scripts" not found Dec 03 12:49:05 crc kubenswrapper[5052]: E1203 12:49:05.314896 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:49:05 crc kubenswrapper[5052]: E1203 12:49:05.314956 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6a5de7fb-b482-4849-a957-e679616aabc0-operator-scripts podName:6a5de7fb-b482-4849-a957-e679616aabc0 nodeName:}" failed. No retries permitted until 2025-12-03 12:49:07.314942478 +0000 UTC m=+5337.903184242 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/6a5de7fb-b482-4849-a957-e679616aabc0-operator-scripts") pod "glanced4a6-account-delete-69llz" (UID: "6a5de7fb-b482-4849-a957-e679616aabc0") : configmap "openstack-scripts" not found Dec 03 12:49:05 crc kubenswrapper[5052]: E1203 12:49:05.314993 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:49:05 crc kubenswrapper[5052]: E1203 12:49:05.315011 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6ceba96c-7e86-4eb2-afa7-444f0fcc27ce-operator-scripts podName:6ceba96c-7e86-4eb2-afa7-444f0fcc27ce nodeName:}" failed. No retries permitted until 2025-12-03 12:49:07.31500462 +0000 UTC m=+5337.903246384 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/6ceba96c-7e86-4eb2-afa7-444f0fcc27ce-operator-scripts") pod "neutrona25a-account-delete-vbr24" (UID: "6ceba96c-7e86-4eb2-afa7-444f0fcc27ce") : configmap "openstack-scripts" not found Dec 03 12:49:05 crc kubenswrapper[5052]: I1203 12:49:05.662148 5052 generic.go:334] "Generic (PLEG): container finished" podID="6a5de7fb-b482-4849-a957-e679616aabc0" containerID="3174df2d1d1a71d59d7128750feffcb2735c01240b7a706e2b5e41abf792ee9e" exitCode=1 Dec 03 12:49:05 crc kubenswrapper[5052]: I1203 12:49:05.662211 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glanced4a6-account-delete-69llz" event={"ID":"6a5de7fb-b482-4849-a957-e679616aabc0","Type":"ContainerDied","Data":"3174df2d1d1a71d59d7128750feffcb2735c01240b7a706e2b5e41abf792ee9e"} Dec 03 12:49:05 crc kubenswrapper[5052]: I1203 12:49:05.666599 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"6c8ad676-e7dd-479c-8d22-f546dc5e50af","Type":"ContainerDied","Data":"f1927cb0d3f87c444e2fca5cdd8c0bdb808bdc516086e07afbe7cd05db09f571"} Dec 03 12:49:05 crc kubenswrapper[5052]: I1203 12:49:05.666629 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f1927cb0d3f87c444e2fca5cdd8c0bdb808bdc516086e07afbe7cd05db09f571" Dec 03 12:49:05 crc kubenswrapper[5052]: I1203 12:49:05.669846 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"098a46f8-2d49-4fea-a95c-5678601aa164","Type":"ContainerDied","Data":"b5f14849b9c0500c19252fb36f88804a02027b9b135c74780ffb794c962f7301"} Dec 03 12:49:05 crc kubenswrapper[5052]: I1203 12:49:05.669876 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b5f14849b9c0500c19252fb36f88804a02027b9b135c74780ffb794c962f7301" Dec 03 12:49:05 crc kubenswrapper[5052]: I1203 12:49:05.671949 5052 generic.go:334] "Generic (PLEG): container finished" podID="6d653682-52e2-4940-af66-633475bfe09a" containerID="2b5c5249476f9fa6344558a0eabe96f98d744f43f1cd4faf1d66ffa9c49d4941" exitCode=0 Dec 03 12:49:05 crc kubenswrapper[5052]: I1203 12:49:05.671980 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-conductor-0" event={"ID":"6d653682-52e2-4940-af66-633475bfe09a","Type":"ContainerDied","Data":"2b5c5249476f9fa6344558a0eabe96f98d744f43f1cd4faf1d66ffa9c49d4941"} Dec 03 12:49:05 crc kubenswrapper[5052]: I1203 12:49:05.675473 5052 generic.go:334] "Generic (PLEG): container finished" podID="a2bc501c-ebef-4e68-8ae2-e8c4217cadff" containerID="032892e966b8241f925ecec443072f0ec039107983eec9fb7785055fdd438546" exitCode=1 Dec 03 12:49:05 crc kubenswrapper[5052]: I1203 12:49:05.684650 5052 generic.go:334] "Generic (PLEG): container finished" podID="3c8e4e3f-f6c1-4362-b64c-b10a973723dd" containerID="86f66b1381c35478a52f56b7c8ce2c98cdc37bae9cd163bbea7b6f45c6c705b9" exitCode=0 Dec 03 12:49:05 crc kubenswrapper[5052]: I1203 12:49:05.688857 5052 generic.go:334] "Generic (PLEG): container finished" podID="456c3fb1-90c0-4d17-863e-414380759f97" containerID="e561d2853dbef5377ec18e773746c93147e4ebef4cf2a400d67d09af44a67531" exitCode=1 Dec 03 12:49:05 crc kubenswrapper[5052]: I1203 12:49:05.691064 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6854341b-41e8-48b0-87f7-aca589364aaf" path="/var/lib/kubelet/pods/6854341b-41e8-48b0-87f7-aca589364aaf/volumes" Dec 03 12:49:05 crc kubenswrapper[5052]: I1203 12:49:05.691500 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d857bbf5-cd76-4d28-863c-1507de5c6cfd" path="/var/lib/kubelet/pods/d857bbf5-cd76-4d28-863c-1507de5c6cfd/volumes" Dec 03 12:49:05 crc kubenswrapper[5052]: I1203 12:49:05.692159 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="edae7848-51ff-40d6-83f2-ffff085f3f3e" path="/var/lib/kubelet/pods/edae7848-51ff-40d6-83f2-ffff085f3f3e/volumes" Dec 03 12:49:05 crc kubenswrapper[5052]: I1203 12:49:05.702664 5052 generic.go:334] "Generic (PLEG): container finished" podID="6ceba96c-7e86-4eb2-afa7-444f0fcc27ce" containerID="4a37bb95201a9ab5a64daccd9272263816dd9e817b0668c7be7e63d522a8fabe" exitCode=1 Dec 03 12:49:05 crc kubenswrapper[5052]: I1203 12:49:05.708693 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/novacell05566-account-delete-2f4dm" event={"ID":"a2bc501c-ebef-4e68-8ae2-e8c4217cadff","Type":"ContainerDied","Data":"032892e966b8241f925ecec443072f0ec039107983eec9fb7785055fdd438546"} Dec 03 12:49:05 crc kubenswrapper[5052]: I1203 12:49:05.708747 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-galera-0" event={"ID":"3c8e4e3f-f6c1-4362-b64c-b10a973723dd","Type":"ContainerDied","Data":"86f66b1381c35478a52f56b7c8ce2c98cdc37bae9cd163bbea7b6f45c6c705b9"} Dec 03 12:49:05 crc kubenswrapper[5052]: I1203 12:49:05.708765 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement361c-account-delete-67ttd" event={"ID":"456c3fb1-90c0-4d17-863e-414380759f97","Type":"ContainerDied","Data":"e561d2853dbef5377ec18e773746c93147e4ebef4cf2a400d67d09af44a67531"} Dec 03 12:49:05 crc kubenswrapper[5052]: I1203 12:49:05.708780 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutrona25a-account-delete-vbr24" event={"ID":"6ceba96c-7e86-4eb2-afa7-444f0fcc27ce","Type":"ContainerDied","Data":"4a37bb95201a9ab5a64daccd9272263816dd9e817b0668c7be7e63d522a8fabe"} Dec 03 12:49:05 crc kubenswrapper[5052]: I1203 12:49:05.708798 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"efb0e60c-2a71-4641-87b2-0f5532b28678","Type":"ContainerDied","Data":"001d3190cef4c1889c7c8aab967ae547a13994c1a2d79e762912f956f402dd93"} Dec 03 12:49:05 crc kubenswrapper[5052]: I1203 12:49:05.708813 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="001d3190cef4c1889c7c8aab967ae547a13994c1a2d79e762912f956f402dd93" Dec 03 12:49:05 crc kubenswrapper[5052]: I1203 12:49:05.708824 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-5f884bdbfd-2xzz4" event={"ID":"bff27172-5e30-474d-a49c-389a35dd3052","Type":"ContainerDied","Data":"01bee3660667320f46e63fda825172057bae6fd00afdd0ff3d3609b7f3093174"} Dec 03 12:49:05 crc kubenswrapper[5052]: I1203 12:49:05.708836 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="01bee3660667320f46e63fda825172057bae6fd00afdd0ff3d3609b7f3093174" Dec 03 12:49:05 crc kubenswrapper[5052]: I1203 12:49:05.723674 5052 generic.go:334] "Generic (PLEG): container finished" podID="a69bfb18-1267-4383-b51a-5e0eba70d1bc" containerID="b1a655b5c3d0ddb68e291790261b40d699c5471d8a606da7a88e93dbced9a7f8" exitCode=0 Dec 03 12:49:05 crc kubenswrapper[5052]: I1203 12:49:05.723778 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"a69bfb18-1267-4383-b51a-5e0eba70d1bc","Type":"ContainerDied","Data":"b1a655b5c3d0ddb68e291790261b40d699c5471d8a606da7a88e93dbced9a7f8"} Dec 03 12:49:05 crc kubenswrapper[5052]: I1203 12:49:05.723812 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"a69bfb18-1267-4383-b51a-5e0eba70d1bc","Type":"ContainerDied","Data":"41ead924f537c723d851acf33ed51e22a3dc127f908a763b554398d819505739"} Dec 03 12:49:05 crc kubenswrapper[5052]: I1203 12:49:05.723824 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="41ead924f537c723d851acf33ed51e22a3dc127f908a763b554398d819505739" Dec 03 12:49:05 crc kubenswrapper[5052]: I1203 12:49:05.728948 5052 generic.go:334] "Generic (PLEG): container finished" podID="857b8a8a-d730-4832-a71c-d45a2f3ce406" containerID="996f7904199a66ff3248f13d4d3a39fb7215e6c68ba1d6d6f445aebc18499890" exitCode=1 Dec 03 12:49:05 crc kubenswrapper[5052]: I1203 12:49:05.729026 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican02cc-account-delete-gf94b" event={"ID":"857b8a8a-d730-4832-a71c-d45a2f3ce406","Type":"ContainerDied","Data":"996f7904199a66ff3248f13d4d3a39fb7215e6c68ba1d6d6f445aebc18499890"} Dec 03 12:49:05 crc kubenswrapper[5052]: I1203 12:49:05.732420 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-75bdf7fcbb-mlkgz" event={"ID":"b8eb690c-165a-4906-a750-bec0178daf6c","Type":"ContainerDied","Data":"23a7cc9313c35930de4dadf709c0ffaca2b9ae4bdea1ed8ac27170c5acee546f"} Dec 03 12:49:05 crc kubenswrapper[5052]: I1203 12:49:05.732458 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="23a7cc9313c35930de4dadf709c0ffaca2b9ae4bdea1ed8ac27170c5acee546f" Dec 03 12:49:05 crc kubenswrapper[5052]: I1203 12:49:05.742444 5052 generic.go:334] "Generic (PLEG): container finished" podID="2846c397-2869-49f3-aeca-cf4b56bc8834" containerID="39fed73e7301681d3f192722739dd4c2c192c9bb0beb359054ef92a552914767" exitCode=1 Dec 03 12:49:05 crc kubenswrapper[5052]: I1203 12:49:05.742647 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/novaapifd1c-account-delete-vhjzv" event={"ID":"2846c397-2869-49f3-aeca-cf4b56bc8834","Type":"ContainerDied","Data":"39fed73e7301681d3f192722739dd4c2c192c9bb0beb359054ef92a552914767"} Dec 03 12:49:05 crc kubenswrapper[5052]: I1203 12:49:05.748408 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-fdd8f875d-xvq79" event={"ID":"77825998-48a8-496e-a523-8e0c371fe532","Type":"ContainerDied","Data":"6185634e3cf243590ca0c31348d7b9fa332bb5339856c6af05dece7a198d886f"} Dec 03 12:49:05 crc kubenswrapper[5052]: I1203 12:49:05.748443 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6185634e3cf243590ca0c31348d7b9fa332bb5339856c6af05dece7a198d886f" Dec 03 12:49:05 crc kubenswrapper[5052]: I1203 12:49:05.752837 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_ovn-northd-0_342cba80-22e1-488a-bbfb-75b67ca536ac/ovn-northd/0.log" Dec 03 12:49:05 crc kubenswrapper[5052]: I1203 12:49:05.752879 5052 generic.go:334] "Generic (PLEG): container finished" podID="342cba80-22e1-488a-bbfb-75b67ca536ac" containerID="6b272b57a393df881b0806ef34c9c3a75d64d13721c0eab27f38fa3374aed7ed" exitCode=139 Dec 03 12:49:05 crc kubenswrapper[5052]: I1203 12:49:05.753165 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovn-northd-0" event={"ID":"342cba80-22e1-488a-bbfb-75b67ca536ac","Type":"ContainerDied","Data":"6b272b57a393df881b0806ef34c9c3a75d64d13721c0eab27f38fa3374aed7ed"} Dec 03 12:49:05 crc kubenswrapper[5052]: I1203 12:49:05.758943 5052 generic.go:334] "Generic (PLEG): container finished" podID="81709983-1fd0-4446-bbbf-fb94653bb3d2" containerID="6964abaffe4177de7695a956baa51adeedd8fc7ae4b4885ca8e38046eae0369a" exitCode=0 Dec 03 12:49:05 crc kubenswrapper[5052]: I1203 12:49:05.759010 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"81709983-1fd0-4446-bbbf-fb94653bb3d2","Type":"ContainerDied","Data":"6964abaffe4177de7695a956baa51adeedd8fc7ae4b4885ca8e38046eae0369a"} Dec 03 12:49:05 crc kubenswrapper[5052]: I1203 12:49:05.761178 5052 generic.go:334] "Generic (PLEG): container finished" podID="5d11b02e-c14a-4d1a-819a-4c3b66aedd33" containerID="573a090ed2c4fc25a35dfd0460dc811780dbcc0a3ea284429ddf8664717e53ef" exitCode=1 Dec 03 12:49:05 crc kubenswrapper[5052]: I1203 12:49:05.761200 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder4345-account-delete-vmdzh" event={"ID":"5d11b02e-c14a-4d1a-819a-4c3b66aedd33","Type":"ContainerDied","Data":"573a090ed2c4fc25a35dfd0460dc811780dbcc0a3ea284429ddf8664717e53ef"} Dec 03 12:49:05 crc kubenswrapper[5052]: I1203 12:49:05.815283 5052 scope.go:117] "RemoveContainer" containerID="f28ffd80db46452a6cf33fcc0825d49c5f14be470338b023f15a46aa3b004932" Dec 03 12:49:05 crc kubenswrapper[5052]: I1203 12:49:05.832109 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:49:05 crc kubenswrapper[5052]: I1203 12:49:05.850574 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-api-75bdf7fcbb-mlkgz" Dec 03 12:49:05 crc kubenswrapper[5052]: I1203 12:49:05.884223 5052 scope.go:117] "RemoveContainer" containerID="9a055bdf916ddbec20efe694430ce069882c0ee2f3b4626ae5ce60fc3fd5c826" Dec 03 12:49:05 crc kubenswrapper[5052]: I1203 12:49:05.926386 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n55nw\" (UniqueName: \"kubernetes.io/projected/6c8ad676-e7dd-479c-8d22-f546dc5e50af-kube-api-access-n55nw\") pod \"6c8ad676-e7dd-479c-8d22-f546dc5e50af\" (UID: \"6c8ad676-e7dd-479c-8d22-f546dc5e50af\") " Dec 03 12:49:05 crc kubenswrapper[5052]: I1203 12:49:05.927395 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6c8ad676-e7dd-479c-8d22-f546dc5e50af-config-data\") pod \"6c8ad676-e7dd-479c-8d22-f546dc5e50af\" (UID: \"6c8ad676-e7dd-479c-8d22-f546dc5e50af\") " Dec 03 12:49:05 crc kubenswrapper[5052]: I1203 12:49:05.927431 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/b8eb690c-165a-4906-a750-bec0178daf6c-internal-tls-certs\") pod \"b8eb690c-165a-4906-a750-bec0178daf6c\" (UID: \"b8eb690c-165a-4906-a750-bec0178daf6c\") " Dec 03 12:49:05 crc kubenswrapper[5052]: I1203 12:49:05.927567 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/6c8ad676-e7dd-479c-8d22-f546dc5e50af-logs\") pod \"6c8ad676-e7dd-479c-8d22-f546dc5e50af\" (UID: \"6c8ad676-e7dd-479c-8d22-f546dc5e50af\") " Dec 03 12:49:05 crc kubenswrapper[5052]: I1203 12:49:05.928081 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6c8ad676-e7dd-479c-8d22-f546dc5e50af-logs" (OuterVolumeSpecName: "logs") pod "6c8ad676-e7dd-479c-8d22-f546dc5e50af" (UID: "6c8ad676-e7dd-479c-8d22-f546dc5e50af"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:49:05 crc kubenswrapper[5052]: I1203 12:49:05.928205 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/6c8ad676-e7dd-479c-8d22-f546dc5e50af-public-tls-certs\") pod \"6c8ad676-e7dd-479c-8d22-f546dc5e50af\" (UID: \"6c8ad676-e7dd-479c-8d22-f546dc5e50af\") " Dec 03 12:49:05 crc kubenswrapper[5052]: I1203 12:49:05.928236 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6c8ad676-e7dd-479c-8d22-f546dc5e50af-combined-ca-bundle\") pod \"6c8ad676-e7dd-479c-8d22-f546dc5e50af\" (UID: \"6c8ad676-e7dd-479c-8d22-f546dc5e50af\") " Dec 03 12:49:05 crc kubenswrapper[5052]: I1203 12:49:05.928895 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/b8eb690c-165a-4906-a750-bec0178daf6c-public-tls-certs\") pod \"b8eb690c-165a-4906-a750-bec0178daf6c\" (UID: \"b8eb690c-165a-4906-a750-bec0178daf6c\") " Dec 03 12:49:05 crc kubenswrapper[5052]: I1203 12:49:05.928948 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-znpn4\" (UniqueName: \"kubernetes.io/projected/b8eb690c-165a-4906-a750-bec0178daf6c-kube-api-access-znpn4\") pod \"b8eb690c-165a-4906-a750-bec0178daf6c\" (UID: \"b8eb690c-165a-4906-a750-bec0178daf6c\") " Dec 03 12:49:05 crc kubenswrapper[5052]: I1203 12:49:05.929382 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/b8eb690c-165a-4906-a750-bec0178daf6c-config-data\") pod \"b8eb690c-165a-4906-a750-bec0178daf6c\" (UID: \"b8eb690c-165a-4906-a750-bec0178daf6c\") " Dec 03 12:49:05 crc kubenswrapper[5052]: I1203 12:49:05.929418 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/b8eb690c-165a-4906-a750-bec0178daf6c-logs\") pod \"b8eb690c-165a-4906-a750-bec0178daf6c\" (UID: \"b8eb690c-165a-4906-a750-bec0178daf6c\") " Dec 03 12:49:05 crc kubenswrapper[5052]: I1203 12:49:05.929445 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/b8eb690c-165a-4906-a750-bec0178daf6c-config-data-custom\") pod \"b8eb690c-165a-4906-a750-bec0178daf6c\" (UID: \"b8eb690c-165a-4906-a750-bec0178daf6c\") " Dec 03 12:49:05 crc kubenswrapper[5052]: I1203 12:49:05.929471 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b8eb690c-165a-4906-a750-bec0178daf6c-combined-ca-bundle\") pod \"b8eb690c-165a-4906-a750-bec0178daf6c\" (UID: \"b8eb690c-165a-4906-a750-bec0178daf6c\") " Dec 03 12:49:05 crc kubenswrapper[5052]: I1203 12:49:05.929543 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/6c8ad676-e7dd-479c-8d22-f546dc5e50af-internal-tls-certs\") pod \"6c8ad676-e7dd-479c-8d22-f546dc5e50af\" (UID: \"6c8ad676-e7dd-479c-8d22-f546dc5e50af\") " Dec 03 12:49:05 crc kubenswrapper[5052]: I1203 12:49:05.929716 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:49:05 crc kubenswrapper[5052]: I1203 12:49:05.931095 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b8eb690c-165a-4906-a750-bec0178daf6c-logs" (OuterVolumeSpecName: "logs") pod "b8eb690c-165a-4906-a750-bec0178daf6c" (UID: "b8eb690c-165a-4906-a750-bec0178daf6c"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:49:05 crc kubenswrapper[5052]: I1203 12:49:05.932442 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/b8eb690c-165a-4906-a750-bec0178daf6c-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:05 crc kubenswrapper[5052]: I1203 12:49:05.932466 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/6c8ad676-e7dd-479c-8d22-f546dc5e50af-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:05 crc kubenswrapper[5052]: I1203 12:49:05.938375 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6c8ad676-e7dd-479c-8d22-f546dc5e50af-kube-api-access-n55nw" (OuterVolumeSpecName: "kube-api-access-n55nw") pod "6c8ad676-e7dd-479c-8d22-f546dc5e50af" (UID: "6c8ad676-e7dd-479c-8d22-f546dc5e50af"). InnerVolumeSpecName "kube-api-access-n55nw". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:49:05 crc kubenswrapper[5052]: I1203 12:49:05.938684 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b8eb690c-165a-4906-a750-bec0178daf6c-config-data-custom" (OuterVolumeSpecName: "config-data-custom") pod "b8eb690c-165a-4906-a750-bec0178daf6c" (UID: "b8eb690c-165a-4906-a750-bec0178daf6c"). InnerVolumeSpecName "config-data-custom". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:49:05 crc kubenswrapper[5052]: I1203 12:49:05.945536 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b8eb690c-165a-4906-a750-bec0178daf6c-kube-api-access-znpn4" (OuterVolumeSpecName: "kube-api-access-znpn4") pod "b8eb690c-165a-4906-a750-bec0178daf6c" (UID: "b8eb690c-165a-4906-a750-bec0178daf6c"). InnerVolumeSpecName "kube-api-access-znpn4". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:49:05 crc kubenswrapper[5052]: I1203 12:49:05.958194 5052 scope.go:117] "RemoveContainer" containerID="3cec53cfcf32bebda98bd1f984b0bb1ae1c9eaa0f2a096f12724a271e1103ca8" Dec 03 12:49:05 crc kubenswrapper[5052]: I1203 12:49:05.971720 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6c8ad676-e7dd-479c-8d22-f546dc5e50af-config-data" (OuterVolumeSpecName: "config-data") pod "6c8ad676-e7dd-479c-8d22-f546dc5e50af" (UID: "6c8ad676-e7dd-479c-8d22-f546dc5e50af"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:49:05 crc kubenswrapper[5052]: I1203 12:49:05.979813 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:49:05 crc kubenswrapper[5052]: I1203 12:49:05.998580 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-worker-58f7cb7fd9-2k5tg"] Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.019705 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/barbican-worker-58f7cb7fd9-2k5tg"] Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.035899 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/efb0e60c-2a71-4641-87b2-0f5532b28678-combined-ca-bundle\") pod \"efb0e60c-2a71-4641-87b2-0f5532b28678\" (UID: \"efb0e60c-2a71-4641-87b2-0f5532b28678\") " Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.035966 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/098a46f8-2d49-4fea-a95c-5678601aa164-logs\") pod \"098a46f8-2d49-4fea-a95c-5678601aa164\" (UID: \"098a46f8-2d49-4fea-a95c-5678601aa164\") " Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.035990 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/efb0e60c-2a71-4641-87b2-0f5532b28678-logs\") pod \"efb0e60c-2a71-4641-87b2-0f5532b28678\" (UID: \"efb0e60c-2a71-4641-87b2-0f5532b28678\") " Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.036015 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jlhbj\" (UniqueName: \"kubernetes.io/projected/098a46f8-2d49-4fea-a95c-5678601aa164-kube-api-access-jlhbj\") pod \"098a46f8-2d49-4fea-a95c-5678601aa164\" (UID: \"098a46f8-2d49-4fea-a95c-5678601aa164\") " Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.036051 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zg457\" (UniqueName: \"kubernetes.io/projected/efb0e60c-2a71-4641-87b2-0f5532b28678-kube-api-access-zg457\") pod \"efb0e60c-2a71-4641-87b2-0f5532b28678\" (UID: \"efb0e60c-2a71-4641-87b2-0f5532b28678\") " Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.036138 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/098a46f8-2d49-4fea-a95c-5678601aa164-config-data\") pod \"098a46f8-2d49-4fea-a95c-5678601aa164\" (UID: \"098a46f8-2d49-4fea-a95c-5678601aa164\") " Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.036181 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/efb0e60c-2a71-4641-87b2-0f5532b28678-config-data\") pod \"efb0e60c-2a71-4641-87b2-0f5532b28678\" (UID: \"efb0e60c-2a71-4641-87b2-0f5532b28678\") " Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.036247 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/098a46f8-2d49-4fea-a95c-5678601aa164-internal-tls-certs\") pod \"098a46f8-2d49-4fea-a95c-5678601aa164\" (UID: \"098a46f8-2d49-4fea-a95c-5678601aa164\") " Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.036296 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/098a46f8-2d49-4fea-a95c-5678601aa164-combined-ca-bundle\") pod \"098a46f8-2d49-4fea-a95c-5678601aa164\" (UID: \"098a46f8-2d49-4fea-a95c-5678601aa164\") " Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.036318 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/098a46f8-2d49-4fea-a95c-5678601aa164-httpd-run\") pod \"098a46f8-2d49-4fea-a95c-5678601aa164\" (UID: \"098a46f8-2d49-4fea-a95c-5678601aa164\") " Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.036372 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/098a46f8-2d49-4fea-a95c-5678601aa164-scripts\") pod \"098a46f8-2d49-4fea-a95c-5678601aa164\" (UID: \"098a46f8-2d49-4fea-a95c-5678601aa164\") " Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.036407 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"glance\" (UniqueName: \"kubernetes.io/local-volume/local-storage18-crc\") pod \"098a46f8-2d49-4fea-a95c-5678601aa164\" (UID: \"098a46f8-2d49-4fea-a95c-5678601aa164\") " Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.036513 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/efb0e60c-2a71-4641-87b2-0f5532b28678-nova-metadata-tls-certs\") pod \"efb0e60c-2a71-4641-87b2-0f5532b28678\" (UID: \"efb0e60c-2a71-4641-87b2-0f5532b28678\") " Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.048093 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/efb0e60c-2a71-4641-87b2-0f5532b28678-logs" (OuterVolumeSpecName: "logs") pod "efb0e60c-2a71-4641-87b2-0f5532b28678" (UID: "efb0e60c-2a71-4641-87b2-0f5532b28678"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.052327 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/098a46f8-2d49-4fea-a95c-5678601aa164-logs" (OuterVolumeSpecName: "logs") pod "098a46f8-2d49-4fea-a95c-5678601aa164" (UID: "098a46f8-2d49-4fea-a95c-5678601aa164"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.052669 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/098a46f8-2d49-4fea-a95c-5678601aa164-httpd-run" (OuterVolumeSpecName: "httpd-run") pod "098a46f8-2d49-4fea-a95c-5678601aa164" (UID: "098a46f8-2d49-4fea-a95c-5678601aa164"). InnerVolumeSpecName "httpd-run". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.052702 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/memcached-0"] Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.056636 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-n55nw\" (UniqueName: \"kubernetes.io/projected/6c8ad676-e7dd-479c-8d22-f546dc5e50af-kube-api-access-n55nw\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.056659 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6c8ad676-e7dd-479c-8d22-f546dc5e50af-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.056669 5052 reconciler_common.go:293] "Volume detached for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/098a46f8-2d49-4fea-a95c-5678601aa164-httpd-run\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.056708 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-znpn4\" (UniqueName: \"kubernetes.io/projected/b8eb690c-165a-4906-a750-bec0178daf6c-kube-api-access-znpn4\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.056719 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/098a46f8-2d49-4fea-a95c-5678601aa164-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.056730 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/efb0e60c-2a71-4641-87b2-0f5532b28678-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.056741 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/b8eb690c-165a-4906-a750-bec0178daf6c-config-data-custom\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.056899 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b8eb690c-165a-4906-a750-bec0178daf6c-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "b8eb690c-165a-4906-a750-bec0178daf6c" (UID: "b8eb690c-165a-4906-a750-bec0178daf6c"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.063839 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/efb0e60c-2a71-4641-87b2-0f5532b28678-kube-api-access-zg457" (OuterVolumeSpecName: "kube-api-access-zg457") pod "efb0e60c-2a71-4641-87b2-0f5532b28678" (UID: "efb0e60c-2a71-4641-87b2-0f5532b28678"). InnerVolumeSpecName "kube-api-access-zg457". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.064153 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/098a46f8-2d49-4fea-a95c-5678601aa164-scripts" (OuterVolumeSpecName: "scripts") pod "098a46f8-2d49-4fea-a95c-5678601aa164" (UID: "098a46f8-2d49-4fea-a95c-5678601aa164"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.064208 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/098a46f8-2d49-4fea-a95c-5678601aa164-kube-api-access-jlhbj" (OuterVolumeSpecName: "kube-api-access-jlhbj") pod "098a46f8-2d49-4fea-a95c-5678601aa164" (UID: "098a46f8-2d49-4fea-a95c-5678601aa164"). InnerVolumeSpecName "kube-api-access-jlhbj". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.064291 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/memcached-0"] Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.067762 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage18-crc" (OuterVolumeSpecName: "glance") pod "098a46f8-2d49-4fea-a95c-5678601aa164" (UID: "098a46f8-2d49-4fea-a95c-5678601aa164"). InnerVolumeSpecName "local-storage18-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.074858 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/openstack-cell1-galera-0"] Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.090471 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/openstack-cell1-galera-0"] Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.102028 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b8eb690c-165a-4906-a750-bec0178daf6c-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "b8eb690c-165a-4906-a750-bec0178daf6c" (UID: "b8eb690c-165a-4906-a750-bec0178daf6c"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.113882 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6c8ad676-e7dd-479c-8d22-f546dc5e50af-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "6c8ad676-e7dd-479c-8d22-f546dc5e50af" (UID: "6c8ad676-e7dd-479c-8d22-f546dc5e50af"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.117840 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6c8ad676-e7dd-479c-8d22-f546dc5e50af-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "6c8ad676-e7dd-479c-8d22-f546dc5e50af" (UID: "6c8ad676-e7dd-479c-8d22-f546dc5e50af"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.135206 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6c8ad676-e7dd-479c-8d22-f546dc5e50af-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "6c8ad676-e7dd-479c-8d22-f546dc5e50af" (UID: "6c8ad676-e7dd-479c-8d22-f546dc5e50af"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.138710 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b8eb690c-165a-4906-a750-bec0178daf6c-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "b8eb690c-165a-4906-a750-bec0178daf6c" (UID: "b8eb690c-165a-4906-a750-bec0178daf6c"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.146863 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b8eb690c-165a-4906-a750-bec0178daf6c-config-data" (OuterVolumeSpecName: "config-data") pod "b8eb690c-165a-4906-a750-bec0178daf6c" (UID: "b8eb690c-165a-4906-a750-bec0178daf6c"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.148386 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/efb0e60c-2a71-4641-87b2-0f5532b28678-config-data" (OuterVolumeSpecName: "config-data") pod "efb0e60c-2a71-4641-87b2-0f5532b28678" (UID: "efb0e60c-2a71-4641-87b2-0f5532b28678"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.156139 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/efb0e60c-2a71-4641-87b2-0f5532b28678-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "efb0e60c-2a71-4641-87b2-0f5532b28678" (UID: "efb0e60c-2a71-4641-87b2-0f5532b28678"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.158578 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/6c8ad676-e7dd-479c-8d22-f546dc5e50af-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.158606 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6c8ad676-e7dd-479c-8d22-f546dc5e50af-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.158616 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/b8eb690c-165a-4906-a750-bec0178daf6c-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.158627 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/efb0e60c-2a71-4641-87b2-0f5532b28678-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.158638 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-jlhbj\" (UniqueName: \"kubernetes.io/projected/098a46f8-2d49-4fea-a95c-5678601aa164-kube-api-access-jlhbj\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.158654 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-zg457\" (UniqueName: \"kubernetes.io/projected/efb0e60c-2a71-4641-87b2-0f5532b28678-kube-api-access-zg457\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.158671 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/b8eb690c-165a-4906-a750-bec0178daf6c-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.158682 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b8eb690c-165a-4906-a750-bec0178daf6c-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.158693 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/6c8ad676-e7dd-479c-8d22-f546dc5e50af-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.158704 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/efb0e60c-2a71-4641-87b2-0f5532b28678-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.158713 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/b8eb690c-165a-4906-a750-bec0178daf6c-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.158721 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/098a46f8-2d49-4fea-a95c-5678601aa164-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.158748 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage18-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage18-crc\") on node \"crc\" " Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.162948 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/098a46f8-2d49-4fea-a95c-5678601aa164-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "098a46f8-2d49-4fea-a95c-5678601aa164" (UID: "098a46f8-2d49-4fea-a95c-5678601aa164"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.163664 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/098a46f8-2d49-4fea-a95c-5678601aa164-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "098a46f8-2d49-4fea-a95c-5678601aa164" (UID: "098a46f8-2d49-4fea-a95c-5678601aa164"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.172324 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/098a46f8-2d49-4fea-a95c-5678601aa164-config-data" (OuterVolumeSpecName: "config-data") pod "098a46f8-2d49-4fea-a95c-5678601aa164" (UID: "098a46f8-2d49-4fea-a95c-5678601aa164"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.187722 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage18-crc" (UniqueName: "kubernetes.io/local-volume/local-storage18-crc") on node "crc" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.212770 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/efb0e60c-2a71-4641-87b2-0f5532b28678-nova-metadata-tls-certs" (OuterVolumeSpecName: "nova-metadata-tls-certs") pod "efb0e60c-2a71-4641-87b2-0f5532b28678" (UID: "efb0e60c-2a71-4641-87b2-0f5532b28678"). InnerVolumeSpecName "nova-metadata-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.261030 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/098a46f8-2d49-4fea-a95c-5678601aa164-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.261074 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/098a46f8-2d49-4fea-a95c-5678601aa164-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.261087 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/098a46f8-2d49-4fea-a95c-5678601aa164-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.261101 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage18-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage18-crc\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.261115 5052 reconciler_common.go:293] "Volume detached for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/efb0e60c-2a71-4641-87b2-0f5532b28678-nova-metadata-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.337559 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-fdd8f875d-xvq79" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.350244 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-5f884bdbfd-2xzz4" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.357496 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.358286 5052 scope.go:117] "RemoveContainer" containerID="4f6a6de545ed43fd89fd3c33b250e3e6c4f74774fb05bed0727171451864fd53" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.374568 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.390108 5052 scope.go:117] "RemoveContainer" containerID="60ff2fa7c14572046a3621f0f09cee6b8670305019e8df8b2b52c2d7c1c39683" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.419113 5052 scope.go:117] "RemoveContainer" containerID="7f4d1f5e1e1d3b41756391a25a136984b2f8629b0050f20cbf09bc7b030ba18f" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.466911 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5mblw\" (UniqueName: \"kubernetes.io/projected/77825998-48a8-496e-a523-8e0c371fe532-kube-api-access-5mblw\") pod \"77825998-48a8-496e-a523-8e0c371fe532\" (UID: \"77825998-48a8-496e-a523-8e0c371fe532\") " Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.466977 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ll4tx\" (UniqueName: \"kubernetes.io/projected/6d653682-52e2-4940-af66-633475bfe09a-kube-api-access-ll4tx\") pod \"6d653682-52e2-4940-af66-633475bfe09a\" (UID: \"6d653682-52e2-4940-af66-633475bfe09a\") " Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.467018 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b6f8l\" (UniqueName: \"kubernetes.io/projected/a69bfb18-1267-4383-b51a-5e0eba70d1bc-kube-api-access-b6f8l\") pod \"a69bfb18-1267-4383-b51a-5e0eba70d1bc\" (UID: \"a69bfb18-1267-4383-b51a-5e0eba70d1bc\") " Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.467048 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/a69bfb18-1267-4383-b51a-5e0eba70d1bc-log-httpd\") pod \"a69bfb18-1267-4383-b51a-5e0eba70d1bc\" (UID: \"a69bfb18-1267-4383-b51a-5e0eba70d1bc\") " Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.467080 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/77825998-48a8-496e-a523-8e0c371fe532-combined-ca-bundle\") pod \"77825998-48a8-496e-a523-8e0c371fe532\" (UID: \"77825998-48a8-496e-a523-8e0c371fe532\") " Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.467118 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/a69bfb18-1267-4383-b51a-5e0eba70d1bc-ceilometer-tls-certs\") pod \"a69bfb18-1267-4383-b51a-5e0eba70d1bc\" (UID: \"a69bfb18-1267-4383-b51a-5e0eba70d1bc\") " Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.467503 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/bff27172-5e30-474d-a49c-389a35dd3052-scripts\") pod \"bff27172-5e30-474d-a49c-389a35dd3052\" (UID: \"bff27172-5e30-474d-a49c-389a35dd3052\") " Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.467584 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/77825998-48a8-496e-a523-8e0c371fe532-internal-tls-certs\") pod \"77825998-48a8-496e-a523-8e0c371fe532\" (UID: \"77825998-48a8-496e-a523-8e0c371fe532\") " Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.468016 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a69bfb18-1267-4383-b51a-5e0eba70d1bc-log-httpd" (OuterVolumeSpecName: "log-httpd") pod "a69bfb18-1267-4383-b51a-5e0eba70d1bc" (UID: "a69bfb18-1267-4383-b51a-5e0eba70d1bc"). InnerVolumeSpecName "log-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.467615 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6d653682-52e2-4940-af66-633475bfe09a-combined-ca-bundle\") pod \"6d653682-52e2-4940-af66-633475bfe09a\" (UID: \"6d653682-52e2-4940-af66-633475bfe09a\") " Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.468531 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/bff27172-5e30-474d-a49c-389a35dd3052-combined-ca-bundle\") pod \"bff27172-5e30-474d-a49c-389a35dd3052\" (UID: \"bff27172-5e30-474d-a49c-389a35dd3052\") " Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.468571 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/77825998-48a8-496e-a523-8e0c371fe532-config\") pod \"77825998-48a8-496e-a523-8e0c371fe532\" (UID: \"77825998-48a8-496e-a523-8e0c371fe532\") " Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.468602 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/a69bfb18-1267-4383-b51a-5e0eba70d1bc-run-httpd\") pod \"a69bfb18-1267-4383-b51a-5e0eba70d1bc\" (UID: \"a69bfb18-1267-4383-b51a-5e0eba70d1bc\") " Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.468639 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ncvgs\" (UniqueName: \"kubernetes.io/projected/bff27172-5e30-474d-a49c-389a35dd3052-kube-api-access-ncvgs\") pod \"bff27172-5e30-474d-a49c-389a35dd3052\" (UID: \"bff27172-5e30-474d-a49c-389a35dd3052\") " Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.468695 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/bff27172-5e30-474d-a49c-389a35dd3052-credential-keys\") pod \"bff27172-5e30-474d-a49c-389a35dd3052\" (UID: \"bff27172-5e30-474d-a49c-389a35dd3052\") " Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.468724 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/bff27172-5e30-474d-a49c-389a35dd3052-public-tls-certs\") pod \"bff27172-5e30-474d-a49c-389a35dd3052\" (UID: \"bff27172-5e30-474d-a49c-389a35dd3052\") " Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.468748 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6d653682-52e2-4940-af66-633475bfe09a-config-data\") pod \"6d653682-52e2-4940-af66-633475bfe09a\" (UID: \"6d653682-52e2-4940-af66-633475bfe09a\") " Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.468777 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/bff27172-5e30-474d-a49c-389a35dd3052-config-data\") pod \"bff27172-5e30-474d-a49c-389a35dd3052\" (UID: \"bff27172-5e30-474d-a49c-389a35dd3052\") " Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.468848 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/bff27172-5e30-474d-a49c-389a35dd3052-fernet-keys\") pod \"bff27172-5e30-474d-a49c-389a35dd3052\" (UID: \"bff27172-5e30-474d-a49c-389a35dd3052\") " Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.468886 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/77825998-48a8-496e-a523-8e0c371fe532-httpd-config\") pod \"77825998-48a8-496e-a523-8e0c371fe532\" (UID: \"77825998-48a8-496e-a523-8e0c371fe532\") " Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.468915 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a69bfb18-1267-4383-b51a-5e0eba70d1bc-combined-ca-bundle\") pod \"a69bfb18-1267-4383-b51a-5e0eba70d1bc\" (UID: \"a69bfb18-1267-4383-b51a-5e0eba70d1bc\") " Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.468950 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/a69bfb18-1267-4383-b51a-5e0eba70d1bc-sg-core-conf-yaml\") pod \"a69bfb18-1267-4383-b51a-5e0eba70d1bc\" (UID: \"a69bfb18-1267-4383-b51a-5e0eba70d1bc\") " Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.468994 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/bff27172-5e30-474d-a49c-389a35dd3052-internal-tls-certs\") pod \"bff27172-5e30-474d-a49c-389a35dd3052\" (UID: \"bff27172-5e30-474d-a49c-389a35dd3052\") " Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.469018 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a69bfb18-1267-4383-b51a-5e0eba70d1bc-config-data\") pod \"a69bfb18-1267-4383-b51a-5e0eba70d1bc\" (UID: \"a69bfb18-1267-4383-b51a-5e0eba70d1bc\") " Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.469050 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/a69bfb18-1267-4383-b51a-5e0eba70d1bc-scripts\") pod \"a69bfb18-1267-4383-b51a-5e0eba70d1bc\" (UID: \"a69bfb18-1267-4383-b51a-5e0eba70d1bc\") " Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.469075 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/77825998-48a8-496e-a523-8e0c371fe532-public-tls-certs\") pod \"77825998-48a8-496e-a523-8e0c371fe532\" (UID: \"77825998-48a8-496e-a523-8e0c371fe532\") " Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.469109 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/77825998-48a8-496e-a523-8e0c371fe532-ovndb-tls-certs\") pod \"77825998-48a8-496e-a523-8e0c371fe532\" (UID: \"77825998-48a8-496e-a523-8e0c371fe532\") " Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.469840 5052 reconciler_common.go:293] "Volume detached for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/a69bfb18-1267-4383-b51a-5e0eba70d1bc-log-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.479504 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/77825998-48a8-496e-a523-8e0c371fe532-kube-api-access-5mblw" (OuterVolumeSpecName: "kube-api-access-5mblw") pod "77825998-48a8-496e-a523-8e0c371fe532" (UID: "77825998-48a8-496e-a523-8e0c371fe532"). InnerVolumeSpecName "kube-api-access-5mblw". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.479607 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6d653682-52e2-4940-af66-633475bfe09a-kube-api-access-ll4tx" (OuterVolumeSpecName: "kube-api-access-ll4tx") pod "6d653682-52e2-4940-af66-633475bfe09a" (UID: "6d653682-52e2-4940-af66-633475bfe09a"). InnerVolumeSpecName "kube-api-access-ll4tx". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.480085 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bff27172-5e30-474d-a49c-389a35dd3052-scripts" (OuterVolumeSpecName: "scripts") pod "bff27172-5e30-474d-a49c-389a35dd3052" (UID: "bff27172-5e30-474d-a49c-389a35dd3052"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.480413 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a69bfb18-1267-4383-b51a-5e0eba70d1bc-run-httpd" (OuterVolumeSpecName: "run-httpd") pod "a69bfb18-1267-4383-b51a-5e0eba70d1bc" (UID: "a69bfb18-1267-4383-b51a-5e0eba70d1bc"). InnerVolumeSpecName "run-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.492007 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a69bfb18-1267-4383-b51a-5e0eba70d1bc-scripts" (OuterVolumeSpecName: "scripts") pod "a69bfb18-1267-4383-b51a-5e0eba70d1bc" (UID: "a69bfb18-1267-4383-b51a-5e0eba70d1bc"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.492177 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a69bfb18-1267-4383-b51a-5e0eba70d1bc-kube-api-access-b6f8l" (OuterVolumeSpecName: "kube-api-access-b6f8l") pod "a69bfb18-1267-4383-b51a-5e0eba70d1bc" (UID: "a69bfb18-1267-4383-b51a-5e0eba70d1bc"). InnerVolumeSpecName "kube-api-access-b6f8l". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.492268 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bff27172-5e30-474d-a49c-389a35dd3052-kube-api-access-ncvgs" (OuterVolumeSpecName: "kube-api-access-ncvgs") pod "bff27172-5e30-474d-a49c-389a35dd3052" (UID: "bff27172-5e30-474d-a49c-389a35dd3052"). InnerVolumeSpecName "kube-api-access-ncvgs". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.495126 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/77825998-48a8-496e-a523-8e0c371fe532-httpd-config" (OuterVolumeSpecName: "httpd-config") pod "77825998-48a8-496e-a523-8e0c371fe532" (UID: "77825998-48a8-496e-a523-8e0c371fe532"). InnerVolumeSpecName "httpd-config". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.496999 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bff27172-5e30-474d-a49c-389a35dd3052-fernet-keys" (OuterVolumeSpecName: "fernet-keys") pod "bff27172-5e30-474d-a49c-389a35dd3052" (UID: "bff27172-5e30-474d-a49c-389a35dd3052"). InnerVolumeSpecName "fernet-keys". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.497838 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bff27172-5e30-474d-a49c-389a35dd3052-credential-keys" (OuterVolumeSpecName: "credential-keys") pod "bff27172-5e30-474d-a49c-389a35dd3052" (UID: "bff27172-5e30-474d-a49c-389a35dd3052"). InnerVolumeSpecName "credential-keys". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.508783 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6d653682-52e2-4940-af66-633475bfe09a-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "6d653682-52e2-4940-af66-633475bfe09a" (UID: "6d653682-52e2-4940-af66-633475bfe09a"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.508808 5052 scope.go:117] "RemoveContainer" containerID="1fdbade92a13691b593283a5b263382dfec190075f1201c77fff78a7be754487" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.537430 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6d653682-52e2-4940-af66-633475bfe09a-config-data" (OuterVolumeSpecName: "config-data") pod "6d653682-52e2-4940-af66-633475bfe09a" (UID: "6d653682-52e2-4940-af66-633475bfe09a"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.537566 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a69bfb18-1267-4383-b51a-5e0eba70d1bc-sg-core-conf-yaml" (OuterVolumeSpecName: "sg-core-conf-yaml") pod "a69bfb18-1267-4383-b51a-5e0eba70d1bc" (UID: "a69bfb18-1267-4383-b51a-5e0eba70d1bc"). InnerVolumeSpecName "sg-core-conf-yaml". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.571848 5052 reconciler_common.go:293] "Volume detached for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/a69bfb18-1267-4383-b51a-5e0eba70d1bc-sg-core-conf-yaml\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.572073 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/a69bfb18-1267-4383-b51a-5e0eba70d1bc-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.572181 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-5mblw\" (UniqueName: \"kubernetes.io/projected/77825998-48a8-496e-a523-8e0c371fe532-kube-api-access-5mblw\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.572318 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-ll4tx\" (UniqueName: \"kubernetes.io/projected/6d653682-52e2-4940-af66-633475bfe09a-kube-api-access-ll4tx\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.572403 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-b6f8l\" (UniqueName: \"kubernetes.io/projected/a69bfb18-1267-4383-b51a-5e0eba70d1bc-kube-api-access-b6f8l\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.572476 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/bff27172-5e30-474d-a49c-389a35dd3052-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.572575 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6d653682-52e2-4940-af66-633475bfe09a-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.572650 5052 reconciler_common.go:293] "Volume detached for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/a69bfb18-1267-4383-b51a-5e0eba70d1bc-run-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.572729 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-ncvgs\" (UniqueName: \"kubernetes.io/projected/bff27172-5e30-474d-a49c-389a35dd3052-kube-api-access-ncvgs\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.572811 5052 reconciler_common.go:293] "Volume detached for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/bff27172-5e30-474d-a49c-389a35dd3052-credential-keys\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.572890 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6d653682-52e2-4940-af66-633475bfe09a-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.572969 5052 reconciler_common.go:293] "Volume detached for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/bff27172-5e30-474d-a49c-389a35dd3052-fernet-keys\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.573047 5052 reconciler_common.go:293] "Volume detached for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/77825998-48a8-496e-a523-8e0c371fe532-httpd-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.578078 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a69bfb18-1267-4383-b51a-5e0eba70d1bc-ceilometer-tls-certs" (OuterVolumeSpecName: "ceilometer-tls-certs") pod "a69bfb18-1267-4383-b51a-5e0eba70d1bc" (UID: "a69bfb18-1267-4383-b51a-5e0eba70d1bc"). InnerVolumeSpecName "ceilometer-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.591174 5052 scope.go:117] "RemoveContainer" containerID="a2b104f1b7080afce6644e6daf4c349c7a2bb8f2c68da0f5a404e6190c2e33ad" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.594271 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bff27172-5e30-474d-a49c-389a35dd3052-config-data" (OuterVolumeSpecName: "config-data") pod "bff27172-5e30-474d-a49c-389a35dd3052" (UID: "bff27172-5e30-474d-a49c-389a35dd3052"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.594259 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bff27172-5e30-474d-a49c-389a35dd3052-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "bff27172-5e30-474d-a49c-389a35dd3052" (UID: "bff27172-5e30-474d-a49c-389a35dd3052"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.594596 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/77825998-48a8-496e-a523-8e0c371fe532-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "77825998-48a8-496e-a523-8e0c371fe532" (UID: "77825998-48a8-496e-a523-8e0c371fe532"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.594986 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bff27172-5e30-474d-a49c-389a35dd3052-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "bff27172-5e30-474d-a49c-389a35dd3052" (UID: "bff27172-5e30-474d-a49c-389a35dd3052"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.595076 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bff27172-5e30-474d-a49c-389a35dd3052-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "bff27172-5e30-474d-a49c-389a35dd3052" (UID: "bff27172-5e30-474d-a49c-389a35dd3052"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.635075 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/77825998-48a8-496e-a523-8e0c371fe532-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "77825998-48a8-496e-a523-8e0c371fe532" (UID: "77825998-48a8-496e-a523-8e0c371fe532"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.635158 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/77825998-48a8-496e-a523-8e0c371fe532-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "77825998-48a8-496e-a523-8e0c371fe532" (UID: "77825998-48a8-496e-a523-8e0c371fe532"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.641018 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/77825998-48a8-496e-a523-8e0c371fe532-config" (OuterVolumeSpecName: "config") pod "77825998-48a8-496e-a523-8e0c371fe532" (UID: "77825998-48a8-496e-a523-8e0c371fe532"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.641270 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement361c-account-delete-67ttd" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.647256 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/77825998-48a8-496e-a523-8e0c371fe532-ovndb-tls-certs" (OuterVolumeSpecName: "ovndb-tls-certs") pod "77825998-48a8-496e-a523-8e0c371fe532" (UID: "77825998-48a8-496e-a523-8e0c371fe532"). InnerVolumeSpecName "ovndb-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.648156 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a69bfb18-1267-4383-b51a-5e0eba70d1bc-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "a69bfb18-1267-4383-b51a-5e0eba70d1bc" (UID: "a69bfb18-1267-4383-b51a-5e0eba70d1bc"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.673600 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/456c3fb1-90c0-4d17-863e-414380759f97-operator-scripts\") pod \"456c3fb1-90c0-4d17-863e-414380759f97\" (UID: \"456c3fb1-90c0-4d17-863e-414380759f97\") " Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.674146 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/bff27172-5e30-474d-a49c-389a35dd3052-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.674164 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/77825998-48a8-496e-a523-8e0c371fe532-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.674174 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/bff27172-5e30-474d-a49c-389a35dd3052-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.674183 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/bff27172-5e30-474d-a49c-389a35dd3052-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.674192 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a69bfb18-1267-4383-b51a-5e0eba70d1bc-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.674201 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/bff27172-5e30-474d-a49c-389a35dd3052-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.674209 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/77825998-48a8-496e-a523-8e0c371fe532-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.674217 5052 reconciler_common.go:293] "Volume detached for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/77825998-48a8-496e-a523-8e0c371fe532-ovndb-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.674226 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/77825998-48a8-496e-a523-8e0c371fe532-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.674235 5052 reconciler_common.go:293] "Volume detached for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/a69bfb18-1267-4383-b51a-5e0eba70d1bc-ceilometer-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.674243 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/77825998-48a8-496e-a523-8e0c371fe532-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.675367 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/456c3fb1-90c0-4d17-863e-414380759f97-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "456c3fb1-90c0-4d17-863e-414380759f97" (UID: "456c3fb1-90c0-4d17-863e-414380759f97"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.697268 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/novaapifd1c-account-delete-vhjzv" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.706624 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.706851 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a69bfb18-1267-4383-b51a-5e0eba70d1bc-config-data" (OuterVolumeSpecName: "config-data") pod "a69bfb18-1267-4383-b51a-5e0eba70d1bc" (UID: "a69bfb18-1267-4383-b51a-5e0eba70d1bc"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.743638 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glanced4a6-account-delete-69llz" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.763161 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/novacell05566-account-delete-2f4dm" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.778210 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/81709983-1fd0-4446-bbbf-fb94653bb3d2-config-data\") pod \"81709983-1fd0-4446-bbbf-fb94653bb3d2\" (UID: \"81709983-1fd0-4446-bbbf-fb94653bb3d2\") " Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.778323 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nqqmx\" (UniqueName: \"kubernetes.io/projected/81709983-1fd0-4446-bbbf-fb94653bb3d2-kube-api-access-nqqmx\") pod \"81709983-1fd0-4446-bbbf-fb94653bb3d2\" (UID: \"81709983-1fd0-4446-bbbf-fb94653bb3d2\") " Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.778349 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/81709983-1fd0-4446-bbbf-fb94653bb3d2-combined-ca-bundle\") pod \"81709983-1fd0-4446-bbbf-fb94653bb3d2\" (UID: \"81709983-1fd0-4446-bbbf-fb94653bb3d2\") " Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.778411 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8jp9d\" (UniqueName: \"kubernetes.io/projected/2846c397-2869-49f3-aeca-cf4b56bc8834-kube-api-access-8jp9d\") pod \"2846c397-2869-49f3-aeca-cf4b56bc8834\" (UID: \"2846c397-2869-49f3-aeca-cf4b56bc8834\") " Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.778489 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/2846c397-2869-49f3-aeca-cf4b56bc8834-operator-scripts\") pod \"2846c397-2869-49f3-aeca-cf4b56bc8834\" (UID: \"2846c397-2869-49f3-aeca-cf4b56bc8834\") " Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.778528 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7t96z\" (UniqueName: \"kubernetes.io/projected/456c3fb1-90c0-4d17-863e-414380759f97-kube-api-access-7t96z\") pod \"456c3fb1-90c0-4d17-863e-414380759f97\" (UID: \"456c3fb1-90c0-4d17-863e-414380759f97\") " Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.778936 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a69bfb18-1267-4383-b51a-5e0eba70d1bc-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.778949 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/456c3fb1-90c0-4d17-863e-414380759f97-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.779953 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2846c397-2869-49f3-aeca-cf4b56bc8834-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "2846c397-2869-49f3-aeca-cf4b56bc8834" (UID: "2846c397-2869-49f3-aeca-cf4b56bc8834"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.780016 5052 scope.go:117] "RemoveContainer" containerID="7427500c7abb1caced6a34336c8c514e1bf6bd1b958e728d97933c47c78d9675" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.782428 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2846c397-2869-49f3-aeca-cf4b56bc8834-kube-api-access-8jp9d" (OuterVolumeSpecName: "kube-api-access-8jp9d") pod "2846c397-2869-49f3-aeca-cf4b56bc8834" (UID: "2846c397-2869-49f3-aeca-cf4b56bc8834"). InnerVolumeSpecName "kube-api-access-8jp9d". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.791807 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-conductor-0" event={"ID":"6d653682-52e2-4940-af66-633475bfe09a","Type":"ContainerDied","Data":"1c60e1611f47b232078a4f996ef3cb8b4bd103c8fd84abb48567e8f4e445ef6f"} Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.791830 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.793398 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutrona25a-account-delete-vbr24" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.793718 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.795305 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/81709983-1fd0-4446-bbbf-fb94653bb3d2-kube-api-access-nqqmx" (OuterVolumeSpecName: "kube-api-access-nqqmx") pod "81709983-1fd0-4446-bbbf-fb94653bb3d2" (UID: "81709983-1fd0-4446-bbbf-fb94653bb3d2"). InnerVolumeSpecName "kube-api-access-nqqmx". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.795325 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/456c3fb1-90c0-4d17-863e-414380759f97-kube-api-access-7t96z" (OuterVolumeSpecName: "kube-api-access-7t96z") pod "456c3fb1-90c0-4d17-863e-414380759f97" (UID: "456c3fb1-90c0-4d17-863e-414380759f97"). InnerVolumeSpecName "kube-api-access-7t96z". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.803832 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-9t4zp" event={"ID":"59ec248d-84c7-4d23-b9f6-29ea8c2f36fe","Type":"ContainerStarted","Data":"35ee8cb46ad46c891647ac8e4f3735c080422ea4c90dd9702c115515d3206886"} Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.804118 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-9t4zp" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.805066 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/81709983-1fd0-4446-bbbf-fb94653bb3d2-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "81709983-1fd0-4446-bbbf-fb94653bb3d2" (UID: "81709983-1fd0-4446-bbbf-fb94653bb3d2"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.807003 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_ovn-northd-0_342cba80-22e1-488a-bbfb-75b67ca536ac/ovn-northd/0.log" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.807356 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.816317 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"81709983-1fd0-4446-bbbf-fb94653bb3d2","Type":"ContainerDied","Data":"2cbcded0899056536b72f9bbc9bb72c1f3ce02cd23f2749dd1c011532129b01c"} Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.816458 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.817119 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/81709983-1fd0-4446-bbbf-fb94653bb3d2-config-data" (OuterVolumeSpecName: "config-data") pod "81709983-1fd0-4446-bbbf-fb94653bb3d2" (UID: "81709983-1fd0-4446-bbbf-fb94653bb3d2"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.823775 5052 generic.go:334] "Generic (PLEG): container finished" podID="488c3529-4c2b-47c0-9dcf-05a064c34d0d" containerID="dfdff0317f61d3fa549a21a9c81b61a342d0bac7de867324e414cf592af4d97d" exitCode=0 Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.823862 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" event={"ID":"488c3529-4c2b-47c0-9dcf-05a064c34d0d","Type":"ContainerDied","Data":"dfdff0317f61d3fa549a21a9c81b61a342d0bac7de867324e414cf592af4d97d"} Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.823884 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" event={"ID":"488c3529-4c2b-47c0-9dcf-05a064c34d0d","Type":"ContainerDied","Data":"9ba12cd8f00246c7ec1948393afaa34606091b41050f719916c5e78d1943d909"} Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.823946 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.829449 5052 scope.go:117] "RemoveContainer" containerID="3b37742fa0a9e71beb41d34a9311dc58c88f7bd963804764ab51a685adf84ccf" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.852711 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutrona25a-account-delete-vbr24" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.852711 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutrona25a-account-delete-vbr24" event={"ID":"6ceba96c-7e86-4eb2-afa7-444f0fcc27ce","Type":"ContainerDied","Data":"4484c628b57167ddc8097fb24cae67ddfbe52a0c2419f2ef5e9d75c47d729dc6"} Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.857655 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/novaapifd1c-account-delete-vhjzv" event={"ID":"2846c397-2869-49f3-aeca-cf4b56bc8834","Type":"ContainerDied","Data":"fd19211cd437571e300e0af34e890cfd3bcac9bd1d280e66811c743e020d56b1"} Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.857788 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/novaapifd1c-account-delete-vhjzv" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.860794 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/novacell05566-account-delete-2f4dm" event={"ID":"a2bc501c-ebef-4e68-8ae2-e8c4217cadff","Type":"ContainerDied","Data":"a06ff1694d0794e101db19e3b416b77efe12f42ffdd886b18c15f4ce2305b6f0"} Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.865252 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glanced4a6-account-delete-69llz" event={"ID":"6a5de7fb-b482-4849-a957-e679616aabc0","Type":"ContainerDied","Data":"b4607ef57b739c3dcf2c9020dc61004b4be30f5f17eab10a2de4e5156869e7dd"} Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.865354 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glanced4a6-account-delete-69llz" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.878039 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/novacell05566-account-delete-2f4dm" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.880619 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"server-conf\" (UniqueName: \"kubernetes.io/configmap/488c3529-4c2b-47c0-9dcf-05a064c34d0d-server-conf\") pod \"488c3529-4c2b-47c0-9dcf-05a064c34d0d\" (UID: \"488c3529-4c2b-47c0-9dcf-05a064c34d0d\") " Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.880663 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d722f\" (UniqueName: \"kubernetes.io/projected/6a5de7fb-b482-4849-a957-e679616aabc0-kube-api-access-d722f\") pod \"6a5de7fb-b482-4849-a957-e679616aabc0\" (UID: \"6a5de7fb-b482-4849-a957-e679616aabc0\") " Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.880729 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2hxgh\" (UniqueName: \"kubernetes.io/projected/342cba80-22e1-488a-bbfb-75b67ca536ac-kube-api-access-2hxgh\") pod \"342cba80-22e1-488a-bbfb-75b67ca536ac\" (UID: \"342cba80-22e1-488a-bbfb-75b67ca536ac\") " Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.880783 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/6a5de7fb-b482-4849-a957-e679616aabc0-operator-scripts\") pod \"6a5de7fb-b482-4849-a957-e679616aabc0\" (UID: \"6a5de7fb-b482-4849-a957-e679616aabc0\") " Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.880821 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/6ceba96c-7e86-4eb2-afa7-444f0fcc27ce-operator-scripts\") pod \"6ceba96c-7e86-4eb2-afa7-444f0fcc27ce\" (UID: \"6ceba96c-7e86-4eb2-afa7-444f0fcc27ce\") " Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.880853 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovn-rundir\" (UniqueName: \"kubernetes.io/empty-dir/342cba80-22e1-488a-bbfb-75b67ca536ac-ovn-rundir\") pod \"342cba80-22e1-488a-bbfb-75b67ca536ac\" (UID: \"342cba80-22e1-488a-bbfb-75b67ca536ac\") " Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.880899 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"rabbitmq-erlang-cookie\" (UniqueName: \"kubernetes.io/empty-dir/488c3529-4c2b-47c0-9dcf-05a064c34d0d-rabbitmq-erlang-cookie\") pod \"488c3529-4c2b-47c0-9dcf-05a064c34d0d\" (UID: \"488c3529-4c2b-47c0-9dcf-05a064c34d0d\") " Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.880923 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tpc6k\" (UniqueName: \"kubernetes.io/projected/6ceba96c-7e86-4eb2-afa7-444f0fcc27ce-kube-api-access-tpc6k\") pod \"6ceba96c-7e86-4eb2-afa7-444f0fcc27ce\" (UID: \"6ceba96c-7e86-4eb2-afa7-444f0fcc27ce\") " Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.880962 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"pod-info\" (UniqueName: \"kubernetes.io/downward-api/488c3529-4c2b-47c0-9dcf-05a064c34d0d-pod-info\") pod \"488c3529-4c2b-47c0-9dcf-05a064c34d0d\" (UID: \"488c3529-4c2b-47c0-9dcf-05a064c34d0d\") " Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.880988 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/342cba80-22e1-488a-bbfb-75b67ca536ac-combined-ca-bundle\") pod \"342cba80-22e1-488a-bbfb-75b67ca536ac\" (UID: \"342cba80-22e1-488a-bbfb-75b67ca536ac\") " Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.881014 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovn-northd-tls-certs\" (UniqueName: \"kubernetes.io/secret/342cba80-22e1-488a-bbfb-75b67ca536ac-ovn-northd-tls-certs\") pod \"342cba80-22e1-488a-bbfb-75b67ca536ac\" (UID: \"342cba80-22e1-488a-bbfb-75b67ca536ac\") " Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.881033 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/342cba80-22e1-488a-bbfb-75b67ca536ac-metrics-certs-tls-certs\") pod \"342cba80-22e1-488a-bbfb-75b67ca536ac\" (UID: \"342cba80-22e1-488a-bbfb-75b67ca536ac\") " Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.881053 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"plugins-conf\" (UniqueName: \"kubernetes.io/configmap/488c3529-4c2b-47c0-9dcf-05a064c34d0d-plugins-conf\") pod \"488c3529-4c2b-47c0-9dcf-05a064c34d0d\" (UID: \"488c3529-4c2b-47c0-9dcf-05a064c34d0d\") " Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.881077 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/a2bc501c-ebef-4e68-8ae2-e8c4217cadff-operator-scripts\") pod \"a2bc501c-ebef-4e68-8ae2-e8c4217cadff\" (UID: \"a2bc501c-ebef-4e68-8ae2-e8c4217cadff\") " Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.881107 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/342cba80-22e1-488a-bbfb-75b67ca536ac-config\") pod \"342cba80-22e1-488a-bbfb-75b67ca536ac\" (UID: \"342cba80-22e1-488a-bbfb-75b67ca536ac\") " Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.881125 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"persistence\" (UniqueName: \"kubernetes.io/local-volume/local-storage20-crc\") pod \"488c3529-4c2b-47c0-9dcf-05a064c34d0d\" (UID: \"488c3529-4c2b-47c0-9dcf-05a064c34d0d\") " Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.881145 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"rabbitmq-confd\" (UniqueName: \"kubernetes.io/projected/488c3529-4c2b-47c0-9dcf-05a064c34d0d-rabbitmq-confd\") pod \"488c3529-4c2b-47c0-9dcf-05a064c34d0d\" (UID: \"488c3529-4c2b-47c0-9dcf-05a064c34d0d\") " Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.881174 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cm686\" (UniqueName: \"kubernetes.io/projected/488c3529-4c2b-47c0-9dcf-05a064c34d0d-kube-api-access-cm686\") pod \"488c3529-4c2b-47c0-9dcf-05a064c34d0d\" (UID: \"488c3529-4c2b-47c0-9dcf-05a064c34d0d\") " Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.881221 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"rabbitmq-tls\" (UniqueName: \"kubernetes.io/projected/488c3529-4c2b-47c0-9dcf-05a064c34d0d-rabbitmq-tls\") pod \"488c3529-4c2b-47c0-9dcf-05a064c34d0d\" (UID: \"488c3529-4c2b-47c0-9dcf-05a064c34d0d\") " Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.881242 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/342cba80-22e1-488a-bbfb-75b67ca536ac-scripts\") pod \"342cba80-22e1-488a-bbfb-75b67ca536ac\" (UID: \"342cba80-22e1-488a-bbfb-75b67ca536ac\") " Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.881268 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z7sjx\" (UniqueName: \"kubernetes.io/projected/a2bc501c-ebef-4e68-8ae2-e8c4217cadff-kube-api-access-z7sjx\") pod \"a2bc501c-ebef-4e68-8ae2-e8c4217cadff\" (UID: \"a2bc501c-ebef-4e68-8ae2-e8c4217cadff\") " Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.881303 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/488c3529-4c2b-47c0-9dcf-05a064c34d0d-config-data\") pod \"488c3529-4c2b-47c0-9dcf-05a064c34d0d\" (UID: \"488c3529-4c2b-47c0-9dcf-05a064c34d0d\") " Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.881324 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"erlang-cookie-secret\" (UniqueName: \"kubernetes.io/secret/488c3529-4c2b-47c0-9dcf-05a064c34d0d-erlang-cookie-secret\") pod \"488c3529-4c2b-47c0-9dcf-05a064c34d0d\" (UID: \"488c3529-4c2b-47c0-9dcf-05a064c34d0d\") " Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.881350 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"rabbitmq-plugins\" (UniqueName: \"kubernetes.io/empty-dir/488c3529-4c2b-47c0-9dcf-05a064c34d0d-rabbitmq-plugins\") pod \"488c3529-4c2b-47c0-9dcf-05a064c34d0d\" (UID: \"488c3529-4c2b-47c0-9dcf-05a064c34d0d\") " Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.881837 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/342cba80-22e1-488a-bbfb-75b67ca536ac-config" (OuterVolumeSpecName: "config") pod "342cba80-22e1-488a-bbfb-75b67ca536ac" (UID: "342cba80-22e1-488a-bbfb-75b67ca536ac"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.881975 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-7t96z\" (UniqueName: \"kubernetes.io/projected/456c3fb1-90c0-4d17-863e-414380759f97-kube-api-access-7t96z\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.881994 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/81709983-1fd0-4446-bbbf-fb94653bb3d2-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.882153 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-nqqmx\" (UniqueName: \"kubernetes.io/projected/81709983-1fd0-4446-bbbf-fb94653bb3d2-kube-api-access-nqqmx\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.882165 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/81709983-1fd0-4446-bbbf-fb94653bb3d2-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.882258 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/342cba80-22e1-488a-bbfb-75b67ca536ac-ovn-rundir" (OuterVolumeSpecName: "ovn-rundir") pod "342cba80-22e1-488a-bbfb-75b67ca536ac" (UID: "342cba80-22e1-488a-bbfb-75b67ca536ac"). InnerVolumeSpecName "ovn-rundir". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.882327 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-8jp9d\" (UniqueName: \"kubernetes.io/projected/2846c397-2869-49f3-aeca-cf4b56bc8834-kube-api-access-8jp9d\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.882354 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/342cba80-22e1-488a-bbfb-75b67ca536ac-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.882365 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/2846c397-2869-49f3-aeca-cf4b56bc8834-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.882642 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6ceba96c-7e86-4eb2-afa7-444f0fcc27ce-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "6ceba96c-7e86-4eb2-afa7-444f0fcc27ce" (UID: "6ceba96c-7e86-4eb2-afa7-444f0fcc27ce"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.883005 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/488c3529-4c2b-47c0-9dcf-05a064c34d0d-rabbitmq-plugins" (OuterVolumeSpecName: "rabbitmq-plugins") pod "488c3529-4c2b-47c0-9dcf-05a064c34d0d" (UID: "488c3529-4c2b-47c0-9dcf-05a064c34d0d"). InnerVolumeSpecName "rabbitmq-plugins". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.883610 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/488c3529-4c2b-47c0-9dcf-05a064c34d0d-rabbitmq-erlang-cookie" (OuterVolumeSpecName: "rabbitmq-erlang-cookie") pod "488c3529-4c2b-47c0-9dcf-05a064c34d0d" (UID: "488c3529-4c2b-47c0-9dcf-05a064c34d0d"). InnerVolumeSpecName "rabbitmq-erlang-cookie". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.887977 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6a5de7fb-b482-4849-a957-e679616aabc0-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "6a5de7fb-b482-4849-a957-e679616aabc0" (UID: "6a5de7fb-b482-4849-a957-e679616aabc0"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.888949 5052 scope.go:117] "RemoveContainer" containerID="626ebae6286b0199eb107048b8e04a3c2f172a614197d9cbb4a4d9047544b8e0" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.891007 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/342cba80-22e1-488a-bbfb-75b67ca536ac-scripts" (OuterVolumeSpecName: "scripts") pod "342cba80-22e1-488a-bbfb-75b67ca536ac" (UID: "342cba80-22e1-488a-bbfb-75b67ca536ac"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.891720 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/488c3529-4c2b-47c0-9dcf-05a064c34d0d-plugins-conf" (OuterVolumeSpecName: "plugins-conf") pod "488c3529-4c2b-47c0-9dcf-05a064c34d0d" (UID: "488c3529-4c2b-47c0-9dcf-05a064c34d0d"). InnerVolumeSpecName "plugins-conf". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.891752 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a2bc501c-ebef-4e68-8ae2-e8c4217cadff-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "a2bc501c-ebef-4e68-8ae2-e8c4217cadff" (UID: "a2bc501c-ebef-4e68-8ae2-e8c4217cadff"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.898059 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/488c3529-4c2b-47c0-9dcf-05a064c34d0d-kube-api-access-cm686" (OuterVolumeSpecName: "kube-api-access-cm686") pod "488c3529-4c2b-47c0-9dcf-05a064c34d0d" (UID: "488c3529-4c2b-47c0-9dcf-05a064c34d0d"). InnerVolumeSpecName "kube-api-access-cm686". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.898198 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.898227 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.898278 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement361c-account-delete-67ttd" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.898291 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement361c-account-delete-67ttd" event={"ID":"456c3fb1-90c0-4d17-863e-414380759f97","Type":"ContainerDied","Data":"b9d37c789843877b9b3d38fbc02a09b3df53f9dbff1743cb3ac630230e61dd79"} Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.898338 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.898385 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-fdd8f875d-xvq79" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.898504 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-5f884bdbfd-2xzz4" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.898584 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.898626 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-api-75bdf7fcbb-mlkgz" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.898643 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/488c3529-4c2b-47c0-9dcf-05a064c34d0d-rabbitmq-tls" (OuterVolumeSpecName: "rabbitmq-tls") pod "488c3529-4c2b-47c0-9dcf-05a064c34d0d" (UID: "488c3529-4c2b-47c0-9dcf-05a064c34d0d"). InnerVolumeSpecName "rabbitmq-tls". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.899088 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-9t4zp" podStartSLOduration=8.899069791 podStartE2EDuration="8.899069791s" podCreationTimestamp="2025-12-03 12:48:58 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:49:06.866326964 +0000 UTC m=+5337.454568728" watchObservedRunningTime="2025-12-03 12:49:06.899069791 +0000 UTC m=+5337.487311555" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.900458 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican02cc-account-delete-gf94b" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.901245 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder4345-account-delete-vmdzh" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.906023 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6a5de7fb-b482-4849-a957-e679616aabc0-kube-api-access-d722f" (OuterVolumeSpecName: "kube-api-access-d722f") pod "6a5de7fb-b482-4849-a957-e679616aabc0" (UID: "6a5de7fb-b482-4849-a957-e679616aabc0"). InnerVolumeSpecName "kube-api-access-d722f". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.906759 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a2bc501c-ebef-4e68-8ae2-e8c4217cadff-kube-api-access-z7sjx" (OuterVolumeSpecName: "kube-api-access-z7sjx") pod "a2bc501c-ebef-4e68-8ae2-e8c4217cadff" (UID: "a2bc501c-ebef-4e68-8ae2-e8c4217cadff"). InnerVolumeSpecName "kube-api-access-z7sjx". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.911813 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.924961 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage20-crc" (OuterVolumeSpecName: "persistence") pod "488c3529-4c2b-47c0-9dcf-05a064c34d0d" (UID: "488c3529-4c2b-47c0-9dcf-05a064c34d0d"). InnerVolumeSpecName "local-storage20-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.925042 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/488c3529-4c2b-47c0-9dcf-05a064c34d0d-pod-info" (OuterVolumeSpecName: "pod-info") pod "488c3529-4c2b-47c0-9dcf-05a064c34d0d" (UID: "488c3529-4c2b-47c0-9dcf-05a064c34d0d"). InnerVolumeSpecName "pod-info". PluginName "kubernetes.io/downward-api", VolumeGidValue "" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.925427 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/488c3529-4c2b-47c0-9dcf-05a064c34d0d-erlang-cookie-secret" (OuterVolumeSpecName: "erlang-cookie-secret") pod "488c3529-4c2b-47c0-9dcf-05a064c34d0d" (UID: "488c3529-4c2b-47c0-9dcf-05a064c34d0d"). InnerVolumeSpecName "erlang-cookie-secret". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.935586 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6ceba96c-7e86-4eb2-afa7-444f0fcc27ce-kube-api-access-tpc6k" (OuterVolumeSpecName: "kube-api-access-tpc6k") pod "6ceba96c-7e86-4eb2-afa7-444f0fcc27ce" (UID: "6ceba96c-7e86-4eb2-afa7-444f0fcc27ce"). InnerVolumeSpecName "kube-api-access-tpc6k". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.936720 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/342cba80-22e1-488a-bbfb-75b67ca536ac-kube-api-access-2hxgh" (OuterVolumeSpecName: "kube-api-access-2hxgh") pod "342cba80-22e1-488a-bbfb-75b67ca536ac" (UID: "342cba80-22e1-488a-bbfb-75b67ca536ac"). InnerVolumeSpecName "kube-api-access-2hxgh". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.946393 5052 scope.go:117] "RemoveContainer" containerID="747dff2aa2b4006f1eacd1565f1e19dcab8b056e4b9a569dfe7735eef649cbe8" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.946403 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/488c3529-4c2b-47c0-9dcf-05a064c34d0d-config-data" (OuterVolumeSpecName: "config-data") pod "488c3529-4c2b-47c0-9dcf-05a064c34d0d" (UID: "488c3529-4c2b-47c0-9dcf-05a064c34d0d"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.983818 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/3c8e4e3f-f6c1-4362-b64c-b10a973723dd-galera-tls-certs\") pod \"3c8e4e3f-f6c1-4362-b64c-b10a973723dd\" (UID: \"3c8e4e3f-f6c1-4362-b64c-b10a973723dd\") " Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.983916 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3c8e4e3f-f6c1-4362-b64c-b10a973723dd-combined-ca-bundle\") pod \"3c8e4e3f-f6c1-4362-b64c-b10a973723dd\" (UID: \"3c8e4e3f-f6c1-4362-b64c-b10a973723dd\") " Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.983948 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/3c8e4e3f-f6c1-4362-b64c-b10a973723dd-config-data-default\") pod \"3c8e4e3f-f6c1-4362-b64c-b10a973723dd\" (UID: \"3c8e4e3f-f6c1-4362-b64c-b10a973723dd\") " Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.983995 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/3c8e4e3f-f6c1-4362-b64c-b10a973723dd-operator-scripts\") pod \"3c8e4e3f-f6c1-4362-b64c-b10a973723dd\" (UID: \"3c8e4e3f-f6c1-4362-b64c-b10a973723dd\") " Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.984030 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/5d11b02e-c14a-4d1a-819a-4c3b66aedd33-operator-scripts\") pod \"5d11b02e-c14a-4d1a-819a-4c3b66aedd33\" (UID: \"5d11b02e-c14a-4d1a-819a-4c3b66aedd33\") " Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.984062 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/857b8a8a-d730-4832-a71c-d45a2f3ce406-operator-scripts\") pod \"857b8a8a-d730-4832-a71c-d45a2f3ce406\" (UID: \"857b8a8a-d730-4832-a71c-d45a2f3ce406\") " Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.984117 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ftscq\" (UniqueName: \"kubernetes.io/projected/3c8e4e3f-f6c1-4362-b64c-b10a973723dd-kube-api-access-ftscq\") pod \"3c8e4e3f-f6c1-4362-b64c-b10a973723dd\" (UID: \"3c8e4e3f-f6c1-4362-b64c-b10a973723dd\") " Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.984211 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/3c8e4e3f-f6c1-4362-b64c-b10a973723dd-config-data-generated\") pod \"3c8e4e3f-f6c1-4362-b64c-b10a973723dd\" (UID: \"3c8e4e3f-f6c1-4362-b64c-b10a973723dd\") " Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.984242 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8clcg\" (UniqueName: \"kubernetes.io/projected/5d11b02e-c14a-4d1a-819a-4c3b66aedd33-kube-api-access-8clcg\") pod \"5d11b02e-c14a-4d1a-819a-4c3b66aedd33\" (UID: \"5d11b02e-c14a-4d1a-819a-4c3b66aedd33\") " Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.984312 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/3c8e4e3f-f6c1-4362-b64c-b10a973723dd-kolla-config\") pod \"3c8e4e3f-f6c1-4362-b64c-b10a973723dd\" (UID: \"3c8e4e3f-f6c1-4362-b64c-b10a973723dd\") " Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.984344 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"mysql-db\" (UniqueName: \"kubernetes.io/local-volume/local-storage07-crc\") pod \"3c8e4e3f-f6c1-4362-b64c-b10a973723dd\" (UID: \"3c8e4e3f-f6c1-4362-b64c-b10a973723dd\") " Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.984384 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5mh5h\" (UniqueName: \"kubernetes.io/projected/857b8a8a-d730-4832-a71c-d45a2f3ce406-kube-api-access-5mh5h\") pod \"857b8a8a-d730-4832-a71c-d45a2f3ce406\" (UID: \"857b8a8a-d730-4832-a71c-d45a2f3ce406\") " Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.984787 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-2hxgh\" (UniqueName: \"kubernetes.io/projected/342cba80-22e1-488a-bbfb-75b67ca536ac-kube-api-access-2hxgh\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.984805 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/6a5de7fb-b482-4849-a957-e679616aabc0-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.984818 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/6ceba96c-7e86-4eb2-afa7-444f0fcc27ce-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.984828 5052 reconciler_common.go:293] "Volume detached for volume \"ovn-rundir\" (UniqueName: \"kubernetes.io/empty-dir/342cba80-22e1-488a-bbfb-75b67ca536ac-ovn-rundir\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.984842 5052 reconciler_common.go:293] "Volume detached for volume \"rabbitmq-erlang-cookie\" (UniqueName: \"kubernetes.io/empty-dir/488c3529-4c2b-47c0-9dcf-05a064c34d0d-rabbitmq-erlang-cookie\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.984855 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-tpc6k\" (UniqueName: \"kubernetes.io/projected/6ceba96c-7e86-4eb2-afa7-444f0fcc27ce-kube-api-access-tpc6k\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.984867 5052 reconciler_common.go:293] "Volume detached for volume \"pod-info\" (UniqueName: \"kubernetes.io/downward-api/488c3529-4c2b-47c0-9dcf-05a064c34d0d-pod-info\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.984877 5052 reconciler_common.go:293] "Volume detached for volume \"plugins-conf\" (UniqueName: \"kubernetes.io/configmap/488c3529-4c2b-47c0-9dcf-05a064c34d0d-plugins-conf\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.984888 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/a2bc501c-ebef-4e68-8ae2-e8c4217cadff-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.984908 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage20-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage20-crc\") on node \"crc\" " Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.984919 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-cm686\" (UniqueName: \"kubernetes.io/projected/488c3529-4c2b-47c0-9dcf-05a064c34d0d-kube-api-access-cm686\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.984930 5052 reconciler_common.go:293] "Volume detached for volume \"rabbitmq-tls\" (UniqueName: \"kubernetes.io/projected/488c3529-4c2b-47c0-9dcf-05a064c34d0d-rabbitmq-tls\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.984939 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/342cba80-22e1-488a-bbfb-75b67ca536ac-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.984950 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-z7sjx\" (UniqueName: \"kubernetes.io/projected/a2bc501c-ebef-4e68-8ae2-e8c4217cadff-kube-api-access-z7sjx\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.984960 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/488c3529-4c2b-47c0-9dcf-05a064c34d0d-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.984972 5052 reconciler_common.go:293] "Volume detached for volume \"erlang-cookie-secret\" (UniqueName: \"kubernetes.io/secret/488c3529-4c2b-47c0-9dcf-05a064c34d0d-erlang-cookie-secret\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.984982 5052 reconciler_common.go:293] "Volume detached for volume \"rabbitmq-plugins\" (UniqueName: \"kubernetes.io/empty-dir/488c3529-4c2b-47c0-9dcf-05a064c34d0d-rabbitmq-plugins\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.984993 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-d722f\" (UniqueName: \"kubernetes.io/projected/6a5de7fb-b482-4849-a957-e679616aabc0-kube-api-access-d722f\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.987397 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c8e4e3f-f6c1-4362-b64c-b10a973723dd-config-data-generated" (OuterVolumeSpecName: "config-data-generated") pod "3c8e4e3f-f6c1-4362-b64c-b10a973723dd" (UID: "3c8e4e3f-f6c1-4362-b64c-b10a973723dd"). InnerVolumeSpecName "config-data-generated". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.988557 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3c8e4e3f-f6c1-4362-b64c-b10a973723dd-config-data-default" (OuterVolumeSpecName: "config-data-default") pod "3c8e4e3f-f6c1-4362-b64c-b10a973723dd" (UID: "3c8e4e3f-f6c1-4362-b64c-b10a973723dd"). InnerVolumeSpecName "config-data-default". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.991387 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3c8e4e3f-f6c1-4362-b64c-b10a973723dd-kolla-config" (OuterVolumeSpecName: "kolla-config") pod "3c8e4e3f-f6c1-4362-b64c-b10a973723dd" (UID: "3c8e4e3f-f6c1-4362-b64c-b10a973723dd"). InnerVolumeSpecName "kolla-config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.993619 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3c8e4e3f-f6c1-4362-b64c-b10a973723dd-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "3c8e4e3f-f6c1-4362-b64c-b10a973723dd" (UID: "3c8e4e3f-f6c1-4362-b64c-b10a973723dd"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.994119 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5d11b02e-c14a-4d1a-819a-4c3b66aedd33-kube-api-access-8clcg" (OuterVolumeSpecName: "kube-api-access-8clcg") pod "5d11b02e-c14a-4d1a-819a-4c3b66aedd33" (UID: "5d11b02e-c14a-4d1a-819a-4c3b66aedd33"). InnerVolumeSpecName "kube-api-access-8clcg". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.994564 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3c8e4e3f-f6c1-4362-b64c-b10a973723dd-kube-api-access-ftscq" (OuterVolumeSpecName: "kube-api-access-ftscq") pod "3c8e4e3f-f6c1-4362-b64c-b10a973723dd" (UID: "3c8e4e3f-f6c1-4362-b64c-b10a973723dd"). InnerVolumeSpecName "kube-api-access-ftscq". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.995164 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/857b8a8a-d730-4832-a71c-d45a2f3ce406-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "857b8a8a-d730-4832-a71c-d45a2f3ce406" (UID: "857b8a8a-d730-4832-a71c-d45a2f3ce406"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:49:06 crc kubenswrapper[5052]: I1203 12:49:06.995603 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5d11b02e-c14a-4d1a-819a-4c3b66aedd33-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "5d11b02e-c14a-4d1a-819a-4c3b66aedd33" (UID: "5d11b02e-c14a-4d1a-819a-4c3b66aedd33"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.009005 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/857b8a8a-d730-4832-a71c-d45a2f3ce406-kube-api-access-5mh5h" (OuterVolumeSpecName: "kube-api-access-5mh5h") pod "857b8a8a-d730-4832-a71c-d45a2f3ce406" (UID: "857b8a8a-d730-4832-a71c-d45a2f3ce406"). InnerVolumeSpecName "kube-api-access-5mh5h". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.011002 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-0"] Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.014378 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/342cba80-22e1-488a-bbfb-75b67ca536ac-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "342cba80-22e1-488a-bbfb-75b67ca536ac" (UID: "342cba80-22e1-488a-bbfb-75b67ca536ac"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.015743 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage07-crc" (OuterVolumeSpecName: "mysql-db") pod "3c8e4e3f-f6c1-4362-b64c-b10a973723dd" (UID: "3c8e4e3f-f6c1-4362-b64c-b10a973723dd"). InnerVolumeSpecName "local-storage07-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.020271 5052 scope.go:117] "RemoveContainer" containerID="b1f27aab2d1cce51772c0ce99bd8dfc4afd0620b38b776e6b968d9114105f165" Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.034112 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-0"] Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.038631 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage20-crc" (UniqueName: "kubernetes.io/local-volume/local-storage20-crc") on node "crc" Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.041748 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/488c3529-4c2b-47c0-9dcf-05a064c34d0d-server-conf" (OuterVolumeSpecName: "server-conf") pod "488c3529-4c2b-47c0-9dcf-05a064c34d0d" (UID: "488c3529-4c2b-47c0-9dcf-05a064c34d0d"). InnerVolumeSpecName "server-conf". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.046567 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/novaapifd1c-account-delete-vhjzv"] Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.054202 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3c8e4e3f-f6c1-4362-b64c-b10a973723dd-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "3c8e4e3f-f6c1-4362-b64c-b10a973723dd" (UID: "3c8e4e3f-f6c1-4362-b64c-b10a973723dd"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.055196 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/342cba80-22e1-488a-bbfb-75b67ca536ac-metrics-certs-tls-certs" (OuterVolumeSpecName: "metrics-certs-tls-certs") pod "342cba80-22e1-488a-bbfb-75b67ca536ac" (UID: "342cba80-22e1-488a-bbfb-75b67ca536ac"). InnerVolumeSpecName "metrics-certs-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.063840 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/novaapifd1c-account-delete-vhjzv"] Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.086741 5052 reconciler_common.go:293] "Volume detached for volume \"server-conf\" (UniqueName: \"kubernetes.io/configmap/488c3529-4c2b-47c0-9dcf-05a064c34d0d-server-conf\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.086763 5052 reconciler_common.go:293] "Volume detached for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/3c8e4e3f-f6c1-4362-b64c-b10a973723dd-kolla-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.086792 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage07-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage07-crc\") on node \"crc\" " Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.086803 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-5mh5h\" (UniqueName: \"kubernetes.io/projected/857b8a8a-d730-4832-a71c-d45a2f3ce406-kube-api-access-5mh5h\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.086815 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3c8e4e3f-f6c1-4362-b64c-b10a973723dd-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.086825 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/3c8e4e3f-f6c1-4362-b64c-b10a973723dd-config-data-default\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.086836 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/3c8e4e3f-f6c1-4362-b64c-b10a973723dd-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.086846 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/5d11b02e-c14a-4d1a-819a-4c3b66aedd33-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.086856 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/857b8a8a-d730-4832-a71c-d45a2f3ce406-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.086864 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/342cba80-22e1-488a-bbfb-75b67ca536ac-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.086873 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-ftscq\" (UniqueName: \"kubernetes.io/projected/3c8e4e3f-f6c1-4362-b64c-b10a973723dd-kube-api-access-ftscq\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.086881 5052 reconciler_common.go:293] "Volume detached for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/342cba80-22e1-488a-bbfb-75b67ca536ac-metrics-certs-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.086889 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage20-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage20-crc\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.086897 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/3c8e4e3f-f6c1-4362-b64c-b10a973723dd-config-data-generated\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.086905 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-8clcg\" (UniqueName: \"kubernetes.io/projected/5d11b02e-c14a-4d1a-819a-4c3b66aedd33-kube-api-access-8clcg\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.097589 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/342cba80-22e1-488a-bbfb-75b67ca536ac-ovn-northd-tls-certs" (OuterVolumeSpecName: "ovn-northd-tls-certs") pod "342cba80-22e1-488a-bbfb-75b67ca536ac" (UID: "342cba80-22e1-488a-bbfb-75b67ca536ac"). InnerVolumeSpecName "ovn-northd-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.104656 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3c8e4e3f-f6c1-4362-b64c-b10a973723dd-galera-tls-certs" (OuterVolumeSpecName: "galera-tls-certs") pod "3c8e4e3f-f6c1-4362-b64c-b10a973723dd" (UID: "3c8e4e3f-f6c1-4362-b64c-b10a973723dd"). InnerVolumeSpecName "galera-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.112177 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage07-crc" (UniqueName: "kubernetes.io/local-volume/local-storage07-crc") on node "crc" Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.114675 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/488c3529-4c2b-47c0-9dcf-05a064c34d0d-rabbitmq-confd" (OuterVolumeSpecName: "rabbitmq-confd") pod "488c3529-4c2b-47c0-9dcf-05a064c34d0d" (UID: "488c3529-4c2b-47c0-9dcf-05a064c34d0d"). InnerVolumeSpecName "rabbitmq-confd". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.188734 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage07-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage07-crc\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.188772 5052 reconciler_common.go:293] "Volume detached for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/3c8e4e3f-f6c1-4362-b64c-b10a973723dd-galera-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.188783 5052 reconciler_common.go:293] "Volume detached for volume \"ovn-northd-tls-certs\" (UniqueName: \"kubernetes.io/secret/342cba80-22e1-488a-bbfb-75b67ca536ac-ovn-northd-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.188794 5052 reconciler_common.go:293] "Volume detached for volume \"rabbitmq-confd\" (UniqueName: \"kubernetes.io/projected/488c3529-4c2b-47c0-9dcf-05a064c34d0d-rabbitmq-confd\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.400725 5052 scope.go:117] "RemoveContainer" containerID="b58ebff57a6b5eca68fc9558830a4aee95150c885bec5d3131628c10c8d77c4e" Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.407256 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.424501 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-api-75bdf7fcbb-mlkgz"] Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.432289 5052 scope.go:117] "RemoveContainer" containerID="2b5c5249476f9fa6344558a0eabe96f98d744f43f1cd4faf1d66ffa9c49d4941" Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.434303 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/barbican-api-75bdf7fcbb-mlkgz"] Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.442480 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.452352 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.470075 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/placement361c-account-delete-67ttd"] Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.477908 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/placement361c-account-delete-67ttd"] Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.495764 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-66gvc\" (UniqueName: \"kubernetes.io/projected/8669832d-c47a-49bd-9491-dc7141ac6281-kube-api-access-66gvc\") pod \"8669832d-c47a-49bd-9491-dc7141ac6281\" (UID: \"8669832d-c47a-49bd-9491-dc7141ac6281\") " Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.495808 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8669832d-c47a-49bd-9491-dc7141ac6281-combined-ca-bundle\") pod \"8669832d-c47a-49bd-9491-dc7141ac6281\" (UID: \"8669832d-c47a-49bd-9491-dc7141ac6281\") " Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.495945 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/8669832d-c47a-49bd-9491-dc7141ac6281-config-data\") pod \"8669832d-c47a-49bd-9491-dc7141ac6281\" (UID: \"8669832d-c47a-49bd-9491-dc7141ac6281\") " Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.502566 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8669832d-c47a-49bd-9491-dc7141ac6281-kube-api-access-66gvc" (OuterVolumeSpecName: "kube-api-access-66gvc") pod "8669832d-c47a-49bd-9491-dc7141ac6281" (UID: "8669832d-c47a-49bd-9491-dc7141ac6281"). InnerVolumeSpecName "kube-api-access-66gvc". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:49:07 crc kubenswrapper[5052]: E1203 12:49:07.514463 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8669832d-c47a-49bd-9491-dc7141ac6281-config-data podName:8669832d-c47a-49bd-9491-dc7141ac6281 nodeName:}" failed. No retries permitted until 2025-12-03 12:49:08.014439619 +0000 UTC m=+5338.602681383 (durationBeforeRetry 500ms). Error: error cleaning subPath mounts for volume "config-data" (UniqueName: "kubernetes.io/secret/8669832d-c47a-49bd-9491-dc7141ac6281-config-data") pod "8669832d-c47a-49bd-9491-dc7141ac6281" (UID: "8669832d-c47a-49bd-9491-dc7141ac6281") : error deleting /var/lib/kubelet/pods/8669832d-c47a-49bd-9491-dc7141ac6281/volume-subpaths: remove /var/lib/kubelet/pods/8669832d-c47a-49bd-9491-dc7141ac6281/volume-subpaths: no such file or directory Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.516624 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8669832d-c47a-49bd-9491-dc7141ac6281-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "8669832d-c47a-49bd-9491-dc7141ac6281" (UID: "8669832d-c47a-49bd-9491-dc7141ac6281"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.558755 5052 scope.go:117] "RemoveContainer" containerID="6964abaffe4177de7695a956baa51adeedd8fc7ae4b4885ca8e38046eae0369a" Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.561229 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.590020 5052 scope.go:117] "RemoveContainer" containerID="b1f27aab2d1cce51772c0ce99bd8dfc4afd0620b38b776e6b968d9114105f165" Dec 03 12:49:07 crc kubenswrapper[5052]: E1203 12:49:07.590729 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b1f27aab2d1cce51772c0ce99bd8dfc4afd0620b38b776e6b968d9114105f165\": container with ID starting with b1f27aab2d1cce51772c0ce99bd8dfc4afd0620b38b776e6b968d9114105f165 not found: ID does not exist" containerID="b1f27aab2d1cce51772c0ce99bd8dfc4afd0620b38b776e6b968d9114105f165" Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.590783 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b1f27aab2d1cce51772c0ce99bd8dfc4afd0620b38b776e6b968d9114105f165"} err="failed to get container status \"b1f27aab2d1cce51772c0ce99bd8dfc4afd0620b38b776e6b968d9114105f165\": rpc error: code = NotFound desc = could not find container \"b1f27aab2d1cce51772c0ce99bd8dfc4afd0620b38b776e6b968d9114105f165\": container with ID starting with b1f27aab2d1cce51772c0ce99bd8dfc4afd0620b38b776e6b968d9114105f165 not found: ID does not exist" Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.590814 5052 scope.go:117] "RemoveContainer" containerID="dfdff0317f61d3fa549a21a9c81b61a342d0bac7de867324e414cf592af4d97d" Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.599400 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gblmn\" (UniqueName: \"kubernetes.io/projected/0cb33a40-6f0a-4e89-a159-4edd3af3a2fa-kube-api-access-gblmn\") pod \"0cb33a40-6f0a-4e89-a159-4edd3af3a2fa\" (UID: \"0cb33a40-6f0a-4e89-a159-4edd3af3a2fa\") " Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.599461 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"pod-info\" (UniqueName: \"kubernetes.io/downward-api/0cb33a40-6f0a-4e89-a159-4edd3af3a2fa-pod-info\") pod \"0cb33a40-6f0a-4e89-a159-4edd3af3a2fa\" (UID: \"0cb33a40-6f0a-4e89-a159-4edd3af3a2fa\") " Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.599501 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/0cb33a40-6f0a-4e89-a159-4edd3af3a2fa-config-data\") pod \"0cb33a40-6f0a-4e89-a159-4edd3af3a2fa\" (UID: \"0cb33a40-6f0a-4e89-a159-4edd3af3a2fa\") " Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.599536 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"rabbitmq-confd\" (UniqueName: \"kubernetes.io/projected/0cb33a40-6f0a-4e89-a159-4edd3af3a2fa-rabbitmq-confd\") pod \"0cb33a40-6f0a-4e89-a159-4edd3af3a2fa\" (UID: \"0cb33a40-6f0a-4e89-a159-4edd3af3a2fa\") " Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.599595 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"erlang-cookie-secret\" (UniqueName: \"kubernetes.io/secret/0cb33a40-6f0a-4e89-a159-4edd3af3a2fa-erlang-cookie-secret\") pod \"0cb33a40-6f0a-4e89-a159-4edd3af3a2fa\" (UID: \"0cb33a40-6f0a-4e89-a159-4edd3af3a2fa\") " Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.599611 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"persistence\" (UniqueName: \"kubernetes.io/local-volume/local-storage14-crc\") pod \"0cb33a40-6f0a-4e89-a159-4edd3af3a2fa\" (UID: \"0cb33a40-6f0a-4e89-a159-4edd3af3a2fa\") " Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.599651 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"server-conf\" (UniqueName: \"kubernetes.io/configmap/0cb33a40-6f0a-4e89-a159-4edd3af3a2fa-server-conf\") pod \"0cb33a40-6f0a-4e89-a159-4edd3af3a2fa\" (UID: \"0cb33a40-6f0a-4e89-a159-4edd3af3a2fa\") " Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.599671 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"rabbitmq-plugins\" (UniqueName: \"kubernetes.io/empty-dir/0cb33a40-6f0a-4e89-a159-4edd3af3a2fa-rabbitmq-plugins\") pod \"0cb33a40-6f0a-4e89-a159-4edd3af3a2fa\" (UID: \"0cb33a40-6f0a-4e89-a159-4edd3af3a2fa\") " Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.599689 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"rabbitmq-tls\" (UniqueName: \"kubernetes.io/projected/0cb33a40-6f0a-4e89-a159-4edd3af3a2fa-rabbitmq-tls\") pod \"0cb33a40-6f0a-4e89-a159-4edd3af3a2fa\" (UID: \"0cb33a40-6f0a-4e89-a159-4edd3af3a2fa\") " Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.599708 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"rabbitmq-erlang-cookie\" (UniqueName: \"kubernetes.io/empty-dir/0cb33a40-6f0a-4e89-a159-4edd3af3a2fa-rabbitmq-erlang-cookie\") pod \"0cb33a40-6f0a-4e89-a159-4edd3af3a2fa\" (UID: \"0cb33a40-6f0a-4e89-a159-4edd3af3a2fa\") " Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.601321 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"plugins-conf\" (UniqueName: \"kubernetes.io/configmap/0cb33a40-6f0a-4e89-a159-4edd3af3a2fa-plugins-conf\") pod \"0cb33a40-6f0a-4e89-a159-4edd3af3a2fa\" (UID: \"0cb33a40-6f0a-4e89-a159-4edd3af3a2fa\") " Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.601806 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-66gvc\" (UniqueName: \"kubernetes.io/projected/8669832d-c47a-49bd-9491-dc7141ac6281-kube-api-access-66gvc\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.601829 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8669832d-c47a-49bd-9491-dc7141ac6281-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.603089 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0cb33a40-6f0a-4e89-a159-4edd3af3a2fa-plugins-conf" (OuterVolumeSpecName: "plugins-conf") pod "0cb33a40-6f0a-4e89-a159-4edd3af3a2fa" (UID: "0cb33a40-6f0a-4e89-a159-4edd3af3a2fa"). InnerVolumeSpecName "plugins-conf". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.604253 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0cb33a40-6f0a-4e89-a159-4edd3af3a2fa-rabbitmq-erlang-cookie" (OuterVolumeSpecName: "rabbitmq-erlang-cookie") pod "0cb33a40-6f0a-4e89-a159-4edd3af3a2fa" (UID: "0cb33a40-6f0a-4e89-a159-4edd3af3a2fa"). InnerVolumeSpecName "rabbitmq-erlang-cookie". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.604748 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0cb33a40-6f0a-4e89-a159-4edd3af3a2fa-rabbitmq-plugins" (OuterVolumeSpecName: "rabbitmq-plugins") pod "0cb33a40-6f0a-4e89-a159-4edd3af3a2fa" (UID: "0cb33a40-6f0a-4e89-a159-4edd3af3a2fa"). InnerVolumeSpecName "rabbitmq-plugins". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.606803 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0cb33a40-6f0a-4e89-a159-4edd3af3a2fa-pod-info" (OuterVolumeSpecName: "pod-info") pod "0cb33a40-6f0a-4e89-a159-4edd3af3a2fa" (UID: "0cb33a40-6f0a-4e89-a159-4edd3af3a2fa"). InnerVolumeSpecName "pod-info". PluginName "kubernetes.io/downward-api", VolumeGidValue "" Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.607148 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage14-crc" (OuterVolumeSpecName: "persistence") pod "0cb33a40-6f0a-4e89-a159-4edd3af3a2fa" (UID: "0cb33a40-6f0a-4e89-a159-4edd3af3a2fa"). InnerVolumeSpecName "local-storage14-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.607820 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0cb33a40-6f0a-4e89-a159-4edd3af3a2fa-rabbitmq-tls" (OuterVolumeSpecName: "rabbitmq-tls") pod "0cb33a40-6f0a-4e89-a159-4edd3af3a2fa" (UID: "0cb33a40-6f0a-4e89-a159-4edd3af3a2fa"). InnerVolumeSpecName "rabbitmq-tls". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.608613 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0cb33a40-6f0a-4e89-a159-4edd3af3a2fa-erlang-cookie-secret" (OuterVolumeSpecName: "erlang-cookie-secret") pod "0cb33a40-6f0a-4e89-a159-4edd3af3a2fa" (UID: "0cb33a40-6f0a-4e89-a159-4edd3af3a2fa"). InnerVolumeSpecName "erlang-cookie-secret". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.611223 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.616774 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0cb33a40-6f0a-4e89-a159-4edd3af3a2fa-kube-api-access-gblmn" (OuterVolumeSpecName: "kube-api-access-gblmn") pod "0cb33a40-6f0a-4e89-a159-4edd3af3a2fa" (UID: "0cb33a40-6f0a-4e89-a159-4edd3af3a2fa"). InnerVolumeSpecName "kube-api-access-gblmn". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.628718 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.646070 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glanced4a6-account-delete-69llz"] Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.656287 5052 scope.go:117] "RemoveContainer" containerID="fd397a6821f16ad2b0e97b5bbbb387d00aed4ab2619cd9d9f06f9bc10900138f" Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.661472 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/glanced4a6-account-delete-69llz"] Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.687340 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0cb33a40-6f0a-4e89-a159-4edd3af3a2fa-config-data" (OuterVolumeSpecName: "config-data") pod "0cb33a40-6f0a-4e89-a159-4edd3af3a2fa" (UID: "0cb33a40-6f0a-4e89-a159-4edd3af3a2fa"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.695871 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2846c397-2869-49f3-aeca-cf4b56bc8834" path="/var/lib/kubelet/pods/2846c397-2869-49f3-aeca-cf4b56bc8834/volumes" Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.700486 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3479f9ad-871f-463e-96ce-50f4069088b3" path="/var/lib/kubelet/pods/3479f9ad-871f-463e-96ce-50f4069088b3/volumes" Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.701541 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="456c3fb1-90c0-4d17-863e-414380759f97" path="/var/lib/kubelet/pods/456c3fb1-90c0-4d17-863e-414380759f97/volumes" Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.701934 5052 scope.go:117] "RemoveContainer" containerID="dfdff0317f61d3fa549a21a9c81b61a342d0bac7de867324e414cf592af4d97d" Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.702021 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6a5de7fb-b482-4849-a957-e679616aabc0" path="/var/lib/kubelet/pods/6a5de7fb-b482-4849-a957-e679616aabc0/volumes" Dec 03 12:49:07 crc kubenswrapper[5052]: E1203 12:49:07.702691 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dfdff0317f61d3fa549a21a9c81b61a342d0bac7de867324e414cf592af4d97d\": container with ID starting with dfdff0317f61d3fa549a21a9c81b61a342d0bac7de867324e414cf592af4d97d not found: ID does not exist" containerID="dfdff0317f61d3fa549a21a9c81b61a342d0bac7de867324e414cf592af4d97d" Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.702895 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dfdff0317f61d3fa549a21a9c81b61a342d0bac7de867324e414cf592af4d97d"} err="failed to get container status \"dfdff0317f61d3fa549a21a9c81b61a342d0bac7de867324e414cf592af4d97d\": rpc error: code = NotFound desc = could not find container \"dfdff0317f61d3fa549a21a9c81b61a342d0bac7de867324e414cf592af4d97d\": container with ID starting with dfdff0317f61d3fa549a21a9c81b61a342d0bac7de867324e414cf592af4d97d not found: ID does not exist" Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.702916 5052 scope.go:117] "RemoveContainer" containerID="fd397a6821f16ad2b0e97b5bbbb387d00aed4ab2619cd9d9f06f9bc10900138f" Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.703044 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-gblmn\" (UniqueName: \"kubernetes.io/projected/0cb33a40-6f0a-4e89-a159-4edd3af3a2fa-kube-api-access-gblmn\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.703225 5052 reconciler_common.go:293] "Volume detached for volume \"pod-info\" (UniqueName: \"kubernetes.io/downward-api/0cb33a40-6f0a-4e89-a159-4edd3af3a2fa-pod-info\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.703245 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/0cb33a40-6f0a-4e89-a159-4edd3af3a2fa-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.703255 5052 reconciler_common.go:293] "Volume detached for volume \"erlang-cookie-secret\" (UniqueName: \"kubernetes.io/secret/0cb33a40-6f0a-4e89-a159-4edd3af3a2fa-erlang-cookie-secret\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.703281 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage14-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage14-crc\") on node \"crc\" " Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.703296 5052 reconciler_common.go:293] "Volume detached for volume \"rabbitmq-plugins\" (UniqueName: \"kubernetes.io/empty-dir/0cb33a40-6f0a-4e89-a159-4edd3af3a2fa-rabbitmq-plugins\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.703307 5052 reconciler_common.go:293] "Volume detached for volume \"rabbitmq-tls\" (UniqueName: \"kubernetes.io/projected/0cb33a40-6f0a-4e89-a159-4edd3af3a2fa-rabbitmq-tls\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.703317 5052 reconciler_common.go:293] "Volume detached for volume \"rabbitmq-erlang-cookie\" (UniqueName: \"kubernetes.io/empty-dir/0cb33a40-6f0a-4e89-a159-4edd3af3a2fa-rabbitmq-erlang-cookie\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.703325 5052 reconciler_common.go:293] "Volume detached for volume \"plugins-conf\" (UniqueName: \"kubernetes.io/configmap/0cb33a40-6f0a-4e89-a159-4edd3af3a2fa-plugins-conf\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.703342 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6d653682-52e2-4940-af66-633475bfe09a" path="/var/lib/kubelet/pods/6d653682-52e2-4940-af66-633475bfe09a/volumes" Dec 03 12:49:07 crc kubenswrapper[5052]: E1203 12:49:07.703381 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fd397a6821f16ad2b0e97b5bbbb387d00aed4ab2619cd9d9f06f9bc10900138f\": container with ID starting with fd397a6821f16ad2b0e97b5bbbb387d00aed4ab2619cd9d9f06f9bc10900138f not found: ID does not exist" containerID="fd397a6821f16ad2b0e97b5bbbb387d00aed4ab2619cd9d9f06f9bc10900138f" Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.703414 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fd397a6821f16ad2b0e97b5bbbb387d00aed4ab2619cd9d9f06f9bc10900138f"} err="failed to get container status \"fd397a6821f16ad2b0e97b5bbbb387d00aed4ab2619cd9d9f06f9bc10900138f\": rpc error: code = NotFound desc = could not find container \"fd397a6821f16ad2b0e97b5bbbb387d00aed4ab2619cd9d9f06f9bc10900138f\": container with ID starting with fd397a6821f16ad2b0e97b5bbbb387d00aed4ab2619cd9d9f06f9bc10900138f not found: ID does not exist" Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.703444 5052 scope.go:117] "RemoveContainer" containerID="4a37bb95201a9ab5a64daccd9272263816dd9e817b0668c7be7e63d522a8fabe" Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.704156 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="81709983-1fd0-4446-bbbf-fb94653bb3d2" path="/var/lib/kubelet/pods/81709983-1fd0-4446-bbbf-fb94653bb3d2/volumes" Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.706296 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b8eb690c-165a-4906-a750-bec0178daf6c" path="/var/lib/kubelet/pods/b8eb690c-165a-4906-a750-bec0178daf6c/volumes" Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.706921 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e14465ed-868a-49ba-b3ab-1e67196229e8" path="/var/lib/kubelet/pods/e14465ed-868a-49ba-b3ab-1e67196229e8/volumes" Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.711648 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eef9c304-b598-4a31-9c47-09be0087c25e" path="/var/lib/kubelet/pods/eef9c304-b598-4a31-9c47-09be0087c25e/volumes" Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.714138 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="efb0e60c-2a71-4641-87b2-0f5532b28678" path="/var/lib/kubelet/pods/efb0e60c-2a71-4641-87b2-0f5532b28678/volumes" Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.728179 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0cb33a40-6f0a-4e89-a159-4edd3af3a2fa-server-conf" (OuterVolumeSpecName: "server-conf") pod "0cb33a40-6f0a-4e89-a159-4edd3af3a2fa" (UID: "0cb33a40-6f0a-4e89-a159-4edd3af3a2fa"). InnerVolumeSpecName "server-conf". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.733234 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage14-crc" (UniqueName: "kubernetes.io/local-volume/local-storage14-crc") on node "crc" Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.741332 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/novacell05566-account-delete-2f4dm"] Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.741369 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/novacell05566-account-delete-2f4dm"] Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.741386 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/rabbitmq-cell1-server-0"] Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.741399 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/rabbitmq-cell1-server-0"] Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.741413 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.741428 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.741440 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.749163 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.749280 5052 scope.go:117] "RemoveContainer" containerID="3b37742fa0a9e71beb41d34a9311dc58c88f7bd963804764ab51a685adf84ccf" Dec 03 12:49:07 crc kubenswrapper[5052]: E1203 12:49:07.749857 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3b37742fa0a9e71beb41d34a9311dc58c88f7bd963804764ab51a685adf84ccf\": container with ID starting with 3b37742fa0a9e71beb41d34a9311dc58c88f7bd963804764ab51a685adf84ccf not found: ID does not exist" containerID="3b37742fa0a9e71beb41d34a9311dc58c88f7bd963804764ab51a685adf84ccf" Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.749952 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3b37742fa0a9e71beb41d34a9311dc58c88f7bd963804764ab51a685adf84ccf"} err="failed to get container status \"3b37742fa0a9e71beb41d34a9311dc58c88f7bd963804764ab51a685adf84ccf\": rpc error: code = NotFound desc = could not find container \"3b37742fa0a9e71beb41d34a9311dc58c88f7bd963804764ab51a685adf84ccf\": container with ID starting with 3b37742fa0a9e71beb41d34a9311dc58c88f7bd963804764ab51a685adf84ccf not found: ID does not exist" Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.750042 5052 scope.go:117] "RemoveContainer" containerID="39fed73e7301681d3f192722739dd4c2c192c9bb0beb359054ef92a552914767" Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.768859 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0cb33a40-6f0a-4e89-a159-4edd3af3a2fa-rabbitmq-confd" (OuterVolumeSpecName: "rabbitmq-confd") pod "0cb33a40-6f0a-4e89-a159-4edd3af3a2fa" (UID: "0cb33a40-6f0a-4e89-a159-4edd3af3a2fa"). InnerVolumeSpecName "rabbitmq-confd". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.789718 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/keystone-5f884bdbfd-2xzz4"] Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.799672 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/keystone-5f884bdbfd-2xzz4"] Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.803350 5052 scope.go:117] "RemoveContainer" containerID="747dff2aa2b4006f1eacd1565f1e19dcab8b056e4b9a569dfe7735eef649cbe8" Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.805109 5052 reconciler_common.go:293] "Volume detached for volume \"rabbitmq-confd\" (UniqueName: \"kubernetes.io/projected/0cb33a40-6f0a-4e89-a159-4edd3af3a2fa-rabbitmq-confd\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.805131 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage14-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage14-crc\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.805143 5052 reconciler_common.go:293] "Volume detached for volume \"server-conf\" (UniqueName: \"kubernetes.io/configmap/0cb33a40-6f0a-4e89-a159-4edd3af3a2fa-server-conf\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:07 crc kubenswrapper[5052]: E1203 12:49:07.807145 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"747dff2aa2b4006f1eacd1565f1e19dcab8b056e4b9a569dfe7735eef649cbe8\": container with ID starting with 747dff2aa2b4006f1eacd1565f1e19dcab8b056e4b9a569dfe7735eef649cbe8 not found: ID does not exist" containerID="747dff2aa2b4006f1eacd1565f1e19dcab8b056e4b9a569dfe7735eef649cbe8" Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.807185 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"747dff2aa2b4006f1eacd1565f1e19dcab8b056e4b9a569dfe7735eef649cbe8"} err="failed to get container status \"747dff2aa2b4006f1eacd1565f1e19dcab8b056e4b9a569dfe7735eef649cbe8\": rpc error: code = NotFound desc = could not find container \"747dff2aa2b4006f1eacd1565f1e19dcab8b056e4b9a569dfe7735eef649cbe8\": container with ID starting with 747dff2aa2b4006f1eacd1565f1e19dcab8b056e4b9a569dfe7735eef649cbe8 not found: ID does not exist" Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.807211 5052 scope.go:117] "RemoveContainer" containerID="032892e966b8241f925ecec443072f0ec039107983eec9fb7785055fdd438546" Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.812787 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/neutrona25a-account-delete-vbr24"] Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.831641 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/neutrona25a-account-delete-vbr24"] Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.836860 5052 scope.go:117] "RemoveContainer" containerID="3174df2d1d1a71d59d7128750feffcb2735c01240b7a706e2b5e41abf792ee9e" Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.838191 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/neutron-fdd8f875d-xvq79"] Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.846767 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/neutron-fdd8f875d-xvq79"] Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.856654 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.865902 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.866070 5052 scope.go:117] "RemoveContainer" containerID="e561d2853dbef5377ec18e773746c93147e4ebef4cf2a400d67d09af44a67531" Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.910189 5052 generic.go:334] "Generic (PLEG): container finished" podID="0cb33a40-6f0a-4e89-a159-4edd3af3a2fa" containerID="e4371252da37cb77d701ae89da98daa7bb376b68ff8df323b2456a8b5876c5ba" exitCode=0 Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.910290 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-server-0" event={"ID":"0cb33a40-6f0a-4e89-a159-4edd3af3a2fa","Type":"ContainerDied","Data":"e4371252da37cb77d701ae89da98daa7bb376b68ff8df323b2456a8b5876c5ba"} Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.910328 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-server-0" event={"ID":"0cb33a40-6f0a-4e89-a159-4edd3af3a2fa","Type":"ContainerDied","Data":"649e9b73822db914ddce53841f5f090c49bb4102dea8d827fabb769084e11db2"} Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.910352 5052 scope.go:117] "RemoveContainer" containerID="e4371252da37cb77d701ae89da98daa7bb376b68ff8df323b2456a8b5876c5ba" Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.910354 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.914236 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-keystone-listener-68fd6d997d-7tjjw" Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.916379 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder4345-account-delete-vmdzh" event={"ID":"5d11b02e-c14a-4d1a-819a-4c3b66aedd33","Type":"ContainerDied","Data":"edd916e2f697136555c97bf7187eabf8e7e3c3a04c9c08fec4f86f0bc3ca9a5c"} Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.916465 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder4345-account-delete-vmdzh" Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.925457 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-galera-0" event={"ID":"3c8e4e3f-f6c1-4362-b64c-b10a973723dd","Type":"ContainerDied","Data":"18d688987807d911f1579610334ad2972251071bd9402179512a647c3ef36c10"} Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.925652 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.929687 5052 generic.go:334] "Generic (PLEG): container finished" podID="8669832d-c47a-49bd-9491-dc7141ac6281" containerID="caf28df7281c196a4c4d326b47a9f4c24851d6defc41fd8fa20f0e5325c7cdac" exitCode=0 Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.929737 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-conductor-0" event={"ID":"8669832d-c47a-49bd-9491-dc7141ac6281","Type":"ContainerDied","Data":"caf28df7281c196a4c4d326b47a9f4c24851d6defc41fd8fa20f0e5325c7cdac"} Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.929760 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-conductor-0" event={"ID":"8669832d-c47a-49bd-9491-dc7141ac6281","Type":"ContainerDied","Data":"71aa97adfbba84a05616bbe88826b6bf5072c85c532f501d3d903dd85d92b0bb"} Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.929828 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.936390 5052 scope.go:117] "RemoveContainer" containerID="40637a2936ebcaa4c4b798867fbd94260367dfcf9f69160a782dfc1f71402096" Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.943967 5052 generic.go:334] "Generic (PLEG): container finished" podID="2065f5bb-88eb-4e84-88b6-9982833d413d" containerID="92eeaf97de94d65f0f332a8172dd776af0421be37c42d90e9dab8739d2cb83e3" exitCode=0 Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.944038 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-keystone-listener-68fd6d997d-7tjjw" Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.944056 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-keystone-listener-68fd6d997d-7tjjw" event={"ID":"2065f5bb-88eb-4e84-88b6-9982833d413d","Type":"ContainerDied","Data":"92eeaf97de94d65f0f332a8172dd776af0421be37c42d90e9dab8739d2cb83e3"} Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.944080 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-keystone-listener-68fd6d997d-7tjjw" event={"ID":"2065f5bb-88eb-4e84-88b6-9982833d413d","Type":"ContainerDied","Data":"d9800cff12dbc4c6cb5054864b4a94efa21b1761b5e25554829174d9dbd0716f"} Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.958236 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_ovn-northd-0_342cba80-22e1-488a-bbfb-75b67ca536ac/ovn-northd/0.log" Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.958428 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.958510 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovn-northd-0" event={"ID":"342cba80-22e1-488a-bbfb-75b67ca536ac","Type":"ContainerDied","Data":"ce49b1f529433537dacac11dc47b93030dd9670b6a41b713c883a09a1fe19146"} Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.960645 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/rabbitmq-server-0"] Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.962255 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican02cc-account-delete-gf94b" event={"ID":"857b8a8a-d730-4832-a71c-d45a2f3ce406","Type":"ContainerDied","Data":"823c9ad272bf683331ba7b76dc06ffdad3a303ada7a89dddc6c6456b71e08dd0"} Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.962394 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican02cc-account-delete-gf94b" Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.971320 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/rabbitmq-server-0"] Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.984890 5052 scope.go:117] "RemoveContainer" containerID="e4371252da37cb77d701ae89da98daa7bb376b68ff8df323b2456a8b5876c5ba" Dec 03 12:49:07 crc kubenswrapper[5052]: E1203 12:49:07.985440 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e4371252da37cb77d701ae89da98daa7bb376b68ff8df323b2456a8b5876c5ba\": container with ID starting with e4371252da37cb77d701ae89da98daa7bb376b68ff8df323b2456a8b5876c5ba not found: ID does not exist" containerID="e4371252da37cb77d701ae89da98daa7bb376b68ff8df323b2456a8b5876c5ba" Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.985538 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e4371252da37cb77d701ae89da98daa7bb376b68ff8df323b2456a8b5876c5ba"} err="failed to get container status \"e4371252da37cb77d701ae89da98daa7bb376b68ff8df323b2456a8b5876c5ba\": rpc error: code = NotFound desc = could not find container \"e4371252da37cb77d701ae89da98daa7bb376b68ff8df323b2456a8b5876c5ba\": container with ID starting with e4371252da37cb77d701ae89da98daa7bb376b68ff8df323b2456a8b5876c5ba not found: ID does not exist" Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.985619 5052 scope.go:117] "RemoveContainer" containerID="40637a2936ebcaa4c4b798867fbd94260367dfcf9f69160a782dfc1f71402096" Dec 03 12:49:07 crc kubenswrapper[5052]: E1203 12:49:07.985913 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"40637a2936ebcaa4c4b798867fbd94260367dfcf9f69160a782dfc1f71402096\": container with ID starting with 40637a2936ebcaa4c4b798867fbd94260367dfcf9f69160a782dfc1f71402096 not found: ID does not exist" containerID="40637a2936ebcaa4c4b798867fbd94260367dfcf9f69160a782dfc1f71402096" Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.985984 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"40637a2936ebcaa4c4b798867fbd94260367dfcf9f69160a782dfc1f71402096"} err="failed to get container status \"40637a2936ebcaa4c4b798867fbd94260367dfcf9f69160a782dfc1f71402096\": rpc error: code = NotFound desc = could not find container \"40637a2936ebcaa4c4b798867fbd94260367dfcf9f69160a782dfc1f71402096\": container with ID starting with 40637a2936ebcaa4c4b798867fbd94260367dfcf9f69160a782dfc1f71402096 not found: ID does not exist" Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.986045 5052 scope.go:117] "RemoveContainer" containerID="573a090ed2c4fc25a35dfd0460dc811780dbcc0a3ea284429ddf8664717e53ef" Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.989293 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder4345-account-delete-vmdzh"] Dec 03 12:49:07 crc kubenswrapper[5052]: I1203 12:49:07.998027 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/cinder4345-account-delete-vmdzh"] Dec 03 12:49:08 crc kubenswrapper[5052]: I1203 12:49:08.006762 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bswmw\" (UniqueName: \"kubernetes.io/projected/2065f5bb-88eb-4e84-88b6-9982833d413d-kube-api-access-bswmw\") pod \"2065f5bb-88eb-4e84-88b6-9982833d413d\" (UID: \"2065f5bb-88eb-4e84-88b6-9982833d413d\") " Dec 03 12:49:08 crc kubenswrapper[5052]: I1203 12:49:08.007203 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/2065f5bb-88eb-4e84-88b6-9982833d413d-config-data-custom\") pod \"2065f5bb-88eb-4e84-88b6-9982833d413d\" (UID: \"2065f5bb-88eb-4e84-88b6-9982833d413d\") " Dec 03 12:49:08 crc kubenswrapper[5052]: I1203 12:49:08.007322 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2065f5bb-88eb-4e84-88b6-9982833d413d-combined-ca-bundle\") pod \"2065f5bb-88eb-4e84-88b6-9982833d413d\" (UID: \"2065f5bb-88eb-4e84-88b6-9982833d413d\") " Dec 03 12:49:08 crc kubenswrapper[5052]: I1203 12:49:08.007673 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/2065f5bb-88eb-4e84-88b6-9982833d413d-logs\") pod \"2065f5bb-88eb-4e84-88b6-9982833d413d\" (UID: \"2065f5bb-88eb-4e84-88b6-9982833d413d\") " Dec 03 12:49:08 crc kubenswrapper[5052]: I1203 12:49:08.007762 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/2065f5bb-88eb-4e84-88b6-9982833d413d-config-data\") pod \"2065f5bb-88eb-4e84-88b6-9982833d413d\" (UID: \"2065f5bb-88eb-4e84-88b6-9982833d413d\") " Dec 03 12:49:08 crc kubenswrapper[5052]: I1203 12:49:08.008494 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2065f5bb-88eb-4e84-88b6-9982833d413d-logs" (OuterVolumeSpecName: "logs") pod "2065f5bb-88eb-4e84-88b6-9982833d413d" (UID: "2065f5bb-88eb-4e84-88b6-9982833d413d"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:49:08 crc kubenswrapper[5052]: I1203 12:49:08.010741 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2065f5bb-88eb-4e84-88b6-9982833d413d-config-data-custom" (OuterVolumeSpecName: "config-data-custom") pod "2065f5bb-88eb-4e84-88b6-9982833d413d" (UID: "2065f5bb-88eb-4e84-88b6-9982833d413d"). InnerVolumeSpecName "config-data-custom". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:49:08 crc kubenswrapper[5052]: I1203 12:49:08.011679 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2065f5bb-88eb-4e84-88b6-9982833d413d-kube-api-access-bswmw" (OuterVolumeSpecName: "kube-api-access-bswmw") pod "2065f5bb-88eb-4e84-88b6-9982833d413d" (UID: "2065f5bb-88eb-4e84-88b6-9982833d413d"). InnerVolumeSpecName "kube-api-access-bswmw". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:49:08 crc kubenswrapper[5052]: I1203 12:49:08.011748 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican02cc-account-delete-gf94b"] Dec 03 12:49:08 crc kubenswrapper[5052]: I1203 12:49:08.022066 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/barbican02cc-account-delete-gf94b"] Dec 03 12:49:08 crc kubenswrapper[5052]: I1203 12:49:08.034840 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ovn-northd-0"] Dec 03 12:49:08 crc kubenswrapper[5052]: I1203 12:49:08.043354 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ovn-northd-0"] Dec 03 12:49:08 crc kubenswrapper[5052]: I1203 12:49:08.050746 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/openstack-galera-0"] Dec 03 12:49:08 crc kubenswrapper[5052]: I1203 12:49:08.056881 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/openstack-galera-0"] Dec 03 12:49:08 crc kubenswrapper[5052]: I1203 12:49:08.066463 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2065f5bb-88eb-4e84-88b6-9982833d413d-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "2065f5bb-88eb-4e84-88b6-9982833d413d" (UID: "2065f5bb-88eb-4e84-88b6-9982833d413d"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:49:08 crc kubenswrapper[5052]: I1203 12:49:08.066645 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2065f5bb-88eb-4e84-88b6-9982833d413d-config-data" (OuterVolumeSpecName: "config-data") pod "2065f5bb-88eb-4e84-88b6-9982833d413d" (UID: "2065f5bb-88eb-4e84-88b6-9982833d413d"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:49:08 crc kubenswrapper[5052]: I1203 12:49:08.109681 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/8669832d-c47a-49bd-9491-dc7141ac6281-config-data\") pod \"8669832d-c47a-49bd-9491-dc7141ac6281\" (UID: \"8669832d-c47a-49bd-9491-dc7141ac6281\") " Dec 03 12:49:08 crc kubenswrapper[5052]: I1203 12:49:08.110084 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-bswmw\" (UniqueName: \"kubernetes.io/projected/2065f5bb-88eb-4e84-88b6-9982833d413d-kube-api-access-bswmw\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:08 crc kubenswrapper[5052]: I1203 12:49:08.110106 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/2065f5bb-88eb-4e84-88b6-9982833d413d-config-data-custom\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:08 crc kubenswrapper[5052]: I1203 12:49:08.110117 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2065f5bb-88eb-4e84-88b6-9982833d413d-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:08 crc kubenswrapper[5052]: I1203 12:49:08.110127 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/2065f5bb-88eb-4e84-88b6-9982833d413d-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:08 crc kubenswrapper[5052]: I1203 12:49:08.110137 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/2065f5bb-88eb-4e84-88b6-9982833d413d-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:08 crc kubenswrapper[5052]: I1203 12:49:08.113378 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8669832d-c47a-49bd-9491-dc7141ac6281-config-data" (OuterVolumeSpecName: "config-data") pod "8669832d-c47a-49bd-9491-dc7141ac6281" (UID: "8669832d-c47a-49bd-9491-dc7141ac6281"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:49:08 crc kubenswrapper[5052]: I1203 12:49:08.163999 5052 scope.go:117] "RemoveContainer" containerID="86f66b1381c35478a52f56b7c8ce2c98cdc37bae9cd163bbea7b6f45c6c705b9" Dec 03 12:49:08 crc kubenswrapper[5052]: I1203 12:49:08.194530 5052 scope.go:117] "RemoveContainer" containerID="4ba4f75a72e1d824adaefd96f9adda9fa4cebaac7aadd0b439e99a73bbb29f1e" Dec 03 12:49:08 crc kubenswrapper[5052]: I1203 12:49:08.211596 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/8669832d-c47a-49bd-9491-dc7141ac6281-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:08 crc kubenswrapper[5052]: I1203 12:49:08.220231 5052 scope.go:117] "RemoveContainer" containerID="caf28df7281c196a4c4d326b47a9f4c24851d6defc41fd8fa20f0e5325c7cdac" Dec 03 12:49:08 crc kubenswrapper[5052]: I1203 12:49:08.245417 5052 scope.go:117] "RemoveContainer" containerID="8e3764a05c784a75a82ccff0fcb9f4aaac35f5d6845cca81b14deee31318466f" Dec 03 12:49:08 crc kubenswrapper[5052]: I1203 12:49:08.272006 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-0"] Dec 03 12:49:08 crc kubenswrapper[5052]: I1203 12:49:08.290044 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-0"] Dec 03 12:49:08 crc kubenswrapper[5052]: I1203 12:49:08.294096 5052 scope.go:117] "RemoveContainer" containerID="caf28df7281c196a4c4d326b47a9f4c24851d6defc41fd8fa20f0e5325c7cdac" Dec 03 12:49:08 crc kubenswrapper[5052]: E1203 12:49:08.294787 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"caf28df7281c196a4c4d326b47a9f4c24851d6defc41fd8fa20f0e5325c7cdac\": container with ID starting with caf28df7281c196a4c4d326b47a9f4c24851d6defc41fd8fa20f0e5325c7cdac not found: ID does not exist" containerID="caf28df7281c196a4c4d326b47a9f4c24851d6defc41fd8fa20f0e5325c7cdac" Dec 03 12:49:08 crc kubenswrapper[5052]: I1203 12:49:08.294815 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"caf28df7281c196a4c4d326b47a9f4c24851d6defc41fd8fa20f0e5325c7cdac"} err="failed to get container status \"caf28df7281c196a4c4d326b47a9f4c24851d6defc41fd8fa20f0e5325c7cdac\": rpc error: code = NotFound desc = could not find container \"caf28df7281c196a4c4d326b47a9f4c24851d6defc41fd8fa20f0e5325c7cdac\": container with ID starting with caf28df7281c196a4c4d326b47a9f4c24851d6defc41fd8fa20f0e5325c7cdac not found: ID does not exist" Dec 03 12:49:08 crc kubenswrapper[5052]: I1203 12:49:08.294835 5052 scope.go:117] "RemoveContainer" containerID="8e3764a05c784a75a82ccff0fcb9f4aaac35f5d6845cca81b14deee31318466f" Dec 03 12:49:08 crc kubenswrapper[5052]: E1203 12:49:08.295129 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8e3764a05c784a75a82ccff0fcb9f4aaac35f5d6845cca81b14deee31318466f\": container with ID starting with 8e3764a05c784a75a82ccff0fcb9f4aaac35f5d6845cca81b14deee31318466f not found: ID does not exist" containerID="8e3764a05c784a75a82ccff0fcb9f4aaac35f5d6845cca81b14deee31318466f" Dec 03 12:49:08 crc kubenswrapper[5052]: I1203 12:49:08.295155 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8e3764a05c784a75a82ccff0fcb9f4aaac35f5d6845cca81b14deee31318466f"} err="failed to get container status \"8e3764a05c784a75a82ccff0fcb9f4aaac35f5d6845cca81b14deee31318466f\": rpc error: code = NotFound desc = could not find container \"8e3764a05c784a75a82ccff0fcb9f4aaac35f5d6845cca81b14deee31318466f\": container with ID starting with 8e3764a05c784a75a82ccff0fcb9f4aaac35f5d6845cca81b14deee31318466f not found: ID does not exist" Dec 03 12:49:08 crc kubenswrapper[5052]: I1203 12:49:08.295171 5052 scope.go:117] "RemoveContainer" containerID="92eeaf97de94d65f0f332a8172dd776af0421be37c42d90e9dab8739d2cb83e3" Dec 03 12:49:08 crc kubenswrapper[5052]: I1203 12:49:08.301042 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-keystone-listener-68fd6d997d-7tjjw"] Dec 03 12:49:08 crc kubenswrapper[5052]: I1203 12:49:08.307839 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/barbican-keystone-listener-68fd6d997d-7tjjw"] Dec 03 12:49:08 crc kubenswrapper[5052]: I1203 12:49:08.320492 5052 scope.go:117] "RemoveContainer" containerID="2e1db89e52b83cc4d03ffdc49b1a121ba636bb18c0424300ea015664b45cff9a" Dec 03 12:49:08 crc kubenswrapper[5052]: I1203 12:49:08.341113 5052 scope.go:117] "RemoveContainer" containerID="92eeaf97de94d65f0f332a8172dd776af0421be37c42d90e9dab8739d2cb83e3" Dec 03 12:49:08 crc kubenswrapper[5052]: E1203 12:49:08.341782 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"92eeaf97de94d65f0f332a8172dd776af0421be37c42d90e9dab8739d2cb83e3\": container with ID starting with 92eeaf97de94d65f0f332a8172dd776af0421be37c42d90e9dab8739d2cb83e3 not found: ID does not exist" containerID="92eeaf97de94d65f0f332a8172dd776af0421be37c42d90e9dab8739d2cb83e3" Dec 03 12:49:08 crc kubenswrapper[5052]: I1203 12:49:08.341843 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"92eeaf97de94d65f0f332a8172dd776af0421be37c42d90e9dab8739d2cb83e3"} err="failed to get container status \"92eeaf97de94d65f0f332a8172dd776af0421be37c42d90e9dab8739d2cb83e3\": rpc error: code = NotFound desc = could not find container \"92eeaf97de94d65f0f332a8172dd776af0421be37c42d90e9dab8739d2cb83e3\": container with ID starting with 92eeaf97de94d65f0f332a8172dd776af0421be37c42d90e9dab8739d2cb83e3 not found: ID does not exist" Dec 03 12:49:08 crc kubenswrapper[5052]: I1203 12:49:08.341876 5052 scope.go:117] "RemoveContainer" containerID="2e1db89e52b83cc4d03ffdc49b1a121ba636bb18c0424300ea015664b45cff9a" Dec 03 12:49:08 crc kubenswrapper[5052]: E1203 12:49:08.342311 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2e1db89e52b83cc4d03ffdc49b1a121ba636bb18c0424300ea015664b45cff9a\": container with ID starting with 2e1db89e52b83cc4d03ffdc49b1a121ba636bb18c0424300ea015664b45cff9a not found: ID does not exist" containerID="2e1db89e52b83cc4d03ffdc49b1a121ba636bb18c0424300ea015664b45cff9a" Dec 03 12:49:08 crc kubenswrapper[5052]: I1203 12:49:08.342336 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2e1db89e52b83cc4d03ffdc49b1a121ba636bb18c0424300ea015664b45cff9a"} err="failed to get container status \"2e1db89e52b83cc4d03ffdc49b1a121ba636bb18c0424300ea015664b45cff9a\": rpc error: code = NotFound desc = could not find container \"2e1db89e52b83cc4d03ffdc49b1a121ba636bb18c0424300ea015664b45cff9a\": container with ID starting with 2e1db89e52b83cc4d03ffdc49b1a121ba636bb18c0424300ea015664b45cff9a not found: ID does not exist" Dec 03 12:49:08 crc kubenswrapper[5052]: I1203 12:49:08.342349 5052 scope.go:117] "RemoveContainer" containerID="8df9cc82a4e4e7032e7155d5c8c0fda7507b5b3d7b99882037f3bf0a8995f1d7" Dec 03 12:49:08 crc kubenswrapper[5052]: I1203 12:49:08.365950 5052 scope.go:117] "RemoveContainer" containerID="6b272b57a393df881b0806ef34c9c3a75d64d13721c0eab27f38fa3374aed7ed" Dec 03 12:49:08 crc kubenswrapper[5052]: I1203 12:49:08.388646 5052 scope.go:117] "RemoveContainer" containerID="996f7904199a66ff3248f13d4d3a39fb7215e6c68ba1d6d6f445aebc18499890" Dec 03 12:49:09 crc kubenswrapper[5052]: I1203 12:49:09.709376 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="098a46f8-2d49-4fea-a95c-5678601aa164" path="/var/lib/kubelet/pods/098a46f8-2d49-4fea-a95c-5678601aa164/volumes" Dec 03 12:49:09 crc kubenswrapper[5052]: I1203 12:49:09.710485 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0cb33a40-6f0a-4e89-a159-4edd3af3a2fa" path="/var/lib/kubelet/pods/0cb33a40-6f0a-4e89-a159-4edd3af3a2fa/volumes" Dec 03 12:49:09 crc kubenswrapper[5052]: I1203 12:49:09.711676 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2065f5bb-88eb-4e84-88b6-9982833d413d" path="/var/lib/kubelet/pods/2065f5bb-88eb-4e84-88b6-9982833d413d/volumes" Dec 03 12:49:09 crc kubenswrapper[5052]: I1203 12:49:09.712477 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="342cba80-22e1-488a-bbfb-75b67ca536ac" path="/var/lib/kubelet/pods/342cba80-22e1-488a-bbfb-75b67ca536ac/volumes" Dec 03 12:49:09 crc kubenswrapper[5052]: I1203 12:49:09.713291 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3c8e4e3f-f6c1-4362-b64c-b10a973723dd" path="/var/lib/kubelet/pods/3c8e4e3f-f6c1-4362-b64c-b10a973723dd/volumes" Dec 03 12:49:09 crc kubenswrapper[5052]: I1203 12:49:09.714686 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="488c3529-4c2b-47c0-9dcf-05a064c34d0d" path="/var/lib/kubelet/pods/488c3529-4c2b-47c0-9dcf-05a064c34d0d/volumes" Dec 03 12:49:09 crc kubenswrapper[5052]: I1203 12:49:09.715290 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5d11b02e-c14a-4d1a-819a-4c3b66aedd33" path="/var/lib/kubelet/pods/5d11b02e-c14a-4d1a-819a-4c3b66aedd33/volumes" Dec 03 12:49:09 crc kubenswrapper[5052]: I1203 12:49:09.716203 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6c8ad676-e7dd-479c-8d22-f546dc5e50af" path="/var/lib/kubelet/pods/6c8ad676-e7dd-479c-8d22-f546dc5e50af/volumes" Dec 03 12:49:09 crc kubenswrapper[5052]: I1203 12:49:09.716773 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6ceba96c-7e86-4eb2-afa7-444f0fcc27ce" path="/var/lib/kubelet/pods/6ceba96c-7e86-4eb2-afa7-444f0fcc27ce/volumes" Dec 03 12:49:09 crc kubenswrapper[5052]: I1203 12:49:09.717294 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="77825998-48a8-496e-a523-8e0c371fe532" path="/var/lib/kubelet/pods/77825998-48a8-496e-a523-8e0c371fe532/volumes" Dec 03 12:49:09 crc kubenswrapper[5052]: I1203 12:49:09.718404 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="857b8a8a-d730-4832-a71c-d45a2f3ce406" path="/var/lib/kubelet/pods/857b8a8a-d730-4832-a71c-d45a2f3ce406/volumes" Dec 03 12:49:09 crc kubenswrapper[5052]: I1203 12:49:09.719005 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8669832d-c47a-49bd-9491-dc7141ac6281" path="/var/lib/kubelet/pods/8669832d-c47a-49bd-9491-dc7141ac6281/volumes" Dec 03 12:49:09 crc kubenswrapper[5052]: I1203 12:49:09.719703 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a2bc501c-ebef-4e68-8ae2-e8c4217cadff" path="/var/lib/kubelet/pods/a2bc501c-ebef-4e68-8ae2-e8c4217cadff/volumes" Dec 03 12:49:09 crc kubenswrapper[5052]: I1203 12:49:09.720723 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a69bfb18-1267-4383-b51a-5e0eba70d1bc" path="/var/lib/kubelet/pods/a69bfb18-1267-4383-b51a-5e0eba70d1bc/volumes" Dec 03 12:49:09 crc kubenswrapper[5052]: I1203 12:49:09.721385 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bff27172-5e30-474d-a49c-389a35dd3052" path="/var/lib/kubelet/pods/bff27172-5e30-474d-a49c-389a35dd3052/volumes" Dec 03 12:49:10 crc kubenswrapper[5052]: I1203 12:49:10.649625 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/nova-metadata-0" podUID="efb0e60c-2a71-4641-87b2-0f5532b28678" containerName="nova-metadata-metadata" probeResult="failure" output="Get \"https://10.217.0.119:8775/\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Dec 03 12:49:10 crc kubenswrapper[5052]: I1203 12:49:10.650384 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/nova-metadata-0" podUID="efb0e60c-2a71-4641-87b2-0f5532b28678" containerName="nova-metadata-log" probeResult="failure" output="Get \"https://10.217.0.119:8775/\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Dec 03 12:49:15 crc kubenswrapper[5052]: I1203 12:49:15.832658 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-9t4zp" Dec 03 12:49:15 crc kubenswrapper[5052]: I1203 12:49:15.881304 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-8554887475-6pl7l"] Dec 03 12:49:15 crc kubenswrapper[5052]: I1203 12:49:15.881551 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-8554887475-6pl7l" podUID="ee63b3ae-44c4-4225-8373-75f5950af9d5" containerName="dnsmasq-dns" containerID="cri-o://aad48c211b1075e99f24229e8da89586a407d1f73f3bdf7e13056ec73c5db24c" gracePeriod=10 Dec 03 12:49:16 crc kubenswrapper[5052]: I1203 12:49:16.050209 5052 generic.go:334] "Generic (PLEG): container finished" podID="ee63b3ae-44c4-4225-8373-75f5950af9d5" containerID="aad48c211b1075e99f24229e8da89586a407d1f73f3bdf7e13056ec73c5db24c" exitCode=0 Dec 03 12:49:16 crc kubenswrapper[5052]: I1203 12:49:16.050284 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-8554887475-6pl7l" event={"ID":"ee63b3ae-44c4-4225-8373-75f5950af9d5","Type":"ContainerDied","Data":"aad48c211b1075e99f24229e8da89586a407d1f73f3bdf7e13056ec73c5db24c"} Dec 03 12:49:16 crc kubenswrapper[5052]: I1203 12:49:16.349392 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-8554887475-6pl7l" Dec 03 12:49:16 crc kubenswrapper[5052]: I1203 12:49:16.432406 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"dns-swift-storage-0\" (UniqueName: \"kubernetes.io/configmap/ee63b3ae-44c4-4225-8373-75f5950af9d5-dns-swift-storage-0\") pod \"ee63b3ae-44c4-4225-8373-75f5950af9d5\" (UID: \"ee63b3ae-44c4-4225-8373-75f5950af9d5\") " Dec 03 12:49:16 crc kubenswrapper[5052]: I1203 12:49:16.432623 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/ee63b3ae-44c4-4225-8373-75f5950af9d5-dnsmasq-svc\") pod \"ee63b3ae-44c4-4225-8373-75f5950af9d5\" (UID: \"ee63b3ae-44c4-4225-8373-75f5950af9d5\") " Dec 03 12:49:16 crc kubenswrapper[5052]: I1203 12:49:16.432794 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wwx7n\" (UniqueName: \"kubernetes.io/projected/ee63b3ae-44c4-4225-8373-75f5950af9d5-kube-api-access-wwx7n\") pod \"ee63b3ae-44c4-4225-8373-75f5950af9d5\" (UID: \"ee63b3ae-44c4-4225-8373-75f5950af9d5\") " Dec 03 12:49:16 crc kubenswrapper[5052]: I1203 12:49:16.432927 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ee63b3ae-44c4-4225-8373-75f5950af9d5-config\") pod \"ee63b3ae-44c4-4225-8373-75f5950af9d5\" (UID: \"ee63b3ae-44c4-4225-8373-75f5950af9d5\") " Dec 03 12:49:16 crc kubenswrapper[5052]: I1203 12:49:16.441627 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ee63b3ae-44c4-4225-8373-75f5950af9d5-kube-api-access-wwx7n" (OuterVolumeSpecName: "kube-api-access-wwx7n") pod "ee63b3ae-44c4-4225-8373-75f5950af9d5" (UID: "ee63b3ae-44c4-4225-8373-75f5950af9d5"). InnerVolumeSpecName "kube-api-access-wwx7n". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:49:16 crc kubenswrapper[5052]: I1203 12:49:16.480094 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ee63b3ae-44c4-4225-8373-75f5950af9d5-dns-swift-storage-0" (OuterVolumeSpecName: "dns-swift-storage-0") pod "ee63b3ae-44c4-4225-8373-75f5950af9d5" (UID: "ee63b3ae-44c4-4225-8373-75f5950af9d5"). InnerVolumeSpecName "dns-swift-storage-0". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:49:16 crc kubenswrapper[5052]: I1203 12:49:16.480487 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ee63b3ae-44c4-4225-8373-75f5950af9d5-dnsmasq-svc" (OuterVolumeSpecName: "dnsmasq-svc") pod "ee63b3ae-44c4-4225-8373-75f5950af9d5" (UID: "ee63b3ae-44c4-4225-8373-75f5950af9d5"). InnerVolumeSpecName "dnsmasq-svc". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:49:16 crc kubenswrapper[5052]: I1203 12:49:16.491112 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ee63b3ae-44c4-4225-8373-75f5950af9d5-config" (OuterVolumeSpecName: "config") pod "ee63b3ae-44c4-4225-8373-75f5950af9d5" (UID: "ee63b3ae-44c4-4225-8373-75f5950af9d5"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:49:16 crc kubenswrapper[5052]: I1203 12:49:16.535864 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-wwx7n\" (UniqueName: \"kubernetes.io/projected/ee63b3ae-44c4-4225-8373-75f5950af9d5-kube-api-access-wwx7n\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:16 crc kubenswrapper[5052]: I1203 12:49:16.535939 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ee63b3ae-44c4-4225-8373-75f5950af9d5-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:16 crc kubenswrapper[5052]: I1203 12:49:16.535956 5052 reconciler_common.go:293] "Volume detached for volume \"dns-swift-storage-0\" (UniqueName: \"kubernetes.io/configmap/ee63b3ae-44c4-4225-8373-75f5950af9d5-dns-swift-storage-0\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:16 crc kubenswrapper[5052]: I1203 12:49:16.535970 5052 reconciler_common.go:293] "Volume detached for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/ee63b3ae-44c4-4225-8373-75f5950af9d5-dnsmasq-svc\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:17 crc kubenswrapper[5052]: I1203 12:49:17.060111 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-8554887475-6pl7l" event={"ID":"ee63b3ae-44c4-4225-8373-75f5950af9d5","Type":"ContainerDied","Data":"d42b09e9571187e0cc5b0c0de500425a97201c0ae57f4351e072b6ecc809e171"} Dec 03 12:49:17 crc kubenswrapper[5052]: I1203 12:49:17.060200 5052 scope.go:117] "RemoveContainer" containerID="aad48c211b1075e99f24229e8da89586a407d1f73f3bdf7e13056ec73c5db24c" Dec 03 12:49:17 crc kubenswrapper[5052]: I1203 12:49:17.060144 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-8554887475-6pl7l" Dec 03 12:49:17 crc kubenswrapper[5052]: I1203 12:49:17.085815 5052 scope.go:117] "RemoveContainer" containerID="88f4f14061896d001d9cdb55644ba6c0aba013ab1758be59354f8602ded7f643" Dec 03 12:49:17 crc kubenswrapper[5052]: I1203 12:49:17.093862 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-8554887475-6pl7l"] Dec 03 12:49:17 crc kubenswrapper[5052]: I1203 12:49:17.104008 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-8554887475-6pl7l"] Dec 03 12:49:17 crc kubenswrapper[5052]: I1203 12:49:17.682392 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ee63b3ae-44c4-4225-8373-75f5950af9d5" path="/var/lib/kubelet/pods/ee63b3ae-44c4-4225-8373-75f5950af9d5/volumes" Dec 03 12:49:29 crc kubenswrapper[5052]: I1203 12:49:29.108238 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:49:29 crc kubenswrapper[5052]: I1203 12:49:29.133811 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8t7k4\" (UniqueName: \"kubernetes.io/projected/bd4392a2-9c70-4f3a-9050-ae6a6db01082-kube-api-access-8t7k4\") pod \"bd4392a2-9c70-4f3a-9050-ae6a6db01082\" (UID: \"bd4392a2-9c70-4f3a-9050-ae6a6db01082\") " Dec 03 12:49:29 crc kubenswrapper[5052]: I1203 12:49:29.133885 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/bd4392a2-9c70-4f3a-9050-ae6a6db01082-etc-swift\") pod \"bd4392a2-9c70-4f3a-9050-ae6a6db01082\" (UID: \"bd4392a2-9c70-4f3a-9050-ae6a6db01082\") " Dec 03 12:49:29 crc kubenswrapper[5052]: I1203 12:49:29.133943 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"lock\" (UniqueName: \"kubernetes.io/empty-dir/bd4392a2-9c70-4f3a-9050-ae6a6db01082-lock\") pod \"bd4392a2-9c70-4f3a-9050-ae6a6db01082\" (UID: \"bd4392a2-9c70-4f3a-9050-ae6a6db01082\") " Dec 03 12:49:29 crc kubenswrapper[5052]: I1203 12:49:29.133976 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"swift\" (UniqueName: \"kubernetes.io/local-volume/local-storage13-crc\") pod \"bd4392a2-9c70-4f3a-9050-ae6a6db01082\" (UID: \"bd4392a2-9c70-4f3a-9050-ae6a6db01082\") " Dec 03 12:49:29 crc kubenswrapper[5052]: I1203 12:49:29.134050 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"cache\" (UniqueName: \"kubernetes.io/empty-dir/bd4392a2-9c70-4f3a-9050-ae6a6db01082-cache\") pod \"bd4392a2-9c70-4f3a-9050-ae6a6db01082\" (UID: \"bd4392a2-9c70-4f3a-9050-ae6a6db01082\") " Dec 03 12:49:29 crc kubenswrapper[5052]: I1203 12:49:29.135129 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd4392a2-9c70-4f3a-9050-ae6a6db01082-cache" (OuterVolumeSpecName: "cache") pod "bd4392a2-9c70-4f3a-9050-ae6a6db01082" (UID: "bd4392a2-9c70-4f3a-9050-ae6a6db01082"). InnerVolumeSpecName "cache". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:49:29 crc kubenswrapper[5052]: I1203 12:49:29.135398 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd4392a2-9c70-4f3a-9050-ae6a6db01082-lock" (OuterVolumeSpecName: "lock") pod "bd4392a2-9c70-4f3a-9050-ae6a6db01082" (UID: "bd4392a2-9c70-4f3a-9050-ae6a6db01082"). InnerVolumeSpecName "lock". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:49:29 crc kubenswrapper[5052]: I1203 12:49:29.140090 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage13-crc" (OuterVolumeSpecName: "swift") pod "bd4392a2-9c70-4f3a-9050-ae6a6db01082" (UID: "bd4392a2-9c70-4f3a-9050-ae6a6db01082"). InnerVolumeSpecName "local-storage13-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 12:49:29 crc kubenswrapper[5052]: I1203 12:49:29.141812 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bd4392a2-9c70-4f3a-9050-ae6a6db01082-kube-api-access-8t7k4" (OuterVolumeSpecName: "kube-api-access-8t7k4") pod "bd4392a2-9c70-4f3a-9050-ae6a6db01082" (UID: "bd4392a2-9c70-4f3a-9050-ae6a6db01082"). InnerVolumeSpecName "kube-api-access-8t7k4". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:49:29 crc kubenswrapper[5052]: I1203 12:49:29.145798 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bd4392a2-9c70-4f3a-9050-ae6a6db01082-etc-swift" (OuterVolumeSpecName: "etc-swift") pod "bd4392a2-9c70-4f3a-9050-ae6a6db01082" (UID: "bd4392a2-9c70-4f3a-9050-ae6a6db01082"). InnerVolumeSpecName "etc-swift". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:49:29 crc kubenswrapper[5052]: I1203 12:49:29.235255 5052 reconciler_common.go:293] "Volume detached for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/bd4392a2-9c70-4f3a-9050-ae6a6db01082-etc-swift\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:29 crc kubenswrapper[5052]: I1203 12:49:29.235294 5052 reconciler_common.go:293] "Volume detached for volume \"lock\" (UniqueName: \"kubernetes.io/empty-dir/bd4392a2-9c70-4f3a-9050-ae6a6db01082-lock\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:29 crc kubenswrapper[5052]: I1203 12:49:29.235327 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage13-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage13-crc\") on node \"crc\" " Dec 03 12:49:29 crc kubenswrapper[5052]: I1203 12:49:29.235337 5052 reconciler_common.go:293] "Volume detached for volume \"cache\" (UniqueName: \"kubernetes.io/empty-dir/bd4392a2-9c70-4f3a-9050-ae6a6db01082-cache\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:29 crc kubenswrapper[5052]: I1203 12:49:29.235354 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-8t7k4\" (UniqueName: \"kubernetes.io/projected/bd4392a2-9c70-4f3a-9050-ae6a6db01082-kube-api-access-8t7k4\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:29 crc kubenswrapper[5052]: I1203 12:49:29.253181 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage13-crc" (UniqueName: "kubernetes.io/local-volume/local-storage13-crc") on node "crc" Dec 03 12:49:29 crc kubenswrapper[5052]: I1203 12:49:29.336538 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage13-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage13-crc\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:29 crc kubenswrapper[5052]: I1203 12:49:29.408535 5052 generic.go:334] "Generic (PLEG): container finished" podID="bd4392a2-9c70-4f3a-9050-ae6a6db01082" containerID="6eeb3694b4e9534ae61ec9fcf173cc77b70bcda76de435e9b1ede55531edf6ec" exitCode=137 Dec 03 12:49:29 crc kubenswrapper[5052]: I1203 12:49:29.408585 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"bd4392a2-9c70-4f3a-9050-ae6a6db01082","Type":"ContainerDied","Data":"6eeb3694b4e9534ae61ec9fcf173cc77b70bcda76de435e9b1ede55531edf6ec"} Dec 03 12:49:29 crc kubenswrapper[5052]: I1203 12:49:29.408617 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"bd4392a2-9c70-4f3a-9050-ae6a6db01082","Type":"ContainerDied","Data":"e445f81374349289566e935c5fcf09fc81e82d45facdd44b73e17b255cbb20c6"} Dec 03 12:49:29 crc kubenswrapper[5052]: I1203 12:49:29.408637 5052 scope.go:117] "RemoveContainer" containerID="6eeb3694b4e9534ae61ec9fcf173cc77b70bcda76de435e9b1ede55531edf6ec" Dec 03 12:49:29 crc kubenswrapper[5052]: I1203 12:49:29.408663 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:49:29 crc kubenswrapper[5052]: I1203 12:49:29.436221 5052 scope.go:117] "RemoveContainer" containerID="44d343c040a52a96a2507d69db579564483888cfc2ab0051211cae74e09f32f3" Dec 03 12:49:29 crc kubenswrapper[5052]: I1203 12:49:29.451640 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/swift-storage-0"] Dec 03 12:49:29 crc kubenswrapper[5052]: I1203 12:49:29.458595 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/swift-storage-0"] Dec 03 12:49:29 crc kubenswrapper[5052]: I1203 12:49:29.459992 5052 scope.go:117] "RemoveContainer" containerID="ef01830e5c368fdf7b4e0af6d27790eef08c24d4f1d55b39ccbab91a7ba97c10" Dec 03 12:49:29 crc kubenswrapper[5052]: I1203 12:49:29.481591 5052 scope.go:117] "RemoveContainer" containerID="4f5b00a3217544c70607eacf64edc7dd926767e810b84e309cbc071e09b652d5" Dec 03 12:49:29 crc kubenswrapper[5052]: I1203 12:49:29.503904 5052 scope.go:117] "RemoveContainer" containerID="d17825037af7c0e226677e493c307a512bb167e46667dd9c705ac03cff6758bb" Dec 03 12:49:29 crc kubenswrapper[5052]: I1203 12:49:29.534880 5052 scope.go:117] "RemoveContainer" containerID="257a3713d34f8297bc83998342be63e79d0964dc86ce021417f8cb3fc862ff41" Dec 03 12:49:29 crc kubenswrapper[5052]: I1203 12:49:29.556118 5052 scope.go:117] "RemoveContainer" containerID="073f8906a127eaad4d2dcbdbaf2738671488e4b5da8001af8eb90b508703cf7a" Dec 03 12:49:29 crc kubenswrapper[5052]: I1203 12:49:29.579571 5052 scope.go:117] "RemoveContainer" containerID="0420f9807b79aa9a35f7e00db76c776194c37e61b2e6f2e9e3e83e3f355d270c" Dec 03 12:49:29 crc kubenswrapper[5052]: I1203 12:49:29.597967 5052 scope.go:117] "RemoveContainer" containerID="49448581e8cc46afbfef30edf584ba024f9fdfe5baf0de42a0abed3885f0ef23" Dec 03 12:49:29 crc kubenswrapper[5052]: I1203 12:49:29.614754 5052 scope.go:117] "RemoveContainer" containerID="4aa02cc6c01a66c5bcc1c0d61505c02dbf6374ff0a6550b9d548733640117138" Dec 03 12:49:29 crc kubenswrapper[5052]: I1203 12:49:29.631914 5052 scope.go:117] "RemoveContainer" containerID="2eaf7736f8bbe27fc3320dbe4500d1c1b7388bb8194a9d8a9bb7ef68f452b97f" Dec 03 12:49:29 crc kubenswrapper[5052]: I1203 12:49:29.648062 5052 scope.go:117] "RemoveContainer" containerID="409983b9eafc0681b072e5508199566f3fbeaf0cc55e055beaee8282045a003e" Dec 03 12:49:29 crc kubenswrapper[5052]: I1203 12:49:29.665552 5052 scope.go:117] "RemoveContainer" containerID="e4afa7a612ce1826fda8750703ccc15d8110e17d1d419c076a60957b33af8adb" Dec 03 12:49:29 crc kubenswrapper[5052]: I1203 12:49:29.683018 5052 scope.go:117] "RemoveContainer" containerID="8ea80229aee1ce207c0394efd61dfb519308fde6a2a25c517795af19f4c72da4" Dec 03 12:49:29 crc kubenswrapper[5052]: I1203 12:49:29.683081 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bd4392a2-9c70-4f3a-9050-ae6a6db01082" path="/var/lib/kubelet/pods/bd4392a2-9c70-4f3a-9050-ae6a6db01082/volumes" Dec 03 12:49:29 crc kubenswrapper[5052]: I1203 12:49:29.700944 5052 scope.go:117] "RemoveContainer" containerID="ee68253d5346b57e39fc0b0fed73d3c548e33dc2c36cd2673c5add72f235bb79" Dec 03 12:49:29 crc kubenswrapper[5052]: I1203 12:49:29.731375 5052 scope.go:117] "RemoveContainer" containerID="6eeb3694b4e9534ae61ec9fcf173cc77b70bcda76de435e9b1ede55531edf6ec" Dec 03 12:49:29 crc kubenswrapper[5052]: E1203 12:49:29.732590 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6eeb3694b4e9534ae61ec9fcf173cc77b70bcda76de435e9b1ede55531edf6ec\": container with ID starting with 6eeb3694b4e9534ae61ec9fcf173cc77b70bcda76de435e9b1ede55531edf6ec not found: ID does not exist" containerID="6eeb3694b4e9534ae61ec9fcf173cc77b70bcda76de435e9b1ede55531edf6ec" Dec 03 12:49:29 crc kubenswrapper[5052]: I1203 12:49:29.732629 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6eeb3694b4e9534ae61ec9fcf173cc77b70bcda76de435e9b1ede55531edf6ec"} err="failed to get container status \"6eeb3694b4e9534ae61ec9fcf173cc77b70bcda76de435e9b1ede55531edf6ec\": rpc error: code = NotFound desc = could not find container \"6eeb3694b4e9534ae61ec9fcf173cc77b70bcda76de435e9b1ede55531edf6ec\": container with ID starting with 6eeb3694b4e9534ae61ec9fcf173cc77b70bcda76de435e9b1ede55531edf6ec not found: ID does not exist" Dec 03 12:49:29 crc kubenswrapper[5052]: I1203 12:49:29.732654 5052 scope.go:117] "RemoveContainer" containerID="44d343c040a52a96a2507d69db579564483888cfc2ab0051211cae74e09f32f3" Dec 03 12:49:29 crc kubenswrapper[5052]: E1203 12:49:29.733259 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"44d343c040a52a96a2507d69db579564483888cfc2ab0051211cae74e09f32f3\": container with ID starting with 44d343c040a52a96a2507d69db579564483888cfc2ab0051211cae74e09f32f3 not found: ID does not exist" containerID="44d343c040a52a96a2507d69db579564483888cfc2ab0051211cae74e09f32f3" Dec 03 12:49:29 crc kubenswrapper[5052]: I1203 12:49:29.733286 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"44d343c040a52a96a2507d69db579564483888cfc2ab0051211cae74e09f32f3"} err="failed to get container status \"44d343c040a52a96a2507d69db579564483888cfc2ab0051211cae74e09f32f3\": rpc error: code = NotFound desc = could not find container \"44d343c040a52a96a2507d69db579564483888cfc2ab0051211cae74e09f32f3\": container with ID starting with 44d343c040a52a96a2507d69db579564483888cfc2ab0051211cae74e09f32f3 not found: ID does not exist" Dec 03 12:49:29 crc kubenswrapper[5052]: I1203 12:49:29.733302 5052 scope.go:117] "RemoveContainer" containerID="ef01830e5c368fdf7b4e0af6d27790eef08c24d4f1d55b39ccbab91a7ba97c10" Dec 03 12:49:29 crc kubenswrapper[5052]: E1203 12:49:29.733553 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ef01830e5c368fdf7b4e0af6d27790eef08c24d4f1d55b39ccbab91a7ba97c10\": container with ID starting with ef01830e5c368fdf7b4e0af6d27790eef08c24d4f1d55b39ccbab91a7ba97c10 not found: ID does not exist" containerID="ef01830e5c368fdf7b4e0af6d27790eef08c24d4f1d55b39ccbab91a7ba97c10" Dec 03 12:49:29 crc kubenswrapper[5052]: I1203 12:49:29.733578 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ef01830e5c368fdf7b4e0af6d27790eef08c24d4f1d55b39ccbab91a7ba97c10"} err="failed to get container status \"ef01830e5c368fdf7b4e0af6d27790eef08c24d4f1d55b39ccbab91a7ba97c10\": rpc error: code = NotFound desc = could not find container \"ef01830e5c368fdf7b4e0af6d27790eef08c24d4f1d55b39ccbab91a7ba97c10\": container with ID starting with ef01830e5c368fdf7b4e0af6d27790eef08c24d4f1d55b39ccbab91a7ba97c10 not found: ID does not exist" Dec 03 12:49:29 crc kubenswrapper[5052]: I1203 12:49:29.733593 5052 scope.go:117] "RemoveContainer" containerID="4f5b00a3217544c70607eacf64edc7dd926767e810b84e309cbc071e09b652d5" Dec 03 12:49:29 crc kubenswrapper[5052]: E1203 12:49:29.733847 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4f5b00a3217544c70607eacf64edc7dd926767e810b84e309cbc071e09b652d5\": container with ID starting with 4f5b00a3217544c70607eacf64edc7dd926767e810b84e309cbc071e09b652d5 not found: ID does not exist" containerID="4f5b00a3217544c70607eacf64edc7dd926767e810b84e309cbc071e09b652d5" Dec 03 12:49:29 crc kubenswrapper[5052]: I1203 12:49:29.733866 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4f5b00a3217544c70607eacf64edc7dd926767e810b84e309cbc071e09b652d5"} err="failed to get container status \"4f5b00a3217544c70607eacf64edc7dd926767e810b84e309cbc071e09b652d5\": rpc error: code = NotFound desc = could not find container \"4f5b00a3217544c70607eacf64edc7dd926767e810b84e309cbc071e09b652d5\": container with ID starting with 4f5b00a3217544c70607eacf64edc7dd926767e810b84e309cbc071e09b652d5 not found: ID does not exist" Dec 03 12:49:29 crc kubenswrapper[5052]: I1203 12:49:29.733877 5052 scope.go:117] "RemoveContainer" containerID="d17825037af7c0e226677e493c307a512bb167e46667dd9c705ac03cff6758bb" Dec 03 12:49:29 crc kubenswrapper[5052]: E1203 12:49:29.734081 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d17825037af7c0e226677e493c307a512bb167e46667dd9c705ac03cff6758bb\": container with ID starting with d17825037af7c0e226677e493c307a512bb167e46667dd9c705ac03cff6758bb not found: ID does not exist" containerID="d17825037af7c0e226677e493c307a512bb167e46667dd9c705ac03cff6758bb" Dec 03 12:49:29 crc kubenswrapper[5052]: I1203 12:49:29.734108 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d17825037af7c0e226677e493c307a512bb167e46667dd9c705ac03cff6758bb"} err="failed to get container status \"d17825037af7c0e226677e493c307a512bb167e46667dd9c705ac03cff6758bb\": rpc error: code = NotFound desc = could not find container \"d17825037af7c0e226677e493c307a512bb167e46667dd9c705ac03cff6758bb\": container with ID starting with d17825037af7c0e226677e493c307a512bb167e46667dd9c705ac03cff6758bb not found: ID does not exist" Dec 03 12:49:29 crc kubenswrapper[5052]: I1203 12:49:29.734125 5052 scope.go:117] "RemoveContainer" containerID="257a3713d34f8297bc83998342be63e79d0964dc86ce021417f8cb3fc862ff41" Dec 03 12:49:29 crc kubenswrapper[5052]: E1203 12:49:29.734752 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"257a3713d34f8297bc83998342be63e79d0964dc86ce021417f8cb3fc862ff41\": container with ID starting with 257a3713d34f8297bc83998342be63e79d0964dc86ce021417f8cb3fc862ff41 not found: ID does not exist" containerID="257a3713d34f8297bc83998342be63e79d0964dc86ce021417f8cb3fc862ff41" Dec 03 12:49:29 crc kubenswrapper[5052]: I1203 12:49:29.734780 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"257a3713d34f8297bc83998342be63e79d0964dc86ce021417f8cb3fc862ff41"} err="failed to get container status \"257a3713d34f8297bc83998342be63e79d0964dc86ce021417f8cb3fc862ff41\": rpc error: code = NotFound desc = could not find container \"257a3713d34f8297bc83998342be63e79d0964dc86ce021417f8cb3fc862ff41\": container with ID starting with 257a3713d34f8297bc83998342be63e79d0964dc86ce021417f8cb3fc862ff41 not found: ID does not exist" Dec 03 12:49:29 crc kubenswrapper[5052]: I1203 12:49:29.734798 5052 scope.go:117] "RemoveContainer" containerID="073f8906a127eaad4d2dcbdbaf2738671488e4b5da8001af8eb90b508703cf7a" Dec 03 12:49:29 crc kubenswrapper[5052]: E1203 12:49:29.735071 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"073f8906a127eaad4d2dcbdbaf2738671488e4b5da8001af8eb90b508703cf7a\": container with ID starting with 073f8906a127eaad4d2dcbdbaf2738671488e4b5da8001af8eb90b508703cf7a not found: ID does not exist" containerID="073f8906a127eaad4d2dcbdbaf2738671488e4b5da8001af8eb90b508703cf7a" Dec 03 12:49:29 crc kubenswrapper[5052]: I1203 12:49:29.735097 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"073f8906a127eaad4d2dcbdbaf2738671488e4b5da8001af8eb90b508703cf7a"} err="failed to get container status \"073f8906a127eaad4d2dcbdbaf2738671488e4b5da8001af8eb90b508703cf7a\": rpc error: code = NotFound desc = could not find container \"073f8906a127eaad4d2dcbdbaf2738671488e4b5da8001af8eb90b508703cf7a\": container with ID starting with 073f8906a127eaad4d2dcbdbaf2738671488e4b5da8001af8eb90b508703cf7a not found: ID does not exist" Dec 03 12:49:29 crc kubenswrapper[5052]: I1203 12:49:29.735113 5052 scope.go:117] "RemoveContainer" containerID="0420f9807b79aa9a35f7e00db76c776194c37e61b2e6f2e9e3e83e3f355d270c" Dec 03 12:49:29 crc kubenswrapper[5052]: E1203 12:49:29.735350 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0420f9807b79aa9a35f7e00db76c776194c37e61b2e6f2e9e3e83e3f355d270c\": container with ID starting with 0420f9807b79aa9a35f7e00db76c776194c37e61b2e6f2e9e3e83e3f355d270c not found: ID does not exist" containerID="0420f9807b79aa9a35f7e00db76c776194c37e61b2e6f2e9e3e83e3f355d270c" Dec 03 12:49:29 crc kubenswrapper[5052]: I1203 12:49:29.735381 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0420f9807b79aa9a35f7e00db76c776194c37e61b2e6f2e9e3e83e3f355d270c"} err="failed to get container status \"0420f9807b79aa9a35f7e00db76c776194c37e61b2e6f2e9e3e83e3f355d270c\": rpc error: code = NotFound desc = could not find container \"0420f9807b79aa9a35f7e00db76c776194c37e61b2e6f2e9e3e83e3f355d270c\": container with ID starting with 0420f9807b79aa9a35f7e00db76c776194c37e61b2e6f2e9e3e83e3f355d270c not found: ID does not exist" Dec 03 12:49:29 crc kubenswrapper[5052]: I1203 12:49:29.735401 5052 scope.go:117] "RemoveContainer" containerID="49448581e8cc46afbfef30edf584ba024f9fdfe5baf0de42a0abed3885f0ef23" Dec 03 12:49:29 crc kubenswrapper[5052]: E1203 12:49:29.735711 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"49448581e8cc46afbfef30edf584ba024f9fdfe5baf0de42a0abed3885f0ef23\": container with ID starting with 49448581e8cc46afbfef30edf584ba024f9fdfe5baf0de42a0abed3885f0ef23 not found: ID does not exist" containerID="49448581e8cc46afbfef30edf584ba024f9fdfe5baf0de42a0abed3885f0ef23" Dec 03 12:49:29 crc kubenswrapper[5052]: I1203 12:49:29.735731 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"49448581e8cc46afbfef30edf584ba024f9fdfe5baf0de42a0abed3885f0ef23"} err="failed to get container status \"49448581e8cc46afbfef30edf584ba024f9fdfe5baf0de42a0abed3885f0ef23\": rpc error: code = NotFound desc = could not find container \"49448581e8cc46afbfef30edf584ba024f9fdfe5baf0de42a0abed3885f0ef23\": container with ID starting with 49448581e8cc46afbfef30edf584ba024f9fdfe5baf0de42a0abed3885f0ef23 not found: ID does not exist" Dec 03 12:49:29 crc kubenswrapper[5052]: I1203 12:49:29.735743 5052 scope.go:117] "RemoveContainer" containerID="4aa02cc6c01a66c5bcc1c0d61505c02dbf6374ff0a6550b9d548733640117138" Dec 03 12:49:29 crc kubenswrapper[5052]: E1203 12:49:29.735961 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4aa02cc6c01a66c5bcc1c0d61505c02dbf6374ff0a6550b9d548733640117138\": container with ID starting with 4aa02cc6c01a66c5bcc1c0d61505c02dbf6374ff0a6550b9d548733640117138 not found: ID does not exist" containerID="4aa02cc6c01a66c5bcc1c0d61505c02dbf6374ff0a6550b9d548733640117138" Dec 03 12:49:29 crc kubenswrapper[5052]: I1203 12:49:29.735981 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4aa02cc6c01a66c5bcc1c0d61505c02dbf6374ff0a6550b9d548733640117138"} err="failed to get container status \"4aa02cc6c01a66c5bcc1c0d61505c02dbf6374ff0a6550b9d548733640117138\": rpc error: code = NotFound desc = could not find container \"4aa02cc6c01a66c5bcc1c0d61505c02dbf6374ff0a6550b9d548733640117138\": container with ID starting with 4aa02cc6c01a66c5bcc1c0d61505c02dbf6374ff0a6550b9d548733640117138 not found: ID does not exist" Dec 03 12:49:29 crc kubenswrapper[5052]: I1203 12:49:29.735992 5052 scope.go:117] "RemoveContainer" containerID="2eaf7736f8bbe27fc3320dbe4500d1c1b7388bb8194a9d8a9bb7ef68f452b97f" Dec 03 12:49:29 crc kubenswrapper[5052]: E1203 12:49:29.736424 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2eaf7736f8bbe27fc3320dbe4500d1c1b7388bb8194a9d8a9bb7ef68f452b97f\": container with ID starting with 2eaf7736f8bbe27fc3320dbe4500d1c1b7388bb8194a9d8a9bb7ef68f452b97f not found: ID does not exist" containerID="2eaf7736f8bbe27fc3320dbe4500d1c1b7388bb8194a9d8a9bb7ef68f452b97f" Dec 03 12:49:29 crc kubenswrapper[5052]: I1203 12:49:29.736452 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2eaf7736f8bbe27fc3320dbe4500d1c1b7388bb8194a9d8a9bb7ef68f452b97f"} err="failed to get container status \"2eaf7736f8bbe27fc3320dbe4500d1c1b7388bb8194a9d8a9bb7ef68f452b97f\": rpc error: code = NotFound desc = could not find container \"2eaf7736f8bbe27fc3320dbe4500d1c1b7388bb8194a9d8a9bb7ef68f452b97f\": container with ID starting with 2eaf7736f8bbe27fc3320dbe4500d1c1b7388bb8194a9d8a9bb7ef68f452b97f not found: ID does not exist" Dec 03 12:49:29 crc kubenswrapper[5052]: I1203 12:49:29.736470 5052 scope.go:117] "RemoveContainer" containerID="409983b9eafc0681b072e5508199566f3fbeaf0cc55e055beaee8282045a003e" Dec 03 12:49:29 crc kubenswrapper[5052]: E1203 12:49:29.737005 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"409983b9eafc0681b072e5508199566f3fbeaf0cc55e055beaee8282045a003e\": container with ID starting with 409983b9eafc0681b072e5508199566f3fbeaf0cc55e055beaee8282045a003e not found: ID does not exist" containerID="409983b9eafc0681b072e5508199566f3fbeaf0cc55e055beaee8282045a003e" Dec 03 12:49:29 crc kubenswrapper[5052]: I1203 12:49:29.737078 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"409983b9eafc0681b072e5508199566f3fbeaf0cc55e055beaee8282045a003e"} err="failed to get container status \"409983b9eafc0681b072e5508199566f3fbeaf0cc55e055beaee8282045a003e\": rpc error: code = NotFound desc = could not find container \"409983b9eafc0681b072e5508199566f3fbeaf0cc55e055beaee8282045a003e\": container with ID starting with 409983b9eafc0681b072e5508199566f3fbeaf0cc55e055beaee8282045a003e not found: ID does not exist" Dec 03 12:49:29 crc kubenswrapper[5052]: I1203 12:49:29.737091 5052 scope.go:117] "RemoveContainer" containerID="e4afa7a612ce1826fda8750703ccc15d8110e17d1d419c076a60957b33af8adb" Dec 03 12:49:29 crc kubenswrapper[5052]: E1203 12:49:29.738248 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e4afa7a612ce1826fda8750703ccc15d8110e17d1d419c076a60957b33af8adb\": container with ID starting with e4afa7a612ce1826fda8750703ccc15d8110e17d1d419c076a60957b33af8adb not found: ID does not exist" containerID="e4afa7a612ce1826fda8750703ccc15d8110e17d1d419c076a60957b33af8adb" Dec 03 12:49:29 crc kubenswrapper[5052]: I1203 12:49:29.738275 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e4afa7a612ce1826fda8750703ccc15d8110e17d1d419c076a60957b33af8adb"} err="failed to get container status \"e4afa7a612ce1826fda8750703ccc15d8110e17d1d419c076a60957b33af8adb\": rpc error: code = NotFound desc = could not find container \"e4afa7a612ce1826fda8750703ccc15d8110e17d1d419c076a60957b33af8adb\": container with ID starting with e4afa7a612ce1826fda8750703ccc15d8110e17d1d419c076a60957b33af8adb not found: ID does not exist" Dec 03 12:49:29 crc kubenswrapper[5052]: I1203 12:49:29.738290 5052 scope.go:117] "RemoveContainer" containerID="8ea80229aee1ce207c0394efd61dfb519308fde6a2a25c517795af19f4c72da4" Dec 03 12:49:29 crc kubenswrapper[5052]: E1203 12:49:29.738925 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8ea80229aee1ce207c0394efd61dfb519308fde6a2a25c517795af19f4c72da4\": container with ID starting with 8ea80229aee1ce207c0394efd61dfb519308fde6a2a25c517795af19f4c72da4 not found: ID does not exist" containerID="8ea80229aee1ce207c0394efd61dfb519308fde6a2a25c517795af19f4c72da4" Dec 03 12:49:29 crc kubenswrapper[5052]: I1203 12:49:29.738950 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8ea80229aee1ce207c0394efd61dfb519308fde6a2a25c517795af19f4c72da4"} err="failed to get container status \"8ea80229aee1ce207c0394efd61dfb519308fde6a2a25c517795af19f4c72da4\": rpc error: code = NotFound desc = could not find container \"8ea80229aee1ce207c0394efd61dfb519308fde6a2a25c517795af19f4c72da4\": container with ID starting with 8ea80229aee1ce207c0394efd61dfb519308fde6a2a25c517795af19f4c72da4 not found: ID does not exist" Dec 03 12:49:29 crc kubenswrapper[5052]: I1203 12:49:29.738965 5052 scope.go:117] "RemoveContainer" containerID="ee68253d5346b57e39fc0b0fed73d3c548e33dc2c36cd2673c5add72f235bb79" Dec 03 12:49:29 crc kubenswrapper[5052]: E1203 12:49:29.739242 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ee68253d5346b57e39fc0b0fed73d3c548e33dc2c36cd2673c5add72f235bb79\": container with ID starting with ee68253d5346b57e39fc0b0fed73d3c548e33dc2c36cd2673c5add72f235bb79 not found: ID does not exist" containerID="ee68253d5346b57e39fc0b0fed73d3c548e33dc2c36cd2673c5add72f235bb79" Dec 03 12:49:29 crc kubenswrapper[5052]: I1203 12:49:29.739261 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ee68253d5346b57e39fc0b0fed73d3c548e33dc2c36cd2673c5add72f235bb79"} err="failed to get container status \"ee68253d5346b57e39fc0b0fed73d3c548e33dc2c36cd2673c5add72f235bb79\": rpc error: code = NotFound desc = could not find container \"ee68253d5346b57e39fc0b0fed73d3c548e33dc2c36cd2673c5add72f235bb79\": container with ID starting with ee68253d5346b57e39fc0b0fed73d3c548e33dc2c36cd2673c5add72f235bb79 not found: ID does not exist" Dec 03 12:49:32 crc kubenswrapper[5052]: I1203 12:49:32.789106 5052 scope.go:117] "RemoveContainer" containerID="5b04eb43cca61516127ebbe98e362a70bd97994640daed64b640e02a6705f087" Dec 03 12:49:32 crc kubenswrapper[5052]: I1203 12:49:32.813100 5052 scope.go:117] "RemoveContainer" containerID="f48e73f815035e727aa3a398a4b9910157004cad319377fa765a601efb172b96" Dec 03 12:49:32 crc kubenswrapper[5052]: I1203 12:49:32.862526 5052 scope.go:117] "RemoveContainer" containerID="3c9394ad9a076c5005649d998f0ffa542de55379118fcfb363d27cb846a45d9f" Dec 03 12:49:32 crc kubenswrapper[5052]: I1203 12:49:32.910108 5052 scope.go:117] "RemoveContainer" containerID="353555b1bb776ead8959c338289c90f24c0ffc2347769f7b77a175865eb773e9" Dec 03 12:49:32 crc kubenswrapper[5052]: I1203 12:49:32.929132 5052 scope.go:117] "RemoveContainer" containerID="918f9e1c99cf9114714c32008148f123af54d1f2de2fdbb10e539c82c6d0d7a7" Dec 03 12:49:32 crc kubenswrapper[5052]: I1203 12:49:32.952009 5052 scope.go:117] "RemoveContainer" containerID="c38db48afa4b344c32d00e4db4d452c8f75d832d0c682df5c1a48a83e977e312" Dec 03 12:49:32 crc kubenswrapper[5052]: I1203 12:49:32.979792 5052 scope.go:117] "RemoveContainer" containerID="060ad9813ab6a37e3a1a8a5496ce0ed7a7fb29d84ac18787732f368231785872" Dec 03 12:49:33 crc kubenswrapper[5052]: I1203 12:49:33.007286 5052 scope.go:117] "RemoveContainer" containerID="55b9e1923f83215ee935eb7dac25724eaea8aa946882abe2860d9d47a8b78111" Dec 03 12:49:33 crc kubenswrapper[5052]: I1203 12:49:33.034130 5052 scope.go:117] "RemoveContainer" containerID="925b2497e4f642ed4fb1967329c7ab704e5ce009931ac3070b5ab19c386d16f2" Dec 03 12:49:33 crc kubenswrapper[5052]: I1203 12:49:33.058430 5052 scope.go:117] "RemoveContainer" containerID="f9c7ff2956392c8515764b57f36f4a4495d905e7a71d9a81e290f937bf0e1dcd" Dec 03 12:49:33 crc kubenswrapper[5052]: I1203 12:49:33.086688 5052 scope.go:117] "RemoveContainer" containerID="93ea570e117e1af074faa4804b0c9dd26bf11892c6a87661f85fc221a9464d09" Dec 03 12:49:33 crc kubenswrapper[5052]: I1203 12:49:33.103748 5052 scope.go:117] "RemoveContainer" containerID="fd0a7f48fe60812e7073999bafe14759c449ef5d471df89f6fa9c0b085b41b7c" Dec 03 12:49:33 crc kubenswrapper[5052]: I1203 12:49:33.123749 5052 scope.go:117] "RemoveContainer" containerID="851a4c98712831fdb1a18e01a6741b469ff7b63efd9396a566b09e32d17f05ba" Dec 03 12:49:33 crc kubenswrapper[5052]: I1203 12:49:33.145852 5052 scope.go:117] "RemoveContainer" containerID="d871ecf0facb775d7ef4d9dacb2a2746b353a7fee55e6f036959cc141b2fcf0e" Dec 03 12:49:33 crc kubenswrapper[5052]: I1203 12:49:33.170100 5052 scope.go:117] "RemoveContainer" containerID="1e18bfb707a7b7ab93495e154e2c9ee1ca6a50570f41008061ee3a82fb2bf49f" Dec 03 12:49:33 crc kubenswrapper[5052]: I1203 12:49:33.189726 5052 scope.go:117] "RemoveContainer" containerID="4fc9099c8e6b1d6bde19c5a50f204a4526a781a027bf3b6727e6310b27729d5f" Dec 03 12:49:33 crc kubenswrapper[5052]: I1203 12:49:33.208779 5052 scope.go:117] "RemoveContainer" containerID="5d97dda03f4bb7760eec9dbecdc78fddde19c90f53bee737bf5ab871d649c4ae" Dec 03 12:49:33 crc kubenswrapper[5052]: I1203 12:49:33.233719 5052 scope.go:117] "RemoveContainer" containerID="265de199d1a82bc77793a76bb9691418b9980aa4b80f4223f554b36dc4b53abe" Dec 03 12:49:33 crc kubenswrapper[5052]: I1203 12:49:33.259867 5052 scope.go:117] "RemoveContainer" containerID="b623beb1d698577700860795ffcf72792b261e732bcbf58537ca90d6307df621" Dec 03 12:49:33 crc kubenswrapper[5052]: I1203 12:49:33.280553 5052 scope.go:117] "RemoveContainer" containerID="2268353a2de5ede55fc8582a63a842701e59a57d84ce16fa95b9377d33da7455" Dec 03 12:49:33 crc kubenswrapper[5052]: I1203 12:49:33.324629 5052 scope.go:117] "RemoveContainer" containerID="0fcd271f98e04087a547c0155f1cd70b4b5b3a96bfb2a5c4a2f74fcf8ba00936" Dec 03 12:49:33 crc kubenswrapper[5052]: I1203 12:49:33.355977 5052 scope.go:117] "RemoveContainer" containerID="2b66f8fd287d88b09b2bdfd5155d95c7605918d968768e443682215d572dbf83" Dec 03 12:49:33 crc kubenswrapper[5052]: I1203 12:49:33.375477 5052 scope.go:117] "RemoveContainer" containerID="5574cf1b9637d9dc4df4d731d497a66be92d5061b8f70847d57c773861022281" Dec 03 12:49:33 crc kubenswrapper[5052]: I1203 12:49:33.394629 5052 scope.go:117] "RemoveContainer" containerID="e88eca179985f518ccd057e45f232aca0ad9b2db7b2c59e4e34612eeb6405a5a" Dec 03 12:49:33 crc kubenswrapper[5052]: I1203 12:49:33.416359 5052 scope.go:117] "RemoveContainer" containerID="28741b5100ab5f3a042ecef7948d376f4cfc37deb45f39e79dcf1bdea4115ee9" Dec 03 12:49:33 crc kubenswrapper[5052]: I1203 12:49:33.437639 5052 scope.go:117] "RemoveContainer" containerID="65e1fdf0d5abf9142c0d5dd8fd15f412c134a90fb4aecb28999c1caba228326b" Dec 03 12:49:33 crc kubenswrapper[5052]: I1203 12:49:33.468468 5052 scope.go:117] "RemoveContainer" containerID="8b321f78db963a5c8fd8a06d8d437e337dea72a0472cde308cbba093a65a6e60" Dec 03 12:49:33 crc kubenswrapper[5052]: I1203 12:49:33.486884 5052 scope.go:117] "RemoveContainer" containerID="fc4f4321d2ac241b4a797a64c285f190f073d7f5f8f626ec4d2711dd1d73a837" Dec 03 12:49:33 crc kubenswrapper[5052]: I1203 12:49:33.510215 5052 scope.go:117] "RemoveContainer" containerID="29577d728aa7f4df45f66cf1e60698a7460652a0971153a774ed4bab7555b7e7" Dec 03 12:49:35 crc kubenswrapper[5052]: I1203 12:49:35.832442 5052 pod_container_manager_linux.go:210] "Failed to delete cgroup paths" cgroupName=["kubepods","besteffort","pod5201872c-fde1-49d6-9784-6a7f54497990"] err="unable to destroy cgroup paths for cgroup [kubepods besteffort pod5201872c-fde1-49d6-9784-6a7f54497990] : Timed out while waiting for systemd to remove kubepods-besteffort-pod5201872c_fde1_49d6_9784_6a7f54497990.slice" Dec 03 12:49:35 crc kubenswrapper[5052]: E1203 12:49:35.832892 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to delete cgroup paths for [kubepods besteffort pod5201872c-fde1-49d6-9784-6a7f54497990] : unable to destroy cgroup paths for cgroup [kubepods besteffort pod5201872c-fde1-49d6-9784-6a7f54497990] : Timed out while waiting for systemd to remove kubepods-besteffort-pod5201872c_fde1_49d6_9784_6a7f54497990.slice" pod="openstack-kuttl-tests/placement-74746c7f4-h6sx8" podUID="5201872c-fde1-49d6-9784-6a7f54497990" Dec 03 12:49:35 crc kubenswrapper[5052]: I1203 12:49:35.840393 5052 pod_container_manager_linux.go:210] "Failed to delete cgroup paths" cgroupName=["kubepods","besteffort","podb1a74998-b009-4bb6-a0dc-d94ffae8b641"] err="unable to destroy cgroup paths for cgroup [kubepods besteffort podb1a74998-b009-4bb6-a0dc-d94ffae8b641] : Timed out while waiting for systemd to remove kubepods-besteffort-podb1a74998_b009_4bb6_a0dc_d94ffae8b641.slice" Dec 03 12:49:35 crc kubenswrapper[5052]: E1203 12:49:35.840476 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to delete cgroup paths for [kubepods besteffort podb1a74998-b009-4bb6-a0dc-d94ffae8b641] : unable to destroy cgroup paths for cgroup [kubepods besteffort podb1a74998-b009-4bb6-a0dc-d94ffae8b641] : Timed out while waiting for systemd to remove kubepods-besteffort-podb1a74998_b009_4bb6_a0dc_d94ffae8b641.slice" pod="openstack-kuttl-tests/glance-default-external-api-0" podUID="b1a74998-b009-4bb6-a0dc-d94ffae8b641" Dec 03 12:49:35 crc kubenswrapper[5052]: I1203 12:49:35.844569 5052 pod_container_manager_linux.go:210] "Failed to delete cgroup paths" cgroupName=["kubepods","besteffort","pod557175a9-81e5-4605-bd67-f1dfa575fb1b"] err="unable to destroy cgroup paths for cgroup [kubepods besteffort pod557175a9-81e5-4605-bd67-f1dfa575fb1b] : Timed out while waiting for systemd to remove kubepods-besteffort-pod557175a9_81e5_4605_bd67_f1dfa575fb1b.slice" Dec 03 12:49:35 crc kubenswrapper[5052]: E1203 12:49:35.844627 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to delete cgroup paths for [kubepods besteffort pod557175a9-81e5-4605-bd67-f1dfa575fb1b] : unable to destroy cgroup paths for cgroup [kubepods besteffort pod557175a9-81e5-4605-bd67-f1dfa575fb1b] : Timed out while waiting for systemd to remove kubepods-besteffort-pod557175a9_81e5_4605_bd67_f1dfa575fb1b.slice" pod="openstack-kuttl-tests/cinder-api-0" podUID="557175a9-81e5-4605-bd67-f1dfa575fb1b" Dec 03 12:49:36 crc kubenswrapper[5052]: I1203 12:49:36.506762 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-74746c7f4-h6sx8" Dec 03 12:49:36 crc kubenswrapper[5052]: I1203 12:49:36.506792 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:49:36 crc kubenswrapper[5052]: I1203 12:49:36.506761 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:49:36 crc kubenswrapper[5052]: I1203 12:49:36.569905 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/placement-74746c7f4-h6sx8"] Dec 03 12:49:36 crc kubenswrapper[5052]: I1203 12:49:36.575307 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/placement-74746c7f4-h6sx8"] Dec 03 12:49:36 crc kubenswrapper[5052]: I1203 12:49:36.603784 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 12:49:36 crc kubenswrapper[5052]: I1203 12:49:36.609559 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 12:49:36 crc kubenswrapper[5052]: I1203 12:49:36.620068 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:49:36 crc kubenswrapper[5052]: I1203 12:49:36.625184 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:49:37 crc kubenswrapper[5052]: I1203 12:49:37.684733 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5201872c-fde1-49d6-9784-6a7f54497990" path="/var/lib/kubelet/pods/5201872c-fde1-49d6-9784-6a7f54497990/volumes" Dec 03 12:49:37 crc kubenswrapper[5052]: I1203 12:49:37.686062 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="557175a9-81e5-4605-bd67-f1dfa575fb1b" path="/var/lib/kubelet/pods/557175a9-81e5-4605-bd67-f1dfa575fb1b/volumes" Dec 03 12:49:37 crc kubenswrapper[5052]: I1203 12:49:37.687045 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b1a74998-b009-4bb6-a0dc-d94ffae8b641" path="/var/lib/kubelet/pods/b1a74998-b009-4bb6-a0dc-d94ffae8b641/volumes" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.804668 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["crc-storage/crc-storage-crc-mz9xv"] Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.811374 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["crc-storage/crc-storage-crc-mz9xv"] Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.958606 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["crc-storage/crc-storage-crc-jb5w2"] Dec 03 12:49:43 crc kubenswrapper[5052]: E1203 12:49:43.958896 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="efb0e60c-2a71-4641-87b2-0f5532b28678" containerName="nova-metadata-log" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.958908 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="efb0e60c-2a71-4641-87b2-0f5532b28678" containerName="nova-metadata-log" Dec 03 12:49:43 crc kubenswrapper[5052]: E1203 12:49:43.958929 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="098a46f8-2d49-4fea-a95c-5678601aa164" containerName="glance-log" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.958936 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="098a46f8-2d49-4fea-a95c-5678601aa164" containerName="glance-log" Dec 03 12:49:43 crc kubenswrapper[5052]: E1203 12:49:43.958948 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="a2bc501c-ebef-4e68-8ae2-e8c4217cadff" containerName="mariadb-account-delete" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.958954 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="a2bc501c-ebef-4e68-8ae2-e8c4217cadff" containerName="mariadb-account-delete" Dec 03 12:49:43 crc kubenswrapper[5052]: E1203 12:49:43.958961 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="2846c397-2869-49f3-aeca-cf4b56bc8834" containerName="mariadb-account-delete" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.958969 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="2846c397-2869-49f3-aeca-cf4b56bc8834" containerName="mariadb-account-delete" Dec 03 12:49:43 crc kubenswrapper[5052]: E1203 12:49:43.958979 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="2005f89c-8a19-403c-acc9-c13ede717372" containerName="openstack-network-exporter" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.958984 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="2005f89c-8a19-403c-acc9-c13ede717372" containerName="openstack-network-exporter" Dec 03 12:49:43 crc kubenswrapper[5052]: E1203 12:49:43.958990 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="edae7848-51ff-40d6-83f2-ffff085f3f3e" containerName="nova-cell1-novncproxy-novncproxy" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.958996 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="edae7848-51ff-40d6-83f2-ffff085f3f3e" containerName="nova-cell1-novncproxy-novncproxy" Dec 03 12:49:43 crc kubenswrapper[5052]: E1203 12:49:43.959010 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="8669832d-c47a-49bd-9491-dc7141ac6281" containerName="nova-cell1-conductor-conductor" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.959015 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="8669832d-c47a-49bd-9491-dc7141ac6281" containerName="nova-cell1-conductor-conductor" Dec 03 12:49:43 crc kubenswrapper[5052]: E1203 12:49:43.959023 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="bd4392a2-9c70-4f3a-9050-ae6a6db01082" containerName="account-server" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.959028 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd4392a2-9c70-4f3a-9050-ae6a6db01082" containerName="account-server" Dec 03 12:49:43 crc kubenswrapper[5052]: E1203 12:49:43.959035 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="bd4392a2-9c70-4f3a-9050-ae6a6db01082" containerName="container-updater" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.959041 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd4392a2-9c70-4f3a-9050-ae6a6db01082" containerName="container-updater" Dec 03 12:49:43 crc kubenswrapper[5052]: E1203 12:49:43.959047 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="bd4392a2-9c70-4f3a-9050-ae6a6db01082" containerName="object-expirer" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.959052 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd4392a2-9c70-4f3a-9050-ae6a6db01082" containerName="object-expirer" Dec 03 12:49:43 crc kubenswrapper[5052]: E1203 12:49:43.959064 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="b1a74998-b009-4bb6-a0dc-d94ffae8b641" containerName="glance-log" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.959069 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="b1a74998-b009-4bb6-a0dc-d94ffae8b641" containerName="glance-log" Dec 03 12:49:43 crc kubenswrapper[5052]: E1203 12:49:43.959080 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="876d06e6-0187-4e3d-9183-ed2083fe5189" containerName="kube-state-metrics" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.959085 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="876d06e6-0187-4e3d-9183-ed2083fe5189" containerName="kube-state-metrics" Dec 03 12:49:43 crc kubenswrapper[5052]: E1203 12:49:43.959091 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="6a5de7fb-b482-4849-a957-e679616aabc0" containerName="mariadb-account-delete" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.959099 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="6a5de7fb-b482-4849-a957-e679616aabc0" containerName="mariadb-account-delete" Dec 03 12:49:43 crc kubenswrapper[5052]: E1203 12:49:43.959105 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="3479f9ad-871f-463e-96ce-50f4069088b3" containerName="mysql-bootstrap" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.959110 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="3479f9ad-871f-463e-96ce-50f4069088b3" containerName="mysql-bootstrap" Dec 03 12:49:43 crc kubenswrapper[5052]: E1203 12:49:43.959117 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="456c3fb1-90c0-4d17-863e-414380759f97" containerName="mariadb-account-delete" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.959123 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="456c3fb1-90c0-4d17-863e-414380759f97" containerName="mariadb-account-delete" Dec 03 12:49:43 crc kubenswrapper[5052]: E1203 12:49:43.959133 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="2065f5bb-88eb-4e84-88b6-9982833d413d" containerName="barbican-keystone-listener" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.959138 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="2065f5bb-88eb-4e84-88b6-9982833d413d" containerName="barbican-keystone-listener" Dec 03 12:49:43 crc kubenswrapper[5052]: E1203 12:49:43.959146 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="0cb33a40-6f0a-4e89-a159-4edd3af3a2fa" containerName="rabbitmq" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.959151 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="0cb33a40-6f0a-4e89-a159-4edd3af3a2fa" containerName="rabbitmq" Dec 03 12:49:43 crc kubenswrapper[5052]: E1203 12:49:43.959161 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="6d653682-52e2-4940-af66-633475bfe09a" containerName="nova-cell0-conductor-conductor" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.959167 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="6d653682-52e2-4940-af66-633475bfe09a" containerName="nova-cell0-conductor-conductor" Dec 03 12:49:43 crc kubenswrapper[5052]: E1203 12:49:43.959174 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="8669832d-c47a-49bd-9491-dc7141ac6281" containerName="nova-cell1-conductor-conductor" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.959181 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="8669832d-c47a-49bd-9491-dc7141ac6281" containerName="nova-cell1-conductor-conductor" Dec 03 12:49:43 crc kubenswrapper[5052]: E1203 12:49:43.959187 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="342cba80-22e1-488a-bbfb-75b67ca536ac" containerName="openstack-network-exporter" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.959193 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="342cba80-22e1-488a-bbfb-75b67ca536ac" containerName="openstack-network-exporter" Dec 03 12:49:43 crc kubenswrapper[5052]: E1203 12:49:43.959201 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="456c3fb1-90c0-4d17-863e-414380759f97" containerName="mariadb-account-delete" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.959206 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="456c3fb1-90c0-4d17-863e-414380759f97" containerName="mariadb-account-delete" Dec 03 12:49:43 crc kubenswrapper[5052]: E1203 12:49:43.959212 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="488c3529-4c2b-47c0-9dcf-05a064c34d0d" containerName="setup-container" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.959218 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="488c3529-4c2b-47c0-9dcf-05a064c34d0d" containerName="setup-container" Dec 03 12:49:43 crc kubenswrapper[5052]: E1203 12:49:43.959226 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="5201872c-fde1-49d6-9784-6a7f54497990" containerName="placement-log" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.959232 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="5201872c-fde1-49d6-9784-6a7f54497990" containerName="placement-log" Dec 03 12:49:43 crc kubenswrapper[5052]: E1203 12:49:43.959241 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="2065f5bb-88eb-4e84-88b6-9982833d413d" containerName="barbican-keystone-listener-log" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.959248 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="2065f5bb-88eb-4e84-88b6-9982833d413d" containerName="barbican-keystone-listener-log" Dec 03 12:49:43 crc kubenswrapper[5052]: E1203 12:49:43.959255 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="bd4392a2-9c70-4f3a-9050-ae6a6db01082" containerName="account-auditor" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.959261 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd4392a2-9c70-4f3a-9050-ae6a6db01082" containerName="account-auditor" Dec 03 12:49:43 crc kubenswrapper[5052]: E1203 12:49:43.959268 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="2005f89c-8a19-403c-acc9-c13ede717372" containerName="ovsdbserver-nb" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.959273 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="2005f89c-8a19-403c-acc9-c13ede717372" containerName="ovsdbserver-nb" Dec 03 12:49:43 crc kubenswrapper[5052]: E1203 12:49:43.959278 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="3479f9ad-871f-463e-96ce-50f4069088b3" containerName="galera" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.959284 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="3479f9ad-871f-463e-96ce-50f4069088b3" containerName="galera" Dec 03 12:49:43 crc kubenswrapper[5052]: E1203 12:49:43.959291 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="bd4392a2-9c70-4f3a-9050-ae6a6db01082" containerName="account-replicator" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.959296 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd4392a2-9c70-4f3a-9050-ae6a6db01082" containerName="account-replicator" Dec 03 12:49:43 crc kubenswrapper[5052]: E1203 12:49:43.959307 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="a69bfb18-1267-4383-b51a-5e0eba70d1bc" containerName="proxy-httpd" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.959312 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="a69bfb18-1267-4383-b51a-5e0eba70d1bc" containerName="proxy-httpd" Dec 03 12:49:43 crc kubenswrapper[5052]: E1203 12:49:43.959321 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="5d11b02e-c14a-4d1a-819a-4c3b66aedd33" containerName="mariadb-account-delete" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.959327 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="5d11b02e-c14a-4d1a-819a-4c3b66aedd33" containerName="mariadb-account-delete" Dec 03 12:49:43 crc kubenswrapper[5052]: E1203 12:49:43.959338 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="bd4392a2-9c70-4f3a-9050-ae6a6db01082" containerName="container-auditor" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.959344 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd4392a2-9c70-4f3a-9050-ae6a6db01082" containerName="container-auditor" Dec 03 12:49:43 crc kubenswrapper[5052]: E1203 12:49:43.959354 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="5d11b02e-c14a-4d1a-819a-4c3b66aedd33" containerName="mariadb-account-delete" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.959361 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="5d11b02e-c14a-4d1a-819a-4c3b66aedd33" containerName="mariadb-account-delete" Dec 03 12:49:43 crc kubenswrapper[5052]: E1203 12:49:43.959375 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="aa557411-6b0a-4596-a5f5-0e23f8686dc4" containerName="probe" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.959383 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa557411-6b0a-4596-a5f5-0e23f8686dc4" containerName="probe" Dec 03 12:49:43 crc kubenswrapper[5052]: E1203 12:49:43.959394 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="8669832d-c47a-49bd-9491-dc7141ac6281" containerName="nova-cell1-conductor-conductor" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.959400 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="8669832d-c47a-49bd-9491-dc7141ac6281" containerName="nova-cell1-conductor-conductor" Dec 03 12:49:43 crc kubenswrapper[5052]: E1203 12:49:43.959407 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="6a5de7fb-b482-4849-a957-e679616aabc0" containerName="mariadb-account-delete" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.959413 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="6a5de7fb-b482-4849-a957-e679616aabc0" containerName="mariadb-account-delete" Dec 03 12:49:43 crc kubenswrapper[5052]: E1203 12:49:43.959422 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="557175a9-81e5-4605-bd67-f1dfa575fb1b" containerName="cinder-api-log" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.959427 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="557175a9-81e5-4605-bd67-f1dfa575fb1b" containerName="cinder-api-log" Dec 03 12:49:43 crc kubenswrapper[5052]: E1203 12:49:43.959438 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="bd4392a2-9c70-4f3a-9050-ae6a6db01082" containerName="account-reaper" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.959445 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd4392a2-9c70-4f3a-9050-ae6a6db01082" containerName="account-reaper" Dec 03 12:49:43 crc kubenswrapper[5052]: E1203 12:49:43.959454 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="81709983-1fd0-4446-bbbf-fb94653bb3d2" containerName="nova-scheduler-scheduler" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.959461 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="81709983-1fd0-4446-bbbf-fb94653bb3d2" containerName="nova-scheduler-scheduler" Dec 03 12:49:43 crc kubenswrapper[5052]: E1203 12:49:43.959468 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="bd4392a2-9c70-4f3a-9050-ae6a6db01082" containerName="object-replicator" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.959475 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd4392a2-9c70-4f3a-9050-ae6a6db01082" containerName="object-replicator" Dec 03 12:49:43 crc kubenswrapper[5052]: E1203 12:49:43.959485 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="dd8df313-3680-4dd1-a57d-ff85b0b2c6f1" containerName="proxy-httpd" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.959492 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="dd8df313-3680-4dd1-a57d-ff85b0b2c6f1" containerName="proxy-httpd" Dec 03 12:49:43 crc kubenswrapper[5052]: E1203 12:49:43.959500 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="b8eb690c-165a-4906-a750-bec0178daf6c" containerName="barbican-api" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.959524 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="b8eb690c-165a-4906-a750-bec0178daf6c" containerName="barbican-api" Dec 03 12:49:43 crc kubenswrapper[5052]: E1203 12:49:43.959536 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="bd4392a2-9c70-4f3a-9050-ae6a6db01082" containerName="container-server" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.959543 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd4392a2-9c70-4f3a-9050-ae6a6db01082" containerName="container-server" Dec 03 12:49:43 crc kubenswrapper[5052]: E1203 12:49:43.959556 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="77825998-48a8-496e-a523-8e0c371fe532" containerName="neutron-httpd" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.959561 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="77825998-48a8-496e-a523-8e0c371fe532" containerName="neutron-httpd" Dec 03 12:49:43 crc kubenswrapper[5052]: E1203 12:49:43.959569 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="557175a9-81e5-4605-bd67-f1dfa575fb1b" containerName="cinder-api" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.959576 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="557175a9-81e5-4605-bd67-f1dfa575fb1b" containerName="cinder-api" Dec 03 12:49:43 crc kubenswrapper[5052]: E1203 12:49:43.959589 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="ee63b3ae-44c4-4225-8373-75f5950af9d5" containerName="dnsmasq-dns" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.959596 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee63b3ae-44c4-4225-8373-75f5950af9d5" containerName="dnsmasq-dns" Dec 03 12:49:43 crc kubenswrapper[5052]: E1203 12:49:43.959608 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="b8eb690c-165a-4906-a750-bec0178daf6c" containerName="barbican-api-log" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.959614 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="b8eb690c-165a-4906-a750-bec0178daf6c" containerName="barbican-api-log" Dec 03 12:49:43 crc kubenswrapper[5052]: E1203 12:49:43.959625 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="84f26afe-5b0c-43bb-a4c8-a9ac21d24330" containerName="ovsdbserver-sb" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.959633 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="84f26afe-5b0c-43bb-a4c8-a9ac21d24330" containerName="ovsdbserver-sb" Dec 03 12:49:43 crc kubenswrapper[5052]: E1203 12:49:43.959644 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="6ceba96c-7e86-4eb2-afa7-444f0fcc27ce" containerName="mariadb-account-delete" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.959652 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="6ceba96c-7e86-4eb2-afa7-444f0fcc27ce" containerName="mariadb-account-delete" Dec 03 12:49:43 crc kubenswrapper[5052]: E1203 12:49:43.959658 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="81709983-1fd0-4446-bbbf-fb94653bb3d2" containerName="nova-scheduler-scheduler" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.959664 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="81709983-1fd0-4446-bbbf-fb94653bb3d2" containerName="nova-scheduler-scheduler" Dec 03 12:49:43 crc kubenswrapper[5052]: E1203 12:49:43.959671 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="81709983-1fd0-4446-bbbf-fb94653bb3d2" containerName="nova-scheduler-scheduler" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.959676 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="81709983-1fd0-4446-bbbf-fb94653bb3d2" containerName="nova-scheduler-scheduler" Dec 03 12:49:43 crc kubenswrapper[5052]: E1203 12:49:43.959683 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="6d653682-52e2-4940-af66-633475bfe09a" containerName="nova-cell0-conductor-conductor" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.959688 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="6d653682-52e2-4940-af66-633475bfe09a" containerName="nova-cell0-conductor-conductor" Dec 03 12:49:43 crc kubenswrapper[5052]: E1203 12:49:43.959695 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="bd4392a2-9c70-4f3a-9050-ae6a6db01082" containerName="container-replicator" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.959700 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd4392a2-9c70-4f3a-9050-ae6a6db01082" containerName="container-replicator" Dec 03 12:49:43 crc kubenswrapper[5052]: E1203 12:49:43.959707 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="a2bc501c-ebef-4e68-8ae2-e8c4217cadff" containerName="mariadb-account-delete" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.959712 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="a2bc501c-ebef-4e68-8ae2-e8c4217cadff" containerName="mariadb-account-delete" Dec 03 12:49:43 crc kubenswrapper[5052]: E1203 12:49:43.959718 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="e14465ed-868a-49ba-b3ab-1e67196229e8" containerName="barbican-worker" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.959724 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="e14465ed-868a-49ba-b3ab-1e67196229e8" containerName="barbican-worker" Dec 03 12:49:43 crc kubenswrapper[5052]: E1203 12:49:43.959732 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="aa557411-6b0a-4596-a5f5-0e23f8686dc4" containerName="cinder-scheduler" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.959738 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa557411-6b0a-4596-a5f5-0e23f8686dc4" containerName="cinder-scheduler" Dec 03 12:49:43 crc kubenswrapper[5052]: E1203 12:49:43.959746 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="84f26afe-5b0c-43bb-a4c8-a9ac21d24330" containerName="openstack-network-exporter" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.959752 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="84f26afe-5b0c-43bb-a4c8-a9ac21d24330" containerName="openstack-network-exporter" Dec 03 12:49:43 crc kubenswrapper[5052]: E1203 12:49:43.959760 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="81709983-1fd0-4446-bbbf-fb94653bb3d2" containerName="nova-scheduler-scheduler" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.959766 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="81709983-1fd0-4446-bbbf-fb94653bb3d2" containerName="nova-scheduler-scheduler" Dec 03 12:49:43 crc kubenswrapper[5052]: E1203 12:49:43.959775 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="488c3529-4c2b-47c0-9dcf-05a064c34d0d" containerName="rabbitmq" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.959781 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="488c3529-4c2b-47c0-9dcf-05a064c34d0d" containerName="rabbitmq" Dec 03 12:49:43 crc kubenswrapper[5052]: E1203 12:49:43.959790 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="342cba80-22e1-488a-bbfb-75b67ca536ac" containerName="ovn-northd" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.959795 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="342cba80-22e1-488a-bbfb-75b67ca536ac" containerName="ovn-northd" Dec 03 12:49:43 crc kubenswrapper[5052]: E1203 12:49:43.959802 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="eef9c304-b598-4a31-9c47-09be0087c25e" containerName="memcached" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.959808 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="eef9c304-b598-4a31-9c47-09be0087c25e" containerName="memcached" Dec 03 12:49:43 crc kubenswrapper[5052]: E1203 12:49:43.959814 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="bff27172-5e30-474d-a49c-389a35dd3052" containerName="keystone-api" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.959819 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="bff27172-5e30-474d-a49c-389a35dd3052" containerName="keystone-api" Dec 03 12:49:43 crc kubenswrapper[5052]: E1203 12:49:43.959829 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="6d653682-52e2-4940-af66-633475bfe09a" containerName="nova-cell0-conductor-conductor" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.959835 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="6d653682-52e2-4940-af66-633475bfe09a" containerName="nova-cell0-conductor-conductor" Dec 03 12:49:43 crc kubenswrapper[5052]: E1203 12:49:43.959843 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="6ceba96c-7e86-4eb2-afa7-444f0fcc27ce" containerName="mariadb-account-delete" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.959924 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="6ceba96c-7e86-4eb2-afa7-444f0fcc27ce" containerName="mariadb-account-delete" Dec 03 12:49:43 crc kubenswrapper[5052]: E1203 12:49:43.959934 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="857b8a8a-d730-4832-a71c-d45a2f3ce406" containerName="mariadb-account-delete" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.959941 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="857b8a8a-d730-4832-a71c-d45a2f3ce406" containerName="mariadb-account-delete" Dec 03 12:49:43 crc kubenswrapper[5052]: E1203 12:49:43.959949 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="bd4392a2-9c70-4f3a-9050-ae6a6db01082" containerName="swift-recon-cron" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.959954 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd4392a2-9c70-4f3a-9050-ae6a6db01082" containerName="swift-recon-cron" Dec 03 12:49:43 crc kubenswrapper[5052]: E1203 12:49:43.959966 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="e14465ed-868a-49ba-b3ab-1e67196229e8" containerName="barbican-worker-log" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.959972 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="e14465ed-868a-49ba-b3ab-1e67196229e8" containerName="barbican-worker-log" Dec 03 12:49:43 crc kubenswrapper[5052]: E1203 12:49:43.959982 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="8669832d-c47a-49bd-9491-dc7141ac6281" containerName="nova-cell1-conductor-conductor" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.959988 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="8669832d-c47a-49bd-9491-dc7141ac6281" containerName="nova-cell1-conductor-conductor" Dec 03 12:49:43 crc kubenswrapper[5052]: E1203 12:49:43.959995 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="6d653682-52e2-4940-af66-633475bfe09a" containerName="nova-cell0-conductor-conductor" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.960001 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="6d653682-52e2-4940-af66-633475bfe09a" containerName="nova-cell0-conductor-conductor" Dec 03 12:49:43 crc kubenswrapper[5052]: E1203 12:49:43.960010 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="098a46f8-2d49-4fea-a95c-5678601aa164" containerName="glance-httpd" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.960016 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="098a46f8-2d49-4fea-a95c-5678601aa164" containerName="glance-httpd" Dec 03 12:49:43 crc kubenswrapper[5052]: E1203 12:49:43.960025 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="b1a74998-b009-4bb6-a0dc-d94ffae8b641" containerName="glance-httpd" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.960031 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="b1a74998-b009-4bb6-a0dc-d94ffae8b641" containerName="glance-httpd" Dec 03 12:49:43 crc kubenswrapper[5052]: E1203 12:49:43.960038 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="ee63b3ae-44c4-4225-8373-75f5950af9d5" containerName="init" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.960043 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee63b3ae-44c4-4225-8373-75f5950af9d5" containerName="init" Dec 03 12:49:43 crc kubenswrapper[5052]: E1203 12:49:43.960050 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="efb0e60c-2a71-4641-87b2-0f5532b28678" containerName="nova-metadata-metadata" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.960056 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="efb0e60c-2a71-4641-87b2-0f5532b28678" containerName="nova-metadata-metadata" Dec 03 12:49:43 crc kubenswrapper[5052]: E1203 12:49:43.960067 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="77825998-48a8-496e-a523-8e0c371fe532" containerName="neutron-api" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.960073 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="77825998-48a8-496e-a523-8e0c371fe532" containerName="neutron-api" Dec 03 12:49:43 crc kubenswrapper[5052]: E1203 12:49:43.960084 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="bd4392a2-9c70-4f3a-9050-ae6a6db01082" containerName="object-server" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.960090 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd4392a2-9c70-4f3a-9050-ae6a6db01082" containerName="object-server" Dec 03 12:49:43 crc kubenswrapper[5052]: E1203 12:49:43.960099 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="5201872c-fde1-49d6-9784-6a7f54497990" containerName="placement-api" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.960105 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="5201872c-fde1-49d6-9784-6a7f54497990" containerName="placement-api" Dec 03 12:49:43 crc kubenswrapper[5052]: E1203 12:49:43.960114 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="3c8e4e3f-f6c1-4362-b64c-b10a973723dd" containerName="mysql-bootstrap" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.960119 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="3c8e4e3f-f6c1-4362-b64c-b10a973723dd" containerName="mysql-bootstrap" Dec 03 12:49:43 crc kubenswrapper[5052]: E1203 12:49:43.960128 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="6c8ad676-e7dd-479c-8d22-f546dc5e50af" containerName="nova-api-api" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.960135 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="6c8ad676-e7dd-479c-8d22-f546dc5e50af" containerName="nova-api-api" Dec 03 12:49:43 crc kubenswrapper[5052]: E1203 12:49:43.960142 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="3c8e4e3f-f6c1-4362-b64c-b10a973723dd" containerName="galera" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.960148 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="3c8e4e3f-f6c1-4362-b64c-b10a973723dd" containerName="galera" Dec 03 12:49:43 crc kubenswrapper[5052]: E1203 12:49:43.960155 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="6c8ad676-e7dd-479c-8d22-f546dc5e50af" containerName="nova-api-log" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.960161 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="6c8ad676-e7dd-479c-8d22-f546dc5e50af" containerName="nova-api-log" Dec 03 12:49:43 crc kubenswrapper[5052]: E1203 12:49:43.960171 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="bd4392a2-9c70-4f3a-9050-ae6a6db01082" containerName="object-auditor" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.960177 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd4392a2-9c70-4f3a-9050-ae6a6db01082" containerName="object-auditor" Dec 03 12:49:43 crc kubenswrapper[5052]: E1203 12:49:43.960188 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="a69bfb18-1267-4383-b51a-5e0eba70d1bc" containerName="ceilometer-central-agent" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.960195 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="a69bfb18-1267-4383-b51a-5e0eba70d1bc" containerName="ceilometer-central-agent" Dec 03 12:49:43 crc kubenswrapper[5052]: E1203 12:49:43.960202 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="0cb33a40-6f0a-4e89-a159-4edd3af3a2fa" containerName="setup-container" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.960208 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="0cb33a40-6f0a-4e89-a159-4edd3af3a2fa" containerName="setup-container" Dec 03 12:49:43 crc kubenswrapper[5052]: E1203 12:49:43.960215 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="bd4392a2-9c70-4f3a-9050-ae6a6db01082" containerName="rsync" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.960221 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd4392a2-9c70-4f3a-9050-ae6a6db01082" containerName="rsync" Dec 03 12:49:43 crc kubenswrapper[5052]: E1203 12:49:43.960230 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="bd4392a2-9c70-4f3a-9050-ae6a6db01082" containerName="object-updater" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.960235 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd4392a2-9c70-4f3a-9050-ae6a6db01082" containerName="object-updater" Dec 03 12:49:43 crc kubenswrapper[5052]: E1203 12:49:43.960243 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="a69bfb18-1267-4383-b51a-5e0eba70d1bc" containerName="sg-core" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.960249 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="a69bfb18-1267-4383-b51a-5e0eba70d1bc" containerName="sg-core" Dec 03 12:49:43 crc kubenswrapper[5052]: E1203 12:49:43.960256 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="dd8df313-3680-4dd1-a57d-ff85b0b2c6f1" containerName="proxy-server" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.960262 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="dd8df313-3680-4dd1-a57d-ff85b0b2c6f1" containerName="proxy-server" Dec 03 12:49:43 crc kubenswrapper[5052]: E1203 12:49:43.960270 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="a69bfb18-1267-4383-b51a-5e0eba70d1bc" containerName="ceilometer-notification-agent" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.960275 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="a69bfb18-1267-4383-b51a-5e0eba70d1bc" containerName="ceilometer-notification-agent" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.960425 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="2065f5bb-88eb-4e84-88b6-9982833d413d" containerName="barbican-keystone-listener-log" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.960437 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="81709983-1fd0-4446-bbbf-fb94653bb3d2" containerName="nova-scheduler-scheduler" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.960442 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="3c8e4e3f-f6c1-4362-b64c-b10a973723dd" containerName="galera" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.960455 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="8669832d-c47a-49bd-9491-dc7141ac6281" containerName="nova-cell1-conductor-conductor" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.960460 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="5201872c-fde1-49d6-9784-6a7f54497990" containerName="placement-log" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.960471 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="5d11b02e-c14a-4d1a-819a-4c3b66aedd33" containerName="mariadb-account-delete" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.960483 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="a69bfb18-1267-4383-b51a-5e0eba70d1bc" containerName="proxy-httpd" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.960493 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="5d11b02e-c14a-4d1a-819a-4c3b66aedd33" containerName="mariadb-account-delete" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.960519 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="84f26afe-5b0c-43bb-a4c8-a9ac21d24330" containerName="openstack-network-exporter" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.960527 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="342cba80-22e1-488a-bbfb-75b67ca536ac" containerName="openstack-network-exporter" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.960535 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="bd4392a2-9c70-4f3a-9050-ae6a6db01082" containerName="account-auditor" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.960544 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="bd4392a2-9c70-4f3a-9050-ae6a6db01082" containerName="object-replicator" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.960550 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="6a5de7fb-b482-4849-a957-e679616aabc0" containerName="mariadb-account-delete" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.960556 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="bd4392a2-9c70-4f3a-9050-ae6a6db01082" containerName="account-server" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.960565 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="bd4392a2-9c70-4f3a-9050-ae6a6db01082" containerName="container-updater" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.960572 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="bd4392a2-9c70-4f3a-9050-ae6a6db01082" containerName="account-reaper" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.960580 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="488c3529-4c2b-47c0-9dcf-05a064c34d0d" containerName="rabbitmq" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.960585 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="ee63b3ae-44c4-4225-8373-75f5950af9d5" containerName="dnsmasq-dns" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.960595 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="bd4392a2-9c70-4f3a-9050-ae6a6db01082" containerName="account-replicator" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.960602 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="342cba80-22e1-488a-bbfb-75b67ca536ac" containerName="ovn-northd" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.960612 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="6c8ad676-e7dd-479c-8d22-f546dc5e50af" containerName="nova-api-api" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.960623 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="b1a74998-b009-4bb6-a0dc-d94ffae8b641" containerName="glance-log" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.960630 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="bd4392a2-9c70-4f3a-9050-ae6a6db01082" containerName="container-server" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.960639 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="bd4392a2-9c70-4f3a-9050-ae6a6db01082" containerName="object-server" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.960648 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="edae7848-51ff-40d6-83f2-ffff085f3f3e" containerName="nova-cell1-novncproxy-novncproxy" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.960657 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="b8eb690c-165a-4906-a750-bec0178daf6c" containerName="barbican-api" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.960664 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="bd4392a2-9c70-4f3a-9050-ae6a6db01082" containerName="object-expirer" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.960671 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="aa557411-6b0a-4596-a5f5-0e23f8686dc4" containerName="probe" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.960678 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="a69bfb18-1267-4383-b51a-5e0eba70d1bc" containerName="ceilometer-notification-agent" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.960686 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="bd4392a2-9c70-4f3a-9050-ae6a6db01082" containerName="object-updater" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.960693 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="e14465ed-868a-49ba-b3ab-1e67196229e8" containerName="barbican-worker-log" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.960699 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="098a46f8-2d49-4fea-a95c-5678601aa164" containerName="glance-httpd" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.960705 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="bff27172-5e30-474d-a49c-389a35dd3052" containerName="keystone-api" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.960713 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="efb0e60c-2a71-4641-87b2-0f5532b28678" containerName="nova-metadata-metadata" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.960719 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="8669832d-c47a-49bd-9491-dc7141ac6281" containerName="nova-cell1-conductor-conductor" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.960728 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="aa557411-6b0a-4596-a5f5-0e23f8686dc4" containerName="cinder-scheduler" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.960734 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="6d653682-52e2-4940-af66-633475bfe09a" containerName="nova-cell0-conductor-conductor" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.960743 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="6c8ad676-e7dd-479c-8d22-f546dc5e50af" containerName="nova-api-log" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.960751 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="efb0e60c-2a71-4641-87b2-0f5532b28678" containerName="nova-metadata-log" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.960759 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="2846c397-2869-49f3-aeca-cf4b56bc8834" containerName="mariadb-account-delete" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.960768 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="6ceba96c-7e86-4eb2-afa7-444f0fcc27ce" containerName="mariadb-account-delete" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.960774 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="dd8df313-3680-4dd1-a57d-ff85b0b2c6f1" containerName="proxy-httpd" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.960780 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="0cb33a40-6f0a-4e89-a159-4edd3af3a2fa" containerName="rabbitmq" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.960789 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="6d653682-52e2-4940-af66-633475bfe09a" containerName="nova-cell0-conductor-conductor" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.960795 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="2846c397-2869-49f3-aeca-cf4b56bc8834" containerName="mariadb-account-delete" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.960803 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="6d653682-52e2-4940-af66-633475bfe09a" containerName="nova-cell0-conductor-conductor" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.960811 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="857b8a8a-d730-4832-a71c-d45a2f3ce406" containerName="mariadb-account-delete" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.960817 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="098a46f8-2d49-4fea-a95c-5678601aa164" containerName="glance-log" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.960827 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="bd4392a2-9c70-4f3a-9050-ae6a6db01082" containerName="object-auditor" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.960833 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="456c3fb1-90c0-4d17-863e-414380759f97" containerName="mariadb-account-delete" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.960840 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="bd4392a2-9c70-4f3a-9050-ae6a6db01082" containerName="container-auditor" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.960849 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="6d653682-52e2-4940-af66-633475bfe09a" containerName="nova-cell0-conductor-conductor" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.960857 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="a69bfb18-1267-4383-b51a-5e0eba70d1bc" containerName="sg-core" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.960866 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="81709983-1fd0-4446-bbbf-fb94653bb3d2" containerName="nova-scheduler-scheduler" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.960873 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="a2bc501c-ebef-4e68-8ae2-e8c4217cadff" containerName="mariadb-account-delete" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.960881 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="b8eb690c-165a-4906-a750-bec0178daf6c" containerName="barbican-api-log" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.960889 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="3479f9ad-871f-463e-96ce-50f4069088b3" containerName="galera" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.960899 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="81709983-1fd0-4446-bbbf-fb94653bb3d2" containerName="nova-scheduler-scheduler" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.960906 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="557175a9-81e5-4605-bd67-f1dfa575fb1b" containerName="cinder-api-log" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.960913 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="a2bc501c-ebef-4e68-8ae2-e8c4217cadff" containerName="mariadb-account-delete" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.960920 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="2005f89c-8a19-403c-acc9-c13ede717372" containerName="openstack-network-exporter" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.960927 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="84f26afe-5b0c-43bb-a4c8-a9ac21d24330" containerName="ovsdbserver-sb" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.960935 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="77825998-48a8-496e-a523-8e0c371fe532" containerName="neutron-api" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.960943 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="876d06e6-0187-4e3d-9183-ed2083fe5189" containerName="kube-state-metrics" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.960951 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="2005f89c-8a19-403c-acc9-c13ede717372" containerName="ovsdbserver-nb" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.960960 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="557175a9-81e5-4605-bd67-f1dfa575fb1b" containerName="cinder-api" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.960968 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="2065f5bb-88eb-4e84-88b6-9982833d413d" containerName="barbican-keystone-listener" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.960977 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="81709983-1fd0-4446-bbbf-fb94653bb3d2" containerName="nova-scheduler-scheduler" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.960984 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="77825998-48a8-496e-a523-8e0c371fe532" containerName="neutron-httpd" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.960991 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="bd4392a2-9c70-4f3a-9050-ae6a6db01082" containerName="container-replicator" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.960997 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="6ceba96c-7e86-4eb2-afa7-444f0fcc27ce" containerName="mariadb-account-delete" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.961004 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="eef9c304-b598-4a31-9c47-09be0087c25e" containerName="memcached" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.961013 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="bd4392a2-9c70-4f3a-9050-ae6a6db01082" containerName="rsync" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.961020 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="a69bfb18-1267-4383-b51a-5e0eba70d1bc" containerName="ceilometer-central-agent" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.961028 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="5201872c-fde1-49d6-9784-6a7f54497990" containerName="placement-api" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.961034 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="bd4392a2-9c70-4f3a-9050-ae6a6db01082" containerName="swift-recon-cron" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.961041 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="8669832d-c47a-49bd-9491-dc7141ac6281" containerName="nova-cell1-conductor-conductor" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.961049 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="456c3fb1-90c0-4d17-863e-414380759f97" containerName="mariadb-account-delete" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.961056 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="e14465ed-868a-49ba-b3ab-1e67196229e8" containerName="barbican-worker" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.961063 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="dd8df313-3680-4dd1-a57d-ff85b0b2c6f1" containerName="proxy-server" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.961071 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="b1a74998-b009-4bb6-a0dc-d94ffae8b641" containerName="glance-httpd" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.961595 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="crc-storage/crc-storage-crc-jb5w2" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.966074 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"crc-storage"/"openshift-service-ca.crt" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.966285 5052 reflector.go:368] Caches populated for *v1.Secret from object-"crc-storage"/"crc-storage-dockercfg-mf76g" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.966361 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["crc-storage/crc-storage-crc-jb5w2"] Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.966383 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"crc-storage"/"kube-root-ca.crt" Dec 03 12:49:43 crc kubenswrapper[5052]: I1203 12:49:43.966525 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"crc-storage"/"crc-storage" Dec 03 12:49:44 crc kubenswrapper[5052]: I1203 12:49:44.143034 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/021ea517-b7b9-493e-ac58-efc4e2cdfbb8-node-mnt\") pod \"crc-storage-crc-jb5w2\" (UID: \"021ea517-b7b9-493e-ac58-efc4e2cdfbb8\") " pod="crc-storage/crc-storage-crc-jb5w2" Dec 03 12:49:44 crc kubenswrapper[5052]: I1203 12:49:44.143454 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/021ea517-b7b9-493e-ac58-efc4e2cdfbb8-crc-storage\") pod \"crc-storage-crc-jb5w2\" (UID: \"021ea517-b7b9-493e-ac58-efc4e2cdfbb8\") " pod="crc-storage/crc-storage-crc-jb5w2" Dec 03 12:49:44 crc kubenswrapper[5052]: I1203 12:49:44.143573 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4zqbq\" (UniqueName: \"kubernetes.io/projected/021ea517-b7b9-493e-ac58-efc4e2cdfbb8-kube-api-access-4zqbq\") pod \"crc-storage-crc-jb5w2\" (UID: \"021ea517-b7b9-493e-ac58-efc4e2cdfbb8\") " pod="crc-storage/crc-storage-crc-jb5w2" Dec 03 12:49:44 crc kubenswrapper[5052]: I1203 12:49:44.244668 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-4zqbq\" (UniqueName: \"kubernetes.io/projected/021ea517-b7b9-493e-ac58-efc4e2cdfbb8-kube-api-access-4zqbq\") pod \"crc-storage-crc-jb5w2\" (UID: \"021ea517-b7b9-493e-ac58-efc4e2cdfbb8\") " pod="crc-storage/crc-storage-crc-jb5w2" Dec 03 12:49:44 crc kubenswrapper[5052]: I1203 12:49:44.244766 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/021ea517-b7b9-493e-ac58-efc4e2cdfbb8-node-mnt\") pod \"crc-storage-crc-jb5w2\" (UID: \"021ea517-b7b9-493e-ac58-efc4e2cdfbb8\") " pod="crc-storage/crc-storage-crc-jb5w2" Dec 03 12:49:44 crc kubenswrapper[5052]: I1203 12:49:44.244809 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/021ea517-b7b9-493e-ac58-efc4e2cdfbb8-crc-storage\") pod \"crc-storage-crc-jb5w2\" (UID: \"021ea517-b7b9-493e-ac58-efc4e2cdfbb8\") " pod="crc-storage/crc-storage-crc-jb5w2" Dec 03 12:49:44 crc kubenswrapper[5052]: I1203 12:49:44.245312 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/021ea517-b7b9-493e-ac58-efc4e2cdfbb8-node-mnt\") pod \"crc-storage-crc-jb5w2\" (UID: \"021ea517-b7b9-493e-ac58-efc4e2cdfbb8\") " pod="crc-storage/crc-storage-crc-jb5w2" Dec 03 12:49:44 crc kubenswrapper[5052]: I1203 12:49:44.245778 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/021ea517-b7b9-493e-ac58-efc4e2cdfbb8-crc-storage\") pod \"crc-storage-crc-jb5w2\" (UID: \"021ea517-b7b9-493e-ac58-efc4e2cdfbb8\") " pod="crc-storage/crc-storage-crc-jb5w2" Dec 03 12:49:44 crc kubenswrapper[5052]: I1203 12:49:44.266416 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-4zqbq\" (UniqueName: \"kubernetes.io/projected/021ea517-b7b9-493e-ac58-efc4e2cdfbb8-kube-api-access-4zqbq\") pod \"crc-storage-crc-jb5w2\" (UID: \"021ea517-b7b9-493e-ac58-efc4e2cdfbb8\") " pod="crc-storage/crc-storage-crc-jb5w2" Dec 03 12:49:44 crc kubenswrapper[5052]: I1203 12:49:44.280973 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="crc-storage/crc-storage-crc-jb5w2" Dec 03 12:49:44 crc kubenswrapper[5052]: I1203 12:49:44.783603 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["crc-storage/crc-storage-crc-jb5w2"] Dec 03 12:49:45 crc kubenswrapper[5052]: I1203 12:49:45.587130 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="crc-storage/crc-storage-crc-jb5w2" event={"ID":"021ea517-b7b9-493e-ac58-efc4e2cdfbb8","Type":"ContainerStarted","Data":"9bb9bf8967b521c1d44404dd2ed1c484593b76d4621c4b47a49b8f616824a1f3"} Dec 03 12:49:45 crc kubenswrapper[5052]: I1203 12:49:45.682489 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b6dc9243-9b95-4639-b4b9-a0adce5f25b8" path="/var/lib/kubelet/pods/b6dc9243-9b95-4639-b4b9-a0adce5f25b8/volumes" Dec 03 12:49:47 crc kubenswrapper[5052]: I1203 12:49:47.603639 5052 generic.go:334] "Generic (PLEG): container finished" podID="021ea517-b7b9-493e-ac58-efc4e2cdfbb8" containerID="ce01d82e85a4543523154c1e02105bb4626f45bd57dd40b49d3a01ee746c0546" exitCode=0 Dec 03 12:49:47 crc kubenswrapper[5052]: I1203 12:49:47.603698 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="crc-storage/crc-storage-crc-jb5w2" event={"ID":"021ea517-b7b9-493e-ac58-efc4e2cdfbb8","Type":"ContainerDied","Data":"ce01d82e85a4543523154c1e02105bb4626f45bd57dd40b49d3a01ee746c0546"} Dec 03 12:49:48 crc kubenswrapper[5052]: I1203 12:49:48.890469 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="crc-storage/crc-storage-crc-jb5w2" Dec 03 12:49:49 crc kubenswrapper[5052]: I1203 12:49:49.050538 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/021ea517-b7b9-493e-ac58-efc4e2cdfbb8-node-mnt\") pod \"021ea517-b7b9-493e-ac58-efc4e2cdfbb8\" (UID: \"021ea517-b7b9-493e-ac58-efc4e2cdfbb8\") " Dec 03 12:49:49 crc kubenswrapper[5052]: I1203 12:49:49.050672 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4zqbq\" (UniqueName: \"kubernetes.io/projected/021ea517-b7b9-493e-ac58-efc4e2cdfbb8-kube-api-access-4zqbq\") pod \"021ea517-b7b9-493e-ac58-efc4e2cdfbb8\" (UID: \"021ea517-b7b9-493e-ac58-efc4e2cdfbb8\") " Dec 03 12:49:49 crc kubenswrapper[5052]: I1203 12:49:49.050688 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/021ea517-b7b9-493e-ac58-efc4e2cdfbb8-node-mnt" (OuterVolumeSpecName: "node-mnt") pod "021ea517-b7b9-493e-ac58-efc4e2cdfbb8" (UID: "021ea517-b7b9-493e-ac58-efc4e2cdfbb8"). InnerVolumeSpecName "node-mnt". PluginName "kubernetes.io/host-path", VolumeGidValue "" Dec 03 12:49:49 crc kubenswrapper[5052]: I1203 12:49:49.050797 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/021ea517-b7b9-493e-ac58-efc4e2cdfbb8-crc-storage\") pod \"021ea517-b7b9-493e-ac58-efc4e2cdfbb8\" (UID: \"021ea517-b7b9-493e-ac58-efc4e2cdfbb8\") " Dec 03 12:49:49 crc kubenswrapper[5052]: I1203 12:49:49.051039 5052 reconciler_common.go:293] "Volume detached for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/021ea517-b7b9-493e-ac58-efc4e2cdfbb8-node-mnt\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:49 crc kubenswrapper[5052]: I1203 12:49:49.057001 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/021ea517-b7b9-493e-ac58-efc4e2cdfbb8-kube-api-access-4zqbq" (OuterVolumeSpecName: "kube-api-access-4zqbq") pod "021ea517-b7b9-493e-ac58-efc4e2cdfbb8" (UID: "021ea517-b7b9-493e-ac58-efc4e2cdfbb8"). InnerVolumeSpecName "kube-api-access-4zqbq". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:49:49 crc kubenswrapper[5052]: I1203 12:49:49.070543 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/021ea517-b7b9-493e-ac58-efc4e2cdfbb8-crc-storage" (OuterVolumeSpecName: "crc-storage") pod "021ea517-b7b9-493e-ac58-efc4e2cdfbb8" (UID: "021ea517-b7b9-493e-ac58-efc4e2cdfbb8"). InnerVolumeSpecName "crc-storage". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:49:49 crc kubenswrapper[5052]: I1203 12:49:49.151867 5052 reconciler_common.go:293] "Volume detached for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/021ea517-b7b9-493e-ac58-efc4e2cdfbb8-crc-storage\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:49 crc kubenswrapper[5052]: I1203 12:49:49.151903 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-4zqbq\" (UniqueName: \"kubernetes.io/projected/021ea517-b7b9-493e-ac58-efc4e2cdfbb8-kube-api-access-4zqbq\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:49 crc kubenswrapper[5052]: I1203 12:49:49.621366 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="crc-storage/crc-storage-crc-jb5w2" event={"ID":"021ea517-b7b9-493e-ac58-efc4e2cdfbb8","Type":"ContainerDied","Data":"9bb9bf8967b521c1d44404dd2ed1c484593b76d4621c4b47a49b8f616824a1f3"} Dec 03 12:49:49 crc kubenswrapper[5052]: I1203 12:49:49.621426 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9bb9bf8967b521c1d44404dd2ed1c484593b76d4621c4b47a49b8f616824a1f3" Dec 03 12:49:49 crc kubenswrapper[5052]: I1203 12:49:49.621498 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="crc-storage/crc-storage-crc-jb5w2" Dec 03 12:49:52 crc kubenswrapper[5052]: I1203 12:49:52.734627 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["crc-storage/crc-storage-crc-jb5w2"] Dec 03 12:49:52 crc kubenswrapper[5052]: I1203 12:49:52.742309 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["crc-storage/crc-storage-crc-jb5w2"] Dec 03 12:49:52 crc kubenswrapper[5052]: I1203 12:49:52.888432 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["crc-storage/crc-storage-crc-br5r4"] Dec 03 12:49:52 crc kubenswrapper[5052]: E1203 12:49:52.888938 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="81709983-1fd0-4446-bbbf-fb94653bb3d2" containerName="nova-scheduler-scheduler" Dec 03 12:49:52 crc kubenswrapper[5052]: I1203 12:49:52.888956 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="81709983-1fd0-4446-bbbf-fb94653bb3d2" containerName="nova-scheduler-scheduler" Dec 03 12:49:52 crc kubenswrapper[5052]: E1203 12:49:52.888985 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="857b8a8a-d730-4832-a71c-d45a2f3ce406" containerName="mariadb-account-delete" Dec 03 12:49:52 crc kubenswrapper[5052]: I1203 12:49:52.888993 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="857b8a8a-d730-4832-a71c-d45a2f3ce406" containerName="mariadb-account-delete" Dec 03 12:49:52 crc kubenswrapper[5052]: E1203 12:49:52.889003 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="8669832d-c47a-49bd-9491-dc7141ac6281" containerName="nova-cell1-conductor-conductor" Dec 03 12:49:52 crc kubenswrapper[5052]: I1203 12:49:52.889010 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="8669832d-c47a-49bd-9491-dc7141ac6281" containerName="nova-cell1-conductor-conductor" Dec 03 12:49:52 crc kubenswrapper[5052]: E1203 12:49:52.889026 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="6d653682-52e2-4940-af66-633475bfe09a" containerName="nova-cell0-conductor-conductor" Dec 03 12:49:52 crc kubenswrapper[5052]: I1203 12:49:52.889032 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="6d653682-52e2-4940-af66-633475bfe09a" containerName="nova-cell0-conductor-conductor" Dec 03 12:49:52 crc kubenswrapper[5052]: E1203 12:49:52.889054 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="2846c397-2869-49f3-aeca-cf4b56bc8834" containerName="mariadb-account-delete" Dec 03 12:49:52 crc kubenswrapper[5052]: I1203 12:49:52.889061 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="2846c397-2869-49f3-aeca-cf4b56bc8834" containerName="mariadb-account-delete" Dec 03 12:49:52 crc kubenswrapper[5052]: E1203 12:49:52.889079 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="021ea517-b7b9-493e-ac58-efc4e2cdfbb8" containerName="storage" Dec 03 12:49:52 crc kubenswrapper[5052]: I1203 12:49:52.889086 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="021ea517-b7b9-493e-ac58-efc4e2cdfbb8" containerName="storage" Dec 03 12:49:52 crc kubenswrapper[5052]: I1203 12:49:52.889261 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="81709983-1fd0-4446-bbbf-fb94653bb3d2" containerName="nova-scheduler-scheduler" Dec 03 12:49:52 crc kubenswrapper[5052]: I1203 12:49:52.889279 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="8669832d-c47a-49bd-9491-dc7141ac6281" containerName="nova-cell1-conductor-conductor" Dec 03 12:49:52 crc kubenswrapper[5052]: I1203 12:49:52.889290 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="6a5de7fb-b482-4849-a957-e679616aabc0" containerName="mariadb-account-delete" Dec 03 12:49:52 crc kubenswrapper[5052]: I1203 12:49:52.889303 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="857b8a8a-d730-4832-a71c-d45a2f3ce406" containerName="mariadb-account-delete" Dec 03 12:49:52 crc kubenswrapper[5052]: I1203 12:49:52.889311 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="6d653682-52e2-4940-af66-633475bfe09a" containerName="nova-cell0-conductor-conductor" Dec 03 12:49:52 crc kubenswrapper[5052]: I1203 12:49:52.889324 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="021ea517-b7b9-493e-ac58-efc4e2cdfbb8" containerName="storage" Dec 03 12:49:52 crc kubenswrapper[5052]: I1203 12:49:52.889992 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="crc-storage/crc-storage-crc-br5r4" Dec 03 12:49:52 crc kubenswrapper[5052]: I1203 12:49:52.895357 5052 reflector.go:368] Caches populated for *v1.Secret from object-"crc-storage"/"crc-storage-dockercfg-mf76g" Dec 03 12:49:52 crc kubenswrapper[5052]: I1203 12:49:52.895720 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"crc-storage"/"kube-root-ca.crt" Dec 03 12:49:52 crc kubenswrapper[5052]: I1203 12:49:52.895361 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"crc-storage"/"openshift-service-ca.crt" Dec 03 12:49:52 crc kubenswrapper[5052]: I1203 12:49:52.896050 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"crc-storage"/"crc-storage" Dec 03 12:49:52 crc kubenswrapper[5052]: I1203 12:49:52.898043 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["crc-storage/crc-storage-crc-br5r4"] Dec 03 12:49:53 crc kubenswrapper[5052]: I1203 12:49:53.009003 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/abdb8807-9006-4c35-9d4b-41dcd672416f-node-mnt\") pod \"crc-storage-crc-br5r4\" (UID: \"abdb8807-9006-4c35-9d4b-41dcd672416f\") " pod="crc-storage/crc-storage-crc-br5r4" Dec 03 12:49:53 crc kubenswrapper[5052]: I1203 12:49:53.009322 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/abdb8807-9006-4c35-9d4b-41dcd672416f-crc-storage\") pod \"crc-storage-crc-br5r4\" (UID: \"abdb8807-9006-4c35-9d4b-41dcd672416f\") " pod="crc-storage/crc-storage-crc-br5r4" Dec 03 12:49:53 crc kubenswrapper[5052]: I1203 12:49:53.009377 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9sdn6\" (UniqueName: \"kubernetes.io/projected/abdb8807-9006-4c35-9d4b-41dcd672416f-kube-api-access-9sdn6\") pod \"crc-storage-crc-br5r4\" (UID: \"abdb8807-9006-4c35-9d4b-41dcd672416f\") " pod="crc-storage/crc-storage-crc-br5r4" Dec 03 12:49:53 crc kubenswrapper[5052]: I1203 12:49:53.111860 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/abdb8807-9006-4c35-9d4b-41dcd672416f-crc-storage\") pod \"crc-storage-crc-br5r4\" (UID: \"abdb8807-9006-4c35-9d4b-41dcd672416f\") " pod="crc-storage/crc-storage-crc-br5r4" Dec 03 12:49:53 crc kubenswrapper[5052]: I1203 12:49:53.111939 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-9sdn6\" (UniqueName: \"kubernetes.io/projected/abdb8807-9006-4c35-9d4b-41dcd672416f-kube-api-access-9sdn6\") pod \"crc-storage-crc-br5r4\" (UID: \"abdb8807-9006-4c35-9d4b-41dcd672416f\") " pod="crc-storage/crc-storage-crc-br5r4" Dec 03 12:49:53 crc kubenswrapper[5052]: I1203 12:49:53.112068 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/abdb8807-9006-4c35-9d4b-41dcd672416f-node-mnt\") pod \"crc-storage-crc-br5r4\" (UID: \"abdb8807-9006-4c35-9d4b-41dcd672416f\") " pod="crc-storage/crc-storage-crc-br5r4" Dec 03 12:49:53 crc kubenswrapper[5052]: I1203 12:49:53.112562 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/abdb8807-9006-4c35-9d4b-41dcd672416f-node-mnt\") pod \"crc-storage-crc-br5r4\" (UID: \"abdb8807-9006-4c35-9d4b-41dcd672416f\") " pod="crc-storage/crc-storage-crc-br5r4" Dec 03 12:49:53 crc kubenswrapper[5052]: I1203 12:49:53.112898 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/abdb8807-9006-4c35-9d4b-41dcd672416f-crc-storage\") pod \"crc-storage-crc-br5r4\" (UID: \"abdb8807-9006-4c35-9d4b-41dcd672416f\") " pod="crc-storage/crc-storage-crc-br5r4" Dec 03 12:49:53 crc kubenswrapper[5052]: I1203 12:49:53.134695 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-9sdn6\" (UniqueName: \"kubernetes.io/projected/abdb8807-9006-4c35-9d4b-41dcd672416f-kube-api-access-9sdn6\") pod \"crc-storage-crc-br5r4\" (UID: \"abdb8807-9006-4c35-9d4b-41dcd672416f\") " pod="crc-storage/crc-storage-crc-br5r4" Dec 03 12:49:53 crc kubenswrapper[5052]: I1203 12:49:53.220540 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="crc-storage/crc-storage-crc-br5r4" Dec 03 12:49:53 crc kubenswrapper[5052]: I1203 12:49:53.627944 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["crc-storage/crc-storage-crc-br5r4"] Dec 03 12:49:53 crc kubenswrapper[5052]: I1203 12:49:53.662458 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="crc-storage/crc-storage-crc-br5r4" event={"ID":"abdb8807-9006-4c35-9d4b-41dcd672416f","Type":"ContainerStarted","Data":"2f11fe5a3f7fdff95899bd8b674264a7c4c0a0dd603e412388cd5cd7129c2cbc"} Dec 03 12:49:53 crc kubenswrapper[5052]: I1203 12:49:53.685708 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="021ea517-b7b9-493e-ac58-efc4e2cdfbb8" path="/var/lib/kubelet/pods/021ea517-b7b9-493e-ac58-efc4e2cdfbb8/volumes" Dec 03 12:49:54 crc kubenswrapper[5052]: I1203 12:49:54.672739 5052 generic.go:334] "Generic (PLEG): container finished" podID="abdb8807-9006-4c35-9d4b-41dcd672416f" containerID="909e761cd205a7fdfc227201b2626c6b3ce6dbab281adadf66ab0f0073e2725c" exitCode=0 Dec 03 12:49:54 crc kubenswrapper[5052]: I1203 12:49:54.673186 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="crc-storage/crc-storage-crc-br5r4" event={"ID":"abdb8807-9006-4c35-9d4b-41dcd672416f","Type":"ContainerDied","Data":"909e761cd205a7fdfc227201b2626c6b3ce6dbab281adadf66ab0f0073e2725c"} Dec 03 12:49:55 crc kubenswrapper[5052]: I1203 12:49:55.951871 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="crc-storage/crc-storage-crc-br5r4" Dec 03 12:49:56 crc kubenswrapper[5052]: I1203 12:49:56.059933 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/abdb8807-9006-4c35-9d4b-41dcd672416f-crc-storage\") pod \"abdb8807-9006-4c35-9d4b-41dcd672416f\" (UID: \"abdb8807-9006-4c35-9d4b-41dcd672416f\") " Dec 03 12:49:56 crc kubenswrapper[5052]: I1203 12:49:56.060051 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9sdn6\" (UniqueName: \"kubernetes.io/projected/abdb8807-9006-4c35-9d4b-41dcd672416f-kube-api-access-9sdn6\") pod \"abdb8807-9006-4c35-9d4b-41dcd672416f\" (UID: \"abdb8807-9006-4c35-9d4b-41dcd672416f\") " Dec 03 12:49:56 crc kubenswrapper[5052]: I1203 12:49:56.060092 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/abdb8807-9006-4c35-9d4b-41dcd672416f-node-mnt\") pod \"abdb8807-9006-4c35-9d4b-41dcd672416f\" (UID: \"abdb8807-9006-4c35-9d4b-41dcd672416f\") " Dec 03 12:49:56 crc kubenswrapper[5052]: I1203 12:49:56.060396 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/abdb8807-9006-4c35-9d4b-41dcd672416f-node-mnt" (OuterVolumeSpecName: "node-mnt") pod "abdb8807-9006-4c35-9d4b-41dcd672416f" (UID: "abdb8807-9006-4c35-9d4b-41dcd672416f"). InnerVolumeSpecName "node-mnt". PluginName "kubernetes.io/host-path", VolumeGidValue "" Dec 03 12:49:56 crc kubenswrapper[5052]: I1203 12:49:56.067762 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/abdb8807-9006-4c35-9d4b-41dcd672416f-kube-api-access-9sdn6" (OuterVolumeSpecName: "kube-api-access-9sdn6") pod "abdb8807-9006-4c35-9d4b-41dcd672416f" (UID: "abdb8807-9006-4c35-9d4b-41dcd672416f"). InnerVolumeSpecName "kube-api-access-9sdn6". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:49:56 crc kubenswrapper[5052]: I1203 12:49:56.082401 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/abdb8807-9006-4c35-9d4b-41dcd672416f-crc-storage" (OuterVolumeSpecName: "crc-storage") pod "abdb8807-9006-4c35-9d4b-41dcd672416f" (UID: "abdb8807-9006-4c35-9d4b-41dcd672416f"). InnerVolumeSpecName "crc-storage". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:49:56 crc kubenswrapper[5052]: I1203 12:49:56.162530 5052 reconciler_common.go:293] "Volume detached for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/abdb8807-9006-4c35-9d4b-41dcd672416f-crc-storage\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:56 crc kubenswrapper[5052]: I1203 12:49:56.162597 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-9sdn6\" (UniqueName: \"kubernetes.io/projected/abdb8807-9006-4c35-9d4b-41dcd672416f-kube-api-access-9sdn6\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:56 crc kubenswrapper[5052]: I1203 12:49:56.162610 5052 reconciler_common.go:293] "Volume detached for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/abdb8807-9006-4c35-9d4b-41dcd672416f-node-mnt\") on node \"crc\" DevicePath \"\"" Dec 03 12:49:56 crc kubenswrapper[5052]: I1203 12:49:56.693729 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="crc-storage/crc-storage-crc-br5r4" event={"ID":"abdb8807-9006-4c35-9d4b-41dcd672416f","Type":"ContainerDied","Data":"2f11fe5a3f7fdff95899bd8b674264a7c4c0a0dd603e412388cd5cd7129c2cbc"} Dec 03 12:49:56 crc kubenswrapper[5052]: I1203 12:49:56.693793 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2f11fe5a3f7fdff95899bd8b674264a7c4c0a0dd603e412388cd5cd7129c2cbc" Dec 03 12:49:56 crc kubenswrapper[5052]: I1203 12:49:56.693756 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="crc-storage/crc-storage-crc-br5r4" Dec 03 12:50:08 crc kubenswrapper[5052]: I1203 12:50:08.386675 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/rabbitmq-cell1-server-0"] Dec 03 12:50:08 crc kubenswrapper[5052]: E1203 12:50:08.387821 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="abdb8807-9006-4c35-9d4b-41dcd672416f" containerName="storage" Dec 03 12:50:08 crc kubenswrapper[5052]: I1203 12:50:08.387836 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="abdb8807-9006-4c35-9d4b-41dcd672416f" containerName="storage" Dec 03 12:50:08 crc kubenswrapper[5052]: I1203 12:50:08.388012 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="abdb8807-9006-4c35-9d4b-41dcd672416f" containerName="storage" Dec 03 12:50:08 crc kubenswrapper[5052]: I1203 12:50:08.388046 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="8669832d-c47a-49bd-9491-dc7141ac6281" containerName="nova-cell1-conductor-conductor" Dec 03 12:50:08 crc kubenswrapper[5052]: I1203 12:50:08.389071 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:50:08 crc kubenswrapper[5052]: I1203 12:50:08.393167 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"rabbitmq-cell1-config-data" Dec 03 12:50:08 crc kubenswrapper[5052]: I1203 12:50:08.393418 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"rabbitmq-cell1-server-dockercfg-zglz6" Dec 03 12:50:08 crc kubenswrapper[5052]: I1203 12:50:08.393640 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"rabbitmq-cell1-default-user" Dec 03 12:50:08 crc kubenswrapper[5052]: I1203 12:50:08.393927 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-rabbitmq-cell1-svc" Dec 03 12:50:08 crc kubenswrapper[5052]: I1203 12:50:08.393980 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"rabbitmq-cell1-plugins-conf" Dec 03 12:50:08 crc kubenswrapper[5052]: I1203 12:50:08.396367 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"rabbitmq-cell1-erlang-cookie" Dec 03 12:50:08 crc kubenswrapper[5052]: I1203 12:50:08.397383 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"rabbitmq-cell1-server-conf" Dec 03 12:50:08 crc kubenswrapper[5052]: I1203 12:50:08.413969 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/rabbitmq-cell1-server-0"] Dec 03 12:50:08 crc kubenswrapper[5052]: I1203 12:50:08.559549 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"rabbitmq-confd\" (UniqueName: \"kubernetes.io/projected/568addd3-4ad3-47c3-adae-693691e7cd40-rabbitmq-confd\") pod \"rabbitmq-cell1-server-0\" (UID: \"568addd3-4ad3-47c3-adae-693691e7cd40\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:50:08 crc kubenswrapper[5052]: I1203 12:50:08.559601 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-conf\" (UniqueName: \"kubernetes.io/configmap/568addd3-4ad3-47c3-adae-693691e7cd40-plugins-conf\") pod \"rabbitmq-cell1-server-0\" (UID: \"568addd3-4ad3-47c3-adae-693691e7cd40\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:50:08 crc kubenswrapper[5052]: I1203 12:50:08.559634 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage02-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage02-crc\") pod \"rabbitmq-cell1-server-0\" (UID: \"568addd3-4ad3-47c3-adae-693691e7cd40\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:50:08 crc kubenswrapper[5052]: I1203 12:50:08.559661 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"rabbitmq-erlang-cookie\" (UniqueName: \"kubernetes.io/empty-dir/568addd3-4ad3-47c3-adae-693691e7cd40-rabbitmq-erlang-cookie\") pod \"rabbitmq-cell1-server-0\" (UID: \"568addd3-4ad3-47c3-adae-693691e7cd40\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:50:08 crc kubenswrapper[5052]: I1203 12:50:08.559690 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/568addd3-4ad3-47c3-adae-693691e7cd40-config-data\") pod \"rabbitmq-cell1-server-0\" (UID: \"568addd3-4ad3-47c3-adae-693691e7cd40\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:50:08 crc kubenswrapper[5052]: I1203 12:50:08.559704 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"rabbitmq-tls\" (UniqueName: \"kubernetes.io/projected/568addd3-4ad3-47c3-adae-693691e7cd40-rabbitmq-tls\") pod \"rabbitmq-cell1-server-0\" (UID: \"568addd3-4ad3-47c3-adae-693691e7cd40\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:50:08 crc kubenswrapper[5052]: I1203 12:50:08.559727 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"rabbitmq-plugins\" (UniqueName: \"kubernetes.io/empty-dir/568addd3-4ad3-47c3-adae-693691e7cd40-rabbitmq-plugins\") pod \"rabbitmq-cell1-server-0\" (UID: \"568addd3-4ad3-47c3-adae-693691e7cd40\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:50:08 crc kubenswrapper[5052]: I1203 12:50:08.559741 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"pod-info\" (UniqueName: \"kubernetes.io/downward-api/568addd3-4ad3-47c3-adae-693691e7cd40-pod-info\") pod \"rabbitmq-cell1-server-0\" (UID: \"568addd3-4ad3-47c3-adae-693691e7cd40\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:50:08 crc kubenswrapper[5052]: I1203 12:50:08.559765 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"erlang-cookie-secret\" (UniqueName: \"kubernetes.io/secret/568addd3-4ad3-47c3-adae-693691e7cd40-erlang-cookie-secret\") pod \"rabbitmq-cell1-server-0\" (UID: \"568addd3-4ad3-47c3-adae-693691e7cd40\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:50:08 crc kubenswrapper[5052]: I1203 12:50:08.559793 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9hxs4\" (UniqueName: \"kubernetes.io/projected/568addd3-4ad3-47c3-adae-693691e7cd40-kube-api-access-9hxs4\") pod \"rabbitmq-cell1-server-0\" (UID: \"568addd3-4ad3-47c3-adae-693691e7cd40\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:50:08 crc kubenswrapper[5052]: I1203 12:50:08.559808 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"server-conf\" (UniqueName: \"kubernetes.io/configmap/568addd3-4ad3-47c3-adae-693691e7cd40-server-conf\") pod \"rabbitmq-cell1-server-0\" (UID: \"568addd3-4ad3-47c3-adae-693691e7cd40\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:50:08 crc kubenswrapper[5052]: I1203 12:50:08.660691 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"erlang-cookie-secret\" (UniqueName: \"kubernetes.io/secret/568addd3-4ad3-47c3-adae-693691e7cd40-erlang-cookie-secret\") pod \"rabbitmq-cell1-server-0\" (UID: \"568addd3-4ad3-47c3-adae-693691e7cd40\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:50:08 crc kubenswrapper[5052]: I1203 12:50:08.661790 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-9hxs4\" (UniqueName: \"kubernetes.io/projected/568addd3-4ad3-47c3-adae-693691e7cd40-kube-api-access-9hxs4\") pod \"rabbitmq-cell1-server-0\" (UID: \"568addd3-4ad3-47c3-adae-693691e7cd40\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:50:08 crc kubenswrapper[5052]: I1203 12:50:08.661830 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"server-conf\" (UniqueName: \"kubernetes.io/configmap/568addd3-4ad3-47c3-adae-693691e7cd40-server-conf\") pod \"rabbitmq-cell1-server-0\" (UID: \"568addd3-4ad3-47c3-adae-693691e7cd40\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:50:08 crc kubenswrapper[5052]: I1203 12:50:08.661898 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"rabbitmq-confd\" (UniqueName: \"kubernetes.io/projected/568addd3-4ad3-47c3-adae-693691e7cd40-rabbitmq-confd\") pod \"rabbitmq-cell1-server-0\" (UID: \"568addd3-4ad3-47c3-adae-693691e7cd40\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:50:08 crc kubenswrapper[5052]: I1203 12:50:08.661935 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"plugins-conf\" (UniqueName: \"kubernetes.io/configmap/568addd3-4ad3-47c3-adae-693691e7cd40-plugins-conf\") pod \"rabbitmq-cell1-server-0\" (UID: \"568addd3-4ad3-47c3-adae-693691e7cd40\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:50:08 crc kubenswrapper[5052]: I1203 12:50:08.662002 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage02-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage02-crc\") pod \"rabbitmq-cell1-server-0\" (UID: \"568addd3-4ad3-47c3-adae-693691e7cd40\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:50:08 crc kubenswrapper[5052]: I1203 12:50:08.662044 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"rabbitmq-erlang-cookie\" (UniqueName: \"kubernetes.io/empty-dir/568addd3-4ad3-47c3-adae-693691e7cd40-rabbitmq-erlang-cookie\") pod \"rabbitmq-cell1-server-0\" (UID: \"568addd3-4ad3-47c3-adae-693691e7cd40\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:50:08 crc kubenswrapper[5052]: I1203 12:50:08.662095 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/568addd3-4ad3-47c3-adae-693691e7cd40-config-data\") pod \"rabbitmq-cell1-server-0\" (UID: \"568addd3-4ad3-47c3-adae-693691e7cd40\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:50:08 crc kubenswrapper[5052]: I1203 12:50:08.662122 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"rabbitmq-tls\" (UniqueName: \"kubernetes.io/projected/568addd3-4ad3-47c3-adae-693691e7cd40-rabbitmq-tls\") pod \"rabbitmq-cell1-server-0\" (UID: \"568addd3-4ad3-47c3-adae-693691e7cd40\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:50:08 crc kubenswrapper[5052]: I1203 12:50:08.662148 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"pod-info\" (UniqueName: \"kubernetes.io/downward-api/568addd3-4ad3-47c3-adae-693691e7cd40-pod-info\") pod \"rabbitmq-cell1-server-0\" (UID: \"568addd3-4ad3-47c3-adae-693691e7cd40\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:50:08 crc kubenswrapper[5052]: I1203 12:50:08.662172 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"rabbitmq-plugins\" (UniqueName: \"kubernetes.io/empty-dir/568addd3-4ad3-47c3-adae-693691e7cd40-rabbitmq-plugins\") pod \"rabbitmq-cell1-server-0\" (UID: \"568addd3-4ad3-47c3-adae-693691e7cd40\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:50:08 crc kubenswrapper[5052]: I1203 12:50:08.662822 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage02-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage02-crc\") pod \"rabbitmq-cell1-server-0\" (UID: \"568addd3-4ad3-47c3-adae-693691e7cd40\") device mount path \"/mnt/openstack/pv02\"" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:50:08 crc kubenswrapper[5052]: I1203 12:50:08.662914 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"rabbitmq-plugins\" (UniqueName: \"kubernetes.io/empty-dir/568addd3-4ad3-47c3-adae-693691e7cd40-rabbitmq-plugins\") pod \"rabbitmq-cell1-server-0\" (UID: \"568addd3-4ad3-47c3-adae-693691e7cd40\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:50:08 crc kubenswrapper[5052]: I1203 12:50:08.663641 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"rabbitmq-erlang-cookie\" (UniqueName: \"kubernetes.io/empty-dir/568addd3-4ad3-47c3-adae-693691e7cd40-rabbitmq-erlang-cookie\") pod \"rabbitmq-cell1-server-0\" (UID: \"568addd3-4ad3-47c3-adae-693691e7cd40\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:50:08 crc kubenswrapper[5052]: I1203 12:50:08.663767 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/568addd3-4ad3-47c3-adae-693691e7cd40-config-data\") pod \"rabbitmq-cell1-server-0\" (UID: \"568addd3-4ad3-47c3-adae-693691e7cd40\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:50:08 crc kubenswrapper[5052]: I1203 12:50:08.663835 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"plugins-conf\" (UniqueName: \"kubernetes.io/configmap/568addd3-4ad3-47c3-adae-693691e7cd40-plugins-conf\") pod \"rabbitmq-cell1-server-0\" (UID: \"568addd3-4ad3-47c3-adae-693691e7cd40\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:50:08 crc kubenswrapper[5052]: I1203 12:50:08.664292 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"server-conf\" (UniqueName: \"kubernetes.io/configmap/568addd3-4ad3-47c3-adae-693691e7cd40-server-conf\") pod \"rabbitmq-cell1-server-0\" (UID: \"568addd3-4ad3-47c3-adae-693691e7cd40\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:50:08 crc kubenswrapper[5052]: I1203 12:50:08.667217 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"rabbitmq-confd\" (UniqueName: \"kubernetes.io/projected/568addd3-4ad3-47c3-adae-693691e7cd40-rabbitmq-confd\") pod \"rabbitmq-cell1-server-0\" (UID: \"568addd3-4ad3-47c3-adae-693691e7cd40\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:50:08 crc kubenswrapper[5052]: I1203 12:50:08.667915 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"pod-info\" (UniqueName: \"kubernetes.io/downward-api/568addd3-4ad3-47c3-adae-693691e7cd40-pod-info\") pod \"rabbitmq-cell1-server-0\" (UID: \"568addd3-4ad3-47c3-adae-693691e7cd40\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:50:08 crc kubenswrapper[5052]: I1203 12:50:08.668252 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"erlang-cookie-secret\" (UniqueName: \"kubernetes.io/secret/568addd3-4ad3-47c3-adae-693691e7cd40-erlang-cookie-secret\") pod \"rabbitmq-cell1-server-0\" (UID: \"568addd3-4ad3-47c3-adae-693691e7cd40\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:50:08 crc kubenswrapper[5052]: I1203 12:50:08.668852 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"rabbitmq-tls\" (UniqueName: \"kubernetes.io/projected/568addd3-4ad3-47c3-adae-693691e7cd40-rabbitmq-tls\") pod \"rabbitmq-cell1-server-0\" (UID: \"568addd3-4ad3-47c3-adae-693691e7cd40\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:50:08 crc kubenswrapper[5052]: I1203 12:50:08.682658 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-9hxs4\" (UniqueName: \"kubernetes.io/projected/568addd3-4ad3-47c3-adae-693691e7cd40-kube-api-access-9hxs4\") pod \"rabbitmq-cell1-server-0\" (UID: \"568addd3-4ad3-47c3-adae-693691e7cd40\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:50:08 crc kubenswrapper[5052]: I1203 12:50:08.690722 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage02-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage02-crc\") pod \"rabbitmq-cell1-server-0\" (UID: \"568addd3-4ad3-47c3-adae-693691e7cd40\") " pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:50:08 crc kubenswrapper[5052]: I1203 12:50:08.709902 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:50:08 crc kubenswrapper[5052]: I1203 12:50:08.822613 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/rabbitmq-server-0"] Dec 03 12:50:08 crc kubenswrapper[5052]: I1203 12:50:08.826580 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:50:08 crc kubenswrapper[5052]: I1203 12:50:08.831729 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"rabbitmq-server-conf" Dec 03 12:50:08 crc kubenswrapper[5052]: I1203 12:50:08.832273 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"rabbitmq-server-dockercfg-7ztmk" Dec 03 12:50:08 crc kubenswrapper[5052]: I1203 12:50:08.832362 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"rabbitmq-erlang-cookie" Dec 03 12:50:08 crc kubenswrapper[5052]: I1203 12:50:08.832440 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-rabbitmq-svc" Dec 03 12:50:08 crc kubenswrapper[5052]: I1203 12:50:08.832527 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"rabbitmq-default-user" Dec 03 12:50:08 crc kubenswrapper[5052]: I1203 12:50:08.834575 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"rabbitmq-config-data" Dec 03 12:50:08 crc kubenswrapper[5052]: I1203 12:50:08.834848 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"rabbitmq-plugins-conf" Dec 03 12:50:08 crc kubenswrapper[5052]: I1203 12:50:08.845350 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/rabbitmq-server-0"] Dec 03 12:50:08 crc kubenswrapper[5052]: I1203 12:50:08.971160 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"rabbitmq-confd\" (UniqueName: \"kubernetes.io/projected/e73813f5-f97f-41dd-8a79-6196557d4d1a-rabbitmq-confd\") pod \"rabbitmq-server-0\" (UID: \"e73813f5-f97f-41dd-8a79-6196557d4d1a\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:50:08 crc kubenswrapper[5052]: I1203 12:50:08.971194 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"rabbitmq-plugins\" (UniqueName: \"kubernetes.io/empty-dir/e73813f5-f97f-41dd-8a79-6196557d4d1a-rabbitmq-plugins\") pod \"rabbitmq-server-0\" (UID: \"e73813f5-f97f-41dd-8a79-6196557d4d1a\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:50:08 crc kubenswrapper[5052]: I1203 12:50:08.971337 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"server-conf\" (UniqueName: \"kubernetes.io/configmap/e73813f5-f97f-41dd-8a79-6196557d4d1a-server-conf\") pod \"rabbitmq-server-0\" (UID: \"e73813f5-f97f-41dd-8a79-6196557d4d1a\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:50:08 crc kubenswrapper[5052]: I1203 12:50:08.971419 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"rabbitmq-tls\" (UniqueName: \"kubernetes.io/projected/e73813f5-f97f-41dd-8a79-6196557d4d1a-rabbitmq-tls\") pod \"rabbitmq-server-0\" (UID: \"e73813f5-f97f-41dd-8a79-6196557d4d1a\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:50:08 crc kubenswrapper[5052]: I1203 12:50:08.971475 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"erlang-cookie-secret\" (UniqueName: \"kubernetes.io/secret/e73813f5-f97f-41dd-8a79-6196557d4d1a-erlang-cookie-secret\") pod \"rabbitmq-server-0\" (UID: \"e73813f5-f97f-41dd-8a79-6196557d4d1a\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:50:08 crc kubenswrapper[5052]: I1203 12:50:08.971561 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"rabbitmq-erlang-cookie\" (UniqueName: \"kubernetes.io/empty-dir/e73813f5-f97f-41dd-8a79-6196557d4d1a-rabbitmq-erlang-cookie\") pod \"rabbitmq-server-0\" (UID: \"e73813f5-f97f-41dd-8a79-6196557d4d1a\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:50:08 crc kubenswrapper[5052]: I1203 12:50:08.971579 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-conf\" (UniqueName: \"kubernetes.io/configmap/e73813f5-f97f-41dd-8a79-6196557d4d1a-plugins-conf\") pod \"rabbitmq-server-0\" (UID: \"e73813f5-f97f-41dd-8a79-6196557d4d1a\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:50:08 crc kubenswrapper[5052]: I1203 12:50:08.971617 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5b8c9\" (UniqueName: \"kubernetes.io/projected/e73813f5-f97f-41dd-8a79-6196557d4d1a-kube-api-access-5b8c9\") pod \"rabbitmq-server-0\" (UID: \"e73813f5-f97f-41dd-8a79-6196557d4d1a\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:50:08 crc kubenswrapper[5052]: I1203 12:50:08.971668 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage03-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage03-crc\") pod \"rabbitmq-server-0\" (UID: \"e73813f5-f97f-41dd-8a79-6196557d4d1a\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:50:08 crc kubenswrapper[5052]: I1203 12:50:08.971731 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"pod-info\" (UniqueName: \"kubernetes.io/downward-api/e73813f5-f97f-41dd-8a79-6196557d4d1a-pod-info\") pod \"rabbitmq-server-0\" (UID: \"e73813f5-f97f-41dd-8a79-6196557d4d1a\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:50:08 crc kubenswrapper[5052]: I1203 12:50:08.971811 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/e73813f5-f97f-41dd-8a79-6196557d4d1a-config-data\") pod \"rabbitmq-server-0\" (UID: \"e73813f5-f97f-41dd-8a79-6196557d4d1a\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:50:09 crc kubenswrapper[5052]: I1203 12:50:09.073671 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"rabbitmq-confd\" (UniqueName: \"kubernetes.io/projected/e73813f5-f97f-41dd-8a79-6196557d4d1a-rabbitmq-confd\") pod \"rabbitmq-server-0\" (UID: \"e73813f5-f97f-41dd-8a79-6196557d4d1a\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:50:09 crc kubenswrapper[5052]: I1203 12:50:09.073714 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"rabbitmq-plugins\" (UniqueName: \"kubernetes.io/empty-dir/e73813f5-f97f-41dd-8a79-6196557d4d1a-rabbitmq-plugins\") pod \"rabbitmq-server-0\" (UID: \"e73813f5-f97f-41dd-8a79-6196557d4d1a\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:50:09 crc kubenswrapper[5052]: I1203 12:50:09.073732 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"server-conf\" (UniqueName: \"kubernetes.io/configmap/e73813f5-f97f-41dd-8a79-6196557d4d1a-server-conf\") pod \"rabbitmq-server-0\" (UID: \"e73813f5-f97f-41dd-8a79-6196557d4d1a\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:50:09 crc kubenswrapper[5052]: I1203 12:50:09.073755 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"rabbitmq-tls\" (UniqueName: \"kubernetes.io/projected/e73813f5-f97f-41dd-8a79-6196557d4d1a-rabbitmq-tls\") pod \"rabbitmq-server-0\" (UID: \"e73813f5-f97f-41dd-8a79-6196557d4d1a\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:50:09 crc kubenswrapper[5052]: I1203 12:50:09.073780 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"erlang-cookie-secret\" (UniqueName: \"kubernetes.io/secret/e73813f5-f97f-41dd-8a79-6196557d4d1a-erlang-cookie-secret\") pod \"rabbitmq-server-0\" (UID: \"e73813f5-f97f-41dd-8a79-6196557d4d1a\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:50:09 crc kubenswrapper[5052]: I1203 12:50:09.073804 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"rabbitmq-erlang-cookie\" (UniqueName: \"kubernetes.io/empty-dir/e73813f5-f97f-41dd-8a79-6196557d4d1a-rabbitmq-erlang-cookie\") pod \"rabbitmq-server-0\" (UID: \"e73813f5-f97f-41dd-8a79-6196557d4d1a\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:50:09 crc kubenswrapper[5052]: I1203 12:50:09.073820 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"plugins-conf\" (UniqueName: \"kubernetes.io/configmap/e73813f5-f97f-41dd-8a79-6196557d4d1a-plugins-conf\") pod \"rabbitmq-server-0\" (UID: \"e73813f5-f97f-41dd-8a79-6196557d4d1a\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:50:09 crc kubenswrapper[5052]: I1203 12:50:09.073842 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-5b8c9\" (UniqueName: \"kubernetes.io/projected/e73813f5-f97f-41dd-8a79-6196557d4d1a-kube-api-access-5b8c9\") pod \"rabbitmq-server-0\" (UID: \"e73813f5-f97f-41dd-8a79-6196557d4d1a\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:50:09 crc kubenswrapper[5052]: I1203 12:50:09.073869 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage03-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage03-crc\") pod \"rabbitmq-server-0\" (UID: \"e73813f5-f97f-41dd-8a79-6196557d4d1a\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:50:09 crc kubenswrapper[5052]: I1203 12:50:09.073901 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"pod-info\" (UniqueName: \"kubernetes.io/downward-api/e73813f5-f97f-41dd-8a79-6196557d4d1a-pod-info\") pod \"rabbitmq-server-0\" (UID: \"e73813f5-f97f-41dd-8a79-6196557d4d1a\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:50:09 crc kubenswrapper[5052]: I1203 12:50:09.073926 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/e73813f5-f97f-41dd-8a79-6196557d4d1a-config-data\") pod \"rabbitmq-server-0\" (UID: \"e73813f5-f97f-41dd-8a79-6196557d4d1a\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:50:09 crc kubenswrapper[5052]: I1203 12:50:09.074156 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage03-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage03-crc\") pod \"rabbitmq-server-0\" (UID: \"e73813f5-f97f-41dd-8a79-6196557d4d1a\") device mount path \"/mnt/openstack/pv03\"" pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:50:09 crc kubenswrapper[5052]: I1203 12:50:09.074443 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"rabbitmq-plugins\" (UniqueName: \"kubernetes.io/empty-dir/e73813f5-f97f-41dd-8a79-6196557d4d1a-rabbitmq-plugins\") pod \"rabbitmq-server-0\" (UID: \"e73813f5-f97f-41dd-8a79-6196557d4d1a\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:50:09 crc kubenswrapper[5052]: I1203 12:50:09.074682 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"rabbitmq-erlang-cookie\" (UniqueName: \"kubernetes.io/empty-dir/e73813f5-f97f-41dd-8a79-6196557d4d1a-rabbitmq-erlang-cookie\") pod \"rabbitmq-server-0\" (UID: \"e73813f5-f97f-41dd-8a79-6196557d4d1a\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:50:09 crc kubenswrapper[5052]: I1203 12:50:09.074918 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"plugins-conf\" (UniqueName: \"kubernetes.io/configmap/e73813f5-f97f-41dd-8a79-6196557d4d1a-plugins-conf\") pod \"rabbitmq-server-0\" (UID: \"e73813f5-f97f-41dd-8a79-6196557d4d1a\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:50:09 crc kubenswrapper[5052]: I1203 12:50:09.074941 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/e73813f5-f97f-41dd-8a79-6196557d4d1a-config-data\") pod \"rabbitmq-server-0\" (UID: \"e73813f5-f97f-41dd-8a79-6196557d4d1a\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:50:09 crc kubenswrapper[5052]: I1203 12:50:09.075361 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"server-conf\" (UniqueName: \"kubernetes.io/configmap/e73813f5-f97f-41dd-8a79-6196557d4d1a-server-conf\") pod \"rabbitmq-server-0\" (UID: \"e73813f5-f97f-41dd-8a79-6196557d4d1a\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:50:09 crc kubenswrapper[5052]: I1203 12:50:09.078588 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"pod-info\" (UniqueName: \"kubernetes.io/downward-api/e73813f5-f97f-41dd-8a79-6196557d4d1a-pod-info\") pod \"rabbitmq-server-0\" (UID: \"e73813f5-f97f-41dd-8a79-6196557d4d1a\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:50:09 crc kubenswrapper[5052]: I1203 12:50:09.080602 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"erlang-cookie-secret\" (UniqueName: \"kubernetes.io/secret/e73813f5-f97f-41dd-8a79-6196557d4d1a-erlang-cookie-secret\") pod \"rabbitmq-server-0\" (UID: \"e73813f5-f97f-41dd-8a79-6196557d4d1a\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:50:09 crc kubenswrapper[5052]: I1203 12:50:09.084019 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"rabbitmq-confd\" (UniqueName: \"kubernetes.io/projected/e73813f5-f97f-41dd-8a79-6196557d4d1a-rabbitmq-confd\") pod \"rabbitmq-server-0\" (UID: \"e73813f5-f97f-41dd-8a79-6196557d4d1a\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:50:09 crc kubenswrapper[5052]: I1203 12:50:09.084507 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"rabbitmq-tls\" (UniqueName: \"kubernetes.io/projected/e73813f5-f97f-41dd-8a79-6196557d4d1a-rabbitmq-tls\") pod \"rabbitmq-server-0\" (UID: \"e73813f5-f97f-41dd-8a79-6196557d4d1a\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:50:09 crc kubenswrapper[5052]: I1203 12:50:09.112884 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage03-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage03-crc\") pod \"rabbitmq-server-0\" (UID: \"e73813f5-f97f-41dd-8a79-6196557d4d1a\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:50:09 crc kubenswrapper[5052]: I1203 12:50:09.124863 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-5b8c9\" (UniqueName: \"kubernetes.io/projected/e73813f5-f97f-41dd-8a79-6196557d4d1a-kube-api-access-5b8c9\") pod \"rabbitmq-server-0\" (UID: \"e73813f5-f97f-41dd-8a79-6196557d4d1a\") " pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:50:09 crc kubenswrapper[5052]: I1203 12:50:09.147232 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/rabbitmq-cell1-server-0"] Dec 03 12:50:09 crc kubenswrapper[5052]: I1203 12:50:09.153994 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:50:09 crc kubenswrapper[5052]: I1203 12:50:09.629065 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/rabbitmq-server-0"] Dec 03 12:50:09 crc kubenswrapper[5052]: I1203 12:50:09.838962 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" event={"ID":"568addd3-4ad3-47c3-adae-693691e7cd40","Type":"ContainerStarted","Data":"b1fadf1ac3bfe95435852744a04e0c29a811c11270cfcc0a4f8a5ac0ba88ad99"} Dec 03 12:50:09 crc kubenswrapper[5052]: I1203 12:50:09.839712 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-server-0" event={"ID":"e73813f5-f97f-41dd-8a79-6196557d4d1a","Type":"ContainerStarted","Data":"6bc8c20d005acb0e60c7ac12f491adff39620c48a24066fc45557f898f3a9e66"} Dec 03 12:50:10 crc kubenswrapper[5052]: I1203 12:50:10.445208 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/openstack-galera-0"] Dec 03 12:50:10 crc kubenswrapper[5052]: I1203 12:50:10.446748 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:50:10 crc kubenswrapper[5052]: I1203 12:50:10.449133 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"openstack-config-data" Dec 03 12:50:10 crc kubenswrapper[5052]: I1203 12:50:10.449177 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-galera-openstack-svc" Dec 03 12:50:10 crc kubenswrapper[5052]: I1203 12:50:10.449980 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"openstack-scripts" Dec 03 12:50:10 crc kubenswrapper[5052]: I1203 12:50:10.460697 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"galera-openstack-dockercfg-fkczj" Dec 03 12:50:10 crc kubenswrapper[5052]: I1203 12:50:10.461340 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/openstack-galera-0"] Dec 03 12:50:10 crc kubenswrapper[5052]: I1203 12:50:10.469651 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"combined-ca-bundle" Dec 03 12:50:10 crc kubenswrapper[5052]: I1203 12:50:10.596124 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/b3363d6f-656e-4e22-a3eb-b1016380a0c8-kolla-config\") pod \"openstack-galera-0\" (UID: \"b3363d6f-656e-4e22-a3eb-b1016380a0c8\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:50:10 crc kubenswrapper[5052]: I1203 12:50:10.596176 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/b3363d6f-656e-4e22-a3eb-b1016380a0c8-config-data-generated\") pod \"openstack-galera-0\" (UID: \"b3363d6f-656e-4e22-a3eb-b1016380a0c8\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:50:10 crc kubenswrapper[5052]: I1203 12:50:10.596210 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/b3363d6f-656e-4e22-a3eb-b1016380a0c8-config-data-default\") pod \"openstack-galera-0\" (UID: \"b3363d6f-656e-4e22-a3eb-b1016380a0c8\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:50:10 crc kubenswrapper[5052]: I1203 12:50:10.596368 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/b3363d6f-656e-4e22-a3eb-b1016380a0c8-operator-scripts\") pod \"openstack-galera-0\" (UID: \"b3363d6f-656e-4e22-a3eb-b1016380a0c8\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:50:10 crc kubenswrapper[5052]: I1203 12:50:10.596430 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage09-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage09-crc\") pod \"openstack-galera-0\" (UID: \"b3363d6f-656e-4e22-a3eb-b1016380a0c8\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:50:10 crc kubenswrapper[5052]: I1203 12:50:10.596448 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b3363d6f-656e-4e22-a3eb-b1016380a0c8-combined-ca-bundle\") pod \"openstack-galera-0\" (UID: \"b3363d6f-656e-4e22-a3eb-b1016380a0c8\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:50:10 crc kubenswrapper[5052]: I1203 12:50:10.596553 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bl5h2\" (UniqueName: \"kubernetes.io/projected/b3363d6f-656e-4e22-a3eb-b1016380a0c8-kube-api-access-bl5h2\") pod \"openstack-galera-0\" (UID: \"b3363d6f-656e-4e22-a3eb-b1016380a0c8\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:50:10 crc kubenswrapper[5052]: I1203 12:50:10.596580 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/b3363d6f-656e-4e22-a3eb-b1016380a0c8-galera-tls-certs\") pod \"openstack-galera-0\" (UID: \"b3363d6f-656e-4e22-a3eb-b1016380a0c8\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:50:10 crc kubenswrapper[5052]: I1203 12:50:10.698427 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/b3363d6f-656e-4e22-a3eb-b1016380a0c8-kolla-config\") pod \"openstack-galera-0\" (UID: \"b3363d6f-656e-4e22-a3eb-b1016380a0c8\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:50:10 crc kubenswrapper[5052]: I1203 12:50:10.698477 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/b3363d6f-656e-4e22-a3eb-b1016380a0c8-config-data-generated\") pod \"openstack-galera-0\" (UID: \"b3363d6f-656e-4e22-a3eb-b1016380a0c8\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:50:10 crc kubenswrapper[5052]: I1203 12:50:10.698502 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/b3363d6f-656e-4e22-a3eb-b1016380a0c8-config-data-default\") pod \"openstack-galera-0\" (UID: \"b3363d6f-656e-4e22-a3eb-b1016380a0c8\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:50:10 crc kubenswrapper[5052]: I1203 12:50:10.698561 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/b3363d6f-656e-4e22-a3eb-b1016380a0c8-operator-scripts\") pod \"openstack-galera-0\" (UID: \"b3363d6f-656e-4e22-a3eb-b1016380a0c8\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:50:10 crc kubenswrapper[5052]: I1203 12:50:10.698588 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage09-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage09-crc\") pod \"openstack-galera-0\" (UID: \"b3363d6f-656e-4e22-a3eb-b1016380a0c8\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:50:10 crc kubenswrapper[5052]: I1203 12:50:10.698607 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b3363d6f-656e-4e22-a3eb-b1016380a0c8-combined-ca-bundle\") pod \"openstack-galera-0\" (UID: \"b3363d6f-656e-4e22-a3eb-b1016380a0c8\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:50:10 crc kubenswrapper[5052]: I1203 12:50:10.698667 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-bl5h2\" (UniqueName: \"kubernetes.io/projected/b3363d6f-656e-4e22-a3eb-b1016380a0c8-kube-api-access-bl5h2\") pod \"openstack-galera-0\" (UID: \"b3363d6f-656e-4e22-a3eb-b1016380a0c8\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:50:10 crc kubenswrapper[5052]: I1203 12:50:10.698689 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/b3363d6f-656e-4e22-a3eb-b1016380a0c8-galera-tls-certs\") pod \"openstack-galera-0\" (UID: \"b3363d6f-656e-4e22-a3eb-b1016380a0c8\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:50:10 crc kubenswrapper[5052]: I1203 12:50:10.698943 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage09-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage09-crc\") pod \"openstack-galera-0\" (UID: \"b3363d6f-656e-4e22-a3eb-b1016380a0c8\") device mount path \"/mnt/openstack/pv09\"" pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:50:10 crc kubenswrapper[5052]: I1203 12:50:10.699708 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/b3363d6f-656e-4e22-a3eb-b1016380a0c8-config-data-generated\") pod \"openstack-galera-0\" (UID: \"b3363d6f-656e-4e22-a3eb-b1016380a0c8\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:50:10 crc kubenswrapper[5052]: I1203 12:50:10.699967 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/b3363d6f-656e-4e22-a3eb-b1016380a0c8-kolla-config\") pod \"openstack-galera-0\" (UID: \"b3363d6f-656e-4e22-a3eb-b1016380a0c8\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:50:10 crc kubenswrapper[5052]: I1203 12:50:10.700113 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/b3363d6f-656e-4e22-a3eb-b1016380a0c8-config-data-default\") pod \"openstack-galera-0\" (UID: \"b3363d6f-656e-4e22-a3eb-b1016380a0c8\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:50:10 crc kubenswrapper[5052]: I1203 12:50:10.700777 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/b3363d6f-656e-4e22-a3eb-b1016380a0c8-operator-scripts\") pod \"openstack-galera-0\" (UID: \"b3363d6f-656e-4e22-a3eb-b1016380a0c8\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:50:10 crc kubenswrapper[5052]: I1203 12:50:10.706448 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/b3363d6f-656e-4e22-a3eb-b1016380a0c8-galera-tls-certs\") pod \"openstack-galera-0\" (UID: \"b3363d6f-656e-4e22-a3eb-b1016380a0c8\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:50:10 crc kubenswrapper[5052]: I1203 12:50:10.710235 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b3363d6f-656e-4e22-a3eb-b1016380a0c8-combined-ca-bundle\") pod \"openstack-galera-0\" (UID: \"b3363d6f-656e-4e22-a3eb-b1016380a0c8\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:50:10 crc kubenswrapper[5052]: I1203 12:50:10.718292 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-bl5h2\" (UniqueName: \"kubernetes.io/projected/b3363d6f-656e-4e22-a3eb-b1016380a0c8-kube-api-access-bl5h2\") pod \"openstack-galera-0\" (UID: \"b3363d6f-656e-4e22-a3eb-b1016380a0c8\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:50:10 crc kubenswrapper[5052]: I1203 12:50:10.727367 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage09-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage09-crc\") pod \"openstack-galera-0\" (UID: \"b3363d6f-656e-4e22-a3eb-b1016380a0c8\") " pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:50:10 crc kubenswrapper[5052]: I1203 12:50:10.769344 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:50:10 crc kubenswrapper[5052]: I1203 12:50:10.855637 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" event={"ID":"568addd3-4ad3-47c3-adae-693691e7cd40","Type":"ContainerStarted","Data":"d3e528340d7b7c51ad079f738747e4effb2e76401a14409088fb9ae1abff9903"} Dec 03 12:50:10 crc kubenswrapper[5052]: I1203 12:50:10.858802 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-server-0" event={"ID":"e73813f5-f97f-41dd-8a79-6196557d4d1a","Type":"ContainerStarted","Data":"0ef2bfbc937b79d5fdccf169b0b5adaf3d759d2b9428459842a9476d8aacd0c1"} Dec 03 12:50:11 crc kubenswrapper[5052]: W1203 12:50:11.269725 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb3363d6f_656e_4e22_a3eb_b1016380a0c8.slice/crio-5e15fc793d245bf9358ad40489776225b5cbe20697b0c58370c4fa680dc083c4 WatchSource:0}: Error finding container 5e15fc793d245bf9358ad40489776225b5cbe20697b0c58370c4fa680dc083c4: Status 404 returned error can't find the container with id 5e15fc793d245bf9358ad40489776225b5cbe20697b0c58370c4fa680dc083c4 Dec 03 12:50:11 crc kubenswrapper[5052]: I1203 12:50:11.270034 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/openstack-galera-0"] Dec 03 12:50:11 crc kubenswrapper[5052]: I1203 12:50:11.868704 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-galera-0" event={"ID":"b3363d6f-656e-4e22-a3eb-b1016380a0c8","Type":"ContainerStarted","Data":"5a735baf9cea071acdfeb65e88d849bce23a03817704ca1e54362fd9b759b7b2"} Dec 03 12:50:11 crc kubenswrapper[5052]: I1203 12:50:11.869292 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/openstack-cell1-galera-0"] Dec 03 12:50:11 crc kubenswrapper[5052]: I1203 12:50:11.870674 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-galera-0" event={"ID":"b3363d6f-656e-4e22-a3eb-b1016380a0c8","Type":"ContainerStarted","Data":"5e15fc793d245bf9358ad40489776225b5cbe20697b0c58370c4fa680dc083c4"} Dec 03 12:50:11 crc kubenswrapper[5052]: I1203 12:50:11.870772 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:50:11 crc kubenswrapper[5052]: I1203 12:50:11.876211 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"galera-openstack-cell1-dockercfg-vwgxg" Dec 03 12:50:11 crc kubenswrapper[5052]: I1203 12:50:11.876286 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-galera-openstack-cell1-svc" Dec 03 12:50:11 crc kubenswrapper[5052]: I1203 12:50:11.876459 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"openstack-cell1-config-data" Dec 03 12:50:11 crc kubenswrapper[5052]: I1203 12:50:11.876594 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"openstack-cell1-scripts" Dec 03 12:50:11 crc kubenswrapper[5052]: I1203 12:50:11.885710 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/openstack-cell1-galera-0"] Dec 03 12:50:12 crc kubenswrapper[5052]: I1203 12:50:12.022379 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/64ada63f-b17d-411b-a547-fa1c15107ab5-combined-ca-bundle\") pod \"openstack-cell1-galera-0\" (UID: \"64ada63f-b17d-411b-a547-fa1c15107ab5\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:50:12 crc kubenswrapper[5052]: I1203 12:50:12.022447 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9zh24\" (UniqueName: \"kubernetes.io/projected/64ada63f-b17d-411b-a547-fa1c15107ab5-kube-api-access-9zh24\") pod \"openstack-cell1-galera-0\" (UID: \"64ada63f-b17d-411b-a547-fa1c15107ab5\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:50:12 crc kubenswrapper[5052]: I1203 12:50:12.022587 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/64ada63f-b17d-411b-a547-fa1c15107ab5-operator-scripts\") pod \"openstack-cell1-galera-0\" (UID: \"64ada63f-b17d-411b-a547-fa1c15107ab5\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:50:12 crc kubenswrapper[5052]: I1203 12:50:12.022634 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage11-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage11-crc\") pod \"openstack-cell1-galera-0\" (UID: \"64ada63f-b17d-411b-a547-fa1c15107ab5\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:50:12 crc kubenswrapper[5052]: I1203 12:50:12.022689 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/64ada63f-b17d-411b-a547-fa1c15107ab5-galera-tls-certs\") pod \"openstack-cell1-galera-0\" (UID: \"64ada63f-b17d-411b-a547-fa1c15107ab5\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:50:12 crc kubenswrapper[5052]: I1203 12:50:12.022791 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/64ada63f-b17d-411b-a547-fa1c15107ab5-kolla-config\") pod \"openstack-cell1-galera-0\" (UID: \"64ada63f-b17d-411b-a547-fa1c15107ab5\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:50:12 crc kubenswrapper[5052]: I1203 12:50:12.022820 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/64ada63f-b17d-411b-a547-fa1c15107ab5-config-data-generated\") pod \"openstack-cell1-galera-0\" (UID: \"64ada63f-b17d-411b-a547-fa1c15107ab5\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:50:12 crc kubenswrapper[5052]: I1203 12:50:12.022884 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/64ada63f-b17d-411b-a547-fa1c15107ab5-config-data-default\") pod \"openstack-cell1-galera-0\" (UID: \"64ada63f-b17d-411b-a547-fa1c15107ab5\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:50:12 crc kubenswrapper[5052]: I1203 12:50:12.124811 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage11-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage11-crc\") pod \"openstack-cell1-galera-0\" (UID: \"64ada63f-b17d-411b-a547-fa1c15107ab5\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:50:12 crc kubenswrapper[5052]: I1203 12:50:12.124889 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/64ada63f-b17d-411b-a547-fa1c15107ab5-galera-tls-certs\") pod \"openstack-cell1-galera-0\" (UID: \"64ada63f-b17d-411b-a547-fa1c15107ab5\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:50:12 crc kubenswrapper[5052]: I1203 12:50:12.124942 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/64ada63f-b17d-411b-a547-fa1c15107ab5-kolla-config\") pod \"openstack-cell1-galera-0\" (UID: \"64ada63f-b17d-411b-a547-fa1c15107ab5\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:50:12 crc kubenswrapper[5052]: I1203 12:50:12.124971 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/64ada63f-b17d-411b-a547-fa1c15107ab5-config-data-generated\") pod \"openstack-cell1-galera-0\" (UID: \"64ada63f-b17d-411b-a547-fa1c15107ab5\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:50:12 crc kubenswrapper[5052]: I1203 12:50:12.125012 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/64ada63f-b17d-411b-a547-fa1c15107ab5-config-data-default\") pod \"openstack-cell1-galera-0\" (UID: \"64ada63f-b17d-411b-a547-fa1c15107ab5\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:50:12 crc kubenswrapper[5052]: I1203 12:50:12.125076 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/64ada63f-b17d-411b-a547-fa1c15107ab5-combined-ca-bundle\") pod \"openstack-cell1-galera-0\" (UID: \"64ada63f-b17d-411b-a547-fa1c15107ab5\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:50:12 crc kubenswrapper[5052]: I1203 12:50:12.125100 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-9zh24\" (UniqueName: \"kubernetes.io/projected/64ada63f-b17d-411b-a547-fa1c15107ab5-kube-api-access-9zh24\") pod \"openstack-cell1-galera-0\" (UID: \"64ada63f-b17d-411b-a547-fa1c15107ab5\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:50:12 crc kubenswrapper[5052]: I1203 12:50:12.125142 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/64ada63f-b17d-411b-a547-fa1c15107ab5-operator-scripts\") pod \"openstack-cell1-galera-0\" (UID: \"64ada63f-b17d-411b-a547-fa1c15107ab5\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:50:12 crc kubenswrapper[5052]: I1203 12:50:12.125226 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage11-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage11-crc\") pod \"openstack-cell1-galera-0\" (UID: \"64ada63f-b17d-411b-a547-fa1c15107ab5\") device mount path \"/mnt/openstack/pv11\"" pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:50:12 crc kubenswrapper[5052]: I1203 12:50:12.125425 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/64ada63f-b17d-411b-a547-fa1c15107ab5-config-data-generated\") pod \"openstack-cell1-galera-0\" (UID: \"64ada63f-b17d-411b-a547-fa1c15107ab5\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:50:12 crc kubenswrapper[5052]: I1203 12:50:12.126190 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/64ada63f-b17d-411b-a547-fa1c15107ab5-kolla-config\") pod \"openstack-cell1-galera-0\" (UID: \"64ada63f-b17d-411b-a547-fa1c15107ab5\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:50:12 crc kubenswrapper[5052]: I1203 12:50:12.126501 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/64ada63f-b17d-411b-a547-fa1c15107ab5-config-data-default\") pod \"openstack-cell1-galera-0\" (UID: \"64ada63f-b17d-411b-a547-fa1c15107ab5\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:50:12 crc kubenswrapper[5052]: I1203 12:50:12.127093 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/64ada63f-b17d-411b-a547-fa1c15107ab5-operator-scripts\") pod \"openstack-cell1-galera-0\" (UID: \"64ada63f-b17d-411b-a547-fa1c15107ab5\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:50:12 crc kubenswrapper[5052]: I1203 12:50:12.130437 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/64ada63f-b17d-411b-a547-fa1c15107ab5-galera-tls-certs\") pod \"openstack-cell1-galera-0\" (UID: \"64ada63f-b17d-411b-a547-fa1c15107ab5\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:50:12 crc kubenswrapper[5052]: I1203 12:50:12.131885 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/64ada63f-b17d-411b-a547-fa1c15107ab5-combined-ca-bundle\") pod \"openstack-cell1-galera-0\" (UID: \"64ada63f-b17d-411b-a547-fa1c15107ab5\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:50:12 crc kubenswrapper[5052]: I1203 12:50:12.146818 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage11-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage11-crc\") pod \"openstack-cell1-galera-0\" (UID: \"64ada63f-b17d-411b-a547-fa1c15107ab5\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:50:12 crc kubenswrapper[5052]: I1203 12:50:12.150591 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-9zh24\" (UniqueName: \"kubernetes.io/projected/64ada63f-b17d-411b-a547-fa1c15107ab5-kube-api-access-9zh24\") pod \"openstack-cell1-galera-0\" (UID: \"64ada63f-b17d-411b-a547-fa1c15107ab5\") " pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:50:12 crc kubenswrapper[5052]: I1203 12:50:12.204377 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:50:12 crc kubenswrapper[5052]: I1203 12:50:12.211179 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/memcached-0"] Dec 03 12:50:12 crc kubenswrapper[5052]: I1203 12:50:12.212490 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/memcached-0" Dec 03 12:50:12 crc kubenswrapper[5052]: I1203 12:50:12.214946 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-memcached-svc" Dec 03 12:50:12 crc kubenswrapper[5052]: I1203 12:50:12.215808 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"memcached-memcached-dockercfg-29bq9" Dec 03 12:50:12 crc kubenswrapper[5052]: I1203 12:50:12.216643 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"memcached-config-data" Dec 03 12:50:12 crc kubenswrapper[5052]: I1203 12:50:12.221115 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/memcached-0"] Dec 03 12:50:12 crc kubenswrapper[5052]: I1203 12:50:12.327819 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"memcached-tls-certs\" (UniqueName: \"kubernetes.io/secret/d1ff5100-3ac7-4633-811c-ea470242d858-memcached-tls-certs\") pod \"memcached-0\" (UID: \"d1ff5100-3ac7-4633-811c-ea470242d858\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:50:12 crc kubenswrapper[5052]: I1203 12:50:12.327938 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v69rd\" (UniqueName: \"kubernetes.io/projected/d1ff5100-3ac7-4633-811c-ea470242d858-kube-api-access-v69rd\") pod \"memcached-0\" (UID: \"d1ff5100-3ac7-4633-811c-ea470242d858\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:50:12 crc kubenswrapper[5052]: I1203 12:50:12.327977 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/d1ff5100-3ac7-4633-811c-ea470242d858-kolla-config\") pod \"memcached-0\" (UID: \"d1ff5100-3ac7-4633-811c-ea470242d858\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:50:12 crc kubenswrapper[5052]: I1203 12:50:12.328266 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d1ff5100-3ac7-4633-811c-ea470242d858-combined-ca-bundle\") pod \"memcached-0\" (UID: \"d1ff5100-3ac7-4633-811c-ea470242d858\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:50:12 crc kubenswrapper[5052]: I1203 12:50:12.328306 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/d1ff5100-3ac7-4633-811c-ea470242d858-config-data\") pod \"memcached-0\" (UID: \"d1ff5100-3ac7-4633-811c-ea470242d858\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:50:12 crc kubenswrapper[5052]: I1203 12:50:12.430286 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-v69rd\" (UniqueName: \"kubernetes.io/projected/d1ff5100-3ac7-4633-811c-ea470242d858-kube-api-access-v69rd\") pod \"memcached-0\" (UID: \"d1ff5100-3ac7-4633-811c-ea470242d858\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:50:12 crc kubenswrapper[5052]: I1203 12:50:12.430336 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/d1ff5100-3ac7-4633-811c-ea470242d858-kolla-config\") pod \"memcached-0\" (UID: \"d1ff5100-3ac7-4633-811c-ea470242d858\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:50:12 crc kubenswrapper[5052]: I1203 12:50:12.430364 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d1ff5100-3ac7-4633-811c-ea470242d858-combined-ca-bundle\") pod \"memcached-0\" (UID: \"d1ff5100-3ac7-4633-811c-ea470242d858\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:50:12 crc kubenswrapper[5052]: I1203 12:50:12.430395 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/d1ff5100-3ac7-4633-811c-ea470242d858-config-data\") pod \"memcached-0\" (UID: \"d1ff5100-3ac7-4633-811c-ea470242d858\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:50:12 crc kubenswrapper[5052]: I1203 12:50:12.430446 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"memcached-tls-certs\" (UniqueName: \"kubernetes.io/secret/d1ff5100-3ac7-4633-811c-ea470242d858-memcached-tls-certs\") pod \"memcached-0\" (UID: \"d1ff5100-3ac7-4633-811c-ea470242d858\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:50:12 crc kubenswrapper[5052]: I1203 12:50:12.431619 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/d1ff5100-3ac7-4633-811c-ea470242d858-config-data\") pod \"memcached-0\" (UID: \"d1ff5100-3ac7-4633-811c-ea470242d858\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:50:12 crc kubenswrapper[5052]: I1203 12:50:12.432252 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/d1ff5100-3ac7-4633-811c-ea470242d858-kolla-config\") pod \"memcached-0\" (UID: \"d1ff5100-3ac7-4633-811c-ea470242d858\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:50:12 crc kubenswrapper[5052]: I1203 12:50:12.436044 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"memcached-tls-certs\" (UniqueName: \"kubernetes.io/secret/d1ff5100-3ac7-4633-811c-ea470242d858-memcached-tls-certs\") pod \"memcached-0\" (UID: \"d1ff5100-3ac7-4633-811c-ea470242d858\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:50:12 crc kubenswrapper[5052]: I1203 12:50:12.437209 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d1ff5100-3ac7-4633-811c-ea470242d858-combined-ca-bundle\") pod \"memcached-0\" (UID: \"d1ff5100-3ac7-4633-811c-ea470242d858\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:50:12 crc kubenswrapper[5052]: I1203 12:50:12.465183 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-v69rd\" (UniqueName: \"kubernetes.io/projected/d1ff5100-3ac7-4633-811c-ea470242d858-kube-api-access-v69rd\") pod \"memcached-0\" (UID: \"d1ff5100-3ac7-4633-811c-ea470242d858\") " pod="openstack-kuttl-tests/memcached-0" Dec 03 12:50:12 crc kubenswrapper[5052]: I1203 12:50:12.555037 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/openstack-cell1-galera-0"] Dec 03 12:50:12 crc kubenswrapper[5052]: I1203 12:50:12.586333 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/memcached-0" Dec 03 12:50:12 crc kubenswrapper[5052]: I1203 12:50:12.886411 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-cell1-galera-0" event={"ID":"64ada63f-b17d-411b-a547-fa1c15107ab5","Type":"ContainerStarted","Data":"1748e5136286c097ba05ce84d0037c1e32b588c257e53061127e3d8c8fa44d52"} Dec 03 12:50:13 crc kubenswrapper[5052]: I1203 12:50:13.149906 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/memcached-0"] Dec 03 12:50:13 crc kubenswrapper[5052]: I1203 12:50:13.897724 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-cell1-galera-0" event={"ID":"64ada63f-b17d-411b-a547-fa1c15107ab5","Type":"ContainerStarted","Data":"0dc2d2dd56838e5a1533bde8204be1b738b226cd95e31ea3ee837e37c7cc98a8"} Dec 03 12:50:13 crc kubenswrapper[5052]: I1203 12:50:13.901204 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/memcached-0" event={"ID":"d1ff5100-3ac7-4633-811c-ea470242d858","Type":"ContainerStarted","Data":"27f09642bc3ec7488d9032aa069fc921aca6a6f0c2792bc5d84326beb5487829"} Dec 03 12:50:13 crc kubenswrapper[5052]: I1203 12:50:13.901259 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/memcached-0" event={"ID":"d1ff5100-3ac7-4633-811c-ea470242d858","Type":"ContainerStarted","Data":"7e94672ce1b201b8722d0a3d689750d098504a74ebe9de66246c9bc7e834b074"} Dec 03 12:50:13 crc kubenswrapper[5052]: I1203 12:50:13.901278 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/memcached-0" Dec 03 12:50:13 crc kubenswrapper[5052]: I1203 12:50:13.968080 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/memcached-0" podStartSLOduration=1.968055384 podStartE2EDuration="1.968055384s" podCreationTimestamp="2025-12-03 12:50:12 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:50:13.962472714 +0000 UTC m=+5404.550714488" watchObservedRunningTime="2025-12-03 12:50:13.968055384 +0000 UTC m=+5404.556297148" Dec 03 12:50:14 crc kubenswrapper[5052]: I1203 12:50:14.120410 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/kube-state-metrics-0"] Dec 03 12:50:14 crc kubenswrapper[5052]: I1203 12:50:14.122157 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:50:14 crc kubenswrapper[5052]: I1203 12:50:14.131608 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"telemetry-ceilometer-dockercfg-lg2gn" Dec 03 12:50:14 crc kubenswrapper[5052]: I1203 12:50:14.139405 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/kube-state-metrics-0"] Dec 03 12:50:14 crc kubenswrapper[5052]: I1203 12:50:14.254645 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kczb2\" (UniqueName: \"kubernetes.io/projected/571d31d9-6750-41fe-8905-ea6de9cfb44d-kube-api-access-kczb2\") pod \"kube-state-metrics-0\" (UID: \"571d31d9-6750-41fe-8905-ea6de9cfb44d\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:50:14 crc kubenswrapper[5052]: I1203 12:50:14.355917 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-kczb2\" (UniqueName: \"kubernetes.io/projected/571d31d9-6750-41fe-8905-ea6de9cfb44d-kube-api-access-kczb2\") pod \"kube-state-metrics-0\" (UID: \"571d31d9-6750-41fe-8905-ea6de9cfb44d\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:50:14 crc kubenswrapper[5052]: I1203 12:50:14.377039 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-kczb2\" (UniqueName: \"kubernetes.io/projected/571d31d9-6750-41fe-8905-ea6de9cfb44d-kube-api-access-kczb2\") pod \"kube-state-metrics-0\" (UID: \"571d31d9-6750-41fe-8905-ea6de9cfb44d\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:50:14 crc kubenswrapper[5052]: I1203 12:50:14.447577 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:50:14 crc kubenswrapper[5052]: I1203 12:50:14.925265 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/kube-state-metrics-0"] Dec 03 12:50:14 crc kubenswrapper[5052]: W1203 12:50:14.936808 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod571d31d9_6750_41fe_8905_ea6de9cfb44d.slice/crio-ed01d2e0e418b4c44af203e889c8edd46f3780911714a3a903eb50cb9667a55a WatchSource:0}: Error finding container ed01d2e0e418b4c44af203e889c8edd46f3780911714a3a903eb50cb9667a55a: Status 404 returned error can't find the container with id ed01d2e0e418b4c44af203e889c8edd46f3780911714a3a903eb50cb9667a55a Dec 03 12:50:15 crc kubenswrapper[5052]: I1203 12:50:15.917724 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/kube-state-metrics-0" event={"ID":"571d31d9-6750-41fe-8905-ea6de9cfb44d","Type":"ContainerStarted","Data":"dac5b0bc9b047df6eb67eb15e4e6c404248602afea4f49a377b09a6f2b2fe314"} Dec 03 12:50:15 crc kubenswrapper[5052]: I1203 12:50:15.918283 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:50:15 crc kubenswrapper[5052]: I1203 12:50:15.918352 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/kube-state-metrics-0" event={"ID":"571d31d9-6750-41fe-8905-ea6de9cfb44d","Type":"ContainerStarted","Data":"ed01d2e0e418b4c44af203e889c8edd46f3780911714a3a903eb50cb9667a55a"} Dec 03 12:50:15 crc kubenswrapper[5052]: I1203 12:50:15.921061 5052 generic.go:334] "Generic (PLEG): container finished" podID="b3363d6f-656e-4e22-a3eb-b1016380a0c8" containerID="5a735baf9cea071acdfeb65e88d849bce23a03817704ca1e54362fd9b759b7b2" exitCode=0 Dec 03 12:50:15 crc kubenswrapper[5052]: I1203 12:50:15.921106 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-galera-0" event={"ID":"b3363d6f-656e-4e22-a3eb-b1016380a0c8","Type":"ContainerDied","Data":"5a735baf9cea071acdfeb65e88d849bce23a03817704ca1e54362fd9b759b7b2"} Dec 03 12:50:15 crc kubenswrapper[5052]: I1203 12:50:15.942188 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/kube-state-metrics-0" podStartSLOduration=1.495747403 podStartE2EDuration="1.94216641s" podCreationTimestamp="2025-12-03 12:50:14 +0000 UTC" firstStartedPulling="2025-12-03 12:50:14.938753554 +0000 UTC m=+5405.526995318" lastFinishedPulling="2025-12-03 12:50:15.385172561 +0000 UTC m=+5405.973414325" observedRunningTime="2025-12-03 12:50:15.935076068 +0000 UTC m=+5406.523317832" watchObservedRunningTime="2025-12-03 12:50:15.94216641 +0000 UTC m=+5406.530408194" Dec 03 12:50:16 crc kubenswrapper[5052]: I1203 12:50:16.930945 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-galera-0" event={"ID":"b3363d6f-656e-4e22-a3eb-b1016380a0c8","Type":"ContainerStarted","Data":"b8ecae5dcbbd55bfb5a128bba207c90d7a6b85badc20f265e843e97165d22a9c"} Dec 03 12:50:16 crc kubenswrapper[5052]: I1203 12:50:16.934306 5052 generic.go:334] "Generic (PLEG): container finished" podID="64ada63f-b17d-411b-a547-fa1c15107ab5" containerID="0dc2d2dd56838e5a1533bde8204be1b738b226cd95e31ea3ee837e37c7cc98a8" exitCode=0 Dec 03 12:50:16 crc kubenswrapper[5052]: I1203 12:50:16.934648 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-cell1-galera-0" event={"ID":"64ada63f-b17d-411b-a547-fa1c15107ab5","Type":"ContainerDied","Data":"0dc2d2dd56838e5a1533bde8204be1b738b226cd95e31ea3ee837e37c7cc98a8"} Dec 03 12:50:17 crc kubenswrapper[5052]: I1203 12:50:17.010012 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/openstack-galera-0" podStartSLOduration=8.009988978 podStartE2EDuration="8.009988978s" podCreationTimestamp="2025-12-03 12:50:09 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:50:17.002487684 +0000 UTC m=+5407.590729448" watchObservedRunningTime="2025-12-03 12:50:17.009988978 +0000 UTC m=+5407.598230742" Dec 03 12:50:17 crc kubenswrapper[5052]: I1203 12:50:17.622743 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ovsdbserver-nb-0"] Dec 03 12:50:17 crc kubenswrapper[5052]: I1203 12:50:17.625000 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:50:17 crc kubenswrapper[5052]: I1203 12:50:17.626859 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"ovndbcluster-nb-scripts" Dec 03 12:50:17 crc kubenswrapper[5052]: I1203 12:50:17.627671 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ovncluster-ovndbcluster-nb-dockercfg-j6fqr" Dec 03 12:50:17 crc kubenswrapper[5052]: I1203 12:50:17.628178 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-ovndbcluster-nb-ovndbs" Dec 03 12:50:17 crc kubenswrapper[5052]: I1203 12:50:17.628230 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-ovn-metrics" Dec 03 12:50:17 crc kubenswrapper[5052]: I1203 12:50:17.629445 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"ovndbcluster-nb-config" Dec 03 12:50:17 crc kubenswrapper[5052]: I1203 12:50:17.640167 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-nb-0"] Dec 03 12:50:17 crc kubenswrapper[5052]: I1203 12:50:17.713668 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/3a5e94ea-faeb-4202-843a-bd691d8895ce-metrics-certs-tls-certs\") pod \"ovsdbserver-nb-0\" (UID: \"3a5e94ea-faeb-4202-843a-bd691d8895ce\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:50:17 crc kubenswrapper[5052]: I1203 12:50:17.713721 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage15-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage15-crc\") pod \"ovsdbserver-nb-0\" (UID: \"3a5e94ea-faeb-4202-843a-bd691d8895ce\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:50:17 crc kubenswrapper[5052]: I1203 12:50:17.713742 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/3a5e94ea-faeb-4202-843a-bd691d8895ce-ovsdb-rundir\") pod \"ovsdbserver-nb-0\" (UID: \"3a5e94ea-faeb-4202-843a-bd691d8895ce\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:50:17 crc kubenswrapper[5052]: I1203 12:50:17.713871 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3a5e94ea-faeb-4202-843a-bd691d8895ce-combined-ca-bundle\") pod \"ovsdbserver-nb-0\" (UID: \"3a5e94ea-faeb-4202-843a-bd691d8895ce\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:50:17 crc kubenswrapper[5052]: I1203 12:50:17.713950 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovsdbserver-nb-tls-certs\" (UniqueName: \"kubernetes.io/secret/3a5e94ea-faeb-4202-843a-bd691d8895ce-ovsdbserver-nb-tls-certs\") pod \"ovsdbserver-nb-0\" (UID: \"3a5e94ea-faeb-4202-843a-bd691d8895ce\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:50:17 crc kubenswrapper[5052]: I1203 12:50:17.714065 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/3a5e94ea-faeb-4202-843a-bd691d8895ce-scripts\") pod \"ovsdbserver-nb-0\" (UID: \"3a5e94ea-faeb-4202-843a-bd691d8895ce\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:50:17 crc kubenswrapper[5052]: I1203 12:50:17.714093 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8nd6t\" (UniqueName: \"kubernetes.io/projected/3a5e94ea-faeb-4202-843a-bd691d8895ce-kube-api-access-8nd6t\") pod \"ovsdbserver-nb-0\" (UID: \"3a5e94ea-faeb-4202-843a-bd691d8895ce\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:50:17 crc kubenswrapper[5052]: I1203 12:50:17.714110 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3a5e94ea-faeb-4202-843a-bd691d8895ce-config\") pod \"ovsdbserver-nb-0\" (UID: \"3a5e94ea-faeb-4202-843a-bd691d8895ce\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:50:17 crc kubenswrapper[5052]: I1203 12:50:17.815784 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/3a5e94ea-faeb-4202-843a-bd691d8895ce-metrics-certs-tls-certs\") pod \"ovsdbserver-nb-0\" (UID: \"3a5e94ea-faeb-4202-843a-bd691d8895ce\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:50:17 crc kubenswrapper[5052]: I1203 12:50:17.815835 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage15-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage15-crc\") pod \"ovsdbserver-nb-0\" (UID: \"3a5e94ea-faeb-4202-843a-bd691d8895ce\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:50:17 crc kubenswrapper[5052]: I1203 12:50:17.815856 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/3a5e94ea-faeb-4202-843a-bd691d8895ce-ovsdb-rundir\") pod \"ovsdbserver-nb-0\" (UID: \"3a5e94ea-faeb-4202-843a-bd691d8895ce\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:50:17 crc kubenswrapper[5052]: I1203 12:50:17.815876 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3a5e94ea-faeb-4202-843a-bd691d8895ce-combined-ca-bundle\") pod \"ovsdbserver-nb-0\" (UID: \"3a5e94ea-faeb-4202-843a-bd691d8895ce\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:50:17 crc kubenswrapper[5052]: I1203 12:50:17.815905 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovsdbserver-nb-tls-certs\" (UniqueName: \"kubernetes.io/secret/3a5e94ea-faeb-4202-843a-bd691d8895ce-ovsdbserver-nb-tls-certs\") pod \"ovsdbserver-nb-0\" (UID: \"3a5e94ea-faeb-4202-843a-bd691d8895ce\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:50:17 crc kubenswrapper[5052]: I1203 12:50:17.815979 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/3a5e94ea-faeb-4202-843a-bd691d8895ce-scripts\") pod \"ovsdbserver-nb-0\" (UID: \"3a5e94ea-faeb-4202-843a-bd691d8895ce\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:50:17 crc kubenswrapper[5052]: I1203 12:50:17.816008 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-8nd6t\" (UniqueName: \"kubernetes.io/projected/3a5e94ea-faeb-4202-843a-bd691d8895ce-kube-api-access-8nd6t\") pod \"ovsdbserver-nb-0\" (UID: \"3a5e94ea-faeb-4202-843a-bd691d8895ce\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:50:17 crc kubenswrapper[5052]: I1203 12:50:17.816026 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3a5e94ea-faeb-4202-843a-bd691d8895ce-config\") pod \"ovsdbserver-nb-0\" (UID: \"3a5e94ea-faeb-4202-843a-bd691d8895ce\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:50:17 crc kubenswrapper[5052]: I1203 12:50:17.816502 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/3a5e94ea-faeb-4202-843a-bd691d8895ce-ovsdb-rundir\") pod \"ovsdbserver-nb-0\" (UID: \"3a5e94ea-faeb-4202-843a-bd691d8895ce\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:50:17 crc kubenswrapper[5052]: I1203 12:50:17.816618 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage15-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage15-crc\") pod \"ovsdbserver-nb-0\" (UID: \"3a5e94ea-faeb-4202-843a-bd691d8895ce\") device mount path \"/mnt/openstack/pv15\"" pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:50:17 crc kubenswrapper[5052]: I1203 12:50:17.817209 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3a5e94ea-faeb-4202-843a-bd691d8895ce-config\") pod \"ovsdbserver-nb-0\" (UID: \"3a5e94ea-faeb-4202-843a-bd691d8895ce\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:50:17 crc kubenswrapper[5052]: I1203 12:50:17.818176 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/3a5e94ea-faeb-4202-843a-bd691d8895ce-scripts\") pod \"ovsdbserver-nb-0\" (UID: \"3a5e94ea-faeb-4202-843a-bd691d8895ce\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:50:17 crc kubenswrapper[5052]: I1203 12:50:17.820549 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3a5e94ea-faeb-4202-843a-bd691d8895ce-combined-ca-bundle\") pod \"ovsdbserver-nb-0\" (UID: \"3a5e94ea-faeb-4202-843a-bd691d8895ce\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:50:17 crc kubenswrapper[5052]: I1203 12:50:17.821134 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovsdbserver-nb-tls-certs\" (UniqueName: \"kubernetes.io/secret/3a5e94ea-faeb-4202-843a-bd691d8895ce-ovsdbserver-nb-tls-certs\") pod \"ovsdbserver-nb-0\" (UID: \"3a5e94ea-faeb-4202-843a-bd691d8895ce\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:50:17 crc kubenswrapper[5052]: I1203 12:50:17.822280 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/3a5e94ea-faeb-4202-843a-bd691d8895ce-metrics-certs-tls-certs\") pod \"ovsdbserver-nb-0\" (UID: \"3a5e94ea-faeb-4202-843a-bd691d8895ce\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:50:17 crc kubenswrapper[5052]: I1203 12:50:17.838312 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-8nd6t\" (UniqueName: \"kubernetes.io/projected/3a5e94ea-faeb-4202-843a-bd691d8895ce-kube-api-access-8nd6t\") pod \"ovsdbserver-nb-0\" (UID: \"3a5e94ea-faeb-4202-843a-bd691d8895ce\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:50:17 crc kubenswrapper[5052]: I1203 12:50:17.841828 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage15-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage15-crc\") pod \"ovsdbserver-nb-0\" (UID: \"3a5e94ea-faeb-4202-843a-bd691d8895ce\") " pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:50:17 crc kubenswrapper[5052]: I1203 12:50:17.943601 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-cell1-galera-0" event={"ID":"64ada63f-b17d-411b-a547-fa1c15107ab5","Type":"ContainerStarted","Data":"594b90765c644ac99bd8e077dd10ec16881d4e9f1364dcdc6541052085119849"} Dec 03 12:50:17 crc kubenswrapper[5052]: I1203 12:50:17.943670 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:50:17 crc kubenswrapper[5052]: I1203 12:50:17.969418 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/openstack-cell1-galera-0" podStartSLOduration=7.969399615 podStartE2EDuration="7.969399615s" podCreationTimestamp="2025-12-03 12:50:10 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:50:17.966376689 +0000 UTC m=+5408.554618463" watchObservedRunningTime="2025-12-03 12:50:17.969399615 +0000 UTC m=+5408.557641379" Dec 03 12:50:18 crc kubenswrapper[5052]: I1203 12:50:18.389794 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-nb-0"] Dec 03 12:50:18 crc kubenswrapper[5052]: I1203 12:50:18.953574 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-nb-0" event={"ID":"3a5e94ea-faeb-4202-843a-bd691d8895ce","Type":"ContainerStarted","Data":"5dbfbe34fded1c834b0fb2487d51d6da9334318d91fb67fd92b8f7457b35a22f"} Dec 03 12:50:18 crc kubenswrapper[5052]: I1203 12:50:18.953625 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-nb-0" event={"ID":"3a5e94ea-faeb-4202-843a-bd691d8895ce","Type":"ContainerStarted","Data":"4a6055cf9cff80734b8d6c28ebdc2f8003fb5d319d561fbaf2f95617ced8483e"} Dec 03 12:50:18 crc kubenswrapper[5052]: I1203 12:50:18.953642 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-nb-0" event={"ID":"3a5e94ea-faeb-4202-843a-bd691d8895ce","Type":"ContainerStarted","Data":"5ce9123099cbd6bdd7b0ddb2ed641312926d368d6d76b979c601a45165285cb5"} Dec 03 12:50:19 crc kubenswrapper[5052]: I1203 12:50:19.803965 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/ovsdbserver-nb-0" podStartSLOduration=3.80394414 podStartE2EDuration="3.80394414s" podCreationTimestamp="2025-12-03 12:50:16 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:50:18.977148055 +0000 UTC m=+5409.565389839" watchObservedRunningTime="2025-12-03 12:50:19.80394414 +0000 UTC m=+5410.392185904" Dec 03 12:50:19 crc kubenswrapper[5052]: I1203 12:50:19.807799 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ovsdbserver-sb-0"] Dec 03 12:50:19 crc kubenswrapper[5052]: I1203 12:50:19.809435 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:50:19 crc kubenswrapper[5052]: I1203 12:50:19.818180 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-ovndbcluster-sb-ovndbs" Dec 03 12:50:19 crc kubenswrapper[5052]: I1203 12:50:19.818242 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"ovndbcluster-sb-config" Dec 03 12:50:19 crc kubenswrapper[5052]: I1203 12:50:19.818192 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"ovndbcluster-sb-scripts" Dec 03 12:50:19 crc kubenswrapper[5052]: I1203 12:50:19.818651 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ovncluster-ovndbcluster-sb-dockercfg-p25sj" Dec 03 12:50:19 crc kubenswrapper[5052]: I1203 12:50:19.825848 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-sb-0"] Dec 03 12:50:19 crc kubenswrapper[5052]: I1203 12:50:19.950850 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d16285c0-7e62-4b3c-b3e8-a62b8abb49cb-combined-ca-bundle\") pod \"ovsdbserver-sb-0\" (UID: \"d16285c0-7e62-4b3c-b3e8-a62b8abb49cb\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:50:19 crc kubenswrapper[5052]: I1203 12:50:19.950928 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/d16285c0-7e62-4b3c-b3e8-a62b8abb49cb-scripts\") pod \"ovsdbserver-sb-0\" (UID: \"d16285c0-7e62-4b3c-b3e8-a62b8abb49cb\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:50:19 crc kubenswrapper[5052]: I1203 12:50:19.950960 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/d16285c0-7e62-4b3c-b3e8-a62b8abb49cb-ovsdb-rundir\") pod \"ovsdbserver-sb-0\" (UID: \"d16285c0-7e62-4b3c-b3e8-a62b8abb49cb\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:50:19 crc kubenswrapper[5052]: I1203 12:50:19.951124 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage04-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage04-crc\") pod \"ovsdbserver-sb-0\" (UID: \"d16285c0-7e62-4b3c-b3e8-a62b8abb49cb\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:50:19 crc kubenswrapper[5052]: I1203 12:50:19.951487 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/d16285c0-7e62-4b3c-b3e8-a62b8abb49cb-metrics-certs-tls-certs\") pod \"ovsdbserver-sb-0\" (UID: \"d16285c0-7e62-4b3c-b3e8-a62b8abb49cb\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:50:19 crc kubenswrapper[5052]: I1203 12:50:19.951725 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4fxgg\" (UniqueName: \"kubernetes.io/projected/d16285c0-7e62-4b3c-b3e8-a62b8abb49cb-kube-api-access-4fxgg\") pod \"ovsdbserver-sb-0\" (UID: \"d16285c0-7e62-4b3c-b3e8-a62b8abb49cb\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:50:19 crc kubenswrapper[5052]: I1203 12:50:19.951798 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d16285c0-7e62-4b3c-b3e8-a62b8abb49cb-config\") pod \"ovsdbserver-sb-0\" (UID: \"d16285c0-7e62-4b3c-b3e8-a62b8abb49cb\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:50:19 crc kubenswrapper[5052]: I1203 12:50:19.951875 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovsdbserver-sb-tls-certs\" (UniqueName: \"kubernetes.io/secret/d16285c0-7e62-4b3c-b3e8-a62b8abb49cb-ovsdbserver-sb-tls-certs\") pod \"ovsdbserver-sb-0\" (UID: \"d16285c0-7e62-4b3c-b3e8-a62b8abb49cb\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:50:20 crc kubenswrapper[5052]: I1203 12:50:20.053297 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovsdbserver-sb-tls-certs\" (UniqueName: \"kubernetes.io/secret/d16285c0-7e62-4b3c-b3e8-a62b8abb49cb-ovsdbserver-sb-tls-certs\") pod \"ovsdbserver-sb-0\" (UID: \"d16285c0-7e62-4b3c-b3e8-a62b8abb49cb\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:50:20 crc kubenswrapper[5052]: I1203 12:50:20.053386 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d16285c0-7e62-4b3c-b3e8-a62b8abb49cb-combined-ca-bundle\") pod \"ovsdbserver-sb-0\" (UID: \"d16285c0-7e62-4b3c-b3e8-a62b8abb49cb\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:50:20 crc kubenswrapper[5052]: I1203 12:50:20.053448 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/d16285c0-7e62-4b3c-b3e8-a62b8abb49cb-scripts\") pod \"ovsdbserver-sb-0\" (UID: \"d16285c0-7e62-4b3c-b3e8-a62b8abb49cb\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:50:20 crc kubenswrapper[5052]: I1203 12:50:20.053483 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/d16285c0-7e62-4b3c-b3e8-a62b8abb49cb-ovsdb-rundir\") pod \"ovsdbserver-sb-0\" (UID: \"d16285c0-7e62-4b3c-b3e8-a62b8abb49cb\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:50:20 crc kubenswrapper[5052]: I1203 12:50:20.053535 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage04-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage04-crc\") pod \"ovsdbserver-sb-0\" (UID: \"d16285c0-7e62-4b3c-b3e8-a62b8abb49cb\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:50:20 crc kubenswrapper[5052]: I1203 12:50:20.053590 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/d16285c0-7e62-4b3c-b3e8-a62b8abb49cb-metrics-certs-tls-certs\") pod \"ovsdbserver-sb-0\" (UID: \"d16285c0-7e62-4b3c-b3e8-a62b8abb49cb\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:50:20 crc kubenswrapper[5052]: I1203 12:50:20.053675 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-4fxgg\" (UniqueName: \"kubernetes.io/projected/d16285c0-7e62-4b3c-b3e8-a62b8abb49cb-kube-api-access-4fxgg\") pod \"ovsdbserver-sb-0\" (UID: \"d16285c0-7e62-4b3c-b3e8-a62b8abb49cb\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:50:20 crc kubenswrapper[5052]: I1203 12:50:20.053709 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d16285c0-7e62-4b3c-b3e8-a62b8abb49cb-config\") pod \"ovsdbserver-sb-0\" (UID: \"d16285c0-7e62-4b3c-b3e8-a62b8abb49cb\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:50:20 crc kubenswrapper[5052]: I1203 12:50:20.054386 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/d16285c0-7e62-4b3c-b3e8-a62b8abb49cb-ovsdb-rundir\") pod \"ovsdbserver-sb-0\" (UID: \"d16285c0-7e62-4b3c-b3e8-a62b8abb49cb\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:50:20 crc kubenswrapper[5052]: I1203 12:50:20.054744 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d16285c0-7e62-4b3c-b3e8-a62b8abb49cb-config\") pod \"ovsdbserver-sb-0\" (UID: \"d16285c0-7e62-4b3c-b3e8-a62b8abb49cb\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:50:20 crc kubenswrapper[5052]: I1203 12:50:20.055057 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage04-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage04-crc\") pod \"ovsdbserver-sb-0\" (UID: \"d16285c0-7e62-4b3c-b3e8-a62b8abb49cb\") device mount path \"/mnt/openstack/pv04\"" pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:50:20 crc kubenswrapper[5052]: I1203 12:50:20.056660 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/d16285c0-7e62-4b3c-b3e8-a62b8abb49cb-scripts\") pod \"ovsdbserver-sb-0\" (UID: \"d16285c0-7e62-4b3c-b3e8-a62b8abb49cb\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:50:20 crc kubenswrapper[5052]: I1203 12:50:20.058801 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovsdbserver-sb-tls-certs\" (UniqueName: \"kubernetes.io/secret/d16285c0-7e62-4b3c-b3e8-a62b8abb49cb-ovsdbserver-sb-tls-certs\") pod \"ovsdbserver-sb-0\" (UID: \"d16285c0-7e62-4b3c-b3e8-a62b8abb49cb\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:50:20 crc kubenswrapper[5052]: I1203 12:50:20.059027 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d16285c0-7e62-4b3c-b3e8-a62b8abb49cb-combined-ca-bundle\") pod \"ovsdbserver-sb-0\" (UID: \"d16285c0-7e62-4b3c-b3e8-a62b8abb49cb\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:50:20 crc kubenswrapper[5052]: I1203 12:50:20.061993 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/d16285c0-7e62-4b3c-b3e8-a62b8abb49cb-metrics-certs-tls-certs\") pod \"ovsdbserver-sb-0\" (UID: \"d16285c0-7e62-4b3c-b3e8-a62b8abb49cb\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:50:20 crc kubenswrapper[5052]: I1203 12:50:20.073856 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-4fxgg\" (UniqueName: \"kubernetes.io/projected/d16285c0-7e62-4b3c-b3e8-a62b8abb49cb-kube-api-access-4fxgg\") pod \"ovsdbserver-sb-0\" (UID: \"d16285c0-7e62-4b3c-b3e8-a62b8abb49cb\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:50:20 crc kubenswrapper[5052]: I1203 12:50:20.078434 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage04-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage04-crc\") pod \"ovsdbserver-sb-0\" (UID: \"d16285c0-7e62-4b3c-b3e8-a62b8abb49cb\") " pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:50:20 crc kubenswrapper[5052]: I1203 12:50:20.137984 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:50:20 crc kubenswrapper[5052]: E1203 12:50:20.347710 5052 upgradeaware.go:427] Error proxying data from client to backend: readfrom tcp 38.129.56.252:41568->38.129.56.252:34849: write tcp 38.129.56.252:41568->38.129.56.252:34849: write: broken pipe Dec 03 12:50:20 crc kubenswrapper[5052]: I1203 12:50:20.612355 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-sb-0"] Dec 03 12:50:20 crc kubenswrapper[5052]: I1203 12:50:20.770003 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:50:20 crc kubenswrapper[5052]: I1203 12:50:20.770064 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:50:20 crc kubenswrapper[5052]: I1203 12:50:20.861418 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:50:20 crc kubenswrapper[5052]: I1203 12:50:20.944252 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:50:20 crc kubenswrapper[5052]: I1203 12:50:20.984192 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-sb-0" event={"ID":"d16285c0-7e62-4b3c-b3e8-a62b8abb49cb","Type":"ContainerStarted","Data":"89cea0c9094ed2fb8bb3973f66f25388cbb772d78cc9dc394bfc10058b78d468"} Dec 03 12:50:20 crc kubenswrapper[5052]: I1203 12:50:20.984254 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-sb-0" event={"ID":"d16285c0-7e62-4b3c-b3e8-a62b8abb49cb","Type":"ContainerStarted","Data":"468387bd146ee57d9dbb25ad0e76ab64875756a52288e01c0d2ec19a6cddb217"} Dec 03 12:50:20 crc kubenswrapper[5052]: I1203 12:50:20.984271 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-sb-0" event={"ID":"d16285c0-7e62-4b3c-b3e8-a62b8abb49cb","Type":"ContainerStarted","Data":"32b977d602b5b173414e41de53d7cf4d2880cb47f71b460727db4cca78a1c064"} Dec 03 12:50:21 crc kubenswrapper[5052]: I1203 12:50:21.007834 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/ovsdbserver-sb-0" podStartSLOduration=3.007815589 podStartE2EDuration="3.007815589s" podCreationTimestamp="2025-12-03 12:50:18 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:50:20.999559503 +0000 UTC m=+5411.587801267" watchObservedRunningTime="2025-12-03 12:50:21.007815589 +0000 UTC m=+5411.596057343" Dec 03 12:50:21 crc kubenswrapper[5052]: I1203 12:50:21.066230 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:50:22 crc kubenswrapper[5052]: I1203 12:50:22.203490 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/keystone-c555-account-create-update-6zkgw"] Dec 03 12:50:22 crc kubenswrapper[5052]: I1203 12:50:22.205079 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:50:22 crc kubenswrapper[5052]: I1203 12:50:22.205110 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:50:22 crc kubenswrapper[5052]: I1203 12:50:22.205233 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-c555-account-create-update-6zkgw" Dec 03 12:50:22 crc kubenswrapper[5052]: I1203 12:50:22.214432 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone-db-secret" Dec 03 12:50:22 crc kubenswrapper[5052]: I1203 12:50:22.215207 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-c555-account-create-update-6zkgw"] Dec 03 12:50:22 crc kubenswrapper[5052]: I1203 12:50:22.303243 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/5f5af30b-fb44-40c4-94a2-33965abf72ef-operator-scripts\") pod \"keystone-c555-account-create-update-6zkgw\" (UID: \"5f5af30b-fb44-40c4-94a2-33965abf72ef\") " pod="openstack-kuttl-tests/keystone-c555-account-create-update-6zkgw" Dec 03 12:50:22 crc kubenswrapper[5052]: I1203 12:50:22.303400 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bsfl7\" (UniqueName: \"kubernetes.io/projected/5f5af30b-fb44-40c4-94a2-33965abf72ef-kube-api-access-bsfl7\") pod \"keystone-c555-account-create-update-6zkgw\" (UID: \"5f5af30b-fb44-40c4-94a2-33965abf72ef\") " pod="openstack-kuttl-tests/keystone-c555-account-create-update-6zkgw" Dec 03 12:50:22 crc kubenswrapper[5052]: I1203 12:50:22.405155 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/5f5af30b-fb44-40c4-94a2-33965abf72ef-operator-scripts\") pod \"keystone-c555-account-create-update-6zkgw\" (UID: \"5f5af30b-fb44-40c4-94a2-33965abf72ef\") " pod="openstack-kuttl-tests/keystone-c555-account-create-update-6zkgw" Dec 03 12:50:22 crc kubenswrapper[5052]: I1203 12:50:22.405267 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-bsfl7\" (UniqueName: \"kubernetes.io/projected/5f5af30b-fb44-40c4-94a2-33965abf72ef-kube-api-access-bsfl7\") pod \"keystone-c555-account-create-update-6zkgw\" (UID: \"5f5af30b-fb44-40c4-94a2-33965abf72ef\") " pod="openstack-kuttl-tests/keystone-c555-account-create-update-6zkgw" Dec 03 12:50:22 crc kubenswrapper[5052]: I1203 12:50:22.406084 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/5f5af30b-fb44-40c4-94a2-33965abf72ef-operator-scripts\") pod \"keystone-c555-account-create-update-6zkgw\" (UID: \"5f5af30b-fb44-40c4-94a2-33965abf72ef\") " pod="openstack-kuttl-tests/keystone-c555-account-create-update-6zkgw" Dec 03 12:50:22 crc kubenswrapper[5052]: I1203 12:50:22.431034 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-bsfl7\" (UniqueName: \"kubernetes.io/projected/5f5af30b-fb44-40c4-94a2-33965abf72ef-kube-api-access-bsfl7\") pod \"keystone-c555-account-create-update-6zkgw\" (UID: \"5f5af30b-fb44-40c4-94a2-33965abf72ef\") " pod="openstack-kuttl-tests/keystone-c555-account-create-update-6zkgw" Dec 03 12:50:22 crc kubenswrapper[5052]: I1203 12:50:22.480539 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/placement-db-create-7cr6f"] Dec 03 12:50:22 crc kubenswrapper[5052]: I1203 12:50:22.481829 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-db-create-7cr6f" Dec 03 12:50:22 crc kubenswrapper[5052]: I1203 12:50:22.489144 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/placement-db-create-7cr6f"] Dec 03 12:50:22 crc kubenswrapper[5052]: I1203 12:50:22.527242 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-c555-account-create-update-6zkgw" Dec 03 12:50:22 crc kubenswrapper[5052]: I1203 12:50:22.586203 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/placement-bfa4-account-create-update-84nm8"] Dec 03 12:50:22 crc kubenswrapper[5052]: I1203 12:50:22.587651 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-bfa4-account-create-update-84nm8" Dec 03 12:50:22 crc kubenswrapper[5052]: I1203 12:50:22.588689 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/memcached-0" Dec 03 12:50:22 crc kubenswrapper[5052]: I1203 12:50:22.591013 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"placement-db-secret" Dec 03 12:50:22 crc kubenswrapper[5052]: I1203 12:50:22.600079 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/placement-bfa4-account-create-update-84nm8"] Dec 03 12:50:22 crc kubenswrapper[5052]: I1203 12:50:22.608755 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/218f2098-bd98-44e2-a2d5-8f30733da613-operator-scripts\") pod \"placement-db-create-7cr6f\" (UID: \"218f2098-bd98-44e2-a2d5-8f30733da613\") " pod="openstack-kuttl-tests/placement-db-create-7cr6f" Dec 03 12:50:22 crc kubenswrapper[5052]: I1203 12:50:22.608824 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vg8sd\" (UniqueName: \"kubernetes.io/projected/218f2098-bd98-44e2-a2d5-8f30733da613-kube-api-access-vg8sd\") pod \"placement-db-create-7cr6f\" (UID: \"218f2098-bd98-44e2-a2d5-8f30733da613\") " pod="openstack-kuttl-tests/placement-db-create-7cr6f" Dec 03 12:50:22 crc kubenswrapper[5052]: I1203 12:50:22.709972 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-vg8sd\" (UniqueName: \"kubernetes.io/projected/218f2098-bd98-44e2-a2d5-8f30733da613-kube-api-access-vg8sd\") pod \"placement-db-create-7cr6f\" (UID: \"218f2098-bd98-44e2-a2d5-8f30733da613\") " pod="openstack-kuttl-tests/placement-db-create-7cr6f" Dec 03 12:50:22 crc kubenswrapper[5052]: I1203 12:50:22.710128 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/f3314e52-7982-44b7-89dd-73b819efe84e-operator-scripts\") pod \"placement-bfa4-account-create-update-84nm8\" (UID: \"f3314e52-7982-44b7-89dd-73b819efe84e\") " pod="openstack-kuttl-tests/placement-bfa4-account-create-update-84nm8" Dec 03 12:50:22 crc kubenswrapper[5052]: I1203 12:50:22.710160 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xlhp4\" (UniqueName: \"kubernetes.io/projected/f3314e52-7982-44b7-89dd-73b819efe84e-kube-api-access-xlhp4\") pod \"placement-bfa4-account-create-update-84nm8\" (UID: \"f3314e52-7982-44b7-89dd-73b819efe84e\") " pod="openstack-kuttl-tests/placement-bfa4-account-create-update-84nm8" Dec 03 12:50:22 crc kubenswrapper[5052]: I1203 12:50:22.710188 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/218f2098-bd98-44e2-a2d5-8f30733da613-operator-scripts\") pod \"placement-db-create-7cr6f\" (UID: \"218f2098-bd98-44e2-a2d5-8f30733da613\") " pod="openstack-kuttl-tests/placement-db-create-7cr6f" Dec 03 12:50:22 crc kubenswrapper[5052]: I1203 12:50:22.710836 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/218f2098-bd98-44e2-a2d5-8f30733da613-operator-scripts\") pod \"placement-db-create-7cr6f\" (UID: \"218f2098-bd98-44e2-a2d5-8f30733da613\") " pod="openstack-kuttl-tests/placement-db-create-7cr6f" Dec 03 12:50:22 crc kubenswrapper[5052]: I1203 12:50:22.727571 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-vg8sd\" (UniqueName: \"kubernetes.io/projected/218f2098-bd98-44e2-a2d5-8f30733da613-kube-api-access-vg8sd\") pod \"placement-db-create-7cr6f\" (UID: \"218f2098-bd98-44e2-a2d5-8f30733da613\") " pod="openstack-kuttl-tests/placement-db-create-7cr6f" Dec 03 12:50:22 crc kubenswrapper[5052]: I1203 12:50:22.772788 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/glance-db-create-42dbc"] Dec 03 12:50:22 crc kubenswrapper[5052]: I1203 12:50:22.774067 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-db-create-42dbc" Dec 03 12:50:22 crc kubenswrapper[5052]: I1203 12:50:22.811107 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-db-create-7cr6f" Dec 03 12:50:22 crc kubenswrapper[5052]: I1203 12:50:22.811800 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/f3314e52-7982-44b7-89dd-73b819efe84e-operator-scripts\") pod \"placement-bfa4-account-create-update-84nm8\" (UID: \"f3314e52-7982-44b7-89dd-73b819efe84e\") " pod="openstack-kuttl-tests/placement-bfa4-account-create-update-84nm8" Dec 03 12:50:22 crc kubenswrapper[5052]: I1203 12:50:22.811868 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-xlhp4\" (UniqueName: \"kubernetes.io/projected/f3314e52-7982-44b7-89dd-73b819efe84e-kube-api-access-xlhp4\") pod \"placement-bfa4-account-create-update-84nm8\" (UID: \"f3314e52-7982-44b7-89dd-73b819efe84e\") " pod="openstack-kuttl-tests/placement-bfa4-account-create-update-84nm8" Dec 03 12:50:22 crc kubenswrapper[5052]: I1203 12:50:22.812893 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/f3314e52-7982-44b7-89dd-73b819efe84e-operator-scripts\") pod \"placement-bfa4-account-create-update-84nm8\" (UID: \"f3314e52-7982-44b7-89dd-73b819efe84e\") " pod="openstack-kuttl-tests/placement-bfa4-account-create-update-84nm8" Dec 03 12:50:22 crc kubenswrapper[5052]: I1203 12:50:22.841870 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-db-create-42dbc"] Dec 03 12:50:22 crc kubenswrapper[5052]: I1203 12:50:22.855110 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-xlhp4\" (UniqueName: \"kubernetes.io/projected/f3314e52-7982-44b7-89dd-73b819efe84e-kube-api-access-xlhp4\") pod \"placement-bfa4-account-create-update-84nm8\" (UID: \"f3314e52-7982-44b7-89dd-73b819efe84e\") " pod="openstack-kuttl-tests/placement-bfa4-account-create-update-84nm8" Dec 03 12:50:22 crc kubenswrapper[5052]: I1203 12:50:22.892264 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/glance-7c80-account-create-update-kl5f4"] Dec 03 12:50:22 crc kubenswrapper[5052]: I1203 12:50:22.893377 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-7c80-account-create-update-kl5f4" Dec 03 12:50:22 crc kubenswrapper[5052]: I1203 12:50:22.895677 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-db-secret" Dec 03 12:50:22 crc kubenswrapper[5052]: I1203 12:50:22.909907 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-7c80-account-create-update-kl5f4"] Dec 03 12:50:22 crc kubenswrapper[5052]: I1203 12:50:22.937952 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-bfa4-account-create-update-84nm8" Dec 03 12:50:22 crc kubenswrapper[5052]: I1203 12:50:22.938168 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bpn5w\" (UniqueName: \"kubernetes.io/projected/f8491d70-84bb-4aa6-a969-fc03097c3745-kube-api-access-bpn5w\") pod \"glance-db-create-42dbc\" (UID: \"f8491d70-84bb-4aa6-a969-fc03097c3745\") " pod="openstack-kuttl-tests/glance-db-create-42dbc" Dec 03 12:50:22 crc kubenswrapper[5052]: I1203 12:50:22.938327 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/f8491d70-84bb-4aa6-a969-fc03097c3745-operator-scripts\") pod \"glance-db-create-42dbc\" (UID: \"f8491d70-84bb-4aa6-a969-fc03097c3745\") " pod="openstack-kuttl-tests/glance-db-create-42dbc" Dec 03 12:50:22 crc kubenswrapper[5052]: I1203 12:50:22.954264 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:50:23 crc kubenswrapper[5052]: I1203 12:50:23.040415 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bkqdd\" (UniqueName: \"kubernetes.io/projected/3db493c4-0372-454e-b802-53ffc8cc3e9b-kube-api-access-bkqdd\") pod \"glance-7c80-account-create-update-kl5f4\" (UID: \"3db493c4-0372-454e-b802-53ffc8cc3e9b\") " pod="openstack-kuttl-tests/glance-7c80-account-create-update-kl5f4" Dec 03 12:50:23 crc kubenswrapper[5052]: I1203 12:50:23.040472 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-bpn5w\" (UniqueName: \"kubernetes.io/projected/f8491d70-84bb-4aa6-a969-fc03097c3745-kube-api-access-bpn5w\") pod \"glance-db-create-42dbc\" (UID: \"f8491d70-84bb-4aa6-a969-fc03097c3745\") " pod="openstack-kuttl-tests/glance-db-create-42dbc" Dec 03 12:50:23 crc kubenswrapper[5052]: I1203 12:50:23.040556 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/3db493c4-0372-454e-b802-53ffc8cc3e9b-operator-scripts\") pod \"glance-7c80-account-create-update-kl5f4\" (UID: \"3db493c4-0372-454e-b802-53ffc8cc3e9b\") " pod="openstack-kuttl-tests/glance-7c80-account-create-update-kl5f4" Dec 03 12:50:23 crc kubenswrapper[5052]: I1203 12:50:23.040995 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/f8491d70-84bb-4aa6-a969-fc03097c3745-operator-scripts\") pod \"glance-db-create-42dbc\" (UID: \"f8491d70-84bb-4aa6-a969-fc03097c3745\") " pod="openstack-kuttl-tests/glance-db-create-42dbc" Dec 03 12:50:23 crc kubenswrapper[5052]: I1203 12:50:23.041783 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/f8491d70-84bb-4aa6-a969-fc03097c3745-operator-scripts\") pod \"glance-db-create-42dbc\" (UID: \"f8491d70-84bb-4aa6-a969-fc03097c3745\") " pod="openstack-kuttl-tests/glance-db-create-42dbc" Dec 03 12:50:23 crc kubenswrapper[5052]: I1203 12:50:23.061891 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-bpn5w\" (UniqueName: \"kubernetes.io/projected/f8491d70-84bb-4aa6-a969-fc03097c3745-kube-api-access-bpn5w\") pod \"glance-db-create-42dbc\" (UID: \"f8491d70-84bb-4aa6-a969-fc03097c3745\") " pod="openstack-kuttl-tests/glance-db-create-42dbc" Dec 03 12:50:23 crc kubenswrapper[5052]: I1203 12:50:23.092210 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-db-create-42dbc" Dec 03 12:50:23 crc kubenswrapper[5052]: I1203 12:50:23.138963 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:50:23 crc kubenswrapper[5052]: I1203 12:50:23.142009 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-bkqdd\" (UniqueName: \"kubernetes.io/projected/3db493c4-0372-454e-b802-53ffc8cc3e9b-kube-api-access-bkqdd\") pod \"glance-7c80-account-create-update-kl5f4\" (UID: \"3db493c4-0372-454e-b802-53ffc8cc3e9b\") " pod="openstack-kuttl-tests/glance-7c80-account-create-update-kl5f4" Dec 03 12:50:23 crc kubenswrapper[5052]: I1203 12:50:23.142057 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/3db493c4-0372-454e-b802-53ffc8cc3e9b-operator-scripts\") pod \"glance-7c80-account-create-update-kl5f4\" (UID: \"3db493c4-0372-454e-b802-53ffc8cc3e9b\") " pod="openstack-kuttl-tests/glance-7c80-account-create-update-kl5f4" Dec 03 12:50:23 crc kubenswrapper[5052]: I1203 12:50:23.143148 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/3db493c4-0372-454e-b802-53ffc8cc3e9b-operator-scripts\") pod \"glance-7c80-account-create-update-kl5f4\" (UID: \"3db493c4-0372-454e-b802-53ffc8cc3e9b\") " pod="openstack-kuttl-tests/glance-7c80-account-create-update-kl5f4" Dec 03 12:50:23 crc kubenswrapper[5052]: I1203 12:50:23.169043 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-bkqdd\" (UniqueName: \"kubernetes.io/projected/3db493c4-0372-454e-b802-53ffc8cc3e9b-kube-api-access-bkqdd\") pod \"glance-7c80-account-create-update-kl5f4\" (UID: \"3db493c4-0372-454e-b802-53ffc8cc3e9b\") " pod="openstack-kuttl-tests/glance-7c80-account-create-update-kl5f4" Dec 03 12:50:23 crc kubenswrapper[5052]: I1203 12:50:23.191422 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:50:23 crc kubenswrapper[5052]: I1203 12:50:23.274985 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-7c80-account-create-update-kl5f4" Dec 03 12:50:23 crc kubenswrapper[5052]: I1203 12:50:23.276332 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-c555-account-create-update-6zkgw"] Dec 03 12:50:23 crc kubenswrapper[5052]: W1203 12:50:23.281644 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5f5af30b_fb44_40c4_94a2_33965abf72ef.slice/crio-be1aab86577ab460ad127ec4548bd5c34b04e72bfae69bdd5168fbcd6c035d97 WatchSource:0}: Error finding container be1aab86577ab460ad127ec4548bd5c34b04e72bfae69bdd5168fbcd6c035d97: Status 404 returned error can't find the container with id be1aab86577ab460ad127ec4548bd5c34b04e72bfae69bdd5168fbcd6c035d97 Dec 03 12:50:23 crc kubenswrapper[5052]: I1203 12:50:23.374867 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/placement-db-create-7cr6f"] Dec 03 12:50:23 crc kubenswrapper[5052]: I1203 12:50:23.527394 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/placement-bfa4-account-create-update-84nm8"] Dec 03 12:50:23 crc kubenswrapper[5052]: I1203 12:50:23.619764 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-7c80-account-create-update-kl5f4"] Dec 03 12:50:23 crc kubenswrapper[5052]: W1203 12:50:23.624613 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3db493c4_0372_454e_b802_53ffc8cc3e9b.slice/crio-534c4a4402063e5d1b3086cd12d56bee9554c112c60319f1dc9db8368325da63 WatchSource:0}: Error finding container 534c4a4402063e5d1b3086cd12d56bee9554c112c60319f1dc9db8368325da63: Status 404 returned error can't find the container with id 534c4a4402063e5d1b3086cd12d56bee9554c112c60319f1dc9db8368325da63 Dec 03 12:50:23 crc kubenswrapper[5052]: I1203 12:50:23.648217 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-db-create-42dbc"] Dec 03 12:50:23 crc kubenswrapper[5052]: I1203 12:50:23.988874 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:50:24 crc kubenswrapper[5052]: I1203 12:50:24.050293 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:50:24 crc kubenswrapper[5052]: I1203 12:50:24.052186 5052 generic.go:334] "Generic (PLEG): container finished" podID="f8491d70-84bb-4aa6-a969-fc03097c3745" containerID="3337bf147a1198a7d374ebe43a83d3f902342362f5770d4c6b8d236f37a55306" exitCode=0 Dec 03 12:50:24 crc kubenswrapper[5052]: I1203 12:50:24.052293 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-db-create-42dbc" event={"ID":"f8491d70-84bb-4aa6-a969-fc03097c3745","Type":"ContainerDied","Data":"3337bf147a1198a7d374ebe43a83d3f902342362f5770d4c6b8d236f37a55306"} Dec 03 12:50:24 crc kubenswrapper[5052]: I1203 12:50:24.052365 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-db-create-42dbc" event={"ID":"f8491d70-84bb-4aa6-a969-fc03097c3745","Type":"ContainerStarted","Data":"90efba77b859214ef3ab77d5c74d90c6afb2c7ed8c01ce5dfcad40e3bc483c62"} Dec 03 12:50:24 crc kubenswrapper[5052]: I1203 12:50:24.057599 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-7c80-account-create-update-kl5f4" event={"ID":"3db493c4-0372-454e-b802-53ffc8cc3e9b","Type":"ContainerStarted","Data":"63038026ed4d3a0cffe0b438d1ef983112e5fb0e27b4ce5efd2e7a5e9ad2fe3c"} Dec 03 12:50:24 crc kubenswrapper[5052]: I1203 12:50:24.057681 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-7c80-account-create-update-kl5f4" event={"ID":"3db493c4-0372-454e-b802-53ffc8cc3e9b","Type":"ContainerStarted","Data":"534c4a4402063e5d1b3086cd12d56bee9554c112c60319f1dc9db8368325da63"} Dec 03 12:50:24 crc kubenswrapper[5052]: I1203 12:50:24.064684 5052 generic.go:334] "Generic (PLEG): container finished" podID="218f2098-bd98-44e2-a2d5-8f30733da613" containerID="b2aadea167aa77dc784f9f8aa60f73ab130df902022eed62b5b88d51e19cde7b" exitCode=0 Dec 03 12:50:24 crc kubenswrapper[5052]: I1203 12:50:24.064795 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-db-create-7cr6f" event={"ID":"218f2098-bd98-44e2-a2d5-8f30733da613","Type":"ContainerDied","Data":"b2aadea167aa77dc784f9f8aa60f73ab130df902022eed62b5b88d51e19cde7b"} Dec 03 12:50:24 crc kubenswrapper[5052]: I1203 12:50:24.064846 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-db-create-7cr6f" event={"ID":"218f2098-bd98-44e2-a2d5-8f30733da613","Type":"ContainerStarted","Data":"ada9143b9b3e3497dd50534ad43c47f4911522fc2ece1b8b1437b79a373f951a"} Dec 03 12:50:24 crc kubenswrapper[5052]: I1203 12:50:24.077705 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-bfa4-account-create-update-84nm8" event={"ID":"f3314e52-7982-44b7-89dd-73b819efe84e","Type":"ContainerStarted","Data":"4c91d3477ec8fee9de89adfe339684f3d73d2d5166f4c936f3b80973b8b0fcbb"} Dec 03 12:50:24 crc kubenswrapper[5052]: I1203 12:50:24.077753 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-bfa4-account-create-update-84nm8" event={"ID":"f3314e52-7982-44b7-89dd-73b819efe84e","Type":"ContainerStarted","Data":"faa3cf9d5555f9d7ddd7e0ffb128d19d162615347fba97490b3f39cc066654de"} Dec 03 12:50:24 crc kubenswrapper[5052]: I1203 12:50:24.080626 5052 generic.go:334] "Generic (PLEG): container finished" podID="5f5af30b-fb44-40c4-94a2-33965abf72ef" containerID="474a137afe0eb8c21a373aa9a2430c6cb8a18815dfd7c1dc6300a9e03b4cbac5" exitCode=0 Dec 03 12:50:24 crc kubenswrapper[5052]: I1203 12:50:24.082015 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-c555-account-create-update-6zkgw" event={"ID":"5f5af30b-fb44-40c4-94a2-33965abf72ef","Type":"ContainerDied","Data":"474a137afe0eb8c21a373aa9a2430c6cb8a18815dfd7c1dc6300a9e03b4cbac5"} Dec 03 12:50:24 crc kubenswrapper[5052]: I1203 12:50:24.082177 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:50:24 crc kubenswrapper[5052]: I1203 12:50:24.082197 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-c555-account-create-update-6zkgw" event={"ID":"5f5af30b-fb44-40c4-94a2-33965abf72ef","Type":"ContainerStarted","Data":"be1aab86577ab460ad127ec4548bd5c34b04e72bfae69bdd5168fbcd6c035d97"} Dec 03 12:50:24 crc kubenswrapper[5052]: I1203 12:50:24.405705 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:50:24 crc kubenswrapper[5052]: I1203 12:50:24.463694 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:50:24 crc kubenswrapper[5052]: I1203 12:50:24.588914 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:50:25 crc kubenswrapper[5052]: I1203 12:50:25.089252 5052 generic.go:334] "Generic (PLEG): container finished" podID="f3314e52-7982-44b7-89dd-73b819efe84e" containerID="4c91d3477ec8fee9de89adfe339684f3d73d2d5166f4c936f3b80973b8b0fcbb" exitCode=0 Dec 03 12:50:25 crc kubenswrapper[5052]: I1203 12:50:25.089395 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-bfa4-account-create-update-84nm8" event={"ID":"f3314e52-7982-44b7-89dd-73b819efe84e","Type":"ContainerDied","Data":"4c91d3477ec8fee9de89adfe339684f3d73d2d5166f4c936f3b80973b8b0fcbb"} Dec 03 12:50:25 crc kubenswrapper[5052]: I1203 12:50:25.091288 5052 generic.go:334] "Generic (PLEG): container finished" podID="3db493c4-0372-454e-b802-53ffc8cc3e9b" containerID="63038026ed4d3a0cffe0b438d1ef983112e5fb0e27b4ce5efd2e7a5e9ad2fe3c" exitCode=0 Dec 03 12:50:25 crc kubenswrapper[5052]: I1203 12:50:25.091554 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-7c80-account-create-update-kl5f4" event={"ID":"3db493c4-0372-454e-b802-53ffc8cc3e9b","Type":"ContainerDied","Data":"63038026ed4d3a0cffe0b438d1ef983112e5fb0e27b4ce5efd2e7a5e9ad2fe3c"} Dec 03 12:50:25 crc kubenswrapper[5052]: I1203 12:50:25.138475 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:50:25 crc kubenswrapper[5052]: I1203 12:50:25.406357 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ovn-northd-0"] Dec 03 12:50:25 crc kubenswrapper[5052]: I1203 12:50:25.409432 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:50:25 crc kubenswrapper[5052]: I1203 12:50:25.418716 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"ovnnorthd-config" Dec 03 12:50:25 crc kubenswrapper[5052]: I1203 12:50:25.419706 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ovnnorthd-ovnnorthd-dockercfg-7qznl" Dec 03 12:50:25 crc kubenswrapper[5052]: I1203 12:50:25.419877 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"ovnnorthd-scripts" Dec 03 12:50:25 crc kubenswrapper[5052]: I1203 12:50:25.420115 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-ovnnorthd-ovndbs" Dec 03 12:50:25 crc kubenswrapper[5052]: I1203 12:50:25.430392 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ovn-northd-0"] Dec 03 12:50:25 crc kubenswrapper[5052]: I1203 12:50:25.505408 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/3b93bc75-db87-4677-be2f-910d07bb696c-scripts\") pod \"ovn-northd-0\" (UID: \"3b93bc75-db87-4677-be2f-910d07bb696c\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:50:25 crc kubenswrapper[5052]: I1203 12:50:25.505496 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3b93bc75-db87-4677-be2f-910d07bb696c-combined-ca-bundle\") pod \"ovn-northd-0\" (UID: \"3b93bc75-db87-4677-be2f-910d07bb696c\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:50:25 crc kubenswrapper[5052]: I1203 12:50:25.505565 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-northd-tls-certs\" (UniqueName: \"kubernetes.io/secret/3b93bc75-db87-4677-be2f-910d07bb696c-ovn-northd-tls-certs\") pod \"ovn-northd-0\" (UID: \"3b93bc75-db87-4677-be2f-910d07bb696c\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:50:25 crc kubenswrapper[5052]: I1203 12:50:25.505598 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2wx2s\" (UniqueName: \"kubernetes.io/projected/3b93bc75-db87-4677-be2f-910d07bb696c-kube-api-access-2wx2s\") pod \"ovn-northd-0\" (UID: \"3b93bc75-db87-4677-be2f-910d07bb696c\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:50:25 crc kubenswrapper[5052]: I1203 12:50:25.505620 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-rundir\" (UniqueName: \"kubernetes.io/empty-dir/3b93bc75-db87-4677-be2f-910d07bb696c-ovn-rundir\") pod \"ovn-northd-0\" (UID: \"3b93bc75-db87-4677-be2f-910d07bb696c\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:50:25 crc kubenswrapper[5052]: I1203 12:50:25.505640 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/3b93bc75-db87-4677-be2f-910d07bb696c-metrics-certs-tls-certs\") pod \"ovn-northd-0\" (UID: \"3b93bc75-db87-4677-be2f-910d07bb696c\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:50:25 crc kubenswrapper[5052]: I1203 12:50:25.505661 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3b93bc75-db87-4677-be2f-910d07bb696c-config\") pod \"ovn-northd-0\" (UID: \"3b93bc75-db87-4677-be2f-910d07bb696c\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:50:25 crc kubenswrapper[5052]: I1203 12:50:25.511430 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-7c80-account-create-update-kl5f4" Dec 03 12:50:25 crc kubenswrapper[5052]: I1203 12:50:25.573573 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/swift-storage-0"] Dec 03 12:50:25 crc kubenswrapper[5052]: E1203 12:50:25.574000 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="3db493c4-0372-454e-b802-53ffc8cc3e9b" containerName="mariadb-account-create-update" Dec 03 12:50:25 crc kubenswrapper[5052]: I1203 12:50:25.574016 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="3db493c4-0372-454e-b802-53ffc8cc3e9b" containerName="mariadb-account-create-update" Dec 03 12:50:25 crc kubenswrapper[5052]: I1203 12:50:25.574369 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="3db493c4-0372-454e-b802-53ffc8cc3e9b" containerName="mariadb-account-create-update" Dec 03 12:50:25 crc kubenswrapper[5052]: I1203 12:50:25.579543 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:50:25 crc kubenswrapper[5052]: I1203 12:50:25.585160 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"swift-swift-dockercfg-cbnpw" Dec 03 12:50:25 crc kubenswrapper[5052]: I1203 12:50:25.589074 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/swift-storage-0"] Dec 03 12:50:25 crc kubenswrapper[5052]: I1203 12:50:25.591081 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"swift-storage-config-data" Dec 03 12:50:25 crc kubenswrapper[5052]: I1203 12:50:25.591216 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"swift-ring-files" Dec 03 12:50:25 crc kubenswrapper[5052]: I1203 12:50:25.591324 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"swift-conf" Dec 03 12:50:25 crc kubenswrapper[5052]: I1203 12:50:25.607003 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bkqdd\" (UniqueName: \"kubernetes.io/projected/3db493c4-0372-454e-b802-53ffc8cc3e9b-kube-api-access-bkqdd\") pod \"3db493c4-0372-454e-b802-53ffc8cc3e9b\" (UID: \"3db493c4-0372-454e-b802-53ffc8cc3e9b\") " Dec 03 12:50:25 crc kubenswrapper[5052]: I1203 12:50:25.607254 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/3db493c4-0372-454e-b802-53ffc8cc3e9b-operator-scripts\") pod \"3db493c4-0372-454e-b802-53ffc8cc3e9b\" (UID: \"3db493c4-0372-454e-b802-53ffc8cc3e9b\") " Dec 03 12:50:25 crc kubenswrapper[5052]: I1203 12:50:25.607547 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/3b93bc75-db87-4677-be2f-910d07bb696c-scripts\") pod \"ovn-northd-0\" (UID: \"3b93bc75-db87-4677-be2f-910d07bb696c\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:50:25 crc kubenswrapper[5052]: I1203 12:50:25.607616 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3b93bc75-db87-4677-be2f-910d07bb696c-combined-ca-bundle\") pod \"ovn-northd-0\" (UID: \"3b93bc75-db87-4677-be2f-910d07bb696c\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:50:25 crc kubenswrapper[5052]: I1203 12:50:25.607672 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovn-northd-tls-certs\" (UniqueName: \"kubernetes.io/secret/3b93bc75-db87-4677-be2f-910d07bb696c-ovn-northd-tls-certs\") pod \"ovn-northd-0\" (UID: \"3b93bc75-db87-4677-be2f-910d07bb696c\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:50:25 crc kubenswrapper[5052]: I1203 12:50:25.607700 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-2wx2s\" (UniqueName: \"kubernetes.io/projected/3b93bc75-db87-4677-be2f-910d07bb696c-kube-api-access-2wx2s\") pod \"ovn-northd-0\" (UID: \"3b93bc75-db87-4677-be2f-910d07bb696c\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:50:25 crc kubenswrapper[5052]: I1203 12:50:25.607722 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovn-rundir\" (UniqueName: \"kubernetes.io/empty-dir/3b93bc75-db87-4677-be2f-910d07bb696c-ovn-rundir\") pod \"ovn-northd-0\" (UID: \"3b93bc75-db87-4677-be2f-910d07bb696c\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:50:25 crc kubenswrapper[5052]: I1203 12:50:25.607746 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/3b93bc75-db87-4677-be2f-910d07bb696c-metrics-certs-tls-certs\") pod \"ovn-northd-0\" (UID: \"3b93bc75-db87-4677-be2f-910d07bb696c\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:50:25 crc kubenswrapper[5052]: I1203 12:50:25.607771 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3b93bc75-db87-4677-be2f-910d07bb696c-config\") pod \"ovn-northd-0\" (UID: \"3b93bc75-db87-4677-be2f-910d07bb696c\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:50:25 crc kubenswrapper[5052]: I1203 12:50:25.608840 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3b93bc75-db87-4677-be2f-910d07bb696c-config\") pod \"ovn-northd-0\" (UID: \"3b93bc75-db87-4677-be2f-910d07bb696c\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:50:25 crc kubenswrapper[5052]: I1203 12:50:25.609469 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovn-rundir\" (UniqueName: \"kubernetes.io/empty-dir/3b93bc75-db87-4677-be2f-910d07bb696c-ovn-rundir\") pod \"ovn-northd-0\" (UID: \"3b93bc75-db87-4677-be2f-910d07bb696c\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:50:25 crc kubenswrapper[5052]: I1203 12:50:25.609466 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/3b93bc75-db87-4677-be2f-910d07bb696c-scripts\") pod \"ovn-northd-0\" (UID: \"3b93bc75-db87-4677-be2f-910d07bb696c\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:50:25 crc kubenswrapper[5052]: I1203 12:50:25.609840 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3db493c4-0372-454e-b802-53ffc8cc3e9b-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "3db493c4-0372-454e-b802-53ffc8cc3e9b" (UID: "3db493c4-0372-454e-b802-53ffc8cc3e9b"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:50:25 crc kubenswrapper[5052]: I1203 12:50:25.623096 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/3b93bc75-db87-4677-be2f-910d07bb696c-metrics-certs-tls-certs\") pod \"ovn-northd-0\" (UID: \"3b93bc75-db87-4677-be2f-910d07bb696c\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:50:25 crc kubenswrapper[5052]: I1203 12:50:25.630028 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3db493c4-0372-454e-b802-53ffc8cc3e9b-kube-api-access-bkqdd" (OuterVolumeSpecName: "kube-api-access-bkqdd") pod "3db493c4-0372-454e-b802-53ffc8cc3e9b" (UID: "3db493c4-0372-454e-b802-53ffc8cc3e9b"). InnerVolumeSpecName "kube-api-access-bkqdd". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:50:25 crc kubenswrapper[5052]: I1203 12:50:25.635611 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3b93bc75-db87-4677-be2f-910d07bb696c-combined-ca-bundle\") pod \"ovn-northd-0\" (UID: \"3b93bc75-db87-4677-be2f-910d07bb696c\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:50:25 crc kubenswrapper[5052]: I1203 12:50:25.637139 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovn-northd-tls-certs\" (UniqueName: \"kubernetes.io/secret/3b93bc75-db87-4677-be2f-910d07bb696c-ovn-northd-tls-certs\") pod \"ovn-northd-0\" (UID: \"3b93bc75-db87-4677-be2f-910d07bb696c\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:50:25 crc kubenswrapper[5052]: I1203 12:50:25.653053 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-2wx2s\" (UniqueName: \"kubernetes.io/projected/3b93bc75-db87-4677-be2f-910d07bb696c-kube-api-access-2wx2s\") pod \"ovn-northd-0\" (UID: \"3b93bc75-db87-4677-be2f-910d07bb696c\") " pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:50:25 crc kubenswrapper[5052]: I1203 12:50:25.709224 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage10-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage10-crc\") pod \"swift-storage-0\" (UID: \"5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:50:25 crc kubenswrapper[5052]: I1203 12:50:25.709302 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gp5lf\" (UniqueName: \"kubernetes.io/projected/5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4-kube-api-access-gp5lf\") pod \"swift-storage-0\" (UID: \"5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:50:25 crc kubenswrapper[5052]: I1203 12:50:25.709322 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lock\" (UniqueName: \"kubernetes.io/empty-dir/5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4-lock\") pod \"swift-storage-0\" (UID: \"5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:50:25 crc kubenswrapper[5052]: I1203 12:50:25.709369 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4-etc-swift\") pod \"swift-storage-0\" (UID: \"5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:50:25 crc kubenswrapper[5052]: I1203 12:50:25.709398 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cache\" (UniqueName: \"kubernetes.io/empty-dir/5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4-cache\") pod \"swift-storage-0\" (UID: \"5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:50:25 crc kubenswrapper[5052]: I1203 12:50:25.709443 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/3db493c4-0372-454e-b802-53ffc8cc3e9b-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:50:25 crc kubenswrapper[5052]: I1203 12:50:25.709454 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-bkqdd\" (UniqueName: \"kubernetes.io/projected/3db493c4-0372-454e-b802-53ffc8cc3e9b-kube-api-access-bkqdd\") on node \"crc\" DevicePath \"\"" Dec 03 12:50:25 crc kubenswrapper[5052]: I1203 12:50:25.799982 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:50:25 crc kubenswrapper[5052]: I1203 12:50:25.811128 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-gp5lf\" (UniqueName: \"kubernetes.io/projected/5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4-kube-api-access-gp5lf\") pod \"swift-storage-0\" (UID: \"5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:50:25 crc kubenswrapper[5052]: I1203 12:50:25.811180 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"lock\" (UniqueName: \"kubernetes.io/empty-dir/5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4-lock\") pod \"swift-storage-0\" (UID: \"5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:50:25 crc kubenswrapper[5052]: I1203 12:50:25.811237 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4-etc-swift\") pod \"swift-storage-0\" (UID: \"5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:50:25 crc kubenswrapper[5052]: I1203 12:50:25.811268 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"cache\" (UniqueName: \"kubernetes.io/empty-dir/5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4-cache\") pod \"swift-storage-0\" (UID: \"5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:50:25 crc kubenswrapper[5052]: I1203 12:50:25.811328 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage10-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage10-crc\") pod \"swift-storage-0\" (UID: \"5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:50:25 crc kubenswrapper[5052]: I1203 12:50:25.811692 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage10-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage10-crc\") pod \"swift-storage-0\" (UID: \"5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4\") device mount path \"/mnt/openstack/pv10\"" pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:50:25 crc kubenswrapper[5052]: I1203 12:50:25.813014 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"lock\" (UniqueName: \"kubernetes.io/empty-dir/5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4-lock\") pod \"swift-storage-0\" (UID: \"5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:50:25 crc kubenswrapper[5052]: E1203 12:50:25.813454 5052 projected.go:288] Couldn't get configMap openstack-kuttl-tests/swift-ring-files: configmap "swift-ring-files" not found Dec 03 12:50:25 crc kubenswrapper[5052]: E1203 12:50:25.813470 5052 projected.go:194] Error preparing data for projected volume etc-swift for pod openstack-kuttl-tests/swift-storage-0: configmap "swift-ring-files" not found Dec 03 12:50:25 crc kubenswrapper[5052]: E1203 12:50:25.813654 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4-etc-swift podName:5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4 nodeName:}" failed. No retries permitted until 2025-12-03 12:50:26.313498142 +0000 UTC m=+5416.901739906 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "etc-swift" (UniqueName: "kubernetes.io/projected/5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4-etc-swift") pod "swift-storage-0" (UID: "5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4") : configmap "swift-ring-files" not found Dec 03 12:50:25 crc kubenswrapper[5052]: I1203 12:50:25.814269 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"cache\" (UniqueName: \"kubernetes.io/empty-dir/5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4-cache\") pod \"swift-storage-0\" (UID: \"5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:50:25 crc kubenswrapper[5052]: I1203 12:50:25.841494 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage10-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage10-crc\") pod \"swift-storage-0\" (UID: \"5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:50:25 crc kubenswrapper[5052]: I1203 12:50:25.849664 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-gp5lf\" (UniqueName: \"kubernetes.io/projected/5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4-kube-api-access-gp5lf\") pod \"swift-storage-0\" (UID: \"5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:50:25 crc kubenswrapper[5052]: I1203 12:50:25.915688 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-db-create-42dbc" Dec 03 12:50:25 crc kubenswrapper[5052]: I1203 12:50:25.945934 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-db-create-7cr6f" Dec 03 12:50:25 crc kubenswrapper[5052]: I1203 12:50:25.951875 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-bfa4-account-create-update-84nm8" Dec 03 12:50:25 crc kubenswrapper[5052]: I1203 12:50:25.980095 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-c555-account-create-update-6zkgw" Dec 03 12:50:26 crc kubenswrapper[5052]: I1203 12:50:26.014452 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xlhp4\" (UniqueName: \"kubernetes.io/projected/f3314e52-7982-44b7-89dd-73b819efe84e-kube-api-access-xlhp4\") pod \"f3314e52-7982-44b7-89dd-73b819efe84e\" (UID: \"f3314e52-7982-44b7-89dd-73b819efe84e\") " Dec 03 12:50:26 crc kubenswrapper[5052]: I1203 12:50:26.014488 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/f3314e52-7982-44b7-89dd-73b819efe84e-operator-scripts\") pod \"f3314e52-7982-44b7-89dd-73b819efe84e\" (UID: \"f3314e52-7982-44b7-89dd-73b819efe84e\") " Dec 03 12:50:26 crc kubenswrapper[5052]: I1203 12:50:26.014526 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/f8491d70-84bb-4aa6-a969-fc03097c3745-operator-scripts\") pod \"f8491d70-84bb-4aa6-a969-fc03097c3745\" (UID: \"f8491d70-84bb-4aa6-a969-fc03097c3745\") " Dec 03 12:50:26 crc kubenswrapper[5052]: I1203 12:50:26.014960 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vg8sd\" (UniqueName: \"kubernetes.io/projected/218f2098-bd98-44e2-a2d5-8f30733da613-kube-api-access-vg8sd\") pod \"218f2098-bd98-44e2-a2d5-8f30733da613\" (UID: \"218f2098-bd98-44e2-a2d5-8f30733da613\") " Dec 03 12:50:26 crc kubenswrapper[5052]: I1203 12:50:26.015007 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f3314e52-7982-44b7-89dd-73b819efe84e-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "f3314e52-7982-44b7-89dd-73b819efe84e" (UID: "f3314e52-7982-44b7-89dd-73b819efe84e"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:50:26 crc kubenswrapper[5052]: I1203 12:50:26.015024 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/218f2098-bd98-44e2-a2d5-8f30733da613-operator-scripts\") pod \"218f2098-bd98-44e2-a2d5-8f30733da613\" (UID: \"218f2098-bd98-44e2-a2d5-8f30733da613\") " Dec 03 12:50:26 crc kubenswrapper[5052]: I1203 12:50:26.015174 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bpn5w\" (UniqueName: \"kubernetes.io/projected/f8491d70-84bb-4aa6-a969-fc03097c3745-kube-api-access-bpn5w\") pod \"f8491d70-84bb-4aa6-a969-fc03097c3745\" (UID: \"f8491d70-84bb-4aa6-a969-fc03097c3745\") " Dec 03 12:50:26 crc kubenswrapper[5052]: I1203 12:50:26.015285 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f8491d70-84bb-4aa6-a969-fc03097c3745-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "f8491d70-84bb-4aa6-a969-fc03097c3745" (UID: "f8491d70-84bb-4aa6-a969-fc03097c3745"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:50:26 crc kubenswrapper[5052]: I1203 12:50:26.015870 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/f3314e52-7982-44b7-89dd-73b819efe84e-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:50:26 crc kubenswrapper[5052]: I1203 12:50:26.015891 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/f8491d70-84bb-4aa6-a969-fc03097c3745-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:50:26 crc kubenswrapper[5052]: I1203 12:50:26.016011 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/218f2098-bd98-44e2-a2d5-8f30733da613-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "218f2098-bd98-44e2-a2d5-8f30733da613" (UID: "218f2098-bd98-44e2-a2d5-8f30733da613"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:50:26 crc kubenswrapper[5052]: I1203 12:50:26.019196 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f3314e52-7982-44b7-89dd-73b819efe84e-kube-api-access-xlhp4" (OuterVolumeSpecName: "kube-api-access-xlhp4") pod "f3314e52-7982-44b7-89dd-73b819efe84e" (UID: "f3314e52-7982-44b7-89dd-73b819efe84e"). InnerVolumeSpecName "kube-api-access-xlhp4". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:50:26 crc kubenswrapper[5052]: I1203 12:50:26.019326 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/218f2098-bd98-44e2-a2d5-8f30733da613-kube-api-access-vg8sd" (OuterVolumeSpecName: "kube-api-access-vg8sd") pod "218f2098-bd98-44e2-a2d5-8f30733da613" (UID: "218f2098-bd98-44e2-a2d5-8f30733da613"). InnerVolumeSpecName "kube-api-access-vg8sd". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:50:26 crc kubenswrapper[5052]: I1203 12:50:26.019657 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f8491d70-84bb-4aa6-a969-fc03097c3745-kube-api-access-bpn5w" (OuterVolumeSpecName: "kube-api-access-bpn5w") pod "f8491d70-84bb-4aa6-a969-fc03097c3745" (UID: "f8491d70-84bb-4aa6-a969-fc03097c3745"). InnerVolumeSpecName "kube-api-access-bpn5w". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:50:26 crc kubenswrapper[5052]: I1203 12:50:26.100726 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-bfa4-account-create-update-84nm8" event={"ID":"f3314e52-7982-44b7-89dd-73b819efe84e","Type":"ContainerDied","Data":"faa3cf9d5555f9d7ddd7e0ffb128d19d162615347fba97490b3f39cc066654de"} Dec 03 12:50:26 crc kubenswrapper[5052]: I1203 12:50:26.100765 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="faa3cf9d5555f9d7ddd7e0ffb128d19d162615347fba97490b3f39cc066654de" Dec 03 12:50:26 crc kubenswrapper[5052]: I1203 12:50:26.100823 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-bfa4-account-create-update-84nm8" Dec 03 12:50:26 crc kubenswrapper[5052]: I1203 12:50:26.106562 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-c555-account-create-update-6zkgw" event={"ID":"5f5af30b-fb44-40c4-94a2-33965abf72ef","Type":"ContainerDied","Data":"be1aab86577ab460ad127ec4548bd5c34b04e72bfae69bdd5168fbcd6c035d97"} Dec 03 12:50:26 crc kubenswrapper[5052]: I1203 12:50:26.106610 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="be1aab86577ab460ad127ec4548bd5c34b04e72bfae69bdd5168fbcd6c035d97" Dec 03 12:50:26 crc kubenswrapper[5052]: I1203 12:50:26.106673 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-c555-account-create-update-6zkgw" Dec 03 12:50:26 crc kubenswrapper[5052]: I1203 12:50:26.107648 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-db-create-42dbc" event={"ID":"f8491d70-84bb-4aa6-a969-fc03097c3745","Type":"ContainerDied","Data":"90efba77b859214ef3ab77d5c74d90c6afb2c7ed8c01ce5dfcad40e3bc483c62"} Dec 03 12:50:26 crc kubenswrapper[5052]: I1203 12:50:26.107674 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="90efba77b859214ef3ab77d5c74d90c6afb2c7ed8c01ce5dfcad40e3bc483c62" Dec 03 12:50:26 crc kubenswrapper[5052]: I1203 12:50:26.107720 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-db-create-42dbc" Dec 03 12:50:26 crc kubenswrapper[5052]: I1203 12:50:26.111726 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-7c80-account-create-update-kl5f4" event={"ID":"3db493c4-0372-454e-b802-53ffc8cc3e9b","Type":"ContainerDied","Data":"534c4a4402063e5d1b3086cd12d56bee9554c112c60319f1dc9db8368325da63"} Dec 03 12:50:26 crc kubenswrapper[5052]: I1203 12:50:26.111760 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="534c4a4402063e5d1b3086cd12d56bee9554c112c60319f1dc9db8368325da63" Dec 03 12:50:26 crc kubenswrapper[5052]: I1203 12:50:26.111825 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-7c80-account-create-update-kl5f4" Dec 03 12:50:26 crc kubenswrapper[5052]: I1203 12:50:26.116390 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/5f5af30b-fb44-40c4-94a2-33965abf72ef-operator-scripts\") pod \"5f5af30b-fb44-40c4-94a2-33965abf72ef\" (UID: \"5f5af30b-fb44-40c4-94a2-33965abf72ef\") " Dec 03 12:50:26 crc kubenswrapper[5052]: I1203 12:50:26.116527 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bsfl7\" (UniqueName: \"kubernetes.io/projected/5f5af30b-fb44-40c4-94a2-33965abf72ef-kube-api-access-bsfl7\") pod \"5f5af30b-fb44-40c4-94a2-33965abf72ef\" (UID: \"5f5af30b-fb44-40c4-94a2-33965abf72ef\") " Dec 03 12:50:26 crc kubenswrapper[5052]: I1203 12:50:26.116914 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5f5af30b-fb44-40c4-94a2-33965abf72ef-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "5f5af30b-fb44-40c4-94a2-33965abf72ef" (UID: "5f5af30b-fb44-40c4-94a2-33965abf72ef"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:50:26 crc kubenswrapper[5052]: I1203 12:50:26.116994 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-vg8sd\" (UniqueName: \"kubernetes.io/projected/218f2098-bd98-44e2-a2d5-8f30733da613-kube-api-access-vg8sd\") on node \"crc\" DevicePath \"\"" Dec 03 12:50:26 crc kubenswrapper[5052]: I1203 12:50:26.117016 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/218f2098-bd98-44e2-a2d5-8f30733da613-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:50:26 crc kubenswrapper[5052]: I1203 12:50:26.117030 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-bpn5w\" (UniqueName: \"kubernetes.io/projected/f8491d70-84bb-4aa6-a969-fc03097c3745-kube-api-access-bpn5w\") on node \"crc\" DevicePath \"\"" Dec 03 12:50:26 crc kubenswrapper[5052]: I1203 12:50:26.117043 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-xlhp4\" (UniqueName: \"kubernetes.io/projected/f3314e52-7982-44b7-89dd-73b819efe84e-kube-api-access-xlhp4\") on node \"crc\" DevicePath \"\"" Dec 03 12:50:26 crc kubenswrapper[5052]: I1203 12:50:26.120061 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-db-create-7cr6f" event={"ID":"218f2098-bd98-44e2-a2d5-8f30733da613","Type":"ContainerDied","Data":"ada9143b9b3e3497dd50534ad43c47f4911522fc2ece1b8b1437b79a373f951a"} Dec 03 12:50:26 crc kubenswrapper[5052]: I1203 12:50:26.120100 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ada9143b9b3e3497dd50534ad43c47f4911522fc2ece1b8b1437b79a373f951a" Dec 03 12:50:26 crc kubenswrapper[5052]: I1203 12:50:26.120174 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-db-create-7cr6f" Dec 03 12:50:26 crc kubenswrapper[5052]: I1203 12:50:26.120866 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5f5af30b-fb44-40c4-94a2-33965abf72ef-kube-api-access-bsfl7" (OuterVolumeSpecName: "kube-api-access-bsfl7") pod "5f5af30b-fb44-40c4-94a2-33965abf72ef" (UID: "5f5af30b-fb44-40c4-94a2-33965abf72ef"). InnerVolumeSpecName "kube-api-access-bsfl7". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:50:27 crc kubenswrapper[5052]: I1203 12:50:26.219363 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/5f5af30b-fb44-40c4-94a2-33965abf72ef-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:50:27 crc kubenswrapper[5052]: I1203 12:50:26.219798 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-bsfl7\" (UniqueName: \"kubernetes.io/projected/5f5af30b-fb44-40c4-94a2-33965abf72ef-kube-api-access-bsfl7\") on node \"crc\" DevicePath \"\"" Dec 03 12:50:27 crc kubenswrapper[5052]: I1203 12:50:26.321958 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4-etc-swift\") pod \"swift-storage-0\" (UID: \"5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:50:27 crc kubenswrapper[5052]: E1203 12:50:26.322190 5052 projected.go:288] Couldn't get configMap openstack-kuttl-tests/swift-ring-files: configmap "swift-ring-files" not found Dec 03 12:50:27 crc kubenswrapper[5052]: E1203 12:50:26.322219 5052 projected.go:194] Error preparing data for projected volume etc-swift for pod openstack-kuttl-tests/swift-storage-0: configmap "swift-ring-files" not found Dec 03 12:50:27 crc kubenswrapper[5052]: E1203 12:50:26.322283 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4-etc-swift podName:5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4 nodeName:}" failed. No retries permitted until 2025-12-03 12:50:27.322262812 +0000 UTC m=+5417.910504576 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "etc-swift" (UniqueName: "kubernetes.io/projected/5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4-etc-swift") pod "swift-storage-0" (UID: "5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4") : configmap "swift-ring-files" not found Dec 03 12:50:27 crc kubenswrapper[5052]: I1203 12:50:26.422430 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ovn-northd-0"] Dec 03 12:50:27 crc kubenswrapper[5052]: I1203 12:50:27.130152 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovn-northd-0" event={"ID":"3b93bc75-db87-4677-be2f-910d07bb696c","Type":"ContainerStarted","Data":"06bf7c042266ac81635b44d190310f81518aa47f7b8008ddde3f54ca515b15fd"} Dec 03 12:50:27 crc kubenswrapper[5052]: I1203 12:50:27.130452 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovn-northd-0" event={"ID":"3b93bc75-db87-4677-be2f-910d07bb696c","Type":"ContainerStarted","Data":"b0ffc7b4f3e19ca93256e9a6d7d13686986ce1b654547d9489545b25ee4fe92a"} Dec 03 12:50:27 crc kubenswrapper[5052]: I1203 12:50:27.342016 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4-etc-swift\") pod \"swift-storage-0\" (UID: \"5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:50:27 crc kubenswrapper[5052]: E1203 12:50:27.342198 5052 projected.go:288] Couldn't get configMap openstack-kuttl-tests/swift-ring-files: configmap "swift-ring-files" not found Dec 03 12:50:27 crc kubenswrapper[5052]: E1203 12:50:27.342211 5052 projected.go:194] Error preparing data for projected volume etc-swift for pod openstack-kuttl-tests/swift-storage-0: configmap "swift-ring-files" not found Dec 03 12:50:27 crc kubenswrapper[5052]: E1203 12:50:27.342253 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4-etc-swift podName:5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4 nodeName:}" failed. No retries permitted until 2025-12-03 12:50:29.342239352 +0000 UTC m=+5419.930481116 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "etc-swift" (UniqueName: "kubernetes.io/projected/5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4-etc-swift") pod "swift-storage-0" (UID: "5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4") : configmap "swift-ring-files" not found Dec 03 12:50:28 crc kubenswrapper[5052]: I1203 12:50:28.140756 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/glance-db-sync-p8q2q"] Dec 03 12:50:28 crc kubenswrapper[5052]: E1203 12:50:28.141384 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="f3314e52-7982-44b7-89dd-73b819efe84e" containerName="mariadb-account-create-update" Dec 03 12:50:28 crc kubenswrapper[5052]: I1203 12:50:28.141400 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="f3314e52-7982-44b7-89dd-73b819efe84e" containerName="mariadb-account-create-update" Dec 03 12:50:28 crc kubenswrapper[5052]: E1203 12:50:28.141423 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="5f5af30b-fb44-40c4-94a2-33965abf72ef" containerName="mariadb-account-create-update" Dec 03 12:50:28 crc kubenswrapper[5052]: I1203 12:50:28.141430 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f5af30b-fb44-40c4-94a2-33965abf72ef" containerName="mariadb-account-create-update" Dec 03 12:50:28 crc kubenswrapper[5052]: E1203 12:50:28.141444 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="218f2098-bd98-44e2-a2d5-8f30733da613" containerName="mariadb-database-create" Dec 03 12:50:28 crc kubenswrapper[5052]: I1203 12:50:28.141450 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="218f2098-bd98-44e2-a2d5-8f30733da613" containerName="mariadb-database-create" Dec 03 12:50:28 crc kubenswrapper[5052]: E1203 12:50:28.141459 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="f8491d70-84bb-4aa6-a969-fc03097c3745" containerName="mariadb-database-create" Dec 03 12:50:28 crc kubenswrapper[5052]: I1203 12:50:28.141465 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="f8491d70-84bb-4aa6-a969-fc03097c3745" containerName="mariadb-database-create" Dec 03 12:50:28 crc kubenswrapper[5052]: I1203 12:50:28.142367 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="f8491d70-84bb-4aa6-a969-fc03097c3745" containerName="mariadb-database-create" Dec 03 12:50:28 crc kubenswrapper[5052]: I1203 12:50:28.142395 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="f3314e52-7982-44b7-89dd-73b819efe84e" containerName="mariadb-account-create-update" Dec 03 12:50:28 crc kubenswrapper[5052]: I1203 12:50:28.142411 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="5f5af30b-fb44-40c4-94a2-33965abf72ef" containerName="mariadb-account-create-update" Dec 03 12:50:28 crc kubenswrapper[5052]: I1203 12:50:28.142422 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="218f2098-bd98-44e2-a2d5-8f30733da613" containerName="mariadb-database-create" Dec 03 12:50:28 crc kubenswrapper[5052]: I1203 12:50:28.142954 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:50:28 crc kubenswrapper[5052]: I1203 12:50:28.142973 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovn-northd-0" event={"ID":"3b93bc75-db87-4677-be2f-910d07bb696c","Type":"ContainerStarted","Data":"7cdee3173161e7591d4f168eb2ed4975b7806623d040f607514a1bb1ab4a5542"} Dec 03 12:50:28 crc kubenswrapper[5052]: I1203 12:50:28.143046 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-db-sync-p8q2q" Dec 03 12:50:28 crc kubenswrapper[5052]: I1203 12:50:28.145197 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-config-data" Dec 03 12:50:28 crc kubenswrapper[5052]: I1203 12:50:28.145222 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-glance-dockercfg-zph2l" Dec 03 12:50:28 crc kubenswrapper[5052]: I1203 12:50:28.153317 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-db-sync-p8q2q"] Dec 03 12:50:28 crc kubenswrapper[5052]: I1203 12:50:28.169058 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/ovn-northd-0" podStartSLOduration=3.169030067 podStartE2EDuration="3.169030067s" podCreationTimestamp="2025-12-03 12:50:25 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:50:28.165131825 +0000 UTC m=+5418.753373589" watchObservedRunningTime="2025-12-03 12:50:28.169030067 +0000 UTC m=+5418.757271831" Dec 03 12:50:28 crc kubenswrapper[5052]: I1203 12:50:28.254219 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5282791b-cdf2-4e89-a273-89bc6550f531-combined-ca-bundle\") pod \"glance-db-sync-p8q2q\" (UID: \"5282791b-cdf2-4e89-a273-89bc6550f531\") " pod="openstack-kuttl-tests/glance-db-sync-p8q2q" Dec 03 12:50:28 crc kubenswrapper[5052]: I1203 12:50:28.254296 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kmz6b\" (UniqueName: \"kubernetes.io/projected/5282791b-cdf2-4e89-a273-89bc6550f531-kube-api-access-kmz6b\") pod \"glance-db-sync-p8q2q\" (UID: \"5282791b-cdf2-4e89-a273-89bc6550f531\") " pod="openstack-kuttl-tests/glance-db-sync-p8q2q" Dec 03 12:50:28 crc kubenswrapper[5052]: I1203 12:50:28.254314 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/5282791b-cdf2-4e89-a273-89bc6550f531-db-sync-config-data\") pod \"glance-db-sync-p8q2q\" (UID: \"5282791b-cdf2-4e89-a273-89bc6550f531\") " pod="openstack-kuttl-tests/glance-db-sync-p8q2q" Dec 03 12:50:28 crc kubenswrapper[5052]: I1203 12:50:28.254716 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/5282791b-cdf2-4e89-a273-89bc6550f531-config-data\") pod \"glance-db-sync-p8q2q\" (UID: \"5282791b-cdf2-4e89-a273-89bc6550f531\") " pod="openstack-kuttl-tests/glance-db-sync-p8q2q" Dec 03 12:50:28 crc kubenswrapper[5052]: I1203 12:50:28.356463 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5282791b-cdf2-4e89-a273-89bc6550f531-combined-ca-bundle\") pod \"glance-db-sync-p8q2q\" (UID: \"5282791b-cdf2-4e89-a273-89bc6550f531\") " pod="openstack-kuttl-tests/glance-db-sync-p8q2q" Dec 03 12:50:28 crc kubenswrapper[5052]: I1203 12:50:28.356563 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-kmz6b\" (UniqueName: \"kubernetes.io/projected/5282791b-cdf2-4e89-a273-89bc6550f531-kube-api-access-kmz6b\") pod \"glance-db-sync-p8q2q\" (UID: \"5282791b-cdf2-4e89-a273-89bc6550f531\") " pod="openstack-kuttl-tests/glance-db-sync-p8q2q" Dec 03 12:50:28 crc kubenswrapper[5052]: I1203 12:50:28.356584 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/5282791b-cdf2-4e89-a273-89bc6550f531-db-sync-config-data\") pod \"glance-db-sync-p8q2q\" (UID: \"5282791b-cdf2-4e89-a273-89bc6550f531\") " pod="openstack-kuttl-tests/glance-db-sync-p8q2q" Dec 03 12:50:28 crc kubenswrapper[5052]: I1203 12:50:28.356662 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/5282791b-cdf2-4e89-a273-89bc6550f531-config-data\") pod \"glance-db-sync-p8q2q\" (UID: \"5282791b-cdf2-4e89-a273-89bc6550f531\") " pod="openstack-kuttl-tests/glance-db-sync-p8q2q" Dec 03 12:50:28 crc kubenswrapper[5052]: I1203 12:50:28.361092 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/5282791b-cdf2-4e89-a273-89bc6550f531-config-data\") pod \"glance-db-sync-p8q2q\" (UID: \"5282791b-cdf2-4e89-a273-89bc6550f531\") " pod="openstack-kuttl-tests/glance-db-sync-p8q2q" Dec 03 12:50:28 crc kubenswrapper[5052]: I1203 12:50:28.361113 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5282791b-cdf2-4e89-a273-89bc6550f531-combined-ca-bundle\") pod \"glance-db-sync-p8q2q\" (UID: \"5282791b-cdf2-4e89-a273-89bc6550f531\") " pod="openstack-kuttl-tests/glance-db-sync-p8q2q" Dec 03 12:50:28 crc kubenswrapper[5052]: I1203 12:50:28.361806 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/5282791b-cdf2-4e89-a273-89bc6550f531-db-sync-config-data\") pod \"glance-db-sync-p8q2q\" (UID: \"5282791b-cdf2-4e89-a273-89bc6550f531\") " pod="openstack-kuttl-tests/glance-db-sync-p8q2q" Dec 03 12:50:28 crc kubenswrapper[5052]: I1203 12:50:28.381123 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-kmz6b\" (UniqueName: \"kubernetes.io/projected/5282791b-cdf2-4e89-a273-89bc6550f531-kube-api-access-kmz6b\") pod \"glance-db-sync-p8q2q\" (UID: \"5282791b-cdf2-4e89-a273-89bc6550f531\") " pod="openstack-kuttl-tests/glance-db-sync-p8q2q" Dec 03 12:50:28 crc kubenswrapper[5052]: I1203 12:50:28.458810 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-db-sync-p8q2q" Dec 03 12:50:28 crc kubenswrapper[5052]: I1203 12:50:28.932843 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-db-sync-p8q2q"] Dec 03 12:50:29 crc kubenswrapper[5052]: I1203 12:50:29.164665 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-db-sync-p8q2q" event={"ID":"5282791b-cdf2-4e89-a273-89bc6550f531","Type":"ContainerStarted","Data":"2efcd82d98adb393d64bf27c02580db3f2b6676b34341c0817867550f41ca4e4"} Dec 03 12:50:29 crc kubenswrapper[5052]: I1203 12:50:29.284898 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/swift-ring-rebalance-7jc4g"] Dec 03 12:50:29 crc kubenswrapper[5052]: I1203 12:50:29.286288 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-ring-rebalance-7jc4g" Dec 03 12:50:29 crc kubenswrapper[5052]: I1203 12:50:29.288894 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"swift-proxy-config-data" Dec 03 12:50:29 crc kubenswrapper[5052]: I1203 12:50:29.289199 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"swift-ring-config-data" Dec 03 12:50:29 crc kubenswrapper[5052]: I1203 12:50:29.289552 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"swift-ring-scripts" Dec 03 12:50:29 crc kubenswrapper[5052]: I1203 12:50:29.324384 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/swift-ring-rebalance-7jc4g"] Dec 03 12:50:29 crc kubenswrapper[5052]: E1203 12:50:29.331431 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[combined-ca-bundle dispersionconf etc-swift kube-api-access-4w4lm ring-data-devices scripts swiftconf], unattached volumes=[], failed to process volumes=[]: context canceled" pod="openstack-kuttl-tests/swift-ring-rebalance-7jc4g" podUID="7dec2754-b782-480a-aec7-047d024dd80d" Dec 03 12:50:29 crc kubenswrapper[5052]: I1203 12:50:29.341799 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/swift-ring-rebalance-sfbr9"] Dec 03 12:50:29 crc kubenswrapper[5052]: I1203 12:50:29.343203 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-ring-rebalance-sfbr9" Dec 03 12:50:29 crc kubenswrapper[5052]: I1203 12:50:29.351650 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/swift-ring-rebalance-7jc4g"] Dec 03 12:50:29 crc kubenswrapper[5052]: I1203 12:50:29.360322 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/swift-ring-rebalance-sfbr9"] Dec 03 12:50:29 crc kubenswrapper[5052]: I1203 12:50:29.373651 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/7dec2754-b782-480a-aec7-047d024dd80d-scripts\") pod \"swift-ring-rebalance-7jc4g\" (UID: \"7dec2754-b782-480a-aec7-047d024dd80d\") " pod="openstack-kuttl-tests/swift-ring-rebalance-7jc4g" Dec 03 12:50:29 crc kubenswrapper[5052]: I1203 12:50:29.373730 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4-etc-swift\") pod \"swift-storage-0\" (UID: \"5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:50:29 crc kubenswrapper[5052]: I1203 12:50:29.373779 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/7dec2754-b782-480a-aec7-047d024dd80d-combined-ca-bundle\") pod \"swift-ring-rebalance-7jc4g\" (UID: \"7dec2754-b782-480a-aec7-047d024dd80d\") " pod="openstack-kuttl-tests/swift-ring-rebalance-7jc4g" Dec 03 12:50:29 crc kubenswrapper[5052]: I1203 12:50:29.373815 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"swiftconf\" (UniqueName: \"kubernetes.io/secret/7dec2754-b782-480a-aec7-047d024dd80d-swiftconf\") pod \"swift-ring-rebalance-7jc4g\" (UID: \"7dec2754-b782-480a-aec7-047d024dd80d\") " pod="openstack-kuttl-tests/swift-ring-rebalance-7jc4g" Dec 03 12:50:29 crc kubenswrapper[5052]: I1203 12:50:29.373843 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ring-data-devices\" (UniqueName: \"kubernetes.io/configmap/7dec2754-b782-480a-aec7-047d024dd80d-ring-data-devices\") pod \"swift-ring-rebalance-7jc4g\" (UID: \"7dec2754-b782-480a-aec7-047d024dd80d\") " pod="openstack-kuttl-tests/swift-ring-rebalance-7jc4g" Dec 03 12:50:29 crc kubenswrapper[5052]: I1203 12:50:29.373896 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dispersionconf\" (UniqueName: \"kubernetes.io/secret/7dec2754-b782-480a-aec7-047d024dd80d-dispersionconf\") pod \"swift-ring-rebalance-7jc4g\" (UID: \"7dec2754-b782-480a-aec7-047d024dd80d\") " pod="openstack-kuttl-tests/swift-ring-rebalance-7jc4g" Dec 03 12:50:29 crc kubenswrapper[5052]: I1203 12:50:29.373928 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/empty-dir/7dec2754-b782-480a-aec7-047d024dd80d-etc-swift\") pod \"swift-ring-rebalance-7jc4g\" (UID: \"7dec2754-b782-480a-aec7-047d024dd80d\") " pod="openstack-kuttl-tests/swift-ring-rebalance-7jc4g" Dec 03 12:50:29 crc kubenswrapper[5052]: I1203 12:50:29.373978 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4w4lm\" (UniqueName: \"kubernetes.io/projected/7dec2754-b782-480a-aec7-047d024dd80d-kube-api-access-4w4lm\") pod \"swift-ring-rebalance-7jc4g\" (UID: \"7dec2754-b782-480a-aec7-047d024dd80d\") " pod="openstack-kuttl-tests/swift-ring-rebalance-7jc4g" Dec 03 12:50:29 crc kubenswrapper[5052]: E1203 12:50:29.374165 5052 projected.go:288] Couldn't get configMap openstack-kuttl-tests/swift-ring-files: configmap "swift-ring-files" not found Dec 03 12:50:29 crc kubenswrapper[5052]: E1203 12:50:29.374186 5052 projected.go:194] Error preparing data for projected volume etc-swift for pod openstack-kuttl-tests/swift-storage-0: configmap "swift-ring-files" not found Dec 03 12:50:29 crc kubenswrapper[5052]: E1203 12:50:29.374231 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4-etc-swift podName:5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4 nodeName:}" failed. No retries permitted until 2025-12-03 12:50:33.374213872 +0000 UTC m=+5423.962455646 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "etc-swift" (UniqueName: "kubernetes.io/projected/5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4-etc-swift") pod "swift-storage-0" (UID: "5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4") : configmap "swift-ring-files" not found Dec 03 12:50:29 crc kubenswrapper[5052]: I1203 12:50:29.475277 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/empty-dir/7dec2754-b782-480a-aec7-047d024dd80d-etc-swift\") pod \"swift-ring-rebalance-7jc4g\" (UID: \"7dec2754-b782-480a-aec7-047d024dd80d\") " pod="openstack-kuttl-tests/swift-ring-rebalance-7jc4g" Dec 03 12:50:29 crc kubenswrapper[5052]: I1203 12:50:29.475325 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/empty-dir/efa403f3-536d-4c75-adbe-79ed07819dff-etc-swift\") pod \"swift-ring-rebalance-sfbr9\" (UID: \"efa403f3-536d-4c75-adbe-79ed07819dff\") " pod="openstack-kuttl-tests/swift-ring-rebalance-sfbr9" Dec 03 12:50:29 crc kubenswrapper[5052]: I1203 12:50:29.475355 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hk6fk\" (UniqueName: \"kubernetes.io/projected/efa403f3-536d-4c75-adbe-79ed07819dff-kube-api-access-hk6fk\") pod \"swift-ring-rebalance-sfbr9\" (UID: \"efa403f3-536d-4c75-adbe-79ed07819dff\") " pod="openstack-kuttl-tests/swift-ring-rebalance-sfbr9" Dec 03 12:50:29 crc kubenswrapper[5052]: I1203 12:50:29.475373 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dispersionconf\" (UniqueName: \"kubernetes.io/secret/efa403f3-536d-4c75-adbe-79ed07819dff-dispersionconf\") pod \"swift-ring-rebalance-sfbr9\" (UID: \"efa403f3-536d-4c75-adbe-79ed07819dff\") " pod="openstack-kuttl-tests/swift-ring-rebalance-sfbr9" Dec 03 12:50:29 crc kubenswrapper[5052]: I1203 12:50:29.475388 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ring-data-devices\" (UniqueName: \"kubernetes.io/configmap/efa403f3-536d-4c75-adbe-79ed07819dff-ring-data-devices\") pod \"swift-ring-rebalance-sfbr9\" (UID: \"efa403f3-536d-4c75-adbe-79ed07819dff\") " pod="openstack-kuttl-tests/swift-ring-rebalance-sfbr9" Dec 03 12:50:29 crc kubenswrapper[5052]: I1203 12:50:29.475421 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-4w4lm\" (UniqueName: \"kubernetes.io/projected/7dec2754-b782-480a-aec7-047d024dd80d-kube-api-access-4w4lm\") pod \"swift-ring-rebalance-7jc4g\" (UID: \"7dec2754-b782-480a-aec7-047d024dd80d\") " pod="openstack-kuttl-tests/swift-ring-rebalance-7jc4g" Dec 03 12:50:29 crc kubenswrapper[5052]: I1203 12:50:29.475449 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"swiftconf\" (UniqueName: \"kubernetes.io/secret/efa403f3-536d-4c75-adbe-79ed07819dff-swiftconf\") pod \"swift-ring-rebalance-sfbr9\" (UID: \"efa403f3-536d-4c75-adbe-79ed07819dff\") " pod="openstack-kuttl-tests/swift-ring-rebalance-sfbr9" Dec 03 12:50:29 crc kubenswrapper[5052]: I1203 12:50:29.475478 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/7dec2754-b782-480a-aec7-047d024dd80d-scripts\") pod \"swift-ring-rebalance-7jc4g\" (UID: \"7dec2754-b782-480a-aec7-047d024dd80d\") " pod="openstack-kuttl-tests/swift-ring-rebalance-7jc4g" Dec 03 12:50:29 crc kubenswrapper[5052]: I1203 12:50:29.475541 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/efa403f3-536d-4c75-adbe-79ed07819dff-scripts\") pod \"swift-ring-rebalance-sfbr9\" (UID: \"efa403f3-536d-4c75-adbe-79ed07819dff\") " pod="openstack-kuttl-tests/swift-ring-rebalance-sfbr9" Dec 03 12:50:29 crc kubenswrapper[5052]: I1203 12:50:29.475565 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/efa403f3-536d-4c75-adbe-79ed07819dff-combined-ca-bundle\") pod \"swift-ring-rebalance-sfbr9\" (UID: \"efa403f3-536d-4c75-adbe-79ed07819dff\") " pod="openstack-kuttl-tests/swift-ring-rebalance-sfbr9" Dec 03 12:50:29 crc kubenswrapper[5052]: I1203 12:50:29.475598 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/7dec2754-b782-480a-aec7-047d024dd80d-combined-ca-bundle\") pod \"swift-ring-rebalance-7jc4g\" (UID: \"7dec2754-b782-480a-aec7-047d024dd80d\") " pod="openstack-kuttl-tests/swift-ring-rebalance-7jc4g" Dec 03 12:50:29 crc kubenswrapper[5052]: I1203 12:50:29.475625 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"swiftconf\" (UniqueName: \"kubernetes.io/secret/7dec2754-b782-480a-aec7-047d024dd80d-swiftconf\") pod \"swift-ring-rebalance-7jc4g\" (UID: \"7dec2754-b782-480a-aec7-047d024dd80d\") " pod="openstack-kuttl-tests/swift-ring-rebalance-7jc4g" Dec 03 12:50:29 crc kubenswrapper[5052]: I1203 12:50:29.475649 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ring-data-devices\" (UniqueName: \"kubernetes.io/configmap/7dec2754-b782-480a-aec7-047d024dd80d-ring-data-devices\") pod \"swift-ring-rebalance-7jc4g\" (UID: \"7dec2754-b782-480a-aec7-047d024dd80d\") " pod="openstack-kuttl-tests/swift-ring-rebalance-7jc4g" Dec 03 12:50:29 crc kubenswrapper[5052]: I1203 12:50:29.475693 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"dispersionconf\" (UniqueName: \"kubernetes.io/secret/7dec2754-b782-480a-aec7-047d024dd80d-dispersionconf\") pod \"swift-ring-rebalance-7jc4g\" (UID: \"7dec2754-b782-480a-aec7-047d024dd80d\") " pod="openstack-kuttl-tests/swift-ring-rebalance-7jc4g" Dec 03 12:50:29 crc kubenswrapper[5052]: I1203 12:50:29.476408 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"etc-swift\" (UniqueName: \"kubernetes.io/empty-dir/7dec2754-b782-480a-aec7-047d024dd80d-etc-swift\") pod \"swift-ring-rebalance-7jc4g\" (UID: \"7dec2754-b782-480a-aec7-047d024dd80d\") " pod="openstack-kuttl-tests/swift-ring-rebalance-7jc4g" Dec 03 12:50:29 crc kubenswrapper[5052]: I1203 12:50:29.477079 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/7dec2754-b782-480a-aec7-047d024dd80d-scripts\") pod \"swift-ring-rebalance-7jc4g\" (UID: \"7dec2754-b782-480a-aec7-047d024dd80d\") " pod="openstack-kuttl-tests/swift-ring-rebalance-7jc4g" Dec 03 12:50:29 crc kubenswrapper[5052]: I1203 12:50:29.479611 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"dispersionconf\" (UniqueName: \"kubernetes.io/secret/7dec2754-b782-480a-aec7-047d024dd80d-dispersionconf\") pod \"swift-ring-rebalance-7jc4g\" (UID: \"7dec2754-b782-480a-aec7-047d024dd80d\") " pod="openstack-kuttl-tests/swift-ring-rebalance-7jc4g" Dec 03 12:50:29 crc kubenswrapper[5052]: I1203 12:50:29.479933 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ring-data-devices\" (UniqueName: \"kubernetes.io/configmap/7dec2754-b782-480a-aec7-047d024dd80d-ring-data-devices\") pod \"swift-ring-rebalance-7jc4g\" (UID: \"7dec2754-b782-480a-aec7-047d024dd80d\") " pod="openstack-kuttl-tests/swift-ring-rebalance-7jc4g" Dec 03 12:50:29 crc kubenswrapper[5052]: I1203 12:50:29.482403 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"swiftconf\" (UniqueName: \"kubernetes.io/secret/7dec2754-b782-480a-aec7-047d024dd80d-swiftconf\") pod \"swift-ring-rebalance-7jc4g\" (UID: \"7dec2754-b782-480a-aec7-047d024dd80d\") " pod="openstack-kuttl-tests/swift-ring-rebalance-7jc4g" Dec 03 12:50:29 crc kubenswrapper[5052]: I1203 12:50:29.482769 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/7dec2754-b782-480a-aec7-047d024dd80d-combined-ca-bundle\") pod \"swift-ring-rebalance-7jc4g\" (UID: \"7dec2754-b782-480a-aec7-047d024dd80d\") " pod="openstack-kuttl-tests/swift-ring-rebalance-7jc4g" Dec 03 12:50:29 crc kubenswrapper[5052]: I1203 12:50:29.496537 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-4w4lm\" (UniqueName: \"kubernetes.io/projected/7dec2754-b782-480a-aec7-047d024dd80d-kube-api-access-4w4lm\") pod \"swift-ring-rebalance-7jc4g\" (UID: \"7dec2754-b782-480a-aec7-047d024dd80d\") " pod="openstack-kuttl-tests/swift-ring-rebalance-7jc4g" Dec 03 12:50:29 crc kubenswrapper[5052]: I1203 12:50:29.577459 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/empty-dir/efa403f3-536d-4c75-adbe-79ed07819dff-etc-swift\") pod \"swift-ring-rebalance-sfbr9\" (UID: \"efa403f3-536d-4c75-adbe-79ed07819dff\") " pod="openstack-kuttl-tests/swift-ring-rebalance-sfbr9" Dec 03 12:50:29 crc kubenswrapper[5052]: I1203 12:50:29.577559 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-hk6fk\" (UniqueName: \"kubernetes.io/projected/efa403f3-536d-4c75-adbe-79ed07819dff-kube-api-access-hk6fk\") pod \"swift-ring-rebalance-sfbr9\" (UID: \"efa403f3-536d-4c75-adbe-79ed07819dff\") " pod="openstack-kuttl-tests/swift-ring-rebalance-sfbr9" Dec 03 12:50:29 crc kubenswrapper[5052]: I1203 12:50:29.577584 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"dispersionconf\" (UniqueName: \"kubernetes.io/secret/efa403f3-536d-4c75-adbe-79ed07819dff-dispersionconf\") pod \"swift-ring-rebalance-sfbr9\" (UID: \"efa403f3-536d-4c75-adbe-79ed07819dff\") " pod="openstack-kuttl-tests/swift-ring-rebalance-sfbr9" Dec 03 12:50:29 crc kubenswrapper[5052]: I1203 12:50:29.577599 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ring-data-devices\" (UniqueName: \"kubernetes.io/configmap/efa403f3-536d-4c75-adbe-79ed07819dff-ring-data-devices\") pod \"swift-ring-rebalance-sfbr9\" (UID: \"efa403f3-536d-4c75-adbe-79ed07819dff\") " pod="openstack-kuttl-tests/swift-ring-rebalance-sfbr9" Dec 03 12:50:29 crc kubenswrapper[5052]: I1203 12:50:29.577639 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"swiftconf\" (UniqueName: \"kubernetes.io/secret/efa403f3-536d-4c75-adbe-79ed07819dff-swiftconf\") pod \"swift-ring-rebalance-sfbr9\" (UID: \"efa403f3-536d-4c75-adbe-79ed07819dff\") " pod="openstack-kuttl-tests/swift-ring-rebalance-sfbr9" Dec 03 12:50:29 crc kubenswrapper[5052]: I1203 12:50:29.577696 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/efa403f3-536d-4c75-adbe-79ed07819dff-scripts\") pod \"swift-ring-rebalance-sfbr9\" (UID: \"efa403f3-536d-4c75-adbe-79ed07819dff\") " pod="openstack-kuttl-tests/swift-ring-rebalance-sfbr9" Dec 03 12:50:29 crc kubenswrapper[5052]: I1203 12:50:29.577714 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/efa403f3-536d-4c75-adbe-79ed07819dff-combined-ca-bundle\") pod \"swift-ring-rebalance-sfbr9\" (UID: \"efa403f3-536d-4c75-adbe-79ed07819dff\") " pod="openstack-kuttl-tests/swift-ring-rebalance-sfbr9" Dec 03 12:50:29 crc kubenswrapper[5052]: I1203 12:50:29.578278 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"etc-swift\" (UniqueName: \"kubernetes.io/empty-dir/efa403f3-536d-4c75-adbe-79ed07819dff-etc-swift\") pod \"swift-ring-rebalance-sfbr9\" (UID: \"efa403f3-536d-4c75-adbe-79ed07819dff\") " pod="openstack-kuttl-tests/swift-ring-rebalance-sfbr9" Dec 03 12:50:29 crc kubenswrapper[5052]: I1203 12:50:29.578505 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ring-data-devices\" (UniqueName: \"kubernetes.io/configmap/efa403f3-536d-4c75-adbe-79ed07819dff-ring-data-devices\") pod \"swift-ring-rebalance-sfbr9\" (UID: \"efa403f3-536d-4c75-adbe-79ed07819dff\") " pod="openstack-kuttl-tests/swift-ring-rebalance-sfbr9" Dec 03 12:50:29 crc kubenswrapper[5052]: I1203 12:50:29.579059 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/efa403f3-536d-4c75-adbe-79ed07819dff-scripts\") pod \"swift-ring-rebalance-sfbr9\" (UID: \"efa403f3-536d-4c75-adbe-79ed07819dff\") " pod="openstack-kuttl-tests/swift-ring-rebalance-sfbr9" Dec 03 12:50:29 crc kubenswrapper[5052]: I1203 12:50:29.581872 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"dispersionconf\" (UniqueName: \"kubernetes.io/secret/efa403f3-536d-4c75-adbe-79ed07819dff-dispersionconf\") pod \"swift-ring-rebalance-sfbr9\" (UID: \"efa403f3-536d-4c75-adbe-79ed07819dff\") " pod="openstack-kuttl-tests/swift-ring-rebalance-sfbr9" Dec 03 12:50:29 crc kubenswrapper[5052]: I1203 12:50:29.582015 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"swiftconf\" (UniqueName: \"kubernetes.io/secret/efa403f3-536d-4c75-adbe-79ed07819dff-swiftconf\") pod \"swift-ring-rebalance-sfbr9\" (UID: \"efa403f3-536d-4c75-adbe-79ed07819dff\") " pod="openstack-kuttl-tests/swift-ring-rebalance-sfbr9" Dec 03 12:50:29 crc kubenswrapper[5052]: I1203 12:50:29.582436 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/efa403f3-536d-4c75-adbe-79ed07819dff-combined-ca-bundle\") pod \"swift-ring-rebalance-sfbr9\" (UID: \"efa403f3-536d-4c75-adbe-79ed07819dff\") " pod="openstack-kuttl-tests/swift-ring-rebalance-sfbr9" Dec 03 12:50:29 crc kubenswrapper[5052]: I1203 12:50:29.597888 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-hk6fk\" (UniqueName: \"kubernetes.io/projected/efa403f3-536d-4c75-adbe-79ed07819dff-kube-api-access-hk6fk\") pod \"swift-ring-rebalance-sfbr9\" (UID: \"efa403f3-536d-4c75-adbe-79ed07819dff\") " pod="openstack-kuttl-tests/swift-ring-rebalance-sfbr9" Dec 03 12:50:29 crc kubenswrapper[5052]: I1203 12:50:29.690420 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-ring-rebalance-sfbr9" Dec 03 12:50:30 crc kubenswrapper[5052]: I1203 12:50:30.125627 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/swift-ring-rebalance-sfbr9"] Dec 03 12:50:30 crc kubenswrapper[5052]: W1203 12:50:30.135991 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podefa403f3_536d_4c75_adbe_79ed07819dff.slice/crio-34978c536837553a8c79ceeab56200fd5b7011762297e49eb7ee1ba7c0dae054 WatchSource:0}: Error finding container 34978c536837553a8c79ceeab56200fd5b7011762297e49eb7ee1ba7c0dae054: Status 404 returned error can't find the container with id 34978c536837553a8c79ceeab56200fd5b7011762297e49eb7ee1ba7c0dae054 Dec 03 12:50:30 crc kubenswrapper[5052]: I1203 12:50:30.181035 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-db-sync-p8q2q" event={"ID":"5282791b-cdf2-4e89-a273-89bc6550f531","Type":"ContainerStarted","Data":"5c9c7136b870fdaefa424e5ed10b8c43b181bc99a764b6eae51d7e74f33874d0"} Dec 03 12:50:30 crc kubenswrapper[5052]: I1203 12:50:30.184070 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-ring-rebalance-7jc4g" Dec 03 12:50:30 crc kubenswrapper[5052]: I1203 12:50:30.184642 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-ring-rebalance-sfbr9" event={"ID":"efa403f3-536d-4c75-adbe-79ed07819dff","Type":"ContainerStarted","Data":"34978c536837553a8c79ceeab56200fd5b7011762297e49eb7ee1ba7c0dae054"} Dec 03 12:50:30 crc kubenswrapper[5052]: I1203 12:50:30.195969 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-ring-rebalance-7jc4g" Dec 03 12:50:30 crc kubenswrapper[5052]: I1203 12:50:30.291554 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ring-data-devices\" (UniqueName: \"kubernetes.io/configmap/7dec2754-b782-480a-aec7-047d024dd80d-ring-data-devices\") pod \"7dec2754-b782-480a-aec7-047d024dd80d\" (UID: \"7dec2754-b782-480a-aec7-047d024dd80d\") " Dec 03 12:50:30 crc kubenswrapper[5052]: I1203 12:50:30.292024 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/7dec2754-b782-480a-aec7-047d024dd80d-scripts\") pod \"7dec2754-b782-480a-aec7-047d024dd80d\" (UID: \"7dec2754-b782-480a-aec7-047d024dd80d\") " Dec 03 12:50:30 crc kubenswrapper[5052]: I1203 12:50:30.292054 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4w4lm\" (UniqueName: \"kubernetes.io/projected/7dec2754-b782-480a-aec7-047d024dd80d-kube-api-access-4w4lm\") pod \"7dec2754-b782-480a-aec7-047d024dd80d\" (UID: \"7dec2754-b782-480a-aec7-047d024dd80d\") " Dec 03 12:50:30 crc kubenswrapper[5052]: I1203 12:50:30.292133 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7dec2754-b782-480a-aec7-047d024dd80d-ring-data-devices" (OuterVolumeSpecName: "ring-data-devices") pod "7dec2754-b782-480a-aec7-047d024dd80d" (UID: "7dec2754-b782-480a-aec7-047d024dd80d"). InnerVolumeSpecName "ring-data-devices". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:50:30 crc kubenswrapper[5052]: I1203 12:50:30.292334 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7dec2754-b782-480a-aec7-047d024dd80d-scripts" (OuterVolumeSpecName: "scripts") pod "7dec2754-b782-480a-aec7-047d024dd80d" (UID: "7dec2754-b782-480a-aec7-047d024dd80d"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:50:30 crc kubenswrapper[5052]: I1203 12:50:30.292375 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"swiftconf\" (UniqueName: \"kubernetes.io/secret/7dec2754-b782-480a-aec7-047d024dd80d-swiftconf\") pod \"7dec2754-b782-480a-aec7-047d024dd80d\" (UID: \"7dec2754-b782-480a-aec7-047d024dd80d\") " Dec 03 12:50:30 crc kubenswrapper[5052]: I1203 12:50:30.292448 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"dispersionconf\" (UniqueName: \"kubernetes.io/secret/7dec2754-b782-480a-aec7-047d024dd80d-dispersionconf\") pod \"7dec2754-b782-480a-aec7-047d024dd80d\" (UID: \"7dec2754-b782-480a-aec7-047d024dd80d\") " Dec 03 12:50:30 crc kubenswrapper[5052]: I1203 12:50:30.292485 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/empty-dir/7dec2754-b782-480a-aec7-047d024dd80d-etc-swift\") pod \"7dec2754-b782-480a-aec7-047d024dd80d\" (UID: \"7dec2754-b782-480a-aec7-047d024dd80d\") " Dec 03 12:50:30 crc kubenswrapper[5052]: I1203 12:50:30.292581 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/7dec2754-b782-480a-aec7-047d024dd80d-combined-ca-bundle\") pod \"7dec2754-b782-480a-aec7-047d024dd80d\" (UID: \"7dec2754-b782-480a-aec7-047d024dd80d\") " Dec 03 12:50:30 crc kubenswrapper[5052]: I1203 12:50:30.292799 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7dec2754-b782-480a-aec7-047d024dd80d-etc-swift" (OuterVolumeSpecName: "etc-swift") pod "7dec2754-b782-480a-aec7-047d024dd80d" (UID: "7dec2754-b782-480a-aec7-047d024dd80d"). InnerVolumeSpecName "etc-swift". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:50:30 crc kubenswrapper[5052]: I1203 12:50:30.293062 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/7dec2754-b782-480a-aec7-047d024dd80d-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:50:30 crc kubenswrapper[5052]: I1203 12:50:30.293083 5052 reconciler_common.go:293] "Volume detached for volume \"etc-swift\" (UniqueName: \"kubernetes.io/empty-dir/7dec2754-b782-480a-aec7-047d024dd80d-etc-swift\") on node \"crc\" DevicePath \"\"" Dec 03 12:50:30 crc kubenswrapper[5052]: I1203 12:50:30.293095 5052 reconciler_common.go:293] "Volume detached for volume \"ring-data-devices\" (UniqueName: \"kubernetes.io/configmap/7dec2754-b782-480a-aec7-047d024dd80d-ring-data-devices\") on node \"crc\" DevicePath \"\"" Dec 03 12:50:30 crc kubenswrapper[5052]: I1203 12:50:30.300960 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7dec2754-b782-480a-aec7-047d024dd80d-swiftconf" (OuterVolumeSpecName: "swiftconf") pod "7dec2754-b782-480a-aec7-047d024dd80d" (UID: "7dec2754-b782-480a-aec7-047d024dd80d"). InnerVolumeSpecName "swiftconf". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:50:30 crc kubenswrapper[5052]: I1203 12:50:30.304681 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7dec2754-b782-480a-aec7-047d024dd80d-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "7dec2754-b782-480a-aec7-047d024dd80d" (UID: "7dec2754-b782-480a-aec7-047d024dd80d"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:50:30 crc kubenswrapper[5052]: I1203 12:50:30.304769 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7dec2754-b782-480a-aec7-047d024dd80d-dispersionconf" (OuterVolumeSpecName: "dispersionconf") pod "7dec2754-b782-480a-aec7-047d024dd80d" (UID: "7dec2754-b782-480a-aec7-047d024dd80d"). InnerVolumeSpecName "dispersionconf". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:50:30 crc kubenswrapper[5052]: I1203 12:50:30.311958 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7dec2754-b782-480a-aec7-047d024dd80d-kube-api-access-4w4lm" (OuterVolumeSpecName: "kube-api-access-4w4lm") pod "7dec2754-b782-480a-aec7-047d024dd80d" (UID: "7dec2754-b782-480a-aec7-047d024dd80d"). InnerVolumeSpecName "kube-api-access-4w4lm". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:50:30 crc kubenswrapper[5052]: I1203 12:50:30.394714 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/7dec2754-b782-480a-aec7-047d024dd80d-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:50:30 crc kubenswrapper[5052]: I1203 12:50:30.394772 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-4w4lm\" (UniqueName: \"kubernetes.io/projected/7dec2754-b782-480a-aec7-047d024dd80d-kube-api-access-4w4lm\") on node \"crc\" DevicePath \"\"" Dec 03 12:50:30 crc kubenswrapper[5052]: I1203 12:50:30.394789 5052 reconciler_common.go:293] "Volume detached for volume \"swiftconf\" (UniqueName: \"kubernetes.io/secret/7dec2754-b782-480a-aec7-047d024dd80d-swiftconf\") on node \"crc\" DevicePath \"\"" Dec 03 12:50:30 crc kubenswrapper[5052]: I1203 12:50:30.394802 5052 reconciler_common.go:293] "Volume detached for volume \"dispersionconf\" (UniqueName: \"kubernetes.io/secret/7dec2754-b782-480a-aec7-047d024dd80d-dispersionconf\") on node \"crc\" DevicePath \"\"" Dec 03 12:50:31 crc kubenswrapper[5052]: I1203 12:50:31.194967 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-ring-rebalance-sfbr9" event={"ID":"efa403f3-536d-4c75-adbe-79ed07819dff","Type":"ContainerStarted","Data":"a106c3bce606737209a4ddb9c54e0f19e9031d68589a8d8a5db30f62902c3da1"} Dec 03 12:50:31 crc kubenswrapper[5052]: I1203 12:50:31.195005 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-ring-rebalance-7jc4g" Dec 03 12:50:31 crc kubenswrapper[5052]: I1203 12:50:31.217987 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/glance-db-sync-p8q2q" podStartSLOduration=3.217969541 podStartE2EDuration="3.217969541s" podCreationTimestamp="2025-12-03 12:50:28 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:50:30.205105544 +0000 UTC m=+5420.793347298" watchObservedRunningTime="2025-12-03 12:50:31.217969541 +0000 UTC m=+5421.806211305" Dec 03 12:50:31 crc kubenswrapper[5052]: I1203 12:50:31.244455 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/swift-ring-rebalance-sfbr9" podStartSLOduration=2.244439238 podStartE2EDuration="2.244439238s" podCreationTimestamp="2025-12-03 12:50:29 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:50:31.215644104 +0000 UTC m=+5421.803885868" watchObservedRunningTime="2025-12-03 12:50:31.244439238 +0000 UTC m=+5421.832680992" Dec 03 12:50:31 crc kubenswrapper[5052]: I1203 12:50:31.266495 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/swift-ring-rebalance-7jc4g"] Dec 03 12:50:31 crc kubenswrapper[5052]: I1203 12:50:31.266926 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/swift-ring-rebalance-7jc4g"] Dec 03 12:50:31 crc kubenswrapper[5052]: I1203 12:50:31.683541 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7dec2754-b782-480a-aec7-047d024dd80d" path="/var/lib/kubelet/pods/7dec2754-b782-480a-aec7-047d024dd80d/volumes" Dec 03 12:50:32 crc kubenswrapper[5052]: I1203 12:50:32.102871 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/keystone-db-create-zdrgs"] Dec 03 12:50:32 crc kubenswrapper[5052]: I1203 12:50:32.104115 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-db-create-zdrgs" Dec 03 12:50:32 crc kubenswrapper[5052]: I1203 12:50:32.112328 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-db-create-zdrgs"] Dec 03 12:50:32 crc kubenswrapper[5052]: I1203 12:50:32.223853 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/1e8517f3-2136-4b41-89bc-115f5c14b265-operator-scripts\") pod \"keystone-db-create-zdrgs\" (UID: \"1e8517f3-2136-4b41-89bc-115f5c14b265\") " pod="openstack-kuttl-tests/keystone-db-create-zdrgs" Dec 03 12:50:32 crc kubenswrapper[5052]: I1203 12:50:32.224320 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hpr7b\" (UniqueName: \"kubernetes.io/projected/1e8517f3-2136-4b41-89bc-115f5c14b265-kube-api-access-hpr7b\") pod \"keystone-db-create-zdrgs\" (UID: \"1e8517f3-2136-4b41-89bc-115f5c14b265\") " pod="openstack-kuttl-tests/keystone-db-create-zdrgs" Dec 03 12:50:32 crc kubenswrapper[5052]: I1203 12:50:32.326238 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-hpr7b\" (UniqueName: \"kubernetes.io/projected/1e8517f3-2136-4b41-89bc-115f5c14b265-kube-api-access-hpr7b\") pod \"keystone-db-create-zdrgs\" (UID: \"1e8517f3-2136-4b41-89bc-115f5c14b265\") " pod="openstack-kuttl-tests/keystone-db-create-zdrgs" Dec 03 12:50:32 crc kubenswrapper[5052]: I1203 12:50:32.326361 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/1e8517f3-2136-4b41-89bc-115f5c14b265-operator-scripts\") pod \"keystone-db-create-zdrgs\" (UID: \"1e8517f3-2136-4b41-89bc-115f5c14b265\") " pod="openstack-kuttl-tests/keystone-db-create-zdrgs" Dec 03 12:50:32 crc kubenswrapper[5052]: I1203 12:50:32.327643 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/1e8517f3-2136-4b41-89bc-115f5c14b265-operator-scripts\") pod \"keystone-db-create-zdrgs\" (UID: \"1e8517f3-2136-4b41-89bc-115f5c14b265\") " pod="openstack-kuttl-tests/keystone-db-create-zdrgs" Dec 03 12:50:32 crc kubenswrapper[5052]: I1203 12:50:32.356380 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-hpr7b\" (UniqueName: \"kubernetes.io/projected/1e8517f3-2136-4b41-89bc-115f5c14b265-kube-api-access-hpr7b\") pod \"keystone-db-create-zdrgs\" (UID: \"1e8517f3-2136-4b41-89bc-115f5c14b265\") " pod="openstack-kuttl-tests/keystone-db-create-zdrgs" Dec 03 12:50:32 crc kubenswrapper[5052]: I1203 12:50:32.433280 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-db-create-zdrgs" Dec 03 12:50:32 crc kubenswrapper[5052]: I1203 12:50:32.871641 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-db-create-zdrgs"] Dec 03 12:50:33 crc kubenswrapper[5052]: I1203 12:50:33.212137 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-db-create-zdrgs" event={"ID":"1e8517f3-2136-4b41-89bc-115f5c14b265","Type":"ContainerStarted","Data":"6d62042146fa6a8867f0255494a4c5121abaf607cbfb5898a3d90f59b0dfc75c"} Dec 03 12:50:33 crc kubenswrapper[5052]: I1203 12:50:33.446377 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4-etc-swift\") pod \"swift-storage-0\" (UID: \"5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:50:33 crc kubenswrapper[5052]: E1203 12:50:33.446539 5052 projected.go:288] Couldn't get configMap openstack-kuttl-tests/swift-ring-files: configmap "swift-ring-files" not found Dec 03 12:50:33 crc kubenswrapper[5052]: E1203 12:50:33.446557 5052 projected.go:194] Error preparing data for projected volume etc-swift for pod openstack-kuttl-tests/swift-storage-0: configmap "swift-ring-files" not found Dec 03 12:50:33 crc kubenswrapper[5052]: E1203 12:50:33.446600 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4-etc-swift podName:5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4 nodeName:}" failed. No retries permitted until 2025-12-03 12:50:41.446587105 +0000 UTC m=+5432.034828869 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "etc-swift" (UniqueName: "kubernetes.io/projected/5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4-etc-swift") pod "swift-storage-0" (UID: "5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4") : configmap "swift-ring-files" not found Dec 03 12:50:34 crc kubenswrapper[5052]: I1203 12:50:34.239776 5052 generic.go:334] "Generic (PLEG): container finished" podID="1e8517f3-2136-4b41-89bc-115f5c14b265" containerID="d3bec13b250dc5d3da3916fc0a01000daac906f8a9e04bab954c7cd2f5198046" exitCode=0 Dec 03 12:50:34 crc kubenswrapper[5052]: I1203 12:50:34.239816 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-db-create-zdrgs" event={"ID":"1e8517f3-2136-4b41-89bc-115f5c14b265","Type":"ContainerDied","Data":"d3bec13b250dc5d3da3916fc0a01000daac906f8a9e04bab954c7cd2f5198046"} Dec 03 12:50:34 crc kubenswrapper[5052]: I1203 12:50:34.302533 5052 scope.go:117] "RemoveContainer" containerID="840307e68c72bbebca98f5317317ad1e1192eb2196b68e8a95a9c08f71d63e8b" Dec 03 12:50:34 crc kubenswrapper[5052]: I1203 12:50:34.327873 5052 scope.go:117] "RemoveContainer" containerID="abc3050de7da767805b268c2d3b5fa32accdabf063de3d1794a02867a3ae5a02" Dec 03 12:50:34 crc kubenswrapper[5052]: I1203 12:50:34.358198 5052 scope.go:117] "RemoveContainer" containerID="80c94eaeae20bd3937789c76d0b2f5e78f8cdbd6d6a0f6314ae1b3855ffcc22b" Dec 03 12:50:34 crc kubenswrapper[5052]: I1203 12:50:34.392562 5052 scope.go:117] "RemoveContainer" containerID="8af40cfb10f5db16ec28ea25c0561fa79c3cc14b977ccb702d77cd50dcbe5974" Dec 03 12:50:34 crc kubenswrapper[5052]: I1203 12:50:34.451097 5052 scope.go:117] "RemoveContainer" containerID="242b3296cda414e10917d8871e4c46cde20cc551a78ee9308847f958d3879db1" Dec 03 12:50:34 crc kubenswrapper[5052]: I1203 12:50:34.470339 5052 scope.go:117] "RemoveContainer" containerID="c00bc3cdc6070763956a6f62a9aae570b12ea54c5b9b331777f9d6473f462dac" Dec 03 12:50:34 crc kubenswrapper[5052]: I1203 12:50:34.500296 5052 scope.go:117] "RemoveContainer" containerID="588b5370b468d35103c4a171221a0e2f4784fb7d8d8468ab69a7e96a141588db" Dec 03 12:50:34 crc kubenswrapper[5052]: I1203 12:50:34.518848 5052 scope.go:117] "RemoveContainer" containerID="945804751d5a870a849fdb2c2fced6c4170c19fc2099d177cfebd94e18f4ec26" Dec 03 12:50:34 crc kubenswrapper[5052]: I1203 12:50:34.539102 5052 scope.go:117] "RemoveContainer" containerID="3568fe20f05b7aa147cdfcb12cb7bb90f5c1e4b924211cc0ad6668d18a082ef5" Dec 03 12:50:34 crc kubenswrapper[5052]: I1203 12:50:34.570327 5052 scope.go:117] "RemoveContainer" containerID="3ac4a2059b84064ce39da686d421b429257a9d750e421df9c7a06c8da2d6d181" Dec 03 12:50:34 crc kubenswrapper[5052]: I1203 12:50:34.590641 5052 scope.go:117] "RemoveContainer" containerID="06fb78df44fbba56afa104136ade5fc0d9d9f01772c05059cf7f5e2ec493e511" Dec 03 12:50:34 crc kubenswrapper[5052]: I1203 12:50:34.609907 5052 scope.go:117] "RemoveContainer" containerID="c3413a78a793e89bd0b3f298b6e5e3d40cfcd493e84ae55bfbc2ea5d5d154265" Dec 03 12:50:34 crc kubenswrapper[5052]: I1203 12:50:34.630705 5052 scope.go:117] "RemoveContainer" containerID="2d10f32dbe1a375fa4b22fdd2415c34559e7c30be12f35f8ef9503c291f76702" Dec 03 12:50:34 crc kubenswrapper[5052]: I1203 12:50:34.659369 5052 scope.go:117] "RemoveContainer" containerID="bbc2755b5351566cbdd1dc7fd68849cb7ea314f303defbebf723032464036311" Dec 03 12:50:35 crc kubenswrapper[5052]: I1203 12:50:35.599254 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-db-create-zdrgs" Dec 03 12:50:35 crc kubenswrapper[5052]: I1203 12:50:35.687675 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/1e8517f3-2136-4b41-89bc-115f5c14b265-operator-scripts\") pod \"1e8517f3-2136-4b41-89bc-115f5c14b265\" (UID: \"1e8517f3-2136-4b41-89bc-115f5c14b265\") " Dec 03 12:50:35 crc kubenswrapper[5052]: I1203 12:50:35.688024 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hpr7b\" (UniqueName: \"kubernetes.io/projected/1e8517f3-2136-4b41-89bc-115f5c14b265-kube-api-access-hpr7b\") pod \"1e8517f3-2136-4b41-89bc-115f5c14b265\" (UID: \"1e8517f3-2136-4b41-89bc-115f5c14b265\") " Dec 03 12:50:35 crc kubenswrapper[5052]: I1203 12:50:35.688583 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1e8517f3-2136-4b41-89bc-115f5c14b265-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "1e8517f3-2136-4b41-89bc-115f5c14b265" (UID: "1e8517f3-2136-4b41-89bc-115f5c14b265"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:50:35 crc kubenswrapper[5052]: I1203 12:50:35.695763 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1e8517f3-2136-4b41-89bc-115f5c14b265-kube-api-access-hpr7b" (OuterVolumeSpecName: "kube-api-access-hpr7b") pod "1e8517f3-2136-4b41-89bc-115f5c14b265" (UID: "1e8517f3-2136-4b41-89bc-115f5c14b265"). InnerVolumeSpecName "kube-api-access-hpr7b". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:50:35 crc kubenswrapper[5052]: I1203 12:50:35.790485 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-hpr7b\" (UniqueName: \"kubernetes.io/projected/1e8517f3-2136-4b41-89bc-115f5c14b265-kube-api-access-hpr7b\") on node \"crc\" DevicePath \"\"" Dec 03 12:50:35 crc kubenswrapper[5052]: I1203 12:50:35.790540 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/1e8517f3-2136-4b41-89bc-115f5c14b265-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:50:36 crc kubenswrapper[5052]: I1203 12:50:36.255969 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-db-create-zdrgs" event={"ID":"1e8517f3-2136-4b41-89bc-115f5c14b265","Type":"ContainerDied","Data":"6d62042146fa6a8867f0255494a4c5121abaf607cbfb5898a3d90f59b0dfc75c"} Dec 03 12:50:36 crc kubenswrapper[5052]: I1203 12:50:36.256305 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6d62042146fa6a8867f0255494a4c5121abaf607cbfb5898a3d90f59b0dfc75c" Dec 03 12:50:36 crc kubenswrapper[5052]: I1203 12:50:36.255983 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-db-create-zdrgs" Dec 03 12:50:36 crc kubenswrapper[5052]: I1203 12:50:36.258277 5052 generic.go:334] "Generic (PLEG): container finished" podID="5282791b-cdf2-4e89-a273-89bc6550f531" containerID="5c9c7136b870fdaefa424e5ed10b8c43b181bc99a764b6eae51d7e74f33874d0" exitCode=0 Dec 03 12:50:36 crc kubenswrapper[5052]: I1203 12:50:36.258356 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-db-sync-p8q2q" event={"ID":"5282791b-cdf2-4e89-a273-89bc6550f531","Type":"ContainerDied","Data":"5c9c7136b870fdaefa424e5ed10b8c43b181bc99a764b6eae51d7e74f33874d0"} Dec 03 12:50:37 crc kubenswrapper[5052]: I1203 12:50:37.631095 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-db-sync-p8q2q" Dec 03 12:50:37 crc kubenswrapper[5052]: I1203 12:50:37.724562 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/5282791b-cdf2-4e89-a273-89bc6550f531-db-sync-config-data\") pod \"5282791b-cdf2-4e89-a273-89bc6550f531\" (UID: \"5282791b-cdf2-4e89-a273-89bc6550f531\") " Dec 03 12:50:37 crc kubenswrapper[5052]: I1203 12:50:37.724619 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/5282791b-cdf2-4e89-a273-89bc6550f531-config-data\") pod \"5282791b-cdf2-4e89-a273-89bc6550f531\" (UID: \"5282791b-cdf2-4e89-a273-89bc6550f531\") " Dec 03 12:50:37 crc kubenswrapper[5052]: I1203 12:50:37.724734 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kmz6b\" (UniqueName: \"kubernetes.io/projected/5282791b-cdf2-4e89-a273-89bc6550f531-kube-api-access-kmz6b\") pod \"5282791b-cdf2-4e89-a273-89bc6550f531\" (UID: \"5282791b-cdf2-4e89-a273-89bc6550f531\") " Dec 03 12:50:37 crc kubenswrapper[5052]: I1203 12:50:37.724824 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5282791b-cdf2-4e89-a273-89bc6550f531-combined-ca-bundle\") pod \"5282791b-cdf2-4e89-a273-89bc6550f531\" (UID: \"5282791b-cdf2-4e89-a273-89bc6550f531\") " Dec 03 12:50:37 crc kubenswrapper[5052]: I1203 12:50:37.734176 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5282791b-cdf2-4e89-a273-89bc6550f531-kube-api-access-kmz6b" (OuterVolumeSpecName: "kube-api-access-kmz6b") pod "5282791b-cdf2-4e89-a273-89bc6550f531" (UID: "5282791b-cdf2-4e89-a273-89bc6550f531"). InnerVolumeSpecName "kube-api-access-kmz6b". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:50:37 crc kubenswrapper[5052]: I1203 12:50:37.743612 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5282791b-cdf2-4e89-a273-89bc6550f531-db-sync-config-data" (OuterVolumeSpecName: "db-sync-config-data") pod "5282791b-cdf2-4e89-a273-89bc6550f531" (UID: "5282791b-cdf2-4e89-a273-89bc6550f531"). InnerVolumeSpecName "db-sync-config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:50:37 crc kubenswrapper[5052]: I1203 12:50:37.767881 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5282791b-cdf2-4e89-a273-89bc6550f531-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "5282791b-cdf2-4e89-a273-89bc6550f531" (UID: "5282791b-cdf2-4e89-a273-89bc6550f531"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:50:37 crc kubenswrapper[5052]: I1203 12:50:37.784716 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5282791b-cdf2-4e89-a273-89bc6550f531-config-data" (OuterVolumeSpecName: "config-data") pod "5282791b-cdf2-4e89-a273-89bc6550f531" (UID: "5282791b-cdf2-4e89-a273-89bc6550f531"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:50:37 crc kubenswrapper[5052]: I1203 12:50:37.826492 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5282791b-cdf2-4e89-a273-89bc6550f531-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:50:37 crc kubenswrapper[5052]: I1203 12:50:37.826540 5052 reconciler_common.go:293] "Volume detached for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/5282791b-cdf2-4e89-a273-89bc6550f531-db-sync-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:50:37 crc kubenswrapper[5052]: I1203 12:50:37.826550 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/5282791b-cdf2-4e89-a273-89bc6550f531-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:50:37 crc kubenswrapper[5052]: I1203 12:50:37.826561 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-kmz6b\" (UniqueName: \"kubernetes.io/projected/5282791b-cdf2-4e89-a273-89bc6550f531-kube-api-access-kmz6b\") on node \"crc\" DevicePath \"\"" Dec 03 12:50:38 crc kubenswrapper[5052]: I1203 12:50:38.281282 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-db-sync-p8q2q" event={"ID":"5282791b-cdf2-4e89-a273-89bc6550f531","Type":"ContainerDied","Data":"2efcd82d98adb393d64bf27c02580db3f2b6676b34341c0817867550f41ca4e4"} Dec 03 12:50:38 crc kubenswrapper[5052]: I1203 12:50:38.281334 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2efcd82d98adb393d64bf27c02580db3f2b6676b34341c0817867550f41ca4e4" Dec 03 12:50:38 crc kubenswrapper[5052]: I1203 12:50:38.281359 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-db-sync-p8q2q" Dec 03 12:50:38 crc kubenswrapper[5052]: I1203 12:50:38.283828 5052 generic.go:334] "Generic (PLEG): container finished" podID="efa403f3-536d-4c75-adbe-79ed07819dff" containerID="a106c3bce606737209a4ddb9c54e0f19e9031d68589a8d8a5db30f62902c3da1" exitCode=0 Dec 03 12:50:38 crc kubenswrapper[5052]: I1203 12:50:38.283959 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-ring-rebalance-sfbr9" event={"ID":"efa403f3-536d-4c75-adbe-79ed07819dff","Type":"ContainerDied","Data":"a106c3bce606737209a4ddb9c54e0f19e9031d68589a8d8a5db30f62902c3da1"} Dec 03 12:50:39 crc kubenswrapper[5052]: I1203 12:50:39.625634 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-ring-rebalance-sfbr9" Dec 03 12:50:39 crc kubenswrapper[5052]: I1203 12:50:39.757261 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ring-data-devices\" (UniqueName: \"kubernetes.io/configmap/efa403f3-536d-4c75-adbe-79ed07819dff-ring-data-devices\") pod \"efa403f3-536d-4c75-adbe-79ed07819dff\" (UID: \"efa403f3-536d-4c75-adbe-79ed07819dff\") " Dec 03 12:50:39 crc kubenswrapper[5052]: I1203 12:50:39.757415 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"swiftconf\" (UniqueName: \"kubernetes.io/secret/efa403f3-536d-4c75-adbe-79ed07819dff-swiftconf\") pod \"efa403f3-536d-4c75-adbe-79ed07819dff\" (UID: \"efa403f3-536d-4c75-adbe-79ed07819dff\") " Dec 03 12:50:39 crc kubenswrapper[5052]: I1203 12:50:39.757641 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/efa403f3-536d-4c75-adbe-79ed07819dff-scripts\") pod \"efa403f3-536d-4c75-adbe-79ed07819dff\" (UID: \"efa403f3-536d-4c75-adbe-79ed07819dff\") " Dec 03 12:50:39 crc kubenswrapper[5052]: I1203 12:50:39.757752 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/empty-dir/efa403f3-536d-4c75-adbe-79ed07819dff-etc-swift\") pod \"efa403f3-536d-4c75-adbe-79ed07819dff\" (UID: \"efa403f3-536d-4c75-adbe-79ed07819dff\") " Dec 03 12:50:39 crc kubenswrapper[5052]: I1203 12:50:39.757871 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/efa403f3-536d-4c75-adbe-79ed07819dff-combined-ca-bundle\") pod \"efa403f3-536d-4c75-adbe-79ed07819dff\" (UID: \"efa403f3-536d-4c75-adbe-79ed07819dff\") " Dec 03 12:50:39 crc kubenswrapper[5052]: I1203 12:50:39.758072 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hk6fk\" (UniqueName: \"kubernetes.io/projected/efa403f3-536d-4c75-adbe-79ed07819dff-kube-api-access-hk6fk\") pod \"efa403f3-536d-4c75-adbe-79ed07819dff\" (UID: \"efa403f3-536d-4c75-adbe-79ed07819dff\") " Dec 03 12:50:39 crc kubenswrapper[5052]: I1203 12:50:39.758131 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"dispersionconf\" (UniqueName: \"kubernetes.io/secret/efa403f3-536d-4c75-adbe-79ed07819dff-dispersionconf\") pod \"efa403f3-536d-4c75-adbe-79ed07819dff\" (UID: \"efa403f3-536d-4c75-adbe-79ed07819dff\") " Dec 03 12:50:39 crc kubenswrapper[5052]: I1203 12:50:39.758306 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/efa403f3-536d-4c75-adbe-79ed07819dff-ring-data-devices" (OuterVolumeSpecName: "ring-data-devices") pod "efa403f3-536d-4c75-adbe-79ed07819dff" (UID: "efa403f3-536d-4c75-adbe-79ed07819dff"). InnerVolumeSpecName "ring-data-devices". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:50:39 crc kubenswrapper[5052]: I1203 12:50:39.758550 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/efa403f3-536d-4c75-adbe-79ed07819dff-etc-swift" (OuterVolumeSpecName: "etc-swift") pod "efa403f3-536d-4c75-adbe-79ed07819dff" (UID: "efa403f3-536d-4c75-adbe-79ed07819dff"). InnerVolumeSpecName "etc-swift". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:50:39 crc kubenswrapper[5052]: I1203 12:50:39.758883 5052 reconciler_common.go:293] "Volume detached for volume \"ring-data-devices\" (UniqueName: \"kubernetes.io/configmap/efa403f3-536d-4c75-adbe-79ed07819dff-ring-data-devices\") on node \"crc\" DevicePath \"\"" Dec 03 12:50:39 crc kubenswrapper[5052]: I1203 12:50:39.758903 5052 reconciler_common.go:293] "Volume detached for volume \"etc-swift\" (UniqueName: \"kubernetes.io/empty-dir/efa403f3-536d-4c75-adbe-79ed07819dff-etc-swift\") on node \"crc\" DevicePath \"\"" Dec 03 12:50:39 crc kubenswrapper[5052]: I1203 12:50:39.763709 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/efa403f3-536d-4c75-adbe-79ed07819dff-kube-api-access-hk6fk" (OuterVolumeSpecName: "kube-api-access-hk6fk") pod "efa403f3-536d-4c75-adbe-79ed07819dff" (UID: "efa403f3-536d-4c75-adbe-79ed07819dff"). InnerVolumeSpecName "kube-api-access-hk6fk". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:50:39 crc kubenswrapper[5052]: I1203 12:50:39.766755 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/efa403f3-536d-4c75-adbe-79ed07819dff-dispersionconf" (OuterVolumeSpecName: "dispersionconf") pod "efa403f3-536d-4c75-adbe-79ed07819dff" (UID: "efa403f3-536d-4c75-adbe-79ed07819dff"). InnerVolumeSpecName "dispersionconf". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:50:39 crc kubenswrapper[5052]: I1203 12:50:39.778415 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/efa403f3-536d-4c75-adbe-79ed07819dff-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "efa403f3-536d-4c75-adbe-79ed07819dff" (UID: "efa403f3-536d-4c75-adbe-79ed07819dff"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:50:39 crc kubenswrapper[5052]: I1203 12:50:39.780334 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/efa403f3-536d-4c75-adbe-79ed07819dff-swiftconf" (OuterVolumeSpecName: "swiftconf") pod "efa403f3-536d-4c75-adbe-79ed07819dff" (UID: "efa403f3-536d-4c75-adbe-79ed07819dff"). InnerVolumeSpecName "swiftconf". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:50:39 crc kubenswrapper[5052]: I1203 12:50:39.793463 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/efa403f3-536d-4c75-adbe-79ed07819dff-scripts" (OuterVolumeSpecName: "scripts") pod "efa403f3-536d-4c75-adbe-79ed07819dff" (UID: "efa403f3-536d-4c75-adbe-79ed07819dff"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:50:39 crc kubenswrapper[5052]: I1203 12:50:39.861013 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/efa403f3-536d-4c75-adbe-79ed07819dff-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:50:39 crc kubenswrapper[5052]: I1203 12:50:39.861048 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/efa403f3-536d-4c75-adbe-79ed07819dff-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:50:39 crc kubenswrapper[5052]: I1203 12:50:39.861065 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-hk6fk\" (UniqueName: \"kubernetes.io/projected/efa403f3-536d-4c75-adbe-79ed07819dff-kube-api-access-hk6fk\") on node \"crc\" DevicePath \"\"" Dec 03 12:50:39 crc kubenswrapper[5052]: I1203 12:50:39.861078 5052 reconciler_common.go:293] "Volume detached for volume \"dispersionconf\" (UniqueName: \"kubernetes.io/secret/efa403f3-536d-4c75-adbe-79ed07819dff-dispersionconf\") on node \"crc\" DevicePath \"\"" Dec 03 12:50:39 crc kubenswrapper[5052]: I1203 12:50:39.861087 5052 reconciler_common.go:293] "Volume detached for volume \"swiftconf\" (UniqueName: \"kubernetes.io/secret/efa403f3-536d-4c75-adbe-79ed07819dff-swiftconf\") on node \"crc\" DevicePath \"\"" Dec 03 12:50:40 crc kubenswrapper[5052]: I1203 12:50:40.301772 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-ring-rebalance-sfbr9" event={"ID":"efa403f3-536d-4c75-adbe-79ed07819dff","Type":"ContainerDied","Data":"34978c536837553a8c79ceeab56200fd5b7011762297e49eb7ee1ba7c0dae054"} Dec 03 12:50:40 crc kubenswrapper[5052]: I1203 12:50:40.301999 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="34978c536837553a8c79ceeab56200fd5b7011762297e49eb7ee1ba7c0dae054" Dec 03 12:50:40 crc kubenswrapper[5052]: I1203 12:50:40.302022 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-ring-rebalance-sfbr9" Dec 03 12:50:40 crc kubenswrapper[5052]: I1203 12:50:40.863434 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:50:41 crc kubenswrapper[5052]: I1203 12:50:41.488420 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4-etc-swift\") pod \"swift-storage-0\" (UID: \"5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:50:41 crc kubenswrapper[5052]: I1203 12:50:41.494596 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4-etc-swift\") pod \"swift-storage-0\" (UID: \"5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4\") " pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:50:41 crc kubenswrapper[5052]: I1203 12:50:41.539700 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:50:42 crc kubenswrapper[5052]: I1203 12:50:42.034167 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/swift-storage-0"] Dec 03 12:50:42 crc kubenswrapper[5052]: W1203 12:50:42.036340 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5a63be9b_cdff_4d47_b2ad_a5039b1b9fc4.slice/crio-341fe1e40774ad3e533141e2a85fa7ab7d9d5691734e70031925b6befaf69079 WatchSource:0}: Error finding container 341fe1e40774ad3e533141e2a85fa7ab7d9d5691734e70031925b6befaf69079: Status 404 returned error can't find the container with id 341fe1e40774ad3e533141e2a85fa7ab7d9d5691734e70031925b6befaf69079 Dec 03 12:50:42 crc kubenswrapper[5052]: I1203 12:50:42.318963 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4","Type":"ContainerStarted","Data":"f162a9aaff5508763fc99c270ac25a9155335aac10652767b1730627c88c0d13"} Dec 03 12:50:42 crc kubenswrapper[5052]: I1203 12:50:42.319251 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4","Type":"ContainerStarted","Data":"341fe1e40774ad3e533141e2a85fa7ab7d9d5691734e70031925b6befaf69079"} Dec 03 12:50:43 crc kubenswrapper[5052]: I1203 12:50:43.333010 5052 generic.go:334] "Generic (PLEG): container finished" podID="568addd3-4ad3-47c3-adae-693691e7cd40" containerID="d3e528340d7b7c51ad079f738747e4effb2e76401a14409088fb9ae1abff9903" exitCode=0 Dec 03 12:50:43 crc kubenswrapper[5052]: I1203 12:50:43.333104 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" event={"ID":"568addd3-4ad3-47c3-adae-693691e7cd40","Type":"ContainerDied","Data":"d3e528340d7b7c51ad079f738747e4effb2e76401a14409088fb9ae1abff9903"} Dec 03 12:50:43 crc kubenswrapper[5052]: I1203 12:50:43.345211 5052 generic.go:334] "Generic (PLEG): container finished" podID="e73813f5-f97f-41dd-8a79-6196557d4d1a" containerID="0ef2bfbc937b79d5fdccf169b0b5adaf3d759d2b9428459842a9476d8aacd0c1" exitCode=0 Dec 03 12:50:43 crc kubenswrapper[5052]: I1203 12:50:43.345376 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-server-0" event={"ID":"e73813f5-f97f-41dd-8a79-6196557d4d1a","Type":"ContainerDied","Data":"0ef2bfbc937b79d5fdccf169b0b5adaf3d759d2b9428459842a9476d8aacd0c1"} Dec 03 12:50:43 crc kubenswrapper[5052]: I1203 12:50:43.352725 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4","Type":"ContainerStarted","Data":"79c2a3a0d0be04c4ece62e264a2a1afc1ddae2cbe0714bda786215f70574370c"} Dec 03 12:50:43 crc kubenswrapper[5052]: I1203 12:50:43.352993 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4","Type":"ContainerStarted","Data":"e9ef673f7a7a6d771b2dac62256d6ff0ffb0a20ed29ae2461ff57a96282f22a4"} Dec 03 12:50:43 crc kubenswrapper[5052]: I1203 12:50:43.353112 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4","Type":"ContainerStarted","Data":"722f9b2f73e09eeb2d10bcd607cd1d2c2250a2aefb6a7b7fc5b762f09df66896"} Dec 03 12:50:43 crc kubenswrapper[5052]: I1203 12:50:43.353209 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4","Type":"ContainerStarted","Data":"93e148f1fa8fb3a45b149f7c61848865b732bc5668fbd9e528494e58a56b6237"} Dec 03 12:50:43 crc kubenswrapper[5052]: I1203 12:50:43.353289 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4","Type":"ContainerStarted","Data":"7cd62df542d638c1af171b7f7bdfb4bfa0fa061cd5e93db34b71eef1fee65081"} Dec 03 12:50:43 crc kubenswrapper[5052]: I1203 12:50:43.353373 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4","Type":"ContainerStarted","Data":"5606a2414fb1d7623fb77afb101b103f800b56ca86bd25e193447a537a91edbe"} Dec 03 12:50:44 crc kubenswrapper[5052]: I1203 12:50:44.367874 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4","Type":"ContainerStarted","Data":"71586c32f42d5da42bcbffc73b3fed04e796afb63a8b7d88e60890c23b73e5c0"} Dec 03 12:50:44 crc kubenswrapper[5052]: I1203 12:50:44.368254 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4","Type":"ContainerStarted","Data":"44f9d9990406d3045f888637bfcfe158544e8d9bceeb3e129858be5d63d5b0ef"} Dec 03 12:50:44 crc kubenswrapper[5052]: I1203 12:50:44.368273 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4","Type":"ContainerStarted","Data":"b47992e8799bd9270966dcef43cbed9c315f16df636f79081295a474d02ec3b9"} Dec 03 12:50:44 crc kubenswrapper[5052]: I1203 12:50:44.368287 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4","Type":"ContainerStarted","Data":"1087472cb2f4b1be538bcf920658b4a2f73a08130ae74a54e44b3322500d6407"} Dec 03 12:50:44 crc kubenswrapper[5052]: I1203 12:50:44.368300 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4","Type":"ContainerStarted","Data":"5e0e53488533218e7d56486c1da9b5627e233ae560832e3a38ad6cb8cf881504"} Dec 03 12:50:44 crc kubenswrapper[5052]: I1203 12:50:44.370227 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" event={"ID":"568addd3-4ad3-47c3-adae-693691e7cd40","Type":"ContainerStarted","Data":"dc308258e1e18beb76be744d3270524d74c7f8be911cbba0e2cd35f103d54af5"} Dec 03 12:50:44 crc kubenswrapper[5052]: I1203 12:50:44.370615 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:50:44 crc kubenswrapper[5052]: I1203 12:50:44.373414 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-server-0" event={"ID":"e73813f5-f97f-41dd-8a79-6196557d4d1a","Type":"ContainerStarted","Data":"db4b319560fefd949092d7f9bbfd3ae7aab4d1e6d1c4c90e729f406df14b8b79"} Dec 03 12:50:44 crc kubenswrapper[5052]: I1203 12:50:44.374037 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:50:44 crc kubenswrapper[5052]: I1203 12:50:44.420311 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" podStartSLOduration=37.420290063 podStartE2EDuration="37.420290063s" podCreationTimestamp="2025-12-03 12:50:07 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:50:44.396587106 +0000 UTC m=+5434.984828890" watchObservedRunningTime="2025-12-03 12:50:44.420290063 +0000 UTC m=+5435.008531837" Dec 03 12:50:44 crc kubenswrapper[5052]: I1203 12:50:44.424539 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/rabbitmq-server-0" podStartSLOduration=37.424531615 podStartE2EDuration="37.424531615s" podCreationTimestamp="2025-12-03 12:50:07 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:50:44.419646615 +0000 UTC m=+5435.007888389" watchObservedRunningTime="2025-12-03 12:50:44.424531615 +0000 UTC m=+5435.012773389" Dec 03 12:50:45 crc kubenswrapper[5052]: I1203 12:50:45.397348 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4","Type":"ContainerStarted","Data":"5f94534eaf62ac523a8208a315b484dcd4fe50a4df83c2d6cfacb2dcdfd740c6"} Dec 03 12:50:45 crc kubenswrapper[5052]: I1203 12:50:45.397784 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4","Type":"ContainerStarted","Data":"7bdd0d115f77d5d7457272b4632a96a1cbaef3a87e7a1296d68acdce465508f1"} Dec 03 12:50:45 crc kubenswrapper[5052]: I1203 12:50:45.397817 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4","Type":"ContainerStarted","Data":"62e46329d5f90f5463064339e9bbe330b943da63b752db809975d250c3c5e1ba"} Dec 03 12:50:45 crc kubenswrapper[5052]: I1203 12:50:45.463065 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/swift-storage-0" podStartSLOduration=21.463043155 podStartE2EDuration="21.463043155s" podCreationTimestamp="2025-12-03 12:50:24 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:50:45.433687685 +0000 UTC m=+5436.021929469" watchObservedRunningTime="2025-12-03 12:50:45.463043155 +0000 UTC m=+5436.051284909" Dec 03 12:50:45 crc kubenswrapper[5052]: I1203 12:50:45.607344 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-85c5f46bf7-pqjs4"] Dec 03 12:50:45 crc kubenswrapper[5052]: E1203 12:50:45.607786 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="5282791b-cdf2-4e89-a273-89bc6550f531" containerName="glance-db-sync" Dec 03 12:50:45 crc kubenswrapper[5052]: I1203 12:50:45.607808 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="5282791b-cdf2-4e89-a273-89bc6550f531" containerName="glance-db-sync" Dec 03 12:50:45 crc kubenswrapper[5052]: E1203 12:50:45.607835 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="1e8517f3-2136-4b41-89bc-115f5c14b265" containerName="mariadb-database-create" Dec 03 12:50:45 crc kubenswrapper[5052]: I1203 12:50:45.607843 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="1e8517f3-2136-4b41-89bc-115f5c14b265" containerName="mariadb-database-create" Dec 03 12:50:45 crc kubenswrapper[5052]: E1203 12:50:45.607869 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="efa403f3-536d-4c75-adbe-79ed07819dff" containerName="swift-ring-rebalance" Dec 03 12:50:45 crc kubenswrapper[5052]: I1203 12:50:45.607876 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="efa403f3-536d-4c75-adbe-79ed07819dff" containerName="swift-ring-rebalance" Dec 03 12:50:45 crc kubenswrapper[5052]: I1203 12:50:45.608059 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="5282791b-cdf2-4e89-a273-89bc6550f531" containerName="glance-db-sync" Dec 03 12:50:45 crc kubenswrapper[5052]: I1203 12:50:45.608103 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="1e8517f3-2136-4b41-89bc-115f5c14b265" containerName="mariadb-database-create" Dec 03 12:50:45 crc kubenswrapper[5052]: I1203 12:50:45.608122 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="efa403f3-536d-4c75-adbe-79ed07819dff" containerName="swift-ring-rebalance" Dec 03 12:50:45 crc kubenswrapper[5052]: I1203 12:50:45.609337 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-85c5f46bf7-pqjs4" Dec 03 12:50:45 crc kubenswrapper[5052]: I1203 12:50:45.613658 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"dns-swift-storage-0" Dec 03 12:50:45 crc kubenswrapper[5052]: I1203 12:50:45.635834 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-85c5f46bf7-pqjs4"] Dec 03 12:50:45 crc kubenswrapper[5052]: I1203 12:50:45.763275 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1cf6bdc8-8a27-4cc6-a2e9-f6de73fc1ea6-config\") pod \"dnsmasq-dnsmasq-85c5f46bf7-pqjs4\" (UID: \"1cf6bdc8-8a27-4cc6-a2e9-f6de73fc1ea6\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-85c5f46bf7-pqjs4" Dec 03 12:50:45 crc kubenswrapper[5052]: I1203 12:50:45.763348 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dns-swift-storage-0\" (UniqueName: \"kubernetes.io/configmap/1cf6bdc8-8a27-4cc6-a2e9-f6de73fc1ea6-dns-swift-storage-0\") pod \"dnsmasq-dnsmasq-85c5f46bf7-pqjs4\" (UID: \"1cf6bdc8-8a27-4cc6-a2e9-f6de73fc1ea6\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-85c5f46bf7-pqjs4" Dec 03 12:50:45 crc kubenswrapper[5052]: I1203 12:50:45.763430 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/1cf6bdc8-8a27-4cc6-a2e9-f6de73fc1ea6-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-85c5f46bf7-pqjs4\" (UID: \"1cf6bdc8-8a27-4cc6-a2e9-f6de73fc1ea6\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-85c5f46bf7-pqjs4" Dec 03 12:50:45 crc kubenswrapper[5052]: I1203 12:50:45.763463 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nplrw\" (UniqueName: \"kubernetes.io/projected/1cf6bdc8-8a27-4cc6-a2e9-f6de73fc1ea6-kube-api-access-nplrw\") pod \"dnsmasq-dnsmasq-85c5f46bf7-pqjs4\" (UID: \"1cf6bdc8-8a27-4cc6-a2e9-f6de73fc1ea6\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-85c5f46bf7-pqjs4" Dec 03 12:50:45 crc kubenswrapper[5052]: I1203 12:50:45.865129 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1cf6bdc8-8a27-4cc6-a2e9-f6de73fc1ea6-config\") pod \"dnsmasq-dnsmasq-85c5f46bf7-pqjs4\" (UID: \"1cf6bdc8-8a27-4cc6-a2e9-f6de73fc1ea6\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-85c5f46bf7-pqjs4" Dec 03 12:50:45 crc kubenswrapper[5052]: I1203 12:50:45.865460 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"dns-swift-storage-0\" (UniqueName: \"kubernetes.io/configmap/1cf6bdc8-8a27-4cc6-a2e9-f6de73fc1ea6-dns-swift-storage-0\") pod \"dnsmasq-dnsmasq-85c5f46bf7-pqjs4\" (UID: \"1cf6bdc8-8a27-4cc6-a2e9-f6de73fc1ea6\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-85c5f46bf7-pqjs4" Dec 03 12:50:45 crc kubenswrapper[5052]: I1203 12:50:45.865560 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/1cf6bdc8-8a27-4cc6-a2e9-f6de73fc1ea6-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-85c5f46bf7-pqjs4\" (UID: \"1cf6bdc8-8a27-4cc6-a2e9-f6de73fc1ea6\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-85c5f46bf7-pqjs4" Dec 03 12:50:45 crc kubenswrapper[5052]: I1203 12:50:45.865592 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-nplrw\" (UniqueName: \"kubernetes.io/projected/1cf6bdc8-8a27-4cc6-a2e9-f6de73fc1ea6-kube-api-access-nplrw\") pod \"dnsmasq-dnsmasq-85c5f46bf7-pqjs4\" (UID: \"1cf6bdc8-8a27-4cc6-a2e9-f6de73fc1ea6\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-85c5f46bf7-pqjs4" Dec 03 12:50:45 crc kubenswrapper[5052]: I1203 12:50:45.866021 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1cf6bdc8-8a27-4cc6-a2e9-f6de73fc1ea6-config\") pod \"dnsmasq-dnsmasq-85c5f46bf7-pqjs4\" (UID: \"1cf6bdc8-8a27-4cc6-a2e9-f6de73fc1ea6\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-85c5f46bf7-pqjs4" Dec 03 12:50:45 crc kubenswrapper[5052]: I1203 12:50:45.866245 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"dns-swift-storage-0\" (UniqueName: \"kubernetes.io/configmap/1cf6bdc8-8a27-4cc6-a2e9-f6de73fc1ea6-dns-swift-storage-0\") pod \"dnsmasq-dnsmasq-85c5f46bf7-pqjs4\" (UID: \"1cf6bdc8-8a27-4cc6-a2e9-f6de73fc1ea6\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-85c5f46bf7-pqjs4" Dec 03 12:50:45 crc kubenswrapper[5052]: I1203 12:50:45.866722 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/1cf6bdc8-8a27-4cc6-a2e9-f6de73fc1ea6-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-85c5f46bf7-pqjs4\" (UID: \"1cf6bdc8-8a27-4cc6-a2e9-f6de73fc1ea6\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-85c5f46bf7-pqjs4" Dec 03 12:50:45 crc kubenswrapper[5052]: I1203 12:50:45.886079 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-nplrw\" (UniqueName: \"kubernetes.io/projected/1cf6bdc8-8a27-4cc6-a2e9-f6de73fc1ea6-kube-api-access-nplrw\") pod \"dnsmasq-dnsmasq-85c5f46bf7-pqjs4\" (UID: \"1cf6bdc8-8a27-4cc6-a2e9-f6de73fc1ea6\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-85c5f46bf7-pqjs4" Dec 03 12:50:45 crc kubenswrapper[5052]: I1203 12:50:45.927016 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-85c5f46bf7-pqjs4" Dec 03 12:50:46 crc kubenswrapper[5052]: I1203 12:50:46.411001 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-85c5f46bf7-pqjs4"] Dec 03 12:50:46 crc kubenswrapper[5052]: W1203 12:50:46.417504 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1cf6bdc8_8a27_4cc6_a2e9_f6de73fc1ea6.slice/crio-316b115f05fd73e9974f2538b9459c33cf0d446a63b85964b862e1f96aed1152 WatchSource:0}: Error finding container 316b115f05fd73e9974f2538b9459c33cf0d446a63b85964b862e1f96aed1152: Status 404 returned error can't find the container with id 316b115f05fd73e9974f2538b9459c33cf0d446a63b85964b862e1f96aed1152 Dec 03 12:50:47 crc kubenswrapper[5052]: I1203 12:50:47.417798 5052 generic.go:334] "Generic (PLEG): container finished" podID="1cf6bdc8-8a27-4cc6-a2e9-f6de73fc1ea6" containerID="07e3ceee7a3a89781a782a0bbd0e4e55633ebaf42ef3fc33c4f41527c0bc4140" exitCode=0 Dec 03 12:50:47 crc kubenswrapper[5052]: I1203 12:50:47.417844 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-85c5f46bf7-pqjs4" event={"ID":"1cf6bdc8-8a27-4cc6-a2e9-f6de73fc1ea6","Type":"ContainerDied","Data":"07e3ceee7a3a89781a782a0bbd0e4e55633ebaf42ef3fc33c4f41527c0bc4140"} Dec 03 12:50:47 crc kubenswrapper[5052]: I1203 12:50:47.418112 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-85c5f46bf7-pqjs4" event={"ID":"1cf6bdc8-8a27-4cc6-a2e9-f6de73fc1ea6","Type":"ContainerStarted","Data":"316b115f05fd73e9974f2538b9459c33cf0d446a63b85964b862e1f96aed1152"} Dec 03 12:50:48 crc kubenswrapper[5052]: I1203 12:50:48.431159 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-85c5f46bf7-pqjs4" event={"ID":"1cf6bdc8-8a27-4cc6-a2e9-f6de73fc1ea6","Type":"ContainerStarted","Data":"7764c0629dd26dc1a0c178458e4042481ac430367224d3fc71d16036893f0163"} Dec 03 12:50:48 crc kubenswrapper[5052]: I1203 12:50:48.431599 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-85c5f46bf7-pqjs4" Dec 03 12:50:48 crc kubenswrapper[5052]: I1203 12:50:48.468456 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-85c5f46bf7-pqjs4" podStartSLOduration=3.468439494 podStartE2EDuration="3.468439494s" podCreationTimestamp="2025-12-03 12:50:45 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:50:48.459724824 +0000 UTC m=+5439.047966598" watchObservedRunningTime="2025-12-03 12:50:48.468439494 +0000 UTC m=+5439.056681258" Dec 03 12:50:55 crc kubenswrapper[5052]: I1203 12:50:55.928813 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-85c5f46bf7-pqjs4" Dec 03 12:50:56 crc kubenswrapper[5052]: I1203 12:50:56.000989 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-9t4zp"] Dec 03 12:50:56 crc kubenswrapper[5052]: I1203 12:50:56.001293 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-9t4zp" podUID="59ec248d-84c7-4d23-b9f6-29ea8c2f36fe" containerName="dnsmasq-dns" containerID="cri-o://35ee8cb46ad46c891647ac8e4f3735c080422ea4c90dd9702c115515d3206886" gracePeriod=10 Dec 03 12:50:56 crc kubenswrapper[5052]: I1203 12:50:56.449334 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-9t4zp" Dec 03 12:50:56 crc kubenswrapper[5052]: I1203 12:50:56.512864 5052 generic.go:334] "Generic (PLEG): container finished" podID="59ec248d-84c7-4d23-b9f6-29ea8c2f36fe" containerID="35ee8cb46ad46c891647ac8e4f3735c080422ea4c90dd9702c115515d3206886" exitCode=0 Dec 03 12:50:56 crc kubenswrapper[5052]: I1203 12:50:56.512902 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-9t4zp" event={"ID":"59ec248d-84c7-4d23-b9f6-29ea8c2f36fe","Type":"ContainerDied","Data":"35ee8cb46ad46c891647ac8e4f3735c080422ea4c90dd9702c115515d3206886"} Dec 03 12:50:56 crc kubenswrapper[5052]: I1203 12:50:56.512927 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-9t4zp" event={"ID":"59ec248d-84c7-4d23-b9f6-29ea8c2f36fe","Type":"ContainerDied","Data":"4a1dccd1b6083cf5d0f44566001da9de050803d7e6b9a2639d43f390c4686981"} Dec 03 12:50:56 crc kubenswrapper[5052]: I1203 12:50:56.512942 5052 scope.go:117] "RemoveContainer" containerID="35ee8cb46ad46c891647ac8e4f3735c080422ea4c90dd9702c115515d3206886" Dec 03 12:50:56 crc kubenswrapper[5052]: I1203 12:50:56.513252 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-9t4zp" Dec 03 12:50:56 crc kubenswrapper[5052]: I1203 12:50:56.534822 5052 scope.go:117] "RemoveContainer" containerID="1e281e7872fa27702c42c19852a987a152988cec19cb89cd3614fa3d4e9a5e56" Dec 03 12:50:56 crc kubenswrapper[5052]: I1203 12:50:56.550092 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lj2nq\" (UniqueName: \"kubernetes.io/projected/59ec248d-84c7-4d23-b9f6-29ea8c2f36fe-kube-api-access-lj2nq\") pod \"59ec248d-84c7-4d23-b9f6-29ea8c2f36fe\" (UID: \"59ec248d-84c7-4d23-b9f6-29ea8c2f36fe\") " Dec 03 12:50:56 crc kubenswrapper[5052]: I1203 12:50:56.550528 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/59ec248d-84c7-4d23-b9f6-29ea8c2f36fe-dnsmasq-svc\") pod \"59ec248d-84c7-4d23-b9f6-29ea8c2f36fe\" (UID: \"59ec248d-84c7-4d23-b9f6-29ea8c2f36fe\") " Dec 03 12:50:56 crc kubenswrapper[5052]: I1203 12:50:56.550780 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/59ec248d-84c7-4d23-b9f6-29ea8c2f36fe-config\") pod \"59ec248d-84c7-4d23-b9f6-29ea8c2f36fe\" (UID: \"59ec248d-84c7-4d23-b9f6-29ea8c2f36fe\") " Dec 03 12:50:56 crc kubenswrapper[5052]: I1203 12:50:56.553918 5052 scope.go:117] "RemoveContainer" containerID="35ee8cb46ad46c891647ac8e4f3735c080422ea4c90dd9702c115515d3206886" Dec 03 12:50:56 crc kubenswrapper[5052]: E1203 12:50:56.555725 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"35ee8cb46ad46c891647ac8e4f3735c080422ea4c90dd9702c115515d3206886\": container with ID starting with 35ee8cb46ad46c891647ac8e4f3735c080422ea4c90dd9702c115515d3206886 not found: ID does not exist" containerID="35ee8cb46ad46c891647ac8e4f3735c080422ea4c90dd9702c115515d3206886" Dec 03 12:50:56 crc kubenswrapper[5052]: I1203 12:50:56.555765 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"35ee8cb46ad46c891647ac8e4f3735c080422ea4c90dd9702c115515d3206886"} err="failed to get container status \"35ee8cb46ad46c891647ac8e4f3735c080422ea4c90dd9702c115515d3206886\": rpc error: code = NotFound desc = could not find container \"35ee8cb46ad46c891647ac8e4f3735c080422ea4c90dd9702c115515d3206886\": container with ID starting with 35ee8cb46ad46c891647ac8e4f3735c080422ea4c90dd9702c115515d3206886 not found: ID does not exist" Dec 03 12:50:56 crc kubenswrapper[5052]: I1203 12:50:56.555791 5052 scope.go:117] "RemoveContainer" containerID="1e281e7872fa27702c42c19852a987a152988cec19cb89cd3614fa3d4e9a5e56" Dec 03 12:50:56 crc kubenswrapper[5052]: I1203 12:50:56.556139 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/59ec248d-84c7-4d23-b9f6-29ea8c2f36fe-kube-api-access-lj2nq" (OuterVolumeSpecName: "kube-api-access-lj2nq") pod "59ec248d-84c7-4d23-b9f6-29ea8c2f36fe" (UID: "59ec248d-84c7-4d23-b9f6-29ea8c2f36fe"). InnerVolumeSpecName "kube-api-access-lj2nq". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:50:56 crc kubenswrapper[5052]: E1203 12:50:56.556878 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1e281e7872fa27702c42c19852a987a152988cec19cb89cd3614fa3d4e9a5e56\": container with ID starting with 1e281e7872fa27702c42c19852a987a152988cec19cb89cd3614fa3d4e9a5e56 not found: ID does not exist" containerID="1e281e7872fa27702c42c19852a987a152988cec19cb89cd3614fa3d4e9a5e56" Dec 03 12:50:56 crc kubenswrapper[5052]: I1203 12:50:56.556903 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1e281e7872fa27702c42c19852a987a152988cec19cb89cd3614fa3d4e9a5e56"} err="failed to get container status \"1e281e7872fa27702c42c19852a987a152988cec19cb89cd3614fa3d4e9a5e56\": rpc error: code = NotFound desc = could not find container \"1e281e7872fa27702c42c19852a987a152988cec19cb89cd3614fa3d4e9a5e56\": container with ID starting with 1e281e7872fa27702c42c19852a987a152988cec19cb89cd3614fa3d4e9a5e56 not found: ID does not exist" Dec 03 12:50:56 crc kubenswrapper[5052]: I1203 12:50:56.600493 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/59ec248d-84c7-4d23-b9f6-29ea8c2f36fe-dnsmasq-svc" (OuterVolumeSpecName: "dnsmasq-svc") pod "59ec248d-84c7-4d23-b9f6-29ea8c2f36fe" (UID: "59ec248d-84c7-4d23-b9f6-29ea8c2f36fe"). InnerVolumeSpecName "dnsmasq-svc". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:50:56 crc kubenswrapper[5052]: I1203 12:50:56.605676 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/59ec248d-84c7-4d23-b9f6-29ea8c2f36fe-config" (OuterVolumeSpecName: "config") pod "59ec248d-84c7-4d23-b9f6-29ea8c2f36fe" (UID: "59ec248d-84c7-4d23-b9f6-29ea8c2f36fe"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:50:56 crc kubenswrapper[5052]: I1203 12:50:56.653830 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-lj2nq\" (UniqueName: \"kubernetes.io/projected/59ec248d-84c7-4d23-b9f6-29ea8c2f36fe-kube-api-access-lj2nq\") on node \"crc\" DevicePath \"\"" Dec 03 12:50:56 crc kubenswrapper[5052]: I1203 12:50:56.653860 5052 reconciler_common.go:293] "Volume detached for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/59ec248d-84c7-4d23-b9f6-29ea8c2f36fe-dnsmasq-svc\") on node \"crc\" DevicePath \"\"" Dec 03 12:50:56 crc kubenswrapper[5052]: I1203 12:50:56.653870 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/59ec248d-84c7-4d23-b9f6-29ea8c2f36fe-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:50:56 crc kubenswrapper[5052]: I1203 12:50:56.857089 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-9t4zp"] Dec 03 12:50:56 crc kubenswrapper[5052]: I1203 12:50:56.862646 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-9t4zp"] Dec 03 12:50:57 crc kubenswrapper[5052]: I1203 12:50:57.686339 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="59ec248d-84c7-4d23-b9f6-29ea8c2f36fe" path="/var/lib/kubelet/pods/59ec248d-84c7-4d23-b9f6-29ea8c2f36fe/volumes" Dec 03 12:50:58 crc kubenswrapper[5052]: I1203 12:50:58.713792 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:50:59 crc kubenswrapper[5052]: I1203 12:50:59.156760 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:50:59 crc kubenswrapper[5052]: I1203 12:50:59.379889 5052 patch_prober.go:28] interesting pod/machine-config-daemon-74bj6 container/machine-config-daemon namespace/openshift-machine-config-operator: Liveness probe status=failure output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" start-of-body= Dec 03 12:50:59 crc kubenswrapper[5052]: I1203 12:50:59.379960 5052 prober.go:107] "Probe failed" probeType="Liveness" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" containerName="machine-config-daemon" probeResult="failure" output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" Dec 03 12:51:00 crc kubenswrapper[5052]: I1203 12:51:00.626033 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/cinder-db-create-9vhcz"] Dec 03 12:51:00 crc kubenswrapper[5052]: E1203 12:51:00.626375 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="59ec248d-84c7-4d23-b9f6-29ea8c2f36fe" containerName="init" Dec 03 12:51:00 crc kubenswrapper[5052]: I1203 12:51:00.626387 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="59ec248d-84c7-4d23-b9f6-29ea8c2f36fe" containerName="init" Dec 03 12:51:00 crc kubenswrapper[5052]: E1203 12:51:00.626403 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="59ec248d-84c7-4d23-b9f6-29ea8c2f36fe" containerName="dnsmasq-dns" Dec 03 12:51:00 crc kubenswrapper[5052]: I1203 12:51:00.626410 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="59ec248d-84c7-4d23-b9f6-29ea8c2f36fe" containerName="dnsmasq-dns" Dec 03 12:51:00 crc kubenswrapper[5052]: I1203 12:51:00.626579 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="59ec248d-84c7-4d23-b9f6-29ea8c2f36fe" containerName="dnsmasq-dns" Dec 03 12:51:00 crc kubenswrapper[5052]: I1203 12:51:00.630874 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-db-create-9vhcz" Dec 03 12:51:00 crc kubenswrapper[5052]: I1203 12:51:00.646310 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/cinder-a2a9-account-create-update-2z8wq"] Dec 03 12:51:00 crc kubenswrapper[5052]: I1203 12:51:00.647747 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-a2a9-account-create-update-2z8wq" Dec 03 12:51:00 crc kubenswrapper[5052]: I1203 12:51:00.654794 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cinder-db-secret" Dec 03 12:51:00 crc kubenswrapper[5052]: I1203 12:51:00.659816 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-a2a9-account-create-update-2z8wq"] Dec 03 12:51:00 crc kubenswrapper[5052]: I1203 12:51:00.679055 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-db-create-9vhcz"] Dec 03 12:51:00 crc kubenswrapper[5052]: I1203 12:51:00.730437 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/6436478b-ce40-4335-930c-df0c19d4cb1a-operator-scripts\") pod \"cinder-db-create-9vhcz\" (UID: \"6436478b-ce40-4335-930c-df0c19d4cb1a\") " pod="openstack-kuttl-tests/cinder-db-create-9vhcz" Dec 03 12:51:00 crc kubenswrapper[5052]: I1203 12:51:00.730969 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mv8c6\" (UniqueName: \"kubernetes.io/projected/9b10d490-7528-42d2-83b7-d54974959b36-kube-api-access-mv8c6\") pod \"cinder-a2a9-account-create-update-2z8wq\" (UID: \"9b10d490-7528-42d2-83b7-d54974959b36\") " pod="openstack-kuttl-tests/cinder-a2a9-account-create-update-2z8wq" Dec 03 12:51:00 crc kubenswrapper[5052]: I1203 12:51:00.731014 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rjhk8\" (UniqueName: \"kubernetes.io/projected/6436478b-ce40-4335-930c-df0c19d4cb1a-kube-api-access-rjhk8\") pod \"cinder-db-create-9vhcz\" (UID: \"6436478b-ce40-4335-930c-df0c19d4cb1a\") " pod="openstack-kuttl-tests/cinder-db-create-9vhcz" Dec 03 12:51:00 crc kubenswrapper[5052]: I1203 12:51:00.731037 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/9b10d490-7528-42d2-83b7-d54974959b36-operator-scripts\") pod \"cinder-a2a9-account-create-update-2z8wq\" (UID: \"9b10d490-7528-42d2-83b7-d54974959b36\") " pod="openstack-kuttl-tests/cinder-a2a9-account-create-update-2z8wq" Dec 03 12:51:00 crc kubenswrapper[5052]: I1203 12:51:00.774562 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/barbican-6e62-account-create-update-5xsrz"] Dec 03 12:51:00 crc kubenswrapper[5052]: I1203 12:51:00.782762 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-6e62-account-create-update-5xsrz" Dec 03 12:51:00 crc kubenswrapper[5052]: I1203 12:51:00.789835 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"barbican-db-secret" Dec 03 12:51:00 crc kubenswrapper[5052]: I1203 12:51:00.833960 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-mv8c6\" (UniqueName: \"kubernetes.io/projected/9b10d490-7528-42d2-83b7-d54974959b36-kube-api-access-mv8c6\") pod \"cinder-a2a9-account-create-update-2z8wq\" (UID: \"9b10d490-7528-42d2-83b7-d54974959b36\") " pod="openstack-kuttl-tests/cinder-a2a9-account-create-update-2z8wq" Dec 03 12:51:00 crc kubenswrapper[5052]: I1203 12:51:00.834037 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-rjhk8\" (UniqueName: \"kubernetes.io/projected/6436478b-ce40-4335-930c-df0c19d4cb1a-kube-api-access-rjhk8\") pod \"cinder-db-create-9vhcz\" (UID: \"6436478b-ce40-4335-930c-df0c19d4cb1a\") " pod="openstack-kuttl-tests/cinder-db-create-9vhcz" Dec 03 12:51:00 crc kubenswrapper[5052]: I1203 12:51:00.834067 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/9b10d490-7528-42d2-83b7-d54974959b36-operator-scripts\") pod \"cinder-a2a9-account-create-update-2z8wq\" (UID: \"9b10d490-7528-42d2-83b7-d54974959b36\") " pod="openstack-kuttl-tests/cinder-a2a9-account-create-update-2z8wq" Dec 03 12:51:00 crc kubenswrapper[5052]: I1203 12:51:00.834149 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9ktrp\" (UniqueName: \"kubernetes.io/projected/8b9df814-7cdb-4696-9045-28d40e5b478c-kube-api-access-9ktrp\") pod \"barbican-6e62-account-create-update-5xsrz\" (UID: \"8b9df814-7cdb-4696-9045-28d40e5b478c\") " pod="openstack-kuttl-tests/barbican-6e62-account-create-update-5xsrz" Dec 03 12:51:00 crc kubenswrapper[5052]: I1203 12:51:00.834204 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/8b9df814-7cdb-4696-9045-28d40e5b478c-operator-scripts\") pod \"barbican-6e62-account-create-update-5xsrz\" (UID: \"8b9df814-7cdb-4696-9045-28d40e5b478c\") " pod="openstack-kuttl-tests/barbican-6e62-account-create-update-5xsrz" Dec 03 12:51:00 crc kubenswrapper[5052]: I1203 12:51:00.834251 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/6436478b-ce40-4335-930c-df0c19d4cb1a-operator-scripts\") pod \"cinder-db-create-9vhcz\" (UID: \"6436478b-ce40-4335-930c-df0c19d4cb1a\") " pod="openstack-kuttl-tests/cinder-db-create-9vhcz" Dec 03 12:51:00 crc kubenswrapper[5052]: I1203 12:51:00.835034 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-6e62-account-create-update-5xsrz"] Dec 03 12:51:00 crc kubenswrapper[5052]: I1203 12:51:00.835419 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/6436478b-ce40-4335-930c-df0c19d4cb1a-operator-scripts\") pod \"cinder-db-create-9vhcz\" (UID: \"6436478b-ce40-4335-930c-df0c19d4cb1a\") " pod="openstack-kuttl-tests/cinder-db-create-9vhcz" Dec 03 12:51:00 crc kubenswrapper[5052]: I1203 12:51:00.835645 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/9b10d490-7528-42d2-83b7-d54974959b36-operator-scripts\") pod \"cinder-a2a9-account-create-update-2z8wq\" (UID: \"9b10d490-7528-42d2-83b7-d54974959b36\") " pod="openstack-kuttl-tests/cinder-a2a9-account-create-update-2z8wq" Dec 03 12:51:00 crc kubenswrapper[5052]: I1203 12:51:00.869010 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-rjhk8\" (UniqueName: \"kubernetes.io/projected/6436478b-ce40-4335-930c-df0c19d4cb1a-kube-api-access-rjhk8\") pod \"cinder-db-create-9vhcz\" (UID: \"6436478b-ce40-4335-930c-df0c19d4cb1a\") " pod="openstack-kuttl-tests/cinder-db-create-9vhcz" Dec 03 12:51:00 crc kubenswrapper[5052]: I1203 12:51:00.876111 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-mv8c6\" (UniqueName: \"kubernetes.io/projected/9b10d490-7528-42d2-83b7-d54974959b36-kube-api-access-mv8c6\") pod \"cinder-a2a9-account-create-update-2z8wq\" (UID: \"9b10d490-7528-42d2-83b7-d54974959b36\") " pod="openstack-kuttl-tests/cinder-a2a9-account-create-update-2z8wq" Dec 03 12:51:00 crc kubenswrapper[5052]: I1203 12:51:00.888319 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/barbican-db-create-66nr2"] Dec 03 12:51:00 crc kubenswrapper[5052]: I1203 12:51:00.902837 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-db-create-66nr2"] Dec 03 12:51:00 crc kubenswrapper[5052]: I1203 12:51:00.903096 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-db-create-66nr2" Dec 03 12:51:00 crc kubenswrapper[5052]: I1203 12:51:00.935630 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-9ktrp\" (UniqueName: \"kubernetes.io/projected/8b9df814-7cdb-4696-9045-28d40e5b478c-kube-api-access-9ktrp\") pod \"barbican-6e62-account-create-update-5xsrz\" (UID: \"8b9df814-7cdb-4696-9045-28d40e5b478c\") " pod="openstack-kuttl-tests/barbican-6e62-account-create-update-5xsrz" Dec 03 12:51:00 crc kubenswrapper[5052]: I1203 12:51:00.935687 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/8b9df814-7cdb-4696-9045-28d40e5b478c-operator-scripts\") pod \"barbican-6e62-account-create-update-5xsrz\" (UID: \"8b9df814-7cdb-4696-9045-28d40e5b478c\") " pod="openstack-kuttl-tests/barbican-6e62-account-create-update-5xsrz" Dec 03 12:51:00 crc kubenswrapper[5052]: I1203 12:51:00.936643 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/8b9df814-7cdb-4696-9045-28d40e5b478c-operator-scripts\") pod \"barbican-6e62-account-create-update-5xsrz\" (UID: \"8b9df814-7cdb-4696-9045-28d40e5b478c\") " pod="openstack-kuttl-tests/barbican-6e62-account-create-update-5xsrz" Dec 03 12:51:00 crc kubenswrapper[5052]: I1203 12:51:00.963040 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-9ktrp\" (UniqueName: \"kubernetes.io/projected/8b9df814-7cdb-4696-9045-28d40e5b478c-kube-api-access-9ktrp\") pod \"barbican-6e62-account-create-update-5xsrz\" (UID: \"8b9df814-7cdb-4696-9045-28d40e5b478c\") " pod="openstack-kuttl-tests/barbican-6e62-account-create-update-5xsrz" Dec 03 12:51:00 crc kubenswrapper[5052]: I1203 12:51:00.986065 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/neutron-db-create-mn57r"] Dec 03 12:51:00 crc kubenswrapper[5052]: I1203 12:51:00.987427 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-db-create-mn57r" Dec 03 12:51:00 crc kubenswrapper[5052]: I1203 12:51:00.994561 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-db-create-9vhcz" Dec 03 12:51:01 crc kubenswrapper[5052]: I1203 12:51:01.018579 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-a2a9-account-create-update-2z8wq" Dec 03 12:51:01 crc kubenswrapper[5052]: I1203 12:51:01.029295 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/neutron-6c26-account-create-update-b4jfv"] Dec 03 12:51:01 crc kubenswrapper[5052]: I1203 12:51:01.030574 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-6c26-account-create-update-b4jfv" Dec 03 12:51:01 crc kubenswrapper[5052]: I1203 12:51:01.037315 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/69a643e8-8b6e-4fd4-b445-481364d71485-operator-scripts\") pod \"barbican-db-create-66nr2\" (UID: \"69a643e8-8b6e-4fd4-b445-481364d71485\") " pod="openstack-kuttl-tests/barbican-db-create-66nr2" Dec 03 12:51:01 crc kubenswrapper[5052]: I1203 12:51:01.037409 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6wp5l\" (UniqueName: \"kubernetes.io/projected/69a643e8-8b6e-4fd4-b445-481364d71485-kube-api-access-6wp5l\") pod \"barbican-db-create-66nr2\" (UID: \"69a643e8-8b6e-4fd4-b445-481364d71485\") " pod="openstack-kuttl-tests/barbican-db-create-66nr2" Dec 03 12:51:01 crc kubenswrapper[5052]: I1203 12:51:01.037461 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ss9xx\" (UniqueName: \"kubernetes.io/projected/38348d87-fbe3-4196-8819-8d90f2720b33-kube-api-access-ss9xx\") pod \"neutron-db-create-mn57r\" (UID: \"38348d87-fbe3-4196-8819-8d90f2720b33\") " pod="openstack-kuttl-tests/neutron-db-create-mn57r" Dec 03 12:51:01 crc kubenswrapper[5052]: I1203 12:51:01.037496 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/38348d87-fbe3-4196-8819-8d90f2720b33-operator-scripts\") pod \"neutron-db-create-mn57r\" (UID: \"38348d87-fbe3-4196-8819-8d90f2720b33\") " pod="openstack-kuttl-tests/neutron-db-create-mn57r" Dec 03 12:51:01 crc kubenswrapper[5052]: I1203 12:51:01.050421 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"neutron-db-secret" Dec 03 12:51:01 crc kubenswrapper[5052]: I1203 12:51:01.063378 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-db-create-mn57r"] Dec 03 12:51:01 crc kubenswrapper[5052]: I1203 12:51:01.094354 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-6c26-account-create-update-b4jfv"] Dec 03 12:51:01 crc kubenswrapper[5052]: I1203 12:51:01.106006 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-6e62-account-create-update-5xsrz" Dec 03 12:51:01 crc kubenswrapper[5052]: I1203 12:51:01.136867 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/keystone-db-sync-skdtp"] Dec 03 12:51:01 crc kubenswrapper[5052]: I1203 12:51:01.138430 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-db-sync-skdtp" Dec 03 12:51:01 crc kubenswrapper[5052]: I1203 12:51:01.139387 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-ss9xx\" (UniqueName: \"kubernetes.io/projected/38348d87-fbe3-4196-8819-8d90f2720b33-kube-api-access-ss9xx\") pod \"neutron-db-create-mn57r\" (UID: \"38348d87-fbe3-4196-8819-8d90f2720b33\") " pod="openstack-kuttl-tests/neutron-db-create-mn57r" Dec 03 12:51:01 crc kubenswrapper[5052]: I1203 12:51:01.139450 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/0a272fa7-1c64-49b1-81a0-b339a5cc686a-operator-scripts\") pod \"neutron-6c26-account-create-update-b4jfv\" (UID: \"0a272fa7-1c64-49b1-81a0-b339a5cc686a\") " pod="openstack-kuttl-tests/neutron-6c26-account-create-update-b4jfv" Dec 03 12:51:01 crc kubenswrapper[5052]: I1203 12:51:01.139491 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/38348d87-fbe3-4196-8819-8d90f2720b33-operator-scripts\") pod \"neutron-db-create-mn57r\" (UID: \"38348d87-fbe3-4196-8819-8d90f2720b33\") " pod="openstack-kuttl-tests/neutron-db-create-mn57r" Dec 03 12:51:01 crc kubenswrapper[5052]: I1203 12:51:01.139580 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/69a643e8-8b6e-4fd4-b445-481364d71485-operator-scripts\") pod \"barbican-db-create-66nr2\" (UID: \"69a643e8-8b6e-4fd4-b445-481364d71485\") " pod="openstack-kuttl-tests/barbican-db-create-66nr2" Dec 03 12:51:01 crc kubenswrapper[5052]: I1203 12:51:01.139631 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-txpl8\" (UniqueName: \"kubernetes.io/projected/0a272fa7-1c64-49b1-81a0-b339a5cc686a-kube-api-access-txpl8\") pod \"neutron-6c26-account-create-update-b4jfv\" (UID: \"0a272fa7-1c64-49b1-81a0-b339a5cc686a\") " pod="openstack-kuttl-tests/neutron-6c26-account-create-update-b4jfv" Dec 03 12:51:01 crc kubenswrapper[5052]: I1203 12:51:01.139681 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-6wp5l\" (UniqueName: \"kubernetes.io/projected/69a643e8-8b6e-4fd4-b445-481364d71485-kube-api-access-6wp5l\") pod \"barbican-db-create-66nr2\" (UID: \"69a643e8-8b6e-4fd4-b445-481364d71485\") " pod="openstack-kuttl-tests/barbican-db-create-66nr2" Dec 03 12:51:01 crc kubenswrapper[5052]: I1203 12:51:01.140412 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/38348d87-fbe3-4196-8819-8d90f2720b33-operator-scripts\") pod \"neutron-db-create-mn57r\" (UID: \"38348d87-fbe3-4196-8819-8d90f2720b33\") " pod="openstack-kuttl-tests/neutron-db-create-mn57r" Dec 03 12:51:01 crc kubenswrapper[5052]: I1203 12:51:01.140947 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/69a643e8-8b6e-4fd4-b445-481364d71485-operator-scripts\") pod \"barbican-db-create-66nr2\" (UID: \"69a643e8-8b6e-4fd4-b445-481364d71485\") " pod="openstack-kuttl-tests/barbican-db-create-66nr2" Dec 03 12:51:01 crc kubenswrapper[5052]: I1203 12:51:01.141109 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone-keystone-dockercfg-fr5sp" Dec 03 12:51:01 crc kubenswrapper[5052]: I1203 12:51:01.150631 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone" Dec 03 12:51:01 crc kubenswrapper[5052]: I1203 12:51:01.150921 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone-config-data" Dec 03 12:51:01 crc kubenswrapper[5052]: I1203 12:51:01.150967 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone-scripts" Dec 03 12:51:01 crc kubenswrapper[5052]: I1203 12:51:01.175612 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-6wp5l\" (UniqueName: \"kubernetes.io/projected/69a643e8-8b6e-4fd4-b445-481364d71485-kube-api-access-6wp5l\") pod \"barbican-db-create-66nr2\" (UID: \"69a643e8-8b6e-4fd4-b445-481364d71485\") " pod="openstack-kuttl-tests/barbican-db-create-66nr2" Dec 03 12:51:01 crc kubenswrapper[5052]: I1203 12:51:01.182387 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-db-sync-skdtp"] Dec 03 12:51:01 crc kubenswrapper[5052]: I1203 12:51:01.184560 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-ss9xx\" (UniqueName: \"kubernetes.io/projected/38348d87-fbe3-4196-8819-8d90f2720b33-kube-api-access-ss9xx\") pod \"neutron-db-create-mn57r\" (UID: \"38348d87-fbe3-4196-8819-8d90f2720b33\") " pod="openstack-kuttl-tests/neutron-db-create-mn57r" Dec 03 12:51:01 crc kubenswrapper[5052]: I1203 12:51:01.241485 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6e6cb4e4-e8b3-4e38-8c40-f4829e4348a9-config-data\") pod \"keystone-db-sync-skdtp\" (UID: \"6e6cb4e4-e8b3-4e38-8c40-f4829e4348a9\") " pod="openstack-kuttl-tests/keystone-db-sync-skdtp" Dec 03 12:51:01 crc kubenswrapper[5052]: I1203 12:51:01.241862 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6e6cb4e4-e8b3-4e38-8c40-f4829e4348a9-combined-ca-bundle\") pod \"keystone-db-sync-skdtp\" (UID: \"6e6cb4e4-e8b3-4e38-8c40-f4829e4348a9\") " pod="openstack-kuttl-tests/keystone-db-sync-skdtp" Dec 03 12:51:01 crc kubenswrapper[5052]: I1203 12:51:01.241966 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/0a272fa7-1c64-49b1-81a0-b339a5cc686a-operator-scripts\") pod \"neutron-6c26-account-create-update-b4jfv\" (UID: \"0a272fa7-1c64-49b1-81a0-b339a5cc686a\") " pod="openstack-kuttl-tests/neutron-6c26-account-create-update-b4jfv" Dec 03 12:51:01 crc kubenswrapper[5052]: I1203 12:51:01.242080 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dj92f\" (UniqueName: \"kubernetes.io/projected/6e6cb4e4-e8b3-4e38-8c40-f4829e4348a9-kube-api-access-dj92f\") pod \"keystone-db-sync-skdtp\" (UID: \"6e6cb4e4-e8b3-4e38-8c40-f4829e4348a9\") " pod="openstack-kuttl-tests/keystone-db-sync-skdtp" Dec 03 12:51:01 crc kubenswrapper[5052]: I1203 12:51:01.242243 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-txpl8\" (UniqueName: \"kubernetes.io/projected/0a272fa7-1c64-49b1-81a0-b339a5cc686a-kube-api-access-txpl8\") pod \"neutron-6c26-account-create-update-b4jfv\" (UID: \"0a272fa7-1c64-49b1-81a0-b339a5cc686a\") " pod="openstack-kuttl-tests/neutron-6c26-account-create-update-b4jfv" Dec 03 12:51:01 crc kubenswrapper[5052]: I1203 12:51:01.242588 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/0a272fa7-1c64-49b1-81a0-b339a5cc686a-operator-scripts\") pod \"neutron-6c26-account-create-update-b4jfv\" (UID: \"0a272fa7-1c64-49b1-81a0-b339a5cc686a\") " pod="openstack-kuttl-tests/neutron-6c26-account-create-update-b4jfv" Dec 03 12:51:01 crc kubenswrapper[5052]: I1203 12:51:01.260151 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-db-create-66nr2" Dec 03 12:51:01 crc kubenswrapper[5052]: I1203 12:51:01.263179 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-txpl8\" (UniqueName: \"kubernetes.io/projected/0a272fa7-1c64-49b1-81a0-b339a5cc686a-kube-api-access-txpl8\") pod \"neutron-6c26-account-create-update-b4jfv\" (UID: \"0a272fa7-1c64-49b1-81a0-b339a5cc686a\") " pod="openstack-kuttl-tests/neutron-6c26-account-create-update-b4jfv" Dec 03 12:51:01 crc kubenswrapper[5052]: I1203 12:51:01.336840 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-db-create-mn57r" Dec 03 12:51:01 crc kubenswrapper[5052]: I1203 12:51:01.344284 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6e6cb4e4-e8b3-4e38-8c40-f4829e4348a9-combined-ca-bundle\") pod \"keystone-db-sync-skdtp\" (UID: \"6e6cb4e4-e8b3-4e38-8c40-f4829e4348a9\") " pod="openstack-kuttl-tests/keystone-db-sync-skdtp" Dec 03 12:51:01 crc kubenswrapper[5052]: I1203 12:51:01.344363 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-dj92f\" (UniqueName: \"kubernetes.io/projected/6e6cb4e4-e8b3-4e38-8c40-f4829e4348a9-kube-api-access-dj92f\") pod \"keystone-db-sync-skdtp\" (UID: \"6e6cb4e4-e8b3-4e38-8c40-f4829e4348a9\") " pod="openstack-kuttl-tests/keystone-db-sync-skdtp" Dec 03 12:51:01 crc kubenswrapper[5052]: I1203 12:51:01.344434 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6e6cb4e4-e8b3-4e38-8c40-f4829e4348a9-config-data\") pod \"keystone-db-sync-skdtp\" (UID: \"6e6cb4e4-e8b3-4e38-8c40-f4829e4348a9\") " pod="openstack-kuttl-tests/keystone-db-sync-skdtp" Dec 03 12:51:01 crc kubenswrapper[5052]: I1203 12:51:01.351381 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6e6cb4e4-e8b3-4e38-8c40-f4829e4348a9-combined-ca-bundle\") pod \"keystone-db-sync-skdtp\" (UID: \"6e6cb4e4-e8b3-4e38-8c40-f4829e4348a9\") " pod="openstack-kuttl-tests/keystone-db-sync-skdtp" Dec 03 12:51:01 crc kubenswrapper[5052]: I1203 12:51:01.352228 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6e6cb4e4-e8b3-4e38-8c40-f4829e4348a9-config-data\") pod \"keystone-db-sync-skdtp\" (UID: \"6e6cb4e4-e8b3-4e38-8c40-f4829e4348a9\") " pod="openstack-kuttl-tests/keystone-db-sync-skdtp" Dec 03 12:51:01 crc kubenswrapper[5052]: I1203 12:51:01.361685 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-dj92f\" (UniqueName: \"kubernetes.io/projected/6e6cb4e4-e8b3-4e38-8c40-f4829e4348a9-kube-api-access-dj92f\") pod \"keystone-db-sync-skdtp\" (UID: \"6e6cb4e4-e8b3-4e38-8c40-f4829e4348a9\") " pod="openstack-kuttl-tests/keystone-db-sync-skdtp" Dec 03 12:51:01 crc kubenswrapper[5052]: I1203 12:51:01.423063 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-6c26-account-create-update-b4jfv" Dec 03 12:51:01 crc kubenswrapper[5052]: I1203 12:51:01.423981 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-db-create-9vhcz"] Dec 03 12:51:01 crc kubenswrapper[5052]: W1203 12:51:01.445645 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6436478b_ce40_4335_930c_df0c19d4cb1a.slice/crio-fcb8c989032beca63ff17b25e36c67dc7c5a631deab199b322b06140bee88633 WatchSource:0}: Error finding container fcb8c989032beca63ff17b25e36c67dc7c5a631deab199b322b06140bee88633: Status 404 returned error can't find the container with id fcb8c989032beca63ff17b25e36c67dc7c5a631deab199b322b06140bee88633 Dec 03 12:51:01 crc kubenswrapper[5052]: I1203 12:51:01.489358 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-db-sync-skdtp" Dec 03 12:51:01 crc kubenswrapper[5052]: I1203 12:51:01.568338 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-a2a9-account-create-update-2z8wq"] Dec 03 12:51:01 crc kubenswrapper[5052]: I1203 12:51:01.619726 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-db-create-9vhcz" event={"ID":"6436478b-ce40-4335-930c-df0c19d4cb1a","Type":"ContainerStarted","Data":"fcb8c989032beca63ff17b25e36c67dc7c5a631deab199b322b06140bee88633"} Dec 03 12:51:01 crc kubenswrapper[5052]: I1203 12:51:01.712906 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-6e62-account-create-update-5xsrz"] Dec 03 12:51:01 crc kubenswrapper[5052]: W1203 12:51:01.733986 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8b9df814_7cdb_4696_9045_28d40e5b478c.slice/crio-b5e68dd5f1c0992a2ebe01e13e59cbe26715ac7adf66cd7bfac7e76315cae6c4 WatchSource:0}: Error finding container b5e68dd5f1c0992a2ebe01e13e59cbe26715ac7adf66cd7bfac7e76315cae6c4: Status 404 returned error can't find the container with id b5e68dd5f1c0992a2ebe01e13e59cbe26715ac7adf66cd7bfac7e76315cae6c4 Dec 03 12:51:01 crc kubenswrapper[5052]: I1203 12:51:01.817136 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-db-create-66nr2"] Dec 03 12:51:01 crc kubenswrapper[5052]: W1203 12:51:01.829710 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod69a643e8_8b6e_4fd4_b445_481364d71485.slice/crio-4bb2be3fff9ebeea805525edfb3894f3350c1b9ff5f576086d741bc52c7eb29c WatchSource:0}: Error finding container 4bb2be3fff9ebeea805525edfb3894f3350c1b9ff5f576086d741bc52c7eb29c: Status 404 returned error can't find the container with id 4bb2be3fff9ebeea805525edfb3894f3350c1b9ff5f576086d741bc52c7eb29c Dec 03 12:51:01 crc kubenswrapper[5052]: I1203 12:51:01.939387 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-db-create-mn57r"] Dec 03 12:51:01 crc kubenswrapper[5052]: W1203 12:51:01.952605 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod38348d87_fbe3_4196_8819_8d90f2720b33.slice/crio-ffe5f42be5d88ee969b8f7c6d9415efef77c061d1c13b4f7487405cabcf17d30 WatchSource:0}: Error finding container ffe5f42be5d88ee969b8f7c6d9415efef77c061d1c13b4f7487405cabcf17d30: Status 404 returned error can't find the container with id ffe5f42be5d88ee969b8f7c6d9415efef77c061d1c13b4f7487405cabcf17d30 Dec 03 12:51:02 crc kubenswrapper[5052]: I1203 12:51:02.058580 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-6c26-account-create-update-b4jfv"] Dec 03 12:51:02 crc kubenswrapper[5052]: I1203 12:51:02.122224 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-db-sync-skdtp"] Dec 03 12:51:02 crc kubenswrapper[5052]: W1203 12:51:02.122743 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0a272fa7_1c64_49b1_81a0_b339a5cc686a.slice/crio-baf4cd1356b72328b4446324695812fb0a2910d32dea0da3d58820c9f22a2854 WatchSource:0}: Error finding container baf4cd1356b72328b4446324695812fb0a2910d32dea0da3d58820c9f22a2854: Status 404 returned error can't find the container with id baf4cd1356b72328b4446324695812fb0a2910d32dea0da3d58820c9f22a2854 Dec 03 12:51:02 crc kubenswrapper[5052]: W1203 12:51:02.124168 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6e6cb4e4_e8b3_4e38_8c40_f4829e4348a9.slice/crio-412c4de49318470be85d855599948b08cfdbdb14b979a54c973e4264cbe52c9d WatchSource:0}: Error finding container 412c4de49318470be85d855599948b08cfdbdb14b979a54c973e4264cbe52c9d: Status 404 returned error can't find the container with id 412c4de49318470be85d855599948b08cfdbdb14b979a54c973e4264cbe52c9d Dec 03 12:51:02 crc kubenswrapper[5052]: I1203 12:51:02.637690 5052 generic.go:334] "Generic (PLEG): container finished" podID="0a272fa7-1c64-49b1-81a0-b339a5cc686a" containerID="906c443738f38d4f8d4282c9fd33ffbebb16c2aec1b556f25ccc7a7e8c6e748c" exitCode=0 Dec 03 12:51:02 crc kubenswrapper[5052]: I1203 12:51:02.638048 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-6c26-account-create-update-b4jfv" event={"ID":"0a272fa7-1c64-49b1-81a0-b339a5cc686a","Type":"ContainerDied","Data":"906c443738f38d4f8d4282c9fd33ffbebb16c2aec1b556f25ccc7a7e8c6e748c"} Dec 03 12:51:02 crc kubenswrapper[5052]: I1203 12:51:02.638082 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-6c26-account-create-update-b4jfv" event={"ID":"0a272fa7-1c64-49b1-81a0-b339a5cc686a","Type":"ContainerStarted","Data":"baf4cd1356b72328b4446324695812fb0a2910d32dea0da3d58820c9f22a2854"} Dec 03 12:51:02 crc kubenswrapper[5052]: I1203 12:51:02.643262 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-db-sync-skdtp" event={"ID":"6e6cb4e4-e8b3-4e38-8c40-f4829e4348a9","Type":"ContainerStarted","Data":"b4f45322078c29bf05e1e6c5ce39ffbdb6524a51a8a145e7edb2b7b7325e566f"} Dec 03 12:51:02 crc kubenswrapper[5052]: I1203 12:51:02.643311 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-db-sync-skdtp" event={"ID":"6e6cb4e4-e8b3-4e38-8c40-f4829e4348a9","Type":"ContainerStarted","Data":"412c4de49318470be85d855599948b08cfdbdb14b979a54c973e4264cbe52c9d"} Dec 03 12:51:02 crc kubenswrapper[5052]: I1203 12:51:02.646136 5052 generic.go:334] "Generic (PLEG): container finished" podID="38348d87-fbe3-4196-8819-8d90f2720b33" containerID="4b943db92344d3538d476fedbfe7197f925dbe4641448a2e28f5c8d79d9ed4b5" exitCode=0 Dec 03 12:51:02 crc kubenswrapper[5052]: I1203 12:51:02.646186 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-db-create-mn57r" event={"ID":"38348d87-fbe3-4196-8819-8d90f2720b33","Type":"ContainerDied","Data":"4b943db92344d3538d476fedbfe7197f925dbe4641448a2e28f5c8d79d9ed4b5"} Dec 03 12:51:02 crc kubenswrapper[5052]: I1203 12:51:02.646206 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-db-create-mn57r" event={"ID":"38348d87-fbe3-4196-8819-8d90f2720b33","Type":"ContainerStarted","Data":"ffe5f42be5d88ee969b8f7c6d9415efef77c061d1c13b4f7487405cabcf17d30"} Dec 03 12:51:02 crc kubenswrapper[5052]: I1203 12:51:02.647988 5052 generic.go:334] "Generic (PLEG): container finished" podID="8b9df814-7cdb-4696-9045-28d40e5b478c" containerID="405a9b22b01a4f49fa4335e876211ec12001bfaed5b30631c5e1d909cd57aa18" exitCode=0 Dec 03 12:51:02 crc kubenswrapper[5052]: I1203 12:51:02.648045 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-6e62-account-create-update-5xsrz" event={"ID":"8b9df814-7cdb-4696-9045-28d40e5b478c","Type":"ContainerDied","Data":"405a9b22b01a4f49fa4335e876211ec12001bfaed5b30631c5e1d909cd57aa18"} Dec 03 12:51:02 crc kubenswrapper[5052]: I1203 12:51:02.648087 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-6e62-account-create-update-5xsrz" event={"ID":"8b9df814-7cdb-4696-9045-28d40e5b478c","Type":"ContainerStarted","Data":"b5e68dd5f1c0992a2ebe01e13e59cbe26715ac7adf66cd7bfac7e76315cae6c4"} Dec 03 12:51:02 crc kubenswrapper[5052]: I1203 12:51:02.652043 5052 generic.go:334] "Generic (PLEG): container finished" podID="69a643e8-8b6e-4fd4-b445-481364d71485" containerID="327f0066a0601f7b4c70c175a1a3262b3ce3a6c6fc204b7124a0b69083969d6f" exitCode=0 Dec 03 12:51:02 crc kubenswrapper[5052]: I1203 12:51:02.652109 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-db-create-66nr2" event={"ID":"69a643e8-8b6e-4fd4-b445-481364d71485","Type":"ContainerDied","Data":"327f0066a0601f7b4c70c175a1a3262b3ce3a6c6fc204b7124a0b69083969d6f"} Dec 03 12:51:02 crc kubenswrapper[5052]: I1203 12:51:02.652142 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-db-create-66nr2" event={"ID":"69a643e8-8b6e-4fd4-b445-481364d71485","Type":"ContainerStarted","Data":"4bb2be3fff9ebeea805525edfb3894f3350c1b9ff5f576086d741bc52c7eb29c"} Dec 03 12:51:02 crc kubenswrapper[5052]: I1203 12:51:02.657726 5052 generic.go:334] "Generic (PLEG): container finished" podID="9b10d490-7528-42d2-83b7-d54974959b36" containerID="9a5124afa6da140d0ad66a43ce6d9f495f3b80943f462de57cae97790359ac58" exitCode=0 Dec 03 12:51:02 crc kubenswrapper[5052]: I1203 12:51:02.657764 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-a2a9-account-create-update-2z8wq" event={"ID":"9b10d490-7528-42d2-83b7-d54974959b36","Type":"ContainerDied","Data":"9a5124afa6da140d0ad66a43ce6d9f495f3b80943f462de57cae97790359ac58"} Dec 03 12:51:02 crc kubenswrapper[5052]: I1203 12:51:02.657809 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-a2a9-account-create-update-2z8wq" event={"ID":"9b10d490-7528-42d2-83b7-d54974959b36","Type":"ContainerStarted","Data":"86a7818499f167a2a3ec2765dbcfd94d241d7e57151d4df7cf0676f2b4444428"} Dec 03 12:51:02 crc kubenswrapper[5052]: I1203 12:51:02.659128 5052 generic.go:334] "Generic (PLEG): container finished" podID="6436478b-ce40-4335-930c-df0c19d4cb1a" containerID="3049ef615fa201f9bcee00bec5b96cdf452b84cc54465cb8739a8fe34c22f4f6" exitCode=0 Dec 03 12:51:02 crc kubenswrapper[5052]: I1203 12:51:02.659164 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-db-create-9vhcz" event={"ID":"6436478b-ce40-4335-930c-df0c19d4cb1a","Type":"ContainerDied","Data":"3049ef615fa201f9bcee00bec5b96cdf452b84cc54465cb8739a8fe34c22f4f6"} Dec 03 12:51:02 crc kubenswrapper[5052]: I1203 12:51:02.719123 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/keystone-db-sync-skdtp" podStartSLOduration=1.719085317 podStartE2EDuration="1.719085317s" podCreationTimestamp="2025-12-03 12:51:01 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:51:02.707882566 +0000 UTC m=+5453.296124340" watchObservedRunningTime="2025-12-03 12:51:02.719085317 +0000 UTC m=+5453.307327081" Dec 03 12:51:02 crc kubenswrapper[5052]: E1203 12:51:02.873229 5052 cadvisor_stats_provider.go:516] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod38348d87_fbe3_4196_8819_8d90f2720b33.slice/crio-conmon-4b943db92344d3538d476fedbfe7197f925dbe4641448a2e28f5c8d79d9ed4b5.scope\": RecentStats: unable to find data in memory cache]" Dec 03 12:51:04 crc kubenswrapper[5052]: I1203 12:51:04.043101 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-db-create-9vhcz" Dec 03 12:51:04 crc kubenswrapper[5052]: I1203 12:51:04.090996 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rjhk8\" (UniqueName: \"kubernetes.io/projected/6436478b-ce40-4335-930c-df0c19d4cb1a-kube-api-access-rjhk8\") pod \"6436478b-ce40-4335-930c-df0c19d4cb1a\" (UID: \"6436478b-ce40-4335-930c-df0c19d4cb1a\") " Dec 03 12:51:04 crc kubenswrapper[5052]: I1203 12:51:04.091068 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/6436478b-ce40-4335-930c-df0c19d4cb1a-operator-scripts\") pod \"6436478b-ce40-4335-930c-df0c19d4cb1a\" (UID: \"6436478b-ce40-4335-930c-df0c19d4cb1a\") " Dec 03 12:51:04 crc kubenswrapper[5052]: I1203 12:51:04.092151 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6436478b-ce40-4335-930c-df0c19d4cb1a-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "6436478b-ce40-4335-930c-df0c19d4cb1a" (UID: "6436478b-ce40-4335-930c-df0c19d4cb1a"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:51:04 crc kubenswrapper[5052]: I1203 12:51:04.114207 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6436478b-ce40-4335-930c-df0c19d4cb1a-kube-api-access-rjhk8" (OuterVolumeSpecName: "kube-api-access-rjhk8") pod "6436478b-ce40-4335-930c-df0c19d4cb1a" (UID: "6436478b-ce40-4335-930c-df0c19d4cb1a"). InnerVolumeSpecName "kube-api-access-rjhk8". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:51:04 crc kubenswrapper[5052]: I1203 12:51:04.201715 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/6436478b-ce40-4335-930c-df0c19d4cb1a-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:51:04 crc kubenswrapper[5052]: I1203 12:51:04.201755 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-rjhk8\" (UniqueName: \"kubernetes.io/projected/6436478b-ce40-4335-930c-df0c19d4cb1a-kube-api-access-rjhk8\") on node \"crc\" DevicePath \"\"" Dec 03 12:51:04 crc kubenswrapper[5052]: I1203 12:51:04.249312 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-db-create-66nr2" Dec 03 12:51:04 crc kubenswrapper[5052]: I1203 12:51:04.253913 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-db-create-mn57r" Dec 03 12:51:04 crc kubenswrapper[5052]: I1203 12:51:04.261757 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-6c26-account-create-update-b4jfv" Dec 03 12:51:04 crc kubenswrapper[5052]: I1203 12:51:04.282452 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-a2a9-account-create-update-2z8wq" Dec 03 12:51:04 crc kubenswrapper[5052]: I1203 12:51:04.289432 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-6e62-account-create-update-5xsrz" Dec 03 12:51:04 crc kubenswrapper[5052]: I1203 12:51:04.407735 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9ktrp\" (UniqueName: \"kubernetes.io/projected/8b9df814-7cdb-4696-9045-28d40e5b478c-kube-api-access-9ktrp\") pod \"8b9df814-7cdb-4696-9045-28d40e5b478c\" (UID: \"8b9df814-7cdb-4696-9045-28d40e5b478c\") " Dec 03 12:51:04 crc kubenswrapper[5052]: I1203 12:51:04.408435 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mv8c6\" (UniqueName: \"kubernetes.io/projected/9b10d490-7528-42d2-83b7-d54974959b36-kube-api-access-mv8c6\") pod \"9b10d490-7528-42d2-83b7-d54974959b36\" (UID: \"9b10d490-7528-42d2-83b7-d54974959b36\") " Dec 03 12:51:04 crc kubenswrapper[5052]: I1203 12:51:04.408693 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ss9xx\" (UniqueName: \"kubernetes.io/projected/38348d87-fbe3-4196-8819-8d90f2720b33-kube-api-access-ss9xx\") pod \"38348d87-fbe3-4196-8819-8d90f2720b33\" (UID: \"38348d87-fbe3-4196-8819-8d90f2720b33\") " Dec 03 12:51:04 crc kubenswrapper[5052]: I1203 12:51:04.408842 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/9b10d490-7528-42d2-83b7-d54974959b36-operator-scripts\") pod \"9b10d490-7528-42d2-83b7-d54974959b36\" (UID: \"9b10d490-7528-42d2-83b7-d54974959b36\") " Dec 03 12:51:04 crc kubenswrapper[5052]: I1203 12:51:04.408889 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/8b9df814-7cdb-4696-9045-28d40e5b478c-operator-scripts\") pod \"8b9df814-7cdb-4696-9045-28d40e5b478c\" (UID: \"8b9df814-7cdb-4696-9045-28d40e5b478c\") " Dec 03 12:51:04 crc kubenswrapper[5052]: I1203 12:51:04.408969 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6wp5l\" (UniqueName: \"kubernetes.io/projected/69a643e8-8b6e-4fd4-b445-481364d71485-kube-api-access-6wp5l\") pod \"69a643e8-8b6e-4fd4-b445-481364d71485\" (UID: \"69a643e8-8b6e-4fd4-b445-481364d71485\") " Dec 03 12:51:04 crc kubenswrapper[5052]: I1203 12:51:04.409043 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-txpl8\" (UniqueName: \"kubernetes.io/projected/0a272fa7-1c64-49b1-81a0-b339a5cc686a-kube-api-access-txpl8\") pod \"0a272fa7-1c64-49b1-81a0-b339a5cc686a\" (UID: \"0a272fa7-1c64-49b1-81a0-b339a5cc686a\") " Dec 03 12:51:04 crc kubenswrapper[5052]: I1203 12:51:04.409100 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/38348d87-fbe3-4196-8819-8d90f2720b33-operator-scripts\") pod \"38348d87-fbe3-4196-8819-8d90f2720b33\" (UID: \"38348d87-fbe3-4196-8819-8d90f2720b33\") " Dec 03 12:51:04 crc kubenswrapper[5052]: I1203 12:51:04.409154 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/0a272fa7-1c64-49b1-81a0-b339a5cc686a-operator-scripts\") pod \"0a272fa7-1c64-49b1-81a0-b339a5cc686a\" (UID: \"0a272fa7-1c64-49b1-81a0-b339a5cc686a\") " Dec 03 12:51:04 crc kubenswrapper[5052]: I1203 12:51:04.409199 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/69a643e8-8b6e-4fd4-b445-481364d71485-operator-scripts\") pod \"69a643e8-8b6e-4fd4-b445-481364d71485\" (UID: \"69a643e8-8b6e-4fd4-b445-481364d71485\") " Dec 03 12:51:04 crc kubenswrapper[5052]: I1203 12:51:04.409389 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8b9df814-7cdb-4696-9045-28d40e5b478c-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "8b9df814-7cdb-4696-9045-28d40e5b478c" (UID: "8b9df814-7cdb-4696-9045-28d40e5b478c"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:51:04 crc kubenswrapper[5052]: I1203 12:51:04.409836 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/8b9df814-7cdb-4696-9045-28d40e5b478c-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:51:04 crc kubenswrapper[5052]: I1203 12:51:04.409884 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9b10d490-7528-42d2-83b7-d54974959b36-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "9b10d490-7528-42d2-83b7-d54974959b36" (UID: "9b10d490-7528-42d2-83b7-d54974959b36"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:51:04 crc kubenswrapper[5052]: I1203 12:51:04.409948 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0a272fa7-1c64-49b1-81a0-b339a5cc686a-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "0a272fa7-1c64-49b1-81a0-b339a5cc686a" (UID: "0a272fa7-1c64-49b1-81a0-b339a5cc686a"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:51:04 crc kubenswrapper[5052]: I1203 12:51:04.409963 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/38348d87-fbe3-4196-8819-8d90f2720b33-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "38348d87-fbe3-4196-8819-8d90f2720b33" (UID: "38348d87-fbe3-4196-8819-8d90f2720b33"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:51:04 crc kubenswrapper[5052]: I1203 12:51:04.410066 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/69a643e8-8b6e-4fd4-b445-481364d71485-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "69a643e8-8b6e-4fd4-b445-481364d71485" (UID: "69a643e8-8b6e-4fd4-b445-481364d71485"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:51:04 crc kubenswrapper[5052]: I1203 12:51:04.412007 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0a272fa7-1c64-49b1-81a0-b339a5cc686a-kube-api-access-txpl8" (OuterVolumeSpecName: "kube-api-access-txpl8") pod "0a272fa7-1c64-49b1-81a0-b339a5cc686a" (UID: "0a272fa7-1c64-49b1-81a0-b339a5cc686a"). InnerVolumeSpecName "kube-api-access-txpl8". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:51:04 crc kubenswrapper[5052]: I1203 12:51:04.412490 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8b9df814-7cdb-4696-9045-28d40e5b478c-kube-api-access-9ktrp" (OuterVolumeSpecName: "kube-api-access-9ktrp") pod "8b9df814-7cdb-4696-9045-28d40e5b478c" (UID: "8b9df814-7cdb-4696-9045-28d40e5b478c"). InnerVolumeSpecName "kube-api-access-9ktrp". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:51:04 crc kubenswrapper[5052]: I1203 12:51:04.413078 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/69a643e8-8b6e-4fd4-b445-481364d71485-kube-api-access-6wp5l" (OuterVolumeSpecName: "kube-api-access-6wp5l") pod "69a643e8-8b6e-4fd4-b445-481364d71485" (UID: "69a643e8-8b6e-4fd4-b445-481364d71485"). InnerVolumeSpecName "kube-api-access-6wp5l". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:51:04 crc kubenswrapper[5052]: I1203 12:51:04.413229 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9b10d490-7528-42d2-83b7-d54974959b36-kube-api-access-mv8c6" (OuterVolumeSpecName: "kube-api-access-mv8c6") pod "9b10d490-7528-42d2-83b7-d54974959b36" (UID: "9b10d490-7528-42d2-83b7-d54974959b36"). InnerVolumeSpecName "kube-api-access-mv8c6". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:51:04 crc kubenswrapper[5052]: I1203 12:51:04.414260 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/38348d87-fbe3-4196-8819-8d90f2720b33-kube-api-access-ss9xx" (OuterVolumeSpecName: "kube-api-access-ss9xx") pod "38348d87-fbe3-4196-8819-8d90f2720b33" (UID: "38348d87-fbe3-4196-8819-8d90f2720b33"). InnerVolumeSpecName "kube-api-access-ss9xx". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:51:04 crc kubenswrapper[5052]: I1203 12:51:04.511022 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-9ktrp\" (UniqueName: \"kubernetes.io/projected/8b9df814-7cdb-4696-9045-28d40e5b478c-kube-api-access-9ktrp\") on node \"crc\" DevicePath \"\"" Dec 03 12:51:04 crc kubenswrapper[5052]: I1203 12:51:04.511060 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-mv8c6\" (UniqueName: \"kubernetes.io/projected/9b10d490-7528-42d2-83b7-d54974959b36-kube-api-access-mv8c6\") on node \"crc\" DevicePath \"\"" Dec 03 12:51:04 crc kubenswrapper[5052]: I1203 12:51:04.511072 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-ss9xx\" (UniqueName: \"kubernetes.io/projected/38348d87-fbe3-4196-8819-8d90f2720b33-kube-api-access-ss9xx\") on node \"crc\" DevicePath \"\"" Dec 03 12:51:04 crc kubenswrapper[5052]: I1203 12:51:04.511086 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/9b10d490-7528-42d2-83b7-d54974959b36-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:51:04 crc kubenswrapper[5052]: I1203 12:51:04.511098 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-6wp5l\" (UniqueName: \"kubernetes.io/projected/69a643e8-8b6e-4fd4-b445-481364d71485-kube-api-access-6wp5l\") on node \"crc\" DevicePath \"\"" Dec 03 12:51:04 crc kubenswrapper[5052]: I1203 12:51:04.511110 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-txpl8\" (UniqueName: \"kubernetes.io/projected/0a272fa7-1c64-49b1-81a0-b339a5cc686a-kube-api-access-txpl8\") on node \"crc\" DevicePath \"\"" Dec 03 12:51:04 crc kubenswrapper[5052]: I1203 12:51:04.511121 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/38348d87-fbe3-4196-8819-8d90f2720b33-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:51:04 crc kubenswrapper[5052]: I1203 12:51:04.511132 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/0a272fa7-1c64-49b1-81a0-b339a5cc686a-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:51:04 crc kubenswrapper[5052]: I1203 12:51:04.511143 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/69a643e8-8b6e-4fd4-b445-481364d71485-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:51:04 crc kubenswrapper[5052]: I1203 12:51:04.676056 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-6c26-account-create-update-b4jfv" Dec 03 12:51:04 crc kubenswrapper[5052]: I1203 12:51:04.676057 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-6c26-account-create-update-b4jfv" event={"ID":"0a272fa7-1c64-49b1-81a0-b339a5cc686a","Type":"ContainerDied","Data":"baf4cd1356b72328b4446324695812fb0a2910d32dea0da3d58820c9f22a2854"} Dec 03 12:51:04 crc kubenswrapper[5052]: I1203 12:51:04.676199 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="baf4cd1356b72328b4446324695812fb0a2910d32dea0da3d58820c9f22a2854" Dec 03 12:51:04 crc kubenswrapper[5052]: I1203 12:51:04.677491 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-db-create-mn57r" event={"ID":"38348d87-fbe3-4196-8819-8d90f2720b33","Type":"ContainerDied","Data":"ffe5f42be5d88ee969b8f7c6d9415efef77c061d1c13b4f7487405cabcf17d30"} Dec 03 12:51:04 crc kubenswrapper[5052]: I1203 12:51:04.677584 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ffe5f42be5d88ee969b8f7c6d9415efef77c061d1c13b4f7487405cabcf17d30" Dec 03 12:51:04 crc kubenswrapper[5052]: I1203 12:51:04.677496 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-db-create-mn57r" Dec 03 12:51:04 crc kubenswrapper[5052]: I1203 12:51:04.679531 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-6e62-account-create-update-5xsrz" event={"ID":"8b9df814-7cdb-4696-9045-28d40e5b478c","Type":"ContainerDied","Data":"b5e68dd5f1c0992a2ebe01e13e59cbe26715ac7adf66cd7bfac7e76315cae6c4"} Dec 03 12:51:04 crc kubenswrapper[5052]: I1203 12:51:04.679554 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-6e62-account-create-update-5xsrz" Dec 03 12:51:04 crc kubenswrapper[5052]: I1203 12:51:04.679557 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b5e68dd5f1c0992a2ebe01e13e59cbe26715ac7adf66cd7bfac7e76315cae6c4" Dec 03 12:51:04 crc kubenswrapper[5052]: I1203 12:51:04.681471 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-db-create-66nr2" event={"ID":"69a643e8-8b6e-4fd4-b445-481364d71485","Type":"ContainerDied","Data":"4bb2be3fff9ebeea805525edfb3894f3350c1b9ff5f576086d741bc52c7eb29c"} Dec 03 12:51:04 crc kubenswrapper[5052]: I1203 12:51:04.681500 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-db-create-66nr2" Dec 03 12:51:04 crc kubenswrapper[5052]: I1203 12:51:04.681501 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4bb2be3fff9ebeea805525edfb3894f3350c1b9ff5f576086d741bc52c7eb29c" Dec 03 12:51:04 crc kubenswrapper[5052]: I1203 12:51:04.683087 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-a2a9-account-create-update-2z8wq" event={"ID":"9b10d490-7528-42d2-83b7-d54974959b36","Type":"ContainerDied","Data":"86a7818499f167a2a3ec2765dbcfd94d241d7e57151d4df7cf0676f2b4444428"} Dec 03 12:51:04 crc kubenswrapper[5052]: I1203 12:51:04.683114 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="86a7818499f167a2a3ec2765dbcfd94d241d7e57151d4df7cf0676f2b4444428" Dec 03 12:51:04 crc kubenswrapper[5052]: I1203 12:51:04.683154 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-a2a9-account-create-update-2z8wq" Dec 03 12:51:04 crc kubenswrapper[5052]: I1203 12:51:04.687259 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-db-create-9vhcz" event={"ID":"6436478b-ce40-4335-930c-df0c19d4cb1a","Type":"ContainerDied","Data":"fcb8c989032beca63ff17b25e36c67dc7c5a631deab199b322b06140bee88633"} Dec 03 12:51:04 crc kubenswrapper[5052]: I1203 12:51:04.687295 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-db-create-9vhcz" Dec 03 12:51:04 crc kubenswrapper[5052]: I1203 12:51:04.687313 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fcb8c989032beca63ff17b25e36c67dc7c5a631deab199b322b06140bee88633" Dec 03 12:51:05 crc kubenswrapper[5052]: I1203 12:51:05.697595 5052 generic.go:334] "Generic (PLEG): container finished" podID="6e6cb4e4-e8b3-4e38-8c40-f4829e4348a9" containerID="b4f45322078c29bf05e1e6c5ce39ffbdb6524a51a8a145e7edb2b7b7325e566f" exitCode=0 Dec 03 12:51:05 crc kubenswrapper[5052]: I1203 12:51:05.697642 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-db-sync-skdtp" event={"ID":"6e6cb4e4-e8b3-4e38-8c40-f4829e4348a9","Type":"ContainerDied","Data":"b4f45322078c29bf05e1e6c5ce39ffbdb6524a51a8a145e7edb2b7b7325e566f"} Dec 03 12:51:07 crc kubenswrapper[5052]: I1203 12:51:07.025148 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-db-sync-skdtp" Dec 03 12:51:07 crc kubenswrapper[5052]: I1203 12:51:07.155463 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dj92f\" (UniqueName: \"kubernetes.io/projected/6e6cb4e4-e8b3-4e38-8c40-f4829e4348a9-kube-api-access-dj92f\") pod \"6e6cb4e4-e8b3-4e38-8c40-f4829e4348a9\" (UID: \"6e6cb4e4-e8b3-4e38-8c40-f4829e4348a9\") " Dec 03 12:51:07 crc kubenswrapper[5052]: I1203 12:51:07.155814 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6e6cb4e4-e8b3-4e38-8c40-f4829e4348a9-combined-ca-bundle\") pod \"6e6cb4e4-e8b3-4e38-8c40-f4829e4348a9\" (UID: \"6e6cb4e4-e8b3-4e38-8c40-f4829e4348a9\") " Dec 03 12:51:07 crc kubenswrapper[5052]: I1203 12:51:07.155846 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6e6cb4e4-e8b3-4e38-8c40-f4829e4348a9-config-data\") pod \"6e6cb4e4-e8b3-4e38-8c40-f4829e4348a9\" (UID: \"6e6cb4e4-e8b3-4e38-8c40-f4829e4348a9\") " Dec 03 12:51:07 crc kubenswrapper[5052]: I1203 12:51:07.162645 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6e6cb4e4-e8b3-4e38-8c40-f4829e4348a9-kube-api-access-dj92f" (OuterVolumeSpecName: "kube-api-access-dj92f") pod "6e6cb4e4-e8b3-4e38-8c40-f4829e4348a9" (UID: "6e6cb4e4-e8b3-4e38-8c40-f4829e4348a9"). InnerVolumeSpecName "kube-api-access-dj92f". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:51:07 crc kubenswrapper[5052]: I1203 12:51:07.181917 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6e6cb4e4-e8b3-4e38-8c40-f4829e4348a9-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "6e6cb4e4-e8b3-4e38-8c40-f4829e4348a9" (UID: "6e6cb4e4-e8b3-4e38-8c40-f4829e4348a9"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:51:07 crc kubenswrapper[5052]: I1203 12:51:07.197774 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6e6cb4e4-e8b3-4e38-8c40-f4829e4348a9-config-data" (OuterVolumeSpecName: "config-data") pod "6e6cb4e4-e8b3-4e38-8c40-f4829e4348a9" (UID: "6e6cb4e4-e8b3-4e38-8c40-f4829e4348a9"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:51:07 crc kubenswrapper[5052]: I1203 12:51:07.257379 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6e6cb4e4-e8b3-4e38-8c40-f4829e4348a9-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:51:07 crc kubenswrapper[5052]: I1203 12:51:07.257428 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-dj92f\" (UniqueName: \"kubernetes.io/projected/6e6cb4e4-e8b3-4e38-8c40-f4829e4348a9-kube-api-access-dj92f\") on node \"crc\" DevicePath \"\"" Dec 03 12:51:07 crc kubenswrapper[5052]: I1203 12:51:07.257439 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6e6cb4e4-e8b3-4e38-8c40-f4829e4348a9-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:51:07 crc kubenswrapper[5052]: I1203 12:51:07.714016 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-db-sync-skdtp" event={"ID":"6e6cb4e4-e8b3-4e38-8c40-f4829e4348a9","Type":"ContainerDied","Data":"412c4de49318470be85d855599948b08cfdbdb14b979a54c973e4264cbe52c9d"} Dec 03 12:51:07 crc kubenswrapper[5052]: I1203 12:51:07.714062 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="412c4de49318470be85d855599948b08cfdbdb14b979a54c973e4264cbe52c9d" Dec 03 12:51:07 crc kubenswrapper[5052]: I1203 12:51:07.714120 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-db-sync-skdtp" Dec 03 12:51:07 crc kubenswrapper[5052]: I1203 12:51:07.869494 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/keystone-bootstrap-mgkjf"] Dec 03 12:51:07 crc kubenswrapper[5052]: E1203 12:51:07.871026 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="6e6cb4e4-e8b3-4e38-8c40-f4829e4348a9" containerName="keystone-db-sync" Dec 03 12:51:07 crc kubenswrapper[5052]: I1203 12:51:07.871053 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="6e6cb4e4-e8b3-4e38-8c40-f4829e4348a9" containerName="keystone-db-sync" Dec 03 12:51:07 crc kubenswrapper[5052]: E1203 12:51:07.871840 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="38348d87-fbe3-4196-8819-8d90f2720b33" containerName="mariadb-database-create" Dec 03 12:51:07 crc kubenswrapper[5052]: I1203 12:51:07.871869 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="38348d87-fbe3-4196-8819-8d90f2720b33" containerName="mariadb-database-create" Dec 03 12:51:07 crc kubenswrapper[5052]: E1203 12:51:07.871895 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="9b10d490-7528-42d2-83b7-d54974959b36" containerName="mariadb-account-create-update" Dec 03 12:51:07 crc kubenswrapper[5052]: I1203 12:51:07.871907 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b10d490-7528-42d2-83b7-d54974959b36" containerName="mariadb-account-create-update" Dec 03 12:51:07 crc kubenswrapper[5052]: E1203 12:51:07.871938 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="69a643e8-8b6e-4fd4-b445-481364d71485" containerName="mariadb-database-create" Dec 03 12:51:07 crc kubenswrapper[5052]: I1203 12:51:07.871951 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="69a643e8-8b6e-4fd4-b445-481364d71485" containerName="mariadb-database-create" Dec 03 12:51:07 crc kubenswrapper[5052]: E1203 12:51:07.871984 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="8b9df814-7cdb-4696-9045-28d40e5b478c" containerName="mariadb-account-create-update" Dec 03 12:51:07 crc kubenswrapper[5052]: I1203 12:51:07.872006 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="8b9df814-7cdb-4696-9045-28d40e5b478c" containerName="mariadb-account-create-update" Dec 03 12:51:07 crc kubenswrapper[5052]: E1203 12:51:07.872035 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="0a272fa7-1c64-49b1-81a0-b339a5cc686a" containerName="mariadb-account-create-update" Dec 03 12:51:07 crc kubenswrapper[5052]: I1203 12:51:07.872048 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a272fa7-1c64-49b1-81a0-b339a5cc686a" containerName="mariadb-account-create-update" Dec 03 12:51:07 crc kubenswrapper[5052]: E1203 12:51:07.872099 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="6436478b-ce40-4335-930c-df0c19d4cb1a" containerName="mariadb-database-create" Dec 03 12:51:07 crc kubenswrapper[5052]: I1203 12:51:07.872113 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="6436478b-ce40-4335-930c-df0c19d4cb1a" containerName="mariadb-database-create" Dec 03 12:51:07 crc kubenswrapper[5052]: I1203 12:51:07.872421 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="6e6cb4e4-e8b3-4e38-8c40-f4829e4348a9" containerName="keystone-db-sync" Dec 03 12:51:07 crc kubenswrapper[5052]: I1203 12:51:07.872447 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="6436478b-ce40-4335-930c-df0c19d4cb1a" containerName="mariadb-database-create" Dec 03 12:51:07 crc kubenswrapper[5052]: I1203 12:51:07.875316 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="9b10d490-7528-42d2-83b7-d54974959b36" containerName="mariadb-account-create-update" Dec 03 12:51:07 crc kubenswrapper[5052]: I1203 12:51:07.875414 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="38348d87-fbe3-4196-8819-8d90f2720b33" containerName="mariadb-database-create" Dec 03 12:51:07 crc kubenswrapper[5052]: I1203 12:51:07.875438 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="8b9df814-7cdb-4696-9045-28d40e5b478c" containerName="mariadb-account-create-update" Dec 03 12:51:07 crc kubenswrapper[5052]: I1203 12:51:07.875495 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="69a643e8-8b6e-4fd4-b445-481364d71485" containerName="mariadb-database-create" Dec 03 12:51:07 crc kubenswrapper[5052]: I1203 12:51:07.875543 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="0a272fa7-1c64-49b1-81a0-b339a5cc686a" containerName="mariadb-account-create-update" Dec 03 12:51:07 crc kubenswrapper[5052]: I1203 12:51:07.876671 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-bootstrap-mgkjf" Dec 03 12:51:07 crc kubenswrapper[5052]: I1203 12:51:07.879120 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone" Dec 03 12:51:07 crc kubenswrapper[5052]: I1203 12:51:07.879343 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone-keystone-dockercfg-fr5sp" Dec 03 12:51:07 crc kubenswrapper[5052]: I1203 12:51:07.879829 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"osp-secret" Dec 03 12:51:07 crc kubenswrapper[5052]: I1203 12:51:07.881640 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone-scripts" Dec 03 12:51:07 crc kubenswrapper[5052]: I1203 12:51:07.881806 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone-config-data" Dec 03 12:51:07 crc kubenswrapper[5052]: I1203 12:51:07.902434 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-bootstrap-mgkjf"] Dec 03 12:51:07 crc kubenswrapper[5052]: I1203 12:51:07.977363 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/83062467-67e1-4cab-b536-6edf3ea916b4-scripts\") pod \"keystone-bootstrap-mgkjf\" (UID: \"83062467-67e1-4cab-b536-6edf3ea916b4\") " pod="openstack-kuttl-tests/keystone-bootstrap-mgkjf" Dec 03 12:51:07 crc kubenswrapper[5052]: I1203 12:51:07.977724 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/83062467-67e1-4cab-b536-6edf3ea916b4-fernet-keys\") pod \"keystone-bootstrap-mgkjf\" (UID: \"83062467-67e1-4cab-b536-6edf3ea916b4\") " pod="openstack-kuttl-tests/keystone-bootstrap-mgkjf" Dec 03 12:51:07 crc kubenswrapper[5052]: I1203 12:51:07.977815 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/83062467-67e1-4cab-b536-6edf3ea916b4-combined-ca-bundle\") pod \"keystone-bootstrap-mgkjf\" (UID: \"83062467-67e1-4cab-b536-6edf3ea916b4\") " pod="openstack-kuttl-tests/keystone-bootstrap-mgkjf" Dec 03 12:51:07 crc kubenswrapper[5052]: I1203 12:51:07.977914 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-99mb9\" (UniqueName: \"kubernetes.io/projected/83062467-67e1-4cab-b536-6edf3ea916b4-kube-api-access-99mb9\") pod \"keystone-bootstrap-mgkjf\" (UID: \"83062467-67e1-4cab-b536-6edf3ea916b4\") " pod="openstack-kuttl-tests/keystone-bootstrap-mgkjf" Dec 03 12:51:07 crc kubenswrapper[5052]: I1203 12:51:07.978045 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/83062467-67e1-4cab-b536-6edf3ea916b4-credential-keys\") pod \"keystone-bootstrap-mgkjf\" (UID: \"83062467-67e1-4cab-b536-6edf3ea916b4\") " pod="openstack-kuttl-tests/keystone-bootstrap-mgkjf" Dec 03 12:51:07 crc kubenswrapper[5052]: I1203 12:51:07.978126 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/83062467-67e1-4cab-b536-6edf3ea916b4-config-data\") pod \"keystone-bootstrap-mgkjf\" (UID: \"83062467-67e1-4cab-b536-6edf3ea916b4\") " pod="openstack-kuttl-tests/keystone-bootstrap-mgkjf" Dec 03 12:51:08 crc kubenswrapper[5052]: I1203 12:51:08.082907 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/83062467-67e1-4cab-b536-6edf3ea916b4-credential-keys\") pod \"keystone-bootstrap-mgkjf\" (UID: \"83062467-67e1-4cab-b536-6edf3ea916b4\") " pod="openstack-kuttl-tests/keystone-bootstrap-mgkjf" Dec 03 12:51:08 crc kubenswrapper[5052]: I1203 12:51:08.083768 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/83062467-67e1-4cab-b536-6edf3ea916b4-config-data\") pod \"keystone-bootstrap-mgkjf\" (UID: \"83062467-67e1-4cab-b536-6edf3ea916b4\") " pod="openstack-kuttl-tests/keystone-bootstrap-mgkjf" Dec 03 12:51:08 crc kubenswrapper[5052]: I1203 12:51:08.083859 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/83062467-67e1-4cab-b536-6edf3ea916b4-scripts\") pod \"keystone-bootstrap-mgkjf\" (UID: \"83062467-67e1-4cab-b536-6edf3ea916b4\") " pod="openstack-kuttl-tests/keystone-bootstrap-mgkjf" Dec 03 12:51:08 crc kubenswrapper[5052]: I1203 12:51:08.083942 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/83062467-67e1-4cab-b536-6edf3ea916b4-fernet-keys\") pod \"keystone-bootstrap-mgkjf\" (UID: \"83062467-67e1-4cab-b536-6edf3ea916b4\") " pod="openstack-kuttl-tests/keystone-bootstrap-mgkjf" Dec 03 12:51:08 crc kubenswrapper[5052]: I1203 12:51:08.084065 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/83062467-67e1-4cab-b536-6edf3ea916b4-combined-ca-bundle\") pod \"keystone-bootstrap-mgkjf\" (UID: \"83062467-67e1-4cab-b536-6edf3ea916b4\") " pod="openstack-kuttl-tests/keystone-bootstrap-mgkjf" Dec 03 12:51:08 crc kubenswrapper[5052]: I1203 12:51:08.084164 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-99mb9\" (UniqueName: \"kubernetes.io/projected/83062467-67e1-4cab-b536-6edf3ea916b4-kube-api-access-99mb9\") pod \"keystone-bootstrap-mgkjf\" (UID: \"83062467-67e1-4cab-b536-6edf3ea916b4\") " pod="openstack-kuttl-tests/keystone-bootstrap-mgkjf" Dec 03 12:51:08 crc kubenswrapper[5052]: I1203 12:51:08.087279 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/83062467-67e1-4cab-b536-6edf3ea916b4-credential-keys\") pod \"keystone-bootstrap-mgkjf\" (UID: \"83062467-67e1-4cab-b536-6edf3ea916b4\") " pod="openstack-kuttl-tests/keystone-bootstrap-mgkjf" Dec 03 12:51:08 crc kubenswrapper[5052]: I1203 12:51:08.089264 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/83062467-67e1-4cab-b536-6edf3ea916b4-config-data\") pod \"keystone-bootstrap-mgkjf\" (UID: \"83062467-67e1-4cab-b536-6edf3ea916b4\") " pod="openstack-kuttl-tests/keystone-bootstrap-mgkjf" Dec 03 12:51:08 crc kubenswrapper[5052]: I1203 12:51:08.090080 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/83062467-67e1-4cab-b536-6edf3ea916b4-scripts\") pod \"keystone-bootstrap-mgkjf\" (UID: \"83062467-67e1-4cab-b536-6edf3ea916b4\") " pod="openstack-kuttl-tests/keystone-bootstrap-mgkjf" Dec 03 12:51:08 crc kubenswrapper[5052]: I1203 12:51:08.095683 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/83062467-67e1-4cab-b536-6edf3ea916b4-fernet-keys\") pod \"keystone-bootstrap-mgkjf\" (UID: \"83062467-67e1-4cab-b536-6edf3ea916b4\") " pod="openstack-kuttl-tests/keystone-bootstrap-mgkjf" Dec 03 12:51:08 crc kubenswrapper[5052]: I1203 12:51:08.095907 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/83062467-67e1-4cab-b536-6edf3ea916b4-combined-ca-bundle\") pod \"keystone-bootstrap-mgkjf\" (UID: \"83062467-67e1-4cab-b536-6edf3ea916b4\") " pod="openstack-kuttl-tests/keystone-bootstrap-mgkjf" Dec 03 12:51:08 crc kubenswrapper[5052]: I1203 12:51:08.115066 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:51:08 crc kubenswrapper[5052]: I1203 12:51:08.117817 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:51:08 crc kubenswrapper[5052]: I1203 12:51:08.133604 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-scripts" Dec 03 12:51:08 crc kubenswrapper[5052]: I1203 12:51:08.133868 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-config-data" Dec 03 12:51:08 crc kubenswrapper[5052]: I1203 12:51:08.137055 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-99mb9\" (UniqueName: \"kubernetes.io/projected/83062467-67e1-4cab-b536-6edf3ea916b4-kube-api-access-99mb9\") pod \"keystone-bootstrap-mgkjf\" (UID: \"83062467-67e1-4cab-b536-6edf3ea916b4\") " pod="openstack-kuttl-tests/keystone-bootstrap-mgkjf" Dec 03 12:51:08 crc kubenswrapper[5052]: I1203 12:51:08.141976 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/cinder-db-sync-2b42x"] Dec 03 12:51:08 crc kubenswrapper[5052]: I1203 12:51:08.143384 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-db-sync-2b42x" Dec 03 12:51:08 crc kubenswrapper[5052]: I1203 12:51:08.147688 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cinder-scripts" Dec 03 12:51:08 crc kubenswrapper[5052]: I1203 12:51:08.147949 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cinder-config-data" Dec 03 12:51:08 crc kubenswrapper[5052]: I1203 12:51:08.148375 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cinder-cinder-dockercfg-8p42q" Dec 03 12:51:08 crc kubenswrapper[5052]: I1203 12:51:08.163495 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/placement-db-sync-xbbsx"] Dec 03 12:51:08 crc kubenswrapper[5052]: I1203 12:51:08.164945 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-db-sync-xbbsx" Dec 03 12:51:08 crc kubenswrapper[5052]: I1203 12:51:08.167916 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"placement-scripts" Dec 03 12:51:08 crc kubenswrapper[5052]: I1203 12:51:08.168206 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"placement-placement-dockercfg-tlqbn" Dec 03 12:51:08 crc kubenswrapper[5052]: I1203 12:51:08.175186 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"placement-config-data" Dec 03 12:51:08 crc kubenswrapper[5052]: I1203 12:51:08.188484 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-69m5r\" (UniqueName: \"kubernetes.io/projected/c5398d23-68fd-4a1c-af2b-d0a7500cc973-kube-api-access-69m5r\") pod \"ceilometer-0\" (UID: \"c5398d23-68fd-4a1c-af2b-d0a7500cc973\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:51:08 crc kubenswrapper[5052]: I1203 12:51:08.188555 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/c5398d23-68fd-4a1c-af2b-d0a7500cc973-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"c5398d23-68fd-4a1c-af2b-d0a7500cc973\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:51:08 crc kubenswrapper[5052]: I1203 12:51:08.188580 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/c5398d23-68fd-4a1c-af2b-d0a7500cc973-log-httpd\") pod \"ceilometer-0\" (UID: \"c5398d23-68fd-4a1c-af2b-d0a7500cc973\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:51:08 crc kubenswrapper[5052]: I1203 12:51:08.188602 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c5398d23-68fd-4a1c-af2b-d0a7500cc973-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"c5398d23-68fd-4a1c-af2b-d0a7500cc973\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:51:08 crc kubenswrapper[5052]: I1203 12:51:08.188696 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/c5398d23-68fd-4a1c-af2b-d0a7500cc973-scripts\") pod \"ceilometer-0\" (UID: \"c5398d23-68fd-4a1c-af2b-d0a7500cc973\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:51:08 crc kubenswrapper[5052]: I1203 12:51:08.188741 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:51:08 crc kubenswrapper[5052]: I1203 12:51:08.188840 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/c5398d23-68fd-4a1c-af2b-d0a7500cc973-run-httpd\") pod \"ceilometer-0\" (UID: \"c5398d23-68fd-4a1c-af2b-d0a7500cc973\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:51:08 crc kubenswrapper[5052]: I1203 12:51:08.188873 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/c5398d23-68fd-4a1c-af2b-d0a7500cc973-config-data\") pod \"ceilometer-0\" (UID: \"c5398d23-68fd-4a1c-af2b-d0a7500cc973\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:51:08 crc kubenswrapper[5052]: I1203 12:51:08.203219 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-db-sync-2b42x"] Dec 03 12:51:08 crc kubenswrapper[5052]: I1203 12:51:08.211891 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-bootstrap-mgkjf" Dec 03 12:51:08 crc kubenswrapper[5052]: I1203 12:51:08.227665 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/placement-db-sync-xbbsx"] Dec 03 12:51:08 crc kubenswrapper[5052]: I1203 12:51:08.251780 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/neutron-db-sync-jxbww"] Dec 03 12:51:08 crc kubenswrapper[5052]: I1203 12:51:08.253480 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-db-sync-jxbww" Dec 03 12:51:08 crc kubenswrapper[5052]: I1203 12:51:08.263038 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"neutron-httpd-config" Dec 03 12:51:08 crc kubenswrapper[5052]: I1203 12:51:08.263270 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"neutron-config" Dec 03 12:51:08 crc kubenswrapper[5052]: I1203 12:51:08.263376 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"neutron-neutron-dockercfg-mhnq6" Dec 03 12:51:08 crc kubenswrapper[5052]: I1203 12:51:08.290293 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/468756d2-d46f-411a-a8ce-0c160d58c278-logs\") pod \"placement-db-sync-xbbsx\" (UID: \"468756d2-d46f-411a-a8ce-0c160d58c278\") " pod="openstack-kuttl-tests/placement-db-sync-xbbsx" Dec 03 12:51:08 crc kubenswrapper[5052]: I1203 12:51:08.290335 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f56tk\" (UniqueName: \"kubernetes.io/projected/468756d2-d46f-411a-a8ce-0c160d58c278-kube-api-access-f56tk\") pod \"placement-db-sync-xbbsx\" (UID: \"468756d2-d46f-411a-a8ce-0c160d58c278\") " pod="openstack-kuttl-tests/placement-db-sync-xbbsx" Dec 03 12:51:08 crc kubenswrapper[5052]: I1203 12:51:08.290355 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/468756d2-d46f-411a-a8ce-0c160d58c278-scripts\") pod \"placement-db-sync-xbbsx\" (UID: \"468756d2-d46f-411a-a8ce-0c160d58c278\") " pod="openstack-kuttl-tests/placement-db-sync-xbbsx" Dec 03 12:51:08 crc kubenswrapper[5052]: I1203 12:51:08.290385 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/8212a97d-b76a-4b60-b16b-914a4bbe5b9d-db-sync-config-data\") pod \"cinder-db-sync-2b42x\" (UID: \"8212a97d-b76a-4b60-b16b-914a4bbe5b9d\") " pod="openstack-kuttl-tests/cinder-db-sync-2b42x" Dec 03 12:51:08 crc kubenswrapper[5052]: I1203 12:51:08.290408 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/c5398d23-68fd-4a1c-af2b-d0a7500cc973-run-httpd\") pod \"ceilometer-0\" (UID: \"c5398d23-68fd-4a1c-af2b-d0a7500cc973\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:51:08 crc kubenswrapper[5052]: I1203 12:51:08.290427 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/c5398d23-68fd-4a1c-af2b-d0a7500cc973-config-data\") pod \"ceilometer-0\" (UID: \"c5398d23-68fd-4a1c-af2b-d0a7500cc973\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:51:08 crc kubenswrapper[5052]: I1203 12:51:08.290447 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/8212a97d-b76a-4b60-b16b-914a4bbe5b9d-config-data\") pod \"cinder-db-sync-2b42x\" (UID: \"8212a97d-b76a-4b60-b16b-914a4bbe5b9d\") " pod="openstack-kuttl-tests/cinder-db-sync-2b42x" Dec 03 12:51:08 crc kubenswrapper[5052]: I1203 12:51:08.290478 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/8212a97d-b76a-4b60-b16b-914a4bbe5b9d-etc-machine-id\") pod \"cinder-db-sync-2b42x\" (UID: \"8212a97d-b76a-4b60-b16b-914a4bbe5b9d\") " pod="openstack-kuttl-tests/cinder-db-sync-2b42x" Dec 03 12:51:08 crc kubenswrapper[5052]: I1203 12:51:08.290500 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9z5l6\" (UniqueName: \"kubernetes.io/projected/8212a97d-b76a-4b60-b16b-914a4bbe5b9d-kube-api-access-9z5l6\") pod \"cinder-db-sync-2b42x\" (UID: \"8212a97d-b76a-4b60-b16b-914a4bbe5b9d\") " pod="openstack-kuttl-tests/cinder-db-sync-2b42x" Dec 03 12:51:08 crc kubenswrapper[5052]: I1203 12:51:08.291108 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-69m5r\" (UniqueName: \"kubernetes.io/projected/c5398d23-68fd-4a1c-af2b-d0a7500cc973-kube-api-access-69m5r\") pod \"ceilometer-0\" (UID: \"c5398d23-68fd-4a1c-af2b-d0a7500cc973\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:51:08 crc kubenswrapper[5052]: I1203 12:51:08.291136 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/c5398d23-68fd-4a1c-af2b-d0a7500cc973-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"c5398d23-68fd-4a1c-af2b-d0a7500cc973\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:51:08 crc kubenswrapper[5052]: I1203 12:51:08.291150 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/c5398d23-68fd-4a1c-af2b-d0a7500cc973-log-httpd\") pod \"ceilometer-0\" (UID: \"c5398d23-68fd-4a1c-af2b-d0a7500cc973\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:51:08 crc kubenswrapper[5052]: I1203 12:51:08.291166 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c5398d23-68fd-4a1c-af2b-d0a7500cc973-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"c5398d23-68fd-4a1c-af2b-d0a7500cc973\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:51:08 crc kubenswrapper[5052]: I1203 12:51:08.291201 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/8212a97d-b76a-4b60-b16b-914a4bbe5b9d-scripts\") pod \"cinder-db-sync-2b42x\" (UID: \"8212a97d-b76a-4b60-b16b-914a4bbe5b9d\") " pod="openstack-kuttl-tests/cinder-db-sync-2b42x" Dec 03 12:51:08 crc kubenswrapper[5052]: I1203 12:51:08.291228 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8212a97d-b76a-4b60-b16b-914a4bbe5b9d-combined-ca-bundle\") pod \"cinder-db-sync-2b42x\" (UID: \"8212a97d-b76a-4b60-b16b-914a4bbe5b9d\") " pod="openstack-kuttl-tests/cinder-db-sync-2b42x" Dec 03 12:51:08 crc kubenswrapper[5052]: I1203 12:51:08.291256 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/c5398d23-68fd-4a1c-af2b-d0a7500cc973-scripts\") pod \"ceilometer-0\" (UID: \"c5398d23-68fd-4a1c-af2b-d0a7500cc973\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:51:08 crc kubenswrapper[5052]: I1203 12:51:08.291279 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/468756d2-d46f-411a-a8ce-0c160d58c278-combined-ca-bundle\") pod \"placement-db-sync-xbbsx\" (UID: \"468756d2-d46f-411a-a8ce-0c160d58c278\") " pod="openstack-kuttl-tests/placement-db-sync-xbbsx" Dec 03 12:51:08 crc kubenswrapper[5052]: I1203 12:51:08.291299 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/468756d2-d46f-411a-a8ce-0c160d58c278-config-data\") pod \"placement-db-sync-xbbsx\" (UID: \"468756d2-d46f-411a-a8ce-0c160d58c278\") " pod="openstack-kuttl-tests/placement-db-sync-xbbsx" Dec 03 12:51:08 crc kubenswrapper[5052]: I1203 12:51:08.291945 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/c5398d23-68fd-4a1c-af2b-d0a7500cc973-run-httpd\") pod \"ceilometer-0\" (UID: \"c5398d23-68fd-4a1c-af2b-d0a7500cc973\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:51:08 crc kubenswrapper[5052]: I1203 12:51:08.295992 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/c5398d23-68fd-4a1c-af2b-d0a7500cc973-log-httpd\") pod \"ceilometer-0\" (UID: \"c5398d23-68fd-4a1c-af2b-d0a7500cc973\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:51:08 crc kubenswrapper[5052]: I1203 12:51:08.296536 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-db-sync-jxbww"] Dec 03 12:51:08 crc kubenswrapper[5052]: I1203 12:51:08.304259 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/c5398d23-68fd-4a1c-af2b-d0a7500cc973-scripts\") pod \"ceilometer-0\" (UID: \"c5398d23-68fd-4a1c-af2b-d0a7500cc973\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:51:08 crc kubenswrapper[5052]: I1203 12:51:08.305080 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/c5398d23-68fd-4a1c-af2b-d0a7500cc973-config-data\") pod \"ceilometer-0\" (UID: \"c5398d23-68fd-4a1c-af2b-d0a7500cc973\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:51:08 crc kubenswrapper[5052]: I1203 12:51:08.305708 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c5398d23-68fd-4a1c-af2b-d0a7500cc973-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"c5398d23-68fd-4a1c-af2b-d0a7500cc973\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:51:08 crc kubenswrapper[5052]: I1203 12:51:08.317215 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/c5398d23-68fd-4a1c-af2b-d0a7500cc973-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"c5398d23-68fd-4a1c-af2b-d0a7500cc973\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:51:08 crc kubenswrapper[5052]: I1203 12:51:08.321623 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/barbican-db-sync-rsmgs"] Dec 03 12:51:08 crc kubenswrapper[5052]: I1203 12:51:08.322834 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-db-sync-rsmgs" Dec 03 12:51:08 crc kubenswrapper[5052]: I1203 12:51:08.327223 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"barbican-barbican-dockercfg-74cwb" Dec 03 12:51:08 crc kubenswrapper[5052]: I1203 12:51:08.327914 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"barbican-config-data" Dec 03 12:51:08 crc kubenswrapper[5052]: I1203 12:51:08.331468 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-69m5r\" (UniqueName: \"kubernetes.io/projected/c5398d23-68fd-4a1c-af2b-d0a7500cc973-kube-api-access-69m5r\") pod \"ceilometer-0\" (UID: \"c5398d23-68fd-4a1c-af2b-d0a7500cc973\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:51:08 crc kubenswrapper[5052]: I1203 12:51:08.349609 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-db-sync-rsmgs"] Dec 03 12:51:08 crc kubenswrapper[5052]: I1203 12:51:08.392780 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/8212a97d-b76a-4b60-b16b-914a4bbe5b9d-config-data\") pod \"cinder-db-sync-2b42x\" (UID: \"8212a97d-b76a-4b60-b16b-914a4bbe5b9d\") " pod="openstack-kuttl-tests/cinder-db-sync-2b42x" Dec 03 12:51:08 crc kubenswrapper[5052]: I1203 12:51:08.392839 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lnhfk\" (UniqueName: \"kubernetes.io/projected/2fab7846-f3ad-4673-b813-36c20d4b7611-kube-api-access-lnhfk\") pod \"barbican-db-sync-rsmgs\" (UID: \"2fab7846-f3ad-4673-b813-36c20d4b7611\") " pod="openstack-kuttl-tests/barbican-db-sync-rsmgs" Dec 03 12:51:08 crc kubenswrapper[5052]: I1203 12:51:08.392874 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/8212a97d-b76a-4b60-b16b-914a4bbe5b9d-etc-machine-id\") pod \"cinder-db-sync-2b42x\" (UID: \"8212a97d-b76a-4b60-b16b-914a4bbe5b9d\") " pod="openstack-kuttl-tests/cinder-db-sync-2b42x" Dec 03 12:51:08 crc kubenswrapper[5052]: I1203 12:51:08.392940 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-9z5l6\" (UniqueName: \"kubernetes.io/projected/8212a97d-b76a-4b60-b16b-914a4bbe5b9d-kube-api-access-9z5l6\") pod \"cinder-db-sync-2b42x\" (UID: \"8212a97d-b76a-4b60-b16b-914a4bbe5b9d\") " pod="openstack-kuttl-tests/cinder-db-sync-2b42x" Dec 03 12:51:08 crc kubenswrapper[5052]: I1203 12:51:08.392973 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/ffe1f27e-ba05-4da0-a02b-a73b3bf63bc3-config\") pod \"neutron-db-sync-jxbww\" (UID: \"ffe1f27e-ba05-4da0-a02b-a73b3bf63bc3\") " pod="openstack-kuttl-tests/neutron-db-sync-jxbww" Dec 03 12:51:08 crc kubenswrapper[5052]: I1203 12:51:08.393044 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/8212a97d-b76a-4b60-b16b-914a4bbe5b9d-scripts\") pod \"cinder-db-sync-2b42x\" (UID: \"8212a97d-b76a-4b60-b16b-914a4bbe5b9d\") " pod="openstack-kuttl-tests/cinder-db-sync-2b42x" Dec 03 12:51:08 crc kubenswrapper[5052]: I1203 12:51:08.393074 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ffe1f27e-ba05-4da0-a02b-a73b3bf63bc3-combined-ca-bundle\") pod \"neutron-db-sync-jxbww\" (UID: \"ffe1f27e-ba05-4da0-a02b-a73b3bf63bc3\") " pod="openstack-kuttl-tests/neutron-db-sync-jxbww" Dec 03 12:51:08 crc kubenswrapper[5052]: I1203 12:51:08.393141 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8212a97d-b76a-4b60-b16b-914a4bbe5b9d-combined-ca-bundle\") pod \"cinder-db-sync-2b42x\" (UID: \"8212a97d-b76a-4b60-b16b-914a4bbe5b9d\") " pod="openstack-kuttl-tests/cinder-db-sync-2b42x" Dec 03 12:51:08 crc kubenswrapper[5052]: I1203 12:51:08.393202 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nhfzw\" (UniqueName: \"kubernetes.io/projected/ffe1f27e-ba05-4da0-a02b-a73b3bf63bc3-kube-api-access-nhfzw\") pod \"neutron-db-sync-jxbww\" (UID: \"ffe1f27e-ba05-4da0-a02b-a73b3bf63bc3\") " pod="openstack-kuttl-tests/neutron-db-sync-jxbww" Dec 03 12:51:08 crc kubenswrapper[5052]: I1203 12:51:08.393241 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2fab7846-f3ad-4673-b813-36c20d4b7611-combined-ca-bundle\") pod \"barbican-db-sync-rsmgs\" (UID: \"2fab7846-f3ad-4673-b813-36c20d4b7611\") " pod="openstack-kuttl-tests/barbican-db-sync-rsmgs" Dec 03 12:51:08 crc kubenswrapper[5052]: I1203 12:51:08.393269 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/468756d2-d46f-411a-a8ce-0c160d58c278-combined-ca-bundle\") pod \"placement-db-sync-xbbsx\" (UID: \"468756d2-d46f-411a-a8ce-0c160d58c278\") " pod="openstack-kuttl-tests/placement-db-sync-xbbsx" Dec 03 12:51:08 crc kubenswrapper[5052]: I1203 12:51:08.393294 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/2fab7846-f3ad-4673-b813-36c20d4b7611-db-sync-config-data\") pod \"barbican-db-sync-rsmgs\" (UID: \"2fab7846-f3ad-4673-b813-36c20d4b7611\") " pod="openstack-kuttl-tests/barbican-db-sync-rsmgs" Dec 03 12:51:08 crc kubenswrapper[5052]: I1203 12:51:08.393323 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/468756d2-d46f-411a-a8ce-0c160d58c278-config-data\") pod \"placement-db-sync-xbbsx\" (UID: \"468756d2-d46f-411a-a8ce-0c160d58c278\") " pod="openstack-kuttl-tests/placement-db-sync-xbbsx" Dec 03 12:51:08 crc kubenswrapper[5052]: I1203 12:51:08.393365 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/468756d2-d46f-411a-a8ce-0c160d58c278-logs\") pod \"placement-db-sync-xbbsx\" (UID: \"468756d2-d46f-411a-a8ce-0c160d58c278\") " pod="openstack-kuttl-tests/placement-db-sync-xbbsx" Dec 03 12:51:08 crc kubenswrapper[5052]: I1203 12:51:08.393392 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-f56tk\" (UniqueName: \"kubernetes.io/projected/468756d2-d46f-411a-a8ce-0c160d58c278-kube-api-access-f56tk\") pod \"placement-db-sync-xbbsx\" (UID: \"468756d2-d46f-411a-a8ce-0c160d58c278\") " pod="openstack-kuttl-tests/placement-db-sync-xbbsx" Dec 03 12:51:08 crc kubenswrapper[5052]: I1203 12:51:08.393418 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/468756d2-d46f-411a-a8ce-0c160d58c278-scripts\") pod \"placement-db-sync-xbbsx\" (UID: \"468756d2-d46f-411a-a8ce-0c160d58c278\") " pod="openstack-kuttl-tests/placement-db-sync-xbbsx" Dec 03 12:51:08 crc kubenswrapper[5052]: I1203 12:51:08.393457 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/8212a97d-b76a-4b60-b16b-914a4bbe5b9d-db-sync-config-data\") pod \"cinder-db-sync-2b42x\" (UID: \"8212a97d-b76a-4b60-b16b-914a4bbe5b9d\") " pod="openstack-kuttl-tests/cinder-db-sync-2b42x" Dec 03 12:51:08 crc kubenswrapper[5052]: I1203 12:51:08.395938 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/8212a97d-b76a-4b60-b16b-914a4bbe5b9d-etc-machine-id\") pod \"cinder-db-sync-2b42x\" (UID: \"8212a97d-b76a-4b60-b16b-914a4bbe5b9d\") " pod="openstack-kuttl-tests/cinder-db-sync-2b42x" Dec 03 12:51:08 crc kubenswrapper[5052]: I1203 12:51:08.396841 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/468756d2-d46f-411a-a8ce-0c160d58c278-logs\") pod \"placement-db-sync-xbbsx\" (UID: \"468756d2-d46f-411a-a8ce-0c160d58c278\") " pod="openstack-kuttl-tests/placement-db-sync-xbbsx" Dec 03 12:51:08 crc kubenswrapper[5052]: I1203 12:51:08.400491 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/8212a97d-b76a-4b60-b16b-914a4bbe5b9d-db-sync-config-data\") pod \"cinder-db-sync-2b42x\" (UID: \"8212a97d-b76a-4b60-b16b-914a4bbe5b9d\") " pod="openstack-kuttl-tests/cinder-db-sync-2b42x" Dec 03 12:51:08 crc kubenswrapper[5052]: I1203 12:51:08.400529 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/8212a97d-b76a-4b60-b16b-914a4bbe5b9d-scripts\") pod \"cinder-db-sync-2b42x\" (UID: \"8212a97d-b76a-4b60-b16b-914a4bbe5b9d\") " pod="openstack-kuttl-tests/cinder-db-sync-2b42x" Dec 03 12:51:08 crc kubenswrapper[5052]: I1203 12:51:08.400587 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/468756d2-d46f-411a-a8ce-0c160d58c278-combined-ca-bundle\") pod \"placement-db-sync-xbbsx\" (UID: \"468756d2-d46f-411a-a8ce-0c160d58c278\") " pod="openstack-kuttl-tests/placement-db-sync-xbbsx" Dec 03 12:51:08 crc kubenswrapper[5052]: I1203 12:51:08.401133 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/468756d2-d46f-411a-a8ce-0c160d58c278-scripts\") pod \"placement-db-sync-xbbsx\" (UID: \"468756d2-d46f-411a-a8ce-0c160d58c278\") " pod="openstack-kuttl-tests/placement-db-sync-xbbsx" Dec 03 12:51:08 crc kubenswrapper[5052]: I1203 12:51:08.401461 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/8212a97d-b76a-4b60-b16b-914a4bbe5b9d-config-data\") pod \"cinder-db-sync-2b42x\" (UID: \"8212a97d-b76a-4b60-b16b-914a4bbe5b9d\") " pod="openstack-kuttl-tests/cinder-db-sync-2b42x" Dec 03 12:51:08 crc kubenswrapper[5052]: I1203 12:51:08.401582 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8212a97d-b76a-4b60-b16b-914a4bbe5b9d-combined-ca-bundle\") pod \"cinder-db-sync-2b42x\" (UID: \"8212a97d-b76a-4b60-b16b-914a4bbe5b9d\") " pod="openstack-kuttl-tests/cinder-db-sync-2b42x" Dec 03 12:51:08 crc kubenswrapper[5052]: I1203 12:51:08.405428 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/468756d2-d46f-411a-a8ce-0c160d58c278-config-data\") pod \"placement-db-sync-xbbsx\" (UID: \"468756d2-d46f-411a-a8ce-0c160d58c278\") " pod="openstack-kuttl-tests/placement-db-sync-xbbsx" Dec 03 12:51:08 crc kubenswrapper[5052]: I1203 12:51:08.418556 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-f56tk\" (UniqueName: \"kubernetes.io/projected/468756d2-d46f-411a-a8ce-0c160d58c278-kube-api-access-f56tk\") pod \"placement-db-sync-xbbsx\" (UID: \"468756d2-d46f-411a-a8ce-0c160d58c278\") " pod="openstack-kuttl-tests/placement-db-sync-xbbsx" Dec 03 12:51:08 crc kubenswrapper[5052]: I1203 12:51:08.418936 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-9z5l6\" (UniqueName: \"kubernetes.io/projected/8212a97d-b76a-4b60-b16b-914a4bbe5b9d-kube-api-access-9z5l6\") pod \"cinder-db-sync-2b42x\" (UID: \"8212a97d-b76a-4b60-b16b-914a4bbe5b9d\") " pod="openstack-kuttl-tests/cinder-db-sync-2b42x" Dec 03 12:51:08 crc kubenswrapper[5052]: I1203 12:51:08.496451 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/ffe1f27e-ba05-4da0-a02b-a73b3bf63bc3-config\") pod \"neutron-db-sync-jxbww\" (UID: \"ffe1f27e-ba05-4da0-a02b-a73b3bf63bc3\") " pod="openstack-kuttl-tests/neutron-db-sync-jxbww" Dec 03 12:51:08 crc kubenswrapper[5052]: I1203 12:51:08.496543 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ffe1f27e-ba05-4da0-a02b-a73b3bf63bc3-combined-ca-bundle\") pod \"neutron-db-sync-jxbww\" (UID: \"ffe1f27e-ba05-4da0-a02b-a73b3bf63bc3\") " pod="openstack-kuttl-tests/neutron-db-sync-jxbww" Dec 03 12:51:08 crc kubenswrapper[5052]: I1203 12:51:08.496581 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-nhfzw\" (UniqueName: \"kubernetes.io/projected/ffe1f27e-ba05-4da0-a02b-a73b3bf63bc3-kube-api-access-nhfzw\") pod \"neutron-db-sync-jxbww\" (UID: \"ffe1f27e-ba05-4da0-a02b-a73b3bf63bc3\") " pod="openstack-kuttl-tests/neutron-db-sync-jxbww" Dec 03 12:51:08 crc kubenswrapper[5052]: I1203 12:51:08.496607 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2fab7846-f3ad-4673-b813-36c20d4b7611-combined-ca-bundle\") pod \"barbican-db-sync-rsmgs\" (UID: \"2fab7846-f3ad-4673-b813-36c20d4b7611\") " pod="openstack-kuttl-tests/barbican-db-sync-rsmgs" Dec 03 12:51:08 crc kubenswrapper[5052]: I1203 12:51:08.496629 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/2fab7846-f3ad-4673-b813-36c20d4b7611-db-sync-config-data\") pod \"barbican-db-sync-rsmgs\" (UID: \"2fab7846-f3ad-4673-b813-36c20d4b7611\") " pod="openstack-kuttl-tests/barbican-db-sync-rsmgs" Dec 03 12:51:08 crc kubenswrapper[5052]: I1203 12:51:08.496701 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-lnhfk\" (UniqueName: \"kubernetes.io/projected/2fab7846-f3ad-4673-b813-36c20d4b7611-kube-api-access-lnhfk\") pod \"barbican-db-sync-rsmgs\" (UID: \"2fab7846-f3ad-4673-b813-36c20d4b7611\") " pod="openstack-kuttl-tests/barbican-db-sync-rsmgs" Dec 03 12:51:08 crc kubenswrapper[5052]: I1203 12:51:08.501563 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/ffe1f27e-ba05-4da0-a02b-a73b3bf63bc3-config\") pod \"neutron-db-sync-jxbww\" (UID: \"ffe1f27e-ba05-4da0-a02b-a73b3bf63bc3\") " pod="openstack-kuttl-tests/neutron-db-sync-jxbww" Dec 03 12:51:08 crc kubenswrapper[5052]: I1203 12:51:08.502249 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/2fab7846-f3ad-4673-b813-36c20d4b7611-db-sync-config-data\") pod \"barbican-db-sync-rsmgs\" (UID: \"2fab7846-f3ad-4673-b813-36c20d4b7611\") " pod="openstack-kuttl-tests/barbican-db-sync-rsmgs" Dec 03 12:51:08 crc kubenswrapper[5052]: I1203 12:51:08.502416 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ffe1f27e-ba05-4da0-a02b-a73b3bf63bc3-combined-ca-bundle\") pod \"neutron-db-sync-jxbww\" (UID: \"ffe1f27e-ba05-4da0-a02b-a73b3bf63bc3\") " pod="openstack-kuttl-tests/neutron-db-sync-jxbww" Dec 03 12:51:08 crc kubenswrapper[5052]: I1203 12:51:08.504946 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:51:08 crc kubenswrapper[5052]: I1203 12:51:08.506076 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2fab7846-f3ad-4673-b813-36c20d4b7611-combined-ca-bundle\") pod \"barbican-db-sync-rsmgs\" (UID: \"2fab7846-f3ad-4673-b813-36c20d4b7611\") " pod="openstack-kuttl-tests/barbican-db-sync-rsmgs" Dec 03 12:51:08 crc kubenswrapper[5052]: I1203 12:51:08.516825 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-nhfzw\" (UniqueName: \"kubernetes.io/projected/ffe1f27e-ba05-4da0-a02b-a73b3bf63bc3-kube-api-access-nhfzw\") pod \"neutron-db-sync-jxbww\" (UID: \"ffe1f27e-ba05-4da0-a02b-a73b3bf63bc3\") " pod="openstack-kuttl-tests/neutron-db-sync-jxbww" Dec 03 12:51:08 crc kubenswrapper[5052]: I1203 12:51:08.519199 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-lnhfk\" (UniqueName: \"kubernetes.io/projected/2fab7846-f3ad-4673-b813-36c20d4b7611-kube-api-access-lnhfk\") pod \"barbican-db-sync-rsmgs\" (UID: \"2fab7846-f3ad-4673-b813-36c20d4b7611\") " pod="openstack-kuttl-tests/barbican-db-sync-rsmgs" Dec 03 12:51:08 crc kubenswrapper[5052]: I1203 12:51:08.549723 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-db-sync-2b42x" Dec 03 12:51:08 crc kubenswrapper[5052]: I1203 12:51:08.556018 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-db-sync-xbbsx" Dec 03 12:51:08 crc kubenswrapper[5052]: I1203 12:51:08.674379 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-db-sync-jxbww" Dec 03 12:51:08 crc kubenswrapper[5052]: I1203 12:51:08.686822 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-db-sync-rsmgs" Dec 03 12:51:08 crc kubenswrapper[5052]: I1203 12:51:08.741218 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-bootstrap-mgkjf"] Dec 03 12:51:08 crc kubenswrapper[5052]: I1203 12:51:08.961121 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:51:08 crc kubenswrapper[5052]: I1203 12:51:08.963374 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:51:08 crc kubenswrapper[5052]: I1203 12:51:08.967422 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-glance-dockercfg-zph2l" Dec 03 12:51:08 crc kubenswrapper[5052]: I1203 12:51:08.968330 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-glance-default-internal-svc" Dec 03 12:51:08 crc kubenswrapper[5052]: I1203 12:51:08.968387 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-scripts" Dec 03 12:51:08 crc kubenswrapper[5052]: I1203 12:51:08.969850 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-default-internal-config-data" Dec 03 12:51:08 crc kubenswrapper[5052]: I1203 12:51:08.994570 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:51:09 crc kubenswrapper[5052]: I1203 12:51:09.007883 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage06-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage06-crc\") pod \"glance-default-internal-api-0\" (UID: \"37d28f5c-4b30-437d-a68e-44bc3ef970a5\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:51:09 crc kubenswrapper[5052]: I1203 12:51:09.007936 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/37d28f5c-4b30-437d-a68e-44bc3ef970a5-config-data\") pod \"glance-default-internal-api-0\" (UID: \"37d28f5c-4b30-437d-a68e-44bc3ef970a5\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:51:09 crc kubenswrapper[5052]: I1203 12:51:09.007958 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/37d28f5c-4b30-437d-a68e-44bc3ef970a5-logs\") pod \"glance-default-internal-api-0\" (UID: \"37d28f5c-4b30-437d-a68e-44bc3ef970a5\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:51:09 crc kubenswrapper[5052]: I1203 12:51:09.008030 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/37d28f5c-4b30-437d-a68e-44bc3ef970a5-scripts\") pod \"glance-default-internal-api-0\" (UID: \"37d28f5c-4b30-437d-a68e-44bc3ef970a5\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:51:09 crc kubenswrapper[5052]: I1203 12:51:09.008151 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/37d28f5c-4b30-437d-a68e-44bc3ef970a5-combined-ca-bundle\") pod \"glance-default-internal-api-0\" (UID: \"37d28f5c-4b30-437d-a68e-44bc3ef970a5\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:51:09 crc kubenswrapper[5052]: I1203 12:51:09.008234 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6rz7r\" (UniqueName: \"kubernetes.io/projected/37d28f5c-4b30-437d-a68e-44bc3ef970a5-kube-api-access-6rz7r\") pod \"glance-default-internal-api-0\" (UID: \"37d28f5c-4b30-437d-a68e-44bc3ef970a5\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:51:09 crc kubenswrapper[5052]: I1203 12:51:09.008334 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/37d28f5c-4b30-437d-a68e-44bc3ef970a5-internal-tls-certs\") pod \"glance-default-internal-api-0\" (UID: \"37d28f5c-4b30-437d-a68e-44bc3ef970a5\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:51:09 crc kubenswrapper[5052]: I1203 12:51:09.008431 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/37d28f5c-4b30-437d-a68e-44bc3ef970a5-httpd-run\") pod \"glance-default-internal-api-0\" (UID: \"37d28f5c-4b30-437d-a68e-44bc3ef970a5\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:51:09 crc kubenswrapper[5052]: I1203 12:51:09.033640 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:51:09 crc kubenswrapper[5052]: I1203 12:51:09.035234 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:51:09 crc kubenswrapper[5052]: I1203 12:51:09.041054 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-default-external-config-data" Dec 03 12:51:09 crc kubenswrapper[5052]: I1203 12:51:09.041098 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-glance-default-public-svc" Dec 03 12:51:09 crc kubenswrapper[5052]: I1203 12:51:09.052835 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:51:09 crc kubenswrapper[5052]: I1203 12:51:09.065120 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:51:09 crc kubenswrapper[5052]: I1203 12:51:09.115821 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/placement-db-sync-xbbsx"] Dec 03 12:51:09 crc kubenswrapper[5052]: I1203 12:51:09.116667 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/9127a05d-9fe5-42b6-b54a-acc1290ad6ee-httpd-run\") pod \"glance-default-external-api-0\" (UID: \"9127a05d-9fe5-42b6-b54a-acc1290ad6ee\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:51:09 crc kubenswrapper[5052]: I1203 12:51:09.116695 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage06-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage06-crc\") pod \"glance-default-internal-api-0\" (UID: \"37d28f5c-4b30-437d-a68e-44bc3ef970a5\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:51:09 crc kubenswrapper[5052]: I1203 12:51:09.116730 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/37d28f5c-4b30-437d-a68e-44bc3ef970a5-config-data\") pod \"glance-default-internal-api-0\" (UID: \"37d28f5c-4b30-437d-a68e-44bc3ef970a5\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:51:09 crc kubenswrapper[5052]: I1203 12:51:09.116746 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/37d28f5c-4b30-437d-a68e-44bc3ef970a5-logs\") pod \"glance-default-internal-api-0\" (UID: \"37d28f5c-4b30-437d-a68e-44bc3ef970a5\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:51:09 crc kubenswrapper[5052]: I1203 12:51:09.116770 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/9127a05d-9fe5-42b6-b54a-acc1290ad6ee-config-data\") pod \"glance-default-external-api-0\" (UID: \"9127a05d-9fe5-42b6-b54a-acc1290ad6ee\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:51:09 crc kubenswrapper[5052]: I1203 12:51:09.116793 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dntdw\" (UniqueName: \"kubernetes.io/projected/9127a05d-9fe5-42b6-b54a-acc1290ad6ee-kube-api-access-dntdw\") pod \"glance-default-external-api-0\" (UID: \"9127a05d-9fe5-42b6-b54a-acc1290ad6ee\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:51:09 crc kubenswrapper[5052]: I1203 12:51:09.116810 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/9127a05d-9fe5-42b6-b54a-acc1290ad6ee-scripts\") pod \"glance-default-external-api-0\" (UID: \"9127a05d-9fe5-42b6-b54a-acc1290ad6ee\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:51:09 crc kubenswrapper[5052]: I1203 12:51:09.116853 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9127a05d-9fe5-42b6-b54a-acc1290ad6ee-combined-ca-bundle\") pod \"glance-default-external-api-0\" (UID: \"9127a05d-9fe5-42b6-b54a-acc1290ad6ee\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:51:09 crc kubenswrapper[5052]: I1203 12:51:09.116889 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/9127a05d-9fe5-42b6-b54a-acc1290ad6ee-logs\") pod \"glance-default-external-api-0\" (UID: \"9127a05d-9fe5-42b6-b54a-acc1290ad6ee\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:51:09 crc kubenswrapper[5052]: I1203 12:51:09.116915 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/37d28f5c-4b30-437d-a68e-44bc3ef970a5-scripts\") pod \"glance-default-internal-api-0\" (UID: \"37d28f5c-4b30-437d-a68e-44bc3ef970a5\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:51:09 crc kubenswrapper[5052]: I1203 12:51:09.116930 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/9127a05d-9fe5-42b6-b54a-acc1290ad6ee-public-tls-certs\") pod \"glance-default-external-api-0\" (UID: \"9127a05d-9fe5-42b6-b54a-acc1290ad6ee\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:51:09 crc kubenswrapper[5052]: I1203 12:51:09.116950 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/37d28f5c-4b30-437d-a68e-44bc3ef970a5-combined-ca-bundle\") pod \"glance-default-internal-api-0\" (UID: \"37d28f5c-4b30-437d-a68e-44bc3ef970a5\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:51:09 crc kubenswrapper[5052]: I1203 12:51:09.116972 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-6rz7r\" (UniqueName: \"kubernetes.io/projected/37d28f5c-4b30-437d-a68e-44bc3ef970a5-kube-api-access-6rz7r\") pod \"glance-default-internal-api-0\" (UID: \"37d28f5c-4b30-437d-a68e-44bc3ef970a5\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:51:09 crc kubenswrapper[5052]: I1203 12:51:09.117003 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage19-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage19-crc\") pod \"glance-default-external-api-0\" (UID: \"9127a05d-9fe5-42b6-b54a-acc1290ad6ee\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:51:09 crc kubenswrapper[5052]: I1203 12:51:09.117019 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/37d28f5c-4b30-437d-a68e-44bc3ef970a5-internal-tls-certs\") pod \"glance-default-internal-api-0\" (UID: \"37d28f5c-4b30-437d-a68e-44bc3ef970a5\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:51:09 crc kubenswrapper[5052]: I1203 12:51:09.117037 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/37d28f5c-4b30-437d-a68e-44bc3ef970a5-httpd-run\") pod \"glance-default-internal-api-0\" (UID: \"37d28f5c-4b30-437d-a68e-44bc3ef970a5\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:51:09 crc kubenswrapper[5052]: I1203 12:51:09.117045 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage06-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage06-crc\") pod \"glance-default-internal-api-0\" (UID: \"37d28f5c-4b30-437d-a68e-44bc3ef970a5\") device mount path \"/mnt/openstack/pv06\"" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:51:09 crc kubenswrapper[5052]: I1203 12:51:09.119338 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/37d28f5c-4b30-437d-a68e-44bc3ef970a5-logs\") pod \"glance-default-internal-api-0\" (UID: \"37d28f5c-4b30-437d-a68e-44bc3ef970a5\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:51:09 crc kubenswrapper[5052]: I1203 12:51:09.121274 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/37d28f5c-4b30-437d-a68e-44bc3ef970a5-httpd-run\") pod \"glance-default-internal-api-0\" (UID: \"37d28f5c-4b30-437d-a68e-44bc3ef970a5\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:51:09 crc kubenswrapper[5052]: I1203 12:51:09.138905 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/37d28f5c-4b30-437d-a68e-44bc3ef970a5-combined-ca-bundle\") pod \"glance-default-internal-api-0\" (UID: \"37d28f5c-4b30-437d-a68e-44bc3ef970a5\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:51:09 crc kubenswrapper[5052]: I1203 12:51:09.139254 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/37d28f5c-4b30-437d-a68e-44bc3ef970a5-scripts\") pod \"glance-default-internal-api-0\" (UID: \"37d28f5c-4b30-437d-a68e-44bc3ef970a5\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:51:09 crc kubenswrapper[5052]: I1203 12:51:09.142507 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/37d28f5c-4b30-437d-a68e-44bc3ef970a5-internal-tls-certs\") pod \"glance-default-internal-api-0\" (UID: \"37d28f5c-4b30-437d-a68e-44bc3ef970a5\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:51:09 crc kubenswrapper[5052]: I1203 12:51:09.144034 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-6rz7r\" (UniqueName: \"kubernetes.io/projected/37d28f5c-4b30-437d-a68e-44bc3ef970a5-kube-api-access-6rz7r\") pod \"glance-default-internal-api-0\" (UID: \"37d28f5c-4b30-437d-a68e-44bc3ef970a5\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:51:09 crc kubenswrapper[5052]: I1203 12:51:09.145168 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/37d28f5c-4b30-437d-a68e-44bc3ef970a5-config-data\") pod \"glance-default-internal-api-0\" (UID: \"37d28f5c-4b30-437d-a68e-44bc3ef970a5\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:51:09 crc kubenswrapper[5052]: I1203 12:51:09.152963 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage06-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage06-crc\") pod \"glance-default-internal-api-0\" (UID: \"37d28f5c-4b30-437d-a68e-44bc3ef970a5\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:51:09 crc kubenswrapper[5052]: I1203 12:51:09.210891 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-db-sync-2b42x"] Dec 03 12:51:09 crc kubenswrapper[5052]: I1203 12:51:09.222836 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/9127a05d-9fe5-42b6-b54a-acc1290ad6ee-logs\") pod \"glance-default-external-api-0\" (UID: \"9127a05d-9fe5-42b6-b54a-acc1290ad6ee\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:51:09 crc kubenswrapper[5052]: I1203 12:51:09.222945 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/9127a05d-9fe5-42b6-b54a-acc1290ad6ee-public-tls-certs\") pod \"glance-default-external-api-0\" (UID: \"9127a05d-9fe5-42b6-b54a-acc1290ad6ee\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:51:09 crc kubenswrapper[5052]: I1203 12:51:09.223003 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage19-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage19-crc\") pod \"glance-default-external-api-0\" (UID: \"9127a05d-9fe5-42b6-b54a-acc1290ad6ee\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:51:09 crc kubenswrapper[5052]: I1203 12:51:09.223061 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/9127a05d-9fe5-42b6-b54a-acc1290ad6ee-httpd-run\") pod \"glance-default-external-api-0\" (UID: \"9127a05d-9fe5-42b6-b54a-acc1290ad6ee\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:51:09 crc kubenswrapper[5052]: I1203 12:51:09.223110 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/9127a05d-9fe5-42b6-b54a-acc1290ad6ee-config-data\") pod \"glance-default-external-api-0\" (UID: \"9127a05d-9fe5-42b6-b54a-acc1290ad6ee\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:51:09 crc kubenswrapper[5052]: I1203 12:51:09.223142 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-dntdw\" (UniqueName: \"kubernetes.io/projected/9127a05d-9fe5-42b6-b54a-acc1290ad6ee-kube-api-access-dntdw\") pod \"glance-default-external-api-0\" (UID: \"9127a05d-9fe5-42b6-b54a-acc1290ad6ee\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:51:09 crc kubenswrapper[5052]: I1203 12:51:09.223165 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/9127a05d-9fe5-42b6-b54a-acc1290ad6ee-scripts\") pod \"glance-default-external-api-0\" (UID: \"9127a05d-9fe5-42b6-b54a-acc1290ad6ee\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:51:09 crc kubenswrapper[5052]: I1203 12:51:09.223206 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9127a05d-9fe5-42b6-b54a-acc1290ad6ee-combined-ca-bundle\") pod \"glance-default-external-api-0\" (UID: \"9127a05d-9fe5-42b6-b54a-acc1290ad6ee\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:51:09 crc kubenswrapper[5052]: I1203 12:51:09.223590 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/9127a05d-9fe5-42b6-b54a-acc1290ad6ee-logs\") pod \"glance-default-external-api-0\" (UID: \"9127a05d-9fe5-42b6-b54a-acc1290ad6ee\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:51:09 crc kubenswrapper[5052]: I1203 12:51:09.223778 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage19-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage19-crc\") pod \"glance-default-external-api-0\" (UID: \"9127a05d-9fe5-42b6-b54a-acc1290ad6ee\") device mount path \"/mnt/openstack/pv19\"" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:51:09 crc kubenswrapper[5052]: I1203 12:51:09.224245 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/9127a05d-9fe5-42b6-b54a-acc1290ad6ee-httpd-run\") pod \"glance-default-external-api-0\" (UID: \"9127a05d-9fe5-42b6-b54a-acc1290ad6ee\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:51:09 crc kubenswrapper[5052]: I1203 12:51:09.226894 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/9127a05d-9fe5-42b6-b54a-acc1290ad6ee-public-tls-certs\") pod \"glance-default-external-api-0\" (UID: \"9127a05d-9fe5-42b6-b54a-acc1290ad6ee\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:51:09 crc kubenswrapper[5052]: I1203 12:51:09.227256 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9127a05d-9fe5-42b6-b54a-acc1290ad6ee-combined-ca-bundle\") pod \"glance-default-external-api-0\" (UID: \"9127a05d-9fe5-42b6-b54a-acc1290ad6ee\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:51:09 crc kubenswrapper[5052]: I1203 12:51:09.229502 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/9127a05d-9fe5-42b6-b54a-acc1290ad6ee-scripts\") pod \"glance-default-external-api-0\" (UID: \"9127a05d-9fe5-42b6-b54a-acc1290ad6ee\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:51:09 crc kubenswrapper[5052]: I1203 12:51:09.238182 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/9127a05d-9fe5-42b6-b54a-acc1290ad6ee-config-data\") pod \"glance-default-external-api-0\" (UID: \"9127a05d-9fe5-42b6-b54a-acc1290ad6ee\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:51:09 crc kubenswrapper[5052]: I1203 12:51:09.250574 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-dntdw\" (UniqueName: \"kubernetes.io/projected/9127a05d-9fe5-42b6-b54a-acc1290ad6ee-kube-api-access-dntdw\") pod \"glance-default-external-api-0\" (UID: \"9127a05d-9fe5-42b6-b54a-acc1290ad6ee\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:51:09 crc kubenswrapper[5052]: W1203 12:51:09.277174 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2fab7846_f3ad_4673_b813_36c20d4b7611.slice/crio-36fdcc55bb30d997fc9fb3be40b70b2d853ca78f5277296f366a15b6a0cd179f WatchSource:0}: Error finding container 36fdcc55bb30d997fc9fb3be40b70b2d853ca78f5277296f366a15b6a0cd179f: Status 404 returned error can't find the container with id 36fdcc55bb30d997fc9fb3be40b70b2d853ca78f5277296f366a15b6a0cd179f Dec 03 12:51:09 crc kubenswrapper[5052]: I1203 12:51:09.287021 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-db-sync-jxbww"] Dec 03 12:51:09 crc kubenswrapper[5052]: I1203 12:51:09.288762 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage19-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage19-crc\") pod \"glance-default-external-api-0\" (UID: \"9127a05d-9fe5-42b6-b54a-acc1290ad6ee\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:51:09 crc kubenswrapper[5052]: I1203 12:51:09.299347 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:51:09 crc kubenswrapper[5052]: I1203 12:51:09.302574 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-db-sync-rsmgs"] Dec 03 12:51:09 crc kubenswrapper[5052]: I1203 12:51:09.358363 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:51:09 crc kubenswrapper[5052]: I1203 12:51:09.803755 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-bootstrap-mgkjf" event={"ID":"83062467-67e1-4cab-b536-6edf3ea916b4","Type":"ContainerStarted","Data":"99aeae0bb50f3692e222705b7174440017ada583bd010ad7e324069e888cc242"} Dec 03 12:51:09 crc kubenswrapper[5052]: I1203 12:51:09.804272 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-bootstrap-mgkjf" event={"ID":"83062467-67e1-4cab-b536-6edf3ea916b4","Type":"ContainerStarted","Data":"40e91b05245027d186c016daca121b70283f9ca6be3d354d07c4aab376ef7d66"} Dec 03 12:51:09 crc kubenswrapper[5052]: I1203 12:51:09.810707 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"c5398d23-68fd-4a1c-af2b-d0a7500cc973","Type":"ContainerStarted","Data":"926bd638eec5757ee27004c7c18d3ceafab3a0dd3c091e38104af18db40f699a"} Dec 03 12:51:09 crc kubenswrapper[5052]: I1203 12:51:09.814916 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-db-sync-jxbww" event={"ID":"ffe1f27e-ba05-4da0-a02b-a73b3bf63bc3","Type":"ContainerStarted","Data":"174dc76c1d02fad9f57931bbb0c9542148e337ada369ca2b722e53cd15f06251"} Dec 03 12:51:09 crc kubenswrapper[5052]: I1203 12:51:09.814961 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-db-sync-jxbww" event={"ID":"ffe1f27e-ba05-4da0-a02b-a73b3bf63bc3","Type":"ContainerStarted","Data":"3e995213ff11ab92b3c3d6997def76b4e409dfb320b2e2d8ea6e50d98cd1e651"} Dec 03 12:51:09 crc kubenswrapper[5052]: I1203 12:51:09.819664 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-db-sync-2b42x" event={"ID":"8212a97d-b76a-4b60-b16b-914a4bbe5b9d","Type":"ContainerStarted","Data":"20c10208455f5a41166905ab0c82bb2f4b84aaf5e03bf97368f12b7272263c80"} Dec 03 12:51:09 crc kubenswrapper[5052]: I1203 12:51:09.829211 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-db-sync-rsmgs" event={"ID":"2fab7846-f3ad-4673-b813-36c20d4b7611","Type":"ContainerStarted","Data":"1a5bf9dc6916f04b426e2efc9f68b9c5e9ea7f21f2b45e2410c8d21dce7a7067"} Dec 03 12:51:09 crc kubenswrapper[5052]: I1203 12:51:09.829655 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-db-sync-rsmgs" event={"ID":"2fab7846-f3ad-4673-b813-36c20d4b7611","Type":"ContainerStarted","Data":"36fdcc55bb30d997fc9fb3be40b70b2d853ca78f5277296f366a15b6a0cd179f"} Dec 03 12:51:09 crc kubenswrapper[5052]: I1203 12:51:09.831688 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-db-sync-xbbsx" event={"ID":"468756d2-d46f-411a-a8ce-0c160d58c278","Type":"ContainerStarted","Data":"892e9cbb41eca0f11cf27ace38277c207623a5c2ba56eb827b340a0fc3b17b83"} Dec 03 12:51:09 crc kubenswrapper[5052]: I1203 12:51:09.831893 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-db-sync-xbbsx" event={"ID":"468756d2-d46f-411a-a8ce-0c160d58c278","Type":"ContainerStarted","Data":"908bf14390a4882004191c33dcf6574357daba67c6584dcf8b14650cdd533dbe"} Dec 03 12:51:09 crc kubenswrapper[5052]: I1203 12:51:09.851864 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/keystone-bootstrap-mgkjf" podStartSLOduration=2.851843721 podStartE2EDuration="2.851843721s" podCreationTimestamp="2025-12-03 12:51:07 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:51:09.844440519 +0000 UTC m=+5460.432682283" watchObservedRunningTime="2025-12-03 12:51:09.851843721 +0000 UTC m=+5460.440085485" Dec 03 12:51:09 crc kubenswrapper[5052]: I1203 12:51:09.864564 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:51:09 crc kubenswrapper[5052]: I1203 12:51:09.882300 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/neutron-db-sync-jxbww" podStartSLOduration=1.882281541 podStartE2EDuration="1.882281541s" podCreationTimestamp="2025-12-03 12:51:08 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:51:09.872375518 +0000 UTC m=+5460.460617292" watchObservedRunningTime="2025-12-03 12:51:09.882281541 +0000 UTC m=+5460.470523305" Dec 03 12:51:09 crc kubenswrapper[5052]: I1203 12:51:09.892026 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/barbican-db-sync-rsmgs" podStartSLOduration=1.8920062899999999 podStartE2EDuration="1.89200629s" podCreationTimestamp="2025-12-03 12:51:08 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:51:09.888368015 +0000 UTC m=+5460.476609779" watchObservedRunningTime="2025-12-03 12:51:09.89200629 +0000 UTC m=+5460.480248054" Dec 03 12:51:09 crc kubenswrapper[5052]: I1203 12:51:09.913364 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/placement-db-sync-xbbsx" podStartSLOduration=1.9133447700000001 podStartE2EDuration="1.91334477s" podCreationTimestamp="2025-12-03 12:51:08 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:51:09.910012504 +0000 UTC m=+5460.498254268" watchObservedRunningTime="2025-12-03 12:51:09.91334477 +0000 UTC m=+5460.501586534" Dec 03 12:51:10 crc kubenswrapper[5052]: I1203 12:51:10.085034 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:51:10 crc kubenswrapper[5052]: I1203 12:51:10.336682 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:51:10 crc kubenswrapper[5052]: I1203 12:51:10.382794 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:51:10 crc kubenswrapper[5052]: I1203 12:51:10.420287 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:51:10 crc kubenswrapper[5052]: I1203 12:51:10.872418 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"c5398d23-68fd-4a1c-af2b-d0a7500cc973","Type":"ContainerStarted","Data":"9cd573f6f7ac6e4412f9a87ff3c6e0e029d47569495ab458cd84bc64613ea8ee"} Dec 03 12:51:10 crc kubenswrapper[5052]: I1203 12:51:10.872797 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"c5398d23-68fd-4a1c-af2b-d0a7500cc973","Type":"ContainerStarted","Data":"4c1d3ef0fbbfab8064033b53eb26d8cd4924d5ae1aa7dde7609e5705ca05a452"} Dec 03 12:51:10 crc kubenswrapper[5052]: I1203 12:51:10.874432 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"9127a05d-9fe5-42b6-b54a-acc1290ad6ee","Type":"ContainerStarted","Data":"39c969cb5aad3addd4ac449d5141323b403d3bc2ca5badb45cdb3f27f04a63a7"} Dec 03 12:51:10 crc kubenswrapper[5052]: I1203 12:51:10.876050 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"37d28f5c-4b30-437d-a68e-44bc3ef970a5","Type":"ContainerStarted","Data":"2f4e35d6916133515e62e80be4fe6d8fb3da75a35545da6e36bf7983883f3c18"} Dec 03 12:51:10 crc kubenswrapper[5052]: I1203 12:51:10.876142 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"37d28f5c-4b30-437d-a68e-44bc3ef970a5","Type":"ContainerStarted","Data":"a473a626b80e988798ea23c1675faef75f8b22aa04e807387e5562079e4ff353"} Dec 03 12:51:10 crc kubenswrapper[5052]: I1203 12:51:10.877675 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-db-sync-2b42x" event={"ID":"8212a97d-b76a-4b60-b16b-914a4bbe5b9d","Type":"ContainerStarted","Data":"c424cb522d3b69b3e111498d4aa3be9ef046469f0fdba589af66d4e85ff8160a"} Dec 03 12:51:10 crc kubenswrapper[5052]: I1203 12:51:10.909259 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/cinder-db-sync-2b42x" podStartSLOduration=2.909240541 podStartE2EDuration="2.909240541s" podCreationTimestamp="2025-12-03 12:51:08 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:51:10.89558311 +0000 UTC m=+5461.483824874" watchObservedRunningTime="2025-12-03 12:51:10.909240541 +0000 UTC m=+5461.497482305" Dec 03 12:51:11 crc kubenswrapper[5052]: I1203 12:51:11.888504 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"37d28f5c-4b30-437d-a68e-44bc3ef970a5","Type":"ContainerStarted","Data":"2be6dd451ec5517bd6ad4162a739b4a81b981611340d97c9ea6683edb26868fd"} Dec 03 12:51:11 crc kubenswrapper[5052]: I1203 12:51:11.888643 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-internal-api-0" podUID="37d28f5c-4b30-437d-a68e-44bc3ef970a5" containerName="glance-log" containerID="cri-o://2f4e35d6916133515e62e80be4fe6d8fb3da75a35545da6e36bf7983883f3c18" gracePeriod=30 Dec 03 12:51:11 crc kubenswrapper[5052]: I1203 12:51:11.888679 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-internal-api-0" podUID="37d28f5c-4b30-437d-a68e-44bc3ef970a5" containerName="glance-httpd" containerID="cri-o://2be6dd451ec5517bd6ad4162a739b4a81b981611340d97c9ea6683edb26868fd" gracePeriod=30 Dec 03 12:51:11 crc kubenswrapper[5052]: I1203 12:51:11.898230 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"c5398d23-68fd-4a1c-af2b-d0a7500cc973","Type":"ContainerStarted","Data":"7f4892364d811e394b52fa07f11dbd2d15da8c9336b319b27a65bef091b1d9c5"} Dec 03 12:51:11 crc kubenswrapper[5052]: I1203 12:51:11.902287 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-external-api-0" podUID="9127a05d-9fe5-42b6-b54a-acc1290ad6ee" containerName="glance-log" containerID="cri-o://534fc02272cfcc81c822e99ea520e9a3371776af939322cdb1eed78b5425de86" gracePeriod=30 Dec 03 12:51:11 crc kubenswrapper[5052]: I1203 12:51:11.906111 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"9127a05d-9fe5-42b6-b54a-acc1290ad6ee","Type":"ContainerStarted","Data":"1bee40688c891ee07452290068931c4b106d7e46615e9a44b71d11a16bf3a098"} Dec 03 12:51:11 crc kubenswrapper[5052]: I1203 12:51:11.906149 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"9127a05d-9fe5-42b6-b54a-acc1290ad6ee","Type":"ContainerStarted","Data":"534fc02272cfcc81c822e99ea520e9a3371776af939322cdb1eed78b5425de86"} Dec 03 12:51:11 crc kubenswrapper[5052]: I1203 12:51:11.906204 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-external-api-0" podUID="9127a05d-9fe5-42b6-b54a-acc1290ad6ee" containerName="glance-httpd" containerID="cri-o://1bee40688c891ee07452290068931c4b106d7e46615e9a44b71d11a16bf3a098" gracePeriod=30 Dec 03 12:51:11 crc kubenswrapper[5052]: I1203 12:51:11.928255 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/glance-default-internal-api-0" podStartSLOduration=4.928239302 podStartE2EDuration="4.928239302s" podCreationTimestamp="2025-12-03 12:51:07 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:51:11.918187654 +0000 UTC m=+5462.506429418" watchObservedRunningTime="2025-12-03 12:51:11.928239302 +0000 UTC m=+5462.516481066" Dec 03 12:51:11 crc kubenswrapper[5052]: I1203 12:51:11.953839 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/glance-default-external-api-0" podStartSLOduration=4.953817333 podStartE2EDuration="4.953817333s" podCreationTimestamp="2025-12-03 12:51:07 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:51:11.939462773 +0000 UTC m=+5462.527704547" watchObservedRunningTime="2025-12-03 12:51:11.953817333 +0000 UTC m=+5462.542059097" Dec 03 12:51:12 crc kubenswrapper[5052]: I1203 12:51:12.552932 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:51:12 crc kubenswrapper[5052]: I1203 12:51:12.718163 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/9127a05d-9fe5-42b6-b54a-acc1290ad6ee-scripts\") pod \"9127a05d-9fe5-42b6-b54a-acc1290ad6ee\" (UID: \"9127a05d-9fe5-42b6-b54a-acc1290ad6ee\") " Dec 03 12:51:12 crc kubenswrapper[5052]: I1203 12:51:12.718227 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/9127a05d-9fe5-42b6-b54a-acc1290ad6ee-logs\") pod \"9127a05d-9fe5-42b6-b54a-acc1290ad6ee\" (UID: \"9127a05d-9fe5-42b6-b54a-acc1290ad6ee\") " Dec 03 12:51:12 crc kubenswrapper[5052]: I1203 12:51:12.718307 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dntdw\" (UniqueName: \"kubernetes.io/projected/9127a05d-9fe5-42b6-b54a-acc1290ad6ee-kube-api-access-dntdw\") pod \"9127a05d-9fe5-42b6-b54a-acc1290ad6ee\" (UID: \"9127a05d-9fe5-42b6-b54a-acc1290ad6ee\") " Dec 03 12:51:12 crc kubenswrapper[5052]: I1203 12:51:12.718328 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/9127a05d-9fe5-42b6-b54a-acc1290ad6ee-public-tls-certs\") pod \"9127a05d-9fe5-42b6-b54a-acc1290ad6ee\" (UID: \"9127a05d-9fe5-42b6-b54a-acc1290ad6ee\") " Dec 03 12:51:12 crc kubenswrapper[5052]: I1203 12:51:12.718382 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"glance\" (UniqueName: \"kubernetes.io/local-volume/local-storage19-crc\") pod \"9127a05d-9fe5-42b6-b54a-acc1290ad6ee\" (UID: \"9127a05d-9fe5-42b6-b54a-acc1290ad6ee\") " Dec 03 12:51:12 crc kubenswrapper[5052]: I1203 12:51:12.718403 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/9127a05d-9fe5-42b6-b54a-acc1290ad6ee-httpd-run\") pod \"9127a05d-9fe5-42b6-b54a-acc1290ad6ee\" (UID: \"9127a05d-9fe5-42b6-b54a-acc1290ad6ee\") " Dec 03 12:51:12 crc kubenswrapper[5052]: I1203 12:51:12.718454 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/9127a05d-9fe5-42b6-b54a-acc1290ad6ee-config-data\") pod \"9127a05d-9fe5-42b6-b54a-acc1290ad6ee\" (UID: \"9127a05d-9fe5-42b6-b54a-acc1290ad6ee\") " Dec 03 12:51:12 crc kubenswrapper[5052]: I1203 12:51:12.718492 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9127a05d-9fe5-42b6-b54a-acc1290ad6ee-combined-ca-bundle\") pod \"9127a05d-9fe5-42b6-b54a-acc1290ad6ee\" (UID: \"9127a05d-9fe5-42b6-b54a-acc1290ad6ee\") " Dec 03 12:51:12 crc kubenswrapper[5052]: I1203 12:51:12.718742 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9127a05d-9fe5-42b6-b54a-acc1290ad6ee-logs" (OuterVolumeSpecName: "logs") pod "9127a05d-9fe5-42b6-b54a-acc1290ad6ee" (UID: "9127a05d-9fe5-42b6-b54a-acc1290ad6ee"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:51:12 crc kubenswrapper[5052]: I1203 12:51:12.719083 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/9127a05d-9fe5-42b6-b54a-acc1290ad6ee-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:51:12 crc kubenswrapper[5052]: I1203 12:51:12.719321 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9127a05d-9fe5-42b6-b54a-acc1290ad6ee-httpd-run" (OuterVolumeSpecName: "httpd-run") pod "9127a05d-9fe5-42b6-b54a-acc1290ad6ee" (UID: "9127a05d-9fe5-42b6-b54a-acc1290ad6ee"). InnerVolumeSpecName "httpd-run". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:51:12 crc kubenswrapper[5052]: I1203 12:51:12.723928 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9127a05d-9fe5-42b6-b54a-acc1290ad6ee-scripts" (OuterVolumeSpecName: "scripts") pod "9127a05d-9fe5-42b6-b54a-acc1290ad6ee" (UID: "9127a05d-9fe5-42b6-b54a-acc1290ad6ee"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:51:12 crc kubenswrapper[5052]: I1203 12:51:12.726167 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9127a05d-9fe5-42b6-b54a-acc1290ad6ee-kube-api-access-dntdw" (OuterVolumeSpecName: "kube-api-access-dntdw") pod "9127a05d-9fe5-42b6-b54a-acc1290ad6ee" (UID: "9127a05d-9fe5-42b6-b54a-acc1290ad6ee"). InnerVolumeSpecName "kube-api-access-dntdw". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:51:12 crc kubenswrapper[5052]: I1203 12:51:12.726849 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage19-crc" (OuterVolumeSpecName: "glance") pod "9127a05d-9fe5-42b6-b54a-acc1290ad6ee" (UID: "9127a05d-9fe5-42b6-b54a-acc1290ad6ee"). InnerVolumeSpecName "local-storage19-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 12:51:12 crc kubenswrapper[5052]: I1203 12:51:12.748704 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9127a05d-9fe5-42b6-b54a-acc1290ad6ee-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "9127a05d-9fe5-42b6-b54a-acc1290ad6ee" (UID: "9127a05d-9fe5-42b6-b54a-acc1290ad6ee"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:51:12 crc kubenswrapper[5052]: I1203 12:51:12.764719 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9127a05d-9fe5-42b6-b54a-acc1290ad6ee-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "9127a05d-9fe5-42b6-b54a-acc1290ad6ee" (UID: "9127a05d-9fe5-42b6-b54a-acc1290ad6ee"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:51:12 crc kubenswrapper[5052]: I1203 12:51:12.766185 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9127a05d-9fe5-42b6-b54a-acc1290ad6ee-config-data" (OuterVolumeSpecName: "config-data") pod "9127a05d-9fe5-42b6-b54a-acc1290ad6ee" (UID: "9127a05d-9fe5-42b6-b54a-acc1290ad6ee"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:51:12 crc kubenswrapper[5052]: I1203 12:51:12.832160 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-dntdw\" (UniqueName: \"kubernetes.io/projected/9127a05d-9fe5-42b6-b54a-acc1290ad6ee-kube-api-access-dntdw\") on node \"crc\" DevicePath \"\"" Dec 03 12:51:12 crc kubenswrapper[5052]: I1203 12:51:12.832368 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/9127a05d-9fe5-42b6-b54a-acc1290ad6ee-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:51:12 crc kubenswrapper[5052]: I1203 12:51:12.832482 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage19-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage19-crc\") on node \"crc\" " Dec 03 12:51:12 crc kubenswrapper[5052]: I1203 12:51:12.832678 5052 reconciler_common.go:293] "Volume detached for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/9127a05d-9fe5-42b6-b54a-acc1290ad6ee-httpd-run\") on node \"crc\" DevicePath \"\"" Dec 03 12:51:12 crc kubenswrapper[5052]: I1203 12:51:12.833018 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/9127a05d-9fe5-42b6-b54a-acc1290ad6ee-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:51:12 crc kubenswrapper[5052]: I1203 12:51:12.833101 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9127a05d-9fe5-42b6-b54a-acc1290ad6ee-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:51:12 crc kubenswrapper[5052]: I1203 12:51:12.833175 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/9127a05d-9fe5-42b6-b54a-acc1290ad6ee-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:51:12 crc kubenswrapper[5052]: I1203 12:51:12.854912 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage19-crc" (UniqueName: "kubernetes.io/local-volume/local-storage19-crc") on node "crc" Dec 03 12:51:12 crc kubenswrapper[5052]: I1203 12:51:12.923292 5052 generic.go:334] "Generic (PLEG): container finished" podID="468756d2-d46f-411a-a8ce-0c160d58c278" containerID="892e9cbb41eca0f11cf27ace38277c207623a5c2ba56eb827b340a0fc3b17b83" exitCode=0 Dec 03 12:51:12 crc kubenswrapper[5052]: I1203 12:51:12.923368 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-db-sync-xbbsx" event={"ID":"468756d2-d46f-411a-a8ce-0c160d58c278","Type":"ContainerDied","Data":"892e9cbb41eca0f11cf27ace38277c207623a5c2ba56eb827b340a0fc3b17b83"} Dec 03 12:51:12 crc kubenswrapper[5052]: I1203 12:51:12.927654 5052 generic.go:334] "Generic (PLEG): container finished" podID="9127a05d-9fe5-42b6-b54a-acc1290ad6ee" containerID="1bee40688c891ee07452290068931c4b106d7e46615e9a44b71d11a16bf3a098" exitCode=143 Dec 03 12:51:12 crc kubenswrapper[5052]: I1203 12:51:12.927676 5052 generic.go:334] "Generic (PLEG): container finished" podID="9127a05d-9fe5-42b6-b54a-acc1290ad6ee" containerID="534fc02272cfcc81c822e99ea520e9a3371776af939322cdb1eed78b5425de86" exitCode=143 Dec 03 12:51:12 crc kubenswrapper[5052]: I1203 12:51:12.927710 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"9127a05d-9fe5-42b6-b54a-acc1290ad6ee","Type":"ContainerDied","Data":"1bee40688c891ee07452290068931c4b106d7e46615e9a44b71d11a16bf3a098"} Dec 03 12:51:12 crc kubenswrapper[5052]: I1203 12:51:12.927731 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"9127a05d-9fe5-42b6-b54a-acc1290ad6ee","Type":"ContainerDied","Data":"534fc02272cfcc81c822e99ea520e9a3371776af939322cdb1eed78b5425de86"} Dec 03 12:51:12 crc kubenswrapper[5052]: I1203 12:51:12.927741 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"9127a05d-9fe5-42b6-b54a-acc1290ad6ee","Type":"ContainerDied","Data":"39c969cb5aad3addd4ac449d5141323b403d3bc2ca5badb45cdb3f27f04a63a7"} Dec 03 12:51:12 crc kubenswrapper[5052]: I1203 12:51:12.927755 5052 scope.go:117] "RemoveContainer" containerID="1bee40688c891ee07452290068931c4b106d7e46615e9a44b71d11a16bf3a098" Dec 03 12:51:12 crc kubenswrapper[5052]: I1203 12:51:12.927866 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:51:12 crc kubenswrapper[5052]: I1203 12:51:12.945417 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage19-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage19-crc\") on node \"crc\" DevicePath \"\"" Dec 03 12:51:12 crc kubenswrapper[5052]: I1203 12:51:12.960025 5052 generic.go:334] "Generic (PLEG): container finished" podID="37d28f5c-4b30-437d-a68e-44bc3ef970a5" containerID="2be6dd451ec5517bd6ad4162a739b4a81b981611340d97c9ea6683edb26868fd" exitCode=0 Dec 03 12:51:12 crc kubenswrapper[5052]: I1203 12:51:12.960062 5052 generic.go:334] "Generic (PLEG): container finished" podID="37d28f5c-4b30-437d-a68e-44bc3ef970a5" containerID="2f4e35d6916133515e62e80be4fe6d8fb3da75a35545da6e36bf7983883f3c18" exitCode=143 Dec 03 12:51:12 crc kubenswrapper[5052]: I1203 12:51:12.960086 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"37d28f5c-4b30-437d-a68e-44bc3ef970a5","Type":"ContainerDied","Data":"2be6dd451ec5517bd6ad4162a739b4a81b981611340d97c9ea6683edb26868fd"} Dec 03 12:51:12 crc kubenswrapper[5052]: I1203 12:51:12.960116 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"37d28f5c-4b30-437d-a68e-44bc3ef970a5","Type":"ContainerDied","Data":"2f4e35d6916133515e62e80be4fe6d8fb3da75a35545da6e36bf7983883f3c18"} Dec 03 12:51:12 crc kubenswrapper[5052]: I1203 12:51:12.998424 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:51:13 crc kubenswrapper[5052]: I1203 12:51:13.006594 5052 scope.go:117] "RemoveContainer" containerID="534fc02272cfcc81c822e99ea520e9a3371776af939322cdb1eed78b5425de86" Dec 03 12:51:13 crc kubenswrapper[5052]: I1203 12:51:13.025596 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:51:13 crc kubenswrapper[5052]: I1203 12:51:13.051048 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:51:13 crc kubenswrapper[5052]: I1203 12:51:13.052717 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:51:13 crc kubenswrapper[5052]: E1203 12:51:13.053170 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="37d28f5c-4b30-437d-a68e-44bc3ef970a5" containerName="glance-httpd" Dec 03 12:51:13 crc kubenswrapper[5052]: I1203 12:51:13.053182 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="37d28f5c-4b30-437d-a68e-44bc3ef970a5" containerName="glance-httpd" Dec 03 12:51:13 crc kubenswrapper[5052]: E1203 12:51:13.053198 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="37d28f5c-4b30-437d-a68e-44bc3ef970a5" containerName="glance-log" Dec 03 12:51:13 crc kubenswrapper[5052]: I1203 12:51:13.053206 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="37d28f5c-4b30-437d-a68e-44bc3ef970a5" containerName="glance-log" Dec 03 12:51:13 crc kubenswrapper[5052]: E1203 12:51:13.053217 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="9127a05d-9fe5-42b6-b54a-acc1290ad6ee" containerName="glance-log" Dec 03 12:51:13 crc kubenswrapper[5052]: I1203 12:51:13.053223 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="9127a05d-9fe5-42b6-b54a-acc1290ad6ee" containerName="glance-log" Dec 03 12:51:13 crc kubenswrapper[5052]: E1203 12:51:13.053234 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="9127a05d-9fe5-42b6-b54a-acc1290ad6ee" containerName="glance-httpd" Dec 03 12:51:13 crc kubenswrapper[5052]: I1203 12:51:13.053240 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="9127a05d-9fe5-42b6-b54a-acc1290ad6ee" containerName="glance-httpd" Dec 03 12:51:13 crc kubenswrapper[5052]: I1203 12:51:13.053418 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="9127a05d-9fe5-42b6-b54a-acc1290ad6ee" containerName="glance-log" Dec 03 12:51:13 crc kubenswrapper[5052]: I1203 12:51:13.053441 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="37d28f5c-4b30-437d-a68e-44bc3ef970a5" containerName="glance-log" Dec 03 12:51:13 crc kubenswrapper[5052]: I1203 12:51:13.053453 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="9127a05d-9fe5-42b6-b54a-acc1290ad6ee" containerName="glance-httpd" Dec 03 12:51:13 crc kubenswrapper[5052]: I1203 12:51:13.053465 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="37d28f5c-4b30-437d-a68e-44bc3ef970a5" containerName="glance-httpd" Dec 03 12:51:13 crc kubenswrapper[5052]: I1203 12:51:13.054352 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:51:13 crc kubenswrapper[5052]: I1203 12:51:13.055830 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-default-external-config-data" Dec 03 12:51:13 crc kubenswrapper[5052]: I1203 12:51:13.056759 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-glance-default-public-svc" Dec 03 12:51:13 crc kubenswrapper[5052]: I1203 12:51:13.065238 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:51:13 crc kubenswrapper[5052]: I1203 12:51:13.099470 5052 scope.go:117] "RemoveContainer" containerID="1bee40688c891ee07452290068931c4b106d7e46615e9a44b71d11a16bf3a098" Dec 03 12:51:13 crc kubenswrapper[5052]: E1203 12:51:13.100136 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1bee40688c891ee07452290068931c4b106d7e46615e9a44b71d11a16bf3a098\": container with ID starting with 1bee40688c891ee07452290068931c4b106d7e46615e9a44b71d11a16bf3a098 not found: ID does not exist" containerID="1bee40688c891ee07452290068931c4b106d7e46615e9a44b71d11a16bf3a098" Dec 03 12:51:13 crc kubenswrapper[5052]: I1203 12:51:13.100169 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1bee40688c891ee07452290068931c4b106d7e46615e9a44b71d11a16bf3a098"} err="failed to get container status \"1bee40688c891ee07452290068931c4b106d7e46615e9a44b71d11a16bf3a098\": rpc error: code = NotFound desc = could not find container \"1bee40688c891ee07452290068931c4b106d7e46615e9a44b71d11a16bf3a098\": container with ID starting with 1bee40688c891ee07452290068931c4b106d7e46615e9a44b71d11a16bf3a098 not found: ID does not exist" Dec 03 12:51:13 crc kubenswrapper[5052]: I1203 12:51:13.100188 5052 scope.go:117] "RemoveContainer" containerID="534fc02272cfcc81c822e99ea520e9a3371776af939322cdb1eed78b5425de86" Dec 03 12:51:13 crc kubenswrapper[5052]: E1203 12:51:13.120839 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"534fc02272cfcc81c822e99ea520e9a3371776af939322cdb1eed78b5425de86\": container with ID starting with 534fc02272cfcc81c822e99ea520e9a3371776af939322cdb1eed78b5425de86 not found: ID does not exist" containerID="534fc02272cfcc81c822e99ea520e9a3371776af939322cdb1eed78b5425de86" Dec 03 12:51:13 crc kubenswrapper[5052]: I1203 12:51:13.121375 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"534fc02272cfcc81c822e99ea520e9a3371776af939322cdb1eed78b5425de86"} err="failed to get container status \"534fc02272cfcc81c822e99ea520e9a3371776af939322cdb1eed78b5425de86\": rpc error: code = NotFound desc = could not find container \"534fc02272cfcc81c822e99ea520e9a3371776af939322cdb1eed78b5425de86\": container with ID starting with 534fc02272cfcc81c822e99ea520e9a3371776af939322cdb1eed78b5425de86 not found: ID does not exist" Dec 03 12:51:13 crc kubenswrapper[5052]: I1203 12:51:13.121412 5052 scope.go:117] "RemoveContainer" containerID="1bee40688c891ee07452290068931c4b106d7e46615e9a44b71d11a16bf3a098" Dec 03 12:51:13 crc kubenswrapper[5052]: I1203 12:51:13.121990 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1bee40688c891ee07452290068931c4b106d7e46615e9a44b71d11a16bf3a098"} err="failed to get container status \"1bee40688c891ee07452290068931c4b106d7e46615e9a44b71d11a16bf3a098\": rpc error: code = NotFound desc = could not find container \"1bee40688c891ee07452290068931c4b106d7e46615e9a44b71d11a16bf3a098\": container with ID starting with 1bee40688c891ee07452290068931c4b106d7e46615e9a44b71d11a16bf3a098 not found: ID does not exist" Dec 03 12:51:13 crc kubenswrapper[5052]: I1203 12:51:13.122062 5052 scope.go:117] "RemoveContainer" containerID="534fc02272cfcc81c822e99ea520e9a3371776af939322cdb1eed78b5425de86" Dec 03 12:51:13 crc kubenswrapper[5052]: I1203 12:51:13.122647 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"534fc02272cfcc81c822e99ea520e9a3371776af939322cdb1eed78b5425de86"} err="failed to get container status \"534fc02272cfcc81c822e99ea520e9a3371776af939322cdb1eed78b5425de86\": rpc error: code = NotFound desc = could not find container \"534fc02272cfcc81c822e99ea520e9a3371776af939322cdb1eed78b5425de86\": container with ID starting with 534fc02272cfcc81c822e99ea520e9a3371776af939322cdb1eed78b5425de86 not found: ID does not exist" Dec 03 12:51:13 crc kubenswrapper[5052]: E1203 12:51:13.189683 5052 cadvisor_stats_provider.go:516] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9127a05d_9fe5_42b6_b54a_acc1290ad6ee.slice/crio-39c969cb5aad3addd4ac449d5141323b403d3bc2ca5badb45cdb3f27f04a63a7\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9127a05d_9fe5_42b6_b54a_acc1290ad6ee.slice\": RecentStats: unable to find data in memory cache]" Dec 03 12:51:13 crc kubenswrapper[5052]: I1203 12:51:13.249101 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/37d28f5c-4b30-437d-a68e-44bc3ef970a5-logs\") pod \"37d28f5c-4b30-437d-a68e-44bc3ef970a5\" (UID: \"37d28f5c-4b30-437d-a68e-44bc3ef970a5\") " Dec 03 12:51:13 crc kubenswrapper[5052]: I1203 12:51:13.249451 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/37d28f5c-4b30-437d-a68e-44bc3ef970a5-internal-tls-certs\") pod \"37d28f5c-4b30-437d-a68e-44bc3ef970a5\" (UID: \"37d28f5c-4b30-437d-a68e-44bc3ef970a5\") " Dec 03 12:51:13 crc kubenswrapper[5052]: I1203 12:51:13.249554 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"glance\" (UniqueName: \"kubernetes.io/local-volume/local-storage06-crc\") pod \"37d28f5c-4b30-437d-a68e-44bc3ef970a5\" (UID: \"37d28f5c-4b30-437d-a68e-44bc3ef970a5\") " Dec 03 12:51:13 crc kubenswrapper[5052]: I1203 12:51:13.249574 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/37d28f5c-4b30-437d-a68e-44bc3ef970a5-config-data\") pod \"37d28f5c-4b30-437d-a68e-44bc3ef970a5\" (UID: \"37d28f5c-4b30-437d-a68e-44bc3ef970a5\") " Dec 03 12:51:13 crc kubenswrapper[5052]: I1203 12:51:13.249627 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/37d28f5c-4b30-437d-a68e-44bc3ef970a5-httpd-run\") pod \"37d28f5c-4b30-437d-a68e-44bc3ef970a5\" (UID: \"37d28f5c-4b30-437d-a68e-44bc3ef970a5\") " Dec 03 12:51:13 crc kubenswrapper[5052]: I1203 12:51:13.249684 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37d28f5c-4b30-437d-a68e-44bc3ef970a5-logs" (OuterVolumeSpecName: "logs") pod "37d28f5c-4b30-437d-a68e-44bc3ef970a5" (UID: "37d28f5c-4b30-437d-a68e-44bc3ef970a5"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:51:13 crc kubenswrapper[5052]: I1203 12:51:13.249712 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/37d28f5c-4b30-437d-a68e-44bc3ef970a5-combined-ca-bundle\") pod \"37d28f5c-4b30-437d-a68e-44bc3ef970a5\" (UID: \"37d28f5c-4b30-437d-a68e-44bc3ef970a5\") " Dec 03 12:51:13 crc kubenswrapper[5052]: I1203 12:51:13.249752 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6rz7r\" (UniqueName: \"kubernetes.io/projected/37d28f5c-4b30-437d-a68e-44bc3ef970a5-kube-api-access-6rz7r\") pod \"37d28f5c-4b30-437d-a68e-44bc3ef970a5\" (UID: \"37d28f5c-4b30-437d-a68e-44bc3ef970a5\") " Dec 03 12:51:13 crc kubenswrapper[5052]: I1203 12:51:13.249808 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/37d28f5c-4b30-437d-a68e-44bc3ef970a5-scripts\") pod \"37d28f5c-4b30-437d-a68e-44bc3ef970a5\" (UID: \"37d28f5c-4b30-437d-a68e-44bc3ef970a5\") " Dec 03 12:51:13 crc kubenswrapper[5052]: I1203 12:51:13.250095 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37d28f5c-4b30-437d-a68e-44bc3ef970a5-httpd-run" (OuterVolumeSpecName: "httpd-run") pod "37d28f5c-4b30-437d-a68e-44bc3ef970a5" (UID: "37d28f5c-4b30-437d-a68e-44bc3ef970a5"). InnerVolumeSpecName "httpd-run". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:51:13 crc kubenswrapper[5052]: I1203 12:51:13.250374 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/cd023d37-ce83-4ca4-ab52-f89e94d0b721-logs\") pod \"glance-default-external-api-0\" (UID: \"cd023d37-ce83-4ca4-ab52-f89e94d0b721\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:51:13 crc kubenswrapper[5052]: I1203 12:51:13.250424 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/cd023d37-ce83-4ca4-ab52-f89e94d0b721-combined-ca-bundle\") pod \"glance-default-external-api-0\" (UID: \"cd023d37-ce83-4ca4-ab52-f89e94d0b721\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:51:13 crc kubenswrapper[5052]: I1203 12:51:13.250482 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/cd023d37-ce83-4ca4-ab52-f89e94d0b721-scripts\") pod \"glance-default-external-api-0\" (UID: \"cd023d37-ce83-4ca4-ab52-f89e94d0b721\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:51:13 crc kubenswrapper[5052]: I1203 12:51:13.250525 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/cd023d37-ce83-4ca4-ab52-f89e94d0b721-public-tls-certs\") pod \"glance-default-external-api-0\" (UID: \"cd023d37-ce83-4ca4-ab52-f89e94d0b721\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:51:13 crc kubenswrapper[5052]: I1203 12:51:13.250635 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/cd023d37-ce83-4ca4-ab52-f89e94d0b721-config-data\") pod \"glance-default-external-api-0\" (UID: \"cd023d37-ce83-4ca4-ab52-f89e94d0b721\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:51:13 crc kubenswrapper[5052]: I1203 12:51:13.250804 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage19-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage19-crc\") pod \"glance-default-external-api-0\" (UID: \"cd023d37-ce83-4ca4-ab52-f89e94d0b721\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:51:13 crc kubenswrapper[5052]: I1203 12:51:13.250942 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/cd023d37-ce83-4ca4-ab52-f89e94d0b721-httpd-run\") pod \"glance-default-external-api-0\" (UID: \"cd023d37-ce83-4ca4-ab52-f89e94d0b721\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:51:13 crc kubenswrapper[5052]: I1203 12:51:13.251197 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bkssv\" (UniqueName: \"kubernetes.io/projected/cd023d37-ce83-4ca4-ab52-f89e94d0b721-kube-api-access-bkssv\") pod \"glance-default-external-api-0\" (UID: \"cd023d37-ce83-4ca4-ab52-f89e94d0b721\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:51:13 crc kubenswrapper[5052]: I1203 12:51:13.251409 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/37d28f5c-4b30-437d-a68e-44bc3ef970a5-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:51:13 crc kubenswrapper[5052]: I1203 12:51:13.251497 5052 reconciler_common.go:293] "Volume detached for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/37d28f5c-4b30-437d-a68e-44bc3ef970a5-httpd-run\") on node \"crc\" DevicePath \"\"" Dec 03 12:51:13 crc kubenswrapper[5052]: I1203 12:51:13.265950 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage06-crc" (OuterVolumeSpecName: "glance") pod "37d28f5c-4b30-437d-a68e-44bc3ef970a5" (UID: "37d28f5c-4b30-437d-a68e-44bc3ef970a5"). InnerVolumeSpecName "local-storage06-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 12:51:13 crc kubenswrapper[5052]: I1203 12:51:13.272712 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/37d28f5c-4b30-437d-a68e-44bc3ef970a5-kube-api-access-6rz7r" (OuterVolumeSpecName: "kube-api-access-6rz7r") pod "37d28f5c-4b30-437d-a68e-44bc3ef970a5" (UID: "37d28f5c-4b30-437d-a68e-44bc3ef970a5"). InnerVolumeSpecName "kube-api-access-6rz7r". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:51:13 crc kubenswrapper[5052]: I1203 12:51:13.288937 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/37d28f5c-4b30-437d-a68e-44bc3ef970a5-scripts" (OuterVolumeSpecName: "scripts") pod "37d28f5c-4b30-437d-a68e-44bc3ef970a5" (UID: "37d28f5c-4b30-437d-a68e-44bc3ef970a5"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:51:13 crc kubenswrapper[5052]: I1203 12:51:13.297907 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/37d28f5c-4b30-437d-a68e-44bc3ef970a5-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "37d28f5c-4b30-437d-a68e-44bc3ef970a5" (UID: "37d28f5c-4b30-437d-a68e-44bc3ef970a5"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:51:13 crc kubenswrapper[5052]: I1203 12:51:13.324846 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/37d28f5c-4b30-437d-a68e-44bc3ef970a5-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "37d28f5c-4b30-437d-a68e-44bc3ef970a5" (UID: "37d28f5c-4b30-437d-a68e-44bc3ef970a5"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:51:13 crc kubenswrapper[5052]: I1203 12:51:13.353495 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage19-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage19-crc\") pod \"glance-default-external-api-0\" (UID: \"cd023d37-ce83-4ca4-ab52-f89e94d0b721\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:51:13 crc kubenswrapper[5052]: I1203 12:51:13.353819 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/cd023d37-ce83-4ca4-ab52-f89e94d0b721-httpd-run\") pod \"glance-default-external-api-0\" (UID: \"cd023d37-ce83-4ca4-ab52-f89e94d0b721\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:51:13 crc kubenswrapper[5052]: I1203 12:51:13.353949 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-bkssv\" (UniqueName: \"kubernetes.io/projected/cd023d37-ce83-4ca4-ab52-f89e94d0b721-kube-api-access-bkssv\") pod \"glance-default-external-api-0\" (UID: \"cd023d37-ce83-4ca4-ab52-f89e94d0b721\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:51:13 crc kubenswrapper[5052]: I1203 12:51:13.354085 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/cd023d37-ce83-4ca4-ab52-f89e94d0b721-logs\") pod \"glance-default-external-api-0\" (UID: \"cd023d37-ce83-4ca4-ab52-f89e94d0b721\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:51:13 crc kubenswrapper[5052]: I1203 12:51:13.354165 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/cd023d37-ce83-4ca4-ab52-f89e94d0b721-combined-ca-bundle\") pod \"glance-default-external-api-0\" (UID: \"cd023d37-ce83-4ca4-ab52-f89e94d0b721\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:51:13 crc kubenswrapper[5052]: I1203 12:51:13.354250 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/cd023d37-ce83-4ca4-ab52-f89e94d0b721-scripts\") pod \"glance-default-external-api-0\" (UID: \"cd023d37-ce83-4ca4-ab52-f89e94d0b721\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:51:13 crc kubenswrapper[5052]: I1203 12:51:13.354324 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/cd023d37-ce83-4ca4-ab52-f89e94d0b721-public-tls-certs\") pod \"glance-default-external-api-0\" (UID: \"cd023d37-ce83-4ca4-ab52-f89e94d0b721\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:51:13 crc kubenswrapper[5052]: I1203 12:51:13.354398 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/cd023d37-ce83-4ca4-ab52-f89e94d0b721-config-data\") pod \"glance-default-external-api-0\" (UID: \"cd023d37-ce83-4ca4-ab52-f89e94d0b721\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:51:13 crc kubenswrapper[5052]: I1203 12:51:13.354496 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/cd023d37-ce83-4ca4-ab52-f89e94d0b721-httpd-run\") pod \"glance-default-external-api-0\" (UID: \"cd023d37-ce83-4ca4-ab52-f89e94d0b721\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:51:13 crc kubenswrapper[5052]: I1203 12:51:13.354528 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/37d28f5c-4b30-437d-a68e-44bc3ef970a5-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:51:13 crc kubenswrapper[5052]: I1203 12:51:13.354611 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage06-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage06-crc\") on node \"crc\" " Dec 03 12:51:13 crc kubenswrapper[5052]: I1203 12:51:13.354632 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/37d28f5c-4b30-437d-a68e-44bc3ef970a5-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:51:13 crc kubenswrapper[5052]: I1203 12:51:13.354653 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-6rz7r\" (UniqueName: \"kubernetes.io/projected/37d28f5c-4b30-437d-a68e-44bc3ef970a5-kube-api-access-6rz7r\") on node \"crc\" DevicePath \"\"" Dec 03 12:51:13 crc kubenswrapper[5052]: I1203 12:51:13.354669 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/37d28f5c-4b30-437d-a68e-44bc3ef970a5-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:51:13 crc kubenswrapper[5052]: I1203 12:51:13.354629 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/cd023d37-ce83-4ca4-ab52-f89e94d0b721-logs\") pod \"glance-default-external-api-0\" (UID: \"cd023d37-ce83-4ca4-ab52-f89e94d0b721\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:51:13 crc kubenswrapper[5052]: I1203 12:51:13.355392 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage19-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage19-crc\") pod \"glance-default-external-api-0\" (UID: \"cd023d37-ce83-4ca4-ab52-f89e94d0b721\") device mount path \"/mnt/openstack/pv19\"" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:51:13 crc kubenswrapper[5052]: I1203 12:51:13.360381 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/cd023d37-ce83-4ca4-ab52-f89e94d0b721-scripts\") pod \"glance-default-external-api-0\" (UID: \"cd023d37-ce83-4ca4-ab52-f89e94d0b721\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:51:13 crc kubenswrapper[5052]: I1203 12:51:13.360454 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/cd023d37-ce83-4ca4-ab52-f89e94d0b721-public-tls-certs\") pod \"glance-default-external-api-0\" (UID: \"cd023d37-ce83-4ca4-ab52-f89e94d0b721\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:51:13 crc kubenswrapper[5052]: I1203 12:51:13.366807 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/cd023d37-ce83-4ca4-ab52-f89e94d0b721-combined-ca-bundle\") pod \"glance-default-external-api-0\" (UID: \"cd023d37-ce83-4ca4-ab52-f89e94d0b721\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:51:13 crc kubenswrapper[5052]: I1203 12:51:13.368712 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/cd023d37-ce83-4ca4-ab52-f89e94d0b721-config-data\") pod \"glance-default-external-api-0\" (UID: \"cd023d37-ce83-4ca4-ab52-f89e94d0b721\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:51:13 crc kubenswrapper[5052]: I1203 12:51:13.374833 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-bkssv\" (UniqueName: \"kubernetes.io/projected/cd023d37-ce83-4ca4-ab52-f89e94d0b721-kube-api-access-bkssv\") pod \"glance-default-external-api-0\" (UID: \"cd023d37-ce83-4ca4-ab52-f89e94d0b721\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:51:13 crc kubenswrapper[5052]: I1203 12:51:13.377063 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/37d28f5c-4b30-437d-a68e-44bc3ef970a5-config-data" (OuterVolumeSpecName: "config-data") pod "37d28f5c-4b30-437d-a68e-44bc3ef970a5" (UID: "37d28f5c-4b30-437d-a68e-44bc3ef970a5"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:51:13 crc kubenswrapper[5052]: I1203 12:51:13.389913 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage06-crc" (UniqueName: "kubernetes.io/local-volume/local-storage06-crc") on node "crc" Dec 03 12:51:13 crc kubenswrapper[5052]: I1203 12:51:13.406479 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage19-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage19-crc\") pod \"glance-default-external-api-0\" (UID: \"cd023d37-ce83-4ca4-ab52-f89e94d0b721\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:51:13 crc kubenswrapper[5052]: I1203 12:51:13.453945 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:51:13 crc kubenswrapper[5052]: I1203 12:51:13.455640 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage06-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage06-crc\") on node \"crc\" DevicePath \"\"" Dec 03 12:51:13 crc kubenswrapper[5052]: I1203 12:51:13.455666 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/37d28f5c-4b30-437d-a68e-44bc3ef970a5-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:51:13 crc kubenswrapper[5052]: I1203 12:51:13.688857 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9127a05d-9fe5-42b6-b54a-acc1290ad6ee" path="/var/lib/kubelet/pods/9127a05d-9fe5-42b6-b54a-acc1290ad6ee/volumes" Dec 03 12:51:13 crc kubenswrapper[5052]: I1203 12:51:13.896650 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:51:13 crc kubenswrapper[5052]: I1203 12:51:13.971439 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"cd023d37-ce83-4ca4-ab52-f89e94d0b721","Type":"ContainerStarted","Data":"9bff0490e9efac8c13794ea92cd1e5524942afd11c37264a3f09ab026012843e"} Dec 03 12:51:13 crc kubenswrapper[5052]: I1203 12:51:13.976742 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"37d28f5c-4b30-437d-a68e-44bc3ef970a5","Type":"ContainerDied","Data":"a473a626b80e988798ea23c1675faef75f8b22aa04e807387e5562079e4ff353"} Dec 03 12:51:13 crc kubenswrapper[5052]: I1203 12:51:13.976755 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:51:13 crc kubenswrapper[5052]: I1203 12:51:13.976799 5052 scope.go:117] "RemoveContainer" containerID="2be6dd451ec5517bd6ad4162a739b4a81b981611340d97c9ea6683edb26868fd" Dec 03 12:51:13 crc kubenswrapper[5052]: I1203 12:51:13.981058 5052 generic.go:334] "Generic (PLEG): container finished" podID="2fab7846-f3ad-4673-b813-36c20d4b7611" containerID="1a5bf9dc6916f04b426e2efc9f68b9c5e9ea7f21f2b45e2410c8d21dce7a7067" exitCode=0 Dec 03 12:51:13 crc kubenswrapper[5052]: I1203 12:51:13.981111 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-db-sync-rsmgs" event={"ID":"2fab7846-f3ad-4673-b813-36c20d4b7611","Type":"ContainerDied","Data":"1a5bf9dc6916f04b426e2efc9f68b9c5e9ea7f21f2b45e2410c8d21dce7a7067"} Dec 03 12:51:13 crc kubenswrapper[5052]: I1203 12:51:13.983793 5052 generic.go:334] "Generic (PLEG): container finished" podID="83062467-67e1-4cab-b536-6edf3ea916b4" containerID="99aeae0bb50f3692e222705b7174440017ada583bd010ad7e324069e888cc242" exitCode=0 Dec 03 12:51:13 crc kubenswrapper[5052]: I1203 12:51:13.983859 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-bootstrap-mgkjf" event={"ID":"83062467-67e1-4cab-b536-6edf3ea916b4","Type":"ContainerDied","Data":"99aeae0bb50f3692e222705b7174440017ada583bd010ad7e324069e888cc242"} Dec 03 12:51:13 crc kubenswrapper[5052]: I1203 12:51:13.990567 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"c5398d23-68fd-4a1c-af2b-d0a7500cc973","Type":"ContainerStarted","Data":"4daedd289d651c6e85f6a5a2c3b29d4274a156f001635aa0c260ea51ca5ecc5d"} Dec 03 12:51:13 crc kubenswrapper[5052]: I1203 12:51:13.990658 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="c5398d23-68fd-4a1c-af2b-d0a7500cc973" containerName="ceilometer-central-agent" containerID="cri-o://4c1d3ef0fbbfab8064033b53eb26d8cd4924d5ae1aa7dde7609e5705ca05a452" gracePeriod=30 Dec 03 12:51:13 crc kubenswrapper[5052]: I1203 12:51:13.990748 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="c5398d23-68fd-4a1c-af2b-d0a7500cc973" containerName="proxy-httpd" containerID="cri-o://4daedd289d651c6e85f6a5a2c3b29d4274a156f001635aa0c260ea51ca5ecc5d" gracePeriod=30 Dec 03 12:51:13 crc kubenswrapper[5052]: I1203 12:51:13.990794 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="c5398d23-68fd-4a1c-af2b-d0a7500cc973" containerName="sg-core" containerID="cri-o://7f4892364d811e394b52fa07f11dbd2d15da8c9336b319b27a65bef091b1d9c5" gracePeriod=30 Dec 03 12:51:13 crc kubenswrapper[5052]: I1203 12:51:13.990851 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="c5398d23-68fd-4a1c-af2b-d0a7500cc973" containerName="ceilometer-notification-agent" containerID="cri-o://9cd573f6f7ac6e4412f9a87ff3c6e0e029d47569495ab458cd84bc64613ea8ee" gracePeriod=30 Dec 03 12:51:14 crc kubenswrapper[5052]: I1203 12:51:14.006045 5052 scope.go:117] "RemoveContainer" containerID="2f4e35d6916133515e62e80be4fe6d8fb3da75a35545da6e36bf7983883f3c18" Dec 03 12:51:14 crc kubenswrapper[5052]: I1203 12:51:14.027229 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/ceilometer-0" podStartSLOduration=2.236521932 podStartE2EDuration="6.027210429s" podCreationTimestamp="2025-12-03 12:51:08 +0000 UTC" firstStartedPulling="2025-12-03 12:51:09.063363872 +0000 UTC m=+5459.651605646" lastFinishedPulling="2025-12-03 12:51:12.854052369 +0000 UTC m=+5463.442294143" observedRunningTime="2025-12-03 12:51:14.017897733 +0000 UTC m=+5464.606139497" watchObservedRunningTime="2025-12-03 12:51:14.027210429 +0000 UTC m=+5464.615452193" Dec 03 12:51:14 crc kubenswrapper[5052]: I1203 12:51:14.082683 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:51:14 crc kubenswrapper[5052]: I1203 12:51:14.096338 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:51:14 crc kubenswrapper[5052]: I1203 12:51:14.132943 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:51:14 crc kubenswrapper[5052]: I1203 12:51:14.136060 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:51:14 crc kubenswrapper[5052]: I1203 12:51:14.153065 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:51:14 crc kubenswrapper[5052]: I1203 12:51:14.154271 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-glance-default-internal-svc" Dec 03 12:51:14 crc kubenswrapper[5052]: I1203 12:51:14.154670 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-default-internal-config-data" Dec 03 12:51:14 crc kubenswrapper[5052]: I1203 12:51:14.268898 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c266x\" (UniqueName: \"kubernetes.io/projected/fc57d1ed-34a0-42b5-b1be-5bf914492701-kube-api-access-c266x\") pod \"glance-default-internal-api-0\" (UID: \"fc57d1ed-34a0-42b5-b1be-5bf914492701\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:51:14 crc kubenswrapper[5052]: I1203 12:51:14.268987 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/fc57d1ed-34a0-42b5-b1be-5bf914492701-internal-tls-certs\") pod \"glance-default-internal-api-0\" (UID: \"fc57d1ed-34a0-42b5-b1be-5bf914492701\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:51:14 crc kubenswrapper[5052]: I1203 12:51:14.269031 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/fc57d1ed-34a0-42b5-b1be-5bf914492701-httpd-run\") pod \"glance-default-internal-api-0\" (UID: \"fc57d1ed-34a0-42b5-b1be-5bf914492701\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:51:14 crc kubenswrapper[5052]: I1203 12:51:14.269080 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/fc57d1ed-34a0-42b5-b1be-5bf914492701-logs\") pod \"glance-default-internal-api-0\" (UID: \"fc57d1ed-34a0-42b5-b1be-5bf914492701\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:51:14 crc kubenswrapper[5052]: I1203 12:51:14.269102 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/fc57d1ed-34a0-42b5-b1be-5bf914492701-config-data\") pod \"glance-default-internal-api-0\" (UID: \"fc57d1ed-34a0-42b5-b1be-5bf914492701\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:51:14 crc kubenswrapper[5052]: I1203 12:51:14.269115 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/fc57d1ed-34a0-42b5-b1be-5bf914492701-scripts\") pod \"glance-default-internal-api-0\" (UID: \"fc57d1ed-34a0-42b5-b1be-5bf914492701\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:51:14 crc kubenswrapper[5052]: I1203 12:51:14.269142 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage06-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage06-crc\") pod \"glance-default-internal-api-0\" (UID: \"fc57d1ed-34a0-42b5-b1be-5bf914492701\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:51:14 crc kubenswrapper[5052]: I1203 12:51:14.269175 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/fc57d1ed-34a0-42b5-b1be-5bf914492701-combined-ca-bundle\") pod \"glance-default-internal-api-0\" (UID: \"fc57d1ed-34a0-42b5-b1be-5bf914492701\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:51:14 crc kubenswrapper[5052]: I1203 12:51:14.370880 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/fc57d1ed-34a0-42b5-b1be-5bf914492701-httpd-run\") pod \"glance-default-internal-api-0\" (UID: \"fc57d1ed-34a0-42b5-b1be-5bf914492701\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:51:14 crc kubenswrapper[5052]: I1203 12:51:14.370974 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/fc57d1ed-34a0-42b5-b1be-5bf914492701-logs\") pod \"glance-default-internal-api-0\" (UID: \"fc57d1ed-34a0-42b5-b1be-5bf914492701\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:51:14 crc kubenswrapper[5052]: I1203 12:51:14.371011 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/fc57d1ed-34a0-42b5-b1be-5bf914492701-config-data\") pod \"glance-default-internal-api-0\" (UID: \"fc57d1ed-34a0-42b5-b1be-5bf914492701\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:51:14 crc kubenswrapper[5052]: I1203 12:51:14.371035 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/fc57d1ed-34a0-42b5-b1be-5bf914492701-scripts\") pod \"glance-default-internal-api-0\" (UID: \"fc57d1ed-34a0-42b5-b1be-5bf914492701\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:51:14 crc kubenswrapper[5052]: I1203 12:51:14.371065 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage06-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage06-crc\") pod \"glance-default-internal-api-0\" (UID: \"fc57d1ed-34a0-42b5-b1be-5bf914492701\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:51:14 crc kubenswrapper[5052]: I1203 12:51:14.371093 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/fc57d1ed-34a0-42b5-b1be-5bf914492701-combined-ca-bundle\") pod \"glance-default-internal-api-0\" (UID: \"fc57d1ed-34a0-42b5-b1be-5bf914492701\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:51:14 crc kubenswrapper[5052]: I1203 12:51:14.371138 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-c266x\" (UniqueName: \"kubernetes.io/projected/fc57d1ed-34a0-42b5-b1be-5bf914492701-kube-api-access-c266x\") pod \"glance-default-internal-api-0\" (UID: \"fc57d1ed-34a0-42b5-b1be-5bf914492701\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:51:14 crc kubenswrapper[5052]: I1203 12:51:14.371220 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/fc57d1ed-34a0-42b5-b1be-5bf914492701-internal-tls-certs\") pod \"glance-default-internal-api-0\" (UID: \"fc57d1ed-34a0-42b5-b1be-5bf914492701\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:51:14 crc kubenswrapper[5052]: I1203 12:51:14.371366 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/fc57d1ed-34a0-42b5-b1be-5bf914492701-httpd-run\") pod \"glance-default-internal-api-0\" (UID: \"fc57d1ed-34a0-42b5-b1be-5bf914492701\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:51:14 crc kubenswrapper[5052]: I1203 12:51:14.372559 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/fc57d1ed-34a0-42b5-b1be-5bf914492701-logs\") pod \"glance-default-internal-api-0\" (UID: \"fc57d1ed-34a0-42b5-b1be-5bf914492701\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:51:14 crc kubenswrapper[5052]: I1203 12:51:14.373303 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage06-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage06-crc\") pod \"glance-default-internal-api-0\" (UID: \"fc57d1ed-34a0-42b5-b1be-5bf914492701\") device mount path \"/mnt/openstack/pv06\"" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:51:14 crc kubenswrapper[5052]: I1203 12:51:14.380379 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/fc57d1ed-34a0-42b5-b1be-5bf914492701-scripts\") pod \"glance-default-internal-api-0\" (UID: \"fc57d1ed-34a0-42b5-b1be-5bf914492701\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:51:14 crc kubenswrapper[5052]: I1203 12:51:14.381361 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/fc57d1ed-34a0-42b5-b1be-5bf914492701-combined-ca-bundle\") pod \"glance-default-internal-api-0\" (UID: \"fc57d1ed-34a0-42b5-b1be-5bf914492701\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:51:14 crc kubenswrapper[5052]: I1203 12:51:14.381780 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/fc57d1ed-34a0-42b5-b1be-5bf914492701-config-data\") pod \"glance-default-internal-api-0\" (UID: \"fc57d1ed-34a0-42b5-b1be-5bf914492701\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:51:14 crc kubenswrapper[5052]: I1203 12:51:14.382715 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/fc57d1ed-34a0-42b5-b1be-5bf914492701-internal-tls-certs\") pod \"glance-default-internal-api-0\" (UID: \"fc57d1ed-34a0-42b5-b1be-5bf914492701\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:51:14 crc kubenswrapper[5052]: I1203 12:51:14.389854 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-c266x\" (UniqueName: \"kubernetes.io/projected/fc57d1ed-34a0-42b5-b1be-5bf914492701-kube-api-access-c266x\") pod \"glance-default-internal-api-0\" (UID: \"fc57d1ed-34a0-42b5-b1be-5bf914492701\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:51:14 crc kubenswrapper[5052]: I1203 12:51:14.409146 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage06-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage06-crc\") pod \"glance-default-internal-api-0\" (UID: \"fc57d1ed-34a0-42b5-b1be-5bf914492701\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:51:14 crc kubenswrapper[5052]: I1203 12:51:14.534061 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:51:14 crc kubenswrapper[5052]: I1203 12:51:14.572010 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-db-sync-xbbsx" Dec 03 12:51:14 crc kubenswrapper[5052]: I1203 12:51:14.675196 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/468756d2-d46f-411a-a8ce-0c160d58c278-config-data\") pod \"468756d2-d46f-411a-a8ce-0c160d58c278\" (UID: \"468756d2-d46f-411a-a8ce-0c160d58c278\") " Dec 03 12:51:14 crc kubenswrapper[5052]: I1203 12:51:14.675537 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/468756d2-d46f-411a-a8ce-0c160d58c278-scripts\") pod \"468756d2-d46f-411a-a8ce-0c160d58c278\" (UID: \"468756d2-d46f-411a-a8ce-0c160d58c278\") " Dec 03 12:51:14 crc kubenswrapper[5052]: I1203 12:51:14.675714 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/468756d2-d46f-411a-a8ce-0c160d58c278-logs\") pod \"468756d2-d46f-411a-a8ce-0c160d58c278\" (UID: \"468756d2-d46f-411a-a8ce-0c160d58c278\") " Dec 03 12:51:14 crc kubenswrapper[5052]: I1203 12:51:14.675770 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f56tk\" (UniqueName: \"kubernetes.io/projected/468756d2-d46f-411a-a8ce-0c160d58c278-kube-api-access-f56tk\") pod \"468756d2-d46f-411a-a8ce-0c160d58c278\" (UID: \"468756d2-d46f-411a-a8ce-0c160d58c278\") " Dec 03 12:51:14 crc kubenswrapper[5052]: I1203 12:51:14.675810 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/468756d2-d46f-411a-a8ce-0c160d58c278-combined-ca-bundle\") pod \"468756d2-d46f-411a-a8ce-0c160d58c278\" (UID: \"468756d2-d46f-411a-a8ce-0c160d58c278\") " Dec 03 12:51:14 crc kubenswrapper[5052]: I1203 12:51:14.678116 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/468756d2-d46f-411a-a8ce-0c160d58c278-logs" (OuterVolumeSpecName: "logs") pod "468756d2-d46f-411a-a8ce-0c160d58c278" (UID: "468756d2-d46f-411a-a8ce-0c160d58c278"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:51:14 crc kubenswrapper[5052]: I1203 12:51:14.685467 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/468756d2-d46f-411a-a8ce-0c160d58c278-kube-api-access-f56tk" (OuterVolumeSpecName: "kube-api-access-f56tk") pod "468756d2-d46f-411a-a8ce-0c160d58c278" (UID: "468756d2-d46f-411a-a8ce-0c160d58c278"). InnerVolumeSpecName "kube-api-access-f56tk". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:51:14 crc kubenswrapper[5052]: I1203 12:51:14.690138 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/468756d2-d46f-411a-a8ce-0c160d58c278-scripts" (OuterVolumeSpecName: "scripts") pod "468756d2-d46f-411a-a8ce-0c160d58c278" (UID: "468756d2-d46f-411a-a8ce-0c160d58c278"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:51:14 crc kubenswrapper[5052]: I1203 12:51:14.710927 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/468756d2-d46f-411a-a8ce-0c160d58c278-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "468756d2-d46f-411a-a8ce-0c160d58c278" (UID: "468756d2-d46f-411a-a8ce-0c160d58c278"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:51:14 crc kubenswrapper[5052]: I1203 12:51:14.715458 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/468756d2-d46f-411a-a8ce-0c160d58c278-config-data" (OuterVolumeSpecName: "config-data") pod "468756d2-d46f-411a-a8ce-0c160d58c278" (UID: "468756d2-d46f-411a-a8ce-0c160d58c278"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:51:14 crc kubenswrapper[5052]: I1203 12:51:14.778289 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/468756d2-d46f-411a-a8ce-0c160d58c278-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:51:14 crc kubenswrapper[5052]: I1203 12:51:14.778313 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-f56tk\" (UniqueName: \"kubernetes.io/projected/468756d2-d46f-411a-a8ce-0c160d58c278-kube-api-access-f56tk\") on node \"crc\" DevicePath \"\"" Dec 03 12:51:14 crc kubenswrapper[5052]: I1203 12:51:14.778324 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/468756d2-d46f-411a-a8ce-0c160d58c278-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:51:14 crc kubenswrapper[5052]: I1203 12:51:14.778332 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/468756d2-d46f-411a-a8ce-0c160d58c278-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:51:14 crc kubenswrapper[5052]: I1203 12:51:14.778343 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/468756d2-d46f-411a-a8ce-0c160d58c278-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:51:14 crc kubenswrapper[5052]: I1203 12:51:14.790687 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:51:14 crc kubenswrapper[5052]: I1203 12:51:14.878960 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/c5398d23-68fd-4a1c-af2b-d0a7500cc973-config-data\") pod \"c5398d23-68fd-4a1c-af2b-d0a7500cc973\" (UID: \"c5398d23-68fd-4a1c-af2b-d0a7500cc973\") " Dec 03 12:51:14 crc kubenswrapper[5052]: I1203 12:51:14.879066 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c5398d23-68fd-4a1c-af2b-d0a7500cc973-combined-ca-bundle\") pod \"c5398d23-68fd-4a1c-af2b-d0a7500cc973\" (UID: \"c5398d23-68fd-4a1c-af2b-d0a7500cc973\") " Dec 03 12:51:14 crc kubenswrapper[5052]: I1203 12:51:14.879154 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-69m5r\" (UniqueName: \"kubernetes.io/projected/c5398d23-68fd-4a1c-af2b-d0a7500cc973-kube-api-access-69m5r\") pod \"c5398d23-68fd-4a1c-af2b-d0a7500cc973\" (UID: \"c5398d23-68fd-4a1c-af2b-d0a7500cc973\") " Dec 03 12:51:14 crc kubenswrapper[5052]: I1203 12:51:14.879229 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/c5398d23-68fd-4a1c-af2b-d0a7500cc973-run-httpd\") pod \"c5398d23-68fd-4a1c-af2b-d0a7500cc973\" (UID: \"c5398d23-68fd-4a1c-af2b-d0a7500cc973\") " Dec 03 12:51:14 crc kubenswrapper[5052]: I1203 12:51:14.879277 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/c5398d23-68fd-4a1c-af2b-d0a7500cc973-scripts\") pod \"c5398d23-68fd-4a1c-af2b-d0a7500cc973\" (UID: \"c5398d23-68fd-4a1c-af2b-d0a7500cc973\") " Dec 03 12:51:14 crc kubenswrapper[5052]: I1203 12:51:14.879364 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/c5398d23-68fd-4a1c-af2b-d0a7500cc973-sg-core-conf-yaml\") pod \"c5398d23-68fd-4a1c-af2b-d0a7500cc973\" (UID: \"c5398d23-68fd-4a1c-af2b-d0a7500cc973\") " Dec 03 12:51:14 crc kubenswrapper[5052]: I1203 12:51:14.879427 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/c5398d23-68fd-4a1c-af2b-d0a7500cc973-log-httpd\") pod \"c5398d23-68fd-4a1c-af2b-d0a7500cc973\" (UID: \"c5398d23-68fd-4a1c-af2b-d0a7500cc973\") " Dec 03 12:51:14 crc kubenswrapper[5052]: I1203 12:51:14.880095 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c5398d23-68fd-4a1c-af2b-d0a7500cc973-run-httpd" (OuterVolumeSpecName: "run-httpd") pod "c5398d23-68fd-4a1c-af2b-d0a7500cc973" (UID: "c5398d23-68fd-4a1c-af2b-d0a7500cc973"). InnerVolumeSpecName "run-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:51:14 crc kubenswrapper[5052]: I1203 12:51:14.880340 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c5398d23-68fd-4a1c-af2b-d0a7500cc973-log-httpd" (OuterVolumeSpecName: "log-httpd") pod "c5398d23-68fd-4a1c-af2b-d0a7500cc973" (UID: "c5398d23-68fd-4a1c-af2b-d0a7500cc973"). InnerVolumeSpecName "log-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:51:14 crc kubenswrapper[5052]: I1203 12:51:14.886578 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c5398d23-68fd-4a1c-af2b-d0a7500cc973-kube-api-access-69m5r" (OuterVolumeSpecName: "kube-api-access-69m5r") pod "c5398d23-68fd-4a1c-af2b-d0a7500cc973" (UID: "c5398d23-68fd-4a1c-af2b-d0a7500cc973"). InnerVolumeSpecName "kube-api-access-69m5r". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:51:14 crc kubenswrapper[5052]: I1203 12:51:14.889122 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c5398d23-68fd-4a1c-af2b-d0a7500cc973-scripts" (OuterVolumeSpecName: "scripts") pod "c5398d23-68fd-4a1c-af2b-d0a7500cc973" (UID: "c5398d23-68fd-4a1c-af2b-d0a7500cc973"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:51:14 crc kubenswrapper[5052]: I1203 12:51:14.919181 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c5398d23-68fd-4a1c-af2b-d0a7500cc973-sg-core-conf-yaml" (OuterVolumeSpecName: "sg-core-conf-yaml") pod "c5398d23-68fd-4a1c-af2b-d0a7500cc973" (UID: "c5398d23-68fd-4a1c-af2b-d0a7500cc973"). InnerVolumeSpecName "sg-core-conf-yaml". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:51:14 crc kubenswrapper[5052]: I1203 12:51:14.983362 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-69m5r\" (UniqueName: \"kubernetes.io/projected/c5398d23-68fd-4a1c-af2b-d0a7500cc973-kube-api-access-69m5r\") on node \"crc\" DevicePath \"\"" Dec 03 12:51:14 crc kubenswrapper[5052]: I1203 12:51:14.983826 5052 reconciler_common.go:293] "Volume detached for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/c5398d23-68fd-4a1c-af2b-d0a7500cc973-run-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:51:14 crc kubenswrapper[5052]: I1203 12:51:14.983840 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/c5398d23-68fd-4a1c-af2b-d0a7500cc973-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:51:14 crc kubenswrapper[5052]: I1203 12:51:14.983853 5052 reconciler_common.go:293] "Volume detached for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/c5398d23-68fd-4a1c-af2b-d0a7500cc973-sg-core-conf-yaml\") on node \"crc\" DevicePath \"\"" Dec 03 12:51:14 crc kubenswrapper[5052]: I1203 12:51:14.983865 5052 reconciler_common.go:293] "Volume detached for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/c5398d23-68fd-4a1c-af2b-d0a7500cc973-log-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:51:14 crc kubenswrapper[5052]: I1203 12:51:14.985531 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c5398d23-68fd-4a1c-af2b-d0a7500cc973-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "c5398d23-68fd-4a1c-af2b-d0a7500cc973" (UID: "c5398d23-68fd-4a1c-af2b-d0a7500cc973"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:51:15 crc kubenswrapper[5052]: I1203 12:51:15.008383 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c5398d23-68fd-4a1c-af2b-d0a7500cc973-config-data" (OuterVolumeSpecName: "config-data") pod "c5398d23-68fd-4a1c-af2b-d0a7500cc973" (UID: "c5398d23-68fd-4a1c-af2b-d0a7500cc973"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:51:15 crc kubenswrapper[5052]: I1203 12:51:15.013247 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-db-sync-xbbsx" event={"ID":"468756d2-d46f-411a-a8ce-0c160d58c278","Type":"ContainerDied","Data":"908bf14390a4882004191c33dcf6574357daba67c6584dcf8b14650cdd533dbe"} Dec 03 12:51:15 crc kubenswrapper[5052]: I1203 12:51:15.013304 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="908bf14390a4882004191c33dcf6574357daba67c6584dcf8b14650cdd533dbe" Dec 03 12:51:15 crc kubenswrapper[5052]: I1203 12:51:15.013267 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-db-sync-xbbsx" Dec 03 12:51:15 crc kubenswrapper[5052]: I1203 12:51:15.028537 5052 generic.go:334] "Generic (PLEG): container finished" podID="c5398d23-68fd-4a1c-af2b-d0a7500cc973" containerID="4daedd289d651c6e85f6a5a2c3b29d4274a156f001635aa0c260ea51ca5ecc5d" exitCode=0 Dec 03 12:51:15 crc kubenswrapper[5052]: I1203 12:51:15.028564 5052 generic.go:334] "Generic (PLEG): container finished" podID="c5398d23-68fd-4a1c-af2b-d0a7500cc973" containerID="7f4892364d811e394b52fa07f11dbd2d15da8c9336b319b27a65bef091b1d9c5" exitCode=2 Dec 03 12:51:15 crc kubenswrapper[5052]: I1203 12:51:15.028571 5052 generic.go:334] "Generic (PLEG): container finished" podID="c5398d23-68fd-4a1c-af2b-d0a7500cc973" containerID="9cd573f6f7ac6e4412f9a87ff3c6e0e029d47569495ab458cd84bc64613ea8ee" exitCode=0 Dec 03 12:51:15 crc kubenswrapper[5052]: I1203 12:51:15.028578 5052 generic.go:334] "Generic (PLEG): container finished" podID="c5398d23-68fd-4a1c-af2b-d0a7500cc973" containerID="4c1d3ef0fbbfab8064033b53eb26d8cd4924d5ae1aa7dde7609e5705ca05a452" exitCode=0 Dec 03 12:51:15 crc kubenswrapper[5052]: I1203 12:51:15.028644 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"c5398d23-68fd-4a1c-af2b-d0a7500cc973","Type":"ContainerDied","Data":"4daedd289d651c6e85f6a5a2c3b29d4274a156f001635aa0c260ea51ca5ecc5d"} Dec 03 12:51:15 crc kubenswrapper[5052]: I1203 12:51:15.028680 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"c5398d23-68fd-4a1c-af2b-d0a7500cc973","Type":"ContainerDied","Data":"7f4892364d811e394b52fa07f11dbd2d15da8c9336b319b27a65bef091b1d9c5"} Dec 03 12:51:15 crc kubenswrapper[5052]: I1203 12:51:15.028700 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"c5398d23-68fd-4a1c-af2b-d0a7500cc973","Type":"ContainerDied","Data":"9cd573f6f7ac6e4412f9a87ff3c6e0e029d47569495ab458cd84bc64613ea8ee"} Dec 03 12:51:15 crc kubenswrapper[5052]: I1203 12:51:15.028713 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"c5398d23-68fd-4a1c-af2b-d0a7500cc973","Type":"ContainerDied","Data":"4c1d3ef0fbbfab8064033b53eb26d8cd4924d5ae1aa7dde7609e5705ca05a452"} Dec 03 12:51:15 crc kubenswrapper[5052]: I1203 12:51:15.028724 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"c5398d23-68fd-4a1c-af2b-d0a7500cc973","Type":"ContainerDied","Data":"926bd638eec5757ee27004c7c18d3ceafab3a0dd3c091e38104af18db40f699a"} Dec 03 12:51:15 crc kubenswrapper[5052]: I1203 12:51:15.028743 5052 scope.go:117] "RemoveContainer" containerID="4daedd289d651c6e85f6a5a2c3b29d4274a156f001635aa0c260ea51ca5ecc5d" Dec 03 12:51:15 crc kubenswrapper[5052]: I1203 12:51:15.028925 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:51:15 crc kubenswrapper[5052]: I1203 12:51:15.036070 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"cd023d37-ce83-4ca4-ab52-f89e94d0b721","Type":"ContainerStarted","Data":"c6b845ca325df0412cbeac659e0056c26815c15f2c2df4fafcf3c74b16e2590a"} Dec 03 12:51:15 crc kubenswrapper[5052]: I1203 12:51:15.071349 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:51:15 crc kubenswrapper[5052]: I1203 12:51:15.083522 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:51:15 crc kubenswrapper[5052]: I1203 12:51:15.086885 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/c5398d23-68fd-4a1c-af2b-d0a7500cc973-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:51:15 crc kubenswrapper[5052]: I1203 12:51:15.086920 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c5398d23-68fd-4a1c-af2b-d0a7500cc973-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:51:15 crc kubenswrapper[5052]: I1203 12:51:15.094208 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:51:15 crc kubenswrapper[5052]: I1203 12:51:15.107065 5052 scope.go:117] "RemoveContainer" containerID="7f4892364d811e394b52fa07f11dbd2d15da8c9336b319b27a65bef091b1d9c5" Dec 03 12:51:15 crc kubenswrapper[5052]: I1203 12:51:15.107388 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:51:15 crc kubenswrapper[5052]: E1203 12:51:15.109243 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="468756d2-d46f-411a-a8ce-0c160d58c278" containerName="placement-db-sync" Dec 03 12:51:15 crc kubenswrapper[5052]: I1203 12:51:15.109264 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="468756d2-d46f-411a-a8ce-0c160d58c278" containerName="placement-db-sync" Dec 03 12:51:15 crc kubenswrapper[5052]: E1203 12:51:15.109290 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="c5398d23-68fd-4a1c-af2b-d0a7500cc973" containerName="ceilometer-notification-agent" Dec 03 12:51:15 crc kubenswrapper[5052]: I1203 12:51:15.109296 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="c5398d23-68fd-4a1c-af2b-d0a7500cc973" containerName="ceilometer-notification-agent" Dec 03 12:51:15 crc kubenswrapper[5052]: E1203 12:51:15.109310 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="c5398d23-68fd-4a1c-af2b-d0a7500cc973" containerName="proxy-httpd" Dec 03 12:51:15 crc kubenswrapper[5052]: I1203 12:51:15.109317 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="c5398d23-68fd-4a1c-af2b-d0a7500cc973" containerName="proxy-httpd" Dec 03 12:51:15 crc kubenswrapper[5052]: E1203 12:51:15.109337 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="c5398d23-68fd-4a1c-af2b-d0a7500cc973" containerName="ceilometer-central-agent" Dec 03 12:51:15 crc kubenswrapper[5052]: I1203 12:51:15.109342 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="c5398d23-68fd-4a1c-af2b-d0a7500cc973" containerName="ceilometer-central-agent" Dec 03 12:51:15 crc kubenswrapper[5052]: E1203 12:51:15.109354 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="c5398d23-68fd-4a1c-af2b-d0a7500cc973" containerName="sg-core" Dec 03 12:51:15 crc kubenswrapper[5052]: I1203 12:51:15.109360 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="c5398d23-68fd-4a1c-af2b-d0a7500cc973" containerName="sg-core" Dec 03 12:51:15 crc kubenswrapper[5052]: I1203 12:51:15.109564 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="c5398d23-68fd-4a1c-af2b-d0a7500cc973" containerName="ceilometer-central-agent" Dec 03 12:51:15 crc kubenswrapper[5052]: I1203 12:51:15.109580 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="468756d2-d46f-411a-a8ce-0c160d58c278" containerName="placement-db-sync" Dec 03 12:51:15 crc kubenswrapper[5052]: I1203 12:51:15.109587 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="c5398d23-68fd-4a1c-af2b-d0a7500cc973" containerName="sg-core" Dec 03 12:51:15 crc kubenswrapper[5052]: I1203 12:51:15.109595 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="c5398d23-68fd-4a1c-af2b-d0a7500cc973" containerName="proxy-httpd" Dec 03 12:51:15 crc kubenswrapper[5052]: I1203 12:51:15.109607 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="c5398d23-68fd-4a1c-af2b-d0a7500cc973" containerName="ceilometer-notification-agent" Dec 03 12:51:15 crc kubenswrapper[5052]: I1203 12:51:15.111243 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:51:15 crc kubenswrapper[5052]: I1203 12:51:15.114440 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-config-data" Dec 03 12:51:15 crc kubenswrapper[5052]: I1203 12:51:15.114791 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-scripts" Dec 03 12:51:15 crc kubenswrapper[5052]: I1203 12:51:15.128882 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:51:15 crc kubenswrapper[5052]: I1203 12:51:15.165063 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/placement-7b465d48b8-4bj69"] Dec 03 12:51:15 crc kubenswrapper[5052]: I1203 12:51:15.166596 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-7b465d48b8-4bj69" Dec 03 12:51:15 crc kubenswrapper[5052]: I1203 12:51:15.171765 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"placement-scripts" Dec 03 12:51:15 crc kubenswrapper[5052]: I1203 12:51:15.172107 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"placement-config-data" Dec 03 12:51:15 crc kubenswrapper[5052]: I1203 12:51:15.172561 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-placement-internal-svc" Dec 03 12:51:15 crc kubenswrapper[5052]: I1203 12:51:15.172730 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"placement-placement-dockercfg-tlqbn" Dec 03 12:51:15 crc kubenswrapper[5052]: I1203 12:51:15.172874 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-placement-public-svc" Dec 03 12:51:15 crc kubenswrapper[5052]: I1203 12:51:15.206456 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/placement-7b465d48b8-4bj69"] Dec 03 12:51:15 crc kubenswrapper[5052]: I1203 12:51:15.274647 5052 scope.go:117] "RemoveContainer" containerID="9cd573f6f7ac6e4412f9a87ff3c6e0e029d47569495ab458cd84bc64613ea8ee" Dec 03 12:51:15 crc kubenswrapper[5052]: I1203 12:51:15.295388 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/f3733be6-a76c-4c85-ad81-58ca3f3e3d2d-scripts\") pod \"ceilometer-0\" (UID: \"f3733be6-a76c-4c85-ad81-58ca3f3e3d2d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:51:15 crc kubenswrapper[5052]: I1203 12:51:15.295498 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/f3733be6-a76c-4c85-ad81-58ca3f3e3d2d-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"f3733be6-a76c-4c85-ad81-58ca3f3e3d2d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:51:15 crc kubenswrapper[5052]: I1203 12:51:15.295553 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/f3733be6-a76c-4c85-ad81-58ca3f3e3d2d-config-data\") pod \"ceilometer-0\" (UID: \"f3733be6-a76c-4c85-ad81-58ca3f3e3d2d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:51:15 crc kubenswrapper[5052]: I1203 12:51:15.295585 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/8df65ee4-b91d-4ffd-bff2-65e5e6559130-internal-tls-certs\") pod \"placement-7b465d48b8-4bj69\" (UID: \"8df65ee4-b91d-4ffd-bff2-65e5e6559130\") " pod="openstack-kuttl-tests/placement-7b465d48b8-4bj69" Dec 03 12:51:15 crc kubenswrapper[5052]: I1203 12:51:15.295642 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/8df65ee4-b91d-4ffd-bff2-65e5e6559130-public-tls-certs\") pod \"placement-7b465d48b8-4bj69\" (UID: \"8df65ee4-b91d-4ffd-bff2-65e5e6559130\") " pod="openstack-kuttl-tests/placement-7b465d48b8-4bj69" Dec 03 12:51:15 crc kubenswrapper[5052]: I1203 12:51:15.295689 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f3733be6-a76c-4c85-ad81-58ca3f3e3d2d-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"f3733be6-a76c-4c85-ad81-58ca3f3e3d2d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:51:15 crc kubenswrapper[5052]: I1203 12:51:15.295710 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8df65ee4-b91d-4ffd-bff2-65e5e6559130-combined-ca-bundle\") pod \"placement-7b465d48b8-4bj69\" (UID: \"8df65ee4-b91d-4ffd-bff2-65e5e6559130\") " pod="openstack-kuttl-tests/placement-7b465d48b8-4bj69" Dec 03 12:51:15 crc kubenswrapper[5052]: I1203 12:51:15.295737 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qj6ss\" (UniqueName: \"kubernetes.io/projected/8df65ee4-b91d-4ffd-bff2-65e5e6559130-kube-api-access-qj6ss\") pod \"placement-7b465d48b8-4bj69\" (UID: \"8df65ee4-b91d-4ffd-bff2-65e5e6559130\") " pod="openstack-kuttl-tests/placement-7b465d48b8-4bj69" Dec 03 12:51:15 crc kubenswrapper[5052]: I1203 12:51:15.295770 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/8df65ee4-b91d-4ffd-bff2-65e5e6559130-logs\") pod \"placement-7b465d48b8-4bj69\" (UID: \"8df65ee4-b91d-4ffd-bff2-65e5e6559130\") " pod="openstack-kuttl-tests/placement-7b465d48b8-4bj69" Dec 03 12:51:15 crc kubenswrapper[5052]: I1203 12:51:15.295796 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/f3733be6-a76c-4c85-ad81-58ca3f3e3d2d-log-httpd\") pod \"ceilometer-0\" (UID: \"f3733be6-a76c-4c85-ad81-58ca3f3e3d2d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:51:15 crc kubenswrapper[5052]: I1203 12:51:15.295822 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dcndt\" (UniqueName: \"kubernetes.io/projected/f3733be6-a76c-4c85-ad81-58ca3f3e3d2d-kube-api-access-dcndt\") pod \"ceilometer-0\" (UID: \"f3733be6-a76c-4c85-ad81-58ca3f3e3d2d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:51:15 crc kubenswrapper[5052]: I1203 12:51:15.296271 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/8df65ee4-b91d-4ffd-bff2-65e5e6559130-scripts\") pod \"placement-7b465d48b8-4bj69\" (UID: \"8df65ee4-b91d-4ffd-bff2-65e5e6559130\") " pod="openstack-kuttl-tests/placement-7b465d48b8-4bj69" Dec 03 12:51:15 crc kubenswrapper[5052]: I1203 12:51:15.296432 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/f3733be6-a76c-4c85-ad81-58ca3f3e3d2d-run-httpd\") pod \"ceilometer-0\" (UID: \"f3733be6-a76c-4c85-ad81-58ca3f3e3d2d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:51:15 crc kubenswrapper[5052]: I1203 12:51:15.296547 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/8df65ee4-b91d-4ffd-bff2-65e5e6559130-config-data\") pod \"placement-7b465d48b8-4bj69\" (UID: \"8df65ee4-b91d-4ffd-bff2-65e5e6559130\") " pod="openstack-kuttl-tests/placement-7b465d48b8-4bj69" Dec 03 12:51:15 crc kubenswrapper[5052]: I1203 12:51:15.298644 5052 scope.go:117] "RemoveContainer" containerID="4c1d3ef0fbbfab8064033b53eb26d8cd4924d5ae1aa7dde7609e5705ca05a452" Dec 03 12:51:15 crc kubenswrapper[5052]: I1203 12:51:15.331588 5052 scope.go:117] "RemoveContainer" containerID="4daedd289d651c6e85f6a5a2c3b29d4274a156f001635aa0c260ea51ca5ecc5d" Dec 03 12:51:15 crc kubenswrapper[5052]: E1203 12:51:15.333287 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4daedd289d651c6e85f6a5a2c3b29d4274a156f001635aa0c260ea51ca5ecc5d\": container with ID starting with 4daedd289d651c6e85f6a5a2c3b29d4274a156f001635aa0c260ea51ca5ecc5d not found: ID does not exist" containerID="4daedd289d651c6e85f6a5a2c3b29d4274a156f001635aa0c260ea51ca5ecc5d" Dec 03 12:51:15 crc kubenswrapper[5052]: I1203 12:51:15.333320 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4daedd289d651c6e85f6a5a2c3b29d4274a156f001635aa0c260ea51ca5ecc5d"} err="failed to get container status \"4daedd289d651c6e85f6a5a2c3b29d4274a156f001635aa0c260ea51ca5ecc5d\": rpc error: code = NotFound desc = could not find container \"4daedd289d651c6e85f6a5a2c3b29d4274a156f001635aa0c260ea51ca5ecc5d\": container with ID starting with 4daedd289d651c6e85f6a5a2c3b29d4274a156f001635aa0c260ea51ca5ecc5d not found: ID does not exist" Dec 03 12:51:15 crc kubenswrapper[5052]: I1203 12:51:15.333341 5052 scope.go:117] "RemoveContainer" containerID="7f4892364d811e394b52fa07f11dbd2d15da8c9336b319b27a65bef091b1d9c5" Dec 03 12:51:15 crc kubenswrapper[5052]: E1203 12:51:15.333883 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7f4892364d811e394b52fa07f11dbd2d15da8c9336b319b27a65bef091b1d9c5\": container with ID starting with 7f4892364d811e394b52fa07f11dbd2d15da8c9336b319b27a65bef091b1d9c5 not found: ID does not exist" containerID="7f4892364d811e394b52fa07f11dbd2d15da8c9336b319b27a65bef091b1d9c5" Dec 03 12:51:15 crc kubenswrapper[5052]: I1203 12:51:15.333921 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7f4892364d811e394b52fa07f11dbd2d15da8c9336b319b27a65bef091b1d9c5"} err="failed to get container status \"7f4892364d811e394b52fa07f11dbd2d15da8c9336b319b27a65bef091b1d9c5\": rpc error: code = NotFound desc = could not find container \"7f4892364d811e394b52fa07f11dbd2d15da8c9336b319b27a65bef091b1d9c5\": container with ID starting with 7f4892364d811e394b52fa07f11dbd2d15da8c9336b319b27a65bef091b1d9c5 not found: ID does not exist" Dec 03 12:51:15 crc kubenswrapper[5052]: I1203 12:51:15.333946 5052 scope.go:117] "RemoveContainer" containerID="9cd573f6f7ac6e4412f9a87ff3c6e0e029d47569495ab458cd84bc64613ea8ee" Dec 03 12:51:15 crc kubenswrapper[5052]: E1203 12:51:15.337213 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9cd573f6f7ac6e4412f9a87ff3c6e0e029d47569495ab458cd84bc64613ea8ee\": container with ID starting with 9cd573f6f7ac6e4412f9a87ff3c6e0e029d47569495ab458cd84bc64613ea8ee not found: ID does not exist" containerID="9cd573f6f7ac6e4412f9a87ff3c6e0e029d47569495ab458cd84bc64613ea8ee" Dec 03 12:51:15 crc kubenswrapper[5052]: I1203 12:51:15.337244 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9cd573f6f7ac6e4412f9a87ff3c6e0e029d47569495ab458cd84bc64613ea8ee"} err="failed to get container status \"9cd573f6f7ac6e4412f9a87ff3c6e0e029d47569495ab458cd84bc64613ea8ee\": rpc error: code = NotFound desc = could not find container \"9cd573f6f7ac6e4412f9a87ff3c6e0e029d47569495ab458cd84bc64613ea8ee\": container with ID starting with 9cd573f6f7ac6e4412f9a87ff3c6e0e029d47569495ab458cd84bc64613ea8ee not found: ID does not exist" Dec 03 12:51:15 crc kubenswrapper[5052]: I1203 12:51:15.337266 5052 scope.go:117] "RemoveContainer" containerID="4c1d3ef0fbbfab8064033b53eb26d8cd4924d5ae1aa7dde7609e5705ca05a452" Dec 03 12:51:15 crc kubenswrapper[5052]: E1203 12:51:15.338903 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4c1d3ef0fbbfab8064033b53eb26d8cd4924d5ae1aa7dde7609e5705ca05a452\": container with ID starting with 4c1d3ef0fbbfab8064033b53eb26d8cd4924d5ae1aa7dde7609e5705ca05a452 not found: ID does not exist" containerID="4c1d3ef0fbbfab8064033b53eb26d8cd4924d5ae1aa7dde7609e5705ca05a452" Dec 03 12:51:15 crc kubenswrapper[5052]: I1203 12:51:15.338944 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4c1d3ef0fbbfab8064033b53eb26d8cd4924d5ae1aa7dde7609e5705ca05a452"} err="failed to get container status \"4c1d3ef0fbbfab8064033b53eb26d8cd4924d5ae1aa7dde7609e5705ca05a452\": rpc error: code = NotFound desc = could not find container \"4c1d3ef0fbbfab8064033b53eb26d8cd4924d5ae1aa7dde7609e5705ca05a452\": container with ID starting with 4c1d3ef0fbbfab8064033b53eb26d8cd4924d5ae1aa7dde7609e5705ca05a452 not found: ID does not exist" Dec 03 12:51:15 crc kubenswrapper[5052]: I1203 12:51:15.338977 5052 scope.go:117] "RemoveContainer" containerID="4daedd289d651c6e85f6a5a2c3b29d4274a156f001635aa0c260ea51ca5ecc5d" Dec 03 12:51:15 crc kubenswrapper[5052]: I1203 12:51:15.339449 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4daedd289d651c6e85f6a5a2c3b29d4274a156f001635aa0c260ea51ca5ecc5d"} err="failed to get container status \"4daedd289d651c6e85f6a5a2c3b29d4274a156f001635aa0c260ea51ca5ecc5d\": rpc error: code = NotFound desc = could not find container \"4daedd289d651c6e85f6a5a2c3b29d4274a156f001635aa0c260ea51ca5ecc5d\": container with ID starting with 4daedd289d651c6e85f6a5a2c3b29d4274a156f001635aa0c260ea51ca5ecc5d not found: ID does not exist" Dec 03 12:51:15 crc kubenswrapper[5052]: I1203 12:51:15.339482 5052 scope.go:117] "RemoveContainer" containerID="7f4892364d811e394b52fa07f11dbd2d15da8c9336b319b27a65bef091b1d9c5" Dec 03 12:51:15 crc kubenswrapper[5052]: I1203 12:51:15.339759 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7f4892364d811e394b52fa07f11dbd2d15da8c9336b319b27a65bef091b1d9c5"} err="failed to get container status \"7f4892364d811e394b52fa07f11dbd2d15da8c9336b319b27a65bef091b1d9c5\": rpc error: code = NotFound desc = could not find container \"7f4892364d811e394b52fa07f11dbd2d15da8c9336b319b27a65bef091b1d9c5\": container with ID starting with 7f4892364d811e394b52fa07f11dbd2d15da8c9336b319b27a65bef091b1d9c5 not found: ID does not exist" Dec 03 12:51:15 crc kubenswrapper[5052]: I1203 12:51:15.339776 5052 scope.go:117] "RemoveContainer" containerID="9cd573f6f7ac6e4412f9a87ff3c6e0e029d47569495ab458cd84bc64613ea8ee" Dec 03 12:51:15 crc kubenswrapper[5052]: I1203 12:51:15.340120 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9cd573f6f7ac6e4412f9a87ff3c6e0e029d47569495ab458cd84bc64613ea8ee"} err="failed to get container status \"9cd573f6f7ac6e4412f9a87ff3c6e0e029d47569495ab458cd84bc64613ea8ee\": rpc error: code = NotFound desc = could not find container \"9cd573f6f7ac6e4412f9a87ff3c6e0e029d47569495ab458cd84bc64613ea8ee\": container with ID starting with 9cd573f6f7ac6e4412f9a87ff3c6e0e029d47569495ab458cd84bc64613ea8ee not found: ID does not exist" Dec 03 12:51:15 crc kubenswrapper[5052]: I1203 12:51:15.340143 5052 scope.go:117] "RemoveContainer" containerID="4c1d3ef0fbbfab8064033b53eb26d8cd4924d5ae1aa7dde7609e5705ca05a452" Dec 03 12:51:15 crc kubenswrapper[5052]: I1203 12:51:15.340606 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4c1d3ef0fbbfab8064033b53eb26d8cd4924d5ae1aa7dde7609e5705ca05a452"} err="failed to get container status \"4c1d3ef0fbbfab8064033b53eb26d8cd4924d5ae1aa7dde7609e5705ca05a452\": rpc error: code = NotFound desc = could not find container \"4c1d3ef0fbbfab8064033b53eb26d8cd4924d5ae1aa7dde7609e5705ca05a452\": container with ID starting with 4c1d3ef0fbbfab8064033b53eb26d8cd4924d5ae1aa7dde7609e5705ca05a452 not found: ID does not exist" Dec 03 12:51:15 crc kubenswrapper[5052]: I1203 12:51:15.340625 5052 scope.go:117] "RemoveContainer" containerID="4daedd289d651c6e85f6a5a2c3b29d4274a156f001635aa0c260ea51ca5ecc5d" Dec 03 12:51:15 crc kubenswrapper[5052]: I1203 12:51:15.340908 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4daedd289d651c6e85f6a5a2c3b29d4274a156f001635aa0c260ea51ca5ecc5d"} err="failed to get container status \"4daedd289d651c6e85f6a5a2c3b29d4274a156f001635aa0c260ea51ca5ecc5d\": rpc error: code = NotFound desc = could not find container \"4daedd289d651c6e85f6a5a2c3b29d4274a156f001635aa0c260ea51ca5ecc5d\": container with ID starting with 4daedd289d651c6e85f6a5a2c3b29d4274a156f001635aa0c260ea51ca5ecc5d not found: ID does not exist" Dec 03 12:51:15 crc kubenswrapper[5052]: I1203 12:51:15.340926 5052 scope.go:117] "RemoveContainer" containerID="7f4892364d811e394b52fa07f11dbd2d15da8c9336b319b27a65bef091b1d9c5" Dec 03 12:51:15 crc kubenswrapper[5052]: I1203 12:51:15.341178 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7f4892364d811e394b52fa07f11dbd2d15da8c9336b319b27a65bef091b1d9c5"} err="failed to get container status \"7f4892364d811e394b52fa07f11dbd2d15da8c9336b319b27a65bef091b1d9c5\": rpc error: code = NotFound desc = could not find container \"7f4892364d811e394b52fa07f11dbd2d15da8c9336b319b27a65bef091b1d9c5\": container with ID starting with 7f4892364d811e394b52fa07f11dbd2d15da8c9336b319b27a65bef091b1d9c5 not found: ID does not exist" Dec 03 12:51:15 crc kubenswrapper[5052]: I1203 12:51:15.341195 5052 scope.go:117] "RemoveContainer" containerID="9cd573f6f7ac6e4412f9a87ff3c6e0e029d47569495ab458cd84bc64613ea8ee" Dec 03 12:51:15 crc kubenswrapper[5052]: I1203 12:51:15.341402 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9cd573f6f7ac6e4412f9a87ff3c6e0e029d47569495ab458cd84bc64613ea8ee"} err="failed to get container status \"9cd573f6f7ac6e4412f9a87ff3c6e0e029d47569495ab458cd84bc64613ea8ee\": rpc error: code = NotFound desc = could not find container \"9cd573f6f7ac6e4412f9a87ff3c6e0e029d47569495ab458cd84bc64613ea8ee\": container with ID starting with 9cd573f6f7ac6e4412f9a87ff3c6e0e029d47569495ab458cd84bc64613ea8ee not found: ID does not exist" Dec 03 12:51:15 crc kubenswrapper[5052]: I1203 12:51:15.341421 5052 scope.go:117] "RemoveContainer" containerID="4c1d3ef0fbbfab8064033b53eb26d8cd4924d5ae1aa7dde7609e5705ca05a452" Dec 03 12:51:15 crc kubenswrapper[5052]: I1203 12:51:15.341688 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4c1d3ef0fbbfab8064033b53eb26d8cd4924d5ae1aa7dde7609e5705ca05a452"} err="failed to get container status \"4c1d3ef0fbbfab8064033b53eb26d8cd4924d5ae1aa7dde7609e5705ca05a452\": rpc error: code = NotFound desc = could not find container \"4c1d3ef0fbbfab8064033b53eb26d8cd4924d5ae1aa7dde7609e5705ca05a452\": container with ID starting with 4c1d3ef0fbbfab8064033b53eb26d8cd4924d5ae1aa7dde7609e5705ca05a452 not found: ID does not exist" Dec 03 12:51:15 crc kubenswrapper[5052]: I1203 12:51:15.341705 5052 scope.go:117] "RemoveContainer" containerID="4daedd289d651c6e85f6a5a2c3b29d4274a156f001635aa0c260ea51ca5ecc5d" Dec 03 12:51:15 crc kubenswrapper[5052]: I1203 12:51:15.341882 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4daedd289d651c6e85f6a5a2c3b29d4274a156f001635aa0c260ea51ca5ecc5d"} err="failed to get container status \"4daedd289d651c6e85f6a5a2c3b29d4274a156f001635aa0c260ea51ca5ecc5d\": rpc error: code = NotFound desc = could not find container \"4daedd289d651c6e85f6a5a2c3b29d4274a156f001635aa0c260ea51ca5ecc5d\": container with ID starting with 4daedd289d651c6e85f6a5a2c3b29d4274a156f001635aa0c260ea51ca5ecc5d not found: ID does not exist" Dec 03 12:51:15 crc kubenswrapper[5052]: I1203 12:51:15.341899 5052 scope.go:117] "RemoveContainer" containerID="7f4892364d811e394b52fa07f11dbd2d15da8c9336b319b27a65bef091b1d9c5" Dec 03 12:51:15 crc kubenswrapper[5052]: I1203 12:51:15.342324 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7f4892364d811e394b52fa07f11dbd2d15da8c9336b319b27a65bef091b1d9c5"} err="failed to get container status \"7f4892364d811e394b52fa07f11dbd2d15da8c9336b319b27a65bef091b1d9c5\": rpc error: code = NotFound desc = could not find container \"7f4892364d811e394b52fa07f11dbd2d15da8c9336b319b27a65bef091b1d9c5\": container with ID starting with 7f4892364d811e394b52fa07f11dbd2d15da8c9336b319b27a65bef091b1d9c5 not found: ID does not exist" Dec 03 12:51:15 crc kubenswrapper[5052]: I1203 12:51:15.342348 5052 scope.go:117] "RemoveContainer" containerID="9cd573f6f7ac6e4412f9a87ff3c6e0e029d47569495ab458cd84bc64613ea8ee" Dec 03 12:51:15 crc kubenswrapper[5052]: I1203 12:51:15.342908 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9cd573f6f7ac6e4412f9a87ff3c6e0e029d47569495ab458cd84bc64613ea8ee"} err="failed to get container status \"9cd573f6f7ac6e4412f9a87ff3c6e0e029d47569495ab458cd84bc64613ea8ee\": rpc error: code = NotFound desc = could not find container \"9cd573f6f7ac6e4412f9a87ff3c6e0e029d47569495ab458cd84bc64613ea8ee\": container with ID starting with 9cd573f6f7ac6e4412f9a87ff3c6e0e029d47569495ab458cd84bc64613ea8ee not found: ID does not exist" Dec 03 12:51:15 crc kubenswrapper[5052]: I1203 12:51:15.342933 5052 scope.go:117] "RemoveContainer" containerID="4c1d3ef0fbbfab8064033b53eb26d8cd4924d5ae1aa7dde7609e5705ca05a452" Dec 03 12:51:15 crc kubenswrapper[5052]: I1203 12:51:15.343155 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4c1d3ef0fbbfab8064033b53eb26d8cd4924d5ae1aa7dde7609e5705ca05a452"} err="failed to get container status \"4c1d3ef0fbbfab8064033b53eb26d8cd4924d5ae1aa7dde7609e5705ca05a452\": rpc error: code = NotFound desc = could not find container \"4c1d3ef0fbbfab8064033b53eb26d8cd4924d5ae1aa7dde7609e5705ca05a452\": container with ID starting with 4c1d3ef0fbbfab8064033b53eb26d8cd4924d5ae1aa7dde7609e5705ca05a452 not found: ID does not exist" Dec 03 12:51:15 crc kubenswrapper[5052]: I1203 12:51:15.397807 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/8df65ee4-b91d-4ffd-bff2-65e5e6559130-logs\") pod \"placement-7b465d48b8-4bj69\" (UID: \"8df65ee4-b91d-4ffd-bff2-65e5e6559130\") " pod="openstack-kuttl-tests/placement-7b465d48b8-4bj69" Dec 03 12:51:15 crc kubenswrapper[5052]: I1203 12:51:15.397845 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/f3733be6-a76c-4c85-ad81-58ca3f3e3d2d-log-httpd\") pod \"ceilometer-0\" (UID: \"f3733be6-a76c-4c85-ad81-58ca3f3e3d2d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:51:15 crc kubenswrapper[5052]: I1203 12:51:15.397876 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-dcndt\" (UniqueName: \"kubernetes.io/projected/f3733be6-a76c-4c85-ad81-58ca3f3e3d2d-kube-api-access-dcndt\") pod \"ceilometer-0\" (UID: \"f3733be6-a76c-4c85-ad81-58ca3f3e3d2d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:51:15 crc kubenswrapper[5052]: I1203 12:51:15.397916 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/8df65ee4-b91d-4ffd-bff2-65e5e6559130-scripts\") pod \"placement-7b465d48b8-4bj69\" (UID: \"8df65ee4-b91d-4ffd-bff2-65e5e6559130\") " pod="openstack-kuttl-tests/placement-7b465d48b8-4bj69" Dec 03 12:51:15 crc kubenswrapper[5052]: I1203 12:51:15.397944 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/f3733be6-a76c-4c85-ad81-58ca3f3e3d2d-run-httpd\") pod \"ceilometer-0\" (UID: \"f3733be6-a76c-4c85-ad81-58ca3f3e3d2d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:51:15 crc kubenswrapper[5052]: I1203 12:51:15.397971 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/8df65ee4-b91d-4ffd-bff2-65e5e6559130-config-data\") pod \"placement-7b465d48b8-4bj69\" (UID: \"8df65ee4-b91d-4ffd-bff2-65e5e6559130\") " pod="openstack-kuttl-tests/placement-7b465d48b8-4bj69" Dec 03 12:51:15 crc kubenswrapper[5052]: I1203 12:51:15.398014 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/f3733be6-a76c-4c85-ad81-58ca3f3e3d2d-scripts\") pod \"ceilometer-0\" (UID: \"f3733be6-a76c-4c85-ad81-58ca3f3e3d2d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:51:15 crc kubenswrapper[5052]: I1203 12:51:15.398040 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/f3733be6-a76c-4c85-ad81-58ca3f3e3d2d-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"f3733be6-a76c-4c85-ad81-58ca3f3e3d2d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:51:15 crc kubenswrapper[5052]: I1203 12:51:15.398055 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/f3733be6-a76c-4c85-ad81-58ca3f3e3d2d-config-data\") pod \"ceilometer-0\" (UID: \"f3733be6-a76c-4c85-ad81-58ca3f3e3d2d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:51:15 crc kubenswrapper[5052]: I1203 12:51:15.398076 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/8df65ee4-b91d-4ffd-bff2-65e5e6559130-internal-tls-certs\") pod \"placement-7b465d48b8-4bj69\" (UID: \"8df65ee4-b91d-4ffd-bff2-65e5e6559130\") " pod="openstack-kuttl-tests/placement-7b465d48b8-4bj69" Dec 03 12:51:15 crc kubenswrapper[5052]: I1203 12:51:15.398104 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/8df65ee4-b91d-4ffd-bff2-65e5e6559130-public-tls-certs\") pod \"placement-7b465d48b8-4bj69\" (UID: \"8df65ee4-b91d-4ffd-bff2-65e5e6559130\") " pod="openstack-kuttl-tests/placement-7b465d48b8-4bj69" Dec 03 12:51:15 crc kubenswrapper[5052]: I1203 12:51:15.398133 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f3733be6-a76c-4c85-ad81-58ca3f3e3d2d-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"f3733be6-a76c-4c85-ad81-58ca3f3e3d2d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:51:15 crc kubenswrapper[5052]: I1203 12:51:15.398149 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8df65ee4-b91d-4ffd-bff2-65e5e6559130-combined-ca-bundle\") pod \"placement-7b465d48b8-4bj69\" (UID: \"8df65ee4-b91d-4ffd-bff2-65e5e6559130\") " pod="openstack-kuttl-tests/placement-7b465d48b8-4bj69" Dec 03 12:51:15 crc kubenswrapper[5052]: I1203 12:51:15.398199 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-qj6ss\" (UniqueName: \"kubernetes.io/projected/8df65ee4-b91d-4ffd-bff2-65e5e6559130-kube-api-access-qj6ss\") pod \"placement-7b465d48b8-4bj69\" (UID: \"8df65ee4-b91d-4ffd-bff2-65e5e6559130\") " pod="openstack-kuttl-tests/placement-7b465d48b8-4bj69" Dec 03 12:51:15 crc kubenswrapper[5052]: I1203 12:51:15.398257 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/8df65ee4-b91d-4ffd-bff2-65e5e6559130-logs\") pod \"placement-7b465d48b8-4bj69\" (UID: \"8df65ee4-b91d-4ffd-bff2-65e5e6559130\") " pod="openstack-kuttl-tests/placement-7b465d48b8-4bj69" Dec 03 12:51:15 crc kubenswrapper[5052]: I1203 12:51:15.402953 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/f3733be6-a76c-4c85-ad81-58ca3f3e3d2d-log-httpd\") pod \"ceilometer-0\" (UID: \"f3733be6-a76c-4c85-ad81-58ca3f3e3d2d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:51:15 crc kubenswrapper[5052]: I1203 12:51:15.403190 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/f3733be6-a76c-4c85-ad81-58ca3f3e3d2d-run-httpd\") pod \"ceilometer-0\" (UID: \"f3733be6-a76c-4c85-ad81-58ca3f3e3d2d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:51:15 crc kubenswrapper[5052]: I1203 12:51:15.404932 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/8df65ee4-b91d-4ffd-bff2-65e5e6559130-internal-tls-certs\") pod \"placement-7b465d48b8-4bj69\" (UID: \"8df65ee4-b91d-4ffd-bff2-65e5e6559130\") " pod="openstack-kuttl-tests/placement-7b465d48b8-4bj69" Dec 03 12:51:15 crc kubenswrapper[5052]: I1203 12:51:15.406676 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8df65ee4-b91d-4ffd-bff2-65e5e6559130-combined-ca-bundle\") pod \"placement-7b465d48b8-4bj69\" (UID: \"8df65ee4-b91d-4ffd-bff2-65e5e6559130\") " pod="openstack-kuttl-tests/placement-7b465d48b8-4bj69" Dec 03 12:51:15 crc kubenswrapper[5052]: I1203 12:51:15.413318 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/f3733be6-a76c-4c85-ad81-58ca3f3e3d2d-scripts\") pod \"ceilometer-0\" (UID: \"f3733be6-a76c-4c85-ad81-58ca3f3e3d2d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:51:15 crc kubenswrapper[5052]: I1203 12:51:15.417989 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/8df65ee4-b91d-4ffd-bff2-65e5e6559130-config-data\") pod \"placement-7b465d48b8-4bj69\" (UID: \"8df65ee4-b91d-4ffd-bff2-65e5e6559130\") " pod="openstack-kuttl-tests/placement-7b465d48b8-4bj69" Dec 03 12:51:15 crc kubenswrapper[5052]: I1203 12:51:15.418789 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/8df65ee4-b91d-4ffd-bff2-65e5e6559130-scripts\") pod \"placement-7b465d48b8-4bj69\" (UID: \"8df65ee4-b91d-4ffd-bff2-65e5e6559130\") " pod="openstack-kuttl-tests/placement-7b465d48b8-4bj69" Dec 03 12:51:15 crc kubenswrapper[5052]: I1203 12:51:15.420193 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/8df65ee4-b91d-4ffd-bff2-65e5e6559130-public-tls-certs\") pod \"placement-7b465d48b8-4bj69\" (UID: \"8df65ee4-b91d-4ffd-bff2-65e5e6559130\") " pod="openstack-kuttl-tests/placement-7b465d48b8-4bj69" Dec 03 12:51:15 crc kubenswrapper[5052]: I1203 12:51:15.420485 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f3733be6-a76c-4c85-ad81-58ca3f3e3d2d-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"f3733be6-a76c-4c85-ad81-58ca3f3e3d2d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:51:15 crc kubenswrapper[5052]: I1203 12:51:15.422843 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/f3733be6-a76c-4c85-ad81-58ca3f3e3d2d-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"f3733be6-a76c-4c85-ad81-58ca3f3e3d2d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:51:15 crc kubenswrapper[5052]: I1203 12:51:15.426068 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-qj6ss\" (UniqueName: \"kubernetes.io/projected/8df65ee4-b91d-4ffd-bff2-65e5e6559130-kube-api-access-qj6ss\") pod \"placement-7b465d48b8-4bj69\" (UID: \"8df65ee4-b91d-4ffd-bff2-65e5e6559130\") " pod="openstack-kuttl-tests/placement-7b465d48b8-4bj69" Dec 03 12:51:15 crc kubenswrapper[5052]: I1203 12:51:15.426257 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-dcndt\" (UniqueName: \"kubernetes.io/projected/f3733be6-a76c-4c85-ad81-58ca3f3e3d2d-kube-api-access-dcndt\") pod \"ceilometer-0\" (UID: \"f3733be6-a76c-4c85-ad81-58ca3f3e3d2d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:51:15 crc kubenswrapper[5052]: I1203 12:51:15.426448 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/f3733be6-a76c-4c85-ad81-58ca3f3e3d2d-config-data\") pod \"ceilometer-0\" (UID: \"f3733be6-a76c-4c85-ad81-58ca3f3e3d2d\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:51:15 crc kubenswrapper[5052]: I1203 12:51:15.546996 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:51:15 crc kubenswrapper[5052]: I1203 12:51:15.562985 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-7b465d48b8-4bj69" Dec 03 12:51:15 crc kubenswrapper[5052]: I1203 12:51:15.570351 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-bootstrap-mgkjf" Dec 03 12:51:15 crc kubenswrapper[5052]: I1203 12:51:15.593826 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-db-sync-rsmgs" Dec 03 12:51:15 crc kubenswrapper[5052]: I1203 12:51:15.701603 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/83062467-67e1-4cab-b536-6edf3ea916b4-fernet-keys\") pod \"83062467-67e1-4cab-b536-6edf3ea916b4\" (UID: \"83062467-67e1-4cab-b536-6edf3ea916b4\") " Dec 03 12:51:15 crc kubenswrapper[5052]: I1203 12:51:15.702686 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/83062467-67e1-4cab-b536-6edf3ea916b4-credential-keys\") pod \"83062467-67e1-4cab-b536-6edf3ea916b4\" (UID: \"83062467-67e1-4cab-b536-6edf3ea916b4\") " Dec 03 12:51:15 crc kubenswrapper[5052]: I1203 12:51:15.703575 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-99mb9\" (UniqueName: \"kubernetes.io/projected/83062467-67e1-4cab-b536-6edf3ea916b4-kube-api-access-99mb9\") pod \"83062467-67e1-4cab-b536-6edf3ea916b4\" (UID: \"83062467-67e1-4cab-b536-6edf3ea916b4\") " Dec 03 12:51:15 crc kubenswrapper[5052]: I1203 12:51:15.703751 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lnhfk\" (UniqueName: \"kubernetes.io/projected/2fab7846-f3ad-4673-b813-36c20d4b7611-kube-api-access-lnhfk\") pod \"2fab7846-f3ad-4673-b813-36c20d4b7611\" (UID: \"2fab7846-f3ad-4673-b813-36c20d4b7611\") " Dec 03 12:51:15 crc kubenswrapper[5052]: I1203 12:51:15.703817 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/83062467-67e1-4cab-b536-6edf3ea916b4-config-data\") pod \"83062467-67e1-4cab-b536-6edf3ea916b4\" (UID: \"83062467-67e1-4cab-b536-6edf3ea916b4\") " Dec 03 12:51:15 crc kubenswrapper[5052]: I1203 12:51:15.703937 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2fab7846-f3ad-4673-b813-36c20d4b7611-combined-ca-bundle\") pod \"2fab7846-f3ad-4673-b813-36c20d4b7611\" (UID: \"2fab7846-f3ad-4673-b813-36c20d4b7611\") " Dec 03 12:51:15 crc kubenswrapper[5052]: I1203 12:51:15.703999 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/83062467-67e1-4cab-b536-6edf3ea916b4-combined-ca-bundle\") pod \"83062467-67e1-4cab-b536-6edf3ea916b4\" (UID: \"83062467-67e1-4cab-b536-6edf3ea916b4\") " Dec 03 12:51:15 crc kubenswrapper[5052]: I1203 12:51:15.704030 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/2fab7846-f3ad-4673-b813-36c20d4b7611-db-sync-config-data\") pod \"2fab7846-f3ad-4673-b813-36c20d4b7611\" (UID: \"2fab7846-f3ad-4673-b813-36c20d4b7611\") " Dec 03 12:51:15 crc kubenswrapper[5052]: I1203 12:51:15.704090 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/83062467-67e1-4cab-b536-6edf3ea916b4-scripts\") pod \"83062467-67e1-4cab-b536-6edf3ea916b4\" (UID: \"83062467-67e1-4cab-b536-6edf3ea916b4\") " Dec 03 12:51:15 crc kubenswrapper[5052]: I1203 12:51:15.706468 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="37d28f5c-4b30-437d-a68e-44bc3ef970a5" path="/var/lib/kubelet/pods/37d28f5c-4b30-437d-a68e-44bc3ef970a5/volumes" Dec 03 12:51:15 crc kubenswrapper[5052]: I1203 12:51:15.711699 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c5398d23-68fd-4a1c-af2b-d0a7500cc973" path="/var/lib/kubelet/pods/c5398d23-68fd-4a1c-af2b-d0a7500cc973/volumes" Dec 03 12:51:15 crc kubenswrapper[5052]: I1203 12:51:15.714724 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/83062467-67e1-4cab-b536-6edf3ea916b4-credential-keys" (OuterVolumeSpecName: "credential-keys") pod "83062467-67e1-4cab-b536-6edf3ea916b4" (UID: "83062467-67e1-4cab-b536-6edf3ea916b4"). InnerVolumeSpecName "credential-keys". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:51:15 crc kubenswrapper[5052]: I1203 12:51:15.718669 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/83062467-67e1-4cab-b536-6edf3ea916b4-scripts" (OuterVolumeSpecName: "scripts") pod "83062467-67e1-4cab-b536-6edf3ea916b4" (UID: "83062467-67e1-4cab-b536-6edf3ea916b4"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:51:15 crc kubenswrapper[5052]: I1203 12:51:15.718683 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/83062467-67e1-4cab-b536-6edf3ea916b4-fernet-keys" (OuterVolumeSpecName: "fernet-keys") pod "83062467-67e1-4cab-b536-6edf3ea916b4" (UID: "83062467-67e1-4cab-b536-6edf3ea916b4"). InnerVolumeSpecName "fernet-keys". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:51:15 crc kubenswrapper[5052]: I1203 12:51:15.718776 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2fab7846-f3ad-4673-b813-36c20d4b7611-db-sync-config-data" (OuterVolumeSpecName: "db-sync-config-data") pod "2fab7846-f3ad-4673-b813-36c20d4b7611" (UID: "2fab7846-f3ad-4673-b813-36c20d4b7611"). InnerVolumeSpecName "db-sync-config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:51:15 crc kubenswrapper[5052]: I1203 12:51:15.718982 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2fab7846-f3ad-4673-b813-36c20d4b7611-kube-api-access-lnhfk" (OuterVolumeSpecName: "kube-api-access-lnhfk") pod "2fab7846-f3ad-4673-b813-36c20d4b7611" (UID: "2fab7846-f3ad-4673-b813-36c20d4b7611"). InnerVolumeSpecName "kube-api-access-lnhfk". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:51:15 crc kubenswrapper[5052]: I1203 12:51:15.719102 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/83062467-67e1-4cab-b536-6edf3ea916b4-kube-api-access-99mb9" (OuterVolumeSpecName: "kube-api-access-99mb9") pod "83062467-67e1-4cab-b536-6edf3ea916b4" (UID: "83062467-67e1-4cab-b536-6edf3ea916b4"). InnerVolumeSpecName "kube-api-access-99mb9". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:51:15 crc kubenswrapper[5052]: I1203 12:51:15.740308 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/83062467-67e1-4cab-b536-6edf3ea916b4-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "83062467-67e1-4cab-b536-6edf3ea916b4" (UID: "83062467-67e1-4cab-b536-6edf3ea916b4"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:51:15 crc kubenswrapper[5052]: I1203 12:51:15.748816 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/83062467-67e1-4cab-b536-6edf3ea916b4-config-data" (OuterVolumeSpecName: "config-data") pod "83062467-67e1-4cab-b536-6edf3ea916b4" (UID: "83062467-67e1-4cab-b536-6edf3ea916b4"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:51:15 crc kubenswrapper[5052]: I1203 12:51:15.755697 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2fab7846-f3ad-4673-b813-36c20d4b7611-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "2fab7846-f3ad-4673-b813-36c20d4b7611" (UID: "2fab7846-f3ad-4673-b813-36c20d4b7611"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:51:15 crc kubenswrapper[5052]: I1203 12:51:15.806662 5052 reconciler_common.go:293] "Volume detached for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/83062467-67e1-4cab-b536-6edf3ea916b4-fernet-keys\") on node \"crc\" DevicePath \"\"" Dec 03 12:51:15 crc kubenswrapper[5052]: I1203 12:51:15.806845 5052 reconciler_common.go:293] "Volume detached for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/83062467-67e1-4cab-b536-6edf3ea916b4-credential-keys\") on node \"crc\" DevicePath \"\"" Dec 03 12:51:15 crc kubenswrapper[5052]: I1203 12:51:15.806866 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-99mb9\" (UniqueName: \"kubernetes.io/projected/83062467-67e1-4cab-b536-6edf3ea916b4-kube-api-access-99mb9\") on node \"crc\" DevicePath \"\"" Dec 03 12:51:15 crc kubenswrapper[5052]: I1203 12:51:15.806879 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-lnhfk\" (UniqueName: \"kubernetes.io/projected/2fab7846-f3ad-4673-b813-36c20d4b7611-kube-api-access-lnhfk\") on node \"crc\" DevicePath \"\"" Dec 03 12:51:15 crc kubenswrapper[5052]: I1203 12:51:15.806893 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/83062467-67e1-4cab-b536-6edf3ea916b4-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:51:15 crc kubenswrapper[5052]: I1203 12:51:15.806905 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2fab7846-f3ad-4673-b813-36c20d4b7611-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:51:15 crc kubenswrapper[5052]: I1203 12:51:15.806916 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/83062467-67e1-4cab-b536-6edf3ea916b4-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:51:15 crc kubenswrapper[5052]: I1203 12:51:15.806929 5052 reconciler_common.go:293] "Volume detached for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/2fab7846-f3ad-4673-b813-36c20d4b7611-db-sync-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:51:15 crc kubenswrapper[5052]: I1203 12:51:15.806943 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/83062467-67e1-4cab-b536-6edf3ea916b4-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:51:16 crc kubenswrapper[5052]: I1203 12:51:16.050429 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-db-sync-rsmgs" event={"ID":"2fab7846-f3ad-4673-b813-36c20d4b7611","Type":"ContainerDied","Data":"36fdcc55bb30d997fc9fb3be40b70b2d853ca78f5277296f366a15b6a0cd179f"} Dec 03 12:51:16 crc kubenswrapper[5052]: I1203 12:51:16.050750 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="36fdcc55bb30d997fc9fb3be40b70b2d853ca78f5277296f366a15b6a0cd179f" Dec 03 12:51:16 crc kubenswrapper[5052]: I1203 12:51:16.050449 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-db-sync-rsmgs" Dec 03 12:51:16 crc kubenswrapper[5052]: I1203 12:51:16.055415 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"fc57d1ed-34a0-42b5-b1be-5bf914492701","Type":"ContainerStarted","Data":"5e85ff655790c57c0d7ecc6f00fca41e5f274eea76a18c0b7ebbd12438ea1d83"} Dec 03 12:51:16 crc kubenswrapper[5052]: I1203 12:51:16.055467 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"fc57d1ed-34a0-42b5-b1be-5bf914492701","Type":"ContainerStarted","Data":"04405b5bbd959325af164a93fe039b735439fcd2c5549ded39bf0b3f9298962f"} Dec 03 12:51:16 crc kubenswrapper[5052]: I1203 12:51:16.059576 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-bootstrap-mgkjf" event={"ID":"83062467-67e1-4cab-b536-6edf3ea916b4","Type":"ContainerDied","Data":"40e91b05245027d186c016daca121b70283f9ca6be3d354d07c4aab376ef7d66"} Dec 03 12:51:16 crc kubenswrapper[5052]: I1203 12:51:16.059647 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-bootstrap-mgkjf" Dec 03 12:51:16 crc kubenswrapper[5052]: I1203 12:51:16.059652 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="40e91b05245027d186c016daca121b70283f9ca6be3d354d07c4aab376ef7d66" Dec 03 12:51:16 crc kubenswrapper[5052]: I1203 12:51:16.082531 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"cd023d37-ce83-4ca4-ab52-f89e94d0b721","Type":"ContainerStarted","Data":"1ffdd15a3a5713be01c664e2b6740ea79e1cb391fcfc3f99d37c27c80a734363"} Dec 03 12:51:16 crc kubenswrapper[5052]: I1203 12:51:16.112196 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/glance-default-external-api-0" podStartSLOduration=4.112176885 podStartE2EDuration="4.112176885s" podCreationTimestamp="2025-12-03 12:51:12 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:51:16.103889898 +0000 UTC m=+5466.692131662" watchObservedRunningTime="2025-12-03 12:51:16.112176885 +0000 UTC m=+5466.700418639" Dec 03 12:51:16 crc kubenswrapper[5052]: I1203 12:51:16.113131 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:51:16 crc kubenswrapper[5052]: W1203 12:51:16.118671 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf3733be6_a76c_4c85_ad81_58ca3f3e3d2d.slice/crio-59d9cca171d0e5f3ea0e246db187ae008de4e9af8261af120aa2d4a36e96f396 WatchSource:0}: Error finding container 59d9cca171d0e5f3ea0e246db187ae008de4e9af8261af120aa2d4a36e96f396: Status 404 returned error can't find the container with id 59d9cca171d0e5f3ea0e246db187ae008de4e9af8261af120aa2d4a36e96f396 Dec 03 12:51:16 crc kubenswrapper[5052]: I1203 12:51:16.179144 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/keystone-bootstrap-mgkjf"] Dec 03 12:51:16 crc kubenswrapper[5052]: I1203 12:51:16.190139 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/keystone-bootstrap-mgkjf"] Dec 03 12:51:16 crc kubenswrapper[5052]: I1203 12:51:16.200414 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/placement-7b465d48b8-4bj69"] Dec 03 12:51:16 crc kubenswrapper[5052]: I1203 12:51:16.255659 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/barbican-keystone-listener-7b66cf7bb6-fsc4x"] Dec 03 12:51:16 crc kubenswrapper[5052]: E1203 12:51:16.256148 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="83062467-67e1-4cab-b536-6edf3ea916b4" containerName="keystone-bootstrap" Dec 03 12:51:16 crc kubenswrapper[5052]: I1203 12:51:16.256169 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="83062467-67e1-4cab-b536-6edf3ea916b4" containerName="keystone-bootstrap" Dec 03 12:51:16 crc kubenswrapper[5052]: E1203 12:51:16.256217 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="2fab7846-f3ad-4673-b813-36c20d4b7611" containerName="barbican-db-sync" Dec 03 12:51:16 crc kubenswrapper[5052]: I1203 12:51:16.256226 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="2fab7846-f3ad-4673-b813-36c20d4b7611" containerName="barbican-db-sync" Dec 03 12:51:16 crc kubenswrapper[5052]: I1203 12:51:16.256436 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="2fab7846-f3ad-4673-b813-36c20d4b7611" containerName="barbican-db-sync" Dec 03 12:51:16 crc kubenswrapper[5052]: I1203 12:51:16.256454 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="83062467-67e1-4cab-b536-6edf3ea916b4" containerName="keystone-bootstrap" Dec 03 12:51:16 crc kubenswrapper[5052]: I1203 12:51:16.257381 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-keystone-listener-7b66cf7bb6-fsc4x" Dec 03 12:51:16 crc kubenswrapper[5052]: I1203 12:51:16.260650 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"barbican-keystone-listener-config-data" Dec 03 12:51:16 crc kubenswrapper[5052]: I1203 12:51:16.262640 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"barbican-config-data" Dec 03 12:51:16 crc kubenswrapper[5052]: I1203 12:51:16.262798 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"barbican-barbican-dockercfg-74cwb" Dec 03 12:51:16 crc kubenswrapper[5052]: I1203 12:51:16.312263 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/keystone-bootstrap-d2z7m"] Dec 03 12:51:16 crc kubenswrapper[5052]: I1203 12:51:16.315091 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-bootstrap-d2z7m" Dec 03 12:51:16 crc kubenswrapper[5052]: W1203 12:51:16.320214 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8df65ee4_b91d_4ffd_bff2_65e5e6559130.slice/crio-97761280aaf2fdc7736eab0f506725b769a3dfd0ac8b0fcd7a289fffa429cb9a WatchSource:0}: Error finding container 97761280aaf2fdc7736eab0f506725b769a3dfd0ac8b0fcd7a289fffa429cb9a: Status 404 returned error can't find the container with id 97761280aaf2fdc7736eab0f506725b769a3dfd0ac8b0fcd7a289fffa429cb9a Dec 03 12:51:16 crc kubenswrapper[5052]: I1203 12:51:16.325626 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/020bfa4c-3dba-43d2-9ac2-0bbbae660709-config-data-custom\") pod \"barbican-keystone-listener-7b66cf7bb6-fsc4x\" (UID: \"020bfa4c-3dba-43d2-9ac2-0bbbae660709\") " pod="openstack-kuttl-tests/barbican-keystone-listener-7b66cf7bb6-fsc4x" Dec 03 12:51:16 crc kubenswrapper[5052]: I1203 12:51:16.325668 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/020bfa4c-3dba-43d2-9ac2-0bbbae660709-logs\") pod \"barbican-keystone-listener-7b66cf7bb6-fsc4x\" (UID: \"020bfa4c-3dba-43d2-9ac2-0bbbae660709\") " pod="openstack-kuttl-tests/barbican-keystone-listener-7b66cf7bb6-fsc4x" Dec 03 12:51:16 crc kubenswrapper[5052]: I1203 12:51:16.326445 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/020bfa4c-3dba-43d2-9ac2-0bbbae660709-combined-ca-bundle\") pod \"barbican-keystone-listener-7b66cf7bb6-fsc4x\" (UID: \"020bfa4c-3dba-43d2-9ac2-0bbbae660709\") " pod="openstack-kuttl-tests/barbican-keystone-listener-7b66cf7bb6-fsc4x" Dec 03 12:51:16 crc kubenswrapper[5052]: I1203 12:51:16.326480 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fjbl5\" (UniqueName: \"kubernetes.io/projected/020bfa4c-3dba-43d2-9ac2-0bbbae660709-kube-api-access-fjbl5\") pod \"barbican-keystone-listener-7b66cf7bb6-fsc4x\" (UID: \"020bfa4c-3dba-43d2-9ac2-0bbbae660709\") " pod="openstack-kuttl-tests/barbican-keystone-listener-7b66cf7bb6-fsc4x" Dec 03 12:51:16 crc kubenswrapper[5052]: I1203 12:51:16.326559 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/020bfa4c-3dba-43d2-9ac2-0bbbae660709-config-data\") pod \"barbican-keystone-listener-7b66cf7bb6-fsc4x\" (UID: \"020bfa4c-3dba-43d2-9ac2-0bbbae660709\") " pod="openstack-kuttl-tests/barbican-keystone-listener-7b66cf7bb6-fsc4x" Dec 03 12:51:16 crc kubenswrapper[5052]: I1203 12:51:16.345326 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone" Dec 03 12:51:16 crc kubenswrapper[5052]: I1203 12:51:16.345792 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone-config-data" Dec 03 12:51:16 crc kubenswrapper[5052]: I1203 12:51:16.346040 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"osp-secret" Dec 03 12:51:16 crc kubenswrapper[5052]: I1203 12:51:16.347024 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone-keystone-dockercfg-fr5sp" Dec 03 12:51:16 crc kubenswrapper[5052]: I1203 12:51:16.358409 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone-scripts" Dec 03 12:51:16 crc kubenswrapper[5052]: I1203 12:51:16.380920 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-keystone-listener-7b66cf7bb6-fsc4x"] Dec 03 12:51:16 crc kubenswrapper[5052]: I1203 12:51:16.397382 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/barbican-worker-b7f5dfb6f-pj6bx"] Dec 03 12:51:16 crc kubenswrapper[5052]: I1203 12:51:16.399442 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-worker-b7f5dfb6f-pj6bx" Dec 03 12:51:16 crc kubenswrapper[5052]: I1203 12:51:16.401081 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"barbican-worker-config-data" Dec 03 12:51:16 crc kubenswrapper[5052]: I1203 12:51:16.427684 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/cd29fd9a-ac8a-4424-b6ac-e4b81bf292f4-scripts\") pod \"keystone-bootstrap-d2z7m\" (UID: \"cd29fd9a-ac8a-4424-b6ac-e4b81bf292f4\") " pod="openstack-kuttl-tests/keystone-bootstrap-d2z7m" Dec 03 12:51:16 crc kubenswrapper[5052]: I1203 12:51:16.427762 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dpdq2\" (UniqueName: \"kubernetes.io/projected/cd29fd9a-ac8a-4424-b6ac-e4b81bf292f4-kube-api-access-dpdq2\") pod \"keystone-bootstrap-d2z7m\" (UID: \"cd29fd9a-ac8a-4424-b6ac-e4b81bf292f4\") " pod="openstack-kuttl-tests/keystone-bootstrap-d2z7m" Dec 03 12:51:16 crc kubenswrapper[5052]: I1203 12:51:16.427831 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/020bfa4c-3dba-43d2-9ac2-0bbbae660709-config-data\") pod \"barbican-keystone-listener-7b66cf7bb6-fsc4x\" (UID: \"020bfa4c-3dba-43d2-9ac2-0bbbae660709\") " pod="openstack-kuttl-tests/barbican-keystone-listener-7b66cf7bb6-fsc4x" Dec 03 12:51:16 crc kubenswrapper[5052]: I1203 12:51:16.427854 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/cd29fd9a-ac8a-4424-b6ac-e4b81bf292f4-credential-keys\") pod \"keystone-bootstrap-d2z7m\" (UID: \"cd29fd9a-ac8a-4424-b6ac-e4b81bf292f4\") " pod="openstack-kuttl-tests/keystone-bootstrap-d2z7m" Dec 03 12:51:16 crc kubenswrapper[5052]: I1203 12:51:16.428183 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/020bfa4c-3dba-43d2-9ac2-0bbbae660709-config-data-custom\") pod \"barbican-keystone-listener-7b66cf7bb6-fsc4x\" (UID: \"020bfa4c-3dba-43d2-9ac2-0bbbae660709\") " pod="openstack-kuttl-tests/barbican-keystone-listener-7b66cf7bb6-fsc4x" Dec 03 12:51:16 crc kubenswrapper[5052]: I1203 12:51:16.428218 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/020bfa4c-3dba-43d2-9ac2-0bbbae660709-logs\") pod \"barbican-keystone-listener-7b66cf7bb6-fsc4x\" (UID: \"020bfa4c-3dba-43d2-9ac2-0bbbae660709\") " pod="openstack-kuttl-tests/barbican-keystone-listener-7b66cf7bb6-fsc4x" Dec 03 12:51:16 crc kubenswrapper[5052]: I1203 12:51:16.430608 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/cd29fd9a-ac8a-4424-b6ac-e4b81bf292f4-config-data\") pod \"keystone-bootstrap-d2z7m\" (UID: \"cd29fd9a-ac8a-4424-b6ac-e4b81bf292f4\") " pod="openstack-kuttl-tests/keystone-bootstrap-d2z7m" Dec 03 12:51:16 crc kubenswrapper[5052]: I1203 12:51:16.430644 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/cd29fd9a-ac8a-4424-b6ac-e4b81bf292f4-fernet-keys\") pod \"keystone-bootstrap-d2z7m\" (UID: \"cd29fd9a-ac8a-4424-b6ac-e4b81bf292f4\") " pod="openstack-kuttl-tests/keystone-bootstrap-d2z7m" Dec 03 12:51:16 crc kubenswrapper[5052]: I1203 12:51:16.430692 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/cd29fd9a-ac8a-4424-b6ac-e4b81bf292f4-combined-ca-bundle\") pod \"keystone-bootstrap-d2z7m\" (UID: \"cd29fd9a-ac8a-4424-b6ac-e4b81bf292f4\") " pod="openstack-kuttl-tests/keystone-bootstrap-d2z7m" Dec 03 12:51:16 crc kubenswrapper[5052]: I1203 12:51:16.430724 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/020bfa4c-3dba-43d2-9ac2-0bbbae660709-combined-ca-bundle\") pod \"barbican-keystone-listener-7b66cf7bb6-fsc4x\" (UID: \"020bfa4c-3dba-43d2-9ac2-0bbbae660709\") " pod="openstack-kuttl-tests/barbican-keystone-listener-7b66cf7bb6-fsc4x" Dec 03 12:51:16 crc kubenswrapper[5052]: I1203 12:51:16.430762 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-fjbl5\" (UniqueName: \"kubernetes.io/projected/020bfa4c-3dba-43d2-9ac2-0bbbae660709-kube-api-access-fjbl5\") pod \"barbican-keystone-listener-7b66cf7bb6-fsc4x\" (UID: \"020bfa4c-3dba-43d2-9ac2-0bbbae660709\") " pod="openstack-kuttl-tests/barbican-keystone-listener-7b66cf7bb6-fsc4x" Dec 03 12:51:16 crc kubenswrapper[5052]: I1203 12:51:16.431027 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/020bfa4c-3dba-43d2-9ac2-0bbbae660709-logs\") pod \"barbican-keystone-listener-7b66cf7bb6-fsc4x\" (UID: \"020bfa4c-3dba-43d2-9ac2-0bbbae660709\") " pod="openstack-kuttl-tests/barbican-keystone-listener-7b66cf7bb6-fsc4x" Dec 03 12:51:16 crc kubenswrapper[5052]: I1203 12:51:16.433905 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/020bfa4c-3dba-43d2-9ac2-0bbbae660709-config-data\") pod \"barbican-keystone-listener-7b66cf7bb6-fsc4x\" (UID: \"020bfa4c-3dba-43d2-9ac2-0bbbae660709\") " pod="openstack-kuttl-tests/barbican-keystone-listener-7b66cf7bb6-fsc4x" Dec 03 12:51:16 crc kubenswrapper[5052]: I1203 12:51:16.440094 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/020bfa4c-3dba-43d2-9ac2-0bbbae660709-config-data-custom\") pod \"barbican-keystone-listener-7b66cf7bb6-fsc4x\" (UID: \"020bfa4c-3dba-43d2-9ac2-0bbbae660709\") " pod="openstack-kuttl-tests/barbican-keystone-listener-7b66cf7bb6-fsc4x" Dec 03 12:51:16 crc kubenswrapper[5052]: I1203 12:51:16.445351 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/020bfa4c-3dba-43d2-9ac2-0bbbae660709-combined-ca-bundle\") pod \"barbican-keystone-listener-7b66cf7bb6-fsc4x\" (UID: \"020bfa4c-3dba-43d2-9ac2-0bbbae660709\") " pod="openstack-kuttl-tests/barbican-keystone-listener-7b66cf7bb6-fsc4x" Dec 03 12:51:16 crc kubenswrapper[5052]: I1203 12:51:16.453876 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-fjbl5\" (UniqueName: \"kubernetes.io/projected/020bfa4c-3dba-43d2-9ac2-0bbbae660709-kube-api-access-fjbl5\") pod \"barbican-keystone-listener-7b66cf7bb6-fsc4x\" (UID: \"020bfa4c-3dba-43d2-9ac2-0bbbae660709\") " pod="openstack-kuttl-tests/barbican-keystone-listener-7b66cf7bb6-fsc4x" Dec 03 12:51:16 crc kubenswrapper[5052]: I1203 12:51:16.468884 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-worker-b7f5dfb6f-pj6bx"] Dec 03 12:51:16 crc kubenswrapper[5052]: I1203 12:51:16.481290 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-bootstrap-d2z7m"] Dec 03 12:51:16 crc kubenswrapper[5052]: I1203 12:51:16.506446 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/barbican-api-5bdcd5dc6-vqlkq"] Dec 03 12:51:16 crc kubenswrapper[5052]: I1203 12:51:16.515935 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-api-5bdcd5dc6-vqlkq" Dec 03 12:51:16 crc kubenswrapper[5052]: I1203 12:51:16.519332 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-api-5bdcd5dc6-vqlkq"] Dec 03 12:51:16 crc kubenswrapper[5052]: I1203 12:51:16.521331 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"barbican-api-config-data" Dec 03 12:51:16 crc kubenswrapper[5052]: I1203 12:51:16.532901 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/b2933134-7226-423a-8120-3148845b808a-config-data-custom\") pod \"barbican-worker-b7f5dfb6f-pj6bx\" (UID: \"b2933134-7226-423a-8120-3148845b808a\") " pod="openstack-kuttl-tests/barbican-worker-b7f5dfb6f-pj6bx" Dec 03 12:51:16 crc kubenswrapper[5052]: I1203 12:51:16.532977 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/cd29fd9a-ac8a-4424-b6ac-e4b81bf292f4-scripts\") pod \"keystone-bootstrap-d2z7m\" (UID: \"cd29fd9a-ac8a-4424-b6ac-e4b81bf292f4\") " pod="openstack-kuttl-tests/keystone-bootstrap-d2z7m" Dec 03 12:51:16 crc kubenswrapper[5052]: I1203 12:51:16.533013 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-dpdq2\" (UniqueName: \"kubernetes.io/projected/cd29fd9a-ac8a-4424-b6ac-e4b81bf292f4-kube-api-access-dpdq2\") pod \"keystone-bootstrap-d2z7m\" (UID: \"cd29fd9a-ac8a-4424-b6ac-e4b81bf292f4\") " pod="openstack-kuttl-tests/keystone-bootstrap-d2z7m" Dec 03 12:51:16 crc kubenswrapper[5052]: I1203 12:51:16.533048 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/cd29fd9a-ac8a-4424-b6ac-e4b81bf292f4-credential-keys\") pod \"keystone-bootstrap-d2z7m\" (UID: \"cd29fd9a-ac8a-4424-b6ac-e4b81bf292f4\") " pod="openstack-kuttl-tests/keystone-bootstrap-d2z7m" Dec 03 12:51:16 crc kubenswrapper[5052]: I1203 12:51:16.533087 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8t9x7\" (UniqueName: \"kubernetes.io/projected/b2933134-7226-423a-8120-3148845b808a-kube-api-access-8t9x7\") pod \"barbican-worker-b7f5dfb6f-pj6bx\" (UID: \"b2933134-7226-423a-8120-3148845b808a\") " pod="openstack-kuttl-tests/barbican-worker-b7f5dfb6f-pj6bx" Dec 03 12:51:16 crc kubenswrapper[5052]: I1203 12:51:16.533130 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b2933134-7226-423a-8120-3148845b808a-combined-ca-bundle\") pod \"barbican-worker-b7f5dfb6f-pj6bx\" (UID: \"b2933134-7226-423a-8120-3148845b808a\") " pod="openstack-kuttl-tests/barbican-worker-b7f5dfb6f-pj6bx" Dec 03 12:51:16 crc kubenswrapper[5052]: I1203 12:51:16.533207 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/b2933134-7226-423a-8120-3148845b808a-config-data\") pod \"barbican-worker-b7f5dfb6f-pj6bx\" (UID: \"b2933134-7226-423a-8120-3148845b808a\") " pod="openstack-kuttl-tests/barbican-worker-b7f5dfb6f-pj6bx" Dec 03 12:51:16 crc kubenswrapper[5052]: I1203 12:51:16.533251 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/cd29fd9a-ac8a-4424-b6ac-e4b81bf292f4-config-data\") pod \"keystone-bootstrap-d2z7m\" (UID: \"cd29fd9a-ac8a-4424-b6ac-e4b81bf292f4\") " pod="openstack-kuttl-tests/keystone-bootstrap-d2z7m" Dec 03 12:51:16 crc kubenswrapper[5052]: I1203 12:51:16.533277 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/cd29fd9a-ac8a-4424-b6ac-e4b81bf292f4-fernet-keys\") pod \"keystone-bootstrap-d2z7m\" (UID: \"cd29fd9a-ac8a-4424-b6ac-e4b81bf292f4\") " pod="openstack-kuttl-tests/keystone-bootstrap-d2z7m" Dec 03 12:51:16 crc kubenswrapper[5052]: I1203 12:51:16.533309 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/b2933134-7226-423a-8120-3148845b808a-logs\") pod \"barbican-worker-b7f5dfb6f-pj6bx\" (UID: \"b2933134-7226-423a-8120-3148845b808a\") " pod="openstack-kuttl-tests/barbican-worker-b7f5dfb6f-pj6bx" Dec 03 12:51:16 crc kubenswrapper[5052]: I1203 12:51:16.533337 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/cd29fd9a-ac8a-4424-b6ac-e4b81bf292f4-combined-ca-bundle\") pod \"keystone-bootstrap-d2z7m\" (UID: \"cd29fd9a-ac8a-4424-b6ac-e4b81bf292f4\") " pod="openstack-kuttl-tests/keystone-bootstrap-d2z7m" Dec 03 12:51:16 crc kubenswrapper[5052]: I1203 12:51:16.537919 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/cd29fd9a-ac8a-4424-b6ac-e4b81bf292f4-scripts\") pod \"keystone-bootstrap-d2z7m\" (UID: \"cd29fd9a-ac8a-4424-b6ac-e4b81bf292f4\") " pod="openstack-kuttl-tests/keystone-bootstrap-d2z7m" Dec 03 12:51:16 crc kubenswrapper[5052]: I1203 12:51:16.539970 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/cd29fd9a-ac8a-4424-b6ac-e4b81bf292f4-combined-ca-bundle\") pod \"keystone-bootstrap-d2z7m\" (UID: \"cd29fd9a-ac8a-4424-b6ac-e4b81bf292f4\") " pod="openstack-kuttl-tests/keystone-bootstrap-d2z7m" Dec 03 12:51:16 crc kubenswrapper[5052]: I1203 12:51:16.541162 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/cd29fd9a-ac8a-4424-b6ac-e4b81bf292f4-credential-keys\") pod \"keystone-bootstrap-d2z7m\" (UID: \"cd29fd9a-ac8a-4424-b6ac-e4b81bf292f4\") " pod="openstack-kuttl-tests/keystone-bootstrap-d2z7m" Dec 03 12:51:16 crc kubenswrapper[5052]: I1203 12:51:16.543940 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/cd29fd9a-ac8a-4424-b6ac-e4b81bf292f4-fernet-keys\") pod \"keystone-bootstrap-d2z7m\" (UID: \"cd29fd9a-ac8a-4424-b6ac-e4b81bf292f4\") " pod="openstack-kuttl-tests/keystone-bootstrap-d2z7m" Dec 03 12:51:16 crc kubenswrapper[5052]: I1203 12:51:16.546492 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/cd29fd9a-ac8a-4424-b6ac-e4b81bf292f4-config-data\") pod \"keystone-bootstrap-d2z7m\" (UID: \"cd29fd9a-ac8a-4424-b6ac-e4b81bf292f4\") " pod="openstack-kuttl-tests/keystone-bootstrap-d2z7m" Dec 03 12:51:16 crc kubenswrapper[5052]: I1203 12:51:16.561703 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-dpdq2\" (UniqueName: \"kubernetes.io/projected/cd29fd9a-ac8a-4424-b6ac-e4b81bf292f4-kube-api-access-dpdq2\") pod \"keystone-bootstrap-d2z7m\" (UID: \"cd29fd9a-ac8a-4424-b6ac-e4b81bf292f4\") " pod="openstack-kuttl-tests/keystone-bootstrap-d2z7m" Dec 03 12:51:16 crc kubenswrapper[5052]: I1203 12:51:16.635365 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-8t9x7\" (UniqueName: \"kubernetes.io/projected/b2933134-7226-423a-8120-3148845b808a-kube-api-access-8t9x7\") pod \"barbican-worker-b7f5dfb6f-pj6bx\" (UID: \"b2933134-7226-423a-8120-3148845b808a\") " pod="openstack-kuttl-tests/barbican-worker-b7f5dfb6f-pj6bx" Dec 03 12:51:16 crc kubenswrapper[5052]: I1203 12:51:16.635444 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/539d9404-7518-4f03-815d-568a43eedb0b-config-data-custom\") pod \"barbican-api-5bdcd5dc6-vqlkq\" (UID: \"539d9404-7518-4f03-815d-568a43eedb0b\") " pod="openstack-kuttl-tests/barbican-api-5bdcd5dc6-vqlkq" Dec 03 12:51:16 crc kubenswrapper[5052]: I1203 12:51:16.635480 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b2933134-7226-423a-8120-3148845b808a-combined-ca-bundle\") pod \"barbican-worker-b7f5dfb6f-pj6bx\" (UID: \"b2933134-7226-423a-8120-3148845b808a\") " pod="openstack-kuttl-tests/barbican-worker-b7f5dfb6f-pj6bx" Dec 03 12:51:16 crc kubenswrapper[5052]: I1203 12:51:16.635535 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kw6x2\" (UniqueName: \"kubernetes.io/projected/539d9404-7518-4f03-815d-568a43eedb0b-kube-api-access-kw6x2\") pod \"barbican-api-5bdcd5dc6-vqlkq\" (UID: \"539d9404-7518-4f03-815d-568a43eedb0b\") " pod="openstack-kuttl-tests/barbican-api-5bdcd5dc6-vqlkq" Dec 03 12:51:16 crc kubenswrapper[5052]: I1203 12:51:16.635554 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/539d9404-7518-4f03-815d-568a43eedb0b-logs\") pod \"barbican-api-5bdcd5dc6-vqlkq\" (UID: \"539d9404-7518-4f03-815d-568a43eedb0b\") " pod="openstack-kuttl-tests/barbican-api-5bdcd5dc6-vqlkq" Dec 03 12:51:16 crc kubenswrapper[5052]: I1203 12:51:16.635589 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/b2933134-7226-423a-8120-3148845b808a-config-data\") pod \"barbican-worker-b7f5dfb6f-pj6bx\" (UID: \"b2933134-7226-423a-8120-3148845b808a\") " pod="openstack-kuttl-tests/barbican-worker-b7f5dfb6f-pj6bx" Dec 03 12:51:16 crc kubenswrapper[5052]: I1203 12:51:16.635626 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/b2933134-7226-423a-8120-3148845b808a-logs\") pod \"barbican-worker-b7f5dfb6f-pj6bx\" (UID: \"b2933134-7226-423a-8120-3148845b808a\") " pod="openstack-kuttl-tests/barbican-worker-b7f5dfb6f-pj6bx" Dec 03 12:51:16 crc kubenswrapper[5052]: I1203 12:51:16.635664 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/539d9404-7518-4f03-815d-568a43eedb0b-combined-ca-bundle\") pod \"barbican-api-5bdcd5dc6-vqlkq\" (UID: \"539d9404-7518-4f03-815d-568a43eedb0b\") " pod="openstack-kuttl-tests/barbican-api-5bdcd5dc6-vqlkq" Dec 03 12:51:16 crc kubenswrapper[5052]: I1203 12:51:16.635695 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/539d9404-7518-4f03-815d-568a43eedb0b-config-data\") pod \"barbican-api-5bdcd5dc6-vqlkq\" (UID: \"539d9404-7518-4f03-815d-568a43eedb0b\") " pod="openstack-kuttl-tests/barbican-api-5bdcd5dc6-vqlkq" Dec 03 12:51:16 crc kubenswrapper[5052]: I1203 12:51:16.635720 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/b2933134-7226-423a-8120-3148845b808a-config-data-custom\") pod \"barbican-worker-b7f5dfb6f-pj6bx\" (UID: \"b2933134-7226-423a-8120-3148845b808a\") " pod="openstack-kuttl-tests/barbican-worker-b7f5dfb6f-pj6bx" Dec 03 12:51:16 crc kubenswrapper[5052]: I1203 12:51:16.636563 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/b2933134-7226-423a-8120-3148845b808a-logs\") pod \"barbican-worker-b7f5dfb6f-pj6bx\" (UID: \"b2933134-7226-423a-8120-3148845b808a\") " pod="openstack-kuttl-tests/barbican-worker-b7f5dfb6f-pj6bx" Dec 03 12:51:16 crc kubenswrapper[5052]: I1203 12:51:16.645192 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-keystone-listener-7b66cf7bb6-fsc4x" Dec 03 12:51:16 crc kubenswrapper[5052]: I1203 12:51:16.646881 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/b2933134-7226-423a-8120-3148845b808a-config-data-custom\") pod \"barbican-worker-b7f5dfb6f-pj6bx\" (UID: \"b2933134-7226-423a-8120-3148845b808a\") " pod="openstack-kuttl-tests/barbican-worker-b7f5dfb6f-pj6bx" Dec 03 12:51:16 crc kubenswrapper[5052]: I1203 12:51:16.648540 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/b2933134-7226-423a-8120-3148845b808a-config-data\") pod \"barbican-worker-b7f5dfb6f-pj6bx\" (UID: \"b2933134-7226-423a-8120-3148845b808a\") " pod="openstack-kuttl-tests/barbican-worker-b7f5dfb6f-pj6bx" Dec 03 12:51:16 crc kubenswrapper[5052]: I1203 12:51:16.651261 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b2933134-7226-423a-8120-3148845b808a-combined-ca-bundle\") pod \"barbican-worker-b7f5dfb6f-pj6bx\" (UID: \"b2933134-7226-423a-8120-3148845b808a\") " pod="openstack-kuttl-tests/barbican-worker-b7f5dfb6f-pj6bx" Dec 03 12:51:16 crc kubenswrapper[5052]: I1203 12:51:16.665295 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-8t9x7\" (UniqueName: \"kubernetes.io/projected/b2933134-7226-423a-8120-3148845b808a-kube-api-access-8t9x7\") pod \"barbican-worker-b7f5dfb6f-pj6bx\" (UID: \"b2933134-7226-423a-8120-3148845b808a\") " pod="openstack-kuttl-tests/barbican-worker-b7f5dfb6f-pj6bx" Dec 03 12:51:16 crc kubenswrapper[5052]: I1203 12:51:16.737356 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/539d9404-7518-4f03-815d-568a43eedb0b-combined-ca-bundle\") pod \"barbican-api-5bdcd5dc6-vqlkq\" (UID: \"539d9404-7518-4f03-815d-568a43eedb0b\") " pod="openstack-kuttl-tests/barbican-api-5bdcd5dc6-vqlkq" Dec 03 12:51:16 crc kubenswrapper[5052]: I1203 12:51:16.737470 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/539d9404-7518-4f03-815d-568a43eedb0b-config-data\") pod \"barbican-api-5bdcd5dc6-vqlkq\" (UID: \"539d9404-7518-4f03-815d-568a43eedb0b\") " pod="openstack-kuttl-tests/barbican-api-5bdcd5dc6-vqlkq" Dec 03 12:51:16 crc kubenswrapper[5052]: I1203 12:51:16.739878 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/539d9404-7518-4f03-815d-568a43eedb0b-config-data-custom\") pod \"barbican-api-5bdcd5dc6-vqlkq\" (UID: \"539d9404-7518-4f03-815d-568a43eedb0b\") " pod="openstack-kuttl-tests/barbican-api-5bdcd5dc6-vqlkq" Dec 03 12:51:16 crc kubenswrapper[5052]: I1203 12:51:16.740418 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-kw6x2\" (UniqueName: \"kubernetes.io/projected/539d9404-7518-4f03-815d-568a43eedb0b-kube-api-access-kw6x2\") pod \"barbican-api-5bdcd5dc6-vqlkq\" (UID: \"539d9404-7518-4f03-815d-568a43eedb0b\") " pod="openstack-kuttl-tests/barbican-api-5bdcd5dc6-vqlkq" Dec 03 12:51:16 crc kubenswrapper[5052]: I1203 12:51:16.740505 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/539d9404-7518-4f03-815d-568a43eedb0b-logs\") pod \"barbican-api-5bdcd5dc6-vqlkq\" (UID: \"539d9404-7518-4f03-815d-568a43eedb0b\") " pod="openstack-kuttl-tests/barbican-api-5bdcd5dc6-vqlkq" Dec 03 12:51:16 crc kubenswrapper[5052]: I1203 12:51:16.741351 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-bootstrap-d2z7m" Dec 03 12:51:16 crc kubenswrapper[5052]: I1203 12:51:16.741686 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/539d9404-7518-4f03-815d-568a43eedb0b-logs\") pod \"barbican-api-5bdcd5dc6-vqlkq\" (UID: \"539d9404-7518-4f03-815d-568a43eedb0b\") " pod="openstack-kuttl-tests/barbican-api-5bdcd5dc6-vqlkq" Dec 03 12:51:16 crc kubenswrapper[5052]: I1203 12:51:16.745045 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/539d9404-7518-4f03-815d-568a43eedb0b-config-data\") pod \"barbican-api-5bdcd5dc6-vqlkq\" (UID: \"539d9404-7518-4f03-815d-568a43eedb0b\") " pod="openstack-kuttl-tests/barbican-api-5bdcd5dc6-vqlkq" Dec 03 12:51:16 crc kubenswrapper[5052]: I1203 12:51:16.745093 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/539d9404-7518-4f03-815d-568a43eedb0b-combined-ca-bundle\") pod \"barbican-api-5bdcd5dc6-vqlkq\" (UID: \"539d9404-7518-4f03-815d-568a43eedb0b\") " pod="openstack-kuttl-tests/barbican-api-5bdcd5dc6-vqlkq" Dec 03 12:51:16 crc kubenswrapper[5052]: I1203 12:51:16.747316 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/539d9404-7518-4f03-815d-568a43eedb0b-config-data-custom\") pod \"barbican-api-5bdcd5dc6-vqlkq\" (UID: \"539d9404-7518-4f03-815d-568a43eedb0b\") " pod="openstack-kuttl-tests/barbican-api-5bdcd5dc6-vqlkq" Dec 03 12:51:16 crc kubenswrapper[5052]: I1203 12:51:16.764304 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-kw6x2\" (UniqueName: \"kubernetes.io/projected/539d9404-7518-4f03-815d-568a43eedb0b-kube-api-access-kw6x2\") pod \"barbican-api-5bdcd5dc6-vqlkq\" (UID: \"539d9404-7518-4f03-815d-568a43eedb0b\") " pod="openstack-kuttl-tests/barbican-api-5bdcd5dc6-vqlkq" Dec 03 12:51:16 crc kubenswrapper[5052]: I1203 12:51:16.797223 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-worker-b7f5dfb6f-pj6bx" Dec 03 12:51:16 crc kubenswrapper[5052]: I1203 12:51:16.859152 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-api-5bdcd5dc6-vqlkq" Dec 03 12:51:17 crc kubenswrapper[5052]: I1203 12:51:17.016430 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-keystone-listener-7b66cf7bb6-fsc4x"] Dec 03 12:51:17 crc kubenswrapper[5052]: I1203 12:51:17.112131 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-7b465d48b8-4bj69" event={"ID":"8df65ee4-b91d-4ffd-bff2-65e5e6559130","Type":"ContainerStarted","Data":"4cc0973d6be860027762d01ac994ad7b9ec6e3602b0a54d18cb4bc9da4071fbb"} Dec 03 12:51:17 crc kubenswrapper[5052]: I1203 12:51:17.112184 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-7b465d48b8-4bj69" event={"ID":"8df65ee4-b91d-4ffd-bff2-65e5e6559130","Type":"ContainerStarted","Data":"06f463922e62266ed96092189ec4fb9925ca28e41dfc81fbad694a526afdc991"} Dec 03 12:51:17 crc kubenswrapper[5052]: I1203 12:51:17.112201 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-7b465d48b8-4bj69" event={"ID":"8df65ee4-b91d-4ffd-bff2-65e5e6559130","Type":"ContainerStarted","Data":"97761280aaf2fdc7736eab0f506725b769a3dfd0ac8b0fcd7a289fffa429cb9a"} Dec 03 12:51:17 crc kubenswrapper[5052]: I1203 12:51:17.130005 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/placement-7b465d48b8-4bj69" Dec 03 12:51:17 crc kubenswrapper[5052]: I1203 12:51:17.130271 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/placement-7b465d48b8-4bj69" Dec 03 12:51:17 crc kubenswrapper[5052]: I1203 12:51:17.205088 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"fc57d1ed-34a0-42b5-b1be-5bf914492701","Type":"ContainerStarted","Data":"b2120f51b650e3bfe33d9abfe02816028d28060266d5498f4adcf7f4a0048dab"} Dec 03 12:51:17 crc kubenswrapper[5052]: I1203 12:51:17.227968 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-keystone-listener-7b66cf7bb6-fsc4x" event={"ID":"020bfa4c-3dba-43d2-9ac2-0bbbae660709","Type":"ContainerStarted","Data":"9e873a757d76efed4f69f46f02db362ab015158690671368a180b4a2cb792abd"} Dec 03 12:51:17 crc kubenswrapper[5052]: I1203 12:51:17.264889 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/placement-7b465d48b8-4bj69" podStartSLOduration=2.2648596899999998 podStartE2EDuration="2.26485969s" podCreationTimestamp="2025-12-03 12:51:15 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:51:17.193744456 +0000 UTC m=+5467.781986220" watchObservedRunningTime="2025-12-03 12:51:17.26485969 +0000 UTC m=+5467.853101454" Dec 03 12:51:17 crc kubenswrapper[5052]: I1203 12:51:17.282718 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/glance-default-internal-api-0" podStartSLOduration=3.28267951 podStartE2EDuration="3.28267951s" podCreationTimestamp="2025-12-03 12:51:14 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:51:17.241040129 +0000 UTC m=+5467.829281883" watchObservedRunningTime="2025-12-03 12:51:17.28267951 +0000 UTC m=+5467.870921294" Dec 03 12:51:17 crc kubenswrapper[5052]: I1203 12:51:17.284890 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"f3733be6-a76c-4c85-ad81-58ca3f3e3d2d","Type":"ContainerStarted","Data":"8fb1ea412deb631d1ed8163bda6feae84ba3cd74f0818f033958f4a2a30de564"} Dec 03 12:51:17 crc kubenswrapper[5052]: I1203 12:51:17.284928 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"f3733be6-a76c-4c85-ad81-58ca3f3e3d2d","Type":"ContainerStarted","Data":"59d9cca171d0e5f3ea0e246db187ae008de4e9af8261af120aa2d4a36e96f396"} Dec 03 12:51:17 crc kubenswrapper[5052]: I1203 12:51:17.360736 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-worker-b7f5dfb6f-pj6bx"] Dec 03 12:51:17 crc kubenswrapper[5052]: I1203 12:51:17.377586 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-bootstrap-d2z7m"] Dec 03 12:51:17 crc kubenswrapper[5052]: I1203 12:51:17.577063 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-api-5bdcd5dc6-vqlkq"] Dec 03 12:51:17 crc kubenswrapper[5052]: W1203 12:51:17.577455 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod539d9404_7518_4f03_815d_568a43eedb0b.slice/crio-dcaa6ec2052c2553cd436993c1572efa22c63b65ebb45e135ea74e0effccb612 WatchSource:0}: Error finding container dcaa6ec2052c2553cd436993c1572efa22c63b65ebb45e135ea74e0effccb612: Status 404 returned error can't find the container with id dcaa6ec2052c2553cd436993c1572efa22c63b65ebb45e135ea74e0effccb612 Dec 03 12:51:17 crc kubenswrapper[5052]: I1203 12:51:17.686928 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="83062467-67e1-4cab-b536-6edf3ea916b4" path="/var/lib/kubelet/pods/83062467-67e1-4cab-b536-6edf3ea916b4/volumes" Dec 03 12:51:18 crc kubenswrapper[5052]: I1203 12:51:18.294755 5052 generic.go:334] "Generic (PLEG): container finished" podID="8212a97d-b76a-4b60-b16b-914a4bbe5b9d" containerID="c424cb522d3b69b3e111498d4aa3be9ef046469f0fdba589af66d4e85ff8160a" exitCode=0 Dec 03 12:51:18 crc kubenswrapper[5052]: I1203 12:51:18.294866 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-db-sync-2b42x" event={"ID":"8212a97d-b76a-4b60-b16b-914a4bbe5b9d","Type":"ContainerDied","Data":"c424cb522d3b69b3e111498d4aa3be9ef046469f0fdba589af66d4e85ff8160a"} Dec 03 12:51:18 crc kubenswrapper[5052]: I1203 12:51:18.297851 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"f3733be6-a76c-4c85-ad81-58ca3f3e3d2d","Type":"ContainerStarted","Data":"262d5eb89c22ac292d99479055ae3d43a813219c6939b2f60660b7ee0bf8ae6d"} Dec 03 12:51:18 crc kubenswrapper[5052]: I1203 12:51:18.299320 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-bootstrap-d2z7m" event={"ID":"cd29fd9a-ac8a-4424-b6ac-e4b81bf292f4","Type":"ContainerStarted","Data":"b78075e349099e5081c2a933e5a387e49bcc03246f78f07abd5d255f951c6969"} Dec 03 12:51:18 crc kubenswrapper[5052]: I1203 12:51:18.299365 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-bootstrap-d2z7m" event={"ID":"cd29fd9a-ac8a-4424-b6ac-e4b81bf292f4","Type":"ContainerStarted","Data":"10936a6922451e4775ed678a70def6ea5bbab6cdd2a4c6978e1c1c7c1b4b2731"} Dec 03 12:51:18 crc kubenswrapper[5052]: I1203 12:51:18.301884 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-worker-b7f5dfb6f-pj6bx" event={"ID":"b2933134-7226-423a-8120-3148845b808a","Type":"ContainerStarted","Data":"6ef42ed115f2247455a35e6dba9e2519ebc77a9c515e4efa379aa58e493e2dea"} Dec 03 12:51:18 crc kubenswrapper[5052]: I1203 12:51:18.301928 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-worker-b7f5dfb6f-pj6bx" event={"ID":"b2933134-7226-423a-8120-3148845b808a","Type":"ContainerStarted","Data":"c45182c71bc38ac598875105b0f4d0cd30926ee88268d53941e68205eaeb487b"} Dec 03 12:51:18 crc kubenswrapper[5052]: I1203 12:51:18.301938 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-worker-b7f5dfb6f-pj6bx" event={"ID":"b2933134-7226-423a-8120-3148845b808a","Type":"ContainerStarted","Data":"d9a0bc41dd7a93bb685e52d2df88081b1cd6cec771ed9a5f4fa07db63fea42e5"} Dec 03 12:51:18 crc kubenswrapper[5052]: I1203 12:51:18.303861 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-5bdcd5dc6-vqlkq" event={"ID":"539d9404-7518-4f03-815d-568a43eedb0b","Type":"ContainerStarted","Data":"5e8481f4e60d3362b7d1a8fe46676fcecfc4e5f3f8c24cf300a5d3c419311f59"} Dec 03 12:51:18 crc kubenswrapper[5052]: I1203 12:51:18.303909 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-5bdcd5dc6-vqlkq" event={"ID":"539d9404-7518-4f03-815d-568a43eedb0b","Type":"ContainerStarted","Data":"f30c8553eba37c12be3afb2daedc6c8c9d49487bdc18a5edc36f729b1ad6083f"} Dec 03 12:51:18 crc kubenswrapper[5052]: I1203 12:51:18.303922 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-5bdcd5dc6-vqlkq" event={"ID":"539d9404-7518-4f03-815d-568a43eedb0b","Type":"ContainerStarted","Data":"dcaa6ec2052c2553cd436993c1572efa22c63b65ebb45e135ea74e0effccb612"} Dec 03 12:51:18 crc kubenswrapper[5052]: I1203 12:51:18.304639 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/barbican-api-5bdcd5dc6-vqlkq" Dec 03 12:51:18 crc kubenswrapper[5052]: I1203 12:51:18.304671 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/barbican-api-5bdcd5dc6-vqlkq" Dec 03 12:51:18 crc kubenswrapper[5052]: I1203 12:51:18.308630 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-keystone-listener-7b66cf7bb6-fsc4x" event={"ID":"020bfa4c-3dba-43d2-9ac2-0bbbae660709","Type":"ContainerStarted","Data":"bab03b01c66d5661abe6c60c8c81496efcefb2ea684c5751d1ee1113198a69ca"} Dec 03 12:51:18 crc kubenswrapper[5052]: I1203 12:51:18.308693 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-keystone-listener-7b66cf7bb6-fsc4x" event={"ID":"020bfa4c-3dba-43d2-9ac2-0bbbae660709","Type":"ContainerStarted","Data":"27212e7a0c6918bbf263320719b8b0731ba8a5d005b2aed71d83eab909ffb6f0"} Dec 03 12:51:18 crc kubenswrapper[5052]: I1203 12:51:18.337342 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/barbican-keystone-listener-7b66cf7bb6-fsc4x" podStartSLOduration=2.337323621 podStartE2EDuration="2.337323621s" podCreationTimestamp="2025-12-03 12:51:16 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:51:18.331395381 +0000 UTC m=+5468.919637145" watchObservedRunningTime="2025-12-03 12:51:18.337323621 +0000 UTC m=+5468.925565375" Dec 03 12:51:18 crc kubenswrapper[5052]: I1203 12:51:18.361140 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/barbican-api-5bdcd5dc6-vqlkq" podStartSLOduration=2.361120682 podStartE2EDuration="2.361120682s" podCreationTimestamp="2025-12-03 12:51:16 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:51:18.356452608 +0000 UTC m=+5468.944694372" watchObservedRunningTime="2025-12-03 12:51:18.361120682 +0000 UTC m=+5468.949362446" Dec 03 12:51:18 crc kubenswrapper[5052]: I1203 12:51:18.389041 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/keystone-bootstrap-d2z7m" podStartSLOduration=2.389025319 podStartE2EDuration="2.389025319s" podCreationTimestamp="2025-12-03 12:51:16 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:51:18.384029126 +0000 UTC m=+5468.972270900" watchObservedRunningTime="2025-12-03 12:51:18.389025319 +0000 UTC m=+5468.977267083" Dec 03 12:51:18 crc kubenswrapper[5052]: I1203 12:51:18.407311 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/barbican-worker-b7f5dfb6f-pj6bx" podStartSLOduration=2.407292591 podStartE2EDuration="2.407292591s" podCreationTimestamp="2025-12-03 12:51:16 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:51:18.405202681 +0000 UTC m=+5468.993444465" watchObservedRunningTime="2025-12-03 12:51:18.407292591 +0000 UTC m=+5468.995534355" Dec 03 12:51:19 crc kubenswrapper[5052]: I1203 12:51:19.328880 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"f3733be6-a76c-4c85-ad81-58ca3f3e3d2d","Type":"ContainerStarted","Data":"bca28d625281627cc7675e3115c216e4fdf5af87f408df91e6006b4d6c37f376"} Dec 03 12:51:19 crc kubenswrapper[5052]: I1203 12:51:19.360049 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/barbican-api-76dc68d888-zmmcs"] Dec 03 12:51:19 crc kubenswrapper[5052]: I1203 12:51:19.363220 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-api-76dc68d888-zmmcs" Dec 03 12:51:19 crc kubenswrapper[5052]: I1203 12:51:19.366355 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-barbican-internal-svc" Dec 03 12:51:19 crc kubenswrapper[5052]: I1203 12:51:19.370298 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-barbican-public-svc" Dec 03 12:51:19 crc kubenswrapper[5052]: I1203 12:51:19.378065 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-api-76dc68d888-zmmcs"] Dec 03 12:51:19 crc kubenswrapper[5052]: I1203 12:51:19.510185 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/25eead1e-d17c-4b36-8a9f-ef563189d53b-config-data-custom\") pod \"barbican-api-76dc68d888-zmmcs\" (UID: \"25eead1e-d17c-4b36-8a9f-ef563189d53b\") " pod="openstack-kuttl-tests/barbican-api-76dc68d888-zmmcs" Dec 03 12:51:19 crc kubenswrapper[5052]: I1203 12:51:19.510453 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/25eead1e-d17c-4b36-8a9f-ef563189d53b-public-tls-certs\") pod \"barbican-api-76dc68d888-zmmcs\" (UID: \"25eead1e-d17c-4b36-8a9f-ef563189d53b\") " pod="openstack-kuttl-tests/barbican-api-76dc68d888-zmmcs" Dec 03 12:51:19 crc kubenswrapper[5052]: I1203 12:51:19.510980 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/25eead1e-d17c-4b36-8a9f-ef563189d53b-combined-ca-bundle\") pod \"barbican-api-76dc68d888-zmmcs\" (UID: \"25eead1e-d17c-4b36-8a9f-ef563189d53b\") " pod="openstack-kuttl-tests/barbican-api-76dc68d888-zmmcs" Dec 03 12:51:19 crc kubenswrapper[5052]: I1203 12:51:19.511103 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/25eead1e-d17c-4b36-8a9f-ef563189d53b-config-data\") pod \"barbican-api-76dc68d888-zmmcs\" (UID: \"25eead1e-d17c-4b36-8a9f-ef563189d53b\") " pod="openstack-kuttl-tests/barbican-api-76dc68d888-zmmcs" Dec 03 12:51:19 crc kubenswrapper[5052]: I1203 12:51:19.511275 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/25eead1e-d17c-4b36-8a9f-ef563189d53b-internal-tls-certs\") pod \"barbican-api-76dc68d888-zmmcs\" (UID: \"25eead1e-d17c-4b36-8a9f-ef563189d53b\") " pod="openstack-kuttl-tests/barbican-api-76dc68d888-zmmcs" Dec 03 12:51:19 crc kubenswrapper[5052]: I1203 12:51:19.511398 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/25eead1e-d17c-4b36-8a9f-ef563189d53b-logs\") pod \"barbican-api-76dc68d888-zmmcs\" (UID: \"25eead1e-d17c-4b36-8a9f-ef563189d53b\") " pod="openstack-kuttl-tests/barbican-api-76dc68d888-zmmcs" Dec 03 12:51:19 crc kubenswrapper[5052]: I1203 12:51:19.511586 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ksl6m\" (UniqueName: \"kubernetes.io/projected/25eead1e-d17c-4b36-8a9f-ef563189d53b-kube-api-access-ksl6m\") pod \"barbican-api-76dc68d888-zmmcs\" (UID: \"25eead1e-d17c-4b36-8a9f-ef563189d53b\") " pod="openstack-kuttl-tests/barbican-api-76dc68d888-zmmcs" Dec 03 12:51:19 crc kubenswrapper[5052]: I1203 12:51:19.614426 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-ksl6m\" (UniqueName: \"kubernetes.io/projected/25eead1e-d17c-4b36-8a9f-ef563189d53b-kube-api-access-ksl6m\") pod \"barbican-api-76dc68d888-zmmcs\" (UID: \"25eead1e-d17c-4b36-8a9f-ef563189d53b\") " pod="openstack-kuttl-tests/barbican-api-76dc68d888-zmmcs" Dec 03 12:51:19 crc kubenswrapper[5052]: I1203 12:51:19.614538 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/25eead1e-d17c-4b36-8a9f-ef563189d53b-config-data-custom\") pod \"barbican-api-76dc68d888-zmmcs\" (UID: \"25eead1e-d17c-4b36-8a9f-ef563189d53b\") " pod="openstack-kuttl-tests/barbican-api-76dc68d888-zmmcs" Dec 03 12:51:19 crc kubenswrapper[5052]: I1203 12:51:19.614577 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/25eead1e-d17c-4b36-8a9f-ef563189d53b-public-tls-certs\") pod \"barbican-api-76dc68d888-zmmcs\" (UID: \"25eead1e-d17c-4b36-8a9f-ef563189d53b\") " pod="openstack-kuttl-tests/barbican-api-76dc68d888-zmmcs" Dec 03 12:51:19 crc kubenswrapper[5052]: I1203 12:51:19.614619 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/25eead1e-d17c-4b36-8a9f-ef563189d53b-combined-ca-bundle\") pod \"barbican-api-76dc68d888-zmmcs\" (UID: \"25eead1e-d17c-4b36-8a9f-ef563189d53b\") " pod="openstack-kuttl-tests/barbican-api-76dc68d888-zmmcs" Dec 03 12:51:19 crc kubenswrapper[5052]: I1203 12:51:19.614645 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/25eead1e-d17c-4b36-8a9f-ef563189d53b-config-data\") pod \"barbican-api-76dc68d888-zmmcs\" (UID: \"25eead1e-d17c-4b36-8a9f-ef563189d53b\") " pod="openstack-kuttl-tests/barbican-api-76dc68d888-zmmcs" Dec 03 12:51:19 crc kubenswrapper[5052]: I1203 12:51:19.614719 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/25eead1e-d17c-4b36-8a9f-ef563189d53b-internal-tls-certs\") pod \"barbican-api-76dc68d888-zmmcs\" (UID: \"25eead1e-d17c-4b36-8a9f-ef563189d53b\") " pod="openstack-kuttl-tests/barbican-api-76dc68d888-zmmcs" Dec 03 12:51:19 crc kubenswrapper[5052]: I1203 12:51:19.614766 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/25eead1e-d17c-4b36-8a9f-ef563189d53b-logs\") pod \"barbican-api-76dc68d888-zmmcs\" (UID: \"25eead1e-d17c-4b36-8a9f-ef563189d53b\") " pod="openstack-kuttl-tests/barbican-api-76dc68d888-zmmcs" Dec 03 12:51:19 crc kubenswrapper[5052]: I1203 12:51:19.615181 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/25eead1e-d17c-4b36-8a9f-ef563189d53b-logs\") pod \"barbican-api-76dc68d888-zmmcs\" (UID: \"25eead1e-d17c-4b36-8a9f-ef563189d53b\") " pod="openstack-kuttl-tests/barbican-api-76dc68d888-zmmcs" Dec 03 12:51:19 crc kubenswrapper[5052]: I1203 12:51:19.620109 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/25eead1e-d17c-4b36-8a9f-ef563189d53b-internal-tls-certs\") pod \"barbican-api-76dc68d888-zmmcs\" (UID: \"25eead1e-d17c-4b36-8a9f-ef563189d53b\") " pod="openstack-kuttl-tests/barbican-api-76dc68d888-zmmcs" Dec 03 12:51:19 crc kubenswrapper[5052]: I1203 12:51:19.620809 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/25eead1e-d17c-4b36-8a9f-ef563189d53b-public-tls-certs\") pod \"barbican-api-76dc68d888-zmmcs\" (UID: \"25eead1e-d17c-4b36-8a9f-ef563189d53b\") " pod="openstack-kuttl-tests/barbican-api-76dc68d888-zmmcs" Dec 03 12:51:19 crc kubenswrapper[5052]: I1203 12:51:19.621202 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/25eead1e-d17c-4b36-8a9f-ef563189d53b-combined-ca-bundle\") pod \"barbican-api-76dc68d888-zmmcs\" (UID: \"25eead1e-d17c-4b36-8a9f-ef563189d53b\") " pod="openstack-kuttl-tests/barbican-api-76dc68d888-zmmcs" Dec 03 12:51:19 crc kubenswrapper[5052]: I1203 12:51:19.626813 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/25eead1e-d17c-4b36-8a9f-ef563189d53b-config-data-custom\") pod \"barbican-api-76dc68d888-zmmcs\" (UID: \"25eead1e-d17c-4b36-8a9f-ef563189d53b\") " pod="openstack-kuttl-tests/barbican-api-76dc68d888-zmmcs" Dec 03 12:51:19 crc kubenswrapper[5052]: I1203 12:51:19.626842 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/25eead1e-d17c-4b36-8a9f-ef563189d53b-config-data\") pod \"barbican-api-76dc68d888-zmmcs\" (UID: \"25eead1e-d17c-4b36-8a9f-ef563189d53b\") " pod="openstack-kuttl-tests/barbican-api-76dc68d888-zmmcs" Dec 03 12:51:19 crc kubenswrapper[5052]: I1203 12:51:19.639716 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-ksl6m\" (UniqueName: \"kubernetes.io/projected/25eead1e-d17c-4b36-8a9f-ef563189d53b-kube-api-access-ksl6m\") pod \"barbican-api-76dc68d888-zmmcs\" (UID: \"25eead1e-d17c-4b36-8a9f-ef563189d53b\") " pod="openstack-kuttl-tests/barbican-api-76dc68d888-zmmcs" Dec 03 12:51:19 crc kubenswrapper[5052]: I1203 12:51:19.683647 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-api-76dc68d888-zmmcs" Dec 03 12:51:19 crc kubenswrapper[5052]: I1203 12:51:19.798556 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-db-sync-2b42x" Dec 03 12:51:19 crc kubenswrapper[5052]: I1203 12:51:19.919539 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/8212a97d-b76a-4b60-b16b-914a4bbe5b9d-db-sync-config-data\") pod \"8212a97d-b76a-4b60-b16b-914a4bbe5b9d\" (UID: \"8212a97d-b76a-4b60-b16b-914a4bbe5b9d\") " Dec 03 12:51:19 crc kubenswrapper[5052]: I1203 12:51:19.919588 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/8212a97d-b76a-4b60-b16b-914a4bbe5b9d-config-data\") pod \"8212a97d-b76a-4b60-b16b-914a4bbe5b9d\" (UID: \"8212a97d-b76a-4b60-b16b-914a4bbe5b9d\") " Dec 03 12:51:19 crc kubenswrapper[5052]: I1203 12:51:19.919653 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/8212a97d-b76a-4b60-b16b-914a4bbe5b9d-scripts\") pod \"8212a97d-b76a-4b60-b16b-914a4bbe5b9d\" (UID: \"8212a97d-b76a-4b60-b16b-914a4bbe5b9d\") " Dec 03 12:51:19 crc kubenswrapper[5052]: I1203 12:51:19.919697 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8212a97d-b76a-4b60-b16b-914a4bbe5b9d-combined-ca-bundle\") pod \"8212a97d-b76a-4b60-b16b-914a4bbe5b9d\" (UID: \"8212a97d-b76a-4b60-b16b-914a4bbe5b9d\") " Dec 03 12:51:19 crc kubenswrapper[5052]: I1203 12:51:19.919844 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9z5l6\" (UniqueName: \"kubernetes.io/projected/8212a97d-b76a-4b60-b16b-914a4bbe5b9d-kube-api-access-9z5l6\") pod \"8212a97d-b76a-4b60-b16b-914a4bbe5b9d\" (UID: \"8212a97d-b76a-4b60-b16b-914a4bbe5b9d\") " Dec 03 12:51:19 crc kubenswrapper[5052]: I1203 12:51:19.919894 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/8212a97d-b76a-4b60-b16b-914a4bbe5b9d-etc-machine-id\") pod \"8212a97d-b76a-4b60-b16b-914a4bbe5b9d\" (UID: \"8212a97d-b76a-4b60-b16b-914a4bbe5b9d\") " Dec 03 12:51:19 crc kubenswrapper[5052]: I1203 12:51:19.920360 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/8212a97d-b76a-4b60-b16b-914a4bbe5b9d-etc-machine-id" (OuterVolumeSpecName: "etc-machine-id") pod "8212a97d-b76a-4b60-b16b-914a4bbe5b9d" (UID: "8212a97d-b76a-4b60-b16b-914a4bbe5b9d"). InnerVolumeSpecName "etc-machine-id". PluginName "kubernetes.io/host-path", VolumeGidValue "" Dec 03 12:51:19 crc kubenswrapper[5052]: I1203 12:51:19.923473 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8212a97d-b76a-4b60-b16b-914a4bbe5b9d-kube-api-access-9z5l6" (OuterVolumeSpecName: "kube-api-access-9z5l6") pod "8212a97d-b76a-4b60-b16b-914a4bbe5b9d" (UID: "8212a97d-b76a-4b60-b16b-914a4bbe5b9d"). InnerVolumeSpecName "kube-api-access-9z5l6". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:51:19 crc kubenswrapper[5052]: I1203 12:51:19.923791 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8212a97d-b76a-4b60-b16b-914a4bbe5b9d-db-sync-config-data" (OuterVolumeSpecName: "db-sync-config-data") pod "8212a97d-b76a-4b60-b16b-914a4bbe5b9d" (UID: "8212a97d-b76a-4b60-b16b-914a4bbe5b9d"). InnerVolumeSpecName "db-sync-config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:51:19 crc kubenswrapper[5052]: I1203 12:51:19.933859 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8212a97d-b76a-4b60-b16b-914a4bbe5b9d-scripts" (OuterVolumeSpecName: "scripts") pod "8212a97d-b76a-4b60-b16b-914a4bbe5b9d" (UID: "8212a97d-b76a-4b60-b16b-914a4bbe5b9d"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:51:19 crc kubenswrapper[5052]: I1203 12:51:19.950073 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8212a97d-b76a-4b60-b16b-914a4bbe5b9d-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "8212a97d-b76a-4b60-b16b-914a4bbe5b9d" (UID: "8212a97d-b76a-4b60-b16b-914a4bbe5b9d"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:51:19 crc kubenswrapper[5052]: I1203 12:51:19.977643 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8212a97d-b76a-4b60-b16b-914a4bbe5b9d-config-data" (OuterVolumeSpecName: "config-data") pod "8212a97d-b76a-4b60-b16b-914a4bbe5b9d" (UID: "8212a97d-b76a-4b60-b16b-914a4bbe5b9d"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:51:20 crc kubenswrapper[5052]: I1203 12:51:20.021595 5052 reconciler_common.go:293] "Volume detached for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/8212a97d-b76a-4b60-b16b-914a4bbe5b9d-db-sync-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:51:20 crc kubenswrapper[5052]: I1203 12:51:20.021631 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/8212a97d-b76a-4b60-b16b-914a4bbe5b9d-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:51:20 crc kubenswrapper[5052]: I1203 12:51:20.021639 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/8212a97d-b76a-4b60-b16b-914a4bbe5b9d-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:51:20 crc kubenswrapper[5052]: I1203 12:51:20.021650 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8212a97d-b76a-4b60-b16b-914a4bbe5b9d-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:51:20 crc kubenswrapper[5052]: I1203 12:51:20.021661 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-9z5l6\" (UniqueName: \"kubernetes.io/projected/8212a97d-b76a-4b60-b16b-914a4bbe5b9d-kube-api-access-9z5l6\") on node \"crc\" DevicePath \"\"" Dec 03 12:51:20 crc kubenswrapper[5052]: I1203 12:51:20.021670 5052 reconciler_common.go:293] "Volume detached for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/8212a97d-b76a-4b60-b16b-914a4bbe5b9d-etc-machine-id\") on node \"crc\" DevicePath \"\"" Dec 03 12:51:20 crc kubenswrapper[5052]: I1203 12:51:20.138842 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican-api-76dc68d888-zmmcs"] Dec 03 12:51:20 crc kubenswrapper[5052]: W1203 12:51:20.140578 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod25eead1e_d17c_4b36_8a9f_ef563189d53b.slice/crio-58a2643eb7cb7af49c47147ea6a9b799b0eb13ca461c58ef022848439681488b WatchSource:0}: Error finding container 58a2643eb7cb7af49c47147ea6a9b799b0eb13ca461c58ef022848439681488b: Status 404 returned error can't find the container with id 58a2643eb7cb7af49c47147ea6a9b799b0eb13ca461c58ef022848439681488b Dec 03 12:51:20 crc kubenswrapper[5052]: I1203 12:51:20.338745 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-76dc68d888-zmmcs" event={"ID":"25eead1e-d17c-4b36-8a9f-ef563189d53b","Type":"ContainerStarted","Data":"4c38a9546e116479167e49dc563abec492318ccfbe7e54b5db501a8e8665f5aa"} Dec 03 12:51:20 crc kubenswrapper[5052]: I1203 12:51:20.338803 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-76dc68d888-zmmcs" event={"ID":"25eead1e-d17c-4b36-8a9f-ef563189d53b","Type":"ContainerStarted","Data":"58a2643eb7cb7af49c47147ea6a9b799b0eb13ca461c58ef022848439681488b"} Dec 03 12:51:20 crc kubenswrapper[5052]: I1203 12:51:20.342955 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-db-sync-2b42x" event={"ID":"8212a97d-b76a-4b60-b16b-914a4bbe5b9d","Type":"ContainerDied","Data":"20c10208455f5a41166905ab0c82bb2f4b84aaf5e03bf97368f12b7272263c80"} Dec 03 12:51:20 crc kubenswrapper[5052]: I1203 12:51:20.343011 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="20c10208455f5a41166905ab0c82bb2f4b84aaf5e03bf97368f12b7272263c80" Dec 03 12:51:20 crc kubenswrapper[5052]: I1203 12:51:20.343206 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-db-sync-2b42x" Dec 03 12:51:20 crc kubenswrapper[5052]: I1203 12:51:20.533302 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 12:51:20 crc kubenswrapper[5052]: E1203 12:51:20.533903 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="8212a97d-b76a-4b60-b16b-914a4bbe5b9d" containerName="cinder-db-sync" Dec 03 12:51:20 crc kubenswrapper[5052]: I1203 12:51:20.533919 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="8212a97d-b76a-4b60-b16b-914a4bbe5b9d" containerName="cinder-db-sync" Dec 03 12:51:20 crc kubenswrapper[5052]: I1203 12:51:20.534116 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="8212a97d-b76a-4b60-b16b-914a4bbe5b9d" containerName="cinder-db-sync" Dec 03 12:51:20 crc kubenswrapper[5052]: I1203 12:51:20.535402 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:51:20 crc kubenswrapper[5052]: I1203 12:51:20.540499 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cinder-cinder-dockercfg-8p42q" Dec 03 12:51:20 crc kubenswrapper[5052]: I1203 12:51:20.540963 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cinder-config-data" Dec 03 12:51:20 crc kubenswrapper[5052]: I1203 12:51:20.541153 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cinder-scripts" Dec 03 12:51:20 crc kubenswrapper[5052]: I1203 12:51:20.541298 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cinder-scheduler-config-data" Dec 03 12:51:20 crc kubenswrapper[5052]: I1203 12:51:20.558702 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 12:51:20 crc kubenswrapper[5052]: I1203 12:51:20.610505 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 12:51:20 crc kubenswrapper[5052]: I1203 12:51:20.612361 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:51:20 crc kubenswrapper[5052]: I1203 12:51:20.616575 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cinder-api-config-data" Dec 03 12:51:20 crc kubenswrapper[5052]: I1203 12:51:20.640094 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qb97f\" (UniqueName: \"kubernetes.io/projected/ee74f2f2-174b-4a8c-9fac-812be78ad1a8-kube-api-access-qb97f\") pod \"cinder-scheduler-0\" (UID: \"ee74f2f2-174b-4a8c-9fac-812be78ad1a8\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:51:20 crc kubenswrapper[5052]: I1203 12:51:20.640163 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/ee74f2f2-174b-4a8c-9fac-812be78ad1a8-config-data-custom\") pod \"cinder-scheduler-0\" (UID: \"ee74f2f2-174b-4a8c-9fac-812be78ad1a8\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:51:20 crc kubenswrapper[5052]: I1203 12:51:20.640244 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/ee74f2f2-174b-4a8c-9fac-812be78ad1a8-scripts\") pod \"cinder-scheduler-0\" (UID: \"ee74f2f2-174b-4a8c-9fac-812be78ad1a8\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:51:20 crc kubenswrapper[5052]: I1203 12:51:20.640300 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/ee74f2f2-174b-4a8c-9fac-812be78ad1a8-etc-machine-id\") pod \"cinder-scheduler-0\" (UID: \"ee74f2f2-174b-4a8c-9fac-812be78ad1a8\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:51:20 crc kubenswrapper[5052]: I1203 12:51:20.640336 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ee74f2f2-174b-4a8c-9fac-812be78ad1a8-combined-ca-bundle\") pod \"cinder-scheduler-0\" (UID: \"ee74f2f2-174b-4a8c-9fac-812be78ad1a8\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:51:20 crc kubenswrapper[5052]: I1203 12:51:20.640363 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/ee74f2f2-174b-4a8c-9fac-812be78ad1a8-config-data\") pod \"cinder-scheduler-0\" (UID: \"ee74f2f2-174b-4a8c-9fac-812be78ad1a8\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:51:20 crc kubenswrapper[5052]: I1203 12:51:20.640470 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 12:51:20 crc kubenswrapper[5052]: I1203 12:51:20.741964 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/3758f9f1-0328-4860-a346-e3691983a552-etc-machine-id\") pod \"cinder-api-0\" (UID: \"3758f9f1-0328-4860-a346-e3691983a552\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:51:20 crc kubenswrapper[5052]: I1203 12:51:20.742017 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/ee74f2f2-174b-4a8c-9fac-812be78ad1a8-etc-machine-id\") pod \"cinder-scheduler-0\" (UID: \"ee74f2f2-174b-4a8c-9fac-812be78ad1a8\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:51:20 crc kubenswrapper[5052]: I1203 12:51:20.742055 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mxdqb\" (UniqueName: \"kubernetes.io/projected/3758f9f1-0328-4860-a346-e3691983a552-kube-api-access-mxdqb\") pod \"cinder-api-0\" (UID: \"3758f9f1-0328-4860-a346-e3691983a552\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:51:20 crc kubenswrapper[5052]: I1203 12:51:20.742076 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ee74f2f2-174b-4a8c-9fac-812be78ad1a8-combined-ca-bundle\") pod \"cinder-scheduler-0\" (UID: \"ee74f2f2-174b-4a8c-9fac-812be78ad1a8\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:51:20 crc kubenswrapper[5052]: I1203 12:51:20.742090 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/3758f9f1-0328-4860-a346-e3691983a552-scripts\") pod \"cinder-api-0\" (UID: \"3758f9f1-0328-4860-a346-e3691983a552\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:51:20 crc kubenswrapper[5052]: I1203 12:51:20.742113 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/ee74f2f2-174b-4a8c-9fac-812be78ad1a8-config-data\") pod \"cinder-scheduler-0\" (UID: \"ee74f2f2-174b-4a8c-9fac-812be78ad1a8\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:51:20 crc kubenswrapper[5052]: I1203 12:51:20.742158 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/3758f9f1-0328-4860-a346-e3691983a552-logs\") pod \"cinder-api-0\" (UID: \"3758f9f1-0328-4860-a346-e3691983a552\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:51:20 crc kubenswrapper[5052]: I1203 12:51:20.742175 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-qb97f\" (UniqueName: \"kubernetes.io/projected/ee74f2f2-174b-4a8c-9fac-812be78ad1a8-kube-api-access-qb97f\") pod \"cinder-scheduler-0\" (UID: \"ee74f2f2-174b-4a8c-9fac-812be78ad1a8\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:51:20 crc kubenswrapper[5052]: I1203 12:51:20.742299 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/ee74f2f2-174b-4a8c-9fac-812be78ad1a8-config-data-custom\") pod \"cinder-scheduler-0\" (UID: \"ee74f2f2-174b-4a8c-9fac-812be78ad1a8\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:51:20 crc kubenswrapper[5052]: I1203 12:51:20.742333 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/ee74f2f2-174b-4a8c-9fac-812be78ad1a8-etc-machine-id\") pod \"cinder-scheduler-0\" (UID: \"ee74f2f2-174b-4a8c-9fac-812be78ad1a8\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:51:20 crc kubenswrapper[5052]: I1203 12:51:20.742391 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/3758f9f1-0328-4860-a346-e3691983a552-config-data\") pod \"cinder-api-0\" (UID: \"3758f9f1-0328-4860-a346-e3691983a552\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:51:20 crc kubenswrapper[5052]: I1203 12:51:20.742472 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/3758f9f1-0328-4860-a346-e3691983a552-config-data-custom\") pod \"cinder-api-0\" (UID: \"3758f9f1-0328-4860-a346-e3691983a552\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:51:20 crc kubenswrapper[5052]: I1203 12:51:20.742501 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3758f9f1-0328-4860-a346-e3691983a552-combined-ca-bundle\") pod \"cinder-api-0\" (UID: \"3758f9f1-0328-4860-a346-e3691983a552\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:51:20 crc kubenswrapper[5052]: I1203 12:51:20.742640 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/ee74f2f2-174b-4a8c-9fac-812be78ad1a8-scripts\") pod \"cinder-scheduler-0\" (UID: \"ee74f2f2-174b-4a8c-9fac-812be78ad1a8\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:51:20 crc kubenswrapper[5052]: I1203 12:51:20.750117 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/ee74f2f2-174b-4a8c-9fac-812be78ad1a8-scripts\") pod \"cinder-scheduler-0\" (UID: \"ee74f2f2-174b-4a8c-9fac-812be78ad1a8\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:51:20 crc kubenswrapper[5052]: I1203 12:51:20.752049 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/ee74f2f2-174b-4a8c-9fac-812be78ad1a8-config-data-custom\") pod \"cinder-scheduler-0\" (UID: \"ee74f2f2-174b-4a8c-9fac-812be78ad1a8\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:51:20 crc kubenswrapper[5052]: I1203 12:51:20.752810 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/ee74f2f2-174b-4a8c-9fac-812be78ad1a8-config-data\") pod \"cinder-scheduler-0\" (UID: \"ee74f2f2-174b-4a8c-9fac-812be78ad1a8\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:51:20 crc kubenswrapper[5052]: I1203 12:51:20.754169 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ee74f2f2-174b-4a8c-9fac-812be78ad1a8-combined-ca-bundle\") pod \"cinder-scheduler-0\" (UID: \"ee74f2f2-174b-4a8c-9fac-812be78ad1a8\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:51:20 crc kubenswrapper[5052]: I1203 12:51:20.765023 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-qb97f\" (UniqueName: \"kubernetes.io/projected/ee74f2f2-174b-4a8c-9fac-812be78ad1a8-kube-api-access-qb97f\") pod \"cinder-scheduler-0\" (UID: \"ee74f2f2-174b-4a8c-9fac-812be78ad1a8\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:51:20 crc kubenswrapper[5052]: I1203 12:51:20.844016 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/3758f9f1-0328-4860-a346-e3691983a552-logs\") pod \"cinder-api-0\" (UID: \"3758f9f1-0328-4860-a346-e3691983a552\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:51:20 crc kubenswrapper[5052]: I1203 12:51:20.844110 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/3758f9f1-0328-4860-a346-e3691983a552-config-data\") pod \"cinder-api-0\" (UID: \"3758f9f1-0328-4860-a346-e3691983a552\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:51:20 crc kubenswrapper[5052]: I1203 12:51:20.844134 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/3758f9f1-0328-4860-a346-e3691983a552-config-data-custom\") pod \"cinder-api-0\" (UID: \"3758f9f1-0328-4860-a346-e3691983a552\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:51:20 crc kubenswrapper[5052]: I1203 12:51:20.844150 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3758f9f1-0328-4860-a346-e3691983a552-combined-ca-bundle\") pod \"cinder-api-0\" (UID: \"3758f9f1-0328-4860-a346-e3691983a552\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:51:20 crc kubenswrapper[5052]: I1203 12:51:20.844400 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/3758f9f1-0328-4860-a346-e3691983a552-etc-machine-id\") pod \"cinder-api-0\" (UID: \"3758f9f1-0328-4860-a346-e3691983a552\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:51:20 crc kubenswrapper[5052]: I1203 12:51:20.844448 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-mxdqb\" (UniqueName: \"kubernetes.io/projected/3758f9f1-0328-4860-a346-e3691983a552-kube-api-access-mxdqb\") pod \"cinder-api-0\" (UID: \"3758f9f1-0328-4860-a346-e3691983a552\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:51:20 crc kubenswrapper[5052]: I1203 12:51:20.844469 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/3758f9f1-0328-4860-a346-e3691983a552-scripts\") pod \"cinder-api-0\" (UID: \"3758f9f1-0328-4860-a346-e3691983a552\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:51:20 crc kubenswrapper[5052]: I1203 12:51:20.844808 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/3758f9f1-0328-4860-a346-e3691983a552-etc-machine-id\") pod \"cinder-api-0\" (UID: \"3758f9f1-0328-4860-a346-e3691983a552\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:51:20 crc kubenswrapper[5052]: I1203 12:51:20.849272 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/3758f9f1-0328-4860-a346-e3691983a552-logs\") pod \"cinder-api-0\" (UID: \"3758f9f1-0328-4860-a346-e3691983a552\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:51:20 crc kubenswrapper[5052]: I1203 12:51:20.858076 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/3758f9f1-0328-4860-a346-e3691983a552-scripts\") pod \"cinder-api-0\" (UID: \"3758f9f1-0328-4860-a346-e3691983a552\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:51:20 crc kubenswrapper[5052]: I1203 12:51:20.865086 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/3758f9f1-0328-4860-a346-e3691983a552-config-data\") pod \"cinder-api-0\" (UID: \"3758f9f1-0328-4860-a346-e3691983a552\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:51:20 crc kubenswrapper[5052]: I1203 12:51:20.866792 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:51:20 crc kubenswrapper[5052]: I1203 12:51:20.869092 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/3758f9f1-0328-4860-a346-e3691983a552-config-data-custom\") pod \"cinder-api-0\" (UID: \"3758f9f1-0328-4860-a346-e3691983a552\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:51:20 crc kubenswrapper[5052]: I1203 12:51:20.871083 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3758f9f1-0328-4860-a346-e3691983a552-combined-ca-bundle\") pod \"cinder-api-0\" (UID: \"3758f9f1-0328-4860-a346-e3691983a552\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:51:20 crc kubenswrapper[5052]: I1203 12:51:20.881707 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-mxdqb\" (UniqueName: \"kubernetes.io/projected/3758f9f1-0328-4860-a346-e3691983a552-kube-api-access-mxdqb\") pod \"cinder-api-0\" (UID: \"3758f9f1-0328-4860-a346-e3691983a552\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:51:21 crc kubenswrapper[5052]: I1203 12:51:21.134610 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:51:21 crc kubenswrapper[5052]: I1203 12:51:21.358957 5052 generic.go:334] "Generic (PLEG): container finished" podID="cd29fd9a-ac8a-4424-b6ac-e4b81bf292f4" containerID="b78075e349099e5081c2a933e5a387e49bcc03246f78f07abd5d255f951c6969" exitCode=0 Dec 03 12:51:21 crc kubenswrapper[5052]: I1203 12:51:21.359213 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-bootstrap-d2z7m" event={"ID":"cd29fd9a-ac8a-4424-b6ac-e4b81bf292f4","Type":"ContainerDied","Data":"b78075e349099e5081c2a933e5a387e49bcc03246f78f07abd5d255f951c6969"} Dec 03 12:51:21 crc kubenswrapper[5052]: I1203 12:51:21.365375 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-76dc68d888-zmmcs" event={"ID":"25eead1e-d17c-4b36-8a9f-ef563189d53b","Type":"ContainerStarted","Data":"018816f9a78e7b2315d8c4e733edf99eeec2819f8f83cbe3fa3669341863f8d1"} Dec 03 12:51:21 crc kubenswrapper[5052]: I1203 12:51:21.365477 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/barbican-api-76dc68d888-zmmcs" Dec 03 12:51:21 crc kubenswrapper[5052]: I1203 12:51:21.365506 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/barbican-api-76dc68d888-zmmcs" Dec 03 12:51:21 crc kubenswrapper[5052]: I1203 12:51:21.371629 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"f3733be6-a76c-4c85-ad81-58ca3f3e3d2d","Type":"ContainerStarted","Data":"9753117b9d1d7a9e8974b3b7100aceebac6f8ff8d17dbee469702e6582413ceb"} Dec 03 12:51:21 crc kubenswrapper[5052]: I1203 12:51:21.372716 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:51:21 crc kubenswrapper[5052]: I1203 12:51:21.373666 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 12:51:21 crc kubenswrapper[5052]: I1203 12:51:21.405714 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/barbican-api-76dc68d888-zmmcs" podStartSLOduration=2.405690271 podStartE2EDuration="2.405690271s" podCreationTimestamp="2025-12-03 12:51:19 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:51:21.398488255 +0000 UTC m=+5471.986730049" watchObservedRunningTime="2025-12-03 12:51:21.405690271 +0000 UTC m=+5471.993932045" Dec 03 12:51:21 crc kubenswrapper[5052]: I1203 12:51:21.434025 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/ceilometer-0" podStartSLOduration=1.87042442 podStartE2EDuration="6.43399992s" podCreationTimestamp="2025-12-03 12:51:15 +0000 UTC" firstStartedPulling="2025-12-03 12:51:16.121107621 +0000 UTC m=+5466.709349385" lastFinishedPulling="2025-12-03 12:51:20.684683111 +0000 UTC m=+5471.272924885" observedRunningTime="2025-12-03 12:51:21.423575822 +0000 UTC m=+5472.011817606" watchObservedRunningTime="2025-12-03 12:51:21.43399992 +0000 UTC m=+5472.022241694" Dec 03 12:51:21 crc kubenswrapper[5052]: I1203 12:51:21.593903 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 12:51:21 crc kubenswrapper[5052]: W1203 12:51:21.599478 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3758f9f1_0328_4860_a346_e3691983a552.slice/crio-2ca85e3c4d60693e2e1ffb8494d1730c75ce2e53ca64040724c027491fcfc3bc WatchSource:0}: Error finding container 2ca85e3c4d60693e2e1ffb8494d1730c75ce2e53ca64040724c027491fcfc3bc: Status 404 returned error can't find the container with id 2ca85e3c4d60693e2e1ffb8494d1730c75ce2e53ca64040724c027491fcfc3bc Dec 03 12:51:22 crc kubenswrapper[5052]: I1203 12:51:22.403806 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"ee74f2f2-174b-4a8c-9fac-812be78ad1a8","Type":"ContainerStarted","Data":"ca86cc05c83430edadfad14e0307d0069ae23ed1a778ca9dc8d0eaad71818ac1"} Dec 03 12:51:22 crc kubenswrapper[5052]: I1203 12:51:22.404112 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"ee74f2f2-174b-4a8c-9fac-812be78ad1a8","Type":"ContainerStarted","Data":"0342943988af94769a8ba341d5ff2e919423c64f7796d1453d4fc55c187d2831"} Dec 03 12:51:22 crc kubenswrapper[5052]: I1203 12:51:22.407481 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"3758f9f1-0328-4860-a346-e3691983a552","Type":"ContainerStarted","Data":"2ca85e3c4d60693e2e1ffb8494d1730c75ce2e53ca64040724c027491fcfc3bc"} Dec 03 12:51:22 crc kubenswrapper[5052]: I1203 12:51:22.834129 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-bootstrap-d2z7m" Dec 03 12:51:23 crc kubenswrapper[5052]: I1203 12:51:23.002179 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/cd29fd9a-ac8a-4424-b6ac-e4b81bf292f4-fernet-keys\") pod \"cd29fd9a-ac8a-4424-b6ac-e4b81bf292f4\" (UID: \"cd29fd9a-ac8a-4424-b6ac-e4b81bf292f4\") " Dec 03 12:51:23 crc kubenswrapper[5052]: I1203 12:51:23.002492 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/cd29fd9a-ac8a-4424-b6ac-e4b81bf292f4-scripts\") pod \"cd29fd9a-ac8a-4424-b6ac-e4b81bf292f4\" (UID: \"cd29fd9a-ac8a-4424-b6ac-e4b81bf292f4\") " Dec 03 12:51:23 crc kubenswrapper[5052]: I1203 12:51:23.002712 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dpdq2\" (UniqueName: \"kubernetes.io/projected/cd29fd9a-ac8a-4424-b6ac-e4b81bf292f4-kube-api-access-dpdq2\") pod \"cd29fd9a-ac8a-4424-b6ac-e4b81bf292f4\" (UID: \"cd29fd9a-ac8a-4424-b6ac-e4b81bf292f4\") " Dec 03 12:51:23 crc kubenswrapper[5052]: I1203 12:51:23.002783 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/cd29fd9a-ac8a-4424-b6ac-e4b81bf292f4-credential-keys\") pod \"cd29fd9a-ac8a-4424-b6ac-e4b81bf292f4\" (UID: \"cd29fd9a-ac8a-4424-b6ac-e4b81bf292f4\") " Dec 03 12:51:23 crc kubenswrapper[5052]: I1203 12:51:23.002826 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/cd29fd9a-ac8a-4424-b6ac-e4b81bf292f4-combined-ca-bundle\") pod \"cd29fd9a-ac8a-4424-b6ac-e4b81bf292f4\" (UID: \"cd29fd9a-ac8a-4424-b6ac-e4b81bf292f4\") " Dec 03 12:51:23 crc kubenswrapper[5052]: I1203 12:51:23.002856 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/cd29fd9a-ac8a-4424-b6ac-e4b81bf292f4-config-data\") pod \"cd29fd9a-ac8a-4424-b6ac-e4b81bf292f4\" (UID: \"cd29fd9a-ac8a-4424-b6ac-e4b81bf292f4\") " Dec 03 12:51:23 crc kubenswrapper[5052]: I1203 12:51:23.024077 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cd29fd9a-ac8a-4424-b6ac-e4b81bf292f4-scripts" (OuterVolumeSpecName: "scripts") pod "cd29fd9a-ac8a-4424-b6ac-e4b81bf292f4" (UID: "cd29fd9a-ac8a-4424-b6ac-e4b81bf292f4"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:51:23 crc kubenswrapper[5052]: I1203 12:51:23.024113 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cd29fd9a-ac8a-4424-b6ac-e4b81bf292f4-credential-keys" (OuterVolumeSpecName: "credential-keys") pod "cd29fd9a-ac8a-4424-b6ac-e4b81bf292f4" (UID: "cd29fd9a-ac8a-4424-b6ac-e4b81bf292f4"). InnerVolumeSpecName "credential-keys". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:51:23 crc kubenswrapper[5052]: I1203 12:51:23.024191 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cd29fd9a-ac8a-4424-b6ac-e4b81bf292f4-kube-api-access-dpdq2" (OuterVolumeSpecName: "kube-api-access-dpdq2") pod "cd29fd9a-ac8a-4424-b6ac-e4b81bf292f4" (UID: "cd29fd9a-ac8a-4424-b6ac-e4b81bf292f4"). InnerVolumeSpecName "kube-api-access-dpdq2". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:51:23 crc kubenswrapper[5052]: I1203 12:51:23.028288 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cd29fd9a-ac8a-4424-b6ac-e4b81bf292f4-fernet-keys" (OuterVolumeSpecName: "fernet-keys") pod "cd29fd9a-ac8a-4424-b6ac-e4b81bf292f4" (UID: "cd29fd9a-ac8a-4424-b6ac-e4b81bf292f4"). InnerVolumeSpecName "fernet-keys". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:51:23 crc kubenswrapper[5052]: I1203 12:51:23.076674 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cd29fd9a-ac8a-4424-b6ac-e4b81bf292f4-config-data" (OuterVolumeSpecName: "config-data") pod "cd29fd9a-ac8a-4424-b6ac-e4b81bf292f4" (UID: "cd29fd9a-ac8a-4424-b6ac-e4b81bf292f4"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:51:23 crc kubenswrapper[5052]: I1203 12:51:23.077332 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cd29fd9a-ac8a-4424-b6ac-e4b81bf292f4-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "cd29fd9a-ac8a-4424-b6ac-e4b81bf292f4" (UID: "cd29fd9a-ac8a-4424-b6ac-e4b81bf292f4"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:51:23 crc kubenswrapper[5052]: I1203 12:51:23.104795 5052 reconciler_common.go:293] "Volume detached for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/cd29fd9a-ac8a-4424-b6ac-e4b81bf292f4-credential-keys\") on node \"crc\" DevicePath \"\"" Dec 03 12:51:23 crc kubenswrapper[5052]: I1203 12:51:23.104836 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/cd29fd9a-ac8a-4424-b6ac-e4b81bf292f4-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:51:23 crc kubenswrapper[5052]: I1203 12:51:23.104851 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/cd29fd9a-ac8a-4424-b6ac-e4b81bf292f4-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:51:23 crc kubenswrapper[5052]: I1203 12:51:23.104866 5052 reconciler_common.go:293] "Volume detached for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/cd29fd9a-ac8a-4424-b6ac-e4b81bf292f4-fernet-keys\") on node \"crc\" DevicePath \"\"" Dec 03 12:51:23 crc kubenswrapper[5052]: I1203 12:51:23.104879 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/cd29fd9a-ac8a-4424-b6ac-e4b81bf292f4-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:51:23 crc kubenswrapper[5052]: I1203 12:51:23.104895 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-dpdq2\" (UniqueName: \"kubernetes.io/projected/cd29fd9a-ac8a-4424-b6ac-e4b81bf292f4-kube-api-access-dpdq2\") on node \"crc\" DevicePath \"\"" Dec 03 12:51:23 crc kubenswrapper[5052]: I1203 12:51:23.432087 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"ee74f2f2-174b-4a8c-9fac-812be78ad1a8","Type":"ContainerStarted","Data":"ffe8aa10ffea0f8a58b990b6040c05d187bb5616dea3df67a9e84c15341fba1a"} Dec 03 12:51:23 crc kubenswrapper[5052]: I1203 12:51:23.437832 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"3758f9f1-0328-4860-a346-e3691983a552","Type":"ContainerStarted","Data":"de321d5d5126d4d9f5462912231b30a9ea3db3810ab591537461b693b09d3c5a"} Dec 03 12:51:23 crc kubenswrapper[5052]: I1203 12:51:23.454840 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:51:23 crc kubenswrapper[5052]: I1203 12:51:23.456533 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:51:23 crc kubenswrapper[5052]: I1203 12:51:23.462413 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-bootstrap-d2z7m" Dec 03 12:51:23 crc kubenswrapper[5052]: I1203 12:51:23.463633 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-bootstrap-d2z7m" event={"ID":"cd29fd9a-ac8a-4424-b6ac-e4b81bf292f4","Type":"ContainerDied","Data":"10936a6922451e4775ed678a70def6ea5bbab6cdd2a4c6978e1c1c7c1b4b2731"} Dec 03 12:51:23 crc kubenswrapper[5052]: I1203 12:51:23.463682 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="10936a6922451e4775ed678a70def6ea5bbab6cdd2a4c6978e1c1c7c1b4b2731" Dec 03 12:51:23 crc kubenswrapper[5052]: I1203 12:51:23.496541 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/cinder-scheduler-0" podStartSLOduration=3.496489023 podStartE2EDuration="3.496489023s" podCreationTimestamp="2025-12-03 12:51:20 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:51:23.463936942 +0000 UTC m=+5474.052178706" watchObservedRunningTime="2025-12-03 12:51:23.496489023 +0000 UTC m=+5474.084730787" Dec 03 12:51:23 crc kubenswrapper[5052]: I1203 12:51:23.511045 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 12:51:23 crc kubenswrapper[5052]: I1203 12:51:23.544766 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/keystone-7fd545bd9-2lc7s"] Dec 03 12:51:23 crc kubenswrapper[5052]: E1203 12:51:23.545168 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="cd29fd9a-ac8a-4424-b6ac-e4b81bf292f4" containerName="keystone-bootstrap" Dec 03 12:51:23 crc kubenswrapper[5052]: I1203 12:51:23.545183 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="cd29fd9a-ac8a-4424-b6ac-e4b81bf292f4" containerName="keystone-bootstrap" Dec 03 12:51:23 crc kubenswrapper[5052]: I1203 12:51:23.545380 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="cd29fd9a-ac8a-4424-b6ac-e4b81bf292f4" containerName="keystone-bootstrap" Dec 03 12:51:23 crc kubenswrapper[5052]: I1203 12:51:23.546036 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-7fd545bd9-2lc7s" Dec 03 12:51:23 crc kubenswrapper[5052]: I1203 12:51:23.546418 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:51:23 crc kubenswrapper[5052]: I1203 12:51:23.552990 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone-keystone-dockercfg-fr5sp" Dec 03 12:51:23 crc kubenswrapper[5052]: I1203 12:51:23.553168 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone-scripts" Dec 03 12:51:23 crc kubenswrapper[5052]: I1203 12:51:23.554303 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-keystone-public-svc" Dec 03 12:51:23 crc kubenswrapper[5052]: I1203 12:51:23.554472 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-keystone-internal-svc" Dec 03 12:51:23 crc kubenswrapper[5052]: I1203 12:51:23.554618 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone-config-data" Dec 03 12:51:23 crc kubenswrapper[5052]: I1203 12:51:23.554726 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"keystone" Dec 03 12:51:23 crc kubenswrapper[5052]: I1203 12:51:23.579877 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:51:23 crc kubenswrapper[5052]: I1203 12:51:23.589838 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-7fd545bd9-2lc7s"] Dec 03 12:51:23 crc kubenswrapper[5052]: I1203 12:51:23.726961 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/0191d138-7aff-42c2-88d2-fbbaf1722bfd-credential-keys\") pod \"keystone-7fd545bd9-2lc7s\" (UID: \"0191d138-7aff-42c2-88d2-fbbaf1722bfd\") " pod="openstack-kuttl-tests/keystone-7fd545bd9-2lc7s" Dec 03 12:51:23 crc kubenswrapper[5052]: I1203 12:51:23.727087 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xqxxd\" (UniqueName: \"kubernetes.io/projected/0191d138-7aff-42c2-88d2-fbbaf1722bfd-kube-api-access-xqxxd\") pod \"keystone-7fd545bd9-2lc7s\" (UID: \"0191d138-7aff-42c2-88d2-fbbaf1722bfd\") " pod="openstack-kuttl-tests/keystone-7fd545bd9-2lc7s" Dec 03 12:51:23 crc kubenswrapper[5052]: I1203 12:51:23.727143 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/0191d138-7aff-42c2-88d2-fbbaf1722bfd-internal-tls-certs\") pod \"keystone-7fd545bd9-2lc7s\" (UID: \"0191d138-7aff-42c2-88d2-fbbaf1722bfd\") " pod="openstack-kuttl-tests/keystone-7fd545bd9-2lc7s" Dec 03 12:51:23 crc kubenswrapper[5052]: I1203 12:51:23.727272 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/0191d138-7aff-42c2-88d2-fbbaf1722bfd-fernet-keys\") pod \"keystone-7fd545bd9-2lc7s\" (UID: \"0191d138-7aff-42c2-88d2-fbbaf1722bfd\") " pod="openstack-kuttl-tests/keystone-7fd545bd9-2lc7s" Dec 03 12:51:23 crc kubenswrapper[5052]: I1203 12:51:23.727384 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/0191d138-7aff-42c2-88d2-fbbaf1722bfd-public-tls-certs\") pod \"keystone-7fd545bd9-2lc7s\" (UID: \"0191d138-7aff-42c2-88d2-fbbaf1722bfd\") " pod="openstack-kuttl-tests/keystone-7fd545bd9-2lc7s" Dec 03 12:51:23 crc kubenswrapper[5052]: I1203 12:51:23.727439 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/0191d138-7aff-42c2-88d2-fbbaf1722bfd-config-data\") pod \"keystone-7fd545bd9-2lc7s\" (UID: \"0191d138-7aff-42c2-88d2-fbbaf1722bfd\") " pod="openstack-kuttl-tests/keystone-7fd545bd9-2lc7s" Dec 03 12:51:23 crc kubenswrapper[5052]: I1203 12:51:23.727477 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0191d138-7aff-42c2-88d2-fbbaf1722bfd-combined-ca-bundle\") pod \"keystone-7fd545bd9-2lc7s\" (UID: \"0191d138-7aff-42c2-88d2-fbbaf1722bfd\") " pod="openstack-kuttl-tests/keystone-7fd545bd9-2lc7s" Dec 03 12:51:23 crc kubenswrapper[5052]: I1203 12:51:23.727526 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/0191d138-7aff-42c2-88d2-fbbaf1722bfd-scripts\") pod \"keystone-7fd545bd9-2lc7s\" (UID: \"0191d138-7aff-42c2-88d2-fbbaf1722bfd\") " pod="openstack-kuttl-tests/keystone-7fd545bd9-2lc7s" Dec 03 12:51:23 crc kubenswrapper[5052]: I1203 12:51:23.829163 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/0191d138-7aff-42c2-88d2-fbbaf1722bfd-fernet-keys\") pod \"keystone-7fd545bd9-2lc7s\" (UID: \"0191d138-7aff-42c2-88d2-fbbaf1722bfd\") " pod="openstack-kuttl-tests/keystone-7fd545bd9-2lc7s" Dec 03 12:51:23 crc kubenswrapper[5052]: I1203 12:51:23.829239 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/0191d138-7aff-42c2-88d2-fbbaf1722bfd-public-tls-certs\") pod \"keystone-7fd545bd9-2lc7s\" (UID: \"0191d138-7aff-42c2-88d2-fbbaf1722bfd\") " pod="openstack-kuttl-tests/keystone-7fd545bd9-2lc7s" Dec 03 12:51:23 crc kubenswrapper[5052]: I1203 12:51:23.829284 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/0191d138-7aff-42c2-88d2-fbbaf1722bfd-config-data\") pod \"keystone-7fd545bd9-2lc7s\" (UID: \"0191d138-7aff-42c2-88d2-fbbaf1722bfd\") " pod="openstack-kuttl-tests/keystone-7fd545bd9-2lc7s" Dec 03 12:51:23 crc kubenswrapper[5052]: I1203 12:51:23.829311 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0191d138-7aff-42c2-88d2-fbbaf1722bfd-combined-ca-bundle\") pod \"keystone-7fd545bd9-2lc7s\" (UID: \"0191d138-7aff-42c2-88d2-fbbaf1722bfd\") " pod="openstack-kuttl-tests/keystone-7fd545bd9-2lc7s" Dec 03 12:51:23 crc kubenswrapper[5052]: I1203 12:51:23.829335 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/0191d138-7aff-42c2-88d2-fbbaf1722bfd-scripts\") pod \"keystone-7fd545bd9-2lc7s\" (UID: \"0191d138-7aff-42c2-88d2-fbbaf1722bfd\") " pod="openstack-kuttl-tests/keystone-7fd545bd9-2lc7s" Dec 03 12:51:23 crc kubenswrapper[5052]: I1203 12:51:23.829369 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/0191d138-7aff-42c2-88d2-fbbaf1722bfd-credential-keys\") pod \"keystone-7fd545bd9-2lc7s\" (UID: \"0191d138-7aff-42c2-88d2-fbbaf1722bfd\") " pod="openstack-kuttl-tests/keystone-7fd545bd9-2lc7s" Dec 03 12:51:23 crc kubenswrapper[5052]: I1203 12:51:23.829437 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-xqxxd\" (UniqueName: \"kubernetes.io/projected/0191d138-7aff-42c2-88d2-fbbaf1722bfd-kube-api-access-xqxxd\") pod \"keystone-7fd545bd9-2lc7s\" (UID: \"0191d138-7aff-42c2-88d2-fbbaf1722bfd\") " pod="openstack-kuttl-tests/keystone-7fd545bd9-2lc7s" Dec 03 12:51:23 crc kubenswrapper[5052]: I1203 12:51:23.829499 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/0191d138-7aff-42c2-88d2-fbbaf1722bfd-internal-tls-certs\") pod \"keystone-7fd545bd9-2lc7s\" (UID: \"0191d138-7aff-42c2-88d2-fbbaf1722bfd\") " pod="openstack-kuttl-tests/keystone-7fd545bd9-2lc7s" Dec 03 12:51:23 crc kubenswrapper[5052]: I1203 12:51:23.835248 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/0191d138-7aff-42c2-88d2-fbbaf1722bfd-scripts\") pod \"keystone-7fd545bd9-2lc7s\" (UID: \"0191d138-7aff-42c2-88d2-fbbaf1722bfd\") " pod="openstack-kuttl-tests/keystone-7fd545bd9-2lc7s" Dec 03 12:51:23 crc kubenswrapper[5052]: I1203 12:51:23.835647 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/0191d138-7aff-42c2-88d2-fbbaf1722bfd-fernet-keys\") pod \"keystone-7fd545bd9-2lc7s\" (UID: \"0191d138-7aff-42c2-88d2-fbbaf1722bfd\") " pod="openstack-kuttl-tests/keystone-7fd545bd9-2lc7s" Dec 03 12:51:23 crc kubenswrapper[5052]: I1203 12:51:23.835665 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/0191d138-7aff-42c2-88d2-fbbaf1722bfd-config-data\") pod \"keystone-7fd545bd9-2lc7s\" (UID: \"0191d138-7aff-42c2-88d2-fbbaf1722bfd\") " pod="openstack-kuttl-tests/keystone-7fd545bd9-2lc7s" Dec 03 12:51:23 crc kubenswrapper[5052]: I1203 12:51:23.835704 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0191d138-7aff-42c2-88d2-fbbaf1722bfd-combined-ca-bundle\") pod \"keystone-7fd545bd9-2lc7s\" (UID: \"0191d138-7aff-42c2-88d2-fbbaf1722bfd\") " pod="openstack-kuttl-tests/keystone-7fd545bd9-2lc7s" Dec 03 12:51:23 crc kubenswrapper[5052]: I1203 12:51:23.835928 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/0191d138-7aff-42c2-88d2-fbbaf1722bfd-public-tls-certs\") pod \"keystone-7fd545bd9-2lc7s\" (UID: \"0191d138-7aff-42c2-88d2-fbbaf1722bfd\") " pod="openstack-kuttl-tests/keystone-7fd545bd9-2lc7s" Dec 03 12:51:23 crc kubenswrapper[5052]: I1203 12:51:23.836914 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/0191d138-7aff-42c2-88d2-fbbaf1722bfd-credential-keys\") pod \"keystone-7fd545bd9-2lc7s\" (UID: \"0191d138-7aff-42c2-88d2-fbbaf1722bfd\") " pod="openstack-kuttl-tests/keystone-7fd545bd9-2lc7s" Dec 03 12:51:23 crc kubenswrapper[5052]: I1203 12:51:23.860035 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/0191d138-7aff-42c2-88d2-fbbaf1722bfd-internal-tls-certs\") pod \"keystone-7fd545bd9-2lc7s\" (UID: \"0191d138-7aff-42c2-88d2-fbbaf1722bfd\") " pod="openstack-kuttl-tests/keystone-7fd545bd9-2lc7s" Dec 03 12:51:23 crc kubenswrapper[5052]: I1203 12:51:23.869046 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-xqxxd\" (UniqueName: \"kubernetes.io/projected/0191d138-7aff-42c2-88d2-fbbaf1722bfd-kube-api-access-xqxxd\") pod \"keystone-7fd545bd9-2lc7s\" (UID: \"0191d138-7aff-42c2-88d2-fbbaf1722bfd\") " pod="openstack-kuttl-tests/keystone-7fd545bd9-2lc7s" Dec 03 12:51:23 crc kubenswrapper[5052]: I1203 12:51:23.893895 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-7fd545bd9-2lc7s" Dec 03 12:51:24 crc kubenswrapper[5052]: I1203 12:51:24.383700 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/keystone-7fd545bd9-2lc7s"] Dec 03 12:51:24 crc kubenswrapper[5052]: I1203 12:51:24.472981 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"3758f9f1-0328-4860-a346-e3691983a552","Type":"ContainerStarted","Data":"dc5958e625b0b6fa167d00e8a75177456bfdc7cc243434c1e02e56ad6156011c"} Dec 03 12:51:24 crc kubenswrapper[5052]: I1203 12:51:24.473079 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/cinder-api-0" podUID="3758f9f1-0328-4860-a346-e3691983a552" containerName="cinder-api-log" containerID="cri-o://de321d5d5126d4d9f5462912231b30a9ea3db3810ab591537461b693b09d3c5a" gracePeriod=30 Dec 03 12:51:24 crc kubenswrapper[5052]: I1203 12:51:24.473190 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/cinder-api-0" podUID="3758f9f1-0328-4860-a346-e3691983a552" containerName="cinder-api" containerID="cri-o://dc5958e625b0b6fa167d00e8a75177456bfdc7cc243434c1e02e56ad6156011c" gracePeriod=30 Dec 03 12:51:24 crc kubenswrapper[5052]: I1203 12:51:24.473369 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:51:24 crc kubenswrapper[5052]: I1203 12:51:24.483091 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-7fd545bd9-2lc7s" event={"ID":"0191d138-7aff-42c2-88d2-fbbaf1722bfd","Type":"ContainerStarted","Data":"1e486dffdb8f8e897858840beacc3d7ec4d117da56b1dca2b0d95d7723129830"} Dec 03 12:51:24 crc kubenswrapper[5052]: I1203 12:51:24.483138 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:51:24 crc kubenswrapper[5052]: I1203 12:51:24.483442 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:51:24 crc kubenswrapper[5052]: I1203 12:51:24.504657 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/cinder-api-0" podStartSLOduration=4.5046302449999995 podStartE2EDuration="4.504630245s" podCreationTimestamp="2025-12-03 12:51:20 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:51:24.501461084 +0000 UTC m=+5475.089702858" watchObservedRunningTime="2025-12-03 12:51:24.504630245 +0000 UTC m=+5475.092872019" Dec 03 12:51:24 crc kubenswrapper[5052]: I1203 12:51:24.535186 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:51:24 crc kubenswrapper[5052]: I1203 12:51:24.535229 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:51:24 crc kubenswrapper[5052]: I1203 12:51:24.581404 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:51:24 crc kubenswrapper[5052]: I1203 12:51:24.601335 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:51:25 crc kubenswrapper[5052]: I1203 12:51:25.008459 5052 prober.go:107] "Probe failed" probeType="Liveness" pod="openstack-kuttl-tests/barbican-api-5bdcd5dc6-vqlkq" podUID="539d9404-7518-4f03-815d-568a43eedb0b" containerName="barbican-api-log" probeResult="failure" output="HTTP probe failed with statuscode: 500" Dec 03 12:51:25 crc kubenswrapper[5052]: I1203 12:51:25.104103 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:51:25 crc kubenswrapper[5052]: I1203 12:51:25.260357 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/3758f9f1-0328-4860-a346-e3691983a552-etc-machine-id\") pod \"3758f9f1-0328-4860-a346-e3691983a552\" (UID: \"3758f9f1-0328-4860-a346-e3691983a552\") " Dec 03 12:51:25 crc kubenswrapper[5052]: I1203 12:51:25.260432 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/3758f9f1-0328-4860-a346-e3691983a552-logs\") pod \"3758f9f1-0328-4860-a346-e3691983a552\" (UID: \"3758f9f1-0328-4860-a346-e3691983a552\") " Dec 03 12:51:25 crc kubenswrapper[5052]: I1203 12:51:25.260495 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3758f9f1-0328-4860-a346-e3691983a552-combined-ca-bundle\") pod \"3758f9f1-0328-4860-a346-e3691983a552\" (UID: \"3758f9f1-0328-4860-a346-e3691983a552\") " Dec 03 12:51:25 crc kubenswrapper[5052]: I1203 12:51:25.260507 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/3758f9f1-0328-4860-a346-e3691983a552-etc-machine-id" (OuterVolumeSpecName: "etc-machine-id") pod "3758f9f1-0328-4860-a346-e3691983a552" (UID: "3758f9f1-0328-4860-a346-e3691983a552"). InnerVolumeSpecName "etc-machine-id". PluginName "kubernetes.io/host-path", VolumeGidValue "" Dec 03 12:51:25 crc kubenswrapper[5052]: I1203 12:51:25.260546 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/3758f9f1-0328-4860-a346-e3691983a552-config-data\") pod \"3758f9f1-0328-4860-a346-e3691983a552\" (UID: \"3758f9f1-0328-4860-a346-e3691983a552\") " Dec 03 12:51:25 crc kubenswrapper[5052]: I1203 12:51:25.260565 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/3758f9f1-0328-4860-a346-e3691983a552-scripts\") pod \"3758f9f1-0328-4860-a346-e3691983a552\" (UID: \"3758f9f1-0328-4860-a346-e3691983a552\") " Dec 03 12:51:25 crc kubenswrapper[5052]: I1203 12:51:25.260667 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/3758f9f1-0328-4860-a346-e3691983a552-config-data-custom\") pod \"3758f9f1-0328-4860-a346-e3691983a552\" (UID: \"3758f9f1-0328-4860-a346-e3691983a552\") " Dec 03 12:51:25 crc kubenswrapper[5052]: I1203 12:51:25.260687 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mxdqb\" (UniqueName: \"kubernetes.io/projected/3758f9f1-0328-4860-a346-e3691983a552-kube-api-access-mxdqb\") pod \"3758f9f1-0328-4860-a346-e3691983a552\" (UID: \"3758f9f1-0328-4860-a346-e3691983a552\") " Dec 03 12:51:25 crc kubenswrapper[5052]: I1203 12:51:25.261048 5052 reconciler_common.go:293] "Volume detached for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/3758f9f1-0328-4860-a346-e3691983a552-etc-machine-id\") on node \"crc\" DevicePath \"\"" Dec 03 12:51:25 crc kubenswrapper[5052]: I1203 12:51:25.261102 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3758f9f1-0328-4860-a346-e3691983a552-logs" (OuterVolumeSpecName: "logs") pod "3758f9f1-0328-4860-a346-e3691983a552" (UID: "3758f9f1-0328-4860-a346-e3691983a552"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:51:25 crc kubenswrapper[5052]: I1203 12:51:25.269811 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3758f9f1-0328-4860-a346-e3691983a552-scripts" (OuterVolumeSpecName: "scripts") pod "3758f9f1-0328-4860-a346-e3691983a552" (UID: "3758f9f1-0328-4860-a346-e3691983a552"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:51:25 crc kubenswrapper[5052]: I1203 12:51:25.269944 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3758f9f1-0328-4860-a346-e3691983a552-kube-api-access-mxdqb" (OuterVolumeSpecName: "kube-api-access-mxdqb") pod "3758f9f1-0328-4860-a346-e3691983a552" (UID: "3758f9f1-0328-4860-a346-e3691983a552"). InnerVolumeSpecName "kube-api-access-mxdqb". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:51:25 crc kubenswrapper[5052]: I1203 12:51:25.270036 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3758f9f1-0328-4860-a346-e3691983a552-config-data-custom" (OuterVolumeSpecName: "config-data-custom") pod "3758f9f1-0328-4860-a346-e3691983a552" (UID: "3758f9f1-0328-4860-a346-e3691983a552"). InnerVolumeSpecName "config-data-custom". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:51:25 crc kubenswrapper[5052]: I1203 12:51:25.295420 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3758f9f1-0328-4860-a346-e3691983a552-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "3758f9f1-0328-4860-a346-e3691983a552" (UID: "3758f9f1-0328-4860-a346-e3691983a552"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:51:25 crc kubenswrapper[5052]: I1203 12:51:25.327022 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3758f9f1-0328-4860-a346-e3691983a552-config-data" (OuterVolumeSpecName: "config-data") pod "3758f9f1-0328-4860-a346-e3691983a552" (UID: "3758f9f1-0328-4860-a346-e3691983a552"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:51:25 crc kubenswrapper[5052]: I1203 12:51:25.363338 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/3758f9f1-0328-4860-a346-e3691983a552-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:51:25 crc kubenswrapper[5052]: I1203 12:51:25.363392 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3758f9f1-0328-4860-a346-e3691983a552-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:51:25 crc kubenswrapper[5052]: I1203 12:51:25.363409 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/3758f9f1-0328-4860-a346-e3691983a552-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:51:25 crc kubenswrapper[5052]: I1203 12:51:25.363421 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/3758f9f1-0328-4860-a346-e3691983a552-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:51:25 crc kubenswrapper[5052]: I1203 12:51:25.363429 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/3758f9f1-0328-4860-a346-e3691983a552-config-data-custom\") on node \"crc\" DevicePath \"\"" Dec 03 12:51:25 crc kubenswrapper[5052]: I1203 12:51:25.363440 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-mxdqb\" (UniqueName: \"kubernetes.io/projected/3758f9f1-0328-4860-a346-e3691983a552-kube-api-access-mxdqb\") on node \"crc\" DevicePath \"\"" Dec 03 12:51:25 crc kubenswrapper[5052]: I1203 12:51:25.490700 5052 generic.go:334] "Generic (PLEG): container finished" podID="3758f9f1-0328-4860-a346-e3691983a552" containerID="dc5958e625b0b6fa167d00e8a75177456bfdc7cc243434c1e02e56ad6156011c" exitCode=0 Dec 03 12:51:25 crc kubenswrapper[5052]: I1203 12:51:25.490740 5052 generic.go:334] "Generic (PLEG): container finished" podID="3758f9f1-0328-4860-a346-e3691983a552" containerID="de321d5d5126d4d9f5462912231b30a9ea3db3810ab591537461b693b09d3c5a" exitCode=143 Dec 03 12:51:25 crc kubenswrapper[5052]: I1203 12:51:25.490782 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"3758f9f1-0328-4860-a346-e3691983a552","Type":"ContainerDied","Data":"dc5958e625b0b6fa167d00e8a75177456bfdc7cc243434c1e02e56ad6156011c"} Dec 03 12:51:25 crc kubenswrapper[5052]: I1203 12:51:25.490808 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"3758f9f1-0328-4860-a346-e3691983a552","Type":"ContainerDied","Data":"de321d5d5126d4d9f5462912231b30a9ea3db3810ab591537461b693b09d3c5a"} Dec 03 12:51:25 crc kubenswrapper[5052]: I1203 12:51:25.490818 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"3758f9f1-0328-4860-a346-e3691983a552","Type":"ContainerDied","Data":"2ca85e3c4d60693e2e1ffb8494d1730c75ce2e53ca64040724c027491fcfc3bc"} Dec 03 12:51:25 crc kubenswrapper[5052]: I1203 12:51:25.490833 5052 scope.go:117] "RemoveContainer" containerID="dc5958e625b0b6fa167d00e8a75177456bfdc7cc243434c1e02e56ad6156011c" Dec 03 12:51:25 crc kubenswrapper[5052]: I1203 12:51:25.490961 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:51:25 crc kubenswrapper[5052]: I1203 12:51:25.496820 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-7fd545bd9-2lc7s" event={"ID":"0191d138-7aff-42c2-88d2-fbbaf1722bfd","Type":"ContainerStarted","Data":"ac4ecfa5627541f93acf9cc63e185d94bb2e833e592e7703865bb5d4026f322f"} Dec 03 12:51:25 crc kubenswrapper[5052]: I1203 12:51:25.496862 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:51:25 crc kubenswrapper[5052]: I1203 12:51:25.496876 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/keystone-7fd545bd9-2lc7s" Dec 03 12:51:25 crc kubenswrapper[5052]: I1203 12:51:25.497557 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:51:25 crc kubenswrapper[5052]: I1203 12:51:25.525415 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/keystone-7fd545bd9-2lc7s" podStartSLOduration=2.525393637 podStartE2EDuration="2.525393637s" podCreationTimestamp="2025-12-03 12:51:23 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:51:25.524296196 +0000 UTC m=+5476.112537960" watchObservedRunningTime="2025-12-03 12:51:25.525393637 +0000 UTC m=+5476.113635401" Dec 03 12:51:25 crc kubenswrapper[5052]: I1203 12:51:25.548952 5052 scope.go:117] "RemoveContainer" containerID="de321d5d5126d4d9f5462912231b30a9ea3db3810ab591537461b693b09d3c5a" Dec 03 12:51:25 crc kubenswrapper[5052]: I1203 12:51:25.577571 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 12:51:25 crc kubenswrapper[5052]: I1203 12:51:25.583150 5052 scope.go:117] "RemoveContainer" containerID="dc5958e625b0b6fa167d00e8a75177456bfdc7cc243434c1e02e56ad6156011c" Dec 03 12:51:25 crc kubenswrapper[5052]: E1203 12:51:25.584692 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dc5958e625b0b6fa167d00e8a75177456bfdc7cc243434c1e02e56ad6156011c\": container with ID starting with dc5958e625b0b6fa167d00e8a75177456bfdc7cc243434c1e02e56ad6156011c not found: ID does not exist" containerID="dc5958e625b0b6fa167d00e8a75177456bfdc7cc243434c1e02e56ad6156011c" Dec 03 12:51:25 crc kubenswrapper[5052]: I1203 12:51:25.584724 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dc5958e625b0b6fa167d00e8a75177456bfdc7cc243434c1e02e56ad6156011c"} err="failed to get container status \"dc5958e625b0b6fa167d00e8a75177456bfdc7cc243434c1e02e56ad6156011c\": rpc error: code = NotFound desc = could not find container \"dc5958e625b0b6fa167d00e8a75177456bfdc7cc243434c1e02e56ad6156011c\": container with ID starting with dc5958e625b0b6fa167d00e8a75177456bfdc7cc243434c1e02e56ad6156011c not found: ID does not exist" Dec 03 12:51:25 crc kubenswrapper[5052]: I1203 12:51:25.584754 5052 scope.go:117] "RemoveContainer" containerID="de321d5d5126d4d9f5462912231b30a9ea3db3810ab591537461b693b09d3c5a" Dec 03 12:51:25 crc kubenswrapper[5052]: E1203 12:51:25.586848 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"de321d5d5126d4d9f5462912231b30a9ea3db3810ab591537461b693b09d3c5a\": container with ID starting with de321d5d5126d4d9f5462912231b30a9ea3db3810ab591537461b693b09d3c5a not found: ID does not exist" containerID="de321d5d5126d4d9f5462912231b30a9ea3db3810ab591537461b693b09d3c5a" Dec 03 12:51:25 crc kubenswrapper[5052]: I1203 12:51:25.586880 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"de321d5d5126d4d9f5462912231b30a9ea3db3810ab591537461b693b09d3c5a"} err="failed to get container status \"de321d5d5126d4d9f5462912231b30a9ea3db3810ab591537461b693b09d3c5a\": rpc error: code = NotFound desc = could not find container \"de321d5d5126d4d9f5462912231b30a9ea3db3810ab591537461b693b09d3c5a\": container with ID starting with de321d5d5126d4d9f5462912231b30a9ea3db3810ab591537461b693b09d3c5a not found: ID does not exist" Dec 03 12:51:25 crc kubenswrapper[5052]: I1203 12:51:25.586902 5052 scope.go:117] "RemoveContainer" containerID="dc5958e625b0b6fa167d00e8a75177456bfdc7cc243434c1e02e56ad6156011c" Dec 03 12:51:25 crc kubenswrapper[5052]: I1203 12:51:25.587141 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dc5958e625b0b6fa167d00e8a75177456bfdc7cc243434c1e02e56ad6156011c"} err="failed to get container status \"dc5958e625b0b6fa167d00e8a75177456bfdc7cc243434c1e02e56ad6156011c\": rpc error: code = NotFound desc = could not find container \"dc5958e625b0b6fa167d00e8a75177456bfdc7cc243434c1e02e56ad6156011c\": container with ID starting with dc5958e625b0b6fa167d00e8a75177456bfdc7cc243434c1e02e56ad6156011c not found: ID does not exist" Dec 03 12:51:25 crc kubenswrapper[5052]: I1203 12:51:25.587160 5052 scope.go:117] "RemoveContainer" containerID="de321d5d5126d4d9f5462912231b30a9ea3db3810ab591537461b693b09d3c5a" Dec 03 12:51:25 crc kubenswrapper[5052]: I1203 12:51:25.592993 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"de321d5d5126d4d9f5462912231b30a9ea3db3810ab591537461b693b09d3c5a"} err="failed to get container status \"de321d5d5126d4d9f5462912231b30a9ea3db3810ab591537461b693b09d3c5a\": rpc error: code = NotFound desc = could not find container \"de321d5d5126d4d9f5462912231b30a9ea3db3810ab591537461b693b09d3c5a\": container with ID starting with de321d5d5126d4d9f5462912231b30a9ea3db3810ab591537461b693b09d3c5a not found: ID does not exist" Dec 03 12:51:25 crc kubenswrapper[5052]: I1203 12:51:25.599142 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 12:51:25 crc kubenswrapper[5052]: I1203 12:51:25.608438 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 12:51:25 crc kubenswrapper[5052]: E1203 12:51:25.612956 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="3758f9f1-0328-4860-a346-e3691983a552" containerName="cinder-api-log" Dec 03 12:51:25 crc kubenswrapper[5052]: I1203 12:51:25.612998 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="3758f9f1-0328-4860-a346-e3691983a552" containerName="cinder-api-log" Dec 03 12:51:25 crc kubenswrapper[5052]: E1203 12:51:25.613027 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="3758f9f1-0328-4860-a346-e3691983a552" containerName="cinder-api" Dec 03 12:51:25 crc kubenswrapper[5052]: I1203 12:51:25.613034 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="3758f9f1-0328-4860-a346-e3691983a552" containerName="cinder-api" Dec 03 12:51:25 crc kubenswrapper[5052]: I1203 12:51:25.613738 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="3758f9f1-0328-4860-a346-e3691983a552" containerName="cinder-api-log" Dec 03 12:51:25 crc kubenswrapper[5052]: I1203 12:51:25.613791 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="3758f9f1-0328-4860-a346-e3691983a552" containerName="cinder-api" Dec 03 12:51:25 crc kubenswrapper[5052]: I1203 12:51:25.618241 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:51:25 crc kubenswrapper[5052]: I1203 12:51:25.631320 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 12:51:25 crc kubenswrapper[5052]: I1203 12:51:25.654641 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-cinder-public-svc" Dec 03 12:51:25 crc kubenswrapper[5052]: I1203 12:51:25.654696 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-cinder-internal-svc" Dec 03 12:51:25 crc kubenswrapper[5052]: I1203 12:51:25.654877 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cinder-api-config-data" Dec 03 12:51:25 crc kubenswrapper[5052]: I1203 12:51:25.668681 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/ed40ab59-7f38-4b2a-ad68-85592fe22ef2-scripts\") pod \"cinder-api-0\" (UID: \"ed40ab59-7f38-4b2a-ad68-85592fe22ef2\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:51:25 crc kubenswrapper[5052]: I1203 12:51:25.668889 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p9xm5\" (UniqueName: \"kubernetes.io/projected/ed40ab59-7f38-4b2a-ad68-85592fe22ef2-kube-api-access-p9xm5\") pod \"cinder-api-0\" (UID: \"ed40ab59-7f38-4b2a-ad68-85592fe22ef2\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:51:25 crc kubenswrapper[5052]: I1203 12:51:25.668923 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ed40ab59-7f38-4b2a-ad68-85592fe22ef2-combined-ca-bundle\") pod \"cinder-api-0\" (UID: \"ed40ab59-7f38-4b2a-ad68-85592fe22ef2\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:51:25 crc kubenswrapper[5052]: I1203 12:51:25.668967 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/ed40ab59-7f38-4b2a-ad68-85592fe22ef2-logs\") pod \"cinder-api-0\" (UID: \"ed40ab59-7f38-4b2a-ad68-85592fe22ef2\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:51:25 crc kubenswrapper[5052]: I1203 12:51:25.668988 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/ed40ab59-7f38-4b2a-ad68-85592fe22ef2-config-data\") pod \"cinder-api-0\" (UID: \"ed40ab59-7f38-4b2a-ad68-85592fe22ef2\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:51:25 crc kubenswrapper[5052]: I1203 12:51:25.669033 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/ed40ab59-7f38-4b2a-ad68-85592fe22ef2-internal-tls-certs\") pod \"cinder-api-0\" (UID: \"ed40ab59-7f38-4b2a-ad68-85592fe22ef2\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:51:25 crc kubenswrapper[5052]: I1203 12:51:25.669057 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/ed40ab59-7f38-4b2a-ad68-85592fe22ef2-config-data-custom\") pod \"cinder-api-0\" (UID: \"ed40ab59-7f38-4b2a-ad68-85592fe22ef2\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:51:25 crc kubenswrapper[5052]: I1203 12:51:25.669100 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/ed40ab59-7f38-4b2a-ad68-85592fe22ef2-etc-machine-id\") pod \"cinder-api-0\" (UID: \"ed40ab59-7f38-4b2a-ad68-85592fe22ef2\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:51:25 crc kubenswrapper[5052]: I1203 12:51:25.669144 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/ed40ab59-7f38-4b2a-ad68-85592fe22ef2-public-tls-certs\") pod \"cinder-api-0\" (UID: \"ed40ab59-7f38-4b2a-ad68-85592fe22ef2\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:51:25 crc kubenswrapper[5052]: I1203 12:51:25.692022 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3758f9f1-0328-4860-a346-e3691983a552" path="/var/lib/kubelet/pods/3758f9f1-0328-4860-a346-e3691983a552/volumes" Dec 03 12:51:25 crc kubenswrapper[5052]: I1203 12:51:25.771886 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-p9xm5\" (UniqueName: \"kubernetes.io/projected/ed40ab59-7f38-4b2a-ad68-85592fe22ef2-kube-api-access-p9xm5\") pod \"cinder-api-0\" (UID: \"ed40ab59-7f38-4b2a-ad68-85592fe22ef2\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:51:25 crc kubenswrapper[5052]: I1203 12:51:25.771947 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ed40ab59-7f38-4b2a-ad68-85592fe22ef2-combined-ca-bundle\") pod \"cinder-api-0\" (UID: \"ed40ab59-7f38-4b2a-ad68-85592fe22ef2\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:51:25 crc kubenswrapper[5052]: I1203 12:51:25.772008 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/ed40ab59-7f38-4b2a-ad68-85592fe22ef2-logs\") pod \"cinder-api-0\" (UID: \"ed40ab59-7f38-4b2a-ad68-85592fe22ef2\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:51:25 crc kubenswrapper[5052]: I1203 12:51:25.772044 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/ed40ab59-7f38-4b2a-ad68-85592fe22ef2-config-data\") pod \"cinder-api-0\" (UID: \"ed40ab59-7f38-4b2a-ad68-85592fe22ef2\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:51:25 crc kubenswrapper[5052]: I1203 12:51:25.772070 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/ed40ab59-7f38-4b2a-ad68-85592fe22ef2-internal-tls-certs\") pod \"cinder-api-0\" (UID: \"ed40ab59-7f38-4b2a-ad68-85592fe22ef2\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:51:25 crc kubenswrapper[5052]: I1203 12:51:25.772090 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/ed40ab59-7f38-4b2a-ad68-85592fe22ef2-config-data-custom\") pod \"cinder-api-0\" (UID: \"ed40ab59-7f38-4b2a-ad68-85592fe22ef2\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:51:25 crc kubenswrapper[5052]: I1203 12:51:25.772138 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/ed40ab59-7f38-4b2a-ad68-85592fe22ef2-etc-machine-id\") pod \"cinder-api-0\" (UID: \"ed40ab59-7f38-4b2a-ad68-85592fe22ef2\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:51:25 crc kubenswrapper[5052]: I1203 12:51:25.772527 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/ed40ab59-7f38-4b2a-ad68-85592fe22ef2-etc-machine-id\") pod \"cinder-api-0\" (UID: \"ed40ab59-7f38-4b2a-ad68-85592fe22ef2\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:51:25 crc kubenswrapper[5052]: I1203 12:51:25.772198 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/ed40ab59-7f38-4b2a-ad68-85592fe22ef2-public-tls-certs\") pod \"cinder-api-0\" (UID: \"ed40ab59-7f38-4b2a-ad68-85592fe22ef2\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:51:25 crc kubenswrapper[5052]: I1203 12:51:25.772885 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/ed40ab59-7f38-4b2a-ad68-85592fe22ef2-logs\") pod \"cinder-api-0\" (UID: \"ed40ab59-7f38-4b2a-ad68-85592fe22ef2\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:51:25 crc kubenswrapper[5052]: I1203 12:51:25.774209 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/ed40ab59-7f38-4b2a-ad68-85592fe22ef2-scripts\") pod \"cinder-api-0\" (UID: \"ed40ab59-7f38-4b2a-ad68-85592fe22ef2\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:51:25 crc kubenswrapper[5052]: I1203 12:51:25.778138 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/ed40ab59-7f38-4b2a-ad68-85592fe22ef2-public-tls-certs\") pod \"cinder-api-0\" (UID: \"ed40ab59-7f38-4b2a-ad68-85592fe22ef2\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:51:25 crc kubenswrapper[5052]: I1203 12:51:25.780125 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/ed40ab59-7f38-4b2a-ad68-85592fe22ef2-config-data\") pod \"cinder-api-0\" (UID: \"ed40ab59-7f38-4b2a-ad68-85592fe22ef2\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:51:25 crc kubenswrapper[5052]: I1203 12:51:25.780370 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ed40ab59-7f38-4b2a-ad68-85592fe22ef2-combined-ca-bundle\") pod \"cinder-api-0\" (UID: \"ed40ab59-7f38-4b2a-ad68-85592fe22ef2\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:51:25 crc kubenswrapper[5052]: I1203 12:51:25.782625 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/ed40ab59-7f38-4b2a-ad68-85592fe22ef2-scripts\") pod \"cinder-api-0\" (UID: \"ed40ab59-7f38-4b2a-ad68-85592fe22ef2\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:51:25 crc kubenswrapper[5052]: I1203 12:51:25.784166 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/ed40ab59-7f38-4b2a-ad68-85592fe22ef2-config-data-custom\") pod \"cinder-api-0\" (UID: \"ed40ab59-7f38-4b2a-ad68-85592fe22ef2\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:51:25 crc kubenswrapper[5052]: I1203 12:51:25.787772 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-p9xm5\" (UniqueName: \"kubernetes.io/projected/ed40ab59-7f38-4b2a-ad68-85592fe22ef2-kube-api-access-p9xm5\") pod \"cinder-api-0\" (UID: \"ed40ab59-7f38-4b2a-ad68-85592fe22ef2\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:51:25 crc kubenswrapper[5052]: I1203 12:51:25.795627 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/ed40ab59-7f38-4b2a-ad68-85592fe22ef2-internal-tls-certs\") pod \"cinder-api-0\" (UID: \"ed40ab59-7f38-4b2a-ad68-85592fe22ef2\") " pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:51:25 crc kubenswrapper[5052]: I1203 12:51:25.867709 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:51:25 crc kubenswrapper[5052]: I1203 12:51:25.989851 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:51:26 crc kubenswrapper[5052]: I1203 12:51:26.519972 5052 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Dec 03 12:51:26 crc kubenswrapper[5052]: I1203 12:51:26.520312 5052 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Dec 03 12:51:26 crc kubenswrapper[5052]: W1203 12:51:26.522065 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poded40ab59_7f38_4b2a_ad68_85592fe22ef2.slice/crio-cc40798a75bb61e54d557a7c32d26f21e44ee687a79c1f4c496f8ad7b15c4381 WatchSource:0}: Error finding container cc40798a75bb61e54d557a7c32d26f21e44ee687a79c1f4c496f8ad7b15c4381: Status 404 returned error can't find the container with id cc40798a75bb61e54d557a7c32d26f21e44ee687a79c1f4c496f8ad7b15c4381 Dec 03 12:51:26 crc kubenswrapper[5052]: I1203 12:51:26.525604 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 12:51:26 crc kubenswrapper[5052]: I1203 12:51:26.748713 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:51:26 crc kubenswrapper[5052]: I1203 12:51:26.950376 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:51:27 crc kubenswrapper[5052]: I1203 12:51:27.533653 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"ed40ab59-7f38-4b2a-ad68-85592fe22ef2","Type":"ContainerStarted","Data":"717a9f8ebf7eb71e44fa8c9d63200534cfb506f60207616c8ff568f62f9287a8"} Dec 03 12:51:27 crc kubenswrapper[5052]: I1203 12:51:27.533936 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"ed40ab59-7f38-4b2a-ad68-85592fe22ef2","Type":"ContainerStarted","Data":"cc40798a75bb61e54d557a7c32d26f21e44ee687a79c1f4c496f8ad7b15c4381"} Dec 03 12:51:27 crc kubenswrapper[5052]: I1203 12:51:27.533983 5052 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Dec 03 12:51:27 crc kubenswrapper[5052]: I1203 12:51:27.534028 5052 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Dec 03 12:51:27 crc kubenswrapper[5052]: I1203 12:51:27.687194 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:51:27 crc kubenswrapper[5052]: I1203 12:51:27.689817 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:51:28 crc kubenswrapper[5052]: I1203 12:51:28.543438 5052 generic.go:334] "Generic (PLEG): container finished" podID="ffe1f27e-ba05-4da0-a02b-a73b3bf63bc3" containerID="174dc76c1d02fad9f57931bbb0c9542148e337ada369ca2b722e53cd15f06251" exitCode=0 Dec 03 12:51:28 crc kubenswrapper[5052]: I1203 12:51:28.543646 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-db-sync-jxbww" event={"ID":"ffe1f27e-ba05-4da0-a02b-a73b3bf63bc3","Type":"ContainerDied","Data":"174dc76c1d02fad9f57931bbb0c9542148e337ada369ca2b722e53cd15f06251"} Dec 03 12:51:28 crc kubenswrapper[5052]: I1203 12:51:28.545480 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"ed40ab59-7f38-4b2a-ad68-85592fe22ef2","Type":"ContainerStarted","Data":"0d0807a346577955962a640bd5fe5f8e366a538b40576ef0017fa65770ea3118"} Dec 03 12:51:28 crc kubenswrapper[5052]: I1203 12:51:28.589876 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/cinder-api-0" podStartSLOduration=3.589855455 podStartE2EDuration="3.589855455s" podCreationTimestamp="2025-12-03 12:51:25 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:51:28.585111249 +0000 UTC m=+5479.173353023" watchObservedRunningTime="2025-12-03 12:51:28.589855455 +0000 UTC m=+5479.178097219" Dec 03 12:51:28 crc kubenswrapper[5052]: I1203 12:51:28.691585 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/barbican-api-5bdcd5dc6-vqlkq" Dec 03 12:51:28 crc kubenswrapper[5052]: I1203 12:51:28.856572 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/barbican-api-5bdcd5dc6-vqlkq" Dec 03 12:51:29 crc kubenswrapper[5052]: I1203 12:51:29.379887 5052 patch_prober.go:28] interesting pod/machine-config-daemon-74bj6 container/machine-config-daemon namespace/openshift-machine-config-operator: Liveness probe status=failure output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" start-of-body= Dec 03 12:51:29 crc kubenswrapper[5052]: I1203 12:51:29.380266 5052 prober.go:107] "Probe failed" probeType="Liveness" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" containerName="machine-config-daemon" probeResult="failure" output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" Dec 03 12:51:29 crc kubenswrapper[5052]: I1203 12:51:29.558155 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:51:29 crc kubenswrapper[5052]: I1203 12:51:29.966599 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-db-sync-jxbww" Dec 03 12:51:30 crc kubenswrapper[5052]: I1203 12:51:30.160141 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nhfzw\" (UniqueName: \"kubernetes.io/projected/ffe1f27e-ba05-4da0-a02b-a73b3bf63bc3-kube-api-access-nhfzw\") pod \"ffe1f27e-ba05-4da0-a02b-a73b3bf63bc3\" (UID: \"ffe1f27e-ba05-4da0-a02b-a73b3bf63bc3\") " Dec 03 12:51:30 crc kubenswrapper[5052]: I1203 12:51:30.160215 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/ffe1f27e-ba05-4da0-a02b-a73b3bf63bc3-config\") pod \"ffe1f27e-ba05-4da0-a02b-a73b3bf63bc3\" (UID: \"ffe1f27e-ba05-4da0-a02b-a73b3bf63bc3\") " Dec 03 12:51:30 crc kubenswrapper[5052]: I1203 12:51:30.160276 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ffe1f27e-ba05-4da0-a02b-a73b3bf63bc3-combined-ca-bundle\") pod \"ffe1f27e-ba05-4da0-a02b-a73b3bf63bc3\" (UID: \"ffe1f27e-ba05-4da0-a02b-a73b3bf63bc3\") " Dec 03 12:51:30 crc kubenswrapper[5052]: I1203 12:51:30.177352 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ffe1f27e-ba05-4da0-a02b-a73b3bf63bc3-kube-api-access-nhfzw" (OuterVolumeSpecName: "kube-api-access-nhfzw") pod "ffe1f27e-ba05-4da0-a02b-a73b3bf63bc3" (UID: "ffe1f27e-ba05-4da0-a02b-a73b3bf63bc3"). InnerVolumeSpecName "kube-api-access-nhfzw". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:51:30 crc kubenswrapper[5052]: I1203 12:51:30.193698 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ffe1f27e-ba05-4da0-a02b-a73b3bf63bc3-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "ffe1f27e-ba05-4da0-a02b-a73b3bf63bc3" (UID: "ffe1f27e-ba05-4da0-a02b-a73b3bf63bc3"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:51:30 crc kubenswrapper[5052]: I1203 12:51:30.209233 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ffe1f27e-ba05-4da0-a02b-a73b3bf63bc3-config" (OuterVolumeSpecName: "config") pod "ffe1f27e-ba05-4da0-a02b-a73b3bf63bc3" (UID: "ffe1f27e-ba05-4da0-a02b-a73b3bf63bc3"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:51:30 crc kubenswrapper[5052]: I1203 12:51:30.262223 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-nhfzw\" (UniqueName: \"kubernetes.io/projected/ffe1f27e-ba05-4da0-a02b-a73b3bf63bc3-kube-api-access-nhfzw\") on node \"crc\" DevicePath \"\"" Dec 03 12:51:30 crc kubenswrapper[5052]: I1203 12:51:30.262253 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/ffe1f27e-ba05-4da0-a02b-a73b3bf63bc3-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:51:30 crc kubenswrapper[5052]: I1203 12:51:30.262264 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ffe1f27e-ba05-4da0-a02b-a73b3bf63bc3-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:51:30 crc kubenswrapper[5052]: I1203 12:51:30.567887 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-db-sync-jxbww" event={"ID":"ffe1f27e-ba05-4da0-a02b-a73b3bf63bc3","Type":"ContainerDied","Data":"3e995213ff11ab92b3c3d6997def76b4e409dfb320b2e2d8ea6e50d98cd1e651"} Dec 03 12:51:30 crc kubenswrapper[5052]: I1203 12:51:30.567951 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3e995213ff11ab92b3c3d6997def76b4e409dfb320b2e2d8ea6e50d98cd1e651" Dec 03 12:51:30 crc kubenswrapper[5052]: I1203 12:51:30.567899 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-db-sync-jxbww" Dec 03 12:51:30 crc kubenswrapper[5052]: I1203 12:51:30.718070 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/neutron-57cc9dddbf-g8d9w"] Dec 03 12:51:30 crc kubenswrapper[5052]: E1203 12:51:30.718780 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="ffe1f27e-ba05-4da0-a02b-a73b3bf63bc3" containerName="neutron-db-sync" Dec 03 12:51:30 crc kubenswrapper[5052]: I1203 12:51:30.718807 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="ffe1f27e-ba05-4da0-a02b-a73b3bf63bc3" containerName="neutron-db-sync" Dec 03 12:51:30 crc kubenswrapper[5052]: I1203 12:51:30.719060 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="ffe1f27e-ba05-4da0-a02b-a73b3bf63bc3" containerName="neutron-db-sync" Dec 03 12:51:30 crc kubenswrapper[5052]: I1203 12:51:30.720160 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-57cc9dddbf-g8d9w" Dec 03 12:51:30 crc kubenswrapper[5052]: I1203 12:51:30.726128 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"neutron-neutron-dockercfg-mhnq6" Dec 03 12:51:30 crc kubenswrapper[5052]: I1203 12:51:30.726355 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-neutron-ovndbs" Dec 03 12:51:30 crc kubenswrapper[5052]: I1203 12:51:30.726488 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"neutron-httpd-config" Dec 03 12:51:30 crc kubenswrapper[5052]: I1203 12:51:30.726617 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"neutron-config" Dec 03 12:51:30 crc kubenswrapper[5052]: I1203 12:51:30.727663 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-57cc9dddbf-g8d9w"] Dec 03 12:51:30 crc kubenswrapper[5052]: I1203 12:51:30.872487 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/0fc6a196-8a8e-4daf-9bbe-48d28e2e1191-httpd-config\") pod \"neutron-57cc9dddbf-g8d9w\" (UID: \"0fc6a196-8a8e-4daf-9bbe-48d28e2e1191\") " pod="openstack-kuttl-tests/neutron-57cc9dddbf-g8d9w" Dec 03 12:51:30 crc kubenswrapper[5052]: I1203 12:51:30.872582 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6qlpv\" (UniqueName: \"kubernetes.io/projected/0fc6a196-8a8e-4daf-9bbe-48d28e2e1191-kube-api-access-6qlpv\") pod \"neutron-57cc9dddbf-g8d9w\" (UID: \"0fc6a196-8a8e-4daf-9bbe-48d28e2e1191\") " pod="openstack-kuttl-tests/neutron-57cc9dddbf-g8d9w" Dec 03 12:51:30 crc kubenswrapper[5052]: I1203 12:51:30.872668 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0fc6a196-8a8e-4daf-9bbe-48d28e2e1191-combined-ca-bundle\") pod \"neutron-57cc9dddbf-g8d9w\" (UID: \"0fc6a196-8a8e-4daf-9bbe-48d28e2e1191\") " pod="openstack-kuttl-tests/neutron-57cc9dddbf-g8d9w" Dec 03 12:51:30 crc kubenswrapper[5052]: I1203 12:51:30.872704 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/0fc6a196-8a8e-4daf-9bbe-48d28e2e1191-ovndb-tls-certs\") pod \"neutron-57cc9dddbf-g8d9w\" (UID: \"0fc6a196-8a8e-4daf-9bbe-48d28e2e1191\") " pod="openstack-kuttl-tests/neutron-57cc9dddbf-g8d9w" Dec 03 12:51:30 crc kubenswrapper[5052]: I1203 12:51:30.872762 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/0fc6a196-8a8e-4daf-9bbe-48d28e2e1191-config\") pod \"neutron-57cc9dddbf-g8d9w\" (UID: \"0fc6a196-8a8e-4daf-9bbe-48d28e2e1191\") " pod="openstack-kuttl-tests/neutron-57cc9dddbf-g8d9w" Dec 03 12:51:30 crc kubenswrapper[5052]: I1203 12:51:30.974748 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0fc6a196-8a8e-4daf-9bbe-48d28e2e1191-combined-ca-bundle\") pod \"neutron-57cc9dddbf-g8d9w\" (UID: \"0fc6a196-8a8e-4daf-9bbe-48d28e2e1191\") " pod="openstack-kuttl-tests/neutron-57cc9dddbf-g8d9w" Dec 03 12:51:30 crc kubenswrapper[5052]: I1203 12:51:30.974819 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/0fc6a196-8a8e-4daf-9bbe-48d28e2e1191-ovndb-tls-certs\") pod \"neutron-57cc9dddbf-g8d9w\" (UID: \"0fc6a196-8a8e-4daf-9bbe-48d28e2e1191\") " pod="openstack-kuttl-tests/neutron-57cc9dddbf-g8d9w" Dec 03 12:51:30 crc kubenswrapper[5052]: I1203 12:51:30.974904 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/0fc6a196-8a8e-4daf-9bbe-48d28e2e1191-config\") pod \"neutron-57cc9dddbf-g8d9w\" (UID: \"0fc6a196-8a8e-4daf-9bbe-48d28e2e1191\") " pod="openstack-kuttl-tests/neutron-57cc9dddbf-g8d9w" Dec 03 12:51:30 crc kubenswrapper[5052]: I1203 12:51:30.975074 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/0fc6a196-8a8e-4daf-9bbe-48d28e2e1191-httpd-config\") pod \"neutron-57cc9dddbf-g8d9w\" (UID: \"0fc6a196-8a8e-4daf-9bbe-48d28e2e1191\") " pod="openstack-kuttl-tests/neutron-57cc9dddbf-g8d9w" Dec 03 12:51:30 crc kubenswrapper[5052]: I1203 12:51:30.975121 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-6qlpv\" (UniqueName: \"kubernetes.io/projected/0fc6a196-8a8e-4daf-9bbe-48d28e2e1191-kube-api-access-6qlpv\") pod \"neutron-57cc9dddbf-g8d9w\" (UID: \"0fc6a196-8a8e-4daf-9bbe-48d28e2e1191\") " pod="openstack-kuttl-tests/neutron-57cc9dddbf-g8d9w" Dec 03 12:51:30 crc kubenswrapper[5052]: I1203 12:51:30.982288 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/0fc6a196-8a8e-4daf-9bbe-48d28e2e1191-httpd-config\") pod \"neutron-57cc9dddbf-g8d9w\" (UID: \"0fc6a196-8a8e-4daf-9bbe-48d28e2e1191\") " pod="openstack-kuttl-tests/neutron-57cc9dddbf-g8d9w" Dec 03 12:51:30 crc kubenswrapper[5052]: I1203 12:51:30.985380 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/0fc6a196-8a8e-4daf-9bbe-48d28e2e1191-ovndb-tls-certs\") pod \"neutron-57cc9dddbf-g8d9w\" (UID: \"0fc6a196-8a8e-4daf-9bbe-48d28e2e1191\") " pod="openstack-kuttl-tests/neutron-57cc9dddbf-g8d9w" Dec 03 12:51:30 crc kubenswrapper[5052]: I1203 12:51:30.992067 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-6qlpv\" (UniqueName: \"kubernetes.io/projected/0fc6a196-8a8e-4daf-9bbe-48d28e2e1191-kube-api-access-6qlpv\") pod \"neutron-57cc9dddbf-g8d9w\" (UID: \"0fc6a196-8a8e-4daf-9bbe-48d28e2e1191\") " pod="openstack-kuttl-tests/neutron-57cc9dddbf-g8d9w" Dec 03 12:51:30 crc kubenswrapper[5052]: I1203 12:51:30.995375 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/0fc6a196-8a8e-4daf-9bbe-48d28e2e1191-config\") pod \"neutron-57cc9dddbf-g8d9w\" (UID: \"0fc6a196-8a8e-4daf-9bbe-48d28e2e1191\") " pod="openstack-kuttl-tests/neutron-57cc9dddbf-g8d9w" Dec 03 12:51:31 crc kubenswrapper[5052]: I1203 12:51:31.001628 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0fc6a196-8a8e-4daf-9bbe-48d28e2e1191-combined-ca-bundle\") pod \"neutron-57cc9dddbf-g8d9w\" (UID: \"0fc6a196-8a8e-4daf-9bbe-48d28e2e1191\") " pod="openstack-kuttl-tests/neutron-57cc9dddbf-g8d9w" Dec 03 12:51:31 crc kubenswrapper[5052]: I1203 12:51:31.040542 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-57cc9dddbf-g8d9w" Dec 03 12:51:31 crc kubenswrapper[5052]: I1203 12:51:31.132237 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:51:31 crc kubenswrapper[5052]: I1203 12:51:31.202811 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 12:51:31 crc kubenswrapper[5052]: I1203 12:51:31.273156 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/barbican-api-76dc68d888-zmmcs" Dec 03 12:51:31 crc kubenswrapper[5052]: I1203 12:51:31.274658 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/barbican-api-76dc68d888-zmmcs" Dec 03 12:51:31 crc kubenswrapper[5052]: I1203 12:51:31.357277 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-api-5bdcd5dc6-vqlkq"] Dec 03 12:51:31 crc kubenswrapper[5052]: I1203 12:51:31.357500 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/barbican-api-5bdcd5dc6-vqlkq" podUID="539d9404-7518-4f03-815d-568a43eedb0b" containerName="barbican-api-log" containerID="cri-o://f30c8553eba37c12be3afb2daedc6c8c9d49487bdc18a5edc36f729b1ad6083f" gracePeriod=30 Dec 03 12:51:31 crc kubenswrapper[5052]: I1203 12:51:31.357672 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/barbican-api-5bdcd5dc6-vqlkq" podUID="539d9404-7518-4f03-815d-568a43eedb0b" containerName="barbican-api" containerID="cri-o://5e8481f4e60d3362b7d1a8fe46676fcecfc4e5f3f8c24cf300a5d3c419311f59" gracePeriod=30 Dec 03 12:51:31 crc kubenswrapper[5052]: I1203 12:51:31.518995 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-57cc9dddbf-g8d9w"] Dec 03 12:51:31 crc kubenswrapper[5052]: W1203 12:51:31.523877 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0fc6a196_8a8e_4daf_9bbe_48d28e2e1191.slice/crio-3327c71b7d5574d8c41d7448686bddf428aeb8a282cb62195991720398ebc15b WatchSource:0}: Error finding container 3327c71b7d5574d8c41d7448686bddf428aeb8a282cb62195991720398ebc15b: Status 404 returned error can't find the container with id 3327c71b7d5574d8c41d7448686bddf428aeb8a282cb62195991720398ebc15b Dec 03 12:51:31 crc kubenswrapper[5052]: I1203 12:51:31.596194 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-57cc9dddbf-g8d9w" event={"ID":"0fc6a196-8a8e-4daf-9bbe-48d28e2e1191","Type":"ContainerStarted","Data":"3327c71b7d5574d8c41d7448686bddf428aeb8a282cb62195991720398ebc15b"} Dec 03 12:51:31 crc kubenswrapper[5052]: I1203 12:51:31.598756 5052 generic.go:334] "Generic (PLEG): container finished" podID="539d9404-7518-4f03-815d-568a43eedb0b" containerID="f30c8553eba37c12be3afb2daedc6c8c9d49487bdc18a5edc36f729b1ad6083f" exitCode=143 Dec 03 12:51:31 crc kubenswrapper[5052]: I1203 12:51:31.598835 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-5bdcd5dc6-vqlkq" event={"ID":"539d9404-7518-4f03-815d-568a43eedb0b","Type":"ContainerDied","Data":"f30c8553eba37c12be3afb2daedc6c8c9d49487bdc18a5edc36f729b1ad6083f"} Dec 03 12:51:31 crc kubenswrapper[5052]: I1203 12:51:31.599287 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/cinder-scheduler-0" podUID="ee74f2f2-174b-4a8c-9fac-812be78ad1a8" containerName="cinder-scheduler" containerID="cri-o://ca86cc05c83430edadfad14e0307d0069ae23ed1a778ca9dc8d0eaad71818ac1" gracePeriod=30 Dec 03 12:51:31 crc kubenswrapper[5052]: I1203 12:51:31.599302 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/cinder-scheduler-0" podUID="ee74f2f2-174b-4a8c-9fac-812be78ad1a8" containerName="probe" containerID="cri-o://ffe8aa10ffea0f8a58b990b6040c05d187bb5616dea3df67a9e84c15341fba1a" gracePeriod=30 Dec 03 12:51:32 crc kubenswrapper[5052]: I1203 12:51:32.608653 5052 generic.go:334] "Generic (PLEG): container finished" podID="ee74f2f2-174b-4a8c-9fac-812be78ad1a8" containerID="ffe8aa10ffea0f8a58b990b6040c05d187bb5616dea3df67a9e84c15341fba1a" exitCode=0 Dec 03 12:51:32 crc kubenswrapper[5052]: I1203 12:51:32.608741 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"ee74f2f2-174b-4a8c-9fac-812be78ad1a8","Type":"ContainerDied","Data":"ffe8aa10ffea0f8a58b990b6040c05d187bb5616dea3df67a9e84c15341fba1a"} Dec 03 12:51:32 crc kubenswrapper[5052]: I1203 12:51:32.610835 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-57cc9dddbf-g8d9w" event={"ID":"0fc6a196-8a8e-4daf-9bbe-48d28e2e1191","Type":"ContainerStarted","Data":"a546a8af3f11336976a636365b701a3471736636285673c4dd7e4ce181388249"} Dec 03 12:51:32 crc kubenswrapper[5052]: I1203 12:51:32.610872 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-57cc9dddbf-g8d9w" event={"ID":"0fc6a196-8a8e-4daf-9bbe-48d28e2e1191","Type":"ContainerStarted","Data":"5d7518740b9f063814d21fa5094564958d8f7a70cc9e607d2a89ceff536afeb6"} Dec 03 12:51:32 crc kubenswrapper[5052]: I1203 12:51:32.611973 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/neutron-57cc9dddbf-g8d9w" Dec 03 12:51:32 crc kubenswrapper[5052]: I1203 12:51:32.640996 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/neutron-57cc9dddbf-g8d9w" podStartSLOduration=2.64097364 podStartE2EDuration="2.64097364s" podCreationTimestamp="2025-12-03 12:51:30 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:51:32.630967224 +0000 UTC m=+5483.219208988" watchObservedRunningTime="2025-12-03 12:51:32.64097364 +0000 UTC m=+5483.229215404" Dec 03 12:51:33 crc kubenswrapper[5052]: I1203 12:51:33.645204 5052 generic.go:334] "Generic (PLEG): container finished" podID="ee74f2f2-174b-4a8c-9fac-812be78ad1a8" containerID="ca86cc05c83430edadfad14e0307d0069ae23ed1a778ca9dc8d0eaad71818ac1" exitCode=0 Dec 03 12:51:33 crc kubenswrapper[5052]: I1203 12:51:33.645299 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"ee74f2f2-174b-4a8c-9fac-812be78ad1a8","Type":"ContainerDied","Data":"ca86cc05c83430edadfad14e0307d0069ae23ed1a778ca9dc8d0eaad71818ac1"} Dec 03 12:51:33 crc kubenswrapper[5052]: I1203 12:51:33.846496 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:51:33 crc kubenswrapper[5052]: I1203 12:51:33.927084 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/ee74f2f2-174b-4a8c-9fac-812be78ad1a8-scripts\") pod \"ee74f2f2-174b-4a8c-9fac-812be78ad1a8\" (UID: \"ee74f2f2-174b-4a8c-9fac-812be78ad1a8\") " Dec 03 12:51:33 crc kubenswrapper[5052]: I1203 12:51:33.927139 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ee74f2f2-174b-4a8c-9fac-812be78ad1a8-combined-ca-bundle\") pod \"ee74f2f2-174b-4a8c-9fac-812be78ad1a8\" (UID: \"ee74f2f2-174b-4a8c-9fac-812be78ad1a8\") " Dec 03 12:51:33 crc kubenswrapper[5052]: I1203 12:51:33.927227 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/ee74f2f2-174b-4a8c-9fac-812be78ad1a8-config-data-custom\") pod \"ee74f2f2-174b-4a8c-9fac-812be78ad1a8\" (UID: \"ee74f2f2-174b-4a8c-9fac-812be78ad1a8\") " Dec 03 12:51:33 crc kubenswrapper[5052]: I1203 12:51:33.927268 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qb97f\" (UniqueName: \"kubernetes.io/projected/ee74f2f2-174b-4a8c-9fac-812be78ad1a8-kube-api-access-qb97f\") pod \"ee74f2f2-174b-4a8c-9fac-812be78ad1a8\" (UID: \"ee74f2f2-174b-4a8c-9fac-812be78ad1a8\") " Dec 03 12:51:33 crc kubenswrapper[5052]: I1203 12:51:33.927290 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/ee74f2f2-174b-4a8c-9fac-812be78ad1a8-etc-machine-id\") pod \"ee74f2f2-174b-4a8c-9fac-812be78ad1a8\" (UID: \"ee74f2f2-174b-4a8c-9fac-812be78ad1a8\") " Dec 03 12:51:33 crc kubenswrapper[5052]: I1203 12:51:33.927310 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/ee74f2f2-174b-4a8c-9fac-812be78ad1a8-config-data\") pod \"ee74f2f2-174b-4a8c-9fac-812be78ad1a8\" (UID: \"ee74f2f2-174b-4a8c-9fac-812be78ad1a8\") " Dec 03 12:51:33 crc kubenswrapper[5052]: I1203 12:51:33.930429 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/ee74f2f2-174b-4a8c-9fac-812be78ad1a8-etc-machine-id" (OuterVolumeSpecName: "etc-machine-id") pod "ee74f2f2-174b-4a8c-9fac-812be78ad1a8" (UID: "ee74f2f2-174b-4a8c-9fac-812be78ad1a8"). InnerVolumeSpecName "etc-machine-id". PluginName "kubernetes.io/host-path", VolumeGidValue "" Dec 03 12:51:33 crc kubenswrapper[5052]: I1203 12:51:33.933476 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ee74f2f2-174b-4a8c-9fac-812be78ad1a8-scripts" (OuterVolumeSpecName: "scripts") pod "ee74f2f2-174b-4a8c-9fac-812be78ad1a8" (UID: "ee74f2f2-174b-4a8c-9fac-812be78ad1a8"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:51:33 crc kubenswrapper[5052]: I1203 12:51:33.933973 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ee74f2f2-174b-4a8c-9fac-812be78ad1a8-kube-api-access-qb97f" (OuterVolumeSpecName: "kube-api-access-qb97f") pod "ee74f2f2-174b-4a8c-9fac-812be78ad1a8" (UID: "ee74f2f2-174b-4a8c-9fac-812be78ad1a8"). InnerVolumeSpecName "kube-api-access-qb97f". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:51:33 crc kubenswrapper[5052]: I1203 12:51:33.934335 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ee74f2f2-174b-4a8c-9fac-812be78ad1a8-config-data-custom" (OuterVolumeSpecName: "config-data-custom") pod "ee74f2f2-174b-4a8c-9fac-812be78ad1a8" (UID: "ee74f2f2-174b-4a8c-9fac-812be78ad1a8"). InnerVolumeSpecName "config-data-custom". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:51:33 crc kubenswrapper[5052]: I1203 12:51:33.992770 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ee74f2f2-174b-4a8c-9fac-812be78ad1a8-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "ee74f2f2-174b-4a8c-9fac-812be78ad1a8" (UID: "ee74f2f2-174b-4a8c-9fac-812be78ad1a8"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:51:34 crc kubenswrapper[5052]: I1203 12:51:34.029727 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/ee74f2f2-174b-4a8c-9fac-812be78ad1a8-config-data-custom\") on node \"crc\" DevicePath \"\"" Dec 03 12:51:34 crc kubenswrapper[5052]: I1203 12:51:34.029764 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-qb97f\" (UniqueName: \"kubernetes.io/projected/ee74f2f2-174b-4a8c-9fac-812be78ad1a8-kube-api-access-qb97f\") on node \"crc\" DevicePath \"\"" Dec 03 12:51:34 crc kubenswrapper[5052]: I1203 12:51:34.029775 5052 reconciler_common.go:293] "Volume detached for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/ee74f2f2-174b-4a8c-9fac-812be78ad1a8-etc-machine-id\") on node \"crc\" DevicePath \"\"" Dec 03 12:51:34 crc kubenswrapper[5052]: I1203 12:51:34.029785 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/ee74f2f2-174b-4a8c-9fac-812be78ad1a8-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:51:34 crc kubenswrapper[5052]: I1203 12:51:34.029794 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ee74f2f2-174b-4a8c-9fac-812be78ad1a8-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:51:34 crc kubenswrapper[5052]: I1203 12:51:34.050147 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ee74f2f2-174b-4a8c-9fac-812be78ad1a8-config-data" (OuterVolumeSpecName: "config-data") pod "ee74f2f2-174b-4a8c-9fac-812be78ad1a8" (UID: "ee74f2f2-174b-4a8c-9fac-812be78ad1a8"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:51:34 crc kubenswrapper[5052]: I1203 12:51:34.132266 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/ee74f2f2-174b-4a8c-9fac-812be78ad1a8-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:51:34 crc kubenswrapper[5052]: I1203 12:51:34.389245 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/neutron-698d8977cd-ppz9n"] Dec 03 12:51:34 crc kubenswrapper[5052]: E1203 12:51:34.390233 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="ee74f2f2-174b-4a8c-9fac-812be78ad1a8" containerName="probe" Dec 03 12:51:34 crc kubenswrapper[5052]: I1203 12:51:34.390263 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee74f2f2-174b-4a8c-9fac-812be78ad1a8" containerName="probe" Dec 03 12:51:34 crc kubenswrapper[5052]: E1203 12:51:34.390286 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="ee74f2f2-174b-4a8c-9fac-812be78ad1a8" containerName="cinder-scheduler" Dec 03 12:51:34 crc kubenswrapper[5052]: I1203 12:51:34.390295 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee74f2f2-174b-4a8c-9fac-812be78ad1a8" containerName="cinder-scheduler" Dec 03 12:51:34 crc kubenswrapper[5052]: I1203 12:51:34.390528 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="ee74f2f2-174b-4a8c-9fac-812be78ad1a8" containerName="probe" Dec 03 12:51:34 crc kubenswrapper[5052]: I1203 12:51:34.390555 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="ee74f2f2-174b-4a8c-9fac-812be78ad1a8" containerName="cinder-scheduler" Dec 03 12:51:34 crc kubenswrapper[5052]: I1203 12:51:34.391777 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-698d8977cd-ppz9n" Dec 03 12:51:34 crc kubenswrapper[5052]: I1203 12:51:34.395151 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-neutron-public-svc" Dec 03 12:51:34 crc kubenswrapper[5052]: I1203 12:51:34.395393 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-neutron-internal-svc" Dec 03 12:51:34 crc kubenswrapper[5052]: I1203 12:51:34.400127 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-698d8977cd-ppz9n"] Dec 03 12:51:34 crc kubenswrapper[5052]: I1203 12:51:34.503886 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/barbican-api-5bdcd5dc6-vqlkq" podUID="539d9404-7518-4f03-815d-568a43eedb0b" containerName="barbican-api-log" probeResult="failure" output="Get \"http://10.217.0.180:9311/healthcheck\": read tcp 10.217.0.2:53626->10.217.0.180:9311: read: connection reset by peer" Dec 03 12:51:34 crc kubenswrapper[5052]: I1203 12:51:34.503911 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/barbican-api-5bdcd5dc6-vqlkq" podUID="539d9404-7518-4f03-815d-568a43eedb0b" containerName="barbican-api" probeResult="failure" output="Get \"http://10.217.0.180:9311/healthcheck\": read tcp 10.217.0.2:53622->10.217.0.180:9311: read: connection reset by peer" Dec 03 12:51:34 crc kubenswrapper[5052]: I1203 12:51:34.539681 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/e2225738-b0d1-4306-bbee-a61dd9558c80-ovndb-tls-certs\") pod \"neutron-698d8977cd-ppz9n\" (UID: \"e2225738-b0d1-4306-bbee-a61dd9558c80\") " pod="openstack-kuttl-tests/neutron-698d8977cd-ppz9n" Dec 03 12:51:34 crc kubenswrapper[5052]: I1203 12:51:34.539745 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/e2225738-b0d1-4306-bbee-a61dd9558c80-public-tls-certs\") pod \"neutron-698d8977cd-ppz9n\" (UID: \"e2225738-b0d1-4306-bbee-a61dd9558c80\") " pod="openstack-kuttl-tests/neutron-698d8977cd-ppz9n" Dec 03 12:51:34 crc kubenswrapper[5052]: I1203 12:51:34.539776 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/e2225738-b0d1-4306-bbee-a61dd9558c80-internal-tls-certs\") pod \"neutron-698d8977cd-ppz9n\" (UID: \"e2225738-b0d1-4306-bbee-a61dd9558c80\") " pod="openstack-kuttl-tests/neutron-698d8977cd-ppz9n" Dec 03 12:51:34 crc kubenswrapper[5052]: I1203 12:51:34.539809 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wj7x6\" (UniqueName: \"kubernetes.io/projected/e2225738-b0d1-4306-bbee-a61dd9558c80-kube-api-access-wj7x6\") pod \"neutron-698d8977cd-ppz9n\" (UID: \"e2225738-b0d1-4306-bbee-a61dd9558c80\") " pod="openstack-kuttl-tests/neutron-698d8977cd-ppz9n" Dec 03 12:51:34 crc kubenswrapper[5052]: I1203 12:51:34.539877 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e2225738-b0d1-4306-bbee-a61dd9558c80-combined-ca-bundle\") pod \"neutron-698d8977cd-ppz9n\" (UID: \"e2225738-b0d1-4306-bbee-a61dd9558c80\") " pod="openstack-kuttl-tests/neutron-698d8977cd-ppz9n" Dec 03 12:51:34 crc kubenswrapper[5052]: I1203 12:51:34.539911 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/e2225738-b0d1-4306-bbee-a61dd9558c80-config\") pod \"neutron-698d8977cd-ppz9n\" (UID: \"e2225738-b0d1-4306-bbee-a61dd9558c80\") " pod="openstack-kuttl-tests/neutron-698d8977cd-ppz9n" Dec 03 12:51:34 crc kubenswrapper[5052]: I1203 12:51:34.540022 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/e2225738-b0d1-4306-bbee-a61dd9558c80-httpd-config\") pod \"neutron-698d8977cd-ppz9n\" (UID: \"e2225738-b0d1-4306-bbee-a61dd9558c80\") " pod="openstack-kuttl-tests/neutron-698d8977cd-ppz9n" Dec 03 12:51:34 crc kubenswrapper[5052]: I1203 12:51:34.641058 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e2225738-b0d1-4306-bbee-a61dd9558c80-combined-ca-bundle\") pod \"neutron-698d8977cd-ppz9n\" (UID: \"e2225738-b0d1-4306-bbee-a61dd9558c80\") " pod="openstack-kuttl-tests/neutron-698d8977cd-ppz9n" Dec 03 12:51:34 crc kubenswrapper[5052]: I1203 12:51:34.641329 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/e2225738-b0d1-4306-bbee-a61dd9558c80-config\") pod \"neutron-698d8977cd-ppz9n\" (UID: \"e2225738-b0d1-4306-bbee-a61dd9558c80\") " pod="openstack-kuttl-tests/neutron-698d8977cd-ppz9n" Dec 03 12:51:34 crc kubenswrapper[5052]: I1203 12:51:34.641433 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/e2225738-b0d1-4306-bbee-a61dd9558c80-httpd-config\") pod \"neutron-698d8977cd-ppz9n\" (UID: \"e2225738-b0d1-4306-bbee-a61dd9558c80\") " pod="openstack-kuttl-tests/neutron-698d8977cd-ppz9n" Dec 03 12:51:34 crc kubenswrapper[5052]: I1203 12:51:34.641471 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/e2225738-b0d1-4306-bbee-a61dd9558c80-ovndb-tls-certs\") pod \"neutron-698d8977cd-ppz9n\" (UID: \"e2225738-b0d1-4306-bbee-a61dd9558c80\") " pod="openstack-kuttl-tests/neutron-698d8977cd-ppz9n" Dec 03 12:51:34 crc kubenswrapper[5052]: I1203 12:51:34.641501 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/e2225738-b0d1-4306-bbee-a61dd9558c80-public-tls-certs\") pod \"neutron-698d8977cd-ppz9n\" (UID: \"e2225738-b0d1-4306-bbee-a61dd9558c80\") " pod="openstack-kuttl-tests/neutron-698d8977cd-ppz9n" Dec 03 12:51:34 crc kubenswrapper[5052]: I1203 12:51:34.641552 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/e2225738-b0d1-4306-bbee-a61dd9558c80-internal-tls-certs\") pod \"neutron-698d8977cd-ppz9n\" (UID: \"e2225738-b0d1-4306-bbee-a61dd9558c80\") " pod="openstack-kuttl-tests/neutron-698d8977cd-ppz9n" Dec 03 12:51:34 crc kubenswrapper[5052]: I1203 12:51:34.641583 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-wj7x6\" (UniqueName: \"kubernetes.io/projected/e2225738-b0d1-4306-bbee-a61dd9558c80-kube-api-access-wj7x6\") pod \"neutron-698d8977cd-ppz9n\" (UID: \"e2225738-b0d1-4306-bbee-a61dd9558c80\") " pod="openstack-kuttl-tests/neutron-698d8977cd-ppz9n" Dec 03 12:51:34 crc kubenswrapper[5052]: I1203 12:51:34.646406 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/e2225738-b0d1-4306-bbee-a61dd9558c80-config\") pod \"neutron-698d8977cd-ppz9n\" (UID: \"e2225738-b0d1-4306-bbee-a61dd9558c80\") " pod="openstack-kuttl-tests/neutron-698d8977cd-ppz9n" Dec 03 12:51:34 crc kubenswrapper[5052]: I1203 12:51:34.646549 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/e2225738-b0d1-4306-bbee-a61dd9558c80-public-tls-certs\") pod \"neutron-698d8977cd-ppz9n\" (UID: \"e2225738-b0d1-4306-bbee-a61dd9558c80\") " pod="openstack-kuttl-tests/neutron-698d8977cd-ppz9n" Dec 03 12:51:34 crc kubenswrapper[5052]: I1203 12:51:34.648296 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/e2225738-b0d1-4306-bbee-a61dd9558c80-ovndb-tls-certs\") pod \"neutron-698d8977cd-ppz9n\" (UID: \"e2225738-b0d1-4306-bbee-a61dd9558c80\") " pod="openstack-kuttl-tests/neutron-698d8977cd-ppz9n" Dec 03 12:51:34 crc kubenswrapper[5052]: I1203 12:51:34.648394 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/e2225738-b0d1-4306-bbee-a61dd9558c80-internal-tls-certs\") pod \"neutron-698d8977cd-ppz9n\" (UID: \"e2225738-b0d1-4306-bbee-a61dd9558c80\") " pod="openstack-kuttl-tests/neutron-698d8977cd-ppz9n" Dec 03 12:51:34 crc kubenswrapper[5052]: I1203 12:51:34.648713 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e2225738-b0d1-4306-bbee-a61dd9558c80-combined-ca-bundle\") pod \"neutron-698d8977cd-ppz9n\" (UID: \"e2225738-b0d1-4306-bbee-a61dd9558c80\") " pod="openstack-kuttl-tests/neutron-698d8977cd-ppz9n" Dec 03 12:51:34 crc kubenswrapper[5052]: I1203 12:51:34.665539 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/e2225738-b0d1-4306-bbee-a61dd9558c80-httpd-config\") pod \"neutron-698d8977cd-ppz9n\" (UID: \"e2225738-b0d1-4306-bbee-a61dd9558c80\") " pod="openstack-kuttl-tests/neutron-698d8977cd-ppz9n" Dec 03 12:51:34 crc kubenswrapper[5052]: I1203 12:51:34.667958 5052 generic.go:334] "Generic (PLEG): container finished" podID="539d9404-7518-4f03-815d-568a43eedb0b" containerID="5e8481f4e60d3362b7d1a8fe46676fcecfc4e5f3f8c24cf300a5d3c419311f59" exitCode=0 Dec 03 12:51:34 crc kubenswrapper[5052]: I1203 12:51:34.668048 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-5bdcd5dc6-vqlkq" event={"ID":"539d9404-7518-4f03-815d-568a43eedb0b","Type":"ContainerDied","Data":"5e8481f4e60d3362b7d1a8fe46676fcecfc4e5f3f8c24cf300a5d3c419311f59"} Dec 03 12:51:34 crc kubenswrapper[5052]: I1203 12:51:34.669036 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-wj7x6\" (UniqueName: \"kubernetes.io/projected/e2225738-b0d1-4306-bbee-a61dd9558c80-kube-api-access-wj7x6\") pod \"neutron-698d8977cd-ppz9n\" (UID: \"e2225738-b0d1-4306-bbee-a61dd9558c80\") " pod="openstack-kuttl-tests/neutron-698d8977cd-ppz9n" Dec 03 12:51:34 crc kubenswrapper[5052]: I1203 12:51:34.675296 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:51:34 crc kubenswrapper[5052]: I1203 12:51:34.675340 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"ee74f2f2-174b-4a8c-9fac-812be78ad1a8","Type":"ContainerDied","Data":"0342943988af94769a8ba341d5ff2e919423c64f7796d1453d4fc55c187d2831"} Dec 03 12:51:34 crc kubenswrapper[5052]: I1203 12:51:34.675370 5052 scope.go:117] "RemoveContainer" containerID="ffe8aa10ffea0f8a58b990b6040c05d187bb5616dea3df67a9e84c15341fba1a" Dec 03 12:51:34 crc kubenswrapper[5052]: I1203 12:51:34.712929 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-698d8977cd-ppz9n" Dec 03 12:51:34 crc kubenswrapper[5052]: I1203 12:51:34.753955 5052 scope.go:117] "RemoveContainer" containerID="ca86cc05c83430edadfad14e0307d0069ae23ed1a778ca9dc8d0eaad71818ac1" Dec 03 12:51:34 crc kubenswrapper[5052]: I1203 12:51:34.805352 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 12:51:34 crc kubenswrapper[5052]: I1203 12:51:34.840729 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 12:51:34 crc kubenswrapper[5052]: I1203 12:51:34.855335 5052 scope.go:117] "RemoveContainer" containerID="722c7be4df16aafcde67740e1e015e377d9008419180cd45892921e7e51a19ef" Dec 03 12:51:34 crc kubenswrapper[5052]: I1203 12:51:34.870158 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 12:51:34 crc kubenswrapper[5052]: I1203 12:51:34.872742 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:51:34 crc kubenswrapper[5052]: I1203 12:51:34.875206 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cinder-scheduler-config-data" Dec 03 12:51:34 crc kubenswrapper[5052]: I1203 12:51:34.893591 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 12:51:34 crc kubenswrapper[5052]: I1203 12:51:34.905024 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-api-5bdcd5dc6-vqlkq" Dec 03 12:51:34 crc kubenswrapper[5052]: I1203 12:51:34.917979 5052 scope.go:117] "RemoveContainer" containerID="67e80f3b2185e807d12e6cb73fcdb3e6261d02a42c22e32ff2b8b1ff0732f99b" Dec 03 12:51:34 crc kubenswrapper[5052]: I1203 12:51:34.949832 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/2b0bf9ae-2d1d-48da-8c4b-2973b47985a3-config-data\") pod \"cinder-scheduler-0\" (UID: \"2b0bf9ae-2d1d-48da-8c4b-2973b47985a3\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:51:34 crc kubenswrapper[5052]: I1203 12:51:34.949901 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ql9sg\" (UniqueName: \"kubernetes.io/projected/2b0bf9ae-2d1d-48da-8c4b-2973b47985a3-kube-api-access-ql9sg\") pod \"cinder-scheduler-0\" (UID: \"2b0bf9ae-2d1d-48da-8c4b-2973b47985a3\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:51:34 crc kubenswrapper[5052]: I1203 12:51:34.949970 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/2b0bf9ae-2d1d-48da-8c4b-2973b47985a3-config-data-custom\") pod \"cinder-scheduler-0\" (UID: \"2b0bf9ae-2d1d-48da-8c4b-2973b47985a3\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:51:34 crc kubenswrapper[5052]: I1203 12:51:34.950040 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2b0bf9ae-2d1d-48da-8c4b-2973b47985a3-combined-ca-bundle\") pod \"cinder-scheduler-0\" (UID: \"2b0bf9ae-2d1d-48da-8c4b-2973b47985a3\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:51:34 crc kubenswrapper[5052]: I1203 12:51:34.950091 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/2b0bf9ae-2d1d-48da-8c4b-2973b47985a3-scripts\") pod \"cinder-scheduler-0\" (UID: \"2b0bf9ae-2d1d-48da-8c4b-2973b47985a3\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:51:34 crc kubenswrapper[5052]: I1203 12:51:34.950147 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/2b0bf9ae-2d1d-48da-8c4b-2973b47985a3-etc-machine-id\") pod \"cinder-scheduler-0\" (UID: \"2b0bf9ae-2d1d-48da-8c4b-2973b47985a3\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:51:34 crc kubenswrapper[5052]: I1203 12:51:34.969974 5052 scope.go:117] "RemoveContainer" containerID="bd4fb5170e931a743cac2a0ebfb0fe722c35f2535a615030fafe0f1d840d184e" Dec 03 12:51:34 crc kubenswrapper[5052]: I1203 12:51:34.997582 5052 scope.go:117] "RemoveContainer" containerID="3d1c3119a6baffe670b9369900b68bd9fa52d242fa306af736795fb6724df163" Dec 03 12:51:35 crc kubenswrapper[5052]: I1203 12:51:35.021546 5052 scope.go:117] "RemoveContainer" containerID="0746aff23d59af40796b6fe2a1bbd7fdff16310788905f24c7ce1f98b626c483" Dec 03 12:51:35 crc kubenswrapper[5052]: I1203 12:51:35.051096 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/539d9404-7518-4f03-815d-568a43eedb0b-config-data\") pod \"539d9404-7518-4f03-815d-568a43eedb0b\" (UID: \"539d9404-7518-4f03-815d-568a43eedb0b\") " Dec 03 12:51:35 crc kubenswrapper[5052]: I1203 12:51:35.051255 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/539d9404-7518-4f03-815d-568a43eedb0b-config-data-custom\") pod \"539d9404-7518-4f03-815d-568a43eedb0b\" (UID: \"539d9404-7518-4f03-815d-568a43eedb0b\") " Dec 03 12:51:35 crc kubenswrapper[5052]: I1203 12:51:35.051278 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/539d9404-7518-4f03-815d-568a43eedb0b-combined-ca-bundle\") pod \"539d9404-7518-4f03-815d-568a43eedb0b\" (UID: \"539d9404-7518-4f03-815d-568a43eedb0b\") " Dec 03 12:51:35 crc kubenswrapper[5052]: I1203 12:51:35.051307 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kw6x2\" (UniqueName: \"kubernetes.io/projected/539d9404-7518-4f03-815d-568a43eedb0b-kube-api-access-kw6x2\") pod \"539d9404-7518-4f03-815d-568a43eedb0b\" (UID: \"539d9404-7518-4f03-815d-568a43eedb0b\") " Dec 03 12:51:35 crc kubenswrapper[5052]: I1203 12:51:35.051331 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/539d9404-7518-4f03-815d-568a43eedb0b-logs\") pod \"539d9404-7518-4f03-815d-568a43eedb0b\" (UID: \"539d9404-7518-4f03-815d-568a43eedb0b\") " Dec 03 12:51:35 crc kubenswrapper[5052]: I1203 12:51:35.051579 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2b0bf9ae-2d1d-48da-8c4b-2973b47985a3-combined-ca-bundle\") pod \"cinder-scheduler-0\" (UID: \"2b0bf9ae-2d1d-48da-8c4b-2973b47985a3\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:51:35 crc kubenswrapper[5052]: I1203 12:51:35.051628 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/2b0bf9ae-2d1d-48da-8c4b-2973b47985a3-scripts\") pod \"cinder-scheduler-0\" (UID: \"2b0bf9ae-2d1d-48da-8c4b-2973b47985a3\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:51:35 crc kubenswrapper[5052]: I1203 12:51:35.051670 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/2b0bf9ae-2d1d-48da-8c4b-2973b47985a3-etc-machine-id\") pod \"cinder-scheduler-0\" (UID: \"2b0bf9ae-2d1d-48da-8c4b-2973b47985a3\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:51:35 crc kubenswrapper[5052]: I1203 12:51:35.051752 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/2b0bf9ae-2d1d-48da-8c4b-2973b47985a3-config-data\") pod \"cinder-scheduler-0\" (UID: \"2b0bf9ae-2d1d-48da-8c4b-2973b47985a3\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:51:35 crc kubenswrapper[5052]: I1203 12:51:35.051770 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-ql9sg\" (UniqueName: \"kubernetes.io/projected/2b0bf9ae-2d1d-48da-8c4b-2973b47985a3-kube-api-access-ql9sg\") pod \"cinder-scheduler-0\" (UID: \"2b0bf9ae-2d1d-48da-8c4b-2973b47985a3\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:51:35 crc kubenswrapper[5052]: I1203 12:51:35.051809 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/2b0bf9ae-2d1d-48da-8c4b-2973b47985a3-config-data-custom\") pod \"cinder-scheduler-0\" (UID: \"2b0bf9ae-2d1d-48da-8c4b-2973b47985a3\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:51:35 crc kubenswrapper[5052]: I1203 12:51:35.052957 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/539d9404-7518-4f03-815d-568a43eedb0b-logs" (OuterVolumeSpecName: "logs") pod "539d9404-7518-4f03-815d-568a43eedb0b" (UID: "539d9404-7518-4f03-815d-568a43eedb0b"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:51:35 crc kubenswrapper[5052]: I1203 12:51:35.057089 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/2b0bf9ae-2d1d-48da-8c4b-2973b47985a3-etc-machine-id\") pod \"cinder-scheduler-0\" (UID: \"2b0bf9ae-2d1d-48da-8c4b-2973b47985a3\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:51:35 crc kubenswrapper[5052]: I1203 12:51:35.059313 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2b0bf9ae-2d1d-48da-8c4b-2973b47985a3-combined-ca-bundle\") pod \"cinder-scheduler-0\" (UID: \"2b0bf9ae-2d1d-48da-8c4b-2973b47985a3\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:51:35 crc kubenswrapper[5052]: I1203 12:51:35.061677 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/539d9404-7518-4f03-815d-568a43eedb0b-kube-api-access-kw6x2" (OuterVolumeSpecName: "kube-api-access-kw6x2") pod "539d9404-7518-4f03-815d-568a43eedb0b" (UID: "539d9404-7518-4f03-815d-568a43eedb0b"). InnerVolumeSpecName "kube-api-access-kw6x2". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:51:35 crc kubenswrapper[5052]: I1203 12:51:35.062149 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/2b0bf9ae-2d1d-48da-8c4b-2973b47985a3-config-data-custom\") pod \"cinder-scheduler-0\" (UID: \"2b0bf9ae-2d1d-48da-8c4b-2973b47985a3\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:51:35 crc kubenswrapper[5052]: I1203 12:51:35.062376 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/539d9404-7518-4f03-815d-568a43eedb0b-config-data-custom" (OuterVolumeSpecName: "config-data-custom") pod "539d9404-7518-4f03-815d-568a43eedb0b" (UID: "539d9404-7518-4f03-815d-568a43eedb0b"). InnerVolumeSpecName "config-data-custom". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:51:35 crc kubenswrapper[5052]: I1203 12:51:35.064909 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/2b0bf9ae-2d1d-48da-8c4b-2973b47985a3-scripts\") pod \"cinder-scheduler-0\" (UID: \"2b0bf9ae-2d1d-48da-8c4b-2973b47985a3\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:51:35 crc kubenswrapper[5052]: I1203 12:51:35.070488 5052 scope.go:117] "RemoveContainer" containerID="a9b87dccd0e7dcc3aa151cb7ead35d696cd2b5970c1fdb43b6dea2d3e0aee83d" Dec 03 12:51:35 crc kubenswrapper[5052]: I1203 12:51:35.071896 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/2b0bf9ae-2d1d-48da-8c4b-2973b47985a3-config-data\") pod \"cinder-scheduler-0\" (UID: \"2b0bf9ae-2d1d-48da-8c4b-2973b47985a3\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:51:35 crc kubenswrapper[5052]: I1203 12:51:35.076114 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-ql9sg\" (UniqueName: \"kubernetes.io/projected/2b0bf9ae-2d1d-48da-8c4b-2973b47985a3-kube-api-access-ql9sg\") pod \"cinder-scheduler-0\" (UID: \"2b0bf9ae-2d1d-48da-8c4b-2973b47985a3\") " pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:51:35 crc kubenswrapper[5052]: I1203 12:51:35.100231 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/539d9404-7518-4f03-815d-568a43eedb0b-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "539d9404-7518-4f03-815d-568a43eedb0b" (UID: "539d9404-7518-4f03-815d-568a43eedb0b"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:51:35 crc kubenswrapper[5052]: I1203 12:51:35.105382 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/539d9404-7518-4f03-815d-568a43eedb0b-config-data" (OuterVolumeSpecName: "config-data") pod "539d9404-7518-4f03-815d-568a43eedb0b" (UID: "539d9404-7518-4f03-815d-568a43eedb0b"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:51:35 crc kubenswrapper[5052]: I1203 12:51:35.118911 5052 scope.go:117] "RemoveContainer" containerID="2027079a1fe5e803e78ab8612d4ce2d26d54f6fbbe2c40d7cccb7ba4a5c1f6f0" Dec 03 12:51:35 crc kubenswrapper[5052]: I1203 12:51:35.153320 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/539d9404-7518-4f03-815d-568a43eedb0b-config-data-custom\") on node \"crc\" DevicePath \"\"" Dec 03 12:51:35 crc kubenswrapper[5052]: I1203 12:51:35.153353 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/539d9404-7518-4f03-815d-568a43eedb0b-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:51:35 crc kubenswrapper[5052]: I1203 12:51:35.153365 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-kw6x2\" (UniqueName: \"kubernetes.io/projected/539d9404-7518-4f03-815d-568a43eedb0b-kube-api-access-kw6x2\") on node \"crc\" DevicePath \"\"" Dec 03 12:51:35 crc kubenswrapper[5052]: I1203 12:51:35.153376 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/539d9404-7518-4f03-815d-568a43eedb0b-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:51:35 crc kubenswrapper[5052]: I1203 12:51:35.153385 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/539d9404-7518-4f03-815d-568a43eedb0b-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:51:35 crc kubenswrapper[5052]: I1203 12:51:35.201948 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:51:35 crc kubenswrapper[5052]: I1203 12:51:35.250021 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-698d8977cd-ppz9n"] Dec 03 12:51:35 crc kubenswrapper[5052]: I1203 12:51:35.685929 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ee74f2f2-174b-4a8c-9fac-812be78ad1a8" path="/var/lib/kubelet/pods/ee74f2f2-174b-4a8c-9fac-812be78ad1a8/volumes" Dec 03 12:51:35 crc kubenswrapper[5052]: I1203 12:51:35.688418 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-698d8977cd-ppz9n" event={"ID":"e2225738-b0d1-4306-bbee-a61dd9558c80","Type":"ContainerStarted","Data":"b7a1da6db38db2f0ed2c6f73c89998af0cc42716d17e3513590eabdf3cc5aa71"} Dec 03 12:51:35 crc kubenswrapper[5052]: I1203 12:51:35.688473 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-698d8977cd-ppz9n" event={"ID":"e2225738-b0d1-4306-bbee-a61dd9558c80","Type":"ContainerStarted","Data":"1f4b0685f379f72ab1ff145175ef6378f1e0a0c2caf829598faeca90aa64dadf"} Dec 03 12:51:35 crc kubenswrapper[5052]: I1203 12:51:35.690149 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-5bdcd5dc6-vqlkq" event={"ID":"539d9404-7518-4f03-815d-568a43eedb0b","Type":"ContainerDied","Data":"dcaa6ec2052c2553cd436993c1572efa22c63b65ebb45e135ea74e0effccb612"} Dec 03 12:51:35 crc kubenswrapper[5052]: I1203 12:51:35.690184 5052 scope.go:117] "RemoveContainer" containerID="5e8481f4e60d3362b7d1a8fe46676fcecfc4e5f3f8c24cf300a5d3c419311f59" Dec 03 12:51:35 crc kubenswrapper[5052]: I1203 12:51:35.690332 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-api-5bdcd5dc6-vqlkq" Dec 03 12:51:35 crc kubenswrapper[5052]: I1203 12:51:35.691159 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 12:51:35 crc kubenswrapper[5052]: W1203 12:51:35.695793 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2b0bf9ae_2d1d_48da_8c4b_2973b47985a3.slice/crio-c6e9ea130ef6779b2c89f649e9b2cc204bd62f4ce28406054fb1e75ea1417b4a WatchSource:0}: Error finding container c6e9ea130ef6779b2c89f649e9b2cc204bd62f4ce28406054fb1e75ea1417b4a: Status 404 returned error can't find the container with id c6e9ea130ef6779b2c89f649e9b2cc204bd62f4ce28406054fb1e75ea1417b4a Dec 03 12:51:35 crc kubenswrapper[5052]: I1203 12:51:35.766057 5052 scope.go:117] "RemoveContainer" containerID="f30c8553eba37c12be3afb2daedc6c8c9d49487bdc18a5edc36f729b1ad6083f" Dec 03 12:51:36 crc kubenswrapper[5052]: I1203 12:51:36.700096 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"2b0bf9ae-2d1d-48da-8c4b-2973b47985a3","Type":"ContainerStarted","Data":"34ec2e3b276084c3a6169386d90460245b79be1c0922971153b1615fd7e2d686"} Dec 03 12:51:36 crc kubenswrapper[5052]: I1203 12:51:36.700473 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"2b0bf9ae-2d1d-48da-8c4b-2973b47985a3","Type":"ContainerStarted","Data":"c6e9ea130ef6779b2c89f649e9b2cc204bd62f4ce28406054fb1e75ea1417b4a"} Dec 03 12:51:36 crc kubenswrapper[5052]: I1203 12:51:36.709789 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-698d8977cd-ppz9n" event={"ID":"e2225738-b0d1-4306-bbee-a61dd9558c80","Type":"ContainerStarted","Data":"89798d3d34dfd5e2ea4266a0c6cf4bb6c1e2078d3038566fe9362ddc07ef9734"} Dec 03 12:51:36 crc kubenswrapper[5052]: I1203 12:51:36.709853 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/neutron-698d8977cd-ppz9n" Dec 03 12:51:36 crc kubenswrapper[5052]: I1203 12:51:36.732730 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/neutron-698d8977cd-ppz9n" podStartSLOduration=2.732706846 podStartE2EDuration="2.732706846s" podCreationTimestamp="2025-12-03 12:51:34 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:51:36.727466256 +0000 UTC m=+5487.315708020" watchObservedRunningTime="2025-12-03 12:51:36.732706846 +0000 UTC m=+5487.320948620" Dec 03 12:51:37 crc kubenswrapper[5052]: I1203 12:51:37.721835 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"2b0bf9ae-2d1d-48da-8c4b-2973b47985a3","Type":"ContainerStarted","Data":"78894a79ffab8e64787cde1b1f24aa5fa09f830cec10c1b466b6619c2b33ae10"} Dec 03 12:51:37 crc kubenswrapper[5052]: I1203 12:51:37.756452 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/cinder-scheduler-0" podStartSLOduration=3.756430892 podStartE2EDuration="3.756430892s" podCreationTimestamp="2025-12-03 12:51:34 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:51:37.744174992 +0000 UTC m=+5488.332416766" watchObservedRunningTime="2025-12-03 12:51:37.756430892 +0000 UTC m=+5488.344672657" Dec 03 12:51:37 crc kubenswrapper[5052]: I1203 12:51:37.990130 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:51:40 crc kubenswrapper[5052]: I1203 12:51:40.202238 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:51:45 crc kubenswrapper[5052]: I1203 12:51:45.554041 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:51:45 crc kubenswrapper[5052]: I1203 12:51:45.705480 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:51:46 crc kubenswrapper[5052]: I1203 12:51:46.663349 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/placement-7b465d48b8-4bj69" Dec 03 12:51:46 crc kubenswrapper[5052]: I1203 12:51:46.715261 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/placement-7b465d48b8-4bj69" Dec 03 12:51:55 crc kubenswrapper[5052]: I1203 12:51:55.401646 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/keystone-7fd545bd9-2lc7s" Dec 03 12:51:58 crc kubenswrapper[5052]: I1203 12:51:58.884644 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/openstackclient"] Dec 03 12:51:58 crc kubenswrapper[5052]: E1203 12:51:58.886238 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="539d9404-7518-4f03-815d-568a43eedb0b" containerName="barbican-api" Dec 03 12:51:58 crc kubenswrapper[5052]: I1203 12:51:58.886275 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="539d9404-7518-4f03-815d-568a43eedb0b" containerName="barbican-api" Dec 03 12:51:58 crc kubenswrapper[5052]: E1203 12:51:58.886335 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="539d9404-7518-4f03-815d-568a43eedb0b" containerName="barbican-api-log" Dec 03 12:51:58 crc kubenswrapper[5052]: I1203 12:51:58.886354 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="539d9404-7518-4f03-815d-568a43eedb0b" containerName="barbican-api-log" Dec 03 12:51:58 crc kubenswrapper[5052]: I1203 12:51:58.886816 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="539d9404-7518-4f03-815d-568a43eedb0b" containerName="barbican-api" Dec 03 12:51:58 crc kubenswrapper[5052]: I1203 12:51:58.886872 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="539d9404-7518-4f03-815d-568a43eedb0b" containerName="barbican-api-log" Dec 03 12:51:58 crc kubenswrapper[5052]: I1203 12:51:58.888281 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstackclient" Dec 03 12:51:58 crc kubenswrapper[5052]: I1203 12:51:58.891764 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"openstack-config-secret" Dec 03 12:51:58 crc kubenswrapper[5052]: I1203 12:51:58.891985 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"openstackclient-openstackclient-dockercfg-jfg5h" Dec 03 12:51:58 crc kubenswrapper[5052]: I1203 12:51:58.903294 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"openstack-config" Dec 03 12:51:58 crc kubenswrapper[5052]: I1203 12:51:58.914695 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/openstackclient"] Dec 03 12:51:59 crc kubenswrapper[5052]: I1203 12:51:59.012339 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openstack-config\" (UniqueName: \"kubernetes.io/configmap/c4b7bd04-32be-43c6-82f1-968f2e368b22-openstack-config\") pod \"openstackclient\" (UID: \"c4b7bd04-32be-43c6-82f1-968f2e368b22\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 12:51:59 crc kubenswrapper[5052]: I1203 12:51:59.012470 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t6gvj\" (UniqueName: \"kubernetes.io/projected/c4b7bd04-32be-43c6-82f1-968f2e368b22-kube-api-access-t6gvj\") pod \"openstackclient\" (UID: \"c4b7bd04-32be-43c6-82f1-968f2e368b22\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 12:51:59 crc kubenswrapper[5052]: I1203 12:51:59.012683 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openstack-config-secret\" (UniqueName: \"kubernetes.io/secret/c4b7bd04-32be-43c6-82f1-968f2e368b22-openstack-config-secret\") pod \"openstackclient\" (UID: \"c4b7bd04-32be-43c6-82f1-968f2e368b22\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 12:51:59 crc kubenswrapper[5052]: I1203 12:51:59.012789 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c4b7bd04-32be-43c6-82f1-968f2e368b22-combined-ca-bundle\") pod \"openstackclient\" (UID: \"c4b7bd04-32be-43c6-82f1-968f2e368b22\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 12:51:59 crc kubenswrapper[5052]: I1203 12:51:59.114763 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"openstack-config-secret\" (UniqueName: \"kubernetes.io/secret/c4b7bd04-32be-43c6-82f1-968f2e368b22-openstack-config-secret\") pod \"openstackclient\" (UID: \"c4b7bd04-32be-43c6-82f1-968f2e368b22\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 12:51:59 crc kubenswrapper[5052]: I1203 12:51:59.114844 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c4b7bd04-32be-43c6-82f1-968f2e368b22-combined-ca-bundle\") pod \"openstackclient\" (UID: \"c4b7bd04-32be-43c6-82f1-968f2e368b22\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 12:51:59 crc kubenswrapper[5052]: I1203 12:51:59.114955 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"openstack-config\" (UniqueName: \"kubernetes.io/configmap/c4b7bd04-32be-43c6-82f1-968f2e368b22-openstack-config\") pod \"openstackclient\" (UID: \"c4b7bd04-32be-43c6-82f1-968f2e368b22\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 12:51:59 crc kubenswrapper[5052]: I1203 12:51:59.115057 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-t6gvj\" (UniqueName: \"kubernetes.io/projected/c4b7bd04-32be-43c6-82f1-968f2e368b22-kube-api-access-t6gvj\") pod \"openstackclient\" (UID: \"c4b7bd04-32be-43c6-82f1-968f2e368b22\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 12:51:59 crc kubenswrapper[5052]: I1203 12:51:59.115842 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"openstack-config\" (UniqueName: \"kubernetes.io/configmap/c4b7bd04-32be-43c6-82f1-968f2e368b22-openstack-config\") pod \"openstackclient\" (UID: \"c4b7bd04-32be-43c6-82f1-968f2e368b22\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 12:51:59 crc kubenswrapper[5052]: I1203 12:51:59.120732 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"openstack-config-secret\" (UniqueName: \"kubernetes.io/secret/c4b7bd04-32be-43c6-82f1-968f2e368b22-openstack-config-secret\") pod \"openstackclient\" (UID: \"c4b7bd04-32be-43c6-82f1-968f2e368b22\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 12:51:59 crc kubenswrapper[5052]: I1203 12:51:59.123433 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c4b7bd04-32be-43c6-82f1-968f2e368b22-combined-ca-bundle\") pod \"openstackclient\" (UID: \"c4b7bd04-32be-43c6-82f1-968f2e368b22\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 12:51:59 crc kubenswrapper[5052]: I1203 12:51:59.131154 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-t6gvj\" (UniqueName: \"kubernetes.io/projected/c4b7bd04-32be-43c6-82f1-968f2e368b22-kube-api-access-t6gvj\") pod \"openstackclient\" (UID: \"c4b7bd04-32be-43c6-82f1-968f2e368b22\") " pod="openstack-kuttl-tests/openstackclient" Dec 03 12:51:59 crc kubenswrapper[5052]: I1203 12:51:59.215492 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstackclient" Dec 03 12:51:59 crc kubenswrapper[5052]: I1203 12:51:59.383790 5052 patch_prober.go:28] interesting pod/machine-config-daemon-74bj6 container/machine-config-daemon namespace/openshift-machine-config-operator: Liveness probe status=failure output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" start-of-body= Dec 03 12:51:59 crc kubenswrapper[5052]: I1203 12:51:59.384202 5052 prober.go:107] "Probe failed" probeType="Liveness" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" containerName="machine-config-daemon" probeResult="failure" output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" Dec 03 12:51:59 crc kubenswrapper[5052]: I1203 12:51:59.384245 5052 kubelet.go:2542] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" Dec 03 12:51:59 crc kubenswrapper[5052]: I1203 12:51:59.384785 5052 kuberuntime_manager.go:1027] "Message for Container of pod" containerName="machine-config-daemon" containerStatusID={"Type":"cri-o","ID":"75d5669ef68596efd94ee7584a4e5e570e7db6cc9f424db1fa9cd7f60acbab7b"} pod="openshift-machine-config-operator/machine-config-daemon-74bj6" containerMessage="Container machine-config-daemon failed liveness probe, will be restarted" Dec 03 12:51:59 crc kubenswrapper[5052]: I1203 12:51:59.384837 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" containerName="machine-config-daemon" containerID="cri-o://75d5669ef68596efd94ee7584a4e5e570e7db6cc9f424db1fa9cd7f60acbab7b" gracePeriod=600 Dec 03 12:51:59 crc kubenswrapper[5052]: I1203 12:51:59.686648 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/openstackclient"] Dec 03 12:51:59 crc kubenswrapper[5052]: I1203 12:51:59.962304 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstackclient" event={"ID":"c4b7bd04-32be-43c6-82f1-968f2e368b22","Type":"ContainerStarted","Data":"863f9a5dead36dfcd4e960256f81adece8f398e82c1cc6eb7b8c1e26766d0633"} Dec 03 12:51:59 crc kubenswrapper[5052]: I1203 12:51:59.962441 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstackclient" event={"ID":"c4b7bd04-32be-43c6-82f1-968f2e368b22","Type":"ContainerStarted","Data":"5d6a57a701e1824adcf99a1cdf4ef76b22e6865f5543d0707e653c83480e4eb4"} Dec 03 12:51:59 crc kubenswrapper[5052]: I1203 12:51:59.967275 5052 generic.go:334] "Generic (PLEG): container finished" podID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" containerID="75d5669ef68596efd94ee7584a4e5e570e7db6cc9f424db1fa9cd7f60acbab7b" exitCode=0 Dec 03 12:51:59 crc kubenswrapper[5052]: I1203 12:51:59.967329 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" event={"ID":"10c237bc-72e7-4da3-9f2c-b1688395bf6e","Type":"ContainerDied","Data":"75d5669ef68596efd94ee7584a4e5e570e7db6cc9f424db1fa9cd7f60acbab7b"} Dec 03 12:51:59 crc kubenswrapper[5052]: I1203 12:51:59.967376 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" event={"ID":"10c237bc-72e7-4da3-9f2c-b1688395bf6e","Type":"ContainerStarted","Data":"69fa7fbcd2aceb7c53703a19508876f93a97dc9adabb79c16fb691d90f606d58"} Dec 03 12:51:59 crc kubenswrapper[5052]: I1203 12:51:59.967403 5052 scope.go:117] "RemoveContainer" containerID="2aed1c760a326f3d008799e902b1093313df3764a57c879f0b3686410127d37d" Dec 03 12:52:00 crc kubenswrapper[5052]: I1203 12:52:00.013075 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/openstackclient" podStartSLOduration=2.013051942 podStartE2EDuration="2.013051942s" podCreationTimestamp="2025-12-03 12:51:58 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:51:59.990976611 +0000 UTC m=+5510.579218375" watchObservedRunningTime="2025-12-03 12:52:00.013051942 +0000 UTC m=+5510.601293726" Dec 03 12:52:01 crc kubenswrapper[5052]: I1203 12:52:01.048814 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/neutron-57cc9dddbf-g8d9w" Dec 03 12:52:02 crc kubenswrapper[5052]: I1203 12:52:02.533139 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/swift-proxy-5479cb94c8-q2snc"] Dec 03 12:52:02 crc kubenswrapper[5052]: I1203 12:52:02.547779 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-proxy-5479cb94c8-q2snc" Dec 03 12:52:02 crc kubenswrapper[5052]: I1203 12:52:02.553242 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"swift-proxy-config-data" Dec 03 12:52:02 crc kubenswrapper[5052]: I1203 12:52:02.553493 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-swift-public-svc" Dec 03 12:52:02 crc kubenswrapper[5052]: I1203 12:52:02.553712 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-swift-internal-svc" Dec 03 12:52:02 crc kubenswrapper[5052]: I1203 12:52:02.554215 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/swift-proxy-5479cb94c8-q2snc"] Dec 03 12:52:02 crc kubenswrapper[5052]: I1203 12:52:02.685548 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/eefe83f8-0235-4b16-aefb-5ec7b2d92fcf-internal-tls-certs\") pod \"swift-proxy-5479cb94c8-q2snc\" (UID: \"eefe83f8-0235-4b16-aefb-5ec7b2d92fcf\") " pod="openstack-kuttl-tests/swift-proxy-5479cb94c8-q2snc" Dec 03 12:52:02 crc kubenswrapper[5052]: I1203 12:52:02.685619 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/eefe83f8-0235-4b16-aefb-5ec7b2d92fcf-log-httpd\") pod \"swift-proxy-5479cb94c8-q2snc\" (UID: \"eefe83f8-0235-4b16-aefb-5ec7b2d92fcf\") " pod="openstack-kuttl-tests/swift-proxy-5479cb94c8-q2snc" Dec 03 12:52:02 crc kubenswrapper[5052]: I1203 12:52:02.685650 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/eefe83f8-0235-4b16-aefb-5ec7b2d92fcf-etc-swift\") pod \"swift-proxy-5479cb94c8-q2snc\" (UID: \"eefe83f8-0235-4b16-aefb-5ec7b2d92fcf\") " pod="openstack-kuttl-tests/swift-proxy-5479cb94c8-q2snc" Dec 03 12:52:02 crc kubenswrapper[5052]: I1203 12:52:02.685726 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/eefe83f8-0235-4b16-aefb-5ec7b2d92fcf-run-httpd\") pod \"swift-proxy-5479cb94c8-q2snc\" (UID: \"eefe83f8-0235-4b16-aefb-5ec7b2d92fcf\") " pod="openstack-kuttl-tests/swift-proxy-5479cb94c8-q2snc" Dec 03 12:52:02 crc kubenswrapper[5052]: I1203 12:52:02.685786 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v7gp8\" (UniqueName: \"kubernetes.io/projected/eefe83f8-0235-4b16-aefb-5ec7b2d92fcf-kube-api-access-v7gp8\") pod \"swift-proxy-5479cb94c8-q2snc\" (UID: \"eefe83f8-0235-4b16-aefb-5ec7b2d92fcf\") " pod="openstack-kuttl-tests/swift-proxy-5479cb94c8-q2snc" Dec 03 12:52:02 crc kubenswrapper[5052]: I1203 12:52:02.685810 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/eefe83f8-0235-4b16-aefb-5ec7b2d92fcf-public-tls-certs\") pod \"swift-proxy-5479cb94c8-q2snc\" (UID: \"eefe83f8-0235-4b16-aefb-5ec7b2d92fcf\") " pod="openstack-kuttl-tests/swift-proxy-5479cb94c8-q2snc" Dec 03 12:52:02 crc kubenswrapper[5052]: I1203 12:52:02.685880 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/eefe83f8-0235-4b16-aefb-5ec7b2d92fcf-combined-ca-bundle\") pod \"swift-proxy-5479cb94c8-q2snc\" (UID: \"eefe83f8-0235-4b16-aefb-5ec7b2d92fcf\") " pod="openstack-kuttl-tests/swift-proxy-5479cb94c8-q2snc" Dec 03 12:52:02 crc kubenswrapper[5052]: I1203 12:52:02.685905 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/eefe83f8-0235-4b16-aefb-5ec7b2d92fcf-config-data\") pod \"swift-proxy-5479cb94c8-q2snc\" (UID: \"eefe83f8-0235-4b16-aefb-5ec7b2d92fcf\") " pod="openstack-kuttl-tests/swift-proxy-5479cb94c8-q2snc" Dec 03 12:52:02 crc kubenswrapper[5052]: I1203 12:52:02.787595 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/eefe83f8-0235-4b16-aefb-5ec7b2d92fcf-combined-ca-bundle\") pod \"swift-proxy-5479cb94c8-q2snc\" (UID: \"eefe83f8-0235-4b16-aefb-5ec7b2d92fcf\") " pod="openstack-kuttl-tests/swift-proxy-5479cb94c8-q2snc" Dec 03 12:52:02 crc kubenswrapper[5052]: I1203 12:52:02.787933 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/eefe83f8-0235-4b16-aefb-5ec7b2d92fcf-config-data\") pod \"swift-proxy-5479cb94c8-q2snc\" (UID: \"eefe83f8-0235-4b16-aefb-5ec7b2d92fcf\") " pod="openstack-kuttl-tests/swift-proxy-5479cb94c8-q2snc" Dec 03 12:52:02 crc kubenswrapper[5052]: I1203 12:52:02.788077 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/eefe83f8-0235-4b16-aefb-5ec7b2d92fcf-internal-tls-certs\") pod \"swift-proxy-5479cb94c8-q2snc\" (UID: \"eefe83f8-0235-4b16-aefb-5ec7b2d92fcf\") " pod="openstack-kuttl-tests/swift-proxy-5479cb94c8-q2snc" Dec 03 12:52:02 crc kubenswrapper[5052]: I1203 12:52:02.788169 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/eefe83f8-0235-4b16-aefb-5ec7b2d92fcf-log-httpd\") pod \"swift-proxy-5479cb94c8-q2snc\" (UID: \"eefe83f8-0235-4b16-aefb-5ec7b2d92fcf\") " pod="openstack-kuttl-tests/swift-proxy-5479cb94c8-q2snc" Dec 03 12:52:02 crc kubenswrapper[5052]: I1203 12:52:02.788245 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/eefe83f8-0235-4b16-aefb-5ec7b2d92fcf-etc-swift\") pod \"swift-proxy-5479cb94c8-q2snc\" (UID: \"eefe83f8-0235-4b16-aefb-5ec7b2d92fcf\") " pod="openstack-kuttl-tests/swift-proxy-5479cb94c8-q2snc" Dec 03 12:52:02 crc kubenswrapper[5052]: I1203 12:52:02.788370 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/eefe83f8-0235-4b16-aefb-5ec7b2d92fcf-run-httpd\") pod \"swift-proxy-5479cb94c8-q2snc\" (UID: \"eefe83f8-0235-4b16-aefb-5ec7b2d92fcf\") " pod="openstack-kuttl-tests/swift-proxy-5479cb94c8-q2snc" Dec 03 12:52:02 crc kubenswrapper[5052]: I1203 12:52:02.788472 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-v7gp8\" (UniqueName: \"kubernetes.io/projected/eefe83f8-0235-4b16-aefb-5ec7b2d92fcf-kube-api-access-v7gp8\") pod \"swift-proxy-5479cb94c8-q2snc\" (UID: \"eefe83f8-0235-4b16-aefb-5ec7b2d92fcf\") " pod="openstack-kuttl-tests/swift-proxy-5479cb94c8-q2snc" Dec 03 12:52:02 crc kubenswrapper[5052]: I1203 12:52:02.788579 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/eefe83f8-0235-4b16-aefb-5ec7b2d92fcf-public-tls-certs\") pod \"swift-proxy-5479cb94c8-q2snc\" (UID: \"eefe83f8-0235-4b16-aefb-5ec7b2d92fcf\") " pod="openstack-kuttl-tests/swift-proxy-5479cb94c8-q2snc" Dec 03 12:52:02 crc kubenswrapper[5052]: I1203 12:52:02.788856 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/eefe83f8-0235-4b16-aefb-5ec7b2d92fcf-log-httpd\") pod \"swift-proxy-5479cb94c8-q2snc\" (UID: \"eefe83f8-0235-4b16-aefb-5ec7b2d92fcf\") " pod="openstack-kuttl-tests/swift-proxy-5479cb94c8-q2snc" Dec 03 12:52:02 crc kubenswrapper[5052]: I1203 12:52:02.788931 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/eefe83f8-0235-4b16-aefb-5ec7b2d92fcf-run-httpd\") pod \"swift-proxy-5479cb94c8-q2snc\" (UID: \"eefe83f8-0235-4b16-aefb-5ec7b2d92fcf\") " pod="openstack-kuttl-tests/swift-proxy-5479cb94c8-q2snc" Dec 03 12:52:02 crc kubenswrapper[5052]: I1203 12:52:02.807877 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/eefe83f8-0235-4b16-aefb-5ec7b2d92fcf-public-tls-certs\") pod \"swift-proxy-5479cb94c8-q2snc\" (UID: \"eefe83f8-0235-4b16-aefb-5ec7b2d92fcf\") " pod="openstack-kuttl-tests/swift-proxy-5479cb94c8-q2snc" Dec 03 12:52:02 crc kubenswrapper[5052]: I1203 12:52:02.807903 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/eefe83f8-0235-4b16-aefb-5ec7b2d92fcf-config-data\") pod \"swift-proxy-5479cb94c8-q2snc\" (UID: \"eefe83f8-0235-4b16-aefb-5ec7b2d92fcf\") " pod="openstack-kuttl-tests/swift-proxy-5479cb94c8-q2snc" Dec 03 12:52:02 crc kubenswrapper[5052]: I1203 12:52:02.807898 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/eefe83f8-0235-4b16-aefb-5ec7b2d92fcf-internal-tls-certs\") pod \"swift-proxy-5479cb94c8-q2snc\" (UID: \"eefe83f8-0235-4b16-aefb-5ec7b2d92fcf\") " pod="openstack-kuttl-tests/swift-proxy-5479cb94c8-q2snc" Dec 03 12:52:02 crc kubenswrapper[5052]: I1203 12:52:02.808784 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/eefe83f8-0235-4b16-aefb-5ec7b2d92fcf-etc-swift\") pod \"swift-proxy-5479cb94c8-q2snc\" (UID: \"eefe83f8-0235-4b16-aefb-5ec7b2d92fcf\") " pod="openstack-kuttl-tests/swift-proxy-5479cb94c8-q2snc" Dec 03 12:52:02 crc kubenswrapper[5052]: I1203 12:52:02.809549 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/eefe83f8-0235-4b16-aefb-5ec7b2d92fcf-combined-ca-bundle\") pod \"swift-proxy-5479cb94c8-q2snc\" (UID: \"eefe83f8-0235-4b16-aefb-5ec7b2d92fcf\") " pod="openstack-kuttl-tests/swift-proxy-5479cb94c8-q2snc" Dec 03 12:52:02 crc kubenswrapper[5052]: I1203 12:52:02.812670 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-v7gp8\" (UniqueName: \"kubernetes.io/projected/eefe83f8-0235-4b16-aefb-5ec7b2d92fcf-kube-api-access-v7gp8\") pod \"swift-proxy-5479cb94c8-q2snc\" (UID: \"eefe83f8-0235-4b16-aefb-5ec7b2d92fcf\") " pod="openstack-kuttl-tests/swift-proxy-5479cb94c8-q2snc" Dec 03 12:52:02 crc kubenswrapper[5052]: I1203 12:52:02.883295 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-proxy-5479cb94c8-q2snc" Dec 03 12:52:03 crc kubenswrapper[5052]: W1203 12:52:03.378206 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podeefe83f8_0235_4b16_aefb_5ec7b2d92fcf.slice/crio-7a0e8c623d922e9f99403581b92083b2a8cdbfa4f8428dce82708efd3a82fb94 WatchSource:0}: Error finding container 7a0e8c623d922e9f99403581b92083b2a8cdbfa4f8428dce82708efd3a82fb94: Status 404 returned error can't find the container with id 7a0e8c623d922e9f99403581b92083b2a8cdbfa4f8428dce82708efd3a82fb94 Dec 03 12:52:03 crc kubenswrapper[5052]: I1203 12:52:03.382434 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/swift-proxy-5479cb94c8-q2snc"] Dec 03 12:52:04 crc kubenswrapper[5052]: I1203 12:52:04.007191 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-proxy-5479cb94c8-q2snc" event={"ID":"eefe83f8-0235-4b16-aefb-5ec7b2d92fcf","Type":"ContainerStarted","Data":"29117e54764872df757ac95e94dbab057dd9c2d6f5908051e97419768ed8628a"} Dec 03 12:52:04 crc kubenswrapper[5052]: I1203 12:52:04.007483 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-proxy-5479cb94c8-q2snc" event={"ID":"eefe83f8-0235-4b16-aefb-5ec7b2d92fcf","Type":"ContainerStarted","Data":"ac795f902a20e66db9af6d237bc550efac4a63fdac0b163e8ceb8138a8c4a16f"} Dec 03 12:52:04 crc kubenswrapper[5052]: I1203 12:52:04.007493 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-proxy-5479cb94c8-q2snc" event={"ID":"eefe83f8-0235-4b16-aefb-5ec7b2d92fcf","Type":"ContainerStarted","Data":"7a0e8c623d922e9f99403581b92083b2a8cdbfa4f8428dce82708efd3a82fb94"} Dec 03 12:52:04 crc kubenswrapper[5052]: I1203 12:52:04.009011 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/swift-proxy-5479cb94c8-q2snc" Dec 03 12:52:04 crc kubenswrapper[5052]: I1203 12:52:04.009085 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/swift-proxy-5479cb94c8-q2snc" Dec 03 12:52:04 crc kubenswrapper[5052]: I1203 12:52:04.051912 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/swift-proxy-5479cb94c8-q2snc" podStartSLOduration=2.051892976 podStartE2EDuration="2.051892976s" podCreationTimestamp="2025-12-03 12:52:02 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:52:04.042077045 +0000 UTC m=+5514.630318829" watchObservedRunningTime="2025-12-03 12:52:04.051892976 +0000 UTC m=+5514.640134740" Dec 03 12:52:04 crc kubenswrapper[5052]: I1203 12:52:04.731092 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/neutron-698d8977cd-ppz9n" Dec 03 12:52:04 crc kubenswrapper[5052]: I1203 12:52:04.812222 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/neutron-57cc9dddbf-g8d9w"] Dec 03 12:52:04 crc kubenswrapper[5052]: I1203 12:52:04.812490 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/neutron-57cc9dddbf-g8d9w" podUID="0fc6a196-8a8e-4daf-9bbe-48d28e2e1191" containerName="neutron-api" containerID="cri-o://5d7518740b9f063814d21fa5094564958d8f7a70cc9e607d2a89ceff536afeb6" gracePeriod=30 Dec 03 12:52:04 crc kubenswrapper[5052]: I1203 12:52:04.812658 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/neutron-57cc9dddbf-g8d9w" podUID="0fc6a196-8a8e-4daf-9bbe-48d28e2e1191" containerName="neutron-httpd" containerID="cri-o://a546a8af3f11336976a636365b701a3471736636285673c4dd7e4ce181388249" gracePeriod=30 Dec 03 12:52:05 crc kubenswrapper[5052]: I1203 12:52:05.745647 5052 pod_container_manager_linux.go:210] "Failed to delete cgroup paths" cgroupName=["kubepods","besteffort","pod539d9404-7518-4f03-815d-568a43eedb0b"] err="unable to destroy cgroup paths for cgroup [kubepods besteffort pod539d9404-7518-4f03-815d-568a43eedb0b] : Timed out while waiting for systemd to remove kubepods-besteffort-pod539d9404_7518_4f03_815d_568a43eedb0b.slice" Dec 03 12:52:05 crc kubenswrapper[5052]: E1203 12:52:05.745987 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to delete cgroup paths for [kubepods besteffort pod539d9404-7518-4f03-815d-568a43eedb0b] : unable to destroy cgroup paths for cgroup [kubepods besteffort pod539d9404-7518-4f03-815d-568a43eedb0b] : Timed out while waiting for systemd to remove kubepods-besteffort-pod539d9404_7518_4f03_815d_568a43eedb0b.slice" pod="openstack-kuttl-tests/barbican-api-5bdcd5dc6-vqlkq" podUID="539d9404-7518-4f03-815d-568a43eedb0b" Dec 03 12:52:06 crc kubenswrapper[5052]: I1203 12:52:06.047113 5052 generic.go:334] "Generic (PLEG): container finished" podID="0fc6a196-8a8e-4daf-9bbe-48d28e2e1191" containerID="a546a8af3f11336976a636365b701a3471736636285673c4dd7e4ce181388249" exitCode=0 Dec 03 12:52:06 crc kubenswrapper[5052]: I1203 12:52:06.047164 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-57cc9dddbf-g8d9w" event={"ID":"0fc6a196-8a8e-4daf-9bbe-48d28e2e1191","Type":"ContainerDied","Data":"a546a8af3f11336976a636365b701a3471736636285673c4dd7e4ce181388249"} Dec 03 12:52:06 crc kubenswrapper[5052]: I1203 12:52:06.047255 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-api-5bdcd5dc6-vqlkq" Dec 03 12:52:06 crc kubenswrapper[5052]: I1203 12:52:06.071231 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-api-5bdcd5dc6-vqlkq"] Dec 03 12:52:06 crc kubenswrapper[5052]: I1203 12:52:06.078821 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/barbican-api-5bdcd5dc6-vqlkq"] Dec 03 12:52:07 crc kubenswrapper[5052]: I1203 12:52:07.685546 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="539d9404-7518-4f03-815d-568a43eedb0b" path="/var/lib/kubelet/pods/539d9404-7518-4f03-815d-568a43eedb0b/volumes" Dec 03 12:52:07 crc kubenswrapper[5052]: I1203 12:52:07.920414 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-57cc9dddbf-g8d9w" Dec 03 12:52:08 crc kubenswrapper[5052]: I1203 12:52:07.999897 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0fc6a196-8a8e-4daf-9bbe-48d28e2e1191-combined-ca-bundle\") pod \"0fc6a196-8a8e-4daf-9bbe-48d28e2e1191\" (UID: \"0fc6a196-8a8e-4daf-9bbe-48d28e2e1191\") " Dec 03 12:52:08 crc kubenswrapper[5052]: I1203 12:52:07.999954 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/0fc6a196-8a8e-4daf-9bbe-48d28e2e1191-httpd-config\") pod \"0fc6a196-8a8e-4daf-9bbe-48d28e2e1191\" (UID: \"0fc6a196-8a8e-4daf-9bbe-48d28e2e1191\") " Dec 03 12:52:08 crc kubenswrapper[5052]: I1203 12:52:08.000014 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/0fc6a196-8a8e-4daf-9bbe-48d28e2e1191-config\") pod \"0fc6a196-8a8e-4daf-9bbe-48d28e2e1191\" (UID: \"0fc6a196-8a8e-4daf-9bbe-48d28e2e1191\") " Dec 03 12:52:08 crc kubenswrapper[5052]: I1203 12:52:08.000103 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/0fc6a196-8a8e-4daf-9bbe-48d28e2e1191-ovndb-tls-certs\") pod \"0fc6a196-8a8e-4daf-9bbe-48d28e2e1191\" (UID: \"0fc6a196-8a8e-4daf-9bbe-48d28e2e1191\") " Dec 03 12:52:08 crc kubenswrapper[5052]: I1203 12:52:08.000139 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6qlpv\" (UniqueName: \"kubernetes.io/projected/0fc6a196-8a8e-4daf-9bbe-48d28e2e1191-kube-api-access-6qlpv\") pod \"0fc6a196-8a8e-4daf-9bbe-48d28e2e1191\" (UID: \"0fc6a196-8a8e-4daf-9bbe-48d28e2e1191\") " Dec 03 12:52:08 crc kubenswrapper[5052]: I1203 12:52:08.005486 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0fc6a196-8a8e-4daf-9bbe-48d28e2e1191-httpd-config" (OuterVolumeSpecName: "httpd-config") pod "0fc6a196-8a8e-4daf-9bbe-48d28e2e1191" (UID: "0fc6a196-8a8e-4daf-9bbe-48d28e2e1191"). InnerVolumeSpecName "httpd-config". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:52:08 crc kubenswrapper[5052]: I1203 12:52:08.005882 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0fc6a196-8a8e-4daf-9bbe-48d28e2e1191-kube-api-access-6qlpv" (OuterVolumeSpecName: "kube-api-access-6qlpv") pod "0fc6a196-8a8e-4daf-9bbe-48d28e2e1191" (UID: "0fc6a196-8a8e-4daf-9bbe-48d28e2e1191"). InnerVolumeSpecName "kube-api-access-6qlpv". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:52:08 crc kubenswrapper[5052]: I1203 12:52:08.057836 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0fc6a196-8a8e-4daf-9bbe-48d28e2e1191-config" (OuterVolumeSpecName: "config") pod "0fc6a196-8a8e-4daf-9bbe-48d28e2e1191" (UID: "0fc6a196-8a8e-4daf-9bbe-48d28e2e1191"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:52:08 crc kubenswrapper[5052]: I1203 12:52:08.069482 5052 generic.go:334] "Generic (PLEG): container finished" podID="0fc6a196-8a8e-4daf-9bbe-48d28e2e1191" containerID="5d7518740b9f063814d21fa5094564958d8f7a70cc9e607d2a89ceff536afeb6" exitCode=0 Dec 03 12:52:08 crc kubenswrapper[5052]: I1203 12:52:08.069584 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-57cc9dddbf-g8d9w" event={"ID":"0fc6a196-8a8e-4daf-9bbe-48d28e2e1191","Type":"ContainerDied","Data":"5d7518740b9f063814d21fa5094564958d8f7a70cc9e607d2a89ceff536afeb6"} Dec 03 12:52:08 crc kubenswrapper[5052]: I1203 12:52:08.069617 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-57cc9dddbf-g8d9w" event={"ID":"0fc6a196-8a8e-4daf-9bbe-48d28e2e1191","Type":"ContainerDied","Data":"3327c71b7d5574d8c41d7448686bddf428aeb8a282cb62195991720398ebc15b"} Dec 03 12:52:08 crc kubenswrapper[5052]: I1203 12:52:08.069669 5052 scope.go:117] "RemoveContainer" containerID="a546a8af3f11336976a636365b701a3471736636285673c4dd7e4ce181388249" Dec 03 12:52:08 crc kubenswrapper[5052]: I1203 12:52:08.070221 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-57cc9dddbf-g8d9w" Dec 03 12:52:08 crc kubenswrapper[5052]: I1203 12:52:08.084506 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0fc6a196-8a8e-4daf-9bbe-48d28e2e1191-ovndb-tls-certs" (OuterVolumeSpecName: "ovndb-tls-certs") pod "0fc6a196-8a8e-4daf-9bbe-48d28e2e1191" (UID: "0fc6a196-8a8e-4daf-9bbe-48d28e2e1191"). InnerVolumeSpecName "ovndb-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:52:08 crc kubenswrapper[5052]: I1203 12:52:08.086481 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0fc6a196-8a8e-4daf-9bbe-48d28e2e1191-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "0fc6a196-8a8e-4daf-9bbe-48d28e2e1191" (UID: "0fc6a196-8a8e-4daf-9bbe-48d28e2e1191"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:52:08 crc kubenswrapper[5052]: I1203 12:52:08.102223 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0fc6a196-8a8e-4daf-9bbe-48d28e2e1191-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:52:08 crc kubenswrapper[5052]: I1203 12:52:08.102267 5052 reconciler_common.go:293] "Volume detached for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/0fc6a196-8a8e-4daf-9bbe-48d28e2e1191-httpd-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:52:08 crc kubenswrapper[5052]: I1203 12:52:08.102281 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/0fc6a196-8a8e-4daf-9bbe-48d28e2e1191-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:52:08 crc kubenswrapper[5052]: I1203 12:52:08.102294 5052 reconciler_common.go:293] "Volume detached for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/0fc6a196-8a8e-4daf-9bbe-48d28e2e1191-ovndb-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:52:08 crc kubenswrapper[5052]: I1203 12:52:08.102308 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-6qlpv\" (UniqueName: \"kubernetes.io/projected/0fc6a196-8a8e-4daf-9bbe-48d28e2e1191-kube-api-access-6qlpv\") on node \"crc\" DevicePath \"\"" Dec 03 12:52:08 crc kubenswrapper[5052]: I1203 12:52:08.136139 5052 scope.go:117] "RemoveContainer" containerID="5d7518740b9f063814d21fa5094564958d8f7a70cc9e607d2a89ceff536afeb6" Dec 03 12:52:08 crc kubenswrapper[5052]: I1203 12:52:08.159313 5052 scope.go:117] "RemoveContainer" containerID="a546a8af3f11336976a636365b701a3471736636285673c4dd7e4ce181388249" Dec 03 12:52:08 crc kubenswrapper[5052]: E1203 12:52:08.159736 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a546a8af3f11336976a636365b701a3471736636285673c4dd7e4ce181388249\": container with ID starting with a546a8af3f11336976a636365b701a3471736636285673c4dd7e4ce181388249 not found: ID does not exist" containerID="a546a8af3f11336976a636365b701a3471736636285673c4dd7e4ce181388249" Dec 03 12:52:08 crc kubenswrapper[5052]: I1203 12:52:08.159777 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a546a8af3f11336976a636365b701a3471736636285673c4dd7e4ce181388249"} err="failed to get container status \"a546a8af3f11336976a636365b701a3471736636285673c4dd7e4ce181388249\": rpc error: code = NotFound desc = could not find container \"a546a8af3f11336976a636365b701a3471736636285673c4dd7e4ce181388249\": container with ID starting with a546a8af3f11336976a636365b701a3471736636285673c4dd7e4ce181388249 not found: ID does not exist" Dec 03 12:52:08 crc kubenswrapper[5052]: I1203 12:52:08.159803 5052 scope.go:117] "RemoveContainer" containerID="5d7518740b9f063814d21fa5094564958d8f7a70cc9e607d2a89ceff536afeb6" Dec 03 12:52:08 crc kubenswrapper[5052]: E1203 12:52:08.160237 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5d7518740b9f063814d21fa5094564958d8f7a70cc9e607d2a89ceff536afeb6\": container with ID starting with 5d7518740b9f063814d21fa5094564958d8f7a70cc9e607d2a89ceff536afeb6 not found: ID does not exist" containerID="5d7518740b9f063814d21fa5094564958d8f7a70cc9e607d2a89ceff536afeb6" Dec 03 12:52:08 crc kubenswrapper[5052]: I1203 12:52:08.160263 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5d7518740b9f063814d21fa5094564958d8f7a70cc9e607d2a89ceff536afeb6"} err="failed to get container status \"5d7518740b9f063814d21fa5094564958d8f7a70cc9e607d2a89ceff536afeb6\": rpc error: code = NotFound desc = could not find container \"5d7518740b9f063814d21fa5094564958d8f7a70cc9e607d2a89ceff536afeb6\": container with ID starting with 5d7518740b9f063814d21fa5094564958d8f7a70cc9e607d2a89ceff536afeb6 not found: ID does not exist" Dec 03 12:52:08 crc kubenswrapper[5052]: I1203 12:52:08.406608 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/neutron-57cc9dddbf-g8d9w"] Dec 03 12:52:08 crc kubenswrapper[5052]: I1203 12:52:08.414952 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/neutron-57cc9dddbf-g8d9w"] Dec 03 12:52:09 crc kubenswrapper[5052]: I1203 12:52:09.014839 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:52:09 crc kubenswrapper[5052]: I1203 12:52:09.015563 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="f3733be6-a76c-4c85-ad81-58ca3f3e3d2d" containerName="ceilometer-central-agent" containerID="cri-o://8fb1ea412deb631d1ed8163bda6feae84ba3cd74f0818f033958f4a2a30de564" gracePeriod=30 Dec 03 12:52:09 crc kubenswrapper[5052]: I1203 12:52:09.015595 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="f3733be6-a76c-4c85-ad81-58ca3f3e3d2d" containerName="sg-core" containerID="cri-o://bca28d625281627cc7675e3115c216e4fdf5af87f408df91e6006b4d6c37f376" gracePeriod=30 Dec 03 12:52:09 crc kubenswrapper[5052]: I1203 12:52:09.015633 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="f3733be6-a76c-4c85-ad81-58ca3f3e3d2d" containerName="proxy-httpd" containerID="cri-o://9753117b9d1d7a9e8974b3b7100aceebac6f8ff8d17dbee469702e6582413ceb" gracePeriod=30 Dec 03 12:52:09 crc kubenswrapper[5052]: I1203 12:52:09.015647 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="f3733be6-a76c-4c85-ad81-58ca3f3e3d2d" containerName="ceilometer-notification-agent" containerID="cri-o://262d5eb89c22ac292d99479055ae3d43a813219c6939b2f60660b7ee0bf8ae6d" gracePeriod=30 Dec 03 12:52:09 crc kubenswrapper[5052]: I1203 12:52:09.684628 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0fc6a196-8a8e-4daf-9bbe-48d28e2e1191" path="/var/lib/kubelet/pods/0fc6a196-8a8e-4daf-9bbe-48d28e2e1191/volumes" Dec 03 12:52:10 crc kubenswrapper[5052]: I1203 12:52:10.094498 5052 generic.go:334] "Generic (PLEG): container finished" podID="f3733be6-a76c-4c85-ad81-58ca3f3e3d2d" containerID="9753117b9d1d7a9e8974b3b7100aceebac6f8ff8d17dbee469702e6582413ceb" exitCode=0 Dec 03 12:52:10 crc kubenswrapper[5052]: I1203 12:52:10.094858 5052 generic.go:334] "Generic (PLEG): container finished" podID="f3733be6-a76c-4c85-ad81-58ca3f3e3d2d" containerID="bca28d625281627cc7675e3115c216e4fdf5af87f408df91e6006b4d6c37f376" exitCode=2 Dec 03 12:52:10 crc kubenswrapper[5052]: I1203 12:52:10.094871 5052 generic.go:334] "Generic (PLEG): container finished" podID="f3733be6-a76c-4c85-ad81-58ca3f3e3d2d" containerID="8fb1ea412deb631d1ed8163bda6feae84ba3cd74f0818f033958f4a2a30de564" exitCode=0 Dec 03 12:52:10 crc kubenswrapper[5052]: I1203 12:52:10.094640 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"f3733be6-a76c-4c85-ad81-58ca3f3e3d2d","Type":"ContainerDied","Data":"9753117b9d1d7a9e8974b3b7100aceebac6f8ff8d17dbee469702e6582413ceb"} Dec 03 12:52:10 crc kubenswrapper[5052]: I1203 12:52:10.094925 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"f3733be6-a76c-4c85-ad81-58ca3f3e3d2d","Type":"ContainerDied","Data":"bca28d625281627cc7675e3115c216e4fdf5af87f408df91e6006b4d6c37f376"} Dec 03 12:52:10 crc kubenswrapper[5052]: I1203 12:52:10.094954 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"f3733be6-a76c-4c85-ad81-58ca3f3e3d2d","Type":"ContainerDied","Data":"8fb1ea412deb631d1ed8163bda6feae84ba3cd74f0818f033958f4a2a30de564"} Dec 03 12:52:11 crc kubenswrapper[5052]: I1203 12:52:11.902180 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-api-db-create-xmgf4"] Dec 03 12:52:11 crc kubenswrapper[5052]: E1203 12:52:11.903691 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="0fc6a196-8a8e-4daf-9bbe-48d28e2e1191" containerName="neutron-api" Dec 03 12:52:11 crc kubenswrapper[5052]: I1203 12:52:11.903707 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="0fc6a196-8a8e-4daf-9bbe-48d28e2e1191" containerName="neutron-api" Dec 03 12:52:11 crc kubenswrapper[5052]: E1203 12:52:11.914641 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="0fc6a196-8a8e-4daf-9bbe-48d28e2e1191" containerName="neutron-httpd" Dec 03 12:52:11 crc kubenswrapper[5052]: I1203 12:52:11.914666 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="0fc6a196-8a8e-4daf-9bbe-48d28e2e1191" containerName="neutron-httpd" Dec 03 12:52:11 crc kubenswrapper[5052]: I1203 12:52:11.914991 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="0fc6a196-8a8e-4daf-9bbe-48d28e2e1191" containerName="neutron-api" Dec 03 12:52:11 crc kubenswrapper[5052]: I1203 12:52:11.915022 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="0fc6a196-8a8e-4daf-9bbe-48d28e2e1191" containerName="neutron-httpd" Dec 03 12:52:11 crc kubenswrapper[5052]: I1203 12:52:11.915651 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-db-create-xmgf4" Dec 03 12:52:11 crc kubenswrapper[5052]: I1203 12:52:11.925268 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-db-create-xmgf4"] Dec 03 12:52:12 crc kubenswrapper[5052]: I1203 12:52:12.000947 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell0-db-create-nvztf"] Dec 03 12:52:12 crc kubenswrapper[5052]: I1203 12:52:12.002161 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-db-create-nvztf" Dec 03 12:52:12 crc kubenswrapper[5052]: I1203 12:52:12.012751 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-api-7066-account-create-update-4xwgf"] Dec 03 12:52:12 crc kubenswrapper[5052]: I1203 12:52:12.014003 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-7066-account-create-update-4xwgf" Dec 03 12:52:12 crc kubenswrapper[5052]: I1203 12:52:12.015382 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-api-db-secret" Dec 03 12:52:12 crc kubenswrapper[5052]: I1203 12:52:12.026674 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell0-db-create-nvztf"] Dec 03 12:52:12 crc kubenswrapper[5052]: I1203 12:52:12.040272 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-7066-account-create-update-4xwgf"] Dec 03 12:52:12 crc kubenswrapper[5052]: I1203 12:52:12.098472 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bl9s5\" (UniqueName: \"kubernetes.io/projected/a3142a17-c4e9-49a1-95f3-f7b725516c93-kube-api-access-bl9s5\") pod \"nova-api-db-create-xmgf4\" (UID: \"a3142a17-c4e9-49a1-95f3-f7b725516c93\") " pod="openstack-kuttl-tests/nova-api-db-create-xmgf4" Dec 03 12:52:12 crc kubenswrapper[5052]: I1203 12:52:12.098532 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/a3142a17-c4e9-49a1-95f3-f7b725516c93-operator-scripts\") pod \"nova-api-db-create-xmgf4\" (UID: \"a3142a17-c4e9-49a1-95f3-f7b725516c93\") " pod="openstack-kuttl-tests/nova-api-db-create-xmgf4" Dec 03 12:52:12 crc kubenswrapper[5052]: I1203 12:52:12.119444 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell1-db-create-tp5g8"] Dec 03 12:52:12 crc kubenswrapper[5052]: I1203 12:52:12.121389 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-db-create-tp5g8" Dec 03 12:52:12 crc kubenswrapper[5052]: I1203 12:52:12.129525 5052 generic.go:334] "Generic (PLEG): container finished" podID="f3733be6-a76c-4c85-ad81-58ca3f3e3d2d" containerID="262d5eb89c22ac292d99479055ae3d43a813219c6939b2f60660b7ee0bf8ae6d" exitCode=0 Dec 03 12:52:12 crc kubenswrapper[5052]: I1203 12:52:12.129757 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"f3733be6-a76c-4c85-ad81-58ca3f3e3d2d","Type":"ContainerDied","Data":"262d5eb89c22ac292d99479055ae3d43a813219c6939b2f60660b7ee0bf8ae6d"} Dec 03 12:52:12 crc kubenswrapper[5052]: I1203 12:52:12.129864 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"f3733be6-a76c-4c85-ad81-58ca3f3e3d2d","Type":"ContainerDied","Data":"59d9cca171d0e5f3ea0e246db187ae008de4e9af8261af120aa2d4a36e96f396"} Dec 03 12:52:12 crc kubenswrapper[5052]: I1203 12:52:12.129943 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="59d9cca171d0e5f3ea0e246db187ae008de4e9af8261af120aa2d4a36e96f396" Dec 03 12:52:12 crc kubenswrapper[5052]: I1203 12:52:12.136386 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-db-create-tp5g8"] Dec 03 12:52:12 crc kubenswrapper[5052]: I1203 12:52:12.197560 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:52:12 crc kubenswrapper[5052]: I1203 12:52:12.200010 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/4d9b46e0-ffbc-4a61-8b07-f84d5b630f5a-operator-scripts\") pod \"nova-api-7066-account-create-update-4xwgf\" (UID: \"4d9b46e0-ffbc-4a61-8b07-f84d5b630f5a\") " pod="openstack-kuttl-tests/nova-api-7066-account-create-update-4xwgf" Dec 03 12:52:12 crc kubenswrapper[5052]: I1203 12:52:12.200071 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-bl9s5\" (UniqueName: \"kubernetes.io/projected/a3142a17-c4e9-49a1-95f3-f7b725516c93-kube-api-access-bl9s5\") pod \"nova-api-db-create-xmgf4\" (UID: \"a3142a17-c4e9-49a1-95f3-f7b725516c93\") " pod="openstack-kuttl-tests/nova-api-db-create-xmgf4" Dec 03 12:52:12 crc kubenswrapper[5052]: I1203 12:52:12.200107 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/a3142a17-c4e9-49a1-95f3-f7b725516c93-operator-scripts\") pod \"nova-api-db-create-xmgf4\" (UID: \"a3142a17-c4e9-49a1-95f3-f7b725516c93\") " pod="openstack-kuttl-tests/nova-api-db-create-xmgf4" Dec 03 12:52:12 crc kubenswrapper[5052]: I1203 12:52:12.200201 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/e26fce77-6037-4d8f-bb16-f06cb151fdef-operator-scripts\") pod \"nova-cell0-db-create-nvztf\" (UID: \"e26fce77-6037-4d8f-bb16-f06cb151fdef\") " pod="openstack-kuttl-tests/nova-cell0-db-create-nvztf" Dec 03 12:52:12 crc kubenswrapper[5052]: I1203 12:52:12.200239 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s6pbf\" (UniqueName: \"kubernetes.io/projected/e26fce77-6037-4d8f-bb16-f06cb151fdef-kube-api-access-s6pbf\") pod \"nova-cell0-db-create-nvztf\" (UID: \"e26fce77-6037-4d8f-bb16-f06cb151fdef\") " pod="openstack-kuttl-tests/nova-cell0-db-create-nvztf" Dec 03 12:52:12 crc kubenswrapper[5052]: I1203 12:52:12.200266 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p6jmn\" (UniqueName: \"kubernetes.io/projected/4d9b46e0-ffbc-4a61-8b07-f84d5b630f5a-kube-api-access-p6jmn\") pod \"nova-api-7066-account-create-update-4xwgf\" (UID: \"4d9b46e0-ffbc-4a61-8b07-f84d5b630f5a\") " pod="openstack-kuttl-tests/nova-api-7066-account-create-update-4xwgf" Dec 03 12:52:12 crc kubenswrapper[5052]: I1203 12:52:12.201478 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/a3142a17-c4e9-49a1-95f3-f7b725516c93-operator-scripts\") pod \"nova-api-db-create-xmgf4\" (UID: \"a3142a17-c4e9-49a1-95f3-f7b725516c93\") " pod="openstack-kuttl-tests/nova-api-db-create-xmgf4" Dec 03 12:52:12 crc kubenswrapper[5052]: I1203 12:52:12.209580 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell0-2629-account-create-update-c78fj"] Dec 03 12:52:12 crc kubenswrapper[5052]: E1203 12:52:12.210010 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="f3733be6-a76c-4c85-ad81-58ca3f3e3d2d" containerName="proxy-httpd" Dec 03 12:52:12 crc kubenswrapper[5052]: I1203 12:52:12.210024 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="f3733be6-a76c-4c85-ad81-58ca3f3e3d2d" containerName="proxy-httpd" Dec 03 12:52:12 crc kubenswrapper[5052]: E1203 12:52:12.210043 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="f3733be6-a76c-4c85-ad81-58ca3f3e3d2d" containerName="sg-core" Dec 03 12:52:12 crc kubenswrapper[5052]: I1203 12:52:12.210049 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="f3733be6-a76c-4c85-ad81-58ca3f3e3d2d" containerName="sg-core" Dec 03 12:52:12 crc kubenswrapper[5052]: E1203 12:52:12.210060 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="f3733be6-a76c-4c85-ad81-58ca3f3e3d2d" containerName="ceilometer-notification-agent" Dec 03 12:52:12 crc kubenswrapper[5052]: I1203 12:52:12.210065 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="f3733be6-a76c-4c85-ad81-58ca3f3e3d2d" containerName="ceilometer-notification-agent" Dec 03 12:52:12 crc kubenswrapper[5052]: E1203 12:52:12.210088 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="f3733be6-a76c-4c85-ad81-58ca3f3e3d2d" containerName="ceilometer-central-agent" Dec 03 12:52:12 crc kubenswrapper[5052]: I1203 12:52:12.210094 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="f3733be6-a76c-4c85-ad81-58ca3f3e3d2d" containerName="ceilometer-central-agent" Dec 03 12:52:12 crc kubenswrapper[5052]: I1203 12:52:12.210270 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="f3733be6-a76c-4c85-ad81-58ca3f3e3d2d" containerName="ceilometer-central-agent" Dec 03 12:52:12 crc kubenswrapper[5052]: I1203 12:52:12.210290 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="f3733be6-a76c-4c85-ad81-58ca3f3e3d2d" containerName="sg-core" Dec 03 12:52:12 crc kubenswrapper[5052]: I1203 12:52:12.210301 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="f3733be6-a76c-4c85-ad81-58ca3f3e3d2d" containerName="proxy-httpd" Dec 03 12:52:12 crc kubenswrapper[5052]: I1203 12:52:12.210311 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="f3733be6-a76c-4c85-ad81-58ca3f3e3d2d" containerName="ceilometer-notification-agent" Dec 03 12:52:12 crc kubenswrapper[5052]: I1203 12:52:12.210920 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-2629-account-create-update-c78fj" Dec 03 12:52:12 crc kubenswrapper[5052]: I1203 12:52:12.212875 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell0-db-secret" Dec 03 12:52:12 crc kubenswrapper[5052]: I1203 12:52:12.229965 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell0-2629-account-create-update-c78fj"] Dec 03 12:52:12 crc kubenswrapper[5052]: I1203 12:52:12.241959 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-bl9s5\" (UniqueName: \"kubernetes.io/projected/a3142a17-c4e9-49a1-95f3-f7b725516c93-kube-api-access-bl9s5\") pod \"nova-api-db-create-xmgf4\" (UID: \"a3142a17-c4e9-49a1-95f3-f7b725516c93\") " pod="openstack-kuttl-tests/nova-api-db-create-xmgf4" Dec 03 12:52:12 crc kubenswrapper[5052]: I1203 12:52:12.301048 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dcndt\" (UniqueName: \"kubernetes.io/projected/f3733be6-a76c-4c85-ad81-58ca3f3e3d2d-kube-api-access-dcndt\") pod \"f3733be6-a76c-4c85-ad81-58ca3f3e3d2d\" (UID: \"f3733be6-a76c-4c85-ad81-58ca3f3e3d2d\") " Dec 03 12:52:12 crc kubenswrapper[5052]: I1203 12:52:12.301160 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/f3733be6-a76c-4c85-ad81-58ca3f3e3d2d-sg-core-conf-yaml\") pod \"f3733be6-a76c-4c85-ad81-58ca3f3e3d2d\" (UID: \"f3733be6-a76c-4c85-ad81-58ca3f3e3d2d\") " Dec 03 12:52:12 crc kubenswrapper[5052]: I1203 12:52:12.301215 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/f3733be6-a76c-4c85-ad81-58ca3f3e3d2d-scripts\") pod \"f3733be6-a76c-4c85-ad81-58ca3f3e3d2d\" (UID: \"f3733be6-a76c-4c85-ad81-58ca3f3e3d2d\") " Dec 03 12:52:12 crc kubenswrapper[5052]: I1203 12:52:12.301234 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/f3733be6-a76c-4c85-ad81-58ca3f3e3d2d-run-httpd\") pod \"f3733be6-a76c-4c85-ad81-58ca3f3e3d2d\" (UID: \"f3733be6-a76c-4c85-ad81-58ca3f3e3d2d\") " Dec 03 12:52:12 crc kubenswrapper[5052]: I1203 12:52:12.301256 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f3733be6-a76c-4c85-ad81-58ca3f3e3d2d-combined-ca-bundle\") pod \"f3733be6-a76c-4c85-ad81-58ca3f3e3d2d\" (UID: \"f3733be6-a76c-4c85-ad81-58ca3f3e3d2d\") " Dec 03 12:52:12 crc kubenswrapper[5052]: I1203 12:52:12.301279 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/f3733be6-a76c-4c85-ad81-58ca3f3e3d2d-config-data\") pod \"f3733be6-a76c-4c85-ad81-58ca3f3e3d2d\" (UID: \"f3733be6-a76c-4c85-ad81-58ca3f3e3d2d\") " Dec 03 12:52:12 crc kubenswrapper[5052]: I1203 12:52:12.301329 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/f3733be6-a76c-4c85-ad81-58ca3f3e3d2d-log-httpd\") pod \"f3733be6-a76c-4c85-ad81-58ca3f3e3d2d\" (UID: \"f3733be6-a76c-4c85-ad81-58ca3f3e3d2d\") " Dec 03 12:52:12 crc kubenswrapper[5052]: I1203 12:52:12.301725 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/e26fce77-6037-4d8f-bb16-f06cb151fdef-operator-scripts\") pod \"nova-cell0-db-create-nvztf\" (UID: \"e26fce77-6037-4d8f-bb16-f06cb151fdef\") " pod="openstack-kuttl-tests/nova-cell0-db-create-nvztf" Dec 03 12:52:12 crc kubenswrapper[5052]: I1203 12:52:12.301773 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-s6pbf\" (UniqueName: \"kubernetes.io/projected/e26fce77-6037-4d8f-bb16-f06cb151fdef-kube-api-access-s6pbf\") pod \"nova-cell0-db-create-nvztf\" (UID: \"e26fce77-6037-4d8f-bb16-f06cb151fdef\") " pod="openstack-kuttl-tests/nova-cell0-db-create-nvztf" Dec 03 12:52:12 crc kubenswrapper[5052]: I1203 12:52:12.301798 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-p6jmn\" (UniqueName: \"kubernetes.io/projected/4d9b46e0-ffbc-4a61-8b07-f84d5b630f5a-kube-api-access-p6jmn\") pod \"nova-api-7066-account-create-update-4xwgf\" (UID: \"4d9b46e0-ffbc-4a61-8b07-f84d5b630f5a\") " pod="openstack-kuttl-tests/nova-api-7066-account-create-update-4xwgf" Dec 03 12:52:12 crc kubenswrapper[5052]: I1203 12:52:12.301830 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/2ff4a526-fb1e-48e9-ae88-6b84ce5eaba2-operator-scripts\") pod \"nova-cell1-db-create-tp5g8\" (UID: \"2ff4a526-fb1e-48e9-ae88-6b84ce5eaba2\") " pod="openstack-kuttl-tests/nova-cell1-db-create-tp5g8" Dec 03 12:52:12 crc kubenswrapper[5052]: I1203 12:52:12.301821 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f3733be6-a76c-4c85-ad81-58ca3f3e3d2d-run-httpd" (OuterVolumeSpecName: "run-httpd") pod "f3733be6-a76c-4c85-ad81-58ca3f3e3d2d" (UID: "f3733be6-a76c-4c85-ad81-58ca3f3e3d2d"). InnerVolumeSpecName "run-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:52:12 crc kubenswrapper[5052]: I1203 12:52:12.301968 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-svd4h\" (UniqueName: \"kubernetes.io/projected/2ff4a526-fb1e-48e9-ae88-6b84ce5eaba2-kube-api-access-svd4h\") pod \"nova-cell1-db-create-tp5g8\" (UID: \"2ff4a526-fb1e-48e9-ae88-6b84ce5eaba2\") " pod="openstack-kuttl-tests/nova-cell1-db-create-tp5g8" Dec 03 12:52:12 crc kubenswrapper[5052]: I1203 12:52:12.302112 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/4d9b46e0-ffbc-4a61-8b07-f84d5b630f5a-operator-scripts\") pod \"nova-api-7066-account-create-update-4xwgf\" (UID: \"4d9b46e0-ffbc-4a61-8b07-f84d5b630f5a\") " pod="openstack-kuttl-tests/nova-api-7066-account-create-update-4xwgf" Dec 03 12:52:12 crc kubenswrapper[5052]: I1203 12:52:12.302265 5052 reconciler_common.go:293] "Volume detached for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/f3733be6-a76c-4c85-ad81-58ca3f3e3d2d-run-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:52:12 crc kubenswrapper[5052]: I1203 12:52:12.302738 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f3733be6-a76c-4c85-ad81-58ca3f3e3d2d-log-httpd" (OuterVolumeSpecName: "log-httpd") pod "f3733be6-a76c-4c85-ad81-58ca3f3e3d2d" (UID: "f3733be6-a76c-4c85-ad81-58ca3f3e3d2d"). InnerVolumeSpecName "log-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:52:12 crc kubenswrapper[5052]: I1203 12:52:12.302964 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/4d9b46e0-ffbc-4a61-8b07-f84d5b630f5a-operator-scripts\") pod \"nova-api-7066-account-create-update-4xwgf\" (UID: \"4d9b46e0-ffbc-4a61-8b07-f84d5b630f5a\") " pod="openstack-kuttl-tests/nova-api-7066-account-create-update-4xwgf" Dec 03 12:52:12 crc kubenswrapper[5052]: I1203 12:52:12.303130 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/e26fce77-6037-4d8f-bb16-f06cb151fdef-operator-scripts\") pod \"nova-cell0-db-create-nvztf\" (UID: \"e26fce77-6037-4d8f-bb16-f06cb151fdef\") " pod="openstack-kuttl-tests/nova-cell0-db-create-nvztf" Dec 03 12:52:12 crc kubenswrapper[5052]: I1203 12:52:12.319346 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f3733be6-a76c-4c85-ad81-58ca3f3e3d2d-scripts" (OuterVolumeSpecName: "scripts") pod "f3733be6-a76c-4c85-ad81-58ca3f3e3d2d" (UID: "f3733be6-a76c-4c85-ad81-58ca3f3e3d2d"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:52:12 crc kubenswrapper[5052]: I1203 12:52:12.322592 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f3733be6-a76c-4c85-ad81-58ca3f3e3d2d-kube-api-access-dcndt" (OuterVolumeSpecName: "kube-api-access-dcndt") pod "f3733be6-a76c-4c85-ad81-58ca3f3e3d2d" (UID: "f3733be6-a76c-4c85-ad81-58ca3f3e3d2d"). InnerVolumeSpecName "kube-api-access-dcndt". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:52:12 crc kubenswrapper[5052]: I1203 12:52:12.323105 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-p6jmn\" (UniqueName: \"kubernetes.io/projected/4d9b46e0-ffbc-4a61-8b07-f84d5b630f5a-kube-api-access-p6jmn\") pod \"nova-api-7066-account-create-update-4xwgf\" (UID: \"4d9b46e0-ffbc-4a61-8b07-f84d5b630f5a\") " pod="openstack-kuttl-tests/nova-api-7066-account-create-update-4xwgf" Dec 03 12:52:12 crc kubenswrapper[5052]: I1203 12:52:12.324922 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-s6pbf\" (UniqueName: \"kubernetes.io/projected/e26fce77-6037-4d8f-bb16-f06cb151fdef-kube-api-access-s6pbf\") pod \"nova-cell0-db-create-nvztf\" (UID: \"e26fce77-6037-4d8f-bb16-f06cb151fdef\") " pod="openstack-kuttl-tests/nova-cell0-db-create-nvztf" Dec 03 12:52:12 crc kubenswrapper[5052]: I1203 12:52:12.329786 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-db-create-nvztf" Dec 03 12:52:12 crc kubenswrapper[5052]: I1203 12:52:12.335580 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f3733be6-a76c-4c85-ad81-58ca3f3e3d2d-sg-core-conf-yaml" (OuterVolumeSpecName: "sg-core-conf-yaml") pod "f3733be6-a76c-4c85-ad81-58ca3f3e3d2d" (UID: "f3733be6-a76c-4c85-ad81-58ca3f3e3d2d"). InnerVolumeSpecName "sg-core-conf-yaml". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:52:12 crc kubenswrapper[5052]: I1203 12:52:12.347075 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-7066-account-create-update-4xwgf" Dec 03 12:52:12 crc kubenswrapper[5052]: I1203 12:52:12.404482 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/2ff4a526-fb1e-48e9-ae88-6b84ce5eaba2-operator-scripts\") pod \"nova-cell1-db-create-tp5g8\" (UID: \"2ff4a526-fb1e-48e9-ae88-6b84ce5eaba2\") " pod="openstack-kuttl-tests/nova-cell1-db-create-tp5g8" Dec 03 12:52:12 crc kubenswrapper[5052]: I1203 12:52:12.404577 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-svd4h\" (UniqueName: \"kubernetes.io/projected/2ff4a526-fb1e-48e9-ae88-6b84ce5eaba2-kube-api-access-svd4h\") pod \"nova-cell1-db-create-tp5g8\" (UID: \"2ff4a526-fb1e-48e9-ae88-6b84ce5eaba2\") " pod="openstack-kuttl-tests/nova-cell1-db-create-tp5g8" Dec 03 12:52:12 crc kubenswrapper[5052]: I1203 12:52:12.404645 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-66qg4\" (UniqueName: \"kubernetes.io/projected/38bf6fea-894f-4f1c-841b-374540c44a38-kube-api-access-66qg4\") pod \"nova-cell0-2629-account-create-update-c78fj\" (UID: \"38bf6fea-894f-4f1c-841b-374540c44a38\") " pod="openstack-kuttl-tests/nova-cell0-2629-account-create-update-c78fj" Dec 03 12:52:12 crc kubenswrapper[5052]: I1203 12:52:12.404718 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/38bf6fea-894f-4f1c-841b-374540c44a38-operator-scripts\") pod \"nova-cell0-2629-account-create-update-c78fj\" (UID: \"38bf6fea-894f-4f1c-841b-374540c44a38\") " pod="openstack-kuttl-tests/nova-cell0-2629-account-create-update-c78fj" Dec 03 12:52:12 crc kubenswrapper[5052]: I1203 12:52:12.404811 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-dcndt\" (UniqueName: \"kubernetes.io/projected/f3733be6-a76c-4c85-ad81-58ca3f3e3d2d-kube-api-access-dcndt\") on node \"crc\" DevicePath \"\"" Dec 03 12:52:12 crc kubenswrapper[5052]: I1203 12:52:12.404828 5052 reconciler_common.go:293] "Volume detached for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/f3733be6-a76c-4c85-ad81-58ca3f3e3d2d-sg-core-conf-yaml\") on node \"crc\" DevicePath \"\"" Dec 03 12:52:12 crc kubenswrapper[5052]: I1203 12:52:12.404840 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/f3733be6-a76c-4c85-ad81-58ca3f3e3d2d-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:52:12 crc kubenswrapper[5052]: I1203 12:52:12.404854 5052 reconciler_common.go:293] "Volume detached for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/f3733be6-a76c-4c85-ad81-58ca3f3e3d2d-log-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:52:12 crc kubenswrapper[5052]: I1203 12:52:12.407109 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f3733be6-a76c-4c85-ad81-58ca3f3e3d2d-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "f3733be6-a76c-4c85-ad81-58ca3f3e3d2d" (UID: "f3733be6-a76c-4c85-ad81-58ca3f3e3d2d"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:52:12 crc kubenswrapper[5052]: I1203 12:52:12.407408 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/2ff4a526-fb1e-48e9-ae88-6b84ce5eaba2-operator-scripts\") pod \"nova-cell1-db-create-tp5g8\" (UID: \"2ff4a526-fb1e-48e9-ae88-6b84ce5eaba2\") " pod="openstack-kuttl-tests/nova-cell1-db-create-tp5g8" Dec 03 12:52:12 crc kubenswrapper[5052]: I1203 12:52:12.418232 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell1-0a46-account-create-update-4n2r8"] Dec 03 12:52:12 crc kubenswrapper[5052]: I1203 12:52:12.419902 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-0a46-account-create-update-4n2r8" Dec 03 12:52:12 crc kubenswrapper[5052]: I1203 12:52:12.421994 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell1-db-secret" Dec 03 12:52:12 crc kubenswrapper[5052]: I1203 12:52:12.437209 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-svd4h\" (UniqueName: \"kubernetes.io/projected/2ff4a526-fb1e-48e9-ae88-6b84ce5eaba2-kube-api-access-svd4h\") pod \"nova-cell1-db-create-tp5g8\" (UID: \"2ff4a526-fb1e-48e9-ae88-6b84ce5eaba2\") " pod="openstack-kuttl-tests/nova-cell1-db-create-tp5g8" Dec 03 12:52:12 crc kubenswrapper[5052]: I1203 12:52:12.464740 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-0a46-account-create-update-4n2r8"] Dec 03 12:52:12 crc kubenswrapper[5052]: I1203 12:52:12.507902 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-66qg4\" (UniqueName: \"kubernetes.io/projected/38bf6fea-894f-4f1c-841b-374540c44a38-kube-api-access-66qg4\") pod \"nova-cell0-2629-account-create-update-c78fj\" (UID: \"38bf6fea-894f-4f1c-841b-374540c44a38\") " pod="openstack-kuttl-tests/nova-cell0-2629-account-create-update-c78fj" Dec 03 12:52:12 crc kubenswrapper[5052]: I1203 12:52:12.507985 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/38bf6fea-894f-4f1c-841b-374540c44a38-operator-scripts\") pod \"nova-cell0-2629-account-create-update-c78fj\" (UID: \"38bf6fea-894f-4f1c-841b-374540c44a38\") " pod="openstack-kuttl-tests/nova-cell0-2629-account-create-update-c78fj" Dec 03 12:52:12 crc kubenswrapper[5052]: I1203 12:52:12.508054 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f3733be6-a76c-4c85-ad81-58ca3f3e3d2d-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:52:12 crc kubenswrapper[5052]: I1203 12:52:12.508745 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/38bf6fea-894f-4f1c-841b-374540c44a38-operator-scripts\") pod \"nova-cell0-2629-account-create-update-c78fj\" (UID: \"38bf6fea-894f-4f1c-841b-374540c44a38\") " pod="openstack-kuttl-tests/nova-cell0-2629-account-create-update-c78fj" Dec 03 12:52:12 crc kubenswrapper[5052]: I1203 12:52:12.513730 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-db-create-tp5g8" Dec 03 12:52:12 crc kubenswrapper[5052]: I1203 12:52:12.531109 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f3733be6-a76c-4c85-ad81-58ca3f3e3d2d-config-data" (OuterVolumeSpecName: "config-data") pod "f3733be6-a76c-4c85-ad81-58ca3f3e3d2d" (UID: "f3733be6-a76c-4c85-ad81-58ca3f3e3d2d"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:52:12 crc kubenswrapper[5052]: I1203 12:52:12.532085 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-66qg4\" (UniqueName: \"kubernetes.io/projected/38bf6fea-894f-4f1c-841b-374540c44a38-kube-api-access-66qg4\") pod \"nova-cell0-2629-account-create-update-c78fj\" (UID: \"38bf6fea-894f-4f1c-841b-374540c44a38\") " pod="openstack-kuttl-tests/nova-cell0-2629-account-create-update-c78fj" Dec 03 12:52:12 crc kubenswrapper[5052]: I1203 12:52:12.535808 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-db-create-xmgf4" Dec 03 12:52:12 crc kubenswrapper[5052]: I1203 12:52:12.537315 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-2629-account-create-update-c78fj" Dec 03 12:52:12 crc kubenswrapper[5052]: I1203 12:52:12.610137 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/69d15184-ac62-4679-aadf-61fd17294318-operator-scripts\") pod \"nova-cell1-0a46-account-create-update-4n2r8\" (UID: \"69d15184-ac62-4679-aadf-61fd17294318\") " pod="openstack-kuttl-tests/nova-cell1-0a46-account-create-update-4n2r8" Dec 03 12:52:12 crc kubenswrapper[5052]: I1203 12:52:12.610235 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xtqn7\" (UniqueName: \"kubernetes.io/projected/69d15184-ac62-4679-aadf-61fd17294318-kube-api-access-xtqn7\") pod \"nova-cell1-0a46-account-create-update-4n2r8\" (UID: \"69d15184-ac62-4679-aadf-61fd17294318\") " pod="openstack-kuttl-tests/nova-cell1-0a46-account-create-update-4n2r8" Dec 03 12:52:12 crc kubenswrapper[5052]: I1203 12:52:12.610313 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/f3733be6-a76c-4c85-ad81-58ca3f3e3d2d-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:52:12 crc kubenswrapper[5052]: I1203 12:52:12.712150 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/69d15184-ac62-4679-aadf-61fd17294318-operator-scripts\") pod \"nova-cell1-0a46-account-create-update-4n2r8\" (UID: \"69d15184-ac62-4679-aadf-61fd17294318\") " pod="openstack-kuttl-tests/nova-cell1-0a46-account-create-update-4n2r8" Dec 03 12:52:12 crc kubenswrapper[5052]: I1203 12:52:12.712613 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-xtqn7\" (UniqueName: \"kubernetes.io/projected/69d15184-ac62-4679-aadf-61fd17294318-kube-api-access-xtqn7\") pod \"nova-cell1-0a46-account-create-update-4n2r8\" (UID: \"69d15184-ac62-4679-aadf-61fd17294318\") " pod="openstack-kuttl-tests/nova-cell1-0a46-account-create-update-4n2r8" Dec 03 12:52:12 crc kubenswrapper[5052]: I1203 12:52:12.712944 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/69d15184-ac62-4679-aadf-61fd17294318-operator-scripts\") pod \"nova-cell1-0a46-account-create-update-4n2r8\" (UID: \"69d15184-ac62-4679-aadf-61fd17294318\") " pod="openstack-kuttl-tests/nova-cell1-0a46-account-create-update-4n2r8" Dec 03 12:52:12 crc kubenswrapper[5052]: I1203 12:52:12.745578 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-xtqn7\" (UniqueName: \"kubernetes.io/projected/69d15184-ac62-4679-aadf-61fd17294318-kube-api-access-xtqn7\") pod \"nova-cell1-0a46-account-create-update-4n2r8\" (UID: \"69d15184-ac62-4679-aadf-61fd17294318\") " pod="openstack-kuttl-tests/nova-cell1-0a46-account-create-update-4n2r8" Dec 03 12:52:12 crc kubenswrapper[5052]: I1203 12:52:12.761695 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-0a46-account-create-update-4n2r8" Dec 03 12:52:12 crc kubenswrapper[5052]: I1203 12:52:12.891216 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell0-db-create-nvztf"] Dec 03 12:52:12 crc kubenswrapper[5052]: I1203 12:52:12.897752 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/swift-proxy-5479cb94c8-q2snc" Dec 03 12:52:12 crc kubenswrapper[5052]: I1203 12:52:12.899193 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/swift-proxy-5479cb94c8-q2snc" Dec 03 12:52:12 crc kubenswrapper[5052]: I1203 12:52:12.906056 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-7066-account-create-update-4xwgf"] Dec 03 12:52:12 crc kubenswrapper[5052]: W1203 12:52:12.948469 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4d9b46e0_ffbc_4a61_8b07_f84d5b630f5a.slice/crio-6e53f88812f5f9916e69d41d02e253e0924cfb89d78d92ecc7d5745e82af1129 WatchSource:0}: Error finding container 6e53f88812f5f9916e69d41d02e253e0924cfb89d78d92ecc7d5745e82af1129: Status 404 returned error can't find the container with id 6e53f88812f5f9916e69d41d02e253e0924cfb89d78d92ecc7d5745e82af1129 Dec 03 12:52:13 crc kubenswrapper[5052]: I1203 12:52:13.078883 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-db-create-tp5g8"] Dec 03 12:52:13 crc kubenswrapper[5052]: W1203 12:52:13.103648 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2ff4a526_fb1e_48e9_ae88_6b84ce5eaba2.slice/crio-b633210ccd963782f79ca4056e0b24683556127758e280f24de54630567dbe4a WatchSource:0}: Error finding container b633210ccd963782f79ca4056e0b24683556127758e280f24de54630567dbe4a: Status 404 returned error can't find the container with id b633210ccd963782f79ca4056e0b24683556127758e280f24de54630567dbe4a Dec 03 12:52:13 crc kubenswrapper[5052]: I1203 12:52:13.143440 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-db-create-tp5g8" event={"ID":"2ff4a526-fb1e-48e9-ae88-6b84ce5eaba2","Type":"ContainerStarted","Data":"b633210ccd963782f79ca4056e0b24683556127758e280f24de54630567dbe4a"} Dec 03 12:52:13 crc kubenswrapper[5052]: I1203 12:52:13.144531 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-db-create-nvztf" event={"ID":"e26fce77-6037-4d8f-bb16-f06cb151fdef","Type":"ContainerStarted","Data":"1fe6a36d5702f1d96c201f260f8be2f8bd2e9e9d1f1a9ff3b89ebbbfd3ddb8f7"} Dec 03 12:52:13 crc kubenswrapper[5052]: I1203 12:52:13.146988 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-7066-account-create-update-4xwgf" event={"ID":"4d9b46e0-ffbc-4a61-8b07-f84d5b630f5a","Type":"ContainerStarted","Data":"6e53f88812f5f9916e69d41d02e253e0924cfb89d78d92ecc7d5745e82af1129"} Dec 03 12:52:13 crc kubenswrapper[5052]: I1203 12:52:13.147082 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:52:13 crc kubenswrapper[5052]: I1203 12:52:13.179804 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell0-2629-account-create-update-c78fj"] Dec 03 12:52:13 crc kubenswrapper[5052]: W1203 12:52:13.205722 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod38bf6fea_894f_4f1c_841b_374540c44a38.slice/crio-1449b24b0e1393996115428adad6bb23fe3cf6ce02755f1ffd75a5d41aa08d56 WatchSource:0}: Error finding container 1449b24b0e1393996115428adad6bb23fe3cf6ce02755f1ffd75a5d41aa08d56: Status 404 returned error can't find the container with id 1449b24b0e1393996115428adad6bb23fe3cf6ce02755f1ffd75a5d41aa08d56 Dec 03 12:52:13 crc kubenswrapper[5052]: I1203 12:52:13.216322 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:52:13 crc kubenswrapper[5052]: I1203 12:52:13.252271 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:52:13 crc kubenswrapper[5052]: I1203 12:52:13.275093 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-db-create-xmgf4"] Dec 03 12:52:13 crc kubenswrapper[5052]: I1203 12:52:13.287080 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:52:13 crc kubenswrapper[5052]: I1203 12:52:13.291018 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:52:13 crc kubenswrapper[5052]: I1203 12:52:13.296075 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-config-data" Dec 03 12:52:13 crc kubenswrapper[5052]: I1203 12:52:13.296247 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-scripts" Dec 03 12:52:13 crc kubenswrapper[5052]: I1203 12:52:13.298842 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:52:13 crc kubenswrapper[5052]: I1203 12:52:13.367101 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-0a46-account-create-update-4n2r8"] Dec 03 12:52:13 crc kubenswrapper[5052]: W1203 12:52:13.378892 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod69d15184_ac62_4679_aadf_61fd17294318.slice/crio-bfd79b797e67a642bf41fac48f3b919ba3a1f5efd9f87128842a33d6f21be6de WatchSource:0}: Error finding container bfd79b797e67a642bf41fac48f3b919ba3a1f5efd9f87128842a33d6f21be6de: Status 404 returned error can't find the container with id bfd79b797e67a642bf41fac48f3b919ba3a1f5efd9f87128842a33d6f21be6de Dec 03 12:52:13 crc kubenswrapper[5052]: I1203 12:52:13.432414 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-65dkq\" (UniqueName: \"kubernetes.io/projected/186b516e-0d01-4c65-8538-94d9f7d30df3-kube-api-access-65dkq\") pod \"ceilometer-0\" (UID: \"186b516e-0d01-4c65-8538-94d9f7d30df3\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:52:13 crc kubenswrapper[5052]: I1203 12:52:13.432479 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/186b516e-0d01-4c65-8538-94d9f7d30df3-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"186b516e-0d01-4c65-8538-94d9f7d30df3\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:52:13 crc kubenswrapper[5052]: I1203 12:52:13.432505 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/186b516e-0d01-4c65-8538-94d9f7d30df3-scripts\") pod \"ceilometer-0\" (UID: \"186b516e-0d01-4c65-8538-94d9f7d30df3\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:52:13 crc kubenswrapper[5052]: I1203 12:52:13.432568 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/186b516e-0d01-4c65-8538-94d9f7d30df3-log-httpd\") pod \"ceilometer-0\" (UID: \"186b516e-0d01-4c65-8538-94d9f7d30df3\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:52:13 crc kubenswrapper[5052]: I1203 12:52:13.432623 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/186b516e-0d01-4c65-8538-94d9f7d30df3-config-data\") pod \"ceilometer-0\" (UID: \"186b516e-0d01-4c65-8538-94d9f7d30df3\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:52:13 crc kubenswrapper[5052]: I1203 12:52:13.432648 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/186b516e-0d01-4c65-8538-94d9f7d30df3-run-httpd\") pod \"ceilometer-0\" (UID: \"186b516e-0d01-4c65-8538-94d9f7d30df3\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:52:13 crc kubenswrapper[5052]: I1203 12:52:13.432711 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/186b516e-0d01-4c65-8538-94d9f7d30df3-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"186b516e-0d01-4c65-8538-94d9f7d30df3\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:52:13 crc kubenswrapper[5052]: I1203 12:52:13.534084 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-65dkq\" (UniqueName: \"kubernetes.io/projected/186b516e-0d01-4c65-8538-94d9f7d30df3-kube-api-access-65dkq\") pod \"ceilometer-0\" (UID: \"186b516e-0d01-4c65-8538-94d9f7d30df3\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:52:13 crc kubenswrapper[5052]: I1203 12:52:13.534136 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/186b516e-0d01-4c65-8538-94d9f7d30df3-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"186b516e-0d01-4c65-8538-94d9f7d30df3\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:52:13 crc kubenswrapper[5052]: I1203 12:52:13.534154 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/186b516e-0d01-4c65-8538-94d9f7d30df3-scripts\") pod \"ceilometer-0\" (UID: \"186b516e-0d01-4c65-8538-94d9f7d30df3\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:52:13 crc kubenswrapper[5052]: I1203 12:52:13.534191 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/186b516e-0d01-4c65-8538-94d9f7d30df3-log-httpd\") pod \"ceilometer-0\" (UID: \"186b516e-0d01-4c65-8538-94d9f7d30df3\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:52:13 crc kubenswrapper[5052]: I1203 12:52:13.534215 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/186b516e-0d01-4c65-8538-94d9f7d30df3-config-data\") pod \"ceilometer-0\" (UID: \"186b516e-0d01-4c65-8538-94d9f7d30df3\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:52:13 crc kubenswrapper[5052]: I1203 12:52:13.534232 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/186b516e-0d01-4c65-8538-94d9f7d30df3-run-httpd\") pod \"ceilometer-0\" (UID: \"186b516e-0d01-4c65-8538-94d9f7d30df3\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:52:13 crc kubenswrapper[5052]: I1203 12:52:13.534284 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/186b516e-0d01-4c65-8538-94d9f7d30df3-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"186b516e-0d01-4c65-8538-94d9f7d30df3\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:52:13 crc kubenswrapper[5052]: I1203 12:52:13.535249 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/186b516e-0d01-4c65-8538-94d9f7d30df3-log-httpd\") pod \"ceilometer-0\" (UID: \"186b516e-0d01-4c65-8538-94d9f7d30df3\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:52:13 crc kubenswrapper[5052]: I1203 12:52:13.535546 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/186b516e-0d01-4c65-8538-94d9f7d30df3-run-httpd\") pod \"ceilometer-0\" (UID: \"186b516e-0d01-4c65-8538-94d9f7d30df3\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:52:13 crc kubenswrapper[5052]: I1203 12:52:13.541034 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/186b516e-0d01-4c65-8538-94d9f7d30df3-scripts\") pod \"ceilometer-0\" (UID: \"186b516e-0d01-4c65-8538-94d9f7d30df3\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:52:13 crc kubenswrapper[5052]: I1203 12:52:13.541639 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/186b516e-0d01-4c65-8538-94d9f7d30df3-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"186b516e-0d01-4c65-8538-94d9f7d30df3\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:52:13 crc kubenswrapper[5052]: I1203 12:52:13.546020 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/186b516e-0d01-4c65-8538-94d9f7d30df3-config-data\") pod \"ceilometer-0\" (UID: \"186b516e-0d01-4c65-8538-94d9f7d30df3\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:52:13 crc kubenswrapper[5052]: I1203 12:52:13.551090 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-65dkq\" (UniqueName: \"kubernetes.io/projected/186b516e-0d01-4c65-8538-94d9f7d30df3-kube-api-access-65dkq\") pod \"ceilometer-0\" (UID: \"186b516e-0d01-4c65-8538-94d9f7d30df3\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:52:13 crc kubenswrapper[5052]: I1203 12:52:13.551531 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/186b516e-0d01-4c65-8538-94d9f7d30df3-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"186b516e-0d01-4c65-8538-94d9f7d30df3\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:52:13 crc kubenswrapper[5052]: I1203 12:52:13.615296 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:52:13 crc kubenswrapper[5052]: I1203 12:52:13.687398 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f3733be6-a76c-4c85-ad81-58ca3f3e3d2d" path="/var/lib/kubelet/pods/f3733be6-a76c-4c85-ad81-58ca3f3e3d2d/volumes" Dec 03 12:52:14 crc kubenswrapper[5052]: I1203 12:52:14.171760 5052 generic.go:334] "Generic (PLEG): container finished" podID="4d9b46e0-ffbc-4a61-8b07-f84d5b630f5a" containerID="067b2bdc88fb8db62ca52315daa392d562dd49266ba0cde5f73a3a78796ed357" exitCode=0 Dec 03 12:52:14 crc kubenswrapper[5052]: I1203 12:52:14.172128 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-7066-account-create-update-4xwgf" event={"ID":"4d9b46e0-ffbc-4a61-8b07-f84d5b630f5a","Type":"ContainerDied","Data":"067b2bdc88fb8db62ca52315daa392d562dd49266ba0cde5f73a3a78796ed357"} Dec 03 12:52:14 crc kubenswrapper[5052]: I1203 12:52:14.175283 5052 generic.go:334] "Generic (PLEG): container finished" podID="69d15184-ac62-4679-aadf-61fd17294318" containerID="6dd5b4c49e1b08f4974bb1ad78b09649b89f7ce5636ef4999c8c4847ad475556" exitCode=0 Dec 03 12:52:14 crc kubenswrapper[5052]: I1203 12:52:14.175346 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-0a46-account-create-update-4n2r8" event={"ID":"69d15184-ac62-4679-aadf-61fd17294318","Type":"ContainerDied","Data":"6dd5b4c49e1b08f4974bb1ad78b09649b89f7ce5636ef4999c8c4847ad475556"} Dec 03 12:52:14 crc kubenswrapper[5052]: I1203 12:52:14.175368 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-0a46-account-create-update-4n2r8" event={"ID":"69d15184-ac62-4679-aadf-61fd17294318","Type":"ContainerStarted","Data":"bfd79b797e67a642bf41fac48f3b919ba3a1f5efd9f87128842a33d6f21be6de"} Dec 03 12:52:14 crc kubenswrapper[5052]: I1203 12:52:14.176997 5052 generic.go:334] "Generic (PLEG): container finished" podID="38bf6fea-894f-4f1c-841b-374540c44a38" containerID="5e8290f1b75b23e41346e6776bb910c41c35f1e744bda7029a7a83d431241314" exitCode=0 Dec 03 12:52:14 crc kubenswrapper[5052]: I1203 12:52:14.177173 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-2629-account-create-update-c78fj" event={"ID":"38bf6fea-894f-4f1c-841b-374540c44a38","Type":"ContainerDied","Data":"5e8290f1b75b23e41346e6776bb910c41c35f1e744bda7029a7a83d431241314"} Dec 03 12:52:14 crc kubenswrapper[5052]: I1203 12:52:14.177229 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-2629-account-create-update-c78fj" event={"ID":"38bf6fea-894f-4f1c-841b-374540c44a38","Type":"ContainerStarted","Data":"1449b24b0e1393996115428adad6bb23fe3cf6ce02755f1ffd75a5d41aa08d56"} Dec 03 12:52:14 crc kubenswrapper[5052]: I1203 12:52:14.178379 5052 generic.go:334] "Generic (PLEG): container finished" podID="a3142a17-c4e9-49a1-95f3-f7b725516c93" containerID="8edec2df8b21527fbab0a86cfd43f0ada2120365035a681121d82c8e08fe2868" exitCode=0 Dec 03 12:52:14 crc kubenswrapper[5052]: I1203 12:52:14.178446 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-db-create-xmgf4" event={"ID":"a3142a17-c4e9-49a1-95f3-f7b725516c93","Type":"ContainerDied","Data":"8edec2df8b21527fbab0a86cfd43f0ada2120365035a681121d82c8e08fe2868"} Dec 03 12:52:14 crc kubenswrapper[5052]: I1203 12:52:14.178473 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-db-create-xmgf4" event={"ID":"a3142a17-c4e9-49a1-95f3-f7b725516c93","Type":"ContainerStarted","Data":"05fe440b1a8728bebae2e000d85874f1ee0039eb2f5ca51a66eeaceed998b8db"} Dec 03 12:52:14 crc kubenswrapper[5052]: I1203 12:52:14.180980 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:52:14 crc kubenswrapper[5052]: I1203 12:52:14.181500 5052 generic.go:334] "Generic (PLEG): container finished" podID="2ff4a526-fb1e-48e9-ae88-6b84ce5eaba2" containerID="6529ebafbdabe5682caf908893845491ea6c63023a1e09a075b791c88d910044" exitCode=0 Dec 03 12:52:14 crc kubenswrapper[5052]: I1203 12:52:14.181703 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-db-create-tp5g8" event={"ID":"2ff4a526-fb1e-48e9-ae88-6b84ce5eaba2","Type":"ContainerDied","Data":"6529ebafbdabe5682caf908893845491ea6c63023a1e09a075b791c88d910044"} Dec 03 12:52:14 crc kubenswrapper[5052]: I1203 12:52:14.184224 5052 generic.go:334] "Generic (PLEG): container finished" podID="e26fce77-6037-4d8f-bb16-f06cb151fdef" containerID="2af5f1d759ac9d0ae0920501c0ea568edd510c60defdbffba8b9de9b0b0ba71c" exitCode=0 Dec 03 12:52:14 crc kubenswrapper[5052]: I1203 12:52:14.184272 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-db-create-nvztf" event={"ID":"e26fce77-6037-4d8f-bb16-f06cb151fdef","Type":"ContainerDied","Data":"2af5f1d759ac9d0ae0920501c0ea568edd510c60defdbffba8b9de9b0b0ba71c"} Dec 03 12:52:14 crc kubenswrapper[5052]: W1203 12:52:14.197662 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod186b516e_0d01_4c65_8538_94d9f7d30df3.slice/crio-50c8e90c89f0255bf4a704fb25fba49af8b3c17101af68d6d2dc43e2fb344c0b WatchSource:0}: Error finding container 50c8e90c89f0255bf4a704fb25fba49af8b3c17101af68d6d2dc43e2fb344c0b: Status 404 returned error can't find the container with id 50c8e90c89f0255bf4a704fb25fba49af8b3c17101af68d6d2dc43e2fb344c0b Dec 03 12:52:14 crc kubenswrapper[5052]: I1203 12:52:14.885390 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/kube-state-metrics-0"] Dec 03 12:52:14 crc kubenswrapper[5052]: I1203 12:52:14.885611 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/kube-state-metrics-0" podUID="571d31d9-6750-41fe-8905-ea6de9cfb44d" containerName="kube-state-metrics" containerID="cri-o://dac5b0bc9b047df6eb67eb15e4e6c404248602afea4f49a377b09a6f2b2fe314" gracePeriod=30 Dec 03 12:52:15 crc kubenswrapper[5052]: I1203 12:52:15.199571 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"186b516e-0d01-4c65-8538-94d9f7d30df3","Type":"ContainerStarted","Data":"50c8e90c89f0255bf4a704fb25fba49af8b3c17101af68d6d2dc43e2fb344c0b"} Dec 03 12:52:15 crc kubenswrapper[5052]: I1203 12:52:15.201699 5052 generic.go:334] "Generic (PLEG): container finished" podID="571d31d9-6750-41fe-8905-ea6de9cfb44d" containerID="dac5b0bc9b047df6eb67eb15e4e6c404248602afea4f49a377b09a6f2b2fe314" exitCode=2 Dec 03 12:52:15 crc kubenswrapper[5052]: I1203 12:52:15.201950 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/kube-state-metrics-0" event={"ID":"571d31d9-6750-41fe-8905-ea6de9cfb44d","Type":"ContainerDied","Data":"dac5b0bc9b047df6eb67eb15e4e6c404248602afea4f49a377b09a6f2b2fe314"} Dec 03 12:52:15 crc kubenswrapper[5052]: I1203 12:52:15.666951 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-db-create-xmgf4" Dec 03 12:52:15 crc kubenswrapper[5052]: I1203 12:52:15.786612 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/a3142a17-c4e9-49a1-95f3-f7b725516c93-operator-scripts\") pod \"a3142a17-c4e9-49a1-95f3-f7b725516c93\" (UID: \"a3142a17-c4e9-49a1-95f3-f7b725516c93\") " Dec 03 12:52:15 crc kubenswrapper[5052]: I1203 12:52:15.786713 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bl9s5\" (UniqueName: \"kubernetes.io/projected/a3142a17-c4e9-49a1-95f3-f7b725516c93-kube-api-access-bl9s5\") pod \"a3142a17-c4e9-49a1-95f3-f7b725516c93\" (UID: \"a3142a17-c4e9-49a1-95f3-f7b725516c93\") " Dec 03 12:52:15 crc kubenswrapper[5052]: I1203 12:52:15.787469 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a3142a17-c4e9-49a1-95f3-f7b725516c93-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "a3142a17-c4e9-49a1-95f3-f7b725516c93" (UID: "a3142a17-c4e9-49a1-95f3-f7b725516c93"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:52:15 crc kubenswrapper[5052]: I1203 12:52:15.794837 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a3142a17-c4e9-49a1-95f3-f7b725516c93-kube-api-access-bl9s5" (OuterVolumeSpecName: "kube-api-access-bl9s5") pod "a3142a17-c4e9-49a1-95f3-f7b725516c93" (UID: "a3142a17-c4e9-49a1-95f3-f7b725516c93"). InnerVolumeSpecName "kube-api-access-bl9s5". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:52:15 crc kubenswrapper[5052]: I1203 12:52:15.889347 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/a3142a17-c4e9-49a1-95f3-f7b725516c93-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:52:15 crc kubenswrapper[5052]: I1203 12:52:15.889380 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-bl9s5\" (UniqueName: \"kubernetes.io/projected/a3142a17-c4e9-49a1-95f3-f7b725516c93-kube-api-access-bl9s5\") on node \"crc\" DevicePath \"\"" Dec 03 12:52:15 crc kubenswrapper[5052]: I1203 12:52:15.935346 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-2629-account-create-update-c78fj" Dec 03 12:52:15 crc kubenswrapper[5052]: I1203 12:52:15.954649 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-db-create-tp5g8" Dec 03 12:52:15 crc kubenswrapper[5052]: I1203 12:52:15.968417 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-7066-account-create-update-4xwgf" Dec 03 12:52:16 crc kubenswrapper[5052]: I1203 12:52:16.007739 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:52:16 crc kubenswrapper[5052]: I1203 12:52:16.011730 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-0a46-account-create-update-4n2r8" Dec 03 12:52:16 crc kubenswrapper[5052]: I1203 12:52:16.092161 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/69d15184-ac62-4679-aadf-61fd17294318-operator-scripts\") pod \"69d15184-ac62-4679-aadf-61fd17294318\" (UID: \"69d15184-ac62-4679-aadf-61fd17294318\") " Dec 03 12:52:16 crc kubenswrapper[5052]: I1203 12:52:16.092215 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-svd4h\" (UniqueName: \"kubernetes.io/projected/2ff4a526-fb1e-48e9-ae88-6b84ce5eaba2-kube-api-access-svd4h\") pod \"2ff4a526-fb1e-48e9-ae88-6b84ce5eaba2\" (UID: \"2ff4a526-fb1e-48e9-ae88-6b84ce5eaba2\") " Dec 03 12:52:16 crc kubenswrapper[5052]: I1203 12:52:16.092255 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-66qg4\" (UniqueName: \"kubernetes.io/projected/38bf6fea-894f-4f1c-841b-374540c44a38-kube-api-access-66qg4\") pod \"38bf6fea-894f-4f1c-841b-374540c44a38\" (UID: \"38bf6fea-894f-4f1c-841b-374540c44a38\") " Dec 03 12:52:16 crc kubenswrapper[5052]: I1203 12:52:16.092291 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kczb2\" (UniqueName: \"kubernetes.io/projected/571d31d9-6750-41fe-8905-ea6de9cfb44d-kube-api-access-kczb2\") pod \"571d31d9-6750-41fe-8905-ea6de9cfb44d\" (UID: \"571d31d9-6750-41fe-8905-ea6de9cfb44d\") " Dec 03 12:52:16 crc kubenswrapper[5052]: I1203 12:52:16.092341 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/2ff4a526-fb1e-48e9-ae88-6b84ce5eaba2-operator-scripts\") pod \"2ff4a526-fb1e-48e9-ae88-6b84ce5eaba2\" (UID: \"2ff4a526-fb1e-48e9-ae88-6b84ce5eaba2\") " Dec 03 12:52:16 crc kubenswrapper[5052]: I1203 12:52:16.092366 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p6jmn\" (UniqueName: \"kubernetes.io/projected/4d9b46e0-ffbc-4a61-8b07-f84d5b630f5a-kube-api-access-p6jmn\") pod \"4d9b46e0-ffbc-4a61-8b07-f84d5b630f5a\" (UID: \"4d9b46e0-ffbc-4a61-8b07-f84d5b630f5a\") " Dec 03 12:52:16 crc kubenswrapper[5052]: I1203 12:52:16.092401 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/38bf6fea-894f-4f1c-841b-374540c44a38-operator-scripts\") pod \"38bf6fea-894f-4f1c-841b-374540c44a38\" (UID: \"38bf6fea-894f-4f1c-841b-374540c44a38\") " Dec 03 12:52:16 crc kubenswrapper[5052]: I1203 12:52:16.092423 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xtqn7\" (UniqueName: \"kubernetes.io/projected/69d15184-ac62-4679-aadf-61fd17294318-kube-api-access-xtqn7\") pod \"69d15184-ac62-4679-aadf-61fd17294318\" (UID: \"69d15184-ac62-4679-aadf-61fd17294318\") " Dec 03 12:52:16 crc kubenswrapper[5052]: I1203 12:52:16.092534 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/4d9b46e0-ffbc-4a61-8b07-f84d5b630f5a-operator-scripts\") pod \"4d9b46e0-ffbc-4a61-8b07-f84d5b630f5a\" (UID: \"4d9b46e0-ffbc-4a61-8b07-f84d5b630f5a\") " Dec 03 12:52:16 crc kubenswrapper[5052]: I1203 12:52:16.093949 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4d9b46e0-ffbc-4a61-8b07-f84d5b630f5a-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "4d9b46e0-ffbc-4a61-8b07-f84d5b630f5a" (UID: "4d9b46e0-ffbc-4a61-8b07-f84d5b630f5a"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:52:16 crc kubenswrapper[5052]: I1203 12:52:16.094000 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/69d15184-ac62-4679-aadf-61fd17294318-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "69d15184-ac62-4679-aadf-61fd17294318" (UID: "69d15184-ac62-4679-aadf-61fd17294318"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:52:16 crc kubenswrapper[5052]: I1203 12:52:16.094275 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2ff4a526-fb1e-48e9-ae88-6b84ce5eaba2-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "2ff4a526-fb1e-48e9-ae88-6b84ce5eaba2" (UID: "2ff4a526-fb1e-48e9-ae88-6b84ce5eaba2"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:52:16 crc kubenswrapper[5052]: I1203 12:52:16.094425 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/38bf6fea-894f-4f1c-841b-374540c44a38-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "38bf6fea-894f-4f1c-841b-374540c44a38" (UID: "38bf6fea-894f-4f1c-841b-374540c44a38"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:52:16 crc kubenswrapper[5052]: I1203 12:52:16.097734 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2ff4a526-fb1e-48e9-ae88-6b84ce5eaba2-kube-api-access-svd4h" (OuterVolumeSpecName: "kube-api-access-svd4h") pod "2ff4a526-fb1e-48e9-ae88-6b84ce5eaba2" (UID: "2ff4a526-fb1e-48e9-ae88-6b84ce5eaba2"). InnerVolumeSpecName "kube-api-access-svd4h". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:52:16 crc kubenswrapper[5052]: I1203 12:52:16.098767 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/571d31d9-6750-41fe-8905-ea6de9cfb44d-kube-api-access-kczb2" (OuterVolumeSpecName: "kube-api-access-kczb2") pod "571d31d9-6750-41fe-8905-ea6de9cfb44d" (UID: "571d31d9-6750-41fe-8905-ea6de9cfb44d"). InnerVolumeSpecName "kube-api-access-kczb2". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:52:16 crc kubenswrapper[5052]: I1203 12:52:16.099149 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/38bf6fea-894f-4f1c-841b-374540c44a38-kube-api-access-66qg4" (OuterVolumeSpecName: "kube-api-access-66qg4") pod "38bf6fea-894f-4f1c-841b-374540c44a38" (UID: "38bf6fea-894f-4f1c-841b-374540c44a38"). InnerVolumeSpecName "kube-api-access-66qg4". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:52:16 crc kubenswrapper[5052]: I1203 12:52:16.100634 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4d9b46e0-ffbc-4a61-8b07-f84d5b630f5a-kube-api-access-p6jmn" (OuterVolumeSpecName: "kube-api-access-p6jmn") pod "4d9b46e0-ffbc-4a61-8b07-f84d5b630f5a" (UID: "4d9b46e0-ffbc-4a61-8b07-f84d5b630f5a"). InnerVolumeSpecName "kube-api-access-p6jmn". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:52:16 crc kubenswrapper[5052]: I1203 12:52:16.103021 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/69d15184-ac62-4679-aadf-61fd17294318-kube-api-access-xtqn7" (OuterVolumeSpecName: "kube-api-access-xtqn7") pod "69d15184-ac62-4679-aadf-61fd17294318" (UID: "69d15184-ac62-4679-aadf-61fd17294318"). InnerVolumeSpecName "kube-api-access-xtqn7". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:52:16 crc kubenswrapper[5052]: I1203 12:52:16.194537 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-kczb2\" (UniqueName: \"kubernetes.io/projected/571d31d9-6750-41fe-8905-ea6de9cfb44d-kube-api-access-kczb2\") on node \"crc\" DevicePath \"\"" Dec 03 12:52:16 crc kubenswrapper[5052]: I1203 12:52:16.194578 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/2ff4a526-fb1e-48e9-ae88-6b84ce5eaba2-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:52:16 crc kubenswrapper[5052]: I1203 12:52:16.194604 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-p6jmn\" (UniqueName: \"kubernetes.io/projected/4d9b46e0-ffbc-4a61-8b07-f84d5b630f5a-kube-api-access-p6jmn\") on node \"crc\" DevicePath \"\"" Dec 03 12:52:16 crc kubenswrapper[5052]: I1203 12:52:16.194615 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/38bf6fea-894f-4f1c-841b-374540c44a38-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:52:16 crc kubenswrapper[5052]: I1203 12:52:16.194624 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-xtqn7\" (UniqueName: \"kubernetes.io/projected/69d15184-ac62-4679-aadf-61fd17294318-kube-api-access-xtqn7\") on node \"crc\" DevicePath \"\"" Dec 03 12:52:16 crc kubenswrapper[5052]: I1203 12:52:16.194633 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/4d9b46e0-ffbc-4a61-8b07-f84d5b630f5a-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:52:16 crc kubenswrapper[5052]: I1203 12:52:16.194641 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/69d15184-ac62-4679-aadf-61fd17294318-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:52:16 crc kubenswrapper[5052]: I1203 12:52:16.194650 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-svd4h\" (UniqueName: \"kubernetes.io/projected/2ff4a526-fb1e-48e9-ae88-6b84ce5eaba2-kube-api-access-svd4h\") on node \"crc\" DevicePath \"\"" Dec 03 12:52:16 crc kubenswrapper[5052]: I1203 12:52:16.194659 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-66qg4\" (UniqueName: \"kubernetes.io/projected/38bf6fea-894f-4f1c-841b-374540c44a38-kube-api-access-66qg4\") on node \"crc\" DevicePath \"\"" Dec 03 12:52:16 crc kubenswrapper[5052]: I1203 12:52:16.214586 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-7066-account-create-update-4xwgf" Dec 03 12:52:16 crc kubenswrapper[5052]: I1203 12:52:16.214640 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-7066-account-create-update-4xwgf" event={"ID":"4d9b46e0-ffbc-4a61-8b07-f84d5b630f5a","Type":"ContainerDied","Data":"6e53f88812f5f9916e69d41d02e253e0924cfb89d78d92ecc7d5745e82af1129"} Dec 03 12:52:16 crc kubenswrapper[5052]: I1203 12:52:16.214800 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6e53f88812f5f9916e69d41d02e253e0924cfb89d78d92ecc7d5745e82af1129" Dec 03 12:52:16 crc kubenswrapper[5052]: I1203 12:52:16.216853 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-0a46-account-create-update-4n2r8" event={"ID":"69d15184-ac62-4679-aadf-61fd17294318","Type":"ContainerDied","Data":"bfd79b797e67a642bf41fac48f3b919ba3a1f5efd9f87128842a33d6f21be6de"} Dec 03 12:52:16 crc kubenswrapper[5052]: I1203 12:52:16.216904 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bfd79b797e67a642bf41fac48f3b919ba3a1f5efd9f87128842a33d6f21be6de" Dec 03 12:52:16 crc kubenswrapper[5052]: I1203 12:52:16.217143 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-0a46-account-create-update-4n2r8" Dec 03 12:52:16 crc kubenswrapper[5052]: I1203 12:52:16.218241 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-2629-account-create-update-c78fj" Dec 03 12:52:16 crc kubenswrapper[5052]: I1203 12:52:16.218273 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-2629-account-create-update-c78fj" event={"ID":"38bf6fea-894f-4f1c-841b-374540c44a38","Type":"ContainerDied","Data":"1449b24b0e1393996115428adad6bb23fe3cf6ce02755f1ffd75a5d41aa08d56"} Dec 03 12:52:16 crc kubenswrapper[5052]: I1203 12:52:16.218302 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1449b24b0e1393996115428adad6bb23fe3cf6ce02755f1ffd75a5d41aa08d56" Dec 03 12:52:16 crc kubenswrapper[5052]: I1203 12:52:16.228460 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/kube-state-metrics-0" event={"ID":"571d31d9-6750-41fe-8905-ea6de9cfb44d","Type":"ContainerDied","Data":"ed01d2e0e418b4c44af203e889c8edd46f3780911714a3a903eb50cb9667a55a"} Dec 03 12:52:16 crc kubenswrapper[5052]: I1203 12:52:16.228523 5052 scope.go:117] "RemoveContainer" containerID="dac5b0bc9b047df6eb67eb15e4e6c404248602afea4f49a377b09a6f2b2fe314" Dec 03 12:52:16 crc kubenswrapper[5052]: I1203 12:52:16.228631 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:52:16 crc kubenswrapper[5052]: I1203 12:52:16.232931 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-db-create-nvztf" Dec 03 12:52:16 crc kubenswrapper[5052]: I1203 12:52:16.234731 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-db-create-xmgf4" event={"ID":"a3142a17-c4e9-49a1-95f3-f7b725516c93","Type":"ContainerDied","Data":"05fe440b1a8728bebae2e000d85874f1ee0039eb2f5ca51a66eeaceed998b8db"} Dec 03 12:52:16 crc kubenswrapper[5052]: I1203 12:52:16.234771 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="05fe440b1a8728bebae2e000d85874f1ee0039eb2f5ca51a66eeaceed998b8db" Dec 03 12:52:16 crc kubenswrapper[5052]: I1203 12:52:16.234831 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-db-create-xmgf4" Dec 03 12:52:16 crc kubenswrapper[5052]: I1203 12:52:16.241180 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-db-create-tp5g8" event={"ID":"2ff4a526-fb1e-48e9-ae88-6b84ce5eaba2","Type":"ContainerDied","Data":"b633210ccd963782f79ca4056e0b24683556127758e280f24de54630567dbe4a"} Dec 03 12:52:16 crc kubenswrapper[5052]: I1203 12:52:16.241233 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b633210ccd963782f79ca4056e0b24683556127758e280f24de54630567dbe4a" Dec 03 12:52:16 crc kubenswrapper[5052]: I1203 12:52:16.241312 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-db-create-tp5g8" Dec 03 12:52:16 crc kubenswrapper[5052]: I1203 12:52:16.252627 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-db-create-nvztf" event={"ID":"e26fce77-6037-4d8f-bb16-f06cb151fdef","Type":"ContainerDied","Data":"1fe6a36d5702f1d96c201f260f8be2f8bd2e9e9d1f1a9ff3b89ebbbfd3ddb8f7"} Dec 03 12:52:16 crc kubenswrapper[5052]: I1203 12:52:16.252693 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1fe6a36d5702f1d96c201f260f8be2f8bd2e9e9d1f1a9ff3b89ebbbfd3ddb8f7" Dec 03 12:52:16 crc kubenswrapper[5052]: I1203 12:52:16.252756 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-db-create-nvztf" Dec 03 12:52:16 crc kubenswrapper[5052]: I1203 12:52:16.296220 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/kube-state-metrics-0"] Dec 03 12:52:16 crc kubenswrapper[5052]: I1203 12:52:16.308456 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/kube-state-metrics-0"] Dec 03 12:52:16 crc kubenswrapper[5052]: I1203 12:52:16.321089 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/kube-state-metrics-0"] Dec 03 12:52:16 crc kubenswrapper[5052]: E1203 12:52:16.321547 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="571d31d9-6750-41fe-8905-ea6de9cfb44d" containerName="kube-state-metrics" Dec 03 12:52:16 crc kubenswrapper[5052]: I1203 12:52:16.321570 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="571d31d9-6750-41fe-8905-ea6de9cfb44d" containerName="kube-state-metrics" Dec 03 12:52:16 crc kubenswrapper[5052]: E1203 12:52:16.321591 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="2ff4a526-fb1e-48e9-ae88-6b84ce5eaba2" containerName="mariadb-database-create" Dec 03 12:52:16 crc kubenswrapper[5052]: I1203 12:52:16.321599 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="2ff4a526-fb1e-48e9-ae88-6b84ce5eaba2" containerName="mariadb-database-create" Dec 03 12:52:16 crc kubenswrapper[5052]: E1203 12:52:16.321611 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="a3142a17-c4e9-49a1-95f3-f7b725516c93" containerName="mariadb-database-create" Dec 03 12:52:16 crc kubenswrapper[5052]: I1203 12:52:16.321619 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="a3142a17-c4e9-49a1-95f3-f7b725516c93" containerName="mariadb-database-create" Dec 03 12:52:16 crc kubenswrapper[5052]: E1203 12:52:16.321640 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="4d9b46e0-ffbc-4a61-8b07-f84d5b630f5a" containerName="mariadb-account-create-update" Dec 03 12:52:16 crc kubenswrapper[5052]: I1203 12:52:16.321651 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="4d9b46e0-ffbc-4a61-8b07-f84d5b630f5a" containerName="mariadb-account-create-update" Dec 03 12:52:16 crc kubenswrapper[5052]: E1203 12:52:16.321666 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="69d15184-ac62-4679-aadf-61fd17294318" containerName="mariadb-account-create-update" Dec 03 12:52:16 crc kubenswrapper[5052]: I1203 12:52:16.321674 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="69d15184-ac62-4679-aadf-61fd17294318" containerName="mariadb-account-create-update" Dec 03 12:52:16 crc kubenswrapper[5052]: E1203 12:52:16.321697 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="38bf6fea-894f-4f1c-841b-374540c44a38" containerName="mariadb-account-create-update" Dec 03 12:52:16 crc kubenswrapper[5052]: I1203 12:52:16.321705 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="38bf6fea-894f-4f1c-841b-374540c44a38" containerName="mariadb-account-create-update" Dec 03 12:52:16 crc kubenswrapper[5052]: E1203 12:52:16.321716 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="e26fce77-6037-4d8f-bb16-f06cb151fdef" containerName="mariadb-database-create" Dec 03 12:52:16 crc kubenswrapper[5052]: I1203 12:52:16.321722 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="e26fce77-6037-4d8f-bb16-f06cb151fdef" containerName="mariadb-database-create" Dec 03 12:52:16 crc kubenswrapper[5052]: I1203 12:52:16.321924 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="e26fce77-6037-4d8f-bb16-f06cb151fdef" containerName="mariadb-database-create" Dec 03 12:52:16 crc kubenswrapper[5052]: I1203 12:52:16.321945 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="4d9b46e0-ffbc-4a61-8b07-f84d5b630f5a" containerName="mariadb-account-create-update" Dec 03 12:52:16 crc kubenswrapper[5052]: I1203 12:52:16.321961 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="38bf6fea-894f-4f1c-841b-374540c44a38" containerName="mariadb-account-create-update" Dec 03 12:52:16 crc kubenswrapper[5052]: I1203 12:52:16.321979 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="a3142a17-c4e9-49a1-95f3-f7b725516c93" containerName="mariadb-database-create" Dec 03 12:52:16 crc kubenswrapper[5052]: I1203 12:52:16.321994 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="69d15184-ac62-4679-aadf-61fd17294318" containerName="mariadb-account-create-update" Dec 03 12:52:16 crc kubenswrapper[5052]: I1203 12:52:16.322004 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="2ff4a526-fb1e-48e9-ae88-6b84ce5eaba2" containerName="mariadb-database-create" Dec 03 12:52:16 crc kubenswrapper[5052]: I1203 12:52:16.322021 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="571d31d9-6750-41fe-8905-ea6de9cfb44d" containerName="kube-state-metrics" Dec 03 12:52:16 crc kubenswrapper[5052]: I1203 12:52:16.322795 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:52:16 crc kubenswrapper[5052]: I1203 12:52:16.324897 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"kube-state-metrics-tls-config" Dec 03 12:52:16 crc kubenswrapper[5052]: I1203 12:52:16.324934 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-kube-state-metrics-svc" Dec 03 12:52:16 crc kubenswrapper[5052]: I1203 12:52:16.336726 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/kube-state-metrics-0"] Dec 03 12:52:16 crc kubenswrapper[5052]: I1203 12:52:16.396919 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/e26fce77-6037-4d8f-bb16-f06cb151fdef-operator-scripts\") pod \"e26fce77-6037-4d8f-bb16-f06cb151fdef\" (UID: \"e26fce77-6037-4d8f-bb16-f06cb151fdef\") " Dec 03 12:52:16 crc kubenswrapper[5052]: I1203 12:52:16.397198 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s6pbf\" (UniqueName: \"kubernetes.io/projected/e26fce77-6037-4d8f-bb16-f06cb151fdef-kube-api-access-s6pbf\") pod \"e26fce77-6037-4d8f-bb16-f06cb151fdef\" (UID: \"e26fce77-6037-4d8f-bb16-f06cb151fdef\") " Dec 03 12:52:16 crc kubenswrapper[5052]: I1203 12:52:16.397534 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e26fce77-6037-4d8f-bb16-f06cb151fdef-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "e26fce77-6037-4d8f-bb16-f06cb151fdef" (UID: "e26fce77-6037-4d8f-bb16-f06cb151fdef"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:52:16 crc kubenswrapper[5052]: I1203 12:52:16.397888 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/e26fce77-6037-4d8f-bb16-f06cb151fdef-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:52:16 crc kubenswrapper[5052]: I1203 12:52:16.401081 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e26fce77-6037-4d8f-bb16-f06cb151fdef-kube-api-access-s6pbf" (OuterVolumeSpecName: "kube-api-access-s6pbf") pod "e26fce77-6037-4d8f-bb16-f06cb151fdef" (UID: "e26fce77-6037-4d8f-bb16-f06cb151fdef"). InnerVolumeSpecName "kube-api-access-s6pbf". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:52:16 crc kubenswrapper[5052]: I1203 12:52:16.500095 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls-certs\" (UniqueName: \"kubernetes.io/secret/3c553114-e0c8-4af9-992f-5421b31497ed-kube-state-metrics-tls-certs\") pod \"kube-state-metrics-0\" (UID: \"3c553114-e0c8-4af9-992f-5421b31497ed\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:52:16 crc kubenswrapper[5052]: I1203 12:52:16.500172 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3c553114-e0c8-4af9-992f-5421b31497ed-combined-ca-bundle\") pod \"kube-state-metrics-0\" (UID: \"3c553114-e0c8-4af9-992f-5421b31497ed\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:52:16 crc kubenswrapper[5052]: I1203 12:52:16.500198 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hpx5g\" (UniqueName: \"kubernetes.io/projected/3c553114-e0c8-4af9-992f-5421b31497ed-kube-api-access-hpx5g\") pod \"kube-state-metrics-0\" (UID: \"3c553114-e0c8-4af9-992f-5421b31497ed\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:52:16 crc kubenswrapper[5052]: I1203 12:52:16.500256 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls-config\" (UniqueName: \"kubernetes.io/secret/3c553114-e0c8-4af9-992f-5421b31497ed-kube-state-metrics-tls-config\") pod \"kube-state-metrics-0\" (UID: \"3c553114-e0c8-4af9-992f-5421b31497ed\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:52:16 crc kubenswrapper[5052]: I1203 12:52:16.500330 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-s6pbf\" (UniqueName: \"kubernetes.io/projected/e26fce77-6037-4d8f-bb16-f06cb151fdef-kube-api-access-s6pbf\") on node \"crc\" DevicePath \"\"" Dec 03 12:52:16 crc kubenswrapper[5052]: I1203 12:52:16.602063 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls-config\" (UniqueName: \"kubernetes.io/secret/3c553114-e0c8-4af9-992f-5421b31497ed-kube-state-metrics-tls-config\") pod \"kube-state-metrics-0\" (UID: \"3c553114-e0c8-4af9-992f-5421b31497ed\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:52:16 crc kubenswrapper[5052]: I1203 12:52:16.602205 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls-certs\" (UniqueName: \"kubernetes.io/secret/3c553114-e0c8-4af9-992f-5421b31497ed-kube-state-metrics-tls-certs\") pod \"kube-state-metrics-0\" (UID: \"3c553114-e0c8-4af9-992f-5421b31497ed\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:52:16 crc kubenswrapper[5052]: I1203 12:52:16.602287 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3c553114-e0c8-4af9-992f-5421b31497ed-combined-ca-bundle\") pod \"kube-state-metrics-0\" (UID: \"3c553114-e0c8-4af9-992f-5421b31497ed\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:52:16 crc kubenswrapper[5052]: I1203 12:52:16.602317 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-hpx5g\" (UniqueName: \"kubernetes.io/projected/3c553114-e0c8-4af9-992f-5421b31497ed-kube-api-access-hpx5g\") pod \"kube-state-metrics-0\" (UID: \"3c553114-e0c8-4af9-992f-5421b31497ed\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:52:16 crc kubenswrapper[5052]: I1203 12:52:16.606247 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3c553114-e0c8-4af9-992f-5421b31497ed-combined-ca-bundle\") pod \"kube-state-metrics-0\" (UID: \"3c553114-e0c8-4af9-992f-5421b31497ed\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:52:16 crc kubenswrapper[5052]: I1203 12:52:16.606592 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls-certs\" (UniqueName: \"kubernetes.io/secret/3c553114-e0c8-4af9-992f-5421b31497ed-kube-state-metrics-tls-certs\") pod \"kube-state-metrics-0\" (UID: \"3c553114-e0c8-4af9-992f-5421b31497ed\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:52:16 crc kubenswrapper[5052]: I1203 12:52:16.607008 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls-config\" (UniqueName: \"kubernetes.io/secret/3c553114-e0c8-4af9-992f-5421b31497ed-kube-state-metrics-tls-config\") pod \"kube-state-metrics-0\" (UID: \"3c553114-e0c8-4af9-992f-5421b31497ed\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:52:16 crc kubenswrapper[5052]: I1203 12:52:16.619296 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-hpx5g\" (UniqueName: \"kubernetes.io/projected/3c553114-e0c8-4af9-992f-5421b31497ed-kube-api-access-hpx5g\") pod \"kube-state-metrics-0\" (UID: \"3c553114-e0c8-4af9-992f-5421b31497ed\") " pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:52:16 crc kubenswrapper[5052]: I1203 12:52:16.705054 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:52:16 crc kubenswrapper[5052]: I1203 12:52:16.814902 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:52:17 crc kubenswrapper[5052]: I1203 12:52:17.178785 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/kube-state-metrics-0"] Dec 03 12:52:17 crc kubenswrapper[5052]: W1203 12:52:17.189007 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3c553114_e0c8_4af9_992f_5421b31497ed.slice/crio-53edc2154195b4da6f1b98de61e5c7482fcb0dd5f0bf268ab7464297f0dac516 WatchSource:0}: Error finding container 53edc2154195b4da6f1b98de61e5c7482fcb0dd5f0bf268ab7464297f0dac516: Status 404 returned error can't find the container with id 53edc2154195b4da6f1b98de61e5c7482fcb0dd5f0bf268ab7464297f0dac516 Dec 03 12:52:17 crc kubenswrapper[5052]: I1203 12:52:17.263868 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"186b516e-0d01-4c65-8538-94d9f7d30df3","Type":"ContainerStarted","Data":"beece57396ac6ecad4be85951176d559b1d2e56ef0c8a6748cf3bf59994c1e23"} Dec 03 12:52:17 crc kubenswrapper[5052]: I1203 12:52:17.265128 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/kube-state-metrics-0" event={"ID":"3c553114-e0c8-4af9-992f-5421b31497ed","Type":"ContainerStarted","Data":"53edc2154195b4da6f1b98de61e5c7482fcb0dd5f0bf268ab7464297f0dac516"} Dec 03 12:52:17 crc kubenswrapper[5052]: I1203 12:52:17.685799 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="571d31d9-6750-41fe-8905-ea6de9cfb44d" path="/var/lib/kubelet/pods/571d31d9-6750-41fe-8905-ea6de9cfb44d/volumes" Dec 03 12:52:18 crc kubenswrapper[5052]: I1203 12:52:18.283265 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"186b516e-0d01-4c65-8538-94d9f7d30df3","Type":"ContainerStarted","Data":"7f9b15965f678038775ed4ad92775e0ebdc5a3c4a507d030dfd30dafb03b5000"} Dec 03 12:52:18 crc kubenswrapper[5052]: I1203 12:52:18.286591 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/kube-state-metrics-0" event={"ID":"3c553114-e0c8-4af9-992f-5421b31497ed","Type":"ContainerStarted","Data":"77bdaa2aad097bc96adc6a7e702b0e7e164f8b659c20495278f4d679ff5255fc"} Dec 03 12:52:18 crc kubenswrapper[5052]: I1203 12:52:18.286771 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:52:18 crc kubenswrapper[5052]: I1203 12:52:18.314605 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/kube-state-metrics-0" podStartSLOduration=1.804214628 podStartE2EDuration="2.314587794s" podCreationTimestamp="2025-12-03 12:52:16 +0000 UTC" firstStartedPulling="2025-12-03 12:52:17.192878895 +0000 UTC m=+5527.781120659" lastFinishedPulling="2025-12-03 12:52:17.703252061 +0000 UTC m=+5528.291493825" observedRunningTime="2025-12-03 12:52:18.304827355 +0000 UTC m=+5528.893069139" watchObservedRunningTime="2025-12-03 12:52:18.314587794 +0000 UTC m=+5528.902829558" Dec 03 12:52:19 crc kubenswrapper[5052]: I1203 12:52:19.299685 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"186b516e-0d01-4c65-8538-94d9f7d30df3","Type":"ContainerStarted","Data":"5d5740aaad5dc10d850b2392ecd2c0287191a26d2120cdd8009f96e88219597d"} Dec 03 12:52:22 crc kubenswrapper[5052]: I1203 12:52:22.260215 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-db-sync-8f92x"] Dec 03 12:52:22 crc kubenswrapper[5052]: I1203 12:52:22.262197 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-8f92x" Dec 03 12:52:22 crc kubenswrapper[5052]: I1203 12:52:22.265777 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-nova-dockercfg-zsqnb" Dec 03 12:52:22 crc kubenswrapper[5052]: I1203 12:52:22.266699 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell0-conductor-scripts" Dec 03 12:52:22 crc kubenswrapper[5052]: I1203 12:52:22.266929 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell0-conductor-config-data" Dec 03 12:52:22 crc kubenswrapper[5052]: I1203 12:52:22.279420 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-db-sync-8f92x"] Dec 03 12:52:22 crc kubenswrapper[5052]: I1203 12:52:22.331050 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"186b516e-0d01-4c65-8538-94d9f7d30df3","Type":"ContainerStarted","Data":"68f5c8c9b9ea2d938f4f548913daba2af1ec2af9d890140096d327a28b69500c"} Dec 03 12:52:22 crc kubenswrapper[5052]: I1203 12:52:22.331251 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:52:22 crc kubenswrapper[5052]: I1203 12:52:22.331254 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="186b516e-0d01-4c65-8538-94d9f7d30df3" containerName="proxy-httpd" containerID="cri-o://68f5c8c9b9ea2d938f4f548913daba2af1ec2af9d890140096d327a28b69500c" gracePeriod=30 Dec 03 12:52:22 crc kubenswrapper[5052]: I1203 12:52:22.331305 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="186b516e-0d01-4c65-8538-94d9f7d30df3" containerName="ceilometer-notification-agent" containerID="cri-o://7f9b15965f678038775ed4ad92775e0ebdc5a3c4a507d030dfd30dafb03b5000" gracePeriod=30 Dec 03 12:52:22 crc kubenswrapper[5052]: I1203 12:52:22.331261 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="186b516e-0d01-4c65-8538-94d9f7d30df3" containerName="sg-core" containerID="cri-o://5d5740aaad5dc10d850b2392ecd2c0287191a26d2120cdd8009f96e88219597d" gracePeriod=30 Dec 03 12:52:22 crc kubenswrapper[5052]: I1203 12:52:22.331201 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="186b516e-0d01-4c65-8538-94d9f7d30df3" containerName="ceilometer-central-agent" containerID="cri-o://beece57396ac6ecad4be85951176d559b1d2e56ef0c8a6748cf3bf59994c1e23" gracePeriod=30 Dec 03 12:52:22 crc kubenswrapper[5052]: I1203 12:52:22.343004 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/6bed880a-56d4-4fed-bec7-6fea476ceb58-scripts\") pod \"nova-cell0-conductor-db-sync-8f92x\" (UID: \"6bed880a-56d4-4fed-bec7-6fea476ceb58\") " pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-8f92x" Dec 03 12:52:22 crc kubenswrapper[5052]: I1203 12:52:22.343053 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6bed880a-56d4-4fed-bec7-6fea476ceb58-combined-ca-bundle\") pod \"nova-cell0-conductor-db-sync-8f92x\" (UID: \"6bed880a-56d4-4fed-bec7-6fea476ceb58\") " pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-8f92x" Dec 03 12:52:22 crc kubenswrapper[5052]: I1203 12:52:22.343109 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6bed880a-56d4-4fed-bec7-6fea476ceb58-config-data\") pod \"nova-cell0-conductor-db-sync-8f92x\" (UID: \"6bed880a-56d4-4fed-bec7-6fea476ceb58\") " pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-8f92x" Dec 03 12:52:22 crc kubenswrapper[5052]: I1203 12:52:22.343192 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vxsgf\" (UniqueName: \"kubernetes.io/projected/6bed880a-56d4-4fed-bec7-6fea476ceb58-kube-api-access-vxsgf\") pod \"nova-cell0-conductor-db-sync-8f92x\" (UID: \"6bed880a-56d4-4fed-bec7-6fea476ceb58\") " pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-8f92x" Dec 03 12:52:22 crc kubenswrapper[5052]: I1203 12:52:22.449718 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/6bed880a-56d4-4fed-bec7-6fea476ceb58-scripts\") pod \"nova-cell0-conductor-db-sync-8f92x\" (UID: \"6bed880a-56d4-4fed-bec7-6fea476ceb58\") " pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-8f92x" Dec 03 12:52:22 crc kubenswrapper[5052]: I1203 12:52:22.449763 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6bed880a-56d4-4fed-bec7-6fea476ceb58-combined-ca-bundle\") pod \"nova-cell0-conductor-db-sync-8f92x\" (UID: \"6bed880a-56d4-4fed-bec7-6fea476ceb58\") " pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-8f92x" Dec 03 12:52:22 crc kubenswrapper[5052]: I1203 12:52:22.449813 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6bed880a-56d4-4fed-bec7-6fea476ceb58-config-data\") pod \"nova-cell0-conductor-db-sync-8f92x\" (UID: \"6bed880a-56d4-4fed-bec7-6fea476ceb58\") " pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-8f92x" Dec 03 12:52:22 crc kubenswrapper[5052]: I1203 12:52:22.449858 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-vxsgf\" (UniqueName: \"kubernetes.io/projected/6bed880a-56d4-4fed-bec7-6fea476ceb58-kube-api-access-vxsgf\") pod \"nova-cell0-conductor-db-sync-8f92x\" (UID: \"6bed880a-56d4-4fed-bec7-6fea476ceb58\") " pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-8f92x" Dec 03 12:52:22 crc kubenswrapper[5052]: I1203 12:52:22.454497 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6bed880a-56d4-4fed-bec7-6fea476ceb58-config-data\") pod \"nova-cell0-conductor-db-sync-8f92x\" (UID: \"6bed880a-56d4-4fed-bec7-6fea476ceb58\") " pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-8f92x" Dec 03 12:52:22 crc kubenswrapper[5052]: I1203 12:52:22.459155 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6bed880a-56d4-4fed-bec7-6fea476ceb58-combined-ca-bundle\") pod \"nova-cell0-conductor-db-sync-8f92x\" (UID: \"6bed880a-56d4-4fed-bec7-6fea476ceb58\") " pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-8f92x" Dec 03 12:52:22 crc kubenswrapper[5052]: I1203 12:52:22.462867 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/6bed880a-56d4-4fed-bec7-6fea476ceb58-scripts\") pod \"nova-cell0-conductor-db-sync-8f92x\" (UID: \"6bed880a-56d4-4fed-bec7-6fea476ceb58\") " pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-8f92x" Dec 03 12:52:22 crc kubenswrapper[5052]: I1203 12:52:22.469029 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-vxsgf\" (UniqueName: \"kubernetes.io/projected/6bed880a-56d4-4fed-bec7-6fea476ceb58-kube-api-access-vxsgf\") pod \"nova-cell0-conductor-db-sync-8f92x\" (UID: \"6bed880a-56d4-4fed-bec7-6fea476ceb58\") " pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-8f92x" Dec 03 12:52:22 crc kubenswrapper[5052]: I1203 12:52:22.581362 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-8f92x" Dec 03 12:52:23 crc kubenswrapper[5052]: I1203 12:52:23.055094 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/ceilometer-0" podStartSLOduration=2.548770356 podStartE2EDuration="10.055070133s" podCreationTimestamp="2025-12-03 12:52:13 +0000 UTC" firstStartedPulling="2025-12-03 12:52:14.20123909 +0000 UTC m=+5524.789480854" lastFinishedPulling="2025-12-03 12:52:21.707538867 +0000 UTC m=+5532.295780631" observedRunningTime="2025-12-03 12:52:22.362202879 +0000 UTC m=+5532.950444643" watchObservedRunningTime="2025-12-03 12:52:23.055070133 +0000 UTC m=+5533.643311897" Dec 03 12:52:23 crc kubenswrapper[5052]: W1203 12:52:23.057593 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6bed880a_56d4_4fed_bec7_6fea476ceb58.slice/crio-a208a94a1930048f74ca260eb08531acef36473d7ba9ad67f6179a05ee533181 WatchSource:0}: Error finding container a208a94a1930048f74ca260eb08531acef36473d7ba9ad67f6179a05ee533181: Status 404 returned error can't find the container with id a208a94a1930048f74ca260eb08531acef36473d7ba9ad67f6179a05ee533181 Dec 03 12:52:23 crc kubenswrapper[5052]: I1203 12:52:23.059164 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-db-sync-8f92x"] Dec 03 12:52:23 crc kubenswrapper[5052]: I1203 12:52:23.341881 5052 generic.go:334] "Generic (PLEG): container finished" podID="186b516e-0d01-4c65-8538-94d9f7d30df3" containerID="68f5c8c9b9ea2d938f4f548913daba2af1ec2af9d890140096d327a28b69500c" exitCode=0 Dec 03 12:52:23 crc kubenswrapper[5052]: I1203 12:52:23.341911 5052 generic.go:334] "Generic (PLEG): container finished" podID="186b516e-0d01-4c65-8538-94d9f7d30df3" containerID="5d5740aaad5dc10d850b2392ecd2c0287191a26d2120cdd8009f96e88219597d" exitCode=2 Dec 03 12:52:23 crc kubenswrapper[5052]: I1203 12:52:23.341919 5052 generic.go:334] "Generic (PLEG): container finished" podID="186b516e-0d01-4c65-8538-94d9f7d30df3" containerID="7f9b15965f678038775ed4ad92775e0ebdc5a3c4a507d030dfd30dafb03b5000" exitCode=0 Dec 03 12:52:23 crc kubenswrapper[5052]: I1203 12:52:23.341966 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"186b516e-0d01-4c65-8538-94d9f7d30df3","Type":"ContainerDied","Data":"68f5c8c9b9ea2d938f4f548913daba2af1ec2af9d890140096d327a28b69500c"} Dec 03 12:52:23 crc kubenswrapper[5052]: I1203 12:52:23.342012 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"186b516e-0d01-4c65-8538-94d9f7d30df3","Type":"ContainerDied","Data":"5d5740aaad5dc10d850b2392ecd2c0287191a26d2120cdd8009f96e88219597d"} Dec 03 12:52:23 crc kubenswrapper[5052]: I1203 12:52:23.342023 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"186b516e-0d01-4c65-8538-94d9f7d30df3","Type":"ContainerDied","Data":"7f9b15965f678038775ed4ad92775e0ebdc5a3c4a507d030dfd30dafb03b5000"} Dec 03 12:52:23 crc kubenswrapper[5052]: I1203 12:52:23.343412 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-8f92x" event={"ID":"6bed880a-56d4-4fed-bec7-6fea476ceb58","Type":"ContainerStarted","Data":"96b654a5e42ddde9fc6e98cc635309cb5313602a1cc9c873755f04530d171682"} Dec 03 12:52:23 crc kubenswrapper[5052]: I1203 12:52:23.343449 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-8f92x" event={"ID":"6bed880a-56d4-4fed-bec7-6fea476ceb58","Type":"ContainerStarted","Data":"a208a94a1930048f74ca260eb08531acef36473d7ba9ad67f6179a05ee533181"} Dec 03 12:52:26 crc kubenswrapper[5052]: I1203 12:52:26.384264 5052 generic.go:334] "Generic (PLEG): container finished" podID="186b516e-0d01-4c65-8538-94d9f7d30df3" containerID="beece57396ac6ecad4be85951176d559b1d2e56ef0c8a6748cf3bf59994c1e23" exitCode=0 Dec 03 12:52:26 crc kubenswrapper[5052]: I1203 12:52:26.384337 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"186b516e-0d01-4c65-8538-94d9f7d30df3","Type":"ContainerDied","Data":"beece57396ac6ecad4be85951176d559b1d2e56ef0c8a6748cf3bf59994c1e23"} Dec 03 12:52:26 crc kubenswrapper[5052]: I1203 12:52:26.563369 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:52:26 crc kubenswrapper[5052]: I1203 12:52:26.604343 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-8f92x" podStartSLOduration=4.604322175 podStartE2EDuration="4.604322175s" podCreationTimestamp="2025-12-03 12:52:22 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:52:23.367357444 +0000 UTC m=+5533.955599208" watchObservedRunningTime="2025-12-03 12:52:26.604322175 +0000 UTC m=+5537.192563939" Dec 03 12:52:26 crc kubenswrapper[5052]: I1203 12:52:26.727080 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-65dkq\" (UniqueName: \"kubernetes.io/projected/186b516e-0d01-4c65-8538-94d9f7d30df3-kube-api-access-65dkq\") pod \"186b516e-0d01-4c65-8538-94d9f7d30df3\" (UID: \"186b516e-0d01-4c65-8538-94d9f7d30df3\") " Dec 03 12:52:26 crc kubenswrapper[5052]: I1203 12:52:26.727137 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/186b516e-0d01-4c65-8538-94d9f7d30df3-config-data\") pod \"186b516e-0d01-4c65-8538-94d9f7d30df3\" (UID: \"186b516e-0d01-4c65-8538-94d9f7d30df3\") " Dec 03 12:52:26 crc kubenswrapper[5052]: I1203 12:52:26.727218 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/186b516e-0d01-4c65-8538-94d9f7d30df3-sg-core-conf-yaml\") pod \"186b516e-0d01-4c65-8538-94d9f7d30df3\" (UID: \"186b516e-0d01-4c65-8538-94d9f7d30df3\") " Dec 03 12:52:26 crc kubenswrapper[5052]: I1203 12:52:26.727331 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/186b516e-0d01-4c65-8538-94d9f7d30df3-run-httpd\") pod \"186b516e-0d01-4c65-8538-94d9f7d30df3\" (UID: \"186b516e-0d01-4c65-8538-94d9f7d30df3\") " Dec 03 12:52:26 crc kubenswrapper[5052]: I1203 12:52:26.727457 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/186b516e-0d01-4c65-8538-94d9f7d30df3-log-httpd\") pod \"186b516e-0d01-4c65-8538-94d9f7d30df3\" (UID: \"186b516e-0d01-4c65-8538-94d9f7d30df3\") " Dec 03 12:52:26 crc kubenswrapper[5052]: I1203 12:52:26.727487 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/186b516e-0d01-4c65-8538-94d9f7d30df3-scripts\") pod \"186b516e-0d01-4c65-8538-94d9f7d30df3\" (UID: \"186b516e-0d01-4c65-8538-94d9f7d30df3\") " Dec 03 12:52:26 crc kubenswrapper[5052]: I1203 12:52:26.727565 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/186b516e-0d01-4c65-8538-94d9f7d30df3-combined-ca-bundle\") pod \"186b516e-0d01-4c65-8538-94d9f7d30df3\" (UID: \"186b516e-0d01-4c65-8538-94d9f7d30df3\") " Dec 03 12:52:26 crc kubenswrapper[5052]: I1203 12:52:26.727858 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/186b516e-0d01-4c65-8538-94d9f7d30df3-run-httpd" (OuterVolumeSpecName: "run-httpd") pod "186b516e-0d01-4c65-8538-94d9f7d30df3" (UID: "186b516e-0d01-4c65-8538-94d9f7d30df3"). InnerVolumeSpecName "run-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:52:26 crc kubenswrapper[5052]: I1203 12:52:26.727956 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/186b516e-0d01-4c65-8538-94d9f7d30df3-log-httpd" (OuterVolumeSpecName: "log-httpd") pod "186b516e-0d01-4c65-8538-94d9f7d30df3" (UID: "186b516e-0d01-4c65-8538-94d9f7d30df3"). InnerVolumeSpecName "log-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:52:26 crc kubenswrapper[5052]: I1203 12:52:26.728895 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:52:26 crc kubenswrapper[5052]: I1203 12:52:26.735937 5052 reconciler_common.go:293] "Volume detached for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/186b516e-0d01-4c65-8538-94d9f7d30df3-run-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:52:26 crc kubenswrapper[5052]: I1203 12:52:26.735973 5052 reconciler_common.go:293] "Volume detached for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/186b516e-0d01-4c65-8538-94d9f7d30df3-log-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:52:26 crc kubenswrapper[5052]: I1203 12:52:26.758658 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/186b516e-0d01-4c65-8538-94d9f7d30df3-scripts" (OuterVolumeSpecName: "scripts") pod "186b516e-0d01-4c65-8538-94d9f7d30df3" (UID: "186b516e-0d01-4c65-8538-94d9f7d30df3"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:52:26 crc kubenswrapper[5052]: I1203 12:52:26.761877 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/186b516e-0d01-4c65-8538-94d9f7d30df3-kube-api-access-65dkq" (OuterVolumeSpecName: "kube-api-access-65dkq") pod "186b516e-0d01-4c65-8538-94d9f7d30df3" (UID: "186b516e-0d01-4c65-8538-94d9f7d30df3"). InnerVolumeSpecName "kube-api-access-65dkq". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:52:26 crc kubenswrapper[5052]: I1203 12:52:26.779670 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/186b516e-0d01-4c65-8538-94d9f7d30df3-sg-core-conf-yaml" (OuterVolumeSpecName: "sg-core-conf-yaml") pod "186b516e-0d01-4c65-8538-94d9f7d30df3" (UID: "186b516e-0d01-4c65-8538-94d9f7d30df3"). InnerVolumeSpecName "sg-core-conf-yaml". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:52:26 crc kubenswrapper[5052]: I1203 12:52:26.848964 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/186b516e-0d01-4c65-8538-94d9f7d30df3-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:52:26 crc kubenswrapper[5052]: I1203 12:52:26.849296 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-65dkq\" (UniqueName: \"kubernetes.io/projected/186b516e-0d01-4c65-8538-94d9f7d30df3-kube-api-access-65dkq\") on node \"crc\" DevicePath \"\"" Dec 03 12:52:26 crc kubenswrapper[5052]: I1203 12:52:26.849309 5052 reconciler_common.go:293] "Volume detached for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/186b516e-0d01-4c65-8538-94d9f7d30df3-sg-core-conf-yaml\") on node \"crc\" DevicePath \"\"" Dec 03 12:52:26 crc kubenswrapper[5052]: I1203 12:52:26.856117 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/186b516e-0d01-4c65-8538-94d9f7d30df3-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "186b516e-0d01-4c65-8538-94d9f7d30df3" (UID: "186b516e-0d01-4c65-8538-94d9f7d30df3"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:52:26 crc kubenswrapper[5052]: I1203 12:52:26.881687 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/186b516e-0d01-4c65-8538-94d9f7d30df3-config-data" (OuterVolumeSpecName: "config-data") pod "186b516e-0d01-4c65-8538-94d9f7d30df3" (UID: "186b516e-0d01-4c65-8538-94d9f7d30df3"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:52:26 crc kubenswrapper[5052]: I1203 12:52:26.951257 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/186b516e-0d01-4c65-8538-94d9f7d30df3-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:52:26 crc kubenswrapper[5052]: I1203 12:52:26.951288 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/186b516e-0d01-4c65-8538-94d9f7d30df3-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:52:27 crc kubenswrapper[5052]: I1203 12:52:27.396404 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"186b516e-0d01-4c65-8538-94d9f7d30df3","Type":"ContainerDied","Data":"50c8e90c89f0255bf4a704fb25fba49af8b3c17101af68d6d2dc43e2fb344c0b"} Dec 03 12:52:27 crc kubenswrapper[5052]: I1203 12:52:27.396464 5052 scope.go:117] "RemoveContainer" containerID="68f5c8c9b9ea2d938f4f548913daba2af1ec2af9d890140096d327a28b69500c" Dec 03 12:52:27 crc kubenswrapper[5052]: I1203 12:52:27.397487 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:52:27 crc kubenswrapper[5052]: I1203 12:52:27.417312 5052 scope.go:117] "RemoveContainer" containerID="5d5740aaad5dc10d850b2392ecd2c0287191a26d2120cdd8009f96e88219597d" Dec 03 12:52:27 crc kubenswrapper[5052]: I1203 12:52:27.441526 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:52:27 crc kubenswrapper[5052]: I1203 12:52:27.451598 5052 scope.go:117] "RemoveContainer" containerID="7f9b15965f678038775ed4ad92775e0ebdc5a3c4a507d030dfd30dafb03b5000" Dec 03 12:52:27 crc kubenswrapper[5052]: I1203 12:52:27.453178 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:52:27 crc kubenswrapper[5052]: I1203 12:52:27.468914 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:52:27 crc kubenswrapper[5052]: E1203 12:52:27.469334 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="186b516e-0d01-4c65-8538-94d9f7d30df3" containerName="sg-core" Dec 03 12:52:27 crc kubenswrapper[5052]: I1203 12:52:27.469352 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="186b516e-0d01-4c65-8538-94d9f7d30df3" containerName="sg-core" Dec 03 12:52:27 crc kubenswrapper[5052]: E1203 12:52:27.469367 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="186b516e-0d01-4c65-8538-94d9f7d30df3" containerName="proxy-httpd" Dec 03 12:52:27 crc kubenswrapper[5052]: I1203 12:52:27.469374 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="186b516e-0d01-4c65-8538-94d9f7d30df3" containerName="proxy-httpd" Dec 03 12:52:27 crc kubenswrapper[5052]: E1203 12:52:27.469394 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="186b516e-0d01-4c65-8538-94d9f7d30df3" containerName="ceilometer-notification-agent" Dec 03 12:52:27 crc kubenswrapper[5052]: I1203 12:52:27.469401 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="186b516e-0d01-4c65-8538-94d9f7d30df3" containerName="ceilometer-notification-agent" Dec 03 12:52:27 crc kubenswrapper[5052]: E1203 12:52:27.469409 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="186b516e-0d01-4c65-8538-94d9f7d30df3" containerName="ceilometer-central-agent" Dec 03 12:52:27 crc kubenswrapper[5052]: I1203 12:52:27.469414 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="186b516e-0d01-4c65-8538-94d9f7d30df3" containerName="ceilometer-central-agent" Dec 03 12:52:27 crc kubenswrapper[5052]: I1203 12:52:27.469599 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="186b516e-0d01-4c65-8538-94d9f7d30df3" containerName="proxy-httpd" Dec 03 12:52:27 crc kubenswrapper[5052]: I1203 12:52:27.469610 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="186b516e-0d01-4c65-8538-94d9f7d30df3" containerName="ceilometer-notification-agent" Dec 03 12:52:27 crc kubenswrapper[5052]: I1203 12:52:27.469633 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="186b516e-0d01-4c65-8538-94d9f7d30df3" containerName="ceilometer-central-agent" Dec 03 12:52:27 crc kubenswrapper[5052]: I1203 12:52:27.469645 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="186b516e-0d01-4c65-8538-94d9f7d30df3" containerName="sg-core" Dec 03 12:52:27 crc kubenswrapper[5052]: I1203 12:52:27.471581 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:52:27 crc kubenswrapper[5052]: I1203 12:52:27.475191 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-scripts" Dec 03 12:52:27 crc kubenswrapper[5052]: I1203 12:52:27.475521 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-config-data" Dec 03 12:52:27 crc kubenswrapper[5052]: I1203 12:52:27.475675 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-ceilometer-internal-svc" Dec 03 12:52:27 crc kubenswrapper[5052]: I1203 12:52:27.478728 5052 scope.go:117] "RemoveContainer" containerID="beece57396ac6ecad4be85951176d559b1d2e56ef0c8a6748cf3bf59994c1e23" Dec 03 12:52:27 crc kubenswrapper[5052]: I1203 12:52:27.493527 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:52:27 crc kubenswrapper[5052]: I1203 12:52:27.560149 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gcrw5\" (UniqueName: \"kubernetes.io/projected/165e6ac4-9875-4750-afb3-c77088cdc884-kube-api-access-gcrw5\") pod \"ceilometer-0\" (UID: \"165e6ac4-9875-4750-afb3-c77088cdc884\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:52:27 crc kubenswrapper[5052]: I1203 12:52:27.560196 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/165e6ac4-9875-4750-afb3-c77088cdc884-scripts\") pod \"ceilometer-0\" (UID: \"165e6ac4-9875-4750-afb3-c77088cdc884\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:52:27 crc kubenswrapper[5052]: I1203 12:52:27.560251 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/165e6ac4-9875-4750-afb3-c77088cdc884-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"165e6ac4-9875-4750-afb3-c77088cdc884\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:52:27 crc kubenswrapper[5052]: I1203 12:52:27.560296 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/165e6ac4-9875-4750-afb3-c77088cdc884-run-httpd\") pod \"ceilometer-0\" (UID: \"165e6ac4-9875-4750-afb3-c77088cdc884\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:52:27 crc kubenswrapper[5052]: I1203 12:52:27.560396 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/165e6ac4-9875-4750-afb3-c77088cdc884-config-data\") pod \"ceilometer-0\" (UID: \"165e6ac4-9875-4750-afb3-c77088cdc884\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:52:27 crc kubenswrapper[5052]: I1203 12:52:27.560438 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/165e6ac4-9875-4750-afb3-c77088cdc884-ceilometer-tls-certs\") pod \"ceilometer-0\" (UID: \"165e6ac4-9875-4750-afb3-c77088cdc884\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:52:27 crc kubenswrapper[5052]: I1203 12:52:27.560529 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/165e6ac4-9875-4750-afb3-c77088cdc884-log-httpd\") pod \"ceilometer-0\" (UID: \"165e6ac4-9875-4750-afb3-c77088cdc884\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:52:27 crc kubenswrapper[5052]: I1203 12:52:27.560559 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/165e6ac4-9875-4750-afb3-c77088cdc884-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"165e6ac4-9875-4750-afb3-c77088cdc884\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:52:27 crc kubenswrapper[5052]: I1203 12:52:27.662337 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/165e6ac4-9875-4750-afb3-c77088cdc884-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"165e6ac4-9875-4750-afb3-c77088cdc884\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:52:27 crc kubenswrapper[5052]: I1203 12:52:27.662410 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/165e6ac4-9875-4750-afb3-c77088cdc884-run-httpd\") pod \"ceilometer-0\" (UID: \"165e6ac4-9875-4750-afb3-c77088cdc884\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:52:27 crc kubenswrapper[5052]: I1203 12:52:27.662523 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/165e6ac4-9875-4750-afb3-c77088cdc884-config-data\") pod \"ceilometer-0\" (UID: \"165e6ac4-9875-4750-afb3-c77088cdc884\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:52:27 crc kubenswrapper[5052]: I1203 12:52:27.662563 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/165e6ac4-9875-4750-afb3-c77088cdc884-ceilometer-tls-certs\") pod \"ceilometer-0\" (UID: \"165e6ac4-9875-4750-afb3-c77088cdc884\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:52:27 crc kubenswrapper[5052]: I1203 12:52:27.662619 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/165e6ac4-9875-4750-afb3-c77088cdc884-log-httpd\") pod \"ceilometer-0\" (UID: \"165e6ac4-9875-4750-afb3-c77088cdc884\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:52:27 crc kubenswrapper[5052]: I1203 12:52:27.662647 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/165e6ac4-9875-4750-afb3-c77088cdc884-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"165e6ac4-9875-4750-afb3-c77088cdc884\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:52:27 crc kubenswrapper[5052]: I1203 12:52:27.662710 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-gcrw5\" (UniqueName: \"kubernetes.io/projected/165e6ac4-9875-4750-afb3-c77088cdc884-kube-api-access-gcrw5\") pod \"ceilometer-0\" (UID: \"165e6ac4-9875-4750-afb3-c77088cdc884\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:52:27 crc kubenswrapper[5052]: I1203 12:52:27.662735 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/165e6ac4-9875-4750-afb3-c77088cdc884-scripts\") pod \"ceilometer-0\" (UID: \"165e6ac4-9875-4750-afb3-c77088cdc884\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:52:27 crc kubenswrapper[5052]: I1203 12:52:27.662985 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/165e6ac4-9875-4750-afb3-c77088cdc884-run-httpd\") pod \"ceilometer-0\" (UID: \"165e6ac4-9875-4750-afb3-c77088cdc884\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:52:27 crc kubenswrapper[5052]: I1203 12:52:27.664112 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/165e6ac4-9875-4750-afb3-c77088cdc884-log-httpd\") pod \"ceilometer-0\" (UID: \"165e6ac4-9875-4750-afb3-c77088cdc884\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:52:27 crc kubenswrapper[5052]: I1203 12:52:27.667270 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/165e6ac4-9875-4750-afb3-c77088cdc884-ceilometer-tls-certs\") pod \"ceilometer-0\" (UID: \"165e6ac4-9875-4750-afb3-c77088cdc884\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:52:27 crc kubenswrapper[5052]: I1203 12:52:27.667545 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/165e6ac4-9875-4750-afb3-c77088cdc884-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"165e6ac4-9875-4750-afb3-c77088cdc884\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:52:27 crc kubenswrapper[5052]: I1203 12:52:27.673870 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/165e6ac4-9875-4750-afb3-c77088cdc884-config-data\") pod \"ceilometer-0\" (UID: \"165e6ac4-9875-4750-afb3-c77088cdc884\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:52:27 crc kubenswrapper[5052]: I1203 12:52:27.674900 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/165e6ac4-9875-4750-afb3-c77088cdc884-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"165e6ac4-9875-4750-afb3-c77088cdc884\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:52:27 crc kubenswrapper[5052]: I1203 12:52:27.680132 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/165e6ac4-9875-4750-afb3-c77088cdc884-scripts\") pod \"ceilometer-0\" (UID: \"165e6ac4-9875-4750-afb3-c77088cdc884\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:52:27 crc kubenswrapper[5052]: I1203 12:52:27.683615 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-gcrw5\" (UniqueName: \"kubernetes.io/projected/165e6ac4-9875-4750-afb3-c77088cdc884-kube-api-access-gcrw5\") pod \"ceilometer-0\" (UID: \"165e6ac4-9875-4750-afb3-c77088cdc884\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:52:27 crc kubenswrapper[5052]: I1203 12:52:27.688353 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="186b516e-0d01-4c65-8538-94d9f7d30df3" path="/var/lib/kubelet/pods/186b516e-0d01-4c65-8538-94d9f7d30df3/volumes" Dec 03 12:52:27 crc kubenswrapper[5052]: I1203 12:52:27.845361 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:52:28 crc kubenswrapper[5052]: I1203 12:52:28.334256 5052 provider.go:102] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Dec 03 12:52:28 crc kubenswrapper[5052]: I1203 12:52:28.343177 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:52:28 crc kubenswrapper[5052]: I1203 12:52:28.409860 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"165e6ac4-9875-4750-afb3-c77088cdc884","Type":"ContainerStarted","Data":"789597ebd1d30a87780d6928f58709eecf3ac8ebd6ec7832c998cf41b5da4a3e"} Dec 03 12:52:30 crc kubenswrapper[5052]: I1203 12:52:30.431355 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"165e6ac4-9875-4750-afb3-c77088cdc884","Type":"ContainerStarted","Data":"d14603df1efe703bc136b352c1731d60aa5b4abdbd67a9414cc8be1a2069f9fa"} Dec 03 12:52:31 crc kubenswrapper[5052]: I1203 12:52:31.444476 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"165e6ac4-9875-4750-afb3-c77088cdc884","Type":"ContainerStarted","Data":"a3c0984ec47119deb78f745f0836fd837688c8ac4b2d2615e858c0c3c8890cc6"} Dec 03 12:52:32 crc kubenswrapper[5052]: I1203 12:52:32.455446 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"165e6ac4-9875-4750-afb3-c77088cdc884","Type":"ContainerStarted","Data":"66e47b99705eb8f855b5604562315e914b5edc1fc44a531356994b77c3a644a8"} Dec 03 12:52:32 crc kubenswrapper[5052]: I1203 12:52:32.585364 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:52:32 crc kubenswrapper[5052]: I1203 12:52:32.585606 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-external-api-0" podUID="cd023d37-ce83-4ca4-ab52-f89e94d0b721" containerName="glance-log" containerID="cri-o://c6b845ca325df0412cbeac659e0056c26815c15f2c2df4fafcf3c74b16e2590a" gracePeriod=30 Dec 03 12:52:32 crc kubenswrapper[5052]: I1203 12:52:32.585719 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-external-api-0" podUID="cd023d37-ce83-4ca4-ab52-f89e94d0b721" containerName="glance-httpd" containerID="cri-o://1ffdd15a3a5713be01c664e2b6740ea79e1cb391fcfc3f99d37c27c80a734363" gracePeriod=30 Dec 03 12:52:33 crc kubenswrapper[5052]: I1203 12:52:33.437120 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:52:33 crc kubenswrapper[5052]: I1203 12:52:33.441255 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-internal-api-0" podUID="fc57d1ed-34a0-42b5-b1be-5bf914492701" containerName="glance-httpd" containerID="cri-o://b2120f51b650e3bfe33d9abfe02816028d28060266d5498f4adcf7f4a0048dab" gracePeriod=30 Dec 03 12:52:33 crc kubenswrapper[5052]: I1203 12:52:33.441373 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-internal-api-0" podUID="fc57d1ed-34a0-42b5-b1be-5bf914492701" containerName="glance-log" containerID="cri-o://5e85ff655790c57c0d7ecc6f00fca41e5f274eea76a18c0b7ebbd12438ea1d83" gracePeriod=30 Dec 03 12:52:33 crc kubenswrapper[5052]: I1203 12:52:33.468991 5052 generic.go:334] "Generic (PLEG): container finished" podID="cd023d37-ce83-4ca4-ab52-f89e94d0b721" containerID="c6b845ca325df0412cbeac659e0056c26815c15f2c2df4fafcf3c74b16e2590a" exitCode=143 Dec 03 12:52:33 crc kubenswrapper[5052]: I1203 12:52:33.469044 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"cd023d37-ce83-4ca4-ab52-f89e94d0b721","Type":"ContainerDied","Data":"c6b845ca325df0412cbeac659e0056c26815c15f2c2df4fafcf3c74b16e2590a"} Dec 03 12:52:34 crc kubenswrapper[5052]: I1203 12:52:34.481167 5052 generic.go:334] "Generic (PLEG): container finished" podID="fc57d1ed-34a0-42b5-b1be-5bf914492701" containerID="5e85ff655790c57c0d7ecc6f00fca41e5f274eea76a18c0b7ebbd12438ea1d83" exitCode=143 Dec 03 12:52:34 crc kubenswrapper[5052]: I1203 12:52:34.481265 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"fc57d1ed-34a0-42b5-b1be-5bf914492701","Type":"ContainerDied","Data":"5e85ff655790c57c0d7ecc6f00fca41e5f274eea76a18c0b7ebbd12438ea1d83"} Dec 03 12:52:34 crc kubenswrapper[5052]: I1203 12:52:34.484240 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"165e6ac4-9875-4750-afb3-c77088cdc884","Type":"ContainerStarted","Data":"9690e5377d7ca39f7e310c5292008842e2f2c8ddceb5a7692800c84e88d1cacb"} Dec 03 12:52:34 crc kubenswrapper[5052]: I1203 12:52:34.484450 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:52:34 crc kubenswrapper[5052]: I1203 12:52:34.519546 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/ceilometer-0" podStartSLOduration=2.601226521 podStartE2EDuration="7.519528716s" podCreationTimestamp="2025-12-03 12:52:27 +0000 UTC" firstStartedPulling="2025-12-03 12:52:28.334052473 +0000 UTC m=+5538.922294237" lastFinishedPulling="2025-12-03 12:52:33.252354658 +0000 UTC m=+5543.840596432" observedRunningTime="2025-12-03 12:52:34.515499941 +0000 UTC m=+5545.103741705" watchObservedRunningTime="2025-12-03 12:52:34.519528716 +0000 UTC m=+5545.107770480" Dec 03 12:52:34 crc kubenswrapper[5052]: I1203 12:52:34.727701 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:52:35 crc kubenswrapper[5052]: I1203 12:52:35.470402 5052 scope.go:117] "RemoveContainer" containerID="962dbb81a097bd1736b9740b6fea46c9644f48229fd42d2dc734698665523114" Dec 03 12:52:35 crc kubenswrapper[5052]: I1203 12:52:35.496190 5052 scope.go:117] "RemoveContainer" containerID="c222a3a51a9f475798e3bef53ce522fe0044f6c1d5020c5fa20ba4e43d10b518" Dec 03 12:52:35 crc kubenswrapper[5052]: I1203 12:52:35.518637 5052 scope.go:117] "RemoveContainer" containerID="805cd92e51a39e19c7bde5260dca999018b3ba92ebb95c79b560a3986eb6d94a" Dec 03 12:52:36 crc kubenswrapper[5052]: I1203 12:52:36.510303 5052 generic.go:334] "Generic (PLEG): container finished" podID="cd023d37-ce83-4ca4-ab52-f89e94d0b721" containerID="1ffdd15a3a5713be01c664e2b6740ea79e1cb391fcfc3f99d37c27c80a734363" exitCode=0 Dec 03 12:52:36 crc kubenswrapper[5052]: I1203 12:52:36.510366 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"cd023d37-ce83-4ca4-ab52-f89e94d0b721","Type":"ContainerDied","Data":"1ffdd15a3a5713be01c664e2b6740ea79e1cb391fcfc3f99d37c27c80a734363"} Dec 03 12:52:36 crc kubenswrapper[5052]: I1203 12:52:36.511474 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="165e6ac4-9875-4750-afb3-c77088cdc884" containerName="ceilometer-central-agent" containerID="cri-o://d14603df1efe703bc136b352c1731d60aa5b4abdbd67a9414cc8be1a2069f9fa" gracePeriod=30 Dec 03 12:52:36 crc kubenswrapper[5052]: I1203 12:52:36.511643 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="165e6ac4-9875-4750-afb3-c77088cdc884" containerName="sg-core" containerID="cri-o://66e47b99705eb8f855b5604562315e914b5edc1fc44a531356994b77c3a644a8" gracePeriod=30 Dec 03 12:52:36 crc kubenswrapper[5052]: I1203 12:52:36.511668 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="165e6ac4-9875-4750-afb3-c77088cdc884" containerName="ceilometer-notification-agent" containerID="cri-o://a3c0984ec47119deb78f745f0836fd837688c8ac4b2d2615e858c0c3c8890cc6" gracePeriod=30 Dec 03 12:52:36 crc kubenswrapper[5052]: I1203 12:52:36.511709 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="165e6ac4-9875-4750-afb3-c77088cdc884" containerName="proxy-httpd" containerID="cri-o://9690e5377d7ca39f7e310c5292008842e2f2c8ddceb5a7692800c84e88d1cacb" gracePeriod=30 Dec 03 12:52:36 crc kubenswrapper[5052]: I1203 12:52:36.976831 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:52:37 crc kubenswrapper[5052]: I1203 12:52:37.151347 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/cd023d37-ce83-4ca4-ab52-f89e94d0b721-httpd-run\") pod \"cd023d37-ce83-4ca4-ab52-f89e94d0b721\" (UID: \"cd023d37-ce83-4ca4-ab52-f89e94d0b721\") " Dec 03 12:52:37 crc kubenswrapper[5052]: I1203 12:52:37.151424 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/cd023d37-ce83-4ca4-ab52-f89e94d0b721-logs\") pod \"cd023d37-ce83-4ca4-ab52-f89e94d0b721\" (UID: \"cd023d37-ce83-4ca4-ab52-f89e94d0b721\") " Dec 03 12:52:37 crc kubenswrapper[5052]: I1203 12:52:37.151489 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/cd023d37-ce83-4ca4-ab52-f89e94d0b721-combined-ca-bundle\") pod \"cd023d37-ce83-4ca4-ab52-f89e94d0b721\" (UID: \"cd023d37-ce83-4ca4-ab52-f89e94d0b721\") " Dec 03 12:52:37 crc kubenswrapper[5052]: I1203 12:52:37.151544 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/cd023d37-ce83-4ca4-ab52-f89e94d0b721-config-data\") pod \"cd023d37-ce83-4ca4-ab52-f89e94d0b721\" (UID: \"cd023d37-ce83-4ca4-ab52-f89e94d0b721\") " Dec 03 12:52:37 crc kubenswrapper[5052]: I1203 12:52:37.151616 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/cd023d37-ce83-4ca4-ab52-f89e94d0b721-public-tls-certs\") pod \"cd023d37-ce83-4ca4-ab52-f89e94d0b721\" (UID: \"cd023d37-ce83-4ca4-ab52-f89e94d0b721\") " Dec 03 12:52:37 crc kubenswrapper[5052]: I1203 12:52:37.151654 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"glance\" (UniqueName: \"kubernetes.io/local-volume/local-storage19-crc\") pod \"cd023d37-ce83-4ca4-ab52-f89e94d0b721\" (UID: \"cd023d37-ce83-4ca4-ab52-f89e94d0b721\") " Dec 03 12:52:37 crc kubenswrapper[5052]: I1203 12:52:37.151748 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/cd023d37-ce83-4ca4-ab52-f89e94d0b721-scripts\") pod \"cd023d37-ce83-4ca4-ab52-f89e94d0b721\" (UID: \"cd023d37-ce83-4ca4-ab52-f89e94d0b721\") " Dec 03 12:52:37 crc kubenswrapper[5052]: I1203 12:52:37.151827 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bkssv\" (UniqueName: \"kubernetes.io/projected/cd023d37-ce83-4ca4-ab52-f89e94d0b721-kube-api-access-bkssv\") pod \"cd023d37-ce83-4ca4-ab52-f89e94d0b721\" (UID: \"cd023d37-ce83-4ca4-ab52-f89e94d0b721\") " Dec 03 12:52:37 crc kubenswrapper[5052]: I1203 12:52:37.151877 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd023d37-ce83-4ca4-ab52-f89e94d0b721-httpd-run" (OuterVolumeSpecName: "httpd-run") pod "cd023d37-ce83-4ca4-ab52-f89e94d0b721" (UID: "cd023d37-ce83-4ca4-ab52-f89e94d0b721"). InnerVolumeSpecName "httpd-run". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:52:37 crc kubenswrapper[5052]: I1203 12:52:37.151964 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd023d37-ce83-4ca4-ab52-f89e94d0b721-logs" (OuterVolumeSpecName: "logs") pod "cd023d37-ce83-4ca4-ab52-f89e94d0b721" (UID: "cd023d37-ce83-4ca4-ab52-f89e94d0b721"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:52:37 crc kubenswrapper[5052]: I1203 12:52:37.152488 5052 reconciler_common.go:293] "Volume detached for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/cd023d37-ce83-4ca4-ab52-f89e94d0b721-httpd-run\") on node \"crc\" DevicePath \"\"" Dec 03 12:52:37 crc kubenswrapper[5052]: I1203 12:52:37.152501 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/cd023d37-ce83-4ca4-ab52-f89e94d0b721-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:52:37 crc kubenswrapper[5052]: I1203 12:52:37.158434 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:52:37 crc kubenswrapper[5052]: I1203 12:52:37.158538 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage19-crc" (OuterVolumeSpecName: "glance") pod "cd023d37-ce83-4ca4-ab52-f89e94d0b721" (UID: "cd023d37-ce83-4ca4-ab52-f89e94d0b721"). InnerVolumeSpecName "local-storage19-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 12:52:37 crc kubenswrapper[5052]: I1203 12:52:37.159466 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cd023d37-ce83-4ca4-ab52-f89e94d0b721-kube-api-access-bkssv" (OuterVolumeSpecName: "kube-api-access-bkssv") pod "cd023d37-ce83-4ca4-ab52-f89e94d0b721" (UID: "cd023d37-ce83-4ca4-ab52-f89e94d0b721"). InnerVolumeSpecName "kube-api-access-bkssv". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:52:37 crc kubenswrapper[5052]: I1203 12:52:37.170745 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cd023d37-ce83-4ca4-ab52-f89e94d0b721-scripts" (OuterVolumeSpecName: "scripts") pod "cd023d37-ce83-4ca4-ab52-f89e94d0b721" (UID: "cd023d37-ce83-4ca4-ab52-f89e94d0b721"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:52:37 crc kubenswrapper[5052]: I1203 12:52:37.188635 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cd023d37-ce83-4ca4-ab52-f89e94d0b721-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "cd023d37-ce83-4ca4-ab52-f89e94d0b721" (UID: "cd023d37-ce83-4ca4-ab52-f89e94d0b721"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:52:37 crc kubenswrapper[5052]: I1203 12:52:37.235007 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cd023d37-ce83-4ca4-ab52-f89e94d0b721-config-data" (OuterVolumeSpecName: "config-data") pod "cd023d37-ce83-4ca4-ab52-f89e94d0b721" (UID: "cd023d37-ce83-4ca4-ab52-f89e94d0b721"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:52:37 crc kubenswrapper[5052]: I1203 12:52:37.245196 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cd023d37-ce83-4ca4-ab52-f89e94d0b721-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "cd023d37-ce83-4ca4-ab52-f89e94d0b721" (UID: "cd023d37-ce83-4ca4-ab52-f89e94d0b721"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:52:37 crc kubenswrapper[5052]: I1203 12:52:37.253970 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c266x\" (UniqueName: \"kubernetes.io/projected/fc57d1ed-34a0-42b5-b1be-5bf914492701-kube-api-access-c266x\") pod \"fc57d1ed-34a0-42b5-b1be-5bf914492701\" (UID: \"fc57d1ed-34a0-42b5-b1be-5bf914492701\") " Dec 03 12:52:37 crc kubenswrapper[5052]: I1203 12:52:37.254498 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/cd023d37-ce83-4ca4-ab52-f89e94d0b721-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:52:37 crc kubenswrapper[5052]: I1203 12:52:37.254537 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/cd023d37-ce83-4ca4-ab52-f89e94d0b721-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:52:37 crc kubenswrapper[5052]: I1203 12:52:37.254550 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/cd023d37-ce83-4ca4-ab52-f89e94d0b721-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:52:37 crc kubenswrapper[5052]: I1203 12:52:37.254569 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage19-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage19-crc\") on node \"crc\" " Dec 03 12:52:37 crc kubenswrapper[5052]: I1203 12:52:37.254578 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/cd023d37-ce83-4ca4-ab52-f89e94d0b721-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:52:37 crc kubenswrapper[5052]: I1203 12:52:37.254587 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-bkssv\" (UniqueName: \"kubernetes.io/projected/cd023d37-ce83-4ca4-ab52-f89e94d0b721-kube-api-access-bkssv\") on node \"crc\" DevicePath \"\"" Dec 03 12:52:37 crc kubenswrapper[5052]: I1203 12:52:37.261540 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fc57d1ed-34a0-42b5-b1be-5bf914492701-kube-api-access-c266x" (OuterVolumeSpecName: "kube-api-access-c266x") pod "fc57d1ed-34a0-42b5-b1be-5bf914492701" (UID: "fc57d1ed-34a0-42b5-b1be-5bf914492701"). InnerVolumeSpecName "kube-api-access-c266x". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:52:37 crc kubenswrapper[5052]: I1203 12:52:37.274386 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage19-crc" (UniqueName: "kubernetes.io/local-volume/local-storage19-crc") on node "crc" Dec 03 12:52:37 crc kubenswrapper[5052]: I1203 12:52:37.355400 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/fc57d1ed-34a0-42b5-b1be-5bf914492701-httpd-run\") pod \"fc57d1ed-34a0-42b5-b1be-5bf914492701\" (UID: \"fc57d1ed-34a0-42b5-b1be-5bf914492701\") " Dec 03 12:52:37 crc kubenswrapper[5052]: I1203 12:52:37.355475 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"glance\" (UniqueName: \"kubernetes.io/local-volume/local-storage06-crc\") pod \"fc57d1ed-34a0-42b5-b1be-5bf914492701\" (UID: \"fc57d1ed-34a0-42b5-b1be-5bf914492701\") " Dec 03 12:52:37 crc kubenswrapper[5052]: I1203 12:52:37.355574 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/fc57d1ed-34a0-42b5-b1be-5bf914492701-combined-ca-bundle\") pod \"fc57d1ed-34a0-42b5-b1be-5bf914492701\" (UID: \"fc57d1ed-34a0-42b5-b1be-5bf914492701\") " Dec 03 12:52:37 crc kubenswrapper[5052]: I1203 12:52:37.355689 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/fc57d1ed-34a0-42b5-b1be-5bf914492701-logs\") pod \"fc57d1ed-34a0-42b5-b1be-5bf914492701\" (UID: \"fc57d1ed-34a0-42b5-b1be-5bf914492701\") " Dec 03 12:52:37 crc kubenswrapper[5052]: I1203 12:52:37.355717 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/fc57d1ed-34a0-42b5-b1be-5bf914492701-internal-tls-certs\") pod \"fc57d1ed-34a0-42b5-b1be-5bf914492701\" (UID: \"fc57d1ed-34a0-42b5-b1be-5bf914492701\") " Dec 03 12:52:37 crc kubenswrapper[5052]: I1203 12:52:37.355777 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/fc57d1ed-34a0-42b5-b1be-5bf914492701-config-data\") pod \"fc57d1ed-34a0-42b5-b1be-5bf914492701\" (UID: \"fc57d1ed-34a0-42b5-b1be-5bf914492701\") " Dec 03 12:52:37 crc kubenswrapper[5052]: I1203 12:52:37.355776 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fc57d1ed-34a0-42b5-b1be-5bf914492701-httpd-run" (OuterVolumeSpecName: "httpd-run") pod "fc57d1ed-34a0-42b5-b1be-5bf914492701" (UID: "fc57d1ed-34a0-42b5-b1be-5bf914492701"). InnerVolumeSpecName "httpd-run". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:52:37 crc kubenswrapper[5052]: I1203 12:52:37.355807 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/fc57d1ed-34a0-42b5-b1be-5bf914492701-scripts\") pod \"fc57d1ed-34a0-42b5-b1be-5bf914492701\" (UID: \"fc57d1ed-34a0-42b5-b1be-5bf914492701\") " Dec 03 12:52:37 crc kubenswrapper[5052]: I1203 12:52:37.356113 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fc57d1ed-34a0-42b5-b1be-5bf914492701-logs" (OuterVolumeSpecName: "logs") pod "fc57d1ed-34a0-42b5-b1be-5bf914492701" (UID: "fc57d1ed-34a0-42b5-b1be-5bf914492701"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:52:37 crc kubenswrapper[5052]: I1203 12:52:37.356528 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-c266x\" (UniqueName: \"kubernetes.io/projected/fc57d1ed-34a0-42b5-b1be-5bf914492701-kube-api-access-c266x\") on node \"crc\" DevicePath \"\"" Dec 03 12:52:37 crc kubenswrapper[5052]: I1203 12:52:37.356552 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/fc57d1ed-34a0-42b5-b1be-5bf914492701-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:52:37 crc kubenswrapper[5052]: I1203 12:52:37.356567 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage19-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage19-crc\") on node \"crc\" DevicePath \"\"" Dec 03 12:52:37 crc kubenswrapper[5052]: I1203 12:52:37.356576 5052 reconciler_common.go:293] "Volume detached for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/fc57d1ed-34a0-42b5-b1be-5bf914492701-httpd-run\") on node \"crc\" DevicePath \"\"" Dec 03 12:52:37 crc kubenswrapper[5052]: I1203 12:52:37.361638 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fc57d1ed-34a0-42b5-b1be-5bf914492701-scripts" (OuterVolumeSpecName: "scripts") pod "fc57d1ed-34a0-42b5-b1be-5bf914492701" (UID: "fc57d1ed-34a0-42b5-b1be-5bf914492701"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:52:37 crc kubenswrapper[5052]: I1203 12:52:37.361732 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage06-crc" (OuterVolumeSpecName: "glance") pod "fc57d1ed-34a0-42b5-b1be-5bf914492701" (UID: "fc57d1ed-34a0-42b5-b1be-5bf914492701"). InnerVolumeSpecName "local-storage06-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 12:52:37 crc kubenswrapper[5052]: I1203 12:52:37.383232 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fc57d1ed-34a0-42b5-b1be-5bf914492701-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "fc57d1ed-34a0-42b5-b1be-5bf914492701" (UID: "fc57d1ed-34a0-42b5-b1be-5bf914492701"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:52:37 crc kubenswrapper[5052]: I1203 12:52:37.412313 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fc57d1ed-34a0-42b5-b1be-5bf914492701-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "fc57d1ed-34a0-42b5-b1be-5bf914492701" (UID: "fc57d1ed-34a0-42b5-b1be-5bf914492701"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:52:37 crc kubenswrapper[5052]: I1203 12:52:37.416055 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fc57d1ed-34a0-42b5-b1be-5bf914492701-config-data" (OuterVolumeSpecName: "config-data") pod "fc57d1ed-34a0-42b5-b1be-5bf914492701" (UID: "fc57d1ed-34a0-42b5-b1be-5bf914492701"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:52:37 crc kubenswrapper[5052]: I1203 12:52:37.458099 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/fc57d1ed-34a0-42b5-b1be-5bf914492701-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:52:37 crc kubenswrapper[5052]: I1203 12:52:37.458188 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/fc57d1ed-34a0-42b5-b1be-5bf914492701-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:52:37 crc kubenswrapper[5052]: I1203 12:52:37.458206 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/fc57d1ed-34a0-42b5-b1be-5bf914492701-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:52:37 crc kubenswrapper[5052]: I1203 12:52:37.458252 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage06-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage06-crc\") on node \"crc\" " Dec 03 12:52:37 crc kubenswrapper[5052]: I1203 12:52:37.458267 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/fc57d1ed-34a0-42b5-b1be-5bf914492701-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:52:37 crc kubenswrapper[5052]: I1203 12:52:37.505405 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage06-crc" (UniqueName: "kubernetes.io/local-volume/local-storage06-crc") on node "crc" Dec 03 12:52:37 crc kubenswrapper[5052]: I1203 12:52:37.536751 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"cd023d37-ce83-4ca4-ab52-f89e94d0b721","Type":"ContainerDied","Data":"9bff0490e9efac8c13794ea92cd1e5524942afd11c37264a3f09ab026012843e"} Dec 03 12:52:37 crc kubenswrapper[5052]: I1203 12:52:37.536802 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:52:37 crc kubenswrapper[5052]: I1203 12:52:37.536872 5052 scope.go:117] "RemoveContainer" containerID="1ffdd15a3a5713be01c664e2b6740ea79e1cb391fcfc3f99d37c27c80a734363" Dec 03 12:52:37 crc kubenswrapper[5052]: I1203 12:52:37.543988 5052 generic.go:334] "Generic (PLEG): container finished" podID="165e6ac4-9875-4750-afb3-c77088cdc884" containerID="9690e5377d7ca39f7e310c5292008842e2f2c8ddceb5a7692800c84e88d1cacb" exitCode=0 Dec 03 12:52:37 crc kubenswrapper[5052]: I1203 12:52:37.544029 5052 generic.go:334] "Generic (PLEG): container finished" podID="165e6ac4-9875-4750-afb3-c77088cdc884" containerID="66e47b99705eb8f855b5604562315e914b5edc1fc44a531356994b77c3a644a8" exitCode=2 Dec 03 12:52:37 crc kubenswrapper[5052]: I1203 12:52:37.544037 5052 generic.go:334] "Generic (PLEG): container finished" podID="165e6ac4-9875-4750-afb3-c77088cdc884" containerID="a3c0984ec47119deb78f745f0836fd837688c8ac4b2d2615e858c0c3c8890cc6" exitCode=0 Dec 03 12:52:37 crc kubenswrapper[5052]: I1203 12:52:37.544061 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"165e6ac4-9875-4750-afb3-c77088cdc884","Type":"ContainerDied","Data":"9690e5377d7ca39f7e310c5292008842e2f2c8ddceb5a7692800c84e88d1cacb"} Dec 03 12:52:37 crc kubenswrapper[5052]: I1203 12:52:37.544568 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"165e6ac4-9875-4750-afb3-c77088cdc884","Type":"ContainerDied","Data":"66e47b99705eb8f855b5604562315e914b5edc1fc44a531356994b77c3a644a8"} Dec 03 12:52:37 crc kubenswrapper[5052]: I1203 12:52:37.544593 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"165e6ac4-9875-4750-afb3-c77088cdc884","Type":"ContainerDied","Data":"a3c0984ec47119deb78f745f0836fd837688c8ac4b2d2615e858c0c3c8890cc6"} Dec 03 12:52:37 crc kubenswrapper[5052]: I1203 12:52:37.546320 5052 generic.go:334] "Generic (PLEG): container finished" podID="fc57d1ed-34a0-42b5-b1be-5bf914492701" containerID="b2120f51b650e3bfe33d9abfe02816028d28060266d5498f4adcf7f4a0048dab" exitCode=0 Dec 03 12:52:37 crc kubenswrapper[5052]: I1203 12:52:37.546372 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"fc57d1ed-34a0-42b5-b1be-5bf914492701","Type":"ContainerDied","Data":"b2120f51b650e3bfe33d9abfe02816028d28060266d5498f4adcf7f4a0048dab"} Dec 03 12:52:37 crc kubenswrapper[5052]: I1203 12:52:37.546399 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"fc57d1ed-34a0-42b5-b1be-5bf914492701","Type":"ContainerDied","Data":"04405b5bbd959325af164a93fe039b735439fcd2c5549ded39bf0b3f9298962f"} Dec 03 12:52:37 crc kubenswrapper[5052]: I1203 12:52:37.546501 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:52:37 crc kubenswrapper[5052]: I1203 12:52:37.559752 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage06-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage06-crc\") on node \"crc\" DevicePath \"\"" Dec 03 12:52:37 crc kubenswrapper[5052]: I1203 12:52:37.566848 5052 scope.go:117] "RemoveContainer" containerID="c6b845ca325df0412cbeac659e0056c26815c15f2c2df4fafcf3c74b16e2590a" Dec 03 12:52:37 crc kubenswrapper[5052]: I1203 12:52:37.584928 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:52:37 crc kubenswrapper[5052]: I1203 12:52:37.597897 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:52:37 crc kubenswrapper[5052]: I1203 12:52:37.599671 5052 scope.go:117] "RemoveContainer" containerID="b2120f51b650e3bfe33d9abfe02816028d28060266d5498f4adcf7f4a0048dab" Dec 03 12:52:37 crc kubenswrapper[5052]: I1203 12:52:37.615138 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:52:37 crc kubenswrapper[5052]: I1203 12:52:37.625305 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:52:37 crc kubenswrapper[5052]: I1203 12:52:37.635924 5052 scope.go:117] "RemoveContainer" containerID="5e85ff655790c57c0d7ecc6f00fca41e5f274eea76a18c0b7ebbd12438ea1d83" Dec 03 12:52:37 crc kubenswrapper[5052]: I1203 12:52:37.641125 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:52:37 crc kubenswrapper[5052]: E1203 12:52:37.641672 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="cd023d37-ce83-4ca4-ab52-f89e94d0b721" containerName="glance-httpd" Dec 03 12:52:37 crc kubenswrapper[5052]: I1203 12:52:37.641697 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="cd023d37-ce83-4ca4-ab52-f89e94d0b721" containerName="glance-httpd" Dec 03 12:52:37 crc kubenswrapper[5052]: E1203 12:52:37.641717 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="fc57d1ed-34a0-42b5-b1be-5bf914492701" containerName="glance-log" Dec 03 12:52:37 crc kubenswrapper[5052]: I1203 12:52:37.641726 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="fc57d1ed-34a0-42b5-b1be-5bf914492701" containerName="glance-log" Dec 03 12:52:37 crc kubenswrapper[5052]: E1203 12:52:37.641746 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="cd023d37-ce83-4ca4-ab52-f89e94d0b721" containerName="glance-log" Dec 03 12:52:37 crc kubenswrapper[5052]: I1203 12:52:37.641755 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="cd023d37-ce83-4ca4-ab52-f89e94d0b721" containerName="glance-log" Dec 03 12:52:37 crc kubenswrapper[5052]: E1203 12:52:37.641775 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="fc57d1ed-34a0-42b5-b1be-5bf914492701" containerName="glance-httpd" Dec 03 12:52:37 crc kubenswrapper[5052]: I1203 12:52:37.641783 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="fc57d1ed-34a0-42b5-b1be-5bf914492701" containerName="glance-httpd" Dec 03 12:52:37 crc kubenswrapper[5052]: I1203 12:52:37.642047 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="fc57d1ed-34a0-42b5-b1be-5bf914492701" containerName="glance-log" Dec 03 12:52:37 crc kubenswrapper[5052]: I1203 12:52:37.642084 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="cd023d37-ce83-4ca4-ab52-f89e94d0b721" containerName="glance-log" Dec 03 12:52:37 crc kubenswrapper[5052]: I1203 12:52:37.642108 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="cd023d37-ce83-4ca4-ab52-f89e94d0b721" containerName="glance-httpd" Dec 03 12:52:37 crc kubenswrapper[5052]: I1203 12:52:37.642350 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="fc57d1ed-34a0-42b5-b1be-5bf914492701" containerName="glance-httpd" Dec 03 12:52:37 crc kubenswrapper[5052]: I1203 12:52:37.643642 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:52:37 crc kubenswrapper[5052]: I1203 12:52:37.647590 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-scripts" Dec 03 12:52:37 crc kubenswrapper[5052]: I1203 12:52:37.647733 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-glance-dockercfg-zph2l" Dec 03 12:52:37 crc kubenswrapper[5052]: I1203 12:52:37.647899 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-default-external-config-data" Dec 03 12:52:37 crc kubenswrapper[5052]: I1203 12:52:37.647995 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-glance-default-public-svc" Dec 03 12:52:37 crc kubenswrapper[5052]: I1203 12:52:37.659756 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:52:37 crc kubenswrapper[5052]: I1203 12:52:37.661303 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:52:37 crc kubenswrapper[5052]: I1203 12:52:37.663984 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"glance-default-internal-config-data" Dec 03 12:52:37 crc kubenswrapper[5052]: I1203 12:52:37.664151 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-glance-default-internal-svc" Dec 03 12:52:37 crc kubenswrapper[5052]: I1203 12:52:37.666589 5052 scope.go:117] "RemoveContainer" containerID="b2120f51b650e3bfe33d9abfe02816028d28060266d5498f4adcf7f4a0048dab" Dec 03 12:52:37 crc kubenswrapper[5052]: E1203 12:52:37.667982 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b2120f51b650e3bfe33d9abfe02816028d28060266d5498f4adcf7f4a0048dab\": container with ID starting with b2120f51b650e3bfe33d9abfe02816028d28060266d5498f4adcf7f4a0048dab not found: ID does not exist" containerID="b2120f51b650e3bfe33d9abfe02816028d28060266d5498f4adcf7f4a0048dab" Dec 03 12:52:37 crc kubenswrapper[5052]: I1203 12:52:37.668042 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b2120f51b650e3bfe33d9abfe02816028d28060266d5498f4adcf7f4a0048dab"} err="failed to get container status \"b2120f51b650e3bfe33d9abfe02816028d28060266d5498f4adcf7f4a0048dab\": rpc error: code = NotFound desc = could not find container \"b2120f51b650e3bfe33d9abfe02816028d28060266d5498f4adcf7f4a0048dab\": container with ID starting with b2120f51b650e3bfe33d9abfe02816028d28060266d5498f4adcf7f4a0048dab not found: ID does not exist" Dec 03 12:52:37 crc kubenswrapper[5052]: I1203 12:52:37.668067 5052 scope.go:117] "RemoveContainer" containerID="5e85ff655790c57c0d7ecc6f00fca41e5f274eea76a18c0b7ebbd12438ea1d83" Dec 03 12:52:37 crc kubenswrapper[5052]: E1203 12:52:37.668740 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5e85ff655790c57c0d7ecc6f00fca41e5f274eea76a18c0b7ebbd12438ea1d83\": container with ID starting with 5e85ff655790c57c0d7ecc6f00fca41e5f274eea76a18c0b7ebbd12438ea1d83 not found: ID does not exist" containerID="5e85ff655790c57c0d7ecc6f00fca41e5f274eea76a18c0b7ebbd12438ea1d83" Dec 03 12:52:37 crc kubenswrapper[5052]: I1203 12:52:37.668772 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5e85ff655790c57c0d7ecc6f00fca41e5f274eea76a18c0b7ebbd12438ea1d83"} err="failed to get container status \"5e85ff655790c57c0d7ecc6f00fca41e5f274eea76a18c0b7ebbd12438ea1d83\": rpc error: code = NotFound desc = could not find container \"5e85ff655790c57c0d7ecc6f00fca41e5f274eea76a18c0b7ebbd12438ea1d83\": container with ID starting with 5e85ff655790c57c0d7ecc6f00fca41e5f274eea76a18c0b7ebbd12438ea1d83 not found: ID does not exist" Dec 03 12:52:37 crc kubenswrapper[5052]: I1203 12:52:37.690452 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cd023d37-ce83-4ca4-ab52-f89e94d0b721" path="/var/lib/kubelet/pods/cd023d37-ce83-4ca4-ab52-f89e94d0b721/volumes" Dec 03 12:52:37 crc kubenswrapper[5052]: I1203 12:52:37.691668 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fc57d1ed-34a0-42b5-b1be-5bf914492701" path="/var/lib/kubelet/pods/fc57d1ed-34a0-42b5-b1be-5bf914492701/volumes" Dec 03 12:52:37 crc kubenswrapper[5052]: I1203 12:52:37.692365 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:52:37 crc kubenswrapper[5052]: I1203 12:52:37.702931 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:52:37 crc kubenswrapper[5052]: I1203 12:52:37.762711 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/cfc68960-6006-486c-bc03-848875d3bf5b-logs\") pod \"glance-default-external-api-0\" (UID: \"cfc68960-6006-486c-bc03-848875d3bf5b\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:52:37 crc kubenswrapper[5052]: I1203 12:52:37.762776 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/981c3156-805a-44d1-b4ec-f688a8bc3cf8-logs\") pod \"glance-default-internal-api-0\" (UID: \"981c3156-805a-44d1-b4ec-f688a8bc3cf8\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:52:37 crc kubenswrapper[5052]: I1203 12:52:37.762819 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage19-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage19-crc\") pod \"glance-default-external-api-0\" (UID: \"cfc68960-6006-486c-bc03-848875d3bf5b\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:52:37 crc kubenswrapper[5052]: I1203 12:52:37.762878 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/cfc68960-6006-486c-bc03-848875d3bf5b-config-data\") pod \"glance-default-external-api-0\" (UID: \"cfc68960-6006-486c-bc03-848875d3bf5b\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:52:37 crc kubenswrapper[5052]: I1203 12:52:37.762909 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dvtfh\" (UniqueName: \"kubernetes.io/projected/981c3156-805a-44d1-b4ec-f688a8bc3cf8-kube-api-access-dvtfh\") pod \"glance-default-internal-api-0\" (UID: \"981c3156-805a-44d1-b4ec-f688a8bc3cf8\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:52:37 crc kubenswrapper[5052]: I1203 12:52:37.762998 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage06-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage06-crc\") pod \"glance-default-internal-api-0\" (UID: \"981c3156-805a-44d1-b4ec-f688a8bc3cf8\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:52:37 crc kubenswrapper[5052]: I1203 12:52:37.763024 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/981c3156-805a-44d1-b4ec-f688a8bc3cf8-httpd-run\") pod \"glance-default-internal-api-0\" (UID: \"981c3156-805a-44d1-b4ec-f688a8bc3cf8\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:52:37 crc kubenswrapper[5052]: I1203 12:52:37.763051 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/981c3156-805a-44d1-b4ec-f688a8bc3cf8-config-data\") pod \"glance-default-internal-api-0\" (UID: \"981c3156-805a-44d1-b4ec-f688a8bc3cf8\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:52:37 crc kubenswrapper[5052]: I1203 12:52:37.763086 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/cfc68960-6006-486c-bc03-848875d3bf5b-combined-ca-bundle\") pod \"glance-default-external-api-0\" (UID: \"cfc68960-6006-486c-bc03-848875d3bf5b\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:52:37 crc kubenswrapper[5052]: I1203 12:52:37.763126 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/cfc68960-6006-486c-bc03-848875d3bf5b-httpd-run\") pod \"glance-default-external-api-0\" (UID: \"cfc68960-6006-486c-bc03-848875d3bf5b\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:52:37 crc kubenswrapper[5052]: I1203 12:52:37.763150 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/cfc68960-6006-486c-bc03-848875d3bf5b-public-tls-certs\") pod \"glance-default-external-api-0\" (UID: \"cfc68960-6006-486c-bc03-848875d3bf5b\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:52:37 crc kubenswrapper[5052]: I1203 12:52:37.763174 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/981c3156-805a-44d1-b4ec-f688a8bc3cf8-scripts\") pod \"glance-default-internal-api-0\" (UID: \"981c3156-805a-44d1-b4ec-f688a8bc3cf8\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:52:37 crc kubenswrapper[5052]: I1203 12:52:37.763198 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/981c3156-805a-44d1-b4ec-f688a8bc3cf8-internal-tls-certs\") pod \"glance-default-internal-api-0\" (UID: \"981c3156-805a-44d1-b4ec-f688a8bc3cf8\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:52:37 crc kubenswrapper[5052]: I1203 12:52:37.763229 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/981c3156-805a-44d1-b4ec-f688a8bc3cf8-combined-ca-bundle\") pod \"glance-default-internal-api-0\" (UID: \"981c3156-805a-44d1-b4ec-f688a8bc3cf8\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:52:37 crc kubenswrapper[5052]: I1203 12:52:37.763250 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wdxzl\" (UniqueName: \"kubernetes.io/projected/cfc68960-6006-486c-bc03-848875d3bf5b-kube-api-access-wdxzl\") pod \"glance-default-external-api-0\" (UID: \"cfc68960-6006-486c-bc03-848875d3bf5b\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:52:37 crc kubenswrapper[5052]: I1203 12:52:37.763285 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/cfc68960-6006-486c-bc03-848875d3bf5b-scripts\") pod \"glance-default-external-api-0\" (UID: \"cfc68960-6006-486c-bc03-848875d3bf5b\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:52:37 crc kubenswrapper[5052]: I1203 12:52:37.865404 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/cfc68960-6006-486c-bc03-848875d3bf5b-logs\") pod \"glance-default-external-api-0\" (UID: \"cfc68960-6006-486c-bc03-848875d3bf5b\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:52:37 crc kubenswrapper[5052]: I1203 12:52:37.865484 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/981c3156-805a-44d1-b4ec-f688a8bc3cf8-logs\") pod \"glance-default-internal-api-0\" (UID: \"981c3156-805a-44d1-b4ec-f688a8bc3cf8\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:52:37 crc kubenswrapper[5052]: I1203 12:52:37.865537 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage19-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage19-crc\") pod \"glance-default-external-api-0\" (UID: \"cfc68960-6006-486c-bc03-848875d3bf5b\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:52:37 crc kubenswrapper[5052]: I1203 12:52:37.865568 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/cfc68960-6006-486c-bc03-848875d3bf5b-config-data\") pod \"glance-default-external-api-0\" (UID: \"cfc68960-6006-486c-bc03-848875d3bf5b\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:52:37 crc kubenswrapper[5052]: I1203 12:52:37.865608 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-dvtfh\" (UniqueName: \"kubernetes.io/projected/981c3156-805a-44d1-b4ec-f688a8bc3cf8-kube-api-access-dvtfh\") pod \"glance-default-internal-api-0\" (UID: \"981c3156-805a-44d1-b4ec-f688a8bc3cf8\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:52:37 crc kubenswrapper[5052]: I1203 12:52:37.865646 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage06-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage06-crc\") pod \"glance-default-internal-api-0\" (UID: \"981c3156-805a-44d1-b4ec-f688a8bc3cf8\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:52:37 crc kubenswrapper[5052]: I1203 12:52:37.865683 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/981c3156-805a-44d1-b4ec-f688a8bc3cf8-httpd-run\") pod \"glance-default-internal-api-0\" (UID: \"981c3156-805a-44d1-b4ec-f688a8bc3cf8\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:52:37 crc kubenswrapper[5052]: I1203 12:52:37.865712 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/981c3156-805a-44d1-b4ec-f688a8bc3cf8-config-data\") pod \"glance-default-internal-api-0\" (UID: \"981c3156-805a-44d1-b4ec-f688a8bc3cf8\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:52:37 crc kubenswrapper[5052]: I1203 12:52:37.865755 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/cfc68960-6006-486c-bc03-848875d3bf5b-combined-ca-bundle\") pod \"glance-default-external-api-0\" (UID: \"cfc68960-6006-486c-bc03-848875d3bf5b\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:52:37 crc kubenswrapper[5052]: I1203 12:52:37.865775 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/cfc68960-6006-486c-bc03-848875d3bf5b-httpd-run\") pod \"glance-default-external-api-0\" (UID: \"cfc68960-6006-486c-bc03-848875d3bf5b\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:52:37 crc kubenswrapper[5052]: I1203 12:52:37.865792 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/cfc68960-6006-486c-bc03-848875d3bf5b-public-tls-certs\") pod \"glance-default-external-api-0\" (UID: \"cfc68960-6006-486c-bc03-848875d3bf5b\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:52:37 crc kubenswrapper[5052]: I1203 12:52:37.865810 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/981c3156-805a-44d1-b4ec-f688a8bc3cf8-scripts\") pod \"glance-default-internal-api-0\" (UID: \"981c3156-805a-44d1-b4ec-f688a8bc3cf8\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:52:37 crc kubenswrapper[5052]: I1203 12:52:37.865831 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/981c3156-805a-44d1-b4ec-f688a8bc3cf8-internal-tls-certs\") pod \"glance-default-internal-api-0\" (UID: \"981c3156-805a-44d1-b4ec-f688a8bc3cf8\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:52:37 crc kubenswrapper[5052]: I1203 12:52:37.865879 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/981c3156-805a-44d1-b4ec-f688a8bc3cf8-combined-ca-bundle\") pod \"glance-default-internal-api-0\" (UID: \"981c3156-805a-44d1-b4ec-f688a8bc3cf8\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:52:37 crc kubenswrapper[5052]: I1203 12:52:37.865895 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-wdxzl\" (UniqueName: \"kubernetes.io/projected/cfc68960-6006-486c-bc03-848875d3bf5b-kube-api-access-wdxzl\") pod \"glance-default-external-api-0\" (UID: \"cfc68960-6006-486c-bc03-848875d3bf5b\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:52:37 crc kubenswrapper[5052]: I1203 12:52:37.865929 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/cfc68960-6006-486c-bc03-848875d3bf5b-scripts\") pod \"glance-default-external-api-0\" (UID: \"cfc68960-6006-486c-bc03-848875d3bf5b\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:52:37 crc kubenswrapper[5052]: I1203 12:52:37.865952 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/cfc68960-6006-486c-bc03-848875d3bf5b-logs\") pod \"glance-default-external-api-0\" (UID: \"cfc68960-6006-486c-bc03-848875d3bf5b\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:52:37 crc kubenswrapper[5052]: I1203 12:52:37.866163 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage19-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage19-crc\") pod \"glance-default-external-api-0\" (UID: \"cfc68960-6006-486c-bc03-848875d3bf5b\") device mount path \"/mnt/openstack/pv19\"" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:52:37 crc kubenswrapper[5052]: I1203 12:52:37.866182 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage06-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage06-crc\") pod \"glance-default-internal-api-0\" (UID: \"981c3156-805a-44d1-b4ec-f688a8bc3cf8\") device mount path \"/mnt/openstack/pv06\"" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:52:37 crc kubenswrapper[5052]: I1203 12:52:37.866959 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/cfc68960-6006-486c-bc03-848875d3bf5b-httpd-run\") pod \"glance-default-external-api-0\" (UID: \"cfc68960-6006-486c-bc03-848875d3bf5b\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:52:37 crc kubenswrapper[5052]: I1203 12:52:37.867468 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/981c3156-805a-44d1-b4ec-f688a8bc3cf8-logs\") pod \"glance-default-internal-api-0\" (UID: \"981c3156-805a-44d1-b4ec-f688a8bc3cf8\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:52:37 crc kubenswrapper[5052]: I1203 12:52:37.867474 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/981c3156-805a-44d1-b4ec-f688a8bc3cf8-httpd-run\") pod \"glance-default-internal-api-0\" (UID: \"981c3156-805a-44d1-b4ec-f688a8bc3cf8\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:52:37 crc kubenswrapper[5052]: I1203 12:52:37.878126 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/cfc68960-6006-486c-bc03-848875d3bf5b-config-data\") pod \"glance-default-external-api-0\" (UID: \"cfc68960-6006-486c-bc03-848875d3bf5b\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:52:37 crc kubenswrapper[5052]: I1203 12:52:37.882841 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/981c3156-805a-44d1-b4ec-f688a8bc3cf8-combined-ca-bundle\") pod \"glance-default-internal-api-0\" (UID: \"981c3156-805a-44d1-b4ec-f688a8bc3cf8\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:52:37 crc kubenswrapper[5052]: I1203 12:52:37.882938 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/cfc68960-6006-486c-bc03-848875d3bf5b-public-tls-certs\") pod \"glance-default-external-api-0\" (UID: \"cfc68960-6006-486c-bc03-848875d3bf5b\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:52:37 crc kubenswrapper[5052]: I1203 12:52:37.883206 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/981c3156-805a-44d1-b4ec-f688a8bc3cf8-scripts\") pod \"glance-default-internal-api-0\" (UID: \"981c3156-805a-44d1-b4ec-f688a8bc3cf8\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:52:37 crc kubenswrapper[5052]: I1203 12:52:37.883257 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/981c3156-805a-44d1-b4ec-f688a8bc3cf8-internal-tls-certs\") pod \"glance-default-internal-api-0\" (UID: \"981c3156-805a-44d1-b4ec-f688a8bc3cf8\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:52:37 crc kubenswrapper[5052]: I1203 12:52:37.883646 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/cfc68960-6006-486c-bc03-848875d3bf5b-scripts\") pod \"glance-default-external-api-0\" (UID: \"cfc68960-6006-486c-bc03-848875d3bf5b\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:52:37 crc kubenswrapper[5052]: I1203 12:52:37.883863 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/981c3156-805a-44d1-b4ec-f688a8bc3cf8-config-data\") pod \"glance-default-internal-api-0\" (UID: \"981c3156-805a-44d1-b4ec-f688a8bc3cf8\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:52:37 crc kubenswrapper[5052]: I1203 12:52:37.888142 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-wdxzl\" (UniqueName: \"kubernetes.io/projected/cfc68960-6006-486c-bc03-848875d3bf5b-kube-api-access-wdxzl\") pod \"glance-default-external-api-0\" (UID: \"cfc68960-6006-486c-bc03-848875d3bf5b\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:52:37 crc kubenswrapper[5052]: I1203 12:52:37.888151 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-dvtfh\" (UniqueName: \"kubernetes.io/projected/981c3156-805a-44d1-b4ec-f688a8bc3cf8-kube-api-access-dvtfh\") pod \"glance-default-internal-api-0\" (UID: \"981c3156-805a-44d1-b4ec-f688a8bc3cf8\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:52:37 crc kubenswrapper[5052]: I1203 12:52:37.888424 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/cfc68960-6006-486c-bc03-848875d3bf5b-combined-ca-bundle\") pod \"glance-default-external-api-0\" (UID: \"cfc68960-6006-486c-bc03-848875d3bf5b\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:52:37 crc kubenswrapper[5052]: I1203 12:52:37.907570 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage06-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage06-crc\") pod \"glance-default-internal-api-0\" (UID: \"981c3156-805a-44d1-b4ec-f688a8bc3cf8\") " pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:52:37 crc kubenswrapper[5052]: I1203 12:52:37.908663 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage19-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage19-crc\") pod \"glance-default-external-api-0\" (UID: \"cfc68960-6006-486c-bc03-848875d3bf5b\") " pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:52:37 crc kubenswrapper[5052]: I1203 12:52:37.976957 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:52:37 crc kubenswrapper[5052]: I1203 12:52:37.993872 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:52:38 crc kubenswrapper[5052]: I1203 12:52:38.517718 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:52:38 crc kubenswrapper[5052]: I1203 12:52:38.576228 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"cfc68960-6006-486c-bc03-848875d3bf5b","Type":"ContainerStarted","Data":"2f252f5162cf02247034307341a5c641a56a84fe5d75bbe8d22f19b1c247c3cc"} Dec 03 12:52:38 crc kubenswrapper[5052]: I1203 12:52:38.596497 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:52:39 crc kubenswrapper[5052]: I1203 12:52:39.617250 5052 generic.go:334] "Generic (PLEG): container finished" podID="165e6ac4-9875-4750-afb3-c77088cdc884" containerID="d14603df1efe703bc136b352c1731d60aa5b4abdbd67a9414cc8be1a2069f9fa" exitCode=0 Dec 03 12:52:39 crc kubenswrapper[5052]: I1203 12:52:39.617941 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"165e6ac4-9875-4750-afb3-c77088cdc884","Type":"ContainerDied","Data":"d14603df1efe703bc136b352c1731d60aa5b4abdbd67a9414cc8be1a2069f9fa"} Dec 03 12:52:39 crc kubenswrapper[5052]: I1203 12:52:39.623734 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"981c3156-805a-44d1-b4ec-f688a8bc3cf8","Type":"ContainerStarted","Data":"f9733bd0ad6d0ebfc83dace1f11042df0c2c68afe8d846ce7105b5cf6873ea6f"} Dec 03 12:52:39 crc kubenswrapper[5052]: I1203 12:52:39.634301 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"cfc68960-6006-486c-bc03-848875d3bf5b","Type":"ContainerStarted","Data":"e4a30a41f3f5c0bc5325054597c83dfd7f01768f25444ec3d94e71d8f4c168ed"} Dec 03 12:52:39 crc kubenswrapper[5052]: I1203 12:52:39.770535 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:52:39 crc kubenswrapper[5052]: I1203 12:52:39.914364 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/165e6ac4-9875-4750-afb3-c77088cdc884-combined-ca-bundle\") pod \"165e6ac4-9875-4750-afb3-c77088cdc884\" (UID: \"165e6ac4-9875-4750-afb3-c77088cdc884\") " Dec 03 12:52:39 crc kubenswrapper[5052]: I1203 12:52:39.914430 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/165e6ac4-9875-4750-afb3-c77088cdc884-scripts\") pod \"165e6ac4-9875-4750-afb3-c77088cdc884\" (UID: \"165e6ac4-9875-4750-afb3-c77088cdc884\") " Dec 03 12:52:39 crc kubenswrapper[5052]: I1203 12:52:39.914621 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gcrw5\" (UniqueName: \"kubernetes.io/projected/165e6ac4-9875-4750-afb3-c77088cdc884-kube-api-access-gcrw5\") pod \"165e6ac4-9875-4750-afb3-c77088cdc884\" (UID: \"165e6ac4-9875-4750-afb3-c77088cdc884\") " Dec 03 12:52:39 crc kubenswrapper[5052]: I1203 12:52:39.914737 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/165e6ac4-9875-4750-afb3-c77088cdc884-run-httpd\") pod \"165e6ac4-9875-4750-afb3-c77088cdc884\" (UID: \"165e6ac4-9875-4750-afb3-c77088cdc884\") " Dec 03 12:52:39 crc kubenswrapper[5052]: I1203 12:52:39.914811 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/165e6ac4-9875-4750-afb3-c77088cdc884-log-httpd\") pod \"165e6ac4-9875-4750-afb3-c77088cdc884\" (UID: \"165e6ac4-9875-4750-afb3-c77088cdc884\") " Dec 03 12:52:39 crc kubenswrapper[5052]: I1203 12:52:39.914846 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/165e6ac4-9875-4750-afb3-c77088cdc884-config-data\") pod \"165e6ac4-9875-4750-afb3-c77088cdc884\" (UID: \"165e6ac4-9875-4750-afb3-c77088cdc884\") " Dec 03 12:52:39 crc kubenswrapper[5052]: I1203 12:52:39.914919 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/165e6ac4-9875-4750-afb3-c77088cdc884-ceilometer-tls-certs\") pod \"165e6ac4-9875-4750-afb3-c77088cdc884\" (UID: \"165e6ac4-9875-4750-afb3-c77088cdc884\") " Dec 03 12:52:39 crc kubenswrapper[5052]: I1203 12:52:39.914959 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/165e6ac4-9875-4750-afb3-c77088cdc884-sg-core-conf-yaml\") pod \"165e6ac4-9875-4750-afb3-c77088cdc884\" (UID: \"165e6ac4-9875-4750-afb3-c77088cdc884\") " Dec 03 12:52:39 crc kubenswrapper[5052]: I1203 12:52:39.916165 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/165e6ac4-9875-4750-afb3-c77088cdc884-log-httpd" (OuterVolumeSpecName: "log-httpd") pod "165e6ac4-9875-4750-afb3-c77088cdc884" (UID: "165e6ac4-9875-4750-afb3-c77088cdc884"). InnerVolumeSpecName "log-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:52:39 crc kubenswrapper[5052]: I1203 12:52:39.916274 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/165e6ac4-9875-4750-afb3-c77088cdc884-run-httpd" (OuterVolumeSpecName: "run-httpd") pod "165e6ac4-9875-4750-afb3-c77088cdc884" (UID: "165e6ac4-9875-4750-afb3-c77088cdc884"). InnerVolumeSpecName "run-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:52:39 crc kubenswrapper[5052]: I1203 12:52:39.919792 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/165e6ac4-9875-4750-afb3-c77088cdc884-kube-api-access-gcrw5" (OuterVolumeSpecName: "kube-api-access-gcrw5") pod "165e6ac4-9875-4750-afb3-c77088cdc884" (UID: "165e6ac4-9875-4750-afb3-c77088cdc884"). InnerVolumeSpecName "kube-api-access-gcrw5". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:52:39 crc kubenswrapper[5052]: I1203 12:52:39.919876 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/165e6ac4-9875-4750-afb3-c77088cdc884-scripts" (OuterVolumeSpecName: "scripts") pod "165e6ac4-9875-4750-afb3-c77088cdc884" (UID: "165e6ac4-9875-4750-afb3-c77088cdc884"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:52:39 crc kubenswrapper[5052]: I1203 12:52:39.971699 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/165e6ac4-9875-4750-afb3-c77088cdc884-ceilometer-tls-certs" (OuterVolumeSpecName: "ceilometer-tls-certs") pod "165e6ac4-9875-4750-afb3-c77088cdc884" (UID: "165e6ac4-9875-4750-afb3-c77088cdc884"). InnerVolumeSpecName "ceilometer-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:52:39 crc kubenswrapper[5052]: I1203 12:52:39.981979 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/165e6ac4-9875-4750-afb3-c77088cdc884-sg-core-conf-yaml" (OuterVolumeSpecName: "sg-core-conf-yaml") pod "165e6ac4-9875-4750-afb3-c77088cdc884" (UID: "165e6ac4-9875-4750-afb3-c77088cdc884"). InnerVolumeSpecName "sg-core-conf-yaml". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:52:40 crc kubenswrapper[5052]: I1203 12:52:40.017065 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-gcrw5\" (UniqueName: \"kubernetes.io/projected/165e6ac4-9875-4750-afb3-c77088cdc884-kube-api-access-gcrw5\") on node \"crc\" DevicePath \"\"" Dec 03 12:52:40 crc kubenswrapper[5052]: I1203 12:52:40.017312 5052 reconciler_common.go:293] "Volume detached for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/165e6ac4-9875-4750-afb3-c77088cdc884-run-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:52:40 crc kubenswrapper[5052]: I1203 12:52:40.017389 5052 reconciler_common.go:293] "Volume detached for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/165e6ac4-9875-4750-afb3-c77088cdc884-log-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:52:40 crc kubenswrapper[5052]: I1203 12:52:40.017486 5052 reconciler_common.go:293] "Volume detached for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/165e6ac4-9875-4750-afb3-c77088cdc884-ceilometer-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:52:40 crc kubenswrapper[5052]: I1203 12:52:40.017571 5052 reconciler_common.go:293] "Volume detached for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/165e6ac4-9875-4750-afb3-c77088cdc884-sg-core-conf-yaml\") on node \"crc\" DevicePath \"\"" Dec 03 12:52:40 crc kubenswrapper[5052]: I1203 12:52:40.017874 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/165e6ac4-9875-4750-afb3-c77088cdc884-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:52:40 crc kubenswrapper[5052]: I1203 12:52:40.018399 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/165e6ac4-9875-4750-afb3-c77088cdc884-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "165e6ac4-9875-4750-afb3-c77088cdc884" (UID: "165e6ac4-9875-4750-afb3-c77088cdc884"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:52:40 crc kubenswrapper[5052]: I1203 12:52:40.039947 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/165e6ac4-9875-4750-afb3-c77088cdc884-config-data" (OuterVolumeSpecName: "config-data") pod "165e6ac4-9875-4750-afb3-c77088cdc884" (UID: "165e6ac4-9875-4750-afb3-c77088cdc884"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:52:40 crc kubenswrapper[5052]: I1203 12:52:40.119488 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/165e6ac4-9875-4750-afb3-c77088cdc884-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:52:40 crc kubenswrapper[5052]: I1203 12:52:40.119542 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/165e6ac4-9875-4750-afb3-c77088cdc884-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:52:40 crc kubenswrapper[5052]: I1203 12:52:40.647910 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"981c3156-805a-44d1-b4ec-f688a8bc3cf8","Type":"ContainerStarted","Data":"91f773e54ad4ec35e1b76f707be75a9985aeb5b002636c9dd379bb836c31c220"} Dec 03 12:52:40 crc kubenswrapper[5052]: I1203 12:52:40.648187 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"981c3156-805a-44d1-b4ec-f688a8bc3cf8","Type":"ContainerStarted","Data":"691c45bd03a54c99bcffac732eb2f453f8046a69209f84bde56957c8b1cd60f9"} Dec 03 12:52:40 crc kubenswrapper[5052]: I1203 12:52:40.651376 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"cfc68960-6006-486c-bc03-848875d3bf5b","Type":"ContainerStarted","Data":"21405273563ddcf40333c0b6ddfcd119d733ef5beb5a7399909f33a285ac39e7"} Dec 03 12:52:40 crc kubenswrapper[5052]: I1203 12:52:40.656782 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"165e6ac4-9875-4750-afb3-c77088cdc884","Type":"ContainerDied","Data":"789597ebd1d30a87780d6928f58709eecf3ac8ebd6ec7832c998cf41b5da4a3e"} Dec 03 12:52:40 crc kubenswrapper[5052]: I1203 12:52:40.656836 5052 scope.go:117] "RemoveContainer" containerID="9690e5377d7ca39f7e310c5292008842e2f2c8ddceb5a7692800c84e88d1cacb" Dec 03 12:52:40 crc kubenswrapper[5052]: I1203 12:52:40.657011 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:52:40 crc kubenswrapper[5052]: I1203 12:52:40.682253 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/glance-default-internal-api-0" podStartSLOduration=3.6818061269999998 podStartE2EDuration="3.681806127s" podCreationTimestamp="2025-12-03 12:52:37 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:52:40.672845141 +0000 UTC m=+5551.261086925" watchObservedRunningTime="2025-12-03 12:52:40.681806127 +0000 UTC m=+5551.270047891" Dec 03 12:52:40 crc kubenswrapper[5052]: I1203 12:52:40.695847 5052 scope.go:117] "RemoveContainer" containerID="66e47b99705eb8f855b5604562315e914b5edc1fc44a531356994b77c3a644a8" Dec 03 12:52:40 crc kubenswrapper[5052]: I1203 12:52:40.721462 5052 scope.go:117] "RemoveContainer" containerID="a3c0984ec47119deb78f745f0836fd837688c8ac4b2d2615e858c0c3c8890cc6" Dec 03 12:52:40 crc kubenswrapper[5052]: I1203 12:52:40.726664 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/glance-default-external-api-0" podStartSLOduration=3.726642629 podStartE2EDuration="3.726642629s" podCreationTimestamp="2025-12-03 12:52:37 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:52:40.695978093 +0000 UTC m=+5551.284219867" watchObservedRunningTime="2025-12-03 12:52:40.726642629 +0000 UTC m=+5551.314884393" Dec 03 12:52:40 crc kubenswrapper[5052]: I1203 12:52:40.757562 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:52:40 crc kubenswrapper[5052]: I1203 12:52:40.759128 5052 scope.go:117] "RemoveContainer" containerID="d14603df1efe703bc136b352c1731d60aa5b4abdbd67a9414cc8be1a2069f9fa" Dec 03 12:52:40 crc kubenswrapper[5052]: I1203 12:52:40.766957 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:52:40 crc kubenswrapper[5052]: I1203 12:52:40.786694 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:52:40 crc kubenswrapper[5052]: E1203 12:52:40.787117 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="165e6ac4-9875-4750-afb3-c77088cdc884" containerName="sg-core" Dec 03 12:52:40 crc kubenswrapper[5052]: I1203 12:52:40.787134 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="165e6ac4-9875-4750-afb3-c77088cdc884" containerName="sg-core" Dec 03 12:52:40 crc kubenswrapper[5052]: E1203 12:52:40.787155 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="165e6ac4-9875-4750-afb3-c77088cdc884" containerName="proxy-httpd" Dec 03 12:52:40 crc kubenswrapper[5052]: I1203 12:52:40.787162 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="165e6ac4-9875-4750-afb3-c77088cdc884" containerName="proxy-httpd" Dec 03 12:52:40 crc kubenswrapper[5052]: E1203 12:52:40.787181 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="165e6ac4-9875-4750-afb3-c77088cdc884" containerName="ceilometer-central-agent" Dec 03 12:52:40 crc kubenswrapper[5052]: I1203 12:52:40.787187 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="165e6ac4-9875-4750-afb3-c77088cdc884" containerName="ceilometer-central-agent" Dec 03 12:52:40 crc kubenswrapper[5052]: E1203 12:52:40.787211 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="165e6ac4-9875-4750-afb3-c77088cdc884" containerName="ceilometer-notification-agent" Dec 03 12:52:40 crc kubenswrapper[5052]: I1203 12:52:40.787217 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="165e6ac4-9875-4750-afb3-c77088cdc884" containerName="ceilometer-notification-agent" Dec 03 12:52:40 crc kubenswrapper[5052]: I1203 12:52:40.787383 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="165e6ac4-9875-4750-afb3-c77088cdc884" containerName="proxy-httpd" Dec 03 12:52:40 crc kubenswrapper[5052]: I1203 12:52:40.787399 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="165e6ac4-9875-4750-afb3-c77088cdc884" containerName="ceilometer-notification-agent" Dec 03 12:52:40 crc kubenswrapper[5052]: I1203 12:52:40.787410 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="165e6ac4-9875-4750-afb3-c77088cdc884" containerName="ceilometer-central-agent" Dec 03 12:52:40 crc kubenswrapper[5052]: I1203 12:52:40.787424 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="165e6ac4-9875-4750-afb3-c77088cdc884" containerName="sg-core" Dec 03 12:52:40 crc kubenswrapper[5052]: I1203 12:52:40.789158 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:52:40 crc kubenswrapper[5052]: I1203 12:52:40.791899 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-ceilometer-internal-svc" Dec 03 12:52:40 crc kubenswrapper[5052]: I1203 12:52:40.791931 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-scripts" Dec 03 12:52:40 crc kubenswrapper[5052]: I1203 12:52:40.792251 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-config-data" Dec 03 12:52:40 crc kubenswrapper[5052]: I1203 12:52:40.797696 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:52:40 crc kubenswrapper[5052]: I1203 12:52:40.831068 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/f99b50df-1d5b-4535-bc09-86078a1318bd-log-httpd\") pod \"ceilometer-0\" (UID: \"f99b50df-1d5b-4535-bc09-86078a1318bd\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:52:40 crc kubenswrapper[5052]: I1203 12:52:40.831114 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f99b50df-1d5b-4535-bc09-86078a1318bd-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"f99b50df-1d5b-4535-bc09-86078a1318bd\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:52:40 crc kubenswrapper[5052]: I1203 12:52:40.831150 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8v4sn\" (UniqueName: \"kubernetes.io/projected/f99b50df-1d5b-4535-bc09-86078a1318bd-kube-api-access-8v4sn\") pod \"ceilometer-0\" (UID: \"f99b50df-1d5b-4535-bc09-86078a1318bd\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:52:40 crc kubenswrapper[5052]: I1203 12:52:40.831181 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/f99b50df-1d5b-4535-bc09-86078a1318bd-run-httpd\") pod \"ceilometer-0\" (UID: \"f99b50df-1d5b-4535-bc09-86078a1318bd\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:52:40 crc kubenswrapper[5052]: I1203 12:52:40.831203 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/f99b50df-1d5b-4535-bc09-86078a1318bd-ceilometer-tls-certs\") pod \"ceilometer-0\" (UID: \"f99b50df-1d5b-4535-bc09-86078a1318bd\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:52:40 crc kubenswrapper[5052]: I1203 12:52:40.831220 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/f99b50df-1d5b-4535-bc09-86078a1318bd-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"f99b50df-1d5b-4535-bc09-86078a1318bd\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:52:40 crc kubenswrapper[5052]: I1203 12:52:40.831238 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/f99b50df-1d5b-4535-bc09-86078a1318bd-scripts\") pod \"ceilometer-0\" (UID: \"f99b50df-1d5b-4535-bc09-86078a1318bd\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:52:40 crc kubenswrapper[5052]: I1203 12:52:40.831368 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/f99b50df-1d5b-4535-bc09-86078a1318bd-config-data\") pod \"ceilometer-0\" (UID: \"f99b50df-1d5b-4535-bc09-86078a1318bd\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:52:40 crc kubenswrapper[5052]: I1203 12:52:40.933361 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/f99b50df-1d5b-4535-bc09-86078a1318bd-log-httpd\") pod \"ceilometer-0\" (UID: \"f99b50df-1d5b-4535-bc09-86078a1318bd\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:52:40 crc kubenswrapper[5052]: I1203 12:52:40.933407 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f99b50df-1d5b-4535-bc09-86078a1318bd-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"f99b50df-1d5b-4535-bc09-86078a1318bd\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:52:40 crc kubenswrapper[5052]: I1203 12:52:40.933438 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-8v4sn\" (UniqueName: \"kubernetes.io/projected/f99b50df-1d5b-4535-bc09-86078a1318bd-kube-api-access-8v4sn\") pod \"ceilometer-0\" (UID: \"f99b50df-1d5b-4535-bc09-86078a1318bd\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:52:40 crc kubenswrapper[5052]: I1203 12:52:40.933468 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/f99b50df-1d5b-4535-bc09-86078a1318bd-run-httpd\") pod \"ceilometer-0\" (UID: \"f99b50df-1d5b-4535-bc09-86078a1318bd\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:52:40 crc kubenswrapper[5052]: I1203 12:52:40.933494 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/f99b50df-1d5b-4535-bc09-86078a1318bd-ceilometer-tls-certs\") pod \"ceilometer-0\" (UID: \"f99b50df-1d5b-4535-bc09-86078a1318bd\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:52:40 crc kubenswrapper[5052]: I1203 12:52:40.933529 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/f99b50df-1d5b-4535-bc09-86078a1318bd-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"f99b50df-1d5b-4535-bc09-86078a1318bd\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:52:40 crc kubenswrapper[5052]: I1203 12:52:40.933556 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/f99b50df-1d5b-4535-bc09-86078a1318bd-scripts\") pod \"ceilometer-0\" (UID: \"f99b50df-1d5b-4535-bc09-86078a1318bd\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:52:40 crc kubenswrapper[5052]: I1203 12:52:40.933851 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/f99b50df-1d5b-4535-bc09-86078a1318bd-log-httpd\") pod \"ceilometer-0\" (UID: \"f99b50df-1d5b-4535-bc09-86078a1318bd\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:52:40 crc kubenswrapper[5052]: I1203 12:52:40.933898 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/f99b50df-1d5b-4535-bc09-86078a1318bd-run-httpd\") pod \"ceilometer-0\" (UID: \"f99b50df-1d5b-4535-bc09-86078a1318bd\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:52:40 crc kubenswrapper[5052]: I1203 12:52:40.934311 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/f99b50df-1d5b-4535-bc09-86078a1318bd-config-data\") pod \"ceilometer-0\" (UID: \"f99b50df-1d5b-4535-bc09-86078a1318bd\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:52:40 crc kubenswrapper[5052]: I1203 12:52:40.939078 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f99b50df-1d5b-4535-bc09-86078a1318bd-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"f99b50df-1d5b-4535-bc09-86078a1318bd\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:52:40 crc kubenswrapper[5052]: I1203 12:52:40.940202 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/f99b50df-1d5b-4535-bc09-86078a1318bd-config-data\") pod \"ceilometer-0\" (UID: \"f99b50df-1d5b-4535-bc09-86078a1318bd\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:52:40 crc kubenswrapper[5052]: I1203 12:52:40.943159 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/f99b50df-1d5b-4535-bc09-86078a1318bd-scripts\") pod \"ceilometer-0\" (UID: \"f99b50df-1d5b-4535-bc09-86078a1318bd\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:52:40 crc kubenswrapper[5052]: I1203 12:52:40.943201 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/f99b50df-1d5b-4535-bc09-86078a1318bd-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"f99b50df-1d5b-4535-bc09-86078a1318bd\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:52:40 crc kubenswrapper[5052]: I1203 12:52:40.951639 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-8v4sn\" (UniqueName: \"kubernetes.io/projected/f99b50df-1d5b-4535-bc09-86078a1318bd-kube-api-access-8v4sn\") pod \"ceilometer-0\" (UID: \"f99b50df-1d5b-4535-bc09-86078a1318bd\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:52:40 crc kubenswrapper[5052]: I1203 12:52:40.957235 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/f99b50df-1d5b-4535-bc09-86078a1318bd-ceilometer-tls-certs\") pod \"ceilometer-0\" (UID: \"f99b50df-1d5b-4535-bc09-86078a1318bd\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:52:41 crc kubenswrapper[5052]: I1203 12:52:41.111249 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:52:41 crc kubenswrapper[5052]: I1203 12:52:41.632439 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:52:41 crc kubenswrapper[5052]: I1203 12:52:41.669553 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"f99b50df-1d5b-4535-bc09-86078a1318bd","Type":"ContainerStarted","Data":"8a1c4ae133793732d4f09ef113339e8f652716264b5560426daf38d948a4d695"} Dec 03 12:52:41 crc kubenswrapper[5052]: I1203 12:52:41.689919 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="165e6ac4-9875-4750-afb3-c77088cdc884" path="/var/lib/kubelet/pods/165e6ac4-9875-4750-afb3-c77088cdc884/volumes" Dec 03 12:52:42 crc kubenswrapper[5052]: I1203 12:52:42.682975 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"f99b50df-1d5b-4535-bc09-86078a1318bd","Type":"ContainerStarted","Data":"8752f37ec5e86db977fed1536c7c034e712d2037bc9e10abf57acd410669f7b7"} Dec 03 12:52:43 crc kubenswrapper[5052]: I1203 12:52:43.694648 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"f99b50df-1d5b-4535-bc09-86078a1318bd","Type":"ContainerStarted","Data":"8ea80ef5cbeb1e7ba8f177b0594824485b3da4da0c66afa4536804902bdf70e1"} Dec 03 12:52:43 crc kubenswrapper[5052]: I1203 12:52:43.694885 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"f99b50df-1d5b-4535-bc09-86078a1318bd","Type":"ContainerStarted","Data":"c3e875e412556baa33eaca07af1a9b1c17c206e2f38da5c9a1a7982dbc3ff9da"} Dec 03 12:52:45 crc kubenswrapper[5052]: I1203 12:52:45.714734 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"f99b50df-1d5b-4535-bc09-86078a1318bd","Type":"ContainerStarted","Data":"f367321ea2b1658db4fe5944e4616d0065be3a170ac7d1db301f3314e4cbe754"} Dec 03 12:52:45 crc kubenswrapper[5052]: I1203 12:52:45.715319 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:52:45 crc kubenswrapper[5052]: I1203 12:52:45.755000 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/ceilometer-0" podStartSLOduration=2.666161336 podStartE2EDuration="5.754977331s" podCreationTimestamp="2025-12-03 12:52:40 +0000 UTC" firstStartedPulling="2025-12-03 12:52:41.638417104 +0000 UTC m=+5552.226658868" lastFinishedPulling="2025-12-03 12:52:44.727233109 +0000 UTC m=+5555.315474863" observedRunningTime="2025-12-03 12:52:45.739539039 +0000 UTC m=+5556.327780813" watchObservedRunningTime="2025-12-03 12:52:45.754977331 +0000 UTC m=+5556.343219095" Dec 03 12:52:47 crc kubenswrapper[5052]: I1203 12:52:47.977707 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:52:47 crc kubenswrapper[5052]: I1203 12:52:47.978196 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:52:47 crc kubenswrapper[5052]: I1203 12:52:47.994432 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:52:47 crc kubenswrapper[5052]: I1203 12:52:47.994471 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:52:48 crc kubenswrapper[5052]: I1203 12:52:48.016564 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:52:48 crc kubenswrapper[5052]: I1203 12:52:48.048098 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:52:48 crc kubenswrapper[5052]: I1203 12:52:48.059812 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:52:48 crc kubenswrapper[5052]: I1203 12:52:48.061156 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:52:48 crc kubenswrapper[5052]: I1203 12:52:48.749943 5052 generic.go:334] "Generic (PLEG): container finished" podID="6bed880a-56d4-4fed-bec7-6fea476ceb58" containerID="96b654a5e42ddde9fc6e98cc635309cb5313602a1cc9c873755f04530d171682" exitCode=0 Dec 03 12:52:48 crc kubenswrapper[5052]: I1203 12:52:48.750152 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-8f92x" event={"ID":"6bed880a-56d4-4fed-bec7-6fea476ceb58","Type":"ContainerDied","Data":"96b654a5e42ddde9fc6e98cc635309cb5313602a1cc9c873755f04530d171682"} Dec 03 12:52:48 crc kubenswrapper[5052]: I1203 12:52:48.751861 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:52:48 crc kubenswrapper[5052]: I1203 12:52:48.751883 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:52:48 crc kubenswrapper[5052]: I1203 12:52:48.752845 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:52:48 crc kubenswrapper[5052]: I1203 12:52:48.752895 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:52:50 crc kubenswrapper[5052]: I1203 12:52:50.102342 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-8f92x" Dec 03 12:52:50 crc kubenswrapper[5052]: I1203 12:52:50.228912 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6bed880a-56d4-4fed-bec7-6fea476ceb58-combined-ca-bundle\") pod \"6bed880a-56d4-4fed-bec7-6fea476ceb58\" (UID: \"6bed880a-56d4-4fed-bec7-6fea476ceb58\") " Dec 03 12:52:50 crc kubenswrapper[5052]: I1203 12:52:50.228963 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vxsgf\" (UniqueName: \"kubernetes.io/projected/6bed880a-56d4-4fed-bec7-6fea476ceb58-kube-api-access-vxsgf\") pod \"6bed880a-56d4-4fed-bec7-6fea476ceb58\" (UID: \"6bed880a-56d4-4fed-bec7-6fea476ceb58\") " Dec 03 12:52:50 crc kubenswrapper[5052]: I1203 12:52:50.229037 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/6bed880a-56d4-4fed-bec7-6fea476ceb58-scripts\") pod \"6bed880a-56d4-4fed-bec7-6fea476ceb58\" (UID: \"6bed880a-56d4-4fed-bec7-6fea476ceb58\") " Dec 03 12:52:50 crc kubenswrapper[5052]: I1203 12:52:50.229107 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6bed880a-56d4-4fed-bec7-6fea476ceb58-config-data\") pod \"6bed880a-56d4-4fed-bec7-6fea476ceb58\" (UID: \"6bed880a-56d4-4fed-bec7-6fea476ceb58\") " Dec 03 12:52:50 crc kubenswrapper[5052]: I1203 12:52:50.242570 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6bed880a-56d4-4fed-bec7-6fea476ceb58-scripts" (OuterVolumeSpecName: "scripts") pod "6bed880a-56d4-4fed-bec7-6fea476ceb58" (UID: "6bed880a-56d4-4fed-bec7-6fea476ceb58"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:52:50 crc kubenswrapper[5052]: I1203 12:52:50.249940 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6bed880a-56d4-4fed-bec7-6fea476ceb58-kube-api-access-vxsgf" (OuterVolumeSpecName: "kube-api-access-vxsgf") pod "6bed880a-56d4-4fed-bec7-6fea476ceb58" (UID: "6bed880a-56d4-4fed-bec7-6fea476ceb58"). InnerVolumeSpecName "kube-api-access-vxsgf". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:52:50 crc kubenswrapper[5052]: I1203 12:52:50.295611 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6bed880a-56d4-4fed-bec7-6fea476ceb58-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "6bed880a-56d4-4fed-bec7-6fea476ceb58" (UID: "6bed880a-56d4-4fed-bec7-6fea476ceb58"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:52:50 crc kubenswrapper[5052]: I1203 12:52:50.298774 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6bed880a-56d4-4fed-bec7-6fea476ceb58-config-data" (OuterVolumeSpecName: "config-data") pod "6bed880a-56d4-4fed-bec7-6fea476ceb58" (UID: "6bed880a-56d4-4fed-bec7-6fea476ceb58"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:52:50 crc kubenswrapper[5052]: I1203 12:52:50.331707 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6bed880a-56d4-4fed-bec7-6fea476ceb58-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:52:50 crc kubenswrapper[5052]: I1203 12:52:50.331738 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-vxsgf\" (UniqueName: \"kubernetes.io/projected/6bed880a-56d4-4fed-bec7-6fea476ceb58-kube-api-access-vxsgf\") on node \"crc\" DevicePath \"\"" Dec 03 12:52:50 crc kubenswrapper[5052]: I1203 12:52:50.331748 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/6bed880a-56d4-4fed-bec7-6fea476ceb58-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:52:50 crc kubenswrapper[5052]: I1203 12:52:50.331758 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6bed880a-56d4-4fed-bec7-6fea476ceb58-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:52:50 crc kubenswrapper[5052]: I1203 12:52:50.738884 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:52:50 crc kubenswrapper[5052]: I1203 12:52:50.776491 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:52:50 crc kubenswrapper[5052]: I1203 12:52:50.780026 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-8f92x" event={"ID":"6bed880a-56d4-4fed-bec7-6fea476ceb58","Type":"ContainerDied","Data":"a208a94a1930048f74ca260eb08531acef36473d7ba9ad67f6179a05ee533181"} Dec 03 12:52:50 crc kubenswrapper[5052]: I1203 12:52:50.780064 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a208a94a1930048f74ca260eb08531acef36473d7ba9ad67f6179a05ee533181" Dec 03 12:52:50 crc kubenswrapper[5052]: I1203 12:52:50.780125 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-conductor-db-sync-8f92x" Dec 03 12:52:50 crc kubenswrapper[5052]: I1203 12:52:50.780140 5052 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Dec 03 12:52:50 crc kubenswrapper[5052]: I1203 12:52:50.780159 5052 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Dec 03 12:52:50 crc kubenswrapper[5052]: I1203 12:52:50.816574 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:52:50 crc kubenswrapper[5052]: I1203 12:52:50.897802 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-0"] Dec 03 12:52:50 crc kubenswrapper[5052]: E1203 12:52:50.898272 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="6bed880a-56d4-4fed-bec7-6fea476ceb58" containerName="nova-cell0-conductor-db-sync" Dec 03 12:52:50 crc kubenswrapper[5052]: I1203 12:52:50.898289 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="6bed880a-56d4-4fed-bec7-6fea476ceb58" containerName="nova-cell0-conductor-db-sync" Dec 03 12:52:50 crc kubenswrapper[5052]: I1203 12:52:50.898478 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="6bed880a-56d4-4fed-bec7-6fea476ceb58" containerName="nova-cell0-conductor-db-sync" Dec 03 12:52:50 crc kubenswrapper[5052]: I1203 12:52:50.899151 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:52:50 crc kubenswrapper[5052]: I1203 12:52:50.906942 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-nova-dockercfg-zsqnb" Dec 03 12:52:50 crc kubenswrapper[5052]: I1203 12:52:50.907700 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell0-conductor-config-data" Dec 03 12:52:50 crc kubenswrapper[5052]: I1203 12:52:50.913224 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:52:50 crc kubenswrapper[5052]: I1203 12:52:50.913485 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-0"] Dec 03 12:52:51 crc kubenswrapper[5052]: I1203 12:52:51.044601 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s9695\" (UniqueName: \"kubernetes.io/projected/3c6d824e-c7da-41ff-adde-dbafb7535476-kube-api-access-s9695\") pod \"nova-cell0-conductor-0\" (UID: \"3c6d824e-c7da-41ff-adde-dbafb7535476\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:52:51 crc kubenswrapper[5052]: I1203 12:52:51.045038 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/3c6d824e-c7da-41ff-adde-dbafb7535476-config-data\") pod \"nova-cell0-conductor-0\" (UID: \"3c6d824e-c7da-41ff-adde-dbafb7535476\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:52:51 crc kubenswrapper[5052]: I1203 12:52:51.045113 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3c6d824e-c7da-41ff-adde-dbafb7535476-combined-ca-bundle\") pod \"nova-cell0-conductor-0\" (UID: \"3c6d824e-c7da-41ff-adde-dbafb7535476\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:52:51 crc kubenswrapper[5052]: I1203 12:52:51.147009 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-s9695\" (UniqueName: \"kubernetes.io/projected/3c6d824e-c7da-41ff-adde-dbafb7535476-kube-api-access-s9695\") pod \"nova-cell0-conductor-0\" (UID: \"3c6d824e-c7da-41ff-adde-dbafb7535476\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:52:51 crc kubenswrapper[5052]: I1203 12:52:51.147147 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/3c6d824e-c7da-41ff-adde-dbafb7535476-config-data\") pod \"nova-cell0-conductor-0\" (UID: \"3c6d824e-c7da-41ff-adde-dbafb7535476\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:52:51 crc kubenswrapper[5052]: I1203 12:52:51.147175 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3c6d824e-c7da-41ff-adde-dbafb7535476-combined-ca-bundle\") pod \"nova-cell0-conductor-0\" (UID: \"3c6d824e-c7da-41ff-adde-dbafb7535476\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:52:51 crc kubenswrapper[5052]: I1203 12:52:51.151715 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3c6d824e-c7da-41ff-adde-dbafb7535476-combined-ca-bundle\") pod \"nova-cell0-conductor-0\" (UID: \"3c6d824e-c7da-41ff-adde-dbafb7535476\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:52:51 crc kubenswrapper[5052]: I1203 12:52:51.151757 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/3c6d824e-c7da-41ff-adde-dbafb7535476-config-data\") pod \"nova-cell0-conductor-0\" (UID: \"3c6d824e-c7da-41ff-adde-dbafb7535476\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:52:51 crc kubenswrapper[5052]: I1203 12:52:51.163908 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-s9695\" (UniqueName: \"kubernetes.io/projected/3c6d824e-c7da-41ff-adde-dbafb7535476-kube-api-access-s9695\") pod \"nova-cell0-conductor-0\" (UID: \"3c6d824e-c7da-41ff-adde-dbafb7535476\") " pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:52:51 crc kubenswrapper[5052]: I1203 12:52:51.237842 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:52:51 crc kubenswrapper[5052]: I1203 12:52:51.709938 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-0"] Dec 03 12:52:51 crc kubenswrapper[5052]: I1203 12:52:51.790167 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-conductor-0" event={"ID":"3c6d824e-c7da-41ff-adde-dbafb7535476","Type":"ContainerStarted","Data":"d1ac337c7422916c6958db2e1cf8f4b67a9f638fef8862b8806ef4c84da04cd6"} Dec 03 12:52:52 crc kubenswrapper[5052]: I1203 12:52:52.801684 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-conductor-0" event={"ID":"3c6d824e-c7da-41ff-adde-dbafb7535476","Type":"ContainerStarted","Data":"eb28bc8efa7fd43a1e5b7da097e9679a59f690173e27b20d4dae0a08837dde86"} Dec 03 12:52:52 crc kubenswrapper[5052]: I1203 12:52:52.824433 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-cell0-conductor-0" podStartSLOduration=2.824411325 podStartE2EDuration="2.824411325s" podCreationTimestamp="2025-12-03 12:52:50 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:52:52.816334604 +0000 UTC m=+5563.404576378" watchObservedRunningTime="2025-12-03 12:52:52.824411325 +0000 UTC m=+5563.412653099" Dec 03 12:52:53 crc kubenswrapper[5052]: I1203 12:52:53.810464 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:52:56 crc kubenswrapper[5052]: I1203 12:52:56.277078 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:52:56 crc kubenswrapper[5052]: I1203 12:52:56.710398 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell0-cell-mapping-cnck7"] Dec 03 12:52:56 crc kubenswrapper[5052]: I1203 12:52:56.711614 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-cell-mapping-cnck7" Dec 03 12:52:56 crc kubenswrapper[5052]: I1203 12:52:56.715921 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell0-manage-config-data" Dec 03 12:52:56 crc kubenswrapper[5052]: I1203 12:52:56.715936 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell0-manage-scripts" Dec 03 12:52:56 crc kubenswrapper[5052]: I1203 12:52:56.739188 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell0-cell-mapping-cnck7"] Dec 03 12:52:56 crc kubenswrapper[5052]: I1203 12:52:56.771041 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2a7f3d6f-9d21-4071-a9eb-644a23bd704c-combined-ca-bundle\") pod \"nova-cell0-cell-mapping-cnck7\" (UID: \"2a7f3d6f-9d21-4071-a9eb-644a23bd704c\") " pod="openstack-kuttl-tests/nova-cell0-cell-mapping-cnck7" Dec 03 12:52:56 crc kubenswrapper[5052]: I1203 12:52:56.771091 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/2a7f3d6f-9d21-4071-a9eb-644a23bd704c-config-data\") pod \"nova-cell0-cell-mapping-cnck7\" (UID: \"2a7f3d6f-9d21-4071-a9eb-644a23bd704c\") " pod="openstack-kuttl-tests/nova-cell0-cell-mapping-cnck7" Dec 03 12:52:56 crc kubenswrapper[5052]: I1203 12:52:56.771159 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hzchv\" (UniqueName: \"kubernetes.io/projected/2a7f3d6f-9d21-4071-a9eb-644a23bd704c-kube-api-access-hzchv\") pod \"nova-cell0-cell-mapping-cnck7\" (UID: \"2a7f3d6f-9d21-4071-a9eb-644a23bd704c\") " pod="openstack-kuttl-tests/nova-cell0-cell-mapping-cnck7" Dec 03 12:52:56 crc kubenswrapper[5052]: I1203 12:52:56.771192 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/2a7f3d6f-9d21-4071-a9eb-644a23bd704c-scripts\") pod \"nova-cell0-cell-mapping-cnck7\" (UID: \"2a7f3d6f-9d21-4071-a9eb-644a23bd704c\") " pod="openstack-kuttl-tests/nova-cell0-cell-mapping-cnck7" Dec 03 12:52:56 crc kubenswrapper[5052]: I1203 12:52:56.872949 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2a7f3d6f-9d21-4071-a9eb-644a23bd704c-combined-ca-bundle\") pod \"nova-cell0-cell-mapping-cnck7\" (UID: \"2a7f3d6f-9d21-4071-a9eb-644a23bd704c\") " pod="openstack-kuttl-tests/nova-cell0-cell-mapping-cnck7" Dec 03 12:52:56 crc kubenswrapper[5052]: I1203 12:52:56.873029 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/2a7f3d6f-9d21-4071-a9eb-644a23bd704c-config-data\") pod \"nova-cell0-cell-mapping-cnck7\" (UID: \"2a7f3d6f-9d21-4071-a9eb-644a23bd704c\") " pod="openstack-kuttl-tests/nova-cell0-cell-mapping-cnck7" Dec 03 12:52:56 crc kubenswrapper[5052]: I1203 12:52:56.873141 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-hzchv\" (UniqueName: \"kubernetes.io/projected/2a7f3d6f-9d21-4071-a9eb-644a23bd704c-kube-api-access-hzchv\") pod \"nova-cell0-cell-mapping-cnck7\" (UID: \"2a7f3d6f-9d21-4071-a9eb-644a23bd704c\") " pod="openstack-kuttl-tests/nova-cell0-cell-mapping-cnck7" Dec 03 12:52:56 crc kubenswrapper[5052]: I1203 12:52:56.873205 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/2a7f3d6f-9d21-4071-a9eb-644a23bd704c-scripts\") pod \"nova-cell0-cell-mapping-cnck7\" (UID: \"2a7f3d6f-9d21-4071-a9eb-644a23bd704c\") " pod="openstack-kuttl-tests/nova-cell0-cell-mapping-cnck7" Dec 03 12:52:56 crc kubenswrapper[5052]: I1203 12:52:56.888456 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2a7f3d6f-9d21-4071-a9eb-644a23bd704c-combined-ca-bundle\") pod \"nova-cell0-cell-mapping-cnck7\" (UID: \"2a7f3d6f-9d21-4071-a9eb-644a23bd704c\") " pod="openstack-kuttl-tests/nova-cell0-cell-mapping-cnck7" Dec 03 12:52:56 crc kubenswrapper[5052]: I1203 12:52:56.893255 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 12:52:56 crc kubenswrapper[5052]: I1203 12:52:56.895255 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:52:56 crc kubenswrapper[5052]: I1203 12:52:56.902683 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell1-novncproxy-config-data" Dec 03 12:52:56 crc kubenswrapper[5052]: I1203 12:52:56.903014 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/2a7f3d6f-9d21-4071-a9eb-644a23bd704c-config-data\") pod \"nova-cell0-cell-mapping-cnck7\" (UID: \"2a7f3d6f-9d21-4071-a9eb-644a23bd704c\") " pod="openstack-kuttl-tests/nova-cell0-cell-mapping-cnck7" Dec 03 12:52:56 crc kubenswrapper[5052]: I1203 12:52:56.904963 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/2a7f3d6f-9d21-4071-a9eb-644a23bd704c-scripts\") pod \"nova-cell0-cell-mapping-cnck7\" (UID: \"2a7f3d6f-9d21-4071-a9eb-644a23bd704c\") " pod="openstack-kuttl-tests/nova-cell0-cell-mapping-cnck7" Dec 03 12:52:56 crc kubenswrapper[5052]: I1203 12:52:56.906728 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-hzchv\" (UniqueName: \"kubernetes.io/projected/2a7f3d6f-9d21-4071-a9eb-644a23bd704c-kube-api-access-hzchv\") pod \"nova-cell0-cell-mapping-cnck7\" (UID: \"2a7f3d6f-9d21-4071-a9eb-644a23bd704c\") " pod="openstack-kuttl-tests/nova-cell0-cell-mapping-cnck7" Dec 03 12:52:56 crc kubenswrapper[5052]: I1203 12:52:56.947890 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:52:56 crc kubenswrapper[5052]: I1203 12:52:56.949607 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:52:56 crc kubenswrapper[5052]: I1203 12:52:56.954961 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-metadata-config-data" Dec 03 12:52:56 crc kubenswrapper[5052]: I1203 12:52:56.974913 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/551cb42c-c141-482a-93cf-d1253fb141f8-config-data\") pod \"nova-cell1-novncproxy-0\" (UID: \"551cb42c-c141-482a-93cf-d1253fb141f8\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:52:56 crc kubenswrapper[5052]: I1203 12:52:56.975033 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v58mc\" (UniqueName: \"kubernetes.io/projected/551cb42c-c141-482a-93cf-d1253fb141f8-kube-api-access-v58mc\") pod \"nova-cell1-novncproxy-0\" (UID: \"551cb42c-c141-482a-93cf-d1253fb141f8\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:52:56 crc kubenswrapper[5052]: I1203 12:52:56.975076 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/551cb42c-c141-482a-93cf-d1253fb141f8-combined-ca-bundle\") pod \"nova-cell1-novncproxy-0\" (UID: \"551cb42c-c141-482a-93cf-d1253fb141f8\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:52:56 crc kubenswrapper[5052]: I1203 12:52:56.991094 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 12:52:57 crc kubenswrapper[5052]: I1203 12:52:57.018396 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:52:57 crc kubenswrapper[5052]: I1203 12:52:57.038565 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:52:57 crc kubenswrapper[5052]: I1203 12:52:57.040087 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:52:57 crc kubenswrapper[5052]: I1203 12:52:57.041195 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-cell-mapping-cnck7" Dec 03 12:52:57 crc kubenswrapper[5052]: I1203 12:52:57.042228 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-scheduler-config-data" Dec 03 12:52:57 crc kubenswrapper[5052]: I1203 12:52:57.076664 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/551cb42c-c141-482a-93cf-d1253fb141f8-combined-ca-bundle\") pod \"nova-cell1-novncproxy-0\" (UID: \"551cb42c-c141-482a-93cf-d1253fb141f8\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:52:57 crc kubenswrapper[5052]: I1203 12:52:57.076790 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/940d64b0-e4f6-413d-abf7-3b00f710792a-config-data\") pod \"nova-metadata-0\" (UID: \"940d64b0-e4f6-413d-abf7-3b00f710792a\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:52:57 crc kubenswrapper[5052]: I1203 12:52:57.076823 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rgk6g\" (UniqueName: \"kubernetes.io/projected/940d64b0-e4f6-413d-abf7-3b00f710792a-kube-api-access-rgk6g\") pod \"nova-metadata-0\" (UID: \"940d64b0-e4f6-413d-abf7-3b00f710792a\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:52:57 crc kubenswrapper[5052]: I1203 12:52:57.076867 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2a32ea2d-a02d-4dd9-bc81-7d89aed33c38-combined-ca-bundle\") pod \"nova-scheduler-0\" (UID: \"2a32ea2d-a02d-4dd9-bc81-7d89aed33c38\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:52:57 crc kubenswrapper[5052]: I1203 12:52:57.076901 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/551cb42c-c141-482a-93cf-d1253fb141f8-config-data\") pod \"nova-cell1-novncproxy-0\" (UID: \"551cb42c-c141-482a-93cf-d1253fb141f8\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:52:57 crc kubenswrapper[5052]: I1203 12:52:57.076952 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/940d64b0-e4f6-413d-abf7-3b00f710792a-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"940d64b0-e4f6-413d-abf7-3b00f710792a\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:52:57 crc kubenswrapper[5052]: I1203 12:52:57.076974 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/940d64b0-e4f6-413d-abf7-3b00f710792a-logs\") pod \"nova-metadata-0\" (UID: \"940d64b0-e4f6-413d-abf7-3b00f710792a\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:52:57 crc kubenswrapper[5052]: I1203 12:52:57.077034 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b5x2t\" (UniqueName: \"kubernetes.io/projected/2a32ea2d-a02d-4dd9-bc81-7d89aed33c38-kube-api-access-b5x2t\") pod \"nova-scheduler-0\" (UID: \"2a32ea2d-a02d-4dd9-bc81-7d89aed33c38\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:52:57 crc kubenswrapper[5052]: I1203 12:52:57.077099 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/2a32ea2d-a02d-4dd9-bc81-7d89aed33c38-config-data\") pod \"nova-scheduler-0\" (UID: \"2a32ea2d-a02d-4dd9-bc81-7d89aed33c38\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:52:57 crc kubenswrapper[5052]: I1203 12:52:57.077132 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-v58mc\" (UniqueName: \"kubernetes.io/projected/551cb42c-c141-482a-93cf-d1253fb141f8-kube-api-access-v58mc\") pod \"nova-cell1-novncproxy-0\" (UID: \"551cb42c-c141-482a-93cf-d1253fb141f8\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:52:57 crc kubenswrapper[5052]: I1203 12:52:57.083628 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:52:57 crc kubenswrapper[5052]: I1203 12:52:57.094235 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/551cb42c-c141-482a-93cf-d1253fb141f8-config-data\") pod \"nova-cell1-novncproxy-0\" (UID: \"551cb42c-c141-482a-93cf-d1253fb141f8\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:52:57 crc kubenswrapper[5052]: I1203 12:52:57.102892 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/551cb42c-c141-482a-93cf-d1253fb141f8-combined-ca-bundle\") pod \"nova-cell1-novncproxy-0\" (UID: \"551cb42c-c141-482a-93cf-d1253fb141f8\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:52:57 crc kubenswrapper[5052]: I1203 12:52:57.113335 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-v58mc\" (UniqueName: \"kubernetes.io/projected/551cb42c-c141-482a-93cf-d1253fb141f8-kube-api-access-v58mc\") pod \"nova-cell1-novncproxy-0\" (UID: \"551cb42c-c141-482a-93cf-d1253fb141f8\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:52:57 crc kubenswrapper[5052]: I1203 12:52:57.126045 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:52:57 crc kubenswrapper[5052]: I1203 12:52:57.127680 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:52:57 crc kubenswrapper[5052]: I1203 12:52:57.134412 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-api-config-data" Dec 03 12:52:57 crc kubenswrapper[5052]: I1203 12:52:57.134707 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:52:57 crc kubenswrapper[5052]: I1203 12:52:57.178693 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-b5x2t\" (UniqueName: \"kubernetes.io/projected/2a32ea2d-a02d-4dd9-bc81-7d89aed33c38-kube-api-access-b5x2t\") pod \"nova-scheduler-0\" (UID: \"2a32ea2d-a02d-4dd9-bc81-7d89aed33c38\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:52:57 crc kubenswrapper[5052]: I1203 12:52:57.178757 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/2a32ea2d-a02d-4dd9-bc81-7d89aed33c38-config-data\") pod \"nova-scheduler-0\" (UID: \"2a32ea2d-a02d-4dd9-bc81-7d89aed33c38\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:52:57 crc kubenswrapper[5052]: I1203 12:52:57.178786 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a6cfb9d3-77e4-4aaf-91f6-8963a30ed7c1-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"a6cfb9d3-77e4-4aaf-91f6-8963a30ed7c1\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:52:57 crc kubenswrapper[5052]: I1203 12:52:57.178832 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/a6cfb9d3-77e4-4aaf-91f6-8963a30ed7c1-logs\") pod \"nova-api-0\" (UID: \"a6cfb9d3-77e4-4aaf-91f6-8963a30ed7c1\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:52:57 crc kubenswrapper[5052]: I1203 12:52:57.178884 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qn62w\" (UniqueName: \"kubernetes.io/projected/a6cfb9d3-77e4-4aaf-91f6-8963a30ed7c1-kube-api-access-qn62w\") pod \"nova-api-0\" (UID: \"a6cfb9d3-77e4-4aaf-91f6-8963a30ed7c1\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:52:57 crc kubenswrapper[5052]: I1203 12:52:57.178908 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/940d64b0-e4f6-413d-abf7-3b00f710792a-config-data\") pod \"nova-metadata-0\" (UID: \"940d64b0-e4f6-413d-abf7-3b00f710792a\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:52:57 crc kubenswrapper[5052]: I1203 12:52:57.178925 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-rgk6g\" (UniqueName: \"kubernetes.io/projected/940d64b0-e4f6-413d-abf7-3b00f710792a-kube-api-access-rgk6g\") pod \"nova-metadata-0\" (UID: \"940d64b0-e4f6-413d-abf7-3b00f710792a\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:52:57 crc kubenswrapper[5052]: I1203 12:52:57.178947 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2a32ea2d-a02d-4dd9-bc81-7d89aed33c38-combined-ca-bundle\") pod \"nova-scheduler-0\" (UID: \"2a32ea2d-a02d-4dd9-bc81-7d89aed33c38\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:52:57 crc kubenswrapper[5052]: I1203 12:52:57.178984 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/940d64b0-e4f6-413d-abf7-3b00f710792a-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"940d64b0-e4f6-413d-abf7-3b00f710792a\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:52:57 crc kubenswrapper[5052]: I1203 12:52:57.179000 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/940d64b0-e4f6-413d-abf7-3b00f710792a-logs\") pod \"nova-metadata-0\" (UID: \"940d64b0-e4f6-413d-abf7-3b00f710792a\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:52:57 crc kubenswrapper[5052]: I1203 12:52:57.179014 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a6cfb9d3-77e4-4aaf-91f6-8963a30ed7c1-config-data\") pod \"nova-api-0\" (UID: \"a6cfb9d3-77e4-4aaf-91f6-8963a30ed7c1\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:52:57 crc kubenswrapper[5052]: I1203 12:52:57.180861 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/940d64b0-e4f6-413d-abf7-3b00f710792a-logs\") pod \"nova-metadata-0\" (UID: \"940d64b0-e4f6-413d-abf7-3b00f710792a\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:52:57 crc kubenswrapper[5052]: I1203 12:52:57.182939 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/940d64b0-e4f6-413d-abf7-3b00f710792a-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"940d64b0-e4f6-413d-abf7-3b00f710792a\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:52:57 crc kubenswrapper[5052]: I1203 12:52:57.183160 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/2a32ea2d-a02d-4dd9-bc81-7d89aed33c38-config-data\") pod \"nova-scheduler-0\" (UID: \"2a32ea2d-a02d-4dd9-bc81-7d89aed33c38\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:52:57 crc kubenswrapper[5052]: I1203 12:52:57.183468 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2a32ea2d-a02d-4dd9-bc81-7d89aed33c38-combined-ca-bundle\") pod \"nova-scheduler-0\" (UID: \"2a32ea2d-a02d-4dd9-bc81-7d89aed33c38\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:52:57 crc kubenswrapper[5052]: I1203 12:52:57.188226 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/940d64b0-e4f6-413d-abf7-3b00f710792a-config-data\") pod \"nova-metadata-0\" (UID: \"940d64b0-e4f6-413d-abf7-3b00f710792a\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:52:57 crc kubenswrapper[5052]: I1203 12:52:57.200571 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-rgk6g\" (UniqueName: \"kubernetes.io/projected/940d64b0-e4f6-413d-abf7-3b00f710792a-kube-api-access-rgk6g\") pod \"nova-metadata-0\" (UID: \"940d64b0-e4f6-413d-abf7-3b00f710792a\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:52:57 crc kubenswrapper[5052]: I1203 12:52:57.201408 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-b5x2t\" (UniqueName: \"kubernetes.io/projected/2a32ea2d-a02d-4dd9-bc81-7d89aed33c38-kube-api-access-b5x2t\") pod \"nova-scheduler-0\" (UID: \"2a32ea2d-a02d-4dd9-bc81-7d89aed33c38\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:52:57 crc kubenswrapper[5052]: I1203 12:52:57.282655 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-qn62w\" (UniqueName: \"kubernetes.io/projected/a6cfb9d3-77e4-4aaf-91f6-8963a30ed7c1-kube-api-access-qn62w\") pod \"nova-api-0\" (UID: \"a6cfb9d3-77e4-4aaf-91f6-8963a30ed7c1\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:52:57 crc kubenswrapper[5052]: I1203 12:52:57.283206 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a6cfb9d3-77e4-4aaf-91f6-8963a30ed7c1-config-data\") pod \"nova-api-0\" (UID: \"a6cfb9d3-77e4-4aaf-91f6-8963a30ed7c1\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:52:57 crc kubenswrapper[5052]: I1203 12:52:57.283404 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a6cfb9d3-77e4-4aaf-91f6-8963a30ed7c1-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"a6cfb9d3-77e4-4aaf-91f6-8963a30ed7c1\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:52:57 crc kubenswrapper[5052]: I1203 12:52:57.283494 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/a6cfb9d3-77e4-4aaf-91f6-8963a30ed7c1-logs\") pod \"nova-api-0\" (UID: \"a6cfb9d3-77e4-4aaf-91f6-8963a30ed7c1\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:52:57 crc kubenswrapper[5052]: I1203 12:52:57.284134 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/a6cfb9d3-77e4-4aaf-91f6-8963a30ed7c1-logs\") pod \"nova-api-0\" (UID: \"a6cfb9d3-77e4-4aaf-91f6-8963a30ed7c1\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:52:57 crc kubenswrapper[5052]: I1203 12:52:57.289035 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a6cfb9d3-77e4-4aaf-91f6-8963a30ed7c1-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"a6cfb9d3-77e4-4aaf-91f6-8963a30ed7c1\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:52:57 crc kubenswrapper[5052]: I1203 12:52:57.289222 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a6cfb9d3-77e4-4aaf-91f6-8963a30ed7c1-config-data\") pod \"nova-api-0\" (UID: \"a6cfb9d3-77e4-4aaf-91f6-8963a30ed7c1\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:52:57 crc kubenswrapper[5052]: I1203 12:52:57.300224 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:52:57 crc kubenswrapper[5052]: I1203 12:52:57.301793 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-qn62w\" (UniqueName: \"kubernetes.io/projected/a6cfb9d3-77e4-4aaf-91f6-8963a30ed7c1-kube-api-access-qn62w\") pod \"nova-api-0\" (UID: \"a6cfb9d3-77e4-4aaf-91f6-8963a30ed7c1\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:52:57 crc kubenswrapper[5052]: I1203 12:52:57.318795 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:52:57 crc kubenswrapper[5052]: I1203 12:52:57.360606 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:52:57 crc kubenswrapper[5052]: I1203 12:52:57.506370 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:52:57 crc kubenswrapper[5052]: I1203 12:52:57.624993 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell0-cell-mapping-cnck7"] Dec 03 12:52:57 crc kubenswrapper[5052]: W1203 12:52:57.641317 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2a7f3d6f_9d21_4071_a9eb_644a23bd704c.slice/crio-c1987b38945af8a7269075505fcf064a25c89889e3f597c257edacaaf5dfcba1 WatchSource:0}: Error finding container c1987b38945af8a7269075505fcf064a25c89889e3f597c257edacaaf5dfcba1: Status 404 returned error can't find the container with id c1987b38945af8a7269075505fcf064a25c89889e3f597c257edacaaf5dfcba1 Dec 03 12:52:57 crc kubenswrapper[5052]: I1203 12:52:57.848568 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:52:57 crc kubenswrapper[5052]: I1203 12:52:57.861681 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-cell-mapping-cnck7" event={"ID":"2a7f3d6f-9d21-4071-a9eb-644a23bd704c","Type":"ContainerStarted","Data":"c1987b38945af8a7269075505fcf064a25c89889e3f597c257edacaaf5dfcba1"} Dec 03 12:52:57 crc kubenswrapper[5052]: W1203 12:52:57.867659 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod551cb42c_c141_482a_93cf_d1253fb141f8.slice/crio-85477a62a0767889aee9786eacd62837c95b82f7e65cd3a417927eb0887ee457 WatchSource:0}: Error finding container 85477a62a0767889aee9786eacd62837c95b82f7e65cd3a417927eb0887ee457: Status 404 returned error can't find the container with id 85477a62a0767889aee9786eacd62837c95b82f7e65cd3a417927eb0887ee457 Dec 03 12:52:57 crc kubenswrapper[5052]: I1203 12:52:57.886022 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 12:52:57 crc kubenswrapper[5052]: I1203 12:52:57.955044 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-db-sync-gqn79"] Dec 03 12:52:57 crc kubenswrapper[5052]: I1203 12:52:57.958167 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-gqn79" Dec 03 12:52:57 crc kubenswrapper[5052]: I1203 12:52:57.963441 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell1-conductor-scripts" Dec 03 12:52:57 crc kubenswrapper[5052]: I1203 12:52:57.963578 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell1-conductor-config-data" Dec 03 12:52:57 crc kubenswrapper[5052]: I1203 12:52:57.974090 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-db-sync-gqn79"] Dec 03 12:52:58 crc kubenswrapper[5052]: I1203 12:52:58.016930 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:52:58 crc kubenswrapper[5052]: W1203 12:52:58.022683 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2a32ea2d_a02d_4dd9_bc81_7d89aed33c38.slice/crio-8ea41cb52da6ab5b71123f750aa156133f7e1fbe2f8356cc7d3cb664be9d29ed WatchSource:0}: Error finding container 8ea41cb52da6ab5b71123f750aa156133f7e1fbe2f8356cc7d3cb664be9d29ed: Status 404 returned error can't find the container with id 8ea41cb52da6ab5b71123f750aa156133f7e1fbe2f8356cc7d3cb664be9d29ed Dec 03 12:52:58 crc kubenswrapper[5052]: I1203 12:52:58.028798 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b2498fe2-65ee-4470-a52b-e84da2ea42fb-combined-ca-bundle\") pod \"nova-cell1-conductor-db-sync-gqn79\" (UID: \"b2498fe2-65ee-4470-a52b-e84da2ea42fb\") " pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-gqn79" Dec 03 12:52:58 crc kubenswrapper[5052]: I1203 12:52:58.028859 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kdf7t\" (UniqueName: \"kubernetes.io/projected/b2498fe2-65ee-4470-a52b-e84da2ea42fb-kube-api-access-kdf7t\") pod \"nova-cell1-conductor-db-sync-gqn79\" (UID: \"b2498fe2-65ee-4470-a52b-e84da2ea42fb\") " pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-gqn79" Dec 03 12:52:58 crc kubenswrapper[5052]: I1203 12:52:58.028954 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/b2498fe2-65ee-4470-a52b-e84da2ea42fb-scripts\") pod \"nova-cell1-conductor-db-sync-gqn79\" (UID: \"b2498fe2-65ee-4470-a52b-e84da2ea42fb\") " pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-gqn79" Dec 03 12:52:58 crc kubenswrapper[5052]: I1203 12:52:58.028987 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/b2498fe2-65ee-4470-a52b-e84da2ea42fb-config-data\") pod \"nova-cell1-conductor-db-sync-gqn79\" (UID: \"b2498fe2-65ee-4470-a52b-e84da2ea42fb\") " pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-gqn79" Dec 03 12:52:58 crc kubenswrapper[5052]: I1203 12:52:58.083709 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:52:58 crc kubenswrapper[5052]: W1203 12:52:58.090837 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda6cfb9d3_77e4_4aaf_91f6_8963a30ed7c1.slice/crio-22ca6f5d511d80e32577cf024e3fb0537488f43fc358de48173eeceb1d748fe6 WatchSource:0}: Error finding container 22ca6f5d511d80e32577cf024e3fb0537488f43fc358de48173eeceb1d748fe6: Status 404 returned error can't find the container with id 22ca6f5d511d80e32577cf024e3fb0537488f43fc358de48173eeceb1d748fe6 Dec 03 12:52:58 crc kubenswrapper[5052]: I1203 12:52:58.130330 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/b2498fe2-65ee-4470-a52b-e84da2ea42fb-scripts\") pod \"nova-cell1-conductor-db-sync-gqn79\" (UID: \"b2498fe2-65ee-4470-a52b-e84da2ea42fb\") " pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-gqn79" Dec 03 12:52:58 crc kubenswrapper[5052]: I1203 12:52:58.130389 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/b2498fe2-65ee-4470-a52b-e84da2ea42fb-config-data\") pod \"nova-cell1-conductor-db-sync-gqn79\" (UID: \"b2498fe2-65ee-4470-a52b-e84da2ea42fb\") " pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-gqn79" Dec 03 12:52:58 crc kubenswrapper[5052]: I1203 12:52:58.130452 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b2498fe2-65ee-4470-a52b-e84da2ea42fb-combined-ca-bundle\") pod \"nova-cell1-conductor-db-sync-gqn79\" (UID: \"b2498fe2-65ee-4470-a52b-e84da2ea42fb\") " pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-gqn79" Dec 03 12:52:58 crc kubenswrapper[5052]: I1203 12:52:58.130485 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-kdf7t\" (UniqueName: \"kubernetes.io/projected/b2498fe2-65ee-4470-a52b-e84da2ea42fb-kube-api-access-kdf7t\") pod \"nova-cell1-conductor-db-sync-gqn79\" (UID: \"b2498fe2-65ee-4470-a52b-e84da2ea42fb\") " pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-gqn79" Dec 03 12:52:58 crc kubenswrapper[5052]: I1203 12:52:58.134154 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/b2498fe2-65ee-4470-a52b-e84da2ea42fb-scripts\") pod \"nova-cell1-conductor-db-sync-gqn79\" (UID: \"b2498fe2-65ee-4470-a52b-e84da2ea42fb\") " pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-gqn79" Dec 03 12:52:58 crc kubenswrapper[5052]: I1203 12:52:58.134311 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b2498fe2-65ee-4470-a52b-e84da2ea42fb-combined-ca-bundle\") pod \"nova-cell1-conductor-db-sync-gqn79\" (UID: \"b2498fe2-65ee-4470-a52b-e84da2ea42fb\") " pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-gqn79" Dec 03 12:52:58 crc kubenswrapper[5052]: I1203 12:52:58.135761 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/b2498fe2-65ee-4470-a52b-e84da2ea42fb-config-data\") pod \"nova-cell1-conductor-db-sync-gqn79\" (UID: \"b2498fe2-65ee-4470-a52b-e84da2ea42fb\") " pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-gqn79" Dec 03 12:52:58 crc kubenswrapper[5052]: I1203 12:52:58.147816 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-kdf7t\" (UniqueName: \"kubernetes.io/projected/b2498fe2-65ee-4470-a52b-e84da2ea42fb-kube-api-access-kdf7t\") pod \"nova-cell1-conductor-db-sync-gqn79\" (UID: \"b2498fe2-65ee-4470-a52b-e84da2ea42fb\") " pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-gqn79" Dec 03 12:52:58 crc kubenswrapper[5052]: I1203 12:52:58.412475 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-gqn79" Dec 03 12:52:58 crc kubenswrapper[5052]: I1203 12:52:58.873294 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" event={"ID":"551cb42c-c141-482a-93cf-d1253fb141f8","Type":"ContainerStarted","Data":"a0e1c63c3b2b66fda7d707e3eda117a587806b0161c34e8b4d808ac027b748cf"} Dec 03 12:52:58 crc kubenswrapper[5052]: I1203 12:52:58.873742 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" event={"ID":"551cb42c-c141-482a-93cf-d1253fb141f8","Type":"ContainerStarted","Data":"85477a62a0767889aee9786eacd62837c95b82f7e65cd3a417927eb0887ee457"} Dec 03 12:52:58 crc kubenswrapper[5052]: I1203 12:52:58.877089 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"a6cfb9d3-77e4-4aaf-91f6-8963a30ed7c1","Type":"ContainerStarted","Data":"e6d15bca323a72231788b116d26ddd36e8b803c7e170a8fcf032f090e75035bc"} Dec 03 12:52:58 crc kubenswrapper[5052]: I1203 12:52:58.877137 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"a6cfb9d3-77e4-4aaf-91f6-8963a30ed7c1","Type":"ContainerStarted","Data":"082d99fe436dabac3d83b3ec24e280ac88af76ef43fbecaf1ad5c375478b46b8"} Dec 03 12:52:58 crc kubenswrapper[5052]: I1203 12:52:58.877151 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"a6cfb9d3-77e4-4aaf-91f6-8963a30ed7c1","Type":"ContainerStarted","Data":"22ca6f5d511d80e32577cf024e3fb0537488f43fc358de48173eeceb1d748fe6"} Dec 03 12:52:58 crc kubenswrapper[5052]: I1203 12:52:58.879903 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-cell-mapping-cnck7" event={"ID":"2a7f3d6f-9d21-4071-a9eb-644a23bd704c","Type":"ContainerStarted","Data":"c2531bb89e226cd0c966b054b54647ebd280ef61bc50e35addc38acce583274c"} Dec 03 12:52:58 crc kubenswrapper[5052]: I1203 12:52:58.887318 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"2a32ea2d-a02d-4dd9-bc81-7d89aed33c38","Type":"ContainerStarted","Data":"1ae333246c18a5a9547848319849f522de511b1ad9a59932cc0b25dbb4046a47"} Dec 03 12:52:58 crc kubenswrapper[5052]: I1203 12:52:58.887355 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"2a32ea2d-a02d-4dd9-bc81-7d89aed33c38","Type":"ContainerStarted","Data":"8ea41cb52da6ab5b71123f750aa156133f7e1fbe2f8356cc7d3cb664be9d29ed"} Dec 03 12:52:58 crc kubenswrapper[5052]: I1203 12:52:58.890320 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"940d64b0-e4f6-413d-abf7-3b00f710792a","Type":"ContainerStarted","Data":"096e333b4be102c9585f93b9c91db493f7adfe911a8b08d60ce3468c4b5273d8"} Dec 03 12:52:58 crc kubenswrapper[5052]: I1203 12:52:58.890345 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"940d64b0-e4f6-413d-abf7-3b00f710792a","Type":"ContainerStarted","Data":"5a546208859d2318969c05730d9cccefc4ceae4d6295ae49e76408ece16215ce"} Dec 03 12:52:58 crc kubenswrapper[5052]: I1203 12:52:58.890354 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"940d64b0-e4f6-413d-abf7-3b00f710792a","Type":"ContainerStarted","Data":"8fe275a4c5b882c995736a2b51779c26303956b9650d63caa9fd9c087fbf6094"} Dec 03 12:52:58 crc kubenswrapper[5052]: W1203 12:52:58.894043 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb2498fe2_65ee_4470_a52b_e84da2ea42fb.slice/crio-01ab80b86542e89af59c3d442e2b4254f33ebbc7229aa37cece24b65575abfe9 WatchSource:0}: Error finding container 01ab80b86542e89af59c3d442e2b4254f33ebbc7229aa37cece24b65575abfe9: Status 404 returned error can't find the container with id 01ab80b86542e89af59c3d442e2b4254f33ebbc7229aa37cece24b65575abfe9 Dec 03 12:52:58 crc kubenswrapper[5052]: I1203 12:52:58.900360 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-db-sync-gqn79"] Dec 03 12:52:58 crc kubenswrapper[5052]: I1203 12:52:58.905712 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" podStartSLOduration=2.905695378 podStartE2EDuration="2.905695378s" podCreationTimestamp="2025-12-03 12:52:56 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:52:58.90505892 +0000 UTC m=+5569.493300684" watchObservedRunningTime="2025-12-03 12:52:58.905695378 +0000 UTC m=+5569.493937142" Dec 03 12:52:58 crc kubenswrapper[5052]: I1203 12:52:58.922925 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-scheduler-0" podStartSLOduration=2.922905611 podStartE2EDuration="2.922905611s" podCreationTimestamp="2025-12-03 12:52:56 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:52:58.922248052 +0000 UTC m=+5569.510489826" watchObservedRunningTime="2025-12-03 12:52:58.922905611 +0000 UTC m=+5569.511147375" Dec 03 12:52:58 crc kubenswrapper[5052]: I1203 12:52:58.958924 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-cell0-cell-mapping-cnck7" podStartSLOduration=2.95890665 podStartE2EDuration="2.95890665s" podCreationTimestamp="2025-12-03 12:52:56 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:52:58.952471056 +0000 UTC m=+5569.540712870" watchObservedRunningTime="2025-12-03 12:52:58.95890665 +0000 UTC m=+5569.547148414" Dec 03 12:52:58 crc kubenswrapper[5052]: I1203 12:52:58.979070 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-metadata-0" podStartSLOduration=2.979048986 podStartE2EDuration="2.979048986s" podCreationTimestamp="2025-12-03 12:52:56 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:52:58.976010349 +0000 UTC m=+5569.564252133" watchObservedRunningTime="2025-12-03 12:52:58.979048986 +0000 UTC m=+5569.567290750" Dec 03 12:52:59 crc kubenswrapper[5052]: I1203 12:52:59.011405 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-api-0" podStartSLOduration=2.011380401 podStartE2EDuration="2.011380401s" podCreationTimestamp="2025-12-03 12:52:57 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:52:59.001305623 +0000 UTC m=+5569.589547397" watchObservedRunningTime="2025-12-03 12:52:59.011380401 +0000 UTC m=+5569.599622165" Dec 03 12:52:59 crc kubenswrapper[5052]: I1203 12:52:59.902335 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-gqn79" event={"ID":"b2498fe2-65ee-4470-a52b-e84da2ea42fb","Type":"ContainerStarted","Data":"01de615203f552d9ea312a07e2ddf729b58c05565d1119d00cbb8d733dbaa190"} Dec 03 12:52:59 crc kubenswrapper[5052]: I1203 12:52:59.902661 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-gqn79" event={"ID":"b2498fe2-65ee-4470-a52b-e84da2ea42fb","Type":"ContainerStarted","Data":"01ab80b86542e89af59c3d442e2b4254f33ebbc7229aa37cece24b65575abfe9"} Dec 03 12:52:59 crc kubenswrapper[5052]: I1203 12:52:59.919051 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-gqn79" podStartSLOduration=2.919030518 podStartE2EDuration="2.919030518s" podCreationTimestamp="2025-12-03 12:52:57 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:52:59.916676831 +0000 UTC m=+5570.504918615" watchObservedRunningTime="2025-12-03 12:52:59.919030518 +0000 UTC m=+5570.507272282" Dec 03 12:53:00 crc kubenswrapper[5052]: I1203 12:53:00.357816 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 12:53:00 crc kubenswrapper[5052]: I1203 12:53:00.370599 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:53:00 crc kubenswrapper[5052]: I1203 12:53:00.913149 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-metadata-0" podUID="940d64b0-e4f6-413d-abf7-3b00f710792a" containerName="nova-metadata-log" containerID="cri-o://5a546208859d2318969c05730d9cccefc4ceae4d6295ae49e76408ece16215ce" gracePeriod=30 Dec 03 12:53:00 crc kubenswrapper[5052]: I1203 12:53:00.913261 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-metadata-0" podUID="940d64b0-e4f6-413d-abf7-3b00f710792a" containerName="nova-metadata-metadata" containerID="cri-o://096e333b4be102c9585f93b9c91db493f7adfe911a8b08d60ce3468c4b5273d8" gracePeriod=30 Dec 03 12:53:00 crc kubenswrapper[5052]: I1203 12:53:00.913727 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" podUID="551cb42c-c141-482a-93cf-d1253fb141f8" containerName="nova-cell1-novncproxy-novncproxy" containerID="cri-o://a0e1c63c3b2b66fda7d707e3eda117a587806b0161c34e8b4d808ac027b748cf" gracePeriod=30 Dec 03 12:53:01 crc kubenswrapper[5052]: I1203 12:53:01.743384 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:53:01 crc kubenswrapper[5052]: I1203 12:53:01.801995 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:53:01 crc kubenswrapper[5052]: I1203 12:53:01.804399 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v58mc\" (UniqueName: \"kubernetes.io/projected/551cb42c-c141-482a-93cf-d1253fb141f8-kube-api-access-v58mc\") pod \"551cb42c-c141-482a-93cf-d1253fb141f8\" (UID: \"551cb42c-c141-482a-93cf-d1253fb141f8\") " Dec 03 12:53:01 crc kubenswrapper[5052]: I1203 12:53:01.804483 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/551cb42c-c141-482a-93cf-d1253fb141f8-config-data\") pod \"551cb42c-c141-482a-93cf-d1253fb141f8\" (UID: \"551cb42c-c141-482a-93cf-d1253fb141f8\") " Dec 03 12:53:01 crc kubenswrapper[5052]: I1203 12:53:01.804555 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/551cb42c-c141-482a-93cf-d1253fb141f8-combined-ca-bundle\") pod \"551cb42c-c141-482a-93cf-d1253fb141f8\" (UID: \"551cb42c-c141-482a-93cf-d1253fb141f8\") " Dec 03 12:53:01 crc kubenswrapper[5052]: I1203 12:53:01.809980 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/551cb42c-c141-482a-93cf-d1253fb141f8-kube-api-access-v58mc" (OuterVolumeSpecName: "kube-api-access-v58mc") pod "551cb42c-c141-482a-93cf-d1253fb141f8" (UID: "551cb42c-c141-482a-93cf-d1253fb141f8"). InnerVolumeSpecName "kube-api-access-v58mc". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:53:01 crc kubenswrapper[5052]: I1203 12:53:01.840052 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/551cb42c-c141-482a-93cf-d1253fb141f8-config-data" (OuterVolumeSpecName: "config-data") pod "551cb42c-c141-482a-93cf-d1253fb141f8" (UID: "551cb42c-c141-482a-93cf-d1253fb141f8"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:53:01 crc kubenswrapper[5052]: I1203 12:53:01.847258 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/551cb42c-c141-482a-93cf-d1253fb141f8-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "551cb42c-c141-482a-93cf-d1253fb141f8" (UID: "551cb42c-c141-482a-93cf-d1253fb141f8"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:53:01 crc kubenswrapper[5052]: I1203 12:53:01.906562 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/940d64b0-e4f6-413d-abf7-3b00f710792a-combined-ca-bundle\") pod \"940d64b0-e4f6-413d-abf7-3b00f710792a\" (UID: \"940d64b0-e4f6-413d-abf7-3b00f710792a\") " Dec 03 12:53:01 crc kubenswrapper[5052]: I1203 12:53:01.906673 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/940d64b0-e4f6-413d-abf7-3b00f710792a-config-data\") pod \"940d64b0-e4f6-413d-abf7-3b00f710792a\" (UID: \"940d64b0-e4f6-413d-abf7-3b00f710792a\") " Dec 03 12:53:01 crc kubenswrapper[5052]: I1203 12:53:01.906720 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rgk6g\" (UniqueName: \"kubernetes.io/projected/940d64b0-e4f6-413d-abf7-3b00f710792a-kube-api-access-rgk6g\") pod \"940d64b0-e4f6-413d-abf7-3b00f710792a\" (UID: \"940d64b0-e4f6-413d-abf7-3b00f710792a\") " Dec 03 12:53:01 crc kubenswrapper[5052]: I1203 12:53:01.906811 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/940d64b0-e4f6-413d-abf7-3b00f710792a-logs\") pod \"940d64b0-e4f6-413d-abf7-3b00f710792a\" (UID: \"940d64b0-e4f6-413d-abf7-3b00f710792a\") " Dec 03 12:53:01 crc kubenswrapper[5052]: I1203 12:53:01.907394 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-v58mc\" (UniqueName: \"kubernetes.io/projected/551cb42c-c141-482a-93cf-d1253fb141f8-kube-api-access-v58mc\") on node \"crc\" DevicePath \"\"" Dec 03 12:53:01 crc kubenswrapper[5052]: I1203 12:53:01.907418 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/551cb42c-c141-482a-93cf-d1253fb141f8-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:53:01 crc kubenswrapper[5052]: I1203 12:53:01.907431 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/551cb42c-c141-482a-93cf-d1253fb141f8-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:53:01 crc kubenswrapper[5052]: I1203 12:53:01.907686 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/940d64b0-e4f6-413d-abf7-3b00f710792a-logs" (OuterVolumeSpecName: "logs") pod "940d64b0-e4f6-413d-abf7-3b00f710792a" (UID: "940d64b0-e4f6-413d-abf7-3b00f710792a"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:53:01 crc kubenswrapper[5052]: I1203 12:53:01.910157 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/940d64b0-e4f6-413d-abf7-3b00f710792a-kube-api-access-rgk6g" (OuterVolumeSpecName: "kube-api-access-rgk6g") pod "940d64b0-e4f6-413d-abf7-3b00f710792a" (UID: "940d64b0-e4f6-413d-abf7-3b00f710792a"). InnerVolumeSpecName "kube-api-access-rgk6g". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:53:01 crc kubenswrapper[5052]: I1203 12:53:01.925789 5052 generic.go:334] "Generic (PLEG): container finished" podID="940d64b0-e4f6-413d-abf7-3b00f710792a" containerID="096e333b4be102c9585f93b9c91db493f7adfe911a8b08d60ce3468c4b5273d8" exitCode=0 Dec 03 12:53:01 crc kubenswrapper[5052]: I1203 12:53:01.925823 5052 generic.go:334] "Generic (PLEG): container finished" podID="940d64b0-e4f6-413d-abf7-3b00f710792a" containerID="5a546208859d2318969c05730d9cccefc4ceae4d6295ae49e76408ece16215ce" exitCode=143 Dec 03 12:53:01 crc kubenswrapper[5052]: I1203 12:53:01.925834 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:53:01 crc kubenswrapper[5052]: I1203 12:53:01.925900 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"940d64b0-e4f6-413d-abf7-3b00f710792a","Type":"ContainerDied","Data":"096e333b4be102c9585f93b9c91db493f7adfe911a8b08d60ce3468c4b5273d8"} Dec 03 12:53:01 crc kubenswrapper[5052]: I1203 12:53:01.925929 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"940d64b0-e4f6-413d-abf7-3b00f710792a","Type":"ContainerDied","Data":"5a546208859d2318969c05730d9cccefc4ceae4d6295ae49e76408ece16215ce"} Dec 03 12:53:01 crc kubenswrapper[5052]: I1203 12:53:01.925944 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"940d64b0-e4f6-413d-abf7-3b00f710792a","Type":"ContainerDied","Data":"8fe275a4c5b882c995736a2b51779c26303956b9650d63caa9fd9c087fbf6094"} Dec 03 12:53:01 crc kubenswrapper[5052]: I1203 12:53:01.925963 5052 scope.go:117] "RemoveContainer" containerID="096e333b4be102c9585f93b9c91db493f7adfe911a8b08d60ce3468c4b5273d8" Dec 03 12:53:01 crc kubenswrapper[5052]: I1203 12:53:01.928190 5052 generic.go:334] "Generic (PLEG): container finished" podID="551cb42c-c141-482a-93cf-d1253fb141f8" containerID="a0e1c63c3b2b66fda7d707e3eda117a587806b0161c34e8b4d808ac027b748cf" exitCode=0 Dec 03 12:53:01 crc kubenswrapper[5052]: I1203 12:53:01.928785 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:53:01 crc kubenswrapper[5052]: I1203 12:53:01.929067 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" event={"ID":"551cb42c-c141-482a-93cf-d1253fb141f8","Type":"ContainerDied","Data":"a0e1c63c3b2b66fda7d707e3eda117a587806b0161c34e8b4d808ac027b748cf"} Dec 03 12:53:01 crc kubenswrapper[5052]: I1203 12:53:01.929106 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" event={"ID":"551cb42c-c141-482a-93cf-d1253fb141f8","Type":"ContainerDied","Data":"85477a62a0767889aee9786eacd62837c95b82f7e65cd3a417927eb0887ee457"} Dec 03 12:53:01 crc kubenswrapper[5052]: I1203 12:53:01.936273 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/940d64b0-e4f6-413d-abf7-3b00f710792a-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "940d64b0-e4f6-413d-abf7-3b00f710792a" (UID: "940d64b0-e4f6-413d-abf7-3b00f710792a"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:53:01 crc kubenswrapper[5052]: I1203 12:53:01.938399 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/940d64b0-e4f6-413d-abf7-3b00f710792a-config-data" (OuterVolumeSpecName: "config-data") pod "940d64b0-e4f6-413d-abf7-3b00f710792a" (UID: "940d64b0-e4f6-413d-abf7-3b00f710792a"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:53:01 crc kubenswrapper[5052]: I1203 12:53:01.962966 5052 scope.go:117] "RemoveContainer" containerID="5a546208859d2318969c05730d9cccefc4ceae4d6295ae49e76408ece16215ce" Dec 03 12:53:01 crc kubenswrapper[5052]: I1203 12:53:01.971622 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 12:53:01 crc kubenswrapper[5052]: I1203 12:53:01.993792 5052 scope.go:117] "RemoveContainer" containerID="096e333b4be102c9585f93b9c91db493f7adfe911a8b08d60ce3468c4b5273d8" Dec 03 12:53:01 crc kubenswrapper[5052]: E1203 12:53:01.994594 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"096e333b4be102c9585f93b9c91db493f7adfe911a8b08d60ce3468c4b5273d8\": container with ID starting with 096e333b4be102c9585f93b9c91db493f7adfe911a8b08d60ce3468c4b5273d8 not found: ID does not exist" containerID="096e333b4be102c9585f93b9c91db493f7adfe911a8b08d60ce3468c4b5273d8" Dec 03 12:53:01 crc kubenswrapper[5052]: I1203 12:53:01.994647 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"096e333b4be102c9585f93b9c91db493f7adfe911a8b08d60ce3468c4b5273d8"} err="failed to get container status \"096e333b4be102c9585f93b9c91db493f7adfe911a8b08d60ce3468c4b5273d8\": rpc error: code = NotFound desc = could not find container \"096e333b4be102c9585f93b9c91db493f7adfe911a8b08d60ce3468c4b5273d8\": container with ID starting with 096e333b4be102c9585f93b9c91db493f7adfe911a8b08d60ce3468c4b5273d8 not found: ID does not exist" Dec 03 12:53:01 crc kubenswrapper[5052]: I1203 12:53:01.994667 5052 scope.go:117] "RemoveContainer" containerID="5a546208859d2318969c05730d9cccefc4ceae4d6295ae49e76408ece16215ce" Dec 03 12:53:01 crc kubenswrapper[5052]: E1203 12:53:01.996757 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5a546208859d2318969c05730d9cccefc4ceae4d6295ae49e76408ece16215ce\": container with ID starting with 5a546208859d2318969c05730d9cccefc4ceae4d6295ae49e76408ece16215ce not found: ID does not exist" containerID="5a546208859d2318969c05730d9cccefc4ceae4d6295ae49e76408ece16215ce" Dec 03 12:53:01 crc kubenswrapper[5052]: I1203 12:53:01.996819 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5a546208859d2318969c05730d9cccefc4ceae4d6295ae49e76408ece16215ce"} err="failed to get container status \"5a546208859d2318969c05730d9cccefc4ceae4d6295ae49e76408ece16215ce\": rpc error: code = NotFound desc = could not find container \"5a546208859d2318969c05730d9cccefc4ceae4d6295ae49e76408ece16215ce\": container with ID starting with 5a546208859d2318969c05730d9cccefc4ceae4d6295ae49e76408ece16215ce not found: ID does not exist" Dec 03 12:53:01 crc kubenswrapper[5052]: I1203 12:53:01.996855 5052 scope.go:117] "RemoveContainer" containerID="096e333b4be102c9585f93b9c91db493f7adfe911a8b08d60ce3468c4b5273d8" Dec 03 12:53:01 crc kubenswrapper[5052]: I1203 12:53:01.998767 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"096e333b4be102c9585f93b9c91db493f7adfe911a8b08d60ce3468c4b5273d8"} err="failed to get container status \"096e333b4be102c9585f93b9c91db493f7adfe911a8b08d60ce3468c4b5273d8\": rpc error: code = NotFound desc = could not find container \"096e333b4be102c9585f93b9c91db493f7adfe911a8b08d60ce3468c4b5273d8\": container with ID starting with 096e333b4be102c9585f93b9c91db493f7adfe911a8b08d60ce3468c4b5273d8 not found: ID does not exist" Dec 03 12:53:01 crc kubenswrapper[5052]: I1203 12:53:01.998868 5052 scope.go:117] "RemoveContainer" containerID="5a546208859d2318969c05730d9cccefc4ceae4d6295ae49e76408ece16215ce" Dec 03 12:53:01 crc kubenswrapper[5052]: I1203 12:53:01.999198 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5a546208859d2318969c05730d9cccefc4ceae4d6295ae49e76408ece16215ce"} err="failed to get container status \"5a546208859d2318969c05730d9cccefc4ceae4d6295ae49e76408ece16215ce\": rpc error: code = NotFound desc = could not find container \"5a546208859d2318969c05730d9cccefc4ceae4d6295ae49e76408ece16215ce\": container with ID starting with 5a546208859d2318969c05730d9cccefc4ceae4d6295ae49e76408ece16215ce not found: ID does not exist" Dec 03 12:53:01 crc kubenswrapper[5052]: I1203 12:53:01.999220 5052 scope.go:117] "RemoveContainer" containerID="a0e1c63c3b2b66fda7d707e3eda117a587806b0161c34e8b4d808ac027b748cf" Dec 03 12:53:01 crc kubenswrapper[5052]: I1203 12:53:01.999846 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 12:53:02 crc kubenswrapper[5052]: I1203 12:53:02.013781 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/940d64b0-e4f6-413d-abf7-3b00f710792a-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:53:02 crc kubenswrapper[5052]: I1203 12:53:02.013808 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/940d64b0-e4f6-413d-abf7-3b00f710792a-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:53:02 crc kubenswrapper[5052]: I1203 12:53:02.013821 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-rgk6g\" (UniqueName: \"kubernetes.io/projected/940d64b0-e4f6-413d-abf7-3b00f710792a-kube-api-access-rgk6g\") on node \"crc\" DevicePath \"\"" Dec 03 12:53:02 crc kubenswrapper[5052]: I1203 12:53:02.013833 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/940d64b0-e4f6-413d-abf7-3b00f710792a-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:53:02 crc kubenswrapper[5052]: I1203 12:53:02.021260 5052 scope.go:117] "RemoveContainer" containerID="a0e1c63c3b2b66fda7d707e3eda117a587806b0161c34e8b4d808ac027b748cf" Dec 03 12:53:02 crc kubenswrapper[5052]: E1203 12:53:02.021929 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a0e1c63c3b2b66fda7d707e3eda117a587806b0161c34e8b4d808ac027b748cf\": container with ID starting with a0e1c63c3b2b66fda7d707e3eda117a587806b0161c34e8b4d808ac027b748cf not found: ID does not exist" containerID="a0e1c63c3b2b66fda7d707e3eda117a587806b0161c34e8b4d808ac027b748cf" Dec 03 12:53:02 crc kubenswrapper[5052]: I1203 12:53:02.022024 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a0e1c63c3b2b66fda7d707e3eda117a587806b0161c34e8b4d808ac027b748cf"} err="failed to get container status \"a0e1c63c3b2b66fda7d707e3eda117a587806b0161c34e8b4d808ac027b748cf\": rpc error: code = NotFound desc = could not find container \"a0e1c63c3b2b66fda7d707e3eda117a587806b0161c34e8b4d808ac027b748cf\": container with ID starting with a0e1c63c3b2b66fda7d707e3eda117a587806b0161c34e8b4d808ac027b748cf not found: ID does not exist" Dec 03 12:53:02 crc kubenswrapper[5052]: I1203 12:53:02.023425 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 12:53:02 crc kubenswrapper[5052]: E1203 12:53:02.023881 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="940d64b0-e4f6-413d-abf7-3b00f710792a" containerName="nova-metadata-log" Dec 03 12:53:02 crc kubenswrapper[5052]: I1203 12:53:02.023961 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="940d64b0-e4f6-413d-abf7-3b00f710792a" containerName="nova-metadata-log" Dec 03 12:53:02 crc kubenswrapper[5052]: E1203 12:53:02.024030 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="551cb42c-c141-482a-93cf-d1253fb141f8" containerName="nova-cell1-novncproxy-novncproxy" Dec 03 12:53:02 crc kubenswrapper[5052]: I1203 12:53:02.024080 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="551cb42c-c141-482a-93cf-d1253fb141f8" containerName="nova-cell1-novncproxy-novncproxy" Dec 03 12:53:02 crc kubenswrapper[5052]: E1203 12:53:02.024153 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="940d64b0-e4f6-413d-abf7-3b00f710792a" containerName="nova-metadata-metadata" Dec 03 12:53:02 crc kubenswrapper[5052]: I1203 12:53:02.024211 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="940d64b0-e4f6-413d-abf7-3b00f710792a" containerName="nova-metadata-metadata" Dec 03 12:53:02 crc kubenswrapper[5052]: I1203 12:53:02.024461 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="551cb42c-c141-482a-93cf-d1253fb141f8" containerName="nova-cell1-novncproxy-novncproxy" Dec 03 12:53:02 crc kubenswrapper[5052]: I1203 12:53:02.024555 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="940d64b0-e4f6-413d-abf7-3b00f710792a" containerName="nova-metadata-metadata" Dec 03 12:53:02 crc kubenswrapper[5052]: I1203 12:53:02.024620 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="940d64b0-e4f6-413d-abf7-3b00f710792a" containerName="nova-metadata-log" Dec 03 12:53:02 crc kubenswrapper[5052]: I1203 12:53:02.025252 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:53:02 crc kubenswrapper[5052]: I1203 12:53:02.027726 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-nova-novncproxy-cell1-vencrypt" Dec 03 12:53:02 crc kubenswrapper[5052]: I1203 12:53:02.027974 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-nova-novncproxy-cell1-public-svc" Dec 03 12:53:02 crc kubenswrapper[5052]: I1203 12:53:02.027970 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell1-novncproxy-config-data" Dec 03 12:53:02 crc kubenswrapper[5052]: I1203 12:53:02.031747 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 12:53:02 crc kubenswrapper[5052]: I1203 12:53:02.115109 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/43b07f3c-f9d7-451b-b0ab-61b293cff35d-config-data\") pod \"nova-cell1-novncproxy-0\" (UID: \"43b07f3c-f9d7-451b-b0ab-61b293cff35d\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:53:02 crc kubenswrapper[5052]: I1203 12:53:02.115231 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vfmg6\" (UniqueName: \"kubernetes.io/projected/43b07f3c-f9d7-451b-b0ab-61b293cff35d-kube-api-access-vfmg6\") pod \"nova-cell1-novncproxy-0\" (UID: \"43b07f3c-f9d7-451b-b0ab-61b293cff35d\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:53:02 crc kubenswrapper[5052]: I1203 12:53:02.115290 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/43b07f3c-f9d7-451b-b0ab-61b293cff35d-combined-ca-bundle\") pod \"nova-cell1-novncproxy-0\" (UID: \"43b07f3c-f9d7-451b-b0ab-61b293cff35d\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:53:02 crc kubenswrapper[5052]: I1203 12:53:02.115317 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"vencrypt-tls-certs\" (UniqueName: \"kubernetes.io/secret/43b07f3c-f9d7-451b-b0ab-61b293cff35d-vencrypt-tls-certs\") pod \"nova-cell1-novncproxy-0\" (UID: \"43b07f3c-f9d7-451b-b0ab-61b293cff35d\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:53:02 crc kubenswrapper[5052]: I1203 12:53:02.115424 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nova-novncproxy-tls-certs\" (UniqueName: \"kubernetes.io/secret/43b07f3c-f9d7-451b-b0ab-61b293cff35d-nova-novncproxy-tls-certs\") pod \"nova-cell1-novncproxy-0\" (UID: \"43b07f3c-f9d7-451b-b0ab-61b293cff35d\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:53:02 crc kubenswrapper[5052]: I1203 12:53:02.217555 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-vfmg6\" (UniqueName: \"kubernetes.io/projected/43b07f3c-f9d7-451b-b0ab-61b293cff35d-kube-api-access-vfmg6\") pod \"nova-cell1-novncproxy-0\" (UID: \"43b07f3c-f9d7-451b-b0ab-61b293cff35d\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:53:02 crc kubenswrapper[5052]: I1203 12:53:02.217603 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/43b07f3c-f9d7-451b-b0ab-61b293cff35d-combined-ca-bundle\") pod \"nova-cell1-novncproxy-0\" (UID: \"43b07f3c-f9d7-451b-b0ab-61b293cff35d\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:53:02 crc kubenswrapper[5052]: I1203 12:53:02.217624 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"vencrypt-tls-certs\" (UniqueName: \"kubernetes.io/secret/43b07f3c-f9d7-451b-b0ab-61b293cff35d-vencrypt-tls-certs\") pod \"nova-cell1-novncproxy-0\" (UID: \"43b07f3c-f9d7-451b-b0ab-61b293cff35d\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:53:02 crc kubenswrapper[5052]: I1203 12:53:02.217762 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"nova-novncproxy-tls-certs\" (UniqueName: \"kubernetes.io/secret/43b07f3c-f9d7-451b-b0ab-61b293cff35d-nova-novncproxy-tls-certs\") pod \"nova-cell1-novncproxy-0\" (UID: \"43b07f3c-f9d7-451b-b0ab-61b293cff35d\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:53:02 crc kubenswrapper[5052]: I1203 12:53:02.217840 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/43b07f3c-f9d7-451b-b0ab-61b293cff35d-config-data\") pod \"nova-cell1-novncproxy-0\" (UID: \"43b07f3c-f9d7-451b-b0ab-61b293cff35d\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:53:02 crc kubenswrapper[5052]: I1203 12:53:02.222577 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/43b07f3c-f9d7-451b-b0ab-61b293cff35d-combined-ca-bundle\") pod \"nova-cell1-novncproxy-0\" (UID: \"43b07f3c-f9d7-451b-b0ab-61b293cff35d\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:53:02 crc kubenswrapper[5052]: I1203 12:53:02.222806 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"vencrypt-tls-certs\" (UniqueName: \"kubernetes.io/secret/43b07f3c-f9d7-451b-b0ab-61b293cff35d-vencrypt-tls-certs\") pod \"nova-cell1-novncproxy-0\" (UID: \"43b07f3c-f9d7-451b-b0ab-61b293cff35d\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:53:02 crc kubenswrapper[5052]: I1203 12:53:02.223430 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"nova-novncproxy-tls-certs\" (UniqueName: \"kubernetes.io/secret/43b07f3c-f9d7-451b-b0ab-61b293cff35d-nova-novncproxy-tls-certs\") pod \"nova-cell1-novncproxy-0\" (UID: \"43b07f3c-f9d7-451b-b0ab-61b293cff35d\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:53:02 crc kubenswrapper[5052]: I1203 12:53:02.233661 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/43b07f3c-f9d7-451b-b0ab-61b293cff35d-config-data\") pod \"nova-cell1-novncproxy-0\" (UID: \"43b07f3c-f9d7-451b-b0ab-61b293cff35d\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:53:02 crc kubenswrapper[5052]: I1203 12:53:02.237693 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-vfmg6\" (UniqueName: \"kubernetes.io/projected/43b07f3c-f9d7-451b-b0ab-61b293cff35d-kube-api-access-vfmg6\") pod \"nova-cell1-novncproxy-0\" (UID: \"43b07f3c-f9d7-451b-b0ab-61b293cff35d\") " pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:53:02 crc kubenswrapper[5052]: I1203 12:53:02.323526 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:53:02 crc kubenswrapper[5052]: I1203 12:53:02.340627 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:53:02 crc kubenswrapper[5052]: I1203 12:53:02.350439 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:53:02 crc kubenswrapper[5052]: I1203 12:53:02.353481 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:53:02 crc kubenswrapper[5052]: I1203 12:53:02.356895 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-nova-metadata-internal-svc" Dec 03 12:53:02 crc kubenswrapper[5052]: I1203 12:53:02.357150 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-metadata-config-data" Dec 03 12:53:02 crc kubenswrapper[5052]: I1203 12:53:02.361257 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:53:02 crc kubenswrapper[5052]: I1203 12:53:02.371164 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:53:02 crc kubenswrapper[5052]: I1203 12:53:02.405835 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:53:02 crc kubenswrapper[5052]: I1203 12:53:02.421313 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/b0bb2e93-76f1-4485-b822-41e83b2a4f59-nova-metadata-tls-certs\") pod \"nova-metadata-0\" (UID: \"b0bb2e93-76f1-4485-b822-41e83b2a4f59\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:53:02 crc kubenswrapper[5052]: I1203 12:53:02.421383 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b0bb2e93-76f1-4485-b822-41e83b2a4f59-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"b0bb2e93-76f1-4485-b822-41e83b2a4f59\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:53:02 crc kubenswrapper[5052]: I1203 12:53:02.421575 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/b0bb2e93-76f1-4485-b822-41e83b2a4f59-config-data\") pod \"nova-metadata-0\" (UID: \"b0bb2e93-76f1-4485-b822-41e83b2a4f59\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:53:02 crc kubenswrapper[5052]: I1203 12:53:02.421698 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/b0bb2e93-76f1-4485-b822-41e83b2a4f59-logs\") pod \"nova-metadata-0\" (UID: \"b0bb2e93-76f1-4485-b822-41e83b2a4f59\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:53:02 crc kubenswrapper[5052]: I1203 12:53:02.421747 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q8bdk\" (UniqueName: \"kubernetes.io/projected/b0bb2e93-76f1-4485-b822-41e83b2a4f59-kube-api-access-q8bdk\") pod \"nova-metadata-0\" (UID: \"b0bb2e93-76f1-4485-b822-41e83b2a4f59\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:53:02 crc kubenswrapper[5052]: I1203 12:53:02.525756 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/b0bb2e93-76f1-4485-b822-41e83b2a4f59-config-data\") pod \"nova-metadata-0\" (UID: \"b0bb2e93-76f1-4485-b822-41e83b2a4f59\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:53:02 crc kubenswrapper[5052]: I1203 12:53:02.527353 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/b0bb2e93-76f1-4485-b822-41e83b2a4f59-logs\") pod \"nova-metadata-0\" (UID: \"b0bb2e93-76f1-4485-b822-41e83b2a4f59\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:53:02 crc kubenswrapper[5052]: I1203 12:53:02.527387 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-q8bdk\" (UniqueName: \"kubernetes.io/projected/b0bb2e93-76f1-4485-b822-41e83b2a4f59-kube-api-access-q8bdk\") pod \"nova-metadata-0\" (UID: \"b0bb2e93-76f1-4485-b822-41e83b2a4f59\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:53:02 crc kubenswrapper[5052]: I1203 12:53:02.527435 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/b0bb2e93-76f1-4485-b822-41e83b2a4f59-nova-metadata-tls-certs\") pod \"nova-metadata-0\" (UID: \"b0bb2e93-76f1-4485-b822-41e83b2a4f59\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:53:02 crc kubenswrapper[5052]: I1203 12:53:02.528650 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b0bb2e93-76f1-4485-b822-41e83b2a4f59-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"b0bb2e93-76f1-4485-b822-41e83b2a4f59\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:53:02 crc kubenswrapper[5052]: I1203 12:53:02.528118 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/b0bb2e93-76f1-4485-b822-41e83b2a4f59-logs\") pod \"nova-metadata-0\" (UID: \"b0bb2e93-76f1-4485-b822-41e83b2a4f59\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:53:02 crc kubenswrapper[5052]: I1203 12:53:02.533836 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/b0bb2e93-76f1-4485-b822-41e83b2a4f59-config-data\") pod \"nova-metadata-0\" (UID: \"b0bb2e93-76f1-4485-b822-41e83b2a4f59\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:53:02 crc kubenswrapper[5052]: I1203 12:53:02.534689 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b0bb2e93-76f1-4485-b822-41e83b2a4f59-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"b0bb2e93-76f1-4485-b822-41e83b2a4f59\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:53:02 crc kubenswrapper[5052]: I1203 12:53:02.535761 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/b0bb2e93-76f1-4485-b822-41e83b2a4f59-nova-metadata-tls-certs\") pod \"nova-metadata-0\" (UID: \"b0bb2e93-76f1-4485-b822-41e83b2a4f59\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:53:02 crc kubenswrapper[5052]: I1203 12:53:02.593386 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-q8bdk\" (UniqueName: \"kubernetes.io/projected/b0bb2e93-76f1-4485-b822-41e83b2a4f59-kube-api-access-q8bdk\") pod \"nova-metadata-0\" (UID: \"b0bb2e93-76f1-4485-b822-41e83b2a4f59\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:53:02 crc kubenswrapper[5052]: I1203 12:53:02.672589 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:53:02 crc kubenswrapper[5052]: I1203 12:53:02.887348 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 12:53:02 crc kubenswrapper[5052]: W1203 12:53:02.900473 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod43b07f3c_f9d7_451b_b0ab_61b293cff35d.slice/crio-c241edf0de6c454dc695c93815be67873e73d06d1177654a8e8937f82063df1b WatchSource:0}: Error finding container c241edf0de6c454dc695c93815be67873e73d06d1177654a8e8937f82063df1b: Status 404 returned error can't find the container with id c241edf0de6c454dc695c93815be67873e73d06d1177654a8e8937f82063df1b Dec 03 12:53:02 crc kubenswrapper[5052]: I1203 12:53:02.938634 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" event={"ID":"43b07f3c-f9d7-451b-b0ab-61b293cff35d","Type":"ContainerStarted","Data":"c241edf0de6c454dc695c93815be67873e73d06d1177654a8e8937f82063df1b"} Dec 03 12:53:03 crc kubenswrapper[5052]: W1203 12:53:03.151814 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb0bb2e93_76f1_4485_b822_41e83b2a4f59.slice/crio-29b21fda6544a2343736d96bac4c7f7476a926e688d5808f8b0fa0235fbd671a WatchSource:0}: Error finding container 29b21fda6544a2343736d96bac4c7f7476a926e688d5808f8b0fa0235fbd671a: Status 404 returned error can't find the container with id 29b21fda6544a2343736d96bac4c7f7476a926e688d5808f8b0fa0235fbd671a Dec 03 12:53:03 crc kubenswrapper[5052]: I1203 12:53:03.152178 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:53:03 crc kubenswrapper[5052]: I1203 12:53:03.685010 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="551cb42c-c141-482a-93cf-d1253fb141f8" path="/var/lib/kubelet/pods/551cb42c-c141-482a-93cf-d1253fb141f8/volumes" Dec 03 12:53:03 crc kubenswrapper[5052]: I1203 12:53:03.685894 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="940d64b0-e4f6-413d-abf7-3b00f710792a" path="/var/lib/kubelet/pods/940d64b0-e4f6-413d-abf7-3b00f710792a/volumes" Dec 03 12:53:03 crc kubenswrapper[5052]: I1203 12:53:03.951477 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"b0bb2e93-76f1-4485-b822-41e83b2a4f59","Type":"ContainerStarted","Data":"69a3ade9bda7116a28fb4f0f91180a4ccd72fa2cc1845e27fb6c9c30376c1c4a"} Dec 03 12:53:03 crc kubenswrapper[5052]: I1203 12:53:03.951544 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"b0bb2e93-76f1-4485-b822-41e83b2a4f59","Type":"ContainerStarted","Data":"29b21fda6544a2343736d96bac4c7f7476a926e688d5808f8b0fa0235fbd671a"} Dec 03 12:53:03 crc kubenswrapper[5052]: I1203 12:53:03.953176 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" event={"ID":"43b07f3c-f9d7-451b-b0ab-61b293cff35d","Type":"ContainerStarted","Data":"92269ae03a0728af8ad79f1a3cc73e6323d763c9c370a177f9ea91e0fc11bf91"} Dec 03 12:53:03 crc kubenswrapper[5052]: I1203 12:53:03.988070 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" podStartSLOduration=2.9880476849999997 podStartE2EDuration="2.988047685s" podCreationTimestamp="2025-12-03 12:53:01 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:53:03.970560345 +0000 UTC m=+5574.558802109" watchObservedRunningTime="2025-12-03 12:53:03.988047685 +0000 UTC m=+5574.576289469" Dec 03 12:53:04 crc kubenswrapper[5052]: I1203 12:53:04.966219 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"b0bb2e93-76f1-4485-b822-41e83b2a4f59","Type":"ContainerStarted","Data":"9dc693897c1dfd24dbd13e3487a399cfa90b9e6b8ddb3101d227b8fb6896172d"} Dec 03 12:53:06 crc kubenswrapper[5052]: I1203 12:53:06.001656 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-metadata-0" podStartSLOduration=4.00163595 podStartE2EDuration="4.00163595s" podCreationTimestamp="2025-12-03 12:53:02 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:53:05.995791113 +0000 UTC m=+5576.584032877" watchObservedRunningTime="2025-12-03 12:53:06.00163595 +0000 UTC m=+5576.589877714" Dec 03 12:53:06 crc kubenswrapper[5052]: E1203 12:53:06.111360 5052 cadvisor_stats_provider.go:516] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2a7f3d6f_9d21_4071_a9eb_644a23bd704c.slice/crio-c2531bb89e226cd0c966b054b54647ebd280ef61bc50e35addc38acce583274c.scope\": RecentStats: unable to find data in memory cache]" Dec 03 12:53:06 crc kubenswrapper[5052]: I1203 12:53:06.990244 5052 generic.go:334] "Generic (PLEG): container finished" podID="2a7f3d6f-9d21-4071-a9eb-644a23bd704c" containerID="c2531bb89e226cd0c966b054b54647ebd280ef61bc50e35addc38acce583274c" exitCode=0 Dec 03 12:53:06 crc kubenswrapper[5052]: I1203 12:53:06.990300 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-cell-mapping-cnck7" event={"ID":"2a7f3d6f-9d21-4071-a9eb-644a23bd704c","Type":"ContainerDied","Data":"c2531bb89e226cd0c966b054b54647ebd280ef61bc50e35addc38acce583274c"} Dec 03 12:53:07 crc kubenswrapper[5052]: I1203 12:53:07.362331 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:53:07 crc kubenswrapper[5052]: I1203 12:53:07.406662 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:53:07 crc kubenswrapper[5052]: I1203 12:53:07.418917 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:53:07 crc kubenswrapper[5052]: I1203 12:53:07.507943 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:53:07 crc kubenswrapper[5052]: I1203 12:53:07.507991 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:53:07 crc kubenswrapper[5052]: I1203 12:53:07.693275 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:53:07 crc kubenswrapper[5052]: I1203 12:53:07.693316 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:53:08 crc kubenswrapper[5052]: I1203 12:53:08.038244 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:53:08 crc kubenswrapper[5052]: I1203 12:53:08.373231 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-cell-mapping-cnck7" Dec 03 12:53:08 crc kubenswrapper[5052]: I1203 12:53:08.450842 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2a7f3d6f-9d21-4071-a9eb-644a23bd704c-combined-ca-bundle\") pod \"2a7f3d6f-9d21-4071-a9eb-644a23bd704c\" (UID: \"2a7f3d6f-9d21-4071-a9eb-644a23bd704c\") " Dec 03 12:53:08 crc kubenswrapper[5052]: I1203 12:53:08.451395 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/2a7f3d6f-9d21-4071-a9eb-644a23bd704c-config-data\") pod \"2a7f3d6f-9d21-4071-a9eb-644a23bd704c\" (UID: \"2a7f3d6f-9d21-4071-a9eb-644a23bd704c\") " Dec 03 12:53:08 crc kubenswrapper[5052]: I1203 12:53:08.451478 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/2a7f3d6f-9d21-4071-a9eb-644a23bd704c-scripts\") pod \"2a7f3d6f-9d21-4071-a9eb-644a23bd704c\" (UID: \"2a7f3d6f-9d21-4071-a9eb-644a23bd704c\") " Dec 03 12:53:08 crc kubenswrapper[5052]: I1203 12:53:08.451524 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hzchv\" (UniqueName: \"kubernetes.io/projected/2a7f3d6f-9d21-4071-a9eb-644a23bd704c-kube-api-access-hzchv\") pod \"2a7f3d6f-9d21-4071-a9eb-644a23bd704c\" (UID: \"2a7f3d6f-9d21-4071-a9eb-644a23bd704c\") " Dec 03 12:53:08 crc kubenswrapper[5052]: I1203 12:53:08.465332 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2a7f3d6f-9d21-4071-a9eb-644a23bd704c-scripts" (OuterVolumeSpecName: "scripts") pod "2a7f3d6f-9d21-4071-a9eb-644a23bd704c" (UID: "2a7f3d6f-9d21-4071-a9eb-644a23bd704c"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:53:08 crc kubenswrapper[5052]: I1203 12:53:08.465617 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2a7f3d6f-9d21-4071-a9eb-644a23bd704c-kube-api-access-hzchv" (OuterVolumeSpecName: "kube-api-access-hzchv") pod "2a7f3d6f-9d21-4071-a9eb-644a23bd704c" (UID: "2a7f3d6f-9d21-4071-a9eb-644a23bd704c"). InnerVolumeSpecName "kube-api-access-hzchv". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:53:08 crc kubenswrapper[5052]: I1203 12:53:08.487902 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2a7f3d6f-9d21-4071-a9eb-644a23bd704c-config-data" (OuterVolumeSpecName: "config-data") pod "2a7f3d6f-9d21-4071-a9eb-644a23bd704c" (UID: "2a7f3d6f-9d21-4071-a9eb-644a23bd704c"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:53:08 crc kubenswrapper[5052]: I1203 12:53:08.488084 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2a7f3d6f-9d21-4071-a9eb-644a23bd704c-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "2a7f3d6f-9d21-4071-a9eb-644a23bd704c" (UID: "2a7f3d6f-9d21-4071-a9eb-644a23bd704c"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:53:08 crc kubenswrapper[5052]: I1203 12:53:08.554187 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2a7f3d6f-9d21-4071-a9eb-644a23bd704c-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:53:08 crc kubenswrapper[5052]: I1203 12:53:08.554227 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/2a7f3d6f-9d21-4071-a9eb-644a23bd704c-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:53:08 crc kubenswrapper[5052]: I1203 12:53:08.554242 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/2a7f3d6f-9d21-4071-a9eb-644a23bd704c-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:53:08 crc kubenswrapper[5052]: I1203 12:53:08.554256 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-hzchv\" (UniqueName: \"kubernetes.io/projected/2a7f3d6f-9d21-4071-a9eb-644a23bd704c-kube-api-access-hzchv\") on node \"crc\" DevicePath \"\"" Dec 03 12:53:08 crc kubenswrapper[5052]: I1203 12:53:08.590947 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-api-0" podUID="a6cfb9d3-77e4-4aaf-91f6-8963a30ed7c1" containerName="nova-api-log" probeResult="failure" output="Get \"http://10.217.0.209:8774/\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Dec 03 12:53:08 crc kubenswrapper[5052]: I1203 12:53:08.591354 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-api-0" podUID="a6cfb9d3-77e4-4aaf-91f6-8963a30ed7c1" containerName="nova-api-api" probeResult="failure" output="Get \"http://10.217.0.209:8774/\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Dec 03 12:53:09 crc kubenswrapper[5052]: I1203 12:53:09.017653 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-cell-mapping-cnck7" Dec 03 12:53:09 crc kubenswrapper[5052]: I1203 12:53:09.018785 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-cell-mapping-cnck7" event={"ID":"2a7f3d6f-9d21-4071-a9eb-644a23bd704c","Type":"ContainerDied","Data":"c1987b38945af8a7269075505fcf064a25c89889e3f597c257edacaaf5dfcba1"} Dec 03 12:53:09 crc kubenswrapper[5052]: I1203 12:53:09.018839 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c1987b38945af8a7269075505fcf064a25c89889e3f597c257edacaaf5dfcba1" Dec 03 12:53:09 crc kubenswrapper[5052]: I1203 12:53:09.199380 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:53:09 crc kubenswrapper[5052]: I1203 12:53:09.199872 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-api-0" podUID="a6cfb9d3-77e4-4aaf-91f6-8963a30ed7c1" containerName="nova-api-log" containerID="cri-o://082d99fe436dabac3d83b3ec24e280ac88af76ef43fbecaf1ad5c375478b46b8" gracePeriod=30 Dec 03 12:53:09 crc kubenswrapper[5052]: I1203 12:53:09.200178 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-api-0" podUID="a6cfb9d3-77e4-4aaf-91f6-8963a30ed7c1" containerName="nova-api-api" containerID="cri-o://e6d15bca323a72231788b116d26ddd36e8b803c7e170a8fcf032f090e75035bc" gracePeriod=30 Dec 03 12:53:09 crc kubenswrapper[5052]: I1203 12:53:09.214341 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:53:09 crc kubenswrapper[5052]: I1203 12:53:09.276208 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:53:09 crc kubenswrapper[5052]: I1203 12:53:09.276536 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-metadata-0" podUID="b0bb2e93-76f1-4485-b822-41e83b2a4f59" containerName="nova-metadata-log" containerID="cri-o://69a3ade9bda7116a28fb4f0f91180a4ccd72fa2cc1845e27fb6c9c30376c1c4a" gracePeriod=30 Dec 03 12:53:09 crc kubenswrapper[5052]: I1203 12:53:09.276706 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-metadata-0" podUID="b0bb2e93-76f1-4485-b822-41e83b2a4f59" containerName="nova-metadata-metadata" containerID="cri-o://9dc693897c1dfd24dbd13e3487a399cfa90b9e6b8ddb3101d227b8fb6896172d" gracePeriod=30 Dec 03 12:53:09 crc kubenswrapper[5052]: I1203 12:53:09.945798 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:53:09 crc kubenswrapper[5052]: I1203 12:53:09.994283 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q8bdk\" (UniqueName: \"kubernetes.io/projected/b0bb2e93-76f1-4485-b822-41e83b2a4f59-kube-api-access-q8bdk\") pod \"b0bb2e93-76f1-4485-b822-41e83b2a4f59\" (UID: \"b0bb2e93-76f1-4485-b822-41e83b2a4f59\") " Dec 03 12:53:09 crc kubenswrapper[5052]: I1203 12:53:09.994350 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/b0bb2e93-76f1-4485-b822-41e83b2a4f59-logs\") pod \"b0bb2e93-76f1-4485-b822-41e83b2a4f59\" (UID: \"b0bb2e93-76f1-4485-b822-41e83b2a4f59\") " Dec 03 12:53:09 crc kubenswrapper[5052]: I1203 12:53:09.994401 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/b0bb2e93-76f1-4485-b822-41e83b2a4f59-config-data\") pod \"b0bb2e93-76f1-4485-b822-41e83b2a4f59\" (UID: \"b0bb2e93-76f1-4485-b822-41e83b2a4f59\") " Dec 03 12:53:09 crc kubenswrapper[5052]: I1203 12:53:09.994446 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/b0bb2e93-76f1-4485-b822-41e83b2a4f59-nova-metadata-tls-certs\") pod \"b0bb2e93-76f1-4485-b822-41e83b2a4f59\" (UID: \"b0bb2e93-76f1-4485-b822-41e83b2a4f59\") " Dec 03 12:53:09 crc kubenswrapper[5052]: I1203 12:53:09.994883 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b0bb2e93-76f1-4485-b822-41e83b2a4f59-logs" (OuterVolumeSpecName: "logs") pod "b0bb2e93-76f1-4485-b822-41e83b2a4f59" (UID: "b0bb2e93-76f1-4485-b822-41e83b2a4f59"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:53:09 crc kubenswrapper[5052]: I1203 12:53:09.998597 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b0bb2e93-76f1-4485-b822-41e83b2a4f59-combined-ca-bundle\") pod \"b0bb2e93-76f1-4485-b822-41e83b2a4f59\" (UID: \"b0bb2e93-76f1-4485-b822-41e83b2a4f59\") " Dec 03 12:53:09 crc kubenswrapper[5052]: I1203 12:53:09.999355 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/b0bb2e93-76f1-4485-b822-41e83b2a4f59-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:53:10 crc kubenswrapper[5052]: I1203 12:53:10.002242 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b0bb2e93-76f1-4485-b822-41e83b2a4f59-kube-api-access-q8bdk" (OuterVolumeSpecName: "kube-api-access-q8bdk") pod "b0bb2e93-76f1-4485-b822-41e83b2a4f59" (UID: "b0bb2e93-76f1-4485-b822-41e83b2a4f59"). InnerVolumeSpecName "kube-api-access-q8bdk". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:53:10 crc kubenswrapper[5052]: I1203 12:53:10.030959 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b0bb2e93-76f1-4485-b822-41e83b2a4f59-config-data" (OuterVolumeSpecName: "config-data") pod "b0bb2e93-76f1-4485-b822-41e83b2a4f59" (UID: "b0bb2e93-76f1-4485-b822-41e83b2a4f59"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:53:10 crc kubenswrapper[5052]: I1203 12:53:10.034199 5052 generic.go:334] "Generic (PLEG): container finished" podID="b0bb2e93-76f1-4485-b822-41e83b2a4f59" containerID="9dc693897c1dfd24dbd13e3487a399cfa90b9e6b8ddb3101d227b8fb6896172d" exitCode=0 Dec 03 12:53:10 crc kubenswrapper[5052]: I1203 12:53:10.034375 5052 generic.go:334] "Generic (PLEG): container finished" podID="b0bb2e93-76f1-4485-b822-41e83b2a4f59" containerID="69a3ade9bda7116a28fb4f0f91180a4ccd72fa2cc1845e27fb6c9c30376c1c4a" exitCode=143 Dec 03 12:53:10 crc kubenswrapper[5052]: I1203 12:53:10.034259 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"b0bb2e93-76f1-4485-b822-41e83b2a4f59","Type":"ContainerDied","Data":"9dc693897c1dfd24dbd13e3487a399cfa90b9e6b8ddb3101d227b8fb6896172d"} Dec 03 12:53:10 crc kubenswrapper[5052]: I1203 12:53:10.034488 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"b0bb2e93-76f1-4485-b822-41e83b2a4f59","Type":"ContainerDied","Data":"69a3ade9bda7116a28fb4f0f91180a4ccd72fa2cc1845e27fb6c9c30376c1c4a"} Dec 03 12:53:10 crc kubenswrapper[5052]: I1203 12:53:10.034310 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:53:10 crc kubenswrapper[5052]: I1203 12:53:10.034821 5052 scope.go:117] "RemoveContainer" containerID="9dc693897c1dfd24dbd13e3487a399cfa90b9e6b8ddb3101d227b8fb6896172d" Dec 03 12:53:10 crc kubenswrapper[5052]: I1203 12:53:10.035686 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"b0bb2e93-76f1-4485-b822-41e83b2a4f59","Type":"ContainerDied","Data":"29b21fda6544a2343736d96bac4c7f7476a926e688d5808f8b0fa0235fbd671a"} Dec 03 12:53:10 crc kubenswrapper[5052]: I1203 12:53:10.038064 5052 generic.go:334] "Generic (PLEG): container finished" podID="a6cfb9d3-77e4-4aaf-91f6-8963a30ed7c1" containerID="082d99fe436dabac3d83b3ec24e280ac88af76ef43fbecaf1ad5c375478b46b8" exitCode=143 Dec 03 12:53:10 crc kubenswrapper[5052]: I1203 12:53:10.038131 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"a6cfb9d3-77e4-4aaf-91f6-8963a30ed7c1","Type":"ContainerDied","Data":"082d99fe436dabac3d83b3ec24e280ac88af76ef43fbecaf1ad5c375478b46b8"} Dec 03 12:53:10 crc kubenswrapper[5052]: I1203 12:53:10.046527 5052 generic.go:334] "Generic (PLEG): container finished" podID="b2498fe2-65ee-4470-a52b-e84da2ea42fb" containerID="01de615203f552d9ea312a07e2ddf729b58c05565d1119d00cbb8d733dbaa190" exitCode=0 Dec 03 12:53:10 crc kubenswrapper[5052]: I1203 12:53:10.046913 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-scheduler-0" podUID="2a32ea2d-a02d-4dd9-bc81-7d89aed33c38" containerName="nova-scheduler-scheduler" containerID="cri-o://1ae333246c18a5a9547848319849f522de511b1ad9a59932cc0b25dbb4046a47" gracePeriod=30 Dec 03 12:53:10 crc kubenswrapper[5052]: I1203 12:53:10.047607 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-gqn79" event={"ID":"b2498fe2-65ee-4470-a52b-e84da2ea42fb","Type":"ContainerDied","Data":"01de615203f552d9ea312a07e2ddf729b58c05565d1119d00cbb8d733dbaa190"} Dec 03 12:53:10 crc kubenswrapper[5052]: I1203 12:53:10.117693 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b0bb2e93-76f1-4485-b822-41e83b2a4f59-nova-metadata-tls-certs" (OuterVolumeSpecName: "nova-metadata-tls-certs") pod "b0bb2e93-76f1-4485-b822-41e83b2a4f59" (UID: "b0bb2e93-76f1-4485-b822-41e83b2a4f59"). InnerVolumeSpecName "nova-metadata-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:53:10 crc kubenswrapper[5052]: I1203 12:53:10.128918 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-q8bdk\" (UniqueName: \"kubernetes.io/projected/b0bb2e93-76f1-4485-b822-41e83b2a4f59-kube-api-access-q8bdk\") on node \"crc\" DevicePath \"\"" Dec 03 12:53:10 crc kubenswrapper[5052]: I1203 12:53:10.129255 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/b0bb2e93-76f1-4485-b822-41e83b2a4f59-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:53:10 crc kubenswrapper[5052]: I1203 12:53:10.129327 5052 reconciler_common.go:293] "Volume detached for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/b0bb2e93-76f1-4485-b822-41e83b2a4f59-nova-metadata-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:53:10 crc kubenswrapper[5052]: I1203 12:53:10.141668 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b0bb2e93-76f1-4485-b822-41e83b2a4f59-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "b0bb2e93-76f1-4485-b822-41e83b2a4f59" (UID: "b0bb2e93-76f1-4485-b822-41e83b2a4f59"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:53:10 crc kubenswrapper[5052]: I1203 12:53:10.145500 5052 scope.go:117] "RemoveContainer" containerID="69a3ade9bda7116a28fb4f0f91180a4ccd72fa2cc1845e27fb6c9c30376c1c4a" Dec 03 12:53:10 crc kubenswrapper[5052]: I1203 12:53:10.197612 5052 scope.go:117] "RemoveContainer" containerID="9dc693897c1dfd24dbd13e3487a399cfa90b9e6b8ddb3101d227b8fb6896172d" Dec 03 12:53:10 crc kubenswrapper[5052]: E1203 12:53:10.204904 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9dc693897c1dfd24dbd13e3487a399cfa90b9e6b8ddb3101d227b8fb6896172d\": container with ID starting with 9dc693897c1dfd24dbd13e3487a399cfa90b9e6b8ddb3101d227b8fb6896172d not found: ID does not exist" containerID="9dc693897c1dfd24dbd13e3487a399cfa90b9e6b8ddb3101d227b8fb6896172d" Dec 03 12:53:10 crc kubenswrapper[5052]: I1203 12:53:10.205174 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9dc693897c1dfd24dbd13e3487a399cfa90b9e6b8ddb3101d227b8fb6896172d"} err="failed to get container status \"9dc693897c1dfd24dbd13e3487a399cfa90b9e6b8ddb3101d227b8fb6896172d\": rpc error: code = NotFound desc = could not find container \"9dc693897c1dfd24dbd13e3487a399cfa90b9e6b8ddb3101d227b8fb6896172d\": container with ID starting with 9dc693897c1dfd24dbd13e3487a399cfa90b9e6b8ddb3101d227b8fb6896172d not found: ID does not exist" Dec 03 12:53:10 crc kubenswrapper[5052]: I1203 12:53:10.205274 5052 scope.go:117] "RemoveContainer" containerID="69a3ade9bda7116a28fb4f0f91180a4ccd72fa2cc1845e27fb6c9c30376c1c4a" Dec 03 12:53:10 crc kubenswrapper[5052]: E1203 12:53:10.205678 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"69a3ade9bda7116a28fb4f0f91180a4ccd72fa2cc1845e27fb6c9c30376c1c4a\": container with ID starting with 69a3ade9bda7116a28fb4f0f91180a4ccd72fa2cc1845e27fb6c9c30376c1c4a not found: ID does not exist" containerID="69a3ade9bda7116a28fb4f0f91180a4ccd72fa2cc1845e27fb6c9c30376c1c4a" Dec 03 12:53:10 crc kubenswrapper[5052]: I1203 12:53:10.205790 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"69a3ade9bda7116a28fb4f0f91180a4ccd72fa2cc1845e27fb6c9c30376c1c4a"} err="failed to get container status \"69a3ade9bda7116a28fb4f0f91180a4ccd72fa2cc1845e27fb6c9c30376c1c4a\": rpc error: code = NotFound desc = could not find container \"69a3ade9bda7116a28fb4f0f91180a4ccd72fa2cc1845e27fb6c9c30376c1c4a\": container with ID starting with 69a3ade9bda7116a28fb4f0f91180a4ccd72fa2cc1845e27fb6c9c30376c1c4a not found: ID does not exist" Dec 03 12:53:10 crc kubenswrapper[5052]: I1203 12:53:10.205859 5052 scope.go:117] "RemoveContainer" containerID="9dc693897c1dfd24dbd13e3487a399cfa90b9e6b8ddb3101d227b8fb6896172d" Dec 03 12:53:10 crc kubenswrapper[5052]: I1203 12:53:10.206112 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9dc693897c1dfd24dbd13e3487a399cfa90b9e6b8ddb3101d227b8fb6896172d"} err="failed to get container status \"9dc693897c1dfd24dbd13e3487a399cfa90b9e6b8ddb3101d227b8fb6896172d\": rpc error: code = NotFound desc = could not find container \"9dc693897c1dfd24dbd13e3487a399cfa90b9e6b8ddb3101d227b8fb6896172d\": container with ID starting with 9dc693897c1dfd24dbd13e3487a399cfa90b9e6b8ddb3101d227b8fb6896172d not found: ID does not exist" Dec 03 12:53:10 crc kubenswrapper[5052]: I1203 12:53:10.206215 5052 scope.go:117] "RemoveContainer" containerID="69a3ade9bda7116a28fb4f0f91180a4ccd72fa2cc1845e27fb6c9c30376c1c4a" Dec 03 12:53:10 crc kubenswrapper[5052]: I1203 12:53:10.206439 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"69a3ade9bda7116a28fb4f0f91180a4ccd72fa2cc1845e27fb6c9c30376c1c4a"} err="failed to get container status \"69a3ade9bda7116a28fb4f0f91180a4ccd72fa2cc1845e27fb6c9c30376c1c4a\": rpc error: code = NotFound desc = could not find container \"69a3ade9bda7116a28fb4f0f91180a4ccd72fa2cc1845e27fb6c9c30376c1c4a\": container with ID starting with 69a3ade9bda7116a28fb4f0f91180a4ccd72fa2cc1845e27fb6c9c30376c1c4a not found: ID does not exist" Dec 03 12:53:10 crc kubenswrapper[5052]: I1203 12:53:10.231455 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b0bb2e93-76f1-4485-b822-41e83b2a4f59-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:53:10 crc kubenswrapper[5052]: I1203 12:53:10.389913 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:53:10 crc kubenswrapper[5052]: I1203 12:53:10.399333 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:53:10 crc kubenswrapper[5052]: I1203 12:53:10.417273 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:53:10 crc kubenswrapper[5052]: E1203 12:53:10.417954 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="b0bb2e93-76f1-4485-b822-41e83b2a4f59" containerName="nova-metadata-log" Dec 03 12:53:10 crc kubenswrapper[5052]: I1203 12:53:10.418367 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="b0bb2e93-76f1-4485-b822-41e83b2a4f59" containerName="nova-metadata-log" Dec 03 12:53:10 crc kubenswrapper[5052]: E1203 12:53:10.418473 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="b0bb2e93-76f1-4485-b822-41e83b2a4f59" containerName="nova-metadata-metadata" Dec 03 12:53:10 crc kubenswrapper[5052]: I1203 12:53:10.418564 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="b0bb2e93-76f1-4485-b822-41e83b2a4f59" containerName="nova-metadata-metadata" Dec 03 12:53:10 crc kubenswrapper[5052]: E1203 12:53:10.418657 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="2a7f3d6f-9d21-4071-a9eb-644a23bd704c" containerName="nova-manage" Dec 03 12:53:10 crc kubenswrapper[5052]: I1203 12:53:10.418844 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="2a7f3d6f-9d21-4071-a9eb-644a23bd704c" containerName="nova-manage" Dec 03 12:53:10 crc kubenswrapper[5052]: I1203 12:53:10.419223 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="2a7f3d6f-9d21-4071-a9eb-644a23bd704c" containerName="nova-manage" Dec 03 12:53:10 crc kubenswrapper[5052]: I1203 12:53:10.419258 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="b0bb2e93-76f1-4485-b822-41e83b2a4f59" containerName="nova-metadata-log" Dec 03 12:53:10 crc kubenswrapper[5052]: I1203 12:53:10.419292 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="b0bb2e93-76f1-4485-b822-41e83b2a4f59" containerName="nova-metadata-metadata" Dec 03 12:53:10 crc kubenswrapper[5052]: I1203 12:53:10.420685 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:53:10 crc kubenswrapper[5052]: I1203 12:53:10.422193 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-metadata-config-data" Dec 03 12:53:10 crc kubenswrapper[5052]: I1203 12:53:10.425660 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-nova-metadata-internal-svc" Dec 03 12:53:10 crc kubenswrapper[5052]: I1203 12:53:10.434771 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/1e7ef579-72f5-41e2-9374-d2dcab169471-logs\") pod \"nova-metadata-0\" (UID: \"1e7ef579-72f5-41e2-9374-d2dcab169471\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:53:10 crc kubenswrapper[5052]: I1203 12:53:10.434909 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/1e7ef579-72f5-41e2-9374-d2dcab169471-nova-metadata-tls-certs\") pod \"nova-metadata-0\" (UID: \"1e7ef579-72f5-41e2-9374-d2dcab169471\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:53:10 crc kubenswrapper[5052]: I1203 12:53:10.435073 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wpzm4\" (UniqueName: \"kubernetes.io/projected/1e7ef579-72f5-41e2-9374-d2dcab169471-kube-api-access-wpzm4\") pod \"nova-metadata-0\" (UID: \"1e7ef579-72f5-41e2-9374-d2dcab169471\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:53:10 crc kubenswrapper[5052]: I1203 12:53:10.435130 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/1e7ef579-72f5-41e2-9374-d2dcab169471-config-data\") pod \"nova-metadata-0\" (UID: \"1e7ef579-72f5-41e2-9374-d2dcab169471\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:53:10 crc kubenswrapper[5052]: I1203 12:53:10.435181 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1e7ef579-72f5-41e2-9374-d2dcab169471-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"1e7ef579-72f5-41e2-9374-d2dcab169471\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:53:10 crc kubenswrapper[5052]: I1203 12:53:10.445724 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:53:10 crc kubenswrapper[5052]: I1203 12:53:10.537089 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/1e7ef579-72f5-41e2-9374-d2dcab169471-logs\") pod \"nova-metadata-0\" (UID: \"1e7ef579-72f5-41e2-9374-d2dcab169471\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:53:10 crc kubenswrapper[5052]: I1203 12:53:10.537143 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/1e7ef579-72f5-41e2-9374-d2dcab169471-nova-metadata-tls-certs\") pod \"nova-metadata-0\" (UID: \"1e7ef579-72f5-41e2-9374-d2dcab169471\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:53:10 crc kubenswrapper[5052]: I1203 12:53:10.537204 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-wpzm4\" (UniqueName: \"kubernetes.io/projected/1e7ef579-72f5-41e2-9374-d2dcab169471-kube-api-access-wpzm4\") pod \"nova-metadata-0\" (UID: \"1e7ef579-72f5-41e2-9374-d2dcab169471\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:53:10 crc kubenswrapper[5052]: I1203 12:53:10.537245 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/1e7ef579-72f5-41e2-9374-d2dcab169471-config-data\") pod \"nova-metadata-0\" (UID: \"1e7ef579-72f5-41e2-9374-d2dcab169471\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:53:10 crc kubenswrapper[5052]: I1203 12:53:10.537280 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1e7ef579-72f5-41e2-9374-d2dcab169471-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"1e7ef579-72f5-41e2-9374-d2dcab169471\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:53:10 crc kubenswrapper[5052]: I1203 12:53:10.537929 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/1e7ef579-72f5-41e2-9374-d2dcab169471-logs\") pod \"nova-metadata-0\" (UID: \"1e7ef579-72f5-41e2-9374-d2dcab169471\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:53:10 crc kubenswrapper[5052]: I1203 12:53:10.541437 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/1e7ef579-72f5-41e2-9374-d2dcab169471-nova-metadata-tls-certs\") pod \"nova-metadata-0\" (UID: \"1e7ef579-72f5-41e2-9374-d2dcab169471\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:53:10 crc kubenswrapper[5052]: I1203 12:53:10.542096 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1e7ef579-72f5-41e2-9374-d2dcab169471-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"1e7ef579-72f5-41e2-9374-d2dcab169471\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:53:10 crc kubenswrapper[5052]: I1203 12:53:10.550757 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/1e7ef579-72f5-41e2-9374-d2dcab169471-config-data\") pod \"nova-metadata-0\" (UID: \"1e7ef579-72f5-41e2-9374-d2dcab169471\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:53:10 crc kubenswrapper[5052]: I1203 12:53:10.557079 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-wpzm4\" (UniqueName: \"kubernetes.io/projected/1e7ef579-72f5-41e2-9374-d2dcab169471-kube-api-access-wpzm4\") pod \"nova-metadata-0\" (UID: \"1e7ef579-72f5-41e2-9374-d2dcab169471\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:53:10 crc kubenswrapper[5052]: I1203 12:53:10.742268 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:53:11 crc kubenswrapper[5052]: I1203 12:53:11.118995 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:53:11 crc kubenswrapper[5052]: I1203 12:53:11.243435 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:53:11 crc kubenswrapper[5052]: I1203 12:53:11.435641 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-gqn79" Dec 03 12:53:11 crc kubenswrapper[5052]: I1203 12:53:11.457594 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b2498fe2-65ee-4470-a52b-e84da2ea42fb-combined-ca-bundle\") pod \"b2498fe2-65ee-4470-a52b-e84da2ea42fb\" (UID: \"b2498fe2-65ee-4470-a52b-e84da2ea42fb\") " Dec 03 12:53:11 crc kubenswrapper[5052]: I1203 12:53:11.457649 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/b2498fe2-65ee-4470-a52b-e84da2ea42fb-config-data\") pod \"b2498fe2-65ee-4470-a52b-e84da2ea42fb\" (UID: \"b2498fe2-65ee-4470-a52b-e84da2ea42fb\") " Dec 03 12:53:11 crc kubenswrapper[5052]: I1203 12:53:11.457725 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kdf7t\" (UniqueName: \"kubernetes.io/projected/b2498fe2-65ee-4470-a52b-e84da2ea42fb-kube-api-access-kdf7t\") pod \"b2498fe2-65ee-4470-a52b-e84da2ea42fb\" (UID: \"b2498fe2-65ee-4470-a52b-e84da2ea42fb\") " Dec 03 12:53:11 crc kubenswrapper[5052]: I1203 12:53:11.457765 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/b2498fe2-65ee-4470-a52b-e84da2ea42fb-scripts\") pod \"b2498fe2-65ee-4470-a52b-e84da2ea42fb\" (UID: \"b2498fe2-65ee-4470-a52b-e84da2ea42fb\") " Dec 03 12:53:11 crc kubenswrapper[5052]: I1203 12:53:11.464491 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b2498fe2-65ee-4470-a52b-e84da2ea42fb-kube-api-access-kdf7t" (OuterVolumeSpecName: "kube-api-access-kdf7t") pod "b2498fe2-65ee-4470-a52b-e84da2ea42fb" (UID: "b2498fe2-65ee-4470-a52b-e84da2ea42fb"). InnerVolumeSpecName "kube-api-access-kdf7t". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:53:11 crc kubenswrapper[5052]: I1203 12:53:11.466013 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b2498fe2-65ee-4470-a52b-e84da2ea42fb-scripts" (OuterVolumeSpecName: "scripts") pod "b2498fe2-65ee-4470-a52b-e84da2ea42fb" (UID: "b2498fe2-65ee-4470-a52b-e84da2ea42fb"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:53:11 crc kubenswrapper[5052]: I1203 12:53:11.505266 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b2498fe2-65ee-4470-a52b-e84da2ea42fb-config-data" (OuterVolumeSpecName: "config-data") pod "b2498fe2-65ee-4470-a52b-e84da2ea42fb" (UID: "b2498fe2-65ee-4470-a52b-e84da2ea42fb"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:53:11 crc kubenswrapper[5052]: I1203 12:53:11.508864 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b2498fe2-65ee-4470-a52b-e84da2ea42fb-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "b2498fe2-65ee-4470-a52b-e84da2ea42fb" (UID: "b2498fe2-65ee-4470-a52b-e84da2ea42fb"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:53:11 crc kubenswrapper[5052]: I1203 12:53:11.559450 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/b2498fe2-65ee-4470-a52b-e84da2ea42fb-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:53:11 crc kubenswrapper[5052]: I1203 12:53:11.559983 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-kdf7t\" (UniqueName: \"kubernetes.io/projected/b2498fe2-65ee-4470-a52b-e84da2ea42fb-kube-api-access-kdf7t\") on node \"crc\" DevicePath \"\"" Dec 03 12:53:11 crc kubenswrapper[5052]: I1203 12:53:11.560078 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/b2498fe2-65ee-4470-a52b-e84da2ea42fb-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:53:11 crc kubenswrapper[5052]: I1203 12:53:11.560159 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b2498fe2-65ee-4470-a52b-e84da2ea42fb-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:53:11 crc kubenswrapper[5052]: I1203 12:53:11.682772 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b0bb2e93-76f1-4485-b822-41e83b2a4f59" path="/var/lib/kubelet/pods/b0bb2e93-76f1-4485-b822-41e83b2a4f59/volumes" Dec 03 12:53:12 crc kubenswrapper[5052]: I1203 12:53:12.072850 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-gqn79" event={"ID":"b2498fe2-65ee-4470-a52b-e84da2ea42fb","Type":"ContainerDied","Data":"01ab80b86542e89af59c3d442e2b4254f33ebbc7229aa37cece24b65575abfe9"} Dec 03 12:53:12 crc kubenswrapper[5052]: I1203 12:53:12.072888 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="01ab80b86542e89af59c3d442e2b4254f33ebbc7229aa37cece24b65575abfe9" Dec 03 12:53:12 crc kubenswrapper[5052]: I1203 12:53:12.072865 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-conductor-db-sync-gqn79" Dec 03 12:53:12 crc kubenswrapper[5052]: I1203 12:53:12.075812 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"1e7ef579-72f5-41e2-9374-d2dcab169471","Type":"ContainerStarted","Data":"19c711c57e77c66d39bf6c4bd8d5e28dab2b329bf459ff2daed1570bc217ec16"} Dec 03 12:53:12 crc kubenswrapper[5052]: I1203 12:53:12.075998 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"1e7ef579-72f5-41e2-9374-d2dcab169471","Type":"ContainerStarted","Data":"d6031286fa2a90c842cf1e6e7b88e513c06f9131324674efcd3cb78de18ca18f"} Dec 03 12:53:12 crc kubenswrapper[5052]: I1203 12:53:12.076574 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"1e7ef579-72f5-41e2-9374-d2dcab169471","Type":"ContainerStarted","Data":"e52e82d33656856b2e2f3d6c3e4423ea16101ee7e57fcb2c0e9ec0af57ba0695"} Dec 03 12:53:12 crc kubenswrapper[5052]: I1203 12:53:12.105921 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-metadata-0" podStartSLOduration=2.105903252 podStartE2EDuration="2.105903252s" podCreationTimestamp="2025-12-03 12:53:10 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:53:12.099496948 +0000 UTC m=+5582.687738722" watchObservedRunningTime="2025-12-03 12:53:12.105903252 +0000 UTC m=+5582.694145016" Dec 03 12:53:12 crc kubenswrapper[5052]: I1203 12:53:12.161250 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-0"] Dec 03 12:53:12 crc kubenswrapper[5052]: E1203 12:53:12.161921 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="b2498fe2-65ee-4470-a52b-e84da2ea42fb" containerName="nova-cell1-conductor-db-sync" Dec 03 12:53:12 crc kubenswrapper[5052]: I1203 12:53:12.161935 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="b2498fe2-65ee-4470-a52b-e84da2ea42fb" containerName="nova-cell1-conductor-db-sync" Dec 03 12:53:12 crc kubenswrapper[5052]: I1203 12:53:12.162164 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="b2498fe2-65ee-4470-a52b-e84da2ea42fb" containerName="nova-cell1-conductor-db-sync" Dec 03 12:53:12 crc kubenswrapper[5052]: I1203 12:53:12.162852 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:53:12 crc kubenswrapper[5052]: I1203 12:53:12.165250 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell1-conductor-config-data" Dec 03 12:53:12 crc kubenswrapper[5052]: I1203 12:53:12.172071 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-0"] Dec 03 12:53:12 crc kubenswrapper[5052]: I1203 12:53:12.173146 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/2adad00d-01e3-4441-a3fc-87d162a5e309-config-data\") pod \"nova-cell1-conductor-0\" (UID: \"2adad00d-01e3-4441-a3fc-87d162a5e309\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:53:12 crc kubenswrapper[5052]: I1203 12:53:12.173234 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2adad00d-01e3-4441-a3fc-87d162a5e309-combined-ca-bundle\") pod \"nova-cell1-conductor-0\" (UID: \"2adad00d-01e3-4441-a3fc-87d162a5e309\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:53:12 crc kubenswrapper[5052]: I1203 12:53:12.173266 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ffrfx\" (UniqueName: \"kubernetes.io/projected/2adad00d-01e3-4441-a3fc-87d162a5e309-kube-api-access-ffrfx\") pod \"nova-cell1-conductor-0\" (UID: \"2adad00d-01e3-4441-a3fc-87d162a5e309\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:53:12 crc kubenswrapper[5052]: I1203 12:53:12.274982 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/2adad00d-01e3-4441-a3fc-87d162a5e309-config-data\") pod \"nova-cell1-conductor-0\" (UID: \"2adad00d-01e3-4441-a3fc-87d162a5e309\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:53:12 crc kubenswrapper[5052]: I1203 12:53:12.275320 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2adad00d-01e3-4441-a3fc-87d162a5e309-combined-ca-bundle\") pod \"nova-cell1-conductor-0\" (UID: \"2adad00d-01e3-4441-a3fc-87d162a5e309\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:53:12 crc kubenswrapper[5052]: I1203 12:53:12.275421 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-ffrfx\" (UniqueName: \"kubernetes.io/projected/2adad00d-01e3-4441-a3fc-87d162a5e309-kube-api-access-ffrfx\") pod \"nova-cell1-conductor-0\" (UID: \"2adad00d-01e3-4441-a3fc-87d162a5e309\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:53:12 crc kubenswrapper[5052]: I1203 12:53:12.280186 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/2adad00d-01e3-4441-a3fc-87d162a5e309-config-data\") pod \"nova-cell1-conductor-0\" (UID: \"2adad00d-01e3-4441-a3fc-87d162a5e309\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:53:12 crc kubenswrapper[5052]: I1203 12:53:12.280790 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2adad00d-01e3-4441-a3fc-87d162a5e309-combined-ca-bundle\") pod \"nova-cell1-conductor-0\" (UID: \"2adad00d-01e3-4441-a3fc-87d162a5e309\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:53:12 crc kubenswrapper[5052]: I1203 12:53:12.301583 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-ffrfx\" (UniqueName: \"kubernetes.io/projected/2adad00d-01e3-4441-a3fc-87d162a5e309-kube-api-access-ffrfx\") pod \"nova-cell1-conductor-0\" (UID: \"2adad00d-01e3-4441-a3fc-87d162a5e309\") " pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:53:12 crc kubenswrapper[5052]: E1203 12:53:12.364715 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="1ae333246c18a5a9547848319849f522de511b1ad9a59932cc0b25dbb4046a47" cmd=["/usr/bin/pgrep","-r","DRST","nova-scheduler"] Dec 03 12:53:12 crc kubenswrapper[5052]: E1203 12:53:12.367183 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="1ae333246c18a5a9547848319849f522de511b1ad9a59932cc0b25dbb4046a47" cmd=["/usr/bin/pgrep","-r","DRST","nova-scheduler"] Dec 03 12:53:12 crc kubenswrapper[5052]: E1203 12:53:12.369436 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="1ae333246c18a5a9547848319849f522de511b1ad9a59932cc0b25dbb4046a47" cmd=["/usr/bin/pgrep","-r","DRST","nova-scheduler"] Dec 03 12:53:12 crc kubenswrapper[5052]: E1203 12:53:12.369577 5052 prober.go:104] "Probe errored" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" probeType="Readiness" pod="openstack-kuttl-tests/nova-scheduler-0" podUID="2a32ea2d-a02d-4dd9-bc81-7d89aed33c38" containerName="nova-scheduler-scheduler" Dec 03 12:53:12 crc kubenswrapper[5052]: I1203 12:53:12.406199 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:53:12 crc kubenswrapper[5052]: I1203 12:53:12.427913 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:53:12 crc kubenswrapper[5052]: I1203 12:53:12.530258 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:53:12 crc kubenswrapper[5052]: I1203 12:53:12.976794 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-0"] Dec 03 12:53:13 crc kubenswrapper[5052]: I1203 12:53:13.090550 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-conductor-0" event={"ID":"2adad00d-01e3-4441-a3fc-87d162a5e309","Type":"ContainerStarted","Data":"069817b8bb632c54923379bdbcb28b2aeaddd9b64161f40bc504fc2f25ddd00f"} Dec 03 12:53:13 crc kubenswrapper[5052]: I1203 12:53:13.117529 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:53:14 crc kubenswrapper[5052]: I1203 12:53:14.105925 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-conductor-0" event={"ID":"2adad00d-01e3-4441-a3fc-87d162a5e309","Type":"ContainerStarted","Data":"a0d54bbd0da0ce78d0c6c5dcb48378864ee5ffb0c8e5280b729c579e83ef32f1"} Dec 03 12:53:14 crc kubenswrapper[5052]: I1203 12:53:14.106273 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:53:14 crc kubenswrapper[5052]: I1203 12:53:14.122123 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-cell1-conductor-0" podStartSLOduration=2.122106771 podStartE2EDuration="2.122106771s" podCreationTimestamp="2025-12-03 12:53:12 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:53:14.12207692 +0000 UTC m=+5584.710318684" watchObservedRunningTime="2025-12-03 12:53:14.122106771 +0000 UTC m=+5584.710348535" Dec 03 12:53:15 crc kubenswrapper[5052]: I1203 12:53:15.114336 5052 generic.go:334] "Generic (PLEG): container finished" podID="a6cfb9d3-77e4-4aaf-91f6-8963a30ed7c1" containerID="e6d15bca323a72231788b116d26ddd36e8b803c7e170a8fcf032f090e75035bc" exitCode=0 Dec 03 12:53:15 crc kubenswrapper[5052]: I1203 12:53:15.115580 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"a6cfb9d3-77e4-4aaf-91f6-8963a30ed7c1","Type":"ContainerDied","Data":"e6d15bca323a72231788b116d26ddd36e8b803c7e170a8fcf032f090e75035bc"} Dec 03 12:53:15 crc kubenswrapper[5052]: I1203 12:53:15.743443 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:53:15 crc kubenswrapper[5052]: I1203 12:53:15.743787 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:53:16 crc kubenswrapper[5052]: I1203 12:53:16.068506 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:53:16 crc kubenswrapper[5052]: I1203 12:53:16.127315 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:53:16 crc kubenswrapper[5052]: I1203 12:53:16.127309 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"a6cfb9d3-77e4-4aaf-91f6-8963a30ed7c1","Type":"ContainerDied","Data":"22ca6f5d511d80e32577cf024e3fb0537488f43fc358de48173eeceb1d748fe6"} Dec 03 12:53:16 crc kubenswrapper[5052]: I1203 12:53:16.127580 5052 scope.go:117] "RemoveContainer" containerID="e6d15bca323a72231788b116d26ddd36e8b803c7e170a8fcf032f090e75035bc" Dec 03 12:53:16 crc kubenswrapper[5052]: I1203 12:53:16.131019 5052 generic.go:334] "Generic (PLEG): container finished" podID="2a32ea2d-a02d-4dd9-bc81-7d89aed33c38" containerID="1ae333246c18a5a9547848319849f522de511b1ad9a59932cc0b25dbb4046a47" exitCode=0 Dec 03 12:53:16 crc kubenswrapper[5052]: I1203 12:53:16.131059 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"2a32ea2d-a02d-4dd9-bc81-7d89aed33c38","Type":"ContainerDied","Data":"1ae333246c18a5a9547848319849f522de511b1ad9a59932cc0b25dbb4046a47"} Dec 03 12:53:16 crc kubenswrapper[5052]: I1203 12:53:16.151124 5052 scope.go:117] "RemoveContainer" containerID="082d99fe436dabac3d83b3ec24e280ac88af76ef43fbecaf1ad5c375478b46b8" Dec 03 12:53:16 crc kubenswrapper[5052]: I1203 12:53:16.168557 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qn62w\" (UniqueName: \"kubernetes.io/projected/a6cfb9d3-77e4-4aaf-91f6-8963a30ed7c1-kube-api-access-qn62w\") pod \"a6cfb9d3-77e4-4aaf-91f6-8963a30ed7c1\" (UID: \"a6cfb9d3-77e4-4aaf-91f6-8963a30ed7c1\") " Dec 03 12:53:16 crc kubenswrapper[5052]: I1203 12:53:16.168637 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/a6cfb9d3-77e4-4aaf-91f6-8963a30ed7c1-logs\") pod \"a6cfb9d3-77e4-4aaf-91f6-8963a30ed7c1\" (UID: \"a6cfb9d3-77e4-4aaf-91f6-8963a30ed7c1\") " Dec 03 12:53:16 crc kubenswrapper[5052]: I1203 12:53:16.168658 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a6cfb9d3-77e4-4aaf-91f6-8963a30ed7c1-combined-ca-bundle\") pod \"a6cfb9d3-77e4-4aaf-91f6-8963a30ed7c1\" (UID: \"a6cfb9d3-77e4-4aaf-91f6-8963a30ed7c1\") " Dec 03 12:53:16 crc kubenswrapper[5052]: I1203 12:53:16.168862 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a6cfb9d3-77e4-4aaf-91f6-8963a30ed7c1-config-data\") pod \"a6cfb9d3-77e4-4aaf-91f6-8963a30ed7c1\" (UID: \"a6cfb9d3-77e4-4aaf-91f6-8963a30ed7c1\") " Dec 03 12:53:16 crc kubenswrapper[5052]: I1203 12:53:16.170110 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a6cfb9d3-77e4-4aaf-91f6-8963a30ed7c1-logs" (OuterVolumeSpecName: "logs") pod "a6cfb9d3-77e4-4aaf-91f6-8963a30ed7c1" (UID: "a6cfb9d3-77e4-4aaf-91f6-8963a30ed7c1"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:53:16 crc kubenswrapper[5052]: I1203 12:53:16.184557 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a6cfb9d3-77e4-4aaf-91f6-8963a30ed7c1-kube-api-access-qn62w" (OuterVolumeSpecName: "kube-api-access-qn62w") pod "a6cfb9d3-77e4-4aaf-91f6-8963a30ed7c1" (UID: "a6cfb9d3-77e4-4aaf-91f6-8963a30ed7c1"). InnerVolumeSpecName "kube-api-access-qn62w". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:53:16 crc kubenswrapper[5052]: I1203 12:53:16.231769 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a6cfb9d3-77e4-4aaf-91f6-8963a30ed7c1-config-data" (OuterVolumeSpecName: "config-data") pod "a6cfb9d3-77e4-4aaf-91f6-8963a30ed7c1" (UID: "a6cfb9d3-77e4-4aaf-91f6-8963a30ed7c1"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:53:16 crc kubenswrapper[5052]: I1203 12:53:16.231833 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a6cfb9d3-77e4-4aaf-91f6-8963a30ed7c1-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "a6cfb9d3-77e4-4aaf-91f6-8963a30ed7c1" (UID: "a6cfb9d3-77e4-4aaf-91f6-8963a30ed7c1"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:53:16 crc kubenswrapper[5052]: I1203 12:53:16.271290 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-qn62w\" (UniqueName: \"kubernetes.io/projected/a6cfb9d3-77e4-4aaf-91f6-8963a30ed7c1-kube-api-access-qn62w\") on node \"crc\" DevicePath \"\"" Dec 03 12:53:16 crc kubenswrapper[5052]: I1203 12:53:16.271354 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/a6cfb9d3-77e4-4aaf-91f6-8963a30ed7c1-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:53:16 crc kubenswrapper[5052]: I1203 12:53:16.271369 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a6cfb9d3-77e4-4aaf-91f6-8963a30ed7c1-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:53:16 crc kubenswrapper[5052]: I1203 12:53:16.271383 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/a6cfb9d3-77e4-4aaf-91f6-8963a30ed7c1-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:53:16 crc kubenswrapper[5052]: E1203 12:53:16.341890 5052 cadvisor_stats_provider.go:516] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb0bb2e93_76f1_4485_b822_41e83b2a4f59.slice/crio-29b21fda6544a2343736d96bac4c7f7476a926e688d5808f8b0fa0235fbd671a\": RecentStats: unable to find data in memory cache]" Dec 03 12:53:16 crc kubenswrapper[5052]: I1203 12:53:16.465887 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:53:16 crc kubenswrapper[5052]: I1203 12:53:16.477681 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:53:16 crc kubenswrapper[5052]: I1203 12:53:16.488465 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:53:16 crc kubenswrapper[5052]: E1203 12:53:16.488922 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="a6cfb9d3-77e4-4aaf-91f6-8963a30ed7c1" containerName="nova-api-api" Dec 03 12:53:16 crc kubenswrapper[5052]: I1203 12:53:16.488944 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="a6cfb9d3-77e4-4aaf-91f6-8963a30ed7c1" containerName="nova-api-api" Dec 03 12:53:16 crc kubenswrapper[5052]: E1203 12:53:16.488960 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="a6cfb9d3-77e4-4aaf-91f6-8963a30ed7c1" containerName="nova-api-log" Dec 03 12:53:16 crc kubenswrapper[5052]: I1203 12:53:16.488966 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="a6cfb9d3-77e4-4aaf-91f6-8963a30ed7c1" containerName="nova-api-log" Dec 03 12:53:16 crc kubenswrapper[5052]: I1203 12:53:16.489174 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="a6cfb9d3-77e4-4aaf-91f6-8963a30ed7c1" containerName="nova-api-log" Dec 03 12:53:16 crc kubenswrapper[5052]: I1203 12:53:16.489203 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="a6cfb9d3-77e4-4aaf-91f6-8963a30ed7c1" containerName="nova-api-api" Dec 03 12:53:16 crc kubenswrapper[5052]: I1203 12:53:16.490155 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:53:16 crc kubenswrapper[5052]: I1203 12:53:16.492854 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-api-config-data" Dec 03 12:53:16 crc kubenswrapper[5052]: I1203 12:53:16.498220 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:53:16 crc kubenswrapper[5052]: I1203 12:53:16.683675 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c7d71496-0273-4f3d-9f21-3848037800ff-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"c7d71496-0273-4f3d-9f21-3848037800ff\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:53:16 crc kubenswrapper[5052]: I1203 12:53:16.683755 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/c7d71496-0273-4f3d-9f21-3848037800ff-logs\") pod \"nova-api-0\" (UID: \"c7d71496-0273-4f3d-9f21-3848037800ff\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:53:16 crc kubenswrapper[5052]: I1203 12:53:16.684553 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6fg52\" (UniqueName: \"kubernetes.io/projected/c7d71496-0273-4f3d-9f21-3848037800ff-kube-api-access-6fg52\") pod \"nova-api-0\" (UID: \"c7d71496-0273-4f3d-9f21-3848037800ff\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:53:16 crc kubenswrapper[5052]: I1203 12:53:16.684603 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/c7d71496-0273-4f3d-9f21-3848037800ff-config-data\") pod \"nova-api-0\" (UID: \"c7d71496-0273-4f3d-9f21-3848037800ff\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:53:16 crc kubenswrapper[5052]: I1203 12:53:16.786453 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/c7d71496-0273-4f3d-9f21-3848037800ff-logs\") pod \"nova-api-0\" (UID: \"c7d71496-0273-4f3d-9f21-3848037800ff\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:53:16 crc kubenswrapper[5052]: I1203 12:53:16.786608 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-6fg52\" (UniqueName: \"kubernetes.io/projected/c7d71496-0273-4f3d-9f21-3848037800ff-kube-api-access-6fg52\") pod \"nova-api-0\" (UID: \"c7d71496-0273-4f3d-9f21-3848037800ff\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:53:16 crc kubenswrapper[5052]: I1203 12:53:16.786639 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/c7d71496-0273-4f3d-9f21-3848037800ff-config-data\") pod \"nova-api-0\" (UID: \"c7d71496-0273-4f3d-9f21-3848037800ff\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:53:16 crc kubenswrapper[5052]: I1203 12:53:16.786723 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c7d71496-0273-4f3d-9f21-3848037800ff-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"c7d71496-0273-4f3d-9f21-3848037800ff\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:53:16 crc kubenswrapper[5052]: I1203 12:53:16.786877 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/c7d71496-0273-4f3d-9f21-3848037800ff-logs\") pod \"nova-api-0\" (UID: \"c7d71496-0273-4f3d-9f21-3848037800ff\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:53:16 crc kubenswrapper[5052]: I1203 12:53:16.792819 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/c7d71496-0273-4f3d-9f21-3848037800ff-config-data\") pod \"nova-api-0\" (UID: \"c7d71496-0273-4f3d-9f21-3848037800ff\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:53:16 crc kubenswrapper[5052]: I1203 12:53:16.793445 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c7d71496-0273-4f3d-9f21-3848037800ff-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"c7d71496-0273-4f3d-9f21-3848037800ff\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:53:16 crc kubenswrapper[5052]: I1203 12:53:16.803255 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-6fg52\" (UniqueName: \"kubernetes.io/projected/c7d71496-0273-4f3d-9f21-3848037800ff-kube-api-access-6fg52\") pod \"nova-api-0\" (UID: \"c7d71496-0273-4f3d-9f21-3848037800ff\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:53:16 crc kubenswrapper[5052]: I1203 12:53:16.817559 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:53:16 crc kubenswrapper[5052]: I1203 12:53:16.856031 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:53:16 crc kubenswrapper[5052]: I1203 12:53:16.989888 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b5x2t\" (UniqueName: \"kubernetes.io/projected/2a32ea2d-a02d-4dd9-bc81-7d89aed33c38-kube-api-access-b5x2t\") pod \"2a32ea2d-a02d-4dd9-bc81-7d89aed33c38\" (UID: \"2a32ea2d-a02d-4dd9-bc81-7d89aed33c38\") " Dec 03 12:53:16 crc kubenswrapper[5052]: I1203 12:53:16.990423 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2a32ea2d-a02d-4dd9-bc81-7d89aed33c38-combined-ca-bundle\") pod \"2a32ea2d-a02d-4dd9-bc81-7d89aed33c38\" (UID: \"2a32ea2d-a02d-4dd9-bc81-7d89aed33c38\") " Dec 03 12:53:16 crc kubenswrapper[5052]: I1203 12:53:16.990595 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/2a32ea2d-a02d-4dd9-bc81-7d89aed33c38-config-data\") pod \"2a32ea2d-a02d-4dd9-bc81-7d89aed33c38\" (UID: \"2a32ea2d-a02d-4dd9-bc81-7d89aed33c38\") " Dec 03 12:53:16 crc kubenswrapper[5052]: I1203 12:53:16.996457 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2a32ea2d-a02d-4dd9-bc81-7d89aed33c38-kube-api-access-b5x2t" (OuterVolumeSpecName: "kube-api-access-b5x2t") pod "2a32ea2d-a02d-4dd9-bc81-7d89aed33c38" (UID: "2a32ea2d-a02d-4dd9-bc81-7d89aed33c38"). InnerVolumeSpecName "kube-api-access-b5x2t". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:53:17 crc kubenswrapper[5052]: I1203 12:53:17.016982 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2a32ea2d-a02d-4dd9-bc81-7d89aed33c38-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "2a32ea2d-a02d-4dd9-bc81-7d89aed33c38" (UID: "2a32ea2d-a02d-4dd9-bc81-7d89aed33c38"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:53:17 crc kubenswrapper[5052]: I1203 12:53:17.047665 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2a32ea2d-a02d-4dd9-bc81-7d89aed33c38-config-data" (OuterVolumeSpecName: "config-data") pod "2a32ea2d-a02d-4dd9-bc81-7d89aed33c38" (UID: "2a32ea2d-a02d-4dd9-bc81-7d89aed33c38"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:53:17 crc kubenswrapper[5052]: I1203 12:53:17.093671 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-b5x2t\" (UniqueName: \"kubernetes.io/projected/2a32ea2d-a02d-4dd9-bc81-7d89aed33c38-kube-api-access-b5x2t\") on node \"crc\" DevicePath \"\"" Dec 03 12:53:17 crc kubenswrapper[5052]: I1203 12:53:17.093708 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2a32ea2d-a02d-4dd9-bc81-7d89aed33c38-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:53:17 crc kubenswrapper[5052]: I1203 12:53:17.093718 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/2a32ea2d-a02d-4dd9-bc81-7d89aed33c38-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:53:17 crc kubenswrapper[5052]: I1203 12:53:17.145046 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"2a32ea2d-a02d-4dd9-bc81-7d89aed33c38","Type":"ContainerDied","Data":"8ea41cb52da6ab5b71123f750aa156133f7e1fbe2f8356cc7d3cb664be9d29ed"} Dec 03 12:53:17 crc kubenswrapper[5052]: I1203 12:53:17.145074 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:53:17 crc kubenswrapper[5052]: I1203 12:53:17.145098 5052 scope.go:117] "RemoveContainer" containerID="1ae333246c18a5a9547848319849f522de511b1ad9a59932cc0b25dbb4046a47" Dec 03 12:53:17 crc kubenswrapper[5052]: I1203 12:53:17.191094 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:53:17 crc kubenswrapper[5052]: I1203 12:53:17.206051 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:53:17 crc kubenswrapper[5052]: I1203 12:53:17.224125 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:53:17 crc kubenswrapper[5052]: E1203 12:53:17.225649 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="2a32ea2d-a02d-4dd9-bc81-7d89aed33c38" containerName="nova-scheduler-scheduler" Dec 03 12:53:17 crc kubenswrapper[5052]: I1203 12:53:17.225671 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="2a32ea2d-a02d-4dd9-bc81-7d89aed33c38" containerName="nova-scheduler-scheduler" Dec 03 12:53:17 crc kubenswrapper[5052]: I1203 12:53:17.226010 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="2a32ea2d-a02d-4dd9-bc81-7d89aed33c38" containerName="nova-scheduler-scheduler" Dec 03 12:53:17 crc kubenswrapper[5052]: I1203 12:53:17.226749 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:53:17 crc kubenswrapper[5052]: I1203 12:53:17.228799 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-scheduler-config-data" Dec 03 12:53:17 crc kubenswrapper[5052]: I1203 12:53:17.238144 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:53:17 crc kubenswrapper[5052]: I1203 12:53:17.294791 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:53:17 crc kubenswrapper[5052]: I1203 12:53:17.397966 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/5930ff4b-2768-4636-af3f-e5395bb93f49-config-data\") pod \"nova-scheduler-0\" (UID: \"5930ff4b-2768-4636-af3f-e5395bb93f49\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:53:17 crc kubenswrapper[5052]: I1203 12:53:17.398032 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5930ff4b-2768-4636-af3f-e5395bb93f49-combined-ca-bundle\") pod \"nova-scheduler-0\" (UID: \"5930ff4b-2768-4636-af3f-e5395bb93f49\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:53:17 crc kubenswrapper[5052]: I1203 12:53:17.398164 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mnj5b\" (UniqueName: \"kubernetes.io/projected/5930ff4b-2768-4636-af3f-e5395bb93f49-kube-api-access-mnj5b\") pod \"nova-scheduler-0\" (UID: \"5930ff4b-2768-4636-af3f-e5395bb93f49\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:53:17 crc kubenswrapper[5052]: I1203 12:53:17.499285 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5930ff4b-2768-4636-af3f-e5395bb93f49-combined-ca-bundle\") pod \"nova-scheduler-0\" (UID: \"5930ff4b-2768-4636-af3f-e5395bb93f49\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:53:17 crc kubenswrapper[5052]: I1203 12:53:17.500190 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-mnj5b\" (UniqueName: \"kubernetes.io/projected/5930ff4b-2768-4636-af3f-e5395bb93f49-kube-api-access-mnj5b\") pod \"nova-scheduler-0\" (UID: \"5930ff4b-2768-4636-af3f-e5395bb93f49\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:53:17 crc kubenswrapper[5052]: I1203 12:53:17.500237 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/5930ff4b-2768-4636-af3f-e5395bb93f49-config-data\") pod \"nova-scheduler-0\" (UID: \"5930ff4b-2768-4636-af3f-e5395bb93f49\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:53:17 crc kubenswrapper[5052]: I1203 12:53:17.503583 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5930ff4b-2768-4636-af3f-e5395bb93f49-combined-ca-bundle\") pod \"nova-scheduler-0\" (UID: \"5930ff4b-2768-4636-af3f-e5395bb93f49\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:53:17 crc kubenswrapper[5052]: I1203 12:53:17.503905 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/5930ff4b-2768-4636-af3f-e5395bb93f49-config-data\") pod \"nova-scheduler-0\" (UID: \"5930ff4b-2768-4636-af3f-e5395bb93f49\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:53:17 crc kubenswrapper[5052]: I1203 12:53:17.519035 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-mnj5b\" (UniqueName: \"kubernetes.io/projected/5930ff4b-2768-4636-af3f-e5395bb93f49-kube-api-access-mnj5b\") pod \"nova-scheduler-0\" (UID: \"5930ff4b-2768-4636-af3f-e5395bb93f49\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:53:17 crc kubenswrapper[5052]: I1203 12:53:17.630688 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:53:17 crc kubenswrapper[5052]: I1203 12:53:17.686773 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2a32ea2d-a02d-4dd9-bc81-7d89aed33c38" path="/var/lib/kubelet/pods/2a32ea2d-a02d-4dd9-bc81-7d89aed33c38/volumes" Dec 03 12:53:17 crc kubenswrapper[5052]: I1203 12:53:17.687692 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a6cfb9d3-77e4-4aaf-91f6-8963a30ed7c1" path="/var/lib/kubelet/pods/a6cfb9d3-77e4-4aaf-91f6-8963a30ed7c1/volumes" Dec 03 12:53:18 crc kubenswrapper[5052]: I1203 12:53:18.100859 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:53:18 crc kubenswrapper[5052]: W1203 12:53:18.107085 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5930ff4b_2768_4636_af3f_e5395bb93f49.slice/crio-6f20fa9d8650553f0998b0bce070f5158cd0e6889e646f6de06461b8a4aa07d0 WatchSource:0}: Error finding container 6f20fa9d8650553f0998b0bce070f5158cd0e6889e646f6de06461b8a4aa07d0: Status 404 returned error can't find the container with id 6f20fa9d8650553f0998b0bce070f5158cd0e6889e646f6de06461b8a4aa07d0 Dec 03 12:53:18 crc kubenswrapper[5052]: I1203 12:53:18.159288 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"c7d71496-0273-4f3d-9f21-3848037800ff","Type":"ContainerStarted","Data":"e3ed6fc1c0c369792c4bf196651ad2c3593bedb2a55fc0a57ff66f53c8f194ab"} Dec 03 12:53:18 crc kubenswrapper[5052]: I1203 12:53:18.159883 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"c7d71496-0273-4f3d-9f21-3848037800ff","Type":"ContainerStarted","Data":"dce448abb2df885cc3e486c3248a809b055e7460ef4b57579e4f233091b421b8"} Dec 03 12:53:18 crc kubenswrapper[5052]: I1203 12:53:18.161157 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"5930ff4b-2768-4636-af3f-e5395bb93f49","Type":"ContainerStarted","Data":"6f20fa9d8650553f0998b0bce070f5158cd0e6889e646f6de06461b8a4aa07d0"} Dec 03 12:53:19 crc kubenswrapper[5052]: I1203 12:53:19.171872 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"5930ff4b-2768-4636-af3f-e5395bb93f49","Type":"ContainerStarted","Data":"7498f540d70df6b99e8086ff89ab5331440cd2c5e65aed3c60c8d119b3260aba"} Dec 03 12:53:19 crc kubenswrapper[5052]: I1203 12:53:19.173540 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"c7d71496-0273-4f3d-9f21-3848037800ff","Type":"ContainerStarted","Data":"2b509ad5f1c30962f5faa8d131f05da4b7c14268ac8f7cec4bee50543949b838"} Dec 03 12:53:19 crc kubenswrapper[5052]: I1203 12:53:19.191385 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-scheduler-0" podStartSLOduration=2.191361283 podStartE2EDuration="2.191361283s" podCreationTimestamp="2025-12-03 12:53:17 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:53:19.190295803 +0000 UTC m=+5589.778537567" watchObservedRunningTime="2025-12-03 12:53:19.191361283 +0000 UTC m=+5589.779603047" Dec 03 12:53:19 crc kubenswrapper[5052]: I1203 12:53:19.215827 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-api-0" podStartSLOduration=3.215804822 podStartE2EDuration="3.215804822s" podCreationTimestamp="2025-12-03 12:53:16 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:53:19.208956976 +0000 UTC m=+5589.797198750" watchObservedRunningTime="2025-12-03 12:53:19.215804822 +0000 UTC m=+5589.804046586" Dec 03 12:53:20 crc kubenswrapper[5052]: I1203 12:53:20.743270 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:53:20 crc kubenswrapper[5052]: I1203 12:53:20.745719 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:53:21 crc kubenswrapper[5052]: I1203 12:53:21.749794 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-metadata-0" podUID="1e7ef579-72f5-41e2-9374-d2dcab169471" containerName="nova-metadata-log" probeResult="failure" output="Get \"https://10.217.0.213:8775/\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Dec 03 12:53:21 crc kubenswrapper[5052]: I1203 12:53:21.755832 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-metadata-0" podUID="1e7ef579-72f5-41e2-9374-d2dcab169471" containerName="nova-metadata-metadata" probeResult="failure" output="Get \"https://10.217.0.213:8775/\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Dec 03 12:53:22 crc kubenswrapper[5052]: I1203 12:53:22.566016 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:53:22 crc kubenswrapper[5052]: I1203 12:53:22.631492 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:53:22 crc kubenswrapper[5052]: I1203 12:53:22.991357 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-cell1-cell-mapping-t6brm"] Dec 03 12:53:22 crc kubenswrapper[5052]: I1203 12:53:22.992608 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-cell-mapping-t6brm" Dec 03 12:53:22 crc kubenswrapper[5052]: I1203 12:53:22.996471 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell1-manage-config-data" Dec 03 12:53:23 crc kubenswrapper[5052]: I1203 12:53:23.001137 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell1-manage-scripts" Dec 03 12:53:23 crc kubenswrapper[5052]: I1203 12:53:23.021269 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-cell-mapping-t6brm"] Dec 03 12:53:23 crc kubenswrapper[5052]: I1203 12:53:23.116425 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/337505df-59b3-4007-9855-05616985bfc6-scripts\") pod \"nova-cell1-cell-mapping-t6brm\" (UID: \"337505df-59b3-4007-9855-05616985bfc6\") " pod="openstack-kuttl-tests/nova-cell1-cell-mapping-t6brm" Dec 03 12:53:23 crc kubenswrapper[5052]: I1203 12:53:23.116533 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/337505df-59b3-4007-9855-05616985bfc6-config-data\") pod \"nova-cell1-cell-mapping-t6brm\" (UID: \"337505df-59b3-4007-9855-05616985bfc6\") " pod="openstack-kuttl-tests/nova-cell1-cell-mapping-t6brm" Dec 03 12:53:23 crc kubenswrapper[5052]: I1203 12:53:23.116560 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fx2cb\" (UniqueName: \"kubernetes.io/projected/337505df-59b3-4007-9855-05616985bfc6-kube-api-access-fx2cb\") pod \"nova-cell1-cell-mapping-t6brm\" (UID: \"337505df-59b3-4007-9855-05616985bfc6\") " pod="openstack-kuttl-tests/nova-cell1-cell-mapping-t6brm" Dec 03 12:53:23 crc kubenswrapper[5052]: I1203 12:53:23.116605 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/337505df-59b3-4007-9855-05616985bfc6-combined-ca-bundle\") pod \"nova-cell1-cell-mapping-t6brm\" (UID: \"337505df-59b3-4007-9855-05616985bfc6\") " pod="openstack-kuttl-tests/nova-cell1-cell-mapping-t6brm" Dec 03 12:53:23 crc kubenswrapper[5052]: I1203 12:53:23.218166 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/337505df-59b3-4007-9855-05616985bfc6-scripts\") pod \"nova-cell1-cell-mapping-t6brm\" (UID: \"337505df-59b3-4007-9855-05616985bfc6\") " pod="openstack-kuttl-tests/nova-cell1-cell-mapping-t6brm" Dec 03 12:53:23 crc kubenswrapper[5052]: I1203 12:53:23.218296 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/337505df-59b3-4007-9855-05616985bfc6-config-data\") pod \"nova-cell1-cell-mapping-t6brm\" (UID: \"337505df-59b3-4007-9855-05616985bfc6\") " pod="openstack-kuttl-tests/nova-cell1-cell-mapping-t6brm" Dec 03 12:53:23 crc kubenswrapper[5052]: I1203 12:53:23.218321 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-fx2cb\" (UniqueName: \"kubernetes.io/projected/337505df-59b3-4007-9855-05616985bfc6-kube-api-access-fx2cb\") pod \"nova-cell1-cell-mapping-t6brm\" (UID: \"337505df-59b3-4007-9855-05616985bfc6\") " pod="openstack-kuttl-tests/nova-cell1-cell-mapping-t6brm" Dec 03 12:53:23 crc kubenswrapper[5052]: I1203 12:53:23.218376 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/337505df-59b3-4007-9855-05616985bfc6-combined-ca-bundle\") pod \"nova-cell1-cell-mapping-t6brm\" (UID: \"337505df-59b3-4007-9855-05616985bfc6\") " pod="openstack-kuttl-tests/nova-cell1-cell-mapping-t6brm" Dec 03 12:53:23 crc kubenswrapper[5052]: I1203 12:53:23.225095 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/337505df-59b3-4007-9855-05616985bfc6-scripts\") pod \"nova-cell1-cell-mapping-t6brm\" (UID: \"337505df-59b3-4007-9855-05616985bfc6\") " pod="openstack-kuttl-tests/nova-cell1-cell-mapping-t6brm" Dec 03 12:53:23 crc kubenswrapper[5052]: I1203 12:53:23.227118 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/337505df-59b3-4007-9855-05616985bfc6-combined-ca-bundle\") pod \"nova-cell1-cell-mapping-t6brm\" (UID: \"337505df-59b3-4007-9855-05616985bfc6\") " pod="openstack-kuttl-tests/nova-cell1-cell-mapping-t6brm" Dec 03 12:53:23 crc kubenswrapper[5052]: I1203 12:53:23.231311 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/337505df-59b3-4007-9855-05616985bfc6-config-data\") pod \"nova-cell1-cell-mapping-t6brm\" (UID: \"337505df-59b3-4007-9855-05616985bfc6\") " pod="openstack-kuttl-tests/nova-cell1-cell-mapping-t6brm" Dec 03 12:53:23 crc kubenswrapper[5052]: I1203 12:53:23.241643 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-fx2cb\" (UniqueName: \"kubernetes.io/projected/337505df-59b3-4007-9855-05616985bfc6-kube-api-access-fx2cb\") pod \"nova-cell1-cell-mapping-t6brm\" (UID: \"337505df-59b3-4007-9855-05616985bfc6\") " pod="openstack-kuttl-tests/nova-cell1-cell-mapping-t6brm" Dec 03 12:53:23 crc kubenswrapper[5052]: I1203 12:53:23.316141 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-cell-mapping-t6brm" Dec 03 12:53:23 crc kubenswrapper[5052]: I1203 12:53:23.789461 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-cell1-cell-mapping-t6brm"] Dec 03 12:53:24 crc kubenswrapper[5052]: I1203 12:53:24.534031 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-cell-mapping-t6brm" event={"ID":"337505df-59b3-4007-9855-05616985bfc6","Type":"ContainerStarted","Data":"d2b9e7f45cd74b7356316ded6d8d6691148221f1ac450e577bbac1e71ab3d68c"} Dec 03 12:53:24 crc kubenswrapper[5052]: I1203 12:53:24.534394 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-cell-mapping-t6brm" event={"ID":"337505df-59b3-4007-9855-05616985bfc6","Type":"ContainerStarted","Data":"2e77c3ab0db5cb09236f51501a1292aa4e27c7dd34cb0d8c3be6bf93754b2efa"} Dec 03 12:53:24 crc kubenswrapper[5052]: I1203 12:53:24.556131 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-cell1-cell-mapping-t6brm" podStartSLOduration=2.556109955 podStartE2EDuration="2.556109955s" podCreationTimestamp="2025-12-03 12:53:22 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:53:24.553876481 +0000 UTC m=+5595.142118245" watchObservedRunningTime="2025-12-03 12:53:24.556109955 +0000 UTC m=+5595.144351719" Dec 03 12:53:26 crc kubenswrapper[5052]: E1203 12:53:26.601732 5052 cadvisor_stats_provider.go:516] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb0bb2e93_76f1_4485_b822_41e83b2a4f59.slice/crio-29b21fda6544a2343736d96bac4c7f7476a926e688d5808f8b0fa0235fbd671a\": RecentStats: unable to find data in memory cache]" Dec 03 12:53:26 crc kubenswrapper[5052]: I1203 12:53:26.818370 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:53:26 crc kubenswrapper[5052]: I1203 12:53:26.818425 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:53:27 crc kubenswrapper[5052]: I1203 12:53:27.630910 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:53:27 crc kubenswrapper[5052]: I1203 12:53:27.661494 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:53:27 crc kubenswrapper[5052]: I1203 12:53:27.900731 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-api-0" podUID="c7d71496-0273-4f3d-9f21-3848037800ff" containerName="nova-api-api" probeResult="failure" output="Get \"http://10.217.0.215:8774/\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Dec 03 12:53:27 crc kubenswrapper[5052]: I1203 12:53:27.900752 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-api-0" podUID="c7d71496-0273-4f3d-9f21-3848037800ff" containerName="nova-api-log" probeResult="failure" output="Get \"http://10.217.0.215:8774/\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Dec 03 12:53:28 crc kubenswrapper[5052]: I1203 12:53:28.609897 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:53:29 crc kubenswrapper[5052]: I1203 12:53:29.578546 5052 generic.go:334] "Generic (PLEG): container finished" podID="337505df-59b3-4007-9855-05616985bfc6" containerID="d2b9e7f45cd74b7356316ded6d8d6691148221f1ac450e577bbac1e71ab3d68c" exitCode=0 Dec 03 12:53:29 crc kubenswrapper[5052]: I1203 12:53:29.578644 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-cell-mapping-t6brm" event={"ID":"337505df-59b3-4007-9855-05616985bfc6","Type":"ContainerDied","Data":"d2b9e7f45cd74b7356316ded6d8d6691148221f1ac450e577bbac1e71ab3d68c"} Dec 03 12:53:30 crc kubenswrapper[5052]: I1203 12:53:30.752173 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:53:30 crc kubenswrapper[5052]: I1203 12:53:30.762034 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:53:30 crc kubenswrapper[5052]: I1203 12:53:30.762377 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:53:30 crc kubenswrapper[5052]: I1203 12:53:30.956498 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-cell-mapping-t6brm" Dec 03 12:53:31 crc kubenswrapper[5052]: I1203 12:53:31.086002 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/337505df-59b3-4007-9855-05616985bfc6-combined-ca-bundle\") pod \"337505df-59b3-4007-9855-05616985bfc6\" (UID: \"337505df-59b3-4007-9855-05616985bfc6\") " Dec 03 12:53:31 crc kubenswrapper[5052]: I1203 12:53:31.086070 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fx2cb\" (UniqueName: \"kubernetes.io/projected/337505df-59b3-4007-9855-05616985bfc6-kube-api-access-fx2cb\") pod \"337505df-59b3-4007-9855-05616985bfc6\" (UID: \"337505df-59b3-4007-9855-05616985bfc6\") " Dec 03 12:53:31 crc kubenswrapper[5052]: I1203 12:53:31.086097 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/337505df-59b3-4007-9855-05616985bfc6-config-data\") pod \"337505df-59b3-4007-9855-05616985bfc6\" (UID: \"337505df-59b3-4007-9855-05616985bfc6\") " Dec 03 12:53:31 crc kubenswrapper[5052]: I1203 12:53:31.086211 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/337505df-59b3-4007-9855-05616985bfc6-scripts\") pod \"337505df-59b3-4007-9855-05616985bfc6\" (UID: \"337505df-59b3-4007-9855-05616985bfc6\") " Dec 03 12:53:31 crc kubenswrapper[5052]: I1203 12:53:31.092339 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/337505df-59b3-4007-9855-05616985bfc6-kube-api-access-fx2cb" (OuterVolumeSpecName: "kube-api-access-fx2cb") pod "337505df-59b3-4007-9855-05616985bfc6" (UID: "337505df-59b3-4007-9855-05616985bfc6"). InnerVolumeSpecName "kube-api-access-fx2cb". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:53:31 crc kubenswrapper[5052]: I1203 12:53:31.092651 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/337505df-59b3-4007-9855-05616985bfc6-scripts" (OuterVolumeSpecName: "scripts") pod "337505df-59b3-4007-9855-05616985bfc6" (UID: "337505df-59b3-4007-9855-05616985bfc6"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:53:31 crc kubenswrapper[5052]: I1203 12:53:31.120773 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/337505df-59b3-4007-9855-05616985bfc6-config-data" (OuterVolumeSpecName: "config-data") pod "337505df-59b3-4007-9855-05616985bfc6" (UID: "337505df-59b3-4007-9855-05616985bfc6"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:53:31 crc kubenswrapper[5052]: I1203 12:53:31.129310 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/337505df-59b3-4007-9855-05616985bfc6-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "337505df-59b3-4007-9855-05616985bfc6" (UID: "337505df-59b3-4007-9855-05616985bfc6"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:53:31 crc kubenswrapper[5052]: I1203 12:53:31.188633 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/337505df-59b3-4007-9855-05616985bfc6-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:53:31 crc kubenswrapper[5052]: I1203 12:53:31.188675 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/337505df-59b3-4007-9855-05616985bfc6-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:53:31 crc kubenswrapper[5052]: I1203 12:53:31.188693 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-fx2cb\" (UniqueName: \"kubernetes.io/projected/337505df-59b3-4007-9855-05616985bfc6-kube-api-access-fx2cb\") on node \"crc\" DevicePath \"\"" Dec 03 12:53:31 crc kubenswrapper[5052]: I1203 12:53:31.188706 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/337505df-59b3-4007-9855-05616985bfc6-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:53:31 crc kubenswrapper[5052]: I1203 12:53:31.602000 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-cell-mapping-t6brm" Dec 03 12:53:31 crc kubenswrapper[5052]: I1203 12:53:31.602047 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-cell-mapping-t6brm" event={"ID":"337505df-59b3-4007-9855-05616985bfc6","Type":"ContainerDied","Data":"2e77c3ab0db5cb09236f51501a1292aa4e27c7dd34cb0d8c3be6bf93754b2efa"} Dec 03 12:53:31 crc kubenswrapper[5052]: I1203 12:53:31.602070 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2e77c3ab0db5cb09236f51501a1292aa4e27c7dd34cb0d8c3be6bf93754b2efa" Dec 03 12:53:31 crc kubenswrapper[5052]: I1203 12:53:31.609473 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:53:31 crc kubenswrapper[5052]: I1203 12:53:31.787811 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:53:31 crc kubenswrapper[5052]: I1203 12:53:31.788085 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-api-0" podUID="c7d71496-0273-4f3d-9f21-3848037800ff" containerName="nova-api-log" containerID="cri-o://e3ed6fc1c0c369792c4bf196651ad2c3593bedb2a55fc0a57ff66f53c8f194ab" gracePeriod=30 Dec 03 12:53:31 crc kubenswrapper[5052]: I1203 12:53:31.788170 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-api-0" podUID="c7d71496-0273-4f3d-9f21-3848037800ff" containerName="nova-api-api" containerID="cri-o://2b509ad5f1c30962f5faa8d131f05da4b7c14268ac8f7cec4bee50543949b838" gracePeriod=30 Dec 03 12:53:31 crc kubenswrapper[5052]: I1203 12:53:31.856899 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:53:31 crc kubenswrapper[5052]: I1203 12:53:31.857157 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-scheduler-0" podUID="5930ff4b-2768-4636-af3f-e5395bb93f49" containerName="nova-scheduler-scheduler" containerID="cri-o://7498f540d70df6b99e8086ff89ab5331440cd2c5e65aed3c60c8d119b3260aba" gracePeriod=30 Dec 03 12:53:31 crc kubenswrapper[5052]: I1203 12:53:31.871493 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:53:32 crc kubenswrapper[5052]: I1203 12:53:32.613573 5052 generic.go:334] "Generic (PLEG): container finished" podID="c7d71496-0273-4f3d-9f21-3848037800ff" containerID="e3ed6fc1c0c369792c4bf196651ad2c3593bedb2a55fc0a57ff66f53c8f194ab" exitCode=143 Dec 03 12:53:32 crc kubenswrapper[5052]: I1203 12:53:32.613646 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"c7d71496-0273-4f3d-9f21-3848037800ff","Type":"ContainerDied","Data":"e3ed6fc1c0c369792c4bf196651ad2c3593bedb2a55fc0a57ff66f53c8f194ab"} Dec 03 12:53:32 crc kubenswrapper[5052]: E1203 12:53:32.633687 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="7498f540d70df6b99e8086ff89ab5331440cd2c5e65aed3c60c8d119b3260aba" cmd=["/usr/bin/pgrep","-r","DRST","nova-scheduler"] Dec 03 12:53:32 crc kubenswrapper[5052]: E1203 12:53:32.635391 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="7498f540d70df6b99e8086ff89ab5331440cd2c5e65aed3c60c8d119b3260aba" cmd=["/usr/bin/pgrep","-r","DRST","nova-scheduler"] Dec 03 12:53:32 crc kubenswrapper[5052]: E1203 12:53:32.637075 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="7498f540d70df6b99e8086ff89ab5331440cd2c5e65aed3c60c8d119b3260aba" cmd=["/usr/bin/pgrep","-r","DRST","nova-scheduler"] Dec 03 12:53:32 crc kubenswrapper[5052]: E1203 12:53:32.637172 5052 prober.go:104] "Probe errored" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" probeType="Readiness" pod="openstack-kuttl-tests/nova-scheduler-0" podUID="5930ff4b-2768-4636-af3f-e5395bb93f49" containerName="nova-scheduler-scheduler" Dec 03 12:53:33 crc kubenswrapper[5052]: I1203 12:53:33.637795 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-metadata-0" podUID="1e7ef579-72f5-41e2-9374-d2dcab169471" containerName="nova-metadata-log" containerID="cri-o://d6031286fa2a90c842cf1e6e7b88e513c06f9131324674efcd3cb78de18ca18f" gracePeriod=30 Dec 03 12:53:33 crc kubenswrapper[5052]: I1203 12:53:33.638165 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-metadata-0" podUID="1e7ef579-72f5-41e2-9374-d2dcab169471" containerName="nova-metadata-metadata" containerID="cri-o://19c711c57e77c66d39bf6c4bd8d5e28dab2b329bf459ff2daed1570bc217ec16" gracePeriod=30 Dec 03 12:53:34 crc kubenswrapper[5052]: I1203 12:53:34.648938 5052 generic.go:334] "Generic (PLEG): container finished" podID="1e7ef579-72f5-41e2-9374-d2dcab169471" containerID="d6031286fa2a90c842cf1e6e7b88e513c06f9131324674efcd3cb78de18ca18f" exitCode=143 Dec 03 12:53:34 crc kubenswrapper[5052]: I1203 12:53:34.649077 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"1e7ef579-72f5-41e2-9374-d2dcab169471","Type":"ContainerDied","Data":"d6031286fa2a90c842cf1e6e7b88e513c06f9131324674efcd3cb78de18ca18f"} Dec 03 12:53:35 crc kubenswrapper[5052]: I1203 12:53:35.127372 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:53:35 crc kubenswrapper[5052]: I1203 12:53:35.272666 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/5930ff4b-2768-4636-af3f-e5395bb93f49-config-data\") pod \"5930ff4b-2768-4636-af3f-e5395bb93f49\" (UID: \"5930ff4b-2768-4636-af3f-e5395bb93f49\") " Dec 03 12:53:35 crc kubenswrapper[5052]: I1203 12:53:35.273078 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5930ff4b-2768-4636-af3f-e5395bb93f49-combined-ca-bundle\") pod \"5930ff4b-2768-4636-af3f-e5395bb93f49\" (UID: \"5930ff4b-2768-4636-af3f-e5395bb93f49\") " Dec 03 12:53:35 crc kubenswrapper[5052]: I1203 12:53:35.273291 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mnj5b\" (UniqueName: \"kubernetes.io/projected/5930ff4b-2768-4636-af3f-e5395bb93f49-kube-api-access-mnj5b\") pod \"5930ff4b-2768-4636-af3f-e5395bb93f49\" (UID: \"5930ff4b-2768-4636-af3f-e5395bb93f49\") " Dec 03 12:53:35 crc kubenswrapper[5052]: I1203 12:53:35.278269 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5930ff4b-2768-4636-af3f-e5395bb93f49-kube-api-access-mnj5b" (OuterVolumeSpecName: "kube-api-access-mnj5b") pod "5930ff4b-2768-4636-af3f-e5395bb93f49" (UID: "5930ff4b-2768-4636-af3f-e5395bb93f49"). InnerVolumeSpecName "kube-api-access-mnj5b". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:53:35 crc kubenswrapper[5052]: I1203 12:53:35.309639 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5930ff4b-2768-4636-af3f-e5395bb93f49-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "5930ff4b-2768-4636-af3f-e5395bb93f49" (UID: "5930ff4b-2768-4636-af3f-e5395bb93f49"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:53:35 crc kubenswrapper[5052]: I1203 12:53:35.309721 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5930ff4b-2768-4636-af3f-e5395bb93f49-config-data" (OuterVolumeSpecName: "config-data") pod "5930ff4b-2768-4636-af3f-e5395bb93f49" (UID: "5930ff4b-2768-4636-af3f-e5395bb93f49"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:53:35 crc kubenswrapper[5052]: I1203 12:53:35.358467 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:53:35 crc kubenswrapper[5052]: I1203 12:53:35.375230 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/c7d71496-0273-4f3d-9f21-3848037800ff-logs\") pod \"c7d71496-0273-4f3d-9f21-3848037800ff\" (UID: \"c7d71496-0273-4f3d-9f21-3848037800ff\") " Dec 03 12:53:35 crc kubenswrapper[5052]: I1203 12:53:35.375301 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6fg52\" (UniqueName: \"kubernetes.io/projected/c7d71496-0273-4f3d-9f21-3848037800ff-kube-api-access-6fg52\") pod \"c7d71496-0273-4f3d-9f21-3848037800ff\" (UID: \"c7d71496-0273-4f3d-9f21-3848037800ff\") " Dec 03 12:53:35 crc kubenswrapper[5052]: I1203 12:53:35.375374 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c7d71496-0273-4f3d-9f21-3848037800ff-combined-ca-bundle\") pod \"c7d71496-0273-4f3d-9f21-3848037800ff\" (UID: \"c7d71496-0273-4f3d-9f21-3848037800ff\") " Dec 03 12:53:35 crc kubenswrapper[5052]: I1203 12:53:35.375781 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c7d71496-0273-4f3d-9f21-3848037800ff-logs" (OuterVolumeSpecName: "logs") pod "c7d71496-0273-4f3d-9f21-3848037800ff" (UID: "c7d71496-0273-4f3d-9f21-3848037800ff"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:53:35 crc kubenswrapper[5052]: I1203 12:53:35.375986 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-mnj5b\" (UniqueName: \"kubernetes.io/projected/5930ff4b-2768-4636-af3f-e5395bb93f49-kube-api-access-mnj5b\") on node \"crc\" DevicePath \"\"" Dec 03 12:53:35 crc kubenswrapper[5052]: I1203 12:53:35.376015 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/5930ff4b-2768-4636-af3f-e5395bb93f49-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:53:35 crc kubenswrapper[5052]: I1203 12:53:35.376031 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5930ff4b-2768-4636-af3f-e5395bb93f49-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:53:35 crc kubenswrapper[5052]: I1203 12:53:35.376042 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/c7d71496-0273-4f3d-9f21-3848037800ff-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:53:35 crc kubenswrapper[5052]: I1203 12:53:35.379287 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c7d71496-0273-4f3d-9f21-3848037800ff-kube-api-access-6fg52" (OuterVolumeSpecName: "kube-api-access-6fg52") pod "c7d71496-0273-4f3d-9f21-3848037800ff" (UID: "c7d71496-0273-4f3d-9f21-3848037800ff"). InnerVolumeSpecName "kube-api-access-6fg52". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:53:35 crc kubenswrapper[5052]: I1203 12:53:35.405190 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c7d71496-0273-4f3d-9f21-3848037800ff-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "c7d71496-0273-4f3d-9f21-3848037800ff" (UID: "c7d71496-0273-4f3d-9f21-3848037800ff"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:53:35 crc kubenswrapper[5052]: I1203 12:53:35.476905 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/c7d71496-0273-4f3d-9f21-3848037800ff-config-data\") pod \"c7d71496-0273-4f3d-9f21-3848037800ff\" (UID: \"c7d71496-0273-4f3d-9f21-3848037800ff\") " Dec 03 12:53:35 crc kubenswrapper[5052]: I1203 12:53:35.477675 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-6fg52\" (UniqueName: \"kubernetes.io/projected/c7d71496-0273-4f3d-9f21-3848037800ff-kube-api-access-6fg52\") on node \"crc\" DevicePath \"\"" Dec 03 12:53:35 crc kubenswrapper[5052]: I1203 12:53:35.477715 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c7d71496-0273-4f3d-9f21-3848037800ff-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:53:35 crc kubenswrapper[5052]: I1203 12:53:35.507773 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c7d71496-0273-4f3d-9f21-3848037800ff-config-data" (OuterVolumeSpecName: "config-data") pod "c7d71496-0273-4f3d-9f21-3848037800ff" (UID: "c7d71496-0273-4f3d-9f21-3848037800ff"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:53:35 crc kubenswrapper[5052]: I1203 12:53:35.579662 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/c7d71496-0273-4f3d-9f21-3848037800ff-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:53:35 crc kubenswrapper[5052]: I1203 12:53:35.648482 5052 scope.go:117] "RemoveContainer" containerID="f8b9d65f292776aa251d63e395ac7e5031feb16b30ea525e3b1072e08b062178" Dec 03 12:53:35 crc kubenswrapper[5052]: I1203 12:53:35.663362 5052 generic.go:334] "Generic (PLEG): container finished" podID="c7d71496-0273-4f3d-9f21-3848037800ff" containerID="2b509ad5f1c30962f5faa8d131f05da4b7c14268ac8f7cec4bee50543949b838" exitCode=0 Dec 03 12:53:35 crc kubenswrapper[5052]: I1203 12:53:35.663436 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:53:35 crc kubenswrapper[5052]: I1203 12:53:35.663460 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"c7d71496-0273-4f3d-9f21-3848037800ff","Type":"ContainerDied","Data":"2b509ad5f1c30962f5faa8d131f05da4b7c14268ac8f7cec4bee50543949b838"} Dec 03 12:53:35 crc kubenswrapper[5052]: I1203 12:53:35.663501 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"c7d71496-0273-4f3d-9f21-3848037800ff","Type":"ContainerDied","Data":"dce448abb2df885cc3e486c3248a809b055e7460ef4b57579e4f233091b421b8"} Dec 03 12:53:35 crc kubenswrapper[5052]: I1203 12:53:35.663541 5052 scope.go:117] "RemoveContainer" containerID="2b509ad5f1c30962f5faa8d131f05da4b7c14268ac8f7cec4bee50543949b838" Dec 03 12:53:35 crc kubenswrapper[5052]: I1203 12:53:35.667355 5052 generic.go:334] "Generic (PLEG): container finished" podID="5930ff4b-2768-4636-af3f-e5395bb93f49" containerID="7498f540d70df6b99e8086ff89ab5331440cd2c5e65aed3c60c8d119b3260aba" exitCode=0 Dec 03 12:53:35 crc kubenswrapper[5052]: I1203 12:53:35.667389 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"5930ff4b-2768-4636-af3f-e5395bb93f49","Type":"ContainerDied","Data":"7498f540d70df6b99e8086ff89ab5331440cd2c5e65aed3c60c8d119b3260aba"} Dec 03 12:53:35 crc kubenswrapper[5052]: I1203 12:53:35.667416 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"5930ff4b-2768-4636-af3f-e5395bb93f49","Type":"ContainerDied","Data":"6f20fa9d8650553f0998b0bce070f5158cd0e6889e646f6de06461b8a4aa07d0"} Dec 03 12:53:35 crc kubenswrapper[5052]: I1203 12:53:35.667424 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:53:35 crc kubenswrapper[5052]: I1203 12:53:35.679158 5052 scope.go:117] "RemoveContainer" containerID="d42c23e387ad7773d4b53a13fb7267499e89a43ce6fa0b570fa164ddcc0ed712" Dec 03 12:53:35 crc kubenswrapper[5052]: I1203 12:53:35.700750 5052 scope.go:117] "RemoveContainer" containerID="e3ed6fc1c0c369792c4bf196651ad2c3593bedb2a55fc0a57ff66f53c8f194ab" Dec 03 12:53:35 crc kubenswrapper[5052]: I1203 12:53:35.723308 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:53:35 crc kubenswrapper[5052]: I1203 12:53:35.771147 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:53:35 crc kubenswrapper[5052]: I1203 12:53:35.776645 5052 scope.go:117] "RemoveContainer" containerID="bf37583e842ac2986eebf906db0e42704630e142c5ffc66b3b257042bb9c2bf5" Dec 03 12:53:35 crc kubenswrapper[5052]: I1203 12:53:35.789636 5052 scope.go:117] "RemoveContainer" containerID="2b509ad5f1c30962f5faa8d131f05da4b7c14268ac8f7cec4bee50543949b838" Dec 03 12:53:35 crc kubenswrapper[5052]: E1203 12:53:35.790075 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2b509ad5f1c30962f5faa8d131f05da4b7c14268ac8f7cec4bee50543949b838\": container with ID starting with 2b509ad5f1c30962f5faa8d131f05da4b7c14268ac8f7cec4bee50543949b838 not found: ID does not exist" containerID="2b509ad5f1c30962f5faa8d131f05da4b7c14268ac8f7cec4bee50543949b838" Dec 03 12:53:35 crc kubenswrapper[5052]: I1203 12:53:35.790112 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2b509ad5f1c30962f5faa8d131f05da4b7c14268ac8f7cec4bee50543949b838"} err="failed to get container status \"2b509ad5f1c30962f5faa8d131f05da4b7c14268ac8f7cec4bee50543949b838\": rpc error: code = NotFound desc = could not find container \"2b509ad5f1c30962f5faa8d131f05da4b7c14268ac8f7cec4bee50543949b838\": container with ID starting with 2b509ad5f1c30962f5faa8d131f05da4b7c14268ac8f7cec4bee50543949b838 not found: ID does not exist" Dec 03 12:53:35 crc kubenswrapper[5052]: I1203 12:53:35.790140 5052 scope.go:117] "RemoveContainer" containerID="e3ed6fc1c0c369792c4bf196651ad2c3593bedb2a55fc0a57ff66f53c8f194ab" Dec 03 12:53:35 crc kubenswrapper[5052]: E1203 12:53:35.790610 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e3ed6fc1c0c369792c4bf196651ad2c3593bedb2a55fc0a57ff66f53c8f194ab\": container with ID starting with e3ed6fc1c0c369792c4bf196651ad2c3593bedb2a55fc0a57ff66f53c8f194ab not found: ID does not exist" containerID="e3ed6fc1c0c369792c4bf196651ad2c3593bedb2a55fc0a57ff66f53c8f194ab" Dec 03 12:53:35 crc kubenswrapper[5052]: I1203 12:53:35.790637 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e3ed6fc1c0c369792c4bf196651ad2c3593bedb2a55fc0a57ff66f53c8f194ab"} err="failed to get container status \"e3ed6fc1c0c369792c4bf196651ad2c3593bedb2a55fc0a57ff66f53c8f194ab\": rpc error: code = NotFound desc = could not find container \"e3ed6fc1c0c369792c4bf196651ad2c3593bedb2a55fc0a57ff66f53c8f194ab\": container with ID starting with e3ed6fc1c0c369792c4bf196651ad2c3593bedb2a55fc0a57ff66f53c8f194ab not found: ID does not exist" Dec 03 12:53:35 crc kubenswrapper[5052]: I1203 12:53:35.790652 5052 scope.go:117] "RemoveContainer" containerID="7498f540d70df6b99e8086ff89ab5331440cd2c5e65aed3c60c8d119b3260aba" Dec 03 12:53:35 crc kubenswrapper[5052]: I1203 12:53:35.796464 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:53:35 crc kubenswrapper[5052]: I1203 12:53:35.807193 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:53:35 crc kubenswrapper[5052]: I1203 12:53:35.812786 5052 scope.go:117] "RemoveContainer" containerID="786d9178f57d4b0d285a0e72f6cd93f8939028d39964463dca3a233c79a6225d" Dec 03 12:53:35 crc kubenswrapper[5052]: I1203 12:53:35.812984 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:53:35 crc kubenswrapper[5052]: E1203 12:53:35.813537 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="c7d71496-0273-4f3d-9f21-3848037800ff" containerName="nova-api-log" Dec 03 12:53:35 crc kubenswrapper[5052]: I1203 12:53:35.813558 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="c7d71496-0273-4f3d-9f21-3848037800ff" containerName="nova-api-log" Dec 03 12:53:35 crc kubenswrapper[5052]: E1203 12:53:35.813587 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="337505df-59b3-4007-9855-05616985bfc6" containerName="nova-manage" Dec 03 12:53:35 crc kubenswrapper[5052]: I1203 12:53:35.813597 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="337505df-59b3-4007-9855-05616985bfc6" containerName="nova-manage" Dec 03 12:53:35 crc kubenswrapper[5052]: E1203 12:53:35.813622 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="5930ff4b-2768-4636-af3f-e5395bb93f49" containerName="nova-scheduler-scheduler" Dec 03 12:53:35 crc kubenswrapper[5052]: I1203 12:53:35.813630 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="5930ff4b-2768-4636-af3f-e5395bb93f49" containerName="nova-scheduler-scheduler" Dec 03 12:53:35 crc kubenswrapper[5052]: E1203 12:53:35.813644 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="c7d71496-0273-4f3d-9f21-3848037800ff" containerName="nova-api-api" Dec 03 12:53:35 crc kubenswrapper[5052]: I1203 12:53:35.813651 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="c7d71496-0273-4f3d-9f21-3848037800ff" containerName="nova-api-api" Dec 03 12:53:35 crc kubenswrapper[5052]: I1203 12:53:35.813886 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="337505df-59b3-4007-9855-05616985bfc6" containerName="nova-manage" Dec 03 12:53:35 crc kubenswrapper[5052]: I1203 12:53:35.813907 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="5930ff4b-2768-4636-af3f-e5395bb93f49" containerName="nova-scheduler-scheduler" Dec 03 12:53:35 crc kubenswrapper[5052]: I1203 12:53:35.813927 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="c7d71496-0273-4f3d-9f21-3848037800ff" containerName="nova-api-api" Dec 03 12:53:35 crc kubenswrapper[5052]: I1203 12:53:35.813954 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="c7d71496-0273-4f3d-9f21-3848037800ff" containerName="nova-api-log" Dec 03 12:53:35 crc kubenswrapper[5052]: I1203 12:53:35.815255 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:53:35 crc kubenswrapper[5052]: I1203 12:53:35.817349 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-api-config-data" Dec 03 12:53:35 crc kubenswrapper[5052]: I1203 12:53:35.821892 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:53:35 crc kubenswrapper[5052]: I1203 12:53:35.829716 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:53:35 crc kubenswrapper[5052]: I1203 12:53:35.831393 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:53:35 crc kubenswrapper[5052]: I1203 12:53:35.837269 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-scheduler-config-data" Dec 03 12:53:35 crc kubenswrapper[5052]: I1203 12:53:35.838738 5052 scope.go:117] "RemoveContainer" containerID="7498f540d70df6b99e8086ff89ab5331440cd2c5e65aed3c60c8d119b3260aba" Dec 03 12:53:35 crc kubenswrapper[5052]: I1203 12:53:35.839149 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:53:35 crc kubenswrapper[5052]: E1203 12:53:35.839286 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7498f540d70df6b99e8086ff89ab5331440cd2c5e65aed3c60c8d119b3260aba\": container with ID starting with 7498f540d70df6b99e8086ff89ab5331440cd2c5e65aed3c60c8d119b3260aba not found: ID does not exist" containerID="7498f540d70df6b99e8086ff89ab5331440cd2c5e65aed3c60c8d119b3260aba" Dec 03 12:53:35 crc kubenswrapper[5052]: I1203 12:53:35.839307 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7498f540d70df6b99e8086ff89ab5331440cd2c5e65aed3c60c8d119b3260aba"} err="failed to get container status \"7498f540d70df6b99e8086ff89ab5331440cd2c5e65aed3c60c8d119b3260aba\": rpc error: code = NotFound desc = could not find container \"7498f540d70df6b99e8086ff89ab5331440cd2c5e65aed3c60c8d119b3260aba\": container with ID starting with 7498f540d70df6b99e8086ff89ab5331440cd2c5e65aed3c60c8d119b3260aba not found: ID does not exist" Dec 03 12:53:35 crc kubenswrapper[5052]: I1203 12:53:35.856739 5052 scope.go:117] "RemoveContainer" containerID="fde954c2e50f163494a08fec446c2659dbced2f64031d8d4fb5557a44bc92f99" Dec 03 12:53:35 crc kubenswrapper[5052]: I1203 12:53:35.878703 5052 scope.go:117] "RemoveContainer" containerID="275c7f7b179720a472f88be78dd4725e7d963350bb431c4a491cf0a780e6019a" Dec 03 12:53:35 crc kubenswrapper[5052]: I1203 12:53:35.897097 5052 scope.go:117] "RemoveContainer" containerID="c79f26b11c1bba88e99d71d696bb0decec843bf868e7a4fdf02ceb21feddb964" Dec 03 12:53:35 crc kubenswrapper[5052]: I1203 12:53:35.931471 5052 scope.go:117] "RemoveContainer" containerID="363ded64823f176012eadd8f4efa965cf7bb21ad177ce1c3b4cc513c83f5ebab" Dec 03 12:53:35 crc kubenswrapper[5052]: I1203 12:53:35.982872 5052 scope.go:117] "RemoveContainer" containerID="795d64dcbb859f297be1ee826fd9ec0faf4da92063117b21eb577b822e4b7657" Dec 03 12:53:35 crc kubenswrapper[5052]: I1203 12:53:35.986322 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4ec49e76-2597-47d6-811b-f65cbe0580ff-combined-ca-bundle\") pod \"nova-scheduler-0\" (UID: \"4ec49e76-2597-47d6-811b-f65cbe0580ff\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:53:35 crc kubenswrapper[5052]: I1203 12:53:35.986384 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/cb5f26d2-4434-4f20-a7bc-49d58108d9ce-logs\") pod \"nova-api-0\" (UID: \"cb5f26d2-4434-4f20-a7bc-49d58108d9ce\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:53:35 crc kubenswrapper[5052]: I1203 12:53:35.986530 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r69xw\" (UniqueName: \"kubernetes.io/projected/cb5f26d2-4434-4f20-a7bc-49d58108d9ce-kube-api-access-r69xw\") pod \"nova-api-0\" (UID: \"cb5f26d2-4434-4f20-a7bc-49d58108d9ce\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:53:35 crc kubenswrapper[5052]: I1203 12:53:35.986578 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hnfq6\" (UniqueName: \"kubernetes.io/projected/4ec49e76-2597-47d6-811b-f65cbe0580ff-kube-api-access-hnfq6\") pod \"nova-scheduler-0\" (UID: \"4ec49e76-2597-47d6-811b-f65cbe0580ff\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:53:35 crc kubenswrapper[5052]: I1203 12:53:35.986607 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/cb5f26d2-4434-4f20-a7bc-49d58108d9ce-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"cb5f26d2-4434-4f20-a7bc-49d58108d9ce\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:53:35 crc kubenswrapper[5052]: I1203 12:53:35.987679 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/cb5f26d2-4434-4f20-a7bc-49d58108d9ce-config-data\") pod \"nova-api-0\" (UID: \"cb5f26d2-4434-4f20-a7bc-49d58108d9ce\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:53:35 crc kubenswrapper[5052]: I1203 12:53:35.987708 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/4ec49e76-2597-47d6-811b-f65cbe0580ff-config-data\") pod \"nova-scheduler-0\" (UID: \"4ec49e76-2597-47d6-811b-f65cbe0580ff\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:53:36 crc kubenswrapper[5052]: I1203 12:53:36.015234 5052 scope.go:117] "RemoveContainer" containerID="1986f1fabd06880b71b76b4c94e01ab46c4e8d1cff3eae60d3b1cd7579b2c785" Dec 03 12:53:36 crc kubenswrapper[5052]: I1203 12:53:36.036439 5052 scope.go:117] "RemoveContainer" containerID="3a02ca07996460062957faa0b466285e3ee2c66925c520449ab5df8d346825d8" Dec 03 12:53:36 crc kubenswrapper[5052]: I1203 12:53:36.055499 5052 scope.go:117] "RemoveContainer" containerID="56d9b1a087e1cc1a1e32bb0b52dc44b84176e308fe03f914bdbda9333ae12466" Dec 03 12:53:36 crc kubenswrapper[5052]: I1203 12:53:36.075541 5052 scope.go:117] "RemoveContainer" containerID="15655b574d1d35cf1c932536528e9b95711b3ebc8dd150537384f5080772cd5e" Dec 03 12:53:36 crc kubenswrapper[5052]: I1203 12:53:36.089144 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4ec49e76-2597-47d6-811b-f65cbe0580ff-combined-ca-bundle\") pod \"nova-scheduler-0\" (UID: \"4ec49e76-2597-47d6-811b-f65cbe0580ff\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:53:36 crc kubenswrapper[5052]: I1203 12:53:36.089206 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/cb5f26d2-4434-4f20-a7bc-49d58108d9ce-logs\") pod \"nova-api-0\" (UID: \"cb5f26d2-4434-4f20-a7bc-49d58108d9ce\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:53:36 crc kubenswrapper[5052]: I1203 12:53:36.089259 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-r69xw\" (UniqueName: \"kubernetes.io/projected/cb5f26d2-4434-4f20-a7bc-49d58108d9ce-kube-api-access-r69xw\") pod \"nova-api-0\" (UID: \"cb5f26d2-4434-4f20-a7bc-49d58108d9ce\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:53:36 crc kubenswrapper[5052]: I1203 12:53:36.089295 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-hnfq6\" (UniqueName: \"kubernetes.io/projected/4ec49e76-2597-47d6-811b-f65cbe0580ff-kube-api-access-hnfq6\") pod \"nova-scheduler-0\" (UID: \"4ec49e76-2597-47d6-811b-f65cbe0580ff\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:53:36 crc kubenswrapper[5052]: I1203 12:53:36.089320 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/cb5f26d2-4434-4f20-a7bc-49d58108d9ce-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"cb5f26d2-4434-4f20-a7bc-49d58108d9ce\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:53:36 crc kubenswrapper[5052]: I1203 12:53:36.089349 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/cb5f26d2-4434-4f20-a7bc-49d58108d9ce-config-data\") pod \"nova-api-0\" (UID: \"cb5f26d2-4434-4f20-a7bc-49d58108d9ce\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:53:36 crc kubenswrapper[5052]: I1203 12:53:36.089367 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/4ec49e76-2597-47d6-811b-f65cbe0580ff-config-data\") pod \"nova-scheduler-0\" (UID: \"4ec49e76-2597-47d6-811b-f65cbe0580ff\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:53:36 crc kubenswrapper[5052]: I1203 12:53:36.089945 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/cb5f26d2-4434-4f20-a7bc-49d58108d9ce-logs\") pod \"nova-api-0\" (UID: \"cb5f26d2-4434-4f20-a7bc-49d58108d9ce\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:53:36 crc kubenswrapper[5052]: I1203 12:53:36.094303 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/cb5f26d2-4434-4f20-a7bc-49d58108d9ce-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"cb5f26d2-4434-4f20-a7bc-49d58108d9ce\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:53:36 crc kubenswrapper[5052]: I1203 12:53:36.094393 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4ec49e76-2597-47d6-811b-f65cbe0580ff-combined-ca-bundle\") pod \"nova-scheduler-0\" (UID: \"4ec49e76-2597-47d6-811b-f65cbe0580ff\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:53:36 crc kubenswrapper[5052]: I1203 12:53:36.097134 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/cb5f26d2-4434-4f20-a7bc-49d58108d9ce-config-data\") pod \"nova-api-0\" (UID: \"cb5f26d2-4434-4f20-a7bc-49d58108d9ce\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:53:36 crc kubenswrapper[5052]: I1203 12:53:36.103318 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/4ec49e76-2597-47d6-811b-f65cbe0580ff-config-data\") pod \"nova-scheduler-0\" (UID: \"4ec49e76-2597-47d6-811b-f65cbe0580ff\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:53:36 crc kubenswrapper[5052]: I1203 12:53:36.106110 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-hnfq6\" (UniqueName: \"kubernetes.io/projected/4ec49e76-2597-47d6-811b-f65cbe0580ff-kube-api-access-hnfq6\") pod \"nova-scheduler-0\" (UID: \"4ec49e76-2597-47d6-811b-f65cbe0580ff\") " pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:53:36 crc kubenswrapper[5052]: I1203 12:53:36.109297 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-r69xw\" (UniqueName: \"kubernetes.io/projected/cb5f26d2-4434-4f20-a7bc-49d58108d9ce-kube-api-access-r69xw\") pod \"nova-api-0\" (UID: \"cb5f26d2-4434-4f20-a7bc-49d58108d9ce\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:53:36 crc kubenswrapper[5052]: I1203 12:53:36.147347 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:53:36 crc kubenswrapper[5052]: I1203 12:53:36.168556 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:53:36 crc kubenswrapper[5052]: I1203 12:53:36.650174 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:53:36 crc kubenswrapper[5052]: I1203 12:53:36.686402 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"4ec49e76-2597-47d6-811b-f65cbe0580ff","Type":"ContainerStarted","Data":"dbda807a085f0dde69e7b6f3a0f75c39b7d48940e339baa86060bf949e62e21c"} Dec 03 12:53:36 crc kubenswrapper[5052]: I1203 12:53:36.739446 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:53:36 crc kubenswrapper[5052]: W1203 12:53:36.740246 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcb5f26d2_4434_4f20_a7bc_49d58108d9ce.slice/crio-d46a4912699adbf3859fb864b472e197b4bd54f2770ed821a5b9be07f3630f75 WatchSource:0}: Error finding container d46a4912699adbf3859fb864b472e197b4bd54f2770ed821a5b9be07f3630f75: Status 404 returned error can't find the container with id d46a4912699adbf3859fb864b472e197b4bd54f2770ed821a5b9be07f3630f75 Dec 03 12:53:36 crc kubenswrapper[5052]: I1203 12:53:36.801179 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/nova-metadata-0" podUID="1e7ef579-72f5-41e2-9374-d2dcab169471" containerName="nova-metadata-metadata" probeResult="failure" output="Get \"https://10.217.0.213:8775/\": read tcp 10.217.0.2:55612->10.217.0.213:8775: read: connection reset by peer" Dec 03 12:53:36 crc kubenswrapper[5052]: I1203 12:53:36.801199 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/nova-metadata-0" podUID="1e7ef579-72f5-41e2-9374-d2dcab169471" containerName="nova-metadata-log" probeResult="failure" output="Get \"https://10.217.0.213:8775/\": read tcp 10.217.0.2:55602->10.217.0.213:8775: read: connection reset by peer" Dec 03 12:53:36 crc kubenswrapper[5052]: E1203 12:53:36.897073 5052 cadvisor_stats_provider.go:516] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb0bb2e93_76f1_4485_b822_41e83b2a4f59.slice/crio-29b21fda6544a2343736d96bac4c7f7476a926e688d5808f8b0fa0235fbd671a\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1e7ef579_72f5_41e2_9374_d2dcab169471.slice/crio-19c711c57e77c66d39bf6c4bd8d5e28dab2b329bf459ff2daed1570bc217ec16.scope\": RecentStats: unable to find data in memory cache]" Dec 03 12:53:37 crc kubenswrapper[5052]: I1203 12:53:37.195979 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:53:37 crc kubenswrapper[5052]: I1203 12:53:37.326690 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1e7ef579-72f5-41e2-9374-d2dcab169471-combined-ca-bundle\") pod \"1e7ef579-72f5-41e2-9374-d2dcab169471\" (UID: \"1e7ef579-72f5-41e2-9374-d2dcab169471\") " Dec 03 12:53:37 crc kubenswrapper[5052]: I1203 12:53:37.326814 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wpzm4\" (UniqueName: \"kubernetes.io/projected/1e7ef579-72f5-41e2-9374-d2dcab169471-kube-api-access-wpzm4\") pod \"1e7ef579-72f5-41e2-9374-d2dcab169471\" (UID: \"1e7ef579-72f5-41e2-9374-d2dcab169471\") " Dec 03 12:53:37 crc kubenswrapper[5052]: I1203 12:53:37.326881 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/1e7ef579-72f5-41e2-9374-d2dcab169471-nova-metadata-tls-certs\") pod \"1e7ef579-72f5-41e2-9374-d2dcab169471\" (UID: \"1e7ef579-72f5-41e2-9374-d2dcab169471\") " Dec 03 12:53:37 crc kubenswrapper[5052]: I1203 12:53:37.326906 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/1e7ef579-72f5-41e2-9374-d2dcab169471-logs\") pod \"1e7ef579-72f5-41e2-9374-d2dcab169471\" (UID: \"1e7ef579-72f5-41e2-9374-d2dcab169471\") " Dec 03 12:53:37 crc kubenswrapper[5052]: I1203 12:53:37.326952 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/1e7ef579-72f5-41e2-9374-d2dcab169471-config-data\") pod \"1e7ef579-72f5-41e2-9374-d2dcab169471\" (UID: \"1e7ef579-72f5-41e2-9374-d2dcab169471\") " Dec 03 12:53:37 crc kubenswrapper[5052]: I1203 12:53:37.327395 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1e7ef579-72f5-41e2-9374-d2dcab169471-logs" (OuterVolumeSpecName: "logs") pod "1e7ef579-72f5-41e2-9374-d2dcab169471" (UID: "1e7ef579-72f5-41e2-9374-d2dcab169471"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:53:37 crc kubenswrapper[5052]: I1203 12:53:37.331791 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1e7ef579-72f5-41e2-9374-d2dcab169471-kube-api-access-wpzm4" (OuterVolumeSpecName: "kube-api-access-wpzm4") pod "1e7ef579-72f5-41e2-9374-d2dcab169471" (UID: "1e7ef579-72f5-41e2-9374-d2dcab169471"). InnerVolumeSpecName "kube-api-access-wpzm4". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:53:37 crc kubenswrapper[5052]: I1203 12:53:37.356204 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1e7ef579-72f5-41e2-9374-d2dcab169471-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "1e7ef579-72f5-41e2-9374-d2dcab169471" (UID: "1e7ef579-72f5-41e2-9374-d2dcab169471"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:53:37 crc kubenswrapper[5052]: I1203 12:53:37.359919 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1e7ef579-72f5-41e2-9374-d2dcab169471-config-data" (OuterVolumeSpecName: "config-data") pod "1e7ef579-72f5-41e2-9374-d2dcab169471" (UID: "1e7ef579-72f5-41e2-9374-d2dcab169471"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:53:37 crc kubenswrapper[5052]: I1203 12:53:37.375871 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1e7ef579-72f5-41e2-9374-d2dcab169471-nova-metadata-tls-certs" (OuterVolumeSpecName: "nova-metadata-tls-certs") pod "1e7ef579-72f5-41e2-9374-d2dcab169471" (UID: "1e7ef579-72f5-41e2-9374-d2dcab169471"). InnerVolumeSpecName "nova-metadata-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:53:37 crc kubenswrapper[5052]: I1203 12:53:37.428856 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1e7ef579-72f5-41e2-9374-d2dcab169471-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:53:37 crc kubenswrapper[5052]: I1203 12:53:37.428898 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-wpzm4\" (UniqueName: \"kubernetes.io/projected/1e7ef579-72f5-41e2-9374-d2dcab169471-kube-api-access-wpzm4\") on node \"crc\" DevicePath \"\"" Dec 03 12:53:37 crc kubenswrapper[5052]: I1203 12:53:37.428914 5052 reconciler_common.go:293] "Volume detached for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/1e7ef579-72f5-41e2-9374-d2dcab169471-nova-metadata-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:53:37 crc kubenswrapper[5052]: I1203 12:53:37.428927 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/1e7ef579-72f5-41e2-9374-d2dcab169471-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:53:37 crc kubenswrapper[5052]: I1203 12:53:37.428940 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/1e7ef579-72f5-41e2-9374-d2dcab169471-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:53:37 crc kubenswrapper[5052]: I1203 12:53:37.682631 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5930ff4b-2768-4636-af3f-e5395bb93f49" path="/var/lib/kubelet/pods/5930ff4b-2768-4636-af3f-e5395bb93f49/volumes" Dec 03 12:53:37 crc kubenswrapper[5052]: I1203 12:53:37.683194 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c7d71496-0273-4f3d-9f21-3848037800ff" path="/var/lib/kubelet/pods/c7d71496-0273-4f3d-9f21-3848037800ff/volumes" Dec 03 12:53:37 crc kubenswrapper[5052]: I1203 12:53:37.697618 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"cb5f26d2-4434-4f20-a7bc-49d58108d9ce","Type":"ContainerStarted","Data":"41c8290b9e5fa58a7f3d6b59794a41b3e1646cb946f63f4436263b8c5f5de28a"} Dec 03 12:53:37 crc kubenswrapper[5052]: I1203 12:53:37.697665 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"cb5f26d2-4434-4f20-a7bc-49d58108d9ce","Type":"ContainerStarted","Data":"0cb6fd060ff53407ddd9999edc7d4e22f3a7033c40e984ce5173cef5e4eb20f1"} Dec 03 12:53:37 crc kubenswrapper[5052]: I1203 12:53:37.697676 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"cb5f26d2-4434-4f20-a7bc-49d58108d9ce","Type":"ContainerStarted","Data":"d46a4912699adbf3859fb864b472e197b4bd54f2770ed821a5b9be07f3630f75"} Dec 03 12:53:37 crc kubenswrapper[5052]: I1203 12:53:37.701524 5052 generic.go:334] "Generic (PLEG): container finished" podID="1e7ef579-72f5-41e2-9374-d2dcab169471" containerID="19c711c57e77c66d39bf6c4bd8d5e28dab2b329bf459ff2daed1570bc217ec16" exitCode=0 Dec 03 12:53:37 crc kubenswrapper[5052]: I1203 12:53:37.701609 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"1e7ef579-72f5-41e2-9374-d2dcab169471","Type":"ContainerDied","Data":"19c711c57e77c66d39bf6c4bd8d5e28dab2b329bf459ff2daed1570bc217ec16"} Dec 03 12:53:37 crc kubenswrapper[5052]: I1203 12:53:37.701644 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"1e7ef579-72f5-41e2-9374-d2dcab169471","Type":"ContainerDied","Data":"e52e82d33656856b2e2f3d6c3e4423ea16101ee7e57fcb2c0e9ec0af57ba0695"} Dec 03 12:53:37 crc kubenswrapper[5052]: I1203 12:53:37.701668 5052 scope.go:117] "RemoveContainer" containerID="19c711c57e77c66d39bf6c4bd8d5e28dab2b329bf459ff2daed1570bc217ec16" Dec 03 12:53:37 crc kubenswrapper[5052]: I1203 12:53:37.701822 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:53:37 crc kubenswrapper[5052]: I1203 12:53:37.704789 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"4ec49e76-2597-47d6-811b-f65cbe0580ff","Type":"ContainerStarted","Data":"ec5a6e0000f85ac80b044c5e702c36b00720989c7bd27aac0130a6739a2d1bd8"} Dec 03 12:53:37 crc kubenswrapper[5052]: I1203 12:53:37.722911 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-api-0" podStartSLOduration=2.722890752 podStartE2EDuration="2.722890752s" podCreationTimestamp="2025-12-03 12:53:35 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:53:37.716238112 +0000 UTC m=+5608.304479876" watchObservedRunningTime="2025-12-03 12:53:37.722890752 +0000 UTC m=+5608.311132516" Dec 03 12:53:37 crc kubenswrapper[5052]: I1203 12:53:37.727397 5052 scope.go:117] "RemoveContainer" containerID="d6031286fa2a90c842cf1e6e7b88e513c06f9131324674efcd3cb78de18ca18f" Dec 03 12:53:37 crc kubenswrapper[5052]: I1203 12:53:37.747767 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-scheduler-0" podStartSLOduration=2.747748033 podStartE2EDuration="2.747748033s" podCreationTimestamp="2025-12-03 12:53:35 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:53:37.735850463 +0000 UTC m=+5608.324092227" watchObservedRunningTime="2025-12-03 12:53:37.747748033 +0000 UTC m=+5608.335989797" Dec 03 12:53:37 crc kubenswrapper[5052]: I1203 12:53:37.755297 5052 scope.go:117] "RemoveContainer" containerID="19c711c57e77c66d39bf6c4bd8d5e28dab2b329bf459ff2daed1570bc217ec16" Dec 03 12:53:37 crc kubenswrapper[5052]: E1203 12:53:37.755780 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"19c711c57e77c66d39bf6c4bd8d5e28dab2b329bf459ff2daed1570bc217ec16\": container with ID starting with 19c711c57e77c66d39bf6c4bd8d5e28dab2b329bf459ff2daed1570bc217ec16 not found: ID does not exist" containerID="19c711c57e77c66d39bf6c4bd8d5e28dab2b329bf459ff2daed1570bc217ec16" Dec 03 12:53:37 crc kubenswrapper[5052]: I1203 12:53:37.755821 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"19c711c57e77c66d39bf6c4bd8d5e28dab2b329bf459ff2daed1570bc217ec16"} err="failed to get container status \"19c711c57e77c66d39bf6c4bd8d5e28dab2b329bf459ff2daed1570bc217ec16\": rpc error: code = NotFound desc = could not find container \"19c711c57e77c66d39bf6c4bd8d5e28dab2b329bf459ff2daed1570bc217ec16\": container with ID starting with 19c711c57e77c66d39bf6c4bd8d5e28dab2b329bf459ff2daed1570bc217ec16 not found: ID does not exist" Dec 03 12:53:37 crc kubenswrapper[5052]: I1203 12:53:37.755847 5052 scope.go:117] "RemoveContainer" containerID="d6031286fa2a90c842cf1e6e7b88e513c06f9131324674efcd3cb78de18ca18f" Dec 03 12:53:37 crc kubenswrapper[5052]: E1203 12:53:37.756745 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d6031286fa2a90c842cf1e6e7b88e513c06f9131324674efcd3cb78de18ca18f\": container with ID starting with d6031286fa2a90c842cf1e6e7b88e513c06f9131324674efcd3cb78de18ca18f not found: ID does not exist" containerID="d6031286fa2a90c842cf1e6e7b88e513c06f9131324674efcd3cb78de18ca18f" Dec 03 12:53:37 crc kubenswrapper[5052]: I1203 12:53:37.756772 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d6031286fa2a90c842cf1e6e7b88e513c06f9131324674efcd3cb78de18ca18f"} err="failed to get container status \"d6031286fa2a90c842cf1e6e7b88e513c06f9131324674efcd3cb78de18ca18f\": rpc error: code = NotFound desc = could not find container \"d6031286fa2a90c842cf1e6e7b88e513c06f9131324674efcd3cb78de18ca18f\": container with ID starting with d6031286fa2a90c842cf1e6e7b88e513c06f9131324674efcd3cb78de18ca18f not found: ID does not exist" Dec 03 12:53:37 crc kubenswrapper[5052]: I1203 12:53:37.767657 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:53:37 crc kubenswrapper[5052]: I1203 12:53:37.778563 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:53:37 crc kubenswrapper[5052]: I1203 12:53:37.792301 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:53:37 crc kubenswrapper[5052]: E1203 12:53:37.792867 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="1e7ef579-72f5-41e2-9374-d2dcab169471" containerName="nova-metadata-log" Dec 03 12:53:37 crc kubenswrapper[5052]: I1203 12:53:37.792894 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="1e7ef579-72f5-41e2-9374-d2dcab169471" containerName="nova-metadata-log" Dec 03 12:53:37 crc kubenswrapper[5052]: E1203 12:53:37.792946 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="1e7ef579-72f5-41e2-9374-d2dcab169471" containerName="nova-metadata-metadata" Dec 03 12:53:37 crc kubenswrapper[5052]: I1203 12:53:37.792956 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="1e7ef579-72f5-41e2-9374-d2dcab169471" containerName="nova-metadata-metadata" Dec 03 12:53:37 crc kubenswrapper[5052]: I1203 12:53:37.793190 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="1e7ef579-72f5-41e2-9374-d2dcab169471" containerName="nova-metadata-log" Dec 03 12:53:37 crc kubenswrapper[5052]: I1203 12:53:37.793297 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="1e7ef579-72f5-41e2-9374-d2dcab169471" containerName="nova-metadata-metadata" Dec 03 12:53:37 crc kubenswrapper[5052]: I1203 12:53:37.794620 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:53:37 crc kubenswrapper[5052]: I1203 12:53:37.796800 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-nova-metadata-internal-svc" Dec 03 12:53:37 crc kubenswrapper[5052]: I1203 12:53:37.797285 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-metadata-config-data" Dec 03 12:53:37 crc kubenswrapper[5052]: I1203 12:53:37.809999 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:53:37 crc kubenswrapper[5052]: I1203 12:53:37.938567 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xf64v\" (UniqueName: \"kubernetes.io/projected/022746a5-1c20-4729-8688-5903831f57ed-kube-api-access-xf64v\") pod \"nova-metadata-0\" (UID: \"022746a5-1c20-4729-8688-5903831f57ed\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:53:37 crc kubenswrapper[5052]: I1203 12:53:37.938623 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/022746a5-1c20-4729-8688-5903831f57ed-nova-metadata-tls-certs\") pod \"nova-metadata-0\" (UID: \"022746a5-1c20-4729-8688-5903831f57ed\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:53:37 crc kubenswrapper[5052]: I1203 12:53:37.938679 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/022746a5-1c20-4729-8688-5903831f57ed-logs\") pod \"nova-metadata-0\" (UID: \"022746a5-1c20-4729-8688-5903831f57ed\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:53:37 crc kubenswrapper[5052]: I1203 12:53:37.938717 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/022746a5-1c20-4729-8688-5903831f57ed-config-data\") pod \"nova-metadata-0\" (UID: \"022746a5-1c20-4729-8688-5903831f57ed\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:53:37 crc kubenswrapper[5052]: I1203 12:53:37.938737 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/022746a5-1c20-4729-8688-5903831f57ed-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"022746a5-1c20-4729-8688-5903831f57ed\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:53:38 crc kubenswrapper[5052]: I1203 12:53:38.040195 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/022746a5-1c20-4729-8688-5903831f57ed-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"022746a5-1c20-4729-8688-5903831f57ed\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:53:38 crc kubenswrapper[5052]: I1203 12:53:38.040735 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-xf64v\" (UniqueName: \"kubernetes.io/projected/022746a5-1c20-4729-8688-5903831f57ed-kube-api-access-xf64v\") pod \"nova-metadata-0\" (UID: \"022746a5-1c20-4729-8688-5903831f57ed\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:53:38 crc kubenswrapper[5052]: I1203 12:53:38.040813 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/022746a5-1c20-4729-8688-5903831f57ed-nova-metadata-tls-certs\") pod \"nova-metadata-0\" (UID: \"022746a5-1c20-4729-8688-5903831f57ed\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:53:38 crc kubenswrapper[5052]: I1203 12:53:38.040861 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/022746a5-1c20-4729-8688-5903831f57ed-logs\") pod \"nova-metadata-0\" (UID: \"022746a5-1c20-4729-8688-5903831f57ed\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:53:38 crc kubenswrapper[5052]: I1203 12:53:38.040892 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/022746a5-1c20-4729-8688-5903831f57ed-config-data\") pod \"nova-metadata-0\" (UID: \"022746a5-1c20-4729-8688-5903831f57ed\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:53:38 crc kubenswrapper[5052]: I1203 12:53:38.041298 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/022746a5-1c20-4729-8688-5903831f57ed-logs\") pod \"nova-metadata-0\" (UID: \"022746a5-1c20-4729-8688-5903831f57ed\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:53:38 crc kubenswrapper[5052]: I1203 12:53:38.050022 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/022746a5-1c20-4729-8688-5903831f57ed-nova-metadata-tls-certs\") pod \"nova-metadata-0\" (UID: \"022746a5-1c20-4729-8688-5903831f57ed\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:53:38 crc kubenswrapper[5052]: I1203 12:53:38.050134 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/022746a5-1c20-4729-8688-5903831f57ed-combined-ca-bundle\") pod \"nova-metadata-0\" (UID: \"022746a5-1c20-4729-8688-5903831f57ed\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:53:38 crc kubenswrapper[5052]: I1203 12:53:38.050220 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/022746a5-1c20-4729-8688-5903831f57ed-config-data\") pod \"nova-metadata-0\" (UID: \"022746a5-1c20-4729-8688-5903831f57ed\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:53:38 crc kubenswrapper[5052]: I1203 12:53:38.068991 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-xf64v\" (UniqueName: \"kubernetes.io/projected/022746a5-1c20-4729-8688-5903831f57ed-kube-api-access-xf64v\") pod \"nova-metadata-0\" (UID: \"022746a5-1c20-4729-8688-5903831f57ed\") " pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:53:38 crc kubenswrapper[5052]: I1203 12:53:38.132448 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:53:38 crc kubenswrapper[5052]: I1203 12:53:38.604344 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:53:38 crc kubenswrapper[5052]: W1203 12:53:38.610040 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod022746a5_1c20_4729_8688_5903831f57ed.slice/crio-ebb56e141b46e0dc6c812313228476a17019089f5138ec3df4797aac16a09903 WatchSource:0}: Error finding container ebb56e141b46e0dc6c812313228476a17019089f5138ec3df4797aac16a09903: Status 404 returned error can't find the container with id ebb56e141b46e0dc6c812313228476a17019089f5138ec3df4797aac16a09903 Dec 03 12:53:38 crc kubenswrapper[5052]: I1203 12:53:38.715278 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"022746a5-1c20-4729-8688-5903831f57ed","Type":"ContainerStarted","Data":"ebb56e141b46e0dc6c812313228476a17019089f5138ec3df4797aac16a09903"} Dec 03 12:53:39 crc kubenswrapper[5052]: I1203 12:53:39.697766 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1e7ef579-72f5-41e2-9374-d2dcab169471" path="/var/lib/kubelet/pods/1e7ef579-72f5-41e2-9374-d2dcab169471/volumes" Dec 03 12:53:39 crc kubenswrapper[5052]: I1203 12:53:39.726327 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"022746a5-1c20-4729-8688-5903831f57ed","Type":"ContainerStarted","Data":"e304d74be1d9347456c21077ccfef718ab1e897114fc49c1d7da35457693360a"} Dec 03 12:53:39 crc kubenswrapper[5052]: I1203 12:53:39.726418 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"022746a5-1c20-4729-8688-5903831f57ed","Type":"ContainerStarted","Data":"3157794aa44d4b21b9c104515fcfd5ee6e33807ee7b2753cff2132e2facd1f75"} Dec 03 12:53:39 crc kubenswrapper[5052]: I1203 12:53:39.757344 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-metadata-0" podStartSLOduration=2.757321393 podStartE2EDuration="2.757321393s" podCreationTimestamp="2025-12-03 12:53:37 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:53:39.746758361 +0000 UTC m=+5610.335000125" watchObservedRunningTime="2025-12-03 12:53:39.757321393 +0000 UTC m=+5610.345563167" Dec 03 12:53:41 crc kubenswrapper[5052]: I1203 12:53:41.169249 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:53:43 crc kubenswrapper[5052]: I1203 12:53:43.132569 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:53:43 crc kubenswrapper[5052]: I1203 12:53:43.132865 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:53:46 crc kubenswrapper[5052]: I1203 12:53:46.148033 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:53:46 crc kubenswrapper[5052]: I1203 12:53:46.148347 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:53:46 crc kubenswrapper[5052]: I1203 12:53:46.169416 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:53:46 crc kubenswrapper[5052]: I1203 12:53:46.199147 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:53:46 crc kubenswrapper[5052]: I1203 12:53:46.818546 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:53:47 crc kubenswrapper[5052]: E1203 12:53:47.155214 5052 cadvisor_stats_provider.go:516] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb0bb2e93_76f1_4485_b822_41e83b2a4f59.slice/crio-29b21fda6544a2343736d96bac4c7f7476a926e688d5808f8b0fa0235fbd671a\": RecentStats: unable to find data in memory cache]" Dec 03 12:53:47 crc kubenswrapper[5052]: I1203 12:53:47.230844 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-api-0" podUID="cb5f26d2-4434-4f20-a7bc-49d58108d9ce" containerName="nova-api-log" probeResult="failure" output="Get \"http://10.217.0.218:8774/\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Dec 03 12:53:47 crc kubenswrapper[5052]: I1203 12:53:47.230849 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-api-0" podUID="cb5f26d2-4434-4f20-a7bc-49d58108d9ce" containerName="nova-api-api" probeResult="failure" output="Get \"http://10.217.0.218:8774/\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Dec 03 12:53:48 crc kubenswrapper[5052]: I1203 12:53:48.142843 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:53:48 crc kubenswrapper[5052]: I1203 12:53:48.144561 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:53:49 crc kubenswrapper[5052]: I1203 12:53:49.193725 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-metadata-0" podUID="022746a5-1c20-4729-8688-5903831f57ed" containerName="nova-metadata-metadata" probeResult="failure" output="Get \"https://10.217.0.220:8775/\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Dec 03 12:53:49 crc kubenswrapper[5052]: I1203 12:53:49.193721 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-metadata-0" podUID="022746a5-1c20-4729-8688-5903831f57ed" containerName="nova-metadata-log" probeResult="failure" output="Get \"https://10.217.0.220:8775/\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Dec 03 12:53:53 crc kubenswrapper[5052]: I1203 12:53:53.902018 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openshift-marketplace/community-operators-42jr4"] Dec 03 12:53:53 crc kubenswrapper[5052]: I1203 12:53:53.906253 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/community-operators-42jr4" Dec 03 12:53:53 crc kubenswrapper[5052]: I1203 12:53:53.917527 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/community-operators-42jr4"] Dec 03 12:53:53 crc kubenswrapper[5052]: I1203 12:53:53.966121 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/b6eacded-e668-400c-ba59-4de7e7f8bf78-utilities\") pod \"community-operators-42jr4\" (UID: \"b6eacded-e668-400c-ba59-4de7e7f8bf78\") " pod="openshift-marketplace/community-operators-42jr4" Dec 03 12:53:53 crc kubenswrapper[5052]: I1203 12:53:53.966711 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n28p4\" (UniqueName: \"kubernetes.io/projected/b6eacded-e668-400c-ba59-4de7e7f8bf78-kube-api-access-n28p4\") pod \"community-operators-42jr4\" (UID: \"b6eacded-e668-400c-ba59-4de7e7f8bf78\") " pod="openshift-marketplace/community-operators-42jr4" Dec 03 12:53:53 crc kubenswrapper[5052]: I1203 12:53:53.966798 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/b6eacded-e668-400c-ba59-4de7e7f8bf78-catalog-content\") pod \"community-operators-42jr4\" (UID: \"b6eacded-e668-400c-ba59-4de7e7f8bf78\") " pod="openshift-marketplace/community-operators-42jr4" Dec 03 12:53:54 crc kubenswrapper[5052]: I1203 12:53:54.068861 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-n28p4\" (UniqueName: \"kubernetes.io/projected/b6eacded-e668-400c-ba59-4de7e7f8bf78-kube-api-access-n28p4\") pod \"community-operators-42jr4\" (UID: \"b6eacded-e668-400c-ba59-4de7e7f8bf78\") " pod="openshift-marketplace/community-operators-42jr4" Dec 03 12:53:54 crc kubenswrapper[5052]: I1203 12:53:54.068908 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/b6eacded-e668-400c-ba59-4de7e7f8bf78-catalog-content\") pod \"community-operators-42jr4\" (UID: \"b6eacded-e668-400c-ba59-4de7e7f8bf78\") " pod="openshift-marketplace/community-operators-42jr4" Dec 03 12:53:54 crc kubenswrapper[5052]: I1203 12:53:54.068968 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/b6eacded-e668-400c-ba59-4de7e7f8bf78-utilities\") pod \"community-operators-42jr4\" (UID: \"b6eacded-e668-400c-ba59-4de7e7f8bf78\") " pod="openshift-marketplace/community-operators-42jr4" Dec 03 12:53:54 crc kubenswrapper[5052]: I1203 12:53:54.069614 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/b6eacded-e668-400c-ba59-4de7e7f8bf78-utilities\") pod \"community-operators-42jr4\" (UID: \"b6eacded-e668-400c-ba59-4de7e7f8bf78\") " pod="openshift-marketplace/community-operators-42jr4" Dec 03 12:53:54 crc kubenswrapper[5052]: I1203 12:53:54.069741 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/b6eacded-e668-400c-ba59-4de7e7f8bf78-catalog-content\") pod \"community-operators-42jr4\" (UID: \"b6eacded-e668-400c-ba59-4de7e7f8bf78\") " pod="openshift-marketplace/community-operators-42jr4" Dec 03 12:53:54 crc kubenswrapper[5052]: I1203 12:53:54.095608 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-n28p4\" (UniqueName: \"kubernetes.io/projected/b6eacded-e668-400c-ba59-4de7e7f8bf78-kube-api-access-n28p4\") pod \"community-operators-42jr4\" (UID: \"b6eacded-e668-400c-ba59-4de7e7f8bf78\") " pod="openshift-marketplace/community-operators-42jr4" Dec 03 12:53:54 crc kubenswrapper[5052]: I1203 12:53:54.228562 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/community-operators-42jr4" Dec 03 12:53:54 crc kubenswrapper[5052]: W1203 12:53:54.835782 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6eacded_e668_400c_ba59_4de7e7f8bf78.slice/crio-d7c505d4c4561acb5e8829f2ed564173b44dffb5b6223e37e95fc69a47785021 WatchSource:0}: Error finding container d7c505d4c4561acb5e8829f2ed564173b44dffb5b6223e37e95fc69a47785021: Status 404 returned error can't find the container with id d7c505d4c4561acb5e8829f2ed564173b44dffb5b6223e37e95fc69a47785021 Dec 03 12:53:54 crc kubenswrapper[5052]: I1203 12:53:54.838045 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/community-operators-42jr4"] Dec 03 12:53:54 crc kubenswrapper[5052]: I1203 12:53:54.875377 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/community-operators-42jr4" event={"ID":"b6eacded-e668-400c-ba59-4de7e7f8bf78","Type":"ContainerStarted","Data":"d7c505d4c4561acb5e8829f2ed564173b44dffb5b6223e37e95fc69a47785021"} Dec 03 12:53:55 crc kubenswrapper[5052]: I1203 12:53:55.888732 5052 generic.go:334] "Generic (PLEG): container finished" podID="b6eacded-e668-400c-ba59-4de7e7f8bf78" containerID="771f29a89558d77fd33244f67e65bfe1da83beb6967c685b37a418114b7cce80" exitCode=0 Dec 03 12:53:55 crc kubenswrapper[5052]: I1203 12:53:55.888788 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/community-operators-42jr4" event={"ID":"b6eacded-e668-400c-ba59-4de7e7f8bf78","Type":"ContainerDied","Data":"771f29a89558d77fd33244f67e65bfe1da83beb6967c685b37a418114b7cce80"} Dec 03 12:53:56 crc kubenswrapper[5052]: I1203 12:53:56.151643 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:53:56 crc kubenswrapper[5052]: I1203 12:53:56.151727 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:53:56 crc kubenswrapper[5052]: I1203 12:53:56.152429 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:53:56 crc kubenswrapper[5052]: I1203 12:53:56.152456 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:53:56 crc kubenswrapper[5052]: I1203 12:53:56.155772 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:53:56 crc kubenswrapper[5052]: I1203 12:53:56.158548 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:53:57 crc kubenswrapper[5052]: E1203 12:53:57.540542 5052 cadvisor_stats_provider.go:516] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb0bb2e93_76f1_4485_b822_41e83b2a4f59.slice/crio-29b21fda6544a2343736d96bac4c7f7476a926e688d5808f8b0fa0235fbd671a\": RecentStats: unable to find data in memory cache]" Dec 03 12:53:57 crc kubenswrapper[5052]: I1203 12:53:57.925118 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/community-operators-42jr4" event={"ID":"b6eacded-e668-400c-ba59-4de7e7f8bf78","Type":"ContainerStarted","Data":"24d9b7547faf46f13ce8052fefe424803375ae067d57181d41efb5c19b519092"} Dec 03 12:53:58 crc kubenswrapper[5052]: I1203 12:53:58.142307 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:53:58 crc kubenswrapper[5052]: I1203 12:53:58.147040 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:53:58 crc kubenswrapper[5052]: I1203 12:53:58.160210 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:53:58 crc kubenswrapper[5052]: I1203 12:53:58.622247 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:53:58 crc kubenswrapper[5052]: I1203 12:53:58.622922 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="f99b50df-1d5b-4535-bc09-86078a1318bd" containerName="ceilometer-central-agent" containerID="cri-o://8752f37ec5e86db977fed1536c7c034e712d2037bc9e10abf57acd410669f7b7" gracePeriod=30 Dec 03 12:53:58 crc kubenswrapper[5052]: I1203 12:53:58.622982 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="f99b50df-1d5b-4535-bc09-86078a1318bd" containerName="proxy-httpd" containerID="cri-o://f367321ea2b1658db4fe5944e4616d0065be3a170ac7d1db301f3314e4cbe754" gracePeriod=30 Dec 03 12:53:58 crc kubenswrapper[5052]: I1203 12:53:58.623023 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="f99b50df-1d5b-4535-bc09-86078a1318bd" containerName="sg-core" containerID="cri-o://8ea80ef5cbeb1e7ba8f177b0594824485b3da4da0c66afa4536804902bdf70e1" gracePeriod=30 Dec 03 12:53:58 crc kubenswrapper[5052]: I1203 12:53:58.623036 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="f99b50df-1d5b-4535-bc09-86078a1318bd" containerName="ceilometer-notification-agent" containerID="cri-o://c3e875e412556baa33eaca07af1a9b1c17c206e2f38da5c9a1a7982dbc3ff9da" gracePeriod=30 Dec 03 12:53:58 crc kubenswrapper[5052]: I1203 12:53:58.944328 5052 generic.go:334] "Generic (PLEG): container finished" podID="f99b50df-1d5b-4535-bc09-86078a1318bd" containerID="f367321ea2b1658db4fe5944e4616d0065be3a170ac7d1db301f3314e4cbe754" exitCode=0 Dec 03 12:53:58 crc kubenswrapper[5052]: I1203 12:53:58.944356 5052 generic.go:334] "Generic (PLEG): container finished" podID="f99b50df-1d5b-4535-bc09-86078a1318bd" containerID="8ea80ef5cbeb1e7ba8f177b0594824485b3da4da0c66afa4536804902bdf70e1" exitCode=2 Dec 03 12:53:58 crc kubenswrapper[5052]: I1203 12:53:58.944390 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"f99b50df-1d5b-4535-bc09-86078a1318bd","Type":"ContainerDied","Data":"f367321ea2b1658db4fe5944e4616d0065be3a170ac7d1db301f3314e4cbe754"} Dec 03 12:53:58 crc kubenswrapper[5052]: I1203 12:53:58.944415 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"f99b50df-1d5b-4535-bc09-86078a1318bd","Type":"ContainerDied","Data":"8ea80ef5cbeb1e7ba8f177b0594824485b3da4da0c66afa4536804902bdf70e1"} Dec 03 12:53:58 crc kubenswrapper[5052]: I1203 12:53:58.948813 5052 generic.go:334] "Generic (PLEG): container finished" podID="b6eacded-e668-400c-ba59-4de7e7f8bf78" containerID="24d9b7547faf46f13ce8052fefe424803375ae067d57181d41efb5c19b519092" exitCode=0 Dec 03 12:53:58 crc kubenswrapper[5052]: I1203 12:53:58.948909 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/community-operators-42jr4" event={"ID":"b6eacded-e668-400c-ba59-4de7e7f8bf78","Type":"ContainerDied","Data":"24d9b7547faf46f13ce8052fefe424803375ae067d57181d41efb5c19b519092"} Dec 03 12:53:58 crc kubenswrapper[5052]: I1203 12:53:58.958836 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:53:59 crc kubenswrapper[5052]: I1203 12:53:59.176556 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:53:59 crc kubenswrapper[5052]: I1203 12:53:59.176823 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-api-0" podUID="cb5f26d2-4434-4f20-a7bc-49d58108d9ce" containerName="nova-api-log" containerID="cri-o://0cb6fd060ff53407ddd9999edc7d4e22f3a7033c40e984ce5173cef5e4eb20f1" gracePeriod=30 Dec 03 12:53:59 crc kubenswrapper[5052]: I1203 12:53:59.176968 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-api-0" podUID="cb5f26d2-4434-4f20-a7bc-49d58108d9ce" containerName="nova-api-api" containerID="cri-o://41c8290b9e5fa58a7f3d6b59794a41b3e1646cb946f63f4436263b8c5f5de28a" gracePeriod=30 Dec 03 12:53:59 crc kubenswrapper[5052]: I1203 12:53:59.379710 5052 patch_prober.go:28] interesting pod/machine-config-daemon-74bj6 container/machine-config-daemon namespace/openshift-machine-config-operator: Liveness probe status=failure output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" start-of-body= Dec 03 12:53:59 crc kubenswrapper[5052]: I1203 12:53:59.380100 5052 prober.go:107] "Probe failed" probeType="Liveness" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" containerName="machine-config-daemon" probeResult="failure" output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" Dec 03 12:53:59 crc kubenswrapper[5052]: I1203 12:53:59.960283 5052 generic.go:334] "Generic (PLEG): container finished" podID="f99b50df-1d5b-4535-bc09-86078a1318bd" containerID="8752f37ec5e86db977fed1536c7c034e712d2037bc9e10abf57acd410669f7b7" exitCode=0 Dec 03 12:53:59 crc kubenswrapper[5052]: I1203 12:53:59.960372 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"f99b50df-1d5b-4535-bc09-86078a1318bd","Type":"ContainerDied","Data":"8752f37ec5e86db977fed1536c7c034e712d2037bc9e10abf57acd410669f7b7"} Dec 03 12:53:59 crc kubenswrapper[5052]: I1203 12:53:59.962349 5052 generic.go:334] "Generic (PLEG): container finished" podID="cb5f26d2-4434-4f20-a7bc-49d58108d9ce" containerID="0cb6fd060ff53407ddd9999edc7d4e22f3a7033c40e984ce5173cef5e4eb20f1" exitCode=143 Dec 03 12:53:59 crc kubenswrapper[5052]: I1203 12:53:59.963270 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"cb5f26d2-4434-4f20-a7bc-49d58108d9ce","Type":"ContainerDied","Data":"0cb6fd060ff53407ddd9999edc7d4e22f3a7033c40e984ce5173cef5e4eb20f1"} Dec 03 12:54:01 crc kubenswrapper[5052]: I1203 12:54:01.991161 5052 generic.go:334] "Generic (PLEG): container finished" podID="f99b50df-1d5b-4535-bc09-86078a1318bd" containerID="c3e875e412556baa33eaca07af1a9b1c17c206e2f38da5c9a1a7982dbc3ff9da" exitCode=0 Dec 03 12:54:01 crc kubenswrapper[5052]: I1203 12:54:01.991235 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"f99b50df-1d5b-4535-bc09-86078a1318bd","Type":"ContainerDied","Data":"c3e875e412556baa33eaca07af1a9b1c17c206e2f38da5c9a1a7982dbc3ff9da"} Dec 03 12:54:01 crc kubenswrapper[5052]: I1203 12:54:01.991834 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"f99b50df-1d5b-4535-bc09-86078a1318bd","Type":"ContainerDied","Data":"8a1c4ae133793732d4f09ef113339e8f652716264b5560426daf38d948a4d695"} Dec 03 12:54:01 crc kubenswrapper[5052]: I1203 12:54:01.991851 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8a1c4ae133793732d4f09ef113339e8f652716264b5560426daf38d948a4d695" Dec 03 12:54:01 crc kubenswrapper[5052]: I1203 12:54:01.994841 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/community-operators-42jr4" event={"ID":"b6eacded-e668-400c-ba59-4de7e7f8bf78","Type":"ContainerStarted","Data":"3342bd0b32d00537ebb8c5f6c934d9afc76399f580767282f48c1be38c55e5c1"} Dec 03 12:54:02 crc kubenswrapper[5052]: I1203 12:54:02.011086 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:54:02 crc kubenswrapper[5052]: I1203 12:54:02.026889 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-marketplace/community-operators-42jr4" podStartSLOduration=4.354058678 podStartE2EDuration="9.026870072s" podCreationTimestamp="2025-12-03 12:53:53 +0000 UTC" firstStartedPulling="2025-12-03 12:53:55.892676685 +0000 UTC m=+5626.480918449" lastFinishedPulling="2025-12-03 12:54:00.565488079 +0000 UTC m=+5631.153729843" observedRunningTime="2025-12-03 12:54:02.017494334 +0000 UTC m=+5632.605736088" watchObservedRunningTime="2025-12-03 12:54:02.026870072 +0000 UTC m=+5632.615111836" Dec 03 12:54:02 crc kubenswrapper[5052]: I1203 12:54:02.122377 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/f99b50df-1d5b-4535-bc09-86078a1318bd-config-data\") pod \"f99b50df-1d5b-4535-bc09-86078a1318bd\" (UID: \"f99b50df-1d5b-4535-bc09-86078a1318bd\") " Dec 03 12:54:02 crc kubenswrapper[5052]: I1203 12:54:02.122434 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/f99b50df-1d5b-4535-bc09-86078a1318bd-log-httpd\") pod \"f99b50df-1d5b-4535-bc09-86078a1318bd\" (UID: \"f99b50df-1d5b-4535-bc09-86078a1318bd\") " Dec 03 12:54:02 crc kubenswrapper[5052]: I1203 12:54:02.122451 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/f99b50df-1d5b-4535-bc09-86078a1318bd-sg-core-conf-yaml\") pod \"f99b50df-1d5b-4535-bc09-86078a1318bd\" (UID: \"f99b50df-1d5b-4535-bc09-86078a1318bd\") " Dec 03 12:54:02 crc kubenswrapper[5052]: I1203 12:54:02.122479 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8v4sn\" (UniqueName: \"kubernetes.io/projected/f99b50df-1d5b-4535-bc09-86078a1318bd-kube-api-access-8v4sn\") pod \"f99b50df-1d5b-4535-bc09-86078a1318bd\" (UID: \"f99b50df-1d5b-4535-bc09-86078a1318bd\") " Dec 03 12:54:02 crc kubenswrapper[5052]: I1203 12:54:02.122594 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/f99b50df-1d5b-4535-bc09-86078a1318bd-run-httpd\") pod \"f99b50df-1d5b-4535-bc09-86078a1318bd\" (UID: \"f99b50df-1d5b-4535-bc09-86078a1318bd\") " Dec 03 12:54:02 crc kubenswrapper[5052]: I1203 12:54:02.122638 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/f99b50df-1d5b-4535-bc09-86078a1318bd-ceilometer-tls-certs\") pod \"f99b50df-1d5b-4535-bc09-86078a1318bd\" (UID: \"f99b50df-1d5b-4535-bc09-86078a1318bd\") " Dec 03 12:54:02 crc kubenswrapper[5052]: I1203 12:54:02.122692 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/f99b50df-1d5b-4535-bc09-86078a1318bd-scripts\") pod \"f99b50df-1d5b-4535-bc09-86078a1318bd\" (UID: \"f99b50df-1d5b-4535-bc09-86078a1318bd\") " Dec 03 12:54:02 crc kubenswrapper[5052]: I1203 12:54:02.122722 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f99b50df-1d5b-4535-bc09-86078a1318bd-combined-ca-bundle\") pod \"f99b50df-1d5b-4535-bc09-86078a1318bd\" (UID: \"f99b50df-1d5b-4535-bc09-86078a1318bd\") " Dec 03 12:54:02 crc kubenswrapper[5052]: I1203 12:54:02.122906 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f99b50df-1d5b-4535-bc09-86078a1318bd-log-httpd" (OuterVolumeSpecName: "log-httpd") pod "f99b50df-1d5b-4535-bc09-86078a1318bd" (UID: "f99b50df-1d5b-4535-bc09-86078a1318bd"). InnerVolumeSpecName "log-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:54:02 crc kubenswrapper[5052]: I1203 12:54:02.123503 5052 reconciler_common.go:293] "Volume detached for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/f99b50df-1d5b-4535-bc09-86078a1318bd-log-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:02 crc kubenswrapper[5052]: I1203 12:54:02.123492 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f99b50df-1d5b-4535-bc09-86078a1318bd-run-httpd" (OuterVolumeSpecName: "run-httpd") pod "f99b50df-1d5b-4535-bc09-86078a1318bd" (UID: "f99b50df-1d5b-4535-bc09-86078a1318bd"). InnerVolumeSpecName "run-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:54:02 crc kubenswrapper[5052]: I1203 12:54:02.135103 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f99b50df-1d5b-4535-bc09-86078a1318bd-kube-api-access-8v4sn" (OuterVolumeSpecName: "kube-api-access-8v4sn") pod "f99b50df-1d5b-4535-bc09-86078a1318bd" (UID: "f99b50df-1d5b-4535-bc09-86078a1318bd"). InnerVolumeSpecName "kube-api-access-8v4sn". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:54:02 crc kubenswrapper[5052]: I1203 12:54:02.135140 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f99b50df-1d5b-4535-bc09-86078a1318bd-scripts" (OuterVolumeSpecName: "scripts") pod "f99b50df-1d5b-4535-bc09-86078a1318bd" (UID: "f99b50df-1d5b-4535-bc09-86078a1318bd"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:54:02 crc kubenswrapper[5052]: I1203 12:54:02.167984 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f99b50df-1d5b-4535-bc09-86078a1318bd-sg-core-conf-yaml" (OuterVolumeSpecName: "sg-core-conf-yaml") pod "f99b50df-1d5b-4535-bc09-86078a1318bd" (UID: "f99b50df-1d5b-4535-bc09-86078a1318bd"). InnerVolumeSpecName "sg-core-conf-yaml". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:54:02 crc kubenswrapper[5052]: I1203 12:54:02.212670 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f99b50df-1d5b-4535-bc09-86078a1318bd-ceilometer-tls-certs" (OuterVolumeSpecName: "ceilometer-tls-certs") pod "f99b50df-1d5b-4535-bc09-86078a1318bd" (UID: "f99b50df-1d5b-4535-bc09-86078a1318bd"). InnerVolumeSpecName "ceilometer-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:54:02 crc kubenswrapper[5052]: I1203 12:54:02.235627 5052 reconciler_common.go:293] "Volume detached for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/f99b50df-1d5b-4535-bc09-86078a1318bd-ceilometer-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:02 crc kubenswrapper[5052]: I1203 12:54:02.235667 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/f99b50df-1d5b-4535-bc09-86078a1318bd-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:02 crc kubenswrapper[5052]: I1203 12:54:02.235680 5052 reconciler_common.go:293] "Volume detached for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/f99b50df-1d5b-4535-bc09-86078a1318bd-sg-core-conf-yaml\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:02 crc kubenswrapper[5052]: I1203 12:54:02.235694 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-8v4sn\" (UniqueName: \"kubernetes.io/projected/f99b50df-1d5b-4535-bc09-86078a1318bd-kube-api-access-8v4sn\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:02 crc kubenswrapper[5052]: I1203 12:54:02.235702 5052 reconciler_common.go:293] "Volume detached for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/f99b50df-1d5b-4535-bc09-86078a1318bd-run-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:02 crc kubenswrapper[5052]: I1203 12:54:02.240833 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f99b50df-1d5b-4535-bc09-86078a1318bd-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "f99b50df-1d5b-4535-bc09-86078a1318bd" (UID: "f99b50df-1d5b-4535-bc09-86078a1318bd"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:54:02 crc kubenswrapper[5052]: I1203 12:54:02.253396 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f99b50df-1d5b-4535-bc09-86078a1318bd-config-data" (OuterVolumeSpecName: "config-data") pod "f99b50df-1d5b-4535-bc09-86078a1318bd" (UID: "f99b50df-1d5b-4535-bc09-86078a1318bd"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:54:02 crc kubenswrapper[5052]: I1203 12:54:02.337185 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/f99b50df-1d5b-4535-bc09-86078a1318bd-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:02 crc kubenswrapper[5052]: I1203 12:54:02.337228 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f99b50df-1d5b-4535-bc09-86078a1318bd-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:03 crc kubenswrapper[5052]: I1203 12:54:03.018805 5052 generic.go:334] "Generic (PLEG): container finished" podID="cb5f26d2-4434-4f20-a7bc-49d58108d9ce" containerID="41c8290b9e5fa58a7f3d6b59794a41b3e1646cb946f63f4436263b8c5f5de28a" exitCode=0 Dec 03 12:54:03 crc kubenswrapper[5052]: I1203 12:54:03.019322 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:54:03 crc kubenswrapper[5052]: I1203 12:54:03.018870 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"cb5f26d2-4434-4f20-a7bc-49d58108d9ce","Type":"ContainerDied","Data":"41c8290b9e5fa58a7f3d6b59794a41b3e1646cb946f63f4436263b8c5f5de28a"} Dec 03 12:54:03 crc kubenswrapper[5052]: I1203 12:54:03.063163 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:54:03 crc kubenswrapper[5052]: I1203 12:54:03.070703 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:54:03 crc kubenswrapper[5052]: I1203 12:54:03.089537 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:54:03 crc kubenswrapper[5052]: E1203 12:54:03.090053 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="f99b50df-1d5b-4535-bc09-86078a1318bd" containerName="sg-core" Dec 03 12:54:03 crc kubenswrapper[5052]: I1203 12:54:03.090075 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="f99b50df-1d5b-4535-bc09-86078a1318bd" containerName="sg-core" Dec 03 12:54:03 crc kubenswrapper[5052]: E1203 12:54:03.090108 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="f99b50df-1d5b-4535-bc09-86078a1318bd" containerName="ceilometer-notification-agent" Dec 03 12:54:03 crc kubenswrapper[5052]: I1203 12:54:03.090116 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="f99b50df-1d5b-4535-bc09-86078a1318bd" containerName="ceilometer-notification-agent" Dec 03 12:54:03 crc kubenswrapper[5052]: E1203 12:54:03.090138 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="f99b50df-1d5b-4535-bc09-86078a1318bd" containerName="proxy-httpd" Dec 03 12:54:03 crc kubenswrapper[5052]: I1203 12:54:03.090146 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="f99b50df-1d5b-4535-bc09-86078a1318bd" containerName="proxy-httpd" Dec 03 12:54:03 crc kubenswrapper[5052]: E1203 12:54:03.090159 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="f99b50df-1d5b-4535-bc09-86078a1318bd" containerName="ceilometer-central-agent" Dec 03 12:54:03 crc kubenswrapper[5052]: I1203 12:54:03.090165 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="f99b50df-1d5b-4535-bc09-86078a1318bd" containerName="ceilometer-central-agent" Dec 03 12:54:03 crc kubenswrapper[5052]: I1203 12:54:03.090389 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="f99b50df-1d5b-4535-bc09-86078a1318bd" containerName="ceilometer-notification-agent" Dec 03 12:54:03 crc kubenswrapper[5052]: I1203 12:54:03.090414 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="f99b50df-1d5b-4535-bc09-86078a1318bd" containerName="ceilometer-central-agent" Dec 03 12:54:03 crc kubenswrapper[5052]: I1203 12:54:03.090423 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="f99b50df-1d5b-4535-bc09-86078a1318bd" containerName="proxy-httpd" Dec 03 12:54:03 crc kubenswrapper[5052]: I1203 12:54:03.090434 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="f99b50df-1d5b-4535-bc09-86078a1318bd" containerName="sg-core" Dec 03 12:54:03 crc kubenswrapper[5052]: I1203 12:54:03.104041 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:54:03 crc kubenswrapper[5052]: I1203 12:54:03.104160 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:54:03 crc kubenswrapper[5052]: I1203 12:54:03.107777 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-ceilometer-internal-svc" Dec 03 12:54:03 crc kubenswrapper[5052]: I1203 12:54:03.107858 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-scripts" Dec 03 12:54:03 crc kubenswrapper[5052]: I1203 12:54:03.107784 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-config-data" Dec 03 12:54:03 crc kubenswrapper[5052]: I1203 12:54:03.152442 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/7c1938cc-b16b-4acd-b39c-3af04dda7436-log-httpd\") pod \"ceilometer-0\" (UID: \"7c1938cc-b16b-4acd-b39c-3af04dda7436\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:54:03 crc kubenswrapper[5052]: I1203 12:54:03.152652 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/7c1938cc-b16b-4acd-b39c-3af04dda7436-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"7c1938cc-b16b-4acd-b39c-3af04dda7436\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:54:03 crc kubenswrapper[5052]: I1203 12:54:03.152792 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/7c1938cc-b16b-4acd-b39c-3af04dda7436-config-data\") pod \"ceilometer-0\" (UID: \"7c1938cc-b16b-4acd-b39c-3af04dda7436\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:54:03 crc kubenswrapper[5052]: I1203 12:54:03.152846 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/7c1938cc-b16b-4acd-b39c-3af04dda7436-scripts\") pod \"ceilometer-0\" (UID: \"7c1938cc-b16b-4acd-b39c-3af04dda7436\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:54:03 crc kubenswrapper[5052]: I1203 12:54:03.152894 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/7c1938cc-b16b-4acd-b39c-3af04dda7436-run-httpd\") pod \"ceilometer-0\" (UID: \"7c1938cc-b16b-4acd-b39c-3af04dda7436\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:54:03 crc kubenswrapper[5052]: I1203 12:54:03.153127 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/7c1938cc-b16b-4acd-b39c-3af04dda7436-ceilometer-tls-certs\") pod \"ceilometer-0\" (UID: \"7c1938cc-b16b-4acd-b39c-3af04dda7436\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:54:03 crc kubenswrapper[5052]: I1203 12:54:03.153198 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/7c1938cc-b16b-4acd-b39c-3af04dda7436-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"7c1938cc-b16b-4acd-b39c-3af04dda7436\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:54:03 crc kubenswrapper[5052]: I1203 12:54:03.153296 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kwghq\" (UniqueName: \"kubernetes.io/projected/7c1938cc-b16b-4acd-b39c-3af04dda7436-kube-api-access-kwghq\") pod \"ceilometer-0\" (UID: \"7c1938cc-b16b-4acd-b39c-3af04dda7436\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:54:03 crc kubenswrapper[5052]: I1203 12:54:03.245760 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:54:03 crc kubenswrapper[5052]: E1203 12:54:03.246485 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[ceilometer-tls-certs combined-ca-bundle config-data kube-api-access-kwghq log-httpd run-httpd scripts sg-core-conf-yaml], unattached volumes=[], failed to process volumes=[]: context canceled" pod="openstack-kuttl-tests/ceilometer-0" podUID="7c1938cc-b16b-4acd-b39c-3af04dda7436" Dec 03 12:54:03 crc kubenswrapper[5052]: I1203 12:54:03.256619 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/7c1938cc-b16b-4acd-b39c-3af04dda7436-scripts\") pod \"ceilometer-0\" (UID: \"7c1938cc-b16b-4acd-b39c-3af04dda7436\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:54:03 crc kubenswrapper[5052]: I1203 12:54:03.256757 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/7c1938cc-b16b-4acd-b39c-3af04dda7436-run-httpd\") pod \"ceilometer-0\" (UID: \"7c1938cc-b16b-4acd-b39c-3af04dda7436\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:54:03 crc kubenswrapper[5052]: I1203 12:54:03.256832 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/7c1938cc-b16b-4acd-b39c-3af04dda7436-ceilometer-tls-certs\") pod \"ceilometer-0\" (UID: \"7c1938cc-b16b-4acd-b39c-3af04dda7436\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:54:03 crc kubenswrapper[5052]: I1203 12:54:03.256870 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/7c1938cc-b16b-4acd-b39c-3af04dda7436-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"7c1938cc-b16b-4acd-b39c-3af04dda7436\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:54:03 crc kubenswrapper[5052]: I1203 12:54:03.256913 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-kwghq\" (UniqueName: \"kubernetes.io/projected/7c1938cc-b16b-4acd-b39c-3af04dda7436-kube-api-access-kwghq\") pod \"ceilometer-0\" (UID: \"7c1938cc-b16b-4acd-b39c-3af04dda7436\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:54:03 crc kubenswrapper[5052]: I1203 12:54:03.256937 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/7c1938cc-b16b-4acd-b39c-3af04dda7436-log-httpd\") pod \"ceilometer-0\" (UID: \"7c1938cc-b16b-4acd-b39c-3af04dda7436\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:54:03 crc kubenswrapper[5052]: I1203 12:54:03.256958 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/7c1938cc-b16b-4acd-b39c-3af04dda7436-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"7c1938cc-b16b-4acd-b39c-3af04dda7436\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:54:03 crc kubenswrapper[5052]: I1203 12:54:03.257008 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/7c1938cc-b16b-4acd-b39c-3af04dda7436-config-data\") pod \"ceilometer-0\" (UID: \"7c1938cc-b16b-4acd-b39c-3af04dda7436\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:54:03 crc kubenswrapper[5052]: I1203 12:54:03.259006 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/7c1938cc-b16b-4acd-b39c-3af04dda7436-log-httpd\") pod \"ceilometer-0\" (UID: \"7c1938cc-b16b-4acd-b39c-3af04dda7436\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:54:03 crc kubenswrapper[5052]: I1203 12:54:03.259063 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/7c1938cc-b16b-4acd-b39c-3af04dda7436-run-httpd\") pod \"ceilometer-0\" (UID: \"7c1938cc-b16b-4acd-b39c-3af04dda7436\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:54:03 crc kubenswrapper[5052]: I1203 12:54:03.264890 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/7c1938cc-b16b-4acd-b39c-3af04dda7436-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"7c1938cc-b16b-4acd-b39c-3af04dda7436\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:54:03 crc kubenswrapper[5052]: I1203 12:54:03.266457 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/7c1938cc-b16b-4acd-b39c-3af04dda7436-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"7c1938cc-b16b-4acd-b39c-3af04dda7436\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:54:03 crc kubenswrapper[5052]: I1203 12:54:03.266890 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/7c1938cc-b16b-4acd-b39c-3af04dda7436-ceilometer-tls-certs\") pod \"ceilometer-0\" (UID: \"7c1938cc-b16b-4acd-b39c-3af04dda7436\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:54:03 crc kubenswrapper[5052]: I1203 12:54:03.269085 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/7c1938cc-b16b-4acd-b39c-3af04dda7436-scripts\") pod \"ceilometer-0\" (UID: \"7c1938cc-b16b-4acd-b39c-3af04dda7436\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:54:03 crc kubenswrapper[5052]: I1203 12:54:03.280752 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-kwghq\" (UniqueName: \"kubernetes.io/projected/7c1938cc-b16b-4acd-b39c-3af04dda7436-kube-api-access-kwghq\") pod \"ceilometer-0\" (UID: \"7c1938cc-b16b-4acd-b39c-3af04dda7436\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:54:03 crc kubenswrapper[5052]: I1203 12:54:03.280937 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/7c1938cc-b16b-4acd-b39c-3af04dda7436-config-data\") pod \"ceilometer-0\" (UID: \"7c1938cc-b16b-4acd-b39c-3af04dda7436\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:54:03 crc kubenswrapper[5052]: I1203 12:54:03.373599 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:54:03 crc kubenswrapper[5052]: I1203 12:54:03.459850 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/cb5f26d2-4434-4f20-a7bc-49d58108d9ce-logs\") pod \"cb5f26d2-4434-4f20-a7bc-49d58108d9ce\" (UID: \"cb5f26d2-4434-4f20-a7bc-49d58108d9ce\") " Dec 03 12:54:03 crc kubenswrapper[5052]: I1203 12:54:03.460440 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r69xw\" (UniqueName: \"kubernetes.io/projected/cb5f26d2-4434-4f20-a7bc-49d58108d9ce-kube-api-access-r69xw\") pod \"cb5f26d2-4434-4f20-a7bc-49d58108d9ce\" (UID: \"cb5f26d2-4434-4f20-a7bc-49d58108d9ce\") " Dec 03 12:54:03 crc kubenswrapper[5052]: I1203 12:54:03.460488 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cb5f26d2-4434-4f20-a7bc-49d58108d9ce-logs" (OuterVolumeSpecName: "logs") pod "cb5f26d2-4434-4f20-a7bc-49d58108d9ce" (UID: "cb5f26d2-4434-4f20-a7bc-49d58108d9ce"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:54:03 crc kubenswrapper[5052]: I1203 12:54:03.460594 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/cb5f26d2-4434-4f20-a7bc-49d58108d9ce-config-data\") pod \"cb5f26d2-4434-4f20-a7bc-49d58108d9ce\" (UID: \"cb5f26d2-4434-4f20-a7bc-49d58108d9ce\") " Dec 03 12:54:03 crc kubenswrapper[5052]: I1203 12:54:03.460641 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/cb5f26d2-4434-4f20-a7bc-49d58108d9ce-combined-ca-bundle\") pod \"cb5f26d2-4434-4f20-a7bc-49d58108d9ce\" (UID: \"cb5f26d2-4434-4f20-a7bc-49d58108d9ce\") " Dec 03 12:54:03 crc kubenswrapper[5052]: I1203 12:54:03.461142 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/cb5f26d2-4434-4f20-a7bc-49d58108d9ce-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:03 crc kubenswrapper[5052]: I1203 12:54:03.491552 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cb5f26d2-4434-4f20-a7bc-49d58108d9ce-kube-api-access-r69xw" (OuterVolumeSpecName: "kube-api-access-r69xw") pod "cb5f26d2-4434-4f20-a7bc-49d58108d9ce" (UID: "cb5f26d2-4434-4f20-a7bc-49d58108d9ce"). InnerVolumeSpecName "kube-api-access-r69xw". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:54:03 crc kubenswrapper[5052]: I1203 12:54:03.492377 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cb5f26d2-4434-4f20-a7bc-49d58108d9ce-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "cb5f26d2-4434-4f20-a7bc-49d58108d9ce" (UID: "cb5f26d2-4434-4f20-a7bc-49d58108d9ce"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:54:03 crc kubenswrapper[5052]: I1203 12:54:03.518335 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cb5f26d2-4434-4f20-a7bc-49d58108d9ce-config-data" (OuterVolumeSpecName: "config-data") pod "cb5f26d2-4434-4f20-a7bc-49d58108d9ce" (UID: "cb5f26d2-4434-4f20-a7bc-49d58108d9ce"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:54:03 crc kubenswrapper[5052]: I1203 12:54:03.563193 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-r69xw\" (UniqueName: \"kubernetes.io/projected/cb5f26d2-4434-4f20-a7bc-49d58108d9ce-kube-api-access-r69xw\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:03 crc kubenswrapper[5052]: I1203 12:54:03.563225 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/cb5f26d2-4434-4f20-a7bc-49d58108d9ce-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:03 crc kubenswrapper[5052]: I1203 12:54:03.563236 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/cb5f26d2-4434-4f20-a7bc-49d58108d9ce-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:03 crc kubenswrapper[5052]: I1203 12:54:03.685459 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f99b50df-1d5b-4535-bc09-86078a1318bd" path="/var/lib/kubelet/pods/f99b50df-1d5b-4535-bc09-86078a1318bd/volumes" Dec 03 12:54:04 crc kubenswrapper[5052]: I1203 12:54:04.031587 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"cb5f26d2-4434-4f20-a7bc-49d58108d9ce","Type":"ContainerDied","Data":"d46a4912699adbf3859fb864b472e197b4bd54f2770ed821a5b9be07f3630f75"} Dec 03 12:54:04 crc kubenswrapper[5052]: I1203 12:54:04.031644 5052 scope.go:117] "RemoveContainer" containerID="41c8290b9e5fa58a7f3d6b59794a41b3e1646cb946f63f4436263b8c5f5de28a" Dec 03 12:54:04 crc kubenswrapper[5052]: I1203 12:54:04.031601 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:54:04 crc kubenswrapper[5052]: I1203 12:54:04.033109 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:54:04 crc kubenswrapper[5052]: I1203 12:54:04.049724 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:54:04 crc kubenswrapper[5052]: I1203 12:54:04.067827 5052 scope.go:117] "RemoveContainer" containerID="0cb6fd060ff53407ddd9999edc7d4e22f3a7033c40e984ce5173cef5e4eb20f1" Dec 03 12:54:04 crc kubenswrapper[5052]: I1203 12:54:04.075018 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/7c1938cc-b16b-4acd-b39c-3af04dda7436-ceilometer-tls-certs\") pod \"7c1938cc-b16b-4acd-b39c-3af04dda7436\" (UID: \"7c1938cc-b16b-4acd-b39c-3af04dda7436\") " Dec 03 12:54:04 crc kubenswrapper[5052]: I1203 12:54:04.075121 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kwghq\" (UniqueName: \"kubernetes.io/projected/7c1938cc-b16b-4acd-b39c-3af04dda7436-kube-api-access-kwghq\") pod \"7c1938cc-b16b-4acd-b39c-3af04dda7436\" (UID: \"7c1938cc-b16b-4acd-b39c-3af04dda7436\") " Dec 03 12:54:04 crc kubenswrapper[5052]: I1203 12:54:04.075187 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/7c1938cc-b16b-4acd-b39c-3af04dda7436-run-httpd\") pod \"7c1938cc-b16b-4acd-b39c-3af04dda7436\" (UID: \"7c1938cc-b16b-4acd-b39c-3af04dda7436\") " Dec 03 12:54:04 crc kubenswrapper[5052]: I1203 12:54:04.075216 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/7c1938cc-b16b-4acd-b39c-3af04dda7436-sg-core-conf-yaml\") pod \"7c1938cc-b16b-4acd-b39c-3af04dda7436\" (UID: \"7c1938cc-b16b-4acd-b39c-3af04dda7436\") " Dec 03 12:54:04 crc kubenswrapper[5052]: I1203 12:54:04.075305 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/7c1938cc-b16b-4acd-b39c-3af04dda7436-combined-ca-bundle\") pod \"7c1938cc-b16b-4acd-b39c-3af04dda7436\" (UID: \"7c1938cc-b16b-4acd-b39c-3af04dda7436\") " Dec 03 12:54:04 crc kubenswrapper[5052]: I1203 12:54:04.075339 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/7c1938cc-b16b-4acd-b39c-3af04dda7436-log-httpd\") pod \"7c1938cc-b16b-4acd-b39c-3af04dda7436\" (UID: \"7c1938cc-b16b-4acd-b39c-3af04dda7436\") " Dec 03 12:54:04 crc kubenswrapper[5052]: I1203 12:54:04.075398 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/7c1938cc-b16b-4acd-b39c-3af04dda7436-scripts\") pod \"7c1938cc-b16b-4acd-b39c-3af04dda7436\" (UID: \"7c1938cc-b16b-4acd-b39c-3af04dda7436\") " Dec 03 12:54:04 crc kubenswrapper[5052]: I1203 12:54:04.075448 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/7c1938cc-b16b-4acd-b39c-3af04dda7436-config-data\") pod \"7c1938cc-b16b-4acd-b39c-3af04dda7436\" (UID: \"7c1938cc-b16b-4acd-b39c-3af04dda7436\") " Dec 03 12:54:04 crc kubenswrapper[5052]: I1203 12:54:04.079421 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7c1938cc-b16b-4acd-b39c-3af04dda7436-log-httpd" (OuterVolumeSpecName: "log-httpd") pod "7c1938cc-b16b-4acd-b39c-3af04dda7436" (UID: "7c1938cc-b16b-4acd-b39c-3af04dda7436"). InnerVolumeSpecName "log-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:54:04 crc kubenswrapper[5052]: I1203 12:54:04.084994 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7c1938cc-b16b-4acd-b39c-3af04dda7436-kube-api-access-kwghq" (OuterVolumeSpecName: "kube-api-access-kwghq") pod "7c1938cc-b16b-4acd-b39c-3af04dda7436" (UID: "7c1938cc-b16b-4acd-b39c-3af04dda7436"). InnerVolumeSpecName "kube-api-access-kwghq". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:54:04 crc kubenswrapper[5052]: I1203 12:54:04.088038 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7c1938cc-b16b-4acd-b39c-3af04dda7436-ceilometer-tls-certs" (OuterVolumeSpecName: "ceilometer-tls-certs") pod "7c1938cc-b16b-4acd-b39c-3af04dda7436" (UID: "7c1938cc-b16b-4acd-b39c-3af04dda7436"). InnerVolumeSpecName "ceilometer-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:54:04 crc kubenswrapper[5052]: I1203 12:54:04.100614 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7c1938cc-b16b-4acd-b39c-3af04dda7436-scripts" (OuterVolumeSpecName: "scripts") pod "7c1938cc-b16b-4acd-b39c-3af04dda7436" (UID: "7c1938cc-b16b-4acd-b39c-3af04dda7436"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:54:04 crc kubenswrapper[5052]: I1203 12:54:04.101081 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7c1938cc-b16b-4acd-b39c-3af04dda7436-run-httpd" (OuterVolumeSpecName: "run-httpd") pod "7c1938cc-b16b-4acd-b39c-3af04dda7436" (UID: "7c1938cc-b16b-4acd-b39c-3af04dda7436"). InnerVolumeSpecName "run-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:54:04 crc kubenswrapper[5052]: I1203 12:54:04.100779 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:54:04 crc kubenswrapper[5052]: I1203 12:54:04.107704 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7c1938cc-b16b-4acd-b39c-3af04dda7436-config-data" (OuterVolumeSpecName: "config-data") pod "7c1938cc-b16b-4acd-b39c-3af04dda7436" (UID: "7c1938cc-b16b-4acd-b39c-3af04dda7436"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:54:04 crc kubenswrapper[5052]: I1203 12:54:04.114858 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7c1938cc-b16b-4acd-b39c-3af04dda7436-sg-core-conf-yaml" (OuterVolumeSpecName: "sg-core-conf-yaml") pod "7c1938cc-b16b-4acd-b39c-3af04dda7436" (UID: "7c1938cc-b16b-4acd-b39c-3af04dda7436"). InnerVolumeSpecName "sg-core-conf-yaml". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:54:04 crc kubenswrapper[5052]: I1203 12:54:04.121049 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7c1938cc-b16b-4acd-b39c-3af04dda7436-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "7c1938cc-b16b-4acd-b39c-3af04dda7436" (UID: "7c1938cc-b16b-4acd-b39c-3af04dda7436"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:54:04 crc kubenswrapper[5052]: I1203 12:54:04.134919 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:54:04 crc kubenswrapper[5052]: I1203 12:54:04.148617 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:54:04 crc kubenswrapper[5052]: E1203 12:54:04.149093 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="cb5f26d2-4434-4f20-a7bc-49d58108d9ce" containerName="nova-api-log" Dec 03 12:54:04 crc kubenswrapper[5052]: I1203 12:54:04.149115 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="cb5f26d2-4434-4f20-a7bc-49d58108d9ce" containerName="nova-api-log" Dec 03 12:54:04 crc kubenswrapper[5052]: E1203 12:54:04.149154 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="cb5f26d2-4434-4f20-a7bc-49d58108d9ce" containerName="nova-api-api" Dec 03 12:54:04 crc kubenswrapper[5052]: I1203 12:54:04.149161 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="cb5f26d2-4434-4f20-a7bc-49d58108d9ce" containerName="nova-api-api" Dec 03 12:54:04 crc kubenswrapper[5052]: I1203 12:54:04.149338 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="cb5f26d2-4434-4f20-a7bc-49d58108d9ce" containerName="nova-api-api" Dec 03 12:54:04 crc kubenswrapper[5052]: I1203 12:54:04.149368 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="cb5f26d2-4434-4f20-a7bc-49d58108d9ce" containerName="nova-api-log" Dec 03 12:54:04 crc kubenswrapper[5052]: I1203 12:54:04.150664 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:54:04 crc kubenswrapper[5052]: I1203 12:54:04.152859 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-api-config-data" Dec 03 12:54:04 crc kubenswrapper[5052]: I1203 12:54:04.153247 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-nova-internal-svc" Dec 03 12:54:04 crc kubenswrapper[5052]: I1203 12:54:04.153240 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-nova-public-svc" Dec 03 12:54:04 crc kubenswrapper[5052]: I1203 12:54:04.167099 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:54:04 crc kubenswrapper[5052]: I1203 12:54:04.177661 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/7d297f47-a73f-4a88-bba0-3108deb8df4d-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"7d297f47-a73f-4a88-bba0-3108deb8df4d\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:54:04 crc kubenswrapper[5052]: I1203 12:54:04.177771 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/7d297f47-a73f-4a88-bba0-3108deb8df4d-internal-tls-certs\") pod \"nova-api-0\" (UID: \"7d297f47-a73f-4a88-bba0-3108deb8df4d\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:54:04 crc kubenswrapper[5052]: I1203 12:54:04.177809 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/7d297f47-a73f-4a88-bba0-3108deb8df4d-config-data\") pod \"nova-api-0\" (UID: \"7d297f47-a73f-4a88-bba0-3108deb8df4d\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:54:04 crc kubenswrapper[5052]: I1203 12:54:04.177834 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/7d297f47-a73f-4a88-bba0-3108deb8df4d-public-tls-certs\") pod \"nova-api-0\" (UID: \"7d297f47-a73f-4a88-bba0-3108deb8df4d\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:54:04 crc kubenswrapper[5052]: I1203 12:54:04.177896 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5sp7g\" (UniqueName: \"kubernetes.io/projected/7d297f47-a73f-4a88-bba0-3108deb8df4d-kube-api-access-5sp7g\") pod \"nova-api-0\" (UID: \"7d297f47-a73f-4a88-bba0-3108deb8df4d\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:54:04 crc kubenswrapper[5052]: I1203 12:54:04.177919 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/7d297f47-a73f-4a88-bba0-3108deb8df4d-logs\") pod \"nova-api-0\" (UID: \"7d297f47-a73f-4a88-bba0-3108deb8df4d\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:54:04 crc kubenswrapper[5052]: I1203 12:54:04.177990 5052 reconciler_common.go:293] "Volume detached for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/7c1938cc-b16b-4acd-b39c-3af04dda7436-run-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:04 crc kubenswrapper[5052]: I1203 12:54:04.178005 5052 reconciler_common.go:293] "Volume detached for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/7c1938cc-b16b-4acd-b39c-3af04dda7436-sg-core-conf-yaml\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:04 crc kubenswrapper[5052]: I1203 12:54:04.178020 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/7c1938cc-b16b-4acd-b39c-3af04dda7436-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:04 crc kubenswrapper[5052]: I1203 12:54:04.178031 5052 reconciler_common.go:293] "Volume detached for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/7c1938cc-b16b-4acd-b39c-3af04dda7436-log-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:04 crc kubenswrapper[5052]: I1203 12:54:04.178043 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/7c1938cc-b16b-4acd-b39c-3af04dda7436-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:04 crc kubenswrapper[5052]: I1203 12:54:04.178055 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/7c1938cc-b16b-4acd-b39c-3af04dda7436-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:04 crc kubenswrapper[5052]: I1203 12:54:04.178067 5052 reconciler_common.go:293] "Volume detached for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/7c1938cc-b16b-4acd-b39c-3af04dda7436-ceilometer-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:04 crc kubenswrapper[5052]: I1203 12:54:04.178079 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-kwghq\" (UniqueName: \"kubernetes.io/projected/7c1938cc-b16b-4acd-b39c-3af04dda7436-kube-api-access-kwghq\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:04 crc kubenswrapper[5052]: I1203 12:54:04.229596 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openshift-marketplace/community-operators-42jr4" Dec 03 12:54:04 crc kubenswrapper[5052]: I1203 12:54:04.229813 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-marketplace/community-operators-42jr4" Dec 03 12:54:04 crc kubenswrapper[5052]: I1203 12:54:04.279717 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/7d297f47-a73f-4a88-bba0-3108deb8df4d-internal-tls-certs\") pod \"nova-api-0\" (UID: \"7d297f47-a73f-4a88-bba0-3108deb8df4d\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:54:04 crc kubenswrapper[5052]: I1203 12:54:04.279803 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/7d297f47-a73f-4a88-bba0-3108deb8df4d-config-data\") pod \"nova-api-0\" (UID: \"7d297f47-a73f-4a88-bba0-3108deb8df4d\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:54:04 crc kubenswrapper[5052]: I1203 12:54:04.279838 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/7d297f47-a73f-4a88-bba0-3108deb8df4d-public-tls-certs\") pod \"nova-api-0\" (UID: \"7d297f47-a73f-4a88-bba0-3108deb8df4d\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:54:04 crc kubenswrapper[5052]: I1203 12:54:04.279843 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-marketplace/community-operators-42jr4" Dec 03 12:54:04 crc kubenswrapper[5052]: I1203 12:54:04.279911 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-5sp7g\" (UniqueName: \"kubernetes.io/projected/7d297f47-a73f-4a88-bba0-3108deb8df4d-kube-api-access-5sp7g\") pod \"nova-api-0\" (UID: \"7d297f47-a73f-4a88-bba0-3108deb8df4d\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:54:04 crc kubenswrapper[5052]: I1203 12:54:04.279933 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/7d297f47-a73f-4a88-bba0-3108deb8df4d-logs\") pod \"nova-api-0\" (UID: \"7d297f47-a73f-4a88-bba0-3108deb8df4d\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:54:04 crc kubenswrapper[5052]: I1203 12:54:04.279996 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/7d297f47-a73f-4a88-bba0-3108deb8df4d-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"7d297f47-a73f-4a88-bba0-3108deb8df4d\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:54:04 crc kubenswrapper[5052]: I1203 12:54:04.282037 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/7d297f47-a73f-4a88-bba0-3108deb8df4d-logs\") pod \"nova-api-0\" (UID: \"7d297f47-a73f-4a88-bba0-3108deb8df4d\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:54:04 crc kubenswrapper[5052]: I1203 12:54:04.284132 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/7d297f47-a73f-4a88-bba0-3108deb8df4d-combined-ca-bundle\") pod \"nova-api-0\" (UID: \"7d297f47-a73f-4a88-bba0-3108deb8df4d\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:54:04 crc kubenswrapper[5052]: I1203 12:54:04.284387 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/7d297f47-a73f-4a88-bba0-3108deb8df4d-config-data\") pod \"nova-api-0\" (UID: \"7d297f47-a73f-4a88-bba0-3108deb8df4d\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:54:04 crc kubenswrapper[5052]: I1203 12:54:04.289447 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/7d297f47-a73f-4a88-bba0-3108deb8df4d-public-tls-certs\") pod \"nova-api-0\" (UID: \"7d297f47-a73f-4a88-bba0-3108deb8df4d\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:54:04 crc kubenswrapper[5052]: I1203 12:54:04.297125 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/7d297f47-a73f-4a88-bba0-3108deb8df4d-internal-tls-certs\") pod \"nova-api-0\" (UID: \"7d297f47-a73f-4a88-bba0-3108deb8df4d\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:54:04 crc kubenswrapper[5052]: I1203 12:54:04.305559 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-5sp7g\" (UniqueName: \"kubernetes.io/projected/7d297f47-a73f-4a88-bba0-3108deb8df4d-kube-api-access-5sp7g\") pod \"nova-api-0\" (UID: \"7d297f47-a73f-4a88-bba0-3108deb8df4d\") " pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:54:04 crc kubenswrapper[5052]: I1203 12:54:04.527530 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:54:05 crc kubenswrapper[5052]: I1203 12:54:05.030012 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:54:05 crc kubenswrapper[5052]: I1203 12:54:05.042955 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:54:05 crc kubenswrapper[5052]: I1203 12:54:05.045636 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"7d297f47-a73f-4a88-bba0-3108deb8df4d","Type":"ContainerStarted","Data":"cf937784e1e4cbf8b442e984fb4bf2a8b4cc9faedeeed62447a5cd38c718719e"} Dec 03 12:54:05 crc kubenswrapper[5052]: I1203 12:54:05.125072 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:54:05 crc kubenswrapper[5052]: I1203 12:54:05.135205 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:54:05 crc kubenswrapper[5052]: I1203 12:54:05.143637 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:54:05 crc kubenswrapper[5052]: I1203 12:54:05.145840 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:54:05 crc kubenswrapper[5052]: I1203 12:54:05.150374 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-scripts" Dec 03 12:54:05 crc kubenswrapper[5052]: I1203 12:54:05.150768 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"ceilometer-config-data" Dec 03 12:54:05 crc kubenswrapper[5052]: I1203 12:54:05.156170 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"cert-ceilometer-internal-svc" Dec 03 12:54:05 crc kubenswrapper[5052]: I1203 12:54:05.172634 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:54:05 crc kubenswrapper[5052]: I1203 12:54:05.199769 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/02edb9e9-3f42-4aea-9fe6-7efa332d5fb8-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"02edb9e9-3f42-4aea-9fe6-7efa332d5fb8\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:54:05 crc kubenswrapper[5052]: I1203 12:54:05.199809 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rtlhb\" (UniqueName: \"kubernetes.io/projected/02edb9e9-3f42-4aea-9fe6-7efa332d5fb8-kube-api-access-rtlhb\") pod \"ceilometer-0\" (UID: \"02edb9e9-3f42-4aea-9fe6-7efa332d5fb8\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:54:05 crc kubenswrapper[5052]: I1203 12:54:05.199845 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/02edb9e9-3f42-4aea-9fe6-7efa332d5fb8-ceilometer-tls-certs\") pod \"ceilometer-0\" (UID: \"02edb9e9-3f42-4aea-9fe6-7efa332d5fb8\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:54:05 crc kubenswrapper[5052]: I1203 12:54:05.199872 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/02edb9e9-3f42-4aea-9fe6-7efa332d5fb8-run-httpd\") pod \"ceilometer-0\" (UID: \"02edb9e9-3f42-4aea-9fe6-7efa332d5fb8\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:54:05 crc kubenswrapper[5052]: I1203 12:54:05.199901 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/02edb9e9-3f42-4aea-9fe6-7efa332d5fb8-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"02edb9e9-3f42-4aea-9fe6-7efa332d5fb8\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:54:05 crc kubenswrapper[5052]: I1203 12:54:05.199920 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/02edb9e9-3f42-4aea-9fe6-7efa332d5fb8-log-httpd\") pod \"ceilometer-0\" (UID: \"02edb9e9-3f42-4aea-9fe6-7efa332d5fb8\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:54:05 crc kubenswrapper[5052]: I1203 12:54:05.200112 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/02edb9e9-3f42-4aea-9fe6-7efa332d5fb8-scripts\") pod \"ceilometer-0\" (UID: \"02edb9e9-3f42-4aea-9fe6-7efa332d5fb8\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:54:05 crc kubenswrapper[5052]: I1203 12:54:05.200217 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/02edb9e9-3f42-4aea-9fe6-7efa332d5fb8-config-data\") pod \"ceilometer-0\" (UID: \"02edb9e9-3f42-4aea-9fe6-7efa332d5fb8\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:54:05 crc kubenswrapper[5052]: I1203 12:54:05.302347 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/02edb9e9-3f42-4aea-9fe6-7efa332d5fb8-scripts\") pod \"ceilometer-0\" (UID: \"02edb9e9-3f42-4aea-9fe6-7efa332d5fb8\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:54:05 crc kubenswrapper[5052]: I1203 12:54:05.302736 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/02edb9e9-3f42-4aea-9fe6-7efa332d5fb8-config-data\") pod \"ceilometer-0\" (UID: \"02edb9e9-3f42-4aea-9fe6-7efa332d5fb8\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:54:05 crc kubenswrapper[5052]: I1203 12:54:05.302828 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/02edb9e9-3f42-4aea-9fe6-7efa332d5fb8-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"02edb9e9-3f42-4aea-9fe6-7efa332d5fb8\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:54:05 crc kubenswrapper[5052]: I1203 12:54:05.302854 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-rtlhb\" (UniqueName: \"kubernetes.io/projected/02edb9e9-3f42-4aea-9fe6-7efa332d5fb8-kube-api-access-rtlhb\") pod \"ceilometer-0\" (UID: \"02edb9e9-3f42-4aea-9fe6-7efa332d5fb8\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:54:05 crc kubenswrapper[5052]: I1203 12:54:05.302887 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/02edb9e9-3f42-4aea-9fe6-7efa332d5fb8-ceilometer-tls-certs\") pod \"ceilometer-0\" (UID: \"02edb9e9-3f42-4aea-9fe6-7efa332d5fb8\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:54:05 crc kubenswrapper[5052]: I1203 12:54:05.302918 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/02edb9e9-3f42-4aea-9fe6-7efa332d5fb8-run-httpd\") pod \"ceilometer-0\" (UID: \"02edb9e9-3f42-4aea-9fe6-7efa332d5fb8\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:54:05 crc kubenswrapper[5052]: I1203 12:54:05.302950 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/02edb9e9-3f42-4aea-9fe6-7efa332d5fb8-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"02edb9e9-3f42-4aea-9fe6-7efa332d5fb8\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:54:05 crc kubenswrapper[5052]: I1203 12:54:05.302971 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/02edb9e9-3f42-4aea-9fe6-7efa332d5fb8-log-httpd\") pod \"ceilometer-0\" (UID: \"02edb9e9-3f42-4aea-9fe6-7efa332d5fb8\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:54:05 crc kubenswrapper[5052]: I1203 12:54:05.303497 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/02edb9e9-3f42-4aea-9fe6-7efa332d5fb8-log-httpd\") pod \"ceilometer-0\" (UID: \"02edb9e9-3f42-4aea-9fe6-7efa332d5fb8\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:54:05 crc kubenswrapper[5052]: I1203 12:54:05.305325 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/02edb9e9-3f42-4aea-9fe6-7efa332d5fb8-run-httpd\") pod \"ceilometer-0\" (UID: \"02edb9e9-3f42-4aea-9fe6-7efa332d5fb8\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:54:05 crc kubenswrapper[5052]: I1203 12:54:05.308248 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/02edb9e9-3f42-4aea-9fe6-7efa332d5fb8-ceilometer-tls-certs\") pod \"ceilometer-0\" (UID: \"02edb9e9-3f42-4aea-9fe6-7efa332d5fb8\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:54:05 crc kubenswrapper[5052]: I1203 12:54:05.308578 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/02edb9e9-3f42-4aea-9fe6-7efa332d5fb8-scripts\") pod \"ceilometer-0\" (UID: \"02edb9e9-3f42-4aea-9fe6-7efa332d5fb8\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:54:05 crc kubenswrapper[5052]: I1203 12:54:05.308727 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/02edb9e9-3f42-4aea-9fe6-7efa332d5fb8-config-data\") pod \"ceilometer-0\" (UID: \"02edb9e9-3f42-4aea-9fe6-7efa332d5fb8\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:54:05 crc kubenswrapper[5052]: I1203 12:54:05.309010 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/02edb9e9-3f42-4aea-9fe6-7efa332d5fb8-combined-ca-bundle\") pod \"ceilometer-0\" (UID: \"02edb9e9-3f42-4aea-9fe6-7efa332d5fb8\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:54:05 crc kubenswrapper[5052]: I1203 12:54:05.317663 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/02edb9e9-3f42-4aea-9fe6-7efa332d5fb8-sg-core-conf-yaml\") pod \"ceilometer-0\" (UID: \"02edb9e9-3f42-4aea-9fe6-7efa332d5fb8\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:54:05 crc kubenswrapper[5052]: I1203 12:54:05.323640 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-rtlhb\" (UniqueName: \"kubernetes.io/projected/02edb9e9-3f42-4aea-9fe6-7efa332d5fb8-kube-api-access-rtlhb\") pod \"ceilometer-0\" (UID: \"02edb9e9-3f42-4aea-9fe6-7efa332d5fb8\") " pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:54:05 crc kubenswrapper[5052]: I1203 12:54:05.489620 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:54:05 crc kubenswrapper[5052]: I1203 12:54:05.691210 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7c1938cc-b16b-4acd-b39c-3af04dda7436" path="/var/lib/kubelet/pods/7c1938cc-b16b-4acd-b39c-3af04dda7436/volumes" Dec 03 12:54:05 crc kubenswrapper[5052]: I1203 12:54:05.694453 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cb5f26d2-4434-4f20-a7bc-49d58108d9ce" path="/var/lib/kubelet/pods/cb5f26d2-4434-4f20-a7bc-49d58108d9ce/volumes" Dec 03 12:54:05 crc kubenswrapper[5052]: W1203 12:54:05.946652 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod02edb9e9_3f42_4aea_9fe6_7efa332d5fb8.slice/crio-49eac351cf028c70490c6de088af1b224540eb2aa17cd045c37778ceb031307d WatchSource:0}: Error finding container 49eac351cf028c70490c6de088af1b224540eb2aa17cd045c37778ceb031307d: Status 404 returned error can't find the container with id 49eac351cf028c70490c6de088af1b224540eb2aa17cd045c37778ceb031307d Dec 03 12:54:05 crc kubenswrapper[5052]: I1203 12:54:05.950017 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:54:06 crc kubenswrapper[5052]: I1203 12:54:06.054260 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"7d297f47-a73f-4a88-bba0-3108deb8df4d","Type":"ContainerStarted","Data":"f15af8c0a8823cb666fbe94253b10ca1d40d1a5303a03c6ac25035b6456a7a85"} Dec 03 12:54:06 crc kubenswrapper[5052]: I1203 12:54:06.054318 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"7d297f47-a73f-4a88-bba0-3108deb8df4d","Type":"ContainerStarted","Data":"52ef14e30c84a7e38ff36a4c93fbfa8a565c2ed55f66905b14b2fdd87cd69077"} Dec 03 12:54:06 crc kubenswrapper[5052]: I1203 12:54:06.055468 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"02edb9e9-3f42-4aea-9fe6-7efa332d5fb8","Type":"ContainerStarted","Data":"49eac351cf028c70490c6de088af1b224540eb2aa17cd045c37778ceb031307d"} Dec 03 12:54:06 crc kubenswrapper[5052]: I1203 12:54:06.079149 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-api-0" podStartSLOduration=2.07912904 podStartE2EDuration="2.07912904s" podCreationTimestamp="2025-12-03 12:54:04 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:54:06.07528234 +0000 UTC m=+5636.663524104" watchObservedRunningTime="2025-12-03 12:54:06.07912904 +0000 UTC m=+5636.667370804" Dec 03 12:54:07 crc kubenswrapper[5052]: I1203 12:54:07.066982 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"02edb9e9-3f42-4aea-9fe6-7efa332d5fb8","Type":"ContainerStarted","Data":"86ed75c4a9b117c08c2a3ecec9878ef557fc9d6d68fea18aa83751770d37d20b"} Dec 03 12:54:07 crc kubenswrapper[5052]: E1203 12:54:07.842455 5052 cadvisor_stats_provider.go:516] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb0bb2e93_76f1_4485_b822_41e83b2a4f59.slice/crio-29b21fda6544a2343736d96bac4c7f7476a926e688d5808f8b0fa0235fbd671a\": RecentStats: unable to find data in memory cache]" Dec 03 12:54:09 crc kubenswrapper[5052]: I1203 12:54:09.090313 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"02edb9e9-3f42-4aea-9fe6-7efa332d5fb8","Type":"ContainerStarted","Data":"59b945b4df7623c03eb622cbe50e816dadd5224b5097d5e7e646cca960ca90e1"} Dec 03 12:54:09 crc kubenswrapper[5052]: I1203 12:54:09.090829 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"02edb9e9-3f42-4aea-9fe6-7efa332d5fb8","Type":"ContainerStarted","Data":"72785ef6236001625d6d7f1e690671c78dc1f11c3518cfcc3bf9d6de5ab4c2fb"} Dec 03 12:54:14 crc kubenswrapper[5052]: I1203 12:54:14.274565 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-marketplace/community-operators-42jr4" Dec 03 12:54:14 crc kubenswrapper[5052]: I1203 12:54:14.334005 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openshift-marketplace/community-operators-42jr4"] Dec 03 12:54:14 crc kubenswrapper[5052]: I1203 12:54:14.528530 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:54:14 crc kubenswrapper[5052]: I1203 12:54:14.529396 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:54:15 crc kubenswrapper[5052]: I1203 12:54:15.161211 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"02edb9e9-3f42-4aea-9fe6-7efa332d5fb8","Type":"ContainerStarted","Data":"990e0730d2387a55431e533e2f7232d5369fd95354ef6d99da9341e99b6b7c58"} Dec 03 12:54:15 crc kubenswrapper[5052]: I1203 12:54:15.161370 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openshift-marketplace/community-operators-42jr4" podUID="b6eacded-e668-400c-ba59-4de7e7f8bf78" containerName="registry-server" containerID="cri-o://3342bd0b32d00537ebb8c5f6c934d9afc76399f580767282f48c1be38c55e5c1" gracePeriod=2 Dec 03 12:54:15 crc kubenswrapper[5052]: I1203 12:54:15.161540 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:54:15 crc kubenswrapper[5052]: I1203 12:54:15.199430 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/ceilometer-0" podStartSLOduration=1.8703746780000001 podStartE2EDuration="10.199411843s" podCreationTimestamp="2025-12-03 12:54:05 +0000 UTC" firstStartedPulling="2025-12-03 12:54:05.94975995 +0000 UTC m=+5636.538001714" lastFinishedPulling="2025-12-03 12:54:14.278797115 +0000 UTC m=+5644.867038879" observedRunningTime="2025-12-03 12:54:15.188620785 +0000 UTC m=+5645.776862559" watchObservedRunningTime="2025-12-03 12:54:15.199411843 +0000 UTC m=+5645.787653607" Dec 03 12:54:15 crc kubenswrapper[5052]: I1203 12:54:15.579766 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-api-0" podUID="7d297f47-a73f-4a88-bba0-3108deb8df4d" containerName="nova-api-api" probeResult="failure" output="Get \"https://10.217.0.223:8774/\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Dec 03 12:54:15 crc kubenswrapper[5052]: I1203 12:54:15.580408 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openstack-kuttl-tests/nova-api-0" podUID="7d297f47-a73f-4a88-bba0-3108deb8df4d" containerName="nova-api-log" probeResult="failure" output="Get \"https://10.217.0.223:8774/\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Dec 03 12:54:15 crc kubenswrapper[5052]: I1203 12:54:15.757425 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/community-operators-42jr4" Dec 03 12:54:15 crc kubenswrapper[5052]: I1203 12:54:15.835325 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n28p4\" (UniqueName: \"kubernetes.io/projected/b6eacded-e668-400c-ba59-4de7e7f8bf78-kube-api-access-n28p4\") pod \"b6eacded-e668-400c-ba59-4de7e7f8bf78\" (UID: \"b6eacded-e668-400c-ba59-4de7e7f8bf78\") " Dec 03 12:54:15 crc kubenswrapper[5052]: I1203 12:54:15.835391 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/b6eacded-e668-400c-ba59-4de7e7f8bf78-catalog-content\") pod \"b6eacded-e668-400c-ba59-4de7e7f8bf78\" (UID: \"b6eacded-e668-400c-ba59-4de7e7f8bf78\") " Dec 03 12:54:15 crc kubenswrapper[5052]: I1203 12:54:15.835532 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/b6eacded-e668-400c-ba59-4de7e7f8bf78-utilities\") pod \"b6eacded-e668-400c-ba59-4de7e7f8bf78\" (UID: \"b6eacded-e668-400c-ba59-4de7e7f8bf78\") " Dec 03 12:54:15 crc kubenswrapper[5052]: I1203 12:54:15.844718 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b6eacded-e668-400c-ba59-4de7e7f8bf78-utilities" (OuterVolumeSpecName: "utilities") pod "b6eacded-e668-400c-ba59-4de7e7f8bf78" (UID: "b6eacded-e668-400c-ba59-4de7e7f8bf78"). InnerVolumeSpecName "utilities". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:54:15 crc kubenswrapper[5052]: I1203 12:54:15.866337 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b6eacded-e668-400c-ba59-4de7e7f8bf78-kube-api-access-n28p4" (OuterVolumeSpecName: "kube-api-access-n28p4") pod "b6eacded-e668-400c-ba59-4de7e7f8bf78" (UID: "b6eacded-e668-400c-ba59-4de7e7f8bf78"). InnerVolumeSpecName "kube-api-access-n28p4". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:54:15 crc kubenswrapper[5052]: I1203 12:54:15.938563 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-n28p4\" (UniqueName: \"kubernetes.io/projected/b6eacded-e668-400c-ba59-4de7e7f8bf78-kube-api-access-n28p4\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:15 crc kubenswrapper[5052]: I1203 12:54:15.938606 5052 reconciler_common.go:293] "Volume detached for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/b6eacded-e668-400c-ba59-4de7e7f8bf78-utilities\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:15 crc kubenswrapper[5052]: I1203 12:54:15.976392 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b6eacded-e668-400c-ba59-4de7e7f8bf78-catalog-content" (OuterVolumeSpecName: "catalog-content") pod "b6eacded-e668-400c-ba59-4de7e7f8bf78" (UID: "b6eacded-e668-400c-ba59-4de7e7f8bf78"). InnerVolumeSpecName "catalog-content". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:54:16 crc kubenswrapper[5052]: I1203 12:54:16.039990 5052 reconciler_common.go:293] "Volume detached for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/b6eacded-e668-400c-ba59-4de7e7f8bf78-catalog-content\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:16 crc kubenswrapper[5052]: I1203 12:54:16.173453 5052 generic.go:334] "Generic (PLEG): container finished" podID="b6eacded-e668-400c-ba59-4de7e7f8bf78" containerID="3342bd0b32d00537ebb8c5f6c934d9afc76399f580767282f48c1be38c55e5c1" exitCode=0 Dec 03 12:54:16 crc kubenswrapper[5052]: I1203 12:54:16.174621 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/community-operators-42jr4" Dec 03 12:54:16 crc kubenswrapper[5052]: I1203 12:54:16.175444 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/community-operators-42jr4" event={"ID":"b6eacded-e668-400c-ba59-4de7e7f8bf78","Type":"ContainerDied","Data":"3342bd0b32d00537ebb8c5f6c934d9afc76399f580767282f48c1be38c55e5c1"} Dec 03 12:54:16 crc kubenswrapper[5052]: I1203 12:54:16.175538 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/community-operators-42jr4" event={"ID":"b6eacded-e668-400c-ba59-4de7e7f8bf78","Type":"ContainerDied","Data":"d7c505d4c4561acb5e8829f2ed564173b44dffb5b6223e37e95fc69a47785021"} Dec 03 12:54:16 crc kubenswrapper[5052]: I1203 12:54:16.175561 5052 scope.go:117] "RemoveContainer" containerID="3342bd0b32d00537ebb8c5f6c934d9afc76399f580767282f48c1be38c55e5c1" Dec 03 12:54:16 crc kubenswrapper[5052]: I1203 12:54:16.231344 5052 scope.go:117] "RemoveContainer" containerID="24d9b7547faf46f13ce8052fefe424803375ae067d57181d41efb5c19b519092" Dec 03 12:54:16 crc kubenswrapper[5052]: I1203 12:54:16.234197 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openshift-marketplace/community-operators-42jr4"] Dec 03 12:54:16 crc kubenswrapper[5052]: I1203 12:54:16.246225 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openshift-marketplace/community-operators-42jr4"] Dec 03 12:54:16 crc kubenswrapper[5052]: I1203 12:54:16.266885 5052 scope.go:117] "RemoveContainer" containerID="771f29a89558d77fd33244f67e65bfe1da83beb6967c685b37a418114b7cce80" Dec 03 12:54:16 crc kubenswrapper[5052]: I1203 12:54:16.310599 5052 scope.go:117] "RemoveContainer" containerID="3342bd0b32d00537ebb8c5f6c934d9afc76399f580767282f48c1be38c55e5c1" Dec 03 12:54:16 crc kubenswrapper[5052]: E1203 12:54:16.310965 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3342bd0b32d00537ebb8c5f6c934d9afc76399f580767282f48c1be38c55e5c1\": container with ID starting with 3342bd0b32d00537ebb8c5f6c934d9afc76399f580767282f48c1be38c55e5c1 not found: ID does not exist" containerID="3342bd0b32d00537ebb8c5f6c934d9afc76399f580767282f48c1be38c55e5c1" Dec 03 12:54:16 crc kubenswrapper[5052]: I1203 12:54:16.310994 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3342bd0b32d00537ebb8c5f6c934d9afc76399f580767282f48c1be38c55e5c1"} err="failed to get container status \"3342bd0b32d00537ebb8c5f6c934d9afc76399f580767282f48c1be38c55e5c1\": rpc error: code = NotFound desc = could not find container \"3342bd0b32d00537ebb8c5f6c934d9afc76399f580767282f48c1be38c55e5c1\": container with ID starting with 3342bd0b32d00537ebb8c5f6c934d9afc76399f580767282f48c1be38c55e5c1 not found: ID does not exist" Dec 03 12:54:16 crc kubenswrapper[5052]: I1203 12:54:16.311021 5052 scope.go:117] "RemoveContainer" containerID="24d9b7547faf46f13ce8052fefe424803375ae067d57181d41efb5c19b519092" Dec 03 12:54:16 crc kubenswrapper[5052]: E1203 12:54:16.311219 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"24d9b7547faf46f13ce8052fefe424803375ae067d57181d41efb5c19b519092\": container with ID starting with 24d9b7547faf46f13ce8052fefe424803375ae067d57181d41efb5c19b519092 not found: ID does not exist" containerID="24d9b7547faf46f13ce8052fefe424803375ae067d57181d41efb5c19b519092" Dec 03 12:54:16 crc kubenswrapper[5052]: I1203 12:54:16.311246 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"24d9b7547faf46f13ce8052fefe424803375ae067d57181d41efb5c19b519092"} err="failed to get container status \"24d9b7547faf46f13ce8052fefe424803375ae067d57181d41efb5c19b519092\": rpc error: code = NotFound desc = could not find container \"24d9b7547faf46f13ce8052fefe424803375ae067d57181d41efb5c19b519092\": container with ID starting with 24d9b7547faf46f13ce8052fefe424803375ae067d57181d41efb5c19b519092 not found: ID does not exist" Dec 03 12:54:16 crc kubenswrapper[5052]: I1203 12:54:16.311264 5052 scope.go:117] "RemoveContainer" containerID="771f29a89558d77fd33244f67e65bfe1da83beb6967c685b37a418114b7cce80" Dec 03 12:54:16 crc kubenswrapper[5052]: E1203 12:54:16.311437 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"771f29a89558d77fd33244f67e65bfe1da83beb6967c685b37a418114b7cce80\": container with ID starting with 771f29a89558d77fd33244f67e65bfe1da83beb6967c685b37a418114b7cce80 not found: ID does not exist" containerID="771f29a89558d77fd33244f67e65bfe1da83beb6967c685b37a418114b7cce80" Dec 03 12:54:16 crc kubenswrapper[5052]: I1203 12:54:16.311464 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"771f29a89558d77fd33244f67e65bfe1da83beb6967c685b37a418114b7cce80"} err="failed to get container status \"771f29a89558d77fd33244f67e65bfe1da83beb6967c685b37a418114b7cce80\": rpc error: code = NotFound desc = could not find container \"771f29a89558d77fd33244f67e65bfe1da83beb6967c685b37a418114b7cce80\": container with ID starting with 771f29a89558d77fd33244f67e65bfe1da83beb6967c685b37a418114b7cce80 not found: ID does not exist" Dec 03 12:54:17 crc kubenswrapper[5052]: I1203 12:54:17.683914 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b6eacded-e668-400c-ba59-4de7e7f8bf78" path="/var/lib/kubelet/pods/b6eacded-e668-400c-ba59-4de7e7f8bf78/volumes" Dec 03 12:54:24 crc kubenswrapper[5052]: I1203 12:54:24.535856 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:54:24 crc kubenswrapper[5052]: I1203 12:54:24.537893 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:54:24 crc kubenswrapper[5052]: I1203 12:54:24.538695 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:54:24 crc kubenswrapper[5052]: I1203 12:54:24.538718 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:54:24 crc kubenswrapper[5052]: I1203 12:54:24.543708 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:54:24 crc kubenswrapper[5052]: I1203 12:54:24.545270 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:54:29 crc kubenswrapper[5052]: I1203 12:54:29.378942 5052 patch_prober.go:28] interesting pod/machine-config-daemon-74bj6 container/machine-config-daemon namespace/openshift-machine-config-operator: Liveness probe status=failure output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" start-of-body= Dec 03 12:54:29 crc kubenswrapper[5052]: I1203 12:54:29.379472 5052 prober.go:107] "Probe failed" probeType="Liveness" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" containerName="machine-config-daemon" probeResult="failure" output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" Dec 03 12:54:35 crc kubenswrapper[5052]: I1203 12:54:35.497029 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:54:36 crc kubenswrapper[5052]: I1203 12:54:36.490629 5052 scope.go:117] "RemoveContainer" containerID="b1a655b5c3d0ddb68e291790261b40d699c5471d8a606da7a88e93dbced9a7f8" Dec 03 12:54:36 crc kubenswrapper[5052]: I1203 12:54:36.518822 5052 scope.go:117] "RemoveContainer" containerID="1d02d8acfd7135f1ee2b76bfcdb23259e05bd05d52a412e5809afd7e88dcf52b" Dec 03 12:54:36 crc kubenswrapper[5052]: I1203 12:54:36.559126 5052 scope.go:117] "RemoveContainer" containerID="18022901c9e16d4c4eb5c5f284061eb245203a106525b3fb6b0192a33f4b392e" Dec 03 12:54:36 crc kubenswrapper[5052]: I1203 12:54:36.605228 5052 scope.go:117] "RemoveContainer" containerID="f38a12088ebf63cf61a94810998aa693db18e09b88bf0391d892ee960b99e9ce" Dec 03 12:54:36 crc kubenswrapper[5052]: I1203 12:54:36.646436 5052 scope.go:117] "RemoveContainer" containerID="c97e72423d1d83658766dc2e82aa17152e2bd056591379659920ae4ef1d9a268" Dec 03 12:54:36 crc kubenswrapper[5052]: I1203 12:54:36.684517 5052 scope.go:117] "RemoveContainer" containerID="4f74304b5716b547aed360588b0a9be626bd9def7e57532f38549b982366ac6f" Dec 03 12:54:36 crc kubenswrapper[5052]: I1203 12:54:36.731849 5052 scope.go:117] "RemoveContainer" containerID="a3f4f93f3787a5482025f013961d9f934ba51e800bf3b24730bce1b1e2696ee6" Dec 03 12:54:36 crc kubenswrapper[5052]: I1203 12:54:36.771861 5052 scope.go:117] "RemoveContainer" containerID="36278016e7f1acc45cc98e13ca70cbede708ca3f836f4836fe1a85096a93bdcf" Dec 03 12:54:36 crc kubenswrapper[5052]: I1203 12:54:36.797723 5052 scope.go:117] "RemoveContainer" containerID="74e52eb4f69076f917a1b3ab5d18438ec4c2d708d4bc29581185ab517d52919e" Dec 03 12:54:40 crc kubenswrapper[5052]: I1203 12:54:40.939857 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/openstackclient"] Dec 03 12:54:40 crc kubenswrapper[5052]: I1203 12:54:40.940706 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/openstackclient" podUID="c4b7bd04-32be-43c6-82f1-968f2e368b22" containerName="openstackclient" containerID="cri-o://863f9a5dead36dfcd4e960256f81adece8f398e82c1cc6eb7b8c1e26766d0633" gracePeriod=2 Dec 03 12:54:40 crc kubenswrapper[5052]: I1203 12:54:40.959673 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/openstackclient"] Dec 03 12:54:41 crc kubenswrapper[5052]: I1203 12:54:41.019359 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/rabbitmq-server-0"] Dec 03 12:54:41 crc kubenswrapper[5052]: E1203 12:54:41.243408 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/rabbitmq-config-data: configmap "rabbitmq-config-data" not found Dec 03 12:54:41 crc kubenswrapper[5052]: E1203 12:54:41.243459 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e73813f5-f97f-41dd-8a79-6196557d4d1a-config-data podName:e73813f5-f97f-41dd-8a79-6196557d4d1a nodeName:}" failed. No retries permitted until 2025-12-03 12:54:41.743446376 +0000 UTC m=+5672.331688140 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "config-data" (UniqueName: "kubernetes.io/configmap/e73813f5-f97f-41dd-8a79-6196557d4d1a-config-data") pod "rabbitmq-server-0" (UID: "e73813f5-f97f-41dd-8a79-6196557d4d1a") : configmap "rabbitmq-config-data" not found Dec 03 12:54:41 crc kubenswrapper[5052]: I1203 12:54:41.285973 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/placementbfa4-account-delete-flc9f"] Dec 03 12:54:41 crc kubenswrapper[5052]: E1203 12:54:41.286650 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="b6eacded-e668-400c-ba59-4de7e7f8bf78" containerName="registry-server" Dec 03 12:54:41 crc kubenswrapper[5052]: I1203 12:54:41.286729 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="b6eacded-e668-400c-ba59-4de7e7f8bf78" containerName="registry-server" Dec 03 12:54:41 crc kubenswrapper[5052]: E1203 12:54:41.286806 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="b6eacded-e668-400c-ba59-4de7e7f8bf78" containerName="extract-content" Dec 03 12:54:41 crc kubenswrapper[5052]: I1203 12:54:41.286868 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="b6eacded-e668-400c-ba59-4de7e7f8bf78" containerName="extract-content" Dec 03 12:54:41 crc kubenswrapper[5052]: E1203 12:54:41.286932 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="c4b7bd04-32be-43c6-82f1-968f2e368b22" containerName="openstackclient" Dec 03 12:54:41 crc kubenswrapper[5052]: I1203 12:54:41.286984 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="c4b7bd04-32be-43c6-82f1-968f2e368b22" containerName="openstackclient" Dec 03 12:54:41 crc kubenswrapper[5052]: E1203 12:54:41.287119 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="b6eacded-e668-400c-ba59-4de7e7f8bf78" containerName="extract-utilities" Dec 03 12:54:41 crc kubenswrapper[5052]: I1203 12:54:41.287177 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="b6eacded-e668-400c-ba59-4de7e7f8bf78" containerName="extract-utilities" Dec 03 12:54:41 crc kubenswrapper[5052]: I1203 12:54:41.287421 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="b6eacded-e668-400c-ba59-4de7e7f8bf78" containerName="registry-server" Dec 03 12:54:41 crc kubenswrapper[5052]: I1203 12:54:41.287495 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="c4b7bd04-32be-43c6-82f1-968f2e368b22" containerName="openstackclient" Dec 03 12:54:41 crc kubenswrapper[5052]: I1203 12:54:41.288176 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placementbfa4-account-delete-flc9f" Dec 03 12:54:41 crc kubenswrapper[5052]: I1203 12:54:41.318969 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/placementbfa4-account-delete-flc9f"] Dec 03 12:54:41 crc kubenswrapper[5052]: I1203 12:54:41.356585 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ovn-northd-0"] Dec 03 12:54:41 crc kubenswrapper[5052]: I1203 12:54:41.356915 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ovn-northd-0" podUID="3b93bc75-db87-4677-be2f-910d07bb696c" containerName="ovn-northd" containerID="cri-o://06bf7c042266ac81635b44d190310f81518aa47f7b8008ddde3f54ca515b15fd" gracePeriod=30 Dec 03 12:54:41 crc kubenswrapper[5052]: I1203 12:54:41.357451 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ovn-northd-0" podUID="3b93bc75-db87-4677-be2f-910d07bb696c" containerName="openstack-network-exporter" containerID="cri-o://7cdee3173161e7591d4f168eb2ed4975b7806623d040f607514a1bb1ab4a5542" gracePeriod=30 Dec 03 12:54:41 crc kubenswrapper[5052]: I1203 12:54:41.429435 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-nb-0"] Dec 03 12:54:41 crc kubenswrapper[5052]: I1203 12:54:41.430297 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ovsdbserver-nb-0" podUID="3a5e94ea-faeb-4202-843a-bd691d8895ce" containerName="openstack-network-exporter" containerID="cri-o://5dbfbe34fded1c834b0fb2487d51d6da9334318d91fb67fd92b8f7457b35a22f" gracePeriod=300 Dec 03 12:54:41 crc kubenswrapper[5052]: I1203 12:54:41.445818 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/c4d5c183-8df4-4c43-9afb-2b28c6aae9e1-operator-scripts\") pod \"placementbfa4-account-delete-flc9f\" (UID: \"c4d5c183-8df4-4c43-9afb-2b28c6aae9e1\") " pod="openstack-kuttl-tests/placementbfa4-account-delete-flc9f" Dec 03 12:54:41 crc kubenswrapper[5052]: I1203 12:54:41.446120 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-68vcw\" (UniqueName: \"kubernetes.io/projected/c4d5c183-8df4-4c43-9afb-2b28c6aae9e1-kube-api-access-68vcw\") pod \"placementbfa4-account-delete-flc9f\" (UID: \"c4d5c183-8df4-4c43-9afb-2b28c6aae9e1\") " pod="openstack-kuttl-tests/placementbfa4-account-delete-flc9f" Dec 03 12:54:41 crc kubenswrapper[5052]: I1203 12:54:41.548761 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-68vcw\" (UniqueName: \"kubernetes.io/projected/c4d5c183-8df4-4c43-9afb-2b28c6aae9e1-kube-api-access-68vcw\") pod \"placementbfa4-account-delete-flc9f\" (UID: \"c4d5c183-8df4-4c43-9afb-2b28c6aae9e1\") " pod="openstack-kuttl-tests/placementbfa4-account-delete-flc9f" Dec 03 12:54:41 crc kubenswrapper[5052]: I1203 12:54:41.549025 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/c4d5c183-8df4-4c43-9afb-2b28c6aae9e1-operator-scripts\") pod \"placementbfa4-account-delete-flc9f\" (UID: \"c4d5c183-8df4-4c43-9afb-2b28c6aae9e1\") " pod="openstack-kuttl-tests/placementbfa4-account-delete-flc9f" Dec 03 12:54:41 crc kubenswrapper[5052]: I1203 12:54:41.550041 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/c4d5c183-8df4-4c43-9afb-2b28c6aae9e1-operator-scripts\") pod \"placementbfa4-account-delete-flc9f\" (UID: \"c4d5c183-8df4-4c43-9afb-2b28c6aae9e1\") " pod="openstack-kuttl-tests/placementbfa4-account-delete-flc9f" Dec 03 12:54:41 crc kubenswrapper[5052]: I1203 12:54:41.573562 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/placement-db-sync-xbbsx"] Dec 03 12:54:41 crc kubenswrapper[5052]: I1203 12:54:41.705591 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-68vcw\" (UniqueName: \"kubernetes.io/projected/c4d5c183-8df4-4c43-9afb-2b28c6aae9e1-kube-api-access-68vcw\") pod \"placementbfa4-account-delete-flc9f\" (UID: \"c4d5c183-8df4-4c43-9afb-2b28c6aae9e1\") " pod="openstack-kuttl-tests/placementbfa4-account-delete-flc9f" Dec 03 12:54:41 crc kubenswrapper[5052]: E1203 12:54:41.797640 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/rabbitmq-config-data: configmap "rabbitmq-config-data" not found Dec 03 12:54:41 crc kubenswrapper[5052]: E1203 12:54:41.797694 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e73813f5-f97f-41dd-8a79-6196557d4d1a-config-data podName:e73813f5-f97f-41dd-8a79-6196557d4d1a nodeName:}" failed. No retries permitted until 2025-12-03 12:54:42.797679177 +0000 UTC m=+5673.385920941 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "config-data" (UniqueName: "kubernetes.io/configmap/e73813f5-f97f-41dd-8a79-6196557d4d1a-config-data") pod "rabbitmq-server-0" (UID: "e73813f5-f97f-41dd-8a79-6196557d4d1a") : configmap "rabbitmq-config-data" not found Dec 03 12:54:41 crc kubenswrapper[5052]: I1203 12:54:41.865581 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/placement-db-sync-xbbsx"] Dec 03 12:54:41 crc kubenswrapper[5052]: I1203 12:54:41.925295 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placementbfa4-account-delete-flc9f" Dec 03 12:54:41 crc kubenswrapper[5052]: I1203 12:54:41.979672 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/rabbitmq-cell1-server-0"] Dec 03 12:54:42 crc kubenswrapper[5052]: I1203 12:54:42.033748 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/neutron6c26-account-delete-d7wtw"] Dec 03 12:54:42 crc kubenswrapper[5052]: I1203 12:54:42.034982 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron6c26-account-delete-d7wtw" Dec 03 12:54:42 crc kubenswrapper[5052]: I1203 12:54:42.085601 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/neutron-db-sync-jxbww"] Dec 03 12:54:42 crc kubenswrapper[5052]: E1203 12:54:42.106262 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/rabbitmq-cell1-config-data: configmap "rabbitmq-cell1-config-data" not found Dec 03 12:54:42 crc kubenswrapper[5052]: E1203 12:54:42.106317 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/568addd3-4ad3-47c3-adae-693691e7cd40-config-data podName:568addd3-4ad3-47c3-adae-693691e7cd40 nodeName:}" failed. No retries permitted until 2025-12-03 12:54:42.606300843 +0000 UTC m=+5673.194542607 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "config-data" (UniqueName: "kubernetes.io/configmap/568addd3-4ad3-47c3-adae-693691e7cd40-config-data") pod "rabbitmq-cell1-server-0" (UID: "568addd3-4ad3-47c3-adae-693691e7cd40") : configmap "rabbitmq-cell1-config-data" not found Dec 03 12:54:42 crc kubenswrapper[5052]: I1203 12:54:42.134922 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/neutron-db-sync-jxbww"] Dec 03 12:54:42 crc kubenswrapper[5052]: I1203 12:54:42.189585 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron6c26-account-delete-d7wtw"] Dec 03 12:54:42 crc kubenswrapper[5052]: I1203 12:54:42.208967 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/941d063f-c834-47a1-bc0c-2096b005e09c-operator-scripts\") pod \"neutron6c26-account-delete-d7wtw\" (UID: \"941d063f-c834-47a1-bc0c-2096b005e09c\") " pod="openstack-kuttl-tests/neutron6c26-account-delete-d7wtw" Dec 03 12:54:42 crc kubenswrapper[5052]: I1203 12:54:42.209256 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gwqrq\" (UniqueName: \"kubernetes.io/projected/941d063f-c834-47a1-bc0c-2096b005e09c-kube-api-access-gwqrq\") pod \"neutron6c26-account-delete-d7wtw\" (UID: \"941d063f-c834-47a1-bc0c-2096b005e09c\") " pod="openstack-kuttl-tests/neutron6c26-account-delete-d7wtw" Dec 03 12:54:42 crc kubenswrapper[5052]: I1203 12:54:42.231766 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-sb-0"] Dec 03 12:54:42 crc kubenswrapper[5052]: I1203 12:54:42.232101 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ovsdbserver-sb-0" podUID="d16285c0-7e62-4b3c-b3e8-a62b8abb49cb" containerName="openstack-network-exporter" containerID="cri-o://89cea0c9094ed2fb8bb3973f66f25388cbb772d78cc9dc394bfc10058b78d468" gracePeriod=300 Dec 03 12:54:42 crc kubenswrapper[5052]: I1203 12:54:42.245303 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-db-sync-p8q2q"] Dec 03 12:54:42 crc kubenswrapper[5052]: I1203 12:54:42.297460 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder-db-sync-2b42x"] Dec 03 12:54:42 crc kubenswrapper[5052]: I1203 12:54:42.318325 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/glance-db-sync-p8q2q"] Dec 03 12:54:42 crc kubenswrapper[5052]: I1203 12:54:42.318643 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ovsdbserver-nb-0" podUID="3a5e94ea-faeb-4202-843a-bd691d8895ce" containerName="ovsdbserver-nb" containerID="cri-o://4a6055cf9cff80734b8d6c28ebdc2f8003fb5d319d561fbaf2f95617ced8483e" gracePeriod=300 Dec 03 12:54:42 crc kubenswrapper[5052]: I1203 12:54:42.319702 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/941d063f-c834-47a1-bc0c-2096b005e09c-operator-scripts\") pod \"neutron6c26-account-delete-d7wtw\" (UID: \"941d063f-c834-47a1-bc0c-2096b005e09c\") " pod="openstack-kuttl-tests/neutron6c26-account-delete-d7wtw" Dec 03 12:54:42 crc kubenswrapper[5052]: I1203 12:54:42.319728 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-gwqrq\" (UniqueName: \"kubernetes.io/projected/941d063f-c834-47a1-bc0c-2096b005e09c-kube-api-access-gwqrq\") pod \"neutron6c26-account-delete-d7wtw\" (UID: \"941d063f-c834-47a1-bc0c-2096b005e09c\") " pod="openstack-kuttl-tests/neutron6c26-account-delete-d7wtw" Dec 03 12:54:42 crc kubenswrapper[5052]: I1203 12:54:42.321058 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/941d063f-c834-47a1-bc0c-2096b005e09c-operator-scripts\") pod \"neutron6c26-account-delete-d7wtw\" (UID: \"941d063f-c834-47a1-bc0c-2096b005e09c\") " pod="openstack-kuttl-tests/neutron6c26-account-delete-d7wtw" Dec 03 12:54:42 crc kubenswrapper[5052]: I1203 12:54:42.346203 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/cinder-db-sync-2b42x"] Dec 03 12:54:42 crc kubenswrapper[5052]: I1203 12:54:42.371901 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/glance7c80-account-delete-knmlm"] Dec 03 12:54:42 crc kubenswrapper[5052]: I1203 12:54:42.373495 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance7c80-account-delete-knmlm" Dec 03 12:54:42 crc kubenswrapper[5052]: I1203 12:54:42.403642 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/placement-7b465d48b8-4bj69"] Dec 03 12:54:42 crc kubenswrapper[5052]: I1203 12:54:42.404100 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/placement-7b465d48b8-4bj69" podUID="8df65ee4-b91d-4ffd-bff2-65e5e6559130" containerName="placement-log" containerID="cri-o://06f463922e62266ed96092189ec4fb9925ca28e41dfc81fbad694a526afdc991" gracePeriod=30 Dec 03 12:54:42 crc kubenswrapper[5052]: I1203 12:54:42.404226 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/placement-7b465d48b8-4bj69" podUID="8df65ee4-b91d-4ffd-bff2-65e5e6559130" containerName="placement-api" containerID="cri-o://4cc0973d6be860027762d01ac994ad7b9ec6e3602b0a54d18cb4bc9da4071fbb" gracePeriod=30 Dec 03 12:54:42 crc kubenswrapper[5052]: I1203 12:54:42.438856 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-gwqrq\" (UniqueName: \"kubernetes.io/projected/941d063f-c834-47a1-bc0c-2096b005e09c-kube-api-access-gwqrq\") pod \"neutron6c26-account-delete-d7wtw\" (UID: \"941d063f-c834-47a1-bc0c-2096b005e09c\") " pod="openstack-kuttl-tests/neutron6c26-account-delete-d7wtw" Dec 03 12:54:42 crc kubenswrapper[5052]: I1203 12:54:42.454579 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance7c80-account-delete-knmlm"] Dec 03 12:54:42 crc kubenswrapper[5052]: I1203 12:54:42.472108 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/swift-ring-rebalance-sfbr9"] Dec 03 12:54:42 crc kubenswrapper[5052]: I1203 12:54:42.524419 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron6c26-account-delete-d7wtw" Dec 03 12:54:42 crc kubenswrapper[5052]: I1203 12:54:42.529677 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-spfp6\" (UniqueName: \"kubernetes.io/projected/79efd225-5ce4-428f-aef9-3f4884931fef-kube-api-access-spfp6\") pod \"glance7c80-account-delete-knmlm\" (UID: \"79efd225-5ce4-428f-aef9-3f4884931fef\") " pod="openstack-kuttl-tests/glance7c80-account-delete-knmlm" Dec 03 12:54:42 crc kubenswrapper[5052]: I1203 12:54:42.531022 5052 generic.go:334] "Generic (PLEG): container finished" podID="3b93bc75-db87-4677-be2f-910d07bb696c" containerID="7cdee3173161e7591d4f168eb2ed4975b7806623d040f607514a1bb1ab4a5542" exitCode=2 Dec 03 12:54:42 crc kubenswrapper[5052]: I1203 12:54:42.531127 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovn-northd-0" event={"ID":"3b93bc75-db87-4677-be2f-910d07bb696c","Type":"ContainerDied","Data":"7cdee3173161e7591d4f168eb2ed4975b7806623d040f607514a1bb1ab4a5542"} Dec 03 12:54:42 crc kubenswrapper[5052]: I1203 12:54:42.540575 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-db-sync-rsmgs"] Dec 03 12:54:42 crc kubenswrapper[5052]: I1203 12:54:42.545790 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/79efd225-5ce4-428f-aef9-3f4884931fef-operator-scripts\") pod \"glance7c80-account-delete-knmlm\" (UID: \"79efd225-5ce4-428f-aef9-3f4884931fef\") " pod="openstack-kuttl-tests/glance7c80-account-delete-knmlm" Dec 03 12:54:42 crc kubenswrapper[5052]: I1203 12:54:42.574279 5052 generic.go:334] "Generic (PLEG): container finished" podID="3a5e94ea-faeb-4202-843a-bd691d8895ce" containerID="5dbfbe34fded1c834b0fb2487d51d6da9334318d91fb67fd92b8f7457b35a22f" exitCode=2 Dec 03 12:54:42 crc kubenswrapper[5052]: I1203 12:54:42.574319 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-nb-0" event={"ID":"3a5e94ea-faeb-4202-843a-bd691d8895ce","Type":"ContainerDied","Data":"5dbfbe34fded1c834b0fb2487d51d6da9334318d91fb67fd92b8f7457b35a22f"} Dec 03 12:54:42 crc kubenswrapper[5052]: I1203 12:54:42.585458 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/cindera2a9-account-delete-pmspg"] Dec 03 12:54:42 crc kubenswrapper[5052]: I1203 12:54:42.586869 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cindera2a9-account-delete-pmspg" Dec 03 12:54:42 crc kubenswrapper[5052]: I1203 12:54:42.663365 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/barbican-db-sync-rsmgs"] Dec 03 12:54:42 crc kubenswrapper[5052]: I1203 12:54:42.707940 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ovsdbserver-sb-0" podUID="d16285c0-7e62-4b3c-b3e8-a62b8abb49cb" containerName="ovsdbserver-sb" containerID="cri-o://468387bd146ee57d9dbb25ad0e76ab64875756a52288e01c0d2ec19a6cddb217" gracePeriod=300 Dec 03 12:54:42 crc kubenswrapper[5052]: I1203 12:54:42.752234 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jv2g8\" (UniqueName: \"kubernetes.io/projected/e8a26759-2926-41cf-a1ec-286fc9962bb3-kube-api-access-jv2g8\") pod \"cindera2a9-account-delete-pmspg\" (UID: \"e8a26759-2926-41cf-a1ec-286fc9962bb3\") " pod="openstack-kuttl-tests/cindera2a9-account-delete-pmspg" Dec 03 12:54:42 crc kubenswrapper[5052]: I1203 12:54:42.752338 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-spfp6\" (UniqueName: \"kubernetes.io/projected/79efd225-5ce4-428f-aef9-3f4884931fef-kube-api-access-spfp6\") pod \"glance7c80-account-delete-knmlm\" (UID: \"79efd225-5ce4-428f-aef9-3f4884931fef\") " pod="openstack-kuttl-tests/glance7c80-account-delete-knmlm" Dec 03 12:54:42 crc kubenswrapper[5052]: I1203 12:54:42.752437 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/79efd225-5ce4-428f-aef9-3f4884931fef-operator-scripts\") pod \"glance7c80-account-delete-knmlm\" (UID: \"79efd225-5ce4-428f-aef9-3f4884931fef\") " pod="openstack-kuttl-tests/glance7c80-account-delete-knmlm" Dec 03 12:54:42 crc kubenswrapper[5052]: I1203 12:54:42.752527 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/e8a26759-2926-41cf-a1ec-286fc9962bb3-operator-scripts\") pod \"cindera2a9-account-delete-pmspg\" (UID: \"e8a26759-2926-41cf-a1ec-286fc9962bb3\") " pod="openstack-kuttl-tests/cindera2a9-account-delete-pmspg" Dec 03 12:54:42 crc kubenswrapper[5052]: I1203 12:54:42.764792 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/swift-ring-rebalance-sfbr9"] Dec 03 12:54:42 crc kubenswrapper[5052]: E1203 12:54:42.787426 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/rabbitmq-cell1-config-data: configmap "rabbitmq-cell1-config-data" not found Dec 03 12:54:42 crc kubenswrapper[5052]: E1203 12:54:42.787505 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/568addd3-4ad3-47c3-adae-693691e7cd40-config-data podName:568addd3-4ad3-47c3-adae-693691e7cd40 nodeName:}" failed. No retries permitted until 2025-12-03 12:54:43.787486583 +0000 UTC m=+5674.375728347 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "config-data" (UniqueName: "kubernetes.io/configmap/568addd3-4ad3-47c3-adae-693691e7cd40-config-data") pod "rabbitmq-cell1-server-0" (UID: "568addd3-4ad3-47c3-adae-693691e7cd40") : configmap "rabbitmq-cell1-config-data" not found Dec 03 12:54:42 crc kubenswrapper[5052]: I1203 12:54:42.799907 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/79efd225-5ce4-428f-aef9-3f4884931fef-operator-scripts\") pod \"glance7c80-account-delete-knmlm\" (UID: \"79efd225-5ce4-428f-aef9-3f4884931fef\") " pod="openstack-kuttl-tests/glance7c80-account-delete-knmlm" Dec 03 12:54:42 crc kubenswrapper[5052]: I1203 12:54:42.807598 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cindera2a9-account-delete-pmspg"] Dec 03 12:54:42 crc kubenswrapper[5052]: I1203 12:54:42.862613 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/e8a26759-2926-41cf-a1ec-286fc9962bb3-operator-scripts\") pod \"cindera2a9-account-delete-pmspg\" (UID: \"e8a26759-2926-41cf-a1ec-286fc9962bb3\") " pod="openstack-kuttl-tests/cindera2a9-account-delete-pmspg" Dec 03 12:54:42 crc kubenswrapper[5052]: I1203 12:54:42.862743 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-jv2g8\" (UniqueName: \"kubernetes.io/projected/e8a26759-2926-41cf-a1ec-286fc9962bb3-kube-api-access-jv2g8\") pod \"cindera2a9-account-delete-pmspg\" (UID: \"e8a26759-2926-41cf-a1ec-286fc9962bb3\") " pod="openstack-kuttl-tests/cindera2a9-account-delete-pmspg" Dec 03 12:54:42 crc kubenswrapper[5052]: E1203 12:54:42.863182 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/rabbitmq-config-data: configmap "rabbitmq-config-data" not found Dec 03 12:54:42 crc kubenswrapper[5052]: E1203 12:54:42.863223 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e73813f5-f97f-41dd-8a79-6196557d4d1a-config-data podName:e73813f5-f97f-41dd-8a79-6196557d4d1a nodeName:}" failed. No retries permitted until 2025-12-03 12:54:44.863210838 +0000 UTC m=+5675.451452602 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "config-data" (UniqueName: "kubernetes.io/configmap/e73813f5-f97f-41dd-8a79-6196557d4d1a-config-data") pod "rabbitmq-server-0" (UID: "e73813f5-f97f-41dd-8a79-6196557d4d1a") : configmap "rabbitmq-config-data" not found Dec 03 12:54:42 crc kubenswrapper[5052]: I1203 12:54:42.863705 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/e8a26759-2926-41cf-a1ec-286fc9962bb3-operator-scripts\") pod \"cindera2a9-account-delete-pmspg\" (UID: \"e8a26759-2926-41cf-a1ec-286fc9962bb3\") " pod="openstack-kuttl-tests/cindera2a9-account-delete-pmspg" Dec 03 12:54:42 crc kubenswrapper[5052]: I1203 12:54:42.881493 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/barbican6e62-account-delete-6skhm"] Dec 03 12:54:42 crc kubenswrapper[5052]: I1203 12:54:42.889029 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican6e62-account-delete-6skhm" Dec 03 12:54:42 crc kubenswrapper[5052]: I1203 12:54:42.913725 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-jv2g8\" (UniqueName: \"kubernetes.io/projected/e8a26759-2926-41cf-a1ec-286fc9962bb3-kube-api-access-jv2g8\") pod \"cindera2a9-account-delete-pmspg\" (UID: \"e8a26759-2926-41cf-a1ec-286fc9962bb3\") " pod="openstack-kuttl-tests/cindera2a9-account-delete-pmspg" Dec 03 12:54:42 crc kubenswrapper[5052]: I1203 12:54:42.942570 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican6e62-account-delete-6skhm"] Dec 03 12:54:42 crc kubenswrapper[5052]: I1203 12:54:42.963024 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-spfp6\" (UniqueName: \"kubernetes.io/projected/79efd225-5ce4-428f-aef9-3f4884931fef-kube-api-access-spfp6\") pod \"glance7c80-account-delete-knmlm\" (UID: \"79efd225-5ce4-428f-aef9-3f4884931fef\") " pod="openstack-kuttl-tests/glance7c80-account-delete-knmlm" Dec 03 12:54:42 crc kubenswrapper[5052]: I1203 12:54:42.965739 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/d9bf91f1-099c-4e72-97cf-1828241aeaac-operator-scripts\") pod \"barbican6e62-account-delete-6skhm\" (UID: \"d9bf91f1-099c-4e72-97cf-1828241aeaac\") " pod="openstack-kuttl-tests/barbican6e62-account-delete-6skhm" Dec 03 12:54:42 crc kubenswrapper[5052]: I1203 12:54:42.973390 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-grf5h\" (UniqueName: \"kubernetes.io/projected/d9bf91f1-099c-4e72-97cf-1828241aeaac-kube-api-access-grf5h\") pod \"barbican6e62-account-delete-6skhm\" (UID: \"d9bf91f1-099c-4e72-97cf-1828241aeaac\") " pod="openstack-kuttl-tests/barbican6e62-account-delete-6skhm" Dec 03 12:54:42 crc kubenswrapper[5052]: I1203 12:54:42.986588 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/swift-storage-0"] Dec 03 12:54:42 crc kubenswrapper[5052]: I1203 12:54:42.987275 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4" containerName="account-server" containerID="cri-o://f162a9aaff5508763fc99c270ac25a9155335aac10652767b1730627c88c0d13" gracePeriod=30 Dec 03 12:54:42 crc kubenswrapper[5052]: I1203 12:54:42.987404 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4" containerName="swift-recon-cron" containerID="cri-o://5f94534eaf62ac523a8208a315b484dcd4fe50a4df83c2d6cfacb2dcdfd740c6" gracePeriod=30 Dec 03 12:54:42 crc kubenswrapper[5052]: I1203 12:54:42.987451 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4" containerName="rsync" containerID="cri-o://7bdd0d115f77d5d7457272b4632a96a1cbaef3a87e7a1296d68acdce465508f1" gracePeriod=30 Dec 03 12:54:42 crc kubenswrapper[5052]: I1203 12:54:42.987495 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4" containerName="object-expirer" containerID="cri-o://62e46329d5f90f5463064339e9bbe330b943da63b752db809975d250c3c5e1ba" gracePeriod=30 Dec 03 12:54:42 crc kubenswrapper[5052]: I1203 12:54:42.987561 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4" containerName="object-updater" containerID="cri-o://71586c32f42d5da42bcbffc73b3fed04e796afb63a8b7d88e60890c23b73e5c0" gracePeriod=30 Dec 03 12:54:42 crc kubenswrapper[5052]: I1203 12:54:42.987610 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4" containerName="object-auditor" containerID="cri-o://44f9d9990406d3045f888637bfcfe158544e8d9bceeb3e129858be5d63d5b0ef" gracePeriod=30 Dec 03 12:54:42 crc kubenswrapper[5052]: I1203 12:54:42.987649 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4" containerName="object-replicator" containerID="cri-o://b47992e8799bd9270966dcef43cbed9c315f16df636f79081295a474d02ec3b9" gracePeriod=30 Dec 03 12:54:42 crc kubenswrapper[5052]: I1203 12:54:42.987697 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4" containerName="object-server" containerID="cri-o://1087472cb2f4b1be538bcf920658b4a2f73a08130ae74a54e44b3322500d6407" gracePeriod=30 Dec 03 12:54:42 crc kubenswrapper[5052]: I1203 12:54:42.987744 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4" containerName="container-updater" containerID="cri-o://5e0e53488533218e7d56486c1da9b5627e233ae560832e3a38ad6cb8cf881504" gracePeriod=30 Dec 03 12:54:42 crc kubenswrapper[5052]: I1203 12:54:42.987789 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4" containerName="container-auditor" containerID="cri-o://79c2a3a0d0be04c4ece62e264a2a1afc1ddae2cbe0714bda786215f70574370c" gracePeriod=30 Dec 03 12:54:42 crc kubenswrapper[5052]: I1203 12:54:42.987839 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4" containerName="container-replicator" containerID="cri-o://e9ef673f7a7a6d771b2dac62256d6ff0ffb0a20ed29ae2461ff57a96282f22a4" gracePeriod=30 Dec 03 12:54:42 crc kubenswrapper[5052]: I1203 12:54:42.987885 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4" containerName="container-server" containerID="cri-o://722f9b2f73e09eeb2d10bcd607cd1d2c2250a2aefb6a7b7fc5b762f09df66896" gracePeriod=30 Dec 03 12:54:42 crc kubenswrapper[5052]: I1203 12:54:42.987931 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4" containerName="account-reaper" containerID="cri-o://93e148f1fa8fb3a45b149f7c61848865b732bc5668fbd9e528494e58a56b6237" gracePeriod=30 Dec 03 12:54:42 crc kubenswrapper[5052]: I1203 12:54:42.987970 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4" containerName="account-auditor" containerID="cri-o://7cd62df542d638c1af171b7f7bdfb4bfa0fa061cd5e93db34b71eef1fee65081" gracePeriod=30 Dec 03 12:54:42 crc kubenswrapper[5052]: I1203 12:54:42.988032 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-storage-0" podUID="5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4" containerName="account-replicator" containerID="cri-o://5606a2414fb1d7623fb77afb101b103f800b56ca86bd25e193447a537a91edbe" gracePeriod=30 Dec 03 12:54:43 crc kubenswrapper[5052]: E1203 12:54:43.021896 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="4a6055cf9cff80734b8d6c28ebdc2f8003fb5d319d561fbaf2f95617ced8483e" cmd=["/usr/bin/pidof","ovsdb-server"] Dec 03 12:54:43 crc kubenswrapper[5052]: I1203 12:54:43.068313 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/neutron-698d8977cd-ppz9n"] Dec 03 12:54:43 crc kubenswrapper[5052]: I1203 12:54:43.068691 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/neutron-698d8977cd-ppz9n" podUID="e2225738-b0d1-4306-bbee-a61dd9558c80" containerName="neutron-api" containerID="cri-o://b7a1da6db38db2f0ed2c6f73c89998af0cc42716d17e3513590eabdf3cc5aa71" gracePeriod=30 Dec 03 12:54:43 crc kubenswrapper[5052]: I1203 12:54:43.069310 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/neutron-698d8977cd-ppz9n" podUID="e2225738-b0d1-4306-bbee-a61dd9558c80" containerName="neutron-httpd" containerID="cri-o://89798d3d34dfd5e2ea4266a0c6cf4bb6c1e2078d3038566fe9362ddc07ef9734" gracePeriod=30 Dec 03 12:54:43 crc kubenswrapper[5052]: I1203 12:54:43.075090 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/d9bf91f1-099c-4e72-97cf-1828241aeaac-operator-scripts\") pod \"barbican6e62-account-delete-6skhm\" (UID: \"d9bf91f1-099c-4e72-97cf-1828241aeaac\") " pod="openstack-kuttl-tests/barbican6e62-account-delete-6skhm" Dec 03 12:54:43 crc kubenswrapper[5052]: I1203 12:54:43.075210 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-grf5h\" (UniqueName: \"kubernetes.io/projected/d9bf91f1-099c-4e72-97cf-1828241aeaac-kube-api-access-grf5h\") pod \"barbican6e62-account-delete-6skhm\" (UID: \"d9bf91f1-099c-4e72-97cf-1828241aeaac\") " pod="openstack-kuttl-tests/barbican6e62-account-delete-6skhm" Dec 03 12:54:43 crc kubenswrapper[5052]: I1203 12:54:43.076835 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/d9bf91f1-099c-4e72-97cf-1828241aeaac-operator-scripts\") pod \"barbican6e62-account-delete-6skhm\" (UID: \"d9bf91f1-099c-4e72-97cf-1828241aeaac\") " pod="openstack-kuttl-tests/barbican6e62-account-delete-6skhm" Dec 03 12:54:43 crc kubenswrapper[5052]: I1203 12:54:43.151963 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 12:54:43 crc kubenswrapper[5052]: I1203 12:54:43.152503 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/cinder-scheduler-0" podUID="2b0bf9ae-2d1d-48da-8c4b-2973b47985a3" containerName="cinder-scheduler" containerID="cri-o://34ec2e3b276084c3a6169386d90460245b79be1c0922971153b1615fd7e2d686" gracePeriod=30 Dec 03 12:54:43 crc kubenswrapper[5052]: I1203 12:54:43.153169 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/cinder-scheduler-0" podUID="2b0bf9ae-2d1d-48da-8c4b-2973b47985a3" containerName="probe" containerID="cri-o://78894a79ffab8e64787cde1b1f24aa5fa09f830cec10c1b466b6619c2b33ae10" gracePeriod=30 Dec 03 12:54:43 crc kubenswrapper[5052]: E1203 12:54:43.157267 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="4a6055cf9cff80734b8d6c28ebdc2f8003fb5d319d561fbaf2f95617ced8483e" cmd=["/usr/bin/pidof","ovsdb-server"] Dec 03 12:54:43 crc kubenswrapper[5052]: E1203 12:54:43.163960 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of 4a6055cf9cff80734b8d6c28ebdc2f8003fb5d319d561fbaf2f95617ced8483e is running failed: container process not found" containerID="4a6055cf9cff80734b8d6c28ebdc2f8003fb5d319d561fbaf2f95617ced8483e" cmd=["/usr/bin/pidof","ovsdb-server"] Dec 03 12:54:43 crc kubenswrapper[5052]: E1203 12:54:43.164021 5052 prober.go:104] "Probe errored" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of 4a6055cf9cff80734b8d6c28ebdc2f8003fb5d319d561fbaf2f95617ced8483e is running failed: container process not found" probeType="Readiness" pod="openstack-kuttl-tests/ovsdbserver-nb-0" podUID="3a5e94ea-faeb-4202-843a-bd691d8895ce" containerName="ovsdbserver-nb" Dec 03 12:54:43 crc kubenswrapper[5052]: I1203 12:54:43.165099 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:54:43 crc kubenswrapper[5052]: I1203 12:54:43.165342 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-external-api-0" podUID="cfc68960-6006-486c-bc03-848875d3bf5b" containerName="glance-log" containerID="cri-o://e4a30a41f3f5c0bc5325054597c83dfd7f01768f25444ec3d94e71d8f4c168ed" gracePeriod=30 Dec 03 12:54:43 crc kubenswrapper[5052]: I1203 12:54:43.165460 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-external-api-0" podUID="cfc68960-6006-486c-bc03-848875d3bf5b" containerName="glance-httpd" containerID="cri-o://21405273563ddcf40333c0b6ddfcd119d733ef5beb5a7399909f33a285ac39e7" gracePeriod=30 Dec 03 12:54:43 crc kubenswrapper[5052]: I1203 12:54:43.177750 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 12:54:43 crc kubenswrapper[5052]: I1203 12:54:43.178035 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/cinder-api-0" podUID="ed40ab59-7f38-4b2a-ad68-85592fe22ef2" containerName="cinder-api-log" containerID="cri-o://717a9f8ebf7eb71e44fa8c9d63200534cfb506f60207616c8ff568f62f9287a8" gracePeriod=30 Dec 03 12:54:43 crc kubenswrapper[5052]: I1203 12:54:43.178330 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/cinder-api-0" podUID="ed40ab59-7f38-4b2a-ad68-85592fe22ef2" containerName="cinder-api" containerID="cri-o://0d0807a346577955962a640bd5fe5f8e366a538b40576ef0017fa65770ea3118" gracePeriod=30 Dec 03 12:54:43 crc kubenswrapper[5052]: I1203 12:54:43.195238 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cindera2a9-account-delete-pmspg" Dec 03 12:54:43 crc kubenswrapper[5052]: I1203 12:54:43.197091 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/swift-proxy-5479cb94c8-q2snc"] Dec 03 12:54:43 crc kubenswrapper[5052]: I1203 12:54:43.197286 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-proxy-5479cb94c8-q2snc" podUID="eefe83f8-0235-4b16-aefb-5ec7b2d92fcf" containerName="proxy-httpd" containerID="cri-o://ac795f902a20e66db9af6d237bc550efac4a63fdac0b163e8ceb8138a8c4a16f" gracePeriod=30 Dec 03 12:54:43 crc kubenswrapper[5052]: I1203 12:54:43.198143 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-grf5h\" (UniqueName: \"kubernetes.io/projected/d9bf91f1-099c-4e72-97cf-1828241aeaac-kube-api-access-grf5h\") pod \"barbican6e62-account-delete-6skhm\" (UID: \"d9bf91f1-099c-4e72-97cf-1828241aeaac\") " pod="openstack-kuttl-tests/barbican6e62-account-delete-6skhm" Dec 03 12:54:43 crc kubenswrapper[5052]: I1203 12:54:43.198244 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/swift-proxy-5479cb94c8-q2snc" podUID="eefe83f8-0235-4b16-aefb-5ec7b2d92fcf" containerName="proxy-server" containerID="cri-o://29117e54764872df757ac95e94dbab057dd9c2d6f5908051e97419768ed8628a" gracePeriod=30 Dec 03 12:54:43 crc kubenswrapper[5052]: I1203 12:54:43.206884 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance7c80-account-delete-knmlm" Dec 03 12:54:43 crc kubenswrapper[5052]: I1203 12:54:43.214901 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/rabbitmq-server-0"] Dec 03 12:54:43 crc kubenswrapper[5052]: I1203 12:54:43.239563 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-shdj6"] Dec 03 12:54:43 crc kubenswrapper[5052]: I1203 12:54:43.274486 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican6e62-account-delete-6skhm" Dec 03 12:54:43 crc kubenswrapper[5052]: I1203 12:54:43.288255 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-shdj6" Dec 03 12:54:43 crc kubenswrapper[5052]: I1203 12:54:43.293116 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-shdj6"] Dec 03 12:54:43 crc kubenswrapper[5052]: I1203 12:54:43.312979 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/novacell10a46-account-delete-h8x7f"] Dec 03 12:54:43 crc kubenswrapper[5052]: I1203 12:54:43.351152 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell0-cell-mapping-cnck7"] Dec 03 12:54:43 crc kubenswrapper[5052]: I1203 12:54:43.351342 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/novacell10a46-account-delete-h8x7f" Dec 03 12:54:43 crc kubenswrapper[5052]: I1203 12:54:43.379175 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell1-cell-mapping-t6brm"] Dec 03 12:54:43 crc kubenswrapper[5052]: I1203 12:54:43.448343 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/6d56c6dc-38c9-42cf-ac6d-04ba49860d31-operator-scripts\") pod \"novacell10a46-account-delete-h8x7f\" (UID: \"6d56c6dc-38c9-42cf-ac6d-04ba49860d31\") " pod="openstack-kuttl-tests/novacell10a46-account-delete-h8x7f" Dec 03 12:54:43 crc kubenswrapper[5052]: I1203 12:54:43.448668 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/7bed2e44-93da-465d-9e10-6879ff1c82f5-config\") pod \"dnsmasq-dnsmasq-84b9f45d47-shdj6\" (UID: \"7bed2e44-93da-465d-9e10-6879ff1c82f5\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-shdj6" Dec 03 12:54:43 crc kubenswrapper[5052]: I1203 12:54:43.448819 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/7bed2e44-93da-465d-9e10-6879ff1c82f5-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-84b9f45d47-shdj6\" (UID: \"7bed2e44-93da-465d-9e10-6879ff1c82f5\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-shdj6" Dec 03 12:54:43 crc kubenswrapper[5052]: I1203 12:54:43.448888 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hs7rk\" (UniqueName: \"kubernetes.io/projected/6d56c6dc-38c9-42cf-ac6d-04ba49860d31-kube-api-access-hs7rk\") pod \"novacell10a46-account-delete-h8x7f\" (UID: \"6d56c6dc-38c9-42cf-ac6d-04ba49860d31\") " pod="openstack-kuttl-tests/novacell10a46-account-delete-h8x7f" Dec 03 12:54:43 crc kubenswrapper[5052]: I1203 12:54:43.449021 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2cd75\" (UniqueName: \"kubernetes.io/projected/7bed2e44-93da-465d-9e10-6879ff1c82f5-kube-api-access-2cd75\") pod \"dnsmasq-dnsmasq-84b9f45d47-shdj6\" (UID: \"7bed2e44-93da-465d-9e10-6879ff1c82f5\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-shdj6" Dec 03 12:54:43 crc kubenswrapper[5052]: I1203 12:54:43.458578 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell1-cell-mapping-t6brm"] Dec 03 12:54:43 crc kubenswrapper[5052]: I1203 12:54:43.469598 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/novacell10a46-account-delete-h8x7f"] Dec 03 12:54:43 crc kubenswrapper[5052]: I1203 12:54:43.479593 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell0-cell-mapping-cnck7"] Dec 03 12:54:43 crc kubenswrapper[5052]: I1203 12:54:43.493240 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/novaapi7066-account-delete-whswm"] Dec 03 12:54:43 crc kubenswrapper[5052]: I1203 12:54:43.494739 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/novaapi7066-account-delete-whswm" Dec 03 12:54:43 crc kubenswrapper[5052]: I1203 12:54:43.524819 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:54:43 crc kubenswrapper[5052]: I1203 12:54:43.524864 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/novaapi7066-account-delete-whswm"] Dec 03 12:54:43 crc kubenswrapper[5052]: I1203 12:54:43.525307 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-internal-api-0" podUID="981c3156-805a-44d1-b4ec-f688a8bc3cf8" containerName="glance-log" containerID="cri-o://691c45bd03a54c99bcffac732eb2f453f8046a69209f84bde56957c8b1cd60f9" gracePeriod=30 Dec 03 12:54:43 crc kubenswrapper[5052]: I1203 12:54:43.525459 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/glance-default-internal-api-0" podUID="981c3156-805a-44d1-b4ec-f688a8bc3cf8" containerName="glance-httpd" containerID="cri-o://91f773e54ad4ec35e1b76f707be75a9985aeb5b002636c9dd379bb836c31c220" gracePeriod=30 Dec 03 12:54:43 crc kubenswrapper[5052]: I1203 12:54:43.528167 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-keystone-listener-7b66cf7bb6-fsc4x"] Dec 03 12:54:43 crc kubenswrapper[5052]: I1203 12:54:43.528376 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/barbican-keystone-listener-7b66cf7bb6-fsc4x" podUID="020bfa4c-3dba-43d2-9ac2-0bbbae660709" containerName="barbican-keystone-listener-log" containerID="cri-o://27212e7a0c6918bbf263320719b8b0731ba8a5d005b2aed71d83eab909ffb6f0" gracePeriod=30 Dec 03 12:54:43 crc kubenswrapper[5052]: I1203 12:54:43.528441 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/barbican-keystone-listener-7b66cf7bb6-fsc4x" podUID="020bfa4c-3dba-43d2-9ac2-0bbbae660709" containerName="barbican-keystone-listener" containerID="cri-o://bab03b01c66d5661abe6c60c8c81496efcefb2ea684c5751d1ee1113198a69ca" gracePeriod=30 Dec 03 12:54:43 crc kubenswrapper[5052]: I1203 12:54:43.551630 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/7bed2e44-93da-465d-9e10-6879ff1c82f5-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-84b9f45d47-shdj6\" (UID: \"7bed2e44-93da-465d-9e10-6879ff1c82f5\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-shdj6" Dec 03 12:54:43 crc kubenswrapper[5052]: I1203 12:54:43.551727 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-hs7rk\" (UniqueName: \"kubernetes.io/projected/6d56c6dc-38c9-42cf-ac6d-04ba49860d31-kube-api-access-hs7rk\") pod \"novacell10a46-account-delete-h8x7f\" (UID: \"6d56c6dc-38c9-42cf-ac6d-04ba49860d31\") " pod="openstack-kuttl-tests/novacell10a46-account-delete-h8x7f" Dec 03 12:54:43 crc kubenswrapper[5052]: I1203 12:54:43.551781 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-2cd75\" (UniqueName: \"kubernetes.io/projected/7bed2e44-93da-465d-9e10-6879ff1c82f5-kube-api-access-2cd75\") pod \"dnsmasq-dnsmasq-84b9f45d47-shdj6\" (UID: \"7bed2e44-93da-465d-9e10-6879ff1c82f5\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-shdj6" Dec 03 12:54:43 crc kubenswrapper[5052]: I1203 12:54:43.551817 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/6d56c6dc-38c9-42cf-ac6d-04ba49860d31-operator-scripts\") pod \"novacell10a46-account-delete-h8x7f\" (UID: \"6d56c6dc-38c9-42cf-ac6d-04ba49860d31\") " pod="openstack-kuttl-tests/novacell10a46-account-delete-h8x7f" Dec 03 12:54:43 crc kubenswrapper[5052]: I1203 12:54:43.551911 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/7bed2e44-93da-465d-9e10-6879ff1c82f5-config\") pod \"dnsmasq-dnsmasq-84b9f45d47-shdj6\" (UID: \"7bed2e44-93da-465d-9e10-6879ff1c82f5\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-shdj6" Dec 03 12:54:43 crc kubenswrapper[5052]: E1203 12:54:43.552870 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-cell1-scripts: configmap "openstack-cell1-scripts" not found Dec 03 12:54:43 crc kubenswrapper[5052]: E1203 12:54:43.552913 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6d56c6dc-38c9-42cf-ac6d-04ba49860d31-operator-scripts podName:6d56c6dc-38c9-42cf-ac6d-04ba49860d31 nodeName:}" failed. No retries permitted until 2025-12-03 12:54:44.052896182 +0000 UTC m=+5674.641137946 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/6d56c6dc-38c9-42cf-ac6d-04ba49860d31-operator-scripts") pod "novacell10a46-account-delete-h8x7f" (UID: "6d56c6dc-38c9-42cf-ac6d-04ba49860d31") : configmap "openstack-cell1-scripts" not found Dec 03 12:54:43 crc kubenswrapper[5052]: I1203 12:54:43.553471 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/7bed2e44-93da-465d-9e10-6879ff1c82f5-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-84b9f45d47-shdj6\" (UID: \"7bed2e44-93da-465d-9e10-6879ff1c82f5\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-shdj6" Dec 03 12:54:43 crc kubenswrapper[5052]: I1203 12:54:43.553813 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/7bed2e44-93da-465d-9e10-6879ff1c82f5-config\") pod \"dnsmasq-dnsmasq-84b9f45d47-shdj6\" (UID: \"7bed2e44-93da-465d-9e10-6879ff1c82f5\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-shdj6" Dec 03 12:54:43 crc kubenswrapper[5052]: I1203 12:54:43.560028 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/novacell02629-account-delete-wsshh"] Dec 03 12:54:43 crc kubenswrapper[5052]: I1203 12:54:43.561338 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/novacell02629-account-delete-wsshh" Dec 03 12:54:43 crc kubenswrapper[5052]: I1203 12:54:43.564943 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/rabbitmq-server-0" podUID="e73813f5-f97f-41dd-8a79-6196557d4d1a" containerName="rabbitmq" containerID="cri-o://db4b319560fefd949092d7f9bbfd3ae7aab4d1e6d1c4c90e729f406df14b8b79" gracePeriod=604800 Dec 03 12:54:43 crc kubenswrapper[5052]: E1203 12:54:43.565857 5052 projected.go:194] Error preparing data for projected volume kube-api-access-hs7rk for pod openstack-kuttl-tests/novacell10a46-account-delete-h8x7f: failed to fetch token: serviceaccounts "galera-openstack-cell1" not found Dec 03 12:54:43 crc kubenswrapper[5052]: E1203 12:54:43.565922 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6d56c6dc-38c9-42cf-ac6d-04ba49860d31-kube-api-access-hs7rk podName:6d56c6dc-38c9-42cf-ac6d-04ba49860d31 nodeName:}" failed. No retries permitted until 2025-12-03 12:54:44.065898854 +0000 UTC m=+5674.654140618 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-hs7rk" (UniqueName: "kubernetes.io/projected/6d56c6dc-38c9-42cf-ac6d-04ba49860d31-kube-api-access-hs7rk") pod "novacell10a46-account-delete-h8x7f" (UID: "6d56c6dc-38c9-42cf-ac6d-04ba49860d31") : failed to fetch token: serviceaccounts "galera-openstack-cell1" not found Dec 03 12:54:43 crc kubenswrapper[5052]: I1203 12:54:43.597739 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-api-76dc68d888-zmmcs"] Dec 03 12:54:43 crc kubenswrapper[5052]: I1203 12:54:43.597959 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/barbican-api-76dc68d888-zmmcs" podUID="25eead1e-d17c-4b36-8a9f-ef563189d53b" containerName="barbican-api-log" containerID="cri-o://4c38a9546e116479167e49dc563abec492318ccfbe7e54b5db501a8e8665f5aa" gracePeriod=30 Dec 03 12:54:43 crc kubenswrapper[5052]: I1203 12:54:43.598326 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/barbican-api-76dc68d888-zmmcs" podUID="25eead1e-d17c-4b36-8a9f-ef563189d53b" containerName="barbican-api" containerID="cri-o://018816f9a78e7b2315d8c4e733edf99eeec2819f8f83cbe3fa3669341863f8d1" gracePeriod=30 Dec 03 12:54:43 crc kubenswrapper[5052]: I1203 12:54:43.652356 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-2cd75\" (UniqueName: \"kubernetes.io/projected/7bed2e44-93da-465d-9e10-6879ff1c82f5-kube-api-access-2cd75\") pod \"dnsmasq-dnsmasq-84b9f45d47-shdj6\" (UID: \"7bed2e44-93da-465d-9e10-6879ff1c82f5\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-shdj6" Dec 03 12:54:43 crc kubenswrapper[5052]: I1203 12:54:43.666773 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/1b7ef5eb-406e-4e2c-9d9c-d763919f56fd-operator-scripts\") pod \"novacell02629-account-delete-wsshh\" (UID: \"1b7ef5eb-406e-4e2c-9d9c-d763919f56fd\") " pod="openstack-kuttl-tests/novacell02629-account-delete-wsshh" Dec 03 12:54:43 crc kubenswrapper[5052]: I1203 12:54:43.673876 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s76qn\" (UniqueName: \"kubernetes.io/projected/1b7ef5eb-406e-4e2c-9d9c-d763919f56fd-kube-api-access-s76qn\") pod \"novacell02629-account-delete-wsshh\" (UID: \"1b7ef5eb-406e-4e2c-9d9c-d763919f56fd\") " pod="openstack-kuttl-tests/novacell02629-account-delete-wsshh" Dec 03 12:54:43 crc kubenswrapper[5052]: I1203 12:54:43.674150 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/13e7e748-92f4-4255-a8eb-aa5a6faf361f-operator-scripts\") pod \"novaapi7066-account-delete-whswm\" (UID: \"13e7e748-92f4-4255-a8eb-aa5a6faf361f\") " pod="openstack-kuttl-tests/novaapi7066-account-delete-whswm" Dec 03 12:54:43 crc kubenswrapper[5052]: I1203 12:54:43.674304 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5vk45\" (UniqueName: \"kubernetes.io/projected/13e7e748-92f4-4255-a8eb-aa5a6faf361f-kube-api-access-5vk45\") pod \"novaapi7066-account-delete-whswm\" (UID: \"13e7e748-92f4-4255-a8eb-aa5a6faf361f\") " pod="openstack-kuttl-tests/novaapi7066-account-delete-whswm" Dec 03 12:54:43 crc kubenswrapper[5052]: I1203 12:54:43.705699 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2a7f3d6f-9d21-4071-a9eb-644a23bd704c" path="/var/lib/kubelet/pods/2a7f3d6f-9d21-4071-a9eb-644a23bd704c/volumes" Dec 03 12:54:43 crc kubenswrapper[5052]: I1203 12:54:43.706249 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2fab7846-f3ad-4673-b813-36c20d4b7611" path="/var/lib/kubelet/pods/2fab7846-f3ad-4673-b813-36c20d4b7611/volumes" Dec 03 12:54:43 crc kubenswrapper[5052]: I1203 12:54:43.706772 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="337505df-59b3-4007-9855-05616985bfc6" path="/var/lib/kubelet/pods/337505df-59b3-4007-9855-05616985bfc6/volumes" Dec 03 12:54:43 crc kubenswrapper[5052]: I1203 12:54:43.712232 5052 generic.go:334] "Generic (PLEG): container finished" podID="5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4" containerID="62e46329d5f90f5463064339e9bbe330b943da63b752db809975d250c3c5e1ba" exitCode=0 Dec 03 12:54:43 crc kubenswrapper[5052]: I1203 12:54:43.712266 5052 generic.go:334] "Generic (PLEG): container finished" podID="5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4" containerID="71586c32f42d5da42bcbffc73b3fed04e796afb63a8b7d88e60890c23b73e5c0" exitCode=0 Dec 03 12:54:43 crc kubenswrapper[5052]: I1203 12:54:43.712272 5052 generic.go:334] "Generic (PLEG): container finished" podID="5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4" containerID="44f9d9990406d3045f888637bfcfe158544e8d9bceeb3e129858be5d63d5b0ef" exitCode=0 Dec 03 12:54:43 crc kubenswrapper[5052]: I1203 12:54:43.712279 5052 generic.go:334] "Generic (PLEG): container finished" podID="5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4" containerID="b47992e8799bd9270966dcef43cbed9c315f16df636f79081295a474d02ec3b9" exitCode=0 Dec 03 12:54:43 crc kubenswrapper[5052]: I1203 12:54:43.712287 5052 generic.go:334] "Generic (PLEG): container finished" podID="5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4" containerID="5e0e53488533218e7d56486c1da9b5627e233ae560832e3a38ad6cb8cf881504" exitCode=0 Dec 03 12:54:43 crc kubenswrapper[5052]: I1203 12:54:43.712293 5052 generic.go:334] "Generic (PLEG): container finished" podID="5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4" containerID="79c2a3a0d0be04c4ece62e264a2a1afc1ddae2cbe0714bda786215f70574370c" exitCode=0 Dec 03 12:54:43 crc kubenswrapper[5052]: I1203 12:54:43.712299 5052 generic.go:334] "Generic (PLEG): container finished" podID="5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4" containerID="e9ef673f7a7a6d771b2dac62256d6ff0ffb0a20ed29ae2461ff57a96282f22a4" exitCode=0 Dec 03 12:54:43 crc kubenswrapper[5052]: I1203 12:54:43.712305 5052 generic.go:334] "Generic (PLEG): container finished" podID="5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4" containerID="722f9b2f73e09eeb2d10bcd607cd1d2c2250a2aefb6a7b7fc5b762f09df66896" exitCode=0 Dec 03 12:54:43 crc kubenswrapper[5052]: I1203 12:54:43.712313 5052 generic.go:334] "Generic (PLEG): container finished" podID="5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4" containerID="93e148f1fa8fb3a45b149f7c61848865b732bc5668fbd9e528494e58a56b6237" exitCode=0 Dec 03 12:54:43 crc kubenswrapper[5052]: I1203 12:54:43.712318 5052 generic.go:334] "Generic (PLEG): container finished" podID="5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4" containerID="7cd62df542d638c1af171b7f7bdfb4bfa0fa061cd5e93db34b71eef1fee65081" exitCode=0 Dec 03 12:54:43 crc kubenswrapper[5052]: I1203 12:54:43.712324 5052 generic.go:334] "Generic (PLEG): container finished" podID="5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4" containerID="5606a2414fb1d7623fb77afb101b103f800b56ca86bd25e193447a537a91edbe" exitCode=0 Dec 03 12:54:43 crc kubenswrapper[5052]: I1203 12:54:43.715702 5052 generic.go:334] "Generic (PLEG): container finished" podID="cfc68960-6006-486c-bc03-848875d3bf5b" containerID="e4a30a41f3f5c0bc5325054597c83dfd7f01768f25444ec3d94e71d8f4c168ed" exitCode=143 Dec 03 12:54:43 crc kubenswrapper[5052]: I1203 12:54:43.719565 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="468756d2-d46f-411a-a8ce-0c160d58c278" path="/var/lib/kubelet/pods/468756d2-d46f-411a-a8ce-0c160d58c278/volumes" Dec 03 12:54:43 crc kubenswrapper[5052]: I1203 12:54:43.719947 5052 generic.go:334] "Generic (PLEG): container finished" podID="eefe83f8-0235-4b16-aefb-5ec7b2d92fcf" containerID="ac795f902a20e66db9af6d237bc550efac4a63fdac0b163e8ceb8138a8c4a16f" exitCode=0 Dec 03 12:54:43 crc kubenswrapper[5052]: I1203 12:54:43.720632 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5282791b-cdf2-4e89-a273-89bc6550f531" path="/var/lib/kubelet/pods/5282791b-cdf2-4e89-a273-89bc6550f531/volumes" Dec 03 12:54:43 crc kubenswrapper[5052]: I1203 12:54:43.721326 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8212a97d-b76a-4b60-b16b-914a4bbe5b9d" path="/var/lib/kubelet/pods/8212a97d-b76a-4b60-b16b-914a4bbe5b9d/volumes" Dec 03 12:54:43 crc kubenswrapper[5052]: I1203 12:54:43.721986 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="efa403f3-536d-4c75-adbe-79ed07819dff" path="/var/lib/kubelet/pods/efa403f3-536d-4c75-adbe-79ed07819dff/volumes" Dec 03 12:54:43 crc kubenswrapper[5052]: I1203 12:54:43.730595 5052 generic.go:334] "Generic (PLEG): container finished" podID="c4b7bd04-32be-43c6-82f1-968f2e368b22" containerID="863f9a5dead36dfcd4e960256f81adece8f398e82c1cc6eb7b8c1e26766d0633" exitCode=137 Dec 03 12:54:43 crc kubenswrapper[5052]: I1203 12:54:43.732787 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ffe1f27e-ba05-4da0-a02b-a73b3bf63bc3" path="/var/lib/kubelet/pods/ffe1f27e-ba05-4da0-a02b-a73b3bf63bc3/volumes" Dec 03 12:54:43 crc kubenswrapper[5052]: I1203 12:54:43.733499 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4","Type":"ContainerDied","Data":"62e46329d5f90f5463064339e9bbe330b943da63b752db809975d250c3c5e1ba"} Dec 03 12:54:43 crc kubenswrapper[5052]: I1203 12:54:43.733548 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4","Type":"ContainerDied","Data":"71586c32f42d5da42bcbffc73b3fed04e796afb63a8b7d88e60890c23b73e5c0"} Dec 03 12:54:43 crc kubenswrapper[5052]: I1203 12:54:43.733565 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4","Type":"ContainerDied","Data":"44f9d9990406d3045f888637bfcfe158544e8d9bceeb3e129858be5d63d5b0ef"} Dec 03 12:54:43 crc kubenswrapper[5052]: I1203 12:54:43.733577 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4","Type":"ContainerDied","Data":"b47992e8799bd9270966dcef43cbed9c315f16df636f79081295a474d02ec3b9"} Dec 03 12:54:43 crc kubenswrapper[5052]: I1203 12:54:43.733589 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4","Type":"ContainerDied","Data":"5e0e53488533218e7d56486c1da9b5627e233ae560832e3a38ad6cb8cf881504"} Dec 03 12:54:43 crc kubenswrapper[5052]: I1203 12:54:43.733601 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4","Type":"ContainerDied","Data":"79c2a3a0d0be04c4ece62e264a2a1afc1ddae2cbe0714bda786215f70574370c"} Dec 03 12:54:43 crc kubenswrapper[5052]: I1203 12:54:43.733611 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4","Type":"ContainerDied","Data":"e9ef673f7a7a6d771b2dac62256d6ff0ffb0a20ed29ae2461ff57a96282f22a4"} Dec 03 12:54:43 crc kubenswrapper[5052]: I1203 12:54:43.733625 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4","Type":"ContainerDied","Data":"722f9b2f73e09eeb2d10bcd607cd1d2c2250a2aefb6a7b7fc5b762f09df66896"} Dec 03 12:54:43 crc kubenswrapper[5052]: I1203 12:54:43.733637 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4","Type":"ContainerDied","Data":"93e148f1fa8fb3a45b149f7c61848865b732bc5668fbd9e528494e58a56b6237"} Dec 03 12:54:43 crc kubenswrapper[5052]: I1203 12:54:43.733659 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4","Type":"ContainerDied","Data":"7cd62df542d638c1af171b7f7bdfb4bfa0fa061cd5e93db34b71eef1fee65081"} Dec 03 12:54:43 crc kubenswrapper[5052]: I1203 12:54:43.733671 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4","Type":"ContainerDied","Data":"5606a2414fb1d7623fb77afb101b103f800b56ca86bd25e193447a537a91edbe"} Dec 03 12:54:43 crc kubenswrapper[5052]: I1203 12:54:43.733683 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"cfc68960-6006-486c-bc03-848875d3bf5b","Type":"ContainerDied","Data":"e4a30a41f3f5c0bc5325054597c83dfd7f01768f25444ec3d94e71d8f4c168ed"} Dec 03 12:54:43 crc kubenswrapper[5052]: I1203 12:54:43.733704 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-proxy-5479cb94c8-q2snc" event={"ID":"eefe83f8-0235-4b16-aefb-5ec7b2d92fcf","Type":"ContainerDied","Data":"ac795f902a20e66db9af6d237bc550efac4a63fdac0b163e8ceb8138a8c4a16f"} Dec 03 12:54:43 crc kubenswrapper[5052]: I1203 12:54:43.734552 5052 generic.go:334] "Generic (PLEG): container finished" podID="ed40ab59-7f38-4b2a-ad68-85592fe22ef2" containerID="717a9f8ebf7eb71e44fa8c9d63200534cfb506f60207616c8ff568f62f9287a8" exitCode=143 Dec 03 12:54:43 crc kubenswrapper[5052]: I1203 12:54:43.734597 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"ed40ab59-7f38-4b2a-ad68-85592fe22ef2","Type":"ContainerDied","Data":"717a9f8ebf7eb71e44fa8c9d63200534cfb506f60207616c8ff568f62f9287a8"} Dec 03 12:54:43 crc kubenswrapper[5052]: I1203 12:54:43.736125 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_ovsdbserver-sb-0_d16285c0-7e62-4b3c-b3e8-a62b8abb49cb/ovsdbserver-sb/0.log" Dec 03 12:54:43 crc kubenswrapper[5052]: I1203 12:54:43.736151 5052 generic.go:334] "Generic (PLEG): container finished" podID="d16285c0-7e62-4b3c-b3e8-a62b8abb49cb" containerID="89cea0c9094ed2fb8bb3973f66f25388cbb772d78cc9dc394bfc10058b78d468" exitCode=2 Dec 03 12:54:43 crc kubenswrapper[5052]: I1203 12:54:43.736162 5052 generic.go:334] "Generic (PLEG): container finished" podID="d16285c0-7e62-4b3c-b3e8-a62b8abb49cb" containerID="468387bd146ee57d9dbb25ad0e76ab64875756a52288e01c0d2ec19a6cddb217" exitCode=143 Dec 03 12:54:43 crc kubenswrapper[5052]: I1203 12:54:43.736202 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-sb-0" event={"ID":"d16285c0-7e62-4b3c-b3e8-a62b8abb49cb","Type":"ContainerDied","Data":"89cea0c9094ed2fb8bb3973f66f25388cbb772d78cc9dc394bfc10058b78d468"} Dec 03 12:54:43 crc kubenswrapper[5052]: I1203 12:54:43.736217 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-sb-0" event={"ID":"d16285c0-7e62-4b3c-b3e8-a62b8abb49cb","Type":"ContainerDied","Data":"468387bd146ee57d9dbb25ad0e76ab64875756a52288e01c0d2ec19a6cddb217"} Dec 03 12:54:43 crc kubenswrapper[5052]: I1203 12:54:43.737541 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_ovsdbserver-nb-0_3a5e94ea-faeb-4202-843a-bd691d8895ce/ovsdbserver-nb/0.log" Dec 03 12:54:43 crc kubenswrapper[5052]: I1203 12:54:43.737569 5052 generic.go:334] "Generic (PLEG): container finished" podID="3a5e94ea-faeb-4202-843a-bd691d8895ce" containerID="4a6055cf9cff80734b8d6c28ebdc2f8003fb5d319d561fbaf2f95617ced8483e" exitCode=143 Dec 03 12:54:43 crc kubenswrapper[5052]: I1203 12:54:43.737600 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-nb-0" event={"ID":"3a5e94ea-faeb-4202-843a-bd691d8895ce","Type":"ContainerDied","Data":"4a6055cf9cff80734b8d6c28ebdc2f8003fb5d319d561fbaf2f95617ced8483e"} Dec 03 12:54:43 crc kubenswrapper[5052]: I1203 12:54:43.738592 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/novacell02629-account-delete-wsshh"] Dec 03 12:54:43 crc kubenswrapper[5052]: I1203 12:54:43.738994 5052 generic.go:334] "Generic (PLEG): container finished" podID="8df65ee4-b91d-4ffd-bff2-65e5e6559130" containerID="06f463922e62266ed96092189ec4fb9925ca28e41dfc81fbad694a526afdc991" exitCode=143 Dec 03 12:54:43 crc kubenswrapper[5052]: I1203 12:54:43.739014 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-7b465d48b8-4bj69" event={"ID":"8df65ee4-b91d-4ffd-bff2-65e5e6559130","Type":"ContainerDied","Data":"06f463922e62266ed96092189ec4fb9925ca28e41dfc81fbad694a526afdc991"} Dec 03 12:54:43 crc kubenswrapper[5052]: I1203 12:54:43.785645 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-worker-b7f5dfb6f-pj6bx"] Dec 03 12:54:43 crc kubenswrapper[5052]: I1203 12:54:43.785942 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/barbican-worker-b7f5dfb6f-pj6bx" podUID="b2933134-7226-423a-8120-3148845b808a" containerName="barbican-worker-log" containerID="cri-o://c45182c71bc38ac598875105b0f4d0cd30926ee88268d53941e68205eaeb487b" gracePeriod=30 Dec 03 12:54:43 crc kubenswrapper[5052]: I1203 12:54:43.786083 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/barbican-worker-b7f5dfb6f-pj6bx" podUID="b2933134-7226-423a-8120-3148845b808a" containerName="barbican-worker" containerID="cri-o://6ef42ed115f2247455a35e6dba9e2519ebc77a9c515e4efa379aa58e493e2dea" gracePeriod=30 Dec 03 12:54:43 crc kubenswrapper[5052]: I1203 12:54:43.787488 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-5vk45\" (UniqueName: \"kubernetes.io/projected/13e7e748-92f4-4255-a8eb-aa5a6faf361f-kube-api-access-5vk45\") pod \"novaapi7066-account-delete-whswm\" (UID: \"13e7e748-92f4-4255-a8eb-aa5a6faf361f\") " pod="openstack-kuttl-tests/novaapi7066-account-delete-whswm" Dec 03 12:54:43 crc kubenswrapper[5052]: I1203 12:54:43.787762 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/1b7ef5eb-406e-4e2c-9d9c-d763919f56fd-operator-scripts\") pod \"novacell02629-account-delete-wsshh\" (UID: \"1b7ef5eb-406e-4e2c-9d9c-d763919f56fd\") " pod="openstack-kuttl-tests/novacell02629-account-delete-wsshh" Dec 03 12:54:43 crc kubenswrapper[5052]: I1203 12:54:43.787843 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-s76qn\" (UniqueName: \"kubernetes.io/projected/1b7ef5eb-406e-4e2c-9d9c-d763919f56fd-kube-api-access-s76qn\") pod \"novacell02629-account-delete-wsshh\" (UID: \"1b7ef5eb-406e-4e2c-9d9c-d763919f56fd\") " pod="openstack-kuttl-tests/novacell02629-account-delete-wsshh" Dec 03 12:54:43 crc kubenswrapper[5052]: I1203 12:54:43.788031 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/13e7e748-92f4-4255-a8eb-aa5a6faf361f-operator-scripts\") pod \"novaapi7066-account-delete-whswm\" (UID: \"13e7e748-92f4-4255-a8eb-aa5a6faf361f\") " pod="openstack-kuttl-tests/novaapi7066-account-delete-whswm" Dec 03 12:54:43 crc kubenswrapper[5052]: E1203 12:54:43.790439 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/rabbitmq-cell1-config-data: configmap "rabbitmq-cell1-config-data" not found Dec 03 12:54:43 crc kubenswrapper[5052]: E1203 12:54:43.790542 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/568addd3-4ad3-47c3-adae-693691e7cd40-config-data podName:568addd3-4ad3-47c3-adae-693691e7cd40 nodeName:}" failed. No retries permitted until 2025-12-03 12:54:45.790520238 +0000 UTC m=+5676.378762002 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "config-data" (UniqueName: "kubernetes.io/configmap/568addd3-4ad3-47c3-adae-693691e7cd40-config-data") pod "rabbitmq-cell1-server-0" (UID: "568addd3-4ad3-47c3-adae-693691e7cd40") : configmap "rabbitmq-cell1-config-data" not found Dec 03 12:54:43 crc kubenswrapper[5052]: I1203 12:54:43.791799 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/1b7ef5eb-406e-4e2c-9d9c-d763919f56fd-operator-scripts\") pod \"novacell02629-account-delete-wsshh\" (UID: \"1b7ef5eb-406e-4e2c-9d9c-d763919f56fd\") " pod="openstack-kuttl-tests/novacell02629-account-delete-wsshh" Dec 03 12:54:43 crc kubenswrapper[5052]: I1203 12:54:43.799199 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/13e7e748-92f4-4255-a8eb-aa5a6faf361f-operator-scripts\") pod \"novaapi7066-account-delete-whswm\" (UID: \"13e7e748-92f4-4255-a8eb-aa5a6faf361f\") " pod="openstack-kuttl-tests/novaapi7066-account-delete-whswm" Dec 03 12:54:43 crc kubenswrapper[5052]: I1203 12:54:43.811295 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/rabbitmq-cell1-server-0"] Dec 03 12:54:43 crc kubenswrapper[5052]: I1203 12:54:43.822346 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-s76qn\" (UniqueName: \"kubernetes.io/projected/1b7ef5eb-406e-4e2c-9d9c-d763919f56fd-kube-api-access-s76qn\") pod \"novacell02629-account-delete-wsshh\" (UID: \"1b7ef5eb-406e-4e2c-9d9c-d763919f56fd\") " pod="openstack-kuttl-tests/novacell02629-account-delete-wsshh" Dec 03 12:54:43 crc kubenswrapper[5052]: I1203 12:54:43.857217 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:54:43 crc kubenswrapper[5052]: I1203 12:54:43.857444 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-scheduler-0" podUID="4ec49e76-2597-47d6-811b-f65cbe0580ff" containerName="nova-scheduler-scheduler" containerID="cri-o://ec5a6e0000f85ac80b044c5e702c36b00720989c7bd27aac0130a6739a2d1bd8" gracePeriod=30 Dec 03 12:54:43 crc kubenswrapper[5052]: I1203 12:54:43.858362 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-5vk45\" (UniqueName: \"kubernetes.io/projected/13e7e748-92f4-4255-a8eb-aa5a6faf361f-kube-api-access-5vk45\") pod \"novaapi7066-account-delete-whswm\" (UID: \"13e7e748-92f4-4255-a8eb-aa5a6faf361f\") " pod="openstack-kuttl-tests/novaapi7066-account-delete-whswm" Dec 03 12:54:43 crc kubenswrapper[5052]: I1203 12:54:43.894966 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/openstack-cell1-galera-0"] Dec 03 12:54:43 crc kubenswrapper[5052]: I1203 12:54:43.904741 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" podUID="568addd3-4ad3-47c3-adae-693691e7cd40" containerName="rabbitmq" containerID="cri-o://dc308258e1e18beb76be744d3270524d74c7f8be911cbba0e2cd35f103d54af5" gracePeriod=604800 Dec 03 12:54:43 crc kubenswrapper[5052]: I1203 12:54:43.904868 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:54:43 crc kubenswrapper[5052]: I1203 12:54:43.905066 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-api-0" podUID="7d297f47-a73f-4a88-bba0-3108deb8df4d" containerName="nova-api-log" containerID="cri-o://52ef14e30c84a7e38ff36a4c93fbfa8a565c2ed55f66905b14b2fdd87cd69077" gracePeriod=30 Dec 03 12:54:43 crc kubenswrapper[5052]: I1203 12:54:43.905452 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-api-0" podUID="7d297f47-a73f-4a88-bba0-3108deb8df4d" containerName="nova-api-api" containerID="cri-o://f15af8c0a8823cb666fbe94253b10ca1d40d1a5303a03c6ac25035b6456a7a85" gracePeriod=30 Dec 03 12:54:43 crc kubenswrapper[5052]: I1203 12:54:43.928503 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:54:43 crc kubenswrapper[5052]: I1203 12:54:43.928825 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-metadata-0" podUID="022746a5-1c20-4729-8688-5903831f57ed" containerName="nova-metadata-log" containerID="cri-o://3157794aa44d4b21b9c104515fcfd5ee6e33807ee7b2753cff2132e2facd1f75" gracePeriod=30 Dec 03 12:54:43 crc kubenswrapper[5052]: I1203 12:54:43.930171 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-metadata-0" podUID="022746a5-1c20-4729-8688-5903831f57ed" containerName="nova-metadata-metadata" containerID="cri-o://e304d74be1d9347456c21077ccfef718ab1e897114fc49c1d7da35457693360a" gracePeriod=30 Dec 03 12:54:43 crc kubenswrapper[5052]: I1203 12:54:43.953252 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 12:54:43 crc kubenswrapper[5052]: I1203 12:54:43.953558 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" podUID="43b07f3c-f9d7-451b-b0ab-61b293cff35d" containerName="nova-cell1-novncproxy-novncproxy" containerID="cri-o://92269ae03a0728af8ad79f1a3cc73e6323d763c9c370a177f9ea91e0fc11bf91" gracePeriod=30 Dec 03 12:54:43 crc kubenswrapper[5052]: I1203 12:54:43.980665 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell1-0a46-account-create-update-4n2r8"] Dec 03 12:54:44 crc kubenswrapper[5052]: I1203 12:54:44.034790 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell1-db-create-tp5g8"] Dec 03 12:54:44 crc kubenswrapper[5052]: I1203 12:54:44.047593 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/novacell10a46-account-delete-h8x7f"] Dec 03 12:54:44 crc kubenswrapper[5052]: I1203 12:54:44.055264 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell1-0a46-account-create-update-4n2r8"] Dec 03 12:54:44 crc kubenswrapper[5052]: I1203 12:54:44.079424 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell1-db-create-tp5g8"] Dec 03 12:54:44 crc kubenswrapper[5052]: E1203 12:54:44.082278 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-hs7rk operator-scripts], unattached volumes=[], failed to process volumes=[]: context canceled" pod="openstack-kuttl-tests/novacell10a46-account-delete-h8x7f" podUID="6d56c6dc-38c9-42cf-ac6d-04ba49860d31" Dec 03 12:54:44 crc kubenswrapper[5052]: I1203 12:54:44.100356 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-shdj6" Dec 03 12:54:44 crc kubenswrapper[5052]: I1203 12:54:44.102052 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/placementbfa4-account-delete-flc9f"] Dec 03 12:54:44 crc kubenswrapper[5052]: I1203 12:54:44.110290 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-hs7rk\" (UniqueName: \"kubernetes.io/projected/6d56c6dc-38c9-42cf-ac6d-04ba49860d31-kube-api-access-hs7rk\") pod \"novacell10a46-account-delete-h8x7f\" (UID: \"6d56c6dc-38c9-42cf-ac6d-04ba49860d31\") " pod="openstack-kuttl-tests/novacell10a46-account-delete-h8x7f" Dec 03 12:54:44 crc kubenswrapper[5052]: I1203 12:54:44.110372 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/6d56c6dc-38c9-42cf-ac6d-04ba49860d31-operator-scripts\") pod \"novacell10a46-account-delete-h8x7f\" (UID: \"6d56c6dc-38c9-42cf-ac6d-04ba49860d31\") " pod="openstack-kuttl-tests/novacell10a46-account-delete-h8x7f" Dec 03 12:54:44 crc kubenswrapper[5052]: E1203 12:54:44.110470 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-cell1-scripts: configmap "openstack-cell1-scripts" not found Dec 03 12:54:44 crc kubenswrapper[5052]: E1203 12:54:44.110528 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6d56c6dc-38c9-42cf-ac6d-04ba49860d31-operator-scripts podName:6d56c6dc-38c9-42cf-ac6d-04ba49860d31 nodeName:}" failed. No retries permitted until 2025-12-03 12:54:45.110499709 +0000 UTC m=+5675.698741473 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/6d56c6dc-38c9-42cf-ac6d-04ba49860d31-operator-scripts") pod "novacell10a46-account-delete-h8x7f" (UID: "6d56c6dc-38c9-42cf-ac6d-04ba49860d31") : configmap "openstack-cell1-scripts" not found Dec 03 12:54:44 crc kubenswrapper[5052]: E1203 12:54:44.113126 5052 projected.go:194] Error preparing data for projected volume kube-api-access-hs7rk for pod openstack-kuttl-tests/novacell10a46-account-delete-h8x7f: failed to fetch token: serviceaccounts "galera-openstack-cell1" not found Dec 03 12:54:44 crc kubenswrapper[5052]: E1203 12:54:44.113175 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6d56c6dc-38c9-42cf-ac6d-04ba49860d31-kube-api-access-hs7rk podName:6d56c6dc-38c9-42cf-ac6d-04ba49860d31 nodeName:}" failed. No retries permitted until 2025-12-03 12:54:45.113162255 +0000 UTC m=+5675.701404019 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-hs7rk" (UniqueName: "kubernetes.io/projected/6d56c6dc-38c9-42cf-ac6d-04ba49860d31-kube-api-access-hs7rk") pod "novacell10a46-account-delete-h8x7f" (UID: "6d56c6dc-38c9-42cf-ac6d-04ba49860d31") : failed to fetch token: serviceaccounts "galera-openstack-cell1" not found Dec 03 12:54:44 crc kubenswrapper[5052]: I1203 12:54:44.120428 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/novaapi7066-account-delete-whswm" Dec 03 12:54:44 crc kubenswrapper[5052]: I1203 12:54:44.148207 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/novacell02629-account-delete-wsshh" Dec 03 12:54:44 crc kubenswrapper[5052]: I1203 12:54:44.185428 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-0"] Dec 03 12:54:44 crc kubenswrapper[5052]: I1203 12:54:44.185637 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-cell1-conductor-0" podUID="2adad00d-01e3-4441-a3fc-87d162a5e309" containerName="nova-cell1-conductor-conductor" containerID="cri-o://a0d54bbd0da0ce78d0c6c5dcb48378864ee5ffb0c8e5280b729c579e83ef32f1" gracePeriod=30 Dec 03 12:54:44 crc kubenswrapper[5052]: I1203 12:54:44.197145 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-db-sync-gqn79"] Dec 03 12:54:44 crc kubenswrapper[5052]: I1203 12:54:44.202072 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/openstack-cell1-galera-0" podUID="64ada63f-b17d-411b-a547-fa1c15107ab5" containerName="galera" containerID="cri-o://594b90765c644ac99bd8e077dd10ec16881d4e9f1364dcdc6541052085119849" gracePeriod=30 Dec 03 12:54:44 crc kubenswrapper[5052]: I1203 12:54:44.204904 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-db-sync-gqn79"] Dec 03 12:54:44 crc kubenswrapper[5052]: I1203 12:54:44.211890 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-db-sync-8f92x"] Dec 03 12:54:44 crc kubenswrapper[5052]: I1203 12:54:44.225862 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-0"] Dec 03 12:54:44 crc kubenswrapper[5052]: I1203 12:54:44.226557 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/nova-cell0-conductor-0" podUID="3c6d824e-c7da-41ff-adde-dbafb7535476" containerName="nova-cell0-conductor-conductor" containerID="cri-o://eb28bc8efa7fd43a1e5b7da097e9679a59f690173e27b20d4dae0a08837dde86" gracePeriod=30 Dec 03 12:54:44 crc kubenswrapper[5052]: I1203 12:54:44.254228 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-db-sync-8f92x"] Dec 03 12:54:44 crc kubenswrapper[5052]: I1203 12:54:44.264727 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron6c26-account-delete-d7wtw"] Dec 03 12:54:44 crc kubenswrapper[5052]: W1203 12:54:44.373419 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod941d063f_c834_47a1_bc0c_2096b005e09c.slice/crio-fe894db25a6b4c1c2ece51721c19741c414732bda90808788f9d1eebd4775640 WatchSource:0}: Error finding container fe894db25a6b4c1c2ece51721c19741c414732bda90808788f9d1eebd4775640: Status 404 returned error can't find the container with id fe894db25a6b4c1c2ece51721c19741c414732bda90808788f9d1eebd4775640 Dec 03 12:54:44 crc kubenswrapper[5052]: I1203 12:54:44.407836 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstackclient" Dec 03 12:54:44 crc kubenswrapper[5052]: I1203 12:54:44.502423 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_ovsdbserver-sb-0_d16285c0-7e62-4b3c-b3e8-a62b8abb49cb/ovsdbserver-sb/0.log" Dec 03 12:54:44 crc kubenswrapper[5052]: I1203 12:54:44.502493 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:54:44 crc kubenswrapper[5052]: I1203 12:54:44.517464 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t6gvj\" (UniqueName: \"kubernetes.io/projected/c4b7bd04-32be-43c6-82f1-968f2e368b22-kube-api-access-t6gvj\") pod \"c4b7bd04-32be-43c6-82f1-968f2e368b22\" (UID: \"c4b7bd04-32be-43c6-82f1-968f2e368b22\") " Dec 03 12:54:44 crc kubenswrapper[5052]: I1203 12:54:44.517612 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"openstack-config-secret\" (UniqueName: \"kubernetes.io/secret/c4b7bd04-32be-43c6-82f1-968f2e368b22-openstack-config-secret\") pod \"c4b7bd04-32be-43c6-82f1-968f2e368b22\" (UID: \"c4b7bd04-32be-43c6-82f1-968f2e368b22\") " Dec 03 12:54:44 crc kubenswrapper[5052]: I1203 12:54:44.517763 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c4b7bd04-32be-43c6-82f1-968f2e368b22-combined-ca-bundle\") pod \"c4b7bd04-32be-43c6-82f1-968f2e368b22\" (UID: \"c4b7bd04-32be-43c6-82f1-968f2e368b22\") " Dec 03 12:54:44 crc kubenswrapper[5052]: I1203 12:54:44.517873 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"openstack-config\" (UniqueName: \"kubernetes.io/configmap/c4b7bd04-32be-43c6-82f1-968f2e368b22-openstack-config\") pod \"c4b7bd04-32be-43c6-82f1-968f2e368b22\" (UID: \"c4b7bd04-32be-43c6-82f1-968f2e368b22\") " Dec 03 12:54:44 crc kubenswrapper[5052]: I1203 12:54:44.538060 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_ovsdbserver-nb-0_3a5e94ea-faeb-4202-843a-bd691d8895ce/ovsdbserver-nb/0.log" Dec 03 12:54:44 crc kubenswrapper[5052]: I1203 12:54:44.538383 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:54:44 crc kubenswrapper[5052]: I1203 12:54:44.539934 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:54:44 crc kubenswrapper[5052]: I1203 12:54:44.542338 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="02edb9e9-3f42-4aea-9fe6-7efa332d5fb8" containerName="ceilometer-central-agent" containerID="cri-o://86ed75c4a9b117c08c2a3ecec9878ef557fc9d6d68fea18aa83751770d37d20b" gracePeriod=30 Dec 03 12:54:44 crc kubenswrapper[5052]: I1203 12:54:44.542536 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="02edb9e9-3f42-4aea-9fe6-7efa332d5fb8" containerName="proxy-httpd" containerID="cri-o://990e0730d2387a55431e533e2f7232d5369fd95354ef6d99da9341e99b6b7c58" gracePeriod=30 Dec 03 12:54:44 crc kubenswrapper[5052]: I1203 12:54:44.542589 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="02edb9e9-3f42-4aea-9fe6-7efa332d5fb8" containerName="sg-core" containerID="cri-o://59b945b4df7623c03eb622cbe50e816dadd5224b5097d5e7e646cca960ca90e1" gracePeriod=30 Dec 03 12:54:44 crc kubenswrapper[5052]: I1203 12:54:44.542644 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/ceilometer-0" podUID="02edb9e9-3f42-4aea-9fe6-7efa332d5fb8" containerName="ceilometer-notification-agent" containerID="cri-o://72785ef6236001625d6d7f1e690671c78dc1f11c3518cfcc3bf9d6de5ab4c2fb" gracePeriod=30 Dec 03 12:54:44 crc kubenswrapper[5052]: I1203 12:54:44.637227 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovsdbserver-sb-tls-certs\" (UniqueName: \"kubernetes.io/secret/d16285c0-7e62-4b3c-b3e8-a62b8abb49cb-ovsdbserver-sb-tls-certs\") pod \"d16285c0-7e62-4b3c-b3e8-a62b8abb49cb\" (UID: \"d16285c0-7e62-4b3c-b3e8-a62b8abb49cb\") " Dec 03 12:54:44 crc kubenswrapper[5052]: I1203 12:54:44.637423 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d16285c0-7e62-4b3c-b3e8-a62b8abb49cb-config\") pod \"d16285c0-7e62-4b3c-b3e8-a62b8abb49cb\" (UID: \"d16285c0-7e62-4b3c-b3e8-a62b8abb49cb\") " Dec 03 12:54:44 crc kubenswrapper[5052]: I1203 12:54:44.637451 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d16285c0-7e62-4b3c-b3e8-a62b8abb49cb-combined-ca-bundle\") pod \"d16285c0-7e62-4b3c-b3e8-a62b8abb49cb\" (UID: \"d16285c0-7e62-4b3c-b3e8-a62b8abb49cb\") " Dec 03 12:54:44 crc kubenswrapper[5052]: I1203 12:54:44.637534 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/d16285c0-7e62-4b3c-b3e8-a62b8abb49cb-ovsdb-rundir\") pod \"d16285c0-7e62-4b3c-b3e8-a62b8abb49cb\" (UID: \"d16285c0-7e62-4b3c-b3e8-a62b8abb49cb\") " Dec 03 12:54:44 crc kubenswrapper[5052]: I1203 12:54:44.637583 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovndbcluster-sb-etc-ovn\" (UniqueName: \"kubernetes.io/local-volume/local-storage04-crc\") pod \"d16285c0-7e62-4b3c-b3e8-a62b8abb49cb\" (UID: \"d16285c0-7e62-4b3c-b3e8-a62b8abb49cb\") " Dec 03 12:54:44 crc kubenswrapper[5052]: I1203 12:54:44.637655 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/d16285c0-7e62-4b3c-b3e8-a62b8abb49cb-scripts\") pod \"d16285c0-7e62-4b3c-b3e8-a62b8abb49cb\" (UID: \"d16285c0-7e62-4b3c-b3e8-a62b8abb49cb\") " Dec 03 12:54:44 crc kubenswrapper[5052]: I1203 12:54:44.637719 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/d16285c0-7e62-4b3c-b3e8-a62b8abb49cb-metrics-certs-tls-certs\") pod \"d16285c0-7e62-4b3c-b3e8-a62b8abb49cb\" (UID: \"d16285c0-7e62-4b3c-b3e8-a62b8abb49cb\") " Dec 03 12:54:44 crc kubenswrapper[5052]: I1203 12:54:44.637765 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4fxgg\" (UniqueName: \"kubernetes.io/projected/d16285c0-7e62-4b3c-b3e8-a62b8abb49cb-kube-api-access-4fxgg\") pod \"d16285c0-7e62-4b3c-b3e8-a62b8abb49cb\" (UID: \"d16285c0-7e62-4b3c-b3e8-a62b8abb49cb\") " Dec 03 12:54:44 crc kubenswrapper[5052]: I1203 12:54:44.639488 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d16285c0-7e62-4b3c-b3e8-a62b8abb49cb-ovsdb-rundir" (OuterVolumeSpecName: "ovsdb-rundir") pod "d16285c0-7e62-4b3c-b3e8-a62b8abb49cb" (UID: "d16285c0-7e62-4b3c-b3e8-a62b8abb49cb"). InnerVolumeSpecName "ovsdb-rundir". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:54:44 crc kubenswrapper[5052]: I1203 12:54:44.641888 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d16285c0-7e62-4b3c-b3e8-a62b8abb49cb-config" (OuterVolumeSpecName: "config") pod "d16285c0-7e62-4b3c-b3e8-a62b8abb49cb" (UID: "d16285c0-7e62-4b3c-b3e8-a62b8abb49cb"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:54:44 crc kubenswrapper[5052]: I1203 12:54:44.647601 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c4b7bd04-32be-43c6-82f1-968f2e368b22-kube-api-access-t6gvj" (OuterVolumeSpecName: "kube-api-access-t6gvj") pod "c4b7bd04-32be-43c6-82f1-968f2e368b22" (UID: "c4b7bd04-32be-43c6-82f1-968f2e368b22"). InnerVolumeSpecName "kube-api-access-t6gvj". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:54:44 crc kubenswrapper[5052]: I1203 12:54:44.663157 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d16285c0-7e62-4b3c-b3e8-a62b8abb49cb-kube-api-access-4fxgg" (OuterVolumeSpecName: "kube-api-access-4fxgg") pod "d16285c0-7e62-4b3c-b3e8-a62b8abb49cb" (UID: "d16285c0-7e62-4b3c-b3e8-a62b8abb49cb"). InnerVolumeSpecName "kube-api-access-4fxgg". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:54:44 crc kubenswrapper[5052]: I1203 12:54:44.664726 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d16285c0-7e62-4b3c-b3e8-a62b8abb49cb-scripts" (OuterVolumeSpecName: "scripts") pod "d16285c0-7e62-4b3c-b3e8-a62b8abb49cb" (UID: "d16285c0-7e62-4b3c-b3e8-a62b8abb49cb"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:54:44 crc kubenswrapper[5052]: I1203 12:54:44.736970 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/kube-state-metrics-0"] Dec 03 12:54:44 crc kubenswrapper[5052]: I1203 12:54:44.737419 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/kube-state-metrics-0" podUID="3c553114-e0c8-4af9-992f-5421b31497ed" containerName="kube-state-metrics" containerID="cri-o://77bdaa2aad097bc96adc6a7e702b0e7e164f8b659c20495278f4d679ff5255fc" gracePeriod=30 Dec 03 12:54:44 crc kubenswrapper[5052]: I1203 12:54:44.748220 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3a5e94ea-faeb-4202-843a-bd691d8895ce-combined-ca-bundle\") pod \"3a5e94ea-faeb-4202-843a-bd691d8895ce\" (UID: \"3a5e94ea-faeb-4202-843a-bd691d8895ce\") " Dec 03 12:54:44 crc kubenswrapper[5052]: I1203 12:54:44.748331 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8nd6t\" (UniqueName: \"kubernetes.io/projected/3a5e94ea-faeb-4202-843a-bd691d8895ce-kube-api-access-8nd6t\") pod \"3a5e94ea-faeb-4202-843a-bd691d8895ce\" (UID: \"3a5e94ea-faeb-4202-843a-bd691d8895ce\") " Dec 03 12:54:44 crc kubenswrapper[5052]: I1203 12:54:44.748787 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3a5e94ea-faeb-4202-843a-bd691d8895ce-ovsdb-rundir" (OuterVolumeSpecName: "ovsdb-rundir") pod "3a5e94ea-faeb-4202-843a-bd691d8895ce" (UID: "3a5e94ea-faeb-4202-843a-bd691d8895ce"). InnerVolumeSpecName "ovsdb-rundir". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:54:44 crc kubenswrapper[5052]: I1203 12:54:44.751745 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/3a5e94ea-faeb-4202-843a-bd691d8895ce-ovsdb-rundir\") pod \"3a5e94ea-faeb-4202-843a-bd691d8895ce\" (UID: \"3a5e94ea-faeb-4202-843a-bd691d8895ce\") " Dec 03 12:54:44 crc kubenswrapper[5052]: I1203 12:54:44.751848 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/3a5e94ea-faeb-4202-843a-bd691d8895ce-scripts\") pod \"3a5e94ea-faeb-4202-843a-bd691d8895ce\" (UID: \"3a5e94ea-faeb-4202-843a-bd691d8895ce\") " Dec 03 12:54:44 crc kubenswrapper[5052]: I1203 12:54:44.751918 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovsdbserver-nb-tls-certs\" (UniqueName: \"kubernetes.io/secret/3a5e94ea-faeb-4202-843a-bd691d8895ce-ovsdbserver-nb-tls-certs\") pod \"3a5e94ea-faeb-4202-843a-bd691d8895ce\" (UID: \"3a5e94ea-faeb-4202-843a-bd691d8895ce\") " Dec 03 12:54:44 crc kubenswrapper[5052]: I1203 12:54:44.752224 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovndbcluster-nb-etc-ovn\" (UniqueName: \"kubernetes.io/local-volume/local-storage15-crc\") pod \"3a5e94ea-faeb-4202-843a-bd691d8895ce\" (UID: \"3a5e94ea-faeb-4202-843a-bd691d8895ce\") " Dec 03 12:54:44 crc kubenswrapper[5052]: I1203 12:54:44.752279 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/3a5e94ea-faeb-4202-843a-bd691d8895ce-metrics-certs-tls-certs\") pod \"3a5e94ea-faeb-4202-843a-bd691d8895ce\" (UID: \"3a5e94ea-faeb-4202-843a-bd691d8895ce\") " Dec 03 12:54:44 crc kubenswrapper[5052]: I1203 12:54:44.752438 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3a5e94ea-faeb-4202-843a-bd691d8895ce-config\") pod \"3a5e94ea-faeb-4202-843a-bd691d8895ce\" (UID: \"3a5e94ea-faeb-4202-843a-bd691d8895ce\") " Dec 03 12:54:44 crc kubenswrapper[5052]: I1203 12:54:44.754298 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3a5e94ea-faeb-4202-843a-bd691d8895ce-config" (OuterVolumeSpecName: "config") pod "3a5e94ea-faeb-4202-843a-bd691d8895ce" (UID: "3a5e94ea-faeb-4202-843a-bd691d8895ce"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:54:44 crc kubenswrapper[5052]: I1203 12:54:44.754739 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3a5e94ea-faeb-4202-843a-bd691d8895ce-scripts" (OuterVolumeSpecName: "scripts") pod "3a5e94ea-faeb-4202-843a-bd691d8895ce" (UID: "3a5e94ea-faeb-4202-843a-bd691d8895ce"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:54:44 crc kubenswrapper[5052]: I1203 12:54:44.762586 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage04-crc" (OuterVolumeSpecName: "ovndbcluster-sb-etc-ovn") pod "d16285c0-7e62-4b3c-b3e8-a62b8abb49cb" (UID: "d16285c0-7e62-4b3c-b3e8-a62b8abb49cb"). InnerVolumeSpecName "local-storage04-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 12:54:44 crc kubenswrapper[5052]: I1203 12:54:44.764089 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-t6gvj\" (UniqueName: \"kubernetes.io/projected/c4b7bd04-32be-43c6-82f1-968f2e368b22-kube-api-access-t6gvj\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:44 crc kubenswrapper[5052]: I1203 12:54:44.764129 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d16285c0-7e62-4b3c-b3e8-a62b8abb49cb-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:44 crc kubenswrapper[5052]: I1203 12:54:44.764145 5052 reconciler_common.go:293] "Volume detached for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/d16285c0-7e62-4b3c-b3e8-a62b8abb49cb-ovsdb-rundir\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:44 crc kubenswrapper[5052]: I1203 12:54:44.764174 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/d16285c0-7e62-4b3c-b3e8-a62b8abb49cb-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:44 crc kubenswrapper[5052]: I1203 12:54:44.764190 5052 reconciler_common.go:293] "Volume detached for volume \"ovsdb-rundir\" (UniqueName: \"kubernetes.io/empty-dir/3a5e94ea-faeb-4202-843a-bd691d8895ce-ovsdb-rundir\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:44 crc kubenswrapper[5052]: I1203 12:54:44.764209 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-4fxgg\" (UniqueName: \"kubernetes.io/projected/d16285c0-7e62-4b3c-b3e8-a62b8abb49cb-kube-api-access-4fxgg\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:44 crc kubenswrapper[5052]: I1203 12:54:44.811401 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/glance7c80-account-delete-knmlm"] Dec 03 12:54:44 crc kubenswrapper[5052]: I1203 12:54:44.815260 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage15-crc" (OuterVolumeSpecName: "ovndbcluster-nb-etc-ovn") pod "3a5e94ea-faeb-4202-843a-bd691d8895ce" (UID: "3a5e94ea-faeb-4202-843a-bd691d8895ce"). InnerVolumeSpecName "local-storage15-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 12:54:44 crc kubenswrapper[5052]: I1203 12:54:44.815441 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3a5e94ea-faeb-4202-843a-bd691d8895ce-kube-api-access-8nd6t" (OuterVolumeSpecName: "kube-api-access-8nd6t") pod "3a5e94ea-faeb-4202-843a-bd691d8895ce" (UID: "3a5e94ea-faeb-4202-843a-bd691d8895ce"). InnerVolumeSpecName "kube-api-access-8nd6t". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:54:44 crc kubenswrapper[5052]: I1203 12:54:44.863845 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_ovsdbserver-sb-0_d16285c0-7e62-4b3c-b3e8-a62b8abb49cb/ovsdbserver-sb/0.log" Dec 03 12:54:44 crc kubenswrapper[5052]: I1203 12:54:44.863934 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-sb-0" event={"ID":"d16285c0-7e62-4b3c-b3e8-a62b8abb49cb","Type":"ContainerDied","Data":"32b977d602b5b173414e41de53d7cf4d2880cb47f71b460727db4cca78a1c064"} Dec 03 12:54:44 crc kubenswrapper[5052]: I1203 12:54:44.863972 5052 scope.go:117] "RemoveContainer" containerID="89cea0c9094ed2fb8bb3973f66f25388cbb772d78cc9dc394bfc10058b78d468" Dec 03 12:54:44 crc kubenswrapper[5052]: I1203 12:54:44.864133 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovsdbserver-sb-0" Dec 03 12:54:44 crc kubenswrapper[5052]: I1203 12:54:44.866357 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3a5e94ea-faeb-4202-843a-bd691d8895ce-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:44 crc kubenswrapper[5052]: I1203 12:54:44.866389 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage04-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage04-crc\") on node \"crc\" " Dec 03 12:54:44 crc kubenswrapper[5052]: I1203 12:54:44.866401 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-8nd6t\" (UniqueName: \"kubernetes.io/projected/3a5e94ea-faeb-4202-843a-bd691d8895ce-kube-api-access-8nd6t\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:44 crc kubenswrapper[5052]: I1203 12:54:44.866413 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/3a5e94ea-faeb-4202-843a-bd691d8895ce-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:44 crc kubenswrapper[5052]: I1203 12:54:44.866430 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage15-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage15-crc\") on node \"crc\" " Dec 03 12:54:44 crc kubenswrapper[5052]: E1203 12:54:44.875002 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/rabbitmq-config-data: configmap "rabbitmq-config-data" not found Dec 03 12:54:44 crc kubenswrapper[5052]: E1203 12:54:44.875079 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e73813f5-f97f-41dd-8a79-6196557d4d1a-config-data podName:e73813f5-f97f-41dd-8a79-6196557d4d1a nodeName:}" failed. No retries permitted until 2025-12-03 12:54:48.875058024 +0000 UTC m=+5679.463299788 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "config-data" (UniqueName: "kubernetes.io/configmap/e73813f5-f97f-41dd-8a79-6196557d4d1a-config-data") pod "rabbitmq-server-0" (UID: "e73813f5-f97f-41dd-8a79-6196557d4d1a") : configmap "rabbitmq-config-data" not found Dec 03 12:54:44 crc kubenswrapper[5052]: I1203 12:54:44.935578 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/memcached-0"] Dec 03 12:54:44 crc kubenswrapper[5052]: I1203 12:54:44.936178 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/memcached-0" podUID="d1ff5100-3ac7-4633-811c-ea470242d858" containerName="memcached" containerID="cri-o://27f09642bc3ec7488d9032aa069fc921aca6a6f0c2792bc5d84326beb5487829" gracePeriod=30 Dec 03 12:54:44 crc kubenswrapper[5052]: I1203 12:54:44.962532 5052 generic.go:334] "Generic (PLEG): container finished" podID="7d297f47-a73f-4a88-bba0-3108deb8df4d" containerID="52ef14e30c84a7e38ff36a4c93fbfa8a565c2ed55f66905b14b2fdd87cd69077" exitCode=143 Dec 03 12:54:44 crc kubenswrapper[5052]: I1203 12:54:44.962651 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"7d297f47-a73f-4a88-bba0-3108deb8df4d","Type":"ContainerDied","Data":"52ef14e30c84a7e38ff36a4c93fbfa8a565c2ed55f66905b14b2fdd87cd69077"} Dec 03 12:54:44 crc kubenswrapper[5052]: I1203 12:54:44.967555 5052 generic.go:334] "Generic (PLEG): container finished" podID="2b0bf9ae-2d1d-48da-8c4b-2973b47985a3" containerID="78894a79ffab8e64787cde1b1f24aa5fa09f830cec10c1b466b6619c2b33ae10" exitCode=0 Dec 03 12:54:44 crc kubenswrapper[5052]: I1203 12:54:44.967582 5052 generic.go:334] "Generic (PLEG): container finished" podID="2b0bf9ae-2d1d-48da-8c4b-2973b47985a3" containerID="34ec2e3b276084c3a6169386d90460245b79be1c0922971153b1615fd7e2d686" exitCode=0 Dec 03 12:54:44 crc kubenswrapper[5052]: I1203 12:54:44.967628 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"2b0bf9ae-2d1d-48da-8c4b-2973b47985a3","Type":"ContainerDied","Data":"78894a79ffab8e64787cde1b1f24aa5fa09f830cec10c1b466b6619c2b33ae10"} Dec 03 12:54:44 crc kubenswrapper[5052]: I1203 12:54:44.967663 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"2b0bf9ae-2d1d-48da-8c4b-2973b47985a3","Type":"ContainerDied","Data":"34ec2e3b276084c3a6169386d90460245b79be1c0922971153b1615fd7e2d686"} Dec 03 12:54:46 crc kubenswrapper[5052]: I1203 12:54:44.982328 5052 generic.go:334] "Generic (PLEG): container finished" podID="e2225738-b0d1-4306-bbee-a61dd9558c80" containerID="89798d3d34dfd5e2ea4266a0c6cf4bb6c1e2078d3038566fe9362ddc07ef9734" exitCode=0 Dec 03 12:54:46 crc kubenswrapper[5052]: I1203 12:54:44.982543 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-698d8977cd-ppz9n" event={"ID":"e2225738-b0d1-4306-bbee-a61dd9558c80","Type":"ContainerDied","Data":"89798d3d34dfd5e2ea4266a0c6cf4bb6c1e2078d3038566fe9362ddc07ef9734"} Dec 03 12:54:46 crc kubenswrapper[5052]: I1203 12:54:44.986202 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c4b7bd04-32be-43c6-82f1-968f2e368b22-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "c4b7bd04-32be-43c6-82f1-968f2e368b22" (UID: "c4b7bd04-32be-43c6-82f1-968f2e368b22"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:54:46 crc kubenswrapper[5052]: I1203 12:54:45.019613 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/cindera2a9-account-delete-pmspg"] Dec 03 12:54:46 crc kubenswrapper[5052]: I1203 12:54:45.043322 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstackclient" Dec 03 12:54:46 crc kubenswrapper[5052]: I1203 12:54:45.049763 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/keystone-bootstrap-d2z7m"] Dec 03 12:54:46 crc kubenswrapper[5052]: I1203 12:54:45.063645 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/keystone-db-sync-skdtp"] Dec 03 12:54:46 crc kubenswrapper[5052]: I1203 12:54:45.070266 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placementbfa4-account-delete-flc9f" event={"ID":"c4d5c183-8df4-4c43-9afb-2b28c6aae9e1","Type":"ContainerStarted","Data":"ca28d19362660f7ce08f01405dcf6a9e78f1824720527f0865bf72dc4425f918"} Dec 03 12:54:46 crc kubenswrapper[5052]: I1203 12:54:45.072227 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c4b7bd04-32be-43c6-82f1-968f2e368b22-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:46 crc kubenswrapper[5052]: I1203 12:54:45.098169 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/keystone-bootstrap-d2z7m"] Dec 03 12:54:46 crc kubenswrapper[5052]: I1203 12:54:45.102893 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/keystone-db-sync-skdtp"] Dec 03 12:54:46 crc kubenswrapper[5052]: I1203 12:54:45.104151 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c4b7bd04-32be-43c6-82f1-968f2e368b22-openstack-config" (OuterVolumeSpecName: "openstack-config") pod "c4b7bd04-32be-43c6-82f1-968f2e368b22" (UID: "c4b7bd04-32be-43c6-82f1-968f2e368b22"). InnerVolumeSpecName "openstack-config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:54:46 crc kubenswrapper[5052]: I1203 12:54:45.110610 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/barbican6e62-account-delete-6skhm"] Dec 03 12:54:46 crc kubenswrapper[5052]: I1203 12:54:45.132451 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3a5e94ea-faeb-4202-843a-bd691d8895ce-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "3a5e94ea-faeb-4202-843a-bd691d8895ce" (UID: "3a5e94ea-faeb-4202-843a-bd691d8895ce"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:54:46 crc kubenswrapper[5052]: I1203 12:54:45.142472 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage04-crc" (UniqueName: "kubernetes.io/local-volume/local-storage04-crc") on node "crc" Dec 03 12:54:46 crc kubenswrapper[5052]: I1203 12:54:45.146603 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/keystone-7fd545bd9-2lc7s"] Dec 03 12:54:46 crc kubenswrapper[5052]: I1203 12:54:45.146877 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/keystone-7fd545bd9-2lc7s" podUID="0191d138-7aff-42c2-88d2-fbbaf1722bfd" containerName="keystone-api" containerID="cri-o://ac4ecfa5627541f93acf9cc63e185d94bb2e833e592e7703865bb5d4026f322f" gracePeriod=30 Dec 03 12:54:46 crc kubenswrapper[5052]: W1203 12:54:45.163825 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode8a26759_2926_41cf_a1ec_286fc9962bb3.slice/crio-7f88d68507ef2d369c5fe3dce319cce0c03f0265b1bc25ed04d6d8be9cafb545 WatchSource:0}: Error finding container 7f88d68507ef2d369c5fe3dce319cce0c03f0265b1bc25ed04d6d8be9cafb545: Status 404 returned error can't find the container with id 7f88d68507ef2d369c5fe3dce319cce0c03f0265b1bc25ed04d6d8be9cafb545 Dec 03 12:54:46 crc kubenswrapper[5052]: I1203 12:54:45.175213 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-hs7rk\" (UniqueName: \"kubernetes.io/projected/6d56c6dc-38c9-42cf-ac6d-04ba49860d31-kube-api-access-hs7rk\") pod \"novacell10a46-account-delete-h8x7f\" (UID: \"6d56c6dc-38c9-42cf-ac6d-04ba49860d31\") " pod="openstack-kuttl-tests/novacell10a46-account-delete-h8x7f" Dec 03 12:54:46 crc kubenswrapper[5052]: I1203 12:54:45.175351 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/6d56c6dc-38c9-42cf-ac6d-04ba49860d31-operator-scripts\") pod \"novacell10a46-account-delete-h8x7f\" (UID: \"6d56c6dc-38c9-42cf-ac6d-04ba49860d31\") " pod="openstack-kuttl-tests/novacell10a46-account-delete-h8x7f" Dec 03 12:54:46 crc kubenswrapper[5052]: I1203 12:54:45.175586 5052 reconciler_common.go:293] "Volume detached for volume \"openstack-config\" (UniqueName: \"kubernetes.io/configmap/c4b7bd04-32be-43c6-82f1-968f2e368b22-openstack-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:46 crc kubenswrapper[5052]: I1203 12:54:45.175603 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage04-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage04-crc\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:46 crc kubenswrapper[5052]: I1203 12:54:45.175615 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3a5e94ea-faeb-4202-843a-bd691d8895ce-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:46 crc kubenswrapper[5052]: E1203 12:54:45.175683 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-cell1-scripts: configmap "openstack-cell1-scripts" not found Dec 03 12:54:46 crc kubenswrapper[5052]: E1203 12:54:45.175745 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6d56c6dc-38c9-42cf-ac6d-04ba49860d31-operator-scripts podName:6d56c6dc-38c9-42cf-ac6d-04ba49860d31 nodeName:}" failed. No retries permitted until 2025-12-03 12:54:47.175726753 +0000 UTC m=+5677.763968517 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/6d56c6dc-38c9-42cf-ac6d-04ba49860d31-operator-scripts") pod "novacell10a46-account-delete-h8x7f" (UID: "6d56c6dc-38c9-42cf-ac6d-04ba49860d31") : configmap "openstack-cell1-scripts" not found Dec 03 12:54:46 crc kubenswrapper[5052]: E1203 12:54:45.178126 5052 projected.go:194] Error preparing data for projected volume kube-api-access-hs7rk for pod openstack-kuttl-tests/novacell10a46-account-delete-h8x7f: failed to fetch token: serviceaccounts "galera-openstack-cell1" not found Dec 03 12:54:46 crc kubenswrapper[5052]: E1203 12:54:45.178199 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6d56c6dc-38c9-42cf-ac6d-04ba49860d31-kube-api-access-hs7rk podName:6d56c6dc-38c9-42cf-ac6d-04ba49860d31 nodeName:}" failed. No retries permitted until 2025-12-03 12:54:47.178177243 +0000 UTC m=+5677.766419007 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-hs7rk" (UniqueName: "kubernetes.io/projected/6d56c6dc-38c9-42cf-ac6d-04ba49860d31-kube-api-access-hs7rk") pod "novacell10a46-account-delete-h8x7f" (UID: "6d56c6dc-38c9-42cf-ac6d-04ba49860d31") : failed to fetch token: serviceaccounts "galera-openstack-cell1" not found Dec 03 12:54:46 crc kubenswrapper[5052]: I1203 12:54:45.186652 5052 generic.go:334] "Generic (PLEG): container finished" podID="5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4" containerID="7bdd0d115f77d5d7457272b4632a96a1cbaef3a87e7a1296d68acdce465508f1" exitCode=0 Dec 03 12:54:46 crc kubenswrapper[5052]: I1203 12:54:45.187024 5052 generic.go:334] "Generic (PLEG): container finished" podID="5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4" containerID="1087472cb2f4b1be538bcf920658b4a2f73a08130ae74a54e44b3322500d6407" exitCode=0 Dec 03 12:54:46 crc kubenswrapper[5052]: I1203 12:54:45.187039 5052 generic.go:334] "Generic (PLEG): container finished" podID="5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4" containerID="f162a9aaff5508763fc99c270ac25a9155335aac10652767b1730627c88c0d13" exitCode=0 Dec 03 12:54:46 crc kubenswrapper[5052]: I1203 12:54:45.187090 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4","Type":"ContainerDied","Data":"7bdd0d115f77d5d7457272b4632a96a1cbaef3a87e7a1296d68acdce465508f1"} Dec 03 12:54:46 crc kubenswrapper[5052]: I1203 12:54:45.187133 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4","Type":"ContainerDied","Data":"1087472cb2f4b1be538bcf920658b4a2f73a08130ae74a54e44b3322500d6407"} Dec 03 12:54:46 crc kubenswrapper[5052]: I1203 12:54:45.187144 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4","Type":"ContainerDied","Data":"f162a9aaff5508763fc99c270ac25a9155335aac10652767b1730627c88c0d13"} Dec 03 12:54:46 crc kubenswrapper[5052]: I1203 12:54:45.189613 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d16285c0-7e62-4b3c-b3e8-a62b8abb49cb-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "d16285c0-7e62-4b3c-b3e8-a62b8abb49cb" (UID: "d16285c0-7e62-4b3c-b3e8-a62b8abb49cb"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:54:46 crc kubenswrapper[5052]: I1203 12:54:45.193978 5052 generic.go:334] "Generic (PLEG): container finished" podID="b2933134-7226-423a-8120-3148845b808a" containerID="c45182c71bc38ac598875105b0f4d0cd30926ee88268d53941e68205eaeb487b" exitCode=143 Dec 03 12:54:46 crc kubenswrapper[5052]: I1203 12:54:45.194084 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-worker-b7f5dfb6f-pj6bx" event={"ID":"b2933134-7226-423a-8120-3148845b808a","Type":"ContainerDied","Data":"c45182c71bc38ac598875105b0f4d0cd30926ee88268d53941e68205eaeb487b"} Dec 03 12:54:46 crc kubenswrapper[5052]: I1203 12:54:45.196147 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron6c26-account-delete-d7wtw" event={"ID":"941d063f-c834-47a1-bc0c-2096b005e09c","Type":"ContainerStarted","Data":"fe894db25a6b4c1c2ece51721c19741c414732bda90808788f9d1eebd4775640"} Dec 03 12:54:46 crc kubenswrapper[5052]: I1203 12:54:45.206823 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/openstack-galera-0"] Dec 03 12:54:46 crc kubenswrapper[5052]: I1203 12:54:45.236783 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/keystone-c555-account-create-update-6zkgw"] Dec 03 12:54:46 crc kubenswrapper[5052]: I1203 12:54:45.237241 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage15-crc" (UniqueName: "kubernetes.io/local-volume/local-storage15-crc") on node "crc" Dec 03 12:54:46 crc kubenswrapper[5052]: I1203 12:54:45.243437 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_ovsdbserver-nb-0_3a5e94ea-faeb-4202-843a-bd691d8895ce/ovsdbserver-nb/0.log" Dec 03 12:54:46 crc kubenswrapper[5052]: I1203 12:54:45.243594 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovsdbserver-nb-0" Dec 03 12:54:46 crc kubenswrapper[5052]: I1203 12:54:45.243737 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovsdbserver-nb-0" event={"ID":"3a5e94ea-faeb-4202-843a-bd691d8895ce","Type":"ContainerDied","Data":"5ce9123099cbd6bdd7b0ddb2ed641312926d368d6d76b979c601a45165285cb5"} Dec 03 12:54:46 crc kubenswrapper[5052]: I1203 12:54:45.255573 5052 generic.go:334] "Generic (PLEG): container finished" podID="25eead1e-d17c-4b36-8a9f-ef563189d53b" containerID="4c38a9546e116479167e49dc563abec492318ccfbe7e54b5db501a8e8665f5aa" exitCode=143 Dec 03 12:54:46 crc kubenswrapper[5052]: I1203 12:54:45.255632 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-76dc68d888-zmmcs" event={"ID":"25eead1e-d17c-4b36-8a9f-ef563189d53b","Type":"ContainerDied","Data":"4c38a9546e116479167e49dc563abec492318ccfbe7e54b5db501a8e8665f5aa"} Dec 03 12:54:46 crc kubenswrapper[5052]: I1203 12:54:45.257397 5052 generic.go:334] "Generic (PLEG): container finished" podID="020bfa4c-3dba-43d2-9ac2-0bbbae660709" containerID="27212e7a0c6918bbf263320719b8b0731ba8a5d005b2aed71d83eab909ffb6f0" exitCode=143 Dec 03 12:54:46 crc kubenswrapper[5052]: I1203 12:54:45.257442 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-keystone-listener-7b66cf7bb6-fsc4x" event={"ID":"020bfa4c-3dba-43d2-9ac2-0bbbae660709","Type":"ContainerDied","Data":"27212e7a0c6918bbf263320719b8b0731ba8a5d005b2aed71d83eab909ffb6f0"} Dec 03 12:54:46 crc kubenswrapper[5052]: I1203 12:54:45.260430 5052 generic.go:334] "Generic (PLEG): container finished" podID="981c3156-805a-44d1-b4ec-f688a8bc3cf8" containerID="691c45bd03a54c99bcffac732eb2f453f8046a69209f84bde56957c8b1cd60f9" exitCode=143 Dec 03 12:54:46 crc kubenswrapper[5052]: I1203 12:54:45.260497 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"981c3156-805a-44d1-b4ec-f688a8bc3cf8","Type":"ContainerDied","Data":"691c45bd03a54c99bcffac732eb2f453f8046a69209f84bde56957c8b1cd60f9"} Dec 03 12:54:46 crc kubenswrapper[5052]: I1203 12:54:45.264532 5052 generic.go:334] "Generic (PLEG): container finished" podID="02edb9e9-3f42-4aea-9fe6-7efa332d5fb8" containerID="59b945b4df7623c03eb622cbe50e816dadd5224b5097d5e7e646cca960ca90e1" exitCode=2 Dec 03 12:54:46 crc kubenswrapper[5052]: I1203 12:54:45.264604 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"02edb9e9-3f42-4aea-9fe6-7efa332d5fb8","Type":"ContainerDied","Data":"59b945b4df7623c03eb622cbe50e816dadd5224b5097d5e7e646cca960ca90e1"} Dec 03 12:54:46 crc kubenswrapper[5052]: I1203 12:54:45.267632 5052 generic.go:334] "Generic (PLEG): container finished" podID="eefe83f8-0235-4b16-aefb-5ec7b2d92fcf" containerID="29117e54764872df757ac95e94dbab057dd9c2d6f5908051e97419768ed8628a" exitCode=0 Dec 03 12:54:46 crc kubenswrapper[5052]: I1203 12:54:45.267716 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-proxy-5479cb94c8-q2snc" event={"ID":"eefe83f8-0235-4b16-aefb-5ec7b2d92fcf","Type":"ContainerDied","Data":"29117e54764872df757ac95e94dbab057dd9c2d6f5908051e97419768ed8628a"} Dec 03 12:54:46 crc kubenswrapper[5052]: I1203 12:54:45.277212 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/keystone-c555-account-create-update-6zkgw"] Dec 03 12:54:46 crc kubenswrapper[5052]: I1203 12:54:45.278897 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage15-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage15-crc\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:46 crc kubenswrapper[5052]: I1203 12:54:45.278949 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d16285c0-7e62-4b3c-b3e8-a62b8abb49cb-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:46 crc kubenswrapper[5052]: I1203 12:54:45.298792 5052 generic.go:334] "Generic (PLEG): container finished" podID="022746a5-1c20-4729-8688-5903831f57ed" containerID="3157794aa44d4b21b9c104515fcfd5ee6e33807ee7b2753cff2132e2facd1f75" exitCode=143 Dec 03 12:54:46 crc kubenswrapper[5052]: I1203 12:54:45.298908 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/novacell10a46-account-delete-h8x7f" Dec 03 12:54:46 crc kubenswrapper[5052]: I1203 12:54:45.298946 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d16285c0-7e62-4b3c-b3e8-a62b8abb49cb-metrics-certs-tls-certs" (OuterVolumeSpecName: "metrics-certs-tls-certs") pod "d16285c0-7e62-4b3c-b3e8-a62b8abb49cb" (UID: "d16285c0-7e62-4b3c-b3e8-a62b8abb49cb"). InnerVolumeSpecName "metrics-certs-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:54:46 crc kubenswrapper[5052]: I1203 12:54:45.299015 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"022746a5-1c20-4729-8688-5903831f57ed","Type":"ContainerDied","Data":"3157794aa44d4b21b9c104515fcfd5ee6e33807ee7b2753cff2132e2facd1f75"} Dec 03 12:54:46 crc kubenswrapper[5052]: I1203 12:54:45.302644 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/keystone-db-create-zdrgs"] Dec 03 12:54:46 crc kubenswrapper[5052]: I1203 12:54:45.325599 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/keystone-db-create-zdrgs"] Dec 03 12:54:46 crc kubenswrapper[5052]: I1203 12:54:45.372137 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c4b7bd04-32be-43c6-82f1-968f2e368b22-openstack-config-secret" (OuterVolumeSpecName: "openstack-config-secret") pod "c4b7bd04-32be-43c6-82f1-968f2e368b22" (UID: "c4b7bd04-32be-43c6-82f1-968f2e368b22"). InnerVolumeSpecName "openstack-config-secret". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:54:46 crc kubenswrapper[5052]: I1203 12:54:45.375611 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-shdj6"] Dec 03 12:54:46 crc kubenswrapper[5052]: I1203 12:54:45.381794 5052 reconciler_common.go:293] "Volume detached for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/d16285c0-7e62-4b3c-b3e8-a62b8abb49cb-metrics-certs-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:46 crc kubenswrapper[5052]: I1203 12:54:45.381819 5052 reconciler_common.go:293] "Volume detached for volume \"openstack-config-secret\" (UniqueName: \"kubernetes.io/secret/c4b7bd04-32be-43c6-82f1-968f2e368b22-openstack-config-secret\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:46 crc kubenswrapper[5052]: I1203 12:54:45.418495 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3a5e94ea-faeb-4202-843a-bd691d8895ce-metrics-certs-tls-certs" (OuterVolumeSpecName: "metrics-certs-tls-certs") pod "3a5e94ea-faeb-4202-843a-bd691d8895ce" (UID: "3a5e94ea-faeb-4202-843a-bd691d8895ce"). InnerVolumeSpecName "metrics-certs-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:54:46 crc kubenswrapper[5052]: I1203 12:54:45.420190 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d16285c0-7e62-4b3c-b3e8-a62b8abb49cb-ovsdbserver-sb-tls-certs" (OuterVolumeSpecName: "ovsdbserver-sb-tls-certs") pod "d16285c0-7e62-4b3c-b3e8-a62b8abb49cb" (UID: "d16285c0-7e62-4b3c-b3e8-a62b8abb49cb"). InnerVolumeSpecName "ovsdbserver-sb-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:54:46 crc kubenswrapper[5052]: I1203 12:54:45.438101 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3a5e94ea-faeb-4202-843a-bd691d8895ce-ovsdbserver-nb-tls-certs" (OuterVolumeSpecName: "ovsdbserver-nb-tls-certs") pod "3a5e94ea-faeb-4202-843a-bd691d8895ce" (UID: "3a5e94ea-faeb-4202-843a-bd691d8895ce"). InnerVolumeSpecName "ovsdbserver-nb-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:54:46 crc kubenswrapper[5052]: I1203 12:54:45.461759 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/novacell10a46-account-delete-h8x7f" Dec 03 12:54:46 crc kubenswrapper[5052]: I1203 12:54:45.468209 5052 scope.go:117] "RemoveContainer" containerID="468387bd146ee57d9dbb25ad0e76ab64875756a52288e01c0d2ec19a6cddb217" Dec 03 12:54:46 crc kubenswrapper[5052]: I1203 12:54:45.483956 5052 reconciler_common.go:293] "Volume detached for volume \"ovsdbserver-nb-tls-certs\" (UniqueName: \"kubernetes.io/secret/3a5e94ea-faeb-4202-843a-bd691d8895ce-ovsdbserver-nb-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:46 crc kubenswrapper[5052]: I1203 12:54:45.483987 5052 reconciler_common.go:293] "Volume detached for volume \"ovsdbserver-sb-tls-certs\" (UniqueName: \"kubernetes.io/secret/d16285c0-7e62-4b3c-b3e8-a62b8abb49cb-ovsdbserver-sb-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:46 crc kubenswrapper[5052]: I1203 12:54:45.484000 5052 reconciler_common.go:293] "Volume detached for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/3a5e94ea-faeb-4202-843a-bd691d8895ce-metrics-certs-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:46 crc kubenswrapper[5052]: I1203 12:54:45.553090 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/openstack-galera-0" podUID="b3363d6f-656e-4e22-a3eb-b1016380a0c8" containerName="galera" containerID="cri-o://b8ecae5dcbbd55bfb5a128bba207c90d7a6b85badc20f265e843e97165d22a9c" gracePeriod=30 Dec 03 12:54:46 crc kubenswrapper[5052]: I1203 12:54:45.653309 5052 scope.go:117] "RemoveContainer" containerID="863f9a5dead36dfcd4e960256f81adece8f398e82c1cc6eb7b8c1e26766d0633" Dec 03 12:54:46 crc kubenswrapper[5052]: I1203 12:54:45.689206 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1e8517f3-2136-4b41-89bc-115f5c14b265" path="/var/lib/kubelet/pods/1e8517f3-2136-4b41-89bc-115f5c14b265/volumes" Dec 03 12:54:46 crc kubenswrapper[5052]: I1203 12:54:45.689978 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2ff4a526-fb1e-48e9-ae88-6b84ce5eaba2" path="/var/lib/kubelet/pods/2ff4a526-fb1e-48e9-ae88-6b84ce5eaba2/volumes" Dec 03 12:54:46 crc kubenswrapper[5052]: I1203 12:54:45.690630 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5f5af30b-fb44-40c4-94a2-33965abf72ef" path="/var/lib/kubelet/pods/5f5af30b-fb44-40c4-94a2-33965abf72ef/volumes" Dec 03 12:54:46 crc kubenswrapper[5052]: I1203 12:54:45.691746 5052 scope.go:117] "RemoveContainer" containerID="5dbfbe34fded1c834b0fb2487d51d6da9334318d91fb67fd92b8f7457b35a22f" Dec 03 12:54:46 crc kubenswrapper[5052]: I1203 12:54:45.692328 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="69d15184-ac62-4679-aadf-61fd17294318" path="/var/lib/kubelet/pods/69d15184-ac62-4679-aadf-61fd17294318/volumes" Dec 03 12:54:46 crc kubenswrapper[5052]: I1203 12:54:45.699266 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6bed880a-56d4-4fed-bec7-6fea476ceb58" path="/var/lib/kubelet/pods/6bed880a-56d4-4fed-bec7-6fea476ceb58/volumes" Dec 03 12:54:46 crc kubenswrapper[5052]: I1203 12:54:45.706825 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6e6cb4e4-e8b3-4e38-8c40-f4829e4348a9" path="/var/lib/kubelet/pods/6e6cb4e4-e8b3-4e38-8c40-f4829e4348a9/volumes" Dec 03 12:54:46 crc kubenswrapper[5052]: I1203 12:54:45.708369 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b2498fe2-65ee-4470-a52b-e84da2ea42fb" path="/var/lib/kubelet/pods/b2498fe2-65ee-4470-a52b-e84da2ea42fb/volumes" Dec 03 12:54:46 crc kubenswrapper[5052]: I1203 12:54:45.709343 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c4b7bd04-32be-43c6-82f1-968f2e368b22" path="/var/lib/kubelet/pods/c4b7bd04-32be-43c6-82f1-968f2e368b22/volumes" Dec 03 12:54:46 crc kubenswrapper[5052]: I1203 12:54:45.710449 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cd29fd9a-ac8a-4424-b6ac-e4b81bf292f4" path="/var/lib/kubelet/pods/cd29fd9a-ac8a-4424-b6ac-e4b81bf292f4/volumes" Dec 03 12:54:46 crc kubenswrapper[5052]: I1203 12:54:45.713748 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-nb-0"] Dec 03 12:54:46 crc kubenswrapper[5052]: I1203 12:54:45.713778 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-nb-0"] Dec 03 12:54:46 crc kubenswrapper[5052]: I1203 12:54:45.713795 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-sb-0"] Dec 03 12:54:46 crc kubenswrapper[5052]: I1203 12:54:45.727425 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ovsdbserver-sb-0"] Dec 03 12:54:46 crc kubenswrapper[5052]: I1203 12:54:45.728374 5052 scope.go:117] "RemoveContainer" containerID="4a6055cf9cff80734b8d6c28ebdc2f8003fb5d319d561fbaf2f95617ced8483e" Dec 03 12:54:46 crc kubenswrapper[5052]: E1203 12:54:45.805179 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/rabbitmq-cell1-config-data: configmap "rabbitmq-cell1-config-data" not found Dec 03 12:54:46 crc kubenswrapper[5052]: E1203 12:54:45.805254 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/568addd3-4ad3-47c3-adae-693691e7cd40-config-data podName:568addd3-4ad3-47c3-adae-693691e7cd40 nodeName:}" failed. No retries permitted until 2025-12-03 12:54:49.805233796 +0000 UTC m=+5680.393475580 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "config-data" (UniqueName: "kubernetes.io/configmap/568addd3-4ad3-47c3-adae-693691e7cd40-config-data") pod "rabbitmq-cell1-server-0" (UID: "568addd3-4ad3-47c3-adae-693691e7cd40") : configmap "rabbitmq-cell1-config-data" not found Dec 03 12:54:46 crc kubenswrapper[5052]: E1203 12:54:45.806531 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="06bf7c042266ac81635b44d190310f81518aa47f7b8008ddde3f54ca515b15fd" cmd=["/usr/local/bin/container-scripts/status_check.sh"] Dec 03 12:54:46 crc kubenswrapper[5052]: E1203 12:54:45.809464 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="06bf7c042266ac81635b44d190310f81518aa47f7b8008ddde3f54ca515b15fd" cmd=["/usr/local/bin/container-scripts/status_check.sh"] Dec 03 12:54:46 crc kubenswrapper[5052]: E1203 12:54:45.812342 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="06bf7c042266ac81635b44d190310f81518aa47f7b8008ddde3f54ca515b15fd" cmd=["/usr/local/bin/container-scripts/status_check.sh"] Dec 03 12:54:46 crc kubenswrapper[5052]: E1203 12:54:45.812380 5052 prober.go:104] "Probe errored" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" probeType="Readiness" pod="openstack-kuttl-tests/ovn-northd-0" podUID="3b93bc75-db87-4677-be2f-910d07bb696c" containerName="ovn-northd" Dec 03 12:54:46 crc kubenswrapper[5052]: E1203 12:54:46.170743 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="ec5a6e0000f85ac80b044c5e702c36b00720989c7bd27aac0130a6739a2d1bd8" cmd=["/usr/bin/pgrep","-r","DRST","nova-scheduler"] Dec 03 12:54:46 crc kubenswrapper[5052]: E1203 12:54:46.171837 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="ec5a6e0000f85ac80b044c5e702c36b00720989c7bd27aac0130a6739a2d1bd8" cmd=["/usr/bin/pgrep","-r","DRST","nova-scheduler"] Dec 03 12:54:46 crc kubenswrapper[5052]: E1203 12:54:46.173561 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="ec5a6e0000f85ac80b044c5e702c36b00720989c7bd27aac0130a6739a2d1bd8" cmd=["/usr/bin/pgrep","-r","DRST","nova-scheduler"] Dec 03 12:54:46 crc kubenswrapper[5052]: E1203 12:54:46.173590 5052 prober.go:104] "Probe errored" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" probeType="Readiness" pod="openstack-kuttl-tests/nova-scheduler-0" podUID="4ec49e76-2597-47d6-811b-f65cbe0580ff" containerName="nova-scheduler-scheduler" Dec 03 12:54:46 crc kubenswrapper[5052]: E1203 12:54:46.239902 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of eb28bc8efa7fd43a1e5b7da097e9679a59f690173e27b20d4dae0a08837dde86 is running failed: container process not found" containerID="eb28bc8efa7fd43a1e5b7da097e9679a59f690173e27b20d4dae0a08837dde86" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:54:46 crc kubenswrapper[5052]: E1203 12:54:46.240377 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of eb28bc8efa7fd43a1e5b7da097e9679a59f690173e27b20d4dae0a08837dde86 is running failed: container process not found" containerID="eb28bc8efa7fd43a1e5b7da097e9679a59f690173e27b20d4dae0a08837dde86" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:54:46 crc kubenswrapper[5052]: E1203 12:54:46.242711 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of eb28bc8efa7fd43a1e5b7da097e9679a59f690173e27b20d4dae0a08837dde86 is running failed: container process not found" containerID="eb28bc8efa7fd43a1e5b7da097e9679a59f690173e27b20d4dae0a08837dde86" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:54:46 crc kubenswrapper[5052]: E1203 12:54:46.242778 5052 prober.go:104] "Probe errored" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of eb28bc8efa7fd43a1e5b7da097e9679a59f690173e27b20d4dae0a08837dde86 is running failed: container process not found" probeType="Readiness" pod="openstack-kuttl-tests/nova-cell0-conductor-0" podUID="3c6d824e-c7da-41ff-adde-dbafb7535476" containerName="nova-cell0-conductor-conductor" Dec 03 12:54:46 crc kubenswrapper[5052]: I1203 12:54:46.295089 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/placement-db-create-7cr6f"] Dec 03 12:54:46 crc kubenswrapper[5052]: I1203 12:54:46.302421 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/placement-db-create-7cr6f"] Dec 03 12:54:46 crc kubenswrapper[5052]: I1203 12:54:46.311490 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican6e62-account-delete-6skhm" event={"ID":"d9bf91f1-099c-4e72-97cf-1828241aeaac","Type":"ContainerStarted","Data":"11ddaf754d754f510a35d327364ee83e653bb8146394aaffa51c37770ca3011a"} Dec 03 12:54:46 crc kubenswrapper[5052]: I1203 12:54:46.311552 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican6e62-account-delete-6skhm" event={"ID":"d9bf91f1-099c-4e72-97cf-1828241aeaac","Type":"ContainerStarted","Data":"f24f8c65ed321344bff617dc2efb47214ab4325842fbde63688138a762aadbdb"} Dec 03 12:54:46 crc kubenswrapper[5052]: I1203 12:54:46.312295 5052 kubelet_pods.go:1007] "Unable to retrieve pull secret, the image pull may not succeed." pod="openstack-kuttl-tests/barbican6e62-account-delete-6skhm" secret="" err="secret \"galera-openstack-dockercfg-fkczj\" not found" Dec 03 12:54:46 crc kubenswrapper[5052]: I1203 12:54:46.320577 5052 generic.go:334] "Generic (PLEG): container finished" podID="3c6d824e-c7da-41ff-adde-dbafb7535476" containerID="eb28bc8efa7fd43a1e5b7da097e9679a59f690173e27b20d4dae0a08837dde86" exitCode=0 Dec 03 12:54:46 crc kubenswrapper[5052]: I1203 12:54:46.320615 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-conductor-0" event={"ID":"3c6d824e-c7da-41ff-adde-dbafb7535476","Type":"ContainerDied","Data":"eb28bc8efa7fd43a1e5b7da097e9679a59f690173e27b20d4dae0a08837dde86"} Dec 03 12:54:46 crc kubenswrapper[5052]: I1203 12:54:46.324865 5052 generic.go:334] "Generic (PLEG): container finished" podID="02edb9e9-3f42-4aea-9fe6-7efa332d5fb8" containerID="990e0730d2387a55431e533e2f7232d5369fd95354ef6d99da9341e99b6b7c58" exitCode=0 Dec 03 12:54:46 crc kubenswrapper[5052]: I1203 12:54:46.324886 5052 generic.go:334] "Generic (PLEG): container finished" podID="02edb9e9-3f42-4aea-9fe6-7efa332d5fb8" containerID="86ed75c4a9b117c08c2a3ecec9878ef557fc9d6d68fea18aa83751770d37d20b" exitCode=0 Dec 03 12:54:46 crc kubenswrapper[5052]: I1203 12:54:46.324927 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"02edb9e9-3f42-4aea-9fe6-7efa332d5fb8","Type":"ContainerDied","Data":"990e0730d2387a55431e533e2f7232d5369fd95354ef6d99da9341e99b6b7c58"} Dec 03 12:54:46 crc kubenswrapper[5052]: I1203 12:54:46.324955 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"02edb9e9-3f42-4aea-9fe6-7efa332d5fb8","Type":"ContainerDied","Data":"86ed75c4a9b117c08c2a3ecec9878ef557fc9d6d68fea18aa83751770d37d20b"} Dec 03 12:54:46 crc kubenswrapper[5052]: I1203 12:54:46.331863 5052 generic.go:334] "Generic (PLEG): container finished" podID="7bed2e44-93da-465d-9e10-6879ff1c82f5" containerID="a27fb96a63938a81b5f484c71200a5edcde938947a173488bc4333ca4777eefc" exitCode=0 Dec 03 12:54:46 crc kubenswrapper[5052]: I1203 12:54:46.331955 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-shdj6" event={"ID":"7bed2e44-93da-465d-9e10-6879ff1c82f5","Type":"ContainerDied","Data":"a27fb96a63938a81b5f484c71200a5edcde938947a173488bc4333ca4777eefc"} Dec 03 12:54:46 crc kubenswrapper[5052]: I1203 12:54:46.332009 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-shdj6" event={"ID":"7bed2e44-93da-465d-9e10-6879ff1c82f5","Type":"ContainerStarted","Data":"a586b878b7096b7313dc8e2f549516d2b891f1eb73b67e3bdb6b74dd77488878"} Dec 03 12:54:46 crc kubenswrapper[5052]: I1203 12:54:46.341480 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron6c26-account-delete-d7wtw" event={"ID":"941d063f-c834-47a1-bc0c-2096b005e09c","Type":"ContainerStarted","Data":"c89e8180cefdc0d2331c24dc4ecea16dbdbd25fd8e2135fbbbde4e84750348ce"} Dec 03 12:54:46 crc kubenswrapper[5052]: I1203 12:54:46.346056 5052 generic.go:334] "Generic (PLEG): container finished" podID="43b07f3c-f9d7-451b-b0ab-61b293cff35d" containerID="92269ae03a0728af8ad79f1a3cc73e6323d763c9c370a177f9ea91e0fc11bf91" exitCode=0 Dec 03 12:54:46 crc kubenswrapper[5052]: I1203 12:54:46.346111 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" event={"ID":"43b07f3c-f9d7-451b-b0ab-61b293cff35d","Type":"ContainerDied","Data":"92269ae03a0728af8ad79f1a3cc73e6323d763c9c370a177f9ea91e0fc11bf91"} Dec 03 12:54:46 crc kubenswrapper[5052]: I1203 12:54:46.348546 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/placement-bfa4-account-create-update-84nm8"] Dec 03 12:54:46 crc kubenswrapper[5052]: I1203 12:54:46.348678 5052 generic.go:334] "Generic (PLEG): container finished" podID="b2933134-7226-423a-8120-3148845b808a" containerID="6ef42ed115f2247455a35e6dba9e2519ebc77a9c515e4efa379aa58e493e2dea" exitCode=0 Dec 03 12:54:46 crc kubenswrapper[5052]: I1203 12:54:46.348736 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-worker-b7f5dfb6f-pj6bx" event={"ID":"b2933134-7226-423a-8120-3148845b808a","Type":"ContainerDied","Data":"6ef42ed115f2247455a35e6dba9e2519ebc77a9c515e4efa379aa58e493e2dea"} Dec 03 12:54:46 crc kubenswrapper[5052]: I1203 12:54:46.349471 5052 kubelet_pods.go:1007] "Unable to retrieve pull secret, the image pull may not succeed." pod="openstack-kuttl-tests/neutron6c26-account-delete-d7wtw" secret="" err="secret \"galera-openstack-dockercfg-fkczj\" not found" Dec 03 12:54:46 crc kubenswrapper[5052]: I1203 12:54:46.351995 5052 generic.go:334] "Generic (PLEG): container finished" podID="c4d5c183-8df4-4c43-9afb-2b28c6aae9e1" containerID="cf0936e9c4c87ab125307449bfe7ff21ea78778fbaef0f8f0360dbecdf7e1591" exitCode=1 Dec 03 12:54:46 crc kubenswrapper[5052]: I1203 12:54:46.352073 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placementbfa4-account-delete-flc9f" event={"ID":"c4d5c183-8df4-4c43-9afb-2b28c6aae9e1","Type":"ContainerDied","Data":"cf0936e9c4c87ab125307449bfe7ff21ea78778fbaef0f8f0360dbecdf7e1591"} Dec 03 12:54:46 crc kubenswrapper[5052]: I1203 12:54:46.352936 5052 kubelet_pods.go:1007] "Unable to retrieve pull secret, the image pull may not succeed." pod="openstack-kuttl-tests/placementbfa4-account-delete-flc9f" secret="" err="secret \"galera-openstack-dockercfg-fkczj\" not found" Dec 03 12:54:46 crc kubenswrapper[5052]: I1203 12:54:46.352977 5052 scope.go:117] "RemoveContainer" containerID="cf0936e9c4c87ab125307449bfe7ff21ea78778fbaef0f8f0360dbecdf7e1591" Dec 03 12:54:46 crc kubenswrapper[5052]: I1203 12:54:46.366652 5052 generic.go:334] "Generic (PLEG): container finished" podID="3c553114-e0c8-4af9-992f-5421b31497ed" containerID="77bdaa2aad097bc96adc6a7e702b0e7e164f8b659c20495278f4d679ff5255fc" exitCode=2 Dec 03 12:54:46 crc kubenswrapper[5052]: I1203 12:54:46.366756 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/kube-state-metrics-0" event={"ID":"3c553114-e0c8-4af9-992f-5421b31497ed","Type":"ContainerDied","Data":"77bdaa2aad097bc96adc6a7e702b0e7e164f8b659c20495278f4d679ff5255fc"} Dec 03 12:54:46 crc kubenswrapper[5052]: I1203 12:54:46.383171 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cindera2a9-account-delete-pmspg" event={"ID":"e8a26759-2926-41cf-a1ec-286fc9962bb3","Type":"ContainerStarted","Data":"42f16d033db22635e368a85dc53ea3877fb872db34db75e3fb79daeb9c825a36"} Dec 03 12:54:46 crc kubenswrapper[5052]: I1203 12:54:46.383232 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cindera2a9-account-delete-pmspg" event={"ID":"e8a26759-2926-41cf-a1ec-286fc9962bb3","Type":"ContainerStarted","Data":"7f88d68507ef2d369c5fe3dce319cce0c03f0265b1bc25ed04d6d8be9cafb545"} Dec 03 12:54:46 crc kubenswrapper[5052]: I1203 12:54:46.383918 5052 kubelet_pods.go:1007] "Unable to retrieve pull secret, the image pull may not succeed." pod="openstack-kuttl-tests/cindera2a9-account-delete-pmspg" secret="" err="secret \"galera-openstack-dockercfg-fkczj\" not found" Dec 03 12:54:46 crc kubenswrapper[5052]: I1203 12:54:46.394849 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/placement-bfa4-account-create-update-84nm8"] Dec 03 12:54:46 crc kubenswrapper[5052]: E1203 12:54:46.425790 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:54:46 crc kubenswrapper[5052]: E1203 12:54:46.425890 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c4d5c183-8df4-4c43-9afb-2b28c6aae9e1-operator-scripts podName:c4d5c183-8df4-4c43-9afb-2b28c6aae9e1 nodeName:}" failed. No retries permitted until 2025-12-03 12:54:46.925867284 +0000 UTC m=+5677.514109048 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/c4d5c183-8df4-4c43-9afb-2b28c6aae9e1-operator-scripts") pod "placementbfa4-account-delete-flc9f" (UID: "c4d5c183-8df4-4c43-9afb-2b28c6aae9e1") : configmap "openstack-scripts" not found Dec 03 12:54:46 crc kubenswrapper[5052]: E1203 12:54:46.426148 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:54:46 crc kubenswrapper[5052]: E1203 12:54:46.426215 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d9bf91f1-099c-4e72-97cf-1828241aeaac-operator-scripts podName:d9bf91f1-099c-4e72-97cf-1828241aeaac nodeName:}" failed. No retries permitted until 2025-12-03 12:54:46.926192943 +0000 UTC m=+5677.514434717 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/d9bf91f1-099c-4e72-97cf-1828241aeaac-operator-scripts") pod "barbican6e62-account-delete-6skhm" (UID: "d9bf91f1-099c-4e72-97cf-1828241aeaac") : configmap "openstack-scripts" not found Dec 03 12:54:46 crc kubenswrapper[5052]: E1203 12:54:46.427219 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:54:46 crc kubenswrapper[5052]: E1203 12:54:46.427265 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/941d063f-c834-47a1-bc0c-2096b005e09c-operator-scripts podName:941d063f-c834-47a1-bc0c-2096b005e09c nodeName:}" failed. No retries permitted until 2025-12-03 12:54:46.927254274 +0000 UTC m=+5677.515496038 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/941d063f-c834-47a1-bc0c-2096b005e09c-operator-scripts") pod "neutron6c26-account-delete-d7wtw" (UID: "941d063f-c834-47a1-bc0c-2096b005e09c") : configmap "openstack-scripts" not found Dec 03 12:54:46 crc kubenswrapper[5052]: I1203 12:54:46.439208 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/placementbfa4-account-delete-flc9f"] Dec 03 12:54:46 crc kubenswrapper[5052]: I1203 12:54:46.457547 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/barbican6e62-account-delete-6skhm" podStartSLOduration=4.457521849 podStartE2EDuration="4.457521849s" podCreationTimestamp="2025-12-03 12:54:42 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:54:46.32816693 +0000 UTC m=+5676.916408724" watchObservedRunningTime="2025-12-03 12:54:46.457521849 +0000 UTC m=+5677.045763613" Dec 03 12:54:46 crc kubenswrapper[5052]: I1203 12:54:46.464364 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance7c80-account-delete-knmlm" event={"ID":"79efd225-5ce4-428f-aef9-3f4884931fef","Type":"ContainerStarted","Data":"a1b6cbc8b7e6a360fb1952ecd8d7fe6374e2e1fc02dd0db8aad15d96a40e30e9"} Dec 03 12:54:46 crc kubenswrapper[5052]: I1203 12:54:46.469129 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance7c80-account-delete-knmlm" event={"ID":"79efd225-5ce4-428f-aef9-3f4884931fef","Type":"ContainerStarted","Data":"7e9917be03250448576169c69cea7030e3fc164ff5f88cdf282c32c5d212bbdb"} Dec 03 12:54:46 crc kubenswrapper[5052]: I1203 12:54:46.471498 5052 kubelet_pods.go:1007] "Unable to retrieve pull secret, the image pull may not succeed." pod="openstack-kuttl-tests/glance7c80-account-delete-knmlm" secret="" err="secret \"galera-openstack-dockercfg-fkczj\" not found" Dec 03 12:54:46 crc kubenswrapper[5052]: I1203 12:54:46.482284 5052 generic.go:334] "Generic (PLEG): container finished" podID="020bfa4c-3dba-43d2-9ac2-0bbbae660709" containerID="bab03b01c66d5661abe6c60c8c81496efcefb2ea684c5751d1ee1113198a69ca" exitCode=0 Dec 03 12:54:46 crc kubenswrapper[5052]: I1203 12:54:46.482371 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/novacell10a46-account-delete-h8x7f" Dec 03 12:54:46 crc kubenswrapper[5052]: I1203 12:54:46.482403 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-keystone-listener-7b66cf7bb6-fsc4x" event={"ID":"020bfa4c-3dba-43d2-9ac2-0bbbae660709","Type":"ContainerDied","Data":"bab03b01c66d5661abe6c60c8c81496efcefb2ea684c5751d1ee1113198a69ca"} Dec 03 12:54:46 crc kubenswrapper[5052]: I1203 12:54:46.483139 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/cinder-api-0" podUID="ed40ab59-7f38-4b2a-ad68-85592fe22ef2" containerName="cinder-api" probeResult="failure" output="Get \"https://10.217.0.185:8776/healthcheck\": read tcp 10.217.0.2:57254->10.217.0.185:8776: read: connection reset by peer" Dec 03 12:54:46 crc kubenswrapper[5052]: I1203 12:54:46.515388 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/neutron6c26-account-delete-d7wtw" podStartSLOduration=5.515366383 podStartE2EDuration="5.515366383s" podCreationTimestamp="2025-12-03 12:54:41 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:54:46.427574033 +0000 UTC m=+5677.015815797" watchObservedRunningTime="2025-12-03 12:54:46.515366383 +0000 UTC m=+5677.103608147" Dec 03 12:54:46 crc kubenswrapper[5052]: E1203 12:54:46.526602 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:54:46 crc kubenswrapper[5052]: E1203 12:54:46.526668 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e8a26759-2926-41cf-a1ec-286fc9962bb3-operator-scripts podName:e8a26759-2926-41cf-a1ec-286fc9962bb3 nodeName:}" failed. No retries permitted until 2025-12-03 12:54:47.026649216 +0000 UTC m=+5677.614890980 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/e8a26759-2926-41cf-a1ec-286fc9962bb3-operator-scripts") pod "cindera2a9-account-delete-pmspg" (UID: "e8a26759-2926-41cf-a1ec-286fc9962bb3") : configmap "openstack-scripts" not found Dec 03 12:54:46 crc kubenswrapper[5052]: I1203 12:54:46.562159 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/cindera2a9-account-delete-pmspg" podStartSLOduration=4.562139041 podStartE2EDuration="4.562139041s" podCreationTimestamp="2025-12-03 12:54:42 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:54:46.481964198 +0000 UTC m=+5677.070205962" watchObservedRunningTime="2025-12-03 12:54:46.562139041 +0000 UTC m=+5677.150380805" Dec 03 12:54:46 crc kubenswrapper[5052]: I1203 12:54:46.566831 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/glance7c80-account-delete-knmlm" podStartSLOduration=5.566808715 podStartE2EDuration="5.566808715s" podCreationTimestamp="2025-12-03 12:54:41 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:54:46.503636278 +0000 UTC m=+5677.091878062" watchObservedRunningTime="2025-12-03 12:54:46.566808715 +0000 UTC m=+5677.155050479" Dec 03 12:54:46 crc kubenswrapper[5052]: E1203 12:54:46.629065 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:54:46 crc kubenswrapper[5052]: E1203 12:54:46.629139 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/79efd225-5ce4-428f-aef9-3f4884931fef-operator-scripts podName:79efd225-5ce4-428f-aef9-3f4884931fef nodeName:}" failed. No retries permitted until 2025-12-03 12:54:47.129124427 +0000 UTC m=+5677.717366191 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/79efd225-5ce4-428f-aef9-3f4884931fef-operator-scripts") pod "glance7c80-account-delete-knmlm" (UID: "79efd225-5ce4-428f-aef9-3f4884931fef") : configmap "openstack-scripts" not found Dec 03 12:54:46 crc kubenswrapper[5052]: I1203 12:54:46.713321 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/neutron-db-create-mn57r"] Dec 03 12:54:46 crc kubenswrapper[5052]: I1203 12:54:46.733667 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/neutron-db-create-mn57r"] Dec 03 12:54:46 crc kubenswrapper[5052]: I1203 12:54:46.756855 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/neutron6c26-account-delete-d7wtw"] Dec 03 12:54:46 crc kubenswrapper[5052]: I1203 12:54:46.772167 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/neutron-6c26-account-create-update-b4jfv"] Dec 03 12:54:46 crc kubenswrapper[5052]: I1203 12:54:46.783872 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/neutron-6c26-account-create-update-b4jfv"] Dec 03 12:54:46 crc kubenswrapper[5052]: I1203 12:54:46.903693 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/placement-7b465d48b8-4bj69" podUID="8df65ee4-b91d-4ffd-bff2-65e5e6559130" containerName="placement-api" probeResult="failure" output="Get \"https://10.217.0.176:8778/\": read tcp 10.217.0.2:57212->10.217.0.176:8778: read: connection reset by peer" Dec 03 12:54:46 crc kubenswrapper[5052]: I1203 12:54:46.903697 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/placement-7b465d48b8-4bj69" podUID="8df65ee4-b91d-4ffd-bff2-65e5e6559130" containerName="placement-log" probeResult="failure" output="Get \"https://10.217.0.176:8778/\": read tcp 10.217.0.2:57228->10.217.0.176:8778: read: connection reset by peer" Dec 03 12:54:46 crc kubenswrapper[5052]: I1203 12:54:46.933983 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-db-create-42dbc"] Dec 03 12:54:46 crc kubenswrapper[5052]: E1203 12:54:46.941816 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:54:46 crc kubenswrapper[5052]: E1203 12:54:46.941882 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d9bf91f1-099c-4e72-97cf-1828241aeaac-operator-scripts podName:d9bf91f1-099c-4e72-97cf-1828241aeaac nodeName:}" failed. No retries permitted until 2025-12-03 12:54:47.941868091 +0000 UTC m=+5678.530109855 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/d9bf91f1-099c-4e72-97cf-1828241aeaac-operator-scripts") pod "barbican6e62-account-delete-6skhm" (UID: "d9bf91f1-099c-4e72-97cf-1828241aeaac") : configmap "openstack-scripts" not found Dec 03 12:54:46 crc kubenswrapper[5052]: E1203 12:54:46.942179 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:54:46 crc kubenswrapper[5052]: E1203 12:54:46.942270 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:54:46 crc kubenswrapper[5052]: E1203 12:54:46.942356 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c4d5c183-8df4-4c43-9afb-2b28c6aae9e1-operator-scripts podName:c4d5c183-8df4-4c43-9afb-2b28c6aae9e1 nodeName:}" failed. No retries permitted until 2025-12-03 12:54:47.942282533 +0000 UTC m=+5678.530524367 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/c4d5c183-8df4-4c43-9afb-2b28c6aae9e1-operator-scripts") pod "placementbfa4-account-delete-flc9f" (UID: "c4d5c183-8df4-4c43-9afb-2b28c6aae9e1") : configmap "openstack-scripts" not found Dec 03 12:54:46 crc kubenswrapper[5052]: E1203 12:54:46.942430 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/941d063f-c834-47a1-bc0c-2096b005e09c-operator-scripts podName:941d063f-c834-47a1-bc0c-2096b005e09c nodeName:}" failed. No retries permitted until 2025-12-03 12:54:47.942417966 +0000 UTC m=+5678.530659780 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/941d063f-c834-47a1-bc0c-2096b005e09c-operator-scripts") pod "neutron6c26-account-delete-d7wtw" (UID: "941d063f-c834-47a1-bc0c-2096b005e09c") : configmap "openstack-scripts" not found Dec 03 12:54:46 crc kubenswrapper[5052]: I1203 12:54:46.961598 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/glance-db-create-42dbc"] Dec 03 12:54:46 crc kubenswrapper[5052]: I1203 12:54:46.977364 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-7c80-account-create-update-kl5f4"] Dec 03 12:54:47 crc kubenswrapper[5052]: I1203 12:54:47.001542 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance7c80-account-delete-knmlm"] Dec 03 12:54:47 crc kubenswrapper[5052]: E1203 12:54:47.050051 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:54:47 crc kubenswrapper[5052]: E1203 12:54:47.050137 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e8a26759-2926-41cf-a1ec-286fc9962bb3-operator-scripts podName:e8a26759-2926-41cf-a1ec-286fc9962bb3 nodeName:}" failed. No retries permitted until 2025-12-03 12:54:48.050120237 +0000 UTC m=+5678.638362001 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/e8a26759-2926-41cf-a1ec-286fc9962bb3-operator-scripts") pod "cindera2a9-account-delete-pmspg" (UID: "e8a26759-2926-41cf-a1ec-286fc9962bb3") : configmap "openstack-scripts" not found Dec 03 12:54:47 crc kubenswrapper[5052]: I1203 12:54:47.070795 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/glance-7c80-account-create-update-kl5f4"] Dec 03 12:54:47 crc kubenswrapper[5052]: E1203 12:54:47.152842 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:54:47 crc kubenswrapper[5052]: E1203 12:54:47.152905 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/79efd225-5ce4-428f-aef9-3f4884931fef-operator-scripts podName:79efd225-5ce4-428f-aef9-3f4884931fef nodeName:}" failed. No retries permitted until 2025-12-03 12:54:48.152890076 +0000 UTC m=+5678.741131840 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/79efd225-5ce4-428f-aef9-3f4884931fef-operator-scripts") pod "glance7c80-account-delete-knmlm" (UID: "79efd225-5ce4-428f-aef9-3f4884931fef") : configmap "openstack-scripts" not found Dec 03 12:54:47 crc kubenswrapper[5052]: I1203 12:54:47.160571 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder-db-create-9vhcz"] Dec 03 12:54:47 crc kubenswrapper[5052]: I1203 12:54:47.184583 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/cinder-db-create-9vhcz"] Dec 03 12:54:47 crc kubenswrapper[5052]: I1203 12:54:47.215563 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cindera2a9-account-delete-pmspg"] Dec 03 12:54:47 crc kubenswrapper[5052]: I1203 12:54:47.233582 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder-a2a9-account-create-update-2z8wq"] Dec 03 12:54:47 crc kubenswrapper[5052]: I1203 12:54:47.234487 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/cinder-a2a9-account-create-update-2z8wq"] Dec 03 12:54:47 crc kubenswrapper[5052]: I1203 12:54:47.254864 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/6d56c6dc-38c9-42cf-ac6d-04ba49860d31-operator-scripts\") pod \"novacell10a46-account-delete-h8x7f\" (UID: \"6d56c6dc-38c9-42cf-ac6d-04ba49860d31\") " pod="openstack-kuttl-tests/novacell10a46-account-delete-h8x7f" Dec 03 12:54:47 crc kubenswrapper[5052]: I1203 12:54:47.255381 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-hs7rk\" (UniqueName: \"kubernetes.io/projected/6d56c6dc-38c9-42cf-ac6d-04ba49860d31-kube-api-access-hs7rk\") pod \"novacell10a46-account-delete-h8x7f\" (UID: \"6d56c6dc-38c9-42cf-ac6d-04ba49860d31\") " pod="openstack-kuttl-tests/novacell10a46-account-delete-h8x7f" Dec 03 12:54:47 crc kubenswrapper[5052]: E1203 12:54:47.254904 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-cell1-scripts: configmap "openstack-cell1-scripts" not found Dec 03 12:54:47 crc kubenswrapper[5052]: E1203 12:54:47.255876 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6d56c6dc-38c9-42cf-ac6d-04ba49860d31-operator-scripts podName:6d56c6dc-38c9-42cf-ac6d-04ba49860d31 nodeName:}" failed. No retries permitted until 2025-12-03 12:54:51.255860651 +0000 UTC m=+5681.844102415 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/6d56c6dc-38c9-42cf-ac6d-04ba49860d31-operator-scripts") pod "novacell10a46-account-delete-h8x7f" (UID: "6d56c6dc-38c9-42cf-ac6d-04ba49860d31") : configmap "openstack-cell1-scripts" not found Dec 03 12:54:47 crc kubenswrapper[5052]: E1203 12:54:47.262922 5052 projected.go:194] Error preparing data for projected volume kube-api-access-hs7rk for pod openstack-kuttl-tests/novacell10a46-account-delete-h8x7f: failed to fetch token: serviceaccounts "galera-openstack-cell1" not found Dec 03 12:54:47 crc kubenswrapper[5052]: E1203 12:54:47.263008 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6d56c6dc-38c9-42cf-ac6d-04ba49860d31-kube-api-access-hs7rk podName:6d56c6dc-38c9-42cf-ac6d-04ba49860d31 nodeName:}" failed. No retries permitted until 2025-12-03 12:54:51.262988144 +0000 UTC m=+5681.851229908 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-hs7rk" (UniqueName: "kubernetes.io/projected/6d56c6dc-38c9-42cf-ac6d-04ba49860d31-kube-api-access-hs7rk") pod "novacell10a46-account-delete-h8x7f" (UID: "6d56c6dc-38c9-42cf-ac6d-04ba49860d31") : failed to fetch token: serviceaccounts "galera-openstack-cell1" not found Dec 03 12:54:47 crc kubenswrapper[5052]: I1203 12:54:47.319609 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-db-create-66nr2"] Dec 03 12:54:47 crc kubenswrapper[5052]: I1203 12:54:47.331879 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/barbican-db-create-66nr2"] Dec 03 12:54:47 crc kubenswrapper[5052]: I1203 12:54:47.364241 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican6e62-account-delete-6skhm"] Dec 03 12:54:47 crc kubenswrapper[5052]: I1203 12:54:47.394985 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:54:47 crc kubenswrapper[5052]: I1203 12:54:47.439585 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-6e62-account-create-update-5xsrz"] Dec 03 12:54:47 crc kubenswrapper[5052]: I1203 12:54:47.465343 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s9695\" (UniqueName: \"kubernetes.io/projected/3c6d824e-c7da-41ff-adde-dbafb7535476-kube-api-access-s9695\") pod \"3c6d824e-c7da-41ff-adde-dbafb7535476\" (UID: \"3c6d824e-c7da-41ff-adde-dbafb7535476\") " Dec 03 12:54:47 crc kubenswrapper[5052]: I1203 12:54:47.465654 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/3c6d824e-c7da-41ff-adde-dbafb7535476-config-data\") pod \"3c6d824e-c7da-41ff-adde-dbafb7535476\" (UID: \"3c6d824e-c7da-41ff-adde-dbafb7535476\") " Dec 03 12:54:47 crc kubenswrapper[5052]: I1203 12:54:47.465844 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3c6d824e-c7da-41ff-adde-dbafb7535476-combined-ca-bundle\") pod \"3c6d824e-c7da-41ff-adde-dbafb7535476\" (UID: \"3c6d824e-c7da-41ff-adde-dbafb7535476\") " Dec 03 12:54:47 crc kubenswrapper[5052]: I1203 12:54:47.467637 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/barbican-6e62-account-create-update-5xsrz"] Dec 03 12:54:47 crc kubenswrapper[5052]: E1203 12:54:47.548844 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="a0d54bbd0da0ce78d0c6c5dcb48378864ee5ffb0c8e5280b729c579e83ef32f1" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:54:47 crc kubenswrapper[5052]: I1203 12:54:47.557264 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3c6d824e-c7da-41ff-adde-dbafb7535476-kube-api-access-s9695" (OuterVolumeSpecName: "kube-api-access-s9695") pod "3c6d824e-c7da-41ff-adde-dbafb7535476" (UID: "3c6d824e-c7da-41ff-adde-dbafb7535476"). InnerVolumeSpecName "kube-api-access-s9695". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:54:47 crc kubenswrapper[5052]: E1203 12:54:47.568492 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="a0d54bbd0da0ce78d0c6c5dcb48378864ee5ffb0c8e5280b729c579e83ef32f1" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:54:47 crc kubenswrapper[5052]: I1203 12:54:47.569814 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-s9695\" (UniqueName: \"kubernetes.io/projected/3c6d824e-c7da-41ff-adde-dbafb7535476-kube-api-access-s9695\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:47 crc kubenswrapper[5052]: E1203 12:54:47.570528 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="a0d54bbd0da0ce78d0c6c5dcb48378864ee5ffb0c8e5280b729c579e83ef32f1" cmd=["/usr/bin/pgrep","-r","DRST","nova-conductor"] Dec 03 12:54:47 crc kubenswrapper[5052]: E1203 12:54:47.570592 5052 prober.go:104] "Probe errored" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" probeType="Readiness" pod="openstack-kuttl-tests/nova-cell1-conductor-0" podUID="2adad00d-01e3-4441-a3fc-87d162a5e309" containerName="nova-cell1-conductor-conductor" Dec 03 12:54:47 crc kubenswrapper[5052]: I1203 12:54:47.573839 5052 generic.go:334] "Generic (PLEG): container finished" podID="941d063f-c834-47a1-bc0c-2096b005e09c" containerID="c89e8180cefdc0d2331c24dc4ecea16dbdbd25fd8e2135fbbbde4e84750348ce" exitCode=1 Dec 03 12:54:47 crc kubenswrapper[5052]: I1203 12:54:47.574263 5052 kubelet_pods.go:1007] "Unable to retrieve pull secret, the image pull may not succeed." pod="openstack-kuttl-tests/neutron6c26-account-delete-d7wtw" secret="" err="secret \"galera-openstack-dockercfg-fkczj\" not found" Dec 03 12:54:47 crc kubenswrapper[5052]: I1203 12:54:47.574304 5052 scope.go:117] "RemoveContainer" containerID="c89e8180cefdc0d2331c24dc4ecea16dbdbd25fd8e2135fbbbde4e84750348ce" Dec 03 12:54:47 crc kubenswrapper[5052]: I1203 12:54:47.574974 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron6c26-account-delete-d7wtw" event={"ID":"941d063f-c834-47a1-bc0c-2096b005e09c","Type":"ContainerDied","Data":"c89e8180cefdc0d2331c24dc4ecea16dbdbd25fd8e2135fbbbde4e84750348ce"} Dec 03 12:54:47 crc kubenswrapper[5052]: I1203 12:54:47.593005 5052 generic.go:334] "Generic (PLEG): container finished" podID="981c3156-805a-44d1-b4ec-f688a8bc3cf8" containerID="91f773e54ad4ec35e1b76f707be75a9985aeb5b002636c9dd379bb836c31c220" exitCode=0 Dec 03 12:54:47 crc kubenswrapper[5052]: I1203 12:54:47.593364 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"981c3156-805a-44d1-b4ec-f688a8bc3cf8","Type":"ContainerDied","Data":"91f773e54ad4ec35e1b76f707be75a9985aeb5b002636c9dd379bb836c31c220"} Dec 03 12:54:47 crc kubenswrapper[5052]: I1203 12:54:47.617810 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3c6d824e-c7da-41ff-adde-dbafb7535476-config-data" (OuterVolumeSpecName: "config-data") pod "3c6d824e-c7da-41ff-adde-dbafb7535476" (UID: "3c6d824e-c7da-41ff-adde-dbafb7535476"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:54:47 crc kubenswrapper[5052]: I1203 12:54:47.632463 5052 generic.go:334] "Generic (PLEG): container finished" podID="79efd225-5ce4-428f-aef9-3f4884931fef" containerID="a1b6cbc8b7e6a360fb1952ecd8d7fe6374e2e1fc02dd0db8aad15d96a40e30e9" exitCode=1 Dec 03 12:54:47 crc kubenswrapper[5052]: I1203 12:54:47.633147 5052 kubelet_pods.go:1007] "Unable to retrieve pull secret, the image pull may not succeed." pod="openstack-kuttl-tests/glance7c80-account-delete-knmlm" secret="" err="secret \"galera-openstack-dockercfg-fkczj\" not found" Dec 03 12:54:47 crc kubenswrapper[5052]: I1203 12:54:47.633191 5052 scope.go:117] "RemoveContainer" containerID="a1b6cbc8b7e6a360fb1952ecd8d7fe6374e2e1fc02dd0db8aad15d96a40e30e9" Dec 03 12:54:47 crc kubenswrapper[5052]: I1203 12:54:47.633401 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance7c80-account-delete-knmlm" event={"ID":"79efd225-5ce4-428f-aef9-3f4884931fef","Type":"ContainerDied","Data":"a1b6cbc8b7e6a360fb1952ecd8d7fe6374e2e1fc02dd0db8aad15d96a40e30e9"} Dec 03 12:54:47 crc kubenswrapper[5052]: I1203 12:54:47.645037 5052 generic.go:334] "Generic (PLEG): container finished" podID="64ada63f-b17d-411b-a547-fa1c15107ab5" containerID="594b90765c644ac99bd8e077dd10ec16881d4e9f1364dcdc6541052085119849" exitCode=0 Dec 03 12:54:47 crc kubenswrapper[5052]: I1203 12:54:47.645137 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-cell1-galera-0" event={"ID":"64ada63f-b17d-411b-a547-fa1c15107ab5","Type":"ContainerDied","Data":"594b90765c644ac99bd8e077dd10ec16881d4e9f1364dcdc6541052085119849"} Dec 03 12:54:47 crc kubenswrapper[5052]: I1203 12:54:47.659658 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/novacell10a46-account-delete-h8x7f"] Dec 03 12:54:47 crc kubenswrapper[5052]: I1203 12:54:47.661924 5052 generic.go:334] "Generic (PLEG): container finished" podID="d9bf91f1-099c-4e72-97cf-1828241aeaac" containerID="11ddaf754d754f510a35d327364ee83e653bb8146394aaffa51c37770ca3011a" exitCode=1 Dec 03 12:54:47 crc kubenswrapper[5052]: I1203 12:54:47.662028 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican6e62-account-delete-6skhm" event={"ID":"d9bf91f1-099c-4e72-97cf-1828241aeaac","Type":"ContainerDied","Data":"11ddaf754d754f510a35d327364ee83e653bb8146394aaffa51c37770ca3011a"} Dec 03 12:54:47 crc kubenswrapper[5052]: I1203 12:54:47.662618 5052 kubelet_pods.go:1007] "Unable to retrieve pull secret, the image pull may not succeed." pod="openstack-kuttl-tests/barbican6e62-account-delete-6skhm" secret="" err="secret \"galera-openstack-dockercfg-fkczj\" not found" Dec 03 12:54:47 crc kubenswrapper[5052]: I1203 12:54:47.662657 5052 scope.go:117] "RemoveContainer" containerID="11ddaf754d754f510a35d327364ee83e653bb8146394aaffa51c37770ca3011a" Dec 03 12:54:47 crc kubenswrapper[5052]: I1203 12:54:47.673902 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/3c6d824e-c7da-41ff-adde-dbafb7535476-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:47 crc kubenswrapper[5052]: I1203 12:54:47.692563 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell0-conductor-0" Dec 03 12:54:47 crc kubenswrapper[5052]: I1203 12:54:47.693960 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0a272fa7-1c64-49b1-81a0-b339a5cc686a" path="/var/lib/kubelet/pods/0a272fa7-1c64-49b1-81a0-b339a5cc686a/volumes" Dec 03 12:54:47 crc kubenswrapper[5052]: I1203 12:54:47.694557 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="218f2098-bd98-44e2-a2d5-8f30733da613" path="/var/lib/kubelet/pods/218f2098-bd98-44e2-a2d5-8f30733da613/volumes" Dec 03 12:54:47 crc kubenswrapper[5052]: I1203 12:54:47.695098 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="38348d87-fbe3-4196-8819-8d90f2720b33" path="/var/lib/kubelet/pods/38348d87-fbe3-4196-8819-8d90f2720b33/volumes" Dec 03 12:54:47 crc kubenswrapper[5052]: I1203 12:54:47.696764 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3a5e94ea-faeb-4202-843a-bd691d8895ce" path="/var/lib/kubelet/pods/3a5e94ea-faeb-4202-843a-bd691d8895ce/volumes" Dec 03 12:54:47 crc kubenswrapper[5052]: I1203 12:54:47.699752 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3db493c4-0372-454e-b802-53ffc8cc3e9b" path="/var/lib/kubelet/pods/3db493c4-0372-454e-b802-53ffc8cc3e9b/volumes" Dec 03 12:54:47 crc kubenswrapper[5052]: I1203 12:54:47.700545 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6436478b-ce40-4335-930c-df0c19d4cb1a" path="/var/lib/kubelet/pods/6436478b-ce40-4335-930c-df0c19d4cb1a/volumes" Dec 03 12:54:47 crc kubenswrapper[5052]: I1203 12:54:47.701805 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="69a643e8-8b6e-4fd4-b445-481364d71485" path="/var/lib/kubelet/pods/69a643e8-8b6e-4fd4-b445-481364d71485/volumes" Dec 03 12:54:47 crc kubenswrapper[5052]: I1203 12:54:47.704278 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8b9df814-7cdb-4696-9045-28d40e5b478c" path="/var/lib/kubelet/pods/8b9df814-7cdb-4696-9045-28d40e5b478c/volumes" Dec 03 12:54:47 crc kubenswrapper[5052]: I1203 12:54:47.707487 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9b10d490-7528-42d2-83b7-d54974959b36" path="/var/lib/kubelet/pods/9b10d490-7528-42d2-83b7-d54974959b36/volumes" Dec 03 12:54:47 crc kubenswrapper[5052]: I1203 12:54:47.708156 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d16285c0-7e62-4b3c-b3e8-a62b8abb49cb" path="/var/lib/kubelet/pods/d16285c0-7e62-4b3c-b3e8-a62b8abb49cb/volumes" Dec 03 12:54:47 crc kubenswrapper[5052]: I1203 12:54:47.708548 5052 generic.go:334] "Generic (PLEG): container finished" podID="cfc68960-6006-486c-bc03-848875d3bf5b" containerID="21405273563ddcf40333c0b6ddfcd119d733ef5beb5a7399909f33a285ac39e7" exitCode=0 Dec 03 12:54:47 crc kubenswrapper[5052]: I1203 12:54:47.710465 5052 generic.go:334] "Generic (PLEG): container finished" podID="d1ff5100-3ac7-4633-811c-ea470242d858" containerID="27f09642bc3ec7488d9032aa069fc921aca6a6f0c2792bc5d84326beb5487829" exitCode=0 Dec 03 12:54:47 crc kubenswrapper[5052]: I1203 12:54:47.712632 5052 generic.go:334] "Generic (PLEG): container finished" podID="e8a26759-2926-41cf-a1ec-286fc9962bb3" containerID="42f16d033db22635e368a85dc53ea3877fb872db34db75e3fb79daeb9c825a36" exitCode=1 Dec 03 12:54:47 crc kubenswrapper[5052]: I1203 12:54:47.713219 5052 kubelet_pods.go:1007] "Unable to retrieve pull secret, the image pull may not succeed." pod="openstack-kuttl-tests/cindera2a9-account-delete-pmspg" secret="" err="secret \"galera-openstack-dockercfg-fkczj\" not found" Dec 03 12:54:47 crc kubenswrapper[5052]: I1203 12:54:47.713258 5052 scope.go:117] "RemoveContainer" containerID="42f16d033db22635e368a85dc53ea3877fb872db34db75e3fb79daeb9c825a36" Dec 03 12:54:47 crc kubenswrapper[5052]: I1203 12:54:47.724093 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f3314e52-7982-44b7-89dd-73b819efe84e" path="/var/lib/kubelet/pods/f3314e52-7982-44b7-89dd-73b819efe84e/volumes" Dec 03 12:54:47 crc kubenswrapper[5052]: I1203 12:54:47.724776 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f8491d70-84bb-4aa6-a969-fc03097c3745" path="/var/lib/kubelet/pods/f8491d70-84bb-4aa6-a969-fc03097c3745/volumes" Dec 03 12:54:47 crc kubenswrapper[5052]: I1203 12:54:47.747693 5052 generic.go:334] "Generic (PLEG): container finished" podID="ed40ab59-7f38-4b2a-ad68-85592fe22ef2" containerID="0d0807a346577955962a640bd5fe5f8e366a538b40576ef0017fa65770ea3118" exitCode=0 Dec 03 12:54:47 crc kubenswrapper[5052]: I1203 12:54:47.750335 5052 generic.go:334] "Generic (PLEG): container finished" podID="8df65ee4-b91d-4ffd-bff2-65e5e6559130" containerID="4cc0973d6be860027762d01ac994ad7b9ec6e3602b0a54d18cb4bc9da4071fbb" exitCode=0 Dec 03 12:54:47 crc kubenswrapper[5052]: I1203 12:54:47.770721 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3c6d824e-c7da-41ff-adde-dbafb7535476-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "3c6d824e-c7da-41ff-adde-dbafb7535476" (UID: "3c6d824e-c7da-41ff-adde-dbafb7535476"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:54:47 crc kubenswrapper[5052]: I1203 12:54:47.773504 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/barbican-api-76dc68d888-zmmcs" podUID="25eead1e-d17c-4b36-8a9f-ef563189d53b" containerName="barbican-api" probeResult="failure" output="Get \"https://10.217.0.181:9311/healthcheck\": read tcp 10.217.0.2:34028->10.217.0.181:9311: read: connection reset by peer" Dec 03 12:54:47 crc kubenswrapper[5052]: I1203 12:54:47.773932 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/barbican-api-76dc68d888-zmmcs" podUID="25eead1e-d17c-4b36-8a9f-ef563189d53b" containerName="barbican-api-log" probeResult="failure" output="Get \"https://10.217.0.181:9311/healthcheck\": read tcp 10.217.0.2:34032->10.217.0.181:9311: read: connection reset by peer" Dec 03 12:54:47 crc kubenswrapper[5052]: I1203 12:54:47.777430 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3c6d824e-c7da-41ff-adde-dbafb7535476-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:48 crc kubenswrapper[5052]: E1203 12:54:48.030306 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:54:48 crc kubenswrapper[5052]: E1203 12:54:48.030345 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:54:48 crc kubenswrapper[5052]: E1203 12:54:48.030395 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d9bf91f1-099c-4e72-97cf-1828241aeaac-operator-scripts podName:d9bf91f1-099c-4e72-97cf-1828241aeaac nodeName:}" failed. No retries permitted until 2025-12-03 12:54:50.03037386 +0000 UTC m=+5680.618615624 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/d9bf91f1-099c-4e72-97cf-1828241aeaac-operator-scripts") pod "barbican6e62-account-delete-6skhm" (UID: "d9bf91f1-099c-4e72-97cf-1828241aeaac") : configmap "openstack-scripts" not found Dec 03 12:54:48 crc kubenswrapper[5052]: E1203 12:54:48.030416 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/941d063f-c834-47a1-bc0c-2096b005e09c-operator-scripts podName:941d063f-c834-47a1-bc0c-2096b005e09c nodeName:}" failed. No retries permitted until 2025-12-03 12:54:50.030407251 +0000 UTC m=+5680.618649015 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/941d063f-c834-47a1-bc0c-2096b005e09c-operator-scripts") pod "neutron6c26-account-delete-d7wtw" (UID: "941d063f-c834-47a1-bc0c-2096b005e09c") : configmap "openstack-scripts" not found Dec 03 12:54:48 crc kubenswrapper[5052]: E1203 12:54:48.030422 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:54:48 crc kubenswrapper[5052]: E1203 12:54:48.030458 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c4d5c183-8df4-4c43-9afb-2b28c6aae9e1-operator-scripts podName:c4d5c183-8df4-4c43-9afb-2b28c6aae9e1 nodeName:}" failed. No retries permitted until 2025-12-03 12:54:50.030443632 +0000 UTC m=+5680.618685396 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/c4d5c183-8df4-4c43-9afb-2b28c6aae9e1-operator-scripts") pod "placementbfa4-account-delete-flc9f" (UID: "c4d5c183-8df4-4c43-9afb-2b28c6aae9e1") : configmap "openstack-scripts" not found Dec 03 12:54:48 crc kubenswrapper[5052]: E1203 12:54:48.133179 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:54:48 crc kubenswrapper[5052]: E1203 12:54:48.133250 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e8a26759-2926-41cf-a1ec-286fc9962bb3-operator-scripts podName:e8a26759-2926-41cf-a1ec-286fc9962bb3 nodeName:}" failed. No retries permitted until 2025-12-03 12:54:50.133232062 +0000 UTC m=+5680.721473826 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/e8a26759-2926-41cf-a1ec-286fc9962bb3-operator-scripts") pod "cindera2a9-account-delete-pmspg" (UID: "e8a26759-2926-41cf-a1ec-286fc9962bb3") : configmap "openstack-scripts" not found Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.133263 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/nova-metadata-0" podUID="022746a5-1c20-4729-8688-5903831f57ed" containerName="nova-metadata-metadata" probeResult="failure" output="Get \"https://10.217.0.220:8775/\": dial tcp 10.217.0.220:8775: connect: connection refused" Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.133648 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/nova-metadata-0" podUID="022746a5-1c20-4729-8688-5903831f57ed" containerName="nova-metadata-log" probeResult="failure" output="Get \"https://10.217.0.220:8775/\": dial tcp 10.217.0.220:8775: connect: connection refused" Dec 03 12:54:48 crc kubenswrapper[5052]: E1203 12:54:48.235272 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:54:48 crc kubenswrapper[5052]: E1203 12:54:48.235405 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/79efd225-5ce4-428f-aef9-3f4884931fef-operator-scripts podName:79efd225-5ce4-428f-aef9-3f4884931fef nodeName:}" failed. No retries permitted until 2025-12-03 12:54:50.235376773 +0000 UTC m=+5680.823618607 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/79efd225-5ce4-428f-aef9-3f4884931fef-operator-scripts") pod "glance7c80-account-delete-knmlm" (UID: "79efd225-5ce4-428f-aef9-3f4884931fef") : configmap "openstack-scripts" not found Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.310504 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.314907 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/novacell10a46-account-delete-h8x7f"] Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.314941 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell0-conductor-0" event={"ID":"3c6d824e-c7da-41ff-adde-dbafb7535476","Type":"ContainerDied","Data":"d1ac337c7422916c6958db2e1cf8f4b67a9f638fef8862b8806ef4c84da04cd6"} Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.315057 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-api-db-create-xmgf4"] Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.315073 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-api-db-create-xmgf4"] Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.315086 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/novaapi7066-account-delete-whswm"] Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.315100 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-api-7066-account-create-update-4xwgf"] Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.315110 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"cfc68960-6006-486c-bc03-848875d3bf5b","Type":"ContainerDied","Data":"21405273563ddcf40333c0b6ddfcd119d733ef5beb5a7399909f33a285ac39e7"} Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.315122 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/memcached-0" event={"ID":"d1ff5100-3ac7-4633-811c-ea470242d858","Type":"ContainerDied","Data":"27f09642bc3ec7488d9032aa069fc921aca6a6f0c2792bc5d84326beb5487829"} Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.315133 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cindera2a9-account-delete-pmspg" event={"ID":"e8a26759-2926-41cf-a1ec-286fc9962bb3","Type":"ContainerDied","Data":"42f16d033db22635e368a85dc53ea3877fb872db34db75e3fb79daeb9c825a36"} Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.315144 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"ed40ab59-7f38-4b2a-ad68-85592fe22ef2","Type":"ContainerDied","Data":"0d0807a346577955962a640bd5fe5f8e366a538b40576ef0017fa65770ea3118"} Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.315154 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-7b465d48b8-4bj69" event={"ID":"8df65ee4-b91d-4ffd-bff2-65e5e6559130","Type":"ContainerDied","Data":"4cc0973d6be860027762d01ac994ad7b9ec6e3602b0a54d18cb4bc9da4071fbb"} Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.315168 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-api-7066-account-create-update-4xwgf"] Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.315177 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell0-db-create-nvztf"] Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.315186 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell0-db-create-nvztf"] Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.315195 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell0-2629-account-create-update-c78fj"] Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.315203 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell0-2629-account-create-update-c78fj"] Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.315213 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/novacell02629-account-delete-wsshh"] Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.315230 5052 scope.go:117] "RemoveContainer" containerID="eb28bc8efa7fd43a1e5b7da097e9679a59f690173e27b20d4dae0a08837dde86" Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.406068 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.432481 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-0"] Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.440303 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-state-metrics-tls-config\" (UniqueName: \"kubernetes.io/secret/3c553114-e0c8-4af9-992f-5421b31497ed-kube-state-metrics-tls-config\") pod \"3c553114-e0c8-4af9-992f-5421b31497ed\" (UID: \"3c553114-e0c8-4af9-992f-5421b31497ed\") " Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.440361 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hpx5g\" (UniqueName: \"kubernetes.io/projected/3c553114-e0c8-4af9-992f-5421b31497ed-kube-api-access-hpx5g\") pod \"3c553114-e0c8-4af9-992f-5421b31497ed\" (UID: \"3c553114-e0c8-4af9-992f-5421b31497ed\") " Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.440387 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-state-metrics-tls-certs\" (UniqueName: \"kubernetes.io/secret/3c553114-e0c8-4af9-992f-5421b31497ed-kube-state-metrics-tls-certs\") pod \"3c553114-e0c8-4af9-992f-5421b31497ed\" (UID: \"3c553114-e0c8-4af9-992f-5421b31497ed\") " Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.440600 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3c553114-e0c8-4af9-992f-5421b31497ed-combined-ca-bundle\") pod \"3c553114-e0c8-4af9-992f-5421b31497ed\" (UID: \"3c553114-e0c8-4af9-992f-5421b31497ed\") " Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.450917 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-proxy-5479cb94c8-q2snc" Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.456666 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/6d56c6dc-38c9-42cf-ac6d-04ba49860d31-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.456697 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-hs7rk\" (UniqueName: \"kubernetes.io/projected/6d56c6dc-38c9-42cf-ac6d-04ba49860d31-kube-api-access-hs7rk\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.470387 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3c553114-e0c8-4af9-992f-5421b31497ed-kube-api-access-hpx5g" (OuterVolumeSpecName: "kube-api-access-hpx5g") pod "3c553114-e0c8-4af9-992f-5421b31497ed" (UID: "3c553114-e0c8-4af9-992f-5421b31497ed"). InnerVolumeSpecName "kube-api-access-hpx5g". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.485137 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-keystone-listener-7b66cf7bb6-fsc4x" Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.511922 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell0-conductor-0"] Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.519591 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.538823 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-worker-b7f5dfb6f-pj6bx" Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.540152 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.553067 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.558214 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/2b0bf9ae-2d1d-48da-8c4b-2973b47985a3-config-data-custom\") pod \"2b0bf9ae-2d1d-48da-8c4b-2973b47985a3\" (UID: \"2b0bf9ae-2d1d-48da-8c4b-2973b47985a3\") " Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.558287 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/2b0bf9ae-2d1d-48da-8c4b-2973b47985a3-etc-machine-id\") pod \"2b0bf9ae-2d1d-48da-8c4b-2973b47985a3\" (UID: \"2b0bf9ae-2d1d-48da-8c4b-2973b47985a3\") " Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.558320 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/2b0bf9ae-2d1d-48da-8c4b-2973b47985a3-config-data\") pod \"2b0bf9ae-2d1d-48da-8c4b-2973b47985a3\" (UID: \"2b0bf9ae-2d1d-48da-8c4b-2973b47985a3\") " Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.558378 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v7gp8\" (UniqueName: \"kubernetes.io/projected/eefe83f8-0235-4b16-aefb-5ec7b2d92fcf-kube-api-access-v7gp8\") pod \"eefe83f8-0235-4b16-aefb-5ec7b2d92fcf\" (UID: \"eefe83f8-0235-4b16-aefb-5ec7b2d92fcf\") " Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.558467 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/eefe83f8-0235-4b16-aefb-5ec7b2d92fcf-combined-ca-bundle\") pod \"eefe83f8-0235-4b16-aefb-5ec7b2d92fcf\" (UID: \"eefe83f8-0235-4b16-aefb-5ec7b2d92fcf\") " Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.558542 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/2b0bf9ae-2d1d-48da-8c4b-2973b47985a3-scripts\") pod \"2b0bf9ae-2d1d-48da-8c4b-2973b47985a3\" (UID: \"2b0bf9ae-2d1d-48da-8c4b-2973b47985a3\") " Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.558575 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/eefe83f8-0235-4b16-aefb-5ec7b2d92fcf-internal-tls-certs\") pod \"eefe83f8-0235-4b16-aefb-5ec7b2d92fcf\" (UID: \"eefe83f8-0235-4b16-aefb-5ec7b2d92fcf\") " Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.561960 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/eefe83f8-0235-4b16-aefb-5ec7b2d92fcf-config-data\") pod \"eefe83f8-0235-4b16-aefb-5ec7b2d92fcf\" (UID: \"eefe83f8-0235-4b16-aefb-5ec7b2d92fcf\") " Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.559169 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/2b0bf9ae-2d1d-48da-8c4b-2973b47985a3-etc-machine-id" (OuterVolumeSpecName: "etc-machine-id") pod "2b0bf9ae-2d1d-48da-8c4b-2973b47985a3" (UID: "2b0bf9ae-2d1d-48da-8c4b-2973b47985a3"). InnerVolumeSpecName "etc-machine-id". PluginName "kubernetes.io/host-path", VolumeGidValue "" Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.562225 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/eefe83f8-0235-4b16-aefb-5ec7b2d92fcf-etc-swift\") pod \"eefe83f8-0235-4b16-aefb-5ec7b2d92fcf\" (UID: \"eefe83f8-0235-4b16-aefb-5ec7b2d92fcf\") " Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.562318 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ql9sg\" (UniqueName: \"kubernetes.io/projected/2b0bf9ae-2d1d-48da-8c4b-2973b47985a3-kube-api-access-ql9sg\") pod \"2b0bf9ae-2d1d-48da-8c4b-2973b47985a3\" (UID: \"2b0bf9ae-2d1d-48da-8c4b-2973b47985a3\") " Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.562481 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/eefe83f8-0235-4b16-aefb-5ec7b2d92fcf-log-httpd\") pod \"eefe83f8-0235-4b16-aefb-5ec7b2d92fcf\" (UID: \"eefe83f8-0235-4b16-aefb-5ec7b2d92fcf\") " Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.563071 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/eefe83f8-0235-4b16-aefb-5ec7b2d92fcf-run-httpd\") pod \"eefe83f8-0235-4b16-aefb-5ec7b2d92fcf\" (UID: \"eefe83f8-0235-4b16-aefb-5ec7b2d92fcf\") " Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.563135 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/eefe83f8-0235-4b16-aefb-5ec7b2d92fcf-public-tls-certs\") pod \"eefe83f8-0235-4b16-aefb-5ec7b2d92fcf\" (UID: \"eefe83f8-0235-4b16-aefb-5ec7b2d92fcf\") " Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.563169 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2b0bf9ae-2d1d-48da-8c4b-2973b47985a3-combined-ca-bundle\") pod \"2b0bf9ae-2d1d-48da-8c4b-2973b47985a3\" (UID: \"2b0bf9ae-2d1d-48da-8c4b-2973b47985a3\") " Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.564033 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eefe83f8-0235-4b16-aefb-5ec7b2d92fcf-run-httpd" (OuterVolumeSpecName: "run-httpd") pod "eefe83f8-0235-4b16-aefb-5ec7b2d92fcf" (UID: "eefe83f8-0235-4b16-aefb-5ec7b2d92fcf"). InnerVolumeSpecName "run-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.564315 5052 reconciler_common.go:293] "Volume detached for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/2b0bf9ae-2d1d-48da-8c4b-2973b47985a3-etc-machine-id\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.564389 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-hpx5g\" (UniqueName: \"kubernetes.io/projected/3c553114-e0c8-4af9-992f-5421b31497ed-kube-api-access-hpx5g\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.564442 5052 reconciler_common.go:293] "Volume detached for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/eefe83f8-0235-4b16-aefb-5ec7b2d92fcf-run-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.564762 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eefe83f8-0235-4b16-aefb-5ec7b2d92fcf-log-httpd" (OuterVolumeSpecName: "log-httpd") pod "eefe83f8-0235-4b16-aefb-5ec7b2d92fcf" (UID: "eefe83f8-0235-4b16-aefb-5ec7b2d92fcf"). InnerVolumeSpecName "log-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.637898 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eefe83f8-0235-4b16-aefb-5ec7b2d92fcf-kube-api-access-v7gp8" (OuterVolumeSpecName: "kube-api-access-v7gp8") pod "eefe83f8-0235-4b16-aefb-5ec7b2d92fcf" (UID: "eefe83f8-0235-4b16-aefb-5ec7b2d92fcf"). InnerVolumeSpecName "kube-api-access-v7gp8". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.639539 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2b0bf9ae-2d1d-48da-8c4b-2973b47985a3-scripts" (OuterVolumeSpecName: "scripts") pod "2b0bf9ae-2d1d-48da-8c4b-2973b47985a3" (UID: "2b0bf9ae-2d1d-48da-8c4b-2973b47985a3"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.639637 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3c553114-e0c8-4af9-992f-5421b31497ed-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "3c553114-e0c8-4af9-992f-5421b31497ed" (UID: "3c553114-e0c8-4af9-992f-5421b31497ed"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.652747 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2b0bf9ae-2d1d-48da-8c4b-2973b47985a3-config-data-custom" (OuterVolumeSpecName: "config-data-custom") pod "2b0bf9ae-2d1d-48da-8c4b-2973b47985a3" (UID: "2b0bf9ae-2d1d-48da-8c4b-2973b47985a3"). InnerVolumeSpecName "config-data-custom". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.652833 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eefe83f8-0235-4b16-aefb-5ec7b2d92fcf-etc-swift" (OuterVolumeSpecName: "etc-swift") pod "eefe83f8-0235-4b16-aefb-5ec7b2d92fcf" (UID: "eefe83f8-0235-4b16-aefb-5ec7b2d92fcf"). InnerVolumeSpecName "etc-swift". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.653301 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2b0bf9ae-2d1d-48da-8c4b-2973b47985a3-kube-api-access-ql9sg" (OuterVolumeSpecName: "kube-api-access-ql9sg") pod "2b0bf9ae-2d1d-48da-8c4b-2973b47985a3" (UID: "2b0bf9ae-2d1d-48da-8c4b-2973b47985a3"). InnerVolumeSpecName "kube-api-access-ql9sg". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.679989 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/ed40ab59-7f38-4b2a-ad68-85592fe22ef2-logs\") pod \"ed40ab59-7f38-4b2a-ad68-85592fe22ef2\" (UID: \"ed40ab59-7f38-4b2a-ad68-85592fe22ef2\") " Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.680040 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"nova-novncproxy-tls-certs\" (UniqueName: \"kubernetes.io/secret/43b07f3c-f9d7-451b-b0ab-61b293cff35d-nova-novncproxy-tls-certs\") pod \"43b07f3c-f9d7-451b-b0ab-61b293cff35d\" (UID: \"43b07f3c-f9d7-451b-b0ab-61b293cff35d\") " Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.680082 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/020bfa4c-3dba-43d2-9ac2-0bbbae660709-config-data-custom\") pod \"020bfa4c-3dba-43d2-9ac2-0bbbae660709\" (UID: \"020bfa4c-3dba-43d2-9ac2-0bbbae660709\") " Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.680124 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/64ada63f-b17d-411b-a547-fa1c15107ab5-galera-tls-certs\") pod \"64ada63f-b17d-411b-a547-fa1c15107ab5\" (UID: \"64ada63f-b17d-411b-a547-fa1c15107ab5\") " Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.680153 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"vencrypt-tls-certs\" (UniqueName: \"kubernetes.io/secret/43b07f3c-f9d7-451b-b0ab-61b293cff35d-vencrypt-tls-certs\") pod \"43b07f3c-f9d7-451b-b0ab-61b293cff35d\" (UID: \"43b07f3c-f9d7-451b-b0ab-61b293cff35d\") " Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.680196 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/020bfa4c-3dba-43d2-9ac2-0bbbae660709-combined-ca-bundle\") pod \"020bfa4c-3dba-43d2-9ac2-0bbbae660709\" (UID: \"020bfa4c-3dba-43d2-9ac2-0bbbae660709\") " Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.680220 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/43b07f3c-f9d7-451b-b0ab-61b293cff35d-combined-ca-bundle\") pod \"43b07f3c-f9d7-451b-b0ab-61b293cff35d\" (UID: \"43b07f3c-f9d7-451b-b0ab-61b293cff35d\") " Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.680256 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/64ada63f-b17d-411b-a547-fa1c15107ab5-kolla-config\") pod \"64ada63f-b17d-411b-a547-fa1c15107ab5\" (UID: \"64ada63f-b17d-411b-a547-fa1c15107ab5\") " Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.680317 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/b2933134-7226-423a-8120-3148845b808a-config-data\") pod \"b2933134-7226-423a-8120-3148845b808a\" (UID: \"b2933134-7226-423a-8120-3148845b808a\") " Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.680368 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/ed40ab59-7f38-4b2a-ad68-85592fe22ef2-config-data\") pod \"ed40ab59-7f38-4b2a-ad68-85592fe22ef2\" (UID: \"ed40ab59-7f38-4b2a-ad68-85592fe22ef2\") " Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.680406 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8t9x7\" (UniqueName: \"kubernetes.io/projected/b2933134-7226-423a-8120-3148845b808a-kube-api-access-8t9x7\") pod \"b2933134-7226-423a-8120-3148845b808a\" (UID: \"b2933134-7226-423a-8120-3148845b808a\") " Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.680439 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9zh24\" (UniqueName: \"kubernetes.io/projected/64ada63f-b17d-411b-a547-fa1c15107ab5-kube-api-access-9zh24\") pod \"64ada63f-b17d-411b-a547-fa1c15107ab5\" (UID: \"64ada63f-b17d-411b-a547-fa1c15107ab5\") " Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.680466 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p9xm5\" (UniqueName: \"kubernetes.io/projected/ed40ab59-7f38-4b2a-ad68-85592fe22ef2-kube-api-access-p9xm5\") pod \"ed40ab59-7f38-4b2a-ad68-85592fe22ef2\" (UID: \"ed40ab59-7f38-4b2a-ad68-85592fe22ef2\") " Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.680488 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/ed40ab59-7f38-4b2a-ad68-85592fe22ef2-scripts\") pod \"ed40ab59-7f38-4b2a-ad68-85592fe22ef2\" (UID: \"ed40ab59-7f38-4b2a-ad68-85592fe22ef2\") " Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.680524 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/ed40ab59-7f38-4b2a-ad68-85592fe22ef2-etc-machine-id\") pod \"ed40ab59-7f38-4b2a-ad68-85592fe22ef2\" (UID: \"ed40ab59-7f38-4b2a-ad68-85592fe22ef2\") " Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.680554 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"mysql-db\" (UniqueName: \"kubernetes.io/local-volume/local-storage11-crc\") pod \"64ada63f-b17d-411b-a547-fa1c15107ab5\" (UID: \"64ada63f-b17d-411b-a547-fa1c15107ab5\") " Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.680581 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ed40ab59-7f38-4b2a-ad68-85592fe22ef2-combined-ca-bundle\") pod \"ed40ab59-7f38-4b2a-ad68-85592fe22ef2\" (UID: \"ed40ab59-7f38-4b2a-ad68-85592fe22ef2\") " Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.680623 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed40ab59-7f38-4b2a-ad68-85592fe22ef2-logs" (OuterVolumeSpecName: "logs") pod "ed40ab59-7f38-4b2a-ad68-85592fe22ef2" (UID: "ed40ab59-7f38-4b2a-ad68-85592fe22ef2"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.680632 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/43b07f3c-f9d7-451b-b0ab-61b293cff35d-config-data\") pod \"43b07f3c-f9d7-451b-b0ab-61b293cff35d\" (UID: \"43b07f3c-f9d7-451b-b0ab-61b293cff35d\") " Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.680707 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/64ada63f-b17d-411b-a547-fa1c15107ab5-combined-ca-bundle\") pod \"64ada63f-b17d-411b-a547-fa1c15107ab5\" (UID: \"64ada63f-b17d-411b-a547-fa1c15107ab5\") " Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.680737 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/64ada63f-b17d-411b-a547-fa1c15107ab5-operator-scripts\") pod \"64ada63f-b17d-411b-a547-fa1c15107ab5\" (UID: \"64ada63f-b17d-411b-a547-fa1c15107ab5\") " Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.680767 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/b2933134-7226-423a-8120-3148845b808a-config-data-custom\") pod \"b2933134-7226-423a-8120-3148845b808a\" (UID: \"b2933134-7226-423a-8120-3148845b808a\") " Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.680796 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/b2933134-7226-423a-8120-3148845b808a-logs\") pod \"b2933134-7226-423a-8120-3148845b808a\" (UID: \"b2933134-7226-423a-8120-3148845b808a\") " Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.680836 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vfmg6\" (UniqueName: \"kubernetes.io/projected/43b07f3c-f9d7-451b-b0ab-61b293cff35d-kube-api-access-vfmg6\") pod \"43b07f3c-f9d7-451b-b0ab-61b293cff35d\" (UID: \"43b07f3c-f9d7-451b-b0ab-61b293cff35d\") " Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.680878 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b2933134-7226-423a-8120-3148845b808a-combined-ca-bundle\") pod \"b2933134-7226-423a-8120-3148845b808a\" (UID: \"b2933134-7226-423a-8120-3148845b808a\") " Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.680938 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/020bfa4c-3dba-43d2-9ac2-0bbbae660709-config-data\") pod \"020bfa4c-3dba-43d2-9ac2-0bbbae660709\" (UID: \"020bfa4c-3dba-43d2-9ac2-0bbbae660709\") " Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.680987 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/64ada63f-b17d-411b-a547-fa1c15107ab5-config-data-generated\") pod \"64ada63f-b17d-411b-a547-fa1c15107ab5\" (UID: \"64ada63f-b17d-411b-a547-fa1c15107ab5\") " Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.681016 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/64ada63f-b17d-411b-a547-fa1c15107ab5-config-data-default\") pod \"64ada63f-b17d-411b-a547-fa1c15107ab5\" (UID: \"64ada63f-b17d-411b-a547-fa1c15107ab5\") " Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.681049 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/ed40ab59-7f38-4b2a-ad68-85592fe22ef2-internal-tls-certs\") pod \"ed40ab59-7f38-4b2a-ad68-85592fe22ef2\" (UID: \"ed40ab59-7f38-4b2a-ad68-85592fe22ef2\") " Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.681078 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/ed40ab59-7f38-4b2a-ad68-85592fe22ef2-public-tls-certs\") pod \"ed40ab59-7f38-4b2a-ad68-85592fe22ef2\" (UID: \"ed40ab59-7f38-4b2a-ad68-85592fe22ef2\") " Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.681112 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/020bfa4c-3dba-43d2-9ac2-0bbbae660709-logs\") pod \"020bfa4c-3dba-43d2-9ac2-0bbbae660709\" (UID: \"020bfa4c-3dba-43d2-9ac2-0bbbae660709\") " Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.681153 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/ed40ab59-7f38-4b2a-ad68-85592fe22ef2-config-data-custom\") pod \"ed40ab59-7f38-4b2a-ad68-85592fe22ef2\" (UID: \"ed40ab59-7f38-4b2a-ad68-85592fe22ef2\") " Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.681177 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fjbl5\" (UniqueName: \"kubernetes.io/projected/020bfa4c-3dba-43d2-9ac2-0bbbae660709-kube-api-access-fjbl5\") pod \"020bfa4c-3dba-43d2-9ac2-0bbbae660709\" (UID: \"020bfa4c-3dba-43d2-9ac2-0bbbae660709\") " Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.681978 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3c553114-e0c8-4af9-992f-5421b31497ed-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.681993 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/2b0bf9ae-2d1d-48da-8c4b-2973b47985a3-config-data-custom\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.682004 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-v7gp8\" (UniqueName: \"kubernetes.io/projected/eefe83f8-0235-4b16-aefb-5ec7b2d92fcf-kube-api-access-v7gp8\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.682014 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/ed40ab59-7f38-4b2a-ad68-85592fe22ef2-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.682027 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/2b0bf9ae-2d1d-48da-8c4b-2973b47985a3-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.682036 5052 reconciler_common.go:293] "Volume detached for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/eefe83f8-0235-4b16-aefb-5ec7b2d92fcf-etc-swift\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.682045 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-ql9sg\" (UniqueName: \"kubernetes.io/projected/2b0bf9ae-2d1d-48da-8c4b-2973b47985a3-kube-api-access-ql9sg\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.682055 5052 reconciler_common.go:293] "Volume detached for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/eefe83f8-0235-4b16-aefb-5ec7b2d92fcf-log-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.687340 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/020bfa4c-3dba-43d2-9ac2-0bbbae660709-logs" (OuterVolumeSpecName: "logs") pod "020bfa4c-3dba-43d2-9ac2-0bbbae660709" (UID: "020bfa4c-3dba-43d2-9ac2-0bbbae660709"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.689610 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b2933134-7226-423a-8120-3148845b808a-logs" (OuterVolumeSpecName: "logs") pod "b2933134-7226-423a-8120-3148845b808a" (UID: "b2933134-7226-423a-8120-3148845b808a"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.690066 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/64ada63f-b17d-411b-a547-fa1c15107ab5-config-data-generated" (OuterVolumeSpecName: "config-data-generated") pod "64ada63f-b17d-411b-a547-fa1c15107ab5" (UID: "64ada63f-b17d-411b-a547-fa1c15107ab5"). InnerVolumeSpecName "config-data-generated". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.690544 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/64ada63f-b17d-411b-a547-fa1c15107ab5-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "64ada63f-b17d-411b-a547-fa1c15107ab5" (UID: "64ada63f-b17d-411b-a547-fa1c15107ab5"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.690601 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/ed40ab59-7f38-4b2a-ad68-85592fe22ef2-etc-machine-id" (OuterVolumeSpecName: "etc-machine-id") pod "ed40ab59-7f38-4b2a-ad68-85592fe22ef2" (UID: "ed40ab59-7f38-4b2a-ad68-85592fe22ef2"). InnerVolumeSpecName "etc-machine-id". PluginName "kubernetes.io/host-path", VolumeGidValue "" Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.690840 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/64ada63f-b17d-411b-a547-fa1c15107ab5-config-data-default" (OuterVolumeSpecName: "config-data-default") pod "64ada63f-b17d-411b-a547-fa1c15107ab5" (UID: "64ada63f-b17d-411b-a547-fa1c15107ab5"). InnerVolumeSpecName "config-data-default". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.696326 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/64ada63f-b17d-411b-a547-fa1c15107ab5-kolla-config" (OuterVolumeSpecName: "kolla-config") pod "64ada63f-b17d-411b-a547-fa1c15107ab5" (UID: "64ada63f-b17d-411b-a547-fa1c15107ab5"). InnerVolumeSpecName "kolla-config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.705269 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/novacell02629-account-delete-wsshh"] Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.713315 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" podUID="568addd3-4ad3-47c3-adae-693691e7cd40" containerName="rabbitmq" probeResult="failure" output="dial tcp 10.217.0.137:5671: connect: connection refused" Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.738538 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b2933134-7226-423a-8120-3148845b808a-config-data-custom" (OuterVolumeSpecName: "config-data-custom") pod "b2933134-7226-423a-8120-3148845b808a" (UID: "b2933134-7226-423a-8120-3148845b808a"). InnerVolumeSpecName "config-data-custom". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.738660 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/43b07f3c-f9d7-451b-b0ab-61b293cff35d-kube-api-access-vfmg6" (OuterVolumeSpecName: "kube-api-access-vfmg6") pod "43b07f3c-f9d7-451b-b0ab-61b293cff35d" (UID: "43b07f3c-f9d7-451b-b0ab-61b293cff35d"). InnerVolumeSpecName "kube-api-access-vfmg6". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.752938 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/020bfa4c-3dba-43d2-9ac2-0bbbae660709-kube-api-access-fjbl5" (OuterVolumeSpecName: "kube-api-access-fjbl5") pod "020bfa4c-3dba-43d2-9ac2-0bbbae660709" (UID: "020bfa4c-3dba-43d2-9ac2-0bbbae660709"). InnerVolumeSpecName "kube-api-access-fjbl5". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.764611 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ed40ab59-7f38-4b2a-ad68-85592fe22ef2-config-data-custom" (OuterVolumeSpecName: "config-data-custom") pod "ed40ab59-7f38-4b2a-ad68-85592fe22ef2" (UID: "ed40ab59-7f38-4b2a-ad68-85592fe22ef2"). InnerVolumeSpecName "config-data-custom". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.775802 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/020bfa4c-3dba-43d2-9ac2-0bbbae660709-config-data-custom" (OuterVolumeSpecName: "config-data-custom") pod "020bfa4c-3dba-43d2-9ac2-0bbbae660709" (UID: "020bfa4c-3dba-43d2-9ac2-0bbbae660709"). InnerVolumeSpecName "config-data-custom". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.779716 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ed40ab59-7f38-4b2a-ad68-85592fe22ef2-kube-api-access-p9xm5" (OuterVolumeSpecName: "kube-api-access-p9xm5") pod "ed40ab59-7f38-4b2a-ad68-85592fe22ef2" (UID: "ed40ab59-7f38-4b2a-ad68-85592fe22ef2"). InnerVolumeSpecName "kube-api-access-p9xm5". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.781273 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/64ada63f-b17d-411b-a547-fa1c15107ab5-kube-api-access-9zh24" (OuterVolumeSpecName: "kube-api-access-9zh24") pod "64ada63f-b17d-411b-a547-fa1c15107ab5" (UID: "64ada63f-b17d-411b-a547-fa1c15107ab5"). InnerVolumeSpecName "kube-api-access-9zh24". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.782422 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ed40ab59-7f38-4b2a-ad68-85592fe22ef2-scripts" (OuterVolumeSpecName: "scripts") pod "ed40ab59-7f38-4b2a-ad68-85592fe22ef2" (UID: "ed40ab59-7f38-4b2a-ad68-85592fe22ef2"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.783678 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-9zh24\" (UniqueName: \"kubernetes.io/projected/64ada63f-b17d-411b-a547-fa1c15107ab5-kube-api-access-9zh24\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.783699 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-p9xm5\" (UniqueName: \"kubernetes.io/projected/ed40ab59-7f38-4b2a-ad68-85592fe22ef2-kube-api-access-p9xm5\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.783708 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/ed40ab59-7f38-4b2a-ad68-85592fe22ef2-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.783718 5052 reconciler_common.go:293] "Volume detached for volume \"etc-machine-id\" (UniqueName: \"kubernetes.io/host-path/ed40ab59-7f38-4b2a-ad68-85592fe22ef2-etc-machine-id\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.783730 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/64ada63f-b17d-411b-a547-fa1c15107ab5-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.783739 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/b2933134-7226-423a-8120-3148845b808a-config-data-custom\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.783748 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/b2933134-7226-423a-8120-3148845b808a-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.783757 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-vfmg6\" (UniqueName: \"kubernetes.io/projected/43b07f3c-f9d7-451b-b0ab-61b293cff35d-kube-api-access-vfmg6\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.783765 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/64ada63f-b17d-411b-a547-fa1c15107ab5-config-data-generated\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.783774 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/64ada63f-b17d-411b-a547-fa1c15107ab5-config-data-default\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.783782 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/020bfa4c-3dba-43d2-9ac2-0bbbae660709-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.783791 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/ed40ab59-7f38-4b2a-ad68-85592fe22ef2-config-data-custom\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.783800 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-fjbl5\" (UniqueName: \"kubernetes.io/projected/020bfa4c-3dba-43d2-9ac2-0bbbae660709-kube-api-access-fjbl5\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.783810 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/020bfa4c-3dba-43d2-9ac2-0bbbae660709-config-data-custom\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.783819 5052 reconciler_common.go:293] "Volume detached for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/64ada63f-b17d-411b-a547-fa1c15107ab5-kolla-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.791340 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b2933134-7226-423a-8120-3148845b808a-kube-api-access-8t9x7" (OuterVolumeSpecName: "kube-api-access-8t9x7") pod "b2933134-7226-423a-8120-3148845b808a" (UID: "b2933134-7226-423a-8120-3148845b808a"). InnerVolumeSpecName "kube-api-access-8t9x7". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.807905 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-proxy-5479cb94c8-q2snc" Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.807905 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-proxy-5479cb94c8-q2snc" event={"ID":"eefe83f8-0235-4b16-aefb-5ec7b2d92fcf","Type":"ContainerDied","Data":"7a0e8c623d922e9f99403581b92083b2a8cdbfa4f8428dce82708efd3a82fb94"} Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.808032 5052 scope.go:117] "RemoveContainer" containerID="29117e54764872df757ac95e94dbab057dd9c2d6f5908051e97419768ed8628a" Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.813452 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-internal-api-0" event={"ID":"981c3156-805a-44d1-b4ec-f688a8bc3cf8","Type":"ContainerDied","Data":"f9733bd0ad6d0ebfc83dace1f11042df0c2c68afe8d846ce7105b5cf6873ea6f"} Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.813494 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f9733bd0ad6d0ebfc83dace1f11042df0c2c68afe8d846ce7105b5cf6873ea6f" Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.815835 5052 generic.go:334] "Generic (PLEG): container finished" podID="2adad00d-01e3-4441-a3fc-87d162a5e309" containerID="a0d54bbd0da0ce78d0c6c5dcb48378864ee5ffb0c8e5280b729c579e83ef32f1" exitCode=0 Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.816374 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-conductor-0" event={"ID":"2adad00d-01e3-4441-a3fc-87d162a5e309","Type":"ContainerDied","Data":"a0d54bbd0da0ce78d0c6c5dcb48378864ee5ffb0c8e5280b729c579e83ef32f1"} Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.818042 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-worker-b7f5dfb6f-pj6bx" event={"ID":"b2933134-7226-423a-8120-3148845b808a","Type":"ContainerDied","Data":"d9a0bc41dd7a93bb685e52d2df88081b1cd6cec771ed9a5f4fa07db63fea42e5"} Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.818059 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-worker-b7f5dfb6f-pj6bx" Dec 03 12:54:48 crc kubenswrapper[5052]: W1203 12:54:48.826863 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1b7ef5eb_406e_4e2c_9d9c_d763919f56fd.slice/crio-b90af27119e2a44164e92cfa6f8e83a087021fe45af6e255a1db739e22bee63b WatchSource:0}: Error finding container b90af27119e2a44164e92cfa6f8e83a087021fe45af6e255a1db739e22bee63b: Status 404 returned error can't find the container with id b90af27119e2a44164e92cfa6f8e83a087021fe45af6e255a1db739e22bee63b Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.829030 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/kube-state-metrics-0" event={"ID":"3c553114-e0c8-4af9-992f-5421b31497ed","Type":"ContainerDied","Data":"53edc2154195b4da6f1b98de61e5c7482fcb0dd5f0bf268ab7464297f0dac516"} Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.829123 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/kube-state-metrics-0" Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.833141 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-api-0" event={"ID":"ed40ab59-7f38-4b2a-ad68-85592fe22ef2","Type":"ContainerDied","Data":"cc40798a75bb61e54d557a7c32d26f21e44ee687a79c1f4c496f8ad7b15c4381"} Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.833232 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-api-0" Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.837405 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-cell1-galera-0" event={"ID":"64ada63f-b17d-411b-a547-fa1c15107ab5","Type":"ContainerDied","Data":"1748e5136286c097ba05ce84d0037c1e32b588c257e53061127e3d8c8fa44d52"} Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.837426 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.842195 5052 generic.go:334] "Generic (PLEG): container finished" podID="02edb9e9-3f42-4aea-9fe6-7efa332d5fb8" containerID="72785ef6236001625d6d7f1e690671c78dc1f11c3518cfcc3bf9d6de5ab4c2fb" exitCode=0 Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.842261 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"02edb9e9-3f42-4aea-9fe6-7efa332d5fb8","Type":"ContainerDied","Data":"72785ef6236001625d6d7f1e690671c78dc1f11c3518cfcc3bf9d6de5ab4c2fb"} Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.844756 5052 generic.go:334] "Generic (PLEG): container finished" podID="022746a5-1c20-4729-8688-5903831f57ed" containerID="e304d74be1d9347456c21077ccfef718ab1e897114fc49c1d7da35457693360a" exitCode=0 Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.844831 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"022746a5-1c20-4729-8688-5903831f57ed","Type":"ContainerDied","Data":"e304d74be1d9347456c21077ccfef718ab1e897114fc49c1d7da35457693360a"} Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.846644 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-keystone-listener-7b66cf7bb6-fsc4x" event={"ID":"020bfa4c-3dba-43d2-9ac2-0bbbae660709","Type":"ContainerDied","Data":"9e873a757d76efed4f69f46f02db362ab015158690671368a180b4a2cb792abd"} Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.846675 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-keystone-listener-7b66cf7bb6-fsc4x" Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.848788 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/memcached-0" event={"ID":"d1ff5100-3ac7-4633-811c-ea470242d858","Type":"ContainerDied","Data":"7e94672ce1b201b8722d0a3d689750d098504a74ebe9de66246c9bc7e834b074"} Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.848814 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7e94672ce1b201b8722d0a3d689750d098504a74ebe9de66246c9bc7e834b074" Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.850433 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" event={"ID":"43b07f3c-f9d7-451b-b0ab-61b293cff35d","Type":"ContainerDied","Data":"c241edf0de6c454dc695c93815be67873e73d06d1177654a8e8937f82063df1b"} Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.850493 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.865375 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage11-crc" (OuterVolumeSpecName: "mysql-db") pod "64ada63f-b17d-411b-a547-fa1c15107ab5" (UID: "64ada63f-b17d-411b-a547-fa1c15107ab5"). InnerVolumeSpecName "local-storage11-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.866559 5052 generic.go:334] "Generic (PLEG): container finished" podID="25eead1e-d17c-4b36-8a9f-ef563189d53b" containerID="018816f9a78e7b2315d8c4e733edf99eeec2819f8f83cbe3fa3669341863f8d1" exitCode=0 Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.866635 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-76dc68d888-zmmcs" event={"ID":"25eead1e-d17c-4b36-8a9f-ef563189d53b","Type":"ContainerDied","Data":"018816f9a78e7b2315d8c4e733edf99eeec2819f8f83cbe3fa3669341863f8d1"} Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.873269 5052 generic.go:334] "Generic (PLEG): container finished" podID="7d297f47-a73f-4a88-bba0-3108deb8df4d" containerID="f15af8c0a8823cb666fbe94253b10ca1d40d1a5303a03c6ac25035b6456a7a85" exitCode=0 Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.873339 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"7d297f47-a73f-4a88-bba0-3108deb8df4d","Type":"ContainerDied","Data":"f15af8c0a8823cb666fbe94253b10ca1d40d1a5303a03c6ac25035b6456a7a85"} Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.885841 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-8t9x7\" (UniqueName: \"kubernetes.io/projected/b2933134-7226-423a-8120-3148845b808a-kube-api-access-8t9x7\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.885886 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage11-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage11-crc\") on node \"crc\" " Dec 03 12:54:48 crc kubenswrapper[5052]: E1203 12:54:48.886394 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/rabbitmq-config-data: configmap "rabbitmq-config-data" not found Dec 03 12:54:48 crc kubenswrapper[5052]: E1203 12:54:48.886455 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e73813f5-f97f-41dd-8a79-6196557d4d1a-config-data podName:e73813f5-f97f-41dd-8a79-6196557d4d1a nodeName:}" failed. No retries permitted until 2025-12-03 12:54:56.886439253 +0000 UTC m=+5687.474681017 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "config-data" (UniqueName: "kubernetes.io/configmap/e73813f5-f97f-41dd-8a79-6196557d4d1a-config-data") pod "rabbitmq-server-0" (UID: "e73813f5-f97f-41dd-8a79-6196557d4d1a") : configmap "rabbitmq-config-data" not found Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.887482 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cinder-scheduler-0" event={"ID":"2b0bf9ae-2d1d-48da-8c4b-2973b47985a3","Type":"ContainerDied","Data":"c6e9ea130ef6779b2c89f649e9b2cc204bd62f4ce28406054fb1e75ea1417b4a"} Dec 03 12:54:48 crc kubenswrapper[5052]: I1203 12:54:48.887586 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cinder-scheduler-0" Dec 03 12:54:49 crc kubenswrapper[5052]: I1203 12:54:49.154802 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/rabbitmq-server-0" podUID="e73813f5-f97f-41dd-8a79-6196557d4d1a" containerName="rabbitmq" probeResult="failure" output="dial tcp 10.217.0.138:5671: connect: connection refused" Dec 03 12:54:49 crc kubenswrapper[5052]: I1203 12:54:49.366761 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b2933134-7226-423a-8120-3148845b808a-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "b2933134-7226-423a-8120-3148845b808a" (UID: "b2933134-7226-423a-8120-3148845b808a"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:54:49 crc kubenswrapper[5052]: I1203 12:54:49.405267 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b2933134-7226-423a-8120-3148845b808a-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:49 crc kubenswrapper[5052]: I1203 12:54:49.446454 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/43b07f3c-f9d7-451b-b0ab-61b293cff35d-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "43b07f3c-f9d7-451b-b0ab-61b293cff35d" (UID: "43b07f3c-f9d7-451b-b0ab-61b293cff35d"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:54:49 crc kubenswrapper[5052]: I1203 12:54:49.459664 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/43b07f3c-f9d7-451b-b0ab-61b293cff35d-vencrypt-tls-certs" (OuterVolumeSpecName: "vencrypt-tls-certs") pod "43b07f3c-f9d7-451b-b0ab-61b293cff35d" (UID: "43b07f3c-f9d7-451b-b0ab-61b293cff35d"). InnerVolumeSpecName "vencrypt-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:54:49 crc kubenswrapper[5052]: I1203 12:54:49.462148 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/43b07f3c-f9d7-451b-b0ab-61b293cff35d-config-data" (OuterVolumeSpecName: "config-data") pod "43b07f3c-f9d7-451b-b0ab-61b293cff35d" (UID: "43b07f3c-f9d7-451b-b0ab-61b293cff35d"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:54:49 crc kubenswrapper[5052]: I1203 12:54:49.481655 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2b0bf9ae-2d1d-48da-8c4b-2973b47985a3-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "2b0bf9ae-2d1d-48da-8c4b-2973b47985a3" (UID: "2b0bf9ae-2d1d-48da-8c4b-2973b47985a3"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:54:49 crc kubenswrapper[5052]: I1203 12:54:49.494670 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/64ada63f-b17d-411b-a547-fa1c15107ab5-galera-tls-certs" (OuterVolumeSpecName: "galera-tls-certs") pod "64ada63f-b17d-411b-a547-fa1c15107ab5" (UID: "64ada63f-b17d-411b-a547-fa1c15107ab5"). InnerVolumeSpecName "galera-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:54:49 crc kubenswrapper[5052]: I1203 12:54:49.507117 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eefe83f8-0235-4b16-aefb-5ec7b2d92fcf-config-data" (OuterVolumeSpecName: "config-data") pod "eefe83f8-0235-4b16-aefb-5ec7b2d92fcf" (UID: "eefe83f8-0235-4b16-aefb-5ec7b2d92fcf"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:54:49 crc kubenswrapper[5052]: I1203 12:54:49.508384 5052 reconciler_common.go:293] "Volume detached for volume \"vencrypt-tls-certs\" (UniqueName: \"kubernetes.io/secret/43b07f3c-f9d7-451b-b0ab-61b293cff35d-vencrypt-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:49 crc kubenswrapper[5052]: I1203 12:54:49.508442 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/43b07f3c-f9d7-451b-b0ab-61b293cff35d-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:49 crc kubenswrapper[5052]: I1203 12:54:49.508455 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/eefe83f8-0235-4b16-aefb-5ec7b2d92fcf-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:49 crc kubenswrapper[5052]: I1203 12:54:49.508467 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/43b07f3c-f9d7-451b-b0ab-61b293cff35d-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:49 crc kubenswrapper[5052]: I1203 12:54:49.508480 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2b0bf9ae-2d1d-48da-8c4b-2973b47985a3-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:49 crc kubenswrapper[5052]: I1203 12:54:49.508808 5052 reconciler_common.go:293] "Volume detached for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/64ada63f-b17d-411b-a547-fa1c15107ab5-galera-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:49 crc kubenswrapper[5052]: I1203 12:54:49.523180 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/020bfa4c-3dba-43d2-9ac2-0bbbae660709-config-data" (OuterVolumeSpecName: "config-data") pod "020bfa4c-3dba-43d2-9ac2-0bbbae660709" (UID: "020bfa4c-3dba-43d2-9ac2-0bbbae660709"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:54:49 crc kubenswrapper[5052]: I1203 12:54:49.529166 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3c553114-e0c8-4af9-992f-5421b31497ed-kube-state-metrics-tls-config" (OuterVolumeSpecName: "kube-state-metrics-tls-config") pod "3c553114-e0c8-4af9-992f-5421b31497ed" (UID: "3c553114-e0c8-4af9-992f-5421b31497ed"). InnerVolumeSpecName "kube-state-metrics-tls-config". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:54:49 crc kubenswrapper[5052]: I1203 12:54:49.540282 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eefe83f8-0235-4b16-aefb-5ec7b2d92fcf-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "eefe83f8-0235-4b16-aefb-5ec7b2d92fcf" (UID: "eefe83f8-0235-4b16-aefb-5ec7b2d92fcf"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:54:49 crc kubenswrapper[5052]: I1203 12:54:49.554720 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ed40ab59-7f38-4b2a-ad68-85592fe22ef2-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "ed40ab59-7f38-4b2a-ad68-85592fe22ef2" (UID: "ed40ab59-7f38-4b2a-ad68-85592fe22ef2"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:54:49 crc kubenswrapper[5052]: I1203 12:54:49.589278 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ed40ab59-7f38-4b2a-ad68-85592fe22ef2-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "ed40ab59-7f38-4b2a-ad68-85592fe22ef2" (UID: "ed40ab59-7f38-4b2a-ad68-85592fe22ef2"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:54:49 crc kubenswrapper[5052]: I1203 12:54:49.605788 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/43b07f3c-f9d7-451b-b0ab-61b293cff35d-nova-novncproxy-tls-certs" (OuterVolumeSpecName: "nova-novncproxy-tls-certs") pod "43b07f3c-f9d7-451b-b0ab-61b293cff35d" (UID: "43b07f3c-f9d7-451b-b0ab-61b293cff35d"). InnerVolumeSpecName "nova-novncproxy-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:54:49 crc kubenswrapper[5052]: I1203 12:54:49.620885 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ed40ab59-7f38-4b2a-ad68-85592fe22ef2-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:49 crc kubenswrapper[5052]: I1203 12:54:49.620926 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/eefe83f8-0235-4b16-aefb-5ec7b2d92fcf-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:49 crc kubenswrapper[5052]: I1203 12:54:49.620939 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/020bfa4c-3dba-43d2-9ac2-0bbbae660709-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:49 crc kubenswrapper[5052]: I1203 12:54:49.620948 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/ed40ab59-7f38-4b2a-ad68-85592fe22ef2-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:49 crc kubenswrapper[5052]: I1203 12:54:49.621018 5052 reconciler_common.go:293] "Volume detached for volume \"kube-state-metrics-tls-config\" (UniqueName: \"kubernetes.io/secret/3c553114-e0c8-4af9-992f-5421b31497ed-kube-state-metrics-tls-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:49 crc kubenswrapper[5052]: I1203 12:54:49.621032 5052 reconciler_common.go:293] "Volume detached for volume \"nova-novncproxy-tls-certs\" (UniqueName: \"kubernetes.io/secret/43b07f3c-f9d7-451b-b0ab-61b293cff35d-nova-novncproxy-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:49 crc kubenswrapper[5052]: I1203 12:54:49.622170 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/novaapi7066-account-delete-whswm"] Dec 03 12:54:49 crc kubenswrapper[5052]: I1203 12:54:49.625032 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage11-crc" (UniqueName: "kubernetes.io/local-volume/local-storage11-crc") on node "crc" Dec 03 12:54:49 crc kubenswrapper[5052]: I1203 12:54:49.669810 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eefe83f8-0235-4b16-aefb-5ec7b2d92fcf-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "eefe83f8-0235-4b16-aefb-5ec7b2d92fcf" (UID: "eefe83f8-0235-4b16-aefb-5ec7b2d92fcf"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:54:49 crc kubenswrapper[5052]: I1203 12:54:49.709458 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ed40ab59-7f38-4b2a-ad68-85592fe22ef2-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "ed40ab59-7f38-4b2a-ad68-85592fe22ef2" (UID: "ed40ab59-7f38-4b2a-ad68-85592fe22ef2"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:54:49 crc kubenswrapper[5052]: I1203 12:54:49.714739 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="38bf6fea-894f-4f1c-841b-374540c44a38" path="/var/lib/kubelet/pods/38bf6fea-894f-4f1c-841b-374540c44a38/volumes" Dec 03 12:54:49 crc kubenswrapper[5052]: I1203 12:54:49.719802 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3c6d824e-c7da-41ff-adde-dbafb7535476" path="/var/lib/kubelet/pods/3c6d824e-c7da-41ff-adde-dbafb7535476/volumes" Dec 03 12:54:49 crc kubenswrapper[5052]: I1203 12:54:49.722707 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4d9b46e0-ffbc-4a61-8b07-f84d5b630f5a" path="/var/lib/kubelet/pods/4d9b46e0-ffbc-4a61-8b07-f84d5b630f5a/volumes" Dec 03 12:54:49 crc kubenswrapper[5052]: I1203 12:54:49.725189 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6d56c6dc-38c9-42cf-ac6d-04ba49860d31" path="/var/lib/kubelet/pods/6d56c6dc-38c9-42cf-ac6d-04ba49860d31/volumes" Dec 03 12:54:49 crc kubenswrapper[5052]: I1203 12:54:49.725852 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a3142a17-c4e9-49a1-95f3-f7b725516c93" path="/var/lib/kubelet/pods/a3142a17-c4e9-49a1-95f3-f7b725516c93/volumes" Dec 03 12:54:49 crc kubenswrapper[5052]: I1203 12:54:49.726424 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/ed40ab59-7f38-4b2a-ad68-85592fe22ef2-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:49 crc kubenswrapper[5052]: I1203 12:54:49.727010 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/eefe83f8-0235-4b16-aefb-5ec7b2d92fcf-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:49 crc kubenswrapper[5052]: I1203 12:54:49.727135 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage11-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage11-crc\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:49 crc kubenswrapper[5052]: I1203 12:54:49.726547 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e26fce77-6037-4d8f-bb16-f06cb151fdef" path="/var/lib/kubelet/pods/e26fce77-6037-4d8f-bb16-f06cb151fdef/volumes" Dec 03 12:54:49 crc kubenswrapper[5052]: I1203 12:54:49.736366 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3c553114-e0c8-4af9-992f-5421b31497ed-kube-state-metrics-tls-certs" (OuterVolumeSpecName: "kube-state-metrics-tls-certs") pod "3c553114-e0c8-4af9-992f-5421b31497ed" (UID: "3c553114-e0c8-4af9-992f-5421b31497ed"). InnerVolumeSpecName "kube-state-metrics-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:54:49 crc kubenswrapper[5052]: I1203 12:54:49.747250 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eefe83f8-0235-4b16-aefb-5ec7b2d92fcf-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "eefe83f8-0235-4b16-aefb-5ec7b2d92fcf" (UID: "eefe83f8-0235-4b16-aefb-5ec7b2d92fcf"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:54:49 crc kubenswrapper[5052]: I1203 12:54:49.754730 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ed40ab59-7f38-4b2a-ad68-85592fe22ef2-config-data" (OuterVolumeSpecName: "config-data") pod "ed40ab59-7f38-4b2a-ad68-85592fe22ef2" (UID: "ed40ab59-7f38-4b2a-ad68-85592fe22ef2"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:54:49 crc kubenswrapper[5052]: I1203 12:54:49.754973 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/64ada63f-b17d-411b-a547-fa1c15107ab5-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "64ada63f-b17d-411b-a547-fa1c15107ab5" (UID: "64ada63f-b17d-411b-a547-fa1c15107ab5"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:54:49 crc kubenswrapper[5052]: I1203 12:54:49.775038 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/020bfa4c-3dba-43d2-9ac2-0bbbae660709-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "020bfa4c-3dba-43d2-9ac2-0bbbae660709" (UID: "020bfa4c-3dba-43d2-9ac2-0bbbae660709"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:54:49 crc kubenswrapper[5052]: I1203 12:54:49.783989 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2b0bf9ae-2d1d-48da-8c4b-2973b47985a3-config-data" (OuterVolumeSpecName: "config-data") pod "2b0bf9ae-2d1d-48da-8c4b-2973b47985a3" (UID: "2b0bf9ae-2d1d-48da-8c4b-2973b47985a3"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:54:49 crc kubenswrapper[5052]: I1203 12:54:49.819771 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b2933134-7226-423a-8120-3148845b808a-config-data" (OuterVolumeSpecName: "config-data") pod "b2933134-7226-423a-8120-3148845b808a" (UID: "b2933134-7226-423a-8120-3148845b808a"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:54:49 crc kubenswrapper[5052]: I1203 12:54:49.829153 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/b2933134-7226-423a-8120-3148845b808a-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:49 crc kubenswrapper[5052]: I1203 12:54:49.829191 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/ed40ab59-7f38-4b2a-ad68-85592fe22ef2-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:49 crc kubenswrapper[5052]: I1203 12:54:49.829205 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/64ada63f-b17d-411b-a547-fa1c15107ab5-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:49 crc kubenswrapper[5052]: I1203 12:54:49.829217 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/2b0bf9ae-2d1d-48da-8c4b-2973b47985a3-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:49 crc kubenswrapper[5052]: I1203 12:54:49.829232 5052 reconciler_common.go:293] "Volume detached for volume \"kube-state-metrics-tls-certs\" (UniqueName: \"kubernetes.io/secret/3c553114-e0c8-4af9-992f-5421b31497ed-kube-state-metrics-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:49 crc kubenswrapper[5052]: I1203 12:54:49.829245 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/020bfa4c-3dba-43d2-9ac2-0bbbae660709-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:49 crc kubenswrapper[5052]: I1203 12:54:49.829256 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/eefe83f8-0235-4b16-aefb-5ec7b2d92fcf-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:49 crc kubenswrapper[5052]: E1203 12:54:49.829150 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/rabbitmq-cell1-config-data: configmap "rabbitmq-cell1-config-data" not found Dec 03 12:54:49 crc kubenswrapper[5052]: E1203 12:54:49.829331 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/568addd3-4ad3-47c3-adae-693691e7cd40-config-data podName:568addd3-4ad3-47c3-adae-693691e7cd40 nodeName:}" failed. No retries permitted until 2025-12-03 12:54:57.829310376 +0000 UTC m=+5688.417552140 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "config-data" (UniqueName: "kubernetes.io/configmap/568addd3-4ad3-47c3-adae-693691e7cd40-config-data") pod "rabbitmq-cell1-server-0" (UID: "568addd3-4ad3-47c3-adae-693691e7cd40") : configmap "rabbitmq-cell1-config-data" not found Dec 03 12:54:49 crc kubenswrapper[5052]: I1203 12:54:49.925265 5052 generic.go:334] "Generic (PLEG): container finished" podID="0191d138-7aff-42c2-88d2-fbbaf1722bfd" containerID="ac4ecfa5627541f93acf9cc63e185d94bb2e833e592e7703865bb5d4026f322f" exitCode=0 Dec 03 12:54:49 crc kubenswrapper[5052]: I1203 12:54:49.932967 5052 generic.go:334] "Generic (PLEG): container finished" podID="79efd225-5ce4-428f-aef9-3f4884931fef" containerID="833e52f94069f90f5fa1a410cc52bd28c6338e81c70d46f601fe9f83a3f59a7e" exitCode=1 Dec 03 12:54:50 crc kubenswrapper[5052]: I1203 12:54:50.001219 5052 generic.go:334] "Generic (PLEG): container finished" podID="d9bf91f1-099c-4e72-97cf-1828241aeaac" containerID="8c4808921d6ec6221a9f281a4ad3dbd4fefd44edf1b417c22977010e21618367" exitCode=1 Dec 03 12:54:50 crc kubenswrapper[5052]: E1203 12:54:50.032198 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:54:50 crc kubenswrapper[5052]: E1203 12:54:50.032221 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:54:50 crc kubenswrapper[5052]: E1203 12:54:50.032257 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c4d5c183-8df4-4c43-9afb-2b28c6aae9e1-operator-scripts podName:c4d5c183-8df4-4c43-9afb-2b28c6aae9e1 nodeName:}" failed. No retries permitted until 2025-12-03 12:54:54.03224326 +0000 UTC m=+5684.620485024 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/c4d5c183-8df4-4c43-9afb-2b28c6aae9e1-operator-scripts") pod "placementbfa4-account-delete-flc9f" (UID: "c4d5c183-8df4-4c43-9afb-2b28c6aae9e1") : configmap "openstack-scripts" not found Dec 03 12:54:50 crc kubenswrapper[5052]: E1203 12:54:50.032302 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/941d063f-c834-47a1-bc0c-2096b005e09c-operator-scripts podName:941d063f-c834-47a1-bc0c-2096b005e09c nodeName:}" failed. No retries permitted until 2025-12-03 12:54:54.032278831 +0000 UTC m=+5684.620520645 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/941d063f-c834-47a1-bc0c-2096b005e09c-operator-scripts") pod "neutron6c26-account-delete-d7wtw" (UID: "941d063f-c834-47a1-bc0c-2096b005e09c") : configmap "openstack-scripts" not found Dec 03 12:54:50 crc kubenswrapper[5052]: E1203 12:54:50.032313 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:54:50 crc kubenswrapper[5052]: E1203 12:54:50.032398 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d9bf91f1-099c-4e72-97cf-1828241aeaac-operator-scripts podName:d9bf91f1-099c-4e72-97cf-1828241aeaac nodeName:}" failed. No retries permitted until 2025-12-03 12:54:54.032375984 +0000 UTC m=+5684.620617808 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/d9bf91f1-099c-4e72-97cf-1828241aeaac-operator-scripts") pod "barbican6e62-account-delete-6skhm" (UID: "d9bf91f1-099c-4e72-97cf-1828241aeaac") : configmap "openstack-scripts" not found Dec 03 12:54:50 crc kubenswrapper[5052]: I1203 12:54:50.070797 5052 generic.go:334] "Generic (PLEG): container finished" podID="c4d5c183-8df4-4c43-9afb-2b28c6aae9e1" containerID="dddf220d5101c12f66ac552ba347af307cf1e8ac13a83847f15138397c57745d" exitCode=1 Dec 03 12:54:50 crc kubenswrapper[5052]: E1203 12:54:50.134572 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:54:50 crc kubenswrapper[5052]: E1203 12:54:50.134638 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e8a26759-2926-41cf-a1ec-286fc9962bb3-operator-scripts podName:e8a26759-2926-41cf-a1ec-286fc9962bb3 nodeName:}" failed. No retries permitted until 2025-12-03 12:54:54.134620198 +0000 UTC m=+5684.722861962 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/e8a26759-2926-41cf-a1ec-286fc9962bb3-operator-scripts") pod "cindera2a9-account-delete-pmspg" (UID: "e8a26759-2926-41cf-a1ec-286fc9962bb3") : configmap "openstack-scripts" not found Dec 03 12:54:50 crc kubenswrapper[5052]: I1203 12:54:50.159252 5052 generic.go:334] "Generic (PLEG): container finished" podID="b3363d6f-656e-4e22-a3eb-b1016380a0c8" containerID="b8ecae5dcbbd55bfb5a128bba207c90d7a6b85badc20f265e843e97165d22a9c" exitCode=0 Dec 03 12:54:50 crc kubenswrapper[5052]: I1203 12:54:50.168557 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-shdj6" podStartSLOduration=8.168530438 podStartE2EDuration="8.168530438s" podCreationTimestamp="2025-12-03 12:54:42 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:54:50.159667384 +0000 UTC m=+5680.747909138" watchObservedRunningTime="2025-12-03 12:54:50.168530438 +0000 UTC m=+5680.756772202" Dec 03 12:54:50 crc kubenswrapper[5052]: I1203 12:54:50.172357 5052 generic.go:334] "Generic (PLEG): container finished" podID="e8a26759-2926-41cf-a1ec-286fc9962bb3" containerID="caab470e87cf074ae88fc5a8c4c3ffe707912bf57f9da9988867733b15f35a11" exitCode=1 Dec 03 12:54:50 crc kubenswrapper[5052]: E1203 12:54:50.236126 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 12:54:50 crc kubenswrapper[5052]: E1203 12:54:50.236223 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/79efd225-5ce4-428f-aef9-3f4884931fef-operator-scripts podName:79efd225-5ce4-428f-aef9-3f4884931fef nodeName:}" failed. No retries permitted until 2025-12-03 12:54:54.236203383 +0000 UTC m=+5684.824445177 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/79efd225-5ce4-428f-aef9-3f4884931fef-operator-scripts") pod "glance7c80-account-delete-knmlm" (UID: "79efd225-5ce4-428f-aef9-3f4884931fef") : configmap "openstack-scripts" not found Dec 03 12:54:50 crc kubenswrapper[5052]: I1203 12:54:50.253177 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/novacell02629-account-delete-wsshh" event={"ID":"1b7ef5eb-406e-4e2c-9d9c-d763919f56fd","Type":"ContainerStarted","Data":"b90af27119e2a44164e92cfa6f8e83a087021fe45af6e255a1db739e22bee63b"} Dec 03 12:54:50 crc kubenswrapper[5052]: I1203 12:54:50.253991 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-shdj6" Dec 03 12:54:50 crc kubenswrapper[5052]: I1203 12:54:50.254036 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/novaapi7066-account-delete-whswm" event={"ID":"13e7e748-92f4-4255-a8eb-aa5a6faf361f","Type":"ContainerStarted","Data":"2323845f988980cd21781bad44c86e11b8a5edcb5a8f3351715e8e37eaba09a0"} Dec 03 12:54:50 crc kubenswrapper[5052]: I1203 12:54:50.254066 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-api-0" event={"ID":"7d297f47-a73f-4a88-bba0-3108deb8df4d","Type":"ContainerDied","Data":"cf937784e1e4cbf8b442e984fb4bf2a8b4cc9faedeeed62447a5cd38c718719e"} Dec 03 12:54:50 crc kubenswrapper[5052]: I1203 12:54:50.254089 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cf937784e1e4cbf8b442e984fb4bf2a8b4cc9faedeeed62447a5cd38c718719e" Dec 03 12:54:50 crc kubenswrapper[5052]: I1203 12:54:50.254106 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican-api-76dc68d888-zmmcs" event={"ID":"25eead1e-d17c-4b36-8a9f-ef563189d53b","Type":"ContainerDied","Data":"58a2643eb7cb7af49c47147ea6a9b799b0eb13ca461c58ef022848439681488b"} Dec 03 12:54:50 crc kubenswrapper[5052]: I1203 12:54:50.254123 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="58a2643eb7cb7af49c47147ea6a9b799b0eb13ca461c58ef022848439681488b" Dec 03 12:54:50 crc kubenswrapper[5052]: I1203 12:54:50.254143 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-7fd545bd9-2lc7s" event={"ID":"0191d138-7aff-42c2-88d2-fbbaf1722bfd","Type":"ContainerDied","Data":"ac4ecfa5627541f93acf9cc63e185d94bb2e833e592e7703865bb5d4026f322f"} Dec 03 12:54:50 crc kubenswrapper[5052]: I1203 12:54:50.254165 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/keystone-7fd545bd9-2lc7s" event={"ID":"0191d138-7aff-42c2-88d2-fbbaf1722bfd","Type":"ContainerDied","Data":"1e486dffdb8f8e897858840beacc3d7ec4d117da56b1dca2b0d95d7723129830"} Dec 03 12:54:50 crc kubenswrapper[5052]: I1203 12:54:50.254176 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1e486dffdb8f8e897858840beacc3d7ec4d117da56b1dca2b0d95d7723129830" Dec 03 12:54:50 crc kubenswrapper[5052]: I1203 12:54:50.254187 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-metadata-0" event={"ID":"022746a5-1c20-4729-8688-5903831f57ed","Type":"ContainerDied","Data":"ebb56e141b46e0dc6c812313228476a17019089f5138ec3df4797aac16a09903"} Dec 03 12:54:50 crc kubenswrapper[5052]: I1203 12:54:50.254199 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ebb56e141b46e0dc6c812313228476a17019089f5138ec3df4797aac16a09903" Dec 03 12:54:50 crc kubenswrapper[5052]: I1203 12:54:50.254219 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance7c80-account-delete-knmlm" event={"ID":"79efd225-5ce4-428f-aef9-3f4884931fef","Type":"ContainerDied","Data":"833e52f94069f90f5fa1a410cc52bd28c6338e81c70d46f601fe9f83a3f59a7e"} Dec 03 12:54:50 crc kubenswrapper[5052]: I1203 12:54:50.254239 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ceilometer-0" event={"ID":"02edb9e9-3f42-4aea-9fe6-7efa332d5fb8","Type":"ContainerDied","Data":"49eac351cf028c70490c6de088af1b224540eb2aa17cd045c37778ceb031307d"} Dec 03 12:54:50 crc kubenswrapper[5052]: I1203 12:54:50.254253 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="49eac351cf028c70490c6de088af1b224540eb2aa17cd045c37778ceb031307d" Dec 03 12:54:50 crc kubenswrapper[5052]: I1203 12:54:50.254268 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican6e62-account-delete-6skhm" event={"ID":"d9bf91f1-099c-4e72-97cf-1828241aeaac","Type":"ContainerDied","Data":"8c4808921d6ec6221a9f281a4ad3dbd4fefd44edf1b417c22977010e21618367"} Dec 03 12:54:50 crc kubenswrapper[5052]: I1203 12:54:50.254283 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-cell1-conductor-0" event={"ID":"2adad00d-01e3-4441-a3fc-87d162a5e309","Type":"ContainerDied","Data":"069817b8bb632c54923379bdbcb28b2aeaddd9b64161f40bc504fc2f25ddd00f"} Dec 03 12:54:50 crc kubenswrapper[5052]: I1203 12:54:50.254297 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="069817b8bb632c54923379bdbcb28b2aeaddd9b64161f40bc504fc2f25ddd00f" Dec 03 12:54:50 crc kubenswrapper[5052]: I1203 12:54:50.254307 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance-default-external-api-0" event={"ID":"cfc68960-6006-486c-bc03-848875d3bf5b","Type":"ContainerDied","Data":"2f252f5162cf02247034307341a5c641a56a84fe5d75bbe8d22f19b1c247c3cc"} Dec 03 12:54:50 crc kubenswrapper[5052]: I1203 12:54:50.254321 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2f252f5162cf02247034307341a5c641a56a84fe5d75bbe8d22f19b1c247c3cc" Dec 03 12:54:50 crc kubenswrapper[5052]: I1203 12:54:50.254340 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placementbfa4-account-delete-flc9f" event={"ID":"c4d5c183-8df4-4c43-9afb-2b28c6aae9e1","Type":"ContainerDied","Data":"dddf220d5101c12f66ac552ba347af307cf1e8ac13a83847f15138397c57745d"} Dec 03 12:54:50 crc kubenswrapper[5052]: I1203 12:54:50.254355 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placement-7b465d48b8-4bj69" event={"ID":"8df65ee4-b91d-4ffd-bff2-65e5e6559130","Type":"ContainerDied","Data":"97761280aaf2fdc7736eab0f506725b769a3dfd0ac8b0fcd7a289fffa429cb9a"} Dec 03 12:54:50 crc kubenswrapper[5052]: I1203 12:54:50.254368 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="97761280aaf2fdc7736eab0f506725b769a3dfd0ac8b0fcd7a289fffa429cb9a" Dec 03 12:54:50 crc kubenswrapper[5052]: I1203 12:54:50.254380 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-shdj6" event={"ID":"7bed2e44-93da-465d-9e10-6879ff1c82f5","Type":"ContainerStarted","Data":"7df281d8985229e1ba628e120c93ab3f824fd9bd67ac5fb03caed49d8f4608a9"} Dec 03 12:54:50 crc kubenswrapper[5052]: I1203 12:54:50.254392 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-galera-0" event={"ID":"b3363d6f-656e-4e22-a3eb-b1016380a0c8","Type":"ContainerDied","Data":"b8ecae5dcbbd55bfb5a128bba207c90d7a6b85badc20f265e843e97165d22a9c"} Dec 03 12:54:50 crc kubenswrapper[5052]: I1203 12:54:50.259495 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cindera2a9-account-delete-pmspg" event={"ID":"e8a26759-2926-41cf-a1ec-286fc9962bb3","Type":"ContainerDied","Data":"caab470e87cf074ae88fc5a8c4c3ffe707912bf57f9da9988867733b15f35a11"} Dec 03 12:54:50 crc kubenswrapper[5052]: I1203 12:54:50.283086 5052 scope.go:117] "RemoveContainer" containerID="ac795f902a20e66db9af6d237bc550efac4a63fdac0b163e8ceb8138a8c4a16f" Dec 03 12:54:50 crc kubenswrapper[5052]: I1203 12:54:50.348480 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/memcached-0" Dec 03 12:54:50 crc kubenswrapper[5052]: I1203 12:54:50.400566 5052 scope.go:117] "RemoveContainer" containerID="6ef42ed115f2247455a35e6dba9e2519ebc77a9c515e4efa379aa58e493e2dea" Dec 03 12:54:50 crc kubenswrapper[5052]: I1203 12:54:50.430038 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-7b465d48b8-4bj69" Dec 03 12:54:50 crc kubenswrapper[5052]: I1203 12:54:50.435880 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:54:50 crc kubenswrapper[5052]: I1203 12:54:50.440440 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d1ff5100-3ac7-4633-811c-ea470242d858-combined-ca-bundle\") pod \"d1ff5100-3ac7-4633-811c-ea470242d858\" (UID: \"d1ff5100-3ac7-4633-811c-ea470242d858\") " Dec 03 12:54:50 crc kubenswrapper[5052]: I1203 12:54:50.440607 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"memcached-tls-certs\" (UniqueName: \"kubernetes.io/secret/d1ff5100-3ac7-4633-811c-ea470242d858-memcached-tls-certs\") pod \"d1ff5100-3ac7-4633-811c-ea470242d858\" (UID: \"d1ff5100-3ac7-4633-811c-ea470242d858\") " Dec 03 12:54:50 crc kubenswrapper[5052]: I1203 12:54:50.440653 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v69rd\" (UniqueName: \"kubernetes.io/projected/d1ff5100-3ac7-4633-811c-ea470242d858-kube-api-access-v69rd\") pod \"d1ff5100-3ac7-4633-811c-ea470242d858\" (UID: \"d1ff5100-3ac7-4633-811c-ea470242d858\") " Dec 03 12:54:50 crc kubenswrapper[5052]: I1203 12:54:50.440843 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/d1ff5100-3ac7-4633-811c-ea470242d858-config-data\") pod \"d1ff5100-3ac7-4633-811c-ea470242d858\" (UID: \"d1ff5100-3ac7-4633-811c-ea470242d858\") " Dec 03 12:54:50 crc kubenswrapper[5052]: I1203 12:54:50.440890 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/d1ff5100-3ac7-4633-811c-ea470242d858-kolla-config\") pod \"d1ff5100-3ac7-4633-811c-ea470242d858\" (UID: \"d1ff5100-3ac7-4633-811c-ea470242d858\") " Dec 03 12:54:50 crc kubenswrapper[5052]: I1203 12:54:50.442884 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d1ff5100-3ac7-4633-811c-ea470242d858-kolla-config" (OuterVolumeSpecName: "kolla-config") pod "d1ff5100-3ac7-4633-811c-ea470242d858" (UID: "d1ff5100-3ac7-4633-811c-ea470242d858"). InnerVolumeSpecName "kolla-config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:54:50 crc kubenswrapper[5052]: I1203 12:54:50.443464 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d1ff5100-3ac7-4633-811c-ea470242d858-config-data" (OuterVolumeSpecName: "config-data") pod "d1ff5100-3ac7-4633-811c-ea470242d858" (UID: "d1ff5100-3ac7-4633-811c-ea470242d858"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:54:50 crc kubenswrapper[5052]: I1203 12:54:50.502764 5052 scope.go:117] "RemoveContainer" containerID="c45182c71bc38ac598875105b0f4d0cd30926ee88268d53941e68205eaeb487b" Dec 03 12:54:50 crc kubenswrapper[5052]: I1203 12:54:50.503072 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d1ff5100-3ac7-4633-811c-ea470242d858-kube-api-access-v69rd" (OuterVolumeSpecName: "kube-api-access-v69rd") pod "d1ff5100-3ac7-4633-811c-ea470242d858" (UID: "d1ff5100-3ac7-4633-811c-ea470242d858"). InnerVolumeSpecName "kube-api-access-v69rd". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:54:50 crc kubenswrapper[5052]: I1203 12:54:50.512358 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:54:50 crc kubenswrapper[5052]: I1203 12:54:50.526248 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d1ff5100-3ac7-4633-811c-ea470242d858-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "d1ff5100-3ac7-4633-811c-ea470242d858" (UID: "d1ff5100-3ac7-4633-811c-ea470242d858"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:54:50 crc kubenswrapper[5052]: I1203 12:54:50.541015 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:54:50 crc kubenswrapper[5052]: I1203 12:54:50.542595 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/981c3156-805a-44d1-b4ec-f688a8bc3cf8-internal-tls-certs\") pod \"981c3156-805a-44d1-b4ec-f688a8bc3cf8\" (UID: \"981c3156-805a-44d1-b4ec-f688a8bc3cf8\") " Dec 03 12:54:50 crc kubenswrapper[5052]: I1203 12:54:50.542644 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/8df65ee4-b91d-4ffd-bff2-65e5e6559130-scripts\") pod \"8df65ee4-b91d-4ffd-bff2-65e5e6559130\" (UID: \"8df65ee4-b91d-4ffd-bff2-65e5e6559130\") " Dec 03 12:54:50 crc kubenswrapper[5052]: I1203 12:54:50.542715 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/981c3156-805a-44d1-b4ec-f688a8bc3cf8-config-data\") pod \"981c3156-805a-44d1-b4ec-f688a8bc3cf8\" (UID: \"981c3156-805a-44d1-b4ec-f688a8bc3cf8\") " Dec 03 12:54:50 crc kubenswrapper[5052]: I1203 12:54:50.542762 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/8df65ee4-b91d-4ffd-bff2-65e5e6559130-internal-tls-certs\") pod \"8df65ee4-b91d-4ffd-bff2-65e5e6559130\" (UID: \"8df65ee4-b91d-4ffd-bff2-65e5e6559130\") " Dec 03 12:54:50 crc kubenswrapper[5052]: I1203 12:54:50.542796 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"glance\" (UniqueName: \"kubernetes.io/local-volume/local-storage06-crc\") pod \"981c3156-805a-44d1-b4ec-f688a8bc3cf8\" (UID: \"981c3156-805a-44d1-b4ec-f688a8bc3cf8\") " Dec 03 12:54:50 crc kubenswrapper[5052]: I1203 12:54:50.542826 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/981c3156-805a-44d1-b4ec-f688a8bc3cf8-scripts\") pod \"981c3156-805a-44d1-b4ec-f688a8bc3cf8\" (UID: \"981c3156-805a-44d1-b4ec-f688a8bc3cf8\") " Dec 03 12:54:50 crc kubenswrapper[5052]: I1203 12:54:50.542864 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/8df65ee4-b91d-4ffd-bff2-65e5e6559130-public-tls-certs\") pod \"8df65ee4-b91d-4ffd-bff2-65e5e6559130\" (UID: \"8df65ee4-b91d-4ffd-bff2-65e5e6559130\") " Dec 03 12:54:50 crc kubenswrapper[5052]: I1203 12:54:50.542979 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8df65ee4-b91d-4ffd-bff2-65e5e6559130-combined-ca-bundle\") pod \"8df65ee4-b91d-4ffd-bff2-65e5e6559130\" (UID: \"8df65ee4-b91d-4ffd-bff2-65e5e6559130\") " Dec 03 12:54:50 crc kubenswrapper[5052]: I1203 12:54:50.543021 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qj6ss\" (UniqueName: \"kubernetes.io/projected/8df65ee4-b91d-4ffd-bff2-65e5e6559130-kube-api-access-qj6ss\") pod \"8df65ee4-b91d-4ffd-bff2-65e5e6559130\" (UID: \"8df65ee4-b91d-4ffd-bff2-65e5e6559130\") " Dec 03 12:54:50 crc kubenswrapper[5052]: I1203 12:54:50.543093 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/8df65ee4-b91d-4ffd-bff2-65e5e6559130-logs\") pod \"8df65ee4-b91d-4ffd-bff2-65e5e6559130\" (UID: \"8df65ee4-b91d-4ffd-bff2-65e5e6559130\") " Dec 03 12:54:50 crc kubenswrapper[5052]: I1203 12:54:50.543123 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/981c3156-805a-44d1-b4ec-f688a8bc3cf8-combined-ca-bundle\") pod \"981c3156-805a-44d1-b4ec-f688a8bc3cf8\" (UID: \"981c3156-805a-44d1-b4ec-f688a8bc3cf8\") " Dec 03 12:54:50 crc kubenswrapper[5052]: I1203 12:54:50.543159 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dvtfh\" (UniqueName: \"kubernetes.io/projected/981c3156-805a-44d1-b4ec-f688a8bc3cf8-kube-api-access-dvtfh\") pod \"981c3156-805a-44d1-b4ec-f688a8bc3cf8\" (UID: \"981c3156-805a-44d1-b4ec-f688a8bc3cf8\") " Dec 03 12:54:50 crc kubenswrapper[5052]: I1203 12:54:50.543184 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/981c3156-805a-44d1-b4ec-f688a8bc3cf8-logs\") pod \"981c3156-805a-44d1-b4ec-f688a8bc3cf8\" (UID: \"981c3156-805a-44d1-b4ec-f688a8bc3cf8\") " Dec 03 12:54:50 crc kubenswrapper[5052]: I1203 12:54:50.543219 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/981c3156-805a-44d1-b4ec-f688a8bc3cf8-httpd-run\") pod \"981c3156-805a-44d1-b4ec-f688a8bc3cf8\" (UID: \"981c3156-805a-44d1-b4ec-f688a8bc3cf8\") " Dec 03 12:54:50 crc kubenswrapper[5052]: I1203 12:54:50.543251 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/8df65ee4-b91d-4ffd-bff2-65e5e6559130-config-data\") pod \"8df65ee4-b91d-4ffd-bff2-65e5e6559130\" (UID: \"8df65ee4-b91d-4ffd-bff2-65e5e6559130\") " Dec 03 12:54:50 crc kubenswrapper[5052]: I1203 12:54:50.543792 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/d1ff5100-3ac7-4633-811c-ea470242d858-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:50 crc kubenswrapper[5052]: I1203 12:54:50.543811 5052 reconciler_common.go:293] "Volume detached for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/d1ff5100-3ac7-4633-811c-ea470242d858-kolla-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:50 crc kubenswrapper[5052]: I1203 12:54:50.543826 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d1ff5100-3ac7-4633-811c-ea470242d858-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:50 crc kubenswrapper[5052]: I1203 12:54:50.543837 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-v69rd\" (UniqueName: \"kubernetes.io/projected/d1ff5100-3ac7-4633-811c-ea470242d858-kube-api-access-v69rd\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:50 crc kubenswrapper[5052]: I1203 12:54:50.555026 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/981c3156-805a-44d1-b4ec-f688a8bc3cf8-logs" (OuterVolumeSpecName: "logs") pod "981c3156-805a-44d1-b4ec-f688a8bc3cf8" (UID: "981c3156-805a-44d1-b4ec-f688a8bc3cf8"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:54:50 crc kubenswrapper[5052]: I1203 12:54:50.558874 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8df65ee4-b91d-4ffd-bff2-65e5e6559130-logs" (OuterVolumeSpecName: "logs") pod "8df65ee4-b91d-4ffd-bff2-65e5e6559130" (UID: "8df65ee4-b91d-4ffd-bff2-65e5e6559130"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:54:50 crc kubenswrapper[5052]: I1203 12:54:50.560331 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/981c3156-805a-44d1-b4ec-f688a8bc3cf8-httpd-run" (OuterVolumeSpecName: "httpd-run") pod "981c3156-805a-44d1-b4ec-f688a8bc3cf8" (UID: "981c3156-805a-44d1-b4ec-f688a8bc3cf8"). InnerVolumeSpecName "httpd-run". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:54:50 crc kubenswrapper[5052]: I1203 12:54:50.609147 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage06-crc" (OuterVolumeSpecName: "glance") pod "981c3156-805a-44d1-b4ec-f688a8bc3cf8" (UID: "981c3156-805a-44d1-b4ec-f688a8bc3cf8"). InnerVolumeSpecName "local-storage06-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 12:54:50 crc kubenswrapper[5052]: I1203 12:54:50.610437 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8df65ee4-b91d-4ffd-bff2-65e5e6559130-kube-api-access-qj6ss" (OuterVolumeSpecName: "kube-api-access-qj6ss") pod "8df65ee4-b91d-4ffd-bff2-65e5e6559130" (UID: "8df65ee4-b91d-4ffd-bff2-65e5e6559130"). InnerVolumeSpecName "kube-api-access-qj6ss". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:54:50 crc kubenswrapper[5052]: I1203 12:54:50.610513 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-worker-b7f5dfb6f-pj6bx"] Dec 03 12:54:50 crc kubenswrapper[5052]: I1203 12:54:50.615976 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/981c3156-805a-44d1-b4ec-f688a8bc3cf8-kube-api-access-dvtfh" (OuterVolumeSpecName: "kube-api-access-dvtfh") pod "981c3156-805a-44d1-b4ec-f688a8bc3cf8" (UID: "981c3156-805a-44d1-b4ec-f688a8bc3cf8"). InnerVolumeSpecName "kube-api-access-dvtfh". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:54:50 crc kubenswrapper[5052]: I1203 12:54:50.624944 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/barbican-worker-b7f5dfb6f-pj6bx"] Dec 03 12:54:50 crc kubenswrapper[5052]: I1203 12:54:50.631792 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 12:54:50 crc kubenswrapper[5052]: I1203 12:54:50.638889 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8df65ee4-b91d-4ffd-bff2-65e5e6559130-scripts" (OuterVolumeSpecName: "scripts") pod "8df65ee4-b91d-4ffd-bff2-65e5e6559130" (UID: "8df65ee4-b91d-4ffd-bff2-65e5e6559130"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:54:50 crc kubenswrapper[5052]: I1203 12:54:50.646663 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/981c3156-805a-44d1-b4ec-f688a8bc3cf8-scripts" (OuterVolumeSpecName: "scripts") pod "981c3156-805a-44d1-b4ec-f688a8bc3cf8" (UID: "981c3156-805a-44d1-b4ec-f688a8bc3cf8"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:54:50 crc kubenswrapper[5052]: I1203 12:54:50.646843 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/7d297f47-a73f-4a88-bba0-3108deb8df4d-internal-tls-certs\") pod \"7d297f47-a73f-4a88-bba0-3108deb8df4d\" (UID: \"7d297f47-a73f-4a88-bba0-3108deb8df4d\") " Dec 03 12:54:50 crc kubenswrapper[5052]: I1203 12:54:50.646887 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/7d297f47-a73f-4a88-bba0-3108deb8df4d-combined-ca-bundle\") pod \"7d297f47-a73f-4a88-bba0-3108deb8df4d\" (UID: \"7d297f47-a73f-4a88-bba0-3108deb8df4d\") " Dec 03 12:54:50 crc kubenswrapper[5052]: I1203 12:54:50.647022 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/cfc68960-6006-486c-bc03-848875d3bf5b-httpd-run\") pod \"cfc68960-6006-486c-bc03-848875d3bf5b\" (UID: \"cfc68960-6006-486c-bc03-848875d3bf5b\") " Dec 03 12:54:50 crc kubenswrapper[5052]: I1203 12:54:50.647083 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"glance\" (UniqueName: \"kubernetes.io/local-volume/local-storage19-crc\") pod \"cfc68960-6006-486c-bc03-848875d3bf5b\" (UID: \"cfc68960-6006-486c-bc03-848875d3bf5b\") " Dec 03 12:54:50 crc kubenswrapper[5052]: I1203 12:54:50.647120 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/7d297f47-a73f-4a88-bba0-3108deb8df4d-public-tls-certs\") pod \"7d297f47-a73f-4a88-bba0-3108deb8df4d\" (UID: \"7d297f47-a73f-4a88-bba0-3108deb8df4d\") " Dec 03 12:54:50 crc kubenswrapper[5052]: I1203 12:54:50.647147 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/cfc68960-6006-486c-bc03-848875d3bf5b-combined-ca-bundle\") pod \"cfc68960-6006-486c-bc03-848875d3bf5b\" (UID: \"cfc68960-6006-486c-bc03-848875d3bf5b\") " Dec 03 12:54:50 crc kubenswrapper[5052]: I1203 12:54:50.647169 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/cfc68960-6006-486c-bc03-848875d3bf5b-config-data\") pod \"cfc68960-6006-486c-bc03-848875d3bf5b\" (UID: \"cfc68960-6006-486c-bc03-848875d3bf5b\") " Dec 03 12:54:50 crc kubenswrapper[5052]: I1203 12:54:50.647206 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wdxzl\" (UniqueName: \"kubernetes.io/projected/cfc68960-6006-486c-bc03-848875d3bf5b-kube-api-access-wdxzl\") pod \"cfc68960-6006-486c-bc03-848875d3bf5b\" (UID: \"cfc68960-6006-486c-bc03-848875d3bf5b\") " Dec 03 12:54:50 crc kubenswrapper[5052]: I1203 12:54:50.647234 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5sp7g\" (UniqueName: \"kubernetes.io/projected/7d297f47-a73f-4a88-bba0-3108deb8df4d-kube-api-access-5sp7g\") pod \"7d297f47-a73f-4a88-bba0-3108deb8df4d\" (UID: \"7d297f47-a73f-4a88-bba0-3108deb8df4d\") " Dec 03 12:54:50 crc kubenswrapper[5052]: I1203 12:54:50.647261 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/cfc68960-6006-486c-bc03-848875d3bf5b-logs\") pod \"cfc68960-6006-486c-bc03-848875d3bf5b\" (UID: \"cfc68960-6006-486c-bc03-848875d3bf5b\") " Dec 03 12:54:50 crc kubenswrapper[5052]: I1203 12:54:50.647313 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/7d297f47-a73f-4a88-bba0-3108deb8df4d-config-data\") pod \"7d297f47-a73f-4a88-bba0-3108deb8df4d\" (UID: \"7d297f47-a73f-4a88-bba0-3108deb8df4d\") " Dec 03 12:54:50 crc kubenswrapper[5052]: I1203 12:54:50.647347 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/cfc68960-6006-486c-bc03-848875d3bf5b-public-tls-certs\") pod \"cfc68960-6006-486c-bc03-848875d3bf5b\" (UID: \"cfc68960-6006-486c-bc03-848875d3bf5b\") " Dec 03 12:54:50 crc kubenswrapper[5052]: I1203 12:54:50.647415 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/7d297f47-a73f-4a88-bba0-3108deb8df4d-logs\") pod \"7d297f47-a73f-4a88-bba0-3108deb8df4d\" (UID: \"7d297f47-a73f-4a88-bba0-3108deb8df4d\") " Dec 03 12:54:50 crc kubenswrapper[5052]: I1203 12:54:50.647451 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/cfc68960-6006-486c-bc03-848875d3bf5b-scripts\") pod \"cfc68960-6006-486c-bc03-848875d3bf5b\" (UID: \"cfc68960-6006-486c-bc03-848875d3bf5b\") " Dec 03 12:54:50 crc kubenswrapper[5052]: I1203 12:54:50.652823 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/8df65ee4-b91d-4ffd-bff2-65e5e6559130-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:50 crc kubenswrapper[5052]: I1203 12:54:50.652861 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage06-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage06-crc\") on node \"crc\" " Dec 03 12:54:50 crc kubenswrapper[5052]: I1203 12:54:50.652872 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/981c3156-805a-44d1-b4ec-f688a8bc3cf8-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:50 crc kubenswrapper[5052]: I1203 12:54:50.652884 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-qj6ss\" (UniqueName: \"kubernetes.io/projected/8df65ee4-b91d-4ffd-bff2-65e5e6559130-kube-api-access-qj6ss\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:50 crc kubenswrapper[5052]: I1203 12:54:50.652895 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/8df65ee4-b91d-4ffd-bff2-65e5e6559130-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:50 crc kubenswrapper[5052]: I1203 12:54:50.652904 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-dvtfh\" (UniqueName: \"kubernetes.io/projected/981c3156-805a-44d1-b4ec-f688a8bc3cf8-kube-api-access-dvtfh\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:50 crc kubenswrapper[5052]: I1203 12:54:50.652915 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/981c3156-805a-44d1-b4ec-f688a8bc3cf8-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:50 crc kubenswrapper[5052]: I1203 12:54:50.652923 5052 reconciler_common.go:293] "Volume detached for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/981c3156-805a-44d1-b4ec-f688a8bc3cf8-httpd-run\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:50 crc kubenswrapper[5052]: I1203 12:54:50.653841 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cfc68960-6006-486c-bc03-848875d3bf5b-logs" (OuterVolumeSpecName: "logs") pod "cfc68960-6006-486c-bc03-848875d3bf5b" (UID: "cfc68960-6006-486c-bc03-848875d3bf5b"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:54:50 crc kubenswrapper[5052]: I1203 12:54:50.655244 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cfc68960-6006-486c-bc03-848875d3bf5b-httpd-run" (OuterVolumeSpecName: "httpd-run") pod "cfc68960-6006-486c-bc03-848875d3bf5b" (UID: "cfc68960-6006-486c-bc03-848875d3bf5b"). InnerVolumeSpecName "httpd-run". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:54:50 crc kubenswrapper[5052]: I1203 12:54:50.655708 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d297f47-a73f-4a88-bba0-3108deb8df4d-logs" (OuterVolumeSpecName: "logs") pod "7d297f47-a73f-4a88-bba0-3108deb8df4d" (UID: "7d297f47-a73f-4a88-bba0-3108deb8df4d"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:54:50 crc kubenswrapper[5052]: I1203 12:54:50.660125 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/cinder-scheduler-0"] Dec 03 12:54:50 crc kubenswrapper[5052]: I1203 12:54:50.662611 5052 scope.go:117] "RemoveContainer" containerID="77bdaa2aad097bc96adc6a7e702b0e7e164f8b659c20495278f4d679ff5255fc" Dec 03 12:54:50 crc kubenswrapper[5052]: I1203 12:54:50.680564 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-keystone-listener-7b66cf7bb6-fsc4x"] Dec 03 12:54:50 crc kubenswrapper[5052]: I1203 12:54:50.690393 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/barbican-keystone-listener-7b66cf7bb6-fsc4x"] Dec 03 12:54:50 crc kubenswrapper[5052]: I1203 12:54:50.695872 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cfc68960-6006-486c-bc03-848875d3bf5b-scripts" (OuterVolumeSpecName: "scripts") pod "cfc68960-6006-486c-bc03-848875d3bf5b" (UID: "cfc68960-6006-486c-bc03-848875d3bf5b"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:54:50 crc kubenswrapper[5052]: I1203 12:54:50.702959 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/swift-proxy-5479cb94c8-q2snc"] Dec 03 12:54:50 crc kubenswrapper[5052]: I1203 12:54:50.711737 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/swift-proxy-5479cb94c8-q2snc"] Dec 03 12:54:50 crc kubenswrapper[5052]: I1203 12:54:50.721064 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/kube-state-metrics-0"] Dec 03 12:54:50 crc kubenswrapper[5052]: I1203 12:54:50.722561 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cfc68960-6006-486c-bc03-848875d3bf5b-kube-api-access-wdxzl" (OuterVolumeSpecName: "kube-api-access-wdxzl") pod "cfc68960-6006-486c-bc03-848875d3bf5b" (UID: "cfc68960-6006-486c-bc03-848875d3bf5b"). InnerVolumeSpecName "kube-api-access-wdxzl". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:54:50 crc kubenswrapper[5052]: I1203 12:54:50.727957 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage19-crc" (OuterVolumeSpecName: "glance") pod "cfc68960-6006-486c-bc03-848875d3bf5b" (UID: "cfc68960-6006-486c-bc03-848875d3bf5b"). InnerVolumeSpecName "local-storage19-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 12:54:50 crc kubenswrapper[5052]: I1203 12:54:50.729774 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7d297f47-a73f-4a88-bba0-3108deb8df4d-kube-api-access-5sp7g" (OuterVolumeSpecName: "kube-api-access-5sp7g") pod "7d297f47-a73f-4a88-bba0-3108deb8df4d" (UID: "7d297f47-a73f-4a88-bba0-3108deb8df4d"). InnerVolumeSpecName "kube-api-access-5sp7g". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:54:50 crc kubenswrapper[5052]: I1203 12:54:50.740495 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/kube-state-metrics-0"] Dec 03 12:54:50 crc kubenswrapper[5052]: I1203 12:54:50.751797 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 12:54:50 crc kubenswrapper[5052]: I1203 12:54:50.760706 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell1-novncproxy-0"] Dec 03 12:54:50 crc kubenswrapper[5052]: I1203 12:54:50.762130 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-5sp7g\" (UniqueName: \"kubernetes.io/projected/7d297f47-a73f-4a88-bba0-3108deb8df4d-kube-api-access-5sp7g\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:50 crc kubenswrapper[5052]: I1203 12:54:50.762171 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/cfc68960-6006-486c-bc03-848875d3bf5b-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:50 crc kubenswrapper[5052]: I1203 12:54:50.762183 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/7d297f47-a73f-4a88-bba0-3108deb8df4d-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:50 crc kubenswrapper[5052]: I1203 12:54:50.762195 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/cfc68960-6006-486c-bc03-848875d3bf5b-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:50 crc kubenswrapper[5052]: I1203 12:54:50.762209 5052 reconciler_common.go:293] "Volume detached for volume \"httpd-run\" (UniqueName: \"kubernetes.io/empty-dir/cfc68960-6006-486c-bc03-848875d3bf5b-httpd-run\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:50 crc kubenswrapper[5052]: I1203 12:54:50.762236 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage19-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage19-crc\") on node \"crc\" " Dec 03 12:54:50 crc kubenswrapper[5052]: I1203 12:54:50.762250 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-wdxzl\" (UniqueName: \"kubernetes.io/projected/cfc68960-6006-486c-bc03-848875d3bf5b-kube-api-access-wdxzl\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:50 crc kubenswrapper[5052]: I1203 12:54:50.774651 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 12:54:50 crc kubenswrapper[5052]: E1203 12:54:50.775879 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of b8ecae5dcbbd55bfb5a128bba207c90d7a6b85badc20f265e843e97165d22a9c is running failed: container process not found" containerID="b8ecae5dcbbd55bfb5a128bba207c90d7a6b85badc20f265e843e97165d22a9c" cmd=["/bin/bash","/var/lib/operator-scripts/mysql_probe.sh","readiness"] Dec 03 12:54:50 crc kubenswrapper[5052]: E1203 12:54:50.784983 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of b8ecae5dcbbd55bfb5a128bba207c90d7a6b85badc20f265e843e97165d22a9c is running failed: container process not found" containerID="b8ecae5dcbbd55bfb5a128bba207c90d7a6b85badc20f265e843e97165d22a9c" cmd=["/bin/bash","/var/lib/operator-scripts/mysql_probe.sh","readiness"] Dec 03 12:54:50 crc kubenswrapper[5052]: I1203 12:54:50.785091 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/cinder-api-0"] Dec 03 12:54:50 crc kubenswrapper[5052]: E1203 12:54:50.788728 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of b8ecae5dcbbd55bfb5a128bba207c90d7a6b85badc20f265e843e97165d22a9c is running failed: container process not found" containerID="b8ecae5dcbbd55bfb5a128bba207c90d7a6b85badc20f265e843e97165d22a9c" cmd=["/bin/bash","/var/lib/operator-scripts/mysql_probe.sh","readiness"] Dec 03 12:54:50 crc kubenswrapper[5052]: E1203 12:54:50.788802 5052 prober.go:104] "Probe errored" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of b8ecae5dcbbd55bfb5a128bba207c90d7a6b85badc20f265e843e97165d22a9c is running failed: container process not found" probeType="Readiness" pod="openstack-kuttl-tests/openstack-galera-0" podUID="b3363d6f-656e-4e22-a3eb-b1016380a0c8" containerName="galera" Dec 03 12:54:50 crc kubenswrapper[5052]: E1203 12:54:50.814732 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of 06bf7c042266ac81635b44d190310f81518aa47f7b8008ddde3f54ca515b15fd is running failed: container process not found" containerID="06bf7c042266ac81635b44d190310f81518aa47f7b8008ddde3f54ca515b15fd" cmd=["/usr/local/bin/container-scripts/status_check.sh"] Dec 03 12:54:50 crc kubenswrapper[5052]: E1203 12:54:50.821834 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of 06bf7c042266ac81635b44d190310f81518aa47f7b8008ddde3f54ca515b15fd is running failed: container process not found" containerID="06bf7c042266ac81635b44d190310f81518aa47f7b8008ddde3f54ca515b15fd" cmd=["/usr/local/bin/container-scripts/status_check.sh"] Dec 03 12:54:50 crc kubenswrapper[5052]: E1203 12:54:50.833894 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of 06bf7c042266ac81635b44d190310f81518aa47f7b8008ddde3f54ca515b15fd is running failed: container process not found" containerID="06bf7c042266ac81635b44d190310f81518aa47f7b8008ddde3f54ca515b15fd" cmd=["/usr/local/bin/container-scripts/status_check.sh"] Dec 03 12:54:50 crc kubenswrapper[5052]: E1203 12:54:50.834026 5052 prober.go:104] "Probe errored" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of 06bf7c042266ac81635b44d190310f81518aa47f7b8008ddde3f54ca515b15fd is running failed: container process not found" probeType="Readiness" pod="openstack-kuttl-tests/ovn-northd-0" podUID="3b93bc75-db87-4677-be2f-910d07bb696c" containerName="ovn-northd" Dec 03 12:54:51 crc kubenswrapper[5052]: I1203 12:54:51.027278 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7d297f47-a73f-4a88-bba0-3108deb8df4d-config-data" (OuterVolumeSpecName: "config-data") pod "7d297f47-a73f-4a88-bba0-3108deb8df4d" (UID: "7d297f47-a73f-4a88-bba0-3108deb8df4d"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:54:51 crc kubenswrapper[5052]: I1203 12:54:51.068443 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/7d297f47-a73f-4a88-bba0-3108deb8df4d-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:51 crc kubenswrapper[5052]: I1203 12:54:51.110048 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7d297f47-a73f-4a88-bba0-3108deb8df4d-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "7d297f47-a73f-4a88-bba0-3108deb8df4d" (UID: "7d297f47-a73f-4a88-bba0-3108deb8df4d"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:54:51 crc kubenswrapper[5052]: I1203 12:54:51.167970 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage06-crc" (UniqueName: "kubernetes.io/local-volume/local-storage06-crc") on node "crc" Dec 03 12:54:51 crc kubenswrapper[5052]: I1203 12:54:51.179112 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage06-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage06-crc\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:51 crc kubenswrapper[5052]: I1203 12:54:51.179159 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/7d297f47-a73f-4a88-bba0-3108deb8df4d-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:51 crc kubenswrapper[5052]: I1203 12:54:51.202085 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cfc68960-6006-486c-bc03-848875d3bf5b-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "cfc68960-6006-486c-bc03-848875d3bf5b" (UID: "cfc68960-6006-486c-bc03-848875d3bf5b"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:54:51 crc kubenswrapper[5052]: E1203 12:54:51.207861 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="ec5a6e0000f85ac80b044c5e702c36b00720989c7bd27aac0130a6739a2d1bd8" cmd=["/usr/bin/pgrep","-r","DRST","nova-scheduler"] Dec 03 12:54:51 crc kubenswrapper[5052]: E1203 12:54:51.210951 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="ec5a6e0000f85ac80b044c5e702c36b00720989c7bd27aac0130a6739a2d1bd8" cmd=["/usr/bin/pgrep","-r","DRST","nova-scheduler"] Dec 03 12:54:51 crc kubenswrapper[5052]: E1203 12:54:51.212662 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="ec5a6e0000f85ac80b044c5e702c36b00720989c7bd27aac0130a6739a2d1bd8" cmd=["/usr/bin/pgrep","-r","DRST","nova-scheduler"] Dec 03 12:54:51 crc kubenswrapper[5052]: E1203 12:54:51.212721 5052 prober.go:104] "Probe errored" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" probeType="Readiness" pod="openstack-kuttl-tests/nova-scheduler-0" podUID="4ec49e76-2597-47d6-811b-f65cbe0580ff" containerName="nova-scheduler-scheduler" Dec 03 12:54:51 crc kubenswrapper[5052]: I1203 12:54:51.257693 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cfc68960-6006-486c-bc03-848875d3bf5b-config-data" (OuterVolumeSpecName: "config-data") pod "cfc68960-6006-486c-bc03-848875d3bf5b" (UID: "cfc68960-6006-486c-bc03-848875d3bf5b"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:54:51 crc kubenswrapper[5052]: I1203 12:54:51.271819 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d1ff5100-3ac7-4633-811c-ea470242d858-memcached-tls-certs" (OuterVolumeSpecName: "memcached-tls-certs") pod "d1ff5100-3ac7-4633-811c-ea470242d858" (UID: "d1ff5100-3ac7-4633-811c-ea470242d858"). InnerVolumeSpecName "memcached-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:54:51 crc kubenswrapper[5052]: I1203 12:54:51.279639 5052 generic.go:334] "Generic (PLEG): container finished" podID="e2225738-b0d1-4306-bbee-a61dd9558c80" containerID="b7a1da6db38db2f0ed2c6f73c89998af0cc42716d17e3513590eabdf3cc5aa71" exitCode=0 Dec 03 12:54:51 crc kubenswrapper[5052]: I1203 12:54:51.279675 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-698d8977cd-ppz9n" event={"ID":"e2225738-b0d1-4306-bbee-a61dd9558c80","Type":"ContainerDied","Data":"b7a1da6db38db2f0ed2c6f73c89998af0cc42716d17e3513590eabdf3cc5aa71"} Dec 03 12:54:51 crc kubenswrapper[5052]: I1203 12:54:51.280836 5052 reconciler_common.go:293] "Volume detached for volume \"memcached-tls-certs\" (UniqueName: \"kubernetes.io/secret/d1ff5100-3ac7-4633-811c-ea470242d858-memcached-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:51 crc kubenswrapper[5052]: I1203 12:54:51.280873 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/cfc68960-6006-486c-bc03-848875d3bf5b-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:51 crc kubenswrapper[5052]: I1203 12:54:51.280886 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/cfc68960-6006-486c-bc03-848875d3bf5b-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:51 crc kubenswrapper[5052]: I1203 12:54:51.288748 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/barbican6e62-account-delete-6skhm" event={"ID":"d9bf91f1-099c-4e72-97cf-1828241aeaac","Type":"ContainerDied","Data":"f24f8c65ed321344bff617dc2efb47214ab4325842fbde63688138a762aadbdb"} Dec 03 12:54:51 crc kubenswrapper[5052]: I1203 12:54:51.288799 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f24f8c65ed321344bff617dc2efb47214ab4325842fbde63688138a762aadbdb" Dec 03 12:54:51 crc kubenswrapper[5052]: I1203 12:54:51.293466 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/openstack-galera-0" event={"ID":"b3363d6f-656e-4e22-a3eb-b1016380a0c8","Type":"ContainerDied","Data":"5e15fc793d245bf9358ad40489776225b5cbe20697b0c58370c4fa680dc083c4"} Dec 03 12:54:51 crc kubenswrapper[5052]: I1203 12:54:51.293552 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5e15fc793d245bf9358ad40489776225b5cbe20697b0c58370c4fa680dc083c4" Dec 03 12:54:51 crc kubenswrapper[5052]: I1203 12:54:51.300254 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/cindera2a9-account-delete-pmspg" event={"ID":"e8a26759-2926-41cf-a1ec-286fc9962bb3","Type":"ContainerDied","Data":"7f88d68507ef2d369c5fe3dce319cce0c03f0265b1bc25ed04d6d8be9cafb545"} Dec 03 12:54:51 crc kubenswrapper[5052]: I1203 12:54:51.300306 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7f88d68507ef2d369c5fe3dce319cce0c03f0265b1bc25ed04d6d8be9cafb545" Dec 03 12:54:51 crc kubenswrapper[5052]: I1203 12:54:51.302817 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_ovn-northd-0_3b93bc75-db87-4677-be2f-910d07bb696c/ovn-northd/0.log" Dec 03 12:54:51 crc kubenswrapper[5052]: I1203 12:54:51.302872 5052 generic.go:334] "Generic (PLEG): container finished" podID="3b93bc75-db87-4677-be2f-910d07bb696c" containerID="06bf7c042266ac81635b44d190310f81518aa47f7b8008ddde3f54ca515b15fd" exitCode=139 Dec 03 12:54:51 crc kubenswrapper[5052]: I1203 12:54:51.302939 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovn-northd-0" event={"ID":"3b93bc75-db87-4677-be2f-910d07bb696c","Type":"ContainerDied","Data":"06bf7c042266ac81635b44d190310f81518aa47f7b8008ddde3f54ca515b15fd"} Dec 03 12:54:51 crc kubenswrapper[5052]: I1203 12:54:51.308572 5052 generic.go:334] "Generic (PLEG): container finished" podID="e73813f5-f97f-41dd-8a79-6196557d4d1a" containerID="db4b319560fefd949092d7f9bbfd3ae7aab4d1e6d1c4c90e729f406df14b8b79" exitCode=0 Dec 03 12:54:51 crc kubenswrapper[5052]: I1203 12:54:51.308623 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-server-0" event={"ID":"e73813f5-f97f-41dd-8a79-6196557d4d1a","Type":"ContainerDied","Data":"db4b319560fefd949092d7f9bbfd3ae7aab4d1e6d1c4c90e729f406df14b8b79"} Dec 03 12:54:51 crc kubenswrapper[5052]: I1203 12:54:51.316165 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/981c3156-805a-44d1-b4ec-f688a8bc3cf8-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "981c3156-805a-44d1-b4ec-f688a8bc3cf8" (UID: "981c3156-805a-44d1-b4ec-f688a8bc3cf8"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:54:51 crc kubenswrapper[5052]: I1203 12:54:51.318771 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage19-crc" (UniqueName: "kubernetes.io/local-volume/local-storage19-crc") on node "crc" Dec 03 12:54:51 crc kubenswrapper[5052]: I1203 12:54:51.328576 5052 generic.go:334] "Generic (PLEG): container finished" podID="568addd3-4ad3-47c3-adae-693691e7cd40" containerID="dc308258e1e18beb76be744d3270524d74c7f8be911cbba0e2cd35f103d54af5" exitCode=0 Dec 03 12:54:51 crc kubenswrapper[5052]: I1203 12:54:51.328694 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" event={"ID":"568addd3-4ad3-47c3-adae-693691e7cd40","Type":"ContainerDied","Data":"dc308258e1e18beb76be744d3270524d74c7f8be911cbba0e2cd35f103d54af5"} Dec 03 12:54:51 crc kubenswrapper[5052]: I1203 12:54:51.331963 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/glance7c80-account-delete-knmlm" event={"ID":"79efd225-5ce4-428f-aef9-3f4884931fef","Type":"ContainerDied","Data":"7e9917be03250448576169c69cea7030e3fc164ff5f88cdf282c32c5d212bbdb"} Dec 03 12:54:51 crc kubenswrapper[5052]: I1203 12:54:51.332122 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7e9917be03250448576169c69cea7030e3fc164ff5f88cdf282c32c5d212bbdb" Dec 03 12:54:51 crc kubenswrapper[5052]: I1203 12:54:51.334412 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/placementbfa4-account-delete-flc9f" event={"ID":"c4d5c183-8df4-4c43-9afb-2b28c6aae9e1","Type":"ContainerDied","Data":"ca28d19362660f7ce08f01405dcf6a9e78f1824720527f0865bf72dc4425f918"} Dec 03 12:54:51 crc kubenswrapper[5052]: I1203 12:54:51.334650 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ca28d19362660f7ce08f01405dcf6a9e78f1824720527f0865bf72dc4425f918" Dec 03 12:54:51 crc kubenswrapper[5052]: I1203 12:54:51.338170 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/novaapi7066-account-delete-whswm" event={"ID":"13e7e748-92f4-4255-a8eb-aa5a6faf361f","Type":"ContainerStarted","Data":"9c5fb38f25954ef5355f2862f8e1ab7d9a859ef546a60f10dfb32dbd818c660c"} Dec 03 12:54:51 crc kubenswrapper[5052]: I1203 12:54:51.347387 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/novacell02629-account-delete-wsshh" event={"ID":"1b7ef5eb-406e-4e2c-9d9c-d763919f56fd","Type":"ContainerStarted","Data":"c9f66104fbca885ffe17ff3f4d917326325460800a0171775b450385bf7d5784"} Dec 03 12:54:51 crc kubenswrapper[5052]: I1203 12:54:51.347892 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/novacell02629-account-delete-wsshh" podUID="1b7ef5eb-406e-4e2c-9d9c-d763919f56fd" containerName="mariadb-account-delete" containerID="cri-o://c9f66104fbca885ffe17ff3f4d917326325460800a0171775b450385bf7d5784" gracePeriod=30 Dec 03 12:54:51 crc kubenswrapper[5052]: I1203 12:54:51.384316 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-internal-api-0" Dec 03 12:54:51 crc kubenswrapper[5052]: I1203 12:54:51.384518 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron6c26-account-delete-d7wtw" event={"ID":"941d063f-c834-47a1-bc0c-2096b005e09c","Type":"ContainerStarted","Data":"4db372b51214205aef5ecd524e9175a7ef98e7227daf49311eab08a95f8ca2a4"} Dec 03 12:54:51 crc kubenswrapper[5052]: I1203 12:54:51.384701 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placement-7b465d48b8-4bj69" Dec 03 12:54:51 crc kubenswrapper[5052]: I1203 12:54:51.384866 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance-default-external-api-0" Dec 03 12:54:51 crc kubenswrapper[5052]: I1203 12:54:51.384907 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/memcached-0" Dec 03 12:54:51 crc kubenswrapper[5052]: I1203 12:54:51.387096 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-api-0" Dec 03 12:54:51 crc kubenswrapper[5052]: I1203 12:54:51.391833 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage19-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage19-crc\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:51 crc kubenswrapper[5052]: I1203 12:54:51.391869 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/981c3156-805a-44d1-b4ec-f688a8bc3cf8-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:51 crc kubenswrapper[5052]: I1203 12:54:51.423965 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/novacell02629-account-delete-wsshh" podStartSLOduration=9.4239401 podStartE2EDuration="9.4239401s" podCreationTimestamp="2025-12-03 12:54:42 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:54:51.38441863 +0000 UTC m=+5681.972660394" watchObservedRunningTime="2025-12-03 12:54:51.4239401 +0000 UTC m=+5682.012181864" Dec 03 12:54:51 crc kubenswrapper[5052]: I1203 12:54:51.426940 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/981c3156-805a-44d1-b4ec-f688a8bc3cf8-config-data" (OuterVolumeSpecName: "config-data") pod "981c3156-805a-44d1-b4ec-f688a8bc3cf8" (UID: "981c3156-805a-44d1-b4ec-f688a8bc3cf8"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:54:51 crc kubenswrapper[5052]: I1203 12:54:51.427192 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7d297f47-a73f-4a88-bba0-3108deb8df4d-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "7d297f47-a73f-4a88-bba0-3108deb8df4d" (UID: "7d297f47-a73f-4a88-bba0-3108deb8df4d"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:54:51 crc kubenswrapper[5052]: I1203 12:54:51.442885 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7d297f47-a73f-4a88-bba0-3108deb8df4d-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "7d297f47-a73f-4a88-bba0-3108deb8df4d" (UID: "7d297f47-a73f-4a88-bba0-3108deb8df4d"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:54:51 crc kubenswrapper[5052]: I1203 12:54:51.446312 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8df65ee4-b91d-4ffd-bff2-65e5e6559130-config-data" (OuterVolumeSpecName: "config-data") pod "8df65ee4-b91d-4ffd-bff2-65e5e6559130" (UID: "8df65ee4-b91d-4ffd-bff2-65e5e6559130"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:54:51 crc kubenswrapper[5052]: I1203 12:54:51.470536 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cfc68960-6006-486c-bc03-848875d3bf5b-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "cfc68960-6006-486c-bc03-848875d3bf5b" (UID: "cfc68960-6006-486c-bc03-848875d3bf5b"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:54:51 crc kubenswrapper[5052]: I1203 12:54:51.485657 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8df65ee4-b91d-4ffd-bff2-65e5e6559130-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "8df65ee4-b91d-4ffd-bff2-65e5e6559130" (UID: "8df65ee4-b91d-4ffd-bff2-65e5e6559130"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:54:51 crc kubenswrapper[5052]: I1203 12:54:51.499031 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/7d297f47-a73f-4a88-bba0-3108deb8df4d-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:51 crc kubenswrapper[5052]: I1203 12:54:51.499071 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8df65ee4-b91d-4ffd-bff2-65e5e6559130-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:51 crc kubenswrapper[5052]: I1203 12:54:51.499084 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/7d297f47-a73f-4a88-bba0-3108deb8df4d-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:51 crc kubenswrapper[5052]: I1203 12:54:51.499096 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/8df65ee4-b91d-4ffd-bff2-65e5e6559130-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:51 crc kubenswrapper[5052]: I1203 12:54:51.499110 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/cfc68960-6006-486c-bc03-848875d3bf5b-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:51 crc kubenswrapper[5052]: I1203 12:54:51.499121 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/981c3156-805a-44d1-b4ec-f688a8bc3cf8-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:51 crc kubenswrapper[5052]: I1203 12:54:51.508708 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/981c3156-805a-44d1-b4ec-f688a8bc3cf8-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "981c3156-805a-44d1-b4ec-f688a8bc3cf8" (UID: "981c3156-805a-44d1-b4ec-f688a8bc3cf8"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:54:51 crc kubenswrapper[5052]: I1203 12:54:51.596889 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8df65ee4-b91d-4ffd-bff2-65e5e6559130-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "8df65ee4-b91d-4ffd-bff2-65e5e6559130" (UID: "8df65ee4-b91d-4ffd-bff2-65e5e6559130"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:54:51 crc kubenswrapper[5052]: I1203 12:54:51.601153 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/981c3156-805a-44d1-b4ec-f688a8bc3cf8-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:51 crc kubenswrapper[5052]: I1203 12:54:51.601198 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/8df65ee4-b91d-4ffd-bff2-65e5e6559130-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:51 crc kubenswrapper[5052]: I1203 12:54:51.639138 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8df65ee4-b91d-4ffd-bff2-65e5e6559130-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "8df65ee4-b91d-4ffd-bff2-65e5e6559130" (UID: "8df65ee4-b91d-4ffd-bff2-65e5e6559130"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:54:51 crc kubenswrapper[5052]: I1203 12:54:51.702522 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/8df65ee4-b91d-4ffd-bff2-65e5e6559130-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:51 crc kubenswrapper[5052]: I1203 12:54:51.707975 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/kube-state-metrics-0" podUID="3c553114-e0c8-4af9-992f-5421b31497ed" containerName="kube-state-metrics" probeResult="failure" output="Get \"https://10.217.0.198:8081/readyz\": dial tcp 10.217.0.198:8081: i/o timeout" Dec 03 12:54:51 crc kubenswrapper[5052]: I1203 12:54:51.710932 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="020bfa4c-3dba-43d2-9ac2-0bbbae660709" path="/var/lib/kubelet/pods/020bfa4c-3dba-43d2-9ac2-0bbbae660709/volumes" Dec 03 12:54:51 crc kubenswrapper[5052]: I1203 12:54:51.711742 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2b0bf9ae-2d1d-48da-8c4b-2973b47985a3" path="/var/lib/kubelet/pods/2b0bf9ae-2d1d-48da-8c4b-2973b47985a3/volumes" Dec 03 12:54:51 crc kubenswrapper[5052]: I1203 12:54:51.712893 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3c553114-e0c8-4af9-992f-5421b31497ed" path="/var/lib/kubelet/pods/3c553114-e0c8-4af9-992f-5421b31497ed/volumes" Dec 03 12:54:51 crc kubenswrapper[5052]: I1203 12:54:51.714645 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="43b07f3c-f9d7-451b-b0ab-61b293cff35d" path="/var/lib/kubelet/pods/43b07f3c-f9d7-451b-b0ab-61b293cff35d/volumes" Dec 03 12:54:51 crc kubenswrapper[5052]: I1203 12:54:51.715882 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b2933134-7226-423a-8120-3148845b808a" path="/var/lib/kubelet/pods/b2933134-7226-423a-8120-3148845b808a/volumes" Dec 03 12:54:51 crc kubenswrapper[5052]: I1203 12:54:51.717104 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ed40ab59-7f38-4b2a-ad68-85592fe22ef2" path="/var/lib/kubelet/pods/ed40ab59-7f38-4b2a-ad68-85592fe22ef2/volumes" Dec 03 12:54:51 crc kubenswrapper[5052]: I1203 12:54:51.718869 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eefe83f8-0235-4b16-aefb-5ec7b2d92fcf" path="/var/lib/kubelet/pods/eefe83f8-0235-4b16-aefb-5ec7b2d92fcf/volumes" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.127894 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.140318 5052 scope.go:117] "RemoveContainer" containerID="0d0807a346577955962a640bd5fe5f8e366a538b40576ef0017fa65770ea3118" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.156886 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-api-76dc68d888-zmmcs" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.173747 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.196277 5052 scope.go:117] "RemoveContainer" containerID="717a9f8ebf7eb71e44fa8c9d63200534cfb506f60207616c8ff568f62f9287a8" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.202248 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.204191 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.219162 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-7fd545bd9-2lc7s" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.219750 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/glance-default-external-api-0"] Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.242453 5052 scope.go:117] "RemoveContainer" containerID="594b90765c644ac99bd8e077dd10ec16881d4e9f1364dcdc6541052085119849" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.242594 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/placement-7b465d48b8-4bj69"] Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.244713 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.253796 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance7c80-account-delete-knmlm" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.267331 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cindera2a9-account-delete-pmspg" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.270374 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/placement-7b465d48b8-4bj69"] Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.277574 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican6e62-account-delete-6skhm" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.285114 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placementbfa4-account-delete-flc9f" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.290745 5052 scope.go:117] "RemoveContainer" containerID="0dc2d2dd56838e5a1533bde8204be1b738b226cd95e31ea3ee837e37c7cc98a8" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.297542 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.302064 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.304503 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/glance-default-internal-api-0"] Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.311392 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.316916 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/0191d138-7aff-42c2-88d2-fbbaf1722bfd-credential-keys\") pod \"0191d138-7aff-42c2-88d2-fbbaf1722bfd\" (UID: \"0191d138-7aff-42c2-88d2-fbbaf1722bfd\") " Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.317245 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2adad00d-01e3-4441-a3fc-87d162a5e309-combined-ca-bundle\") pod \"2adad00d-01e3-4441-a3fc-87d162a5e309\" (UID: \"2adad00d-01e3-4441-a3fc-87d162a5e309\") " Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.317857 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/0191d138-7aff-42c2-88d2-fbbaf1722bfd-internal-tls-certs\") pod \"0191d138-7aff-42c2-88d2-fbbaf1722bfd\" (UID: \"0191d138-7aff-42c2-88d2-fbbaf1722bfd\") " Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.317964 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/25eead1e-d17c-4b36-8a9f-ef563189d53b-combined-ca-bundle\") pod \"25eead1e-d17c-4b36-8a9f-ef563189d53b\" (UID: \"25eead1e-d17c-4b36-8a9f-ef563189d53b\") " Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.318262 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/022746a5-1c20-4729-8688-5903831f57ed-nova-metadata-tls-certs\") pod \"022746a5-1c20-4729-8688-5903831f57ed\" (UID: \"022746a5-1c20-4729-8688-5903831f57ed\") " Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.319468 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/0191d138-7aff-42c2-88d2-fbbaf1722bfd-scripts\") pod \"0191d138-7aff-42c2-88d2-fbbaf1722bfd\" (UID: \"0191d138-7aff-42c2-88d2-fbbaf1722bfd\") " Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.319565 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/022746a5-1c20-4729-8688-5903831f57ed-combined-ca-bundle\") pod \"022746a5-1c20-4729-8688-5903831f57ed\" (UID: \"022746a5-1c20-4729-8688-5903831f57ed\") " Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.319641 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/02edb9e9-3f42-4aea-9fe6-7efa332d5fb8-log-httpd\") pod \"02edb9e9-3f42-4aea-9fe6-7efa332d5fb8\" (UID: \"02edb9e9-3f42-4aea-9fe6-7efa332d5fb8\") " Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.319743 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/2adad00d-01e3-4441-a3fc-87d162a5e309-config-data\") pod \"2adad00d-01e3-4441-a3fc-87d162a5e309\" (UID: \"2adad00d-01e3-4441-a3fc-87d162a5e309\") " Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.319838 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xqxxd\" (UniqueName: \"kubernetes.io/projected/0191d138-7aff-42c2-88d2-fbbaf1722bfd-kube-api-access-xqxxd\") pod \"0191d138-7aff-42c2-88d2-fbbaf1722bfd\" (UID: \"0191d138-7aff-42c2-88d2-fbbaf1722bfd\") " Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.319903 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/25eead1e-d17c-4b36-8a9f-ef563189d53b-config-data-custom\") pod \"25eead1e-d17c-4b36-8a9f-ef563189d53b\" (UID: \"25eead1e-d17c-4b36-8a9f-ef563189d53b\") " Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.319978 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/0191d138-7aff-42c2-88d2-fbbaf1722bfd-public-tls-certs\") pod \"0191d138-7aff-42c2-88d2-fbbaf1722bfd\" (UID: \"0191d138-7aff-42c2-88d2-fbbaf1722bfd\") " Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.320069 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rtlhb\" (UniqueName: \"kubernetes.io/projected/02edb9e9-3f42-4aea-9fe6-7efa332d5fb8-kube-api-access-rtlhb\") pod \"02edb9e9-3f42-4aea-9fe6-7efa332d5fb8\" (UID: \"02edb9e9-3f42-4aea-9fe6-7efa332d5fb8\") " Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.320158 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/02edb9e9-3f42-4aea-9fe6-7efa332d5fb8-ceilometer-tls-certs\") pod \"02edb9e9-3f42-4aea-9fe6-7efa332d5fb8\" (UID: \"02edb9e9-3f42-4aea-9fe6-7efa332d5fb8\") " Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.320244 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xf64v\" (UniqueName: \"kubernetes.io/projected/022746a5-1c20-4729-8688-5903831f57ed-kube-api-access-xf64v\") pod \"022746a5-1c20-4729-8688-5903831f57ed\" (UID: \"022746a5-1c20-4729-8688-5903831f57ed\") " Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.320377 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/0191d138-7aff-42c2-88d2-fbbaf1722bfd-fernet-keys\") pod \"0191d138-7aff-42c2-88d2-fbbaf1722bfd\" (UID: \"0191d138-7aff-42c2-88d2-fbbaf1722bfd\") " Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.320457 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/0191d138-7aff-42c2-88d2-fbbaf1722bfd-config-data\") pod \"0191d138-7aff-42c2-88d2-fbbaf1722bfd\" (UID: \"0191d138-7aff-42c2-88d2-fbbaf1722bfd\") " Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.320562 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/25eead1e-d17c-4b36-8a9f-ef563189d53b-logs\") pod \"25eead1e-d17c-4b36-8a9f-ef563189d53b\" (UID: \"25eead1e-d17c-4b36-8a9f-ef563189d53b\") " Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.320639 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/02edb9e9-3f42-4aea-9fe6-7efa332d5fb8-run-httpd\") pod \"02edb9e9-3f42-4aea-9fe6-7efa332d5fb8\" (UID: \"02edb9e9-3f42-4aea-9fe6-7efa332d5fb8\") " Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.320709 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0191d138-7aff-42c2-88d2-fbbaf1722bfd-combined-ca-bundle\") pod \"0191d138-7aff-42c2-88d2-fbbaf1722bfd\" (UID: \"0191d138-7aff-42c2-88d2-fbbaf1722bfd\") " Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.320770 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/02edb9e9-3f42-4aea-9fe6-7efa332d5fb8-combined-ca-bundle\") pod \"02edb9e9-3f42-4aea-9fe6-7efa332d5fb8\" (UID: \"02edb9e9-3f42-4aea-9fe6-7efa332d5fb8\") " Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.320874 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/25eead1e-d17c-4b36-8a9f-ef563189d53b-public-tls-certs\") pod \"25eead1e-d17c-4b36-8a9f-ef563189d53b\" (UID: \"25eead1e-d17c-4b36-8a9f-ef563189d53b\") " Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.321111 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/02edb9e9-3f42-4aea-9fe6-7efa332d5fb8-scripts\") pod \"02edb9e9-3f42-4aea-9fe6-7efa332d5fb8\" (UID: \"02edb9e9-3f42-4aea-9fe6-7efa332d5fb8\") " Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.321226 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/02edb9e9-3f42-4aea-9fe6-7efa332d5fb8-config-data\") pod \"02edb9e9-3f42-4aea-9fe6-7efa332d5fb8\" (UID: \"02edb9e9-3f42-4aea-9fe6-7efa332d5fb8\") " Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.321389 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/022746a5-1c20-4729-8688-5903831f57ed-config-data\") pod \"022746a5-1c20-4729-8688-5903831f57ed\" (UID: \"022746a5-1c20-4729-8688-5903831f57ed\") " Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.321465 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/25eead1e-d17c-4b36-8a9f-ef563189d53b-config-data\") pod \"25eead1e-d17c-4b36-8a9f-ef563189d53b\" (UID: \"25eead1e-d17c-4b36-8a9f-ef563189d53b\") " Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.321570 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/25eead1e-d17c-4b36-8a9f-ef563189d53b-internal-tls-certs\") pod \"25eead1e-d17c-4b36-8a9f-ef563189d53b\" (UID: \"25eead1e-d17c-4b36-8a9f-ef563189d53b\") " Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.321648 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/022746a5-1c20-4729-8688-5903831f57ed-logs\") pod \"022746a5-1c20-4729-8688-5903831f57ed\" (UID: \"022746a5-1c20-4729-8688-5903831f57ed\") " Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.321724 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/02edb9e9-3f42-4aea-9fe6-7efa332d5fb8-sg-core-conf-yaml\") pod \"02edb9e9-3f42-4aea-9fe6-7efa332d5fb8\" (UID: \"02edb9e9-3f42-4aea-9fe6-7efa332d5fb8\") " Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.321808 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ksl6m\" (UniqueName: \"kubernetes.io/projected/25eead1e-d17c-4b36-8a9f-ef563189d53b-kube-api-access-ksl6m\") pod \"25eead1e-d17c-4b36-8a9f-ef563189d53b\" (UID: \"25eead1e-d17c-4b36-8a9f-ef563189d53b\") " Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.321876 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ffrfx\" (UniqueName: \"kubernetes.io/projected/2adad00d-01e3-4441-a3fc-87d162a5e309-kube-api-access-ffrfx\") pod \"2adad00d-01e3-4441-a3fc-87d162a5e309\" (UID: \"2adad00d-01e3-4441-a3fc-87d162a5e309\") " Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.325315 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2adad00d-01e3-4441-a3fc-87d162a5e309-kube-api-access-ffrfx" (OuterVolumeSpecName: "kube-api-access-ffrfx") pod "2adad00d-01e3-4441-a3fc-87d162a5e309" (UID: "2adad00d-01e3-4441-a3fc-87d162a5e309"). InnerVolumeSpecName "kube-api-access-ffrfx". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.326336 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.328009 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/25eead1e-d17c-4b36-8a9f-ef563189d53b-logs" (OuterVolumeSpecName: "logs") pod "25eead1e-d17c-4b36-8a9f-ef563189d53b" (UID: "25eead1e-d17c-4b36-8a9f-ef563189d53b"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.334295 5052 scope.go:117] "RemoveContainer" containerID="bab03b01c66d5661abe6c60c8c81496efcefb2ea684c5751d1ee1113198a69ca" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.336381 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-api-0"] Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.339599 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/022746a5-1c20-4729-8688-5903831f57ed-logs" (OuterVolumeSpecName: "logs") pod "022746a5-1c20-4729-8688-5903831f57ed" (UID: "022746a5-1c20-4729-8688-5903831f57ed"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.340743 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_ovn-northd-0_3b93bc75-db87-4677-be2f-910d07bb696c/ovn-northd/0.log" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.340816 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.343073 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/memcached-0"] Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.344492 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/02edb9e9-3f42-4aea-9fe6-7efa332d5fb8-run-httpd" (OuterVolumeSpecName: "run-httpd") pod "02edb9e9-3f42-4aea-9fe6-7efa332d5fb8" (UID: "02edb9e9-3f42-4aea-9fe6-7efa332d5fb8"). InnerVolumeSpecName "run-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.346964 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0191d138-7aff-42c2-88d2-fbbaf1722bfd-credential-keys" (OuterVolumeSpecName: "credential-keys") pod "0191d138-7aff-42c2-88d2-fbbaf1722bfd" (UID: "0191d138-7aff-42c2-88d2-fbbaf1722bfd"). InnerVolumeSpecName "credential-keys". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.347044 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/02edb9e9-3f42-4aea-9fe6-7efa332d5fb8-log-httpd" (OuterVolumeSpecName: "log-httpd") pod "02edb9e9-3f42-4aea-9fe6-7efa332d5fb8" (UID: "02edb9e9-3f42-4aea-9fe6-7efa332d5fb8"). InnerVolumeSpecName "log-httpd". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.347174 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/022746a5-1c20-4729-8688-5903831f57ed-kube-api-access-xf64v" (OuterVolumeSpecName: "kube-api-access-xf64v") pod "022746a5-1c20-4729-8688-5903831f57ed" (UID: "022746a5-1c20-4729-8688-5903831f57ed"). InnerVolumeSpecName "kube-api-access-xf64v". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.355028 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron6c26-account-delete-d7wtw" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.355377 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0191d138-7aff-42c2-88d2-fbbaf1722bfd-kube-api-access-xqxxd" (OuterVolumeSpecName: "kube-api-access-xqxxd") pod "0191d138-7aff-42c2-88d2-fbbaf1722bfd" (UID: "0191d138-7aff-42c2-88d2-fbbaf1722bfd"). InnerVolumeSpecName "kube-api-access-xqxxd". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.356930 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/25eead1e-d17c-4b36-8a9f-ef563189d53b-config-data-custom" (OuterVolumeSpecName: "config-data-custom") pod "25eead1e-d17c-4b36-8a9f-ef563189d53b" (UID: "25eead1e-d17c-4b36-8a9f-ef563189d53b"). InnerVolumeSpecName "config-data-custom". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.359547 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/memcached-0"] Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.361564 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0191d138-7aff-42c2-88d2-fbbaf1722bfd-fernet-keys" (OuterVolumeSpecName: "fernet-keys") pod "0191d138-7aff-42c2-88d2-fbbaf1722bfd" (UID: "0191d138-7aff-42c2-88d2-fbbaf1722bfd"). InnerVolumeSpecName "fernet-keys". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.362698 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/novaapi7066-account-delete-whswm" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.363128 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/02edb9e9-3f42-4aea-9fe6-7efa332d5fb8-scripts" (OuterVolumeSpecName: "scripts") pod "02edb9e9-3f42-4aea-9fe6-7efa332d5fb8" (UID: "02edb9e9-3f42-4aea-9fe6-7efa332d5fb8"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.363235 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0191d138-7aff-42c2-88d2-fbbaf1722bfd-scripts" (OuterVolumeSpecName: "scripts") pod "0191d138-7aff-42c2-88d2-fbbaf1722bfd" (UID: "0191d138-7aff-42c2-88d2-fbbaf1722bfd"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.371930 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/02edb9e9-3f42-4aea-9fe6-7efa332d5fb8-kube-api-access-rtlhb" (OuterVolumeSpecName: "kube-api-access-rtlhb") pod "02edb9e9-3f42-4aea-9fe6-7efa332d5fb8" (UID: "02edb9e9-3f42-4aea-9fe6-7efa332d5fb8"). InnerVolumeSpecName "kube-api-access-rtlhb". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.373718 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/25eead1e-d17c-4b36-8a9f-ef563189d53b-kube-api-access-ksl6m" (OuterVolumeSpecName: "kube-api-access-ksl6m") pod "25eead1e-d17c-4b36-8a9f-ef563189d53b" (UID: "25eead1e-d17c-4b36-8a9f-ef563189d53b"). InnerVolumeSpecName "kube-api-access-ksl6m". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.402013 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-698d8977cd-ppz9n" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.406101 5052 scope.go:117] "RemoveContainer" containerID="27212e7a0c6918bbf263320719b8b0731ba8a5d005b2aed71d83eab909ffb6f0" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.406611 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.407509 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.407526 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-cell1-server-0" event={"ID":"568addd3-4ad3-47c3-adae-693691e7cd40","Type":"ContainerDied","Data":"b1fadf1ac3bfe95435852744a04e0c29a811c11270cfcc0a4f8a5ac0ba88ad99"} Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.412772 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/nova-cell1-novncproxy-0" podUID="43b07f3c-f9d7-451b-b0ab-61b293cff35d" containerName="nova-cell1-novncproxy-novncproxy" probeResult="failure" output="Get \"https://10.217.0.211:6080/vnc_lite.html\": dial tcp 10.217.0.211:6080: i/o timeout" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.424204 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovn-rundir\" (UniqueName: \"kubernetes.io/empty-dir/3b93bc75-db87-4677-be2f-910d07bb696c-ovn-rundir\") pod \"3b93bc75-db87-4677-be2f-910d07bb696c\" (UID: \"3b93bc75-db87-4677-be2f-910d07bb696c\") " Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.424383 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/c4d5c183-8df4-4c43-9afb-2b28c6aae9e1-operator-scripts\") pod \"c4d5c183-8df4-4c43-9afb-2b28c6aae9e1\" (UID: \"c4d5c183-8df4-4c43-9afb-2b28c6aae9e1\") " Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.424475 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/568addd3-4ad3-47c3-adae-693691e7cd40-config-data\") pod \"568addd3-4ad3-47c3-adae-693691e7cd40\" (UID: \"568addd3-4ad3-47c3-adae-693691e7cd40\") " Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.424583 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"plugins-conf\" (UniqueName: \"kubernetes.io/configmap/568addd3-4ad3-47c3-adae-693691e7cd40-plugins-conf\") pod \"568addd3-4ad3-47c3-adae-693691e7cd40\" (UID: \"568addd3-4ad3-47c3-adae-693691e7cd40\") " Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.424686 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jv2g8\" (UniqueName: \"kubernetes.io/projected/e8a26759-2926-41cf-a1ec-286fc9962bb3-kube-api-access-jv2g8\") pod \"e8a26759-2926-41cf-a1ec-286fc9962bb3\" (UID: \"e8a26759-2926-41cf-a1ec-286fc9962bb3\") " Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.424762 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9hxs4\" (UniqueName: \"kubernetes.io/projected/568addd3-4ad3-47c3-adae-693691e7cd40-kube-api-access-9hxs4\") pod \"568addd3-4ad3-47c3-adae-693691e7cd40\" (UID: \"568addd3-4ad3-47c3-adae-693691e7cd40\") " Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.424839 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-68vcw\" (UniqueName: \"kubernetes.io/projected/c4d5c183-8df4-4c43-9afb-2b28c6aae9e1-kube-api-access-68vcw\") pod \"c4d5c183-8df4-4c43-9afb-2b28c6aae9e1\" (UID: \"c4d5c183-8df4-4c43-9afb-2b28c6aae9e1\") " Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.425223 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"plugins-conf\" (UniqueName: \"kubernetes.io/configmap/e73813f5-f97f-41dd-8a79-6196557d4d1a-plugins-conf\") pod \"e73813f5-f97f-41dd-8a79-6196557d4d1a\" (UID: \"e73813f5-f97f-41dd-8a79-6196557d4d1a\") " Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.425308 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"rabbitmq-erlang-cookie\" (UniqueName: \"kubernetes.io/empty-dir/568addd3-4ad3-47c3-adae-693691e7cd40-rabbitmq-erlang-cookie\") pod \"568addd3-4ad3-47c3-adae-693691e7cd40\" (UID: \"568addd3-4ad3-47c3-adae-693691e7cd40\") " Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.425383 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/b3363d6f-656e-4e22-a3eb-b1016380a0c8-kolla-config\") pod \"b3363d6f-656e-4e22-a3eb-b1016380a0c8\" (UID: \"b3363d6f-656e-4e22-a3eb-b1016380a0c8\") " Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.425432 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b93bc75-db87-4677-be2f-910d07bb696c-ovn-rundir" (OuterVolumeSpecName: "ovn-rundir") pod "3b93bc75-db87-4677-be2f-910d07bb696c" (UID: "3b93bc75-db87-4677-be2f-910d07bb696c"). InnerVolumeSpecName "ovn-rundir". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.425555 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b3363d6f-656e-4e22-a3eb-b1016380a0c8-combined-ca-bundle\") pod \"b3363d6f-656e-4e22-a3eb-b1016380a0c8\" (UID: \"b3363d6f-656e-4e22-a3eb-b1016380a0c8\") " Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.425656 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3b93bc75-db87-4677-be2f-910d07bb696c-config\") pod \"3b93bc75-db87-4677-be2f-910d07bb696c\" (UID: \"3b93bc75-db87-4677-be2f-910d07bb696c\") " Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.425758 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"rabbitmq-tls\" (UniqueName: \"kubernetes.io/projected/e73813f5-f97f-41dd-8a79-6196557d4d1a-rabbitmq-tls\") pod \"e73813f5-f97f-41dd-8a79-6196557d4d1a\" (UID: \"e73813f5-f97f-41dd-8a79-6196557d4d1a\") " Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.425836 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"persistence\" (UniqueName: \"kubernetes.io/local-volume/local-storage03-crc\") pod \"e73813f5-f97f-41dd-8a79-6196557d4d1a\" (UID: \"e73813f5-f97f-41dd-8a79-6196557d4d1a\") " Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.425904 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"rabbitmq-plugins\" (UniqueName: \"kubernetes.io/empty-dir/568addd3-4ad3-47c3-adae-693691e7cd40-rabbitmq-plugins\") pod \"568addd3-4ad3-47c3-adae-693691e7cd40\" (UID: \"568addd3-4ad3-47c3-adae-693691e7cd40\") " Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.425944 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c4d5c183-8df4-4c43-9afb-2b28c6aae9e1-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "c4d5c183-8df4-4c43-9afb-2b28c6aae9e1" (UID: "c4d5c183-8df4-4c43-9afb-2b28c6aae9e1"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.426077 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"rabbitmq-confd\" (UniqueName: \"kubernetes.io/projected/e73813f5-f97f-41dd-8a79-6196557d4d1a-rabbitmq-confd\") pod \"e73813f5-f97f-41dd-8a79-6196557d4d1a\" (UID: \"e73813f5-f97f-41dd-8a79-6196557d4d1a\") " Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.426147 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/e8a26759-2926-41cf-a1ec-286fc9962bb3-operator-scripts\") pod \"e8a26759-2926-41cf-a1ec-286fc9962bb3\" (UID: \"e8a26759-2926-41cf-a1ec-286fc9962bb3\") " Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.426232 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bl5h2\" (UniqueName: \"kubernetes.io/projected/b3363d6f-656e-4e22-a3eb-b1016380a0c8-kube-api-access-bl5h2\") pod \"b3363d6f-656e-4e22-a3eb-b1016380a0c8\" (UID: \"b3363d6f-656e-4e22-a3eb-b1016380a0c8\") " Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.426353 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/b3363d6f-656e-4e22-a3eb-b1016380a0c8-operator-scripts\") pod \"b3363d6f-656e-4e22-a3eb-b1016380a0c8\" (UID: \"b3363d6f-656e-4e22-a3eb-b1016380a0c8\") " Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.426436 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/d9bf91f1-099c-4e72-97cf-1828241aeaac-operator-scripts\") pod \"d9bf91f1-099c-4e72-97cf-1828241aeaac\" (UID: \"d9bf91f1-099c-4e72-97cf-1828241aeaac\") " Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.426499 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"rabbitmq-confd\" (UniqueName: \"kubernetes.io/projected/568addd3-4ad3-47c3-adae-693691e7cd40-rabbitmq-confd\") pod \"568addd3-4ad3-47c3-adae-693691e7cd40\" (UID: \"568addd3-4ad3-47c3-adae-693691e7cd40\") " Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.426673 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"server-conf\" (UniqueName: \"kubernetes.io/configmap/568addd3-4ad3-47c3-adae-693691e7cd40-server-conf\") pod \"568addd3-4ad3-47c3-adae-693691e7cd40\" (UID: \"568addd3-4ad3-47c3-adae-693691e7cd40\") " Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.426763 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/3b93bc75-db87-4677-be2f-910d07bb696c-scripts\") pod \"3b93bc75-db87-4677-be2f-910d07bb696c\" (UID: \"3b93bc75-db87-4677-be2f-910d07bb696c\") " Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.426849 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"rabbitmq-tls\" (UniqueName: \"kubernetes.io/projected/568addd3-4ad3-47c3-adae-693691e7cd40-rabbitmq-tls\") pod \"568addd3-4ad3-47c3-adae-693691e7cd40\" (UID: \"568addd3-4ad3-47c3-adae-693691e7cd40\") " Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.426923 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"rabbitmq-plugins\" (UniqueName: \"kubernetes.io/empty-dir/e73813f5-f97f-41dd-8a79-6196557d4d1a-rabbitmq-plugins\") pod \"e73813f5-f97f-41dd-8a79-6196557d4d1a\" (UID: \"e73813f5-f97f-41dd-8a79-6196557d4d1a\") " Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.426985 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2wx2s\" (UniqueName: \"kubernetes.io/projected/3b93bc75-db87-4677-be2f-910d07bb696c-kube-api-access-2wx2s\") pod \"3b93bc75-db87-4677-be2f-910d07bb696c\" (UID: \"3b93bc75-db87-4677-be2f-910d07bb696c\") " Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.427050 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/b3363d6f-656e-4e22-a3eb-b1016380a0c8-galera-tls-certs\") pod \"b3363d6f-656e-4e22-a3eb-b1016380a0c8\" (UID: \"b3363d6f-656e-4e22-a3eb-b1016380a0c8\") " Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.427120 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/b3363d6f-656e-4e22-a3eb-b1016380a0c8-config-data-default\") pod \"b3363d6f-656e-4e22-a3eb-b1016380a0c8\" (UID: \"b3363d6f-656e-4e22-a3eb-b1016380a0c8\") " Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.427208 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"pod-info\" (UniqueName: \"kubernetes.io/downward-api/e73813f5-f97f-41dd-8a79-6196557d4d1a-pod-info\") pod \"e73813f5-f97f-41dd-8a79-6196557d4d1a\" (UID: \"e73813f5-f97f-41dd-8a79-6196557d4d1a\") " Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.427278 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"persistence\" (UniqueName: \"kubernetes.io/local-volume/local-storage02-crc\") pod \"568addd3-4ad3-47c3-adae-693691e7cd40\" (UID: \"568addd3-4ad3-47c3-adae-693691e7cd40\") " Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.427407 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/79efd225-5ce4-428f-aef9-3f4884931fef-operator-scripts\") pod \"79efd225-5ce4-428f-aef9-3f4884931fef\" (UID: \"79efd225-5ce4-428f-aef9-3f4884931fef\") " Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.427484 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3b93bc75-db87-4677-be2f-910d07bb696c-combined-ca-bundle\") pod \"3b93bc75-db87-4677-be2f-910d07bb696c\" (UID: \"3b93bc75-db87-4677-be2f-910d07bb696c\") " Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.427642 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5b8c9\" (UniqueName: \"kubernetes.io/projected/e73813f5-f97f-41dd-8a79-6196557d4d1a-kube-api-access-5b8c9\") pod \"e73813f5-f97f-41dd-8a79-6196557d4d1a\" (UID: \"e73813f5-f97f-41dd-8a79-6196557d4d1a\") " Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.427713 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"mysql-db\" (UniqueName: \"kubernetes.io/local-volume/local-storage09-crc\") pod \"b3363d6f-656e-4e22-a3eb-b1016380a0c8\" (UID: \"b3363d6f-656e-4e22-a3eb-b1016380a0c8\") " Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.427773 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"server-conf\" (UniqueName: \"kubernetes.io/configmap/e73813f5-f97f-41dd-8a79-6196557d4d1a-server-conf\") pod \"e73813f5-f97f-41dd-8a79-6196557d4d1a\" (UID: \"e73813f5-f97f-41dd-8a79-6196557d4d1a\") " Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.427836 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/b3363d6f-656e-4e22-a3eb-b1016380a0c8-config-data-generated\") pod \"b3363d6f-656e-4e22-a3eb-b1016380a0c8\" (UID: \"b3363d6f-656e-4e22-a3eb-b1016380a0c8\") " Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.427907 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/3b93bc75-db87-4677-be2f-910d07bb696c-metrics-certs-tls-certs\") pod \"3b93bc75-db87-4677-be2f-910d07bb696c\" (UID: \"3b93bc75-db87-4677-be2f-910d07bb696c\") " Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.427997 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-spfp6\" (UniqueName: \"kubernetes.io/projected/79efd225-5ce4-428f-aef9-3f4884931fef-kube-api-access-spfp6\") pod \"79efd225-5ce4-428f-aef9-3f4884931fef\" (UID: \"79efd225-5ce4-428f-aef9-3f4884931fef\") " Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.428084 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/e73813f5-f97f-41dd-8a79-6196557d4d1a-config-data\") pod \"e73813f5-f97f-41dd-8a79-6196557d4d1a\" (UID: \"e73813f5-f97f-41dd-8a79-6196557d4d1a\") " Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.428162 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"erlang-cookie-secret\" (UniqueName: \"kubernetes.io/secret/568addd3-4ad3-47c3-adae-693691e7cd40-erlang-cookie-secret\") pod \"568addd3-4ad3-47c3-adae-693691e7cd40\" (UID: \"568addd3-4ad3-47c3-adae-693691e7cd40\") " Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.428234 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovn-northd-tls-certs\" (UniqueName: \"kubernetes.io/secret/3b93bc75-db87-4677-be2f-910d07bb696c-ovn-northd-tls-certs\") pod \"3b93bc75-db87-4677-be2f-910d07bb696c\" (UID: \"3b93bc75-db87-4677-be2f-910d07bb696c\") " Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.428350 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"erlang-cookie-secret\" (UniqueName: \"kubernetes.io/secret/e73813f5-f97f-41dd-8a79-6196557d4d1a-erlang-cookie-secret\") pod \"e73813f5-f97f-41dd-8a79-6196557d4d1a\" (UID: \"e73813f5-f97f-41dd-8a79-6196557d4d1a\") " Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.428455 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"pod-info\" (UniqueName: \"kubernetes.io/downward-api/568addd3-4ad3-47c3-adae-693691e7cd40-pod-info\") pod \"568addd3-4ad3-47c3-adae-693691e7cd40\" (UID: \"568addd3-4ad3-47c3-adae-693691e7cd40\") " Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.429262 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/568addd3-4ad3-47c3-adae-693691e7cd40-rabbitmq-erlang-cookie" (OuterVolumeSpecName: "rabbitmq-erlang-cookie") pod "568addd3-4ad3-47c3-adae-693691e7cd40" (UID: "568addd3-4ad3-47c3-adae-693691e7cd40"). InnerVolumeSpecName "rabbitmq-erlang-cookie". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.429305 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b3363d6f-656e-4e22-a3eb-b1016380a0c8-kolla-config" (OuterVolumeSpecName: "kolla-config") pod "b3363d6f-656e-4e22-a3eb-b1016380a0c8" (UID: "b3363d6f-656e-4e22-a3eb-b1016380a0c8"). InnerVolumeSpecName "kolla-config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.429838 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e73813f5-f97f-41dd-8a79-6196557d4d1a-plugins-conf" (OuterVolumeSpecName: "plugins-conf") pod "e73813f5-f97f-41dd-8a79-6196557d4d1a" (UID: "e73813f5-f97f-41dd-8a79-6196557d4d1a"). InnerVolumeSpecName "plugins-conf". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.429887 5052 generic.go:334] "Generic (PLEG): container finished" podID="1b7ef5eb-406e-4e2c-9d9c-d763919f56fd" containerID="c9f66104fbca885ffe17ff3f4d917326325460800a0171775b450385bf7d5784" exitCode=1 Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.429959 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/novacell02629-account-delete-wsshh" event={"ID":"1b7ef5eb-406e-4e2c-9d9c-d763919f56fd","Type":"ContainerDied","Data":"c9f66104fbca885ffe17ff3f4d917326325460800a0171775b450385bf7d5784"} Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.442147 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/568addd3-4ad3-47c3-adae-693691e7cd40-plugins-conf" (OuterVolumeSpecName: "plugins-conf") pod "568addd3-4ad3-47c3-adae-693691e7cd40" (UID: "568addd3-4ad3-47c3-adae-693691e7cd40"). InnerVolumeSpecName "plugins-conf". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.445473 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3b93bc75-db87-4677-be2f-910d07bb696c-config" (OuterVolumeSpecName: "config") pod "3b93bc75-db87-4677-be2f-910d07bb696c" (UID: "3b93bc75-db87-4677-be2f-910d07bb696c"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.445605 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-grf5h\" (UniqueName: \"kubernetes.io/projected/d9bf91f1-099c-4e72-97cf-1828241aeaac-kube-api-access-grf5h\") pod \"d9bf91f1-099c-4e72-97cf-1828241aeaac\" (UID: \"d9bf91f1-099c-4e72-97cf-1828241aeaac\") " Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.445682 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"rabbitmq-erlang-cookie\" (UniqueName: \"kubernetes.io/empty-dir/e73813f5-f97f-41dd-8a79-6196557d4d1a-rabbitmq-erlang-cookie\") pod \"e73813f5-f97f-41dd-8a79-6196557d4d1a\" (UID: \"e73813f5-f97f-41dd-8a79-6196557d4d1a\") " Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.446255 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/0191d138-7aff-42c2-88d2-fbbaf1722bfd-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.446272 5052 reconciler_common.go:293] "Volume detached for volume \"log-httpd\" (UniqueName: \"kubernetes.io/empty-dir/02edb9e9-3f42-4aea-9fe6-7efa332d5fb8-log-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.446285 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-xqxxd\" (UniqueName: \"kubernetes.io/projected/0191d138-7aff-42c2-88d2-fbbaf1722bfd-kube-api-access-xqxxd\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.446297 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/25eead1e-d17c-4b36-8a9f-ef563189d53b-config-data-custom\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.446309 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-rtlhb\" (UniqueName: \"kubernetes.io/projected/02edb9e9-3f42-4aea-9fe6-7efa332d5fb8-kube-api-access-rtlhb\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.446322 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-xf64v\" (UniqueName: \"kubernetes.io/projected/022746a5-1c20-4729-8688-5903831f57ed-kube-api-access-xf64v\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.446336 5052 reconciler_common.go:293] "Volume detached for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/0191d138-7aff-42c2-88d2-fbbaf1722bfd-fernet-keys\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.446347 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/25eead1e-d17c-4b36-8a9f-ef563189d53b-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.446359 5052 reconciler_common.go:293] "Volume detached for volume \"run-httpd\" (UniqueName: \"kubernetes.io/empty-dir/02edb9e9-3f42-4aea-9fe6-7efa332d5fb8-run-httpd\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.446374 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/02edb9e9-3f42-4aea-9fe6-7efa332d5fb8-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.446389 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/022746a5-1c20-4729-8688-5903831f57ed-logs\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.446403 5052 reconciler_common.go:293] "Volume detached for volume \"ovn-rundir\" (UniqueName: \"kubernetes.io/empty-dir/3b93bc75-db87-4677-be2f-910d07bb696c-ovn-rundir\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.446417 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-ksl6m\" (UniqueName: \"kubernetes.io/projected/25eead1e-d17c-4b36-8a9f-ef563189d53b-kube-api-access-ksl6m\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.446428 5052 reconciler_common.go:293] "Volume detached for volume \"plugins-conf\" (UniqueName: \"kubernetes.io/configmap/568addd3-4ad3-47c3-adae-693691e7cd40-plugins-conf\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.446441 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-ffrfx\" (UniqueName: \"kubernetes.io/projected/2adad00d-01e3-4441-a3fc-87d162a5e309-kube-api-access-ffrfx\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.446451 5052 reconciler_common.go:293] "Volume detached for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/0191d138-7aff-42c2-88d2-fbbaf1722bfd-credential-keys\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.446459 5052 reconciler_common.go:293] "Volume detached for volume \"plugins-conf\" (UniqueName: \"kubernetes.io/configmap/e73813f5-f97f-41dd-8a79-6196557d4d1a-plugins-conf\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.446468 5052 reconciler_common.go:293] "Volume detached for volume \"rabbitmq-erlang-cookie\" (UniqueName: \"kubernetes.io/empty-dir/568addd3-4ad3-47c3-adae-693691e7cd40-rabbitmq-erlang-cookie\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.446476 5052 reconciler_common.go:293] "Volume detached for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/b3363d6f-656e-4e22-a3eb-b1016380a0c8-kolla-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.446495 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3b93bc75-db87-4677-be2f-910d07bb696c-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.447306 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b3363d6f-656e-4e22-a3eb-b1016380a0c8-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "b3363d6f-656e-4e22-a3eb-b1016380a0c8" (UID: "b3363d6f-656e-4e22-a3eb-b1016380a0c8"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.453294 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e73813f5-f97f-41dd-8a79-6196557d4d1a-rabbitmq-plugins" (OuterVolumeSpecName: "rabbitmq-plugins") pod "e73813f5-f97f-41dd-8a79-6196557d4d1a" (UID: "e73813f5-f97f-41dd-8a79-6196557d4d1a"). InnerVolumeSpecName "rabbitmq-plugins". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.454467 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e73813f5-f97f-41dd-8a79-6196557d4d1a-rabbitmq-erlang-cookie" (OuterVolumeSpecName: "rabbitmq-erlang-cookie") pod "e73813f5-f97f-41dd-8a79-6196557d4d1a" (UID: "e73813f5-f97f-41dd-8a79-6196557d4d1a"). InnerVolumeSpecName "rabbitmq-erlang-cookie". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.459080 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3b93bc75-db87-4677-be2f-910d07bb696c-scripts" (OuterVolumeSpecName: "scripts") pod "3b93bc75-db87-4677-be2f-910d07bb696c" (UID: "3b93bc75-db87-4677-be2f-910d07bb696c"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.459066 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b3363d6f-656e-4e22-a3eb-b1016380a0c8-config-data-generated" (OuterVolumeSpecName: "config-data-generated") pod "b3363d6f-656e-4e22-a3eb-b1016380a0c8" (UID: "b3363d6f-656e-4e22-a3eb-b1016380a0c8"). InnerVolumeSpecName "config-data-generated". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.459571 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d9bf91f1-099c-4e72-97cf-1828241aeaac-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "d9bf91f1-099c-4e72-97cf-1828241aeaac" (UID: "d9bf91f1-099c-4e72-97cf-1828241aeaac"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.464553 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b3363d6f-656e-4e22-a3eb-b1016380a0c8-config-data-default" (OuterVolumeSpecName: "config-data-default") pod "b3363d6f-656e-4e22-a3eb-b1016380a0c8" (UID: "b3363d6f-656e-4e22-a3eb-b1016380a0c8"). InnerVolumeSpecName "config-data-default". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.466184 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e8a26759-2926-41cf-a1ec-286fc9962bb3-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "e8a26759-2926-41cf-a1ec-286fc9962bb3" (UID: "e8a26759-2926-41cf-a1ec-286fc9962bb3"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.471022 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/79efd225-5ce4-428f-aef9-3f4884931fef-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "79efd225-5ce4-428f-aef9-3f4884931fef" (UID: "79efd225-5ce4-428f-aef9-3f4884931fef"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.482851 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/568addd3-4ad3-47c3-adae-693691e7cd40-rabbitmq-plugins" (OuterVolumeSpecName: "rabbitmq-plugins") pod "568addd3-4ad3-47c3-adae-693691e7cd40" (UID: "568addd3-4ad3-47c3-adae-693691e7cd40"). InnerVolumeSpecName "rabbitmq-plugins". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.496018 5052 generic.go:334] "Generic (PLEG): container finished" podID="13e7e748-92f4-4255-a8eb-aa5a6faf361f" containerID="9c5fb38f25954ef5355f2862f8e1ab7d9a859ef546a60f10dfb32dbd818c660c" exitCode=1 Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.496129 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/novaapi7066-account-delete-whswm" event={"ID":"13e7e748-92f4-4255-a8eb-aa5a6faf361f","Type":"ContainerDied","Data":"9c5fb38f25954ef5355f2862f8e1ab7d9a859ef546a60f10dfb32dbd818c660c"} Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.496161 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/novaapi7066-account-delete-whswm" event={"ID":"13e7e748-92f4-4255-a8eb-aa5a6faf361f","Type":"ContainerDied","Data":"2323845f988980cd21781bad44c86e11b8a5edcb5a8f3351715e8e37eaba09a0"} Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.496240 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/novaapi7066-account-delete-whswm" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.548061 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gwqrq\" (UniqueName: \"kubernetes.io/projected/941d063f-c834-47a1-bc0c-2096b005e09c-kube-api-access-gwqrq\") pod \"941d063f-c834-47a1-bc0c-2096b005e09c\" (UID: \"941d063f-c834-47a1-bc0c-2096b005e09c\") " Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.548221 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/e2225738-b0d1-4306-bbee-a61dd9558c80-ovndb-tls-certs\") pod \"e2225738-b0d1-4306-bbee-a61dd9558c80\" (UID: \"e2225738-b0d1-4306-bbee-a61dd9558c80\") " Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.548272 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/e2225738-b0d1-4306-bbee-a61dd9558c80-internal-tls-certs\") pod \"e2225738-b0d1-4306-bbee-a61dd9558c80\" (UID: \"e2225738-b0d1-4306-bbee-a61dd9558c80\") " Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.548425 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/941d063f-c834-47a1-bc0c-2096b005e09c-operator-scripts\") pod \"941d063f-c834-47a1-bc0c-2096b005e09c\" (UID: \"941d063f-c834-47a1-bc0c-2096b005e09c\") " Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.548449 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/13e7e748-92f4-4255-a8eb-aa5a6faf361f-operator-scripts\") pod \"13e7e748-92f4-4255-a8eb-aa5a6faf361f\" (UID: \"13e7e748-92f4-4255-a8eb-aa5a6faf361f\") " Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.548471 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/e2225738-b0d1-4306-bbee-a61dd9558c80-public-tls-certs\") pod \"e2225738-b0d1-4306-bbee-a61dd9558c80\" (UID: \"e2225738-b0d1-4306-bbee-a61dd9558c80\") " Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.548579 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/4ec49e76-2597-47d6-811b-f65cbe0580ff-config-data\") pod \"4ec49e76-2597-47d6-811b-f65cbe0580ff\" (UID: \"4ec49e76-2597-47d6-811b-f65cbe0580ff\") " Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.548603 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hnfq6\" (UniqueName: \"kubernetes.io/projected/4ec49e76-2597-47d6-811b-f65cbe0580ff-kube-api-access-hnfq6\") pod \"4ec49e76-2597-47d6-811b-f65cbe0580ff\" (UID: \"4ec49e76-2597-47d6-811b-f65cbe0580ff\") " Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.548621 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wj7x6\" (UniqueName: \"kubernetes.io/projected/e2225738-b0d1-4306-bbee-a61dd9558c80-kube-api-access-wj7x6\") pod \"e2225738-b0d1-4306-bbee-a61dd9558c80\" (UID: \"e2225738-b0d1-4306-bbee-a61dd9558c80\") " Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.548637 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/e2225738-b0d1-4306-bbee-a61dd9558c80-config\") pod \"e2225738-b0d1-4306-bbee-a61dd9558c80\" (UID: \"e2225738-b0d1-4306-bbee-a61dd9558c80\") " Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.548717 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5vk45\" (UniqueName: \"kubernetes.io/projected/13e7e748-92f4-4255-a8eb-aa5a6faf361f-kube-api-access-5vk45\") pod \"13e7e748-92f4-4255-a8eb-aa5a6faf361f\" (UID: \"13e7e748-92f4-4255-a8eb-aa5a6faf361f\") " Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.548739 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/e2225738-b0d1-4306-bbee-a61dd9558c80-httpd-config\") pod \"e2225738-b0d1-4306-bbee-a61dd9558c80\" (UID: \"e2225738-b0d1-4306-bbee-a61dd9558c80\") " Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.548789 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4ec49e76-2597-47d6-811b-f65cbe0580ff-combined-ca-bundle\") pod \"4ec49e76-2597-47d6-811b-f65cbe0580ff\" (UID: \"4ec49e76-2597-47d6-811b-f65cbe0580ff\") " Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.548847 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e2225738-b0d1-4306-bbee-a61dd9558c80-combined-ca-bundle\") pod \"e2225738-b0d1-4306-bbee-a61dd9558c80\" (UID: \"e2225738-b0d1-4306-bbee-a61dd9558c80\") " Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.549201 5052 reconciler_common.go:293] "Volume detached for volume \"rabbitmq-erlang-cookie\" (UniqueName: \"kubernetes.io/empty-dir/e73813f5-f97f-41dd-8a79-6196557d4d1a-rabbitmq-erlang-cookie\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.549215 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/c4d5c183-8df4-4c43-9afb-2b28c6aae9e1-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.549226 5052 reconciler_common.go:293] "Volume detached for volume \"rabbitmq-plugins\" (UniqueName: \"kubernetes.io/empty-dir/568addd3-4ad3-47c3-adae-693691e7cd40-rabbitmq-plugins\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.549240 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/e8a26759-2926-41cf-a1ec-286fc9962bb3-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.549251 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/b3363d6f-656e-4e22-a3eb-b1016380a0c8-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.549262 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/d9bf91f1-099c-4e72-97cf-1828241aeaac-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.549271 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/3b93bc75-db87-4677-be2f-910d07bb696c-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.549280 5052 reconciler_common.go:293] "Volume detached for volume \"rabbitmq-plugins\" (UniqueName: \"kubernetes.io/empty-dir/e73813f5-f97f-41dd-8a79-6196557d4d1a-rabbitmq-plugins\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.549288 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/b3363d6f-656e-4e22-a3eb-b1016380a0c8-config-data-default\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.549300 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/79efd225-5ce4-428f-aef9-3f4884931fef-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.549311 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/b3363d6f-656e-4e22-a3eb-b1016380a0c8-config-data-generated\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.551887 5052 generic.go:334] "Generic (PLEG): container finished" podID="4ec49e76-2597-47d6-811b-f65cbe0580ff" containerID="ec5a6e0000f85ac80b044c5e702c36b00720989c7bd27aac0130a6739a2d1bd8" exitCode=0 Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.552023 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-scheduler-0" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.556112 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/13e7e748-92f4-4255-a8eb-aa5a6faf361f-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "13e7e748-92f4-4255-a8eb-aa5a6faf361f" (UID: "13e7e748-92f4-4255-a8eb-aa5a6faf361f"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.556597 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/941d063f-c834-47a1-bc0c-2096b005e09c-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "941d063f-c834-47a1-bc0c-2096b005e09c" (UID: "941d063f-c834-47a1-bc0c-2096b005e09c"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.567019 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_ovn-northd-0_3b93bc75-db87-4677-be2f-910d07bb696c/ovn-northd/0.log" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.567157 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovn-northd-0" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.567958 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"4ec49e76-2597-47d6-811b-f65cbe0580ff","Type":"ContainerDied","Data":"ec5a6e0000f85ac80b044c5e702c36b00720989c7bd27aac0130a6739a2d1bd8"} Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.568026 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-scheduler-0" event={"ID":"4ec49e76-2597-47d6-811b-f65cbe0580ff","Type":"ContainerDied","Data":"dbda807a085f0dde69e7b6f3a0f75c39b7d48940e339baa86060bf949e62e21c"} Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.568047 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovn-northd-0" event={"ID":"3b93bc75-db87-4677-be2f-910d07bb696c","Type":"ContainerDied","Data":"b0ffc7b4f3e19ca93256e9a6d7d13686986ce1b654547d9489545b25ee4fe92a"} Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.625283 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c4d5c183-8df4-4c43-9afb-2b28c6aae9e1-kube-api-access-68vcw" (OuterVolumeSpecName: "kube-api-access-68vcw") pod "c4d5c183-8df4-4c43-9afb-2b28c6aae9e1" (UID: "c4d5c183-8df4-4c43-9afb-2b28c6aae9e1"). InnerVolumeSpecName "kube-api-access-68vcw". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.625435 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3b93bc75-db87-4677-be2f-910d07bb696c-kube-api-access-2wx2s" (OuterVolumeSpecName: "kube-api-access-2wx2s") pod "3b93bc75-db87-4677-be2f-910d07bb696c" (UID: "3b93bc75-db87-4677-be2f-910d07bb696c"). InnerVolumeSpecName "kube-api-access-2wx2s". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.625551 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/568addd3-4ad3-47c3-adae-693691e7cd40-kube-api-access-9hxs4" (OuterVolumeSpecName: "kube-api-access-9hxs4") pod "568addd3-4ad3-47c3-adae-693691e7cd40" (UID: "568addd3-4ad3-47c3-adae-693691e7cd40"). InnerVolumeSpecName "kube-api-access-9hxs4". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.625635 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e8a26759-2926-41cf-a1ec-286fc9962bb3-kube-api-access-jv2g8" (OuterVolumeSpecName: "kube-api-access-jv2g8") pod "e8a26759-2926-41cf-a1ec-286fc9962bb3" (UID: "e8a26759-2926-41cf-a1ec-286fc9962bb3"). InnerVolumeSpecName "kube-api-access-jv2g8". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.630779 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e2225738-b0d1-4306-bbee-a61dd9558c80-kube-api-access-wj7x6" (OuterVolumeSpecName: "kube-api-access-wj7x6") pod "e2225738-b0d1-4306-bbee-a61dd9558c80" (UID: "e2225738-b0d1-4306-bbee-a61dd9558c80"). InnerVolumeSpecName "kube-api-access-wj7x6". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.630924 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b3363d6f-656e-4e22-a3eb-b1016380a0c8-kube-api-access-bl5h2" (OuterVolumeSpecName: "kube-api-access-bl5h2") pod "b3363d6f-656e-4e22-a3eb-b1016380a0c8" (UID: "b3363d6f-656e-4e22-a3eb-b1016380a0c8"). InnerVolumeSpecName "kube-api-access-bl5h2". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.636227 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/memcached-0" podUID="d1ff5100-3ac7-4633-811c-ea470242d858" containerName="memcached" probeResult="failure" output="dial tcp 10.217.0.141:11211: i/o timeout" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.650637 5052 scope.go:117] "RemoveContainer" containerID="92269ae03a0728af8ad79f1a3cc73e6323d763c9c370a177f9ea91e0fc11bf91" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.660407 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/rabbitmq-server-0" event={"ID":"e73813f5-f97f-41dd-8a79-6196557d4d1a","Type":"ContainerDied","Data":"6bc8c20d005acb0e60c7ac12f491adff39620c48a24066fc45557f898f3a9e66"} Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.661775 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/rabbitmq-server-0" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.669096 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-bl5h2\" (UniqueName: \"kubernetes.io/projected/b3363d6f-656e-4e22-a3eb-b1016380a0c8-kube-api-access-bl5h2\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.671047 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/941d063f-c834-47a1-bc0c-2096b005e09c-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.671088 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/13e7e748-92f4-4255-a8eb-aa5a6faf361f-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.671104 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-2wx2s\" (UniqueName: \"kubernetes.io/projected/3b93bc75-db87-4677-be2f-910d07bb696c-kube-api-access-2wx2s\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.671124 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-wj7x6\" (UniqueName: \"kubernetes.io/projected/e2225738-b0d1-4306-bbee-a61dd9558c80-kube-api-access-wj7x6\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.671141 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-jv2g8\" (UniqueName: \"kubernetes.io/projected/e8a26759-2926-41cf-a1ec-286fc9962bb3-kube-api-access-jv2g8\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.671159 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-9hxs4\" (UniqueName: \"kubernetes.io/projected/568addd3-4ad3-47c3-adae-693691e7cd40-kube-api-access-9hxs4\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.671172 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-68vcw\" (UniqueName: \"kubernetes.io/projected/c4d5c183-8df4-4c43-9afb-2b28c6aae9e1-kube-api-access-68vcw\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.681687 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/568addd3-4ad3-47c3-adae-693691e7cd40-pod-info" (OuterVolumeSpecName: "pod-info") pod "568addd3-4ad3-47c3-adae-693691e7cd40" (UID: "568addd3-4ad3-47c3-adae-693691e7cd40"). InnerVolumeSpecName "pod-info". PluginName "kubernetes.io/downward-api", VolumeGidValue "" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.682818 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e73813f5-f97f-41dd-8a79-6196557d4d1a-kube-api-access-5b8c9" (OuterVolumeSpecName: "kube-api-access-5b8c9") pod "e73813f5-f97f-41dd-8a79-6196557d4d1a" (UID: "e73813f5-f97f-41dd-8a79-6196557d4d1a"). InnerVolumeSpecName "kube-api-access-5b8c9". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.684552 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/novacell02629-account-delete-wsshh" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.684816 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/568addd3-4ad3-47c3-adae-693691e7cd40-rabbitmq-tls" (OuterVolumeSpecName: "rabbitmq-tls") pod "568addd3-4ad3-47c3-adae-693691e7cd40" (UID: "568addd3-4ad3-47c3-adae-693691e7cd40"). InnerVolumeSpecName "rabbitmq-tls". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.693412 5052 generic.go:334] "Generic (PLEG): container finished" podID="941d063f-c834-47a1-bc0c-2096b005e09c" containerID="4db372b51214205aef5ecd524e9175a7ef98e7227daf49311eab08a95f8ca2a4" exitCode=1 Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.693487 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron6c26-account-delete-d7wtw" event={"ID":"941d063f-c834-47a1-bc0c-2096b005e09c","Type":"ContainerDied","Data":"4db372b51214205aef5ecd524e9175a7ef98e7227daf49311eab08a95f8ca2a4"} Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.693523 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron6c26-account-delete-d7wtw" event={"ID":"941d063f-c834-47a1-bc0c-2096b005e09c","Type":"ContainerDied","Data":"fe894db25a6b4c1c2ece51721c19741c414732bda90808788f9d1eebd4775640"} Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.693642 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron6c26-account-delete-d7wtw" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.693953 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e73813f5-f97f-41dd-8a79-6196557d4d1a-rabbitmq-tls" (OuterVolumeSpecName: "rabbitmq-tls") pod "e73813f5-f97f-41dd-8a79-6196557d4d1a" (UID: "e73813f5-f97f-41dd-8a79-6196557d4d1a"). InnerVolumeSpecName "rabbitmq-tls". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.694035 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage03-crc" (OuterVolumeSpecName: "persistence") pod "e73813f5-f97f-41dd-8a79-6196557d4d1a" (UID: "e73813f5-f97f-41dd-8a79-6196557d4d1a"). InnerVolumeSpecName "local-storage03-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.694080 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/568addd3-4ad3-47c3-adae-693691e7cd40-erlang-cookie-secret" (OuterVolumeSpecName: "erlang-cookie-secret") pod "568addd3-4ad3-47c3-adae-693691e7cd40" (UID: "568addd3-4ad3-47c3-adae-693691e7cd40"). InnerVolumeSpecName "erlang-cookie-secret". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.700699 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e73813f5-f97f-41dd-8a79-6196557d4d1a-pod-info" (OuterVolumeSpecName: "pod-info") pod "e73813f5-f97f-41dd-8a79-6196557d4d1a" (UID: "e73813f5-f97f-41dd-8a79-6196557d4d1a"). InnerVolumeSpecName "pod-info". PluginName "kubernetes.io/downward-api", VolumeGidValue "" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.701576 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/placementbfa4-account-delete-flc9f" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.702964 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-metadata-0" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.711934 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-698d8977cd-ppz9n" event={"ID":"e2225738-b0d1-4306-bbee-a61dd9558c80","Type":"ContainerDied","Data":"1f4b0685f379f72ab1ff145175ef6378f1e0a0c2caf829598faeca90aa64dadf"} Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.725732 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e73813f5-f97f-41dd-8a79-6196557d4d1a-erlang-cookie-secret" (OuterVolumeSpecName: "erlang-cookie-secret") pod "e73813f5-f97f-41dd-8a79-6196557d4d1a" (UID: "e73813f5-f97f-41dd-8a79-6196557d4d1a"). InnerVolumeSpecName "erlang-cookie-secret". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.726655 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ceilometer-0" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.726741 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstack-galera-0" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.726803 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican-api-76dc68d888-zmmcs" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.726850 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/keystone-7fd545bd9-2lc7s" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.726903 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/cindera2a9-account-delete-pmspg" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.726953 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/glance7c80-account-delete-knmlm" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.726997 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-cell1-conductor-0" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.727052 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-698d8977cd-ppz9n" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.727106 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/barbican6e62-account-delete-6skhm" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.734856 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d9bf91f1-099c-4e72-97cf-1828241aeaac-kube-api-access-grf5h" (OuterVolumeSpecName: "kube-api-access-grf5h") pod "d9bf91f1-099c-4e72-97cf-1828241aeaac" (UID: "d9bf91f1-099c-4e72-97cf-1828241aeaac"). InnerVolumeSpecName "kube-api-access-grf5h". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.735100 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/941d063f-c834-47a1-bc0c-2096b005e09c-kube-api-access-gwqrq" (OuterVolumeSpecName: "kube-api-access-gwqrq") pod "941d063f-c834-47a1-bc0c-2096b005e09c" (UID: "941d063f-c834-47a1-bc0c-2096b005e09c"). InnerVolumeSpecName "kube-api-access-gwqrq". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.736854 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage09-crc" (OuterVolumeSpecName: "mysql-db") pod "b3363d6f-656e-4e22-a3eb-b1016380a0c8" (UID: "b3363d6f-656e-4e22-a3eb-b1016380a0c8"). InnerVolumeSpecName "local-storage09-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.744709 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/25eead1e-d17c-4b36-8a9f-ef563189d53b-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "25eead1e-d17c-4b36-8a9f-ef563189d53b" (UID: "25eead1e-d17c-4b36-8a9f-ef563189d53b"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.751300 5052 scope.go:117] "RemoveContainer" containerID="78894a79ffab8e64787cde1b1f24aa5fa09f830cec10c1b466b6619c2b33ae10" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.780504 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/13e7e748-92f4-4255-a8eb-aa5a6faf361f-kube-api-access-5vk45" (OuterVolumeSpecName: "kube-api-access-5vk45") pod "13e7e748-92f4-4255-a8eb-aa5a6faf361f" (UID: "13e7e748-92f4-4255-a8eb-aa5a6faf361f"). InnerVolumeSpecName "kube-api-access-5vk45". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.784076 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-gwqrq\" (UniqueName: \"kubernetes.io/projected/941d063f-c834-47a1-bc0c-2096b005e09c-kube-api-access-gwqrq\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.786309 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/25eead1e-d17c-4b36-8a9f-ef563189d53b-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.786458 5052 reconciler_common.go:293] "Volume detached for volume \"rabbitmq-tls\" (UniqueName: \"kubernetes.io/projected/e73813f5-f97f-41dd-8a79-6196557d4d1a-rabbitmq-tls\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.793985 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage03-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage03-crc\") on node \"crc\" " Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.794113 5052 reconciler_common.go:293] "Volume detached for volume \"rabbitmq-tls\" (UniqueName: \"kubernetes.io/projected/568addd3-4ad3-47c3-adae-693691e7cd40-rabbitmq-tls\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.794245 5052 reconciler_common.go:293] "Volume detached for volume \"pod-info\" (UniqueName: \"kubernetes.io/downward-api/e73813f5-f97f-41dd-8a79-6196557d4d1a-pod-info\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.794350 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-5b8c9\" (UniqueName: \"kubernetes.io/projected/e73813f5-f97f-41dd-8a79-6196557d4d1a-kube-api-access-5b8c9\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.794461 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage09-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage09-crc\") on node \"crc\" " Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.794588 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-5vk45\" (UniqueName: \"kubernetes.io/projected/13e7e748-92f4-4255-a8eb-aa5a6faf361f-kube-api-access-5vk45\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.794671 5052 reconciler_common.go:293] "Volume detached for volume \"erlang-cookie-secret\" (UniqueName: \"kubernetes.io/secret/568addd3-4ad3-47c3-adae-693691e7cd40-erlang-cookie-secret\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.794759 5052 reconciler_common.go:293] "Volume detached for volume \"erlang-cookie-secret\" (UniqueName: \"kubernetes.io/secret/e73813f5-f97f-41dd-8a79-6196557d4d1a-erlang-cookie-secret\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.794838 5052 reconciler_common.go:293] "Volume detached for volume \"pod-info\" (UniqueName: \"kubernetes.io/downward-api/568addd3-4ad3-47c3-adae-693691e7cd40-pod-info\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.794924 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-grf5h\" (UniqueName: \"kubernetes.io/projected/d9bf91f1-099c-4e72-97cf-1828241aeaac-kube-api-access-grf5h\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.817629 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4ec49e76-2597-47d6-811b-f65cbe0580ff-kube-api-access-hnfq6" (OuterVolumeSpecName: "kube-api-access-hnfq6") pod "4ec49e76-2597-47d6-811b-f65cbe0580ff" (UID: "4ec49e76-2597-47d6-811b-f65cbe0580ff"). InnerVolumeSpecName "kube-api-access-hnfq6". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.831969 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/79efd225-5ce4-428f-aef9-3f4884931fef-kube-api-access-spfp6" (OuterVolumeSpecName: "kube-api-access-spfp6") pod "79efd225-5ce4-428f-aef9-3f4884931fef" (UID: "79efd225-5ce4-428f-aef9-3f4884931fef"). InnerVolumeSpecName "kube-api-access-spfp6". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.832095 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage02-crc" (OuterVolumeSpecName: "persistence") pod "568addd3-4ad3-47c3-adae-693691e7cd40" (UID: "568addd3-4ad3-47c3-adae-693691e7cd40"). InnerVolumeSpecName "local-storage02-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.835023 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e2225738-b0d1-4306-bbee-a61dd9558c80-httpd-config" (OuterVolumeSpecName: "httpd-config") pod "e2225738-b0d1-4306-bbee-a61dd9558c80" (UID: "e2225738-b0d1-4306-bbee-a61dd9558c80"). InnerVolumeSpecName "httpd-config". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.887569 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage09-crc" (UniqueName: "kubernetes.io/local-volume/local-storage09-crc") on node "crc" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.894618 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/swift-proxy-5479cb94c8-q2snc" podUID="eefe83f8-0235-4b16-aefb-5ec7b2d92fcf" containerName="proxy-server" probeResult="failure" output="Get \"https://10.217.0.190:8080/healthcheck\": dial tcp 10.217.0.190:8080: i/o timeout" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.895303 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/swift-proxy-5479cb94c8-q2snc" podUID="eefe83f8-0235-4b16-aefb-5ec7b2d92fcf" containerName="proxy-httpd" probeResult="failure" output="Get \"https://10.217.0.190:8080/healthcheck\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.896419 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s76qn\" (UniqueName: \"kubernetes.io/projected/1b7ef5eb-406e-4e2c-9d9c-d763919f56fd-kube-api-access-s76qn\") pod \"1b7ef5eb-406e-4e2c-9d9c-d763919f56fd\" (UID: \"1b7ef5eb-406e-4e2c-9d9c-d763919f56fd\") " Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.897073 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/1b7ef5eb-406e-4e2c-9d9c-d763919f56fd-operator-scripts\") pod \"1b7ef5eb-406e-4e2c-9d9c-d763919f56fd\" (UID: \"1b7ef5eb-406e-4e2c-9d9c-d763919f56fd\") " Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.897929 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-spfp6\" (UniqueName: \"kubernetes.io/projected/79efd225-5ce4-428f-aef9-3f4884931fef-kube-api-access-spfp6\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.897950 5052 reconciler_common.go:293] "Volume detached for volume \"httpd-config\" (UniqueName: \"kubernetes.io/secret/e2225738-b0d1-4306-bbee-a61dd9558c80-httpd-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.897984 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage02-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage02-crc\") on node \"crc\" " Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.897998 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-hnfq6\" (UniqueName: \"kubernetes.io/projected/4ec49e76-2597-47d6-811b-f65cbe0580ff-kube-api-access-hnfq6\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.898010 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage09-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage09-crc\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.898897 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1b7ef5eb-406e-4e2c-9d9c-d763919f56fd-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "1b7ef5eb-406e-4e2c-9d9c-d763919f56fd" (UID: "1b7ef5eb-406e-4e2c-9d9c-d763919f56fd"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.919527 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1b7ef5eb-406e-4e2c-9d9c-d763919f56fd-kube-api-access-s76qn" (OuterVolumeSpecName: "kube-api-access-s76qn") pod "1b7ef5eb-406e-4e2c-9d9c-d763919f56fd" (UID: "1b7ef5eb-406e-4e2c-9d9c-d763919f56fd"). InnerVolumeSpecName "kube-api-access-s76qn". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.934152 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2adad00d-01e3-4441-a3fc-87d162a5e309-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "2adad00d-01e3-4441-a3fc-87d162a5e309" (UID: "2adad00d-01e3-4441-a3fc-87d162a5e309"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.950869 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/568addd3-4ad3-47c3-adae-693691e7cd40-server-conf" (OuterVolumeSpecName: "server-conf") pod "568addd3-4ad3-47c3-adae-693691e7cd40" (UID: "568addd3-4ad3-47c3-adae-693691e7cd40"). InnerVolumeSpecName "server-conf". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.965537 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/022746a5-1c20-4729-8688-5903831f57ed-config-data" (OuterVolumeSpecName: "config-data") pod "022746a5-1c20-4729-8688-5903831f57ed" (UID: "022746a5-1c20-4729-8688-5903831f57ed"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.977012 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0191d138-7aff-42c2-88d2-fbbaf1722bfd-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "0191d138-7aff-42c2-88d2-fbbaf1722bfd" (UID: "0191d138-7aff-42c2-88d2-fbbaf1722bfd"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:54:52 crc kubenswrapper[5052]: I1203 12:54:52.980094 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage03-crc" (UniqueName: "kubernetes.io/local-volume/local-storage03-crc") on node "crc" Dec 03 12:54:53 crc kubenswrapper[5052]: I1203 12:54:53.001738 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e2225738-b0d1-4306-bbee-a61dd9558c80-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "e2225738-b0d1-4306-bbee-a61dd9558c80" (UID: "e2225738-b0d1-4306-bbee-a61dd9558c80"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:54:53 crc kubenswrapper[5052]: I1203 12:54:53.002053 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/1b7ef5eb-406e-4e2c-9d9c-d763919f56fd-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:53 crc kubenswrapper[5052]: I1203 12:54:53.002102 5052 reconciler_common.go:293] "Volume detached for volume \"server-conf\" (UniqueName: \"kubernetes.io/configmap/568addd3-4ad3-47c3-adae-693691e7cd40-server-conf\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:53 crc kubenswrapper[5052]: I1203 12:54:53.002115 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0191d138-7aff-42c2-88d2-fbbaf1722bfd-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:53 crc kubenswrapper[5052]: I1203 12:54:53.002128 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-s76qn\" (UniqueName: \"kubernetes.io/projected/1b7ef5eb-406e-4e2c-9d9c-d763919f56fd-kube-api-access-s76qn\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:53 crc kubenswrapper[5052]: I1203 12:54:53.002137 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/022746a5-1c20-4729-8688-5903831f57ed-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:53 crc kubenswrapper[5052]: I1203 12:54:53.002147 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2adad00d-01e3-4441-a3fc-87d162a5e309-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:53 crc kubenswrapper[5052]: I1203 12:54:53.002156 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage03-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage03-crc\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:53 crc kubenswrapper[5052]: I1203 12:54:53.005417 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0191d138-7aff-42c2-88d2-fbbaf1722bfd-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "0191d138-7aff-42c2-88d2-fbbaf1722bfd" (UID: "0191d138-7aff-42c2-88d2-fbbaf1722bfd"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:54:53 crc kubenswrapper[5052]: I1203 12:54:53.016619 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/02edb9e9-3f42-4aea-9fe6-7efa332d5fb8-ceilometer-tls-certs" (OuterVolumeSpecName: "ceilometer-tls-certs") pod "02edb9e9-3f42-4aea-9fe6-7efa332d5fb8" (UID: "02edb9e9-3f42-4aea-9fe6-7efa332d5fb8"). InnerVolumeSpecName "ceilometer-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:54:53 crc kubenswrapper[5052]: I1203 12:54:53.022967 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0191d138-7aff-42c2-88d2-fbbaf1722bfd-config-data" (OuterVolumeSpecName: "config-data") pod "0191d138-7aff-42c2-88d2-fbbaf1722bfd" (UID: "0191d138-7aff-42c2-88d2-fbbaf1722bfd"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:54:53 crc kubenswrapper[5052]: I1203 12:54:53.025954 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/02edb9e9-3f42-4aea-9fe6-7efa332d5fb8-sg-core-conf-yaml" (OuterVolumeSpecName: "sg-core-conf-yaml") pod "02edb9e9-3f42-4aea-9fe6-7efa332d5fb8" (UID: "02edb9e9-3f42-4aea-9fe6-7efa332d5fb8"). InnerVolumeSpecName "sg-core-conf-yaml". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:54:53 crc kubenswrapper[5052]: I1203 12:54:53.032721 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e2225738-b0d1-4306-bbee-a61dd9558c80-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "e2225738-b0d1-4306-bbee-a61dd9558c80" (UID: "e2225738-b0d1-4306-bbee-a61dd9558c80"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:54:53 crc kubenswrapper[5052]: I1203 12:54:53.046926 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage02-crc" (UniqueName: "kubernetes.io/local-volume/local-storage02-crc") on node "crc" Dec 03 12:54:53 crc kubenswrapper[5052]: I1203 12:54:53.048125 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/022746a5-1c20-4729-8688-5903831f57ed-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "022746a5-1c20-4729-8688-5903831f57ed" (UID: "022746a5-1c20-4729-8688-5903831f57ed"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:54:53 crc kubenswrapper[5052]: I1203 12:54:53.064029 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e2225738-b0d1-4306-bbee-a61dd9558c80-ovndb-tls-certs" (OuterVolumeSpecName: "ovndb-tls-certs") pod "e2225738-b0d1-4306-bbee-a61dd9558c80" (UID: "e2225738-b0d1-4306-bbee-a61dd9558c80"). InnerVolumeSpecName "ovndb-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:54:53 crc kubenswrapper[5052]: I1203 12:54:53.069375 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2adad00d-01e3-4441-a3fc-87d162a5e309-config-data" (OuterVolumeSpecName: "config-data") pod "2adad00d-01e3-4441-a3fc-87d162a5e309" (UID: "2adad00d-01e3-4441-a3fc-87d162a5e309"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:54:53 crc kubenswrapper[5052]: I1203 12:54:53.080925 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/568addd3-4ad3-47c3-adae-693691e7cd40-config-data" (OuterVolumeSpecName: "config-data") pod "568addd3-4ad3-47c3-adae-693691e7cd40" (UID: "568addd3-4ad3-47c3-adae-693691e7cd40"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:54:53 crc kubenswrapper[5052]: I1203 12:54:53.087503 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b3363d6f-656e-4e22-a3eb-b1016380a0c8-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "b3363d6f-656e-4e22-a3eb-b1016380a0c8" (UID: "b3363d6f-656e-4e22-a3eb-b1016380a0c8"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:54:53 crc kubenswrapper[5052]: I1203 12:54:53.088234 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/25eead1e-d17c-4b36-8a9f-ef563189d53b-config-data" (OuterVolumeSpecName: "config-data") pod "25eead1e-d17c-4b36-8a9f-ef563189d53b" (UID: "25eead1e-d17c-4b36-8a9f-ef563189d53b"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:54:53 crc kubenswrapper[5052]: I1203 12:54:53.093461 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/02edb9e9-3f42-4aea-9fe6-7efa332d5fb8-config-data" (OuterVolumeSpecName: "config-data") pod "02edb9e9-3f42-4aea-9fe6-7efa332d5fb8" (UID: "02edb9e9-3f42-4aea-9fe6-7efa332d5fb8"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:54:53 crc kubenswrapper[5052]: I1203 12:54:53.100121 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e73813f5-f97f-41dd-8a79-6196557d4d1a-config-data" (OuterVolumeSpecName: "config-data") pod "e73813f5-f97f-41dd-8a79-6196557d4d1a" (UID: "e73813f5-f97f-41dd-8a79-6196557d4d1a"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:54:53 crc kubenswrapper[5052]: I1203 12:54:53.105051 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/022746a5-1c20-4729-8688-5903831f57ed-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:53 crc kubenswrapper[5052]: I1203 12:54:53.105081 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/2adad00d-01e3-4441-a3fc-87d162a5e309-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:53 crc kubenswrapper[5052]: I1203 12:54:53.105091 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/e2225738-b0d1-4306-bbee-a61dd9558c80-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:53 crc kubenswrapper[5052]: I1203 12:54:53.105100 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/0191d138-7aff-42c2-88d2-fbbaf1722bfd-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:53 crc kubenswrapper[5052]: I1203 12:54:53.105109 5052 reconciler_common.go:293] "Volume detached for volume \"ceilometer-tls-certs\" (UniqueName: \"kubernetes.io/secret/02edb9e9-3f42-4aea-9fe6-7efa332d5fb8-ceilometer-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:53 crc kubenswrapper[5052]: I1203 12:54:53.105119 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage02-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage02-crc\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:53 crc kubenswrapper[5052]: I1203 12:54:53.105128 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/0191d138-7aff-42c2-88d2-fbbaf1722bfd-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:53 crc kubenswrapper[5052]: I1203 12:54:53.105140 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/e73813f5-f97f-41dd-8a79-6196557d4d1a-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:53 crc kubenswrapper[5052]: I1203 12:54:53.105152 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/02edb9e9-3f42-4aea-9fe6-7efa332d5fb8-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:53 crc kubenswrapper[5052]: I1203 12:54:53.105163 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e2225738-b0d1-4306-bbee-a61dd9558c80-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:53 crc kubenswrapper[5052]: I1203 12:54:53.105174 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/25eead1e-d17c-4b36-8a9f-ef563189d53b-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:53 crc kubenswrapper[5052]: I1203 12:54:53.105185 5052 reconciler_common.go:293] "Volume detached for volume \"sg-core-conf-yaml\" (UniqueName: \"kubernetes.io/secret/02edb9e9-3f42-4aea-9fe6-7efa332d5fb8-sg-core-conf-yaml\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:53 crc kubenswrapper[5052]: I1203 12:54:53.105195 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/568addd3-4ad3-47c3-adae-693691e7cd40-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:53 crc kubenswrapper[5052]: I1203 12:54:53.105206 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b3363d6f-656e-4e22-a3eb-b1016380a0c8-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:53 crc kubenswrapper[5052]: I1203 12:54:53.105216 5052 reconciler_common.go:293] "Volume detached for volume \"ovndb-tls-certs\" (UniqueName: \"kubernetes.io/secret/e2225738-b0d1-4306-bbee-a61dd9558c80-ovndb-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:53 crc kubenswrapper[5052]: I1203 12:54:53.115968 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0191d138-7aff-42c2-88d2-fbbaf1722bfd-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "0191d138-7aff-42c2-88d2-fbbaf1722bfd" (UID: "0191d138-7aff-42c2-88d2-fbbaf1722bfd"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:54:53 crc kubenswrapper[5052]: I1203 12:54:53.127456 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3b93bc75-db87-4677-be2f-910d07bb696c-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "3b93bc75-db87-4677-be2f-910d07bb696c" (UID: "3b93bc75-db87-4677-be2f-910d07bb696c"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:54:53 crc kubenswrapper[5052]: I1203 12:54:53.130255 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b3363d6f-656e-4e22-a3eb-b1016380a0c8-galera-tls-certs" (OuterVolumeSpecName: "galera-tls-certs") pod "b3363d6f-656e-4e22-a3eb-b1016380a0c8" (UID: "b3363d6f-656e-4e22-a3eb-b1016380a0c8"). InnerVolumeSpecName "galera-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:54:53 crc kubenswrapper[5052]: I1203 12:54:53.137114 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/022746a5-1c20-4729-8688-5903831f57ed-nova-metadata-tls-certs" (OuterVolumeSpecName: "nova-metadata-tls-certs") pod "022746a5-1c20-4729-8688-5903831f57ed" (UID: "022746a5-1c20-4729-8688-5903831f57ed"). InnerVolumeSpecName "nova-metadata-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:54:53 crc kubenswrapper[5052]: I1203 12:54:53.143884 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/25eead1e-d17c-4b36-8a9f-ef563189d53b-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "25eead1e-d17c-4b36-8a9f-ef563189d53b" (UID: "25eead1e-d17c-4b36-8a9f-ef563189d53b"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:54:53 crc kubenswrapper[5052]: I1203 12:54:53.144388 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e2225738-b0d1-4306-bbee-a61dd9558c80-config" (OuterVolumeSpecName: "config") pod "e2225738-b0d1-4306-bbee-a61dd9558c80" (UID: "e2225738-b0d1-4306-bbee-a61dd9558c80"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:54:53 crc kubenswrapper[5052]: I1203 12:54:53.147741 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4ec49e76-2597-47d6-811b-f65cbe0580ff-config-data" (OuterVolumeSpecName: "config-data") pod "4ec49e76-2597-47d6-811b-f65cbe0580ff" (UID: "4ec49e76-2597-47d6-811b-f65cbe0580ff"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:54:53 crc kubenswrapper[5052]: I1203 12:54:53.148421 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4ec49e76-2597-47d6-811b-f65cbe0580ff-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "4ec49e76-2597-47d6-811b-f65cbe0580ff" (UID: "4ec49e76-2597-47d6-811b-f65cbe0580ff"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:54:53 crc kubenswrapper[5052]: I1203 12:54:53.153492 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e73813f5-f97f-41dd-8a79-6196557d4d1a-server-conf" (OuterVolumeSpecName: "server-conf") pod "e73813f5-f97f-41dd-8a79-6196557d4d1a" (UID: "e73813f5-f97f-41dd-8a79-6196557d4d1a"). InnerVolumeSpecName "server-conf". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:54:53 crc kubenswrapper[5052]: I1203 12:54:53.161676 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e2225738-b0d1-4306-bbee-a61dd9558c80-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "e2225738-b0d1-4306-bbee-a61dd9558c80" (UID: "e2225738-b0d1-4306-bbee-a61dd9558c80"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:54:53 crc kubenswrapper[5052]: I1203 12:54:53.163278 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/25eead1e-d17c-4b36-8a9f-ef563189d53b-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "25eead1e-d17c-4b36-8a9f-ef563189d53b" (UID: "25eead1e-d17c-4b36-8a9f-ef563189d53b"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:54:53 crc kubenswrapper[5052]: I1203 12:54:53.166511 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3b93bc75-db87-4677-be2f-910d07bb696c-ovn-northd-tls-certs" (OuterVolumeSpecName: "ovn-northd-tls-certs") pod "3b93bc75-db87-4677-be2f-910d07bb696c" (UID: "3b93bc75-db87-4677-be2f-910d07bb696c"). InnerVolumeSpecName "ovn-northd-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:54:53 crc kubenswrapper[5052]: I1203 12:54:53.179342 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/02edb9e9-3f42-4aea-9fe6-7efa332d5fb8-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "02edb9e9-3f42-4aea-9fe6-7efa332d5fb8" (UID: "02edb9e9-3f42-4aea-9fe6-7efa332d5fb8"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:54:53 crc kubenswrapper[5052]: I1203 12:54:53.184311 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3b93bc75-db87-4677-be2f-910d07bb696c-metrics-certs-tls-certs" (OuterVolumeSpecName: "metrics-certs-tls-certs") pod "3b93bc75-db87-4677-be2f-910d07bb696c" (UID: "3b93bc75-db87-4677-be2f-910d07bb696c"). InnerVolumeSpecName "metrics-certs-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:54:53 crc kubenswrapper[5052]: I1203 12:54:53.202881 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/568addd3-4ad3-47c3-adae-693691e7cd40-rabbitmq-confd" (OuterVolumeSpecName: "rabbitmq-confd") pod "568addd3-4ad3-47c3-adae-693691e7cd40" (UID: "568addd3-4ad3-47c3-adae-693691e7cd40"). InnerVolumeSpecName "rabbitmq-confd". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:54:53 crc kubenswrapper[5052]: I1203 12:54:53.208148 5052 reconciler_common.go:293] "Volume detached for volume \"rabbitmq-confd\" (UniqueName: \"kubernetes.io/projected/568addd3-4ad3-47c3-adae-693691e7cd40-rabbitmq-confd\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:53 crc kubenswrapper[5052]: I1203 12:54:53.208354 5052 reconciler_common.go:293] "Volume detached for volume \"galera-tls-certs\" (UniqueName: \"kubernetes.io/secret/b3363d6f-656e-4e22-a3eb-b1016380a0c8-galera-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:53 crc kubenswrapper[5052]: I1203 12:54:53.208372 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/4ec49e76-2597-47d6-811b-f65cbe0580ff-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:53 crc kubenswrapper[5052]: I1203 12:54:53.208385 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3b93bc75-db87-4677-be2f-910d07bb696c-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:53 crc kubenswrapper[5052]: I1203 12:54:53.208396 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/e2225738-b0d1-4306-bbee-a61dd9558c80-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:53 crc kubenswrapper[5052]: I1203 12:54:53.208408 5052 reconciler_common.go:293] "Volume detached for volume \"server-conf\" (UniqueName: \"kubernetes.io/configmap/e73813f5-f97f-41dd-8a79-6196557d4d1a-server-conf\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:53 crc kubenswrapper[5052]: I1203 12:54:53.208419 5052 reconciler_common.go:293] "Volume detached for volume \"metrics-certs-tls-certs\" (UniqueName: \"kubernetes.io/secret/3b93bc75-db87-4677-be2f-910d07bb696c-metrics-certs-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:53 crc kubenswrapper[5052]: I1203 12:54:53.208434 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/02edb9e9-3f42-4aea-9fe6-7efa332d5fb8-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:53 crc kubenswrapper[5052]: I1203 12:54:53.208445 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/25eead1e-d17c-4b36-8a9f-ef563189d53b-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:53 crc kubenswrapper[5052]: I1203 12:54:53.208457 5052 reconciler_common.go:293] "Volume detached for volume \"ovn-northd-tls-certs\" (UniqueName: \"kubernetes.io/secret/3b93bc75-db87-4677-be2f-910d07bb696c-ovn-northd-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:53 crc kubenswrapper[5052]: I1203 12:54:53.208469 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4ec49e76-2597-47d6-811b-f65cbe0580ff-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:53 crc kubenswrapper[5052]: I1203 12:54:53.208483 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/25eead1e-d17c-4b36-8a9f-ef563189d53b-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:53 crc kubenswrapper[5052]: I1203 12:54:53.208496 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/0191d138-7aff-42c2-88d2-fbbaf1722bfd-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:53 crc kubenswrapper[5052]: I1203 12:54:53.208513 5052 reconciler_common.go:293] "Volume detached for volume \"nova-metadata-tls-certs\" (UniqueName: \"kubernetes.io/secret/022746a5-1c20-4729-8688-5903831f57ed-nova-metadata-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:53 crc kubenswrapper[5052]: I1203 12:54:53.208524 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/e2225738-b0d1-4306-bbee-a61dd9558c80-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:53 crc kubenswrapper[5052]: I1203 12:54:53.226030 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e73813f5-f97f-41dd-8a79-6196557d4d1a-rabbitmq-confd" (OuterVolumeSpecName: "rabbitmq-confd") pod "e73813f5-f97f-41dd-8a79-6196557d4d1a" (UID: "e73813f5-f97f-41dd-8a79-6196557d4d1a"). InnerVolumeSpecName "rabbitmq-confd". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:54:53 crc kubenswrapper[5052]: I1203 12:54:53.255187 5052 scope.go:117] "RemoveContainer" containerID="34ec2e3b276084c3a6169386d90460245b79be1c0922971153b1615fd7e2d686" Dec 03 12:54:53 crc kubenswrapper[5052]: I1203 12:54:53.278904 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-0"] Dec 03 12:54:53 crc kubenswrapper[5052]: I1203 12:54:53.291086 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-cell1-conductor-0"] Dec 03 12:54:53 crc kubenswrapper[5052]: I1203 12:54:53.298082 5052 scope.go:117] "RemoveContainer" containerID="a1b6cbc8b7e6a360fb1952ecd8d7fe6374e2e1fc02dd0db8aad15d96a40e30e9" Dec 03 12:54:53 crc kubenswrapper[5052]: I1203 12:54:53.303563 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/cindera2a9-account-delete-pmspg"] Dec 03 12:54:53 crc kubenswrapper[5052]: I1203 12:54:53.311589 5052 reconciler_common.go:293] "Volume detached for volume \"rabbitmq-confd\" (UniqueName: \"kubernetes.io/projected/e73813f5-f97f-41dd-8a79-6196557d4d1a-rabbitmq-confd\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:53 crc kubenswrapper[5052]: I1203 12:54:53.314034 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/cindera2a9-account-delete-pmspg"] Dec 03 12:54:53 crc kubenswrapper[5052]: I1203 12:54:53.323510 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:54:53 crc kubenswrapper[5052]: I1203 12:54:53.333031 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-scheduler-0"] Dec 03 12:54:53 crc kubenswrapper[5052]: I1203 12:54:53.334792 5052 scope.go:117] "RemoveContainer" containerID="11ddaf754d754f510a35d327364ee83e653bb8146394aaffa51c37770ca3011a" Dec 03 12:54:53 crc kubenswrapper[5052]: I1203 12:54:53.345058 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/glance7c80-account-delete-knmlm"] Dec 03 12:54:53 crc kubenswrapper[5052]: I1203 12:54:53.363320 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/glance7c80-account-delete-knmlm"] Dec 03 12:54:53 crc kubenswrapper[5052]: I1203 12:54:53.374665 5052 scope.go:117] "RemoveContainer" containerID="cf0936e9c4c87ab125307449bfe7ff21ea78778fbaef0f8f0360dbecdf7e1591" Dec 03 12:54:53 crc kubenswrapper[5052]: I1203 12:54:53.377016 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/neutron6c26-account-delete-d7wtw"] Dec 03 12:54:53 crc kubenswrapper[5052]: I1203 12:54:53.389694 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/neutron6c26-account-delete-d7wtw"] Dec 03 12:54:53 crc kubenswrapper[5052]: I1203 12:54:53.396939 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican6e62-account-delete-6skhm"] Dec 03 12:54:53 crc kubenswrapper[5052]: I1203 12:54:53.412290 5052 scope.go:117] "RemoveContainer" containerID="42f16d033db22635e368a85dc53ea3877fb872db34db75e3fb79daeb9c825a36" Dec 03 12:54:53 crc kubenswrapper[5052]: I1203 12:54:53.416698 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/barbican6e62-account-delete-6skhm"] Dec 03 12:54:53 crc kubenswrapper[5052]: I1203 12:54:53.426772 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/novaapi7066-account-delete-whswm"] Dec 03 12:54:53 crc kubenswrapper[5052]: I1203 12:54:53.436757 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/novaapi7066-account-delete-whswm"] Dec 03 12:54:53 crc kubenswrapper[5052]: I1203 12:54:53.448199 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/placementbfa4-account-delete-flc9f"] Dec 03 12:54:53 crc kubenswrapper[5052]: I1203 12:54:53.450217 5052 scope.go:117] "RemoveContainer" containerID="dc308258e1e18beb76be744d3270524d74c7f8be911cbba0e2cd35f103d54af5" Dec 03 12:54:53 crc kubenswrapper[5052]: I1203 12:54:53.468186 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/placementbfa4-account-delete-flc9f"] Dec 03 12:54:53 crc kubenswrapper[5052]: I1203 12:54:53.480663 5052 scope.go:117] "RemoveContainer" containerID="d3e528340d7b7c51ad079f738747e4effb2e76401a14409088fb9ae1abff9903" Dec 03 12:54:53 crc kubenswrapper[5052]: I1203 12:54:53.488522 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/rabbitmq-server-0"] Dec 03 12:54:53 crc kubenswrapper[5052]: I1203 12:54:53.503052 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/rabbitmq-server-0"] Dec 03 12:54:53 crc kubenswrapper[5052]: I1203 12:54:53.504204 5052 scope.go:117] "RemoveContainer" containerID="9c5fb38f25954ef5355f2862f8e1ab7d9a859ef546a60f10dfb32dbd818c660c" Dec 03 12:54:53 crc kubenswrapper[5052]: I1203 12:54:53.513113 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:54:53 crc kubenswrapper[5052]: I1203 12:54:53.520793 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ceilometer-0"] Dec 03 12:54:53 crc kubenswrapper[5052]: I1203 12:54:53.527320 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:54:53 crc kubenswrapper[5052]: I1203 12:54:53.532189 5052 scope.go:117] "RemoveContainer" containerID="9c5fb38f25954ef5355f2862f8e1ab7d9a859ef546a60f10dfb32dbd818c660c" Dec 03 12:54:53 crc kubenswrapper[5052]: E1203 12:54:53.532707 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9c5fb38f25954ef5355f2862f8e1ab7d9a859ef546a60f10dfb32dbd818c660c\": container with ID starting with 9c5fb38f25954ef5355f2862f8e1ab7d9a859ef546a60f10dfb32dbd818c660c not found: ID does not exist" containerID="9c5fb38f25954ef5355f2862f8e1ab7d9a859ef546a60f10dfb32dbd818c660c" Dec 03 12:54:53 crc kubenswrapper[5052]: I1203 12:54:53.532749 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9c5fb38f25954ef5355f2862f8e1ab7d9a859ef546a60f10dfb32dbd818c660c"} err="failed to get container status \"9c5fb38f25954ef5355f2862f8e1ab7d9a859ef546a60f10dfb32dbd818c660c\": rpc error: code = NotFound desc = could not find container \"9c5fb38f25954ef5355f2862f8e1ab7d9a859ef546a60f10dfb32dbd818c660c\": container with ID starting with 9c5fb38f25954ef5355f2862f8e1ab7d9a859ef546a60f10dfb32dbd818c660c not found: ID does not exist" Dec 03 12:54:53 crc kubenswrapper[5052]: I1203 12:54:53.532775 5052 scope.go:117] "RemoveContainer" containerID="ec5a6e0000f85ac80b044c5e702c36b00720989c7bd27aac0130a6739a2d1bd8" Dec 03 12:54:53 crc kubenswrapper[5052]: I1203 12:54:53.534031 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-metadata-0"] Dec 03 12:54:53 crc kubenswrapper[5052]: I1203 12:54:53.541373 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/rabbitmq-cell1-server-0"] Dec 03 12:54:53 crc kubenswrapper[5052]: I1203 12:54:53.546591 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/rabbitmq-cell1-server-0"] Dec 03 12:54:53 crc kubenswrapper[5052]: I1203 12:54:53.550920 5052 scope.go:117] "RemoveContainer" containerID="ec5a6e0000f85ac80b044c5e702c36b00720989c7bd27aac0130a6739a2d1bd8" Dec 03 12:54:53 crc kubenswrapper[5052]: E1203 12:54:53.551284 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ec5a6e0000f85ac80b044c5e702c36b00720989c7bd27aac0130a6739a2d1bd8\": container with ID starting with ec5a6e0000f85ac80b044c5e702c36b00720989c7bd27aac0130a6739a2d1bd8 not found: ID does not exist" containerID="ec5a6e0000f85ac80b044c5e702c36b00720989c7bd27aac0130a6739a2d1bd8" Dec 03 12:54:53 crc kubenswrapper[5052]: I1203 12:54:53.551322 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ec5a6e0000f85ac80b044c5e702c36b00720989c7bd27aac0130a6739a2d1bd8"} err="failed to get container status \"ec5a6e0000f85ac80b044c5e702c36b00720989c7bd27aac0130a6739a2d1bd8\": rpc error: code = NotFound desc = could not find container \"ec5a6e0000f85ac80b044c5e702c36b00720989c7bd27aac0130a6739a2d1bd8\": container with ID starting with ec5a6e0000f85ac80b044c5e702c36b00720989c7bd27aac0130a6739a2d1bd8 not found: ID does not exist" Dec 03 12:54:53 crc kubenswrapper[5052]: I1203 12:54:53.551346 5052 scope.go:117] "RemoveContainer" containerID="7cdee3173161e7591d4f168eb2ed4975b7806623d040f607514a1bb1ab4a5542" Dec 03 12:54:53 crc kubenswrapper[5052]: I1203 12:54:53.552680 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/barbican-api-76dc68d888-zmmcs"] Dec 03 12:54:53 crc kubenswrapper[5052]: I1203 12:54:53.559522 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/barbican-api-76dc68d888-zmmcs"] Dec 03 12:54:53 crc kubenswrapper[5052]: I1203 12:54:53.566320 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/neutron-698d8977cd-ppz9n"] Dec 03 12:54:53 crc kubenswrapper[5052]: I1203 12:54:53.571631 5052 scope.go:117] "RemoveContainer" containerID="06bf7c042266ac81635b44d190310f81518aa47f7b8008ddde3f54ca515b15fd" Dec 03 12:54:53 crc kubenswrapper[5052]: I1203 12:54:53.574744 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/neutron-698d8977cd-ppz9n"] Dec 03 12:54:53 crc kubenswrapper[5052]: I1203 12:54:53.584618 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/openstack-galera-0"] Dec 03 12:54:53 crc kubenswrapper[5052]: I1203 12:54:53.600088 5052 scope.go:117] "RemoveContainer" containerID="db4b319560fefd949092d7f9bbfd3ae7aab4d1e6d1c4c90e729f406df14b8b79" Dec 03 12:54:53 crc kubenswrapper[5052]: I1203 12:54:53.617368 5052 scope.go:117] "RemoveContainer" containerID="0ef2bfbc937b79d5fdccf169b0b5adaf3d759d2b9428459842a9476d8aacd0c1" Dec 03 12:54:53 crc kubenswrapper[5052]: I1203 12:54:53.621123 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/openstack-galera-0"] Dec 03 12:54:53 crc kubenswrapper[5052]: I1203 12:54:53.630204 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/keystone-7fd545bd9-2lc7s"] Dec 03 12:54:53 crc kubenswrapper[5052]: I1203 12:54:53.637811 5052 scope.go:117] "RemoveContainer" containerID="4db372b51214205aef5ecd524e9175a7ef98e7227daf49311eab08a95f8ca2a4" Dec 03 12:54:53 crc kubenswrapper[5052]: I1203 12:54:53.637807 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/keystone-7fd545bd9-2lc7s"] Dec 03 12:54:53 crc kubenswrapper[5052]: I1203 12:54:53.651507 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ovn-northd-0"] Dec 03 12:54:53 crc kubenswrapper[5052]: I1203 12:54:53.660683 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ovn-northd-0"] Dec 03 12:54:53 crc kubenswrapper[5052]: I1203 12:54:53.663319 5052 scope.go:117] "RemoveContainer" containerID="c89e8180cefdc0d2331c24dc4ecea16dbdbd25fd8e2135fbbbde4e84750348ce" Dec 03 12:54:53 crc kubenswrapper[5052]: I1203 12:54:53.682749 5052 scope.go:117] "RemoveContainer" containerID="4db372b51214205aef5ecd524e9175a7ef98e7227daf49311eab08a95f8ca2a4" Dec 03 12:54:53 crc kubenswrapper[5052]: E1203 12:54:53.683052 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4db372b51214205aef5ecd524e9175a7ef98e7227daf49311eab08a95f8ca2a4\": container with ID starting with 4db372b51214205aef5ecd524e9175a7ef98e7227daf49311eab08a95f8ca2a4 not found: ID does not exist" containerID="4db372b51214205aef5ecd524e9175a7ef98e7227daf49311eab08a95f8ca2a4" Dec 03 12:54:53 crc kubenswrapper[5052]: I1203 12:54:53.683088 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4db372b51214205aef5ecd524e9175a7ef98e7227daf49311eab08a95f8ca2a4"} err="failed to get container status \"4db372b51214205aef5ecd524e9175a7ef98e7227daf49311eab08a95f8ca2a4\": rpc error: code = NotFound desc = could not find container \"4db372b51214205aef5ecd524e9175a7ef98e7227daf49311eab08a95f8ca2a4\": container with ID starting with 4db372b51214205aef5ecd524e9175a7ef98e7227daf49311eab08a95f8ca2a4 not found: ID does not exist" Dec 03 12:54:53 crc kubenswrapper[5052]: I1203 12:54:53.683108 5052 scope.go:117] "RemoveContainer" containerID="c89e8180cefdc0d2331c24dc4ecea16dbdbd25fd8e2135fbbbde4e84750348ce" Dec 03 12:54:53 crc kubenswrapper[5052]: E1203 12:54:53.683447 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c89e8180cefdc0d2331c24dc4ecea16dbdbd25fd8e2135fbbbde4e84750348ce\": container with ID starting with c89e8180cefdc0d2331c24dc4ecea16dbdbd25fd8e2135fbbbde4e84750348ce not found: ID does not exist" containerID="c89e8180cefdc0d2331c24dc4ecea16dbdbd25fd8e2135fbbbde4e84750348ce" Dec 03 12:54:53 crc kubenswrapper[5052]: I1203 12:54:53.683488 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c89e8180cefdc0d2331c24dc4ecea16dbdbd25fd8e2135fbbbde4e84750348ce"} err="failed to get container status \"c89e8180cefdc0d2331c24dc4ecea16dbdbd25fd8e2135fbbbde4e84750348ce\": rpc error: code = NotFound desc = could not find container \"c89e8180cefdc0d2331c24dc4ecea16dbdbd25fd8e2135fbbbde4e84750348ce\": container with ID starting with c89e8180cefdc0d2331c24dc4ecea16dbdbd25fd8e2135fbbbde4e84750348ce not found: ID does not exist" Dec 03 12:54:53 crc kubenswrapper[5052]: I1203 12:54:53.683534 5052 scope.go:117] "RemoveContainer" containerID="89798d3d34dfd5e2ea4266a0c6cf4bb6c1e2078d3038566fe9362ddc07ef9734" Dec 03 12:54:53 crc kubenswrapper[5052]: I1203 12:54:53.685654 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0191d138-7aff-42c2-88d2-fbbaf1722bfd" path="/var/lib/kubelet/pods/0191d138-7aff-42c2-88d2-fbbaf1722bfd/volumes" Dec 03 12:54:53 crc kubenswrapper[5052]: I1203 12:54:53.686379 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="022746a5-1c20-4729-8688-5903831f57ed" path="/var/lib/kubelet/pods/022746a5-1c20-4729-8688-5903831f57ed/volumes" Dec 03 12:54:53 crc kubenswrapper[5052]: I1203 12:54:53.687889 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="02edb9e9-3f42-4aea-9fe6-7efa332d5fb8" path="/var/lib/kubelet/pods/02edb9e9-3f42-4aea-9fe6-7efa332d5fb8/volumes" Dec 03 12:54:53 crc kubenswrapper[5052]: I1203 12:54:53.689860 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="13e7e748-92f4-4255-a8eb-aa5a6faf361f" path="/var/lib/kubelet/pods/13e7e748-92f4-4255-a8eb-aa5a6faf361f/volumes" Dec 03 12:54:53 crc kubenswrapper[5052]: I1203 12:54:53.690622 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="25eead1e-d17c-4b36-8a9f-ef563189d53b" path="/var/lib/kubelet/pods/25eead1e-d17c-4b36-8a9f-ef563189d53b/volumes" Dec 03 12:54:53 crc kubenswrapper[5052]: I1203 12:54:53.691931 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2adad00d-01e3-4441-a3fc-87d162a5e309" path="/var/lib/kubelet/pods/2adad00d-01e3-4441-a3fc-87d162a5e309/volumes" Dec 03 12:54:53 crc kubenswrapper[5052]: I1203 12:54:53.692893 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3b93bc75-db87-4677-be2f-910d07bb696c" path="/var/lib/kubelet/pods/3b93bc75-db87-4677-be2f-910d07bb696c/volumes" Dec 03 12:54:53 crc kubenswrapper[5052]: I1203 12:54:53.694683 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4ec49e76-2597-47d6-811b-f65cbe0580ff" path="/var/lib/kubelet/pods/4ec49e76-2597-47d6-811b-f65cbe0580ff/volumes" Dec 03 12:54:53 crc kubenswrapper[5052]: I1203 12:54:53.695718 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="568addd3-4ad3-47c3-adae-693691e7cd40" path="/var/lib/kubelet/pods/568addd3-4ad3-47c3-adae-693691e7cd40/volumes" Dec 03 12:54:53 crc kubenswrapper[5052]: I1203 12:54:53.696912 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="79efd225-5ce4-428f-aef9-3f4884931fef" path="/var/lib/kubelet/pods/79efd225-5ce4-428f-aef9-3f4884931fef/volumes" Dec 03 12:54:53 crc kubenswrapper[5052]: I1203 12:54:53.698361 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7d297f47-a73f-4a88-bba0-3108deb8df4d" path="/var/lib/kubelet/pods/7d297f47-a73f-4a88-bba0-3108deb8df4d/volumes" Dec 03 12:54:53 crc kubenswrapper[5052]: I1203 12:54:53.699010 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8df65ee4-b91d-4ffd-bff2-65e5e6559130" path="/var/lib/kubelet/pods/8df65ee4-b91d-4ffd-bff2-65e5e6559130/volumes" Dec 03 12:54:53 crc kubenswrapper[5052]: I1203 12:54:53.699561 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="941d063f-c834-47a1-bc0c-2096b005e09c" path="/var/lib/kubelet/pods/941d063f-c834-47a1-bc0c-2096b005e09c/volumes" Dec 03 12:54:53 crc kubenswrapper[5052]: I1203 12:54:53.700880 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="981c3156-805a-44d1-b4ec-f688a8bc3cf8" path="/var/lib/kubelet/pods/981c3156-805a-44d1-b4ec-f688a8bc3cf8/volumes" Dec 03 12:54:53 crc kubenswrapper[5052]: I1203 12:54:53.701906 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b3363d6f-656e-4e22-a3eb-b1016380a0c8" path="/var/lib/kubelet/pods/b3363d6f-656e-4e22-a3eb-b1016380a0c8/volumes" Dec 03 12:54:53 crc kubenswrapper[5052]: I1203 12:54:53.703575 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c4d5c183-8df4-4c43-9afb-2b28c6aae9e1" path="/var/lib/kubelet/pods/c4d5c183-8df4-4c43-9afb-2b28c6aae9e1/volumes" Dec 03 12:54:53 crc kubenswrapper[5052]: I1203 12:54:53.704696 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cfc68960-6006-486c-bc03-848875d3bf5b" path="/var/lib/kubelet/pods/cfc68960-6006-486c-bc03-848875d3bf5b/volumes" Dec 03 12:54:53 crc kubenswrapper[5052]: I1203 12:54:53.705516 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d1ff5100-3ac7-4633-811c-ea470242d858" path="/var/lib/kubelet/pods/d1ff5100-3ac7-4633-811c-ea470242d858/volumes" Dec 03 12:54:53 crc kubenswrapper[5052]: I1203 12:54:53.705702 5052 scope.go:117] "RemoveContainer" containerID="b7a1da6db38db2f0ed2c6f73c89998af0cc42716d17e3513590eabdf3cc5aa71" Dec 03 12:54:53 crc kubenswrapper[5052]: I1203 12:54:53.706523 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d9bf91f1-099c-4e72-97cf-1828241aeaac" path="/var/lib/kubelet/pods/d9bf91f1-099c-4e72-97cf-1828241aeaac/volumes" Dec 03 12:54:53 crc kubenswrapper[5052]: I1203 12:54:53.707053 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e2225738-b0d1-4306-bbee-a61dd9558c80" path="/var/lib/kubelet/pods/e2225738-b0d1-4306-bbee-a61dd9558c80/volumes" Dec 03 12:54:53 crc kubenswrapper[5052]: I1203 12:54:53.708454 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e73813f5-f97f-41dd-8a79-6196557d4d1a" path="/var/lib/kubelet/pods/e73813f5-f97f-41dd-8a79-6196557d4d1a/volumes" Dec 03 12:54:53 crc kubenswrapper[5052]: I1203 12:54:53.709945 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e8a26759-2926-41cf-a1ec-286fc9962bb3" path="/var/lib/kubelet/pods/e8a26759-2926-41cf-a1ec-286fc9962bb3/volumes" Dec 03 12:54:53 crc kubenswrapper[5052]: I1203 12:54:53.728990 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/novacell02629-account-delete-wsshh" Dec 03 12:54:53 crc kubenswrapper[5052]: I1203 12:54:53.728991 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/novacell02629-account-delete-wsshh" event={"ID":"1b7ef5eb-406e-4e2c-9d9c-d763919f56fd","Type":"ContainerDied","Data":"b90af27119e2a44164e92cfa6f8e83a087021fe45af6e255a1db739e22bee63b"} Dec 03 12:54:53 crc kubenswrapper[5052]: I1203 12:54:53.745172 5052 scope.go:117] "RemoveContainer" containerID="c9f66104fbca885ffe17ff3f4d917326325460800a0171775b450385bf7d5784" Dec 03 12:54:53 crc kubenswrapper[5052]: I1203 12:54:53.816016 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/novacell02629-account-delete-wsshh"] Dec 03 12:54:53 crc kubenswrapper[5052]: I1203 12:54:53.823826 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/novacell02629-account-delete-wsshh"] Dec 03 12:54:54 crc kubenswrapper[5052]: I1203 12:54:54.103927 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-shdj6" Dec 03 12:54:54 crc kubenswrapper[5052]: I1203 12:54:54.157966 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-85c5f46bf7-pqjs4"] Dec 03 12:54:54 crc kubenswrapper[5052]: I1203 12:54:54.158192 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-85c5f46bf7-pqjs4" podUID="1cf6bdc8-8a27-4cc6-a2e9-f6de73fc1ea6" containerName="dnsmasq-dns" containerID="cri-o://7764c0629dd26dc1a0c178458e4042481ac430367224d3fc71d16036893f0163" gracePeriod=10 Dec 03 12:54:54 crc kubenswrapper[5052]: I1203 12:54:54.685294 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/barbican-api-76dc68d888-zmmcs" podUID="25eead1e-d17c-4b36-8a9f-ef563189d53b" containerName="barbican-api" probeResult="failure" output="Get \"https://10.217.0.181:9311/healthcheck\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Dec 03 12:54:54 crc kubenswrapper[5052]: I1203 12:54:54.685438 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/barbican-api-76dc68d888-zmmcs" podUID="25eead1e-d17c-4b36-8a9f-ef563189d53b" containerName="barbican-api-log" probeResult="failure" output="Get \"https://10.217.0.181:9311/healthcheck\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Dec 03 12:54:54 crc kubenswrapper[5052]: I1203 12:54:54.766745 5052 generic.go:334] "Generic (PLEG): container finished" podID="1cf6bdc8-8a27-4cc6-a2e9-f6de73fc1ea6" containerID="7764c0629dd26dc1a0c178458e4042481ac430367224d3fc71d16036893f0163" exitCode=0 Dec 03 12:54:54 crc kubenswrapper[5052]: I1203 12:54:54.766836 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-85c5f46bf7-pqjs4" event={"ID":"1cf6bdc8-8a27-4cc6-a2e9-f6de73fc1ea6","Type":"ContainerDied","Data":"7764c0629dd26dc1a0c178458e4042481ac430367224d3fc71d16036893f0163"} Dec 03 12:54:55 crc kubenswrapper[5052]: I1203 12:54:55.105318 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-85c5f46bf7-pqjs4" Dec 03 12:54:55 crc kubenswrapper[5052]: I1203 12:54:55.244229 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1cf6bdc8-8a27-4cc6-a2e9-f6de73fc1ea6-config\") pod \"1cf6bdc8-8a27-4cc6-a2e9-f6de73fc1ea6\" (UID: \"1cf6bdc8-8a27-4cc6-a2e9-f6de73fc1ea6\") " Dec 03 12:54:55 crc kubenswrapper[5052]: I1203 12:54:55.244312 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"dns-swift-storage-0\" (UniqueName: \"kubernetes.io/configmap/1cf6bdc8-8a27-4cc6-a2e9-f6de73fc1ea6-dns-swift-storage-0\") pod \"1cf6bdc8-8a27-4cc6-a2e9-f6de73fc1ea6\" (UID: \"1cf6bdc8-8a27-4cc6-a2e9-f6de73fc1ea6\") " Dec 03 12:54:55 crc kubenswrapper[5052]: I1203 12:54:55.244375 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/1cf6bdc8-8a27-4cc6-a2e9-f6de73fc1ea6-dnsmasq-svc\") pod \"1cf6bdc8-8a27-4cc6-a2e9-f6de73fc1ea6\" (UID: \"1cf6bdc8-8a27-4cc6-a2e9-f6de73fc1ea6\") " Dec 03 12:54:55 crc kubenswrapper[5052]: I1203 12:54:55.244564 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nplrw\" (UniqueName: \"kubernetes.io/projected/1cf6bdc8-8a27-4cc6-a2e9-f6de73fc1ea6-kube-api-access-nplrw\") pod \"1cf6bdc8-8a27-4cc6-a2e9-f6de73fc1ea6\" (UID: \"1cf6bdc8-8a27-4cc6-a2e9-f6de73fc1ea6\") " Dec 03 12:54:55 crc kubenswrapper[5052]: I1203 12:54:55.250556 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1cf6bdc8-8a27-4cc6-a2e9-f6de73fc1ea6-kube-api-access-nplrw" (OuterVolumeSpecName: "kube-api-access-nplrw") pod "1cf6bdc8-8a27-4cc6-a2e9-f6de73fc1ea6" (UID: "1cf6bdc8-8a27-4cc6-a2e9-f6de73fc1ea6"). InnerVolumeSpecName "kube-api-access-nplrw". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:54:55 crc kubenswrapper[5052]: I1203 12:54:55.287437 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1cf6bdc8-8a27-4cc6-a2e9-f6de73fc1ea6-dnsmasq-svc" (OuterVolumeSpecName: "dnsmasq-svc") pod "1cf6bdc8-8a27-4cc6-a2e9-f6de73fc1ea6" (UID: "1cf6bdc8-8a27-4cc6-a2e9-f6de73fc1ea6"). InnerVolumeSpecName "dnsmasq-svc". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:54:55 crc kubenswrapper[5052]: I1203 12:54:55.289792 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1cf6bdc8-8a27-4cc6-a2e9-f6de73fc1ea6-dns-swift-storage-0" (OuterVolumeSpecName: "dns-swift-storage-0") pod "1cf6bdc8-8a27-4cc6-a2e9-f6de73fc1ea6" (UID: "1cf6bdc8-8a27-4cc6-a2e9-f6de73fc1ea6"). InnerVolumeSpecName "dns-swift-storage-0". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:54:55 crc kubenswrapper[5052]: I1203 12:54:55.299763 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1cf6bdc8-8a27-4cc6-a2e9-f6de73fc1ea6-config" (OuterVolumeSpecName: "config") pod "1cf6bdc8-8a27-4cc6-a2e9-f6de73fc1ea6" (UID: "1cf6bdc8-8a27-4cc6-a2e9-f6de73fc1ea6"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:54:55 crc kubenswrapper[5052]: I1203 12:54:55.347151 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-nplrw\" (UniqueName: \"kubernetes.io/projected/1cf6bdc8-8a27-4cc6-a2e9-f6de73fc1ea6-kube-api-access-nplrw\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:55 crc kubenswrapper[5052]: I1203 12:54:55.347205 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1cf6bdc8-8a27-4cc6-a2e9-f6de73fc1ea6-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:55 crc kubenswrapper[5052]: I1203 12:54:55.347230 5052 reconciler_common.go:293] "Volume detached for volume \"dns-swift-storage-0\" (UniqueName: \"kubernetes.io/configmap/1cf6bdc8-8a27-4cc6-a2e9-f6de73fc1ea6-dns-swift-storage-0\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:55 crc kubenswrapper[5052]: I1203 12:54:55.347250 5052 reconciler_common.go:293] "Volume detached for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/1cf6bdc8-8a27-4cc6-a2e9-f6de73fc1ea6-dnsmasq-svc\") on node \"crc\" DevicePath \"\"" Dec 03 12:54:55 crc kubenswrapper[5052]: I1203 12:54:55.684308 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1b7ef5eb-406e-4e2c-9d9c-d763919f56fd" path="/var/lib/kubelet/pods/1b7ef5eb-406e-4e2c-9d9c-d763919f56fd/volumes" Dec 03 12:54:55 crc kubenswrapper[5052]: I1203 12:54:55.779589 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-85c5f46bf7-pqjs4" event={"ID":"1cf6bdc8-8a27-4cc6-a2e9-f6de73fc1ea6","Type":"ContainerDied","Data":"316b115f05fd73e9974f2538b9459c33cf0d446a63b85964b862e1f96aed1152"} Dec 03 12:54:55 crc kubenswrapper[5052]: I1203 12:54:55.779626 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-85c5f46bf7-pqjs4" Dec 03 12:54:55 crc kubenswrapper[5052]: I1203 12:54:55.779643 5052 scope.go:117] "RemoveContainer" containerID="7764c0629dd26dc1a0c178458e4042481ac430367224d3fc71d16036893f0163" Dec 03 12:54:55 crc kubenswrapper[5052]: I1203 12:54:55.801002 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-85c5f46bf7-pqjs4"] Dec 03 12:54:55 crc kubenswrapper[5052]: I1203 12:54:55.809467 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-85c5f46bf7-pqjs4"] Dec 03 12:54:55 crc kubenswrapper[5052]: I1203 12:54:55.811421 5052 scope.go:117] "RemoveContainer" containerID="07e3ceee7a3a89781a782a0bbd0e4e55633ebaf42ef3fc33c4f41527c0bc4140" Dec 03 12:54:57 crc kubenswrapper[5052]: I1203 12:54:57.685561 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1cf6bdc8-8a27-4cc6-a2e9-f6de73fc1ea6" path="/var/lib/kubelet/pods/1cf6bdc8-8a27-4cc6-a2e9-f6de73fc1ea6/volumes" Dec 03 12:54:59 crc kubenswrapper[5052]: I1203 12:54:59.379896 5052 patch_prober.go:28] interesting pod/machine-config-daemon-74bj6 container/machine-config-daemon namespace/openshift-machine-config-operator: Liveness probe status=failure output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" start-of-body= Dec 03 12:54:59 crc kubenswrapper[5052]: I1203 12:54:59.380260 5052 prober.go:107] "Probe failed" probeType="Liveness" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" containerName="machine-config-daemon" probeResult="failure" output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" Dec 03 12:54:59 crc kubenswrapper[5052]: I1203 12:54:59.380328 5052 kubelet.go:2542] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" Dec 03 12:54:59 crc kubenswrapper[5052]: I1203 12:54:59.381090 5052 kuberuntime_manager.go:1027] "Message for Container of pod" containerName="machine-config-daemon" containerStatusID={"Type":"cri-o","ID":"69fa7fbcd2aceb7c53703a19508876f93a97dc9adabb79c16fb691d90f606d58"} pod="openshift-machine-config-operator/machine-config-daemon-74bj6" containerMessage="Container machine-config-daemon failed liveness probe, will be restarted" Dec 03 12:54:59 crc kubenswrapper[5052]: I1203 12:54:59.381155 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" containerName="machine-config-daemon" containerID="cri-o://69fa7fbcd2aceb7c53703a19508876f93a97dc9adabb79c16fb691d90f606d58" gracePeriod=600 Dec 03 12:54:59 crc kubenswrapper[5052]: E1203 12:54:59.502890 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 12:54:59 crc kubenswrapper[5052]: I1203 12:54:59.847916 5052 generic.go:334] "Generic (PLEG): container finished" podID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" containerID="69fa7fbcd2aceb7c53703a19508876f93a97dc9adabb79c16fb691d90f606d58" exitCode=0 Dec 03 12:54:59 crc kubenswrapper[5052]: I1203 12:54:59.847968 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" event={"ID":"10c237bc-72e7-4da3-9f2c-b1688395bf6e","Type":"ContainerDied","Data":"69fa7fbcd2aceb7c53703a19508876f93a97dc9adabb79c16fb691d90f606d58"} Dec 03 12:54:59 crc kubenswrapper[5052]: I1203 12:54:59.848008 5052 scope.go:117] "RemoveContainer" containerID="75d5669ef68596efd94ee7584a4e5e570e7db6cc9f424db1fa9cd7f60acbab7b" Dec 03 12:54:59 crc kubenswrapper[5052]: I1203 12:54:59.848774 5052 scope.go:117] "RemoveContainer" containerID="69fa7fbcd2aceb7c53703a19508876f93a97dc9adabb79c16fb691d90f606d58" Dec 03 12:54:59 crc kubenswrapper[5052]: E1203 12:54:59.851742 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 12:55:13 crc kubenswrapper[5052]: I1203 12:55:13.673050 5052 scope.go:117] "RemoveContainer" containerID="69fa7fbcd2aceb7c53703a19508876f93a97dc9adabb79c16fb691d90f606d58" Dec 03 12:55:13 crc kubenswrapper[5052]: E1203 12:55:13.673883 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 12:55:13 crc kubenswrapper[5052]: I1203 12:55:13.934972 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:55:13 crc kubenswrapper[5052]: I1203 12:55:13.990529 5052 generic.go:334] "Generic (PLEG): container finished" podID="5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4" containerID="5f94534eaf62ac523a8208a315b484dcd4fe50a4df83c2d6cfacb2dcdfd740c6" exitCode=137 Dec 03 12:55:13 crc kubenswrapper[5052]: I1203 12:55:13.990550 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4","Type":"ContainerDied","Data":"5f94534eaf62ac523a8208a315b484dcd4fe50a4df83c2d6cfacb2dcdfd740c6"} Dec 03 12:55:13 crc kubenswrapper[5052]: I1203 12:55:13.990608 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/swift-storage-0" event={"ID":"5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4","Type":"ContainerDied","Data":"341fe1e40774ad3e533141e2a85fa7ab7d9d5691734e70031925b6befaf69079"} Dec 03 12:55:13 crc kubenswrapper[5052]: I1203 12:55:13.990628 5052 scope.go:117] "RemoveContainer" containerID="5f94534eaf62ac523a8208a315b484dcd4fe50a4df83c2d6cfacb2dcdfd740c6" Dec 03 12:55:13 crc kubenswrapper[5052]: I1203 12:55:13.990647 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/swift-storage-0" Dec 03 12:55:14 crc kubenswrapper[5052]: I1203 12:55:14.011255 5052 scope.go:117] "RemoveContainer" containerID="7bdd0d115f77d5d7457272b4632a96a1cbaef3a87e7a1296d68acdce465508f1" Dec 03 12:55:14 crc kubenswrapper[5052]: I1203 12:55:14.017834 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"lock\" (UniqueName: \"kubernetes.io/empty-dir/5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4-lock\") pod \"5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4\" (UID: \"5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4\") " Dec 03 12:55:14 crc kubenswrapper[5052]: I1203 12:55:14.018019 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"swift\" (UniqueName: \"kubernetes.io/local-volume/local-storage10-crc\") pod \"5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4\" (UID: \"5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4\") " Dec 03 12:55:14 crc kubenswrapper[5052]: I1203 12:55:14.018142 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4-etc-swift\") pod \"5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4\" (UID: \"5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4\") " Dec 03 12:55:14 crc kubenswrapper[5052]: I1203 12:55:14.018248 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gp5lf\" (UniqueName: \"kubernetes.io/projected/5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4-kube-api-access-gp5lf\") pod \"5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4\" (UID: \"5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4\") " Dec 03 12:55:14 crc kubenswrapper[5052]: I1203 12:55:14.018345 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"cache\" (UniqueName: \"kubernetes.io/empty-dir/5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4-cache\") pod \"5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4\" (UID: \"5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4\") " Dec 03 12:55:14 crc kubenswrapper[5052]: I1203 12:55:14.018941 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4-cache" (OuterVolumeSpecName: "cache") pod "5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4" (UID: "5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4"). InnerVolumeSpecName "cache". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:55:14 crc kubenswrapper[5052]: I1203 12:55:14.019106 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4-lock" (OuterVolumeSpecName: "lock") pod "5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4" (UID: "5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4"). InnerVolumeSpecName "lock". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:55:14 crc kubenswrapper[5052]: I1203 12:55:14.029301 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4-kube-api-access-gp5lf" (OuterVolumeSpecName: "kube-api-access-gp5lf") pod "5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4" (UID: "5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4"). InnerVolumeSpecName "kube-api-access-gp5lf". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:55:14 crc kubenswrapper[5052]: I1203 12:55:14.030249 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage10-crc" (OuterVolumeSpecName: "swift") pod "5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4" (UID: "5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4"). InnerVolumeSpecName "local-storage10-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 12:55:14 crc kubenswrapper[5052]: I1203 12:55:14.031697 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4-etc-swift" (OuterVolumeSpecName: "etc-swift") pod "5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4" (UID: "5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4"). InnerVolumeSpecName "etc-swift". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:55:14 crc kubenswrapper[5052]: I1203 12:55:14.047869 5052 scope.go:117] "RemoveContainer" containerID="62e46329d5f90f5463064339e9bbe330b943da63b752db809975d250c3c5e1ba" Dec 03 12:55:14 crc kubenswrapper[5052]: I1203 12:55:14.096335 5052 scope.go:117] "RemoveContainer" containerID="71586c32f42d5da42bcbffc73b3fed04e796afb63a8b7d88e60890c23b73e5c0" Dec 03 12:55:14 crc kubenswrapper[5052]: I1203 12:55:14.113943 5052 scope.go:117] "RemoveContainer" containerID="44f9d9990406d3045f888637bfcfe158544e8d9bceeb3e129858be5d63d5b0ef" Dec 03 12:55:14 crc kubenswrapper[5052]: I1203 12:55:14.119852 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage10-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage10-crc\") on node \"crc\" " Dec 03 12:55:14 crc kubenswrapper[5052]: I1203 12:55:14.119902 5052 reconciler_common.go:293] "Volume detached for volume \"etc-swift\" (UniqueName: \"kubernetes.io/projected/5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4-etc-swift\") on node \"crc\" DevicePath \"\"" Dec 03 12:55:14 crc kubenswrapper[5052]: I1203 12:55:14.119920 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-gp5lf\" (UniqueName: \"kubernetes.io/projected/5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4-kube-api-access-gp5lf\") on node \"crc\" DevicePath \"\"" Dec 03 12:55:14 crc kubenswrapper[5052]: I1203 12:55:14.119935 5052 reconciler_common.go:293] "Volume detached for volume \"cache\" (UniqueName: \"kubernetes.io/empty-dir/5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4-cache\") on node \"crc\" DevicePath \"\"" Dec 03 12:55:14 crc kubenswrapper[5052]: I1203 12:55:14.119947 5052 reconciler_common.go:293] "Volume detached for volume \"lock\" (UniqueName: \"kubernetes.io/empty-dir/5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4-lock\") on node \"crc\" DevicePath \"\"" Dec 03 12:55:14 crc kubenswrapper[5052]: I1203 12:55:14.134745 5052 scope.go:117] "RemoveContainer" containerID="b47992e8799bd9270966dcef43cbed9c315f16df636f79081295a474d02ec3b9" Dec 03 12:55:14 crc kubenswrapper[5052]: I1203 12:55:14.135722 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage10-crc" (UniqueName: "kubernetes.io/local-volume/local-storage10-crc") on node "crc" Dec 03 12:55:14 crc kubenswrapper[5052]: I1203 12:55:14.156919 5052 scope.go:117] "RemoveContainer" containerID="1087472cb2f4b1be538bcf920658b4a2f73a08130ae74a54e44b3322500d6407" Dec 03 12:55:14 crc kubenswrapper[5052]: I1203 12:55:14.174052 5052 scope.go:117] "RemoveContainer" containerID="5e0e53488533218e7d56486c1da9b5627e233ae560832e3a38ad6cb8cf881504" Dec 03 12:55:14 crc kubenswrapper[5052]: I1203 12:55:14.189509 5052 scope.go:117] "RemoveContainer" containerID="79c2a3a0d0be04c4ece62e264a2a1afc1ddae2cbe0714bda786215f70574370c" Dec 03 12:55:14 crc kubenswrapper[5052]: I1203 12:55:14.203948 5052 scope.go:117] "RemoveContainer" containerID="e9ef673f7a7a6d771b2dac62256d6ff0ffb0a20ed29ae2461ff57a96282f22a4" Dec 03 12:55:14 crc kubenswrapper[5052]: I1203 12:55:14.218454 5052 scope.go:117] "RemoveContainer" containerID="722f9b2f73e09eeb2d10bcd607cd1d2c2250a2aefb6a7b7fc5b762f09df66896" Dec 03 12:55:14 crc kubenswrapper[5052]: I1203 12:55:14.220834 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage10-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage10-crc\") on node \"crc\" DevicePath \"\"" Dec 03 12:55:14 crc kubenswrapper[5052]: I1203 12:55:14.234064 5052 scope.go:117] "RemoveContainer" containerID="93e148f1fa8fb3a45b149f7c61848865b732bc5668fbd9e528494e58a56b6237" Dec 03 12:55:14 crc kubenswrapper[5052]: I1203 12:55:14.250336 5052 scope.go:117] "RemoveContainer" containerID="7cd62df542d638c1af171b7f7bdfb4bfa0fa061cd5e93db34b71eef1fee65081" Dec 03 12:55:14 crc kubenswrapper[5052]: I1203 12:55:14.267357 5052 scope.go:117] "RemoveContainer" containerID="5606a2414fb1d7623fb77afb101b103f800b56ca86bd25e193447a537a91edbe" Dec 03 12:55:14 crc kubenswrapper[5052]: I1203 12:55:14.284616 5052 scope.go:117] "RemoveContainer" containerID="f162a9aaff5508763fc99c270ac25a9155335aac10652767b1730627c88c0d13" Dec 03 12:55:14 crc kubenswrapper[5052]: I1203 12:55:14.306703 5052 scope.go:117] "RemoveContainer" containerID="5f94534eaf62ac523a8208a315b484dcd4fe50a4df83c2d6cfacb2dcdfd740c6" Dec 03 12:55:14 crc kubenswrapper[5052]: E1203 12:55:14.307307 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5f94534eaf62ac523a8208a315b484dcd4fe50a4df83c2d6cfacb2dcdfd740c6\": container with ID starting with 5f94534eaf62ac523a8208a315b484dcd4fe50a4df83c2d6cfacb2dcdfd740c6 not found: ID does not exist" containerID="5f94534eaf62ac523a8208a315b484dcd4fe50a4df83c2d6cfacb2dcdfd740c6" Dec 03 12:55:14 crc kubenswrapper[5052]: I1203 12:55:14.307342 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5f94534eaf62ac523a8208a315b484dcd4fe50a4df83c2d6cfacb2dcdfd740c6"} err="failed to get container status \"5f94534eaf62ac523a8208a315b484dcd4fe50a4df83c2d6cfacb2dcdfd740c6\": rpc error: code = NotFound desc = could not find container \"5f94534eaf62ac523a8208a315b484dcd4fe50a4df83c2d6cfacb2dcdfd740c6\": container with ID starting with 5f94534eaf62ac523a8208a315b484dcd4fe50a4df83c2d6cfacb2dcdfd740c6 not found: ID does not exist" Dec 03 12:55:14 crc kubenswrapper[5052]: I1203 12:55:14.307367 5052 scope.go:117] "RemoveContainer" containerID="7bdd0d115f77d5d7457272b4632a96a1cbaef3a87e7a1296d68acdce465508f1" Dec 03 12:55:14 crc kubenswrapper[5052]: E1203 12:55:14.308130 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7bdd0d115f77d5d7457272b4632a96a1cbaef3a87e7a1296d68acdce465508f1\": container with ID starting with 7bdd0d115f77d5d7457272b4632a96a1cbaef3a87e7a1296d68acdce465508f1 not found: ID does not exist" containerID="7bdd0d115f77d5d7457272b4632a96a1cbaef3a87e7a1296d68acdce465508f1" Dec 03 12:55:14 crc kubenswrapper[5052]: I1203 12:55:14.308197 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7bdd0d115f77d5d7457272b4632a96a1cbaef3a87e7a1296d68acdce465508f1"} err="failed to get container status \"7bdd0d115f77d5d7457272b4632a96a1cbaef3a87e7a1296d68acdce465508f1\": rpc error: code = NotFound desc = could not find container \"7bdd0d115f77d5d7457272b4632a96a1cbaef3a87e7a1296d68acdce465508f1\": container with ID starting with 7bdd0d115f77d5d7457272b4632a96a1cbaef3a87e7a1296d68acdce465508f1 not found: ID does not exist" Dec 03 12:55:14 crc kubenswrapper[5052]: I1203 12:55:14.308244 5052 scope.go:117] "RemoveContainer" containerID="62e46329d5f90f5463064339e9bbe330b943da63b752db809975d250c3c5e1ba" Dec 03 12:55:14 crc kubenswrapper[5052]: E1203 12:55:14.308970 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"62e46329d5f90f5463064339e9bbe330b943da63b752db809975d250c3c5e1ba\": container with ID starting with 62e46329d5f90f5463064339e9bbe330b943da63b752db809975d250c3c5e1ba not found: ID does not exist" containerID="62e46329d5f90f5463064339e9bbe330b943da63b752db809975d250c3c5e1ba" Dec 03 12:55:14 crc kubenswrapper[5052]: I1203 12:55:14.309009 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"62e46329d5f90f5463064339e9bbe330b943da63b752db809975d250c3c5e1ba"} err="failed to get container status \"62e46329d5f90f5463064339e9bbe330b943da63b752db809975d250c3c5e1ba\": rpc error: code = NotFound desc = could not find container \"62e46329d5f90f5463064339e9bbe330b943da63b752db809975d250c3c5e1ba\": container with ID starting with 62e46329d5f90f5463064339e9bbe330b943da63b752db809975d250c3c5e1ba not found: ID does not exist" Dec 03 12:55:14 crc kubenswrapper[5052]: I1203 12:55:14.309032 5052 scope.go:117] "RemoveContainer" containerID="71586c32f42d5da42bcbffc73b3fed04e796afb63a8b7d88e60890c23b73e5c0" Dec 03 12:55:14 crc kubenswrapper[5052]: E1203 12:55:14.309688 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"71586c32f42d5da42bcbffc73b3fed04e796afb63a8b7d88e60890c23b73e5c0\": container with ID starting with 71586c32f42d5da42bcbffc73b3fed04e796afb63a8b7d88e60890c23b73e5c0 not found: ID does not exist" containerID="71586c32f42d5da42bcbffc73b3fed04e796afb63a8b7d88e60890c23b73e5c0" Dec 03 12:55:14 crc kubenswrapper[5052]: I1203 12:55:14.309753 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"71586c32f42d5da42bcbffc73b3fed04e796afb63a8b7d88e60890c23b73e5c0"} err="failed to get container status \"71586c32f42d5da42bcbffc73b3fed04e796afb63a8b7d88e60890c23b73e5c0\": rpc error: code = NotFound desc = could not find container \"71586c32f42d5da42bcbffc73b3fed04e796afb63a8b7d88e60890c23b73e5c0\": container with ID starting with 71586c32f42d5da42bcbffc73b3fed04e796afb63a8b7d88e60890c23b73e5c0 not found: ID does not exist" Dec 03 12:55:14 crc kubenswrapper[5052]: I1203 12:55:14.309792 5052 scope.go:117] "RemoveContainer" containerID="44f9d9990406d3045f888637bfcfe158544e8d9bceeb3e129858be5d63d5b0ef" Dec 03 12:55:14 crc kubenswrapper[5052]: E1203 12:55:14.310200 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"44f9d9990406d3045f888637bfcfe158544e8d9bceeb3e129858be5d63d5b0ef\": container with ID starting with 44f9d9990406d3045f888637bfcfe158544e8d9bceeb3e129858be5d63d5b0ef not found: ID does not exist" containerID="44f9d9990406d3045f888637bfcfe158544e8d9bceeb3e129858be5d63d5b0ef" Dec 03 12:55:14 crc kubenswrapper[5052]: I1203 12:55:14.310224 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"44f9d9990406d3045f888637bfcfe158544e8d9bceeb3e129858be5d63d5b0ef"} err="failed to get container status \"44f9d9990406d3045f888637bfcfe158544e8d9bceeb3e129858be5d63d5b0ef\": rpc error: code = NotFound desc = could not find container \"44f9d9990406d3045f888637bfcfe158544e8d9bceeb3e129858be5d63d5b0ef\": container with ID starting with 44f9d9990406d3045f888637bfcfe158544e8d9bceeb3e129858be5d63d5b0ef not found: ID does not exist" Dec 03 12:55:14 crc kubenswrapper[5052]: I1203 12:55:14.310240 5052 scope.go:117] "RemoveContainer" containerID="b47992e8799bd9270966dcef43cbed9c315f16df636f79081295a474d02ec3b9" Dec 03 12:55:14 crc kubenswrapper[5052]: E1203 12:55:14.310698 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b47992e8799bd9270966dcef43cbed9c315f16df636f79081295a474d02ec3b9\": container with ID starting with b47992e8799bd9270966dcef43cbed9c315f16df636f79081295a474d02ec3b9 not found: ID does not exist" containerID="b47992e8799bd9270966dcef43cbed9c315f16df636f79081295a474d02ec3b9" Dec 03 12:55:14 crc kubenswrapper[5052]: I1203 12:55:14.310742 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b47992e8799bd9270966dcef43cbed9c315f16df636f79081295a474d02ec3b9"} err="failed to get container status \"b47992e8799bd9270966dcef43cbed9c315f16df636f79081295a474d02ec3b9\": rpc error: code = NotFound desc = could not find container \"b47992e8799bd9270966dcef43cbed9c315f16df636f79081295a474d02ec3b9\": container with ID starting with b47992e8799bd9270966dcef43cbed9c315f16df636f79081295a474d02ec3b9 not found: ID does not exist" Dec 03 12:55:14 crc kubenswrapper[5052]: I1203 12:55:14.310762 5052 scope.go:117] "RemoveContainer" containerID="1087472cb2f4b1be538bcf920658b4a2f73a08130ae74a54e44b3322500d6407" Dec 03 12:55:14 crc kubenswrapper[5052]: E1203 12:55:14.311072 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1087472cb2f4b1be538bcf920658b4a2f73a08130ae74a54e44b3322500d6407\": container with ID starting with 1087472cb2f4b1be538bcf920658b4a2f73a08130ae74a54e44b3322500d6407 not found: ID does not exist" containerID="1087472cb2f4b1be538bcf920658b4a2f73a08130ae74a54e44b3322500d6407" Dec 03 12:55:14 crc kubenswrapper[5052]: I1203 12:55:14.311100 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1087472cb2f4b1be538bcf920658b4a2f73a08130ae74a54e44b3322500d6407"} err="failed to get container status \"1087472cb2f4b1be538bcf920658b4a2f73a08130ae74a54e44b3322500d6407\": rpc error: code = NotFound desc = could not find container \"1087472cb2f4b1be538bcf920658b4a2f73a08130ae74a54e44b3322500d6407\": container with ID starting with 1087472cb2f4b1be538bcf920658b4a2f73a08130ae74a54e44b3322500d6407 not found: ID does not exist" Dec 03 12:55:14 crc kubenswrapper[5052]: I1203 12:55:14.311113 5052 scope.go:117] "RemoveContainer" containerID="5e0e53488533218e7d56486c1da9b5627e233ae560832e3a38ad6cb8cf881504" Dec 03 12:55:14 crc kubenswrapper[5052]: E1203 12:55:14.311402 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5e0e53488533218e7d56486c1da9b5627e233ae560832e3a38ad6cb8cf881504\": container with ID starting with 5e0e53488533218e7d56486c1da9b5627e233ae560832e3a38ad6cb8cf881504 not found: ID does not exist" containerID="5e0e53488533218e7d56486c1da9b5627e233ae560832e3a38ad6cb8cf881504" Dec 03 12:55:14 crc kubenswrapper[5052]: I1203 12:55:14.311430 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5e0e53488533218e7d56486c1da9b5627e233ae560832e3a38ad6cb8cf881504"} err="failed to get container status \"5e0e53488533218e7d56486c1da9b5627e233ae560832e3a38ad6cb8cf881504\": rpc error: code = NotFound desc = could not find container \"5e0e53488533218e7d56486c1da9b5627e233ae560832e3a38ad6cb8cf881504\": container with ID starting with 5e0e53488533218e7d56486c1da9b5627e233ae560832e3a38ad6cb8cf881504 not found: ID does not exist" Dec 03 12:55:14 crc kubenswrapper[5052]: I1203 12:55:14.311446 5052 scope.go:117] "RemoveContainer" containerID="79c2a3a0d0be04c4ece62e264a2a1afc1ddae2cbe0714bda786215f70574370c" Dec 03 12:55:14 crc kubenswrapper[5052]: E1203 12:55:14.311868 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"79c2a3a0d0be04c4ece62e264a2a1afc1ddae2cbe0714bda786215f70574370c\": container with ID starting with 79c2a3a0d0be04c4ece62e264a2a1afc1ddae2cbe0714bda786215f70574370c not found: ID does not exist" containerID="79c2a3a0d0be04c4ece62e264a2a1afc1ddae2cbe0714bda786215f70574370c" Dec 03 12:55:14 crc kubenswrapper[5052]: I1203 12:55:14.311894 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"79c2a3a0d0be04c4ece62e264a2a1afc1ddae2cbe0714bda786215f70574370c"} err="failed to get container status \"79c2a3a0d0be04c4ece62e264a2a1afc1ddae2cbe0714bda786215f70574370c\": rpc error: code = NotFound desc = could not find container \"79c2a3a0d0be04c4ece62e264a2a1afc1ddae2cbe0714bda786215f70574370c\": container with ID starting with 79c2a3a0d0be04c4ece62e264a2a1afc1ddae2cbe0714bda786215f70574370c not found: ID does not exist" Dec 03 12:55:14 crc kubenswrapper[5052]: I1203 12:55:14.311910 5052 scope.go:117] "RemoveContainer" containerID="e9ef673f7a7a6d771b2dac62256d6ff0ffb0a20ed29ae2461ff57a96282f22a4" Dec 03 12:55:14 crc kubenswrapper[5052]: E1203 12:55:14.312184 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e9ef673f7a7a6d771b2dac62256d6ff0ffb0a20ed29ae2461ff57a96282f22a4\": container with ID starting with e9ef673f7a7a6d771b2dac62256d6ff0ffb0a20ed29ae2461ff57a96282f22a4 not found: ID does not exist" containerID="e9ef673f7a7a6d771b2dac62256d6ff0ffb0a20ed29ae2461ff57a96282f22a4" Dec 03 12:55:14 crc kubenswrapper[5052]: I1203 12:55:14.312209 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e9ef673f7a7a6d771b2dac62256d6ff0ffb0a20ed29ae2461ff57a96282f22a4"} err="failed to get container status \"e9ef673f7a7a6d771b2dac62256d6ff0ffb0a20ed29ae2461ff57a96282f22a4\": rpc error: code = NotFound desc = could not find container \"e9ef673f7a7a6d771b2dac62256d6ff0ffb0a20ed29ae2461ff57a96282f22a4\": container with ID starting with e9ef673f7a7a6d771b2dac62256d6ff0ffb0a20ed29ae2461ff57a96282f22a4 not found: ID does not exist" Dec 03 12:55:14 crc kubenswrapper[5052]: I1203 12:55:14.312225 5052 scope.go:117] "RemoveContainer" containerID="722f9b2f73e09eeb2d10bcd607cd1d2c2250a2aefb6a7b7fc5b762f09df66896" Dec 03 12:55:14 crc kubenswrapper[5052]: E1203 12:55:14.312470 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"722f9b2f73e09eeb2d10bcd607cd1d2c2250a2aefb6a7b7fc5b762f09df66896\": container with ID starting with 722f9b2f73e09eeb2d10bcd607cd1d2c2250a2aefb6a7b7fc5b762f09df66896 not found: ID does not exist" containerID="722f9b2f73e09eeb2d10bcd607cd1d2c2250a2aefb6a7b7fc5b762f09df66896" Dec 03 12:55:14 crc kubenswrapper[5052]: I1203 12:55:14.312498 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"722f9b2f73e09eeb2d10bcd607cd1d2c2250a2aefb6a7b7fc5b762f09df66896"} err="failed to get container status \"722f9b2f73e09eeb2d10bcd607cd1d2c2250a2aefb6a7b7fc5b762f09df66896\": rpc error: code = NotFound desc = could not find container \"722f9b2f73e09eeb2d10bcd607cd1d2c2250a2aefb6a7b7fc5b762f09df66896\": container with ID starting with 722f9b2f73e09eeb2d10bcd607cd1d2c2250a2aefb6a7b7fc5b762f09df66896 not found: ID does not exist" Dec 03 12:55:14 crc kubenswrapper[5052]: I1203 12:55:14.312541 5052 scope.go:117] "RemoveContainer" containerID="93e148f1fa8fb3a45b149f7c61848865b732bc5668fbd9e528494e58a56b6237" Dec 03 12:55:14 crc kubenswrapper[5052]: E1203 12:55:14.312818 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"93e148f1fa8fb3a45b149f7c61848865b732bc5668fbd9e528494e58a56b6237\": container with ID starting with 93e148f1fa8fb3a45b149f7c61848865b732bc5668fbd9e528494e58a56b6237 not found: ID does not exist" containerID="93e148f1fa8fb3a45b149f7c61848865b732bc5668fbd9e528494e58a56b6237" Dec 03 12:55:14 crc kubenswrapper[5052]: I1203 12:55:14.312844 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"93e148f1fa8fb3a45b149f7c61848865b732bc5668fbd9e528494e58a56b6237"} err="failed to get container status \"93e148f1fa8fb3a45b149f7c61848865b732bc5668fbd9e528494e58a56b6237\": rpc error: code = NotFound desc = could not find container \"93e148f1fa8fb3a45b149f7c61848865b732bc5668fbd9e528494e58a56b6237\": container with ID starting with 93e148f1fa8fb3a45b149f7c61848865b732bc5668fbd9e528494e58a56b6237 not found: ID does not exist" Dec 03 12:55:14 crc kubenswrapper[5052]: I1203 12:55:14.312864 5052 scope.go:117] "RemoveContainer" containerID="7cd62df542d638c1af171b7f7bdfb4bfa0fa061cd5e93db34b71eef1fee65081" Dec 03 12:55:14 crc kubenswrapper[5052]: E1203 12:55:14.313274 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7cd62df542d638c1af171b7f7bdfb4bfa0fa061cd5e93db34b71eef1fee65081\": container with ID starting with 7cd62df542d638c1af171b7f7bdfb4bfa0fa061cd5e93db34b71eef1fee65081 not found: ID does not exist" containerID="7cd62df542d638c1af171b7f7bdfb4bfa0fa061cd5e93db34b71eef1fee65081" Dec 03 12:55:14 crc kubenswrapper[5052]: I1203 12:55:14.313304 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7cd62df542d638c1af171b7f7bdfb4bfa0fa061cd5e93db34b71eef1fee65081"} err="failed to get container status \"7cd62df542d638c1af171b7f7bdfb4bfa0fa061cd5e93db34b71eef1fee65081\": rpc error: code = NotFound desc = could not find container \"7cd62df542d638c1af171b7f7bdfb4bfa0fa061cd5e93db34b71eef1fee65081\": container with ID starting with 7cd62df542d638c1af171b7f7bdfb4bfa0fa061cd5e93db34b71eef1fee65081 not found: ID does not exist" Dec 03 12:55:14 crc kubenswrapper[5052]: I1203 12:55:14.313322 5052 scope.go:117] "RemoveContainer" containerID="5606a2414fb1d7623fb77afb101b103f800b56ca86bd25e193447a537a91edbe" Dec 03 12:55:14 crc kubenswrapper[5052]: E1203 12:55:14.313576 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5606a2414fb1d7623fb77afb101b103f800b56ca86bd25e193447a537a91edbe\": container with ID starting with 5606a2414fb1d7623fb77afb101b103f800b56ca86bd25e193447a537a91edbe not found: ID does not exist" containerID="5606a2414fb1d7623fb77afb101b103f800b56ca86bd25e193447a537a91edbe" Dec 03 12:55:14 crc kubenswrapper[5052]: I1203 12:55:14.313602 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5606a2414fb1d7623fb77afb101b103f800b56ca86bd25e193447a537a91edbe"} err="failed to get container status \"5606a2414fb1d7623fb77afb101b103f800b56ca86bd25e193447a537a91edbe\": rpc error: code = NotFound desc = could not find container \"5606a2414fb1d7623fb77afb101b103f800b56ca86bd25e193447a537a91edbe\": container with ID starting with 5606a2414fb1d7623fb77afb101b103f800b56ca86bd25e193447a537a91edbe not found: ID does not exist" Dec 03 12:55:14 crc kubenswrapper[5052]: I1203 12:55:14.313622 5052 scope.go:117] "RemoveContainer" containerID="f162a9aaff5508763fc99c270ac25a9155335aac10652767b1730627c88c0d13" Dec 03 12:55:14 crc kubenswrapper[5052]: E1203 12:55:14.313863 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f162a9aaff5508763fc99c270ac25a9155335aac10652767b1730627c88c0d13\": container with ID starting with f162a9aaff5508763fc99c270ac25a9155335aac10652767b1730627c88c0d13 not found: ID does not exist" containerID="f162a9aaff5508763fc99c270ac25a9155335aac10652767b1730627c88c0d13" Dec 03 12:55:14 crc kubenswrapper[5052]: I1203 12:55:14.313894 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f162a9aaff5508763fc99c270ac25a9155335aac10652767b1730627c88c0d13"} err="failed to get container status \"f162a9aaff5508763fc99c270ac25a9155335aac10652767b1730627c88c0d13\": rpc error: code = NotFound desc = could not find container \"f162a9aaff5508763fc99c270ac25a9155335aac10652767b1730627c88c0d13\": container with ID starting with f162a9aaff5508763fc99c270ac25a9155335aac10652767b1730627c88c0d13 not found: ID does not exist" Dec 03 12:55:14 crc kubenswrapper[5052]: I1203 12:55:14.332086 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/swift-storage-0"] Dec 03 12:55:14 crc kubenswrapper[5052]: I1203 12:55:14.341184 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/swift-storage-0"] Dec 03 12:55:15 crc kubenswrapper[5052]: I1203 12:55:15.681941 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4" path="/var/lib/kubelet/pods/5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4/volumes" Dec 03 12:55:20 crc kubenswrapper[5052]: I1203 12:55:20.407951 5052 pod_container_manager_linux.go:210] "Failed to delete cgroup paths" cgroupName=["kubepods","besteffort","pod64ada63f-b17d-411b-a547-fa1c15107ab5"] err="unable to destroy cgroup paths for cgroup [kubepods besteffort pod64ada63f-b17d-411b-a547-fa1c15107ab5] : Timed out while waiting for systemd to remove kubepods-besteffort-pod64ada63f_b17d_411b_a547_fa1c15107ab5.slice" Dec 03 12:55:20 crc kubenswrapper[5052]: E1203 12:55:20.408207 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to delete cgroup paths for [kubepods besteffort pod64ada63f-b17d-411b-a547-fa1c15107ab5] : unable to destroy cgroup paths for cgroup [kubepods besteffort pod64ada63f-b17d-411b-a547-fa1c15107ab5] : Timed out while waiting for systemd to remove kubepods-besteffort-pod64ada63f_b17d_411b_a547_fa1c15107ab5.slice" pod="openstack-kuttl-tests/openstack-cell1-galera-0" podUID="64ada63f-b17d-411b-a547-fa1c15107ab5" Dec 03 12:55:21 crc kubenswrapper[5052]: I1203 12:55:21.057579 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/openstack-cell1-galera-0" Dec 03 12:55:21 crc kubenswrapper[5052]: I1203 12:55:21.078205 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/openstack-cell1-galera-0"] Dec 03 12:55:21 crc kubenswrapper[5052]: I1203 12:55:21.083953 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/openstack-cell1-galera-0"] Dec 03 12:55:21 crc kubenswrapper[5052]: I1203 12:55:21.683126 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="64ada63f-b17d-411b-a547-fa1c15107ab5" path="/var/lib/kubelet/pods/64ada63f-b17d-411b-a547-fa1c15107ab5/volumes" Dec 03 12:55:23 crc kubenswrapper[5052]: I1203 12:55:23.793709 5052 pod_container_manager_linux.go:210] "Failed to delete cgroup paths" cgroupName=["kubepods","besteffort","pod1b7ef5eb-406e-4e2c-9d9c-d763919f56fd"] err="unable to destroy cgroup paths for cgroup [kubepods besteffort pod1b7ef5eb-406e-4e2c-9d9c-d763919f56fd] : Timed out while waiting for systemd to remove kubepods-besteffort-pod1b7ef5eb_406e_4e2c_9d9c_d763919f56fd.slice" Dec 03 12:55:24 crc kubenswrapper[5052]: I1203 12:55:24.673086 5052 scope.go:117] "RemoveContainer" containerID="69fa7fbcd2aceb7c53703a19508876f93a97dc9adabb79c16fb691d90f606d58" Dec 03 12:55:24 crc kubenswrapper[5052]: E1203 12:55:24.673552 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.230102 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["crc-storage/crc-storage-crc-br5r4"] Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.241270 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["crc-storage/crc-storage-crc-br5r4"] Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.378211 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["crc-storage/crc-storage-crc-pskkk"] Dec 03 12:55:28 crc kubenswrapper[5052]: E1203 12:55:28.378606 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="d1ff5100-3ac7-4633-811c-ea470242d858" containerName="memcached" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.378629 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="d1ff5100-3ac7-4633-811c-ea470242d858" containerName="memcached" Dec 03 12:55:28 crc kubenswrapper[5052]: E1203 12:55:28.378644 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4" containerName="account-reaper" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.378652 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4" containerName="account-reaper" Dec 03 12:55:28 crc kubenswrapper[5052]: E1203 12:55:28.378667 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="2b0bf9ae-2d1d-48da-8c4b-2973b47985a3" containerName="probe" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.378675 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="2b0bf9ae-2d1d-48da-8c4b-2973b47985a3" containerName="probe" Dec 03 12:55:28 crc kubenswrapper[5052]: E1203 12:55:28.378690 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="25eead1e-d17c-4b36-8a9f-ef563189d53b" containerName="barbican-api-log" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.378697 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="25eead1e-d17c-4b36-8a9f-ef563189d53b" containerName="barbican-api-log" Dec 03 12:55:28 crc kubenswrapper[5052]: E1203 12:55:28.378705 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="0191d138-7aff-42c2-88d2-fbbaf1722bfd" containerName="keystone-api" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.378712 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="0191d138-7aff-42c2-88d2-fbbaf1722bfd" containerName="keystone-api" Dec 03 12:55:28 crc kubenswrapper[5052]: E1203 12:55:28.378722 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="cfc68960-6006-486c-bc03-848875d3bf5b" containerName="glance-log" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.378730 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="cfc68960-6006-486c-bc03-848875d3bf5b" containerName="glance-log" Dec 03 12:55:28 crc kubenswrapper[5052]: E1203 12:55:28.378743 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="79efd225-5ce4-428f-aef9-3f4884931fef" containerName="mariadb-account-delete" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.378750 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="79efd225-5ce4-428f-aef9-3f4884931fef" containerName="mariadb-account-delete" Dec 03 12:55:28 crc kubenswrapper[5052]: E1203 12:55:28.378764 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="e2225738-b0d1-4306-bbee-a61dd9558c80" containerName="neutron-httpd" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.378771 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="e2225738-b0d1-4306-bbee-a61dd9558c80" containerName="neutron-httpd" Dec 03 12:55:28 crc kubenswrapper[5052]: E1203 12:55:28.378788 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="e73813f5-f97f-41dd-8a79-6196557d4d1a" containerName="rabbitmq" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.378795 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="e73813f5-f97f-41dd-8a79-6196557d4d1a" containerName="rabbitmq" Dec 03 12:55:28 crc kubenswrapper[5052]: E1203 12:55:28.378804 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="43b07f3c-f9d7-451b-b0ab-61b293cff35d" containerName="nova-cell1-novncproxy-novncproxy" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.378811 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="43b07f3c-f9d7-451b-b0ab-61b293cff35d" containerName="nova-cell1-novncproxy-novncproxy" Dec 03 12:55:28 crc kubenswrapper[5052]: E1203 12:55:28.378823 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="022746a5-1c20-4729-8688-5903831f57ed" containerName="nova-metadata-metadata" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.378830 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="022746a5-1c20-4729-8688-5903831f57ed" containerName="nova-metadata-metadata" Dec 03 12:55:28 crc kubenswrapper[5052]: E1203 12:55:28.378841 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="d16285c0-7e62-4b3c-b3e8-a62b8abb49cb" containerName="openstack-network-exporter" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.378848 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="d16285c0-7e62-4b3c-b3e8-a62b8abb49cb" containerName="openstack-network-exporter" Dec 03 12:55:28 crc kubenswrapper[5052]: E1203 12:55:28.378861 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="e8a26759-2926-41cf-a1ec-286fc9962bb3" containerName="mariadb-account-delete" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.378869 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="e8a26759-2926-41cf-a1ec-286fc9962bb3" containerName="mariadb-account-delete" Dec 03 12:55:28 crc kubenswrapper[5052]: E1203 12:55:28.378882 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="020bfa4c-3dba-43d2-9ac2-0bbbae660709" containerName="barbican-keystone-listener-log" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.378890 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="020bfa4c-3dba-43d2-9ac2-0bbbae660709" containerName="barbican-keystone-listener-log" Dec 03 12:55:28 crc kubenswrapper[5052]: E1203 12:55:28.378929 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="e2225738-b0d1-4306-bbee-a61dd9558c80" containerName="neutron-api" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.378936 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="e2225738-b0d1-4306-bbee-a61dd9558c80" containerName="neutron-api" Dec 03 12:55:28 crc kubenswrapper[5052]: E1203 12:55:28.378949 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="568addd3-4ad3-47c3-adae-693691e7cd40" containerName="setup-container" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.378958 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="568addd3-4ad3-47c3-adae-693691e7cd40" containerName="setup-container" Dec 03 12:55:28 crc kubenswrapper[5052]: E1203 12:55:28.378971 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="64ada63f-b17d-411b-a547-fa1c15107ab5" containerName="galera" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.378979 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="64ada63f-b17d-411b-a547-fa1c15107ab5" containerName="galera" Dec 03 12:55:28 crc kubenswrapper[5052]: E1203 12:55:28.378989 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="eefe83f8-0235-4b16-aefb-5ec7b2d92fcf" containerName="proxy-httpd" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.378996 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="eefe83f8-0235-4b16-aefb-5ec7b2d92fcf" containerName="proxy-httpd" Dec 03 12:55:28 crc kubenswrapper[5052]: E1203 12:55:28.379004 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="e73813f5-f97f-41dd-8a79-6196557d4d1a" containerName="setup-container" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.379011 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="e73813f5-f97f-41dd-8a79-6196557d4d1a" containerName="setup-container" Dec 03 12:55:28 crc kubenswrapper[5052]: E1203 12:55:28.379019 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="020bfa4c-3dba-43d2-9ac2-0bbbae660709" containerName="barbican-keystone-listener" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.379025 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="020bfa4c-3dba-43d2-9ac2-0bbbae660709" containerName="barbican-keystone-listener" Dec 03 12:55:28 crc kubenswrapper[5052]: E1203 12:55:28.379036 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="1b7ef5eb-406e-4e2c-9d9c-d763919f56fd" containerName="mariadb-account-delete" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.379042 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="1b7ef5eb-406e-4e2c-9d9c-d763919f56fd" containerName="mariadb-account-delete" Dec 03 12:55:28 crc kubenswrapper[5052]: E1203 12:55:28.379050 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="8df65ee4-b91d-4ffd-bff2-65e5e6559130" containerName="placement-log" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.379056 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="8df65ee4-b91d-4ffd-bff2-65e5e6559130" containerName="placement-log" Dec 03 12:55:28 crc kubenswrapper[5052]: E1203 12:55:28.379062 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="941d063f-c834-47a1-bc0c-2096b005e09c" containerName="mariadb-account-delete" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.379068 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="941d063f-c834-47a1-bc0c-2096b005e09c" containerName="mariadb-account-delete" Dec 03 12:55:28 crc kubenswrapper[5052]: E1203 12:55:28.379076 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="eefe83f8-0235-4b16-aefb-5ec7b2d92fcf" containerName="proxy-server" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.379082 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="eefe83f8-0235-4b16-aefb-5ec7b2d92fcf" containerName="proxy-server" Dec 03 12:55:28 crc kubenswrapper[5052]: E1203 12:55:28.379090 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="ed40ab59-7f38-4b2a-ad68-85592fe22ef2" containerName="cinder-api-log" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.379096 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="ed40ab59-7f38-4b2a-ad68-85592fe22ef2" containerName="cinder-api-log" Dec 03 12:55:28 crc kubenswrapper[5052]: E1203 12:55:28.379102 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4" containerName="rsync" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.379108 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4" containerName="rsync" Dec 03 12:55:28 crc kubenswrapper[5052]: E1203 12:55:28.379117 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="02edb9e9-3f42-4aea-9fe6-7efa332d5fb8" containerName="ceilometer-notification-agent" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.379124 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="02edb9e9-3f42-4aea-9fe6-7efa332d5fb8" containerName="ceilometer-notification-agent" Dec 03 12:55:28 crc kubenswrapper[5052]: E1203 12:55:28.379134 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="3c553114-e0c8-4af9-992f-5421b31497ed" containerName="kube-state-metrics" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.379141 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="3c553114-e0c8-4af9-992f-5421b31497ed" containerName="kube-state-metrics" Dec 03 12:55:28 crc kubenswrapper[5052]: E1203 12:55:28.379149 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="3c6d824e-c7da-41ff-adde-dbafb7535476" containerName="nova-cell0-conductor-conductor" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.379154 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="3c6d824e-c7da-41ff-adde-dbafb7535476" containerName="nova-cell0-conductor-conductor" Dec 03 12:55:28 crc kubenswrapper[5052]: E1203 12:55:28.379166 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="7d297f47-a73f-4a88-bba0-3108deb8df4d" containerName="nova-api-log" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.379173 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d297f47-a73f-4a88-bba0-3108deb8df4d" containerName="nova-api-log" Dec 03 12:55:28 crc kubenswrapper[5052]: E1203 12:55:28.379181 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="13e7e748-92f4-4255-a8eb-aa5a6faf361f" containerName="mariadb-account-delete" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.379188 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="13e7e748-92f4-4255-a8eb-aa5a6faf361f" containerName="mariadb-account-delete" Dec 03 12:55:28 crc kubenswrapper[5052]: E1203 12:55:28.379199 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="b3363d6f-656e-4e22-a3eb-b1016380a0c8" containerName="mysql-bootstrap" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.379204 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="b3363d6f-656e-4e22-a3eb-b1016380a0c8" containerName="mysql-bootstrap" Dec 03 12:55:28 crc kubenswrapper[5052]: E1203 12:55:28.379214 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="7d297f47-a73f-4a88-bba0-3108deb8df4d" containerName="nova-api-api" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.379219 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d297f47-a73f-4a88-bba0-3108deb8df4d" containerName="nova-api-api" Dec 03 12:55:28 crc kubenswrapper[5052]: E1203 12:55:28.379228 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="3b93bc75-db87-4677-be2f-910d07bb696c" containerName="openstack-network-exporter" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.379235 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b93bc75-db87-4677-be2f-910d07bb696c" containerName="openstack-network-exporter" Dec 03 12:55:28 crc kubenswrapper[5052]: E1203 12:55:28.379242 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="1cf6bdc8-8a27-4cc6-a2e9-f6de73fc1ea6" containerName="dnsmasq-dns" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.379250 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="1cf6bdc8-8a27-4cc6-a2e9-f6de73fc1ea6" containerName="dnsmasq-dns" Dec 03 12:55:28 crc kubenswrapper[5052]: E1203 12:55:28.379263 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="3b93bc75-db87-4677-be2f-910d07bb696c" containerName="ovn-northd" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.379269 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b93bc75-db87-4677-be2f-910d07bb696c" containerName="ovn-northd" Dec 03 12:55:28 crc kubenswrapper[5052]: E1203 12:55:28.379281 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="981c3156-805a-44d1-b4ec-f688a8bc3cf8" containerName="glance-log" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.379287 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="981c3156-805a-44d1-b4ec-f688a8bc3cf8" containerName="glance-log" Dec 03 12:55:28 crc kubenswrapper[5052]: E1203 12:55:28.379301 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="3a5e94ea-faeb-4202-843a-bd691d8895ce" containerName="ovsdbserver-nb" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.379326 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="3a5e94ea-faeb-4202-843a-bd691d8895ce" containerName="ovsdbserver-nb" Dec 03 12:55:28 crc kubenswrapper[5052]: E1203 12:55:28.379335 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="d16285c0-7e62-4b3c-b3e8-a62b8abb49cb" containerName="ovsdbserver-sb" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.379341 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="d16285c0-7e62-4b3c-b3e8-a62b8abb49cb" containerName="ovsdbserver-sb" Dec 03 12:55:28 crc kubenswrapper[5052]: E1203 12:55:28.379353 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4" containerName="swift-recon-cron" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.379359 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4" containerName="swift-recon-cron" Dec 03 12:55:28 crc kubenswrapper[5052]: E1203 12:55:28.379367 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="b2933134-7226-423a-8120-3148845b808a" containerName="barbican-worker" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.379373 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="b2933134-7226-423a-8120-3148845b808a" containerName="barbican-worker" Dec 03 12:55:28 crc kubenswrapper[5052]: E1203 12:55:28.379386 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="c4d5c183-8df4-4c43-9afb-2b28c6aae9e1" containerName="mariadb-account-delete" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.379393 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="c4d5c183-8df4-4c43-9afb-2b28c6aae9e1" containerName="mariadb-account-delete" Dec 03 12:55:28 crc kubenswrapper[5052]: E1203 12:55:28.379401 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="3a5e94ea-faeb-4202-843a-bd691d8895ce" containerName="openstack-network-exporter" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.379407 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="3a5e94ea-faeb-4202-843a-bd691d8895ce" containerName="openstack-network-exporter" Dec 03 12:55:28 crc kubenswrapper[5052]: E1203 12:55:28.379420 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="2b0bf9ae-2d1d-48da-8c4b-2973b47985a3" containerName="cinder-scheduler" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.379426 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="2b0bf9ae-2d1d-48da-8c4b-2973b47985a3" containerName="cinder-scheduler" Dec 03 12:55:28 crc kubenswrapper[5052]: E1203 12:55:28.379434 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="02edb9e9-3f42-4aea-9fe6-7efa332d5fb8" containerName="proxy-httpd" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.379440 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="02edb9e9-3f42-4aea-9fe6-7efa332d5fb8" containerName="proxy-httpd" Dec 03 12:55:28 crc kubenswrapper[5052]: E1203 12:55:28.379448 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4" containerName="container-updater" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.379454 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4" containerName="container-updater" Dec 03 12:55:28 crc kubenswrapper[5052]: E1203 12:55:28.379462 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="02edb9e9-3f42-4aea-9fe6-7efa332d5fb8" containerName="sg-core" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.379468 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="02edb9e9-3f42-4aea-9fe6-7efa332d5fb8" containerName="sg-core" Dec 03 12:55:28 crc kubenswrapper[5052]: E1203 12:55:28.379477 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="d9bf91f1-099c-4e72-97cf-1828241aeaac" containerName="mariadb-account-delete" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.379482 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="d9bf91f1-099c-4e72-97cf-1828241aeaac" containerName="mariadb-account-delete" Dec 03 12:55:28 crc kubenswrapper[5052]: E1203 12:55:28.379493 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="8df65ee4-b91d-4ffd-bff2-65e5e6559130" containerName="placement-api" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.379500 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="8df65ee4-b91d-4ffd-bff2-65e5e6559130" containerName="placement-api" Dec 03 12:55:28 crc kubenswrapper[5052]: E1203 12:55:28.379508 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="981c3156-805a-44d1-b4ec-f688a8bc3cf8" containerName="glance-httpd" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.379518 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="981c3156-805a-44d1-b4ec-f688a8bc3cf8" containerName="glance-httpd" Dec 03 12:55:28 crc kubenswrapper[5052]: E1203 12:55:28.379539 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="022746a5-1c20-4729-8688-5903831f57ed" containerName="nova-metadata-log" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.379545 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="022746a5-1c20-4729-8688-5903831f57ed" containerName="nova-metadata-log" Dec 03 12:55:28 crc kubenswrapper[5052]: E1203 12:55:28.379551 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="64ada63f-b17d-411b-a547-fa1c15107ab5" containerName="mysql-bootstrap" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.379557 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="64ada63f-b17d-411b-a547-fa1c15107ab5" containerName="mysql-bootstrap" Dec 03 12:55:28 crc kubenswrapper[5052]: E1203 12:55:28.379564 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4" containerName="object-auditor" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.379570 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4" containerName="object-auditor" Dec 03 12:55:28 crc kubenswrapper[5052]: E1203 12:55:28.379576 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="568addd3-4ad3-47c3-adae-693691e7cd40" containerName="rabbitmq" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.379581 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="568addd3-4ad3-47c3-adae-693691e7cd40" containerName="rabbitmq" Dec 03 12:55:28 crc kubenswrapper[5052]: E1203 12:55:28.379591 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4" containerName="object-replicator" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.379597 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4" containerName="object-replicator" Dec 03 12:55:28 crc kubenswrapper[5052]: E1203 12:55:28.379603 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4" containerName="container-server" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.379609 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4" containerName="container-server" Dec 03 12:55:28 crc kubenswrapper[5052]: E1203 12:55:28.379617 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="4ec49e76-2597-47d6-811b-f65cbe0580ff" containerName="nova-scheduler-scheduler" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.379622 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="4ec49e76-2597-47d6-811b-f65cbe0580ff" containerName="nova-scheduler-scheduler" Dec 03 12:55:28 crc kubenswrapper[5052]: E1203 12:55:28.379631 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4" containerName="container-replicator" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.379636 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4" containerName="container-replicator" Dec 03 12:55:28 crc kubenswrapper[5052]: E1203 12:55:28.379645 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="c4d5c183-8df4-4c43-9afb-2b28c6aae9e1" containerName="mariadb-account-delete" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.379651 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="c4d5c183-8df4-4c43-9afb-2b28c6aae9e1" containerName="mariadb-account-delete" Dec 03 12:55:28 crc kubenswrapper[5052]: E1203 12:55:28.379659 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="b2933134-7226-423a-8120-3148845b808a" containerName="barbican-worker-log" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.379665 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="b2933134-7226-423a-8120-3148845b808a" containerName="barbican-worker-log" Dec 03 12:55:28 crc kubenswrapper[5052]: E1203 12:55:28.379671 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="d9bf91f1-099c-4e72-97cf-1828241aeaac" containerName="mariadb-account-delete" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.379677 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="d9bf91f1-099c-4e72-97cf-1828241aeaac" containerName="mariadb-account-delete" Dec 03 12:55:28 crc kubenswrapper[5052]: E1203 12:55:28.379688 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4" containerName="account-replicator" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.379693 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4" containerName="account-replicator" Dec 03 12:55:28 crc kubenswrapper[5052]: E1203 12:55:28.379704 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4" containerName="container-auditor" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.379710 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4" containerName="container-auditor" Dec 03 12:55:28 crc kubenswrapper[5052]: E1203 12:55:28.379719 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="1cf6bdc8-8a27-4cc6-a2e9-f6de73fc1ea6" containerName="init" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.379725 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="1cf6bdc8-8a27-4cc6-a2e9-f6de73fc1ea6" containerName="init" Dec 03 12:55:28 crc kubenswrapper[5052]: E1203 12:55:28.379736 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="b3363d6f-656e-4e22-a3eb-b1016380a0c8" containerName="galera" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.379742 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="b3363d6f-656e-4e22-a3eb-b1016380a0c8" containerName="galera" Dec 03 12:55:28 crc kubenswrapper[5052]: E1203 12:55:28.379752 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="ed40ab59-7f38-4b2a-ad68-85592fe22ef2" containerName="cinder-api" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.379757 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="ed40ab59-7f38-4b2a-ad68-85592fe22ef2" containerName="cinder-api" Dec 03 12:55:28 crc kubenswrapper[5052]: E1203 12:55:28.379769 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="2adad00d-01e3-4441-a3fc-87d162a5e309" containerName="nova-cell1-conductor-conductor" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.379774 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="2adad00d-01e3-4441-a3fc-87d162a5e309" containerName="nova-cell1-conductor-conductor" Dec 03 12:55:28 crc kubenswrapper[5052]: E1203 12:55:28.379784 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4" containerName="object-expirer" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.379790 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4" containerName="object-expirer" Dec 03 12:55:28 crc kubenswrapper[5052]: E1203 12:55:28.379800 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4" containerName="account-auditor" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.379805 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4" containerName="account-auditor" Dec 03 12:55:28 crc kubenswrapper[5052]: E1203 12:55:28.379814 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4" containerName="object-updater" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.379820 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4" containerName="object-updater" Dec 03 12:55:28 crc kubenswrapper[5052]: E1203 12:55:28.379828 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="cfc68960-6006-486c-bc03-848875d3bf5b" containerName="glance-httpd" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.379833 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="cfc68960-6006-486c-bc03-848875d3bf5b" containerName="glance-httpd" Dec 03 12:55:28 crc kubenswrapper[5052]: E1203 12:55:28.379841 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4" containerName="account-server" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.379847 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4" containerName="account-server" Dec 03 12:55:28 crc kubenswrapper[5052]: E1203 12:55:28.379855 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4" containerName="object-server" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.379860 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4" containerName="object-server" Dec 03 12:55:28 crc kubenswrapper[5052]: E1203 12:55:28.379870 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="25eead1e-d17c-4b36-8a9f-ef563189d53b" containerName="barbican-api" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.379876 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="25eead1e-d17c-4b36-8a9f-ef563189d53b" containerName="barbican-api" Dec 03 12:55:28 crc kubenswrapper[5052]: E1203 12:55:28.379883 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="02edb9e9-3f42-4aea-9fe6-7efa332d5fb8" containerName="ceilometer-central-agent" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.379889 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="02edb9e9-3f42-4aea-9fe6-7efa332d5fb8" containerName="ceilometer-central-agent" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.380025 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="ed40ab59-7f38-4b2a-ad68-85592fe22ef2" containerName="cinder-api-log" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.380042 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="2b0bf9ae-2d1d-48da-8c4b-2973b47985a3" containerName="probe" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.380052 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="02edb9e9-3f42-4aea-9fe6-7efa332d5fb8" containerName="ceilometer-central-agent" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.380058 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="e8a26759-2926-41cf-a1ec-286fc9962bb3" containerName="mariadb-account-delete" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.380067 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="3c6d824e-c7da-41ff-adde-dbafb7535476" containerName="nova-cell0-conductor-conductor" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.380075 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="02edb9e9-3f42-4aea-9fe6-7efa332d5fb8" containerName="proxy-httpd" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.380086 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4" containerName="object-updater" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.380096 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="8df65ee4-b91d-4ffd-bff2-65e5e6559130" containerName="placement-api" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.380108 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="e8a26759-2926-41cf-a1ec-286fc9962bb3" containerName="mariadb-account-delete" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.380115 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="b2933134-7226-423a-8120-3148845b808a" containerName="barbican-worker-log" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.380123 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="022746a5-1c20-4729-8688-5903831f57ed" containerName="nova-metadata-metadata" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.380129 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="1b7ef5eb-406e-4e2c-9d9c-d763919f56fd" containerName="mariadb-account-delete" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.380135 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4" containerName="account-replicator" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.380145 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="eefe83f8-0235-4b16-aefb-5ec7b2d92fcf" containerName="proxy-httpd" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.380154 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="020bfa4c-3dba-43d2-9ac2-0bbbae660709" containerName="barbican-keystone-listener" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.380160 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4" containerName="container-server" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.380169 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="02edb9e9-3f42-4aea-9fe6-7efa332d5fb8" containerName="sg-core" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.380177 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4" containerName="rsync" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.380187 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="cfc68960-6006-486c-bc03-848875d3bf5b" containerName="glance-httpd" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.380195 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="3a5e94ea-faeb-4202-843a-bd691d8895ce" containerName="ovsdbserver-nb" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.380203 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="981c3156-805a-44d1-b4ec-f688a8bc3cf8" containerName="glance-httpd" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.380214 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="d16285c0-7e62-4b3c-b3e8-a62b8abb49cb" containerName="ovsdbserver-sb" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.380223 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="0191d138-7aff-42c2-88d2-fbbaf1722bfd" containerName="keystone-api" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.380231 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="2adad00d-01e3-4441-a3fc-87d162a5e309" containerName="nova-cell1-conductor-conductor" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.380238 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4" containerName="object-server" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.380249 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="d16285c0-7e62-4b3c-b3e8-a62b8abb49cb" containerName="openstack-network-exporter" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.380259 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="8df65ee4-b91d-4ffd-bff2-65e5e6559130" containerName="placement-log" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.380270 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="981c3156-805a-44d1-b4ec-f688a8bc3cf8" containerName="glance-log" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.380277 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="b3363d6f-656e-4e22-a3eb-b1016380a0c8" containerName="galera" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.380284 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="e2225738-b0d1-4306-bbee-a61dd9558c80" containerName="neutron-api" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.380291 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4" containerName="object-expirer" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.380298 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4" containerName="account-server" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.380307 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4" containerName="object-replicator" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.380315 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="cfc68960-6006-486c-bc03-848875d3bf5b" containerName="glance-log" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.380323 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="e73813f5-f97f-41dd-8a79-6196557d4d1a" containerName="rabbitmq" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.380329 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="568addd3-4ad3-47c3-adae-693691e7cd40" containerName="rabbitmq" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.380337 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="941d063f-c834-47a1-bc0c-2096b005e09c" containerName="mariadb-account-delete" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.380345 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4" containerName="container-auditor" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.380353 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="d9bf91f1-099c-4e72-97cf-1828241aeaac" containerName="mariadb-account-delete" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.380385 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="79efd225-5ce4-428f-aef9-3f4884931fef" containerName="mariadb-account-delete" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.380392 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="020bfa4c-3dba-43d2-9ac2-0bbbae660709" containerName="barbican-keystone-listener-log" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.380401 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="ed40ab59-7f38-4b2a-ad68-85592fe22ef2" containerName="cinder-api" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.380408 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="eefe83f8-0235-4b16-aefb-5ec7b2d92fcf" containerName="proxy-server" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.380416 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="c4d5c183-8df4-4c43-9afb-2b28c6aae9e1" containerName="mariadb-account-delete" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.380424 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="7d297f47-a73f-4a88-bba0-3108deb8df4d" containerName="nova-api-api" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.380431 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="d9bf91f1-099c-4e72-97cf-1828241aeaac" containerName="mariadb-account-delete" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.380438 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="3b93bc75-db87-4677-be2f-910d07bb696c" containerName="openstack-network-exporter" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.380448 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4" containerName="account-auditor" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.380455 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="2b0bf9ae-2d1d-48da-8c4b-2973b47985a3" containerName="cinder-scheduler" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.380461 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="3b93bc75-db87-4677-be2f-910d07bb696c" containerName="ovn-northd" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.380467 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="25eead1e-d17c-4b36-8a9f-ef563189d53b" containerName="barbican-api-log" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.380477 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4" containerName="container-replicator" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.380488 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="c4d5c183-8df4-4c43-9afb-2b28c6aae9e1" containerName="mariadb-account-delete" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.380496 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4" containerName="swift-recon-cron" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.380504 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="02edb9e9-3f42-4aea-9fe6-7efa332d5fb8" containerName="ceilometer-notification-agent" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.380517 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="941d063f-c834-47a1-bc0c-2096b005e09c" containerName="mariadb-account-delete" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.380537 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="13e7e748-92f4-4255-a8eb-aa5a6faf361f" containerName="mariadb-account-delete" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.380544 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="4ec49e76-2597-47d6-811b-f65cbe0580ff" containerName="nova-scheduler-scheduler" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.380553 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="e2225738-b0d1-4306-bbee-a61dd9558c80" containerName="neutron-httpd" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.380560 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="7d297f47-a73f-4a88-bba0-3108deb8df4d" containerName="nova-api-log" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.380567 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4" containerName="account-reaper" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.380574 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="25eead1e-d17c-4b36-8a9f-ef563189d53b" containerName="barbican-api" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.380582 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="022746a5-1c20-4729-8688-5903831f57ed" containerName="nova-metadata-log" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.380588 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="3c553114-e0c8-4af9-992f-5421b31497ed" containerName="kube-state-metrics" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.380596 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="43b07f3c-f9d7-451b-b0ab-61b293cff35d" containerName="nova-cell1-novncproxy-novncproxy" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.380604 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="d1ff5100-3ac7-4633-811c-ea470242d858" containerName="memcached" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.380611 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="1cf6bdc8-8a27-4cc6-a2e9-f6de73fc1ea6" containerName="dnsmasq-dns" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.380621 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4" containerName="object-auditor" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.380629 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="b2933134-7226-423a-8120-3148845b808a" containerName="barbican-worker" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.380640 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="3a5e94ea-faeb-4202-843a-bd691d8895ce" containerName="openstack-network-exporter" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.380646 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="64ada63f-b17d-411b-a547-fa1c15107ab5" containerName="galera" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.380653 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="5a63be9b-cdff-4d47-b2ad-a5039b1b9fc4" containerName="container-updater" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.381158 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="crc-storage/crc-storage-crc-pskkk" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.383443 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"crc-storage"/"crc-storage" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.385392 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"crc-storage"/"kube-root-ca.crt" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.385661 5052 reflector.go:368] Caches populated for *v1.Secret from object-"crc-storage"/"crc-storage-dockercfg-mf76g" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.391946 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"crc-storage"/"openshift-service-ca.crt" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.401977 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["crc-storage/crc-storage-crc-pskkk"] Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.543716 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/ee6f39b6-8d98-4537-a057-6fbd091b721f-node-mnt\") pod \"crc-storage-crc-pskkk\" (UID: \"ee6f39b6-8d98-4537-a057-6fbd091b721f\") " pod="crc-storage/crc-storage-crc-pskkk" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.543828 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/ee6f39b6-8d98-4537-a057-6fbd091b721f-crc-storage\") pod \"crc-storage-crc-pskkk\" (UID: \"ee6f39b6-8d98-4537-a057-6fbd091b721f\") " pod="crc-storage/crc-storage-crc-pskkk" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.543859 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vprpg\" (UniqueName: \"kubernetes.io/projected/ee6f39b6-8d98-4537-a057-6fbd091b721f-kube-api-access-vprpg\") pod \"crc-storage-crc-pskkk\" (UID: \"ee6f39b6-8d98-4537-a057-6fbd091b721f\") " pod="crc-storage/crc-storage-crc-pskkk" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.645884 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/ee6f39b6-8d98-4537-a057-6fbd091b721f-node-mnt\") pod \"crc-storage-crc-pskkk\" (UID: \"ee6f39b6-8d98-4537-a057-6fbd091b721f\") " pod="crc-storage/crc-storage-crc-pskkk" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.645977 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-vprpg\" (UniqueName: \"kubernetes.io/projected/ee6f39b6-8d98-4537-a057-6fbd091b721f-kube-api-access-vprpg\") pod \"crc-storage-crc-pskkk\" (UID: \"ee6f39b6-8d98-4537-a057-6fbd091b721f\") " pod="crc-storage/crc-storage-crc-pskkk" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.646003 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/ee6f39b6-8d98-4537-a057-6fbd091b721f-crc-storage\") pod \"crc-storage-crc-pskkk\" (UID: \"ee6f39b6-8d98-4537-a057-6fbd091b721f\") " pod="crc-storage/crc-storage-crc-pskkk" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.646159 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/ee6f39b6-8d98-4537-a057-6fbd091b721f-node-mnt\") pod \"crc-storage-crc-pskkk\" (UID: \"ee6f39b6-8d98-4537-a057-6fbd091b721f\") " pod="crc-storage/crc-storage-crc-pskkk" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.647025 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/ee6f39b6-8d98-4537-a057-6fbd091b721f-crc-storage\") pod \"crc-storage-crc-pskkk\" (UID: \"ee6f39b6-8d98-4537-a057-6fbd091b721f\") " pod="crc-storage/crc-storage-crc-pskkk" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.665971 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-vprpg\" (UniqueName: \"kubernetes.io/projected/ee6f39b6-8d98-4537-a057-6fbd091b721f-kube-api-access-vprpg\") pod \"crc-storage-crc-pskkk\" (UID: \"ee6f39b6-8d98-4537-a057-6fbd091b721f\") " pod="crc-storage/crc-storage-crc-pskkk" Dec 03 12:55:28 crc kubenswrapper[5052]: I1203 12:55:28.707396 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="crc-storage/crc-storage-crc-pskkk" Dec 03 12:55:29 crc kubenswrapper[5052]: I1203 12:55:29.200030 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["crc-storage/crc-storage-crc-pskkk"] Dec 03 12:55:29 crc kubenswrapper[5052]: I1203 12:55:29.712710 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="abdb8807-9006-4c35-9d4b-41dcd672416f" path="/var/lib/kubelet/pods/abdb8807-9006-4c35-9d4b-41dcd672416f/volumes" Dec 03 12:55:30 crc kubenswrapper[5052]: I1203 12:55:30.139587 5052 generic.go:334] "Generic (PLEG): container finished" podID="ee6f39b6-8d98-4537-a057-6fbd091b721f" containerID="124318204b3b140e1a75ced489756deb52efc6f18a43f8898a678f726dabdad6" exitCode=0 Dec 03 12:55:30 crc kubenswrapper[5052]: I1203 12:55:30.139642 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="crc-storage/crc-storage-crc-pskkk" event={"ID":"ee6f39b6-8d98-4537-a057-6fbd091b721f","Type":"ContainerDied","Data":"124318204b3b140e1a75ced489756deb52efc6f18a43f8898a678f726dabdad6"} Dec 03 12:55:30 crc kubenswrapper[5052]: I1203 12:55:30.139676 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="crc-storage/crc-storage-crc-pskkk" event={"ID":"ee6f39b6-8d98-4537-a057-6fbd091b721f","Type":"ContainerStarted","Data":"4b5bbe0f4165a719668398ccc9dd05073d5e0d737fa3064d9a1699cfca0c9fd7"} Dec 03 12:55:31 crc kubenswrapper[5052]: I1203 12:55:31.270428 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openshift-marketplace/redhat-marketplace-vxs42"] Dec 03 12:55:31 crc kubenswrapper[5052]: E1203 12:55:31.270942 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="79efd225-5ce4-428f-aef9-3f4884931fef" containerName="mariadb-account-delete" Dec 03 12:55:31 crc kubenswrapper[5052]: I1203 12:55:31.270960 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="79efd225-5ce4-428f-aef9-3f4884931fef" containerName="mariadb-account-delete" Dec 03 12:55:31 crc kubenswrapper[5052]: E1203 12:55:31.270997 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="e8a26759-2926-41cf-a1ec-286fc9962bb3" containerName="mariadb-account-delete" Dec 03 12:55:31 crc kubenswrapper[5052]: I1203 12:55:31.271005 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="e8a26759-2926-41cf-a1ec-286fc9962bb3" containerName="mariadb-account-delete" Dec 03 12:55:31 crc kubenswrapper[5052]: E1203 12:55:31.271025 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="941d063f-c834-47a1-bc0c-2096b005e09c" containerName="mariadb-account-delete" Dec 03 12:55:31 crc kubenswrapper[5052]: I1203 12:55:31.271034 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="941d063f-c834-47a1-bc0c-2096b005e09c" containerName="mariadb-account-delete" Dec 03 12:55:31 crc kubenswrapper[5052]: I1203 12:55:31.271263 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="79efd225-5ce4-428f-aef9-3f4884931fef" containerName="mariadb-account-delete" Dec 03 12:55:31 crc kubenswrapper[5052]: I1203 12:55:31.273780 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/redhat-marketplace-vxs42" Dec 03 12:55:31 crc kubenswrapper[5052]: I1203 12:55:31.284439 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/redhat-marketplace-vxs42"] Dec 03 12:55:31 crc kubenswrapper[5052]: I1203 12:55:31.385560 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nhgnk\" (UniqueName: \"kubernetes.io/projected/5f0a6e44-08a6-454e-8580-1c33335219b3-kube-api-access-nhgnk\") pod \"redhat-marketplace-vxs42\" (UID: \"5f0a6e44-08a6-454e-8580-1c33335219b3\") " pod="openshift-marketplace/redhat-marketplace-vxs42" Dec 03 12:55:31 crc kubenswrapper[5052]: I1203 12:55:31.385699 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/5f0a6e44-08a6-454e-8580-1c33335219b3-catalog-content\") pod \"redhat-marketplace-vxs42\" (UID: \"5f0a6e44-08a6-454e-8580-1c33335219b3\") " pod="openshift-marketplace/redhat-marketplace-vxs42" Dec 03 12:55:31 crc kubenswrapper[5052]: I1203 12:55:31.385722 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/5f0a6e44-08a6-454e-8580-1c33335219b3-utilities\") pod \"redhat-marketplace-vxs42\" (UID: \"5f0a6e44-08a6-454e-8580-1c33335219b3\") " pod="openshift-marketplace/redhat-marketplace-vxs42" Dec 03 12:55:31 crc kubenswrapper[5052]: I1203 12:55:31.455076 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="crc-storage/crc-storage-crc-pskkk" Dec 03 12:55:31 crc kubenswrapper[5052]: I1203 12:55:31.487621 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/5f0a6e44-08a6-454e-8580-1c33335219b3-catalog-content\") pod \"redhat-marketplace-vxs42\" (UID: \"5f0a6e44-08a6-454e-8580-1c33335219b3\") " pod="openshift-marketplace/redhat-marketplace-vxs42" Dec 03 12:55:31 crc kubenswrapper[5052]: I1203 12:55:31.487664 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/5f0a6e44-08a6-454e-8580-1c33335219b3-utilities\") pod \"redhat-marketplace-vxs42\" (UID: \"5f0a6e44-08a6-454e-8580-1c33335219b3\") " pod="openshift-marketplace/redhat-marketplace-vxs42" Dec 03 12:55:31 crc kubenswrapper[5052]: I1203 12:55:31.487707 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-nhgnk\" (UniqueName: \"kubernetes.io/projected/5f0a6e44-08a6-454e-8580-1c33335219b3-kube-api-access-nhgnk\") pod \"redhat-marketplace-vxs42\" (UID: \"5f0a6e44-08a6-454e-8580-1c33335219b3\") " pod="openshift-marketplace/redhat-marketplace-vxs42" Dec 03 12:55:31 crc kubenswrapper[5052]: I1203 12:55:31.488220 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/5f0a6e44-08a6-454e-8580-1c33335219b3-catalog-content\") pod \"redhat-marketplace-vxs42\" (UID: \"5f0a6e44-08a6-454e-8580-1c33335219b3\") " pod="openshift-marketplace/redhat-marketplace-vxs42" Dec 03 12:55:31 crc kubenswrapper[5052]: I1203 12:55:31.488390 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/5f0a6e44-08a6-454e-8580-1c33335219b3-utilities\") pod \"redhat-marketplace-vxs42\" (UID: \"5f0a6e44-08a6-454e-8580-1c33335219b3\") " pod="openshift-marketplace/redhat-marketplace-vxs42" Dec 03 12:55:31 crc kubenswrapper[5052]: I1203 12:55:31.509018 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-nhgnk\" (UniqueName: \"kubernetes.io/projected/5f0a6e44-08a6-454e-8580-1c33335219b3-kube-api-access-nhgnk\") pod \"redhat-marketplace-vxs42\" (UID: \"5f0a6e44-08a6-454e-8580-1c33335219b3\") " pod="openshift-marketplace/redhat-marketplace-vxs42" Dec 03 12:55:31 crc kubenswrapper[5052]: I1203 12:55:31.588709 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vprpg\" (UniqueName: \"kubernetes.io/projected/ee6f39b6-8d98-4537-a057-6fbd091b721f-kube-api-access-vprpg\") pod \"ee6f39b6-8d98-4537-a057-6fbd091b721f\" (UID: \"ee6f39b6-8d98-4537-a057-6fbd091b721f\") " Dec 03 12:55:31 crc kubenswrapper[5052]: I1203 12:55:31.588810 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/ee6f39b6-8d98-4537-a057-6fbd091b721f-node-mnt\") pod \"ee6f39b6-8d98-4537-a057-6fbd091b721f\" (UID: \"ee6f39b6-8d98-4537-a057-6fbd091b721f\") " Dec 03 12:55:31 crc kubenswrapper[5052]: I1203 12:55:31.588877 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/ee6f39b6-8d98-4537-a057-6fbd091b721f-crc-storage\") pod \"ee6f39b6-8d98-4537-a057-6fbd091b721f\" (UID: \"ee6f39b6-8d98-4537-a057-6fbd091b721f\") " Dec 03 12:55:31 crc kubenswrapper[5052]: I1203 12:55:31.588961 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/ee6f39b6-8d98-4537-a057-6fbd091b721f-node-mnt" (OuterVolumeSpecName: "node-mnt") pod "ee6f39b6-8d98-4537-a057-6fbd091b721f" (UID: "ee6f39b6-8d98-4537-a057-6fbd091b721f"). InnerVolumeSpecName "node-mnt". PluginName "kubernetes.io/host-path", VolumeGidValue "" Dec 03 12:55:31 crc kubenswrapper[5052]: I1203 12:55:31.589207 5052 reconciler_common.go:293] "Volume detached for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/ee6f39b6-8d98-4537-a057-6fbd091b721f-node-mnt\") on node \"crc\" DevicePath \"\"" Dec 03 12:55:31 crc kubenswrapper[5052]: I1203 12:55:31.596581 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ee6f39b6-8d98-4537-a057-6fbd091b721f-kube-api-access-vprpg" (OuterVolumeSpecName: "kube-api-access-vprpg") pod "ee6f39b6-8d98-4537-a057-6fbd091b721f" (UID: "ee6f39b6-8d98-4537-a057-6fbd091b721f"). InnerVolumeSpecName "kube-api-access-vprpg". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:55:31 crc kubenswrapper[5052]: I1203 12:55:31.602352 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/redhat-marketplace-vxs42" Dec 03 12:55:31 crc kubenswrapper[5052]: I1203 12:55:31.612159 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ee6f39b6-8d98-4537-a057-6fbd091b721f-crc-storage" (OuterVolumeSpecName: "crc-storage") pod "ee6f39b6-8d98-4537-a057-6fbd091b721f" (UID: "ee6f39b6-8d98-4537-a057-6fbd091b721f"). InnerVolumeSpecName "crc-storage". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:55:31 crc kubenswrapper[5052]: I1203 12:55:31.690286 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-vprpg\" (UniqueName: \"kubernetes.io/projected/ee6f39b6-8d98-4537-a057-6fbd091b721f-kube-api-access-vprpg\") on node \"crc\" DevicePath \"\"" Dec 03 12:55:31 crc kubenswrapper[5052]: I1203 12:55:31.690326 5052 reconciler_common.go:293] "Volume detached for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/ee6f39b6-8d98-4537-a057-6fbd091b721f-crc-storage\") on node \"crc\" DevicePath \"\"" Dec 03 12:55:31 crc kubenswrapper[5052]: I1203 12:55:31.878795 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/redhat-marketplace-vxs42"] Dec 03 12:55:31 crc kubenswrapper[5052]: W1203 12:55:31.881363 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f0a6e44_08a6_454e_8580_1c33335219b3.slice/crio-69e44b423bf9087390deb6500ab45e4beba0bb286287c7246bc0cf13c588fd00 WatchSource:0}: Error finding container 69e44b423bf9087390deb6500ab45e4beba0bb286287c7246bc0cf13c588fd00: Status 404 returned error can't find the container with id 69e44b423bf9087390deb6500ab45e4beba0bb286287c7246bc0cf13c588fd00 Dec 03 12:55:32 crc kubenswrapper[5052]: I1203 12:55:32.162811 5052 generic.go:334] "Generic (PLEG): container finished" podID="5f0a6e44-08a6-454e-8580-1c33335219b3" containerID="e36fa0e3ea12d83fd9cc0e0a2e39ec6811e8022e1a80e53458e96fd128c7c897" exitCode=0 Dec 03 12:55:32 crc kubenswrapper[5052]: I1203 12:55:32.162922 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-marketplace-vxs42" event={"ID":"5f0a6e44-08a6-454e-8580-1c33335219b3","Type":"ContainerDied","Data":"e36fa0e3ea12d83fd9cc0e0a2e39ec6811e8022e1a80e53458e96fd128c7c897"} Dec 03 12:55:32 crc kubenswrapper[5052]: I1203 12:55:32.163448 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-marketplace-vxs42" event={"ID":"5f0a6e44-08a6-454e-8580-1c33335219b3","Type":"ContainerStarted","Data":"69e44b423bf9087390deb6500ab45e4beba0bb286287c7246bc0cf13c588fd00"} Dec 03 12:55:32 crc kubenswrapper[5052]: I1203 12:55:32.165959 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="crc-storage/crc-storage-crc-pskkk" event={"ID":"ee6f39b6-8d98-4537-a057-6fbd091b721f","Type":"ContainerDied","Data":"4b5bbe0f4165a719668398ccc9dd05073d5e0d737fa3064d9a1699cfca0c9fd7"} Dec 03 12:55:32 crc kubenswrapper[5052]: I1203 12:55:32.166009 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4b5bbe0f4165a719668398ccc9dd05073d5e0d737fa3064d9a1699cfca0c9fd7" Dec 03 12:55:32 crc kubenswrapper[5052]: I1203 12:55:32.166072 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="crc-storage/crc-storage-crc-pskkk" Dec 03 12:55:33 crc kubenswrapper[5052]: I1203 12:55:33.177610 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-marketplace-vxs42" event={"ID":"5f0a6e44-08a6-454e-8580-1c33335219b3","Type":"ContainerStarted","Data":"7a4643f480701372ff7a2c0e33933975605568c6046e8665fe0d5caa11ba3e66"} Dec 03 12:55:34 crc kubenswrapper[5052]: I1203 12:55:34.196542 5052 generic.go:334] "Generic (PLEG): container finished" podID="5f0a6e44-08a6-454e-8580-1c33335219b3" containerID="7a4643f480701372ff7a2c0e33933975605568c6046e8665fe0d5caa11ba3e66" exitCode=0 Dec 03 12:55:34 crc kubenswrapper[5052]: I1203 12:55:34.196604 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-marketplace-vxs42" event={"ID":"5f0a6e44-08a6-454e-8580-1c33335219b3","Type":"ContainerDied","Data":"7a4643f480701372ff7a2c0e33933975605568c6046e8665fe0d5caa11ba3e66"} Dec 03 12:55:35 crc kubenswrapper[5052]: I1203 12:55:35.003725 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["crc-storage/crc-storage-crc-pskkk"] Dec 03 12:55:35 crc kubenswrapper[5052]: I1203 12:55:35.010006 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["crc-storage/crc-storage-crc-pskkk"] Dec 03 12:55:35 crc kubenswrapper[5052]: I1203 12:55:35.136097 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["crc-storage/crc-storage-crc-z9h5z"] Dec 03 12:55:35 crc kubenswrapper[5052]: E1203 12:55:35.136455 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="ee6f39b6-8d98-4537-a057-6fbd091b721f" containerName="storage" Dec 03 12:55:35 crc kubenswrapper[5052]: I1203 12:55:35.136475 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee6f39b6-8d98-4537-a057-6fbd091b721f" containerName="storage" Dec 03 12:55:35 crc kubenswrapper[5052]: I1203 12:55:35.136689 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="ee6f39b6-8d98-4537-a057-6fbd091b721f" containerName="storage" Dec 03 12:55:35 crc kubenswrapper[5052]: I1203 12:55:35.137328 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="crc-storage/crc-storage-crc-z9h5z" Dec 03 12:55:35 crc kubenswrapper[5052]: I1203 12:55:35.140031 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"crc-storage"/"openshift-service-ca.crt" Dec 03 12:55:35 crc kubenswrapper[5052]: I1203 12:55:35.140953 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"crc-storage"/"kube-root-ca.crt" Dec 03 12:55:35 crc kubenswrapper[5052]: I1203 12:55:35.141170 5052 reflector.go:368] Caches populated for *v1.Secret from object-"crc-storage"/"crc-storage-dockercfg-mf76g" Dec 03 12:55:35 crc kubenswrapper[5052]: I1203 12:55:35.141218 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"crc-storage"/"crc-storage" Dec 03 12:55:35 crc kubenswrapper[5052]: I1203 12:55:35.155646 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["crc-storage/crc-storage-crc-z9h5z"] Dec 03 12:55:35 crc kubenswrapper[5052]: I1203 12:55:35.220927 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-marketplace-vxs42" event={"ID":"5f0a6e44-08a6-454e-8580-1c33335219b3","Type":"ContainerStarted","Data":"6d4d7eaec35ce7907e463eb02c5339e1763ae84419bc401edcbc5f956a4f58fa"} Dec 03 12:55:35 crc kubenswrapper[5052]: I1203 12:55:35.245441 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-marketplace/redhat-marketplace-vxs42" podStartSLOduration=1.535559278 podStartE2EDuration="4.245416575s" podCreationTimestamp="2025-12-03 12:55:31 +0000 UTC" firstStartedPulling="2025-12-03 12:55:32.164877847 +0000 UTC m=+5722.753119621" lastFinishedPulling="2025-12-03 12:55:34.874735154 +0000 UTC m=+5725.462976918" observedRunningTime="2025-12-03 12:55:35.240798023 +0000 UTC m=+5725.829039787" watchObservedRunningTime="2025-12-03 12:55:35.245416575 +0000 UTC m=+5725.833658359" Dec 03 12:55:35 crc kubenswrapper[5052]: I1203 12:55:35.248061 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/337d1568-3061-4799-bc8b-b8acd9280356-crc-storage\") pod \"crc-storage-crc-z9h5z\" (UID: \"337d1568-3061-4799-bc8b-b8acd9280356\") " pod="crc-storage/crc-storage-crc-z9h5z" Dec 03 12:55:35 crc kubenswrapper[5052]: I1203 12:55:35.248116 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r8r9h\" (UniqueName: \"kubernetes.io/projected/337d1568-3061-4799-bc8b-b8acd9280356-kube-api-access-r8r9h\") pod \"crc-storage-crc-z9h5z\" (UID: \"337d1568-3061-4799-bc8b-b8acd9280356\") " pod="crc-storage/crc-storage-crc-z9h5z" Dec 03 12:55:35 crc kubenswrapper[5052]: I1203 12:55:35.248191 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/337d1568-3061-4799-bc8b-b8acd9280356-node-mnt\") pod \"crc-storage-crc-z9h5z\" (UID: \"337d1568-3061-4799-bc8b-b8acd9280356\") " pod="crc-storage/crc-storage-crc-z9h5z" Dec 03 12:55:35 crc kubenswrapper[5052]: I1203 12:55:35.349498 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-r8r9h\" (UniqueName: \"kubernetes.io/projected/337d1568-3061-4799-bc8b-b8acd9280356-kube-api-access-r8r9h\") pod \"crc-storage-crc-z9h5z\" (UID: \"337d1568-3061-4799-bc8b-b8acd9280356\") " pod="crc-storage/crc-storage-crc-z9h5z" Dec 03 12:55:35 crc kubenswrapper[5052]: I1203 12:55:35.349635 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/337d1568-3061-4799-bc8b-b8acd9280356-node-mnt\") pod \"crc-storage-crc-z9h5z\" (UID: \"337d1568-3061-4799-bc8b-b8acd9280356\") " pod="crc-storage/crc-storage-crc-z9h5z" Dec 03 12:55:35 crc kubenswrapper[5052]: I1203 12:55:35.350044 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/337d1568-3061-4799-bc8b-b8acd9280356-node-mnt\") pod \"crc-storage-crc-z9h5z\" (UID: \"337d1568-3061-4799-bc8b-b8acd9280356\") " pod="crc-storage/crc-storage-crc-z9h5z" Dec 03 12:55:35 crc kubenswrapper[5052]: I1203 12:55:35.350134 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/337d1568-3061-4799-bc8b-b8acd9280356-crc-storage\") pod \"crc-storage-crc-z9h5z\" (UID: \"337d1568-3061-4799-bc8b-b8acd9280356\") " pod="crc-storage/crc-storage-crc-z9h5z" Dec 03 12:55:35 crc kubenswrapper[5052]: I1203 12:55:35.351100 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/337d1568-3061-4799-bc8b-b8acd9280356-crc-storage\") pod \"crc-storage-crc-z9h5z\" (UID: \"337d1568-3061-4799-bc8b-b8acd9280356\") " pod="crc-storage/crc-storage-crc-z9h5z" Dec 03 12:55:35 crc kubenswrapper[5052]: I1203 12:55:35.368640 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-r8r9h\" (UniqueName: \"kubernetes.io/projected/337d1568-3061-4799-bc8b-b8acd9280356-kube-api-access-r8r9h\") pod \"crc-storage-crc-z9h5z\" (UID: \"337d1568-3061-4799-bc8b-b8acd9280356\") " pod="crc-storage/crc-storage-crc-z9h5z" Dec 03 12:55:35 crc kubenswrapper[5052]: I1203 12:55:35.457676 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="crc-storage/crc-storage-crc-z9h5z" Dec 03 12:55:35 crc kubenswrapper[5052]: I1203 12:55:35.684243 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ee6f39b6-8d98-4537-a057-6fbd091b721f" path="/var/lib/kubelet/pods/ee6f39b6-8d98-4537-a057-6fbd091b721f/volumes" Dec 03 12:55:35 crc kubenswrapper[5052]: I1203 12:55:35.905981 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["crc-storage/crc-storage-crc-z9h5z"] Dec 03 12:55:36 crc kubenswrapper[5052]: I1203 12:55:36.233809 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="crc-storage/crc-storage-crc-z9h5z" event={"ID":"337d1568-3061-4799-bc8b-b8acd9280356","Type":"ContainerStarted","Data":"b7ad2e9f92f5bbe97296af1e0dd4d461e28e2164c378e4d83994bd639a1934ba"} Dec 03 12:55:36 crc kubenswrapper[5052]: I1203 12:55:36.673444 5052 scope.go:117] "RemoveContainer" containerID="69fa7fbcd2aceb7c53703a19508876f93a97dc9adabb79c16fb691d90f606d58" Dec 03 12:55:36 crc kubenswrapper[5052]: E1203 12:55:36.673730 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 12:55:37 crc kubenswrapper[5052]: I1203 12:55:37.249241 5052 generic.go:334] "Generic (PLEG): container finished" podID="337d1568-3061-4799-bc8b-b8acd9280356" containerID="c4e366e272363beb523cf40ee21631ae5804252f9f976436f4bc9bc3040b1fc5" exitCode=0 Dec 03 12:55:37 crc kubenswrapper[5052]: I1203 12:55:37.249292 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="crc-storage/crc-storage-crc-z9h5z" event={"ID":"337d1568-3061-4799-bc8b-b8acd9280356","Type":"ContainerDied","Data":"c4e366e272363beb523cf40ee21631ae5804252f9f976436f4bc9bc3040b1fc5"} Dec 03 12:55:38 crc kubenswrapper[5052]: I1203 12:55:38.535666 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="crc-storage/crc-storage-crc-z9h5z" Dec 03 12:55:38 crc kubenswrapper[5052]: I1203 12:55:38.697053 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/337d1568-3061-4799-bc8b-b8acd9280356-node-mnt\") pod \"337d1568-3061-4799-bc8b-b8acd9280356\" (UID: \"337d1568-3061-4799-bc8b-b8acd9280356\") " Dec 03 12:55:38 crc kubenswrapper[5052]: I1203 12:55:38.697188 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/337d1568-3061-4799-bc8b-b8acd9280356-node-mnt" (OuterVolumeSpecName: "node-mnt") pod "337d1568-3061-4799-bc8b-b8acd9280356" (UID: "337d1568-3061-4799-bc8b-b8acd9280356"). InnerVolumeSpecName "node-mnt". PluginName "kubernetes.io/host-path", VolumeGidValue "" Dec 03 12:55:38 crc kubenswrapper[5052]: I1203 12:55:38.697228 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r8r9h\" (UniqueName: \"kubernetes.io/projected/337d1568-3061-4799-bc8b-b8acd9280356-kube-api-access-r8r9h\") pod \"337d1568-3061-4799-bc8b-b8acd9280356\" (UID: \"337d1568-3061-4799-bc8b-b8acd9280356\") " Dec 03 12:55:38 crc kubenswrapper[5052]: I1203 12:55:38.697303 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/337d1568-3061-4799-bc8b-b8acd9280356-crc-storage\") pod \"337d1568-3061-4799-bc8b-b8acd9280356\" (UID: \"337d1568-3061-4799-bc8b-b8acd9280356\") " Dec 03 12:55:38 crc kubenswrapper[5052]: I1203 12:55:38.697616 5052 reconciler_common.go:293] "Volume detached for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/337d1568-3061-4799-bc8b-b8acd9280356-node-mnt\") on node \"crc\" DevicePath \"\"" Dec 03 12:55:38 crc kubenswrapper[5052]: I1203 12:55:38.703947 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/337d1568-3061-4799-bc8b-b8acd9280356-kube-api-access-r8r9h" (OuterVolumeSpecName: "kube-api-access-r8r9h") pod "337d1568-3061-4799-bc8b-b8acd9280356" (UID: "337d1568-3061-4799-bc8b-b8acd9280356"). InnerVolumeSpecName "kube-api-access-r8r9h". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:55:38 crc kubenswrapper[5052]: I1203 12:55:38.715307 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/337d1568-3061-4799-bc8b-b8acd9280356-crc-storage" (OuterVolumeSpecName: "crc-storage") pod "337d1568-3061-4799-bc8b-b8acd9280356" (UID: "337d1568-3061-4799-bc8b-b8acd9280356"). InnerVolumeSpecName "crc-storage". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:55:38 crc kubenswrapper[5052]: I1203 12:55:38.799401 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-r8r9h\" (UniqueName: \"kubernetes.io/projected/337d1568-3061-4799-bc8b-b8acd9280356-kube-api-access-r8r9h\") on node \"crc\" DevicePath \"\"" Dec 03 12:55:38 crc kubenswrapper[5052]: I1203 12:55:38.799453 5052 reconciler_common.go:293] "Volume detached for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/337d1568-3061-4799-bc8b-b8acd9280356-crc-storage\") on node \"crc\" DevicePath \"\"" Dec 03 12:55:39 crc kubenswrapper[5052]: I1203 12:55:39.268210 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="crc-storage/crc-storage-crc-z9h5z" event={"ID":"337d1568-3061-4799-bc8b-b8acd9280356","Type":"ContainerDied","Data":"b7ad2e9f92f5bbe97296af1e0dd4d461e28e2164c378e4d83994bd639a1934ba"} Dec 03 12:55:39 crc kubenswrapper[5052]: I1203 12:55:39.268253 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b7ad2e9f92f5bbe97296af1e0dd4d461e28e2164c378e4d83994bd639a1934ba" Dec 03 12:55:39 crc kubenswrapper[5052]: I1203 12:55:39.268265 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="crc-storage/crc-storage-crc-z9h5z" Dec 03 12:55:41 crc kubenswrapper[5052]: I1203 12:55:41.602981 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openshift-marketplace/redhat-marketplace-vxs42" Dec 03 12:55:41 crc kubenswrapper[5052]: I1203 12:55:41.603331 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-marketplace/redhat-marketplace-vxs42" Dec 03 12:55:41 crc kubenswrapper[5052]: I1203 12:55:41.646300 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-marketplace/redhat-marketplace-vxs42" Dec 03 12:55:42 crc kubenswrapper[5052]: I1203 12:55:42.342125 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-marketplace/redhat-marketplace-vxs42" Dec 03 12:55:42 crc kubenswrapper[5052]: I1203 12:55:42.390297 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openshift-marketplace/redhat-marketplace-vxs42"] Dec 03 12:55:42 crc kubenswrapper[5052]: I1203 12:55:42.739650 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-5b68c79f89-gtfxz"] Dec 03 12:55:42 crc kubenswrapper[5052]: E1203 12:55:42.740020 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="337d1568-3061-4799-bc8b-b8acd9280356" containerName="storage" Dec 03 12:55:42 crc kubenswrapper[5052]: I1203 12:55:42.740037 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="337d1568-3061-4799-bc8b-b8acd9280356" containerName="storage" Dec 03 12:55:42 crc kubenswrapper[5052]: I1203 12:55:42.740204 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="337d1568-3061-4799-bc8b-b8acd9280356" containerName="storage" Dec 03 12:55:42 crc kubenswrapper[5052]: I1203 12:55:42.741173 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-5b68c79f89-gtfxz" Dec 03 12:55:42 crc kubenswrapper[5052]: I1203 12:55:42.744360 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"openstack-edpm-ipam" Dec 03 12:55:42 crc kubenswrapper[5052]: I1203 12:55:42.757243 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-5b68c79f89-gtfxz"] Dec 03 12:55:42 crc kubenswrapper[5052]: I1203 12:55:42.826124 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-5b68c79f89-gtfxz"] Dec 03 12:55:42 crc kubenswrapper[5052]: E1203 12:55:42.826697 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[config dnsmasq-svc kube-api-access-6mxbj openstack-edpm-ipam], unattached volumes=[], failed to process volumes=[]: context canceled" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-5b68c79f89-gtfxz" podUID="372d1c7c-c698-4db9-ac41-267d1dd4dd7a" Dec 03 12:55:42 crc kubenswrapper[5052]: I1203 12:55:42.852903 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-79667f9c49-dtkj9"] Dec 03 12:55:42 crc kubenswrapper[5052]: I1203 12:55:42.854466 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-79667f9c49-dtkj9" Dec 03 12:55:42 crc kubenswrapper[5052]: I1203 12:55:42.860346 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/372d1c7c-c698-4db9-ac41-267d1dd4dd7a-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-5b68c79f89-gtfxz\" (UID: \"372d1c7c-c698-4db9-ac41-267d1dd4dd7a\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-5b68c79f89-gtfxz" Dec 03 12:55:42 crc kubenswrapper[5052]: I1203 12:55:42.860427 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6mxbj\" (UniqueName: \"kubernetes.io/projected/372d1c7c-c698-4db9-ac41-267d1dd4dd7a-kube-api-access-6mxbj\") pod \"dnsmasq-dnsmasq-5b68c79f89-gtfxz\" (UID: \"372d1c7c-c698-4db9-ac41-267d1dd4dd7a\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-5b68c79f89-gtfxz" Dec 03 12:55:42 crc kubenswrapper[5052]: I1203 12:55:42.860535 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/372d1c7c-c698-4db9-ac41-267d1dd4dd7a-config\") pod \"dnsmasq-dnsmasq-5b68c79f89-gtfxz\" (UID: \"372d1c7c-c698-4db9-ac41-267d1dd4dd7a\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-5b68c79f89-gtfxz" Dec 03 12:55:42 crc kubenswrapper[5052]: I1203 12:55:42.860594 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openstack-edpm-ipam\" (UniqueName: \"kubernetes.io/configmap/372d1c7c-c698-4db9-ac41-267d1dd4dd7a-openstack-edpm-ipam\") pod \"dnsmasq-dnsmasq-5b68c79f89-gtfxz\" (UID: \"372d1c7c-c698-4db9-ac41-267d1dd4dd7a\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-5b68c79f89-gtfxz" Dec 03 12:55:42 crc kubenswrapper[5052]: I1203 12:55:42.868247 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-79667f9c49-dtkj9"] Dec 03 12:55:42 crc kubenswrapper[5052]: I1203 12:55:42.962155 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/372d1c7c-c698-4db9-ac41-267d1dd4dd7a-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-5b68c79f89-gtfxz\" (UID: \"372d1c7c-c698-4db9-ac41-267d1dd4dd7a\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-5b68c79f89-gtfxz" Dec 03 12:55:42 crc kubenswrapper[5052]: I1203 12:55:42.962204 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d6e33718-c0d4-4c6e-b764-cc9d7506facb-config\") pod \"dnsmasq-dnsmasq-79667f9c49-dtkj9\" (UID: \"d6e33718-c0d4-4c6e-b764-cc9d7506facb\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-79667f9c49-dtkj9" Dec 03 12:55:42 crc kubenswrapper[5052]: I1203 12:55:42.962243 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w8w8d\" (UniqueName: \"kubernetes.io/projected/d6e33718-c0d4-4c6e-b764-cc9d7506facb-kube-api-access-w8w8d\") pod \"dnsmasq-dnsmasq-79667f9c49-dtkj9\" (UID: \"d6e33718-c0d4-4c6e-b764-cc9d7506facb\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-79667f9c49-dtkj9" Dec 03 12:55:42 crc kubenswrapper[5052]: I1203 12:55:42.962274 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/d6e33718-c0d4-4c6e-b764-cc9d7506facb-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-79667f9c49-dtkj9\" (UID: \"d6e33718-c0d4-4c6e-b764-cc9d7506facb\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-79667f9c49-dtkj9" Dec 03 12:55:42 crc kubenswrapper[5052]: I1203 12:55:42.962299 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-6mxbj\" (UniqueName: \"kubernetes.io/projected/372d1c7c-c698-4db9-ac41-267d1dd4dd7a-kube-api-access-6mxbj\") pod \"dnsmasq-dnsmasq-5b68c79f89-gtfxz\" (UID: \"372d1c7c-c698-4db9-ac41-267d1dd4dd7a\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-5b68c79f89-gtfxz" Dec 03 12:55:42 crc kubenswrapper[5052]: I1203 12:55:42.962331 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/372d1c7c-c698-4db9-ac41-267d1dd4dd7a-config\") pod \"dnsmasq-dnsmasq-5b68c79f89-gtfxz\" (UID: \"372d1c7c-c698-4db9-ac41-267d1dd4dd7a\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-5b68c79f89-gtfxz" Dec 03 12:55:42 crc kubenswrapper[5052]: I1203 12:55:42.962397 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openstack-edpm-ipam\" (UniqueName: \"kubernetes.io/configmap/d6e33718-c0d4-4c6e-b764-cc9d7506facb-openstack-edpm-ipam\") pod \"dnsmasq-dnsmasq-79667f9c49-dtkj9\" (UID: \"d6e33718-c0d4-4c6e-b764-cc9d7506facb\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-79667f9c49-dtkj9" Dec 03 12:55:42 crc kubenswrapper[5052]: I1203 12:55:42.962505 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"openstack-edpm-ipam\" (UniqueName: \"kubernetes.io/configmap/372d1c7c-c698-4db9-ac41-267d1dd4dd7a-openstack-edpm-ipam\") pod \"dnsmasq-dnsmasq-5b68c79f89-gtfxz\" (UID: \"372d1c7c-c698-4db9-ac41-267d1dd4dd7a\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-5b68c79f89-gtfxz" Dec 03 12:55:42 crc kubenswrapper[5052]: I1203 12:55:42.963145 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/372d1c7c-c698-4db9-ac41-267d1dd4dd7a-config\") pod \"dnsmasq-dnsmasq-5b68c79f89-gtfxz\" (UID: \"372d1c7c-c698-4db9-ac41-267d1dd4dd7a\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-5b68c79f89-gtfxz" Dec 03 12:55:42 crc kubenswrapper[5052]: I1203 12:55:42.963148 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/372d1c7c-c698-4db9-ac41-267d1dd4dd7a-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-5b68c79f89-gtfxz\" (UID: \"372d1c7c-c698-4db9-ac41-267d1dd4dd7a\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-5b68c79f89-gtfxz" Dec 03 12:55:42 crc kubenswrapper[5052]: I1203 12:55:42.963358 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"openstack-edpm-ipam\" (UniqueName: \"kubernetes.io/configmap/372d1c7c-c698-4db9-ac41-267d1dd4dd7a-openstack-edpm-ipam\") pod \"dnsmasq-dnsmasq-5b68c79f89-gtfxz\" (UID: \"372d1c7c-c698-4db9-ac41-267d1dd4dd7a\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-5b68c79f89-gtfxz" Dec 03 12:55:42 crc kubenswrapper[5052]: I1203 12:55:42.980253 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-6mxbj\" (UniqueName: \"kubernetes.io/projected/372d1c7c-c698-4db9-ac41-267d1dd4dd7a-kube-api-access-6mxbj\") pod \"dnsmasq-dnsmasq-5b68c79f89-gtfxz\" (UID: \"372d1c7c-c698-4db9-ac41-267d1dd4dd7a\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-5b68c79f89-gtfxz" Dec 03 12:55:43 crc kubenswrapper[5052]: I1203 12:55:43.063526 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d6e33718-c0d4-4c6e-b764-cc9d7506facb-config\") pod \"dnsmasq-dnsmasq-79667f9c49-dtkj9\" (UID: \"d6e33718-c0d4-4c6e-b764-cc9d7506facb\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-79667f9c49-dtkj9" Dec 03 12:55:43 crc kubenswrapper[5052]: I1203 12:55:43.063580 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-w8w8d\" (UniqueName: \"kubernetes.io/projected/d6e33718-c0d4-4c6e-b764-cc9d7506facb-kube-api-access-w8w8d\") pod \"dnsmasq-dnsmasq-79667f9c49-dtkj9\" (UID: \"d6e33718-c0d4-4c6e-b764-cc9d7506facb\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-79667f9c49-dtkj9" Dec 03 12:55:43 crc kubenswrapper[5052]: I1203 12:55:43.063615 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/d6e33718-c0d4-4c6e-b764-cc9d7506facb-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-79667f9c49-dtkj9\" (UID: \"d6e33718-c0d4-4c6e-b764-cc9d7506facb\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-79667f9c49-dtkj9" Dec 03 12:55:43 crc kubenswrapper[5052]: I1203 12:55:43.063657 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"openstack-edpm-ipam\" (UniqueName: \"kubernetes.io/configmap/d6e33718-c0d4-4c6e-b764-cc9d7506facb-openstack-edpm-ipam\") pod \"dnsmasq-dnsmasq-79667f9c49-dtkj9\" (UID: \"d6e33718-c0d4-4c6e-b764-cc9d7506facb\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-79667f9c49-dtkj9" Dec 03 12:55:43 crc kubenswrapper[5052]: I1203 12:55:43.064433 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d6e33718-c0d4-4c6e-b764-cc9d7506facb-config\") pod \"dnsmasq-dnsmasq-79667f9c49-dtkj9\" (UID: \"d6e33718-c0d4-4c6e-b764-cc9d7506facb\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-79667f9c49-dtkj9" Dec 03 12:55:43 crc kubenswrapper[5052]: I1203 12:55:43.064559 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"openstack-edpm-ipam\" (UniqueName: \"kubernetes.io/configmap/d6e33718-c0d4-4c6e-b764-cc9d7506facb-openstack-edpm-ipam\") pod \"dnsmasq-dnsmasq-79667f9c49-dtkj9\" (UID: \"d6e33718-c0d4-4c6e-b764-cc9d7506facb\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-79667f9c49-dtkj9" Dec 03 12:55:43 crc kubenswrapper[5052]: I1203 12:55:43.064650 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/d6e33718-c0d4-4c6e-b764-cc9d7506facb-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-79667f9c49-dtkj9\" (UID: \"d6e33718-c0d4-4c6e-b764-cc9d7506facb\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-79667f9c49-dtkj9" Dec 03 12:55:43 crc kubenswrapper[5052]: I1203 12:55:43.081897 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-w8w8d\" (UniqueName: \"kubernetes.io/projected/d6e33718-c0d4-4c6e-b764-cc9d7506facb-kube-api-access-w8w8d\") pod \"dnsmasq-dnsmasq-79667f9c49-dtkj9\" (UID: \"d6e33718-c0d4-4c6e-b764-cc9d7506facb\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-79667f9c49-dtkj9" Dec 03 12:55:43 crc kubenswrapper[5052]: I1203 12:55:43.173237 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-79667f9c49-dtkj9" Dec 03 12:55:43 crc kubenswrapper[5052]: I1203 12:55:43.299620 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-5b68c79f89-gtfxz" Dec 03 12:55:43 crc kubenswrapper[5052]: I1203 12:55:43.311492 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-5b68c79f89-gtfxz" Dec 03 12:55:43 crc kubenswrapper[5052]: I1203 12:55:43.468978 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"openstack-edpm-ipam\" (UniqueName: \"kubernetes.io/configmap/372d1c7c-c698-4db9-ac41-267d1dd4dd7a-openstack-edpm-ipam\") pod \"372d1c7c-c698-4db9-ac41-267d1dd4dd7a\" (UID: \"372d1c7c-c698-4db9-ac41-267d1dd4dd7a\") " Dec 03 12:55:43 crc kubenswrapper[5052]: I1203 12:55:43.469036 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/372d1c7c-c698-4db9-ac41-267d1dd4dd7a-config\") pod \"372d1c7c-c698-4db9-ac41-267d1dd4dd7a\" (UID: \"372d1c7c-c698-4db9-ac41-267d1dd4dd7a\") " Dec 03 12:55:43 crc kubenswrapper[5052]: I1203 12:55:43.469063 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/372d1c7c-c698-4db9-ac41-267d1dd4dd7a-dnsmasq-svc\") pod \"372d1c7c-c698-4db9-ac41-267d1dd4dd7a\" (UID: \"372d1c7c-c698-4db9-ac41-267d1dd4dd7a\") " Dec 03 12:55:43 crc kubenswrapper[5052]: I1203 12:55:43.469141 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6mxbj\" (UniqueName: \"kubernetes.io/projected/372d1c7c-c698-4db9-ac41-267d1dd4dd7a-kube-api-access-6mxbj\") pod \"372d1c7c-c698-4db9-ac41-267d1dd4dd7a\" (UID: \"372d1c7c-c698-4db9-ac41-267d1dd4dd7a\") " Dec 03 12:55:43 crc kubenswrapper[5052]: I1203 12:55:43.469576 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/372d1c7c-c698-4db9-ac41-267d1dd4dd7a-openstack-edpm-ipam" (OuterVolumeSpecName: "openstack-edpm-ipam") pod "372d1c7c-c698-4db9-ac41-267d1dd4dd7a" (UID: "372d1c7c-c698-4db9-ac41-267d1dd4dd7a"). InnerVolumeSpecName "openstack-edpm-ipam". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:55:43 crc kubenswrapper[5052]: I1203 12:55:43.469616 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/372d1c7c-c698-4db9-ac41-267d1dd4dd7a-config" (OuterVolumeSpecName: "config") pod "372d1c7c-c698-4db9-ac41-267d1dd4dd7a" (UID: "372d1c7c-c698-4db9-ac41-267d1dd4dd7a"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:55:43 crc kubenswrapper[5052]: I1203 12:55:43.469707 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/372d1c7c-c698-4db9-ac41-267d1dd4dd7a-dnsmasq-svc" (OuterVolumeSpecName: "dnsmasq-svc") pod "372d1c7c-c698-4db9-ac41-267d1dd4dd7a" (UID: "372d1c7c-c698-4db9-ac41-267d1dd4dd7a"). InnerVolumeSpecName "dnsmasq-svc". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:55:43 crc kubenswrapper[5052]: I1203 12:55:43.470286 5052 reconciler_common.go:293] "Volume detached for volume \"openstack-edpm-ipam\" (UniqueName: \"kubernetes.io/configmap/372d1c7c-c698-4db9-ac41-267d1dd4dd7a-openstack-edpm-ipam\") on node \"crc\" DevicePath \"\"" Dec 03 12:55:43 crc kubenswrapper[5052]: I1203 12:55:43.470345 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/372d1c7c-c698-4db9-ac41-267d1dd4dd7a-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:55:43 crc kubenswrapper[5052]: I1203 12:55:43.470359 5052 reconciler_common.go:293] "Volume detached for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/372d1c7c-c698-4db9-ac41-267d1dd4dd7a-dnsmasq-svc\") on node \"crc\" DevicePath \"\"" Dec 03 12:55:43 crc kubenswrapper[5052]: I1203 12:55:43.473685 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/372d1c7c-c698-4db9-ac41-267d1dd4dd7a-kube-api-access-6mxbj" (OuterVolumeSpecName: "kube-api-access-6mxbj") pod "372d1c7c-c698-4db9-ac41-267d1dd4dd7a" (UID: "372d1c7c-c698-4db9-ac41-267d1dd4dd7a"). InnerVolumeSpecName "kube-api-access-6mxbj". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:55:43 crc kubenswrapper[5052]: I1203 12:55:43.572040 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-6mxbj\" (UniqueName: \"kubernetes.io/projected/372d1c7c-c698-4db9-ac41-267d1dd4dd7a-kube-api-access-6mxbj\") on node \"crc\" DevicePath \"\"" Dec 03 12:55:43 crc kubenswrapper[5052]: I1203 12:55:43.621802 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-79667f9c49-dtkj9"] Dec 03 12:55:44 crc kubenswrapper[5052]: I1203 12:55:44.095545 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-79667f9c49-dtkj9"] Dec 03 12:55:44 crc kubenswrapper[5052]: I1203 12:55:44.310733 5052 generic.go:334] "Generic (PLEG): container finished" podID="d6e33718-c0d4-4c6e-b764-cc9d7506facb" containerID="9fbce0d32ede246cb7f606b05fc16925371a3bbb91e8feefca3d305d73a5d20e" exitCode=0 Dec 03 12:55:44 crc kubenswrapper[5052]: I1203 12:55:44.310834 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-79667f9c49-dtkj9" event={"ID":"d6e33718-c0d4-4c6e-b764-cc9d7506facb","Type":"ContainerDied","Data":"9fbce0d32ede246cb7f606b05fc16925371a3bbb91e8feefca3d305d73a5d20e"} Dec 03 12:55:44 crc kubenswrapper[5052]: I1203 12:55:44.310874 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-79667f9c49-dtkj9" event={"ID":"d6e33718-c0d4-4c6e-b764-cc9d7506facb","Type":"ContainerStarted","Data":"0791032e4451946f2f84fcc244f6b5ff1529d693be7934196451ef24a9e07686"} Dec 03 12:55:44 crc kubenswrapper[5052]: I1203 12:55:44.311063 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-5b68c79f89-gtfxz" Dec 03 12:55:44 crc kubenswrapper[5052]: I1203 12:55:44.311069 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openshift-marketplace/redhat-marketplace-vxs42" podUID="5f0a6e44-08a6-454e-8580-1c33335219b3" containerName="registry-server" containerID="cri-o://6d4d7eaec35ce7907e463eb02c5339e1763ae84419bc401edcbc5f956a4f58fa" gracePeriod=2 Dec 03 12:55:44 crc kubenswrapper[5052]: E1203 12:55:44.383440 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-edpm-ipam: configmap "openstack-edpm-ipam" not found Dec 03 12:55:44 crc kubenswrapper[5052]: E1203 12:55:44.383928 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d6e33718-c0d4-4c6e-b764-cc9d7506facb-openstack-edpm-ipam podName:d6e33718-c0d4-4c6e-b764-cc9d7506facb nodeName:}" failed. No retries permitted until 2025-12-03 12:55:44.883902688 +0000 UTC m=+5735.472144452 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openstack-edpm-ipam" (UniqueName: "kubernetes.io/configmap/d6e33718-c0d4-4c6e-b764-cc9d7506facb-openstack-edpm-ipam") pod "dnsmasq-dnsmasq-79667f9c49-dtkj9" (UID: "d6e33718-c0d4-4c6e-b764-cc9d7506facb") : configmap "openstack-edpm-ipam" not found Dec 03 12:55:44 crc kubenswrapper[5052]: I1203 12:55:44.529195 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-5b68c79f89-gtfxz"] Dec 03 12:55:44 crc kubenswrapper[5052]: I1203 12:55:44.536698 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-5b68c79f89-gtfxz"] Dec 03 12:55:44 crc kubenswrapper[5052]: I1203 12:55:44.671882 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/redhat-marketplace-vxs42" Dec 03 12:55:44 crc kubenswrapper[5052]: I1203 12:55:44.789529 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nhgnk\" (UniqueName: \"kubernetes.io/projected/5f0a6e44-08a6-454e-8580-1c33335219b3-kube-api-access-nhgnk\") pod \"5f0a6e44-08a6-454e-8580-1c33335219b3\" (UID: \"5f0a6e44-08a6-454e-8580-1c33335219b3\") " Dec 03 12:55:44 crc kubenswrapper[5052]: I1203 12:55:44.789726 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/5f0a6e44-08a6-454e-8580-1c33335219b3-catalog-content\") pod \"5f0a6e44-08a6-454e-8580-1c33335219b3\" (UID: \"5f0a6e44-08a6-454e-8580-1c33335219b3\") " Dec 03 12:55:44 crc kubenswrapper[5052]: I1203 12:55:44.789773 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/5f0a6e44-08a6-454e-8580-1c33335219b3-utilities\") pod \"5f0a6e44-08a6-454e-8580-1c33335219b3\" (UID: \"5f0a6e44-08a6-454e-8580-1c33335219b3\") " Dec 03 12:55:44 crc kubenswrapper[5052]: I1203 12:55:44.790747 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f0a6e44-08a6-454e-8580-1c33335219b3-utilities" (OuterVolumeSpecName: "utilities") pod "5f0a6e44-08a6-454e-8580-1c33335219b3" (UID: "5f0a6e44-08a6-454e-8580-1c33335219b3"). InnerVolumeSpecName "utilities". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:55:44 crc kubenswrapper[5052]: I1203 12:55:44.797367 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5f0a6e44-08a6-454e-8580-1c33335219b3-kube-api-access-nhgnk" (OuterVolumeSpecName: "kube-api-access-nhgnk") pod "5f0a6e44-08a6-454e-8580-1c33335219b3" (UID: "5f0a6e44-08a6-454e-8580-1c33335219b3"). InnerVolumeSpecName "kube-api-access-nhgnk". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:55:44 crc kubenswrapper[5052]: I1203 12:55:44.810794 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f0a6e44-08a6-454e-8580-1c33335219b3-catalog-content" (OuterVolumeSpecName: "catalog-content") pod "5f0a6e44-08a6-454e-8580-1c33335219b3" (UID: "5f0a6e44-08a6-454e-8580-1c33335219b3"). InnerVolumeSpecName "catalog-content". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:55:44 crc kubenswrapper[5052]: I1203 12:55:44.892233 5052 reconciler_common.go:293] "Volume detached for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/5f0a6e44-08a6-454e-8580-1c33335219b3-catalog-content\") on node \"crc\" DevicePath \"\"" Dec 03 12:55:44 crc kubenswrapper[5052]: I1203 12:55:44.892270 5052 reconciler_common.go:293] "Volume detached for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/5f0a6e44-08a6-454e-8580-1c33335219b3-utilities\") on node \"crc\" DevicePath \"\"" Dec 03 12:55:44 crc kubenswrapper[5052]: I1203 12:55:44.892288 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-nhgnk\" (UniqueName: \"kubernetes.io/projected/5f0a6e44-08a6-454e-8580-1c33335219b3-kube-api-access-nhgnk\") on node \"crc\" DevicePath \"\"" Dec 03 12:55:44 crc kubenswrapper[5052]: E1203 12:55:44.892349 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/openstack-edpm-ipam: configmap "openstack-edpm-ipam" not found Dec 03 12:55:44 crc kubenswrapper[5052]: E1203 12:55:44.892417 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d6e33718-c0d4-4c6e-b764-cc9d7506facb-openstack-edpm-ipam podName:d6e33718-c0d4-4c6e-b764-cc9d7506facb nodeName:}" failed. No retries permitted until 2025-12-03 12:55:45.89239858 +0000 UTC m=+5736.480640344 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "openstack-edpm-ipam" (UniqueName: "kubernetes.io/configmap/d6e33718-c0d4-4c6e-b764-cc9d7506facb-openstack-edpm-ipam") pod "dnsmasq-dnsmasq-79667f9c49-dtkj9" (UID: "d6e33718-c0d4-4c6e-b764-cc9d7506facb") : configmap "openstack-edpm-ipam" not found Dec 03 12:55:45 crc kubenswrapper[5052]: I1203 12:55:45.321918 5052 generic.go:334] "Generic (PLEG): container finished" podID="5f0a6e44-08a6-454e-8580-1c33335219b3" containerID="6d4d7eaec35ce7907e463eb02c5339e1763ae84419bc401edcbc5f956a4f58fa" exitCode=0 Dec 03 12:55:45 crc kubenswrapper[5052]: I1203 12:55:45.321984 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/redhat-marketplace-vxs42" Dec 03 12:55:45 crc kubenswrapper[5052]: I1203 12:55:45.322000 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-marketplace-vxs42" event={"ID":"5f0a6e44-08a6-454e-8580-1c33335219b3","Type":"ContainerDied","Data":"6d4d7eaec35ce7907e463eb02c5339e1763ae84419bc401edcbc5f956a4f58fa"} Dec 03 12:55:45 crc kubenswrapper[5052]: I1203 12:55:45.322028 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-marketplace-vxs42" event={"ID":"5f0a6e44-08a6-454e-8580-1c33335219b3","Type":"ContainerDied","Data":"69e44b423bf9087390deb6500ab45e4beba0bb286287c7246bc0cf13c588fd00"} Dec 03 12:55:45 crc kubenswrapper[5052]: I1203 12:55:45.322063 5052 scope.go:117] "RemoveContainer" containerID="6d4d7eaec35ce7907e463eb02c5339e1763ae84419bc401edcbc5f956a4f58fa" Dec 03 12:55:45 crc kubenswrapper[5052]: I1203 12:55:45.324321 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-79667f9c49-dtkj9" event={"ID":"d6e33718-c0d4-4c6e-b764-cc9d7506facb","Type":"ContainerStarted","Data":"bb662c2d445a4909b0d4cff01b0d0c2bee8bdec10afab01b6e9c64a98e974702"} Dec 03 12:55:45 crc kubenswrapper[5052]: I1203 12:55:45.324543 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-79667f9c49-dtkj9" podUID="d6e33718-c0d4-4c6e-b764-cc9d7506facb" containerName="dnsmasq-dns" containerID="cri-o://bb662c2d445a4909b0d4cff01b0d0c2bee8bdec10afab01b6e9c64a98e974702" gracePeriod=10 Dec 03 12:55:45 crc kubenswrapper[5052]: I1203 12:55:45.324564 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-79667f9c49-dtkj9" Dec 03 12:55:45 crc kubenswrapper[5052]: I1203 12:55:45.348700 5052 scope.go:117] "RemoveContainer" containerID="7a4643f480701372ff7a2c0e33933975605568c6046e8665fe0d5caa11ba3e66" Dec 03 12:55:45 crc kubenswrapper[5052]: I1203 12:55:45.349060 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-79667f9c49-dtkj9" podStartSLOduration=3.349043329 podStartE2EDuration="3.349043329s" podCreationTimestamp="2025-12-03 12:55:42 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:55:45.343688846 +0000 UTC m=+5735.931930610" watchObservedRunningTime="2025-12-03 12:55:45.349043329 +0000 UTC m=+5735.937285093" Dec 03 12:55:45 crc kubenswrapper[5052]: I1203 12:55:45.370183 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openshift-marketplace/redhat-marketplace-vxs42"] Dec 03 12:55:45 crc kubenswrapper[5052]: I1203 12:55:45.376011 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openshift-marketplace/redhat-marketplace-vxs42"] Dec 03 12:55:45 crc kubenswrapper[5052]: I1203 12:55:45.411169 5052 scope.go:117] "RemoveContainer" containerID="e36fa0e3ea12d83fd9cc0e0a2e39ec6811e8022e1a80e53458e96fd128c7c897" Dec 03 12:55:45 crc kubenswrapper[5052]: I1203 12:55:45.447805 5052 scope.go:117] "RemoveContainer" containerID="6d4d7eaec35ce7907e463eb02c5339e1763ae84419bc401edcbc5f956a4f58fa" Dec 03 12:55:45 crc kubenswrapper[5052]: E1203 12:55:45.449236 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6d4d7eaec35ce7907e463eb02c5339e1763ae84419bc401edcbc5f956a4f58fa\": container with ID starting with 6d4d7eaec35ce7907e463eb02c5339e1763ae84419bc401edcbc5f956a4f58fa not found: ID does not exist" containerID="6d4d7eaec35ce7907e463eb02c5339e1763ae84419bc401edcbc5f956a4f58fa" Dec 03 12:55:45 crc kubenswrapper[5052]: I1203 12:55:45.449286 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6d4d7eaec35ce7907e463eb02c5339e1763ae84419bc401edcbc5f956a4f58fa"} err="failed to get container status \"6d4d7eaec35ce7907e463eb02c5339e1763ae84419bc401edcbc5f956a4f58fa\": rpc error: code = NotFound desc = could not find container \"6d4d7eaec35ce7907e463eb02c5339e1763ae84419bc401edcbc5f956a4f58fa\": container with ID starting with 6d4d7eaec35ce7907e463eb02c5339e1763ae84419bc401edcbc5f956a4f58fa not found: ID does not exist" Dec 03 12:55:45 crc kubenswrapper[5052]: I1203 12:55:45.449332 5052 scope.go:117] "RemoveContainer" containerID="7a4643f480701372ff7a2c0e33933975605568c6046e8665fe0d5caa11ba3e66" Dec 03 12:55:45 crc kubenswrapper[5052]: E1203 12:55:45.449983 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7a4643f480701372ff7a2c0e33933975605568c6046e8665fe0d5caa11ba3e66\": container with ID starting with 7a4643f480701372ff7a2c0e33933975605568c6046e8665fe0d5caa11ba3e66 not found: ID does not exist" containerID="7a4643f480701372ff7a2c0e33933975605568c6046e8665fe0d5caa11ba3e66" Dec 03 12:55:45 crc kubenswrapper[5052]: I1203 12:55:45.450026 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7a4643f480701372ff7a2c0e33933975605568c6046e8665fe0d5caa11ba3e66"} err="failed to get container status \"7a4643f480701372ff7a2c0e33933975605568c6046e8665fe0d5caa11ba3e66\": rpc error: code = NotFound desc = could not find container \"7a4643f480701372ff7a2c0e33933975605568c6046e8665fe0d5caa11ba3e66\": container with ID starting with 7a4643f480701372ff7a2c0e33933975605568c6046e8665fe0d5caa11ba3e66 not found: ID does not exist" Dec 03 12:55:45 crc kubenswrapper[5052]: I1203 12:55:45.450060 5052 scope.go:117] "RemoveContainer" containerID="e36fa0e3ea12d83fd9cc0e0a2e39ec6811e8022e1a80e53458e96fd128c7c897" Dec 03 12:55:45 crc kubenswrapper[5052]: E1203 12:55:45.451564 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e36fa0e3ea12d83fd9cc0e0a2e39ec6811e8022e1a80e53458e96fd128c7c897\": container with ID starting with e36fa0e3ea12d83fd9cc0e0a2e39ec6811e8022e1a80e53458e96fd128c7c897 not found: ID does not exist" containerID="e36fa0e3ea12d83fd9cc0e0a2e39ec6811e8022e1a80e53458e96fd128c7c897" Dec 03 12:55:45 crc kubenswrapper[5052]: I1203 12:55:45.451595 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e36fa0e3ea12d83fd9cc0e0a2e39ec6811e8022e1a80e53458e96fd128c7c897"} err="failed to get container status \"e36fa0e3ea12d83fd9cc0e0a2e39ec6811e8022e1a80e53458e96fd128c7c897\": rpc error: code = NotFound desc = could not find container \"e36fa0e3ea12d83fd9cc0e0a2e39ec6811e8022e1a80e53458e96fd128c7c897\": container with ID starting with e36fa0e3ea12d83fd9cc0e0a2e39ec6811e8022e1a80e53458e96fd128c7c897 not found: ID does not exist" Dec 03 12:55:45 crc kubenswrapper[5052]: I1203 12:55:45.664132 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-79667f9c49-dtkj9" Dec 03 12:55:45 crc kubenswrapper[5052]: I1203 12:55:45.728777 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="372d1c7c-c698-4db9-ac41-267d1dd4dd7a" path="/var/lib/kubelet/pods/372d1c7c-c698-4db9-ac41-267d1dd4dd7a/volumes" Dec 03 12:55:45 crc kubenswrapper[5052]: I1203 12:55:45.729391 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5f0a6e44-08a6-454e-8580-1c33335219b3" path="/var/lib/kubelet/pods/5f0a6e44-08a6-454e-8580-1c33335219b3/volumes" Dec 03 12:55:45 crc kubenswrapper[5052]: I1203 12:55:45.825272 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d6e33718-c0d4-4c6e-b764-cc9d7506facb-config\") pod \"d6e33718-c0d4-4c6e-b764-cc9d7506facb\" (UID: \"d6e33718-c0d4-4c6e-b764-cc9d7506facb\") " Dec 03 12:55:45 crc kubenswrapper[5052]: I1203 12:55:45.825439 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w8w8d\" (UniqueName: \"kubernetes.io/projected/d6e33718-c0d4-4c6e-b764-cc9d7506facb-kube-api-access-w8w8d\") pod \"d6e33718-c0d4-4c6e-b764-cc9d7506facb\" (UID: \"d6e33718-c0d4-4c6e-b764-cc9d7506facb\") " Dec 03 12:55:45 crc kubenswrapper[5052]: I1203 12:55:45.825478 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"openstack-edpm-ipam\" (UniqueName: \"kubernetes.io/configmap/d6e33718-c0d4-4c6e-b764-cc9d7506facb-openstack-edpm-ipam\") pod \"d6e33718-c0d4-4c6e-b764-cc9d7506facb\" (UID: \"d6e33718-c0d4-4c6e-b764-cc9d7506facb\") " Dec 03 12:55:45 crc kubenswrapper[5052]: I1203 12:55:45.825609 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/d6e33718-c0d4-4c6e-b764-cc9d7506facb-dnsmasq-svc\") pod \"d6e33718-c0d4-4c6e-b764-cc9d7506facb\" (UID: \"d6e33718-c0d4-4c6e-b764-cc9d7506facb\") " Dec 03 12:55:45 crc kubenswrapper[5052]: I1203 12:55:45.830682 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d6e33718-c0d4-4c6e-b764-cc9d7506facb-kube-api-access-w8w8d" (OuterVolumeSpecName: "kube-api-access-w8w8d") pod "d6e33718-c0d4-4c6e-b764-cc9d7506facb" (UID: "d6e33718-c0d4-4c6e-b764-cc9d7506facb"). InnerVolumeSpecName "kube-api-access-w8w8d". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:55:45 crc kubenswrapper[5052]: I1203 12:55:45.861809 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d6e33718-c0d4-4c6e-b764-cc9d7506facb-dnsmasq-svc" (OuterVolumeSpecName: "dnsmasq-svc") pod "d6e33718-c0d4-4c6e-b764-cc9d7506facb" (UID: "d6e33718-c0d4-4c6e-b764-cc9d7506facb"). InnerVolumeSpecName "dnsmasq-svc". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:55:45 crc kubenswrapper[5052]: I1203 12:55:45.863153 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d6e33718-c0d4-4c6e-b764-cc9d7506facb-config" (OuterVolumeSpecName: "config") pod "d6e33718-c0d4-4c6e-b764-cc9d7506facb" (UID: "d6e33718-c0d4-4c6e-b764-cc9d7506facb"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:55:45 crc kubenswrapper[5052]: I1203 12:55:45.868038 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d6e33718-c0d4-4c6e-b764-cc9d7506facb-openstack-edpm-ipam" (OuterVolumeSpecName: "openstack-edpm-ipam") pod "d6e33718-c0d4-4c6e-b764-cc9d7506facb" (UID: "d6e33718-c0d4-4c6e-b764-cc9d7506facb"). InnerVolumeSpecName "openstack-edpm-ipam". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:55:45 crc kubenswrapper[5052]: I1203 12:55:45.926926 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-w8w8d\" (UniqueName: \"kubernetes.io/projected/d6e33718-c0d4-4c6e-b764-cc9d7506facb-kube-api-access-w8w8d\") on node \"crc\" DevicePath \"\"" Dec 03 12:55:45 crc kubenswrapper[5052]: I1203 12:55:45.926963 5052 reconciler_common.go:293] "Volume detached for volume \"openstack-edpm-ipam\" (UniqueName: \"kubernetes.io/configmap/d6e33718-c0d4-4c6e-b764-cc9d7506facb-openstack-edpm-ipam\") on node \"crc\" DevicePath \"\"" Dec 03 12:55:45 crc kubenswrapper[5052]: I1203 12:55:45.926979 5052 reconciler_common.go:293] "Volume detached for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/d6e33718-c0d4-4c6e-b764-cc9d7506facb-dnsmasq-svc\") on node \"crc\" DevicePath \"\"" Dec 03 12:55:45 crc kubenswrapper[5052]: I1203 12:55:45.926992 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d6e33718-c0d4-4c6e-b764-cc9d7506facb-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:55:46 crc kubenswrapper[5052]: I1203 12:55:46.335451 5052 generic.go:334] "Generic (PLEG): container finished" podID="d6e33718-c0d4-4c6e-b764-cc9d7506facb" containerID="bb662c2d445a4909b0d4cff01b0d0c2bee8bdec10afab01b6e9c64a98e974702" exitCode=0 Dec 03 12:55:46 crc kubenswrapper[5052]: I1203 12:55:46.335505 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-79667f9c49-dtkj9" event={"ID":"d6e33718-c0d4-4c6e-b764-cc9d7506facb","Type":"ContainerDied","Data":"bb662c2d445a4909b0d4cff01b0d0c2bee8bdec10afab01b6e9c64a98e974702"} Dec 03 12:55:46 crc kubenswrapper[5052]: I1203 12:55:46.335545 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-79667f9c49-dtkj9" Dec 03 12:55:46 crc kubenswrapper[5052]: I1203 12:55:46.335552 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-79667f9c49-dtkj9" event={"ID":"d6e33718-c0d4-4c6e-b764-cc9d7506facb","Type":"ContainerDied","Data":"0791032e4451946f2f84fcc244f6b5ff1529d693be7934196451ef24a9e07686"} Dec 03 12:55:46 crc kubenswrapper[5052]: I1203 12:55:46.335587 5052 scope.go:117] "RemoveContainer" containerID="bb662c2d445a4909b0d4cff01b0d0c2bee8bdec10afab01b6e9c64a98e974702" Dec 03 12:55:46 crc kubenswrapper[5052]: I1203 12:55:46.357846 5052 scope.go:117] "RemoveContainer" containerID="9fbce0d32ede246cb7f606b05fc16925371a3bbb91e8feefca3d305d73a5d20e" Dec 03 12:55:46 crc kubenswrapper[5052]: I1203 12:55:46.366948 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-79667f9c49-dtkj9"] Dec 03 12:55:46 crc kubenswrapper[5052]: I1203 12:55:46.373143 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-79667f9c49-dtkj9"] Dec 03 12:55:46 crc kubenswrapper[5052]: I1203 12:55:46.385133 5052 scope.go:117] "RemoveContainer" containerID="bb662c2d445a4909b0d4cff01b0d0c2bee8bdec10afab01b6e9c64a98e974702" Dec 03 12:55:46 crc kubenswrapper[5052]: E1203 12:55:46.385792 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bb662c2d445a4909b0d4cff01b0d0c2bee8bdec10afab01b6e9c64a98e974702\": container with ID starting with bb662c2d445a4909b0d4cff01b0d0c2bee8bdec10afab01b6e9c64a98e974702 not found: ID does not exist" containerID="bb662c2d445a4909b0d4cff01b0d0c2bee8bdec10afab01b6e9c64a98e974702" Dec 03 12:55:46 crc kubenswrapper[5052]: I1203 12:55:46.385846 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bb662c2d445a4909b0d4cff01b0d0c2bee8bdec10afab01b6e9c64a98e974702"} err="failed to get container status \"bb662c2d445a4909b0d4cff01b0d0c2bee8bdec10afab01b6e9c64a98e974702\": rpc error: code = NotFound desc = could not find container \"bb662c2d445a4909b0d4cff01b0d0c2bee8bdec10afab01b6e9c64a98e974702\": container with ID starting with bb662c2d445a4909b0d4cff01b0d0c2bee8bdec10afab01b6e9c64a98e974702 not found: ID does not exist" Dec 03 12:55:46 crc kubenswrapper[5052]: I1203 12:55:46.385866 5052 scope.go:117] "RemoveContainer" containerID="9fbce0d32ede246cb7f606b05fc16925371a3bbb91e8feefca3d305d73a5d20e" Dec 03 12:55:46 crc kubenswrapper[5052]: E1203 12:55:46.386126 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9fbce0d32ede246cb7f606b05fc16925371a3bbb91e8feefca3d305d73a5d20e\": container with ID starting with 9fbce0d32ede246cb7f606b05fc16925371a3bbb91e8feefca3d305d73a5d20e not found: ID does not exist" containerID="9fbce0d32ede246cb7f606b05fc16925371a3bbb91e8feefca3d305d73a5d20e" Dec 03 12:55:46 crc kubenswrapper[5052]: I1203 12:55:46.386162 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9fbce0d32ede246cb7f606b05fc16925371a3bbb91e8feefca3d305d73a5d20e"} err="failed to get container status \"9fbce0d32ede246cb7f606b05fc16925371a3bbb91e8feefca3d305d73a5d20e\": rpc error: code = NotFound desc = could not find container \"9fbce0d32ede246cb7f606b05fc16925371a3bbb91e8feefca3d305d73a5d20e\": container with ID starting with 9fbce0d32ede246cb7f606b05fc16925371a3bbb91e8feefca3d305d73a5d20e not found: ID does not exist" Dec 03 12:55:47 crc kubenswrapper[5052]: I1203 12:55:47.687054 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d6e33718-c0d4-4c6e-b764-cc9d7506facb" path="/var/lib/kubelet/pods/d6e33718-c0d4-4c6e-b764-cc9d7506facb/volumes" Dec 03 12:55:50 crc kubenswrapper[5052]: I1203 12:55:50.673202 5052 scope.go:117] "RemoveContainer" containerID="69fa7fbcd2aceb7c53703a19508876f93a97dc9adabb79c16fb691d90f606d58" Dec 03 12:55:50 crc kubenswrapper[5052]: E1203 12:55:50.673983 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 12:55:52 crc kubenswrapper[5052]: I1203 12:55:52.484082 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["crc-storage/crc-storage-crc-z9h5z"] Dec 03 12:55:52 crc kubenswrapper[5052]: I1203 12:55:52.491747 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["crc-storage/crc-storage-crc-z9h5z"] Dec 03 12:55:52 crc kubenswrapper[5052]: I1203 12:55:52.640081 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["crc-storage/crc-storage-crc-dpbp5"] Dec 03 12:55:52 crc kubenswrapper[5052]: E1203 12:55:52.640550 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="5f0a6e44-08a6-454e-8580-1c33335219b3" containerName="extract-content" Dec 03 12:55:52 crc kubenswrapper[5052]: I1203 12:55:52.640571 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f0a6e44-08a6-454e-8580-1c33335219b3" containerName="extract-content" Dec 03 12:55:52 crc kubenswrapper[5052]: E1203 12:55:52.640602 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="5f0a6e44-08a6-454e-8580-1c33335219b3" containerName="extract-utilities" Dec 03 12:55:52 crc kubenswrapper[5052]: I1203 12:55:52.640612 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f0a6e44-08a6-454e-8580-1c33335219b3" containerName="extract-utilities" Dec 03 12:55:52 crc kubenswrapper[5052]: E1203 12:55:52.640636 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="d6e33718-c0d4-4c6e-b764-cc9d7506facb" containerName="init" Dec 03 12:55:52 crc kubenswrapper[5052]: I1203 12:55:52.640644 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="d6e33718-c0d4-4c6e-b764-cc9d7506facb" containerName="init" Dec 03 12:55:52 crc kubenswrapper[5052]: E1203 12:55:52.640665 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="5f0a6e44-08a6-454e-8580-1c33335219b3" containerName="registry-server" Dec 03 12:55:52 crc kubenswrapper[5052]: I1203 12:55:52.640673 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f0a6e44-08a6-454e-8580-1c33335219b3" containerName="registry-server" Dec 03 12:55:52 crc kubenswrapper[5052]: E1203 12:55:52.640686 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="d6e33718-c0d4-4c6e-b764-cc9d7506facb" containerName="dnsmasq-dns" Dec 03 12:55:52 crc kubenswrapper[5052]: I1203 12:55:52.640695 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="d6e33718-c0d4-4c6e-b764-cc9d7506facb" containerName="dnsmasq-dns" Dec 03 12:55:52 crc kubenswrapper[5052]: I1203 12:55:52.640886 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="d6e33718-c0d4-4c6e-b764-cc9d7506facb" containerName="dnsmasq-dns" Dec 03 12:55:52 crc kubenswrapper[5052]: I1203 12:55:52.640920 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="5f0a6e44-08a6-454e-8580-1c33335219b3" containerName="registry-server" Dec 03 12:55:52 crc kubenswrapper[5052]: I1203 12:55:52.641618 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="crc-storage/crc-storage-crc-dpbp5" Dec 03 12:55:52 crc kubenswrapper[5052]: I1203 12:55:52.644486 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"crc-storage"/"crc-storage" Dec 03 12:55:52 crc kubenswrapper[5052]: I1203 12:55:52.644575 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"crc-storage"/"kube-root-ca.crt" Dec 03 12:55:52 crc kubenswrapper[5052]: I1203 12:55:52.644705 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"crc-storage"/"openshift-service-ca.crt" Dec 03 12:55:52 crc kubenswrapper[5052]: I1203 12:55:52.644753 5052 reflector.go:368] Caches populated for *v1.Secret from object-"crc-storage"/"crc-storage-dockercfg-mf76g" Dec 03 12:55:52 crc kubenswrapper[5052]: I1203 12:55:52.655221 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["crc-storage/crc-storage-crc-dpbp5"] Dec 03 12:55:52 crc kubenswrapper[5052]: I1203 12:55:52.737150 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n6tl5\" (UniqueName: \"kubernetes.io/projected/72503b2b-faf7-4642-97d5-ba12d91ab587-kube-api-access-n6tl5\") pod \"crc-storage-crc-dpbp5\" (UID: \"72503b2b-faf7-4642-97d5-ba12d91ab587\") " pod="crc-storage/crc-storage-crc-dpbp5" Dec 03 12:55:52 crc kubenswrapper[5052]: I1203 12:55:52.737410 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/72503b2b-faf7-4642-97d5-ba12d91ab587-crc-storage\") pod \"crc-storage-crc-dpbp5\" (UID: \"72503b2b-faf7-4642-97d5-ba12d91ab587\") " pod="crc-storage/crc-storage-crc-dpbp5" Dec 03 12:55:52 crc kubenswrapper[5052]: I1203 12:55:52.737900 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/72503b2b-faf7-4642-97d5-ba12d91ab587-node-mnt\") pod \"crc-storage-crc-dpbp5\" (UID: \"72503b2b-faf7-4642-97d5-ba12d91ab587\") " pod="crc-storage/crc-storage-crc-dpbp5" Dec 03 12:55:52 crc kubenswrapper[5052]: I1203 12:55:52.839525 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-n6tl5\" (UniqueName: \"kubernetes.io/projected/72503b2b-faf7-4642-97d5-ba12d91ab587-kube-api-access-n6tl5\") pod \"crc-storage-crc-dpbp5\" (UID: \"72503b2b-faf7-4642-97d5-ba12d91ab587\") " pod="crc-storage/crc-storage-crc-dpbp5" Dec 03 12:55:52 crc kubenswrapper[5052]: I1203 12:55:52.839622 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/72503b2b-faf7-4642-97d5-ba12d91ab587-crc-storage\") pod \"crc-storage-crc-dpbp5\" (UID: \"72503b2b-faf7-4642-97d5-ba12d91ab587\") " pod="crc-storage/crc-storage-crc-dpbp5" Dec 03 12:55:52 crc kubenswrapper[5052]: I1203 12:55:52.839913 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/72503b2b-faf7-4642-97d5-ba12d91ab587-node-mnt\") pod \"crc-storage-crc-dpbp5\" (UID: \"72503b2b-faf7-4642-97d5-ba12d91ab587\") " pod="crc-storage/crc-storage-crc-dpbp5" Dec 03 12:55:52 crc kubenswrapper[5052]: I1203 12:55:52.840012 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/72503b2b-faf7-4642-97d5-ba12d91ab587-node-mnt\") pod \"crc-storage-crc-dpbp5\" (UID: \"72503b2b-faf7-4642-97d5-ba12d91ab587\") " pod="crc-storage/crc-storage-crc-dpbp5" Dec 03 12:55:52 crc kubenswrapper[5052]: I1203 12:55:52.841217 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/72503b2b-faf7-4642-97d5-ba12d91ab587-crc-storage\") pod \"crc-storage-crc-dpbp5\" (UID: \"72503b2b-faf7-4642-97d5-ba12d91ab587\") " pod="crc-storage/crc-storage-crc-dpbp5" Dec 03 12:55:52 crc kubenswrapper[5052]: I1203 12:55:52.859045 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-n6tl5\" (UniqueName: \"kubernetes.io/projected/72503b2b-faf7-4642-97d5-ba12d91ab587-kube-api-access-n6tl5\") pod \"crc-storage-crc-dpbp5\" (UID: \"72503b2b-faf7-4642-97d5-ba12d91ab587\") " pod="crc-storage/crc-storage-crc-dpbp5" Dec 03 12:55:52 crc kubenswrapper[5052]: I1203 12:55:52.968178 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="crc-storage/crc-storage-crc-dpbp5" Dec 03 12:55:53 crc kubenswrapper[5052]: I1203 12:55:53.391674 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["crc-storage/crc-storage-crc-dpbp5"] Dec 03 12:55:53 crc kubenswrapper[5052]: I1203 12:55:53.425530 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="crc-storage/crc-storage-crc-dpbp5" event={"ID":"72503b2b-faf7-4642-97d5-ba12d91ab587","Type":"ContainerStarted","Data":"28a823faa8d3733c90e1a87f000ded90c8ee9a0fc7d8f217e40056009a8b6826"} Dec 03 12:55:53 crc kubenswrapper[5052]: I1203 12:55:53.681463 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="337d1568-3061-4799-bc8b-b8acd9280356" path="/var/lib/kubelet/pods/337d1568-3061-4799-bc8b-b8acd9280356/volumes" Dec 03 12:55:54 crc kubenswrapper[5052]: I1203 12:55:54.436278 5052 generic.go:334] "Generic (PLEG): container finished" podID="72503b2b-faf7-4642-97d5-ba12d91ab587" containerID="593c544f232e09a05f83aba11107c032efde7fcb81fb9b1a34b1c8ba7ff35e4b" exitCode=0 Dec 03 12:55:54 crc kubenswrapper[5052]: I1203 12:55:54.436323 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="crc-storage/crc-storage-crc-dpbp5" event={"ID":"72503b2b-faf7-4642-97d5-ba12d91ab587","Type":"ContainerDied","Data":"593c544f232e09a05f83aba11107c032efde7fcb81fb9b1a34b1c8ba7ff35e4b"} Dec 03 12:55:55 crc kubenswrapper[5052]: I1203 12:55:55.705477 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="crc-storage/crc-storage-crc-dpbp5" Dec 03 12:55:55 crc kubenswrapper[5052]: I1203 12:55:55.886747 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n6tl5\" (UniqueName: \"kubernetes.io/projected/72503b2b-faf7-4642-97d5-ba12d91ab587-kube-api-access-n6tl5\") pod \"72503b2b-faf7-4642-97d5-ba12d91ab587\" (UID: \"72503b2b-faf7-4642-97d5-ba12d91ab587\") " Dec 03 12:55:55 crc kubenswrapper[5052]: I1203 12:55:55.886820 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/72503b2b-faf7-4642-97d5-ba12d91ab587-crc-storage\") pod \"72503b2b-faf7-4642-97d5-ba12d91ab587\" (UID: \"72503b2b-faf7-4642-97d5-ba12d91ab587\") " Dec 03 12:55:55 crc kubenswrapper[5052]: I1203 12:55:55.886932 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/72503b2b-faf7-4642-97d5-ba12d91ab587-node-mnt\") pod \"72503b2b-faf7-4642-97d5-ba12d91ab587\" (UID: \"72503b2b-faf7-4642-97d5-ba12d91ab587\") " Dec 03 12:55:55 crc kubenswrapper[5052]: I1203 12:55:55.887125 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/72503b2b-faf7-4642-97d5-ba12d91ab587-node-mnt" (OuterVolumeSpecName: "node-mnt") pod "72503b2b-faf7-4642-97d5-ba12d91ab587" (UID: "72503b2b-faf7-4642-97d5-ba12d91ab587"). InnerVolumeSpecName "node-mnt". PluginName "kubernetes.io/host-path", VolumeGidValue "" Dec 03 12:55:55 crc kubenswrapper[5052]: I1203 12:55:55.887270 5052 reconciler_common.go:293] "Volume detached for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/72503b2b-faf7-4642-97d5-ba12d91ab587-node-mnt\") on node \"crc\" DevicePath \"\"" Dec 03 12:55:55 crc kubenswrapper[5052]: I1203 12:55:55.891167 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/72503b2b-faf7-4642-97d5-ba12d91ab587-kube-api-access-n6tl5" (OuterVolumeSpecName: "kube-api-access-n6tl5") pod "72503b2b-faf7-4642-97d5-ba12d91ab587" (UID: "72503b2b-faf7-4642-97d5-ba12d91ab587"). InnerVolumeSpecName "kube-api-access-n6tl5". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:55:55 crc kubenswrapper[5052]: I1203 12:55:55.904636 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/72503b2b-faf7-4642-97d5-ba12d91ab587-crc-storage" (OuterVolumeSpecName: "crc-storage") pod "72503b2b-faf7-4642-97d5-ba12d91ab587" (UID: "72503b2b-faf7-4642-97d5-ba12d91ab587"). InnerVolumeSpecName "crc-storage". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:55:55 crc kubenswrapper[5052]: I1203 12:55:55.988507 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-n6tl5\" (UniqueName: \"kubernetes.io/projected/72503b2b-faf7-4642-97d5-ba12d91ab587-kube-api-access-n6tl5\") on node \"crc\" DevicePath \"\"" Dec 03 12:55:55 crc kubenswrapper[5052]: I1203 12:55:55.988572 5052 reconciler_common.go:293] "Volume detached for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/72503b2b-faf7-4642-97d5-ba12d91ab587-crc-storage\") on node \"crc\" DevicePath \"\"" Dec 03 12:55:56 crc kubenswrapper[5052]: I1203 12:55:56.461153 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="crc-storage/crc-storage-crc-dpbp5" event={"ID":"72503b2b-faf7-4642-97d5-ba12d91ab587","Type":"ContainerDied","Data":"28a823faa8d3733c90e1a87f000ded90c8ee9a0fc7d8f217e40056009a8b6826"} Dec 03 12:55:56 crc kubenswrapper[5052]: I1203 12:55:56.461181 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="crc-storage/crc-storage-crc-dpbp5" Dec 03 12:55:56 crc kubenswrapper[5052]: I1203 12:55:56.461194 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="28a823faa8d3733c90e1a87f000ded90c8ee9a0fc7d8f217e40056009a8b6826" Dec 03 12:55:59 crc kubenswrapper[5052]: I1203 12:55:59.865744 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["crc-storage/crc-storage-crc-dpbp5"] Dec 03 12:55:59 crc kubenswrapper[5052]: I1203 12:55:59.870658 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["crc-storage/crc-storage-crc-dpbp5"] Dec 03 12:56:00 crc kubenswrapper[5052]: I1203 12:56:00.020287 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["crc-storage/crc-storage-crc-mhzf5"] Dec 03 12:56:00 crc kubenswrapper[5052]: E1203 12:56:00.020728 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="72503b2b-faf7-4642-97d5-ba12d91ab587" containerName="storage" Dec 03 12:56:00 crc kubenswrapper[5052]: I1203 12:56:00.020748 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="72503b2b-faf7-4642-97d5-ba12d91ab587" containerName="storage" Dec 03 12:56:00 crc kubenswrapper[5052]: I1203 12:56:00.020933 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="72503b2b-faf7-4642-97d5-ba12d91ab587" containerName="storage" Dec 03 12:56:00 crc kubenswrapper[5052]: I1203 12:56:00.021611 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="crc-storage/crc-storage-crc-mhzf5" Dec 03 12:56:00 crc kubenswrapper[5052]: I1203 12:56:00.024006 5052 reflector.go:368] Caches populated for *v1.Secret from object-"crc-storage"/"crc-storage-dockercfg-mf76g" Dec 03 12:56:00 crc kubenswrapper[5052]: I1203 12:56:00.024263 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"crc-storage"/"openshift-service-ca.crt" Dec 03 12:56:00 crc kubenswrapper[5052]: I1203 12:56:00.026687 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"crc-storage"/"kube-root-ca.crt" Dec 03 12:56:00 crc kubenswrapper[5052]: I1203 12:56:00.026765 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"crc-storage"/"crc-storage" Dec 03 12:56:00 crc kubenswrapper[5052]: I1203 12:56:00.029691 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["crc-storage/crc-storage-crc-mhzf5"] Dec 03 12:56:00 crc kubenswrapper[5052]: I1203 12:56:00.149779 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/87727c3f-e310-47a7-a7dc-eb96211b6208-node-mnt\") pod \"crc-storage-crc-mhzf5\" (UID: \"87727c3f-e310-47a7-a7dc-eb96211b6208\") " pod="crc-storage/crc-storage-crc-mhzf5" Dec 03 12:56:00 crc kubenswrapper[5052]: I1203 12:56:00.149893 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/87727c3f-e310-47a7-a7dc-eb96211b6208-crc-storage\") pod \"crc-storage-crc-mhzf5\" (UID: \"87727c3f-e310-47a7-a7dc-eb96211b6208\") " pod="crc-storage/crc-storage-crc-mhzf5" Dec 03 12:56:00 crc kubenswrapper[5052]: I1203 12:56:00.150152 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rb9j8\" (UniqueName: \"kubernetes.io/projected/87727c3f-e310-47a7-a7dc-eb96211b6208-kube-api-access-rb9j8\") pod \"crc-storage-crc-mhzf5\" (UID: \"87727c3f-e310-47a7-a7dc-eb96211b6208\") " pod="crc-storage/crc-storage-crc-mhzf5" Dec 03 12:56:00 crc kubenswrapper[5052]: I1203 12:56:00.251270 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-rb9j8\" (UniqueName: \"kubernetes.io/projected/87727c3f-e310-47a7-a7dc-eb96211b6208-kube-api-access-rb9j8\") pod \"crc-storage-crc-mhzf5\" (UID: \"87727c3f-e310-47a7-a7dc-eb96211b6208\") " pod="crc-storage/crc-storage-crc-mhzf5" Dec 03 12:56:00 crc kubenswrapper[5052]: I1203 12:56:00.251432 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/87727c3f-e310-47a7-a7dc-eb96211b6208-node-mnt\") pod \"crc-storage-crc-mhzf5\" (UID: \"87727c3f-e310-47a7-a7dc-eb96211b6208\") " pod="crc-storage/crc-storage-crc-mhzf5" Dec 03 12:56:00 crc kubenswrapper[5052]: I1203 12:56:00.251469 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/87727c3f-e310-47a7-a7dc-eb96211b6208-crc-storage\") pod \"crc-storage-crc-mhzf5\" (UID: \"87727c3f-e310-47a7-a7dc-eb96211b6208\") " pod="crc-storage/crc-storage-crc-mhzf5" Dec 03 12:56:00 crc kubenswrapper[5052]: I1203 12:56:00.251769 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/87727c3f-e310-47a7-a7dc-eb96211b6208-node-mnt\") pod \"crc-storage-crc-mhzf5\" (UID: \"87727c3f-e310-47a7-a7dc-eb96211b6208\") " pod="crc-storage/crc-storage-crc-mhzf5" Dec 03 12:56:00 crc kubenswrapper[5052]: I1203 12:56:00.252258 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/87727c3f-e310-47a7-a7dc-eb96211b6208-crc-storage\") pod \"crc-storage-crc-mhzf5\" (UID: \"87727c3f-e310-47a7-a7dc-eb96211b6208\") " pod="crc-storage/crc-storage-crc-mhzf5" Dec 03 12:56:00 crc kubenswrapper[5052]: I1203 12:56:00.270577 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-rb9j8\" (UniqueName: \"kubernetes.io/projected/87727c3f-e310-47a7-a7dc-eb96211b6208-kube-api-access-rb9j8\") pod \"crc-storage-crc-mhzf5\" (UID: \"87727c3f-e310-47a7-a7dc-eb96211b6208\") " pod="crc-storage/crc-storage-crc-mhzf5" Dec 03 12:56:00 crc kubenswrapper[5052]: I1203 12:56:00.348753 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="crc-storage/crc-storage-crc-mhzf5" Dec 03 12:56:00 crc kubenswrapper[5052]: I1203 12:56:00.747659 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["crc-storage/crc-storage-crc-mhzf5"] Dec 03 12:56:01 crc kubenswrapper[5052]: I1203 12:56:01.505805 5052 generic.go:334] "Generic (PLEG): container finished" podID="87727c3f-e310-47a7-a7dc-eb96211b6208" containerID="3b9a73789825a69ab6325c9009912283dd8eb27d7f4cafb3e836235ef7fd206c" exitCode=0 Dec 03 12:56:01 crc kubenswrapper[5052]: I1203 12:56:01.505872 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="crc-storage/crc-storage-crc-mhzf5" event={"ID":"87727c3f-e310-47a7-a7dc-eb96211b6208","Type":"ContainerDied","Data":"3b9a73789825a69ab6325c9009912283dd8eb27d7f4cafb3e836235ef7fd206c"} Dec 03 12:56:01 crc kubenswrapper[5052]: I1203 12:56:01.506192 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="crc-storage/crc-storage-crc-mhzf5" event={"ID":"87727c3f-e310-47a7-a7dc-eb96211b6208","Type":"ContainerStarted","Data":"a7ce6c9bc516917554bb32e50601f687bd27c65021f9051eb26c333e0f0b92cf"} Dec 03 12:56:01 crc kubenswrapper[5052]: I1203 12:56:01.683993 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="72503b2b-faf7-4642-97d5-ba12d91ab587" path="/var/lib/kubelet/pods/72503b2b-faf7-4642-97d5-ba12d91ab587/volumes" Dec 03 12:56:02 crc kubenswrapper[5052]: I1203 12:56:02.812487 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="crc-storage/crc-storage-crc-mhzf5" Dec 03 12:56:03 crc kubenswrapper[5052]: I1203 12:56:03.005601 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/87727c3f-e310-47a7-a7dc-eb96211b6208-node-mnt\") pod \"87727c3f-e310-47a7-a7dc-eb96211b6208\" (UID: \"87727c3f-e310-47a7-a7dc-eb96211b6208\") " Dec 03 12:56:03 crc kubenswrapper[5052]: I1203 12:56:03.005722 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rb9j8\" (UniqueName: \"kubernetes.io/projected/87727c3f-e310-47a7-a7dc-eb96211b6208-kube-api-access-rb9j8\") pod \"87727c3f-e310-47a7-a7dc-eb96211b6208\" (UID: \"87727c3f-e310-47a7-a7dc-eb96211b6208\") " Dec 03 12:56:03 crc kubenswrapper[5052]: I1203 12:56:03.005753 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/87727c3f-e310-47a7-a7dc-eb96211b6208-node-mnt" (OuterVolumeSpecName: "node-mnt") pod "87727c3f-e310-47a7-a7dc-eb96211b6208" (UID: "87727c3f-e310-47a7-a7dc-eb96211b6208"). InnerVolumeSpecName "node-mnt". PluginName "kubernetes.io/host-path", VolumeGidValue "" Dec 03 12:56:03 crc kubenswrapper[5052]: I1203 12:56:03.005782 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/87727c3f-e310-47a7-a7dc-eb96211b6208-crc-storage\") pod \"87727c3f-e310-47a7-a7dc-eb96211b6208\" (UID: \"87727c3f-e310-47a7-a7dc-eb96211b6208\") " Dec 03 12:56:03 crc kubenswrapper[5052]: I1203 12:56:03.006193 5052 reconciler_common.go:293] "Volume detached for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/87727c3f-e310-47a7-a7dc-eb96211b6208-node-mnt\") on node \"crc\" DevicePath \"\"" Dec 03 12:56:03 crc kubenswrapper[5052]: I1203 12:56:03.011077 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/87727c3f-e310-47a7-a7dc-eb96211b6208-kube-api-access-rb9j8" (OuterVolumeSpecName: "kube-api-access-rb9j8") pod "87727c3f-e310-47a7-a7dc-eb96211b6208" (UID: "87727c3f-e310-47a7-a7dc-eb96211b6208"). InnerVolumeSpecName "kube-api-access-rb9j8". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:56:03 crc kubenswrapper[5052]: I1203 12:56:03.025551 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/87727c3f-e310-47a7-a7dc-eb96211b6208-crc-storage" (OuterVolumeSpecName: "crc-storage") pod "87727c3f-e310-47a7-a7dc-eb96211b6208" (UID: "87727c3f-e310-47a7-a7dc-eb96211b6208"). InnerVolumeSpecName "crc-storage". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:56:03 crc kubenswrapper[5052]: I1203 12:56:03.108266 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-rb9j8\" (UniqueName: \"kubernetes.io/projected/87727c3f-e310-47a7-a7dc-eb96211b6208-kube-api-access-rb9j8\") on node \"crc\" DevicePath \"\"" Dec 03 12:56:03 crc kubenswrapper[5052]: I1203 12:56:03.108320 5052 reconciler_common.go:293] "Volume detached for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/87727c3f-e310-47a7-a7dc-eb96211b6208-crc-storage\") on node \"crc\" DevicePath \"\"" Dec 03 12:56:03 crc kubenswrapper[5052]: I1203 12:56:03.525265 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="crc-storage/crc-storage-crc-mhzf5" event={"ID":"87727c3f-e310-47a7-a7dc-eb96211b6208","Type":"ContainerDied","Data":"a7ce6c9bc516917554bb32e50601f687bd27c65021f9051eb26c333e0f0b92cf"} Dec 03 12:56:03 crc kubenswrapper[5052]: I1203 12:56:03.525298 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a7ce6c9bc516917554bb32e50601f687bd27c65021f9051eb26c333e0f0b92cf" Dec 03 12:56:03 crc kubenswrapper[5052]: I1203 12:56:03.525305 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="crc-storage/crc-storage-crc-mhzf5" Dec 03 12:56:04 crc kubenswrapper[5052]: I1203 12:56:04.673635 5052 scope.go:117] "RemoveContainer" containerID="69fa7fbcd2aceb7c53703a19508876f93a97dc9adabb79c16fb691d90f606d58" Dec 03 12:56:04 crc kubenswrapper[5052]: E1203 12:56:04.673843 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 12:56:06 crc kubenswrapper[5052]: I1203 12:56:06.351698 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-7d78464677-jj2j9"] Dec 03 12:56:06 crc kubenswrapper[5052]: E1203 12:56:06.352794 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="87727c3f-e310-47a7-a7dc-eb96211b6208" containerName="storage" Dec 03 12:56:06 crc kubenswrapper[5052]: I1203 12:56:06.352815 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="87727c3f-e310-47a7-a7dc-eb96211b6208" containerName="storage" Dec 03 12:56:06 crc kubenswrapper[5052]: I1203 12:56:06.353099 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="87727c3f-e310-47a7-a7dc-eb96211b6208" containerName="storage" Dec 03 12:56:06 crc kubenswrapper[5052]: I1203 12:56:06.354532 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-7d78464677-jj2j9" Dec 03 12:56:06 crc kubenswrapper[5052]: I1203 12:56:06.359387 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"edpm-compute-global" Dec 03 12:56:06 crc kubenswrapper[5052]: I1203 12:56:06.363609 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-7d78464677-jj2j9"] Dec 03 12:56:06 crc kubenswrapper[5052]: I1203 12:56:06.455263 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/916bd1d4-f942-4b7b-8efd-94e63cc84208-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-7d78464677-jj2j9\" (UID: \"916bd1d4-f942-4b7b-8efd-94e63cc84208\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-7d78464677-jj2j9" Dec 03 12:56:06 crc kubenswrapper[5052]: I1203 12:56:06.455333 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"edpm-compute-global\" (UniqueName: \"kubernetes.io/configmap/916bd1d4-f942-4b7b-8efd-94e63cc84208-edpm-compute-global\") pod \"dnsmasq-dnsmasq-7d78464677-jj2j9\" (UID: \"916bd1d4-f942-4b7b-8efd-94e63cc84208\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-7d78464677-jj2j9" Dec 03 12:56:06 crc kubenswrapper[5052]: I1203 12:56:06.455352 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/916bd1d4-f942-4b7b-8efd-94e63cc84208-config\") pod \"dnsmasq-dnsmasq-7d78464677-jj2j9\" (UID: \"916bd1d4-f942-4b7b-8efd-94e63cc84208\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-7d78464677-jj2j9" Dec 03 12:56:06 crc kubenswrapper[5052]: I1203 12:56:06.455418 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wxrwc\" (UniqueName: \"kubernetes.io/projected/916bd1d4-f942-4b7b-8efd-94e63cc84208-kube-api-access-wxrwc\") pod \"dnsmasq-dnsmasq-7d78464677-jj2j9\" (UID: \"916bd1d4-f942-4b7b-8efd-94e63cc84208\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-7d78464677-jj2j9" Dec 03 12:56:06 crc kubenswrapper[5052]: I1203 12:56:06.556719 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"edpm-compute-global\" (UniqueName: \"kubernetes.io/configmap/916bd1d4-f942-4b7b-8efd-94e63cc84208-edpm-compute-global\") pod \"dnsmasq-dnsmasq-7d78464677-jj2j9\" (UID: \"916bd1d4-f942-4b7b-8efd-94e63cc84208\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-7d78464677-jj2j9" Dec 03 12:56:06 crc kubenswrapper[5052]: I1203 12:56:06.556775 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/916bd1d4-f942-4b7b-8efd-94e63cc84208-config\") pod \"dnsmasq-dnsmasq-7d78464677-jj2j9\" (UID: \"916bd1d4-f942-4b7b-8efd-94e63cc84208\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-7d78464677-jj2j9" Dec 03 12:56:06 crc kubenswrapper[5052]: I1203 12:56:06.556831 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-wxrwc\" (UniqueName: \"kubernetes.io/projected/916bd1d4-f942-4b7b-8efd-94e63cc84208-kube-api-access-wxrwc\") pod \"dnsmasq-dnsmasq-7d78464677-jj2j9\" (UID: \"916bd1d4-f942-4b7b-8efd-94e63cc84208\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-7d78464677-jj2j9" Dec 03 12:56:06 crc kubenswrapper[5052]: I1203 12:56:06.556894 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/916bd1d4-f942-4b7b-8efd-94e63cc84208-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-7d78464677-jj2j9\" (UID: \"916bd1d4-f942-4b7b-8efd-94e63cc84208\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-7d78464677-jj2j9" Dec 03 12:56:06 crc kubenswrapper[5052]: I1203 12:56:06.557694 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"edpm-compute-global\" (UniqueName: \"kubernetes.io/configmap/916bd1d4-f942-4b7b-8efd-94e63cc84208-edpm-compute-global\") pod \"dnsmasq-dnsmasq-7d78464677-jj2j9\" (UID: \"916bd1d4-f942-4b7b-8efd-94e63cc84208\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-7d78464677-jj2j9" Dec 03 12:56:06 crc kubenswrapper[5052]: I1203 12:56:06.557821 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/916bd1d4-f942-4b7b-8efd-94e63cc84208-config\") pod \"dnsmasq-dnsmasq-7d78464677-jj2j9\" (UID: \"916bd1d4-f942-4b7b-8efd-94e63cc84208\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-7d78464677-jj2j9" Dec 03 12:56:06 crc kubenswrapper[5052]: I1203 12:56:06.557847 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/916bd1d4-f942-4b7b-8efd-94e63cc84208-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-7d78464677-jj2j9\" (UID: \"916bd1d4-f942-4b7b-8efd-94e63cc84208\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-7d78464677-jj2j9" Dec 03 12:56:06 crc kubenswrapper[5052]: I1203 12:56:06.581969 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-wxrwc\" (UniqueName: \"kubernetes.io/projected/916bd1d4-f942-4b7b-8efd-94e63cc84208-kube-api-access-wxrwc\") pod \"dnsmasq-dnsmasq-7d78464677-jj2j9\" (UID: \"916bd1d4-f942-4b7b-8efd-94e63cc84208\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-7d78464677-jj2j9" Dec 03 12:56:06 crc kubenswrapper[5052]: I1203 12:56:06.672825 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-7d78464677-jj2j9" Dec 03 12:56:07 crc kubenswrapper[5052]: I1203 12:56:07.112018 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-7d78464677-jj2j9"] Dec 03 12:56:07 crc kubenswrapper[5052]: I1203 12:56:07.569802 5052 generic.go:334] "Generic (PLEG): container finished" podID="916bd1d4-f942-4b7b-8efd-94e63cc84208" containerID="3e41d127a12f55e3cf5052b7a21b9b23bae57de90a79214a7348f35c20f30a79" exitCode=0 Dec 03 12:56:07 crc kubenswrapper[5052]: I1203 12:56:07.569872 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-7d78464677-jj2j9" event={"ID":"916bd1d4-f942-4b7b-8efd-94e63cc84208","Type":"ContainerDied","Data":"3e41d127a12f55e3cf5052b7a21b9b23bae57de90a79214a7348f35c20f30a79"} Dec 03 12:56:07 crc kubenswrapper[5052]: I1203 12:56:07.569954 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-7d78464677-jj2j9" event={"ID":"916bd1d4-f942-4b7b-8efd-94e63cc84208","Type":"ContainerStarted","Data":"e86e7f53b2d887287605094fc27bb6874b8e78b07816242a428ab81c68878a53"} Dec 03 12:56:08 crc kubenswrapper[5052]: I1203 12:56:08.582631 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-7d78464677-jj2j9" event={"ID":"916bd1d4-f942-4b7b-8efd-94e63cc84208","Type":"ContainerStarted","Data":"309819d445a2c8d65f21aab056f5e431c8c087cdafd3b00934fc169d30dda8fa"} Dec 03 12:56:08 crc kubenswrapper[5052]: I1203 12:56:08.582796 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-7d78464677-jj2j9" Dec 03 12:56:08 crc kubenswrapper[5052]: I1203 12:56:08.606305 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-7d78464677-jj2j9" podStartSLOduration=2.606286055 podStartE2EDuration="2.606286055s" podCreationTimestamp="2025-12-03 12:56:06 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:56:08.600140079 +0000 UTC m=+5759.188381843" watchObservedRunningTime="2025-12-03 12:56:08.606286055 +0000 UTC m=+5759.194527829" Dec 03 12:56:15 crc kubenswrapper[5052]: I1203 12:56:15.672637 5052 scope.go:117] "RemoveContainer" containerID="69fa7fbcd2aceb7c53703a19508876f93a97dc9adabb79c16fb691d90f606d58" Dec 03 12:56:15 crc kubenswrapper[5052]: E1203 12:56:15.673712 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 12:56:16 crc kubenswrapper[5052]: I1203 12:56:16.675263 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-7d78464677-jj2j9" Dec 03 12:56:16 crc kubenswrapper[5052]: I1203 12:56:16.737246 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-shdj6"] Dec 03 12:56:16 crc kubenswrapper[5052]: I1203 12:56:16.737564 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-shdj6" podUID="7bed2e44-93da-465d-9e10-6879ff1c82f5" containerName="dnsmasq-dns" containerID="cri-o://7df281d8985229e1ba628e120c93ab3f824fd9bd67ac5fb03caed49d8f4608a9" gracePeriod=10 Dec 03 12:56:17 crc kubenswrapper[5052]: I1203 12:56:17.171431 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-shdj6" Dec 03 12:56:17 crc kubenswrapper[5052]: I1203 12:56:17.310115 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/7bed2e44-93da-465d-9e10-6879ff1c82f5-dnsmasq-svc\") pod \"7bed2e44-93da-465d-9e10-6879ff1c82f5\" (UID: \"7bed2e44-93da-465d-9e10-6879ff1c82f5\") " Dec 03 12:56:17 crc kubenswrapper[5052]: I1203 12:56:17.310266 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2cd75\" (UniqueName: \"kubernetes.io/projected/7bed2e44-93da-465d-9e10-6879ff1c82f5-kube-api-access-2cd75\") pod \"7bed2e44-93da-465d-9e10-6879ff1c82f5\" (UID: \"7bed2e44-93da-465d-9e10-6879ff1c82f5\") " Dec 03 12:56:17 crc kubenswrapper[5052]: I1203 12:56:17.310318 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/7bed2e44-93da-465d-9e10-6879ff1c82f5-config\") pod \"7bed2e44-93da-465d-9e10-6879ff1c82f5\" (UID: \"7bed2e44-93da-465d-9e10-6879ff1c82f5\") " Dec 03 12:56:17 crc kubenswrapper[5052]: I1203 12:56:17.315053 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7bed2e44-93da-465d-9e10-6879ff1c82f5-kube-api-access-2cd75" (OuterVolumeSpecName: "kube-api-access-2cd75") pod "7bed2e44-93da-465d-9e10-6879ff1c82f5" (UID: "7bed2e44-93da-465d-9e10-6879ff1c82f5"). InnerVolumeSpecName "kube-api-access-2cd75". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:56:17 crc kubenswrapper[5052]: I1203 12:56:17.345157 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7bed2e44-93da-465d-9e10-6879ff1c82f5-dnsmasq-svc" (OuterVolumeSpecName: "dnsmasq-svc") pod "7bed2e44-93da-465d-9e10-6879ff1c82f5" (UID: "7bed2e44-93da-465d-9e10-6879ff1c82f5"). InnerVolumeSpecName "dnsmasq-svc". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:56:17 crc kubenswrapper[5052]: I1203 12:56:17.345722 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7bed2e44-93da-465d-9e10-6879ff1c82f5-config" (OuterVolumeSpecName: "config") pod "7bed2e44-93da-465d-9e10-6879ff1c82f5" (UID: "7bed2e44-93da-465d-9e10-6879ff1c82f5"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:56:17 crc kubenswrapper[5052]: I1203 12:56:17.413437 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/7bed2e44-93da-465d-9e10-6879ff1c82f5-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:56:17 crc kubenswrapper[5052]: I1203 12:56:17.413496 5052 reconciler_common.go:293] "Volume detached for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/7bed2e44-93da-465d-9e10-6879ff1c82f5-dnsmasq-svc\") on node \"crc\" DevicePath \"\"" Dec 03 12:56:17 crc kubenswrapper[5052]: I1203 12:56:17.414602 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-2cd75\" (UniqueName: \"kubernetes.io/projected/7bed2e44-93da-465d-9e10-6879ff1c82f5-kube-api-access-2cd75\") on node \"crc\" DevicePath \"\"" Dec 03 12:56:17 crc kubenswrapper[5052]: I1203 12:56:17.430692 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/download-cache-edpm-compute-global-edpm-compute-global-jvs5j"] Dec 03 12:56:17 crc kubenswrapper[5052]: E1203 12:56:17.431164 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="7bed2e44-93da-465d-9e10-6879ff1c82f5" containerName="dnsmasq-dns" Dec 03 12:56:17 crc kubenswrapper[5052]: I1203 12:56:17.431245 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="7bed2e44-93da-465d-9e10-6879ff1c82f5" containerName="dnsmasq-dns" Dec 03 12:56:17 crc kubenswrapper[5052]: E1203 12:56:17.431319 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="7bed2e44-93da-465d-9e10-6879ff1c82f5" containerName="init" Dec 03 12:56:17 crc kubenswrapper[5052]: I1203 12:56:17.431378 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="7bed2e44-93da-465d-9e10-6879ff1c82f5" containerName="init" Dec 03 12:56:17 crc kubenswrapper[5052]: I1203 12:56:17.431598 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="7bed2e44-93da-465d-9e10-6879ff1c82f5" containerName="dnsmasq-dns" Dec 03 12:56:17 crc kubenswrapper[5052]: I1203 12:56:17.432135 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/download-cache-edpm-compute-global-edpm-compute-global-jvs5j" Dec 03 12:56:17 crc kubenswrapper[5052]: I1203 12:56:17.438484 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"dataplanenodeset-edpm-compute-global" Dec 03 12:56:17 crc kubenswrapper[5052]: I1203 12:56:17.438585 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"openstack-aee-default-env" Dec 03 12:56:17 crc kubenswrapper[5052]: I1203 12:56:17.438800 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"dataplane-ansible-ssh-private-key-secret" Dec 03 12:56:17 crc kubenswrapper[5052]: I1203 12:56:17.438897 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"edpm-compute-global-dockercfg-p5rjx" Dec 03 12:56:17 crc kubenswrapper[5052]: I1203 12:56:17.456694 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/download-cache-edpm-compute-global-edpm-compute-global-jvs5j"] Dec 03 12:56:17 crc kubenswrapper[5052]: I1203 12:56:17.515510 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ntk6p\" (UniqueName: \"kubernetes.io/projected/83a97e99-dd8c-49c7-a503-c09c1768be86-kube-api-access-ntk6p\") pod \"download-cache-edpm-compute-global-edpm-compute-global-jvs5j\" (UID: \"83a97e99-dd8c-49c7-a503-c09c1768be86\") " pod="openstack-kuttl-tests/download-cache-edpm-compute-global-edpm-compute-global-jvs5j" Dec 03 12:56:17 crc kubenswrapper[5052]: I1203 12:56:17.515593 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/83a97e99-dd8c-49c7-a503-c09c1768be86-inventory\") pod \"download-cache-edpm-compute-global-edpm-compute-global-jvs5j\" (UID: \"83a97e99-dd8c-49c7-a503-c09c1768be86\") " pod="openstack-kuttl-tests/download-cache-edpm-compute-global-edpm-compute-global-jvs5j" Dec 03 12:56:17 crc kubenswrapper[5052]: I1203 12:56:17.515639 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/83a97e99-dd8c-49c7-a503-c09c1768be86-ssh-key\") pod \"download-cache-edpm-compute-global-edpm-compute-global-jvs5j\" (UID: \"83a97e99-dd8c-49c7-a503-c09c1768be86\") " pod="openstack-kuttl-tests/download-cache-edpm-compute-global-edpm-compute-global-jvs5j" Dec 03 12:56:17 crc kubenswrapper[5052]: I1203 12:56:17.616726 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/83a97e99-dd8c-49c7-a503-c09c1768be86-ssh-key\") pod \"download-cache-edpm-compute-global-edpm-compute-global-jvs5j\" (UID: \"83a97e99-dd8c-49c7-a503-c09c1768be86\") " pod="openstack-kuttl-tests/download-cache-edpm-compute-global-edpm-compute-global-jvs5j" Dec 03 12:56:17 crc kubenswrapper[5052]: I1203 12:56:17.616833 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-ntk6p\" (UniqueName: \"kubernetes.io/projected/83a97e99-dd8c-49c7-a503-c09c1768be86-kube-api-access-ntk6p\") pod \"download-cache-edpm-compute-global-edpm-compute-global-jvs5j\" (UID: \"83a97e99-dd8c-49c7-a503-c09c1768be86\") " pod="openstack-kuttl-tests/download-cache-edpm-compute-global-edpm-compute-global-jvs5j" Dec 03 12:56:17 crc kubenswrapper[5052]: I1203 12:56:17.616900 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/83a97e99-dd8c-49c7-a503-c09c1768be86-inventory\") pod \"download-cache-edpm-compute-global-edpm-compute-global-jvs5j\" (UID: \"83a97e99-dd8c-49c7-a503-c09c1768be86\") " pod="openstack-kuttl-tests/download-cache-edpm-compute-global-edpm-compute-global-jvs5j" Dec 03 12:56:17 crc kubenswrapper[5052]: I1203 12:56:17.620617 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/83a97e99-dd8c-49c7-a503-c09c1768be86-ssh-key\") pod \"download-cache-edpm-compute-global-edpm-compute-global-jvs5j\" (UID: \"83a97e99-dd8c-49c7-a503-c09c1768be86\") " pod="openstack-kuttl-tests/download-cache-edpm-compute-global-edpm-compute-global-jvs5j" Dec 03 12:56:17 crc kubenswrapper[5052]: I1203 12:56:17.621055 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/83a97e99-dd8c-49c7-a503-c09c1768be86-inventory\") pod \"download-cache-edpm-compute-global-edpm-compute-global-jvs5j\" (UID: \"83a97e99-dd8c-49c7-a503-c09c1768be86\") " pod="openstack-kuttl-tests/download-cache-edpm-compute-global-edpm-compute-global-jvs5j" Dec 03 12:56:17 crc kubenswrapper[5052]: I1203 12:56:17.634388 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-ntk6p\" (UniqueName: \"kubernetes.io/projected/83a97e99-dd8c-49c7-a503-c09c1768be86-kube-api-access-ntk6p\") pod \"download-cache-edpm-compute-global-edpm-compute-global-jvs5j\" (UID: \"83a97e99-dd8c-49c7-a503-c09c1768be86\") " pod="openstack-kuttl-tests/download-cache-edpm-compute-global-edpm-compute-global-jvs5j" Dec 03 12:56:17 crc kubenswrapper[5052]: I1203 12:56:17.670664 5052 generic.go:334] "Generic (PLEG): container finished" podID="7bed2e44-93da-465d-9e10-6879ff1c82f5" containerID="7df281d8985229e1ba628e120c93ab3f824fd9bd67ac5fb03caed49d8f4608a9" exitCode=0 Dec 03 12:56:17 crc kubenswrapper[5052]: I1203 12:56:17.670715 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-shdj6" Dec 03 12:56:17 crc kubenswrapper[5052]: I1203 12:56:17.670716 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-shdj6" event={"ID":"7bed2e44-93da-465d-9e10-6879ff1c82f5","Type":"ContainerDied","Data":"7df281d8985229e1ba628e120c93ab3f824fd9bd67ac5fb03caed49d8f4608a9"} Dec 03 12:56:17 crc kubenswrapper[5052]: I1203 12:56:17.670873 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-shdj6" event={"ID":"7bed2e44-93da-465d-9e10-6879ff1c82f5","Type":"ContainerDied","Data":"a586b878b7096b7313dc8e2f549516d2b891f1eb73b67e3bdb6b74dd77488878"} Dec 03 12:56:17 crc kubenswrapper[5052]: I1203 12:56:17.670903 5052 scope.go:117] "RemoveContainer" containerID="7df281d8985229e1ba628e120c93ab3f824fd9bd67ac5fb03caed49d8f4608a9" Dec 03 12:56:17 crc kubenswrapper[5052]: I1203 12:56:17.692627 5052 scope.go:117] "RemoveContainer" containerID="a27fb96a63938a81b5f484c71200a5edcde938947a173488bc4333ca4777eefc" Dec 03 12:56:17 crc kubenswrapper[5052]: I1203 12:56:17.704732 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-shdj6"] Dec 03 12:56:17 crc kubenswrapper[5052]: I1203 12:56:17.710999 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-shdj6"] Dec 03 12:56:17 crc kubenswrapper[5052]: I1203 12:56:17.712577 5052 scope.go:117] "RemoveContainer" containerID="7df281d8985229e1ba628e120c93ab3f824fd9bd67ac5fb03caed49d8f4608a9" Dec 03 12:56:17 crc kubenswrapper[5052]: E1203 12:56:17.713029 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7df281d8985229e1ba628e120c93ab3f824fd9bd67ac5fb03caed49d8f4608a9\": container with ID starting with 7df281d8985229e1ba628e120c93ab3f824fd9bd67ac5fb03caed49d8f4608a9 not found: ID does not exist" containerID="7df281d8985229e1ba628e120c93ab3f824fd9bd67ac5fb03caed49d8f4608a9" Dec 03 12:56:17 crc kubenswrapper[5052]: I1203 12:56:17.713082 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7df281d8985229e1ba628e120c93ab3f824fd9bd67ac5fb03caed49d8f4608a9"} err="failed to get container status \"7df281d8985229e1ba628e120c93ab3f824fd9bd67ac5fb03caed49d8f4608a9\": rpc error: code = NotFound desc = could not find container \"7df281d8985229e1ba628e120c93ab3f824fd9bd67ac5fb03caed49d8f4608a9\": container with ID starting with 7df281d8985229e1ba628e120c93ab3f824fd9bd67ac5fb03caed49d8f4608a9 not found: ID does not exist" Dec 03 12:56:17 crc kubenswrapper[5052]: I1203 12:56:17.713113 5052 scope.go:117] "RemoveContainer" containerID="a27fb96a63938a81b5f484c71200a5edcde938947a173488bc4333ca4777eefc" Dec 03 12:56:17 crc kubenswrapper[5052]: E1203 12:56:17.713457 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a27fb96a63938a81b5f484c71200a5edcde938947a173488bc4333ca4777eefc\": container with ID starting with a27fb96a63938a81b5f484c71200a5edcde938947a173488bc4333ca4777eefc not found: ID does not exist" containerID="a27fb96a63938a81b5f484c71200a5edcde938947a173488bc4333ca4777eefc" Dec 03 12:56:17 crc kubenswrapper[5052]: I1203 12:56:17.713487 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a27fb96a63938a81b5f484c71200a5edcde938947a173488bc4333ca4777eefc"} err="failed to get container status \"a27fb96a63938a81b5f484c71200a5edcde938947a173488bc4333ca4777eefc\": rpc error: code = NotFound desc = could not find container \"a27fb96a63938a81b5f484c71200a5edcde938947a173488bc4333ca4777eefc\": container with ID starting with a27fb96a63938a81b5f484c71200a5edcde938947a173488bc4333ca4777eefc not found: ID does not exist" Dec 03 12:56:17 crc kubenswrapper[5052]: I1203 12:56:17.749449 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/download-cache-edpm-compute-global-edpm-compute-global-jvs5j" Dec 03 12:56:18 crc kubenswrapper[5052]: I1203 12:56:18.198201 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/download-cache-edpm-compute-global-edpm-compute-global-jvs5j"] Dec 03 12:56:18 crc kubenswrapper[5052]: I1203 12:56:18.683265 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/download-cache-edpm-compute-global-edpm-compute-global-jvs5j" event={"ID":"83a97e99-dd8c-49c7-a503-c09c1768be86","Type":"ContainerStarted","Data":"e3b5cc9bdbfa715a4a0f9b88355e28b9e246f2da78d874693022df994414b7c8"} Dec 03 12:56:19 crc kubenswrapper[5052]: I1203 12:56:19.685365 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7bed2e44-93da-465d-9e10-6879ff1c82f5" path="/var/lib/kubelet/pods/7bed2e44-93da-465d-9e10-6879ff1c82f5/volumes" Dec 03 12:56:27 crc kubenswrapper[5052]: I1203 12:56:27.673162 5052 scope.go:117] "RemoveContainer" containerID="69fa7fbcd2aceb7c53703a19508876f93a97dc9adabb79c16fb691d90f606d58" Dec 03 12:56:27 crc kubenswrapper[5052]: E1203 12:56:27.674433 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 12:56:27 crc kubenswrapper[5052]: I1203 12:56:27.771771 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/download-cache-edpm-compute-global-edpm-compute-global-jvs5j" event={"ID":"83a97e99-dd8c-49c7-a503-c09c1768be86","Type":"ContainerStarted","Data":"31cbbc5dbfdab5d11d7e88f471e14d96eb1f12655abcc00af0fa9704571c07a9"} Dec 03 12:56:27 crc kubenswrapper[5052]: I1203 12:56:27.806322 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/download-cache-edpm-compute-global-edpm-compute-global-jvs5j" podStartSLOduration=2.28087695 podStartE2EDuration="10.806298071s" podCreationTimestamp="2025-12-03 12:56:17 +0000 UTC" firstStartedPulling="2025-12-03 12:56:18.20565013 +0000 UTC m=+5768.793891894" lastFinishedPulling="2025-12-03 12:56:26.731071251 +0000 UTC m=+5777.319313015" observedRunningTime="2025-12-03 12:56:27.800790123 +0000 UTC m=+5778.389031897" watchObservedRunningTime="2025-12-03 12:56:27.806298071 +0000 UTC m=+5778.394539835" Dec 03 12:56:28 crc kubenswrapper[5052]: I1203 12:56:28.783159 5052 generic.go:334] "Generic (PLEG): container finished" podID="83a97e99-dd8c-49c7-a503-c09c1768be86" containerID="31cbbc5dbfdab5d11d7e88f471e14d96eb1f12655abcc00af0fa9704571c07a9" exitCode=0 Dec 03 12:56:28 crc kubenswrapper[5052]: I1203 12:56:28.783208 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/download-cache-edpm-compute-global-edpm-compute-global-jvs5j" event={"ID":"83a97e99-dd8c-49c7-a503-c09c1768be86","Type":"ContainerDied","Data":"31cbbc5dbfdab5d11d7e88f471e14d96eb1f12655abcc00af0fa9704571c07a9"} Dec 03 12:56:30 crc kubenswrapper[5052]: I1203 12:56:30.097545 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/download-cache-edpm-compute-global-edpm-compute-global-jvs5j" Dec 03 12:56:30 crc kubenswrapper[5052]: I1203 12:56:30.209138 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/83a97e99-dd8c-49c7-a503-c09c1768be86-ssh-key\") pod \"83a97e99-dd8c-49c7-a503-c09c1768be86\" (UID: \"83a97e99-dd8c-49c7-a503-c09c1768be86\") " Dec 03 12:56:30 crc kubenswrapper[5052]: I1203 12:56:30.209232 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/83a97e99-dd8c-49c7-a503-c09c1768be86-inventory\") pod \"83a97e99-dd8c-49c7-a503-c09c1768be86\" (UID: \"83a97e99-dd8c-49c7-a503-c09c1768be86\") " Dec 03 12:56:30 crc kubenswrapper[5052]: I1203 12:56:30.209269 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ntk6p\" (UniqueName: \"kubernetes.io/projected/83a97e99-dd8c-49c7-a503-c09c1768be86-kube-api-access-ntk6p\") pod \"83a97e99-dd8c-49c7-a503-c09c1768be86\" (UID: \"83a97e99-dd8c-49c7-a503-c09c1768be86\") " Dec 03 12:56:30 crc kubenswrapper[5052]: I1203 12:56:30.213705 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/83a97e99-dd8c-49c7-a503-c09c1768be86-kube-api-access-ntk6p" (OuterVolumeSpecName: "kube-api-access-ntk6p") pod "83a97e99-dd8c-49c7-a503-c09c1768be86" (UID: "83a97e99-dd8c-49c7-a503-c09c1768be86"). InnerVolumeSpecName "kube-api-access-ntk6p". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:56:30 crc kubenswrapper[5052]: I1203 12:56:30.231251 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/83a97e99-dd8c-49c7-a503-c09c1768be86-inventory" (OuterVolumeSpecName: "inventory") pod "83a97e99-dd8c-49c7-a503-c09c1768be86" (UID: "83a97e99-dd8c-49c7-a503-c09c1768be86"). InnerVolumeSpecName "inventory". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:56:30 crc kubenswrapper[5052]: I1203 12:56:30.231646 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/83a97e99-dd8c-49c7-a503-c09c1768be86-ssh-key" (OuterVolumeSpecName: "ssh-key") pod "83a97e99-dd8c-49c7-a503-c09c1768be86" (UID: "83a97e99-dd8c-49c7-a503-c09c1768be86"). InnerVolumeSpecName "ssh-key". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:56:30 crc kubenswrapper[5052]: I1203 12:56:30.311173 5052 reconciler_common.go:293] "Volume detached for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/83a97e99-dd8c-49c7-a503-c09c1768be86-ssh-key\") on node \"crc\" DevicePath \"\"" Dec 03 12:56:30 crc kubenswrapper[5052]: I1203 12:56:30.311226 5052 reconciler_common.go:293] "Volume detached for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/83a97e99-dd8c-49c7-a503-c09c1768be86-inventory\") on node \"crc\" DevicePath \"\"" Dec 03 12:56:30 crc kubenswrapper[5052]: I1203 12:56:30.311252 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-ntk6p\" (UniqueName: \"kubernetes.io/projected/83a97e99-dd8c-49c7-a503-c09c1768be86-kube-api-access-ntk6p\") on node \"crc\" DevicePath \"\"" Dec 03 12:56:30 crc kubenswrapper[5052]: I1203 12:56:30.803428 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/download-cache-edpm-compute-global-edpm-compute-global-jvs5j" event={"ID":"83a97e99-dd8c-49c7-a503-c09c1768be86","Type":"ContainerDied","Data":"e3b5cc9bdbfa715a4a0f9b88355e28b9e246f2da78d874693022df994414b7c8"} Dec 03 12:56:30 crc kubenswrapper[5052]: I1203 12:56:30.803488 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e3b5cc9bdbfa715a4a0f9b88355e28b9e246f2da78d874693022df994414b7c8" Dec 03 12:56:30 crc kubenswrapper[5052]: I1203 12:56:30.803553 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/download-cache-edpm-compute-global-edpm-compute-global-jvs5j" Dec 03 12:56:30 crc kubenswrapper[5052]: E1203 12:56:30.962374 5052 cadvisor_stats_provider.go:516] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod83a97e99_dd8c_49c7_a503_c09c1768be86.slice/crio-e3b5cc9bdbfa715a4a0f9b88355e28b9e246f2da78d874693022df994414b7c8\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod83a97e99_dd8c_49c7_a503_c09c1768be86.slice\": RecentStats: unable to find data in memory cache]" Dec 03 12:56:31 crc kubenswrapper[5052]: I1203 12:56:31.172392 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/bootstrap-edpm-compute-global-edpm-compute-global-rb7xv"] Dec 03 12:56:31 crc kubenswrapper[5052]: E1203 12:56:31.173021 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="83a97e99-dd8c-49c7-a503-c09c1768be86" containerName="download-cache-edpm-compute-global-edpm-compute-global" Dec 03 12:56:31 crc kubenswrapper[5052]: I1203 12:56:31.173034 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="83a97e99-dd8c-49c7-a503-c09c1768be86" containerName="download-cache-edpm-compute-global-edpm-compute-global" Dec 03 12:56:31 crc kubenswrapper[5052]: I1203 12:56:31.173167 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="83a97e99-dd8c-49c7-a503-c09c1768be86" containerName="download-cache-edpm-compute-global-edpm-compute-global" Dec 03 12:56:31 crc kubenswrapper[5052]: I1203 12:56:31.173629 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/bootstrap-edpm-compute-global-edpm-compute-global-rb7xv" Dec 03 12:56:31 crc kubenswrapper[5052]: I1203 12:56:31.175683 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"dataplane-ansible-ssh-private-key-secret" Dec 03 12:56:31 crc kubenswrapper[5052]: I1203 12:56:31.176372 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"dataplanenodeset-edpm-compute-global" Dec 03 12:56:31 crc kubenswrapper[5052]: I1203 12:56:31.177218 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"openstack-aee-default-env" Dec 03 12:56:31 crc kubenswrapper[5052]: I1203 12:56:31.177247 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"edpm-compute-global-dockercfg-p5rjx" Dec 03 12:56:31 crc kubenswrapper[5052]: I1203 12:56:31.184133 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"combined-ca-bundle" Dec 03 12:56:31 crc kubenswrapper[5052]: I1203 12:56:31.185958 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/bootstrap-edpm-compute-global-edpm-compute-global-rb7xv"] Dec 03 12:56:31 crc kubenswrapper[5052]: I1203 12:56:31.326741 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bootstrap-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/626db185-326b-47f7-b41f-b973e9379b94-bootstrap-combined-ca-bundle\") pod \"bootstrap-edpm-compute-global-edpm-compute-global-rb7xv\" (UID: \"626db185-326b-47f7-b41f-b973e9379b94\") " pod="openstack-kuttl-tests/bootstrap-edpm-compute-global-edpm-compute-global-rb7xv" Dec 03 12:56:31 crc kubenswrapper[5052]: I1203 12:56:31.326814 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/626db185-326b-47f7-b41f-b973e9379b94-ssh-key\") pod \"bootstrap-edpm-compute-global-edpm-compute-global-rb7xv\" (UID: \"626db185-326b-47f7-b41f-b973e9379b94\") " pod="openstack-kuttl-tests/bootstrap-edpm-compute-global-edpm-compute-global-rb7xv" Dec 03 12:56:31 crc kubenswrapper[5052]: I1203 12:56:31.326928 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ppzdf\" (UniqueName: \"kubernetes.io/projected/626db185-326b-47f7-b41f-b973e9379b94-kube-api-access-ppzdf\") pod \"bootstrap-edpm-compute-global-edpm-compute-global-rb7xv\" (UID: \"626db185-326b-47f7-b41f-b973e9379b94\") " pod="openstack-kuttl-tests/bootstrap-edpm-compute-global-edpm-compute-global-rb7xv" Dec 03 12:56:31 crc kubenswrapper[5052]: I1203 12:56:31.326961 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/626db185-326b-47f7-b41f-b973e9379b94-inventory\") pod \"bootstrap-edpm-compute-global-edpm-compute-global-rb7xv\" (UID: \"626db185-326b-47f7-b41f-b973e9379b94\") " pod="openstack-kuttl-tests/bootstrap-edpm-compute-global-edpm-compute-global-rb7xv" Dec 03 12:56:31 crc kubenswrapper[5052]: I1203 12:56:31.428400 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-ppzdf\" (UniqueName: \"kubernetes.io/projected/626db185-326b-47f7-b41f-b973e9379b94-kube-api-access-ppzdf\") pod \"bootstrap-edpm-compute-global-edpm-compute-global-rb7xv\" (UID: \"626db185-326b-47f7-b41f-b973e9379b94\") " pod="openstack-kuttl-tests/bootstrap-edpm-compute-global-edpm-compute-global-rb7xv" Dec 03 12:56:31 crc kubenswrapper[5052]: I1203 12:56:31.428468 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/626db185-326b-47f7-b41f-b973e9379b94-inventory\") pod \"bootstrap-edpm-compute-global-edpm-compute-global-rb7xv\" (UID: \"626db185-326b-47f7-b41f-b973e9379b94\") " pod="openstack-kuttl-tests/bootstrap-edpm-compute-global-edpm-compute-global-rb7xv" Dec 03 12:56:31 crc kubenswrapper[5052]: I1203 12:56:31.428637 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"bootstrap-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/626db185-326b-47f7-b41f-b973e9379b94-bootstrap-combined-ca-bundle\") pod \"bootstrap-edpm-compute-global-edpm-compute-global-rb7xv\" (UID: \"626db185-326b-47f7-b41f-b973e9379b94\") " pod="openstack-kuttl-tests/bootstrap-edpm-compute-global-edpm-compute-global-rb7xv" Dec 03 12:56:31 crc kubenswrapper[5052]: I1203 12:56:31.428679 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/626db185-326b-47f7-b41f-b973e9379b94-ssh-key\") pod \"bootstrap-edpm-compute-global-edpm-compute-global-rb7xv\" (UID: \"626db185-326b-47f7-b41f-b973e9379b94\") " pod="openstack-kuttl-tests/bootstrap-edpm-compute-global-edpm-compute-global-rb7xv" Dec 03 12:56:31 crc kubenswrapper[5052]: I1203 12:56:31.433056 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/626db185-326b-47f7-b41f-b973e9379b94-ssh-key\") pod \"bootstrap-edpm-compute-global-edpm-compute-global-rb7xv\" (UID: \"626db185-326b-47f7-b41f-b973e9379b94\") " pod="openstack-kuttl-tests/bootstrap-edpm-compute-global-edpm-compute-global-rb7xv" Dec 03 12:56:31 crc kubenswrapper[5052]: I1203 12:56:31.433453 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/626db185-326b-47f7-b41f-b973e9379b94-inventory\") pod \"bootstrap-edpm-compute-global-edpm-compute-global-rb7xv\" (UID: \"626db185-326b-47f7-b41f-b973e9379b94\") " pod="openstack-kuttl-tests/bootstrap-edpm-compute-global-edpm-compute-global-rb7xv" Dec 03 12:56:31 crc kubenswrapper[5052]: I1203 12:56:31.434037 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"bootstrap-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/626db185-326b-47f7-b41f-b973e9379b94-bootstrap-combined-ca-bundle\") pod \"bootstrap-edpm-compute-global-edpm-compute-global-rb7xv\" (UID: \"626db185-326b-47f7-b41f-b973e9379b94\") " pod="openstack-kuttl-tests/bootstrap-edpm-compute-global-edpm-compute-global-rb7xv" Dec 03 12:56:31 crc kubenswrapper[5052]: I1203 12:56:31.445941 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-ppzdf\" (UniqueName: \"kubernetes.io/projected/626db185-326b-47f7-b41f-b973e9379b94-kube-api-access-ppzdf\") pod \"bootstrap-edpm-compute-global-edpm-compute-global-rb7xv\" (UID: \"626db185-326b-47f7-b41f-b973e9379b94\") " pod="openstack-kuttl-tests/bootstrap-edpm-compute-global-edpm-compute-global-rb7xv" Dec 03 12:56:31 crc kubenswrapper[5052]: I1203 12:56:31.490659 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/bootstrap-edpm-compute-global-edpm-compute-global-rb7xv" Dec 03 12:56:31 crc kubenswrapper[5052]: I1203 12:56:31.912080 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/bootstrap-edpm-compute-global-edpm-compute-global-rb7xv"] Dec 03 12:56:32 crc kubenswrapper[5052]: I1203 12:56:32.824874 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/bootstrap-edpm-compute-global-edpm-compute-global-rb7xv" event={"ID":"626db185-326b-47f7-b41f-b973e9379b94","Type":"ContainerStarted","Data":"483ab837917d2bb0cee2cc97f9b75239ff0e75f486cdf8e5b738db16fa469c02"} Dec 03 12:56:32 crc kubenswrapper[5052]: I1203 12:56:32.825706 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/bootstrap-edpm-compute-global-edpm-compute-global-rb7xv" event={"ID":"626db185-326b-47f7-b41f-b973e9379b94","Type":"ContainerStarted","Data":"ab073c89b61637a49f3e128f067860ac6e7208cf7f6024b17da24096e2839e71"} Dec 03 12:56:32 crc kubenswrapper[5052]: I1203 12:56:32.849653 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/bootstrap-edpm-compute-global-edpm-compute-global-rb7xv" podStartSLOduration=1.3294351039999999 podStartE2EDuration="1.849638101s" podCreationTimestamp="2025-12-03 12:56:31 +0000 UTC" firstStartedPulling="2025-12-03 12:56:31.920705495 +0000 UTC m=+5782.508947249" lastFinishedPulling="2025-12-03 12:56:32.440908482 +0000 UTC m=+5783.029150246" observedRunningTime="2025-12-03 12:56:32.84611682 +0000 UTC m=+5783.434358614" watchObservedRunningTime="2025-12-03 12:56:32.849638101 +0000 UTC m=+5783.437879865" Dec 03 12:56:34 crc kubenswrapper[5052]: I1203 12:56:34.844555 5052 generic.go:334] "Generic (PLEG): container finished" podID="626db185-326b-47f7-b41f-b973e9379b94" containerID="483ab837917d2bb0cee2cc97f9b75239ff0e75f486cdf8e5b738db16fa469c02" exitCode=0 Dec 03 12:56:34 crc kubenswrapper[5052]: I1203 12:56:34.844694 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/bootstrap-edpm-compute-global-edpm-compute-global-rb7xv" event={"ID":"626db185-326b-47f7-b41f-b973e9379b94","Type":"ContainerDied","Data":"483ab837917d2bb0cee2cc97f9b75239ff0e75f486cdf8e5b738db16fa469c02"} Dec 03 12:56:36 crc kubenswrapper[5052]: I1203 12:56:36.150708 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/bootstrap-edpm-compute-global-edpm-compute-global-rb7xv" Dec 03 12:56:36 crc kubenswrapper[5052]: I1203 12:56:36.302045 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/626db185-326b-47f7-b41f-b973e9379b94-ssh-key\") pod \"626db185-326b-47f7-b41f-b973e9379b94\" (UID: \"626db185-326b-47f7-b41f-b973e9379b94\") " Dec 03 12:56:36 crc kubenswrapper[5052]: I1203 12:56:36.302223 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/626db185-326b-47f7-b41f-b973e9379b94-inventory\") pod \"626db185-326b-47f7-b41f-b973e9379b94\" (UID: \"626db185-326b-47f7-b41f-b973e9379b94\") " Dec 03 12:56:36 crc kubenswrapper[5052]: I1203 12:56:36.302345 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"bootstrap-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/626db185-326b-47f7-b41f-b973e9379b94-bootstrap-combined-ca-bundle\") pod \"626db185-326b-47f7-b41f-b973e9379b94\" (UID: \"626db185-326b-47f7-b41f-b973e9379b94\") " Dec 03 12:56:36 crc kubenswrapper[5052]: I1203 12:56:36.302414 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ppzdf\" (UniqueName: \"kubernetes.io/projected/626db185-326b-47f7-b41f-b973e9379b94-kube-api-access-ppzdf\") pod \"626db185-326b-47f7-b41f-b973e9379b94\" (UID: \"626db185-326b-47f7-b41f-b973e9379b94\") " Dec 03 12:56:36 crc kubenswrapper[5052]: I1203 12:56:36.307215 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/626db185-326b-47f7-b41f-b973e9379b94-kube-api-access-ppzdf" (OuterVolumeSpecName: "kube-api-access-ppzdf") pod "626db185-326b-47f7-b41f-b973e9379b94" (UID: "626db185-326b-47f7-b41f-b973e9379b94"). InnerVolumeSpecName "kube-api-access-ppzdf". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:56:36 crc kubenswrapper[5052]: I1203 12:56:36.307439 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/626db185-326b-47f7-b41f-b973e9379b94-bootstrap-combined-ca-bundle" (OuterVolumeSpecName: "bootstrap-combined-ca-bundle") pod "626db185-326b-47f7-b41f-b973e9379b94" (UID: "626db185-326b-47f7-b41f-b973e9379b94"). InnerVolumeSpecName "bootstrap-combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:56:36 crc kubenswrapper[5052]: I1203 12:56:36.323643 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/626db185-326b-47f7-b41f-b973e9379b94-inventory" (OuterVolumeSpecName: "inventory") pod "626db185-326b-47f7-b41f-b973e9379b94" (UID: "626db185-326b-47f7-b41f-b973e9379b94"). InnerVolumeSpecName "inventory". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:56:36 crc kubenswrapper[5052]: I1203 12:56:36.327322 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/626db185-326b-47f7-b41f-b973e9379b94-ssh-key" (OuterVolumeSpecName: "ssh-key") pod "626db185-326b-47f7-b41f-b973e9379b94" (UID: "626db185-326b-47f7-b41f-b973e9379b94"). InnerVolumeSpecName "ssh-key". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:56:36 crc kubenswrapper[5052]: I1203 12:56:36.404060 5052 reconciler_common.go:293] "Volume detached for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/626db185-326b-47f7-b41f-b973e9379b94-inventory\") on node \"crc\" DevicePath \"\"" Dec 03 12:56:36 crc kubenswrapper[5052]: I1203 12:56:36.404098 5052 reconciler_common.go:293] "Volume detached for volume \"bootstrap-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/626db185-326b-47f7-b41f-b973e9379b94-bootstrap-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:56:36 crc kubenswrapper[5052]: I1203 12:56:36.404134 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-ppzdf\" (UniqueName: \"kubernetes.io/projected/626db185-326b-47f7-b41f-b973e9379b94-kube-api-access-ppzdf\") on node \"crc\" DevicePath \"\"" Dec 03 12:56:36 crc kubenswrapper[5052]: I1203 12:56:36.404143 5052 reconciler_common.go:293] "Volume detached for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/626db185-326b-47f7-b41f-b973e9379b94-ssh-key\") on node \"crc\" DevicePath \"\"" Dec 03 12:56:36 crc kubenswrapper[5052]: I1203 12:56:36.872075 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/bootstrap-edpm-compute-global-edpm-compute-global-rb7xv" event={"ID":"626db185-326b-47f7-b41f-b973e9379b94","Type":"ContainerDied","Data":"ab073c89b61637a49f3e128f067860ac6e7208cf7f6024b17da24096e2839e71"} Dec 03 12:56:36 crc kubenswrapper[5052]: I1203 12:56:36.872145 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ab073c89b61637a49f3e128f067860ac6e7208cf7f6024b17da24096e2839e71" Dec 03 12:56:36 crc kubenswrapper[5052]: I1203 12:56:36.872162 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/bootstrap-edpm-compute-global-edpm-compute-global-rb7xv" Dec 03 12:56:36 crc kubenswrapper[5052]: I1203 12:56:36.953469 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/configure-network-edpm-compute-global-edpm-compute-global-qf7vq"] Dec 03 12:56:36 crc kubenswrapper[5052]: E1203 12:56:36.953934 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="626db185-326b-47f7-b41f-b973e9379b94" containerName="bootstrap-edpm-compute-global-edpm-compute-global" Dec 03 12:56:36 crc kubenswrapper[5052]: I1203 12:56:36.954709 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="626db185-326b-47f7-b41f-b973e9379b94" containerName="bootstrap-edpm-compute-global-edpm-compute-global" Dec 03 12:56:36 crc kubenswrapper[5052]: I1203 12:56:36.954932 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="626db185-326b-47f7-b41f-b973e9379b94" containerName="bootstrap-edpm-compute-global-edpm-compute-global" Dec 03 12:56:36 crc kubenswrapper[5052]: I1203 12:56:36.955491 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/configure-network-edpm-compute-global-edpm-compute-global-qf7vq" Dec 03 12:56:36 crc kubenswrapper[5052]: I1203 12:56:36.958701 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"dataplanenodeset-edpm-compute-global" Dec 03 12:56:36 crc kubenswrapper[5052]: I1203 12:56:36.958972 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"edpm-compute-global-dockercfg-p5rjx" Dec 03 12:56:36 crc kubenswrapper[5052]: I1203 12:56:36.959128 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"openstack-aee-default-env" Dec 03 12:56:36 crc kubenswrapper[5052]: I1203 12:56:36.959536 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"dataplane-ansible-ssh-private-key-secret" Dec 03 12:56:36 crc kubenswrapper[5052]: I1203 12:56:36.963135 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/configure-network-edpm-compute-global-edpm-compute-global-qf7vq"] Dec 03 12:56:37 crc kubenswrapper[5052]: I1203 12:56:37.113240 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/c8557042-afbc-4b35-996c-e2c060db2f2c-ssh-key\") pod \"configure-network-edpm-compute-global-edpm-compute-global-qf7vq\" (UID: \"c8557042-afbc-4b35-996c-e2c060db2f2c\") " pod="openstack-kuttl-tests/configure-network-edpm-compute-global-edpm-compute-global-qf7vq" Dec 03 12:56:37 crc kubenswrapper[5052]: I1203 12:56:37.113387 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bd46x\" (UniqueName: \"kubernetes.io/projected/c8557042-afbc-4b35-996c-e2c060db2f2c-kube-api-access-bd46x\") pod \"configure-network-edpm-compute-global-edpm-compute-global-qf7vq\" (UID: \"c8557042-afbc-4b35-996c-e2c060db2f2c\") " pod="openstack-kuttl-tests/configure-network-edpm-compute-global-edpm-compute-global-qf7vq" Dec 03 12:56:37 crc kubenswrapper[5052]: I1203 12:56:37.113792 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/c8557042-afbc-4b35-996c-e2c060db2f2c-inventory\") pod \"configure-network-edpm-compute-global-edpm-compute-global-qf7vq\" (UID: \"c8557042-afbc-4b35-996c-e2c060db2f2c\") " pod="openstack-kuttl-tests/configure-network-edpm-compute-global-edpm-compute-global-qf7vq" Dec 03 12:56:37 crc kubenswrapper[5052]: I1203 12:56:37.215484 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/c8557042-afbc-4b35-996c-e2c060db2f2c-inventory\") pod \"configure-network-edpm-compute-global-edpm-compute-global-qf7vq\" (UID: \"c8557042-afbc-4b35-996c-e2c060db2f2c\") " pod="openstack-kuttl-tests/configure-network-edpm-compute-global-edpm-compute-global-qf7vq" Dec 03 12:56:37 crc kubenswrapper[5052]: I1203 12:56:37.215590 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/c8557042-afbc-4b35-996c-e2c060db2f2c-ssh-key\") pod \"configure-network-edpm-compute-global-edpm-compute-global-qf7vq\" (UID: \"c8557042-afbc-4b35-996c-e2c060db2f2c\") " pod="openstack-kuttl-tests/configure-network-edpm-compute-global-edpm-compute-global-qf7vq" Dec 03 12:56:37 crc kubenswrapper[5052]: I1203 12:56:37.215646 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-bd46x\" (UniqueName: \"kubernetes.io/projected/c8557042-afbc-4b35-996c-e2c060db2f2c-kube-api-access-bd46x\") pod \"configure-network-edpm-compute-global-edpm-compute-global-qf7vq\" (UID: \"c8557042-afbc-4b35-996c-e2c060db2f2c\") " pod="openstack-kuttl-tests/configure-network-edpm-compute-global-edpm-compute-global-qf7vq" Dec 03 12:56:37 crc kubenswrapper[5052]: I1203 12:56:37.219825 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/c8557042-afbc-4b35-996c-e2c060db2f2c-inventory\") pod \"configure-network-edpm-compute-global-edpm-compute-global-qf7vq\" (UID: \"c8557042-afbc-4b35-996c-e2c060db2f2c\") " pod="openstack-kuttl-tests/configure-network-edpm-compute-global-edpm-compute-global-qf7vq" Dec 03 12:56:37 crc kubenswrapper[5052]: I1203 12:56:37.229651 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/c8557042-afbc-4b35-996c-e2c060db2f2c-ssh-key\") pod \"configure-network-edpm-compute-global-edpm-compute-global-qf7vq\" (UID: \"c8557042-afbc-4b35-996c-e2c060db2f2c\") " pod="openstack-kuttl-tests/configure-network-edpm-compute-global-edpm-compute-global-qf7vq" Dec 03 12:56:37 crc kubenswrapper[5052]: I1203 12:56:37.233129 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-bd46x\" (UniqueName: \"kubernetes.io/projected/c8557042-afbc-4b35-996c-e2c060db2f2c-kube-api-access-bd46x\") pod \"configure-network-edpm-compute-global-edpm-compute-global-qf7vq\" (UID: \"c8557042-afbc-4b35-996c-e2c060db2f2c\") " pod="openstack-kuttl-tests/configure-network-edpm-compute-global-edpm-compute-global-qf7vq" Dec 03 12:56:37 crc kubenswrapper[5052]: I1203 12:56:37.277586 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/configure-network-edpm-compute-global-edpm-compute-global-qf7vq" Dec 03 12:56:37 crc kubenswrapper[5052]: I1203 12:56:37.280763 5052 scope.go:117] "RemoveContainer" containerID="ce01d82e85a4543523154c1e02105bb4626f45bd57dd40b49d3a01ee746c0546" Dec 03 12:56:37 crc kubenswrapper[5052]: I1203 12:56:37.308327 5052 scope.go:117] "RemoveContainer" containerID="5a735baf9cea071acdfeb65e88d849bce23a03817704ca1e54362fd9b759b7b2" Dec 03 12:56:37 crc kubenswrapper[5052]: I1203 12:56:37.431187 5052 scope.go:117] "RemoveContainer" containerID="909e761cd205a7fdfc227201b2626c6b3ce6dbab281adadf66ab0f0073e2725c" Dec 03 12:56:37 crc kubenswrapper[5052]: I1203 12:56:37.459353 5052 scope.go:117] "RemoveContainer" containerID="b2aadea167aa77dc784f9f8aa60f73ab130df902022eed62b5b88d51e19cde7b" Dec 03 12:56:37 crc kubenswrapper[5052]: I1203 12:56:37.479619 5052 scope.go:117] "RemoveContainer" containerID="5c9c7136b870fdaefa424e5ed10b8c43b181bc99a764b6eae51d7e74f33874d0" Dec 03 12:56:37 crc kubenswrapper[5052]: I1203 12:56:37.519426 5052 scope.go:117] "RemoveContainer" containerID="3337bf147a1198a7d374ebe43a83d3f902342362f5770d4c6b8d236f37a55306" Dec 03 12:56:37 crc kubenswrapper[5052]: I1203 12:56:37.538301 5052 scope.go:117] "RemoveContainer" containerID="63038026ed4d3a0cffe0b438d1ef983112e5fb0e27b4ce5efd2e7a5e9ad2fe3c" Dec 03 12:56:37 crc kubenswrapper[5052]: I1203 12:56:37.558848 5052 scope.go:117] "RemoveContainer" containerID="4c91d3477ec8fee9de89adfe339684f3d73d2d5166f4c936f3b80973b8b0fcbb" Dec 03 12:56:37 crc kubenswrapper[5052]: I1203 12:56:37.581669 5052 scope.go:117] "RemoveContainer" containerID="b8ecae5dcbbd55bfb5a128bba207c90d7a6b85badc20f265e843e97165d22a9c" Dec 03 12:56:37 crc kubenswrapper[5052]: I1203 12:56:37.602254 5052 scope.go:117] "RemoveContainer" containerID="a106c3bce606737209a4ddb9c54e0f19e9031d68589a8d8a5db30f62902c3da1" Dec 03 12:56:37 crc kubenswrapper[5052]: I1203 12:56:37.627605 5052 scope.go:117] "RemoveContainer" containerID="27f09642bc3ec7488d9032aa069fc921aca6a6f0c2792bc5d84326beb5487829" Dec 03 12:56:37 crc kubenswrapper[5052]: I1203 12:56:37.646750 5052 scope.go:117] "RemoveContainer" containerID="d3bec13b250dc5d3da3916fc0a01000daac906f8a9e04bab954c7cd2f5198046" Dec 03 12:56:37 crc kubenswrapper[5052]: I1203 12:56:37.664095 5052 scope.go:117] "RemoveContainer" containerID="474a137afe0eb8c21a373aa9a2430c6cb8a18815dfd7c1dc6300a9e03b4cbac5" Dec 03 12:56:37 crc kubenswrapper[5052]: I1203 12:56:37.743918 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/configure-network-edpm-compute-global-edpm-compute-global-qf7vq"] Dec 03 12:56:37 crc kubenswrapper[5052]: I1203 12:56:37.883447 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/configure-network-edpm-compute-global-edpm-compute-global-qf7vq" event={"ID":"c8557042-afbc-4b35-996c-e2c060db2f2c","Type":"ContainerStarted","Data":"0970cd99b90b06a9e18bb33920469f41ab6b99df7fd9bdd322626bde7d3faf43"} Dec 03 12:56:38 crc kubenswrapper[5052]: I1203 12:56:38.897394 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/configure-network-edpm-compute-global-edpm-compute-global-qf7vq" event={"ID":"c8557042-afbc-4b35-996c-e2c060db2f2c","Type":"ContainerStarted","Data":"317bade2bc973a40de5b34c60a1f485d2e6ce7d4603e6c7351dc4e69f9f7e1e2"} Dec 03 12:56:38 crc kubenswrapper[5052]: I1203 12:56:38.916012 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/configure-network-edpm-compute-global-edpm-compute-global-qf7vq" podStartSLOduration=2.111278224 podStartE2EDuration="2.915981167s" podCreationTimestamp="2025-12-03 12:56:36 +0000 UTC" firstStartedPulling="2025-12-03 12:56:37.76320194 +0000 UTC m=+5788.351443714" lastFinishedPulling="2025-12-03 12:56:38.567904893 +0000 UTC m=+5789.156146657" observedRunningTime="2025-12-03 12:56:38.909700898 +0000 UTC m=+5789.497942662" watchObservedRunningTime="2025-12-03 12:56:38.915981167 +0000 UTC m=+5789.504222931" Dec 03 12:56:39 crc kubenswrapper[5052]: I1203 12:56:39.677325 5052 scope.go:117] "RemoveContainer" containerID="69fa7fbcd2aceb7c53703a19508876f93a97dc9adabb79c16fb691d90f606d58" Dec 03 12:56:39 crc kubenswrapper[5052]: E1203 12:56:39.678554 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 12:56:40 crc kubenswrapper[5052]: I1203 12:56:40.925849 5052 generic.go:334] "Generic (PLEG): container finished" podID="c8557042-afbc-4b35-996c-e2c060db2f2c" containerID="317bade2bc973a40de5b34c60a1f485d2e6ce7d4603e6c7351dc4e69f9f7e1e2" exitCode=0 Dec 03 12:56:40 crc kubenswrapper[5052]: I1203 12:56:40.926038 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/configure-network-edpm-compute-global-edpm-compute-global-qf7vq" event={"ID":"c8557042-afbc-4b35-996c-e2c060db2f2c","Type":"ContainerDied","Data":"317bade2bc973a40de5b34c60a1f485d2e6ce7d4603e6c7351dc4e69f9f7e1e2"} Dec 03 12:56:42 crc kubenswrapper[5052]: I1203 12:56:42.217944 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/configure-network-edpm-compute-global-edpm-compute-global-qf7vq" Dec 03 12:56:42 crc kubenswrapper[5052]: I1203 12:56:42.388813 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/c8557042-afbc-4b35-996c-e2c060db2f2c-inventory\") pod \"c8557042-afbc-4b35-996c-e2c060db2f2c\" (UID: \"c8557042-afbc-4b35-996c-e2c060db2f2c\") " Dec 03 12:56:42 crc kubenswrapper[5052]: I1203 12:56:42.388882 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bd46x\" (UniqueName: \"kubernetes.io/projected/c8557042-afbc-4b35-996c-e2c060db2f2c-kube-api-access-bd46x\") pod \"c8557042-afbc-4b35-996c-e2c060db2f2c\" (UID: \"c8557042-afbc-4b35-996c-e2c060db2f2c\") " Dec 03 12:56:42 crc kubenswrapper[5052]: I1203 12:56:42.388969 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/c8557042-afbc-4b35-996c-e2c060db2f2c-ssh-key\") pod \"c8557042-afbc-4b35-996c-e2c060db2f2c\" (UID: \"c8557042-afbc-4b35-996c-e2c060db2f2c\") " Dec 03 12:56:42 crc kubenswrapper[5052]: I1203 12:56:42.394658 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c8557042-afbc-4b35-996c-e2c060db2f2c-kube-api-access-bd46x" (OuterVolumeSpecName: "kube-api-access-bd46x") pod "c8557042-afbc-4b35-996c-e2c060db2f2c" (UID: "c8557042-afbc-4b35-996c-e2c060db2f2c"). InnerVolumeSpecName "kube-api-access-bd46x". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:56:42 crc kubenswrapper[5052]: I1203 12:56:42.410662 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c8557042-afbc-4b35-996c-e2c060db2f2c-ssh-key" (OuterVolumeSpecName: "ssh-key") pod "c8557042-afbc-4b35-996c-e2c060db2f2c" (UID: "c8557042-afbc-4b35-996c-e2c060db2f2c"). InnerVolumeSpecName "ssh-key". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:56:42 crc kubenswrapper[5052]: I1203 12:56:42.410708 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c8557042-afbc-4b35-996c-e2c060db2f2c-inventory" (OuterVolumeSpecName: "inventory") pod "c8557042-afbc-4b35-996c-e2c060db2f2c" (UID: "c8557042-afbc-4b35-996c-e2c060db2f2c"). InnerVolumeSpecName "inventory". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:56:42 crc kubenswrapper[5052]: I1203 12:56:42.490727 5052 reconciler_common.go:293] "Volume detached for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/c8557042-afbc-4b35-996c-e2c060db2f2c-inventory\") on node \"crc\" DevicePath \"\"" Dec 03 12:56:42 crc kubenswrapper[5052]: I1203 12:56:42.490780 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-bd46x\" (UniqueName: \"kubernetes.io/projected/c8557042-afbc-4b35-996c-e2c060db2f2c-kube-api-access-bd46x\") on node \"crc\" DevicePath \"\"" Dec 03 12:56:42 crc kubenswrapper[5052]: I1203 12:56:42.490794 5052 reconciler_common.go:293] "Volume detached for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/c8557042-afbc-4b35-996c-e2c060db2f2c-ssh-key\") on node \"crc\" DevicePath \"\"" Dec 03 12:56:42 crc kubenswrapper[5052]: I1203 12:56:42.945135 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/configure-network-edpm-compute-global-edpm-compute-global-qf7vq" event={"ID":"c8557042-afbc-4b35-996c-e2c060db2f2c","Type":"ContainerDied","Data":"0970cd99b90b06a9e18bb33920469f41ab6b99df7fd9bdd322626bde7d3faf43"} Dec 03 12:56:42 crc kubenswrapper[5052]: I1203 12:56:42.945390 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0970cd99b90b06a9e18bb33920469f41ab6b99df7fd9bdd322626bde7d3faf43" Dec 03 12:56:42 crc kubenswrapper[5052]: I1203 12:56:42.945186 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/configure-network-edpm-compute-global-edpm-compute-global-qf7vq" Dec 03 12:56:43 crc kubenswrapper[5052]: I1203 12:56:43.019608 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/validate-network-edpm-compute-global-edpm-compute-global-nnvwn"] Dec 03 12:56:43 crc kubenswrapper[5052]: E1203 12:56:43.020006 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="c8557042-afbc-4b35-996c-e2c060db2f2c" containerName="configure-network-edpm-compute-global-edpm-compute-global" Dec 03 12:56:43 crc kubenswrapper[5052]: I1203 12:56:43.020027 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="c8557042-afbc-4b35-996c-e2c060db2f2c" containerName="configure-network-edpm-compute-global-edpm-compute-global" Dec 03 12:56:43 crc kubenswrapper[5052]: I1203 12:56:43.020224 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="c8557042-afbc-4b35-996c-e2c060db2f2c" containerName="configure-network-edpm-compute-global-edpm-compute-global" Dec 03 12:56:43 crc kubenswrapper[5052]: I1203 12:56:43.020877 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/validate-network-edpm-compute-global-edpm-compute-global-nnvwn" Dec 03 12:56:43 crc kubenswrapper[5052]: I1203 12:56:43.026073 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"dataplane-ansible-ssh-private-key-secret" Dec 03 12:56:43 crc kubenswrapper[5052]: I1203 12:56:43.026265 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"dataplanenodeset-edpm-compute-global" Dec 03 12:56:43 crc kubenswrapper[5052]: I1203 12:56:43.028134 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"edpm-compute-global-dockercfg-p5rjx" Dec 03 12:56:43 crc kubenswrapper[5052]: I1203 12:56:43.030977 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/validate-network-edpm-compute-global-edpm-compute-global-nnvwn"] Dec 03 12:56:43 crc kubenswrapper[5052]: I1203 12:56:43.034245 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"openstack-aee-default-env" Dec 03 12:56:43 crc kubenswrapper[5052]: I1203 12:56:43.099212 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/6d5fab08-b680-425a-a4a2-c0b0858b1d8b-inventory\") pod \"validate-network-edpm-compute-global-edpm-compute-global-nnvwn\" (UID: \"6d5fab08-b680-425a-a4a2-c0b0858b1d8b\") " pod="openstack-kuttl-tests/validate-network-edpm-compute-global-edpm-compute-global-nnvwn" Dec 03 12:56:43 crc kubenswrapper[5052]: I1203 12:56:43.099298 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bbz4k\" (UniqueName: \"kubernetes.io/projected/6d5fab08-b680-425a-a4a2-c0b0858b1d8b-kube-api-access-bbz4k\") pod \"validate-network-edpm-compute-global-edpm-compute-global-nnvwn\" (UID: \"6d5fab08-b680-425a-a4a2-c0b0858b1d8b\") " pod="openstack-kuttl-tests/validate-network-edpm-compute-global-edpm-compute-global-nnvwn" Dec 03 12:56:43 crc kubenswrapper[5052]: I1203 12:56:43.099362 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/6d5fab08-b680-425a-a4a2-c0b0858b1d8b-ssh-key\") pod \"validate-network-edpm-compute-global-edpm-compute-global-nnvwn\" (UID: \"6d5fab08-b680-425a-a4a2-c0b0858b1d8b\") " pod="openstack-kuttl-tests/validate-network-edpm-compute-global-edpm-compute-global-nnvwn" Dec 03 12:56:43 crc kubenswrapper[5052]: I1203 12:56:43.200730 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-bbz4k\" (UniqueName: \"kubernetes.io/projected/6d5fab08-b680-425a-a4a2-c0b0858b1d8b-kube-api-access-bbz4k\") pod \"validate-network-edpm-compute-global-edpm-compute-global-nnvwn\" (UID: \"6d5fab08-b680-425a-a4a2-c0b0858b1d8b\") " pod="openstack-kuttl-tests/validate-network-edpm-compute-global-edpm-compute-global-nnvwn" Dec 03 12:56:43 crc kubenswrapper[5052]: I1203 12:56:43.200802 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/6d5fab08-b680-425a-a4a2-c0b0858b1d8b-ssh-key\") pod \"validate-network-edpm-compute-global-edpm-compute-global-nnvwn\" (UID: \"6d5fab08-b680-425a-a4a2-c0b0858b1d8b\") " pod="openstack-kuttl-tests/validate-network-edpm-compute-global-edpm-compute-global-nnvwn" Dec 03 12:56:43 crc kubenswrapper[5052]: I1203 12:56:43.200845 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/6d5fab08-b680-425a-a4a2-c0b0858b1d8b-inventory\") pod \"validate-network-edpm-compute-global-edpm-compute-global-nnvwn\" (UID: \"6d5fab08-b680-425a-a4a2-c0b0858b1d8b\") " pod="openstack-kuttl-tests/validate-network-edpm-compute-global-edpm-compute-global-nnvwn" Dec 03 12:56:43 crc kubenswrapper[5052]: I1203 12:56:43.204144 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/6d5fab08-b680-425a-a4a2-c0b0858b1d8b-inventory\") pod \"validate-network-edpm-compute-global-edpm-compute-global-nnvwn\" (UID: \"6d5fab08-b680-425a-a4a2-c0b0858b1d8b\") " pod="openstack-kuttl-tests/validate-network-edpm-compute-global-edpm-compute-global-nnvwn" Dec 03 12:56:43 crc kubenswrapper[5052]: I1203 12:56:43.205303 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/6d5fab08-b680-425a-a4a2-c0b0858b1d8b-ssh-key\") pod \"validate-network-edpm-compute-global-edpm-compute-global-nnvwn\" (UID: \"6d5fab08-b680-425a-a4a2-c0b0858b1d8b\") " pod="openstack-kuttl-tests/validate-network-edpm-compute-global-edpm-compute-global-nnvwn" Dec 03 12:56:43 crc kubenswrapper[5052]: I1203 12:56:43.221200 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-bbz4k\" (UniqueName: \"kubernetes.io/projected/6d5fab08-b680-425a-a4a2-c0b0858b1d8b-kube-api-access-bbz4k\") pod \"validate-network-edpm-compute-global-edpm-compute-global-nnvwn\" (UID: \"6d5fab08-b680-425a-a4a2-c0b0858b1d8b\") " pod="openstack-kuttl-tests/validate-network-edpm-compute-global-edpm-compute-global-nnvwn" Dec 03 12:56:43 crc kubenswrapper[5052]: I1203 12:56:43.352116 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/validate-network-edpm-compute-global-edpm-compute-global-nnvwn" Dec 03 12:56:43 crc kubenswrapper[5052]: I1203 12:56:43.602781 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/validate-network-edpm-compute-global-edpm-compute-global-nnvwn"] Dec 03 12:56:43 crc kubenswrapper[5052]: I1203 12:56:43.956458 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/validate-network-edpm-compute-global-edpm-compute-global-nnvwn" event={"ID":"6d5fab08-b680-425a-a4a2-c0b0858b1d8b","Type":"ContainerStarted","Data":"35438b8e0fec532aaacad1d0672462d6d47b43f885943763f1f08896d7d63f3a"} Dec 03 12:56:44 crc kubenswrapper[5052]: I1203 12:56:44.968179 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/validate-network-edpm-compute-global-edpm-compute-global-nnvwn" event={"ID":"6d5fab08-b680-425a-a4a2-c0b0858b1d8b","Type":"ContainerStarted","Data":"31a8f9017cd879bdfe8012e920146b64e25c9b677ff80153e43f799ec1f6e0e2"} Dec 03 12:56:44 crc kubenswrapper[5052]: I1203 12:56:44.985253 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/validate-network-edpm-compute-global-edpm-compute-global-nnvwn" podStartSLOduration=2.213518938 podStartE2EDuration="2.985236337s" podCreationTimestamp="2025-12-03 12:56:42 +0000 UTC" firstStartedPulling="2025-12-03 12:56:43.6097171 +0000 UTC m=+5794.197958864" lastFinishedPulling="2025-12-03 12:56:44.381434469 +0000 UTC m=+5794.969676263" observedRunningTime="2025-12-03 12:56:44.981957593 +0000 UTC m=+5795.570199367" watchObservedRunningTime="2025-12-03 12:56:44.985236337 +0000 UTC m=+5795.573478111" Dec 03 12:56:45 crc kubenswrapper[5052]: I1203 12:56:45.980738 5052 generic.go:334] "Generic (PLEG): container finished" podID="6d5fab08-b680-425a-a4a2-c0b0858b1d8b" containerID="31a8f9017cd879bdfe8012e920146b64e25c9b677ff80153e43f799ec1f6e0e2" exitCode=0 Dec 03 12:56:45 crc kubenswrapper[5052]: I1203 12:56:45.980785 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/validate-network-edpm-compute-global-edpm-compute-global-nnvwn" event={"ID":"6d5fab08-b680-425a-a4a2-c0b0858b1d8b","Type":"ContainerDied","Data":"31a8f9017cd879bdfe8012e920146b64e25c9b677ff80153e43f799ec1f6e0e2"} Dec 03 12:56:47 crc kubenswrapper[5052]: I1203 12:56:47.271328 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/validate-network-edpm-compute-global-edpm-compute-global-nnvwn" Dec 03 12:56:47 crc kubenswrapper[5052]: I1203 12:56:47.362478 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/6d5fab08-b680-425a-a4a2-c0b0858b1d8b-inventory\") pod \"6d5fab08-b680-425a-a4a2-c0b0858b1d8b\" (UID: \"6d5fab08-b680-425a-a4a2-c0b0858b1d8b\") " Dec 03 12:56:47 crc kubenswrapper[5052]: I1203 12:56:47.362657 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bbz4k\" (UniqueName: \"kubernetes.io/projected/6d5fab08-b680-425a-a4a2-c0b0858b1d8b-kube-api-access-bbz4k\") pod \"6d5fab08-b680-425a-a4a2-c0b0858b1d8b\" (UID: \"6d5fab08-b680-425a-a4a2-c0b0858b1d8b\") " Dec 03 12:56:47 crc kubenswrapper[5052]: I1203 12:56:47.362735 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/6d5fab08-b680-425a-a4a2-c0b0858b1d8b-ssh-key\") pod \"6d5fab08-b680-425a-a4a2-c0b0858b1d8b\" (UID: \"6d5fab08-b680-425a-a4a2-c0b0858b1d8b\") " Dec 03 12:56:47 crc kubenswrapper[5052]: I1203 12:56:47.369112 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6d5fab08-b680-425a-a4a2-c0b0858b1d8b-kube-api-access-bbz4k" (OuterVolumeSpecName: "kube-api-access-bbz4k") pod "6d5fab08-b680-425a-a4a2-c0b0858b1d8b" (UID: "6d5fab08-b680-425a-a4a2-c0b0858b1d8b"). InnerVolumeSpecName "kube-api-access-bbz4k". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:56:47 crc kubenswrapper[5052]: I1203 12:56:47.387642 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6d5fab08-b680-425a-a4a2-c0b0858b1d8b-ssh-key" (OuterVolumeSpecName: "ssh-key") pod "6d5fab08-b680-425a-a4a2-c0b0858b1d8b" (UID: "6d5fab08-b680-425a-a4a2-c0b0858b1d8b"). InnerVolumeSpecName "ssh-key". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:56:47 crc kubenswrapper[5052]: I1203 12:56:47.390459 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6d5fab08-b680-425a-a4a2-c0b0858b1d8b-inventory" (OuterVolumeSpecName: "inventory") pod "6d5fab08-b680-425a-a4a2-c0b0858b1d8b" (UID: "6d5fab08-b680-425a-a4a2-c0b0858b1d8b"). InnerVolumeSpecName "inventory". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:56:47 crc kubenswrapper[5052]: I1203 12:56:47.492417 5052 reconciler_common.go:293] "Volume detached for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/6d5fab08-b680-425a-a4a2-c0b0858b1d8b-inventory\") on node \"crc\" DevicePath \"\"" Dec 03 12:56:47 crc kubenswrapper[5052]: I1203 12:56:47.492474 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-bbz4k\" (UniqueName: \"kubernetes.io/projected/6d5fab08-b680-425a-a4a2-c0b0858b1d8b-kube-api-access-bbz4k\") on node \"crc\" DevicePath \"\"" Dec 03 12:56:47 crc kubenswrapper[5052]: I1203 12:56:47.492494 5052 reconciler_common.go:293] "Volume detached for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/6d5fab08-b680-425a-a4a2-c0b0858b1d8b-ssh-key\") on node \"crc\" DevicePath \"\"" Dec 03 12:56:47 crc kubenswrapper[5052]: I1203 12:56:47.999210 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/validate-network-edpm-compute-global-edpm-compute-global-nnvwn" event={"ID":"6d5fab08-b680-425a-a4a2-c0b0858b1d8b","Type":"ContainerDied","Data":"35438b8e0fec532aaacad1d0672462d6d47b43f885943763f1f08896d7d63f3a"} Dec 03 12:56:47 crc kubenswrapper[5052]: I1203 12:56:47.999255 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="35438b8e0fec532aaacad1d0672462d6d47b43f885943763f1f08896d7d63f3a" Dec 03 12:56:47 crc kubenswrapper[5052]: I1203 12:56:47.999309 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/validate-network-edpm-compute-global-edpm-compute-global-nnvwn" Dec 03 12:56:48 crc kubenswrapper[5052]: I1203 12:56:48.060366 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/install-os-edpm-compute-global-edpm-compute-global-xf85w"] Dec 03 12:56:48 crc kubenswrapper[5052]: E1203 12:56:48.061050 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="6d5fab08-b680-425a-a4a2-c0b0858b1d8b" containerName="validate-network-edpm-compute-global-edpm-compute-global" Dec 03 12:56:48 crc kubenswrapper[5052]: I1203 12:56:48.061146 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="6d5fab08-b680-425a-a4a2-c0b0858b1d8b" containerName="validate-network-edpm-compute-global-edpm-compute-global" Dec 03 12:56:48 crc kubenswrapper[5052]: I1203 12:56:48.061407 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="6d5fab08-b680-425a-a4a2-c0b0858b1d8b" containerName="validate-network-edpm-compute-global-edpm-compute-global" Dec 03 12:56:48 crc kubenswrapper[5052]: I1203 12:56:48.062125 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/install-os-edpm-compute-global-edpm-compute-global-xf85w" Dec 03 12:56:48 crc kubenswrapper[5052]: I1203 12:56:48.065009 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"edpm-compute-global-dockercfg-p5rjx" Dec 03 12:56:48 crc kubenswrapper[5052]: I1203 12:56:48.065536 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"dataplanenodeset-edpm-compute-global" Dec 03 12:56:48 crc kubenswrapper[5052]: I1203 12:56:48.065646 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"openstack-aee-default-env" Dec 03 12:56:48 crc kubenswrapper[5052]: I1203 12:56:48.065733 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"dataplane-ansible-ssh-private-key-secret" Dec 03 12:56:48 crc kubenswrapper[5052]: I1203 12:56:48.073938 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/install-os-edpm-compute-global-edpm-compute-global-xf85w"] Dec 03 12:56:48 crc kubenswrapper[5052]: I1203 12:56:48.201926 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/edd8e798-07f9-4ad1-bb97-8b64840c7a4e-inventory\") pod \"install-os-edpm-compute-global-edpm-compute-global-xf85w\" (UID: \"edd8e798-07f9-4ad1-bb97-8b64840c7a4e\") " pod="openstack-kuttl-tests/install-os-edpm-compute-global-edpm-compute-global-xf85w" Dec 03 12:56:48 crc kubenswrapper[5052]: I1203 12:56:48.202405 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/edd8e798-07f9-4ad1-bb97-8b64840c7a4e-ssh-key\") pod \"install-os-edpm-compute-global-edpm-compute-global-xf85w\" (UID: \"edd8e798-07f9-4ad1-bb97-8b64840c7a4e\") " pod="openstack-kuttl-tests/install-os-edpm-compute-global-edpm-compute-global-xf85w" Dec 03 12:56:48 crc kubenswrapper[5052]: I1203 12:56:48.202594 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gcjtb\" (UniqueName: \"kubernetes.io/projected/edd8e798-07f9-4ad1-bb97-8b64840c7a4e-kube-api-access-gcjtb\") pod \"install-os-edpm-compute-global-edpm-compute-global-xf85w\" (UID: \"edd8e798-07f9-4ad1-bb97-8b64840c7a4e\") " pod="openstack-kuttl-tests/install-os-edpm-compute-global-edpm-compute-global-xf85w" Dec 03 12:56:48 crc kubenswrapper[5052]: I1203 12:56:48.303902 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-gcjtb\" (UniqueName: \"kubernetes.io/projected/edd8e798-07f9-4ad1-bb97-8b64840c7a4e-kube-api-access-gcjtb\") pod \"install-os-edpm-compute-global-edpm-compute-global-xf85w\" (UID: \"edd8e798-07f9-4ad1-bb97-8b64840c7a4e\") " pod="openstack-kuttl-tests/install-os-edpm-compute-global-edpm-compute-global-xf85w" Dec 03 12:56:48 crc kubenswrapper[5052]: I1203 12:56:48.303983 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/edd8e798-07f9-4ad1-bb97-8b64840c7a4e-inventory\") pod \"install-os-edpm-compute-global-edpm-compute-global-xf85w\" (UID: \"edd8e798-07f9-4ad1-bb97-8b64840c7a4e\") " pod="openstack-kuttl-tests/install-os-edpm-compute-global-edpm-compute-global-xf85w" Dec 03 12:56:48 crc kubenswrapper[5052]: I1203 12:56:48.304010 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/edd8e798-07f9-4ad1-bb97-8b64840c7a4e-ssh-key\") pod \"install-os-edpm-compute-global-edpm-compute-global-xf85w\" (UID: \"edd8e798-07f9-4ad1-bb97-8b64840c7a4e\") " pod="openstack-kuttl-tests/install-os-edpm-compute-global-edpm-compute-global-xf85w" Dec 03 12:56:48 crc kubenswrapper[5052]: I1203 12:56:48.307794 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/edd8e798-07f9-4ad1-bb97-8b64840c7a4e-ssh-key\") pod \"install-os-edpm-compute-global-edpm-compute-global-xf85w\" (UID: \"edd8e798-07f9-4ad1-bb97-8b64840c7a4e\") " pod="openstack-kuttl-tests/install-os-edpm-compute-global-edpm-compute-global-xf85w" Dec 03 12:56:48 crc kubenswrapper[5052]: I1203 12:56:48.307893 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/edd8e798-07f9-4ad1-bb97-8b64840c7a4e-inventory\") pod \"install-os-edpm-compute-global-edpm-compute-global-xf85w\" (UID: \"edd8e798-07f9-4ad1-bb97-8b64840c7a4e\") " pod="openstack-kuttl-tests/install-os-edpm-compute-global-edpm-compute-global-xf85w" Dec 03 12:56:48 crc kubenswrapper[5052]: I1203 12:56:48.321185 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-gcjtb\" (UniqueName: \"kubernetes.io/projected/edd8e798-07f9-4ad1-bb97-8b64840c7a4e-kube-api-access-gcjtb\") pod \"install-os-edpm-compute-global-edpm-compute-global-xf85w\" (UID: \"edd8e798-07f9-4ad1-bb97-8b64840c7a4e\") " pod="openstack-kuttl-tests/install-os-edpm-compute-global-edpm-compute-global-xf85w" Dec 03 12:56:48 crc kubenswrapper[5052]: I1203 12:56:48.379866 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/install-os-edpm-compute-global-edpm-compute-global-xf85w" Dec 03 12:56:48 crc kubenswrapper[5052]: I1203 12:56:48.778757 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/install-os-edpm-compute-global-edpm-compute-global-xf85w"] Dec 03 12:56:49 crc kubenswrapper[5052]: I1203 12:56:49.007845 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/install-os-edpm-compute-global-edpm-compute-global-xf85w" event={"ID":"edd8e798-07f9-4ad1-bb97-8b64840c7a4e","Type":"ContainerStarted","Data":"301db4df7d6cd84eb659ec1d48a858a37ae4987edf06694b20b6a520bce13d31"} Dec 03 12:56:51 crc kubenswrapper[5052]: I1203 12:56:51.027023 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/install-os-edpm-compute-global-edpm-compute-global-xf85w" event={"ID":"edd8e798-07f9-4ad1-bb97-8b64840c7a4e","Type":"ContainerStarted","Data":"75cdca491987d282e0403d1bab06e238526d67486286e68090da422766bc9093"} Dec 03 12:56:51 crc kubenswrapper[5052]: I1203 12:56:51.055929 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/install-os-edpm-compute-global-edpm-compute-global-xf85w" podStartSLOduration=1.870131547 podStartE2EDuration="3.055904288s" podCreationTimestamp="2025-12-03 12:56:48 +0000 UTC" firstStartedPulling="2025-12-03 12:56:48.778841248 +0000 UTC m=+5799.367083012" lastFinishedPulling="2025-12-03 12:56:49.964613989 +0000 UTC m=+5800.552855753" observedRunningTime="2025-12-03 12:56:51.045048298 +0000 UTC m=+5801.633290062" watchObservedRunningTime="2025-12-03 12:56:51.055904288 +0000 UTC m=+5801.644146062" Dec 03 12:56:51 crc kubenswrapper[5052]: E1203 12:56:51.330571 5052 cadvisor_stats_provider.go:516] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podedd8e798_07f9_4ad1_bb97_8b64840c7a4e.slice/crio-75cdca491987d282e0403d1bab06e238526d67486286e68090da422766bc9093.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podedd8e798_07f9_4ad1_bb97_8b64840c7a4e.slice/crio-conmon-75cdca491987d282e0403d1bab06e238526d67486286e68090da422766bc9093.scope\": RecentStats: unable to find data in memory cache]" Dec 03 12:56:52 crc kubenswrapper[5052]: I1203 12:56:52.038988 5052 generic.go:334] "Generic (PLEG): container finished" podID="edd8e798-07f9-4ad1-bb97-8b64840c7a4e" containerID="75cdca491987d282e0403d1bab06e238526d67486286e68090da422766bc9093" exitCode=0 Dec 03 12:56:52 crc kubenswrapper[5052]: I1203 12:56:52.039044 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/install-os-edpm-compute-global-edpm-compute-global-xf85w" event={"ID":"edd8e798-07f9-4ad1-bb97-8b64840c7a4e","Type":"ContainerDied","Data":"75cdca491987d282e0403d1bab06e238526d67486286e68090da422766bc9093"} Dec 03 12:56:52 crc kubenswrapper[5052]: I1203 12:56:52.673458 5052 scope.go:117] "RemoveContainer" containerID="69fa7fbcd2aceb7c53703a19508876f93a97dc9adabb79c16fb691d90f606d58" Dec 03 12:56:52 crc kubenswrapper[5052]: E1203 12:56:52.673985 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 12:56:53 crc kubenswrapper[5052]: I1203 12:56:53.313090 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/install-os-edpm-compute-global-edpm-compute-global-xf85w" Dec 03 12:56:53 crc kubenswrapper[5052]: I1203 12:56:53.480118 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/edd8e798-07f9-4ad1-bb97-8b64840c7a4e-inventory\") pod \"edd8e798-07f9-4ad1-bb97-8b64840c7a4e\" (UID: \"edd8e798-07f9-4ad1-bb97-8b64840c7a4e\") " Dec 03 12:56:53 crc kubenswrapper[5052]: I1203 12:56:53.480251 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/edd8e798-07f9-4ad1-bb97-8b64840c7a4e-ssh-key\") pod \"edd8e798-07f9-4ad1-bb97-8b64840c7a4e\" (UID: \"edd8e798-07f9-4ad1-bb97-8b64840c7a4e\") " Dec 03 12:56:53 crc kubenswrapper[5052]: I1203 12:56:53.480280 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gcjtb\" (UniqueName: \"kubernetes.io/projected/edd8e798-07f9-4ad1-bb97-8b64840c7a4e-kube-api-access-gcjtb\") pod \"edd8e798-07f9-4ad1-bb97-8b64840c7a4e\" (UID: \"edd8e798-07f9-4ad1-bb97-8b64840c7a4e\") " Dec 03 12:56:53 crc kubenswrapper[5052]: I1203 12:56:53.485333 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/edd8e798-07f9-4ad1-bb97-8b64840c7a4e-kube-api-access-gcjtb" (OuterVolumeSpecName: "kube-api-access-gcjtb") pod "edd8e798-07f9-4ad1-bb97-8b64840c7a4e" (UID: "edd8e798-07f9-4ad1-bb97-8b64840c7a4e"). InnerVolumeSpecName "kube-api-access-gcjtb". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:56:53 crc kubenswrapper[5052]: I1203 12:56:53.501450 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/edd8e798-07f9-4ad1-bb97-8b64840c7a4e-inventory" (OuterVolumeSpecName: "inventory") pod "edd8e798-07f9-4ad1-bb97-8b64840c7a4e" (UID: "edd8e798-07f9-4ad1-bb97-8b64840c7a4e"). InnerVolumeSpecName "inventory". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:56:53 crc kubenswrapper[5052]: I1203 12:56:53.503558 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/edd8e798-07f9-4ad1-bb97-8b64840c7a4e-ssh-key" (OuterVolumeSpecName: "ssh-key") pod "edd8e798-07f9-4ad1-bb97-8b64840c7a4e" (UID: "edd8e798-07f9-4ad1-bb97-8b64840c7a4e"). InnerVolumeSpecName "ssh-key". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:56:53 crc kubenswrapper[5052]: I1203 12:56:53.581821 5052 reconciler_common.go:293] "Volume detached for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/edd8e798-07f9-4ad1-bb97-8b64840c7a4e-inventory\") on node \"crc\" DevicePath \"\"" Dec 03 12:56:53 crc kubenswrapper[5052]: I1203 12:56:53.581860 5052 reconciler_common.go:293] "Volume detached for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/edd8e798-07f9-4ad1-bb97-8b64840c7a4e-ssh-key\") on node \"crc\" DevicePath \"\"" Dec 03 12:56:53 crc kubenswrapper[5052]: I1203 12:56:53.581874 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-gcjtb\" (UniqueName: \"kubernetes.io/projected/edd8e798-07f9-4ad1-bb97-8b64840c7a4e-kube-api-access-gcjtb\") on node \"crc\" DevicePath \"\"" Dec 03 12:56:54 crc kubenswrapper[5052]: I1203 12:56:54.060261 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/install-os-edpm-compute-global-edpm-compute-global-xf85w" event={"ID":"edd8e798-07f9-4ad1-bb97-8b64840c7a4e","Type":"ContainerDied","Data":"301db4df7d6cd84eb659ec1d48a858a37ae4987edf06694b20b6a520bce13d31"} Dec 03 12:56:54 crc kubenswrapper[5052]: I1203 12:56:54.060305 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/install-os-edpm-compute-global-edpm-compute-global-xf85w" Dec 03 12:56:54 crc kubenswrapper[5052]: I1203 12:56:54.060322 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="301db4df7d6cd84eb659ec1d48a858a37ae4987edf06694b20b6a520bce13d31" Dec 03 12:56:54 crc kubenswrapper[5052]: I1203 12:56:54.114946 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/configure-os-edpm-compute-global-edpm-compute-global-bpx67"] Dec 03 12:56:54 crc kubenswrapper[5052]: E1203 12:56:54.115473 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="edd8e798-07f9-4ad1-bb97-8b64840c7a4e" containerName="install-os-edpm-compute-global-edpm-compute-global" Dec 03 12:56:54 crc kubenswrapper[5052]: I1203 12:56:54.115497 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="edd8e798-07f9-4ad1-bb97-8b64840c7a4e" containerName="install-os-edpm-compute-global-edpm-compute-global" Dec 03 12:56:54 crc kubenswrapper[5052]: I1203 12:56:54.115796 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="edd8e798-07f9-4ad1-bb97-8b64840c7a4e" containerName="install-os-edpm-compute-global-edpm-compute-global" Dec 03 12:56:54 crc kubenswrapper[5052]: I1203 12:56:54.116388 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/configure-os-edpm-compute-global-edpm-compute-global-bpx67" Dec 03 12:56:54 crc kubenswrapper[5052]: I1203 12:56:54.125584 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/configure-os-edpm-compute-global-edpm-compute-global-bpx67"] Dec 03 12:56:54 crc kubenswrapper[5052]: I1203 12:56:54.127430 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"dataplane-ansible-ssh-private-key-secret" Dec 03 12:56:54 crc kubenswrapper[5052]: I1203 12:56:54.127494 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"openstack-aee-default-env" Dec 03 12:56:54 crc kubenswrapper[5052]: I1203 12:56:54.127607 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"edpm-compute-global-dockercfg-p5rjx" Dec 03 12:56:54 crc kubenswrapper[5052]: I1203 12:56:54.127720 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"dataplanenodeset-edpm-compute-global" Dec 03 12:56:54 crc kubenswrapper[5052]: I1203 12:56:54.293310 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/4c4ff49c-5eeb-43cb-ab65-1a284cf1e79a-ssh-key\") pod \"configure-os-edpm-compute-global-edpm-compute-global-bpx67\" (UID: \"4c4ff49c-5eeb-43cb-ab65-1a284cf1e79a\") " pod="openstack-kuttl-tests/configure-os-edpm-compute-global-edpm-compute-global-bpx67" Dec 03 12:56:54 crc kubenswrapper[5052]: I1203 12:56:54.293440 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/4c4ff49c-5eeb-43cb-ab65-1a284cf1e79a-inventory\") pod \"configure-os-edpm-compute-global-edpm-compute-global-bpx67\" (UID: \"4c4ff49c-5eeb-43cb-ab65-1a284cf1e79a\") " pod="openstack-kuttl-tests/configure-os-edpm-compute-global-edpm-compute-global-bpx67" Dec 03 12:56:54 crc kubenswrapper[5052]: I1203 12:56:54.293488 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s458q\" (UniqueName: \"kubernetes.io/projected/4c4ff49c-5eeb-43cb-ab65-1a284cf1e79a-kube-api-access-s458q\") pod \"configure-os-edpm-compute-global-edpm-compute-global-bpx67\" (UID: \"4c4ff49c-5eeb-43cb-ab65-1a284cf1e79a\") " pod="openstack-kuttl-tests/configure-os-edpm-compute-global-edpm-compute-global-bpx67" Dec 03 12:56:54 crc kubenswrapper[5052]: I1203 12:56:54.395064 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/4c4ff49c-5eeb-43cb-ab65-1a284cf1e79a-ssh-key\") pod \"configure-os-edpm-compute-global-edpm-compute-global-bpx67\" (UID: \"4c4ff49c-5eeb-43cb-ab65-1a284cf1e79a\") " pod="openstack-kuttl-tests/configure-os-edpm-compute-global-edpm-compute-global-bpx67" Dec 03 12:56:54 crc kubenswrapper[5052]: I1203 12:56:54.395219 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/4c4ff49c-5eeb-43cb-ab65-1a284cf1e79a-inventory\") pod \"configure-os-edpm-compute-global-edpm-compute-global-bpx67\" (UID: \"4c4ff49c-5eeb-43cb-ab65-1a284cf1e79a\") " pod="openstack-kuttl-tests/configure-os-edpm-compute-global-edpm-compute-global-bpx67" Dec 03 12:56:54 crc kubenswrapper[5052]: I1203 12:56:54.395309 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-s458q\" (UniqueName: \"kubernetes.io/projected/4c4ff49c-5eeb-43cb-ab65-1a284cf1e79a-kube-api-access-s458q\") pod \"configure-os-edpm-compute-global-edpm-compute-global-bpx67\" (UID: \"4c4ff49c-5eeb-43cb-ab65-1a284cf1e79a\") " pod="openstack-kuttl-tests/configure-os-edpm-compute-global-edpm-compute-global-bpx67" Dec 03 12:56:54 crc kubenswrapper[5052]: I1203 12:56:54.400174 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/4c4ff49c-5eeb-43cb-ab65-1a284cf1e79a-ssh-key\") pod \"configure-os-edpm-compute-global-edpm-compute-global-bpx67\" (UID: \"4c4ff49c-5eeb-43cb-ab65-1a284cf1e79a\") " pod="openstack-kuttl-tests/configure-os-edpm-compute-global-edpm-compute-global-bpx67" Dec 03 12:56:54 crc kubenswrapper[5052]: I1203 12:56:54.400878 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/4c4ff49c-5eeb-43cb-ab65-1a284cf1e79a-inventory\") pod \"configure-os-edpm-compute-global-edpm-compute-global-bpx67\" (UID: \"4c4ff49c-5eeb-43cb-ab65-1a284cf1e79a\") " pod="openstack-kuttl-tests/configure-os-edpm-compute-global-edpm-compute-global-bpx67" Dec 03 12:56:54 crc kubenswrapper[5052]: I1203 12:56:54.411182 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-s458q\" (UniqueName: \"kubernetes.io/projected/4c4ff49c-5eeb-43cb-ab65-1a284cf1e79a-kube-api-access-s458q\") pod \"configure-os-edpm-compute-global-edpm-compute-global-bpx67\" (UID: \"4c4ff49c-5eeb-43cb-ab65-1a284cf1e79a\") " pod="openstack-kuttl-tests/configure-os-edpm-compute-global-edpm-compute-global-bpx67" Dec 03 12:56:54 crc kubenswrapper[5052]: I1203 12:56:54.433815 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/configure-os-edpm-compute-global-edpm-compute-global-bpx67" Dec 03 12:56:54 crc kubenswrapper[5052]: I1203 12:56:54.848812 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/configure-os-edpm-compute-global-edpm-compute-global-bpx67"] Dec 03 12:56:54 crc kubenswrapper[5052]: W1203 12:56:54.852910 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4c4ff49c_5eeb_43cb_ab65_1a284cf1e79a.slice/crio-bfb4a62398641169e5657368f64194b9b48b1b6434a6ecf7b5c9e4b3920423ef WatchSource:0}: Error finding container bfb4a62398641169e5657368f64194b9b48b1b6434a6ecf7b5c9e4b3920423ef: Status 404 returned error can't find the container with id bfb4a62398641169e5657368f64194b9b48b1b6434a6ecf7b5c9e4b3920423ef Dec 03 12:56:55 crc kubenswrapper[5052]: I1203 12:56:55.069000 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/configure-os-edpm-compute-global-edpm-compute-global-bpx67" event={"ID":"4c4ff49c-5eeb-43cb-ab65-1a284cf1e79a","Type":"ContainerStarted","Data":"bfb4a62398641169e5657368f64194b9b48b1b6434a6ecf7b5c9e4b3920423ef"} Dec 03 12:56:57 crc kubenswrapper[5052]: I1203 12:56:57.093963 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/configure-os-edpm-compute-global-edpm-compute-global-bpx67" event={"ID":"4c4ff49c-5eeb-43cb-ab65-1a284cf1e79a","Type":"ContainerStarted","Data":"48e513f51c2458ff90bf4c4af1efa154b86bfb7487ef23ef47f70cf0942dd19b"} Dec 03 12:56:57 crc kubenswrapper[5052]: I1203 12:56:57.112305 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/configure-os-edpm-compute-global-edpm-compute-global-bpx67" podStartSLOduration=1.260129571 podStartE2EDuration="3.11228679s" podCreationTimestamp="2025-12-03 12:56:54 +0000 UTC" firstStartedPulling="2025-12-03 12:56:54.85694626 +0000 UTC m=+5805.445188024" lastFinishedPulling="2025-12-03 12:56:56.709103479 +0000 UTC m=+5807.297345243" observedRunningTime="2025-12-03 12:56:57.106959827 +0000 UTC m=+5807.695201591" watchObservedRunningTime="2025-12-03 12:56:57.11228679 +0000 UTC m=+5807.700528554" Dec 03 12:56:59 crc kubenswrapper[5052]: I1203 12:56:59.115799 5052 generic.go:334] "Generic (PLEG): container finished" podID="4c4ff49c-5eeb-43cb-ab65-1a284cf1e79a" containerID="48e513f51c2458ff90bf4c4af1efa154b86bfb7487ef23ef47f70cf0942dd19b" exitCode=0 Dec 03 12:56:59 crc kubenswrapper[5052]: I1203 12:56:59.115906 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/configure-os-edpm-compute-global-edpm-compute-global-bpx67" event={"ID":"4c4ff49c-5eeb-43cb-ab65-1a284cf1e79a","Type":"ContainerDied","Data":"48e513f51c2458ff90bf4c4af1efa154b86bfb7487ef23ef47f70cf0942dd19b"} Dec 03 12:57:00 crc kubenswrapper[5052]: I1203 12:57:00.395053 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/configure-os-edpm-compute-global-edpm-compute-global-bpx67" Dec 03 12:57:00 crc kubenswrapper[5052]: I1203 12:57:00.491388 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/4c4ff49c-5eeb-43cb-ab65-1a284cf1e79a-ssh-key\") pod \"4c4ff49c-5eeb-43cb-ab65-1a284cf1e79a\" (UID: \"4c4ff49c-5eeb-43cb-ab65-1a284cf1e79a\") " Dec 03 12:57:00 crc kubenswrapper[5052]: I1203 12:57:00.491439 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s458q\" (UniqueName: \"kubernetes.io/projected/4c4ff49c-5eeb-43cb-ab65-1a284cf1e79a-kube-api-access-s458q\") pod \"4c4ff49c-5eeb-43cb-ab65-1a284cf1e79a\" (UID: \"4c4ff49c-5eeb-43cb-ab65-1a284cf1e79a\") " Dec 03 12:57:00 crc kubenswrapper[5052]: I1203 12:57:00.491494 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/4c4ff49c-5eeb-43cb-ab65-1a284cf1e79a-inventory\") pod \"4c4ff49c-5eeb-43cb-ab65-1a284cf1e79a\" (UID: \"4c4ff49c-5eeb-43cb-ab65-1a284cf1e79a\") " Dec 03 12:57:00 crc kubenswrapper[5052]: I1203 12:57:00.496451 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4c4ff49c-5eeb-43cb-ab65-1a284cf1e79a-kube-api-access-s458q" (OuterVolumeSpecName: "kube-api-access-s458q") pod "4c4ff49c-5eeb-43cb-ab65-1a284cf1e79a" (UID: "4c4ff49c-5eeb-43cb-ab65-1a284cf1e79a"). InnerVolumeSpecName "kube-api-access-s458q". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:57:00 crc kubenswrapper[5052]: I1203 12:57:00.513755 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4c4ff49c-5eeb-43cb-ab65-1a284cf1e79a-ssh-key" (OuterVolumeSpecName: "ssh-key") pod "4c4ff49c-5eeb-43cb-ab65-1a284cf1e79a" (UID: "4c4ff49c-5eeb-43cb-ab65-1a284cf1e79a"). InnerVolumeSpecName "ssh-key". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:57:00 crc kubenswrapper[5052]: I1203 12:57:00.516794 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4c4ff49c-5eeb-43cb-ab65-1a284cf1e79a-inventory" (OuterVolumeSpecName: "inventory") pod "4c4ff49c-5eeb-43cb-ab65-1a284cf1e79a" (UID: "4c4ff49c-5eeb-43cb-ab65-1a284cf1e79a"). InnerVolumeSpecName "inventory". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:57:00 crc kubenswrapper[5052]: I1203 12:57:00.593791 5052 reconciler_common.go:293] "Volume detached for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/4c4ff49c-5eeb-43cb-ab65-1a284cf1e79a-ssh-key\") on node \"crc\" DevicePath \"\"" Dec 03 12:57:00 crc kubenswrapper[5052]: I1203 12:57:00.593843 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-s458q\" (UniqueName: \"kubernetes.io/projected/4c4ff49c-5eeb-43cb-ab65-1a284cf1e79a-kube-api-access-s458q\") on node \"crc\" DevicePath \"\"" Dec 03 12:57:00 crc kubenswrapper[5052]: I1203 12:57:00.593854 5052 reconciler_common.go:293] "Volume detached for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/4c4ff49c-5eeb-43cb-ab65-1a284cf1e79a-inventory\") on node \"crc\" DevicePath \"\"" Dec 03 12:57:01 crc kubenswrapper[5052]: I1203 12:57:01.138259 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/configure-os-edpm-compute-global-edpm-compute-global-bpx67" event={"ID":"4c4ff49c-5eeb-43cb-ab65-1a284cf1e79a","Type":"ContainerDied","Data":"bfb4a62398641169e5657368f64194b9b48b1b6434a6ecf7b5c9e4b3920423ef"} Dec 03 12:57:01 crc kubenswrapper[5052]: I1203 12:57:01.138635 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bfb4a62398641169e5657368f64194b9b48b1b6434a6ecf7b5c9e4b3920423ef" Dec 03 12:57:01 crc kubenswrapper[5052]: I1203 12:57:01.138568 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/configure-os-edpm-compute-global-edpm-compute-global-bpx67" Dec 03 12:57:01 crc kubenswrapper[5052]: I1203 12:57:01.200772 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/run-os-edpm-compute-global-edpm-compute-global-cj7qg"] Dec 03 12:57:01 crc kubenswrapper[5052]: E1203 12:57:01.202069 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="4c4ff49c-5eeb-43cb-ab65-1a284cf1e79a" containerName="configure-os-edpm-compute-global-edpm-compute-global" Dec 03 12:57:01 crc kubenswrapper[5052]: I1203 12:57:01.202099 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="4c4ff49c-5eeb-43cb-ab65-1a284cf1e79a" containerName="configure-os-edpm-compute-global-edpm-compute-global" Dec 03 12:57:01 crc kubenswrapper[5052]: I1203 12:57:01.203138 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="4c4ff49c-5eeb-43cb-ab65-1a284cf1e79a" containerName="configure-os-edpm-compute-global-edpm-compute-global" Dec 03 12:57:01 crc kubenswrapper[5052]: I1203 12:57:01.204374 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/run-os-edpm-compute-global-edpm-compute-global-cj7qg" Dec 03 12:57:01 crc kubenswrapper[5052]: I1203 12:57:01.208387 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"dataplanenodeset-edpm-compute-global" Dec 03 12:57:01 crc kubenswrapper[5052]: I1203 12:57:01.208491 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"edpm-compute-global-dockercfg-p5rjx" Dec 03 12:57:01 crc kubenswrapper[5052]: I1203 12:57:01.208889 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"dataplane-ansible-ssh-private-key-secret" Dec 03 12:57:01 crc kubenswrapper[5052]: I1203 12:57:01.209833 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"openstack-aee-default-env" Dec 03 12:57:01 crc kubenswrapper[5052]: I1203 12:57:01.237091 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/run-os-edpm-compute-global-edpm-compute-global-cj7qg"] Dec 03 12:57:01 crc kubenswrapper[5052]: I1203 12:57:01.310358 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/a2602d97-293c-4ace-a4e0-974deee70a22-ssh-key\") pod \"run-os-edpm-compute-global-edpm-compute-global-cj7qg\" (UID: \"a2602d97-293c-4ace-a4e0-974deee70a22\") " pod="openstack-kuttl-tests/run-os-edpm-compute-global-edpm-compute-global-cj7qg" Dec 03 12:57:01 crc kubenswrapper[5052]: I1203 12:57:01.310444 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jbqln\" (UniqueName: \"kubernetes.io/projected/a2602d97-293c-4ace-a4e0-974deee70a22-kube-api-access-jbqln\") pod \"run-os-edpm-compute-global-edpm-compute-global-cj7qg\" (UID: \"a2602d97-293c-4ace-a4e0-974deee70a22\") " pod="openstack-kuttl-tests/run-os-edpm-compute-global-edpm-compute-global-cj7qg" Dec 03 12:57:01 crc kubenswrapper[5052]: I1203 12:57:01.310557 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/a2602d97-293c-4ace-a4e0-974deee70a22-inventory\") pod \"run-os-edpm-compute-global-edpm-compute-global-cj7qg\" (UID: \"a2602d97-293c-4ace-a4e0-974deee70a22\") " pod="openstack-kuttl-tests/run-os-edpm-compute-global-edpm-compute-global-cj7qg" Dec 03 12:57:01 crc kubenswrapper[5052]: I1203 12:57:01.411404 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-jbqln\" (UniqueName: \"kubernetes.io/projected/a2602d97-293c-4ace-a4e0-974deee70a22-kube-api-access-jbqln\") pod \"run-os-edpm-compute-global-edpm-compute-global-cj7qg\" (UID: \"a2602d97-293c-4ace-a4e0-974deee70a22\") " pod="openstack-kuttl-tests/run-os-edpm-compute-global-edpm-compute-global-cj7qg" Dec 03 12:57:01 crc kubenswrapper[5052]: I1203 12:57:01.411541 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/a2602d97-293c-4ace-a4e0-974deee70a22-inventory\") pod \"run-os-edpm-compute-global-edpm-compute-global-cj7qg\" (UID: \"a2602d97-293c-4ace-a4e0-974deee70a22\") " pod="openstack-kuttl-tests/run-os-edpm-compute-global-edpm-compute-global-cj7qg" Dec 03 12:57:01 crc kubenswrapper[5052]: I1203 12:57:01.411590 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/a2602d97-293c-4ace-a4e0-974deee70a22-ssh-key\") pod \"run-os-edpm-compute-global-edpm-compute-global-cj7qg\" (UID: \"a2602d97-293c-4ace-a4e0-974deee70a22\") " pod="openstack-kuttl-tests/run-os-edpm-compute-global-edpm-compute-global-cj7qg" Dec 03 12:57:01 crc kubenswrapper[5052]: I1203 12:57:01.416964 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/a2602d97-293c-4ace-a4e0-974deee70a22-inventory\") pod \"run-os-edpm-compute-global-edpm-compute-global-cj7qg\" (UID: \"a2602d97-293c-4ace-a4e0-974deee70a22\") " pod="openstack-kuttl-tests/run-os-edpm-compute-global-edpm-compute-global-cj7qg" Dec 03 12:57:01 crc kubenswrapper[5052]: I1203 12:57:01.420717 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/a2602d97-293c-4ace-a4e0-974deee70a22-ssh-key\") pod \"run-os-edpm-compute-global-edpm-compute-global-cj7qg\" (UID: \"a2602d97-293c-4ace-a4e0-974deee70a22\") " pod="openstack-kuttl-tests/run-os-edpm-compute-global-edpm-compute-global-cj7qg" Dec 03 12:57:01 crc kubenswrapper[5052]: I1203 12:57:01.436157 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-jbqln\" (UniqueName: \"kubernetes.io/projected/a2602d97-293c-4ace-a4e0-974deee70a22-kube-api-access-jbqln\") pod \"run-os-edpm-compute-global-edpm-compute-global-cj7qg\" (UID: \"a2602d97-293c-4ace-a4e0-974deee70a22\") " pod="openstack-kuttl-tests/run-os-edpm-compute-global-edpm-compute-global-cj7qg" Dec 03 12:57:01 crc kubenswrapper[5052]: I1203 12:57:01.549042 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/run-os-edpm-compute-global-edpm-compute-global-cj7qg" Dec 03 12:57:02 crc kubenswrapper[5052]: I1203 12:57:02.002526 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/run-os-edpm-compute-global-edpm-compute-global-cj7qg"] Dec 03 12:57:02 crc kubenswrapper[5052]: I1203 12:57:02.146401 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/run-os-edpm-compute-global-edpm-compute-global-cj7qg" event={"ID":"a2602d97-293c-4ace-a4e0-974deee70a22","Type":"ContainerStarted","Data":"2e52675f2ef11bc1f75c6c00be40239dfcce67ba17d3b3ced89252e01ee06cca"} Dec 03 12:57:03 crc kubenswrapper[5052]: I1203 12:57:03.157791 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/run-os-edpm-compute-global-edpm-compute-global-cj7qg" event={"ID":"a2602d97-293c-4ace-a4e0-974deee70a22","Type":"ContainerStarted","Data":"4cf61e454547ee7071ac8e117abaf877c9e8f3c66e33c632e689ad67fc0440df"} Dec 03 12:57:03 crc kubenswrapper[5052]: I1203 12:57:03.173298 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/run-os-edpm-compute-global-edpm-compute-global-cj7qg" podStartSLOduration=1.394294785 podStartE2EDuration="2.173280213s" podCreationTimestamp="2025-12-03 12:57:01 +0000 UTC" firstStartedPulling="2025-12-03 12:57:02.004166748 +0000 UTC m=+5812.592408512" lastFinishedPulling="2025-12-03 12:57:02.783152176 +0000 UTC m=+5813.371393940" observedRunningTime="2025-12-03 12:57:03.172029367 +0000 UTC m=+5813.760271131" watchObservedRunningTime="2025-12-03 12:57:03.173280213 +0000 UTC m=+5813.761521977" Dec 03 12:57:04 crc kubenswrapper[5052]: I1203 12:57:04.168684 5052 generic.go:334] "Generic (PLEG): container finished" podID="a2602d97-293c-4ace-a4e0-974deee70a22" containerID="4cf61e454547ee7071ac8e117abaf877c9e8f3c66e33c632e689ad67fc0440df" exitCode=0 Dec 03 12:57:04 crc kubenswrapper[5052]: I1203 12:57:04.168742 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/run-os-edpm-compute-global-edpm-compute-global-cj7qg" event={"ID":"a2602d97-293c-4ace-a4e0-974deee70a22","Type":"ContainerDied","Data":"4cf61e454547ee7071ac8e117abaf877c9e8f3c66e33c632e689ad67fc0440df"} Dec 03 12:57:05 crc kubenswrapper[5052]: I1203 12:57:05.502279 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/run-os-edpm-compute-global-edpm-compute-global-cj7qg" Dec 03 12:57:05 crc kubenswrapper[5052]: I1203 12:57:05.670060 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jbqln\" (UniqueName: \"kubernetes.io/projected/a2602d97-293c-4ace-a4e0-974deee70a22-kube-api-access-jbqln\") pod \"a2602d97-293c-4ace-a4e0-974deee70a22\" (UID: \"a2602d97-293c-4ace-a4e0-974deee70a22\") " Dec 03 12:57:05 crc kubenswrapper[5052]: I1203 12:57:05.670404 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/a2602d97-293c-4ace-a4e0-974deee70a22-ssh-key\") pod \"a2602d97-293c-4ace-a4e0-974deee70a22\" (UID: \"a2602d97-293c-4ace-a4e0-974deee70a22\") " Dec 03 12:57:05 crc kubenswrapper[5052]: I1203 12:57:05.670614 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/a2602d97-293c-4ace-a4e0-974deee70a22-inventory\") pod \"a2602d97-293c-4ace-a4e0-974deee70a22\" (UID: \"a2602d97-293c-4ace-a4e0-974deee70a22\") " Dec 03 12:57:05 crc kubenswrapper[5052]: I1203 12:57:05.675814 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a2602d97-293c-4ace-a4e0-974deee70a22-kube-api-access-jbqln" (OuterVolumeSpecName: "kube-api-access-jbqln") pod "a2602d97-293c-4ace-a4e0-974deee70a22" (UID: "a2602d97-293c-4ace-a4e0-974deee70a22"). InnerVolumeSpecName "kube-api-access-jbqln". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:57:05 crc kubenswrapper[5052]: I1203 12:57:05.692192 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a2602d97-293c-4ace-a4e0-974deee70a22-inventory" (OuterVolumeSpecName: "inventory") pod "a2602d97-293c-4ace-a4e0-974deee70a22" (UID: "a2602d97-293c-4ace-a4e0-974deee70a22"). InnerVolumeSpecName "inventory". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:57:05 crc kubenswrapper[5052]: I1203 12:57:05.692250 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a2602d97-293c-4ace-a4e0-974deee70a22-ssh-key" (OuterVolumeSpecName: "ssh-key") pod "a2602d97-293c-4ace-a4e0-974deee70a22" (UID: "a2602d97-293c-4ace-a4e0-974deee70a22"). InnerVolumeSpecName "ssh-key". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:57:05 crc kubenswrapper[5052]: I1203 12:57:05.773044 5052 reconciler_common.go:293] "Volume detached for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/a2602d97-293c-4ace-a4e0-974deee70a22-inventory\") on node \"crc\" DevicePath \"\"" Dec 03 12:57:05 crc kubenswrapper[5052]: I1203 12:57:05.773098 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-jbqln\" (UniqueName: \"kubernetes.io/projected/a2602d97-293c-4ace-a4e0-974deee70a22-kube-api-access-jbqln\") on node \"crc\" DevicePath \"\"" Dec 03 12:57:05 crc kubenswrapper[5052]: I1203 12:57:05.773128 5052 reconciler_common.go:293] "Volume detached for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/a2602d97-293c-4ace-a4e0-974deee70a22-ssh-key\") on node \"crc\" DevicePath \"\"" Dec 03 12:57:06 crc kubenswrapper[5052]: I1203 12:57:06.189104 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/run-os-edpm-compute-global-edpm-compute-global-cj7qg" event={"ID":"a2602d97-293c-4ace-a4e0-974deee70a22","Type":"ContainerDied","Data":"2e52675f2ef11bc1f75c6c00be40239dfcce67ba17d3b3ced89252e01ee06cca"} Dec 03 12:57:06 crc kubenswrapper[5052]: I1203 12:57:06.189140 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2e52675f2ef11bc1f75c6c00be40239dfcce67ba17d3b3ced89252e01ee06cca" Dec 03 12:57:06 crc kubenswrapper[5052]: I1203 12:57:06.189158 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/run-os-edpm-compute-global-edpm-compute-global-cj7qg" Dec 03 12:57:06 crc kubenswrapper[5052]: I1203 12:57:06.259276 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/install-certs-edpm-compute-global-edpm-compute-global-l68zr"] Dec 03 12:57:06 crc kubenswrapper[5052]: E1203 12:57:06.260123 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="a2602d97-293c-4ace-a4e0-974deee70a22" containerName="run-os-edpm-compute-global-edpm-compute-global" Dec 03 12:57:06 crc kubenswrapper[5052]: I1203 12:57:06.260143 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="a2602d97-293c-4ace-a4e0-974deee70a22" containerName="run-os-edpm-compute-global-edpm-compute-global" Dec 03 12:57:06 crc kubenswrapper[5052]: I1203 12:57:06.260304 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="a2602d97-293c-4ace-a4e0-974deee70a22" containerName="run-os-edpm-compute-global-edpm-compute-global" Dec 03 12:57:06 crc kubenswrapper[5052]: I1203 12:57:06.260783 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/install-certs-edpm-compute-global-edpm-compute-global-l68zr" Dec 03 12:57:06 crc kubenswrapper[5052]: I1203 12:57:06.262931 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"edpm-compute-global-dockercfg-p5rjx" Dec 03 12:57:06 crc kubenswrapper[5052]: I1203 12:57:06.262942 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"dataplane-ansible-ssh-private-key-secret" Dec 03 12:57:06 crc kubenswrapper[5052]: I1203 12:57:06.266447 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"dataplanenodeset-edpm-compute-global" Dec 03 12:57:06 crc kubenswrapper[5052]: I1203 12:57:06.266702 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"combined-ca-bundle" Dec 03 12:57:06 crc kubenswrapper[5052]: I1203 12:57:06.266878 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"openstack-aee-default-env" Dec 03 12:57:06 crc kubenswrapper[5052]: I1203 12:57:06.273058 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/install-certs-edpm-compute-global-edpm-compute-global-l68zr"] Dec 03 12:57:06 crc kubenswrapper[5052]: I1203 12:57:06.381744 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"neutron-metadata-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e3d7c09b-d632-44e9-b7b4-26c841d13f7f-neutron-metadata-combined-ca-bundle\") pod \"install-certs-edpm-compute-global-edpm-compute-global-l68zr\" (UID: \"e3d7c09b-d632-44e9-b7b4-26c841d13f7f\") " pod="openstack-kuttl-tests/install-certs-edpm-compute-global-edpm-compute-global-l68zr" Dec 03 12:57:06 crc kubenswrapper[5052]: I1203 12:57:06.381809 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/e3d7c09b-d632-44e9-b7b4-26c841d13f7f-ssh-key\") pod \"install-certs-edpm-compute-global-edpm-compute-global-l68zr\" (UID: \"e3d7c09b-d632-44e9-b7b4-26c841d13f7f\") " pod="openstack-kuttl-tests/install-certs-edpm-compute-global-edpm-compute-global-l68zr" Dec 03 12:57:06 crc kubenswrapper[5052]: I1203 12:57:06.381862 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nova-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e3d7c09b-d632-44e9-b7b4-26c841d13f7f-nova-combined-ca-bundle\") pod \"install-certs-edpm-compute-global-edpm-compute-global-l68zr\" (UID: \"e3d7c09b-d632-44e9-b7b4-26c841d13f7f\") " pod="openstack-kuttl-tests/install-certs-edpm-compute-global-edpm-compute-global-l68zr" Dec 03 12:57:06 crc kubenswrapper[5052]: I1203 12:57:06.381892 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"custom-global-service-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e3d7c09b-d632-44e9-b7b4-26c841d13f7f-custom-global-service-combined-ca-bundle\") pod \"install-certs-edpm-compute-global-edpm-compute-global-l68zr\" (UID: \"e3d7c09b-d632-44e9-b7b4-26c841d13f7f\") " pod="openstack-kuttl-tests/install-certs-edpm-compute-global-edpm-compute-global-l68zr" Dec 03 12:57:06 crc kubenswrapper[5052]: I1203 12:57:06.381916 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"neutron-dhcp-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e3d7c09b-d632-44e9-b7b4-26c841d13f7f-neutron-dhcp-combined-ca-bundle\") pod \"install-certs-edpm-compute-global-edpm-compute-global-l68zr\" (UID: \"e3d7c09b-d632-44e9-b7b4-26c841d13f7f\") " pod="openstack-kuttl-tests/install-certs-edpm-compute-global-edpm-compute-global-l68zr" Dec 03 12:57:06 crc kubenswrapper[5052]: I1203 12:57:06.381954 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bootstrap-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e3d7c09b-d632-44e9-b7b4-26c841d13f7f-bootstrap-combined-ca-bundle\") pod \"install-certs-edpm-compute-global-edpm-compute-global-l68zr\" (UID: \"e3d7c09b-d632-44e9-b7b4-26c841d13f7f\") " pod="openstack-kuttl-tests/install-certs-edpm-compute-global-edpm-compute-global-l68zr" Dec 03 12:57:06 crc kubenswrapper[5052]: I1203 12:57:06.381977 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"neutron-sriov-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e3d7c09b-d632-44e9-b7b4-26c841d13f7f-neutron-sriov-combined-ca-bundle\") pod \"install-certs-edpm-compute-global-edpm-compute-global-l68zr\" (UID: \"e3d7c09b-d632-44e9-b7b4-26c841d13f7f\") " pod="openstack-kuttl-tests/install-certs-edpm-compute-global-edpm-compute-global-l68zr" Dec 03 12:57:06 crc kubenswrapper[5052]: I1203 12:57:06.381992 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/e3d7c09b-d632-44e9-b7b4-26c841d13f7f-inventory\") pod \"install-certs-edpm-compute-global-edpm-compute-global-l68zr\" (UID: \"e3d7c09b-d632-44e9-b7b4-26c841d13f7f\") " pod="openstack-kuttl-tests/install-certs-edpm-compute-global-edpm-compute-global-l68zr" Dec 03 12:57:06 crc kubenswrapper[5052]: I1203 12:57:06.382031 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e3d7c09b-d632-44e9-b7b4-26c841d13f7f-ovn-combined-ca-bundle\") pod \"install-certs-edpm-compute-global-edpm-compute-global-l68zr\" (UID: \"e3d7c09b-d632-44e9-b7b4-26c841d13f7f\") " pod="openstack-kuttl-tests/install-certs-edpm-compute-global-edpm-compute-global-l68zr" Dec 03 12:57:06 crc kubenswrapper[5052]: I1203 12:57:06.382068 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"libvirt-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e3d7c09b-d632-44e9-b7b4-26c841d13f7f-libvirt-combined-ca-bundle\") pod \"install-certs-edpm-compute-global-edpm-compute-global-l68zr\" (UID: \"e3d7c09b-d632-44e9-b7b4-26c841d13f7f\") " pod="openstack-kuttl-tests/install-certs-edpm-compute-global-edpm-compute-global-l68zr" Dec 03 12:57:06 crc kubenswrapper[5052]: I1203 12:57:06.382128 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"neutron-ovn-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e3d7c09b-d632-44e9-b7b4-26c841d13f7f-neutron-ovn-combined-ca-bundle\") pod \"install-certs-edpm-compute-global-edpm-compute-global-l68zr\" (UID: \"e3d7c09b-d632-44e9-b7b4-26c841d13f7f\") " pod="openstack-kuttl-tests/install-certs-edpm-compute-global-edpm-compute-global-l68zr" Dec 03 12:57:06 crc kubenswrapper[5052]: I1203 12:57:06.382150 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bqsbn\" (UniqueName: \"kubernetes.io/projected/e3d7c09b-d632-44e9-b7b4-26c841d13f7f-kube-api-access-bqsbn\") pod \"install-certs-edpm-compute-global-edpm-compute-global-l68zr\" (UID: \"e3d7c09b-d632-44e9-b7b4-26c841d13f7f\") " pod="openstack-kuttl-tests/install-certs-edpm-compute-global-edpm-compute-global-l68zr" Dec 03 12:57:06 crc kubenswrapper[5052]: I1203 12:57:06.483542 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"neutron-sriov-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e3d7c09b-d632-44e9-b7b4-26c841d13f7f-neutron-sriov-combined-ca-bundle\") pod \"install-certs-edpm-compute-global-edpm-compute-global-l68zr\" (UID: \"e3d7c09b-d632-44e9-b7b4-26c841d13f7f\") " pod="openstack-kuttl-tests/install-certs-edpm-compute-global-edpm-compute-global-l68zr" Dec 03 12:57:06 crc kubenswrapper[5052]: I1203 12:57:06.483920 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/e3d7c09b-d632-44e9-b7b4-26c841d13f7f-inventory\") pod \"install-certs-edpm-compute-global-edpm-compute-global-l68zr\" (UID: \"e3d7c09b-d632-44e9-b7b4-26c841d13f7f\") " pod="openstack-kuttl-tests/install-certs-edpm-compute-global-edpm-compute-global-l68zr" Dec 03 12:57:06 crc kubenswrapper[5052]: I1203 12:57:06.484030 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovn-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e3d7c09b-d632-44e9-b7b4-26c841d13f7f-ovn-combined-ca-bundle\") pod \"install-certs-edpm-compute-global-edpm-compute-global-l68zr\" (UID: \"e3d7c09b-d632-44e9-b7b4-26c841d13f7f\") " pod="openstack-kuttl-tests/install-certs-edpm-compute-global-edpm-compute-global-l68zr" Dec 03 12:57:06 crc kubenswrapper[5052]: I1203 12:57:06.484170 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"libvirt-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e3d7c09b-d632-44e9-b7b4-26c841d13f7f-libvirt-combined-ca-bundle\") pod \"install-certs-edpm-compute-global-edpm-compute-global-l68zr\" (UID: \"e3d7c09b-d632-44e9-b7b4-26c841d13f7f\") " pod="openstack-kuttl-tests/install-certs-edpm-compute-global-edpm-compute-global-l68zr" Dec 03 12:57:06 crc kubenswrapper[5052]: I1203 12:57:06.484278 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"neutron-ovn-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e3d7c09b-d632-44e9-b7b4-26c841d13f7f-neutron-ovn-combined-ca-bundle\") pod \"install-certs-edpm-compute-global-edpm-compute-global-l68zr\" (UID: \"e3d7c09b-d632-44e9-b7b4-26c841d13f7f\") " pod="openstack-kuttl-tests/install-certs-edpm-compute-global-edpm-compute-global-l68zr" Dec 03 12:57:06 crc kubenswrapper[5052]: I1203 12:57:06.484353 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-bqsbn\" (UniqueName: \"kubernetes.io/projected/e3d7c09b-d632-44e9-b7b4-26c841d13f7f-kube-api-access-bqsbn\") pod \"install-certs-edpm-compute-global-edpm-compute-global-l68zr\" (UID: \"e3d7c09b-d632-44e9-b7b4-26c841d13f7f\") " pod="openstack-kuttl-tests/install-certs-edpm-compute-global-edpm-compute-global-l68zr" Dec 03 12:57:06 crc kubenswrapper[5052]: I1203 12:57:06.484460 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"neutron-metadata-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e3d7c09b-d632-44e9-b7b4-26c841d13f7f-neutron-metadata-combined-ca-bundle\") pod \"install-certs-edpm-compute-global-edpm-compute-global-l68zr\" (UID: \"e3d7c09b-d632-44e9-b7b4-26c841d13f7f\") " pod="openstack-kuttl-tests/install-certs-edpm-compute-global-edpm-compute-global-l68zr" Dec 03 12:57:06 crc kubenswrapper[5052]: I1203 12:57:06.484615 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/e3d7c09b-d632-44e9-b7b4-26c841d13f7f-ssh-key\") pod \"install-certs-edpm-compute-global-edpm-compute-global-l68zr\" (UID: \"e3d7c09b-d632-44e9-b7b4-26c841d13f7f\") " pod="openstack-kuttl-tests/install-certs-edpm-compute-global-edpm-compute-global-l68zr" Dec 03 12:57:06 crc kubenswrapper[5052]: I1203 12:57:06.484766 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"nova-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e3d7c09b-d632-44e9-b7b4-26c841d13f7f-nova-combined-ca-bundle\") pod \"install-certs-edpm-compute-global-edpm-compute-global-l68zr\" (UID: \"e3d7c09b-d632-44e9-b7b4-26c841d13f7f\") " pod="openstack-kuttl-tests/install-certs-edpm-compute-global-edpm-compute-global-l68zr" Dec 03 12:57:06 crc kubenswrapper[5052]: I1203 12:57:06.484893 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"custom-global-service-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e3d7c09b-d632-44e9-b7b4-26c841d13f7f-custom-global-service-combined-ca-bundle\") pod \"install-certs-edpm-compute-global-edpm-compute-global-l68zr\" (UID: \"e3d7c09b-d632-44e9-b7b4-26c841d13f7f\") " pod="openstack-kuttl-tests/install-certs-edpm-compute-global-edpm-compute-global-l68zr" Dec 03 12:57:06 crc kubenswrapper[5052]: I1203 12:57:06.485022 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"neutron-dhcp-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e3d7c09b-d632-44e9-b7b4-26c841d13f7f-neutron-dhcp-combined-ca-bundle\") pod \"install-certs-edpm-compute-global-edpm-compute-global-l68zr\" (UID: \"e3d7c09b-d632-44e9-b7b4-26c841d13f7f\") " pod="openstack-kuttl-tests/install-certs-edpm-compute-global-edpm-compute-global-l68zr" Dec 03 12:57:06 crc kubenswrapper[5052]: I1203 12:57:06.485143 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"bootstrap-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e3d7c09b-d632-44e9-b7b4-26c841d13f7f-bootstrap-combined-ca-bundle\") pod \"install-certs-edpm-compute-global-edpm-compute-global-l68zr\" (UID: \"e3d7c09b-d632-44e9-b7b4-26c841d13f7f\") " pod="openstack-kuttl-tests/install-certs-edpm-compute-global-edpm-compute-global-l68zr" Dec 03 12:57:06 crc kubenswrapper[5052]: I1203 12:57:06.487118 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"neutron-sriov-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e3d7c09b-d632-44e9-b7b4-26c841d13f7f-neutron-sriov-combined-ca-bundle\") pod \"install-certs-edpm-compute-global-edpm-compute-global-l68zr\" (UID: \"e3d7c09b-d632-44e9-b7b4-26c841d13f7f\") " pod="openstack-kuttl-tests/install-certs-edpm-compute-global-edpm-compute-global-l68zr" Dec 03 12:57:06 crc kubenswrapper[5052]: I1203 12:57:06.487822 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"neutron-metadata-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e3d7c09b-d632-44e9-b7b4-26c841d13f7f-neutron-metadata-combined-ca-bundle\") pod \"install-certs-edpm-compute-global-edpm-compute-global-l68zr\" (UID: \"e3d7c09b-d632-44e9-b7b4-26c841d13f7f\") " pod="openstack-kuttl-tests/install-certs-edpm-compute-global-edpm-compute-global-l68zr" Dec 03 12:57:06 crc kubenswrapper[5052]: I1203 12:57:06.487871 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovn-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e3d7c09b-d632-44e9-b7b4-26c841d13f7f-ovn-combined-ca-bundle\") pod \"install-certs-edpm-compute-global-edpm-compute-global-l68zr\" (UID: \"e3d7c09b-d632-44e9-b7b4-26c841d13f7f\") " pod="openstack-kuttl-tests/install-certs-edpm-compute-global-edpm-compute-global-l68zr" Dec 03 12:57:06 crc kubenswrapper[5052]: I1203 12:57:06.487922 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"neutron-dhcp-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e3d7c09b-d632-44e9-b7b4-26c841d13f7f-neutron-dhcp-combined-ca-bundle\") pod \"install-certs-edpm-compute-global-edpm-compute-global-l68zr\" (UID: \"e3d7c09b-d632-44e9-b7b4-26c841d13f7f\") " pod="openstack-kuttl-tests/install-certs-edpm-compute-global-edpm-compute-global-l68zr" Dec 03 12:57:06 crc kubenswrapper[5052]: I1203 12:57:06.488112 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"nova-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e3d7c09b-d632-44e9-b7b4-26c841d13f7f-nova-combined-ca-bundle\") pod \"install-certs-edpm-compute-global-edpm-compute-global-l68zr\" (UID: \"e3d7c09b-d632-44e9-b7b4-26c841d13f7f\") " pod="openstack-kuttl-tests/install-certs-edpm-compute-global-edpm-compute-global-l68zr" Dec 03 12:57:06 crc kubenswrapper[5052]: I1203 12:57:06.488873 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/e3d7c09b-d632-44e9-b7b4-26c841d13f7f-inventory\") pod \"install-certs-edpm-compute-global-edpm-compute-global-l68zr\" (UID: \"e3d7c09b-d632-44e9-b7b4-26c841d13f7f\") " pod="openstack-kuttl-tests/install-certs-edpm-compute-global-edpm-compute-global-l68zr" Dec 03 12:57:06 crc kubenswrapper[5052]: I1203 12:57:06.489382 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"bootstrap-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e3d7c09b-d632-44e9-b7b4-26c841d13f7f-bootstrap-combined-ca-bundle\") pod \"install-certs-edpm-compute-global-edpm-compute-global-l68zr\" (UID: \"e3d7c09b-d632-44e9-b7b4-26c841d13f7f\") " pod="openstack-kuttl-tests/install-certs-edpm-compute-global-edpm-compute-global-l68zr" Dec 03 12:57:06 crc kubenswrapper[5052]: I1203 12:57:06.489501 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"libvirt-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e3d7c09b-d632-44e9-b7b4-26c841d13f7f-libvirt-combined-ca-bundle\") pod \"install-certs-edpm-compute-global-edpm-compute-global-l68zr\" (UID: \"e3d7c09b-d632-44e9-b7b4-26c841d13f7f\") " pod="openstack-kuttl-tests/install-certs-edpm-compute-global-edpm-compute-global-l68zr" Dec 03 12:57:06 crc kubenswrapper[5052]: I1203 12:57:06.490431 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/e3d7c09b-d632-44e9-b7b4-26c841d13f7f-ssh-key\") pod \"install-certs-edpm-compute-global-edpm-compute-global-l68zr\" (UID: \"e3d7c09b-d632-44e9-b7b4-26c841d13f7f\") " pod="openstack-kuttl-tests/install-certs-edpm-compute-global-edpm-compute-global-l68zr" Dec 03 12:57:06 crc kubenswrapper[5052]: I1203 12:57:06.490498 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"custom-global-service-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e3d7c09b-d632-44e9-b7b4-26c841d13f7f-custom-global-service-combined-ca-bundle\") pod \"install-certs-edpm-compute-global-edpm-compute-global-l68zr\" (UID: \"e3d7c09b-d632-44e9-b7b4-26c841d13f7f\") " pod="openstack-kuttl-tests/install-certs-edpm-compute-global-edpm-compute-global-l68zr" Dec 03 12:57:06 crc kubenswrapper[5052]: I1203 12:57:06.493825 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"neutron-ovn-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e3d7c09b-d632-44e9-b7b4-26c841d13f7f-neutron-ovn-combined-ca-bundle\") pod \"install-certs-edpm-compute-global-edpm-compute-global-l68zr\" (UID: \"e3d7c09b-d632-44e9-b7b4-26c841d13f7f\") " pod="openstack-kuttl-tests/install-certs-edpm-compute-global-edpm-compute-global-l68zr" Dec 03 12:57:06 crc kubenswrapper[5052]: I1203 12:57:06.501749 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-bqsbn\" (UniqueName: \"kubernetes.io/projected/e3d7c09b-d632-44e9-b7b4-26c841d13f7f-kube-api-access-bqsbn\") pod \"install-certs-edpm-compute-global-edpm-compute-global-l68zr\" (UID: \"e3d7c09b-d632-44e9-b7b4-26c841d13f7f\") " pod="openstack-kuttl-tests/install-certs-edpm-compute-global-edpm-compute-global-l68zr" Dec 03 12:57:06 crc kubenswrapper[5052]: I1203 12:57:06.576294 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/install-certs-edpm-compute-global-edpm-compute-global-l68zr" Dec 03 12:57:06 crc kubenswrapper[5052]: I1203 12:57:06.979242 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/install-certs-edpm-compute-global-edpm-compute-global-l68zr"] Dec 03 12:57:07 crc kubenswrapper[5052]: I1203 12:57:07.198355 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/install-certs-edpm-compute-global-edpm-compute-global-l68zr" event={"ID":"e3d7c09b-d632-44e9-b7b4-26c841d13f7f","Type":"ContainerStarted","Data":"c8615f02bbc64f3d6aaeac051179f6a88e695939798f59e86eb29a13b0d69a6d"} Dec 03 12:57:07 crc kubenswrapper[5052]: I1203 12:57:07.673003 5052 scope.go:117] "RemoveContainer" containerID="69fa7fbcd2aceb7c53703a19508876f93a97dc9adabb79c16fb691d90f606d58" Dec 03 12:57:07 crc kubenswrapper[5052]: E1203 12:57:07.674610 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 12:57:08 crc kubenswrapper[5052]: I1203 12:57:08.211733 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/install-certs-edpm-compute-global-edpm-compute-global-l68zr" event={"ID":"e3d7c09b-d632-44e9-b7b4-26c841d13f7f","Type":"ContainerStarted","Data":"fe2b87c907b25b19cfafc26126c48aaef2435a532f4d6cb5c96faebd6a8c77cb"} Dec 03 12:57:08 crc kubenswrapper[5052]: I1203 12:57:08.243661 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/install-certs-edpm-compute-global-edpm-compute-global-l68zr" podStartSLOduration=1.754995782 podStartE2EDuration="2.243628056s" podCreationTimestamp="2025-12-03 12:57:06 +0000 UTC" firstStartedPulling="2025-12-03 12:57:06.986323239 +0000 UTC m=+5817.574564993" lastFinishedPulling="2025-12-03 12:57:07.474955493 +0000 UTC m=+5818.063197267" observedRunningTime="2025-12-03 12:57:08.240591009 +0000 UTC m=+5818.828832813" watchObservedRunningTime="2025-12-03 12:57:08.243628056 +0000 UTC m=+5818.831869860" Dec 03 12:57:09 crc kubenswrapper[5052]: I1203 12:57:09.223701 5052 generic.go:334] "Generic (PLEG): container finished" podID="e3d7c09b-d632-44e9-b7b4-26c841d13f7f" containerID="fe2b87c907b25b19cfafc26126c48aaef2435a532f4d6cb5c96faebd6a8c77cb" exitCode=0 Dec 03 12:57:09 crc kubenswrapper[5052]: I1203 12:57:09.223782 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/install-certs-edpm-compute-global-edpm-compute-global-l68zr" event={"ID":"e3d7c09b-d632-44e9-b7b4-26c841d13f7f","Type":"ContainerDied","Data":"fe2b87c907b25b19cfafc26126c48aaef2435a532f4d6cb5c96faebd6a8c77cb"} Dec 03 12:57:10 crc kubenswrapper[5052]: I1203 12:57:10.515061 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/install-certs-edpm-compute-global-edpm-compute-global-l68zr" Dec 03 12:57:10 crc kubenswrapper[5052]: I1203 12:57:10.650015 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"custom-global-service-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e3d7c09b-d632-44e9-b7b4-26c841d13f7f-custom-global-service-combined-ca-bundle\") pod \"e3d7c09b-d632-44e9-b7b4-26c841d13f7f\" (UID: \"e3d7c09b-d632-44e9-b7b4-26c841d13f7f\") " Dec 03 12:57:10 crc kubenswrapper[5052]: I1203 12:57:10.650107 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"nova-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e3d7c09b-d632-44e9-b7b4-26c841d13f7f-nova-combined-ca-bundle\") pod \"e3d7c09b-d632-44e9-b7b4-26c841d13f7f\" (UID: \"e3d7c09b-d632-44e9-b7b4-26c841d13f7f\") " Dec 03 12:57:10 crc kubenswrapper[5052]: I1203 12:57:10.650141 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovn-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e3d7c09b-d632-44e9-b7b4-26c841d13f7f-ovn-combined-ca-bundle\") pod \"e3d7c09b-d632-44e9-b7b4-26c841d13f7f\" (UID: \"e3d7c09b-d632-44e9-b7b4-26c841d13f7f\") " Dec 03 12:57:10 crc kubenswrapper[5052]: I1203 12:57:10.650167 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/e3d7c09b-d632-44e9-b7b4-26c841d13f7f-ssh-key\") pod \"e3d7c09b-d632-44e9-b7b4-26c841d13f7f\" (UID: \"e3d7c09b-d632-44e9-b7b4-26c841d13f7f\") " Dec 03 12:57:10 crc kubenswrapper[5052]: I1203 12:57:10.650667 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"neutron-metadata-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e3d7c09b-d632-44e9-b7b4-26c841d13f7f-neutron-metadata-combined-ca-bundle\") pod \"e3d7c09b-d632-44e9-b7b4-26c841d13f7f\" (UID: \"e3d7c09b-d632-44e9-b7b4-26c841d13f7f\") " Dec 03 12:57:10 crc kubenswrapper[5052]: I1203 12:57:10.650726 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"neutron-dhcp-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e3d7c09b-d632-44e9-b7b4-26c841d13f7f-neutron-dhcp-combined-ca-bundle\") pod \"e3d7c09b-d632-44e9-b7b4-26c841d13f7f\" (UID: \"e3d7c09b-d632-44e9-b7b4-26c841d13f7f\") " Dec 03 12:57:10 crc kubenswrapper[5052]: I1203 12:57:10.650759 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"neutron-sriov-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e3d7c09b-d632-44e9-b7b4-26c841d13f7f-neutron-sriov-combined-ca-bundle\") pod \"e3d7c09b-d632-44e9-b7b4-26c841d13f7f\" (UID: \"e3d7c09b-d632-44e9-b7b4-26c841d13f7f\") " Dec 03 12:57:10 crc kubenswrapper[5052]: I1203 12:57:10.650795 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"libvirt-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e3d7c09b-d632-44e9-b7b4-26c841d13f7f-libvirt-combined-ca-bundle\") pod \"e3d7c09b-d632-44e9-b7b4-26c841d13f7f\" (UID: \"e3d7c09b-d632-44e9-b7b4-26c841d13f7f\") " Dec 03 12:57:10 crc kubenswrapper[5052]: I1203 12:57:10.650820 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"bootstrap-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e3d7c09b-d632-44e9-b7b4-26c841d13f7f-bootstrap-combined-ca-bundle\") pod \"e3d7c09b-d632-44e9-b7b4-26c841d13f7f\" (UID: \"e3d7c09b-d632-44e9-b7b4-26c841d13f7f\") " Dec 03 12:57:10 crc kubenswrapper[5052]: I1203 12:57:10.650853 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bqsbn\" (UniqueName: \"kubernetes.io/projected/e3d7c09b-d632-44e9-b7b4-26c841d13f7f-kube-api-access-bqsbn\") pod \"e3d7c09b-d632-44e9-b7b4-26c841d13f7f\" (UID: \"e3d7c09b-d632-44e9-b7b4-26c841d13f7f\") " Dec 03 12:57:10 crc kubenswrapper[5052]: I1203 12:57:10.650908 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"neutron-ovn-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e3d7c09b-d632-44e9-b7b4-26c841d13f7f-neutron-ovn-combined-ca-bundle\") pod \"e3d7c09b-d632-44e9-b7b4-26c841d13f7f\" (UID: \"e3d7c09b-d632-44e9-b7b4-26c841d13f7f\") " Dec 03 12:57:10 crc kubenswrapper[5052]: I1203 12:57:10.650941 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/e3d7c09b-d632-44e9-b7b4-26c841d13f7f-inventory\") pod \"e3d7c09b-d632-44e9-b7b4-26c841d13f7f\" (UID: \"e3d7c09b-d632-44e9-b7b4-26c841d13f7f\") " Dec 03 12:57:10 crc kubenswrapper[5052]: I1203 12:57:10.656311 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e3d7c09b-d632-44e9-b7b4-26c841d13f7f-libvirt-combined-ca-bundle" (OuterVolumeSpecName: "libvirt-combined-ca-bundle") pod "e3d7c09b-d632-44e9-b7b4-26c841d13f7f" (UID: "e3d7c09b-d632-44e9-b7b4-26c841d13f7f"). InnerVolumeSpecName "libvirt-combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:57:10 crc kubenswrapper[5052]: I1203 12:57:10.656669 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e3d7c09b-d632-44e9-b7b4-26c841d13f7f-kube-api-access-bqsbn" (OuterVolumeSpecName: "kube-api-access-bqsbn") pod "e3d7c09b-d632-44e9-b7b4-26c841d13f7f" (UID: "e3d7c09b-d632-44e9-b7b4-26c841d13f7f"). InnerVolumeSpecName "kube-api-access-bqsbn". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:57:10 crc kubenswrapper[5052]: I1203 12:57:10.656837 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e3d7c09b-d632-44e9-b7b4-26c841d13f7f-custom-global-service-combined-ca-bundle" (OuterVolumeSpecName: "custom-global-service-combined-ca-bundle") pod "e3d7c09b-d632-44e9-b7b4-26c841d13f7f" (UID: "e3d7c09b-d632-44e9-b7b4-26c841d13f7f"). InnerVolumeSpecName "custom-global-service-combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:57:10 crc kubenswrapper[5052]: I1203 12:57:10.656917 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e3d7c09b-d632-44e9-b7b4-26c841d13f7f-neutron-dhcp-combined-ca-bundle" (OuterVolumeSpecName: "neutron-dhcp-combined-ca-bundle") pod "e3d7c09b-d632-44e9-b7b4-26c841d13f7f" (UID: "e3d7c09b-d632-44e9-b7b4-26c841d13f7f"). InnerVolumeSpecName "neutron-dhcp-combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:57:10 crc kubenswrapper[5052]: I1203 12:57:10.656974 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e3d7c09b-d632-44e9-b7b4-26c841d13f7f-neutron-metadata-combined-ca-bundle" (OuterVolumeSpecName: "neutron-metadata-combined-ca-bundle") pod "e3d7c09b-d632-44e9-b7b4-26c841d13f7f" (UID: "e3d7c09b-d632-44e9-b7b4-26c841d13f7f"). InnerVolumeSpecName "neutron-metadata-combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:57:10 crc kubenswrapper[5052]: I1203 12:57:10.657043 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e3d7c09b-d632-44e9-b7b4-26c841d13f7f-neutron-ovn-combined-ca-bundle" (OuterVolumeSpecName: "neutron-ovn-combined-ca-bundle") pod "e3d7c09b-d632-44e9-b7b4-26c841d13f7f" (UID: "e3d7c09b-d632-44e9-b7b4-26c841d13f7f"). InnerVolumeSpecName "neutron-ovn-combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:57:10 crc kubenswrapper[5052]: I1203 12:57:10.657181 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e3d7c09b-d632-44e9-b7b4-26c841d13f7f-bootstrap-combined-ca-bundle" (OuterVolumeSpecName: "bootstrap-combined-ca-bundle") pod "e3d7c09b-d632-44e9-b7b4-26c841d13f7f" (UID: "e3d7c09b-d632-44e9-b7b4-26c841d13f7f"). InnerVolumeSpecName "bootstrap-combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:57:10 crc kubenswrapper[5052]: I1203 12:57:10.657226 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e3d7c09b-d632-44e9-b7b4-26c841d13f7f-neutron-sriov-combined-ca-bundle" (OuterVolumeSpecName: "neutron-sriov-combined-ca-bundle") pod "e3d7c09b-d632-44e9-b7b4-26c841d13f7f" (UID: "e3d7c09b-d632-44e9-b7b4-26c841d13f7f"). InnerVolumeSpecName "neutron-sriov-combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:57:10 crc kubenswrapper[5052]: I1203 12:57:10.657468 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e3d7c09b-d632-44e9-b7b4-26c841d13f7f-nova-combined-ca-bundle" (OuterVolumeSpecName: "nova-combined-ca-bundle") pod "e3d7c09b-d632-44e9-b7b4-26c841d13f7f" (UID: "e3d7c09b-d632-44e9-b7b4-26c841d13f7f"). InnerVolumeSpecName "nova-combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:57:10 crc kubenswrapper[5052]: I1203 12:57:10.661683 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e3d7c09b-d632-44e9-b7b4-26c841d13f7f-ovn-combined-ca-bundle" (OuterVolumeSpecName: "ovn-combined-ca-bundle") pod "e3d7c09b-d632-44e9-b7b4-26c841d13f7f" (UID: "e3d7c09b-d632-44e9-b7b4-26c841d13f7f"). InnerVolumeSpecName "ovn-combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:57:10 crc kubenswrapper[5052]: I1203 12:57:10.674092 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e3d7c09b-d632-44e9-b7b4-26c841d13f7f-inventory" (OuterVolumeSpecName: "inventory") pod "e3d7c09b-d632-44e9-b7b4-26c841d13f7f" (UID: "e3d7c09b-d632-44e9-b7b4-26c841d13f7f"). InnerVolumeSpecName "inventory". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:57:10 crc kubenswrapper[5052]: I1203 12:57:10.674247 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e3d7c09b-d632-44e9-b7b4-26c841d13f7f-ssh-key" (OuterVolumeSpecName: "ssh-key") pod "e3d7c09b-d632-44e9-b7b4-26c841d13f7f" (UID: "e3d7c09b-d632-44e9-b7b4-26c841d13f7f"). InnerVolumeSpecName "ssh-key". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:57:10 crc kubenswrapper[5052]: I1203 12:57:10.753373 5052 reconciler_common.go:293] "Volume detached for volume \"nova-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e3d7c09b-d632-44e9-b7b4-26c841d13f7f-nova-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:57:10 crc kubenswrapper[5052]: I1203 12:57:10.753412 5052 reconciler_common.go:293] "Volume detached for volume \"ovn-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e3d7c09b-d632-44e9-b7b4-26c841d13f7f-ovn-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:57:10 crc kubenswrapper[5052]: I1203 12:57:10.753422 5052 reconciler_common.go:293] "Volume detached for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/e3d7c09b-d632-44e9-b7b4-26c841d13f7f-ssh-key\") on node \"crc\" DevicePath \"\"" Dec 03 12:57:10 crc kubenswrapper[5052]: I1203 12:57:10.753433 5052 reconciler_common.go:293] "Volume detached for volume \"neutron-metadata-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e3d7c09b-d632-44e9-b7b4-26c841d13f7f-neutron-metadata-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:57:10 crc kubenswrapper[5052]: I1203 12:57:10.753442 5052 reconciler_common.go:293] "Volume detached for volume \"neutron-dhcp-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e3d7c09b-d632-44e9-b7b4-26c841d13f7f-neutron-dhcp-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:57:10 crc kubenswrapper[5052]: I1203 12:57:10.753451 5052 reconciler_common.go:293] "Volume detached for volume \"neutron-sriov-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e3d7c09b-d632-44e9-b7b4-26c841d13f7f-neutron-sriov-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:57:10 crc kubenswrapper[5052]: I1203 12:57:10.753465 5052 reconciler_common.go:293] "Volume detached for volume \"libvirt-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e3d7c09b-d632-44e9-b7b4-26c841d13f7f-libvirt-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:57:10 crc kubenswrapper[5052]: I1203 12:57:10.753478 5052 reconciler_common.go:293] "Volume detached for volume \"bootstrap-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e3d7c09b-d632-44e9-b7b4-26c841d13f7f-bootstrap-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:57:10 crc kubenswrapper[5052]: I1203 12:57:10.753494 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-bqsbn\" (UniqueName: \"kubernetes.io/projected/e3d7c09b-d632-44e9-b7b4-26c841d13f7f-kube-api-access-bqsbn\") on node \"crc\" DevicePath \"\"" Dec 03 12:57:10 crc kubenswrapper[5052]: I1203 12:57:10.753506 5052 reconciler_common.go:293] "Volume detached for volume \"neutron-ovn-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e3d7c09b-d632-44e9-b7b4-26c841d13f7f-neutron-ovn-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:57:10 crc kubenswrapper[5052]: I1203 12:57:10.753569 5052 reconciler_common.go:293] "Volume detached for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/e3d7c09b-d632-44e9-b7b4-26c841d13f7f-inventory\") on node \"crc\" DevicePath \"\"" Dec 03 12:57:10 crc kubenswrapper[5052]: I1203 12:57:10.753588 5052 reconciler_common.go:293] "Volume detached for volume \"custom-global-service-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e3d7c09b-d632-44e9-b7b4-26c841d13f7f-custom-global-service-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:57:11 crc kubenswrapper[5052]: I1203 12:57:11.243423 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/install-certs-edpm-compute-global-edpm-compute-global-l68zr" event={"ID":"e3d7c09b-d632-44e9-b7b4-26c841d13f7f","Type":"ContainerDied","Data":"c8615f02bbc64f3d6aaeac051179f6a88e695939798f59e86eb29a13b0d69a6d"} Dec 03 12:57:11 crc kubenswrapper[5052]: I1203 12:57:11.243469 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c8615f02bbc64f3d6aaeac051179f6a88e695939798f59e86eb29a13b0d69a6d" Dec 03 12:57:11 crc kubenswrapper[5052]: I1203 12:57:11.243799 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/install-certs-edpm-compute-global-edpm-compute-global-l68zr" Dec 03 12:57:11 crc kubenswrapper[5052]: I1203 12:57:11.324336 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ovn-edpm-compute-global-edpm-compute-global-dw8gp"] Dec 03 12:57:11 crc kubenswrapper[5052]: E1203 12:57:11.324863 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="e3d7c09b-d632-44e9-b7b4-26c841d13f7f" containerName="install-certs-edpm-compute-global-edpm-compute-global" Dec 03 12:57:11 crc kubenswrapper[5052]: I1203 12:57:11.324891 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="e3d7c09b-d632-44e9-b7b4-26c841d13f7f" containerName="install-certs-edpm-compute-global-edpm-compute-global" Dec 03 12:57:11 crc kubenswrapper[5052]: I1203 12:57:11.325152 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="e3d7c09b-d632-44e9-b7b4-26c841d13f7f" containerName="install-certs-edpm-compute-global-edpm-compute-global" Dec 03 12:57:11 crc kubenswrapper[5052]: I1203 12:57:11.325931 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovn-edpm-compute-global-edpm-compute-global-dw8gp" Dec 03 12:57:11 crc kubenswrapper[5052]: I1203 12:57:11.328507 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"openstack-aee-default-env" Dec 03 12:57:11 crc kubenswrapper[5052]: I1203 12:57:11.328677 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"dataplane-ansible-ssh-private-key-secret" Dec 03 12:57:11 crc kubenswrapper[5052]: I1203 12:57:11.328769 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"ovncontroller-config" Dec 03 12:57:11 crc kubenswrapper[5052]: I1203 12:57:11.329050 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"edpm-compute-global-dockercfg-p5rjx" Dec 03 12:57:11 crc kubenswrapper[5052]: I1203 12:57:11.329851 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"combined-ca-bundle" Dec 03 12:57:11 crc kubenswrapper[5052]: I1203 12:57:11.331468 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ovn-edpm-compute-global-edpm-compute-global-dw8gp"] Dec 03 12:57:11 crc kubenswrapper[5052]: I1203 12:57:11.331968 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"dataplanenodeset-edpm-compute-global" Dec 03 12:57:11 crc kubenswrapper[5052]: I1203 12:57:11.464571 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a3839557-7c97-4814-9c26-48e2c9674a9b-ovn-combined-ca-bundle\") pod \"ovn-edpm-compute-global-edpm-compute-global-dw8gp\" (UID: \"a3839557-7c97-4814-9c26-48e2c9674a9b\") " pod="openstack-kuttl-tests/ovn-edpm-compute-global-edpm-compute-global-dw8gp" Dec 03 12:57:11 crc kubenswrapper[5052]: I1203 12:57:11.464740 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/a3839557-7c97-4814-9c26-48e2c9674a9b-inventory\") pod \"ovn-edpm-compute-global-edpm-compute-global-dw8gp\" (UID: \"a3839557-7c97-4814-9c26-48e2c9674a9b\") " pod="openstack-kuttl-tests/ovn-edpm-compute-global-edpm-compute-global-dw8gp" Dec 03 12:57:11 crc kubenswrapper[5052]: I1203 12:57:11.464846 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/a3839557-7c97-4814-9c26-48e2c9674a9b-ssh-key\") pod \"ovn-edpm-compute-global-edpm-compute-global-dw8gp\" (UID: \"a3839557-7c97-4814-9c26-48e2c9674a9b\") " pod="openstack-kuttl-tests/ovn-edpm-compute-global-edpm-compute-global-dw8gp" Dec 03 12:57:11 crc kubenswrapper[5052]: I1203 12:57:11.464875 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovncontroller-config-0\" (UniqueName: \"kubernetes.io/configmap/a3839557-7c97-4814-9c26-48e2c9674a9b-ovncontroller-config-0\") pod \"ovn-edpm-compute-global-edpm-compute-global-dw8gp\" (UID: \"a3839557-7c97-4814-9c26-48e2c9674a9b\") " pod="openstack-kuttl-tests/ovn-edpm-compute-global-edpm-compute-global-dw8gp" Dec 03 12:57:11 crc kubenswrapper[5052]: I1203 12:57:11.464951 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2mx4b\" (UniqueName: \"kubernetes.io/projected/a3839557-7c97-4814-9c26-48e2c9674a9b-kube-api-access-2mx4b\") pod \"ovn-edpm-compute-global-edpm-compute-global-dw8gp\" (UID: \"a3839557-7c97-4814-9c26-48e2c9674a9b\") " pod="openstack-kuttl-tests/ovn-edpm-compute-global-edpm-compute-global-dw8gp" Dec 03 12:57:11 crc kubenswrapper[5052]: I1203 12:57:11.566192 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovn-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a3839557-7c97-4814-9c26-48e2c9674a9b-ovn-combined-ca-bundle\") pod \"ovn-edpm-compute-global-edpm-compute-global-dw8gp\" (UID: \"a3839557-7c97-4814-9c26-48e2c9674a9b\") " pod="openstack-kuttl-tests/ovn-edpm-compute-global-edpm-compute-global-dw8gp" Dec 03 12:57:11 crc kubenswrapper[5052]: I1203 12:57:11.566331 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/a3839557-7c97-4814-9c26-48e2c9674a9b-inventory\") pod \"ovn-edpm-compute-global-edpm-compute-global-dw8gp\" (UID: \"a3839557-7c97-4814-9c26-48e2c9674a9b\") " pod="openstack-kuttl-tests/ovn-edpm-compute-global-edpm-compute-global-dw8gp" Dec 03 12:57:11 crc kubenswrapper[5052]: I1203 12:57:11.566369 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/a3839557-7c97-4814-9c26-48e2c9674a9b-ssh-key\") pod \"ovn-edpm-compute-global-edpm-compute-global-dw8gp\" (UID: \"a3839557-7c97-4814-9c26-48e2c9674a9b\") " pod="openstack-kuttl-tests/ovn-edpm-compute-global-edpm-compute-global-dw8gp" Dec 03 12:57:11 crc kubenswrapper[5052]: I1203 12:57:11.566404 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovncontroller-config-0\" (UniqueName: \"kubernetes.io/configmap/a3839557-7c97-4814-9c26-48e2c9674a9b-ovncontroller-config-0\") pod \"ovn-edpm-compute-global-edpm-compute-global-dw8gp\" (UID: \"a3839557-7c97-4814-9c26-48e2c9674a9b\") " pod="openstack-kuttl-tests/ovn-edpm-compute-global-edpm-compute-global-dw8gp" Dec 03 12:57:11 crc kubenswrapper[5052]: I1203 12:57:11.566451 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-2mx4b\" (UniqueName: \"kubernetes.io/projected/a3839557-7c97-4814-9c26-48e2c9674a9b-kube-api-access-2mx4b\") pod \"ovn-edpm-compute-global-edpm-compute-global-dw8gp\" (UID: \"a3839557-7c97-4814-9c26-48e2c9674a9b\") " pod="openstack-kuttl-tests/ovn-edpm-compute-global-edpm-compute-global-dw8gp" Dec 03 12:57:11 crc kubenswrapper[5052]: I1203 12:57:11.568476 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovncontroller-config-0\" (UniqueName: \"kubernetes.io/configmap/a3839557-7c97-4814-9c26-48e2c9674a9b-ovncontroller-config-0\") pod \"ovn-edpm-compute-global-edpm-compute-global-dw8gp\" (UID: \"a3839557-7c97-4814-9c26-48e2c9674a9b\") " pod="openstack-kuttl-tests/ovn-edpm-compute-global-edpm-compute-global-dw8gp" Dec 03 12:57:11 crc kubenswrapper[5052]: I1203 12:57:11.569882 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/a3839557-7c97-4814-9c26-48e2c9674a9b-ssh-key\") pod \"ovn-edpm-compute-global-edpm-compute-global-dw8gp\" (UID: \"a3839557-7c97-4814-9c26-48e2c9674a9b\") " pod="openstack-kuttl-tests/ovn-edpm-compute-global-edpm-compute-global-dw8gp" Dec 03 12:57:11 crc kubenswrapper[5052]: I1203 12:57:11.569992 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovn-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a3839557-7c97-4814-9c26-48e2c9674a9b-ovn-combined-ca-bundle\") pod \"ovn-edpm-compute-global-edpm-compute-global-dw8gp\" (UID: \"a3839557-7c97-4814-9c26-48e2c9674a9b\") " pod="openstack-kuttl-tests/ovn-edpm-compute-global-edpm-compute-global-dw8gp" Dec 03 12:57:11 crc kubenswrapper[5052]: I1203 12:57:11.572911 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/a3839557-7c97-4814-9c26-48e2c9674a9b-inventory\") pod \"ovn-edpm-compute-global-edpm-compute-global-dw8gp\" (UID: \"a3839557-7c97-4814-9c26-48e2c9674a9b\") " pod="openstack-kuttl-tests/ovn-edpm-compute-global-edpm-compute-global-dw8gp" Dec 03 12:57:11 crc kubenswrapper[5052]: I1203 12:57:11.588838 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-2mx4b\" (UniqueName: \"kubernetes.io/projected/a3839557-7c97-4814-9c26-48e2c9674a9b-kube-api-access-2mx4b\") pod \"ovn-edpm-compute-global-edpm-compute-global-dw8gp\" (UID: \"a3839557-7c97-4814-9c26-48e2c9674a9b\") " pod="openstack-kuttl-tests/ovn-edpm-compute-global-edpm-compute-global-dw8gp" Dec 03 12:57:11 crc kubenswrapper[5052]: I1203 12:57:11.640499 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovn-edpm-compute-global-edpm-compute-global-dw8gp" Dec 03 12:57:12 crc kubenswrapper[5052]: I1203 12:57:12.060892 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ovn-edpm-compute-global-edpm-compute-global-dw8gp"] Dec 03 12:57:12 crc kubenswrapper[5052]: I1203 12:57:12.252271 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovn-edpm-compute-global-edpm-compute-global-dw8gp" event={"ID":"a3839557-7c97-4814-9c26-48e2c9674a9b","Type":"ContainerStarted","Data":"039ef90283d1b2c10c651af0a2987d9ff5c6cee8e7b190616975fc5135aaf8b5"} Dec 03 12:57:13 crc kubenswrapper[5052]: I1203 12:57:13.289656 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovn-edpm-compute-global-edpm-compute-global-dw8gp" event={"ID":"a3839557-7c97-4814-9c26-48e2c9674a9b","Type":"ContainerStarted","Data":"853f5dcd4e910e741a9e3919fe95580a2245f409cbb1171f83441a8643074e00"} Dec 03 12:57:13 crc kubenswrapper[5052]: I1203 12:57:13.316001 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/ovn-edpm-compute-global-edpm-compute-global-dw8gp" podStartSLOduration=1.903038067 podStartE2EDuration="2.315978146s" podCreationTimestamp="2025-12-03 12:57:11 +0000 UTC" firstStartedPulling="2025-12-03 12:57:12.060675727 +0000 UTC m=+5822.648917491" lastFinishedPulling="2025-12-03 12:57:12.473615786 +0000 UTC m=+5823.061857570" observedRunningTime="2025-12-03 12:57:13.306028051 +0000 UTC m=+5823.894269835" watchObservedRunningTime="2025-12-03 12:57:13.315978146 +0000 UTC m=+5823.904219920" Dec 03 12:57:14 crc kubenswrapper[5052]: I1203 12:57:14.308062 5052 generic.go:334] "Generic (PLEG): container finished" podID="a3839557-7c97-4814-9c26-48e2c9674a9b" containerID="853f5dcd4e910e741a9e3919fe95580a2245f409cbb1171f83441a8643074e00" exitCode=0 Dec 03 12:57:14 crc kubenswrapper[5052]: I1203 12:57:14.308127 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovn-edpm-compute-global-edpm-compute-global-dw8gp" event={"ID":"a3839557-7c97-4814-9c26-48e2c9674a9b","Type":"ContainerDied","Data":"853f5dcd4e910e741a9e3919fe95580a2245f409cbb1171f83441a8643074e00"} Dec 03 12:57:15 crc kubenswrapper[5052]: I1203 12:57:15.616305 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovn-edpm-compute-global-edpm-compute-global-dw8gp" Dec 03 12:57:15 crc kubenswrapper[5052]: I1203 12:57:15.767635 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/a3839557-7c97-4814-9c26-48e2c9674a9b-inventory\") pod \"a3839557-7c97-4814-9c26-48e2c9674a9b\" (UID: \"a3839557-7c97-4814-9c26-48e2c9674a9b\") " Dec 03 12:57:15 crc kubenswrapper[5052]: I1203 12:57:15.767745 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/a3839557-7c97-4814-9c26-48e2c9674a9b-ssh-key\") pod \"a3839557-7c97-4814-9c26-48e2c9674a9b\" (UID: \"a3839557-7c97-4814-9c26-48e2c9674a9b\") " Dec 03 12:57:15 crc kubenswrapper[5052]: I1203 12:57:15.767808 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovn-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a3839557-7c97-4814-9c26-48e2c9674a9b-ovn-combined-ca-bundle\") pod \"a3839557-7c97-4814-9c26-48e2c9674a9b\" (UID: \"a3839557-7c97-4814-9c26-48e2c9674a9b\") " Dec 03 12:57:15 crc kubenswrapper[5052]: I1203 12:57:15.768103 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2mx4b\" (UniqueName: \"kubernetes.io/projected/a3839557-7c97-4814-9c26-48e2c9674a9b-kube-api-access-2mx4b\") pod \"a3839557-7c97-4814-9c26-48e2c9674a9b\" (UID: \"a3839557-7c97-4814-9c26-48e2c9674a9b\") " Dec 03 12:57:15 crc kubenswrapper[5052]: I1203 12:57:15.768186 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovncontroller-config-0\" (UniqueName: \"kubernetes.io/configmap/a3839557-7c97-4814-9c26-48e2c9674a9b-ovncontroller-config-0\") pod \"a3839557-7c97-4814-9c26-48e2c9674a9b\" (UID: \"a3839557-7c97-4814-9c26-48e2c9674a9b\") " Dec 03 12:57:15 crc kubenswrapper[5052]: I1203 12:57:15.772862 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a3839557-7c97-4814-9c26-48e2c9674a9b-ovn-combined-ca-bundle" (OuterVolumeSpecName: "ovn-combined-ca-bundle") pod "a3839557-7c97-4814-9c26-48e2c9674a9b" (UID: "a3839557-7c97-4814-9c26-48e2c9674a9b"). InnerVolumeSpecName "ovn-combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:57:15 crc kubenswrapper[5052]: I1203 12:57:15.773121 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a3839557-7c97-4814-9c26-48e2c9674a9b-kube-api-access-2mx4b" (OuterVolumeSpecName: "kube-api-access-2mx4b") pod "a3839557-7c97-4814-9c26-48e2c9674a9b" (UID: "a3839557-7c97-4814-9c26-48e2c9674a9b"). InnerVolumeSpecName "kube-api-access-2mx4b". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:57:15 crc kubenswrapper[5052]: I1203 12:57:15.788244 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a3839557-7c97-4814-9c26-48e2c9674a9b-ovncontroller-config-0" (OuterVolumeSpecName: "ovncontroller-config-0") pod "a3839557-7c97-4814-9c26-48e2c9674a9b" (UID: "a3839557-7c97-4814-9c26-48e2c9674a9b"). InnerVolumeSpecName "ovncontroller-config-0". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:57:15 crc kubenswrapper[5052]: I1203 12:57:15.791081 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a3839557-7c97-4814-9c26-48e2c9674a9b-ssh-key" (OuterVolumeSpecName: "ssh-key") pod "a3839557-7c97-4814-9c26-48e2c9674a9b" (UID: "a3839557-7c97-4814-9c26-48e2c9674a9b"). InnerVolumeSpecName "ssh-key". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:57:15 crc kubenswrapper[5052]: I1203 12:57:15.791219 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a3839557-7c97-4814-9c26-48e2c9674a9b-inventory" (OuterVolumeSpecName: "inventory") pod "a3839557-7c97-4814-9c26-48e2c9674a9b" (UID: "a3839557-7c97-4814-9c26-48e2c9674a9b"). InnerVolumeSpecName "inventory". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:57:15 crc kubenswrapper[5052]: I1203 12:57:15.869454 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-2mx4b\" (UniqueName: \"kubernetes.io/projected/a3839557-7c97-4814-9c26-48e2c9674a9b-kube-api-access-2mx4b\") on node \"crc\" DevicePath \"\"" Dec 03 12:57:15 crc kubenswrapper[5052]: I1203 12:57:15.869488 5052 reconciler_common.go:293] "Volume detached for volume \"ovncontroller-config-0\" (UniqueName: \"kubernetes.io/configmap/a3839557-7c97-4814-9c26-48e2c9674a9b-ovncontroller-config-0\") on node \"crc\" DevicePath \"\"" Dec 03 12:57:15 crc kubenswrapper[5052]: I1203 12:57:15.869500 5052 reconciler_common.go:293] "Volume detached for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/a3839557-7c97-4814-9c26-48e2c9674a9b-inventory\") on node \"crc\" DevicePath \"\"" Dec 03 12:57:15 crc kubenswrapper[5052]: I1203 12:57:15.869522 5052 reconciler_common.go:293] "Volume detached for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/a3839557-7c97-4814-9c26-48e2c9674a9b-ssh-key\") on node \"crc\" DevicePath \"\"" Dec 03 12:57:15 crc kubenswrapper[5052]: I1203 12:57:15.869531 5052 reconciler_common.go:293] "Volume detached for volume \"ovn-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a3839557-7c97-4814-9c26-48e2c9674a9b-ovn-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:57:16 crc kubenswrapper[5052]: I1203 12:57:16.331780 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovn-edpm-compute-global-edpm-compute-global-dw8gp" event={"ID":"a3839557-7c97-4814-9c26-48e2c9674a9b","Type":"ContainerDied","Data":"039ef90283d1b2c10c651af0a2987d9ff5c6cee8e7b190616975fc5135aaf8b5"} Dec 03 12:57:16 crc kubenswrapper[5052]: I1203 12:57:16.332089 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="039ef90283d1b2c10c651af0a2987d9ff5c6cee8e7b190616975fc5135aaf8b5" Dec 03 12:57:16 crc kubenswrapper[5052]: I1203 12:57:16.331883 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovn-edpm-compute-global-edpm-compute-global-dw8gp" Dec 03 12:57:16 crc kubenswrapper[5052]: I1203 12:57:16.402797 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/neutron-metadata-edpm-compute-global-edpm-compute-global-8p8rw"] Dec 03 12:57:16 crc kubenswrapper[5052]: E1203 12:57:16.403363 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="a3839557-7c97-4814-9c26-48e2c9674a9b" containerName="ovn-edpm-compute-global-edpm-compute-global" Dec 03 12:57:16 crc kubenswrapper[5052]: I1203 12:57:16.403470 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="a3839557-7c97-4814-9c26-48e2c9674a9b" containerName="ovn-edpm-compute-global-edpm-compute-global" Dec 03 12:57:16 crc kubenswrapper[5052]: I1203 12:57:16.403719 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="a3839557-7c97-4814-9c26-48e2c9674a9b" containerName="ovn-edpm-compute-global-edpm-compute-global" Dec 03 12:57:16 crc kubenswrapper[5052]: I1203 12:57:16.404301 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-metadata-edpm-compute-global-edpm-compute-global-8p8rw" Dec 03 12:57:16 crc kubenswrapper[5052]: I1203 12:57:16.406457 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-metadata-neutron-config" Dec 03 12:57:16 crc kubenswrapper[5052]: I1203 12:57:16.408084 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"combined-ca-bundle" Dec 03 12:57:16 crc kubenswrapper[5052]: I1203 12:57:16.408289 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"dataplane-ansible-ssh-private-key-secret" Dec 03 12:57:16 crc kubenswrapper[5052]: I1203 12:57:16.408863 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"edpm-compute-global-dockercfg-p5rjx" Dec 03 12:57:16 crc kubenswrapper[5052]: I1203 12:57:16.409201 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"neutron-ovn-metadata-agent-neutron-config" Dec 03 12:57:16 crc kubenswrapper[5052]: I1203 12:57:16.409361 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"openstack-aee-default-env" Dec 03 12:57:16 crc kubenswrapper[5052]: I1203 12:57:16.415437 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"dataplanenodeset-edpm-compute-global" Dec 03 12:57:16 crc kubenswrapper[5052]: I1203 12:57:16.422057 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-metadata-edpm-compute-global-edpm-compute-global-8p8rw"] Dec 03 12:57:16 crc kubenswrapper[5052]: I1203 12:57:16.579035 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nova-metadata-neutron-config-2\" (UniqueName: \"kubernetes.io/secret/763203d9-f2d9-48d7-8989-aa8f1402c776-nova-metadata-neutron-config-2\") pod \"neutron-metadata-edpm-compute-global-edpm-compute-global-8p8rw\" (UID: \"763203d9-f2d9-48d7-8989-aa8f1402c776\") " pod="openstack-kuttl-tests/neutron-metadata-edpm-compute-global-edpm-compute-global-8p8rw" Dec 03 12:57:16 crc kubenswrapper[5052]: I1203 12:57:16.579102 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nova-metadata-neutron-config-0\" (UniqueName: \"kubernetes.io/secret/763203d9-f2d9-48d7-8989-aa8f1402c776-nova-metadata-neutron-config-0\") pod \"neutron-metadata-edpm-compute-global-edpm-compute-global-8p8rw\" (UID: \"763203d9-f2d9-48d7-8989-aa8f1402c776\") " pod="openstack-kuttl-tests/neutron-metadata-edpm-compute-global-edpm-compute-global-8p8rw" Dec 03 12:57:16 crc kubenswrapper[5052]: I1203 12:57:16.579151 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/763203d9-f2d9-48d7-8989-aa8f1402c776-inventory\") pod \"neutron-metadata-edpm-compute-global-edpm-compute-global-8p8rw\" (UID: \"763203d9-f2d9-48d7-8989-aa8f1402c776\") " pod="openstack-kuttl-tests/neutron-metadata-edpm-compute-global-edpm-compute-global-8p8rw" Dec 03 12:57:16 crc kubenswrapper[5052]: I1203 12:57:16.579210 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4zptp\" (UniqueName: \"kubernetes.io/projected/763203d9-f2d9-48d7-8989-aa8f1402c776-kube-api-access-4zptp\") pod \"neutron-metadata-edpm-compute-global-edpm-compute-global-8p8rw\" (UID: \"763203d9-f2d9-48d7-8989-aa8f1402c776\") " pod="openstack-kuttl-tests/neutron-metadata-edpm-compute-global-edpm-compute-global-8p8rw" Dec 03 12:57:16 crc kubenswrapper[5052]: I1203 12:57:16.579247 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/763203d9-f2d9-48d7-8989-aa8f1402c776-ssh-key\") pod \"neutron-metadata-edpm-compute-global-edpm-compute-global-8p8rw\" (UID: \"763203d9-f2d9-48d7-8989-aa8f1402c776\") " pod="openstack-kuttl-tests/neutron-metadata-edpm-compute-global-edpm-compute-global-8p8rw" Dec 03 12:57:16 crc kubenswrapper[5052]: I1203 12:57:16.579312 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"neutron-ovn-metadata-agent-neutron-config-0\" (UniqueName: \"kubernetes.io/secret/763203d9-f2d9-48d7-8989-aa8f1402c776-neutron-ovn-metadata-agent-neutron-config-0\") pod \"neutron-metadata-edpm-compute-global-edpm-compute-global-8p8rw\" (UID: \"763203d9-f2d9-48d7-8989-aa8f1402c776\") " pod="openstack-kuttl-tests/neutron-metadata-edpm-compute-global-edpm-compute-global-8p8rw" Dec 03 12:57:16 crc kubenswrapper[5052]: I1203 12:57:16.579335 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"neutron-metadata-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/763203d9-f2d9-48d7-8989-aa8f1402c776-neutron-metadata-combined-ca-bundle\") pod \"neutron-metadata-edpm-compute-global-edpm-compute-global-8p8rw\" (UID: \"763203d9-f2d9-48d7-8989-aa8f1402c776\") " pod="openstack-kuttl-tests/neutron-metadata-edpm-compute-global-edpm-compute-global-8p8rw" Dec 03 12:57:16 crc kubenswrapper[5052]: I1203 12:57:16.579355 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nova-metadata-neutron-config-1\" (UniqueName: \"kubernetes.io/secret/763203d9-f2d9-48d7-8989-aa8f1402c776-nova-metadata-neutron-config-1\") pod \"neutron-metadata-edpm-compute-global-edpm-compute-global-8p8rw\" (UID: \"763203d9-f2d9-48d7-8989-aa8f1402c776\") " pod="openstack-kuttl-tests/neutron-metadata-edpm-compute-global-edpm-compute-global-8p8rw" Dec 03 12:57:16 crc kubenswrapper[5052]: I1203 12:57:16.680161 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"neutron-ovn-metadata-agent-neutron-config-0\" (UniqueName: \"kubernetes.io/secret/763203d9-f2d9-48d7-8989-aa8f1402c776-neutron-ovn-metadata-agent-neutron-config-0\") pod \"neutron-metadata-edpm-compute-global-edpm-compute-global-8p8rw\" (UID: \"763203d9-f2d9-48d7-8989-aa8f1402c776\") " pod="openstack-kuttl-tests/neutron-metadata-edpm-compute-global-edpm-compute-global-8p8rw" Dec 03 12:57:16 crc kubenswrapper[5052]: I1203 12:57:16.680205 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"neutron-metadata-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/763203d9-f2d9-48d7-8989-aa8f1402c776-neutron-metadata-combined-ca-bundle\") pod \"neutron-metadata-edpm-compute-global-edpm-compute-global-8p8rw\" (UID: \"763203d9-f2d9-48d7-8989-aa8f1402c776\") " pod="openstack-kuttl-tests/neutron-metadata-edpm-compute-global-edpm-compute-global-8p8rw" Dec 03 12:57:16 crc kubenswrapper[5052]: I1203 12:57:16.680423 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"nova-metadata-neutron-config-1\" (UniqueName: \"kubernetes.io/secret/763203d9-f2d9-48d7-8989-aa8f1402c776-nova-metadata-neutron-config-1\") pod \"neutron-metadata-edpm-compute-global-edpm-compute-global-8p8rw\" (UID: \"763203d9-f2d9-48d7-8989-aa8f1402c776\") " pod="openstack-kuttl-tests/neutron-metadata-edpm-compute-global-edpm-compute-global-8p8rw" Dec 03 12:57:16 crc kubenswrapper[5052]: I1203 12:57:16.680467 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"nova-metadata-neutron-config-2\" (UniqueName: \"kubernetes.io/secret/763203d9-f2d9-48d7-8989-aa8f1402c776-nova-metadata-neutron-config-2\") pod \"neutron-metadata-edpm-compute-global-edpm-compute-global-8p8rw\" (UID: \"763203d9-f2d9-48d7-8989-aa8f1402c776\") " pod="openstack-kuttl-tests/neutron-metadata-edpm-compute-global-edpm-compute-global-8p8rw" Dec 03 12:57:16 crc kubenswrapper[5052]: I1203 12:57:16.680556 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"nova-metadata-neutron-config-0\" (UniqueName: \"kubernetes.io/secret/763203d9-f2d9-48d7-8989-aa8f1402c776-nova-metadata-neutron-config-0\") pod \"neutron-metadata-edpm-compute-global-edpm-compute-global-8p8rw\" (UID: \"763203d9-f2d9-48d7-8989-aa8f1402c776\") " pod="openstack-kuttl-tests/neutron-metadata-edpm-compute-global-edpm-compute-global-8p8rw" Dec 03 12:57:16 crc kubenswrapper[5052]: I1203 12:57:16.680584 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/763203d9-f2d9-48d7-8989-aa8f1402c776-inventory\") pod \"neutron-metadata-edpm-compute-global-edpm-compute-global-8p8rw\" (UID: \"763203d9-f2d9-48d7-8989-aa8f1402c776\") " pod="openstack-kuttl-tests/neutron-metadata-edpm-compute-global-edpm-compute-global-8p8rw" Dec 03 12:57:16 crc kubenswrapper[5052]: I1203 12:57:16.680606 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-4zptp\" (UniqueName: \"kubernetes.io/projected/763203d9-f2d9-48d7-8989-aa8f1402c776-kube-api-access-4zptp\") pod \"neutron-metadata-edpm-compute-global-edpm-compute-global-8p8rw\" (UID: \"763203d9-f2d9-48d7-8989-aa8f1402c776\") " pod="openstack-kuttl-tests/neutron-metadata-edpm-compute-global-edpm-compute-global-8p8rw" Dec 03 12:57:16 crc kubenswrapper[5052]: I1203 12:57:16.680628 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/763203d9-f2d9-48d7-8989-aa8f1402c776-ssh-key\") pod \"neutron-metadata-edpm-compute-global-edpm-compute-global-8p8rw\" (UID: \"763203d9-f2d9-48d7-8989-aa8f1402c776\") " pod="openstack-kuttl-tests/neutron-metadata-edpm-compute-global-edpm-compute-global-8p8rw" Dec 03 12:57:16 crc kubenswrapper[5052]: I1203 12:57:16.685930 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"nova-metadata-neutron-config-0\" (UniqueName: \"kubernetes.io/secret/763203d9-f2d9-48d7-8989-aa8f1402c776-nova-metadata-neutron-config-0\") pod \"neutron-metadata-edpm-compute-global-edpm-compute-global-8p8rw\" (UID: \"763203d9-f2d9-48d7-8989-aa8f1402c776\") " pod="openstack-kuttl-tests/neutron-metadata-edpm-compute-global-edpm-compute-global-8p8rw" Dec 03 12:57:16 crc kubenswrapper[5052]: I1203 12:57:16.686000 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/763203d9-f2d9-48d7-8989-aa8f1402c776-inventory\") pod \"neutron-metadata-edpm-compute-global-edpm-compute-global-8p8rw\" (UID: \"763203d9-f2d9-48d7-8989-aa8f1402c776\") " pod="openstack-kuttl-tests/neutron-metadata-edpm-compute-global-edpm-compute-global-8p8rw" Dec 03 12:57:16 crc kubenswrapper[5052]: I1203 12:57:16.686325 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"neutron-metadata-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/763203d9-f2d9-48d7-8989-aa8f1402c776-neutron-metadata-combined-ca-bundle\") pod \"neutron-metadata-edpm-compute-global-edpm-compute-global-8p8rw\" (UID: \"763203d9-f2d9-48d7-8989-aa8f1402c776\") " pod="openstack-kuttl-tests/neutron-metadata-edpm-compute-global-edpm-compute-global-8p8rw" Dec 03 12:57:16 crc kubenswrapper[5052]: I1203 12:57:16.688230 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/763203d9-f2d9-48d7-8989-aa8f1402c776-ssh-key\") pod \"neutron-metadata-edpm-compute-global-edpm-compute-global-8p8rw\" (UID: \"763203d9-f2d9-48d7-8989-aa8f1402c776\") " pod="openstack-kuttl-tests/neutron-metadata-edpm-compute-global-edpm-compute-global-8p8rw" Dec 03 12:57:16 crc kubenswrapper[5052]: I1203 12:57:16.688725 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"neutron-ovn-metadata-agent-neutron-config-0\" (UniqueName: \"kubernetes.io/secret/763203d9-f2d9-48d7-8989-aa8f1402c776-neutron-ovn-metadata-agent-neutron-config-0\") pod \"neutron-metadata-edpm-compute-global-edpm-compute-global-8p8rw\" (UID: \"763203d9-f2d9-48d7-8989-aa8f1402c776\") " pod="openstack-kuttl-tests/neutron-metadata-edpm-compute-global-edpm-compute-global-8p8rw" Dec 03 12:57:16 crc kubenswrapper[5052]: I1203 12:57:16.689450 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"nova-metadata-neutron-config-2\" (UniqueName: \"kubernetes.io/secret/763203d9-f2d9-48d7-8989-aa8f1402c776-nova-metadata-neutron-config-2\") pod \"neutron-metadata-edpm-compute-global-edpm-compute-global-8p8rw\" (UID: \"763203d9-f2d9-48d7-8989-aa8f1402c776\") " pod="openstack-kuttl-tests/neutron-metadata-edpm-compute-global-edpm-compute-global-8p8rw" Dec 03 12:57:16 crc kubenswrapper[5052]: I1203 12:57:16.689993 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"nova-metadata-neutron-config-1\" (UniqueName: \"kubernetes.io/secret/763203d9-f2d9-48d7-8989-aa8f1402c776-nova-metadata-neutron-config-1\") pod \"neutron-metadata-edpm-compute-global-edpm-compute-global-8p8rw\" (UID: \"763203d9-f2d9-48d7-8989-aa8f1402c776\") " pod="openstack-kuttl-tests/neutron-metadata-edpm-compute-global-edpm-compute-global-8p8rw" Dec 03 12:57:16 crc kubenswrapper[5052]: I1203 12:57:16.718028 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-4zptp\" (UniqueName: \"kubernetes.io/projected/763203d9-f2d9-48d7-8989-aa8f1402c776-kube-api-access-4zptp\") pod \"neutron-metadata-edpm-compute-global-edpm-compute-global-8p8rw\" (UID: \"763203d9-f2d9-48d7-8989-aa8f1402c776\") " pod="openstack-kuttl-tests/neutron-metadata-edpm-compute-global-edpm-compute-global-8p8rw" Dec 03 12:57:16 crc kubenswrapper[5052]: I1203 12:57:16.725664 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-metadata-edpm-compute-global-edpm-compute-global-8p8rw" Dec 03 12:57:17 crc kubenswrapper[5052]: I1203 12:57:17.169043 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-metadata-edpm-compute-global-edpm-compute-global-8p8rw"] Dec 03 12:57:17 crc kubenswrapper[5052]: W1203 12:57:17.178289 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod763203d9_f2d9_48d7_8989_aa8f1402c776.slice/crio-b054a8d5cbc0fde9d61731371a7127a2f86226c73e290c2dcf0e96049c14b15b WatchSource:0}: Error finding container b054a8d5cbc0fde9d61731371a7127a2f86226c73e290c2dcf0e96049c14b15b: Status 404 returned error can't find the container with id b054a8d5cbc0fde9d61731371a7127a2f86226c73e290c2dcf0e96049c14b15b Dec 03 12:57:17 crc kubenswrapper[5052]: I1203 12:57:17.341128 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-metadata-edpm-compute-global-edpm-compute-global-8p8rw" event={"ID":"763203d9-f2d9-48d7-8989-aa8f1402c776","Type":"ContainerStarted","Data":"b054a8d5cbc0fde9d61731371a7127a2f86226c73e290c2dcf0e96049c14b15b"} Dec 03 12:57:18 crc kubenswrapper[5052]: I1203 12:57:18.351197 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-metadata-edpm-compute-global-edpm-compute-global-8p8rw" event={"ID":"763203d9-f2d9-48d7-8989-aa8f1402c776","Type":"ContainerStarted","Data":"d6db46a5be697545a8042aeef3cecb828d2a934319fa3b32c5fa2d569d660329"} Dec 03 12:57:18 crc kubenswrapper[5052]: I1203 12:57:18.384135 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/neutron-metadata-edpm-compute-global-edpm-compute-global-8p8rw" podStartSLOduration=1.856370664 podStartE2EDuration="2.384110996s" podCreationTimestamp="2025-12-03 12:57:16 +0000 UTC" firstStartedPulling="2025-12-03 12:57:17.180796103 +0000 UTC m=+5827.769037867" lastFinishedPulling="2025-12-03 12:57:17.708536435 +0000 UTC m=+5828.296778199" observedRunningTime="2025-12-03 12:57:18.367504161 +0000 UTC m=+5828.955745935" watchObservedRunningTime="2025-12-03 12:57:18.384110996 +0000 UTC m=+5828.972352760" Dec 03 12:57:19 crc kubenswrapper[5052]: I1203 12:57:19.361776 5052 generic.go:334] "Generic (PLEG): container finished" podID="763203d9-f2d9-48d7-8989-aa8f1402c776" containerID="d6db46a5be697545a8042aeef3cecb828d2a934319fa3b32c5fa2d569d660329" exitCode=0 Dec 03 12:57:19 crc kubenswrapper[5052]: I1203 12:57:19.361884 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-metadata-edpm-compute-global-edpm-compute-global-8p8rw" event={"ID":"763203d9-f2d9-48d7-8989-aa8f1402c776","Type":"ContainerDied","Data":"d6db46a5be697545a8042aeef3cecb828d2a934319fa3b32c5fa2d569d660329"} Dec 03 12:57:20 crc kubenswrapper[5052]: I1203 12:57:20.650884 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-metadata-edpm-compute-global-edpm-compute-global-8p8rw" Dec 03 12:57:20 crc kubenswrapper[5052]: I1203 12:57:20.674092 5052 scope.go:117] "RemoveContainer" containerID="69fa7fbcd2aceb7c53703a19508876f93a97dc9adabb79c16fb691d90f606d58" Dec 03 12:57:20 crc kubenswrapper[5052]: E1203 12:57:20.674435 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 12:57:20 crc kubenswrapper[5052]: I1203 12:57:20.839930 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4zptp\" (UniqueName: \"kubernetes.io/projected/763203d9-f2d9-48d7-8989-aa8f1402c776-kube-api-access-4zptp\") pod \"763203d9-f2d9-48d7-8989-aa8f1402c776\" (UID: \"763203d9-f2d9-48d7-8989-aa8f1402c776\") " Dec 03 12:57:20 crc kubenswrapper[5052]: I1203 12:57:20.839982 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"nova-metadata-neutron-config-1\" (UniqueName: \"kubernetes.io/secret/763203d9-f2d9-48d7-8989-aa8f1402c776-nova-metadata-neutron-config-1\") pod \"763203d9-f2d9-48d7-8989-aa8f1402c776\" (UID: \"763203d9-f2d9-48d7-8989-aa8f1402c776\") " Dec 03 12:57:20 crc kubenswrapper[5052]: I1203 12:57:20.840725 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/763203d9-f2d9-48d7-8989-aa8f1402c776-ssh-key\") pod \"763203d9-f2d9-48d7-8989-aa8f1402c776\" (UID: \"763203d9-f2d9-48d7-8989-aa8f1402c776\") " Dec 03 12:57:20 crc kubenswrapper[5052]: I1203 12:57:20.840841 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/763203d9-f2d9-48d7-8989-aa8f1402c776-inventory\") pod \"763203d9-f2d9-48d7-8989-aa8f1402c776\" (UID: \"763203d9-f2d9-48d7-8989-aa8f1402c776\") " Dec 03 12:57:20 crc kubenswrapper[5052]: I1203 12:57:20.840881 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"neutron-ovn-metadata-agent-neutron-config-0\" (UniqueName: \"kubernetes.io/secret/763203d9-f2d9-48d7-8989-aa8f1402c776-neutron-ovn-metadata-agent-neutron-config-0\") pod \"763203d9-f2d9-48d7-8989-aa8f1402c776\" (UID: \"763203d9-f2d9-48d7-8989-aa8f1402c776\") " Dec 03 12:57:20 crc kubenswrapper[5052]: I1203 12:57:20.840902 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"nova-metadata-neutron-config-0\" (UniqueName: \"kubernetes.io/secret/763203d9-f2d9-48d7-8989-aa8f1402c776-nova-metadata-neutron-config-0\") pod \"763203d9-f2d9-48d7-8989-aa8f1402c776\" (UID: \"763203d9-f2d9-48d7-8989-aa8f1402c776\") " Dec 03 12:57:20 crc kubenswrapper[5052]: I1203 12:57:20.840953 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"nova-metadata-neutron-config-2\" (UniqueName: \"kubernetes.io/secret/763203d9-f2d9-48d7-8989-aa8f1402c776-nova-metadata-neutron-config-2\") pod \"763203d9-f2d9-48d7-8989-aa8f1402c776\" (UID: \"763203d9-f2d9-48d7-8989-aa8f1402c776\") " Dec 03 12:57:20 crc kubenswrapper[5052]: I1203 12:57:20.840980 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"neutron-metadata-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/763203d9-f2d9-48d7-8989-aa8f1402c776-neutron-metadata-combined-ca-bundle\") pod \"763203d9-f2d9-48d7-8989-aa8f1402c776\" (UID: \"763203d9-f2d9-48d7-8989-aa8f1402c776\") " Dec 03 12:57:20 crc kubenswrapper[5052]: I1203 12:57:20.846770 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/763203d9-f2d9-48d7-8989-aa8f1402c776-neutron-metadata-combined-ca-bundle" (OuterVolumeSpecName: "neutron-metadata-combined-ca-bundle") pod "763203d9-f2d9-48d7-8989-aa8f1402c776" (UID: "763203d9-f2d9-48d7-8989-aa8f1402c776"). InnerVolumeSpecName "neutron-metadata-combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:57:20 crc kubenswrapper[5052]: I1203 12:57:20.846795 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/763203d9-f2d9-48d7-8989-aa8f1402c776-kube-api-access-4zptp" (OuterVolumeSpecName: "kube-api-access-4zptp") pod "763203d9-f2d9-48d7-8989-aa8f1402c776" (UID: "763203d9-f2d9-48d7-8989-aa8f1402c776"). InnerVolumeSpecName "kube-api-access-4zptp". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:57:20 crc kubenswrapper[5052]: I1203 12:57:20.871149 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/763203d9-f2d9-48d7-8989-aa8f1402c776-nova-metadata-neutron-config-2" (OuterVolumeSpecName: "nova-metadata-neutron-config-2") pod "763203d9-f2d9-48d7-8989-aa8f1402c776" (UID: "763203d9-f2d9-48d7-8989-aa8f1402c776"). InnerVolumeSpecName "nova-metadata-neutron-config-2". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:57:20 crc kubenswrapper[5052]: I1203 12:57:20.883314 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/763203d9-f2d9-48d7-8989-aa8f1402c776-inventory" (OuterVolumeSpecName: "inventory") pod "763203d9-f2d9-48d7-8989-aa8f1402c776" (UID: "763203d9-f2d9-48d7-8989-aa8f1402c776"). InnerVolumeSpecName "inventory". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:57:20 crc kubenswrapper[5052]: I1203 12:57:20.887034 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/763203d9-f2d9-48d7-8989-aa8f1402c776-ssh-key" (OuterVolumeSpecName: "ssh-key") pod "763203d9-f2d9-48d7-8989-aa8f1402c776" (UID: "763203d9-f2d9-48d7-8989-aa8f1402c776"). InnerVolumeSpecName "ssh-key". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:57:20 crc kubenswrapper[5052]: I1203 12:57:20.887098 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/763203d9-f2d9-48d7-8989-aa8f1402c776-nova-metadata-neutron-config-1" (OuterVolumeSpecName: "nova-metadata-neutron-config-1") pod "763203d9-f2d9-48d7-8989-aa8f1402c776" (UID: "763203d9-f2d9-48d7-8989-aa8f1402c776"). InnerVolumeSpecName "nova-metadata-neutron-config-1". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:57:20 crc kubenswrapper[5052]: I1203 12:57:20.888771 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/763203d9-f2d9-48d7-8989-aa8f1402c776-nova-metadata-neutron-config-0" (OuterVolumeSpecName: "nova-metadata-neutron-config-0") pod "763203d9-f2d9-48d7-8989-aa8f1402c776" (UID: "763203d9-f2d9-48d7-8989-aa8f1402c776"). InnerVolumeSpecName "nova-metadata-neutron-config-0". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:57:20 crc kubenswrapper[5052]: I1203 12:57:20.901698 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/763203d9-f2d9-48d7-8989-aa8f1402c776-neutron-ovn-metadata-agent-neutron-config-0" (OuterVolumeSpecName: "neutron-ovn-metadata-agent-neutron-config-0") pod "763203d9-f2d9-48d7-8989-aa8f1402c776" (UID: "763203d9-f2d9-48d7-8989-aa8f1402c776"). InnerVolumeSpecName "neutron-ovn-metadata-agent-neutron-config-0". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:57:20 crc kubenswrapper[5052]: I1203 12:57:20.942491 5052 reconciler_common.go:293] "Volume detached for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/763203d9-f2d9-48d7-8989-aa8f1402c776-inventory\") on node \"crc\" DevicePath \"\"" Dec 03 12:57:20 crc kubenswrapper[5052]: I1203 12:57:20.942544 5052 reconciler_common.go:293] "Volume detached for volume \"neutron-ovn-metadata-agent-neutron-config-0\" (UniqueName: \"kubernetes.io/secret/763203d9-f2d9-48d7-8989-aa8f1402c776-neutron-ovn-metadata-agent-neutron-config-0\") on node \"crc\" DevicePath \"\"" Dec 03 12:57:20 crc kubenswrapper[5052]: I1203 12:57:20.942560 5052 reconciler_common.go:293] "Volume detached for volume \"nova-metadata-neutron-config-0\" (UniqueName: \"kubernetes.io/secret/763203d9-f2d9-48d7-8989-aa8f1402c776-nova-metadata-neutron-config-0\") on node \"crc\" DevicePath \"\"" Dec 03 12:57:20 crc kubenswrapper[5052]: I1203 12:57:20.942573 5052 reconciler_common.go:293] "Volume detached for volume \"nova-metadata-neutron-config-2\" (UniqueName: \"kubernetes.io/secret/763203d9-f2d9-48d7-8989-aa8f1402c776-nova-metadata-neutron-config-2\") on node \"crc\" DevicePath \"\"" Dec 03 12:57:20 crc kubenswrapper[5052]: I1203 12:57:20.942585 5052 reconciler_common.go:293] "Volume detached for volume \"neutron-metadata-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/763203d9-f2d9-48d7-8989-aa8f1402c776-neutron-metadata-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:57:20 crc kubenswrapper[5052]: I1203 12:57:20.942600 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-4zptp\" (UniqueName: \"kubernetes.io/projected/763203d9-f2d9-48d7-8989-aa8f1402c776-kube-api-access-4zptp\") on node \"crc\" DevicePath \"\"" Dec 03 12:57:20 crc kubenswrapper[5052]: I1203 12:57:20.942612 5052 reconciler_common.go:293] "Volume detached for volume \"nova-metadata-neutron-config-1\" (UniqueName: \"kubernetes.io/secret/763203d9-f2d9-48d7-8989-aa8f1402c776-nova-metadata-neutron-config-1\") on node \"crc\" DevicePath \"\"" Dec 03 12:57:20 crc kubenswrapper[5052]: I1203 12:57:20.942625 5052 reconciler_common.go:293] "Volume detached for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/763203d9-f2d9-48d7-8989-aa8f1402c776-ssh-key\") on node \"crc\" DevicePath \"\"" Dec 03 12:57:21 crc kubenswrapper[5052]: I1203 12:57:21.384476 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-metadata-edpm-compute-global-edpm-compute-global-8p8rw" event={"ID":"763203d9-f2d9-48d7-8989-aa8f1402c776","Type":"ContainerDied","Data":"b054a8d5cbc0fde9d61731371a7127a2f86226c73e290c2dcf0e96049c14b15b"} Dec 03 12:57:21 crc kubenswrapper[5052]: I1203 12:57:21.384532 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b054a8d5cbc0fde9d61731371a7127a2f86226c73e290c2dcf0e96049c14b15b" Dec 03 12:57:21 crc kubenswrapper[5052]: I1203 12:57:21.384584 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-metadata-edpm-compute-global-edpm-compute-global-8p8rw" Dec 03 12:57:21 crc kubenswrapper[5052]: I1203 12:57:21.452210 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/neutron-ovn-edpm-compute-global-edpm-compute-global-mvdtr"] Dec 03 12:57:21 crc kubenswrapper[5052]: E1203 12:57:21.453863 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="763203d9-f2d9-48d7-8989-aa8f1402c776" containerName="neutron-metadata-edpm-compute-global-edpm-compute-global" Dec 03 12:57:21 crc kubenswrapper[5052]: I1203 12:57:21.453898 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="763203d9-f2d9-48d7-8989-aa8f1402c776" containerName="neutron-metadata-edpm-compute-global-edpm-compute-global" Dec 03 12:57:21 crc kubenswrapper[5052]: I1203 12:57:21.454125 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="763203d9-f2d9-48d7-8989-aa8f1402c776" containerName="neutron-metadata-edpm-compute-global-edpm-compute-global" Dec 03 12:57:21 crc kubenswrapper[5052]: I1203 12:57:21.454849 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-ovn-edpm-compute-global-edpm-compute-global-mvdtr" Dec 03 12:57:21 crc kubenswrapper[5052]: I1203 12:57:21.459814 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-ovn-edpm-compute-global-edpm-compute-global-mvdtr"] Dec 03 12:57:21 crc kubenswrapper[5052]: I1203 12:57:21.461932 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"openstack-aee-default-env" Dec 03 12:57:21 crc kubenswrapper[5052]: I1203 12:57:21.461963 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"combined-ca-bundle" Dec 03 12:57:21 crc kubenswrapper[5052]: I1203 12:57:21.462009 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"edpm-compute-global-dockercfg-p5rjx" Dec 03 12:57:21 crc kubenswrapper[5052]: I1203 12:57:21.462129 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"dataplane-ansible-ssh-private-key-secret" Dec 03 12:57:21 crc kubenswrapper[5052]: I1203 12:57:21.462243 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"neutron-ovn-agent-neutron-config" Dec 03 12:57:21 crc kubenswrapper[5052]: I1203 12:57:21.462331 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"dataplanenodeset-edpm-compute-global" Dec 03 12:57:21 crc kubenswrapper[5052]: I1203 12:57:21.550678 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mv9tg\" (UniqueName: \"kubernetes.io/projected/86edaec9-e52a-4e74-8cc9-322449b190d9-kube-api-access-mv9tg\") pod \"neutron-ovn-edpm-compute-global-edpm-compute-global-mvdtr\" (UID: \"86edaec9-e52a-4e74-8cc9-322449b190d9\") " pod="openstack-kuttl-tests/neutron-ovn-edpm-compute-global-edpm-compute-global-mvdtr" Dec 03 12:57:21 crc kubenswrapper[5052]: I1203 12:57:21.550748 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"neutron-ovn-agent-neutron-config-0\" (UniqueName: \"kubernetes.io/secret/86edaec9-e52a-4e74-8cc9-322449b190d9-neutron-ovn-agent-neutron-config-0\") pod \"neutron-ovn-edpm-compute-global-edpm-compute-global-mvdtr\" (UID: \"86edaec9-e52a-4e74-8cc9-322449b190d9\") " pod="openstack-kuttl-tests/neutron-ovn-edpm-compute-global-edpm-compute-global-mvdtr" Dec 03 12:57:21 crc kubenswrapper[5052]: I1203 12:57:21.550792 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/86edaec9-e52a-4e74-8cc9-322449b190d9-ssh-key\") pod \"neutron-ovn-edpm-compute-global-edpm-compute-global-mvdtr\" (UID: \"86edaec9-e52a-4e74-8cc9-322449b190d9\") " pod="openstack-kuttl-tests/neutron-ovn-edpm-compute-global-edpm-compute-global-mvdtr" Dec 03 12:57:21 crc kubenswrapper[5052]: I1203 12:57:21.550817 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"neutron-ovn-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/86edaec9-e52a-4e74-8cc9-322449b190d9-neutron-ovn-combined-ca-bundle\") pod \"neutron-ovn-edpm-compute-global-edpm-compute-global-mvdtr\" (UID: \"86edaec9-e52a-4e74-8cc9-322449b190d9\") " pod="openstack-kuttl-tests/neutron-ovn-edpm-compute-global-edpm-compute-global-mvdtr" Dec 03 12:57:21 crc kubenswrapper[5052]: I1203 12:57:21.550849 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/86edaec9-e52a-4e74-8cc9-322449b190d9-inventory\") pod \"neutron-ovn-edpm-compute-global-edpm-compute-global-mvdtr\" (UID: \"86edaec9-e52a-4e74-8cc9-322449b190d9\") " pod="openstack-kuttl-tests/neutron-ovn-edpm-compute-global-edpm-compute-global-mvdtr" Dec 03 12:57:21 crc kubenswrapper[5052]: I1203 12:57:21.652068 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-mv9tg\" (UniqueName: \"kubernetes.io/projected/86edaec9-e52a-4e74-8cc9-322449b190d9-kube-api-access-mv9tg\") pod \"neutron-ovn-edpm-compute-global-edpm-compute-global-mvdtr\" (UID: \"86edaec9-e52a-4e74-8cc9-322449b190d9\") " pod="openstack-kuttl-tests/neutron-ovn-edpm-compute-global-edpm-compute-global-mvdtr" Dec 03 12:57:21 crc kubenswrapper[5052]: I1203 12:57:21.652175 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"neutron-ovn-agent-neutron-config-0\" (UniqueName: \"kubernetes.io/secret/86edaec9-e52a-4e74-8cc9-322449b190d9-neutron-ovn-agent-neutron-config-0\") pod \"neutron-ovn-edpm-compute-global-edpm-compute-global-mvdtr\" (UID: \"86edaec9-e52a-4e74-8cc9-322449b190d9\") " pod="openstack-kuttl-tests/neutron-ovn-edpm-compute-global-edpm-compute-global-mvdtr" Dec 03 12:57:21 crc kubenswrapper[5052]: I1203 12:57:21.652220 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/86edaec9-e52a-4e74-8cc9-322449b190d9-ssh-key\") pod \"neutron-ovn-edpm-compute-global-edpm-compute-global-mvdtr\" (UID: \"86edaec9-e52a-4e74-8cc9-322449b190d9\") " pod="openstack-kuttl-tests/neutron-ovn-edpm-compute-global-edpm-compute-global-mvdtr" Dec 03 12:57:21 crc kubenswrapper[5052]: I1203 12:57:21.652253 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"neutron-ovn-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/86edaec9-e52a-4e74-8cc9-322449b190d9-neutron-ovn-combined-ca-bundle\") pod \"neutron-ovn-edpm-compute-global-edpm-compute-global-mvdtr\" (UID: \"86edaec9-e52a-4e74-8cc9-322449b190d9\") " pod="openstack-kuttl-tests/neutron-ovn-edpm-compute-global-edpm-compute-global-mvdtr" Dec 03 12:57:21 crc kubenswrapper[5052]: I1203 12:57:21.652998 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/86edaec9-e52a-4e74-8cc9-322449b190d9-inventory\") pod \"neutron-ovn-edpm-compute-global-edpm-compute-global-mvdtr\" (UID: \"86edaec9-e52a-4e74-8cc9-322449b190d9\") " pod="openstack-kuttl-tests/neutron-ovn-edpm-compute-global-edpm-compute-global-mvdtr" Dec 03 12:57:21 crc kubenswrapper[5052]: I1203 12:57:21.655906 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/86edaec9-e52a-4e74-8cc9-322449b190d9-ssh-key\") pod \"neutron-ovn-edpm-compute-global-edpm-compute-global-mvdtr\" (UID: \"86edaec9-e52a-4e74-8cc9-322449b190d9\") " pod="openstack-kuttl-tests/neutron-ovn-edpm-compute-global-edpm-compute-global-mvdtr" Dec 03 12:57:21 crc kubenswrapper[5052]: I1203 12:57:21.656270 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/86edaec9-e52a-4e74-8cc9-322449b190d9-inventory\") pod \"neutron-ovn-edpm-compute-global-edpm-compute-global-mvdtr\" (UID: \"86edaec9-e52a-4e74-8cc9-322449b190d9\") " pod="openstack-kuttl-tests/neutron-ovn-edpm-compute-global-edpm-compute-global-mvdtr" Dec 03 12:57:21 crc kubenswrapper[5052]: I1203 12:57:21.656993 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"neutron-ovn-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/86edaec9-e52a-4e74-8cc9-322449b190d9-neutron-ovn-combined-ca-bundle\") pod \"neutron-ovn-edpm-compute-global-edpm-compute-global-mvdtr\" (UID: \"86edaec9-e52a-4e74-8cc9-322449b190d9\") " pod="openstack-kuttl-tests/neutron-ovn-edpm-compute-global-edpm-compute-global-mvdtr" Dec 03 12:57:21 crc kubenswrapper[5052]: I1203 12:57:21.665624 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"neutron-ovn-agent-neutron-config-0\" (UniqueName: \"kubernetes.io/secret/86edaec9-e52a-4e74-8cc9-322449b190d9-neutron-ovn-agent-neutron-config-0\") pod \"neutron-ovn-edpm-compute-global-edpm-compute-global-mvdtr\" (UID: \"86edaec9-e52a-4e74-8cc9-322449b190d9\") " pod="openstack-kuttl-tests/neutron-ovn-edpm-compute-global-edpm-compute-global-mvdtr" Dec 03 12:57:21 crc kubenswrapper[5052]: I1203 12:57:21.670651 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-mv9tg\" (UniqueName: \"kubernetes.io/projected/86edaec9-e52a-4e74-8cc9-322449b190d9-kube-api-access-mv9tg\") pod \"neutron-ovn-edpm-compute-global-edpm-compute-global-mvdtr\" (UID: \"86edaec9-e52a-4e74-8cc9-322449b190d9\") " pod="openstack-kuttl-tests/neutron-ovn-edpm-compute-global-edpm-compute-global-mvdtr" Dec 03 12:57:21 crc kubenswrapper[5052]: I1203 12:57:21.772168 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-ovn-edpm-compute-global-edpm-compute-global-mvdtr" Dec 03 12:57:22 crc kubenswrapper[5052]: I1203 12:57:22.222612 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-ovn-edpm-compute-global-edpm-compute-global-mvdtr"] Dec 03 12:57:22 crc kubenswrapper[5052]: I1203 12:57:22.392682 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-ovn-edpm-compute-global-edpm-compute-global-mvdtr" event={"ID":"86edaec9-e52a-4e74-8cc9-322449b190d9","Type":"ContainerStarted","Data":"e0534af0aaba7a2306d5aec2624466f8f303016dfbdf3bc5fe9ab8b74e012a42"} Dec 03 12:57:23 crc kubenswrapper[5052]: I1203 12:57:23.402728 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-ovn-edpm-compute-global-edpm-compute-global-mvdtr" event={"ID":"86edaec9-e52a-4e74-8cc9-322449b190d9","Type":"ContainerStarted","Data":"82273b9da0043f3ca02df3042d32fcb96cfef15bae8ef0b8a95055b91ecb1233"} Dec 03 12:57:23 crc kubenswrapper[5052]: I1203 12:57:23.424102 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/neutron-ovn-edpm-compute-global-edpm-compute-global-mvdtr" podStartSLOduration=1.932746899 podStartE2EDuration="2.424076721s" podCreationTimestamp="2025-12-03 12:57:21 +0000 UTC" firstStartedPulling="2025-12-03 12:57:22.265689292 +0000 UTC m=+5832.853931056" lastFinishedPulling="2025-12-03 12:57:22.757019114 +0000 UTC m=+5833.345260878" observedRunningTime="2025-12-03 12:57:23.417687398 +0000 UTC m=+5834.005929162" watchObservedRunningTime="2025-12-03 12:57:23.424076721 +0000 UTC m=+5834.012318505" Dec 03 12:57:24 crc kubenswrapper[5052]: I1203 12:57:24.414827 5052 generic.go:334] "Generic (PLEG): container finished" podID="86edaec9-e52a-4e74-8cc9-322449b190d9" containerID="82273b9da0043f3ca02df3042d32fcb96cfef15bae8ef0b8a95055b91ecb1233" exitCode=0 Dec 03 12:57:24 crc kubenswrapper[5052]: I1203 12:57:24.414890 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-ovn-edpm-compute-global-edpm-compute-global-mvdtr" event={"ID":"86edaec9-e52a-4e74-8cc9-322449b190d9","Type":"ContainerDied","Data":"82273b9da0043f3ca02df3042d32fcb96cfef15bae8ef0b8a95055b91ecb1233"} Dec 03 12:57:25 crc kubenswrapper[5052]: I1203 12:57:25.694043 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-ovn-edpm-compute-global-edpm-compute-global-mvdtr" Dec 03 12:57:25 crc kubenswrapper[5052]: I1203 12:57:25.872150 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"neutron-ovn-agent-neutron-config-0\" (UniqueName: \"kubernetes.io/secret/86edaec9-e52a-4e74-8cc9-322449b190d9-neutron-ovn-agent-neutron-config-0\") pod \"86edaec9-e52a-4e74-8cc9-322449b190d9\" (UID: \"86edaec9-e52a-4e74-8cc9-322449b190d9\") " Dec 03 12:57:25 crc kubenswrapper[5052]: I1203 12:57:25.872239 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"neutron-ovn-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/86edaec9-e52a-4e74-8cc9-322449b190d9-neutron-ovn-combined-ca-bundle\") pod \"86edaec9-e52a-4e74-8cc9-322449b190d9\" (UID: \"86edaec9-e52a-4e74-8cc9-322449b190d9\") " Dec 03 12:57:25 crc kubenswrapper[5052]: I1203 12:57:25.872269 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mv9tg\" (UniqueName: \"kubernetes.io/projected/86edaec9-e52a-4e74-8cc9-322449b190d9-kube-api-access-mv9tg\") pod \"86edaec9-e52a-4e74-8cc9-322449b190d9\" (UID: \"86edaec9-e52a-4e74-8cc9-322449b190d9\") " Dec 03 12:57:25 crc kubenswrapper[5052]: I1203 12:57:25.872289 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/86edaec9-e52a-4e74-8cc9-322449b190d9-inventory\") pod \"86edaec9-e52a-4e74-8cc9-322449b190d9\" (UID: \"86edaec9-e52a-4e74-8cc9-322449b190d9\") " Dec 03 12:57:25 crc kubenswrapper[5052]: I1203 12:57:25.872323 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/86edaec9-e52a-4e74-8cc9-322449b190d9-ssh-key\") pod \"86edaec9-e52a-4e74-8cc9-322449b190d9\" (UID: \"86edaec9-e52a-4e74-8cc9-322449b190d9\") " Dec 03 12:57:25 crc kubenswrapper[5052]: I1203 12:57:25.878932 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/86edaec9-e52a-4e74-8cc9-322449b190d9-neutron-ovn-combined-ca-bundle" (OuterVolumeSpecName: "neutron-ovn-combined-ca-bundle") pod "86edaec9-e52a-4e74-8cc9-322449b190d9" (UID: "86edaec9-e52a-4e74-8cc9-322449b190d9"). InnerVolumeSpecName "neutron-ovn-combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:57:25 crc kubenswrapper[5052]: I1203 12:57:25.879263 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/86edaec9-e52a-4e74-8cc9-322449b190d9-kube-api-access-mv9tg" (OuterVolumeSpecName: "kube-api-access-mv9tg") pod "86edaec9-e52a-4e74-8cc9-322449b190d9" (UID: "86edaec9-e52a-4e74-8cc9-322449b190d9"). InnerVolumeSpecName "kube-api-access-mv9tg". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:57:25 crc kubenswrapper[5052]: I1203 12:57:25.896137 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/86edaec9-e52a-4e74-8cc9-322449b190d9-neutron-ovn-agent-neutron-config-0" (OuterVolumeSpecName: "neutron-ovn-agent-neutron-config-0") pod "86edaec9-e52a-4e74-8cc9-322449b190d9" (UID: "86edaec9-e52a-4e74-8cc9-322449b190d9"). InnerVolumeSpecName "neutron-ovn-agent-neutron-config-0". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:57:25 crc kubenswrapper[5052]: I1203 12:57:25.896197 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/86edaec9-e52a-4e74-8cc9-322449b190d9-ssh-key" (OuterVolumeSpecName: "ssh-key") pod "86edaec9-e52a-4e74-8cc9-322449b190d9" (UID: "86edaec9-e52a-4e74-8cc9-322449b190d9"). InnerVolumeSpecName "ssh-key". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:57:25 crc kubenswrapper[5052]: I1203 12:57:25.897940 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/86edaec9-e52a-4e74-8cc9-322449b190d9-inventory" (OuterVolumeSpecName: "inventory") pod "86edaec9-e52a-4e74-8cc9-322449b190d9" (UID: "86edaec9-e52a-4e74-8cc9-322449b190d9"). InnerVolumeSpecName "inventory". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:57:25 crc kubenswrapper[5052]: I1203 12:57:25.974067 5052 reconciler_common.go:293] "Volume detached for volume \"neutron-ovn-agent-neutron-config-0\" (UniqueName: \"kubernetes.io/secret/86edaec9-e52a-4e74-8cc9-322449b190d9-neutron-ovn-agent-neutron-config-0\") on node \"crc\" DevicePath \"\"" Dec 03 12:57:25 crc kubenswrapper[5052]: I1203 12:57:25.974112 5052 reconciler_common.go:293] "Volume detached for volume \"neutron-ovn-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/86edaec9-e52a-4e74-8cc9-322449b190d9-neutron-ovn-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:57:25 crc kubenswrapper[5052]: I1203 12:57:25.974130 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-mv9tg\" (UniqueName: \"kubernetes.io/projected/86edaec9-e52a-4e74-8cc9-322449b190d9-kube-api-access-mv9tg\") on node \"crc\" DevicePath \"\"" Dec 03 12:57:25 crc kubenswrapper[5052]: I1203 12:57:25.974146 5052 reconciler_common.go:293] "Volume detached for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/86edaec9-e52a-4e74-8cc9-322449b190d9-inventory\") on node \"crc\" DevicePath \"\"" Dec 03 12:57:25 crc kubenswrapper[5052]: I1203 12:57:25.974158 5052 reconciler_common.go:293] "Volume detached for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/86edaec9-e52a-4e74-8cc9-322449b190d9-ssh-key\") on node \"crc\" DevicePath \"\"" Dec 03 12:57:26 crc kubenswrapper[5052]: I1203 12:57:26.431785 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-ovn-edpm-compute-global-edpm-compute-global-mvdtr" event={"ID":"86edaec9-e52a-4e74-8cc9-322449b190d9","Type":"ContainerDied","Data":"e0534af0aaba7a2306d5aec2624466f8f303016dfbdf3bc5fe9ab8b74e012a42"} Dec 03 12:57:26 crc kubenswrapper[5052]: I1203 12:57:26.432083 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e0534af0aaba7a2306d5aec2624466f8f303016dfbdf3bc5fe9ab8b74e012a42" Dec 03 12:57:26 crc kubenswrapper[5052]: I1203 12:57:26.431831 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-ovn-edpm-compute-global-edpm-compute-global-mvdtr" Dec 03 12:57:26 crc kubenswrapper[5052]: I1203 12:57:26.497050 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/neutron-sriov-edpm-compute-global-edpm-compute-global-7xq2m"] Dec 03 12:57:26 crc kubenswrapper[5052]: E1203 12:57:26.497413 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="86edaec9-e52a-4e74-8cc9-322449b190d9" containerName="neutron-ovn-edpm-compute-global-edpm-compute-global" Dec 03 12:57:26 crc kubenswrapper[5052]: I1203 12:57:26.497434 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="86edaec9-e52a-4e74-8cc9-322449b190d9" containerName="neutron-ovn-edpm-compute-global-edpm-compute-global" Dec 03 12:57:26 crc kubenswrapper[5052]: I1203 12:57:26.497777 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="86edaec9-e52a-4e74-8cc9-322449b190d9" containerName="neutron-ovn-edpm-compute-global-edpm-compute-global" Dec 03 12:57:26 crc kubenswrapper[5052]: I1203 12:57:26.498489 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-sriov-edpm-compute-global-edpm-compute-global-7xq2m" Dec 03 12:57:26 crc kubenswrapper[5052]: I1203 12:57:26.501654 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"openstack-aee-default-env" Dec 03 12:57:26 crc kubenswrapper[5052]: I1203 12:57:26.501692 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"neutron-sriov-agent-neutron-config" Dec 03 12:57:26 crc kubenswrapper[5052]: I1203 12:57:26.501726 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"combined-ca-bundle" Dec 03 12:57:26 crc kubenswrapper[5052]: I1203 12:57:26.501868 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"dataplane-ansible-ssh-private-key-secret" Dec 03 12:57:26 crc kubenswrapper[5052]: I1203 12:57:26.501983 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"dataplanenodeset-edpm-compute-global" Dec 03 12:57:26 crc kubenswrapper[5052]: I1203 12:57:26.502099 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"edpm-compute-global-dockercfg-p5rjx" Dec 03 12:57:26 crc kubenswrapper[5052]: I1203 12:57:26.507463 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-sriov-edpm-compute-global-edpm-compute-global-7xq2m"] Dec 03 12:57:26 crc kubenswrapper[5052]: I1203 12:57:26.685773 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/c98ad99e-c1f3-46d7-b810-a702c08308f3-inventory\") pod \"neutron-sriov-edpm-compute-global-edpm-compute-global-7xq2m\" (UID: \"c98ad99e-c1f3-46d7-b810-a702c08308f3\") " pod="openstack-kuttl-tests/neutron-sriov-edpm-compute-global-edpm-compute-global-7xq2m" Dec 03 12:57:26 crc kubenswrapper[5052]: I1203 12:57:26.686163 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pvldd\" (UniqueName: \"kubernetes.io/projected/c98ad99e-c1f3-46d7-b810-a702c08308f3-kube-api-access-pvldd\") pod \"neutron-sriov-edpm-compute-global-edpm-compute-global-7xq2m\" (UID: \"c98ad99e-c1f3-46d7-b810-a702c08308f3\") " pod="openstack-kuttl-tests/neutron-sriov-edpm-compute-global-edpm-compute-global-7xq2m" Dec 03 12:57:26 crc kubenswrapper[5052]: I1203 12:57:26.686318 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"neutron-sriov-agent-neutron-config-0\" (UniqueName: \"kubernetes.io/secret/c98ad99e-c1f3-46d7-b810-a702c08308f3-neutron-sriov-agent-neutron-config-0\") pod \"neutron-sriov-edpm-compute-global-edpm-compute-global-7xq2m\" (UID: \"c98ad99e-c1f3-46d7-b810-a702c08308f3\") " pod="openstack-kuttl-tests/neutron-sriov-edpm-compute-global-edpm-compute-global-7xq2m" Dec 03 12:57:26 crc kubenswrapper[5052]: I1203 12:57:26.686434 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/c98ad99e-c1f3-46d7-b810-a702c08308f3-ssh-key\") pod \"neutron-sriov-edpm-compute-global-edpm-compute-global-7xq2m\" (UID: \"c98ad99e-c1f3-46d7-b810-a702c08308f3\") " pod="openstack-kuttl-tests/neutron-sriov-edpm-compute-global-edpm-compute-global-7xq2m" Dec 03 12:57:26 crc kubenswrapper[5052]: I1203 12:57:26.686696 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"neutron-sriov-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c98ad99e-c1f3-46d7-b810-a702c08308f3-neutron-sriov-combined-ca-bundle\") pod \"neutron-sriov-edpm-compute-global-edpm-compute-global-7xq2m\" (UID: \"c98ad99e-c1f3-46d7-b810-a702c08308f3\") " pod="openstack-kuttl-tests/neutron-sriov-edpm-compute-global-edpm-compute-global-7xq2m" Dec 03 12:57:26 crc kubenswrapper[5052]: I1203 12:57:26.787535 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-pvldd\" (UniqueName: \"kubernetes.io/projected/c98ad99e-c1f3-46d7-b810-a702c08308f3-kube-api-access-pvldd\") pod \"neutron-sriov-edpm-compute-global-edpm-compute-global-7xq2m\" (UID: \"c98ad99e-c1f3-46d7-b810-a702c08308f3\") " pod="openstack-kuttl-tests/neutron-sriov-edpm-compute-global-edpm-compute-global-7xq2m" Dec 03 12:57:26 crc kubenswrapper[5052]: I1203 12:57:26.787613 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"neutron-sriov-agent-neutron-config-0\" (UniqueName: \"kubernetes.io/secret/c98ad99e-c1f3-46d7-b810-a702c08308f3-neutron-sriov-agent-neutron-config-0\") pod \"neutron-sriov-edpm-compute-global-edpm-compute-global-7xq2m\" (UID: \"c98ad99e-c1f3-46d7-b810-a702c08308f3\") " pod="openstack-kuttl-tests/neutron-sriov-edpm-compute-global-edpm-compute-global-7xq2m" Dec 03 12:57:26 crc kubenswrapper[5052]: I1203 12:57:26.787645 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/c98ad99e-c1f3-46d7-b810-a702c08308f3-ssh-key\") pod \"neutron-sriov-edpm-compute-global-edpm-compute-global-7xq2m\" (UID: \"c98ad99e-c1f3-46d7-b810-a702c08308f3\") " pod="openstack-kuttl-tests/neutron-sriov-edpm-compute-global-edpm-compute-global-7xq2m" Dec 03 12:57:26 crc kubenswrapper[5052]: I1203 12:57:26.787704 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"neutron-sriov-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c98ad99e-c1f3-46d7-b810-a702c08308f3-neutron-sriov-combined-ca-bundle\") pod \"neutron-sriov-edpm-compute-global-edpm-compute-global-7xq2m\" (UID: \"c98ad99e-c1f3-46d7-b810-a702c08308f3\") " pod="openstack-kuttl-tests/neutron-sriov-edpm-compute-global-edpm-compute-global-7xq2m" Dec 03 12:57:26 crc kubenswrapper[5052]: I1203 12:57:26.787750 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/c98ad99e-c1f3-46d7-b810-a702c08308f3-inventory\") pod \"neutron-sriov-edpm-compute-global-edpm-compute-global-7xq2m\" (UID: \"c98ad99e-c1f3-46d7-b810-a702c08308f3\") " pod="openstack-kuttl-tests/neutron-sriov-edpm-compute-global-edpm-compute-global-7xq2m" Dec 03 12:57:26 crc kubenswrapper[5052]: I1203 12:57:26.791839 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/c98ad99e-c1f3-46d7-b810-a702c08308f3-ssh-key\") pod \"neutron-sriov-edpm-compute-global-edpm-compute-global-7xq2m\" (UID: \"c98ad99e-c1f3-46d7-b810-a702c08308f3\") " pod="openstack-kuttl-tests/neutron-sriov-edpm-compute-global-edpm-compute-global-7xq2m" Dec 03 12:57:26 crc kubenswrapper[5052]: I1203 12:57:26.791870 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"neutron-sriov-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c98ad99e-c1f3-46d7-b810-a702c08308f3-neutron-sriov-combined-ca-bundle\") pod \"neutron-sriov-edpm-compute-global-edpm-compute-global-7xq2m\" (UID: \"c98ad99e-c1f3-46d7-b810-a702c08308f3\") " pod="openstack-kuttl-tests/neutron-sriov-edpm-compute-global-edpm-compute-global-7xq2m" Dec 03 12:57:26 crc kubenswrapper[5052]: I1203 12:57:26.791899 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"neutron-sriov-agent-neutron-config-0\" (UniqueName: \"kubernetes.io/secret/c98ad99e-c1f3-46d7-b810-a702c08308f3-neutron-sriov-agent-neutron-config-0\") pod \"neutron-sriov-edpm-compute-global-edpm-compute-global-7xq2m\" (UID: \"c98ad99e-c1f3-46d7-b810-a702c08308f3\") " pod="openstack-kuttl-tests/neutron-sriov-edpm-compute-global-edpm-compute-global-7xq2m" Dec 03 12:57:26 crc kubenswrapper[5052]: I1203 12:57:26.792244 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/c98ad99e-c1f3-46d7-b810-a702c08308f3-inventory\") pod \"neutron-sriov-edpm-compute-global-edpm-compute-global-7xq2m\" (UID: \"c98ad99e-c1f3-46d7-b810-a702c08308f3\") " pod="openstack-kuttl-tests/neutron-sriov-edpm-compute-global-edpm-compute-global-7xq2m" Dec 03 12:57:26 crc kubenswrapper[5052]: I1203 12:57:26.803925 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-pvldd\" (UniqueName: \"kubernetes.io/projected/c98ad99e-c1f3-46d7-b810-a702c08308f3-kube-api-access-pvldd\") pod \"neutron-sriov-edpm-compute-global-edpm-compute-global-7xq2m\" (UID: \"c98ad99e-c1f3-46d7-b810-a702c08308f3\") " pod="openstack-kuttl-tests/neutron-sriov-edpm-compute-global-edpm-compute-global-7xq2m" Dec 03 12:57:26 crc kubenswrapper[5052]: I1203 12:57:26.826964 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-sriov-edpm-compute-global-edpm-compute-global-7xq2m" Dec 03 12:57:27 crc kubenswrapper[5052]: I1203 12:57:27.250711 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-sriov-edpm-compute-global-edpm-compute-global-7xq2m"] Dec 03 12:57:27 crc kubenswrapper[5052]: I1203 12:57:27.440660 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-sriov-edpm-compute-global-edpm-compute-global-7xq2m" event={"ID":"c98ad99e-c1f3-46d7-b810-a702c08308f3","Type":"ContainerStarted","Data":"eec6fb0649444b604824839d8aeccc6543a76d8bbcf6915056d91713bf08a010"} Dec 03 12:57:28 crc kubenswrapper[5052]: I1203 12:57:28.449188 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-sriov-edpm-compute-global-edpm-compute-global-7xq2m" event={"ID":"c98ad99e-c1f3-46d7-b810-a702c08308f3","Type":"ContainerStarted","Data":"3d11a4a7e3e873380387601c752ad0fe565caf2439bc630b7365094dbe87ea56"} Dec 03 12:57:29 crc kubenswrapper[5052]: I1203 12:57:29.458747 5052 generic.go:334] "Generic (PLEG): container finished" podID="c98ad99e-c1f3-46d7-b810-a702c08308f3" containerID="3d11a4a7e3e873380387601c752ad0fe565caf2439bc630b7365094dbe87ea56" exitCode=0 Dec 03 12:57:29 crc kubenswrapper[5052]: I1203 12:57:29.458796 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-sriov-edpm-compute-global-edpm-compute-global-7xq2m" event={"ID":"c98ad99e-c1f3-46d7-b810-a702c08308f3","Type":"ContainerDied","Data":"3d11a4a7e3e873380387601c752ad0fe565caf2439bc630b7365094dbe87ea56"} Dec 03 12:57:30 crc kubenswrapper[5052]: I1203 12:57:30.733167 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-sriov-edpm-compute-global-edpm-compute-global-7xq2m" Dec 03 12:57:30 crc kubenswrapper[5052]: I1203 12:57:30.846957 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/c98ad99e-c1f3-46d7-b810-a702c08308f3-inventory\") pod \"c98ad99e-c1f3-46d7-b810-a702c08308f3\" (UID: \"c98ad99e-c1f3-46d7-b810-a702c08308f3\") " Dec 03 12:57:30 crc kubenswrapper[5052]: I1203 12:57:30.847029 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pvldd\" (UniqueName: \"kubernetes.io/projected/c98ad99e-c1f3-46d7-b810-a702c08308f3-kube-api-access-pvldd\") pod \"c98ad99e-c1f3-46d7-b810-a702c08308f3\" (UID: \"c98ad99e-c1f3-46d7-b810-a702c08308f3\") " Dec 03 12:57:30 crc kubenswrapper[5052]: I1203 12:57:30.847060 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/c98ad99e-c1f3-46d7-b810-a702c08308f3-ssh-key\") pod \"c98ad99e-c1f3-46d7-b810-a702c08308f3\" (UID: \"c98ad99e-c1f3-46d7-b810-a702c08308f3\") " Dec 03 12:57:30 crc kubenswrapper[5052]: I1203 12:57:30.847130 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"neutron-sriov-agent-neutron-config-0\" (UniqueName: \"kubernetes.io/secret/c98ad99e-c1f3-46d7-b810-a702c08308f3-neutron-sriov-agent-neutron-config-0\") pod \"c98ad99e-c1f3-46d7-b810-a702c08308f3\" (UID: \"c98ad99e-c1f3-46d7-b810-a702c08308f3\") " Dec 03 12:57:30 crc kubenswrapper[5052]: I1203 12:57:30.847209 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"neutron-sriov-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c98ad99e-c1f3-46d7-b810-a702c08308f3-neutron-sriov-combined-ca-bundle\") pod \"c98ad99e-c1f3-46d7-b810-a702c08308f3\" (UID: \"c98ad99e-c1f3-46d7-b810-a702c08308f3\") " Dec 03 12:57:30 crc kubenswrapper[5052]: I1203 12:57:30.859866 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c98ad99e-c1f3-46d7-b810-a702c08308f3-neutron-sriov-combined-ca-bundle" (OuterVolumeSpecName: "neutron-sriov-combined-ca-bundle") pod "c98ad99e-c1f3-46d7-b810-a702c08308f3" (UID: "c98ad99e-c1f3-46d7-b810-a702c08308f3"). InnerVolumeSpecName "neutron-sriov-combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:57:30 crc kubenswrapper[5052]: I1203 12:57:30.862429 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c98ad99e-c1f3-46d7-b810-a702c08308f3-kube-api-access-pvldd" (OuterVolumeSpecName: "kube-api-access-pvldd") pod "c98ad99e-c1f3-46d7-b810-a702c08308f3" (UID: "c98ad99e-c1f3-46d7-b810-a702c08308f3"). InnerVolumeSpecName "kube-api-access-pvldd". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:57:30 crc kubenswrapper[5052]: I1203 12:57:30.867764 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c98ad99e-c1f3-46d7-b810-a702c08308f3-neutron-sriov-agent-neutron-config-0" (OuterVolumeSpecName: "neutron-sriov-agent-neutron-config-0") pod "c98ad99e-c1f3-46d7-b810-a702c08308f3" (UID: "c98ad99e-c1f3-46d7-b810-a702c08308f3"). InnerVolumeSpecName "neutron-sriov-agent-neutron-config-0". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:57:30 crc kubenswrapper[5052]: I1203 12:57:30.868384 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c98ad99e-c1f3-46d7-b810-a702c08308f3-ssh-key" (OuterVolumeSpecName: "ssh-key") pod "c98ad99e-c1f3-46d7-b810-a702c08308f3" (UID: "c98ad99e-c1f3-46d7-b810-a702c08308f3"). InnerVolumeSpecName "ssh-key". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:57:30 crc kubenswrapper[5052]: I1203 12:57:30.868567 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c98ad99e-c1f3-46d7-b810-a702c08308f3-inventory" (OuterVolumeSpecName: "inventory") pod "c98ad99e-c1f3-46d7-b810-a702c08308f3" (UID: "c98ad99e-c1f3-46d7-b810-a702c08308f3"). InnerVolumeSpecName "inventory". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:57:30 crc kubenswrapper[5052]: I1203 12:57:30.948841 5052 reconciler_common.go:293] "Volume detached for volume \"neutron-sriov-agent-neutron-config-0\" (UniqueName: \"kubernetes.io/secret/c98ad99e-c1f3-46d7-b810-a702c08308f3-neutron-sriov-agent-neutron-config-0\") on node \"crc\" DevicePath \"\"" Dec 03 12:57:30 crc kubenswrapper[5052]: I1203 12:57:30.948888 5052 reconciler_common.go:293] "Volume detached for volume \"neutron-sriov-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c98ad99e-c1f3-46d7-b810-a702c08308f3-neutron-sriov-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:57:30 crc kubenswrapper[5052]: I1203 12:57:30.948904 5052 reconciler_common.go:293] "Volume detached for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/c98ad99e-c1f3-46d7-b810-a702c08308f3-inventory\") on node \"crc\" DevicePath \"\"" Dec 03 12:57:30 crc kubenswrapper[5052]: I1203 12:57:30.948950 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-pvldd\" (UniqueName: \"kubernetes.io/projected/c98ad99e-c1f3-46d7-b810-a702c08308f3-kube-api-access-pvldd\") on node \"crc\" DevicePath \"\"" Dec 03 12:57:30 crc kubenswrapper[5052]: I1203 12:57:30.948962 5052 reconciler_common.go:293] "Volume detached for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/c98ad99e-c1f3-46d7-b810-a702c08308f3-ssh-key\") on node \"crc\" DevicePath \"\"" Dec 03 12:57:31 crc kubenswrapper[5052]: I1203 12:57:31.477198 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-sriov-edpm-compute-global-edpm-compute-global-7xq2m" event={"ID":"c98ad99e-c1f3-46d7-b810-a702c08308f3","Type":"ContainerDied","Data":"eec6fb0649444b604824839d8aeccc6543a76d8bbcf6915056d91713bf08a010"} Dec 03 12:57:31 crc kubenswrapper[5052]: I1203 12:57:31.477247 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="eec6fb0649444b604824839d8aeccc6543a76d8bbcf6915056d91713bf08a010" Dec 03 12:57:31 crc kubenswrapper[5052]: I1203 12:57:31.477280 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-sriov-edpm-compute-global-edpm-compute-global-7xq2m" Dec 03 12:57:31 crc kubenswrapper[5052]: I1203 12:57:31.536781 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/neutron-dhcp-edpm-compute-global-edpm-compute-global-22xpf"] Dec 03 12:57:31 crc kubenswrapper[5052]: E1203 12:57:31.537118 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="c98ad99e-c1f3-46d7-b810-a702c08308f3" containerName="neutron-sriov-edpm-compute-global-edpm-compute-global" Dec 03 12:57:31 crc kubenswrapper[5052]: I1203 12:57:31.537136 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="c98ad99e-c1f3-46d7-b810-a702c08308f3" containerName="neutron-sriov-edpm-compute-global-edpm-compute-global" Dec 03 12:57:31 crc kubenswrapper[5052]: I1203 12:57:31.537279 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="c98ad99e-c1f3-46d7-b810-a702c08308f3" containerName="neutron-sriov-edpm-compute-global-edpm-compute-global" Dec 03 12:57:31 crc kubenswrapper[5052]: I1203 12:57:31.537790 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-dhcp-edpm-compute-global-edpm-compute-global-22xpf" Dec 03 12:57:31 crc kubenswrapper[5052]: I1203 12:57:31.541721 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"dataplanenodeset-edpm-compute-global" Dec 03 12:57:31 crc kubenswrapper[5052]: I1203 12:57:31.541915 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"openstack-aee-default-env" Dec 03 12:57:31 crc kubenswrapper[5052]: I1203 12:57:31.541925 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"dataplane-ansible-ssh-private-key-secret" Dec 03 12:57:31 crc kubenswrapper[5052]: I1203 12:57:31.542039 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"combined-ca-bundle" Dec 03 12:57:31 crc kubenswrapper[5052]: I1203 12:57:31.542131 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"neutron-dhcp-agent-neutron-config" Dec 03 12:57:31 crc kubenswrapper[5052]: I1203 12:57:31.545163 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"edpm-compute-global-dockercfg-p5rjx" Dec 03 12:57:31 crc kubenswrapper[5052]: I1203 12:57:31.549015 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-dhcp-edpm-compute-global-edpm-compute-global-22xpf"] Dec 03 12:57:31 crc kubenswrapper[5052]: I1203 12:57:31.555368 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/c7885c2e-f7d6-4cc3-8412-0cf80ebe7950-ssh-key\") pod \"neutron-dhcp-edpm-compute-global-edpm-compute-global-22xpf\" (UID: \"c7885c2e-f7d6-4cc3-8412-0cf80ebe7950\") " pod="openstack-kuttl-tests/neutron-dhcp-edpm-compute-global-edpm-compute-global-22xpf" Dec 03 12:57:31 crc kubenswrapper[5052]: I1203 12:57:31.555416 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"neutron-dhcp-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c7885c2e-f7d6-4cc3-8412-0cf80ebe7950-neutron-dhcp-combined-ca-bundle\") pod \"neutron-dhcp-edpm-compute-global-edpm-compute-global-22xpf\" (UID: \"c7885c2e-f7d6-4cc3-8412-0cf80ebe7950\") " pod="openstack-kuttl-tests/neutron-dhcp-edpm-compute-global-edpm-compute-global-22xpf" Dec 03 12:57:31 crc kubenswrapper[5052]: I1203 12:57:31.555463 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9nzmv\" (UniqueName: \"kubernetes.io/projected/c7885c2e-f7d6-4cc3-8412-0cf80ebe7950-kube-api-access-9nzmv\") pod \"neutron-dhcp-edpm-compute-global-edpm-compute-global-22xpf\" (UID: \"c7885c2e-f7d6-4cc3-8412-0cf80ebe7950\") " pod="openstack-kuttl-tests/neutron-dhcp-edpm-compute-global-edpm-compute-global-22xpf" Dec 03 12:57:31 crc kubenswrapper[5052]: I1203 12:57:31.555494 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/c7885c2e-f7d6-4cc3-8412-0cf80ebe7950-inventory\") pod \"neutron-dhcp-edpm-compute-global-edpm-compute-global-22xpf\" (UID: \"c7885c2e-f7d6-4cc3-8412-0cf80ebe7950\") " pod="openstack-kuttl-tests/neutron-dhcp-edpm-compute-global-edpm-compute-global-22xpf" Dec 03 12:57:31 crc kubenswrapper[5052]: I1203 12:57:31.555847 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"neutron-dhcp-agent-neutron-config-0\" (UniqueName: \"kubernetes.io/secret/c7885c2e-f7d6-4cc3-8412-0cf80ebe7950-neutron-dhcp-agent-neutron-config-0\") pod \"neutron-dhcp-edpm-compute-global-edpm-compute-global-22xpf\" (UID: \"c7885c2e-f7d6-4cc3-8412-0cf80ebe7950\") " pod="openstack-kuttl-tests/neutron-dhcp-edpm-compute-global-edpm-compute-global-22xpf" Dec 03 12:57:31 crc kubenswrapper[5052]: I1203 12:57:31.656552 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/c7885c2e-f7d6-4cc3-8412-0cf80ebe7950-ssh-key\") pod \"neutron-dhcp-edpm-compute-global-edpm-compute-global-22xpf\" (UID: \"c7885c2e-f7d6-4cc3-8412-0cf80ebe7950\") " pod="openstack-kuttl-tests/neutron-dhcp-edpm-compute-global-edpm-compute-global-22xpf" Dec 03 12:57:31 crc kubenswrapper[5052]: I1203 12:57:31.656602 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"neutron-dhcp-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c7885c2e-f7d6-4cc3-8412-0cf80ebe7950-neutron-dhcp-combined-ca-bundle\") pod \"neutron-dhcp-edpm-compute-global-edpm-compute-global-22xpf\" (UID: \"c7885c2e-f7d6-4cc3-8412-0cf80ebe7950\") " pod="openstack-kuttl-tests/neutron-dhcp-edpm-compute-global-edpm-compute-global-22xpf" Dec 03 12:57:31 crc kubenswrapper[5052]: I1203 12:57:31.656644 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-9nzmv\" (UniqueName: \"kubernetes.io/projected/c7885c2e-f7d6-4cc3-8412-0cf80ebe7950-kube-api-access-9nzmv\") pod \"neutron-dhcp-edpm-compute-global-edpm-compute-global-22xpf\" (UID: \"c7885c2e-f7d6-4cc3-8412-0cf80ebe7950\") " pod="openstack-kuttl-tests/neutron-dhcp-edpm-compute-global-edpm-compute-global-22xpf" Dec 03 12:57:31 crc kubenswrapper[5052]: I1203 12:57:31.656672 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/c7885c2e-f7d6-4cc3-8412-0cf80ebe7950-inventory\") pod \"neutron-dhcp-edpm-compute-global-edpm-compute-global-22xpf\" (UID: \"c7885c2e-f7d6-4cc3-8412-0cf80ebe7950\") " pod="openstack-kuttl-tests/neutron-dhcp-edpm-compute-global-edpm-compute-global-22xpf" Dec 03 12:57:31 crc kubenswrapper[5052]: I1203 12:57:31.656694 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"neutron-dhcp-agent-neutron-config-0\" (UniqueName: \"kubernetes.io/secret/c7885c2e-f7d6-4cc3-8412-0cf80ebe7950-neutron-dhcp-agent-neutron-config-0\") pod \"neutron-dhcp-edpm-compute-global-edpm-compute-global-22xpf\" (UID: \"c7885c2e-f7d6-4cc3-8412-0cf80ebe7950\") " pod="openstack-kuttl-tests/neutron-dhcp-edpm-compute-global-edpm-compute-global-22xpf" Dec 03 12:57:31 crc kubenswrapper[5052]: I1203 12:57:31.660106 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/c7885c2e-f7d6-4cc3-8412-0cf80ebe7950-inventory\") pod \"neutron-dhcp-edpm-compute-global-edpm-compute-global-22xpf\" (UID: \"c7885c2e-f7d6-4cc3-8412-0cf80ebe7950\") " pod="openstack-kuttl-tests/neutron-dhcp-edpm-compute-global-edpm-compute-global-22xpf" Dec 03 12:57:31 crc kubenswrapper[5052]: I1203 12:57:31.660138 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"neutron-dhcp-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c7885c2e-f7d6-4cc3-8412-0cf80ebe7950-neutron-dhcp-combined-ca-bundle\") pod \"neutron-dhcp-edpm-compute-global-edpm-compute-global-22xpf\" (UID: \"c7885c2e-f7d6-4cc3-8412-0cf80ebe7950\") " pod="openstack-kuttl-tests/neutron-dhcp-edpm-compute-global-edpm-compute-global-22xpf" Dec 03 12:57:31 crc kubenswrapper[5052]: I1203 12:57:31.660280 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/c7885c2e-f7d6-4cc3-8412-0cf80ebe7950-ssh-key\") pod \"neutron-dhcp-edpm-compute-global-edpm-compute-global-22xpf\" (UID: \"c7885c2e-f7d6-4cc3-8412-0cf80ebe7950\") " pod="openstack-kuttl-tests/neutron-dhcp-edpm-compute-global-edpm-compute-global-22xpf" Dec 03 12:57:31 crc kubenswrapper[5052]: I1203 12:57:31.667006 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"neutron-dhcp-agent-neutron-config-0\" (UniqueName: \"kubernetes.io/secret/c7885c2e-f7d6-4cc3-8412-0cf80ebe7950-neutron-dhcp-agent-neutron-config-0\") pod \"neutron-dhcp-edpm-compute-global-edpm-compute-global-22xpf\" (UID: \"c7885c2e-f7d6-4cc3-8412-0cf80ebe7950\") " pod="openstack-kuttl-tests/neutron-dhcp-edpm-compute-global-edpm-compute-global-22xpf" Dec 03 12:57:31 crc kubenswrapper[5052]: I1203 12:57:31.675919 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-9nzmv\" (UniqueName: \"kubernetes.io/projected/c7885c2e-f7d6-4cc3-8412-0cf80ebe7950-kube-api-access-9nzmv\") pod \"neutron-dhcp-edpm-compute-global-edpm-compute-global-22xpf\" (UID: \"c7885c2e-f7d6-4cc3-8412-0cf80ebe7950\") " pod="openstack-kuttl-tests/neutron-dhcp-edpm-compute-global-edpm-compute-global-22xpf" Dec 03 12:57:31 crc kubenswrapper[5052]: I1203 12:57:31.854930 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-dhcp-edpm-compute-global-edpm-compute-global-22xpf" Dec 03 12:57:32 crc kubenswrapper[5052]: I1203 12:57:32.263913 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-dhcp-edpm-compute-global-edpm-compute-global-22xpf"] Dec 03 12:57:32 crc kubenswrapper[5052]: I1203 12:57:32.277106 5052 provider.go:102] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Dec 03 12:57:32 crc kubenswrapper[5052]: I1203 12:57:32.486914 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-dhcp-edpm-compute-global-edpm-compute-global-22xpf" event={"ID":"c7885c2e-f7d6-4cc3-8412-0cf80ebe7950","Type":"ContainerStarted","Data":"4cf00f77ff3620ce2e67549422bb7e6aee7de51c93bf18553a47b7070a7f2668"} Dec 03 12:57:33 crc kubenswrapper[5052]: I1203 12:57:33.673255 5052 scope.go:117] "RemoveContainer" containerID="69fa7fbcd2aceb7c53703a19508876f93a97dc9adabb79c16fb691d90f606d58" Dec 03 12:57:33 crc kubenswrapper[5052]: E1203 12:57:33.673836 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 12:57:34 crc kubenswrapper[5052]: I1203 12:57:34.504188 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-dhcp-edpm-compute-global-edpm-compute-global-22xpf" event={"ID":"c7885c2e-f7d6-4cc3-8412-0cf80ebe7950","Type":"ContainerStarted","Data":"0e6edd2dc9f4b852bedb05e9840478c19ae05171f6960f1774e3c9cc2d2112ce"} Dec 03 12:57:34 crc kubenswrapper[5052]: I1203 12:57:34.529549 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/neutron-dhcp-edpm-compute-global-edpm-compute-global-22xpf" podStartSLOduration=1.902191496 podStartE2EDuration="3.529483643s" podCreationTimestamp="2025-12-03 12:57:31 +0000 UTC" firstStartedPulling="2025-12-03 12:57:32.276868163 +0000 UTC m=+5842.865109927" lastFinishedPulling="2025-12-03 12:57:33.90416031 +0000 UTC m=+5844.492402074" observedRunningTime="2025-12-03 12:57:34.524523441 +0000 UTC m=+5845.112765205" watchObservedRunningTime="2025-12-03 12:57:34.529483643 +0000 UTC m=+5845.117725407" Dec 03 12:57:35 crc kubenswrapper[5052]: I1203 12:57:35.514463 5052 generic.go:334] "Generic (PLEG): container finished" podID="c7885c2e-f7d6-4cc3-8412-0cf80ebe7950" containerID="0e6edd2dc9f4b852bedb05e9840478c19ae05171f6960f1774e3c9cc2d2112ce" exitCode=0 Dec 03 12:57:35 crc kubenswrapper[5052]: I1203 12:57:35.514501 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-dhcp-edpm-compute-global-edpm-compute-global-22xpf" event={"ID":"c7885c2e-f7d6-4cc3-8412-0cf80ebe7950","Type":"ContainerDied","Data":"0e6edd2dc9f4b852bedb05e9840478c19ae05171f6960f1774e3c9cc2d2112ce"} Dec 03 12:57:36 crc kubenswrapper[5052]: I1203 12:57:36.793328 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-dhcp-edpm-compute-global-edpm-compute-global-22xpf" Dec 03 12:57:36 crc kubenswrapper[5052]: I1203 12:57:36.933560 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/c7885c2e-f7d6-4cc3-8412-0cf80ebe7950-inventory\") pod \"c7885c2e-f7d6-4cc3-8412-0cf80ebe7950\" (UID: \"c7885c2e-f7d6-4cc3-8412-0cf80ebe7950\") " Dec 03 12:57:36 crc kubenswrapper[5052]: I1203 12:57:36.933680 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9nzmv\" (UniqueName: \"kubernetes.io/projected/c7885c2e-f7d6-4cc3-8412-0cf80ebe7950-kube-api-access-9nzmv\") pod \"c7885c2e-f7d6-4cc3-8412-0cf80ebe7950\" (UID: \"c7885c2e-f7d6-4cc3-8412-0cf80ebe7950\") " Dec 03 12:57:36 crc kubenswrapper[5052]: I1203 12:57:36.933726 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/c7885c2e-f7d6-4cc3-8412-0cf80ebe7950-ssh-key\") pod \"c7885c2e-f7d6-4cc3-8412-0cf80ebe7950\" (UID: \"c7885c2e-f7d6-4cc3-8412-0cf80ebe7950\") " Dec 03 12:57:36 crc kubenswrapper[5052]: I1203 12:57:36.933843 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"neutron-dhcp-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c7885c2e-f7d6-4cc3-8412-0cf80ebe7950-neutron-dhcp-combined-ca-bundle\") pod \"c7885c2e-f7d6-4cc3-8412-0cf80ebe7950\" (UID: \"c7885c2e-f7d6-4cc3-8412-0cf80ebe7950\") " Dec 03 12:57:36 crc kubenswrapper[5052]: I1203 12:57:36.933904 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"neutron-dhcp-agent-neutron-config-0\" (UniqueName: \"kubernetes.io/secret/c7885c2e-f7d6-4cc3-8412-0cf80ebe7950-neutron-dhcp-agent-neutron-config-0\") pod \"c7885c2e-f7d6-4cc3-8412-0cf80ebe7950\" (UID: \"c7885c2e-f7d6-4cc3-8412-0cf80ebe7950\") " Dec 03 12:57:36 crc kubenswrapper[5052]: I1203 12:57:36.939551 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c7885c2e-f7d6-4cc3-8412-0cf80ebe7950-neutron-dhcp-combined-ca-bundle" (OuterVolumeSpecName: "neutron-dhcp-combined-ca-bundle") pod "c7885c2e-f7d6-4cc3-8412-0cf80ebe7950" (UID: "c7885c2e-f7d6-4cc3-8412-0cf80ebe7950"). InnerVolumeSpecName "neutron-dhcp-combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:57:36 crc kubenswrapper[5052]: I1203 12:57:36.939571 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c7885c2e-f7d6-4cc3-8412-0cf80ebe7950-kube-api-access-9nzmv" (OuterVolumeSpecName: "kube-api-access-9nzmv") pod "c7885c2e-f7d6-4cc3-8412-0cf80ebe7950" (UID: "c7885c2e-f7d6-4cc3-8412-0cf80ebe7950"). InnerVolumeSpecName "kube-api-access-9nzmv". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:57:36 crc kubenswrapper[5052]: I1203 12:57:36.959763 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c7885c2e-f7d6-4cc3-8412-0cf80ebe7950-ssh-key" (OuterVolumeSpecName: "ssh-key") pod "c7885c2e-f7d6-4cc3-8412-0cf80ebe7950" (UID: "c7885c2e-f7d6-4cc3-8412-0cf80ebe7950"). InnerVolumeSpecName "ssh-key". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:57:36 crc kubenswrapper[5052]: I1203 12:57:36.961235 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c7885c2e-f7d6-4cc3-8412-0cf80ebe7950-neutron-dhcp-agent-neutron-config-0" (OuterVolumeSpecName: "neutron-dhcp-agent-neutron-config-0") pod "c7885c2e-f7d6-4cc3-8412-0cf80ebe7950" (UID: "c7885c2e-f7d6-4cc3-8412-0cf80ebe7950"). InnerVolumeSpecName "neutron-dhcp-agent-neutron-config-0". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:57:36 crc kubenswrapper[5052]: I1203 12:57:36.964987 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c7885c2e-f7d6-4cc3-8412-0cf80ebe7950-inventory" (OuterVolumeSpecName: "inventory") pod "c7885c2e-f7d6-4cc3-8412-0cf80ebe7950" (UID: "c7885c2e-f7d6-4cc3-8412-0cf80ebe7950"). InnerVolumeSpecName "inventory". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:57:37 crc kubenswrapper[5052]: I1203 12:57:37.035751 5052 reconciler_common.go:293] "Volume detached for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/c7885c2e-f7d6-4cc3-8412-0cf80ebe7950-inventory\") on node \"crc\" DevicePath \"\"" Dec 03 12:57:37 crc kubenswrapper[5052]: I1203 12:57:37.035818 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-9nzmv\" (UniqueName: \"kubernetes.io/projected/c7885c2e-f7d6-4cc3-8412-0cf80ebe7950-kube-api-access-9nzmv\") on node \"crc\" DevicePath \"\"" Dec 03 12:57:37 crc kubenswrapper[5052]: I1203 12:57:37.035833 5052 reconciler_common.go:293] "Volume detached for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/c7885c2e-f7d6-4cc3-8412-0cf80ebe7950-ssh-key\") on node \"crc\" DevicePath \"\"" Dec 03 12:57:37 crc kubenswrapper[5052]: I1203 12:57:37.035846 5052 reconciler_common.go:293] "Volume detached for volume \"neutron-dhcp-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c7885c2e-f7d6-4cc3-8412-0cf80ebe7950-neutron-dhcp-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:57:37 crc kubenswrapper[5052]: I1203 12:57:37.035884 5052 reconciler_common.go:293] "Volume detached for volume \"neutron-dhcp-agent-neutron-config-0\" (UniqueName: \"kubernetes.io/secret/c7885c2e-f7d6-4cc3-8412-0cf80ebe7950-neutron-dhcp-agent-neutron-config-0\") on node \"crc\" DevicePath \"\"" Dec 03 12:57:37 crc kubenswrapper[5052]: I1203 12:57:37.533730 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-dhcp-edpm-compute-global-edpm-compute-global-22xpf" event={"ID":"c7885c2e-f7d6-4cc3-8412-0cf80ebe7950","Type":"ContainerDied","Data":"4cf00f77ff3620ce2e67549422bb7e6aee7de51c93bf18553a47b7070a7f2668"} Dec 03 12:57:37 crc kubenswrapper[5052]: I1203 12:57:37.533761 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-dhcp-edpm-compute-global-edpm-compute-global-22xpf" Dec 03 12:57:37 crc kubenswrapper[5052]: I1203 12:57:37.533778 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4cf00f77ff3620ce2e67549422bb7e6aee7de51c93bf18553a47b7070a7f2668" Dec 03 12:57:37 crc kubenswrapper[5052]: I1203 12:57:37.595287 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/libvirt-edpm-compute-global-edpm-compute-global-29fbj"] Dec 03 12:57:37 crc kubenswrapper[5052]: E1203 12:57:37.596002 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="c7885c2e-f7d6-4cc3-8412-0cf80ebe7950" containerName="neutron-dhcp-edpm-compute-global-edpm-compute-global" Dec 03 12:57:37 crc kubenswrapper[5052]: I1203 12:57:37.596031 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="c7885c2e-f7d6-4cc3-8412-0cf80ebe7950" containerName="neutron-dhcp-edpm-compute-global-edpm-compute-global" Dec 03 12:57:37 crc kubenswrapper[5052]: I1203 12:57:37.596235 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="c7885c2e-f7d6-4cc3-8412-0cf80ebe7950" containerName="neutron-dhcp-edpm-compute-global-edpm-compute-global" Dec 03 12:57:37 crc kubenswrapper[5052]: I1203 12:57:37.596886 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/libvirt-edpm-compute-global-edpm-compute-global-29fbj" Dec 03 12:57:37 crc kubenswrapper[5052]: I1203 12:57:37.599539 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"dataplane-ansible-ssh-private-key-secret" Dec 03 12:57:37 crc kubenswrapper[5052]: I1203 12:57:37.599847 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"edpm-compute-global-dockercfg-p5rjx" Dec 03 12:57:37 crc kubenswrapper[5052]: I1203 12:57:37.599950 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"openstack-aee-default-env" Dec 03 12:57:37 crc kubenswrapper[5052]: I1203 12:57:37.599983 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"combined-ca-bundle" Dec 03 12:57:37 crc kubenswrapper[5052]: I1203 12:57:37.600054 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"libvirt-secret" Dec 03 12:57:37 crc kubenswrapper[5052]: I1203 12:57:37.600323 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"dataplanenodeset-edpm-compute-global" Dec 03 12:57:37 crc kubenswrapper[5052]: I1203 12:57:37.608644 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/libvirt-edpm-compute-global-edpm-compute-global-29fbj"] Dec 03 12:57:37 crc kubenswrapper[5052]: I1203 12:57:37.746137 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/a5fbd1bc-e091-459b-97fa-17d728070100-inventory\") pod \"libvirt-edpm-compute-global-edpm-compute-global-29fbj\" (UID: \"a5fbd1bc-e091-459b-97fa-17d728070100\") " pod="openstack-kuttl-tests/libvirt-edpm-compute-global-edpm-compute-global-29fbj" Dec 03 12:57:37 crc kubenswrapper[5052]: I1203 12:57:37.746200 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"libvirt-secret-0\" (UniqueName: \"kubernetes.io/secret/a5fbd1bc-e091-459b-97fa-17d728070100-libvirt-secret-0\") pod \"libvirt-edpm-compute-global-edpm-compute-global-29fbj\" (UID: \"a5fbd1bc-e091-459b-97fa-17d728070100\") " pod="openstack-kuttl-tests/libvirt-edpm-compute-global-edpm-compute-global-29fbj" Dec 03 12:57:37 crc kubenswrapper[5052]: I1203 12:57:37.746345 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rwlsc\" (UniqueName: \"kubernetes.io/projected/a5fbd1bc-e091-459b-97fa-17d728070100-kube-api-access-rwlsc\") pod \"libvirt-edpm-compute-global-edpm-compute-global-29fbj\" (UID: \"a5fbd1bc-e091-459b-97fa-17d728070100\") " pod="openstack-kuttl-tests/libvirt-edpm-compute-global-edpm-compute-global-29fbj" Dec 03 12:57:37 crc kubenswrapper[5052]: I1203 12:57:37.746402 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"libvirt-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a5fbd1bc-e091-459b-97fa-17d728070100-libvirt-combined-ca-bundle\") pod \"libvirt-edpm-compute-global-edpm-compute-global-29fbj\" (UID: \"a5fbd1bc-e091-459b-97fa-17d728070100\") " pod="openstack-kuttl-tests/libvirt-edpm-compute-global-edpm-compute-global-29fbj" Dec 03 12:57:37 crc kubenswrapper[5052]: I1203 12:57:37.746444 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/a5fbd1bc-e091-459b-97fa-17d728070100-ssh-key\") pod \"libvirt-edpm-compute-global-edpm-compute-global-29fbj\" (UID: \"a5fbd1bc-e091-459b-97fa-17d728070100\") " pod="openstack-kuttl-tests/libvirt-edpm-compute-global-edpm-compute-global-29fbj" Dec 03 12:57:37 crc kubenswrapper[5052]: I1203 12:57:37.847569 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/a5fbd1bc-e091-459b-97fa-17d728070100-inventory\") pod \"libvirt-edpm-compute-global-edpm-compute-global-29fbj\" (UID: \"a5fbd1bc-e091-459b-97fa-17d728070100\") " pod="openstack-kuttl-tests/libvirt-edpm-compute-global-edpm-compute-global-29fbj" Dec 03 12:57:37 crc kubenswrapper[5052]: I1203 12:57:37.848242 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"libvirt-secret-0\" (UniqueName: \"kubernetes.io/secret/a5fbd1bc-e091-459b-97fa-17d728070100-libvirt-secret-0\") pod \"libvirt-edpm-compute-global-edpm-compute-global-29fbj\" (UID: \"a5fbd1bc-e091-459b-97fa-17d728070100\") " pod="openstack-kuttl-tests/libvirt-edpm-compute-global-edpm-compute-global-29fbj" Dec 03 12:57:37 crc kubenswrapper[5052]: I1203 12:57:37.848307 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-rwlsc\" (UniqueName: \"kubernetes.io/projected/a5fbd1bc-e091-459b-97fa-17d728070100-kube-api-access-rwlsc\") pod \"libvirt-edpm-compute-global-edpm-compute-global-29fbj\" (UID: \"a5fbd1bc-e091-459b-97fa-17d728070100\") " pod="openstack-kuttl-tests/libvirt-edpm-compute-global-edpm-compute-global-29fbj" Dec 03 12:57:37 crc kubenswrapper[5052]: I1203 12:57:37.848331 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"libvirt-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a5fbd1bc-e091-459b-97fa-17d728070100-libvirt-combined-ca-bundle\") pod \"libvirt-edpm-compute-global-edpm-compute-global-29fbj\" (UID: \"a5fbd1bc-e091-459b-97fa-17d728070100\") " pod="openstack-kuttl-tests/libvirt-edpm-compute-global-edpm-compute-global-29fbj" Dec 03 12:57:37 crc kubenswrapper[5052]: I1203 12:57:37.848354 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/a5fbd1bc-e091-459b-97fa-17d728070100-ssh-key\") pod \"libvirt-edpm-compute-global-edpm-compute-global-29fbj\" (UID: \"a5fbd1bc-e091-459b-97fa-17d728070100\") " pod="openstack-kuttl-tests/libvirt-edpm-compute-global-edpm-compute-global-29fbj" Dec 03 12:57:37 crc kubenswrapper[5052]: I1203 12:57:37.852436 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/a5fbd1bc-e091-459b-97fa-17d728070100-inventory\") pod \"libvirt-edpm-compute-global-edpm-compute-global-29fbj\" (UID: \"a5fbd1bc-e091-459b-97fa-17d728070100\") " pod="openstack-kuttl-tests/libvirt-edpm-compute-global-edpm-compute-global-29fbj" Dec 03 12:57:37 crc kubenswrapper[5052]: I1203 12:57:37.852958 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/a5fbd1bc-e091-459b-97fa-17d728070100-ssh-key\") pod \"libvirt-edpm-compute-global-edpm-compute-global-29fbj\" (UID: \"a5fbd1bc-e091-459b-97fa-17d728070100\") " pod="openstack-kuttl-tests/libvirt-edpm-compute-global-edpm-compute-global-29fbj" Dec 03 12:57:37 crc kubenswrapper[5052]: I1203 12:57:37.853046 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"libvirt-secret-0\" (UniqueName: \"kubernetes.io/secret/a5fbd1bc-e091-459b-97fa-17d728070100-libvirt-secret-0\") pod \"libvirt-edpm-compute-global-edpm-compute-global-29fbj\" (UID: \"a5fbd1bc-e091-459b-97fa-17d728070100\") " pod="openstack-kuttl-tests/libvirt-edpm-compute-global-edpm-compute-global-29fbj" Dec 03 12:57:37 crc kubenswrapper[5052]: I1203 12:57:37.853688 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"libvirt-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a5fbd1bc-e091-459b-97fa-17d728070100-libvirt-combined-ca-bundle\") pod \"libvirt-edpm-compute-global-edpm-compute-global-29fbj\" (UID: \"a5fbd1bc-e091-459b-97fa-17d728070100\") " pod="openstack-kuttl-tests/libvirt-edpm-compute-global-edpm-compute-global-29fbj" Dec 03 12:57:37 crc kubenswrapper[5052]: I1203 12:57:37.864672 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-rwlsc\" (UniqueName: \"kubernetes.io/projected/a5fbd1bc-e091-459b-97fa-17d728070100-kube-api-access-rwlsc\") pod \"libvirt-edpm-compute-global-edpm-compute-global-29fbj\" (UID: \"a5fbd1bc-e091-459b-97fa-17d728070100\") " pod="openstack-kuttl-tests/libvirt-edpm-compute-global-edpm-compute-global-29fbj" Dec 03 12:57:37 crc kubenswrapper[5052]: I1203 12:57:37.919027 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/libvirt-edpm-compute-global-edpm-compute-global-29fbj" Dec 03 12:57:37 crc kubenswrapper[5052]: I1203 12:57:37.931574 5052 scope.go:117] "RemoveContainer" containerID="892e9cbb41eca0f11cf27ace38277c207623a5c2ba56eb827b340a0fc3b17b83" Dec 03 12:57:37 crc kubenswrapper[5052]: I1203 12:57:37.965062 5052 scope.go:117] "RemoveContainer" containerID="4cc0973d6be860027762d01ac994ad7b9ec6e3602b0a54d18cb4bc9da4071fbb" Dec 03 12:57:38 crc kubenswrapper[5052]: I1203 12:57:38.019488 5052 scope.go:117] "RemoveContainer" containerID="906c443738f38d4f8d4282c9fd33ffbebb16c2aec1b556f25ccc7a7e8c6e748c" Dec 03 12:57:38 crc kubenswrapper[5052]: I1203 12:57:38.045241 5052 scope.go:117] "RemoveContainer" containerID="4b943db92344d3538d476fedbfe7197f925dbe4641448a2e28f5c8d79d9ed4b5" Dec 03 12:57:38 crc kubenswrapper[5052]: I1203 12:57:38.074602 5052 scope.go:117] "RemoveContainer" containerID="327f0066a0601f7b4c70c175a1a3262b3ce3a6c6fc204b7124a0b69083969d6f" Dec 03 12:57:38 crc kubenswrapper[5052]: I1203 12:57:38.095416 5052 scope.go:117] "RemoveContainer" containerID="9753117b9d1d7a9e8974b3b7100aceebac6f8ff8d17dbee469702e6582413ceb" Dec 03 12:57:38 crc kubenswrapper[5052]: I1203 12:57:38.122247 5052 scope.go:117] "RemoveContainer" containerID="b78075e349099e5081c2a933e5a387e49bcc03246f78f07abd5d255f951c6969" Dec 03 12:57:38 crc kubenswrapper[5052]: I1203 12:57:38.164674 5052 scope.go:117] "RemoveContainer" containerID="8fb1ea412deb631d1ed8163bda6feae84ba3cd74f0818f033958f4a2a30de564" Dec 03 12:57:38 crc kubenswrapper[5052]: I1203 12:57:38.181495 5052 scope.go:117] "RemoveContainer" containerID="174dc76c1d02fad9f57931bbb0c9542148e337ada369ca2b722e53cd15f06251" Dec 03 12:57:38 crc kubenswrapper[5052]: I1203 12:57:38.213127 5052 scope.go:117] "RemoveContainer" containerID="06f463922e62266ed96092189ec4fb9925ca28e41dfc81fbad694a526afdc991" Dec 03 12:57:38 crc kubenswrapper[5052]: I1203 12:57:38.231942 5052 scope.go:117] "RemoveContainer" containerID="018816f9a78e7b2315d8c4e733edf99eeec2819f8f83cbe3fa3669341863f8d1" Dec 03 12:57:38 crc kubenswrapper[5052]: I1203 12:57:38.251029 5052 scope.go:117] "RemoveContainer" containerID="c424cb522d3b69b3e111498d4aa3be9ef046469f0fdba589af66d4e85ff8160a" Dec 03 12:57:38 crc kubenswrapper[5052]: I1203 12:57:38.314609 5052 scope.go:117] "RemoveContainer" containerID="99aeae0bb50f3692e222705b7174440017ada583bd010ad7e324069e888cc242" Dec 03 12:57:38 crc kubenswrapper[5052]: I1203 12:57:38.332311 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/libvirt-edpm-compute-global-edpm-compute-global-29fbj"] Dec 03 12:57:38 crc kubenswrapper[5052]: W1203 12:57:38.347601 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda5fbd1bc_e091_459b_97fa_17d728070100.slice/crio-f3fcd4f86feef8d8dbb6ab041f4f4b2619ce466958c255a03e5b930c08997007 WatchSource:0}: Error finding container f3fcd4f86feef8d8dbb6ab041f4f4b2619ce466958c255a03e5b930c08997007: Status 404 returned error can't find the container with id f3fcd4f86feef8d8dbb6ab041f4f4b2619ce466958c255a03e5b930c08997007 Dec 03 12:57:38 crc kubenswrapper[5052]: I1203 12:57:38.358721 5052 scope.go:117] "RemoveContainer" containerID="405a9b22b01a4f49fa4335e876211ec12001bfaed5b30631c5e1d909cd57aa18" Dec 03 12:57:38 crc kubenswrapper[5052]: I1203 12:57:38.379073 5052 scope.go:117] "RemoveContainer" containerID="9a5124afa6da140d0ad66a43ce6d9f495f3b80943f462de57cae97790359ac58" Dec 03 12:57:38 crc kubenswrapper[5052]: I1203 12:57:38.404154 5052 scope.go:117] "RemoveContainer" containerID="3049ef615fa201f9bcee00bec5b96cdf452b84cc54465cb8739a8fe34c22f4f6" Dec 03 12:57:38 crc kubenswrapper[5052]: I1203 12:57:38.427670 5052 scope.go:117] "RemoveContainer" containerID="ac4ecfa5627541f93acf9cc63e185d94bb2e833e592e7703865bb5d4026f322f" Dec 03 12:57:38 crc kubenswrapper[5052]: I1203 12:57:38.450712 5052 scope.go:117] "RemoveContainer" containerID="bca28d625281627cc7675e3115c216e4fdf5af87f408df91e6006b4d6c37f376" Dec 03 12:57:38 crc kubenswrapper[5052]: I1203 12:57:38.467365 5052 scope.go:117] "RemoveContainer" containerID="262d5eb89c22ac292d99479055ae3d43a813219c6939b2f60660b7ee0bf8ae6d" Dec 03 12:57:38 crc kubenswrapper[5052]: I1203 12:57:38.485073 5052 scope.go:117] "RemoveContainer" containerID="1a5bf9dc6916f04b426e2efc9f68b9c5e9ea7f21f2b45e2410c8d21dce7a7067" Dec 03 12:57:38 crc kubenswrapper[5052]: I1203 12:57:38.510087 5052 scope.go:117] "RemoveContainer" containerID="b4f45322078c29bf05e1e6c5ce39ffbdb6524a51a8a145e7edb2b7b7325e566f" Dec 03 12:57:38 crc kubenswrapper[5052]: I1203 12:57:38.542274 5052 scope.go:117] "RemoveContainer" containerID="4c38a9546e116479167e49dc563abec492318ccfbe7e54b5db501a8e8665f5aa" Dec 03 12:57:38 crc kubenswrapper[5052]: I1203 12:57:38.555278 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/libvirt-edpm-compute-global-edpm-compute-global-29fbj" event={"ID":"a5fbd1bc-e091-459b-97fa-17d728070100","Type":"ContainerStarted","Data":"f3fcd4f86feef8d8dbb6ab041f4f4b2619ce466958c255a03e5b930c08997007"} Dec 03 12:57:39 crc kubenswrapper[5052]: I1203 12:57:39.581420 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/libvirt-edpm-compute-global-edpm-compute-global-29fbj" event={"ID":"a5fbd1bc-e091-459b-97fa-17d728070100","Type":"ContainerStarted","Data":"dd30be37e0e72ae0ce20ff47e803bd47943e27342d32ad9fb8f062ba7236c0dc"} Dec 03 12:57:39 crc kubenswrapper[5052]: I1203 12:57:39.614572 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/libvirt-edpm-compute-global-edpm-compute-global-29fbj" podStartSLOduration=2.168676996 podStartE2EDuration="2.614552037s" podCreationTimestamp="2025-12-03 12:57:37 +0000 UTC" firstStartedPulling="2025-12-03 12:57:38.359073082 +0000 UTC m=+5848.947314846" lastFinishedPulling="2025-12-03 12:57:38.804948123 +0000 UTC m=+5849.393189887" observedRunningTime="2025-12-03 12:57:39.601276997 +0000 UTC m=+5850.189518751" watchObservedRunningTime="2025-12-03 12:57:39.614552037 +0000 UTC m=+5850.202793811" Dec 03 12:57:40 crc kubenswrapper[5052]: I1203 12:57:40.591814 5052 generic.go:334] "Generic (PLEG): container finished" podID="a5fbd1bc-e091-459b-97fa-17d728070100" containerID="dd30be37e0e72ae0ce20ff47e803bd47943e27342d32ad9fb8f062ba7236c0dc" exitCode=0 Dec 03 12:57:40 crc kubenswrapper[5052]: I1203 12:57:40.591862 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/libvirt-edpm-compute-global-edpm-compute-global-29fbj" event={"ID":"a5fbd1bc-e091-459b-97fa-17d728070100","Type":"ContainerDied","Data":"dd30be37e0e72ae0ce20ff47e803bd47943e27342d32ad9fb8f062ba7236c0dc"} Dec 03 12:57:41 crc kubenswrapper[5052]: I1203 12:57:41.892072 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/libvirt-edpm-compute-global-edpm-compute-global-29fbj" Dec 03 12:57:42 crc kubenswrapper[5052]: I1203 12:57:42.001914 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"libvirt-secret-0\" (UniqueName: \"kubernetes.io/secret/a5fbd1bc-e091-459b-97fa-17d728070100-libvirt-secret-0\") pod \"a5fbd1bc-e091-459b-97fa-17d728070100\" (UID: \"a5fbd1bc-e091-459b-97fa-17d728070100\") " Dec 03 12:57:42 crc kubenswrapper[5052]: I1203 12:57:42.001989 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"libvirt-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a5fbd1bc-e091-459b-97fa-17d728070100-libvirt-combined-ca-bundle\") pod \"a5fbd1bc-e091-459b-97fa-17d728070100\" (UID: \"a5fbd1bc-e091-459b-97fa-17d728070100\") " Dec 03 12:57:42 crc kubenswrapper[5052]: I1203 12:57:42.002041 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/a5fbd1bc-e091-459b-97fa-17d728070100-ssh-key\") pod \"a5fbd1bc-e091-459b-97fa-17d728070100\" (UID: \"a5fbd1bc-e091-459b-97fa-17d728070100\") " Dec 03 12:57:42 crc kubenswrapper[5052]: I1203 12:57:42.002069 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/a5fbd1bc-e091-459b-97fa-17d728070100-inventory\") pod \"a5fbd1bc-e091-459b-97fa-17d728070100\" (UID: \"a5fbd1bc-e091-459b-97fa-17d728070100\") " Dec 03 12:57:42 crc kubenswrapper[5052]: I1203 12:57:42.002103 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rwlsc\" (UniqueName: \"kubernetes.io/projected/a5fbd1bc-e091-459b-97fa-17d728070100-kube-api-access-rwlsc\") pod \"a5fbd1bc-e091-459b-97fa-17d728070100\" (UID: \"a5fbd1bc-e091-459b-97fa-17d728070100\") " Dec 03 12:57:42 crc kubenswrapper[5052]: I1203 12:57:42.007429 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a5fbd1bc-e091-459b-97fa-17d728070100-libvirt-combined-ca-bundle" (OuterVolumeSpecName: "libvirt-combined-ca-bundle") pod "a5fbd1bc-e091-459b-97fa-17d728070100" (UID: "a5fbd1bc-e091-459b-97fa-17d728070100"). InnerVolumeSpecName "libvirt-combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:57:42 crc kubenswrapper[5052]: I1203 12:57:42.007446 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a5fbd1bc-e091-459b-97fa-17d728070100-kube-api-access-rwlsc" (OuterVolumeSpecName: "kube-api-access-rwlsc") pod "a5fbd1bc-e091-459b-97fa-17d728070100" (UID: "a5fbd1bc-e091-459b-97fa-17d728070100"). InnerVolumeSpecName "kube-api-access-rwlsc". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:57:42 crc kubenswrapper[5052]: I1203 12:57:42.023263 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a5fbd1bc-e091-459b-97fa-17d728070100-inventory" (OuterVolumeSpecName: "inventory") pod "a5fbd1bc-e091-459b-97fa-17d728070100" (UID: "a5fbd1bc-e091-459b-97fa-17d728070100"). InnerVolumeSpecName "inventory". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:57:42 crc kubenswrapper[5052]: I1203 12:57:42.024140 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a5fbd1bc-e091-459b-97fa-17d728070100-libvirt-secret-0" (OuterVolumeSpecName: "libvirt-secret-0") pod "a5fbd1bc-e091-459b-97fa-17d728070100" (UID: "a5fbd1bc-e091-459b-97fa-17d728070100"). InnerVolumeSpecName "libvirt-secret-0". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:57:42 crc kubenswrapper[5052]: I1203 12:57:42.024410 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a5fbd1bc-e091-459b-97fa-17d728070100-ssh-key" (OuterVolumeSpecName: "ssh-key") pod "a5fbd1bc-e091-459b-97fa-17d728070100" (UID: "a5fbd1bc-e091-459b-97fa-17d728070100"). InnerVolumeSpecName "ssh-key". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:57:42 crc kubenswrapper[5052]: I1203 12:57:42.103684 5052 reconciler_common.go:293] "Volume detached for volume \"libvirt-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a5fbd1bc-e091-459b-97fa-17d728070100-libvirt-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:57:42 crc kubenswrapper[5052]: I1203 12:57:42.103720 5052 reconciler_common.go:293] "Volume detached for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/a5fbd1bc-e091-459b-97fa-17d728070100-ssh-key\") on node \"crc\" DevicePath \"\"" Dec 03 12:57:42 crc kubenswrapper[5052]: I1203 12:57:42.103733 5052 reconciler_common.go:293] "Volume detached for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/a5fbd1bc-e091-459b-97fa-17d728070100-inventory\") on node \"crc\" DevicePath \"\"" Dec 03 12:57:42 crc kubenswrapper[5052]: I1203 12:57:42.103748 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-rwlsc\" (UniqueName: \"kubernetes.io/projected/a5fbd1bc-e091-459b-97fa-17d728070100-kube-api-access-rwlsc\") on node \"crc\" DevicePath \"\"" Dec 03 12:57:42 crc kubenswrapper[5052]: I1203 12:57:42.103762 5052 reconciler_common.go:293] "Volume detached for volume \"libvirt-secret-0\" (UniqueName: \"kubernetes.io/secret/a5fbd1bc-e091-459b-97fa-17d728070100-libvirt-secret-0\") on node \"crc\" DevicePath \"\"" Dec 03 12:57:42 crc kubenswrapper[5052]: I1203 12:57:42.612571 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/libvirt-edpm-compute-global-edpm-compute-global-29fbj" event={"ID":"a5fbd1bc-e091-459b-97fa-17d728070100","Type":"ContainerDied","Data":"f3fcd4f86feef8d8dbb6ab041f4f4b2619ce466958c255a03e5b930c08997007"} Dec 03 12:57:42 crc kubenswrapper[5052]: I1203 12:57:42.612610 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f3fcd4f86feef8d8dbb6ab041f4f4b2619ce466958c255a03e5b930c08997007" Dec 03 12:57:42 crc kubenswrapper[5052]: I1203 12:57:42.612626 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/libvirt-edpm-compute-global-edpm-compute-global-29fbj" Dec 03 12:57:42 crc kubenswrapper[5052]: I1203 12:57:42.972400 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-edpm-compute-global-edpm-compute-global-94g5g"] Dec 03 12:57:42 crc kubenswrapper[5052]: E1203 12:57:42.973025 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="a5fbd1bc-e091-459b-97fa-17d728070100" containerName="libvirt-edpm-compute-global-edpm-compute-global" Dec 03 12:57:42 crc kubenswrapper[5052]: I1203 12:57:42.973044 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="a5fbd1bc-e091-459b-97fa-17d728070100" containerName="libvirt-edpm-compute-global-edpm-compute-global" Dec 03 12:57:42 crc kubenswrapper[5052]: I1203 12:57:42.973211 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="a5fbd1bc-e091-459b-97fa-17d728070100" containerName="libvirt-edpm-compute-global-edpm-compute-global" Dec 03 12:57:42 crc kubenswrapper[5052]: I1203 12:57:42.973793 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-edpm-compute-global-edpm-compute-global-94g5g" Dec 03 12:57:42 crc kubenswrapper[5052]: I1203 12:57:42.976850 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"edpm-compute-global-dockercfg-p5rjx" Dec 03 12:57:42 crc kubenswrapper[5052]: I1203 12:57:42.977028 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"dataplane-ansible-ssh-private-key-secret" Dec 03 12:57:42 crc kubenswrapper[5052]: W1203 12:57:42.978105 5052 reflector.go:561] object-"openstack-kuttl-tests"/"nova-cell1-compute-config": failed to list *v1.Secret: secrets "nova-cell1-compute-config" is forbidden: User "system:node:crc" cannot list resource "secrets" in API group "" in the namespace "openstack-kuttl-tests": no relationship found between node 'crc' and this object Dec 03 12:57:42 crc kubenswrapper[5052]: E1203 12:57:42.978148 5052 reflector.go:158] "Unhandled Error" err="object-\"openstack-kuttl-tests\"/\"nova-cell1-compute-config\": Failed to watch *v1.Secret: failed to list *v1.Secret: secrets \"nova-cell1-compute-config\" is forbidden: User \"system:node:crc\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openstack-kuttl-tests\": no relationship found between node 'crc' and this object" logger="UnhandledError" Dec 03 12:57:42 crc kubenswrapper[5052]: I1203 12:57:42.978402 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"combined-ca-bundle" Dec 03 12:57:42 crc kubenswrapper[5052]: I1203 12:57:42.979207 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-migration-ssh-key" Dec 03 12:57:42 crc kubenswrapper[5052]: I1203 12:57:42.979207 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"openstack-aee-default-env" Dec 03 12:57:42 crc kubenswrapper[5052]: I1203 12:57:42.979697 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"dataplanenodeset-edpm-compute-global" Dec 03 12:57:42 crc kubenswrapper[5052]: I1203 12:57:42.986316 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-edpm-compute-global-edpm-compute-global-94g5g"] Dec 03 12:57:43 crc kubenswrapper[5052]: I1203 12:57:43.017162 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nova-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0577708f-dd8c-4a0d-9660-0c6640d42bc8-nova-combined-ca-bundle\") pod \"nova-edpm-compute-global-edpm-compute-global-94g5g\" (UID: \"0577708f-dd8c-4a0d-9660-0c6640d42bc8\") " pod="openstack-kuttl-tests/nova-edpm-compute-global-edpm-compute-global-94g5g" Dec 03 12:57:43 crc kubenswrapper[5052]: I1203 12:57:43.017274 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/0577708f-dd8c-4a0d-9660-0c6640d42bc8-inventory\") pod \"nova-edpm-compute-global-edpm-compute-global-94g5g\" (UID: \"0577708f-dd8c-4a0d-9660-0c6640d42bc8\") " pod="openstack-kuttl-tests/nova-edpm-compute-global-edpm-compute-global-94g5g" Dec 03 12:57:43 crc kubenswrapper[5052]: I1203 12:57:43.017319 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nova-migration-ssh-key-0\" (UniqueName: \"kubernetes.io/secret/0577708f-dd8c-4a0d-9660-0c6640d42bc8-nova-migration-ssh-key-0\") pod \"nova-edpm-compute-global-edpm-compute-global-94g5g\" (UID: \"0577708f-dd8c-4a0d-9660-0c6640d42bc8\") " pod="openstack-kuttl-tests/nova-edpm-compute-global-edpm-compute-global-94g5g" Dec 03 12:57:43 crc kubenswrapper[5052]: I1203 12:57:43.017347 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zjzpp\" (UniqueName: \"kubernetes.io/projected/0577708f-dd8c-4a0d-9660-0c6640d42bc8-kube-api-access-zjzpp\") pod \"nova-edpm-compute-global-edpm-compute-global-94g5g\" (UID: \"0577708f-dd8c-4a0d-9660-0c6640d42bc8\") " pod="openstack-kuttl-tests/nova-edpm-compute-global-edpm-compute-global-94g5g" Dec 03 12:57:43 crc kubenswrapper[5052]: I1203 12:57:43.017367 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nova-cell1-compute-config-0\" (UniqueName: \"kubernetes.io/secret/0577708f-dd8c-4a0d-9660-0c6640d42bc8-nova-cell1-compute-config-0\") pod \"nova-edpm-compute-global-edpm-compute-global-94g5g\" (UID: \"0577708f-dd8c-4a0d-9660-0c6640d42bc8\") " pod="openstack-kuttl-tests/nova-edpm-compute-global-edpm-compute-global-94g5g" Dec 03 12:57:43 crc kubenswrapper[5052]: I1203 12:57:43.017405 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/0577708f-dd8c-4a0d-9660-0c6640d42bc8-ssh-key\") pod \"nova-edpm-compute-global-edpm-compute-global-94g5g\" (UID: \"0577708f-dd8c-4a0d-9660-0c6640d42bc8\") " pod="openstack-kuttl-tests/nova-edpm-compute-global-edpm-compute-global-94g5g" Dec 03 12:57:43 crc kubenswrapper[5052]: I1203 12:57:43.017429 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nova-migration-ssh-key-1\" (UniqueName: \"kubernetes.io/secret/0577708f-dd8c-4a0d-9660-0c6640d42bc8-nova-migration-ssh-key-1\") pod \"nova-edpm-compute-global-edpm-compute-global-94g5g\" (UID: \"0577708f-dd8c-4a0d-9660-0c6640d42bc8\") " pod="openstack-kuttl-tests/nova-edpm-compute-global-edpm-compute-global-94g5g" Dec 03 12:57:43 crc kubenswrapper[5052]: I1203 12:57:43.017452 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nova-cell1-compute-config-1\" (UniqueName: \"kubernetes.io/secret/0577708f-dd8c-4a0d-9660-0c6640d42bc8-nova-cell1-compute-config-1\") pod \"nova-edpm-compute-global-edpm-compute-global-94g5g\" (UID: \"0577708f-dd8c-4a0d-9660-0c6640d42bc8\") " pod="openstack-kuttl-tests/nova-edpm-compute-global-edpm-compute-global-94g5g" Dec 03 12:57:43 crc kubenswrapper[5052]: I1203 12:57:43.118828 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"nova-migration-ssh-key-1\" (UniqueName: \"kubernetes.io/secret/0577708f-dd8c-4a0d-9660-0c6640d42bc8-nova-migration-ssh-key-1\") pod \"nova-edpm-compute-global-edpm-compute-global-94g5g\" (UID: \"0577708f-dd8c-4a0d-9660-0c6640d42bc8\") " pod="openstack-kuttl-tests/nova-edpm-compute-global-edpm-compute-global-94g5g" Dec 03 12:57:43 crc kubenswrapper[5052]: I1203 12:57:43.119176 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"nova-cell1-compute-config-1\" (UniqueName: \"kubernetes.io/secret/0577708f-dd8c-4a0d-9660-0c6640d42bc8-nova-cell1-compute-config-1\") pod \"nova-edpm-compute-global-edpm-compute-global-94g5g\" (UID: \"0577708f-dd8c-4a0d-9660-0c6640d42bc8\") " pod="openstack-kuttl-tests/nova-edpm-compute-global-edpm-compute-global-94g5g" Dec 03 12:57:43 crc kubenswrapper[5052]: I1203 12:57:43.119434 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"nova-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0577708f-dd8c-4a0d-9660-0c6640d42bc8-nova-combined-ca-bundle\") pod \"nova-edpm-compute-global-edpm-compute-global-94g5g\" (UID: \"0577708f-dd8c-4a0d-9660-0c6640d42bc8\") " pod="openstack-kuttl-tests/nova-edpm-compute-global-edpm-compute-global-94g5g" Dec 03 12:57:43 crc kubenswrapper[5052]: I1203 12:57:43.119627 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/0577708f-dd8c-4a0d-9660-0c6640d42bc8-inventory\") pod \"nova-edpm-compute-global-edpm-compute-global-94g5g\" (UID: \"0577708f-dd8c-4a0d-9660-0c6640d42bc8\") " pod="openstack-kuttl-tests/nova-edpm-compute-global-edpm-compute-global-94g5g" Dec 03 12:57:43 crc kubenswrapper[5052]: I1203 12:57:43.120226 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"nova-migration-ssh-key-0\" (UniqueName: \"kubernetes.io/secret/0577708f-dd8c-4a0d-9660-0c6640d42bc8-nova-migration-ssh-key-0\") pod \"nova-edpm-compute-global-edpm-compute-global-94g5g\" (UID: \"0577708f-dd8c-4a0d-9660-0c6640d42bc8\") " pod="openstack-kuttl-tests/nova-edpm-compute-global-edpm-compute-global-94g5g" Dec 03 12:57:43 crc kubenswrapper[5052]: I1203 12:57:43.120419 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-zjzpp\" (UniqueName: \"kubernetes.io/projected/0577708f-dd8c-4a0d-9660-0c6640d42bc8-kube-api-access-zjzpp\") pod \"nova-edpm-compute-global-edpm-compute-global-94g5g\" (UID: \"0577708f-dd8c-4a0d-9660-0c6640d42bc8\") " pod="openstack-kuttl-tests/nova-edpm-compute-global-edpm-compute-global-94g5g" Dec 03 12:57:43 crc kubenswrapper[5052]: I1203 12:57:43.120550 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"nova-cell1-compute-config-0\" (UniqueName: \"kubernetes.io/secret/0577708f-dd8c-4a0d-9660-0c6640d42bc8-nova-cell1-compute-config-0\") pod \"nova-edpm-compute-global-edpm-compute-global-94g5g\" (UID: \"0577708f-dd8c-4a0d-9660-0c6640d42bc8\") " pod="openstack-kuttl-tests/nova-edpm-compute-global-edpm-compute-global-94g5g" Dec 03 12:57:43 crc kubenswrapper[5052]: I1203 12:57:43.120694 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/0577708f-dd8c-4a0d-9660-0c6640d42bc8-ssh-key\") pod \"nova-edpm-compute-global-edpm-compute-global-94g5g\" (UID: \"0577708f-dd8c-4a0d-9660-0c6640d42bc8\") " pod="openstack-kuttl-tests/nova-edpm-compute-global-edpm-compute-global-94g5g" Dec 03 12:57:43 crc kubenswrapper[5052]: I1203 12:57:43.122883 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"nova-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0577708f-dd8c-4a0d-9660-0c6640d42bc8-nova-combined-ca-bundle\") pod \"nova-edpm-compute-global-edpm-compute-global-94g5g\" (UID: \"0577708f-dd8c-4a0d-9660-0c6640d42bc8\") " pod="openstack-kuttl-tests/nova-edpm-compute-global-edpm-compute-global-94g5g" Dec 03 12:57:43 crc kubenswrapper[5052]: I1203 12:57:43.124797 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"nova-migration-ssh-key-1\" (UniqueName: \"kubernetes.io/secret/0577708f-dd8c-4a0d-9660-0c6640d42bc8-nova-migration-ssh-key-1\") pod \"nova-edpm-compute-global-edpm-compute-global-94g5g\" (UID: \"0577708f-dd8c-4a0d-9660-0c6640d42bc8\") " pod="openstack-kuttl-tests/nova-edpm-compute-global-edpm-compute-global-94g5g" Dec 03 12:57:43 crc kubenswrapper[5052]: I1203 12:57:43.125463 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"nova-migration-ssh-key-0\" (UniqueName: \"kubernetes.io/secret/0577708f-dd8c-4a0d-9660-0c6640d42bc8-nova-migration-ssh-key-0\") pod \"nova-edpm-compute-global-edpm-compute-global-94g5g\" (UID: \"0577708f-dd8c-4a0d-9660-0c6640d42bc8\") " pod="openstack-kuttl-tests/nova-edpm-compute-global-edpm-compute-global-94g5g" Dec 03 12:57:43 crc kubenswrapper[5052]: I1203 12:57:43.125638 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/0577708f-dd8c-4a0d-9660-0c6640d42bc8-ssh-key\") pod \"nova-edpm-compute-global-edpm-compute-global-94g5g\" (UID: \"0577708f-dd8c-4a0d-9660-0c6640d42bc8\") " pod="openstack-kuttl-tests/nova-edpm-compute-global-edpm-compute-global-94g5g" Dec 03 12:57:43 crc kubenswrapper[5052]: I1203 12:57:43.125800 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/0577708f-dd8c-4a0d-9660-0c6640d42bc8-inventory\") pod \"nova-edpm-compute-global-edpm-compute-global-94g5g\" (UID: \"0577708f-dd8c-4a0d-9660-0c6640d42bc8\") " pod="openstack-kuttl-tests/nova-edpm-compute-global-edpm-compute-global-94g5g" Dec 03 12:57:43 crc kubenswrapper[5052]: I1203 12:57:43.137756 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-zjzpp\" (UniqueName: \"kubernetes.io/projected/0577708f-dd8c-4a0d-9660-0c6640d42bc8-kube-api-access-zjzpp\") pod \"nova-edpm-compute-global-edpm-compute-global-94g5g\" (UID: \"0577708f-dd8c-4a0d-9660-0c6640d42bc8\") " pod="openstack-kuttl-tests/nova-edpm-compute-global-edpm-compute-global-94g5g" Dec 03 12:57:44 crc kubenswrapper[5052]: E1203 12:57:44.120095 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/nova-cell1-compute-config: failed to sync secret cache: timed out waiting for the condition Dec 03 12:57:44 crc kubenswrapper[5052]: E1203 12:57:44.120198 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0577708f-dd8c-4a0d-9660-0c6640d42bc8-nova-cell1-compute-config-1 podName:0577708f-dd8c-4a0d-9660-0c6640d42bc8 nodeName:}" failed. No retries permitted until 2025-12-03 12:57:44.6201749 +0000 UTC m=+5855.208416684 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "nova-cell1-compute-config-1" (UniqueName: "kubernetes.io/secret/0577708f-dd8c-4a0d-9660-0c6640d42bc8-nova-cell1-compute-config-1") pod "nova-edpm-compute-global-edpm-compute-global-94g5g" (UID: "0577708f-dd8c-4a0d-9660-0c6640d42bc8") : failed to sync secret cache: timed out waiting for the condition Dec 03 12:57:44 crc kubenswrapper[5052]: E1203 12:57:44.121175 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/nova-cell1-compute-config: failed to sync secret cache: timed out waiting for the condition Dec 03 12:57:44 crc kubenswrapper[5052]: E1203 12:57:44.121220 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0577708f-dd8c-4a0d-9660-0c6640d42bc8-nova-cell1-compute-config-0 podName:0577708f-dd8c-4a0d-9660-0c6640d42bc8 nodeName:}" failed. No retries permitted until 2025-12-03 12:57:44.62121085 +0000 UTC m=+5855.209452614 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "nova-cell1-compute-config-0" (UniqueName: "kubernetes.io/secret/0577708f-dd8c-4a0d-9660-0c6640d42bc8-nova-cell1-compute-config-0") pod "nova-edpm-compute-global-edpm-compute-global-94g5g" (UID: "0577708f-dd8c-4a0d-9660-0c6640d42bc8") : failed to sync secret cache: timed out waiting for the condition Dec 03 12:57:44 crc kubenswrapper[5052]: I1203 12:57:44.200688 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell1-compute-config" Dec 03 12:57:44 crc kubenswrapper[5052]: I1203 12:57:44.643666 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"nova-cell1-compute-config-0\" (UniqueName: \"kubernetes.io/secret/0577708f-dd8c-4a0d-9660-0c6640d42bc8-nova-cell1-compute-config-0\") pod \"nova-edpm-compute-global-edpm-compute-global-94g5g\" (UID: \"0577708f-dd8c-4a0d-9660-0c6640d42bc8\") " pod="openstack-kuttl-tests/nova-edpm-compute-global-edpm-compute-global-94g5g" Dec 03 12:57:44 crc kubenswrapper[5052]: I1203 12:57:44.643746 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"nova-cell1-compute-config-1\" (UniqueName: \"kubernetes.io/secret/0577708f-dd8c-4a0d-9660-0c6640d42bc8-nova-cell1-compute-config-1\") pod \"nova-edpm-compute-global-edpm-compute-global-94g5g\" (UID: \"0577708f-dd8c-4a0d-9660-0c6640d42bc8\") " pod="openstack-kuttl-tests/nova-edpm-compute-global-edpm-compute-global-94g5g" Dec 03 12:57:44 crc kubenswrapper[5052]: I1203 12:57:44.651237 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"nova-cell1-compute-config-0\" (UniqueName: \"kubernetes.io/secret/0577708f-dd8c-4a0d-9660-0c6640d42bc8-nova-cell1-compute-config-0\") pod \"nova-edpm-compute-global-edpm-compute-global-94g5g\" (UID: \"0577708f-dd8c-4a0d-9660-0c6640d42bc8\") " pod="openstack-kuttl-tests/nova-edpm-compute-global-edpm-compute-global-94g5g" Dec 03 12:57:44 crc kubenswrapper[5052]: I1203 12:57:44.652398 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"nova-cell1-compute-config-1\" (UniqueName: \"kubernetes.io/secret/0577708f-dd8c-4a0d-9660-0c6640d42bc8-nova-cell1-compute-config-1\") pod \"nova-edpm-compute-global-edpm-compute-global-94g5g\" (UID: \"0577708f-dd8c-4a0d-9660-0c6640d42bc8\") " pod="openstack-kuttl-tests/nova-edpm-compute-global-edpm-compute-global-94g5g" Dec 03 12:57:44 crc kubenswrapper[5052]: I1203 12:57:44.796044 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-edpm-compute-global-edpm-compute-global-94g5g" Dec 03 12:57:45 crc kubenswrapper[5052]: I1203 12:57:45.258250 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-edpm-compute-global-edpm-compute-global-94g5g"] Dec 03 12:57:45 crc kubenswrapper[5052]: I1203 12:57:45.638546 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-edpm-compute-global-edpm-compute-global-94g5g" event={"ID":"0577708f-dd8c-4a0d-9660-0c6640d42bc8","Type":"ContainerStarted","Data":"d07beb0438fd34b907af232ce1adb70a481f2cbbca267c598ed6026dd4c13ba7"} Dec 03 12:57:46 crc kubenswrapper[5052]: I1203 12:57:46.648800 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-edpm-compute-global-edpm-compute-global-94g5g" event={"ID":"0577708f-dd8c-4a0d-9660-0c6640d42bc8","Type":"ContainerStarted","Data":"173e3047b0b5ffbab691f7ddc4b2f4f315ad802a64b3aec944a4b88629cd0168"} Dec 03 12:57:46 crc kubenswrapper[5052]: I1203 12:57:46.671209 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-edpm-compute-global-edpm-compute-global-94g5g" podStartSLOduration=4.014514574 podStartE2EDuration="4.671188094s" podCreationTimestamp="2025-12-03 12:57:42 +0000 UTC" firstStartedPulling="2025-12-03 12:57:45.293671369 +0000 UTC m=+5855.881913133" lastFinishedPulling="2025-12-03 12:57:45.950344889 +0000 UTC m=+5856.538586653" observedRunningTime="2025-12-03 12:57:46.664688288 +0000 UTC m=+5857.252930052" watchObservedRunningTime="2025-12-03 12:57:46.671188094 +0000 UTC m=+5857.259429868" Dec 03 12:57:46 crc kubenswrapper[5052]: I1203 12:57:46.673406 5052 scope.go:117] "RemoveContainer" containerID="69fa7fbcd2aceb7c53703a19508876f93a97dc9adabb79c16fb691d90f606d58" Dec 03 12:57:46 crc kubenswrapper[5052]: E1203 12:57:46.673863 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 12:57:47 crc kubenswrapper[5052]: I1203 12:57:47.659909 5052 generic.go:334] "Generic (PLEG): container finished" podID="0577708f-dd8c-4a0d-9660-0c6640d42bc8" containerID="173e3047b0b5ffbab691f7ddc4b2f4f315ad802a64b3aec944a4b88629cd0168" exitCode=0 Dec 03 12:57:47 crc kubenswrapper[5052]: I1203 12:57:47.659970 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-edpm-compute-global-edpm-compute-global-94g5g" event={"ID":"0577708f-dd8c-4a0d-9660-0c6640d42bc8","Type":"ContainerDied","Data":"173e3047b0b5ffbab691f7ddc4b2f4f315ad802a64b3aec944a4b88629cd0168"} Dec 03 12:57:48 crc kubenswrapper[5052]: I1203 12:57:48.947395 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-edpm-compute-global-edpm-compute-global-94g5g" Dec 03 12:57:49 crc kubenswrapper[5052]: I1203 12:57:49.132713 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/0577708f-dd8c-4a0d-9660-0c6640d42bc8-ssh-key\") pod \"0577708f-dd8c-4a0d-9660-0c6640d42bc8\" (UID: \"0577708f-dd8c-4a0d-9660-0c6640d42bc8\") " Dec 03 12:57:49 crc kubenswrapper[5052]: I1203 12:57:49.133311 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zjzpp\" (UniqueName: \"kubernetes.io/projected/0577708f-dd8c-4a0d-9660-0c6640d42bc8-kube-api-access-zjzpp\") pod \"0577708f-dd8c-4a0d-9660-0c6640d42bc8\" (UID: \"0577708f-dd8c-4a0d-9660-0c6640d42bc8\") " Dec 03 12:57:49 crc kubenswrapper[5052]: I1203 12:57:49.133387 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"nova-migration-ssh-key-0\" (UniqueName: \"kubernetes.io/secret/0577708f-dd8c-4a0d-9660-0c6640d42bc8-nova-migration-ssh-key-0\") pod \"0577708f-dd8c-4a0d-9660-0c6640d42bc8\" (UID: \"0577708f-dd8c-4a0d-9660-0c6640d42bc8\") " Dec 03 12:57:49 crc kubenswrapper[5052]: I1203 12:57:49.133435 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"nova-cell1-compute-config-0\" (UniqueName: \"kubernetes.io/secret/0577708f-dd8c-4a0d-9660-0c6640d42bc8-nova-cell1-compute-config-0\") pod \"0577708f-dd8c-4a0d-9660-0c6640d42bc8\" (UID: \"0577708f-dd8c-4a0d-9660-0c6640d42bc8\") " Dec 03 12:57:49 crc kubenswrapper[5052]: I1203 12:57:49.133499 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/0577708f-dd8c-4a0d-9660-0c6640d42bc8-inventory\") pod \"0577708f-dd8c-4a0d-9660-0c6640d42bc8\" (UID: \"0577708f-dd8c-4a0d-9660-0c6640d42bc8\") " Dec 03 12:57:49 crc kubenswrapper[5052]: I1203 12:57:49.133603 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"nova-migration-ssh-key-1\" (UniqueName: \"kubernetes.io/secret/0577708f-dd8c-4a0d-9660-0c6640d42bc8-nova-migration-ssh-key-1\") pod \"0577708f-dd8c-4a0d-9660-0c6640d42bc8\" (UID: \"0577708f-dd8c-4a0d-9660-0c6640d42bc8\") " Dec 03 12:57:49 crc kubenswrapper[5052]: I1203 12:57:49.133641 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"nova-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0577708f-dd8c-4a0d-9660-0c6640d42bc8-nova-combined-ca-bundle\") pod \"0577708f-dd8c-4a0d-9660-0c6640d42bc8\" (UID: \"0577708f-dd8c-4a0d-9660-0c6640d42bc8\") " Dec 03 12:57:49 crc kubenswrapper[5052]: I1203 12:57:49.133695 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"nova-cell1-compute-config-1\" (UniqueName: \"kubernetes.io/secret/0577708f-dd8c-4a0d-9660-0c6640d42bc8-nova-cell1-compute-config-1\") pod \"0577708f-dd8c-4a0d-9660-0c6640d42bc8\" (UID: \"0577708f-dd8c-4a0d-9660-0c6640d42bc8\") " Dec 03 12:57:49 crc kubenswrapper[5052]: I1203 12:57:49.138978 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0577708f-dd8c-4a0d-9660-0c6640d42bc8-nova-combined-ca-bundle" (OuterVolumeSpecName: "nova-combined-ca-bundle") pod "0577708f-dd8c-4a0d-9660-0c6640d42bc8" (UID: "0577708f-dd8c-4a0d-9660-0c6640d42bc8"). InnerVolumeSpecName "nova-combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:57:49 crc kubenswrapper[5052]: I1203 12:57:49.139527 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0577708f-dd8c-4a0d-9660-0c6640d42bc8-kube-api-access-zjzpp" (OuterVolumeSpecName: "kube-api-access-zjzpp") pod "0577708f-dd8c-4a0d-9660-0c6640d42bc8" (UID: "0577708f-dd8c-4a0d-9660-0c6640d42bc8"). InnerVolumeSpecName "kube-api-access-zjzpp". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:57:49 crc kubenswrapper[5052]: I1203 12:57:49.155631 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0577708f-dd8c-4a0d-9660-0c6640d42bc8-nova-migration-ssh-key-0" (OuterVolumeSpecName: "nova-migration-ssh-key-0") pod "0577708f-dd8c-4a0d-9660-0c6640d42bc8" (UID: "0577708f-dd8c-4a0d-9660-0c6640d42bc8"). InnerVolumeSpecName "nova-migration-ssh-key-0". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:57:49 crc kubenswrapper[5052]: I1203 12:57:49.156958 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0577708f-dd8c-4a0d-9660-0c6640d42bc8-ssh-key" (OuterVolumeSpecName: "ssh-key") pod "0577708f-dd8c-4a0d-9660-0c6640d42bc8" (UID: "0577708f-dd8c-4a0d-9660-0c6640d42bc8"). InnerVolumeSpecName "ssh-key". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:57:49 crc kubenswrapper[5052]: I1203 12:57:49.157530 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0577708f-dd8c-4a0d-9660-0c6640d42bc8-nova-cell1-compute-config-0" (OuterVolumeSpecName: "nova-cell1-compute-config-0") pod "0577708f-dd8c-4a0d-9660-0c6640d42bc8" (UID: "0577708f-dd8c-4a0d-9660-0c6640d42bc8"). InnerVolumeSpecName "nova-cell1-compute-config-0". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:57:49 crc kubenswrapper[5052]: I1203 12:57:49.157709 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0577708f-dd8c-4a0d-9660-0c6640d42bc8-nova-cell1-compute-config-1" (OuterVolumeSpecName: "nova-cell1-compute-config-1") pod "0577708f-dd8c-4a0d-9660-0c6640d42bc8" (UID: "0577708f-dd8c-4a0d-9660-0c6640d42bc8"). InnerVolumeSpecName "nova-cell1-compute-config-1". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:57:49 crc kubenswrapper[5052]: I1203 12:57:49.159107 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0577708f-dd8c-4a0d-9660-0c6640d42bc8-inventory" (OuterVolumeSpecName: "inventory") pod "0577708f-dd8c-4a0d-9660-0c6640d42bc8" (UID: "0577708f-dd8c-4a0d-9660-0c6640d42bc8"). InnerVolumeSpecName "inventory". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:57:49 crc kubenswrapper[5052]: I1203 12:57:49.160335 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0577708f-dd8c-4a0d-9660-0c6640d42bc8-nova-migration-ssh-key-1" (OuterVolumeSpecName: "nova-migration-ssh-key-1") pod "0577708f-dd8c-4a0d-9660-0c6640d42bc8" (UID: "0577708f-dd8c-4a0d-9660-0c6640d42bc8"). InnerVolumeSpecName "nova-migration-ssh-key-1". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:57:49 crc kubenswrapper[5052]: I1203 12:57:49.235094 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-zjzpp\" (UniqueName: \"kubernetes.io/projected/0577708f-dd8c-4a0d-9660-0c6640d42bc8-kube-api-access-zjzpp\") on node \"crc\" DevicePath \"\"" Dec 03 12:57:49 crc kubenswrapper[5052]: I1203 12:57:49.235580 5052 reconciler_common.go:293] "Volume detached for volume \"nova-migration-ssh-key-0\" (UniqueName: \"kubernetes.io/secret/0577708f-dd8c-4a0d-9660-0c6640d42bc8-nova-migration-ssh-key-0\") on node \"crc\" DevicePath \"\"" Dec 03 12:57:49 crc kubenswrapper[5052]: I1203 12:57:49.235644 5052 reconciler_common.go:293] "Volume detached for volume \"nova-cell1-compute-config-0\" (UniqueName: \"kubernetes.io/secret/0577708f-dd8c-4a0d-9660-0c6640d42bc8-nova-cell1-compute-config-0\") on node \"crc\" DevicePath \"\"" Dec 03 12:57:49 crc kubenswrapper[5052]: I1203 12:57:49.235716 5052 reconciler_common.go:293] "Volume detached for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/0577708f-dd8c-4a0d-9660-0c6640d42bc8-inventory\") on node \"crc\" DevicePath \"\"" Dec 03 12:57:49 crc kubenswrapper[5052]: I1203 12:57:49.235779 5052 reconciler_common.go:293] "Volume detached for volume \"nova-migration-ssh-key-1\" (UniqueName: \"kubernetes.io/secret/0577708f-dd8c-4a0d-9660-0c6640d42bc8-nova-migration-ssh-key-1\") on node \"crc\" DevicePath \"\"" Dec 03 12:57:49 crc kubenswrapper[5052]: I1203 12:57:49.235837 5052 reconciler_common.go:293] "Volume detached for volume \"nova-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0577708f-dd8c-4a0d-9660-0c6640d42bc8-nova-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:57:49 crc kubenswrapper[5052]: I1203 12:57:49.235897 5052 reconciler_common.go:293] "Volume detached for volume \"nova-cell1-compute-config-1\" (UniqueName: \"kubernetes.io/secret/0577708f-dd8c-4a0d-9660-0c6640d42bc8-nova-cell1-compute-config-1\") on node \"crc\" DevicePath \"\"" Dec 03 12:57:49 crc kubenswrapper[5052]: I1203 12:57:49.235950 5052 reconciler_common.go:293] "Volume detached for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/0577708f-dd8c-4a0d-9660-0c6640d42bc8-ssh-key\") on node \"crc\" DevicePath \"\"" Dec 03 12:57:49 crc kubenswrapper[5052]: I1203 12:57:49.679399 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-edpm-compute-global-edpm-compute-global-94g5g" Dec 03 12:57:49 crc kubenswrapper[5052]: I1203 12:57:49.682668 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-edpm-compute-global-edpm-compute-global-94g5g" event={"ID":"0577708f-dd8c-4a0d-9660-0c6640d42bc8","Type":"ContainerDied","Data":"d07beb0438fd34b907af232ce1adb70a481f2cbbca267c598ed6026dd4c13ba7"} Dec 03 12:57:49 crc kubenswrapper[5052]: I1203 12:57:49.682715 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d07beb0438fd34b907af232ce1adb70a481f2cbbca267c598ed6026dd4c13ba7" Dec 03 12:57:49 crc kubenswrapper[5052]: I1203 12:57:49.753728 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/custom-global-service-edpm-compute-global-ld7cm"] Dec 03 12:57:49 crc kubenswrapper[5052]: E1203 12:57:49.754109 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="0577708f-dd8c-4a0d-9660-0c6640d42bc8" containerName="nova-edpm-compute-global-edpm-compute-global" Dec 03 12:57:49 crc kubenswrapper[5052]: I1203 12:57:49.754124 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="0577708f-dd8c-4a0d-9660-0c6640d42bc8" containerName="nova-edpm-compute-global-edpm-compute-global" Dec 03 12:57:49 crc kubenswrapper[5052]: I1203 12:57:49.754342 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="0577708f-dd8c-4a0d-9660-0c6640d42bc8" containerName="nova-edpm-compute-global-edpm-compute-global" Dec 03 12:57:49 crc kubenswrapper[5052]: I1203 12:57:49.755023 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/custom-global-service-edpm-compute-global-ld7cm" Dec 03 12:57:49 crc kubenswrapper[5052]: I1203 12:57:49.756918 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"edpm-compute-global-dockercfg-p5rjx" Dec 03 12:57:49 crc kubenswrapper[5052]: I1203 12:57:49.756937 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"combined-ca-bundle" Dec 03 12:57:49 crc kubenswrapper[5052]: I1203 12:57:49.757375 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"openstack-aee-default-env" Dec 03 12:57:49 crc kubenswrapper[5052]: I1203 12:57:49.758674 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"dataplane-ansible-ssh-private-key-secret" Dec 03 12:57:49 crc kubenswrapper[5052]: I1203 12:57:49.762448 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/custom-global-service-edpm-compute-global-ld7cm"] Dec 03 12:57:49 crc kubenswrapper[5052]: I1203 12:57:49.812893 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"dataplanenodeset-edpm-compute-global" Dec 03 12:57:49 crc kubenswrapper[5052]: I1203 12:57:49.949981 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ssh-key-edpm-compute-global\" (UniqueName: \"kubernetes.io/secret/5e1054ea-1bb6-4b9b-9393-26fc530fdc71-ssh-key-edpm-compute-global\") pod \"custom-global-service-edpm-compute-global-ld7cm\" (UID: \"5e1054ea-1bb6-4b9b-9393-26fc530fdc71\") " pod="openstack-kuttl-tests/custom-global-service-edpm-compute-global-ld7cm" Dec 03 12:57:49 crc kubenswrapper[5052]: I1203 12:57:49.950039 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"custom-global-service-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5e1054ea-1bb6-4b9b-9393-26fc530fdc71-custom-global-service-combined-ca-bundle\") pod \"custom-global-service-edpm-compute-global-ld7cm\" (UID: \"5e1054ea-1bb6-4b9b-9393-26fc530fdc71\") " pod="openstack-kuttl-tests/custom-global-service-edpm-compute-global-ld7cm" Dec 03 12:57:49 crc kubenswrapper[5052]: I1203 12:57:49.950065 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-99wqk\" (UniqueName: \"kubernetes.io/projected/5e1054ea-1bb6-4b9b-9393-26fc530fdc71-kube-api-access-99wqk\") pod \"custom-global-service-edpm-compute-global-ld7cm\" (UID: \"5e1054ea-1bb6-4b9b-9393-26fc530fdc71\") " pod="openstack-kuttl-tests/custom-global-service-edpm-compute-global-ld7cm" Dec 03 12:57:49 crc kubenswrapper[5052]: I1203 12:57:49.950168 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"inventory-0\" (UniqueName: \"kubernetes.io/secret/5e1054ea-1bb6-4b9b-9393-26fc530fdc71-inventory-0\") pod \"custom-global-service-edpm-compute-global-ld7cm\" (UID: \"5e1054ea-1bb6-4b9b-9393-26fc530fdc71\") " pod="openstack-kuttl-tests/custom-global-service-edpm-compute-global-ld7cm" Dec 03 12:57:50 crc kubenswrapper[5052]: I1203 12:57:50.051810 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"inventory-0\" (UniqueName: \"kubernetes.io/secret/5e1054ea-1bb6-4b9b-9393-26fc530fdc71-inventory-0\") pod \"custom-global-service-edpm-compute-global-ld7cm\" (UID: \"5e1054ea-1bb6-4b9b-9393-26fc530fdc71\") " pod="openstack-kuttl-tests/custom-global-service-edpm-compute-global-ld7cm" Dec 03 12:57:50 crc kubenswrapper[5052]: I1203 12:57:50.051902 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ssh-key-edpm-compute-global\" (UniqueName: \"kubernetes.io/secret/5e1054ea-1bb6-4b9b-9393-26fc530fdc71-ssh-key-edpm-compute-global\") pod \"custom-global-service-edpm-compute-global-ld7cm\" (UID: \"5e1054ea-1bb6-4b9b-9393-26fc530fdc71\") " pod="openstack-kuttl-tests/custom-global-service-edpm-compute-global-ld7cm" Dec 03 12:57:50 crc kubenswrapper[5052]: I1203 12:57:50.051955 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"custom-global-service-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5e1054ea-1bb6-4b9b-9393-26fc530fdc71-custom-global-service-combined-ca-bundle\") pod \"custom-global-service-edpm-compute-global-ld7cm\" (UID: \"5e1054ea-1bb6-4b9b-9393-26fc530fdc71\") " pod="openstack-kuttl-tests/custom-global-service-edpm-compute-global-ld7cm" Dec 03 12:57:50 crc kubenswrapper[5052]: I1203 12:57:50.051985 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-99wqk\" (UniqueName: \"kubernetes.io/projected/5e1054ea-1bb6-4b9b-9393-26fc530fdc71-kube-api-access-99wqk\") pod \"custom-global-service-edpm-compute-global-ld7cm\" (UID: \"5e1054ea-1bb6-4b9b-9393-26fc530fdc71\") " pod="openstack-kuttl-tests/custom-global-service-edpm-compute-global-ld7cm" Dec 03 12:57:50 crc kubenswrapper[5052]: I1203 12:57:50.056714 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ssh-key-edpm-compute-global\" (UniqueName: \"kubernetes.io/secret/5e1054ea-1bb6-4b9b-9393-26fc530fdc71-ssh-key-edpm-compute-global\") pod \"custom-global-service-edpm-compute-global-ld7cm\" (UID: \"5e1054ea-1bb6-4b9b-9393-26fc530fdc71\") " pod="openstack-kuttl-tests/custom-global-service-edpm-compute-global-ld7cm" Dec 03 12:57:50 crc kubenswrapper[5052]: I1203 12:57:50.056758 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"custom-global-service-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5e1054ea-1bb6-4b9b-9393-26fc530fdc71-custom-global-service-combined-ca-bundle\") pod \"custom-global-service-edpm-compute-global-ld7cm\" (UID: \"5e1054ea-1bb6-4b9b-9393-26fc530fdc71\") " pod="openstack-kuttl-tests/custom-global-service-edpm-compute-global-ld7cm" Dec 03 12:57:50 crc kubenswrapper[5052]: I1203 12:57:50.058769 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"inventory-0\" (UniqueName: \"kubernetes.io/secret/5e1054ea-1bb6-4b9b-9393-26fc530fdc71-inventory-0\") pod \"custom-global-service-edpm-compute-global-ld7cm\" (UID: \"5e1054ea-1bb6-4b9b-9393-26fc530fdc71\") " pod="openstack-kuttl-tests/custom-global-service-edpm-compute-global-ld7cm" Dec 03 12:57:50 crc kubenswrapper[5052]: I1203 12:57:50.070047 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-99wqk\" (UniqueName: \"kubernetes.io/projected/5e1054ea-1bb6-4b9b-9393-26fc530fdc71-kube-api-access-99wqk\") pod \"custom-global-service-edpm-compute-global-ld7cm\" (UID: \"5e1054ea-1bb6-4b9b-9393-26fc530fdc71\") " pod="openstack-kuttl-tests/custom-global-service-edpm-compute-global-ld7cm" Dec 03 12:57:50 crc kubenswrapper[5052]: I1203 12:57:50.130407 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/custom-global-service-edpm-compute-global-ld7cm" Dec 03 12:57:50 crc kubenswrapper[5052]: I1203 12:57:50.356818 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/custom-global-service-edpm-compute-global-ld7cm"] Dec 03 12:57:50 crc kubenswrapper[5052]: I1203 12:57:50.688380 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/custom-global-service-edpm-compute-global-ld7cm" event={"ID":"5e1054ea-1bb6-4b9b-9393-26fc530fdc71","Type":"ContainerStarted","Data":"3b7eb80ce8b71c25677bbc0f2908bbce09572beb63cc1b58f5546b241ffc79ad"} Dec 03 12:57:51 crc kubenswrapper[5052]: I1203 12:57:51.698020 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/custom-global-service-edpm-compute-global-ld7cm" event={"ID":"5e1054ea-1bb6-4b9b-9393-26fc530fdc71","Type":"ContainerStarted","Data":"def25f6d5d263d80362de9d1d7e5e67b28a08f4e2bbd4d56a2771bd32a5cf8bf"} Dec 03 12:57:51 crc kubenswrapper[5052]: I1203 12:57:51.716063 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/custom-global-service-edpm-compute-global-ld7cm" podStartSLOduration=2.180158253 podStartE2EDuration="2.716036958s" podCreationTimestamp="2025-12-03 12:57:49 +0000 UTC" firstStartedPulling="2025-12-03 12:57:50.366260987 +0000 UTC m=+5860.954502751" lastFinishedPulling="2025-12-03 12:57:50.902139692 +0000 UTC m=+5861.490381456" observedRunningTime="2025-12-03 12:57:51.714422692 +0000 UTC m=+5862.302664456" watchObservedRunningTime="2025-12-03 12:57:51.716036958 +0000 UTC m=+5862.304278732" Dec 03 12:57:54 crc kubenswrapper[5052]: I1203 12:57:54.723101 5052 generic.go:334] "Generic (PLEG): container finished" podID="5e1054ea-1bb6-4b9b-9393-26fc530fdc71" containerID="def25f6d5d263d80362de9d1d7e5e67b28a08f4e2bbd4d56a2771bd32a5cf8bf" exitCode=0 Dec 03 12:57:54 crc kubenswrapper[5052]: I1203 12:57:54.723324 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/custom-global-service-edpm-compute-global-ld7cm" event={"ID":"5e1054ea-1bb6-4b9b-9393-26fc530fdc71","Type":"ContainerDied","Data":"def25f6d5d263d80362de9d1d7e5e67b28a08f4e2bbd4d56a2771bd32a5cf8bf"} Dec 03 12:57:55 crc kubenswrapper[5052]: I1203 12:57:55.993055 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/custom-global-service-edpm-compute-global-ld7cm" Dec 03 12:57:56 crc kubenswrapper[5052]: I1203 12:57:56.138949 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ssh-key-edpm-compute-global\" (UniqueName: \"kubernetes.io/secret/5e1054ea-1bb6-4b9b-9393-26fc530fdc71-ssh-key-edpm-compute-global\") pod \"5e1054ea-1bb6-4b9b-9393-26fc530fdc71\" (UID: \"5e1054ea-1bb6-4b9b-9393-26fc530fdc71\") " Dec 03 12:57:56 crc kubenswrapper[5052]: I1203 12:57:56.139009 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"custom-global-service-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5e1054ea-1bb6-4b9b-9393-26fc530fdc71-custom-global-service-combined-ca-bundle\") pod \"5e1054ea-1bb6-4b9b-9393-26fc530fdc71\" (UID: \"5e1054ea-1bb6-4b9b-9393-26fc530fdc71\") " Dec 03 12:57:56 crc kubenswrapper[5052]: I1203 12:57:56.139054 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-99wqk\" (UniqueName: \"kubernetes.io/projected/5e1054ea-1bb6-4b9b-9393-26fc530fdc71-kube-api-access-99wqk\") pod \"5e1054ea-1bb6-4b9b-9393-26fc530fdc71\" (UID: \"5e1054ea-1bb6-4b9b-9393-26fc530fdc71\") " Dec 03 12:57:56 crc kubenswrapper[5052]: I1203 12:57:56.139550 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"inventory-0\" (UniqueName: \"kubernetes.io/secret/5e1054ea-1bb6-4b9b-9393-26fc530fdc71-inventory-0\") pod \"5e1054ea-1bb6-4b9b-9393-26fc530fdc71\" (UID: \"5e1054ea-1bb6-4b9b-9393-26fc530fdc71\") " Dec 03 12:57:56 crc kubenswrapper[5052]: I1203 12:57:56.144276 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5e1054ea-1bb6-4b9b-9393-26fc530fdc71-custom-global-service-combined-ca-bundle" (OuterVolumeSpecName: "custom-global-service-combined-ca-bundle") pod "5e1054ea-1bb6-4b9b-9393-26fc530fdc71" (UID: "5e1054ea-1bb6-4b9b-9393-26fc530fdc71"). InnerVolumeSpecName "custom-global-service-combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:57:56 crc kubenswrapper[5052]: I1203 12:57:56.144625 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5e1054ea-1bb6-4b9b-9393-26fc530fdc71-kube-api-access-99wqk" (OuterVolumeSpecName: "kube-api-access-99wqk") pod "5e1054ea-1bb6-4b9b-9393-26fc530fdc71" (UID: "5e1054ea-1bb6-4b9b-9393-26fc530fdc71"). InnerVolumeSpecName "kube-api-access-99wqk". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:57:56 crc kubenswrapper[5052]: I1203 12:57:56.159099 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5e1054ea-1bb6-4b9b-9393-26fc530fdc71-inventory-0" (OuterVolumeSpecName: "inventory-0") pod "5e1054ea-1bb6-4b9b-9393-26fc530fdc71" (UID: "5e1054ea-1bb6-4b9b-9393-26fc530fdc71"). InnerVolumeSpecName "inventory-0". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:57:56 crc kubenswrapper[5052]: I1203 12:57:56.170259 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5e1054ea-1bb6-4b9b-9393-26fc530fdc71-ssh-key-edpm-compute-global" (OuterVolumeSpecName: "ssh-key-edpm-compute-global") pod "5e1054ea-1bb6-4b9b-9393-26fc530fdc71" (UID: "5e1054ea-1bb6-4b9b-9393-26fc530fdc71"). InnerVolumeSpecName "ssh-key-edpm-compute-global". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:57:56 crc kubenswrapper[5052]: I1203 12:57:56.241322 5052 reconciler_common.go:293] "Volume detached for volume \"inventory-0\" (UniqueName: \"kubernetes.io/secret/5e1054ea-1bb6-4b9b-9393-26fc530fdc71-inventory-0\") on node \"crc\" DevicePath \"\"" Dec 03 12:57:56 crc kubenswrapper[5052]: I1203 12:57:56.241365 5052 reconciler_common.go:293] "Volume detached for volume \"ssh-key-edpm-compute-global\" (UniqueName: \"kubernetes.io/secret/5e1054ea-1bb6-4b9b-9393-26fc530fdc71-ssh-key-edpm-compute-global\") on node \"crc\" DevicePath \"\"" Dec 03 12:57:56 crc kubenswrapper[5052]: I1203 12:57:56.241383 5052 reconciler_common.go:293] "Volume detached for volume \"custom-global-service-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5e1054ea-1bb6-4b9b-9393-26fc530fdc71-custom-global-service-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:57:56 crc kubenswrapper[5052]: I1203 12:57:56.241401 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-99wqk\" (UniqueName: \"kubernetes.io/projected/5e1054ea-1bb6-4b9b-9393-26fc530fdc71-kube-api-access-99wqk\") on node \"crc\" DevicePath \"\"" Dec 03 12:57:56 crc kubenswrapper[5052]: I1203 12:57:56.746787 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/custom-global-service-edpm-compute-global-ld7cm" event={"ID":"5e1054ea-1bb6-4b9b-9393-26fc530fdc71","Type":"ContainerDied","Data":"3b7eb80ce8b71c25677bbc0f2908bbce09572beb63cc1b58f5546b241ffc79ad"} Dec 03 12:57:56 crc kubenswrapper[5052]: I1203 12:57:56.747129 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3b7eb80ce8b71c25677bbc0f2908bbce09572beb63cc1b58f5546b241ffc79ad" Dec 03 12:57:56 crc kubenswrapper[5052]: I1203 12:57:56.746833 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/custom-global-service-edpm-compute-global-ld7cm" Dec 03 12:57:57 crc kubenswrapper[5052]: I1203 12:57:57.474895 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-6668544499-v8zl2"] Dec 03 12:57:57 crc kubenswrapper[5052]: E1203 12:57:57.476117 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="5e1054ea-1bb6-4b9b-9393-26fc530fdc71" containerName="custom-global-service-edpm-compute-global" Dec 03 12:57:57 crc kubenswrapper[5052]: I1203 12:57:57.476198 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="5e1054ea-1bb6-4b9b-9393-26fc530fdc71" containerName="custom-global-service-edpm-compute-global" Dec 03 12:57:57 crc kubenswrapper[5052]: I1203 12:57:57.476425 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="5e1054ea-1bb6-4b9b-9393-26fc530fdc71" containerName="custom-global-service-edpm-compute-global" Dec 03 12:57:57 crc kubenswrapper[5052]: I1203 12:57:57.477313 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-6668544499-v8zl2" Dec 03 12:57:57 crc kubenswrapper[5052]: I1203 12:57:57.483887 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"edpm-compute-beta-nodeset" Dec 03 12:57:57 crc kubenswrapper[5052]: I1203 12:57:57.489232 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-6668544499-v8zl2"] Dec 03 12:57:57 crc kubenswrapper[5052]: I1203 12:57:57.661892 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"edpm-compute-beta-nodeset\" (UniqueName: \"kubernetes.io/configmap/65b17e9c-5f92-4aa7-96f3-551e062422c0-edpm-compute-beta-nodeset\") pod \"dnsmasq-dnsmasq-6668544499-v8zl2\" (UID: \"65b17e9c-5f92-4aa7-96f3-551e062422c0\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-6668544499-v8zl2" Dec 03 12:57:57 crc kubenswrapper[5052]: I1203 12:57:57.661931 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"edpm-compute-global\" (UniqueName: \"kubernetes.io/configmap/65b17e9c-5f92-4aa7-96f3-551e062422c0-edpm-compute-global\") pod \"dnsmasq-dnsmasq-6668544499-v8zl2\" (UID: \"65b17e9c-5f92-4aa7-96f3-551e062422c0\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-6668544499-v8zl2" Dec 03 12:57:57 crc kubenswrapper[5052]: I1203 12:57:57.662172 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g5vnv\" (UniqueName: \"kubernetes.io/projected/65b17e9c-5f92-4aa7-96f3-551e062422c0-kube-api-access-g5vnv\") pod \"dnsmasq-dnsmasq-6668544499-v8zl2\" (UID: \"65b17e9c-5f92-4aa7-96f3-551e062422c0\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-6668544499-v8zl2" Dec 03 12:57:57 crc kubenswrapper[5052]: I1203 12:57:57.662245 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/65b17e9c-5f92-4aa7-96f3-551e062422c0-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-6668544499-v8zl2\" (UID: \"65b17e9c-5f92-4aa7-96f3-551e062422c0\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-6668544499-v8zl2" Dec 03 12:57:57 crc kubenswrapper[5052]: I1203 12:57:57.662315 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/65b17e9c-5f92-4aa7-96f3-551e062422c0-config\") pod \"dnsmasq-dnsmasq-6668544499-v8zl2\" (UID: \"65b17e9c-5f92-4aa7-96f3-551e062422c0\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-6668544499-v8zl2" Dec 03 12:57:57 crc kubenswrapper[5052]: I1203 12:57:57.763424 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/65b17e9c-5f92-4aa7-96f3-551e062422c0-config\") pod \"dnsmasq-dnsmasq-6668544499-v8zl2\" (UID: \"65b17e9c-5f92-4aa7-96f3-551e062422c0\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-6668544499-v8zl2" Dec 03 12:57:57 crc kubenswrapper[5052]: I1203 12:57:57.763495 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"edpm-compute-beta-nodeset\" (UniqueName: \"kubernetes.io/configmap/65b17e9c-5f92-4aa7-96f3-551e062422c0-edpm-compute-beta-nodeset\") pod \"dnsmasq-dnsmasq-6668544499-v8zl2\" (UID: \"65b17e9c-5f92-4aa7-96f3-551e062422c0\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-6668544499-v8zl2" Dec 03 12:57:57 crc kubenswrapper[5052]: I1203 12:57:57.763543 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"edpm-compute-global\" (UniqueName: \"kubernetes.io/configmap/65b17e9c-5f92-4aa7-96f3-551e062422c0-edpm-compute-global\") pod \"dnsmasq-dnsmasq-6668544499-v8zl2\" (UID: \"65b17e9c-5f92-4aa7-96f3-551e062422c0\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-6668544499-v8zl2" Dec 03 12:57:57 crc kubenswrapper[5052]: I1203 12:57:57.763642 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-g5vnv\" (UniqueName: \"kubernetes.io/projected/65b17e9c-5f92-4aa7-96f3-551e062422c0-kube-api-access-g5vnv\") pod \"dnsmasq-dnsmasq-6668544499-v8zl2\" (UID: \"65b17e9c-5f92-4aa7-96f3-551e062422c0\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-6668544499-v8zl2" Dec 03 12:57:57 crc kubenswrapper[5052]: I1203 12:57:57.763681 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/65b17e9c-5f92-4aa7-96f3-551e062422c0-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-6668544499-v8zl2\" (UID: \"65b17e9c-5f92-4aa7-96f3-551e062422c0\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-6668544499-v8zl2" Dec 03 12:57:57 crc kubenswrapper[5052]: I1203 12:57:57.764412 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"edpm-compute-global\" (UniqueName: \"kubernetes.io/configmap/65b17e9c-5f92-4aa7-96f3-551e062422c0-edpm-compute-global\") pod \"dnsmasq-dnsmasq-6668544499-v8zl2\" (UID: \"65b17e9c-5f92-4aa7-96f3-551e062422c0\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-6668544499-v8zl2" Dec 03 12:57:57 crc kubenswrapper[5052]: I1203 12:57:57.764543 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"edpm-compute-beta-nodeset\" (UniqueName: \"kubernetes.io/configmap/65b17e9c-5f92-4aa7-96f3-551e062422c0-edpm-compute-beta-nodeset\") pod \"dnsmasq-dnsmasq-6668544499-v8zl2\" (UID: \"65b17e9c-5f92-4aa7-96f3-551e062422c0\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-6668544499-v8zl2" Dec 03 12:57:57 crc kubenswrapper[5052]: I1203 12:57:57.764638 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/65b17e9c-5f92-4aa7-96f3-551e062422c0-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-6668544499-v8zl2\" (UID: \"65b17e9c-5f92-4aa7-96f3-551e062422c0\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-6668544499-v8zl2" Dec 03 12:57:57 crc kubenswrapper[5052]: I1203 12:57:57.764694 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/65b17e9c-5f92-4aa7-96f3-551e062422c0-config\") pod \"dnsmasq-dnsmasq-6668544499-v8zl2\" (UID: \"65b17e9c-5f92-4aa7-96f3-551e062422c0\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-6668544499-v8zl2" Dec 03 12:57:57 crc kubenswrapper[5052]: I1203 12:57:57.783346 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-g5vnv\" (UniqueName: \"kubernetes.io/projected/65b17e9c-5f92-4aa7-96f3-551e062422c0-kube-api-access-g5vnv\") pod \"dnsmasq-dnsmasq-6668544499-v8zl2\" (UID: \"65b17e9c-5f92-4aa7-96f3-551e062422c0\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-6668544499-v8zl2" Dec 03 12:57:57 crc kubenswrapper[5052]: I1203 12:57:57.809711 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-6668544499-v8zl2" Dec 03 12:57:58 crc kubenswrapper[5052]: I1203 12:57:58.235024 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-6668544499-v8zl2"] Dec 03 12:57:58 crc kubenswrapper[5052]: I1203 12:57:58.773082 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-6668544499-v8zl2" event={"ID":"65b17e9c-5f92-4aa7-96f3-551e062422c0","Type":"ContainerStarted","Data":"bc3a37a15d42c8b4e27ed24ccaf8f912294dc73effb2c6589078f541c435a777"} Dec 03 12:57:59 crc kubenswrapper[5052]: I1203 12:57:59.680261 5052 scope.go:117] "RemoveContainer" containerID="69fa7fbcd2aceb7c53703a19508876f93a97dc9adabb79c16fb691d90f606d58" Dec 03 12:57:59 crc kubenswrapper[5052]: E1203 12:57:59.680522 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 12:57:59 crc kubenswrapper[5052]: I1203 12:57:59.785640 5052 generic.go:334] "Generic (PLEG): container finished" podID="65b17e9c-5f92-4aa7-96f3-551e062422c0" containerID="773141ca6b74a4d0763489d6ed12290cd36d743efa51afb643252ff6550cf79d" exitCode=0 Dec 03 12:57:59 crc kubenswrapper[5052]: I1203 12:57:59.785685 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-6668544499-v8zl2" event={"ID":"65b17e9c-5f92-4aa7-96f3-551e062422c0","Type":"ContainerDied","Data":"773141ca6b74a4d0763489d6ed12290cd36d743efa51afb643252ff6550cf79d"} Dec 03 12:58:00 crc kubenswrapper[5052]: I1203 12:58:00.797659 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-6668544499-v8zl2" event={"ID":"65b17e9c-5f92-4aa7-96f3-551e062422c0","Type":"ContainerStarted","Data":"57b2bbf195b6e0e32c45017d766ddec938cf76f66e290154f46f5117d33bf1ac"} Dec 03 12:58:00 crc kubenswrapper[5052]: I1203 12:58:00.797953 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-6668544499-v8zl2" Dec 03 12:58:00 crc kubenswrapper[5052]: I1203 12:58:00.820934 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-6668544499-v8zl2" podStartSLOduration=3.8209123099999998 podStartE2EDuration="3.82091231s" podCreationTimestamp="2025-12-03 12:57:57 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:58:00.815887416 +0000 UTC m=+5871.404129170" watchObservedRunningTime="2025-12-03 12:58:00.82091231 +0000 UTC m=+5871.409154114" Dec 03 12:58:07 crc kubenswrapper[5052]: I1203 12:58:07.811358 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-6668544499-v8zl2" Dec 03 12:58:07 crc kubenswrapper[5052]: I1203 12:58:07.901810 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-7d78464677-jj2j9"] Dec 03 12:58:07 crc kubenswrapper[5052]: I1203 12:58:07.902209 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-7d78464677-jj2j9" podUID="916bd1d4-f942-4b7b-8efd-94e63cc84208" containerName="dnsmasq-dns" containerID="cri-o://309819d445a2c8d65f21aab056f5e431c8c087cdafd3b00934fc169d30dda8fa" gracePeriod=10 Dec 03 12:58:08 crc kubenswrapper[5052]: I1203 12:58:08.308499 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-7d78464677-jj2j9" Dec 03 12:58:08 crc kubenswrapper[5052]: I1203 12:58:08.433340 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/916bd1d4-f942-4b7b-8efd-94e63cc84208-dnsmasq-svc\") pod \"916bd1d4-f942-4b7b-8efd-94e63cc84208\" (UID: \"916bd1d4-f942-4b7b-8efd-94e63cc84208\") " Dec 03 12:58:08 crc kubenswrapper[5052]: I1203 12:58:08.433394 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"edpm-compute-global\" (UniqueName: \"kubernetes.io/configmap/916bd1d4-f942-4b7b-8efd-94e63cc84208-edpm-compute-global\") pod \"916bd1d4-f942-4b7b-8efd-94e63cc84208\" (UID: \"916bd1d4-f942-4b7b-8efd-94e63cc84208\") " Dec 03 12:58:08 crc kubenswrapper[5052]: I1203 12:58:08.433443 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/916bd1d4-f942-4b7b-8efd-94e63cc84208-config\") pod \"916bd1d4-f942-4b7b-8efd-94e63cc84208\" (UID: \"916bd1d4-f942-4b7b-8efd-94e63cc84208\") " Dec 03 12:58:08 crc kubenswrapper[5052]: I1203 12:58:08.433646 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wxrwc\" (UniqueName: \"kubernetes.io/projected/916bd1d4-f942-4b7b-8efd-94e63cc84208-kube-api-access-wxrwc\") pod \"916bd1d4-f942-4b7b-8efd-94e63cc84208\" (UID: \"916bd1d4-f942-4b7b-8efd-94e63cc84208\") " Dec 03 12:58:08 crc kubenswrapper[5052]: I1203 12:58:08.450654 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/916bd1d4-f942-4b7b-8efd-94e63cc84208-kube-api-access-wxrwc" (OuterVolumeSpecName: "kube-api-access-wxrwc") pod "916bd1d4-f942-4b7b-8efd-94e63cc84208" (UID: "916bd1d4-f942-4b7b-8efd-94e63cc84208"). InnerVolumeSpecName "kube-api-access-wxrwc". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:58:08 crc kubenswrapper[5052]: I1203 12:58:08.472155 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/916bd1d4-f942-4b7b-8efd-94e63cc84208-dnsmasq-svc" (OuterVolumeSpecName: "dnsmasq-svc") pod "916bd1d4-f942-4b7b-8efd-94e63cc84208" (UID: "916bd1d4-f942-4b7b-8efd-94e63cc84208"). InnerVolumeSpecName "dnsmasq-svc". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:58:08 crc kubenswrapper[5052]: I1203 12:58:08.476292 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/916bd1d4-f942-4b7b-8efd-94e63cc84208-edpm-compute-global" (OuterVolumeSpecName: "edpm-compute-global") pod "916bd1d4-f942-4b7b-8efd-94e63cc84208" (UID: "916bd1d4-f942-4b7b-8efd-94e63cc84208"). InnerVolumeSpecName "edpm-compute-global". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:58:08 crc kubenswrapper[5052]: I1203 12:58:08.481290 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/916bd1d4-f942-4b7b-8efd-94e63cc84208-config" (OuterVolumeSpecName: "config") pod "916bd1d4-f942-4b7b-8efd-94e63cc84208" (UID: "916bd1d4-f942-4b7b-8efd-94e63cc84208"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:58:08 crc kubenswrapper[5052]: I1203 12:58:08.535207 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-wxrwc\" (UniqueName: \"kubernetes.io/projected/916bd1d4-f942-4b7b-8efd-94e63cc84208-kube-api-access-wxrwc\") on node \"crc\" DevicePath \"\"" Dec 03 12:58:08 crc kubenswrapper[5052]: I1203 12:58:08.535238 5052 reconciler_common.go:293] "Volume detached for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/916bd1d4-f942-4b7b-8efd-94e63cc84208-dnsmasq-svc\") on node \"crc\" DevicePath \"\"" Dec 03 12:58:08 crc kubenswrapper[5052]: I1203 12:58:08.535248 5052 reconciler_common.go:293] "Volume detached for volume \"edpm-compute-global\" (UniqueName: \"kubernetes.io/configmap/916bd1d4-f942-4b7b-8efd-94e63cc84208-edpm-compute-global\") on node \"crc\" DevicePath \"\"" Dec 03 12:58:08 crc kubenswrapper[5052]: I1203 12:58:08.535258 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/916bd1d4-f942-4b7b-8efd-94e63cc84208-config\") on node \"crc\" DevicePath \"\"" Dec 03 12:58:08 crc kubenswrapper[5052]: I1203 12:58:08.893207 5052 generic.go:334] "Generic (PLEG): container finished" podID="916bd1d4-f942-4b7b-8efd-94e63cc84208" containerID="309819d445a2c8d65f21aab056f5e431c8c087cdafd3b00934fc169d30dda8fa" exitCode=0 Dec 03 12:58:08 crc kubenswrapper[5052]: I1203 12:58:08.893250 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-7d78464677-jj2j9" event={"ID":"916bd1d4-f942-4b7b-8efd-94e63cc84208","Type":"ContainerDied","Data":"309819d445a2c8d65f21aab056f5e431c8c087cdafd3b00934fc169d30dda8fa"} Dec 03 12:58:08 crc kubenswrapper[5052]: I1203 12:58:08.893277 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-7d78464677-jj2j9" event={"ID":"916bd1d4-f942-4b7b-8efd-94e63cc84208","Type":"ContainerDied","Data":"e86e7f53b2d887287605094fc27bb6874b8e78b07816242a428ab81c68878a53"} Dec 03 12:58:08 crc kubenswrapper[5052]: I1203 12:58:08.893294 5052 scope.go:117] "RemoveContainer" containerID="309819d445a2c8d65f21aab056f5e431c8c087cdafd3b00934fc169d30dda8fa" Dec 03 12:58:08 crc kubenswrapper[5052]: I1203 12:58:08.893413 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-7d78464677-jj2j9" Dec 03 12:58:08 crc kubenswrapper[5052]: I1203 12:58:08.920796 5052 scope.go:117] "RemoveContainer" containerID="3e41d127a12f55e3cf5052b7a21b9b23bae57de90a79214a7348f35c20f30a79" Dec 03 12:58:08 crc kubenswrapper[5052]: I1203 12:58:08.927774 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-7d78464677-jj2j9"] Dec 03 12:58:08 crc kubenswrapper[5052]: I1203 12:58:08.936320 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-7d78464677-jj2j9"] Dec 03 12:58:08 crc kubenswrapper[5052]: I1203 12:58:08.948049 5052 scope.go:117] "RemoveContainer" containerID="309819d445a2c8d65f21aab056f5e431c8c087cdafd3b00934fc169d30dda8fa" Dec 03 12:58:08 crc kubenswrapper[5052]: E1203 12:58:08.948429 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"309819d445a2c8d65f21aab056f5e431c8c087cdafd3b00934fc169d30dda8fa\": container with ID starting with 309819d445a2c8d65f21aab056f5e431c8c087cdafd3b00934fc169d30dda8fa not found: ID does not exist" containerID="309819d445a2c8d65f21aab056f5e431c8c087cdafd3b00934fc169d30dda8fa" Dec 03 12:58:08 crc kubenswrapper[5052]: I1203 12:58:08.948485 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"309819d445a2c8d65f21aab056f5e431c8c087cdafd3b00934fc169d30dda8fa"} err="failed to get container status \"309819d445a2c8d65f21aab056f5e431c8c087cdafd3b00934fc169d30dda8fa\": rpc error: code = NotFound desc = could not find container \"309819d445a2c8d65f21aab056f5e431c8c087cdafd3b00934fc169d30dda8fa\": container with ID starting with 309819d445a2c8d65f21aab056f5e431c8c087cdafd3b00934fc169d30dda8fa not found: ID does not exist" Dec 03 12:58:08 crc kubenswrapper[5052]: I1203 12:58:08.948571 5052 scope.go:117] "RemoveContainer" containerID="3e41d127a12f55e3cf5052b7a21b9b23bae57de90a79214a7348f35c20f30a79" Dec 03 12:58:08 crc kubenswrapper[5052]: E1203 12:58:08.948881 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3e41d127a12f55e3cf5052b7a21b9b23bae57de90a79214a7348f35c20f30a79\": container with ID starting with 3e41d127a12f55e3cf5052b7a21b9b23bae57de90a79214a7348f35c20f30a79 not found: ID does not exist" containerID="3e41d127a12f55e3cf5052b7a21b9b23bae57de90a79214a7348f35c20f30a79" Dec 03 12:58:08 crc kubenswrapper[5052]: I1203 12:58:08.948922 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3e41d127a12f55e3cf5052b7a21b9b23bae57de90a79214a7348f35c20f30a79"} err="failed to get container status \"3e41d127a12f55e3cf5052b7a21b9b23bae57de90a79214a7348f35c20f30a79\": rpc error: code = NotFound desc = could not find container \"3e41d127a12f55e3cf5052b7a21b9b23bae57de90a79214a7348f35c20f30a79\": container with ID starting with 3e41d127a12f55e3cf5052b7a21b9b23bae57de90a79214a7348f35c20f30a79 not found: ID does not exist" Dec 03 12:58:09 crc kubenswrapper[5052]: I1203 12:58:09.683418 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="916bd1d4-f942-4b7b-8efd-94e63cc84208" path="/var/lib/kubelet/pods/916bd1d4-f942-4b7b-8efd-94e63cc84208/volumes" Dec 03 12:58:12 crc kubenswrapper[5052]: I1203 12:58:12.458485 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/download-cache-edpm-multinodeset-edpm-compute-global-kfr9b"] Dec 03 12:58:12 crc kubenswrapper[5052]: E1203 12:58:12.459182 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="916bd1d4-f942-4b7b-8efd-94e63cc84208" containerName="dnsmasq-dns" Dec 03 12:58:12 crc kubenswrapper[5052]: I1203 12:58:12.459198 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="916bd1d4-f942-4b7b-8efd-94e63cc84208" containerName="dnsmasq-dns" Dec 03 12:58:12 crc kubenswrapper[5052]: E1203 12:58:12.459214 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="916bd1d4-f942-4b7b-8efd-94e63cc84208" containerName="init" Dec 03 12:58:12 crc kubenswrapper[5052]: I1203 12:58:12.459221 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="916bd1d4-f942-4b7b-8efd-94e63cc84208" containerName="init" Dec 03 12:58:12 crc kubenswrapper[5052]: I1203 12:58:12.459398 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="916bd1d4-f942-4b7b-8efd-94e63cc84208" containerName="dnsmasq-dns" Dec 03 12:58:12 crc kubenswrapper[5052]: I1203 12:58:12.459959 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/download-cache-edpm-multinodeset-edpm-compute-global-kfr9b" Dec 03 12:58:12 crc kubenswrapper[5052]: I1203 12:58:12.474549 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/download-cache-edpm-multinodeset-edpm-compute-beta-nodesetsnvv9"] Dec 03 12:58:12 crc kubenswrapper[5052]: I1203 12:58:12.475308 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/download-cache-edpm-multinodeset-edpm-compute-global-kfr9b"] Dec 03 12:58:12 crc kubenswrapper[5052]: I1203 12:58:12.475379 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/download-cache-edpm-multinodeset-edpm-compute-beta-nodesetsnvv9" Dec 03 12:58:12 crc kubenswrapper[5052]: I1203 12:58:12.493117 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"openstack-aee-default-env" Dec 03 12:58:12 crc kubenswrapper[5052]: I1203 12:58:12.493133 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"edpm-compute-global-dockercfg-p5rjx" Dec 03 12:58:12 crc kubenswrapper[5052]: I1203 12:58:12.493199 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"edpm-compute-beta-nodeset-dockercfg-mzvt6" Dec 03 12:58:12 crc kubenswrapper[5052]: I1203 12:58:12.493326 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"dataplanenodeset-edpm-compute-beta-nodeset" Dec 03 12:58:12 crc kubenswrapper[5052]: I1203 12:58:12.493451 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"dataplanenodeset-edpm-compute-global" Dec 03 12:58:12 crc kubenswrapper[5052]: I1203 12:58:12.493672 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"dataplane-ansible-ssh-private-key-secret" Dec 03 12:58:12 crc kubenswrapper[5052]: I1203 12:58:12.504922 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/download-cache-edpm-multinodeset-edpm-compute-beta-nodesetsnvv9"] Dec 03 12:58:12 crc kubenswrapper[5052]: I1203 12:58:12.507692 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/25ab396c-60f7-47e7-a923-709074e77695-ssh-key\") pod \"download-cache-edpm-multinodeset-edpm-compute-global-kfr9b\" (UID: \"25ab396c-60f7-47e7-a923-709074e77695\") " pod="openstack-kuttl-tests/download-cache-edpm-multinodeset-edpm-compute-global-kfr9b" Dec 03 12:58:12 crc kubenswrapper[5052]: I1203 12:58:12.507796 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xqq5f\" (UniqueName: \"kubernetes.io/projected/ca163200-a99d-41cf-972e-862a71328826-kube-api-access-xqq5f\") pod \"download-cache-edpm-multinodeset-edpm-compute-beta-nodesetsnvv9\" (UID: \"ca163200-a99d-41cf-972e-862a71328826\") " pod="openstack-kuttl-tests/download-cache-edpm-multinodeset-edpm-compute-beta-nodesetsnvv9" Dec 03 12:58:12 crc kubenswrapper[5052]: I1203 12:58:12.507829 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/ca163200-a99d-41cf-972e-862a71328826-inventory\") pod \"download-cache-edpm-multinodeset-edpm-compute-beta-nodesetsnvv9\" (UID: \"ca163200-a99d-41cf-972e-862a71328826\") " pod="openstack-kuttl-tests/download-cache-edpm-multinodeset-edpm-compute-beta-nodesetsnvv9" Dec 03 12:58:12 crc kubenswrapper[5052]: I1203 12:58:12.507873 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/25ab396c-60f7-47e7-a923-709074e77695-inventory\") pod \"download-cache-edpm-multinodeset-edpm-compute-global-kfr9b\" (UID: \"25ab396c-60f7-47e7-a923-709074e77695\") " pod="openstack-kuttl-tests/download-cache-edpm-multinodeset-edpm-compute-global-kfr9b" Dec 03 12:58:12 crc kubenswrapper[5052]: I1203 12:58:12.507900 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rbk6x\" (UniqueName: \"kubernetes.io/projected/25ab396c-60f7-47e7-a923-709074e77695-kube-api-access-rbk6x\") pod \"download-cache-edpm-multinodeset-edpm-compute-global-kfr9b\" (UID: \"25ab396c-60f7-47e7-a923-709074e77695\") " pod="openstack-kuttl-tests/download-cache-edpm-multinodeset-edpm-compute-global-kfr9b" Dec 03 12:58:12 crc kubenswrapper[5052]: I1203 12:58:12.508006 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/ca163200-a99d-41cf-972e-862a71328826-ssh-key\") pod \"download-cache-edpm-multinodeset-edpm-compute-beta-nodesetsnvv9\" (UID: \"ca163200-a99d-41cf-972e-862a71328826\") " pod="openstack-kuttl-tests/download-cache-edpm-multinodeset-edpm-compute-beta-nodesetsnvv9" Dec 03 12:58:12 crc kubenswrapper[5052]: I1203 12:58:12.609168 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/ca163200-a99d-41cf-972e-862a71328826-ssh-key\") pod \"download-cache-edpm-multinodeset-edpm-compute-beta-nodesetsnvv9\" (UID: \"ca163200-a99d-41cf-972e-862a71328826\") " pod="openstack-kuttl-tests/download-cache-edpm-multinodeset-edpm-compute-beta-nodesetsnvv9" Dec 03 12:58:12 crc kubenswrapper[5052]: I1203 12:58:12.609225 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/25ab396c-60f7-47e7-a923-709074e77695-ssh-key\") pod \"download-cache-edpm-multinodeset-edpm-compute-global-kfr9b\" (UID: \"25ab396c-60f7-47e7-a923-709074e77695\") " pod="openstack-kuttl-tests/download-cache-edpm-multinodeset-edpm-compute-global-kfr9b" Dec 03 12:58:12 crc kubenswrapper[5052]: I1203 12:58:12.609249 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-xqq5f\" (UniqueName: \"kubernetes.io/projected/ca163200-a99d-41cf-972e-862a71328826-kube-api-access-xqq5f\") pod \"download-cache-edpm-multinodeset-edpm-compute-beta-nodesetsnvv9\" (UID: \"ca163200-a99d-41cf-972e-862a71328826\") " pod="openstack-kuttl-tests/download-cache-edpm-multinodeset-edpm-compute-beta-nodesetsnvv9" Dec 03 12:58:12 crc kubenswrapper[5052]: I1203 12:58:12.609267 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/ca163200-a99d-41cf-972e-862a71328826-inventory\") pod \"download-cache-edpm-multinodeset-edpm-compute-beta-nodesetsnvv9\" (UID: \"ca163200-a99d-41cf-972e-862a71328826\") " pod="openstack-kuttl-tests/download-cache-edpm-multinodeset-edpm-compute-beta-nodesetsnvv9" Dec 03 12:58:12 crc kubenswrapper[5052]: I1203 12:58:12.609291 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/25ab396c-60f7-47e7-a923-709074e77695-inventory\") pod \"download-cache-edpm-multinodeset-edpm-compute-global-kfr9b\" (UID: \"25ab396c-60f7-47e7-a923-709074e77695\") " pod="openstack-kuttl-tests/download-cache-edpm-multinodeset-edpm-compute-global-kfr9b" Dec 03 12:58:12 crc kubenswrapper[5052]: I1203 12:58:12.609307 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-rbk6x\" (UniqueName: \"kubernetes.io/projected/25ab396c-60f7-47e7-a923-709074e77695-kube-api-access-rbk6x\") pod \"download-cache-edpm-multinodeset-edpm-compute-global-kfr9b\" (UID: \"25ab396c-60f7-47e7-a923-709074e77695\") " pod="openstack-kuttl-tests/download-cache-edpm-multinodeset-edpm-compute-global-kfr9b" Dec 03 12:58:12 crc kubenswrapper[5052]: I1203 12:58:12.614161 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/25ab396c-60f7-47e7-a923-709074e77695-inventory\") pod \"download-cache-edpm-multinodeset-edpm-compute-global-kfr9b\" (UID: \"25ab396c-60f7-47e7-a923-709074e77695\") " pod="openstack-kuttl-tests/download-cache-edpm-multinodeset-edpm-compute-global-kfr9b" Dec 03 12:58:12 crc kubenswrapper[5052]: I1203 12:58:12.614504 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/ca163200-a99d-41cf-972e-862a71328826-inventory\") pod \"download-cache-edpm-multinodeset-edpm-compute-beta-nodesetsnvv9\" (UID: \"ca163200-a99d-41cf-972e-862a71328826\") " pod="openstack-kuttl-tests/download-cache-edpm-multinodeset-edpm-compute-beta-nodesetsnvv9" Dec 03 12:58:12 crc kubenswrapper[5052]: I1203 12:58:12.614702 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/ca163200-a99d-41cf-972e-862a71328826-ssh-key\") pod \"download-cache-edpm-multinodeset-edpm-compute-beta-nodesetsnvv9\" (UID: \"ca163200-a99d-41cf-972e-862a71328826\") " pod="openstack-kuttl-tests/download-cache-edpm-multinodeset-edpm-compute-beta-nodesetsnvv9" Dec 03 12:58:12 crc kubenswrapper[5052]: I1203 12:58:12.618991 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/25ab396c-60f7-47e7-a923-709074e77695-ssh-key\") pod \"download-cache-edpm-multinodeset-edpm-compute-global-kfr9b\" (UID: \"25ab396c-60f7-47e7-a923-709074e77695\") " pod="openstack-kuttl-tests/download-cache-edpm-multinodeset-edpm-compute-global-kfr9b" Dec 03 12:58:12 crc kubenswrapper[5052]: I1203 12:58:12.625441 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-rbk6x\" (UniqueName: \"kubernetes.io/projected/25ab396c-60f7-47e7-a923-709074e77695-kube-api-access-rbk6x\") pod \"download-cache-edpm-multinodeset-edpm-compute-global-kfr9b\" (UID: \"25ab396c-60f7-47e7-a923-709074e77695\") " pod="openstack-kuttl-tests/download-cache-edpm-multinodeset-edpm-compute-global-kfr9b" Dec 03 12:58:12 crc kubenswrapper[5052]: I1203 12:58:12.626851 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-xqq5f\" (UniqueName: \"kubernetes.io/projected/ca163200-a99d-41cf-972e-862a71328826-kube-api-access-xqq5f\") pod \"download-cache-edpm-multinodeset-edpm-compute-beta-nodesetsnvv9\" (UID: \"ca163200-a99d-41cf-972e-862a71328826\") " pod="openstack-kuttl-tests/download-cache-edpm-multinodeset-edpm-compute-beta-nodesetsnvv9" Dec 03 12:58:12 crc kubenswrapper[5052]: I1203 12:58:12.672597 5052 scope.go:117] "RemoveContainer" containerID="69fa7fbcd2aceb7c53703a19508876f93a97dc9adabb79c16fb691d90f606d58" Dec 03 12:58:12 crc kubenswrapper[5052]: E1203 12:58:12.672845 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 12:58:12 crc kubenswrapper[5052]: I1203 12:58:12.807103 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/download-cache-edpm-multinodeset-edpm-compute-global-kfr9b" Dec 03 12:58:12 crc kubenswrapper[5052]: I1203 12:58:12.828569 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/download-cache-edpm-multinodeset-edpm-compute-beta-nodesetsnvv9" Dec 03 12:58:13 crc kubenswrapper[5052]: I1203 12:58:13.228729 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/download-cache-edpm-multinodeset-edpm-compute-global-kfr9b"] Dec 03 12:58:13 crc kubenswrapper[5052]: I1203 12:58:13.297174 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/download-cache-edpm-multinodeset-edpm-compute-beta-nodesetsnvv9"] Dec 03 12:58:13 crc kubenswrapper[5052]: W1203 12:58:13.302416 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podca163200_a99d_41cf_972e_862a71328826.slice/crio-fb679455ef92af133fba9d17e92a8663e3248e9d71bb9903478b09b0df961dab WatchSource:0}: Error finding container fb679455ef92af133fba9d17e92a8663e3248e9d71bb9903478b09b0df961dab: Status 404 returned error can't find the container with id fb679455ef92af133fba9d17e92a8663e3248e9d71bb9903478b09b0df961dab Dec 03 12:58:13 crc kubenswrapper[5052]: I1203 12:58:13.953409 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/download-cache-edpm-multinodeset-edpm-compute-beta-nodesetsnvv9" event={"ID":"ca163200-a99d-41cf-972e-862a71328826","Type":"ContainerStarted","Data":"fb679455ef92af133fba9d17e92a8663e3248e9d71bb9903478b09b0df961dab"} Dec 03 12:58:13 crc kubenswrapper[5052]: I1203 12:58:13.957652 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/download-cache-edpm-multinodeset-edpm-compute-global-kfr9b" event={"ID":"25ab396c-60f7-47e7-a923-709074e77695","Type":"ContainerStarted","Data":"e1bcda152b6f19e2894a12a93e3f243905894ce4538dd8e159bff2fcdc72b96a"} Dec 03 12:58:14 crc kubenswrapper[5052]: I1203 12:58:14.966806 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/download-cache-edpm-multinodeset-edpm-compute-beta-nodesetsnvv9" event={"ID":"ca163200-a99d-41cf-972e-862a71328826","Type":"ContainerStarted","Data":"2dbf38c62ebda73087ee02131872825f86e7d69a9f2119ca9bc3dfc54aeaa021"} Dec 03 12:58:14 crc kubenswrapper[5052]: I1203 12:58:14.968603 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/download-cache-edpm-multinodeset-edpm-compute-global-kfr9b" event={"ID":"25ab396c-60f7-47e7-a923-709074e77695","Type":"ContainerStarted","Data":"2a81f6bb9846792fbc11645aa0fd014b92a33f1f4c49de20cbbcdd0f447029af"} Dec 03 12:58:14 crc kubenswrapper[5052]: I1203 12:58:14.990076 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/download-cache-edpm-multinodeset-edpm-compute-beta-nodesetsnvv9" podStartSLOduration=2.433726213 podStartE2EDuration="2.990055962s" podCreationTimestamp="2025-12-03 12:58:12 +0000 UTC" firstStartedPulling="2025-12-03 12:58:13.304759696 +0000 UTC m=+5883.893001460" lastFinishedPulling="2025-12-03 12:58:13.861089445 +0000 UTC m=+5884.449331209" observedRunningTime="2025-12-03 12:58:14.982544797 +0000 UTC m=+5885.570786581" watchObservedRunningTime="2025-12-03 12:58:14.990055962 +0000 UTC m=+5885.578297726" Dec 03 12:58:15 crc kubenswrapper[5052]: I1203 12:58:15.001522 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/download-cache-edpm-multinodeset-edpm-compute-global-kfr9b" podStartSLOduration=2.483034813 podStartE2EDuration="3.001480439s" podCreationTimestamp="2025-12-03 12:58:12 +0000 UTC" firstStartedPulling="2025-12-03 12:58:13.236130243 +0000 UTC m=+5883.824372007" lastFinishedPulling="2025-12-03 12:58:13.754575829 +0000 UTC m=+5884.342817633" observedRunningTime="2025-12-03 12:58:14.998233156 +0000 UTC m=+5885.586474920" watchObservedRunningTime="2025-12-03 12:58:15.001480439 +0000 UTC m=+5885.589722203" Dec 03 12:58:15 crc kubenswrapper[5052]: I1203 12:58:15.978394 5052 generic.go:334] "Generic (PLEG): container finished" podID="ca163200-a99d-41cf-972e-862a71328826" containerID="2dbf38c62ebda73087ee02131872825f86e7d69a9f2119ca9bc3dfc54aeaa021" exitCode=0 Dec 03 12:58:15 crc kubenswrapper[5052]: I1203 12:58:15.978501 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/download-cache-edpm-multinodeset-edpm-compute-beta-nodesetsnvv9" event={"ID":"ca163200-a99d-41cf-972e-862a71328826","Type":"ContainerDied","Data":"2dbf38c62ebda73087ee02131872825f86e7d69a9f2119ca9bc3dfc54aeaa021"} Dec 03 12:58:15 crc kubenswrapper[5052]: I1203 12:58:15.980433 5052 generic.go:334] "Generic (PLEG): container finished" podID="25ab396c-60f7-47e7-a923-709074e77695" containerID="2a81f6bb9846792fbc11645aa0fd014b92a33f1f4c49de20cbbcdd0f447029af" exitCode=0 Dec 03 12:58:15 crc kubenswrapper[5052]: I1203 12:58:15.980462 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/download-cache-edpm-multinodeset-edpm-compute-global-kfr9b" event={"ID":"25ab396c-60f7-47e7-a923-709074e77695","Type":"ContainerDied","Data":"2a81f6bb9846792fbc11645aa0fd014b92a33f1f4c49de20cbbcdd0f447029af"} Dec 03 12:58:17 crc kubenswrapper[5052]: I1203 12:58:17.300609 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/download-cache-edpm-multinodeset-edpm-compute-beta-nodesetsnvv9" Dec 03 12:58:17 crc kubenswrapper[5052]: I1203 12:58:17.308168 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/download-cache-edpm-multinodeset-edpm-compute-global-kfr9b" Dec 03 12:58:17 crc kubenswrapper[5052]: I1203 12:58:17.475435 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/ca163200-a99d-41cf-972e-862a71328826-inventory\") pod \"ca163200-a99d-41cf-972e-862a71328826\" (UID: \"ca163200-a99d-41cf-972e-862a71328826\") " Dec 03 12:58:17 crc kubenswrapper[5052]: I1203 12:58:17.475504 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/25ab396c-60f7-47e7-a923-709074e77695-inventory\") pod \"25ab396c-60f7-47e7-a923-709074e77695\" (UID: \"25ab396c-60f7-47e7-a923-709074e77695\") " Dec 03 12:58:17 crc kubenswrapper[5052]: I1203 12:58:17.475559 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/ca163200-a99d-41cf-972e-862a71328826-ssh-key\") pod \"ca163200-a99d-41cf-972e-862a71328826\" (UID: \"ca163200-a99d-41cf-972e-862a71328826\") " Dec 03 12:58:17 crc kubenswrapper[5052]: I1203 12:58:17.475600 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rbk6x\" (UniqueName: \"kubernetes.io/projected/25ab396c-60f7-47e7-a923-709074e77695-kube-api-access-rbk6x\") pod \"25ab396c-60f7-47e7-a923-709074e77695\" (UID: \"25ab396c-60f7-47e7-a923-709074e77695\") " Dec 03 12:58:17 crc kubenswrapper[5052]: I1203 12:58:17.475633 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/25ab396c-60f7-47e7-a923-709074e77695-ssh-key\") pod \"25ab396c-60f7-47e7-a923-709074e77695\" (UID: \"25ab396c-60f7-47e7-a923-709074e77695\") " Dec 03 12:58:17 crc kubenswrapper[5052]: I1203 12:58:17.475714 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xqq5f\" (UniqueName: \"kubernetes.io/projected/ca163200-a99d-41cf-972e-862a71328826-kube-api-access-xqq5f\") pod \"ca163200-a99d-41cf-972e-862a71328826\" (UID: \"ca163200-a99d-41cf-972e-862a71328826\") " Dec 03 12:58:17 crc kubenswrapper[5052]: I1203 12:58:17.489537 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/25ab396c-60f7-47e7-a923-709074e77695-kube-api-access-rbk6x" (OuterVolumeSpecName: "kube-api-access-rbk6x") pod "25ab396c-60f7-47e7-a923-709074e77695" (UID: "25ab396c-60f7-47e7-a923-709074e77695"). InnerVolumeSpecName "kube-api-access-rbk6x". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:58:17 crc kubenswrapper[5052]: I1203 12:58:17.490727 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ca163200-a99d-41cf-972e-862a71328826-kube-api-access-xqq5f" (OuterVolumeSpecName: "kube-api-access-xqq5f") pod "ca163200-a99d-41cf-972e-862a71328826" (UID: "ca163200-a99d-41cf-972e-862a71328826"). InnerVolumeSpecName "kube-api-access-xqq5f". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:58:17 crc kubenswrapper[5052]: I1203 12:58:17.499725 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/25ab396c-60f7-47e7-a923-709074e77695-ssh-key" (OuterVolumeSpecName: "ssh-key") pod "25ab396c-60f7-47e7-a923-709074e77695" (UID: "25ab396c-60f7-47e7-a923-709074e77695"). InnerVolumeSpecName "ssh-key". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:58:17 crc kubenswrapper[5052]: I1203 12:58:17.504475 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ca163200-a99d-41cf-972e-862a71328826-ssh-key" (OuterVolumeSpecName: "ssh-key") pod "ca163200-a99d-41cf-972e-862a71328826" (UID: "ca163200-a99d-41cf-972e-862a71328826"). InnerVolumeSpecName "ssh-key". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:58:17 crc kubenswrapper[5052]: I1203 12:58:17.506709 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/25ab396c-60f7-47e7-a923-709074e77695-inventory" (OuterVolumeSpecName: "inventory") pod "25ab396c-60f7-47e7-a923-709074e77695" (UID: "25ab396c-60f7-47e7-a923-709074e77695"). InnerVolumeSpecName "inventory". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:58:17 crc kubenswrapper[5052]: I1203 12:58:17.512608 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ca163200-a99d-41cf-972e-862a71328826-inventory" (OuterVolumeSpecName: "inventory") pod "ca163200-a99d-41cf-972e-862a71328826" (UID: "ca163200-a99d-41cf-972e-862a71328826"). InnerVolumeSpecName "inventory". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:58:17 crc kubenswrapper[5052]: I1203 12:58:17.577884 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-xqq5f\" (UniqueName: \"kubernetes.io/projected/ca163200-a99d-41cf-972e-862a71328826-kube-api-access-xqq5f\") on node \"crc\" DevicePath \"\"" Dec 03 12:58:17 crc kubenswrapper[5052]: I1203 12:58:17.577923 5052 reconciler_common.go:293] "Volume detached for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/ca163200-a99d-41cf-972e-862a71328826-inventory\") on node \"crc\" DevicePath \"\"" Dec 03 12:58:17 crc kubenswrapper[5052]: I1203 12:58:17.577934 5052 reconciler_common.go:293] "Volume detached for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/25ab396c-60f7-47e7-a923-709074e77695-inventory\") on node \"crc\" DevicePath \"\"" Dec 03 12:58:17 crc kubenswrapper[5052]: I1203 12:58:17.577942 5052 reconciler_common.go:293] "Volume detached for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/ca163200-a99d-41cf-972e-862a71328826-ssh-key\") on node \"crc\" DevicePath \"\"" Dec 03 12:58:17 crc kubenswrapper[5052]: I1203 12:58:17.577952 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-rbk6x\" (UniqueName: \"kubernetes.io/projected/25ab396c-60f7-47e7-a923-709074e77695-kube-api-access-rbk6x\") on node \"crc\" DevicePath \"\"" Dec 03 12:58:17 crc kubenswrapper[5052]: I1203 12:58:17.577961 5052 reconciler_common.go:293] "Volume detached for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/25ab396c-60f7-47e7-a923-709074e77695-ssh-key\") on node \"crc\" DevicePath \"\"" Dec 03 12:58:18 crc kubenswrapper[5052]: I1203 12:58:18.000753 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/download-cache-edpm-multinodeset-edpm-compute-beta-nodesetsnvv9" event={"ID":"ca163200-a99d-41cf-972e-862a71328826","Type":"ContainerDied","Data":"fb679455ef92af133fba9d17e92a8663e3248e9d71bb9903478b09b0df961dab"} Dec 03 12:58:18 crc kubenswrapper[5052]: I1203 12:58:18.000794 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fb679455ef92af133fba9d17e92a8663e3248e9d71bb9903478b09b0df961dab" Dec 03 12:58:18 crc kubenswrapper[5052]: I1203 12:58:18.000850 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/download-cache-edpm-multinodeset-edpm-compute-beta-nodesetsnvv9" Dec 03 12:58:18 crc kubenswrapper[5052]: I1203 12:58:18.003639 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/download-cache-edpm-multinodeset-edpm-compute-global-kfr9b" event={"ID":"25ab396c-60f7-47e7-a923-709074e77695","Type":"ContainerDied","Data":"e1bcda152b6f19e2894a12a93e3f243905894ce4538dd8e159bff2fcdc72b96a"} Dec 03 12:58:18 crc kubenswrapper[5052]: I1203 12:58:18.003847 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e1bcda152b6f19e2894a12a93e3f243905894ce4538dd8e159bff2fcdc72b96a" Dec 03 12:58:18 crc kubenswrapper[5052]: I1203 12:58:18.004007 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/download-cache-edpm-multinodeset-edpm-compute-global-kfr9b" Dec 03 12:58:18 crc kubenswrapper[5052]: I1203 12:58:18.071685 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/bootstrap-edpm-multinodeset-edpm-compute-beta-nodeset-fbpnq"] Dec 03 12:58:18 crc kubenswrapper[5052]: E1203 12:58:18.072057 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="25ab396c-60f7-47e7-a923-709074e77695" containerName="download-cache-edpm-multinodeset-edpm-compute-global" Dec 03 12:58:18 crc kubenswrapper[5052]: I1203 12:58:18.072081 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="25ab396c-60f7-47e7-a923-709074e77695" containerName="download-cache-edpm-multinodeset-edpm-compute-global" Dec 03 12:58:18 crc kubenswrapper[5052]: E1203 12:58:18.072101 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="ca163200-a99d-41cf-972e-862a71328826" containerName="download-cache-edpm-multinodeset-edpm-compute-beta-nodeset" Dec 03 12:58:18 crc kubenswrapper[5052]: I1203 12:58:18.072113 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="ca163200-a99d-41cf-972e-862a71328826" containerName="download-cache-edpm-multinodeset-edpm-compute-beta-nodeset" Dec 03 12:58:18 crc kubenswrapper[5052]: I1203 12:58:18.072303 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="ca163200-a99d-41cf-972e-862a71328826" containerName="download-cache-edpm-multinodeset-edpm-compute-beta-nodeset" Dec 03 12:58:18 crc kubenswrapper[5052]: I1203 12:58:18.072328 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="25ab396c-60f7-47e7-a923-709074e77695" containerName="download-cache-edpm-multinodeset-edpm-compute-global" Dec 03 12:58:18 crc kubenswrapper[5052]: I1203 12:58:18.072933 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/bootstrap-edpm-multinodeset-edpm-compute-beta-nodeset-fbpnq" Dec 03 12:58:18 crc kubenswrapper[5052]: I1203 12:58:18.077537 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"openstack-aee-default-env" Dec 03 12:58:18 crc kubenswrapper[5052]: I1203 12:58:18.077937 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"dataplanenodeset-edpm-compute-beta-nodeset" Dec 03 12:58:18 crc kubenswrapper[5052]: I1203 12:58:18.078035 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"dataplane-ansible-ssh-private-key-secret" Dec 03 12:58:18 crc kubenswrapper[5052]: I1203 12:58:18.078807 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"combined-ca-bundle" Dec 03 12:58:18 crc kubenswrapper[5052]: I1203 12:58:18.081774 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"edpm-compute-beta-nodeset-dockercfg-mzvt6" Dec 03 12:58:18 crc kubenswrapper[5052]: I1203 12:58:18.084330 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/bootstrap-edpm-multinodeset-edpm-compute-beta-nodeset-fbpnq"] Dec 03 12:58:18 crc kubenswrapper[5052]: I1203 12:58:18.164845 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/bootstrap-edpm-multinodeset-edpm-compute-global-tq75f"] Dec 03 12:58:18 crc kubenswrapper[5052]: I1203 12:58:18.166093 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/bootstrap-edpm-multinodeset-edpm-compute-global-tq75f" Dec 03 12:58:18 crc kubenswrapper[5052]: I1203 12:58:18.168427 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"dataplanenodeset-edpm-compute-global" Dec 03 12:58:18 crc kubenswrapper[5052]: I1203 12:58:18.169771 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"edpm-compute-global-dockercfg-p5rjx" Dec 03 12:58:18 crc kubenswrapper[5052]: I1203 12:58:18.174162 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/bootstrap-edpm-multinodeset-edpm-compute-global-tq75f"] Dec 03 12:58:18 crc kubenswrapper[5052]: I1203 12:58:18.186214 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n4v4z\" (UniqueName: \"kubernetes.io/projected/f1dc7c86-b3e9-456d-919a-b81c077ecdb4-kube-api-access-n4v4z\") pod \"bootstrap-edpm-multinodeset-edpm-compute-global-tq75f\" (UID: \"f1dc7c86-b3e9-456d-919a-b81c077ecdb4\") " pod="openstack-kuttl-tests/bootstrap-edpm-multinodeset-edpm-compute-global-tq75f" Dec 03 12:58:18 crc kubenswrapper[5052]: I1203 12:58:18.186266 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bootstrap-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f1dc7c86-b3e9-456d-919a-b81c077ecdb4-bootstrap-combined-ca-bundle\") pod \"bootstrap-edpm-multinodeset-edpm-compute-global-tq75f\" (UID: \"f1dc7c86-b3e9-456d-919a-b81c077ecdb4\") " pod="openstack-kuttl-tests/bootstrap-edpm-multinodeset-edpm-compute-global-tq75f" Dec 03 12:58:18 crc kubenswrapper[5052]: I1203 12:58:18.186284 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/4c56e6a0-1f3a-4a96-b207-940aadc3c6ef-inventory\") pod \"bootstrap-edpm-multinodeset-edpm-compute-beta-nodeset-fbpnq\" (UID: \"4c56e6a0-1f3a-4a96-b207-940aadc3c6ef\") " pod="openstack-kuttl-tests/bootstrap-edpm-multinodeset-edpm-compute-beta-nodeset-fbpnq" Dec 03 12:58:18 crc kubenswrapper[5052]: I1203 12:58:18.186334 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/f1dc7c86-b3e9-456d-919a-b81c077ecdb4-ssh-key\") pod \"bootstrap-edpm-multinodeset-edpm-compute-global-tq75f\" (UID: \"f1dc7c86-b3e9-456d-919a-b81c077ecdb4\") " pod="openstack-kuttl-tests/bootstrap-edpm-multinodeset-edpm-compute-global-tq75f" Dec 03 12:58:18 crc kubenswrapper[5052]: I1203 12:58:18.186411 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zdm46\" (UniqueName: \"kubernetes.io/projected/4c56e6a0-1f3a-4a96-b207-940aadc3c6ef-kube-api-access-zdm46\") pod \"bootstrap-edpm-multinodeset-edpm-compute-beta-nodeset-fbpnq\" (UID: \"4c56e6a0-1f3a-4a96-b207-940aadc3c6ef\") " pod="openstack-kuttl-tests/bootstrap-edpm-multinodeset-edpm-compute-beta-nodeset-fbpnq" Dec 03 12:58:18 crc kubenswrapper[5052]: I1203 12:58:18.186443 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/f1dc7c86-b3e9-456d-919a-b81c077ecdb4-inventory\") pod \"bootstrap-edpm-multinodeset-edpm-compute-global-tq75f\" (UID: \"f1dc7c86-b3e9-456d-919a-b81c077ecdb4\") " pod="openstack-kuttl-tests/bootstrap-edpm-multinodeset-edpm-compute-global-tq75f" Dec 03 12:58:18 crc kubenswrapper[5052]: I1203 12:58:18.186476 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bootstrap-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4c56e6a0-1f3a-4a96-b207-940aadc3c6ef-bootstrap-combined-ca-bundle\") pod \"bootstrap-edpm-multinodeset-edpm-compute-beta-nodeset-fbpnq\" (UID: \"4c56e6a0-1f3a-4a96-b207-940aadc3c6ef\") " pod="openstack-kuttl-tests/bootstrap-edpm-multinodeset-edpm-compute-beta-nodeset-fbpnq" Dec 03 12:58:18 crc kubenswrapper[5052]: I1203 12:58:18.186499 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/4c56e6a0-1f3a-4a96-b207-940aadc3c6ef-ssh-key\") pod \"bootstrap-edpm-multinodeset-edpm-compute-beta-nodeset-fbpnq\" (UID: \"4c56e6a0-1f3a-4a96-b207-940aadc3c6ef\") " pod="openstack-kuttl-tests/bootstrap-edpm-multinodeset-edpm-compute-beta-nodeset-fbpnq" Dec 03 12:58:18 crc kubenswrapper[5052]: I1203 12:58:18.287349 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/f1dc7c86-b3e9-456d-919a-b81c077ecdb4-inventory\") pod \"bootstrap-edpm-multinodeset-edpm-compute-global-tq75f\" (UID: \"f1dc7c86-b3e9-456d-919a-b81c077ecdb4\") " pod="openstack-kuttl-tests/bootstrap-edpm-multinodeset-edpm-compute-global-tq75f" Dec 03 12:58:18 crc kubenswrapper[5052]: I1203 12:58:18.287407 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"bootstrap-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4c56e6a0-1f3a-4a96-b207-940aadc3c6ef-bootstrap-combined-ca-bundle\") pod \"bootstrap-edpm-multinodeset-edpm-compute-beta-nodeset-fbpnq\" (UID: \"4c56e6a0-1f3a-4a96-b207-940aadc3c6ef\") " pod="openstack-kuttl-tests/bootstrap-edpm-multinodeset-edpm-compute-beta-nodeset-fbpnq" Dec 03 12:58:18 crc kubenswrapper[5052]: I1203 12:58:18.287441 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/4c56e6a0-1f3a-4a96-b207-940aadc3c6ef-ssh-key\") pod \"bootstrap-edpm-multinodeset-edpm-compute-beta-nodeset-fbpnq\" (UID: \"4c56e6a0-1f3a-4a96-b207-940aadc3c6ef\") " pod="openstack-kuttl-tests/bootstrap-edpm-multinodeset-edpm-compute-beta-nodeset-fbpnq" Dec 03 12:58:18 crc kubenswrapper[5052]: I1203 12:58:18.287498 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-n4v4z\" (UniqueName: \"kubernetes.io/projected/f1dc7c86-b3e9-456d-919a-b81c077ecdb4-kube-api-access-n4v4z\") pod \"bootstrap-edpm-multinodeset-edpm-compute-global-tq75f\" (UID: \"f1dc7c86-b3e9-456d-919a-b81c077ecdb4\") " pod="openstack-kuttl-tests/bootstrap-edpm-multinodeset-edpm-compute-global-tq75f" Dec 03 12:58:18 crc kubenswrapper[5052]: I1203 12:58:18.287583 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"bootstrap-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f1dc7c86-b3e9-456d-919a-b81c077ecdb4-bootstrap-combined-ca-bundle\") pod \"bootstrap-edpm-multinodeset-edpm-compute-global-tq75f\" (UID: \"f1dc7c86-b3e9-456d-919a-b81c077ecdb4\") " pod="openstack-kuttl-tests/bootstrap-edpm-multinodeset-edpm-compute-global-tq75f" Dec 03 12:58:18 crc kubenswrapper[5052]: I1203 12:58:18.287608 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/4c56e6a0-1f3a-4a96-b207-940aadc3c6ef-inventory\") pod \"bootstrap-edpm-multinodeset-edpm-compute-beta-nodeset-fbpnq\" (UID: \"4c56e6a0-1f3a-4a96-b207-940aadc3c6ef\") " pod="openstack-kuttl-tests/bootstrap-edpm-multinodeset-edpm-compute-beta-nodeset-fbpnq" Dec 03 12:58:18 crc kubenswrapper[5052]: I1203 12:58:18.287662 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/f1dc7c86-b3e9-456d-919a-b81c077ecdb4-ssh-key\") pod \"bootstrap-edpm-multinodeset-edpm-compute-global-tq75f\" (UID: \"f1dc7c86-b3e9-456d-919a-b81c077ecdb4\") " pod="openstack-kuttl-tests/bootstrap-edpm-multinodeset-edpm-compute-global-tq75f" Dec 03 12:58:18 crc kubenswrapper[5052]: I1203 12:58:18.287712 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-zdm46\" (UniqueName: \"kubernetes.io/projected/4c56e6a0-1f3a-4a96-b207-940aadc3c6ef-kube-api-access-zdm46\") pod \"bootstrap-edpm-multinodeset-edpm-compute-beta-nodeset-fbpnq\" (UID: \"4c56e6a0-1f3a-4a96-b207-940aadc3c6ef\") " pod="openstack-kuttl-tests/bootstrap-edpm-multinodeset-edpm-compute-beta-nodeset-fbpnq" Dec 03 12:58:18 crc kubenswrapper[5052]: I1203 12:58:18.291185 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/4c56e6a0-1f3a-4a96-b207-940aadc3c6ef-ssh-key\") pod \"bootstrap-edpm-multinodeset-edpm-compute-beta-nodeset-fbpnq\" (UID: \"4c56e6a0-1f3a-4a96-b207-940aadc3c6ef\") " pod="openstack-kuttl-tests/bootstrap-edpm-multinodeset-edpm-compute-beta-nodeset-fbpnq" Dec 03 12:58:18 crc kubenswrapper[5052]: I1203 12:58:18.291221 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"bootstrap-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4c56e6a0-1f3a-4a96-b207-940aadc3c6ef-bootstrap-combined-ca-bundle\") pod \"bootstrap-edpm-multinodeset-edpm-compute-beta-nodeset-fbpnq\" (UID: \"4c56e6a0-1f3a-4a96-b207-940aadc3c6ef\") " pod="openstack-kuttl-tests/bootstrap-edpm-multinodeset-edpm-compute-beta-nodeset-fbpnq" Dec 03 12:58:18 crc kubenswrapper[5052]: I1203 12:58:18.291780 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/f1dc7c86-b3e9-456d-919a-b81c077ecdb4-inventory\") pod \"bootstrap-edpm-multinodeset-edpm-compute-global-tq75f\" (UID: \"f1dc7c86-b3e9-456d-919a-b81c077ecdb4\") " pod="openstack-kuttl-tests/bootstrap-edpm-multinodeset-edpm-compute-global-tq75f" Dec 03 12:58:18 crc kubenswrapper[5052]: I1203 12:58:18.292286 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/4c56e6a0-1f3a-4a96-b207-940aadc3c6ef-inventory\") pod \"bootstrap-edpm-multinodeset-edpm-compute-beta-nodeset-fbpnq\" (UID: \"4c56e6a0-1f3a-4a96-b207-940aadc3c6ef\") " pod="openstack-kuttl-tests/bootstrap-edpm-multinodeset-edpm-compute-beta-nodeset-fbpnq" Dec 03 12:58:18 crc kubenswrapper[5052]: I1203 12:58:18.293476 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/f1dc7c86-b3e9-456d-919a-b81c077ecdb4-ssh-key\") pod \"bootstrap-edpm-multinodeset-edpm-compute-global-tq75f\" (UID: \"f1dc7c86-b3e9-456d-919a-b81c077ecdb4\") " pod="openstack-kuttl-tests/bootstrap-edpm-multinodeset-edpm-compute-global-tq75f" Dec 03 12:58:18 crc kubenswrapper[5052]: I1203 12:58:18.294224 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"bootstrap-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f1dc7c86-b3e9-456d-919a-b81c077ecdb4-bootstrap-combined-ca-bundle\") pod \"bootstrap-edpm-multinodeset-edpm-compute-global-tq75f\" (UID: \"f1dc7c86-b3e9-456d-919a-b81c077ecdb4\") " pod="openstack-kuttl-tests/bootstrap-edpm-multinodeset-edpm-compute-global-tq75f" Dec 03 12:58:18 crc kubenswrapper[5052]: I1203 12:58:18.304139 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-zdm46\" (UniqueName: \"kubernetes.io/projected/4c56e6a0-1f3a-4a96-b207-940aadc3c6ef-kube-api-access-zdm46\") pod \"bootstrap-edpm-multinodeset-edpm-compute-beta-nodeset-fbpnq\" (UID: \"4c56e6a0-1f3a-4a96-b207-940aadc3c6ef\") " pod="openstack-kuttl-tests/bootstrap-edpm-multinodeset-edpm-compute-beta-nodeset-fbpnq" Dec 03 12:58:18 crc kubenswrapper[5052]: I1203 12:58:18.306145 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-n4v4z\" (UniqueName: \"kubernetes.io/projected/f1dc7c86-b3e9-456d-919a-b81c077ecdb4-kube-api-access-n4v4z\") pod \"bootstrap-edpm-multinodeset-edpm-compute-global-tq75f\" (UID: \"f1dc7c86-b3e9-456d-919a-b81c077ecdb4\") " pod="openstack-kuttl-tests/bootstrap-edpm-multinodeset-edpm-compute-global-tq75f" Dec 03 12:58:18 crc kubenswrapper[5052]: I1203 12:58:18.397713 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/bootstrap-edpm-multinodeset-edpm-compute-beta-nodeset-fbpnq" Dec 03 12:58:18 crc kubenswrapper[5052]: I1203 12:58:18.486337 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/bootstrap-edpm-multinodeset-edpm-compute-global-tq75f" Dec 03 12:58:18 crc kubenswrapper[5052]: I1203 12:58:18.820231 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/bootstrap-edpm-multinodeset-edpm-compute-beta-nodeset-fbpnq"] Dec 03 12:58:18 crc kubenswrapper[5052]: W1203 12:58:18.823834 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4c56e6a0_1f3a_4a96_b207_940aadc3c6ef.slice/crio-9abf389e379acb5a28a8cbde0024b08c1b9a22a7d8e4ac84a8fef2544516a795 WatchSource:0}: Error finding container 9abf389e379acb5a28a8cbde0024b08c1b9a22a7d8e4ac84a8fef2544516a795: Status 404 returned error can't find the container with id 9abf389e379acb5a28a8cbde0024b08c1b9a22a7d8e4ac84a8fef2544516a795 Dec 03 12:58:18 crc kubenswrapper[5052]: I1203 12:58:18.908571 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/bootstrap-edpm-multinodeset-edpm-compute-global-tq75f"] Dec 03 12:58:18 crc kubenswrapper[5052]: W1203 12:58:18.910172 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf1dc7c86_b3e9_456d_919a_b81c077ecdb4.slice/crio-7d34fe0a3a27e0ba8f31126bf04f4fdc9534c9a13422ef67d59323c5ae792838 WatchSource:0}: Error finding container 7d34fe0a3a27e0ba8f31126bf04f4fdc9534c9a13422ef67d59323c5ae792838: Status 404 returned error can't find the container with id 7d34fe0a3a27e0ba8f31126bf04f4fdc9534c9a13422ef67d59323c5ae792838 Dec 03 12:58:19 crc kubenswrapper[5052]: I1203 12:58:19.013059 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/bootstrap-edpm-multinodeset-edpm-compute-beta-nodeset-fbpnq" event={"ID":"4c56e6a0-1f3a-4a96-b207-940aadc3c6ef","Type":"ContainerStarted","Data":"9abf389e379acb5a28a8cbde0024b08c1b9a22a7d8e4ac84a8fef2544516a795"} Dec 03 12:58:19 crc kubenswrapper[5052]: I1203 12:58:19.014422 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/bootstrap-edpm-multinodeset-edpm-compute-global-tq75f" event={"ID":"f1dc7c86-b3e9-456d-919a-b81c077ecdb4","Type":"ContainerStarted","Data":"7d34fe0a3a27e0ba8f31126bf04f4fdc9534c9a13422ef67d59323c5ae792838"} Dec 03 12:58:20 crc kubenswrapper[5052]: I1203 12:58:20.025277 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/bootstrap-edpm-multinodeset-edpm-compute-global-tq75f" event={"ID":"f1dc7c86-b3e9-456d-919a-b81c077ecdb4","Type":"ContainerStarted","Data":"7dbab62565102d37d525bbacfd4fb0209c6df3d459afec16badc1049d2aa5f14"} Dec 03 12:58:20 crc kubenswrapper[5052]: I1203 12:58:20.028090 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/bootstrap-edpm-multinodeset-edpm-compute-beta-nodeset-fbpnq" event={"ID":"4c56e6a0-1f3a-4a96-b207-940aadc3c6ef","Type":"ContainerStarted","Data":"d735823ce96b93fda511b9255b1132e029a33bd9974ec2e78b0f58272e2cc67f"} Dec 03 12:58:20 crc kubenswrapper[5052]: I1203 12:58:20.044281 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/bootstrap-edpm-multinodeset-edpm-compute-global-tq75f" podStartSLOduration=1.472740389 podStartE2EDuration="2.044265504s" podCreationTimestamp="2025-12-03 12:58:18 +0000 UTC" firstStartedPulling="2025-12-03 12:58:18.912442985 +0000 UTC m=+5889.500684749" lastFinishedPulling="2025-12-03 12:58:19.48396811 +0000 UTC m=+5890.072209864" observedRunningTime="2025-12-03 12:58:20.0437821 +0000 UTC m=+5890.632023884" watchObservedRunningTime="2025-12-03 12:58:20.044265504 +0000 UTC m=+5890.632507268" Dec 03 12:58:20 crc kubenswrapper[5052]: I1203 12:58:20.063292 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/bootstrap-edpm-multinodeset-edpm-compute-beta-nodeset-fbpnq" podStartSLOduration=1.6004569709999998 podStartE2EDuration="2.063271717s" podCreationTimestamp="2025-12-03 12:58:18 +0000 UTC" firstStartedPulling="2025-12-03 12:58:18.826313082 +0000 UTC m=+5889.414554846" lastFinishedPulling="2025-12-03 12:58:19.289127828 +0000 UTC m=+5889.877369592" observedRunningTime="2025-12-03 12:58:20.059907031 +0000 UTC m=+5890.648148795" watchObservedRunningTime="2025-12-03 12:58:20.063271717 +0000 UTC m=+5890.651513491" Dec 03 12:58:21 crc kubenswrapper[5052]: I1203 12:58:21.039810 5052 generic.go:334] "Generic (PLEG): container finished" podID="4c56e6a0-1f3a-4a96-b207-940aadc3c6ef" containerID="d735823ce96b93fda511b9255b1132e029a33bd9974ec2e78b0f58272e2cc67f" exitCode=0 Dec 03 12:58:21 crc kubenswrapper[5052]: I1203 12:58:21.040035 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/bootstrap-edpm-multinodeset-edpm-compute-beta-nodeset-fbpnq" event={"ID":"4c56e6a0-1f3a-4a96-b207-940aadc3c6ef","Type":"ContainerDied","Data":"d735823ce96b93fda511b9255b1132e029a33bd9974ec2e78b0f58272e2cc67f"} Dec 03 12:58:22 crc kubenswrapper[5052]: I1203 12:58:22.052920 5052 generic.go:334] "Generic (PLEG): container finished" podID="f1dc7c86-b3e9-456d-919a-b81c077ecdb4" containerID="7dbab62565102d37d525bbacfd4fb0209c6df3d459afec16badc1049d2aa5f14" exitCode=0 Dec 03 12:58:22 crc kubenswrapper[5052]: I1203 12:58:22.053015 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/bootstrap-edpm-multinodeset-edpm-compute-global-tq75f" event={"ID":"f1dc7c86-b3e9-456d-919a-b81c077ecdb4","Type":"ContainerDied","Data":"7dbab62565102d37d525bbacfd4fb0209c6df3d459afec16badc1049d2aa5f14"} Dec 03 12:58:22 crc kubenswrapper[5052]: I1203 12:58:22.338642 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/bootstrap-edpm-multinodeset-edpm-compute-beta-nodeset-fbpnq" Dec 03 12:58:22 crc kubenswrapper[5052]: I1203 12:58:22.349828 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zdm46\" (UniqueName: \"kubernetes.io/projected/4c56e6a0-1f3a-4a96-b207-940aadc3c6ef-kube-api-access-zdm46\") pod \"4c56e6a0-1f3a-4a96-b207-940aadc3c6ef\" (UID: \"4c56e6a0-1f3a-4a96-b207-940aadc3c6ef\") " Dec 03 12:58:22 crc kubenswrapper[5052]: I1203 12:58:22.350648 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/4c56e6a0-1f3a-4a96-b207-940aadc3c6ef-ssh-key\") pod \"4c56e6a0-1f3a-4a96-b207-940aadc3c6ef\" (UID: \"4c56e6a0-1f3a-4a96-b207-940aadc3c6ef\") " Dec 03 12:58:22 crc kubenswrapper[5052]: I1203 12:58:22.350730 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/4c56e6a0-1f3a-4a96-b207-940aadc3c6ef-inventory\") pod \"4c56e6a0-1f3a-4a96-b207-940aadc3c6ef\" (UID: \"4c56e6a0-1f3a-4a96-b207-940aadc3c6ef\") " Dec 03 12:58:22 crc kubenswrapper[5052]: I1203 12:58:22.350829 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"bootstrap-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4c56e6a0-1f3a-4a96-b207-940aadc3c6ef-bootstrap-combined-ca-bundle\") pod \"4c56e6a0-1f3a-4a96-b207-940aadc3c6ef\" (UID: \"4c56e6a0-1f3a-4a96-b207-940aadc3c6ef\") " Dec 03 12:58:22 crc kubenswrapper[5052]: I1203 12:58:22.359316 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4c56e6a0-1f3a-4a96-b207-940aadc3c6ef-bootstrap-combined-ca-bundle" (OuterVolumeSpecName: "bootstrap-combined-ca-bundle") pod "4c56e6a0-1f3a-4a96-b207-940aadc3c6ef" (UID: "4c56e6a0-1f3a-4a96-b207-940aadc3c6ef"). InnerVolumeSpecName "bootstrap-combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:58:22 crc kubenswrapper[5052]: I1203 12:58:22.361616 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4c56e6a0-1f3a-4a96-b207-940aadc3c6ef-kube-api-access-zdm46" (OuterVolumeSpecName: "kube-api-access-zdm46") pod "4c56e6a0-1f3a-4a96-b207-940aadc3c6ef" (UID: "4c56e6a0-1f3a-4a96-b207-940aadc3c6ef"). InnerVolumeSpecName "kube-api-access-zdm46". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:58:22 crc kubenswrapper[5052]: I1203 12:58:22.373393 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4c56e6a0-1f3a-4a96-b207-940aadc3c6ef-inventory" (OuterVolumeSpecName: "inventory") pod "4c56e6a0-1f3a-4a96-b207-940aadc3c6ef" (UID: "4c56e6a0-1f3a-4a96-b207-940aadc3c6ef"). InnerVolumeSpecName "inventory". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:58:22 crc kubenswrapper[5052]: I1203 12:58:22.374053 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4c56e6a0-1f3a-4a96-b207-940aadc3c6ef-ssh-key" (OuterVolumeSpecName: "ssh-key") pod "4c56e6a0-1f3a-4a96-b207-940aadc3c6ef" (UID: "4c56e6a0-1f3a-4a96-b207-940aadc3c6ef"). InnerVolumeSpecName "ssh-key". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:58:22 crc kubenswrapper[5052]: I1203 12:58:22.453016 5052 reconciler_common.go:293] "Volume detached for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/4c56e6a0-1f3a-4a96-b207-940aadc3c6ef-inventory\") on node \"crc\" DevicePath \"\"" Dec 03 12:58:22 crc kubenswrapper[5052]: I1203 12:58:22.453055 5052 reconciler_common.go:293] "Volume detached for volume \"bootstrap-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4c56e6a0-1f3a-4a96-b207-940aadc3c6ef-bootstrap-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:58:22 crc kubenswrapper[5052]: I1203 12:58:22.453072 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-zdm46\" (UniqueName: \"kubernetes.io/projected/4c56e6a0-1f3a-4a96-b207-940aadc3c6ef-kube-api-access-zdm46\") on node \"crc\" DevicePath \"\"" Dec 03 12:58:22 crc kubenswrapper[5052]: I1203 12:58:22.453085 5052 reconciler_common.go:293] "Volume detached for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/4c56e6a0-1f3a-4a96-b207-940aadc3c6ef-ssh-key\") on node \"crc\" DevicePath \"\"" Dec 03 12:58:23 crc kubenswrapper[5052]: I1203 12:58:23.061701 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/bootstrap-edpm-multinodeset-edpm-compute-beta-nodeset-fbpnq" event={"ID":"4c56e6a0-1f3a-4a96-b207-940aadc3c6ef","Type":"ContainerDied","Data":"9abf389e379acb5a28a8cbde0024b08c1b9a22a7d8e4ac84a8fef2544516a795"} Dec 03 12:58:23 crc kubenswrapper[5052]: I1203 12:58:23.061742 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9abf389e379acb5a28a8cbde0024b08c1b9a22a7d8e4ac84a8fef2544516a795" Dec 03 12:58:23 crc kubenswrapper[5052]: I1203 12:58:23.061723 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/bootstrap-edpm-multinodeset-edpm-compute-beta-nodeset-fbpnq" Dec 03 12:58:23 crc kubenswrapper[5052]: I1203 12:58:23.324636 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/bootstrap-edpm-multinodeset-edpm-compute-global-tq75f" Dec 03 12:58:23 crc kubenswrapper[5052]: I1203 12:58:23.364146 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/f1dc7c86-b3e9-456d-919a-b81c077ecdb4-ssh-key\") pod \"f1dc7c86-b3e9-456d-919a-b81c077ecdb4\" (UID: \"f1dc7c86-b3e9-456d-919a-b81c077ecdb4\") " Dec 03 12:58:23 crc kubenswrapper[5052]: I1203 12:58:23.364218 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n4v4z\" (UniqueName: \"kubernetes.io/projected/f1dc7c86-b3e9-456d-919a-b81c077ecdb4-kube-api-access-n4v4z\") pod \"f1dc7c86-b3e9-456d-919a-b81c077ecdb4\" (UID: \"f1dc7c86-b3e9-456d-919a-b81c077ecdb4\") " Dec 03 12:58:23 crc kubenswrapper[5052]: I1203 12:58:23.364284 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/f1dc7c86-b3e9-456d-919a-b81c077ecdb4-inventory\") pod \"f1dc7c86-b3e9-456d-919a-b81c077ecdb4\" (UID: \"f1dc7c86-b3e9-456d-919a-b81c077ecdb4\") " Dec 03 12:58:23 crc kubenswrapper[5052]: I1203 12:58:23.364346 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"bootstrap-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f1dc7c86-b3e9-456d-919a-b81c077ecdb4-bootstrap-combined-ca-bundle\") pod \"f1dc7c86-b3e9-456d-919a-b81c077ecdb4\" (UID: \"f1dc7c86-b3e9-456d-919a-b81c077ecdb4\") " Dec 03 12:58:23 crc kubenswrapper[5052]: I1203 12:58:23.369404 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f1dc7c86-b3e9-456d-919a-b81c077ecdb4-kube-api-access-n4v4z" (OuterVolumeSpecName: "kube-api-access-n4v4z") pod "f1dc7c86-b3e9-456d-919a-b81c077ecdb4" (UID: "f1dc7c86-b3e9-456d-919a-b81c077ecdb4"). InnerVolumeSpecName "kube-api-access-n4v4z". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:58:23 crc kubenswrapper[5052]: I1203 12:58:23.370054 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f1dc7c86-b3e9-456d-919a-b81c077ecdb4-bootstrap-combined-ca-bundle" (OuterVolumeSpecName: "bootstrap-combined-ca-bundle") pod "f1dc7c86-b3e9-456d-919a-b81c077ecdb4" (UID: "f1dc7c86-b3e9-456d-919a-b81c077ecdb4"). InnerVolumeSpecName "bootstrap-combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:58:23 crc kubenswrapper[5052]: I1203 12:58:23.390063 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f1dc7c86-b3e9-456d-919a-b81c077ecdb4-inventory" (OuterVolumeSpecName: "inventory") pod "f1dc7c86-b3e9-456d-919a-b81c077ecdb4" (UID: "f1dc7c86-b3e9-456d-919a-b81c077ecdb4"). InnerVolumeSpecName "inventory". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:58:23 crc kubenswrapper[5052]: I1203 12:58:23.391558 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f1dc7c86-b3e9-456d-919a-b81c077ecdb4-ssh-key" (OuterVolumeSpecName: "ssh-key") pod "f1dc7c86-b3e9-456d-919a-b81c077ecdb4" (UID: "f1dc7c86-b3e9-456d-919a-b81c077ecdb4"). InnerVolumeSpecName "ssh-key". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:58:23 crc kubenswrapper[5052]: I1203 12:58:23.465698 5052 reconciler_common.go:293] "Volume detached for volume \"bootstrap-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f1dc7c86-b3e9-456d-919a-b81c077ecdb4-bootstrap-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:58:23 crc kubenswrapper[5052]: I1203 12:58:23.465895 5052 reconciler_common.go:293] "Volume detached for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/f1dc7c86-b3e9-456d-919a-b81c077ecdb4-ssh-key\") on node \"crc\" DevicePath \"\"" Dec 03 12:58:23 crc kubenswrapper[5052]: I1203 12:58:23.465918 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-n4v4z\" (UniqueName: \"kubernetes.io/projected/f1dc7c86-b3e9-456d-919a-b81c077ecdb4-kube-api-access-n4v4z\") on node \"crc\" DevicePath \"\"" Dec 03 12:58:23 crc kubenswrapper[5052]: I1203 12:58:23.465931 5052 reconciler_common.go:293] "Volume detached for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/f1dc7c86-b3e9-456d-919a-b81c077ecdb4-inventory\") on node \"crc\" DevicePath \"\"" Dec 03 12:58:24 crc kubenswrapper[5052]: I1203 12:58:24.075409 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/bootstrap-edpm-multinodeset-edpm-compute-global-tq75f" event={"ID":"f1dc7c86-b3e9-456d-919a-b81c077ecdb4","Type":"ContainerDied","Data":"7d34fe0a3a27e0ba8f31126bf04f4fdc9534c9a13422ef67d59323c5ae792838"} Dec 03 12:58:24 crc kubenswrapper[5052]: I1203 12:58:24.075457 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7d34fe0a3a27e0ba8f31126bf04f4fdc9534c9a13422ef67d59323c5ae792838" Dec 03 12:58:24 crc kubenswrapper[5052]: I1203 12:58:24.075588 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/bootstrap-edpm-multinodeset-edpm-compute-global-tq75f" Dec 03 12:58:24 crc kubenswrapper[5052]: I1203 12:58:24.182582 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/configure-network-edpm-multinodeset-edpm-compute-global-z5bnb"] Dec 03 12:58:24 crc kubenswrapper[5052]: E1203 12:58:24.183033 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="f1dc7c86-b3e9-456d-919a-b81c077ecdb4" containerName="bootstrap-edpm-multinodeset-edpm-compute-global" Dec 03 12:58:24 crc kubenswrapper[5052]: I1203 12:58:24.183060 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="f1dc7c86-b3e9-456d-919a-b81c077ecdb4" containerName="bootstrap-edpm-multinodeset-edpm-compute-global" Dec 03 12:58:24 crc kubenswrapper[5052]: E1203 12:58:24.183082 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="4c56e6a0-1f3a-4a96-b207-940aadc3c6ef" containerName="bootstrap-edpm-multinodeset-edpm-compute-beta-nodeset" Dec 03 12:58:24 crc kubenswrapper[5052]: I1203 12:58:24.183093 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="4c56e6a0-1f3a-4a96-b207-940aadc3c6ef" containerName="bootstrap-edpm-multinodeset-edpm-compute-beta-nodeset" Dec 03 12:58:24 crc kubenswrapper[5052]: I1203 12:58:24.183285 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="4c56e6a0-1f3a-4a96-b207-940aadc3c6ef" containerName="bootstrap-edpm-multinodeset-edpm-compute-beta-nodeset" Dec 03 12:58:24 crc kubenswrapper[5052]: I1203 12:58:24.183312 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="f1dc7c86-b3e9-456d-919a-b81c077ecdb4" containerName="bootstrap-edpm-multinodeset-edpm-compute-global" Dec 03 12:58:24 crc kubenswrapper[5052]: I1203 12:58:24.183971 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/configure-network-edpm-multinodeset-edpm-compute-global-z5bnb" Dec 03 12:58:24 crc kubenswrapper[5052]: I1203 12:58:24.187945 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"openstack-aee-default-env" Dec 03 12:58:24 crc kubenswrapper[5052]: I1203 12:58:24.188099 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"dataplane-ansible-ssh-private-key-secret" Dec 03 12:58:24 crc kubenswrapper[5052]: I1203 12:58:24.188584 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"dataplanenodeset-edpm-compute-global" Dec 03 12:58:24 crc kubenswrapper[5052]: I1203 12:58:24.190197 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"edpm-compute-global-dockercfg-p5rjx" Dec 03 12:58:24 crc kubenswrapper[5052]: I1203 12:58:24.202090 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/configure-network-edpm-multinodeset-edpm-compute-global-z5bnb"] Dec 03 12:58:24 crc kubenswrapper[5052]: I1203 12:58:24.378645 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/fa21fede-812c-48f1-84e7-899886f312c3-ssh-key\") pod \"configure-network-edpm-multinodeset-edpm-compute-global-z5bnb\" (UID: \"fa21fede-812c-48f1-84e7-899886f312c3\") " pod="openstack-kuttl-tests/configure-network-edpm-multinodeset-edpm-compute-global-z5bnb" Dec 03 12:58:24 crc kubenswrapper[5052]: I1203 12:58:24.378731 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h6vtr\" (UniqueName: \"kubernetes.io/projected/fa21fede-812c-48f1-84e7-899886f312c3-kube-api-access-h6vtr\") pod \"configure-network-edpm-multinodeset-edpm-compute-global-z5bnb\" (UID: \"fa21fede-812c-48f1-84e7-899886f312c3\") " pod="openstack-kuttl-tests/configure-network-edpm-multinodeset-edpm-compute-global-z5bnb" Dec 03 12:58:24 crc kubenswrapper[5052]: I1203 12:58:24.378777 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/fa21fede-812c-48f1-84e7-899886f312c3-inventory\") pod \"configure-network-edpm-multinodeset-edpm-compute-global-z5bnb\" (UID: \"fa21fede-812c-48f1-84e7-899886f312c3\") " pod="openstack-kuttl-tests/configure-network-edpm-multinodeset-edpm-compute-global-z5bnb" Dec 03 12:58:24 crc kubenswrapper[5052]: I1203 12:58:24.479929 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/fa21fede-812c-48f1-84e7-899886f312c3-ssh-key\") pod \"configure-network-edpm-multinodeset-edpm-compute-global-z5bnb\" (UID: \"fa21fede-812c-48f1-84e7-899886f312c3\") " pod="openstack-kuttl-tests/configure-network-edpm-multinodeset-edpm-compute-global-z5bnb" Dec 03 12:58:24 crc kubenswrapper[5052]: I1203 12:58:24.480020 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-h6vtr\" (UniqueName: \"kubernetes.io/projected/fa21fede-812c-48f1-84e7-899886f312c3-kube-api-access-h6vtr\") pod \"configure-network-edpm-multinodeset-edpm-compute-global-z5bnb\" (UID: \"fa21fede-812c-48f1-84e7-899886f312c3\") " pod="openstack-kuttl-tests/configure-network-edpm-multinodeset-edpm-compute-global-z5bnb" Dec 03 12:58:24 crc kubenswrapper[5052]: I1203 12:58:24.480053 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/fa21fede-812c-48f1-84e7-899886f312c3-inventory\") pod \"configure-network-edpm-multinodeset-edpm-compute-global-z5bnb\" (UID: \"fa21fede-812c-48f1-84e7-899886f312c3\") " pod="openstack-kuttl-tests/configure-network-edpm-multinodeset-edpm-compute-global-z5bnb" Dec 03 12:58:24 crc kubenswrapper[5052]: I1203 12:58:24.483440 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/fa21fede-812c-48f1-84e7-899886f312c3-ssh-key\") pod \"configure-network-edpm-multinodeset-edpm-compute-global-z5bnb\" (UID: \"fa21fede-812c-48f1-84e7-899886f312c3\") " pod="openstack-kuttl-tests/configure-network-edpm-multinodeset-edpm-compute-global-z5bnb" Dec 03 12:58:24 crc kubenswrapper[5052]: I1203 12:58:24.483477 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/fa21fede-812c-48f1-84e7-899886f312c3-inventory\") pod \"configure-network-edpm-multinodeset-edpm-compute-global-z5bnb\" (UID: \"fa21fede-812c-48f1-84e7-899886f312c3\") " pod="openstack-kuttl-tests/configure-network-edpm-multinodeset-edpm-compute-global-z5bnb" Dec 03 12:58:24 crc kubenswrapper[5052]: I1203 12:58:24.501366 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-h6vtr\" (UniqueName: \"kubernetes.io/projected/fa21fede-812c-48f1-84e7-899886f312c3-kube-api-access-h6vtr\") pod \"configure-network-edpm-multinodeset-edpm-compute-global-z5bnb\" (UID: \"fa21fede-812c-48f1-84e7-899886f312c3\") " pod="openstack-kuttl-tests/configure-network-edpm-multinodeset-edpm-compute-global-z5bnb" Dec 03 12:58:24 crc kubenswrapper[5052]: I1203 12:58:24.506332 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/configure-network-edpm-multinodeset-edpm-compute-global-z5bnb" Dec 03 12:58:24 crc kubenswrapper[5052]: I1203 12:58:24.927492 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/configure-network-edpm-multinodeset-edpm-compute-global-z5bnb"] Dec 03 12:58:24 crc kubenswrapper[5052]: W1203 12:58:24.939760 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfa21fede_812c_48f1_84e7_899886f312c3.slice/crio-d40980ffe6411d248fa86935d4753dd0551c06b0073854990201dc76173879e7 WatchSource:0}: Error finding container d40980ffe6411d248fa86935d4753dd0551c06b0073854990201dc76173879e7: Status 404 returned error can't find the container with id d40980ffe6411d248fa86935d4753dd0551c06b0073854990201dc76173879e7 Dec 03 12:58:25 crc kubenswrapper[5052]: I1203 12:58:25.087147 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/configure-network-edpm-multinodeset-edpm-compute-global-z5bnb" event={"ID":"fa21fede-812c-48f1-84e7-899886f312c3","Type":"ContainerStarted","Data":"d40980ffe6411d248fa86935d4753dd0551c06b0073854990201dc76173879e7"} Dec 03 12:58:26 crc kubenswrapper[5052]: I1203 12:58:26.099159 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/configure-network-edpm-multinodeset-edpm-compute-global-z5bnb" event={"ID":"fa21fede-812c-48f1-84e7-899886f312c3","Type":"ContainerStarted","Data":"afc3b9ec0f6569d43e8044e0226a3a0167f6f4148750c2a219b6f5318339c9c1"} Dec 03 12:58:26 crc kubenswrapper[5052]: I1203 12:58:26.118856 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/configure-network-edpm-multinodeset-edpm-compute-global-z5bnb" podStartSLOduration=1.285866733 podStartE2EDuration="2.118838455s" podCreationTimestamp="2025-12-03 12:58:24 +0000 UTC" firstStartedPulling="2025-12-03 12:58:24.941599388 +0000 UTC m=+5895.529841152" lastFinishedPulling="2025-12-03 12:58:25.77457111 +0000 UTC m=+5896.362812874" observedRunningTime="2025-12-03 12:58:26.112134724 +0000 UTC m=+5896.700376498" watchObservedRunningTime="2025-12-03 12:58:26.118838455 +0000 UTC m=+5896.707080219" Dec 03 12:58:27 crc kubenswrapper[5052]: I1203 12:58:27.673483 5052 scope.go:117] "RemoveContainer" containerID="69fa7fbcd2aceb7c53703a19508876f93a97dc9adabb79c16fb691d90f606d58" Dec 03 12:58:27 crc kubenswrapper[5052]: E1203 12:58:27.673965 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 12:58:28 crc kubenswrapper[5052]: I1203 12:58:28.128653 5052 generic.go:334] "Generic (PLEG): container finished" podID="fa21fede-812c-48f1-84e7-899886f312c3" containerID="afc3b9ec0f6569d43e8044e0226a3a0167f6f4148750c2a219b6f5318339c9c1" exitCode=0 Dec 03 12:58:28 crc kubenswrapper[5052]: I1203 12:58:28.128716 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/configure-network-edpm-multinodeset-edpm-compute-global-z5bnb" event={"ID":"fa21fede-812c-48f1-84e7-899886f312c3","Type":"ContainerDied","Data":"afc3b9ec0f6569d43e8044e0226a3a0167f6f4148750c2a219b6f5318339c9c1"} Dec 03 12:58:29 crc kubenswrapper[5052]: I1203 12:58:29.392358 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/configure-network-edpm-multinodeset-edpm-compute-global-z5bnb" Dec 03 12:58:29 crc kubenswrapper[5052]: I1203 12:58:29.551133 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/fa21fede-812c-48f1-84e7-899886f312c3-ssh-key\") pod \"fa21fede-812c-48f1-84e7-899886f312c3\" (UID: \"fa21fede-812c-48f1-84e7-899886f312c3\") " Dec 03 12:58:29 crc kubenswrapper[5052]: I1203 12:58:29.551223 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h6vtr\" (UniqueName: \"kubernetes.io/projected/fa21fede-812c-48f1-84e7-899886f312c3-kube-api-access-h6vtr\") pod \"fa21fede-812c-48f1-84e7-899886f312c3\" (UID: \"fa21fede-812c-48f1-84e7-899886f312c3\") " Dec 03 12:58:29 crc kubenswrapper[5052]: I1203 12:58:29.551371 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/fa21fede-812c-48f1-84e7-899886f312c3-inventory\") pod \"fa21fede-812c-48f1-84e7-899886f312c3\" (UID: \"fa21fede-812c-48f1-84e7-899886f312c3\") " Dec 03 12:58:29 crc kubenswrapper[5052]: I1203 12:58:29.562771 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fa21fede-812c-48f1-84e7-899886f312c3-kube-api-access-h6vtr" (OuterVolumeSpecName: "kube-api-access-h6vtr") pod "fa21fede-812c-48f1-84e7-899886f312c3" (UID: "fa21fede-812c-48f1-84e7-899886f312c3"). InnerVolumeSpecName "kube-api-access-h6vtr". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:58:29 crc kubenswrapper[5052]: I1203 12:58:29.571271 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fa21fede-812c-48f1-84e7-899886f312c3-ssh-key" (OuterVolumeSpecName: "ssh-key") pod "fa21fede-812c-48f1-84e7-899886f312c3" (UID: "fa21fede-812c-48f1-84e7-899886f312c3"). InnerVolumeSpecName "ssh-key". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:58:29 crc kubenswrapper[5052]: I1203 12:58:29.571862 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fa21fede-812c-48f1-84e7-899886f312c3-inventory" (OuterVolumeSpecName: "inventory") pod "fa21fede-812c-48f1-84e7-899886f312c3" (UID: "fa21fede-812c-48f1-84e7-899886f312c3"). InnerVolumeSpecName "inventory". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:58:29 crc kubenswrapper[5052]: I1203 12:58:29.653947 5052 reconciler_common.go:293] "Volume detached for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/fa21fede-812c-48f1-84e7-899886f312c3-inventory\") on node \"crc\" DevicePath \"\"" Dec 03 12:58:29 crc kubenswrapper[5052]: I1203 12:58:29.654242 5052 reconciler_common.go:293] "Volume detached for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/fa21fede-812c-48f1-84e7-899886f312c3-ssh-key\") on node \"crc\" DevicePath \"\"" Dec 03 12:58:29 crc kubenswrapper[5052]: I1203 12:58:29.654261 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-h6vtr\" (UniqueName: \"kubernetes.io/projected/fa21fede-812c-48f1-84e7-899886f312c3-kube-api-access-h6vtr\") on node \"crc\" DevicePath \"\"" Dec 03 12:58:30 crc kubenswrapper[5052]: I1203 12:58:30.149197 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/configure-network-edpm-multinodeset-edpm-compute-global-z5bnb" event={"ID":"fa21fede-812c-48f1-84e7-899886f312c3","Type":"ContainerDied","Data":"d40980ffe6411d248fa86935d4753dd0551c06b0073854990201dc76173879e7"} Dec 03 12:58:30 crc kubenswrapper[5052]: I1203 12:58:30.149273 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d40980ffe6411d248fa86935d4753dd0551c06b0073854990201dc76173879e7" Dec 03 12:58:30 crc kubenswrapper[5052]: I1203 12:58:30.149315 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/configure-network-edpm-multinodeset-edpm-compute-global-z5bnb" Dec 03 12:58:30 crc kubenswrapper[5052]: I1203 12:58:30.259887 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/validate-network-edpm-multinodeset-edpm-compute-global-4rxsd"] Dec 03 12:58:30 crc kubenswrapper[5052]: E1203 12:58:30.260297 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="fa21fede-812c-48f1-84e7-899886f312c3" containerName="configure-network-edpm-multinodeset-edpm-compute-global" Dec 03 12:58:30 crc kubenswrapper[5052]: I1203 12:58:30.260319 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="fa21fede-812c-48f1-84e7-899886f312c3" containerName="configure-network-edpm-multinodeset-edpm-compute-global" Dec 03 12:58:30 crc kubenswrapper[5052]: I1203 12:58:30.260484 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="fa21fede-812c-48f1-84e7-899886f312c3" containerName="configure-network-edpm-multinodeset-edpm-compute-global" Dec 03 12:58:30 crc kubenswrapper[5052]: I1203 12:58:30.261153 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/validate-network-edpm-multinodeset-edpm-compute-global-4rxsd" Dec 03 12:58:30 crc kubenswrapper[5052]: I1203 12:58:30.264115 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/d51c95a7-7ee5-455d-9cc2-5227f70b2c73-ssh-key\") pod \"validate-network-edpm-multinodeset-edpm-compute-global-4rxsd\" (UID: \"d51c95a7-7ee5-455d-9cc2-5227f70b2c73\") " pod="openstack-kuttl-tests/validate-network-edpm-multinodeset-edpm-compute-global-4rxsd" Dec 03 12:58:30 crc kubenswrapper[5052]: I1203 12:58:30.264196 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/d51c95a7-7ee5-455d-9cc2-5227f70b2c73-inventory\") pod \"validate-network-edpm-multinodeset-edpm-compute-global-4rxsd\" (UID: \"d51c95a7-7ee5-455d-9cc2-5227f70b2c73\") " pod="openstack-kuttl-tests/validate-network-edpm-multinodeset-edpm-compute-global-4rxsd" Dec 03 12:58:30 crc kubenswrapper[5052]: I1203 12:58:30.264271 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mc66n\" (UniqueName: \"kubernetes.io/projected/d51c95a7-7ee5-455d-9cc2-5227f70b2c73-kube-api-access-mc66n\") pod \"validate-network-edpm-multinodeset-edpm-compute-global-4rxsd\" (UID: \"d51c95a7-7ee5-455d-9cc2-5227f70b2c73\") " pod="openstack-kuttl-tests/validate-network-edpm-multinodeset-edpm-compute-global-4rxsd" Dec 03 12:58:30 crc kubenswrapper[5052]: I1203 12:58:30.264362 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"openstack-aee-default-env" Dec 03 12:58:30 crc kubenswrapper[5052]: I1203 12:58:30.264534 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"edpm-compute-global-dockercfg-p5rjx" Dec 03 12:58:30 crc kubenswrapper[5052]: I1203 12:58:30.264726 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"dataplane-ansible-ssh-private-key-secret" Dec 03 12:58:30 crc kubenswrapper[5052]: I1203 12:58:30.264996 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"dataplanenodeset-edpm-compute-global" Dec 03 12:58:30 crc kubenswrapper[5052]: I1203 12:58:30.280728 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/validate-network-edpm-multinodeset-edpm-compute-global-4rxsd"] Dec 03 12:58:30 crc kubenswrapper[5052]: I1203 12:58:30.365450 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/d51c95a7-7ee5-455d-9cc2-5227f70b2c73-inventory\") pod \"validate-network-edpm-multinodeset-edpm-compute-global-4rxsd\" (UID: \"d51c95a7-7ee5-455d-9cc2-5227f70b2c73\") " pod="openstack-kuttl-tests/validate-network-edpm-multinodeset-edpm-compute-global-4rxsd" Dec 03 12:58:30 crc kubenswrapper[5052]: I1203 12:58:30.365576 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-mc66n\" (UniqueName: \"kubernetes.io/projected/d51c95a7-7ee5-455d-9cc2-5227f70b2c73-kube-api-access-mc66n\") pod \"validate-network-edpm-multinodeset-edpm-compute-global-4rxsd\" (UID: \"d51c95a7-7ee5-455d-9cc2-5227f70b2c73\") " pod="openstack-kuttl-tests/validate-network-edpm-multinodeset-edpm-compute-global-4rxsd" Dec 03 12:58:30 crc kubenswrapper[5052]: I1203 12:58:30.365634 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/d51c95a7-7ee5-455d-9cc2-5227f70b2c73-ssh-key\") pod \"validate-network-edpm-multinodeset-edpm-compute-global-4rxsd\" (UID: \"d51c95a7-7ee5-455d-9cc2-5227f70b2c73\") " pod="openstack-kuttl-tests/validate-network-edpm-multinodeset-edpm-compute-global-4rxsd" Dec 03 12:58:30 crc kubenswrapper[5052]: I1203 12:58:30.371972 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/d51c95a7-7ee5-455d-9cc2-5227f70b2c73-ssh-key\") pod \"validate-network-edpm-multinodeset-edpm-compute-global-4rxsd\" (UID: \"d51c95a7-7ee5-455d-9cc2-5227f70b2c73\") " pod="openstack-kuttl-tests/validate-network-edpm-multinodeset-edpm-compute-global-4rxsd" Dec 03 12:58:30 crc kubenswrapper[5052]: I1203 12:58:30.372315 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/d51c95a7-7ee5-455d-9cc2-5227f70b2c73-inventory\") pod \"validate-network-edpm-multinodeset-edpm-compute-global-4rxsd\" (UID: \"d51c95a7-7ee5-455d-9cc2-5227f70b2c73\") " pod="openstack-kuttl-tests/validate-network-edpm-multinodeset-edpm-compute-global-4rxsd" Dec 03 12:58:30 crc kubenswrapper[5052]: I1203 12:58:30.394714 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-mc66n\" (UniqueName: \"kubernetes.io/projected/d51c95a7-7ee5-455d-9cc2-5227f70b2c73-kube-api-access-mc66n\") pod \"validate-network-edpm-multinodeset-edpm-compute-global-4rxsd\" (UID: \"d51c95a7-7ee5-455d-9cc2-5227f70b2c73\") " pod="openstack-kuttl-tests/validate-network-edpm-multinodeset-edpm-compute-global-4rxsd" Dec 03 12:58:30 crc kubenswrapper[5052]: I1203 12:58:30.577719 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/validate-network-edpm-multinodeset-edpm-compute-global-4rxsd" Dec 03 12:58:31 crc kubenswrapper[5052]: I1203 12:58:31.002345 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/validate-network-edpm-multinodeset-edpm-compute-global-4rxsd"] Dec 03 12:58:31 crc kubenswrapper[5052]: I1203 12:58:31.158233 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/validate-network-edpm-multinodeset-edpm-compute-global-4rxsd" event={"ID":"d51c95a7-7ee5-455d-9cc2-5227f70b2c73","Type":"ContainerStarted","Data":"199868e892138225608d379b959213c25f4d89a2400899d024de7d24cf5fae51"} Dec 03 12:58:32 crc kubenswrapper[5052]: I1203 12:58:32.167871 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/validate-network-edpm-multinodeset-edpm-compute-global-4rxsd" event={"ID":"d51c95a7-7ee5-455d-9cc2-5227f70b2c73","Type":"ContainerStarted","Data":"1fa4faeb71c3c04c4a155f9896519f0bee40159795401bac2d43ec3f2949c08a"} Dec 03 12:58:33 crc kubenswrapper[5052]: I1203 12:58:33.179979 5052 generic.go:334] "Generic (PLEG): container finished" podID="d51c95a7-7ee5-455d-9cc2-5227f70b2c73" containerID="1fa4faeb71c3c04c4a155f9896519f0bee40159795401bac2d43ec3f2949c08a" exitCode=0 Dec 03 12:58:33 crc kubenswrapper[5052]: I1203 12:58:33.180027 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/validate-network-edpm-multinodeset-edpm-compute-global-4rxsd" event={"ID":"d51c95a7-7ee5-455d-9cc2-5227f70b2c73","Type":"ContainerDied","Data":"1fa4faeb71c3c04c4a155f9896519f0bee40159795401bac2d43ec3f2949c08a"} Dec 03 12:58:33 crc kubenswrapper[5052]: E1203 12:58:33.216680 5052 cadvisor_stats_provider.go:516] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd51c95a7_7ee5_455d_9cc2_5227f70b2c73.slice/crio-conmon-1fa4faeb71c3c04c4a155f9896519f0bee40159795401bac2d43ec3f2949c08a.scope\": RecentStats: unable to find data in memory cache]" Dec 03 12:58:34 crc kubenswrapper[5052]: I1203 12:58:34.537373 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/validate-network-edpm-multinodeset-edpm-compute-global-4rxsd" Dec 03 12:58:34 crc kubenswrapper[5052]: I1203 12:58:34.629871 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/d51c95a7-7ee5-455d-9cc2-5227f70b2c73-ssh-key\") pod \"d51c95a7-7ee5-455d-9cc2-5227f70b2c73\" (UID: \"d51c95a7-7ee5-455d-9cc2-5227f70b2c73\") " Dec 03 12:58:34 crc kubenswrapper[5052]: I1203 12:58:34.629977 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mc66n\" (UniqueName: \"kubernetes.io/projected/d51c95a7-7ee5-455d-9cc2-5227f70b2c73-kube-api-access-mc66n\") pod \"d51c95a7-7ee5-455d-9cc2-5227f70b2c73\" (UID: \"d51c95a7-7ee5-455d-9cc2-5227f70b2c73\") " Dec 03 12:58:34 crc kubenswrapper[5052]: I1203 12:58:34.630044 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/d51c95a7-7ee5-455d-9cc2-5227f70b2c73-inventory\") pod \"d51c95a7-7ee5-455d-9cc2-5227f70b2c73\" (UID: \"d51c95a7-7ee5-455d-9cc2-5227f70b2c73\") " Dec 03 12:58:34 crc kubenswrapper[5052]: I1203 12:58:34.650711 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d51c95a7-7ee5-455d-9cc2-5227f70b2c73-kube-api-access-mc66n" (OuterVolumeSpecName: "kube-api-access-mc66n") pod "d51c95a7-7ee5-455d-9cc2-5227f70b2c73" (UID: "d51c95a7-7ee5-455d-9cc2-5227f70b2c73"). InnerVolumeSpecName "kube-api-access-mc66n". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:58:34 crc kubenswrapper[5052]: I1203 12:58:34.662871 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d51c95a7-7ee5-455d-9cc2-5227f70b2c73-ssh-key" (OuterVolumeSpecName: "ssh-key") pod "d51c95a7-7ee5-455d-9cc2-5227f70b2c73" (UID: "d51c95a7-7ee5-455d-9cc2-5227f70b2c73"). InnerVolumeSpecName "ssh-key". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:58:34 crc kubenswrapper[5052]: I1203 12:58:34.662927 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d51c95a7-7ee5-455d-9cc2-5227f70b2c73-inventory" (OuterVolumeSpecName: "inventory") pod "d51c95a7-7ee5-455d-9cc2-5227f70b2c73" (UID: "d51c95a7-7ee5-455d-9cc2-5227f70b2c73"). InnerVolumeSpecName "inventory". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:58:34 crc kubenswrapper[5052]: I1203 12:58:34.732111 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-mc66n\" (UniqueName: \"kubernetes.io/projected/d51c95a7-7ee5-455d-9cc2-5227f70b2c73-kube-api-access-mc66n\") on node \"crc\" DevicePath \"\"" Dec 03 12:58:34 crc kubenswrapper[5052]: I1203 12:58:34.732416 5052 reconciler_common.go:293] "Volume detached for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/d51c95a7-7ee5-455d-9cc2-5227f70b2c73-inventory\") on node \"crc\" DevicePath \"\"" Dec 03 12:58:34 crc kubenswrapper[5052]: I1203 12:58:34.732485 5052 reconciler_common.go:293] "Volume detached for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/d51c95a7-7ee5-455d-9cc2-5227f70b2c73-ssh-key\") on node \"crc\" DevicePath \"\"" Dec 03 12:58:35 crc kubenswrapper[5052]: I1203 12:58:35.198972 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/validate-network-edpm-multinodeset-edpm-compute-global-4rxsd" event={"ID":"d51c95a7-7ee5-455d-9cc2-5227f70b2c73","Type":"ContainerDied","Data":"199868e892138225608d379b959213c25f4d89a2400899d024de7d24cf5fae51"} Dec 03 12:58:35 crc kubenswrapper[5052]: I1203 12:58:35.199011 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="199868e892138225608d379b959213c25f4d89a2400899d024de7d24cf5fae51" Dec 03 12:58:35 crc kubenswrapper[5052]: I1203 12:58:35.199014 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/validate-network-edpm-multinodeset-edpm-compute-global-4rxsd" Dec 03 12:58:35 crc kubenswrapper[5052]: I1203 12:58:35.269564 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/install-os-edpm-multinodeset-edpm-compute-global-zkjjj"] Dec 03 12:58:35 crc kubenswrapper[5052]: E1203 12:58:35.269977 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="d51c95a7-7ee5-455d-9cc2-5227f70b2c73" containerName="validate-network-edpm-multinodeset-edpm-compute-global" Dec 03 12:58:35 crc kubenswrapper[5052]: I1203 12:58:35.270001 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="d51c95a7-7ee5-455d-9cc2-5227f70b2c73" containerName="validate-network-edpm-multinodeset-edpm-compute-global" Dec 03 12:58:35 crc kubenswrapper[5052]: I1203 12:58:35.270210 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="d51c95a7-7ee5-455d-9cc2-5227f70b2c73" containerName="validate-network-edpm-multinodeset-edpm-compute-global" Dec 03 12:58:35 crc kubenswrapper[5052]: I1203 12:58:35.270858 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/install-os-edpm-multinodeset-edpm-compute-global-zkjjj" Dec 03 12:58:35 crc kubenswrapper[5052]: I1203 12:58:35.272930 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"dataplane-ansible-ssh-private-key-secret" Dec 03 12:58:35 crc kubenswrapper[5052]: I1203 12:58:35.273209 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"openstack-aee-default-env" Dec 03 12:58:35 crc kubenswrapper[5052]: I1203 12:58:35.273538 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"edpm-compute-global-dockercfg-p5rjx" Dec 03 12:58:35 crc kubenswrapper[5052]: I1203 12:58:35.274430 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"dataplanenodeset-edpm-compute-global" Dec 03 12:58:35 crc kubenswrapper[5052]: I1203 12:58:35.280718 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/install-os-edpm-multinodeset-edpm-compute-global-zkjjj"] Dec 03 12:58:35 crc kubenswrapper[5052]: I1203 12:58:35.442919 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/bf5f88a2-5ab8-4878-a8ba-8541fa324a8e-inventory\") pod \"install-os-edpm-multinodeset-edpm-compute-global-zkjjj\" (UID: \"bf5f88a2-5ab8-4878-a8ba-8541fa324a8e\") " pod="openstack-kuttl-tests/install-os-edpm-multinodeset-edpm-compute-global-zkjjj" Dec 03 12:58:35 crc kubenswrapper[5052]: I1203 12:58:35.442976 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wwm42\" (UniqueName: \"kubernetes.io/projected/bf5f88a2-5ab8-4878-a8ba-8541fa324a8e-kube-api-access-wwm42\") pod \"install-os-edpm-multinodeset-edpm-compute-global-zkjjj\" (UID: \"bf5f88a2-5ab8-4878-a8ba-8541fa324a8e\") " pod="openstack-kuttl-tests/install-os-edpm-multinodeset-edpm-compute-global-zkjjj" Dec 03 12:58:35 crc kubenswrapper[5052]: I1203 12:58:35.443012 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/bf5f88a2-5ab8-4878-a8ba-8541fa324a8e-ssh-key\") pod \"install-os-edpm-multinodeset-edpm-compute-global-zkjjj\" (UID: \"bf5f88a2-5ab8-4878-a8ba-8541fa324a8e\") " pod="openstack-kuttl-tests/install-os-edpm-multinodeset-edpm-compute-global-zkjjj" Dec 03 12:58:35 crc kubenswrapper[5052]: I1203 12:58:35.544058 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/bf5f88a2-5ab8-4878-a8ba-8541fa324a8e-inventory\") pod \"install-os-edpm-multinodeset-edpm-compute-global-zkjjj\" (UID: \"bf5f88a2-5ab8-4878-a8ba-8541fa324a8e\") " pod="openstack-kuttl-tests/install-os-edpm-multinodeset-edpm-compute-global-zkjjj" Dec 03 12:58:35 crc kubenswrapper[5052]: I1203 12:58:35.544109 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-wwm42\" (UniqueName: \"kubernetes.io/projected/bf5f88a2-5ab8-4878-a8ba-8541fa324a8e-kube-api-access-wwm42\") pod \"install-os-edpm-multinodeset-edpm-compute-global-zkjjj\" (UID: \"bf5f88a2-5ab8-4878-a8ba-8541fa324a8e\") " pod="openstack-kuttl-tests/install-os-edpm-multinodeset-edpm-compute-global-zkjjj" Dec 03 12:58:35 crc kubenswrapper[5052]: I1203 12:58:35.544133 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/bf5f88a2-5ab8-4878-a8ba-8541fa324a8e-ssh-key\") pod \"install-os-edpm-multinodeset-edpm-compute-global-zkjjj\" (UID: \"bf5f88a2-5ab8-4878-a8ba-8541fa324a8e\") " pod="openstack-kuttl-tests/install-os-edpm-multinodeset-edpm-compute-global-zkjjj" Dec 03 12:58:35 crc kubenswrapper[5052]: I1203 12:58:35.548562 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/bf5f88a2-5ab8-4878-a8ba-8541fa324a8e-inventory\") pod \"install-os-edpm-multinodeset-edpm-compute-global-zkjjj\" (UID: \"bf5f88a2-5ab8-4878-a8ba-8541fa324a8e\") " pod="openstack-kuttl-tests/install-os-edpm-multinodeset-edpm-compute-global-zkjjj" Dec 03 12:58:35 crc kubenswrapper[5052]: I1203 12:58:35.548641 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/bf5f88a2-5ab8-4878-a8ba-8541fa324a8e-ssh-key\") pod \"install-os-edpm-multinodeset-edpm-compute-global-zkjjj\" (UID: \"bf5f88a2-5ab8-4878-a8ba-8541fa324a8e\") " pod="openstack-kuttl-tests/install-os-edpm-multinodeset-edpm-compute-global-zkjjj" Dec 03 12:58:35 crc kubenswrapper[5052]: I1203 12:58:35.561468 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-wwm42\" (UniqueName: \"kubernetes.io/projected/bf5f88a2-5ab8-4878-a8ba-8541fa324a8e-kube-api-access-wwm42\") pod \"install-os-edpm-multinodeset-edpm-compute-global-zkjjj\" (UID: \"bf5f88a2-5ab8-4878-a8ba-8541fa324a8e\") " pod="openstack-kuttl-tests/install-os-edpm-multinodeset-edpm-compute-global-zkjjj" Dec 03 12:58:35 crc kubenswrapper[5052]: I1203 12:58:35.595447 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/install-os-edpm-multinodeset-edpm-compute-global-zkjjj" Dec 03 12:58:36 crc kubenswrapper[5052]: W1203 12:58:36.000604 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbf5f88a2_5ab8_4878_a8ba_8541fa324a8e.slice/crio-f15bf58df4f313fe7ee0a76a23a3cfb4853c163ac59114619a27bd8987d93c6a WatchSource:0}: Error finding container f15bf58df4f313fe7ee0a76a23a3cfb4853c163ac59114619a27bd8987d93c6a: Status 404 returned error can't find the container with id f15bf58df4f313fe7ee0a76a23a3cfb4853c163ac59114619a27bd8987d93c6a Dec 03 12:58:36 crc kubenswrapper[5052]: I1203 12:58:36.007420 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/install-os-edpm-multinodeset-edpm-compute-global-zkjjj"] Dec 03 12:58:36 crc kubenswrapper[5052]: I1203 12:58:36.209580 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/install-os-edpm-multinodeset-edpm-compute-global-zkjjj" event={"ID":"bf5f88a2-5ab8-4878-a8ba-8541fa324a8e","Type":"ContainerStarted","Data":"f15bf58df4f313fe7ee0a76a23a3cfb4853c163ac59114619a27bd8987d93c6a"} Dec 03 12:58:37 crc kubenswrapper[5052]: I1203 12:58:37.218683 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/install-os-edpm-multinodeset-edpm-compute-global-zkjjj" event={"ID":"bf5f88a2-5ab8-4878-a8ba-8541fa324a8e","Type":"ContainerStarted","Data":"1104c2716cc817488412d6a7a780f7984a797cc98e5ee047d93cc2777df6e2e1"} Dec 03 12:58:37 crc kubenswrapper[5052]: I1203 12:58:37.240622 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/install-os-edpm-multinodeset-edpm-compute-global-zkjjj" podStartSLOduration=1.756424051 podStartE2EDuration="2.240598128s" podCreationTimestamp="2025-12-03 12:58:35 +0000 UTC" firstStartedPulling="2025-12-03 12:58:36.003077497 +0000 UTC m=+5906.591319261" lastFinishedPulling="2025-12-03 12:58:36.487251574 +0000 UTC m=+5907.075493338" observedRunningTime="2025-12-03 12:58:37.234389331 +0000 UTC m=+5907.822631095" watchObservedRunningTime="2025-12-03 12:58:37.240598128 +0000 UTC m=+5907.828839892" Dec 03 12:58:38 crc kubenswrapper[5052]: I1203 12:58:38.228150 5052 generic.go:334] "Generic (PLEG): container finished" podID="bf5f88a2-5ab8-4878-a8ba-8541fa324a8e" containerID="1104c2716cc817488412d6a7a780f7984a797cc98e5ee047d93cc2777df6e2e1" exitCode=0 Dec 03 12:58:38 crc kubenswrapper[5052]: I1203 12:58:38.228200 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/install-os-edpm-multinodeset-edpm-compute-global-zkjjj" event={"ID":"bf5f88a2-5ab8-4878-a8ba-8541fa324a8e","Type":"ContainerDied","Data":"1104c2716cc817488412d6a7a780f7984a797cc98e5ee047d93cc2777df6e2e1"} Dec 03 12:58:38 crc kubenswrapper[5052]: I1203 12:58:38.776641 5052 scope.go:117] "RemoveContainer" containerID="8edec2df8b21527fbab0a86cfd43f0ada2120365035a681121d82c8e08fe2868" Dec 03 12:58:38 crc kubenswrapper[5052]: I1203 12:58:38.845960 5052 scope.go:117] "RemoveContainer" containerID="6dd5b4c49e1b08f4974bb1ad78b09649b89f7ce5636ef4999c8c4847ad475556" Dec 03 12:58:38 crc kubenswrapper[5052]: I1203 12:58:38.869054 5052 scope.go:117] "RemoveContainer" containerID="96b654a5e42ddde9fc6e98cc635309cb5313602a1cc9c873755f04530d171682" Dec 03 12:58:38 crc kubenswrapper[5052]: I1203 12:58:38.939428 5052 scope.go:117] "RemoveContainer" containerID="2af5f1d759ac9d0ae0920501c0ea568edd510c60defdbffba8b9de9b0b0ba71c" Dec 03 12:58:38 crc kubenswrapper[5052]: I1203 12:58:38.974498 5052 scope.go:117] "RemoveContainer" containerID="691c45bd03a54c99bcffac732eb2f453f8046a69209f84bde56957c8b1cd60f9" Dec 03 12:58:39 crc kubenswrapper[5052]: I1203 12:58:39.000297 5052 scope.go:117] "RemoveContainer" containerID="e4a30a41f3f5c0bc5325054597c83dfd7f01768f25444ec3d94e71d8f4c168ed" Dec 03 12:58:39 crc kubenswrapper[5052]: I1203 12:58:39.020379 5052 scope.go:117] "RemoveContainer" containerID="5e8290f1b75b23e41346e6776bb910c41c35f1e744bda7029a7a83d431241314" Dec 03 12:58:39 crc kubenswrapper[5052]: I1203 12:58:39.044798 5052 scope.go:117] "RemoveContainer" containerID="067b2bdc88fb8db62ca52315daa392d562dd49266ba0cde5f73a3a78796ed357" Dec 03 12:58:39 crc kubenswrapper[5052]: I1203 12:58:39.069126 5052 scope.go:117] "RemoveContainer" containerID="6529ebafbdabe5682caf908893845491ea6c63023a1e09a075b791c88d910044" Dec 03 12:58:39 crc kubenswrapper[5052]: I1203 12:58:39.432480 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/install-os-edpm-multinodeset-edpm-compute-global-zkjjj" Dec 03 12:58:39 crc kubenswrapper[5052]: I1203 12:58:39.607148 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/bf5f88a2-5ab8-4878-a8ba-8541fa324a8e-ssh-key\") pod \"bf5f88a2-5ab8-4878-a8ba-8541fa324a8e\" (UID: \"bf5f88a2-5ab8-4878-a8ba-8541fa324a8e\") " Dec 03 12:58:39 crc kubenswrapper[5052]: I1203 12:58:39.607196 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/bf5f88a2-5ab8-4878-a8ba-8541fa324a8e-inventory\") pod \"bf5f88a2-5ab8-4878-a8ba-8541fa324a8e\" (UID: \"bf5f88a2-5ab8-4878-a8ba-8541fa324a8e\") " Dec 03 12:58:39 crc kubenswrapper[5052]: I1203 12:58:39.607262 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wwm42\" (UniqueName: \"kubernetes.io/projected/bf5f88a2-5ab8-4878-a8ba-8541fa324a8e-kube-api-access-wwm42\") pod \"bf5f88a2-5ab8-4878-a8ba-8541fa324a8e\" (UID: \"bf5f88a2-5ab8-4878-a8ba-8541fa324a8e\") " Dec 03 12:58:39 crc kubenswrapper[5052]: I1203 12:58:39.612638 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bf5f88a2-5ab8-4878-a8ba-8541fa324a8e-kube-api-access-wwm42" (OuterVolumeSpecName: "kube-api-access-wwm42") pod "bf5f88a2-5ab8-4878-a8ba-8541fa324a8e" (UID: "bf5f88a2-5ab8-4878-a8ba-8541fa324a8e"). InnerVolumeSpecName "kube-api-access-wwm42". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:58:39 crc kubenswrapper[5052]: I1203 12:58:39.628119 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bf5f88a2-5ab8-4878-a8ba-8541fa324a8e-ssh-key" (OuterVolumeSpecName: "ssh-key") pod "bf5f88a2-5ab8-4878-a8ba-8541fa324a8e" (UID: "bf5f88a2-5ab8-4878-a8ba-8541fa324a8e"). InnerVolumeSpecName "ssh-key". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:58:39 crc kubenswrapper[5052]: I1203 12:58:39.631132 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bf5f88a2-5ab8-4878-a8ba-8541fa324a8e-inventory" (OuterVolumeSpecName: "inventory") pod "bf5f88a2-5ab8-4878-a8ba-8541fa324a8e" (UID: "bf5f88a2-5ab8-4878-a8ba-8541fa324a8e"). InnerVolumeSpecName "inventory". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:58:39 crc kubenswrapper[5052]: I1203 12:58:39.708470 5052 reconciler_common.go:293] "Volume detached for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/bf5f88a2-5ab8-4878-a8ba-8541fa324a8e-ssh-key\") on node \"crc\" DevicePath \"\"" Dec 03 12:58:39 crc kubenswrapper[5052]: I1203 12:58:39.708505 5052 reconciler_common.go:293] "Volume detached for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/bf5f88a2-5ab8-4878-a8ba-8541fa324a8e-inventory\") on node \"crc\" DevicePath \"\"" Dec 03 12:58:39 crc kubenswrapper[5052]: I1203 12:58:39.708539 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-wwm42\" (UniqueName: \"kubernetes.io/projected/bf5f88a2-5ab8-4878-a8ba-8541fa324a8e-kube-api-access-wwm42\") on node \"crc\" DevicePath \"\"" Dec 03 12:58:40 crc kubenswrapper[5052]: I1203 12:58:40.248874 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/install-os-edpm-multinodeset-edpm-compute-global-zkjjj" event={"ID":"bf5f88a2-5ab8-4878-a8ba-8541fa324a8e","Type":"ContainerDied","Data":"f15bf58df4f313fe7ee0a76a23a3cfb4853c163ac59114619a27bd8987d93c6a"} Dec 03 12:58:40 crc kubenswrapper[5052]: I1203 12:58:40.248916 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f15bf58df4f313fe7ee0a76a23a3cfb4853c163ac59114619a27bd8987d93c6a" Dec 03 12:58:40 crc kubenswrapper[5052]: I1203 12:58:40.248964 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/install-os-edpm-multinodeset-edpm-compute-global-zkjjj" Dec 03 12:58:40 crc kubenswrapper[5052]: I1203 12:58:40.299918 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/configure-os-edpm-multinodeset-edpm-compute-global-qgc2j"] Dec 03 12:58:40 crc kubenswrapper[5052]: E1203 12:58:40.300238 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="bf5f88a2-5ab8-4878-a8ba-8541fa324a8e" containerName="install-os-edpm-multinodeset-edpm-compute-global" Dec 03 12:58:40 crc kubenswrapper[5052]: I1203 12:58:40.300256 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="bf5f88a2-5ab8-4878-a8ba-8541fa324a8e" containerName="install-os-edpm-multinodeset-edpm-compute-global" Dec 03 12:58:40 crc kubenswrapper[5052]: I1203 12:58:40.300425 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="bf5f88a2-5ab8-4878-a8ba-8541fa324a8e" containerName="install-os-edpm-multinodeset-edpm-compute-global" Dec 03 12:58:40 crc kubenswrapper[5052]: I1203 12:58:40.301111 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/configure-os-edpm-multinodeset-edpm-compute-global-qgc2j" Dec 03 12:58:40 crc kubenswrapper[5052]: I1203 12:58:40.304666 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"openstack-aee-default-env" Dec 03 12:58:40 crc kubenswrapper[5052]: I1203 12:58:40.304708 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"edpm-compute-global-dockercfg-p5rjx" Dec 03 12:58:40 crc kubenswrapper[5052]: I1203 12:58:40.304836 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"dataplane-ansible-ssh-private-key-secret" Dec 03 12:58:40 crc kubenswrapper[5052]: I1203 12:58:40.305387 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"dataplanenodeset-edpm-compute-global" Dec 03 12:58:40 crc kubenswrapper[5052]: I1203 12:58:40.317546 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/configure-os-edpm-multinodeset-edpm-compute-global-qgc2j"] Dec 03 12:58:40 crc kubenswrapper[5052]: I1203 12:58:40.418686 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/35c9f4ce-47f8-4ff2-9716-6ca827dfdf89-inventory\") pod \"configure-os-edpm-multinodeset-edpm-compute-global-qgc2j\" (UID: \"35c9f4ce-47f8-4ff2-9716-6ca827dfdf89\") " pod="openstack-kuttl-tests/configure-os-edpm-multinodeset-edpm-compute-global-qgc2j" Dec 03 12:58:40 crc kubenswrapper[5052]: I1203 12:58:40.418932 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/35c9f4ce-47f8-4ff2-9716-6ca827dfdf89-ssh-key\") pod \"configure-os-edpm-multinodeset-edpm-compute-global-qgc2j\" (UID: \"35c9f4ce-47f8-4ff2-9716-6ca827dfdf89\") " pod="openstack-kuttl-tests/configure-os-edpm-multinodeset-edpm-compute-global-qgc2j" Dec 03 12:58:40 crc kubenswrapper[5052]: I1203 12:58:40.419020 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fxnqv\" (UniqueName: \"kubernetes.io/projected/35c9f4ce-47f8-4ff2-9716-6ca827dfdf89-kube-api-access-fxnqv\") pod \"configure-os-edpm-multinodeset-edpm-compute-global-qgc2j\" (UID: \"35c9f4ce-47f8-4ff2-9716-6ca827dfdf89\") " pod="openstack-kuttl-tests/configure-os-edpm-multinodeset-edpm-compute-global-qgc2j" Dec 03 12:58:40 crc kubenswrapper[5052]: I1203 12:58:40.520480 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/35c9f4ce-47f8-4ff2-9716-6ca827dfdf89-ssh-key\") pod \"configure-os-edpm-multinodeset-edpm-compute-global-qgc2j\" (UID: \"35c9f4ce-47f8-4ff2-9716-6ca827dfdf89\") " pod="openstack-kuttl-tests/configure-os-edpm-multinodeset-edpm-compute-global-qgc2j" Dec 03 12:58:40 crc kubenswrapper[5052]: I1203 12:58:40.520567 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-fxnqv\" (UniqueName: \"kubernetes.io/projected/35c9f4ce-47f8-4ff2-9716-6ca827dfdf89-kube-api-access-fxnqv\") pod \"configure-os-edpm-multinodeset-edpm-compute-global-qgc2j\" (UID: \"35c9f4ce-47f8-4ff2-9716-6ca827dfdf89\") " pod="openstack-kuttl-tests/configure-os-edpm-multinodeset-edpm-compute-global-qgc2j" Dec 03 12:58:40 crc kubenswrapper[5052]: I1203 12:58:40.520646 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/35c9f4ce-47f8-4ff2-9716-6ca827dfdf89-inventory\") pod \"configure-os-edpm-multinodeset-edpm-compute-global-qgc2j\" (UID: \"35c9f4ce-47f8-4ff2-9716-6ca827dfdf89\") " pod="openstack-kuttl-tests/configure-os-edpm-multinodeset-edpm-compute-global-qgc2j" Dec 03 12:58:40 crc kubenswrapper[5052]: I1203 12:58:40.527301 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/35c9f4ce-47f8-4ff2-9716-6ca827dfdf89-ssh-key\") pod \"configure-os-edpm-multinodeset-edpm-compute-global-qgc2j\" (UID: \"35c9f4ce-47f8-4ff2-9716-6ca827dfdf89\") " pod="openstack-kuttl-tests/configure-os-edpm-multinodeset-edpm-compute-global-qgc2j" Dec 03 12:58:40 crc kubenswrapper[5052]: I1203 12:58:40.534998 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/35c9f4ce-47f8-4ff2-9716-6ca827dfdf89-inventory\") pod \"configure-os-edpm-multinodeset-edpm-compute-global-qgc2j\" (UID: \"35c9f4ce-47f8-4ff2-9716-6ca827dfdf89\") " pod="openstack-kuttl-tests/configure-os-edpm-multinodeset-edpm-compute-global-qgc2j" Dec 03 12:58:40 crc kubenswrapper[5052]: I1203 12:58:40.543103 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-fxnqv\" (UniqueName: \"kubernetes.io/projected/35c9f4ce-47f8-4ff2-9716-6ca827dfdf89-kube-api-access-fxnqv\") pod \"configure-os-edpm-multinodeset-edpm-compute-global-qgc2j\" (UID: \"35c9f4ce-47f8-4ff2-9716-6ca827dfdf89\") " pod="openstack-kuttl-tests/configure-os-edpm-multinodeset-edpm-compute-global-qgc2j" Dec 03 12:58:40 crc kubenswrapper[5052]: I1203 12:58:40.620211 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/configure-os-edpm-multinodeset-edpm-compute-global-qgc2j" Dec 03 12:58:41 crc kubenswrapper[5052]: I1203 12:58:41.025613 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/configure-os-edpm-multinodeset-edpm-compute-global-qgc2j"] Dec 03 12:58:41 crc kubenswrapper[5052]: W1203 12:58:41.033723 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod35c9f4ce_47f8_4ff2_9716_6ca827dfdf89.slice/crio-a29a882a254b3fc334f50954a21b995fa77e203c9beb4782b000dcdd18f1df08 WatchSource:0}: Error finding container a29a882a254b3fc334f50954a21b995fa77e203c9beb4782b000dcdd18f1df08: Status 404 returned error can't find the container with id a29a882a254b3fc334f50954a21b995fa77e203c9beb4782b000dcdd18f1df08 Dec 03 12:58:41 crc kubenswrapper[5052]: I1203 12:58:41.258661 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/configure-os-edpm-multinodeset-edpm-compute-global-qgc2j" event={"ID":"35c9f4ce-47f8-4ff2-9716-6ca827dfdf89","Type":"ContainerStarted","Data":"a29a882a254b3fc334f50954a21b995fa77e203c9beb4782b000dcdd18f1df08"} Dec 03 12:58:41 crc kubenswrapper[5052]: I1203 12:58:41.674990 5052 scope.go:117] "RemoveContainer" containerID="69fa7fbcd2aceb7c53703a19508876f93a97dc9adabb79c16fb691d90f606d58" Dec 03 12:58:41 crc kubenswrapper[5052]: E1203 12:58:41.675612 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 12:58:42 crc kubenswrapper[5052]: I1203 12:58:42.270316 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/configure-os-edpm-multinodeset-edpm-compute-global-qgc2j" event={"ID":"35c9f4ce-47f8-4ff2-9716-6ca827dfdf89","Type":"ContainerStarted","Data":"69e36da60df181c814dcb00f2d0f2b1ce08d4683088cf9c43ddd1deeb7ba65a0"} Dec 03 12:58:42 crc kubenswrapper[5052]: I1203 12:58:42.287400 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/configure-os-edpm-multinodeset-edpm-compute-global-qgc2j" podStartSLOduration=1.570725752 podStartE2EDuration="2.287376147s" podCreationTimestamp="2025-12-03 12:58:40 +0000 UTC" firstStartedPulling="2025-12-03 12:58:41.035764534 +0000 UTC m=+5911.624006298" lastFinishedPulling="2025-12-03 12:58:41.752414929 +0000 UTC m=+5912.340656693" observedRunningTime="2025-12-03 12:58:42.283936218 +0000 UTC m=+5912.872177992" watchObservedRunningTime="2025-12-03 12:58:42.287376147 +0000 UTC m=+5912.875617911" Dec 03 12:58:43 crc kubenswrapper[5052]: E1203 12:58:43.454296 5052 cadvisor_stats_provider.go:516] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod35c9f4ce_47f8_4ff2_9716_6ca827dfdf89.slice/crio-69e36da60df181c814dcb00f2d0f2b1ce08d4683088cf9c43ddd1deeb7ba65a0.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod35c9f4ce_47f8_4ff2_9716_6ca827dfdf89.slice/crio-conmon-69e36da60df181c814dcb00f2d0f2b1ce08d4683088cf9c43ddd1deeb7ba65a0.scope\": RecentStats: unable to find data in memory cache]" Dec 03 12:58:44 crc kubenswrapper[5052]: I1203 12:58:44.288304 5052 generic.go:334] "Generic (PLEG): container finished" podID="35c9f4ce-47f8-4ff2-9716-6ca827dfdf89" containerID="69e36da60df181c814dcb00f2d0f2b1ce08d4683088cf9c43ddd1deeb7ba65a0" exitCode=0 Dec 03 12:58:44 crc kubenswrapper[5052]: I1203 12:58:44.288621 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/configure-os-edpm-multinodeset-edpm-compute-global-qgc2j" event={"ID":"35c9f4ce-47f8-4ff2-9716-6ca827dfdf89","Type":"ContainerDied","Data":"69e36da60df181c814dcb00f2d0f2b1ce08d4683088cf9c43ddd1deeb7ba65a0"} Dec 03 12:58:45 crc kubenswrapper[5052]: I1203 12:58:45.573595 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/configure-os-edpm-multinodeset-edpm-compute-global-qgc2j" Dec 03 12:58:45 crc kubenswrapper[5052]: I1203 12:58:45.699122 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fxnqv\" (UniqueName: \"kubernetes.io/projected/35c9f4ce-47f8-4ff2-9716-6ca827dfdf89-kube-api-access-fxnqv\") pod \"35c9f4ce-47f8-4ff2-9716-6ca827dfdf89\" (UID: \"35c9f4ce-47f8-4ff2-9716-6ca827dfdf89\") " Dec 03 12:58:45 crc kubenswrapper[5052]: I1203 12:58:45.699201 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/35c9f4ce-47f8-4ff2-9716-6ca827dfdf89-ssh-key\") pod \"35c9f4ce-47f8-4ff2-9716-6ca827dfdf89\" (UID: \"35c9f4ce-47f8-4ff2-9716-6ca827dfdf89\") " Dec 03 12:58:45 crc kubenswrapper[5052]: I1203 12:58:45.699256 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/35c9f4ce-47f8-4ff2-9716-6ca827dfdf89-inventory\") pod \"35c9f4ce-47f8-4ff2-9716-6ca827dfdf89\" (UID: \"35c9f4ce-47f8-4ff2-9716-6ca827dfdf89\") " Dec 03 12:58:45 crc kubenswrapper[5052]: I1203 12:58:45.705475 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/35c9f4ce-47f8-4ff2-9716-6ca827dfdf89-kube-api-access-fxnqv" (OuterVolumeSpecName: "kube-api-access-fxnqv") pod "35c9f4ce-47f8-4ff2-9716-6ca827dfdf89" (UID: "35c9f4ce-47f8-4ff2-9716-6ca827dfdf89"). InnerVolumeSpecName "kube-api-access-fxnqv". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:58:45 crc kubenswrapper[5052]: I1203 12:58:45.728478 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/35c9f4ce-47f8-4ff2-9716-6ca827dfdf89-ssh-key" (OuterVolumeSpecName: "ssh-key") pod "35c9f4ce-47f8-4ff2-9716-6ca827dfdf89" (UID: "35c9f4ce-47f8-4ff2-9716-6ca827dfdf89"). InnerVolumeSpecName "ssh-key". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:58:45 crc kubenswrapper[5052]: I1203 12:58:45.745671 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/35c9f4ce-47f8-4ff2-9716-6ca827dfdf89-inventory" (OuterVolumeSpecName: "inventory") pod "35c9f4ce-47f8-4ff2-9716-6ca827dfdf89" (UID: "35c9f4ce-47f8-4ff2-9716-6ca827dfdf89"). InnerVolumeSpecName "inventory". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:58:45 crc kubenswrapper[5052]: I1203 12:58:45.801378 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-fxnqv\" (UniqueName: \"kubernetes.io/projected/35c9f4ce-47f8-4ff2-9716-6ca827dfdf89-kube-api-access-fxnqv\") on node \"crc\" DevicePath \"\"" Dec 03 12:58:45 crc kubenswrapper[5052]: I1203 12:58:45.801667 5052 reconciler_common.go:293] "Volume detached for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/35c9f4ce-47f8-4ff2-9716-6ca827dfdf89-ssh-key\") on node \"crc\" DevicePath \"\"" Dec 03 12:58:45 crc kubenswrapper[5052]: I1203 12:58:45.801732 5052 reconciler_common.go:293] "Volume detached for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/35c9f4ce-47f8-4ff2-9716-6ca827dfdf89-inventory\") on node \"crc\" DevicePath \"\"" Dec 03 12:58:46 crc kubenswrapper[5052]: I1203 12:58:46.311221 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/configure-os-edpm-multinodeset-edpm-compute-global-qgc2j" event={"ID":"35c9f4ce-47f8-4ff2-9716-6ca827dfdf89","Type":"ContainerDied","Data":"a29a882a254b3fc334f50954a21b995fa77e203c9beb4782b000dcdd18f1df08"} Dec 03 12:58:46 crc kubenswrapper[5052]: I1203 12:58:46.311278 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a29a882a254b3fc334f50954a21b995fa77e203c9beb4782b000dcdd18f1df08" Dec 03 12:58:46 crc kubenswrapper[5052]: I1203 12:58:46.311346 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/configure-os-edpm-multinodeset-edpm-compute-global-qgc2j" Dec 03 12:58:46 crc kubenswrapper[5052]: I1203 12:58:46.407287 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/run-os-edpm-multinodeset-edpm-compute-global-q76r6"] Dec 03 12:58:46 crc kubenswrapper[5052]: E1203 12:58:46.407667 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="35c9f4ce-47f8-4ff2-9716-6ca827dfdf89" containerName="configure-os-edpm-multinodeset-edpm-compute-global" Dec 03 12:58:46 crc kubenswrapper[5052]: I1203 12:58:46.407690 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="35c9f4ce-47f8-4ff2-9716-6ca827dfdf89" containerName="configure-os-edpm-multinodeset-edpm-compute-global" Dec 03 12:58:46 crc kubenswrapper[5052]: I1203 12:58:46.407849 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="35c9f4ce-47f8-4ff2-9716-6ca827dfdf89" containerName="configure-os-edpm-multinodeset-edpm-compute-global" Dec 03 12:58:46 crc kubenswrapper[5052]: I1203 12:58:46.408373 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/run-os-edpm-multinodeset-edpm-compute-global-q76r6" Dec 03 12:58:46 crc kubenswrapper[5052]: I1203 12:58:46.410891 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"edpm-compute-global-dockercfg-p5rjx" Dec 03 12:58:46 crc kubenswrapper[5052]: I1203 12:58:46.412705 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"dataplane-ansible-ssh-private-key-secret" Dec 03 12:58:46 crc kubenswrapper[5052]: I1203 12:58:46.413503 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"openstack-aee-default-env" Dec 03 12:58:46 crc kubenswrapper[5052]: I1203 12:58:46.414342 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"dataplanenodeset-edpm-compute-global" Dec 03 12:58:46 crc kubenswrapper[5052]: I1203 12:58:46.419067 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/run-os-edpm-multinodeset-edpm-compute-global-q76r6"] Dec 03 12:58:46 crc kubenswrapper[5052]: I1203 12:58:46.511633 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/5abb2379-9bea-40e8-9a8c-d7accf747008-ssh-key\") pod \"run-os-edpm-multinodeset-edpm-compute-global-q76r6\" (UID: \"5abb2379-9bea-40e8-9a8c-d7accf747008\") " pod="openstack-kuttl-tests/run-os-edpm-multinodeset-edpm-compute-global-q76r6" Dec 03 12:58:46 crc kubenswrapper[5052]: I1203 12:58:46.511916 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f788f\" (UniqueName: \"kubernetes.io/projected/5abb2379-9bea-40e8-9a8c-d7accf747008-kube-api-access-f788f\") pod \"run-os-edpm-multinodeset-edpm-compute-global-q76r6\" (UID: \"5abb2379-9bea-40e8-9a8c-d7accf747008\") " pod="openstack-kuttl-tests/run-os-edpm-multinodeset-edpm-compute-global-q76r6" Dec 03 12:58:46 crc kubenswrapper[5052]: I1203 12:58:46.512149 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/5abb2379-9bea-40e8-9a8c-d7accf747008-inventory\") pod \"run-os-edpm-multinodeset-edpm-compute-global-q76r6\" (UID: \"5abb2379-9bea-40e8-9a8c-d7accf747008\") " pod="openstack-kuttl-tests/run-os-edpm-multinodeset-edpm-compute-global-q76r6" Dec 03 12:58:46 crc kubenswrapper[5052]: I1203 12:58:46.614167 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-f788f\" (UniqueName: \"kubernetes.io/projected/5abb2379-9bea-40e8-9a8c-d7accf747008-kube-api-access-f788f\") pod \"run-os-edpm-multinodeset-edpm-compute-global-q76r6\" (UID: \"5abb2379-9bea-40e8-9a8c-d7accf747008\") " pod="openstack-kuttl-tests/run-os-edpm-multinodeset-edpm-compute-global-q76r6" Dec 03 12:58:46 crc kubenswrapper[5052]: I1203 12:58:46.614618 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/5abb2379-9bea-40e8-9a8c-d7accf747008-inventory\") pod \"run-os-edpm-multinodeset-edpm-compute-global-q76r6\" (UID: \"5abb2379-9bea-40e8-9a8c-d7accf747008\") " pod="openstack-kuttl-tests/run-os-edpm-multinodeset-edpm-compute-global-q76r6" Dec 03 12:58:46 crc kubenswrapper[5052]: I1203 12:58:46.614719 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/5abb2379-9bea-40e8-9a8c-d7accf747008-ssh-key\") pod \"run-os-edpm-multinodeset-edpm-compute-global-q76r6\" (UID: \"5abb2379-9bea-40e8-9a8c-d7accf747008\") " pod="openstack-kuttl-tests/run-os-edpm-multinodeset-edpm-compute-global-q76r6" Dec 03 12:58:46 crc kubenswrapper[5052]: I1203 12:58:46.631824 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/5abb2379-9bea-40e8-9a8c-d7accf747008-ssh-key\") pod \"run-os-edpm-multinodeset-edpm-compute-global-q76r6\" (UID: \"5abb2379-9bea-40e8-9a8c-d7accf747008\") " pod="openstack-kuttl-tests/run-os-edpm-multinodeset-edpm-compute-global-q76r6" Dec 03 12:58:46 crc kubenswrapper[5052]: I1203 12:58:46.633346 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/5abb2379-9bea-40e8-9a8c-d7accf747008-inventory\") pod \"run-os-edpm-multinodeset-edpm-compute-global-q76r6\" (UID: \"5abb2379-9bea-40e8-9a8c-d7accf747008\") " pod="openstack-kuttl-tests/run-os-edpm-multinodeset-edpm-compute-global-q76r6" Dec 03 12:58:46 crc kubenswrapper[5052]: I1203 12:58:46.643022 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-f788f\" (UniqueName: \"kubernetes.io/projected/5abb2379-9bea-40e8-9a8c-d7accf747008-kube-api-access-f788f\") pod \"run-os-edpm-multinodeset-edpm-compute-global-q76r6\" (UID: \"5abb2379-9bea-40e8-9a8c-d7accf747008\") " pod="openstack-kuttl-tests/run-os-edpm-multinodeset-edpm-compute-global-q76r6" Dec 03 12:58:46 crc kubenswrapper[5052]: I1203 12:58:46.725928 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/run-os-edpm-multinodeset-edpm-compute-global-q76r6" Dec 03 12:58:47 crc kubenswrapper[5052]: W1203 12:58:47.207494 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5abb2379_9bea_40e8_9a8c_d7accf747008.slice/crio-3b2ddaf22fff23220a74660a374441c7ff71af070a15c1276b4e419b98d46c36 WatchSource:0}: Error finding container 3b2ddaf22fff23220a74660a374441c7ff71af070a15c1276b4e419b98d46c36: Status 404 returned error can't find the container with id 3b2ddaf22fff23220a74660a374441c7ff71af070a15c1276b4e419b98d46c36 Dec 03 12:58:47 crc kubenswrapper[5052]: I1203 12:58:47.217738 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/run-os-edpm-multinodeset-edpm-compute-global-q76r6"] Dec 03 12:58:47 crc kubenswrapper[5052]: I1203 12:58:47.321210 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/run-os-edpm-multinodeset-edpm-compute-global-q76r6" event={"ID":"5abb2379-9bea-40e8-9a8c-d7accf747008","Type":"ContainerStarted","Data":"3b2ddaf22fff23220a74660a374441c7ff71af070a15c1276b4e419b98d46c36"} Dec 03 12:58:48 crc kubenswrapper[5052]: I1203 12:58:48.333527 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/run-os-edpm-multinodeset-edpm-compute-global-q76r6" event={"ID":"5abb2379-9bea-40e8-9a8c-d7accf747008","Type":"ContainerStarted","Data":"c2929449b63508dbdd5ff4b0cb9bdb1f65cf2a3423afd1a23614e1e61951b61c"} Dec 03 12:58:48 crc kubenswrapper[5052]: I1203 12:58:48.352772 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/run-os-edpm-multinodeset-edpm-compute-global-q76r6" podStartSLOduration=1.656326049 podStartE2EDuration="2.352753346s" podCreationTimestamp="2025-12-03 12:58:46 +0000 UTC" firstStartedPulling="2025-12-03 12:58:47.211251281 +0000 UTC m=+5917.799493045" lastFinishedPulling="2025-12-03 12:58:47.907678568 +0000 UTC m=+5918.495920342" observedRunningTime="2025-12-03 12:58:48.351217842 +0000 UTC m=+5918.939459606" watchObservedRunningTime="2025-12-03 12:58:48.352753346 +0000 UTC m=+5918.940995110" Dec 03 12:58:49 crc kubenswrapper[5052]: I1203 12:58:49.343635 5052 generic.go:334] "Generic (PLEG): container finished" podID="5abb2379-9bea-40e8-9a8c-d7accf747008" containerID="c2929449b63508dbdd5ff4b0cb9bdb1f65cf2a3423afd1a23614e1e61951b61c" exitCode=0 Dec 03 12:58:49 crc kubenswrapper[5052]: I1203 12:58:49.343679 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/run-os-edpm-multinodeset-edpm-compute-global-q76r6" event={"ID":"5abb2379-9bea-40e8-9a8c-d7accf747008","Type":"ContainerDied","Data":"c2929449b63508dbdd5ff4b0cb9bdb1f65cf2a3423afd1a23614e1e61951b61c"} Dec 03 12:58:50 crc kubenswrapper[5052]: I1203 12:58:50.647797 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/run-os-edpm-multinodeset-edpm-compute-global-q76r6" Dec 03 12:58:50 crc kubenswrapper[5052]: I1203 12:58:50.777121 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f788f\" (UniqueName: \"kubernetes.io/projected/5abb2379-9bea-40e8-9a8c-d7accf747008-kube-api-access-f788f\") pod \"5abb2379-9bea-40e8-9a8c-d7accf747008\" (UID: \"5abb2379-9bea-40e8-9a8c-d7accf747008\") " Dec 03 12:58:50 crc kubenswrapper[5052]: I1203 12:58:50.777331 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/5abb2379-9bea-40e8-9a8c-d7accf747008-ssh-key\") pod \"5abb2379-9bea-40e8-9a8c-d7accf747008\" (UID: \"5abb2379-9bea-40e8-9a8c-d7accf747008\") " Dec 03 12:58:50 crc kubenswrapper[5052]: I1203 12:58:50.778142 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/5abb2379-9bea-40e8-9a8c-d7accf747008-inventory\") pod \"5abb2379-9bea-40e8-9a8c-d7accf747008\" (UID: \"5abb2379-9bea-40e8-9a8c-d7accf747008\") " Dec 03 12:58:50 crc kubenswrapper[5052]: I1203 12:58:50.783341 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5abb2379-9bea-40e8-9a8c-d7accf747008-kube-api-access-f788f" (OuterVolumeSpecName: "kube-api-access-f788f") pod "5abb2379-9bea-40e8-9a8c-d7accf747008" (UID: "5abb2379-9bea-40e8-9a8c-d7accf747008"). InnerVolumeSpecName "kube-api-access-f788f". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:58:50 crc kubenswrapper[5052]: I1203 12:58:50.803628 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5abb2379-9bea-40e8-9a8c-d7accf747008-ssh-key" (OuterVolumeSpecName: "ssh-key") pod "5abb2379-9bea-40e8-9a8c-d7accf747008" (UID: "5abb2379-9bea-40e8-9a8c-d7accf747008"). InnerVolumeSpecName "ssh-key". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:58:50 crc kubenswrapper[5052]: I1203 12:58:50.809633 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5abb2379-9bea-40e8-9a8c-d7accf747008-inventory" (OuterVolumeSpecName: "inventory") pod "5abb2379-9bea-40e8-9a8c-d7accf747008" (UID: "5abb2379-9bea-40e8-9a8c-d7accf747008"). InnerVolumeSpecName "inventory". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:58:50 crc kubenswrapper[5052]: I1203 12:58:50.879448 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-f788f\" (UniqueName: \"kubernetes.io/projected/5abb2379-9bea-40e8-9a8c-d7accf747008-kube-api-access-f788f\") on node \"crc\" DevicePath \"\"" Dec 03 12:58:50 crc kubenswrapper[5052]: I1203 12:58:50.879491 5052 reconciler_common.go:293] "Volume detached for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/5abb2379-9bea-40e8-9a8c-d7accf747008-ssh-key\") on node \"crc\" DevicePath \"\"" Dec 03 12:58:50 crc kubenswrapper[5052]: I1203 12:58:50.879519 5052 reconciler_common.go:293] "Volume detached for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/5abb2379-9bea-40e8-9a8c-d7accf747008-inventory\") on node \"crc\" DevicePath \"\"" Dec 03 12:58:51 crc kubenswrapper[5052]: I1203 12:58:51.361587 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/run-os-edpm-multinodeset-edpm-compute-global-q76r6" event={"ID":"5abb2379-9bea-40e8-9a8c-d7accf747008","Type":"ContainerDied","Data":"3b2ddaf22fff23220a74660a374441c7ff71af070a15c1276b4e419b98d46c36"} Dec 03 12:58:51 crc kubenswrapper[5052]: I1203 12:58:51.361634 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3b2ddaf22fff23220a74660a374441c7ff71af070a15c1276b4e419b98d46c36" Dec 03 12:58:51 crc kubenswrapper[5052]: I1203 12:58:51.361676 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/run-os-edpm-multinodeset-edpm-compute-global-q76r6" Dec 03 12:58:51 crc kubenswrapper[5052]: I1203 12:58:51.430901 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/install-certs-edpm-multinodeset-edpm-compute-global-6dqgm"] Dec 03 12:58:51 crc kubenswrapper[5052]: E1203 12:58:51.431284 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="5abb2379-9bea-40e8-9a8c-d7accf747008" containerName="run-os-edpm-multinodeset-edpm-compute-global" Dec 03 12:58:51 crc kubenswrapper[5052]: I1203 12:58:51.431306 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="5abb2379-9bea-40e8-9a8c-d7accf747008" containerName="run-os-edpm-multinodeset-edpm-compute-global" Dec 03 12:58:51 crc kubenswrapper[5052]: I1203 12:58:51.431484 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="5abb2379-9bea-40e8-9a8c-d7accf747008" containerName="run-os-edpm-multinodeset-edpm-compute-global" Dec 03 12:58:51 crc kubenswrapper[5052]: I1203 12:58:51.432079 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/install-certs-edpm-multinodeset-edpm-compute-global-6dqgm" Dec 03 12:58:51 crc kubenswrapper[5052]: I1203 12:58:51.433941 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"combined-ca-bundle" Dec 03 12:58:51 crc kubenswrapper[5052]: I1203 12:58:51.435105 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"openstack-aee-default-env" Dec 03 12:58:51 crc kubenswrapper[5052]: I1203 12:58:51.436003 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"edpm-compute-global-dockercfg-p5rjx" Dec 03 12:58:51 crc kubenswrapper[5052]: I1203 12:58:51.436176 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"dataplane-ansible-ssh-private-key-secret" Dec 03 12:58:51 crc kubenswrapper[5052]: I1203 12:58:51.436357 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"dataplanenodeset-edpm-compute-global" Dec 03 12:58:51 crc kubenswrapper[5052]: I1203 12:58:51.454921 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/install-certs-edpm-multinodeset-edpm-compute-global-6dqgm"] Dec 03 12:58:51 crc kubenswrapper[5052]: I1203 12:58:51.591294 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5d8a070f-f37f-42b9-a47b-cca83f8ffda6-ovn-combined-ca-bundle\") pod \"install-certs-edpm-multinodeset-edpm-compute-global-6dqgm\" (UID: \"5d8a070f-f37f-42b9-a47b-cca83f8ffda6\") " pod="openstack-kuttl-tests/install-certs-edpm-multinodeset-edpm-compute-global-6dqgm" Dec 03 12:58:51 crc kubenswrapper[5052]: I1203 12:58:51.591335 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"neutron-sriov-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5d8a070f-f37f-42b9-a47b-cca83f8ffda6-neutron-sriov-combined-ca-bundle\") pod \"install-certs-edpm-multinodeset-edpm-compute-global-6dqgm\" (UID: \"5d8a070f-f37f-42b9-a47b-cca83f8ffda6\") " pod="openstack-kuttl-tests/install-certs-edpm-multinodeset-edpm-compute-global-6dqgm" Dec 03 12:58:51 crc kubenswrapper[5052]: I1203 12:58:51.591361 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nova-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5d8a070f-f37f-42b9-a47b-cca83f8ffda6-nova-combined-ca-bundle\") pod \"install-certs-edpm-multinodeset-edpm-compute-global-6dqgm\" (UID: \"5d8a070f-f37f-42b9-a47b-cca83f8ffda6\") " pod="openstack-kuttl-tests/install-certs-edpm-multinodeset-edpm-compute-global-6dqgm" Dec 03 12:58:51 crc kubenswrapper[5052]: I1203 12:58:51.591381 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/5d8a070f-f37f-42b9-a47b-cca83f8ffda6-ssh-key\") pod \"install-certs-edpm-multinodeset-edpm-compute-global-6dqgm\" (UID: \"5d8a070f-f37f-42b9-a47b-cca83f8ffda6\") " pod="openstack-kuttl-tests/install-certs-edpm-multinodeset-edpm-compute-global-6dqgm" Dec 03 12:58:51 crc kubenswrapper[5052]: I1203 12:58:51.591415 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lf4xg\" (UniqueName: \"kubernetes.io/projected/5d8a070f-f37f-42b9-a47b-cca83f8ffda6-kube-api-access-lf4xg\") pod \"install-certs-edpm-multinodeset-edpm-compute-global-6dqgm\" (UID: \"5d8a070f-f37f-42b9-a47b-cca83f8ffda6\") " pod="openstack-kuttl-tests/install-certs-edpm-multinodeset-edpm-compute-global-6dqgm" Dec 03 12:58:51 crc kubenswrapper[5052]: I1203 12:58:51.591436 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"custom-global-service-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5d8a070f-f37f-42b9-a47b-cca83f8ffda6-custom-global-service-combined-ca-bundle\") pod \"install-certs-edpm-multinodeset-edpm-compute-global-6dqgm\" (UID: \"5d8a070f-f37f-42b9-a47b-cca83f8ffda6\") " pod="openstack-kuttl-tests/install-certs-edpm-multinodeset-edpm-compute-global-6dqgm" Dec 03 12:58:51 crc kubenswrapper[5052]: I1203 12:58:51.591455 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/5d8a070f-f37f-42b9-a47b-cca83f8ffda6-inventory\") pod \"install-certs-edpm-multinodeset-edpm-compute-global-6dqgm\" (UID: \"5d8a070f-f37f-42b9-a47b-cca83f8ffda6\") " pod="openstack-kuttl-tests/install-certs-edpm-multinodeset-edpm-compute-global-6dqgm" Dec 03 12:58:51 crc kubenswrapper[5052]: I1203 12:58:51.591542 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"neutron-dhcp-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5d8a070f-f37f-42b9-a47b-cca83f8ffda6-neutron-dhcp-combined-ca-bundle\") pod \"install-certs-edpm-multinodeset-edpm-compute-global-6dqgm\" (UID: \"5d8a070f-f37f-42b9-a47b-cca83f8ffda6\") " pod="openstack-kuttl-tests/install-certs-edpm-multinodeset-edpm-compute-global-6dqgm" Dec 03 12:58:51 crc kubenswrapper[5052]: I1203 12:58:51.591563 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bootstrap-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5d8a070f-f37f-42b9-a47b-cca83f8ffda6-bootstrap-combined-ca-bundle\") pod \"install-certs-edpm-multinodeset-edpm-compute-global-6dqgm\" (UID: \"5d8a070f-f37f-42b9-a47b-cca83f8ffda6\") " pod="openstack-kuttl-tests/install-certs-edpm-multinodeset-edpm-compute-global-6dqgm" Dec 03 12:58:51 crc kubenswrapper[5052]: I1203 12:58:51.591593 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"neutron-ovn-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5d8a070f-f37f-42b9-a47b-cca83f8ffda6-neutron-ovn-combined-ca-bundle\") pod \"install-certs-edpm-multinodeset-edpm-compute-global-6dqgm\" (UID: \"5d8a070f-f37f-42b9-a47b-cca83f8ffda6\") " pod="openstack-kuttl-tests/install-certs-edpm-multinodeset-edpm-compute-global-6dqgm" Dec 03 12:58:51 crc kubenswrapper[5052]: I1203 12:58:51.591611 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"libvirt-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5d8a070f-f37f-42b9-a47b-cca83f8ffda6-libvirt-combined-ca-bundle\") pod \"install-certs-edpm-multinodeset-edpm-compute-global-6dqgm\" (UID: \"5d8a070f-f37f-42b9-a47b-cca83f8ffda6\") " pod="openstack-kuttl-tests/install-certs-edpm-multinodeset-edpm-compute-global-6dqgm" Dec 03 12:58:51 crc kubenswrapper[5052]: I1203 12:58:51.591629 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"neutron-metadata-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5d8a070f-f37f-42b9-a47b-cca83f8ffda6-neutron-metadata-combined-ca-bundle\") pod \"install-certs-edpm-multinodeset-edpm-compute-global-6dqgm\" (UID: \"5d8a070f-f37f-42b9-a47b-cca83f8ffda6\") " pod="openstack-kuttl-tests/install-certs-edpm-multinodeset-edpm-compute-global-6dqgm" Dec 03 12:58:51 crc kubenswrapper[5052]: I1203 12:58:51.693426 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"neutron-dhcp-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5d8a070f-f37f-42b9-a47b-cca83f8ffda6-neutron-dhcp-combined-ca-bundle\") pod \"install-certs-edpm-multinodeset-edpm-compute-global-6dqgm\" (UID: \"5d8a070f-f37f-42b9-a47b-cca83f8ffda6\") " pod="openstack-kuttl-tests/install-certs-edpm-multinodeset-edpm-compute-global-6dqgm" Dec 03 12:58:51 crc kubenswrapper[5052]: I1203 12:58:51.693478 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"bootstrap-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5d8a070f-f37f-42b9-a47b-cca83f8ffda6-bootstrap-combined-ca-bundle\") pod \"install-certs-edpm-multinodeset-edpm-compute-global-6dqgm\" (UID: \"5d8a070f-f37f-42b9-a47b-cca83f8ffda6\") " pod="openstack-kuttl-tests/install-certs-edpm-multinodeset-edpm-compute-global-6dqgm" Dec 03 12:58:51 crc kubenswrapper[5052]: I1203 12:58:51.693569 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"neutron-ovn-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5d8a070f-f37f-42b9-a47b-cca83f8ffda6-neutron-ovn-combined-ca-bundle\") pod \"install-certs-edpm-multinodeset-edpm-compute-global-6dqgm\" (UID: \"5d8a070f-f37f-42b9-a47b-cca83f8ffda6\") " pod="openstack-kuttl-tests/install-certs-edpm-multinodeset-edpm-compute-global-6dqgm" Dec 03 12:58:51 crc kubenswrapper[5052]: I1203 12:58:51.693789 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"libvirt-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5d8a070f-f37f-42b9-a47b-cca83f8ffda6-libvirt-combined-ca-bundle\") pod \"install-certs-edpm-multinodeset-edpm-compute-global-6dqgm\" (UID: \"5d8a070f-f37f-42b9-a47b-cca83f8ffda6\") " pod="openstack-kuttl-tests/install-certs-edpm-multinodeset-edpm-compute-global-6dqgm" Dec 03 12:58:51 crc kubenswrapper[5052]: I1203 12:58:51.693812 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"neutron-metadata-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5d8a070f-f37f-42b9-a47b-cca83f8ffda6-neutron-metadata-combined-ca-bundle\") pod \"install-certs-edpm-multinodeset-edpm-compute-global-6dqgm\" (UID: \"5d8a070f-f37f-42b9-a47b-cca83f8ffda6\") " pod="openstack-kuttl-tests/install-certs-edpm-multinodeset-edpm-compute-global-6dqgm" Dec 03 12:58:51 crc kubenswrapper[5052]: I1203 12:58:51.693853 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovn-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5d8a070f-f37f-42b9-a47b-cca83f8ffda6-ovn-combined-ca-bundle\") pod \"install-certs-edpm-multinodeset-edpm-compute-global-6dqgm\" (UID: \"5d8a070f-f37f-42b9-a47b-cca83f8ffda6\") " pod="openstack-kuttl-tests/install-certs-edpm-multinodeset-edpm-compute-global-6dqgm" Dec 03 12:58:51 crc kubenswrapper[5052]: I1203 12:58:51.693882 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"neutron-sriov-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5d8a070f-f37f-42b9-a47b-cca83f8ffda6-neutron-sriov-combined-ca-bundle\") pod \"install-certs-edpm-multinodeset-edpm-compute-global-6dqgm\" (UID: \"5d8a070f-f37f-42b9-a47b-cca83f8ffda6\") " pod="openstack-kuttl-tests/install-certs-edpm-multinodeset-edpm-compute-global-6dqgm" Dec 03 12:58:51 crc kubenswrapper[5052]: I1203 12:58:51.693907 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"nova-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5d8a070f-f37f-42b9-a47b-cca83f8ffda6-nova-combined-ca-bundle\") pod \"install-certs-edpm-multinodeset-edpm-compute-global-6dqgm\" (UID: \"5d8a070f-f37f-42b9-a47b-cca83f8ffda6\") " pod="openstack-kuttl-tests/install-certs-edpm-multinodeset-edpm-compute-global-6dqgm" Dec 03 12:58:51 crc kubenswrapper[5052]: I1203 12:58:51.693928 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/5d8a070f-f37f-42b9-a47b-cca83f8ffda6-ssh-key\") pod \"install-certs-edpm-multinodeset-edpm-compute-global-6dqgm\" (UID: \"5d8a070f-f37f-42b9-a47b-cca83f8ffda6\") " pod="openstack-kuttl-tests/install-certs-edpm-multinodeset-edpm-compute-global-6dqgm" Dec 03 12:58:51 crc kubenswrapper[5052]: I1203 12:58:51.693959 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-lf4xg\" (UniqueName: \"kubernetes.io/projected/5d8a070f-f37f-42b9-a47b-cca83f8ffda6-kube-api-access-lf4xg\") pod \"install-certs-edpm-multinodeset-edpm-compute-global-6dqgm\" (UID: \"5d8a070f-f37f-42b9-a47b-cca83f8ffda6\") " pod="openstack-kuttl-tests/install-certs-edpm-multinodeset-edpm-compute-global-6dqgm" Dec 03 12:58:51 crc kubenswrapper[5052]: I1203 12:58:51.693984 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"custom-global-service-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5d8a070f-f37f-42b9-a47b-cca83f8ffda6-custom-global-service-combined-ca-bundle\") pod \"install-certs-edpm-multinodeset-edpm-compute-global-6dqgm\" (UID: \"5d8a070f-f37f-42b9-a47b-cca83f8ffda6\") " pod="openstack-kuttl-tests/install-certs-edpm-multinodeset-edpm-compute-global-6dqgm" Dec 03 12:58:51 crc kubenswrapper[5052]: I1203 12:58:51.694003 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/5d8a070f-f37f-42b9-a47b-cca83f8ffda6-inventory\") pod \"install-certs-edpm-multinodeset-edpm-compute-global-6dqgm\" (UID: \"5d8a070f-f37f-42b9-a47b-cca83f8ffda6\") " pod="openstack-kuttl-tests/install-certs-edpm-multinodeset-edpm-compute-global-6dqgm" Dec 03 12:58:51 crc kubenswrapper[5052]: I1203 12:58:51.698231 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"nova-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5d8a070f-f37f-42b9-a47b-cca83f8ffda6-nova-combined-ca-bundle\") pod \"install-certs-edpm-multinodeset-edpm-compute-global-6dqgm\" (UID: \"5d8a070f-f37f-42b9-a47b-cca83f8ffda6\") " pod="openstack-kuttl-tests/install-certs-edpm-multinodeset-edpm-compute-global-6dqgm" Dec 03 12:58:51 crc kubenswrapper[5052]: I1203 12:58:51.698245 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"neutron-dhcp-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5d8a070f-f37f-42b9-a47b-cca83f8ffda6-neutron-dhcp-combined-ca-bundle\") pod \"install-certs-edpm-multinodeset-edpm-compute-global-6dqgm\" (UID: \"5d8a070f-f37f-42b9-a47b-cca83f8ffda6\") " pod="openstack-kuttl-tests/install-certs-edpm-multinodeset-edpm-compute-global-6dqgm" Dec 03 12:58:51 crc kubenswrapper[5052]: I1203 12:58:51.698359 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"neutron-ovn-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5d8a070f-f37f-42b9-a47b-cca83f8ffda6-neutron-ovn-combined-ca-bundle\") pod \"install-certs-edpm-multinodeset-edpm-compute-global-6dqgm\" (UID: \"5d8a070f-f37f-42b9-a47b-cca83f8ffda6\") " pod="openstack-kuttl-tests/install-certs-edpm-multinodeset-edpm-compute-global-6dqgm" Dec 03 12:58:51 crc kubenswrapper[5052]: I1203 12:58:51.698838 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"neutron-metadata-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5d8a070f-f37f-42b9-a47b-cca83f8ffda6-neutron-metadata-combined-ca-bundle\") pod \"install-certs-edpm-multinodeset-edpm-compute-global-6dqgm\" (UID: \"5d8a070f-f37f-42b9-a47b-cca83f8ffda6\") " pod="openstack-kuttl-tests/install-certs-edpm-multinodeset-edpm-compute-global-6dqgm" Dec 03 12:58:51 crc kubenswrapper[5052]: I1203 12:58:51.699187 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"bootstrap-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5d8a070f-f37f-42b9-a47b-cca83f8ffda6-bootstrap-combined-ca-bundle\") pod \"install-certs-edpm-multinodeset-edpm-compute-global-6dqgm\" (UID: \"5d8a070f-f37f-42b9-a47b-cca83f8ffda6\") " pod="openstack-kuttl-tests/install-certs-edpm-multinodeset-edpm-compute-global-6dqgm" Dec 03 12:58:51 crc kubenswrapper[5052]: I1203 12:58:51.699861 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/5d8a070f-f37f-42b9-a47b-cca83f8ffda6-inventory\") pod \"install-certs-edpm-multinodeset-edpm-compute-global-6dqgm\" (UID: \"5d8a070f-f37f-42b9-a47b-cca83f8ffda6\") " pod="openstack-kuttl-tests/install-certs-edpm-multinodeset-edpm-compute-global-6dqgm" Dec 03 12:58:51 crc kubenswrapper[5052]: I1203 12:58:51.700042 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"libvirt-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5d8a070f-f37f-42b9-a47b-cca83f8ffda6-libvirt-combined-ca-bundle\") pod \"install-certs-edpm-multinodeset-edpm-compute-global-6dqgm\" (UID: \"5d8a070f-f37f-42b9-a47b-cca83f8ffda6\") " pod="openstack-kuttl-tests/install-certs-edpm-multinodeset-edpm-compute-global-6dqgm" Dec 03 12:58:51 crc kubenswrapper[5052]: I1203 12:58:51.701571 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"neutron-sriov-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5d8a070f-f37f-42b9-a47b-cca83f8ffda6-neutron-sriov-combined-ca-bundle\") pod \"install-certs-edpm-multinodeset-edpm-compute-global-6dqgm\" (UID: \"5d8a070f-f37f-42b9-a47b-cca83f8ffda6\") " pod="openstack-kuttl-tests/install-certs-edpm-multinodeset-edpm-compute-global-6dqgm" Dec 03 12:58:51 crc kubenswrapper[5052]: I1203 12:58:51.702070 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"custom-global-service-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5d8a070f-f37f-42b9-a47b-cca83f8ffda6-custom-global-service-combined-ca-bundle\") pod \"install-certs-edpm-multinodeset-edpm-compute-global-6dqgm\" (UID: \"5d8a070f-f37f-42b9-a47b-cca83f8ffda6\") " pod="openstack-kuttl-tests/install-certs-edpm-multinodeset-edpm-compute-global-6dqgm" Dec 03 12:58:51 crc kubenswrapper[5052]: I1203 12:58:51.702824 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/5d8a070f-f37f-42b9-a47b-cca83f8ffda6-ssh-key\") pod \"install-certs-edpm-multinodeset-edpm-compute-global-6dqgm\" (UID: \"5d8a070f-f37f-42b9-a47b-cca83f8ffda6\") " pod="openstack-kuttl-tests/install-certs-edpm-multinodeset-edpm-compute-global-6dqgm" Dec 03 12:58:51 crc kubenswrapper[5052]: I1203 12:58:51.703182 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovn-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5d8a070f-f37f-42b9-a47b-cca83f8ffda6-ovn-combined-ca-bundle\") pod \"install-certs-edpm-multinodeset-edpm-compute-global-6dqgm\" (UID: \"5d8a070f-f37f-42b9-a47b-cca83f8ffda6\") " pod="openstack-kuttl-tests/install-certs-edpm-multinodeset-edpm-compute-global-6dqgm" Dec 03 12:58:51 crc kubenswrapper[5052]: I1203 12:58:51.710859 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-lf4xg\" (UniqueName: \"kubernetes.io/projected/5d8a070f-f37f-42b9-a47b-cca83f8ffda6-kube-api-access-lf4xg\") pod \"install-certs-edpm-multinodeset-edpm-compute-global-6dqgm\" (UID: \"5d8a070f-f37f-42b9-a47b-cca83f8ffda6\") " pod="openstack-kuttl-tests/install-certs-edpm-multinodeset-edpm-compute-global-6dqgm" Dec 03 12:58:51 crc kubenswrapper[5052]: I1203 12:58:51.752826 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/install-certs-edpm-multinodeset-edpm-compute-global-6dqgm" Dec 03 12:58:52 crc kubenswrapper[5052]: I1203 12:58:52.214327 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/install-certs-edpm-multinodeset-edpm-compute-global-6dqgm"] Dec 03 12:58:52 crc kubenswrapper[5052]: W1203 12:58:52.220416 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5d8a070f_f37f_42b9_a47b_cca83f8ffda6.slice/crio-01da662b7039ecb72ebf5b007aa4ac8dfe32873ae30a75470ddf8a914c572c98 WatchSource:0}: Error finding container 01da662b7039ecb72ebf5b007aa4ac8dfe32873ae30a75470ddf8a914c572c98: Status 404 returned error can't find the container with id 01da662b7039ecb72ebf5b007aa4ac8dfe32873ae30a75470ddf8a914c572c98 Dec 03 12:58:52 crc kubenswrapper[5052]: I1203 12:58:52.370352 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/install-certs-edpm-multinodeset-edpm-compute-global-6dqgm" event={"ID":"5d8a070f-f37f-42b9-a47b-cca83f8ffda6","Type":"ContainerStarted","Data":"01da662b7039ecb72ebf5b007aa4ac8dfe32873ae30a75470ddf8a914c572c98"} Dec 03 12:58:53 crc kubenswrapper[5052]: I1203 12:58:53.380828 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/install-certs-edpm-multinodeset-edpm-compute-global-6dqgm" event={"ID":"5d8a070f-f37f-42b9-a47b-cca83f8ffda6","Type":"ContainerStarted","Data":"351eb338913ef40dcaca07a18d88b7d7bee6b0144cace109e505631f705e3ed3"} Dec 03 12:58:53 crc kubenswrapper[5052]: I1203 12:58:53.396493 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/install-certs-edpm-multinodeset-edpm-compute-global-6dqgm" podStartSLOduration=1.8843387310000002 podStartE2EDuration="2.396473647s" podCreationTimestamp="2025-12-03 12:58:51 +0000 UTC" firstStartedPulling="2025-12-03 12:58:52.22374116 +0000 UTC m=+5922.811982924" lastFinishedPulling="2025-12-03 12:58:52.735876076 +0000 UTC m=+5923.324117840" observedRunningTime="2025-12-03 12:58:53.396315133 +0000 UTC m=+5923.984556897" watchObservedRunningTime="2025-12-03 12:58:53.396473647 +0000 UTC m=+5923.984715421" Dec 03 12:58:54 crc kubenswrapper[5052]: I1203 12:58:54.391981 5052 generic.go:334] "Generic (PLEG): container finished" podID="5d8a070f-f37f-42b9-a47b-cca83f8ffda6" containerID="351eb338913ef40dcaca07a18d88b7d7bee6b0144cace109e505631f705e3ed3" exitCode=0 Dec 03 12:58:54 crc kubenswrapper[5052]: I1203 12:58:54.392065 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/install-certs-edpm-multinodeset-edpm-compute-global-6dqgm" event={"ID":"5d8a070f-f37f-42b9-a47b-cca83f8ffda6","Type":"ContainerDied","Data":"351eb338913ef40dcaca07a18d88b7d7bee6b0144cace109e505631f705e3ed3"} Dec 03 12:58:55 crc kubenswrapper[5052]: I1203 12:58:55.677846 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/install-certs-edpm-multinodeset-edpm-compute-global-6dqgm" Dec 03 12:58:55 crc kubenswrapper[5052]: I1203 12:58:55.760368 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"neutron-metadata-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5d8a070f-f37f-42b9-a47b-cca83f8ffda6-neutron-metadata-combined-ca-bundle\") pod \"5d8a070f-f37f-42b9-a47b-cca83f8ffda6\" (UID: \"5d8a070f-f37f-42b9-a47b-cca83f8ffda6\") " Dec 03 12:58:55 crc kubenswrapper[5052]: I1203 12:58:55.760731 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"libvirt-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5d8a070f-f37f-42b9-a47b-cca83f8ffda6-libvirt-combined-ca-bundle\") pod \"5d8a070f-f37f-42b9-a47b-cca83f8ffda6\" (UID: \"5d8a070f-f37f-42b9-a47b-cca83f8ffda6\") " Dec 03 12:58:55 crc kubenswrapper[5052]: I1203 12:58:55.760846 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovn-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5d8a070f-f37f-42b9-a47b-cca83f8ffda6-ovn-combined-ca-bundle\") pod \"5d8a070f-f37f-42b9-a47b-cca83f8ffda6\" (UID: \"5d8a070f-f37f-42b9-a47b-cca83f8ffda6\") " Dec 03 12:58:55 crc kubenswrapper[5052]: I1203 12:58:55.760941 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/5d8a070f-f37f-42b9-a47b-cca83f8ffda6-inventory\") pod \"5d8a070f-f37f-42b9-a47b-cca83f8ffda6\" (UID: \"5d8a070f-f37f-42b9-a47b-cca83f8ffda6\") " Dec 03 12:58:55 crc kubenswrapper[5052]: I1203 12:58:55.761047 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"neutron-sriov-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5d8a070f-f37f-42b9-a47b-cca83f8ffda6-neutron-sriov-combined-ca-bundle\") pod \"5d8a070f-f37f-42b9-a47b-cca83f8ffda6\" (UID: \"5d8a070f-f37f-42b9-a47b-cca83f8ffda6\") " Dec 03 12:58:55 crc kubenswrapper[5052]: I1203 12:58:55.761127 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"neutron-ovn-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5d8a070f-f37f-42b9-a47b-cca83f8ffda6-neutron-ovn-combined-ca-bundle\") pod \"5d8a070f-f37f-42b9-a47b-cca83f8ffda6\" (UID: \"5d8a070f-f37f-42b9-a47b-cca83f8ffda6\") " Dec 03 12:58:55 crc kubenswrapper[5052]: I1203 12:58:55.761231 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"nova-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5d8a070f-f37f-42b9-a47b-cca83f8ffda6-nova-combined-ca-bundle\") pod \"5d8a070f-f37f-42b9-a47b-cca83f8ffda6\" (UID: \"5d8a070f-f37f-42b9-a47b-cca83f8ffda6\") " Dec 03 12:58:55 crc kubenswrapper[5052]: I1203 12:58:55.761477 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"neutron-dhcp-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5d8a070f-f37f-42b9-a47b-cca83f8ffda6-neutron-dhcp-combined-ca-bundle\") pod \"5d8a070f-f37f-42b9-a47b-cca83f8ffda6\" (UID: \"5d8a070f-f37f-42b9-a47b-cca83f8ffda6\") " Dec 03 12:58:55 crc kubenswrapper[5052]: I1203 12:58:55.761558 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"custom-global-service-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5d8a070f-f37f-42b9-a47b-cca83f8ffda6-custom-global-service-combined-ca-bundle\") pod \"5d8a070f-f37f-42b9-a47b-cca83f8ffda6\" (UID: \"5d8a070f-f37f-42b9-a47b-cca83f8ffda6\") " Dec 03 12:58:55 crc kubenswrapper[5052]: I1203 12:58:55.761601 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"bootstrap-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5d8a070f-f37f-42b9-a47b-cca83f8ffda6-bootstrap-combined-ca-bundle\") pod \"5d8a070f-f37f-42b9-a47b-cca83f8ffda6\" (UID: \"5d8a070f-f37f-42b9-a47b-cca83f8ffda6\") " Dec 03 12:58:55 crc kubenswrapper[5052]: I1203 12:58:55.761638 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lf4xg\" (UniqueName: \"kubernetes.io/projected/5d8a070f-f37f-42b9-a47b-cca83f8ffda6-kube-api-access-lf4xg\") pod \"5d8a070f-f37f-42b9-a47b-cca83f8ffda6\" (UID: \"5d8a070f-f37f-42b9-a47b-cca83f8ffda6\") " Dec 03 12:58:55 crc kubenswrapper[5052]: I1203 12:58:55.761682 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/5d8a070f-f37f-42b9-a47b-cca83f8ffda6-ssh-key\") pod \"5d8a070f-f37f-42b9-a47b-cca83f8ffda6\" (UID: \"5d8a070f-f37f-42b9-a47b-cca83f8ffda6\") " Dec 03 12:58:55 crc kubenswrapper[5052]: I1203 12:58:55.766787 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5d8a070f-f37f-42b9-a47b-cca83f8ffda6-neutron-ovn-combined-ca-bundle" (OuterVolumeSpecName: "neutron-ovn-combined-ca-bundle") pod "5d8a070f-f37f-42b9-a47b-cca83f8ffda6" (UID: "5d8a070f-f37f-42b9-a47b-cca83f8ffda6"). InnerVolumeSpecName "neutron-ovn-combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:58:55 crc kubenswrapper[5052]: I1203 12:58:55.766810 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5d8a070f-f37f-42b9-a47b-cca83f8ffda6-custom-global-service-combined-ca-bundle" (OuterVolumeSpecName: "custom-global-service-combined-ca-bundle") pod "5d8a070f-f37f-42b9-a47b-cca83f8ffda6" (UID: "5d8a070f-f37f-42b9-a47b-cca83f8ffda6"). InnerVolumeSpecName "custom-global-service-combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:58:55 crc kubenswrapper[5052]: I1203 12:58:55.767053 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5d8a070f-f37f-42b9-a47b-cca83f8ffda6-neutron-dhcp-combined-ca-bundle" (OuterVolumeSpecName: "neutron-dhcp-combined-ca-bundle") pod "5d8a070f-f37f-42b9-a47b-cca83f8ffda6" (UID: "5d8a070f-f37f-42b9-a47b-cca83f8ffda6"). InnerVolumeSpecName "neutron-dhcp-combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:58:55 crc kubenswrapper[5052]: I1203 12:58:55.767675 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5d8a070f-f37f-42b9-a47b-cca83f8ffda6-bootstrap-combined-ca-bundle" (OuterVolumeSpecName: "bootstrap-combined-ca-bundle") pod "5d8a070f-f37f-42b9-a47b-cca83f8ffda6" (UID: "5d8a070f-f37f-42b9-a47b-cca83f8ffda6"). InnerVolumeSpecName "bootstrap-combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:58:55 crc kubenswrapper[5052]: I1203 12:58:55.767780 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5d8a070f-f37f-42b9-a47b-cca83f8ffda6-neutron-sriov-combined-ca-bundle" (OuterVolumeSpecName: "neutron-sriov-combined-ca-bundle") pod "5d8a070f-f37f-42b9-a47b-cca83f8ffda6" (UID: "5d8a070f-f37f-42b9-a47b-cca83f8ffda6"). InnerVolumeSpecName "neutron-sriov-combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:58:55 crc kubenswrapper[5052]: I1203 12:58:55.767830 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5d8a070f-f37f-42b9-a47b-cca83f8ffda6-libvirt-combined-ca-bundle" (OuterVolumeSpecName: "libvirt-combined-ca-bundle") pod "5d8a070f-f37f-42b9-a47b-cca83f8ffda6" (UID: "5d8a070f-f37f-42b9-a47b-cca83f8ffda6"). InnerVolumeSpecName "libvirt-combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:58:55 crc kubenswrapper[5052]: I1203 12:58:55.768022 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5d8a070f-f37f-42b9-a47b-cca83f8ffda6-ovn-combined-ca-bundle" (OuterVolumeSpecName: "ovn-combined-ca-bundle") pod "5d8a070f-f37f-42b9-a47b-cca83f8ffda6" (UID: "5d8a070f-f37f-42b9-a47b-cca83f8ffda6"). InnerVolumeSpecName "ovn-combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:58:55 crc kubenswrapper[5052]: I1203 12:58:55.768110 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5d8a070f-f37f-42b9-a47b-cca83f8ffda6-nova-combined-ca-bundle" (OuterVolumeSpecName: "nova-combined-ca-bundle") pod "5d8a070f-f37f-42b9-a47b-cca83f8ffda6" (UID: "5d8a070f-f37f-42b9-a47b-cca83f8ffda6"). InnerVolumeSpecName "nova-combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:58:55 crc kubenswrapper[5052]: I1203 12:58:55.768144 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5d8a070f-f37f-42b9-a47b-cca83f8ffda6-neutron-metadata-combined-ca-bundle" (OuterVolumeSpecName: "neutron-metadata-combined-ca-bundle") pod "5d8a070f-f37f-42b9-a47b-cca83f8ffda6" (UID: "5d8a070f-f37f-42b9-a47b-cca83f8ffda6"). InnerVolumeSpecName "neutron-metadata-combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:58:55 crc kubenswrapper[5052]: I1203 12:58:55.768627 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5d8a070f-f37f-42b9-a47b-cca83f8ffda6-kube-api-access-lf4xg" (OuterVolumeSpecName: "kube-api-access-lf4xg") pod "5d8a070f-f37f-42b9-a47b-cca83f8ffda6" (UID: "5d8a070f-f37f-42b9-a47b-cca83f8ffda6"). InnerVolumeSpecName "kube-api-access-lf4xg". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:58:55 crc kubenswrapper[5052]: I1203 12:58:55.783524 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5d8a070f-f37f-42b9-a47b-cca83f8ffda6-ssh-key" (OuterVolumeSpecName: "ssh-key") pod "5d8a070f-f37f-42b9-a47b-cca83f8ffda6" (UID: "5d8a070f-f37f-42b9-a47b-cca83f8ffda6"). InnerVolumeSpecName "ssh-key". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:58:55 crc kubenswrapper[5052]: I1203 12:58:55.790281 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5d8a070f-f37f-42b9-a47b-cca83f8ffda6-inventory" (OuterVolumeSpecName: "inventory") pod "5d8a070f-f37f-42b9-a47b-cca83f8ffda6" (UID: "5d8a070f-f37f-42b9-a47b-cca83f8ffda6"). InnerVolumeSpecName "inventory". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:58:55 crc kubenswrapper[5052]: I1203 12:58:55.862965 5052 reconciler_common.go:293] "Volume detached for volume \"ovn-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5d8a070f-f37f-42b9-a47b-cca83f8ffda6-ovn-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:58:55 crc kubenswrapper[5052]: I1203 12:58:55.863003 5052 reconciler_common.go:293] "Volume detached for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/5d8a070f-f37f-42b9-a47b-cca83f8ffda6-inventory\") on node \"crc\" DevicePath \"\"" Dec 03 12:58:55 crc kubenswrapper[5052]: I1203 12:58:55.863016 5052 reconciler_common.go:293] "Volume detached for volume \"neutron-sriov-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5d8a070f-f37f-42b9-a47b-cca83f8ffda6-neutron-sriov-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:58:55 crc kubenswrapper[5052]: I1203 12:58:55.863028 5052 reconciler_common.go:293] "Volume detached for volume \"neutron-ovn-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5d8a070f-f37f-42b9-a47b-cca83f8ffda6-neutron-ovn-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:58:55 crc kubenswrapper[5052]: I1203 12:58:55.863038 5052 reconciler_common.go:293] "Volume detached for volume \"nova-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5d8a070f-f37f-42b9-a47b-cca83f8ffda6-nova-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:58:55 crc kubenswrapper[5052]: I1203 12:58:55.863047 5052 reconciler_common.go:293] "Volume detached for volume \"neutron-dhcp-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5d8a070f-f37f-42b9-a47b-cca83f8ffda6-neutron-dhcp-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:58:55 crc kubenswrapper[5052]: I1203 12:58:55.863056 5052 reconciler_common.go:293] "Volume detached for volume \"custom-global-service-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5d8a070f-f37f-42b9-a47b-cca83f8ffda6-custom-global-service-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:58:55 crc kubenswrapper[5052]: I1203 12:58:55.863065 5052 reconciler_common.go:293] "Volume detached for volume \"bootstrap-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5d8a070f-f37f-42b9-a47b-cca83f8ffda6-bootstrap-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:58:55 crc kubenswrapper[5052]: I1203 12:58:55.863077 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-lf4xg\" (UniqueName: \"kubernetes.io/projected/5d8a070f-f37f-42b9-a47b-cca83f8ffda6-kube-api-access-lf4xg\") on node \"crc\" DevicePath \"\"" Dec 03 12:58:55 crc kubenswrapper[5052]: I1203 12:58:55.863084 5052 reconciler_common.go:293] "Volume detached for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/5d8a070f-f37f-42b9-a47b-cca83f8ffda6-ssh-key\") on node \"crc\" DevicePath \"\"" Dec 03 12:58:55 crc kubenswrapper[5052]: I1203 12:58:55.863092 5052 reconciler_common.go:293] "Volume detached for volume \"neutron-metadata-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5d8a070f-f37f-42b9-a47b-cca83f8ffda6-neutron-metadata-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:58:55 crc kubenswrapper[5052]: I1203 12:58:55.863103 5052 reconciler_common.go:293] "Volume detached for volume \"libvirt-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5d8a070f-f37f-42b9-a47b-cca83f8ffda6-libvirt-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:58:56 crc kubenswrapper[5052]: I1203 12:58:56.413383 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/install-certs-edpm-multinodeset-edpm-compute-global-6dqgm" event={"ID":"5d8a070f-f37f-42b9-a47b-cca83f8ffda6","Type":"ContainerDied","Data":"01da662b7039ecb72ebf5b007aa4ac8dfe32873ae30a75470ddf8a914c572c98"} Dec 03 12:58:56 crc kubenswrapper[5052]: I1203 12:58:56.413832 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="01da662b7039ecb72ebf5b007aa4ac8dfe32873ae30a75470ddf8a914c572c98" Dec 03 12:58:56 crc kubenswrapper[5052]: I1203 12:58:56.413478 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/install-certs-edpm-multinodeset-edpm-compute-global-6dqgm" Dec 03 12:58:56 crc kubenswrapper[5052]: I1203 12:58:56.484158 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ovn-edpm-multinodeset-edpm-compute-global-m7d25"] Dec 03 12:58:56 crc kubenswrapper[5052]: E1203 12:58:56.484641 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="5d8a070f-f37f-42b9-a47b-cca83f8ffda6" containerName="install-certs-edpm-multinodeset-edpm-compute-global" Dec 03 12:58:56 crc kubenswrapper[5052]: I1203 12:58:56.484667 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="5d8a070f-f37f-42b9-a47b-cca83f8ffda6" containerName="install-certs-edpm-multinodeset-edpm-compute-global" Dec 03 12:58:56 crc kubenswrapper[5052]: I1203 12:58:56.484884 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="5d8a070f-f37f-42b9-a47b-cca83f8ffda6" containerName="install-certs-edpm-multinodeset-edpm-compute-global" Dec 03 12:58:56 crc kubenswrapper[5052]: I1203 12:58:56.485527 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovn-edpm-multinodeset-edpm-compute-global-m7d25" Dec 03 12:58:56 crc kubenswrapper[5052]: I1203 12:58:56.487433 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"combined-ca-bundle" Dec 03 12:58:56 crc kubenswrapper[5052]: I1203 12:58:56.487435 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"edpm-compute-global-dockercfg-p5rjx" Dec 03 12:58:56 crc kubenswrapper[5052]: I1203 12:58:56.488215 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"openstack-aee-default-env" Dec 03 12:58:56 crc kubenswrapper[5052]: I1203 12:58:56.488273 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"dataplanenodeset-edpm-compute-global" Dec 03 12:58:56 crc kubenswrapper[5052]: I1203 12:58:56.489080 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"ovncontroller-config" Dec 03 12:58:56 crc kubenswrapper[5052]: I1203 12:58:56.489713 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"dataplane-ansible-ssh-private-key-secret" Dec 03 12:58:56 crc kubenswrapper[5052]: I1203 12:58:56.507911 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ovn-edpm-multinodeset-edpm-compute-global-m7d25"] Dec 03 12:58:56 crc kubenswrapper[5052]: I1203 12:58:56.573750 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-79qkr\" (UniqueName: \"kubernetes.io/projected/c3c03807-79b5-49d6-b432-be6be9c109bb-kube-api-access-79qkr\") pod \"ovn-edpm-multinodeset-edpm-compute-global-m7d25\" (UID: \"c3c03807-79b5-49d6-b432-be6be9c109bb\") " pod="openstack-kuttl-tests/ovn-edpm-multinodeset-edpm-compute-global-m7d25" Dec 03 12:58:56 crc kubenswrapper[5052]: I1203 12:58:56.573807 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/c3c03807-79b5-49d6-b432-be6be9c109bb-inventory\") pod \"ovn-edpm-multinodeset-edpm-compute-global-m7d25\" (UID: \"c3c03807-79b5-49d6-b432-be6be9c109bb\") " pod="openstack-kuttl-tests/ovn-edpm-multinodeset-edpm-compute-global-m7d25" Dec 03 12:58:56 crc kubenswrapper[5052]: I1203 12:58:56.573838 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovncontroller-config-0\" (UniqueName: \"kubernetes.io/configmap/c3c03807-79b5-49d6-b432-be6be9c109bb-ovncontroller-config-0\") pod \"ovn-edpm-multinodeset-edpm-compute-global-m7d25\" (UID: \"c3c03807-79b5-49d6-b432-be6be9c109bb\") " pod="openstack-kuttl-tests/ovn-edpm-multinodeset-edpm-compute-global-m7d25" Dec 03 12:58:56 crc kubenswrapper[5052]: I1203 12:58:56.573860 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c3c03807-79b5-49d6-b432-be6be9c109bb-ovn-combined-ca-bundle\") pod \"ovn-edpm-multinodeset-edpm-compute-global-m7d25\" (UID: \"c3c03807-79b5-49d6-b432-be6be9c109bb\") " pod="openstack-kuttl-tests/ovn-edpm-multinodeset-edpm-compute-global-m7d25" Dec 03 12:58:56 crc kubenswrapper[5052]: I1203 12:58:56.573877 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/c3c03807-79b5-49d6-b432-be6be9c109bb-ssh-key\") pod \"ovn-edpm-multinodeset-edpm-compute-global-m7d25\" (UID: \"c3c03807-79b5-49d6-b432-be6be9c109bb\") " pod="openstack-kuttl-tests/ovn-edpm-multinodeset-edpm-compute-global-m7d25" Dec 03 12:58:56 crc kubenswrapper[5052]: I1203 12:58:56.673338 5052 scope.go:117] "RemoveContainer" containerID="69fa7fbcd2aceb7c53703a19508876f93a97dc9adabb79c16fb691d90f606d58" Dec 03 12:58:56 crc kubenswrapper[5052]: E1203 12:58:56.674141 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 12:58:56 crc kubenswrapper[5052]: I1203 12:58:56.674576 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovn-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c3c03807-79b5-49d6-b432-be6be9c109bb-ovn-combined-ca-bundle\") pod \"ovn-edpm-multinodeset-edpm-compute-global-m7d25\" (UID: \"c3c03807-79b5-49d6-b432-be6be9c109bb\") " pod="openstack-kuttl-tests/ovn-edpm-multinodeset-edpm-compute-global-m7d25" Dec 03 12:58:56 crc kubenswrapper[5052]: I1203 12:58:56.674616 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/c3c03807-79b5-49d6-b432-be6be9c109bb-ssh-key\") pod \"ovn-edpm-multinodeset-edpm-compute-global-m7d25\" (UID: \"c3c03807-79b5-49d6-b432-be6be9c109bb\") " pod="openstack-kuttl-tests/ovn-edpm-multinodeset-edpm-compute-global-m7d25" Dec 03 12:58:56 crc kubenswrapper[5052]: I1203 12:58:56.674700 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-79qkr\" (UniqueName: \"kubernetes.io/projected/c3c03807-79b5-49d6-b432-be6be9c109bb-kube-api-access-79qkr\") pod \"ovn-edpm-multinodeset-edpm-compute-global-m7d25\" (UID: \"c3c03807-79b5-49d6-b432-be6be9c109bb\") " pod="openstack-kuttl-tests/ovn-edpm-multinodeset-edpm-compute-global-m7d25" Dec 03 12:58:56 crc kubenswrapper[5052]: I1203 12:58:56.674731 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/c3c03807-79b5-49d6-b432-be6be9c109bb-inventory\") pod \"ovn-edpm-multinodeset-edpm-compute-global-m7d25\" (UID: \"c3c03807-79b5-49d6-b432-be6be9c109bb\") " pod="openstack-kuttl-tests/ovn-edpm-multinodeset-edpm-compute-global-m7d25" Dec 03 12:58:56 crc kubenswrapper[5052]: I1203 12:58:56.674755 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovncontroller-config-0\" (UniqueName: \"kubernetes.io/configmap/c3c03807-79b5-49d6-b432-be6be9c109bb-ovncontroller-config-0\") pod \"ovn-edpm-multinodeset-edpm-compute-global-m7d25\" (UID: \"c3c03807-79b5-49d6-b432-be6be9c109bb\") " pod="openstack-kuttl-tests/ovn-edpm-multinodeset-edpm-compute-global-m7d25" Dec 03 12:58:56 crc kubenswrapper[5052]: I1203 12:58:56.675686 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovncontroller-config-0\" (UniqueName: \"kubernetes.io/configmap/c3c03807-79b5-49d6-b432-be6be9c109bb-ovncontroller-config-0\") pod \"ovn-edpm-multinodeset-edpm-compute-global-m7d25\" (UID: \"c3c03807-79b5-49d6-b432-be6be9c109bb\") " pod="openstack-kuttl-tests/ovn-edpm-multinodeset-edpm-compute-global-m7d25" Dec 03 12:58:56 crc kubenswrapper[5052]: I1203 12:58:56.680165 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/c3c03807-79b5-49d6-b432-be6be9c109bb-inventory\") pod \"ovn-edpm-multinodeset-edpm-compute-global-m7d25\" (UID: \"c3c03807-79b5-49d6-b432-be6be9c109bb\") " pod="openstack-kuttl-tests/ovn-edpm-multinodeset-edpm-compute-global-m7d25" Dec 03 12:58:56 crc kubenswrapper[5052]: I1203 12:58:56.680991 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovn-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c3c03807-79b5-49d6-b432-be6be9c109bb-ovn-combined-ca-bundle\") pod \"ovn-edpm-multinodeset-edpm-compute-global-m7d25\" (UID: \"c3c03807-79b5-49d6-b432-be6be9c109bb\") " pod="openstack-kuttl-tests/ovn-edpm-multinodeset-edpm-compute-global-m7d25" Dec 03 12:58:56 crc kubenswrapper[5052]: I1203 12:58:56.681089 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/c3c03807-79b5-49d6-b432-be6be9c109bb-ssh-key\") pod \"ovn-edpm-multinodeset-edpm-compute-global-m7d25\" (UID: \"c3c03807-79b5-49d6-b432-be6be9c109bb\") " pod="openstack-kuttl-tests/ovn-edpm-multinodeset-edpm-compute-global-m7d25" Dec 03 12:58:56 crc kubenswrapper[5052]: I1203 12:58:56.691481 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-79qkr\" (UniqueName: \"kubernetes.io/projected/c3c03807-79b5-49d6-b432-be6be9c109bb-kube-api-access-79qkr\") pod \"ovn-edpm-multinodeset-edpm-compute-global-m7d25\" (UID: \"c3c03807-79b5-49d6-b432-be6be9c109bb\") " pod="openstack-kuttl-tests/ovn-edpm-multinodeset-edpm-compute-global-m7d25" Dec 03 12:58:56 crc kubenswrapper[5052]: I1203 12:58:56.803031 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovn-edpm-multinodeset-edpm-compute-global-m7d25" Dec 03 12:58:57 crc kubenswrapper[5052]: I1203 12:58:57.242964 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ovn-edpm-multinodeset-edpm-compute-global-m7d25"] Dec 03 12:58:57 crc kubenswrapper[5052]: I1203 12:58:57.422151 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovn-edpm-multinodeset-edpm-compute-global-m7d25" event={"ID":"c3c03807-79b5-49d6-b432-be6be9c109bb","Type":"ContainerStarted","Data":"f719c38f8a279963743d50459c61ae564fd7e99c514adebe7902fa766d017f26"} Dec 03 12:58:59 crc kubenswrapper[5052]: I1203 12:58:59.441783 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovn-edpm-multinodeset-edpm-compute-global-m7d25" event={"ID":"c3c03807-79b5-49d6-b432-be6be9c109bb","Type":"ContainerStarted","Data":"150deb6c290fbd42f50cc667850b46c58753348c6b8763611a69647692127bce"} Dec 03 12:58:59 crc kubenswrapper[5052]: I1203 12:58:59.470402 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/ovn-edpm-multinodeset-edpm-compute-global-m7d25" podStartSLOduration=1.89858643 podStartE2EDuration="3.470380271s" podCreationTimestamp="2025-12-03 12:58:56 +0000 UTC" firstStartedPulling="2025-12-03 12:58:57.24995732 +0000 UTC m=+5927.838199084" lastFinishedPulling="2025-12-03 12:58:58.821751151 +0000 UTC m=+5929.409992925" observedRunningTime="2025-12-03 12:58:59.458624895 +0000 UTC m=+5930.046866669" watchObservedRunningTime="2025-12-03 12:58:59.470380271 +0000 UTC m=+5930.058622025" Dec 03 12:59:02 crc kubenswrapper[5052]: I1203 12:59:02.470495 5052 generic.go:334] "Generic (PLEG): container finished" podID="c3c03807-79b5-49d6-b432-be6be9c109bb" containerID="150deb6c290fbd42f50cc667850b46c58753348c6b8763611a69647692127bce" exitCode=0 Dec 03 12:59:02 crc kubenswrapper[5052]: I1203 12:59:02.470595 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovn-edpm-multinodeset-edpm-compute-global-m7d25" event={"ID":"c3c03807-79b5-49d6-b432-be6be9c109bb","Type":"ContainerDied","Data":"150deb6c290fbd42f50cc667850b46c58753348c6b8763611a69647692127bce"} Dec 03 12:59:03 crc kubenswrapper[5052]: I1203 12:59:03.181769 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openshift-marketplace/redhat-operators-xfdvp"] Dec 03 12:59:03 crc kubenswrapper[5052]: I1203 12:59:03.183459 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/redhat-operators-xfdvp" Dec 03 12:59:03 crc kubenswrapper[5052]: I1203 12:59:03.257782 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/redhat-operators-xfdvp"] Dec 03 12:59:03 crc kubenswrapper[5052]: I1203 12:59:03.282811 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/222ac98a-f9c5-4510-87de-7b8a57b77d11-catalog-content\") pod \"redhat-operators-xfdvp\" (UID: \"222ac98a-f9c5-4510-87de-7b8a57b77d11\") " pod="openshift-marketplace/redhat-operators-xfdvp" Dec 03 12:59:03 crc kubenswrapper[5052]: I1203 12:59:03.282867 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/222ac98a-f9c5-4510-87de-7b8a57b77d11-utilities\") pod \"redhat-operators-xfdvp\" (UID: \"222ac98a-f9c5-4510-87de-7b8a57b77d11\") " pod="openshift-marketplace/redhat-operators-xfdvp" Dec 03 12:59:03 crc kubenswrapper[5052]: I1203 12:59:03.282914 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6mtgq\" (UniqueName: \"kubernetes.io/projected/222ac98a-f9c5-4510-87de-7b8a57b77d11-kube-api-access-6mtgq\") pod \"redhat-operators-xfdvp\" (UID: \"222ac98a-f9c5-4510-87de-7b8a57b77d11\") " pod="openshift-marketplace/redhat-operators-xfdvp" Dec 03 12:59:03 crc kubenswrapper[5052]: I1203 12:59:03.384722 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-6mtgq\" (UniqueName: \"kubernetes.io/projected/222ac98a-f9c5-4510-87de-7b8a57b77d11-kube-api-access-6mtgq\") pod \"redhat-operators-xfdvp\" (UID: \"222ac98a-f9c5-4510-87de-7b8a57b77d11\") " pod="openshift-marketplace/redhat-operators-xfdvp" Dec 03 12:59:03 crc kubenswrapper[5052]: I1203 12:59:03.384889 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/222ac98a-f9c5-4510-87de-7b8a57b77d11-catalog-content\") pod \"redhat-operators-xfdvp\" (UID: \"222ac98a-f9c5-4510-87de-7b8a57b77d11\") " pod="openshift-marketplace/redhat-operators-xfdvp" Dec 03 12:59:03 crc kubenswrapper[5052]: I1203 12:59:03.384921 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/222ac98a-f9c5-4510-87de-7b8a57b77d11-utilities\") pod \"redhat-operators-xfdvp\" (UID: \"222ac98a-f9c5-4510-87de-7b8a57b77d11\") " pod="openshift-marketplace/redhat-operators-xfdvp" Dec 03 12:59:03 crc kubenswrapper[5052]: I1203 12:59:03.385646 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/222ac98a-f9c5-4510-87de-7b8a57b77d11-utilities\") pod \"redhat-operators-xfdvp\" (UID: \"222ac98a-f9c5-4510-87de-7b8a57b77d11\") " pod="openshift-marketplace/redhat-operators-xfdvp" Dec 03 12:59:03 crc kubenswrapper[5052]: I1203 12:59:03.385969 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/222ac98a-f9c5-4510-87de-7b8a57b77d11-catalog-content\") pod \"redhat-operators-xfdvp\" (UID: \"222ac98a-f9c5-4510-87de-7b8a57b77d11\") " pod="openshift-marketplace/redhat-operators-xfdvp" Dec 03 12:59:03 crc kubenswrapper[5052]: I1203 12:59:03.419371 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-6mtgq\" (UniqueName: \"kubernetes.io/projected/222ac98a-f9c5-4510-87de-7b8a57b77d11-kube-api-access-6mtgq\") pod \"redhat-operators-xfdvp\" (UID: \"222ac98a-f9c5-4510-87de-7b8a57b77d11\") " pod="openshift-marketplace/redhat-operators-xfdvp" Dec 03 12:59:03 crc kubenswrapper[5052]: I1203 12:59:03.564463 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/redhat-operators-xfdvp" Dec 03 12:59:03 crc kubenswrapper[5052]: I1203 12:59:03.825807 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovn-edpm-multinodeset-edpm-compute-global-m7d25" Dec 03 12:59:03 crc kubenswrapper[5052]: I1203 12:59:03.892333 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-79qkr\" (UniqueName: \"kubernetes.io/projected/c3c03807-79b5-49d6-b432-be6be9c109bb-kube-api-access-79qkr\") pod \"c3c03807-79b5-49d6-b432-be6be9c109bb\" (UID: \"c3c03807-79b5-49d6-b432-be6be9c109bb\") " Dec 03 12:59:03 crc kubenswrapper[5052]: I1203 12:59:03.892441 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovn-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c3c03807-79b5-49d6-b432-be6be9c109bb-ovn-combined-ca-bundle\") pod \"c3c03807-79b5-49d6-b432-be6be9c109bb\" (UID: \"c3c03807-79b5-49d6-b432-be6be9c109bb\") " Dec 03 12:59:03 crc kubenswrapper[5052]: I1203 12:59:03.892564 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovncontroller-config-0\" (UniqueName: \"kubernetes.io/configmap/c3c03807-79b5-49d6-b432-be6be9c109bb-ovncontroller-config-0\") pod \"c3c03807-79b5-49d6-b432-be6be9c109bb\" (UID: \"c3c03807-79b5-49d6-b432-be6be9c109bb\") " Dec 03 12:59:03 crc kubenswrapper[5052]: I1203 12:59:03.892607 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/c3c03807-79b5-49d6-b432-be6be9c109bb-ssh-key\") pod \"c3c03807-79b5-49d6-b432-be6be9c109bb\" (UID: \"c3c03807-79b5-49d6-b432-be6be9c109bb\") " Dec 03 12:59:03 crc kubenswrapper[5052]: I1203 12:59:03.892657 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/c3c03807-79b5-49d6-b432-be6be9c109bb-inventory\") pod \"c3c03807-79b5-49d6-b432-be6be9c109bb\" (UID: \"c3c03807-79b5-49d6-b432-be6be9c109bb\") " Dec 03 12:59:03 crc kubenswrapper[5052]: I1203 12:59:03.893479 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/redhat-operators-xfdvp"] Dec 03 12:59:03 crc kubenswrapper[5052]: I1203 12:59:03.898908 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c3c03807-79b5-49d6-b432-be6be9c109bb-ovn-combined-ca-bundle" (OuterVolumeSpecName: "ovn-combined-ca-bundle") pod "c3c03807-79b5-49d6-b432-be6be9c109bb" (UID: "c3c03807-79b5-49d6-b432-be6be9c109bb"). InnerVolumeSpecName "ovn-combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:59:03 crc kubenswrapper[5052]: I1203 12:59:03.899346 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c3c03807-79b5-49d6-b432-be6be9c109bb-kube-api-access-79qkr" (OuterVolumeSpecName: "kube-api-access-79qkr") pod "c3c03807-79b5-49d6-b432-be6be9c109bb" (UID: "c3c03807-79b5-49d6-b432-be6be9c109bb"). InnerVolumeSpecName "kube-api-access-79qkr". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:59:03 crc kubenswrapper[5052]: I1203 12:59:03.919812 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c3c03807-79b5-49d6-b432-be6be9c109bb-inventory" (OuterVolumeSpecName: "inventory") pod "c3c03807-79b5-49d6-b432-be6be9c109bb" (UID: "c3c03807-79b5-49d6-b432-be6be9c109bb"). InnerVolumeSpecName "inventory". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:59:03 crc kubenswrapper[5052]: I1203 12:59:03.922240 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c3c03807-79b5-49d6-b432-be6be9c109bb-ovncontroller-config-0" (OuterVolumeSpecName: "ovncontroller-config-0") pod "c3c03807-79b5-49d6-b432-be6be9c109bb" (UID: "c3c03807-79b5-49d6-b432-be6be9c109bb"). InnerVolumeSpecName "ovncontroller-config-0". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 12:59:03 crc kubenswrapper[5052]: I1203 12:59:03.926068 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c3c03807-79b5-49d6-b432-be6be9c109bb-ssh-key" (OuterVolumeSpecName: "ssh-key") pod "c3c03807-79b5-49d6-b432-be6be9c109bb" (UID: "c3c03807-79b5-49d6-b432-be6be9c109bb"). InnerVolumeSpecName "ssh-key". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:59:03 crc kubenswrapper[5052]: I1203 12:59:03.994539 5052 reconciler_common.go:293] "Volume detached for volume \"ovn-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c3c03807-79b5-49d6-b432-be6be9c109bb-ovn-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:59:03 crc kubenswrapper[5052]: I1203 12:59:03.994573 5052 reconciler_common.go:293] "Volume detached for volume \"ovncontroller-config-0\" (UniqueName: \"kubernetes.io/configmap/c3c03807-79b5-49d6-b432-be6be9c109bb-ovncontroller-config-0\") on node \"crc\" DevicePath \"\"" Dec 03 12:59:03 crc kubenswrapper[5052]: I1203 12:59:03.994586 5052 reconciler_common.go:293] "Volume detached for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/c3c03807-79b5-49d6-b432-be6be9c109bb-ssh-key\") on node \"crc\" DevicePath \"\"" Dec 03 12:59:03 crc kubenswrapper[5052]: I1203 12:59:03.994622 5052 reconciler_common.go:293] "Volume detached for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/c3c03807-79b5-49d6-b432-be6be9c109bb-inventory\") on node \"crc\" DevicePath \"\"" Dec 03 12:59:03 crc kubenswrapper[5052]: I1203 12:59:03.994638 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-79qkr\" (UniqueName: \"kubernetes.io/projected/c3c03807-79b5-49d6-b432-be6be9c109bb-kube-api-access-79qkr\") on node \"crc\" DevicePath \"\"" Dec 03 12:59:04 crc kubenswrapper[5052]: I1203 12:59:04.487593 5052 generic.go:334] "Generic (PLEG): container finished" podID="222ac98a-f9c5-4510-87de-7b8a57b77d11" containerID="7a69a280ad50714a1a2ebb99042e6e8cb786ccb73ccc495e54af74389c16f831" exitCode=0 Dec 03 12:59:04 crc kubenswrapper[5052]: I1203 12:59:04.487653 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-operators-xfdvp" event={"ID":"222ac98a-f9c5-4510-87de-7b8a57b77d11","Type":"ContainerDied","Data":"7a69a280ad50714a1a2ebb99042e6e8cb786ccb73ccc495e54af74389c16f831"} Dec 03 12:59:04 crc kubenswrapper[5052]: I1203 12:59:04.487969 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-operators-xfdvp" event={"ID":"222ac98a-f9c5-4510-87de-7b8a57b77d11","Type":"ContainerStarted","Data":"35bfdc829107420cf1a250ddaa9094f1bf5373548671a52bcf77df01aac2b251"} Dec 03 12:59:04 crc kubenswrapper[5052]: I1203 12:59:04.490520 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovn-edpm-multinodeset-edpm-compute-global-m7d25" event={"ID":"c3c03807-79b5-49d6-b432-be6be9c109bb","Type":"ContainerDied","Data":"f719c38f8a279963743d50459c61ae564fd7e99c514adebe7902fa766d017f26"} Dec 03 12:59:04 crc kubenswrapper[5052]: I1203 12:59:04.490556 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f719c38f8a279963743d50459c61ae564fd7e99c514adebe7902fa766d017f26" Dec 03 12:59:04 crc kubenswrapper[5052]: I1203 12:59:04.490577 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovn-edpm-multinodeset-edpm-compute-global-m7d25" Dec 03 12:59:04 crc kubenswrapper[5052]: I1203 12:59:04.585936 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/neutron-metadata-edpm-multinodeset-edpm-compute-global-ngmds"] Dec 03 12:59:04 crc kubenswrapper[5052]: E1203 12:59:04.586322 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="c3c03807-79b5-49d6-b432-be6be9c109bb" containerName="ovn-edpm-multinodeset-edpm-compute-global" Dec 03 12:59:04 crc kubenswrapper[5052]: I1203 12:59:04.586340 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="c3c03807-79b5-49d6-b432-be6be9c109bb" containerName="ovn-edpm-multinodeset-edpm-compute-global" Dec 03 12:59:04 crc kubenswrapper[5052]: I1203 12:59:04.586531 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="c3c03807-79b5-49d6-b432-be6be9c109bb" containerName="ovn-edpm-multinodeset-edpm-compute-global" Dec 03 12:59:04 crc kubenswrapper[5052]: I1203 12:59:04.587068 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-metadata-edpm-multinodeset-edpm-compute-global-ngmds" Dec 03 12:59:04 crc kubenswrapper[5052]: I1203 12:59:04.593804 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"edpm-compute-global-dockercfg-p5rjx" Dec 03 12:59:04 crc kubenswrapper[5052]: I1203 12:59:04.593995 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"dataplanenodeset-edpm-compute-global" Dec 03 12:59:04 crc kubenswrapper[5052]: I1203 12:59:04.594135 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"dataplane-ansible-ssh-private-key-secret" Dec 03 12:59:04 crc kubenswrapper[5052]: I1203 12:59:04.594269 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"neutron-ovn-metadata-agent-neutron-config" Dec 03 12:59:04 crc kubenswrapper[5052]: I1203 12:59:04.594319 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-metadata-neutron-config" Dec 03 12:59:04 crc kubenswrapper[5052]: I1203 12:59:04.594442 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"openstack-aee-default-env" Dec 03 12:59:04 crc kubenswrapper[5052]: I1203 12:59:04.594500 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"combined-ca-bundle" Dec 03 12:59:04 crc kubenswrapper[5052]: I1203 12:59:04.601725 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-metadata-edpm-multinodeset-edpm-compute-global-ngmds"] Dec 03 12:59:04 crc kubenswrapper[5052]: I1203 12:59:04.707238 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v4gz4\" (UniqueName: \"kubernetes.io/projected/865fae8c-437a-482e-8b35-30d9384fc9ca-kube-api-access-v4gz4\") pod \"neutron-metadata-edpm-multinodeset-edpm-compute-global-ngmds\" (UID: \"865fae8c-437a-482e-8b35-30d9384fc9ca\") " pod="openstack-kuttl-tests/neutron-metadata-edpm-multinodeset-edpm-compute-global-ngmds" Dec 03 12:59:04 crc kubenswrapper[5052]: I1203 12:59:04.707323 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nova-metadata-neutron-config-0\" (UniqueName: \"kubernetes.io/secret/865fae8c-437a-482e-8b35-30d9384fc9ca-nova-metadata-neutron-config-0\") pod \"neutron-metadata-edpm-multinodeset-edpm-compute-global-ngmds\" (UID: \"865fae8c-437a-482e-8b35-30d9384fc9ca\") " pod="openstack-kuttl-tests/neutron-metadata-edpm-multinodeset-edpm-compute-global-ngmds" Dec 03 12:59:04 crc kubenswrapper[5052]: I1203 12:59:04.707429 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"neutron-ovn-metadata-agent-neutron-config-0\" (UniqueName: \"kubernetes.io/secret/865fae8c-437a-482e-8b35-30d9384fc9ca-neutron-ovn-metadata-agent-neutron-config-0\") pod \"neutron-metadata-edpm-multinodeset-edpm-compute-global-ngmds\" (UID: \"865fae8c-437a-482e-8b35-30d9384fc9ca\") " pod="openstack-kuttl-tests/neutron-metadata-edpm-multinodeset-edpm-compute-global-ngmds" Dec 03 12:59:04 crc kubenswrapper[5052]: I1203 12:59:04.707459 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/865fae8c-437a-482e-8b35-30d9384fc9ca-ssh-key\") pod \"neutron-metadata-edpm-multinodeset-edpm-compute-global-ngmds\" (UID: \"865fae8c-437a-482e-8b35-30d9384fc9ca\") " pod="openstack-kuttl-tests/neutron-metadata-edpm-multinodeset-edpm-compute-global-ngmds" Dec 03 12:59:04 crc kubenswrapper[5052]: I1203 12:59:04.707531 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nova-metadata-neutron-config-2\" (UniqueName: \"kubernetes.io/secret/865fae8c-437a-482e-8b35-30d9384fc9ca-nova-metadata-neutron-config-2\") pod \"neutron-metadata-edpm-multinodeset-edpm-compute-global-ngmds\" (UID: \"865fae8c-437a-482e-8b35-30d9384fc9ca\") " pod="openstack-kuttl-tests/neutron-metadata-edpm-multinodeset-edpm-compute-global-ngmds" Dec 03 12:59:04 crc kubenswrapper[5052]: I1203 12:59:04.707664 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nova-metadata-neutron-config-1\" (UniqueName: \"kubernetes.io/secret/865fae8c-437a-482e-8b35-30d9384fc9ca-nova-metadata-neutron-config-1\") pod \"neutron-metadata-edpm-multinodeset-edpm-compute-global-ngmds\" (UID: \"865fae8c-437a-482e-8b35-30d9384fc9ca\") " pod="openstack-kuttl-tests/neutron-metadata-edpm-multinodeset-edpm-compute-global-ngmds" Dec 03 12:59:04 crc kubenswrapper[5052]: I1203 12:59:04.707720 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"neutron-metadata-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/865fae8c-437a-482e-8b35-30d9384fc9ca-neutron-metadata-combined-ca-bundle\") pod \"neutron-metadata-edpm-multinodeset-edpm-compute-global-ngmds\" (UID: \"865fae8c-437a-482e-8b35-30d9384fc9ca\") " pod="openstack-kuttl-tests/neutron-metadata-edpm-multinodeset-edpm-compute-global-ngmds" Dec 03 12:59:04 crc kubenswrapper[5052]: I1203 12:59:04.707746 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/865fae8c-437a-482e-8b35-30d9384fc9ca-inventory\") pod \"neutron-metadata-edpm-multinodeset-edpm-compute-global-ngmds\" (UID: \"865fae8c-437a-482e-8b35-30d9384fc9ca\") " pod="openstack-kuttl-tests/neutron-metadata-edpm-multinodeset-edpm-compute-global-ngmds" Dec 03 12:59:04 crc kubenswrapper[5052]: I1203 12:59:04.808821 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-v4gz4\" (UniqueName: \"kubernetes.io/projected/865fae8c-437a-482e-8b35-30d9384fc9ca-kube-api-access-v4gz4\") pod \"neutron-metadata-edpm-multinodeset-edpm-compute-global-ngmds\" (UID: \"865fae8c-437a-482e-8b35-30d9384fc9ca\") " pod="openstack-kuttl-tests/neutron-metadata-edpm-multinodeset-edpm-compute-global-ngmds" Dec 03 12:59:04 crc kubenswrapper[5052]: I1203 12:59:04.808879 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"nova-metadata-neutron-config-0\" (UniqueName: \"kubernetes.io/secret/865fae8c-437a-482e-8b35-30d9384fc9ca-nova-metadata-neutron-config-0\") pod \"neutron-metadata-edpm-multinodeset-edpm-compute-global-ngmds\" (UID: \"865fae8c-437a-482e-8b35-30d9384fc9ca\") " pod="openstack-kuttl-tests/neutron-metadata-edpm-multinodeset-edpm-compute-global-ngmds" Dec 03 12:59:04 crc kubenswrapper[5052]: I1203 12:59:04.808919 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"neutron-ovn-metadata-agent-neutron-config-0\" (UniqueName: \"kubernetes.io/secret/865fae8c-437a-482e-8b35-30d9384fc9ca-neutron-ovn-metadata-agent-neutron-config-0\") pod \"neutron-metadata-edpm-multinodeset-edpm-compute-global-ngmds\" (UID: \"865fae8c-437a-482e-8b35-30d9384fc9ca\") " pod="openstack-kuttl-tests/neutron-metadata-edpm-multinodeset-edpm-compute-global-ngmds" Dec 03 12:59:04 crc kubenswrapper[5052]: I1203 12:59:04.808940 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/865fae8c-437a-482e-8b35-30d9384fc9ca-ssh-key\") pod \"neutron-metadata-edpm-multinodeset-edpm-compute-global-ngmds\" (UID: \"865fae8c-437a-482e-8b35-30d9384fc9ca\") " pod="openstack-kuttl-tests/neutron-metadata-edpm-multinodeset-edpm-compute-global-ngmds" Dec 03 12:59:04 crc kubenswrapper[5052]: I1203 12:59:04.808985 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"nova-metadata-neutron-config-2\" (UniqueName: \"kubernetes.io/secret/865fae8c-437a-482e-8b35-30d9384fc9ca-nova-metadata-neutron-config-2\") pod \"neutron-metadata-edpm-multinodeset-edpm-compute-global-ngmds\" (UID: \"865fae8c-437a-482e-8b35-30d9384fc9ca\") " pod="openstack-kuttl-tests/neutron-metadata-edpm-multinodeset-edpm-compute-global-ngmds" Dec 03 12:59:04 crc kubenswrapper[5052]: I1203 12:59:04.809033 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"nova-metadata-neutron-config-1\" (UniqueName: \"kubernetes.io/secret/865fae8c-437a-482e-8b35-30d9384fc9ca-nova-metadata-neutron-config-1\") pod \"neutron-metadata-edpm-multinodeset-edpm-compute-global-ngmds\" (UID: \"865fae8c-437a-482e-8b35-30d9384fc9ca\") " pod="openstack-kuttl-tests/neutron-metadata-edpm-multinodeset-edpm-compute-global-ngmds" Dec 03 12:59:04 crc kubenswrapper[5052]: I1203 12:59:04.809073 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"neutron-metadata-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/865fae8c-437a-482e-8b35-30d9384fc9ca-neutron-metadata-combined-ca-bundle\") pod \"neutron-metadata-edpm-multinodeset-edpm-compute-global-ngmds\" (UID: \"865fae8c-437a-482e-8b35-30d9384fc9ca\") " pod="openstack-kuttl-tests/neutron-metadata-edpm-multinodeset-edpm-compute-global-ngmds" Dec 03 12:59:04 crc kubenswrapper[5052]: I1203 12:59:04.809105 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/865fae8c-437a-482e-8b35-30d9384fc9ca-inventory\") pod \"neutron-metadata-edpm-multinodeset-edpm-compute-global-ngmds\" (UID: \"865fae8c-437a-482e-8b35-30d9384fc9ca\") " pod="openstack-kuttl-tests/neutron-metadata-edpm-multinodeset-edpm-compute-global-ngmds" Dec 03 12:59:04 crc kubenswrapper[5052]: I1203 12:59:04.814107 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"neutron-ovn-metadata-agent-neutron-config-0\" (UniqueName: \"kubernetes.io/secret/865fae8c-437a-482e-8b35-30d9384fc9ca-neutron-ovn-metadata-agent-neutron-config-0\") pod \"neutron-metadata-edpm-multinodeset-edpm-compute-global-ngmds\" (UID: \"865fae8c-437a-482e-8b35-30d9384fc9ca\") " pod="openstack-kuttl-tests/neutron-metadata-edpm-multinodeset-edpm-compute-global-ngmds" Dec 03 12:59:04 crc kubenswrapper[5052]: I1203 12:59:04.814124 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/865fae8c-437a-482e-8b35-30d9384fc9ca-ssh-key\") pod \"neutron-metadata-edpm-multinodeset-edpm-compute-global-ngmds\" (UID: \"865fae8c-437a-482e-8b35-30d9384fc9ca\") " pod="openstack-kuttl-tests/neutron-metadata-edpm-multinodeset-edpm-compute-global-ngmds" Dec 03 12:59:04 crc kubenswrapper[5052]: I1203 12:59:04.814408 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"nova-metadata-neutron-config-1\" (UniqueName: \"kubernetes.io/secret/865fae8c-437a-482e-8b35-30d9384fc9ca-nova-metadata-neutron-config-1\") pod \"neutron-metadata-edpm-multinodeset-edpm-compute-global-ngmds\" (UID: \"865fae8c-437a-482e-8b35-30d9384fc9ca\") " pod="openstack-kuttl-tests/neutron-metadata-edpm-multinodeset-edpm-compute-global-ngmds" Dec 03 12:59:04 crc kubenswrapper[5052]: I1203 12:59:04.815118 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"nova-metadata-neutron-config-0\" (UniqueName: \"kubernetes.io/secret/865fae8c-437a-482e-8b35-30d9384fc9ca-nova-metadata-neutron-config-0\") pod \"neutron-metadata-edpm-multinodeset-edpm-compute-global-ngmds\" (UID: \"865fae8c-437a-482e-8b35-30d9384fc9ca\") " pod="openstack-kuttl-tests/neutron-metadata-edpm-multinodeset-edpm-compute-global-ngmds" Dec 03 12:59:04 crc kubenswrapper[5052]: I1203 12:59:04.815528 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"neutron-metadata-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/865fae8c-437a-482e-8b35-30d9384fc9ca-neutron-metadata-combined-ca-bundle\") pod \"neutron-metadata-edpm-multinodeset-edpm-compute-global-ngmds\" (UID: \"865fae8c-437a-482e-8b35-30d9384fc9ca\") " pod="openstack-kuttl-tests/neutron-metadata-edpm-multinodeset-edpm-compute-global-ngmds" Dec 03 12:59:04 crc kubenswrapper[5052]: I1203 12:59:04.815581 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/865fae8c-437a-482e-8b35-30d9384fc9ca-inventory\") pod \"neutron-metadata-edpm-multinodeset-edpm-compute-global-ngmds\" (UID: \"865fae8c-437a-482e-8b35-30d9384fc9ca\") " pod="openstack-kuttl-tests/neutron-metadata-edpm-multinodeset-edpm-compute-global-ngmds" Dec 03 12:59:04 crc kubenswrapper[5052]: I1203 12:59:04.818948 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"nova-metadata-neutron-config-2\" (UniqueName: \"kubernetes.io/secret/865fae8c-437a-482e-8b35-30d9384fc9ca-nova-metadata-neutron-config-2\") pod \"neutron-metadata-edpm-multinodeset-edpm-compute-global-ngmds\" (UID: \"865fae8c-437a-482e-8b35-30d9384fc9ca\") " pod="openstack-kuttl-tests/neutron-metadata-edpm-multinodeset-edpm-compute-global-ngmds" Dec 03 12:59:04 crc kubenswrapper[5052]: I1203 12:59:04.833200 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-v4gz4\" (UniqueName: \"kubernetes.io/projected/865fae8c-437a-482e-8b35-30d9384fc9ca-kube-api-access-v4gz4\") pod \"neutron-metadata-edpm-multinodeset-edpm-compute-global-ngmds\" (UID: \"865fae8c-437a-482e-8b35-30d9384fc9ca\") " pod="openstack-kuttl-tests/neutron-metadata-edpm-multinodeset-edpm-compute-global-ngmds" Dec 03 12:59:04 crc kubenswrapper[5052]: I1203 12:59:04.916915 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-metadata-edpm-multinodeset-edpm-compute-global-ngmds" Dec 03 12:59:05 crc kubenswrapper[5052]: W1203 12:59:05.145071 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod865fae8c_437a_482e_8b35_30d9384fc9ca.slice/crio-0276ef74c5f3325ad5d464377bc8ab14aad4db9ee6c837c7196e132f1a12ad86 WatchSource:0}: Error finding container 0276ef74c5f3325ad5d464377bc8ab14aad4db9ee6c837c7196e132f1a12ad86: Status 404 returned error can't find the container with id 0276ef74c5f3325ad5d464377bc8ab14aad4db9ee6c837c7196e132f1a12ad86 Dec 03 12:59:05 crc kubenswrapper[5052]: I1203 12:59:05.145383 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-metadata-edpm-multinodeset-edpm-compute-global-ngmds"] Dec 03 12:59:05 crc kubenswrapper[5052]: I1203 12:59:05.501323 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-operators-xfdvp" event={"ID":"222ac98a-f9c5-4510-87de-7b8a57b77d11","Type":"ContainerStarted","Data":"9b3c6c3aa68c86d27aedbad91dcbf96659965b9c49b83b7378f4a79275581e45"} Dec 03 12:59:05 crc kubenswrapper[5052]: I1203 12:59:05.502471 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-metadata-edpm-multinodeset-edpm-compute-global-ngmds" event={"ID":"865fae8c-437a-482e-8b35-30d9384fc9ca","Type":"ContainerStarted","Data":"0276ef74c5f3325ad5d464377bc8ab14aad4db9ee6c837c7196e132f1a12ad86"} Dec 03 12:59:06 crc kubenswrapper[5052]: I1203 12:59:06.514453 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-metadata-edpm-multinodeset-edpm-compute-global-ngmds" event={"ID":"865fae8c-437a-482e-8b35-30d9384fc9ca","Type":"ContainerStarted","Data":"6f57ee592c6c48f09e795dcd724baf8ca6f694926935f16a3ebe7fe88f7a2520"} Dec 03 12:59:06 crc kubenswrapper[5052]: I1203 12:59:06.540067 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/neutron-metadata-edpm-multinodeset-edpm-compute-global-ngmds" podStartSLOduration=1.702093177 podStartE2EDuration="2.540043621s" podCreationTimestamp="2025-12-03 12:59:04 +0000 UTC" firstStartedPulling="2025-12-03 12:59:05.14728707 +0000 UTC m=+5935.735528834" lastFinishedPulling="2025-12-03 12:59:05.985237514 +0000 UTC m=+5936.573479278" observedRunningTime="2025-12-03 12:59:06.533853554 +0000 UTC m=+5937.122095338" watchObservedRunningTime="2025-12-03 12:59:06.540043621 +0000 UTC m=+5937.128285385" Dec 03 12:59:07 crc kubenswrapper[5052]: I1203 12:59:07.525096 5052 generic.go:334] "Generic (PLEG): container finished" podID="222ac98a-f9c5-4510-87de-7b8a57b77d11" containerID="9b3c6c3aa68c86d27aedbad91dcbf96659965b9c49b83b7378f4a79275581e45" exitCode=0 Dec 03 12:59:07 crc kubenswrapper[5052]: I1203 12:59:07.525138 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-operators-xfdvp" event={"ID":"222ac98a-f9c5-4510-87de-7b8a57b77d11","Type":"ContainerDied","Data":"9b3c6c3aa68c86d27aedbad91dcbf96659965b9c49b83b7378f4a79275581e45"} Dec 03 12:59:08 crc kubenswrapper[5052]: I1203 12:59:08.541849 5052 generic.go:334] "Generic (PLEG): container finished" podID="865fae8c-437a-482e-8b35-30d9384fc9ca" containerID="6f57ee592c6c48f09e795dcd724baf8ca6f694926935f16a3ebe7fe88f7a2520" exitCode=0 Dec 03 12:59:08 crc kubenswrapper[5052]: I1203 12:59:08.541971 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-metadata-edpm-multinodeset-edpm-compute-global-ngmds" event={"ID":"865fae8c-437a-482e-8b35-30d9384fc9ca","Type":"ContainerDied","Data":"6f57ee592c6c48f09e795dcd724baf8ca6f694926935f16a3ebe7fe88f7a2520"} Dec 03 12:59:09 crc kubenswrapper[5052]: I1203 12:59:09.554707 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-operators-xfdvp" event={"ID":"222ac98a-f9c5-4510-87de-7b8a57b77d11","Type":"ContainerStarted","Data":"0ab687b343e82464a46088560c98c269f61c5a55c5117f6d88c4a5cac4195e09"} Dec 03 12:59:09 crc kubenswrapper[5052]: I1203 12:59:09.575041 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-marketplace/redhat-operators-xfdvp" podStartSLOduration=2.650515641 podStartE2EDuration="6.575023456s" podCreationTimestamp="2025-12-03 12:59:03 +0000 UTC" firstStartedPulling="2025-12-03 12:59:04.489534729 +0000 UTC m=+5935.077776493" lastFinishedPulling="2025-12-03 12:59:08.414042544 +0000 UTC m=+5939.002284308" observedRunningTime="2025-12-03 12:59:09.574695707 +0000 UTC m=+5940.162937501" watchObservedRunningTime="2025-12-03 12:59:09.575023456 +0000 UTC m=+5940.163265220" Dec 03 12:59:09 crc kubenswrapper[5052]: I1203 12:59:09.905994 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-metadata-edpm-multinodeset-edpm-compute-global-ngmds" Dec 03 12:59:09 crc kubenswrapper[5052]: I1203 12:59:09.989226 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/865fae8c-437a-482e-8b35-30d9384fc9ca-inventory\") pod \"865fae8c-437a-482e-8b35-30d9384fc9ca\" (UID: \"865fae8c-437a-482e-8b35-30d9384fc9ca\") " Dec 03 12:59:09 crc kubenswrapper[5052]: I1203 12:59:09.989345 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"neutron-ovn-metadata-agent-neutron-config-0\" (UniqueName: \"kubernetes.io/secret/865fae8c-437a-482e-8b35-30d9384fc9ca-neutron-ovn-metadata-agent-neutron-config-0\") pod \"865fae8c-437a-482e-8b35-30d9384fc9ca\" (UID: \"865fae8c-437a-482e-8b35-30d9384fc9ca\") " Dec 03 12:59:09 crc kubenswrapper[5052]: I1203 12:59:09.989384 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"neutron-metadata-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/865fae8c-437a-482e-8b35-30d9384fc9ca-neutron-metadata-combined-ca-bundle\") pod \"865fae8c-437a-482e-8b35-30d9384fc9ca\" (UID: \"865fae8c-437a-482e-8b35-30d9384fc9ca\") " Dec 03 12:59:09 crc kubenswrapper[5052]: I1203 12:59:09.989424 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v4gz4\" (UniqueName: \"kubernetes.io/projected/865fae8c-437a-482e-8b35-30d9384fc9ca-kube-api-access-v4gz4\") pod \"865fae8c-437a-482e-8b35-30d9384fc9ca\" (UID: \"865fae8c-437a-482e-8b35-30d9384fc9ca\") " Dec 03 12:59:09 crc kubenswrapper[5052]: I1203 12:59:09.989450 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"nova-metadata-neutron-config-0\" (UniqueName: \"kubernetes.io/secret/865fae8c-437a-482e-8b35-30d9384fc9ca-nova-metadata-neutron-config-0\") pod \"865fae8c-437a-482e-8b35-30d9384fc9ca\" (UID: \"865fae8c-437a-482e-8b35-30d9384fc9ca\") " Dec 03 12:59:09 crc kubenswrapper[5052]: I1203 12:59:09.989480 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"nova-metadata-neutron-config-2\" (UniqueName: \"kubernetes.io/secret/865fae8c-437a-482e-8b35-30d9384fc9ca-nova-metadata-neutron-config-2\") pod \"865fae8c-437a-482e-8b35-30d9384fc9ca\" (UID: \"865fae8c-437a-482e-8b35-30d9384fc9ca\") " Dec 03 12:59:09 crc kubenswrapper[5052]: I1203 12:59:09.989591 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/865fae8c-437a-482e-8b35-30d9384fc9ca-ssh-key\") pod \"865fae8c-437a-482e-8b35-30d9384fc9ca\" (UID: \"865fae8c-437a-482e-8b35-30d9384fc9ca\") " Dec 03 12:59:09 crc kubenswrapper[5052]: I1203 12:59:09.989619 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"nova-metadata-neutron-config-1\" (UniqueName: \"kubernetes.io/secret/865fae8c-437a-482e-8b35-30d9384fc9ca-nova-metadata-neutron-config-1\") pod \"865fae8c-437a-482e-8b35-30d9384fc9ca\" (UID: \"865fae8c-437a-482e-8b35-30d9384fc9ca\") " Dec 03 12:59:10 crc kubenswrapper[5052]: I1203 12:59:10.010043 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/865fae8c-437a-482e-8b35-30d9384fc9ca-kube-api-access-v4gz4" (OuterVolumeSpecName: "kube-api-access-v4gz4") pod "865fae8c-437a-482e-8b35-30d9384fc9ca" (UID: "865fae8c-437a-482e-8b35-30d9384fc9ca"). InnerVolumeSpecName "kube-api-access-v4gz4". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:59:10 crc kubenswrapper[5052]: I1203 12:59:10.011323 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/865fae8c-437a-482e-8b35-30d9384fc9ca-neutron-metadata-combined-ca-bundle" (OuterVolumeSpecName: "neutron-metadata-combined-ca-bundle") pod "865fae8c-437a-482e-8b35-30d9384fc9ca" (UID: "865fae8c-437a-482e-8b35-30d9384fc9ca"). InnerVolumeSpecName "neutron-metadata-combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:59:10 crc kubenswrapper[5052]: I1203 12:59:10.016623 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/865fae8c-437a-482e-8b35-30d9384fc9ca-neutron-ovn-metadata-agent-neutron-config-0" (OuterVolumeSpecName: "neutron-ovn-metadata-agent-neutron-config-0") pod "865fae8c-437a-482e-8b35-30d9384fc9ca" (UID: "865fae8c-437a-482e-8b35-30d9384fc9ca"). InnerVolumeSpecName "neutron-ovn-metadata-agent-neutron-config-0". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:59:10 crc kubenswrapper[5052]: I1203 12:59:10.016882 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/865fae8c-437a-482e-8b35-30d9384fc9ca-nova-metadata-neutron-config-2" (OuterVolumeSpecName: "nova-metadata-neutron-config-2") pod "865fae8c-437a-482e-8b35-30d9384fc9ca" (UID: "865fae8c-437a-482e-8b35-30d9384fc9ca"). InnerVolumeSpecName "nova-metadata-neutron-config-2". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:59:10 crc kubenswrapper[5052]: I1203 12:59:10.017022 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/865fae8c-437a-482e-8b35-30d9384fc9ca-inventory" (OuterVolumeSpecName: "inventory") pod "865fae8c-437a-482e-8b35-30d9384fc9ca" (UID: "865fae8c-437a-482e-8b35-30d9384fc9ca"). InnerVolumeSpecName "inventory". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:59:10 crc kubenswrapper[5052]: I1203 12:59:10.020699 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/865fae8c-437a-482e-8b35-30d9384fc9ca-nova-metadata-neutron-config-1" (OuterVolumeSpecName: "nova-metadata-neutron-config-1") pod "865fae8c-437a-482e-8b35-30d9384fc9ca" (UID: "865fae8c-437a-482e-8b35-30d9384fc9ca"). InnerVolumeSpecName "nova-metadata-neutron-config-1". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:59:10 crc kubenswrapper[5052]: I1203 12:59:10.029092 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/865fae8c-437a-482e-8b35-30d9384fc9ca-nova-metadata-neutron-config-0" (OuterVolumeSpecName: "nova-metadata-neutron-config-0") pod "865fae8c-437a-482e-8b35-30d9384fc9ca" (UID: "865fae8c-437a-482e-8b35-30d9384fc9ca"). InnerVolumeSpecName "nova-metadata-neutron-config-0". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:59:10 crc kubenswrapper[5052]: I1203 12:59:10.056792 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/865fae8c-437a-482e-8b35-30d9384fc9ca-ssh-key" (OuterVolumeSpecName: "ssh-key") pod "865fae8c-437a-482e-8b35-30d9384fc9ca" (UID: "865fae8c-437a-482e-8b35-30d9384fc9ca"). InnerVolumeSpecName "ssh-key". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:59:10 crc kubenswrapper[5052]: I1203 12:59:10.091657 5052 reconciler_common.go:293] "Volume detached for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/865fae8c-437a-482e-8b35-30d9384fc9ca-inventory\") on node \"crc\" DevicePath \"\"" Dec 03 12:59:10 crc kubenswrapper[5052]: I1203 12:59:10.091694 5052 reconciler_common.go:293] "Volume detached for volume \"neutron-ovn-metadata-agent-neutron-config-0\" (UniqueName: \"kubernetes.io/secret/865fae8c-437a-482e-8b35-30d9384fc9ca-neutron-ovn-metadata-agent-neutron-config-0\") on node \"crc\" DevicePath \"\"" Dec 03 12:59:10 crc kubenswrapper[5052]: I1203 12:59:10.091712 5052 reconciler_common.go:293] "Volume detached for volume \"neutron-metadata-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/865fae8c-437a-482e-8b35-30d9384fc9ca-neutron-metadata-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:59:10 crc kubenswrapper[5052]: I1203 12:59:10.091729 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-v4gz4\" (UniqueName: \"kubernetes.io/projected/865fae8c-437a-482e-8b35-30d9384fc9ca-kube-api-access-v4gz4\") on node \"crc\" DevicePath \"\"" Dec 03 12:59:10 crc kubenswrapper[5052]: I1203 12:59:10.091742 5052 reconciler_common.go:293] "Volume detached for volume \"nova-metadata-neutron-config-0\" (UniqueName: \"kubernetes.io/secret/865fae8c-437a-482e-8b35-30d9384fc9ca-nova-metadata-neutron-config-0\") on node \"crc\" DevicePath \"\"" Dec 03 12:59:10 crc kubenswrapper[5052]: I1203 12:59:10.091757 5052 reconciler_common.go:293] "Volume detached for volume \"nova-metadata-neutron-config-2\" (UniqueName: \"kubernetes.io/secret/865fae8c-437a-482e-8b35-30d9384fc9ca-nova-metadata-neutron-config-2\") on node \"crc\" DevicePath \"\"" Dec 03 12:59:10 crc kubenswrapper[5052]: I1203 12:59:10.091768 5052 reconciler_common.go:293] "Volume detached for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/865fae8c-437a-482e-8b35-30d9384fc9ca-ssh-key\") on node \"crc\" DevicePath \"\"" Dec 03 12:59:10 crc kubenswrapper[5052]: I1203 12:59:10.091780 5052 reconciler_common.go:293] "Volume detached for volume \"nova-metadata-neutron-config-1\" (UniqueName: \"kubernetes.io/secret/865fae8c-437a-482e-8b35-30d9384fc9ca-nova-metadata-neutron-config-1\") on node \"crc\" DevicePath \"\"" Dec 03 12:59:10 crc kubenswrapper[5052]: I1203 12:59:10.566636 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-metadata-edpm-multinodeset-edpm-compute-global-ngmds" event={"ID":"865fae8c-437a-482e-8b35-30d9384fc9ca","Type":"ContainerDied","Data":"0276ef74c5f3325ad5d464377bc8ab14aad4db9ee6c837c7196e132f1a12ad86"} Dec 03 12:59:10 crc kubenswrapper[5052]: I1203 12:59:10.566685 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0276ef74c5f3325ad5d464377bc8ab14aad4db9ee6c837c7196e132f1a12ad86" Dec 03 12:59:10 crc kubenswrapper[5052]: I1203 12:59:10.566842 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-metadata-edpm-multinodeset-edpm-compute-global-ngmds" Dec 03 12:59:10 crc kubenswrapper[5052]: I1203 12:59:10.638850 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/neutron-ovn-edpm-multinodeset-edpm-compute-global-rkfgm"] Dec 03 12:59:10 crc kubenswrapper[5052]: E1203 12:59:10.639533 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="865fae8c-437a-482e-8b35-30d9384fc9ca" containerName="neutron-metadata-edpm-multinodeset-edpm-compute-global" Dec 03 12:59:10 crc kubenswrapper[5052]: I1203 12:59:10.639553 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="865fae8c-437a-482e-8b35-30d9384fc9ca" containerName="neutron-metadata-edpm-multinodeset-edpm-compute-global" Dec 03 12:59:10 crc kubenswrapper[5052]: I1203 12:59:10.639739 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="865fae8c-437a-482e-8b35-30d9384fc9ca" containerName="neutron-metadata-edpm-multinodeset-edpm-compute-global" Dec 03 12:59:10 crc kubenswrapper[5052]: I1203 12:59:10.640410 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-ovn-edpm-multinodeset-edpm-compute-global-rkfgm" Dec 03 12:59:10 crc kubenswrapper[5052]: I1203 12:59:10.642433 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"neutron-ovn-agent-neutron-config" Dec 03 12:59:10 crc kubenswrapper[5052]: I1203 12:59:10.642584 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"dataplanenodeset-edpm-compute-global" Dec 03 12:59:10 crc kubenswrapper[5052]: I1203 12:59:10.643330 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"openstack-aee-default-env" Dec 03 12:59:10 crc kubenswrapper[5052]: I1203 12:59:10.643650 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"combined-ca-bundle" Dec 03 12:59:10 crc kubenswrapper[5052]: I1203 12:59:10.643793 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"edpm-compute-global-dockercfg-p5rjx" Dec 03 12:59:10 crc kubenswrapper[5052]: I1203 12:59:10.643949 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"dataplane-ansible-ssh-private-key-secret" Dec 03 12:59:10 crc kubenswrapper[5052]: I1203 12:59:10.653264 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-ovn-edpm-multinodeset-edpm-compute-global-rkfgm"] Dec 03 12:59:10 crc kubenswrapper[5052]: I1203 12:59:10.699693 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/f649f4e5-2cc1-49bd-b4f3-b6c1e8339999-inventory\") pod \"neutron-ovn-edpm-multinodeset-edpm-compute-global-rkfgm\" (UID: \"f649f4e5-2cc1-49bd-b4f3-b6c1e8339999\") " pod="openstack-kuttl-tests/neutron-ovn-edpm-multinodeset-edpm-compute-global-rkfgm" Dec 03 12:59:10 crc kubenswrapper[5052]: I1203 12:59:10.699789 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"neutron-ovn-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f649f4e5-2cc1-49bd-b4f3-b6c1e8339999-neutron-ovn-combined-ca-bundle\") pod \"neutron-ovn-edpm-multinodeset-edpm-compute-global-rkfgm\" (UID: \"f649f4e5-2cc1-49bd-b4f3-b6c1e8339999\") " pod="openstack-kuttl-tests/neutron-ovn-edpm-multinodeset-edpm-compute-global-rkfgm" Dec 03 12:59:10 crc kubenswrapper[5052]: I1203 12:59:10.699832 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"neutron-ovn-agent-neutron-config-0\" (UniqueName: \"kubernetes.io/secret/f649f4e5-2cc1-49bd-b4f3-b6c1e8339999-neutron-ovn-agent-neutron-config-0\") pod \"neutron-ovn-edpm-multinodeset-edpm-compute-global-rkfgm\" (UID: \"f649f4e5-2cc1-49bd-b4f3-b6c1e8339999\") " pod="openstack-kuttl-tests/neutron-ovn-edpm-multinodeset-edpm-compute-global-rkfgm" Dec 03 12:59:10 crc kubenswrapper[5052]: I1203 12:59:10.699866 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gqm7c\" (UniqueName: \"kubernetes.io/projected/f649f4e5-2cc1-49bd-b4f3-b6c1e8339999-kube-api-access-gqm7c\") pod \"neutron-ovn-edpm-multinodeset-edpm-compute-global-rkfgm\" (UID: \"f649f4e5-2cc1-49bd-b4f3-b6c1e8339999\") " pod="openstack-kuttl-tests/neutron-ovn-edpm-multinodeset-edpm-compute-global-rkfgm" Dec 03 12:59:10 crc kubenswrapper[5052]: I1203 12:59:10.699887 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/f649f4e5-2cc1-49bd-b4f3-b6c1e8339999-ssh-key\") pod \"neutron-ovn-edpm-multinodeset-edpm-compute-global-rkfgm\" (UID: \"f649f4e5-2cc1-49bd-b4f3-b6c1e8339999\") " pod="openstack-kuttl-tests/neutron-ovn-edpm-multinodeset-edpm-compute-global-rkfgm" Dec 03 12:59:10 crc kubenswrapper[5052]: I1203 12:59:10.801057 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-gqm7c\" (UniqueName: \"kubernetes.io/projected/f649f4e5-2cc1-49bd-b4f3-b6c1e8339999-kube-api-access-gqm7c\") pod \"neutron-ovn-edpm-multinodeset-edpm-compute-global-rkfgm\" (UID: \"f649f4e5-2cc1-49bd-b4f3-b6c1e8339999\") " pod="openstack-kuttl-tests/neutron-ovn-edpm-multinodeset-edpm-compute-global-rkfgm" Dec 03 12:59:10 crc kubenswrapper[5052]: I1203 12:59:10.801355 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/f649f4e5-2cc1-49bd-b4f3-b6c1e8339999-ssh-key\") pod \"neutron-ovn-edpm-multinodeset-edpm-compute-global-rkfgm\" (UID: \"f649f4e5-2cc1-49bd-b4f3-b6c1e8339999\") " pod="openstack-kuttl-tests/neutron-ovn-edpm-multinodeset-edpm-compute-global-rkfgm" Dec 03 12:59:10 crc kubenswrapper[5052]: I1203 12:59:10.801537 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/f649f4e5-2cc1-49bd-b4f3-b6c1e8339999-inventory\") pod \"neutron-ovn-edpm-multinodeset-edpm-compute-global-rkfgm\" (UID: \"f649f4e5-2cc1-49bd-b4f3-b6c1e8339999\") " pod="openstack-kuttl-tests/neutron-ovn-edpm-multinodeset-edpm-compute-global-rkfgm" Dec 03 12:59:10 crc kubenswrapper[5052]: I1203 12:59:10.801855 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"neutron-ovn-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f649f4e5-2cc1-49bd-b4f3-b6c1e8339999-neutron-ovn-combined-ca-bundle\") pod \"neutron-ovn-edpm-multinodeset-edpm-compute-global-rkfgm\" (UID: \"f649f4e5-2cc1-49bd-b4f3-b6c1e8339999\") " pod="openstack-kuttl-tests/neutron-ovn-edpm-multinodeset-edpm-compute-global-rkfgm" Dec 03 12:59:10 crc kubenswrapper[5052]: I1203 12:59:10.802000 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"neutron-ovn-agent-neutron-config-0\" (UniqueName: \"kubernetes.io/secret/f649f4e5-2cc1-49bd-b4f3-b6c1e8339999-neutron-ovn-agent-neutron-config-0\") pod \"neutron-ovn-edpm-multinodeset-edpm-compute-global-rkfgm\" (UID: \"f649f4e5-2cc1-49bd-b4f3-b6c1e8339999\") " pod="openstack-kuttl-tests/neutron-ovn-edpm-multinodeset-edpm-compute-global-rkfgm" Dec 03 12:59:10 crc kubenswrapper[5052]: I1203 12:59:10.807245 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/f649f4e5-2cc1-49bd-b4f3-b6c1e8339999-inventory\") pod \"neutron-ovn-edpm-multinodeset-edpm-compute-global-rkfgm\" (UID: \"f649f4e5-2cc1-49bd-b4f3-b6c1e8339999\") " pod="openstack-kuttl-tests/neutron-ovn-edpm-multinodeset-edpm-compute-global-rkfgm" Dec 03 12:59:10 crc kubenswrapper[5052]: I1203 12:59:10.807347 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/f649f4e5-2cc1-49bd-b4f3-b6c1e8339999-ssh-key\") pod \"neutron-ovn-edpm-multinodeset-edpm-compute-global-rkfgm\" (UID: \"f649f4e5-2cc1-49bd-b4f3-b6c1e8339999\") " pod="openstack-kuttl-tests/neutron-ovn-edpm-multinodeset-edpm-compute-global-rkfgm" Dec 03 12:59:10 crc kubenswrapper[5052]: I1203 12:59:10.807959 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"neutron-ovn-agent-neutron-config-0\" (UniqueName: \"kubernetes.io/secret/f649f4e5-2cc1-49bd-b4f3-b6c1e8339999-neutron-ovn-agent-neutron-config-0\") pod \"neutron-ovn-edpm-multinodeset-edpm-compute-global-rkfgm\" (UID: \"f649f4e5-2cc1-49bd-b4f3-b6c1e8339999\") " pod="openstack-kuttl-tests/neutron-ovn-edpm-multinodeset-edpm-compute-global-rkfgm" Dec 03 12:59:10 crc kubenswrapper[5052]: I1203 12:59:10.808623 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"neutron-ovn-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f649f4e5-2cc1-49bd-b4f3-b6c1e8339999-neutron-ovn-combined-ca-bundle\") pod \"neutron-ovn-edpm-multinodeset-edpm-compute-global-rkfgm\" (UID: \"f649f4e5-2cc1-49bd-b4f3-b6c1e8339999\") " pod="openstack-kuttl-tests/neutron-ovn-edpm-multinodeset-edpm-compute-global-rkfgm" Dec 03 12:59:10 crc kubenswrapper[5052]: I1203 12:59:10.820798 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-gqm7c\" (UniqueName: \"kubernetes.io/projected/f649f4e5-2cc1-49bd-b4f3-b6c1e8339999-kube-api-access-gqm7c\") pod \"neutron-ovn-edpm-multinodeset-edpm-compute-global-rkfgm\" (UID: \"f649f4e5-2cc1-49bd-b4f3-b6c1e8339999\") " pod="openstack-kuttl-tests/neutron-ovn-edpm-multinodeset-edpm-compute-global-rkfgm" Dec 03 12:59:10 crc kubenswrapper[5052]: I1203 12:59:10.962102 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-ovn-edpm-multinodeset-edpm-compute-global-rkfgm" Dec 03 12:59:11 crc kubenswrapper[5052]: I1203 12:59:11.214939 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-ovn-edpm-multinodeset-edpm-compute-global-rkfgm"] Dec 03 12:59:11 crc kubenswrapper[5052]: I1203 12:59:11.575409 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-ovn-edpm-multinodeset-edpm-compute-global-rkfgm" event={"ID":"f649f4e5-2cc1-49bd-b4f3-b6c1e8339999","Type":"ContainerStarted","Data":"f7b6b9aec81b34e5142a49e2f320728bd4d3f10d8d15149b3c15f2f6392c9431"} Dec 03 12:59:11 crc kubenswrapper[5052]: I1203 12:59:11.673194 5052 scope.go:117] "RemoveContainer" containerID="69fa7fbcd2aceb7c53703a19508876f93a97dc9adabb79c16fb691d90f606d58" Dec 03 12:59:11 crc kubenswrapper[5052]: E1203 12:59:11.673481 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 12:59:13 crc kubenswrapper[5052]: I1203 12:59:13.566102 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-marketplace/redhat-operators-xfdvp" Dec 03 12:59:13 crc kubenswrapper[5052]: I1203 12:59:13.566494 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openshift-marketplace/redhat-operators-xfdvp" Dec 03 12:59:13 crc kubenswrapper[5052]: I1203 12:59:13.596628 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-ovn-edpm-multinodeset-edpm-compute-global-rkfgm" event={"ID":"f649f4e5-2cc1-49bd-b4f3-b6c1e8339999","Type":"ContainerStarted","Data":"d86a514852064d3f9a1460f1d0c69f1d65af69f5fef6f94d0a52686d3a5d8055"} Dec 03 12:59:13 crc kubenswrapper[5052]: I1203 12:59:13.614888 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/neutron-ovn-edpm-multinodeset-edpm-compute-global-rkfgm" podStartSLOduration=2.393751306 podStartE2EDuration="3.614870378s" podCreationTimestamp="2025-12-03 12:59:10 +0000 UTC" firstStartedPulling="2025-12-03 12:59:11.232857076 +0000 UTC m=+5941.821098840" lastFinishedPulling="2025-12-03 12:59:12.453976148 +0000 UTC m=+5943.042217912" observedRunningTime="2025-12-03 12:59:13.613309634 +0000 UTC m=+5944.201551408" watchObservedRunningTime="2025-12-03 12:59:13.614870378 +0000 UTC m=+5944.203112142" Dec 03 12:59:14 crc kubenswrapper[5052]: E1203 12:59:14.158426 5052 cadvisor_stats_provider.go:516] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf649f4e5_2cc1_49bd_b4f3_b6c1e8339999.slice/crio-conmon-d86a514852064d3f9a1460f1d0c69f1d65af69f5fef6f94d0a52686d3a5d8055.scope\": RecentStats: unable to find data in memory cache]" Dec 03 12:59:14 crc kubenswrapper[5052]: I1203 12:59:14.606438 5052 generic.go:334] "Generic (PLEG): container finished" podID="f649f4e5-2cc1-49bd-b4f3-b6c1e8339999" containerID="d86a514852064d3f9a1460f1d0c69f1d65af69f5fef6f94d0a52686d3a5d8055" exitCode=0 Dec 03 12:59:14 crc kubenswrapper[5052]: I1203 12:59:14.606499 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-ovn-edpm-multinodeset-edpm-compute-global-rkfgm" event={"ID":"f649f4e5-2cc1-49bd-b4f3-b6c1e8339999","Type":"ContainerDied","Data":"d86a514852064d3f9a1460f1d0c69f1d65af69f5fef6f94d0a52686d3a5d8055"} Dec 03 12:59:14 crc kubenswrapper[5052]: I1203 12:59:14.624740 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openshift-marketplace/redhat-operators-xfdvp" podUID="222ac98a-f9c5-4510-87de-7b8a57b77d11" containerName="registry-server" probeResult="failure" output=< Dec 03 12:59:14 crc kubenswrapper[5052]: timeout: failed to connect service ":50051" within 1s Dec 03 12:59:14 crc kubenswrapper[5052]: > Dec 03 12:59:15 crc kubenswrapper[5052]: I1203 12:59:15.922222 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-ovn-edpm-multinodeset-edpm-compute-global-rkfgm" Dec 03 12:59:15 crc kubenswrapper[5052]: I1203 12:59:15.982898 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/f649f4e5-2cc1-49bd-b4f3-b6c1e8339999-ssh-key\") pod \"f649f4e5-2cc1-49bd-b4f3-b6c1e8339999\" (UID: \"f649f4e5-2cc1-49bd-b4f3-b6c1e8339999\") " Dec 03 12:59:15 crc kubenswrapper[5052]: I1203 12:59:15.983027 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gqm7c\" (UniqueName: \"kubernetes.io/projected/f649f4e5-2cc1-49bd-b4f3-b6c1e8339999-kube-api-access-gqm7c\") pod \"f649f4e5-2cc1-49bd-b4f3-b6c1e8339999\" (UID: \"f649f4e5-2cc1-49bd-b4f3-b6c1e8339999\") " Dec 03 12:59:15 crc kubenswrapper[5052]: I1203 12:59:15.983109 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"neutron-ovn-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f649f4e5-2cc1-49bd-b4f3-b6c1e8339999-neutron-ovn-combined-ca-bundle\") pod \"f649f4e5-2cc1-49bd-b4f3-b6c1e8339999\" (UID: \"f649f4e5-2cc1-49bd-b4f3-b6c1e8339999\") " Dec 03 12:59:15 crc kubenswrapper[5052]: I1203 12:59:15.983138 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"neutron-ovn-agent-neutron-config-0\" (UniqueName: \"kubernetes.io/secret/f649f4e5-2cc1-49bd-b4f3-b6c1e8339999-neutron-ovn-agent-neutron-config-0\") pod \"f649f4e5-2cc1-49bd-b4f3-b6c1e8339999\" (UID: \"f649f4e5-2cc1-49bd-b4f3-b6c1e8339999\") " Dec 03 12:59:15 crc kubenswrapper[5052]: I1203 12:59:15.983226 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/f649f4e5-2cc1-49bd-b4f3-b6c1e8339999-inventory\") pod \"f649f4e5-2cc1-49bd-b4f3-b6c1e8339999\" (UID: \"f649f4e5-2cc1-49bd-b4f3-b6c1e8339999\") " Dec 03 12:59:15 crc kubenswrapper[5052]: I1203 12:59:15.988761 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f649f4e5-2cc1-49bd-b4f3-b6c1e8339999-neutron-ovn-combined-ca-bundle" (OuterVolumeSpecName: "neutron-ovn-combined-ca-bundle") pod "f649f4e5-2cc1-49bd-b4f3-b6c1e8339999" (UID: "f649f4e5-2cc1-49bd-b4f3-b6c1e8339999"). InnerVolumeSpecName "neutron-ovn-combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:59:15 crc kubenswrapper[5052]: I1203 12:59:15.989219 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f649f4e5-2cc1-49bd-b4f3-b6c1e8339999-kube-api-access-gqm7c" (OuterVolumeSpecName: "kube-api-access-gqm7c") pod "f649f4e5-2cc1-49bd-b4f3-b6c1e8339999" (UID: "f649f4e5-2cc1-49bd-b4f3-b6c1e8339999"). InnerVolumeSpecName "kube-api-access-gqm7c". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:59:16 crc kubenswrapper[5052]: I1203 12:59:16.007650 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f649f4e5-2cc1-49bd-b4f3-b6c1e8339999-inventory" (OuterVolumeSpecName: "inventory") pod "f649f4e5-2cc1-49bd-b4f3-b6c1e8339999" (UID: "f649f4e5-2cc1-49bd-b4f3-b6c1e8339999"). InnerVolumeSpecName "inventory". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:59:16 crc kubenswrapper[5052]: I1203 12:59:16.007781 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f649f4e5-2cc1-49bd-b4f3-b6c1e8339999-ssh-key" (OuterVolumeSpecName: "ssh-key") pod "f649f4e5-2cc1-49bd-b4f3-b6c1e8339999" (UID: "f649f4e5-2cc1-49bd-b4f3-b6c1e8339999"). InnerVolumeSpecName "ssh-key". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:59:16 crc kubenswrapper[5052]: I1203 12:59:16.013471 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f649f4e5-2cc1-49bd-b4f3-b6c1e8339999-neutron-ovn-agent-neutron-config-0" (OuterVolumeSpecName: "neutron-ovn-agent-neutron-config-0") pod "f649f4e5-2cc1-49bd-b4f3-b6c1e8339999" (UID: "f649f4e5-2cc1-49bd-b4f3-b6c1e8339999"). InnerVolumeSpecName "neutron-ovn-agent-neutron-config-0". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:59:16 crc kubenswrapper[5052]: I1203 12:59:16.085660 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-gqm7c\" (UniqueName: \"kubernetes.io/projected/f649f4e5-2cc1-49bd-b4f3-b6c1e8339999-kube-api-access-gqm7c\") on node \"crc\" DevicePath \"\"" Dec 03 12:59:16 crc kubenswrapper[5052]: I1203 12:59:16.085697 5052 reconciler_common.go:293] "Volume detached for volume \"neutron-ovn-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f649f4e5-2cc1-49bd-b4f3-b6c1e8339999-neutron-ovn-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:59:16 crc kubenswrapper[5052]: I1203 12:59:16.085708 5052 reconciler_common.go:293] "Volume detached for volume \"neutron-ovn-agent-neutron-config-0\" (UniqueName: \"kubernetes.io/secret/f649f4e5-2cc1-49bd-b4f3-b6c1e8339999-neutron-ovn-agent-neutron-config-0\") on node \"crc\" DevicePath \"\"" Dec 03 12:59:16 crc kubenswrapper[5052]: I1203 12:59:16.085719 5052 reconciler_common.go:293] "Volume detached for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/f649f4e5-2cc1-49bd-b4f3-b6c1e8339999-inventory\") on node \"crc\" DevicePath \"\"" Dec 03 12:59:16 crc kubenswrapper[5052]: I1203 12:59:16.085730 5052 reconciler_common.go:293] "Volume detached for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/f649f4e5-2cc1-49bd-b4f3-b6c1e8339999-ssh-key\") on node \"crc\" DevicePath \"\"" Dec 03 12:59:16 crc kubenswrapper[5052]: I1203 12:59:16.628992 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-ovn-edpm-multinodeset-edpm-compute-global-rkfgm" event={"ID":"f649f4e5-2cc1-49bd-b4f3-b6c1e8339999","Type":"ContainerDied","Data":"f7b6b9aec81b34e5142a49e2f320728bd4d3f10d8d15149b3c15f2f6392c9431"} Dec 03 12:59:16 crc kubenswrapper[5052]: I1203 12:59:16.629333 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f7b6b9aec81b34e5142a49e2f320728bd4d3f10d8d15149b3c15f2f6392c9431" Dec 03 12:59:16 crc kubenswrapper[5052]: I1203 12:59:16.629198 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-ovn-edpm-multinodeset-edpm-compute-global-rkfgm" Dec 03 12:59:16 crc kubenswrapper[5052]: I1203 12:59:16.712705 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/neutron-sriov-edpm-multinodeset-edpm-compute-global-k94g4"] Dec 03 12:59:16 crc kubenswrapper[5052]: E1203 12:59:16.713214 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="f649f4e5-2cc1-49bd-b4f3-b6c1e8339999" containerName="neutron-ovn-edpm-multinodeset-edpm-compute-global" Dec 03 12:59:16 crc kubenswrapper[5052]: I1203 12:59:16.713253 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="f649f4e5-2cc1-49bd-b4f3-b6c1e8339999" containerName="neutron-ovn-edpm-multinodeset-edpm-compute-global" Dec 03 12:59:16 crc kubenswrapper[5052]: I1203 12:59:16.713501 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="f649f4e5-2cc1-49bd-b4f3-b6c1e8339999" containerName="neutron-ovn-edpm-multinodeset-edpm-compute-global" Dec 03 12:59:16 crc kubenswrapper[5052]: I1203 12:59:16.714293 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-sriov-edpm-multinodeset-edpm-compute-global-k94g4" Dec 03 12:59:16 crc kubenswrapper[5052]: I1203 12:59:16.716268 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"openstack-aee-default-env" Dec 03 12:59:16 crc kubenswrapper[5052]: I1203 12:59:16.716322 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"edpm-compute-global-dockercfg-p5rjx" Dec 03 12:59:16 crc kubenswrapper[5052]: I1203 12:59:16.716619 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"dataplanenodeset-edpm-compute-global" Dec 03 12:59:16 crc kubenswrapper[5052]: I1203 12:59:16.717046 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"neutron-sriov-agent-neutron-config" Dec 03 12:59:16 crc kubenswrapper[5052]: I1203 12:59:16.721384 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"combined-ca-bundle" Dec 03 12:59:16 crc kubenswrapper[5052]: I1203 12:59:16.721803 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"dataplane-ansible-ssh-private-key-secret" Dec 03 12:59:16 crc kubenswrapper[5052]: I1203 12:59:16.733560 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-sriov-edpm-multinodeset-edpm-compute-global-k94g4"] Dec 03 12:59:16 crc kubenswrapper[5052]: I1203 12:59:16.795790 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/557d8c8e-5008-4943-8f6e-8883fce0562f-inventory\") pod \"neutron-sriov-edpm-multinodeset-edpm-compute-global-k94g4\" (UID: \"557d8c8e-5008-4943-8f6e-8883fce0562f\") " pod="openstack-kuttl-tests/neutron-sriov-edpm-multinodeset-edpm-compute-global-k94g4" Dec 03 12:59:16 crc kubenswrapper[5052]: I1203 12:59:16.795879 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5zfdr\" (UniqueName: \"kubernetes.io/projected/557d8c8e-5008-4943-8f6e-8883fce0562f-kube-api-access-5zfdr\") pod \"neutron-sriov-edpm-multinodeset-edpm-compute-global-k94g4\" (UID: \"557d8c8e-5008-4943-8f6e-8883fce0562f\") " pod="openstack-kuttl-tests/neutron-sriov-edpm-multinodeset-edpm-compute-global-k94g4" Dec 03 12:59:16 crc kubenswrapper[5052]: I1203 12:59:16.795922 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"neutron-sriov-agent-neutron-config-0\" (UniqueName: \"kubernetes.io/secret/557d8c8e-5008-4943-8f6e-8883fce0562f-neutron-sriov-agent-neutron-config-0\") pod \"neutron-sriov-edpm-multinodeset-edpm-compute-global-k94g4\" (UID: \"557d8c8e-5008-4943-8f6e-8883fce0562f\") " pod="openstack-kuttl-tests/neutron-sriov-edpm-multinodeset-edpm-compute-global-k94g4" Dec 03 12:59:16 crc kubenswrapper[5052]: I1203 12:59:16.795961 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"neutron-sriov-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/557d8c8e-5008-4943-8f6e-8883fce0562f-neutron-sriov-combined-ca-bundle\") pod \"neutron-sriov-edpm-multinodeset-edpm-compute-global-k94g4\" (UID: \"557d8c8e-5008-4943-8f6e-8883fce0562f\") " pod="openstack-kuttl-tests/neutron-sriov-edpm-multinodeset-edpm-compute-global-k94g4" Dec 03 12:59:16 crc kubenswrapper[5052]: I1203 12:59:16.795982 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/557d8c8e-5008-4943-8f6e-8883fce0562f-ssh-key\") pod \"neutron-sriov-edpm-multinodeset-edpm-compute-global-k94g4\" (UID: \"557d8c8e-5008-4943-8f6e-8883fce0562f\") " pod="openstack-kuttl-tests/neutron-sriov-edpm-multinodeset-edpm-compute-global-k94g4" Dec 03 12:59:16 crc kubenswrapper[5052]: I1203 12:59:16.897633 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/557d8c8e-5008-4943-8f6e-8883fce0562f-inventory\") pod \"neutron-sriov-edpm-multinodeset-edpm-compute-global-k94g4\" (UID: \"557d8c8e-5008-4943-8f6e-8883fce0562f\") " pod="openstack-kuttl-tests/neutron-sriov-edpm-multinodeset-edpm-compute-global-k94g4" Dec 03 12:59:16 crc kubenswrapper[5052]: I1203 12:59:16.898478 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-5zfdr\" (UniqueName: \"kubernetes.io/projected/557d8c8e-5008-4943-8f6e-8883fce0562f-kube-api-access-5zfdr\") pod \"neutron-sriov-edpm-multinodeset-edpm-compute-global-k94g4\" (UID: \"557d8c8e-5008-4943-8f6e-8883fce0562f\") " pod="openstack-kuttl-tests/neutron-sriov-edpm-multinodeset-edpm-compute-global-k94g4" Dec 03 12:59:16 crc kubenswrapper[5052]: I1203 12:59:16.898747 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"neutron-sriov-agent-neutron-config-0\" (UniqueName: \"kubernetes.io/secret/557d8c8e-5008-4943-8f6e-8883fce0562f-neutron-sriov-agent-neutron-config-0\") pod \"neutron-sriov-edpm-multinodeset-edpm-compute-global-k94g4\" (UID: \"557d8c8e-5008-4943-8f6e-8883fce0562f\") " pod="openstack-kuttl-tests/neutron-sriov-edpm-multinodeset-edpm-compute-global-k94g4" Dec 03 12:59:16 crc kubenswrapper[5052]: I1203 12:59:16.898809 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"neutron-sriov-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/557d8c8e-5008-4943-8f6e-8883fce0562f-neutron-sriov-combined-ca-bundle\") pod \"neutron-sriov-edpm-multinodeset-edpm-compute-global-k94g4\" (UID: \"557d8c8e-5008-4943-8f6e-8883fce0562f\") " pod="openstack-kuttl-tests/neutron-sriov-edpm-multinodeset-edpm-compute-global-k94g4" Dec 03 12:59:16 crc kubenswrapper[5052]: I1203 12:59:16.898847 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/557d8c8e-5008-4943-8f6e-8883fce0562f-ssh-key\") pod \"neutron-sriov-edpm-multinodeset-edpm-compute-global-k94g4\" (UID: \"557d8c8e-5008-4943-8f6e-8883fce0562f\") " pod="openstack-kuttl-tests/neutron-sriov-edpm-multinodeset-edpm-compute-global-k94g4" Dec 03 12:59:16 crc kubenswrapper[5052]: I1203 12:59:16.902389 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/557d8c8e-5008-4943-8f6e-8883fce0562f-inventory\") pod \"neutron-sriov-edpm-multinodeset-edpm-compute-global-k94g4\" (UID: \"557d8c8e-5008-4943-8f6e-8883fce0562f\") " pod="openstack-kuttl-tests/neutron-sriov-edpm-multinodeset-edpm-compute-global-k94g4" Dec 03 12:59:16 crc kubenswrapper[5052]: I1203 12:59:16.902884 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/557d8c8e-5008-4943-8f6e-8883fce0562f-ssh-key\") pod \"neutron-sriov-edpm-multinodeset-edpm-compute-global-k94g4\" (UID: \"557d8c8e-5008-4943-8f6e-8883fce0562f\") " pod="openstack-kuttl-tests/neutron-sriov-edpm-multinodeset-edpm-compute-global-k94g4" Dec 03 12:59:16 crc kubenswrapper[5052]: I1203 12:59:16.907241 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"neutron-sriov-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/557d8c8e-5008-4943-8f6e-8883fce0562f-neutron-sriov-combined-ca-bundle\") pod \"neutron-sriov-edpm-multinodeset-edpm-compute-global-k94g4\" (UID: \"557d8c8e-5008-4943-8f6e-8883fce0562f\") " pod="openstack-kuttl-tests/neutron-sriov-edpm-multinodeset-edpm-compute-global-k94g4" Dec 03 12:59:16 crc kubenswrapper[5052]: I1203 12:59:16.908177 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"neutron-sriov-agent-neutron-config-0\" (UniqueName: \"kubernetes.io/secret/557d8c8e-5008-4943-8f6e-8883fce0562f-neutron-sriov-agent-neutron-config-0\") pod \"neutron-sriov-edpm-multinodeset-edpm-compute-global-k94g4\" (UID: \"557d8c8e-5008-4943-8f6e-8883fce0562f\") " pod="openstack-kuttl-tests/neutron-sriov-edpm-multinodeset-edpm-compute-global-k94g4" Dec 03 12:59:16 crc kubenswrapper[5052]: I1203 12:59:16.920148 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-5zfdr\" (UniqueName: \"kubernetes.io/projected/557d8c8e-5008-4943-8f6e-8883fce0562f-kube-api-access-5zfdr\") pod \"neutron-sriov-edpm-multinodeset-edpm-compute-global-k94g4\" (UID: \"557d8c8e-5008-4943-8f6e-8883fce0562f\") " pod="openstack-kuttl-tests/neutron-sriov-edpm-multinodeset-edpm-compute-global-k94g4" Dec 03 12:59:17 crc kubenswrapper[5052]: I1203 12:59:17.044530 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-sriov-edpm-multinodeset-edpm-compute-global-k94g4" Dec 03 12:59:17 crc kubenswrapper[5052]: I1203 12:59:17.483297 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-sriov-edpm-multinodeset-edpm-compute-global-k94g4"] Dec 03 12:59:17 crc kubenswrapper[5052]: W1203 12:59:17.487899 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod557d8c8e_5008_4943_8f6e_8883fce0562f.slice/crio-9c37fdca1844a9910565d5059b4d30b02ab380e46ac3ea190ad76ceb69042bec WatchSource:0}: Error finding container 9c37fdca1844a9910565d5059b4d30b02ab380e46ac3ea190ad76ceb69042bec: Status 404 returned error can't find the container with id 9c37fdca1844a9910565d5059b4d30b02ab380e46ac3ea190ad76ceb69042bec Dec 03 12:59:17 crc kubenswrapper[5052]: I1203 12:59:17.641800 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-sriov-edpm-multinodeset-edpm-compute-global-k94g4" event={"ID":"557d8c8e-5008-4943-8f6e-8883fce0562f","Type":"ContainerStarted","Data":"9c37fdca1844a9910565d5059b4d30b02ab380e46ac3ea190ad76ceb69042bec"} Dec 03 12:59:18 crc kubenswrapper[5052]: I1203 12:59:18.653696 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-sriov-edpm-multinodeset-edpm-compute-global-k94g4" event={"ID":"557d8c8e-5008-4943-8f6e-8883fce0562f","Type":"ContainerStarted","Data":"0b6c95d92f1b094a11ce62df4323bfac2aba1e665a8e25685952ebf1c6912b04"} Dec 03 12:59:20 crc kubenswrapper[5052]: I1203 12:59:20.671977 5052 generic.go:334] "Generic (PLEG): container finished" podID="557d8c8e-5008-4943-8f6e-8883fce0562f" containerID="0b6c95d92f1b094a11ce62df4323bfac2aba1e665a8e25685952ebf1c6912b04" exitCode=0 Dec 03 12:59:20 crc kubenswrapper[5052]: I1203 12:59:20.672078 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-sriov-edpm-multinodeset-edpm-compute-global-k94g4" event={"ID":"557d8c8e-5008-4943-8f6e-8883fce0562f","Type":"ContainerDied","Data":"0b6c95d92f1b094a11ce62df4323bfac2aba1e665a8e25685952ebf1c6912b04"} Dec 03 12:59:22 crc kubenswrapper[5052]: I1203 12:59:22.031564 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-sriov-edpm-multinodeset-edpm-compute-global-k94g4" Dec 03 12:59:22 crc kubenswrapper[5052]: I1203 12:59:22.078900 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5zfdr\" (UniqueName: \"kubernetes.io/projected/557d8c8e-5008-4943-8f6e-8883fce0562f-kube-api-access-5zfdr\") pod \"557d8c8e-5008-4943-8f6e-8883fce0562f\" (UID: \"557d8c8e-5008-4943-8f6e-8883fce0562f\") " Dec 03 12:59:22 crc kubenswrapper[5052]: I1203 12:59:22.078976 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/557d8c8e-5008-4943-8f6e-8883fce0562f-inventory\") pod \"557d8c8e-5008-4943-8f6e-8883fce0562f\" (UID: \"557d8c8e-5008-4943-8f6e-8883fce0562f\") " Dec 03 12:59:22 crc kubenswrapper[5052]: I1203 12:59:22.079022 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/557d8c8e-5008-4943-8f6e-8883fce0562f-ssh-key\") pod \"557d8c8e-5008-4943-8f6e-8883fce0562f\" (UID: \"557d8c8e-5008-4943-8f6e-8883fce0562f\") " Dec 03 12:59:22 crc kubenswrapper[5052]: I1203 12:59:22.079047 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"neutron-sriov-agent-neutron-config-0\" (UniqueName: \"kubernetes.io/secret/557d8c8e-5008-4943-8f6e-8883fce0562f-neutron-sriov-agent-neutron-config-0\") pod \"557d8c8e-5008-4943-8f6e-8883fce0562f\" (UID: \"557d8c8e-5008-4943-8f6e-8883fce0562f\") " Dec 03 12:59:22 crc kubenswrapper[5052]: I1203 12:59:22.079111 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"neutron-sriov-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/557d8c8e-5008-4943-8f6e-8883fce0562f-neutron-sriov-combined-ca-bundle\") pod \"557d8c8e-5008-4943-8f6e-8883fce0562f\" (UID: \"557d8c8e-5008-4943-8f6e-8883fce0562f\") " Dec 03 12:59:22 crc kubenswrapper[5052]: I1203 12:59:22.084296 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/557d8c8e-5008-4943-8f6e-8883fce0562f-neutron-sriov-combined-ca-bundle" (OuterVolumeSpecName: "neutron-sriov-combined-ca-bundle") pod "557d8c8e-5008-4943-8f6e-8883fce0562f" (UID: "557d8c8e-5008-4943-8f6e-8883fce0562f"). InnerVolumeSpecName "neutron-sriov-combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:59:22 crc kubenswrapper[5052]: I1203 12:59:22.084379 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/557d8c8e-5008-4943-8f6e-8883fce0562f-kube-api-access-5zfdr" (OuterVolumeSpecName: "kube-api-access-5zfdr") pod "557d8c8e-5008-4943-8f6e-8883fce0562f" (UID: "557d8c8e-5008-4943-8f6e-8883fce0562f"). InnerVolumeSpecName "kube-api-access-5zfdr". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:59:22 crc kubenswrapper[5052]: I1203 12:59:22.102447 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/557d8c8e-5008-4943-8f6e-8883fce0562f-ssh-key" (OuterVolumeSpecName: "ssh-key") pod "557d8c8e-5008-4943-8f6e-8883fce0562f" (UID: "557d8c8e-5008-4943-8f6e-8883fce0562f"). InnerVolumeSpecName "ssh-key". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:59:22 crc kubenswrapper[5052]: I1203 12:59:22.105179 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/557d8c8e-5008-4943-8f6e-8883fce0562f-inventory" (OuterVolumeSpecName: "inventory") pod "557d8c8e-5008-4943-8f6e-8883fce0562f" (UID: "557d8c8e-5008-4943-8f6e-8883fce0562f"). InnerVolumeSpecName "inventory". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:59:22 crc kubenswrapper[5052]: I1203 12:59:22.109841 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/557d8c8e-5008-4943-8f6e-8883fce0562f-neutron-sriov-agent-neutron-config-0" (OuterVolumeSpecName: "neutron-sriov-agent-neutron-config-0") pod "557d8c8e-5008-4943-8f6e-8883fce0562f" (UID: "557d8c8e-5008-4943-8f6e-8883fce0562f"). InnerVolumeSpecName "neutron-sriov-agent-neutron-config-0". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:59:22 crc kubenswrapper[5052]: I1203 12:59:22.181565 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-5zfdr\" (UniqueName: \"kubernetes.io/projected/557d8c8e-5008-4943-8f6e-8883fce0562f-kube-api-access-5zfdr\") on node \"crc\" DevicePath \"\"" Dec 03 12:59:22 crc kubenswrapper[5052]: I1203 12:59:22.181626 5052 reconciler_common.go:293] "Volume detached for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/557d8c8e-5008-4943-8f6e-8883fce0562f-inventory\") on node \"crc\" DevicePath \"\"" Dec 03 12:59:22 crc kubenswrapper[5052]: I1203 12:59:22.181640 5052 reconciler_common.go:293] "Volume detached for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/557d8c8e-5008-4943-8f6e-8883fce0562f-ssh-key\") on node \"crc\" DevicePath \"\"" Dec 03 12:59:22 crc kubenswrapper[5052]: I1203 12:59:22.181658 5052 reconciler_common.go:293] "Volume detached for volume \"neutron-sriov-agent-neutron-config-0\" (UniqueName: \"kubernetes.io/secret/557d8c8e-5008-4943-8f6e-8883fce0562f-neutron-sriov-agent-neutron-config-0\") on node \"crc\" DevicePath \"\"" Dec 03 12:59:22 crc kubenswrapper[5052]: I1203 12:59:22.181677 5052 reconciler_common.go:293] "Volume detached for volume \"neutron-sriov-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/557d8c8e-5008-4943-8f6e-8883fce0562f-neutron-sriov-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:59:22 crc kubenswrapper[5052]: I1203 12:59:22.691745 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-sriov-edpm-multinodeset-edpm-compute-global-k94g4" event={"ID":"557d8c8e-5008-4943-8f6e-8883fce0562f","Type":"ContainerDied","Data":"9c37fdca1844a9910565d5059b4d30b02ab380e46ac3ea190ad76ceb69042bec"} Dec 03 12:59:22 crc kubenswrapper[5052]: I1203 12:59:22.691791 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9c37fdca1844a9910565d5059b4d30b02ab380e46ac3ea190ad76ceb69042bec" Dec 03 12:59:22 crc kubenswrapper[5052]: I1203 12:59:22.692222 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-sriov-edpm-multinodeset-edpm-compute-global-k94g4" Dec 03 12:59:22 crc kubenswrapper[5052]: I1203 12:59:22.759557 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/neutron-dhcp-edpm-multinodeset-edpm-compute-global-9f4bn"] Dec 03 12:59:22 crc kubenswrapper[5052]: E1203 12:59:22.759903 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="557d8c8e-5008-4943-8f6e-8883fce0562f" containerName="neutron-sriov-edpm-multinodeset-edpm-compute-global" Dec 03 12:59:22 crc kubenswrapper[5052]: I1203 12:59:22.759916 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="557d8c8e-5008-4943-8f6e-8883fce0562f" containerName="neutron-sriov-edpm-multinodeset-edpm-compute-global" Dec 03 12:59:22 crc kubenswrapper[5052]: I1203 12:59:22.760088 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="557d8c8e-5008-4943-8f6e-8883fce0562f" containerName="neutron-sriov-edpm-multinodeset-edpm-compute-global" Dec 03 12:59:22 crc kubenswrapper[5052]: I1203 12:59:22.760654 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-dhcp-edpm-multinodeset-edpm-compute-global-9f4bn" Dec 03 12:59:22 crc kubenswrapper[5052]: I1203 12:59:22.763395 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"combined-ca-bundle" Dec 03 12:59:22 crc kubenswrapper[5052]: I1203 12:59:22.763652 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"neutron-dhcp-agent-neutron-config" Dec 03 12:59:22 crc kubenswrapper[5052]: I1203 12:59:22.764313 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"openstack-aee-default-env" Dec 03 12:59:22 crc kubenswrapper[5052]: I1203 12:59:22.764930 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"dataplanenodeset-edpm-compute-global" Dec 03 12:59:22 crc kubenswrapper[5052]: I1203 12:59:22.765144 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"edpm-compute-global-dockercfg-p5rjx" Dec 03 12:59:22 crc kubenswrapper[5052]: I1203 12:59:22.774147 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"dataplane-ansible-ssh-private-key-secret" Dec 03 12:59:22 crc kubenswrapper[5052]: I1203 12:59:22.809858 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-dhcp-edpm-multinodeset-edpm-compute-global-9f4bn"] Dec 03 12:59:22 crc kubenswrapper[5052]: I1203 12:59:22.891911 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"neutron-dhcp-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b31d99bd-183e-43e7-90d7-116dc9c307ed-neutron-dhcp-combined-ca-bundle\") pod \"neutron-dhcp-edpm-multinodeset-edpm-compute-global-9f4bn\" (UID: \"b31d99bd-183e-43e7-90d7-116dc9c307ed\") " pod="openstack-kuttl-tests/neutron-dhcp-edpm-multinodeset-edpm-compute-global-9f4bn" Dec 03 12:59:22 crc kubenswrapper[5052]: I1203 12:59:22.892000 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vkzkp\" (UniqueName: \"kubernetes.io/projected/b31d99bd-183e-43e7-90d7-116dc9c307ed-kube-api-access-vkzkp\") pod \"neutron-dhcp-edpm-multinodeset-edpm-compute-global-9f4bn\" (UID: \"b31d99bd-183e-43e7-90d7-116dc9c307ed\") " pod="openstack-kuttl-tests/neutron-dhcp-edpm-multinodeset-edpm-compute-global-9f4bn" Dec 03 12:59:22 crc kubenswrapper[5052]: I1203 12:59:22.892137 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/b31d99bd-183e-43e7-90d7-116dc9c307ed-ssh-key\") pod \"neutron-dhcp-edpm-multinodeset-edpm-compute-global-9f4bn\" (UID: \"b31d99bd-183e-43e7-90d7-116dc9c307ed\") " pod="openstack-kuttl-tests/neutron-dhcp-edpm-multinodeset-edpm-compute-global-9f4bn" Dec 03 12:59:22 crc kubenswrapper[5052]: I1203 12:59:22.892187 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/b31d99bd-183e-43e7-90d7-116dc9c307ed-inventory\") pod \"neutron-dhcp-edpm-multinodeset-edpm-compute-global-9f4bn\" (UID: \"b31d99bd-183e-43e7-90d7-116dc9c307ed\") " pod="openstack-kuttl-tests/neutron-dhcp-edpm-multinodeset-edpm-compute-global-9f4bn" Dec 03 12:59:22 crc kubenswrapper[5052]: I1203 12:59:22.892318 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"neutron-dhcp-agent-neutron-config-0\" (UniqueName: \"kubernetes.io/secret/b31d99bd-183e-43e7-90d7-116dc9c307ed-neutron-dhcp-agent-neutron-config-0\") pod \"neutron-dhcp-edpm-multinodeset-edpm-compute-global-9f4bn\" (UID: \"b31d99bd-183e-43e7-90d7-116dc9c307ed\") " pod="openstack-kuttl-tests/neutron-dhcp-edpm-multinodeset-edpm-compute-global-9f4bn" Dec 03 12:59:22 crc kubenswrapper[5052]: I1203 12:59:22.994462 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"neutron-dhcp-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b31d99bd-183e-43e7-90d7-116dc9c307ed-neutron-dhcp-combined-ca-bundle\") pod \"neutron-dhcp-edpm-multinodeset-edpm-compute-global-9f4bn\" (UID: \"b31d99bd-183e-43e7-90d7-116dc9c307ed\") " pod="openstack-kuttl-tests/neutron-dhcp-edpm-multinodeset-edpm-compute-global-9f4bn" Dec 03 12:59:22 crc kubenswrapper[5052]: I1203 12:59:22.994568 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-vkzkp\" (UniqueName: \"kubernetes.io/projected/b31d99bd-183e-43e7-90d7-116dc9c307ed-kube-api-access-vkzkp\") pod \"neutron-dhcp-edpm-multinodeset-edpm-compute-global-9f4bn\" (UID: \"b31d99bd-183e-43e7-90d7-116dc9c307ed\") " pod="openstack-kuttl-tests/neutron-dhcp-edpm-multinodeset-edpm-compute-global-9f4bn" Dec 03 12:59:22 crc kubenswrapper[5052]: I1203 12:59:22.994609 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/b31d99bd-183e-43e7-90d7-116dc9c307ed-ssh-key\") pod \"neutron-dhcp-edpm-multinodeset-edpm-compute-global-9f4bn\" (UID: \"b31d99bd-183e-43e7-90d7-116dc9c307ed\") " pod="openstack-kuttl-tests/neutron-dhcp-edpm-multinodeset-edpm-compute-global-9f4bn" Dec 03 12:59:22 crc kubenswrapper[5052]: I1203 12:59:22.994629 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/b31d99bd-183e-43e7-90d7-116dc9c307ed-inventory\") pod \"neutron-dhcp-edpm-multinodeset-edpm-compute-global-9f4bn\" (UID: \"b31d99bd-183e-43e7-90d7-116dc9c307ed\") " pod="openstack-kuttl-tests/neutron-dhcp-edpm-multinodeset-edpm-compute-global-9f4bn" Dec 03 12:59:22 crc kubenswrapper[5052]: I1203 12:59:22.994678 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"neutron-dhcp-agent-neutron-config-0\" (UniqueName: \"kubernetes.io/secret/b31d99bd-183e-43e7-90d7-116dc9c307ed-neutron-dhcp-agent-neutron-config-0\") pod \"neutron-dhcp-edpm-multinodeset-edpm-compute-global-9f4bn\" (UID: \"b31d99bd-183e-43e7-90d7-116dc9c307ed\") " pod="openstack-kuttl-tests/neutron-dhcp-edpm-multinodeset-edpm-compute-global-9f4bn" Dec 03 12:59:22 crc kubenswrapper[5052]: I1203 12:59:22.998146 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"neutron-dhcp-agent-neutron-config-0\" (UniqueName: \"kubernetes.io/secret/b31d99bd-183e-43e7-90d7-116dc9c307ed-neutron-dhcp-agent-neutron-config-0\") pod \"neutron-dhcp-edpm-multinodeset-edpm-compute-global-9f4bn\" (UID: \"b31d99bd-183e-43e7-90d7-116dc9c307ed\") " pod="openstack-kuttl-tests/neutron-dhcp-edpm-multinodeset-edpm-compute-global-9f4bn" Dec 03 12:59:22 crc kubenswrapper[5052]: I1203 12:59:22.998224 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"neutron-dhcp-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b31d99bd-183e-43e7-90d7-116dc9c307ed-neutron-dhcp-combined-ca-bundle\") pod \"neutron-dhcp-edpm-multinodeset-edpm-compute-global-9f4bn\" (UID: \"b31d99bd-183e-43e7-90d7-116dc9c307ed\") " pod="openstack-kuttl-tests/neutron-dhcp-edpm-multinodeset-edpm-compute-global-9f4bn" Dec 03 12:59:22 crc kubenswrapper[5052]: I1203 12:59:22.998831 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/b31d99bd-183e-43e7-90d7-116dc9c307ed-ssh-key\") pod \"neutron-dhcp-edpm-multinodeset-edpm-compute-global-9f4bn\" (UID: \"b31d99bd-183e-43e7-90d7-116dc9c307ed\") " pod="openstack-kuttl-tests/neutron-dhcp-edpm-multinodeset-edpm-compute-global-9f4bn" Dec 03 12:59:23 crc kubenswrapper[5052]: I1203 12:59:23.007129 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/b31d99bd-183e-43e7-90d7-116dc9c307ed-inventory\") pod \"neutron-dhcp-edpm-multinodeset-edpm-compute-global-9f4bn\" (UID: \"b31d99bd-183e-43e7-90d7-116dc9c307ed\") " pod="openstack-kuttl-tests/neutron-dhcp-edpm-multinodeset-edpm-compute-global-9f4bn" Dec 03 12:59:23 crc kubenswrapper[5052]: I1203 12:59:23.016524 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-vkzkp\" (UniqueName: \"kubernetes.io/projected/b31d99bd-183e-43e7-90d7-116dc9c307ed-kube-api-access-vkzkp\") pod \"neutron-dhcp-edpm-multinodeset-edpm-compute-global-9f4bn\" (UID: \"b31d99bd-183e-43e7-90d7-116dc9c307ed\") " pod="openstack-kuttl-tests/neutron-dhcp-edpm-multinodeset-edpm-compute-global-9f4bn" Dec 03 12:59:23 crc kubenswrapper[5052]: I1203 12:59:23.079240 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-dhcp-edpm-multinodeset-edpm-compute-global-9f4bn" Dec 03 12:59:23 crc kubenswrapper[5052]: I1203 12:59:23.553186 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-dhcp-edpm-multinodeset-edpm-compute-global-9f4bn"] Dec 03 12:59:23 crc kubenswrapper[5052]: I1203 12:59:23.610114 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-marketplace/redhat-operators-xfdvp" Dec 03 12:59:23 crc kubenswrapper[5052]: I1203 12:59:23.657257 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-marketplace/redhat-operators-xfdvp" Dec 03 12:59:23 crc kubenswrapper[5052]: I1203 12:59:23.702220 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-dhcp-edpm-multinodeset-edpm-compute-global-9f4bn" event={"ID":"b31d99bd-183e-43e7-90d7-116dc9c307ed","Type":"ContainerStarted","Data":"986667ad6157b9c40dc9b83180bc4c9e0bd0badc36004eabaae26688137e0bf6"} Dec 03 12:59:23 crc kubenswrapper[5052]: I1203 12:59:23.850710 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openshift-marketplace/redhat-operators-xfdvp"] Dec 03 12:59:24 crc kubenswrapper[5052]: I1203 12:59:24.675652 5052 scope.go:117] "RemoveContainer" containerID="69fa7fbcd2aceb7c53703a19508876f93a97dc9adabb79c16fb691d90f606d58" Dec 03 12:59:24 crc kubenswrapper[5052]: E1203 12:59:24.676310 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 12:59:24 crc kubenswrapper[5052]: I1203 12:59:24.712328 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openshift-marketplace/redhat-operators-xfdvp" podUID="222ac98a-f9c5-4510-87de-7b8a57b77d11" containerName="registry-server" containerID="cri-o://0ab687b343e82464a46088560c98c269f61c5a55c5117f6d88c4a5cac4195e09" gracePeriod=2 Dec 03 12:59:25 crc kubenswrapper[5052]: I1203 12:59:25.308360 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/redhat-operators-xfdvp" Dec 03 12:59:25 crc kubenswrapper[5052]: I1203 12:59:25.337269 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6mtgq\" (UniqueName: \"kubernetes.io/projected/222ac98a-f9c5-4510-87de-7b8a57b77d11-kube-api-access-6mtgq\") pod \"222ac98a-f9c5-4510-87de-7b8a57b77d11\" (UID: \"222ac98a-f9c5-4510-87de-7b8a57b77d11\") " Dec 03 12:59:25 crc kubenswrapper[5052]: I1203 12:59:25.337320 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/222ac98a-f9c5-4510-87de-7b8a57b77d11-utilities\") pod \"222ac98a-f9c5-4510-87de-7b8a57b77d11\" (UID: \"222ac98a-f9c5-4510-87de-7b8a57b77d11\") " Dec 03 12:59:25 crc kubenswrapper[5052]: I1203 12:59:25.337416 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/222ac98a-f9c5-4510-87de-7b8a57b77d11-catalog-content\") pod \"222ac98a-f9c5-4510-87de-7b8a57b77d11\" (UID: \"222ac98a-f9c5-4510-87de-7b8a57b77d11\") " Dec 03 12:59:25 crc kubenswrapper[5052]: I1203 12:59:25.347625 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/222ac98a-f9c5-4510-87de-7b8a57b77d11-kube-api-access-6mtgq" (OuterVolumeSpecName: "kube-api-access-6mtgq") pod "222ac98a-f9c5-4510-87de-7b8a57b77d11" (UID: "222ac98a-f9c5-4510-87de-7b8a57b77d11"). InnerVolumeSpecName "kube-api-access-6mtgq". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:59:25 crc kubenswrapper[5052]: I1203 12:59:25.348431 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/222ac98a-f9c5-4510-87de-7b8a57b77d11-utilities" (OuterVolumeSpecName: "utilities") pod "222ac98a-f9c5-4510-87de-7b8a57b77d11" (UID: "222ac98a-f9c5-4510-87de-7b8a57b77d11"). InnerVolumeSpecName "utilities". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:59:25 crc kubenswrapper[5052]: I1203 12:59:25.439221 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-6mtgq\" (UniqueName: \"kubernetes.io/projected/222ac98a-f9c5-4510-87de-7b8a57b77d11-kube-api-access-6mtgq\") on node \"crc\" DevicePath \"\"" Dec 03 12:59:25 crc kubenswrapper[5052]: I1203 12:59:25.439258 5052 reconciler_common.go:293] "Volume detached for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/222ac98a-f9c5-4510-87de-7b8a57b77d11-utilities\") on node \"crc\" DevicePath \"\"" Dec 03 12:59:25 crc kubenswrapper[5052]: I1203 12:59:25.477115 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/222ac98a-f9c5-4510-87de-7b8a57b77d11-catalog-content" (OuterVolumeSpecName: "catalog-content") pod "222ac98a-f9c5-4510-87de-7b8a57b77d11" (UID: "222ac98a-f9c5-4510-87de-7b8a57b77d11"). InnerVolumeSpecName "catalog-content". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 12:59:25 crc kubenswrapper[5052]: I1203 12:59:25.540737 5052 reconciler_common.go:293] "Volume detached for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/222ac98a-f9c5-4510-87de-7b8a57b77d11-catalog-content\") on node \"crc\" DevicePath \"\"" Dec 03 12:59:25 crc kubenswrapper[5052]: I1203 12:59:25.724079 5052 generic.go:334] "Generic (PLEG): container finished" podID="222ac98a-f9c5-4510-87de-7b8a57b77d11" containerID="0ab687b343e82464a46088560c98c269f61c5a55c5117f6d88c4a5cac4195e09" exitCode=0 Dec 03 12:59:25 crc kubenswrapper[5052]: I1203 12:59:25.724134 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-operators-xfdvp" event={"ID":"222ac98a-f9c5-4510-87de-7b8a57b77d11","Type":"ContainerDied","Data":"0ab687b343e82464a46088560c98c269f61c5a55c5117f6d88c4a5cac4195e09"} Dec 03 12:59:25 crc kubenswrapper[5052]: I1203 12:59:25.724160 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-operators-xfdvp" event={"ID":"222ac98a-f9c5-4510-87de-7b8a57b77d11","Type":"ContainerDied","Data":"35bfdc829107420cf1a250ddaa9094f1bf5373548671a52bcf77df01aac2b251"} Dec 03 12:59:25 crc kubenswrapper[5052]: I1203 12:59:25.724176 5052 scope.go:117] "RemoveContainer" containerID="0ab687b343e82464a46088560c98c269f61c5a55c5117f6d88c4a5cac4195e09" Dec 03 12:59:25 crc kubenswrapper[5052]: I1203 12:59:25.724294 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/redhat-operators-xfdvp" Dec 03 12:59:25 crc kubenswrapper[5052]: I1203 12:59:25.739231 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-dhcp-edpm-multinodeset-edpm-compute-global-9f4bn" event={"ID":"b31d99bd-183e-43e7-90d7-116dc9c307ed","Type":"ContainerStarted","Data":"305ee98c14899a5167ed67bbb374adfee664b811ba63d26bfab5af7f0bbc880f"} Dec 03 12:59:25 crc kubenswrapper[5052]: I1203 12:59:25.756361 5052 scope.go:117] "RemoveContainer" containerID="9b3c6c3aa68c86d27aedbad91dcbf96659965b9c49b83b7378f4a79275581e45" Dec 03 12:59:25 crc kubenswrapper[5052]: I1203 12:59:25.765062 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openshift-marketplace/redhat-operators-xfdvp"] Dec 03 12:59:25 crc kubenswrapper[5052]: I1203 12:59:25.776186 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openshift-marketplace/redhat-operators-xfdvp"] Dec 03 12:59:25 crc kubenswrapper[5052]: I1203 12:59:25.778737 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/neutron-dhcp-edpm-multinodeset-edpm-compute-global-9f4bn" podStartSLOduration=2.921163638 podStartE2EDuration="3.778692742s" podCreationTimestamp="2025-12-03 12:59:22 +0000 UTC" firstStartedPulling="2025-12-03 12:59:23.560338101 +0000 UTC m=+5954.148579865" lastFinishedPulling="2025-12-03 12:59:24.417867215 +0000 UTC m=+5955.006108969" observedRunningTime="2025-12-03 12:59:25.765878845 +0000 UTC m=+5956.354120609" watchObservedRunningTime="2025-12-03 12:59:25.778692742 +0000 UTC m=+5956.366934506" Dec 03 12:59:25 crc kubenswrapper[5052]: I1203 12:59:25.787783 5052 scope.go:117] "RemoveContainer" containerID="7a69a280ad50714a1a2ebb99042e6e8cb786ccb73ccc495e54af74389c16f831" Dec 03 12:59:25 crc kubenswrapper[5052]: I1203 12:59:25.808972 5052 scope.go:117] "RemoveContainer" containerID="0ab687b343e82464a46088560c98c269f61c5a55c5117f6d88c4a5cac4195e09" Dec 03 12:59:25 crc kubenswrapper[5052]: E1203 12:59:25.809339 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0ab687b343e82464a46088560c98c269f61c5a55c5117f6d88c4a5cac4195e09\": container with ID starting with 0ab687b343e82464a46088560c98c269f61c5a55c5117f6d88c4a5cac4195e09 not found: ID does not exist" containerID="0ab687b343e82464a46088560c98c269f61c5a55c5117f6d88c4a5cac4195e09" Dec 03 12:59:25 crc kubenswrapper[5052]: I1203 12:59:25.809368 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0ab687b343e82464a46088560c98c269f61c5a55c5117f6d88c4a5cac4195e09"} err="failed to get container status \"0ab687b343e82464a46088560c98c269f61c5a55c5117f6d88c4a5cac4195e09\": rpc error: code = NotFound desc = could not find container \"0ab687b343e82464a46088560c98c269f61c5a55c5117f6d88c4a5cac4195e09\": container with ID starting with 0ab687b343e82464a46088560c98c269f61c5a55c5117f6d88c4a5cac4195e09 not found: ID does not exist" Dec 03 12:59:25 crc kubenswrapper[5052]: I1203 12:59:25.809388 5052 scope.go:117] "RemoveContainer" containerID="9b3c6c3aa68c86d27aedbad91dcbf96659965b9c49b83b7378f4a79275581e45" Dec 03 12:59:25 crc kubenswrapper[5052]: E1203 12:59:25.809880 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9b3c6c3aa68c86d27aedbad91dcbf96659965b9c49b83b7378f4a79275581e45\": container with ID starting with 9b3c6c3aa68c86d27aedbad91dcbf96659965b9c49b83b7378f4a79275581e45 not found: ID does not exist" containerID="9b3c6c3aa68c86d27aedbad91dcbf96659965b9c49b83b7378f4a79275581e45" Dec 03 12:59:25 crc kubenswrapper[5052]: I1203 12:59:25.809910 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9b3c6c3aa68c86d27aedbad91dcbf96659965b9c49b83b7378f4a79275581e45"} err="failed to get container status \"9b3c6c3aa68c86d27aedbad91dcbf96659965b9c49b83b7378f4a79275581e45\": rpc error: code = NotFound desc = could not find container \"9b3c6c3aa68c86d27aedbad91dcbf96659965b9c49b83b7378f4a79275581e45\": container with ID starting with 9b3c6c3aa68c86d27aedbad91dcbf96659965b9c49b83b7378f4a79275581e45 not found: ID does not exist" Dec 03 12:59:25 crc kubenswrapper[5052]: I1203 12:59:25.809928 5052 scope.go:117] "RemoveContainer" containerID="7a69a280ad50714a1a2ebb99042e6e8cb786ccb73ccc495e54af74389c16f831" Dec 03 12:59:25 crc kubenswrapper[5052]: E1203 12:59:25.810214 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7a69a280ad50714a1a2ebb99042e6e8cb786ccb73ccc495e54af74389c16f831\": container with ID starting with 7a69a280ad50714a1a2ebb99042e6e8cb786ccb73ccc495e54af74389c16f831 not found: ID does not exist" containerID="7a69a280ad50714a1a2ebb99042e6e8cb786ccb73ccc495e54af74389c16f831" Dec 03 12:59:25 crc kubenswrapper[5052]: I1203 12:59:25.810234 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7a69a280ad50714a1a2ebb99042e6e8cb786ccb73ccc495e54af74389c16f831"} err="failed to get container status \"7a69a280ad50714a1a2ebb99042e6e8cb786ccb73ccc495e54af74389c16f831\": rpc error: code = NotFound desc = could not find container \"7a69a280ad50714a1a2ebb99042e6e8cb786ccb73ccc495e54af74389c16f831\": container with ID starting with 7a69a280ad50714a1a2ebb99042e6e8cb786ccb73ccc495e54af74389c16f831 not found: ID does not exist" Dec 03 12:59:27 crc kubenswrapper[5052]: I1203 12:59:27.685283 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="222ac98a-f9c5-4510-87de-7b8a57b77d11" path="/var/lib/kubelet/pods/222ac98a-f9c5-4510-87de-7b8a57b77d11/volumes" Dec 03 12:59:27 crc kubenswrapper[5052]: I1203 12:59:27.770022 5052 generic.go:334] "Generic (PLEG): container finished" podID="b31d99bd-183e-43e7-90d7-116dc9c307ed" containerID="305ee98c14899a5167ed67bbb374adfee664b811ba63d26bfab5af7f0bbc880f" exitCode=0 Dec 03 12:59:27 crc kubenswrapper[5052]: I1203 12:59:27.770067 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-dhcp-edpm-multinodeset-edpm-compute-global-9f4bn" event={"ID":"b31d99bd-183e-43e7-90d7-116dc9c307ed","Type":"ContainerDied","Data":"305ee98c14899a5167ed67bbb374adfee664b811ba63d26bfab5af7f0bbc880f"} Dec 03 12:59:29 crc kubenswrapper[5052]: I1203 12:59:29.040270 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-dhcp-edpm-multinodeset-edpm-compute-global-9f4bn" Dec 03 12:59:29 crc kubenswrapper[5052]: I1203 12:59:29.099154 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vkzkp\" (UniqueName: \"kubernetes.io/projected/b31d99bd-183e-43e7-90d7-116dc9c307ed-kube-api-access-vkzkp\") pod \"b31d99bd-183e-43e7-90d7-116dc9c307ed\" (UID: \"b31d99bd-183e-43e7-90d7-116dc9c307ed\") " Dec 03 12:59:29 crc kubenswrapper[5052]: I1203 12:59:29.099282 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"neutron-dhcp-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b31d99bd-183e-43e7-90d7-116dc9c307ed-neutron-dhcp-combined-ca-bundle\") pod \"b31d99bd-183e-43e7-90d7-116dc9c307ed\" (UID: \"b31d99bd-183e-43e7-90d7-116dc9c307ed\") " Dec 03 12:59:29 crc kubenswrapper[5052]: I1203 12:59:29.099312 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/b31d99bd-183e-43e7-90d7-116dc9c307ed-ssh-key\") pod \"b31d99bd-183e-43e7-90d7-116dc9c307ed\" (UID: \"b31d99bd-183e-43e7-90d7-116dc9c307ed\") " Dec 03 12:59:29 crc kubenswrapper[5052]: I1203 12:59:29.099352 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"neutron-dhcp-agent-neutron-config-0\" (UniqueName: \"kubernetes.io/secret/b31d99bd-183e-43e7-90d7-116dc9c307ed-neutron-dhcp-agent-neutron-config-0\") pod \"b31d99bd-183e-43e7-90d7-116dc9c307ed\" (UID: \"b31d99bd-183e-43e7-90d7-116dc9c307ed\") " Dec 03 12:59:29 crc kubenswrapper[5052]: I1203 12:59:29.099398 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/b31d99bd-183e-43e7-90d7-116dc9c307ed-inventory\") pod \"b31d99bd-183e-43e7-90d7-116dc9c307ed\" (UID: \"b31d99bd-183e-43e7-90d7-116dc9c307ed\") " Dec 03 12:59:29 crc kubenswrapper[5052]: I1203 12:59:29.104589 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b31d99bd-183e-43e7-90d7-116dc9c307ed-neutron-dhcp-combined-ca-bundle" (OuterVolumeSpecName: "neutron-dhcp-combined-ca-bundle") pod "b31d99bd-183e-43e7-90d7-116dc9c307ed" (UID: "b31d99bd-183e-43e7-90d7-116dc9c307ed"). InnerVolumeSpecName "neutron-dhcp-combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:59:29 crc kubenswrapper[5052]: I1203 12:59:29.104662 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b31d99bd-183e-43e7-90d7-116dc9c307ed-kube-api-access-vkzkp" (OuterVolumeSpecName: "kube-api-access-vkzkp") pod "b31d99bd-183e-43e7-90d7-116dc9c307ed" (UID: "b31d99bd-183e-43e7-90d7-116dc9c307ed"). InnerVolumeSpecName "kube-api-access-vkzkp". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:59:29 crc kubenswrapper[5052]: I1203 12:59:29.119788 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b31d99bd-183e-43e7-90d7-116dc9c307ed-ssh-key" (OuterVolumeSpecName: "ssh-key") pod "b31d99bd-183e-43e7-90d7-116dc9c307ed" (UID: "b31d99bd-183e-43e7-90d7-116dc9c307ed"). InnerVolumeSpecName "ssh-key". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:59:29 crc kubenswrapper[5052]: I1203 12:59:29.119975 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b31d99bd-183e-43e7-90d7-116dc9c307ed-neutron-dhcp-agent-neutron-config-0" (OuterVolumeSpecName: "neutron-dhcp-agent-neutron-config-0") pod "b31d99bd-183e-43e7-90d7-116dc9c307ed" (UID: "b31d99bd-183e-43e7-90d7-116dc9c307ed"). InnerVolumeSpecName "neutron-dhcp-agent-neutron-config-0". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:59:29 crc kubenswrapper[5052]: I1203 12:59:29.122853 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b31d99bd-183e-43e7-90d7-116dc9c307ed-inventory" (OuterVolumeSpecName: "inventory") pod "b31d99bd-183e-43e7-90d7-116dc9c307ed" (UID: "b31d99bd-183e-43e7-90d7-116dc9c307ed"). InnerVolumeSpecName "inventory". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:59:29 crc kubenswrapper[5052]: I1203 12:59:29.201123 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-vkzkp\" (UniqueName: \"kubernetes.io/projected/b31d99bd-183e-43e7-90d7-116dc9c307ed-kube-api-access-vkzkp\") on node \"crc\" DevicePath \"\"" Dec 03 12:59:29 crc kubenswrapper[5052]: I1203 12:59:29.201162 5052 reconciler_common.go:293] "Volume detached for volume \"neutron-dhcp-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b31d99bd-183e-43e7-90d7-116dc9c307ed-neutron-dhcp-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:59:29 crc kubenswrapper[5052]: I1203 12:59:29.201173 5052 reconciler_common.go:293] "Volume detached for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/b31d99bd-183e-43e7-90d7-116dc9c307ed-ssh-key\") on node \"crc\" DevicePath \"\"" Dec 03 12:59:29 crc kubenswrapper[5052]: I1203 12:59:29.201182 5052 reconciler_common.go:293] "Volume detached for volume \"neutron-dhcp-agent-neutron-config-0\" (UniqueName: \"kubernetes.io/secret/b31d99bd-183e-43e7-90d7-116dc9c307ed-neutron-dhcp-agent-neutron-config-0\") on node \"crc\" DevicePath \"\"" Dec 03 12:59:29 crc kubenswrapper[5052]: I1203 12:59:29.201192 5052 reconciler_common.go:293] "Volume detached for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/b31d99bd-183e-43e7-90d7-116dc9c307ed-inventory\") on node \"crc\" DevicePath \"\"" Dec 03 12:59:29 crc kubenswrapper[5052]: I1203 12:59:29.792549 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-dhcp-edpm-multinodeset-edpm-compute-global-9f4bn" event={"ID":"b31d99bd-183e-43e7-90d7-116dc9c307ed","Type":"ContainerDied","Data":"986667ad6157b9c40dc9b83180bc4c9e0bd0badc36004eabaae26688137e0bf6"} Dec 03 12:59:29 crc kubenswrapper[5052]: I1203 12:59:29.792624 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="986667ad6157b9c40dc9b83180bc4c9e0bd0badc36004eabaae26688137e0bf6" Dec 03 12:59:29 crc kubenswrapper[5052]: I1203 12:59:29.793075 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-dhcp-edpm-multinodeset-edpm-compute-global-9f4bn" Dec 03 12:59:29 crc kubenswrapper[5052]: I1203 12:59:29.873999 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/libvirt-edpm-multinodeset-edpm-compute-global-vv2c2"] Dec 03 12:59:29 crc kubenswrapper[5052]: E1203 12:59:29.874368 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="b31d99bd-183e-43e7-90d7-116dc9c307ed" containerName="neutron-dhcp-edpm-multinodeset-edpm-compute-global" Dec 03 12:59:29 crc kubenswrapper[5052]: I1203 12:59:29.874387 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="b31d99bd-183e-43e7-90d7-116dc9c307ed" containerName="neutron-dhcp-edpm-multinodeset-edpm-compute-global" Dec 03 12:59:29 crc kubenswrapper[5052]: E1203 12:59:29.874413 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="222ac98a-f9c5-4510-87de-7b8a57b77d11" containerName="registry-server" Dec 03 12:59:29 crc kubenswrapper[5052]: I1203 12:59:29.874421 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="222ac98a-f9c5-4510-87de-7b8a57b77d11" containerName="registry-server" Dec 03 12:59:29 crc kubenswrapper[5052]: E1203 12:59:29.874434 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="222ac98a-f9c5-4510-87de-7b8a57b77d11" containerName="extract-utilities" Dec 03 12:59:29 crc kubenswrapper[5052]: I1203 12:59:29.874442 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="222ac98a-f9c5-4510-87de-7b8a57b77d11" containerName="extract-utilities" Dec 03 12:59:29 crc kubenswrapper[5052]: E1203 12:59:29.874475 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="222ac98a-f9c5-4510-87de-7b8a57b77d11" containerName="extract-content" Dec 03 12:59:29 crc kubenswrapper[5052]: I1203 12:59:29.874482 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="222ac98a-f9c5-4510-87de-7b8a57b77d11" containerName="extract-content" Dec 03 12:59:29 crc kubenswrapper[5052]: I1203 12:59:29.875954 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="b31d99bd-183e-43e7-90d7-116dc9c307ed" containerName="neutron-dhcp-edpm-multinodeset-edpm-compute-global" Dec 03 12:59:29 crc kubenswrapper[5052]: I1203 12:59:29.875981 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="222ac98a-f9c5-4510-87de-7b8a57b77d11" containerName="registry-server" Dec 03 12:59:29 crc kubenswrapper[5052]: I1203 12:59:29.876637 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/libvirt-edpm-multinodeset-edpm-compute-global-vv2c2" Dec 03 12:59:29 crc kubenswrapper[5052]: I1203 12:59:29.882071 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"libvirt-secret" Dec 03 12:59:29 crc kubenswrapper[5052]: I1203 12:59:29.882484 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"openstack-aee-default-env" Dec 03 12:59:29 crc kubenswrapper[5052]: I1203 12:59:29.882708 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"dataplanenodeset-edpm-compute-global" Dec 03 12:59:29 crc kubenswrapper[5052]: I1203 12:59:29.882868 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"combined-ca-bundle" Dec 03 12:59:29 crc kubenswrapper[5052]: I1203 12:59:29.883046 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"edpm-compute-global-dockercfg-p5rjx" Dec 03 12:59:29 crc kubenswrapper[5052]: I1203 12:59:29.883176 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"dataplane-ansible-ssh-private-key-secret" Dec 03 12:59:29 crc kubenswrapper[5052]: I1203 12:59:29.889456 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/libvirt-edpm-multinodeset-edpm-compute-global-vv2c2"] Dec 03 12:59:29 crc kubenswrapper[5052]: I1203 12:59:29.912911 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"libvirt-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/dbbe9835-e327-492e-b762-2b2e01b6b27e-libvirt-combined-ca-bundle\") pod \"libvirt-edpm-multinodeset-edpm-compute-global-vv2c2\" (UID: \"dbbe9835-e327-492e-b762-2b2e01b6b27e\") " pod="openstack-kuttl-tests/libvirt-edpm-multinodeset-edpm-compute-global-vv2c2" Dec 03 12:59:29 crc kubenswrapper[5052]: I1203 12:59:29.913069 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/dbbe9835-e327-492e-b762-2b2e01b6b27e-ssh-key\") pod \"libvirt-edpm-multinodeset-edpm-compute-global-vv2c2\" (UID: \"dbbe9835-e327-492e-b762-2b2e01b6b27e\") " pod="openstack-kuttl-tests/libvirt-edpm-multinodeset-edpm-compute-global-vv2c2" Dec 03 12:59:29 crc kubenswrapper[5052]: I1203 12:59:29.913101 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"libvirt-secret-0\" (UniqueName: \"kubernetes.io/secret/dbbe9835-e327-492e-b762-2b2e01b6b27e-libvirt-secret-0\") pod \"libvirt-edpm-multinodeset-edpm-compute-global-vv2c2\" (UID: \"dbbe9835-e327-492e-b762-2b2e01b6b27e\") " pod="openstack-kuttl-tests/libvirt-edpm-multinodeset-edpm-compute-global-vv2c2" Dec 03 12:59:29 crc kubenswrapper[5052]: I1203 12:59:29.913126 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/dbbe9835-e327-492e-b762-2b2e01b6b27e-inventory\") pod \"libvirt-edpm-multinodeset-edpm-compute-global-vv2c2\" (UID: \"dbbe9835-e327-492e-b762-2b2e01b6b27e\") " pod="openstack-kuttl-tests/libvirt-edpm-multinodeset-edpm-compute-global-vv2c2" Dec 03 12:59:29 crc kubenswrapper[5052]: I1203 12:59:29.913161 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-htrz2\" (UniqueName: \"kubernetes.io/projected/dbbe9835-e327-492e-b762-2b2e01b6b27e-kube-api-access-htrz2\") pod \"libvirt-edpm-multinodeset-edpm-compute-global-vv2c2\" (UID: \"dbbe9835-e327-492e-b762-2b2e01b6b27e\") " pod="openstack-kuttl-tests/libvirt-edpm-multinodeset-edpm-compute-global-vv2c2" Dec 03 12:59:30 crc kubenswrapper[5052]: I1203 12:59:30.015103 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"libvirt-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/dbbe9835-e327-492e-b762-2b2e01b6b27e-libvirt-combined-ca-bundle\") pod \"libvirt-edpm-multinodeset-edpm-compute-global-vv2c2\" (UID: \"dbbe9835-e327-492e-b762-2b2e01b6b27e\") " pod="openstack-kuttl-tests/libvirt-edpm-multinodeset-edpm-compute-global-vv2c2" Dec 03 12:59:30 crc kubenswrapper[5052]: I1203 12:59:30.015185 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/dbbe9835-e327-492e-b762-2b2e01b6b27e-ssh-key\") pod \"libvirt-edpm-multinodeset-edpm-compute-global-vv2c2\" (UID: \"dbbe9835-e327-492e-b762-2b2e01b6b27e\") " pod="openstack-kuttl-tests/libvirt-edpm-multinodeset-edpm-compute-global-vv2c2" Dec 03 12:59:30 crc kubenswrapper[5052]: I1203 12:59:30.015212 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"libvirt-secret-0\" (UniqueName: \"kubernetes.io/secret/dbbe9835-e327-492e-b762-2b2e01b6b27e-libvirt-secret-0\") pod \"libvirt-edpm-multinodeset-edpm-compute-global-vv2c2\" (UID: \"dbbe9835-e327-492e-b762-2b2e01b6b27e\") " pod="openstack-kuttl-tests/libvirt-edpm-multinodeset-edpm-compute-global-vv2c2" Dec 03 12:59:30 crc kubenswrapper[5052]: I1203 12:59:30.015240 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/dbbe9835-e327-492e-b762-2b2e01b6b27e-inventory\") pod \"libvirt-edpm-multinodeset-edpm-compute-global-vv2c2\" (UID: \"dbbe9835-e327-492e-b762-2b2e01b6b27e\") " pod="openstack-kuttl-tests/libvirt-edpm-multinodeset-edpm-compute-global-vv2c2" Dec 03 12:59:30 crc kubenswrapper[5052]: I1203 12:59:30.015277 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-htrz2\" (UniqueName: \"kubernetes.io/projected/dbbe9835-e327-492e-b762-2b2e01b6b27e-kube-api-access-htrz2\") pod \"libvirt-edpm-multinodeset-edpm-compute-global-vv2c2\" (UID: \"dbbe9835-e327-492e-b762-2b2e01b6b27e\") " pod="openstack-kuttl-tests/libvirt-edpm-multinodeset-edpm-compute-global-vv2c2" Dec 03 12:59:30 crc kubenswrapper[5052]: I1203 12:59:30.020119 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/dbbe9835-e327-492e-b762-2b2e01b6b27e-inventory\") pod \"libvirt-edpm-multinodeset-edpm-compute-global-vv2c2\" (UID: \"dbbe9835-e327-492e-b762-2b2e01b6b27e\") " pod="openstack-kuttl-tests/libvirt-edpm-multinodeset-edpm-compute-global-vv2c2" Dec 03 12:59:30 crc kubenswrapper[5052]: I1203 12:59:30.020258 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"libvirt-secret-0\" (UniqueName: \"kubernetes.io/secret/dbbe9835-e327-492e-b762-2b2e01b6b27e-libvirt-secret-0\") pod \"libvirt-edpm-multinodeset-edpm-compute-global-vv2c2\" (UID: \"dbbe9835-e327-492e-b762-2b2e01b6b27e\") " pod="openstack-kuttl-tests/libvirt-edpm-multinodeset-edpm-compute-global-vv2c2" Dec 03 12:59:30 crc kubenswrapper[5052]: I1203 12:59:30.020556 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"libvirt-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/dbbe9835-e327-492e-b762-2b2e01b6b27e-libvirt-combined-ca-bundle\") pod \"libvirt-edpm-multinodeset-edpm-compute-global-vv2c2\" (UID: \"dbbe9835-e327-492e-b762-2b2e01b6b27e\") " pod="openstack-kuttl-tests/libvirt-edpm-multinodeset-edpm-compute-global-vv2c2" Dec 03 12:59:30 crc kubenswrapper[5052]: I1203 12:59:30.021341 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/dbbe9835-e327-492e-b762-2b2e01b6b27e-ssh-key\") pod \"libvirt-edpm-multinodeset-edpm-compute-global-vv2c2\" (UID: \"dbbe9835-e327-492e-b762-2b2e01b6b27e\") " pod="openstack-kuttl-tests/libvirt-edpm-multinodeset-edpm-compute-global-vv2c2" Dec 03 12:59:30 crc kubenswrapper[5052]: I1203 12:59:30.032455 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-htrz2\" (UniqueName: \"kubernetes.io/projected/dbbe9835-e327-492e-b762-2b2e01b6b27e-kube-api-access-htrz2\") pod \"libvirt-edpm-multinodeset-edpm-compute-global-vv2c2\" (UID: \"dbbe9835-e327-492e-b762-2b2e01b6b27e\") " pod="openstack-kuttl-tests/libvirt-edpm-multinodeset-edpm-compute-global-vv2c2" Dec 03 12:59:30 crc kubenswrapper[5052]: I1203 12:59:30.190819 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/libvirt-edpm-multinodeset-edpm-compute-global-vv2c2" Dec 03 12:59:30 crc kubenswrapper[5052]: I1203 12:59:30.612093 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/libvirt-edpm-multinodeset-edpm-compute-global-vv2c2"] Dec 03 12:59:30 crc kubenswrapper[5052]: W1203 12:59:30.615579 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddbbe9835_e327_492e_b762_2b2e01b6b27e.slice/crio-f90bbca6567cce4706344cf61bfcf6376f8bd447fd509ad60e33dbdba33bbce0 WatchSource:0}: Error finding container f90bbca6567cce4706344cf61bfcf6376f8bd447fd509ad60e33dbdba33bbce0: Status 404 returned error can't find the container with id f90bbca6567cce4706344cf61bfcf6376f8bd447fd509ad60e33dbdba33bbce0 Dec 03 12:59:30 crc kubenswrapper[5052]: I1203 12:59:30.805383 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/libvirt-edpm-multinodeset-edpm-compute-global-vv2c2" event={"ID":"dbbe9835-e327-492e-b762-2b2e01b6b27e","Type":"ContainerStarted","Data":"f90bbca6567cce4706344cf61bfcf6376f8bd447fd509ad60e33dbdba33bbce0"} Dec 03 12:59:31 crc kubenswrapper[5052]: I1203 12:59:31.816850 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/libvirt-edpm-multinodeset-edpm-compute-global-vv2c2" event={"ID":"dbbe9835-e327-492e-b762-2b2e01b6b27e","Type":"ContainerStarted","Data":"f0f5f5c3f2418588d3b27ec433b9cab51b14cc2c52c0e59202556c9b4642bab5"} Dec 03 12:59:31 crc kubenswrapper[5052]: I1203 12:59:31.845907 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/libvirt-edpm-multinodeset-edpm-compute-global-vv2c2" podStartSLOduration=2.154752458 podStartE2EDuration="2.845887153s" podCreationTimestamp="2025-12-03 12:59:29 +0000 UTC" firstStartedPulling="2025-12-03 12:59:30.617800642 +0000 UTC m=+5961.206042406" lastFinishedPulling="2025-12-03 12:59:31.308935337 +0000 UTC m=+5961.897177101" observedRunningTime="2025-12-03 12:59:31.835793005 +0000 UTC m=+5962.424034789" watchObservedRunningTime="2025-12-03 12:59:31.845887153 +0000 UTC m=+5962.434128917" Dec 03 12:59:32 crc kubenswrapper[5052]: I1203 12:59:32.835048 5052 generic.go:334] "Generic (PLEG): container finished" podID="dbbe9835-e327-492e-b762-2b2e01b6b27e" containerID="f0f5f5c3f2418588d3b27ec433b9cab51b14cc2c52c0e59202556c9b4642bab5" exitCode=0 Dec 03 12:59:32 crc kubenswrapper[5052]: I1203 12:59:32.835097 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/libvirt-edpm-multinodeset-edpm-compute-global-vv2c2" event={"ID":"dbbe9835-e327-492e-b762-2b2e01b6b27e","Type":"ContainerDied","Data":"f0f5f5c3f2418588d3b27ec433b9cab51b14cc2c52c0e59202556c9b4642bab5"} Dec 03 12:59:34 crc kubenswrapper[5052]: I1203 12:59:34.171673 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/libvirt-edpm-multinodeset-edpm-compute-global-vv2c2" Dec 03 12:59:34 crc kubenswrapper[5052]: I1203 12:59:34.273942 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-htrz2\" (UniqueName: \"kubernetes.io/projected/dbbe9835-e327-492e-b762-2b2e01b6b27e-kube-api-access-htrz2\") pod \"dbbe9835-e327-492e-b762-2b2e01b6b27e\" (UID: \"dbbe9835-e327-492e-b762-2b2e01b6b27e\") " Dec 03 12:59:34 crc kubenswrapper[5052]: I1203 12:59:34.273998 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/dbbe9835-e327-492e-b762-2b2e01b6b27e-ssh-key\") pod \"dbbe9835-e327-492e-b762-2b2e01b6b27e\" (UID: \"dbbe9835-e327-492e-b762-2b2e01b6b27e\") " Dec 03 12:59:34 crc kubenswrapper[5052]: I1203 12:59:34.280860 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dbbe9835-e327-492e-b762-2b2e01b6b27e-kube-api-access-htrz2" (OuterVolumeSpecName: "kube-api-access-htrz2") pod "dbbe9835-e327-492e-b762-2b2e01b6b27e" (UID: "dbbe9835-e327-492e-b762-2b2e01b6b27e"). InnerVolumeSpecName "kube-api-access-htrz2". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:59:34 crc kubenswrapper[5052]: I1203 12:59:34.299968 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dbbe9835-e327-492e-b762-2b2e01b6b27e-ssh-key" (OuterVolumeSpecName: "ssh-key") pod "dbbe9835-e327-492e-b762-2b2e01b6b27e" (UID: "dbbe9835-e327-492e-b762-2b2e01b6b27e"). InnerVolumeSpecName "ssh-key". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:59:34 crc kubenswrapper[5052]: I1203 12:59:34.374987 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/dbbe9835-e327-492e-b762-2b2e01b6b27e-inventory\") pod \"dbbe9835-e327-492e-b762-2b2e01b6b27e\" (UID: \"dbbe9835-e327-492e-b762-2b2e01b6b27e\") " Dec 03 12:59:34 crc kubenswrapper[5052]: I1203 12:59:34.375180 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"libvirt-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/dbbe9835-e327-492e-b762-2b2e01b6b27e-libvirt-combined-ca-bundle\") pod \"dbbe9835-e327-492e-b762-2b2e01b6b27e\" (UID: \"dbbe9835-e327-492e-b762-2b2e01b6b27e\") " Dec 03 12:59:34 crc kubenswrapper[5052]: I1203 12:59:34.375249 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"libvirt-secret-0\" (UniqueName: \"kubernetes.io/secret/dbbe9835-e327-492e-b762-2b2e01b6b27e-libvirt-secret-0\") pod \"dbbe9835-e327-492e-b762-2b2e01b6b27e\" (UID: \"dbbe9835-e327-492e-b762-2b2e01b6b27e\") " Dec 03 12:59:34 crc kubenswrapper[5052]: I1203 12:59:34.375471 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-htrz2\" (UniqueName: \"kubernetes.io/projected/dbbe9835-e327-492e-b762-2b2e01b6b27e-kube-api-access-htrz2\") on node \"crc\" DevicePath \"\"" Dec 03 12:59:34 crc kubenswrapper[5052]: I1203 12:59:34.375487 5052 reconciler_common.go:293] "Volume detached for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/dbbe9835-e327-492e-b762-2b2e01b6b27e-ssh-key\") on node \"crc\" DevicePath \"\"" Dec 03 12:59:34 crc kubenswrapper[5052]: I1203 12:59:34.385698 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dbbe9835-e327-492e-b762-2b2e01b6b27e-libvirt-combined-ca-bundle" (OuterVolumeSpecName: "libvirt-combined-ca-bundle") pod "dbbe9835-e327-492e-b762-2b2e01b6b27e" (UID: "dbbe9835-e327-492e-b762-2b2e01b6b27e"). InnerVolumeSpecName "libvirt-combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:59:34 crc kubenswrapper[5052]: I1203 12:59:34.396346 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dbbe9835-e327-492e-b762-2b2e01b6b27e-inventory" (OuterVolumeSpecName: "inventory") pod "dbbe9835-e327-492e-b762-2b2e01b6b27e" (UID: "dbbe9835-e327-492e-b762-2b2e01b6b27e"). InnerVolumeSpecName "inventory". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:59:34 crc kubenswrapper[5052]: I1203 12:59:34.397880 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dbbe9835-e327-492e-b762-2b2e01b6b27e-libvirt-secret-0" (OuterVolumeSpecName: "libvirt-secret-0") pod "dbbe9835-e327-492e-b762-2b2e01b6b27e" (UID: "dbbe9835-e327-492e-b762-2b2e01b6b27e"). InnerVolumeSpecName "libvirt-secret-0". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:59:34 crc kubenswrapper[5052]: I1203 12:59:34.476328 5052 reconciler_common.go:293] "Volume detached for volume \"libvirt-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/dbbe9835-e327-492e-b762-2b2e01b6b27e-libvirt-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:59:34 crc kubenswrapper[5052]: I1203 12:59:34.476689 5052 reconciler_common.go:293] "Volume detached for volume \"libvirt-secret-0\" (UniqueName: \"kubernetes.io/secret/dbbe9835-e327-492e-b762-2b2e01b6b27e-libvirt-secret-0\") on node \"crc\" DevicePath \"\"" Dec 03 12:59:34 crc kubenswrapper[5052]: I1203 12:59:34.476716 5052 reconciler_common.go:293] "Volume detached for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/dbbe9835-e327-492e-b762-2b2e01b6b27e-inventory\") on node \"crc\" DevicePath \"\"" Dec 03 12:59:34 crc kubenswrapper[5052]: I1203 12:59:34.857626 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/libvirt-edpm-multinodeset-edpm-compute-global-vv2c2" event={"ID":"dbbe9835-e327-492e-b762-2b2e01b6b27e","Type":"ContainerDied","Data":"f90bbca6567cce4706344cf61bfcf6376f8bd447fd509ad60e33dbdba33bbce0"} Dec 03 12:59:34 crc kubenswrapper[5052]: I1203 12:59:34.857993 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f90bbca6567cce4706344cf61bfcf6376f8bd447fd509ad60e33dbdba33bbce0" Dec 03 12:59:34 crc kubenswrapper[5052]: I1203 12:59:34.857677 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/libvirt-edpm-multinodeset-edpm-compute-global-vv2c2" Dec 03 12:59:34 crc kubenswrapper[5052]: I1203 12:59:34.923290 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-edpm-multinodeset-edpm-compute-global-h9s55"] Dec 03 12:59:34 crc kubenswrapper[5052]: E1203 12:59:34.923667 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="dbbe9835-e327-492e-b762-2b2e01b6b27e" containerName="libvirt-edpm-multinodeset-edpm-compute-global" Dec 03 12:59:34 crc kubenswrapper[5052]: I1203 12:59:34.923685 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="dbbe9835-e327-492e-b762-2b2e01b6b27e" containerName="libvirt-edpm-multinodeset-edpm-compute-global" Dec 03 12:59:34 crc kubenswrapper[5052]: I1203 12:59:34.923865 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="dbbe9835-e327-492e-b762-2b2e01b6b27e" containerName="libvirt-edpm-multinodeset-edpm-compute-global" Dec 03 12:59:34 crc kubenswrapper[5052]: I1203 12:59:34.924465 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-edpm-multinodeset-edpm-compute-global-h9s55" Dec 03 12:59:34 crc kubenswrapper[5052]: I1203 12:59:34.928664 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"combined-ca-bundle" Dec 03 12:59:34 crc kubenswrapper[5052]: I1203 12:59:34.928696 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"edpm-compute-global-dockercfg-p5rjx" Dec 03 12:59:34 crc kubenswrapper[5052]: I1203 12:59:34.928842 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"dataplane-ansible-ssh-private-key-secret" Dec 03 12:59:34 crc kubenswrapper[5052]: I1203 12:59:34.928872 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-migration-ssh-key" Dec 03 12:59:34 crc kubenswrapper[5052]: I1203 12:59:34.928948 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"dataplanenodeset-edpm-compute-global" Dec 03 12:59:34 crc kubenswrapper[5052]: I1203 12:59:34.929055 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell1-compute-config" Dec 03 12:59:34 crc kubenswrapper[5052]: I1203 12:59:34.939308 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"openstack-aee-default-env" Dec 03 12:59:34 crc kubenswrapper[5052]: I1203 12:59:34.947631 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-edpm-multinodeset-edpm-compute-global-h9s55"] Dec 03 12:59:35 crc kubenswrapper[5052]: I1203 12:59:35.084317 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/777858e3-c5c1-46ab-a05f-c1368a6f0ad0-inventory\") pod \"nova-edpm-multinodeset-edpm-compute-global-h9s55\" (UID: \"777858e3-c5c1-46ab-a05f-c1368a6f0ad0\") " pod="openstack-kuttl-tests/nova-edpm-multinodeset-edpm-compute-global-h9s55" Dec 03 12:59:35 crc kubenswrapper[5052]: I1203 12:59:35.084379 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g66cs\" (UniqueName: \"kubernetes.io/projected/777858e3-c5c1-46ab-a05f-c1368a6f0ad0-kube-api-access-g66cs\") pod \"nova-edpm-multinodeset-edpm-compute-global-h9s55\" (UID: \"777858e3-c5c1-46ab-a05f-c1368a6f0ad0\") " pod="openstack-kuttl-tests/nova-edpm-multinodeset-edpm-compute-global-h9s55" Dec 03 12:59:35 crc kubenswrapper[5052]: I1203 12:59:35.084409 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nova-migration-ssh-key-1\" (UniqueName: \"kubernetes.io/secret/777858e3-c5c1-46ab-a05f-c1368a6f0ad0-nova-migration-ssh-key-1\") pod \"nova-edpm-multinodeset-edpm-compute-global-h9s55\" (UID: \"777858e3-c5c1-46ab-a05f-c1368a6f0ad0\") " pod="openstack-kuttl-tests/nova-edpm-multinodeset-edpm-compute-global-h9s55" Dec 03 12:59:35 crc kubenswrapper[5052]: I1203 12:59:35.084439 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nova-cell1-compute-config-0\" (UniqueName: \"kubernetes.io/secret/777858e3-c5c1-46ab-a05f-c1368a6f0ad0-nova-cell1-compute-config-0\") pod \"nova-edpm-multinodeset-edpm-compute-global-h9s55\" (UID: \"777858e3-c5c1-46ab-a05f-c1368a6f0ad0\") " pod="openstack-kuttl-tests/nova-edpm-multinodeset-edpm-compute-global-h9s55" Dec 03 12:59:35 crc kubenswrapper[5052]: I1203 12:59:35.084463 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nova-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/777858e3-c5c1-46ab-a05f-c1368a6f0ad0-nova-combined-ca-bundle\") pod \"nova-edpm-multinodeset-edpm-compute-global-h9s55\" (UID: \"777858e3-c5c1-46ab-a05f-c1368a6f0ad0\") " pod="openstack-kuttl-tests/nova-edpm-multinodeset-edpm-compute-global-h9s55" Dec 03 12:59:35 crc kubenswrapper[5052]: I1203 12:59:35.084495 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/777858e3-c5c1-46ab-a05f-c1368a6f0ad0-ssh-key\") pod \"nova-edpm-multinodeset-edpm-compute-global-h9s55\" (UID: \"777858e3-c5c1-46ab-a05f-c1368a6f0ad0\") " pod="openstack-kuttl-tests/nova-edpm-multinodeset-edpm-compute-global-h9s55" Dec 03 12:59:35 crc kubenswrapper[5052]: I1203 12:59:35.084609 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nova-migration-ssh-key-0\" (UniqueName: \"kubernetes.io/secret/777858e3-c5c1-46ab-a05f-c1368a6f0ad0-nova-migration-ssh-key-0\") pod \"nova-edpm-multinodeset-edpm-compute-global-h9s55\" (UID: \"777858e3-c5c1-46ab-a05f-c1368a6f0ad0\") " pod="openstack-kuttl-tests/nova-edpm-multinodeset-edpm-compute-global-h9s55" Dec 03 12:59:35 crc kubenswrapper[5052]: I1203 12:59:35.084660 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nova-cell1-compute-config-1\" (UniqueName: \"kubernetes.io/secret/777858e3-c5c1-46ab-a05f-c1368a6f0ad0-nova-cell1-compute-config-1\") pod \"nova-edpm-multinodeset-edpm-compute-global-h9s55\" (UID: \"777858e3-c5c1-46ab-a05f-c1368a6f0ad0\") " pod="openstack-kuttl-tests/nova-edpm-multinodeset-edpm-compute-global-h9s55" Dec 03 12:59:35 crc kubenswrapper[5052]: I1203 12:59:35.186582 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"nova-migration-ssh-key-0\" (UniqueName: \"kubernetes.io/secret/777858e3-c5c1-46ab-a05f-c1368a6f0ad0-nova-migration-ssh-key-0\") pod \"nova-edpm-multinodeset-edpm-compute-global-h9s55\" (UID: \"777858e3-c5c1-46ab-a05f-c1368a6f0ad0\") " pod="openstack-kuttl-tests/nova-edpm-multinodeset-edpm-compute-global-h9s55" Dec 03 12:59:35 crc kubenswrapper[5052]: I1203 12:59:35.186649 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"nova-cell1-compute-config-1\" (UniqueName: \"kubernetes.io/secret/777858e3-c5c1-46ab-a05f-c1368a6f0ad0-nova-cell1-compute-config-1\") pod \"nova-edpm-multinodeset-edpm-compute-global-h9s55\" (UID: \"777858e3-c5c1-46ab-a05f-c1368a6f0ad0\") " pod="openstack-kuttl-tests/nova-edpm-multinodeset-edpm-compute-global-h9s55" Dec 03 12:59:35 crc kubenswrapper[5052]: I1203 12:59:35.186694 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/777858e3-c5c1-46ab-a05f-c1368a6f0ad0-inventory\") pod \"nova-edpm-multinodeset-edpm-compute-global-h9s55\" (UID: \"777858e3-c5c1-46ab-a05f-c1368a6f0ad0\") " pod="openstack-kuttl-tests/nova-edpm-multinodeset-edpm-compute-global-h9s55" Dec 03 12:59:35 crc kubenswrapper[5052]: I1203 12:59:35.186720 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-g66cs\" (UniqueName: \"kubernetes.io/projected/777858e3-c5c1-46ab-a05f-c1368a6f0ad0-kube-api-access-g66cs\") pod \"nova-edpm-multinodeset-edpm-compute-global-h9s55\" (UID: \"777858e3-c5c1-46ab-a05f-c1368a6f0ad0\") " pod="openstack-kuttl-tests/nova-edpm-multinodeset-edpm-compute-global-h9s55" Dec 03 12:59:35 crc kubenswrapper[5052]: I1203 12:59:35.186744 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"nova-migration-ssh-key-1\" (UniqueName: \"kubernetes.io/secret/777858e3-c5c1-46ab-a05f-c1368a6f0ad0-nova-migration-ssh-key-1\") pod \"nova-edpm-multinodeset-edpm-compute-global-h9s55\" (UID: \"777858e3-c5c1-46ab-a05f-c1368a6f0ad0\") " pod="openstack-kuttl-tests/nova-edpm-multinodeset-edpm-compute-global-h9s55" Dec 03 12:59:35 crc kubenswrapper[5052]: I1203 12:59:35.186768 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"nova-cell1-compute-config-0\" (UniqueName: \"kubernetes.io/secret/777858e3-c5c1-46ab-a05f-c1368a6f0ad0-nova-cell1-compute-config-0\") pod \"nova-edpm-multinodeset-edpm-compute-global-h9s55\" (UID: \"777858e3-c5c1-46ab-a05f-c1368a6f0ad0\") " pod="openstack-kuttl-tests/nova-edpm-multinodeset-edpm-compute-global-h9s55" Dec 03 12:59:35 crc kubenswrapper[5052]: I1203 12:59:35.186789 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"nova-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/777858e3-c5c1-46ab-a05f-c1368a6f0ad0-nova-combined-ca-bundle\") pod \"nova-edpm-multinodeset-edpm-compute-global-h9s55\" (UID: \"777858e3-c5c1-46ab-a05f-c1368a6f0ad0\") " pod="openstack-kuttl-tests/nova-edpm-multinodeset-edpm-compute-global-h9s55" Dec 03 12:59:35 crc kubenswrapper[5052]: I1203 12:59:35.186816 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/777858e3-c5c1-46ab-a05f-c1368a6f0ad0-ssh-key\") pod \"nova-edpm-multinodeset-edpm-compute-global-h9s55\" (UID: \"777858e3-c5c1-46ab-a05f-c1368a6f0ad0\") " pod="openstack-kuttl-tests/nova-edpm-multinodeset-edpm-compute-global-h9s55" Dec 03 12:59:35 crc kubenswrapper[5052]: I1203 12:59:35.192889 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"nova-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/777858e3-c5c1-46ab-a05f-c1368a6f0ad0-nova-combined-ca-bundle\") pod \"nova-edpm-multinodeset-edpm-compute-global-h9s55\" (UID: \"777858e3-c5c1-46ab-a05f-c1368a6f0ad0\") " pod="openstack-kuttl-tests/nova-edpm-multinodeset-edpm-compute-global-h9s55" Dec 03 12:59:35 crc kubenswrapper[5052]: I1203 12:59:35.192907 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"nova-cell1-compute-config-0\" (UniqueName: \"kubernetes.io/secret/777858e3-c5c1-46ab-a05f-c1368a6f0ad0-nova-cell1-compute-config-0\") pod \"nova-edpm-multinodeset-edpm-compute-global-h9s55\" (UID: \"777858e3-c5c1-46ab-a05f-c1368a6f0ad0\") " pod="openstack-kuttl-tests/nova-edpm-multinodeset-edpm-compute-global-h9s55" Dec 03 12:59:35 crc kubenswrapper[5052]: I1203 12:59:35.192938 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"nova-migration-ssh-key-1\" (UniqueName: \"kubernetes.io/secret/777858e3-c5c1-46ab-a05f-c1368a6f0ad0-nova-migration-ssh-key-1\") pod \"nova-edpm-multinodeset-edpm-compute-global-h9s55\" (UID: \"777858e3-c5c1-46ab-a05f-c1368a6f0ad0\") " pod="openstack-kuttl-tests/nova-edpm-multinodeset-edpm-compute-global-h9s55" Dec 03 12:59:35 crc kubenswrapper[5052]: I1203 12:59:35.193299 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"nova-migration-ssh-key-0\" (UniqueName: \"kubernetes.io/secret/777858e3-c5c1-46ab-a05f-c1368a6f0ad0-nova-migration-ssh-key-0\") pod \"nova-edpm-multinodeset-edpm-compute-global-h9s55\" (UID: \"777858e3-c5c1-46ab-a05f-c1368a6f0ad0\") " pod="openstack-kuttl-tests/nova-edpm-multinodeset-edpm-compute-global-h9s55" Dec 03 12:59:35 crc kubenswrapper[5052]: I1203 12:59:35.193597 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/777858e3-c5c1-46ab-a05f-c1368a6f0ad0-inventory\") pod \"nova-edpm-multinodeset-edpm-compute-global-h9s55\" (UID: \"777858e3-c5c1-46ab-a05f-c1368a6f0ad0\") " pod="openstack-kuttl-tests/nova-edpm-multinodeset-edpm-compute-global-h9s55" Dec 03 12:59:35 crc kubenswrapper[5052]: I1203 12:59:35.196448 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"nova-cell1-compute-config-1\" (UniqueName: \"kubernetes.io/secret/777858e3-c5c1-46ab-a05f-c1368a6f0ad0-nova-cell1-compute-config-1\") pod \"nova-edpm-multinodeset-edpm-compute-global-h9s55\" (UID: \"777858e3-c5c1-46ab-a05f-c1368a6f0ad0\") " pod="openstack-kuttl-tests/nova-edpm-multinodeset-edpm-compute-global-h9s55" Dec 03 12:59:35 crc kubenswrapper[5052]: I1203 12:59:35.205106 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/777858e3-c5c1-46ab-a05f-c1368a6f0ad0-ssh-key\") pod \"nova-edpm-multinodeset-edpm-compute-global-h9s55\" (UID: \"777858e3-c5c1-46ab-a05f-c1368a6f0ad0\") " pod="openstack-kuttl-tests/nova-edpm-multinodeset-edpm-compute-global-h9s55" Dec 03 12:59:35 crc kubenswrapper[5052]: I1203 12:59:35.206797 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-g66cs\" (UniqueName: \"kubernetes.io/projected/777858e3-c5c1-46ab-a05f-c1368a6f0ad0-kube-api-access-g66cs\") pod \"nova-edpm-multinodeset-edpm-compute-global-h9s55\" (UID: \"777858e3-c5c1-46ab-a05f-c1368a6f0ad0\") " pod="openstack-kuttl-tests/nova-edpm-multinodeset-edpm-compute-global-h9s55" Dec 03 12:59:35 crc kubenswrapper[5052]: I1203 12:59:35.238784 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-edpm-multinodeset-edpm-compute-global-h9s55" Dec 03 12:59:35 crc kubenswrapper[5052]: W1203 12:59:35.680695 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod777858e3_c5c1_46ab_a05f_c1368a6f0ad0.slice/crio-70f26e516ab3b0b5d4c57002c262fe04654ad84cb9fb3917b07b417f0312f0f2 WatchSource:0}: Error finding container 70f26e516ab3b0b5d4c57002c262fe04654ad84cb9fb3917b07b417f0312f0f2: Status 404 returned error can't find the container with id 70f26e516ab3b0b5d4c57002c262fe04654ad84cb9fb3917b07b417f0312f0f2 Dec 03 12:59:35 crc kubenswrapper[5052]: I1203 12:59:35.684734 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-edpm-multinodeset-edpm-compute-global-h9s55"] Dec 03 12:59:35 crc kubenswrapper[5052]: I1203 12:59:35.866673 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-edpm-multinodeset-edpm-compute-global-h9s55" event={"ID":"777858e3-c5c1-46ab-a05f-c1368a6f0ad0","Type":"ContainerStarted","Data":"70f26e516ab3b0b5d4c57002c262fe04654ad84cb9fb3917b07b417f0312f0f2"} Dec 03 12:59:37 crc kubenswrapper[5052]: I1203 12:59:37.917813 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-edpm-multinodeset-edpm-compute-global-h9s55" event={"ID":"777858e3-c5c1-46ab-a05f-c1368a6f0ad0","Type":"ContainerStarted","Data":"4359c78028877a7402ffc505753075779053ce031be99a55254ae4756213eff6"} Dec 03 12:59:37 crc kubenswrapper[5052]: I1203 12:59:37.946415 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-edpm-multinodeset-edpm-compute-global-h9s55" podStartSLOduration=2.787002621 podStartE2EDuration="3.946396157s" podCreationTimestamp="2025-12-03 12:59:34 +0000 UTC" firstStartedPulling="2025-12-03 12:59:35.682806473 +0000 UTC m=+5966.271048237" lastFinishedPulling="2025-12-03 12:59:36.842200009 +0000 UTC m=+5967.430441773" observedRunningTime="2025-12-03 12:59:37.941040464 +0000 UTC m=+5968.529282228" watchObservedRunningTime="2025-12-03 12:59:37.946396157 +0000 UTC m=+5968.534637931" Dec 03 12:59:38 crc kubenswrapper[5052]: I1203 12:59:38.928248 5052 generic.go:334] "Generic (PLEG): container finished" podID="777858e3-c5c1-46ab-a05f-c1368a6f0ad0" containerID="4359c78028877a7402ffc505753075779053ce031be99a55254ae4756213eff6" exitCode=0 Dec 03 12:59:38 crc kubenswrapper[5052]: I1203 12:59:38.928293 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-edpm-multinodeset-edpm-compute-global-h9s55" event={"ID":"777858e3-c5c1-46ab-a05f-c1368a6f0ad0","Type":"ContainerDied","Data":"4359c78028877a7402ffc505753075779053ce031be99a55254ae4756213eff6"} Dec 03 12:59:39 crc kubenswrapper[5052]: I1203 12:59:39.201121 5052 scope.go:117] "RemoveContainer" containerID="21405273563ddcf40333c0b6ddfcd119d733ef5beb5a7399909f33a285ac39e7" Dec 03 12:59:39 crc kubenswrapper[5052]: I1203 12:59:39.221538 5052 scope.go:117] "RemoveContainer" containerID="8752f37ec5e86db977fed1536c7c034e712d2037bc9e10abf57acd410669f7b7" Dec 03 12:59:39 crc kubenswrapper[5052]: I1203 12:59:39.240547 5052 scope.go:117] "RemoveContainer" containerID="e304d74be1d9347456c21077ccfef718ab1e897114fc49c1d7da35457693360a" Dec 03 12:59:39 crc kubenswrapper[5052]: I1203 12:59:39.257995 5052 scope.go:117] "RemoveContainer" containerID="3157794aa44d4b21b9c104515fcfd5ee6e33807ee7b2753cff2132e2facd1f75" Dec 03 12:59:39 crc kubenswrapper[5052]: I1203 12:59:39.274725 5052 scope.go:117] "RemoveContainer" containerID="d2b9e7f45cd74b7356316ded6d8d6691148221f1ac450e577bbac1e71ab3d68c" Dec 03 12:59:39 crc kubenswrapper[5052]: I1203 12:59:39.319133 5052 scope.go:117] "RemoveContainer" containerID="f367321ea2b1658db4fe5944e4616d0065be3a170ac7d1db301f3314e4cbe754" Dec 03 12:59:39 crc kubenswrapper[5052]: I1203 12:59:39.345857 5052 scope.go:117] "RemoveContainer" containerID="c2531bb89e226cd0c966b054b54647ebd280ef61bc50e35addc38acce583274c" Dec 03 12:59:39 crc kubenswrapper[5052]: I1203 12:59:39.394854 5052 scope.go:117] "RemoveContainer" containerID="8ea80ef5cbeb1e7ba8f177b0594824485b3da4da0c66afa4536804902bdf70e1" Dec 03 12:59:39 crc kubenswrapper[5052]: I1203 12:59:39.415721 5052 scope.go:117] "RemoveContainer" containerID="91f773e54ad4ec35e1b76f707be75a9985aeb5b002636c9dd379bb836c31c220" Dec 03 12:59:39 crc kubenswrapper[5052]: I1203 12:59:39.438550 5052 scope.go:117] "RemoveContainer" containerID="a0d54bbd0da0ce78d0c6c5dcb48378864ee5ffb0c8e5280b729c579e83ef32f1" Dec 03 12:59:39 crc kubenswrapper[5052]: I1203 12:59:39.457358 5052 scope.go:117] "RemoveContainer" containerID="c3e875e412556baa33eaca07af1a9b1c17c206e2f38da5c9a1a7982dbc3ff9da" Dec 03 12:59:39 crc kubenswrapper[5052]: I1203 12:59:39.473215 5052 scope.go:117] "RemoveContainer" containerID="01de615203f552d9ea312a07e2ddf729b58c05565d1119d00cbb8d733dbaa190" Dec 03 12:59:39 crc kubenswrapper[5052]: I1203 12:59:39.681623 5052 scope.go:117] "RemoveContainer" containerID="69fa7fbcd2aceb7c53703a19508876f93a97dc9adabb79c16fb691d90f606d58" Dec 03 12:59:39 crc kubenswrapper[5052]: E1203 12:59:39.682044 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 12:59:40 crc kubenswrapper[5052]: I1203 12:59:40.224143 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-edpm-multinodeset-edpm-compute-global-h9s55" Dec 03 12:59:40 crc kubenswrapper[5052]: I1203 12:59:40.354077 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"nova-migration-ssh-key-0\" (UniqueName: \"kubernetes.io/secret/777858e3-c5c1-46ab-a05f-c1368a6f0ad0-nova-migration-ssh-key-0\") pod \"777858e3-c5c1-46ab-a05f-c1368a6f0ad0\" (UID: \"777858e3-c5c1-46ab-a05f-c1368a6f0ad0\") " Dec 03 12:59:40 crc kubenswrapper[5052]: I1203 12:59:40.354169 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"nova-cell1-compute-config-0\" (UniqueName: \"kubernetes.io/secret/777858e3-c5c1-46ab-a05f-c1368a6f0ad0-nova-cell1-compute-config-0\") pod \"777858e3-c5c1-46ab-a05f-c1368a6f0ad0\" (UID: \"777858e3-c5c1-46ab-a05f-c1368a6f0ad0\") " Dec 03 12:59:40 crc kubenswrapper[5052]: I1203 12:59:40.354218 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"nova-migration-ssh-key-1\" (UniqueName: \"kubernetes.io/secret/777858e3-c5c1-46ab-a05f-c1368a6f0ad0-nova-migration-ssh-key-1\") pod \"777858e3-c5c1-46ab-a05f-c1368a6f0ad0\" (UID: \"777858e3-c5c1-46ab-a05f-c1368a6f0ad0\") " Dec 03 12:59:40 crc kubenswrapper[5052]: I1203 12:59:40.354243 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/777858e3-c5c1-46ab-a05f-c1368a6f0ad0-ssh-key\") pod \"777858e3-c5c1-46ab-a05f-c1368a6f0ad0\" (UID: \"777858e3-c5c1-46ab-a05f-c1368a6f0ad0\") " Dec 03 12:59:40 crc kubenswrapper[5052]: I1203 12:59:40.354291 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g66cs\" (UniqueName: \"kubernetes.io/projected/777858e3-c5c1-46ab-a05f-c1368a6f0ad0-kube-api-access-g66cs\") pod \"777858e3-c5c1-46ab-a05f-c1368a6f0ad0\" (UID: \"777858e3-c5c1-46ab-a05f-c1368a6f0ad0\") " Dec 03 12:59:40 crc kubenswrapper[5052]: I1203 12:59:40.354350 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"nova-cell1-compute-config-1\" (UniqueName: \"kubernetes.io/secret/777858e3-c5c1-46ab-a05f-c1368a6f0ad0-nova-cell1-compute-config-1\") pod \"777858e3-c5c1-46ab-a05f-c1368a6f0ad0\" (UID: \"777858e3-c5c1-46ab-a05f-c1368a6f0ad0\") " Dec 03 12:59:40 crc kubenswrapper[5052]: I1203 12:59:40.354456 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"nova-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/777858e3-c5c1-46ab-a05f-c1368a6f0ad0-nova-combined-ca-bundle\") pod \"777858e3-c5c1-46ab-a05f-c1368a6f0ad0\" (UID: \"777858e3-c5c1-46ab-a05f-c1368a6f0ad0\") " Dec 03 12:59:40 crc kubenswrapper[5052]: I1203 12:59:40.354506 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/777858e3-c5c1-46ab-a05f-c1368a6f0ad0-inventory\") pod \"777858e3-c5c1-46ab-a05f-c1368a6f0ad0\" (UID: \"777858e3-c5c1-46ab-a05f-c1368a6f0ad0\") " Dec 03 12:59:40 crc kubenswrapper[5052]: I1203 12:59:40.359752 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/777858e3-c5c1-46ab-a05f-c1368a6f0ad0-kube-api-access-g66cs" (OuterVolumeSpecName: "kube-api-access-g66cs") pod "777858e3-c5c1-46ab-a05f-c1368a6f0ad0" (UID: "777858e3-c5c1-46ab-a05f-c1368a6f0ad0"). InnerVolumeSpecName "kube-api-access-g66cs". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:59:40 crc kubenswrapper[5052]: I1203 12:59:40.365027 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/777858e3-c5c1-46ab-a05f-c1368a6f0ad0-nova-combined-ca-bundle" (OuterVolumeSpecName: "nova-combined-ca-bundle") pod "777858e3-c5c1-46ab-a05f-c1368a6f0ad0" (UID: "777858e3-c5c1-46ab-a05f-c1368a6f0ad0"). InnerVolumeSpecName "nova-combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:59:40 crc kubenswrapper[5052]: I1203 12:59:40.377854 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/777858e3-c5c1-46ab-a05f-c1368a6f0ad0-inventory" (OuterVolumeSpecName: "inventory") pod "777858e3-c5c1-46ab-a05f-c1368a6f0ad0" (UID: "777858e3-c5c1-46ab-a05f-c1368a6f0ad0"). InnerVolumeSpecName "inventory". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:59:40 crc kubenswrapper[5052]: I1203 12:59:40.378787 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/777858e3-c5c1-46ab-a05f-c1368a6f0ad0-nova-migration-ssh-key-0" (OuterVolumeSpecName: "nova-migration-ssh-key-0") pod "777858e3-c5c1-46ab-a05f-c1368a6f0ad0" (UID: "777858e3-c5c1-46ab-a05f-c1368a6f0ad0"). InnerVolumeSpecName "nova-migration-ssh-key-0". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:59:40 crc kubenswrapper[5052]: I1203 12:59:40.380251 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/777858e3-c5c1-46ab-a05f-c1368a6f0ad0-nova-cell1-compute-config-0" (OuterVolumeSpecName: "nova-cell1-compute-config-0") pod "777858e3-c5c1-46ab-a05f-c1368a6f0ad0" (UID: "777858e3-c5c1-46ab-a05f-c1368a6f0ad0"). InnerVolumeSpecName "nova-cell1-compute-config-0". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:59:40 crc kubenswrapper[5052]: I1203 12:59:40.380539 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/777858e3-c5c1-46ab-a05f-c1368a6f0ad0-nova-cell1-compute-config-1" (OuterVolumeSpecName: "nova-cell1-compute-config-1") pod "777858e3-c5c1-46ab-a05f-c1368a6f0ad0" (UID: "777858e3-c5c1-46ab-a05f-c1368a6f0ad0"). InnerVolumeSpecName "nova-cell1-compute-config-1". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:59:40 crc kubenswrapper[5052]: I1203 12:59:40.380705 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/777858e3-c5c1-46ab-a05f-c1368a6f0ad0-ssh-key" (OuterVolumeSpecName: "ssh-key") pod "777858e3-c5c1-46ab-a05f-c1368a6f0ad0" (UID: "777858e3-c5c1-46ab-a05f-c1368a6f0ad0"). InnerVolumeSpecName "ssh-key". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:59:40 crc kubenswrapper[5052]: I1203 12:59:40.383955 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/777858e3-c5c1-46ab-a05f-c1368a6f0ad0-nova-migration-ssh-key-1" (OuterVolumeSpecName: "nova-migration-ssh-key-1") pod "777858e3-c5c1-46ab-a05f-c1368a6f0ad0" (UID: "777858e3-c5c1-46ab-a05f-c1368a6f0ad0"). InnerVolumeSpecName "nova-migration-ssh-key-1". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:59:40 crc kubenswrapper[5052]: I1203 12:59:40.456579 5052 reconciler_common.go:293] "Volume detached for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/777858e3-c5c1-46ab-a05f-c1368a6f0ad0-ssh-key\") on node \"crc\" DevicePath \"\"" Dec 03 12:59:40 crc kubenswrapper[5052]: I1203 12:59:40.456621 5052 reconciler_common.go:293] "Volume detached for volume \"nova-migration-ssh-key-1\" (UniqueName: \"kubernetes.io/secret/777858e3-c5c1-46ab-a05f-c1368a6f0ad0-nova-migration-ssh-key-1\") on node \"crc\" DevicePath \"\"" Dec 03 12:59:40 crc kubenswrapper[5052]: I1203 12:59:40.456635 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-g66cs\" (UniqueName: \"kubernetes.io/projected/777858e3-c5c1-46ab-a05f-c1368a6f0ad0-kube-api-access-g66cs\") on node \"crc\" DevicePath \"\"" Dec 03 12:59:40 crc kubenswrapper[5052]: I1203 12:59:40.456649 5052 reconciler_common.go:293] "Volume detached for volume \"nova-cell1-compute-config-1\" (UniqueName: \"kubernetes.io/secret/777858e3-c5c1-46ab-a05f-c1368a6f0ad0-nova-cell1-compute-config-1\") on node \"crc\" DevicePath \"\"" Dec 03 12:59:40 crc kubenswrapper[5052]: I1203 12:59:40.456662 5052 reconciler_common.go:293] "Volume detached for volume \"nova-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/777858e3-c5c1-46ab-a05f-c1368a6f0ad0-nova-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:59:40 crc kubenswrapper[5052]: I1203 12:59:40.456673 5052 reconciler_common.go:293] "Volume detached for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/777858e3-c5c1-46ab-a05f-c1368a6f0ad0-inventory\") on node \"crc\" DevicePath \"\"" Dec 03 12:59:40 crc kubenswrapper[5052]: I1203 12:59:40.456686 5052 reconciler_common.go:293] "Volume detached for volume \"nova-migration-ssh-key-0\" (UniqueName: \"kubernetes.io/secret/777858e3-c5c1-46ab-a05f-c1368a6f0ad0-nova-migration-ssh-key-0\") on node \"crc\" DevicePath \"\"" Dec 03 12:59:40 crc kubenswrapper[5052]: I1203 12:59:40.456697 5052 reconciler_common.go:293] "Volume detached for volume \"nova-cell1-compute-config-0\" (UniqueName: \"kubernetes.io/secret/777858e3-c5c1-46ab-a05f-c1368a6f0ad0-nova-cell1-compute-config-0\") on node \"crc\" DevicePath \"\"" Dec 03 12:59:40 crc kubenswrapper[5052]: I1203 12:59:40.951682 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-edpm-multinodeset-edpm-compute-global-h9s55" event={"ID":"777858e3-c5c1-46ab-a05f-c1368a6f0ad0","Type":"ContainerDied","Data":"70f26e516ab3b0b5d4c57002c262fe04654ad84cb9fb3917b07b417f0312f0f2"} Dec 03 12:59:40 crc kubenswrapper[5052]: I1203 12:59:40.952104 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="70f26e516ab3b0b5d4c57002c262fe04654ad84cb9fb3917b07b417f0312f0f2" Dec 03 12:59:40 crc kubenswrapper[5052]: I1203 12:59:40.951769 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-edpm-multinodeset-edpm-compute-global-h9s55" Dec 03 12:59:41 crc kubenswrapper[5052]: I1203 12:59:41.024763 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/custom-global-service-edpm-multinodeset-qrkz9"] Dec 03 12:59:41 crc kubenswrapper[5052]: E1203 12:59:41.025201 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="777858e3-c5c1-46ab-a05f-c1368a6f0ad0" containerName="nova-edpm-multinodeset-edpm-compute-global" Dec 03 12:59:41 crc kubenswrapper[5052]: I1203 12:59:41.025221 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="777858e3-c5c1-46ab-a05f-c1368a6f0ad0" containerName="nova-edpm-multinodeset-edpm-compute-global" Dec 03 12:59:41 crc kubenswrapper[5052]: I1203 12:59:41.025392 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="777858e3-c5c1-46ab-a05f-c1368a6f0ad0" containerName="nova-edpm-multinodeset-edpm-compute-global" Dec 03 12:59:41 crc kubenswrapper[5052]: I1203 12:59:41.025902 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/custom-global-service-edpm-multinodeset-qrkz9" Dec 03 12:59:41 crc kubenswrapper[5052]: I1203 12:59:41.027649 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"openstack-aee-default-env" Dec 03 12:59:41 crc kubenswrapper[5052]: I1203 12:59:41.028114 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"edpm-compute-global-dockercfg-p5rjx" Dec 03 12:59:41 crc kubenswrapper[5052]: I1203 12:59:41.028631 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"dataplanenodeset-edpm-compute-beta-nodeset" Dec 03 12:59:41 crc kubenswrapper[5052]: I1203 12:59:41.028770 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"combined-ca-bundle" Dec 03 12:59:41 crc kubenswrapper[5052]: I1203 12:59:41.028778 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"dataplane-ansible-ssh-private-key-secret" Dec 03 12:59:41 crc kubenswrapper[5052]: I1203 12:59:41.028941 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"dataplanenodeset-edpm-compute-global" Dec 03 12:59:41 crc kubenswrapper[5052]: I1203 12:59:41.040342 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/custom-global-service-edpm-multinodeset-qrkz9"] Dec 03 12:59:41 crc kubenswrapper[5052]: I1203 12:59:41.065657 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"custom-global-service-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a085d9e4-a428-4db8-b126-2f9819dbc4d2-custom-global-service-combined-ca-bundle\") pod \"custom-global-service-edpm-multinodeset-qrkz9\" (UID: \"a085d9e4-a428-4db8-b126-2f9819dbc4d2\") " pod="openstack-kuttl-tests/custom-global-service-edpm-multinodeset-qrkz9" Dec 03 12:59:41 crc kubenswrapper[5052]: I1203 12:59:41.167090 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ssh-key-edpm-compute-global\" (UniqueName: \"kubernetes.io/secret/a085d9e4-a428-4db8-b126-2f9819dbc4d2-ssh-key-edpm-compute-global\") pod \"custom-global-service-edpm-multinodeset-qrkz9\" (UID: \"a085d9e4-a428-4db8-b126-2f9819dbc4d2\") " pod="openstack-kuttl-tests/custom-global-service-edpm-multinodeset-qrkz9" Dec 03 12:59:41 crc kubenswrapper[5052]: I1203 12:59:41.167152 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ssh-key-edpm-compute-beta-nodeset\" (UniqueName: \"kubernetes.io/secret/a085d9e4-a428-4db8-b126-2f9819dbc4d2-ssh-key-edpm-compute-beta-nodeset\") pod \"custom-global-service-edpm-multinodeset-qrkz9\" (UID: \"a085d9e4-a428-4db8-b126-2f9819dbc4d2\") " pod="openstack-kuttl-tests/custom-global-service-edpm-multinodeset-qrkz9" Dec 03 12:59:41 crc kubenswrapper[5052]: I1203 12:59:41.167179 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"inventory-1\" (UniqueName: \"kubernetes.io/secret/a085d9e4-a428-4db8-b126-2f9819dbc4d2-inventory-1\") pod \"custom-global-service-edpm-multinodeset-qrkz9\" (UID: \"a085d9e4-a428-4db8-b126-2f9819dbc4d2\") " pod="openstack-kuttl-tests/custom-global-service-edpm-multinodeset-qrkz9" Dec 03 12:59:41 crc kubenswrapper[5052]: I1203 12:59:41.167275 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"inventory-0\" (UniqueName: \"kubernetes.io/secret/a085d9e4-a428-4db8-b126-2f9819dbc4d2-inventory-0\") pod \"custom-global-service-edpm-multinodeset-qrkz9\" (UID: \"a085d9e4-a428-4db8-b126-2f9819dbc4d2\") " pod="openstack-kuttl-tests/custom-global-service-edpm-multinodeset-qrkz9" Dec 03 12:59:41 crc kubenswrapper[5052]: I1203 12:59:41.167362 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"custom-global-service-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a085d9e4-a428-4db8-b126-2f9819dbc4d2-custom-global-service-combined-ca-bundle\") pod \"custom-global-service-edpm-multinodeset-qrkz9\" (UID: \"a085d9e4-a428-4db8-b126-2f9819dbc4d2\") " pod="openstack-kuttl-tests/custom-global-service-edpm-multinodeset-qrkz9" Dec 03 12:59:41 crc kubenswrapper[5052]: I1203 12:59:41.167424 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rpstw\" (UniqueName: \"kubernetes.io/projected/a085d9e4-a428-4db8-b126-2f9819dbc4d2-kube-api-access-rpstw\") pod \"custom-global-service-edpm-multinodeset-qrkz9\" (UID: \"a085d9e4-a428-4db8-b126-2f9819dbc4d2\") " pod="openstack-kuttl-tests/custom-global-service-edpm-multinodeset-qrkz9" Dec 03 12:59:41 crc kubenswrapper[5052]: I1203 12:59:41.170849 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"custom-global-service-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a085d9e4-a428-4db8-b126-2f9819dbc4d2-custom-global-service-combined-ca-bundle\") pod \"custom-global-service-edpm-multinodeset-qrkz9\" (UID: \"a085d9e4-a428-4db8-b126-2f9819dbc4d2\") " pod="openstack-kuttl-tests/custom-global-service-edpm-multinodeset-qrkz9" Dec 03 12:59:41 crc kubenswrapper[5052]: I1203 12:59:41.268281 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-rpstw\" (UniqueName: \"kubernetes.io/projected/a085d9e4-a428-4db8-b126-2f9819dbc4d2-kube-api-access-rpstw\") pod \"custom-global-service-edpm-multinodeset-qrkz9\" (UID: \"a085d9e4-a428-4db8-b126-2f9819dbc4d2\") " pod="openstack-kuttl-tests/custom-global-service-edpm-multinodeset-qrkz9" Dec 03 12:59:41 crc kubenswrapper[5052]: I1203 12:59:41.269267 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ssh-key-edpm-compute-global\" (UniqueName: \"kubernetes.io/secret/a085d9e4-a428-4db8-b126-2f9819dbc4d2-ssh-key-edpm-compute-global\") pod \"custom-global-service-edpm-multinodeset-qrkz9\" (UID: \"a085d9e4-a428-4db8-b126-2f9819dbc4d2\") " pod="openstack-kuttl-tests/custom-global-service-edpm-multinodeset-qrkz9" Dec 03 12:59:41 crc kubenswrapper[5052]: I1203 12:59:41.269575 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ssh-key-edpm-compute-beta-nodeset\" (UniqueName: \"kubernetes.io/secret/a085d9e4-a428-4db8-b126-2f9819dbc4d2-ssh-key-edpm-compute-beta-nodeset\") pod \"custom-global-service-edpm-multinodeset-qrkz9\" (UID: \"a085d9e4-a428-4db8-b126-2f9819dbc4d2\") " pod="openstack-kuttl-tests/custom-global-service-edpm-multinodeset-qrkz9" Dec 03 12:59:41 crc kubenswrapper[5052]: I1203 12:59:41.269654 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"inventory-1\" (UniqueName: \"kubernetes.io/secret/a085d9e4-a428-4db8-b126-2f9819dbc4d2-inventory-1\") pod \"custom-global-service-edpm-multinodeset-qrkz9\" (UID: \"a085d9e4-a428-4db8-b126-2f9819dbc4d2\") " pod="openstack-kuttl-tests/custom-global-service-edpm-multinodeset-qrkz9" Dec 03 12:59:41 crc kubenswrapper[5052]: I1203 12:59:41.269794 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"inventory-0\" (UniqueName: \"kubernetes.io/secret/a085d9e4-a428-4db8-b126-2f9819dbc4d2-inventory-0\") pod \"custom-global-service-edpm-multinodeset-qrkz9\" (UID: \"a085d9e4-a428-4db8-b126-2f9819dbc4d2\") " pod="openstack-kuttl-tests/custom-global-service-edpm-multinodeset-qrkz9" Dec 03 12:59:41 crc kubenswrapper[5052]: I1203 12:59:41.274085 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"inventory-0\" (UniqueName: \"kubernetes.io/secret/a085d9e4-a428-4db8-b126-2f9819dbc4d2-inventory-0\") pod \"custom-global-service-edpm-multinodeset-qrkz9\" (UID: \"a085d9e4-a428-4db8-b126-2f9819dbc4d2\") " pod="openstack-kuttl-tests/custom-global-service-edpm-multinodeset-qrkz9" Dec 03 12:59:41 crc kubenswrapper[5052]: I1203 12:59:41.274363 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"inventory-1\" (UniqueName: \"kubernetes.io/secret/a085d9e4-a428-4db8-b126-2f9819dbc4d2-inventory-1\") pod \"custom-global-service-edpm-multinodeset-qrkz9\" (UID: \"a085d9e4-a428-4db8-b126-2f9819dbc4d2\") " pod="openstack-kuttl-tests/custom-global-service-edpm-multinodeset-qrkz9" Dec 03 12:59:41 crc kubenswrapper[5052]: I1203 12:59:41.275079 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ssh-key-edpm-compute-beta-nodeset\" (UniqueName: \"kubernetes.io/secret/a085d9e4-a428-4db8-b126-2f9819dbc4d2-ssh-key-edpm-compute-beta-nodeset\") pod \"custom-global-service-edpm-multinodeset-qrkz9\" (UID: \"a085d9e4-a428-4db8-b126-2f9819dbc4d2\") " pod="openstack-kuttl-tests/custom-global-service-edpm-multinodeset-qrkz9" Dec 03 12:59:41 crc kubenswrapper[5052]: I1203 12:59:41.275378 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ssh-key-edpm-compute-global\" (UniqueName: \"kubernetes.io/secret/a085d9e4-a428-4db8-b126-2f9819dbc4d2-ssh-key-edpm-compute-global\") pod \"custom-global-service-edpm-multinodeset-qrkz9\" (UID: \"a085d9e4-a428-4db8-b126-2f9819dbc4d2\") " pod="openstack-kuttl-tests/custom-global-service-edpm-multinodeset-qrkz9" Dec 03 12:59:41 crc kubenswrapper[5052]: I1203 12:59:41.288396 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-rpstw\" (UniqueName: \"kubernetes.io/projected/a085d9e4-a428-4db8-b126-2f9819dbc4d2-kube-api-access-rpstw\") pod \"custom-global-service-edpm-multinodeset-qrkz9\" (UID: \"a085d9e4-a428-4db8-b126-2f9819dbc4d2\") " pod="openstack-kuttl-tests/custom-global-service-edpm-multinodeset-qrkz9" Dec 03 12:59:41 crc kubenswrapper[5052]: I1203 12:59:41.345096 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/custom-global-service-edpm-multinodeset-qrkz9" Dec 03 12:59:41 crc kubenswrapper[5052]: I1203 12:59:41.755227 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/custom-global-service-edpm-multinodeset-qrkz9"] Dec 03 12:59:41 crc kubenswrapper[5052]: I1203 12:59:41.962176 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/custom-global-service-edpm-multinodeset-qrkz9" event={"ID":"a085d9e4-a428-4db8-b126-2f9819dbc4d2","Type":"ContainerStarted","Data":"8cecc198cd828896381a2ec6a3519c880280a953a8142399920cd0f9ace6ed63"} Dec 03 12:59:42 crc kubenswrapper[5052]: I1203 12:59:42.974049 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/custom-global-service-edpm-multinodeset-qrkz9" event={"ID":"a085d9e4-a428-4db8-b126-2f9819dbc4d2","Type":"ContainerStarted","Data":"f7c4412f149d80155c8075f51b8dc50b33bf84ecc7229551f39e61369acbf604"} Dec 03 12:59:43 crc kubenswrapper[5052]: I1203 12:59:43.000794 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/custom-global-service-edpm-multinodeset-qrkz9" podStartSLOduration=1.495742361 podStartE2EDuration="2.000777224s" podCreationTimestamp="2025-12-03 12:59:41 +0000 UTC" firstStartedPulling="2025-12-03 12:59:41.765675122 +0000 UTC m=+5972.353916886" lastFinishedPulling="2025-12-03 12:59:42.270709985 +0000 UTC m=+5972.858951749" observedRunningTime="2025-12-03 12:59:42.992654042 +0000 UTC m=+5973.580895806" watchObservedRunningTime="2025-12-03 12:59:43.000777224 +0000 UTC m=+5973.589018988" Dec 03 12:59:46 crc kubenswrapper[5052]: I1203 12:59:46.004708 5052 generic.go:334] "Generic (PLEG): container finished" podID="a085d9e4-a428-4db8-b126-2f9819dbc4d2" containerID="f7c4412f149d80155c8075f51b8dc50b33bf84ecc7229551f39e61369acbf604" exitCode=0 Dec 03 12:59:46 crc kubenswrapper[5052]: I1203 12:59:46.004808 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/custom-global-service-edpm-multinodeset-qrkz9" event={"ID":"a085d9e4-a428-4db8-b126-2f9819dbc4d2","Type":"ContainerDied","Data":"f7c4412f149d80155c8075f51b8dc50b33bf84ecc7229551f39e61369acbf604"} Dec 03 12:59:47 crc kubenswrapper[5052]: I1203 12:59:47.335526 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/custom-global-service-edpm-multinodeset-qrkz9" Dec 03 12:59:47 crc kubenswrapper[5052]: I1203 12:59:47.463924 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"inventory-0\" (UniqueName: \"kubernetes.io/secret/a085d9e4-a428-4db8-b126-2f9819dbc4d2-inventory-0\") pod \"a085d9e4-a428-4db8-b126-2f9819dbc4d2\" (UID: \"a085d9e4-a428-4db8-b126-2f9819dbc4d2\") " Dec 03 12:59:47 crc kubenswrapper[5052]: I1203 12:59:47.464016 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rpstw\" (UniqueName: \"kubernetes.io/projected/a085d9e4-a428-4db8-b126-2f9819dbc4d2-kube-api-access-rpstw\") pod \"a085d9e4-a428-4db8-b126-2f9819dbc4d2\" (UID: \"a085d9e4-a428-4db8-b126-2f9819dbc4d2\") " Dec 03 12:59:47 crc kubenswrapper[5052]: I1203 12:59:47.464047 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ssh-key-edpm-compute-beta-nodeset\" (UniqueName: \"kubernetes.io/secret/a085d9e4-a428-4db8-b126-2f9819dbc4d2-ssh-key-edpm-compute-beta-nodeset\") pod \"a085d9e4-a428-4db8-b126-2f9819dbc4d2\" (UID: \"a085d9e4-a428-4db8-b126-2f9819dbc4d2\") " Dec 03 12:59:47 crc kubenswrapper[5052]: I1203 12:59:47.464083 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"custom-global-service-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a085d9e4-a428-4db8-b126-2f9819dbc4d2-custom-global-service-combined-ca-bundle\") pod \"a085d9e4-a428-4db8-b126-2f9819dbc4d2\" (UID: \"a085d9e4-a428-4db8-b126-2f9819dbc4d2\") " Dec 03 12:59:47 crc kubenswrapper[5052]: I1203 12:59:47.464126 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ssh-key-edpm-compute-global\" (UniqueName: \"kubernetes.io/secret/a085d9e4-a428-4db8-b126-2f9819dbc4d2-ssh-key-edpm-compute-global\") pod \"a085d9e4-a428-4db8-b126-2f9819dbc4d2\" (UID: \"a085d9e4-a428-4db8-b126-2f9819dbc4d2\") " Dec 03 12:59:47 crc kubenswrapper[5052]: I1203 12:59:47.464174 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"inventory-1\" (UniqueName: \"kubernetes.io/secret/a085d9e4-a428-4db8-b126-2f9819dbc4d2-inventory-1\") pod \"a085d9e4-a428-4db8-b126-2f9819dbc4d2\" (UID: \"a085d9e4-a428-4db8-b126-2f9819dbc4d2\") " Dec 03 12:59:47 crc kubenswrapper[5052]: I1203 12:59:47.469609 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a085d9e4-a428-4db8-b126-2f9819dbc4d2-custom-global-service-combined-ca-bundle" (OuterVolumeSpecName: "custom-global-service-combined-ca-bundle") pod "a085d9e4-a428-4db8-b126-2f9819dbc4d2" (UID: "a085d9e4-a428-4db8-b126-2f9819dbc4d2"). InnerVolumeSpecName "custom-global-service-combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:59:47 crc kubenswrapper[5052]: I1203 12:59:47.469954 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a085d9e4-a428-4db8-b126-2f9819dbc4d2-kube-api-access-rpstw" (OuterVolumeSpecName: "kube-api-access-rpstw") pod "a085d9e4-a428-4db8-b126-2f9819dbc4d2" (UID: "a085d9e4-a428-4db8-b126-2f9819dbc4d2"). InnerVolumeSpecName "kube-api-access-rpstw". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 12:59:47 crc kubenswrapper[5052]: I1203 12:59:47.486640 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a085d9e4-a428-4db8-b126-2f9819dbc4d2-inventory-1" (OuterVolumeSpecName: "inventory-1") pod "a085d9e4-a428-4db8-b126-2f9819dbc4d2" (UID: "a085d9e4-a428-4db8-b126-2f9819dbc4d2"). InnerVolumeSpecName "inventory-1". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:59:47 crc kubenswrapper[5052]: I1203 12:59:47.487182 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a085d9e4-a428-4db8-b126-2f9819dbc4d2-ssh-key-edpm-compute-beta-nodeset" (OuterVolumeSpecName: "ssh-key-edpm-compute-beta-nodeset") pod "a085d9e4-a428-4db8-b126-2f9819dbc4d2" (UID: "a085d9e4-a428-4db8-b126-2f9819dbc4d2"). InnerVolumeSpecName "ssh-key-edpm-compute-beta-nodeset". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:59:47 crc kubenswrapper[5052]: I1203 12:59:47.492288 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a085d9e4-a428-4db8-b126-2f9819dbc4d2-inventory-0" (OuterVolumeSpecName: "inventory-0") pod "a085d9e4-a428-4db8-b126-2f9819dbc4d2" (UID: "a085d9e4-a428-4db8-b126-2f9819dbc4d2"). InnerVolumeSpecName "inventory-0". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:59:47 crc kubenswrapper[5052]: I1203 12:59:47.493258 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a085d9e4-a428-4db8-b126-2f9819dbc4d2-ssh-key-edpm-compute-global" (OuterVolumeSpecName: "ssh-key-edpm-compute-global") pod "a085d9e4-a428-4db8-b126-2f9819dbc4d2" (UID: "a085d9e4-a428-4db8-b126-2f9819dbc4d2"). InnerVolumeSpecName "ssh-key-edpm-compute-global". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 12:59:47 crc kubenswrapper[5052]: I1203 12:59:47.565571 5052 reconciler_common.go:293] "Volume detached for volume \"inventory-0\" (UniqueName: \"kubernetes.io/secret/a085d9e4-a428-4db8-b126-2f9819dbc4d2-inventory-0\") on node \"crc\" DevicePath \"\"" Dec 03 12:59:47 crc kubenswrapper[5052]: I1203 12:59:47.565608 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-rpstw\" (UniqueName: \"kubernetes.io/projected/a085d9e4-a428-4db8-b126-2f9819dbc4d2-kube-api-access-rpstw\") on node \"crc\" DevicePath \"\"" Dec 03 12:59:47 crc kubenswrapper[5052]: I1203 12:59:47.565623 5052 reconciler_common.go:293] "Volume detached for volume \"ssh-key-edpm-compute-beta-nodeset\" (UniqueName: \"kubernetes.io/secret/a085d9e4-a428-4db8-b126-2f9819dbc4d2-ssh-key-edpm-compute-beta-nodeset\") on node \"crc\" DevicePath \"\"" Dec 03 12:59:47 crc kubenswrapper[5052]: I1203 12:59:47.565634 5052 reconciler_common.go:293] "Volume detached for volume \"custom-global-service-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a085d9e4-a428-4db8-b126-2f9819dbc4d2-custom-global-service-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 12:59:47 crc kubenswrapper[5052]: I1203 12:59:47.565644 5052 reconciler_common.go:293] "Volume detached for volume \"ssh-key-edpm-compute-global\" (UniqueName: \"kubernetes.io/secret/a085d9e4-a428-4db8-b126-2f9819dbc4d2-ssh-key-edpm-compute-global\") on node \"crc\" DevicePath \"\"" Dec 03 12:59:47 crc kubenswrapper[5052]: I1203 12:59:47.565655 5052 reconciler_common.go:293] "Volume detached for volume \"inventory-1\" (UniqueName: \"kubernetes.io/secret/a085d9e4-a428-4db8-b126-2f9819dbc4d2-inventory-1\") on node \"crc\" DevicePath \"\"" Dec 03 12:59:48 crc kubenswrapper[5052]: I1203 12:59:48.028317 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/custom-global-service-edpm-multinodeset-qrkz9" event={"ID":"a085d9e4-a428-4db8-b126-2f9819dbc4d2","Type":"ContainerDied","Data":"8cecc198cd828896381a2ec6a3519c880280a953a8142399920cd0f9ace6ed63"} Dec 03 12:59:48 crc kubenswrapper[5052]: I1203 12:59:48.028673 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8cecc198cd828896381a2ec6a3519c880280a953a8142399920cd0f9ace6ed63" Dec 03 12:59:48 crc kubenswrapper[5052]: I1203 12:59:48.028418 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/custom-global-service-edpm-multinodeset-qrkz9" Dec 03 12:59:49 crc kubenswrapper[5052]: I1203 12:59:49.290209 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/bootstrap-edpm-multinodeset-edpm-compute-beta-nodeset-fbpnq"] Dec 03 12:59:49 crc kubenswrapper[5052]: I1203 12:59:49.310301 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/download-cache-edpm-multinodeset-edpm-compute-beta-nodesetsnvv9"] Dec 03 12:59:49 crc kubenswrapper[5052]: I1203 12:59:49.336581 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ovn-edpm-multinodeset-edpm-compute-global-m7d25"] Dec 03 12:59:49 crc kubenswrapper[5052]: I1203 12:59:49.346709 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/download-cache-edpm-multinodeset-edpm-compute-beta-nodesetsnvv9"] Dec 03 12:59:49 crc kubenswrapper[5052]: I1203 12:59:49.369748 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ovn-edpm-multinodeset-edpm-compute-global-m7d25"] Dec 03 12:59:49 crc kubenswrapper[5052]: I1203 12:59:49.383774 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/bootstrap-edpm-multinodeset-edpm-compute-beta-nodeset-fbpnq"] Dec 03 12:59:49 crc kubenswrapper[5052]: I1203 12:59:49.396480 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/libvirt-edpm-multinodeset-edpm-compute-global-vv2c2"] Dec 03 12:59:49 crc kubenswrapper[5052]: I1203 12:59:49.401570 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/validate-network-edpm-multinodeset-edpm-compute-global-4rxsd"] Dec 03 12:59:49 crc kubenswrapper[5052]: I1203 12:59:49.411977 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/install-certs-edpm-multinodeset-edpm-compute-global-6dqgm"] Dec 03 12:59:49 crc kubenswrapper[5052]: I1203 12:59:49.420467 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/install-os-edpm-multinodeset-edpm-compute-global-zkjjj"] Dec 03 12:59:49 crc kubenswrapper[5052]: I1203 12:59:49.433851 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/configure-os-edpm-multinodeset-edpm-compute-global-qgc2j"] Dec 03 12:59:49 crc kubenswrapper[5052]: I1203 12:59:49.440419 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/bootstrap-edpm-multinodeset-edpm-compute-global-tq75f"] Dec 03 12:59:49 crc kubenswrapper[5052]: I1203 12:59:49.448079 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/libvirt-edpm-multinodeset-edpm-compute-global-vv2c2"] Dec 03 12:59:49 crc kubenswrapper[5052]: I1203 12:59:49.458731 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/neutron-metadata-edpm-multinodeset-edpm-compute-global-ngmds"] Dec 03 12:59:49 crc kubenswrapper[5052]: I1203 12:59:49.467067 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/configure-network-edpm-multinodeset-edpm-compute-global-z5bnb"] Dec 03 12:59:49 crc kubenswrapper[5052]: I1203 12:59:49.476380 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/neutron-sriov-edpm-multinodeset-edpm-compute-global-k94g4"] Dec 03 12:59:49 crc kubenswrapper[5052]: I1203 12:59:49.488288 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-edpm-multinodeset-edpm-compute-global-h9s55"] Dec 03 12:59:49 crc kubenswrapper[5052]: I1203 12:59:49.497041 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/validate-network-edpm-multinodeset-edpm-compute-global-4rxsd"] Dec 03 12:59:49 crc kubenswrapper[5052]: I1203 12:59:49.513211 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/neutron-ovn-edpm-multinodeset-edpm-compute-global-rkfgm"] Dec 03 12:59:49 crc kubenswrapper[5052]: I1203 12:59:49.513286 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/configure-os-edpm-multinodeset-edpm-compute-global-qgc2j"] Dec 03 12:59:49 crc kubenswrapper[5052]: I1203 12:59:49.529927 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/neutron-metadata-edpm-multinodeset-edpm-compute-global-ngmds"] Dec 03 12:59:49 crc kubenswrapper[5052]: I1203 12:59:49.538638 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/install-certs-edpm-multinodeset-edpm-compute-global-6dqgm"] Dec 03 12:59:49 crc kubenswrapper[5052]: I1203 12:59:49.546893 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/install-os-edpm-multinodeset-edpm-compute-global-zkjjj"] Dec 03 12:59:49 crc kubenswrapper[5052]: I1203 12:59:49.554178 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/neutron-sriov-edpm-multinodeset-edpm-compute-global-k94g4"] Dec 03 12:59:49 crc kubenswrapper[5052]: I1203 12:59:49.562501 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/neutron-dhcp-edpm-multinodeset-edpm-compute-global-9f4bn"] Dec 03 12:59:49 crc kubenswrapper[5052]: I1203 12:59:49.570045 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/configure-network-edpm-multinodeset-edpm-compute-global-z5bnb"] Dec 03 12:59:49 crc kubenswrapper[5052]: I1203 12:59:49.578247 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/neutron-ovn-edpm-multinodeset-edpm-compute-global-rkfgm"] Dec 03 12:59:49 crc kubenswrapper[5052]: I1203 12:59:49.586845 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/bootstrap-edpm-multinodeset-edpm-compute-global-tq75f"] Dec 03 12:59:49 crc kubenswrapper[5052]: I1203 12:59:49.596300 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-edpm-multinodeset-edpm-compute-global-h9s55"] Dec 03 12:59:49 crc kubenswrapper[5052]: I1203 12:59:49.605298 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/neutron-dhcp-edpm-multinodeset-edpm-compute-global-9f4bn"] Dec 03 12:59:49 crc kubenswrapper[5052]: I1203 12:59:49.614086 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/custom-global-service-edpm-multinodeset-qrkz9"] Dec 03 12:59:49 crc kubenswrapper[5052]: I1203 12:59:49.623674 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/download-cache-edpm-multinodeset-edpm-compute-global-kfr9b"] Dec 03 12:59:49 crc kubenswrapper[5052]: I1203 12:59:49.632450 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/run-os-edpm-compute-global-edpm-compute-global-cj7qg"] Dec 03 12:59:49 crc kubenswrapper[5052]: I1203 12:59:49.646762 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/run-os-edpm-compute-global-edpm-compute-global-cj7qg"] Dec 03 12:59:49 crc kubenswrapper[5052]: I1203 12:59:49.657725 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/download-cache-edpm-multinodeset-edpm-compute-global-kfr9b"] Dec 03 12:59:49 crc kubenswrapper[5052]: I1203 12:59:49.664330 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/custom-global-service-edpm-multinodeset-qrkz9"] Dec 03 12:59:49 crc kubenswrapper[5052]: I1203 12:59:49.670856 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/bootstrap-edpm-compute-global-edpm-compute-global-rb7xv"] Dec 03 12:59:49 crc kubenswrapper[5052]: I1203 12:59:49.686246 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="25ab396c-60f7-47e7-a923-709074e77695" path="/var/lib/kubelet/pods/25ab396c-60f7-47e7-a923-709074e77695/volumes" Dec 03 12:59:49 crc kubenswrapper[5052]: I1203 12:59:49.687058 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="35c9f4ce-47f8-4ff2-9716-6ca827dfdf89" path="/var/lib/kubelet/pods/35c9f4ce-47f8-4ff2-9716-6ca827dfdf89/volumes" Dec 03 12:59:49 crc kubenswrapper[5052]: I1203 12:59:49.687645 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4c56e6a0-1f3a-4a96-b207-940aadc3c6ef" path="/var/lib/kubelet/pods/4c56e6a0-1f3a-4a96-b207-940aadc3c6ef/volumes" Dec 03 12:59:49 crc kubenswrapper[5052]: I1203 12:59:49.688271 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="557d8c8e-5008-4943-8f6e-8883fce0562f" path="/var/lib/kubelet/pods/557d8c8e-5008-4943-8f6e-8883fce0562f/volumes" Dec 03 12:59:49 crc kubenswrapper[5052]: I1203 12:59:49.689391 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5d8a070f-f37f-42b9-a47b-cca83f8ffda6" path="/var/lib/kubelet/pods/5d8a070f-f37f-42b9-a47b-cca83f8ffda6/volumes" Dec 03 12:59:49 crc kubenswrapper[5052]: I1203 12:59:49.689908 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="777858e3-c5c1-46ab-a05f-c1368a6f0ad0" path="/var/lib/kubelet/pods/777858e3-c5c1-46ab-a05f-c1368a6f0ad0/volumes" Dec 03 12:59:49 crc kubenswrapper[5052]: I1203 12:59:49.690380 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="865fae8c-437a-482e-8b35-30d9384fc9ca" path="/var/lib/kubelet/pods/865fae8c-437a-482e-8b35-30d9384fc9ca/volumes" Dec 03 12:59:49 crc kubenswrapper[5052]: I1203 12:59:49.691338 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a085d9e4-a428-4db8-b126-2f9819dbc4d2" path="/var/lib/kubelet/pods/a085d9e4-a428-4db8-b126-2f9819dbc4d2/volumes" Dec 03 12:59:49 crc kubenswrapper[5052]: I1203 12:59:49.691869 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a2602d97-293c-4ace-a4e0-974deee70a22" path="/var/lib/kubelet/pods/a2602d97-293c-4ace-a4e0-974deee70a22/volumes" Dec 03 12:59:49 crc kubenswrapper[5052]: I1203 12:59:49.692317 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b31d99bd-183e-43e7-90d7-116dc9c307ed" path="/var/lib/kubelet/pods/b31d99bd-183e-43e7-90d7-116dc9c307ed/volumes" Dec 03 12:59:49 crc kubenswrapper[5052]: I1203 12:59:49.693257 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bf5f88a2-5ab8-4878-a8ba-8541fa324a8e" path="/var/lib/kubelet/pods/bf5f88a2-5ab8-4878-a8ba-8541fa324a8e/volumes" Dec 03 12:59:49 crc kubenswrapper[5052]: I1203 12:59:49.693786 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c3c03807-79b5-49d6-b432-be6be9c109bb" path="/var/lib/kubelet/pods/c3c03807-79b5-49d6-b432-be6be9c109bb/volumes" Dec 03 12:59:49 crc kubenswrapper[5052]: I1203 12:59:49.694223 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ca163200-a99d-41cf-972e-862a71328826" path="/var/lib/kubelet/pods/ca163200-a99d-41cf-972e-862a71328826/volumes" Dec 03 12:59:49 crc kubenswrapper[5052]: I1203 12:59:49.694698 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d51c95a7-7ee5-455d-9cc2-5227f70b2c73" path="/var/lib/kubelet/pods/d51c95a7-7ee5-455d-9cc2-5227f70b2c73/volumes" Dec 03 12:59:49 crc kubenswrapper[5052]: I1203 12:59:49.695611 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dbbe9835-e327-492e-b762-2b2e01b6b27e" path="/var/lib/kubelet/pods/dbbe9835-e327-492e-b762-2b2e01b6b27e/volumes" Dec 03 12:59:49 crc kubenswrapper[5052]: I1203 12:59:49.696123 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f1dc7c86-b3e9-456d-919a-b81c077ecdb4" path="/var/lib/kubelet/pods/f1dc7c86-b3e9-456d-919a-b81c077ecdb4/volumes" Dec 03 12:59:49 crc kubenswrapper[5052]: I1203 12:59:49.696600 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f649f4e5-2cc1-49bd-b4f3-b6c1e8339999" path="/var/lib/kubelet/pods/f649f4e5-2cc1-49bd-b4f3-b6c1e8339999/volumes" Dec 03 12:59:49 crc kubenswrapper[5052]: I1203 12:59:49.697475 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fa21fede-812c-48f1-84e7-899886f312c3" path="/var/lib/kubelet/pods/fa21fede-812c-48f1-84e7-899886f312c3/volumes" Dec 03 12:59:49 crc kubenswrapper[5052]: I1203 12:59:49.698105 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/configure-os-edpm-compute-global-edpm-compute-global-bpx67"] Dec 03 12:59:49 crc kubenswrapper[5052]: I1203 12:59:49.698128 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/neutron-sriov-edpm-compute-global-edpm-compute-global-7xq2m"] Dec 03 12:59:49 crc kubenswrapper[5052]: I1203 12:59:49.698138 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/run-os-edpm-multinodeset-edpm-compute-global-q76r6"] Dec 03 12:59:49 crc kubenswrapper[5052]: I1203 12:59:49.704793 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/neutron-sriov-edpm-compute-global-edpm-compute-global-7xq2m"] Dec 03 12:59:49 crc kubenswrapper[5052]: I1203 12:59:49.711748 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/bootstrap-edpm-compute-global-edpm-compute-global-rb7xv"] Dec 03 12:59:49 crc kubenswrapper[5052]: I1203 12:59:49.718324 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/configure-os-edpm-compute-global-edpm-compute-global-bpx67"] Dec 03 12:59:49 crc kubenswrapper[5052]: I1203 12:59:49.724950 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/custom-global-service-edpm-compute-global-ld7cm"] Dec 03 12:59:49 crc kubenswrapper[5052]: I1203 12:59:49.731850 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/download-cache-edpm-compute-global-edpm-compute-global-jvs5j"] Dec 03 12:59:49 crc kubenswrapper[5052]: I1203 12:59:49.739600 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/run-os-edpm-multinodeset-edpm-compute-global-q76r6"] Dec 03 12:59:49 crc kubenswrapper[5052]: I1203 12:59:49.747164 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/install-certs-edpm-compute-global-edpm-compute-global-l68zr"] Dec 03 12:59:49 crc kubenswrapper[5052]: I1203 12:59:49.753848 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ovn-edpm-compute-global-edpm-compute-global-dw8gp"] Dec 03 12:59:49 crc kubenswrapper[5052]: I1203 12:59:49.761851 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/validate-network-edpm-compute-global-edpm-compute-global-nnvwn"] Dec 03 12:59:49 crc kubenswrapper[5052]: I1203 12:59:49.769229 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-edpm-compute-global-edpm-compute-global-94g5g"] Dec 03 12:59:49 crc kubenswrapper[5052]: I1203 12:59:49.775650 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/install-os-edpm-compute-global-edpm-compute-global-xf85w"] Dec 03 12:59:49 crc kubenswrapper[5052]: I1203 12:59:49.781588 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/custom-global-service-edpm-compute-global-ld7cm"] Dec 03 12:59:49 crc kubenswrapper[5052]: I1203 12:59:49.790557 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/neutron-metadata-edpm-compute-global-edpm-compute-global-8p8rw"] Dec 03 12:59:49 crc kubenswrapper[5052]: I1203 12:59:49.797575 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/libvirt-edpm-compute-global-edpm-compute-global-29fbj"] Dec 03 12:59:49 crc kubenswrapper[5052]: I1203 12:59:49.803729 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/download-cache-edpm-compute-global-edpm-compute-global-jvs5j"] Dec 03 12:59:49 crc kubenswrapper[5052]: I1203 12:59:49.811171 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/configure-network-edpm-compute-global-edpm-compute-global-qf7vq"] Dec 03 12:59:49 crc kubenswrapper[5052]: I1203 12:59:49.817847 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/validate-network-edpm-compute-global-edpm-compute-global-nnvwn"] Dec 03 12:59:49 crc kubenswrapper[5052]: I1203 12:59:49.823771 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/install-certs-edpm-compute-global-edpm-compute-global-l68zr"] Dec 03 12:59:49 crc kubenswrapper[5052]: I1203 12:59:49.830201 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-edpm-compute-global-edpm-compute-global-94g5g"] Dec 03 12:59:49 crc kubenswrapper[5052]: I1203 12:59:49.836556 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/neutron-ovn-edpm-compute-global-edpm-compute-global-mvdtr"] Dec 03 12:59:49 crc kubenswrapper[5052]: I1203 12:59:49.841451 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/neutron-dhcp-edpm-compute-global-edpm-compute-global-22xpf"] Dec 03 12:59:49 crc kubenswrapper[5052]: I1203 12:59:49.847054 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ovn-edpm-compute-global-edpm-compute-global-dw8gp"] Dec 03 12:59:49 crc kubenswrapper[5052]: I1203 12:59:49.852713 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/libvirt-edpm-compute-global-edpm-compute-global-29fbj"] Dec 03 12:59:49 crc kubenswrapper[5052]: I1203 12:59:49.858026 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/configure-network-edpm-compute-global-edpm-compute-global-qf7vq"] Dec 03 12:59:49 crc kubenswrapper[5052]: I1203 12:59:49.863306 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/install-os-edpm-compute-global-edpm-compute-global-xf85w"] Dec 03 12:59:49 crc kubenswrapper[5052]: I1203 12:59:49.868719 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/neutron-metadata-edpm-compute-global-edpm-compute-global-8p8rw"] Dec 03 12:59:49 crc kubenswrapper[5052]: I1203 12:59:49.874818 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/neutron-dhcp-edpm-compute-global-edpm-compute-global-22xpf"] Dec 03 12:59:49 crc kubenswrapper[5052]: I1203 12:59:49.881558 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/neutron-ovn-edpm-compute-global-edpm-compute-global-mvdtr"] Dec 03 12:59:49 crc kubenswrapper[5052]: I1203 12:59:49.887404 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-x6wdj"] Dec 03 12:59:49 crc kubenswrapper[5052]: E1203 12:59:49.887856 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="a085d9e4-a428-4db8-b126-2f9819dbc4d2" containerName="custom-global-service-edpm-multinodeset" Dec 03 12:59:49 crc kubenswrapper[5052]: I1203 12:59:49.887884 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="a085d9e4-a428-4db8-b126-2f9819dbc4d2" containerName="custom-global-service-edpm-multinodeset" Dec 03 12:59:49 crc kubenswrapper[5052]: I1203 12:59:49.888073 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="a085d9e4-a428-4db8-b126-2f9819dbc4d2" containerName="custom-global-service-edpm-multinodeset" Dec 03 12:59:49 crc kubenswrapper[5052]: I1203 12:59:49.889104 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-x6wdj" Dec 03 12:59:49 crc kubenswrapper[5052]: I1203 12:59:49.894075 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-x6wdj"] Dec 03 12:59:49 crc kubenswrapper[5052]: I1203 12:59:49.997861 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/08ab2cda-3a42-49b5-843e-6c8b151064df-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-84b9f45d47-x6wdj\" (UID: \"08ab2cda-3a42-49b5-843e-6c8b151064df\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-x6wdj" Dec 03 12:59:49 crc kubenswrapper[5052]: I1203 12:59:49.997934 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7s4f7\" (UniqueName: \"kubernetes.io/projected/08ab2cda-3a42-49b5-843e-6c8b151064df-kube-api-access-7s4f7\") pod \"dnsmasq-dnsmasq-84b9f45d47-x6wdj\" (UID: \"08ab2cda-3a42-49b5-843e-6c8b151064df\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-x6wdj" Dec 03 12:59:49 crc kubenswrapper[5052]: I1203 12:59:49.997968 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/08ab2cda-3a42-49b5-843e-6c8b151064df-config\") pod \"dnsmasq-dnsmasq-84b9f45d47-x6wdj\" (UID: \"08ab2cda-3a42-49b5-843e-6c8b151064df\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-x6wdj" Dec 03 12:59:50 crc kubenswrapper[5052]: I1203 12:59:50.099555 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/08ab2cda-3a42-49b5-843e-6c8b151064df-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-84b9f45d47-x6wdj\" (UID: \"08ab2cda-3a42-49b5-843e-6c8b151064df\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-x6wdj" Dec 03 12:59:50 crc kubenswrapper[5052]: I1203 12:59:50.099821 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-7s4f7\" (UniqueName: \"kubernetes.io/projected/08ab2cda-3a42-49b5-843e-6c8b151064df-kube-api-access-7s4f7\") pod \"dnsmasq-dnsmasq-84b9f45d47-x6wdj\" (UID: \"08ab2cda-3a42-49b5-843e-6c8b151064df\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-x6wdj" Dec 03 12:59:50 crc kubenswrapper[5052]: I1203 12:59:50.099935 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/08ab2cda-3a42-49b5-843e-6c8b151064df-config\") pod \"dnsmasq-dnsmasq-84b9f45d47-x6wdj\" (UID: \"08ab2cda-3a42-49b5-843e-6c8b151064df\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-x6wdj" Dec 03 12:59:50 crc kubenswrapper[5052]: I1203 12:59:50.100576 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/08ab2cda-3a42-49b5-843e-6c8b151064df-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-84b9f45d47-x6wdj\" (UID: \"08ab2cda-3a42-49b5-843e-6c8b151064df\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-x6wdj" Dec 03 12:59:50 crc kubenswrapper[5052]: I1203 12:59:50.100980 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/08ab2cda-3a42-49b5-843e-6c8b151064df-config\") pod \"dnsmasq-dnsmasq-84b9f45d47-x6wdj\" (UID: \"08ab2cda-3a42-49b5-843e-6c8b151064df\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-x6wdj" Dec 03 12:59:50 crc kubenswrapper[5052]: I1203 12:59:50.122525 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-7s4f7\" (UniqueName: \"kubernetes.io/projected/08ab2cda-3a42-49b5-843e-6c8b151064df-kube-api-access-7s4f7\") pod \"dnsmasq-dnsmasq-84b9f45d47-x6wdj\" (UID: \"08ab2cda-3a42-49b5-843e-6c8b151064df\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-x6wdj" Dec 03 12:59:50 crc kubenswrapper[5052]: I1203 12:59:50.206437 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-x6wdj" Dec 03 12:59:50 crc kubenswrapper[5052]: I1203 12:59:50.680320 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-x6wdj"] Dec 03 12:59:51 crc kubenswrapper[5052]: I1203 12:59:51.070230 5052 generic.go:334] "Generic (PLEG): container finished" podID="08ab2cda-3a42-49b5-843e-6c8b151064df" containerID="77898eab798d2ba81ccac49f52ddc5c77eafc3853bfd6053624c031d324209fb" exitCode=0 Dec 03 12:59:51 crc kubenswrapper[5052]: I1203 12:59:51.070266 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-x6wdj" event={"ID":"08ab2cda-3a42-49b5-843e-6c8b151064df","Type":"ContainerDied","Data":"77898eab798d2ba81ccac49f52ddc5c77eafc3853bfd6053624c031d324209fb"} Dec 03 12:59:51 crc kubenswrapper[5052]: I1203 12:59:51.070288 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-x6wdj" event={"ID":"08ab2cda-3a42-49b5-843e-6c8b151064df","Type":"ContainerStarted","Data":"84da4985770c03243888de71571bd4d9a4a900430d1974aa54555c6e639a529d"} Dec 03 12:59:51 crc kubenswrapper[5052]: I1203 12:59:51.683274 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0577708f-dd8c-4a0d-9660-0c6640d42bc8" path="/var/lib/kubelet/pods/0577708f-dd8c-4a0d-9660-0c6640d42bc8/volumes" Dec 03 12:59:51 crc kubenswrapper[5052]: I1203 12:59:51.683820 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4c4ff49c-5eeb-43cb-ab65-1a284cf1e79a" path="/var/lib/kubelet/pods/4c4ff49c-5eeb-43cb-ab65-1a284cf1e79a/volumes" Dec 03 12:59:51 crc kubenswrapper[5052]: I1203 12:59:51.684339 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5abb2379-9bea-40e8-9a8c-d7accf747008" path="/var/lib/kubelet/pods/5abb2379-9bea-40e8-9a8c-d7accf747008/volumes" Dec 03 12:59:51 crc kubenswrapper[5052]: I1203 12:59:51.684813 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5e1054ea-1bb6-4b9b-9393-26fc530fdc71" path="/var/lib/kubelet/pods/5e1054ea-1bb6-4b9b-9393-26fc530fdc71/volumes" Dec 03 12:59:51 crc kubenswrapper[5052]: I1203 12:59:51.685917 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="626db185-326b-47f7-b41f-b973e9379b94" path="/var/lib/kubelet/pods/626db185-326b-47f7-b41f-b973e9379b94/volumes" Dec 03 12:59:51 crc kubenswrapper[5052]: I1203 12:59:51.686355 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6d5fab08-b680-425a-a4a2-c0b0858b1d8b" path="/var/lib/kubelet/pods/6d5fab08-b680-425a-a4a2-c0b0858b1d8b/volumes" Dec 03 12:59:51 crc kubenswrapper[5052]: I1203 12:59:51.686823 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="763203d9-f2d9-48d7-8989-aa8f1402c776" path="/var/lib/kubelet/pods/763203d9-f2d9-48d7-8989-aa8f1402c776/volumes" Dec 03 12:59:51 crc kubenswrapper[5052]: I1203 12:59:51.687787 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="83a97e99-dd8c-49c7-a503-c09c1768be86" path="/var/lib/kubelet/pods/83a97e99-dd8c-49c7-a503-c09c1768be86/volumes" Dec 03 12:59:51 crc kubenswrapper[5052]: I1203 12:59:51.688244 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="86edaec9-e52a-4e74-8cc9-322449b190d9" path="/var/lib/kubelet/pods/86edaec9-e52a-4e74-8cc9-322449b190d9/volumes" Dec 03 12:59:51 crc kubenswrapper[5052]: I1203 12:59:51.688797 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a3839557-7c97-4814-9c26-48e2c9674a9b" path="/var/lib/kubelet/pods/a3839557-7c97-4814-9c26-48e2c9674a9b/volumes" Dec 03 12:59:51 crc kubenswrapper[5052]: I1203 12:59:51.689730 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a5fbd1bc-e091-459b-97fa-17d728070100" path="/var/lib/kubelet/pods/a5fbd1bc-e091-459b-97fa-17d728070100/volumes" Dec 03 12:59:51 crc kubenswrapper[5052]: I1203 12:59:51.690192 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c7885c2e-f7d6-4cc3-8412-0cf80ebe7950" path="/var/lib/kubelet/pods/c7885c2e-f7d6-4cc3-8412-0cf80ebe7950/volumes" Dec 03 12:59:51 crc kubenswrapper[5052]: I1203 12:59:51.690931 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c8557042-afbc-4b35-996c-e2c060db2f2c" path="/var/lib/kubelet/pods/c8557042-afbc-4b35-996c-e2c060db2f2c/volumes" Dec 03 12:59:51 crc kubenswrapper[5052]: I1203 12:59:51.691492 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c98ad99e-c1f3-46d7-b810-a702c08308f3" path="/var/lib/kubelet/pods/c98ad99e-c1f3-46d7-b810-a702c08308f3/volumes" Dec 03 12:59:51 crc kubenswrapper[5052]: I1203 12:59:51.692534 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e3d7c09b-d632-44e9-b7b4-26c841d13f7f" path="/var/lib/kubelet/pods/e3d7c09b-d632-44e9-b7b4-26c841d13f7f/volumes" Dec 03 12:59:51 crc kubenswrapper[5052]: I1203 12:59:51.693071 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="edd8e798-07f9-4ad1-bb97-8b64840c7a4e" path="/var/lib/kubelet/pods/edd8e798-07f9-4ad1-bb97-8b64840c7a4e/volumes" Dec 03 12:59:52 crc kubenswrapper[5052]: I1203 12:59:52.084837 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-x6wdj" event={"ID":"08ab2cda-3a42-49b5-843e-6c8b151064df","Type":"ContainerStarted","Data":"a310ad98499337b886702c931f5d7c2b6343449dfab7c440a0ecf22708504a26"} Dec 03 12:59:52 crc kubenswrapper[5052]: I1203 12:59:52.085948 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-x6wdj" Dec 03 12:59:52 crc kubenswrapper[5052]: I1203 12:59:52.113942 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-x6wdj" podStartSLOduration=3.113921063 podStartE2EDuration="3.113921063s" podCreationTimestamp="2025-12-03 12:59:49 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 12:59:52.113208283 +0000 UTC m=+5982.701450067" watchObservedRunningTime="2025-12-03 12:59:52.113921063 +0000 UTC m=+5982.702162837" Dec 03 12:59:52 crc kubenswrapper[5052]: I1203 12:59:52.673503 5052 scope.go:117] "RemoveContainer" containerID="69fa7fbcd2aceb7c53703a19508876f93a97dc9adabb79c16fb691d90f606d58" Dec 03 12:59:52 crc kubenswrapper[5052]: E1203 12:59:52.674201 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 12:59:56 crc kubenswrapper[5052]: I1203 12:59:56.749810 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openshift-marketplace/certified-operators-dgght"] Dec 03 12:59:56 crc kubenswrapper[5052]: I1203 12:59:56.751581 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/certified-operators-dgght" Dec 03 12:59:56 crc kubenswrapper[5052]: I1203 12:59:56.758890 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/certified-operators-dgght"] Dec 03 12:59:56 crc kubenswrapper[5052]: I1203 12:59:56.801626 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/e2b00b19-1d3b-4097-89cc-c37391b91db6-utilities\") pod \"certified-operators-dgght\" (UID: \"e2b00b19-1d3b-4097-89cc-c37391b91db6\") " pod="openshift-marketplace/certified-operators-dgght" Dec 03 12:59:56 crc kubenswrapper[5052]: I1203 12:59:56.801714 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/e2b00b19-1d3b-4097-89cc-c37391b91db6-catalog-content\") pod \"certified-operators-dgght\" (UID: \"e2b00b19-1d3b-4097-89cc-c37391b91db6\") " pod="openshift-marketplace/certified-operators-dgght" Dec 03 12:59:56 crc kubenswrapper[5052]: I1203 12:59:56.801805 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qk5kp\" (UniqueName: \"kubernetes.io/projected/e2b00b19-1d3b-4097-89cc-c37391b91db6-kube-api-access-qk5kp\") pod \"certified-operators-dgght\" (UID: \"e2b00b19-1d3b-4097-89cc-c37391b91db6\") " pod="openshift-marketplace/certified-operators-dgght" Dec 03 12:59:56 crc kubenswrapper[5052]: I1203 12:59:56.902648 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-qk5kp\" (UniqueName: \"kubernetes.io/projected/e2b00b19-1d3b-4097-89cc-c37391b91db6-kube-api-access-qk5kp\") pod \"certified-operators-dgght\" (UID: \"e2b00b19-1d3b-4097-89cc-c37391b91db6\") " pod="openshift-marketplace/certified-operators-dgght" Dec 03 12:59:56 crc kubenswrapper[5052]: I1203 12:59:56.902726 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/e2b00b19-1d3b-4097-89cc-c37391b91db6-utilities\") pod \"certified-operators-dgght\" (UID: \"e2b00b19-1d3b-4097-89cc-c37391b91db6\") " pod="openshift-marketplace/certified-operators-dgght" Dec 03 12:59:56 crc kubenswrapper[5052]: I1203 12:59:56.902763 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/e2b00b19-1d3b-4097-89cc-c37391b91db6-catalog-content\") pod \"certified-operators-dgght\" (UID: \"e2b00b19-1d3b-4097-89cc-c37391b91db6\") " pod="openshift-marketplace/certified-operators-dgght" Dec 03 12:59:56 crc kubenswrapper[5052]: I1203 12:59:56.903293 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/e2b00b19-1d3b-4097-89cc-c37391b91db6-catalog-content\") pod \"certified-operators-dgght\" (UID: \"e2b00b19-1d3b-4097-89cc-c37391b91db6\") " pod="openshift-marketplace/certified-operators-dgght" Dec 03 12:59:56 crc kubenswrapper[5052]: I1203 12:59:56.903426 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/e2b00b19-1d3b-4097-89cc-c37391b91db6-utilities\") pod \"certified-operators-dgght\" (UID: \"e2b00b19-1d3b-4097-89cc-c37391b91db6\") " pod="openshift-marketplace/certified-operators-dgght" Dec 03 12:59:56 crc kubenswrapper[5052]: I1203 12:59:56.922553 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-qk5kp\" (UniqueName: \"kubernetes.io/projected/e2b00b19-1d3b-4097-89cc-c37391b91db6-kube-api-access-qk5kp\") pod \"certified-operators-dgght\" (UID: \"e2b00b19-1d3b-4097-89cc-c37391b91db6\") " pod="openshift-marketplace/certified-operators-dgght" Dec 03 12:59:57 crc kubenswrapper[5052]: I1203 12:59:57.075119 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/certified-operators-dgght" Dec 03 12:59:57 crc kubenswrapper[5052]: I1203 12:59:57.616194 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/certified-operators-dgght"] Dec 03 12:59:57 crc kubenswrapper[5052]: I1203 12:59:57.711980 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["crc-storage/crc-storage-crc-mhzf5"] Dec 03 12:59:57 crc kubenswrapper[5052]: I1203 12:59:57.720570 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["crc-storage/crc-storage-crc-mhzf5"] Dec 03 12:59:57 crc kubenswrapper[5052]: I1203 12:59:57.854055 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["crc-storage/crc-storage-crc-w8658"] Dec 03 12:59:57 crc kubenswrapper[5052]: I1203 12:59:57.855867 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="crc-storage/crc-storage-crc-w8658" Dec 03 12:59:57 crc kubenswrapper[5052]: I1203 12:59:57.857856 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"crc-storage"/"crc-storage" Dec 03 12:59:57 crc kubenswrapper[5052]: I1203 12:59:57.858300 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"crc-storage"/"kube-root-ca.crt" Dec 03 12:59:57 crc kubenswrapper[5052]: I1203 12:59:57.858473 5052 reflector.go:368] Caches populated for *v1.Secret from object-"crc-storage"/"crc-storage-dockercfg-mf76g" Dec 03 12:59:57 crc kubenswrapper[5052]: I1203 12:59:57.858726 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"crc-storage"/"openshift-service-ca.crt" Dec 03 12:59:57 crc kubenswrapper[5052]: I1203 12:59:57.866496 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["crc-storage/crc-storage-crc-w8658"] Dec 03 12:59:58 crc kubenswrapper[5052]: I1203 12:59:58.021949 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/5a1963fc-eae2-430c-92af-15cf73e27373-crc-storage\") pod \"crc-storage-crc-w8658\" (UID: \"5a1963fc-eae2-430c-92af-15cf73e27373\") " pod="crc-storage/crc-storage-crc-w8658" Dec 03 12:59:58 crc kubenswrapper[5052]: I1203 12:59:58.022024 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/5a1963fc-eae2-430c-92af-15cf73e27373-node-mnt\") pod \"crc-storage-crc-w8658\" (UID: \"5a1963fc-eae2-430c-92af-15cf73e27373\") " pod="crc-storage/crc-storage-crc-w8658" Dec 03 12:59:58 crc kubenswrapper[5052]: I1203 12:59:58.022060 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-frfcq\" (UniqueName: \"kubernetes.io/projected/5a1963fc-eae2-430c-92af-15cf73e27373-kube-api-access-frfcq\") pod \"crc-storage-crc-w8658\" (UID: \"5a1963fc-eae2-430c-92af-15cf73e27373\") " pod="crc-storage/crc-storage-crc-w8658" Dec 03 12:59:58 crc kubenswrapper[5052]: I1203 12:59:58.123665 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/5a1963fc-eae2-430c-92af-15cf73e27373-crc-storage\") pod \"crc-storage-crc-w8658\" (UID: \"5a1963fc-eae2-430c-92af-15cf73e27373\") " pod="crc-storage/crc-storage-crc-w8658" Dec 03 12:59:58 crc kubenswrapper[5052]: I1203 12:59:58.123744 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/5a1963fc-eae2-430c-92af-15cf73e27373-node-mnt\") pod \"crc-storage-crc-w8658\" (UID: \"5a1963fc-eae2-430c-92af-15cf73e27373\") " pod="crc-storage/crc-storage-crc-w8658" Dec 03 12:59:58 crc kubenswrapper[5052]: I1203 12:59:58.123778 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-frfcq\" (UniqueName: \"kubernetes.io/projected/5a1963fc-eae2-430c-92af-15cf73e27373-kube-api-access-frfcq\") pod \"crc-storage-crc-w8658\" (UID: \"5a1963fc-eae2-430c-92af-15cf73e27373\") " pod="crc-storage/crc-storage-crc-w8658" Dec 03 12:59:58 crc kubenswrapper[5052]: I1203 12:59:58.124103 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/5a1963fc-eae2-430c-92af-15cf73e27373-node-mnt\") pod \"crc-storage-crc-w8658\" (UID: \"5a1963fc-eae2-430c-92af-15cf73e27373\") " pod="crc-storage/crc-storage-crc-w8658" Dec 03 12:59:58 crc kubenswrapper[5052]: I1203 12:59:58.124643 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/5a1963fc-eae2-430c-92af-15cf73e27373-crc-storage\") pod \"crc-storage-crc-w8658\" (UID: \"5a1963fc-eae2-430c-92af-15cf73e27373\") " pod="crc-storage/crc-storage-crc-w8658" Dec 03 12:59:58 crc kubenswrapper[5052]: I1203 12:59:58.139463 5052 generic.go:334] "Generic (PLEG): container finished" podID="e2b00b19-1d3b-4097-89cc-c37391b91db6" containerID="5e32971434c7553ce7ecc375b5d743d0dc43163a0f4a021146fba6b002b8ae03" exitCode=0 Dec 03 12:59:58 crc kubenswrapper[5052]: I1203 12:59:58.139504 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/certified-operators-dgght" event={"ID":"e2b00b19-1d3b-4097-89cc-c37391b91db6","Type":"ContainerDied","Data":"5e32971434c7553ce7ecc375b5d743d0dc43163a0f4a021146fba6b002b8ae03"} Dec 03 12:59:58 crc kubenswrapper[5052]: I1203 12:59:58.139545 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/certified-operators-dgght" event={"ID":"e2b00b19-1d3b-4097-89cc-c37391b91db6","Type":"ContainerStarted","Data":"68cbd3a1042668f024777ad2567e80c494d5ed325cdac3880bbb9eff28ec65f7"} Dec 03 12:59:58 crc kubenswrapper[5052]: I1203 12:59:58.143321 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-frfcq\" (UniqueName: \"kubernetes.io/projected/5a1963fc-eae2-430c-92af-15cf73e27373-kube-api-access-frfcq\") pod \"crc-storage-crc-w8658\" (UID: \"5a1963fc-eae2-430c-92af-15cf73e27373\") " pod="crc-storage/crc-storage-crc-w8658" Dec 03 12:59:58 crc kubenswrapper[5052]: I1203 12:59:58.188550 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="crc-storage/crc-storage-crc-w8658" Dec 03 12:59:58 crc kubenswrapper[5052]: I1203 12:59:58.609393 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["crc-storage/crc-storage-crc-w8658"] Dec 03 12:59:59 crc kubenswrapper[5052]: I1203 12:59:59.151858 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="crc-storage/crc-storage-crc-w8658" event={"ID":"5a1963fc-eae2-430c-92af-15cf73e27373","Type":"ContainerStarted","Data":"e629b889e113a3a1c6e9699cdf7945042a4572505342a5c326eaf68f5ab7965a"} Dec 03 12:59:59 crc kubenswrapper[5052]: I1203 12:59:59.688180 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="87727c3f-e310-47a7-a7dc-eb96211b6208" path="/var/lib/kubelet/pods/87727c3f-e310-47a7-a7dc-eb96211b6208/volumes" Dec 03 13:00:00 crc kubenswrapper[5052]: I1203 13:00:00.142607 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openshift-operator-lifecycle-manager/collect-profiles-29412780-k6h9t"] Dec 03 13:00:00 crc kubenswrapper[5052]: I1203 13:00:00.143854 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-operator-lifecycle-manager/collect-profiles-29412780-k6h9t" Dec 03 13:00:00 crc kubenswrapper[5052]: I1203 13:00:00.146361 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-operator-lifecycle-manager"/"collect-profiles-config" Dec 03 13:00:00 crc kubenswrapper[5052]: I1203 13:00:00.146477 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-operator-lifecycle-manager"/"collect-profiles-dockercfg-kzf4t" Dec 03 13:00:00 crc kubenswrapper[5052]: I1203 13:00:00.151408 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-operator-lifecycle-manager/collect-profiles-29412780-k6h9t"] Dec 03 13:00:00 crc kubenswrapper[5052]: I1203 13:00:00.187337 5052 generic.go:334] "Generic (PLEG): container finished" podID="e2b00b19-1d3b-4097-89cc-c37391b91db6" containerID="3ac23fc0406be9a15fb72d13e4f96186af91611322b7cf90e9a443f635960007" exitCode=0 Dec 03 13:00:00 crc kubenswrapper[5052]: I1203 13:00:00.188103 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/certified-operators-dgght" event={"ID":"e2b00b19-1d3b-4097-89cc-c37391b91db6","Type":"ContainerDied","Data":"3ac23fc0406be9a15fb72d13e4f96186af91611322b7cf90e9a443f635960007"} Dec 03 13:00:00 crc kubenswrapper[5052]: I1203 13:00:00.189947 5052 generic.go:334] "Generic (PLEG): container finished" podID="5a1963fc-eae2-430c-92af-15cf73e27373" containerID="03433a1bf72d3b1f122b0c525ee8d41ba085889d0887543f5c99e28ea7edcd7e" exitCode=0 Dec 03 13:00:00 crc kubenswrapper[5052]: I1203 13:00:00.189978 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="crc-storage/crc-storage-crc-w8658" event={"ID":"5a1963fc-eae2-430c-92af-15cf73e27373","Type":"ContainerDied","Data":"03433a1bf72d3b1f122b0c525ee8d41ba085889d0887543f5c99e28ea7edcd7e"} Dec 03 13:00:00 crc kubenswrapper[5052]: I1203 13:00:00.208696 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-x6wdj" Dec 03 13:00:00 crc kubenswrapper[5052]: I1203 13:00:00.259138 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-volume\" (UniqueName: \"kubernetes.io/secret/f3de51af-ca99-42fa-a358-35184a5f1726-secret-volume\") pod \"collect-profiles-29412780-k6h9t\" (UID: \"f3de51af-ca99-42fa-a358-35184a5f1726\") " pod="openshift-operator-lifecycle-manager/collect-profiles-29412780-k6h9t" Dec 03 13:00:00 crc kubenswrapper[5052]: I1203 13:00:00.259187 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f3de51af-ca99-42fa-a358-35184a5f1726-config-volume\") pod \"collect-profiles-29412780-k6h9t\" (UID: \"f3de51af-ca99-42fa-a358-35184a5f1726\") " pod="openshift-operator-lifecycle-manager/collect-profiles-29412780-k6h9t" Dec 03 13:00:00 crc kubenswrapper[5052]: I1203 13:00:00.259219 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jr4ck\" (UniqueName: \"kubernetes.io/projected/f3de51af-ca99-42fa-a358-35184a5f1726-kube-api-access-jr4ck\") pod \"collect-profiles-29412780-k6h9t\" (UID: \"f3de51af-ca99-42fa-a358-35184a5f1726\") " pod="openshift-operator-lifecycle-manager/collect-profiles-29412780-k6h9t" Dec 03 13:00:00 crc kubenswrapper[5052]: I1203 13:00:00.271446 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-6668544499-v8zl2"] Dec 03 13:00:00 crc kubenswrapper[5052]: I1203 13:00:00.271936 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-6668544499-v8zl2" podUID="65b17e9c-5f92-4aa7-96f3-551e062422c0" containerName="dnsmasq-dns" containerID="cri-o://57b2bbf195b6e0e32c45017d766ddec938cf76f66e290154f46f5117d33bf1ac" gracePeriod=10 Dec 03 13:00:00 crc kubenswrapper[5052]: I1203 13:00:00.360202 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"secret-volume\" (UniqueName: \"kubernetes.io/secret/f3de51af-ca99-42fa-a358-35184a5f1726-secret-volume\") pod \"collect-profiles-29412780-k6h9t\" (UID: \"f3de51af-ca99-42fa-a358-35184a5f1726\") " pod="openshift-operator-lifecycle-manager/collect-profiles-29412780-k6h9t" Dec 03 13:00:00 crc kubenswrapper[5052]: I1203 13:00:00.360260 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f3de51af-ca99-42fa-a358-35184a5f1726-config-volume\") pod \"collect-profiles-29412780-k6h9t\" (UID: \"f3de51af-ca99-42fa-a358-35184a5f1726\") " pod="openshift-operator-lifecycle-manager/collect-profiles-29412780-k6h9t" Dec 03 13:00:00 crc kubenswrapper[5052]: I1203 13:00:00.360305 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-jr4ck\" (UniqueName: \"kubernetes.io/projected/f3de51af-ca99-42fa-a358-35184a5f1726-kube-api-access-jr4ck\") pod \"collect-profiles-29412780-k6h9t\" (UID: \"f3de51af-ca99-42fa-a358-35184a5f1726\") " pod="openshift-operator-lifecycle-manager/collect-profiles-29412780-k6h9t" Dec 03 13:00:00 crc kubenswrapper[5052]: I1203 13:00:00.361622 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f3de51af-ca99-42fa-a358-35184a5f1726-config-volume\") pod \"collect-profiles-29412780-k6h9t\" (UID: \"f3de51af-ca99-42fa-a358-35184a5f1726\") " pod="openshift-operator-lifecycle-manager/collect-profiles-29412780-k6h9t" Dec 03 13:00:00 crc kubenswrapper[5052]: I1203 13:00:00.373354 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"secret-volume\" (UniqueName: \"kubernetes.io/secret/f3de51af-ca99-42fa-a358-35184a5f1726-secret-volume\") pod \"collect-profiles-29412780-k6h9t\" (UID: \"f3de51af-ca99-42fa-a358-35184a5f1726\") " pod="openshift-operator-lifecycle-manager/collect-profiles-29412780-k6h9t" Dec 03 13:00:00 crc kubenswrapper[5052]: I1203 13:00:00.378268 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-jr4ck\" (UniqueName: \"kubernetes.io/projected/f3de51af-ca99-42fa-a358-35184a5f1726-kube-api-access-jr4ck\") pod \"collect-profiles-29412780-k6h9t\" (UID: \"f3de51af-ca99-42fa-a358-35184a5f1726\") " pod="openshift-operator-lifecycle-manager/collect-profiles-29412780-k6h9t" Dec 03 13:00:00 crc kubenswrapper[5052]: I1203 13:00:00.481197 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-operator-lifecycle-manager/collect-profiles-29412780-k6h9t" Dec 03 13:00:00 crc kubenswrapper[5052]: I1203 13:00:00.721421 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-6668544499-v8zl2" Dec 03 13:00:01 crc kubenswrapper[5052]: I1203 13:00:00.867560 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"edpm-compute-global\" (UniqueName: \"kubernetes.io/configmap/65b17e9c-5f92-4aa7-96f3-551e062422c0-edpm-compute-global\") pod \"65b17e9c-5f92-4aa7-96f3-551e062422c0\" (UID: \"65b17e9c-5f92-4aa7-96f3-551e062422c0\") " Dec 03 13:00:01 crc kubenswrapper[5052]: I1203 13:00:00.867652 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g5vnv\" (UniqueName: \"kubernetes.io/projected/65b17e9c-5f92-4aa7-96f3-551e062422c0-kube-api-access-g5vnv\") pod \"65b17e9c-5f92-4aa7-96f3-551e062422c0\" (UID: \"65b17e9c-5f92-4aa7-96f3-551e062422c0\") " Dec 03 13:00:01 crc kubenswrapper[5052]: I1203 13:00:00.867694 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/65b17e9c-5f92-4aa7-96f3-551e062422c0-dnsmasq-svc\") pod \"65b17e9c-5f92-4aa7-96f3-551e062422c0\" (UID: \"65b17e9c-5f92-4aa7-96f3-551e062422c0\") " Dec 03 13:00:01 crc kubenswrapper[5052]: I1203 13:00:00.867782 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"edpm-compute-beta-nodeset\" (UniqueName: \"kubernetes.io/configmap/65b17e9c-5f92-4aa7-96f3-551e062422c0-edpm-compute-beta-nodeset\") pod \"65b17e9c-5f92-4aa7-96f3-551e062422c0\" (UID: \"65b17e9c-5f92-4aa7-96f3-551e062422c0\") " Dec 03 13:00:01 crc kubenswrapper[5052]: I1203 13:00:00.867848 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/65b17e9c-5f92-4aa7-96f3-551e062422c0-config\") pod \"65b17e9c-5f92-4aa7-96f3-551e062422c0\" (UID: \"65b17e9c-5f92-4aa7-96f3-551e062422c0\") " Dec 03 13:00:01 crc kubenswrapper[5052]: I1203 13:00:00.872705 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/65b17e9c-5f92-4aa7-96f3-551e062422c0-kube-api-access-g5vnv" (OuterVolumeSpecName: "kube-api-access-g5vnv") pod "65b17e9c-5f92-4aa7-96f3-551e062422c0" (UID: "65b17e9c-5f92-4aa7-96f3-551e062422c0"). InnerVolumeSpecName "kube-api-access-g5vnv". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:00:01 crc kubenswrapper[5052]: I1203 13:00:00.906636 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/65b17e9c-5f92-4aa7-96f3-551e062422c0-edpm-compute-global" (OuterVolumeSpecName: "edpm-compute-global") pod "65b17e9c-5f92-4aa7-96f3-551e062422c0" (UID: "65b17e9c-5f92-4aa7-96f3-551e062422c0"). InnerVolumeSpecName "edpm-compute-global". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:00:01 crc kubenswrapper[5052]: I1203 13:00:00.910766 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/65b17e9c-5f92-4aa7-96f3-551e062422c0-dnsmasq-svc" (OuterVolumeSpecName: "dnsmasq-svc") pod "65b17e9c-5f92-4aa7-96f3-551e062422c0" (UID: "65b17e9c-5f92-4aa7-96f3-551e062422c0"). InnerVolumeSpecName "dnsmasq-svc". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:00:01 crc kubenswrapper[5052]: I1203 13:00:00.911303 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/65b17e9c-5f92-4aa7-96f3-551e062422c0-config" (OuterVolumeSpecName: "config") pod "65b17e9c-5f92-4aa7-96f3-551e062422c0" (UID: "65b17e9c-5f92-4aa7-96f3-551e062422c0"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:00:01 crc kubenswrapper[5052]: I1203 13:00:00.913124 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/65b17e9c-5f92-4aa7-96f3-551e062422c0-edpm-compute-beta-nodeset" (OuterVolumeSpecName: "edpm-compute-beta-nodeset") pod "65b17e9c-5f92-4aa7-96f3-551e062422c0" (UID: "65b17e9c-5f92-4aa7-96f3-551e062422c0"). InnerVolumeSpecName "edpm-compute-beta-nodeset". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:00:01 crc kubenswrapper[5052]: I1203 13:00:00.973554 5052 reconciler_common.go:293] "Volume detached for volume \"edpm-compute-beta-nodeset\" (UniqueName: \"kubernetes.io/configmap/65b17e9c-5f92-4aa7-96f3-551e062422c0-edpm-compute-beta-nodeset\") on node \"crc\" DevicePath \"\"" Dec 03 13:00:01 crc kubenswrapper[5052]: I1203 13:00:00.973593 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/65b17e9c-5f92-4aa7-96f3-551e062422c0-config\") on node \"crc\" DevicePath \"\"" Dec 03 13:00:01 crc kubenswrapper[5052]: I1203 13:00:00.973605 5052 reconciler_common.go:293] "Volume detached for volume \"edpm-compute-global\" (UniqueName: \"kubernetes.io/configmap/65b17e9c-5f92-4aa7-96f3-551e062422c0-edpm-compute-global\") on node \"crc\" DevicePath \"\"" Dec 03 13:00:01 crc kubenswrapper[5052]: I1203 13:00:00.973624 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-g5vnv\" (UniqueName: \"kubernetes.io/projected/65b17e9c-5f92-4aa7-96f3-551e062422c0-kube-api-access-g5vnv\") on node \"crc\" DevicePath \"\"" Dec 03 13:00:01 crc kubenswrapper[5052]: I1203 13:00:00.973636 5052 reconciler_common.go:293] "Volume detached for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/65b17e9c-5f92-4aa7-96f3-551e062422c0-dnsmasq-svc\") on node \"crc\" DevicePath \"\"" Dec 03 13:00:01 crc kubenswrapper[5052]: I1203 13:00:00.974130 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-operator-lifecycle-manager/collect-profiles-29412780-k6h9t"] Dec 03 13:00:01 crc kubenswrapper[5052]: I1203 13:00:01.204081 5052 generic.go:334] "Generic (PLEG): container finished" podID="65b17e9c-5f92-4aa7-96f3-551e062422c0" containerID="57b2bbf195b6e0e32c45017d766ddec938cf76f66e290154f46f5117d33bf1ac" exitCode=0 Dec 03 13:00:01 crc kubenswrapper[5052]: I1203 13:00:01.204201 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-6668544499-v8zl2" Dec 03 13:00:01 crc kubenswrapper[5052]: I1203 13:00:01.204225 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-6668544499-v8zl2" event={"ID":"65b17e9c-5f92-4aa7-96f3-551e062422c0","Type":"ContainerDied","Data":"57b2bbf195b6e0e32c45017d766ddec938cf76f66e290154f46f5117d33bf1ac"} Dec 03 13:00:01 crc kubenswrapper[5052]: I1203 13:00:01.204286 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-6668544499-v8zl2" event={"ID":"65b17e9c-5f92-4aa7-96f3-551e062422c0","Type":"ContainerDied","Data":"bc3a37a15d42c8b4e27ed24ccaf8f912294dc73effb2c6589078f541c435a777"} Dec 03 13:00:01 crc kubenswrapper[5052]: I1203 13:00:01.204305 5052 scope.go:117] "RemoveContainer" containerID="57b2bbf195b6e0e32c45017d766ddec938cf76f66e290154f46f5117d33bf1ac" Dec 03 13:00:01 crc kubenswrapper[5052]: I1203 13:00:01.209486 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/certified-operators-dgght" event={"ID":"e2b00b19-1d3b-4097-89cc-c37391b91db6","Type":"ContainerStarted","Data":"1ebef0065a44d14ba4c3d7cd887592081f8ee24a7688f2546e470ed1df0cdd4e"} Dec 03 13:00:01 crc kubenswrapper[5052]: I1203 13:00:01.215100 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-operator-lifecycle-manager/collect-profiles-29412780-k6h9t" event={"ID":"f3de51af-ca99-42fa-a358-35184a5f1726","Type":"ContainerStarted","Data":"e1e470c082b1a026c1c9a2db17c495f449446b66789eaadeb81061b95f5065de"} Dec 03 13:00:01 crc kubenswrapper[5052]: I1203 13:00:01.233767 5052 scope.go:117] "RemoveContainer" containerID="773141ca6b74a4d0763489d6ed12290cd36d743efa51afb643252ff6550cf79d" Dec 03 13:00:01 crc kubenswrapper[5052]: I1203 13:00:01.249303 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-marketplace/certified-operators-dgght" podStartSLOduration=2.779237128 podStartE2EDuration="5.249278397s" podCreationTimestamp="2025-12-03 12:59:56 +0000 UTC" firstStartedPulling="2025-12-03 12:59:58.142195151 +0000 UTC m=+5988.730436915" lastFinishedPulling="2025-12-03 13:00:00.61223642 +0000 UTC m=+5991.200478184" observedRunningTime="2025-12-03 13:00:01.239060565 +0000 UTC m=+5991.827302349" watchObservedRunningTime="2025-12-03 13:00:01.249278397 +0000 UTC m=+5991.837520161" Dec 03 13:00:01 crc kubenswrapper[5052]: I1203 13:00:01.265038 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-6668544499-v8zl2"] Dec 03 13:00:01 crc kubenswrapper[5052]: I1203 13:00:01.268786 5052 scope.go:117] "RemoveContainer" containerID="57b2bbf195b6e0e32c45017d766ddec938cf76f66e290154f46f5117d33bf1ac" Dec 03 13:00:01 crc kubenswrapper[5052]: E1203 13:00:01.269399 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"57b2bbf195b6e0e32c45017d766ddec938cf76f66e290154f46f5117d33bf1ac\": container with ID starting with 57b2bbf195b6e0e32c45017d766ddec938cf76f66e290154f46f5117d33bf1ac not found: ID does not exist" containerID="57b2bbf195b6e0e32c45017d766ddec938cf76f66e290154f46f5117d33bf1ac" Dec 03 13:00:01 crc kubenswrapper[5052]: I1203 13:00:01.269455 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"57b2bbf195b6e0e32c45017d766ddec938cf76f66e290154f46f5117d33bf1ac"} err="failed to get container status \"57b2bbf195b6e0e32c45017d766ddec938cf76f66e290154f46f5117d33bf1ac\": rpc error: code = NotFound desc = could not find container \"57b2bbf195b6e0e32c45017d766ddec938cf76f66e290154f46f5117d33bf1ac\": container with ID starting with 57b2bbf195b6e0e32c45017d766ddec938cf76f66e290154f46f5117d33bf1ac not found: ID does not exist" Dec 03 13:00:01 crc kubenswrapper[5052]: I1203 13:00:01.270024 5052 scope.go:117] "RemoveContainer" containerID="773141ca6b74a4d0763489d6ed12290cd36d743efa51afb643252ff6550cf79d" Dec 03 13:00:01 crc kubenswrapper[5052]: E1203 13:00:01.270553 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"773141ca6b74a4d0763489d6ed12290cd36d743efa51afb643252ff6550cf79d\": container with ID starting with 773141ca6b74a4d0763489d6ed12290cd36d743efa51afb643252ff6550cf79d not found: ID does not exist" containerID="773141ca6b74a4d0763489d6ed12290cd36d743efa51afb643252ff6550cf79d" Dec 03 13:00:01 crc kubenswrapper[5052]: I1203 13:00:01.270577 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"773141ca6b74a4d0763489d6ed12290cd36d743efa51afb643252ff6550cf79d"} err="failed to get container status \"773141ca6b74a4d0763489d6ed12290cd36d743efa51afb643252ff6550cf79d\": rpc error: code = NotFound desc = could not find container \"773141ca6b74a4d0763489d6ed12290cd36d743efa51afb643252ff6550cf79d\": container with ID starting with 773141ca6b74a4d0763489d6ed12290cd36d743efa51afb643252ff6550cf79d not found: ID does not exist" Dec 03 13:00:01 crc kubenswrapper[5052]: I1203 13:00:01.276618 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-6668544499-v8zl2"] Dec 03 13:00:01 crc kubenswrapper[5052]: I1203 13:00:01.686495 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="65b17e9c-5f92-4aa7-96f3-551e062422c0" path="/var/lib/kubelet/pods/65b17e9c-5f92-4aa7-96f3-551e062422c0/volumes" Dec 03 13:00:01 crc kubenswrapper[5052]: I1203 13:00:01.724871 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="crc-storage/crc-storage-crc-w8658" Dec 03 13:00:01 crc kubenswrapper[5052]: I1203 13:00:01.888458 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/5a1963fc-eae2-430c-92af-15cf73e27373-crc-storage\") pod \"5a1963fc-eae2-430c-92af-15cf73e27373\" (UID: \"5a1963fc-eae2-430c-92af-15cf73e27373\") " Dec 03 13:00:01 crc kubenswrapper[5052]: I1203 13:00:01.888533 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-frfcq\" (UniqueName: \"kubernetes.io/projected/5a1963fc-eae2-430c-92af-15cf73e27373-kube-api-access-frfcq\") pod \"5a1963fc-eae2-430c-92af-15cf73e27373\" (UID: \"5a1963fc-eae2-430c-92af-15cf73e27373\") " Dec 03 13:00:01 crc kubenswrapper[5052]: I1203 13:00:01.888740 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/5a1963fc-eae2-430c-92af-15cf73e27373-node-mnt\") pod \"5a1963fc-eae2-430c-92af-15cf73e27373\" (UID: \"5a1963fc-eae2-430c-92af-15cf73e27373\") " Dec 03 13:00:01 crc kubenswrapper[5052]: I1203 13:00:01.888843 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/5a1963fc-eae2-430c-92af-15cf73e27373-node-mnt" (OuterVolumeSpecName: "node-mnt") pod "5a1963fc-eae2-430c-92af-15cf73e27373" (UID: "5a1963fc-eae2-430c-92af-15cf73e27373"). InnerVolumeSpecName "node-mnt". PluginName "kubernetes.io/host-path", VolumeGidValue "" Dec 03 13:00:01 crc kubenswrapper[5052]: I1203 13:00:01.889327 5052 reconciler_common.go:293] "Volume detached for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/5a1963fc-eae2-430c-92af-15cf73e27373-node-mnt\") on node \"crc\" DevicePath \"\"" Dec 03 13:00:01 crc kubenswrapper[5052]: I1203 13:00:01.895953 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5a1963fc-eae2-430c-92af-15cf73e27373-kube-api-access-frfcq" (OuterVolumeSpecName: "kube-api-access-frfcq") pod "5a1963fc-eae2-430c-92af-15cf73e27373" (UID: "5a1963fc-eae2-430c-92af-15cf73e27373"). InnerVolumeSpecName "kube-api-access-frfcq". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:00:01 crc kubenswrapper[5052]: I1203 13:00:01.908371 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5a1963fc-eae2-430c-92af-15cf73e27373-crc-storage" (OuterVolumeSpecName: "crc-storage") pod "5a1963fc-eae2-430c-92af-15cf73e27373" (UID: "5a1963fc-eae2-430c-92af-15cf73e27373"). InnerVolumeSpecName "crc-storage". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:00:01 crc kubenswrapper[5052]: I1203 13:00:01.990610 5052 reconciler_common.go:293] "Volume detached for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/5a1963fc-eae2-430c-92af-15cf73e27373-crc-storage\") on node \"crc\" DevicePath \"\"" Dec 03 13:00:01 crc kubenswrapper[5052]: I1203 13:00:01.990664 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-frfcq\" (UniqueName: \"kubernetes.io/projected/5a1963fc-eae2-430c-92af-15cf73e27373-kube-api-access-frfcq\") on node \"crc\" DevicePath \"\"" Dec 03 13:00:02 crc kubenswrapper[5052]: I1203 13:00:02.225872 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="crc-storage/crc-storage-crc-w8658" Dec 03 13:00:02 crc kubenswrapper[5052]: I1203 13:00:02.225867 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="crc-storage/crc-storage-crc-w8658" event={"ID":"5a1963fc-eae2-430c-92af-15cf73e27373","Type":"ContainerDied","Data":"e629b889e113a3a1c6e9699cdf7945042a4572505342a5c326eaf68f5ab7965a"} Dec 03 13:00:02 crc kubenswrapper[5052]: I1203 13:00:02.225996 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e629b889e113a3a1c6e9699cdf7945042a4572505342a5c326eaf68f5ab7965a" Dec 03 13:00:02 crc kubenswrapper[5052]: I1203 13:00:02.227757 5052 generic.go:334] "Generic (PLEG): container finished" podID="f3de51af-ca99-42fa-a358-35184a5f1726" containerID="f2e21c2232a9ccb381f0e9b579b8dff5ffc898fae2db699e39b929ba65fa5db0" exitCode=0 Dec 03 13:00:02 crc kubenswrapper[5052]: I1203 13:00:02.227907 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-operator-lifecycle-manager/collect-profiles-29412780-k6h9t" event={"ID":"f3de51af-ca99-42fa-a358-35184a5f1726","Type":"ContainerDied","Data":"f2e21c2232a9ccb381f0e9b579b8dff5ffc898fae2db699e39b929ba65fa5db0"} Dec 03 13:00:03 crc kubenswrapper[5052]: I1203 13:00:03.507381 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-operator-lifecycle-manager/collect-profiles-29412780-k6h9t" Dec 03 13:00:03 crc kubenswrapper[5052]: I1203 13:00:03.613754 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f3de51af-ca99-42fa-a358-35184a5f1726-config-volume\") pod \"f3de51af-ca99-42fa-a358-35184a5f1726\" (UID: \"f3de51af-ca99-42fa-a358-35184a5f1726\") " Dec 03 13:00:03 crc kubenswrapper[5052]: I1203 13:00:03.613879 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"secret-volume\" (UniqueName: \"kubernetes.io/secret/f3de51af-ca99-42fa-a358-35184a5f1726-secret-volume\") pod \"f3de51af-ca99-42fa-a358-35184a5f1726\" (UID: \"f3de51af-ca99-42fa-a358-35184a5f1726\") " Dec 03 13:00:03 crc kubenswrapper[5052]: I1203 13:00:03.613921 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jr4ck\" (UniqueName: \"kubernetes.io/projected/f3de51af-ca99-42fa-a358-35184a5f1726-kube-api-access-jr4ck\") pod \"f3de51af-ca99-42fa-a358-35184a5f1726\" (UID: \"f3de51af-ca99-42fa-a358-35184a5f1726\") " Dec 03 13:00:03 crc kubenswrapper[5052]: I1203 13:00:03.614788 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f3de51af-ca99-42fa-a358-35184a5f1726-config-volume" (OuterVolumeSpecName: "config-volume") pod "f3de51af-ca99-42fa-a358-35184a5f1726" (UID: "f3de51af-ca99-42fa-a358-35184a5f1726"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:00:03 crc kubenswrapper[5052]: I1203 13:00:03.617728 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f3de51af-ca99-42fa-a358-35184a5f1726-secret-volume" (OuterVolumeSpecName: "secret-volume") pod "f3de51af-ca99-42fa-a358-35184a5f1726" (UID: "f3de51af-ca99-42fa-a358-35184a5f1726"). InnerVolumeSpecName "secret-volume". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:00:03 crc kubenswrapper[5052]: I1203 13:00:03.618565 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f3de51af-ca99-42fa-a358-35184a5f1726-kube-api-access-jr4ck" (OuterVolumeSpecName: "kube-api-access-jr4ck") pod "f3de51af-ca99-42fa-a358-35184a5f1726" (UID: "f3de51af-ca99-42fa-a358-35184a5f1726"). InnerVolumeSpecName "kube-api-access-jr4ck". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:00:03 crc kubenswrapper[5052]: I1203 13:00:03.715883 5052 reconciler_common.go:293] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f3de51af-ca99-42fa-a358-35184a5f1726-config-volume\") on node \"crc\" DevicePath \"\"" Dec 03 13:00:03 crc kubenswrapper[5052]: I1203 13:00:03.715923 5052 reconciler_common.go:293] "Volume detached for volume \"secret-volume\" (UniqueName: \"kubernetes.io/secret/f3de51af-ca99-42fa-a358-35184a5f1726-secret-volume\") on node \"crc\" DevicePath \"\"" Dec 03 13:00:03 crc kubenswrapper[5052]: I1203 13:00:03.715939 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-jr4ck\" (UniqueName: \"kubernetes.io/projected/f3de51af-ca99-42fa-a358-35184a5f1726-kube-api-access-jr4ck\") on node \"crc\" DevicePath \"\"" Dec 03 13:00:04 crc kubenswrapper[5052]: I1203 13:00:04.246235 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-operator-lifecycle-manager/collect-profiles-29412780-k6h9t" event={"ID":"f3de51af-ca99-42fa-a358-35184a5f1726","Type":"ContainerDied","Data":"e1e470c082b1a026c1c9a2db17c495f449446b66789eaadeb81061b95f5065de"} Dec 03 13:00:04 crc kubenswrapper[5052]: I1203 13:00:04.246273 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e1e470c082b1a026c1c9a2db17c495f449446b66789eaadeb81061b95f5065de" Dec 03 13:00:04 crc kubenswrapper[5052]: I1203 13:00:04.246311 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-operator-lifecycle-manager/collect-profiles-29412780-k6h9t" Dec 03 13:00:04 crc kubenswrapper[5052]: I1203 13:00:04.570943 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openshift-operator-lifecycle-manager/collect-profiles-29412735-8kbb9"] Dec 03 13:00:04 crc kubenswrapper[5052]: I1203 13:00:04.576146 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openshift-operator-lifecycle-manager/collect-profiles-29412735-8kbb9"] Dec 03 13:00:05 crc kubenswrapper[5052]: I1203 13:00:05.306108 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["crc-storage/crc-storage-crc-w8658"] Dec 03 13:00:05 crc kubenswrapper[5052]: I1203 13:00:05.312847 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["crc-storage/crc-storage-crc-w8658"] Dec 03 13:00:05 crc kubenswrapper[5052]: I1203 13:00:05.462275 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["crc-storage/crc-storage-crc-6gwst"] Dec 03 13:00:05 crc kubenswrapper[5052]: E1203 13:00:05.462918 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="65b17e9c-5f92-4aa7-96f3-551e062422c0" containerName="dnsmasq-dns" Dec 03 13:00:05 crc kubenswrapper[5052]: I1203 13:00:05.463001 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="65b17e9c-5f92-4aa7-96f3-551e062422c0" containerName="dnsmasq-dns" Dec 03 13:00:05 crc kubenswrapper[5052]: E1203 13:00:05.463080 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="65b17e9c-5f92-4aa7-96f3-551e062422c0" containerName="init" Dec 03 13:00:05 crc kubenswrapper[5052]: I1203 13:00:05.463138 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="65b17e9c-5f92-4aa7-96f3-551e062422c0" containerName="init" Dec 03 13:00:05 crc kubenswrapper[5052]: E1203 13:00:05.463234 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="5a1963fc-eae2-430c-92af-15cf73e27373" containerName="storage" Dec 03 13:00:05 crc kubenswrapper[5052]: I1203 13:00:05.463300 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="5a1963fc-eae2-430c-92af-15cf73e27373" containerName="storage" Dec 03 13:00:05 crc kubenswrapper[5052]: E1203 13:00:05.463371 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="f3de51af-ca99-42fa-a358-35184a5f1726" containerName="collect-profiles" Dec 03 13:00:05 crc kubenswrapper[5052]: I1203 13:00:05.463428 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="f3de51af-ca99-42fa-a358-35184a5f1726" containerName="collect-profiles" Dec 03 13:00:05 crc kubenswrapper[5052]: I1203 13:00:05.463655 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="5a1963fc-eae2-430c-92af-15cf73e27373" containerName="storage" Dec 03 13:00:05 crc kubenswrapper[5052]: I1203 13:00:05.463736 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="65b17e9c-5f92-4aa7-96f3-551e062422c0" containerName="dnsmasq-dns" Dec 03 13:00:05 crc kubenswrapper[5052]: I1203 13:00:05.463811 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="f3de51af-ca99-42fa-a358-35184a5f1726" containerName="collect-profiles" Dec 03 13:00:05 crc kubenswrapper[5052]: I1203 13:00:05.464483 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="crc-storage/crc-storage-crc-6gwst" Dec 03 13:00:05 crc kubenswrapper[5052]: I1203 13:00:05.473849 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"crc-storage"/"openshift-service-ca.crt" Dec 03 13:00:05 crc kubenswrapper[5052]: I1203 13:00:05.474057 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"crc-storage"/"kube-root-ca.crt" Dec 03 13:00:05 crc kubenswrapper[5052]: I1203 13:00:05.474178 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"crc-storage"/"crc-storage" Dec 03 13:00:05 crc kubenswrapper[5052]: I1203 13:00:05.477209 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["crc-storage/crc-storage-crc-6gwst"] Dec 03 13:00:05 crc kubenswrapper[5052]: I1203 13:00:05.481290 5052 reflector.go:368] Caches populated for *v1.Secret from object-"crc-storage"/"crc-storage-dockercfg-mf76g" Dec 03 13:00:05 crc kubenswrapper[5052]: I1203 13:00:05.647345 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/583f9858-b0cd-4976-a547-32ba02cbc85b-crc-storage\") pod \"crc-storage-crc-6gwst\" (UID: \"583f9858-b0cd-4976-a547-32ba02cbc85b\") " pod="crc-storage/crc-storage-crc-6gwst" Dec 03 13:00:05 crc kubenswrapper[5052]: I1203 13:00:05.647413 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/583f9858-b0cd-4976-a547-32ba02cbc85b-node-mnt\") pod \"crc-storage-crc-6gwst\" (UID: \"583f9858-b0cd-4976-a547-32ba02cbc85b\") " pod="crc-storage/crc-storage-crc-6gwst" Dec 03 13:00:05 crc kubenswrapper[5052]: I1203 13:00:05.647446 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k4vm2\" (UniqueName: \"kubernetes.io/projected/583f9858-b0cd-4976-a547-32ba02cbc85b-kube-api-access-k4vm2\") pod \"crc-storage-crc-6gwst\" (UID: \"583f9858-b0cd-4976-a547-32ba02cbc85b\") " pod="crc-storage/crc-storage-crc-6gwst" Dec 03 13:00:05 crc kubenswrapper[5052]: I1203 13:00:05.682847 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5a1963fc-eae2-430c-92af-15cf73e27373" path="/var/lib/kubelet/pods/5a1963fc-eae2-430c-92af-15cf73e27373/volumes" Dec 03 13:00:05 crc kubenswrapper[5052]: I1203 13:00:05.683669 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f792d620-f1d1-472d-a013-9f1ae6930b0f" path="/var/lib/kubelet/pods/f792d620-f1d1-472d-a013-9f1ae6930b0f/volumes" Dec 03 13:00:05 crc kubenswrapper[5052]: I1203 13:00:05.748481 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/583f9858-b0cd-4976-a547-32ba02cbc85b-crc-storage\") pod \"crc-storage-crc-6gwst\" (UID: \"583f9858-b0cd-4976-a547-32ba02cbc85b\") " pod="crc-storage/crc-storage-crc-6gwst" Dec 03 13:00:05 crc kubenswrapper[5052]: I1203 13:00:05.748549 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/583f9858-b0cd-4976-a547-32ba02cbc85b-node-mnt\") pod \"crc-storage-crc-6gwst\" (UID: \"583f9858-b0cd-4976-a547-32ba02cbc85b\") " pod="crc-storage/crc-storage-crc-6gwst" Dec 03 13:00:05 crc kubenswrapper[5052]: I1203 13:00:05.748593 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-k4vm2\" (UniqueName: \"kubernetes.io/projected/583f9858-b0cd-4976-a547-32ba02cbc85b-kube-api-access-k4vm2\") pod \"crc-storage-crc-6gwst\" (UID: \"583f9858-b0cd-4976-a547-32ba02cbc85b\") " pod="crc-storage/crc-storage-crc-6gwst" Dec 03 13:00:05 crc kubenswrapper[5052]: I1203 13:00:05.748930 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/583f9858-b0cd-4976-a547-32ba02cbc85b-node-mnt\") pod \"crc-storage-crc-6gwst\" (UID: \"583f9858-b0cd-4976-a547-32ba02cbc85b\") " pod="crc-storage/crc-storage-crc-6gwst" Dec 03 13:00:05 crc kubenswrapper[5052]: I1203 13:00:05.749255 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/583f9858-b0cd-4976-a547-32ba02cbc85b-crc-storage\") pod \"crc-storage-crc-6gwst\" (UID: \"583f9858-b0cd-4976-a547-32ba02cbc85b\") " pod="crc-storage/crc-storage-crc-6gwst" Dec 03 13:00:05 crc kubenswrapper[5052]: I1203 13:00:05.770116 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-k4vm2\" (UniqueName: \"kubernetes.io/projected/583f9858-b0cd-4976-a547-32ba02cbc85b-kube-api-access-k4vm2\") pod \"crc-storage-crc-6gwst\" (UID: \"583f9858-b0cd-4976-a547-32ba02cbc85b\") " pod="crc-storage/crc-storage-crc-6gwst" Dec 03 13:00:05 crc kubenswrapper[5052]: I1203 13:00:05.786400 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="crc-storage/crc-storage-crc-6gwst" Dec 03 13:00:06 crc kubenswrapper[5052]: W1203 13:00:06.186952 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod583f9858_b0cd_4976_a547_32ba02cbc85b.slice/crio-720e00d1beb8fdcf8b5dba0e89b3087a27f5108b7d0fa97c48a482068c1a4b06 WatchSource:0}: Error finding container 720e00d1beb8fdcf8b5dba0e89b3087a27f5108b7d0fa97c48a482068c1a4b06: Status 404 returned error can't find the container with id 720e00d1beb8fdcf8b5dba0e89b3087a27f5108b7d0fa97c48a482068c1a4b06 Dec 03 13:00:06 crc kubenswrapper[5052]: I1203 13:00:06.186964 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["crc-storage/crc-storage-crc-6gwst"] Dec 03 13:00:06 crc kubenswrapper[5052]: I1203 13:00:06.264261 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="crc-storage/crc-storage-crc-6gwst" event={"ID":"583f9858-b0cd-4976-a547-32ba02cbc85b","Type":"ContainerStarted","Data":"720e00d1beb8fdcf8b5dba0e89b3087a27f5108b7d0fa97c48a482068c1a4b06"} Dec 03 13:00:06 crc kubenswrapper[5052]: I1203 13:00:06.673823 5052 scope.go:117] "RemoveContainer" containerID="69fa7fbcd2aceb7c53703a19508876f93a97dc9adabb79c16fb691d90f606d58" Dec 03 13:00:07 crc kubenswrapper[5052]: I1203 13:00:07.076600 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-marketplace/certified-operators-dgght" Dec 03 13:00:07 crc kubenswrapper[5052]: I1203 13:00:07.076706 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openshift-marketplace/certified-operators-dgght" Dec 03 13:00:07 crc kubenswrapper[5052]: I1203 13:00:07.124246 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-marketplace/certified-operators-dgght" Dec 03 13:00:07 crc kubenswrapper[5052]: I1203 13:00:07.276716 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" event={"ID":"10c237bc-72e7-4da3-9f2c-b1688395bf6e","Type":"ContainerStarted","Data":"c2cab872e68bfd02259363286d891f85a8e365964ddb0da9f72bb76c86e2d3d2"} Dec 03 13:00:07 crc kubenswrapper[5052]: I1203 13:00:07.330350 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-marketplace/certified-operators-dgght" Dec 03 13:00:08 crc kubenswrapper[5052]: I1203 13:00:08.287541 5052 generic.go:334] "Generic (PLEG): container finished" podID="583f9858-b0cd-4976-a547-32ba02cbc85b" containerID="440638980aef4abb6f03a0c5400993e44ef78d9662a44adbdc33cec51bb52dd4" exitCode=0 Dec 03 13:00:08 crc kubenswrapper[5052]: I1203 13:00:08.287643 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="crc-storage/crc-storage-crc-6gwst" event={"ID":"583f9858-b0cd-4976-a547-32ba02cbc85b","Type":"ContainerDied","Data":"440638980aef4abb6f03a0c5400993e44ef78d9662a44adbdc33cec51bb52dd4"} Dec 03 13:00:09 crc kubenswrapper[5052]: I1203 13:00:09.558735 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="crc-storage/crc-storage-crc-6gwst" Dec 03 13:00:09 crc kubenswrapper[5052]: I1203 13:00:09.709742 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/583f9858-b0cd-4976-a547-32ba02cbc85b-node-mnt\") pod \"583f9858-b0cd-4976-a547-32ba02cbc85b\" (UID: \"583f9858-b0cd-4976-a547-32ba02cbc85b\") " Dec 03 13:00:09 crc kubenswrapper[5052]: I1203 13:00:09.709833 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/583f9858-b0cd-4976-a547-32ba02cbc85b-node-mnt" (OuterVolumeSpecName: "node-mnt") pod "583f9858-b0cd-4976-a547-32ba02cbc85b" (UID: "583f9858-b0cd-4976-a547-32ba02cbc85b"). InnerVolumeSpecName "node-mnt". PluginName "kubernetes.io/host-path", VolumeGidValue "" Dec 03 13:00:09 crc kubenswrapper[5052]: I1203 13:00:09.710222 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k4vm2\" (UniqueName: \"kubernetes.io/projected/583f9858-b0cd-4976-a547-32ba02cbc85b-kube-api-access-k4vm2\") pod \"583f9858-b0cd-4976-a547-32ba02cbc85b\" (UID: \"583f9858-b0cd-4976-a547-32ba02cbc85b\") " Dec 03 13:00:09 crc kubenswrapper[5052]: I1203 13:00:09.710410 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/583f9858-b0cd-4976-a547-32ba02cbc85b-crc-storage\") pod \"583f9858-b0cd-4976-a547-32ba02cbc85b\" (UID: \"583f9858-b0cd-4976-a547-32ba02cbc85b\") " Dec 03 13:00:09 crc kubenswrapper[5052]: I1203 13:00:09.712926 5052 reconciler_common.go:293] "Volume detached for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/583f9858-b0cd-4976-a547-32ba02cbc85b-node-mnt\") on node \"crc\" DevicePath \"\"" Dec 03 13:00:09 crc kubenswrapper[5052]: I1203 13:00:09.716141 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/583f9858-b0cd-4976-a547-32ba02cbc85b-kube-api-access-k4vm2" (OuterVolumeSpecName: "kube-api-access-k4vm2") pod "583f9858-b0cd-4976-a547-32ba02cbc85b" (UID: "583f9858-b0cd-4976-a547-32ba02cbc85b"). InnerVolumeSpecName "kube-api-access-k4vm2". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:00:09 crc kubenswrapper[5052]: I1203 13:00:09.732495 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/583f9858-b0cd-4976-a547-32ba02cbc85b-crc-storage" (OuterVolumeSpecName: "crc-storage") pod "583f9858-b0cd-4976-a547-32ba02cbc85b" (UID: "583f9858-b0cd-4976-a547-32ba02cbc85b"). InnerVolumeSpecName "crc-storage". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:00:09 crc kubenswrapper[5052]: I1203 13:00:09.815480 5052 reconciler_common.go:293] "Volume detached for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/583f9858-b0cd-4976-a547-32ba02cbc85b-crc-storage\") on node \"crc\" DevicePath \"\"" Dec 03 13:00:09 crc kubenswrapper[5052]: I1203 13:00:09.815638 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-k4vm2\" (UniqueName: \"kubernetes.io/projected/583f9858-b0cd-4976-a547-32ba02cbc85b-kube-api-access-k4vm2\") on node \"crc\" DevicePath \"\"" Dec 03 13:00:10 crc kubenswrapper[5052]: I1203 13:00:10.304751 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="crc-storage/crc-storage-crc-6gwst" event={"ID":"583f9858-b0cd-4976-a547-32ba02cbc85b","Type":"ContainerDied","Data":"720e00d1beb8fdcf8b5dba0e89b3087a27f5108b7d0fa97c48a482068c1a4b06"} Dec 03 13:00:10 crc kubenswrapper[5052]: I1203 13:00:10.305093 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="720e00d1beb8fdcf8b5dba0e89b3087a27f5108b7d0fa97c48a482068c1a4b06" Dec 03 13:00:10 crc kubenswrapper[5052]: I1203 13:00:10.304798 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="crc-storage/crc-storage-crc-6gwst" Dec 03 13:00:12 crc kubenswrapper[5052]: I1203 13:00:12.973374 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-78c7b787f5-9nm92"] Dec 03 13:00:12 crc kubenswrapper[5052]: E1203 13:00:12.974222 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="583f9858-b0cd-4976-a547-32ba02cbc85b" containerName="storage" Dec 03 13:00:12 crc kubenswrapper[5052]: I1203 13:00:12.974237 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="583f9858-b0cd-4976-a547-32ba02cbc85b" containerName="storage" Dec 03 13:00:12 crc kubenswrapper[5052]: I1203 13:00:12.974418 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="583f9858-b0cd-4976-a547-32ba02cbc85b" containerName="storage" Dec 03 13:00:12 crc kubenswrapper[5052]: I1203 13:00:12.975405 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-78c7b787f5-9nm92" Dec 03 13:00:12 crc kubenswrapper[5052]: I1203 13:00:12.977884 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"openstack-edpm-tls" Dec 03 13:00:12 crc kubenswrapper[5052]: I1203 13:00:12.992240 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-78c7b787f5-9nm92"] Dec 03 13:00:13 crc kubenswrapper[5052]: I1203 13:00:13.074825 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/75ea3aab-9805-4150-adda-58a4d500807b-config\") pod \"dnsmasq-dnsmasq-78c7b787f5-9nm92\" (UID: \"75ea3aab-9805-4150-adda-58a4d500807b\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-78c7b787f5-9nm92" Dec 03 13:00:13 crc kubenswrapper[5052]: I1203 13:00:13.074901 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/75ea3aab-9805-4150-adda-58a4d500807b-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-78c7b787f5-9nm92\" (UID: \"75ea3aab-9805-4150-adda-58a4d500807b\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-78c7b787f5-9nm92" Dec 03 13:00:13 crc kubenswrapper[5052]: I1203 13:00:13.074932 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2p6qc\" (UniqueName: \"kubernetes.io/projected/75ea3aab-9805-4150-adda-58a4d500807b-kube-api-access-2p6qc\") pod \"dnsmasq-dnsmasq-78c7b787f5-9nm92\" (UID: \"75ea3aab-9805-4150-adda-58a4d500807b\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-78c7b787f5-9nm92" Dec 03 13:00:13 crc kubenswrapper[5052]: I1203 13:00:13.075186 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openstack-edpm-tls\" (UniqueName: \"kubernetes.io/configmap/75ea3aab-9805-4150-adda-58a4d500807b-openstack-edpm-tls\") pod \"dnsmasq-dnsmasq-78c7b787f5-9nm92\" (UID: \"75ea3aab-9805-4150-adda-58a4d500807b\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-78c7b787f5-9nm92" Dec 03 13:00:13 crc kubenswrapper[5052]: I1203 13:00:13.095125 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-78c7b787f5-9nm92"] Dec 03 13:00:13 crc kubenswrapper[5052]: E1203 13:00:13.095860 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[config dnsmasq-svc kube-api-access-2p6qc openstack-edpm-tls], unattached volumes=[], failed to process volumes=[]: context canceled" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-78c7b787f5-9nm92" podUID="75ea3aab-9805-4150-adda-58a4d500807b" Dec 03 13:00:13 crc kubenswrapper[5052]: I1203 13:00:13.117706 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-57877c5b47-lsnhg"] Dec 03 13:00:13 crc kubenswrapper[5052]: I1203 13:00:13.119548 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-57877c5b47-lsnhg" Dec 03 13:00:13 crc kubenswrapper[5052]: I1203 13:00:13.133599 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-57877c5b47-lsnhg"] Dec 03 13:00:13 crc kubenswrapper[5052]: I1203 13:00:13.176122 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openstack-edpm-tls\" (UniqueName: \"kubernetes.io/configmap/5b86081b-240a-4b7e-9ee4-20f72ed39a31-openstack-edpm-tls\") pod \"dnsmasq-dnsmasq-57877c5b47-lsnhg\" (UID: \"5b86081b-240a-4b7e-9ee4-20f72ed39a31\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-57877c5b47-lsnhg" Dec 03 13:00:13 crc kubenswrapper[5052]: I1203 13:00:13.176187 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/75ea3aab-9805-4150-adda-58a4d500807b-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-78c7b787f5-9nm92\" (UID: \"75ea3aab-9805-4150-adda-58a4d500807b\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-78c7b787f5-9nm92" Dec 03 13:00:13 crc kubenswrapper[5052]: I1203 13:00:13.176208 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-2p6qc\" (UniqueName: \"kubernetes.io/projected/75ea3aab-9805-4150-adda-58a4d500807b-kube-api-access-2p6qc\") pod \"dnsmasq-dnsmasq-78c7b787f5-9nm92\" (UID: \"75ea3aab-9805-4150-adda-58a4d500807b\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-78c7b787f5-9nm92" Dec 03 13:00:13 crc kubenswrapper[5052]: I1203 13:00:13.176261 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5b86081b-240a-4b7e-9ee4-20f72ed39a31-config\") pod \"dnsmasq-dnsmasq-57877c5b47-lsnhg\" (UID: \"5b86081b-240a-4b7e-9ee4-20f72ed39a31\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-57877c5b47-lsnhg" Dec 03 13:00:13 crc kubenswrapper[5052]: I1203 13:00:13.176276 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/5b86081b-240a-4b7e-9ee4-20f72ed39a31-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-57877c5b47-lsnhg\" (UID: \"5b86081b-240a-4b7e-9ee4-20f72ed39a31\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-57877c5b47-lsnhg" Dec 03 13:00:13 crc kubenswrapper[5052]: I1203 13:00:13.176294 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"openstack-edpm-tls\" (UniqueName: \"kubernetes.io/configmap/75ea3aab-9805-4150-adda-58a4d500807b-openstack-edpm-tls\") pod \"dnsmasq-dnsmasq-78c7b787f5-9nm92\" (UID: \"75ea3aab-9805-4150-adda-58a4d500807b\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-78c7b787f5-9nm92" Dec 03 13:00:13 crc kubenswrapper[5052]: I1203 13:00:13.176358 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dzt4n\" (UniqueName: \"kubernetes.io/projected/5b86081b-240a-4b7e-9ee4-20f72ed39a31-kube-api-access-dzt4n\") pod \"dnsmasq-dnsmasq-57877c5b47-lsnhg\" (UID: \"5b86081b-240a-4b7e-9ee4-20f72ed39a31\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-57877c5b47-lsnhg" Dec 03 13:00:13 crc kubenswrapper[5052]: I1203 13:00:13.176395 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/75ea3aab-9805-4150-adda-58a4d500807b-config\") pod \"dnsmasq-dnsmasq-78c7b787f5-9nm92\" (UID: \"75ea3aab-9805-4150-adda-58a4d500807b\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-78c7b787f5-9nm92" Dec 03 13:00:13 crc kubenswrapper[5052]: I1203 13:00:13.177547 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/75ea3aab-9805-4150-adda-58a4d500807b-config\") pod \"dnsmasq-dnsmasq-78c7b787f5-9nm92\" (UID: \"75ea3aab-9805-4150-adda-58a4d500807b\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-78c7b787f5-9nm92" Dec 03 13:00:13 crc kubenswrapper[5052]: I1203 13:00:13.177552 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/75ea3aab-9805-4150-adda-58a4d500807b-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-78c7b787f5-9nm92\" (UID: \"75ea3aab-9805-4150-adda-58a4d500807b\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-78c7b787f5-9nm92" Dec 03 13:00:13 crc kubenswrapper[5052]: I1203 13:00:13.178111 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"openstack-edpm-tls\" (UniqueName: \"kubernetes.io/configmap/75ea3aab-9805-4150-adda-58a4d500807b-openstack-edpm-tls\") pod \"dnsmasq-dnsmasq-78c7b787f5-9nm92\" (UID: \"75ea3aab-9805-4150-adda-58a4d500807b\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-78c7b787f5-9nm92" Dec 03 13:00:13 crc kubenswrapper[5052]: I1203 13:00:13.199906 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-2p6qc\" (UniqueName: \"kubernetes.io/projected/75ea3aab-9805-4150-adda-58a4d500807b-kube-api-access-2p6qc\") pod \"dnsmasq-dnsmasq-78c7b787f5-9nm92\" (UID: \"75ea3aab-9805-4150-adda-58a4d500807b\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-78c7b787f5-9nm92" Dec 03 13:00:13 crc kubenswrapper[5052]: I1203 13:00:13.277967 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5b86081b-240a-4b7e-9ee4-20f72ed39a31-config\") pod \"dnsmasq-dnsmasq-57877c5b47-lsnhg\" (UID: \"5b86081b-240a-4b7e-9ee4-20f72ed39a31\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-57877c5b47-lsnhg" Dec 03 13:00:13 crc kubenswrapper[5052]: I1203 13:00:13.278369 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/5b86081b-240a-4b7e-9ee4-20f72ed39a31-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-57877c5b47-lsnhg\" (UID: \"5b86081b-240a-4b7e-9ee4-20f72ed39a31\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-57877c5b47-lsnhg" Dec 03 13:00:13 crc kubenswrapper[5052]: I1203 13:00:13.278467 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-dzt4n\" (UniqueName: \"kubernetes.io/projected/5b86081b-240a-4b7e-9ee4-20f72ed39a31-kube-api-access-dzt4n\") pod \"dnsmasq-dnsmasq-57877c5b47-lsnhg\" (UID: \"5b86081b-240a-4b7e-9ee4-20f72ed39a31\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-57877c5b47-lsnhg" Dec 03 13:00:13 crc kubenswrapper[5052]: I1203 13:00:13.278634 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"openstack-edpm-tls\" (UniqueName: \"kubernetes.io/configmap/5b86081b-240a-4b7e-9ee4-20f72ed39a31-openstack-edpm-tls\") pod \"dnsmasq-dnsmasq-57877c5b47-lsnhg\" (UID: \"5b86081b-240a-4b7e-9ee4-20f72ed39a31\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-57877c5b47-lsnhg" Dec 03 13:00:13 crc kubenswrapper[5052]: I1203 13:00:13.279412 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5b86081b-240a-4b7e-9ee4-20f72ed39a31-config\") pod \"dnsmasq-dnsmasq-57877c5b47-lsnhg\" (UID: \"5b86081b-240a-4b7e-9ee4-20f72ed39a31\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-57877c5b47-lsnhg" Dec 03 13:00:13 crc kubenswrapper[5052]: I1203 13:00:13.279550 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"openstack-edpm-tls\" (UniqueName: \"kubernetes.io/configmap/5b86081b-240a-4b7e-9ee4-20f72ed39a31-openstack-edpm-tls\") pod \"dnsmasq-dnsmasq-57877c5b47-lsnhg\" (UID: \"5b86081b-240a-4b7e-9ee4-20f72ed39a31\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-57877c5b47-lsnhg" Dec 03 13:00:13 crc kubenswrapper[5052]: I1203 13:00:13.279700 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/5b86081b-240a-4b7e-9ee4-20f72ed39a31-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-57877c5b47-lsnhg\" (UID: \"5b86081b-240a-4b7e-9ee4-20f72ed39a31\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-57877c5b47-lsnhg" Dec 03 13:00:13 crc kubenswrapper[5052]: I1203 13:00:13.302086 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-dzt4n\" (UniqueName: \"kubernetes.io/projected/5b86081b-240a-4b7e-9ee4-20f72ed39a31-kube-api-access-dzt4n\") pod \"dnsmasq-dnsmasq-57877c5b47-lsnhg\" (UID: \"5b86081b-240a-4b7e-9ee4-20f72ed39a31\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-57877c5b47-lsnhg" Dec 03 13:00:13 crc kubenswrapper[5052]: I1203 13:00:13.330115 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-78c7b787f5-9nm92" Dec 03 13:00:13 crc kubenswrapper[5052]: I1203 13:00:13.342773 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-78c7b787f5-9nm92" Dec 03 13:00:13 crc kubenswrapper[5052]: I1203 13:00:13.379616 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/75ea3aab-9805-4150-adda-58a4d500807b-config\") pod \"75ea3aab-9805-4150-adda-58a4d500807b\" (UID: \"75ea3aab-9805-4150-adda-58a4d500807b\") " Dec 03 13:00:13 crc kubenswrapper[5052]: I1203 13:00:13.379694 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2p6qc\" (UniqueName: \"kubernetes.io/projected/75ea3aab-9805-4150-adda-58a4d500807b-kube-api-access-2p6qc\") pod \"75ea3aab-9805-4150-adda-58a4d500807b\" (UID: \"75ea3aab-9805-4150-adda-58a4d500807b\") " Dec 03 13:00:13 crc kubenswrapper[5052]: I1203 13:00:13.379781 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"openstack-edpm-tls\" (UniqueName: \"kubernetes.io/configmap/75ea3aab-9805-4150-adda-58a4d500807b-openstack-edpm-tls\") pod \"75ea3aab-9805-4150-adda-58a4d500807b\" (UID: \"75ea3aab-9805-4150-adda-58a4d500807b\") " Dec 03 13:00:13 crc kubenswrapper[5052]: I1203 13:00:13.379809 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/75ea3aab-9805-4150-adda-58a4d500807b-dnsmasq-svc\") pod \"75ea3aab-9805-4150-adda-58a4d500807b\" (UID: \"75ea3aab-9805-4150-adda-58a4d500807b\") " Dec 03 13:00:13 crc kubenswrapper[5052]: I1203 13:00:13.380160 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/75ea3aab-9805-4150-adda-58a4d500807b-config" (OuterVolumeSpecName: "config") pod "75ea3aab-9805-4150-adda-58a4d500807b" (UID: "75ea3aab-9805-4150-adda-58a4d500807b"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:00:13 crc kubenswrapper[5052]: I1203 13:00:13.380202 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/75ea3aab-9805-4150-adda-58a4d500807b-openstack-edpm-tls" (OuterVolumeSpecName: "openstack-edpm-tls") pod "75ea3aab-9805-4150-adda-58a4d500807b" (UID: "75ea3aab-9805-4150-adda-58a4d500807b"). InnerVolumeSpecName "openstack-edpm-tls". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:00:13 crc kubenswrapper[5052]: I1203 13:00:13.380500 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/75ea3aab-9805-4150-adda-58a4d500807b-dnsmasq-svc" (OuterVolumeSpecName: "dnsmasq-svc") pod "75ea3aab-9805-4150-adda-58a4d500807b" (UID: "75ea3aab-9805-4150-adda-58a4d500807b"). InnerVolumeSpecName "dnsmasq-svc". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:00:13 crc kubenswrapper[5052]: I1203 13:00:13.382942 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/75ea3aab-9805-4150-adda-58a4d500807b-kube-api-access-2p6qc" (OuterVolumeSpecName: "kube-api-access-2p6qc") pod "75ea3aab-9805-4150-adda-58a4d500807b" (UID: "75ea3aab-9805-4150-adda-58a4d500807b"). InnerVolumeSpecName "kube-api-access-2p6qc". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:00:13 crc kubenswrapper[5052]: I1203 13:00:13.435658 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-57877c5b47-lsnhg" Dec 03 13:00:13 crc kubenswrapper[5052]: I1203 13:00:13.471890 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openshift-marketplace/certified-operators-dgght"] Dec 03 13:00:13 crc kubenswrapper[5052]: I1203 13:00:13.472163 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openshift-marketplace/certified-operators-dgght" podUID="e2b00b19-1d3b-4097-89cc-c37391b91db6" containerName="registry-server" containerID="cri-o://1ebef0065a44d14ba4c3d7cd887592081f8ee24a7688f2546e470ed1df0cdd4e" gracePeriod=2 Dec 03 13:00:13 crc kubenswrapper[5052]: I1203 13:00:13.481901 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-2p6qc\" (UniqueName: \"kubernetes.io/projected/75ea3aab-9805-4150-adda-58a4d500807b-kube-api-access-2p6qc\") on node \"crc\" DevicePath \"\"" Dec 03 13:00:13 crc kubenswrapper[5052]: I1203 13:00:13.481989 5052 reconciler_common.go:293] "Volume detached for volume \"openstack-edpm-tls\" (UniqueName: \"kubernetes.io/configmap/75ea3aab-9805-4150-adda-58a4d500807b-openstack-edpm-tls\") on node \"crc\" DevicePath \"\"" Dec 03 13:00:13 crc kubenswrapper[5052]: I1203 13:00:13.482002 5052 reconciler_common.go:293] "Volume detached for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/75ea3aab-9805-4150-adda-58a4d500807b-dnsmasq-svc\") on node \"crc\" DevicePath \"\"" Dec 03 13:00:13 crc kubenswrapper[5052]: I1203 13:00:13.482014 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/75ea3aab-9805-4150-adda-58a4d500807b-config\") on node \"crc\" DevicePath \"\"" Dec 03 13:00:13 crc kubenswrapper[5052]: I1203 13:00:13.819608 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/certified-operators-dgght" Dec 03 13:00:13 crc kubenswrapper[5052]: I1203 13:00:13.880863 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-57877c5b47-lsnhg"] Dec 03 13:00:13 crc kubenswrapper[5052]: I1203 13:00:13.887289 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/e2b00b19-1d3b-4097-89cc-c37391b91db6-utilities\") pod \"e2b00b19-1d3b-4097-89cc-c37391b91db6\" (UID: \"e2b00b19-1d3b-4097-89cc-c37391b91db6\") " Dec 03 13:00:13 crc kubenswrapper[5052]: I1203 13:00:13.887350 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/e2b00b19-1d3b-4097-89cc-c37391b91db6-catalog-content\") pod \"e2b00b19-1d3b-4097-89cc-c37391b91db6\" (UID: \"e2b00b19-1d3b-4097-89cc-c37391b91db6\") " Dec 03 13:00:13 crc kubenswrapper[5052]: I1203 13:00:13.887421 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qk5kp\" (UniqueName: \"kubernetes.io/projected/e2b00b19-1d3b-4097-89cc-c37391b91db6-kube-api-access-qk5kp\") pod \"e2b00b19-1d3b-4097-89cc-c37391b91db6\" (UID: \"e2b00b19-1d3b-4097-89cc-c37391b91db6\") " Dec 03 13:00:13 crc kubenswrapper[5052]: I1203 13:00:13.888988 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e2b00b19-1d3b-4097-89cc-c37391b91db6-utilities" (OuterVolumeSpecName: "utilities") pod "e2b00b19-1d3b-4097-89cc-c37391b91db6" (UID: "e2b00b19-1d3b-4097-89cc-c37391b91db6"). InnerVolumeSpecName "utilities". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 13:00:13 crc kubenswrapper[5052]: I1203 13:00:13.891644 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e2b00b19-1d3b-4097-89cc-c37391b91db6-kube-api-access-qk5kp" (OuterVolumeSpecName: "kube-api-access-qk5kp") pod "e2b00b19-1d3b-4097-89cc-c37391b91db6" (UID: "e2b00b19-1d3b-4097-89cc-c37391b91db6"). InnerVolumeSpecName "kube-api-access-qk5kp". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:00:13 crc kubenswrapper[5052]: W1203 13:00:13.893592 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5b86081b_240a_4b7e_9ee4_20f72ed39a31.slice/crio-c376bb755a72d0205fd01f437b1fdabb7130d27a365d5d581a7503bb6d059b5f WatchSource:0}: Error finding container c376bb755a72d0205fd01f437b1fdabb7130d27a365d5d581a7503bb6d059b5f: Status 404 returned error can't find the container with id c376bb755a72d0205fd01f437b1fdabb7130d27a365d5d581a7503bb6d059b5f Dec 03 13:00:13 crc kubenswrapper[5052]: I1203 13:00:13.939697 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e2b00b19-1d3b-4097-89cc-c37391b91db6-catalog-content" (OuterVolumeSpecName: "catalog-content") pod "e2b00b19-1d3b-4097-89cc-c37391b91db6" (UID: "e2b00b19-1d3b-4097-89cc-c37391b91db6"). InnerVolumeSpecName "catalog-content". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 13:00:13 crc kubenswrapper[5052]: I1203 13:00:13.988776 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-qk5kp\" (UniqueName: \"kubernetes.io/projected/e2b00b19-1d3b-4097-89cc-c37391b91db6-kube-api-access-qk5kp\") on node \"crc\" DevicePath \"\"" Dec 03 13:00:13 crc kubenswrapper[5052]: I1203 13:00:13.988805 5052 reconciler_common.go:293] "Volume detached for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/e2b00b19-1d3b-4097-89cc-c37391b91db6-utilities\") on node \"crc\" DevicePath \"\"" Dec 03 13:00:13 crc kubenswrapper[5052]: I1203 13:00:13.988837 5052 reconciler_common.go:293] "Volume detached for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/e2b00b19-1d3b-4097-89cc-c37391b91db6-catalog-content\") on node \"crc\" DevicePath \"\"" Dec 03 13:00:14 crc kubenswrapper[5052]: I1203 13:00:14.340717 5052 generic.go:334] "Generic (PLEG): container finished" podID="e2b00b19-1d3b-4097-89cc-c37391b91db6" containerID="1ebef0065a44d14ba4c3d7cd887592081f8ee24a7688f2546e470ed1df0cdd4e" exitCode=0 Dec 03 13:00:14 crc kubenswrapper[5052]: I1203 13:00:14.340804 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/certified-operators-dgght" Dec 03 13:00:14 crc kubenswrapper[5052]: I1203 13:00:14.340832 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/certified-operators-dgght" event={"ID":"e2b00b19-1d3b-4097-89cc-c37391b91db6","Type":"ContainerDied","Data":"1ebef0065a44d14ba4c3d7cd887592081f8ee24a7688f2546e470ed1df0cdd4e"} Dec 03 13:00:14 crc kubenswrapper[5052]: I1203 13:00:14.341335 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/certified-operators-dgght" event={"ID":"e2b00b19-1d3b-4097-89cc-c37391b91db6","Type":"ContainerDied","Data":"68cbd3a1042668f024777ad2567e80c494d5ed325cdac3880bbb9eff28ec65f7"} Dec 03 13:00:14 crc kubenswrapper[5052]: I1203 13:00:14.341376 5052 scope.go:117] "RemoveContainer" containerID="1ebef0065a44d14ba4c3d7cd887592081f8ee24a7688f2546e470ed1df0cdd4e" Dec 03 13:00:14 crc kubenswrapper[5052]: I1203 13:00:14.343538 5052 generic.go:334] "Generic (PLEG): container finished" podID="5b86081b-240a-4b7e-9ee4-20f72ed39a31" containerID="6c20b742092df772325f108139591f4ea6f2995c9026f65963a0dd2d7e4de316" exitCode=0 Dec 03 13:00:14 crc kubenswrapper[5052]: I1203 13:00:14.343618 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-78c7b787f5-9nm92" Dec 03 13:00:14 crc kubenswrapper[5052]: I1203 13:00:14.343919 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-57877c5b47-lsnhg" event={"ID":"5b86081b-240a-4b7e-9ee4-20f72ed39a31","Type":"ContainerDied","Data":"6c20b742092df772325f108139591f4ea6f2995c9026f65963a0dd2d7e4de316"} Dec 03 13:00:14 crc kubenswrapper[5052]: I1203 13:00:14.343980 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-57877c5b47-lsnhg" event={"ID":"5b86081b-240a-4b7e-9ee4-20f72ed39a31","Type":"ContainerStarted","Data":"c376bb755a72d0205fd01f437b1fdabb7130d27a365d5d581a7503bb6d059b5f"} Dec 03 13:00:14 crc kubenswrapper[5052]: I1203 13:00:14.430960 5052 scope.go:117] "RemoveContainer" containerID="3ac23fc0406be9a15fb72d13e4f96186af91611322b7cf90e9a443f635960007" Dec 03 13:00:14 crc kubenswrapper[5052]: I1203 13:00:14.485017 5052 scope.go:117] "RemoveContainer" containerID="5e32971434c7553ce7ecc375b5d743d0dc43163a0f4a021146fba6b002b8ae03" Dec 03 13:00:14 crc kubenswrapper[5052]: I1203 13:00:14.493808 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openshift-marketplace/certified-operators-dgght"] Dec 03 13:00:14 crc kubenswrapper[5052]: I1203 13:00:14.507451 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openshift-marketplace/certified-operators-dgght"] Dec 03 13:00:14 crc kubenswrapper[5052]: I1203 13:00:14.521250 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-78c7b787f5-9nm92"] Dec 03 13:00:14 crc kubenswrapper[5052]: I1203 13:00:14.529240 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-78c7b787f5-9nm92"] Dec 03 13:00:14 crc kubenswrapper[5052]: I1203 13:00:14.546429 5052 scope.go:117] "RemoveContainer" containerID="1ebef0065a44d14ba4c3d7cd887592081f8ee24a7688f2546e470ed1df0cdd4e" Dec 03 13:00:14 crc kubenswrapper[5052]: E1203 13:00:14.546930 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1ebef0065a44d14ba4c3d7cd887592081f8ee24a7688f2546e470ed1df0cdd4e\": container with ID starting with 1ebef0065a44d14ba4c3d7cd887592081f8ee24a7688f2546e470ed1df0cdd4e not found: ID does not exist" containerID="1ebef0065a44d14ba4c3d7cd887592081f8ee24a7688f2546e470ed1df0cdd4e" Dec 03 13:00:14 crc kubenswrapper[5052]: I1203 13:00:14.546969 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1ebef0065a44d14ba4c3d7cd887592081f8ee24a7688f2546e470ed1df0cdd4e"} err="failed to get container status \"1ebef0065a44d14ba4c3d7cd887592081f8ee24a7688f2546e470ed1df0cdd4e\": rpc error: code = NotFound desc = could not find container \"1ebef0065a44d14ba4c3d7cd887592081f8ee24a7688f2546e470ed1df0cdd4e\": container with ID starting with 1ebef0065a44d14ba4c3d7cd887592081f8ee24a7688f2546e470ed1df0cdd4e not found: ID does not exist" Dec 03 13:00:14 crc kubenswrapper[5052]: I1203 13:00:14.546989 5052 scope.go:117] "RemoveContainer" containerID="3ac23fc0406be9a15fb72d13e4f96186af91611322b7cf90e9a443f635960007" Dec 03 13:00:14 crc kubenswrapper[5052]: E1203 13:00:14.547505 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3ac23fc0406be9a15fb72d13e4f96186af91611322b7cf90e9a443f635960007\": container with ID starting with 3ac23fc0406be9a15fb72d13e4f96186af91611322b7cf90e9a443f635960007 not found: ID does not exist" containerID="3ac23fc0406be9a15fb72d13e4f96186af91611322b7cf90e9a443f635960007" Dec 03 13:00:14 crc kubenswrapper[5052]: I1203 13:00:14.547571 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3ac23fc0406be9a15fb72d13e4f96186af91611322b7cf90e9a443f635960007"} err="failed to get container status \"3ac23fc0406be9a15fb72d13e4f96186af91611322b7cf90e9a443f635960007\": rpc error: code = NotFound desc = could not find container \"3ac23fc0406be9a15fb72d13e4f96186af91611322b7cf90e9a443f635960007\": container with ID starting with 3ac23fc0406be9a15fb72d13e4f96186af91611322b7cf90e9a443f635960007 not found: ID does not exist" Dec 03 13:00:14 crc kubenswrapper[5052]: I1203 13:00:14.547606 5052 scope.go:117] "RemoveContainer" containerID="5e32971434c7553ce7ecc375b5d743d0dc43163a0f4a021146fba6b002b8ae03" Dec 03 13:00:14 crc kubenswrapper[5052]: E1203 13:00:14.547930 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5e32971434c7553ce7ecc375b5d743d0dc43163a0f4a021146fba6b002b8ae03\": container with ID starting with 5e32971434c7553ce7ecc375b5d743d0dc43163a0f4a021146fba6b002b8ae03 not found: ID does not exist" containerID="5e32971434c7553ce7ecc375b5d743d0dc43163a0f4a021146fba6b002b8ae03" Dec 03 13:00:14 crc kubenswrapper[5052]: I1203 13:00:14.547957 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5e32971434c7553ce7ecc375b5d743d0dc43163a0f4a021146fba6b002b8ae03"} err="failed to get container status \"5e32971434c7553ce7ecc375b5d743d0dc43163a0f4a021146fba6b002b8ae03\": rpc error: code = NotFound desc = could not find container \"5e32971434c7553ce7ecc375b5d743d0dc43163a0f4a021146fba6b002b8ae03\": container with ID starting with 5e32971434c7553ce7ecc375b5d743d0dc43163a0f4a021146fba6b002b8ae03 not found: ID does not exist" Dec 03 13:00:15 crc kubenswrapper[5052]: I1203 13:00:15.354592 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-57877c5b47-lsnhg" event={"ID":"5b86081b-240a-4b7e-9ee4-20f72ed39a31","Type":"ContainerStarted","Data":"1ee85c805793220260e3b321df2901ec536fb275df4bf8d10d50e4c9d78710ad"} Dec 03 13:00:15 crc kubenswrapper[5052]: I1203 13:00:15.355027 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-57877c5b47-lsnhg" Dec 03 13:00:15 crc kubenswrapper[5052]: I1203 13:00:15.385910 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-57877c5b47-lsnhg" podStartSLOduration=2.385881539 podStartE2EDuration="2.385881539s" podCreationTimestamp="2025-12-03 13:00:13 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 13:00:15.375801571 +0000 UTC m=+6005.964043335" watchObservedRunningTime="2025-12-03 13:00:15.385881539 +0000 UTC m=+6005.974123303" Dec 03 13:00:15 crc kubenswrapper[5052]: I1203 13:00:15.682982 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="75ea3aab-9805-4150-adda-58a4d500807b" path="/var/lib/kubelet/pods/75ea3aab-9805-4150-adda-58a4d500807b/volumes" Dec 03 13:00:15 crc kubenswrapper[5052]: I1203 13:00:15.683413 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e2b00b19-1d3b-4097-89cc-c37391b91db6" path="/var/lib/kubelet/pods/e2b00b19-1d3b-4097-89cc-c37391b91db6/volumes" Dec 03 13:00:23 crc kubenswrapper[5052]: I1203 13:00:23.436682 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-57877c5b47-lsnhg" Dec 03 13:00:23 crc kubenswrapper[5052]: I1203 13:00:23.492211 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-x6wdj"] Dec 03 13:00:23 crc kubenswrapper[5052]: I1203 13:00:23.492450 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-x6wdj" podUID="08ab2cda-3a42-49b5-843e-6c8b151064df" containerName="dnsmasq-dns" containerID="cri-o://a310ad98499337b886702c931f5d7c2b6343449dfab7c440a0ecf22708504a26" gracePeriod=10 Dec 03 13:00:23 crc kubenswrapper[5052]: I1203 13:00:23.613180 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-76b7c4d945-lxqvh"] Dec 03 13:00:23 crc kubenswrapper[5052]: E1203 13:00:23.613546 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="e2b00b19-1d3b-4097-89cc-c37391b91db6" containerName="registry-server" Dec 03 13:00:23 crc kubenswrapper[5052]: I1203 13:00:23.613564 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="e2b00b19-1d3b-4097-89cc-c37391b91db6" containerName="registry-server" Dec 03 13:00:23 crc kubenswrapper[5052]: E1203 13:00:23.613587 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="e2b00b19-1d3b-4097-89cc-c37391b91db6" containerName="extract-content" Dec 03 13:00:23 crc kubenswrapper[5052]: I1203 13:00:23.613596 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="e2b00b19-1d3b-4097-89cc-c37391b91db6" containerName="extract-content" Dec 03 13:00:23 crc kubenswrapper[5052]: E1203 13:00:23.613628 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="e2b00b19-1d3b-4097-89cc-c37391b91db6" containerName="extract-utilities" Dec 03 13:00:23 crc kubenswrapper[5052]: I1203 13:00:23.613637 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="e2b00b19-1d3b-4097-89cc-c37391b91db6" containerName="extract-utilities" Dec 03 13:00:23 crc kubenswrapper[5052]: I1203 13:00:23.613818 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="e2b00b19-1d3b-4097-89cc-c37391b91db6" containerName="registry-server" Dec 03 13:00:23 crc kubenswrapper[5052]: I1203 13:00:23.614719 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-76b7c4d945-lxqvh" Dec 03 13:00:23 crc kubenswrapper[5052]: I1203 13:00:23.628845 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d697b768-3215-466f-9438-109e015ad8ed-config\") pod \"dnsmasq-dnsmasq-76b7c4d945-lxqvh\" (UID: \"d697b768-3215-466f-9438-109e015ad8ed\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-76b7c4d945-lxqvh" Dec 03 13:00:23 crc kubenswrapper[5052]: I1203 13:00:23.628893 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openstack-edpm-tls\" (UniqueName: \"kubernetes.io/configmap/d697b768-3215-466f-9438-109e015ad8ed-openstack-edpm-tls\") pod \"dnsmasq-dnsmasq-76b7c4d945-lxqvh\" (UID: \"d697b768-3215-466f-9438-109e015ad8ed\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-76b7c4d945-lxqvh" Dec 03 13:00:23 crc kubenswrapper[5052]: I1203 13:00:23.628951 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b4tsv\" (UniqueName: \"kubernetes.io/projected/d697b768-3215-466f-9438-109e015ad8ed-kube-api-access-b4tsv\") pod \"dnsmasq-dnsmasq-76b7c4d945-lxqvh\" (UID: \"d697b768-3215-466f-9438-109e015ad8ed\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-76b7c4d945-lxqvh" Dec 03 13:00:23 crc kubenswrapper[5052]: I1203 13:00:23.629102 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/d697b768-3215-466f-9438-109e015ad8ed-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-76b7c4d945-lxqvh\" (UID: \"d697b768-3215-466f-9438-109e015ad8ed\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-76b7c4d945-lxqvh" Dec 03 13:00:23 crc kubenswrapper[5052]: I1203 13:00:23.630787 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-76b7c4d945-lxqvh"] Dec 03 13:00:23 crc kubenswrapper[5052]: I1203 13:00:23.730850 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/d697b768-3215-466f-9438-109e015ad8ed-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-76b7c4d945-lxqvh\" (UID: \"d697b768-3215-466f-9438-109e015ad8ed\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-76b7c4d945-lxqvh" Dec 03 13:00:23 crc kubenswrapper[5052]: I1203 13:00:23.731336 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d697b768-3215-466f-9438-109e015ad8ed-config\") pod \"dnsmasq-dnsmasq-76b7c4d945-lxqvh\" (UID: \"d697b768-3215-466f-9438-109e015ad8ed\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-76b7c4d945-lxqvh" Dec 03 13:00:23 crc kubenswrapper[5052]: I1203 13:00:23.731372 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"openstack-edpm-tls\" (UniqueName: \"kubernetes.io/configmap/d697b768-3215-466f-9438-109e015ad8ed-openstack-edpm-tls\") pod \"dnsmasq-dnsmasq-76b7c4d945-lxqvh\" (UID: \"d697b768-3215-466f-9438-109e015ad8ed\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-76b7c4d945-lxqvh" Dec 03 13:00:23 crc kubenswrapper[5052]: I1203 13:00:23.731461 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-b4tsv\" (UniqueName: \"kubernetes.io/projected/d697b768-3215-466f-9438-109e015ad8ed-kube-api-access-b4tsv\") pod \"dnsmasq-dnsmasq-76b7c4d945-lxqvh\" (UID: \"d697b768-3215-466f-9438-109e015ad8ed\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-76b7c4d945-lxqvh" Dec 03 13:00:23 crc kubenswrapper[5052]: I1203 13:00:23.731988 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/d697b768-3215-466f-9438-109e015ad8ed-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-76b7c4d945-lxqvh\" (UID: \"d697b768-3215-466f-9438-109e015ad8ed\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-76b7c4d945-lxqvh" Dec 03 13:00:23 crc kubenswrapper[5052]: I1203 13:00:23.732248 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d697b768-3215-466f-9438-109e015ad8ed-config\") pod \"dnsmasq-dnsmasq-76b7c4d945-lxqvh\" (UID: \"d697b768-3215-466f-9438-109e015ad8ed\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-76b7c4d945-lxqvh" Dec 03 13:00:23 crc kubenswrapper[5052]: I1203 13:00:23.732341 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"openstack-edpm-tls\" (UniqueName: \"kubernetes.io/configmap/d697b768-3215-466f-9438-109e015ad8ed-openstack-edpm-tls\") pod \"dnsmasq-dnsmasq-76b7c4d945-lxqvh\" (UID: \"d697b768-3215-466f-9438-109e015ad8ed\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-76b7c4d945-lxqvh" Dec 03 13:00:23 crc kubenswrapper[5052]: I1203 13:00:23.752112 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-b4tsv\" (UniqueName: \"kubernetes.io/projected/d697b768-3215-466f-9438-109e015ad8ed-kube-api-access-b4tsv\") pod \"dnsmasq-dnsmasq-76b7c4d945-lxqvh\" (UID: \"d697b768-3215-466f-9438-109e015ad8ed\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-76b7c4d945-lxqvh" Dec 03 13:00:23 crc kubenswrapper[5052]: I1203 13:00:23.928091 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-x6wdj" Dec 03 13:00:23 crc kubenswrapper[5052]: I1203 13:00:23.933567 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7s4f7\" (UniqueName: \"kubernetes.io/projected/08ab2cda-3a42-49b5-843e-6c8b151064df-kube-api-access-7s4f7\") pod \"08ab2cda-3a42-49b5-843e-6c8b151064df\" (UID: \"08ab2cda-3a42-49b5-843e-6c8b151064df\") " Dec 03 13:00:23 crc kubenswrapper[5052]: I1203 13:00:23.933787 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/08ab2cda-3a42-49b5-843e-6c8b151064df-dnsmasq-svc\") pod \"08ab2cda-3a42-49b5-843e-6c8b151064df\" (UID: \"08ab2cda-3a42-49b5-843e-6c8b151064df\") " Dec 03 13:00:23 crc kubenswrapper[5052]: I1203 13:00:23.933852 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/08ab2cda-3a42-49b5-843e-6c8b151064df-config\") pod \"08ab2cda-3a42-49b5-843e-6c8b151064df\" (UID: \"08ab2cda-3a42-49b5-843e-6c8b151064df\") " Dec 03 13:00:23 crc kubenswrapper[5052]: I1203 13:00:23.937451 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/08ab2cda-3a42-49b5-843e-6c8b151064df-kube-api-access-7s4f7" (OuterVolumeSpecName: "kube-api-access-7s4f7") pod "08ab2cda-3a42-49b5-843e-6c8b151064df" (UID: "08ab2cda-3a42-49b5-843e-6c8b151064df"). InnerVolumeSpecName "kube-api-access-7s4f7". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:00:23 crc kubenswrapper[5052]: I1203 13:00:23.979700 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-76b7c4d945-lxqvh" Dec 03 13:00:23 crc kubenswrapper[5052]: I1203 13:00:23.982327 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/08ab2cda-3a42-49b5-843e-6c8b151064df-config" (OuterVolumeSpecName: "config") pod "08ab2cda-3a42-49b5-843e-6c8b151064df" (UID: "08ab2cda-3a42-49b5-843e-6c8b151064df"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:00:23 crc kubenswrapper[5052]: I1203 13:00:23.983134 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/08ab2cda-3a42-49b5-843e-6c8b151064df-dnsmasq-svc" (OuterVolumeSpecName: "dnsmasq-svc") pod "08ab2cda-3a42-49b5-843e-6c8b151064df" (UID: "08ab2cda-3a42-49b5-843e-6c8b151064df"). InnerVolumeSpecName "dnsmasq-svc". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:00:24 crc kubenswrapper[5052]: I1203 13:00:24.036006 5052 reconciler_common.go:293] "Volume detached for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/08ab2cda-3a42-49b5-843e-6c8b151064df-dnsmasq-svc\") on node \"crc\" DevicePath \"\"" Dec 03 13:00:24 crc kubenswrapper[5052]: I1203 13:00:24.036072 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/08ab2cda-3a42-49b5-843e-6c8b151064df-config\") on node \"crc\" DevicePath \"\"" Dec 03 13:00:24 crc kubenswrapper[5052]: I1203 13:00:24.036146 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-7s4f7\" (UniqueName: \"kubernetes.io/projected/08ab2cda-3a42-49b5-843e-6c8b151064df-kube-api-access-7s4f7\") on node \"crc\" DevicePath \"\"" Dec 03 13:00:24 crc kubenswrapper[5052]: I1203 13:00:24.388951 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-76b7c4d945-lxqvh"] Dec 03 13:00:24 crc kubenswrapper[5052]: W1203 13:00:24.390071 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd697b768_3215_466f_9438_109e015ad8ed.slice/crio-9a57d57931016bff61e67ec0e8137f80718ea231cd302d708c19ae497a38df4b WatchSource:0}: Error finding container 9a57d57931016bff61e67ec0e8137f80718ea231cd302d708c19ae497a38df4b: Status 404 returned error can't find the container with id 9a57d57931016bff61e67ec0e8137f80718ea231cd302d708c19ae497a38df4b Dec 03 13:00:24 crc kubenswrapper[5052]: I1203 13:00:24.440841 5052 generic.go:334] "Generic (PLEG): container finished" podID="08ab2cda-3a42-49b5-843e-6c8b151064df" containerID="a310ad98499337b886702c931f5d7c2b6343449dfab7c440a0ecf22708504a26" exitCode=0 Dec 03 13:00:24 crc kubenswrapper[5052]: I1203 13:00:24.440900 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-x6wdj" Dec 03 13:00:24 crc kubenswrapper[5052]: I1203 13:00:24.440916 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-x6wdj" event={"ID":"08ab2cda-3a42-49b5-843e-6c8b151064df","Type":"ContainerDied","Data":"a310ad98499337b886702c931f5d7c2b6343449dfab7c440a0ecf22708504a26"} Dec 03 13:00:24 crc kubenswrapper[5052]: I1203 13:00:24.440944 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-x6wdj" event={"ID":"08ab2cda-3a42-49b5-843e-6c8b151064df","Type":"ContainerDied","Data":"84da4985770c03243888de71571bd4d9a4a900430d1974aa54555c6e639a529d"} Dec 03 13:00:24 crc kubenswrapper[5052]: I1203 13:00:24.440966 5052 scope.go:117] "RemoveContainer" containerID="a310ad98499337b886702c931f5d7c2b6343449dfab7c440a0ecf22708504a26" Dec 03 13:00:24 crc kubenswrapper[5052]: I1203 13:00:24.443816 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-76b7c4d945-lxqvh" event={"ID":"d697b768-3215-466f-9438-109e015ad8ed","Type":"ContainerStarted","Data":"9a57d57931016bff61e67ec0e8137f80718ea231cd302d708c19ae497a38df4b"} Dec 03 13:00:24 crc kubenswrapper[5052]: I1203 13:00:24.461531 5052 scope.go:117] "RemoveContainer" containerID="77898eab798d2ba81ccac49f52ddc5c77eafc3853bfd6053624c031d324209fb" Dec 03 13:00:24 crc kubenswrapper[5052]: I1203 13:00:24.473138 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-x6wdj"] Dec 03 13:00:24 crc kubenswrapper[5052]: I1203 13:00:24.480815 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-x6wdj"] Dec 03 13:00:24 crc kubenswrapper[5052]: I1203 13:00:24.495328 5052 scope.go:117] "RemoveContainer" containerID="a310ad98499337b886702c931f5d7c2b6343449dfab7c440a0ecf22708504a26" Dec 03 13:00:24 crc kubenswrapper[5052]: E1203 13:00:24.495831 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a310ad98499337b886702c931f5d7c2b6343449dfab7c440a0ecf22708504a26\": container with ID starting with a310ad98499337b886702c931f5d7c2b6343449dfab7c440a0ecf22708504a26 not found: ID does not exist" containerID="a310ad98499337b886702c931f5d7c2b6343449dfab7c440a0ecf22708504a26" Dec 03 13:00:24 crc kubenswrapper[5052]: I1203 13:00:24.495873 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a310ad98499337b886702c931f5d7c2b6343449dfab7c440a0ecf22708504a26"} err="failed to get container status \"a310ad98499337b886702c931f5d7c2b6343449dfab7c440a0ecf22708504a26\": rpc error: code = NotFound desc = could not find container \"a310ad98499337b886702c931f5d7c2b6343449dfab7c440a0ecf22708504a26\": container with ID starting with a310ad98499337b886702c931f5d7c2b6343449dfab7c440a0ecf22708504a26 not found: ID does not exist" Dec 03 13:00:24 crc kubenswrapper[5052]: I1203 13:00:24.495896 5052 scope.go:117] "RemoveContainer" containerID="77898eab798d2ba81ccac49f52ddc5c77eafc3853bfd6053624c031d324209fb" Dec 03 13:00:24 crc kubenswrapper[5052]: E1203 13:00:24.496195 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"77898eab798d2ba81ccac49f52ddc5c77eafc3853bfd6053624c031d324209fb\": container with ID starting with 77898eab798d2ba81ccac49f52ddc5c77eafc3853bfd6053624c031d324209fb not found: ID does not exist" containerID="77898eab798d2ba81ccac49f52ddc5c77eafc3853bfd6053624c031d324209fb" Dec 03 13:00:24 crc kubenswrapper[5052]: I1203 13:00:24.496222 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"77898eab798d2ba81ccac49f52ddc5c77eafc3853bfd6053624c031d324209fb"} err="failed to get container status \"77898eab798d2ba81ccac49f52ddc5c77eafc3853bfd6053624c031d324209fb\": rpc error: code = NotFound desc = could not find container \"77898eab798d2ba81ccac49f52ddc5c77eafc3853bfd6053624c031d324209fb\": container with ID starting with 77898eab798d2ba81ccac49f52ddc5c77eafc3853bfd6053624c031d324209fb not found: ID does not exist" Dec 03 13:00:25 crc kubenswrapper[5052]: I1203 13:00:25.453883 5052 generic.go:334] "Generic (PLEG): container finished" podID="d697b768-3215-466f-9438-109e015ad8ed" containerID="e10e081ec82236913927fb544daf6e2482f9dbce23297f7d171a2be24f3207b2" exitCode=0 Dec 03 13:00:25 crc kubenswrapper[5052]: I1203 13:00:25.453929 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-76b7c4d945-lxqvh" event={"ID":"d697b768-3215-466f-9438-109e015ad8ed","Type":"ContainerDied","Data":"e10e081ec82236913927fb544daf6e2482f9dbce23297f7d171a2be24f3207b2"} Dec 03 13:00:25 crc kubenswrapper[5052]: I1203 13:00:25.681040 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="08ab2cda-3a42-49b5-843e-6c8b151064df" path="/var/lib/kubelet/pods/08ab2cda-3a42-49b5-843e-6c8b151064df/volumes" Dec 03 13:00:26 crc kubenswrapper[5052]: I1203 13:00:26.468437 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-76b7c4d945-lxqvh" event={"ID":"d697b768-3215-466f-9438-109e015ad8ed","Type":"ContainerStarted","Data":"ed0e4cbe31266dd420112b87e2a8219db6b9f7feb304395f959db983cfc1e6e3"} Dec 03 13:00:26 crc kubenswrapper[5052]: I1203 13:00:26.468906 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-76b7c4d945-lxqvh" Dec 03 13:00:26 crc kubenswrapper[5052]: I1203 13:00:26.489599 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-76b7c4d945-lxqvh" podStartSLOduration=3.489581374 podStartE2EDuration="3.489581374s" podCreationTimestamp="2025-12-03 13:00:23 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 13:00:26.487693911 +0000 UTC m=+6017.075935685" watchObservedRunningTime="2025-12-03 13:00:26.489581374 +0000 UTC m=+6017.077823138" Dec 03 13:00:33 crc kubenswrapper[5052]: I1203 13:00:33.981873 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-76b7c4d945-lxqvh" Dec 03 13:00:34 crc kubenswrapper[5052]: I1203 13:00:34.047923 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-57877c5b47-lsnhg"] Dec 03 13:00:34 crc kubenswrapper[5052]: I1203 13:00:34.048200 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-57877c5b47-lsnhg" podUID="5b86081b-240a-4b7e-9ee4-20f72ed39a31" containerName="dnsmasq-dns" containerID="cri-o://1ee85c805793220260e3b321df2901ec536fb275df4bf8d10d50e4c9d78710ad" gracePeriod=10 Dec 03 13:00:34 crc kubenswrapper[5052]: I1203 13:00:34.419769 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-57877c5b47-lsnhg" Dec 03 13:00:34 crc kubenswrapper[5052]: I1203 13:00:34.545426 5052 generic.go:334] "Generic (PLEG): container finished" podID="5b86081b-240a-4b7e-9ee4-20f72ed39a31" containerID="1ee85c805793220260e3b321df2901ec536fb275df4bf8d10d50e4c9d78710ad" exitCode=0 Dec 03 13:00:34 crc kubenswrapper[5052]: I1203 13:00:34.545482 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-57877c5b47-lsnhg" event={"ID":"5b86081b-240a-4b7e-9ee4-20f72ed39a31","Type":"ContainerDied","Data":"1ee85c805793220260e3b321df2901ec536fb275df4bf8d10d50e4c9d78710ad"} Dec 03 13:00:34 crc kubenswrapper[5052]: I1203 13:00:34.545488 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-57877c5b47-lsnhg" Dec 03 13:00:34 crc kubenswrapper[5052]: I1203 13:00:34.545563 5052 scope.go:117] "RemoveContainer" containerID="1ee85c805793220260e3b321df2901ec536fb275df4bf8d10d50e4c9d78710ad" Dec 03 13:00:34 crc kubenswrapper[5052]: I1203 13:00:34.545547 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-57877c5b47-lsnhg" event={"ID":"5b86081b-240a-4b7e-9ee4-20f72ed39a31","Type":"ContainerDied","Data":"c376bb755a72d0205fd01f437b1fdabb7130d27a365d5d581a7503bb6d059b5f"} Dec 03 13:00:34 crc kubenswrapper[5052]: I1203 13:00:34.567800 5052 scope.go:117] "RemoveContainer" containerID="6c20b742092df772325f108139591f4ea6f2995c9026f65963a0dd2d7e4de316" Dec 03 13:00:34 crc kubenswrapper[5052]: I1203 13:00:34.588989 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5b86081b-240a-4b7e-9ee4-20f72ed39a31-config\") pod \"5b86081b-240a-4b7e-9ee4-20f72ed39a31\" (UID: \"5b86081b-240a-4b7e-9ee4-20f72ed39a31\") " Dec 03 13:00:34 crc kubenswrapper[5052]: I1203 13:00:34.589095 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/5b86081b-240a-4b7e-9ee4-20f72ed39a31-dnsmasq-svc\") pod \"5b86081b-240a-4b7e-9ee4-20f72ed39a31\" (UID: \"5b86081b-240a-4b7e-9ee4-20f72ed39a31\") " Dec 03 13:00:34 crc kubenswrapper[5052]: I1203 13:00:34.589135 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"openstack-edpm-tls\" (UniqueName: \"kubernetes.io/configmap/5b86081b-240a-4b7e-9ee4-20f72ed39a31-openstack-edpm-tls\") pod \"5b86081b-240a-4b7e-9ee4-20f72ed39a31\" (UID: \"5b86081b-240a-4b7e-9ee4-20f72ed39a31\") " Dec 03 13:00:34 crc kubenswrapper[5052]: I1203 13:00:34.589170 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dzt4n\" (UniqueName: \"kubernetes.io/projected/5b86081b-240a-4b7e-9ee4-20f72ed39a31-kube-api-access-dzt4n\") pod \"5b86081b-240a-4b7e-9ee4-20f72ed39a31\" (UID: \"5b86081b-240a-4b7e-9ee4-20f72ed39a31\") " Dec 03 13:00:34 crc kubenswrapper[5052]: I1203 13:00:34.589783 5052 scope.go:117] "RemoveContainer" containerID="1ee85c805793220260e3b321df2901ec536fb275df4bf8d10d50e4c9d78710ad" Dec 03 13:00:34 crc kubenswrapper[5052]: E1203 13:00:34.590682 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1ee85c805793220260e3b321df2901ec536fb275df4bf8d10d50e4c9d78710ad\": container with ID starting with 1ee85c805793220260e3b321df2901ec536fb275df4bf8d10d50e4c9d78710ad not found: ID does not exist" containerID="1ee85c805793220260e3b321df2901ec536fb275df4bf8d10d50e4c9d78710ad" Dec 03 13:00:34 crc kubenswrapper[5052]: I1203 13:00:34.590736 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1ee85c805793220260e3b321df2901ec536fb275df4bf8d10d50e4c9d78710ad"} err="failed to get container status \"1ee85c805793220260e3b321df2901ec536fb275df4bf8d10d50e4c9d78710ad\": rpc error: code = NotFound desc = could not find container \"1ee85c805793220260e3b321df2901ec536fb275df4bf8d10d50e4c9d78710ad\": container with ID starting with 1ee85c805793220260e3b321df2901ec536fb275df4bf8d10d50e4c9d78710ad not found: ID does not exist" Dec 03 13:00:34 crc kubenswrapper[5052]: I1203 13:00:34.590765 5052 scope.go:117] "RemoveContainer" containerID="6c20b742092df772325f108139591f4ea6f2995c9026f65963a0dd2d7e4de316" Dec 03 13:00:34 crc kubenswrapper[5052]: E1203 13:00:34.591092 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6c20b742092df772325f108139591f4ea6f2995c9026f65963a0dd2d7e4de316\": container with ID starting with 6c20b742092df772325f108139591f4ea6f2995c9026f65963a0dd2d7e4de316 not found: ID does not exist" containerID="6c20b742092df772325f108139591f4ea6f2995c9026f65963a0dd2d7e4de316" Dec 03 13:00:34 crc kubenswrapper[5052]: I1203 13:00:34.591119 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6c20b742092df772325f108139591f4ea6f2995c9026f65963a0dd2d7e4de316"} err="failed to get container status \"6c20b742092df772325f108139591f4ea6f2995c9026f65963a0dd2d7e4de316\": rpc error: code = NotFound desc = could not find container \"6c20b742092df772325f108139591f4ea6f2995c9026f65963a0dd2d7e4de316\": container with ID starting with 6c20b742092df772325f108139591f4ea6f2995c9026f65963a0dd2d7e4de316 not found: ID does not exist" Dec 03 13:00:34 crc kubenswrapper[5052]: I1203 13:00:34.594910 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5b86081b-240a-4b7e-9ee4-20f72ed39a31-kube-api-access-dzt4n" (OuterVolumeSpecName: "kube-api-access-dzt4n") pod "5b86081b-240a-4b7e-9ee4-20f72ed39a31" (UID: "5b86081b-240a-4b7e-9ee4-20f72ed39a31"). InnerVolumeSpecName "kube-api-access-dzt4n". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:00:34 crc kubenswrapper[5052]: I1203 13:00:34.626360 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5b86081b-240a-4b7e-9ee4-20f72ed39a31-dnsmasq-svc" (OuterVolumeSpecName: "dnsmasq-svc") pod "5b86081b-240a-4b7e-9ee4-20f72ed39a31" (UID: "5b86081b-240a-4b7e-9ee4-20f72ed39a31"). InnerVolumeSpecName "dnsmasq-svc". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:00:34 crc kubenswrapper[5052]: I1203 13:00:34.628189 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5b86081b-240a-4b7e-9ee4-20f72ed39a31-config" (OuterVolumeSpecName: "config") pod "5b86081b-240a-4b7e-9ee4-20f72ed39a31" (UID: "5b86081b-240a-4b7e-9ee4-20f72ed39a31"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:00:34 crc kubenswrapper[5052]: I1203 13:00:34.629822 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5b86081b-240a-4b7e-9ee4-20f72ed39a31-openstack-edpm-tls" (OuterVolumeSpecName: "openstack-edpm-tls") pod "5b86081b-240a-4b7e-9ee4-20f72ed39a31" (UID: "5b86081b-240a-4b7e-9ee4-20f72ed39a31"). InnerVolumeSpecName "openstack-edpm-tls". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:00:34 crc kubenswrapper[5052]: I1203 13:00:34.690729 5052 reconciler_common.go:293] "Volume detached for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/5b86081b-240a-4b7e-9ee4-20f72ed39a31-dnsmasq-svc\") on node \"crc\" DevicePath \"\"" Dec 03 13:00:34 crc kubenswrapper[5052]: I1203 13:00:34.690812 5052 reconciler_common.go:293] "Volume detached for volume \"openstack-edpm-tls\" (UniqueName: \"kubernetes.io/configmap/5b86081b-240a-4b7e-9ee4-20f72ed39a31-openstack-edpm-tls\") on node \"crc\" DevicePath \"\"" Dec 03 13:00:34 crc kubenswrapper[5052]: I1203 13:00:34.690899 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-dzt4n\" (UniqueName: \"kubernetes.io/projected/5b86081b-240a-4b7e-9ee4-20f72ed39a31-kube-api-access-dzt4n\") on node \"crc\" DevicePath \"\"" Dec 03 13:00:34 crc kubenswrapper[5052]: I1203 13:00:34.690915 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5b86081b-240a-4b7e-9ee4-20f72ed39a31-config\") on node \"crc\" DevicePath \"\"" Dec 03 13:00:34 crc kubenswrapper[5052]: I1203 13:00:34.878665 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-57877c5b47-lsnhg"] Dec 03 13:00:34 crc kubenswrapper[5052]: I1203 13:00:34.885837 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-57877c5b47-lsnhg"] Dec 03 13:00:35 crc kubenswrapper[5052]: I1203 13:00:35.684804 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5b86081b-240a-4b7e-9ee4-20f72ed39a31" path="/var/lib/kubelet/pods/5b86081b-240a-4b7e-9ee4-20f72ed39a31/volumes" Dec 03 13:00:38 crc kubenswrapper[5052]: I1203 13:00:38.086322 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/install-certs-ovr-openstack-edpm-tls-openstack-edpm-tls-2f5qn"] Dec 03 13:00:38 crc kubenswrapper[5052]: E1203 13:00:38.087153 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="5b86081b-240a-4b7e-9ee4-20f72ed39a31" containerName="init" Dec 03 13:00:38 crc kubenswrapper[5052]: I1203 13:00:38.087167 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="5b86081b-240a-4b7e-9ee4-20f72ed39a31" containerName="init" Dec 03 13:00:38 crc kubenswrapper[5052]: E1203 13:00:38.087180 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="08ab2cda-3a42-49b5-843e-6c8b151064df" containerName="dnsmasq-dns" Dec 03 13:00:38 crc kubenswrapper[5052]: I1203 13:00:38.087188 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="08ab2cda-3a42-49b5-843e-6c8b151064df" containerName="dnsmasq-dns" Dec 03 13:00:38 crc kubenswrapper[5052]: E1203 13:00:38.087214 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="08ab2cda-3a42-49b5-843e-6c8b151064df" containerName="init" Dec 03 13:00:38 crc kubenswrapper[5052]: I1203 13:00:38.087221 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="08ab2cda-3a42-49b5-843e-6c8b151064df" containerName="init" Dec 03 13:00:38 crc kubenswrapper[5052]: E1203 13:00:38.087243 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="5b86081b-240a-4b7e-9ee4-20f72ed39a31" containerName="dnsmasq-dns" Dec 03 13:00:38 crc kubenswrapper[5052]: I1203 13:00:38.087250 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="5b86081b-240a-4b7e-9ee4-20f72ed39a31" containerName="dnsmasq-dns" Dec 03 13:00:38 crc kubenswrapper[5052]: I1203 13:00:38.087433 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="5b86081b-240a-4b7e-9ee4-20f72ed39a31" containerName="dnsmasq-dns" Dec 03 13:00:38 crc kubenswrapper[5052]: I1203 13:00:38.087464 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="08ab2cda-3a42-49b5-843e-6c8b151064df" containerName="dnsmasq-dns" Dec 03 13:00:38 crc kubenswrapper[5052]: I1203 13:00:38.088155 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/install-certs-ovr-openstack-edpm-tls-openstack-edpm-tls-2f5qn" Dec 03 13:00:38 crc kubenswrapper[5052]: I1203 13:00:38.090960 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"dataplanenodeset-openstack-edpm-tls" Dec 03 13:00:38 crc kubenswrapper[5052]: I1203 13:00:38.092893 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"openstack-edpm-tls-dockercfg-5dddj" Dec 03 13:00:38 crc kubenswrapper[5052]: I1203 13:00:38.092911 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"openstack-edpm-tls-generic-service1-default-certs-1" Dec 03 13:00:38 crc kubenswrapper[5052]: I1203 13:00:38.093125 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"dataplane-ansible-ssh-private-key-secret" Dec 03 13:00:38 crc kubenswrapper[5052]: I1203 13:00:38.093661 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"openstack-aee-default-env" Dec 03 13:00:38 crc kubenswrapper[5052]: I1203 13:00:38.093820 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"openstack-edpm-tls-generic-service1-default-certs-2" Dec 03 13:00:38 crc kubenswrapper[5052]: I1203 13:00:38.093943 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"openstack-edpm-tls-generic-service1-default-certs-0" Dec 03 13:00:38 crc kubenswrapper[5052]: I1203 13:00:38.094066 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"combined-ca-bundle" Dec 03 13:00:38 crc kubenswrapper[5052]: I1203 13:00:38.101192 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/install-certs-ovr-openstack-edpm-tls-openstack-edpm-tls-2f5qn"] Dec 03 13:00:38 crc kubenswrapper[5052]: I1203 13:00:38.141387 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"install-certs-ovr-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8193174d-9636-4fc6-97bb-1fd0baba30e4-install-certs-ovr-combined-ca-bundle\") pod \"install-certs-ovr-openstack-edpm-tls-openstack-edpm-tls-2f5qn\" (UID: \"8193174d-9636-4fc6-97bb-1fd0baba30e4\") " pod="openstack-kuttl-tests/install-certs-ovr-openstack-edpm-tls-openstack-edpm-tls-2f5qn" Dec 03 13:00:38 crc kubenswrapper[5052]: I1203 13:00:38.141444 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"generic-service1-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8193174d-9636-4fc6-97bb-1fd0baba30e4-generic-service1-combined-ca-bundle\") pod \"install-certs-ovr-openstack-edpm-tls-openstack-edpm-tls-2f5qn\" (UID: \"8193174d-9636-4fc6-97bb-1fd0baba30e4\") " pod="openstack-kuttl-tests/install-certs-ovr-openstack-edpm-tls-openstack-edpm-tls-2f5qn" Dec 03 13:00:38 crc kubenswrapper[5052]: I1203 13:00:38.141506 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/8193174d-9636-4fc6-97bb-1fd0baba30e4-ssh-key\") pod \"install-certs-ovr-openstack-edpm-tls-openstack-edpm-tls-2f5qn\" (UID: \"8193174d-9636-4fc6-97bb-1fd0baba30e4\") " pod="openstack-kuttl-tests/install-certs-ovr-openstack-edpm-tls-openstack-edpm-tls-2f5qn" Dec 03 13:00:38 crc kubenswrapper[5052]: I1203 13:00:38.141605 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vp6mc\" (UniqueName: \"kubernetes.io/projected/8193174d-9636-4fc6-97bb-1fd0baba30e4-kube-api-access-vp6mc\") pod \"install-certs-ovr-openstack-edpm-tls-openstack-edpm-tls-2f5qn\" (UID: \"8193174d-9636-4fc6-97bb-1fd0baba30e4\") " pod="openstack-kuttl-tests/install-certs-ovr-openstack-edpm-tls-openstack-edpm-tls-2f5qn" Dec 03 13:00:38 crc kubenswrapper[5052]: I1203 13:00:38.141631 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/8193174d-9636-4fc6-97bb-1fd0baba30e4-inventory\") pod \"install-certs-ovr-openstack-edpm-tls-openstack-edpm-tls-2f5qn\" (UID: \"8193174d-9636-4fc6-97bb-1fd0baba30e4\") " pod="openstack-kuttl-tests/install-certs-ovr-openstack-edpm-tls-openstack-edpm-tls-2f5qn" Dec 03 13:00:38 crc kubenswrapper[5052]: I1203 13:00:38.141656 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openstack-edpm-tls-generic-service1-default-certs-0\" (UniqueName: \"kubernetes.io/projected/8193174d-9636-4fc6-97bb-1fd0baba30e4-openstack-edpm-tls-generic-service1-default-certs-0\") pod \"install-certs-ovr-openstack-edpm-tls-openstack-edpm-tls-2f5qn\" (UID: \"8193174d-9636-4fc6-97bb-1fd0baba30e4\") " pod="openstack-kuttl-tests/install-certs-ovr-openstack-edpm-tls-openstack-edpm-tls-2f5qn" Dec 03 13:00:38 crc kubenswrapper[5052]: I1203 13:00:38.242585 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"install-certs-ovr-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8193174d-9636-4fc6-97bb-1fd0baba30e4-install-certs-ovr-combined-ca-bundle\") pod \"install-certs-ovr-openstack-edpm-tls-openstack-edpm-tls-2f5qn\" (UID: \"8193174d-9636-4fc6-97bb-1fd0baba30e4\") " pod="openstack-kuttl-tests/install-certs-ovr-openstack-edpm-tls-openstack-edpm-tls-2f5qn" Dec 03 13:00:38 crc kubenswrapper[5052]: I1203 13:00:38.242641 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"generic-service1-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8193174d-9636-4fc6-97bb-1fd0baba30e4-generic-service1-combined-ca-bundle\") pod \"install-certs-ovr-openstack-edpm-tls-openstack-edpm-tls-2f5qn\" (UID: \"8193174d-9636-4fc6-97bb-1fd0baba30e4\") " pod="openstack-kuttl-tests/install-certs-ovr-openstack-edpm-tls-openstack-edpm-tls-2f5qn" Dec 03 13:00:38 crc kubenswrapper[5052]: I1203 13:00:38.242721 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/8193174d-9636-4fc6-97bb-1fd0baba30e4-ssh-key\") pod \"install-certs-ovr-openstack-edpm-tls-openstack-edpm-tls-2f5qn\" (UID: \"8193174d-9636-4fc6-97bb-1fd0baba30e4\") " pod="openstack-kuttl-tests/install-certs-ovr-openstack-edpm-tls-openstack-edpm-tls-2f5qn" Dec 03 13:00:38 crc kubenswrapper[5052]: I1203 13:00:38.242755 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-vp6mc\" (UniqueName: \"kubernetes.io/projected/8193174d-9636-4fc6-97bb-1fd0baba30e4-kube-api-access-vp6mc\") pod \"install-certs-ovr-openstack-edpm-tls-openstack-edpm-tls-2f5qn\" (UID: \"8193174d-9636-4fc6-97bb-1fd0baba30e4\") " pod="openstack-kuttl-tests/install-certs-ovr-openstack-edpm-tls-openstack-edpm-tls-2f5qn" Dec 03 13:00:38 crc kubenswrapper[5052]: I1203 13:00:38.242778 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/8193174d-9636-4fc6-97bb-1fd0baba30e4-inventory\") pod \"install-certs-ovr-openstack-edpm-tls-openstack-edpm-tls-2f5qn\" (UID: \"8193174d-9636-4fc6-97bb-1fd0baba30e4\") " pod="openstack-kuttl-tests/install-certs-ovr-openstack-edpm-tls-openstack-edpm-tls-2f5qn" Dec 03 13:00:38 crc kubenswrapper[5052]: I1203 13:00:38.242804 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"openstack-edpm-tls-generic-service1-default-certs-0\" (UniqueName: \"kubernetes.io/projected/8193174d-9636-4fc6-97bb-1fd0baba30e4-openstack-edpm-tls-generic-service1-default-certs-0\") pod \"install-certs-ovr-openstack-edpm-tls-openstack-edpm-tls-2f5qn\" (UID: \"8193174d-9636-4fc6-97bb-1fd0baba30e4\") " pod="openstack-kuttl-tests/install-certs-ovr-openstack-edpm-tls-openstack-edpm-tls-2f5qn" Dec 03 13:00:38 crc kubenswrapper[5052]: I1203 13:00:38.248810 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/8193174d-9636-4fc6-97bb-1fd0baba30e4-ssh-key\") pod \"install-certs-ovr-openstack-edpm-tls-openstack-edpm-tls-2f5qn\" (UID: \"8193174d-9636-4fc6-97bb-1fd0baba30e4\") " pod="openstack-kuttl-tests/install-certs-ovr-openstack-edpm-tls-openstack-edpm-tls-2f5qn" Dec 03 13:00:38 crc kubenswrapper[5052]: I1203 13:00:38.249327 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"generic-service1-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8193174d-9636-4fc6-97bb-1fd0baba30e4-generic-service1-combined-ca-bundle\") pod \"install-certs-ovr-openstack-edpm-tls-openstack-edpm-tls-2f5qn\" (UID: \"8193174d-9636-4fc6-97bb-1fd0baba30e4\") " pod="openstack-kuttl-tests/install-certs-ovr-openstack-edpm-tls-openstack-edpm-tls-2f5qn" Dec 03 13:00:38 crc kubenswrapper[5052]: I1203 13:00:38.249852 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/8193174d-9636-4fc6-97bb-1fd0baba30e4-inventory\") pod \"install-certs-ovr-openstack-edpm-tls-openstack-edpm-tls-2f5qn\" (UID: \"8193174d-9636-4fc6-97bb-1fd0baba30e4\") " pod="openstack-kuttl-tests/install-certs-ovr-openstack-edpm-tls-openstack-edpm-tls-2f5qn" Dec 03 13:00:38 crc kubenswrapper[5052]: I1203 13:00:38.254144 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"openstack-edpm-tls-generic-service1-default-certs-0\" (UniqueName: \"kubernetes.io/projected/8193174d-9636-4fc6-97bb-1fd0baba30e4-openstack-edpm-tls-generic-service1-default-certs-0\") pod \"install-certs-ovr-openstack-edpm-tls-openstack-edpm-tls-2f5qn\" (UID: \"8193174d-9636-4fc6-97bb-1fd0baba30e4\") " pod="openstack-kuttl-tests/install-certs-ovr-openstack-edpm-tls-openstack-edpm-tls-2f5qn" Dec 03 13:00:38 crc kubenswrapper[5052]: I1203 13:00:38.254499 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"install-certs-ovr-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8193174d-9636-4fc6-97bb-1fd0baba30e4-install-certs-ovr-combined-ca-bundle\") pod \"install-certs-ovr-openstack-edpm-tls-openstack-edpm-tls-2f5qn\" (UID: \"8193174d-9636-4fc6-97bb-1fd0baba30e4\") " pod="openstack-kuttl-tests/install-certs-ovr-openstack-edpm-tls-openstack-edpm-tls-2f5qn" Dec 03 13:00:38 crc kubenswrapper[5052]: I1203 13:00:38.263662 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-vp6mc\" (UniqueName: \"kubernetes.io/projected/8193174d-9636-4fc6-97bb-1fd0baba30e4-kube-api-access-vp6mc\") pod \"install-certs-ovr-openstack-edpm-tls-openstack-edpm-tls-2f5qn\" (UID: \"8193174d-9636-4fc6-97bb-1fd0baba30e4\") " pod="openstack-kuttl-tests/install-certs-ovr-openstack-edpm-tls-openstack-edpm-tls-2f5qn" Dec 03 13:00:38 crc kubenswrapper[5052]: I1203 13:00:38.409697 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/install-certs-ovr-openstack-edpm-tls-openstack-edpm-tls-2f5qn" Dec 03 13:00:38 crc kubenswrapper[5052]: I1203 13:00:38.664419 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/install-certs-ovr-openstack-edpm-tls-openstack-edpm-tls-2f5qn"] Dec 03 13:00:39 crc kubenswrapper[5052]: I1203 13:00:39.594223 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/install-certs-ovr-openstack-edpm-tls-openstack-edpm-tls-2f5qn" event={"ID":"8193174d-9636-4fc6-97bb-1fd0baba30e4","Type":"ContainerStarted","Data":"5a57d728d1afb3e7fc17f47c0c7e311b0f2422272ae2681700e97b1ccd03fa1d"} Dec 03 13:00:39 crc kubenswrapper[5052]: I1203 13:00:39.594584 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/install-certs-ovr-openstack-edpm-tls-openstack-edpm-tls-2f5qn" event={"ID":"8193174d-9636-4fc6-97bb-1fd0baba30e4","Type":"ContainerStarted","Data":"08d63c7fd4fb650411d69fbc8e517cf4e70348852f345cb01d6203b6efb00544"} Dec 03 13:00:39 crc kubenswrapper[5052]: I1203 13:00:39.609866 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/install-certs-ovr-openstack-edpm-tls-openstack-edpm-tls-2f5qn" podStartSLOduration=1.000822164 podStartE2EDuration="1.609849351s" podCreationTimestamp="2025-12-03 13:00:38 +0000 UTC" firstStartedPulling="2025-12-03 13:00:38.676582111 +0000 UTC m=+6029.264823885" lastFinishedPulling="2025-12-03 13:00:39.285609308 +0000 UTC m=+6029.873851072" observedRunningTime="2025-12-03 13:00:39.60772954 +0000 UTC m=+6030.195971304" watchObservedRunningTime="2025-12-03 13:00:39.609849351 +0000 UTC m=+6030.198091115" Dec 03 13:00:39 crc kubenswrapper[5052]: I1203 13:00:39.658860 5052 scope.go:117] "RemoveContainer" containerID="72785ef6236001625d6d7f1e690671c78dc1f11c3518cfcc3bf9d6de5ab4c2fb" Dec 03 13:00:39 crc kubenswrapper[5052]: I1203 13:00:39.680359 5052 scope.go:117] "RemoveContainer" containerID="52ef14e30c84a7e38ff36a4c93fbfa8a565c2ed55f66905b14b2fdd87cd69077" Dec 03 13:00:39 crc kubenswrapper[5052]: I1203 13:00:39.699495 5052 scope.go:117] "RemoveContainer" containerID="59b945b4df7623c03eb622cbe50e816dadd5224b5097d5e7e646cca960ca90e1" Dec 03 13:00:39 crc kubenswrapper[5052]: I1203 13:00:39.718500 5052 scope.go:117] "RemoveContainer" containerID="3e4782da22f3455b238f322c1423c58483f7f0f10c60f639fa7854c8bfb97b96" Dec 03 13:00:39 crc kubenswrapper[5052]: I1203 13:00:39.738556 5052 scope.go:117] "RemoveContainer" containerID="f15af8c0a8823cb666fbe94253b10ca1d40d1a5303a03c6ac25035b6456a7a85" Dec 03 13:00:39 crc kubenswrapper[5052]: I1203 13:00:39.771457 5052 scope.go:117] "RemoveContainer" containerID="86ed75c4a9b117c08c2a3ecec9878ef557fc9d6d68fea18aa83751770d37d20b" Dec 03 13:00:39 crc kubenswrapper[5052]: I1203 13:00:39.794465 5052 scope.go:117] "RemoveContainer" containerID="990e0730d2387a55431e533e2f7232d5369fd95354ef6d99da9341e99b6b7c58" Dec 03 13:00:42 crc kubenswrapper[5052]: I1203 13:00:42.622948 5052 generic.go:334] "Generic (PLEG): container finished" podID="8193174d-9636-4fc6-97bb-1fd0baba30e4" containerID="5a57d728d1afb3e7fc17f47c0c7e311b0f2422272ae2681700e97b1ccd03fa1d" exitCode=0 Dec 03 13:00:42 crc kubenswrapper[5052]: I1203 13:00:42.623031 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/install-certs-ovr-openstack-edpm-tls-openstack-edpm-tls-2f5qn" event={"ID":"8193174d-9636-4fc6-97bb-1fd0baba30e4","Type":"ContainerDied","Data":"5a57d728d1afb3e7fc17f47c0c7e311b0f2422272ae2681700e97b1ccd03fa1d"} Dec 03 13:00:43 crc kubenswrapper[5052]: I1203 13:00:43.893606 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/install-certs-ovr-openstack-edpm-tls-openstack-edpm-tls-2f5qn" Dec 03 13:00:43 crc kubenswrapper[5052]: I1203 13:00:43.938977 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"generic-service1-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8193174d-9636-4fc6-97bb-1fd0baba30e4-generic-service1-combined-ca-bundle\") pod \"8193174d-9636-4fc6-97bb-1fd0baba30e4\" (UID: \"8193174d-9636-4fc6-97bb-1fd0baba30e4\") " Dec 03 13:00:43 crc kubenswrapper[5052]: I1203 13:00:43.939088 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/8193174d-9636-4fc6-97bb-1fd0baba30e4-ssh-key\") pod \"8193174d-9636-4fc6-97bb-1fd0baba30e4\" (UID: \"8193174d-9636-4fc6-97bb-1fd0baba30e4\") " Dec 03 13:00:43 crc kubenswrapper[5052]: I1203 13:00:43.939129 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vp6mc\" (UniqueName: \"kubernetes.io/projected/8193174d-9636-4fc6-97bb-1fd0baba30e4-kube-api-access-vp6mc\") pod \"8193174d-9636-4fc6-97bb-1fd0baba30e4\" (UID: \"8193174d-9636-4fc6-97bb-1fd0baba30e4\") " Dec 03 13:00:43 crc kubenswrapper[5052]: I1203 13:00:43.939154 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/8193174d-9636-4fc6-97bb-1fd0baba30e4-inventory\") pod \"8193174d-9636-4fc6-97bb-1fd0baba30e4\" (UID: \"8193174d-9636-4fc6-97bb-1fd0baba30e4\") " Dec 03 13:00:43 crc kubenswrapper[5052]: I1203 13:00:43.939192 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"openstack-edpm-tls-generic-service1-default-certs-0\" (UniqueName: \"kubernetes.io/projected/8193174d-9636-4fc6-97bb-1fd0baba30e4-openstack-edpm-tls-generic-service1-default-certs-0\") pod \"8193174d-9636-4fc6-97bb-1fd0baba30e4\" (UID: \"8193174d-9636-4fc6-97bb-1fd0baba30e4\") " Dec 03 13:00:43 crc kubenswrapper[5052]: I1203 13:00:43.939222 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"install-certs-ovr-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8193174d-9636-4fc6-97bb-1fd0baba30e4-install-certs-ovr-combined-ca-bundle\") pod \"8193174d-9636-4fc6-97bb-1fd0baba30e4\" (UID: \"8193174d-9636-4fc6-97bb-1fd0baba30e4\") " Dec 03 13:00:43 crc kubenswrapper[5052]: I1203 13:00:43.946223 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8193174d-9636-4fc6-97bb-1fd0baba30e4-install-certs-ovr-combined-ca-bundle" (OuterVolumeSpecName: "install-certs-ovr-combined-ca-bundle") pod "8193174d-9636-4fc6-97bb-1fd0baba30e4" (UID: "8193174d-9636-4fc6-97bb-1fd0baba30e4"). InnerVolumeSpecName "install-certs-ovr-combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:00:43 crc kubenswrapper[5052]: I1203 13:00:43.946843 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8193174d-9636-4fc6-97bb-1fd0baba30e4-openstack-edpm-tls-generic-service1-default-certs-0" (OuterVolumeSpecName: "openstack-edpm-tls-generic-service1-default-certs-0") pod "8193174d-9636-4fc6-97bb-1fd0baba30e4" (UID: "8193174d-9636-4fc6-97bb-1fd0baba30e4"). InnerVolumeSpecName "openstack-edpm-tls-generic-service1-default-certs-0". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:00:43 crc kubenswrapper[5052]: I1203 13:00:43.947240 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8193174d-9636-4fc6-97bb-1fd0baba30e4-kube-api-access-vp6mc" (OuterVolumeSpecName: "kube-api-access-vp6mc") pod "8193174d-9636-4fc6-97bb-1fd0baba30e4" (UID: "8193174d-9636-4fc6-97bb-1fd0baba30e4"). InnerVolumeSpecName "kube-api-access-vp6mc". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:00:43 crc kubenswrapper[5052]: I1203 13:00:43.948659 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8193174d-9636-4fc6-97bb-1fd0baba30e4-generic-service1-combined-ca-bundle" (OuterVolumeSpecName: "generic-service1-combined-ca-bundle") pod "8193174d-9636-4fc6-97bb-1fd0baba30e4" (UID: "8193174d-9636-4fc6-97bb-1fd0baba30e4"). InnerVolumeSpecName "generic-service1-combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:00:43 crc kubenswrapper[5052]: E1203 13:00:43.960849 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8193174d-9636-4fc6-97bb-1fd0baba30e4-ssh-key podName:8193174d-9636-4fc6-97bb-1fd0baba30e4 nodeName:}" failed. No retries permitted until 2025-12-03 13:00:44.46081761 +0000 UTC m=+6035.049059374 (durationBeforeRetry 500ms). Error: error cleaning subPath mounts for volume "ssh-key" (UniqueName: "kubernetes.io/secret/8193174d-9636-4fc6-97bb-1fd0baba30e4-ssh-key") pod "8193174d-9636-4fc6-97bb-1fd0baba30e4" (UID: "8193174d-9636-4fc6-97bb-1fd0baba30e4") : error deleting /var/lib/kubelet/pods/8193174d-9636-4fc6-97bb-1fd0baba30e4/volume-subpaths: remove /var/lib/kubelet/pods/8193174d-9636-4fc6-97bb-1fd0baba30e4/volume-subpaths: no such file or directory Dec 03 13:00:43 crc kubenswrapper[5052]: I1203 13:00:43.963539 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8193174d-9636-4fc6-97bb-1fd0baba30e4-inventory" (OuterVolumeSpecName: "inventory") pod "8193174d-9636-4fc6-97bb-1fd0baba30e4" (UID: "8193174d-9636-4fc6-97bb-1fd0baba30e4"). InnerVolumeSpecName "inventory". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:00:44 crc kubenswrapper[5052]: I1203 13:00:44.041658 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-vp6mc\" (UniqueName: \"kubernetes.io/projected/8193174d-9636-4fc6-97bb-1fd0baba30e4-kube-api-access-vp6mc\") on node \"crc\" DevicePath \"\"" Dec 03 13:00:44 crc kubenswrapper[5052]: I1203 13:00:44.041734 5052 reconciler_common.go:293] "Volume detached for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/8193174d-9636-4fc6-97bb-1fd0baba30e4-inventory\") on node \"crc\" DevicePath \"\"" Dec 03 13:00:44 crc kubenswrapper[5052]: I1203 13:00:44.041750 5052 reconciler_common.go:293] "Volume detached for volume \"openstack-edpm-tls-generic-service1-default-certs-0\" (UniqueName: \"kubernetes.io/projected/8193174d-9636-4fc6-97bb-1fd0baba30e4-openstack-edpm-tls-generic-service1-default-certs-0\") on node \"crc\" DevicePath \"\"" Dec 03 13:00:44 crc kubenswrapper[5052]: I1203 13:00:44.041788 5052 reconciler_common.go:293] "Volume detached for volume \"install-certs-ovr-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8193174d-9636-4fc6-97bb-1fd0baba30e4-install-certs-ovr-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 13:00:44 crc kubenswrapper[5052]: I1203 13:00:44.041802 5052 reconciler_common.go:293] "Volume detached for volume \"generic-service1-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8193174d-9636-4fc6-97bb-1fd0baba30e4-generic-service1-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 13:00:44 crc kubenswrapper[5052]: I1203 13:00:44.547135 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/8193174d-9636-4fc6-97bb-1fd0baba30e4-ssh-key\") pod \"8193174d-9636-4fc6-97bb-1fd0baba30e4\" (UID: \"8193174d-9636-4fc6-97bb-1fd0baba30e4\") " Dec 03 13:00:44 crc kubenswrapper[5052]: I1203 13:00:44.552933 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8193174d-9636-4fc6-97bb-1fd0baba30e4-ssh-key" (OuterVolumeSpecName: "ssh-key") pod "8193174d-9636-4fc6-97bb-1fd0baba30e4" (UID: "8193174d-9636-4fc6-97bb-1fd0baba30e4"). InnerVolumeSpecName "ssh-key". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:00:44 crc kubenswrapper[5052]: I1203 13:00:44.646286 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/install-certs-ovr-openstack-edpm-tls-openstack-edpm-tls-2f5qn" event={"ID":"8193174d-9636-4fc6-97bb-1fd0baba30e4","Type":"ContainerDied","Data":"08d63c7fd4fb650411d69fbc8e517cf4e70348852f345cb01d6203b6efb00544"} Dec 03 13:00:44 crc kubenswrapper[5052]: I1203 13:00:44.646329 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="08d63c7fd4fb650411d69fbc8e517cf4e70348852f345cb01d6203b6efb00544" Dec 03 13:00:44 crc kubenswrapper[5052]: I1203 13:00:44.646352 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/install-certs-ovr-openstack-edpm-tls-openstack-edpm-tls-2f5qn" Dec 03 13:00:44 crc kubenswrapper[5052]: I1203 13:00:44.648703 5052 reconciler_common.go:293] "Volume detached for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/8193174d-9636-4fc6-97bb-1fd0baba30e4-ssh-key\") on node \"crc\" DevicePath \"\"" Dec 03 13:00:44 crc kubenswrapper[5052]: I1203 13:00:44.711643 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/generic-service1-openstack-edpm-tls-openstack-edpm-tls-z569q"] Dec 03 13:00:44 crc kubenswrapper[5052]: E1203 13:00:44.712296 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="8193174d-9636-4fc6-97bb-1fd0baba30e4" containerName="install-certs-ovr-openstack-edpm-tls-openstack-edpm-tls" Dec 03 13:00:44 crc kubenswrapper[5052]: I1203 13:00:44.712380 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="8193174d-9636-4fc6-97bb-1fd0baba30e4" containerName="install-certs-ovr-openstack-edpm-tls-openstack-edpm-tls" Dec 03 13:00:44 crc kubenswrapper[5052]: I1203 13:00:44.712648 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="8193174d-9636-4fc6-97bb-1fd0baba30e4" containerName="install-certs-ovr-openstack-edpm-tls-openstack-edpm-tls" Dec 03 13:00:44 crc kubenswrapper[5052]: I1203 13:00:44.713373 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/generic-service1-openstack-edpm-tls-openstack-edpm-tls-z569q" Dec 03 13:00:44 crc kubenswrapper[5052]: I1203 13:00:44.716911 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"dataplanenodeset-openstack-edpm-tls" Dec 03 13:00:44 crc kubenswrapper[5052]: I1203 13:00:44.717153 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"dataplane-ansible-ssh-private-key-secret" Dec 03 13:00:44 crc kubenswrapper[5052]: I1203 13:00:44.717410 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"combined-ca-bundle" Dec 03 13:00:44 crc kubenswrapper[5052]: I1203 13:00:44.719773 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"openstack-aee-default-env" Dec 03 13:00:44 crc kubenswrapper[5052]: I1203 13:00:44.720989 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/generic-service1-openstack-edpm-tls-openstack-edpm-tls-z569q"] Dec 03 13:00:44 crc kubenswrapper[5052]: I1203 13:00:44.727323 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"openstack-edpm-tls-dockercfg-5dddj" Dec 03 13:00:44 crc kubenswrapper[5052]: I1203 13:00:44.851277 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/b4906f39-828c-4e98-a37d-d627137e55c2-inventory\") pod \"generic-service1-openstack-edpm-tls-openstack-edpm-tls-z569q\" (UID: \"b4906f39-828c-4e98-a37d-d627137e55c2\") " pod="openstack-kuttl-tests/generic-service1-openstack-edpm-tls-openstack-edpm-tls-z569q" Dec 03 13:00:44 crc kubenswrapper[5052]: I1203 13:00:44.851370 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/b4906f39-828c-4e98-a37d-d627137e55c2-ssh-key\") pod \"generic-service1-openstack-edpm-tls-openstack-edpm-tls-z569q\" (UID: \"b4906f39-828c-4e98-a37d-d627137e55c2\") " pod="openstack-kuttl-tests/generic-service1-openstack-edpm-tls-openstack-edpm-tls-z569q" Dec 03 13:00:44 crc kubenswrapper[5052]: I1203 13:00:44.851455 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"generic-service1-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b4906f39-828c-4e98-a37d-d627137e55c2-generic-service1-combined-ca-bundle\") pod \"generic-service1-openstack-edpm-tls-openstack-edpm-tls-z569q\" (UID: \"b4906f39-828c-4e98-a37d-d627137e55c2\") " pod="openstack-kuttl-tests/generic-service1-openstack-edpm-tls-openstack-edpm-tls-z569q" Dec 03 13:00:44 crc kubenswrapper[5052]: I1203 13:00:44.851655 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xx4f4\" (UniqueName: \"kubernetes.io/projected/b4906f39-828c-4e98-a37d-d627137e55c2-kube-api-access-xx4f4\") pod \"generic-service1-openstack-edpm-tls-openstack-edpm-tls-z569q\" (UID: \"b4906f39-828c-4e98-a37d-d627137e55c2\") " pod="openstack-kuttl-tests/generic-service1-openstack-edpm-tls-openstack-edpm-tls-z569q" Dec 03 13:00:44 crc kubenswrapper[5052]: I1203 13:00:44.953274 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/b4906f39-828c-4e98-a37d-d627137e55c2-inventory\") pod \"generic-service1-openstack-edpm-tls-openstack-edpm-tls-z569q\" (UID: \"b4906f39-828c-4e98-a37d-d627137e55c2\") " pod="openstack-kuttl-tests/generic-service1-openstack-edpm-tls-openstack-edpm-tls-z569q" Dec 03 13:00:44 crc kubenswrapper[5052]: I1203 13:00:44.953332 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/b4906f39-828c-4e98-a37d-d627137e55c2-ssh-key\") pod \"generic-service1-openstack-edpm-tls-openstack-edpm-tls-z569q\" (UID: \"b4906f39-828c-4e98-a37d-d627137e55c2\") " pod="openstack-kuttl-tests/generic-service1-openstack-edpm-tls-openstack-edpm-tls-z569q" Dec 03 13:00:44 crc kubenswrapper[5052]: I1203 13:00:44.953363 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"generic-service1-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b4906f39-828c-4e98-a37d-d627137e55c2-generic-service1-combined-ca-bundle\") pod \"generic-service1-openstack-edpm-tls-openstack-edpm-tls-z569q\" (UID: \"b4906f39-828c-4e98-a37d-d627137e55c2\") " pod="openstack-kuttl-tests/generic-service1-openstack-edpm-tls-openstack-edpm-tls-z569q" Dec 03 13:00:44 crc kubenswrapper[5052]: I1203 13:00:44.953393 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-xx4f4\" (UniqueName: \"kubernetes.io/projected/b4906f39-828c-4e98-a37d-d627137e55c2-kube-api-access-xx4f4\") pod \"generic-service1-openstack-edpm-tls-openstack-edpm-tls-z569q\" (UID: \"b4906f39-828c-4e98-a37d-d627137e55c2\") " pod="openstack-kuttl-tests/generic-service1-openstack-edpm-tls-openstack-edpm-tls-z569q" Dec 03 13:00:44 crc kubenswrapper[5052]: I1203 13:00:44.957736 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/b4906f39-828c-4e98-a37d-d627137e55c2-ssh-key\") pod \"generic-service1-openstack-edpm-tls-openstack-edpm-tls-z569q\" (UID: \"b4906f39-828c-4e98-a37d-d627137e55c2\") " pod="openstack-kuttl-tests/generic-service1-openstack-edpm-tls-openstack-edpm-tls-z569q" Dec 03 13:00:44 crc kubenswrapper[5052]: I1203 13:00:44.957765 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/b4906f39-828c-4e98-a37d-d627137e55c2-inventory\") pod \"generic-service1-openstack-edpm-tls-openstack-edpm-tls-z569q\" (UID: \"b4906f39-828c-4e98-a37d-d627137e55c2\") " pod="openstack-kuttl-tests/generic-service1-openstack-edpm-tls-openstack-edpm-tls-z569q" Dec 03 13:00:44 crc kubenswrapper[5052]: I1203 13:00:44.958384 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"generic-service1-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b4906f39-828c-4e98-a37d-d627137e55c2-generic-service1-combined-ca-bundle\") pod \"generic-service1-openstack-edpm-tls-openstack-edpm-tls-z569q\" (UID: \"b4906f39-828c-4e98-a37d-d627137e55c2\") " pod="openstack-kuttl-tests/generic-service1-openstack-edpm-tls-openstack-edpm-tls-z569q" Dec 03 13:00:44 crc kubenswrapper[5052]: I1203 13:00:44.972689 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-xx4f4\" (UniqueName: \"kubernetes.io/projected/b4906f39-828c-4e98-a37d-d627137e55c2-kube-api-access-xx4f4\") pod \"generic-service1-openstack-edpm-tls-openstack-edpm-tls-z569q\" (UID: \"b4906f39-828c-4e98-a37d-d627137e55c2\") " pod="openstack-kuttl-tests/generic-service1-openstack-edpm-tls-openstack-edpm-tls-z569q" Dec 03 13:00:45 crc kubenswrapper[5052]: I1203 13:00:45.029826 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/generic-service1-openstack-edpm-tls-openstack-edpm-tls-z569q" Dec 03 13:00:45 crc kubenswrapper[5052]: I1203 13:00:45.437311 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/generic-service1-openstack-edpm-tls-openstack-edpm-tls-z569q"] Dec 03 13:00:45 crc kubenswrapper[5052]: I1203 13:00:45.657863 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/generic-service1-openstack-edpm-tls-openstack-edpm-tls-z569q" event={"ID":"b4906f39-828c-4e98-a37d-d627137e55c2","Type":"ContainerStarted","Data":"f5b16b5f591ffb20bd85908fa4669e1c695410f9f415aa37543b79176e7c80fc"} Dec 03 13:00:46 crc kubenswrapper[5052]: I1203 13:00:46.669671 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/generic-service1-openstack-edpm-tls-openstack-edpm-tls-z569q" event={"ID":"b4906f39-828c-4e98-a37d-d627137e55c2","Type":"ContainerStarted","Data":"adbfb99d0ce82b3e624c45e32aac97991f21de1f08612bf6ef1b89ddee3aafcd"} Dec 03 13:00:46 crc kubenswrapper[5052]: I1203 13:00:46.688729 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/generic-service1-openstack-edpm-tls-openstack-edpm-tls-z569q" podStartSLOduration=2.253750165 podStartE2EDuration="2.688711724s" podCreationTimestamp="2025-12-03 13:00:44 +0000 UTC" firstStartedPulling="2025-12-03 13:00:45.451973515 +0000 UTC m=+6036.040215279" lastFinishedPulling="2025-12-03 13:00:45.886935074 +0000 UTC m=+6036.475176838" observedRunningTime="2025-12-03 13:00:46.687582571 +0000 UTC m=+6037.275824335" watchObservedRunningTime="2025-12-03 13:00:46.688711724 +0000 UTC m=+6037.276953488" Dec 03 13:00:49 crc kubenswrapper[5052]: I1203 13:00:49.695458 5052 generic.go:334] "Generic (PLEG): container finished" podID="b4906f39-828c-4e98-a37d-d627137e55c2" containerID="adbfb99d0ce82b3e624c45e32aac97991f21de1f08612bf6ef1b89ddee3aafcd" exitCode=0 Dec 03 13:00:49 crc kubenswrapper[5052]: I1203 13:00:49.695558 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/generic-service1-openstack-edpm-tls-openstack-edpm-tls-z569q" event={"ID":"b4906f39-828c-4e98-a37d-d627137e55c2","Type":"ContainerDied","Data":"adbfb99d0ce82b3e624c45e32aac97991f21de1f08612bf6ef1b89ddee3aafcd"} Dec 03 13:00:51 crc kubenswrapper[5052]: I1203 13:00:51.001739 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/generic-service1-openstack-edpm-tls-openstack-edpm-tls-z569q" Dec 03 13:00:51 crc kubenswrapper[5052]: I1203 13:00:51.145852 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/b4906f39-828c-4e98-a37d-d627137e55c2-inventory\") pod \"b4906f39-828c-4e98-a37d-d627137e55c2\" (UID: \"b4906f39-828c-4e98-a37d-d627137e55c2\") " Dec 03 13:00:51 crc kubenswrapper[5052]: I1203 13:00:51.146326 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/b4906f39-828c-4e98-a37d-d627137e55c2-ssh-key\") pod \"b4906f39-828c-4e98-a37d-d627137e55c2\" (UID: \"b4906f39-828c-4e98-a37d-d627137e55c2\") " Dec 03 13:00:51 crc kubenswrapper[5052]: I1203 13:00:51.146383 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xx4f4\" (UniqueName: \"kubernetes.io/projected/b4906f39-828c-4e98-a37d-d627137e55c2-kube-api-access-xx4f4\") pod \"b4906f39-828c-4e98-a37d-d627137e55c2\" (UID: \"b4906f39-828c-4e98-a37d-d627137e55c2\") " Dec 03 13:00:51 crc kubenswrapper[5052]: I1203 13:00:51.146427 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"generic-service1-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b4906f39-828c-4e98-a37d-d627137e55c2-generic-service1-combined-ca-bundle\") pod \"b4906f39-828c-4e98-a37d-d627137e55c2\" (UID: \"b4906f39-828c-4e98-a37d-d627137e55c2\") " Dec 03 13:00:51 crc kubenswrapper[5052]: I1203 13:00:51.151412 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b4906f39-828c-4e98-a37d-d627137e55c2-generic-service1-combined-ca-bundle" (OuterVolumeSpecName: "generic-service1-combined-ca-bundle") pod "b4906f39-828c-4e98-a37d-d627137e55c2" (UID: "b4906f39-828c-4e98-a37d-d627137e55c2"). InnerVolumeSpecName "generic-service1-combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:00:51 crc kubenswrapper[5052]: I1203 13:00:51.151777 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b4906f39-828c-4e98-a37d-d627137e55c2-kube-api-access-xx4f4" (OuterVolumeSpecName: "kube-api-access-xx4f4") pod "b4906f39-828c-4e98-a37d-d627137e55c2" (UID: "b4906f39-828c-4e98-a37d-d627137e55c2"). InnerVolumeSpecName "kube-api-access-xx4f4". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:00:51 crc kubenswrapper[5052]: I1203 13:00:51.168503 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b4906f39-828c-4e98-a37d-d627137e55c2-inventory" (OuterVolumeSpecName: "inventory") pod "b4906f39-828c-4e98-a37d-d627137e55c2" (UID: "b4906f39-828c-4e98-a37d-d627137e55c2"). InnerVolumeSpecName "inventory". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:00:51 crc kubenswrapper[5052]: I1203 13:00:51.170579 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b4906f39-828c-4e98-a37d-d627137e55c2-ssh-key" (OuterVolumeSpecName: "ssh-key") pod "b4906f39-828c-4e98-a37d-d627137e55c2" (UID: "b4906f39-828c-4e98-a37d-d627137e55c2"). InnerVolumeSpecName "ssh-key". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:00:51 crc kubenswrapper[5052]: I1203 13:00:51.249658 5052 reconciler_common.go:293] "Volume detached for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/b4906f39-828c-4e98-a37d-d627137e55c2-ssh-key\") on node \"crc\" DevicePath \"\"" Dec 03 13:00:51 crc kubenswrapper[5052]: I1203 13:00:51.249715 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-xx4f4\" (UniqueName: \"kubernetes.io/projected/b4906f39-828c-4e98-a37d-d627137e55c2-kube-api-access-xx4f4\") on node \"crc\" DevicePath \"\"" Dec 03 13:00:51 crc kubenswrapper[5052]: I1203 13:00:51.249730 5052 reconciler_common.go:293] "Volume detached for volume \"generic-service1-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b4906f39-828c-4e98-a37d-d627137e55c2-generic-service1-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 13:00:51 crc kubenswrapper[5052]: I1203 13:00:51.249747 5052 reconciler_common.go:293] "Volume detached for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/b4906f39-828c-4e98-a37d-d627137e55c2-inventory\") on node \"crc\" DevicePath \"\"" Dec 03 13:00:51 crc kubenswrapper[5052]: I1203 13:00:51.720977 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/generic-service1-openstack-edpm-tls-openstack-edpm-tls-z569q" event={"ID":"b4906f39-828c-4e98-a37d-d627137e55c2","Type":"ContainerDied","Data":"f5b16b5f591ffb20bd85908fa4669e1c695410f9f415aa37543b79176e7c80fc"} Dec 03 13:00:51 crc kubenswrapper[5052]: I1203 13:00:51.721054 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f5b16b5f591ffb20bd85908fa4669e1c695410f9f415aa37543b79176e7c80fc" Dec 03 13:00:51 crc kubenswrapper[5052]: I1203 13:00:51.721072 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/generic-service1-openstack-edpm-tls-openstack-edpm-tls-z569q" Dec 03 13:00:52 crc kubenswrapper[5052]: I1203 13:00:52.540973 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/generic-service1-openstack-edpm-tls-openstack-edpm-tls-z569q"] Dec 03 13:00:52 crc kubenswrapper[5052]: I1203 13:00:52.547437 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/generic-service1-openstack-edpm-tls-openstack-edpm-tls-z569q"] Dec 03 13:00:52 crc kubenswrapper[5052]: I1203 13:00:52.555187 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/install-certs-ovr-openstack-edpm-tls-openstack-edpm-tls-2f5qn"] Dec 03 13:00:52 crc kubenswrapper[5052]: I1203 13:00:52.561195 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/install-certs-ovr-openstack-edpm-tls-openstack-edpm-tls-2f5qn"] Dec 03 13:00:52 crc kubenswrapper[5052]: I1203 13:00:52.627385 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-hrlsl"] Dec 03 13:00:52 crc kubenswrapper[5052]: E1203 13:00:52.627967 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="b4906f39-828c-4e98-a37d-d627137e55c2" containerName="generic-service1-openstack-edpm-tls-openstack-edpm-tls" Dec 03 13:00:52 crc kubenswrapper[5052]: I1203 13:00:52.627991 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="b4906f39-828c-4e98-a37d-d627137e55c2" containerName="generic-service1-openstack-edpm-tls-openstack-edpm-tls" Dec 03 13:00:52 crc kubenswrapper[5052]: I1203 13:00:52.628157 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="b4906f39-828c-4e98-a37d-d627137e55c2" containerName="generic-service1-openstack-edpm-tls-openstack-edpm-tls" Dec 03 13:00:52 crc kubenswrapper[5052]: I1203 13:00:52.629294 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-hrlsl" Dec 03 13:00:52 crc kubenswrapper[5052]: I1203 13:00:52.638700 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-hrlsl"] Dec 03 13:00:52 crc kubenswrapper[5052]: I1203 13:00:52.676982 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0e88a97a-bf2a-41f1-9377-688c7994159c-config\") pod \"dnsmasq-dnsmasq-84b9f45d47-hrlsl\" (UID: \"0e88a97a-bf2a-41f1-9377-688c7994159c\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-hrlsl" Dec 03 13:00:52 crc kubenswrapper[5052]: I1203 13:00:52.677057 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/0e88a97a-bf2a-41f1-9377-688c7994159c-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-84b9f45d47-hrlsl\" (UID: \"0e88a97a-bf2a-41f1-9377-688c7994159c\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-hrlsl" Dec 03 13:00:52 crc kubenswrapper[5052]: I1203 13:00:52.677167 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-znxhx\" (UniqueName: \"kubernetes.io/projected/0e88a97a-bf2a-41f1-9377-688c7994159c-kube-api-access-znxhx\") pod \"dnsmasq-dnsmasq-84b9f45d47-hrlsl\" (UID: \"0e88a97a-bf2a-41f1-9377-688c7994159c\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-hrlsl" Dec 03 13:00:52 crc kubenswrapper[5052]: I1203 13:00:52.779541 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0e88a97a-bf2a-41f1-9377-688c7994159c-config\") pod \"dnsmasq-dnsmasq-84b9f45d47-hrlsl\" (UID: \"0e88a97a-bf2a-41f1-9377-688c7994159c\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-hrlsl" Dec 03 13:00:52 crc kubenswrapper[5052]: I1203 13:00:52.780463 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/0e88a97a-bf2a-41f1-9377-688c7994159c-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-84b9f45d47-hrlsl\" (UID: \"0e88a97a-bf2a-41f1-9377-688c7994159c\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-hrlsl" Dec 03 13:00:52 crc kubenswrapper[5052]: I1203 13:00:52.780607 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-znxhx\" (UniqueName: \"kubernetes.io/projected/0e88a97a-bf2a-41f1-9377-688c7994159c-kube-api-access-znxhx\") pod \"dnsmasq-dnsmasq-84b9f45d47-hrlsl\" (UID: \"0e88a97a-bf2a-41f1-9377-688c7994159c\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-hrlsl" Dec 03 13:00:52 crc kubenswrapper[5052]: I1203 13:00:52.781008 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0e88a97a-bf2a-41f1-9377-688c7994159c-config\") pod \"dnsmasq-dnsmasq-84b9f45d47-hrlsl\" (UID: \"0e88a97a-bf2a-41f1-9377-688c7994159c\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-hrlsl" Dec 03 13:00:52 crc kubenswrapper[5052]: I1203 13:00:52.781395 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/0e88a97a-bf2a-41f1-9377-688c7994159c-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-84b9f45d47-hrlsl\" (UID: \"0e88a97a-bf2a-41f1-9377-688c7994159c\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-hrlsl" Dec 03 13:00:52 crc kubenswrapper[5052]: I1203 13:00:52.800815 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-znxhx\" (UniqueName: \"kubernetes.io/projected/0e88a97a-bf2a-41f1-9377-688c7994159c-kube-api-access-znxhx\") pod \"dnsmasq-dnsmasq-84b9f45d47-hrlsl\" (UID: \"0e88a97a-bf2a-41f1-9377-688c7994159c\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-hrlsl" Dec 03 13:00:52 crc kubenswrapper[5052]: I1203 13:00:52.970562 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-hrlsl" Dec 03 13:00:53 crc kubenswrapper[5052]: I1203 13:00:53.420764 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-hrlsl"] Dec 03 13:00:53 crc kubenswrapper[5052]: I1203 13:00:53.686462 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8193174d-9636-4fc6-97bb-1fd0baba30e4" path="/var/lib/kubelet/pods/8193174d-9636-4fc6-97bb-1fd0baba30e4/volumes" Dec 03 13:00:53 crc kubenswrapper[5052]: I1203 13:00:53.687398 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b4906f39-828c-4e98-a37d-d627137e55c2" path="/var/lib/kubelet/pods/b4906f39-828c-4e98-a37d-d627137e55c2/volumes" Dec 03 13:00:53 crc kubenswrapper[5052]: I1203 13:00:53.743905 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-hrlsl" event={"ID":"0e88a97a-bf2a-41f1-9377-688c7994159c","Type":"ContainerDied","Data":"9220f7f5e1c7e57fd2b6e468d93c19dc8c7d73d51f65490bfd6135547309af2b"} Dec 03 13:00:53 crc kubenswrapper[5052]: I1203 13:00:53.744886 5052 generic.go:334] "Generic (PLEG): container finished" podID="0e88a97a-bf2a-41f1-9377-688c7994159c" containerID="9220f7f5e1c7e57fd2b6e468d93c19dc8c7d73d51f65490bfd6135547309af2b" exitCode=0 Dec 03 13:00:53 crc kubenswrapper[5052]: I1203 13:00:53.745037 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-hrlsl" event={"ID":"0e88a97a-bf2a-41f1-9377-688c7994159c","Type":"ContainerStarted","Data":"5b7cc171298e6de0f6ae700406436883b4842c383337af73f0fb789ff5ef003e"} Dec 03 13:00:54 crc kubenswrapper[5052]: I1203 13:00:54.772614 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-hrlsl" event={"ID":"0e88a97a-bf2a-41f1-9377-688c7994159c","Type":"ContainerStarted","Data":"97a0ff779664ce4b40814258307e0441a7290281ca8a36256521539283d3c3cc"} Dec 03 13:00:54 crc kubenswrapper[5052]: I1203 13:00:54.773042 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-hrlsl" Dec 03 13:00:54 crc kubenswrapper[5052]: I1203 13:00:54.800233 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-hrlsl" podStartSLOduration=2.800200007 podStartE2EDuration="2.800200007s" podCreationTimestamp="2025-12-03 13:00:52 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 13:00:54.796058688 +0000 UTC m=+6045.384300462" watchObservedRunningTime="2025-12-03 13:00:54.800200007 +0000 UTC m=+6045.388441771" Dec 03 13:01:00 crc kubenswrapper[5052]: I1203 13:01:00.602648 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["crc-storage/crc-storage-crc-6gwst"] Dec 03 13:01:00 crc kubenswrapper[5052]: I1203 13:01:00.609971 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["crc-storage/crc-storage-crc-6gwst"] Dec 03 13:01:00 crc kubenswrapper[5052]: I1203 13:01:00.747411 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["crc-storage/crc-storage-crc-glj6h"] Dec 03 13:01:00 crc kubenswrapper[5052]: I1203 13:01:00.748382 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="crc-storage/crc-storage-crc-glj6h" Dec 03 13:01:00 crc kubenswrapper[5052]: I1203 13:01:00.753420 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"crc-storage"/"openshift-service-ca.crt" Dec 03 13:01:00 crc kubenswrapper[5052]: I1203 13:01:00.753849 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"crc-storage"/"crc-storage" Dec 03 13:01:00 crc kubenswrapper[5052]: I1203 13:01:00.754317 5052 reflector.go:368] Caches populated for *v1.Secret from object-"crc-storage"/"crc-storage-dockercfg-mf76g" Dec 03 13:01:00 crc kubenswrapper[5052]: I1203 13:01:00.754436 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"crc-storage"/"kube-root-ca.crt" Dec 03 13:01:00 crc kubenswrapper[5052]: I1203 13:01:00.755530 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["crc-storage/crc-storage-crc-glj6h"] Dec 03 13:01:00 crc kubenswrapper[5052]: I1203 13:01:00.807845 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/30cc2fb5-381a-4602-9bfd-da0fb13c6311-crc-storage\") pod \"crc-storage-crc-glj6h\" (UID: \"30cc2fb5-381a-4602-9bfd-da0fb13c6311\") " pod="crc-storage/crc-storage-crc-glj6h" Dec 03 13:01:00 crc kubenswrapper[5052]: I1203 13:01:00.807930 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bnv7p\" (UniqueName: \"kubernetes.io/projected/30cc2fb5-381a-4602-9bfd-da0fb13c6311-kube-api-access-bnv7p\") pod \"crc-storage-crc-glj6h\" (UID: \"30cc2fb5-381a-4602-9bfd-da0fb13c6311\") " pod="crc-storage/crc-storage-crc-glj6h" Dec 03 13:01:00 crc kubenswrapper[5052]: I1203 13:01:00.808235 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/30cc2fb5-381a-4602-9bfd-da0fb13c6311-node-mnt\") pod \"crc-storage-crc-glj6h\" (UID: \"30cc2fb5-381a-4602-9bfd-da0fb13c6311\") " pod="crc-storage/crc-storage-crc-glj6h" Dec 03 13:01:00 crc kubenswrapper[5052]: I1203 13:01:00.909220 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/30cc2fb5-381a-4602-9bfd-da0fb13c6311-crc-storage\") pod \"crc-storage-crc-glj6h\" (UID: \"30cc2fb5-381a-4602-9bfd-da0fb13c6311\") " pod="crc-storage/crc-storage-crc-glj6h" Dec 03 13:01:00 crc kubenswrapper[5052]: I1203 13:01:00.909293 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-bnv7p\" (UniqueName: \"kubernetes.io/projected/30cc2fb5-381a-4602-9bfd-da0fb13c6311-kube-api-access-bnv7p\") pod \"crc-storage-crc-glj6h\" (UID: \"30cc2fb5-381a-4602-9bfd-da0fb13c6311\") " pod="crc-storage/crc-storage-crc-glj6h" Dec 03 13:01:00 crc kubenswrapper[5052]: I1203 13:01:00.909360 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/30cc2fb5-381a-4602-9bfd-da0fb13c6311-node-mnt\") pod \"crc-storage-crc-glj6h\" (UID: \"30cc2fb5-381a-4602-9bfd-da0fb13c6311\") " pod="crc-storage/crc-storage-crc-glj6h" Dec 03 13:01:00 crc kubenswrapper[5052]: I1203 13:01:00.909692 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/30cc2fb5-381a-4602-9bfd-da0fb13c6311-node-mnt\") pod \"crc-storage-crc-glj6h\" (UID: \"30cc2fb5-381a-4602-9bfd-da0fb13c6311\") " pod="crc-storage/crc-storage-crc-glj6h" Dec 03 13:01:00 crc kubenswrapper[5052]: I1203 13:01:00.910115 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/30cc2fb5-381a-4602-9bfd-da0fb13c6311-crc-storage\") pod \"crc-storage-crc-glj6h\" (UID: \"30cc2fb5-381a-4602-9bfd-da0fb13c6311\") " pod="crc-storage/crc-storage-crc-glj6h" Dec 03 13:01:00 crc kubenswrapper[5052]: I1203 13:01:00.928867 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-bnv7p\" (UniqueName: \"kubernetes.io/projected/30cc2fb5-381a-4602-9bfd-da0fb13c6311-kube-api-access-bnv7p\") pod \"crc-storage-crc-glj6h\" (UID: \"30cc2fb5-381a-4602-9bfd-da0fb13c6311\") " pod="crc-storage/crc-storage-crc-glj6h" Dec 03 13:01:01 crc kubenswrapper[5052]: I1203 13:01:01.066420 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="crc-storage/crc-storage-crc-glj6h" Dec 03 13:01:01 crc kubenswrapper[5052]: I1203 13:01:01.488584 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["crc-storage/crc-storage-crc-glj6h"] Dec 03 13:01:01 crc kubenswrapper[5052]: I1203 13:01:01.690179 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="583f9858-b0cd-4976-a547-32ba02cbc85b" path="/var/lib/kubelet/pods/583f9858-b0cd-4976-a547-32ba02cbc85b/volumes" Dec 03 13:01:01 crc kubenswrapper[5052]: I1203 13:01:01.831818 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="crc-storage/crc-storage-crc-glj6h" event={"ID":"30cc2fb5-381a-4602-9bfd-da0fb13c6311","Type":"ContainerStarted","Data":"75fb4a1b18ffe36be471de8bda6da0d4a9c175cdc5eea5925e836304a0dd866d"} Dec 03 13:01:02 crc kubenswrapper[5052]: I1203 13:01:02.842167 5052 generic.go:334] "Generic (PLEG): container finished" podID="30cc2fb5-381a-4602-9bfd-da0fb13c6311" containerID="aeb2368acbb7a77c33586bb17253462cc38025c7383d25dc21d52cfee3b961e1" exitCode=0 Dec 03 13:01:02 crc kubenswrapper[5052]: I1203 13:01:02.842255 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="crc-storage/crc-storage-crc-glj6h" event={"ID":"30cc2fb5-381a-4602-9bfd-da0fb13c6311","Type":"ContainerDied","Data":"aeb2368acbb7a77c33586bb17253462cc38025c7383d25dc21d52cfee3b961e1"} Dec 03 13:01:02 crc kubenswrapper[5052]: I1203 13:01:02.972698 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-hrlsl" Dec 03 13:01:03 crc kubenswrapper[5052]: I1203 13:01:03.025361 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-76b7c4d945-lxqvh"] Dec 03 13:01:03 crc kubenswrapper[5052]: I1203 13:01:03.026002 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-76b7c4d945-lxqvh" podUID="d697b768-3215-466f-9438-109e015ad8ed" containerName="dnsmasq-dns" containerID="cri-o://ed0e4cbe31266dd420112b87e2a8219db6b9f7feb304395f959db983cfc1e6e3" gracePeriod=10 Dec 03 13:01:03 crc kubenswrapper[5052]: I1203 13:01:03.686611 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-76b7c4d945-lxqvh" Dec 03 13:01:03 crc kubenswrapper[5052]: I1203 13:01:03.850317 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/d697b768-3215-466f-9438-109e015ad8ed-dnsmasq-svc\") pod \"d697b768-3215-466f-9438-109e015ad8ed\" (UID: \"d697b768-3215-466f-9438-109e015ad8ed\") " Dec 03 13:01:03 crc kubenswrapper[5052]: I1203 13:01:03.850364 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"openstack-edpm-tls\" (UniqueName: \"kubernetes.io/configmap/d697b768-3215-466f-9438-109e015ad8ed-openstack-edpm-tls\") pod \"d697b768-3215-466f-9438-109e015ad8ed\" (UID: \"d697b768-3215-466f-9438-109e015ad8ed\") " Dec 03 13:01:03 crc kubenswrapper[5052]: I1203 13:01:03.850433 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b4tsv\" (UniqueName: \"kubernetes.io/projected/d697b768-3215-466f-9438-109e015ad8ed-kube-api-access-b4tsv\") pod \"d697b768-3215-466f-9438-109e015ad8ed\" (UID: \"d697b768-3215-466f-9438-109e015ad8ed\") " Dec 03 13:01:03 crc kubenswrapper[5052]: I1203 13:01:03.850460 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d697b768-3215-466f-9438-109e015ad8ed-config\") pod \"d697b768-3215-466f-9438-109e015ad8ed\" (UID: \"d697b768-3215-466f-9438-109e015ad8ed\") " Dec 03 13:01:03 crc kubenswrapper[5052]: I1203 13:01:03.853033 5052 generic.go:334] "Generic (PLEG): container finished" podID="d697b768-3215-466f-9438-109e015ad8ed" containerID="ed0e4cbe31266dd420112b87e2a8219db6b9f7feb304395f959db983cfc1e6e3" exitCode=0 Dec 03 13:01:03 crc kubenswrapper[5052]: I1203 13:01:03.853082 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-76b7c4d945-lxqvh" Dec 03 13:01:03 crc kubenswrapper[5052]: I1203 13:01:03.853099 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-76b7c4d945-lxqvh" event={"ID":"d697b768-3215-466f-9438-109e015ad8ed","Type":"ContainerDied","Data":"ed0e4cbe31266dd420112b87e2a8219db6b9f7feb304395f959db983cfc1e6e3"} Dec 03 13:01:03 crc kubenswrapper[5052]: I1203 13:01:03.853143 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-76b7c4d945-lxqvh" event={"ID":"d697b768-3215-466f-9438-109e015ad8ed","Type":"ContainerDied","Data":"9a57d57931016bff61e67ec0e8137f80718ea231cd302d708c19ae497a38df4b"} Dec 03 13:01:03 crc kubenswrapper[5052]: I1203 13:01:03.853169 5052 scope.go:117] "RemoveContainer" containerID="ed0e4cbe31266dd420112b87e2a8219db6b9f7feb304395f959db983cfc1e6e3" Dec 03 13:01:03 crc kubenswrapper[5052]: I1203 13:01:03.859475 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d697b768-3215-466f-9438-109e015ad8ed-kube-api-access-b4tsv" (OuterVolumeSpecName: "kube-api-access-b4tsv") pod "d697b768-3215-466f-9438-109e015ad8ed" (UID: "d697b768-3215-466f-9438-109e015ad8ed"). InnerVolumeSpecName "kube-api-access-b4tsv". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:01:03 crc kubenswrapper[5052]: I1203 13:01:03.895781 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d697b768-3215-466f-9438-109e015ad8ed-config" (OuterVolumeSpecName: "config") pod "d697b768-3215-466f-9438-109e015ad8ed" (UID: "d697b768-3215-466f-9438-109e015ad8ed"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:01:03 crc kubenswrapper[5052]: I1203 13:01:03.895778 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d697b768-3215-466f-9438-109e015ad8ed-dnsmasq-svc" (OuterVolumeSpecName: "dnsmasq-svc") pod "d697b768-3215-466f-9438-109e015ad8ed" (UID: "d697b768-3215-466f-9438-109e015ad8ed"). InnerVolumeSpecName "dnsmasq-svc". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:01:03 crc kubenswrapper[5052]: I1203 13:01:03.902011 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d697b768-3215-466f-9438-109e015ad8ed-openstack-edpm-tls" (OuterVolumeSpecName: "openstack-edpm-tls") pod "d697b768-3215-466f-9438-109e015ad8ed" (UID: "d697b768-3215-466f-9438-109e015ad8ed"). InnerVolumeSpecName "openstack-edpm-tls". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:01:03 crc kubenswrapper[5052]: I1203 13:01:03.952741 5052 reconciler_common.go:293] "Volume detached for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/d697b768-3215-466f-9438-109e015ad8ed-dnsmasq-svc\") on node \"crc\" DevicePath \"\"" Dec 03 13:01:03 crc kubenswrapper[5052]: I1203 13:01:03.952772 5052 reconciler_common.go:293] "Volume detached for volume \"openstack-edpm-tls\" (UniqueName: \"kubernetes.io/configmap/d697b768-3215-466f-9438-109e015ad8ed-openstack-edpm-tls\") on node \"crc\" DevicePath \"\"" Dec 03 13:01:03 crc kubenswrapper[5052]: I1203 13:01:03.952784 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-b4tsv\" (UniqueName: \"kubernetes.io/projected/d697b768-3215-466f-9438-109e015ad8ed-kube-api-access-b4tsv\") on node \"crc\" DevicePath \"\"" Dec 03 13:01:03 crc kubenswrapper[5052]: I1203 13:01:03.952795 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d697b768-3215-466f-9438-109e015ad8ed-config\") on node \"crc\" DevicePath \"\"" Dec 03 13:01:03 crc kubenswrapper[5052]: I1203 13:01:03.972793 5052 scope.go:117] "RemoveContainer" containerID="e10e081ec82236913927fb544daf6e2482f9dbce23297f7d171a2be24f3207b2" Dec 03 13:01:04 crc kubenswrapper[5052]: I1203 13:01:04.003155 5052 scope.go:117] "RemoveContainer" containerID="ed0e4cbe31266dd420112b87e2a8219db6b9f7feb304395f959db983cfc1e6e3" Dec 03 13:01:04 crc kubenswrapper[5052]: E1203 13:01:04.003840 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ed0e4cbe31266dd420112b87e2a8219db6b9f7feb304395f959db983cfc1e6e3\": container with ID starting with ed0e4cbe31266dd420112b87e2a8219db6b9f7feb304395f959db983cfc1e6e3 not found: ID does not exist" containerID="ed0e4cbe31266dd420112b87e2a8219db6b9f7feb304395f959db983cfc1e6e3" Dec 03 13:01:04 crc kubenswrapper[5052]: I1203 13:01:04.003901 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ed0e4cbe31266dd420112b87e2a8219db6b9f7feb304395f959db983cfc1e6e3"} err="failed to get container status \"ed0e4cbe31266dd420112b87e2a8219db6b9f7feb304395f959db983cfc1e6e3\": rpc error: code = NotFound desc = could not find container \"ed0e4cbe31266dd420112b87e2a8219db6b9f7feb304395f959db983cfc1e6e3\": container with ID starting with ed0e4cbe31266dd420112b87e2a8219db6b9f7feb304395f959db983cfc1e6e3 not found: ID does not exist" Dec 03 13:01:04 crc kubenswrapper[5052]: I1203 13:01:04.003935 5052 scope.go:117] "RemoveContainer" containerID="e10e081ec82236913927fb544daf6e2482f9dbce23297f7d171a2be24f3207b2" Dec 03 13:01:04 crc kubenswrapper[5052]: E1203 13:01:04.004828 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e10e081ec82236913927fb544daf6e2482f9dbce23297f7d171a2be24f3207b2\": container with ID starting with e10e081ec82236913927fb544daf6e2482f9dbce23297f7d171a2be24f3207b2 not found: ID does not exist" containerID="e10e081ec82236913927fb544daf6e2482f9dbce23297f7d171a2be24f3207b2" Dec 03 13:01:04 crc kubenswrapper[5052]: I1203 13:01:04.004875 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e10e081ec82236913927fb544daf6e2482f9dbce23297f7d171a2be24f3207b2"} err="failed to get container status \"e10e081ec82236913927fb544daf6e2482f9dbce23297f7d171a2be24f3207b2\": rpc error: code = NotFound desc = could not find container \"e10e081ec82236913927fb544daf6e2482f9dbce23297f7d171a2be24f3207b2\": container with ID starting with e10e081ec82236913927fb544daf6e2482f9dbce23297f7d171a2be24f3207b2 not found: ID does not exist" Dec 03 13:01:04 crc kubenswrapper[5052]: I1203 13:01:04.077145 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="crc-storage/crc-storage-crc-glj6h" Dec 03 13:01:04 crc kubenswrapper[5052]: I1203 13:01:04.184945 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-76b7c4d945-lxqvh"] Dec 03 13:01:04 crc kubenswrapper[5052]: I1203 13:01:04.190296 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-76b7c4d945-lxqvh"] Dec 03 13:01:04 crc kubenswrapper[5052]: I1203 13:01:04.255995 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/30cc2fb5-381a-4602-9bfd-da0fb13c6311-crc-storage\") pod \"30cc2fb5-381a-4602-9bfd-da0fb13c6311\" (UID: \"30cc2fb5-381a-4602-9bfd-da0fb13c6311\") " Dec 03 13:01:04 crc kubenswrapper[5052]: I1203 13:01:04.256130 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bnv7p\" (UniqueName: \"kubernetes.io/projected/30cc2fb5-381a-4602-9bfd-da0fb13c6311-kube-api-access-bnv7p\") pod \"30cc2fb5-381a-4602-9bfd-da0fb13c6311\" (UID: \"30cc2fb5-381a-4602-9bfd-da0fb13c6311\") " Dec 03 13:01:04 crc kubenswrapper[5052]: I1203 13:01:04.256304 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/30cc2fb5-381a-4602-9bfd-da0fb13c6311-node-mnt\") pod \"30cc2fb5-381a-4602-9bfd-da0fb13c6311\" (UID: \"30cc2fb5-381a-4602-9bfd-da0fb13c6311\") " Dec 03 13:01:04 crc kubenswrapper[5052]: I1203 13:01:04.256592 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/30cc2fb5-381a-4602-9bfd-da0fb13c6311-node-mnt" (OuterVolumeSpecName: "node-mnt") pod "30cc2fb5-381a-4602-9bfd-da0fb13c6311" (UID: "30cc2fb5-381a-4602-9bfd-da0fb13c6311"). InnerVolumeSpecName "node-mnt". PluginName "kubernetes.io/host-path", VolumeGidValue "" Dec 03 13:01:04 crc kubenswrapper[5052]: I1203 13:01:04.260028 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/30cc2fb5-381a-4602-9bfd-da0fb13c6311-kube-api-access-bnv7p" (OuterVolumeSpecName: "kube-api-access-bnv7p") pod "30cc2fb5-381a-4602-9bfd-da0fb13c6311" (UID: "30cc2fb5-381a-4602-9bfd-da0fb13c6311"). InnerVolumeSpecName "kube-api-access-bnv7p". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:01:04 crc kubenswrapper[5052]: I1203 13:01:04.286540 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/30cc2fb5-381a-4602-9bfd-da0fb13c6311-crc-storage" (OuterVolumeSpecName: "crc-storage") pod "30cc2fb5-381a-4602-9bfd-da0fb13c6311" (UID: "30cc2fb5-381a-4602-9bfd-da0fb13c6311"). InnerVolumeSpecName "crc-storage". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:01:04 crc kubenswrapper[5052]: I1203 13:01:04.358329 5052 reconciler_common.go:293] "Volume detached for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/30cc2fb5-381a-4602-9bfd-da0fb13c6311-node-mnt\") on node \"crc\" DevicePath \"\"" Dec 03 13:01:04 crc kubenswrapper[5052]: I1203 13:01:04.358547 5052 reconciler_common.go:293] "Volume detached for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/30cc2fb5-381a-4602-9bfd-da0fb13c6311-crc-storage\") on node \"crc\" DevicePath \"\"" Dec 03 13:01:04 crc kubenswrapper[5052]: I1203 13:01:04.358564 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-bnv7p\" (UniqueName: \"kubernetes.io/projected/30cc2fb5-381a-4602-9bfd-da0fb13c6311-kube-api-access-bnv7p\") on node \"crc\" DevicePath \"\"" Dec 03 13:01:04 crc kubenswrapper[5052]: I1203 13:01:04.865404 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="crc-storage/crc-storage-crc-glj6h" event={"ID":"30cc2fb5-381a-4602-9bfd-da0fb13c6311","Type":"ContainerDied","Data":"75fb4a1b18ffe36be471de8bda6da0d4a9c175cdc5eea5925e836304a0dd866d"} Dec 03 13:01:04 crc kubenswrapper[5052]: I1203 13:01:04.865464 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="crc-storage/crc-storage-crc-glj6h" Dec 03 13:01:04 crc kubenswrapper[5052]: I1203 13:01:04.865456 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="75fb4a1b18ffe36be471de8bda6da0d4a9c175cdc5eea5925e836304a0dd866d" Dec 03 13:01:05 crc kubenswrapper[5052]: I1203 13:01:05.685500 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d697b768-3215-466f-9438-109e015ad8ed" path="/var/lib/kubelet/pods/d697b768-3215-466f-9438-109e015ad8ed/volumes" Dec 03 13:01:07 crc kubenswrapper[5052]: I1203 13:01:07.752015 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["crc-storage/crc-storage-crc-glj6h"] Dec 03 13:01:07 crc kubenswrapper[5052]: I1203 13:01:07.761591 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["crc-storage/crc-storage-crc-glj6h"] Dec 03 13:01:07 crc kubenswrapper[5052]: I1203 13:01:07.919965 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["crc-storage/crc-storage-crc-f8jwz"] Dec 03 13:01:07 crc kubenswrapper[5052]: E1203 13:01:07.920367 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="d697b768-3215-466f-9438-109e015ad8ed" containerName="dnsmasq-dns" Dec 03 13:01:07 crc kubenswrapper[5052]: I1203 13:01:07.920393 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="d697b768-3215-466f-9438-109e015ad8ed" containerName="dnsmasq-dns" Dec 03 13:01:07 crc kubenswrapper[5052]: E1203 13:01:07.920420 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="30cc2fb5-381a-4602-9bfd-da0fb13c6311" containerName="storage" Dec 03 13:01:07 crc kubenswrapper[5052]: I1203 13:01:07.920429 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="30cc2fb5-381a-4602-9bfd-da0fb13c6311" containerName="storage" Dec 03 13:01:07 crc kubenswrapper[5052]: E1203 13:01:07.920458 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="d697b768-3215-466f-9438-109e015ad8ed" containerName="init" Dec 03 13:01:07 crc kubenswrapper[5052]: I1203 13:01:07.920467 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="d697b768-3215-466f-9438-109e015ad8ed" containerName="init" Dec 03 13:01:07 crc kubenswrapper[5052]: I1203 13:01:07.920712 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="d697b768-3215-466f-9438-109e015ad8ed" containerName="dnsmasq-dns" Dec 03 13:01:07 crc kubenswrapper[5052]: I1203 13:01:07.920742 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="30cc2fb5-381a-4602-9bfd-da0fb13c6311" containerName="storage" Dec 03 13:01:07 crc kubenswrapper[5052]: I1203 13:01:07.921363 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="crc-storage/crc-storage-crc-f8jwz" Dec 03 13:01:07 crc kubenswrapper[5052]: I1203 13:01:07.924276 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"crc-storage"/"openshift-service-ca.crt" Dec 03 13:01:07 crc kubenswrapper[5052]: I1203 13:01:07.924481 5052 reflector.go:368] Caches populated for *v1.Secret from object-"crc-storage"/"crc-storage-dockercfg-mf76g" Dec 03 13:01:07 crc kubenswrapper[5052]: I1203 13:01:07.924555 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"crc-storage"/"crc-storage" Dec 03 13:01:07 crc kubenswrapper[5052]: I1203 13:01:07.925145 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"crc-storage"/"kube-root-ca.crt" Dec 03 13:01:07 crc kubenswrapper[5052]: I1203 13:01:07.928183 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["crc-storage/crc-storage-crc-f8jwz"] Dec 03 13:01:08 crc kubenswrapper[5052]: I1203 13:01:08.113323 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/e7db9821-37e8-440b-961b-2191b4f548ce-crc-storage\") pod \"crc-storage-crc-f8jwz\" (UID: \"e7db9821-37e8-440b-961b-2191b4f548ce\") " pod="crc-storage/crc-storage-crc-f8jwz" Dec 03 13:01:08 crc kubenswrapper[5052]: I1203 13:01:08.113395 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pt9b7\" (UniqueName: \"kubernetes.io/projected/e7db9821-37e8-440b-961b-2191b4f548ce-kube-api-access-pt9b7\") pod \"crc-storage-crc-f8jwz\" (UID: \"e7db9821-37e8-440b-961b-2191b4f548ce\") " pod="crc-storage/crc-storage-crc-f8jwz" Dec 03 13:01:08 crc kubenswrapper[5052]: I1203 13:01:08.113560 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/e7db9821-37e8-440b-961b-2191b4f548ce-node-mnt\") pod \"crc-storage-crc-f8jwz\" (UID: \"e7db9821-37e8-440b-961b-2191b4f548ce\") " pod="crc-storage/crc-storage-crc-f8jwz" Dec 03 13:01:08 crc kubenswrapper[5052]: I1203 13:01:08.214398 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/e7db9821-37e8-440b-961b-2191b4f548ce-node-mnt\") pod \"crc-storage-crc-f8jwz\" (UID: \"e7db9821-37e8-440b-961b-2191b4f548ce\") " pod="crc-storage/crc-storage-crc-f8jwz" Dec 03 13:01:08 crc kubenswrapper[5052]: I1203 13:01:08.214473 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/e7db9821-37e8-440b-961b-2191b4f548ce-crc-storage\") pod \"crc-storage-crc-f8jwz\" (UID: \"e7db9821-37e8-440b-961b-2191b4f548ce\") " pod="crc-storage/crc-storage-crc-f8jwz" Dec 03 13:01:08 crc kubenswrapper[5052]: I1203 13:01:08.214539 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-pt9b7\" (UniqueName: \"kubernetes.io/projected/e7db9821-37e8-440b-961b-2191b4f548ce-kube-api-access-pt9b7\") pod \"crc-storage-crc-f8jwz\" (UID: \"e7db9821-37e8-440b-961b-2191b4f548ce\") " pod="crc-storage/crc-storage-crc-f8jwz" Dec 03 13:01:08 crc kubenswrapper[5052]: I1203 13:01:08.214859 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/e7db9821-37e8-440b-961b-2191b4f548ce-node-mnt\") pod \"crc-storage-crc-f8jwz\" (UID: \"e7db9821-37e8-440b-961b-2191b4f548ce\") " pod="crc-storage/crc-storage-crc-f8jwz" Dec 03 13:01:08 crc kubenswrapper[5052]: I1203 13:01:08.215685 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/e7db9821-37e8-440b-961b-2191b4f548ce-crc-storage\") pod \"crc-storage-crc-f8jwz\" (UID: \"e7db9821-37e8-440b-961b-2191b4f548ce\") " pod="crc-storage/crc-storage-crc-f8jwz" Dec 03 13:01:08 crc kubenswrapper[5052]: I1203 13:01:08.234486 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-pt9b7\" (UniqueName: \"kubernetes.io/projected/e7db9821-37e8-440b-961b-2191b4f548ce-kube-api-access-pt9b7\") pod \"crc-storage-crc-f8jwz\" (UID: \"e7db9821-37e8-440b-961b-2191b4f548ce\") " pod="crc-storage/crc-storage-crc-f8jwz" Dec 03 13:01:08 crc kubenswrapper[5052]: I1203 13:01:08.243403 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="crc-storage/crc-storage-crc-f8jwz" Dec 03 13:01:08 crc kubenswrapper[5052]: I1203 13:01:08.669852 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["crc-storage/crc-storage-crc-f8jwz"] Dec 03 13:01:08 crc kubenswrapper[5052]: I1203 13:01:08.903308 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="crc-storage/crc-storage-crc-f8jwz" event={"ID":"e7db9821-37e8-440b-961b-2191b4f548ce","Type":"ContainerStarted","Data":"180474d711a7d0bfe5fb5388e26f79309197d5cf073b58e40fd4a9622d69ab92"} Dec 03 13:01:09 crc kubenswrapper[5052]: I1203 13:01:09.684139 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="30cc2fb5-381a-4602-9bfd-da0fb13c6311" path="/var/lib/kubelet/pods/30cc2fb5-381a-4602-9bfd-da0fb13c6311/volumes" Dec 03 13:01:09 crc kubenswrapper[5052]: I1203 13:01:09.911853 5052 generic.go:334] "Generic (PLEG): container finished" podID="e7db9821-37e8-440b-961b-2191b4f548ce" containerID="67b390a0268304ca7c8077e64e8d2a96fd60a9d43f440830c178191f3ee252a3" exitCode=0 Dec 03 13:01:09 crc kubenswrapper[5052]: I1203 13:01:09.911913 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="crc-storage/crc-storage-crc-f8jwz" event={"ID":"e7db9821-37e8-440b-961b-2191b4f548ce","Type":"ContainerDied","Data":"67b390a0268304ca7c8077e64e8d2a96fd60a9d43f440830c178191f3ee252a3"} Dec 03 13:01:11 crc kubenswrapper[5052]: I1203 13:01:11.198242 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="crc-storage/crc-storage-crc-f8jwz" Dec 03 13:01:11 crc kubenswrapper[5052]: I1203 13:01:11.370918 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/e7db9821-37e8-440b-961b-2191b4f548ce-node-mnt\") pod \"e7db9821-37e8-440b-961b-2191b4f548ce\" (UID: \"e7db9821-37e8-440b-961b-2191b4f548ce\") " Dec 03 13:01:11 crc kubenswrapper[5052]: I1203 13:01:11.371047 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/e7db9821-37e8-440b-961b-2191b4f548ce-crc-storage\") pod \"e7db9821-37e8-440b-961b-2191b4f548ce\" (UID: \"e7db9821-37e8-440b-961b-2191b4f548ce\") " Dec 03 13:01:11 crc kubenswrapper[5052]: I1203 13:01:11.371131 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pt9b7\" (UniqueName: \"kubernetes.io/projected/e7db9821-37e8-440b-961b-2191b4f548ce-kube-api-access-pt9b7\") pod \"e7db9821-37e8-440b-961b-2191b4f548ce\" (UID: \"e7db9821-37e8-440b-961b-2191b4f548ce\") " Dec 03 13:01:11 crc kubenswrapper[5052]: I1203 13:01:11.371410 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/e7db9821-37e8-440b-961b-2191b4f548ce-node-mnt" (OuterVolumeSpecName: "node-mnt") pod "e7db9821-37e8-440b-961b-2191b4f548ce" (UID: "e7db9821-37e8-440b-961b-2191b4f548ce"). InnerVolumeSpecName "node-mnt". PluginName "kubernetes.io/host-path", VolumeGidValue "" Dec 03 13:01:11 crc kubenswrapper[5052]: I1203 13:01:11.377900 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e7db9821-37e8-440b-961b-2191b4f548ce-kube-api-access-pt9b7" (OuterVolumeSpecName: "kube-api-access-pt9b7") pod "e7db9821-37e8-440b-961b-2191b4f548ce" (UID: "e7db9821-37e8-440b-961b-2191b4f548ce"). InnerVolumeSpecName "kube-api-access-pt9b7". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:01:11 crc kubenswrapper[5052]: I1203 13:01:11.393364 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e7db9821-37e8-440b-961b-2191b4f548ce-crc-storage" (OuterVolumeSpecName: "crc-storage") pod "e7db9821-37e8-440b-961b-2191b4f548ce" (UID: "e7db9821-37e8-440b-961b-2191b4f548ce"). InnerVolumeSpecName "crc-storage". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:01:11 crc kubenswrapper[5052]: I1203 13:01:11.472970 5052 reconciler_common.go:293] "Volume detached for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/e7db9821-37e8-440b-961b-2191b4f548ce-node-mnt\") on node \"crc\" DevicePath \"\"" Dec 03 13:01:11 crc kubenswrapper[5052]: I1203 13:01:11.473006 5052 reconciler_common.go:293] "Volume detached for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/e7db9821-37e8-440b-961b-2191b4f548ce-crc-storage\") on node \"crc\" DevicePath \"\"" Dec 03 13:01:11 crc kubenswrapper[5052]: I1203 13:01:11.473020 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-pt9b7\" (UniqueName: \"kubernetes.io/projected/e7db9821-37e8-440b-961b-2191b4f548ce-kube-api-access-pt9b7\") on node \"crc\" DevicePath \"\"" Dec 03 13:01:11 crc kubenswrapper[5052]: I1203 13:01:11.930026 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="crc-storage/crc-storage-crc-f8jwz" event={"ID":"e7db9821-37e8-440b-961b-2191b4f548ce","Type":"ContainerDied","Data":"180474d711a7d0bfe5fb5388e26f79309197d5cf073b58e40fd4a9622d69ab92"} Dec 03 13:01:11 crc kubenswrapper[5052]: I1203 13:01:11.930078 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="180474d711a7d0bfe5fb5388e26f79309197d5cf073b58e40fd4a9622d69ab92" Dec 03 13:01:11 crc kubenswrapper[5052]: I1203 13:01:11.930137 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="crc-storage/crc-storage-crc-f8jwz" Dec 03 13:01:14 crc kubenswrapper[5052]: I1203 13:01:14.878272 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-64864b6d57-m6vm4"] Dec 03 13:01:14 crc kubenswrapper[5052]: E1203 13:01:14.878618 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="e7db9821-37e8-440b-961b-2191b4f548ce" containerName="storage" Dec 03 13:01:14 crc kubenswrapper[5052]: I1203 13:01:14.878631 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="e7db9821-37e8-440b-961b-2191b4f548ce" containerName="storage" Dec 03 13:01:14 crc kubenswrapper[5052]: I1203 13:01:14.878772 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="e7db9821-37e8-440b-961b-2191b4f548ce" containerName="storage" Dec 03 13:01:14 crc kubenswrapper[5052]: I1203 13:01:14.879531 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-64864b6d57-m6vm4" Dec 03 13:01:14 crc kubenswrapper[5052]: W1203 13:01:14.893528 5052 reflector.go:561] object-"openstack-kuttl-tests"/"edpm-compute-no-nodes": failed to list *v1.ConfigMap: configmaps "edpm-compute-no-nodes" is forbidden: User "system:node:crc" cannot list resource "configmaps" in API group "" in the namespace "openstack-kuttl-tests": no relationship found between node 'crc' and this object Dec 03 13:01:14 crc kubenswrapper[5052]: E1203 13:01:14.893611 5052 reflector.go:158] "Unhandled Error" err="object-\"openstack-kuttl-tests\"/\"edpm-compute-no-nodes\": Failed to watch *v1.ConfigMap: failed to list *v1.ConfigMap: configmaps \"edpm-compute-no-nodes\" is forbidden: User \"system:node:crc\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openstack-kuttl-tests\": no relationship found between node 'crc' and this object" logger="UnhandledError" Dec 03 13:01:14 crc kubenswrapper[5052]: I1203 13:01:14.893885 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-64864b6d57-m6vm4"] Dec 03 13:01:14 crc kubenswrapper[5052]: I1203 13:01:14.939972 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/81730bd7-0103-4be8-86fb-873984597f88-config\") pod \"dnsmasq-dnsmasq-64864b6d57-m6vm4\" (UID: \"81730bd7-0103-4be8-86fb-873984597f88\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-64864b6d57-m6vm4" Dec 03 13:01:14 crc kubenswrapper[5052]: I1203 13:01:14.940080 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/81730bd7-0103-4be8-86fb-873984597f88-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-64864b6d57-m6vm4\" (UID: \"81730bd7-0103-4be8-86fb-873984597f88\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-64864b6d57-m6vm4" Dec 03 13:01:14 crc kubenswrapper[5052]: I1203 13:01:14.940257 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"edpm-compute-no-nodes\" (UniqueName: \"kubernetes.io/configmap/81730bd7-0103-4be8-86fb-873984597f88-edpm-compute-no-nodes\") pod \"dnsmasq-dnsmasq-64864b6d57-m6vm4\" (UID: \"81730bd7-0103-4be8-86fb-873984597f88\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-64864b6d57-m6vm4" Dec 03 13:01:14 crc kubenswrapper[5052]: I1203 13:01:14.940329 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5vj44\" (UniqueName: \"kubernetes.io/projected/81730bd7-0103-4be8-86fb-873984597f88-kube-api-access-5vj44\") pod \"dnsmasq-dnsmasq-64864b6d57-m6vm4\" (UID: \"81730bd7-0103-4be8-86fb-873984597f88\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-64864b6d57-m6vm4" Dec 03 13:01:15 crc kubenswrapper[5052]: I1203 13:01:15.041302 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/81730bd7-0103-4be8-86fb-873984597f88-config\") pod \"dnsmasq-dnsmasq-64864b6d57-m6vm4\" (UID: \"81730bd7-0103-4be8-86fb-873984597f88\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-64864b6d57-m6vm4" Dec 03 13:01:15 crc kubenswrapper[5052]: I1203 13:01:15.041791 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/81730bd7-0103-4be8-86fb-873984597f88-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-64864b6d57-m6vm4\" (UID: \"81730bd7-0103-4be8-86fb-873984597f88\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-64864b6d57-m6vm4" Dec 03 13:01:15 crc kubenswrapper[5052]: I1203 13:01:15.041848 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"edpm-compute-no-nodes\" (UniqueName: \"kubernetes.io/configmap/81730bd7-0103-4be8-86fb-873984597f88-edpm-compute-no-nodes\") pod \"dnsmasq-dnsmasq-64864b6d57-m6vm4\" (UID: \"81730bd7-0103-4be8-86fb-873984597f88\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-64864b6d57-m6vm4" Dec 03 13:01:15 crc kubenswrapper[5052]: I1203 13:01:15.041900 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-5vj44\" (UniqueName: \"kubernetes.io/projected/81730bd7-0103-4be8-86fb-873984597f88-kube-api-access-5vj44\") pod \"dnsmasq-dnsmasq-64864b6d57-m6vm4\" (UID: \"81730bd7-0103-4be8-86fb-873984597f88\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-64864b6d57-m6vm4" Dec 03 13:01:15 crc kubenswrapper[5052]: I1203 13:01:15.042975 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/81730bd7-0103-4be8-86fb-873984597f88-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-64864b6d57-m6vm4\" (UID: \"81730bd7-0103-4be8-86fb-873984597f88\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-64864b6d57-m6vm4" Dec 03 13:01:15 crc kubenswrapper[5052]: I1203 13:01:15.043610 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/81730bd7-0103-4be8-86fb-873984597f88-config\") pod \"dnsmasq-dnsmasq-64864b6d57-m6vm4\" (UID: \"81730bd7-0103-4be8-86fb-873984597f88\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-64864b6d57-m6vm4" Dec 03 13:01:15 crc kubenswrapper[5052]: I1203 13:01:15.065970 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-5vj44\" (UniqueName: \"kubernetes.io/projected/81730bd7-0103-4be8-86fb-873984597f88-kube-api-access-5vj44\") pod \"dnsmasq-dnsmasq-64864b6d57-m6vm4\" (UID: \"81730bd7-0103-4be8-86fb-873984597f88\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-64864b6d57-m6vm4" Dec 03 13:01:16 crc kubenswrapper[5052]: E1203 13:01:16.042428 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/edpm-compute-no-nodes: failed to sync configmap cache: timed out waiting for the condition Dec 03 13:01:16 crc kubenswrapper[5052]: E1203 13:01:16.042536 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/81730bd7-0103-4be8-86fb-873984597f88-edpm-compute-no-nodes podName:81730bd7-0103-4be8-86fb-873984597f88 nodeName:}" failed. No retries permitted until 2025-12-03 13:01:16.542503299 +0000 UTC m=+6067.130745063 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "edpm-compute-no-nodes" (UniqueName: "kubernetes.io/configmap/81730bd7-0103-4be8-86fb-873984597f88-edpm-compute-no-nodes") pod "dnsmasq-dnsmasq-64864b6d57-m6vm4" (UID: "81730bd7-0103-4be8-86fb-873984597f88") : failed to sync configmap cache: timed out waiting for the condition Dec 03 13:01:16 crc kubenswrapper[5052]: I1203 13:01:16.463664 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"edpm-compute-no-nodes" Dec 03 13:01:16 crc kubenswrapper[5052]: I1203 13:01:16.562151 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"edpm-compute-no-nodes\" (UniqueName: \"kubernetes.io/configmap/81730bd7-0103-4be8-86fb-873984597f88-edpm-compute-no-nodes\") pod \"dnsmasq-dnsmasq-64864b6d57-m6vm4\" (UID: \"81730bd7-0103-4be8-86fb-873984597f88\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-64864b6d57-m6vm4" Dec 03 13:01:16 crc kubenswrapper[5052]: I1203 13:01:16.562929 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"edpm-compute-no-nodes\" (UniqueName: \"kubernetes.io/configmap/81730bd7-0103-4be8-86fb-873984597f88-edpm-compute-no-nodes\") pod \"dnsmasq-dnsmasq-64864b6d57-m6vm4\" (UID: \"81730bd7-0103-4be8-86fb-873984597f88\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-64864b6d57-m6vm4" Dec 03 13:01:16 crc kubenswrapper[5052]: I1203 13:01:16.696370 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-64864b6d57-m6vm4" Dec 03 13:01:17 crc kubenswrapper[5052]: I1203 13:01:17.136860 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-64864b6d57-m6vm4"] Dec 03 13:01:17 crc kubenswrapper[5052]: W1203 13:01:17.143045 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod81730bd7_0103_4be8_86fb_873984597f88.slice/crio-09d891f4c1fec9cf21fd6b6e72588534fd1d4d42425eed0a764f74d9cf54f1f5 WatchSource:0}: Error finding container 09d891f4c1fec9cf21fd6b6e72588534fd1d4d42425eed0a764f74d9cf54f1f5: Status 404 returned error can't find the container with id 09d891f4c1fec9cf21fd6b6e72588534fd1d4d42425eed0a764f74d9cf54f1f5 Dec 03 13:01:17 crc kubenswrapper[5052]: I1203 13:01:17.986393 5052 generic.go:334] "Generic (PLEG): container finished" podID="81730bd7-0103-4be8-86fb-873984597f88" containerID="26e26ab53dd348a85275ab39acd2cb9a38d660665660965e13d29fab6e967fc5" exitCode=0 Dec 03 13:01:17 crc kubenswrapper[5052]: I1203 13:01:17.986477 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-64864b6d57-m6vm4" event={"ID":"81730bd7-0103-4be8-86fb-873984597f88","Type":"ContainerDied","Data":"26e26ab53dd348a85275ab39acd2cb9a38d660665660965e13d29fab6e967fc5"} Dec 03 13:01:17 crc kubenswrapper[5052]: I1203 13:01:17.986760 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-64864b6d57-m6vm4" event={"ID":"81730bd7-0103-4be8-86fb-873984597f88","Type":"ContainerStarted","Data":"09d891f4c1fec9cf21fd6b6e72588534fd1d4d42425eed0a764f74d9cf54f1f5"} Dec 03 13:01:19 crc kubenswrapper[5052]: I1203 13:01:19.003180 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-64864b6d57-m6vm4" event={"ID":"81730bd7-0103-4be8-86fb-873984597f88","Type":"ContainerStarted","Data":"fa3231018d1f01da1d43583d08a32a3406f40fecbca74330251b188de857db1f"} Dec 03 13:01:19 crc kubenswrapper[5052]: I1203 13:01:19.003914 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-64864b6d57-m6vm4" Dec 03 13:01:19 crc kubenswrapper[5052]: I1203 13:01:19.029997 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-64864b6d57-m6vm4" podStartSLOduration=5.029973825 podStartE2EDuration="5.029973825s" podCreationTimestamp="2025-12-03 13:01:14 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 13:01:19.026915598 +0000 UTC m=+6069.615157362" watchObservedRunningTime="2025-12-03 13:01:19.029973825 +0000 UTC m=+6069.618215589" Dec 03 13:01:26 crc kubenswrapper[5052]: I1203 13:01:26.697701 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-64864b6d57-m6vm4" Dec 03 13:01:26 crc kubenswrapper[5052]: I1203 13:01:26.767282 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-hrlsl"] Dec 03 13:01:26 crc kubenswrapper[5052]: I1203 13:01:26.767545 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-hrlsl" podUID="0e88a97a-bf2a-41f1-9377-688c7994159c" containerName="dnsmasq-dns" containerID="cri-o://97a0ff779664ce4b40814258307e0441a7290281ca8a36256521539283d3c3cc" gracePeriod=10 Dec 03 13:01:27 crc kubenswrapper[5052]: I1203 13:01:27.035271 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/download-cache-edpm-compute-no-nodes-edpm-compute-no-nodes2tk9d"] Dec 03 13:01:27 crc kubenswrapper[5052]: I1203 13:01:27.036268 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/download-cache-edpm-compute-no-nodes-edpm-compute-no-nodes2tk9d" Dec 03 13:01:27 crc kubenswrapper[5052]: I1203 13:01:27.042300 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/download-cache-edpm-compute-no-nodes-edpm-compute-no-nodes2tk9d"] Dec 03 13:01:27 crc kubenswrapper[5052]: I1203 13:01:27.042353 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"openstack-aee-default-env" Dec 03 13:01:27 crc kubenswrapper[5052]: I1203 13:01:27.042569 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"dataplane-ansible-ssh-private-key-secret" Dec 03 13:01:27 crc kubenswrapper[5052]: I1203 13:01:27.042678 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"edpm-compute-no-nodes-dockercfg-mkcgv" Dec 03 13:01:27 crc kubenswrapper[5052]: I1203 13:01:27.042730 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"dataplanenodeset-edpm-compute-no-nodes" Dec 03 13:01:27 crc kubenswrapper[5052]: I1203 13:01:27.086981 5052 generic.go:334] "Generic (PLEG): container finished" podID="0e88a97a-bf2a-41f1-9377-688c7994159c" containerID="97a0ff779664ce4b40814258307e0441a7290281ca8a36256521539283d3c3cc" exitCode=0 Dec 03 13:01:27 crc kubenswrapper[5052]: I1203 13:01:27.087382 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-hrlsl" event={"ID":"0e88a97a-bf2a-41f1-9377-688c7994159c","Type":"ContainerDied","Data":"97a0ff779664ce4b40814258307e0441a7290281ca8a36256521539283d3c3cc"} Dec 03 13:01:27 crc kubenswrapper[5052]: I1203 13:01:27.144001 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-67s6l\" (UniqueName: \"kubernetes.io/projected/405110f4-d79e-4505-803b-5204317a4bfa-kube-api-access-67s6l\") pod \"download-cache-edpm-compute-no-nodes-edpm-compute-no-nodes2tk9d\" (UID: \"405110f4-d79e-4505-803b-5204317a4bfa\") " pod="openstack-kuttl-tests/download-cache-edpm-compute-no-nodes-edpm-compute-no-nodes2tk9d" Dec 03 13:01:27 crc kubenswrapper[5052]: I1203 13:01:27.144266 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/405110f4-d79e-4505-803b-5204317a4bfa-inventory\") pod \"download-cache-edpm-compute-no-nodes-edpm-compute-no-nodes2tk9d\" (UID: \"405110f4-d79e-4505-803b-5204317a4bfa\") " pod="openstack-kuttl-tests/download-cache-edpm-compute-no-nodes-edpm-compute-no-nodes2tk9d" Dec 03 13:01:27 crc kubenswrapper[5052]: I1203 13:01:27.144390 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/405110f4-d79e-4505-803b-5204317a4bfa-ssh-key\") pod \"download-cache-edpm-compute-no-nodes-edpm-compute-no-nodes2tk9d\" (UID: \"405110f4-d79e-4505-803b-5204317a4bfa\") " pod="openstack-kuttl-tests/download-cache-edpm-compute-no-nodes-edpm-compute-no-nodes2tk9d" Dec 03 13:01:27 crc kubenswrapper[5052]: I1203 13:01:27.237917 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-hrlsl" Dec 03 13:01:27 crc kubenswrapper[5052]: I1203 13:01:27.246094 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-67s6l\" (UniqueName: \"kubernetes.io/projected/405110f4-d79e-4505-803b-5204317a4bfa-kube-api-access-67s6l\") pod \"download-cache-edpm-compute-no-nodes-edpm-compute-no-nodes2tk9d\" (UID: \"405110f4-d79e-4505-803b-5204317a4bfa\") " pod="openstack-kuttl-tests/download-cache-edpm-compute-no-nodes-edpm-compute-no-nodes2tk9d" Dec 03 13:01:27 crc kubenswrapper[5052]: I1203 13:01:27.246176 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/405110f4-d79e-4505-803b-5204317a4bfa-inventory\") pod \"download-cache-edpm-compute-no-nodes-edpm-compute-no-nodes2tk9d\" (UID: \"405110f4-d79e-4505-803b-5204317a4bfa\") " pod="openstack-kuttl-tests/download-cache-edpm-compute-no-nodes-edpm-compute-no-nodes2tk9d" Dec 03 13:01:27 crc kubenswrapper[5052]: I1203 13:01:27.246249 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/405110f4-d79e-4505-803b-5204317a4bfa-ssh-key\") pod \"download-cache-edpm-compute-no-nodes-edpm-compute-no-nodes2tk9d\" (UID: \"405110f4-d79e-4505-803b-5204317a4bfa\") " pod="openstack-kuttl-tests/download-cache-edpm-compute-no-nodes-edpm-compute-no-nodes2tk9d" Dec 03 13:01:27 crc kubenswrapper[5052]: I1203 13:01:27.252431 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/405110f4-d79e-4505-803b-5204317a4bfa-ssh-key\") pod \"download-cache-edpm-compute-no-nodes-edpm-compute-no-nodes2tk9d\" (UID: \"405110f4-d79e-4505-803b-5204317a4bfa\") " pod="openstack-kuttl-tests/download-cache-edpm-compute-no-nodes-edpm-compute-no-nodes2tk9d" Dec 03 13:01:27 crc kubenswrapper[5052]: I1203 13:01:27.255471 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/405110f4-d79e-4505-803b-5204317a4bfa-inventory\") pod \"download-cache-edpm-compute-no-nodes-edpm-compute-no-nodes2tk9d\" (UID: \"405110f4-d79e-4505-803b-5204317a4bfa\") " pod="openstack-kuttl-tests/download-cache-edpm-compute-no-nodes-edpm-compute-no-nodes2tk9d" Dec 03 13:01:27 crc kubenswrapper[5052]: I1203 13:01:27.265215 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-67s6l\" (UniqueName: \"kubernetes.io/projected/405110f4-d79e-4505-803b-5204317a4bfa-kube-api-access-67s6l\") pod \"download-cache-edpm-compute-no-nodes-edpm-compute-no-nodes2tk9d\" (UID: \"405110f4-d79e-4505-803b-5204317a4bfa\") " pod="openstack-kuttl-tests/download-cache-edpm-compute-no-nodes-edpm-compute-no-nodes2tk9d" Dec 03 13:01:27 crc kubenswrapper[5052]: I1203 13:01:27.346882 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0e88a97a-bf2a-41f1-9377-688c7994159c-config\") pod \"0e88a97a-bf2a-41f1-9377-688c7994159c\" (UID: \"0e88a97a-bf2a-41f1-9377-688c7994159c\") " Dec 03 13:01:27 crc kubenswrapper[5052]: I1203 13:01:27.347037 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-znxhx\" (UniqueName: \"kubernetes.io/projected/0e88a97a-bf2a-41f1-9377-688c7994159c-kube-api-access-znxhx\") pod \"0e88a97a-bf2a-41f1-9377-688c7994159c\" (UID: \"0e88a97a-bf2a-41f1-9377-688c7994159c\") " Dec 03 13:01:27 crc kubenswrapper[5052]: I1203 13:01:27.347161 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/0e88a97a-bf2a-41f1-9377-688c7994159c-dnsmasq-svc\") pod \"0e88a97a-bf2a-41f1-9377-688c7994159c\" (UID: \"0e88a97a-bf2a-41f1-9377-688c7994159c\") " Dec 03 13:01:27 crc kubenswrapper[5052]: I1203 13:01:27.350662 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0e88a97a-bf2a-41f1-9377-688c7994159c-kube-api-access-znxhx" (OuterVolumeSpecName: "kube-api-access-znxhx") pod "0e88a97a-bf2a-41f1-9377-688c7994159c" (UID: "0e88a97a-bf2a-41f1-9377-688c7994159c"). InnerVolumeSpecName "kube-api-access-znxhx". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:01:27 crc kubenswrapper[5052]: I1203 13:01:27.363974 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/download-cache-edpm-compute-no-nodes-edpm-compute-no-nodes2tk9d" Dec 03 13:01:27 crc kubenswrapper[5052]: I1203 13:01:27.386119 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0e88a97a-bf2a-41f1-9377-688c7994159c-dnsmasq-svc" (OuterVolumeSpecName: "dnsmasq-svc") pod "0e88a97a-bf2a-41f1-9377-688c7994159c" (UID: "0e88a97a-bf2a-41f1-9377-688c7994159c"). InnerVolumeSpecName "dnsmasq-svc". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:01:27 crc kubenswrapper[5052]: I1203 13:01:27.392337 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0e88a97a-bf2a-41f1-9377-688c7994159c-config" (OuterVolumeSpecName: "config") pod "0e88a97a-bf2a-41f1-9377-688c7994159c" (UID: "0e88a97a-bf2a-41f1-9377-688c7994159c"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:01:27 crc kubenswrapper[5052]: I1203 13:01:27.448762 5052 reconciler_common.go:293] "Volume detached for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/0e88a97a-bf2a-41f1-9377-688c7994159c-dnsmasq-svc\") on node \"crc\" DevicePath \"\"" Dec 03 13:01:27 crc kubenswrapper[5052]: I1203 13:01:27.448802 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0e88a97a-bf2a-41f1-9377-688c7994159c-config\") on node \"crc\" DevicePath \"\"" Dec 03 13:01:27 crc kubenswrapper[5052]: I1203 13:01:27.448819 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-znxhx\" (UniqueName: \"kubernetes.io/projected/0e88a97a-bf2a-41f1-9377-688c7994159c-kube-api-access-znxhx\") on node \"crc\" DevicePath \"\"" Dec 03 13:01:27 crc kubenswrapper[5052]: I1203 13:01:27.764556 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/download-cache-edpm-compute-no-nodes-edpm-compute-no-nodes2tk9d"] Dec 03 13:01:27 crc kubenswrapper[5052]: W1203 13:01:27.768006 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod405110f4_d79e_4505_803b_5204317a4bfa.slice/crio-427c7d7d3cbd4283c440ab4e98f01970716e36e1176f0be66617020a2d84850b WatchSource:0}: Error finding container 427c7d7d3cbd4283c440ab4e98f01970716e36e1176f0be66617020a2d84850b: Status 404 returned error can't find the container with id 427c7d7d3cbd4283c440ab4e98f01970716e36e1176f0be66617020a2d84850b Dec 03 13:01:28 crc kubenswrapper[5052]: I1203 13:01:28.099578 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/download-cache-edpm-compute-no-nodes-edpm-compute-no-nodes2tk9d" event={"ID":"405110f4-d79e-4505-803b-5204317a4bfa","Type":"ContainerStarted","Data":"427c7d7d3cbd4283c440ab4e98f01970716e36e1176f0be66617020a2d84850b"} Dec 03 13:01:28 crc kubenswrapper[5052]: I1203 13:01:28.103618 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-hrlsl" event={"ID":"0e88a97a-bf2a-41f1-9377-688c7994159c","Type":"ContainerDied","Data":"5b7cc171298e6de0f6ae700406436883b4842c383337af73f0fb789ff5ef003e"} Dec 03 13:01:28 crc kubenswrapper[5052]: I1203 13:01:28.103665 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-hrlsl" Dec 03 13:01:28 crc kubenswrapper[5052]: I1203 13:01:28.103694 5052 scope.go:117] "RemoveContainer" containerID="97a0ff779664ce4b40814258307e0441a7290281ca8a36256521539283d3c3cc" Dec 03 13:01:28 crc kubenswrapper[5052]: I1203 13:01:28.130057 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-hrlsl"] Dec 03 13:01:28 crc kubenswrapper[5052]: I1203 13:01:28.135574 5052 scope.go:117] "RemoveContainer" containerID="9220f7f5e1c7e57fd2b6e468d93c19dc8c7d73d51f65490bfd6135547309af2b" Dec 03 13:01:28 crc kubenswrapper[5052]: I1203 13:01:28.145143 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-hrlsl"] Dec 03 13:01:29 crc kubenswrapper[5052]: I1203 13:01:29.121925 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/download-cache-edpm-compute-no-nodes-edpm-compute-no-nodes2tk9d" event={"ID":"405110f4-d79e-4505-803b-5204317a4bfa","Type":"ContainerStarted","Data":"c9486e62f27a2d8c73beccf2e07b988ff983ae8ccc8c27855fbf685b2e59bbdc"} Dec 03 13:01:29 crc kubenswrapper[5052]: I1203 13:01:29.144147 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/download-cache-edpm-compute-no-nodes-edpm-compute-no-nodes2tk9d" podStartSLOduration=1.645494472 podStartE2EDuration="2.144127782s" podCreationTimestamp="2025-12-03 13:01:27 +0000 UTC" firstStartedPulling="2025-12-03 13:01:27.770295622 +0000 UTC m=+6078.358537386" lastFinishedPulling="2025-12-03 13:01:28.268928932 +0000 UTC m=+6078.857170696" observedRunningTime="2025-12-03 13:01:29.141355253 +0000 UTC m=+6079.729597037" watchObservedRunningTime="2025-12-03 13:01:29.144127782 +0000 UTC m=+6079.732369566" Dec 03 13:01:29 crc kubenswrapper[5052]: I1203 13:01:29.683834 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0e88a97a-bf2a-41f1-9377-688c7994159c" path="/var/lib/kubelet/pods/0e88a97a-bf2a-41f1-9377-688c7994159c/volumes" Dec 03 13:01:30 crc kubenswrapper[5052]: I1203 13:01:30.152241 5052 generic.go:334] "Generic (PLEG): container finished" podID="405110f4-d79e-4505-803b-5204317a4bfa" containerID="c9486e62f27a2d8c73beccf2e07b988ff983ae8ccc8c27855fbf685b2e59bbdc" exitCode=0 Dec 03 13:01:30 crc kubenswrapper[5052]: I1203 13:01:30.152295 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/download-cache-edpm-compute-no-nodes-edpm-compute-no-nodes2tk9d" event={"ID":"405110f4-d79e-4505-803b-5204317a4bfa","Type":"ContainerDied","Data":"c9486e62f27a2d8c73beccf2e07b988ff983ae8ccc8c27855fbf685b2e59bbdc"} Dec 03 13:01:31 crc kubenswrapper[5052]: I1203 13:01:31.437110 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/download-cache-edpm-compute-no-nodes-edpm-compute-no-nodes2tk9d" Dec 03 13:01:31 crc kubenswrapper[5052]: I1203 13:01:31.507002 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/405110f4-d79e-4505-803b-5204317a4bfa-inventory\") pod \"405110f4-d79e-4505-803b-5204317a4bfa\" (UID: \"405110f4-d79e-4505-803b-5204317a4bfa\") " Dec 03 13:01:31 crc kubenswrapper[5052]: I1203 13:01:31.507069 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/405110f4-d79e-4505-803b-5204317a4bfa-ssh-key\") pod \"405110f4-d79e-4505-803b-5204317a4bfa\" (UID: \"405110f4-d79e-4505-803b-5204317a4bfa\") " Dec 03 13:01:31 crc kubenswrapper[5052]: I1203 13:01:31.507101 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-67s6l\" (UniqueName: \"kubernetes.io/projected/405110f4-d79e-4505-803b-5204317a4bfa-kube-api-access-67s6l\") pod \"405110f4-d79e-4505-803b-5204317a4bfa\" (UID: \"405110f4-d79e-4505-803b-5204317a4bfa\") " Dec 03 13:01:31 crc kubenswrapper[5052]: I1203 13:01:31.513752 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/405110f4-d79e-4505-803b-5204317a4bfa-kube-api-access-67s6l" (OuterVolumeSpecName: "kube-api-access-67s6l") pod "405110f4-d79e-4505-803b-5204317a4bfa" (UID: "405110f4-d79e-4505-803b-5204317a4bfa"). InnerVolumeSpecName "kube-api-access-67s6l". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:01:31 crc kubenswrapper[5052]: I1203 13:01:31.531483 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/405110f4-d79e-4505-803b-5204317a4bfa-ssh-key" (OuterVolumeSpecName: "ssh-key") pod "405110f4-d79e-4505-803b-5204317a4bfa" (UID: "405110f4-d79e-4505-803b-5204317a4bfa"). InnerVolumeSpecName "ssh-key". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:01:31 crc kubenswrapper[5052]: I1203 13:01:31.531919 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/405110f4-d79e-4505-803b-5204317a4bfa-inventory" (OuterVolumeSpecName: "inventory") pod "405110f4-d79e-4505-803b-5204317a4bfa" (UID: "405110f4-d79e-4505-803b-5204317a4bfa"). InnerVolumeSpecName "inventory". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:01:31 crc kubenswrapper[5052]: I1203 13:01:31.609590 5052 reconciler_common.go:293] "Volume detached for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/405110f4-d79e-4505-803b-5204317a4bfa-inventory\") on node \"crc\" DevicePath \"\"" Dec 03 13:01:31 crc kubenswrapper[5052]: I1203 13:01:31.609657 5052 reconciler_common.go:293] "Volume detached for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/405110f4-d79e-4505-803b-5204317a4bfa-ssh-key\") on node \"crc\" DevicePath \"\"" Dec 03 13:01:31 crc kubenswrapper[5052]: I1203 13:01:31.609676 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-67s6l\" (UniqueName: \"kubernetes.io/projected/405110f4-d79e-4505-803b-5204317a4bfa-kube-api-access-67s6l\") on node \"crc\" DevicePath \"\"" Dec 03 13:01:32 crc kubenswrapper[5052]: I1203 13:01:32.175827 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/download-cache-edpm-compute-no-nodes-edpm-compute-no-nodes2tk9d" event={"ID":"405110f4-d79e-4505-803b-5204317a4bfa","Type":"ContainerDied","Data":"427c7d7d3cbd4283c440ab4e98f01970716e36e1176f0be66617020a2d84850b"} Dec 03 13:01:32 crc kubenswrapper[5052]: I1203 13:01:32.175871 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="427c7d7d3cbd4283c440ab4e98f01970716e36e1176f0be66617020a2d84850b" Dec 03 13:01:32 crc kubenswrapper[5052]: I1203 13:01:32.175922 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/download-cache-edpm-compute-no-nodes-edpm-compute-no-nodes2tk9d" Dec 03 13:01:32 crc kubenswrapper[5052]: I1203 13:01:32.218249 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/bootstrap-edpm-compute-no-nodes-edpm-compute-no-nodes-684xr"] Dec 03 13:01:32 crc kubenswrapper[5052]: E1203 13:01:32.218828 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="405110f4-d79e-4505-803b-5204317a4bfa" containerName="download-cache-edpm-compute-no-nodes-edpm-compute-no-nodes" Dec 03 13:01:32 crc kubenswrapper[5052]: I1203 13:01:32.218853 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="405110f4-d79e-4505-803b-5204317a4bfa" containerName="download-cache-edpm-compute-no-nodes-edpm-compute-no-nodes" Dec 03 13:01:32 crc kubenswrapper[5052]: E1203 13:01:32.218899 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="0e88a97a-bf2a-41f1-9377-688c7994159c" containerName="dnsmasq-dns" Dec 03 13:01:32 crc kubenswrapper[5052]: I1203 13:01:32.218908 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="0e88a97a-bf2a-41f1-9377-688c7994159c" containerName="dnsmasq-dns" Dec 03 13:01:32 crc kubenswrapper[5052]: E1203 13:01:32.218949 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="0e88a97a-bf2a-41f1-9377-688c7994159c" containerName="init" Dec 03 13:01:32 crc kubenswrapper[5052]: I1203 13:01:32.218956 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="0e88a97a-bf2a-41f1-9377-688c7994159c" containerName="init" Dec 03 13:01:32 crc kubenswrapper[5052]: I1203 13:01:32.219235 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="0e88a97a-bf2a-41f1-9377-688c7994159c" containerName="dnsmasq-dns" Dec 03 13:01:32 crc kubenswrapper[5052]: I1203 13:01:32.219253 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="405110f4-d79e-4505-803b-5204317a4bfa" containerName="download-cache-edpm-compute-no-nodes-edpm-compute-no-nodes" Dec 03 13:01:32 crc kubenswrapper[5052]: I1203 13:01:32.220612 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/bootstrap-edpm-compute-no-nodes-edpm-compute-no-nodes-684xr" Dec 03 13:01:32 crc kubenswrapper[5052]: I1203 13:01:32.224173 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"dataplane-ansible-ssh-private-key-secret" Dec 03 13:01:32 crc kubenswrapper[5052]: I1203 13:01:32.224786 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"dataplanenodeset-edpm-compute-no-nodes" Dec 03 13:01:32 crc kubenswrapper[5052]: I1203 13:01:32.224841 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"edpm-compute-no-nodes-dockercfg-mkcgv" Dec 03 13:01:32 crc kubenswrapper[5052]: I1203 13:01:32.224952 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"combined-ca-bundle" Dec 03 13:01:32 crc kubenswrapper[5052]: I1203 13:01:32.225904 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"openstack-aee-default-env" Dec 03 13:01:32 crc kubenswrapper[5052]: I1203 13:01:32.227250 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/bootstrap-edpm-compute-no-nodes-edpm-compute-no-nodes-684xr"] Dec 03 13:01:32 crc kubenswrapper[5052]: I1203 13:01:32.319403 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/5a60accb-4776-478c-b8c4-799e2cb9af8d-inventory\") pod \"bootstrap-edpm-compute-no-nodes-edpm-compute-no-nodes-684xr\" (UID: \"5a60accb-4776-478c-b8c4-799e2cb9af8d\") " pod="openstack-kuttl-tests/bootstrap-edpm-compute-no-nodes-edpm-compute-no-nodes-684xr" Dec 03 13:01:32 crc kubenswrapper[5052]: I1203 13:01:32.319503 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ln9h6\" (UniqueName: \"kubernetes.io/projected/5a60accb-4776-478c-b8c4-799e2cb9af8d-kube-api-access-ln9h6\") pod \"bootstrap-edpm-compute-no-nodes-edpm-compute-no-nodes-684xr\" (UID: \"5a60accb-4776-478c-b8c4-799e2cb9af8d\") " pod="openstack-kuttl-tests/bootstrap-edpm-compute-no-nodes-edpm-compute-no-nodes-684xr" Dec 03 13:01:32 crc kubenswrapper[5052]: I1203 13:01:32.319561 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bootstrap-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5a60accb-4776-478c-b8c4-799e2cb9af8d-bootstrap-combined-ca-bundle\") pod \"bootstrap-edpm-compute-no-nodes-edpm-compute-no-nodes-684xr\" (UID: \"5a60accb-4776-478c-b8c4-799e2cb9af8d\") " pod="openstack-kuttl-tests/bootstrap-edpm-compute-no-nodes-edpm-compute-no-nodes-684xr" Dec 03 13:01:32 crc kubenswrapper[5052]: I1203 13:01:32.319595 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/5a60accb-4776-478c-b8c4-799e2cb9af8d-ssh-key\") pod \"bootstrap-edpm-compute-no-nodes-edpm-compute-no-nodes-684xr\" (UID: \"5a60accb-4776-478c-b8c4-799e2cb9af8d\") " pod="openstack-kuttl-tests/bootstrap-edpm-compute-no-nodes-edpm-compute-no-nodes-684xr" Dec 03 13:01:32 crc kubenswrapper[5052]: I1203 13:01:32.420947 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-ln9h6\" (UniqueName: \"kubernetes.io/projected/5a60accb-4776-478c-b8c4-799e2cb9af8d-kube-api-access-ln9h6\") pod \"bootstrap-edpm-compute-no-nodes-edpm-compute-no-nodes-684xr\" (UID: \"5a60accb-4776-478c-b8c4-799e2cb9af8d\") " pod="openstack-kuttl-tests/bootstrap-edpm-compute-no-nodes-edpm-compute-no-nodes-684xr" Dec 03 13:01:32 crc kubenswrapper[5052]: I1203 13:01:32.421019 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"bootstrap-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5a60accb-4776-478c-b8c4-799e2cb9af8d-bootstrap-combined-ca-bundle\") pod \"bootstrap-edpm-compute-no-nodes-edpm-compute-no-nodes-684xr\" (UID: \"5a60accb-4776-478c-b8c4-799e2cb9af8d\") " pod="openstack-kuttl-tests/bootstrap-edpm-compute-no-nodes-edpm-compute-no-nodes-684xr" Dec 03 13:01:32 crc kubenswrapper[5052]: I1203 13:01:32.421058 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/5a60accb-4776-478c-b8c4-799e2cb9af8d-ssh-key\") pod \"bootstrap-edpm-compute-no-nodes-edpm-compute-no-nodes-684xr\" (UID: \"5a60accb-4776-478c-b8c4-799e2cb9af8d\") " pod="openstack-kuttl-tests/bootstrap-edpm-compute-no-nodes-edpm-compute-no-nodes-684xr" Dec 03 13:01:32 crc kubenswrapper[5052]: I1203 13:01:32.421133 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/5a60accb-4776-478c-b8c4-799e2cb9af8d-inventory\") pod \"bootstrap-edpm-compute-no-nodes-edpm-compute-no-nodes-684xr\" (UID: \"5a60accb-4776-478c-b8c4-799e2cb9af8d\") " pod="openstack-kuttl-tests/bootstrap-edpm-compute-no-nodes-edpm-compute-no-nodes-684xr" Dec 03 13:01:32 crc kubenswrapper[5052]: I1203 13:01:32.425187 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/5a60accb-4776-478c-b8c4-799e2cb9af8d-ssh-key\") pod \"bootstrap-edpm-compute-no-nodes-edpm-compute-no-nodes-684xr\" (UID: \"5a60accb-4776-478c-b8c4-799e2cb9af8d\") " pod="openstack-kuttl-tests/bootstrap-edpm-compute-no-nodes-edpm-compute-no-nodes-684xr" Dec 03 13:01:32 crc kubenswrapper[5052]: I1203 13:01:32.425251 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/5a60accb-4776-478c-b8c4-799e2cb9af8d-inventory\") pod \"bootstrap-edpm-compute-no-nodes-edpm-compute-no-nodes-684xr\" (UID: \"5a60accb-4776-478c-b8c4-799e2cb9af8d\") " pod="openstack-kuttl-tests/bootstrap-edpm-compute-no-nodes-edpm-compute-no-nodes-684xr" Dec 03 13:01:32 crc kubenswrapper[5052]: I1203 13:01:32.425255 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"bootstrap-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5a60accb-4776-478c-b8c4-799e2cb9af8d-bootstrap-combined-ca-bundle\") pod \"bootstrap-edpm-compute-no-nodes-edpm-compute-no-nodes-684xr\" (UID: \"5a60accb-4776-478c-b8c4-799e2cb9af8d\") " pod="openstack-kuttl-tests/bootstrap-edpm-compute-no-nodes-edpm-compute-no-nodes-684xr" Dec 03 13:01:32 crc kubenswrapper[5052]: I1203 13:01:32.439317 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-ln9h6\" (UniqueName: \"kubernetes.io/projected/5a60accb-4776-478c-b8c4-799e2cb9af8d-kube-api-access-ln9h6\") pod \"bootstrap-edpm-compute-no-nodes-edpm-compute-no-nodes-684xr\" (UID: \"5a60accb-4776-478c-b8c4-799e2cb9af8d\") " pod="openstack-kuttl-tests/bootstrap-edpm-compute-no-nodes-edpm-compute-no-nodes-684xr" Dec 03 13:01:32 crc kubenswrapper[5052]: I1203 13:01:32.539293 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/bootstrap-edpm-compute-no-nodes-edpm-compute-no-nodes-684xr" Dec 03 13:01:32 crc kubenswrapper[5052]: I1203 13:01:32.951069 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/bootstrap-edpm-compute-no-nodes-edpm-compute-no-nodes-684xr"] Dec 03 13:01:33 crc kubenswrapper[5052]: I1203 13:01:33.186882 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/bootstrap-edpm-compute-no-nodes-edpm-compute-no-nodes-684xr" event={"ID":"5a60accb-4776-478c-b8c4-799e2cb9af8d","Type":"ContainerStarted","Data":"3210c0dd62a1cdb2e7453b4da7eb6536e8747a39122676fc349cfaa06c790280"} Dec 03 13:01:34 crc kubenswrapper[5052]: I1203 13:01:34.197185 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/bootstrap-edpm-compute-no-nodes-edpm-compute-no-nodes-684xr" event={"ID":"5a60accb-4776-478c-b8c4-799e2cb9af8d","Type":"ContainerStarted","Data":"64c56e75b7425465f6d086bf57aa1a1f81c6dd61a33a86f60578eb2ddf258c12"} Dec 03 13:01:34 crc kubenswrapper[5052]: I1203 13:01:34.215882 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/bootstrap-edpm-compute-no-nodes-edpm-compute-no-nodes-684xr" podStartSLOduration=1.5608170399999999 podStartE2EDuration="2.215865274s" podCreationTimestamp="2025-12-03 13:01:32 +0000 UTC" firstStartedPulling="2025-12-03 13:01:32.960650817 +0000 UTC m=+6083.548892581" lastFinishedPulling="2025-12-03 13:01:33.615699051 +0000 UTC m=+6084.203940815" observedRunningTime="2025-12-03 13:01:34.214062942 +0000 UTC m=+6084.802304706" watchObservedRunningTime="2025-12-03 13:01:34.215865274 +0000 UTC m=+6084.804107038" Dec 03 13:01:36 crc kubenswrapper[5052]: I1203 13:01:36.215323 5052 generic.go:334] "Generic (PLEG): container finished" podID="5a60accb-4776-478c-b8c4-799e2cb9af8d" containerID="64c56e75b7425465f6d086bf57aa1a1f81c6dd61a33a86f60578eb2ddf258c12" exitCode=0 Dec 03 13:01:36 crc kubenswrapper[5052]: I1203 13:01:36.215687 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/bootstrap-edpm-compute-no-nodes-edpm-compute-no-nodes-684xr" event={"ID":"5a60accb-4776-478c-b8c4-799e2cb9af8d","Type":"ContainerDied","Data":"64c56e75b7425465f6d086bf57aa1a1f81c6dd61a33a86f60578eb2ddf258c12"} Dec 03 13:01:37 crc kubenswrapper[5052]: I1203 13:01:37.533271 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/bootstrap-edpm-compute-no-nodes-edpm-compute-no-nodes-684xr" Dec 03 13:01:37 crc kubenswrapper[5052]: I1203 13:01:37.603955 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ln9h6\" (UniqueName: \"kubernetes.io/projected/5a60accb-4776-478c-b8c4-799e2cb9af8d-kube-api-access-ln9h6\") pod \"5a60accb-4776-478c-b8c4-799e2cb9af8d\" (UID: \"5a60accb-4776-478c-b8c4-799e2cb9af8d\") " Dec 03 13:01:37 crc kubenswrapper[5052]: I1203 13:01:37.604009 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/5a60accb-4776-478c-b8c4-799e2cb9af8d-inventory\") pod \"5a60accb-4776-478c-b8c4-799e2cb9af8d\" (UID: \"5a60accb-4776-478c-b8c4-799e2cb9af8d\") " Dec 03 13:01:37 crc kubenswrapper[5052]: I1203 13:01:37.604038 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/5a60accb-4776-478c-b8c4-799e2cb9af8d-ssh-key\") pod \"5a60accb-4776-478c-b8c4-799e2cb9af8d\" (UID: \"5a60accb-4776-478c-b8c4-799e2cb9af8d\") " Dec 03 13:01:37 crc kubenswrapper[5052]: I1203 13:01:37.604094 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"bootstrap-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5a60accb-4776-478c-b8c4-799e2cb9af8d-bootstrap-combined-ca-bundle\") pod \"5a60accb-4776-478c-b8c4-799e2cb9af8d\" (UID: \"5a60accb-4776-478c-b8c4-799e2cb9af8d\") " Dec 03 13:01:37 crc kubenswrapper[5052]: I1203 13:01:37.609811 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5a60accb-4776-478c-b8c4-799e2cb9af8d-bootstrap-combined-ca-bundle" (OuterVolumeSpecName: "bootstrap-combined-ca-bundle") pod "5a60accb-4776-478c-b8c4-799e2cb9af8d" (UID: "5a60accb-4776-478c-b8c4-799e2cb9af8d"). InnerVolumeSpecName "bootstrap-combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:01:37 crc kubenswrapper[5052]: I1203 13:01:37.612994 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5a60accb-4776-478c-b8c4-799e2cb9af8d-kube-api-access-ln9h6" (OuterVolumeSpecName: "kube-api-access-ln9h6") pod "5a60accb-4776-478c-b8c4-799e2cb9af8d" (UID: "5a60accb-4776-478c-b8c4-799e2cb9af8d"). InnerVolumeSpecName "kube-api-access-ln9h6". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:01:37 crc kubenswrapper[5052]: I1203 13:01:37.626835 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5a60accb-4776-478c-b8c4-799e2cb9af8d-ssh-key" (OuterVolumeSpecName: "ssh-key") pod "5a60accb-4776-478c-b8c4-799e2cb9af8d" (UID: "5a60accb-4776-478c-b8c4-799e2cb9af8d"). InnerVolumeSpecName "ssh-key". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:01:37 crc kubenswrapper[5052]: I1203 13:01:37.626914 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5a60accb-4776-478c-b8c4-799e2cb9af8d-inventory" (OuterVolumeSpecName: "inventory") pod "5a60accb-4776-478c-b8c4-799e2cb9af8d" (UID: "5a60accb-4776-478c-b8c4-799e2cb9af8d"). InnerVolumeSpecName "inventory". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:01:37 crc kubenswrapper[5052]: I1203 13:01:37.706376 5052 reconciler_common.go:293] "Volume detached for volume \"bootstrap-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5a60accb-4776-478c-b8c4-799e2cb9af8d-bootstrap-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 13:01:37 crc kubenswrapper[5052]: I1203 13:01:37.706425 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-ln9h6\" (UniqueName: \"kubernetes.io/projected/5a60accb-4776-478c-b8c4-799e2cb9af8d-kube-api-access-ln9h6\") on node \"crc\" DevicePath \"\"" Dec 03 13:01:37 crc kubenswrapper[5052]: I1203 13:01:37.706441 5052 reconciler_common.go:293] "Volume detached for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/5a60accb-4776-478c-b8c4-799e2cb9af8d-inventory\") on node \"crc\" DevicePath \"\"" Dec 03 13:01:37 crc kubenswrapper[5052]: I1203 13:01:37.706456 5052 reconciler_common.go:293] "Volume detached for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/5a60accb-4776-478c-b8c4-799e2cb9af8d-ssh-key\") on node \"crc\" DevicePath \"\"" Dec 03 13:01:38 crc kubenswrapper[5052]: I1203 13:01:38.242557 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/bootstrap-edpm-compute-no-nodes-edpm-compute-no-nodes-684xr" event={"ID":"5a60accb-4776-478c-b8c4-799e2cb9af8d","Type":"ContainerDied","Data":"3210c0dd62a1cdb2e7453b4da7eb6536e8747a39122676fc349cfaa06c790280"} Dec 03 13:01:38 crc kubenswrapper[5052]: I1203 13:01:38.243491 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3210c0dd62a1cdb2e7453b4da7eb6536e8747a39122676fc349cfaa06c790280" Dec 03 13:01:38 crc kubenswrapper[5052]: I1203 13:01:38.242652 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/bootstrap-edpm-compute-no-nodes-edpm-compute-no-nodes-684xr" Dec 03 13:01:38 crc kubenswrapper[5052]: I1203 13:01:38.299677 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/configure-network-edpm-compute-no-nodes-edpm-compute-no-no84qz4"] Dec 03 13:01:38 crc kubenswrapper[5052]: E1203 13:01:38.300111 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="5a60accb-4776-478c-b8c4-799e2cb9af8d" containerName="bootstrap-edpm-compute-no-nodes-edpm-compute-no-nodes" Dec 03 13:01:38 crc kubenswrapper[5052]: I1203 13:01:38.300140 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="5a60accb-4776-478c-b8c4-799e2cb9af8d" containerName="bootstrap-edpm-compute-no-nodes-edpm-compute-no-nodes" Dec 03 13:01:38 crc kubenswrapper[5052]: I1203 13:01:38.300299 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="5a60accb-4776-478c-b8c4-799e2cb9af8d" containerName="bootstrap-edpm-compute-no-nodes-edpm-compute-no-nodes" Dec 03 13:01:38 crc kubenswrapper[5052]: I1203 13:01:38.300923 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/configure-network-edpm-compute-no-nodes-edpm-compute-no-no84qz4" Dec 03 13:01:38 crc kubenswrapper[5052]: I1203 13:01:38.303865 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"edpm-compute-no-nodes-dockercfg-mkcgv" Dec 03 13:01:38 crc kubenswrapper[5052]: I1203 13:01:38.304013 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"openstack-aee-default-env" Dec 03 13:01:38 crc kubenswrapper[5052]: I1203 13:01:38.304294 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"dataplane-ansible-ssh-private-key-secret" Dec 03 13:01:38 crc kubenswrapper[5052]: I1203 13:01:38.307088 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"dataplanenodeset-edpm-compute-no-nodes" Dec 03 13:01:38 crc kubenswrapper[5052]: I1203 13:01:38.310039 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/configure-network-edpm-compute-no-nodes-edpm-compute-no-no84qz4"] Dec 03 13:01:38 crc kubenswrapper[5052]: I1203 13:01:38.419284 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j8626\" (UniqueName: \"kubernetes.io/projected/e16e2bad-fbf3-4817-bb52-422bc88a76b0-kube-api-access-j8626\") pod \"configure-network-edpm-compute-no-nodes-edpm-compute-no-no84qz4\" (UID: \"e16e2bad-fbf3-4817-bb52-422bc88a76b0\") " pod="openstack-kuttl-tests/configure-network-edpm-compute-no-nodes-edpm-compute-no-no84qz4" Dec 03 13:01:38 crc kubenswrapper[5052]: I1203 13:01:38.419491 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/e16e2bad-fbf3-4817-bb52-422bc88a76b0-inventory\") pod \"configure-network-edpm-compute-no-nodes-edpm-compute-no-no84qz4\" (UID: \"e16e2bad-fbf3-4817-bb52-422bc88a76b0\") " pod="openstack-kuttl-tests/configure-network-edpm-compute-no-nodes-edpm-compute-no-no84qz4" Dec 03 13:01:38 crc kubenswrapper[5052]: I1203 13:01:38.419611 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/e16e2bad-fbf3-4817-bb52-422bc88a76b0-ssh-key\") pod \"configure-network-edpm-compute-no-nodes-edpm-compute-no-no84qz4\" (UID: \"e16e2bad-fbf3-4817-bb52-422bc88a76b0\") " pod="openstack-kuttl-tests/configure-network-edpm-compute-no-nodes-edpm-compute-no-no84qz4" Dec 03 13:01:38 crc kubenswrapper[5052]: I1203 13:01:38.520897 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/e16e2bad-fbf3-4817-bb52-422bc88a76b0-inventory\") pod \"configure-network-edpm-compute-no-nodes-edpm-compute-no-no84qz4\" (UID: \"e16e2bad-fbf3-4817-bb52-422bc88a76b0\") " pod="openstack-kuttl-tests/configure-network-edpm-compute-no-nodes-edpm-compute-no-no84qz4" Dec 03 13:01:38 crc kubenswrapper[5052]: I1203 13:01:38.520993 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/e16e2bad-fbf3-4817-bb52-422bc88a76b0-ssh-key\") pod \"configure-network-edpm-compute-no-nodes-edpm-compute-no-no84qz4\" (UID: \"e16e2bad-fbf3-4817-bb52-422bc88a76b0\") " pod="openstack-kuttl-tests/configure-network-edpm-compute-no-nodes-edpm-compute-no-no84qz4" Dec 03 13:01:38 crc kubenswrapper[5052]: I1203 13:01:38.521088 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-j8626\" (UniqueName: \"kubernetes.io/projected/e16e2bad-fbf3-4817-bb52-422bc88a76b0-kube-api-access-j8626\") pod \"configure-network-edpm-compute-no-nodes-edpm-compute-no-no84qz4\" (UID: \"e16e2bad-fbf3-4817-bb52-422bc88a76b0\") " pod="openstack-kuttl-tests/configure-network-edpm-compute-no-nodes-edpm-compute-no-no84qz4" Dec 03 13:01:38 crc kubenswrapper[5052]: I1203 13:01:38.526186 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/e16e2bad-fbf3-4817-bb52-422bc88a76b0-inventory\") pod \"configure-network-edpm-compute-no-nodes-edpm-compute-no-no84qz4\" (UID: \"e16e2bad-fbf3-4817-bb52-422bc88a76b0\") " pod="openstack-kuttl-tests/configure-network-edpm-compute-no-nodes-edpm-compute-no-no84qz4" Dec 03 13:01:38 crc kubenswrapper[5052]: I1203 13:01:38.526215 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/e16e2bad-fbf3-4817-bb52-422bc88a76b0-ssh-key\") pod \"configure-network-edpm-compute-no-nodes-edpm-compute-no-no84qz4\" (UID: \"e16e2bad-fbf3-4817-bb52-422bc88a76b0\") " pod="openstack-kuttl-tests/configure-network-edpm-compute-no-nodes-edpm-compute-no-no84qz4" Dec 03 13:01:38 crc kubenswrapper[5052]: I1203 13:01:38.538285 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-j8626\" (UniqueName: \"kubernetes.io/projected/e16e2bad-fbf3-4817-bb52-422bc88a76b0-kube-api-access-j8626\") pod \"configure-network-edpm-compute-no-nodes-edpm-compute-no-no84qz4\" (UID: \"e16e2bad-fbf3-4817-bb52-422bc88a76b0\") " pod="openstack-kuttl-tests/configure-network-edpm-compute-no-nodes-edpm-compute-no-no84qz4" Dec 03 13:01:38 crc kubenswrapper[5052]: I1203 13:01:38.657721 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/configure-network-edpm-compute-no-nodes-edpm-compute-no-no84qz4" Dec 03 13:01:39 crc kubenswrapper[5052]: I1203 13:01:39.123667 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/configure-network-edpm-compute-no-nodes-edpm-compute-no-no84qz4"] Dec 03 13:01:39 crc kubenswrapper[5052]: I1203 13:01:39.253279 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/configure-network-edpm-compute-no-nodes-edpm-compute-no-no84qz4" event={"ID":"e16e2bad-fbf3-4817-bb52-422bc88a76b0","Type":"ContainerStarted","Data":"ef1ef226bef7e717ce382b441b9421f30d16ec9bfbd8808325f3549eeac6e8f5"} Dec 03 13:01:39 crc kubenswrapper[5052]: I1203 13:01:39.953398 5052 scope.go:117] "RemoveContainer" containerID="c4e366e272363beb523cf40ee21631ae5804252f9f976436f4bc9bc3040b1fc5" Dec 03 13:01:39 crc kubenswrapper[5052]: I1203 13:01:39.977888 5052 scope.go:117] "RemoveContainer" containerID="833e52f94069f90f5fa1a410cc52bd28c6338e81c70d46f601fe9f83a3f59a7e" Dec 03 13:01:40 crc kubenswrapper[5052]: I1203 13:01:40.007246 5052 scope.go:117] "RemoveContainer" containerID="124318204b3b140e1a75ced489756deb52efc6f18a43f8898a678f726dabdad6" Dec 03 13:01:40 crc kubenswrapper[5052]: I1203 13:01:40.040310 5052 scope.go:117] "RemoveContainer" containerID="8c4808921d6ec6221a9f281a4ad3dbd4fefd44edf1b417c22977010e21618367" Dec 03 13:01:40 crc kubenswrapper[5052]: I1203 13:01:40.060455 5052 scope.go:117] "RemoveContainer" containerID="dddf220d5101c12f66ac552ba347af307cf1e8ac13a83847f15138397c57745d" Dec 03 13:01:40 crc kubenswrapper[5052]: I1203 13:01:40.096671 5052 scope.go:117] "RemoveContainer" containerID="caab470e87cf074ae88fc5a8c4c3ffe707912bf57f9da9988867733b15f35a11" Dec 03 13:01:40 crc kubenswrapper[5052]: I1203 13:01:40.266301 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/configure-network-edpm-compute-no-nodes-edpm-compute-no-no84qz4" event={"ID":"e16e2bad-fbf3-4817-bb52-422bc88a76b0","Type":"ContainerStarted","Data":"c2ebf07b50f31375a46146921445745cfa84a1f04033cfe3691aef9bc2a5e576"} Dec 03 13:01:40 crc kubenswrapper[5052]: I1203 13:01:40.287407 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/configure-network-edpm-compute-no-nodes-edpm-compute-no-no84qz4" podStartSLOduration=1.77059781 podStartE2EDuration="2.287389869s" podCreationTimestamp="2025-12-03 13:01:38 +0000 UTC" firstStartedPulling="2025-12-03 13:01:39.132575024 +0000 UTC m=+6089.720816788" lastFinishedPulling="2025-12-03 13:01:39.649367083 +0000 UTC m=+6090.237608847" observedRunningTime="2025-12-03 13:01:40.283665033 +0000 UTC m=+6090.871906807" watchObservedRunningTime="2025-12-03 13:01:40.287389869 +0000 UTC m=+6090.875631633" Dec 03 13:01:41 crc kubenswrapper[5052]: I1203 13:01:41.279908 5052 generic.go:334] "Generic (PLEG): container finished" podID="e16e2bad-fbf3-4817-bb52-422bc88a76b0" containerID="c2ebf07b50f31375a46146921445745cfa84a1f04033cfe3691aef9bc2a5e576" exitCode=0 Dec 03 13:01:41 crc kubenswrapper[5052]: I1203 13:01:41.280132 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/configure-network-edpm-compute-no-nodes-edpm-compute-no-no84qz4" event={"ID":"e16e2bad-fbf3-4817-bb52-422bc88a76b0","Type":"ContainerDied","Data":"c2ebf07b50f31375a46146921445745cfa84a1f04033cfe3691aef9bc2a5e576"} Dec 03 13:01:42 crc kubenswrapper[5052]: I1203 13:01:42.561161 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/configure-network-edpm-compute-no-nodes-edpm-compute-no-no84qz4" Dec 03 13:01:42 crc kubenswrapper[5052]: I1203 13:01:42.583349 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/e16e2bad-fbf3-4817-bb52-422bc88a76b0-ssh-key\") pod \"e16e2bad-fbf3-4817-bb52-422bc88a76b0\" (UID: \"e16e2bad-fbf3-4817-bb52-422bc88a76b0\") " Dec 03 13:01:42 crc kubenswrapper[5052]: I1203 13:01:42.583845 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j8626\" (UniqueName: \"kubernetes.io/projected/e16e2bad-fbf3-4817-bb52-422bc88a76b0-kube-api-access-j8626\") pod \"e16e2bad-fbf3-4817-bb52-422bc88a76b0\" (UID: \"e16e2bad-fbf3-4817-bb52-422bc88a76b0\") " Dec 03 13:01:42 crc kubenswrapper[5052]: I1203 13:01:42.583921 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/e16e2bad-fbf3-4817-bb52-422bc88a76b0-inventory\") pod \"e16e2bad-fbf3-4817-bb52-422bc88a76b0\" (UID: \"e16e2bad-fbf3-4817-bb52-422bc88a76b0\") " Dec 03 13:01:42 crc kubenswrapper[5052]: I1203 13:01:42.596882 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e16e2bad-fbf3-4817-bb52-422bc88a76b0-kube-api-access-j8626" (OuterVolumeSpecName: "kube-api-access-j8626") pod "e16e2bad-fbf3-4817-bb52-422bc88a76b0" (UID: "e16e2bad-fbf3-4817-bb52-422bc88a76b0"). InnerVolumeSpecName "kube-api-access-j8626". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:01:42 crc kubenswrapper[5052]: I1203 13:01:42.607915 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e16e2bad-fbf3-4817-bb52-422bc88a76b0-inventory" (OuterVolumeSpecName: "inventory") pod "e16e2bad-fbf3-4817-bb52-422bc88a76b0" (UID: "e16e2bad-fbf3-4817-bb52-422bc88a76b0"). InnerVolumeSpecName "inventory". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:01:42 crc kubenswrapper[5052]: I1203 13:01:42.610798 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e16e2bad-fbf3-4817-bb52-422bc88a76b0-ssh-key" (OuterVolumeSpecName: "ssh-key") pod "e16e2bad-fbf3-4817-bb52-422bc88a76b0" (UID: "e16e2bad-fbf3-4817-bb52-422bc88a76b0"). InnerVolumeSpecName "ssh-key". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:01:42 crc kubenswrapper[5052]: I1203 13:01:42.686189 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-j8626\" (UniqueName: \"kubernetes.io/projected/e16e2bad-fbf3-4817-bb52-422bc88a76b0-kube-api-access-j8626\") on node \"crc\" DevicePath \"\"" Dec 03 13:01:42 crc kubenswrapper[5052]: I1203 13:01:42.686233 5052 reconciler_common.go:293] "Volume detached for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/e16e2bad-fbf3-4817-bb52-422bc88a76b0-inventory\") on node \"crc\" DevicePath \"\"" Dec 03 13:01:42 crc kubenswrapper[5052]: I1203 13:01:42.686247 5052 reconciler_common.go:293] "Volume detached for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/e16e2bad-fbf3-4817-bb52-422bc88a76b0-ssh-key\") on node \"crc\" DevicePath \"\"" Dec 03 13:01:42 crc kubenswrapper[5052]: I1203 13:01:42.761769 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/validate-network-edpm-compute-no-nodes-edpm-compute-no-nodfw8nd"] Dec 03 13:01:42 crc kubenswrapper[5052]: E1203 13:01:42.762138 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="e16e2bad-fbf3-4817-bb52-422bc88a76b0" containerName="configure-network-edpm-compute-no-nodes-edpm-compute-no-nodes" Dec 03 13:01:42 crc kubenswrapper[5052]: I1203 13:01:42.762163 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="e16e2bad-fbf3-4817-bb52-422bc88a76b0" containerName="configure-network-edpm-compute-no-nodes-edpm-compute-no-nodes" Dec 03 13:01:42 crc kubenswrapper[5052]: I1203 13:01:42.763498 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="e16e2bad-fbf3-4817-bb52-422bc88a76b0" containerName="configure-network-edpm-compute-no-nodes-edpm-compute-no-nodes" Dec 03 13:01:42 crc kubenswrapper[5052]: I1203 13:01:42.764698 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/validate-network-edpm-compute-no-nodes-edpm-compute-no-nodfw8nd" Dec 03 13:01:42 crc kubenswrapper[5052]: I1203 13:01:42.797673 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/validate-network-edpm-compute-no-nodes-edpm-compute-no-nodfw8nd"] Dec 03 13:01:42 crc kubenswrapper[5052]: I1203 13:01:42.797753 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tfqbk\" (UniqueName: \"kubernetes.io/projected/07ec94a0-7a1b-4062-aeca-22402b5f344a-kube-api-access-tfqbk\") pod \"validate-network-edpm-compute-no-nodes-edpm-compute-no-nodfw8nd\" (UID: \"07ec94a0-7a1b-4062-aeca-22402b5f344a\") " pod="openstack-kuttl-tests/validate-network-edpm-compute-no-nodes-edpm-compute-no-nodfw8nd" Dec 03 13:01:42 crc kubenswrapper[5052]: I1203 13:01:42.797916 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/07ec94a0-7a1b-4062-aeca-22402b5f344a-ssh-key\") pod \"validate-network-edpm-compute-no-nodes-edpm-compute-no-nodfw8nd\" (UID: \"07ec94a0-7a1b-4062-aeca-22402b5f344a\") " pod="openstack-kuttl-tests/validate-network-edpm-compute-no-nodes-edpm-compute-no-nodfw8nd" Dec 03 13:01:42 crc kubenswrapper[5052]: I1203 13:01:42.798050 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/07ec94a0-7a1b-4062-aeca-22402b5f344a-inventory\") pod \"validate-network-edpm-compute-no-nodes-edpm-compute-no-nodfw8nd\" (UID: \"07ec94a0-7a1b-4062-aeca-22402b5f344a\") " pod="openstack-kuttl-tests/validate-network-edpm-compute-no-nodes-edpm-compute-no-nodfw8nd" Dec 03 13:01:42 crc kubenswrapper[5052]: I1203 13:01:42.899183 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/07ec94a0-7a1b-4062-aeca-22402b5f344a-inventory\") pod \"validate-network-edpm-compute-no-nodes-edpm-compute-no-nodfw8nd\" (UID: \"07ec94a0-7a1b-4062-aeca-22402b5f344a\") " pod="openstack-kuttl-tests/validate-network-edpm-compute-no-nodes-edpm-compute-no-nodfw8nd" Dec 03 13:01:42 crc kubenswrapper[5052]: I1203 13:01:42.899257 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-tfqbk\" (UniqueName: \"kubernetes.io/projected/07ec94a0-7a1b-4062-aeca-22402b5f344a-kube-api-access-tfqbk\") pod \"validate-network-edpm-compute-no-nodes-edpm-compute-no-nodfw8nd\" (UID: \"07ec94a0-7a1b-4062-aeca-22402b5f344a\") " pod="openstack-kuttl-tests/validate-network-edpm-compute-no-nodes-edpm-compute-no-nodfw8nd" Dec 03 13:01:42 crc kubenswrapper[5052]: I1203 13:01:42.899300 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/07ec94a0-7a1b-4062-aeca-22402b5f344a-ssh-key\") pod \"validate-network-edpm-compute-no-nodes-edpm-compute-no-nodfw8nd\" (UID: \"07ec94a0-7a1b-4062-aeca-22402b5f344a\") " pod="openstack-kuttl-tests/validate-network-edpm-compute-no-nodes-edpm-compute-no-nodfw8nd" Dec 03 13:01:42 crc kubenswrapper[5052]: I1203 13:01:42.903344 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/07ec94a0-7a1b-4062-aeca-22402b5f344a-ssh-key\") pod \"validate-network-edpm-compute-no-nodes-edpm-compute-no-nodfw8nd\" (UID: \"07ec94a0-7a1b-4062-aeca-22402b5f344a\") " pod="openstack-kuttl-tests/validate-network-edpm-compute-no-nodes-edpm-compute-no-nodfw8nd" Dec 03 13:01:42 crc kubenswrapper[5052]: I1203 13:01:42.903423 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/07ec94a0-7a1b-4062-aeca-22402b5f344a-inventory\") pod \"validate-network-edpm-compute-no-nodes-edpm-compute-no-nodfw8nd\" (UID: \"07ec94a0-7a1b-4062-aeca-22402b5f344a\") " pod="openstack-kuttl-tests/validate-network-edpm-compute-no-nodes-edpm-compute-no-nodfw8nd" Dec 03 13:01:42 crc kubenswrapper[5052]: I1203 13:01:42.918293 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-tfqbk\" (UniqueName: \"kubernetes.io/projected/07ec94a0-7a1b-4062-aeca-22402b5f344a-kube-api-access-tfqbk\") pod \"validate-network-edpm-compute-no-nodes-edpm-compute-no-nodfw8nd\" (UID: \"07ec94a0-7a1b-4062-aeca-22402b5f344a\") " pod="openstack-kuttl-tests/validate-network-edpm-compute-no-nodes-edpm-compute-no-nodfw8nd" Dec 03 13:01:43 crc kubenswrapper[5052]: I1203 13:01:43.098733 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/validate-network-edpm-compute-no-nodes-edpm-compute-no-nodfw8nd" Dec 03 13:01:43 crc kubenswrapper[5052]: I1203 13:01:43.305285 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/configure-network-edpm-compute-no-nodes-edpm-compute-no-no84qz4" event={"ID":"e16e2bad-fbf3-4817-bb52-422bc88a76b0","Type":"ContainerDied","Data":"ef1ef226bef7e717ce382b441b9421f30d16ec9bfbd8808325f3549eeac6e8f5"} Dec 03 13:01:43 crc kubenswrapper[5052]: I1203 13:01:43.305698 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ef1ef226bef7e717ce382b441b9421f30d16ec9bfbd8808325f3549eeac6e8f5" Dec 03 13:01:43 crc kubenswrapper[5052]: I1203 13:01:43.305505 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/configure-network-edpm-compute-no-nodes-edpm-compute-no-no84qz4" Dec 03 13:01:43 crc kubenswrapper[5052]: I1203 13:01:43.572472 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/validate-network-edpm-compute-no-nodes-edpm-compute-no-nodfw8nd"] Dec 03 13:01:43 crc kubenswrapper[5052]: W1203 13:01:43.574573 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod07ec94a0_7a1b_4062_aeca_22402b5f344a.slice/crio-ad533acd51d3356d6f51715ec58099ddc7b0648670b005c696ffa971296c7fac WatchSource:0}: Error finding container ad533acd51d3356d6f51715ec58099ddc7b0648670b005c696ffa971296c7fac: Status 404 returned error can't find the container with id ad533acd51d3356d6f51715ec58099ddc7b0648670b005c696ffa971296c7fac Dec 03 13:01:44 crc kubenswrapper[5052]: I1203 13:01:44.314849 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/validate-network-edpm-compute-no-nodes-edpm-compute-no-nodfw8nd" event={"ID":"07ec94a0-7a1b-4062-aeca-22402b5f344a","Type":"ContainerStarted","Data":"c06935f9a7da4acc8f1b8e6b1c55344bff27f44a9b2cb91d2d0103498ab214fd"} Dec 03 13:01:44 crc kubenswrapper[5052]: I1203 13:01:44.315183 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/validate-network-edpm-compute-no-nodes-edpm-compute-no-nodfw8nd" event={"ID":"07ec94a0-7a1b-4062-aeca-22402b5f344a","Type":"ContainerStarted","Data":"ad533acd51d3356d6f51715ec58099ddc7b0648670b005c696ffa971296c7fac"} Dec 03 13:01:44 crc kubenswrapper[5052]: I1203 13:01:44.330753 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/validate-network-edpm-compute-no-nodes-edpm-compute-no-nodfw8nd" podStartSLOduration=1.8242423570000001 podStartE2EDuration="2.330735042s" podCreationTimestamp="2025-12-03 13:01:42 +0000 UTC" firstStartedPulling="2025-12-03 13:01:43.576323397 +0000 UTC m=+6094.164565181" lastFinishedPulling="2025-12-03 13:01:44.082816102 +0000 UTC m=+6094.671057866" observedRunningTime="2025-12-03 13:01:44.328180469 +0000 UTC m=+6094.916422223" watchObservedRunningTime="2025-12-03 13:01:44.330735042 +0000 UTC m=+6094.918976806" Dec 03 13:01:46 crc kubenswrapper[5052]: I1203 13:01:46.334912 5052 generic.go:334] "Generic (PLEG): container finished" podID="07ec94a0-7a1b-4062-aeca-22402b5f344a" containerID="c06935f9a7da4acc8f1b8e6b1c55344bff27f44a9b2cb91d2d0103498ab214fd" exitCode=0 Dec 03 13:01:46 crc kubenswrapper[5052]: I1203 13:01:46.334962 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/validate-network-edpm-compute-no-nodes-edpm-compute-no-nodfw8nd" event={"ID":"07ec94a0-7a1b-4062-aeca-22402b5f344a","Type":"ContainerDied","Data":"c06935f9a7da4acc8f1b8e6b1c55344bff27f44a9b2cb91d2d0103498ab214fd"} Dec 03 13:01:47 crc kubenswrapper[5052]: I1203 13:01:47.648105 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/validate-network-edpm-compute-no-nodes-edpm-compute-no-nodfw8nd" Dec 03 13:01:47 crc kubenswrapper[5052]: I1203 13:01:47.691518 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tfqbk\" (UniqueName: \"kubernetes.io/projected/07ec94a0-7a1b-4062-aeca-22402b5f344a-kube-api-access-tfqbk\") pod \"07ec94a0-7a1b-4062-aeca-22402b5f344a\" (UID: \"07ec94a0-7a1b-4062-aeca-22402b5f344a\") " Dec 03 13:01:47 crc kubenswrapper[5052]: I1203 13:01:47.691659 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/07ec94a0-7a1b-4062-aeca-22402b5f344a-ssh-key\") pod \"07ec94a0-7a1b-4062-aeca-22402b5f344a\" (UID: \"07ec94a0-7a1b-4062-aeca-22402b5f344a\") " Dec 03 13:01:47 crc kubenswrapper[5052]: I1203 13:01:47.691684 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/07ec94a0-7a1b-4062-aeca-22402b5f344a-inventory\") pod \"07ec94a0-7a1b-4062-aeca-22402b5f344a\" (UID: \"07ec94a0-7a1b-4062-aeca-22402b5f344a\") " Dec 03 13:01:47 crc kubenswrapper[5052]: I1203 13:01:47.697188 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/07ec94a0-7a1b-4062-aeca-22402b5f344a-kube-api-access-tfqbk" (OuterVolumeSpecName: "kube-api-access-tfqbk") pod "07ec94a0-7a1b-4062-aeca-22402b5f344a" (UID: "07ec94a0-7a1b-4062-aeca-22402b5f344a"). InnerVolumeSpecName "kube-api-access-tfqbk". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:01:47 crc kubenswrapper[5052]: I1203 13:01:47.714752 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/07ec94a0-7a1b-4062-aeca-22402b5f344a-ssh-key" (OuterVolumeSpecName: "ssh-key") pod "07ec94a0-7a1b-4062-aeca-22402b5f344a" (UID: "07ec94a0-7a1b-4062-aeca-22402b5f344a"). InnerVolumeSpecName "ssh-key". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:01:47 crc kubenswrapper[5052]: I1203 13:01:47.714786 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/07ec94a0-7a1b-4062-aeca-22402b5f344a-inventory" (OuterVolumeSpecName: "inventory") pod "07ec94a0-7a1b-4062-aeca-22402b5f344a" (UID: "07ec94a0-7a1b-4062-aeca-22402b5f344a"). InnerVolumeSpecName "inventory". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:01:47 crc kubenswrapper[5052]: I1203 13:01:47.792990 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-tfqbk\" (UniqueName: \"kubernetes.io/projected/07ec94a0-7a1b-4062-aeca-22402b5f344a-kube-api-access-tfqbk\") on node \"crc\" DevicePath \"\"" Dec 03 13:01:47 crc kubenswrapper[5052]: I1203 13:01:47.793021 5052 reconciler_common.go:293] "Volume detached for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/07ec94a0-7a1b-4062-aeca-22402b5f344a-ssh-key\") on node \"crc\" DevicePath \"\"" Dec 03 13:01:47 crc kubenswrapper[5052]: I1203 13:01:47.793030 5052 reconciler_common.go:293] "Volume detached for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/07ec94a0-7a1b-4062-aeca-22402b5f344a-inventory\") on node \"crc\" DevicePath \"\"" Dec 03 13:01:48 crc kubenswrapper[5052]: I1203 13:01:48.377457 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/validate-network-edpm-compute-no-nodes-edpm-compute-no-nodfw8nd" event={"ID":"07ec94a0-7a1b-4062-aeca-22402b5f344a","Type":"ContainerDied","Data":"ad533acd51d3356d6f51715ec58099ddc7b0648670b005c696ffa971296c7fac"} Dec 03 13:01:48 crc kubenswrapper[5052]: I1203 13:01:48.377566 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ad533acd51d3356d6f51715ec58099ddc7b0648670b005c696ffa971296c7fac" Dec 03 13:01:48 crc kubenswrapper[5052]: I1203 13:01:48.377639 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/validate-network-edpm-compute-no-nodes-edpm-compute-no-nodfw8nd" Dec 03 13:01:48 crc kubenswrapper[5052]: I1203 13:01:48.434299 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/install-os-edpm-compute-no-nodes-edpm-compute-no-nodes-l4h9g"] Dec 03 13:01:48 crc kubenswrapper[5052]: E1203 13:01:48.434680 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="07ec94a0-7a1b-4062-aeca-22402b5f344a" containerName="validate-network-edpm-compute-no-nodes-edpm-compute-no-nodes" Dec 03 13:01:48 crc kubenswrapper[5052]: I1203 13:01:48.434698 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="07ec94a0-7a1b-4062-aeca-22402b5f344a" containerName="validate-network-edpm-compute-no-nodes-edpm-compute-no-nodes" Dec 03 13:01:48 crc kubenswrapper[5052]: I1203 13:01:48.434860 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="07ec94a0-7a1b-4062-aeca-22402b5f344a" containerName="validate-network-edpm-compute-no-nodes-edpm-compute-no-nodes" Dec 03 13:01:48 crc kubenswrapper[5052]: I1203 13:01:48.435365 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/install-os-edpm-compute-no-nodes-edpm-compute-no-nodes-l4h9g" Dec 03 13:01:48 crc kubenswrapper[5052]: I1203 13:01:48.437665 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"openstack-aee-default-env" Dec 03 13:01:48 crc kubenswrapper[5052]: I1203 13:01:48.437995 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"dataplanenodeset-edpm-compute-no-nodes" Dec 03 13:01:48 crc kubenswrapper[5052]: I1203 13:01:48.438104 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"edpm-compute-no-nodes-dockercfg-mkcgv" Dec 03 13:01:48 crc kubenswrapper[5052]: I1203 13:01:48.438214 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"dataplane-ansible-ssh-private-key-secret" Dec 03 13:01:48 crc kubenswrapper[5052]: I1203 13:01:48.441407 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/install-os-edpm-compute-no-nodes-edpm-compute-no-nodes-l4h9g"] Dec 03 13:01:48 crc kubenswrapper[5052]: I1203 13:01:48.506200 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/37fe7283-01d0-4402-a15c-830a8b78aa20-ssh-key\") pod \"install-os-edpm-compute-no-nodes-edpm-compute-no-nodes-l4h9g\" (UID: \"37fe7283-01d0-4402-a15c-830a8b78aa20\") " pod="openstack-kuttl-tests/install-os-edpm-compute-no-nodes-edpm-compute-no-nodes-l4h9g" Dec 03 13:01:48 crc kubenswrapper[5052]: I1203 13:01:48.506360 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/37fe7283-01d0-4402-a15c-830a8b78aa20-inventory\") pod \"install-os-edpm-compute-no-nodes-edpm-compute-no-nodes-l4h9g\" (UID: \"37fe7283-01d0-4402-a15c-830a8b78aa20\") " pod="openstack-kuttl-tests/install-os-edpm-compute-no-nodes-edpm-compute-no-nodes-l4h9g" Dec 03 13:01:48 crc kubenswrapper[5052]: I1203 13:01:48.506419 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tcvj4\" (UniqueName: \"kubernetes.io/projected/37fe7283-01d0-4402-a15c-830a8b78aa20-kube-api-access-tcvj4\") pod \"install-os-edpm-compute-no-nodes-edpm-compute-no-nodes-l4h9g\" (UID: \"37fe7283-01d0-4402-a15c-830a8b78aa20\") " pod="openstack-kuttl-tests/install-os-edpm-compute-no-nodes-edpm-compute-no-nodes-l4h9g" Dec 03 13:01:48 crc kubenswrapper[5052]: I1203 13:01:48.608307 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/37fe7283-01d0-4402-a15c-830a8b78aa20-inventory\") pod \"install-os-edpm-compute-no-nodes-edpm-compute-no-nodes-l4h9g\" (UID: \"37fe7283-01d0-4402-a15c-830a8b78aa20\") " pod="openstack-kuttl-tests/install-os-edpm-compute-no-nodes-edpm-compute-no-nodes-l4h9g" Dec 03 13:01:48 crc kubenswrapper[5052]: I1203 13:01:48.608371 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-tcvj4\" (UniqueName: \"kubernetes.io/projected/37fe7283-01d0-4402-a15c-830a8b78aa20-kube-api-access-tcvj4\") pod \"install-os-edpm-compute-no-nodes-edpm-compute-no-nodes-l4h9g\" (UID: \"37fe7283-01d0-4402-a15c-830a8b78aa20\") " pod="openstack-kuttl-tests/install-os-edpm-compute-no-nodes-edpm-compute-no-nodes-l4h9g" Dec 03 13:01:48 crc kubenswrapper[5052]: I1203 13:01:48.608460 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/37fe7283-01d0-4402-a15c-830a8b78aa20-ssh-key\") pod \"install-os-edpm-compute-no-nodes-edpm-compute-no-nodes-l4h9g\" (UID: \"37fe7283-01d0-4402-a15c-830a8b78aa20\") " pod="openstack-kuttl-tests/install-os-edpm-compute-no-nodes-edpm-compute-no-nodes-l4h9g" Dec 03 13:01:48 crc kubenswrapper[5052]: I1203 13:01:48.613564 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/37fe7283-01d0-4402-a15c-830a8b78aa20-ssh-key\") pod \"install-os-edpm-compute-no-nodes-edpm-compute-no-nodes-l4h9g\" (UID: \"37fe7283-01d0-4402-a15c-830a8b78aa20\") " pod="openstack-kuttl-tests/install-os-edpm-compute-no-nodes-edpm-compute-no-nodes-l4h9g" Dec 03 13:01:48 crc kubenswrapper[5052]: I1203 13:01:48.614399 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/37fe7283-01d0-4402-a15c-830a8b78aa20-inventory\") pod \"install-os-edpm-compute-no-nodes-edpm-compute-no-nodes-l4h9g\" (UID: \"37fe7283-01d0-4402-a15c-830a8b78aa20\") " pod="openstack-kuttl-tests/install-os-edpm-compute-no-nodes-edpm-compute-no-nodes-l4h9g" Dec 03 13:01:48 crc kubenswrapper[5052]: I1203 13:01:48.628104 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-tcvj4\" (UniqueName: \"kubernetes.io/projected/37fe7283-01d0-4402-a15c-830a8b78aa20-kube-api-access-tcvj4\") pod \"install-os-edpm-compute-no-nodes-edpm-compute-no-nodes-l4h9g\" (UID: \"37fe7283-01d0-4402-a15c-830a8b78aa20\") " pod="openstack-kuttl-tests/install-os-edpm-compute-no-nodes-edpm-compute-no-nodes-l4h9g" Dec 03 13:01:48 crc kubenswrapper[5052]: I1203 13:01:48.755477 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/install-os-edpm-compute-no-nodes-edpm-compute-no-nodes-l4h9g" Dec 03 13:01:49 crc kubenswrapper[5052]: I1203 13:01:49.166237 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/install-os-edpm-compute-no-nodes-edpm-compute-no-nodes-l4h9g"] Dec 03 13:01:49 crc kubenswrapper[5052]: I1203 13:01:49.389071 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/install-os-edpm-compute-no-nodes-edpm-compute-no-nodes-l4h9g" event={"ID":"37fe7283-01d0-4402-a15c-830a8b78aa20","Type":"ContainerStarted","Data":"17d7dac3448b15a2431d48884abc701151e1a739d047b533d158f3543acc18f1"} Dec 03 13:01:50 crc kubenswrapper[5052]: I1203 13:01:50.401199 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/install-os-edpm-compute-no-nodes-edpm-compute-no-nodes-l4h9g" event={"ID":"37fe7283-01d0-4402-a15c-830a8b78aa20","Type":"ContainerStarted","Data":"851473ffe16e2f7016240764347d2fcc40e389321b7a88e89031ad0e41410688"} Dec 03 13:01:51 crc kubenswrapper[5052]: I1203 13:01:51.410490 5052 generic.go:334] "Generic (PLEG): container finished" podID="37fe7283-01d0-4402-a15c-830a8b78aa20" containerID="851473ffe16e2f7016240764347d2fcc40e389321b7a88e89031ad0e41410688" exitCode=0 Dec 03 13:01:51 crc kubenswrapper[5052]: I1203 13:01:51.410552 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/install-os-edpm-compute-no-nodes-edpm-compute-no-nodes-l4h9g" event={"ID":"37fe7283-01d0-4402-a15c-830a8b78aa20","Type":"ContainerDied","Data":"851473ffe16e2f7016240764347d2fcc40e389321b7a88e89031ad0e41410688"} Dec 03 13:01:52 crc kubenswrapper[5052]: I1203 13:01:52.675788 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/install-os-edpm-compute-no-nodes-edpm-compute-no-nodes-l4h9g" Dec 03 13:01:52 crc kubenswrapper[5052]: I1203 13:01:52.765337 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/configure-os-edpm-compute-no-nodes-edpm-compute-no-nodes-l6qqd"] Dec 03 13:01:52 crc kubenswrapper[5052]: E1203 13:01:52.765694 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="37fe7283-01d0-4402-a15c-830a8b78aa20" containerName="install-os-edpm-compute-no-nodes-edpm-compute-no-nodes" Dec 03 13:01:52 crc kubenswrapper[5052]: I1203 13:01:52.765719 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="37fe7283-01d0-4402-a15c-830a8b78aa20" containerName="install-os-edpm-compute-no-nodes-edpm-compute-no-nodes" Dec 03 13:01:52 crc kubenswrapper[5052]: I1203 13:01:52.765915 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="37fe7283-01d0-4402-a15c-830a8b78aa20" containerName="install-os-edpm-compute-no-nodes-edpm-compute-no-nodes" Dec 03 13:01:52 crc kubenswrapper[5052]: I1203 13:01:52.766936 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/configure-os-edpm-compute-no-nodes-edpm-compute-no-nodes-l6qqd" Dec 03 13:01:52 crc kubenswrapper[5052]: I1203 13:01:52.774622 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tcvj4\" (UniqueName: \"kubernetes.io/projected/37fe7283-01d0-4402-a15c-830a8b78aa20-kube-api-access-tcvj4\") pod \"37fe7283-01d0-4402-a15c-830a8b78aa20\" (UID: \"37fe7283-01d0-4402-a15c-830a8b78aa20\") " Dec 03 13:01:52 crc kubenswrapper[5052]: I1203 13:01:52.774711 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/37fe7283-01d0-4402-a15c-830a8b78aa20-ssh-key\") pod \"37fe7283-01d0-4402-a15c-830a8b78aa20\" (UID: \"37fe7283-01d0-4402-a15c-830a8b78aa20\") " Dec 03 13:01:52 crc kubenswrapper[5052]: I1203 13:01:52.774730 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/37fe7283-01d0-4402-a15c-830a8b78aa20-inventory\") pod \"37fe7283-01d0-4402-a15c-830a8b78aa20\" (UID: \"37fe7283-01d0-4402-a15c-830a8b78aa20\") " Dec 03 13:01:52 crc kubenswrapper[5052]: I1203 13:01:52.780118 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/configure-os-edpm-compute-no-nodes-edpm-compute-no-nodes-l6qqd"] Dec 03 13:01:52 crc kubenswrapper[5052]: I1203 13:01:52.794007 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/37fe7283-01d0-4402-a15c-830a8b78aa20-kube-api-access-tcvj4" (OuterVolumeSpecName: "kube-api-access-tcvj4") pod "37fe7283-01d0-4402-a15c-830a8b78aa20" (UID: "37fe7283-01d0-4402-a15c-830a8b78aa20"). InnerVolumeSpecName "kube-api-access-tcvj4". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:01:52 crc kubenswrapper[5052]: I1203 13:01:52.799905 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/37fe7283-01d0-4402-a15c-830a8b78aa20-inventory" (OuterVolumeSpecName: "inventory") pod "37fe7283-01d0-4402-a15c-830a8b78aa20" (UID: "37fe7283-01d0-4402-a15c-830a8b78aa20"). InnerVolumeSpecName "inventory". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:01:52 crc kubenswrapper[5052]: I1203 13:01:52.801520 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/37fe7283-01d0-4402-a15c-830a8b78aa20-ssh-key" (OuterVolumeSpecName: "ssh-key") pod "37fe7283-01d0-4402-a15c-830a8b78aa20" (UID: "37fe7283-01d0-4402-a15c-830a8b78aa20"). InnerVolumeSpecName "ssh-key". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:01:52 crc kubenswrapper[5052]: I1203 13:01:52.875919 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/0a9ff100-5a4a-4ba1-aba6-44c45c064fb1-inventory\") pod \"configure-os-edpm-compute-no-nodes-edpm-compute-no-nodes-l6qqd\" (UID: \"0a9ff100-5a4a-4ba1-aba6-44c45c064fb1\") " pod="openstack-kuttl-tests/configure-os-edpm-compute-no-nodes-edpm-compute-no-nodes-l6qqd" Dec 03 13:01:52 crc kubenswrapper[5052]: I1203 13:01:52.875990 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fgbjs\" (UniqueName: \"kubernetes.io/projected/0a9ff100-5a4a-4ba1-aba6-44c45c064fb1-kube-api-access-fgbjs\") pod \"configure-os-edpm-compute-no-nodes-edpm-compute-no-nodes-l6qqd\" (UID: \"0a9ff100-5a4a-4ba1-aba6-44c45c064fb1\") " pod="openstack-kuttl-tests/configure-os-edpm-compute-no-nodes-edpm-compute-no-nodes-l6qqd" Dec 03 13:01:52 crc kubenswrapper[5052]: I1203 13:01:52.876013 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/0a9ff100-5a4a-4ba1-aba6-44c45c064fb1-ssh-key\") pod \"configure-os-edpm-compute-no-nodes-edpm-compute-no-nodes-l6qqd\" (UID: \"0a9ff100-5a4a-4ba1-aba6-44c45c064fb1\") " pod="openstack-kuttl-tests/configure-os-edpm-compute-no-nodes-edpm-compute-no-nodes-l6qqd" Dec 03 13:01:52 crc kubenswrapper[5052]: I1203 13:01:52.876173 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-tcvj4\" (UniqueName: \"kubernetes.io/projected/37fe7283-01d0-4402-a15c-830a8b78aa20-kube-api-access-tcvj4\") on node \"crc\" DevicePath \"\"" Dec 03 13:01:52 crc kubenswrapper[5052]: I1203 13:01:52.876199 5052 reconciler_common.go:293] "Volume detached for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/37fe7283-01d0-4402-a15c-830a8b78aa20-ssh-key\") on node \"crc\" DevicePath \"\"" Dec 03 13:01:52 crc kubenswrapper[5052]: I1203 13:01:52.876209 5052 reconciler_common.go:293] "Volume detached for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/37fe7283-01d0-4402-a15c-830a8b78aa20-inventory\") on node \"crc\" DevicePath \"\"" Dec 03 13:01:52 crc kubenswrapper[5052]: I1203 13:01:52.977997 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-fgbjs\" (UniqueName: \"kubernetes.io/projected/0a9ff100-5a4a-4ba1-aba6-44c45c064fb1-kube-api-access-fgbjs\") pod \"configure-os-edpm-compute-no-nodes-edpm-compute-no-nodes-l6qqd\" (UID: \"0a9ff100-5a4a-4ba1-aba6-44c45c064fb1\") " pod="openstack-kuttl-tests/configure-os-edpm-compute-no-nodes-edpm-compute-no-nodes-l6qqd" Dec 03 13:01:52 crc kubenswrapper[5052]: I1203 13:01:52.978292 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/0a9ff100-5a4a-4ba1-aba6-44c45c064fb1-ssh-key\") pod \"configure-os-edpm-compute-no-nodes-edpm-compute-no-nodes-l6qqd\" (UID: \"0a9ff100-5a4a-4ba1-aba6-44c45c064fb1\") " pod="openstack-kuttl-tests/configure-os-edpm-compute-no-nodes-edpm-compute-no-nodes-l6qqd" Dec 03 13:01:52 crc kubenswrapper[5052]: I1203 13:01:52.978419 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/0a9ff100-5a4a-4ba1-aba6-44c45c064fb1-inventory\") pod \"configure-os-edpm-compute-no-nodes-edpm-compute-no-nodes-l6qqd\" (UID: \"0a9ff100-5a4a-4ba1-aba6-44c45c064fb1\") " pod="openstack-kuttl-tests/configure-os-edpm-compute-no-nodes-edpm-compute-no-nodes-l6qqd" Dec 03 13:01:52 crc kubenswrapper[5052]: I1203 13:01:52.981730 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/0a9ff100-5a4a-4ba1-aba6-44c45c064fb1-inventory\") pod \"configure-os-edpm-compute-no-nodes-edpm-compute-no-nodes-l6qqd\" (UID: \"0a9ff100-5a4a-4ba1-aba6-44c45c064fb1\") " pod="openstack-kuttl-tests/configure-os-edpm-compute-no-nodes-edpm-compute-no-nodes-l6qqd" Dec 03 13:01:52 crc kubenswrapper[5052]: I1203 13:01:52.982386 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/0a9ff100-5a4a-4ba1-aba6-44c45c064fb1-ssh-key\") pod \"configure-os-edpm-compute-no-nodes-edpm-compute-no-nodes-l6qqd\" (UID: \"0a9ff100-5a4a-4ba1-aba6-44c45c064fb1\") " pod="openstack-kuttl-tests/configure-os-edpm-compute-no-nodes-edpm-compute-no-nodes-l6qqd" Dec 03 13:01:52 crc kubenswrapper[5052]: I1203 13:01:52.994137 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-fgbjs\" (UniqueName: \"kubernetes.io/projected/0a9ff100-5a4a-4ba1-aba6-44c45c064fb1-kube-api-access-fgbjs\") pod \"configure-os-edpm-compute-no-nodes-edpm-compute-no-nodes-l6qqd\" (UID: \"0a9ff100-5a4a-4ba1-aba6-44c45c064fb1\") " pod="openstack-kuttl-tests/configure-os-edpm-compute-no-nodes-edpm-compute-no-nodes-l6qqd" Dec 03 13:01:53 crc kubenswrapper[5052]: I1203 13:01:53.095563 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/configure-os-edpm-compute-no-nodes-edpm-compute-no-nodes-l6qqd" Dec 03 13:01:53 crc kubenswrapper[5052]: I1203 13:01:53.430759 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/install-os-edpm-compute-no-nodes-edpm-compute-no-nodes-l4h9g" event={"ID":"37fe7283-01d0-4402-a15c-830a8b78aa20","Type":"ContainerDied","Data":"17d7dac3448b15a2431d48884abc701151e1a739d047b533d158f3543acc18f1"} Dec 03 13:01:53 crc kubenswrapper[5052]: I1203 13:01:53.430806 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="17d7dac3448b15a2431d48884abc701151e1a739d047b533d158f3543acc18f1" Dec 03 13:01:53 crc kubenswrapper[5052]: I1203 13:01:53.430810 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/install-os-edpm-compute-no-nodes-edpm-compute-no-nodes-l4h9g" Dec 03 13:01:53 crc kubenswrapper[5052]: I1203 13:01:53.519068 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/configure-os-edpm-compute-no-nodes-edpm-compute-no-nodes-l6qqd"] Dec 03 13:01:53 crc kubenswrapper[5052]: W1203 13:01:53.525074 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0a9ff100_5a4a_4ba1_aba6_44c45c064fb1.slice/crio-c19a8df56ae2f7e8e2d7831f8be03c1126697cab2185cffb578ac907cb17a0b1 WatchSource:0}: Error finding container c19a8df56ae2f7e8e2d7831f8be03c1126697cab2185cffb578ac907cb17a0b1: Status 404 returned error can't find the container with id c19a8df56ae2f7e8e2d7831f8be03c1126697cab2185cffb578ac907cb17a0b1 Dec 03 13:01:54 crc kubenswrapper[5052]: I1203 13:01:54.450552 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/configure-os-edpm-compute-no-nodes-edpm-compute-no-nodes-l6qqd" event={"ID":"0a9ff100-5a4a-4ba1-aba6-44c45c064fb1","Type":"ContainerStarted","Data":"30828179aa61724ce0355188958eaf87142cd62b593ab9385d1292b37635d56f"} Dec 03 13:01:54 crc kubenswrapper[5052]: I1203 13:01:54.451168 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/configure-os-edpm-compute-no-nodes-edpm-compute-no-nodes-l6qqd" event={"ID":"0a9ff100-5a4a-4ba1-aba6-44c45c064fb1","Type":"ContainerStarted","Data":"c19a8df56ae2f7e8e2d7831f8be03c1126697cab2185cffb578ac907cb17a0b1"} Dec 03 13:01:54 crc kubenswrapper[5052]: I1203 13:01:54.470880 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/configure-os-edpm-compute-no-nodes-edpm-compute-no-nodes-l6qqd" podStartSLOduration=2.005679127 podStartE2EDuration="2.470858021s" podCreationTimestamp="2025-12-03 13:01:52 +0000 UTC" firstStartedPulling="2025-12-03 13:01:53.527880413 +0000 UTC m=+6104.116122177" lastFinishedPulling="2025-12-03 13:01:53.993059287 +0000 UTC m=+6104.581301071" observedRunningTime="2025-12-03 13:01:54.465635952 +0000 UTC m=+6105.053877716" watchObservedRunningTime="2025-12-03 13:01:54.470858021 +0000 UTC m=+6105.059099785" Dec 03 13:01:56 crc kubenswrapper[5052]: I1203 13:01:56.467413 5052 generic.go:334] "Generic (PLEG): container finished" podID="0a9ff100-5a4a-4ba1-aba6-44c45c064fb1" containerID="30828179aa61724ce0355188958eaf87142cd62b593ab9385d1292b37635d56f" exitCode=0 Dec 03 13:01:56 crc kubenswrapper[5052]: I1203 13:01:56.467498 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/configure-os-edpm-compute-no-nodes-edpm-compute-no-nodes-l6qqd" event={"ID":"0a9ff100-5a4a-4ba1-aba6-44c45c064fb1","Type":"ContainerDied","Data":"30828179aa61724ce0355188958eaf87142cd62b593ab9385d1292b37635d56f"} Dec 03 13:01:57 crc kubenswrapper[5052]: I1203 13:01:57.761941 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/configure-os-edpm-compute-no-nodes-edpm-compute-no-nodes-l6qqd" Dec 03 13:01:57 crc kubenswrapper[5052]: I1203 13:01:57.853912 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/0a9ff100-5a4a-4ba1-aba6-44c45c064fb1-ssh-key\") pod \"0a9ff100-5a4a-4ba1-aba6-44c45c064fb1\" (UID: \"0a9ff100-5a4a-4ba1-aba6-44c45c064fb1\") " Dec 03 13:01:57 crc kubenswrapper[5052]: I1203 13:01:57.853984 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fgbjs\" (UniqueName: \"kubernetes.io/projected/0a9ff100-5a4a-4ba1-aba6-44c45c064fb1-kube-api-access-fgbjs\") pod \"0a9ff100-5a4a-4ba1-aba6-44c45c064fb1\" (UID: \"0a9ff100-5a4a-4ba1-aba6-44c45c064fb1\") " Dec 03 13:01:57 crc kubenswrapper[5052]: I1203 13:01:57.854078 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/0a9ff100-5a4a-4ba1-aba6-44c45c064fb1-inventory\") pod \"0a9ff100-5a4a-4ba1-aba6-44c45c064fb1\" (UID: \"0a9ff100-5a4a-4ba1-aba6-44c45c064fb1\") " Dec 03 13:01:57 crc kubenswrapper[5052]: I1203 13:01:57.859833 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0a9ff100-5a4a-4ba1-aba6-44c45c064fb1-kube-api-access-fgbjs" (OuterVolumeSpecName: "kube-api-access-fgbjs") pod "0a9ff100-5a4a-4ba1-aba6-44c45c064fb1" (UID: "0a9ff100-5a4a-4ba1-aba6-44c45c064fb1"). InnerVolumeSpecName "kube-api-access-fgbjs". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:01:57 crc kubenswrapper[5052]: I1203 13:01:57.876388 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0a9ff100-5a4a-4ba1-aba6-44c45c064fb1-ssh-key" (OuterVolumeSpecName: "ssh-key") pod "0a9ff100-5a4a-4ba1-aba6-44c45c064fb1" (UID: "0a9ff100-5a4a-4ba1-aba6-44c45c064fb1"). InnerVolumeSpecName "ssh-key". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:01:57 crc kubenswrapper[5052]: I1203 13:01:57.876707 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0a9ff100-5a4a-4ba1-aba6-44c45c064fb1-inventory" (OuterVolumeSpecName: "inventory") pod "0a9ff100-5a4a-4ba1-aba6-44c45c064fb1" (UID: "0a9ff100-5a4a-4ba1-aba6-44c45c064fb1"). InnerVolumeSpecName "inventory". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:01:57 crc kubenswrapper[5052]: I1203 13:01:57.955868 5052 reconciler_common.go:293] "Volume detached for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/0a9ff100-5a4a-4ba1-aba6-44c45c064fb1-ssh-key\") on node \"crc\" DevicePath \"\"" Dec 03 13:01:57 crc kubenswrapper[5052]: I1203 13:01:57.955914 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-fgbjs\" (UniqueName: \"kubernetes.io/projected/0a9ff100-5a4a-4ba1-aba6-44c45c064fb1-kube-api-access-fgbjs\") on node \"crc\" DevicePath \"\"" Dec 03 13:01:57 crc kubenswrapper[5052]: I1203 13:01:57.955924 5052 reconciler_common.go:293] "Volume detached for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/0a9ff100-5a4a-4ba1-aba6-44c45c064fb1-inventory\") on node \"crc\" DevicePath \"\"" Dec 03 13:01:58 crc kubenswrapper[5052]: I1203 13:01:58.483923 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/configure-os-edpm-compute-no-nodes-edpm-compute-no-nodes-l6qqd" event={"ID":"0a9ff100-5a4a-4ba1-aba6-44c45c064fb1","Type":"ContainerDied","Data":"c19a8df56ae2f7e8e2d7831f8be03c1126697cab2185cffb578ac907cb17a0b1"} Dec 03 13:01:58 crc kubenswrapper[5052]: I1203 13:01:58.483971 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c19a8df56ae2f7e8e2d7831f8be03c1126697cab2185cffb578ac907cb17a0b1" Dec 03 13:01:58 crc kubenswrapper[5052]: I1203 13:01:58.484016 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/configure-os-edpm-compute-no-nodes-edpm-compute-no-nodes-l6qqd" Dec 03 13:01:58 crc kubenswrapper[5052]: I1203 13:01:58.551637 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/run-os-edpm-compute-no-nodes-edpm-compute-no-nodes-d8kqk"] Dec 03 13:01:58 crc kubenswrapper[5052]: E1203 13:01:58.552008 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="0a9ff100-5a4a-4ba1-aba6-44c45c064fb1" containerName="configure-os-edpm-compute-no-nodes-edpm-compute-no-nodes" Dec 03 13:01:58 crc kubenswrapper[5052]: I1203 13:01:58.552033 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a9ff100-5a4a-4ba1-aba6-44c45c064fb1" containerName="configure-os-edpm-compute-no-nodes-edpm-compute-no-nodes" Dec 03 13:01:58 crc kubenswrapper[5052]: I1203 13:01:58.552196 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="0a9ff100-5a4a-4ba1-aba6-44c45c064fb1" containerName="configure-os-edpm-compute-no-nodes-edpm-compute-no-nodes" Dec 03 13:01:58 crc kubenswrapper[5052]: I1203 13:01:58.552681 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/run-os-edpm-compute-no-nodes-edpm-compute-no-nodes-d8kqk" Dec 03 13:01:58 crc kubenswrapper[5052]: I1203 13:01:58.555990 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"openstack-aee-default-env" Dec 03 13:01:58 crc kubenswrapper[5052]: I1203 13:01:58.556197 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"dataplane-ansible-ssh-private-key-secret" Dec 03 13:01:58 crc kubenswrapper[5052]: I1203 13:01:58.556431 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"dataplanenodeset-edpm-compute-no-nodes" Dec 03 13:01:58 crc kubenswrapper[5052]: I1203 13:01:58.556636 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"edpm-compute-no-nodes-dockercfg-mkcgv" Dec 03 13:01:58 crc kubenswrapper[5052]: I1203 13:01:58.564085 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/run-os-edpm-compute-no-nodes-edpm-compute-no-nodes-d8kqk"] Dec 03 13:01:58 crc kubenswrapper[5052]: I1203 13:01:58.665620 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mkfnj\" (UniqueName: \"kubernetes.io/projected/83226f18-0a8d-4fc6-8d02-32056e5fb8ae-kube-api-access-mkfnj\") pod \"run-os-edpm-compute-no-nodes-edpm-compute-no-nodes-d8kqk\" (UID: \"83226f18-0a8d-4fc6-8d02-32056e5fb8ae\") " pod="openstack-kuttl-tests/run-os-edpm-compute-no-nodes-edpm-compute-no-nodes-d8kqk" Dec 03 13:01:58 crc kubenswrapper[5052]: I1203 13:01:58.665673 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/83226f18-0a8d-4fc6-8d02-32056e5fb8ae-ssh-key\") pod \"run-os-edpm-compute-no-nodes-edpm-compute-no-nodes-d8kqk\" (UID: \"83226f18-0a8d-4fc6-8d02-32056e5fb8ae\") " pod="openstack-kuttl-tests/run-os-edpm-compute-no-nodes-edpm-compute-no-nodes-d8kqk" Dec 03 13:01:58 crc kubenswrapper[5052]: I1203 13:01:58.665712 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/83226f18-0a8d-4fc6-8d02-32056e5fb8ae-inventory\") pod \"run-os-edpm-compute-no-nodes-edpm-compute-no-nodes-d8kqk\" (UID: \"83226f18-0a8d-4fc6-8d02-32056e5fb8ae\") " pod="openstack-kuttl-tests/run-os-edpm-compute-no-nodes-edpm-compute-no-nodes-d8kqk" Dec 03 13:01:58 crc kubenswrapper[5052]: I1203 13:01:58.768099 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-mkfnj\" (UniqueName: \"kubernetes.io/projected/83226f18-0a8d-4fc6-8d02-32056e5fb8ae-kube-api-access-mkfnj\") pod \"run-os-edpm-compute-no-nodes-edpm-compute-no-nodes-d8kqk\" (UID: \"83226f18-0a8d-4fc6-8d02-32056e5fb8ae\") " pod="openstack-kuttl-tests/run-os-edpm-compute-no-nodes-edpm-compute-no-nodes-d8kqk" Dec 03 13:01:58 crc kubenswrapper[5052]: I1203 13:01:58.768196 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/83226f18-0a8d-4fc6-8d02-32056e5fb8ae-ssh-key\") pod \"run-os-edpm-compute-no-nodes-edpm-compute-no-nodes-d8kqk\" (UID: \"83226f18-0a8d-4fc6-8d02-32056e5fb8ae\") " pod="openstack-kuttl-tests/run-os-edpm-compute-no-nodes-edpm-compute-no-nodes-d8kqk" Dec 03 13:01:58 crc kubenswrapper[5052]: I1203 13:01:58.768279 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/83226f18-0a8d-4fc6-8d02-32056e5fb8ae-inventory\") pod \"run-os-edpm-compute-no-nodes-edpm-compute-no-nodes-d8kqk\" (UID: \"83226f18-0a8d-4fc6-8d02-32056e5fb8ae\") " pod="openstack-kuttl-tests/run-os-edpm-compute-no-nodes-edpm-compute-no-nodes-d8kqk" Dec 03 13:01:58 crc kubenswrapper[5052]: I1203 13:01:58.773678 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/83226f18-0a8d-4fc6-8d02-32056e5fb8ae-ssh-key\") pod \"run-os-edpm-compute-no-nodes-edpm-compute-no-nodes-d8kqk\" (UID: \"83226f18-0a8d-4fc6-8d02-32056e5fb8ae\") " pod="openstack-kuttl-tests/run-os-edpm-compute-no-nodes-edpm-compute-no-nodes-d8kqk" Dec 03 13:01:58 crc kubenswrapper[5052]: I1203 13:01:58.774133 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/83226f18-0a8d-4fc6-8d02-32056e5fb8ae-inventory\") pod \"run-os-edpm-compute-no-nodes-edpm-compute-no-nodes-d8kqk\" (UID: \"83226f18-0a8d-4fc6-8d02-32056e5fb8ae\") " pod="openstack-kuttl-tests/run-os-edpm-compute-no-nodes-edpm-compute-no-nodes-d8kqk" Dec 03 13:01:58 crc kubenswrapper[5052]: I1203 13:01:58.787392 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-mkfnj\" (UniqueName: \"kubernetes.io/projected/83226f18-0a8d-4fc6-8d02-32056e5fb8ae-kube-api-access-mkfnj\") pod \"run-os-edpm-compute-no-nodes-edpm-compute-no-nodes-d8kqk\" (UID: \"83226f18-0a8d-4fc6-8d02-32056e5fb8ae\") " pod="openstack-kuttl-tests/run-os-edpm-compute-no-nodes-edpm-compute-no-nodes-d8kqk" Dec 03 13:01:58 crc kubenswrapper[5052]: I1203 13:01:58.875351 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/run-os-edpm-compute-no-nodes-edpm-compute-no-nodes-d8kqk" Dec 03 13:01:59 crc kubenswrapper[5052]: I1203 13:01:59.291867 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/run-os-edpm-compute-no-nodes-edpm-compute-no-nodes-d8kqk"] Dec 03 13:01:59 crc kubenswrapper[5052]: I1203 13:01:59.493292 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/run-os-edpm-compute-no-nodes-edpm-compute-no-nodes-d8kqk" event={"ID":"83226f18-0a8d-4fc6-8d02-32056e5fb8ae","Type":"ContainerStarted","Data":"e63d227803d9318e35f75b5c947d10f44fb8a2ec560cd4e139ae52640e06f2e6"} Dec 03 13:02:00 crc kubenswrapper[5052]: I1203 13:02:00.504690 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/run-os-edpm-compute-no-nodes-edpm-compute-no-nodes-d8kqk" event={"ID":"83226f18-0a8d-4fc6-8d02-32056e5fb8ae","Type":"ContainerStarted","Data":"0ed1179c896e0ce5137ca77e2d09904242ece4c5dbb5564773a0003d6228bcd4"} Dec 03 13:02:00 crc kubenswrapper[5052]: I1203 13:02:00.528357 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/run-os-edpm-compute-no-nodes-edpm-compute-no-nodes-d8kqk" podStartSLOduration=2.013545472 podStartE2EDuration="2.528335044s" podCreationTimestamp="2025-12-03 13:01:58 +0000 UTC" firstStartedPulling="2025-12-03 13:01:59.298676548 +0000 UTC m=+6109.886918312" lastFinishedPulling="2025-12-03 13:01:59.81346612 +0000 UTC m=+6110.401707884" observedRunningTime="2025-12-03 13:02:00.518898154 +0000 UTC m=+6111.107139918" watchObservedRunningTime="2025-12-03 13:02:00.528335044 +0000 UTC m=+6111.116576808" Dec 03 13:02:01 crc kubenswrapper[5052]: I1203 13:02:01.515066 5052 generic.go:334] "Generic (PLEG): container finished" podID="83226f18-0a8d-4fc6-8d02-32056e5fb8ae" containerID="0ed1179c896e0ce5137ca77e2d09904242ece4c5dbb5564773a0003d6228bcd4" exitCode=0 Dec 03 13:02:01 crc kubenswrapper[5052]: I1203 13:02:01.515382 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/run-os-edpm-compute-no-nodes-edpm-compute-no-nodes-d8kqk" event={"ID":"83226f18-0a8d-4fc6-8d02-32056e5fb8ae","Type":"ContainerDied","Data":"0ed1179c896e0ce5137ca77e2d09904242ece4c5dbb5564773a0003d6228bcd4"} Dec 03 13:02:02 crc kubenswrapper[5052]: I1203 13:02:02.806283 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/run-os-edpm-compute-no-nodes-edpm-compute-no-nodes-d8kqk" Dec 03 13:02:02 crc kubenswrapper[5052]: I1203 13:02:02.837911 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/83226f18-0a8d-4fc6-8d02-32056e5fb8ae-inventory\") pod \"83226f18-0a8d-4fc6-8d02-32056e5fb8ae\" (UID: \"83226f18-0a8d-4fc6-8d02-32056e5fb8ae\") " Dec 03 13:02:02 crc kubenswrapper[5052]: I1203 13:02:02.838052 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/83226f18-0a8d-4fc6-8d02-32056e5fb8ae-ssh-key\") pod \"83226f18-0a8d-4fc6-8d02-32056e5fb8ae\" (UID: \"83226f18-0a8d-4fc6-8d02-32056e5fb8ae\") " Dec 03 13:02:02 crc kubenswrapper[5052]: I1203 13:02:02.838120 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mkfnj\" (UniqueName: \"kubernetes.io/projected/83226f18-0a8d-4fc6-8d02-32056e5fb8ae-kube-api-access-mkfnj\") pod \"83226f18-0a8d-4fc6-8d02-32056e5fb8ae\" (UID: \"83226f18-0a8d-4fc6-8d02-32056e5fb8ae\") " Dec 03 13:02:02 crc kubenswrapper[5052]: I1203 13:02:02.844083 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/83226f18-0a8d-4fc6-8d02-32056e5fb8ae-kube-api-access-mkfnj" (OuterVolumeSpecName: "kube-api-access-mkfnj") pod "83226f18-0a8d-4fc6-8d02-32056e5fb8ae" (UID: "83226f18-0a8d-4fc6-8d02-32056e5fb8ae"). InnerVolumeSpecName "kube-api-access-mkfnj". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:02:02 crc kubenswrapper[5052]: I1203 13:02:02.862295 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/83226f18-0a8d-4fc6-8d02-32056e5fb8ae-inventory" (OuterVolumeSpecName: "inventory") pod "83226f18-0a8d-4fc6-8d02-32056e5fb8ae" (UID: "83226f18-0a8d-4fc6-8d02-32056e5fb8ae"). InnerVolumeSpecName "inventory". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:02:02 crc kubenswrapper[5052]: I1203 13:02:02.862706 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/83226f18-0a8d-4fc6-8d02-32056e5fb8ae-ssh-key" (OuterVolumeSpecName: "ssh-key") pod "83226f18-0a8d-4fc6-8d02-32056e5fb8ae" (UID: "83226f18-0a8d-4fc6-8d02-32056e5fb8ae"). InnerVolumeSpecName "ssh-key". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:02:02 crc kubenswrapper[5052]: I1203 13:02:02.939902 5052 reconciler_common.go:293] "Volume detached for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/83226f18-0a8d-4fc6-8d02-32056e5fb8ae-inventory\") on node \"crc\" DevicePath \"\"" Dec 03 13:02:02 crc kubenswrapper[5052]: I1203 13:02:02.939959 5052 reconciler_common.go:293] "Volume detached for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/83226f18-0a8d-4fc6-8d02-32056e5fb8ae-ssh-key\") on node \"crc\" DevicePath \"\"" Dec 03 13:02:02 crc kubenswrapper[5052]: I1203 13:02:02.939975 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-mkfnj\" (UniqueName: \"kubernetes.io/projected/83226f18-0a8d-4fc6-8d02-32056e5fb8ae-kube-api-access-mkfnj\") on node \"crc\" DevicePath \"\"" Dec 03 13:02:03 crc kubenswrapper[5052]: I1203 13:02:03.534267 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/run-os-edpm-compute-no-nodes-edpm-compute-no-nodes-d8kqk" event={"ID":"83226f18-0a8d-4fc6-8d02-32056e5fb8ae","Type":"ContainerDied","Data":"e63d227803d9318e35f75b5c947d10f44fb8a2ec560cd4e139ae52640e06f2e6"} Dec 03 13:02:03 crc kubenswrapper[5052]: I1203 13:02:03.534308 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e63d227803d9318e35f75b5c947d10f44fb8a2ec560cd4e139ae52640e06f2e6" Dec 03 13:02:03 crc kubenswrapper[5052]: I1203 13:02:03.534357 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/run-os-edpm-compute-no-nodes-edpm-compute-no-nodes-d8kqk" Dec 03 13:02:03 crc kubenswrapper[5052]: I1203 13:02:03.887324 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/install-certs-edpm-compute-no-nodes-edpm-compute-no-nodes-45bwg"] Dec 03 13:02:03 crc kubenswrapper[5052]: E1203 13:02:03.887657 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="83226f18-0a8d-4fc6-8d02-32056e5fb8ae" containerName="run-os-edpm-compute-no-nodes-edpm-compute-no-nodes" Dec 03 13:02:03 crc kubenswrapper[5052]: I1203 13:02:03.887672 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="83226f18-0a8d-4fc6-8d02-32056e5fb8ae" containerName="run-os-edpm-compute-no-nodes-edpm-compute-no-nodes" Dec 03 13:02:03 crc kubenswrapper[5052]: I1203 13:02:03.887822 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="83226f18-0a8d-4fc6-8d02-32056e5fb8ae" containerName="run-os-edpm-compute-no-nodes-edpm-compute-no-nodes" Dec 03 13:02:03 crc kubenswrapper[5052]: I1203 13:02:03.888293 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/install-certs-edpm-compute-no-nodes-edpm-compute-no-nodes-45bwg" Dec 03 13:02:03 crc kubenswrapper[5052]: I1203 13:02:03.890478 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"combined-ca-bundle" Dec 03 13:02:03 crc kubenswrapper[5052]: I1203 13:02:03.890926 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"openstack-aee-default-env" Dec 03 13:02:03 crc kubenswrapper[5052]: I1203 13:02:03.891086 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"dataplanenodeset-edpm-compute-no-nodes" Dec 03 13:02:03 crc kubenswrapper[5052]: I1203 13:02:03.891341 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"edpm-compute-no-nodes-dockercfg-mkcgv" Dec 03 13:02:03 crc kubenswrapper[5052]: I1203 13:02:03.891479 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"dataplane-ansible-ssh-private-key-secret" Dec 03 13:02:03 crc kubenswrapper[5052]: I1203 13:02:03.947802 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/install-certs-edpm-compute-no-nodes-edpm-compute-no-nodes-45bwg"] Dec 03 13:02:03 crc kubenswrapper[5052]: I1203 13:02:03.958882 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"neutron-sriov-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/58aa9910-8f11-4e46-abd7-816fb0439dc5-neutron-sriov-combined-ca-bundle\") pod \"install-certs-edpm-compute-no-nodes-edpm-compute-no-nodes-45bwg\" (UID: \"58aa9910-8f11-4e46-abd7-816fb0439dc5\") " pod="openstack-kuttl-tests/install-certs-edpm-compute-no-nodes-edpm-compute-no-nodes-45bwg" Dec 03 13:02:03 crc kubenswrapper[5052]: I1203 13:02:03.958977 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"neutron-dhcp-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/58aa9910-8f11-4e46-abd7-816fb0439dc5-neutron-dhcp-combined-ca-bundle\") pod \"install-certs-edpm-compute-no-nodes-edpm-compute-no-nodes-45bwg\" (UID: \"58aa9910-8f11-4e46-abd7-816fb0439dc5\") " pod="openstack-kuttl-tests/install-certs-edpm-compute-no-nodes-edpm-compute-no-nodes-45bwg" Dec 03 13:02:03 crc kubenswrapper[5052]: I1203 13:02:03.959007 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/58aa9910-8f11-4e46-abd7-816fb0439dc5-inventory\") pod \"install-certs-edpm-compute-no-nodes-edpm-compute-no-nodes-45bwg\" (UID: \"58aa9910-8f11-4e46-abd7-816fb0439dc5\") " pod="openstack-kuttl-tests/install-certs-edpm-compute-no-nodes-edpm-compute-no-nodes-45bwg" Dec 03 13:02:03 crc kubenswrapper[5052]: I1203 13:02:03.959056 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"neutron-ovn-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/58aa9910-8f11-4e46-abd7-816fb0439dc5-neutron-ovn-combined-ca-bundle\") pod \"install-certs-edpm-compute-no-nodes-edpm-compute-no-nodes-45bwg\" (UID: \"58aa9910-8f11-4e46-abd7-816fb0439dc5\") " pod="openstack-kuttl-tests/install-certs-edpm-compute-no-nodes-edpm-compute-no-nodes-45bwg" Dec 03 13:02:03 crc kubenswrapper[5052]: I1203 13:02:03.959134 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nova-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/58aa9910-8f11-4e46-abd7-816fb0439dc5-nova-combined-ca-bundle\") pod \"install-certs-edpm-compute-no-nodes-edpm-compute-no-nodes-45bwg\" (UID: \"58aa9910-8f11-4e46-abd7-816fb0439dc5\") " pod="openstack-kuttl-tests/install-certs-edpm-compute-no-nodes-edpm-compute-no-nodes-45bwg" Dec 03 13:02:03 crc kubenswrapper[5052]: I1203 13:02:03.959166 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/58aa9910-8f11-4e46-abd7-816fb0439dc5-ssh-key\") pod \"install-certs-edpm-compute-no-nodes-edpm-compute-no-nodes-45bwg\" (UID: \"58aa9910-8f11-4e46-abd7-816fb0439dc5\") " pod="openstack-kuttl-tests/install-certs-edpm-compute-no-nodes-edpm-compute-no-nodes-45bwg" Dec 03 13:02:03 crc kubenswrapper[5052]: I1203 13:02:03.959221 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"neutron-metadata-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/58aa9910-8f11-4e46-abd7-816fb0439dc5-neutron-metadata-combined-ca-bundle\") pod \"install-certs-edpm-compute-no-nodes-edpm-compute-no-nodes-45bwg\" (UID: \"58aa9910-8f11-4e46-abd7-816fb0439dc5\") " pod="openstack-kuttl-tests/install-certs-edpm-compute-no-nodes-edpm-compute-no-nodes-45bwg" Dec 03 13:02:03 crc kubenswrapper[5052]: I1203 13:02:03.959302 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"libvirt-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/58aa9910-8f11-4e46-abd7-816fb0439dc5-libvirt-combined-ca-bundle\") pod \"install-certs-edpm-compute-no-nodes-edpm-compute-no-nodes-45bwg\" (UID: \"58aa9910-8f11-4e46-abd7-816fb0439dc5\") " pod="openstack-kuttl-tests/install-certs-edpm-compute-no-nodes-edpm-compute-no-nodes-45bwg" Dec 03 13:02:03 crc kubenswrapper[5052]: I1203 13:02:03.959326 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/58aa9910-8f11-4e46-abd7-816fb0439dc5-ovn-combined-ca-bundle\") pod \"install-certs-edpm-compute-no-nodes-edpm-compute-no-nodes-45bwg\" (UID: \"58aa9910-8f11-4e46-abd7-816fb0439dc5\") " pod="openstack-kuttl-tests/install-certs-edpm-compute-no-nodes-edpm-compute-no-nodes-45bwg" Dec 03 13:02:03 crc kubenswrapper[5052]: I1203 13:02:03.959349 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bootstrap-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/58aa9910-8f11-4e46-abd7-816fb0439dc5-bootstrap-combined-ca-bundle\") pod \"install-certs-edpm-compute-no-nodes-edpm-compute-no-nodes-45bwg\" (UID: \"58aa9910-8f11-4e46-abd7-816fb0439dc5\") " pod="openstack-kuttl-tests/install-certs-edpm-compute-no-nodes-edpm-compute-no-nodes-45bwg" Dec 03 13:02:03 crc kubenswrapper[5052]: I1203 13:02:03.959416 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2qnp8\" (UniqueName: \"kubernetes.io/projected/58aa9910-8f11-4e46-abd7-816fb0439dc5-kube-api-access-2qnp8\") pod \"install-certs-edpm-compute-no-nodes-edpm-compute-no-nodes-45bwg\" (UID: \"58aa9910-8f11-4e46-abd7-816fb0439dc5\") " pod="openstack-kuttl-tests/install-certs-edpm-compute-no-nodes-edpm-compute-no-nodes-45bwg" Dec 03 13:02:04 crc kubenswrapper[5052]: I1203 13:02:04.061192 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"libvirt-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/58aa9910-8f11-4e46-abd7-816fb0439dc5-libvirt-combined-ca-bundle\") pod \"install-certs-edpm-compute-no-nodes-edpm-compute-no-nodes-45bwg\" (UID: \"58aa9910-8f11-4e46-abd7-816fb0439dc5\") " pod="openstack-kuttl-tests/install-certs-edpm-compute-no-nodes-edpm-compute-no-nodes-45bwg" Dec 03 13:02:04 crc kubenswrapper[5052]: I1203 13:02:04.061884 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovn-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/58aa9910-8f11-4e46-abd7-816fb0439dc5-ovn-combined-ca-bundle\") pod \"install-certs-edpm-compute-no-nodes-edpm-compute-no-nodes-45bwg\" (UID: \"58aa9910-8f11-4e46-abd7-816fb0439dc5\") " pod="openstack-kuttl-tests/install-certs-edpm-compute-no-nodes-edpm-compute-no-nodes-45bwg" Dec 03 13:02:04 crc kubenswrapper[5052]: I1203 13:02:04.061906 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"bootstrap-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/58aa9910-8f11-4e46-abd7-816fb0439dc5-bootstrap-combined-ca-bundle\") pod \"install-certs-edpm-compute-no-nodes-edpm-compute-no-nodes-45bwg\" (UID: \"58aa9910-8f11-4e46-abd7-816fb0439dc5\") " pod="openstack-kuttl-tests/install-certs-edpm-compute-no-nodes-edpm-compute-no-nodes-45bwg" Dec 03 13:02:04 crc kubenswrapper[5052]: I1203 13:02:04.061949 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-2qnp8\" (UniqueName: \"kubernetes.io/projected/58aa9910-8f11-4e46-abd7-816fb0439dc5-kube-api-access-2qnp8\") pod \"install-certs-edpm-compute-no-nodes-edpm-compute-no-nodes-45bwg\" (UID: \"58aa9910-8f11-4e46-abd7-816fb0439dc5\") " pod="openstack-kuttl-tests/install-certs-edpm-compute-no-nodes-edpm-compute-no-nodes-45bwg" Dec 03 13:02:04 crc kubenswrapper[5052]: I1203 13:02:04.061976 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"neutron-sriov-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/58aa9910-8f11-4e46-abd7-816fb0439dc5-neutron-sriov-combined-ca-bundle\") pod \"install-certs-edpm-compute-no-nodes-edpm-compute-no-nodes-45bwg\" (UID: \"58aa9910-8f11-4e46-abd7-816fb0439dc5\") " pod="openstack-kuttl-tests/install-certs-edpm-compute-no-nodes-edpm-compute-no-nodes-45bwg" Dec 03 13:02:04 crc kubenswrapper[5052]: I1203 13:02:04.062018 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"neutron-dhcp-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/58aa9910-8f11-4e46-abd7-816fb0439dc5-neutron-dhcp-combined-ca-bundle\") pod \"install-certs-edpm-compute-no-nodes-edpm-compute-no-nodes-45bwg\" (UID: \"58aa9910-8f11-4e46-abd7-816fb0439dc5\") " pod="openstack-kuttl-tests/install-certs-edpm-compute-no-nodes-edpm-compute-no-nodes-45bwg" Dec 03 13:02:04 crc kubenswrapper[5052]: I1203 13:02:04.062037 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/58aa9910-8f11-4e46-abd7-816fb0439dc5-inventory\") pod \"install-certs-edpm-compute-no-nodes-edpm-compute-no-nodes-45bwg\" (UID: \"58aa9910-8f11-4e46-abd7-816fb0439dc5\") " pod="openstack-kuttl-tests/install-certs-edpm-compute-no-nodes-edpm-compute-no-nodes-45bwg" Dec 03 13:02:04 crc kubenswrapper[5052]: I1203 13:02:04.062065 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"neutron-ovn-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/58aa9910-8f11-4e46-abd7-816fb0439dc5-neutron-ovn-combined-ca-bundle\") pod \"install-certs-edpm-compute-no-nodes-edpm-compute-no-nodes-45bwg\" (UID: \"58aa9910-8f11-4e46-abd7-816fb0439dc5\") " pod="openstack-kuttl-tests/install-certs-edpm-compute-no-nodes-edpm-compute-no-nodes-45bwg" Dec 03 13:02:04 crc kubenswrapper[5052]: I1203 13:02:04.062109 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"nova-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/58aa9910-8f11-4e46-abd7-816fb0439dc5-nova-combined-ca-bundle\") pod \"install-certs-edpm-compute-no-nodes-edpm-compute-no-nodes-45bwg\" (UID: \"58aa9910-8f11-4e46-abd7-816fb0439dc5\") " pod="openstack-kuttl-tests/install-certs-edpm-compute-no-nodes-edpm-compute-no-nodes-45bwg" Dec 03 13:02:04 crc kubenswrapper[5052]: I1203 13:02:04.062130 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/58aa9910-8f11-4e46-abd7-816fb0439dc5-ssh-key\") pod \"install-certs-edpm-compute-no-nodes-edpm-compute-no-nodes-45bwg\" (UID: \"58aa9910-8f11-4e46-abd7-816fb0439dc5\") " pod="openstack-kuttl-tests/install-certs-edpm-compute-no-nodes-edpm-compute-no-nodes-45bwg" Dec 03 13:02:04 crc kubenswrapper[5052]: I1203 13:02:04.062158 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"neutron-metadata-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/58aa9910-8f11-4e46-abd7-816fb0439dc5-neutron-metadata-combined-ca-bundle\") pod \"install-certs-edpm-compute-no-nodes-edpm-compute-no-nodes-45bwg\" (UID: \"58aa9910-8f11-4e46-abd7-816fb0439dc5\") " pod="openstack-kuttl-tests/install-certs-edpm-compute-no-nodes-edpm-compute-no-nodes-45bwg" Dec 03 13:02:04 crc kubenswrapper[5052]: I1203 13:02:04.070483 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/58aa9910-8f11-4e46-abd7-816fb0439dc5-inventory\") pod \"install-certs-edpm-compute-no-nodes-edpm-compute-no-nodes-45bwg\" (UID: \"58aa9910-8f11-4e46-abd7-816fb0439dc5\") " pod="openstack-kuttl-tests/install-certs-edpm-compute-no-nodes-edpm-compute-no-nodes-45bwg" Dec 03 13:02:04 crc kubenswrapper[5052]: I1203 13:02:04.076183 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"libvirt-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/58aa9910-8f11-4e46-abd7-816fb0439dc5-libvirt-combined-ca-bundle\") pod \"install-certs-edpm-compute-no-nodes-edpm-compute-no-nodes-45bwg\" (UID: \"58aa9910-8f11-4e46-abd7-816fb0439dc5\") " pod="openstack-kuttl-tests/install-certs-edpm-compute-no-nodes-edpm-compute-no-nodes-45bwg" Dec 03 13:02:04 crc kubenswrapper[5052]: I1203 13:02:04.076759 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"bootstrap-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/58aa9910-8f11-4e46-abd7-816fb0439dc5-bootstrap-combined-ca-bundle\") pod \"install-certs-edpm-compute-no-nodes-edpm-compute-no-nodes-45bwg\" (UID: \"58aa9910-8f11-4e46-abd7-816fb0439dc5\") " pod="openstack-kuttl-tests/install-certs-edpm-compute-no-nodes-edpm-compute-no-nodes-45bwg" Dec 03 13:02:04 crc kubenswrapper[5052]: I1203 13:02:04.085350 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"neutron-sriov-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/58aa9910-8f11-4e46-abd7-816fb0439dc5-neutron-sriov-combined-ca-bundle\") pod \"install-certs-edpm-compute-no-nodes-edpm-compute-no-nodes-45bwg\" (UID: \"58aa9910-8f11-4e46-abd7-816fb0439dc5\") " pod="openstack-kuttl-tests/install-certs-edpm-compute-no-nodes-edpm-compute-no-nodes-45bwg" Dec 03 13:02:04 crc kubenswrapper[5052]: I1203 13:02:04.086223 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"neutron-metadata-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/58aa9910-8f11-4e46-abd7-816fb0439dc5-neutron-metadata-combined-ca-bundle\") pod \"install-certs-edpm-compute-no-nodes-edpm-compute-no-nodes-45bwg\" (UID: \"58aa9910-8f11-4e46-abd7-816fb0439dc5\") " pod="openstack-kuttl-tests/install-certs-edpm-compute-no-nodes-edpm-compute-no-nodes-45bwg" Dec 03 13:02:04 crc kubenswrapper[5052]: I1203 13:02:04.086972 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovn-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/58aa9910-8f11-4e46-abd7-816fb0439dc5-ovn-combined-ca-bundle\") pod \"install-certs-edpm-compute-no-nodes-edpm-compute-no-nodes-45bwg\" (UID: \"58aa9910-8f11-4e46-abd7-816fb0439dc5\") " pod="openstack-kuttl-tests/install-certs-edpm-compute-no-nodes-edpm-compute-no-nodes-45bwg" Dec 03 13:02:04 crc kubenswrapper[5052]: I1203 13:02:04.086982 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"nova-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/58aa9910-8f11-4e46-abd7-816fb0439dc5-nova-combined-ca-bundle\") pod \"install-certs-edpm-compute-no-nodes-edpm-compute-no-nodes-45bwg\" (UID: \"58aa9910-8f11-4e46-abd7-816fb0439dc5\") " pod="openstack-kuttl-tests/install-certs-edpm-compute-no-nodes-edpm-compute-no-nodes-45bwg" Dec 03 13:02:04 crc kubenswrapper[5052]: I1203 13:02:04.095204 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"neutron-ovn-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/58aa9910-8f11-4e46-abd7-816fb0439dc5-neutron-ovn-combined-ca-bundle\") pod \"install-certs-edpm-compute-no-nodes-edpm-compute-no-nodes-45bwg\" (UID: \"58aa9910-8f11-4e46-abd7-816fb0439dc5\") " pod="openstack-kuttl-tests/install-certs-edpm-compute-no-nodes-edpm-compute-no-nodes-45bwg" Dec 03 13:02:04 crc kubenswrapper[5052]: I1203 13:02:04.095662 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"neutron-dhcp-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/58aa9910-8f11-4e46-abd7-816fb0439dc5-neutron-dhcp-combined-ca-bundle\") pod \"install-certs-edpm-compute-no-nodes-edpm-compute-no-nodes-45bwg\" (UID: \"58aa9910-8f11-4e46-abd7-816fb0439dc5\") " pod="openstack-kuttl-tests/install-certs-edpm-compute-no-nodes-edpm-compute-no-nodes-45bwg" Dec 03 13:02:04 crc kubenswrapper[5052]: I1203 13:02:04.100041 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/58aa9910-8f11-4e46-abd7-816fb0439dc5-ssh-key\") pod \"install-certs-edpm-compute-no-nodes-edpm-compute-no-nodes-45bwg\" (UID: \"58aa9910-8f11-4e46-abd7-816fb0439dc5\") " pod="openstack-kuttl-tests/install-certs-edpm-compute-no-nodes-edpm-compute-no-nodes-45bwg" Dec 03 13:02:04 crc kubenswrapper[5052]: I1203 13:02:04.109157 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-2qnp8\" (UniqueName: \"kubernetes.io/projected/58aa9910-8f11-4e46-abd7-816fb0439dc5-kube-api-access-2qnp8\") pod \"install-certs-edpm-compute-no-nodes-edpm-compute-no-nodes-45bwg\" (UID: \"58aa9910-8f11-4e46-abd7-816fb0439dc5\") " pod="openstack-kuttl-tests/install-certs-edpm-compute-no-nodes-edpm-compute-no-nodes-45bwg" Dec 03 13:02:04 crc kubenswrapper[5052]: I1203 13:02:04.203796 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/install-certs-edpm-compute-no-nodes-edpm-compute-no-nodes-45bwg" Dec 03 13:02:04 crc kubenswrapper[5052]: I1203 13:02:04.620649 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/install-certs-edpm-compute-no-nodes-edpm-compute-no-nodes-45bwg"] Dec 03 13:02:04 crc kubenswrapper[5052]: W1203 13:02:04.621691 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod58aa9910_8f11_4e46_abd7_816fb0439dc5.slice/crio-7ddbb0348cce0c27b89d2c46bb77d5767e92c1f82b1b2ef93d1eb3a0d846c89c WatchSource:0}: Error finding container 7ddbb0348cce0c27b89d2c46bb77d5767e92c1f82b1b2ef93d1eb3a0d846c89c: Status 404 returned error can't find the container with id 7ddbb0348cce0c27b89d2c46bb77d5767e92c1f82b1b2ef93d1eb3a0d846c89c Dec 03 13:02:05 crc kubenswrapper[5052]: I1203 13:02:05.555393 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/install-certs-edpm-compute-no-nodes-edpm-compute-no-nodes-45bwg" event={"ID":"58aa9910-8f11-4e46-abd7-816fb0439dc5","Type":"ContainerStarted","Data":"dfeed030359f149998b1eacfc553f122f272079d2f91b4885c2f7c259d4e916c"} Dec 03 13:02:05 crc kubenswrapper[5052]: I1203 13:02:05.555807 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/install-certs-edpm-compute-no-nodes-edpm-compute-no-nodes-45bwg" event={"ID":"58aa9910-8f11-4e46-abd7-816fb0439dc5","Type":"ContainerStarted","Data":"7ddbb0348cce0c27b89d2c46bb77d5767e92c1f82b1b2ef93d1eb3a0d846c89c"} Dec 03 13:02:05 crc kubenswrapper[5052]: I1203 13:02:05.579262 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/install-certs-edpm-compute-no-nodes-edpm-compute-no-nodes-45bwg" podStartSLOduration=2.146668941 podStartE2EDuration="2.579239662s" podCreationTimestamp="2025-12-03 13:02:03 +0000 UTC" firstStartedPulling="2025-12-03 13:02:04.623819928 +0000 UTC m=+6115.212061692" lastFinishedPulling="2025-12-03 13:02:05.056390649 +0000 UTC m=+6115.644632413" observedRunningTime="2025-12-03 13:02:05.573072385 +0000 UTC m=+6116.161314149" watchObservedRunningTime="2025-12-03 13:02:05.579239662 +0000 UTC m=+6116.167481436" Dec 03 13:02:06 crc kubenswrapper[5052]: I1203 13:02:06.565969 5052 generic.go:334] "Generic (PLEG): container finished" podID="58aa9910-8f11-4e46-abd7-816fb0439dc5" containerID="dfeed030359f149998b1eacfc553f122f272079d2f91b4885c2f7c259d4e916c" exitCode=0 Dec 03 13:02:06 crc kubenswrapper[5052]: I1203 13:02:06.566034 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/install-certs-edpm-compute-no-nodes-edpm-compute-no-nodes-45bwg" event={"ID":"58aa9910-8f11-4e46-abd7-816fb0439dc5","Type":"ContainerDied","Data":"dfeed030359f149998b1eacfc553f122f272079d2f91b4885c2f7c259d4e916c"} Dec 03 13:02:07 crc kubenswrapper[5052]: I1203 13:02:07.867260 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/install-certs-edpm-compute-no-nodes-edpm-compute-no-nodes-45bwg" Dec 03 13:02:07 crc kubenswrapper[5052]: I1203 13:02:07.914609 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovn-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/58aa9910-8f11-4e46-abd7-816fb0439dc5-ovn-combined-ca-bundle\") pod \"58aa9910-8f11-4e46-abd7-816fb0439dc5\" (UID: \"58aa9910-8f11-4e46-abd7-816fb0439dc5\") " Dec 03 13:02:07 crc kubenswrapper[5052]: I1203 13:02:07.914661 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/58aa9910-8f11-4e46-abd7-816fb0439dc5-inventory\") pod \"58aa9910-8f11-4e46-abd7-816fb0439dc5\" (UID: \"58aa9910-8f11-4e46-abd7-816fb0439dc5\") " Dec 03 13:02:07 crc kubenswrapper[5052]: I1203 13:02:07.914693 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2qnp8\" (UniqueName: \"kubernetes.io/projected/58aa9910-8f11-4e46-abd7-816fb0439dc5-kube-api-access-2qnp8\") pod \"58aa9910-8f11-4e46-abd7-816fb0439dc5\" (UID: \"58aa9910-8f11-4e46-abd7-816fb0439dc5\") " Dec 03 13:02:07 crc kubenswrapper[5052]: I1203 13:02:07.914711 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/58aa9910-8f11-4e46-abd7-816fb0439dc5-ssh-key\") pod \"58aa9910-8f11-4e46-abd7-816fb0439dc5\" (UID: \"58aa9910-8f11-4e46-abd7-816fb0439dc5\") " Dec 03 13:02:07 crc kubenswrapper[5052]: I1203 13:02:07.914735 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"neutron-metadata-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/58aa9910-8f11-4e46-abd7-816fb0439dc5-neutron-metadata-combined-ca-bundle\") pod \"58aa9910-8f11-4e46-abd7-816fb0439dc5\" (UID: \"58aa9910-8f11-4e46-abd7-816fb0439dc5\") " Dec 03 13:02:07 crc kubenswrapper[5052]: I1203 13:02:07.914787 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"nova-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/58aa9910-8f11-4e46-abd7-816fb0439dc5-nova-combined-ca-bundle\") pod \"58aa9910-8f11-4e46-abd7-816fb0439dc5\" (UID: \"58aa9910-8f11-4e46-abd7-816fb0439dc5\") " Dec 03 13:02:07 crc kubenswrapper[5052]: I1203 13:02:07.914809 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"neutron-dhcp-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/58aa9910-8f11-4e46-abd7-816fb0439dc5-neutron-dhcp-combined-ca-bundle\") pod \"58aa9910-8f11-4e46-abd7-816fb0439dc5\" (UID: \"58aa9910-8f11-4e46-abd7-816fb0439dc5\") " Dec 03 13:02:07 crc kubenswrapper[5052]: I1203 13:02:07.914849 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"libvirt-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/58aa9910-8f11-4e46-abd7-816fb0439dc5-libvirt-combined-ca-bundle\") pod \"58aa9910-8f11-4e46-abd7-816fb0439dc5\" (UID: \"58aa9910-8f11-4e46-abd7-816fb0439dc5\") " Dec 03 13:02:07 crc kubenswrapper[5052]: I1203 13:02:07.914902 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"neutron-sriov-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/58aa9910-8f11-4e46-abd7-816fb0439dc5-neutron-sriov-combined-ca-bundle\") pod \"58aa9910-8f11-4e46-abd7-816fb0439dc5\" (UID: \"58aa9910-8f11-4e46-abd7-816fb0439dc5\") " Dec 03 13:02:07 crc kubenswrapper[5052]: I1203 13:02:07.914939 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"bootstrap-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/58aa9910-8f11-4e46-abd7-816fb0439dc5-bootstrap-combined-ca-bundle\") pod \"58aa9910-8f11-4e46-abd7-816fb0439dc5\" (UID: \"58aa9910-8f11-4e46-abd7-816fb0439dc5\") " Dec 03 13:02:07 crc kubenswrapper[5052]: I1203 13:02:07.914962 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"neutron-ovn-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/58aa9910-8f11-4e46-abd7-816fb0439dc5-neutron-ovn-combined-ca-bundle\") pod \"58aa9910-8f11-4e46-abd7-816fb0439dc5\" (UID: \"58aa9910-8f11-4e46-abd7-816fb0439dc5\") " Dec 03 13:02:07 crc kubenswrapper[5052]: I1203 13:02:07.920380 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/58aa9910-8f11-4e46-abd7-816fb0439dc5-bootstrap-combined-ca-bundle" (OuterVolumeSpecName: "bootstrap-combined-ca-bundle") pod "58aa9910-8f11-4e46-abd7-816fb0439dc5" (UID: "58aa9910-8f11-4e46-abd7-816fb0439dc5"). InnerVolumeSpecName "bootstrap-combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:02:07 crc kubenswrapper[5052]: I1203 13:02:07.920670 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/58aa9910-8f11-4e46-abd7-816fb0439dc5-libvirt-combined-ca-bundle" (OuterVolumeSpecName: "libvirt-combined-ca-bundle") pod "58aa9910-8f11-4e46-abd7-816fb0439dc5" (UID: "58aa9910-8f11-4e46-abd7-816fb0439dc5"). InnerVolumeSpecName "libvirt-combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:02:07 crc kubenswrapper[5052]: I1203 13:02:07.920693 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/58aa9910-8f11-4e46-abd7-816fb0439dc5-neutron-dhcp-combined-ca-bundle" (OuterVolumeSpecName: "neutron-dhcp-combined-ca-bundle") pod "58aa9910-8f11-4e46-abd7-816fb0439dc5" (UID: "58aa9910-8f11-4e46-abd7-816fb0439dc5"). InnerVolumeSpecName "neutron-dhcp-combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:02:07 crc kubenswrapper[5052]: I1203 13:02:07.920826 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/58aa9910-8f11-4e46-abd7-816fb0439dc5-ovn-combined-ca-bundle" (OuterVolumeSpecName: "ovn-combined-ca-bundle") pod "58aa9910-8f11-4e46-abd7-816fb0439dc5" (UID: "58aa9910-8f11-4e46-abd7-816fb0439dc5"). InnerVolumeSpecName "ovn-combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:02:07 crc kubenswrapper[5052]: I1203 13:02:07.921026 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/58aa9910-8f11-4e46-abd7-816fb0439dc5-neutron-sriov-combined-ca-bundle" (OuterVolumeSpecName: "neutron-sriov-combined-ca-bundle") pod "58aa9910-8f11-4e46-abd7-816fb0439dc5" (UID: "58aa9910-8f11-4e46-abd7-816fb0439dc5"). InnerVolumeSpecName "neutron-sriov-combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:02:07 crc kubenswrapper[5052]: I1203 13:02:07.921266 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/58aa9910-8f11-4e46-abd7-816fb0439dc5-kube-api-access-2qnp8" (OuterVolumeSpecName: "kube-api-access-2qnp8") pod "58aa9910-8f11-4e46-abd7-816fb0439dc5" (UID: "58aa9910-8f11-4e46-abd7-816fb0439dc5"). InnerVolumeSpecName "kube-api-access-2qnp8". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:02:07 crc kubenswrapper[5052]: I1203 13:02:07.921704 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/58aa9910-8f11-4e46-abd7-816fb0439dc5-nova-combined-ca-bundle" (OuterVolumeSpecName: "nova-combined-ca-bundle") pod "58aa9910-8f11-4e46-abd7-816fb0439dc5" (UID: "58aa9910-8f11-4e46-abd7-816fb0439dc5"). InnerVolumeSpecName "nova-combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:02:07 crc kubenswrapper[5052]: I1203 13:02:07.922134 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/58aa9910-8f11-4e46-abd7-816fb0439dc5-neutron-metadata-combined-ca-bundle" (OuterVolumeSpecName: "neutron-metadata-combined-ca-bundle") pod "58aa9910-8f11-4e46-abd7-816fb0439dc5" (UID: "58aa9910-8f11-4e46-abd7-816fb0439dc5"). InnerVolumeSpecName "neutron-metadata-combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:02:07 crc kubenswrapper[5052]: I1203 13:02:07.922667 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/58aa9910-8f11-4e46-abd7-816fb0439dc5-neutron-ovn-combined-ca-bundle" (OuterVolumeSpecName: "neutron-ovn-combined-ca-bundle") pod "58aa9910-8f11-4e46-abd7-816fb0439dc5" (UID: "58aa9910-8f11-4e46-abd7-816fb0439dc5"). InnerVolumeSpecName "neutron-ovn-combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:02:07 crc kubenswrapper[5052]: I1203 13:02:07.937700 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/58aa9910-8f11-4e46-abd7-816fb0439dc5-ssh-key" (OuterVolumeSpecName: "ssh-key") pod "58aa9910-8f11-4e46-abd7-816fb0439dc5" (UID: "58aa9910-8f11-4e46-abd7-816fb0439dc5"). InnerVolumeSpecName "ssh-key". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:02:07 crc kubenswrapper[5052]: I1203 13:02:07.939493 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/58aa9910-8f11-4e46-abd7-816fb0439dc5-inventory" (OuterVolumeSpecName: "inventory") pod "58aa9910-8f11-4e46-abd7-816fb0439dc5" (UID: "58aa9910-8f11-4e46-abd7-816fb0439dc5"). InnerVolumeSpecName "inventory". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:02:08 crc kubenswrapper[5052]: I1203 13:02:08.016787 5052 reconciler_common.go:293] "Volume detached for volume \"nova-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/58aa9910-8f11-4e46-abd7-816fb0439dc5-nova-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 13:02:08 crc kubenswrapper[5052]: I1203 13:02:08.016839 5052 reconciler_common.go:293] "Volume detached for volume \"neutron-dhcp-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/58aa9910-8f11-4e46-abd7-816fb0439dc5-neutron-dhcp-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 13:02:08 crc kubenswrapper[5052]: I1203 13:02:08.016853 5052 reconciler_common.go:293] "Volume detached for volume \"libvirt-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/58aa9910-8f11-4e46-abd7-816fb0439dc5-libvirt-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 13:02:08 crc kubenswrapper[5052]: I1203 13:02:08.016866 5052 reconciler_common.go:293] "Volume detached for volume \"neutron-sriov-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/58aa9910-8f11-4e46-abd7-816fb0439dc5-neutron-sriov-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 13:02:08 crc kubenswrapper[5052]: I1203 13:02:08.016881 5052 reconciler_common.go:293] "Volume detached for volume \"bootstrap-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/58aa9910-8f11-4e46-abd7-816fb0439dc5-bootstrap-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 13:02:08 crc kubenswrapper[5052]: I1203 13:02:08.016925 5052 reconciler_common.go:293] "Volume detached for volume \"neutron-ovn-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/58aa9910-8f11-4e46-abd7-816fb0439dc5-neutron-ovn-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 13:02:08 crc kubenswrapper[5052]: I1203 13:02:08.016938 5052 reconciler_common.go:293] "Volume detached for volume \"ovn-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/58aa9910-8f11-4e46-abd7-816fb0439dc5-ovn-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 13:02:08 crc kubenswrapper[5052]: I1203 13:02:08.016952 5052 reconciler_common.go:293] "Volume detached for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/58aa9910-8f11-4e46-abd7-816fb0439dc5-inventory\") on node \"crc\" DevicePath \"\"" Dec 03 13:02:08 crc kubenswrapper[5052]: I1203 13:02:08.016969 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-2qnp8\" (UniqueName: \"kubernetes.io/projected/58aa9910-8f11-4e46-abd7-816fb0439dc5-kube-api-access-2qnp8\") on node \"crc\" DevicePath \"\"" Dec 03 13:02:08 crc kubenswrapper[5052]: I1203 13:02:08.016983 5052 reconciler_common.go:293] "Volume detached for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/58aa9910-8f11-4e46-abd7-816fb0439dc5-ssh-key\") on node \"crc\" DevicePath \"\"" Dec 03 13:02:08 crc kubenswrapper[5052]: I1203 13:02:08.016999 5052 reconciler_common.go:293] "Volume detached for volume \"neutron-metadata-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/58aa9910-8f11-4e46-abd7-816fb0439dc5-neutron-metadata-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 13:02:08 crc kubenswrapper[5052]: I1203 13:02:08.584215 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/install-certs-edpm-compute-no-nodes-edpm-compute-no-nodes-45bwg" event={"ID":"58aa9910-8f11-4e46-abd7-816fb0439dc5","Type":"ContainerDied","Data":"7ddbb0348cce0c27b89d2c46bb77d5767e92c1f82b1b2ef93d1eb3a0d846c89c"} Dec 03 13:02:08 crc kubenswrapper[5052]: I1203 13:02:08.584258 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7ddbb0348cce0c27b89d2c46bb77d5767e92c1f82b1b2ef93d1eb3a0d846c89c" Dec 03 13:02:08 crc kubenswrapper[5052]: I1203 13:02:08.584272 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/install-certs-edpm-compute-no-nodes-edpm-compute-no-nodes-45bwg" Dec 03 13:02:08 crc kubenswrapper[5052]: I1203 13:02:08.664441 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ovn-edpm-compute-no-nodes-edpm-compute-no-nodes-gjbzv"] Dec 03 13:02:08 crc kubenswrapper[5052]: E1203 13:02:08.665011 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="58aa9910-8f11-4e46-abd7-816fb0439dc5" containerName="install-certs-edpm-compute-no-nodes-edpm-compute-no-nodes" Dec 03 13:02:08 crc kubenswrapper[5052]: I1203 13:02:08.665040 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="58aa9910-8f11-4e46-abd7-816fb0439dc5" containerName="install-certs-edpm-compute-no-nodes-edpm-compute-no-nodes" Dec 03 13:02:08 crc kubenswrapper[5052]: I1203 13:02:08.665223 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="58aa9910-8f11-4e46-abd7-816fb0439dc5" containerName="install-certs-edpm-compute-no-nodes-edpm-compute-no-nodes" Dec 03 13:02:08 crc kubenswrapper[5052]: I1203 13:02:08.665828 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovn-edpm-compute-no-nodes-edpm-compute-no-nodes-gjbzv" Dec 03 13:02:08 crc kubenswrapper[5052]: I1203 13:02:08.668157 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"ovncontroller-config" Dec 03 13:02:08 crc kubenswrapper[5052]: I1203 13:02:08.668341 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"dataplane-ansible-ssh-private-key-secret" Dec 03 13:02:08 crc kubenswrapper[5052]: I1203 13:02:08.669203 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"openstack-aee-default-env" Dec 03 13:02:08 crc kubenswrapper[5052]: I1203 13:02:08.669267 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"combined-ca-bundle" Dec 03 13:02:08 crc kubenswrapper[5052]: I1203 13:02:08.669314 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"dataplanenodeset-edpm-compute-no-nodes" Dec 03 13:02:08 crc kubenswrapper[5052]: I1203 13:02:08.670148 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"edpm-compute-no-nodes-dockercfg-mkcgv" Dec 03 13:02:08 crc kubenswrapper[5052]: I1203 13:02:08.678860 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ovn-edpm-compute-no-nodes-edpm-compute-no-nodes-gjbzv"] Dec 03 13:02:08 crc kubenswrapper[5052]: I1203 13:02:08.728504 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3c02dad4-5a4f-476f-bf8a-a93e1e397cc8-ovn-combined-ca-bundle\") pod \"ovn-edpm-compute-no-nodes-edpm-compute-no-nodes-gjbzv\" (UID: \"3c02dad4-5a4f-476f-bf8a-a93e1e397cc8\") " pod="openstack-kuttl-tests/ovn-edpm-compute-no-nodes-edpm-compute-no-nodes-gjbzv" Dec 03 13:02:08 crc kubenswrapper[5052]: I1203 13:02:08.728584 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/3c02dad4-5a4f-476f-bf8a-a93e1e397cc8-inventory\") pod \"ovn-edpm-compute-no-nodes-edpm-compute-no-nodes-gjbzv\" (UID: \"3c02dad4-5a4f-476f-bf8a-a93e1e397cc8\") " pod="openstack-kuttl-tests/ovn-edpm-compute-no-nodes-edpm-compute-no-nodes-gjbzv" Dec 03 13:02:08 crc kubenswrapper[5052]: I1203 13:02:08.728631 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/3c02dad4-5a4f-476f-bf8a-a93e1e397cc8-ssh-key\") pod \"ovn-edpm-compute-no-nodes-edpm-compute-no-nodes-gjbzv\" (UID: \"3c02dad4-5a4f-476f-bf8a-a93e1e397cc8\") " pod="openstack-kuttl-tests/ovn-edpm-compute-no-nodes-edpm-compute-no-nodes-gjbzv" Dec 03 13:02:08 crc kubenswrapper[5052]: I1203 13:02:08.728672 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d9w94\" (UniqueName: \"kubernetes.io/projected/3c02dad4-5a4f-476f-bf8a-a93e1e397cc8-kube-api-access-d9w94\") pod \"ovn-edpm-compute-no-nodes-edpm-compute-no-nodes-gjbzv\" (UID: \"3c02dad4-5a4f-476f-bf8a-a93e1e397cc8\") " pod="openstack-kuttl-tests/ovn-edpm-compute-no-nodes-edpm-compute-no-nodes-gjbzv" Dec 03 13:02:08 crc kubenswrapper[5052]: I1203 13:02:08.728756 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovncontroller-config-0\" (UniqueName: \"kubernetes.io/configmap/3c02dad4-5a4f-476f-bf8a-a93e1e397cc8-ovncontroller-config-0\") pod \"ovn-edpm-compute-no-nodes-edpm-compute-no-nodes-gjbzv\" (UID: \"3c02dad4-5a4f-476f-bf8a-a93e1e397cc8\") " pod="openstack-kuttl-tests/ovn-edpm-compute-no-nodes-edpm-compute-no-nodes-gjbzv" Dec 03 13:02:08 crc kubenswrapper[5052]: I1203 13:02:08.833586 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovn-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3c02dad4-5a4f-476f-bf8a-a93e1e397cc8-ovn-combined-ca-bundle\") pod \"ovn-edpm-compute-no-nodes-edpm-compute-no-nodes-gjbzv\" (UID: \"3c02dad4-5a4f-476f-bf8a-a93e1e397cc8\") " pod="openstack-kuttl-tests/ovn-edpm-compute-no-nodes-edpm-compute-no-nodes-gjbzv" Dec 03 13:02:08 crc kubenswrapper[5052]: I1203 13:02:08.833645 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/3c02dad4-5a4f-476f-bf8a-a93e1e397cc8-inventory\") pod \"ovn-edpm-compute-no-nodes-edpm-compute-no-nodes-gjbzv\" (UID: \"3c02dad4-5a4f-476f-bf8a-a93e1e397cc8\") " pod="openstack-kuttl-tests/ovn-edpm-compute-no-nodes-edpm-compute-no-nodes-gjbzv" Dec 03 13:02:08 crc kubenswrapper[5052]: I1203 13:02:08.833681 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/3c02dad4-5a4f-476f-bf8a-a93e1e397cc8-ssh-key\") pod \"ovn-edpm-compute-no-nodes-edpm-compute-no-nodes-gjbzv\" (UID: \"3c02dad4-5a4f-476f-bf8a-a93e1e397cc8\") " pod="openstack-kuttl-tests/ovn-edpm-compute-no-nodes-edpm-compute-no-nodes-gjbzv" Dec 03 13:02:08 crc kubenswrapper[5052]: I1203 13:02:08.833712 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-d9w94\" (UniqueName: \"kubernetes.io/projected/3c02dad4-5a4f-476f-bf8a-a93e1e397cc8-kube-api-access-d9w94\") pod \"ovn-edpm-compute-no-nodes-edpm-compute-no-nodes-gjbzv\" (UID: \"3c02dad4-5a4f-476f-bf8a-a93e1e397cc8\") " pod="openstack-kuttl-tests/ovn-edpm-compute-no-nodes-edpm-compute-no-nodes-gjbzv" Dec 03 13:02:08 crc kubenswrapper[5052]: I1203 13:02:08.833738 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovncontroller-config-0\" (UniqueName: \"kubernetes.io/configmap/3c02dad4-5a4f-476f-bf8a-a93e1e397cc8-ovncontroller-config-0\") pod \"ovn-edpm-compute-no-nodes-edpm-compute-no-nodes-gjbzv\" (UID: \"3c02dad4-5a4f-476f-bf8a-a93e1e397cc8\") " pod="openstack-kuttl-tests/ovn-edpm-compute-no-nodes-edpm-compute-no-nodes-gjbzv" Dec 03 13:02:08 crc kubenswrapper[5052]: I1203 13:02:08.834795 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovncontroller-config-0\" (UniqueName: \"kubernetes.io/configmap/3c02dad4-5a4f-476f-bf8a-a93e1e397cc8-ovncontroller-config-0\") pod \"ovn-edpm-compute-no-nodes-edpm-compute-no-nodes-gjbzv\" (UID: \"3c02dad4-5a4f-476f-bf8a-a93e1e397cc8\") " pod="openstack-kuttl-tests/ovn-edpm-compute-no-nodes-edpm-compute-no-nodes-gjbzv" Dec 03 13:02:08 crc kubenswrapper[5052]: I1203 13:02:08.837469 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/3c02dad4-5a4f-476f-bf8a-a93e1e397cc8-inventory\") pod \"ovn-edpm-compute-no-nodes-edpm-compute-no-nodes-gjbzv\" (UID: \"3c02dad4-5a4f-476f-bf8a-a93e1e397cc8\") " pod="openstack-kuttl-tests/ovn-edpm-compute-no-nodes-edpm-compute-no-nodes-gjbzv" Dec 03 13:02:08 crc kubenswrapper[5052]: I1203 13:02:08.837830 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/3c02dad4-5a4f-476f-bf8a-a93e1e397cc8-ssh-key\") pod \"ovn-edpm-compute-no-nodes-edpm-compute-no-nodes-gjbzv\" (UID: \"3c02dad4-5a4f-476f-bf8a-a93e1e397cc8\") " pod="openstack-kuttl-tests/ovn-edpm-compute-no-nodes-edpm-compute-no-nodes-gjbzv" Dec 03 13:02:08 crc kubenswrapper[5052]: I1203 13:02:08.843023 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovn-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3c02dad4-5a4f-476f-bf8a-a93e1e397cc8-ovn-combined-ca-bundle\") pod \"ovn-edpm-compute-no-nodes-edpm-compute-no-nodes-gjbzv\" (UID: \"3c02dad4-5a4f-476f-bf8a-a93e1e397cc8\") " pod="openstack-kuttl-tests/ovn-edpm-compute-no-nodes-edpm-compute-no-nodes-gjbzv" Dec 03 13:02:08 crc kubenswrapper[5052]: I1203 13:02:08.856460 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-d9w94\" (UniqueName: \"kubernetes.io/projected/3c02dad4-5a4f-476f-bf8a-a93e1e397cc8-kube-api-access-d9w94\") pod \"ovn-edpm-compute-no-nodes-edpm-compute-no-nodes-gjbzv\" (UID: \"3c02dad4-5a4f-476f-bf8a-a93e1e397cc8\") " pod="openstack-kuttl-tests/ovn-edpm-compute-no-nodes-edpm-compute-no-nodes-gjbzv" Dec 03 13:02:08 crc kubenswrapper[5052]: I1203 13:02:08.983506 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovn-edpm-compute-no-nodes-edpm-compute-no-nodes-gjbzv" Dec 03 13:02:09 crc kubenswrapper[5052]: I1203 13:02:09.396565 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ovn-edpm-compute-no-nodes-edpm-compute-no-nodes-gjbzv"] Dec 03 13:02:09 crc kubenswrapper[5052]: I1203 13:02:09.594429 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovn-edpm-compute-no-nodes-edpm-compute-no-nodes-gjbzv" event={"ID":"3c02dad4-5a4f-476f-bf8a-a93e1e397cc8","Type":"ContainerStarted","Data":"befd3362e7340aca3e06197f38bd5a6a4129e219bda5500054a7da328c93d043"} Dec 03 13:02:10 crc kubenswrapper[5052]: I1203 13:02:10.160209 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"openstack-aee-default-env" Dec 03 13:02:10 crc kubenswrapper[5052]: I1203 13:02:10.608040 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovn-edpm-compute-no-nodes-edpm-compute-no-nodes-gjbzv" event={"ID":"3c02dad4-5a4f-476f-bf8a-a93e1e397cc8","Type":"ContainerStarted","Data":"a704128af20876fd571a1aed7e617e6f9b02413124b08ae38ffede0cbff7229b"} Dec 03 13:02:10 crc kubenswrapper[5052]: I1203 13:02:10.631526 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/ovn-edpm-compute-no-nodes-edpm-compute-no-nodes-gjbzv" podStartSLOduration=1.879378948 podStartE2EDuration="2.631489576s" podCreationTimestamp="2025-12-03 13:02:08 +0000 UTC" firstStartedPulling="2025-12-03 13:02:09.405532356 +0000 UTC m=+6119.993774120" lastFinishedPulling="2025-12-03 13:02:10.157642984 +0000 UTC m=+6120.745884748" observedRunningTime="2025-12-03 13:02:10.627769299 +0000 UTC m=+6121.216011063" watchObservedRunningTime="2025-12-03 13:02:10.631489576 +0000 UTC m=+6121.219731360" Dec 03 13:02:12 crc kubenswrapper[5052]: I1203 13:02:12.627438 5052 generic.go:334] "Generic (PLEG): container finished" podID="3c02dad4-5a4f-476f-bf8a-a93e1e397cc8" containerID="a704128af20876fd571a1aed7e617e6f9b02413124b08ae38ffede0cbff7229b" exitCode=0 Dec 03 13:02:12 crc kubenswrapper[5052]: I1203 13:02:12.627482 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovn-edpm-compute-no-nodes-edpm-compute-no-nodes-gjbzv" event={"ID":"3c02dad4-5a4f-476f-bf8a-a93e1e397cc8","Type":"ContainerDied","Data":"a704128af20876fd571a1aed7e617e6f9b02413124b08ae38ffede0cbff7229b"} Dec 03 13:02:13 crc kubenswrapper[5052]: I1203 13:02:13.979229 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovn-edpm-compute-no-nodes-edpm-compute-no-nodes-gjbzv" Dec 03 13:02:14 crc kubenswrapper[5052]: I1203 13:02:14.122357 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovncontroller-config-0\" (UniqueName: \"kubernetes.io/configmap/3c02dad4-5a4f-476f-bf8a-a93e1e397cc8-ovncontroller-config-0\") pod \"3c02dad4-5a4f-476f-bf8a-a93e1e397cc8\" (UID: \"3c02dad4-5a4f-476f-bf8a-a93e1e397cc8\") " Dec 03 13:02:14 crc kubenswrapper[5052]: I1203 13:02:14.122438 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/3c02dad4-5a4f-476f-bf8a-a93e1e397cc8-inventory\") pod \"3c02dad4-5a4f-476f-bf8a-a93e1e397cc8\" (UID: \"3c02dad4-5a4f-476f-bf8a-a93e1e397cc8\") " Dec 03 13:02:14 crc kubenswrapper[5052]: I1203 13:02:14.122462 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovn-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3c02dad4-5a4f-476f-bf8a-a93e1e397cc8-ovn-combined-ca-bundle\") pod \"3c02dad4-5a4f-476f-bf8a-a93e1e397cc8\" (UID: \"3c02dad4-5a4f-476f-bf8a-a93e1e397cc8\") " Dec 03 13:02:14 crc kubenswrapper[5052]: I1203 13:02:14.122545 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d9w94\" (UniqueName: \"kubernetes.io/projected/3c02dad4-5a4f-476f-bf8a-a93e1e397cc8-kube-api-access-d9w94\") pod \"3c02dad4-5a4f-476f-bf8a-a93e1e397cc8\" (UID: \"3c02dad4-5a4f-476f-bf8a-a93e1e397cc8\") " Dec 03 13:02:14 crc kubenswrapper[5052]: I1203 13:02:14.122587 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/3c02dad4-5a4f-476f-bf8a-a93e1e397cc8-ssh-key\") pod \"3c02dad4-5a4f-476f-bf8a-a93e1e397cc8\" (UID: \"3c02dad4-5a4f-476f-bf8a-a93e1e397cc8\") " Dec 03 13:02:14 crc kubenswrapper[5052]: I1203 13:02:14.138927 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3c02dad4-5a4f-476f-bf8a-a93e1e397cc8-ovn-combined-ca-bundle" (OuterVolumeSpecName: "ovn-combined-ca-bundle") pod "3c02dad4-5a4f-476f-bf8a-a93e1e397cc8" (UID: "3c02dad4-5a4f-476f-bf8a-a93e1e397cc8"). InnerVolumeSpecName "ovn-combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:02:14 crc kubenswrapper[5052]: I1203 13:02:14.140428 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3c02dad4-5a4f-476f-bf8a-a93e1e397cc8-kube-api-access-d9w94" (OuterVolumeSpecName: "kube-api-access-d9w94") pod "3c02dad4-5a4f-476f-bf8a-a93e1e397cc8" (UID: "3c02dad4-5a4f-476f-bf8a-a93e1e397cc8"). InnerVolumeSpecName "kube-api-access-d9w94". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:02:14 crc kubenswrapper[5052]: I1203 13:02:14.144316 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3c02dad4-5a4f-476f-bf8a-a93e1e397cc8-ovncontroller-config-0" (OuterVolumeSpecName: "ovncontroller-config-0") pod "3c02dad4-5a4f-476f-bf8a-a93e1e397cc8" (UID: "3c02dad4-5a4f-476f-bf8a-a93e1e397cc8"). InnerVolumeSpecName "ovncontroller-config-0". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:02:14 crc kubenswrapper[5052]: I1203 13:02:14.151945 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3c02dad4-5a4f-476f-bf8a-a93e1e397cc8-ssh-key" (OuterVolumeSpecName: "ssh-key") pod "3c02dad4-5a4f-476f-bf8a-a93e1e397cc8" (UID: "3c02dad4-5a4f-476f-bf8a-a93e1e397cc8"). InnerVolumeSpecName "ssh-key". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:02:14 crc kubenswrapper[5052]: I1203 13:02:14.158293 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3c02dad4-5a4f-476f-bf8a-a93e1e397cc8-inventory" (OuterVolumeSpecName: "inventory") pod "3c02dad4-5a4f-476f-bf8a-a93e1e397cc8" (UID: "3c02dad4-5a4f-476f-bf8a-a93e1e397cc8"). InnerVolumeSpecName "inventory". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:02:14 crc kubenswrapper[5052]: I1203 13:02:14.225071 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-d9w94\" (UniqueName: \"kubernetes.io/projected/3c02dad4-5a4f-476f-bf8a-a93e1e397cc8-kube-api-access-d9w94\") on node \"crc\" DevicePath \"\"" Dec 03 13:02:14 crc kubenswrapper[5052]: I1203 13:02:14.225122 5052 reconciler_common.go:293] "Volume detached for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/3c02dad4-5a4f-476f-bf8a-a93e1e397cc8-ssh-key\") on node \"crc\" DevicePath \"\"" Dec 03 13:02:14 crc kubenswrapper[5052]: I1203 13:02:14.225133 5052 reconciler_common.go:293] "Volume detached for volume \"ovncontroller-config-0\" (UniqueName: \"kubernetes.io/configmap/3c02dad4-5a4f-476f-bf8a-a93e1e397cc8-ovncontroller-config-0\") on node \"crc\" DevicePath \"\"" Dec 03 13:02:14 crc kubenswrapper[5052]: I1203 13:02:14.225144 5052 reconciler_common.go:293] "Volume detached for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/3c02dad4-5a4f-476f-bf8a-a93e1e397cc8-inventory\") on node \"crc\" DevicePath \"\"" Dec 03 13:02:14 crc kubenswrapper[5052]: I1203 13:02:14.225153 5052 reconciler_common.go:293] "Volume detached for volume \"ovn-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3c02dad4-5a4f-476f-bf8a-a93e1e397cc8-ovn-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 13:02:14 crc kubenswrapper[5052]: I1203 13:02:14.646337 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovn-edpm-compute-no-nodes-edpm-compute-no-nodes-gjbzv" event={"ID":"3c02dad4-5a4f-476f-bf8a-a93e1e397cc8","Type":"ContainerDied","Data":"befd3362e7340aca3e06197f38bd5a6a4129e219bda5500054a7da328c93d043"} Dec 03 13:02:14 crc kubenswrapper[5052]: I1203 13:02:14.646386 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="befd3362e7340aca3e06197f38bd5a6a4129e219bda5500054a7da328c93d043" Dec 03 13:02:14 crc kubenswrapper[5052]: I1203 13:02:14.646397 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovn-edpm-compute-no-nodes-edpm-compute-no-nodes-gjbzv" Dec 03 13:02:14 crc kubenswrapper[5052]: I1203 13:02:14.727139 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/neutron-metadata-edpm-compute-no-nodes-edpm-compute-no-nodhbt7h"] Dec 03 13:02:14 crc kubenswrapper[5052]: E1203 13:02:14.727553 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="3c02dad4-5a4f-476f-bf8a-a93e1e397cc8" containerName="ovn-edpm-compute-no-nodes-edpm-compute-no-nodes" Dec 03 13:02:14 crc kubenswrapper[5052]: I1203 13:02:14.727577 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="3c02dad4-5a4f-476f-bf8a-a93e1e397cc8" containerName="ovn-edpm-compute-no-nodes-edpm-compute-no-nodes" Dec 03 13:02:14 crc kubenswrapper[5052]: I1203 13:02:14.727775 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="3c02dad4-5a4f-476f-bf8a-a93e1e397cc8" containerName="ovn-edpm-compute-no-nodes-edpm-compute-no-nodes" Dec 03 13:02:14 crc kubenswrapper[5052]: I1203 13:02:14.728453 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-metadata-edpm-compute-no-nodes-edpm-compute-no-nodhbt7h" Dec 03 13:02:14 crc kubenswrapper[5052]: I1203 13:02:14.730894 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"combined-ca-bundle" Dec 03 13:02:14 crc kubenswrapper[5052]: I1203 13:02:14.731593 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"dataplane-ansible-ssh-private-key-secret" Dec 03 13:02:14 crc kubenswrapper[5052]: I1203 13:02:14.731858 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"neutron-ovn-metadata-agent-neutron-config" Dec 03 13:02:14 crc kubenswrapper[5052]: I1203 13:02:14.731988 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-metadata-neutron-config" Dec 03 13:02:14 crc kubenswrapper[5052]: I1203 13:02:14.732884 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"dataplanenodeset-edpm-compute-no-nodes" Dec 03 13:02:14 crc kubenswrapper[5052]: I1203 13:02:14.733000 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"edpm-compute-no-nodes-dockercfg-mkcgv" Dec 03 13:02:14 crc kubenswrapper[5052]: I1203 13:02:14.733401 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"openstack-aee-default-env" Dec 03 13:02:14 crc kubenswrapper[5052]: I1203 13:02:14.749667 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-metadata-edpm-compute-no-nodes-edpm-compute-no-nodhbt7h"] Dec 03 13:02:14 crc kubenswrapper[5052]: I1203 13:02:14.786315 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nova-metadata-neutron-config-2\" (UniqueName: \"kubernetes.io/secret/4d00433c-3aaf-42ab-ba98-b3ac7452f440-nova-metadata-neutron-config-2\") pod \"neutron-metadata-edpm-compute-no-nodes-edpm-compute-no-nodhbt7h\" (UID: \"4d00433c-3aaf-42ab-ba98-b3ac7452f440\") " pod="openstack-kuttl-tests/neutron-metadata-edpm-compute-no-nodes-edpm-compute-no-nodhbt7h" Dec 03 13:02:14 crc kubenswrapper[5052]: I1203 13:02:14.786395 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"neutron-metadata-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4d00433c-3aaf-42ab-ba98-b3ac7452f440-neutron-metadata-combined-ca-bundle\") pod \"neutron-metadata-edpm-compute-no-nodes-edpm-compute-no-nodhbt7h\" (UID: \"4d00433c-3aaf-42ab-ba98-b3ac7452f440\") " pod="openstack-kuttl-tests/neutron-metadata-edpm-compute-no-nodes-edpm-compute-no-nodhbt7h" Dec 03 13:02:14 crc kubenswrapper[5052]: I1203 13:02:14.787407 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nova-metadata-neutron-config-1\" (UniqueName: \"kubernetes.io/secret/4d00433c-3aaf-42ab-ba98-b3ac7452f440-nova-metadata-neutron-config-1\") pod \"neutron-metadata-edpm-compute-no-nodes-edpm-compute-no-nodhbt7h\" (UID: \"4d00433c-3aaf-42ab-ba98-b3ac7452f440\") " pod="openstack-kuttl-tests/neutron-metadata-edpm-compute-no-nodes-edpm-compute-no-nodhbt7h" Dec 03 13:02:14 crc kubenswrapper[5052]: I1203 13:02:14.787559 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7f5k8\" (UniqueName: \"kubernetes.io/projected/4d00433c-3aaf-42ab-ba98-b3ac7452f440-kube-api-access-7f5k8\") pod \"neutron-metadata-edpm-compute-no-nodes-edpm-compute-no-nodhbt7h\" (UID: \"4d00433c-3aaf-42ab-ba98-b3ac7452f440\") " pod="openstack-kuttl-tests/neutron-metadata-edpm-compute-no-nodes-edpm-compute-no-nodhbt7h" Dec 03 13:02:14 crc kubenswrapper[5052]: I1203 13:02:14.787612 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/4d00433c-3aaf-42ab-ba98-b3ac7452f440-ssh-key\") pod \"neutron-metadata-edpm-compute-no-nodes-edpm-compute-no-nodhbt7h\" (UID: \"4d00433c-3aaf-42ab-ba98-b3ac7452f440\") " pod="openstack-kuttl-tests/neutron-metadata-edpm-compute-no-nodes-edpm-compute-no-nodhbt7h" Dec 03 13:02:14 crc kubenswrapper[5052]: I1203 13:02:14.787649 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/4d00433c-3aaf-42ab-ba98-b3ac7452f440-inventory\") pod \"neutron-metadata-edpm-compute-no-nodes-edpm-compute-no-nodhbt7h\" (UID: \"4d00433c-3aaf-42ab-ba98-b3ac7452f440\") " pod="openstack-kuttl-tests/neutron-metadata-edpm-compute-no-nodes-edpm-compute-no-nodhbt7h" Dec 03 13:02:14 crc kubenswrapper[5052]: I1203 13:02:14.787681 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nova-metadata-neutron-config-0\" (UniqueName: \"kubernetes.io/secret/4d00433c-3aaf-42ab-ba98-b3ac7452f440-nova-metadata-neutron-config-0\") pod \"neutron-metadata-edpm-compute-no-nodes-edpm-compute-no-nodhbt7h\" (UID: \"4d00433c-3aaf-42ab-ba98-b3ac7452f440\") " pod="openstack-kuttl-tests/neutron-metadata-edpm-compute-no-nodes-edpm-compute-no-nodhbt7h" Dec 03 13:02:14 crc kubenswrapper[5052]: I1203 13:02:14.787890 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"neutron-ovn-metadata-agent-neutron-config-0\" (UniqueName: \"kubernetes.io/secret/4d00433c-3aaf-42ab-ba98-b3ac7452f440-neutron-ovn-metadata-agent-neutron-config-0\") pod \"neutron-metadata-edpm-compute-no-nodes-edpm-compute-no-nodhbt7h\" (UID: \"4d00433c-3aaf-42ab-ba98-b3ac7452f440\") " pod="openstack-kuttl-tests/neutron-metadata-edpm-compute-no-nodes-edpm-compute-no-nodhbt7h" Dec 03 13:02:14 crc kubenswrapper[5052]: I1203 13:02:14.890222 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"nova-metadata-neutron-config-1\" (UniqueName: \"kubernetes.io/secret/4d00433c-3aaf-42ab-ba98-b3ac7452f440-nova-metadata-neutron-config-1\") pod \"neutron-metadata-edpm-compute-no-nodes-edpm-compute-no-nodhbt7h\" (UID: \"4d00433c-3aaf-42ab-ba98-b3ac7452f440\") " pod="openstack-kuttl-tests/neutron-metadata-edpm-compute-no-nodes-edpm-compute-no-nodhbt7h" Dec 03 13:02:14 crc kubenswrapper[5052]: I1203 13:02:14.890347 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-7f5k8\" (UniqueName: \"kubernetes.io/projected/4d00433c-3aaf-42ab-ba98-b3ac7452f440-kube-api-access-7f5k8\") pod \"neutron-metadata-edpm-compute-no-nodes-edpm-compute-no-nodhbt7h\" (UID: \"4d00433c-3aaf-42ab-ba98-b3ac7452f440\") " pod="openstack-kuttl-tests/neutron-metadata-edpm-compute-no-nodes-edpm-compute-no-nodhbt7h" Dec 03 13:02:14 crc kubenswrapper[5052]: I1203 13:02:14.890387 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/4d00433c-3aaf-42ab-ba98-b3ac7452f440-ssh-key\") pod \"neutron-metadata-edpm-compute-no-nodes-edpm-compute-no-nodhbt7h\" (UID: \"4d00433c-3aaf-42ab-ba98-b3ac7452f440\") " pod="openstack-kuttl-tests/neutron-metadata-edpm-compute-no-nodes-edpm-compute-no-nodhbt7h" Dec 03 13:02:14 crc kubenswrapper[5052]: I1203 13:02:14.890414 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/4d00433c-3aaf-42ab-ba98-b3ac7452f440-inventory\") pod \"neutron-metadata-edpm-compute-no-nodes-edpm-compute-no-nodhbt7h\" (UID: \"4d00433c-3aaf-42ab-ba98-b3ac7452f440\") " pod="openstack-kuttl-tests/neutron-metadata-edpm-compute-no-nodes-edpm-compute-no-nodhbt7h" Dec 03 13:02:14 crc kubenswrapper[5052]: I1203 13:02:14.890439 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"nova-metadata-neutron-config-0\" (UniqueName: \"kubernetes.io/secret/4d00433c-3aaf-42ab-ba98-b3ac7452f440-nova-metadata-neutron-config-0\") pod \"neutron-metadata-edpm-compute-no-nodes-edpm-compute-no-nodhbt7h\" (UID: \"4d00433c-3aaf-42ab-ba98-b3ac7452f440\") " pod="openstack-kuttl-tests/neutron-metadata-edpm-compute-no-nodes-edpm-compute-no-nodhbt7h" Dec 03 13:02:14 crc kubenswrapper[5052]: I1203 13:02:14.890476 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"neutron-ovn-metadata-agent-neutron-config-0\" (UniqueName: \"kubernetes.io/secret/4d00433c-3aaf-42ab-ba98-b3ac7452f440-neutron-ovn-metadata-agent-neutron-config-0\") pod \"neutron-metadata-edpm-compute-no-nodes-edpm-compute-no-nodhbt7h\" (UID: \"4d00433c-3aaf-42ab-ba98-b3ac7452f440\") " pod="openstack-kuttl-tests/neutron-metadata-edpm-compute-no-nodes-edpm-compute-no-nodhbt7h" Dec 03 13:02:14 crc kubenswrapper[5052]: I1203 13:02:14.890551 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"nova-metadata-neutron-config-2\" (UniqueName: \"kubernetes.io/secret/4d00433c-3aaf-42ab-ba98-b3ac7452f440-nova-metadata-neutron-config-2\") pod \"neutron-metadata-edpm-compute-no-nodes-edpm-compute-no-nodhbt7h\" (UID: \"4d00433c-3aaf-42ab-ba98-b3ac7452f440\") " pod="openstack-kuttl-tests/neutron-metadata-edpm-compute-no-nodes-edpm-compute-no-nodhbt7h" Dec 03 13:02:14 crc kubenswrapper[5052]: I1203 13:02:14.890590 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"neutron-metadata-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4d00433c-3aaf-42ab-ba98-b3ac7452f440-neutron-metadata-combined-ca-bundle\") pod \"neutron-metadata-edpm-compute-no-nodes-edpm-compute-no-nodhbt7h\" (UID: \"4d00433c-3aaf-42ab-ba98-b3ac7452f440\") " pod="openstack-kuttl-tests/neutron-metadata-edpm-compute-no-nodes-edpm-compute-no-nodhbt7h" Dec 03 13:02:14 crc kubenswrapper[5052]: I1203 13:02:14.896098 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"neutron-ovn-metadata-agent-neutron-config-0\" (UniqueName: \"kubernetes.io/secret/4d00433c-3aaf-42ab-ba98-b3ac7452f440-neutron-ovn-metadata-agent-neutron-config-0\") pod \"neutron-metadata-edpm-compute-no-nodes-edpm-compute-no-nodhbt7h\" (UID: \"4d00433c-3aaf-42ab-ba98-b3ac7452f440\") " pod="openstack-kuttl-tests/neutron-metadata-edpm-compute-no-nodes-edpm-compute-no-nodhbt7h" Dec 03 13:02:14 crc kubenswrapper[5052]: I1203 13:02:14.896173 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/4d00433c-3aaf-42ab-ba98-b3ac7452f440-ssh-key\") pod \"neutron-metadata-edpm-compute-no-nodes-edpm-compute-no-nodhbt7h\" (UID: \"4d00433c-3aaf-42ab-ba98-b3ac7452f440\") " pod="openstack-kuttl-tests/neutron-metadata-edpm-compute-no-nodes-edpm-compute-no-nodhbt7h" Dec 03 13:02:14 crc kubenswrapper[5052]: I1203 13:02:14.896194 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"nova-metadata-neutron-config-1\" (UniqueName: \"kubernetes.io/secret/4d00433c-3aaf-42ab-ba98-b3ac7452f440-nova-metadata-neutron-config-1\") pod \"neutron-metadata-edpm-compute-no-nodes-edpm-compute-no-nodhbt7h\" (UID: \"4d00433c-3aaf-42ab-ba98-b3ac7452f440\") " pod="openstack-kuttl-tests/neutron-metadata-edpm-compute-no-nodes-edpm-compute-no-nodhbt7h" Dec 03 13:02:14 crc kubenswrapper[5052]: I1203 13:02:14.896489 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/4d00433c-3aaf-42ab-ba98-b3ac7452f440-inventory\") pod \"neutron-metadata-edpm-compute-no-nodes-edpm-compute-no-nodhbt7h\" (UID: \"4d00433c-3aaf-42ab-ba98-b3ac7452f440\") " pod="openstack-kuttl-tests/neutron-metadata-edpm-compute-no-nodes-edpm-compute-no-nodhbt7h" Dec 03 13:02:14 crc kubenswrapper[5052]: I1203 13:02:14.897069 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"nova-metadata-neutron-config-2\" (UniqueName: \"kubernetes.io/secret/4d00433c-3aaf-42ab-ba98-b3ac7452f440-nova-metadata-neutron-config-2\") pod \"neutron-metadata-edpm-compute-no-nodes-edpm-compute-no-nodhbt7h\" (UID: \"4d00433c-3aaf-42ab-ba98-b3ac7452f440\") " pod="openstack-kuttl-tests/neutron-metadata-edpm-compute-no-nodes-edpm-compute-no-nodhbt7h" Dec 03 13:02:14 crc kubenswrapper[5052]: I1203 13:02:14.897466 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"nova-metadata-neutron-config-0\" (UniqueName: \"kubernetes.io/secret/4d00433c-3aaf-42ab-ba98-b3ac7452f440-nova-metadata-neutron-config-0\") pod \"neutron-metadata-edpm-compute-no-nodes-edpm-compute-no-nodhbt7h\" (UID: \"4d00433c-3aaf-42ab-ba98-b3ac7452f440\") " pod="openstack-kuttl-tests/neutron-metadata-edpm-compute-no-nodes-edpm-compute-no-nodhbt7h" Dec 03 13:02:14 crc kubenswrapper[5052]: I1203 13:02:14.900087 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"neutron-metadata-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4d00433c-3aaf-42ab-ba98-b3ac7452f440-neutron-metadata-combined-ca-bundle\") pod \"neutron-metadata-edpm-compute-no-nodes-edpm-compute-no-nodhbt7h\" (UID: \"4d00433c-3aaf-42ab-ba98-b3ac7452f440\") " pod="openstack-kuttl-tests/neutron-metadata-edpm-compute-no-nodes-edpm-compute-no-nodhbt7h" Dec 03 13:02:14 crc kubenswrapper[5052]: I1203 13:02:14.907264 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-7f5k8\" (UniqueName: \"kubernetes.io/projected/4d00433c-3aaf-42ab-ba98-b3ac7452f440-kube-api-access-7f5k8\") pod \"neutron-metadata-edpm-compute-no-nodes-edpm-compute-no-nodhbt7h\" (UID: \"4d00433c-3aaf-42ab-ba98-b3ac7452f440\") " pod="openstack-kuttl-tests/neutron-metadata-edpm-compute-no-nodes-edpm-compute-no-nodhbt7h" Dec 03 13:02:15 crc kubenswrapper[5052]: I1203 13:02:15.097242 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-metadata-edpm-compute-no-nodes-edpm-compute-no-nodhbt7h" Dec 03 13:02:15 crc kubenswrapper[5052]: I1203 13:02:15.579522 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-metadata-edpm-compute-no-nodes-edpm-compute-no-nodhbt7h"] Dec 03 13:02:15 crc kubenswrapper[5052]: I1203 13:02:15.656154 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-metadata-edpm-compute-no-nodes-edpm-compute-no-nodhbt7h" event={"ID":"4d00433c-3aaf-42ab-ba98-b3ac7452f440","Type":"ContainerStarted","Data":"d273008c611d3524029e832fbfbd2b129b837957f73a232c792ee2f10faec0d6"} Dec 03 13:02:16 crc kubenswrapper[5052]: I1203 13:02:16.677237 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-metadata-edpm-compute-no-nodes-edpm-compute-no-nodhbt7h" event={"ID":"4d00433c-3aaf-42ab-ba98-b3ac7452f440","Type":"ContainerStarted","Data":"7a70ad232f06ff17ca3e174b43c18a67b0887a52348825e075c4580fe817962a"} Dec 03 13:02:16 crc kubenswrapper[5052]: I1203 13:02:16.701541 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/neutron-metadata-edpm-compute-no-nodes-edpm-compute-no-nodhbt7h" podStartSLOduration=2.278663474 podStartE2EDuration="2.701450586s" podCreationTimestamp="2025-12-03 13:02:14 +0000 UTC" firstStartedPulling="2025-12-03 13:02:15.602547949 +0000 UTC m=+6126.190789713" lastFinishedPulling="2025-12-03 13:02:16.025335061 +0000 UTC m=+6126.613576825" observedRunningTime="2025-12-03 13:02:16.700285183 +0000 UTC m=+6127.288526977" watchObservedRunningTime="2025-12-03 13:02:16.701450586 +0000 UTC m=+6127.289692350" Dec 03 13:02:17 crc kubenswrapper[5052]: I1203 13:02:17.686545 5052 generic.go:334] "Generic (PLEG): container finished" podID="4d00433c-3aaf-42ab-ba98-b3ac7452f440" containerID="7a70ad232f06ff17ca3e174b43c18a67b0887a52348825e075c4580fe817962a" exitCode=0 Dec 03 13:02:17 crc kubenswrapper[5052]: I1203 13:02:17.686588 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-metadata-edpm-compute-no-nodes-edpm-compute-no-nodhbt7h" event={"ID":"4d00433c-3aaf-42ab-ba98-b3ac7452f440","Type":"ContainerDied","Data":"7a70ad232f06ff17ca3e174b43c18a67b0887a52348825e075c4580fe817962a"} Dec 03 13:02:18 crc kubenswrapper[5052]: I1203 13:02:18.967456 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-metadata-edpm-compute-no-nodes-edpm-compute-no-nodhbt7h" Dec 03 13:02:18 crc kubenswrapper[5052]: I1203 13:02:18.996672 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7f5k8\" (UniqueName: \"kubernetes.io/projected/4d00433c-3aaf-42ab-ba98-b3ac7452f440-kube-api-access-7f5k8\") pod \"4d00433c-3aaf-42ab-ba98-b3ac7452f440\" (UID: \"4d00433c-3aaf-42ab-ba98-b3ac7452f440\") " Dec 03 13:02:18 crc kubenswrapper[5052]: I1203 13:02:18.996744 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"nova-metadata-neutron-config-1\" (UniqueName: \"kubernetes.io/secret/4d00433c-3aaf-42ab-ba98-b3ac7452f440-nova-metadata-neutron-config-1\") pod \"4d00433c-3aaf-42ab-ba98-b3ac7452f440\" (UID: \"4d00433c-3aaf-42ab-ba98-b3ac7452f440\") " Dec 03 13:02:18 crc kubenswrapper[5052]: I1203 13:02:18.996820 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"neutron-metadata-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4d00433c-3aaf-42ab-ba98-b3ac7452f440-neutron-metadata-combined-ca-bundle\") pod \"4d00433c-3aaf-42ab-ba98-b3ac7452f440\" (UID: \"4d00433c-3aaf-42ab-ba98-b3ac7452f440\") " Dec 03 13:02:18 crc kubenswrapper[5052]: I1203 13:02:18.996867 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"neutron-ovn-metadata-agent-neutron-config-0\" (UniqueName: \"kubernetes.io/secret/4d00433c-3aaf-42ab-ba98-b3ac7452f440-neutron-ovn-metadata-agent-neutron-config-0\") pod \"4d00433c-3aaf-42ab-ba98-b3ac7452f440\" (UID: \"4d00433c-3aaf-42ab-ba98-b3ac7452f440\") " Dec 03 13:02:18 crc kubenswrapper[5052]: I1203 13:02:18.996892 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/4d00433c-3aaf-42ab-ba98-b3ac7452f440-inventory\") pod \"4d00433c-3aaf-42ab-ba98-b3ac7452f440\" (UID: \"4d00433c-3aaf-42ab-ba98-b3ac7452f440\") " Dec 03 13:02:18 crc kubenswrapper[5052]: I1203 13:02:18.996916 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/4d00433c-3aaf-42ab-ba98-b3ac7452f440-ssh-key\") pod \"4d00433c-3aaf-42ab-ba98-b3ac7452f440\" (UID: \"4d00433c-3aaf-42ab-ba98-b3ac7452f440\") " Dec 03 13:02:18 crc kubenswrapper[5052]: I1203 13:02:18.996984 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"nova-metadata-neutron-config-0\" (UniqueName: \"kubernetes.io/secret/4d00433c-3aaf-42ab-ba98-b3ac7452f440-nova-metadata-neutron-config-0\") pod \"4d00433c-3aaf-42ab-ba98-b3ac7452f440\" (UID: \"4d00433c-3aaf-42ab-ba98-b3ac7452f440\") " Dec 03 13:02:18 crc kubenswrapper[5052]: I1203 13:02:18.997010 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"nova-metadata-neutron-config-2\" (UniqueName: \"kubernetes.io/secret/4d00433c-3aaf-42ab-ba98-b3ac7452f440-nova-metadata-neutron-config-2\") pod \"4d00433c-3aaf-42ab-ba98-b3ac7452f440\" (UID: \"4d00433c-3aaf-42ab-ba98-b3ac7452f440\") " Dec 03 13:02:19 crc kubenswrapper[5052]: I1203 13:02:19.002639 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4d00433c-3aaf-42ab-ba98-b3ac7452f440-neutron-metadata-combined-ca-bundle" (OuterVolumeSpecName: "neutron-metadata-combined-ca-bundle") pod "4d00433c-3aaf-42ab-ba98-b3ac7452f440" (UID: "4d00433c-3aaf-42ab-ba98-b3ac7452f440"). InnerVolumeSpecName "neutron-metadata-combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:02:19 crc kubenswrapper[5052]: I1203 13:02:19.003165 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4d00433c-3aaf-42ab-ba98-b3ac7452f440-kube-api-access-7f5k8" (OuterVolumeSpecName: "kube-api-access-7f5k8") pod "4d00433c-3aaf-42ab-ba98-b3ac7452f440" (UID: "4d00433c-3aaf-42ab-ba98-b3ac7452f440"). InnerVolumeSpecName "kube-api-access-7f5k8". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:02:19 crc kubenswrapper[5052]: I1203 13:02:19.021744 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4d00433c-3aaf-42ab-ba98-b3ac7452f440-inventory" (OuterVolumeSpecName: "inventory") pod "4d00433c-3aaf-42ab-ba98-b3ac7452f440" (UID: "4d00433c-3aaf-42ab-ba98-b3ac7452f440"). InnerVolumeSpecName "inventory". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:02:19 crc kubenswrapper[5052]: I1203 13:02:19.021718 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4d00433c-3aaf-42ab-ba98-b3ac7452f440-ssh-key" (OuterVolumeSpecName: "ssh-key") pod "4d00433c-3aaf-42ab-ba98-b3ac7452f440" (UID: "4d00433c-3aaf-42ab-ba98-b3ac7452f440"). InnerVolumeSpecName "ssh-key". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:02:19 crc kubenswrapper[5052]: I1203 13:02:19.022347 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4d00433c-3aaf-42ab-ba98-b3ac7452f440-neutron-ovn-metadata-agent-neutron-config-0" (OuterVolumeSpecName: "neutron-ovn-metadata-agent-neutron-config-0") pod "4d00433c-3aaf-42ab-ba98-b3ac7452f440" (UID: "4d00433c-3aaf-42ab-ba98-b3ac7452f440"). InnerVolumeSpecName "neutron-ovn-metadata-agent-neutron-config-0". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:02:19 crc kubenswrapper[5052]: I1203 13:02:19.024154 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4d00433c-3aaf-42ab-ba98-b3ac7452f440-nova-metadata-neutron-config-0" (OuterVolumeSpecName: "nova-metadata-neutron-config-0") pod "4d00433c-3aaf-42ab-ba98-b3ac7452f440" (UID: "4d00433c-3aaf-42ab-ba98-b3ac7452f440"). InnerVolumeSpecName "nova-metadata-neutron-config-0". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:02:19 crc kubenswrapper[5052]: I1203 13:02:19.026899 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4d00433c-3aaf-42ab-ba98-b3ac7452f440-nova-metadata-neutron-config-2" (OuterVolumeSpecName: "nova-metadata-neutron-config-2") pod "4d00433c-3aaf-42ab-ba98-b3ac7452f440" (UID: "4d00433c-3aaf-42ab-ba98-b3ac7452f440"). InnerVolumeSpecName "nova-metadata-neutron-config-2". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:02:19 crc kubenswrapper[5052]: I1203 13:02:19.029709 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4d00433c-3aaf-42ab-ba98-b3ac7452f440-nova-metadata-neutron-config-1" (OuterVolumeSpecName: "nova-metadata-neutron-config-1") pod "4d00433c-3aaf-42ab-ba98-b3ac7452f440" (UID: "4d00433c-3aaf-42ab-ba98-b3ac7452f440"). InnerVolumeSpecName "nova-metadata-neutron-config-1". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:02:19 crc kubenswrapper[5052]: I1203 13:02:19.099246 5052 reconciler_common.go:293] "Volume detached for volume \"neutron-metadata-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4d00433c-3aaf-42ab-ba98-b3ac7452f440-neutron-metadata-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 13:02:19 crc kubenswrapper[5052]: I1203 13:02:19.099300 5052 reconciler_common.go:293] "Volume detached for volume \"neutron-ovn-metadata-agent-neutron-config-0\" (UniqueName: \"kubernetes.io/secret/4d00433c-3aaf-42ab-ba98-b3ac7452f440-neutron-ovn-metadata-agent-neutron-config-0\") on node \"crc\" DevicePath \"\"" Dec 03 13:02:19 crc kubenswrapper[5052]: I1203 13:02:19.099318 5052 reconciler_common.go:293] "Volume detached for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/4d00433c-3aaf-42ab-ba98-b3ac7452f440-inventory\") on node \"crc\" DevicePath \"\"" Dec 03 13:02:19 crc kubenswrapper[5052]: I1203 13:02:19.099330 5052 reconciler_common.go:293] "Volume detached for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/4d00433c-3aaf-42ab-ba98-b3ac7452f440-ssh-key\") on node \"crc\" DevicePath \"\"" Dec 03 13:02:19 crc kubenswrapper[5052]: I1203 13:02:19.099346 5052 reconciler_common.go:293] "Volume detached for volume \"nova-metadata-neutron-config-0\" (UniqueName: \"kubernetes.io/secret/4d00433c-3aaf-42ab-ba98-b3ac7452f440-nova-metadata-neutron-config-0\") on node \"crc\" DevicePath \"\"" Dec 03 13:02:19 crc kubenswrapper[5052]: I1203 13:02:19.099361 5052 reconciler_common.go:293] "Volume detached for volume \"nova-metadata-neutron-config-2\" (UniqueName: \"kubernetes.io/secret/4d00433c-3aaf-42ab-ba98-b3ac7452f440-nova-metadata-neutron-config-2\") on node \"crc\" DevicePath \"\"" Dec 03 13:02:19 crc kubenswrapper[5052]: I1203 13:02:19.099377 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-7f5k8\" (UniqueName: \"kubernetes.io/projected/4d00433c-3aaf-42ab-ba98-b3ac7452f440-kube-api-access-7f5k8\") on node \"crc\" DevicePath \"\"" Dec 03 13:02:19 crc kubenswrapper[5052]: I1203 13:02:19.099392 5052 reconciler_common.go:293] "Volume detached for volume \"nova-metadata-neutron-config-1\" (UniqueName: \"kubernetes.io/secret/4d00433c-3aaf-42ab-ba98-b3ac7452f440-nova-metadata-neutron-config-1\") on node \"crc\" DevicePath \"\"" Dec 03 13:02:19 crc kubenswrapper[5052]: I1203 13:02:19.707837 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-metadata-edpm-compute-no-nodes-edpm-compute-no-nodhbt7h" event={"ID":"4d00433c-3aaf-42ab-ba98-b3ac7452f440","Type":"ContainerDied","Data":"d273008c611d3524029e832fbfbd2b129b837957f73a232c792ee2f10faec0d6"} Dec 03 13:02:19 crc kubenswrapper[5052]: I1203 13:02:19.707876 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-metadata-edpm-compute-no-nodes-edpm-compute-no-nodhbt7h" Dec 03 13:02:19 crc kubenswrapper[5052]: I1203 13:02:19.707890 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d273008c611d3524029e832fbfbd2b129b837957f73a232c792ee2f10faec0d6" Dec 03 13:02:19 crc kubenswrapper[5052]: I1203 13:02:19.791018 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/neutron-ovn-edpm-compute-no-nodes-edpm-compute-no-nodes-tlp6p"] Dec 03 13:02:19 crc kubenswrapper[5052]: E1203 13:02:19.791383 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="4d00433c-3aaf-42ab-ba98-b3ac7452f440" containerName="neutron-metadata-edpm-compute-no-nodes-edpm-compute-no-nodes" Dec 03 13:02:19 crc kubenswrapper[5052]: I1203 13:02:19.791405 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="4d00433c-3aaf-42ab-ba98-b3ac7452f440" containerName="neutron-metadata-edpm-compute-no-nodes-edpm-compute-no-nodes" Dec 03 13:02:19 crc kubenswrapper[5052]: I1203 13:02:19.791622 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="4d00433c-3aaf-42ab-ba98-b3ac7452f440" containerName="neutron-metadata-edpm-compute-no-nodes-edpm-compute-no-nodes" Dec 03 13:02:19 crc kubenswrapper[5052]: I1203 13:02:19.792142 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-ovn-edpm-compute-no-nodes-edpm-compute-no-nodes-tlp6p" Dec 03 13:02:19 crc kubenswrapper[5052]: I1203 13:02:19.794212 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"openstack-aee-default-env" Dec 03 13:02:19 crc kubenswrapper[5052]: I1203 13:02:19.794435 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"dataplanenodeset-edpm-compute-no-nodes" Dec 03 13:02:19 crc kubenswrapper[5052]: I1203 13:02:19.794587 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"neutron-ovn-agent-neutron-config" Dec 03 13:02:19 crc kubenswrapper[5052]: I1203 13:02:19.794724 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"combined-ca-bundle" Dec 03 13:02:19 crc kubenswrapper[5052]: I1203 13:02:19.794824 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"dataplane-ansible-ssh-private-key-secret" Dec 03 13:02:19 crc kubenswrapper[5052]: I1203 13:02:19.795731 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"edpm-compute-no-nodes-dockercfg-mkcgv" Dec 03 13:02:19 crc kubenswrapper[5052]: I1203 13:02:19.805605 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-ovn-edpm-compute-no-nodes-edpm-compute-no-nodes-tlp6p"] Dec 03 13:02:19 crc kubenswrapper[5052]: I1203 13:02:19.808417 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mtsfn\" (UniqueName: \"kubernetes.io/projected/71a964fa-6056-4f7a-80ff-b792748299ba-kube-api-access-mtsfn\") pod \"neutron-ovn-edpm-compute-no-nodes-edpm-compute-no-nodes-tlp6p\" (UID: \"71a964fa-6056-4f7a-80ff-b792748299ba\") " pod="openstack-kuttl-tests/neutron-ovn-edpm-compute-no-nodes-edpm-compute-no-nodes-tlp6p" Dec 03 13:02:19 crc kubenswrapper[5052]: I1203 13:02:19.808468 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"neutron-ovn-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/71a964fa-6056-4f7a-80ff-b792748299ba-neutron-ovn-combined-ca-bundle\") pod \"neutron-ovn-edpm-compute-no-nodes-edpm-compute-no-nodes-tlp6p\" (UID: \"71a964fa-6056-4f7a-80ff-b792748299ba\") " pod="openstack-kuttl-tests/neutron-ovn-edpm-compute-no-nodes-edpm-compute-no-nodes-tlp6p" Dec 03 13:02:19 crc kubenswrapper[5052]: I1203 13:02:19.808523 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/71a964fa-6056-4f7a-80ff-b792748299ba-inventory\") pod \"neutron-ovn-edpm-compute-no-nodes-edpm-compute-no-nodes-tlp6p\" (UID: \"71a964fa-6056-4f7a-80ff-b792748299ba\") " pod="openstack-kuttl-tests/neutron-ovn-edpm-compute-no-nodes-edpm-compute-no-nodes-tlp6p" Dec 03 13:02:19 crc kubenswrapper[5052]: I1203 13:02:19.808590 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"neutron-ovn-agent-neutron-config-0\" (UniqueName: \"kubernetes.io/secret/71a964fa-6056-4f7a-80ff-b792748299ba-neutron-ovn-agent-neutron-config-0\") pod \"neutron-ovn-edpm-compute-no-nodes-edpm-compute-no-nodes-tlp6p\" (UID: \"71a964fa-6056-4f7a-80ff-b792748299ba\") " pod="openstack-kuttl-tests/neutron-ovn-edpm-compute-no-nodes-edpm-compute-no-nodes-tlp6p" Dec 03 13:02:19 crc kubenswrapper[5052]: I1203 13:02:19.808616 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/71a964fa-6056-4f7a-80ff-b792748299ba-ssh-key\") pod \"neutron-ovn-edpm-compute-no-nodes-edpm-compute-no-nodes-tlp6p\" (UID: \"71a964fa-6056-4f7a-80ff-b792748299ba\") " pod="openstack-kuttl-tests/neutron-ovn-edpm-compute-no-nodes-edpm-compute-no-nodes-tlp6p" Dec 03 13:02:19 crc kubenswrapper[5052]: I1203 13:02:19.909984 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-mtsfn\" (UniqueName: \"kubernetes.io/projected/71a964fa-6056-4f7a-80ff-b792748299ba-kube-api-access-mtsfn\") pod \"neutron-ovn-edpm-compute-no-nodes-edpm-compute-no-nodes-tlp6p\" (UID: \"71a964fa-6056-4f7a-80ff-b792748299ba\") " pod="openstack-kuttl-tests/neutron-ovn-edpm-compute-no-nodes-edpm-compute-no-nodes-tlp6p" Dec 03 13:02:19 crc kubenswrapper[5052]: I1203 13:02:19.910039 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"neutron-ovn-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/71a964fa-6056-4f7a-80ff-b792748299ba-neutron-ovn-combined-ca-bundle\") pod \"neutron-ovn-edpm-compute-no-nodes-edpm-compute-no-nodes-tlp6p\" (UID: \"71a964fa-6056-4f7a-80ff-b792748299ba\") " pod="openstack-kuttl-tests/neutron-ovn-edpm-compute-no-nodes-edpm-compute-no-nodes-tlp6p" Dec 03 13:02:19 crc kubenswrapper[5052]: I1203 13:02:19.910091 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/71a964fa-6056-4f7a-80ff-b792748299ba-inventory\") pod \"neutron-ovn-edpm-compute-no-nodes-edpm-compute-no-nodes-tlp6p\" (UID: \"71a964fa-6056-4f7a-80ff-b792748299ba\") " pod="openstack-kuttl-tests/neutron-ovn-edpm-compute-no-nodes-edpm-compute-no-nodes-tlp6p" Dec 03 13:02:19 crc kubenswrapper[5052]: I1203 13:02:19.910156 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"neutron-ovn-agent-neutron-config-0\" (UniqueName: \"kubernetes.io/secret/71a964fa-6056-4f7a-80ff-b792748299ba-neutron-ovn-agent-neutron-config-0\") pod \"neutron-ovn-edpm-compute-no-nodes-edpm-compute-no-nodes-tlp6p\" (UID: \"71a964fa-6056-4f7a-80ff-b792748299ba\") " pod="openstack-kuttl-tests/neutron-ovn-edpm-compute-no-nodes-edpm-compute-no-nodes-tlp6p" Dec 03 13:02:19 crc kubenswrapper[5052]: I1203 13:02:19.910186 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/71a964fa-6056-4f7a-80ff-b792748299ba-ssh-key\") pod \"neutron-ovn-edpm-compute-no-nodes-edpm-compute-no-nodes-tlp6p\" (UID: \"71a964fa-6056-4f7a-80ff-b792748299ba\") " pod="openstack-kuttl-tests/neutron-ovn-edpm-compute-no-nodes-edpm-compute-no-nodes-tlp6p" Dec 03 13:02:19 crc kubenswrapper[5052]: I1203 13:02:19.914534 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"neutron-ovn-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/71a964fa-6056-4f7a-80ff-b792748299ba-neutron-ovn-combined-ca-bundle\") pod \"neutron-ovn-edpm-compute-no-nodes-edpm-compute-no-nodes-tlp6p\" (UID: \"71a964fa-6056-4f7a-80ff-b792748299ba\") " pod="openstack-kuttl-tests/neutron-ovn-edpm-compute-no-nodes-edpm-compute-no-nodes-tlp6p" Dec 03 13:02:19 crc kubenswrapper[5052]: I1203 13:02:19.914613 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/71a964fa-6056-4f7a-80ff-b792748299ba-inventory\") pod \"neutron-ovn-edpm-compute-no-nodes-edpm-compute-no-nodes-tlp6p\" (UID: \"71a964fa-6056-4f7a-80ff-b792748299ba\") " pod="openstack-kuttl-tests/neutron-ovn-edpm-compute-no-nodes-edpm-compute-no-nodes-tlp6p" Dec 03 13:02:19 crc kubenswrapper[5052]: I1203 13:02:19.919571 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"neutron-ovn-agent-neutron-config-0\" (UniqueName: \"kubernetes.io/secret/71a964fa-6056-4f7a-80ff-b792748299ba-neutron-ovn-agent-neutron-config-0\") pod \"neutron-ovn-edpm-compute-no-nodes-edpm-compute-no-nodes-tlp6p\" (UID: \"71a964fa-6056-4f7a-80ff-b792748299ba\") " pod="openstack-kuttl-tests/neutron-ovn-edpm-compute-no-nodes-edpm-compute-no-nodes-tlp6p" Dec 03 13:02:19 crc kubenswrapper[5052]: I1203 13:02:19.926069 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/71a964fa-6056-4f7a-80ff-b792748299ba-ssh-key\") pod \"neutron-ovn-edpm-compute-no-nodes-edpm-compute-no-nodes-tlp6p\" (UID: \"71a964fa-6056-4f7a-80ff-b792748299ba\") " pod="openstack-kuttl-tests/neutron-ovn-edpm-compute-no-nodes-edpm-compute-no-nodes-tlp6p" Dec 03 13:02:19 crc kubenswrapper[5052]: I1203 13:02:19.929112 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-mtsfn\" (UniqueName: \"kubernetes.io/projected/71a964fa-6056-4f7a-80ff-b792748299ba-kube-api-access-mtsfn\") pod \"neutron-ovn-edpm-compute-no-nodes-edpm-compute-no-nodes-tlp6p\" (UID: \"71a964fa-6056-4f7a-80ff-b792748299ba\") " pod="openstack-kuttl-tests/neutron-ovn-edpm-compute-no-nodes-edpm-compute-no-nodes-tlp6p" Dec 03 13:02:20 crc kubenswrapper[5052]: I1203 13:02:20.111414 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-ovn-edpm-compute-no-nodes-edpm-compute-no-nodes-tlp6p" Dec 03 13:02:20 crc kubenswrapper[5052]: I1203 13:02:20.546873 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-ovn-edpm-compute-no-nodes-edpm-compute-no-nodes-tlp6p"] Dec 03 13:02:20 crc kubenswrapper[5052]: I1203 13:02:20.719226 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-ovn-edpm-compute-no-nodes-edpm-compute-no-nodes-tlp6p" event={"ID":"71a964fa-6056-4f7a-80ff-b792748299ba","Type":"ContainerStarted","Data":"2c1494c50572e9bbaabea0dd118b17377e73a9112e65f9d352621901aae301e9"} Dec 03 13:02:21 crc kubenswrapper[5052]: I1203 13:02:21.739312 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-ovn-edpm-compute-no-nodes-edpm-compute-no-nodes-tlp6p" event={"ID":"71a964fa-6056-4f7a-80ff-b792748299ba","Type":"ContainerStarted","Data":"01a7317e32f22df48f58620df8fc79333f9f7e4dcae549076b81833d21183f92"} Dec 03 13:02:21 crc kubenswrapper[5052]: I1203 13:02:21.758006 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/neutron-ovn-edpm-compute-no-nodes-edpm-compute-no-nodes-tlp6p" podStartSLOduration=2.222313704 podStartE2EDuration="2.757982464s" podCreationTimestamp="2025-12-03 13:02:19 +0000 UTC" firstStartedPulling="2025-12-03 13:02:20.552406256 +0000 UTC m=+6131.140648020" lastFinishedPulling="2025-12-03 13:02:21.088075016 +0000 UTC m=+6131.676316780" observedRunningTime="2025-12-03 13:02:21.757900502 +0000 UTC m=+6132.346142266" watchObservedRunningTime="2025-12-03 13:02:21.757982464 +0000 UTC m=+6132.346224228" Dec 03 13:02:23 crc kubenswrapper[5052]: I1203 13:02:22.751063 5052 generic.go:334] "Generic (PLEG): container finished" podID="71a964fa-6056-4f7a-80ff-b792748299ba" containerID="01a7317e32f22df48f58620df8fc79333f9f7e4dcae549076b81833d21183f92" exitCode=0 Dec 03 13:02:23 crc kubenswrapper[5052]: I1203 13:02:22.751095 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-ovn-edpm-compute-no-nodes-edpm-compute-no-nodes-tlp6p" event={"ID":"71a964fa-6056-4f7a-80ff-b792748299ba","Type":"ContainerDied","Data":"01a7317e32f22df48f58620df8fc79333f9f7e4dcae549076b81833d21183f92"} Dec 03 13:02:24 crc kubenswrapper[5052]: I1203 13:02:24.016565 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-ovn-edpm-compute-no-nodes-edpm-compute-no-nodes-tlp6p" Dec 03 13:02:24 crc kubenswrapper[5052]: I1203 13:02:24.175990 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mtsfn\" (UniqueName: \"kubernetes.io/projected/71a964fa-6056-4f7a-80ff-b792748299ba-kube-api-access-mtsfn\") pod \"71a964fa-6056-4f7a-80ff-b792748299ba\" (UID: \"71a964fa-6056-4f7a-80ff-b792748299ba\") " Dec 03 13:02:24 crc kubenswrapper[5052]: I1203 13:02:24.176058 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/71a964fa-6056-4f7a-80ff-b792748299ba-ssh-key\") pod \"71a964fa-6056-4f7a-80ff-b792748299ba\" (UID: \"71a964fa-6056-4f7a-80ff-b792748299ba\") " Dec 03 13:02:24 crc kubenswrapper[5052]: I1203 13:02:24.176125 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"neutron-ovn-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/71a964fa-6056-4f7a-80ff-b792748299ba-neutron-ovn-combined-ca-bundle\") pod \"71a964fa-6056-4f7a-80ff-b792748299ba\" (UID: \"71a964fa-6056-4f7a-80ff-b792748299ba\") " Dec 03 13:02:24 crc kubenswrapper[5052]: I1203 13:02:24.176210 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"neutron-ovn-agent-neutron-config-0\" (UniqueName: \"kubernetes.io/secret/71a964fa-6056-4f7a-80ff-b792748299ba-neutron-ovn-agent-neutron-config-0\") pod \"71a964fa-6056-4f7a-80ff-b792748299ba\" (UID: \"71a964fa-6056-4f7a-80ff-b792748299ba\") " Dec 03 13:02:24 crc kubenswrapper[5052]: I1203 13:02:24.176412 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/71a964fa-6056-4f7a-80ff-b792748299ba-inventory\") pod \"71a964fa-6056-4f7a-80ff-b792748299ba\" (UID: \"71a964fa-6056-4f7a-80ff-b792748299ba\") " Dec 03 13:02:24 crc kubenswrapper[5052]: I1203 13:02:24.183441 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/71a964fa-6056-4f7a-80ff-b792748299ba-kube-api-access-mtsfn" (OuterVolumeSpecName: "kube-api-access-mtsfn") pod "71a964fa-6056-4f7a-80ff-b792748299ba" (UID: "71a964fa-6056-4f7a-80ff-b792748299ba"). InnerVolumeSpecName "kube-api-access-mtsfn". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:02:24 crc kubenswrapper[5052]: I1203 13:02:24.184084 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/71a964fa-6056-4f7a-80ff-b792748299ba-neutron-ovn-combined-ca-bundle" (OuterVolumeSpecName: "neutron-ovn-combined-ca-bundle") pod "71a964fa-6056-4f7a-80ff-b792748299ba" (UID: "71a964fa-6056-4f7a-80ff-b792748299ba"). InnerVolumeSpecName "neutron-ovn-combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:02:24 crc kubenswrapper[5052]: I1203 13:02:24.202766 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/71a964fa-6056-4f7a-80ff-b792748299ba-neutron-ovn-agent-neutron-config-0" (OuterVolumeSpecName: "neutron-ovn-agent-neutron-config-0") pod "71a964fa-6056-4f7a-80ff-b792748299ba" (UID: "71a964fa-6056-4f7a-80ff-b792748299ba"). InnerVolumeSpecName "neutron-ovn-agent-neutron-config-0". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:02:24 crc kubenswrapper[5052]: I1203 13:02:24.203909 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/71a964fa-6056-4f7a-80ff-b792748299ba-ssh-key" (OuterVolumeSpecName: "ssh-key") pod "71a964fa-6056-4f7a-80ff-b792748299ba" (UID: "71a964fa-6056-4f7a-80ff-b792748299ba"). InnerVolumeSpecName "ssh-key". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:02:24 crc kubenswrapper[5052]: I1203 13:02:24.206644 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/71a964fa-6056-4f7a-80ff-b792748299ba-inventory" (OuterVolumeSpecName: "inventory") pod "71a964fa-6056-4f7a-80ff-b792748299ba" (UID: "71a964fa-6056-4f7a-80ff-b792748299ba"). InnerVolumeSpecName "inventory". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:02:24 crc kubenswrapper[5052]: I1203 13:02:24.277571 5052 reconciler_common.go:293] "Volume detached for volume \"neutron-ovn-agent-neutron-config-0\" (UniqueName: \"kubernetes.io/secret/71a964fa-6056-4f7a-80ff-b792748299ba-neutron-ovn-agent-neutron-config-0\") on node \"crc\" DevicePath \"\"" Dec 03 13:02:24 crc kubenswrapper[5052]: I1203 13:02:24.277621 5052 reconciler_common.go:293] "Volume detached for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/71a964fa-6056-4f7a-80ff-b792748299ba-inventory\") on node \"crc\" DevicePath \"\"" Dec 03 13:02:24 crc kubenswrapper[5052]: I1203 13:02:24.277638 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-mtsfn\" (UniqueName: \"kubernetes.io/projected/71a964fa-6056-4f7a-80ff-b792748299ba-kube-api-access-mtsfn\") on node \"crc\" DevicePath \"\"" Dec 03 13:02:24 crc kubenswrapper[5052]: I1203 13:02:24.277652 5052 reconciler_common.go:293] "Volume detached for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/71a964fa-6056-4f7a-80ff-b792748299ba-ssh-key\") on node \"crc\" DevicePath \"\"" Dec 03 13:02:24 crc kubenswrapper[5052]: I1203 13:02:24.277664 5052 reconciler_common.go:293] "Volume detached for volume \"neutron-ovn-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/71a964fa-6056-4f7a-80ff-b792748299ba-neutron-ovn-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 13:02:24 crc kubenswrapper[5052]: I1203 13:02:24.420662 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/neutron-sriov-edpm-compute-no-nodes-edpm-compute-no-nodes-mjjbz"] Dec 03 13:02:24 crc kubenswrapper[5052]: E1203 13:02:24.421005 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="71a964fa-6056-4f7a-80ff-b792748299ba" containerName="neutron-ovn-edpm-compute-no-nodes-edpm-compute-no-nodes" Dec 03 13:02:24 crc kubenswrapper[5052]: I1203 13:02:24.421019 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="71a964fa-6056-4f7a-80ff-b792748299ba" containerName="neutron-ovn-edpm-compute-no-nodes-edpm-compute-no-nodes" Dec 03 13:02:24 crc kubenswrapper[5052]: I1203 13:02:24.421158 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="71a964fa-6056-4f7a-80ff-b792748299ba" containerName="neutron-ovn-edpm-compute-no-nodes-edpm-compute-no-nodes" Dec 03 13:02:24 crc kubenswrapper[5052]: I1203 13:02:24.423585 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-sriov-edpm-compute-no-nodes-edpm-compute-no-nodes-mjjbz" Dec 03 13:02:24 crc kubenswrapper[5052]: I1203 13:02:24.430837 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"neutron-sriov-agent-neutron-config" Dec 03 13:02:24 crc kubenswrapper[5052]: I1203 13:02:24.435768 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-sriov-edpm-compute-no-nodes-edpm-compute-no-nodes-mjjbz"] Dec 03 13:02:24 crc kubenswrapper[5052]: I1203 13:02:24.581406 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/2d3f5967-7cd5-4400-865f-ad44684747f6-ssh-key\") pod \"neutron-sriov-edpm-compute-no-nodes-edpm-compute-no-nodes-mjjbz\" (UID: \"2d3f5967-7cd5-4400-865f-ad44684747f6\") " pod="openstack-kuttl-tests/neutron-sriov-edpm-compute-no-nodes-edpm-compute-no-nodes-mjjbz" Dec 03 13:02:24 crc kubenswrapper[5052]: I1203 13:02:24.581476 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nw9gc\" (UniqueName: \"kubernetes.io/projected/2d3f5967-7cd5-4400-865f-ad44684747f6-kube-api-access-nw9gc\") pod \"neutron-sriov-edpm-compute-no-nodes-edpm-compute-no-nodes-mjjbz\" (UID: \"2d3f5967-7cd5-4400-865f-ad44684747f6\") " pod="openstack-kuttl-tests/neutron-sriov-edpm-compute-no-nodes-edpm-compute-no-nodes-mjjbz" Dec 03 13:02:24 crc kubenswrapper[5052]: I1203 13:02:24.581539 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"neutron-sriov-agent-neutron-config-0\" (UniqueName: \"kubernetes.io/secret/2d3f5967-7cd5-4400-865f-ad44684747f6-neutron-sriov-agent-neutron-config-0\") pod \"neutron-sriov-edpm-compute-no-nodes-edpm-compute-no-nodes-mjjbz\" (UID: \"2d3f5967-7cd5-4400-865f-ad44684747f6\") " pod="openstack-kuttl-tests/neutron-sriov-edpm-compute-no-nodes-edpm-compute-no-nodes-mjjbz" Dec 03 13:02:24 crc kubenswrapper[5052]: I1203 13:02:24.581679 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/2d3f5967-7cd5-4400-865f-ad44684747f6-inventory\") pod \"neutron-sriov-edpm-compute-no-nodes-edpm-compute-no-nodes-mjjbz\" (UID: \"2d3f5967-7cd5-4400-865f-ad44684747f6\") " pod="openstack-kuttl-tests/neutron-sriov-edpm-compute-no-nodes-edpm-compute-no-nodes-mjjbz" Dec 03 13:02:24 crc kubenswrapper[5052]: I1203 13:02:24.581876 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"neutron-sriov-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2d3f5967-7cd5-4400-865f-ad44684747f6-neutron-sriov-combined-ca-bundle\") pod \"neutron-sriov-edpm-compute-no-nodes-edpm-compute-no-nodes-mjjbz\" (UID: \"2d3f5967-7cd5-4400-865f-ad44684747f6\") " pod="openstack-kuttl-tests/neutron-sriov-edpm-compute-no-nodes-edpm-compute-no-nodes-mjjbz" Dec 03 13:02:24 crc kubenswrapper[5052]: I1203 13:02:24.683827 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"neutron-sriov-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2d3f5967-7cd5-4400-865f-ad44684747f6-neutron-sriov-combined-ca-bundle\") pod \"neutron-sriov-edpm-compute-no-nodes-edpm-compute-no-nodes-mjjbz\" (UID: \"2d3f5967-7cd5-4400-865f-ad44684747f6\") " pod="openstack-kuttl-tests/neutron-sriov-edpm-compute-no-nodes-edpm-compute-no-nodes-mjjbz" Dec 03 13:02:24 crc kubenswrapper[5052]: I1203 13:02:24.683938 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/2d3f5967-7cd5-4400-865f-ad44684747f6-ssh-key\") pod \"neutron-sriov-edpm-compute-no-nodes-edpm-compute-no-nodes-mjjbz\" (UID: \"2d3f5967-7cd5-4400-865f-ad44684747f6\") " pod="openstack-kuttl-tests/neutron-sriov-edpm-compute-no-nodes-edpm-compute-no-nodes-mjjbz" Dec 03 13:02:24 crc kubenswrapper[5052]: I1203 13:02:24.683989 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-nw9gc\" (UniqueName: \"kubernetes.io/projected/2d3f5967-7cd5-4400-865f-ad44684747f6-kube-api-access-nw9gc\") pod \"neutron-sriov-edpm-compute-no-nodes-edpm-compute-no-nodes-mjjbz\" (UID: \"2d3f5967-7cd5-4400-865f-ad44684747f6\") " pod="openstack-kuttl-tests/neutron-sriov-edpm-compute-no-nodes-edpm-compute-no-nodes-mjjbz" Dec 03 13:02:24 crc kubenswrapper[5052]: I1203 13:02:24.684028 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"neutron-sriov-agent-neutron-config-0\" (UniqueName: \"kubernetes.io/secret/2d3f5967-7cd5-4400-865f-ad44684747f6-neutron-sriov-agent-neutron-config-0\") pod \"neutron-sriov-edpm-compute-no-nodes-edpm-compute-no-nodes-mjjbz\" (UID: \"2d3f5967-7cd5-4400-865f-ad44684747f6\") " pod="openstack-kuttl-tests/neutron-sriov-edpm-compute-no-nodes-edpm-compute-no-nodes-mjjbz" Dec 03 13:02:24 crc kubenswrapper[5052]: I1203 13:02:24.684076 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/2d3f5967-7cd5-4400-865f-ad44684747f6-inventory\") pod \"neutron-sriov-edpm-compute-no-nodes-edpm-compute-no-nodes-mjjbz\" (UID: \"2d3f5967-7cd5-4400-865f-ad44684747f6\") " pod="openstack-kuttl-tests/neutron-sriov-edpm-compute-no-nodes-edpm-compute-no-nodes-mjjbz" Dec 03 13:02:24 crc kubenswrapper[5052]: I1203 13:02:24.687878 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"neutron-sriov-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2d3f5967-7cd5-4400-865f-ad44684747f6-neutron-sriov-combined-ca-bundle\") pod \"neutron-sriov-edpm-compute-no-nodes-edpm-compute-no-nodes-mjjbz\" (UID: \"2d3f5967-7cd5-4400-865f-ad44684747f6\") " pod="openstack-kuttl-tests/neutron-sriov-edpm-compute-no-nodes-edpm-compute-no-nodes-mjjbz" Dec 03 13:02:24 crc kubenswrapper[5052]: I1203 13:02:24.687975 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/2d3f5967-7cd5-4400-865f-ad44684747f6-inventory\") pod \"neutron-sriov-edpm-compute-no-nodes-edpm-compute-no-nodes-mjjbz\" (UID: \"2d3f5967-7cd5-4400-865f-ad44684747f6\") " pod="openstack-kuttl-tests/neutron-sriov-edpm-compute-no-nodes-edpm-compute-no-nodes-mjjbz" Dec 03 13:02:24 crc kubenswrapper[5052]: I1203 13:02:24.688575 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"neutron-sriov-agent-neutron-config-0\" (UniqueName: \"kubernetes.io/secret/2d3f5967-7cd5-4400-865f-ad44684747f6-neutron-sriov-agent-neutron-config-0\") pod \"neutron-sriov-edpm-compute-no-nodes-edpm-compute-no-nodes-mjjbz\" (UID: \"2d3f5967-7cd5-4400-865f-ad44684747f6\") " pod="openstack-kuttl-tests/neutron-sriov-edpm-compute-no-nodes-edpm-compute-no-nodes-mjjbz" Dec 03 13:02:24 crc kubenswrapper[5052]: I1203 13:02:24.688969 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/2d3f5967-7cd5-4400-865f-ad44684747f6-ssh-key\") pod \"neutron-sriov-edpm-compute-no-nodes-edpm-compute-no-nodes-mjjbz\" (UID: \"2d3f5967-7cd5-4400-865f-ad44684747f6\") " pod="openstack-kuttl-tests/neutron-sriov-edpm-compute-no-nodes-edpm-compute-no-nodes-mjjbz" Dec 03 13:02:24 crc kubenswrapper[5052]: I1203 13:02:24.704440 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-nw9gc\" (UniqueName: \"kubernetes.io/projected/2d3f5967-7cd5-4400-865f-ad44684747f6-kube-api-access-nw9gc\") pod \"neutron-sriov-edpm-compute-no-nodes-edpm-compute-no-nodes-mjjbz\" (UID: \"2d3f5967-7cd5-4400-865f-ad44684747f6\") " pod="openstack-kuttl-tests/neutron-sriov-edpm-compute-no-nodes-edpm-compute-no-nodes-mjjbz" Dec 03 13:02:24 crc kubenswrapper[5052]: I1203 13:02:24.748202 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-sriov-edpm-compute-no-nodes-edpm-compute-no-nodes-mjjbz" Dec 03 13:02:24 crc kubenswrapper[5052]: I1203 13:02:24.775647 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-ovn-edpm-compute-no-nodes-edpm-compute-no-nodes-tlp6p" event={"ID":"71a964fa-6056-4f7a-80ff-b792748299ba","Type":"ContainerDied","Data":"2c1494c50572e9bbaabea0dd118b17377e73a9112e65f9d352621901aae301e9"} Dec 03 13:02:24 crc kubenswrapper[5052]: I1203 13:02:24.775702 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2c1494c50572e9bbaabea0dd118b17377e73a9112e65f9d352621901aae301e9" Dec 03 13:02:24 crc kubenswrapper[5052]: I1203 13:02:24.775812 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-ovn-edpm-compute-no-nodes-edpm-compute-no-nodes-tlp6p" Dec 03 13:02:25 crc kubenswrapper[5052]: I1203 13:02:25.173938 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-sriov-edpm-compute-no-nodes-edpm-compute-no-nodes-mjjbz"] Dec 03 13:02:25 crc kubenswrapper[5052]: W1203 13:02:25.179479 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2d3f5967_7cd5_4400_865f_ad44684747f6.slice/crio-2adb18571af87f4e9c396ab68e279fab29d0375748880c13b6e7965367a6f9ba WatchSource:0}: Error finding container 2adb18571af87f4e9c396ab68e279fab29d0375748880c13b6e7965367a6f9ba: Status 404 returned error can't find the container with id 2adb18571af87f4e9c396ab68e279fab29d0375748880c13b6e7965367a6f9ba Dec 03 13:02:25 crc kubenswrapper[5052]: I1203 13:02:25.786431 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-sriov-edpm-compute-no-nodes-edpm-compute-no-nodes-mjjbz" event={"ID":"2d3f5967-7cd5-4400-865f-ad44684747f6","Type":"ContainerStarted","Data":"2adb18571af87f4e9c396ab68e279fab29d0375748880c13b6e7965367a6f9ba"} Dec 03 13:02:26 crc kubenswrapper[5052]: I1203 13:02:26.814739 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-sriov-edpm-compute-no-nodes-edpm-compute-no-nodes-mjjbz" event={"ID":"2d3f5967-7cd5-4400-865f-ad44684747f6","Type":"ContainerStarted","Data":"9e4253f194f13ba34b48aaa8ed2cf4fcb7b95dfb4cbf26c38f1e61ae9c3e3b4f"} Dec 03 13:02:26 crc kubenswrapper[5052]: I1203 13:02:26.835005 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/neutron-sriov-edpm-compute-no-nodes-edpm-compute-no-nodes-mjjbz" podStartSLOduration=2.294556132 podStartE2EDuration="2.834984208s" podCreationTimestamp="2025-12-03 13:02:24 +0000 UTC" firstStartedPulling="2025-12-03 13:02:25.184246909 +0000 UTC m=+6135.772488673" lastFinishedPulling="2025-12-03 13:02:25.724674965 +0000 UTC m=+6136.312916749" observedRunningTime="2025-12-03 13:02:26.830900581 +0000 UTC m=+6137.419142345" watchObservedRunningTime="2025-12-03 13:02:26.834984208 +0000 UTC m=+6137.423225972" Dec 03 13:02:27 crc kubenswrapper[5052]: I1203 13:02:27.870855 5052 generic.go:334] "Generic (PLEG): container finished" podID="2d3f5967-7cd5-4400-865f-ad44684747f6" containerID="9e4253f194f13ba34b48aaa8ed2cf4fcb7b95dfb4cbf26c38f1e61ae9c3e3b4f" exitCode=0 Dec 03 13:02:27 crc kubenswrapper[5052]: I1203 13:02:27.871119 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-sriov-edpm-compute-no-nodes-edpm-compute-no-nodes-mjjbz" event={"ID":"2d3f5967-7cd5-4400-865f-ad44684747f6","Type":"ContainerDied","Data":"9e4253f194f13ba34b48aaa8ed2cf4fcb7b95dfb4cbf26c38f1e61ae9c3e3b4f"} Dec 03 13:02:29 crc kubenswrapper[5052]: I1203 13:02:29.155703 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-sriov-edpm-compute-no-nodes-edpm-compute-no-nodes-mjjbz" Dec 03 13:02:29 crc kubenswrapper[5052]: I1203 13:02:29.185133 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"neutron-sriov-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2d3f5967-7cd5-4400-865f-ad44684747f6-neutron-sriov-combined-ca-bundle\") pod \"2d3f5967-7cd5-4400-865f-ad44684747f6\" (UID: \"2d3f5967-7cd5-4400-865f-ad44684747f6\") " Dec 03 13:02:29 crc kubenswrapper[5052]: I1203 13:02:29.185198 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"neutron-sriov-agent-neutron-config-0\" (UniqueName: \"kubernetes.io/secret/2d3f5967-7cd5-4400-865f-ad44684747f6-neutron-sriov-agent-neutron-config-0\") pod \"2d3f5967-7cd5-4400-865f-ad44684747f6\" (UID: \"2d3f5967-7cd5-4400-865f-ad44684747f6\") " Dec 03 13:02:29 crc kubenswrapper[5052]: I1203 13:02:29.185254 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/2d3f5967-7cd5-4400-865f-ad44684747f6-inventory\") pod \"2d3f5967-7cd5-4400-865f-ad44684747f6\" (UID: \"2d3f5967-7cd5-4400-865f-ad44684747f6\") " Dec 03 13:02:29 crc kubenswrapper[5052]: I1203 13:02:29.185303 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nw9gc\" (UniqueName: \"kubernetes.io/projected/2d3f5967-7cd5-4400-865f-ad44684747f6-kube-api-access-nw9gc\") pod \"2d3f5967-7cd5-4400-865f-ad44684747f6\" (UID: \"2d3f5967-7cd5-4400-865f-ad44684747f6\") " Dec 03 13:02:29 crc kubenswrapper[5052]: I1203 13:02:29.185370 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/2d3f5967-7cd5-4400-865f-ad44684747f6-ssh-key\") pod \"2d3f5967-7cd5-4400-865f-ad44684747f6\" (UID: \"2d3f5967-7cd5-4400-865f-ad44684747f6\") " Dec 03 13:02:29 crc kubenswrapper[5052]: I1203 13:02:29.190576 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2d3f5967-7cd5-4400-865f-ad44684747f6-neutron-sriov-combined-ca-bundle" (OuterVolumeSpecName: "neutron-sriov-combined-ca-bundle") pod "2d3f5967-7cd5-4400-865f-ad44684747f6" (UID: "2d3f5967-7cd5-4400-865f-ad44684747f6"). InnerVolumeSpecName "neutron-sriov-combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:02:29 crc kubenswrapper[5052]: I1203 13:02:29.208954 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2d3f5967-7cd5-4400-865f-ad44684747f6-inventory" (OuterVolumeSpecName: "inventory") pod "2d3f5967-7cd5-4400-865f-ad44684747f6" (UID: "2d3f5967-7cd5-4400-865f-ad44684747f6"). InnerVolumeSpecName "inventory". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:02:29 crc kubenswrapper[5052]: I1203 13:02:29.211814 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2d3f5967-7cd5-4400-865f-ad44684747f6-kube-api-access-nw9gc" (OuterVolumeSpecName: "kube-api-access-nw9gc") pod "2d3f5967-7cd5-4400-865f-ad44684747f6" (UID: "2d3f5967-7cd5-4400-865f-ad44684747f6"). InnerVolumeSpecName "kube-api-access-nw9gc". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:02:29 crc kubenswrapper[5052]: I1203 13:02:29.217254 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2d3f5967-7cd5-4400-865f-ad44684747f6-ssh-key" (OuterVolumeSpecName: "ssh-key") pod "2d3f5967-7cd5-4400-865f-ad44684747f6" (UID: "2d3f5967-7cd5-4400-865f-ad44684747f6"). InnerVolumeSpecName "ssh-key". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:02:29 crc kubenswrapper[5052]: I1203 13:02:29.219837 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2d3f5967-7cd5-4400-865f-ad44684747f6-neutron-sriov-agent-neutron-config-0" (OuterVolumeSpecName: "neutron-sriov-agent-neutron-config-0") pod "2d3f5967-7cd5-4400-865f-ad44684747f6" (UID: "2d3f5967-7cd5-4400-865f-ad44684747f6"). InnerVolumeSpecName "neutron-sriov-agent-neutron-config-0". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:02:29 crc kubenswrapper[5052]: I1203 13:02:29.287798 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-nw9gc\" (UniqueName: \"kubernetes.io/projected/2d3f5967-7cd5-4400-865f-ad44684747f6-kube-api-access-nw9gc\") on node \"crc\" DevicePath \"\"" Dec 03 13:02:29 crc kubenswrapper[5052]: I1203 13:02:29.287866 5052 reconciler_common.go:293] "Volume detached for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/2d3f5967-7cd5-4400-865f-ad44684747f6-ssh-key\") on node \"crc\" DevicePath \"\"" Dec 03 13:02:29 crc kubenswrapper[5052]: I1203 13:02:29.287881 5052 reconciler_common.go:293] "Volume detached for volume \"neutron-sriov-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2d3f5967-7cd5-4400-865f-ad44684747f6-neutron-sriov-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 13:02:29 crc kubenswrapper[5052]: I1203 13:02:29.287896 5052 reconciler_common.go:293] "Volume detached for volume \"neutron-sriov-agent-neutron-config-0\" (UniqueName: \"kubernetes.io/secret/2d3f5967-7cd5-4400-865f-ad44684747f6-neutron-sriov-agent-neutron-config-0\") on node \"crc\" DevicePath \"\"" Dec 03 13:02:29 crc kubenswrapper[5052]: I1203 13:02:29.287911 5052 reconciler_common.go:293] "Volume detached for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/2d3f5967-7cd5-4400-865f-ad44684747f6-inventory\") on node \"crc\" DevicePath \"\"" Dec 03 13:02:29 crc kubenswrapper[5052]: I1203 13:02:29.379806 5052 patch_prober.go:28] interesting pod/machine-config-daemon-74bj6 container/machine-config-daemon namespace/openshift-machine-config-operator: Liveness probe status=failure output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" start-of-body= Dec 03 13:02:29 crc kubenswrapper[5052]: I1203 13:02:29.380193 5052 prober.go:107] "Probe failed" probeType="Liveness" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" containerName="machine-config-daemon" probeResult="failure" output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" Dec 03 13:02:29 crc kubenswrapper[5052]: I1203 13:02:29.889346 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-sriov-edpm-compute-no-nodes-edpm-compute-no-nodes-mjjbz" event={"ID":"2d3f5967-7cd5-4400-865f-ad44684747f6","Type":"ContainerDied","Data":"2adb18571af87f4e9c396ab68e279fab29d0375748880c13b6e7965367a6f9ba"} Dec 03 13:02:29 crc kubenswrapper[5052]: I1203 13:02:29.889612 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2adb18571af87f4e9c396ab68e279fab29d0375748880c13b6e7965367a6f9ba" Dec 03 13:02:29 crc kubenswrapper[5052]: I1203 13:02:29.889407 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-sriov-edpm-compute-no-nodes-edpm-compute-no-nodes-mjjbz" Dec 03 13:02:29 crc kubenswrapper[5052]: I1203 13:02:29.951893 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/neutron-dhcp-edpm-compute-no-nodes-edpm-compute-no-nodes-7bvhh"] Dec 03 13:02:29 crc kubenswrapper[5052]: E1203 13:02:29.952264 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="2d3f5967-7cd5-4400-865f-ad44684747f6" containerName="neutron-sriov-edpm-compute-no-nodes-edpm-compute-no-nodes" Dec 03 13:02:29 crc kubenswrapper[5052]: I1203 13:02:29.952288 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d3f5967-7cd5-4400-865f-ad44684747f6" containerName="neutron-sriov-edpm-compute-no-nodes-edpm-compute-no-nodes" Dec 03 13:02:29 crc kubenswrapper[5052]: I1203 13:02:29.952470 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="2d3f5967-7cd5-4400-865f-ad44684747f6" containerName="neutron-sriov-edpm-compute-no-nodes-edpm-compute-no-nodes" Dec 03 13:02:29 crc kubenswrapper[5052]: I1203 13:02:29.953078 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-dhcp-edpm-compute-no-nodes-edpm-compute-no-nodes-7bvhh" Dec 03 13:02:29 crc kubenswrapper[5052]: I1203 13:02:29.954735 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"neutron-dhcp-agent-neutron-config" Dec 03 13:02:29 crc kubenswrapper[5052]: I1203 13:02:29.955953 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"edpm-compute-no-nodes-dockercfg-mkcgv" Dec 03 13:02:29 crc kubenswrapper[5052]: I1203 13:02:29.956229 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"openstack-aee-default-env" Dec 03 13:02:29 crc kubenswrapper[5052]: I1203 13:02:29.956368 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"combined-ca-bundle" Dec 03 13:02:29 crc kubenswrapper[5052]: I1203 13:02:29.956581 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"dataplanenodeset-edpm-compute-no-nodes" Dec 03 13:02:29 crc kubenswrapper[5052]: I1203 13:02:29.956716 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"dataplane-ansible-ssh-private-key-secret" Dec 03 13:02:29 crc kubenswrapper[5052]: I1203 13:02:29.965809 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-dhcp-edpm-compute-no-nodes-edpm-compute-no-nodes-7bvhh"] Dec 03 13:02:29 crc kubenswrapper[5052]: I1203 13:02:29.997336 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/6e7fa506-85c2-4b8f-a21b-1b0e195b2e8e-ssh-key\") pod \"neutron-dhcp-edpm-compute-no-nodes-edpm-compute-no-nodes-7bvhh\" (UID: \"6e7fa506-85c2-4b8f-a21b-1b0e195b2e8e\") " pod="openstack-kuttl-tests/neutron-dhcp-edpm-compute-no-nodes-edpm-compute-no-nodes-7bvhh" Dec 03 13:02:29 crc kubenswrapper[5052]: I1203 13:02:29.997395 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bnsgl\" (UniqueName: \"kubernetes.io/projected/6e7fa506-85c2-4b8f-a21b-1b0e195b2e8e-kube-api-access-bnsgl\") pod \"neutron-dhcp-edpm-compute-no-nodes-edpm-compute-no-nodes-7bvhh\" (UID: \"6e7fa506-85c2-4b8f-a21b-1b0e195b2e8e\") " pod="openstack-kuttl-tests/neutron-dhcp-edpm-compute-no-nodes-edpm-compute-no-nodes-7bvhh" Dec 03 13:02:29 crc kubenswrapper[5052]: I1203 13:02:29.997471 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/6e7fa506-85c2-4b8f-a21b-1b0e195b2e8e-inventory\") pod \"neutron-dhcp-edpm-compute-no-nodes-edpm-compute-no-nodes-7bvhh\" (UID: \"6e7fa506-85c2-4b8f-a21b-1b0e195b2e8e\") " pod="openstack-kuttl-tests/neutron-dhcp-edpm-compute-no-nodes-edpm-compute-no-nodes-7bvhh" Dec 03 13:02:29 crc kubenswrapper[5052]: I1203 13:02:29.997559 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"neutron-dhcp-agent-neutron-config-0\" (UniqueName: \"kubernetes.io/secret/6e7fa506-85c2-4b8f-a21b-1b0e195b2e8e-neutron-dhcp-agent-neutron-config-0\") pod \"neutron-dhcp-edpm-compute-no-nodes-edpm-compute-no-nodes-7bvhh\" (UID: \"6e7fa506-85c2-4b8f-a21b-1b0e195b2e8e\") " pod="openstack-kuttl-tests/neutron-dhcp-edpm-compute-no-nodes-edpm-compute-no-nodes-7bvhh" Dec 03 13:02:29 crc kubenswrapper[5052]: I1203 13:02:29.997586 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"neutron-dhcp-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6e7fa506-85c2-4b8f-a21b-1b0e195b2e8e-neutron-dhcp-combined-ca-bundle\") pod \"neutron-dhcp-edpm-compute-no-nodes-edpm-compute-no-nodes-7bvhh\" (UID: \"6e7fa506-85c2-4b8f-a21b-1b0e195b2e8e\") " pod="openstack-kuttl-tests/neutron-dhcp-edpm-compute-no-nodes-edpm-compute-no-nodes-7bvhh" Dec 03 13:02:30 crc kubenswrapper[5052]: I1203 13:02:30.098897 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"neutron-dhcp-agent-neutron-config-0\" (UniqueName: \"kubernetes.io/secret/6e7fa506-85c2-4b8f-a21b-1b0e195b2e8e-neutron-dhcp-agent-neutron-config-0\") pod \"neutron-dhcp-edpm-compute-no-nodes-edpm-compute-no-nodes-7bvhh\" (UID: \"6e7fa506-85c2-4b8f-a21b-1b0e195b2e8e\") " pod="openstack-kuttl-tests/neutron-dhcp-edpm-compute-no-nodes-edpm-compute-no-nodes-7bvhh" Dec 03 13:02:30 crc kubenswrapper[5052]: I1203 13:02:30.098980 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"neutron-dhcp-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6e7fa506-85c2-4b8f-a21b-1b0e195b2e8e-neutron-dhcp-combined-ca-bundle\") pod \"neutron-dhcp-edpm-compute-no-nodes-edpm-compute-no-nodes-7bvhh\" (UID: \"6e7fa506-85c2-4b8f-a21b-1b0e195b2e8e\") " pod="openstack-kuttl-tests/neutron-dhcp-edpm-compute-no-nodes-edpm-compute-no-nodes-7bvhh" Dec 03 13:02:30 crc kubenswrapper[5052]: I1203 13:02:30.099028 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/6e7fa506-85c2-4b8f-a21b-1b0e195b2e8e-ssh-key\") pod \"neutron-dhcp-edpm-compute-no-nodes-edpm-compute-no-nodes-7bvhh\" (UID: \"6e7fa506-85c2-4b8f-a21b-1b0e195b2e8e\") " pod="openstack-kuttl-tests/neutron-dhcp-edpm-compute-no-nodes-edpm-compute-no-nodes-7bvhh" Dec 03 13:02:30 crc kubenswrapper[5052]: I1203 13:02:30.099066 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-bnsgl\" (UniqueName: \"kubernetes.io/projected/6e7fa506-85c2-4b8f-a21b-1b0e195b2e8e-kube-api-access-bnsgl\") pod \"neutron-dhcp-edpm-compute-no-nodes-edpm-compute-no-nodes-7bvhh\" (UID: \"6e7fa506-85c2-4b8f-a21b-1b0e195b2e8e\") " pod="openstack-kuttl-tests/neutron-dhcp-edpm-compute-no-nodes-edpm-compute-no-nodes-7bvhh" Dec 03 13:02:30 crc kubenswrapper[5052]: I1203 13:02:30.099126 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/6e7fa506-85c2-4b8f-a21b-1b0e195b2e8e-inventory\") pod \"neutron-dhcp-edpm-compute-no-nodes-edpm-compute-no-nodes-7bvhh\" (UID: \"6e7fa506-85c2-4b8f-a21b-1b0e195b2e8e\") " pod="openstack-kuttl-tests/neutron-dhcp-edpm-compute-no-nodes-edpm-compute-no-nodes-7bvhh" Dec 03 13:02:30 crc kubenswrapper[5052]: I1203 13:02:30.103632 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/6e7fa506-85c2-4b8f-a21b-1b0e195b2e8e-ssh-key\") pod \"neutron-dhcp-edpm-compute-no-nodes-edpm-compute-no-nodes-7bvhh\" (UID: \"6e7fa506-85c2-4b8f-a21b-1b0e195b2e8e\") " pod="openstack-kuttl-tests/neutron-dhcp-edpm-compute-no-nodes-edpm-compute-no-nodes-7bvhh" Dec 03 13:02:30 crc kubenswrapper[5052]: I1203 13:02:30.103777 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"neutron-dhcp-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6e7fa506-85c2-4b8f-a21b-1b0e195b2e8e-neutron-dhcp-combined-ca-bundle\") pod \"neutron-dhcp-edpm-compute-no-nodes-edpm-compute-no-nodes-7bvhh\" (UID: \"6e7fa506-85c2-4b8f-a21b-1b0e195b2e8e\") " pod="openstack-kuttl-tests/neutron-dhcp-edpm-compute-no-nodes-edpm-compute-no-nodes-7bvhh" Dec 03 13:02:30 crc kubenswrapper[5052]: I1203 13:02:30.104022 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"neutron-dhcp-agent-neutron-config-0\" (UniqueName: \"kubernetes.io/secret/6e7fa506-85c2-4b8f-a21b-1b0e195b2e8e-neutron-dhcp-agent-neutron-config-0\") pod \"neutron-dhcp-edpm-compute-no-nodes-edpm-compute-no-nodes-7bvhh\" (UID: \"6e7fa506-85c2-4b8f-a21b-1b0e195b2e8e\") " pod="openstack-kuttl-tests/neutron-dhcp-edpm-compute-no-nodes-edpm-compute-no-nodes-7bvhh" Dec 03 13:02:30 crc kubenswrapper[5052]: I1203 13:02:30.104464 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/6e7fa506-85c2-4b8f-a21b-1b0e195b2e8e-inventory\") pod \"neutron-dhcp-edpm-compute-no-nodes-edpm-compute-no-nodes-7bvhh\" (UID: \"6e7fa506-85c2-4b8f-a21b-1b0e195b2e8e\") " pod="openstack-kuttl-tests/neutron-dhcp-edpm-compute-no-nodes-edpm-compute-no-nodes-7bvhh" Dec 03 13:02:30 crc kubenswrapper[5052]: I1203 13:02:30.118367 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-bnsgl\" (UniqueName: \"kubernetes.io/projected/6e7fa506-85c2-4b8f-a21b-1b0e195b2e8e-kube-api-access-bnsgl\") pod \"neutron-dhcp-edpm-compute-no-nodes-edpm-compute-no-nodes-7bvhh\" (UID: \"6e7fa506-85c2-4b8f-a21b-1b0e195b2e8e\") " pod="openstack-kuttl-tests/neutron-dhcp-edpm-compute-no-nodes-edpm-compute-no-nodes-7bvhh" Dec 03 13:02:30 crc kubenswrapper[5052]: I1203 13:02:30.272089 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-dhcp-edpm-compute-no-nodes-edpm-compute-no-nodes-7bvhh" Dec 03 13:02:30 crc kubenswrapper[5052]: I1203 13:02:30.673078 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-dhcp-edpm-compute-no-nodes-edpm-compute-no-nodes-7bvhh"] Dec 03 13:02:30 crc kubenswrapper[5052]: I1203 13:02:30.899725 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-dhcp-edpm-compute-no-nodes-edpm-compute-no-nodes-7bvhh" event={"ID":"6e7fa506-85c2-4b8f-a21b-1b0e195b2e8e","Type":"ContainerStarted","Data":"1312879b6cf1f79ebcd4b2304ee4bead15d7d21bc026546d192475206e7daccd"} Dec 03 13:02:31 crc kubenswrapper[5052]: I1203 13:02:31.910126 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-dhcp-edpm-compute-no-nodes-edpm-compute-no-nodes-7bvhh" event={"ID":"6e7fa506-85c2-4b8f-a21b-1b0e195b2e8e","Type":"ContainerStarted","Data":"5ed07a085c6c74321a783b17fed82c0431e5be607bd95c4d2220c8b12554163d"} Dec 03 13:02:31 crc kubenswrapper[5052]: I1203 13:02:31.932298 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/neutron-dhcp-edpm-compute-no-nodes-edpm-compute-no-nodes-7bvhh" podStartSLOduration=2.289819639 podStartE2EDuration="2.932275191s" podCreationTimestamp="2025-12-03 13:02:29 +0000 UTC" firstStartedPulling="2025-12-03 13:02:30.677291731 +0000 UTC m=+6141.265533495" lastFinishedPulling="2025-12-03 13:02:31.319747283 +0000 UTC m=+6141.907989047" observedRunningTime="2025-12-03 13:02:31.924436527 +0000 UTC m=+6142.512678301" watchObservedRunningTime="2025-12-03 13:02:31.932275191 +0000 UTC m=+6142.520516965" Dec 03 13:02:32 crc kubenswrapper[5052]: I1203 13:02:32.930737 5052 generic.go:334] "Generic (PLEG): container finished" podID="6e7fa506-85c2-4b8f-a21b-1b0e195b2e8e" containerID="5ed07a085c6c74321a783b17fed82c0431e5be607bd95c4d2220c8b12554163d" exitCode=0 Dec 03 13:02:32 crc kubenswrapper[5052]: I1203 13:02:32.931102 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-dhcp-edpm-compute-no-nodes-edpm-compute-no-nodes-7bvhh" event={"ID":"6e7fa506-85c2-4b8f-a21b-1b0e195b2e8e","Type":"ContainerDied","Data":"5ed07a085c6c74321a783b17fed82c0431e5be607bd95c4d2220c8b12554163d"} Dec 03 13:02:34 crc kubenswrapper[5052]: I1203 13:02:34.242815 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-dhcp-edpm-compute-no-nodes-edpm-compute-no-nodes-7bvhh" Dec 03 13:02:34 crc kubenswrapper[5052]: I1203 13:02:34.357610 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/6e7fa506-85c2-4b8f-a21b-1b0e195b2e8e-inventory\") pod \"6e7fa506-85c2-4b8f-a21b-1b0e195b2e8e\" (UID: \"6e7fa506-85c2-4b8f-a21b-1b0e195b2e8e\") " Dec 03 13:02:34 crc kubenswrapper[5052]: I1203 13:02:34.357690 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"neutron-dhcp-agent-neutron-config-0\" (UniqueName: \"kubernetes.io/secret/6e7fa506-85c2-4b8f-a21b-1b0e195b2e8e-neutron-dhcp-agent-neutron-config-0\") pod \"6e7fa506-85c2-4b8f-a21b-1b0e195b2e8e\" (UID: \"6e7fa506-85c2-4b8f-a21b-1b0e195b2e8e\") " Dec 03 13:02:34 crc kubenswrapper[5052]: I1203 13:02:34.357748 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/6e7fa506-85c2-4b8f-a21b-1b0e195b2e8e-ssh-key\") pod \"6e7fa506-85c2-4b8f-a21b-1b0e195b2e8e\" (UID: \"6e7fa506-85c2-4b8f-a21b-1b0e195b2e8e\") " Dec 03 13:02:34 crc kubenswrapper[5052]: I1203 13:02:34.357796 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bnsgl\" (UniqueName: \"kubernetes.io/projected/6e7fa506-85c2-4b8f-a21b-1b0e195b2e8e-kube-api-access-bnsgl\") pod \"6e7fa506-85c2-4b8f-a21b-1b0e195b2e8e\" (UID: \"6e7fa506-85c2-4b8f-a21b-1b0e195b2e8e\") " Dec 03 13:02:34 crc kubenswrapper[5052]: I1203 13:02:34.357858 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"neutron-dhcp-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6e7fa506-85c2-4b8f-a21b-1b0e195b2e8e-neutron-dhcp-combined-ca-bundle\") pod \"6e7fa506-85c2-4b8f-a21b-1b0e195b2e8e\" (UID: \"6e7fa506-85c2-4b8f-a21b-1b0e195b2e8e\") " Dec 03 13:02:34 crc kubenswrapper[5052]: I1203 13:02:34.363061 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6e7fa506-85c2-4b8f-a21b-1b0e195b2e8e-kube-api-access-bnsgl" (OuterVolumeSpecName: "kube-api-access-bnsgl") pod "6e7fa506-85c2-4b8f-a21b-1b0e195b2e8e" (UID: "6e7fa506-85c2-4b8f-a21b-1b0e195b2e8e"). InnerVolumeSpecName "kube-api-access-bnsgl". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:02:34 crc kubenswrapper[5052]: I1203 13:02:34.363666 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6e7fa506-85c2-4b8f-a21b-1b0e195b2e8e-neutron-dhcp-combined-ca-bundle" (OuterVolumeSpecName: "neutron-dhcp-combined-ca-bundle") pod "6e7fa506-85c2-4b8f-a21b-1b0e195b2e8e" (UID: "6e7fa506-85c2-4b8f-a21b-1b0e195b2e8e"). InnerVolumeSpecName "neutron-dhcp-combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:02:34 crc kubenswrapper[5052]: I1203 13:02:34.381214 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6e7fa506-85c2-4b8f-a21b-1b0e195b2e8e-neutron-dhcp-agent-neutron-config-0" (OuterVolumeSpecName: "neutron-dhcp-agent-neutron-config-0") pod "6e7fa506-85c2-4b8f-a21b-1b0e195b2e8e" (UID: "6e7fa506-85c2-4b8f-a21b-1b0e195b2e8e"). InnerVolumeSpecName "neutron-dhcp-agent-neutron-config-0". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:02:34 crc kubenswrapper[5052]: I1203 13:02:34.382094 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6e7fa506-85c2-4b8f-a21b-1b0e195b2e8e-ssh-key" (OuterVolumeSpecName: "ssh-key") pod "6e7fa506-85c2-4b8f-a21b-1b0e195b2e8e" (UID: "6e7fa506-85c2-4b8f-a21b-1b0e195b2e8e"). InnerVolumeSpecName "ssh-key". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:02:34 crc kubenswrapper[5052]: I1203 13:02:34.383174 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6e7fa506-85c2-4b8f-a21b-1b0e195b2e8e-inventory" (OuterVolumeSpecName: "inventory") pod "6e7fa506-85c2-4b8f-a21b-1b0e195b2e8e" (UID: "6e7fa506-85c2-4b8f-a21b-1b0e195b2e8e"). InnerVolumeSpecName "inventory". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:02:34 crc kubenswrapper[5052]: I1203 13:02:34.461040 5052 reconciler_common.go:293] "Volume detached for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/6e7fa506-85c2-4b8f-a21b-1b0e195b2e8e-inventory\") on node \"crc\" DevicePath \"\"" Dec 03 13:02:34 crc kubenswrapper[5052]: I1203 13:02:34.461089 5052 reconciler_common.go:293] "Volume detached for volume \"neutron-dhcp-agent-neutron-config-0\" (UniqueName: \"kubernetes.io/secret/6e7fa506-85c2-4b8f-a21b-1b0e195b2e8e-neutron-dhcp-agent-neutron-config-0\") on node \"crc\" DevicePath \"\"" Dec 03 13:02:34 crc kubenswrapper[5052]: I1203 13:02:34.461104 5052 reconciler_common.go:293] "Volume detached for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/6e7fa506-85c2-4b8f-a21b-1b0e195b2e8e-ssh-key\") on node \"crc\" DevicePath \"\"" Dec 03 13:02:34 crc kubenswrapper[5052]: I1203 13:02:34.461120 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-bnsgl\" (UniqueName: \"kubernetes.io/projected/6e7fa506-85c2-4b8f-a21b-1b0e195b2e8e-kube-api-access-bnsgl\") on node \"crc\" DevicePath \"\"" Dec 03 13:02:34 crc kubenswrapper[5052]: I1203 13:02:34.461136 5052 reconciler_common.go:293] "Volume detached for volume \"neutron-dhcp-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6e7fa506-85c2-4b8f-a21b-1b0e195b2e8e-neutron-dhcp-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 13:02:34 crc kubenswrapper[5052]: I1203 13:02:34.971841 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-dhcp-edpm-compute-no-nodes-edpm-compute-no-nodes-7bvhh" event={"ID":"6e7fa506-85c2-4b8f-a21b-1b0e195b2e8e","Type":"ContainerDied","Data":"1312879b6cf1f79ebcd4b2304ee4bead15d7d21bc026546d192475206e7daccd"} Dec 03 13:02:34 crc kubenswrapper[5052]: I1203 13:02:34.971890 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1312879b6cf1f79ebcd4b2304ee4bead15d7d21bc026546d192475206e7daccd" Dec 03 13:02:34 crc kubenswrapper[5052]: I1203 13:02:34.971954 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-dhcp-edpm-compute-no-nodes-edpm-compute-no-nodes-7bvhh" Dec 03 13:02:35 crc kubenswrapper[5052]: I1203 13:02:35.008609 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/libvirt-edpm-compute-no-nodes-edpm-compute-no-nodes-fb8nw"] Dec 03 13:02:35 crc kubenswrapper[5052]: E1203 13:02:35.009212 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="6e7fa506-85c2-4b8f-a21b-1b0e195b2e8e" containerName="neutron-dhcp-edpm-compute-no-nodes-edpm-compute-no-nodes" Dec 03 13:02:35 crc kubenswrapper[5052]: I1203 13:02:35.009236 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="6e7fa506-85c2-4b8f-a21b-1b0e195b2e8e" containerName="neutron-dhcp-edpm-compute-no-nodes-edpm-compute-no-nodes" Dec 03 13:02:35 crc kubenswrapper[5052]: I1203 13:02:35.009409 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="6e7fa506-85c2-4b8f-a21b-1b0e195b2e8e" containerName="neutron-dhcp-edpm-compute-no-nodes-edpm-compute-no-nodes" Dec 03 13:02:35 crc kubenswrapper[5052]: I1203 13:02:35.010021 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/libvirt-edpm-compute-no-nodes-edpm-compute-no-nodes-fb8nw" Dec 03 13:02:35 crc kubenswrapper[5052]: I1203 13:02:35.012918 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"openstack-aee-default-env" Dec 03 13:02:35 crc kubenswrapper[5052]: I1203 13:02:35.013460 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"dataplanenodeset-edpm-compute-no-nodes" Dec 03 13:02:35 crc kubenswrapper[5052]: I1203 13:02:35.013646 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"edpm-compute-no-nodes-dockercfg-mkcgv" Dec 03 13:02:35 crc kubenswrapper[5052]: I1203 13:02:35.014056 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"combined-ca-bundle" Dec 03 13:02:35 crc kubenswrapper[5052]: I1203 13:02:35.014124 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"libvirt-secret" Dec 03 13:02:35 crc kubenswrapper[5052]: I1203 13:02:35.014223 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"dataplane-ansible-ssh-private-key-secret" Dec 03 13:02:35 crc kubenswrapper[5052]: I1203 13:02:35.022477 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/libvirt-edpm-compute-no-nodes-edpm-compute-no-nodes-fb8nw"] Dec 03 13:02:35 crc kubenswrapper[5052]: I1203 13:02:35.169575 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"libvirt-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/01a181ab-bf69-4454-98c5-828d1e0980c0-libvirt-combined-ca-bundle\") pod \"libvirt-edpm-compute-no-nodes-edpm-compute-no-nodes-fb8nw\" (UID: \"01a181ab-bf69-4454-98c5-828d1e0980c0\") " pod="openstack-kuttl-tests/libvirt-edpm-compute-no-nodes-edpm-compute-no-nodes-fb8nw" Dec 03 13:02:35 crc kubenswrapper[5052]: I1203 13:02:35.169711 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"libvirt-secret-0\" (UniqueName: \"kubernetes.io/secret/01a181ab-bf69-4454-98c5-828d1e0980c0-libvirt-secret-0\") pod \"libvirt-edpm-compute-no-nodes-edpm-compute-no-nodes-fb8nw\" (UID: \"01a181ab-bf69-4454-98c5-828d1e0980c0\") " pod="openstack-kuttl-tests/libvirt-edpm-compute-no-nodes-edpm-compute-no-nodes-fb8nw" Dec 03 13:02:35 crc kubenswrapper[5052]: I1203 13:02:35.169735 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/01a181ab-bf69-4454-98c5-828d1e0980c0-ssh-key\") pod \"libvirt-edpm-compute-no-nodes-edpm-compute-no-nodes-fb8nw\" (UID: \"01a181ab-bf69-4454-98c5-828d1e0980c0\") " pod="openstack-kuttl-tests/libvirt-edpm-compute-no-nodes-edpm-compute-no-nodes-fb8nw" Dec 03 13:02:35 crc kubenswrapper[5052]: I1203 13:02:35.169757 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/01a181ab-bf69-4454-98c5-828d1e0980c0-inventory\") pod \"libvirt-edpm-compute-no-nodes-edpm-compute-no-nodes-fb8nw\" (UID: \"01a181ab-bf69-4454-98c5-828d1e0980c0\") " pod="openstack-kuttl-tests/libvirt-edpm-compute-no-nodes-edpm-compute-no-nodes-fb8nw" Dec 03 13:02:35 crc kubenswrapper[5052]: I1203 13:02:35.169813 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vhktf\" (UniqueName: \"kubernetes.io/projected/01a181ab-bf69-4454-98c5-828d1e0980c0-kube-api-access-vhktf\") pod \"libvirt-edpm-compute-no-nodes-edpm-compute-no-nodes-fb8nw\" (UID: \"01a181ab-bf69-4454-98c5-828d1e0980c0\") " pod="openstack-kuttl-tests/libvirt-edpm-compute-no-nodes-edpm-compute-no-nodes-fb8nw" Dec 03 13:02:35 crc kubenswrapper[5052]: I1203 13:02:35.271258 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"libvirt-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/01a181ab-bf69-4454-98c5-828d1e0980c0-libvirt-combined-ca-bundle\") pod \"libvirt-edpm-compute-no-nodes-edpm-compute-no-nodes-fb8nw\" (UID: \"01a181ab-bf69-4454-98c5-828d1e0980c0\") " pod="openstack-kuttl-tests/libvirt-edpm-compute-no-nodes-edpm-compute-no-nodes-fb8nw" Dec 03 13:02:35 crc kubenswrapper[5052]: I1203 13:02:35.271351 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"libvirt-secret-0\" (UniqueName: \"kubernetes.io/secret/01a181ab-bf69-4454-98c5-828d1e0980c0-libvirt-secret-0\") pod \"libvirt-edpm-compute-no-nodes-edpm-compute-no-nodes-fb8nw\" (UID: \"01a181ab-bf69-4454-98c5-828d1e0980c0\") " pod="openstack-kuttl-tests/libvirt-edpm-compute-no-nodes-edpm-compute-no-nodes-fb8nw" Dec 03 13:02:35 crc kubenswrapper[5052]: I1203 13:02:35.271386 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/01a181ab-bf69-4454-98c5-828d1e0980c0-ssh-key\") pod \"libvirt-edpm-compute-no-nodes-edpm-compute-no-nodes-fb8nw\" (UID: \"01a181ab-bf69-4454-98c5-828d1e0980c0\") " pod="openstack-kuttl-tests/libvirt-edpm-compute-no-nodes-edpm-compute-no-nodes-fb8nw" Dec 03 13:02:35 crc kubenswrapper[5052]: I1203 13:02:35.271410 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/01a181ab-bf69-4454-98c5-828d1e0980c0-inventory\") pod \"libvirt-edpm-compute-no-nodes-edpm-compute-no-nodes-fb8nw\" (UID: \"01a181ab-bf69-4454-98c5-828d1e0980c0\") " pod="openstack-kuttl-tests/libvirt-edpm-compute-no-nodes-edpm-compute-no-nodes-fb8nw" Dec 03 13:02:35 crc kubenswrapper[5052]: I1203 13:02:35.271466 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-vhktf\" (UniqueName: \"kubernetes.io/projected/01a181ab-bf69-4454-98c5-828d1e0980c0-kube-api-access-vhktf\") pod \"libvirt-edpm-compute-no-nodes-edpm-compute-no-nodes-fb8nw\" (UID: \"01a181ab-bf69-4454-98c5-828d1e0980c0\") " pod="openstack-kuttl-tests/libvirt-edpm-compute-no-nodes-edpm-compute-no-nodes-fb8nw" Dec 03 13:02:35 crc kubenswrapper[5052]: I1203 13:02:35.288280 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"libvirt-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/01a181ab-bf69-4454-98c5-828d1e0980c0-libvirt-combined-ca-bundle\") pod \"libvirt-edpm-compute-no-nodes-edpm-compute-no-nodes-fb8nw\" (UID: \"01a181ab-bf69-4454-98c5-828d1e0980c0\") " pod="openstack-kuttl-tests/libvirt-edpm-compute-no-nodes-edpm-compute-no-nodes-fb8nw" Dec 03 13:02:35 crc kubenswrapper[5052]: I1203 13:02:35.288286 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"libvirt-secret-0\" (UniqueName: \"kubernetes.io/secret/01a181ab-bf69-4454-98c5-828d1e0980c0-libvirt-secret-0\") pod \"libvirt-edpm-compute-no-nodes-edpm-compute-no-nodes-fb8nw\" (UID: \"01a181ab-bf69-4454-98c5-828d1e0980c0\") " pod="openstack-kuttl-tests/libvirt-edpm-compute-no-nodes-edpm-compute-no-nodes-fb8nw" Dec 03 13:02:35 crc kubenswrapper[5052]: I1203 13:02:35.288377 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/01a181ab-bf69-4454-98c5-828d1e0980c0-inventory\") pod \"libvirt-edpm-compute-no-nodes-edpm-compute-no-nodes-fb8nw\" (UID: \"01a181ab-bf69-4454-98c5-828d1e0980c0\") " pod="openstack-kuttl-tests/libvirt-edpm-compute-no-nodes-edpm-compute-no-nodes-fb8nw" Dec 03 13:02:35 crc kubenswrapper[5052]: I1203 13:02:35.288784 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/01a181ab-bf69-4454-98c5-828d1e0980c0-ssh-key\") pod \"libvirt-edpm-compute-no-nodes-edpm-compute-no-nodes-fb8nw\" (UID: \"01a181ab-bf69-4454-98c5-828d1e0980c0\") " pod="openstack-kuttl-tests/libvirt-edpm-compute-no-nodes-edpm-compute-no-nodes-fb8nw" Dec 03 13:02:35 crc kubenswrapper[5052]: I1203 13:02:35.292265 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-vhktf\" (UniqueName: \"kubernetes.io/projected/01a181ab-bf69-4454-98c5-828d1e0980c0-kube-api-access-vhktf\") pod \"libvirt-edpm-compute-no-nodes-edpm-compute-no-nodes-fb8nw\" (UID: \"01a181ab-bf69-4454-98c5-828d1e0980c0\") " pod="openstack-kuttl-tests/libvirt-edpm-compute-no-nodes-edpm-compute-no-nodes-fb8nw" Dec 03 13:02:35 crc kubenswrapper[5052]: I1203 13:02:35.346153 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/libvirt-edpm-compute-no-nodes-edpm-compute-no-nodes-fb8nw" Dec 03 13:02:35 crc kubenswrapper[5052]: I1203 13:02:35.798975 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/libvirt-edpm-compute-no-nodes-edpm-compute-no-nodes-fb8nw"] Dec 03 13:02:35 crc kubenswrapper[5052]: I1203 13:02:35.811064 5052 provider.go:102] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Dec 03 13:02:35 crc kubenswrapper[5052]: I1203 13:02:35.985936 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/libvirt-edpm-compute-no-nodes-edpm-compute-no-nodes-fb8nw" event={"ID":"01a181ab-bf69-4454-98c5-828d1e0980c0","Type":"ContainerStarted","Data":"35ba2f4cdef512f4894d1bbab44097e936bc84ab37fe1b82bdff1dc5e1a26d87"} Dec 03 13:02:36 crc kubenswrapper[5052]: I1203 13:02:36.998462 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/libvirt-edpm-compute-no-nodes-edpm-compute-no-nodes-fb8nw" event={"ID":"01a181ab-bf69-4454-98c5-828d1e0980c0","Type":"ContainerStarted","Data":"3d0f2fc795dd0265b63af29b876dbeb43989b9b0b175be2319f1c0399846fb7e"} Dec 03 13:02:37 crc kubenswrapper[5052]: I1203 13:02:37.019060 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/libvirt-edpm-compute-no-nodes-edpm-compute-no-nodes-fb8nw" podStartSLOduration=2.5726548879999998 podStartE2EDuration="3.019044034s" podCreationTimestamp="2025-12-03 13:02:34 +0000 UTC" firstStartedPulling="2025-12-03 13:02:35.810751079 +0000 UTC m=+6146.398992843" lastFinishedPulling="2025-12-03 13:02:36.257140225 +0000 UTC m=+6146.845381989" observedRunningTime="2025-12-03 13:02:37.014303728 +0000 UTC m=+6147.602545492" watchObservedRunningTime="2025-12-03 13:02:37.019044034 +0000 UTC m=+6147.607285798" Dec 03 13:02:38 crc kubenswrapper[5052]: I1203 13:02:38.009980 5052 generic.go:334] "Generic (PLEG): container finished" podID="01a181ab-bf69-4454-98c5-828d1e0980c0" containerID="3d0f2fc795dd0265b63af29b876dbeb43989b9b0b175be2319f1c0399846fb7e" exitCode=0 Dec 03 13:02:38 crc kubenswrapper[5052]: I1203 13:02:38.010022 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/libvirt-edpm-compute-no-nodes-edpm-compute-no-nodes-fb8nw" event={"ID":"01a181ab-bf69-4454-98c5-828d1e0980c0","Type":"ContainerDied","Data":"3d0f2fc795dd0265b63af29b876dbeb43989b9b0b175be2319f1c0399846fb7e"} Dec 03 13:02:39 crc kubenswrapper[5052]: I1203 13:02:39.294970 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/libvirt-edpm-compute-no-nodes-edpm-compute-no-nodes-fb8nw" Dec 03 13:02:39 crc kubenswrapper[5052]: I1203 13:02:39.450589 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"libvirt-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/01a181ab-bf69-4454-98c5-828d1e0980c0-libvirt-combined-ca-bundle\") pod \"01a181ab-bf69-4454-98c5-828d1e0980c0\" (UID: \"01a181ab-bf69-4454-98c5-828d1e0980c0\") " Dec 03 13:02:39 crc kubenswrapper[5052]: I1203 13:02:39.450663 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/01a181ab-bf69-4454-98c5-828d1e0980c0-ssh-key\") pod \"01a181ab-bf69-4454-98c5-828d1e0980c0\" (UID: \"01a181ab-bf69-4454-98c5-828d1e0980c0\") " Dec 03 13:02:39 crc kubenswrapper[5052]: I1203 13:02:39.450711 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"libvirt-secret-0\" (UniqueName: \"kubernetes.io/secret/01a181ab-bf69-4454-98c5-828d1e0980c0-libvirt-secret-0\") pod \"01a181ab-bf69-4454-98c5-828d1e0980c0\" (UID: \"01a181ab-bf69-4454-98c5-828d1e0980c0\") " Dec 03 13:02:39 crc kubenswrapper[5052]: I1203 13:02:39.450756 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/01a181ab-bf69-4454-98c5-828d1e0980c0-inventory\") pod \"01a181ab-bf69-4454-98c5-828d1e0980c0\" (UID: \"01a181ab-bf69-4454-98c5-828d1e0980c0\") " Dec 03 13:02:39 crc kubenswrapper[5052]: I1203 13:02:39.450796 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vhktf\" (UniqueName: \"kubernetes.io/projected/01a181ab-bf69-4454-98c5-828d1e0980c0-kube-api-access-vhktf\") pod \"01a181ab-bf69-4454-98c5-828d1e0980c0\" (UID: \"01a181ab-bf69-4454-98c5-828d1e0980c0\") " Dec 03 13:02:39 crc kubenswrapper[5052]: I1203 13:02:39.457485 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/01a181ab-bf69-4454-98c5-828d1e0980c0-kube-api-access-vhktf" (OuterVolumeSpecName: "kube-api-access-vhktf") pod "01a181ab-bf69-4454-98c5-828d1e0980c0" (UID: "01a181ab-bf69-4454-98c5-828d1e0980c0"). InnerVolumeSpecName "kube-api-access-vhktf". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:02:39 crc kubenswrapper[5052]: I1203 13:02:39.461448 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/01a181ab-bf69-4454-98c5-828d1e0980c0-libvirt-combined-ca-bundle" (OuterVolumeSpecName: "libvirt-combined-ca-bundle") pod "01a181ab-bf69-4454-98c5-828d1e0980c0" (UID: "01a181ab-bf69-4454-98c5-828d1e0980c0"). InnerVolumeSpecName "libvirt-combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:02:39 crc kubenswrapper[5052]: I1203 13:02:39.477136 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/01a181ab-bf69-4454-98c5-828d1e0980c0-libvirt-secret-0" (OuterVolumeSpecName: "libvirt-secret-0") pod "01a181ab-bf69-4454-98c5-828d1e0980c0" (UID: "01a181ab-bf69-4454-98c5-828d1e0980c0"). InnerVolumeSpecName "libvirt-secret-0". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:02:39 crc kubenswrapper[5052]: I1203 13:02:39.477476 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/01a181ab-bf69-4454-98c5-828d1e0980c0-inventory" (OuterVolumeSpecName: "inventory") pod "01a181ab-bf69-4454-98c5-828d1e0980c0" (UID: "01a181ab-bf69-4454-98c5-828d1e0980c0"). InnerVolumeSpecName "inventory". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:02:39 crc kubenswrapper[5052]: I1203 13:02:39.477645 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/01a181ab-bf69-4454-98c5-828d1e0980c0-ssh-key" (OuterVolumeSpecName: "ssh-key") pod "01a181ab-bf69-4454-98c5-828d1e0980c0" (UID: "01a181ab-bf69-4454-98c5-828d1e0980c0"). InnerVolumeSpecName "ssh-key". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:02:39 crc kubenswrapper[5052]: I1203 13:02:39.553152 5052 reconciler_common.go:293] "Volume detached for volume \"libvirt-secret-0\" (UniqueName: \"kubernetes.io/secret/01a181ab-bf69-4454-98c5-828d1e0980c0-libvirt-secret-0\") on node \"crc\" DevicePath \"\"" Dec 03 13:02:39 crc kubenswrapper[5052]: I1203 13:02:39.553206 5052 reconciler_common.go:293] "Volume detached for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/01a181ab-bf69-4454-98c5-828d1e0980c0-inventory\") on node \"crc\" DevicePath \"\"" Dec 03 13:02:39 crc kubenswrapper[5052]: I1203 13:02:39.553223 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-vhktf\" (UniqueName: \"kubernetes.io/projected/01a181ab-bf69-4454-98c5-828d1e0980c0-kube-api-access-vhktf\") on node \"crc\" DevicePath \"\"" Dec 03 13:02:39 crc kubenswrapper[5052]: I1203 13:02:39.553242 5052 reconciler_common.go:293] "Volume detached for volume \"libvirt-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/01a181ab-bf69-4454-98c5-828d1e0980c0-libvirt-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 13:02:39 crc kubenswrapper[5052]: I1203 13:02:39.553255 5052 reconciler_common.go:293] "Volume detached for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/01a181ab-bf69-4454-98c5-828d1e0980c0-ssh-key\") on node \"crc\" DevicePath \"\"" Dec 03 13:02:40 crc kubenswrapper[5052]: I1203 13:02:40.028338 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/libvirt-edpm-compute-no-nodes-edpm-compute-no-nodes-fb8nw" event={"ID":"01a181ab-bf69-4454-98c5-828d1e0980c0","Type":"ContainerDied","Data":"35ba2f4cdef512f4894d1bbab44097e936bc84ab37fe1b82bdff1dc5e1a26d87"} Dec 03 13:02:40 crc kubenswrapper[5052]: I1203 13:02:40.028658 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="35ba2f4cdef512f4894d1bbab44097e936bc84ab37fe1b82bdff1dc5e1a26d87" Dec 03 13:02:40 crc kubenswrapper[5052]: I1203 13:02:40.028410 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/libvirt-edpm-compute-no-nodes-edpm-compute-no-nodes-fb8nw" Dec 03 13:02:40 crc kubenswrapper[5052]: I1203 13:02:40.152918 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-edpm-compute-no-nodes-edpm-compute-no-nodes-lnf9c"] Dec 03 13:02:40 crc kubenswrapper[5052]: E1203 13:02:40.153261 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="01a181ab-bf69-4454-98c5-828d1e0980c0" containerName="libvirt-edpm-compute-no-nodes-edpm-compute-no-nodes" Dec 03 13:02:40 crc kubenswrapper[5052]: I1203 13:02:40.153279 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="01a181ab-bf69-4454-98c5-828d1e0980c0" containerName="libvirt-edpm-compute-no-nodes-edpm-compute-no-nodes" Dec 03 13:02:40 crc kubenswrapper[5052]: I1203 13:02:40.153468 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="01a181ab-bf69-4454-98c5-828d1e0980c0" containerName="libvirt-edpm-compute-no-nodes-edpm-compute-no-nodes" Dec 03 13:02:40 crc kubenswrapper[5052]: I1203 13:02:40.153943 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-edpm-compute-no-nodes-edpm-compute-no-nodes-lnf9c" Dec 03 13:02:40 crc kubenswrapper[5052]: I1203 13:02:40.156421 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell1-compute-config" Dec 03 13:02:40 crc kubenswrapper[5052]: I1203 13:02:40.156900 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-migration-ssh-key" Dec 03 13:02:40 crc kubenswrapper[5052]: I1203 13:02:40.157060 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"openstack-aee-default-env" Dec 03 13:02:40 crc kubenswrapper[5052]: I1203 13:02:40.157086 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"dataplane-ansible-ssh-private-key-secret" Dec 03 13:02:40 crc kubenswrapper[5052]: I1203 13:02:40.157320 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"dataplanenodeset-edpm-compute-no-nodes" Dec 03 13:02:40 crc kubenswrapper[5052]: I1203 13:02:40.158629 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"edpm-compute-no-nodes-dockercfg-mkcgv" Dec 03 13:02:40 crc kubenswrapper[5052]: I1203 13:02:40.168445 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"combined-ca-bundle" Dec 03 13:02:40 crc kubenswrapper[5052]: I1203 13:02:40.169828 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-edpm-compute-no-nodes-edpm-compute-no-nodes-lnf9c"] Dec 03 13:02:40 crc kubenswrapper[5052]: I1203 13:02:40.235728 5052 scope.go:117] "RemoveContainer" containerID="593c544f232e09a05f83aba11107c032efde7fcb81fb9b1a34b1c8ba7ff35e4b" Dec 03 13:02:40 crc kubenswrapper[5052]: I1203 13:02:40.258197 5052 scope.go:117] "RemoveContainer" containerID="3b9a73789825a69ab6325c9009912283dd8eb27d7f4cafb3e836235ef7fd206c" Dec 03 13:02:40 crc kubenswrapper[5052]: I1203 13:02:40.263547 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nova-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/816cc89d-6c4d-4199-a81c-a7b29cc3f85a-nova-combined-ca-bundle\") pod \"nova-edpm-compute-no-nodes-edpm-compute-no-nodes-lnf9c\" (UID: \"816cc89d-6c4d-4199-a81c-a7b29cc3f85a\") " pod="openstack-kuttl-tests/nova-edpm-compute-no-nodes-edpm-compute-no-nodes-lnf9c" Dec 03 13:02:40 crc kubenswrapper[5052]: I1203 13:02:40.263594 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/816cc89d-6c4d-4199-a81c-a7b29cc3f85a-ssh-key\") pod \"nova-edpm-compute-no-nodes-edpm-compute-no-nodes-lnf9c\" (UID: \"816cc89d-6c4d-4199-a81c-a7b29cc3f85a\") " pod="openstack-kuttl-tests/nova-edpm-compute-no-nodes-edpm-compute-no-nodes-lnf9c" Dec 03 13:02:40 crc kubenswrapper[5052]: I1203 13:02:40.263736 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nova-migration-ssh-key-1\" (UniqueName: \"kubernetes.io/secret/816cc89d-6c4d-4199-a81c-a7b29cc3f85a-nova-migration-ssh-key-1\") pod \"nova-edpm-compute-no-nodes-edpm-compute-no-nodes-lnf9c\" (UID: \"816cc89d-6c4d-4199-a81c-a7b29cc3f85a\") " pod="openstack-kuttl-tests/nova-edpm-compute-no-nodes-edpm-compute-no-nodes-lnf9c" Dec 03 13:02:40 crc kubenswrapper[5052]: I1203 13:02:40.263876 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nova-cell1-compute-config-1\" (UniqueName: \"kubernetes.io/secret/816cc89d-6c4d-4199-a81c-a7b29cc3f85a-nova-cell1-compute-config-1\") pod \"nova-edpm-compute-no-nodes-edpm-compute-no-nodes-lnf9c\" (UID: \"816cc89d-6c4d-4199-a81c-a7b29cc3f85a\") " pod="openstack-kuttl-tests/nova-edpm-compute-no-nodes-edpm-compute-no-nodes-lnf9c" Dec 03 13:02:40 crc kubenswrapper[5052]: I1203 13:02:40.264052 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nova-cell1-compute-config-0\" (UniqueName: \"kubernetes.io/secret/816cc89d-6c4d-4199-a81c-a7b29cc3f85a-nova-cell1-compute-config-0\") pod \"nova-edpm-compute-no-nodes-edpm-compute-no-nodes-lnf9c\" (UID: \"816cc89d-6c4d-4199-a81c-a7b29cc3f85a\") " pod="openstack-kuttl-tests/nova-edpm-compute-no-nodes-edpm-compute-no-nodes-lnf9c" Dec 03 13:02:40 crc kubenswrapper[5052]: I1203 13:02:40.264106 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nova-migration-ssh-key-0\" (UniqueName: \"kubernetes.io/secret/816cc89d-6c4d-4199-a81c-a7b29cc3f85a-nova-migration-ssh-key-0\") pod \"nova-edpm-compute-no-nodes-edpm-compute-no-nodes-lnf9c\" (UID: \"816cc89d-6c4d-4199-a81c-a7b29cc3f85a\") " pod="openstack-kuttl-tests/nova-edpm-compute-no-nodes-edpm-compute-no-nodes-lnf9c" Dec 03 13:02:40 crc kubenswrapper[5052]: I1203 13:02:40.264190 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/816cc89d-6c4d-4199-a81c-a7b29cc3f85a-inventory\") pod \"nova-edpm-compute-no-nodes-edpm-compute-no-nodes-lnf9c\" (UID: \"816cc89d-6c4d-4199-a81c-a7b29cc3f85a\") " pod="openstack-kuttl-tests/nova-edpm-compute-no-nodes-edpm-compute-no-nodes-lnf9c" Dec 03 13:02:40 crc kubenswrapper[5052]: I1203 13:02:40.264284 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z2cq9\" (UniqueName: \"kubernetes.io/projected/816cc89d-6c4d-4199-a81c-a7b29cc3f85a-kube-api-access-z2cq9\") pod \"nova-edpm-compute-no-nodes-edpm-compute-no-nodes-lnf9c\" (UID: \"816cc89d-6c4d-4199-a81c-a7b29cc3f85a\") " pod="openstack-kuttl-tests/nova-edpm-compute-no-nodes-edpm-compute-no-nodes-lnf9c" Dec 03 13:02:40 crc kubenswrapper[5052]: I1203 13:02:40.290108 5052 scope.go:117] "RemoveContainer" containerID="317bade2bc973a40de5b34c60a1f485d2e6ce7d4603e6c7351dc4e69f9f7e1e2" Dec 03 13:02:40 crc kubenswrapper[5052]: I1203 13:02:40.337639 5052 scope.go:117] "RemoveContainer" containerID="31cbbc5dbfdab5d11d7e88f471e14d96eb1f12655abcc00af0fa9704571c07a9" Dec 03 13:02:40 crc kubenswrapper[5052]: I1203 13:02:40.366105 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"nova-cell1-compute-config-1\" (UniqueName: \"kubernetes.io/secret/816cc89d-6c4d-4199-a81c-a7b29cc3f85a-nova-cell1-compute-config-1\") pod \"nova-edpm-compute-no-nodes-edpm-compute-no-nodes-lnf9c\" (UID: \"816cc89d-6c4d-4199-a81c-a7b29cc3f85a\") " pod="openstack-kuttl-tests/nova-edpm-compute-no-nodes-edpm-compute-no-nodes-lnf9c" Dec 03 13:02:40 crc kubenswrapper[5052]: I1203 13:02:40.367280 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"nova-cell1-compute-config-0\" (UniqueName: \"kubernetes.io/secret/816cc89d-6c4d-4199-a81c-a7b29cc3f85a-nova-cell1-compute-config-0\") pod \"nova-edpm-compute-no-nodes-edpm-compute-no-nodes-lnf9c\" (UID: \"816cc89d-6c4d-4199-a81c-a7b29cc3f85a\") " pod="openstack-kuttl-tests/nova-edpm-compute-no-nodes-edpm-compute-no-nodes-lnf9c" Dec 03 13:02:40 crc kubenswrapper[5052]: I1203 13:02:40.367439 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"nova-migration-ssh-key-0\" (UniqueName: \"kubernetes.io/secret/816cc89d-6c4d-4199-a81c-a7b29cc3f85a-nova-migration-ssh-key-0\") pod \"nova-edpm-compute-no-nodes-edpm-compute-no-nodes-lnf9c\" (UID: \"816cc89d-6c4d-4199-a81c-a7b29cc3f85a\") " pod="openstack-kuttl-tests/nova-edpm-compute-no-nodes-edpm-compute-no-nodes-lnf9c" Dec 03 13:02:40 crc kubenswrapper[5052]: I1203 13:02:40.367662 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/816cc89d-6c4d-4199-a81c-a7b29cc3f85a-inventory\") pod \"nova-edpm-compute-no-nodes-edpm-compute-no-nodes-lnf9c\" (UID: \"816cc89d-6c4d-4199-a81c-a7b29cc3f85a\") " pod="openstack-kuttl-tests/nova-edpm-compute-no-nodes-edpm-compute-no-nodes-lnf9c" Dec 03 13:02:40 crc kubenswrapper[5052]: I1203 13:02:40.367823 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-z2cq9\" (UniqueName: \"kubernetes.io/projected/816cc89d-6c4d-4199-a81c-a7b29cc3f85a-kube-api-access-z2cq9\") pod \"nova-edpm-compute-no-nodes-edpm-compute-no-nodes-lnf9c\" (UID: \"816cc89d-6c4d-4199-a81c-a7b29cc3f85a\") " pod="openstack-kuttl-tests/nova-edpm-compute-no-nodes-edpm-compute-no-nodes-lnf9c" Dec 03 13:02:40 crc kubenswrapper[5052]: I1203 13:02:40.367957 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"nova-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/816cc89d-6c4d-4199-a81c-a7b29cc3f85a-nova-combined-ca-bundle\") pod \"nova-edpm-compute-no-nodes-edpm-compute-no-nodes-lnf9c\" (UID: \"816cc89d-6c4d-4199-a81c-a7b29cc3f85a\") " pod="openstack-kuttl-tests/nova-edpm-compute-no-nodes-edpm-compute-no-nodes-lnf9c" Dec 03 13:02:40 crc kubenswrapper[5052]: I1203 13:02:40.368081 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/816cc89d-6c4d-4199-a81c-a7b29cc3f85a-ssh-key\") pod \"nova-edpm-compute-no-nodes-edpm-compute-no-nodes-lnf9c\" (UID: \"816cc89d-6c4d-4199-a81c-a7b29cc3f85a\") " pod="openstack-kuttl-tests/nova-edpm-compute-no-nodes-edpm-compute-no-nodes-lnf9c" Dec 03 13:02:40 crc kubenswrapper[5052]: I1203 13:02:40.368207 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"nova-migration-ssh-key-1\" (UniqueName: \"kubernetes.io/secret/816cc89d-6c4d-4199-a81c-a7b29cc3f85a-nova-migration-ssh-key-1\") pod \"nova-edpm-compute-no-nodes-edpm-compute-no-nodes-lnf9c\" (UID: \"816cc89d-6c4d-4199-a81c-a7b29cc3f85a\") " pod="openstack-kuttl-tests/nova-edpm-compute-no-nodes-edpm-compute-no-nodes-lnf9c" Dec 03 13:02:40 crc kubenswrapper[5052]: I1203 13:02:40.369997 5052 scope.go:117] "RemoveContainer" containerID="483ab837917d2bb0cee2cc97f9b75239ff0e75f486cdf8e5b738db16fa469c02" Dec 03 13:02:40 crc kubenswrapper[5052]: I1203 13:02:40.371145 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"nova-migration-ssh-key-0\" (UniqueName: \"kubernetes.io/secret/816cc89d-6c4d-4199-a81c-a7b29cc3f85a-nova-migration-ssh-key-0\") pod \"nova-edpm-compute-no-nodes-edpm-compute-no-nodes-lnf9c\" (UID: \"816cc89d-6c4d-4199-a81c-a7b29cc3f85a\") " pod="openstack-kuttl-tests/nova-edpm-compute-no-nodes-edpm-compute-no-nodes-lnf9c" Dec 03 13:02:40 crc kubenswrapper[5052]: I1203 13:02:40.371629 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"nova-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/816cc89d-6c4d-4199-a81c-a7b29cc3f85a-nova-combined-ca-bundle\") pod \"nova-edpm-compute-no-nodes-edpm-compute-no-nodes-lnf9c\" (UID: \"816cc89d-6c4d-4199-a81c-a7b29cc3f85a\") " pod="openstack-kuttl-tests/nova-edpm-compute-no-nodes-edpm-compute-no-nodes-lnf9c" Dec 03 13:02:40 crc kubenswrapper[5052]: I1203 13:02:40.372101 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"nova-cell1-compute-config-1\" (UniqueName: \"kubernetes.io/secret/816cc89d-6c4d-4199-a81c-a7b29cc3f85a-nova-cell1-compute-config-1\") pod \"nova-edpm-compute-no-nodes-edpm-compute-no-nodes-lnf9c\" (UID: \"816cc89d-6c4d-4199-a81c-a7b29cc3f85a\") " pod="openstack-kuttl-tests/nova-edpm-compute-no-nodes-edpm-compute-no-nodes-lnf9c" Dec 03 13:02:40 crc kubenswrapper[5052]: I1203 13:02:40.372521 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"nova-migration-ssh-key-1\" (UniqueName: \"kubernetes.io/secret/816cc89d-6c4d-4199-a81c-a7b29cc3f85a-nova-migration-ssh-key-1\") pod \"nova-edpm-compute-no-nodes-edpm-compute-no-nodes-lnf9c\" (UID: \"816cc89d-6c4d-4199-a81c-a7b29cc3f85a\") " pod="openstack-kuttl-tests/nova-edpm-compute-no-nodes-edpm-compute-no-nodes-lnf9c" Dec 03 13:02:40 crc kubenswrapper[5052]: I1203 13:02:40.372938 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"nova-cell1-compute-config-0\" (UniqueName: \"kubernetes.io/secret/816cc89d-6c4d-4199-a81c-a7b29cc3f85a-nova-cell1-compute-config-0\") pod \"nova-edpm-compute-no-nodes-edpm-compute-no-nodes-lnf9c\" (UID: \"816cc89d-6c4d-4199-a81c-a7b29cc3f85a\") " pod="openstack-kuttl-tests/nova-edpm-compute-no-nodes-edpm-compute-no-nodes-lnf9c" Dec 03 13:02:40 crc kubenswrapper[5052]: I1203 13:02:40.373328 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/816cc89d-6c4d-4199-a81c-a7b29cc3f85a-inventory\") pod \"nova-edpm-compute-no-nodes-edpm-compute-no-nodes-lnf9c\" (UID: \"816cc89d-6c4d-4199-a81c-a7b29cc3f85a\") " pod="openstack-kuttl-tests/nova-edpm-compute-no-nodes-edpm-compute-no-nodes-lnf9c" Dec 03 13:02:40 crc kubenswrapper[5052]: I1203 13:02:40.374226 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/816cc89d-6c4d-4199-a81c-a7b29cc3f85a-ssh-key\") pod \"nova-edpm-compute-no-nodes-edpm-compute-no-nodes-lnf9c\" (UID: \"816cc89d-6c4d-4199-a81c-a7b29cc3f85a\") " pod="openstack-kuttl-tests/nova-edpm-compute-no-nodes-edpm-compute-no-nodes-lnf9c" Dec 03 13:02:40 crc kubenswrapper[5052]: I1203 13:02:40.386341 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-z2cq9\" (UniqueName: \"kubernetes.io/projected/816cc89d-6c4d-4199-a81c-a7b29cc3f85a-kube-api-access-z2cq9\") pod \"nova-edpm-compute-no-nodes-edpm-compute-no-nodes-lnf9c\" (UID: \"816cc89d-6c4d-4199-a81c-a7b29cc3f85a\") " pod="openstack-kuttl-tests/nova-edpm-compute-no-nodes-edpm-compute-no-nodes-lnf9c" Dec 03 13:02:40 crc kubenswrapper[5052]: I1203 13:02:40.470445 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-edpm-compute-no-nodes-edpm-compute-no-nodes-lnf9c" Dec 03 13:02:40 crc kubenswrapper[5052]: I1203 13:02:40.902032 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-edpm-compute-no-nodes-edpm-compute-no-nodes-lnf9c"] Dec 03 13:02:41 crc kubenswrapper[5052]: I1203 13:02:41.039234 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-edpm-compute-no-nodes-edpm-compute-no-nodes-lnf9c" event={"ID":"816cc89d-6c4d-4199-a81c-a7b29cc3f85a","Type":"ContainerStarted","Data":"3d540a4419c7ad22e68174b384a91f99b9a0fa4a28c97ca3031ac11df26b168f"} Dec 03 13:02:43 crc kubenswrapper[5052]: I1203 13:02:43.059746 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-edpm-compute-no-nodes-edpm-compute-no-nodes-lnf9c" event={"ID":"816cc89d-6c4d-4199-a81c-a7b29cc3f85a","Type":"ContainerStarted","Data":"7590a7faf49819a0d4e82cdb779be314368b5b03ac7d8c6d4aabf90175203276"} Dec 03 13:02:43 crc kubenswrapper[5052]: I1203 13:02:43.081733 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-edpm-compute-no-nodes-edpm-compute-no-nodes-lnf9c" podStartSLOduration=1.685825394 podStartE2EDuration="3.081715994s" podCreationTimestamp="2025-12-03 13:02:40 +0000 UTC" firstStartedPulling="2025-12-03 13:02:40.906275521 +0000 UTC m=+6151.494517285" lastFinishedPulling="2025-12-03 13:02:42.302166111 +0000 UTC m=+6152.890407885" observedRunningTime="2025-12-03 13:02:43.077484143 +0000 UTC m=+6153.665725927" watchObservedRunningTime="2025-12-03 13:02:43.081715994 +0000 UTC m=+6153.669957758" Dec 03 13:02:44 crc kubenswrapper[5052]: I1203 13:02:44.069927 5052 generic.go:334] "Generic (PLEG): container finished" podID="816cc89d-6c4d-4199-a81c-a7b29cc3f85a" containerID="7590a7faf49819a0d4e82cdb779be314368b5b03ac7d8c6d4aabf90175203276" exitCode=0 Dec 03 13:02:44 crc kubenswrapper[5052]: I1203 13:02:44.069978 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-edpm-compute-no-nodes-edpm-compute-no-nodes-lnf9c" event={"ID":"816cc89d-6c4d-4199-a81c-a7b29cc3f85a","Type":"ContainerDied","Data":"7590a7faf49819a0d4e82cdb779be314368b5b03ac7d8c6d4aabf90175203276"} Dec 03 13:02:45 crc kubenswrapper[5052]: I1203 13:02:45.395697 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-edpm-compute-no-nodes-edpm-compute-no-nodes-lnf9c" Dec 03 13:02:45 crc kubenswrapper[5052]: I1203 13:02:45.547669 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"nova-migration-ssh-key-1\" (UniqueName: \"kubernetes.io/secret/816cc89d-6c4d-4199-a81c-a7b29cc3f85a-nova-migration-ssh-key-1\") pod \"816cc89d-6c4d-4199-a81c-a7b29cc3f85a\" (UID: \"816cc89d-6c4d-4199-a81c-a7b29cc3f85a\") " Dec 03 13:02:45 crc kubenswrapper[5052]: I1203 13:02:45.547771 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/816cc89d-6c4d-4199-a81c-a7b29cc3f85a-ssh-key\") pod \"816cc89d-6c4d-4199-a81c-a7b29cc3f85a\" (UID: \"816cc89d-6c4d-4199-a81c-a7b29cc3f85a\") " Dec 03 13:02:45 crc kubenswrapper[5052]: I1203 13:02:45.547801 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z2cq9\" (UniqueName: \"kubernetes.io/projected/816cc89d-6c4d-4199-a81c-a7b29cc3f85a-kube-api-access-z2cq9\") pod \"816cc89d-6c4d-4199-a81c-a7b29cc3f85a\" (UID: \"816cc89d-6c4d-4199-a81c-a7b29cc3f85a\") " Dec 03 13:02:45 crc kubenswrapper[5052]: I1203 13:02:45.547829 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"nova-migration-ssh-key-0\" (UniqueName: \"kubernetes.io/secret/816cc89d-6c4d-4199-a81c-a7b29cc3f85a-nova-migration-ssh-key-0\") pod \"816cc89d-6c4d-4199-a81c-a7b29cc3f85a\" (UID: \"816cc89d-6c4d-4199-a81c-a7b29cc3f85a\") " Dec 03 13:02:45 crc kubenswrapper[5052]: I1203 13:02:45.547911 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/816cc89d-6c4d-4199-a81c-a7b29cc3f85a-inventory\") pod \"816cc89d-6c4d-4199-a81c-a7b29cc3f85a\" (UID: \"816cc89d-6c4d-4199-a81c-a7b29cc3f85a\") " Dec 03 13:02:45 crc kubenswrapper[5052]: I1203 13:02:45.547944 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"nova-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/816cc89d-6c4d-4199-a81c-a7b29cc3f85a-nova-combined-ca-bundle\") pod \"816cc89d-6c4d-4199-a81c-a7b29cc3f85a\" (UID: \"816cc89d-6c4d-4199-a81c-a7b29cc3f85a\") " Dec 03 13:02:45 crc kubenswrapper[5052]: I1203 13:02:45.547987 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"nova-cell1-compute-config-1\" (UniqueName: \"kubernetes.io/secret/816cc89d-6c4d-4199-a81c-a7b29cc3f85a-nova-cell1-compute-config-1\") pod \"816cc89d-6c4d-4199-a81c-a7b29cc3f85a\" (UID: \"816cc89d-6c4d-4199-a81c-a7b29cc3f85a\") " Dec 03 13:02:45 crc kubenswrapper[5052]: I1203 13:02:45.548035 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"nova-cell1-compute-config-0\" (UniqueName: \"kubernetes.io/secret/816cc89d-6c4d-4199-a81c-a7b29cc3f85a-nova-cell1-compute-config-0\") pod \"816cc89d-6c4d-4199-a81c-a7b29cc3f85a\" (UID: \"816cc89d-6c4d-4199-a81c-a7b29cc3f85a\") " Dec 03 13:02:45 crc kubenswrapper[5052]: I1203 13:02:45.552863 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/816cc89d-6c4d-4199-a81c-a7b29cc3f85a-nova-combined-ca-bundle" (OuterVolumeSpecName: "nova-combined-ca-bundle") pod "816cc89d-6c4d-4199-a81c-a7b29cc3f85a" (UID: "816cc89d-6c4d-4199-a81c-a7b29cc3f85a"). InnerVolumeSpecName "nova-combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:02:45 crc kubenswrapper[5052]: I1203 13:02:45.553555 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/816cc89d-6c4d-4199-a81c-a7b29cc3f85a-kube-api-access-z2cq9" (OuterVolumeSpecName: "kube-api-access-z2cq9") pod "816cc89d-6c4d-4199-a81c-a7b29cc3f85a" (UID: "816cc89d-6c4d-4199-a81c-a7b29cc3f85a"). InnerVolumeSpecName "kube-api-access-z2cq9". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:02:45 crc kubenswrapper[5052]: I1203 13:02:45.588653 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/816cc89d-6c4d-4199-a81c-a7b29cc3f85a-nova-migration-ssh-key-1" (OuterVolumeSpecName: "nova-migration-ssh-key-1") pod "816cc89d-6c4d-4199-a81c-a7b29cc3f85a" (UID: "816cc89d-6c4d-4199-a81c-a7b29cc3f85a"). InnerVolumeSpecName "nova-migration-ssh-key-1". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:02:45 crc kubenswrapper[5052]: I1203 13:02:45.595585 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/816cc89d-6c4d-4199-a81c-a7b29cc3f85a-nova-migration-ssh-key-0" (OuterVolumeSpecName: "nova-migration-ssh-key-0") pod "816cc89d-6c4d-4199-a81c-a7b29cc3f85a" (UID: "816cc89d-6c4d-4199-a81c-a7b29cc3f85a"). InnerVolumeSpecName "nova-migration-ssh-key-0". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:02:45 crc kubenswrapper[5052]: I1203 13:02:45.623677 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/816cc89d-6c4d-4199-a81c-a7b29cc3f85a-inventory" (OuterVolumeSpecName: "inventory") pod "816cc89d-6c4d-4199-a81c-a7b29cc3f85a" (UID: "816cc89d-6c4d-4199-a81c-a7b29cc3f85a"). InnerVolumeSpecName "inventory". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:02:45 crc kubenswrapper[5052]: I1203 13:02:45.624672 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/816cc89d-6c4d-4199-a81c-a7b29cc3f85a-nova-cell1-compute-config-0" (OuterVolumeSpecName: "nova-cell1-compute-config-0") pod "816cc89d-6c4d-4199-a81c-a7b29cc3f85a" (UID: "816cc89d-6c4d-4199-a81c-a7b29cc3f85a"). InnerVolumeSpecName "nova-cell1-compute-config-0". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:02:45 crc kubenswrapper[5052]: I1203 13:02:45.640659 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/816cc89d-6c4d-4199-a81c-a7b29cc3f85a-ssh-key" (OuterVolumeSpecName: "ssh-key") pod "816cc89d-6c4d-4199-a81c-a7b29cc3f85a" (UID: "816cc89d-6c4d-4199-a81c-a7b29cc3f85a"). InnerVolumeSpecName "ssh-key". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:02:45 crc kubenswrapper[5052]: I1203 13:02:45.645933 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/816cc89d-6c4d-4199-a81c-a7b29cc3f85a-nova-cell1-compute-config-1" (OuterVolumeSpecName: "nova-cell1-compute-config-1") pod "816cc89d-6c4d-4199-a81c-a7b29cc3f85a" (UID: "816cc89d-6c4d-4199-a81c-a7b29cc3f85a"). InnerVolumeSpecName "nova-cell1-compute-config-1". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:02:45 crc kubenswrapper[5052]: I1203 13:02:45.658862 5052 reconciler_common.go:293] "Volume detached for volume \"nova-cell1-compute-config-1\" (UniqueName: \"kubernetes.io/secret/816cc89d-6c4d-4199-a81c-a7b29cc3f85a-nova-cell1-compute-config-1\") on node \"crc\" DevicePath \"\"" Dec 03 13:02:45 crc kubenswrapper[5052]: I1203 13:02:45.658906 5052 reconciler_common.go:293] "Volume detached for volume \"nova-cell1-compute-config-0\" (UniqueName: \"kubernetes.io/secret/816cc89d-6c4d-4199-a81c-a7b29cc3f85a-nova-cell1-compute-config-0\") on node \"crc\" DevicePath \"\"" Dec 03 13:02:45 crc kubenswrapper[5052]: I1203 13:02:45.658922 5052 reconciler_common.go:293] "Volume detached for volume \"nova-migration-ssh-key-1\" (UniqueName: \"kubernetes.io/secret/816cc89d-6c4d-4199-a81c-a7b29cc3f85a-nova-migration-ssh-key-1\") on node \"crc\" DevicePath \"\"" Dec 03 13:02:45 crc kubenswrapper[5052]: I1203 13:02:45.658934 5052 reconciler_common.go:293] "Volume detached for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/816cc89d-6c4d-4199-a81c-a7b29cc3f85a-ssh-key\") on node \"crc\" DevicePath \"\"" Dec 03 13:02:45 crc kubenswrapper[5052]: I1203 13:02:45.658945 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-z2cq9\" (UniqueName: \"kubernetes.io/projected/816cc89d-6c4d-4199-a81c-a7b29cc3f85a-kube-api-access-z2cq9\") on node \"crc\" DevicePath \"\"" Dec 03 13:02:45 crc kubenswrapper[5052]: I1203 13:02:45.658956 5052 reconciler_common.go:293] "Volume detached for volume \"nova-migration-ssh-key-0\" (UniqueName: \"kubernetes.io/secret/816cc89d-6c4d-4199-a81c-a7b29cc3f85a-nova-migration-ssh-key-0\") on node \"crc\" DevicePath \"\"" Dec 03 13:02:45 crc kubenswrapper[5052]: I1203 13:02:45.658969 5052 reconciler_common.go:293] "Volume detached for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/816cc89d-6c4d-4199-a81c-a7b29cc3f85a-inventory\") on node \"crc\" DevicePath \"\"" Dec 03 13:02:45 crc kubenswrapper[5052]: I1203 13:02:45.658980 5052 reconciler_common.go:293] "Volume detached for volume \"nova-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/816cc89d-6c4d-4199-a81c-a7b29cc3f85a-nova-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 13:02:46 crc kubenswrapper[5052]: I1203 13:02:46.105745 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-edpm-compute-no-nodes-edpm-compute-no-nodes-lnf9c" event={"ID":"816cc89d-6c4d-4199-a81c-a7b29cc3f85a","Type":"ContainerDied","Data":"3d540a4419c7ad22e68174b384a91f99b9a0fa4a28c97ca3031ac11df26b168f"} Dec 03 13:02:46 crc kubenswrapper[5052]: I1203 13:02:46.105795 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3d540a4419c7ad22e68174b384a91f99b9a0fa4a28c97ca3031ac11df26b168f" Dec 03 13:02:46 crc kubenswrapper[5052]: I1203 13:02:46.105880 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-edpm-compute-no-nodes-edpm-compute-no-nodes-lnf9c" Dec 03 13:02:47 crc kubenswrapper[5052]: I1203 13:02:47.123155 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/custom-svc-edpm-compute-no-nodes-ovrd-edpm-compute-no-noded8jt2"] Dec 03 13:02:47 crc kubenswrapper[5052]: E1203 13:02:47.123483 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="816cc89d-6c4d-4199-a81c-a7b29cc3f85a" containerName="nova-edpm-compute-no-nodes-edpm-compute-no-nodes" Dec 03 13:02:47 crc kubenswrapper[5052]: I1203 13:02:47.123499 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="816cc89d-6c4d-4199-a81c-a7b29cc3f85a" containerName="nova-edpm-compute-no-nodes-edpm-compute-no-nodes" Dec 03 13:02:47 crc kubenswrapper[5052]: I1203 13:02:47.123663 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="816cc89d-6c4d-4199-a81c-a7b29cc3f85a" containerName="nova-edpm-compute-no-nodes-edpm-compute-no-nodes" Dec 03 13:02:47 crc kubenswrapper[5052]: I1203 13:02:47.124168 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/custom-svc-edpm-compute-no-nodes-ovrd-edpm-compute-no-noded8jt2" Dec 03 13:02:47 crc kubenswrapper[5052]: I1203 13:02:47.127488 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"openstack-aee-default-env" Dec 03 13:02:47 crc kubenswrapper[5052]: I1203 13:02:47.127501 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"edpm-compute-no-nodes-dockercfg-mkcgv" Dec 03 13:02:47 crc kubenswrapper[5052]: I1203 13:02:47.128419 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"dataplane-ansible-ssh-private-key-secret" Dec 03 13:02:47 crc kubenswrapper[5052]: I1203 13:02:47.128697 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"dataplanenodeset-edpm-compute-no-nodes" Dec 03 13:02:47 crc kubenswrapper[5052]: I1203 13:02:47.136268 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"combined-ca-bundle" Dec 03 13:02:47 crc kubenswrapper[5052]: I1203 13:02:47.136749 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/custom-svc-edpm-compute-no-nodes-ovrd-edpm-compute-no-noded8jt2"] Dec 03 13:02:47 crc kubenswrapper[5052]: I1203 13:02:47.283830 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"custom-svc-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/951b6922-5edc-46f6-8a69-929e7517850c-custom-svc-combined-ca-bundle\") pod \"custom-svc-edpm-compute-no-nodes-ovrd-edpm-compute-no-noded8jt2\" (UID: \"951b6922-5edc-46f6-8a69-929e7517850c\") " pod="openstack-kuttl-tests/custom-svc-edpm-compute-no-nodes-ovrd-edpm-compute-no-noded8jt2" Dec 03 13:02:47 crc kubenswrapper[5052]: I1203 13:02:47.283933 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/951b6922-5edc-46f6-8a69-929e7517850c-ssh-key\") pod \"custom-svc-edpm-compute-no-nodes-ovrd-edpm-compute-no-noded8jt2\" (UID: \"951b6922-5edc-46f6-8a69-929e7517850c\") " pod="openstack-kuttl-tests/custom-svc-edpm-compute-no-nodes-ovrd-edpm-compute-no-noded8jt2" Dec 03 13:02:47 crc kubenswrapper[5052]: I1203 13:02:47.283971 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pfj9v\" (UniqueName: \"kubernetes.io/projected/951b6922-5edc-46f6-8a69-929e7517850c-kube-api-access-pfj9v\") pod \"custom-svc-edpm-compute-no-nodes-ovrd-edpm-compute-no-noded8jt2\" (UID: \"951b6922-5edc-46f6-8a69-929e7517850c\") " pod="openstack-kuttl-tests/custom-svc-edpm-compute-no-nodes-ovrd-edpm-compute-no-noded8jt2" Dec 03 13:02:47 crc kubenswrapper[5052]: I1203 13:02:47.284077 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/951b6922-5edc-46f6-8a69-929e7517850c-inventory\") pod \"custom-svc-edpm-compute-no-nodes-ovrd-edpm-compute-no-noded8jt2\" (UID: \"951b6922-5edc-46f6-8a69-929e7517850c\") " pod="openstack-kuttl-tests/custom-svc-edpm-compute-no-nodes-ovrd-edpm-compute-no-noded8jt2" Dec 03 13:02:47 crc kubenswrapper[5052]: I1203 13:02:47.385365 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"custom-svc-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/951b6922-5edc-46f6-8a69-929e7517850c-custom-svc-combined-ca-bundle\") pod \"custom-svc-edpm-compute-no-nodes-ovrd-edpm-compute-no-noded8jt2\" (UID: \"951b6922-5edc-46f6-8a69-929e7517850c\") " pod="openstack-kuttl-tests/custom-svc-edpm-compute-no-nodes-ovrd-edpm-compute-no-noded8jt2" Dec 03 13:02:47 crc kubenswrapper[5052]: I1203 13:02:47.385464 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/951b6922-5edc-46f6-8a69-929e7517850c-ssh-key\") pod \"custom-svc-edpm-compute-no-nodes-ovrd-edpm-compute-no-noded8jt2\" (UID: \"951b6922-5edc-46f6-8a69-929e7517850c\") " pod="openstack-kuttl-tests/custom-svc-edpm-compute-no-nodes-ovrd-edpm-compute-no-noded8jt2" Dec 03 13:02:47 crc kubenswrapper[5052]: I1203 13:02:47.385503 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-pfj9v\" (UniqueName: \"kubernetes.io/projected/951b6922-5edc-46f6-8a69-929e7517850c-kube-api-access-pfj9v\") pod \"custom-svc-edpm-compute-no-nodes-ovrd-edpm-compute-no-noded8jt2\" (UID: \"951b6922-5edc-46f6-8a69-929e7517850c\") " pod="openstack-kuttl-tests/custom-svc-edpm-compute-no-nodes-ovrd-edpm-compute-no-noded8jt2" Dec 03 13:02:47 crc kubenswrapper[5052]: I1203 13:02:47.385564 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/951b6922-5edc-46f6-8a69-929e7517850c-inventory\") pod \"custom-svc-edpm-compute-no-nodes-ovrd-edpm-compute-no-noded8jt2\" (UID: \"951b6922-5edc-46f6-8a69-929e7517850c\") " pod="openstack-kuttl-tests/custom-svc-edpm-compute-no-nodes-ovrd-edpm-compute-no-noded8jt2" Dec 03 13:02:47 crc kubenswrapper[5052]: I1203 13:02:47.389503 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/951b6922-5edc-46f6-8a69-929e7517850c-inventory\") pod \"custom-svc-edpm-compute-no-nodes-ovrd-edpm-compute-no-noded8jt2\" (UID: \"951b6922-5edc-46f6-8a69-929e7517850c\") " pod="openstack-kuttl-tests/custom-svc-edpm-compute-no-nodes-ovrd-edpm-compute-no-noded8jt2" Dec 03 13:02:47 crc kubenswrapper[5052]: I1203 13:02:47.389678 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/951b6922-5edc-46f6-8a69-929e7517850c-ssh-key\") pod \"custom-svc-edpm-compute-no-nodes-ovrd-edpm-compute-no-noded8jt2\" (UID: \"951b6922-5edc-46f6-8a69-929e7517850c\") " pod="openstack-kuttl-tests/custom-svc-edpm-compute-no-nodes-ovrd-edpm-compute-no-noded8jt2" Dec 03 13:02:47 crc kubenswrapper[5052]: I1203 13:02:47.391300 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"custom-svc-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/951b6922-5edc-46f6-8a69-929e7517850c-custom-svc-combined-ca-bundle\") pod \"custom-svc-edpm-compute-no-nodes-ovrd-edpm-compute-no-noded8jt2\" (UID: \"951b6922-5edc-46f6-8a69-929e7517850c\") " pod="openstack-kuttl-tests/custom-svc-edpm-compute-no-nodes-ovrd-edpm-compute-no-noded8jt2" Dec 03 13:02:47 crc kubenswrapper[5052]: I1203 13:02:47.404339 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-pfj9v\" (UniqueName: \"kubernetes.io/projected/951b6922-5edc-46f6-8a69-929e7517850c-kube-api-access-pfj9v\") pod \"custom-svc-edpm-compute-no-nodes-ovrd-edpm-compute-no-noded8jt2\" (UID: \"951b6922-5edc-46f6-8a69-929e7517850c\") " pod="openstack-kuttl-tests/custom-svc-edpm-compute-no-nodes-ovrd-edpm-compute-no-noded8jt2" Dec 03 13:02:47 crc kubenswrapper[5052]: I1203 13:02:47.441284 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/custom-svc-edpm-compute-no-nodes-ovrd-edpm-compute-no-noded8jt2" Dec 03 13:02:47 crc kubenswrapper[5052]: I1203 13:02:47.855374 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/custom-svc-edpm-compute-no-nodes-ovrd-edpm-compute-no-noded8jt2"] Dec 03 13:02:48 crc kubenswrapper[5052]: I1203 13:02:48.124413 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/custom-svc-edpm-compute-no-nodes-ovrd-edpm-compute-no-noded8jt2" event={"ID":"951b6922-5edc-46f6-8a69-929e7517850c","Type":"ContainerStarted","Data":"4ac4c3d116bb3cefd5635b1e9bf087047d2305acc436a798e60e136ef90e7ab6"} Dec 03 13:02:49 crc kubenswrapper[5052]: I1203 13:02:49.136800 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/custom-svc-edpm-compute-no-nodes-ovrd-edpm-compute-no-noded8jt2" event={"ID":"951b6922-5edc-46f6-8a69-929e7517850c","Type":"ContainerStarted","Data":"74675321570d68686f0bf483f4f778e02335c2f21afd4d74029a906af20aca11"} Dec 03 13:02:49 crc kubenswrapper[5052]: I1203 13:02:49.157544 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/custom-svc-edpm-compute-no-nodes-ovrd-edpm-compute-no-noded8jt2" podStartSLOduration=1.7362321939999998 podStartE2EDuration="2.157520392s" podCreationTimestamp="2025-12-03 13:02:47 +0000 UTC" firstStartedPulling="2025-12-03 13:02:47.860371625 +0000 UTC m=+6158.448613389" lastFinishedPulling="2025-12-03 13:02:48.281659823 +0000 UTC m=+6158.869901587" observedRunningTime="2025-12-03 13:02:49.153445605 +0000 UTC m=+6159.741687369" watchObservedRunningTime="2025-12-03 13:02:49.157520392 +0000 UTC m=+6159.745762146" Dec 03 13:02:51 crc kubenswrapper[5052]: I1203 13:02:51.155073 5052 generic.go:334] "Generic (PLEG): container finished" podID="951b6922-5edc-46f6-8a69-929e7517850c" containerID="74675321570d68686f0bf483f4f778e02335c2f21afd4d74029a906af20aca11" exitCode=0 Dec 03 13:02:51 crc kubenswrapper[5052]: I1203 13:02:51.155137 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/custom-svc-edpm-compute-no-nodes-ovrd-edpm-compute-no-noded8jt2" event={"ID":"951b6922-5edc-46f6-8a69-929e7517850c","Type":"ContainerDied","Data":"74675321570d68686f0bf483f4f778e02335c2f21afd4d74029a906af20aca11"} Dec 03 13:02:52 crc kubenswrapper[5052]: I1203 13:02:52.448059 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/custom-svc-edpm-compute-no-nodes-ovrd-edpm-compute-no-noded8jt2" Dec 03 13:02:52 crc kubenswrapper[5052]: I1203 13:02:52.566094 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/951b6922-5edc-46f6-8a69-929e7517850c-ssh-key\") pod \"951b6922-5edc-46f6-8a69-929e7517850c\" (UID: \"951b6922-5edc-46f6-8a69-929e7517850c\") " Dec 03 13:02:52 crc kubenswrapper[5052]: I1203 13:02:52.566208 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"custom-svc-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/951b6922-5edc-46f6-8a69-929e7517850c-custom-svc-combined-ca-bundle\") pod \"951b6922-5edc-46f6-8a69-929e7517850c\" (UID: \"951b6922-5edc-46f6-8a69-929e7517850c\") " Dec 03 13:02:52 crc kubenswrapper[5052]: I1203 13:02:52.566243 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pfj9v\" (UniqueName: \"kubernetes.io/projected/951b6922-5edc-46f6-8a69-929e7517850c-kube-api-access-pfj9v\") pod \"951b6922-5edc-46f6-8a69-929e7517850c\" (UID: \"951b6922-5edc-46f6-8a69-929e7517850c\") " Dec 03 13:02:52 crc kubenswrapper[5052]: I1203 13:02:52.566294 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/951b6922-5edc-46f6-8a69-929e7517850c-inventory\") pod \"951b6922-5edc-46f6-8a69-929e7517850c\" (UID: \"951b6922-5edc-46f6-8a69-929e7517850c\") " Dec 03 13:02:52 crc kubenswrapper[5052]: I1203 13:02:52.577954 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/951b6922-5edc-46f6-8a69-929e7517850c-custom-svc-combined-ca-bundle" (OuterVolumeSpecName: "custom-svc-combined-ca-bundle") pod "951b6922-5edc-46f6-8a69-929e7517850c" (UID: "951b6922-5edc-46f6-8a69-929e7517850c"). InnerVolumeSpecName "custom-svc-combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:02:52 crc kubenswrapper[5052]: I1203 13:02:52.592537 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/951b6922-5edc-46f6-8a69-929e7517850c-kube-api-access-pfj9v" (OuterVolumeSpecName: "kube-api-access-pfj9v") pod "951b6922-5edc-46f6-8a69-929e7517850c" (UID: "951b6922-5edc-46f6-8a69-929e7517850c"). InnerVolumeSpecName "kube-api-access-pfj9v". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:02:52 crc kubenswrapper[5052]: I1203 13:02:52.601647 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/951b6922-5edc-46f6-8a69-929e7517850c-inventory" (OuterVolumeSpecName: "inventory") pod "951b6922-5edc-46f6-8a69-929e7517850c" (UID: "951b6922-5edc-46f6-8a69-929e7517850c"). InnerVolumeSpecName "inventory". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:02:52 crc kubenswrapper[5052]: I1203 13:02:52.657784 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/951b6922-5edc-46f6-8a69-929e7517850c-ssh-key" (OuterVolumeSpecName: "ssh-key") pod "951b6922-5edc-46f6-8a69-929e7517850c" (UID: "951b6922-5edc-46f6-8a69-929e7517850c"). InnerVolumeSpecName "ssh-key". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:02:52 crc kubenswrapper[5052]: I1203 13:02:52.667813 5052 reconciler_common.go:293] "Volume detached for volume \"custom-svc-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/951b6922-5edc-46f6-8a69-929e7517850c-custom-svc-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 13:02:52 crc kubenswrapper[5052]: I1203 13:02:52.667863 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-pfj9v\" (UniqueName: \"kubernetes.io/projected/951b6922-5edc-46f6-8a69-929e7517850c-kube-api-access-pfj9v\") on node \"crc\" DevicePath \"\"" Dec 03 13:02:52 crc kubenswrapper[5052]: I1203 13:02:52.667882 5052 reconciler_common.go:293] "Volume detached for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/951b6922-5edc-46f6-8a69-929e7517850c-inventory\") on node \"crc\" DevicePath \"\"" Dec 03 13:02:52 crc kubenswrapper[5052]: I1203 13:02:52.667892 5052 reconciler_common.go:293] "Volume detached for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/951b6922-5edc-46f6-8a69-929e7517850c-ssh-key\") on node \"crc\" DevicePath \"\"" Dec 03 13:02:53 crc kubenswrapper[5052]: I1203 13:02:53.173915 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/custom-svc-edpm-compute-no-nodes-ovrd-edpm-compute-no-noded8jt2" event={"ID":"951b6922-5edc-46f6-8a69-929e7517850c","Type":"ContainerDied","Data":"4ac4c3d116bb3cefd5635b1e9bf087047d2305acc436a798e60e136ef90e7ab6"} Dec 03 13:02:53 crc kubenswrapper[5052]: I1203 13:02:53.174221 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4ac4c3d116bb3cefd5635b1e9bf087047d2305acc436a798e60e136ef90e7ab6" Dec 03 13:02:53 crc kubenswrapper[5052]: I1203 13:02:53.173965 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/custom-svc-edpm-compute-no-nodes-ovrd-edpm-compute-no-noded8jt2" Dec 03 13:02:59 crc kubenswrapper[5052]: I1203 13:02:59.379846 5052 patch_prober.go:28] interesting pod/machine-config-daemon-74bj6 container/machine-config-daemon namespace/openshift-machine-config-operator: Liveness probe status=failure output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" start-of-body= Dec 03 13:02:59 crc kubenswrapper[5052]: I1203 13:02:59.380538 5052 prober.go:107] "Probe failed" probeType="Liveness" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" containerName="machine-config-daemon" probeResult="failure" output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" Dec 03 13:03:17 crc kubenswrapper[5052]: I1203 13:03:17.183668 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ovn-edpm-compute-no-nodes-updated-ovn-cm-edpm-compute-no-nlgvrf"] Dec 03 13:03:17 crc kubenswrapper[5052]: E1203 13:03:17.184400 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="951b6922-5edc-46f6-8a69-929e7517850c" containerName="custom-svc-edpm-compute-no-nodes-ovrd-edpm-compute-no-nodes" Dec 03 13:03:17 crc kubenswrapper[5052]: I1203 13:03:17.184414 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="951b6922-5edc-46f6-8a69-929e7517850c" containerName="custom-svc-edpm-compute-no-nodes-ovrd-edpm-compute-no-nodes" Dec 03 13:03:17 crc kubenswrapper[5052]: I1203 13:03:17.184565 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="951b6922-5edc-46f6-8a69-929e7517850c" containerName="custom-svc-edpm-compute-no-nodes-ovrd-edpm-compute-no-nodes" Dec 03 13:03:17 crc kubenswrapper[5052]: I1203 13:03:17.185060 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovn-edpm-compute-no-nodes-updated-ovn-cm-edpm-compute-no-nlgvrf" Dec 03 13:03:17 crc kubenswrapper[5052]: I1203 13:03:17.188109 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"openstack-aee-default-env" Dec 03 13:03:17 crc kubenswrapper[5052]: I1203 13:03:17.188472 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"ovncontroller-config" Dec 03 13:03:17 crc kubenswrapper[5052]: I1203 13:03:17.188774 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"dataplanenodeset-edpm-compute-no-nodes" Dec 03 13:03:17 crc kubenswrapper[5052]: I1203 13:03:17.189172 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"edpm-compute-no-nodes-dockercfg-mkcgv" Dec 03 13:03:17 crc kubenswrapper[5052]: I1203 13:03:17.189453 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"dataplane-ansible-ssh-private-key-secret" Dec 03 13:03:17 crc kubenswrapper[5052]: I1203 13:03:17.189782 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"combined-ca-bundle" Dec 03 13:03:17 crc kubenswrapper[5052]: I1203 13:03:17.192290 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ovn-edpm-compute-no-nodes-updated-ovn-cm-edpm-compute-no-nlgvrf"] Dec 03 13:03:17 crc kubenswrapper[5052]: I1203 13:03:17.315983 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/bf143189-a60c-4f8c-86e8-a2cfaeb07e4f-ovn-combined-ca-bundle\") pod \"ovn-edpm-compute-no-nodes-updated-ovn-cm-edpm-compute-no-nlgvrf\" (UID: \"bf143189-a60c-4f8c-86e8-a2cfaeb07e4f\") " pod="openstack-kuttl-tests/ovn-edpm-compute-no-nodes-updated-ovn-cm-edpm-compute-no-nlgvrf" Dec 03 13:03:17 crc kubenswrapper[5052]: I1203 13:03:17.316040 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/bf143189-a60c-4f8c-86e8-a2cfaeb07e4f-ssh-key\") pod \"ovn-edpm-compute-no-nodes-updated-ovn-cm-edpm-compute-no-nlgvrf\" (UID: \"bf143189-a60c-4f8c-86e8-a2cfaeb07e4f\") " pod="openstack-kuttl-tests/ovn-edpm-compute-no-nodes-updated-ovn-cm-edpm-compute-no-nlgvrf" Dec 03 13:03:17 crc kubenswrapper[5052]: I1203 13:03:17.316121 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/bf143189-a60c-4f8c-86e8-a2cfaeb07e4f-inventory\") pod \"ovn-edpm-compute-no-nodes-updated-ovn-cm-edpm-compute-no-nlgvrf\" (UID: \"bf143189-a60c-4f8c-86e8-a2cfaeb07e4f\") " pod="openstack-kuttl-tests/ovn-edpm-compute-no-nodes-updated-ovn-cm-edpm-compute-no-nlgvrf" Dec 03 13:03:17 crc kubenswrapper[5052]: I1203 13:03:17.316187 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovncontroller-config-0\" (UniqueName: \"kubernetes.io/configmap/bf143189-a60c-4f8c-86e8-a2cfaeb07e4f-ovncontroller-config-0\") pod \"ovn-edpm-compute-no-nodes-updated-ovn-cm-edpm-compute-no-nlgvrf\" (UID: \"bf143189-a60c-4f8c-86e8-a2cfaeb07e4f\") " pod="openstack-kuttl-tests/ovn-edpm-compute-no-nodes-updated-ovn-cm-edpm-compute-no-nlgvrf" Dec 03 13:03:17 crc kubenswrapper[5052]: I1203 13:03:17.316228 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jqfzx\" (UniqueName: \"kubernetes.io/projected/bf143189-a60c-4f8c-86e8-a2cfaeb07e4f-kube-api-access-jqfzx\") pod \"ovn-edpm-compute-no-nodes-updated-ovn-cm-edpm-compute-no-nlgvrf\" (UID: \"bf143189-a60c-4f8c-86e8-a2cfaeb07e4f\") " pod="openstack-kuttl-tests/ovn-edpm-compute-no-nodes-updated-ovn-cm-edpm-compute-no-nlgvrf" Dec 03 13:03:17 crc kubenswrapper[5052]: I1203 13:03:17.417047 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovn-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/bf143189-a60c-4f8c-86e8-a2cfaeb07e4f-ovn-combined-ca-bundle\") pod \"ovn-edpm-compute-no-nodes-updated-ovn-cm-edpm-compute-no-nlgvrf\" (UID: \"bf143189-a60c-4f8c-86e8-a2cfaeb07e4f\") " pod="openstack-kuttl-tests/ovn-edpm-compute-no-nodes-updated-ovn-cm-edpm-compute-no-nlgvrf" Dec 03 13:03:17 crc kubenswrapper[5052]: I1203 13:03:17.418465 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/bf143189-a60c-4f8c-86e8-a2cfaeb07e4f-ssh-key\") pod \"ovn-edpm-compute-no-nodes-updated-ovn-cm-edpm-compute-no-nlgvrf\" (UID: \"bf143189-a60c-4f8c-86e8-a2cfaeb07e4f\") " pod="openstack-kuttl-tests/ovn-edpm-compute-no-nodes-updated-ovn-cm-edpm-compute-no-nlgvrf" Dec 03 13:03:17 crc kubenswrapper[5052]: I1203 13:03:17.418644 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/bf143189-a60c-4f8c-86e8-a2cfaeb07e4f-inventory\") pod \"ovn-edpm-compute-no-nodes-updated-ovn-cm-edpm-compute-no-nlgvrf\" (UID: \"bf143189-a60c-4f8c-86e8-a2cfaeb07e4f\") " pod="openstack-kuttl-tests/ovn-edpm-compute-no-nodes-updated-ovn-cm-edpm-compute-no-nlgvrf" Dec 03 13:03:17 crc kubenswrapper[5052]: I1203 13:03:17.418694 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovncontroller-config-0\" (UniqueName: \"kubernetes.io/configmap/bf143189-a60c-4f8c-86e8-a2cfaeb07e4f-ovncontroller-config-0\") pod \"ovn-edpm-compute-no-nodes-updated-ovn-cm-edpm-compute-no-nlgvrf\" (UID: \"bf143189-a60c-4f8c-86e8-a2cfaeb07e4f\") " pod="openstack-kuttl-tests/ovn-edpm-compute-no-nodes-updated-ovn-cm-edpm-compute-no-nlgvrf" Dec 03 13:03:17 crc kubenswrapper[5052]: I1203 13:03:17.418738 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-jqfzx\" (UniqueName: \"kubernetes.io/projected/bf143189-a60c-4f8c-86e8-a2cfaeb07e4f-kube-api-access-jqfzx\") pod \"ovn-edpm-compute-no-nodes-updated-ovn-cm-edpm-compute-no-nlgvrf\" (UID: \"bf143189-a60c-4f8c-86e8-a2cfaeb07e4f\") " pod="openstack-kuttl-tests/ovn-edpm-compute-no-nodes-updated-ovn-cm-edpm-compute-no-nlgvrf" Dec 03 13:03:17 crc kubenswrapper[5052]: I1203 13:03:17.419647 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovncontroller-config-0\" (UniqueName: \"kubernetes.io/configmap/bf143189-a60c-4f8c-86e8-a2cfaeb07e4f-ovncontroller-config-0\") pod \"ovn-edpm-compute-no-nodes-updated-ovn-cm-edpm-compute-no-nlgvrf\" (UID: \"bf143189-a60c-4f8c-86e8-a2cfaeb07e4f\") " pod="openstack-kuttl-tests/ovn-edpm-compute-no-nodes-updated-ovn-cm-edpm-compute-no-nlgvrf" Dec 03 13:03:17 crc kubenswrapper[5052]: I1203 13:03:17.425078 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/bf143189-a60c-4f8c-86e8-a2cfaeb07e4f-ssh-key\") pod \"ovn-edpm-compute-no-nodes-updated-ovn-cm-edpm-compute-no-nlgvrf\" (UID: \"bf143189-a60c-4f8c-86e8-a2cfaeb07e4f\") " pod="openstack-kuttl-tests/ovn-edpm-compute-no-nodes-updated-ovn-cm-edpm-compute-no-nlgvrf" Dec 03 13:03:17 crc kubenswrapper[5052]: I1203 13:03:17.430191 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/bf143189-a60c-4f8c-86e8-a2cfaeb07e4f-inventory\") pod \"ovn-edpm-compute-no-nodes-updated-ovn-cm-edpm-compute-no-nlgvrf\" (UID: \"bf143189-a60c-4f8c-86e8-a2cfaeb07e4f\") " pod="openstack-kuttl-tests/ovn-edpm-compute-no-nodes-updated-ovn-cm-edpm-compute-no-nlgvrf" Dec 03 13:03:17 crc kubenswrapper[5052]: I1203 13:03:17.438200 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovn-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/bf143189-a60c-4f8c-86e8-a2cfaeb07e4f-ovn-combined-ca-bundle\") pod \"ovn-edpm-compute-no-nodes-updated-ovn-cm-edpm-compute-no-nlgvrf\" (UID: \"bf143189-a60c-4f8c-86e8-a2cfaeb07e4f\") " pod="openstack-kuttl-tests/ovn-edpm-compute-no-nodes-updated-ovn-cm-edpm-compute-no-nlgvrf" Dec 03 13:03:17 crc kubenswrapper[5052]: I1203 13:03:17.438733 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-jqfzx\" (UniqueName: \"kubernetes.io/projected/bf143189-a60c-4f8c-86e8-a2cfaeb07e4f-kube-api-access-jqfzx\") pod \"ovn-edpm-compute-no-nodes-updated-ovn-cm-edpm-compute-no-nlgvrf\" (UID: \"bf143189-a60c-4f8c-86e8-a2cfaeb07e4f\") " pod="openstack-kuttl-tests/ovn-edpm-compute-no-nodes-updated-ovn-cm-edpm-compute-no-nlgvrf" Dec 03 13:03:17 crc kubenswrapper[5052]: I1203 13:03:17.503656 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovn-edpm-compute-no-nodes-updated-ovn-cm-edpm-compute-no-nlgvrf" Dec 03 13:03:17 crc kubenswrapper[5052]: I1203 13:03:17.924460 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ovn-edpm-compute-no-nodes-updated-ovn-cm-edpm-compute-no-nlgvrf"] Dec 03 13:03:18 crc kubenswrapper[5052]: I1203 13:03:18.397104 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovn-edpm-compute-no-nodes-updated-ovn-cm-edpm-compute-no-nlgvrf" event={"ID":"bf143189-a60c-4f8c-86e8-a2cfaeb07e4f","Type":"ContainerStarted","Data":"7a195ee390bbf725dd1a5440a1e4f644e4e5544fe7c71c77a92528b57ddbc508"} Dec 03 13:03:19 crc kubenswrapper[5052]: I1203 13:03:19.408955 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovn-edpm-compute-no-nodes-updated-ovn-cm-edpm-compute-no-nlgvrf" event={"ID":"bf143189-a60c-4f8c-86e8-a2cfaeb07e4f","Type":"ContainerStarted","Data":"3af3678b7ccde3d4dd555b0b72ae1e5848a1e1bdcb3e281e44e225bb38528da2"} Dec 03 13:03:19 crc kubenswrapper[5052]: I1203 13:03:19.433094 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/ovn-edpm-compute-no-nodes-updated-ovn-cm-edpm-compute-no-nlgvrf" podStartSLOduration=1.9783410030000002 podStartE2EDuration="2.433076778s" podCreationTimestamp="2025-12-03 13:03:17 +0000 UTC" firstStartedPulling="2025-12-03 13:03:17.937211538 +0000 UTC m=+6188.525453302" lastFinishedPulling="2025-12-03 13:03:18.391947313 +0000 UTC m=+6188.980189077" observedRunningTime="2025-12-03 13:03:19.426391077 +0000 UTC m=+6190.014632841" watchObservedRunningTime="2025-12-03 13:03:19.433076778 +0000 UTC m=+6190.021318542" Dec 03 13:03:20 crc kubenswrapper[5052]: I1203 13:03:20.421978 5052 generic.go:334] "Generic (PLEG): container finished" podID="bf143189-a60c-4f8c-86e8-a2cfaeb07e4f" containerID="3af3678b7ccde3d4dd555b0b72ae1e5848a1e1bdcb3e281e44e225bb38528da2" exitCode=0 Dec 03 13:03:20 crc kubenswrapper[5052]: I1203 13:03:20.422052 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovn-edpm-compute-no-nodes-updated-ovn-cm-edpm-compute-no-nlgvrf" event={"ID":"bf143189-a60c-4f8c-86e8-a2cfaeb07e4f","Type":"ContainerDied","Data":"3af3678b7ccde3d4dd555b0b72ae1e5848a1e1bdcb3e281e44e225bb38528da2"} Dec 03 13:03:21 crc kubenswrapper[5052]: I1203 13:03:21.730622 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovn-edpm-compute-no-nodes-updated-ovn-cm-edpm-compute-no-nlgvrf" Dec 03 13:03:21 crc kubenswrapper[5052]: I1203 13:03:21.793459 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jqfzx\" (UniqueName: \"kubernetes.io/projected/bf143189-a60c-4f8c-86e8-a2cfaeb07e4f-kube-api-access-jqfzx\") pod \"bf143189-a60c-4f8c-86e8-a2cfaeb07e4f\" (UID: \"bf143189-a60c-4f8c-86e8-a2cfaeb07e4f\") " Dec 03 13:03:21 crc kubenswrapper[5052]: I1203 13:03:21.793548 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovncontroller-config-0\" (UniqueName: \"kubernetes.io/configmap/bf143189-a60c-4f8c-86e8-a2cfaeb07e4f-ovncontroller-config-0\") pod \"bf143189-a60c-4f8c-86e8-a2cfaeb07e4f\" (UID: \"bf143189-a60c-4f8c-86e8-a2cfaeb07e4f\") " Dec 03 13:03:21 crc kubenswrapper[5052]: I1203 13:03:21.793597 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovn-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/bf143189-a60c-4f8c-86e8-a2cfaeb07e4f-ovn-combined-ca-bundle\") pod \"bf143189-a60c-4f8c-86e8-a2cfaeb07e4f\" (UID: \"bf143189-a60c-4f8c-86e8-a2cfaeb07e4f\") " Dec 03 13:03:21 crc kubenswrapper[5052]: I1203 13:03:21.793626 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/bf143189-a60c-4f8c-86e8-a2cfaeb07e4f-inventory\") pod \"bf143189-a60c-4f8c-86e8-a2cfaeb07e4f\" (UID: \"bf143189-a60c-4f8c-86e8-a2cfaeb07e4f\") " Dec 03 13:03:21 crc kubenswrapper[5052]: I1203 13:03:21.793650 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/bf143189-a60c-4f8c-86e8-a2cfaeb07e4f-ssh-key\") pod \"bf143189-a60c-4f8c-86e8-a2cfaeb07e4f\" (UID: \"bf143189-a60c-4f8c-86e8-a2cfaeb07e4f\") " Dec 03 13:03:21 crc kubenswrapper[5052]: I1203 13:03:21.799735 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bf143189-a60c-4f8c-86e8-a2cfaeb07e4f-kube-api-access-jqfzx" (OuterVolumeSpecName: "kube-api-access-jqfzx") pod "bf143189-a60c-4f8c-86e8-a2cfaeb07e4f" (UID: "bf143189-a60c-4f8c-86e8-a2cfaeb07e4f"). InnerVolumeSpecName "kube-api-access-jqfzx". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:03:21 crc kubenswrapper[5052]: I1203 13:03:21.799974 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bf143189-a60c-4f8c-86e8-a2cfaeb07e4f-ovn-combined-ca-bundle" (OuterVolumeSpecName: "ovn-combined-ca-bundle") pod "bf143189-a60c-4f8c-86e8-a2cfaeb07e4f" (UID: "bf143189-a60c-4f8c-86e8-a2cfaeb07e4f"). InnerVolumeSpecName "ovn-combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:03:21 crc kubenswrapper[5052]: I1203 13:03:21.821039 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bf143189-a60c-4f8c-86e8-a2cfaeb07e4f-ovncontroller-config-0" (OuterVolumeSpecName: "ovncontroller-config-0") pod "bf143189-a60c-4f8c-86e8-a2cfaeb07e4f" (UID: "bf143189-a60c-4f8c-86e8-a2cfaeb07e4f"). InnerVolumeSpecName "ovncontroller-config-0". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:03:21 crc kubenswrapper[5052]: I1203 13:03:21.823226 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bf143189-a60c-4f8c-86e8-a2cfaeb07e4f-inventory" (OuterVolumeSpecName: "inventory") pod "bf143189-a60c-4f8c-86e8-a2cfaeb07e4f" (UID: "bf143189-a60c-4f8c-86e8-a2cfaeb07e4f"). InnerVolumeSpecName "inventory". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:03:21 crc kubenswrapper[5052]: I1203 13:03:21.826342 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bf143189-a60c-4f8c-86e8-a2cfaeb07e4f-ssh-key" (OuterVolumeSpecName: "ssh-key") pod "bf143189-a60c-4f8c-86e8-a2cfaeb07e4f" (UID: "bf143189-a60c-4f8c-86e8-a2cfaeb07e4f"). InnerVolumeSpecName "ssh-key". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:03:21 crc kubenswrapper[5052]: I1203 13:03:21.895899 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-jqfzx\" (UniqueName: \"kubernetes.io/projected/bf143189-a60c-4f8c-86e8-a2cfaeb07e4f-kube-api-access-jqfzx\") on node \"crc\" DevicePath \"\"" Dec 03 13:03:21 crc kubenswrapper[5052]: I1203 13:03:21.895938 5052 reconciler_common.go:293] "Volume detached for volume \"ovncontroller-config-0\" (UniqueName: \"kubernetes.io/configmap/bf143189-a60c-4f8c-86e8-a2cfaeb07e4f-ovncontroller-config-0\") on node \"crc\" DevicePath \"\"" Dec 03 13:03:21 crc kubenswrapper[5052]: I1203 13:03:21.895948 5052 reconciler_common.go:293] "Volume detached for volume \"ovn-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/bf143189-a60c-4f8c-86e8-a2cfaeb07e4f-ovn-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 13:03:21 crc kubenswrapper[5052]: I1203 13:03:21.895958 5052 reconciler_common.go:293] "Volume detached for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/bf143189-a60c-4f8c-86e8-a2cfaeb07e4f-inventory\") on node \"crc\" DevicePath \"\"" Dec 03 13:03:21 crc kubenswrapper[5052]: I1203 13:03:21.895967 5052 reconciler_common.go:293] "Volume detached for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/bf143189-a60c-4f8c-86e8-a2cfaeb07e4f-ssh-key\") on node \"crc\" DevicePath \"\"" Dec 03 13:03:22 crc kubenswrapper[5052]: I1203 13:03:22.443474 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovn-edpm-compute-no-nodes-updated-ovn-cm-edpm-compute-no-nlgvrf" event={"ID":"bf143189-a60c-4f8c-86e8-a2cfaeb07e4f","Type":"ContainerDied","Data":"7a195ee390bbf725dd1a5440a1e4f644e4e5544fe7c71c77a92528b57ddbc508"} Dec 03 13:03:22 crc kubenswrapper[5052]: I1203 13:03:22.443538 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7a195ee390bbf725dd1a5440a1e4f644e4e5544fe7c71c77a92528b57ddbc508" Dec 03 13:03:22 crc kubenswrapper[5052]: I1203 13:03:22.443604 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovn-edpm-compute-no-nodes-updated-ovn-cm-edpm-compute-no-nlgvrf" Dec 03 13:03:24 crc kubenswrapper[5052]: I1203 13:03:24.649526 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-67886899f9-xgth2"] Dec 03 13:03:24 crc kubenswrapper[5052]: E1203 13:03:24.650273 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="bf143189-a60c-4f8c-86e8-a2cfaeb07e4f" containerName="ovn-edpm-compute-no-nodes-updated-ovn-cm-edpm-compute-no-nodes" Dec 03 13:03:24 crc kubenswrapper[5052]: I1203 13:03:24.650292 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="bf143189-a60c-4f8c-86e8-a2cfaeb07e4f" containerName="ovn-edpm-compute-no-nodes-updated-ovn-cm-edpm-compute-no-nodes" Dec 03 13:03:24 crc kubenswrapper[5052]: I1203 13:03:24.650499 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="bf143189-a60c-4f8c-86e8-a2cfaeb07e4f" containerName="ovn-edpm-compute-no-nodes-updated-ovn-cm-edpm-compute-no-nodes" Dec 03 13:03:24 crc kubenswrapper[5052]: I1203 13:03:24.651598 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-67886899f9-xgth2" Dec 03 13:03:24 crc kubenswrapper[5052]: I1203 13:03:24.655034 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"edpm-compute-beta-nodeset" Dec 03 13:03:24 crc kubenswrapper[5052]: I1203 13:03:24.670048 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-67886899f9-xgth2"] Dec 03 13:03:24 crc kubenswrapper[5052]: I1203 13:03:24.742717 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c70a070b-903b-4c81-b43a-e129647efea7-config\") pod \"dnsmasq-dnsmasq-67886899f9-xgth2\" (UID: \"c70a070b-903b-4c81-b43a-e129647efea7\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-67886899f9-xgth2" Dec 03 13:03:24 crc kubenswrapper[5052]: I1203 13:03:24.742832 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/c70a070b-903b-4c81-b43a-e129647efea7-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-67886899f9-xgth2\" (UID: \"c70a070b-903b-4c81-b43a-e129647efea7\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-67886899f9-xgth2" Dec 03 13:03:24 crc kubenswrapper[5052]: I1203 13:03:24.742879 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t97q4\" (UniqueName: \"kubernetes.io/projected/c70a070b-903b-4c81-b43a-e129647efea7-kube-api-access-t97q4\") pod \"dnsmasq-dnsmasq-67886899f9-xgth2\" (UID: \"c70a070b-903b-4c81-b43a-e129647efea7\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-67886899f9-xgth2" Dec 03 13:03:24 crc kubenswrapper[5052]: I1203 13:03:24.742915 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"edpm-compute-no-nodes\" (UniqueName: \"kubernetes.io/configmap/c70a070b-903b-4c81-b43a-e129647efea7-edpm-compute-no-nodes\") pod \"dnsmasq-dnsmasq-67886899f9-xgth2\" (UID: \"c70a070b-903b-4c81-b43a-e129647efea7\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-67886899f9-xgth2" Dec 03 13:03:24 crc kubenswrapper[5052]: I1203 13:03:24.742992 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"edpm-compute-beta-nodeset\" (UniqueName: \"kubernetes.io/configmap/c70a070b-903b-4c81-b43a-e129647efea7-edpm-compute-beta-nodeset\") pod \"dnsmasq-dnsmasq-67886899f9-xgth2\" (UID: \"c70a070b-903b-4c81-b43a-e129647efea7\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-67886899f9-xgth2" Dec 03 13:03:24 crc kubenswrapper[5052]: I1203 13:03:24.844706 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-t97q4\" (UniqueName: \"kubernetes.io/projected/c70a070b-903b-4c81-b43a-e129647efea7-kube-api-access-t97q4\") pod \"dnsmasq-dnsmasq-67886899f9-xgth2\" (UID: \"c70a070b-903b-4c81-b43a-e129647efea7\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-67886899f9-xgth2" Dec 03 13:03:24 crc kubenswrapper[5052]: I1203 13:03:24.844789 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"edpm-compute-no-nodes\" (UniqueName: \"kubernetes.io/configmap/c70a070b-903b-4c81-b43a-e129647efea7-edpm-compute-no-nodes\") pod \"dnsmasq-dnsmasq-67886899f9-xgth2\" (UID: \"c70a070b-903b-4c81-b43a-e129647efea7\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-67886899f9-xgth2" Dec 03 13:03:24 crc kubenswrapper[5052]: I1203 13:03:24.844852 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"edpm-compute-beta-nodeset\" (UniqueName: \"kubernetes.io/configmap/c70a070b-903b-4c81-b43a-e129647efea7-edpm-compute-beta-nodeset\") pod \"dnsmasq-dnsmasq-67886899f9-xgth2\" (UID: \"c70a070b-903b-4c81-b43a-e129647efea7\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-67886899f9-xgth2" Dec 03 13:03:24 crc kubenswrapper[5052]: I1203 13:03:24.844918 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c70a070b-903b-4c81-b43a-e129647efea7-config\") pod \"dnsmasq-dnsmasq-67886899f9-xgth2\" (UID: \"c70a070b-903b-4c81-b43a-e129647efea7\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-67886899f9-xgth2" Dec 03 13:03:24 crc kubenswrapper[5052]: I1203 13:03:24.844982 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/c70a070b-903b-4c81-b43a-e129647efea7-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-67886899f9-xgth2\" (UID: \"c70a070b-903b-4c81-b43a-e129647efea7\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-67886899f9-xgth2" Dec 03 13:03:24 crc kubenswrapper[5052]: I1203 13:03:24.845895 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"edpm-compute-beta-nodeset\" (UniqueName: \"kubernetes.io/configmap/c70a070b-903b-4c81-b43a-e129647efea7-edpm-compute-beta-nodeset\") pod \"dnsmasq-dnsmasq-67886899f9-xgth2\" (UID: \"c70a070b-903b-4c81-b43a-e129647efea7\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-67886899f9-xgth2" Dec 03 13:03:24 crc kubenswrapper[5052]: I1203 13:03:24.846064 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/c70a070b-903b-4c81-b43a-e129647efea7-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-67886899f9-xgth2\" (UID: \"c70a070b-903b-4c81-b43a-e129647efea7\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-67886899f9-xgth2" Dec 03 13:03:24 crc kubenswrapper[5052]: I1203 13:03:24.846672 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"edpm-compute-no-nodes\" (UniqueName: \"kubernetes.io/configmap/c70a070b-903b-4c81-b43a-e129647efea7-edpm-compute-no-nodes\") pod \"dnsmasq-dnsmasq-67886899f9-xgth2\" (UID: \"c70a070b-903b-4c81-b43a-e129647efea7\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-67886899f9-xgth2" Dec 03 13:03:24 crc kubenswrapper[5052]: I1203 13:03:24.846970 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c70a070b-903b-4c81-b43a-e129647efea7-config\") pod \"dnsmasq-dnsmasq-67886899f9-xgth2\" (UID: \"c70a070b-903b-4c81-b43a-e129647efea7\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-67886899f9-xgth2" Dec 03 13:03:24 crc kubenswrapper[5052]: I1203 13:03:24.864584 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-t97q4\" (UniqueName: \"kubernetes.io/projected/c70a070b-903b-4c81-b43a-e129647efea7-kube-api-access-t97q4\") pod \"dnsmasq-dnsmasq-67886899f9-xgth2\" (UID: \"c70a070b-903b-4c81-b43a-e129647efea7\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-67886899f9-xgth2" Dec 03 13:03:24 crc kubenswrapper[5052]: I1203 13:03:24.970026 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-67886899f9-xgth2" Dec 03 13:03:25 crc kubenswrapper[5052]: I1203 13:03:25.542234 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-67886899f9-xgth2"] Dec 03 13:03:26 crc kubenswrapper[5052]: I1203 13:03:26.478017 5052 generic.go:334] "Generic (PLEG): container finished" podID="c70a070b-903b-4c81-b43a-e129647efea7" containerID="d631e8d62045cb6cb73a9f326f7b93e176e0760e4fc5e3871604750f68942881" exitCode=0 Dec 03 13:03:26 crc kubenswrapper[5052]: I1203 13:03:26.478083 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-67886899f9-xgth2" event={"ID":"c70a070b-903b-4c81-b43a-e129647efea7","Type":"ContainerDied","Data":"d631e8d62045cb6cb73a9f326f7b93e176e0760e4fc5e3871604750f68942881"} Dec 03 13:03:26 crc kubenswrapper[5052]: I1203 13:03:26.478304 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-67886899f9-xgth2" event={"ID":"c70a070b-903b-4c81-b43a-e129647efea7","Type":"ContainerStarted","Data":"ef52a8ef8e7ebbe9f4814fc966418199c2f75b2d505b6b1be94f319da8d194d4"} Dec 03 13:03:27 crc kubenswrapper[5052]: I1203 13:03:27.491966 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-67886899f9-xgth2" event={"ID":"c70a070b-903b-4c81-b43a-e129647efea7","Type":"ContainerStarted","Data":"db6c7734ee1af3a31951a50895f1410bc81d07a980f8bb934b75e98b775aa33f"} Dec 03 13:03:27 crc kubenswrapper[5052]: I1203 13:03:27.492625 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-67886899f9-xgth2" Dec 03 13:03:27 crc kubenswrapper[5052]: I1203 13:03:27.513011 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-67886899f9-xgth2" podStartSLOduration=3.51299502 podStartE2EDuration="3.51299502s" podCreationTimestamp="2025-12-03 13:03:24 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 13:03:27.511061424 +0000 UTC m=+6198.099303198" watchObservedRunningTime="2025-12-03 13:03:27.51299502 +0000 UTC m=+6198.101236784" Dec 03 13:03:29 crc kubenswrapper[5052]: I1203 13:03:29.379950 5052 patch_prober.go:28] interesting pod/machine-config-daemon-74bj6 container/machine-config-daemon namespace/openshift-machine-config-operator: Liveness probe status=failure output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" start-of-body= Dec 03 13:03:29 crc kubenswrapper[5052]: I1203 13:03:29.380036 5052 prober.go:107] "Probe failed" probeType="Liveness" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" containerName="machine-config-daemon" probeResult="failure" output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" Dec 03 13:03:29 crc kubenswrapper[5052]: I1203 13:03:29.380075 5052 kubelet.go:2542] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" Dec 03 13:03:29 crc kubenswrapper[5052]: I1203 13:03:29.380740 5052 kuberuntime_manager.go:1027] "Message for Container of pod" containerName="machine-config-daemon" containerStatusID={"Type":"cri-o","ID":"c2cab872e68bfd02259363286d891f85a8e365964ddb0da9f72bb76c86e2d3d2"} pod="openshift-machine-config-operator/machine-config-daemon-74bj6" containerMessage="Container machine-config-daemon failed liveness probe, will be restarted" Dec 03 13:03:29 crc kubenswrapper[5052]: I1203 13:03:29.380785 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" containerName="machine-config-daemon" containerID="cri-o://c2cab872e68bfd02259363286d891f85a8e365964ddb0da9f72bb76c86e2d3d2" gracePeriod=600 Dec 03 13:03:30 crc kubenswrapper[5052]: I1203 13:03:30.521386 5052 generic.go:334] "Generic (PLEG): container finished" podID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" containerID="c2cab872e68bfd02259363286d891f85a8e365964ddb0da9f72bb76c86e2d3d2" exitCode=0 Dec 03 13:03:30 crc kubenswrapper[5052]: I1203 13:03:30.521679 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" event={"ID":"10c237bc-72e7-4da3-9f2c-b1688395bf6e","Type":"ContainerDied","Data":"c2cab872e68bfd02259363286d891f85a8e365964ddb0da9f72bb76c86e2d3d2"} Dec 03 13:03:30 crc kubenswrapper[5052]: I1203 13:03:30.521706 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" event={"ID":"10c237bc-72e7-4da3-9f2c-b1688395bf6e","Type":"ContainerStarted","Data":"dbc100c8081ea1a2f5487429cc9d67de6bc8ba11282b64f248c71275375b0e16"} Dec 03 13:03:30 crc kubenswrapper[5052]: I1203 13:03:30.521724 5052 scope.go:117] "RemoveContainer" containerID="69fa7fbcd2aceb7c53703a19508876f93a97dc9adabb79c16fb691d90f606d58" Dec 03 13:03:34 crc kubenswrapper[5052]: I1203 13:03:34.971778 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-67886899f9-xgth2" Dec 03 13:03:35 crc kubenswrapper[5052]: I1203 13:03:35.034289 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-64864b6d57-m6vm4"] Dec 03 13:03:35 crc kubenswrapper[5052]: I1203 13:03:35.035489 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-64864b6d57-m6vm4" podUID="81730bd7-0103-4be8-86fb-873984597f88" containerName="dnsmasq-dns" containerID="cri-o://fa3231018d1f01da1d43583d08a32a3406f40fecbca74330251b188de857db1f" gracePeriod=10 Dec 03 13:03:35 crc kubenswrapper[5052]: I1203 13:03:35.579210 5052 generic.go:334] "Generic (PLEG): container finished" podID="81730bd7-0103-4be8-86fb-873984597f88" containerID="fa3231018d1f01da1d43583d08a32a3406f40fecbca74330251b188de857db1f" exitCode=0 Dec 03 13:03:35 crc kubenswrapper[5052]: I1203 13:03:35.579261 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-64864b6d57-m6vm4" event={"ID":"81730bd7-0103-4be8-86fb-873984597f88","Type":"ContainerDied","Data":"fa3231018d1f01da1d43583d08a32a3406f40fecbca74330251b188de857db1f"} Dec 03 13:03:35 crc kubenswrapper[5052]: I1203 13:03:35.941939 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-64864b6d57-m6vm4" Dec 03 13:03:36 crc kubenswrapper[5052]: I1203 13:03:36.016884 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"edpm-compute-no-nodes\" (UniqueName: \"kubernetes.io/configmap/81730bd7-0103-4be8-86fb-873984597f88-edpm-compute-no-nodes\") pod \"81730bd7-0103-4be8-86fb-873984597f88\" (UID: \"81730bd7-0103-4be8-86fb-873984597f88\") " Dec 03 13:03:36 crc kubenswrapper[5052]: I1203 13:03:36.017017 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5vj44\" (UniqueName: \"kubernetes.io/projected/81730bd7-0103-4be8-86fb-873984597f88-kube-api-access-5vj44\") pod \"81730bd7-0103-4be8-86fb-873984597f88\" (UID: \"81730bd7-0103-4be8-86fb-873984597f88\") " Dec 03 13:03:36 crc kubenswrapper[5052]: I1203 13:03:36.017083 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/81730bd7-0103-4be8-86fb-873984597f88-config\") pod \"81730bd7-0103-4be8-86fb-873984597f88\" (UID: \"81730bd7-0103-4be8-86fb-873984597f88\") " Dec 03 13:03:36 crc kubenswrapper[5052]: I1203 13:03:36.017208 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/81730bd7-0103-4be8-86fb-873984597f88-dnsmasq-svc\") pod \"81730bd7-0103-4be8-86fb-873984597f88\" (UID: \"81730bd7-0103-4be8-86fb-873984597f88\") " Dec 03 13:03:36 crc kubenswrapper[5052]: I1203 13:03:36.023490 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/81730bd7-0103-4be8-86fb-873984597f88-kube-api-access-5vj44" (OuterVolumeSpecName: "kube-api-access-5vj44") pod "81730bd7-0103-4be8-86fb-873984597f88" (UID: "81730bd7-0103-4be8-86fb-873984597f88"). InnerVolumeSpecName "kube-api-access-5vj44". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:03:36 crc kubenswrapper[5052]: I1203 13:03:36.057640 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/81730bd7-0103-4be8-86fb-873984597f88-config" (OuterVolumeSpecName: "config") pod "81730bd7-0103-4be8-86fb-873984597f88" (UID: "81730bd7-0103-4be8-86fb-873984597f88"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:03:36 crc kubenswrapper[5052]: I1203 13:03:36.060921 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/81730bd7-0103-4be8-86fb-873984597f88-edpm-compute-no-nodes" (OuterVolumeSpecName: "edpm-compute-no-nodes") pod "81730bd7-0103-4be8-86fb-873984597f88" (UID: "81730bd7-0103-4be8-86fb-873984597f88"). InnerVolumeSpecName "edpm-compute-no-nodes". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:03:36 crc kubenswrapper[5052]: I1203 13:03:36.061717 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/81730bd7-0103-4be8-86fb-873984597f88-dnsmasq-svc" (OuterVolumeSpecName: "dnsmasq-svc") pod "81730bd7-0103-4be8-86fb-873984597f88" (UID: "81730bd7-0103-4be8-86fb-873984597f88"). InnerVolumeSpecName "dnsmasq-svc". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:03:36 crc kubenswrapper[5052]: I1203 13:03:36.118770 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/81730bd7-0103-4be8-86fb-873984597f88-config\") on node \"crc\" DevicePath \"\"" Dec 03 13:03:36 crc kubenswrapper[5052]: I1203 13:03:36.118816 5052 reconciler_common.go:293] "Volume detached for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/81730bd7-0103-4be8-86fb-873984597f88-dnsmasq-svc\") on node \"crc\" DevicePath \"\"" Dec 03 13:03:36 crc kubenswrapper[5052]: I1203 13:03:36.118839 5052 reconciler_common.go:293] "Volume detached for volume \"edpm-compute-no-nodes\" (UniqueName: \"kubernetes.io/configmap/81730bd7-0103-4be8-86fb-873984597f88-edpm-compute-no-nodes\") on node \"crc\" DevicePath \"\"" Dec 03 13:03:36 crc kubenswrapper[5052]: I1203 13:03:36.118851 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-5vj44\" (UniqueName: \"kubernetes.io/projected/81730bd7-0103-4be8-86fb-873984597f88-kube-api-access-5vj44\") on node \"crc\" DevicePath \"\"" Dec 03 13:03:36 crc kubenswrapper[5052]: I1203 13:03:36.590056 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-64864b6d57-m6vm4" event={"ID":"81730bd7-0103-4be8-86fb-873984597f88","Type":"ContainerDied","Data":"09d891f4c1fec9cf21fd6b6e72588534fd1d4d42425eed0a764f74d9cf54f1f5"} Dec 03 13:03:36 crc kubenswrapper[5052]: I1203 13:03:36.590097 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-64864b6d57-m6vm4" Dec 03 13:03:36 crc kubenswrapper[5052]: I1203 13:03:36.590108 5052 scope.go:117] "RemoveContainer" containerID="fa3231018d1f01da1d43583d08a32a3406f40fecbca74330251b188de857db1f" Dec 03 13:03:36 crc kubenswrapper[5052]: I1203 13:03:36.621138 5052 scope.go:117] "RemoveContainer" containerID="26e26ab53dd348a85275ab39acd2cb9a38d660665660965e13d29fab6e967fc5" Dec 03 13:03:36 crc kubenswrapper[5052]: I1203 13:03:36.629265 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-64864b6d57-m6vm4"] Dec 03 13:03:36 crc kubenswrapper[5052]: I1203 13:03:36.636136 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-64864b6d57-m6vm4"] Dec 03 13:03:37 crc kubenswrapper[5052]: I1203 13:03:37.682249 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="81730bd7-0103-4be8-86fb-873984597f88" path="/var/lib/kubelet/pods/81730bd7-0103-4be8-86fb-873984597f88/volumes" Dec 03 13:03:39 crc kubenswrapper[5052]: I1203 13:03:39.618289 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/download-cache-edpm-multinodeset-edpm-compute-no-nodes-k5p9j"] Dec 03 13:03:39 crc kubenswrapper[5052]: E1203 13:03:39.621195 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="81730bd7-0103-4be8-86fb-873984597f88" containerName="init" Dec 03 13:03:39 crc kubenswrapper[5052]: I1203 13:03:39.621380 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="81730bd7-0103-4be8-86fb-873984597f88" containerName="init" Dec 03 13:03:39 crc kubenswrapper[5052]: E1203 13:03:39.621495 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="81730bd7-0103-4be8-86fb-873984597f88" containerName="dnsmasq-dns" Dec 03 13:03:39 crc kubenswrapper[5052]: I1203 13:03:39.621591 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="81730bd7-0103-4be8-86fb-873984597f88" containerName="dnsmasq-dns" Dec 03 13:03:39 crc kubenswrapper[5052]: I1203 13:03:39.621874 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="81730bd7-0103-4be8-86fb-873984597f88" containerName="dnsmasq-dns" Dec 03 13:03:39 crc kubenswrapper[5052]: I1203 13:03:39.622691 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/download-cache-edpm-multinodeset-edpm-compute-no-nodes-k5p9j" Dec 03 13:03:39 crc kubenswrapper[5052]: I1203 13:03:39.625759 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"dataplane-ansible-ssh-private-key-secret" Dec 03 13:03:39 crc kubenswrapper[5052]: I1203 13:03:39.626889 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"openstack-aee-default-env" Dec 03 13:03:39 crc kubenswrapper[5052]: I1203 13:03:39.627052 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"dataplanenodeset-edpm-compute-no-nodes" Dec 03 13:03:39 crc kubenswrapper[5052]: I1203 13:03:39.627052 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"edpm-compute-no-nodes-dockercfg-mkcgv" Dec 03 13:03:39 crc kubenswrapper[5052]: I1203 13:03:39.630627 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/download-cache-edpm-multinodeset-edpm-compute-beta-nodeset7cf9h"] Dec 03 13:03:39 crc kubenswrapper[5052]: I1203 13:03:39.631751 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/download-cache-edpm-multinodeset-edpm-compute-beta-nodeset7cf9h" Dec 03 13:03:39 crc kubenswrapper[5052]: I1203 13:03:39.634944 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"dataplanenodeset-edpm-compute-beta-nodeset" Dec 03 13:03:39 crc kubenswrapper[5052]: I1203 13:03:39.637234 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"edpm-compute-beta-nodeset-dockercfg-tgg86" Dec 03 13:03:39 crc kubenswrapper[5052]: I1203 13:03:39.642483 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/download-cache-edpm-multinodeset-edpm-compute-no-nodes-k5p9j"] Dec 03 13:03:39 crc kubenswrapper[5052]: I1203 13:03:39.654726 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/download-cache-edpm-multinodeset-edpm-compute-beta-nodeset7cf9h"] Dec 03 13:03:39 crc kubenswrapper[5052]: I1203 13:03:39.678426 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/504b0477-2b1b-403c-9bbf-4dfe2543f8dd-inventory\") pod \"download-cache-edpm-multinodeset-edpm-compute-no-nodes-k5p9j\" (UID: \"504b0477-2b1b-403c-9bbf-4dfe2543f8dd\") " pod="openstack-kuttl-tests/download-cache-edpm-multinodeset-edpm-compute-no-nodes-k5p9j" Dec 03 13:03:39 crc kubenswrapper[5052]: I1203 13:03:39.678466 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/504b0477-2b1b-403c-9bbf-4dfe2543f8dd-ssh-key\") pod \"download-cache-edpm-multinodeset-edpm-compute-no-nodes-k5p9j\" (UID: \"504b0477-2b1b-403c-9bbf-4dfe2543f8dd\") " pod="openstack-kuttl-tests/download-cache-edpm-multinodeset-edpm-compute-no-nodes-k5p9j" Dec 03 13:03:39 crc kubenswrapper[5052]: I1203 13:03:39.678487 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/fa362928-93ec-4d50-98e0-0c3e6bfe2758-ssh-key\") pod \"download-cache-edpm-multinodeset-edpm-compute-beta-nodeset7cf9h\" (UID: \"fa362928-93ec-4d50-98e0-0c3e6bfe2758\") " pod="openstack-kuttl-tests/download-cache-edpm-multinodeset-edpm-compute-beta-nodeset7cf9h" Dec 03 13:03:39 crc kubenswrapper[5052]: I1203 13:03:39.678598 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hb74d\" (UniqueName: \"kubernetes.io/projected/fa362928-93ec-4d50-98e0-0c3e6bfe2758-kube-api-access-hb74d\") pod \"download-cache-edpm-multinodeset-edpm-compute-beta-nodeset7cf9h\" (UID: \"fa362928-93ec-4d50-98e0-0c3e6bfe2758\") " pod="openstack-kuttl-tests/download-cache-edpm-multinodeset-edpm-compute-beta-nodeset7cf9h" Dec 03 13:03:39 crc kubenswrapper[5052]: I1203 13:03:39.678631 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bmtjf\" (UniqueName: \"kubernetes.io/projected/504b0477-2b1b-403c-9bbf-4dfe2543f8dd-kube-api-access-bmtjf\") pod \"download-cache-edpm-multinodeset-edpm-compute-no-nodes-k5p9j\" (UID: \"504b0477-2b1b-403c-9bbf-4dfe2543f8dd\") " pod="openstack-kuttl-tests/download-cache-edpm-multinodeset-edpm-compute-no-nodes-k5p9j" Dec 03 13:03:39 crc kubenswrapper[5052]: I1203 13:03:39.678659 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/fa362928-93ec-4d50-98e0-0c3e6bfe2758-inventory\") pod \"download-cache-edpm-multinodeset-edpm-compute-beta-nodeset7cf9h\" (UID: \"fa362928-93ec-4d50-98e0-0c3e6bfe2758\") " pod="openstack-kuttl-tests/download-cache-edpm-multinodeset-edpm-compute-beta-nodeset7cf9h" Dec 03 13:03:39 crc kubenswrapper[5052]: I1203 13:03:39.781427 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-hb74d\" (UniqueName: \"kubernetes.io/projected/fa362928-93ec-4d50-98e0-0c3e6bfe2758-kube-api-access-hb74d\") pod \"download-cache-edpm-multinodeset-edpm-compute-beta-nodeset7cf9h\" (UID: \"fa362928-93ec-4d50-98e0-0c3e6bfe2758\") " pod="openstack-kuttl-tests/download-cache-edpm-multinodeset-edpm-compute-beta-nodeset7cf9h" Dec 03 13:03:39 crc kubenswrapper[5052]: I1203 13:03:39.781558 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-bmtjf\" (UniqueName: \"kubernetes.io/projected/504b0477-2b1b-403c-9bbf-4dfe2543f8dd-kube-api-access-bmtjf\") pod \"download-cache-edpm-multinodeset-edpm-compute-no-nodes-k5p9j\" (UID: \"504b0477-2b1b-403c-9bbf-4dfe2543f8dd\") " pod="openstack-kuttl-tests/download-cache-edpm-multinodeset-edpm-compute-no-nodes-k5p9j" Dec 03 13:03:39 crc kubenswrapper[5052]: I1203 13:03:39.781621 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/fa362928-93ec-4d50-98e0-0c3e6bfe2758-inventory\") pod \"download-cache-edpm-multinodeset-edpm-compute-beta-nodeset7cf9h\" (UID: \"fa362928-93ec-4d50-98e0-0c3e6bfe2758\") " pod="openstack-kuttl-tests/download-cache-edpm-multinodeset-edpm-compute-beta-nodeset7cf9h" Dec 03 13:03:39 crc kubenswrapper[5052]: I1203 13:03:39.781727 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/504b0477-2b1b-403c-9bbf-4dfe2543f8dd-inventory\") pod \"download-cache-edpm-multinodeset-edpm-compute-no-nodes-k5p9j\" (UID: \"504b0477-2b1b-403c-9bbf-4dfe2543f8dd\") " pod="openstack-kuttl-tests/download-cache-edpm-multinodeset-edpm-compute-no-nodes-k5p9j" Dec 03 13:03:39 crc kubenswrapper[5052]: I1203 13:03:39.781776 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/504b0477-2b1b-403c-9bbf-4dfe2543f8dd-ssh-key\") pod \"download-cache-edpm-multinodeset-edpm-compute-no-nodes-k5p9j\" (UID: \"504b0477-2b1b-403c-9bbf-4dfe2543f8dd\") " pod="openstack-kuttl-tests/download-cache-edpm-multinodeset-edpm-compute-no-nodes-k5p9j" Dec 03 13:03:39 crc kubenswrapper[5052]: I1203 13:03:39.781815 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/fa362928-93ec-4d50-98e0-0c3e6bfe2758-ssh-key\") pod \"download-cache-edpm-multinodeset-edpm-compute-beta-nodeset7cf9h\" (UID: \"fa362928-93ec-4d50-98e0-0c3e6bfe2758\") " pod="openstack-kuttl-tests/download-cache-edpm-multinodeset-edpm-compute-beta-nodeset7cf9h" Dec 03 13:03:39 crc kubenswrapper[5052]: I1203 13:03:39.790088 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/504b0477-2b1b-403c-9bbf-4dfe2543f8dd-inventory\") pod \"download-cache-edpm-multinodeset-edpm-compute-no-nodes-k5p9j\" (UID: \"504b0477-2b1b-403c-9bbf-4dfe2543f8dd\") " pod="openstack-kuttl-tests/download-cache-edpm-multinodeset-edpm-compute-no-nodes-k5p9j" Dec 03 13:03:39 crc kubenswrapper[5052]: I1203 13:03:39.790235 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/fa362928-93ec-4d50-98e0-0c3e6bfe2758-ssh-key\") pod \"download-cache-edpm-multinodeset-edpm-compute-beta-nodeset7cf9h\" (UID: \"fa362928-93ec-4d50-98e0-0c3e6bfe2758\") " pod="openstack-kuttl-tests/download-cache-edpm-multinodeset-edpm-compute-beta-nodeset7cf9h" Dec 03 13:03:39 crc kubenswrapper[5052]: I1203 13:03:39.790783 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/fa362928-93ec-4d50-98e0-0c3e6bfe2758-inventory\") pod \"download-cache-edpm-multinodeset-edpm-compute-beta-nodeset7cf9h\" (UID: \"fa362928-93ec-4d50-98e0-0c3e6bfe2758\") " pod="openstack-kuttl-tests/download-cache-edpm-multinodeset-edpm-compute-beta-nodeset7cf9h" Dec 03 13:03:39 crc kubenswrapper[5052]: I1203 13:03:39.790978 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/504b0477-2b1b-403c-9bbf-4dfe2543f8dd-ssh-key\") pod \"download-cache-edpm-multinodeset-edpm-compute-no-nodes-k5p9j\" (UID: \"504b0477-2b1b-403c-9bbf-4dfe2543f8dd\") " pod="openstack-kuttl-tests/download-cache-edpm-multinodeset-edpm-compute-no-nodes-k5p9j" Dec 03 13:03:39 crc kubenswrapper[5052]: I1203 13:03:39.802097 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-hb74d\" (UniqueName: \"kubernetes.io/projected/fa362928-93ec-4d50-98e0-0c3e6bfe2758-kube-api-access-hb74d\") pod \"download-cache-edpm-multinodeset-edpm-compute-beta-nodeset7cf9h\" (UID: \"fa362928-93ec-4d50-98e0-0c3e6bfe2758\") " pod="openstack-kuttl-tests/download-cache-edpm-multinodeset-edpm-compute-beta-nodeset7cf9h" Dec 03 13:03:39 crc kubenswrapper[5052]: I1203 13:03:39.804226 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-bmtjf\" (UniqueName: \"kubernetes.io/projected/504b0477-2b1b-403c-9bbf-4dfe2543f8dd-kube-api-access-bmtjf\") pod \"download-cache-edpm-multinodeset-edpm-compute-no-nodes-k5p9j\" (UID: \"504b0477-2b1b-403c-9bbf-4dfe2543f8dd\") " pod="openstack-kuttl-tests/download-cache-edpm-multinodeset-edpm-compute-no-nodes-k5p9j" Dec 03 13:03:39 crc kubenswrapper[5052]: I1203 13:03:39.958657 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/download-cache-edpm-multinodeset-edpm-compute-no-nodes-k5p9j" Dec 03 13:03:39 crc kubenswrapper[5052]: I1203 13:03:39.973010 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/download-cache-edpm-multinodeset-edpm-compute-beta-nodeset7cf9h" Dec 03 13:03:40 crc kubenswrapper[5052]: I1203 13:03:40.238593 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/download-cache-edpm-multinodeset-edpm-compute-beta-nodeset7cf9h"] Dec 03 13:03:40 crc kubenswrapper[5052]: W1203 13:03:40.437172 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod504b0477_2b1b_403c_9bbf_4dfe2543f8dd.slice/crio-0b35397af912bdb4f40b29f37239b7575959df327461536c0299adeb556b17f7 WatchSource:0}: Error finding container 0b35397af912bdb4f40b29f37239b7575959df327461536c0299adeb556b17f7: Status 404 returned error can't find the container with id 0b35397af912bdb4f40b29f37239b7575959df327461536c0299adeb556b17f7 Dec 03 13:03:40 crc kubenswrapper[5052]: I1203 13:03:40.438995 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/download-cache-edpm-multinodeset-edpm-compute-no-nodes-k5p9j"] Dec 03 13:03:40 crc kubenswrapper[5052]: I1203 13:03:40.545719 5052 scope.go:117] "RemoveContainer" containerID="4cf61e454547ee7071ac8e117abaf877c9e8f3c66e33c632e689ad67fc0440df" Dec 03 13:03:40 crc kubenswrapper[5052]: I1203 13:03:40.577166 5052 scope.go:117] "RemoveContainer" containerID="75cdca491987d282e0403d1bab06e238526d67486286e68090da422766bc9093" Dec 03 13:03:40 crc kubenswrapper[5052]: I1203 13:03:40.602796 5052 scope.go:117] "RemoveContainer" containerID="48e513f51c2458ff90bf4c4af1efa154b86bfb7487ef23ef47f70cf0942dd19b" Dec 03 13:03:40 crc kubenswrapper[5052]: I1203 13:03:40.633141 5052 scope.go:117] "RemoveContainer" containerID="fe2b87c907b25b19cfafc26126c48aaef2435a532f4d6cb5c96faebd6a8c77cb" Dec 03 13:03:40 crc kubenswrapper[5052]: I1203 13:03:40.638314 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/download-cache-edpm-multinodeset-edpm-compute-no-nodes-k5p9j" event={"ID":"504b0477-2b1b-403c-9bbf-4dfe2543f8dd","Type":"ContainerStarted","Data":"0b35397af912bdb4f40b29f37239b7575959df327461536c0299adeb556b17f7"} Dec 03 13:03:40 crc kubenswrapper[5052]: I1203 13:03:40.645385 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/download-cache-edpm-multinodeset-edpm-compute-beta-nodeset7cf9h" event={"ID":"fa362928-93ec-4d50-98e0-0c3e6bfe2758","Type":"ContainerStarted","Data":"f174296d0b57be00ee3364c048de2140713778ab34252f9a67460e788b2ae9da"} Dec 03 13:03:40 crc kubenswrapper[5052]: I1203 13:03:40.672809 5052 scope.go:117] "RemoveContainer" containerID="82273b9da0043f3ca02df3042d32fcb96cfef15bae8ef0b8a95055b91ecb1233" Dec 03 13:03:40 crc kubenswrapper[5052]: I1203 13:03:40.701505 5052 scope.go:117] "RemoveContainer" containerID="0e6edd2dc9f4b852bedb05e9840478c19ae05171f6960f1774e3c9cc2d2112ce" Dec 03 13:03:40 crc kubenswrapper[5052]: I1203 13:03:40.728654 5052 scope.go:117] "RemoveContainer" containerID="853f5dcd4e910e741a9e3919fe95580a2245f409cbb1171f83441a8643074e00" Dec 03 13:03:40 crc kubenswrapper[5052]: I1203 13:03:40.752871 5052 scope.go:117] "RemoveContainer" containerID="31a8f9017cd879bdfe8012e920146b64e25c9b677ff80153e43f799ec1f6e0e2" Dec 03 13:03:40 crc kubenswrapper[5052]: I1203 13:03:40.782826 5052 scope.go:117] "RemoveContainer" containerID="dd30be37e0e72ae0ce20ff47e803bd47943e27342d32ad9fb8f062ba7236c0dc" Dec 03 13:03:40 crc kubenswrapper[5052]: I1203 13:03:40.817020 5052 scope.go:117] "RemoveContainer" containerID="3d11a4a7e3e873380387601c752ad0fe565caf2439bc630b7365094dbe87ea56" Dec 03 13:03:40 crc kubenswrapper[5052]: I1203 13:03:40.969926 5052 scope.go:117] "RemoveContainer" containerID="d6db46a5be697545a8042aeef3cecb828d2a934319fa3b32c5fa2d569d660329" Dec 03 13:03:41 crc kubenswrapper[5052]: I1203 13:03:41.658937 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/download-cache-edpm-multinodeset-edpm-compute-beta-nodeset7cf9h" event={"ID":"fa362928-93ec-4d50-98e0-0c3e6bfe2758","Type":"ContainerStarted","Data":"4d9cf1e8cdc4000660f6348bc391570c43d1810c852fce65e112085af5c919fc"} Dec 03 13:03:41 crc kubenswrapper[5052]: I1203 13:03:41.660783 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/download-cache-edpm-multinodeset-edpm-compute-no-nodes-k5p9j" event={"ID":"504b0477-2b1b-403c-9bbf-4dfe2543f8dd","Type":"ContainerStarted","Data":"abc46c07438d7d2c153343e71e0c8ac8fc43f50b3e36dfc8d5e8cb94fbdadce4"} Dec 03 13:03:41 crc kubenswrapper[5052]: I1203 13:03:41.684840 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/download-cache-edpm-multinodeset-edpm-compute-beta-nodeset7cf9h" podStartSLOduration=2.113665594 podStartE2EDuration="2.684822128s" podCreationTimestamp="2025-12-03 13:03:39 +0000 UTC" firstStartedPulling="2025-12-03 13:03:40.24599644 +0000 UTC m=+6210.834238204" lastFinishedPulling="2025-12-03 13:03:40.817152974 +0000 UTC m=+6211.405394738" observedRunningTime="2025-12-03 13:03:41.678398175 +0000 UTC m=+6212.266639939" watchObservedRunningTime="2025-12-03 13:03:41.684822128 +0000 UTC m=+6212.273063892" Dec 03 13:03:41 crc kubenswrapper[5052]: I1203 13:03:41.706350 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/download-cache-edpm-multinodeset-edpm-compute-no-nodes-k5p9j" podStartSLOduration=2.248768047 podStartE2EDuration="2.706329613s" podCreationTimestamp="2025-12-03 13:03:39 +0000 UTC" firstStartedPulling="2025-12-03 13:03:40.439999188 +0000 UTC m=+6211.028240952" lastFinishedPulling="2025-12-03 13:03:40.897560754 +0000 UTC m=+6211.485802518" observedRunningTime="2025-12-03 13:03:41.702656128 +0000 UTC m=+6212.290897892" watchObservedRunningTime="2025-12-03 13:03:41.706329613 +0000 UTC m=+6212.294571377" Dec 03 13:03:42 crc kubenswrapper[5052]: I1203 13:03:42.674775 5052 generic.go:334] "Generic (PLEG): container finished" podID="fa362928-93ec-4d50-98e0-0c3e6bfe2758" containerID="4d9cf1e8cdc4000660f6348bc391570c43d1810c852fce65e112085af5c919fc" exitCode=0 Dec 03 13:03:42 crc kubenswrapper[5052]: I1203 13:03:42.674909 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/download-cache-edpm-multinodeset-edpm-compute-beta-nodeset7cf9h" event={"ID":"fa362928-93ec-4d50-98e0-0c3e6bfe2758","Type":"ContainerDied","Data":"4d9cf1e8cdc4000660f6348bc391570c43d1810c852fce65e112085af5c919fc"} Dec 03 13:03:42 crc kubenswrapper[5052]: I1203 13:03:42.676925 5052 generic.go:334] "Generic (PLEG): container finished" podID="504b0477-2b1b-403c-9bbf-4dfe2543f8dd" containerID="abc46c07438d7d2c153343e71e0c8ac8fc43f50b3e36dfc8d5e8cb94fbdadce4" exitCode=0 Dec 03 13:03:42 crc kubenswrapper[5052]: I1203 13:03:42.676969 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/download-cache-edpm-multinodeset-edpm-compute-no-nodes-k5p9j" event={"ID":"504b0477-2b1b-403c-9bbf-4dfe2543f8dd","Type":"ContainerDied","Data":"abc46c07438d7d2c153343e71e0c8ac8fc43f50b3e36dfc8d5e8cb94fbdadce4"} Dec 03 13:03:44 crc kubenswrapper[5052]: I1203 13:03:44.083466 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/download-cache-edpm-multinodeset-edpm-compute-no-nodes-k5p9j" Dec 03 13:03:44 crc kubenswrapper[5052]: I1203 13:03:44.092733 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/download-cache-edpm-multinodeset-edpm-compute-beta-nodeset7cf9h" Dec 03 13:03:44 crc kubenswrapper[5052]: I1203 13:03:44.193760 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bmtjf\" (UniqueName: \"kubernetes.io/projected/504b0477-2b1b-403c-9bbf-4dfe2543f8dd-kube-api-access-bmtjf\") pod \"504b0477-2b1b-403c-9bbf-4dfe2543f8dd\" (UID: \"504b0477-2b1b-403c-9bbf-4dfe2543f8dd\") " Dec 03 13:03:44 crc kubenswrapper[5052]: I1203 13:03:44.193867 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hb74d\" (UniqueName: \"kubernetes.io/projected/fa362928-93ec-4d50-98e0-0c3e6bfe2758-kube-api-access-hb74d\") pod \"fa362928-93ec-4d50-98e0-0c3e6bfe2758\" (UID: \"fa362928-93ec-4d50-98e0-0c3e6bfe2758\") " Dec 03 13:03:44 crc kubenswrapper[5052]: I1203 13:03:44.193896 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/fa362928-93ec-4d50-98e0-0c3e6bfe2758-inventory\") pod \"fa362928-93ec-4d50-98e0-0c3e6bfe2758\" (UID: \"fa362928-93ec-4d50-98e0-0c3e6bfe2758\") " Dec 03 13:03:44 crc kubenswrapper[5052]: I1203 13:03:44.194819 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/504b0477-2b1b-403c-9bbf-4dfe2543f8dd-inventory\") pod \"504b0477-2b1b-403c-9bbf-4dfe2543f8dd\" (UID: \"504b0477-2b1b-403c-9bbf-4dfe2543f8dd\") " Dec 03 13:03:44 crc kubenswrapper[5052]: I1203 13:03:44.194892 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/fa362928-93ec-4d50-98e0-0c3e6bfe2758-ssh-key\") pod \"fa362928-93ec-4d50-98e0-0c3e6bfe2758\" (UID: \"fa362928-93ec-4d50-98e0-0c3e6bfe2758\") " Dec 03 13:03:44 crc kubenswrapper[5052]: I1203 13:03:44.194914 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/504b0477-2b1b-403c-9bbf-4dfe2543f8dd-ssh-key\") pod \"504b0477-2b1b-403c-9bbf-4dfe2543f8dd\" (UID: \"504b0477-2b1b-403c-9bbf-4dfe2543f8dd\") " Dec 03 13:03:44 crc kubenswrapper[5052]: I1203 13:03:44.199741 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fa362928-93ec-4d50-98e0-0c3e6bfe2758-kube-api-access-hb74d" (OuterVolumeSpecName: "kube-api-access-hb74d") pod "fa362928-93ec-4d50-98e0-0c3e6bfe2758" (UID: "fa362928-93ec-4d50-98e0-0c3e6bfe2758"). InnerVolumeSpecName "kube-api-access-hb74d". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:03:44 crc kubenswrapper[5052]: I1203 13:03:44.199793 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/504b0477-2b1b-403c-9bbf-4dfe2543f8dd-kube-api-access-bmtjf" (OuterVolumeSpecName: "kube-api-access-bmtjf") pod "504b0477-2b1b-403c-9bbf-4dfe2543f8dd" (UID: "504b0477-2b1b-403c-9bbf-4dfe2543f8dd"). InnerVolumeSpecName "kube-api-access-bmtjf". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:03:44 crc kubenswrapper[5052]: I1203 13:03:44.215855 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/504b0477-2b1b-403c-9bbf-4dfe2543f8dd-ssh-key" (OuterVolumeSpecName: "ssh-key") pod "504b0477-2b1b-403c-9bbf-4dfe2543f8dd" (UID: "504b0477-2b1b-403c-9bbf-4dfe2543f8dd"). InnerVolumeSpecName "ssh-key". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:03:44 crc kubenswrapper[5052]: I1203 13:03:44.216825 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/504b0477-2b1b-403c-9bbf-4dfe2543f8dd-inventory" (OuterVolumeSpecName: "inventory") pod "504b0477-2b1b-403c-9bbf-4dfe2543f8dd" (UID: "504b0477-2b1b-403c-9bbf-4dfe2543f8dd"). InnerVolumeSpecName "inventory". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:03:44 crc kubenswrapper[5052]: I1203 13:03:44.217237 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fa362928-93ec-4d50-98e0-0c3e6bfe2758-inventory" (OuterVolumeSpecName: "inventory") pod "fa362928-93ec-4d50-98e0-0c3e6bfe2758" (UID: "fa362928-93ec-4d50-98e0-0c3e6bfe2758"). InnerVolumeSpecName "inventory". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:03:44 crc kubenswrapper[5052]: I1203 13:03:44.219071 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fa362928-93ec-4d50-98e0-0c3e6bfe2758-ssh-key" (OuterVolumeSpecName: "ssh-key") pod "fa362928-93ec-4d50-98e0-0c3e6bfe2758" (UID: "fa362928-93ec-4d50-98e0-0c3e6bfe2758"). InnerVolumeSpecName "ssh-key". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:03:44 crc kubenswrapper[5052]: I1203 13:03:44.296313 5052 reconciler_common.go:293] "Volume detached for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/504b0477-2b1b-403c-9bbf-4dfe2543f8dd-inventory\") on node \"crc\" DevicePath \"\"" Dec 03 13:03:44 crc kubenswrapper[5052]: I1203 13:03:44.296378 5052 reconciler_common.go:293] "Volume detached for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/fa362928-93ec-4d50-98e0-0c3e6bfe2758-ssh-key\") on node \"crc\" DevicePath \"\"" Dec 03 13:03:44 crc kubenswrapper[5052]: I1203 13:03:44.296390 5052 reconciler_common.go:293] "Volume detached for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/504b0477-2b1b-403c-9bbf-4dfe2543f8dd-ssh-key\") on node \"crc\" DevicePath \"\"" Dec 03 13:03:44 crc kubenswrapper[5052]: I1203 13:03:44.296404 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-bmtjf\" (UniqueName: \"kubernetes.io/projected/504b0477-2b1b-403c-9bbf-4dfe2543f8dd-kube-api-access-bmtjf\") on node \"crc\" DevicePath \"\"" Dec 03 13:03:44 crc kubenswrapper[5052]: I1203 13:03:44.296419 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-hb74d\" (UniqueName: \"kubernetes.io/projected/fa362928-93ec-4d50-98e0-0c3e6bfe2758-kube-api-access-hb74d\") on node \"crc\" DevicePath \"\"" Dec 03 13:03:44 crc kubenswrapper[5052]: I1203 13:03:44.296427 5052 reconciler_common.go:293] "Volume detached for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/fa362928-93ec-4d50-98e0-0c3e6bfe2758-inventory\") on node \"crc\" DevicePath \"\"" Dec 03 13:03:44 crc kubenswrapper[5052]: I1203 13:03:44.694053 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/download-cache-edpm-multinodeset-edpm-compute-beta-nodeset7cf9h" event={"ID":"fa362928-93ec-4d50-98e0-0c3e6bfe2758","Type":"ContainerDied","Data":"f174296d0b57be00ee3364c048de2140713778ab34252f9a67460e788b2ae9da"} Dec 03 13:03:44 crc kubenswrapper[5052]: I1203 13:03:44.694416 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f174296d0b57be00ee3364c048de2140713778ab34252f9a67460e788b2ae9da" Dec 03 13:03:44 crc kubenswrapper[5052]: I1203 13:03:44.694089 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/download-cache-edpm-multinodeset-edpm-compute-beta-nodeset7cf9h" Dec 03 13:03:44 crc kubenswrapper[5052]: I1203 13:03:44.695335 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/download-cache-edpm-multinodeset-edpm-compute-no-nodes-k5p9j" event={"ID":"504b0477-2b1b-403c-9bbf-4dfe2543f8dd","Type":"ContainerDied","Data":"0b35397af912bdb4f40b29f37239b7575959df327461536c0299adeb556b17f7"} Dec 03 13:03:44 crc kubenswrapper[5052]: I1203 13:03:44.695360 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0b35397af912bdb4f40b29f37239b7575959df327461536c0299adeb556b17f7" Dec 03 13:03:44 crc kubenswrapper[5052]: I1203 13:03:44.695444 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/download-cache-edpm-multinodeset-edpm-compute-no-nodes-k5p9j" Dec 03 13:03:44 crc kubenswrapper[5052]: I1203 13:03:44.761994 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/bootstrap-edpm-multinodeset-edpm-compute-beta-nodeset-b28bm"] Dec 03 13:03:44 crc kubenswrapper[5052]: E1203 13:03:44.762394 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="504b0477-2b1b-403c-9bbf-4dfe2543f8dd" containerName="download-cache-edpm-multinodeset-edpm-compute-no-nodes" Dec 03 13:03:44 crc kubenswrapper[5052]: I1203 13:03:44.762416 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="504b0477-2b1b-403c-9bbf-4dfe2543f8dd" containerName="download-cache-edpm-multinodeset-edpm-compute-no-nodes" Dec 03 13:03:44 crc kubenswrapper[5052]: E1203 13:03:44.762430 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="fa362928-93ec-4d50-98e0-0c3e6bfe2758" containerName="download-cache-edpm-multinodeset-edpm-compute-beta-nodeset" Dec 03 13:03:44 crc kubenswrapper[5052]: I1203 13:03:44.762438 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="fa362928-93ec-4d50-98e0-0c3e6bfe2758" containerName="download-cache-edpm-multinodeset-edpm-compute-beta-nodeset" Dec 03 13:03:44 crc kubenswrapper[5052]: I1203 13:03:44.762690 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="fa362928-93ec-4d50-98e0-0c3e6bfe2758" containerName="download-cache-edpm-multinodeset-edpm-compute-beta-nodeset" Dec 03 13:03:44 crc kubenswrapper[5052]: I1203 13:03:44.762720 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="504b0477-2b1b-403c-9bbf-4dfe2543f8dd" containerName="download-cache-edpm-multinodeset-edpm-compute-no-nodes" Dec 03 13:03:44 crc kubenswrapper[5052]: I1203 13:03:44.763351 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/bootstrap-edpm-multinodeset-edpm-compute-beta-nodeset-b28bm" Dec 03 13:03:44 crc kubenswrapper[5052]: I1203 13:03:44.765001 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"edpm-compute-beta-nodeset-dockercfg-tgg86" Dec 03 13:03:44 crc kubenswrapper[5052]: I1203 13:03:44.765059 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"dataplanenodeset-edpm-compute-beta-nodeset" Dec 03 13:03:44 crc kubenswrapper[5052]: I1203 13:03:44.765794 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"openstack-aee-default-env" Dec 03 13:03:44 crc kubenswrapper[5052]: I1203 13:03:44.766062 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"combined-ca-bundle" Dec 03 13:03:44 crc kubenswrapper[5052]: I1203 13:03:44.766282 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"dataplane-ansible-ssh-private-key-secret" Dec 03 13:03:44 crc kubenswrapper[5052]: I1203 13:03:44.795032 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/bootstrap-edpm-multinodeset-edpm-compute-beta-nodeset-b28bm"] Dec 03 13:03:44 crc kubenswrapper[5052]: I1203 13:03:44.806356 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/bootstrap-edpm-multinodeset-edpm-compute-no-nodes-4cq6w"] Dec 03 13:03:44 crc kubenswrapper[5052]: I1203 13:03:44.807489 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/bootstrap-edpm-multinodeset-edpm-compute-no-nodes-4cq6w" Dec 03 13:03:44 crc kubenswrapper[5052]: I1203 13:03:44.809685 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"dataplanenodeset-edpm-compute-no-nodes" Dec 03 13:03:44 crc kubenswrapper[5052]: I1203 13:03:44.809864 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"edpm-compute-no-nodes-dockercfg-mkcgv" Dec 03 13:03:44 crc kubenswrapper[5052]: I1203 13:03:44.824238 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/bootstrap-edpm-multinodeset-edpm-compute-no-nodes-4cq6w"] Dec 03 13:03:44 crc kubenswrapper[5052]: I1203 13:03:44.906582 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bootstrap-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a913674d-b6c4-442f-9d33-dab50e694cc4-bootstrap-combined-ca-bundle\") pod \"bootstrap-edpm-multinodeset-edpm-compute-no-nodes-4cq6w\" (UID: \"a913674d-b6c4-442f-9d33-dab50e694cc4\") " pod="openstack-kuttl-tests/bootstrap-edpm-multinodeset-edpm-compute-no-nodes-4cq6w" Dec 03 13:03:44 crc kubenswrapper[5052]: I1203 13:03:44.906666 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/cbb40c47-9dc0-4934-8964-7fd7a898d250-ssh-key\") pod \"bootstrap-edpm-multinodeset-edpm-compute-beta-nodeset-b28bm\" (UID: \"cbb40c47-9dc0-4934-8964-7fd7a898d250\") " pod="openstack-kuttl-tests/bootstrap-edpm-multinodeset-edpm-compute-beta-nodeset-b28bm" Dec 03 13:03:44 crc kubenswrapper[5052]: I1203 13:03:44.906712 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bootstrap-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/cbb40c47-9dc0-4934-8964-7fd7a898d250-bootstrap-combined-ca-bundle\") pod \"bootstrap-edpm-multinodeset-edpm-compute-beta-nodeset-b28bm\" (UID: \"cbb40c47-9dc0-4934-8964-7fd7a898d250\") " pod="openstack-kuttl-tests/bootstrap-edpm-multinodeset-edpm-compute-beta-nodeset-b28bm" Dec 03 13:03:44 crc kubenswrapper[5052]: I1203 13:03:44.906766 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/a913674d-b6c4-442f-9d33-dab50e694cc4-inventory\") pod \"bootstrap-edpm-multinodeset-edpm-compute-no-nodes-4cq6w\" (UID: \"a913674d-b6c4-442f-9d33-dab50e694cc4\") " pod="openstack-kuttl-tests/bootstrap-edpm-multinodeset-edpm-compute-no-nodes-4cq6w" Dec 03 13:03:44 crc kubenswrapper[5052]: I1203 13:03:44.906802 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/a913674d-b6c4-442f-9d33-dab50e694cc4-ssh-key\") pod \"bootstrap-edpm-multinodeset-edpm-compute-no-nodes-4cq6w\" (UID: \"a913674d-b6c4-442f-9d33-dab50e694cc4\") " pod="openstack-kuttl-tests/bootstrap-edpm-multinodeset-edpm-compute-no-nodes-4cq6w" Dec 03 13:03:44 crc kubenswrapper[5052]: I1203 13:03:44.906884 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/cbb40c47-9dc0-4934-8964-7fd7a898d250-inventory\") pod \"bootstrap-edpm-multinodeset-edpm-compute-beta-nodeset-b28bm\" (UID: \"cbb40c47-9dc0-4934-8964-7fd7a898d250\") " pod="openstack-kuttl-tests/bootstrap-edpm-multinodeset-edpm-compute-beta-nodeset-b28bm" Dec 03 13:03:44 crc kubenswrapper[5052]: I1203 13:03:44.906972 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xjkf7\" (UniqueName: \"kubernetes.io/projected/a913674d-b6c4-442f-9d33-dab50e694cc4-kube-api-access-xjkf7\") pod \"bootstrap-edpm-multinodeset-edpm-compute-no-nodes-4cq6w\" (UID: \"a913674d-b6c4-442f-9d33-dab50e694cc4\") " pod="openstack-kuttl-tests/bootstrap-edpm-multinodeset-edpm-compute-no-nodes-4cq6w" Dec 03 13:03:44 crc kubenswrapper[5052]: I1203 13:03:44.907008 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-twxxh\" (UniqueName: \"kubernetes.io/projected/cbb40c47-9dc0-4934-8964-7fd7a898d250-kube-api-access-twxxh\") pod \"bootstrap-edpm-multinodeset-edpm-compute-beta-nodeset-b28bm\" (UID: \"cbb40c47-9dc0-4934-8964-7fd7a898d250\") " pod="openstack-kuttl-tests/bootstrap-edpm-multinodeset-edpm-compute-beta-nodeset-b28bm" Dec 03 13:03:45 crc kubenswrapper[5052]: I1203 13:03:45.008743 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/cbb40c47-9dc0-4934-8964-7fd7a898d250-inventory\") pod \"bootstrap-edpm-multinodeset-edpm-compute-beta-nodeset-b28bm\" (UID: \"cbb40c47-9dc0-4934-8964-7fd7a898d250\") " pod="openstack-kuttl-tests/bootstrap-edpm-multinodeset-edpm-compute-beta-nodeset-b28bm" Dec 03 13:03:45 crc kubenswrapper[5052]: I1203 13:03:45.009066 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-xjkf7\" (UniqueName: \"kubernetes.io/projected/a913674d-b6c4-442f-9d33-dab50e694cc4-kube-api-access-xjkf7\") pod \"bootstrap-edpm-multinodeset-edpm-compute-no-nodes-4cq6w\" (UID: \"a913674d-b6c4-442f-9d33-dab50e694cc4\") " pod="openstack-kuttl-tests/bootstrap-edpm-multinodeset-edpm-compute-no-nodes-4cq6w" Dec 03 13:03:45 crc kubenswrapper[5052]: I1203 13:03:45.009170 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-twxxh\" (UniqueName: \"kubernetes.io/projected/cbb40c47-9dc0-4934-8964-7fd7a898d250-kube-api-access-twxxh\") pod \"bootstrap-edpm-multinodeset-edpm-compute-beta-nodeset-b28bm\" (UID: \"cbb40c47-9dc0-4934-8964-7fd7a898d250\") " pod="openstack-kuttl-tests/bootstrap-edpm-multinodeset-edpm-compute-beta-nodeset-b28bm" Dec 03 13:03:45 crc kubenswrapper[5052]: I1203 13:03:45.009297 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"bootstrap-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a913674d-b6c4-442f-9d33-dab50e694cc4-bootstrap-combined-ca-bundle\") pod \"bootstrap-edpm-multinodeset-edpm-compute-no-nodes-4cq6w\" (UID: \"a913674d-b6c4-442f-9d33-dab50e694cc4\") " pod="openstack-kuttl-tests/bootstrap-edpm-multinodeset-edpm-compute-no-nodes-4cq6w" Dec 03 13:03:45 crc kubenswrapper[5052]: I1203 13:03:45.009414 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/cbb40c47-9dc0-4934-8964-7fd7a898d250-ssh-key\") pod \"bootstrap-edpm-multinodeset-edpm-compute-beta-nodeset-b28bm\" (UID: \"cbb40c47-9dc0-4934-8964-7fd7a898d250\") " pod="openstack-kuttl-tests/bootstrap-edpm-multinodeset-edpm-compute-beta-nodeset-b28bm" Dec 03 13:03:45 crc kubenswrapper[5052]: I1203 13:03:45.009527 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"bootstrap-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/cbb40c47-9dc0-4934-8964-7fd7a898d250-bootstrap-combined-ca-bundle\") pod \"bootstrap-edpm-multinodeset-edpm-compute-beta-nodeset-b28bm\" (UID: \"cbb40c47-9dc0-4934-8964-7fd7a898d250\") " pod="openstack-kuttl-tests/bootstrap-edpm-multinodeset-edpm-compute-beta-nodeset-b28bm" Dec 03 13:03:45 crc kubenswrapper[5052]: I1203 13:03:45.009649 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/a913674d-b6c4-442f-9d33-dab50e694cc4-inventory\") pod \"bootstrap-edpm-multinodeset-edpm-compute-no-nodes-4cq6w\" (UID: \"a913674d-b6c4-442f-9d33-dab50e694cc4\") " pod="openstack-kuttl-tests/bootstrap-edpm-multinodeset-edpm-compute-no-nodes-4cq6w" Dec 03 13:03:45 crc kubenswrapper[5052]: I1203 13:03:45.009729 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/a913674d-b6c4-442f-9d33-dab50e694cc4-ssh-key\") pod \"bootstrap-edpm-multinodeset-edpm-compute-no-nodes-4cq6w\" (UID: \"a913674d-b6c4-442f-9d33-dab50e694cc4\") " pod="openstack-kuttl-tests/bootstrap-edpm-multinodeset-edpm-compute-no-nodes-4cq6w" Dec 03 13:03:45 crc kubenswrapper[5052]: I1203 13:03:45.012986 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"bootstrap-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a913674d-b6c4-442f-9d33-dab50e694cc4-bootstrap-combined-ca-bundle\") pod \"bootstrap-edpm-multinodeset-edpm-compute-no-nodes-4cq6w\" (UID: \"a913674d-b6c4-442f-9d33-dab50e694cc4\") " pod="openstack-kuttl-tests/bootstrap-edpm-multinodeset-edpm-compute-no-nodes-4cq6w" Dec 03 13:03:45 crc kubenswrapper[5052]: I1203 13:03:45.013287 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/a913674d-b6c4-442f-9d33-dab50e694cc4-ssh-key\") pod \"bootstrap-edpm-multinodeset-edpm-compute-no-nodes-4cq6w\" (UID: \"a913674d-b6c4-442f-9d33-dab50e694cc4\") " pod="openstack-kuttl-tests/bootstrap-edpm-multinodeset-edpm-compute-no-nodes-4cq6w" Dec 03 13:03:45 crc kubenswrapper[5052]: I1203 13:03:45.016083 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/cbb40c47-9dc0-4934-8964-7fd7a898d250-inventory\") pod \"bootstrap-edpm-multinodeset-edpm-compute-beta-nodeset-b28bm\" (UID: \"cbb40c47-9dc0-4934-8964-7fd7a898d250\") " pod="openstack-kuttl-tests/bootstrap-edpm-multinodeset-edpm-compute-beta-nodeset-b28bm" Dec 03 13:03:45 crc kubenswrapper[5052]: I1203 13:03:45.016402 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/a913674d-b6c4-442f-9d33-dab50e694cc4-inventory\") pod \"bootstrap-edpm-multinodeset-edpm-compute-no-nodes-4cq6w\" (UID: \"a913674d-b6c4-442f-9d33-dab50e694cc4\") " pod="openstack-kuttl-tests/bootstrap-edpm-multinodeset-edpm-compute-no-nodes-4cq6w" Dec 03 13:03:45 crc kubenswrapper[5052]: I1203 13:03:45.016701 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"bootstrap-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/cbb40c47-9dc0-4934-8964-7fd7a898d250-bootstrap-combined-ca-bundle\") pod \"bootstrap-edpm-multinodeset-edpm-compute-beta-nodeset-b28bm\" (UID: \"cbb40c47-9dc0-4934-8964-7fd7a898d250\") " pod="openstack-kuttl-tests/bootstrap-edpm-multinodeset-edpm-compute-beta-nodeset-b28bm" Dec 03 13:03:45 crc kubenswrapper[5052]: I1203 13:03:45.017852 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/cbb40c47-9dc0-4934-8964-7fd7a898d250-ssh-key\") pod \"bootstrap-edpm-multinodeset-edpm-compute-beta-nodeset-b28bm\" (UID: \"cbb40c47-9dc0-4934-8964-7fd7a898d250\") " pod="openstack-kuttl-tests/bootstrap-edpm-multinodeset-edpm-compute-beta-nodeset-b28bm" Dec 03 13:03:45 crc kubenswrapper[5052]: I1203 13:03:45.026079 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-xjkf7\" (UniqueName: \"kubernetes.io/projected/a913674d-b6c4-442f-9d33-dab50e694cc4-kube-api-access-xjkf7\") pod \"bootstrap-edpm-multinodeset-edpm-compute-no-nodes-4cq6w\" (UID: \"a913674d-b6c4-442f-9d33-dab50e694cc4\") " pod="openstack-kuttl-tests/bootstrap-edpm-multinodeset-edpm-compute-no-nodes-4cq6w" Dec 03 13:03:45 crc kubenswrapper[5052]: I1203 13:03:45.035257 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-twxxh\" (UniqueName: \"kubernetes.io/projected/cbb40c47-9dc0-4934-8964-7fd7a898d250-kube-api-access-twxxh\") pod \"bootstrap-edpm-multinodeset-edpm-compute-beta-nodeset-b28bm\" (UID: \"cbb40c47-9dc0-4934-8964-7fd7a898d250\") " pod="openstack-kuttl-tests/bootstrap-edpm-multinodeset-edpm-compute-beta-nodeset-b28bm" Dec 03 13:03:45 crc kubenswrapper[5052]: I1203 13:03:45.085733 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/bootstrap-edpm-multinodeset-edpm-compute-beta-nodeset-b28bm" Dec 03 13:03:45 crc kubenswrapper[5052]: I1203 13:03:45.120122 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/bootstrap-edpm-multinodeset-edpm-compute-no-nodes-4cq6w" Dec 03 13:03:45 crc kubenswrapper[5052]: I1203 13:03:45.521176 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/bootstrap-edpm-multinodeset-edpm-compute-beta-nodeset-b28bm"] Dec 03 13:03:45 crc kubenswrapper[5052]: W1203 13:03:45.521950 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcbb40c47_9dc0_4934_8964_7fd7a898d250.slice/crio-08e7209c159cbd6710661399436451a91dbd0aa29157f32a42c253521693a61a WatchSource:0}: Error finding container 08e7209c159cbd6710661399436451a91dbd0aa29157f32a42c253521693a61a: Status 404 returned error can't find the container with id 08e7209c159cbd6710661399436451a91dbd0aa29157f32a42c253521693a61a Dec 03 13:03:45 crc kubenswrapper[5052]: I1203 13:03:45.627344 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/bootstrap-edpm-multinodeset-edpm-compute-no-nodes-4cq6w"] Dec 03 13:03:45 crc kubenswrapper[5052]: I1203 13:03:45.705873 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/bootstrap-edpm-multinodeset-edpm-compute-no-nodes-4cq6w" event={"ID":"a913674d-b6c4-442f-9d33-dab50e694cc4","Type":"ContainerStarted","Data":"2b61b7d07d62e14c0c638607e81a1259cb4fc6b4afee162fcf09b2237dfaf17c"} Dec 03 13:03:45 crc kubenswrapper[5052]: I1203 13:03:45.707362 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/bootstrap-edpm-multinodeset-edpm-compute-beta-nodeset-b28bm" event={"ID":"cbb40c47-9dc0-4934-8964-7fd7a898d250","Type":"ContainerStarted","Data":"08e7209c159cbd6710661399436451a91dbd0aa29157f32a42c253521693a61a"} Dec 03 13:03:46 crc kubenswrapper[5052]: I1203 13:03:46.715969 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/bootstrap-edpm-multinodeset-edpm-compute-beta-nodeset-b28bm" event={"ID":"cbb40c47-9dc0-4934-8964-7fd7a898d250","Type":"ContainerStarted","Data":"29cabd7861943ebda29291906beab8165e925c6aaf78701fe45479cfc3e5387e"} Dec 03 13:03:46 crc kubenswrapper[5052]: I1203 13:03:46.717724 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/bootstrap-edpm-multinodeset-edpm-compute-no-nodes-4cq6w" event={"ID":"a913674d-b6c4-442f-9d33-dab50e694cc4","Type":"ContainerStarted","Data":"f1f0febd18f3eaa033195f2f7e941a98439672c1a789fc20f5e73209a2dc90d8"} Dec 03 13:03:46 crc kubenswrapper[5052]: I1203 13:03:46.732216 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/bootstrap-edpm-multinodeset-edpm-compute-beta-nodeset-b28bm" podStartSLOduration=2.040466332 podStartE2EDuration="2.732195104s" podCreationTimestamp="2025-12-03 13:03:44 +0000 UTC" firstStartedPulling="2025-12-03 13:03:45.52462861 +0000 UTC m=+6216.112870374" lastFinishedPulling="2025-12-03 13:03:46.216357372 +0000 UTC m=+6216.804599146" observedRunningTime="2025-12-03 13:03:46.729242529 +0000 UTC m=+6217.317484303" watchObservedRunningTime="2025-12-03 13:03:46.732195104 +0000 UTC m=+6217.320436868" Dec 03 13:03:46 crc kubenswrapper[5052]: I1203 13:03:46.764663 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/bootstrap-edpm-multinodeset-edpm-compute-no-nodes-4cq6w" podStartSLOduration=2.160498224 podStartE2EDuration="2.764639301s" podCreationTimestamp="2025-12-03 13:03:44 +0000 UTC" firstStartedPulling="2025-12-03 13:03:45.627279576 +0000 UTC m=+6216.215521340" lastFinishedPulling="2025-12-03 13:03:46.231420653 +0000 UTC m=+6216.819662417" observedRunningTime="2025-12-03 13:03:46.754261245 +0000 UTC m=+6217.342503019" watchObservedRunningTime="2025-12-03 13:03:46.764639301 +0000 UTC m=+6217.352881065" Dec 03 13:03:48 crc kubenswrapper[5052]: I1203 13:03:48.740487 5052 generic.go:334] "Generic (PLEG): container finished" podID="a913674d-b6c4-442f-9d33-dab50e694cc4" containerID="f1f0febd18f3eaa033195f2f7e941a98439672c1a789fc20f5e73209a2dc90d8" exitCode=0 Dec 03 13:03:48 crc kubenswrapper[5052]: I1203 13:03:48.740589 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/bootstrap-edpm-multinodeset-edpm-compute-no-nodes-4cq6w" event={"ID":"a913674d-b6c4-442f-9d33-dab50e694cc4","Type":"ContainerDied","Data":"f1f0febd18f3eaa033195f2f7e941a98439672c1a789fc20f5e73209a2dc90d8"} Dec 03 13:03:48 crc kubenswrapper[5052]: I1203 13:03:48.743635 5052 generic.go:334] "Generic (PLEG): container finished" podID="cbb40c47-9dc0-4934-8964-7fd7a898d250" containerID="29cabd7861943ebda29291906beab8165e925c6aaf78701fe45479cfc3e5387e" exitCode=0 Dec 03 13:03:48 crc kubenswrapper[5052]: I1203 13:03:48.743681 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/bootstrap-edpm-multinodeset-edpm-compute-beta-nodeset-b28bm" event={"ID":"cbb40c47-9dc0-4934-8964-7fd7a898d250","Type":"ContainerDied","Data":"29cabd7861943ebda29291906beab8165e925c6aaf78701fe45479cfc3e5387e"} Dec 03 13:03:50 crc kubenswrapper[5052]: I1203 13:03:50.124243 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/bootstrap-edpm-multinodeset-edpm-compute-beta-nodeset-b28bm" Dec 03 13:03:50 crc kubenswrapper[5052]: I1203 13:03:50.134114 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/bootstrap-edpm-multinodeset-edpm-compute-no-nodes-4cq6w" Dec 03 13:03:50 crc kubenswrapper[5052]: I1203 13:03:50.304002 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xjkf7\" (UniqueName: \"kubernetes.io/projected/a913674d-b6c4-442f-9d33-dab50e694cc4-kube-api-access-xjkf7\") pod \"a913674d-b6c4-442f-9d33-dab50e694cc4\" (UID: \"a913674d-b6c4-442f-9d33-dab50e694cc4\") " Dec 03 13:03:50 crc kubenswrapper[5052]: I1203 13:03:50.304066 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-twxxh\" (UniqueName: \"kubernetes.io/projected/cbb40c47-9dc0-4934-8964-7fd7a898d250-kube-api-access-twxxh\") pod \"cbb40c47-9dc0-4934-8964-7fd7a898d250\" (UID: \"cbb40c47-9dc0-4934-8964-7fd7a898d250\") " Dec 03 13:03:50 crc kubenswrapper[5052]: I1203 13:03:50.304101 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/a913674d-b6c4-442f-9d33-dab50e694cc4-ssh-key\") pod \"a913674d-b6c4-442f-9d33-dab50e694cc4\" (UID: \"a913674d-b6c4-442f-9d33-dab50e694cc4\") " Dec 03 13:03:50 crc kubenswrapper[5052]: I1203 13:03:50.304167 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"bootstrap-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a913674d-b6c4-442f-9d33-dab50e694cc4-bootstrap-combined-ca-bundle\") pod \"a913674d-b6c4-442f-9d33-dab50e694cc4\" (UID: \"a913674d-b6c4-442f-9d33-dab50e694cc4\") " Dec 03 13:03:50 crc kubenswrapper[5052]: I1203 13:03:50.304207 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/a913674d-b6c4-442f-9d33-dab50e694cc4-inventory\") pod \"a913674d-b6c4-442f-9d33-dab50e694cc4\" (UID: \"a913674d-b6c4-442f-9d33-dab50e694cc4\") " Dec 03 13:03:50 crc kubenswrapper[5052]: I1203 13:03:50.304263 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"bootstrap-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/cbb40c47-9dc0-4934-8964-7fd7a898d250-bootstrap-combined-ca-bundle\") pod \"cbb40c47-9dc0-4934-8964-7fd7a898d250\" (UID: \"cbb40c47-9dc0-4934-8964-7fd7a898d250\") " Dec 03 13:03:50 crc kubenswrapper[5052]: I1203 13:03:50.304326 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/cbb40c47-9dc0-4934-8964-7fd7a898d250-ssh-key\") pod \"cbb40c47-9dc0-4934-8964-7fd7a898d250\" (UID: \"cbb40c47-9dc0-4934-8964-7fd7a898d250\") " Dec 03 13:03:50 crc kubenswrapper[5052]: I1203 13:03:50.304365 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/cbb40c47-9dc0-4934-8964-7fd7a898d250-inventory\") pod \"cbb40c47-9dc0-4934-8964-7fd7a898d250\" (UID: \"cbb40c47-9dc0-4934-8964-7fd7a898d250\") " Dec 03 13:03:50 crc kubenswrapper[5052]: I1203 13:03:50.309911 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cbb40c47-9dc0-4934-8964-7fd7a898d250-bootstrap-combined-ca-bundle" (OuterVolumeSpecName: "bootstrap-combined-ca-bundle") pod "cbb40c47-9dc0-4934-8964-7fd7a898d250" (UID: "cbb40c47-9dc0-4934-8964-7fd7a898d250"). InnerVolumeSpecName "bootstrap-combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:03:50 crc kubenswrapper[5052]: I1203 13:03:50.309977 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cbb40c47-9dc0-4934-8964-7fd7a898d250-kube-api-access-twxxh" (OuterVolumeSpecName: "kube-api-access-twxxh") pod "cbb40c47-9dc0-4934-8964-7fd7a898d250" (UID: "cbb40c47-9dc0-4934-8964-7fd7a898d250"). InnerVolumeSpecName "kube-api-access-twxxh". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:03:50 crc kubenswrapper[5052]: I1203 13:03:50.310750 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a913674d-b6c4-442f-9d33-dab50e694cc4-kube-api-access-xjkf7" (OuterVolumeSpecName: "kube-api-access-xjkf7") pod "a913674d-b6c4-442f-9d33-dab50e694cc4" (UID: "a913674d-b6c4-442f-9d33-dab50e694cc4"). InnerVolumeSpecName "kube-api-access-xjkf7". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:03:50 crc kubenswrapper[5052]: I1203 13:03:50.311431 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a913674d-b6c4-442f-9d33-dab50e694cc4-bootstrap-combined-ca-bundle" (OuterVolumeSpecName: "bootstrap-combined-ca-bundle") pod "a913674d-b6c4-442f-9d33-dab50e694cc4" (UID: "a913674d-b6c4-442f-9d33-dab50e694cc4"). InnerVolumeSpecName "bootstrap-combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:03:50 crc kubenswrapper[5052]: I1203 13:03:50.331427 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cbb40c47-9dc0-4934-8964-7fd7a898d250-ssh-key" (OuterVolumeSpecName: "ssh-key") pod "cbb40c47-9dc0-4934-8964-7fd7a898d250" (UID: "cbb40c47-9dc0-4934-8964-7fd7a898d250"). InnerVolumeSpecName "ssh-key". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:03:50 crc kubenswrapper[5052]: I1203 13:03:50.331477 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cbb40c47-9dc0-4934-8964-7fd7a898d250-inventory" (OuterVolumeSpecName: "inventory") pod "cbb40c47-9dc0-4934-8964-7fd7a898d250" (UID: "cbb40c47-9dc0-4934-8964-7fd7a898d250"). InnerVolumeSpecName "inventory". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:03:50 crc kubenswrapper[5052]: I1203 13:03:50.331556 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a913674d-b6c4-442f-9d33-dab50e694cc4-inventory" (OuterVolumeSpecName: "inventory") pod "a913674d-b6c4-442f-9d33-dab50e694cc4" (UID: "a913674d-b6c4-442f-9d33-dab50e694cc4"). InnerVolumeSpecName "inventory". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:03:50 crc kubenswrapper[5052]: I1203 13:03:50.336400 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a913674d-b6c4-442f-9d33-dab50e694cc4-ssh-key" (OuterVolumeSpecName: "ssh-key") pod "a913674d-b6c4-442f-9d33-dab50e694cc4" (UID: "a913674d-b6c4-442f-9d33-dab50e694cc4"). InnerVolumeSpecName "ssh-key". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:03:50 crc kubenswrapper[5052]: I1203 13:03:50.405968 5052 reconciler_common.go:293] "Volume detached for volume \"bootstrap-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a913674d-b6c4-442f-9d33-dab50e694cc4-bootstrap-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 13:03:50 crc kubenswrapper[5052]: I1203 13:03:50.406010 5052 reconciler_common.go:293] "Volume detached for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/a913674d-b6c4-442f-9d33-dab50e694cc4-inventory\") on node \"crc\" DevicePath \"\"" Dec 03 13:03:50 crc kubenswrapper[5052]: I1203 13:03:50.406023 5052 reconciler_common.go:293] "Volume detached for volume \"bootstrap-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/cbb40c47-9dc0-4934-8964-7fd7a898d250-bootstrap-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 13:03:50 crc kubenswrapper[5052]: I1203 13:03:50.406034 5052 reconciler_common.go:293] "Volume detached for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/cbb40c47-9dc0-4934-8964-7fd7a898d250-ssh-key\") on node \"crc\" DevicePath \"\"" Dec 03 13:03:50 crc kubenswrapper[5052]: I1203 13:03:50.406058 5052 reconciler_common.go:293] "Volume detached for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/cbb40c47-9dc0-4934-8964-7fd7a898d250-inventory\") on node \"crc\" DevicePath \"\"" Dec 03 13:03:50 crc kubenswrapper[5052]: I1203 13:03:50.406072 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-xjkf7\" (UniqueName: \"kubernetes.io/projected/a913674d-b6c4-442f-9d33-dab50e694cc4-kube-api-access-xjkf7\") on node \"crc\" DevicePath \"\"" Dec 03 13:03:50 crc kubenswrapper[5052]: I1203 13:03:50.406083 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-twxxh\" (UniqueName: \"kubernetes.io/projected/cbb40c47-9dc0-4934-8964-7fd7a898d250-kube-api-access-twxxh\") on node \"crc\" DevicePath \"\"" Dec 03 13:03:50 crc kubenswrapper[5052]: I1203 13:03:50.406094 5052 reconciler_common.go:293] "Volume detached for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/a913674d-b6c4-442f-9d33-dab50e694cc4-ssh-key\") on node \"crc\" DevicePath \"\"" Dec 03 13:03:50 crc kubenswrapper[5052]: I1203 13:03:50.762432 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/bootstrap-edpm-multinodeset-edpm-compute-no-nodes-4cq6w" event={"ID":"a913674d-b6c4-442f-9d33-dab50e694cc4","Type":"ContainerDied","Data":"2b61b7d07d62e14c0c638607e81a1259cb4fc6b4afee162fcf09b2237dfaf17c"} Dec 03 13:03:50 crc kubenswrapper[5052]: I1203 13:03:50.762794 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2b61b7d07d62e14c0c638607e81a1259cb4fc6b4afee162fcf09b2237dfaf17c" Dec 03 13:03:50 crc kubenswrapper[5052]: I1203 13:03:50.762461 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/bootstrap-edpm-multinodeset-edpm-compute-no-nodes-4cq6w" Dec 03 13:03:50 crc kubenswrapper[5052]: I1203 13:03:50.764227 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/bootstrap-edpm-multinodeset-edpm-compute-beta-nodeset-b28bm" event={"ID":"cbb40c47-9dc0-4934-8964-7fd7a898d250","Type":"ContainerDied","Data":"08e7209c159cbd6710661399436451a91dbd0aa29157f32a42c253521693a61a"} Dec 03 13:03:50 crc kubenswrapper[5052]: I1203 13:03:50.764267 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="08e7209c159cbd6710661399436451a91dbd0aa29157f32a42c253521693a61a" Dec 03 13:03:50 crc kubenswrapper[5052]: I1203 13:03:50.764291 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/bootstrap-edpm-multinodeset-edpm-compute-beta-nodeset-b28bm" Dec 03 13:03:50 crc kubenswrapper[5052]: I1203 13:03:50.853560 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/configure-network-edpm-multinodeset-edpm-compute-no-nodes-v5m6s"] Dec 03 13:03:50 crc kubenswrapper[5052]: E1203 13:03:50.854262 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="cbb40c47-9dc0-4934-8964-7fd7a898d250" containerName="bootstrap-edpm-multinodeset-edpm-compute-beta-nodeset" Dec 03 13:03:50 crc kubenswrapper[5052]: I1203 13:03:50.854363 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="cbb40c47-9dc0-4934-8964-7fd7a898d250" containerName="bootstrap-edpm-multinodeset-edpm-compute-beta-nodeset" Dec 03 13:03:50 crc kubenswrapper[5052]: E1203 13:03:50.854470 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="a913674d-b6c4-442f-9d33-dab50e694cc4" containerName="bootstrap-edpm-multinodeset-edpm-compute-no-nodes" Dec 03 13:03:50 crc kubenswrapper[5052]: I1203 13:03:50.854561 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="a913674d-b6c4-442f-9d33-dab50e694cc4" containerName="bootstrap-edpm-multinodeset-edpm-compute-no-nodes" Dec 03 13:03:50 crc kubenswrapper[5052]: I1203 13:03:50.854817 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="cbb40c47-9dc0-4934-8964-7fd7a898d250" containerName="bootstrap-edpm-multinodeset-edpm-compute-beta-nodeset" Dec 03 13:03:50 crc kubenswrapper[5052]: I1203 13:03:50.854902 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="a913674d-b6c4-442f-9d33-dab50e694cc4" containerName="bootstrap-edpm-multinodeset-edpm-compute-no-nodes" Dec 03 13:03:50 crc kubenswrapper[5052]: I1203 13:03:50.855605 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/configure-network-edpm-multinodeset-edpm-compute-no-nodes-v5m6s" Dec 03 13:03:50 crc kubenswrapper[5052]: I1203 13:03:50.858355 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"openstack-aee-default-env" Dec 03 13:03:50 crc kubenswrapper[5052]: I1203 13:03:50.858383 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"dataplane-ansible-ssh-private-key-secret" Dec 03 13:03:50 crc kubenswrapper[5052]: I1203 13:03:50.858669 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"dataplanenodeset-edpm-compute-no-nodes" Dec 03 13:03:50 crc kubenswrapper[5052]: I1203 13:03:50.859749 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"edpm-compute-no-nodes-dockercfg-mkcgv" Dec 03 13:03:50 crc kubenswrapper[5052]: I1203 13:03:50.886192 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/configure-network-edpm-multinodeset-edpm-compute-no-nodes-v5m6s"] Dec 03 13:03:51 crc kubenswrapper[5052]: I1203 13:03:51.017966 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lgtvp\" (UniqueName: \"kubernetes.io/projected/5f61d52a-bcc2-43f1-bcc5-076fa075a3a1-kube-api-access-lgtvp\") pod \"configure-network-edpm-multinodeset-edpm-compute-no-nodes-v5m6s\" (UID: \"5f61d52a-bcc2-43f1-bcc5-076fa075a3a1\") " pod="openstack-kuttl-tests/configure-network-edpm-multinodeset-edpm-compute-no-nodes-v5m6s" Dec 03 13:03:51 crc kubenswrapper[5052]: I1203 13:03:51.018202 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/5f61d52a-bcc2-43f1-bcc5-076fa075a3a1-ssh-key\") pod \"configure-network-edpm-multinodeset-edpm-compute-no-nodes-v5m6s\" (UID: \"5f61d52a-bcc2-43f1-bcc5-076fa075a3a1\") " pod="openstack-kuttl-tests/configure-network-edpm-multinodeset-edpm-compute-no-nodes-v5m6s" Dec 03 13:03:51 crc kubenswrapper[5052]: I1203 13:03:51.018268 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/5f61d52a-bcc2-43f1-bcc5-076fa075a3a1-inventory\") pod \"configure-network-edpm-multinodeset-edpm-compute-no-nodes-v5m6s\" (UID: \"5f61d52a-bcc2-43f1-bcc5-076fa075a3a1\") " pod="openstack-kuttl-tests/configure-network-edpm-multinodeset-edpm-compute-no-nodes-v5m6s" Dec 03 13:03:51 crc kubenswrapper[5052]: I1203 13:03:51.119499 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-lgtvp\" (UniqueName: \"kubernetes.io/projected/5f61d52a-bcc2-43f1-bcc5-076fa075a3a1-kube-api-access-lgtvp\") pod \"configure-network-edpm-multinodeset-edpm-compute-no-nodes-v5m6s\" (UID: \"5f61d52a-bcc2-43f1-bcc5-076fa075a3a1\") " pod="openstack-kuttl-tests/configure-network-edpm-multinodeset-edpm-compute-no-nodes-v5m6s" Dec 03 13:03:51 crc kubenswrapper[5052]: I1203 13:03:51.119661 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/5f61d52a-bcc2-43f1-bcc5-076fa075a3a1-ssh-key\") pod \"configure-network-edpm-multinodeset-edpm-compute-no-nodes-v5m6s\" (UID: \"5f61d52a-bcc2-43f1-bcc5-076fa075a3a1\") " pod="openstack-kuttl-tests/configure-network-edpm-multinodeset-edpm-compute-no-nodes-v5m6s" Dec 03 13:03:51 crc kubenswrapper[5052]: I1203 13:03:51.119693 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/5f61d52a-bcc2-43f1-bcc5-076fa075a3a1-inventory\") pod \"configure-network-edpm-multinodeset-edpm-compute-no-nodes-v5m6s\" (UID: \"5f61d52a-bcc2-43f1-bcc5-076fa075a3a1\") " pod="openstack-kuttl-tests/configure-network-edpm-multinodeset-edpm-compute-no-nodes-v5m6s" Dec 03 13:03:51 crc kubenswrapper[5052]: I1203 13:03:51.124387 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/5f61d52a-bcc2-43f1-bcc5-076fa075a3a1-ssh-key\") pod \"configure-network-edpm-multinodeset-edpm-compute-no-nodes-v5m6s\" (UID: \"5f61d52a-bcc2-43f1-bcc5-076fa075a3a1\") " pod="openstack-kuttl-tests/configure-network-edpm-multinodeset-edpm-compute-no-nodes-v5m6s" Dec 03 13:03:51 crc kubenswrapper[5052]: I1203 13:03:51.125156 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/5f61d52a-bcc2-43f1-bcc5-076fa075a3a1-inventory\") pod \"configure-network-edpm-multinodeset-edpm-compute-no-nodes-v5m6s\" (UID: \"5f61d52a-bcc2-43f1-bcc5-076fa075a3a1\") " pod="openstack-kuttl-tests/configure-network-edpm-multinodeset-edpm-compute-no-nodes-v5m6s" Dec 03 13:03:51 crc kubenswrapper[5052]: I1203 13:03:51.140174 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-lgtvp\" (UniqueName: \"kubernetes.io/projected/5f61d52a-bcc2-43f1-bcc5-076fa075a3a1-kube-api-access-lgtvp\") pod \"configure-network-edpm-multinodeset-edpm-compute-no-nodes-v5m6s\" (UID: \"5f61d52a-bcc2-43f1-bcc5-076fa075a3a1\") " pod="openstack-kuttl-tests/configure-network-edpm-multinodeset-edpm-compute-no-nodes-v5m6s" Dec 03 13:03:51 crc kubenswrapper[5052]: I1203 13:03:51.172916 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/configure-network-edpm-multinodeset-edpm-compute-no-nodes-v5m6s" Dec 03 13:03:51 crc kubenswrapper[5052]: W1203 13:03:51.577679 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5f61d52a_bcc2_43f1_bcc5_076fa075a3a1.slice/crio-465bbc5d27687d6c1075c432415cf97db52bb1b49c1fa34185cf208d83e02263 WatchSource:0}: Error finding container 465bbc5d27687d6c1075c432415cf97db52bb1b49c1fa34185cf208d83e02263: Status 404 returned error can't find the container with id 465bbc5d27687d6c1075c432415cf97db52bb1b49c1fa34185cf208d83e02263 Dec 03 13:03:51 crc kubenswrapper[5052]: I1203 13:03:51.578743 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/configure-network-edpm-multinodeset-edpm-compute-no-nodes-v5m6s"] Dec 03 13:03:51 crc kubenswrapper[5052]: I1203 13:03:51.773260 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/configure-network-edpm-multinodeset-edpm-compute-no-nodes-v5m6s" event={"ID":"5f61d52a-bcc2-43f1-bcc5-076fa075a3a1","Type":"ContainerStarted","Data":"465bbc5d27687d6c1075c432415cf97db52bb1b49c1fa34185cf208d83e02263"} Dec 03 13:03:52 crc kubenswrapper[5052]: I1203 13:03:52.782807 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/configure-network-edpm-multinodeset-edpm-compute-no-nodes-v5m6s" event={"ID":"5f61d52a-bcc2-43f1-bcc5-076fa075a3a1","Type":"ContainerStarted","Data":"3f496c994d26bdb760c4659da1eaac2a9c50c8691b6b7b40bfd130a5a5621039"} Dec 03 13:03:52 crc kubenswrapper[5052]: I1203 13:03:52.797492 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/configure-network-edpm-multinodeset-edpm-compute-no-nodes-v5m6s" podStartSLOduration=2.340617355 podStartE2EDuration="2.79747295s" podCreationTimestamp="2025-12-03 13:03:50 +0000 UTC" firstStartedPulling="2025-12-03 13:03:51.58517224 +0000 UTC m=+6222.173414004" lastFinishedPulling="2025-12-03 13:03:52.042027835 +0000 UTC m=+6222.630269599" observedRunningTime="2025-12-03 13:03:52.795204805 +0000 UTC m=+6223.383446569" watchObservedRunningTime="2025-12-03 13:03:52.79747295 +0000 UTC m=+6223.385714714" Dec 03 13:03:53 crc kubenswrapper[5052]: I1203 13:03:53.792777 5052 generic.go:334] "Generic (PLEG): container finished" podID="5f61d52a-bcc2-43f1-bcc5-076fa075a3a1" containerID="3f496c994d26bdb760c4659da1eaac2a9c50c8691b6b7b40bfd130a5a5621039" exitCode=0 Dec 03 13:03:53 crc kubenswrapper[5052]: I1203 13:03:53.792836 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/configure-network-edpm-multinodeset-edpm-compute-no-nodes-v5m6s" event={"ID":"5f61d52a-bcc2-43f1-bcc5-076fa075a3a1","Type":"ContainerDied","Data":"3f496c994d26bdb760c4659da1eaac2a9c50c8691b6b7b40bfd130a5a5621039"} Dec 03 13:03:55 crc kubenswrapper[5052]: I1203 13:03:55.081245 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/configure-network-edpm-multinodeset-edpm-compute-no-nodes-v5m6s" Dec 03 13:03:55 crc kubenswrapper[5052]: I1203 13:03:55.175148 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lgtvp\" (UniqueName: \"kubernetes.io/projected/5f61d52a-bcc2-43f1-bcc5-076fa075a3a1-kube-api-access-lgtvp\") pod \"5f61d52a-bcc2-43f1-bcc5-076fa075a3a1\" (UID: \"5f61d52a-bcc2-43f1-bcc5-076fa075a3a1\") " Dec 03 13:03:55 crc kubenswrapper[5052]: I1203 13:03:55.175278 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/5f61d52a-bcc2-43f1-bcc5-076fa075a3a1-ssh-key\") pod \"5f61d52a-bcc2-43f1-bcc5-076fa075a3a1\" (UID: \"5f61d52a-bcc2-43f1-bcc5-076fa075a3a1\") " Dec 03 13:03:55 crc kubenswrapper[5052]: I1203 13:03:55.175311 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/5f61d52a-bcc2-43f1-bcc5-076fa075a3a1-inventory\") pod \"5f61d52a-bcc2-43f1-bcc5-076fa075a3a1\" (UID: \"5f61d52a-bcc2-43f1-bcc5-076fa075a3a1\") " Dec 03 13:03:55 crc kubenswrapper[5052]: I1203 13:03:55.181539 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5f61d52a-bcc2-43f1-bcc5-076fa075a3a1-kube-api-access-lgtvp" (OuterVolumeSpecName: "kube-api-access-lgtvp") pod "5f61d52a-bcc2-43f1-bcc5-076fa075a3a1" (UID: "5f61d52a-bcc2-43f1-bcc5-076fa075a3a1"). InnerVolumeSpecName "kube-api-access-lgtvp". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:03:55 crc kubenswrapper[5052]: I1203 13:03:55.197896 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5f61d52a-bcc2-43f1-bcc5-076fa075a3a1-ssh-key" (OuterVolumeSpecName: "ssh-key") pod "5f61d52a-bcc2-43f1-bcc5-076fa075a3a1" (UID: "5f61d52a-bcc2-43f1-bcc5-076fa075a3a1"). InnerVolumeSpecName "ssh-key". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:03:55 crc kubenswrapper[5052]: I1203 13:03:55.200749 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5f61d52a-bcc2-43f1-bcc5-076fa075a3a1-inventory" (OuterVolumeSpecName: "inventory") pod "5f61d52a-bcc2-43f1-bcc5-076fa075a3a1" (UID: "5f61d52a-bcc2-43f1-bcc5-076fa075a3a1"). InnerVolumeSpecName "inventory". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:03:55 crc kubenswrapper[5052]: I1203 13:03:55.278148 5052 reconciler_common.go:293] "Volume detached for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/5f61d52a-bcc2-43f1-bcc5-076fa075a3a1-ssh-key\") on node \"crc\" DevicePath \"\"" Dec 03 13:03:55 crc kubenswrapper[5052]: I1203 13:03:55.278194 5052 reconciler_common.go:293] "Volume detached for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/5f61d52a-bcc2-43f1-bcc5-076fa075a3a1-inventory\") on node \"crc\" DevicePath \"\"" Dec 03 13:03:55 crc kubenswrapper[5052]: I1203 13:03:55.278212 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-lgtvp\" (UniqueName: \"kubernetes.io/projected/5f61d52a-bcc2-43f1-bcc5-076fa075a3a1-kube-api-access-lgtvp\") on node \"crc\" DevicePath \"\"" Dec 03 13:03:55 crc kubenswrapper[5052]: I1203 13:03:55.814700 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/configure-network-edpm-multinodeset-edpm-compute-no-nodes-v5m6s" event={"ID":"5f61d52a-bcc2-43f1-bcc5-076fa075a3a1","Type":"ContainerDied","Data":"465bbc5d27687d6c1075c432415cf97db52bb1b49c1fa34185cf208d83e02263"} Dec 03 13:03:55 crc kubenswrapper[5052]: I1203 13:03:55.814750 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="465bbc5d27687d6c1075c432415cf97db52bb1b49c1fa34185cf208d83e02263" Dec 03 13:03:55 crc kubenswrapper[5052]: I1203 13:03:55.814815 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/configure-network-edpm-multinodeset-edpm-compute-no-nodes-v5m6s" Dec 03 13:03:55 crc kubenswrapper[5052]: I1203 13:03:55.898243 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/validate-network-edpm-multinodeset-edpm-compute-no-nodes-xqlzc"] Dec 03 13:03:55 crc kubenswrapper[5052]: E1203 13:03:55.900279 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="5f61d52a-bcc2-43f1-bcc5-076fa075a3a1" containerName="configure-network-edpm-multinodeset-edpm-compute-no-nodes" Dec 03 13:03:55 crc kubenswrapper[5052]: I1203 13:03:55.900317 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f61d52a-bcc2-43f1-bcc5-076fa075a3a1" containerName="configure-network-edpm-multinodeset-edpm-compute-no-nodes" Dec 03 13:03:55 crc kubenswrapper[5052]: I1203 13:03:55.900565 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="5f61d52a-bcc2-43f1-bcc5-076fa075a3a1" containerName="configure-network-edpm-multinodeset-edpm-compute-no-nodes" Dec 03 13:03:55 crc kubenswrapper[5052]: I1203 13:03:55.901317 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/validate-network-edpm-multinodeset-edpm-compute-no-nodes-xqlzc" Dec 03 13:03:55 crc kubenswrapper[5052]: I1203 13:03:55.903854 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"dataplanenodeset-edpm-compute-no-nodes" Dec 03 13:03:55 crc kubenswrapper[5052]: I1203 13:03:55.905496 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"dataplane-ansible-ssh-private-key-secret" Dec 03 13:03:55 crc kubenswrapper[5052]: I1203 13:03:55.905631 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"edpm-compute-no-nodes-dockercfg-mkcgv" Dec 03 13:03:55 crc kubenswrapper[5052]: I1203 13:03:55.905978 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"openstack-aee-default-env" Dec 03 13:03:55 crc kubenswrapper[5052]: I1203 13:03:55.926184 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/validate-network-edpm-multinodeset-edpm-compute-no-nodes-xqlzc"] Dec 03 13:03:55 crc kubenswrapper[5052]: I1203 13:03:55.990190 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dmkxz\" (UniqueName: \"kubernetes.io/projected/07266f43-def2-46f4-8a0f-c856ec3d030b-kube-api-access-dmkxz\") pod \"validate-network-edpm-multinodeset-edpm-compute-no-nodes-xqlzc\" (UID: \"07266f43-def2-46f4-8a0f-c856ec3d030b\") " pod="openstack-kuttl-tests/validate-network-edpm-multinodeset-edpm-compute-no-nodes-xqlzc" Dec 03 13:03:55 crc kubenswrapper[5052]: I1203 13:03:55.990302 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/07266f43-def2-46f4-8a0f-c856ec3d030b-ssh-key\") pod \"validate-network-edpm-multinodeset-edpm-compute-no-nodes-xqlzc\" (UID: \"07266f43-def2-46f4-8a0f-c856ec3d030b\") " pod="openstack-kuttl-tests/validate-network-edpm-multinodeset-edpm-compute-no-nodes-xqlzc" Dec 03 13:03:55 crc kubenswrapper[5052]: I1203 13:03:55.990466 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/07266f43-def2-46f4-8a0f-c856ec3d030b-inventory\") pod \"validate-network-edpm-multinodeset-edpm-compute-no-nodes-xqlzc\" (UID: \"07266f43-def2-46f4-8a0f-c856ec3d030b\") " pod="openstack-kuttl-tests/validate-network-edpm-multinodeset-edpm-compute-no-nodes-xqlzc" Dec 03 13:03:56 crc kubenswrapper[5052]: I1203 13:03:56.092460 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/07266f43-def2-46f4-8a0f-c856ec3d030b-ssh-key\") pod \"validate-network-edpm-multinodeset-edpm-compute-no-nodes-xqlzc\" (UID: \"07266f43-def2-46f4-8a0f-c856ec3d030b\") " pod="openstack-kuttl-tests/validate-network-edpm-multinodeset-edpm-compute-no-nodes-xqlzc" Dec 03 13:03:56 crc kubenswrapper[5052]: I1203 13:03:56.092795 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/07266f43-def2-46f4-8a0f-c856ec3d030b-inventory\") pod \"validate-network-edpm-multinodeset-edpm-compute-no-nodes-xqlzc\" (UID: \"07266f43-def2-46f4-8a0f-c856ec3d030b\") " pod="openstack-kuttl-tests/validate-network-edpm-multinodeset-edpm-compute-no-nodes-xqlzc" Dec 03 13:03:56 crc kubenswrapper[5052]: I1203 13:03:56.092952 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-dmkxz\" (UniqueName: \"kubernetes.io/projected/07266f43-def2-46f4-8a0f-c856ec3d030b-kube-api-access-dmkxz\") pod \"validate-network-edpm-multinodeset-edpm-compute-no-nodes-xqlzc\" (UID: \"07266f43-def2-46f4-8a0f-c856ec3d030b\") " pod="openstack-kuttl-tests/validate-network-edpm-multinodeset-edpm-compute-no-nodes-xqlzc" Dec 03 13:03:56 crc kubenswrapper[5052]: I1203 13:03:56.097974 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/07266f43-def2-46f4-8a0f-c856ec3d030b-ssh-key\") pod \"validate-network-edpm-multinodeset-edpm-compute-no-nodes-xqlzc\" (UID: \"07266f43-def2-46f4-8a0f-c856ec3d030b\") " pod="openstack-kuttl-tests/validate-network-edpm-multinodeset-edpm-compute-no-nodes-xqlzc" Dec 03 13:03:56 crc kubenswrapper[5052]: I1203 13:03:56.106165 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/07266f43-def2-46f4-8a0f-c856ec3d030b-inventory\") pod \"validate-network-edpm-multinodeset-edpm-compute-no-nodes-xqlzc\" (UID: \"07266f43-def2-46f4-8a0f-c856ec3d030b\") " pod="openstack-kuttl-tests/validate-network-edpm-multinodeset-edpm-compute-no-nodes-xqlzc" Dec 03 13:03:56 crc kubenswrapper[5052]: I1203 13:03:56.109465 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-dmkxz\" (UniqueName: \"kubernetes.io/projected/07266f43-def2-46f4-8a0f-c856ec3d030b-kube-api-access-dmkxz\") pod \"validate-network-edpm-multinodeset-edpm-compute-no-nodes-xqlzc\" (UID: \"07266f43-def2-46f4-8a0f-c856ec3d030b\") " pod="openstack-kuttl-tests/validate-network-edpm-multinodeset-edpm-compute-no-nodes-xqlzc" Dec 03 13:03:56 crc kubenswrapper[5052]: I1203 13:03:56.221909 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/validate-network-edpm-multinodeset-edpm-compute-no-nodes-xqlzc" Dec 03 13:03:56 crc kubenswrapper[5052]: I1203 13:03:56.635477 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/validate-network-edpm-multinodeset-edpm-compute-no-nodes-xqlzc"] Dec 03 13:03:56 crc kubenswrapper[5052]: W1203 13:03:56.637384 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod07266f43_def2_46f4_8a0f_c856ec3d030b.slice/crio-ad93d0f15acf982fd81efe1b1124e1171b0d20f79d7e0143c34021f1d5e623c8 WatchSource:0}: Error finding container ad93d0f15acf982fd81efe1b1124e1171b0d20f79d7e0143c34021f1d5e623c8: Status 404 returned error can't find the container with id ad93d0f15acf982fd81efe1b1124e1171b0d20f79d7e0143c34021f1d5e623c8 Dec 03 13:03:56 crc kubenswrapper[5052]: I1203 13:03:56.823658 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/validate-network-edpm-multinodeset-edpm-compute-no-nodes-xqlzc" event={"ID":"07266f43-def2-46f4-8a0f-c856ec3d030b","Type":"ContainerStarted","Data":"ad93d0f15acf982fd81efe1b1124e1171b0d20f79d7e0143c34021f1d5e623c8"} Dec 03 13:03:57 crc kubenswrapper[5052]: I1203 13:03:57.838849 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/validate-network-edpm-multinodeset-edpm-compute-no-nodes-xqlzc" event={"ID":"07266f43-def2-46f4-8a0f-c856ec3d030b","Type":"ContainerStarted","Data":"e5342dec4985872b7d7ce685e6185d0580bf743b2ef488b1180b66491c91c7cf"} Dec 03 13:03:57 crc kubenswrapper[5052]: I1203 13:03:57.857342 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/validate-network-edpm-multinodeset-edpm-compute-no-nodes-xqlzc" podStartSLOduration=2.089087653 podStartE2EDuration="2.857323452s" podCreationTimestamp="2025-12-03 13:03:55 +0000 UTC" firstStartedPulling="2025-12-03 13:03:56.639886476 +0000 UTC m=+6227.228128240" lastFinishedPulling="2025-12-03 13:03:57.408122275 +0000 UTC m=+6227.996364039" observedRunningTime="2025-12-03 13:03:57.851494025 +0000 UTC m=+6228.439735779" watchObservedRunningTime="2025-12-03 13:03:57.857323452 +0000 UTC m=+6228.445565216" Dec 03 13:03:58 crc kubenswrapper[5052]: I1203 13:03:58.847628 5052 generic.go:334] "Generic (PLEG): container finished" podID="07266f43-def2-46f4-8a0f-c856ec3d030b" containerID="e5342dec4985872b7d7ce685e6185d0580bf743b2ef488b1180b66491c91c7cf" exitCode=0 Dec 03 13:03:58 crc kubenswrapper[5052]: I1203 13:03:58.847674 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/validate-network-edpm-multinodeset-edpm-compute-no-nodes-xqlzc" event={"ID":"07266f43-def2-46f4-8a0f-c856ec3d030b","Type":"ContainerDied","Data":"e5342dec4985872b7d7ce685e6185d0580bf743b2ef488b1180b66491c91c7cf"} Dec 03 13:04:00 crc kubenswrapper[5052]: I1203 13:04:00.191941 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/validate-network-edpm-multinodeset-edpm-compute-no-nodes-xqlzc" Dec 03 13:04:00 crc kubenswrapper[5052]: I1203 13:04:00.372423 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/07266f43-def2-46f4-8a0f-c856ec3d030b-inventory\") pod \"07266f43-def2-46f4-8a0f-c856ec3d030b\" (UID: \"07266f43-def2-46f4-8a0f-c856ec3d030b\") " Dec 03 13:04:00 crc kubenswrapper[5052]: I1203 13:04:00.372480 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dmkxz\" (UniqueName: \"kubernetes.io/projected/07266f43-def2-46f4-8a0f-c856ec3d030b-kube-api-access-dmkxz\") pod \"07266f43-def2-46f4-8a0f-c856ec3d030b\" (UID: \"07266f43-def2-46f4-8a0f-c856ec3d030b\") " Dec 03 13:04:00 crc kubenswrapper[5052]: I1203 13:04:00.372619 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/07266f43-def2-46f4-8a0f-c856ec3d030b-ssh-key\") pod \"07266f43-def2-46f4-8a0f-c856ec3d030b\" (UID: \"07266f43-def2-46f4-8a0f-c856ec3d030b\") " Dec 03 13:04:00 crc kubenswrapper[5052]: I1203 13:04:00.378208 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/07266f43-def2-46f4-8a0f-c856ec3d030b-kube-api-access-dmkxz" (OuterVolumeSpecName: "kube-api-access-dmkxz") pod "07266f43-def2-46f4-8a0f-c856ec3d030b" (UID: "07266f43-def2-46f4-8a0f-c856ec3d030b"). InnerVolumeSpecName "kube-api-access-dmkxz". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:04:00 crc kubenswrapper[5052]: I1203 13:04:00.398776 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/07266f43-def2-46f4-8a0f-c856ec3d030b-ssh-key" (OuterVolumeSpecName: "ssh-key") pod "07266f43-def2-46f4-8a0f-c856ec3d030b" (UID: "07266f43-def2-46f4-8a0f-c856ec3d030b"). InnerVolumeSpecName "ssh-key". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:04:00 crc kubenswrapper[5052]: I1203 13:04:00.399885 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/07266f43-def2-46f4-8a0f-c856ec3d030b-inventory" (OuterVolumeSpecName: "inventory") pod "07266f43-def2-46f4-8a0f-c856ec3d030b" (UID: "07266f43-def2-46f4-8a0f-c856ec3d030b"). InnerVolumeSpecName "inventory". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:04:00 crc kubenswrapper[5052]: I1203 13:04:00.474382 5052 reconciler_common.go:293] "Volume detached for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/07266f43-def2-46f4-8a0f-c856ec3d030b-inventory\") on node \"crc\" DevicePath \"\"" Dec 03 13:04:00 crc kubenswrapper[5052]: I1203 13:04:00.474427 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-dmkxz\" (UniqueName: \"kubernetes.io/projected/07266f43-def2-46f4-8a0f-c856ec3d030b-kube-api-access-dmkxz\") on node \"crc\" DevicePath \"\"" Dec 03 13:04:00 crc kubenswrapper[5052]: I1203 13:04:00.474438 5052 reconciler_common.go:293] "Volume detached for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/07266f43-def2-46f4-8a0f-c856ec3d030b-ssh-key\") on node \"crc\" DevicePath \"\"" Dec 03 13:04:00 crc kubenswrapper[5052]: I1203 13:04:00.866488 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/validate-network-edpm-multinodeset-edpm-compute-no-nodes-xqlzc" event={"ID":"07266f43-def2-46f4-8a0f-c856ec3d030b","Type":"ContainerDied","Data":"ad93d0f15acf982fd81efe1b1124e1171b0d20f79d7e0143c34021f1d5e623c8"} Dec 03 13:04:00 crc kubenswrapper[5052]: I1203 13:04:00.866559 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ad93d0f15acf982fd81efe1b1124e1171b0d20f79d7e0143c34021f1d5e623c8" Dec 03 13:04:00 crc kubenswrapper[5052]: I1203 13:04:00.866623 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/validate-network-edpm-multinodeset-edpm-compute-no-nodes-xqlzc" Dec 03 13:04:00 crc kubenswrapper[5052]: I1203 13:04:00.928352 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/install-os-edpm-multinodeset-edpm-compute-no-nodes-fwh4z"] Dec 03 13:04:00 crc kubenswrapper[5052]: E1203 13:04:00.928732 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="07266f43-def2-46f4-8a0f-c856ec3d030b" containerName="validate-network-edpm-multinodeset-edpm-compute-no-nodes" Dec 03 13:04:00 crc kubenswrapper[5052]: I1203 13:04:00.928751 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="07266f43-def2-46f4-8a0f-c856ec3d030b" containerName="validate-network-edpm-multinodeset-edpm-compute-no-nodes" Dec 03 13:04:00 crc kubenswrapper[5052]: I1203 13:04:00.928957 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="07266f43-def2-46f4-8a0f-c856ec3d030b" containerName="validate-network-edpm-multinodeset-edpm-compute-no-nodes" Dec 03 13:04:00 crc kubenswrapper[5052]: I1203 13:04:00.929544 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/install-os-edpm-multinodeset-edpm-compute-no-nodes-fwh4z" Dec 03 13:04:00 crc kubenswrapper[5052]: I1203 13:04:00.931873 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"edpm-compute-no-nodes-dockercfg-mkcgv" Dec 03 13:04:00 crc kubenswrapper[5052]: I1203 13:04:00.932488 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"dataplanenodeset-edpm-compute-no-nodes" Dec 03 13:04:00 crc kubenswrapper[5052]: I1203 13:04:00.932717 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"dataplane-ansible-ssh-private-key-secret" Dec 03 13:04:00 crc kubenswrapper[5052]: I1203 13:04:00.932943 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"openstack-aee-default-env" Dec 03 13:04:00 crc kubenswrapper[5052]: I1203 13:04:00.939236 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/install-os-edpm-multinodeset-edpm-compute-no-nodes-fwh4z"] Dec 03 13:04:01 crc kubenswrapper[5052]: I1203 13:04:01.083011 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fkcc8\" (UniqueName: \"kubernetes.io/projected/f042c0a4-0556-430b-8c10-9e662438c938-kube-api-access-fkcc8\") pod \"install-os-edpm-multinodeset-edpm-compute-no-nodes-fwh4z\" (UID: \"f042c0a4-0556-430b-8c10-9e662438c938\") " pod="openstack-kuttl-tests/install-os-edpm-multinodeset-edpm-compute-no-nodes-fwh4z" Dec 03 13:04:01 crc kubenswrapper[5052]: I1203 13:04:01.083110 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/f042c0a4-0556-430b-8c10-9e662438c938-inventory\") pod \"install-os-edpm-multinodeset-edpm-compute-no-nodes-fwh4z\" (UID: \"f042c0a4-0556-430b-8c10-9e662438c938\") " pod="openstack-kuttl-tests/install-os-edpm-multinodeset-edpm-compute-no-nodes-fwh4z" Dec 03 13:04:01 crc kubenswrapper[5052]: I1203 13:04:01.083210 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/f042c0a4-0556-430b-8c10-9e662438c938-ssh-key\") pod \"install-os-edpm-multinodeset-edpm-compute-no-nodes-fwh4z\" (UID: \"f042c0a4-0556-430b-8c10-9e662438c938\") " pod="openstack-kuttl-tests/install-os-edpm-multinodeset-edpm-compute-no-nodes-fwh4z" Dec 03 13:04:01 crc kubenswrapper[5052]: I1203 13:04:01.185071 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/f042c0a4-0556-430b-8c10-9e662438c938-ssh-key\") pod \"install-os-edpm-multinodeset-edpm-compute-no-nodes-fwh4z\" (UID: \"f042c0a4-0556-430b-8c10-9e662438c938\") " pod="openstack-kuttl-tests/install-os-edpm-multinodeset-edpm-compute-no-nodes-fwh4z" Dec 03 13:04:01 crc kubenswrapper[5052]: I1203 13:04:01.185250 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-fkcc8\" (UniqueName: \"kubernetes.io/projected/f042c0a4-0556-430b-8c10-9e662438c938-kube-api-access-fkcc8\") pod \"install-os-edpm-multinodeset-edpm-compute-no-nodes-fwh4z\" (UID: \"f042c0a4-0556-430b-8c10-9e662438c938\") " pod="openstack-kuttl-tests/install-os-edpm-multinodeset-edpm-compute-no-nodes-fwh4z" Dec 03 13:04:01 crc kubenswrapper[5052]: I1203 13:04:01.185302 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/f042c0a4-0556-430b-8c10-9e662438c938-inventory\") pod \"install-os-edpm-multinodeset-edpm-compute-no-nodes-fwh4z\" (UID: \"f042c0a4-0556-430b-8c10-9e662438c938\") " pod="openstack-kuttl-tests/install-os-edpm-multinodeset-edpm-compute-no-nodes-fwh4z" Dec 03 13:04:01 crc kubenswrapper[5052]: I1203 13:04:01.189119 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/f042c0a4-0556-430b-8c10-9e662438c938-inventory\") pod \"install-os-edpm-multinodeset-edpm-compute-no-nodes-fwh4z\" (UID: \"f042c0a4-0556-430b-8c10-9e662438c938\") " pod="openstack-kuttl-tests/install-os-edpm-multinodeset-edpm-compute-no-nodes-fwh4z" Dec 03 13:04:01 crc kubenswrapper[5052]: I1203 13:04:01.197229 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/f042c0a4-0556-430b-8c10-9e662438c938-ssh-key\") pod \"install-os-edpm-multinodeset-edpm-compute-no-nodes-fwh4z\" (UID: \"f042c0a4-0556-430b-8c10-9e662438c938\") " pod="openstack-kuttl-tests/install-os-edpm-multinodeset-edpm-compute-no-nodes-fwh4z" Dec 03 13:04:01 crc kubenswrapper[5052]: I1203 13:04:01.206306 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-fkcc8\" (UniqueName: \"kubernetes.io/projected/f042c0a4-0556-430b-8c10-9e662438c938-kube-api-access-fkcc8\") pod \"install-os-edpm-multinodeset-edpm-compute-no-nodes-fwh4z\" (UID: \"f042c0a4-0556-430b-8c10-9e662438c938\") " pod="openstack-kuttl-tests/install-os-edpm-multinodeset-edpm-compute-no-nodes-fwh4z" Dec 03 13:04:01 crc kubenswrapper[5052]: I1203 13:04:01.246681 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/install-os-edpm-multinodeset-edpm-compute-no-nodes-fwh4z" Dec 03 13:04:01 crc kubenswrapper[5052]: W1203 13:04:01.718950 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf042c0a4_0556_430b_8c10_9e662438c938.slice/crio-e8142dedaeaf888d819502d656bee8dedc57f1a7c91906cd90d58755cc7aa1a7 WatchSource:0}: Error finding container e8142dedaeaf888d819502d656bee8dedc57f1a7c91906cd90d58755cc7aa1a7: Status 404 returned error can't find the container with id e8142dedaeaf888d819502d656bee8dedc57f1a7c91906cd90d58755cc7aa1a7 Dec 03 13:04:01 crc kubenswrapper[5052]: I1203 13:04:01.726214 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/install-os-edpm-multinodeset-edpm-compute-no-nodes-fwh4z"] Dec 03 13:04:01 crc kubenswrapper[5052]: I1203 13:04:01.876485 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/install-os-edpm-multinodeset-edpm-compute-no-nodes-fwh4z" event={"ID":"f042c0a4-0556-430b-8c10-9e662438c938","Type":"ContainerStarted","Data":"e8142dedaeaf888d819502d656bee8dedc57f1a7c91906cd90d58755cc7aa1a7"} Dec 03 13:04:02 crc kubenswrapper[5052]: I1203 13:04:02.886030 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/install-os-edpm-multinodeset-edpm-compute-no-nodes-fwh4z" event={"ID":"f042c0a4-0556-430b-8c10-9e662438c938","Type":"ContainerStarted","Data":"2a430f54ad25c239aaffcfc2cd168c7a1018a6d91ff915b9917a846608c9c8ae"} Dec 03 13:04:02 crc kubenswrapper[5052]: I1203 13:04:02.901292 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/install-os-edpm-multinodeset-edpm-compute-no-nodes-fwh4z" podStartSLOduration=2.400628582 podStartE2EDuration="2.901252159s" podCreationTimestamp="2025-12-03 13:04:00 +0000 UTC" firstStartedPulling="2025-12-03 13:04:01.724960869 +0000 UTC m=+6232.313202653" lastFinishedPulling="2025-12-03 13:04:02.225584466 +0000 UTC m=+6232.813826230" observedRunningTime="2025-12-03 13:04:02.900683683 +0000 UTC m=+6233.488925447" watchObservedRunningTime="2025-12-03 13:04:02.901252159 +0000 UTC m=+6233.489493923" Dec 03 13:04:03 crc kubenswrapper[5052]: I1203 13:04:03.896451 5052 generic.go:334] "Generic (PLEG): container finished" podID="f042c0a4-0556-430b-8c10-9e662438c938" containerID="2a430f54ad25c239aaffcfc2cd168c7a1018a6d91ff915b9917a846608c9c8ae" exitCode=0 Dec 03 13:04:03 crc kubenswrapper[5052]: I1203 13:04:03.896520 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/install-os-edpm-multinodeset-edpm-compute-no-nodes-fwh4z" event={"ID":"f042c0a4-0556-430b-8c10-9e662438c938","Type":"ContainerDied","Data":"2a430f54ad25c239aaffcfc2cd168c7a1018a6d91ff915b9917a846608c9c8ae"} Dec 03 13:04:05 crc kubenswrapper[5052]: I1203 13:04:05.264056 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/install-os-edpm-multinodeset-edpm-compute-no-nodes-fwh4z" Dec 03 13:04:05 crc kubenswrapper[5052]: I1203 13:04:05.345643 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/f042c0a4-0556-430b-8c10-9e662438c938-ssh-key\") pod \"f042c0a4-0556-430b-8c10-9e662438c938\" (UID: \"f042c0a4-0556-430b-8c10-9e662438c938\") " Dec 03 13:04:05 crc kubenswrapper[5052]: I1203 13:04:05.345723 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fkcc8\" (UniqueName: \"kubernetes.io/projected/f042c0a4-0556-430b-8c10-9e662438c938-kube-api-access-fkcc8\") pod \"f042c0a4-0556-430b-8c10-9e662438c938\" (UID: \"f042c0a4-0556-430b-8c10-9e662438c938\") " Dec 03 13:04:05 crc kubenswrapper[5052]: I1203 13:04:05.345760 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/f042c0a4-0556-430b-8c10-9e662438c938-inventory\") pod \"f042c0a4-0556-430b-8c10-9e662438c938\" (UID: \"f042c0a4-0556-430b-8c10-9e662438c938\") " Dec 03 13:04:05 crc kubenswrapper[5052]: I1203 13:04:05.360006 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f042c0a4-0556-430b-8c10-9e662438c938-kube-api-access-fkcc8" (OuterVolumeSpecName: "kube-api-access-fkcc8") pod "f042c0a4-0556-430b-8c10-9e662438c938" (UID: "f042c0a4-0556-430b-8c10-9e662438c938"). InnerVolumeSpecName "kube-api-access-fkcc8". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:04:05 crc kubenswrapper[5052]: I1203 13:04:05.375409 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f042c0a4-0556-430b-8c10-9e662438c938-ssh-key" (OuterVolumeSpecName: "ssh-key") pod "f042c0a4-0556-430b-8c10-9e662438c938" (UID: "f042c0a4-0556-430b-8c10-9e662438c938"). InnerVolumeSpecName "ssh-key". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:04:05 crc kubenswrapper[5052]: I1203 13:04:05.376671 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f042c0a4-0556-430b-8c10-9e662438c938-inventory" (OuterVolumeSpecName: "inventory") pod "f042c0a4-0556-430b-8c10-9e662438c938" (UID: "f042c0a4-0556-430b-8c10-9e662438c938"). InnerVolumeSpecName "inventory". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:04:05 crc kubenswrapper[5052]: I1203 13:04:05.448051 5052 reconciler_common.go:293] "Volume detached for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/f042c0a4-0556-430b-8c10-9e662438c938-ssh-key\") on node \"crc\" DevicePath \"\"" Dec 03 13:04:05 crc kubenswrapper[5052]: I1203 13:04:05.448584 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-fkcc8\" (UniqueName: \"kubernetes.io/projected/f042c0a4-0556-430b-8c10-9e662438c938-kube-api-access-fkcc8\") on node \"crc\" DevicePath \"\"" Dec 03 13:04:05 crc kubenswrapper[5052]: I1203 13:04:05.448695 5052 reconciler_common.go:293] "Volume detached for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/f042c0a4-0556-430b-8c10-9e662438c938-inventory\") on node \"crc\" DevicePath \"\"" Dec 03 13:04:05 crc kubenswrapper[5052]: I1203 13:04:05.918682 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/install-os-edpm-multinodeset-edpm-compute-no-nodes-fwh4z" event={"ID":"f042c0a4-0556-430b-8c10-9e662438c938","Type":"ContainerDied","Data":"e8142dedaeaf888d819502d656bee8dedc57f1a7c91906cd90d58755cc7aa1a7"} Dec 03 13:04:05 crc kubenswrapper[5052]: I1203 13:04:05.919057 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e8142dedaeaf888d819502d656bee8dedc57f1a7c91906cd90d58755cc7aa1a7" Dec 03 13:04:05 crc kubenswrapper[5052]: I1203 13:04:05.918815 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/install-os-edpm-multinodeset-edpm-compute-no-nodes-fwh4z" Dec 03 13:04:05 crc kubenswrapper[5052]: I1203 13:04:05.967830 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/configure-os-edpm-multinodeset-edpm-compute-no-nodes-cndq9"] Dec 03 13:04:05 crc kubenswrapper[5052]: E1203 13:04:05.968153 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="f042c0a4-0556-430b-8c10-9e662438c938" containerName="install-os-edpm-multinodeset-edpm-compute-no-nodes" Dec 03 13:04:05 crc kubenswrapper[5052]: I1203 13:04:05.968173 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="f042c0a4-0556-430b-8c10-9e662438c938" containerName="install-os-edpm-multinodeset-edpm-compute-no-nodes" Dec 03 13:04:05 crc kubenswrapper[5052]: I1203 13:04:05.968361 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="f042c0a4-0556-430b-8c10-9e662438c938" containerName="install-os-edpm-multinodeset-edpm-compute-no-nodes" Dec 03 13:04:05 crc kubenswrapper[5052]: I1203 13:04:05.969335 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/configure-os-edpm-multinodeset-edpm-compute-no-nodes-cndq9" Dec 03 13:04:05 crc kubenswrapper[5052]: I1203 13:04:05.980024 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"dataplane-ansible-ssh-private-key-secret" Dec 03 13:04:05 crc kubenswrapper[5052]: I1203 13:04:05.980320 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"openstack-aee-default-env" Dec 03 13:04:05 crc kubenswrapper[5052]: I1203 13:04:05.980483 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"dataplanenodeset-edpm-compute-no-nodes" Dec 03 13:04:05 crc kubenswrapper[5052]: I1203 13:04:05.981198 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"edpm-compute-no-nodes-dockercfg-mkcgv" Dec 03 13:04:05 crc kubenswrapper[5052]: I1203 13:04:05.985571 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/configure-os-edpm-multinodeset-edpm-compute-no-nodes-cndq9"] Dec 03 13:04:06 crc kubenswrapper[5052]: I1203 13:04:06.057544 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hh6xt\" (UniqueName: \"kubernetes.io/projected/e0cf2a46-be43-44a0-adce-67ceb2cb06ee-kube-api-access-hh6xt\") pod \"configure-os-edpm-multinodeset-edpm-compute-no-nodes-cndq9\" (UID: \"e0cf2a46-be43-44a0-adce-67ceb2cb06ee\") " pod="openstack-kuttl-tests/configure-os-edpm-multinodeset-edpm-compute-no-nodes-cndq9" Dec 03 13:04:06 crc kubenswrapper[5052]: I1203 13:04:06.057608 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/e0cf2a46-be43-44a0-adce-67ceb2cb06ee-inventory\") pod \"configure-os-edpm-multinodeset-edpm-compute-no-nodes-cndq9\" (UID: \"e0cf2a46-be43-44a0-adce-67ceb2cb06ee\") " pod="openstack-kuttl-tests/configure-os-edpm-multinodeset-edpm-compute-no-nodes-cndq9" Dec 03 13:04:06 crc kubenswrapper[5052]: I1203 13:04:06.057835 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/e0cf2a46-be43-44a0-adce-67ceb2cb06ee-ssh-key\") pod \"configure-os-edpm-multinodeset-edpm-compute-no-nodes-cndq9\" (UID: \"e0cf2a46-be43-44a0-adce-67ceb2cb06ee\") " pod="openstack-kuttl-tests/configure-os-edpm-multinodeset-edpm-compute-no-nodes-cndq9" Dec 03 13:04:06 crc kubenswrapper[5052]: I1203 13:04:06.159835 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/e0cf2a46-be43-44a0-adce-67ceb2cb06ee-ssh-key\") pod \"configure-os-edpm-multinodeset-edpm-compute-no-nodes-cndq9\" (UID: \"e0cf2a46-be43-44a0-adce-67ceb2cb06ee\") " pod="openstack-kuttl-tests/configure-os-edpm-multinodeset-edpm-compute-no-nodes-cndq9" Dec 03 13:04:06 crc kubenswrapper[5052]: I1203 13:04:06.159911 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-hh6xt\" (UniqueName: \"kubernetes.io/projected/e0cf2a46-be43-44a0-adce-67ceb2cb06ee-kube-api-access-hh6xt\") pod \"configure-os-edpm-multinodeset-edpm-compute-no-nodes-cndq9\" (UID: \"e0cf2a46-be43-44a0-adce-67ceb2cb06ee\") " pod="openstack-kuttl-tests/configure-os-edpm-multinodeset-edpm-compute-no-nodes-cndq9" Dec 03 13:04:06 crc kubenswrapper[5052]: I1203 13:04:06.159944 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/e0cf2a46-be43-44a0-adce-67ceb2cb06ee-inventory\") pod \"configure-os-edpm-multinodeset-edpm-compute-no-nodes-cndq9\" (UID: \"e0cf2a46-be43-44a0-adce-67ceb2cb06ee\") " pod="openstack-kuttl-tests/configure-os-edpm-multinodeset-edpm-compute-no-nodes-cndq9" Dec 03 13:04:06 crc kubenswrapper[5052]: I1203 13:04:06.164400 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/e0cf2a46-be43-44a0-adce-67ceb2cb06ee-ssh-key\") pod \"configure-os-edpm-multinodeset-edpm-compute-no-nodes-cndq9\" (UID: \"e0cf2a46-be43-44a0-adce-67ceb2cb06ee\") " pod="openstack-kuttl-tests/configure-os-edpm-multinodeset-edpm-compute-no-nodes-cndq9" Dec 03 13:04:06 crc kubenswrapper[5052]: I1203 13:04:06.165366 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/e0cf2a46-be43-44a0-adce-67ceb2cb06ee-inventory\") pod \"configure-os-edpm-multinodeset-edpm-compute-no-nodes-cndq9\" (UID: \"e0cf2a46-be43-44a0-adce-67ceb2cb06ee\") " pod="openstack-kuttl-tests/configure-os-edpm-multinodeset-edpm-compute-no-nodes-cndq9" Dec 03 13:04:06 crc kubenswrapper[5052]: I1203 13:04:06.179727 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-hh6xt\" (UniqueName: \"kubernetes.io/projected/e0cf2a46-be43-44a0-adce-67ceb2cb06ee-kube-api-access-hh6xt\") pod \"configure-os-edpm-multinodeset-edpm-compute-no-nodes-cndq9\" (UID: \"e0cf2a46-be43-44a0-adce-67ceb2cb06ee\") " pod="openstack-kuttl-tests/configure-os-edpm-multinodeset-edpm-compute-no-nodes-cndq9" Dec 03 13:04:06 crc kubenswrapper[5052]: I1203 13:04:06.289659 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/configure-os-edpm-multinodeset-edpm-compute-no-nodes-cndq9" Dec 03 13:04:06 crc kubenswrapper[5052]: I1203 13:04:06.719428 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/configure-os-edpm-multinodeset-edpm-compute-no-nodes-cndq9"] Dec 03 13:04:06 crc kubenswrapper[5052]: W1203 13:04:06.721956 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode0cf2a46_be43_44a0_adce_67ceb2cb06ee.slice/crio-7d57bce184cb830b6345b72d82b689f7e8a0e6511179aaa21ef69128224d736c WatchSource:0}: Error finding container 7d57bce184cb830b6345b72d82b689f7e8a0e6511179aaa21ef69128224d736c: Status 404 returned error can't find the container with id 7d57bce184cb830b6345b72d82b689f7e8a0e6511179aaa21ef69128224d736c Dec 03 13:04:06 crc kubenswrapper[5052]: I1203 13:04:06.928765 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/configure-os-edpm-multinodeset-edpm-compute-no-nodes-cndq9" event={"ID":"e0cf2a46-be43-44a0-adce-67ceb2cb06ee","Type":"ContainerStarted","Data":"7d57bce184cb830b6345b72d82b689f7e8a0e6511179aaa21ef69128224d736c"} Dec 03 13:04:07 crc kubenswrapper[5052]: I1203 13:04:07.938946 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/configure-os-edpm-multinodeset-edpm-compute-no-nodes-cndq9" event={"ID":"e0cf2a46-be43-44a0-adce-67ceb2cb06ee","Type":"ContainerStarted","Data":"b3ba42f32af8ad8ea170cf0a0523a777860aef8c269e25c986888ef8bf7698c4"} Dec 03 13:04:07 crc kubenswrapper[5052]: I1203 13:04:07.957708 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/configure-os-edpm-multinodeset-edpm-compute-no-nodes-cndq9" podStartSLOduration=2.459582319 podStartE2EDuration="2.957669594s" podCreationTimestamp="2025-12-03 13:04:05 +0000 UTC" firstStartedPulling="2025-12-03 13:04:06.724694713 +0000 UTC m=+6237.312936477" lastFinishedPulling="2025-12-03 13:04:07.222781988 +0000 UTC m=+6237.811023752" observedRunningTime="2025-12-03 13:04:07.95405089 +0000 UTC m=+6238.542292674" watchObservedRunningTime="2025-12-03 13:04:07.957669594 +0000 UTC m=+6238.545911358" Dec 03 13:04:08 crc kubenswrapper[5052]: I1203 13:04:08.951624 5052 generic.go:334] "Generic (PLEG): container finished" podID="e0cf2a46-be43-44a0-adce-67ceb2cb06ee" containerID="b3ba42f32af8ad8ea170cf0a0523a777860aef8c269e25c986888ef8bf7698c4" exitCode=0 Dec 03 13:04:08 crc kubenswrapper[5052]: I1203 13:04:08.951676 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/configure-os-edpm-multinodeset-edpm-compute-no-nodes-cndq9" event={"ID":"e0cf2a46-be43-44a0-adce-67ceb2cb06ee","Type":"ContainerDied","Data":"b3ba42f32af8ad8ea170cf0a0523a777860aef8c269e25c986888ef8bf7698c4"} Dec 03 13:04:10 crc kubenswrapper[5052]: I1203 13:04:10.255374 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/configure-os-edpm-multinodeset-edpm-compute-no-nodes-cndq9" Dec 03 13:04:10 crc kubenswrapper[5052]: I1203 13:04:10.429163 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hh6xt\" (UniqueName: \"kubernetes.io/projected/e0cf2a46-be43-44a0-adce-67ceb2cb06ee-kube-api-access-hh6xt\") pod \"e0cf2a46-be43-44a0-adce-67ceb2cb06ee\" (UID: \"e0cf2a46-be43-44a0-adce-67ceb2cb06ee\") " Dec 03 13:04:10 crc kubenswrapper[5052]: I1203 13:04:10.429412 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/e0cf2a46-be43-44a0-adce-67ceb2cb06ee-inventory\") pod \"e0cf2a46-be43-44a0-adce-67ceb2cb06ee\" (UID: \"e0cf2a46-be43-44a0-adce-67ceb2cb06ee\") " Dec 03 13:04:10 crc kubenswrapper[5052]: I1203 13:04:10.429456 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/e0cf2a46-be43-44a0-adce-67ceb2cb06ee-ssh-key\") pod \"e0cf2a46-be43-44a0-adce-67ceb2cb06ee\" (UID: \"e0cf2a46-be43-44a0-adce-67ceb2cb06ee\") " Dec 03 13:04:10 crc kubenswrapper[5052]: I1203 13:04:10.435056 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e0cf2a46-be43-44a0-adce-67ceb2cb06ee-kube-api-access-hh6xt" (OuterVolumeSpecName: "kube-api-access-hh6xt") pod "e0cf2a46-be43-44a0-adce-67ceb2cb06ee" (UID: "e0cf2a46-be43-44a0-adce-67ceb2cb06ee"). InnerVolumeSpecName "kube-api-access-hh6xt". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:04:10 crc kubenswrapper[5052]: I1203 13:04:10.450677 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e0cf2a46-be43-44a0-adce-67ceb2cb06ee-inventory" (OuterVolumeSpecName: "inventory") pod "e0cf2a46-be43-44a0-adce-67ceb2cb06ee" (UID: "e0cf2a46-be43-44a0-adce-67ceb2cb06ee"). InnerVolumeSpecName "inventory". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:04:10 crc kubenswrapper[5052]: I1203 13:04:10.450945 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e0cf2a46-be43-44a0-adce-67ceb2cb06ee-ssh-key" (OuterVolumeSpecName: "ssh-key") pod "e0cf2a46-be43-44a0-adce-67ceb2cb06ee" (UID: "e0cf2a46-be43-44a0-adce-67ceb2cb06ee"). InnerVolumeSpecName "ssh-key". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:04:10 crc kubenswrapper[5052]: I1203 13:04:10.530921 5052 reconciler_common.go:293] "Volume detached for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/e0cf2a46-be43-44a0-adce-67ceb2cb06ee-inventory\") on node \"crc\" DevicePath \"\"" Dec 03 13:04:10 crc kubenswrapper[5052]: I1203 13:04:10.531381 5052 reconciler_common.go:293] "Volume detached for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/e0cf2a46-be43-44a0-adce-67ceb2cb06ee-ssh-key\") on node \"crc\" DevicePath \"\"" Dec 03 13:04:10 crc kubenswrapper[5052]: I1203 13:04:10.531477 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-hh6xt\" (UniqueName: \"kubernetes.io/projected/e0cf2a46-be43-44a0-adce-67ceb2cb06ee-kube-api-access-hh6xt\") on node \"crc\" DevicePath \"\"" Dec 03 13:04:10 crc kubenswrapper[5052]: I1203 13:04:10.971851 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/configure-os-edpm-multinodeset-edpm-compute-no-nodes-cndq9" event={"ID":"e0cf2a46-be43-44a0-adce-67ceb2cb06ee","Type":"ContainerDied","Data":"7d57bce184cb830b6345b72d82b689f7e8a0e6511179aaa21ef69128224d736c"} Dec 03 13:04:10 crc kubenswrapper[5052]: I1203 13:04:10.971900 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7d57bce184cb830b6345b72d82b689f7e8a0e6511179aaa21ef69128224d736c" Dec 03 13:04:10 crc kubenswrapper[5052]: I1203 13:04:10.971900 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/configure-os-edpm-multinodeset-edpm-compute-no-nodes-cndq9" Dec 03 13:04:11 crc kubenswrapper[5052]: I1203 13:04:11.033256 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/run-os-edpm-multinodeset-edpm-compute-no-nodes-ggzzt"] Dec 03 13:04:11 crc kubenswrapper[5052]: E1203 13:04:11.033854 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="e0cf2a46-be43-44a0-adce-67ceb2cb06ee" containerName="configure-os-edpm-multinodeset-edpm-compute-no-nodes" Dec 03 13:04:11 crc kubenswrapper[5052]: I1203 13:04:11.033926 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="e0cf2a46-be43-44a0-adce-67ceb2cb06ee" containerName="configure-os-edpm-multinodeset-edpm-compute-no-nodes" Dec 03 13:04:11 crc kubenswrapper[5052]: I1203 13:04:11.034167 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="e0cf2a46-be43-44a0-adce-67ceb2cb06ee" containerName="configure-os-edpm-multinodeset-edpm-compute-no-nodes" Dec 03 13:04:11 crc kubenswrapper[5052]: I1203 13:04:11.034827 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/run-os-edpm-multinodeset-edpm-compute-no-nodes-ggzzt" Dec 03 13:04:11 crc kubenswrapper[5052]: I1203 13:04:11.037848 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"dataplanenodeset-edpm-compute-no-nodes" Dec 03 13:04:11 crc kubenswrapper[5052]: I1203 13:04:11.038143 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"dataplane-ansible-ssh-private-key-secret" Dec 03 13:04:11 crc kubenswrapper[5052]: I1203 13:04:11.038329 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"openstack-aee-default-env" Dec 03 13:04:11 crc kubenswrapper[5052]: I1203 13:04:11.041195 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"edpm-compute-no-nodes-dockercfg-mkcgv" Dec 03 13:04:11 crc kubenswrapper[5052]: I1203 13:04:11.044629 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/run-os-edpm-multinodeset-edpm-compute-no-nodes-ggzzt"] Dec 03 13:04:11 crc kubenswrapper[5052]: I1203 13:04:11.141499 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/3febf655-343c-4384-a42c-0ee3dc48f21a-ssh-key\") pod \"run-os-edpm-multinodeset-edpm-compute-no-nodes-ggzzt\" (UID: \"3febf655-343c-4384-a42c-0ee3dc48f21a\") " pod="openstack-kuttl-tests/run-os-edpm-multinodeset-edpm-compute-no-nodes-ggzzt" Dec 03 13:04:11 crc kubenswrapper[5052]: I1203 13:04:11.141853 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p49dd\" (UniqueName: \"kubernetes.io/projected/3febf655-343c-4384-a42c-0ee3dc48f21a-kube-api-access-p49dd\") pod \"run-os-edpm-multinodeset-edpm-compute-no-nodes-ggzzt\" (UID: \"3febf655-343c-4384-a42c-0ee3dc48f21a\") " pod="openstack-kuttl-tests/run-os-edpm-multinodeset-edpm-compute-no-nodes-ggzzt" Dec 03 13:04:11 crc kubenswrapper[5052]: I1203 13:04:11.141930 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/3febf655-343c-4384-a42c-0ee3dc48f21a-inventory\") pod \"run-os-edpm-multinodeset-edpm-compute-no-nodes-ggzzt\" (UID: \"3febf655-343c-4384-a42c-0ee3dc48f21a\") " pod="openstack-kuttl-tests/run-os-edpm-multinodeset-edpm-compute-no-nodes-ggzzt" Dec 03 13:04:11 crc kubenswrapper[5052]: I1203 13:04:11.243812 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/3febf655-343c-4384-a42c-0ee3dc48f21a-ssh-key\") pod \"run-os-edpm-multinodeset-edpm-compute-no-nodes-ggzzt\" (UID: \"3febf655-343c-4384-a42c-0ee3dc48f21a\") " pod="openstack-kuttl-tests/run-os-edpm-multinodeset-edpm-compute-no-nodes-ggzzt" Dec 03 13:04:11 crc kubenswrapper[5052]: I1203 13:04:11.243878 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-p49dd\" (UniqueName: \"kubernetes.io/projected/3febf655-343c-4384-a42c-0ee3dc48f21a-kube-api-access-p49dd\") pod \"run-os-edpm-multinodeset-edpm-compute-no-nodes-ggzzt\" (UID: \"3febf655-343c-4384-a42c-0ee3dc48f21a\") " pod="openstack-kuttl-tests/run-os-edpm-multinodeset-edpm-compute-no-nodes-ggzzt" Dec 03 13:04:11 crc kubenswrapper[5052]: I1203 13:04:11.243928 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/3febf655-343c-4384-a42c-0ee3dc48f21a-inventory\") pod \"run-os-edpm-multinodeset-edpm-compute-no-nodes-ggzzt\" (UID: \"3febf655-343c-4384-a42c-0ee3dc48f21a\") " pod="openstack-kuttl-tests/run-os-edpm-multinodeset-edpm-compute-no-nodes-ggzzt" Dec 03 13:04:11 crc kubenswrapper[5052]: I1203 13:04:11.247566 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/3febf655-343c-4384-a42c-0ee3dc48f21a-inventory\") pod \"run-os-edpm-multinodeset-edpm-compute-no-nodes-ggzzt\" (UID: \"3febf655-343c-4384-a42c-0ee3dc48f21a\") " pod="openstack-kuttl-tests/run-os-edpm-multinodeset-edpm-compute-no-nodes-ggzzt" Dec 03 13:04:11 crc kubenswrapper[5052]: I1203 13:04:11.247571 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/3febf655-343c-4384-a42c-0ee3dc48f21a-ssh-key\") pod \"run-os-edpm-multinodeset-edpm-compute-no-nodes-ggzzt\" (UID: \"3febf655-343c-4384-a42c-0ee3dc48f21a\") " pod="openstack-kuttl-tests/run-os-edpm-multinodeset-edpm-compute-no-nodes-ggzzt" Dec 03 13:04:11 crc kubenswrapper[5052]: I1203 13:04:11.259945 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-p49dd\" (UniqueName: \"kubernetes.io/projected/3febf655-343c-4384-a42c-0ee3dc48f21a-kube-api-access-p49dd\") pod \"run-os-edpm-multinodeset-edpm-compute-no-nodes-ggzzt\" (UID: \"3febf655-343c-4384-a42c-0ee3dc48f21a\") " pod="openstack-kuttl-tests/run-os-edpm-multinodeset-edpm-compute-no-nodes-ggzzt" Dec 03 13:04:11 crc kubenswrapper[5052]: I1203 13:04:11.360281 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/run-os-edpm-multinodeset-edpm-compute-no-nodes-ggzzt" Dec 03 13:04:11 crc kubenswrapper[5052]: I1203 13:04:11.792611 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/run-os-edpm-multinodeset-edpm-compute-no-nodes-ggzzt"] Dec 03 13:04:11 crc kubenswrapper[5052]: W1203 13:04:11.798375 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3febf655_343c_4384_a42c_0ee3dc48f21a.slice/crio-317c449d0b52a540f1490fb4021043adb7b0b8a3a7bfe66dcd49627315bf3b4c WatchSource:0}: Error finding container 317c449d0b52a540f1490fb4021043adb7b0b8a3a7bfe66dcd49627315bf3b4c: Status 404 returned error can't find the container with id 317c449d0b52a540f1490fb4021043adb7b0b8a3a7bfe66dcd49627315bf3b4c Dec 03 13:04:11 crc kubenswrapper[5052]: I1203 13:04:11.983067 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/run-os-edpm-multinodeset-edpm-compute-no-nodes-ggzzt" event={"ID":"3febf655-343c-4384-a42c-0ee3dc48f21a","Type":"ContainerStarted","Data":"317c449d0b52a540f1490fb4021043adb7b0b8a3a7bfe66dcd49627315bf3b4c"} Dec 03 13:04:12 crc kubenswrapper[5052]: I1203 13:04:12.992367 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/run-os-edpm-multinodeset-edpm-compute-no-nodes-ggzzt" event={"ID":"3febf655-343c-4384-a42c-0ee3dc48f21a","Type":"ContainerStarted","Data":"feb5b0cb00c367705dfda4372f7ee49d4b7b78b30ec4edcee55d1df737e21b21"} Dec 03 13:04:13 crc kubenswrapper[5052]: I1203 13:04:13.008998 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/run-os-edpm-multinodeset-edpm-compute-no-nodes-ggzzt" podStartSLOduration=1.350949944 podStartE2EDuration="2.008978843s" podCreationTimestamp="2025-12-03 13:04:11 +0000 UTC" firstStartedPulling="2025-12-03 13:04:11.800377198 +0000 UTC m=+6242.388618952" lastFinishedPulling="2025-12-03 13:04:12.458406087 +0000 UTC m=+6243.046647851" observedRunningTime="2025-12-03 13:04:13.006927694 +0000 UTC m=+6243.595169458" watchObservedRunningTime="2025-12-03 13:04:13.008978843 +0000 UTC m=+6243.597220617" Dec 03 13:04:14 crc kubenswrapper[5052]: I1203 13:04:14.002843 5052 generic.go:334] "Generic (PLEG): container finished" podID="3febf655-343c-4384-a42c-0ee3dc48f21a" containerID="feb5b0cb00c367705dfda4372f7ee49d4b7b78b30ec4edcee55d1df737e21b21" exitCode=0 Dec 03 13:04:14 crc kubenswrapper[5052]: I1203 13:04:14.003064 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/run-os-edpm-multinodeset-edpm-compute-no-nodes-ggzzt" event={"ID":"3febf655-343c-4384-a42c-0ee3dc48f21a","Type":"ContainerDied","Data":"feb5b0cb00c367705dfda4372f7ee49d4b7b78b30ec4edcee55d1df737e21b21"} Dec 03 13:04:15 crc kubenswrapper[5052]: I1203 13:04:15.359633 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/run-os-edpm-multinodeset-edpm-compute-no-nodes-ggzzt" Dec 03 13:04:15 crc kubenswrapper[5052]: I1203 13:04:15.505397 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/3febf655-343c-4384-a42c-0ee3dc48f21a-inventory\") pod \"3febf655-343c-4384-a42c-0ee3dc48f21a\" (UID: \"3febf655-343c-4384-a42c-0ee3dc48f21a\") " Dec 03 13:04:15 crc kubenswrapper[5052]: I1203 13:04:15.505543 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p49dd\" (UniqueName: \"kubernetes.io/projected/3febf655-343c-4384-a42c-0ee3dc48f21a-kube-api-access-p49dd\") pod \"3febf655-343c-4384-a42c-0ee3dc48f21a\" (UID: \"3febf655-343c-4384-a42c-0ee3dc48f21a\") " Dec 03 13:04:15 crc kubenswrapper[5052]: I1203 13:04:15.505650 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/3febf655-343c-4384-a42c-0ee3dc48f21a-ssh-key\") pod \"3febf655-343c-4384-a42c-0ee3dc48f21a\" (UID: \"3febf655-343c-4384-a42c-0ee3dc48f21a\") " Dec 03 13:04:15 crc kubenswrapper[5052]: I1203 13:04:15.512051 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3febf655-343c-4384-a42c-0ee3dc48f21a-kube-api-access-p49dd" (OuterVolumeSpecName: "kube-api-access-p49dd") pod "3febf655-343c-4384-a42c-0ee3dc48f21a" (UID: "3febf655-343c-4384-a42c-0ee3dc48f21a"). InnerVolumeSpecName "kube-api-access-p49dd". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:04:15 crc kubenswrapper[5052]: I1203 13:04:15.529708 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3febf655-343c-4384-a42c-0ee3dc48f21a-ssh-key" (OuterVolumeSpecName: "ssh-key") pod "3febf655-343c-4384-a42c-0ee3dc48f21a" (UID: "3febf655-343c-4384-a42c-0ee3dc48f21a"). InnerVolumeSpecName "ssh-key". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:04:15 crc kubenswrapper[5052]: I1203 13:04:15.531080 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3febf655-343c-4384-a42c-0ee3dc48f21a-inventory" (OuterVolumeSpecName: "inventory") pod "3febf655-343c-4384-a42c-0ee3dc48f21a" (UID: "3febf655-343c-4384-a42c-0ee3dc48f21a"). InnerVolumeSpecName "inventory". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:04:15 crc kubenswrapper[5052]: I1203 13:04:15.606865 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-p49dd\" (UniqueName: \"kubernetes.io/projected/3febf655-343c-4384-a42c-0ee3dc48f21a-kube-api-access-p49dd\") on node \"crc\" DevicePath \"\"" Dec 03 13:04:15 crc kubenswrapper[5052]: I1203 13:04:15.606905 5052 reconciler_common.go:293] "Volume detached for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/3febf655-343c-4384-a42c-0ee3dc48f21a-ssh-key\") on node \"crc\" DevicePath \"\"" Dec 03 13:04:15 crc kubenswrapper[5052]: I1203 13:04:15.606917 5052 reconciler_common.go:293] "Volume detached for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/3febf655-343c-4384-a42c-0ee3dc48f21a-inventory\") on node \"crc\" DevicePath \"\"" Dec 03 13:04:16 crc kubenswrapper[5052]: I1203 13:04:16.039652 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/run-os-edpm-multinodeset-edpm-compute-no-nodes-ggzzt" Dec 03 13:04:16 crc kubenswrapper[5052]: I1203 13:04:16.039562 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/run-os-edpm-multinodeset-edpm-compute-no-nodes-ggzzt" event={"ID":"3febf655-343c-4384-a42c-0ee3dc48f21a","Type":"ContainerDied","Data":"317c449d0b52a540f1490fb4021043adb7b0b8a3a7bfe66dcd49627315bf3b4c"} Dec 03 13:04:16 crc kubenswrapper[5052]: I1203 13:04:16.040409 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="317c449d0b52a540f1490fb4021043adb7b0b8a3a7bfe66dcd49627315bf3b4c" Dec 03 13:04:16 crc kubenswrapper[5052]: I1203 13:04:16.124385 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/install-certs-edpm-multinodeset-edpm-compute-no-nodes-6z9jd"] Dec 03 13:04:16 crc kubenswrapper[5052]: E1203 13:04:16.124751 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="3febf655-343c-4384-a42c-0ee3dc48f21a" containerName="run-os-edpm-multinodeset-edpm-compute-no-nodes" Dec 03 13:04:16 crc kubenswrapper[5052]: I1203 13:04:16.124770 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="3febf655-343c-4384-a42c-0ee3dc48f21a" containerName="run-os-edpm-multinodeset-edpm-compute-no-nodes" Dec 03 13:04:16 crc kubenswrapper[5052]: I1203 13:04:16.124909 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="3febf655-343c-4384-a42c-0ee3dc48f21a" containerName="run-os-edpm-multinodeset-edpm-compute-no-nodes" Dec 03 13:04:16 crc kubenswrapper[5052]: I1203 13:04:16.125441 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/install-certs-edpm-multinodeset-edpm-compute-no-nodes-6z9jd" Dec 03 13:04:16 crc kubenswrapper[5052]: I1203 13:04:16.129338 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"openstack-aee-default-env" Dec 03 13:04:16 crc kubenswrapper[5052]: I1203 13:04:16.129679 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"dataplane-ansible-ssh-private-key-secret" Dec 03 13:04:16 crc kubenswrapper[5052]: I1203 13:04:16.129695 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"dataplanenodeset-edpm-compute-no-nodes" Dec 03 13:04:16 crc kubenswrapper[5052]: I1203 13:04:16.131103 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"combined-ca-bundle" Dec 03 13:04:16 crc kubenswrapper[5052]: I1203 13:04:16.131455 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"edpm-compute-no-nodes-dockercfg-mkcgv" Dec 03 13:04:16 crc kubenswrapper[5052]: I1203 13:04:16.137076 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/install-certs-edpm-multinodeset-edpm-compute-no-nodes-6z9jd"] Dec 03 13:04:16 crc kubenswrapper[5052]: I1203 13:04:16.215748 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"neutron-ovn-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/93f47414-793b-41b8-95ac-1e3325bc494c-neutron-ovn-combined-ca-bundle\") pod \"install-certs-edpm-multinodeset-edpm-compute-no-nodes-6z9jd\" (UID: \"93f47414-793b-41b8-95ac-1e3325bc494c\") " pod="openstack-kuttl-tests/install-certs-edpm-multinodeset-edpm-compute-no-nodes-6z9jd" Dec 03 13:04:16 crc kubenswrapper[5052]: I1203 13:04:16.215819 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/93f47414-793b-41b8-95ac-1e3325bc494c-ovn-combined-ca-bundle\") pod \"install-certs-edpm-multinodeset-edpm-compute-no-nodes-6z9jd\" (UID: \"93f47414-793b-41b8-95ac-1e3325bc494c\") " pod="openstack-kuttl-tests/install-certs-edpm-multinodeset-edpm-compute-no-nodes-6z9jd" Dec 03 13:04:16 crc kubenswrapper[5052]: I1203 13:04:16.215876 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/93f47414-793b-41b8-95ac-1e3325bc494c-inventory\") pod \"install-certs-edpm-multinodeset-edpm-compute-no-nodes-6z9jd\" (UID: \"93f47414-793b-41b8-95ac-1e3325bc494c\") " pod="openstack-kuttl-tests/install-certs-edpm-multinodeset-edpm-compute-no-nodes-6z9jd" Dec 03 13:04:16 crc kubenswrapper[5052]: I1203 13:04:16.215916 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/93f47414-793b-41b8-95ac-1e3325bc494c-ssh-key\") pod \"install-certs-edpm-multinodeset-edpm-compute-no-nodes-6z9jd\" (UID: \"93f47414-793b-41b8-95ac-1e3325bc494c\") " pod="openstack-kuttl-tests/install-certs-edpm-multinodeset-edpm-compute-no-nodes-6z9jd" Dec 03 13:04:16 crc kubenswrapper[5052]: I1203 13:04:16.215973 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"neutron-metadata-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/93f47414-793b-41b8-95ac-1e3325bc494c-neutron-metadata-combined-ca-bundle\") pod \"install-certs-edpm-multinodeset-edpm-compute-no-nodes-6z9jd\" (UID: \"93f47414-793b-41b8-95ac-1e3325bc494c\") " pod="openstack-kuttl-tests/install-certs-edpm-multinodeset-edpm-compute-no-nodes-6z9jd" Dec 03 13:04:16 crc kubenswrapper[5052]: I1203 13:04:16.216017 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"neutron-dhcp-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/93f47414-793b-41b8-95ac-1e3325bc494c-neutron-dhcp-combined-ca-bundle\") pod \"install-certs-edpm-multinodeset-edpm-compute-no-nodes-6z9jd\" (UID: \"93f47414-793b-41b8-95ac-1e3325bc494c\") " pod="openstack-kuttl-tests/install-certs-edpm-multinodeset-edpm-compute-no-nodes-6z9jd" Dec 03 13:04:16 crc kubenswrapper[5052]: I1203 13:04:16.216058 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bootstrap-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/93f47414-793b-41b8-95ac-1e3325bc494c-bootstrap-combined-ca-bundle\") pod \"install-certs-edpm-multinodeset-edpm-compute-no-nodes-6z9jd\" (UID: \"93f47414-793b-41b8-95ac-1e3325bc494c\") " pod="openstack-kuttl-tests/install-certs-edpm-multinodeset-edpm-compute-no-nodes-6z9jd" Dec 03 13:04:16 crc kubenswrapper[5052]: I1203 13:04:16.216084 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l695b\" (UniqueName: \"kubernetes.io/projected/93f47414-793b-41b8-95ac-1e3325bc494c-kube-api-access-l695b\") pod \"install-certs-edpm-multinodeset-edpm-compute-no-nodes-6z9jd\" (UID: \"93f47414-793b-41b8-95ac-1e3325bc494c\") " pod="openstack-kuttl-tests/install-certs-edpm-multinodeset-edpm-compute-no-nodes-6z9jd" Dec 03 13:04:16 crc kubenswrapper[5052]: I1203 13:04:16.216146 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"neutron-sriov-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/93f47414-793b-41b8-95ac-1e3325bc494c-neutron-sriov-combined-ca-bundle\") pod \"install-certs-edpm-multinodeset-edpm-compute-no-nodes-6z9jd\" (UID: \"93f47414-793b-41b8-95ac-1e3325bc494c\") " pod="openstack-kuttl-tests/install-certs-edpm-multinodeset-edpm-compute-no-nodes-6z9jd" Dec 03 13:04:16 crc kubenswrapper[5052]: I1203 13:04:16.216211 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nova-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/93f47414-793b-41b8-95ac-1e3325bc494c-nova-combined-ca-bundle\") pod \"install-certs-edpm-multinodeset-edpm-compute-no-nodes-6z9jd\" (UID: \"93f47414-793b-41b8-95ac-1e3325bc494c\") " pod="openstack-kuttl-tests/install-certs-edpm-multinodeset-edpm-compute-no-nodes-6z9jd" Dec 03 13:04:16 crc kubenswrapper[5052]: I1203 13:04:16.216256 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"libvirt-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/93f47414-793b-41b8-95ac-1e3325bc494c-libvirt-combined-ca-bundle\") pod \"install-certs-edpm-multinodeset-edpm-compute-no-nodes-6z9jd\" (UID: \"93f47414-793b-41b8-95ac-1e3325bc494c\") " pod="openstack-kuttl-tests/install-certs-edpm-multinodeset-edpm-compute-no-nodes-6z9jd" Dec 03 13:04:16 crc kubenswrapper[5052]: I1203 13:04:16.317818 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"neutron-ovn-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/93f47414-793b-41b8-95ac-1e3325bc494c-neutron-ovn-combined-ca-bundle\") pod \"install-certs-edpm-multinodeset-edpm-compute-no-nodes-6z9jd\" (UID: \"93f47414-793b-41b8-95ac-1e3325bc494c\") " pod="openstack-kuttl-tests/install-certs-edpm-multinodeset-edpm-compute-no-nodes-6z9jd" Dec 03 13:04:16 crc kubenswrapper[5052]: I1203 13:04:16.317866 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovn-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/93f47414-793b-41b8-95ac-1e3325bc494c-ovn-combined-ca-bundle\") pod \"install-certs-edpm-multinodeset-edpm-compute-no-nodes-6z9jd\" (UID: \"93f47414-793b-41b8-95ac-1e3325bc494c\") " pod="openstack-kuttl-tests/install-certs-edpm-multinodeset-edpm-compute-no-nodes-6z9jd" Dec 03 13:04:16 crc kubenswrapper[5052]: I1203 13:04:16.317895 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/93f47414-793b-41b8-95ac-1e3325bc494c-inventory\") pod \"install-certs-edpm-multinodeset-edpm-compute-no-nodes-6z9jd\" (UID: \"93f47414-793b-41b8-95ac-1e3325bc494c\") " pod="openstack-kuttl-tests/install-certs-edpm-multinodeset-edpm-compute-no-nodes-6z9jd" Dec 03 13:04:16 crc kubenswrapper[5052]: I1203 13:04:16.317940 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/93f47414-793b-41b8-95ac-1e3325bc494c-ssh-key\") pod \"install-certs-edpm-multinodeset-edpm-compute-no-nodes-6z9jd\" (UID: \"93f47414-793b-41b8-95ac-1e3325bc494c\") " pod="openstack-kuttl-tests/install-certs-edpm-multinodeset-edpm-compute-no-nodes-6z9jd" Dec 03 13:04:16 crc kubenswrapper[5052]: I1203 13:04:16.317961 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"neutron-metadata-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/93f47414-793b-41b8-95ac-1e3325bc494c-neutron-metadata-combined-ca-bundle\") pod \"install-certs-edpm-multinodeset-edpm-compute-no-nodes-6z9jd\" (UID: \"93f47414-793b-41b8-95ac-1e3325bc494c\") " pod="openstack-kuttl-tests/install-certs-edpm-multinodeset-edpm-compute-no-nodes-6z9jd" Dec 03 13:04:16 crc kubenswrapper[5052]: I1203 13:04:16.317991 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"neutron-dhcp-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/93f47414-793b-41b8-95ac-1e3325bc494c-neutron-dhcp-combined-ca-bundle\") pod \"install-certs-edpm-multinodeset-edpm-compute-no-nodes-6z9jd\" (UID: \"93f47414-793b-41b8-95ac-1e3325bc494c\") " pod="openstack-kuttl-tests/install-certs-edpm-multinodeset-edpm-compute-no-nodes-6z9jd" Dec 03 13:04:16 crc kubenswrapper[5052]: I1203 13:04:16.318023 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"bootstrap-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/93f47414-793b-41b8-95ac-1e3325bc494c-bootstrap-combined-ca-bundle\") pod \"install-certs-edpm-multinodeset-edpm-compute-no-nodes-6z9jd\" (UID: \"93f47414-793b-41b8-95ac-1e3325bc494c\") " pod="openstack-kuttl-tests/install-certs-edpm-multinodeset-edpm-compute-no-nodes-6z9jd" Dec 03 13:04:16 crc kubenswrapper[5052]: I1203 13:04:16.318042 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-l695b\" (UniqueName: \"kubernetes.io/projected/93f47414-793b-41b8-95ac-1e3325bc494c-kube-api-access-l695b\") pod \"install-certs-edpm-multinodeset-edpm-compute-no-nodes-6z9jd\" (UID: \"93f47414-793b-41b8-95ac-1e3325bc494c\") " pod="openstack-kuttl-tests/install-certs-edpm-multinodeset-edpm-compute-no-nodes-6z9jd" Dec 03 13:04:16 crc kubenswrapper[5052]: I1203 13:04:16.318058 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"neutron-sriov-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/93f47414-793b-41b8-95ac-1e3325bc494c-neutron-sriov-combined-ca-bundle\") pod \"install-certs-edpm-multinodeset-edpm-compute-no-nodes-6z9jd\" (UID: \"93f47414-793b-41b8-95ac-1e3325bc494c\") " pod="openstack-kuttl-tests/install-certs-edpm-multinodeset-edpm-compute-no-nodes-6z9jd" Dec 03 13:04:16 crc kubenswrapper[5052]: I1203 13:04:16.318082 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"nova-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/93f47414-793b-41b8-95ac-1e3325bc494c-nova-combined-ca-bundle\") pod \"install-certs-edpm-multinodeset-edpm-compute-no-nodes-6z9jd\" (UID: \"93f47414-793b-41b8-95ac-1e3325bc494c\") " pod="openstack-kuttl-tests/install-certs-edpm-multinodeset-edpm-compute-no-nodes-6z9jd" Dec 03 13:04:16 crc kubenswrapper[5052]: I1203 13:04:16.318109 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"libvirt-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/93f47414-793b-41b8-95ac-1e3325bc494c-libvirt-combined-ca-bundle\") pod \"install-certs-edpm-multinodeset-edpm-compute-no-nodes-6z9jd\" (UID: \"93f47414-793b-41b8-95ac-1e3325bc494c\") " pod="openstack-kuttl-tests/install-certs-edpm-multinodeset-edpm-compute-no-nodes-6z9jd" Dec 03 13:04:16 crc kubenswrapper[5052]: I1203 13:04:16.323009 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"libvirt-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/93f47414-793b-41b8-95ac-1e3325bc494c-libvirt-combined-ca-bundle\") pod \"install-certs-edpm-multinodeset-edpm-compute-no-nodes-6z9jd\" (UID: \"93f47414-793b-41b8-95ac-1e3325bc494c\") " pod="openstack-kuttl-tests/install-certs-edpm-multinodeset-edpm-compute-no-nodes-6z9jd" Dec 03 13:04:16 crc kubenswrapper[5052]: I1203 13:04:16.324008 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"nova-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/93f47414-793b-41b8-95ac-1e3325bc494c-nova-combined-ca-bundle\") pod \"install-certs-edpm-multinodeset-edpm-compute-no-nodes-6z9jd\" (UID: \"93f47414-793b-41b8-95ac-1e3325bc494c\") " pod="openstack-kuttl-tests/install-certs-edpm-multinodeset-edpm-compute-no-nodes-6z9jd" Dec 03 13:04:16 crc kubenswrapper[5052]: I1203 13:04:16.324064 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"neutron-dhcp-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/93f47414-793b-41b8-95ac-1e3325bc494c-neutron-dhcp-combined-ca-bundle\") pod \"install-certs-edpm-multinodeset-edpm-compute-no-nodes-6z9jd\" (UID: \"93f47414-793b-41b8-95ac-1e3325bc494c\") " pod="openstack-kuttl-tests/install-certs-edpm-multinodeset-edpm-compute-no-nodes-6z9jd" Dec 03 13:04:16 crc kubenswrapper[5052]: I1203 13:04:16.325269 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/93f47414-793b-41b8-95ac-1e3325bc494c-ssh-key\") pod \"install-certs-edpm-multinodeset-edpm-compute-no-nodes-6z9jd\" (UID: \"93f47414-793b-41b8-95ac-1e3325bc494c\") " pod="openstack-kuttl-tests/install-certs-edpm-multinodeset-edpm-compute-no-nodes-6z9jd" Dec 03 13:04:16 crc kubenswrapper[5052]: I1203 13:04:16.325744 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"neutron-sriov-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/93f47414-793b-41b8-95ac-1e3325bc494c-neutron-sriov-combined-ca-bundle\") pod \"install-certs-edpm-multinodeset-edpm-compute-no-nodes-6z9jd\" (UID: \"93f47414-793b-41b8-95ac-1e3325bc494c\") " pod="openstack-kuttl-tests/install-certs-edpm-multinodeset-edpm-compute-no-nodes-6z9jd" Dec 03 13:04:16 crc kubenswrapper[5052]: I1203 13:04:16.326428 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"neutron-metadata-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/93f47414-793b-41b8-95ac-1e3325bc494c-neutron-metadata-combined-ca-bundle\") pod \"install-certs-edpm-multinodeset-edpm-compute-no-nodes-6z9jd\" (UID: \"93f47414-793b-41b8-95ac-1e3325bc494c\") " pod="openstack-kuttl-tests/install-certs-edpm-multinodeset-edpm-compute-no-nodes-6z9jd" Dec 03 13:04:16 crc kubenswrapper[5052]: I1203 13:04:16.327423 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"bootstrap-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/93f47414-793b-41b8-95ac-1e3325bc494c-bootstrap-combined-ca-bundle\") pod \"install-certs-edpm-multinodeset-edpm-compute-no-nodes-6z9jd\" (UID: \"93f47414-793b-41b8-95ac-1e3325bc494c\") " pod="openstack-kuttl-tests/install-certs-edpm-multinodeset-edpm-compute-no-nodes-6z9jd" Dec 03 13:04:16 crc kubenswrapper[5052]: I1203 13:04:16.327934 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/93f47414-793b-41b8-95ac-1e3325bc494c-inventory\") pod \"install-certs-edpm-multinodeset-edpm-compute-no-nodes-6z9jd\" (UID: \"93f47414-793b-41b8-95ac-1e3325bc494c\") " pod="openstack-kuttl-tests/install-certs-edpm-multinodeset-edpm-compute-no-nodes-6z9jd" Dec 03 13:04:16 crc kubenswrapper[5052]: I1203 13:04:16.328308 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovn-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/93f47414-793b-41b8-95ac-1e3325bc494c-ovn-combined-ca-bundle\") pod \"install-certs-edpm-multinodeset-edpm-compute-no-nodes-6z9jd\" (UID: \"93f47414-793b-41b8-95ac-1e3325bc494c\") " pod="openstack-kuttl-tests/install-certs-edpm-multinodeset-edpm-compute-no-nodes-6z9jd" Dec 03 13:04:16 crc kubenswrapper[5052]: I1203 13:04:16.339838 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"neutron-ovn-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/93f47414-793b-41b8-95ac-1e3325bc494c-neutron-ovn-combined-ca-bundle\") pod \"install-certs-edpm-multinodeset-edpm-compute-no-nodes-6z9jd\" (UID: \"93f47414-793b-41b8-95ac-1e3325bc494c\") " pod="openstack-kuttl-tests/install-certs-edpm-multinodeset-edpm-compute-no-nodes-6z9jd" Dec 03 13:04:16 crc kubenswrapper[5052]: I1203 13:04:16.347971 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-l695b\" (UniqueName: \"kubernetes.io/projected/93f47414-793b-41b8-95ac-1e3325bc494c-kube-api-access-l695b\") pod \"install-certs-edpm-multinodeset-edpm-compute-no-nodes-6z9jd\" (UID: \"93f47414-793b-41b8-95ac-1e3325bc494c\") " pod="openstack-kuttl-tests/install-certs-edpm-multinodeset-edpm-compute-no-nodes-6z9jd" Dec 03 13:04:16 crc kubenswrapper[5052]: I1203 13:04:16.476793 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/install-certs-edpm-multinodeset-edpm-compute-no-nodes-6z9jd" Dec 03 13:04:16 crc kubenswrapper[5052]: I1203 13:04:16.965799 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/install-certs-edpm-multinodeset-edpm-compute-no-nodes-6z9jd"] Dec 03 13:04:17 crc kubenswrapper[5052]: I1203 13:04:17.050757 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/install-certs-edpm-multinodeset-edpm-compute-no-nodes-6z9jd" event={"ID":"93f47414-793b-41b8-95ac-1e3325bc494c","Type":"ContainerStarted","Data":"4823eaf31994748efb091dd9e45c9d7fc80e326d483e53ac61ff3de98a7054ed"} Dec 03 13:04:18 crc kubenswrapper[5052]: I1203 13:04:18.063350 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/install-certs-edpm-multinodeset-edpm-compute-no-nodes-6z9jd" event={"ID":"93f47414-793b-41b8-95ac-1e3325bc494c","Type":"ContainerStarted","Data":"e2645a93f6a091a31daccb5c7df437f199acb3979d9c509e0996643c8eed095b"} Dec 03 13:04:18 crc kubenswrapper[5052]: I1203 13:04:18.085136 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/install-certs-edpm-multinodeset-edpm-compute-no-nodes-6z9jd" podStartSLOduration=1.369804325 podStartE2EDuration="2.085116722s" podCreationTimestamp="2025-12-03 13:04:16 +0000 UTC" firstStartedPulling="2025-12-03 13:04:16.976058264 +0000 UTC m=+6247.564300028" lastFinishedPulling="2025-12-03 13:04:17.691370661 +0000 UTC m=+6248.279612425" observedRunningTime="2025-12-03 13:04:18.08261005 +0000 UTC m=+6248.670851814" watchObservedRunningTime="2025-12-03 13:04:18.085116722 +0000 UTC m=+6248.673358486" Dec 03 13:04:19 crc kubenswrapper[5052]: I1203 13:04:19.074556 5052 generic.go:334] "Generic (PLEG): container finished" podID="93f47414-793b-41b8-95ac-1e3325bc494c" containerID="e2645a93f6a091a31daccb5c7df437f199acb3979d9c509e0996643c8eed095b" exitCode=0 Dec 03 13:04:19 crc kubenswrapper[5052]: I1203 13:04:19.074609 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/install-certs-edpm-multinodeset-edpm-compute-no-nodes-6z9jd" event={"ID":"93f47414-793b-41b8-95ac-1e3325bc494c","Type":"ContainerDied","Data":"e2645a93f6a091a31daccb5c7df437f199acb3979d9c509e0996643c8eed095b"} Dec 03 13:04:20 crc kubenswrapper[5052]: I1203 13:04:20.440204 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/install-certs-edpm-multinodeset-edpm-compute-no-nodes-6z9jd" Dec 03 13:04:20 crc kubenswrapper[5052]: I1203 13:04:20.583686 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"neutron-metadata-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/93f47414-793b-41b8-95ac-1e3325bc494c-neutron-metadata-combined-ca-bundle\") pod \"93f47414-793b-41b8-95ac-1e3325bc494c\" (UID: \"93f47414-793b-41b8-95ac-1e3325bc494c\") " Dec 03 13:04:20 crc kubenswrapper[5052]: I1203 13:04:20.583759 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"libvirt-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/93f47414-793b-41b8-95ac-1e3325bc494c-libvirt-combined-ca-bundle\") pod \"93f47414-793b-41b8-95ac-1e3325bc494c\" (UID: \"93f47414-793b-41b8-95ac-1e3325bc494c\") " Dec 03 13:04:20 crc kubenswrapper[5052]: I1203 13:04:20.583822 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/93f47414-793b-41b8-95ac-1e3325bc494c-inventory\") pod \"93f47414-793b-41b8-95ac-1e3325bc494c\" (UID: \"93f47414-793b-41b8-95ac-1e3325bc494c\") " Dec 03 13:04:20 crc kubenswrapper[5052]: I1203 13:04:20.583849 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"nova-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/93f47414-793b-41b8-95ac-1e3325bc494c-nova-combined-ca-bundle\") pod \"93f47414-793b-41b8-95ac-1e3325bc494c\" (UID: \"93f47414-793b-41b8-95ac-1e3325bc494c\") " Dec 03 13:04:20 crc kubenswrapper[5052]: I1203 13:04:20.583928 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"neutron-dhcp-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/93f47414-793b-41b8-95ac-1e3325bc494c-neutron-dhcp-combined-ca-bundle\") pod \"93f47414-793b-41b8-95ac-1e3325bc494c\" (UID: \"93f47414-793b-41b8-95ac-1e3325bc494c\") " Dec 03 13:04:20 crc kubenswrapper[5052]: I1203 13:04:20.583952 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"neutron-sriov-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/93f47414-793b-41b8-95ac-1e3325bc494c-neutron-sriov-combined-ca-bundle\") pod \"93f47414-793b-41b8-95ac-1e3325bc494c\" (UID: \"93f47414-793b-41b8-95ac-1e3325bc494c\") " Dec 03 13:04:20 crc kubenswrapper[5052]: I1203 13:04:20.583974 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"neutron-ovn-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/93f47414-793b-41b8-95ac-1e3325bc494c-neutron-ovn-combined-ca-bundle\") pod \"93f47414-793b-41b8-95ac-1e3325bc494c\" (UID: \"93f47414-793b-41b8-95ac-1e3325bc494c\") " Dec 03 13:04:20 crc kubenswrapper[5052]: I1203 13:04:20.583992 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/93f47414-793b-41b8-95ac-1e3325bc494c-ssh-key\") pod \"93f47414-793b-41b8-95ac-1e3325bc494c\" (UID: \"93f47414-793b-41b8-95ac-1e3325bc494c\") " Dec 03 13:04:20 crc kubenswrapper[5052]: I1203 13:04:20.584035 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovn-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/93f47414-793b-41b8-95ac-1e3325bc494c-ovn-combined-ca-bundle\") pod \"93f47414-793b-41b8-95ac-1e3325bc494c\" (UID: \"93f47414-793b-41b8-95ac-1e3325bc494c\") " Dec 03 13:04:20 crc kubenswrapper[5052]: I1203 13:04:20.584058 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"bootstrap-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/93f47414-793b-41b8-95ac-1e3325bc494c-bootstrap-combined-ca-bundle\") pod \"93f47414-793b-41b8-95ac-1e3325bc494c\" (UID: \"93f47414-793b-41b8-95ac-1e3325bc494c\") " Dec 03 13:04:20 crc kubenswrapper[5052]: I1203 13:04:20.584086 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l695b\" (UniqueName: \"kubernetes.io/projected/93f47414-793b-41b8-95ac-1e3325bc494c-kube-api-access-l695b\") pod \"93f47414-793b-41b8-95ac-1e3325bc494c\" (UID: \"93f47414-793b-41b8-95ac-1e3325bc494c\") " Dec 03 13:04:20 crc kubenswrapper[5052]: I1203 13:04:20.593444 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/93f47414-793b-41b8-95ac-1e3325bc494c-kube-api-access-l695b" (OuterVolumeSpecName: "kube-api-access-l695b") pod "93f47414-793b-41b8-95ac-1e3325bc494c" (UID: "93f47414-793b-41b8-95ac-1e3325bc494c"). InnerVolumeSpecName "kube-api-access-l695b". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:04:20 crc kubenswrapper[5052]: I1203 13:04:20.594669 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/93f47414-793b-41b8-95ac-1e3325bc494c-neutron-dhcp-combined-ca-bundle" (OuterVolumeSpecName: "neutron-dhcp-combined-ca-bundle") pod "93f47414-793b-41b8-95ac-1e3325bc494c" (UID: "93f47414-793b-41b8-95ac-1e3325bc494c"). InnerVolumeSpecName "neutron-dhcp-combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:04:20 crc kubenswrapper[5052]: I1203 13:04:20.602369 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/93f47414-793b-41b8-95ac-1e3325bc494c-neutron-sriov-combined-ca-bundle" (OuterVolumeSpecName: "neutron-sriov-combined-ca-bundle") pod "93f47414-793b-41b8-95ac-1e3325bc494c" (UID: "93f47414-793b-41b8-95ac-1e3325bc494c"). InnerVolumeSpecName "neutron-sriov-combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:04:20 crc kubenswrapper[5052]: I1203 13:04:20.607926 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/93f47414-793b-41b8-95ac-1e3325bc494c-libvirt-combined-ca-bundle" (OuterVolumeSpecName: "libvirt-combined-ca-bundle") pod "93f47414-793b-41b8-95ac-1e3325bc494c" (UID: "93f47414-793b-41b8-95ac-1e3325bc494c"). InnerVolumeSpecName "libvirt-combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:04:20 crc kubenswrapper[5052]: I1203 13:04:20.608939 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/93f47414-793b-41b8-95ac-1e3325bc494c-neutron-metadata-combined-ca-bundle" (OuterVolumeSpecName: "neutron-metadata-combined-ca-bundle") pod "93f47414-793b-41b8-95ac-1e3325bc494c" (UID: "93f47414-793b-41b8-95ac-1e3325bc494c"). InnerVolumeSpecName "neutron-metadata-combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:04:20 crc kubenswrapper[5052]: I1203 13:04:20.609736 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/93f47414-793b-41b8-95ac-1e3325bc494c-bootstrap-combined-ca-bundle" (OuterVolumeSpecName: "bootstrap-combined-ca-bundle") pod "93f47414-793b-41b8-95ac-1e3325bc494c" (UID: "93f47414-793b-41b8-95ac-1e3325bc494c"). InnerVolumeSpecName "bootstrap-combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:04:20 crc kubenswrapper[5052]: I1203 13:04:20.610717 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/93f47414-793b-41b8-95ac-1e3325bc494c-neutron-ovn-combined-ca-bundle" (OuterVolumeSpecName: "neutron-ovn-combined-ca-bundle") pod "93f47414-793b-41b8-95ac-1e3325bc494c" (UID: "93f47414-793b-41b8-95ac-1e3325bc494c"). InnerVolumeSpecName "neutron-ovn-combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:04:20 crc kubenswrapper[5052]: I1203 13:04:20.612695 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/93f47414-793b-41b8-95ac-1e3325bc494c-nova-combined-ca-bundle" (OuterVolumeSpecName: "nova-combined-ca-bundle") pod "93f47414-793b-41b8-95ac-1e3325bc494c" (UID: "93f47414-793b-41b8-95ac-1e3325bc494c"). InnerVolumeSpecName "nova-combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:04:20 crc kubenswrapper[5052]: I1203 13:04:20.632666 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/93f47414-793b-41b8-95ac-1e3325bc494c-ovn-combined-ca-bundle" (OuterVolumeSpecName: "ovn-combined-ca-bundle") pod "93f47414-793b-41b8-95ac-1e3325bc494c" (UID: "93f47414-793b-41b8-95ac-1e3325bc494c"). InnerVolumeSpecName "ovn-combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:04:20 crc kubenswrapper[5052]: I1203 13:04:20.664745 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/93f47414-793b-41b8-95ac-1e3325bc494c-ssh-key" (OuterVolumeSpecName: "ssh-key") pod "93f47414-793b-41b8-95ac-1e3325bc494c" (UID: "93f47414-793b-41b8-95ac-1e3325bc494c"). InnerVolumeSpecName "ssh-key". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:04:20 crc kubenswrapper[5052]: I1203 13:04:20.674669 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/93f47414-793b-41b8-95ac-1e3325bc494c-inventory" (OuterVolumeSpecName: "inventory") pod "93f47414-793b-41b8-95ac-1e3325bc494c" (UID: "93f47414-793b-41b8-95ac-1e3325bc494c"). InnerVolumeSpecName "inventory". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:04:20 crc kubenswrapper[5052]: I1203 13:04:20.698536 5052 reconciler_common.go:293] "Volume detached for volume \"neutron-dhcp-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/93f47414-793b-41b8-95ac-1e3325bc494c-neutron-dhcp-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 13:04:20 crc kubenswrapper[5052]: I1203 13:04:20.698577 5052 reconciler_common.go:293] "Volume detached for volume \"neutron-sriov-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/93f47414-793b-41b8-95ac-1e3325bc494c-neutron-sriov-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 13:04:20 crc kubenswrapper[5052]: I1203 13:04:20.698592 5052 reconciler_common.go:293] "Volume detached for volume \"neutron-ovn-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/93f47414-793b-41b8-95ac-1e3325bc494c-neutron-ovn-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 13:04:20 crc kubenswrapper[5052]: I1203 13:04:20.698605 5052 reconciler_common.go:293] "Volume detached for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/93f47414-793b-41b8-95ac-1e3325bc494c-ssh-key\") on node \"crc\" DevicePath \"\"" Dec 03 13:04:20 crc kubenswrapper[5052]: I1203 13:04:20.698617 5052 reconciler_common.go:293] "Volume detached for volume \"ovn-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/93f47414-793b-41b8-95ac-1e3325bc494c-ovn-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 13:04:20 crc kubenswrapper[5052]: I1203 13:04:20.698628 5052 reconciler_common.go:293] "Volume detached for volume \"bootstrap-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/93f47414-793b-41b8-95ac-1e3325bc494c-bootstrap-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 13:04:20 crc kubenswrapper[5052]: I1203 13:04:20.698643 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-l695b\" (UniqueName: \"kubernetes.io/projected/93f47414-793b-41b8-95ac-1e3325bc494c-kube-api-access-l695b\") on node \"crc\" DevicePath \"\"" Dec 03 13:04:20 crc kubenswrapper[5052]: I1203 13:04:20.698653 5052 reconciler_common.go:293] "Volume detached for volume \"neutron-metadata-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/93f47414-793b-41b8-95ac-1e3325bc494c-neutron-metadata-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 13:04:20 crc kubenswrapper[5052]: I1203 13:04:20.698663 5052 reconciler_common.go:293] "Volume detached for volume \"libvirt-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/93f47414-793b-41b8-95ac-1e3325bc494c-libvirt-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 13:04:20 crc kubenswrapper[5052]: I1203 13:04:20.698674 5052 reconciler_common.go:293] "Volume detached for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/93f47414-793b-41b8-95ac-1e3325bc494c-inventory\") on node \"crc\" DevicePath \"\"" Dec 03 13:04:20 crc kubenswrapper[5052]: I1203 13:04:20.698684 5052 reconciler_common.go:293] "Volume detached for volume \"nova-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/93f47414-793b-41b8-95ac-1e3325bc494c-nova-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 13:04:21 crc kubenswrapper[5052]: I1203 13:04:21.093397 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/install-certs-edpm-multinodeset-edpm-compute-no-nodes-6z9jd" event={"ID":"93f47414-793b-41b8-95ac-1e3325bc494c","Type":"ContainerDied","Data":"4823eaf31994748efb091dd9e45c9d7fc80e326d483e53ac61ff3de98a7054ed"} Dec 03 13:04:21 crc kubenswrapper[5052]: I1203 13:04:21.093434 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4823eaf31994748efb091dd9e45c9d7fc80e326d483e53ac61ff3de98a7054ed" Dec 03 13:04:21 crc kubenswrapper[5052]: I1203 13:04:21.093463 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/install-certs-edpm-multinodeset-edpm-compute-no-nodes-6z9jd" Dec 03 13:04:21 crc kubenswrapper[5052]: I1203 13:04:21.175650 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/ovn-edpm-multinodeset-edpm-compute-no-nodes-shxd8"] Dec 03 13:04:21 crc kubenswrapper[5052]: E1203 13:04:21.176004 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="93f47414-793b-41b8-95ac-1e3325bc494c" containerName="install-certs-edpm-multinodeset-edpm-compute-no-nodes" Dec 03 13:04:21 crc kubenswrapper[5052]: I1203 13:04:21.176021 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="93f47414-793b-41b8-95ac-1e3325bc494c" containerName="install-certs-edpm-multinodeset-edpm-compute-no-nodes" Dec 03 13:04:21 crc kubenswrapper[5052]: I1203 13:04:21.176170 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="93f47414-793b-41b8-95ac-1e3325bc494c" containerName="install-certs-edpm-multinodeset-edpm-compute-no-nodes" Dec 03 13:04:21 crc kubenswrapper[5052]: I1203 13:04:21.176837 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovn-edpm-multinodeset-edpm-compute-no-nodes-shxd8" Dec 03 13:04:21 crc kubenswrapper[5052]: I1203 13:04:21.179024 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"combined-ca-bundle" Dec 03 13:04:21 crc kubenswrapper[5052]: I1203 13:04:21.179105 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"dataplane-ansible-ssh-private-key-secret" Dec 03 13:04:21 crc kubenswrapper[5052]: I1203 13:04:21.181719 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"dataplanenodeset-edpm-compute-no-nodes" Dec 03 13:04:21 crc kubenswrapper[5052]: I1203 13:04:21.181980 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"edpm-compute-no-nodes-dockercfg-mkcgv" Dec 03 13:04:21 crc kubenswrapper[5052]: I1203 13:04:21.182126 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"openstack-aee-default-env" Dec 03 13:04:21 crc kubenswrapper[5052]: I1203 13:04:21.182638 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"ovncontroller-config" Dec 03 13:04:21 crc kubenswrapper[5052]: I1203 13:04:21.197193 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ovn-edpm-multinodeset-edpm-compute-no-nodes-shxd8"] Dec 03 13:04:21 crc kubenswrapper[5052]: I1203 13:04:21.306901 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/6b467669-02e7-48e6-9707-089aac0ef369-inventory\") pod \"ovn-edpm-multinodeset-edpm-compute-no-nodes-shxd8\" (UID: \"6b467669-02e7-48e6-9707-089aac0ef369\") " pod="openstack-kuttl-tests/ovn-edpm-multinodeset-edpm-compute-no-nodes-shxd8" Dec 03 13:04:21 crc kubenswrapper[5052]: I1203 13:04:21.307095 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6b467669-02e7-48e6-9707-089aac0ef369-ovn-combined-ca-bundle\") pod \"ovn-edpm-multinodeset-edpm-compute-no-nodes-shxd8\" (UID: \"6b467669-02e7-48e6-9707-089aac0ef369\") " pod="openstack-kuttl-tests/ovn-edpm-multinodeset-edpm-compute-no-nodes-shxd8" Dec 03 13:04:21 crc kubenswrapper[5052]: I1203 13:04:21.307199 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zjtxv\" (UniqueName: \"kubernetes.io/projected/6b467669-02e7-48e6-9707-089aac0ef369-kube-api-access-zjtxv\") pod \"ovn-edpm-multinodeset-edpm-compute-no-nodes-shxd8\" (UID: \"6b467669-02e7-48e6-9707-089aac0ef369\") " pod="openstack-kuttl-tests/ovn-edpm-multinodeset-edpm-compute-no-nodes-shxd8" Dec 03 13:04:21 crc kubenswrapper[5052]: I1203 13:04:21.307236 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovncontroller-config-0\" (UniqueName: \"kubernetes.io/configmap/6b467669-02e7-48e6-9707-089aac0ef369-ovncontroller-config-0\") pod \"ovn-edpm-multinodeset-edpm-compute-no-nodes-shxd8\" (UID: \"6b467669-02e7-48e6-9707-089aac0ef369\") " pod="openstack-kuttl-tests/ovn-edpm-multinodeset-edpm-compute-no-nodes-shxd8" Dec 03 13:04:21 crc kubenswrapper[5052]: I1203 13:04:21.307294 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/6b467669-02e7-48e6-9707-089aac0ef369-ssh-key\") pod \"ovn-edpm-multinodeset-edpm-compute-no-nodes-shxd8\" (UID: \"6b467669-02e7-48e6-9707-089aac0ef369\") " pod="openstack-kuttl-tests/ovn-edpm-multinodeset-edpm-compute-no-nodes-shxd8" Dec 03 13:04:21 crc kubenswrapper[5052]: I1203 13:04:21.409073 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/6b467669-02e7-48e6-9707-089aac0ef369-ssh-key\") pod \"ovn-edpm-multinodeset-edpm-compute-no-nodes-shxd8\" (UID: \"6b467669-02e7-48e6-9707-089aac0ef369\") " pod="openstack-kuttl-tests/ovn-edpm-multinodeset-edpm-compute-no-nodes-shxd8" Dec 03 13:04:21 crc kubenswrapper[5052]: I1203 13:04:21.409171 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/6b467669-02e7-48e6-9707-089aac0ef369-inventory\") pod \"ovn-edpm-multinodeset-edpm-compute-no-nodes-shxd8\" (UID: \"6b467669-02e7-48e6-9707-089aac0ef369\") " pod="openstack-kuttl-tests/ovn-edpm-multinodeset-edpm-compute-no-nodes-shxd8" Dec 03 13:04:21 crc kubenswrapper[5052]: I1203 13:04:21.409255 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovn-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6b467669-02e7-48e6-9707-089aac0ef369-ovn-combined-ca-bundle\") pod \"ovn-edpm-multinodeset-edpm-compute-no-nodes-shxd8\" (UID: \"6b467669-02e7-48e6-9707-089aac0ef369\") " pod="openstack-kuttl-tests/ovn-edpm-multinodeset-edpm-compute-no-nodes-shxd8" Dec 03 13:04:21 crc kubenswrapper[5052]: I1203 13:04:21.409309 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-zjtxv\" (UniqueName: \"kubernetes.io/projected/6b467669-02e7-48e6-9707-089aac0ef369-kube-api-access-zjtxv\") pod \"ovn-edpm-multinodeset-edpm-compute-no-nodes-shxd8\" (UID: \"6b467669-02e7-48e6-9707-089aac0ef369\") " pod="openstack-kuttl-tests/ovn-edpm-multinodeset-edpm-compute-no-nodes-shxd8" Dec 03 13:04:21 crc kubenswrapper[5052]: I1203 13:04:21.409339 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ovncontroller-config-0\" (UniqueName: \"kubernetes.io/configmap/6b467669-02e7-48e6-9707-089aac0ef369-ovncontroller-config-0\") pod \"ovn-edpm-multinodeset-edpm-compute-no-nodes-shxd8\" (UID: \"6b467669-02e7-48e6-9707-089aac0ef369\") " pod="openstack-kuttl-tests/ovn-edpm-multinodeset-edpm-compute-no-nodes-shxd8" Dec 03 13:04:21 crc kubenswrapper[5052]: I1203 13:04:21.410354 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovncontroller-config-0\" (UniqueName: \"kubernetes.io/configmap/6b467669-02e7-48e6-9707-089aac0ef369-ovncontroller-config-0\") pod \"ovn-edpm-multinodeset-edpm-compute-no-nodes-shxd8\" (UID: \"6b467669-02e7-48e6-9707-089aac0ef369\") " pod="openstack-kuttl-tests/ovn-edpm-multinodeset-edpm-compute-no-nodes-shxd8" Dec 03 13:04:21 crc kubenswrapper[5052]: I1203 13:04:21.414807 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/6b467669-02e7-48e6-9707-089aac0ef369-inventory\") pod \"ovn-edpm-multinodeset-edpm-compute-no-nodes-shxd8\" (UID: \"6b467669-02e7-48e6-9707-089aac0ef369\") " pod="openstack-kuttl-tests/ovn-edpm-multinodeset-edpm-compute-no-nodes-shxd8" Dec 03 13:04:21 crc kubenswrapper[5052]: I1203 13:04:21.414933 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/6b467669-02e7-48e6-9707-089aac0ef369-ssh-key\") pod \"ovn-edpm-multinodeset-edpm-compute-no-nodes-shxd8\" (UID: \"6b467669-02e7-48e6-9707-089aac0ef369\") " pod="openstack-kuttl-tests/ovn-edpm-multinodeset-edpm-compute-no-nodes-shxd8" Dec 03 13:04:21 crc kubenswrapper[5052]: I1203 13:04:21.414953 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ovn-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6b467669-02e7-48e6-9707-089aac0ef369-ovn-combined-ca-bundle\") pod \"ovn-edpm-multinodeset-edpm-compute-no-nodes-shxd8\" (UID: \"6b467669-02e7-48e6-9707-089aac0ef369\") " pod="openstack-kuttl-tests/ovn-edpm-multinodeset-edpm-compute-no-nodes-shxd8" Dec 03 13:04:21 crc kubenswrapper[5052]: I1203 13:04:21.429398 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-zjtxv\" (UniqueName: \"kubernetes.io/projected/6b467669-02e7-48e6-9707-089aac0ef369-kube-api-access-zjtxv\") pod \"ovn-edpm-multinodeset-edpm-compute-no-nodes-shxd8\" (UID: \"6b467669-02e7-48e6-9707-089aac0ef369\") " pod="openstack-kuttl-tests/ovn-edpm-multinodeset-edpm-compute-no-nodes-shxd8" Dec 03 13:04:21 crc kubenswrapper[5052]: I1203 13:04:21.495046 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovn-edpm-multinodeset-edpm-compute-no-nodes-shxd8" Dec 03 13:04:21 crc kubenswrapper[5052]: I1203 13:04:21.972718 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/ovn-edpm-multinodeset-edpm-compute-no-nodes-shxd8"] Dec 03 13:04:22 crc kubenswrapper[5052]: I1203 13:04:22.101645 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovn-edpm-multinodeset-edpm-compute-no-nodes-shxd8" event={"ID":"6b467669-02e7-48e6-9707-089aac0ef369","Type":"ContainerStarted","Data":"3e52755dd3d58a6fdce5da24f397fc03a933621b6685352511bd9d8587827bce"} Dec 03 13:04:23 crc kubenswrapper[5052]: I1203 13:04:23.116093 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovn-edpm-multinodeset-edpm-compute-no-nodes-shxd8" event={"ID":"6b467669-02e7-48e6-9707-089aac0ef369","Type":"ContainerStarted","Data":"27e39a5e00dde973d1e50ee5f6e7b7601cbc03765f47c2836b5134dc9e48ab01"} Dec 03 13:04:23 crc kubenswrapper[5052]: I1203 13:04:23.139054 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/ovn-edpm-multinodeset-edpm-compute-no-nodes-shxd8" podStartSLOduration=1.435107294 podStartE2EDuration="2.139029624s" podCreationTimestamp="2025-12-03 13:04:21 +0000 UTC" firstStartedPulling="2025-12-03 13:04:21.985881287 +0000 UTC m=+6252.574123061" lastFinishedPulling="2025-12-03 13:04:22.689803627 +0000 UTC m=+6253.278045391" observedRunningTime="2025-12-03 13:04:23.132736994 +0000 UTC m=+6253.720978758" watchObservedRunningTime="2025-12-03 13:04:23.139029624 +0000 UTC m=+6253.727271388" Dec 03 13:04:25 crc kubenswrapper[5052]: I1203 13:04:25.134495 5052 generic.go:334] "Generic (PLEG): container finished" podID="6b467669-02e7-48e6-9707-089aac0ef369" containerID="27e39a5e00dde973d1e50ee5f6e7b7601cbc03765f47c2836b5134dc9e48ab01" exitCode=0 Dec 03 13:04:25 crc kubenswrapper[5052]: I1203 13:04:25.134545 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovn-edpm-multinodeset-edpm-compute-no-nodes-shxd8" event={"ID":"6b467669-02e7-48e6-9707-089aac0ef369","Type":"ContainerDied","Data":"27e39a5e00dde973d1e50ee5f6e7b7601cbc03765f47c2836b5134dc9e48ab01"} Dec 03 13:04:26 crc kubenswrapper[5052]: I1203 13:04:26.538082 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovn-edpm-multinodeset-edpm-compute-no-nodes-shxd8" Dec 03 13:04:26 crc kubenswrapper[5052]: I1203 13:04:26.698088 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zjtxv\" (UniqueName: \"kubernetes.io/projected/6b467669-02e7-48e6-9707-089aac0ef369-kube-api-access-zjtxv\") pod \"6b467669-02e7-48e6-9707-089aac0ef369\" (UID: \"6b467669-02e7-48e6-9707-089aac0ef369\") " Dec 03 13:04:26 crc kubenswrapper[5052]: I1203 13:04:26.698264 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/6b467669-02e7-48e6-9707-089aac0ef369-inventory\") pod \"6b467669-02e7-48e6-9707-089aac0ef369\" (UID: \"6b467669-02e7-48e6-9707-089aac0ef369\") " Dec 03 13:04:26 crc kubenswrapper[5052]: I1203 13:04:26.698350 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovn-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6b467669-02e7-48e6-9707-089aac0ef369-ovn-combined-ca-bundle\") pod \"6b467669-02e7-48e6-9707-089aac0ef369\" (UID: \"6b467669-02e7-48e6-9707-089aac0ef369\") " Dec 03 13:04:26 crc kubenswrapper[5052]: I1203 13:04:26.698390 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ovncontroller-config-0\" (UniqueName: \"kubernetes.io/configmap/6b467669-02e7-48e6-9707-089aac0ef369-ovncontroller-config-0\") pod \"6b467669-02e7-48e6-9707-089aac0ef369\" (UID: \"6b467669-02e7-48e6-9707-089aac0ef369\") " Dec 03 13:04:26 crc kubenswrapper[5052]: I1203 13:04:26.698429 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/6b467669-02e7-48e6-9707-089aac0ef369-ssh-key\") pod \"6b467669-02e7-48e6-9707-089aac0ef369\" (UID: \"6b467669-02e7-48e6-9707-089aac0ef369\") " Dec 03 13:04:26 crc kubenswrapper[5052]: I1203 13:04:26.709954 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6b467669-02e7-48e6-9707-089aac0ef369-ovn-combined-ca-bundle" (OuterVolumeSpecName: "ovn-combined-ca-bundle") pod "6b467669-02e7-48e6-9707-089aac0ef369" (UID: "6b467669-02e7-48e6-9707-089aac0ef369"). InnerVolumeSpecName "ovn-combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:04:26 crc kubenswrapper[5052]: I1203 13:04:26.714854 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6b467669-02e7-48e6-9707-089aac0ef369-kube-api-access-zjtxv" (OuterVolumeSpecName: "kube-api-access-zjtxv") pod "6b467669-02e7-48e6-9707-089aac0ef369" (UID: "6b467669-02e7-48e6-9707-089aac0ef369"). InnerVolumeSpecName "kube-api-access-zjtxv". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:04:26 crc kubenswrapper[5052]: I1203 13:04:26.721810 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6b467669-02e7-48e6-9707-089aac0ef369-ovncontroller-config-0" (OuterVolumeSpecName: "ovncontroller-config-0") pod "6b467669-02e7-48e6-9707-089aac0ef369" (UID: "6b467669-02e7-48e6-9707-089aac0ef369"). InnerVolumeSpecName "ovncontroller-config-0". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:04:26 crc kubenswrapper[5052]: I1203 13:04:26.724181 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6b467669-02e7-48e6-9707-089aac0ef369-ssh-key" (OuterVolumeSpecName: "ssh-key") pod "6b467669-02e7-48e6-9707-089aac0ef369" (UID: "6b467669-02e7-48e6-9707-089aac0ef369"). InnerVolumeSpecName "ssh-key". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:04:26 crc kubenswrapper[5052]: I1203 13:04:26.727749 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6b467669-02e7-48e6-9707-089aac0ef369-inventory" (OuterVolumeSpecName: "inventory") pod "6b467669-02e7-48e6-9707-089aac0ef369" (UID: "6b467669-02e7-48e6-9707-089aac0ef369"). InnerVolumeSpecName "inventory". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:04:26 crc kubenswrapper[5052]: I1203 13:04:26.799781 5052 reconciler_common.go:293] "Volume detached for volume \"ovn-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6b467669-02e7-48e6-9707-089aac0ef369-ovn-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 13:04:26 crc kubenswrapper[5052]: I1203 13:04:26.799834 5052 reconciler_common.go:293] "Volume detached for volume \"ovncontroller-config-0\" (UniqueName: \"kubernetes.io/configmap/6b467669-02e7-48e6-9707-089aac0ef369-ovncontroller-config-0\") on node \"crc\" DevicePath \"\"" Dec 03 13:04:26 crc kubenswrapper[5052]: I1203 13:04:26.799843 5052 reconciler_common.go:293] "Volume detached for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/6b467669-02e7-48e6-9707-089aac0ef369-ssh-key\") on node \"crc\" DevicePath \"\"" Dec 03 13:04:26 crc kubenswrapper[5052]: I1203 13:04:26.799853 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-zjtxv\" (UniqueName: \"kubernetes.io/projected/6b467669-02e7-48e6-9707-089aac0ef369-kube-api-access-zjtxv\") on node \"crc\" DevicePath \"\"" Dec 03 13:04:26 crc kubenswrapper[5052]: I1203 13:04:26.799862 5052 reconciler_common.go:293] "Volume detached for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/6b467669-02e7-48e6-9707-089aac0ef369-inventory\") on node \"crc\" DevicePath \"\"" Dec 03 13:04:27 crc kubenswrapper[5052]: I1203 13:04:27.153261 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/ovn-edpm-multinodeset-edpm-compute-no-nodes-shxd8" event={"ID":"6b467669-02e7-48e6-9707-089aac0ef369","Type":"ContainerDied","Data":"3e52755dd3d58a6fdce5da24f397fc03a933621b6685352511bd9d8587827bce"} Dec 03 13:04:27 crc kubenswrapper[5052]: I1203 13:04:27.153300 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3e52755dd3d58a6fdce5da24f397fc03a933621b6685352511bd9d8587827bce" Dec 03 13:04:27 crc kubenswrapper[5052]: I1203 13:04:27.153332 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/ovn-edpm-multinodeset-edpm-compute-no-nodes-shxd8" Dec 03 13:04:27 crc kubenswrapper[5052]: I1203 13:04:27.248012 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/neutron-metadata-edpm-multinodeset-edpm-compute-no-nodes-dflf5"] Dec 03 13:04:27 crc kubenswrapper[5052]: E1203 13:04:27.248335 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="6b467669-02e7-48e6-9707-089aac0ef369" containerName="ovn-edpm-multinodeset-edpm-compute-no-nodes" Dec 03 13:04:27 crc kubenswrapper[5052]: I1203 13:04:27.248352 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="6b467669-02e7-48e6-9707-089aac0ef369" containerName="ovn-edpm-multinodeset-edpm-compute-no-nodes" Dec 03 13:04:27 crc kubenswrapper[5052]: I1203 13:04:27.248535 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="6b467669-02e7-48e6-9707-089aac0ef369" containerName="ovn-edpm-multinodeset-edpm-compute-no-nodes" Dec 03 13:04:27 crc kubenswrapper[5052]: I1203 13:04:27.249099 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-metadata-edpm-multinodeset-edpm-compute-no-nodes-dflf5" Dec 03 13:04:27 crc kubenswrapper[5052]: I1203 13:04:27.251273 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"neutron-ovn-metadata-agent-neutron-config" Dec 03 13:04:27 crc kubenswrapper[5052]: I1203 13:04:27.253012 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"dataplane-ansible-ssh-private-key-secret" Dec 03 13:04:27 crc kubenswrapper[5052]: I1203 13:04:27.253099 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"dataplanenodeset-edpm-compute-no-nodes" Dec 03 13:04:27 crc kubenswrapper[5052]: I1203 13:04:27.253225 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"openstack-aee-default-env" Dec 03 13:04:27 crc kubenswrapper[5052]: I1203 13:04:27.253195 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-metadata-neutron-config" Dec 03 13:04:27 crc kubenswrapper[5052]: I1203 13:04:27.253399 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"combined-ca-bundle" Dec 03 13:04:27 crc kubenswrapper[5052]: I1203 13:04:27.253734 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"edpm-compute-no-nodes-dockercfg-mkcgv" Dec 03 13:04:27 crc kubenswrapper[5052]: I1203 13:04:27.259664 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-metadata-edpm-multinodeset-edpm-compute-no-nodes-dflf5"] Dec 03 13:04:27 crc kubenswrapper[5052]: I1203 13:04:27.408559 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nova-metadata-neutron-config-2\" (UniqueName: \"kubernetes.io/secret/c80a4930-073c-4981-857f-f3fe056f6fd1-nova-metadata-neutron-config-2\") pod \"neutron-metadata-edpm-multinodeset-edpm-compute-no-nodes-dflf5\" (UID: \"c80a4930-073c-4981-857f-f3fe056f6fd1\") " pod="openstack-kuttl-tests/neutron-metadata-edpm-multinodeset-edpm-compute-no-nodes-dflf5" Dec 03 13:04:27 crc kubenswrapper[5052]: I1203 13:04:27.408663 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"neutron-metadata-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c80a4930-073c-4981-857f-f3fe056f6fd1-neutron-metadata-combined-ca-bundle\") pod \"neutron-metadata-edpm-multinodeset-edpm-compute-no-nodes-dflf5\" (UID: \"c80a4930-073c-4981-857f-f3fe056f6fd1\") " pod="openstack-kuttl-tests/neutron-metadata-edpm-multinodeset-edpm-compute-no-nodes-dflf5" Dec 03 13:04:27 crc kubenswrapper[5052]: I1203 13:04:27.408694 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/c80a4930-073c-4981-857f-f3fe056f6fd1-ssh-key\") pod \"neutron-metadata-edpm-multinodeset-edpm-compute-no-nodes-dflf5\" (UID: \"c80a4930-073c-4981-857f-f3fe056f6fd1\") " pod="openstack-kuttl-tests/neutron-metadata-edpm-multinodeset-edpm-compute-no-nodes-dflf5" Dec 03 13:04:27 crc kubenswrapper[5052]: I1203 13:04:27.408726 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nova-metadata-neutron-config-1\" (UniqueName: \"kubernetes.io/secret/c80a4930-073c-4981-857f-f3fe056f6fd1-nova-metadata-neutron-config-1\") pod \"neutron-metadata-edpm-multinodeset-edpm-compute-no-nodes-dflf5\" (UID: \"c80a4930-073c-4981-857f-f3fe056f6fd1\") " pod="openstack-kuttl-tests/neutron-metadata-edpm-multinodeset-edpm-compute-no-nodes-dflf5" Dec 03 13:04:27 crc kubenswrapper[5052]: I1203 13:04:27.408816 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"neutron-ovn-metadata-agent-neutron-config-0\" (UniqueName: \"kubernetes.io/secret/c80a4930-073c-4981-857f-f3fe056f6fd1-neutron-ovn-metadata-agent-neutron-config-0\") pod \"neutron-metadata-edpm-multinodeset-edpm-compute-no-nodes-dflf5\" (UID: \"c80a4930-073c-4981-857f-f3fe056f6fd1\") " pod="openstack-kuttl-tests/neutron-metadata-edpm-multinodeset-edpm-compute-no-nodes-dflf5" Dec 03 13:04:27 crc kubenswrapper[5052]: I1203 13:04:27.408851 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hx2bd\" (UniqueName: \"kubernetes.io/projected/c80a4930-073c-4981-857f-f3fe056f6fd1-kube-api-access-hx2bd\") pod \"neutron-metadata-edpm-multinodeset-edpm-compute-no-nodes-dflf5\" (UID: \"c80a4930-073c-4981-857f-f3fe056f6fd1\") " pod="openstack-kuttl-tests/neutron-metadata-edpm-multinodeset-edpm-compute-no-nodes-dflf5" Dec 03 13:04:27 crc kubenswrapper[5052]: I1203 13:04:27.408883 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nova-metadata-neutron-config-0\" (UniqueName: \"kubernetes.io/secret/c80a4930-073c-4981-857f-f3fe056f6fd1-nova-metadata-neutron-config-0\") pod \"neutron-metadata-edpm-multinodeset-edpm-compute-no-nodes-dflf5\" (UID: \"c80a4930-073c-4981-857f-f3fe056f6fd1\") " pod="openstack-kuttl-tests/neutron-metadata-edpm-multinodeset-edpm-compute-no-nodes-dflf5" Dec 03 13:04:27 crc kubenswrapper[5052]: I1203 13:04:27.409062 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/c80a4930-073c-4981-857f-f3fe056f6fd1-inventory\") pod \"neutron-metadata-edpm-multinodeset-edpm-compute-no-nodes-dflf5\" (UID: \"c80a4930-073c-4981-857f-f3fe056f6fd1\") " pod="openstack-kuttl-tests/neutron-metadata-edpm-multinodeset-edpm-compute-no-nodes-dflf5" Dec 03 13:04:27 crc kubenswrapper[5052]: I1203 13:04:27.510855 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"neutron-metadata-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c80a4930-073c-4981-857f-f3fe056f6fd1-neutron-metadata-combined-ca-bundle\") pod \"neutron-metadata-edpm-multinodeset-edpm-compute-no-nodes-dflf5\" (UID: \"c80a4930-073c-4981-857f-f3fe056f6fd1\") " pod="openstack-kuttl-tests/neutron-metadata-edpm-multinodeset-edpm-compute-no-nodes-dflf5" Dec 03 13:04:27 crc kubenswrapper[5052]: I1203 13:04:27.510920 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/c80a4930-073c-4981-857f-f3fe056f6fd1-ssh-key\") pod \"neutron-metadata-edpm-multinodeset-edpm-compute-no-nodes-dflf5\" (UID: \"c80a4930-073c-4981-857f-f3fe056f6fd1\") " pod="openstack-kuttl-tests/neutron-metadata-edpm-multinodeset-edpm-compute-no-nodes-dflf5" Dec 03 13:04:27 crc kubenswrapper[5052]: I1203 13:04:27.510960 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"nova-metadata-neutron-config-1\" (UniqueName: \"kubernetes.io/secret/c80a4930-073c-4981-857f-f3fe056f6fd1-nova-metadata-neutron-config-1\") pod \"neutron-metadata-edpm-multinodeset-edpm-compute-no-nodes-dflf5\" (UID: \"c80a4930-073c-4981-857f-f3fe056f6fd1\") " pod="openstack-kuttl-tests/neutron-metadata-edpm-multinodeset-edpm-compute-no-nodes-dflf5" Dec 03 13:04:27 crc kubenswrapper[5052]: I1203 13:04:27.511015 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"neutron-ovn-metadata-agent-neutron-config-0\" (UniqueName: \"kubernetes.io/secret/c80a4930-073c-4981-857f-f3fe056f6fd1-neutron-ovn-metadata-agent-neutron-config-0\") pod \"neutron-metadata-edpm-multinodeset-edpm-compute-no-nodes-dflf5\" (UID: \"c80a4930-073c-4981-857f-f3fe056f6fd1\") " pod="openstack-kuttl-tests/neutron-metadata-edpm-multinodeset-edpm-compute-no-nodes-dflf5" Dec 03 13:04:27 crc kubenswrapper[5052]: I1203 13:04:27.511043 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-hx2bd\" (UniqueName: \"kubernetes.io/projected/c80a4930-073c-4981-857f-f3fe056f6fd1-kube-api-access-hx2bd\") pod \"neutron-metadata-edpm-multinodeset-edpm-compute-no-nodes-dflf5\" (UID: \"c80a4930-073c-4981-857f-f3fe056f6fd1\") " pod="openstack-kuttl-tests/neutron-metadata-edpm-multinodeset-edpm-compute-no-nodes-dflf5" Dec 03 13:04:27 crc kubenswrapper[5052]: I1203 13:04:27.511082 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"nova-metadata-neutron-config-0\" (UniqueName: \"kubernetes.io/secret/c80a4930-073c-4981-857f-f3fe056f6fd1-nova-metadata-neutron-config-0\") pod \"neutron-metadata-edpm-multinodeset-edpm-compute-no-nodes-dflf5\" (UID: \"c80a4930-073c-4981-857f-f3fe056f6fd1\") " pod="openstack-kuttl-tests/neutron-metadata-edpm-multinodeset-edpm-compute-no-nodes-dflf5" Dec 03 13:04:27 crc kubenswrapper[5052]: I1203 13:04:27.511152 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/c80a4930-073c-4981-857f-f3fe056f6fd1-inventory\") pod \"neutron-metadata-edpm-multinodeset-edpm-compute-no-nodes-dflf5\" (UID: \"c80a4930-073c-4981-857f-f3fe056f6fd1\") " pod="openstack-kuttl-tests/neutron-metadata-edpm-multinodeset-edpm-compute-no-nodes-dflf5" Dec 03 13:04:27 crc kubenswrapper[5052]: I1203 13:04:27.511180 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"nova-metadata-neutron-config-2\" (UniqueName: \"kubernetes.io/secret/c80a4930-073c-4981-857f-f3fe056f6fd1-nova-metadata-neutron-config-2\") pod \"neutron-metadata-edpm-multinodeset-edpm-compute-no-nodes-dflf5\" (UID: \"c80a4930-073c-4981-857f-f3fe056f6fd1\") " pod="openstack-kuttl-tests/neutron-metadata-edpm-multinodeset-edpm-compute-no-nodes-dflf5" Dec 03 13:04:27 crc kubenswrapper[5052]: I1203 13:04:27.514338 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"nova-metadata-neutron-config-1\" (UniqueName: \"kubernetes.io/secret/c80a4930-073c-4981-857f-f3fe056f6fd1-nova-metadata-neutron-config-1\") pod \"neutron-metadata-edpm-multinodeset-edpm-compute-no-nodes-dflf5\" (UID: \"c80a4930-073c-4981-857f-f3fe056f6fd1\") " pod="openstack-kuttl-tests/neutron-metadata-edpm-multinodeset-edpm-compute-no-nodes-dflf5" Dec 03 13:04:27 crc kubenswrapper[5052]: I1203 13:04:27.514548 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"nova-metadata-neutron-config-2\" (UniqueName: \"kubernetes.io/secret/c80a4930-073c-4981-857f-f3fe056f6fd1-nova-metadata-neutron-config-2\") pod \"neutron-metadata-edpm-multinodeset-edpm-compute-no-nodes-dflf5\" (UID: \"c80a4930-073c-4981-857f-f3fe056f6fd1\") " pod="openstack-kuttl-tests/neutron-metadata-edpm-multinodeset-edpm-compute-no-nodes-dflf5" Dec 03 13:04:27 crc kubenswrapper[5052]: I1203 13:04:27.514670 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/c80a4930-073c-4981-857f-f3fe056f6fd1-ssh-key\") pod \"neutron-metadata-edpm-multinodeset-edpm-compute-no-nodes-dflf5\" (UID: \"c80a4930-073c-4981-857f-f3fe056f6fd1\") " pod="openstack-kuttl-tests/neutron-metadata-edpm-multinodeset-edpm-compute-no-nodes-dflf5" Dec 03 13:04:27 crc kubenswrapper[5052]: I1203 13:04:27.514767 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/c80a4930-073c-4981-857f-f3fe056f6fd1-inventory\") pod \"neutron-metadata-edpm-multinodeset-edpm-compute-no-nodes-dflf5\" (UID: \"c80a4930-073c-4981-857f-f3fe056f6fd1\") " pod="openstack-kuttl-tests/neutron-metadata-edpm-multinodeset-edpm-compute-no-nodes-dflf5" Dec 03 13:04:27 crc kubenswrapper[5052]: I1203 13:04:27.514925 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"neutron-metadata-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c80a4930-073c-4981-857f-f3fe056f6fd1-neutron-metadata-combined-ca-bundle\") pod \"neutron-metadata-edpm-multinodeset-edpm-compute-no-nodes-dflf5\" (UID: \"c80a4930-073c-4981-857f-f3fe056f6fd1\") " pod="openstack-kuttl-tests/neutron-metadata-edpm-multinodeset-edpm-compute-no-nodes-dflf5" Dec 03 13:04:27 crc kubenswrapper[5052]: I1203 13:04:27.516911 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"nova-metadata-neutron-config-0\" (UniqueName: \"kubernetes.io/secret/c80a4930-073c-4981-857f-f3fe056f6fd1-nova-metadata-neutron-config-0\") pod \"neutron-metadata-edpm-multinodeset-edpm-compute-no-nodes-dflf5\" (UID: \"c80a4930-073c-4981-857f-f3fe056f6fd1\") " pod="openstack-kuttl-tests/neutron-metadata-edpm-multinodeset-edpm-compute-no-nodes-dflf5" Dec 03 13:04:27 crc kubenswrapper[5052]: I1203 13:04:27.518094 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"neutron-ovn-metadata-agent-neutron-config-0\" (UniqueName: \"kubernetes.io/secret/c80a4930-073c-4981-857f-f3fe056f6fd1-neutron-ovn-metadata-agent-neutron-config-0\") pod \"neutron-metadata-edpm-multinodeset-edpm-compute-no-nodes-dflf5\" (UID: \"c80a4930-073c-4981-857f-f3fe056f6fd1\") " pod="openstack-kuttl-tests/neutron-metadata-edpm-multinodeset-edpm-compute-no-nodes-dflf5" Dec 03 13:04:27 crc kubenswrapper[5052]: I1203 13:04:27.530612 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-hx2bd\" (UniqueName: \"kubernetes.io/projected/c80a4930-073c-4981-857f-f3fe056f6fd1-kube-api-access-hx2bd\") pod \"neutron-metadata-edpm-multinodeset-edpm-compute-no-nodes-dflf5\" (UID: \"c80a4930-073c-4981-857f-f3fe056f6fd1\") " pod="openstack-kuttl-tests/neutron-metadata-edpm-multinodeset-edpm-compute-no-nodes-dflf5" Dec 03 13:04:27 crc kubenswrapper[5052]: I1203 13:04:27.566375 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-metadata-edpm-multinodeset-edpm-compute-no-nodes-dflf5" Dec 03 13:04:28 crc kubenswrapper[5052]: I1203 13:04:28.010596 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-metadata-edpm-multinodeset-edpm-compute-no-nodes-dflf5"] Dec 03 13:04:28 crc kubenswrapper[5052]: I1203 13:04:28.163267 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-metadata-edpm-multinodeset-edpm-compute-no-nodes-dflf5" event={"ID":"c80a4930-073c-4981-857f-f3fe056f6fd1","Type":"ContainerStarted","Data":"f79dbb6f54aac42cb4dd95782520983a04f9fbda3f9e7a763afc90d913eee112"} Dec 03 13:04:29 crc kubenswrapper[5052]: I1203 13:04:29.178479 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-metadata-edpm-multinodeset-edpm-compute-no-nodes-dflf5" event={"ID":"c80a4930-073c-4981-857f-f3fe056f6fd1","Type":"ContainerStarted","Data":"e38a3ae7079d380dc875cb9151781bdf35b9d38f31c9f182c28fb0c90661bbe5"} Dec 03 13:04:29 crc kubenswrapper[5052]: I1203 13:04:29.195390 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/neutron-metadata-edpm-multinodeset-edpm-compute-no-nodes-dflf5" podStartSLOduration=1.736502472 podStartE2EDuration="2.195367555s" podCreationTimestamp="2025-12-03 13:04:27 +0000 UTC" firstStartedPulling="2025-12-03 13:04:28.004481937 +0000 UTC m=+6258.592723701" lastFinishedPulling="2025-12-03 13:04:28.46334702 +0000 UTC m=+6259.051588784" observedRunningTime="2025-12-03 13:04:29.194733706 +0000 UTC m=+6259.782975470" watchObservedRunningTime="2025-12-03 13:04:29.195367555 +0000 UTC m=+6259.783609319" Dec 03 13:04:30 crc kubenswrapper[5052]: I1203 13:04:30.188790 5052 generic.go:334] "Generic (PLEG): container finished" podID="c80a4930-073c-4981-857f-f3fe056f6fd1" containerID="e38a3ae7079d380dc875cb9151781bdf35b9d38f31c9f182c28fb0c90661bbe5" exitCode=0 Dec 03 13:04:30 crc kubenswrapper[5052]: I1203 13:04:30.188841 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-metadata-edpm-multinodeset-edpm-compute-no-nodes-dflf5" event={"ID":"c80a4930-073c-4981-857f-f3fe056f6fd1","Type":"ContainerDied","Data":"e38a3ae7079d380dc875cb9151781bdf35b9d38f31c9f182c28fb0c90661bbe5"} Dec 03 13:04:31 crc kubenswrapper[5052]: I1203 13:04:31.509032 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-metadata-edpm-multinodeset-edpm-compute-no-nodes-dflf5" Dec 03 13:04:31 crc kubenswrapper[5052]: I1203 13:04:31.667921 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hx2bd\" (UniqueName: \"kubernetes.io/projected/c80a4930-073c-4981-857f-f3fe056f6fd1-kube-api-access-hx2bd\") pod \"c80a4930-073c-4981-857f-f3fe056f6fd1\" (UID: \"c80a4930-073c-4981-857f-f3fe056f6fd1\") " Dec 03 13:04:31 crc kubenswrapper[5052]: I1203 13:04:31.668030 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"nova-metadata-neutron-config-1\" (UniqueName: \"kubernetes.io/secret/c80a4930-073c-4981-857f-f3fe056f6fd1-nova-metadata-neutron-config-1\") pod \"c80a4930-073c-4981-857f-f3fe056f6fd1\" (UID: \"c80a4930-073c-4981-857f-f3fe056f6fd1\") " Dec 03 13:04:31 crc kubenswrapper[5052]: I1203 13:04:31.668076 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"neutron-metadata-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c80a4930-073c-4981-857f-f3fe056f6fd1-neutron-metadata-combined-ca-bundle\") pod \"c80a4930-073c-4981-857f-f3fe056f6fd1\" (UID: \"c80a4930-073c-4981-857f-f3fe056f6fd1\") " Dec 03 13:04:31 crc kubenswrapper[5052]: I1203 13:04:31.668134 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/c80a4930-073c-4981-857f-f3fe056f6fd1-ssh-key\") pod \"c80a4930-073c-4981-857f-f3fe056f6fd1\" (UID: \"c80a4930-073c-4981-857f-f3fe056f6fd1\") " Dec 03 13:04:31 crc kubenswrapper[5052]: I1203 13:04:31.668206 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"neutron-ovn-metadata-agent-neutron-config-0\" (UniqueName: \"kubernetes.io/secret/c80a4930-073c-4981-857f-f3fe056f6fd1-neutron-ovn-metadata-agent-neutron-config-0\") pod \"c80a4930-073c-4981-857f-f3fe056f6fd1\" (UID: \"c80a4930-073c-4981-857f-f3fe056f6fd1\") " Dec 03 13:04:31 crc kubenswrapper[5052]: I1203 13:04:31.668275 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/c80a4930-073c-4981-857f-f3fe056f6fd1-inventory\") pod \"c80a4930-073c-4981-857f-f3fe056f6fd1\" (UID: \"c80a4930-073c-4981-857f-f3fe056f6fd1\") " Dec 03 13:04:31 crc kubenswrapper[5052]: I1203 13:04:31.668459 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"nova-metadata-neutron-config-2\" (UniqueName: \"kubernetes.io/secret/c80a4930-073c-4981-857f-f3fe056f6fd1-nova-metadata-neutron-config-2\") pod \"c80a4930-073c-4981-857f-f3fe056f6fd1\" (UID: \"c80a4930-073c-4981-857f-f3fe056f6fd1\") " Dec 03 13:04:31 crc kubenswrapper[5052]: I1203 13:04:31.668498 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"nova-metadata-neutron-config-0\" (UniqueName: \"kubernetes.io/secret/c80a4930-073c-4981-857f-f3fe056f6fd1-nova-metadata-neutron-config-0\") pod \"c80a4930-073c-4981-857f-f3fe056f6fd1\" (UID: \"c80a4930-073c-4981-857f-f3fe056f6fd1\") " Dec 03 13:04:31 crc kubenswrapper[5052]: I1203 13:04:31.680186 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c80a4930-073c-4981-857f-f3fe056f6fd1-kube-api-access-hx2bd" (OuterVolumeSpecName: "kube-api-access-hx2bd") pod "c80a4930-073c-4981-857f-f3fe056f6fd1" (UID: "c80a4930-073c-4981-857f-f3fe056f6fd1"). InnerVolumeSpecName "kube-api-access-hx2bd". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:04:31 crc kubenswrapper[5052]: I1203 13:04:31.680188 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c80a4930-073c-4981-857f-f3fe056f6fd1-neutron-metadata-combined-ca-bundle" (OuterVolumeSpecName: "neutron-metadata-combined-ca-bundle") pod "c80a4930-073c-4981-857f-f3fe056f6fd1" (UID: "c80a4930-073c-4981-857f-f3fe056f6fd1"). InnerVolumeSpecName "neutron-metadata-combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:04:31 crc kubenswrapper[5052]: I1203 13:04:31.693679 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c80a4930-073c-4981-857f-f3fe056f6fd1-nova-metadata-neutron-config-1" (OuterVolumeSpecName: "nova-metadata-neutron-config-1") pod "c80a4930-073c-4981-857f-f3fe056f6fd1" (UID: "c80a4930-073c-4981-857f-f3fe056f6fd1"). InnerVolumeSpecName "nova-metadata-neutron-config-1". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:04:31 crc kubenswrapper[5052]: I1203 13:04:31.694243 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c80a4930-073c-4981-857f-f3fe056f6fd1-inventory" (OuterVolumeSpecName: "inventory") pod "c80a4930-073c-4981-857f-f3fe056f6fd1" (UID: "c80a4930-073c-4981-857f-f3fe056f6fd1"). InnerVolumeSpecName "inventory". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:04:31 crc kubenswrapper[5052]: I1203 13:04:31.695281 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c80a4930-073c-4981-857f-f3fe056f6fd1-nova-metadata-neutron-config-2" (OuterVolumeSpecName: "nova-metadata-neutron-config-2") pod "c80a4930-073c-4981-857f-f3fe056f6fd1" (UID: "c80a4930-073c-4981-857f-f3fe056f6fd1"). InnerVolumeSpecName "nova-metadata-neutron-config-2". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:04:31 crc kubenswrapper[5052]: I1203 13:04:31.695671 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c80a4930-073c-4981-857f-f3fe056f6fd1-nova-metadata-neutron-config-0" (OuterVolumeSpecName: "nova-metadata-neutron-config-0") pod "c80a4930-073c-4981-857f-f3fe056f6fd1" (UID: "c80a4930-073c-4981-857f-f3fe056f6fd1"). InnerVolumeSpecName "nova-metadata-neutron-config-0". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:04:31 crc kubenswrapper[5052]: I1203 13:04:31.696933 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c80a4930-073c-4981-857f-f3fe056f6fd1-ssh-key" (OuterVolumeSpecName: "ssh-key") pod "c80a4930-073c-4981-857f-f3fe056f6fd1" (UID: "c80a4930-073c-4981-857f-f3fe056f6fd1"). InnerVolumeSpecName "ssh-key". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:04:31 crc kubenswrapper[5052]: I1203 13:04:31.703306 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c80a4930-073c-4981-857f-f3fe056f6fd1-neutron-ovn-metadata-agent-neutron-config-0" (OuterVolumeSpecName: "neutron-ovn-metadata-agent-neutron-config-0") pod "c80a4930-073c-4981-857f-f3fe056f6fd1" (UID: "c80a4930-073c-4981-857f-f3fe056f6fd1"). InnerVolumeSpecName "neutron-ovn-metadata-agent-neutron-config-0". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:04:31 crc kubenswrapper[5052]: I1203 13:04:31.771386 5052 reconciler_common.go:293] "Volume detached for volume \"neutron-ovn-metadata-agent-neutron-config-0\" (UniqueName: \"kubernetes.io/secret/c80a4930-073c-4981-857f-f3fe056f6fd1-neutron-ovn-metadata-agent-neutron-config-0\") on node \"crc\" DevicePath \"\"" Dec 03 13:04:31 crc kubenswrapper[5052]: I1203 13:04:31.771449 5052 reconciler_common.go:293] "Volume detached for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/c80a4930-073c-4981-857f-f3fe056f6fd1-inventory\") on node \"crc\" DevicePath \"\"" Dec 03 13:04:31 crc kubenswrapper[5052]: I1203 13:04:31.771464 5052 reconciler_common.go:293] "Volume detached for volume \"nova-metadata-neutron-config-2\" (UniqueName: \"kubernetes.io/secret/c80a4930-073c-4981-857f-f3fe056f6fd1-nova-metadata-neutron-config-2\") on node \"crc\" DevicePath \"\"" Dec 03 13:04:31 crc kubenswrapper[5052]: I1203 13:04:31.771476 5052 reconciler_common.go:293] "Volume detached for volume \"nova-metadata-neutron-config-0\" (UniqueName: \"kubernetes.io/secret/c80a4930-073c-4981-857f-f3fe056f6fd1-nova-metadata-neutron-config-0\") on node \"crc\" DevicePath \"\"" Dec 03 13:04:31 crc kubenswrapper[5052]: I1203 13:04:31.771489 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-hx2bd\" (UniqueName: \"kubernetes.io/projected/c80a4930-073c-4981-857f-f3fe056f6fd1-kube-api-access-hx2bd\") on node \"crc\" DevicePath \"\"" Dec 03 13:04:31 crc kubenswrapper[5052]: I1203 13:04:31.771502 5052 reconciler_common.go:293] "Volume detached for volume \"nova-metadata-neutron-config-1\" (UniqueName: \"kubernetes.io/secret/c80a4930-073c-4981-857f-f3fe056f6fd1-nova-metadata-neutron-config-1\") on node \"crc\" DevicePath \"\"" Dec 03 13:04:31 crc kubenswrapper[5052]: I1203 13:04:31.771531 5052 reconciler_common.go:293] "Volume detached for volume \"neutron-metadata-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c80a4930-073c-4981-857f-f3fe056f6fd1-neutron-metadata-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 13:04:31 crc kubenswrapper[5052]: I1203 13:04:31.771544 5052 reconciler_common.go:293] "Volume detached for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/c80a4930-073c-4981-857f-f3fe056f6fd1-ssh-key\") on node \"crc\" DevicePath \"\"" Dec 03 13:04:32 crc kubenswrapper[5052]: I1203 13:04:32.207878 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-metadata-edpm-multinodeset-edpm-compute-no-nodes-dflf5" event={"ID":"c80a4930-073c-4981-857f-f3fe056f6fd1","Type":"ContainerDied","Data":"f79dbb6f54aac42cb4dd95782520983a04f9fbda3f9e7a763afc90d913eee112"} Dec 03 13:04:32 crc kubenswrapper[5052]: I1203 13:04:32.207918 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f79dbb6f54aac42cb4dd95782520983a04f9fbda3f9e7a763afc90d913eee112" Dec 03 13:04:32 crc kubenswrapper[5052]: I1203 13:04:32.207952 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-metadata-edpm-multinodeset-edpm-compute-no-nodes-dflf5" Dec 03 13:04:32 crc kubenswrapper[5052]: I1203 13:04:32.289581 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/neutron-ovn-edpm-multinodeset-edpm-compute-no-nodes-jdv6g"] Dec 03 13:04:32 crc kubenswrapper[5052]: E1203 13:04:32.289987 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="c80a4930-073c-4981-857f-f3fe056f6fd1" containerName="neutron-metadata-edpm-multinodeset-edpm-compute-no-nodes" Dec 03 13:04:32 crc kubenswrapper[5052]: I1203 13:04:32.290008 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="c80a4930-073c-4981-857f-f3fe056f6fd1" containerName="neutron-metadata-edpm-multinodeset-edpm-compute-no-nodes" Dec 03 13:04:32 crc kubenswrapper[5052]: I1203 13:04:32.290202 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="c80a4930-073c-4981-857f-f3fe056f6fd1" containerName="neutron-metadata-edpm-multinodeset-edpm-compute-no-nodes" Dec 03 13:04:32 crc kubenswrapper[5052]: I1203 13:04:32.291061 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-ovn-edpm-multinodeset-edpm-compute-no-nodes-jdv6g" Dec 03 13:04:32 crc kubenswrapper[5052]: I1203 13:04:32.294013 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"edpm-compute-no-nodes-dockercfg-mkcgv" Dec 03 13:04:32 crc kubenswrapper[5052]: I1203 13:04:32.294661 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"neutron-ovn-agent-neutron-config" Dec 03 13:04:32 crc kubenswrapper[5052]: I1203 13:04:32.295152 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"combined-ca-bundle" Dec 03 13:04:32 crc kubenswrapper[5052]: I1203 13:04:32.302744 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-ovn-edpm-multinodeset-edpm-compute-no-nodes-jdv6g"] Dec 03 13:04:32 crc kubenswrapper[5052]: I1203 13:04:32.302815 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"openstack-aee-default-env" Dec 03 13:04:32 crc kubenswrapper[5052]: I1203 13:04:32.304745 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"dataplane-ansible-ssh-private-key-secret" Dec 03 13:04:32 crc kubenswrapper[5052]: I1203 13:04:32.323925 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"dataplanenodeset-edpm-compute-no-nodes" Dec 03 13:04:32 crc kubenswrapper[5052]: I1203 13:04:32.381598 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sbkss\" (UniqueName: \"kubernetes.io/projected/10392605-c383-49fd-b4e9-b53bdda69795-kube-api-access-sbkss\") pod \"neutron-ovn-edpm-multinodeset-edpm-compute-no-nodes-jdv6g\" (UID: \"10392605-c383-49fd-b4e9-b53bdda69795\") " pod="openstack-kuttl-tests/neutron-ovn-edpm-multinodeset-edpm-compute-no-nodes-jdv6g" Dec 03 13:04:32 crc kubenswrapper[5052]: I1203 13:04:32.381709 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/10392605-c383-49fd-b4e9-b53bdda69795-inventory\") pod \"neutron-ovn-edpm-multinodeset-edpm-compute-no-nodes-jdv6g\" (UID: \"10392605-c383-49fd-b4e9-b53bdda69795\") " pod="openstack-kuttl-tests/neutron-ovn-edpm-multinodeset-edpm-compute-no-nodes-jdv6g" Dec 03 13:04:32 crc kubenswrapper[5052]: I1203 13:04:32.381751 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"neutron-ovn-agent-neutron-config-0\" (UniqueName: \"kubernetes.io/secret/10392605-c383-49fd-b4e9-b53bdda69795-neutron-ovn-agent-neutron-config-0\") pod \"neutron-ovn-edpm-multinodeset-edpm-compute-no-nodes-jdv6g\" (UID: \"10392605-c383-49fd-b4e9-b53bdda69795\") " pod="openstack-kuttl-tests/neutron-ovn-edpm-multinodeset-edpm-compute-no-nodes-jdv6g" Dec 03 13:04:32 crc kubenswrapper[5052]: I1203 13:04:32.381796 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/10392605-c383-49fd-b4e9-b53bdda69795-ssh-key\") pod \"neutron-ovn-edpm-multinodeset-edpm-compute-no-nodes-jdv6g\" (UID: \"10392605-c383-49fd-b4e9-b53bdda69795\") " pod="openstack-kuttl-tests/neutron-ovn-edpm-multinodeset-edpm-compute-no-nodes-jdv6g" Dec 03 13:04:32 crc kubenswrapper[5052]: I1203 13:04:32.381867 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"neutron-ovn-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/10392605-c383-49fd-b4e9-b53bdda69795-neutron-ovn-combined-ca-bundle\") pod \"neutron-ovn-edpm-multinodeset-edpm-compute-no-nodes-jdv6g\" (UID: \"10392605-c383-49fd-b4e9-b53bdda69795\") " pod="openstack-kuttl-tests/neutron-ovn-edpm-multinodeset-edpm-compute-no-nodes-jdv6g" Dec 03 13:04:32 crc kubenswrapper[5052]: I1203 13:04:32.483405 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-sbkss\" (UniqueName: \"kubernetes.io/projected/10392605-c383-49fd-b4e9-b53bdda69795-kube-api-access-sbkss\") pod \"neutron-ovn-edpm-multinodeset-edpm-compute-no-nodes-jdv6g\" (UID: \"10392605-c383-49fd-b4e9-b53bdda69795\") " pod="openstack-kuttl-tests/neutron-ovn-edpm-multinodeset-edpm-compute-no-nodes-jdv6g" Dec 03 13:04:32 crc kubenswrapper[5052]: I1203 13:04:32.483495 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/10392605-c383-49fd-b4e9-b53bdda69795-inventory\") pod \"neutron-ovn-edpm-multinodeset-edpm-compute-no-nodes-jdv6g\" (UID: \"10392605-c383-49fd-b4e9-b53bdda69795\") " pod="openstack-kuttl-tests/neutron-ovn-edpm-multinodeset-edpm-compute-no-nodes-jdv6g" Dec 03 13:04:32 crc kubenswrapper[5052]: I1203 13:04:32.483539 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"neutron-ovn-agent-neutron-config-0\" (UniqueName: \"kubernetes.io/secret/10392605-c383-49fd-b4e9-b53bdda69795-neutron-ovn-agent-neutron-config-0\") pod \"neutron-ovn-edpm-multinodeset-edpm-compute-no-nodes-jdv6g\" (UID: \"10392605-c383-49fd-b4e9-b53bdda69795\") " pod="openstack-kuttl-tests/neutron-ovn-edpm-multinodeset-edpm-compute-no-nodes-jdv6g" Dec 03 13:04:32 crc kubenswrapper[5052]: I1203 13:04:32.483587 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/10392605-c383-49fd-b4e9-b53bdda69795-ssh-key\") pod \"neutron-ovn-edpm-multinodeset-edpm-compute-no-nodes-jdv6g\" (UID: \"10392605-c383-49fd-b4e9-b53bdda69795\") " pod="openstack-kuttl-tests/neutron-ovn-edpm-multinodeset-edpm-compute-no-nodes-jdv6g" Dec 03 13:04:32 crc kubenswrapper[5052]: I1203 13:04:32.483626 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"neutron-ovn-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/10392605-c383-49fd-b4e9-b53bdda69795-neutron-ovn-combined-ca-bundle\") pod \"neutron-ovn-edpm-multinodeset-edpm-compute-no-nodes-jdv6g\" (UID: \"10392605-c383-49fd-b4e9-b53bdda69795\") " pod="openstack-kuttl-tests/neutron-ovn-edpm-multinodeset-edpm-compute-no-nodes-jdv6g" Dec 03 13:04:32 crc kubenswrapper[5052]: I1203 13:04:32.487818 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/10392605-c383-49fd-b4e9-b53bdda69795-inventory\") pod \"neutron-ovn-edpm-multinodeset-edpm-compute-no-nodes-jdv6g\" (UID: \"10392605-c383-49fd-b4e9-b53bdda69795\") " pod="openstack-kuttl-tests/neutron-ovn-edpm-multinodeset-edpm-compute-no-nodes-jdv6g" Dec 03 13:04:32 crc kubenswrapper[5052]: I1203 13:04:32.489915 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"neutron-ovn-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/10392605-c383-49fd-b4e9-b53bdda69795-neutron-ovn-combined-ca-bundle\") pod \"neutron-ovn-edpm-multinodeset-edpm-compute-no-nodes-jdv6g\" (UID: \"10392605-c383-49fd-b4e9-b53bdda69795\") " pod="openstack-kuttl-tests/neutron-ovn-edpm-multinodeset-edpm-compute-no-nodes-jdv6g" Dec 03 13:04:32 crc kubenswrapper[5052]: I1203 13:04:32.491253 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/10392605-c383-49fd-b4e9-b53bdda69795-ssh-key\") pod \"neutron-ovn-edpm-multinodeset-edpm-compute-no-nodes-jdv6g\" (UID: \"10392605-c383-49fd-b4e9-b53bdda69795\") " pod="openstack-kuttl-tests/neutron-ovn-edpm-multinodeset-edpm-compute-no-nodes-jdv6g" Dec 03 13:04:32 crc kubenswrapper[5052]: I1203 13:04:32.504450 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"neutron-ovn-agent-neutron-config-0\" (UniqueName: \"kubernetes.io/secret/10392605-c383-49fd-b4e9-b53bdda69795-neutron-ovn-agent-neutron-config-0\") pod \"neutron-ovn-edpm-multinodeset-edpm-compute-no-nodes-jdv6g\" (UID: \"10392605-c383-49fd-b4e9-b53bdda69795\") " pod="openstack-kuttl-tests/neutron-ovn-edpm-multinodeset-edpm-compute-no-nodes-jdv6g" Dec 03 13:04:32 crc kubenswrapper[5052]: I1203 13:04:32.505602 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-sbkss\" (UniqueName: \"kubernetes.io/projected/10392605-c383-49fd-b4e9-b53bdda69795-kube-api-access-sbkss\") pod \"neutron-ovn-edpm-multinodeset-edpm-compute-no-nodes-jdv6g\" (UID: \"10392605-c383-49fd-b4e9-b53bdda69795\") " pod="openstack-kuttl-tests/neutron-ovn-edpm-multinodeset-edpm-compute-no-nodes-jdv6g" Dec 03 13:04:32 crc kubenswrapper[5052]: I1203 13:04:32.608903 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-ovn-edpm-multinodeset-edpm-compute-no-nodes-jdv6g" Dec 03 13:04:33 crc kubenswrapper[5052]: I1203 13:04:33.074265 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-ovn-edpm-multinodeset-edpm-compute-no-nodes-jdv6g"] Dec 03 13:04:33 crc kubenswrapper[5052]: I1203 13:04:33.217972 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-ovn-edpm-multinodeset-edpm-compute-no-nodes-jdv6g" event={"ID":"10392605-c383-49fd-b4e9-b53bdda69795","Type":"ContainerStarted","Data":"9449fabb712f0b836f5b91e01f59516d92c938c8c8414ad9defe80d28161623f"} Dec 03 13:04:34 crc kubenswrapper[5052]: I1203 13:04:34.229918 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-ovn-edpm-multinodeset-edpm-compute-no-nodes-jdv6g" event={"ID":"10392605-c383-49fd-b4e9-b53bdda69795","Type":"ContainerStarted","Data":"c13e3f2c1e7922f82cf6b655ebfa96b329f60e10c37d64b31a8eb3d4b17c0e34"} Dec 03 13:04:34 crc kubenswrapper[5052]: I1203 13:04:34.256456 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/neutron-ovn-edpm-multinodeset-edpm-compute-no-nodes-jdv6g" podStartSLOduration=1.645430377 podStartE2EDuration="2.256431051s" podCreationTimestamp="2025-12-03 13:04:32 +0000 UTC" firstStartedPulling="2025-12-03 13:04:33.084628669 +0000 UTC m=+6263.672870423" lastFinishedPulling="2025-12-03 13:04:33.695629333 +0000 UTC m=+6264.283871097" observedRunningTime="2025-12-03 13:04:34.253359524 +0000 UTC m=+6264.841601288" watchObservedRunningTime="2025-12-03 13:04:34.256431051 +0000 UTC m=+6264.844672825" Dec 03 13:04:35 crc kubenswrapper[5052]: I1203 13:04:35.240088 5052 generic.go:334] "Generic (PLEG): container finished" podID="10392605-c383-49fd-b4e9-b53bdda69795" containerID="c13e3f2c1e7922f82cf6b655ebfa96b329f60e10c37d64b31a8eb3d4b17c0e34" exitCode=0 Dec 03 13:04:35 crc kubenswrapper[5052]: I1203 13:04:35.240184 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-ovn-edpm-multinodeset-edpm-compute-no-nodes-jdv6g" event={"ID":"10392605-c383-49fd-b4e9-b53bdda69795","Type":"ContainerDied","Data":"c13e3f2c1e7922f82cf6b655ebfa96b329f60e10c37d64b31a8eb3d4b17c0e34"} Dec 03 13:04:36 crc kubenswrapper[5052]: I1203 13:04:36.594241 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-ovn-edpm-multinodeset-edpm-compute-no-nodes-jdv6g" Dec 03 13:04:36 crc kubenswrapper[5052]: I1203 13:04:36.680107 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sbkss\" (UniqueName: \"kubernetes.io/projected/10392605-c383-49fd-b4e9-b53bdda69795-kube-api-access-sbkss\") pod \"10392605-c383-49fd-b4e9-b53bdda69795\" (UID: \"10392605-c383-49fd-b4e9-b53bdda69795\") " Dec 03 13:04:36 crc kubenswrapper[5052]: I1203 13:04:36.680170 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/10392605-c383-49fd-b4e9-b53bdda69795-ssh-key\") pod \"10392605-c383-49fd-b4e9-b53bdda69795\" (UID: \"10392605-c383-49fd-b4e9-b53bdda69795\") " Dec 03 13:04:36 crc kubenswrapper[5052]: I1203 13:04:36.680227 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"neutron-ovn-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/10392605-c383-49fd-b4e9-b53bdda69795-neutron-ovn-combined-ca-bundle\") pod \"10392605-c383-49fd-b4e9-b53bdda69795\" (UID: \"10392605-c383-49fd-b4e9-b53bdda69795\") " Dec 03 13:04:36 crc kubenswrapper[5052]: I1203 13:04:36.680312 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"neutron-ovn-agent-neutron-config-0\" (UniqueName: \"kubernetes.io/secret/10392605-c383-49fd-b4e9-b53bdda69795-neutron-ovn-agent-neutron-config-0\") pod \"10392605-c383-49fd-b4e9-b53bdda69795\" (UID: \"10392605-c383-49fd-b4e9-b53bdda69795\") " Dec 03 13:04:36 crc kubenswrapper[5052]: I1203 13:04:36.680355 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/10392605-c383-49fd-b4e9-b53bdda69795-inventory\") pod \"10392605-c383-49fd-b4e9-b53bdda69795\" (UID: \"10392605-c383-49fd-b4e9-b53bdda69795\") " Dec 03 13:04:36 crc kubenswrapper[5052]: I1203 13:04:36.686053 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/10392605-c383-49fd-b4e9-b53bdda69795-neutron-ovn-combined-ca-bundle" (OuterVolumeSpecName: "neutron-ovn-combined-ca-bundle") pod "10392605-c383-49fd-b4e9-b53bdda69795" (UID: "10392605-c383-49fd-b4e9-b53bdda69795"). InnerVolumeSpecName "neutron-ovn-combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:04:36 crc kubenswrapper[5052]: I1203 13:04:36.686162 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/10392605-c383-49fd-b4e9-b53bdda69795-kube-api-access-sbkss" (OuterVolumeSpecName: "kube-api-access-sbkss") pod "10392605-c383-49fd-b4e9-b53bdda69795" (UID: "10392605-c383-49fd-b4e9-b53bdda69795"). InnerVolumeSpecName "kube-api-access-sbkss". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:04:36 crc kubenswrapper[5052]: I1203 13:04:36.705021 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/10392605-c383-49fd-b4e9-b53bdda69795-neutron-ovn-agent-neutron-config-0" (OuterVolumeSpecName: "neutron-ovn-agent-neutron-config-0") pod "10392605-c383-49fd-b4e9-b53bdda69795" (UID: "10392605-c383-49fd-b4e9-b53bdda69795"). InnerVolumeSpecName "neutron-ovn-agent-neutron-config-0". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:04:36 crc kubenswrapper[5052]: I1203 13:04:36.705654 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/10392605-c383-49fd-b4e9-b53bdda69795-inventory" (OuterVolumeSpecName: "inventory") pod "10392605-c383-49fd-b4e9-b53bdda69795" (UID: "10392605-c383-49fd-b4e9-b53bdda69795"). InnerVolumeSpecName "inventory". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:04:36 crc kubenswrapper[5052]: I1203 13:04:36.711393 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/10392605-c383-49fd-b4e9-b53bdda69795-ssh-key" (OuterVolumeSpecName: "ssh-key") pod "10392605-c383-49fd-b4e9-b53bdda69795" (UID: "10392605-c383-49fd-b4e9-b53bdda69795"). InnerVolumeSpecName "ssh-key". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:04:36 crc kubenswrapper[5052]: I1203 13:04:36.782594 5052 reconciler_common.go:293] "Volume detached for volume \"neutron-ovn-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/10392605-c383-49fd-b4e9-b53bdda69795-neutron-ovn-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 13:04:36 crc kubenswrapper[5052]: I1203 13:04:36.782628 5052 reconciler_common.go:293] "Volume detached for volume \"neutron-ovn-agent-neutron-config-0\" (UniqueName: \"kubernetes.io/secret/10392605-c383-49fd-b4e9-b53bdda69795-neutron-ovn-agent-neutron-config-0\") on node \"crc\" DevicePath \"\"" Dec 03 13:04:36 crc kubenswrapper[5052]: I1203 13:04:36.782642 5052 reconciler_common.go:293] "Volume detached for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/10392605-c383-49fd-b4e9-b53bdda69795-inventory\") on node \"crc\" DevicePath \"\"" Dec 03 13:04:36 crc kubenswrapper[5052]: I1203 13:04:36.782655 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-sbkss\" (UniqueName: \"kubernetes.io/projected/10392605-c383-49fd-b4e9-b53bdda69795-kube-api-access-sbkss\") on node \"crc\" DevicePath \"\"" Dec 03 13:04:36 crc kubenswrapper[5052]: I1203 13:04:36.782666 5052 reconciler_common.go:293] "Volume detached for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/10392605-c383-49fd-b4e9-b53bdda69795-ssh-key\") on node \"crc\" DevicePath \"\"" Dec 03 13:04:37 crc kubenswrapper[5052]: I1203 13:04:37.265497 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-ovn-edpm-multinodeset-edpm-compute-no-nodes-jdv6g" event={"ID":"10392605-c383-49fd-b4e9-b53bdda69795","Type":"ContainerDied","Data":"9449fabb712f0b836f5b91e01f59516d92c938c8c8414ad9defe80d28161623f"} Dec 03 13:04:37 crc kubenswrapper[5052]: I1203 13:04:37.265551 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-ovn-edpm-multinodeset-edpm-compute-no-nodes-jdv6g" Dec 03 13:04:37 crc kubenswrapper[5052]: I1203 13:04:37.265567 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9449fabb712f0b836f5b91e01f59516d92c938c8c8414ad9defe80d28161623f" Dec 03 13:04:37 crc kubenswrapper[5052]: I1203 13:04:37.327091 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/neutron-sriov-edpm-multinodeset-edpm-compute-no-nodes-ccbd7"] Dec 03 13:04:37 crc kubenswrapper[5052]: E1203 13:04:37.327446 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="10392605-c383-49fd-b4e9-b53bdda69795" containerName="neutron-ovn-edpm-multinodeset-edpm-compute-no-nodes" Dec 03 13:04:37 crc kubenswrapper[5052]: I1203 13:04:37.327468 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="10392605-c383-49fd-b4e9-b53bdda69795" containerName="neutron-ovn-edpm-multinodeset-edpm-compute-no-nodes" Dec 03 13:04:37 crc kubenswrapper[5052]: I1203 13:04:37.327688 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="10392605-c383-49fd-b4e9-b53bdda69795" containerName="neutron-ovn-edpm-multinodeset-edpm-compute-no-nodes" Dec 03 13:04:37 crc kubenswrapper[5052]: I1203 13:04:37.328271 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-sriov-edpm-multinodeset-edpm-compute-no-nodes-ccbd7" Dec 03 13:04:37 crc kubenswrapper[5052]: I1203 13:04:37.330441 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"openstack-aee-default-env" Dec 03 13:04:37 crc kubenswrapper[5052]: I1203 13:04:37.330633 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"neutron-sriov-agent-neutron-config" Dec 03 13:04:37 crc kubenswrapper[5052]: I1203 13:04:37.331872 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"combined-ca-bundle" Dec 03 13:04:37 crc kubenswrapper[5052]: I1203 13:04:37.332060 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"edpm-compute-no-nodes-dockercfg-mkcgv" Dec 03 13:04:37 crc kubenswrapper[5052]: I1203 13:04:37.332215 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"dataplanenodeset-edpm-compute-no-nodes" Dec 03 13:04:37 crc kubenswrapper[5052]: I1203 13:04:37.332575 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"dataplane-ansible-ssh-private-key-secret" Dec 03 13:04:37 crc kubenswrapper[5052]: I1203 13:04:37.344258 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-sriov-edpm-multinodeset-edpm-compute-no-nodes-ccbd7"] Dec 03 13:04:37 crc kubenswrapper[5052]: I1203 13:04:37.394272 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/2b71a6d5-1e66-4601-9e86-f71fc1cd66a0-inventory\") pod \"neutron-sriov-edpm-multinodeset-edpm-compute-no-nodes-ccbd7\" (UID: \"2b71a6d5-1e66-4601-9e86-f71fc1cd66a0\") " pod="openstack-kuttl-tests/neutron-sriov-edpm-multinodeset-edpm-compute-no-nodes-ccbd7" Dec 03 13:04:37 crc kubenswrapper[5052]: I1203 13:04:37.394327 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"neutron-sriov-agent-neutron-config-0\" (UniqueName: \"kubernetes.io/secret/2b71a6d5-1e66-4601-9e86-f71fc1cd66a0-neutron-sriov-agent-neutron-config-0\") pod \"neutron-sriov-edpm-multinodeset-edpm-compute-no-nodes-ccbd7\" (UID: \"2b71a6d5-1e66-4601-9e86-f71fc1cd66a0\") " pod="openstack-kuttl-tests/neutron-sriov-edpm-multinodeset-edpm-compute-no-nodes-ccbd7" Dec 03 13:04:37 crc kubenswrapper[5052]: I1203 13:04:37.394360 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"neutron-sriov-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2b71a6d5-1e66-4601-9e86-f71fc1cd66a0-neutron-sriov-combined-ca-bundle\") pod \"neutron-sriov-edpm-multinodeset-edpm-compute-no-nodes-ccbd7\" (UID: \"2b71a6d5-1e66-4601-9e86-f71fc1cd66a0\") " pod="openstack-kuttl-tests/neutron-sriov-edpm-multinodeset-edpm-compute-no-nodes-ccbd7" Dec 03 13:04:37 crc kubenswrapper[5052]: I1203 13:04:37.394389 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hg5pj\" (UniqueName: \"kubernetes.io/projected/2b71a6d5-1e66-4601-9e86-f71fc1cd66a0-kube-api-access-hg5pj\") pod \"neutron-sriov-edpm-multinodeset-edpm-compute-no-nodes-ccbd7\" (UID: \"2b71a6d5-1e66-4601-9e86-f71fc1cd66a0\") " pod="openstack-kuttl-tests/neutron-sriov-edpm-multinodeset-edpm-compute-no-nodes-ccbd7" Dec 03 13:04:37 crc kubenswrapper[5052]: I1203 13:04:37.394724 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/2b71a6d5-1e66-4601-9e86-f71fc1cd66a0-ssh-key\") pod \"neutron-sriov-edpm-multinodeset-edpm-compute-no-nodes-ccbd7\" (UID: \"2b71a6d5-1e66-4601-9e86-f71fc1cd66a0\") " pod="openstack-kuttl-tests/neutron-sriov-edpm-multinodeset-edpm-compute-no-nodes-ccbd7" Dec 03 13:04:37 crc kubenswrapper[5052]: I1203 13:04:37.496109 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/2b71a6d5-1e66-4601-9e86-f71fc1cd66a0-ssh-key\") pod \"neutron-sriov-edpm-multinodeset-edpm-compute-no-nodes-ccbd7\" (UID: \"2b71a6d5-1e66-4601-9e86-f71fc1cd66a0\") " pod="openstack-kuttl-tests/neutron-sriov-edpm-multinodeset-edpm-compute-no-nodes-ccbd7" Dec 03 13:04:37 crc kubenswrapper[5052]: I1203 13:04:37.496222 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/2b71a6d5-1e66-4601-9e86-f71fc1cd66a0-inventory\") pod \"neutron-sriov-edpm-multinodeset-edpm-compute-no-nodes-ccbd7\" (UID: \"2b71a6d5-1e66-4601-9e86-f71fc1cd66a0\") " pod="openstack-kuttl-tests/neutron-sriov-edpm-multinodeset-edpm-compute-no-nodes-ccbd7" Dec 03 13:04:37 crc kubenswrapper[5052]: I1203 13:04:37.496279 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"neutron-sriov-agent-neutron-config-0\" (UniqueName: \"kubernetes.io/secret/2b71a6d5-1e66-4601-9e86-f71fc1cd66a0-neutron-sriov-agent-neutron-config-0\") pod \"neutron-sriov-edpm-multinodeset-edpm-compute-no-nodes-ccbd7\" (UID: \"2b71a6d5-1e66-4601-9e86-f71fc1cd66a0\") " pod="openstack-kuttl-tests/neutron-sriov-edpm-multinodeset-edpm-compute-no-nodes-ccbd7" Dec 03 13:04:37 crc kubenswrapper[5052]: I1203 13:04:37.496352 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"neutron-sriov-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2b71a6d5-1e66-4601-9e86-f71fc1cd66a0-neutron-sriov-combined-ca-bundle\") pod \"neutron-sriov-edpm-multinodeset-edpm-compute-no-nodes-ccbd7\" (UID: \"2b71a6d5-1e66-4601-9e86-f71fc1cd66a0\") " pod="openstack-kuttl-tests/neutron-sriov-edpm-multinodeset-edpm-compute-no-nodes-ccbd7" Dec 03 13:04:37 crc kubenswrapper[5052]: I1203 13:04:37.496422 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-hg5pj\" (UniqueName: \"kubernetes.io/projected/2b71a6d5-1e66-4601-9e86-f71fc1cd66a0-kube-api-access-hg5pj\") pod \"neutron-sriov-edpm-multinodeset-edpm-compute-no-nodes-ccbd7\" (UID: \"2b71a6d5-1e66-4601-9e86-f71fc1cd66a0\") " pod="openstack-kuttl-tests/neutron-sriov-edpm-multinodeset-edpm-compute-no-nodes-ccbd7" Dec 03 13:04:37 crc kubenswrapper[5052]: I1203 13:04:37.500475 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/2b71a6d5-1e66-4601-9e86-f71fc1cd66a0-ssh-key\") pod \"neutron-sriov-edpm-multinodeset-edpm-compute-no-nodes-ccbd7\" (UID: \"2b71a6d5-1e66-4601-9e86-f71fc1cd66a0\") " pod="openstack-kuttl-tests/neutron-sriov-edpm-multinodeset-edpm-compute-no-nodes-ccbd7" Dec 03 13:04:37 crc kubenswrapper[5052]: I1203 13:04:37.500561 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"neutron-sriov-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2b71a6d5-1e66-4601-9e86-f71fc1cd66a0-neutron-sriov-combined-ca-bundle\") pod \"neutron-sriov-edpm-multinodeset-edpm-compute-no-nodes-ccbd7\" (UID: \"2b71a6d5-1e66-4601-9e86-f71fc1cd66a0\") " pod="openstack-kuttl-tests/neutron-sriov-edpm-multinodeset-edpm-compute-no-nodes-ccbd7" Dec 03 13:04:37 crc kubenswrapper[5052]: I1203 13:04:37.500805 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"neutron-sriov-agent-neutron-config-0\" (UniqueName: \"kubernetes.io/secret/2b71a6d5-1e66-4601-9e86-f71fc1cd66a0-neutron-sriov-agent-neutron-config-0\") pod \"neutron-sriov-edpm-multinodeset-edpm-compute-no-nodes-ccbd7\" (UID: \"2b71a6d5-1e66-4601-9e86-f71fc1cd66a0\") " pod="openstack-kuttl-tests/neutron-sriov-edpm-multinodeset-edpm-compute-no-nodes-ccbd7" Dec 03 13:04:37 crc kubenswrapper[5052]: I1203 13:04:37.500820 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/2b71a6d5-1e66-4601-9e86-f71fc1cd66a0-inventory\") pod \"neutron-sriov-edpm-multinodeset-edpm-compute-no-nodes-ccbd7\" (UID: \"2b71a6d5-1e66-4601-9e86-f71fc1cd66a0\") " pod="openstack-kuttl-tests/neutron-sriov-edpm-multinodeset-edpm-compute-no-nodes-ccbd7" Dec 03 13:04:37 crc kubenswrapper[5052]: I1203 13:04:37.520950 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-hg5pj\" (UniqueName: \"kubernetes.io/projected/2b71a6d5-1e66-4601-9e86-f71fc1cd66a0-kube-api-access-hg5pj\") pod \"neutron-sriov-edpm-multinodeset-edpm-compute-no-nodes-ccbd7\" (UID: \"2b71a6d5-1e66-4601-9e86-f71fc1cd66a0\") " pod="openstack-kuttl-tests/neutron-sriov-edpm-multinodeset-edpm-compute-no-nodes-ccbd7" Dec 03 13:04:37 crc kubenswrapper[5052]: I1203 13:04:37.646707 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-sriov-edpm-multinodeset-edpm-compute-no-nodes-ccbd7" Dec 03 13:04:38 crc kubenswrapper[5052]: I1203 13:04:38.359426 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-sriov-edpm-multinodeset-edpm-compute-no-nodes-ccbd7"] Dec 03 13:04:38 crc kubenswrapper[5052]: W1203 13:04:38.362018 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2b71a6d5_1e66_4601_9e86_f71fc1cd66a0.slice/crio-ea09601b40d8f519400d8daed2487a5faf4a53e03471e513198adea60cefccf7 WatchSource:0}: Error finding container ea09601b40d8f519400d8daed2487a5faf4a53e03471e513198adea60cefccf7: Status 404 returned error can't find the container with id ea09601b40d8f519400d8daed2487a5faf4a53e03471e513198adea60cefccf7 Dec 03 13:04:39 crc kubenswrapper[5052]: I1203 13:04:39.290832 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-sriov-edpm-multinodeset-edpm-compute-no-nodes-ccbd7" event={"ID":"2b71a6d5-1e66-4601-9e86-f71fc1cd66a0","Type":"ContainerStarted","Data":"e15c56c4973e2554de21d4b43c45463964918fa85401b7c3ae40228c1566f441"} Dec 03 13:04:39 crc kubenswrapper[5052]: I1203 13:04:39.291198 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-sriov-edpm-multinodeset-edpm-compute-no-nodes-ccbd7" event={"ID":"2b71a6d5-1e66-4601-9e86-f71fc1cd66a0","Type":"ContainerStarted","Data":"ea09601b40d8f519400d8daed2487a5faf4a53e03471e513198adea60cefccf7"} Dec 03 13:04:39 crc kubenswrapper[5052]: I1203 13:04:39.317443 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/neutron-sriov-edpm-multinodeset-edpm-compute-no-nodes-ccbd7" podStartSLOduration=1.780298267 podStartE2EDuration="2.317426218s" podCreationTimestamp="2025-12-03 13:04:37 +0000 UTC" firstStartedPulling="2025-12-03 13:04:38.364224977 +0000 UTC m=+6268.952466741" lastFinishedPulling="2025-12-03 13:04:38.901352928 +0000 UTC m=+6269.489594692" observedRunningTime="2025-12-03 13:04:39.312248989 +0000 UTC m=+6269.900490753" watchObservedRunningTime="2025-12-03 13:04:39.317426218 +0000 UTC m=+6269.905667982" Dec 03 13:04:41 crc kubenswrapper[5052]: I1203 13:04:41.195491 5052 scope.go:117] "RemoveContainer" containerID="1fa4faeb71c3c04c4a155f9896519f0bee40159795401bac2d43ec3f2949c08a" Dec 03 13:04:41 crc kubenswrapper[5052]: I1203 13:04:41.223589 5052 scope.go:117] "RemoveContainer" containerID="173e3047b0b5ffbab691f7ddc4b2f4f315ad802a64b3aec944a4b88629cd0168" Dec 03 13:04:41 crc kubenswrapper[5052]: I1203 13:04:41.260650 5052 scope.go:117] "RemoveContainer" containerID="1104c2716cc817488412d6a7a780f7984a797cc98e5ee047d93cc2777df6e2e1" Dec 03 13:04:41 crc kubenswrapper[5052]: I1203 13:04:41.284443 5052 scope.go:117] "RemoveContainer" containerID="d735823ce96b93fda511b9255b1132e029a33bd9974ec2e78b0f58272e2cc67f" Dec 03 13:04:41 crc kubenswrapper[5052]: I1203 13:04:41.320188 5052 scope.go:117] "RemoveContainer" containerID="2a81f6bb9846792fbc11645aa0fd014b92a33f1f4c49de20cbbcdd0f447029af" Dec 03 13:04:41 crc kubenswrapper[5052]: I1203 13:04:41.328289 5052 generic.go:334] "Generic (PLEG): container finished" podID="2b71a6d5-1e66-4601-9e86-f71fc1cd66a0" containerID="e15c56c4973e2554de21d4b43c45463964918fa85401b7c3ae40228c1566f441" exitCode=0 Dec 03 13:04:41 crc kubenswrapper[5052]: I1203 13:04:41.328367 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-sriov-edpm-multinodeset-edpm-compute-no-nodes-ccbd7" event={"ID":"2b71a6d5-1e66-4601-9e86-f71fc1cd66a0","Type":"ContainerDied","Data":"e15c56c4973e2554de21d4b43c45463964918fa85401b7c3ae40228c1566f441"} Dec 03 13:04:41 crc kubenswrapper[5052]: I1203 13:04:41.352873 5052 scope.go:117] "RemoveContainer" containerID="afc3b9ec0f6569d43e8044e0226a3a0167f6f4148750c2a219b6f5318339c9c1" Dec 03 13:04:41 crc kubenswrapper[5052]: I1203 13:04:41.380923 5052 scope.go:117] "RemoveContainer" containerID="def25f6d5d263d80362de9d1d7e5e67b28a08f4e2bbd4d56a2771bd32a5cf8bf" Dec 03 13:04:41 crc kubenswrapper[5052]: I1203 13:04:41.409071 5052 scope.go:117] "RemoveContainer" containerID="7dbab62565102d37d525bbacfd4fb0209c6df3d459afec16badc1049d2aa5f14" Dec 03 13:04:41 crc kubenswrapper[5052]: I1203 13:04:41.432082 5052 scope.go:117] "RemoveContainer" containerID="2dbf38c62ebda73087ee02131872825f86e7d69a9f2119ca9bc3dfc54aeaa021" Dec 03 13:04:42 crc kubenswrapper[5052]: I1203 13:04:42.716327 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-sriov-edpm-multinodeset-edpm-compute-no-nodes-ccbd7" Dec 03 13:04:42 crc kubenswrapper[5052]: I1203 13:04:42.784319 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"neutron-sriov-agent-neutron-config-0\" (UniqueName: \"kubernetes.io/secret/2b71a6d5-1e66-4601-9e86-f71fc1cd66a0-neutron-sriov-agent-neutron-config-0\") pod \"2b71a6d5-1e66-4601-9e86-f71fc1cd66a0\" (UID: \"2b71a6d5-1e66-4601-9e86-f71fc1cd66a0\") " Dec 03 13:04:42 crc kubenswrapper[5052]: I1203 13:04:42.784416 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hg5pj\" (UniqueName: \"kubernetes.io/projected/2b71a6d5-1e66-4601-9e86-f71fc1cd66a0-kube-api-access-hg5pj\") pod \"2b71a6d5-1e66-4601-9e86-f71fc1cd66a0\" (UID: \"2b71a6d5-1e66-4601-9e86-f71fc1cd66a0\") " Dec 03 13:04:42 crc kubenswrapper[5052]: I1203 13:04:42.784465 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"neutron-sriov-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2b71a6d5-1e66-4601-9e86-f71fc1cd66a0-neutron-sriov-combined-ca-bundle\") pod \"2b71a6d5-1e66-4601-9e86-f71fc1cd66a0\" (UID: \"2b71a6d5-1e66-4601-9e86-f71fc1cd66a0\") " Dec 03 13:04:42 crc kubenswrapper[5052]: I1203 13:04:42.784572 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/2b71a6d5-1e66-4601-9e86-f71fc1cd66a0-ssh-key\") pod \"2b71a6d5-1e66-4601-9e86-f71fc1cd66a0\" (UID: \"2b71a6d5-1e66-4601-9e86-f71fc1cd66a0\") " Dec 03 13:04:42 crc kubenswrapper[5052]: I1203 13:04:42.784647 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/2b71a6d5-1e66-4601-9e86-f71fc1cd66a0-inventory\") pod \"2b71a6d5-1e66-4601-9e86-f71fc1cd66a0\" (UID: \"2b71a6d5-1e66-4601-9e86-f71fc1cd66a0\") " Dec 03 13:04:42 crc kubenswrapper[5052]: I1203 13:04:42.792800 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2b71a6d5-1e66-4601-9e86-f71fc1cd66a0-kube-api-access-hg5pj" (OuterVolumeSpecName: "kube-api-access-hg5pj") pod "2b71a6d5-1e66-4601-9e86-f71fc1cd66a0" (UID: "2b71a6d5-1e66-4601-9e86-f71fc1cd66a0"). InnerVolumeSpecName "kube-api-access-hg5pj". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:04:42 crc kubenswrapper[5052]: I1203 13:04:42.799656 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2b71a6d5-1e66-4601-9e86-f71fc1cd66a0-neutron-sriov-combined-ca-bundle" (OuterVolumeSpecName: "neutron-sriov-combined-ca-bundle") pod "2b71a6d5-1e66-4601-9e86-f71fc1cd66a0" (UID: "2b71a6d5-1e66-4601-9e86-f71fc1cd66a0"). InnerVolumeSpecName "neutron-sriov-combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:04:42 crc kubenswrapper[5052]: I1203 13:04:42.816975 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2b71a6d5-1e66-4601-9e86-f71fc1cd66a0-ssh-key" (OuterVolumeSpecName: "ssh-key") pod "2b71a6d5-1e66-4601-9e86-f71fc1cd66a0" (UID: "2b71a6d5-1e66-4601-9e86-f71fc1cd66a0"). InnerVolumeSpecName "ssh-key". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:04:42 crc kubenswrapper[5052]: I1203 13:04:42.817432 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2b71a6d5-1e66-4601-9e86-f71fc1cd66a0-inventory" (OuterVolumeSpecName: "inventory") pod "2b71a6d5-1e66-4601-9e86-f71fc1cd66a0" (UID: "2b71a6d5-1e66-4601-9e86-f71fc1cd66a0"). InnerVolumeSpecName "inventory". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:04:42 crc kubenswrapper[5052]: I1203 13:04:42.820809 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2b71a6d5-1e66-4601-9e86-f71fc1cd66a0-neutron-sriov-agent-neutron-config-0" (OuterVolumeSpecName: "neutron-sriov-agent-neutron-config-0") pod "2b71a6d5-1e66-4601-9e86-f71fc1cd66a0" (UID: "2b71a6d5-1e66-4601-9e86-f71fc1cd66a0"). InnerVolumeSpecName "neutron-sriov-agent-neutron-config-0". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:04:42 crc kubenswrapper[5052]: I1203 13:04:42.886372 5052 reconciler_common.go:293] "Volume detached for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/2b71a6d5-1e66-4601-9e86-f71fc1cd66a0-inventory\") on node \"crc\" DevicePath \"\"" Dec 03 13:04:42 crc kubenswrapper[5052]: I1203 13:04:42.886421 5052 reconciler_common.go:293] "Volume detached for volume \"neutron-sriov-agent-neutron-config-0\" (UniqueName: \"kubernetes.io/secret/2b71a6d5-1e66-4601-9e86-f71fc1cd66a0-neutron-sriov-agent-neutron-config-0\") on node \"crc\" DevicePath \"\"" Dec 03 13:04:42 crc kubenswrapper[5052]: I1203 13:04:42.886443 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-hg5pj\" (UniqueName: \"kubernetes.io/projected/2b71a6d5-1e66-4601-9e86-f71fc1cd66a0-kube-api-access-hg5pj\") on node \"crc\" DevicePath \"\"" Dec 03 13:04:42 crc kubenswrapper[5052]: I1203 13:04:42.886458 5052 reconciler_common.go:293] "Volume detached for volume \"neutron-sriov-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2b71a6d5-1e66-4601-9e86-f71fc1cd66a0-neutron-sriov-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 13:04:42 crc kubenswrapper[5052]: I1203 13:04:42.886471 5052 reconciler_common.go:293] "Volume detached for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/2b71a6d5-1e66-4601-9e86-f71fc1cd66a0-ssh-key\") on node \"crc\" DevicePath \"\"" Dec 03 13:04:43 crc kubenswrapper[5052]: I1203 13:04:43.355451 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-sriov-edpm-multinodeset-edpm-compute-no-nodes-ccbd7" event={"ID":"2b71a6d5-1e66-4601-9e86-f71fc1cd66a0","Type":"ContainerDied","Data":"ea09601b40d8f519400d8daed2487a5faf4a53e03471e513198adea60cefccf7"} Dec 03 13:04:43 crc kubenswrapper[5052]: I1203 13:04:43.355494 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-sriov-edpm-multinodeset-edpm-compute-no-nodes-ccbd7" Dec 03 13:04:43 crc kubenswrapper[5052]: I1203 13:04:43.355497 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ea09601b40d8f519400d8daed2487a5faf4a53e03471e513198adea60cefccf7" Dec 03 13:04:43 crc kubenswrapper[5052]: I1203 13:04:43.440586 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/neutron-dhcp-edpm-multinodeset-edpm-compute-no-nodes-269x6"] Dec 03 13:04:43 crc kubenswrapper[5052]: E1203 13:04:43.441098 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="2b71a6d5-1e66-4601-9e86-f71fc1cd66a0" containerName="neutron-sriov-edpm-multinodeset-edpm-compute-no-nodes" Dec 03 13:04:43 crc kubenswrapper[5052]: I1203 13:04:43.441119 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="2b71a6d5-1e66-4601-9e86-f71fc1cd66a0" containerName="neutron-sriov-edpm-multinodeset-edpm-compute-no-nodes" Dec 03 13:04:43 crc kubenswrapper[5052]: I1203 13:04:43.441363 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="2b71a6d5-1e66-4601-9e86-f71fc1cd66a0" containerName="neutron-sriov-edpm-multinodeset-edpm-compute-no-nodes" Dec 03 13:04:43 crc kubenswrapper[5052]: I1203 13:04:43.442263 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-dhcp-edpm-multinodeset-edpm-compute-no-nodes-269x6" Dec 03 13:04:43 crc kubenswrapper[5052]: I1203 13:04:43.445972 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"dataplane-ansible-ssh-private-key-secret" Dec 03 13:04:43 crc kubenswrapper[5052]: I1203 13:04:43.445993 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"openstack-aee-default-env" Dec 03 13:04:43 crc kubenswrapper[5052]: I1203 13:04:43.446017 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"combined-ca-bundle" Dec 03 13:04:43 crc kubenswrapper[5052]: I1203 13:04:43.446465 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"neutron-dhcp-agent-neutron-config" Dec 03 13:04:43 crc kubenswrapper[5052]: I1203 13:04:43.448999 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-dhcp-edpm-multinodeset-edpm-compute-no-nodes-269x6"] Dec 03 13:04:43 crc kubenswrapper[5052]: I1203 13:04:43.449025 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"edpm-compute-no-nodes-dockercfg-mkcgv" Dec 03 13:04:43 crc kubenswrapper[5052]: I1203 13:04:43.450455 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"dataplanenodeset-edpm-compute-no-nodes" Dec 03 13:04:43 crc kubenswrapper[5052]: I1203 13:04:43.598339 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r4rx8\" (UniqueName: \"kubernetes.io/projected/8233e500-8181-426b-aaf3-c7ecf3251d4c-kube-api-access-r4rx8\") pod \"neutron-dhcp-edpm-multinodeset-edpm-compute-no-nodes-269x6\" (UID: \"8233e500-8181-426b-aaf3-c7ecf3251d4c\") " pod="openstack-kuttl-tests/neutron-dhcp-edpm-multinodeset-edpm-compute-no-nodes-269x6" Dec 03 13:04:43 crc kubenswrapper[5052]: I1203 13:04:43.598416 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"neutron-dhcp-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8233e500-8181-426b-aaf3-c7ecf3251d4c-neutron-dhcp-combined-ca-bundle\") pod \"neutron-dhcp-edpm-multinodeset-edpm-compute-no-nodes-269x6\" (UID: \"8233e500-8181-426b-aaf3-c7ecf3251d4c\") " pod="openstack-kuttl-tests/neutron-dhcp-edpm-multinodeset-edpm-compute-no-nodes-269x6" Dec 03 13:04:43 crc kubenswrapper[5052]: I1203 13:04:43.598449 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/8233e500-8181-426b-aaf3-c7ecf3251d4c-ssh-key\") pod \"neutron-dhcp-edpm-multinodeset-edpm-compute-no-nodes-269x6\" (UID: \"8233e500-8181-426b-aaf3-c7ecf3251d4c\") " pod="openstack-kuttl-tests/neutron-dhcp-edpm-multinodeset-edpm-compute-no-nodes-269x6" Dec 03 13:04:43 crc kubenswrapper[5052]: I1203 13:04:43.598480 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"neutron-dhcp-agent-neutron-config-0\" (UniqueName: \"kubernetes.io/secret/8233e500-8181-426b-aaf3-c7ecf3251d4c-neutron-dhcp-agent-neutron-config-0\") pod \"neutron-dhcp-edpm-multinodeset-edpm-compute-no-nodes-269x6\" (UID: \"8233e500-8181-426b-aaf3-c7ecf3251d4c\") " pod="openstack-kuttl-tests/neutron-dhcp-edpm-multinodeset-edpm-compute-no-nodes-269x6" Dec 03 13:04:43 crc kubenswrapper[5052]: I1203 13:04:43.598520 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/8233e500-8181-426b-aaf3-c7ecf3251d4c-inventory\") pod \"neutron-dhcp-edpm-multinodeset-edpm-compute-no-nodes-269x6\" (UID: \"8233e500-8181-426b-aaf3-c7ecf3251d4c\") " pod="openstack-kuttl-tests/neutron-dhcp-edpm-multinodeset-edpm-compute-no-nodes-269x6" Dec 03 13:04:43 crc kubenswrapper[5052]: I1203 13:04:43.700052 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"neutron-dhcp-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8233e500-8181-426b-aaf3-c7ecf3251d4c-neutron-dhcp-combined-ca-bundle\") pod \"neutron-dhcp-edpm-multinodeset-edpm-compute-no-nodes-269x6\" (UID: \"8233e500-8181-426b-aaf3-c7ecf3251d4c\") " pod="openstack-kuttl-tests/neutron-dhcp-edpm-multinodeset-edpm-compute-no-nodes-269x6" Dec 03 13:04:43 crc kubenswrapper[5052]: I1203 13:04:43.700129 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/8233e500-8181-426b-aaf3-c7ecf3251d4c-ssh-key\") pod \"neutron-dhcp-edpm-multinodeset-edpm-compute-no-nodes-269x6\" (UID: \"8233e500-8181-426b-aaf3-c7ecf3251d4c\") " pod="openstack-kuttl-tests/neutron-dhcp-edpm-multinodeset-edpm-compute-no-nodes-269x6" Dec 03 13:04:43 crc kubenswrapper[5052]: I1203 13:04:43.700175 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"neutron-dhcp-agent-neutron-config-0\" (UniqueName: \"kubernetes.io/secret/8233e500-8181-426b-aaf3-c7ecf3251d4c-neutron-dhcp-agent-neutron-config-0\") pod \"neutron-dhcp-edpm-multinodeset-edpm-compute-no-nodes-269x6\" (UID: \"8233e500-8181-426b-aaf3-c7ecf3251d4c\") " pod="openstack-kuttl-tests/neutron-dhcp-edpm-multinodeset-edpm-compute-no-nodes-269x6" Dec 03 13:04:43 crc kubenswrapper[5052]: I1203 13:04:43.700208 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/8233e500-8181-426b-aaf3-c7ecf3251d4c-inventory\") pod \"neutron-dhcp-edpm-multinodeset-edpm-compute-no-nodes-269x6\" (UID: \"8233e500-8181-426b-aaf3-c7ecf3251d4c\") " pod="openstack-kuttl-tests/neutron-dhcp-edpm-multinodeset-edpm-compute-no-nodes-269x6" Dec 03 13:04:43 crc kubenswrapper[5052]: I1203 13:04:43.700411 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-r4rx8\" (UniqueName: \"kubernetes.io/projected/8233e500-8181-426b-aaf3-c7ecf3251d4c-kube-api-access-r4rx8\") pod \"neutron-dhcp-edpm-multinodeset-edpm-compute-no-nodes-269x6\" (UID: \"8233e500-8181-426b-aaf3-c7ecf3251d4c\") " pod="openstack-kuttl-tests/neutron-dhcp-edpm-multinodeset-edpm-compute-no-nodes-269x6" Dec 03 13:04:43 crc kubenswrapper[5052]: I1203 13:04:43.704953 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"neutron-dhcp-agent-neutron-config-0\" (UniqueName: \"kubernetes.io/secret/8233e500-8181-426b-aaf3-c7ecf3251d4c-neutron-dhcp-agent-neutron-config-0\") pod \"neutron-dhcp-edpm-multinodeset-edpm-compute-no-nodes-269x6\" (UID: \"8233e500-8181-426b-aaf3-c7ecf3251d4c\") " pod="openstack-kuttl-tests/neutron-dhcp-edpm-multinodeset-edpm-compute-no-nodes-269x6" Dec 03 13:04:43 crc kubenswrapper[5052]: I1203 13:04:43.705092 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/8233e500-8181-426b-aaf3-c7ecf3251d4c-ssh-key\") pod \"neutron-dhcp-edpm-multinodeset-edpm-compute-no-nodes-269x6\" (UID: \"8233e500-8181-426b-aaf3-c7ecf3251d4c\") " pod="openstack-kuttl-tests/neutron-dhcp-edpm-multinodeset-edpm-compute-no-nodes-269x6" Dec 03 13:04:43 crc kubenswrapper[5052]: I1203 13:04:43.709184 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"neutron-dhcp-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8233e500-8181-426b-aaf3-c7ecf3251d4c-neutron-dhcp-combined-ca-bundle\") pod \"neutron-dhcp-edpm-multinodeset-edpm-compute-no-nodes-269x6\" (UID: \"8233e500-8181-426b-aaf3-c7ecf3251d4c\") " pod="openstack-kuttl-tests/neutron-dhcp-edpm-multinodeset-edpm-compute-no-nodes-269x6" Dec 03 13:04:43 crc kubenswrapper[5052]: I1203 13:04:43.709677 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/8233e500-8181-426b-aaf3-c7ecf3251d4c-inventory\") pod \"neutron-dhcp-edpm-multinodeset-edpm-compute-no-nodes-269x6\" (UID: \"8233e500-8181-426b-aaf3-c7ecf3251d4c\") " pod="openstack-kuttl-tests/neutron-dhcp-edpm-multinodeset-edpm-compute-no-nodes-269x6" Dec 03 13:04:43 crc kubenswrapper[5052]: I1203 13:04:43.727443 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-r4rx8\" (UniqueName: \"kubernetes.io/projected/8233e500-8181-426b-aaf3-c7ecf3251d4c-kube-api-access-r4rx8\") pod \"neutron-dhcp-edpm-multinodeset-edpm-compute-no-nodes-269x6\" (UID: \"8233e500-8181-426b-aaf3-c7ecf3251d4c\") " pod="openstack-kuttl-tests/neutron-dhcp-edpm-multinodeset-edpm-compute-no-nodes-269x6" Dec 03 13:04:43 crc kubenswrapper[5052]: I1203 13:04:43.760121 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-dhcp-edpm-multinodeset-edpm-compute-no-nodes-269x6" Dec 03 13:04:44 crc kubenswrapper[5052]: I1203 13:04:44.166394 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/neutron-dhcp-edpm-multinodeset-edpm-compute-no-nodes-269x6"] Dec 03 13:04:44 crc kubenswrapper[5052]: W1203 13:04:44.172620 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8233e500_8181_426b_aaf3_c7ecf3251d4c.slice/crio-d4daf9b1b2a772433ab6a57574c22c29531dd9f3a7f31f70da4d0acffc6c385b WatchSource:0}: Error finding container d4daf9b1b2a772433ab6a57574c22c29531dd9f3a7f31f70da4d0acffc6c385b: Status 404 returned error can't find the container with id d4daf9b1b2a772433ab6a57574c22c29531dd9f3a7f31f70da4d0acffc6c385b Dec 03 13:04:44 crc kubenswrapper[5052]: I1203 13:04:44.379866 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-dhcp-edpm-multinodeset-edpm-compute-no-nodes-269x6" event={"ID":"8233e500-8181-426b-aaf3-c7ecf3251d4c","Type":"ContainerStarted","Data":"d4daf9b1b2a772433ab6a57574c22c29531dd9f3a7f31f70da4d0acffc6c385b"} Dec 03 13:04:45 crc kubenswrapper[5052]: I1203 13:04:45.389387 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-dhcp-edpm-multinodeset-edpm-compute-no-nodes-269x6" event={"ID":"8233e500-8181-426b-aaf3-c7ecf3251d4c","Type":"ContainerStarted","Data":"df668bb734666f2293cb0286d21d224f05868ec6e232b49bbfa1d595aec11db4"} Dec 03 13:04:45 crc kubenswrapper[5052]: I1203 13:04:45.406880 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/neutron-dhcp-edpm-multinodeset-edpm-compute-no-nodes-269x6" podStartSLOduration=1.605658401 podStartE2EDuration="2.406862224s" podCreationTimestamp="2025-12-03 13:04:43 +0000 UTC" firstStartedPulling="2025-12-03 13:04:44.175417957 +0000 UTC m=+6274.763659731" lastFinishedPulling="2025-12-03 13:04:44.97662178 +0000 UTC m=+6275.564863554" observedRunningTime="2025-12-03 13:04:45.403663193 +0000 UTC m=+6275.991904977" watchObservedRunningTime="2025-12-03 13:04:45.406862224 +0000 UTC m=+6275.995103988" Dec 03 13:04:47 crc kubenswrapper[5052]: I1203 13:04:47.411185 5052 generic.go:334] "Generic (PLEG): container finished" podID="8233e500-8181-426b-aaf3-c7ecf3251d4c" containerID="df668bb734666f2293cb0286d21d224f05868ec6e232b49bbfa1d595aec11db4" exitCode=0 Dec 03 13:04:47 crc kubenswrapper[5052]: I1203 13:04:47.411238 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-dhcp-edpm-multinodeset-edpm-compute-no-nodes-269x6" event={"ID":"8233e500-8181-426b-aaf3-c7ecf3251d4c","Type":"ContainerDied","Data":"df668bb734666f2293cb0286d21d224f05868ec6e232b49bbfa1d595aec11db4"} Dec 03 13:04:48 crc kubenswrapper[5052]: I1203 13:04:48.740872 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-dhcp-edpm-multinodeset-edpm-compute-no-nodes-269x6" Dec 03 13:04:48 crc kubenswrapper[5052]: I1203 13:04:48.877362 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r4rx8\" (UniqueName: \"kubernetes.io/projected/8233e500-8181-426b-aaf3-c7ecf3251d4c-kube-api-access-r4rx8\") pod \"8233e500-8181-426b-aaf3-c7ecf3251d4c\" (UID: \"8233e500-8181-426b-aaf3-c7ecf3251d4c\") " Dec 03 13:04:48 crc kubenswrapper[5052]: I1203 13:04:48.877807 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"neutron-dhcp-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8233e500-8181-426b-aaf3-c7ecf3251d4c-neutron-dhcp-combined-ca-bundle\") pod \"8233e500-8181-426b-aaf3-c7ecf3251d4c\" (UID: \"8233e500-8181-426b-aaf3-c7ecf3251d4c\") " Dec 03 13:04:48 crc kubenswrapper[5052]: I1203 13:04:48.877984 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"neutron-dhcp-agent-neutron-config-0\" (UniqueName: \"kubernetes.io/secret/8233e500-8181-426b-aaf3-c7ecf3251d4c-neutron-dhcp-agent-neutron-config-0\") pod \"8233e500-8181-426b-aaf3-c7ecf3251d4c\" (UID: \"8233e500-8181-426b-aaf3-c7ecf3251d4c\") " Dec 03 13:04:48 crc kubenswrapper[5052]: I1203 13:04:48.878180 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/8233e500-8181-426b-aaf3-c7ecf3251d4c-ssh-key\") pod \"8233e500-8181-426b-aaf3-c7ecf3251d4c\" (UID: \"8233e500-8181-426b-aaf3-c7ecf3251d4c\") " Dec 03 13:04:48 crc kubenswrapper[5052]: I1203 13:04:48.878351 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/8233e500-8181-426b-aaf3-c7ecf3251d4c-inventory\") pod \"8233e500-8181-426b-aaf3-c7ecf3251d4c\" (UID: \"8233e500-8181-426b-aaf3-c7ecf3251d4c\") " Dec 03 13:04:48 crc kubenswrapper[5052]: I1203 13:04:48.884700 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8233e500-8181-426b-aaf3-c7ecf3251d4c-neutron-dhcp-combined-ca-bundle" (OuterVolumeSpecName: "neutron-dhcp-combined-ca-bundle") pod "8233e500-8181-426b-aaf3-c7ecf3251d4c" (UID: "8233e500-8181-426b-aaf3-c7ecf3251d4c"). InnerVolumeSpecName "neutron-dhcp-combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:04:48 crc kubenswrapper[5052]: I1203 13:04:48.896857 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8233e500-8181-426b-aaf3-c7ecf3251d4c-kube-api-access-r4rx8" (OuterVolumeSpecName: "kube-api-access-r4rx8") pod "8233e500-8181-426b-aaf3-c7ecf3251d4c" (UID: "8233e500-8181-426b-aaf3-c7ecf3251d4c"). InnerVolumeSpecName "kube-api-access-r4rx8". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:04:48 crc kubenswrapper[5052]: I1203 13:04:48.906086 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8233e500-8181-426b-aaf3-c7ecf3251d4c-neutron-dhcp-agent-neutron-config-0" (OuterVolumeSpecName: "neutron-dhcp-agent-neutron-config-0") pod "8233e500-8181-426b-aaf3-c7ecf3251d4c" (UID: "8233e500-8181-426b-aaf3-c7ecf3251d4c"). InnerVolumeSpecName "neutron-dhcp-agent-neutron-config-0". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:04:48 crc kubenswrapper[5052]: I1203 13:04:48.910796 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8233e500-8181-426b-aaf3-c7ecf3251d4c-inventory" (OuterVolumeSpecName: "inventory") pod "8233e500-8181-426b-aaf3-c7ecf3251d4c" (UID: "8233e500-8181-426b-aaf3-c7ecf3251d4c"). InnerVolumeSpecName "inventory". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:04:48 crc kubenswrapper[5052]: I1203 13:04:48.911866 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8233e500-8181-426b-aaf3-c7ecf3251d4c-ssh-key" (OuterVolumeSpecName: "ssh-key") pod "8233e500-8181-426b-aaf3-c7ecf3251d4c" (UID: "8233e500-8181-426b-aaf3-c7ecf3251d4c"). InnerVolumeSpecName "ssh-key". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:04:48 crc kubenswrapper[5052]: I1203 13:04:48.980679 5052 reconciler_common.go:293] "Volume detached for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/8233e500-8181-426b-aaf3-c7ecf3251d4c-inventory\") on node \"crc\" DevicePath \"\"" Dec 03 13:04:48 crc kubenswrapper[5052]: I1203 13:04:48.980721 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-r4rx8\" (UniqueName: \"kubernetes.io/projected/8233e500-8181-426b-aaf3-c7ecf3251d4c-kube-api-access-r4rx8\") on node \"crc\" DevicePath \"\"" Dec 03 13:04:48 crc kubenswrapper[5052]: I1203 13:04:48.980740 5052 reconciler_common.go:293] "Volume detached for volume \"neutron-dhcp-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8233e500-8181-426b-aaf3-c7ecf3251d4c-neutron-dhcp-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 13:04:48 crc kubenswrapper[5052]: I1203 13:04:48.980755 5052 reconciler_common.go:293] "Volume detached for volume \"neutron-dhcp-agent-neutron-config-0\" (UniqueName: \"kubernetes.io/secret/8233e500-8181-426b-aaf3-c7ecf3251d4c-neutron-dhcp-agent-neutron-config-0\") on node \"crc\" DevicePath \"\"" Dec 03 13:04:48 crc kubenswrapper[5052]: I1203 13:04:48.980771 5052 reconciler_common.go:293] "Volume detached for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/8233e500-8181-426b-aaf3-c7ecf3251d4c-ssh-key\") on node \"crc\" DevicePath \"\"" Dec 03 13:04:49 crc kubenswrapper[5052]: I1203 13:04:49.427607 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/neutron-dhcp-edpm-multinodeset-edpm-compute-no-nodes-269x6" event={"ID":"8233e500-8181-426b-aaf3-c7ecf3251d4c","Type":"ContainerDied","Data":"d4daf9b1b2a772433ab6a57574c22c29531dd9f3a7f31f70da4d0acffc6c385b"} Dec 03 13:04:49 crc kubenswrapper[5052]: I1203 13:04:49.427650 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d4daf9b1b2a772433ab6a57574c22c29531dd9f3a7f31f70da4d0acffc6c385b" Dec 03 13:04:49 crc kubenswrapper[5052]: I1203 13:04:49.427653 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/neutron-dhcp-edpm-multinodeset-edpm-compute-no-nodes-269x6" Dec 03 13:04:49 crc kubenswrapper[5052]: I1203 13:04:49.513254 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/libvirt-edpm-multinodeset-edpm-compute-no-nodes-fv5kf"] Dec 03 13:04:49 crc kubenswrapper[5052]: E1203 13:04:49.513738 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="8233e500-8181-426b-aaf3-c7ecf3251d4c" containerName="neutron-dhcp-edpm-multinodeset-edpm-compute-no-nodes" Dec 03 13:04:49 crc kubenswrapper[5052]: I1203 13:04:49.513761 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="8233e500-8181-426b-aaf3-c7ecf3251d4c" containerName="neutron-dhcp-edpm-multinodeset-edpm-compute-no-nodes" Dec 03 13:04:49 crc kubenswrapper[5052]: I1203 13:04:49.514004 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="8233e500-8181-426b-aaf3-c7ecf3251d4c" containerName="neutron-dhcp-edpm-multinodeset-edpm-compute-no-nodes" Dec 03 13:04:49 crc kubenswrapper[5052]: I1203 13:04:49.514724 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/libvirt-edpm-multinodeset-edpm-compute-no-nodes-fv5kf" Dec 03 13:04:49 crc kubenswrapper[5052]: I1203 13:04:49.518438 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"combined-ca-bundle" Dec 03 13:04:49 crc kubenswrapper[5052]: I1203 13:04:49.518468 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"libvirt-secret" Dec 03 13:04:49 crc kubenswrapper[5052]: I1203 13:04:49.518626 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"edpm-compute-no-nodes-dockercfg-mkcgv" Dec 03 13:04:49 crc kubenswrapper[5052]: I1203 13:04:49.518490 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"dataplane-ansible-ssh-private-key-secret" Dec 03 13:04:49 crc kubenswrapper[5052]: I1203 13:04:49.519284 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"dataplanenodeset-edpm-compute-no-nodes" Dec 03 13:04:49 crc kubenswrapper[5052]: I1203 13:04:49.519966 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"openstack-aee-default-env" Dec 03 13:04:49 crc kubenswrapper[5052]: I1203 13:04:49.522070 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/libvirt-edpm-multinodeset-edpm-compute-no-nodes-fv5kf"] Dec 03 13:04:49 crc kubenswrapper[5052]: I1203 13:04:49.589582 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/970c0318-3c82-4671-b0b7-e4c573f06226-inventory\") pod \"libvirt-edpm-multinodeset-edpm-compute-no-nodes-fv5kf\" (UID: \"970c0318-3c82-4671-b0b7-e4c573f06226\") " pod="openstack-kuttl-tests/libvirt-edpm-multinodeset-edpm-compute-no-nodes-fv5kf" Dec 03 13:04:49 crc kubenswrapper[5052]: I1203 13:04:49.589699 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/970c0318-3c82-4671-b0b7-e4c573f06226-ssh-key\") pod \"libvirt-edpm-multinodeset-edpm-compute-no-nodes-fv5kf\" (UID: \"970c0318-3c82-4671-b0b7-e4c573f06226\") " pod="openstack-kuttl-tests/libvirt-edpm-multinodeset-edpm-compute-no-nodes-fv5kf" Dec 03 13:04:49 crc kubenswrapper[5052]: I1203 13:04:49.589785 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"libvirt-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/970c0318-3c82-4671-b0b7-e4c573f06226-libvirt-combined-ca-bundle\") pod \"libvirt-edpm-multinodeset-edpm-compute-no-nodes-fv5kf\" (UID: \"970c0318-3c82-4671-b0b7-e4c573f06226\") " pod="openstack-kuttl-tests/libvirt-edpm-multinodeset-edpm-compute-no-nodes-fv5kf" Dec 03 13:04:49 crc kubenswrapper[5052]: I1203 13:04:49.589825 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d5phm\" (UniqueName: \"kubernetes.io/projected/970c0318-3c82-4671-b0b7-e4c573f06226-kube-api-access-d5phm\") pod \"libvirt-edpm-multinodeset-edpm-compute-no-nodes-fv5kf\" (UID: \"970c0318-3c82-4671-b0b7-e4c573f06226\") " pod="openstack-kuttl-tests/libvirt-edpm-multinodeset-edpm-compute-no-nodes-fv5kf" Dec 03 13:04:49 crc kubenswrapper[5052]: I1203 13:04:49.589854 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"libvirt-secret-0\" (UniqueName: \"kubernetes.io/secret/970c0318-3c82-4671-b0b7-e4c573f06226-libvirt-secret-0\") pod \"libvirt-edpm-multinodeset-edpm-compute-no-nodes-fv5kf\" (UID: \"970c0318-3c82-4671-b0b7-e4c573f06226\") " pod="openstack-kuttl-tests/libvirt-edpm-multinodeset-edpm-compute-no-nodes-fv5kf" Dec 03 13:04:49 crc kubenswrapper[5052]: I1203 13:04:49.692330 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/970c0318-3c82-4671-b0b7-e4c573f06226-ssh-key\") pod \"libvirt-edpm-multinodeset-edpm-compute-no-nodes-fv5kf\" (UID: \"970c0318-3c82-4671-b0b7-e4c573f06226\") " pod="openstack-kuttl-tests/libvirt-edpm-multinodeset-edpm-compute-no-nodes-fv5kf" Dec 03 13:04:49 crc kubenswrapper[5052]: I1203 13:04:49.692489 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"libvirt-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/970c0318-3c82-4671-b0b7-e4c573f06226-libvirt-combined-ca-bundle\") pod \"libvirt-edpm-multinodeset-edpm-compute-no-nodes-fv5kf\" (UID: \"970c0318-3c82-4671-b0b7-e4c573f06226\") " pod="openstack-kuttl-tests/libvirt-edpm-multinodeset-edpm-compute-no-nodes-fv5kf" Dec 03 13:04:49 crc kubenswrapper[5052]: I1203 13:04:49.693009 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-d5phm\" (UniqueName: \"kubernetes.io/projected/970c0318-3c82-4671-b0b7-e4c573f06226-kube-api-access-d5phm\") pod \"libvirt-edpm-multinodeset-edpm-compute-no-nodes-fv5kf\" (UID: \"970c0318-3c82-4671-b0b7-e4c573f06226\") " pod="openstack-kuttl-tests/libvirt-edpm-multinodeset-edpm-compute-no-nodes-fv5kf" Dec 03 13:04:49 crc kubenswrapper[5052]: I1203 13:04:49.693058 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"libvirt-secret-0\" (UniqueName: \"kubernetes.io/secret/970c0318-3c82-4671-b0b7-e4c573f06226-libvirt-secret-0\") pod \"libvirt-edpm-multinodeset-edpm-compute-no-nodes-fv5kf\" (UID: \"970c0318-3c82-4671-b0b7-e4c573f06226\") " pod="openstack-kuttl-tests/libvirt-edpm-multinodeset-edpm-compute-no-nodes-fv5kf" Dec 03 13:04:49 crc kubenswrapper[5052]: I1203 13:04:49.693097 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/970c0318-3c82-4671-b0b7-e4c573f06226-inventory\") pod \"libvirt-edpm-multinodeset-edpm-compute-no-nodes-fv5kf\" (UID: \"970c0318-3c82-4671-b0b7-e4c573f06226\") " pod="openstack-kuttl-tests/libvirt-edpm-multinodeset-edpm-compute-no-nodes-fv5kf" Dec 03 13:04:49 crc kubenswrapper[5052]: I1203 13:04:49.698609 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/970c0318-3c82-4671-b0b7-e4c573f06226-inventory\") pod \"libvirt-edpm-multinodeset-edpm-compute-no-nodes-fv5kf\" (UID: \"970c0318-3c82-4671-b0b7-e4c573f06226\") " pod="openstack-kuttl-tests/libvirt-edpm-multinodeset-edpm-compute-no-nodes-fv5kf" Dec 03 13:04:49 crc kubenswrapper[5052]: I1203 13:04:49.700599 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"libvirt-secret-0\" (UniqueName: \"kubernetes.io/secret/970c0318-3c82-4671-b0b7-e4c573f06226-libvirt-secret-0\") pod \"libvirt-edpm-multinodeset-edpm-compute-no-nodes-fv5kf\" (UID: \"970c0318-3c82-4671-b0b7-e4c573f06226\") " pod="openstack-kuttl-tests/libvirt-edpm-multinodeset-edpm-compute-no-nodes-fv5kf" Dec 03 13:04:49 crc kubenswrapper[5052]: I1203 13:04:49.702034 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/970c0318-3c82-4671-b0b7-e4c573f06226-ssh-key\") pod \"libvirt-edpm-multinodeset-edpm-compute-no-nodes-fv5kf\" (UID: \"970c0318-3c82-4671-b0b7-e4c573f06226\") " pod="openstack-kuttl-tests/libvirt-edpm-multinodeset-edpm-compute-no-nodes-fv5kf" Dec 03 13:04:49 crc kubenswrapper[5052]: I1203 13:04:49.702227 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"libvirt-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/970c0318-3c82-4671-b0b7-e4c573f06226-libvirt-combined-ca-bundle\") pod \"libvirt-edpm-multinodeset-edpm-compute-no-nodes-fv5kf\" (UID: \"970c0318-3c82-4671-b0b7-e4c573f06226\") " pod="openstack-kuttl-tests/libvirt-edpm-multinodeset-edpm-compute-no-nodes-fv5kf" Dec 03 13:04:49 crc kubenswrapper[5052]: I1203 13:04:49.721772 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-d5phm\" (UniqueName: \"kubernetes.io/projected/970c0318-3c82-4671-b0b7-e4c573f06226-kube-api-access-d5phm\") pod \"libvirt-edpm-multinodeset-edpm-compute-no-nodes-fv5kf\" (UID: \"970c0318-3c82-4671-b0b7-e4c573f06226\") " pod="openstack-kuttl-tests/libvirt-edpm-multinodeset-edpm-compute-no-nodes-fv5kf" Dec 03 13:04:49 crc kubenswrapper[5052]: I1203 13:04:49.841398 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/libvirt-edpm-multinodeset-edpm-compute-no-nodes-fv5kf" Dec 03 13:04:50 crc kubenswrapper[5052]: I1203 13:04:50.411457 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/libvirt-edpm-multinodeset-edpm-compute-no-nodes-fv5kf"] Dec 03 13:04:50 crc kubenswrapper[5052]: I1203 13:04:50.435443 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/libvirt-edpm-multinodeset-edpm-compute-no-nodes-fv5kf" event={"ID":"970c0318-3c82-4671-b0b7-e4c573f06226","Type":"ContainerStarted","Data":"0a7ef9417c47684203e1a5c2003c33c26d08de1a4d8799885a3232e7b8d03f39"} Dec 03 13:04:51 crc kubenswrapper[5052]: I1203 13:04:51.445699 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/libvirt-edpm-multinodeset-edpm-compute-no-nodes-fv5kf" event={"ID":"970c0318-3c82-4671-b0b7-e4c573f06226","Type":"ContainerStarted","Data":"5fbbcf4e8261ee635f3df3bdf47a7c2e7675875364098a4863ec002d31248f1f"} Dec 03 13:04:51 crc kubenswrapper[5052]: I1203 13:04:51.467567 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/libvirt-edpm-multinodeset-edpm-compute-no-nodes-fv5kf" podStartSLOduration=1.841549647 podStartE2EDuration="2.467548859s" podCreationTimestamp="2025-12-03 13:04:49 +0000 UTC" firstStartedPulling="2025-12-03 13:04:50.419288231 +0000 UTC m=+6281.007529995" lastFinishedPulling="2025-12-03 13:04:51.045287443 +0000 UTC m=+6281.633529207" observedRunningTime="2025-12-03 13:04:51.461494046 +0000 UTC m=+6282.049735810" watchObservedRunningTime="2025-12-03 13:04:51.467548859 +0000 UTC m=+6282.055790623" Dec 03 13:04:53 crc kubenswrapper[5052]: I1203 13:04:53.463317 5052 generic.go:334] "Generic (PLEG): container finished" podID="970c0318-3c82-4671-b0b7-e4c573f06226" containerID="5fbbcf4e8261ee635f3df3bdf47a7c2e7675875364098a4863ec002d31248f1f" exitCode=0 Dec 03 13:04:53 crc kubenswrapper[5052]: I1203 13:04:53.463407 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/libvirt-edpm-multinodeset-edpm-compute-no-nodes-fv5kf" event={"ID":"970c0318-3c82-4671-b0b7-e4c573f06226","Type":"ContainerDied","Data":"5fbbcf4e8261ee635f3df3bdf47a7c2e7675875364098a4863ec002d31248f1f"} Dec 03 13:04:54 crc kubenswrapper[5052]: I1203 13:04:54.817342 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/libvirt-edpm-multinodeset-edpm-compute-no-nodes-fv5kf" Dec 03 13:04:54 crc kubenswrapper[5052]: I1203 13:04:54.977165 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d5phm\" (UniqueName: \"kubernetes.io/projected/970c0318-3c82-4671-b0b7-e4c573f06226-kube-api-access-d5phm\") pod \"970c0318-3c82-4671-b0b7-e4c573f06226\" (UID: \"970c0318-3c82-4671-b0b7-e4c573f06226\") " Dec 03 13:04:54 crc kubenswrapper[5052]: I1203 13:04:54.977273 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"libvirt-secret-0\" (UniqueName: \"kubernetes.io/secret/970c0318-3c82-4671-b0b7-e4c573f06226-libvirt-secret-0\") pod \"970c0318-3c82-4671-b0b7-e4c573f06226\" (UID: \"970c0318-3c82-4671-b0b7-e4c573f06226\") " Dec 03 13:04:54 crc kubenswrapper[5052]: I1203 13:04:54.977300 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"libvirt-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/970c0318-3c82-4671-b0b7-e4c573f06226-libvirt-combined-ca-bundle\") pod \"970c0318-3c82-4671-b0b7-e4c573f06226\" (UID: \"970c0318-3c82-4671-b0b7-e4c573f06226\") " Dec 03 13:04:54 crc kubenswrapper[5052]: I1203 13:04:54.977382 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/970c0318-3c82-4671-b0b7-e4c573f06226-ssh-key\") pod \"970c0318-3c82-4671-b0b7-e4c573f06226\" (UID: \"970c0318-3c82-4671-b0b7-e4c573f06226\") " Dec 03 13:04:54 crc kubenswrapper[5052]: I1203 13:04:54.977431 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/970c0318-3c82-4671-b0b7-e4c573f06226-inventory\") pod \"970c0318-3c82-4671-b0b7-e4c573f06226\" (UID: \"970c0318-3c82-4671-b0b7-e4c573f06226\") " Dec 03 13:04:54 crc kubenswrapper[5052]: I1203 13:04:54.990993 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/970c0318-3c82-4671-b0b7-e4c573f06226-libvirt-combined-ca-bundle" (OuterVolumeSpecName: "libvirt-combined-ca-bundle") pod "970c0318-3c82-4671-b0b7-e4c573f06226" (UID: "970c0318-3c82-4671-b0b7-e4c573f06226"). InnerVolumeSpecName "libvirt-combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:04:54 crc kubenswrapper[5052]: I1203 13:04:54.991062 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/970c0318-3c82-4671-b0b7-e4c573f06226-kube-api-access-d5phm" (OuterVolumeSpecName: "kube-api-access-d5phm") pod "970c0318-3c82-4671-b0b7-e4c573f06226" (UID: "970c0318-3c82-4671-b0b7-e4c573f06226"). InnerVolumeSpecName "kube-api-access-d5phm". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:04:55 crc kubenswrapper[5052]: I1203 13:04:55.003175 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/970c0318-3c82-4671-b0b7-e4c573f06226-libvirt-secret-0" (OuterVolumeSpecName: "libvirt-secret-0") pod "970c0318-3c82-4671-b0b7-e4c573f06226" (UID: "970c0318-3c82-4671-b0b7-e4c573f06226"). InnerVolumeSpecName "libvirt-secret-0". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:04:55 crc kubenswrapper[5052]: I1203 13:04:55.003574 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/970c0318-3c82-4671-b0b7-e4c573f06226-inventory" (OuterVolumeSpecName: "inventory") pod "970c0318-3c82-4671-b0b7-e4c573f06226" (UID: "970c0318-3c82-4671-b0b7-e4c573f06226"). InnerVolumeSpecName "inventory". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:04:55 crc kubenswrapper[5052]: I1203 13:04:55.014320 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/970c0318-3c82-4671-b0b7-e4c573f06226-ssh-key" (OuterVolumeSpecName: "ssh-key") pod "970c0318-3c82-4671-b0b7-e4c573f06226" (UID: "970c0318-3c82-4671-b0b7-e4c573f06226"). InnerVolumeSpecName "ssh-key". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:04:55 crc kubenswrapper[5052]: I1203 13:04:55.079272 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-d5phm\" (UniqueName: \"kubernetes.io/projected/970c0318-3c82-4671-b0b7-e4c573f06226-kube-api-access-d5phm\") on node \"crc\" DevicePath \"\"" Dec 03 13:04:55 crc kubenswrapper[5052]: I1203 13:04:55.079312 5052 reconciler_common.go:293] "Volume detached for volume \"libvirt-secret-0\" (UniqueName: \"kubernetes.io/secret/970c0318-3c82-4671-b0b7-e4c573f06226-libvirt-secret-0\") on node \"crc\" DevicePath \"\"" Dec 03 13:04:55 crc kubenswrapper[5052]: I1203 13:04:55.079322 5052 reconciler_common.go:293] "Volume detached for volume \"libvirt-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/970c0318-3c82-4671-b0b7-e4c573f06226-libvirt-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 13:04:55 crc kubenswrapper[5052]: I1203 13:04:55.079332 5052 reconciler_common.go:293] "Volume detached for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/970c0318-3c82-4671-b0b7-e4c573f06226-ssh-key\") on node \"crc\" DevicePath \"\"" Dec 03 13:04:55 crc kubenswrapper[5052]: I1203 13:04:55.079343 5052 reconciler_common.go:293] "Volume detached for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/970c0318-3c82-4671-b0b7-e4c573f06226-inventory\") on node \"crc\" DevicePath \"\"" Dec 03 13:04:55 crc kubenswrapper[5052]: I1203 13:04:55.502724 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/libvirt-edpm-multinodeset-edpm-compute-no-nodes-fv5kf" event={"ID":"970c0318-3c82-4671-b0b7-e4c573f06226","Type":"ContainerDied","Data":"0a7ef9417c47684203e1a5c2003c33c26d08de1a4d8799885a3232e7b8d03f39"} Dec 03 13:04:55 crc kubenswrapper[5052]: I1203 13:04:55.502765 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0a7ef9417c47684203e1a5c2003c33c26d08de1a4d8799885a3232e7b8d03f39" Dec 03 13:04:55 crc kubenswrapper[5052]: I1203 13:04:55.502802 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/libvirt-edpm-multinodeset-edpm-compute-no-nodes-fv5kf" Dec 03 13:04:55 crc kubenswrapper[5052]: I1203 13:04:55.562715 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/nova-edpm-multinodeset-edpm-compute-no-nodes-8h69r"] Dec 03 13:04:55 crc kubenswrapper[5052]: E1203 13:04:55.563049 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="970c0318-3c82-4671-b0b7-e4c573f06226" containerName="libvirt-edpm-multinodeset-edpm-compute-no-nodes" Dec 03 13:04:55 crc kubenswrapper[5052]: I1203 13:04:55.563064 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="970c0318-3c82-4671-b0b7-e4c573f06226" containerName="libvirt-edpm-multinodeset-edpm-compute-no-nodes" Dec 03 13:04:55 crc kubenswrapper[5052]: I1203 13:04:55.563217 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="970c0318-3c82-4671-b0b7-e4c573f06226" containerName="libvirt-edpm-multinodeset-edpm-compute-no-nodes" Dec 03 13:04:55 crc kubenswrapper[5052]: I1203 13:04:55.563724 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-edpm-multinodeset-edpm-compute-no-nodes-8h69r" Dec 03 13:04:55 crc kubenswrapper[5052]: I1203 13:04:55.566364 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-cell1-compute-config" Dec 03 13:04:55 crc kubenswrapper[5052]: I1203 13:04:55.566395 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"openstack-aee-default-env" Dec 03 13:04:55 crc kubenswrapper[5052]: I1203 13:04:55.567014 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"dataplane-ansible-ssh-private-key-secret" Dec 03 13:04:55 crc kubenswrapper[5052]: I1203 13:04:55.567182 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"dataplanenodeset-edpm-compute-no-nodes" Dec 03 13:04:55 crc kubenswrapper[5052]: I1203 13:04:55.567235 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"nova-migration-ssh-key" Dec 03 13:04:55 crc kubenswrapper[5052]: I1203 13:04:55.567399 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"combined-ca-bundle" Dec 03 13:04:55 crc kubenswrapper[5052]: I1203 13:04:55.569329 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"edpm-compute-no-nodes-dockercfg-mkcgv" Dec 03 13:04:55 crc kubenswrapper[5052]: I1203 13:04:55.573401 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-edpm-multinodeset-edpm-compute-no-nodes-8h69r"] Dec 03 13:04:55 crc kubenswrapper[5052]: I1203 13:04:55.686491 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nova-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/fa0377dd-cc5c-4930-ba00-54f966059f5d-nova-combined-ca-bundle\") pod \"nova-edpm-multinodeset-edpm-compute-no-nodes-8h69r\" (UID: \"fa0377dd-cc5c-4930-ba00-54f966059f5d\") " pod="openstack-kuttl-tests/nova-edpm-multinodeset-edpm-compute-no-nodes-8h69r" Dec 03 13:04:55 crc kubenswrapper[5052]: I1203 13:04:55.686577 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2qx2p\" (UniqueName: \"kubernetes.io/projected/fa0377dd-cc5c-4930-ba00-54f966059f5d-kube-api-access-2qx2p\") pod \"nova-edpm-multinodeset-edpm-compute-no-nodes-8h69r\" (UID: \"fa0377dd-cc5c-4930-ba00-54f966059f5d\") " pod="openstack-kuttl-tests/nova-edpm-multinodeset-edpm-compute-no-nodes-8h69r" Dec 03 13:04:55 crc kubenswrapper[5052]: I1203 13:04:55.686621 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nova-cell1-compute-config-1\" (UniqueName: \"kubernetes.io/secret/fa0377dd-cc5c-4930-ba00-54f966059f5d-nova-cell1-compute-config-1\") pod \"nova-edpm-multinodeset-edpm-compute-no-nodes-8h69r\" (UID: \"fa0377dd-cc5c-4930-ba00-54f966059f5d\") " pod="openstack-kuttl-tests/nova-edpm-multinodeset-edpm-compute-no-nodes-8h69r" Dec 03 13:04:55 crc kubenswrapper[5052]: I1203 13:04:55.686653 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/fa0377dd-cc5c-4930-ba00-54f966059f5d-inventory\") pod \"nova-edpm-multinodeset-edpm-compute-no-nodes-8h69r\" (UID: \"fa0377dd-cc5c-4930-ba00-54f966059f5d\") " pod="openstack-kuttl-tests/nova-edpm-multinodeset-edpm-compute-no-nodes-8h69r" Dec 03 13:04:55 crc kubenswrapper[5052]: I1203 13:04:55.686706 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nova-migration-ssh-key-0\" (UniqueName: \"kubernetes.io/secret/fa0377dd-cc5c-4930-ba00-54f966059f5d-nova-migration-ssh-key-0\") pod \"nova-edpm-multinodeset-edpm-compute-no-nodes-8h69r\" (UID: \"fa0377dd-cc5c-4930-ba00-54f966059f5d\") " pod="openstack-kuttl-tests/nova-edpm-multinodeset-edpm-compute-no-nodes-8h69r" Dec 03 13:04:55 crc kubenswrapper[5052]: I1203 13:04:55.686760 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nova-cell1-compute-config-0\" (UniqueName: \"kubernetes.io/secret/fa0377dd-cc5c-4930-ba00-54f966059f5d-nova-cell1-compute-config-0\") pod \"nova-edpm-multinodeset-edpm-compute-no-nodes-8h69r\" (UID: \"fa0377dd-cc5c-4930-ba00-54f966059f5d\") " pod="openstack-kuttl-tests/nova-edpm-multinodeset-edpm-compute-no-nodes-8h69r" Dec 03 13:04:55 crc kubenswrapper[5052]: I1203 13:04:55.686800 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nova-migration-ssh-key-1\" (UniqueName: \"kubernetes.io/secret/fa0377dd-cc5c-4930-ba00-54f966059f5d-nova-migration-ssh-key-1\") pod \"nova-edpm-multinodeset-edpm-compute-no-nodes-8h69r\" (UID: \"fa0377dd-cc5c-4930-ba00-54f966059f5d\") " pod="openstack-kuttl-tests/nova-edpm-multinodeset-edpm-compute-no-nodes-8h69r" Dec 03 13:04:55 crc kubenswrapper[5052]: I1203 13:04:55.686837 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/fa0377dd-cc5c-4930-ba00-54f966059f5d-ssh-key\") pod \"nova-edpm-multinodeset-edpm-compute-no-nodes-8h69r\" (UID: \"fa0377dd-cc5c-4930-ba00-54f966059f5d\") " pod="openstack-kuttl-tests/nova-edpm-multinodeset-edpm-compute-no-nodes-8h69r" Dec 03 13:04:55 crc kubenswrapper[5052]: I1203 13:04:55.789288 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"nova-cell1-compute-config-1\" (UniqueName: \"kubernetes.io/secret/fa0377dd-cc5c-4930-ba00-54f966059f5d-nova-cell1-compute-config-1\") pod \"nova-edpm-multinodeset-edpm-compute-no-nodes-8h69r\" (UID: \"fa0377dd-cc5c-4930-ba00-54f966059f5d\") " pod="openstack-kuttl-tests/nova-edpm-multinodeset-edpm-compute-no-nodes-8h69r" Dec 03 13:04:55 crc kubenswrapper[5052]: I1203 13:04:55.789383 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/fa0377dd-cc5c-4930-ba00-54f966059f5d-inventory\") pod \"nova-edpm-multinodeset-edpm-compute-no-nodes-8h69r\" (UID: \"fa0377dd-cc5c-4930-ba00-54f966059f5d\") " pod="openstack-kuttl-tests/nova-edpm-multinodeset-edpm-compute-no-nodes-8h69r" Dec 03 13:04:55 crc kubenswrapper[5052]: I1203 13:04:55.789441 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"nova-migration-ssh-key-0\" (UniqueName: \"kubernetes.io/secret/fa0377dd-cc5c-4930-ba00-54f966059f5d-nova-migration-ssh-key-0\") pod \"nova-edpm-multinodeset-edpm-compute-no-nodes-8h69r\" (UID: \"fa0377dd-cc5c-4930-ba00-54f966059f5d\") " pod="openstack-kuttl-tests/nova-edpm-multinodeset-edpm-compute-no-nodes-8h69r" Dec 03 13:04:55 crc kubenswrapper[5052]: I1203 13:04:55.789536 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"nova-cell1-compute-config-0\" (UniqueName: \"kubernetes.io/secret/fa0377dd-cc5c-4930-ba00-54f966059f5d-nova-cell1-compute-config-0\") pod \"nova-edpm-multinodeset-edpm-compute-no-nodes-8h69r\" (UID: \"fa0377dd-cc5c-4930-ba00-54f966059f5d\") " pod="openstack-kuttl-tests/nova-edpm-multinodeset-edpm-compute-no-nodes-8h69r" Dec 03 13:04:55 crc kubenswrapper[5052]: I1203 13:04:55.789608 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"nova-migration-ssh-key-1\" (UniqueName: \"kubernetes.io/secret/fa0377dd-cc5c-4930-ba00-54f966059f5d-nova-migration-ssh-key-1\") pod \"nova-edpm-multinodeset-edpm-compute-no-nodes-8h69r\" (UID: \"fa0377dd-cc5c-4930-ba00-54f966059f5d\") " pod="openstack-kuttl-tests/nova-edpm-multinodeset-edpm-compute-no-nodes-8h69r" Dec 03 13:04:55 crc kubenswrapper[5052]: I1203 13:04:55.789675 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/fa0377dd-cc5c-4930-ba00-54f966059f5d-ssh-key\") pod \"nova-edpm-multinodeset-edpm-compute-no-nodes-8h69r\" (UID: \"fa0377dd-cc5c-4930-ba00-54f966059f5d\") " pod="openstack-kuttl-tests/nova-edpm-multinodeset-edpm-compute-no-nodes-8h69r" Dec 03 13:04:55 crc kubenswrapper[5052]: I1203 13:04:55.789731 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"nova-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/fa0377dd-cc5c-4930-ba00-54f966059f5d-nova-combined-ca-bundle\") pod \"nova-edpm-multinodeset-edpm-compute-no-nodes-8h69r\" (UID: \"fa0377dd-cc5c-4930-ba00-54f966059f5d\") " pod="openstack-kuttl-tests/nova-edpm-multinodeset-edpm-compute-no-nodes-8h69r" Dec 03 13:04:55 crc kubenswrapper[5052]: I1203 13:04:55.789775 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-2qx2p\" (UniqueName: \"kubernetes.io/projected/fa0377dd-cc5c-4930-ba00-54f966059f5d-kube-api-access-2qx2p\") pod \"nova-edpm-multinodeset-edpm-compute-no-nodes-8h69r\" (UID: \"fa0377dd-cc5c-4930-ba00-54f966059f5d\") " pod="openstack-kuttl-tests/nova-edpm-multinodeset-edpm-compute-no-nodes-8h69r" Dec 03 13:04:55 crc kubenswrapper[5052]: I1203 13:04:55.793027 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/fa0377dd-cc5c-4930-ba00-54f966059f5d-ssh-key\") pod \"nova-edpm-multinodeset-edpm-compute-no-nodes-8h69r\" (UID: \"fa0377dd-cc5c-4930-ba00-54f966059f5d\") " pod="openstack-kuttl-tests/nova-edpm-multinodeset-edpm-compute-no-nodes-8h69r" Dec 03 13:04:55 crc kubenswrapper[5052]: I1203 13:04:55.793106 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"nova-migration-ssh-key-0\" (UniqueName: \"kubernetes.io/secret/fa0377dd-cc5c-4930-ba00-54f966059f5d-nova-migration-ssh-key-0\") pod \"nova-edpm-multinodeset-edpm-compute-no-nodes-8h69r\" (UID: \"fa0377dd-cc5c-4930-ba00-54f966059f5d\") " pod="openstack-kuttl-tests/nova-edpm-multinodeset-edpm-compute-no-nodes-8h69r" Dec 03 13:04:55 crc kubenswrapper[5052]: I1203 13:04:55.793850 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"nova-migration-ssh-key-1\" (UniqueName: \"kubernetes.io/secret/fa0377dd-cc5c-4930-ba00-54f966059f5d-nova-migration-ssh-key-1\") pod \"nova-edpm-multinodeset-edpm-compute-no-nodes-8h69r\" (UID: \"fa0377dd-cc5c-4930-ba00-54f966059f5d\") " pod="openstack-kuttl-tests/nova-edpm-multinodeset-edpm-compute-no-nodes-8h69r" Dec 03 13:04:55 crc kubenswrapper[5052]: I1203 13:04:55.794143 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"nova-cell1-compute-config-0\" (UniqueName: \"kubernetes.io/secret/fa0377dd-cc5c-4930-ba00-54f966059f5d-nova-cell1-compute-config-0\") pod \"nova-edpm-multinodeset-edpm-compute-no-nodes-8h69r\" (UID: \"fa0377dd-cc5c-4930-ba00-54f966059f5d\") " pod="openstack-kuttl-tests/nova-edpm-multinodeset-edpm-compute-no-nodes-8h69r" Dec 03 13:04:55 crc kubenswrapper[5052]: I1203 13:04:55.794997 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"nova-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/fa0377dd-cc5c-4930-ba00-54f966059f5d-nova-combined-ca-bundle\") pod \"nova-edpm-multinodeset-edpm-compute-no-nodes-8h69r\" (UID: \"fa0377dd-cc5c-4930-ba00-54f966059f5d\") " pod="openstack-kuttl-tests/nova-edpm-multinodeset-edpm-compute-no-nodes-8h69r" Dec 03 13:04:55 crc kubenswrapper[5052]: I1203 13:04:55.795049 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/fa0377dd-cc5c-4930-ba00-54f966059f5d-inventory\") pod \"nova-edpm-multinodeset-edpm-compute-no-nodes-8h69r\" (UID: \"fa0377dd-cc5c-4930-ba00-54f966059f5d\") " pod="openstack-kuttl-tests/nova-edpm-multinodeset-edpm-compute-no-nodes-8h69r" Dec 03 13:04:55 crc kubenswrapper[5052]: I1203 13:04:55.795928 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"nova-cell1-compute-config-1\" (UniqueName: \"kubernetes.io/secret/fa0377dd-cc5c-4930-ba00-54f966059f5d-nova-cell1-compute-config-1\") pod \"nova-edpm-multinodeset-edpm-compute-no-nodes-8h69r\" (UID: \"fa0377dd-cc5c-4930-ba00-54f966059f5d\") " pod="openstack-kuttl-tests/nova-edpm-multinodeset-edpm-compute-no-nodes-8h69r" Dec 03 13:04:55 crc kubenswrapper[5052]: I1203 13:04:55.813552 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-2qx2p\" (UniqueName: \"kubernetes.io/projected/fa0377dd-cc5c-4930-ba00-54f966059f5d-kube-api-access-2qx2p\") pod \"nova-edpm-multinodeset-edpm-compute-no-nodes-8h69r\" (UID: \"fa0377dd-cc5c-4930-ba00-54f966059f5d\") " pod="openstack-kuttl-tests/nova-edpm-multinodeset-edpm-compute-no-nodes-8h69r" Dec 03 13:04:55 crc kubenswrapper[5052]: I1203 13:04:55.879358 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-edpm-multinodeset-edpm-compute-no-nodes-8h69r" Dec 03 13:04:56 crc kubenswrapper[5052]: I1203 13:04:56.285534 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/nova-edpm-multinodeset-edpm-compute-no-nodes-8h69r"] Dec 03 13:04:56 crc kubenswrapper[5052]: I1203 13:04:56.513220 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-edpm-multinodeset-edpm-compute-no-nodes-8h69r" event={"ID":"fa0377dd-cc5c-4930-ba00-54f966059f5d","Type":"ContainerStarted","Data":"a49cde1761231ec96faafa3913ef119586d63543e64d2b78988747d2309761a6"} Dec 03 13:04:56 crc kubenswrapper[5052]: I1203 13:04:56.929938 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openshift-marketplace/community-operators-5v5wb"] Dec 03 13:04:56 crc kubenswrapper[5052]: I1203 13:04:56.932399 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/community-operators-5v5wb" Dec 03 13:04:56 crc kubenswrapper[5052]: I1203 13:04:56.941983 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/community-operators-5v5wb"] Dec 03 13:04:57 crc kubenswrapper[5052]: I1203 13:04:57.016188 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/271ab8d3-4460-4b19-82c1-e75a4fc488aa-utilities\") pod \"community-operators-5v5wb\" (UID: \"271ab8d3-4460-4b19-82c1-e75a4fc488aa\") " pod="openshift-marketplace/community-operators-5v5wb" Dec 03 13:04:57 crc kubenswrapper[5052]: I1203 13:04:57.016529 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/271ab8d3-4460-4b19-82c1-e75a4fc488aa-catalog-content\") pod \"community-operators-5v5wb\" (UID: \"271ab8d3-4460-4b19-82c1-e75a4fc488aa\") " pod="openshift-marketplace/community-operators-5v5wb" Dec 03 13:04:57 crc kubenswrapper[5052]: I1203 13:04:57.016679 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7x59q\" (UniqueName: \"kubernetes.io/projected/271ab8d3-4460-4b19-82c1-e75a4fc488aa-kube-api-access-7x59q\") pod \"community-operators-5v5wb\" (UID: \"271ab8d3-4460-4b19-82c1-e75a4fc488aa\") " pod="openshift-marketplace/community-operators-5v5wb" Dec 03 13:04:57 crc kubenswrapper[5052]: I1203 13:04:57.118318 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-7x59q\" (UniqueName: \"kubernetes.io/projected/271ab8d3-4460-4b19-82c1-e75a4fc488aa-kube-api-access-7x59q\") pod \"community-operators-5v5wb\" (UID: \"271ab8d3-4460-4b19-82c1-e75a4fc488aa\") " pod="openshift-marketplace/community-operators-5v5wb" Dec 03 13:04:57 crc kubenswrapper[5052]: I1203 13:04:57.118416 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/271ab8d3-4460-4b19-82c1-e75a4fc488aa-utilities\") pod \"community-operators-5v5wb\" (UID: \"271ab8d3-4460-4b19-82c1-e75a4fc488aa\") " pod="openshift-marketplace/community-operators-5v5wb" Dec 03 13:04:57 crc kubenswrapper[5052]: I1203 13:04:57.118446 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/271ab8d3-4460-4b19-82c1-e75a4fc488aa-catalog-content\") pod \"community-operators-5v5wb\" (UID: \"271ab8d3-4460-4b19-82c1-e75a4fc488aa\") " pod="openshift-marketplace/community-operators-5v5wb" Dec 03 13:04:57 crc kubenswrapper[5052]: I1203 13:04:57.119075 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/271ab8d3-4460-4b19-82c1-e75a4fc488aa-catalog-content\") pod \"community-operators-5v5wb\" (UID: \"271ab8d3-4460-4b19-82c1-e75a4fc488aa\") " pod="openshift-marketplace/community-operators-5v5wb" Dec 03 13:04:57 crc kubenswrapper[5052]: I1203 13:04:57.119216 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/271ab8d3-4460-4b19-82c1-e75a4fc488aa-utilities\") pod \"community-operators-5v5wb\" (UID: \"271ab8d3-4460-4b19-82c1-e75a4fc488aa\") " pod="openshift-marketplace/community-operators-5v5wb" Dec 03 13:04:57 crc kubenswrapper[5052]: I1203 13:04:57.140324 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-7x59q\" (UniqueName: \"kubernetes.io/projected/271ab8d3-4460-4b19-82c1-e75a4fc488aa-kube-api-access-7x59q\") pod \"community-operators-5v5wb\" (UID: \"271ab8d3-4460-4b19-82c1-e75a4fc488aa\") " pod="openshift-marketplace/community-operators-5v5wb" Dec 03 13:04:57 crc kubenswrapper[5052]: I1203 13:04:57.288384 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/community-operators-5v5wb" Dec 03 13:04:57 crc kubenswrapper[5052]: I1203 13:04:57.535700 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-edpm-multinodeset-edpm-compute-no-nodes-8h69r" event={"ID":"fa0377dd-cc5c-4930-ba00-54f966059f5d","Type":"ContainerStarted","Data":"91930adc08bdb71680d2e02f8ac8466bfa248f6135ebeeb75dbf410e50719566"} Dec 03 13:04:57 crc kubenswrapper[5052]: I1203 13:04:57.566579 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/nova-edpm-multinodeset-edpm-compute-no-nodes-8h69r" podStartSLOduration=2.095283092 podStartE2EDuration="2.56655999s" podCreationTimestamp="2025-12-03 13:04:55 +0000 UTC" firstStartedPulling="2025-12-03 13:04:56.278617867 +0000 UTC m=+6286.866859641" lastFinishedPulling="2025-12-03 13:04:56.749894765 +0000 UTC m=+6287.338136539" observedRunningTime="2025-12-03 13:04:57.55780951 +0000 UTC m=+6288.146051274" watchObservedRunningTime="2025-12-03 13:04:57.56655999 +0000 UTC m=+6288.154801754" Dec 03 13:04:57 crc kubenswrapper[5052]: I1203 13:04:57.804584 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/community-operators-5v5wb"] Dec 03 13:04:57 crc kubenswrapper[5052]: W1203 13:04:57.811567 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod271ab8d3_4460_4b19_82c1_e75a4fc488aa.slice/crio-7d26e30479b239428368e5b459de9af8f3f30ad038485f5d307ccbed3b4fb9ff WatchSource:0}: Error finding container 7d26e30479b239428368e5b459de9af8f3f30ad038485f5d307ccbed3b4fb9ff: Status 404 returned error can't find the container with id 7d26e30479b239428368e5b459de9af8f3f30ad038485f5d307ccbed3b4fb9ff Dec 03 13:04:58 crc kubenswrapper[5052]: I1203 13:04:58.551567 5052 generic.go:334] "Generic (PLEG): container finished" podID="fa0377dd-cc5c-4930-ba00-54f966059f5d" containerID="91930adc08bdb71680d2e02f8ac8466bfa248f6135ebeeb75dbf410e50719566" exitCode=0 Dec 03 13:04:58 crc kubenswrapper[5052]: I1203 13:04:58.551652 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-edpm-multinodeset-edpm-compute-no-nodes-8h69r" event={"ID":"fa0377dd-cc5c-4930-ba00-54f966059f5d","Type":"ContainerDied","Data":"91930adc08bdb71680d2e02f8ac8466bfa248f6135ebeeb75dbf410e50719566"} Dec 03 13:04:58 crc kubenswrapper[5052]: I1203 13:04:58.556589 5052 generic.go:334] "Generic (PLEG): container finished" podID="271ab8d3-4460-4b19-82c1-e75a4fc488aa" containerID="b3aeab68a5913163936ab01d8f628039248b9f5339d79e193fe8dbc90c820672" exitCode=0 Dec 03 13:04:58 crc kubenswrapper[5052]: I1203 13:04:58.556696 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/community-operators-5v5wb" event={"ID":"271ab8d3-4460-4b19-82c1-e75a4fc488aa","Type":"ContainerDied","Data":"b3aeab68a5913163936ab01d8f628039248b9f5339d79e193fe8dbc90c820672"} Dec 03 13:04:58 crc kubenswrapper[5052]: I1203 13:04:58.556765 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/community-operators-5v5wb" event={"ID":"271ab8d3-4460-4b19-82c1-e75a4fc488aa","Type":"ContainerStarted","Data":"7d26e30479b239428368e5b459de9af8f3f30ad038485f5d307ccbed3b4fb9ff"} Dec 03 13:04:59 crc kubenswrapper[5052]: I1203 13:04:59.569234 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/community-operators-5v5wb" event={"ID":"271ab8d3-4460-4b19-82c1-e75a4fc488aa","Type":"ContainerStarted","Data":"9695bf67bf79d16d036f595c37796668bbc491961ac5c497f389fe969c830b2b"} Dec 03 13:04:59 crc kubenswrapper[5052]: I1203 13:04:59.902462 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-edpm-multinodeset-edpm-compute-no-nodes-8h69r" Dec 03 13:04:59 crc kubenswrapper[5052]: I1203 13:04:59.960990 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"nova-cell1-compute-config-1\" (UniqueName: \"kubernetes.io/secret/fa0377dd-cc5c-4930-ba00-54f966059f5d-nova-cell1-compute-config-1\") pod \"fa0377dd-cc5c-4930-ba00-54f966059f5d\" (UID: \"fa0377dd-cc5c-4930-ba00-54f966059f5d\") " Dec 03 13:04:59 crc kubenswrapper[5052]: I1203 13:04:59.961044 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2qx2p\" (UniqueName: \"kubernetes.io/projected/fa0377dd-cc5c-4930-ba00-54f966059f5d-kube-api-access-2qx2p\") pod \"fa0377dd-cc5c-4930-ba00-54f966059f5d\" (UID: \"fa0377dd-cc5c-4930-ba00-54f966059f5d\") " Dec 03 13:04:59 crc kubenswrapper[5052]: I1203 13:04:59.961079 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/fa0377dd-cc5c-4930-ba00-54f966059f5d-ssh-key\") pod \"fa0377dd-cc5c-4930-ba00-54f966059f5d\" (UID: \"fa0377dd-cc5c-4930-ba00-54f966059f5d\") " Dec 03 13:04:59 crc kubenswrapper[5052]: I1203 13:04:59.961208 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"nova-migration-ssh-key-0\" (UniqueName: \"kubernetes.io/secret/fa0377dd-cc5c-4930-ba00-54f966059f5d-nova-migration-ssh-key-0\") pod \"fa0377dd-cc5c-4930-ba00-54f966059f5d\" (UID: \"fa0377dd-cc5c-4930-ba00-54f966059f5d\") " Dec 03 13:04:59 crc kubenswrapper[5052]: I1203 13:04:59.961236 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"nova-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/fa0377dd-cc5c-4930-ba00-54f966059f5d-nova-combined-ca-bundle\") pod \"fa0377dd-cc5c-4930-ba00-54f966059f5d\" (UID: \"fa0377dd-cc5c-4930-ba00-54f966059f5d\") " Dec 03 13:04:59 crc kubenswrapper[5052]: I1203 13:04:59.961267 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/fa0377dd-cc5c-4930-ba00-54f966059f5d-inventory\") pod \"fa0377dd-cc5c-4930-ba00-54f966059f5d\" (UID: \"fa0377dd-cc5c-4930-ba00-54f966059f5d\") " Dec 03 13:04:59 crc kubenswrapper[5052]: I1203 13:04:59.961297 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"nova-cell1-compute-config-0\" (UniqueName: \"kubernetes.io/secret/fa0377dd-cc5c-4930-ba00-54f966059f5d-nova-cell1-compute-config-0\") pod \"fa0377dd-cc5c-4930-ba00-54f966059f5d\" (UID: \"fa0377dd-cc5c-4930-ba00-54f966059f5d\") " Dec 03 13:04:59 crc kubenswrapper[5052]: I1203 13:04:59.961381 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"nova-migration-ssh-key-1\" (UniqueName: \"kubernetes.io/secret/fa0377dd-cc5c-4930-ba00-54f966059f5d-nova-migration-ssh-key-1\") pod \"fa0377dd-cc5c-4930-ba00-54f966059f5d\" (UID: \"fa0377dd-cc5c-4930-ba00-54f966059f5d\") " Dec 03 13:04:59 crc kubenswrapper[5052]: I1203 13:04:59.968222 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fa0377dd-cc5c-4930-ba00-54f966059f5d-nova-combined-ca-bundle" (OuterVolumeSpecName: "nova-combined-ca-bundle") pod "fa0377dd-cc5c-4930-ba00-54f966059f5d" (UID: "fa0377dd-cc5c-4930-ba00-54f966059f5d"). InnerVolumeSpecName "nova-combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:04:59 crc kubenswrapper[5052]: I1203 13:04:59.968737 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fa0377dd-cc5c-4930-ba00-54f966059f5d-kube-api-access-2qx2p" (OuterVolumeSpecName: "kube-api-access-2qx2p") pod "fa0377dd-cc5c-4930-ba00-54f966059f5d" (UID: "fa0377dd-cc5c-4930-ba00-54f966059f5d"). InnerVolumeSpecName "kube-api-access-2qx2p". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:04:59 crc kubenswrapper[5052]: I1203 13:04:59.986281 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fa0377dd-cc5c-4930-ba00-54f966059f5d-nova-migration-ssh-key-0" (OuterVolumeSpecName: "nova-migration-ssh-key-0") pod "fa0377dd-cc5c-4930-ba00-54f966059f5d" (UID: "fa0377dd-cc5c-4930-ba00-54f966059f5d"). InnerVolumeSpecName "nova-migration-ssh-key-0". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:04:59 crc kubenswrapper[5052]: I1203 13:04:59.989007 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fa0377dd-cc5c-4930-ba00-54f966059f5d-inventory" (OuterVolumeSpecName: "inventory") pod "fa0377dd-cc5c-4930-ba00-54f966059f5d" (UID: "fa0377dd-cc5c-4930-ba00-54f966059f5d"). InnerVolumeSpecName "inventory". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:04:59 crc kubenswrapper[5052]: I1203 13:04:59.989798 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fa0377dd-cc5c-4930-ba00-54f966059f5d-nova-cell1-compute-config-0" (OuterVolumeSpecName: "nova-cell1-compute-config-0") pod "fa0377dd-cc5c-4930-ba00-54f966059f5d" (UID: "fa0377dd-cc5c-4930-ba00-54f966059f5d"). InnerVolumeSpecName "nova-cell1-compute-config-0". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:04:59 crc kubenswrapper[5052]: I1203 13:04:59.990770 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fa0377dd-cc5c-4930-ba00-54f966059f5d-nova-migration-ssh-key-1" (OuterVolumeSpecName: "nova-migration-ssh-key-1") pod "fa0377dd-cc5c-4930-ba00-54f966059f5d" (UID: "fa0377dd-cc5c-4930-ba00-54f966059f5d"). InnerVolumeSpecName "nova-migration-ssh-key-1". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:04:59 crc kubenswrapper[5052]: I1203 13:04:59.993188 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fa0377dd-cc5c-4930-ba00-54f966059f5d-nova-cell1-compute-config-1" (OuterVolumeSpecName: "nova-cell1-compute-config-1") pod "fa0377dd-cc5c-4930-ba00-54f966059f5d" (UID: "fa0377dd-cc5c-4930-ba00-54f966059f5d"). InnerVolumeSpecName "nova-cell1-compute-config-1". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:04:59 crc kubenswrapper[5052]: I1203 13:04:59.995890 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fa0377dd-cc5c-4930-ba00-54f966059f5d-ssh-key" (OuterVolumeSpecName: "ssh-key") pod "fa0377dd-cc5c-4930-ba00-54f966059f5d" (UID: "fa0377dd-cc5c-4930-ba00-54f966059f5d"). InnerVolumeSpecName "ssh-key". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:05:00 crc kubenswrapper[5052]: I1203 13:05:00.064194 5052 reconciler_common.go:293] "Volume detached for volume \"nova-migration-ssh-key-0\" (UniqueName: \"kubernetes.io/secret/fa0377dd-cc5c-4930-ba00-54f966059f5d-nova-migration-ssh-key-0\") on node \"crc\" DevicePath \"\"" Dec 03 13:05:00 crc kubenswrapper[5052]: I1203 13:05:00.064239 5052 reconciler_common.go:293] "Volume detached for volume \"nova-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/fa0377dd-cc5c-4930-ba00-54f966059f5d-nova-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 13:05:00 crc kubenswrapper[5052]: I1203 13:05:00.064252 5052 reconciler_common.go:293] "Volume detached for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/fa0377dd-cc5c-4930-ba00-54f966059f5d-inventory\") on node \"crc\" DevicePath \"\"" Dec 03 13:05:00 crc kubenswrapper[5052]: I1203 13:05:00.064267 5052 reconciler_common.go:293] "Volume detached for volume \"nova-cell1-compute-config-0\" (UniqueName: \"kubernetes.io/secret/fa0377dd-cc5c-4930-ba00-54f966059f5d-nova-cell1-compute-config-0\") on node \"crc\" DevicePath \"\"" Dec 03 13:05:00 crc kubenswrapper[5052]: I1203 13:05:00.064281 5052 reconciler_common.go:293] "Volume detached for volume \"nova-migration-ssh-key-1\" (UniqueName: \"kubernetes.io/secret/fa0377dd-cc5c-4930-ba00-54f966059f5d-nova-migration-ssh-key-1\") on node \"crc\" DevicePath \"\"" Dec 03 13:05:00 crc kubenswrapper[5052]: I1203 13:05:00.064293 5052 reconciler_common.go:293] "Volume detached for volume \"nova-cell1-compute-config-1\" (UniqueName: \"kubernetes.io/secret/fa0377dd-cc5c-4930-ba00-54f966059f5d-nova-cell1-compute-config-1\") on node \"crc\" DevicePath \"\"" Dec 03 13:05:00 crc kubenswrapper[5052]: I1203 13:05:00.064306 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-2qx2p\" (UniqueName: \"kubernetes.io/projected/fa0377dd-cc5c-4930-ba00-54f966059f5d-kube-api-access-2qx2p\") on node \"crc\" DevicePath \"\"" Dec 03 13:05:00 crc kubenswrapper[5052]: I1203 13:05:00.064318 5052 reconciler_common.go:293] "Volume detached for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/fa0377dd-cc5c-4930-ba00-54f966059f5d-ssh-key\") on node \"crc\" DevicePath \"\"" Dec 03 13:05:00 crc kubenswrapper[5052]: I1203 13:05:00.584956 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/nova-edpm-multinodeset-edpm-compute-no-nodes-8h69r" Dec 03 13:05:00 crc kubenswrapper[5052]: I1203 13:05:00.591807 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/nova-edpm-multinodeset-edpm-compute-no-nodes-8h69r" event={"ID":"fa0377dd-cc5c-4930-ba00-54f966059f5d","Type":"ContainerDied","Data":"a49cde1761231ec96faafa3913ef119586d63543e64d2b78988747d2309761a6"} Dec 03 13:05:00 crc kubenswrapper[5052]: I1203 13:05:00.591877 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a49cde1761231ec96faafa3913ef119586d63543e64d2b78988747d2309761a6" Dec 03 13:05:00 crc kubenswrapper[5052]: I1203 13:05:00.593027 5052 generic.go:334] "Generic (PLEG): container finished" podID="271ab8d3-4460-4b19-82c1-e75a4fc488aa" containerID="9695bf67bf79d16d036f595c37796668bbc491961ac5c497f389fe969c830b2b" exitCode=0 Dec 03 13:05:00 crc kubenswrapper[5052]: I1203 13:05:00.593077 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/community-operators-5v5wb" event={"ID":"271ab8d3-4460-4b19-82c1-e75a4fc488aa","Type":"ContainerDied","Data":"9695bf67bf79d16d036f595c37796668bbc491961ac5c497f389fe969c830b2b"} Dec 03 13:05:01 crc kubenswrapper[5052]: I1203 13:05:01.605383 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/community-operators-5v5wb" event={"ID":"271ab8d3-4460-4b19-82c1-e75a4fc488aa","Type":"ContainerStarted","Data":"1389f06196ce48311cde262c298f4468cd48b7d3b3e406e9a3dcc008f6f715a3"} Dec 03 13:05:01 crc kubenswrapper[5052]: I1203 13:05:01.629103 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-marketplace/community-operators-5v5wb" podStartSLOduration=3.14570713 podStartE2EDuration="5.629086061s" podCreationTimestamp="2025-12-03 13:04:56 +0000 UTC" firstStartedPulling="2025-12-03 13:04:58.55857607 +0000 UTC m=+6289.146817834" lastFinishedPulling="2025-12-03 13:05:01.041954991 +0000 UTC m=+6291.630196765" observedRunningTime="2025-12-03 13:05:01.621533545 +0000 UTC m=+6292.209775309" watchObservedRunningTime="2025-12-03 13:05:01.629086061 +0000 UTC m=+6292.217327825" Dec 03 13:05:02 crc kubenswrapper[5052]: I1203 13:05:02.654051 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/bootstrap-edpm-multinodeset-edpm-compute-no-nodes-4cq6w"] Dec 03 13:05:02 crc kubenswrapper[5052]: I1203 13:05:02.674855 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/run-os-edpm-multinodeset-edpm-compute-no-nodes-ggzzt"] Dec 03 13:05:02 crc kubenswrapper[5052]: I1203 13:05:02.688826 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/neutron-dhcp-edpm-multinodeset-edpm-compute-no-nodes-269x6"] Dec 03 13:05:02 crc kubenswrapper[5052]: I1203 13:05:02.699721 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/install-os-edpm-multinodeset-edpm-compute-no-nodes-fwh4z"] Dec 03 13:05:02 crc kubenswrapper[5052]: I1203 13:05:02.709406 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/bootstrap-edpm-multinodeset-edpm-compute-beta-nodeset-b28bm"] Dec 03 13:05:02 crc kubenswrapper[5052]: I1203 13:05:02.719002 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/neutron-metadata-edpm-multinodeset-edpm-compute-no-nodes-dflf5"] Dec 03 13:05:02 crc kubenswrapper[5052]: I1203 13:05:02.728783 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/neutron-ovn-edpm-multinodeset-edpm-compute-no-nodes-jdv6g"] Dec 03 13:05:02 crc kubenswrapper[5052]: I1203 13:05:02.738235 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/neutron-ovn-edpm-multinodeset-edpm-compute-no-nodes-jdv6g"] Dec 03 13:05:02 crc kubenswrapper[5052]: I1203 13:05:02.748376 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/install-os-edpm-multinodeset-edpm-compute-no-nodes-fwh4z"] Dec 03 13:05:02 crc kubenswrapper[5052]: I1203 13:05:02.759408 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/bootstrap-edpm-multinodeset-edpm-compute-beta-nodeset-b28bm"] Dec 03 13:05:02 crc kubenswrapper[5052]: I1203 13:05:02.771168 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/neutron-sriov-edpm-multinodeset-edpm-compute-no-nodes-ccbd7"] Dec 03 13:05:02 crc kubenswrapper[5052]: I1203 13:05:02.784264 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/validate-network-edpm-multinodeset-edpm-compute-no-nodes-xqlzc"] Dec 03 13:05:02 crc kubenswrapper[5052]: I1203 13:05:02.794331 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/configure-network-edpm-multinodeset-edpm-compute-no-nodes-v5m6s"] Dec 03 13:05:02 crc kubenswrapper[5052]: I1203 13:05:02.805693 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/bootstrap-edpm-multinodeset-edpm-compute-no-nodes-4cq6w"] Dec 03 13:05:02 crc kubenswrapper[5052]: I1203 13:05:02.817761 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ovn-edpm-multinodeset-edpm-compute-no-nodes-shxd8"] Dec 03 13:05:02 crc kubenswrapper[5052]: I1203 13:05:02.824604 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/download-cache-edpm-multinodeset-edpm-compute-beta-nodeset7cf9h"] Dec 03 13:05:02 crc kubenswrapper[5052]: I1203 13:05:02.833716 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/run-os-edpm-multinodeset-edpm-compute-no-nodes-ggzzt"] Dec 03 13:05:02 crc kubenswrapper[5052]: I1203 13:05:02.842937 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/neutron-dhcp-edpm-multinodeset-edpm-compute-no-nodes-269x6"] Dec 03 13:05:02 crc kubenswrapper[5052]: I1203 13:05:02.852668 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/neutron-metadata-edpm-multinodeset-edpm-compute-no-nodes-dflf5"] Dec 03 13:05:02 crc kubenswrapper[5052]: I1203 13:05:02.861350 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/libvirt-edpm-multinodeset-edpm-compute-no-nodes-fv5kf"] Dec 03 13:05:02 crc kubenswrapper[5052]: I1203 13:05:02.868289 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/download-cache-edpm-multinodeset-edpm-compute-beta-nodeset7cf9h"] Dec 03 13:05:02 crc kubenswrapper[5052]: I1203 13:05:02.876139 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/download-cache-edpm-multinodeset-edpm-compute-no-nodes-k5p9j"] Dec 03 13:05:02 crc kubenswrapper[5052]: I1203 13:05:02.884679 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/validate-network-edpm-multinodeset-edpm-compute-no-nodes-xqlzc"] Dec 03 13:05:02 crc kubenswrapper[5052]: I1203 13:05:02.893894 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/install-certs-edpm-multinodeset-edpm-compute-no-nodes-6z9jd"] Dec 03 13:05:02 crc kubenswrapper[5052]: I1203 13:05:02.902307 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/neutron-sriov-edpm-multinodeset-edpm-compute-no-nodes-ccbd7"] Dec 03 13:05:02 crc kubenswrapper[5052]: I1203 13:05:02.911165 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/configure-os-edpm-multinodeset-edpm-compute-no-nodes-cndq9"] Dec 03 13:05:02 crc kubenswrapper[5052]: I1203 13:05:02.920035 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/libvirt-edpm-multinodeset-edpm-compute-no-nodes-fv5kf"] Dec 03 13:05:02 crc kubenswrapper[5052]: I1203 13:05:02.929062 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ovn-edpm-multinodeset-edpm-compute-no-nodes-shxd8"] Dec 03 13:05:02 crc kubenswrapper[5052]: I1203 13:05:02.938258 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-edpm-multinodeset-edpm-compute-no-nodes-8h69r"] Dec 03 13:05:02 crc kubenswrapper[5052]: I1203 13:05:02.947168 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/configure-network-edpm-multinodeset-edpm-compute-no-nodes-v5m6s"] Dec 03 13:05:02 crc kubenswrapper[5052]: I1203 13:05:02.957115 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/download-cache-edpm-multinodeset-edpm-compute-no-nodes-k5p9j"] Dec 03 13:05:02 crc kubenswrapper[5052]: I1203 13:05:02.964756 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/configure-os-edpm-multinodeset-edpm-compute-no-nodes-cndq9"] Dec 03 13:05:02 crc kubenswrapper[5052]: I1203 13:05:02.973117 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/install-certs-edpm-multinodeset-edpm-compute-no-nodes-6z9jd"] Dec 03 13:05:02 crc kubenswrapper[5052]: I1203 13:05:02.980450 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-edpm-multinodeset-edpm-compute-no-nodes-8h69r"] Dec 03 13:05:02 crc kubenswrapper[5052]: I1203 13:05:02.987587 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ovn-edpm-compute-no-nodes-updated-ovn-cm-edpm-compute-no-nlgvrf"] Dec 03 13:05:02 crc kubenswrapper[5052]: I1203 13:05:02.996211 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ovn-edpm-compute-no-nodes-updated-ovn-cm-edpm-compute-no-nlgvrf"] Dec 03 13:05:03 crc kubenswrapper[5052]: I1203 13:05:03.003828 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/custom-svc-edpm-compute-no-nodes-ovrd-edpm-compute-no-noded8jt2"] Dec 03 13:05:03 crc kubenswrapper[5052]: I1203 13:05:03.011291 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/custom-svc-edpm-compute-no-nodes-ovrd-edpm-compute-no-noded8jt2"] Dec 03 13:05:03 crc kubenswrapper[5052]: I1203 13:05:03.018541 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/neutron-metadata-edpm-compute-no-nodes-edpm-compute-no-nodhbt7h"] Dec 03 13:05:03 crc kubenswrapper[5052]: I1203 13:05:03.025801 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/neutron-metadata-edpm-compute-no-nodes-edpm-compute-no-nodhbt7h"] Dec 03 13:05:03 crc kubenswrapper[5052]: I1203 13:05:03.032580 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/download-cache-edpm-compute-no-nodes-edpm-compute-no-nodes2tk9d"] Dec 03 13:05:03 crc kubenswrapper[5052]: I1203 13:05:03.040090 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/configure-os-edpm-compute-no-nodes-edpm-compute-no-nodes-l6qqd"] Dec 03 13:05:03 crc kubenswrapper[5052]: I1203 13:05:03.068847 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/run-os-edpm-compute-no-nodes-edpm-compute-no-nodes-d8kqk"] Dec 03 13:05:03 crc kubenswrapper[5052]: I1203 13:05:03.075658 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/configure-network-edpm-compute-no-nodes-edpm-compute-no-no84qz4"] Dec 03 13:05:03 crc kubenswrapper[5052]: I1203 13:05:03.083837 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/ovn-edpm-compute-no-nodes-edpm-compute-no-nodes-gjbzv"] Dec 03 13:05:03 crc kubenswrapper[5052]: I1203 13:05:03.091308 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/libvirt-edpm-compute-no-nodes-edpm-compute-no-nodes-fb8nw"] Dec 03 13:05:03 crc kubenswrapper[5052]: I1203 13:05:03.098837 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/bootstrap-edpm-compute-no-nodes-edpm-compute-no-nodes-684xr"] Dec 03 13:05:03 crc kubenswrapper[5052]: I1203 13:05:03.104526 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/nova-edpm-compute-no-nodes-edpm-compute-no-nodes-lnf9c"] Dec 03 13:05:03 crc kubenswrapper[5052]: I1203 13:05:03.112956 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/validate-network-edpm-compute-no-nodes-edpm-compute-no-nodfw8nd"] Dec 03 13:05:03 crc kubenswrapper[5052]: I1203 13:05:03.121067 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/install-os-edpm-compute-no-nodes-edpm-compute-no-nodes-l4h9g"] Dec 03 13:05:03 crc kubenswrapper[5052]: I1203 13:05:03.129117 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/neutron-ovn-edpm-compute-no-nodes-edpm-compute-no-nodes-tlp6p"] Dec 03 13:05:03 crc kubenswrapper[5052]: I1203 13:05:03.134492 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/neutron-sriov-edpm-compute-no-nodes-edpm-compute-no-nodes-mjjbz"] Dec 03 13:05:03 crc kubenswrapper[5052]: I1203 13:05:03.140723 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/ovn-edpm-compute-no-nodes-edpm-compute-no-nodes-gjbzv"] Dec 03 13:05:03 crc kubenswrapper[5052]: I1203 13:05:03.146965 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/validate-network-edpm-compute-no-nodes-edpm-compute-no-nodfw8nd"] Dec 03 13:05:03 crc kubenswrapper[5052]: I1203 13:05:03.156010 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/configure-os-edpm-compute-no-nodes-edpm-compute-no-nodes-l6qqd"] Dec 03 13:05:03 crc kubenswrapper[5052]: I1203 13:05:03.163129 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/download-cache-edpm-compute-no-nodes-edpm-compute-no-nodes2tk9d"] Dec 03 13:05:03 crc kubenswrapper[5052]: I1203 13:05:03.170389 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/libvirt-edpm-compute-no-nodes-edpm-compute-no-nodes-fb8nw"] Dec 03 13:05:03 crc kubenswrapper[5052]: I1203 13:05:03.180674 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/nova-edpm-compute-no-nodes-edpm-compute-no-nodes-lnf9c"] Dec 03 13:05:03 crc kubenswrapper[5052]: I1203 13:05:03.188015 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/neutron-dhcp-edpm-compute-no-nodes-edpm-compute-no-nodes-7bvhh"] Dec 03 13:05:03 crc kubenswrapper[5052]: I1203 13:05:03.194805 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/bootstrap-edpm-compute-no-nodes-edpm-compute-no-nodes-684xr"] Dec 03 13:05:03 crc kubenswrapper[5052]: I1203 13:05:03.201628 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/run-os-edpm-compute-no-nodes-edpm-compute-no-nodes-d8kqk"] Dec 03 13:05:03 crc kubenswrapper[5052]: I1203 13:05:03.209787 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/configure-network-edpm-compute-no-nodes-edpm-compute-no-no84qz4"] Dec 03 13:05:03 crc kubenswrapper[5052]: I1203 13:05:03.216050 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/neutron-sriov-edpm-compute-no-nodes-edpm-compute-no-nodes-mjjbz"] Dec 03 13:05:03 crc kubenswrapper[5052]: I1203 13:05:03.223599 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/neutron-ovn-edpm-compute-no-nodes-edpm-compute-no-nodes-tlp6p"] Dec 03 13:05:03 crc kubenswrapper[5052]: I1203 13:05:03.231396 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/install-certs-edpm-compute-no-nodes-edpm-compute-no-nodes-45bwg"] Dec 03 13:05:03 crc kubenswrapper[5052]: I1203 13:05:03.237831 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/install-os-edpm-compute-no-nodes-edpm-compute-no-nodes-l4h9g"] Dec 03 13:05:03 crc kubenswrapper[5052]: I1203 13:05:03.244124 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/neutron-dhcp-edpm-compute-no-nodes-edpm-compute-no-nodes-7bvhh"] Dec 03 13:05:03 crc kubenswrapper[5052]: I1203 13:05:03.250237 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/install-certs-edpm-compute-no-nodes-edpm-compute-no-nodes-45bwg"] Dec 03 13:05:03 crc kubenswrapper[5052]: I1203 13:05:03.256528 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-64864b6d57-blmt5"] Dec 03 13:05:03 crc kubenswrapper[5052]: E1203 13:05:03.257140 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="fa0377dd-cc5c-4930-ba00-54f966059f5d" containerName="nova-edpm-multinodeset-edpm-compute-no-nodes" Dec 03 13:05:03 crc kubenswrapper[5052]: I1203 13:05:03.257166 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="fa0377dd-cc5c-4930-ba00-54f966059f5d" containerName="nova-edpm-multinodeset-edpm-compute-no-nodes" Dec 03 13:05:03 crc kubenswrapper[5052]: I1203 13:05:03.257420 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="fa0377dd-cc5c-4930-ba00-54f966059f5d" containerName="nova-edpm-multinodeset-edpm-compute-no-nodes" Dec 03 13:05:03 crc kubenswrapper[5052]: I1203 13:05:03.259478 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-64864b6d57-blmt5" Dec 03 13:05:03 crc kubenswrapper[5052]: I1203 13:05:03.264196 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-64864b6d57-blmt5"] Dec 03 13:05:03 crc kubenswrapper[5052]: I1203 13:05:03.274472 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-64864b6d57-blmt5"] Dec 03 13:05:03 crc kubenswrapper[5052]: E1203 13:05:03.278882 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[config dnsmasq-svc edpm-compute-no-nodes kube-api-access-926t2], unattached volumes=[], failed to process volumes=[]: context canceled" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-64864b6d57-blmt5" podUID="549519fd-33d2-4d52-9751-079808ea167f" Dec 03 13:05:03 crc kubenswrapper[5052]: I1203 13:05:03.282696 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-4s9xh"] Dec 03 13:05:03 crc kubenswrapper[5052]: I1203 13:05:03.284050 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-4s9xh" Dec 03 13:05:03 crc kubenswrapper[5052]: I1203 13:05:03.289829 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-4s9xh"] Dec 03 13:05:03 crc kubenswrapper[5052]: I1203 13:05:03.307181 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/549519fd-33d2-4d52-9751-079808ea167f-config\") pod \"dnsmasq-dnsmasq-64864b6d57-blmt5\" (UID: \"549519fd-33d2-4d52-9751-079808ea167f\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-64864b6d57-blmt5" Dec 03 13:05:03 crc kubenswrapper[5052]: I1203 13:05:03.307235 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/549519fd-33d2-4d52-9751-079808ea167f-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-64864b6d57-blmt5\" (UID: \"549519fd-33d2-4d52-9751-079808ea167f\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-64864b6d57-blmt5" Dec 03 13:05:03 crc kubenswrapper[5052]: I1203 13:05:03.307271 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-926t2\" (UniqueName: \"kubernetes.io/projected/549519fd-33d2-4d52-9751-079808ea167f-kube-api-access-926t2\") pod \"dnsmasq-dnsmasq-64864b6d57-blmt5\" (UID: \"549519fd-33d2-4d52-9751-079808ea167f\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-64864b6d57-blmt5" Dec 03 13:05:03 crc kubenswrapper[5052]: I1203 13:05:03.307476 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"edpm-compute-no-nodes\" (UniqueName: \"kubernetes.io/configmap/549519fd-33d2-4d52-9751-079808ea167f-edpm-compute-no-nodes\") pod \"dnsmasq-dnsmasq-64864b6d57-blmt5\" (UID: \"549519fd-33d2-4d52-9751-079808ea167f\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-64864b6d57-blmt5" Dec 03 13:05:03 crc kubenswrapper[5052]: I1203 13:05:03.409271 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kszbj\" (UniqueName: \"kubernetes.io/projected/030b2288-f819-48fd-91a2-e08b1130e648-kube-api-access-kszbj\") pod \"dnsmasq-dnsmasq-84b9f45d47-4s9xh\" (UID: \"030b2288-f819-48fd-91a2-e08b1130e648\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-4s9xh" Dec 03 13:05:03 crc kubenswrapper[5052]: I1203 13:05:03.409361 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/549519fd-33d2-4d52-9751-079808ea167f-config\") pod \"dnsmasq-dnsmasq-64864b6d57-blmt5\" (UID: \"549519fd-33d2-4d52-9751-079808ea167f\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-64864b6d57-blmt5" Dec 03 13:05:03 crc kubenswrapper[5052]: I1203 13:05:03.409385 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/549519fd-33d2-4d52-9751-079808ea167f-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-64864b6d57-blmt5\" (UID: \"549519fd-33d2-4d52-9751-079808ea167f\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-64864b6d57-blmt5" Dec 03 13:05:03 crc kubenswrapper[5052]: I1203 13:05:03.409403 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-926t2\" (UniqueName: \"kubernetes.io/projected/549519fd-33d2-4d52-9751-079808ea167f-kube-api-access-926t2\") pod \"dnsmasq-dnsmasq-64864b6d57-blmt5\" (UID: \"549519fd-33d2-4d52-9751-079808ea167f\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-64864b6d57-blmt5" Dec 03 13:05:03 crc kubenswrapper[5052]: I1203 13:05:03.409429 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/030b2288-f819-48fd-91a2-e08b1130e648-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-84b9f45d47-4s9xh\" (UID: \"030b2288-f819-48fd-91a2-e08b1130e648\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-4s9xh" Dec 03 13:05:03 crc kubenswrapper[5052]: I1203 13:05:03.409474 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/030b2288-f819-48fd-91a2-e08b1130e648-config\") pod \"dnsmasq-dnsmasq-84b9f45d47-4s9xh\" (UID: \"030b2288-f819-48fd-91a2-e08b1130e648\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-4s9xh" Dec 03 13:05:03 crc kubenswrapper[5052]: I1203 13:05:03.409497 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"edpm-compute-no-nodes\" (UniqueName: \"kubernetes.io/configmap/549519fd-33d2-4d52-9751-079808ea167f-edpm-compute-no-nodes\") pod \"dnsmasq-dnsmasq-64864b6d57-blmt5\" (UID: \"549519fd-33d2-4d52-9751-079808ea167f\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-64864b6d57-blmt5" Dec 03 13:05:03 crc kubenswrapper[5052]: E1203 13:05:03.409622 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/edpm-compute-no-nodes: configmap "edpm-compute-no-nodes" not found Dec 03 13:05:03 crc kubenswrapper[5052]: E1203 13:05:03.409733 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/549519fd-33d2-4d52-9751-079808ea167f-edpm-compute-no-nodes podName:549519fd-33d2-4d52-9751-079808ea167f nodeName:}" failed. No retries permitted until 2025-12-03 13:05:03.909686353 +0000 UTC m=+6294.497928117 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "edpm-compute-no-nodes" (UniqueName: "kubernetes.io/configmap/549519fd-33d2-4d52-9751-079808ea167f-edpm-compute-no-nodes") pod "dnsmasq-dnsmasq-64864b6d57-blmt5" (UID: "549519fd-33d2-4d52-9751-079808ea167f") : configmap "edpm-compute-no-nodes" not found Dec 03 13:05:03 crc kubenswrapper[5052]: I1203 13:05:03.410501 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/549519fd-33d2-4d52-9751-079808ea167f-config\") pod \"dnsmasq-dnsmasq-64864b6d57-blmt5\" (UID: \"549519fd-33d2-4d52-9751-079808ea167f\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-64864b6d57-blmt5" Dec 03 13:05:03 crc kubenswrapper[5052]: I1203 13:05:03.410715 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/549519fd-33d2-4d52-9751-079808ea167f-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-64864b6d57-blmt5\" (UID: \"549519fd-33d2-4d52-9751-079808ea167f\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-64864b6d57-blmt5" Dec 03 13:05:03 crc kubenswrapper[5052]: I1203 13:05:03.428794 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-926t2\" (UniqueName: \"kubernetes.io/projected/549519fd-33d2-4d52-9751-079808ea167f-kube-api-access-926t2\") pod \"dnsmasq-dnsmasq-64864b6d57-blmt5\" (UID: \"549519fd-33d2-4d52-9751-079808ea167f\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-64864b6d57-blmt5" Dec 03 13:05:03 crc kubenswrapper[5052]: I1203 13:05:03.511026 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/030b2288-f819-48fd-91a2-e08b1130e648-config\") pod \"dnsmasq-dnsmasq-84b9f45d47-4s9xh\" (UID: \"030b2288-f819-48fd-91a2-e08b1130e648\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-4s9xh" Dec 03 13:05:03 crc kubenswrapper[5052]: I1203 13:05:03.511152 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-kszbj\" (UniqueName: \"kubernetes.io/projected/030b2288-f819-48fd-91a2-e08b1130e648-kube-api-access-kszbj\") pod \"dnsmasq-dnsmasq-84b9f45d47-4s9xh\" (UID: \"030b2288-f819-48fd-91a2-e08b1130e648\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-4s9xh" Dec 03 13:05:03 crc kubenswrapper[5052]: I1203 13:05:03.511199 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/030b2288-f819-48fd-91a2-e08b1130e648-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-84b9f45d47-4s9xh\" (UID: \"030b2288-f819-48fd-91a2-e08b1130e648\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-4s9xh" Dec 03 13:05:03 crc kubenswrapper[5052]: I1203 13:05:03.512161 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/030b2288-f819-48fd-91a2-e08b1130e648-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-84b9f45d47-4s9xh\" (UID: \"030b2288-f819-48fd-91a2-e08b1130e648\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-4s9xh" Dec 03 13:05:03 crc kubenswrapper[5052]: I1203 13:05:03.512268 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/030b2288-f819-48fd-91a2-e08b1130e648-config\") pod \"dnsmasq-dnsmasq-84b9f45d47-4s9xh\" (UID: \"030b2288-f819-48fd-91a2-e08b1130e648\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-4s9xh" Dec 03 13:05:03 crc kubenswrapper[5052]: I1203 13:05:03.530192 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-kszbj\" (UniqueName: \"kubernetes.io/projected/030b2288-f819-48fd-91a2-e08b1130e648-kube-api-access-kszbj\") pod \"dnsmasq-dnsmasq-84b9f45d47-4s9xh\" (UID: \"030b2288-f819-48fd-91a2-e08b1130e648\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-4s9xh" Dec 03 13:05:03 crc kubenswrapper[5052]: I1203 13:05:03.599643 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-4s9xh" Dec 03 13:05:03 crc kubenswrapper[5052]: I1203 13:05:03.623457 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-64864b6d57-blmt5" Dec 03 13:05:03 crc kubenswrapper[5052]: I1203 13:05:03.682784 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-64864b6d57-blmt5" Dec 03 13:05:03 crc kubenswrapper[5052]: I1203 13:05:03.685373 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="01a181ab-bf69-4454-98c5-828d1e0980c0" path="/var/lib/kubelet/pods/01a181ab-bf69-4454-98c5-828d1e0980c0/volumes" Dec 03 13:05:03 crc kubenswrapper[5052]: I1203 13:05:03.686148 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="07266f43-def2-46f4-8a0f-c856ec3d030b" path="/var/lib/kubelet/pods/07266f43-def2-46f4-8a0f-c856ec3d030b/volumes" Dec 03 13:05:03 crc kubenswrapper[5052]: I1203 13:05:03.688341 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="07ec94a0-7a1b-4062-aeca-22402b5f344a" path="/var/lib/kubelet/pods/07ec94a0-7a1b-4062-aeca-22402b5f344a/volumes" Dec 03 13:05:03 crc kubenswrapper[5052]: I1203 13:05:03.689054 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0a9ff100-5a4a-4ba1-aba6-44c45c064fb1" path="/var/lib/kubelet/pods/0a9ff100-5a4a-4ba1-aba6-44c45c064fb1/volumes" Dec 03 13:05:03 crc kubenswrapper[5052]: I1203 13:05:03.689640 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="10392605-c383-49fd-b4e9-b53bdda69795" path="/var/lib/kubelet/pods/10392605-c383-49fd-b4e9-b53bdda69795/volumes" Dec 03 13:05:03 crc kubenswrapper[5052]: I1203 13:05:03.690188 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2b71a6d5-1e66-4601-9e86-f71fc1cd66a0" path="/var/lib/kubelet/pods/2b71a6d5-1e66-4601-9e86-f71fc1cd66a0/volumes" Dec 03 13:05:03 crc kubenswrapper[5052]: I1203 13:05:03.691402 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2d3f5967-7cd5-4400-865f-ad44684747f6" path="/var/lib/kubelet/pods/2d3f5967-7cd5-4400-865f-ad44684747f6/volumes" Dec 03 13:05:03 crc kubenswrapper[5052]: I1203 13:05:03.691990 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="37fe7283-01d0-4402-a15c-830a8b78aa20" path="/var/lib/kubelet/pods/37fe7283-01d0-4402-a15c-830a8b78aa20/volumes" Dec 03 13:05:03 crc kubenswrapper[5052]: I1203 13:05:03.692844 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3c02dad4-5a4f-476f-bf8a-a93e1e397cc8" path="/var/lib/kubelet/pods/3c02dad4-5a4f-476f-bf8a-a93e1e397cc8/volumes" Dec 03 13:05:03 crc kubenswrapper[5052]: I1203 13:05:03.693960 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3febf655-343c-4384-a42c-0ee3dc48f21a" path="/var/lib/kubelet/pods/3febf655-343c-4384-a42c-0ee3dc48f21a/volumes" Dec 03 13:05:03 crc kubenswrapper[5052]: I1203 13:05:03.694536 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="405110f4-d79e-4505-803b-5204317a4bfa" path="/var/lib/kubelet/pods/405110f4-d79e-4505-803b-5204317a4bfa/volumes" Dec 03 13:05:03 crc kubenswrapper[5052]: I1203 13:05:03.695148 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4d00433c-3aaf-42ab-ba98-b3ac7452f440" path="/var/lib/kubelet/pods/4d00433c-3aaf-42ab-ba98-b3ac7452f440/volumes" Dec 03 13:05:03 crc kubenswrapper[5052]: I1203 13:05:03.696251 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="504b0477-2b1b-403c-9bbf-4dfe2543f8dd" path="/var/lib/kubelet/pods/504b0477-2b1b-403c-9bbf-4dfe2543f8dd/volumes" Dec 03 13:05:03 crc kubenswrapper[5052]: I1203 13:05:03.696996 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="58aa9910-8f11-4e46-abd7-816fb0439dc5" path="/var/lib/kubelet/pods/58aa9910-8f11-4e46-abd7-816fb0439dc5/volumes" Dec 03 13:05:03 crc kubenswrapper[5052]: I1203 13:05:03.697647 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5a60accb-4776-478c-b8c4-799e2cb9af8d" path="/var/lib/kubelet/pods/5a60accb-4776-478c-b8c4-799e2cb9af8d/volumes" Dec 03 13:05:03 crc kubenswrapper[5052]: I1203 13:05:03.698314 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5f61d52a-bcc2-43f1-bcc5-076fa075a3a1" path="/var/lib/kubelet/pods/5f61d52a-bcc2-43f1-bcc5-076fa075a3a1/volumes" Dec 03 13:05:03 crc kubenswrapper[5052]: I1203 13:05:03.699413 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6b467669-02e7-48e6-9707-089aac0ef369" path="/var/lib/kubelet/pods/6b467669-02e7-48e6-9707-089aac0ef369/volumes" Dec 03 13:05:03 crc kubenswrapper[5052]: I1203 13:05:03.700038 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6e7fa506-85c2-4b8f-a21b-1b0e195b2e8e" path="/var/lib/kubelet/pods/6e7fa506-85c2-4b8f-a21b-1b0e195b2e8e/volumes" Dec 03 13:05:03 crc kubenswrapper[5052]: I1203 13:05:03.700633 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="71a964fa-6056-4f7a-80ff-b792748299ba" path="/var/lib/kubelet/pods/71a964fa-6056-4f7a-80ff-b792748299ba/volumes" Dec 03 13:05:03 crc kubenswrapper[5052]: I1203 13:05:03.701869 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="816cc89d-6c4d-4199-a81c-a7b29cc3f85a" path="/var/lib/kubelet/pods/816cc89d-6c4d-4199-a81c-a7b29cc3f85a/volumes" Dec 03 13:05:03 crc kubenswrapper[5052]: I1203 13:05:03.703992 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8233e500-8181-426b-aaf3-c7ecf3251d4c" path="/var/lib/kubelet/pods/8233e500-8181-426b-aaf3-c7ecf3251d4c/volumes" Dec 03 13:05:03 crc kubenswrapper[5052]: I1203 13:05:03.704634 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="83226f18-0a8d-4fc6-8d02-32056e5fb8ae" path="/var/lib/kubelet/pods/83226f18-0a8d-4fc6-8d02-32056e5fb8ae/volumes" Dec 03 13:05:03 crc kubenswrapper[5052]: I1203 13:05:03.705807 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="93f47414-793b-41b8-95ac-1e3325bc494c" path="/var/lib/kubelet/pods/93f47414-793b-41b8-95ac-1e3325bc494c/volumes" Dec 03 13:05:03 crc kubenswrapper[5052]: I1203 13:05:03.706408 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="951b6922-5edc-46f6-8a69-929e7517850c" path="/var/lib/kubelet/pods/951b6922-5edc-46f6-8a69-929e7517850c/volumes" Dec 03 13:05:03 crc kubenswrapper[5052]: I1203 13:05:03.706925 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="970c0318-3c82-4671-b0b7-e4c573f06226" path="/var/lib/kubelet/pods/970c0318-3c82-4671-b0b7-e4c573f06226/volumes" Dec 03 13:05:03 crc kubenswrapper[5052]: I1203 13:05:03.707373 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a913674d-b6c4-442f-9d33-dab50e694cc4" path="/var/lib/kubelet/pods/a913674d-b6c4-442f-9d33-dab50e694cc4/volumes" Dec 03 13:05:03 crc kubenswrapper[5052]: I1203 13:05:03.708337 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bf143189-a60c-4f8c-86e8-a2cfaeb07e4f" path="/var/lib/kubelet/pods/bf143189-a60c-4f8c-86e8-a2cfaeb07e4f/volumes" Dec 03 13:05:03 crc kubenswrapper[5052]: I1203 13:05:03.708830 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c80a4930-073c-4981-857f-f3fe056f6fd1" path="/var/lib/kubelet/pods/c80a4930-073c-4981-857f-f3fe056f6fd1/volumes" Dec 03 13:05:03 crc kubenswrapper[5052]: I1203 13:05:03.709312 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cbb40c47-9dc0-4934-8964-7fd7a898d250" path="/var/lib/kubelet/pods/cbb40c47-9dc0-4934-8964-7fd7a898d250/volumes" Dec 03 13:05:03 crc kubenswrapper[5052]: I1203 13:05:03.710259 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e0cf2a46-be43-44a0-adce-67ceb2cb06ee" path="/var/lib/kubelet/pods/e0cf2a46-be43-44a0-adce-67ceb2cb06ee/volumes" Dec 03 13:05:03 crc kubenswrapper[5052]: I1203 13:05:03.710755 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e16e2bad-fbf3-4817-bb52-422bc88a76b0" path="/var/lib/kubelet/pods/e16e2bad-fbf3-4817-bb52-422bc88a76b0/volumes" Dec 03 13:05:03 crc kubenswrapper[5052]: I1203 13:05:03.711201 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f042c0a4-0556-430b-8c10-9e662438c938" path="/var/lib/kubelet/pods/f042c0a4-0556-430b-8c10-9e662438c938/volumes" Dec 03 13:05:03 crc kubenswrapper[5052]: I1203 13:05:03.712050 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fa0377dd-cc5c-4930-ba00-54f966059f5d" path="/var/lib/kubelet/pods/fa0377dd-cc5c-4930-ba00-54f966059f5d/volumes" Dec 03 13:05:03 crc kubenswrapper[5052]: I1203 13:05:03.712495 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fa362928-93ec-4d50-98e0-0c3e6bfe2758" path="/var/lib/kubelet/pods/fa362928-93ec-4d50-98e0-0c3e6bfe2758/volumes" Dec 03 13:05:03 crc kubenswrapper[5052]: I1203 13:05:03.816949 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/549519fd-33d2-4d52-9751-079808ea167f-config\") pod \"549519fd-33d2-4d52-9751-079808ea167f\" (UID: \"549519fd-33d2-4d52-9751-079808ea167f\") " Dec 03 13:05:03 crc kubenswrapper[5052]: I1203 13:05:03.817106 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-926t2\" (UniqueName: \"kubernetes.io/projected/549519fd-33d2-4d52-9751-079808ea167f-kube-api-access-926t2\") pod \"549519fd-33d2-4d52-9751-079808ea167f\" (UID: \"549519fd-33d2-4d52-9751-079808ea167f\") " Dec 03 13:05:03 crc kubenswrapper[5052]: I1203 13:05:03.817128 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/549519fd-33d2-4d52-9751-079808ea167f-dnsmasq-svc\") pod \"549519fd-33d2-4d52-9751-079808ea167f\" (UID: \"549519fd-33d2-4d52-9751-079808ea167f\") " Dec 03 13:05:03 crc kubenswrapper[5052]: I1203 13:05:03.817487 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/549519fd-33d2-4d52-9751-079808ea167f-config" (OuterVolumeSpecName: "config") pod "549519fd-33d2-4d52-9751-079808ea167f" (UID: "549519fd-33d2-4d52-9751-079808ea167f"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:05:03 crc kubenswrapper[5052]: I1203 13:05:03.818629 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/549519fd-33d2-4d52-9751-079808ea167f-dnsmasq-svc" (OuterVolumeSpecName: "dnsmasq-svc") pod "549519fd-33d2-4d52-9751-079808ea167f" (UID: "549519fd-33d2-4d52-9751-079808ea167f"). InnerVolumeSpecName "dnsmasq-svc". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:05:03 crc kubenswrapper[5052]: I1203 13:05:03.834024 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/549519fd-33d2-4d52-9751-079808ea167f-kube-api-access-926t2" (OuterVolumeSpecName: "kube-api-access-926t2") pod "549519fd-33d2-4d52-9751-079808ea167f" (UID: "549519fd-33d2-4d52-9751-079808ea167f"). InnerVolumeSpecName "kube-api-access-926t2". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:05:03 crc kubenswrapper[5052]: I1203 13:05:03.918899 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"edpm-compute-no-nodes\" (UniqueName: \"kubernetes.io/configmap/549519fd-33d2-4d52-9751-079808ea167f-edpm-compute-no-nodes\") pod \"dnsmasq-dnsmasq-64864b6d57-blmt5\" (UID: \"549519fd-33d2-4d52-9751-079808ea167f\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-64864b6d57-blmt5" Dec 03 13:05:03 crc kubenswrapper[5052]: E1203 13:05:03.919062 5052 configmap.go:193] Couldn't get configMap openstack-kuttl-tests/edpm-compute-no-nodes: configmap "edpm-compute-no-nodes" not found Dec 03 13:05:03 crc kubenswrapper[5052]: E1203 13:05:03.919120 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/549519fd-33d2-4d52-9751-079808ea167f-edpm-compute-no-nodes podName:549519fd-33d2-4d52-9751-079808ea167f nodeName:}" failed. No retries permitted until 2025-12-03 13:05:04.919104202 +0000 UTC m=+6295.507345966 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "edpm-compute-no-nodes" (UniqueName: "kubernetes.io/configmap/549519fd-33d2-4d52-9751-079808ea167f-edpm-compute-no-nodes") pod "dnsmasq-dnsmasq-64864b6d57-blmt5" (UID: "549519fd-33d2-4d52-9751-079808ea167f") : configmap "edpm-compute-no-nodes" not found Dec 03 13:05:03 crc kubenswrapper[5052]: I1203 13:05:03.919068 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-926t2\" (UniqueName: \"kubernetes.io/projected/549519fd-33d2-4d52-9751-079808ea167f-kube-api-access-926t2\") on node \"crc\" DevicePath \"\"" Dec 03 13:05:03 crc kubenswrapper[5052]: I1203 13:05:03.919175 5052 reconciler_common.go:293] "Volume detached for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/549519fd-33d2-4d52-9751-079808ea167f-dnsmasq-svc\") on node \"crc\" DevicePath \"\"" Dec 03 13:05:03 crc kubenswrapper[5052]: I1203 13:05:03.919190 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/549519fd-33d2-4d52-9751-079808ea167f-config\") on node \"crc\" DevicePath \"\"" Dec 03 13:05:04 crc kubenswrapper[5052]: I1203 13:05:04.085345 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-4s9xh"] Dec 03 13:05:04 crc kubenswrapper[5052]: I1203 13:05:04.634682 5052 generic.go:334] "Generic (PLEG): container finished" podID="030b2288-f819-48fd-91a2-e08b1130e648" containerID="b9f57165a0c4c083b6315e54f4620632e3db22e75d5ebfcd465b1fe0467252a9" exitCode=0 Dec 03 13:05:04 crc kubenswrapper[5052]: I1203 13:05:04.634737 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-4s9xh" event={"ID":"030b2288-f819-48fd-91a2-e08b1130e648","Type":"ContainerDied","Data":"b9f57165a0c4c083b6315e54f4620632e3db22e75d5ebfcd465b1fe0467252a9"} Dec 03 13:05:04 crc kubenswrapper[5052]: I1203 13:05:04.635034 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-64864b6d57-blmt5" Dec 03 13:05:04 crc kubenswrapper[5052]: I1203 13:05:04.635057 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-4s9xh" event={"ID":"030b2288-f819-48fd-91a2-e08b1130e648","Type":"ContainerStarted","Data":"7bce5b84beacec4a6e859f708432283096ff43aa0ac5aebd15f1ac5dc49f681d"} Dec 03 13:05:04 crc kubenswrapper[5052]: I1203 13:05:04.819908 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-64864b6d57-blmt5"] Dec 03 13:05:04 crc kubenswrapper[5052]: I1203 13:05:04.823804 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-64864b6d57-blmt5"] Dec 03 13:05:04 crc kubenswrapper[5052]: I1203 13:05:04.945403 5052 reconciler_common.go:293] "Volume detached for volume \"edpm-compute-no-nodes\" (UniqueName: \"kubernetes.io/configmap/549519fd-33d2-4d52-9751-079808ea167f-edpm-compute-no-nodes\") on node \"crc\" DevicePath \"\"" Dec 03 13:05:05 crc kubenswrapper[5052]: I1203 13:05:05.645865 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-4s9xh" event={"ID":"030b2288-f819-48fd-91a2-e08b1130e648","Type":"ContainerStarted","Data":"307141c01a745c9de3c9354a528ed4b1de125eb377db1905522e2c657216d4ef"} Dec 03 13:05:05 crc kubenswrapper[5052]: I1203 13:05:05.646088 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-4s9xh" Dec 03 13:05:05 crc kubenswrapper[5052]: I1203 13:05:05.668264 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-4s9xh" podStartSLOduration=3.668243184 podStartE2EDuration="3.668243184s" podCreationTimestamp="2025-12-03 13:05:02 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 13:05:05.665574168 +0000 UTC m=+6296.253815932" watchObservedRunningTime="2025-12-03 13:05:05.668243184 +0000 UTC m=+6296.256484948" Dec 03 13:05:05 crc kubenswrapper[5052]: I1203 13:05:05.682848 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="549519fd-33d2-4d52-9751-079808ea167f" path="/var/lib/kubelet/pods/549519fd-33d2-4d52-9751-079808ea167f/volumes" Dec 03 13:05:07 crc kubenswrapper[5052]: I1203 13:05:07.289086 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openshift-marketplace/community-operators-5v5wb" Dec 03 13:05:07 crc kubenswrapper[5052]: I1203 13:05:07.289800 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-marketplace/community-operators-5v5wb" Dec 03 13:05:07 crc kubenswrapper[5052]: I1203 13:05:07.337192 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-marketplace/community-operators-5v5wb" Dec 03 13:05:07 crc kubenswrapper[5052]: I1203 13:05:07.704322 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-marketplace/community-operators-5v5wb" Dec 03 13:05:07 crc kubenswrapper[5052]: I1203 13:05:07.755997 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openshift-marketplace/community-operators-5v5wb"] Dec 03 13:05:09 crc kubenswrapper[5052]: I1203 13:05:09.700029 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openshift-marketplace/community-operators-5v5wb" podUID="271ab8d3-4460-4b19-82c1-e75a4fc488aa" containerName="registry-server" containerID="cri-o://1389f06196ce48311cde262c298f4468cd48b7d3b3e406e9a3dcc008f6f715a3" gracePeriod=2 Dec 03 13:05:10 crc kubenswrapper[5052]: I1203 13:05:10.126442 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/community-operators-5v5wb" Dec 03 13:05:10 crc kubenswrapper[5052]: I1203 13:05:10.228062 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7x59q\" (UniqueName: \"kubernetes.io/projected/271ab8d3-4460-4b19-82c1-e75a4fc488aa-kube-api-access-7x59q\") pod \"271ab8d3-4460-4b19-82c1-e75a4fc488aa\" (UID: \"271ab8d3-4460-4b19-82c1-e75a4fc488aa\") " Dec 03 13:05:10 crc kubenswrapper[5052]: I1203 13:05:10.228163 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/271ab8d3-4460-4b19-82c1-e75a4fc488aa-catalog-content\") pod \"271ab8d3-4460-4b19-82c1-e75a4fc488aa\" (UID: \"271ab8d3-4460-4b19-82c1-e75a4fc488aa\") " Dec 03 13:05:10 crc kubenswrapper[5052]: I1203 13:05:10.228331 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/271ab8d3-4460-4b19-82c1-e75a4fc488aa-utilities\") pod \"271ab8d3-4460-4b19-82c1-e75a4fc488aa\" (UID: \"271ab8d3-4460-4b19-82c1-e75a4fc488aa\") " Dec 03 13:05:10 crc kubenswrapper[5052]: I1203 13:05:10.229507 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/271ab8d3-4460-4b19-82c1-e75a4fc488aa-utilities" (OuterVolumeSpecName: "utilities") pod "271ab8d3-4460-4b19-82c1-e75a4fc488aa" (UID: "271ab8d3-4460-4b19-82c1-e75a4fc488aa"). InnerVolumeSpecName "utilities". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 13:05:10 crc kubenswrapper[5052]: I1203 13:05:10.237482 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/271ab8d3-4460-4b19-82c1-e75a4fc488aa-kube-api-access-7x59q" (OuterVolumeSpecName: "kube-api-access-7x59q") pod "271ab8d3-4460-4b19-82c1-e75a4fc488aa" (UID: "271ab8d3-4460-4b19-82c1-e75a4fc488aa"). InnerVolumeSpecName "kube-api-access-7x59q". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:05:10 crc kubenswrapper[5052]: I1203 13:05:10.289587 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/271ab8d3-4460-4b19-82c1-e75a4fc488aa-catalog-content" (OuterVolumeSpecName: "catalog-content") pod "271ab8d3-4460-4b19-82c1-e75a4fc488aa" (UID: "271ab8d3-4460-4b19-82c1-e75a4fc488aa"). InnerVolumeSpecName "catalog-content". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 13:05:10 crc kubenswrapper[5052]: I1203 13:05:10.330536 5052 reconciler_common.go:293] "Volume detached for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/271ab8d3-4460-4b19-82c1-e75a4fc488aa-utilities\") on node \"crc\" DevicePath \"\"" Dec 03 13:05:10 crc kubenswrapper[5052]: I1203 13:05:10.330584 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-7x59q\" (UniqueName: \"kubernetes.io/projected/271ab8d3-4460-4b19-82c1-e75a4fc488aa-kube-api-access-7x59q\") on node \"crc\" DevicePath \"\"" Dec 03 13:05:10 crc kubenswrapper[5052]: I1203 13:05:10.330599 5052 reconciler_common.go:293] "Volume detached for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/271ab8d3-4460-4b19-82c1-e75a4fc488aa-catalog-content\") on node \"crc\" DevicePath \"\"" Dec 03 13:05:10 crc kubenswrapper[5052]: I1203 13:05:10.714138 5052 generic.go:334] "Generic (PLEG): container finished" podID="271ab8d3-4460-4b19-82c1-e75a4fc488aa" containerID="1389f06196ce48311cde262c298f4468cd48b7d3b3e406e9a3dcc008f6f715a3" exitCode=0 Dec 03 13:05:10 crc kubenswrapper[5052]: I1203 13:05:10.714206 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/community-operators-5v5wb" Dec 03 13:05:10 crc kubenswrapper[5052]: I1203 13:05:10.714203 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/community-operators-5v5wb" event={"ID":"271ab8d3-4460-4b19-82c1-e75a4fc488aa","Type":"ContainerDied","Data":"1389f06196ce48311cde262c298f4468cd48b7d3b3e406e9a3dcc008f6f715a3"} Dec 03 13:05:10 crc kubenswrapper[5052]: I1203 13:05:10.714286 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/community-operators-5v5wb" event={"ID":"271ab8d3-4460-4b19-82c1-e75a4fc488aa","Type":"ContainerDied","Data":"7d26e30479b239428368e5b459de9af8f3f30ad038485f5d307ccbed3b4fb9ff"} Dec 03 13:05:10 crc kubenswrapper[5052]: I1203 13:05:10.714315 5052 scope.go:117] "RemoveContainer" containerID="1389f06196ce48311cde262c298f4468cd48b7d3b3e406e9a3dcc008f6f715a3" Dec 03 13:05:10 crc kubenswrapper[5052]: I1203 13:05:10.742154 5052 scope.go:117] "RemoveContainer" containerID="9695bf67bf79d16d036f595c37796668bbc491961ac5c497f389fe969c830b2b" Dec 03 13:05:10 crc kubenswrapper[5052]: I1203 13:05:10.753282 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openshift-marketplace/community-operators-5v5wb"] Dec 03 13:05:10 crc kubenswrapper[5052]: I1203 13:05:10.761658 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openshift-marketplace/community-operators-5v5wb"] Dec 03 13:05:10 crc kubenswrapper[5052]: I1203 13:05:10.780111 5052 scope.go:117] "RemoveContainer" containerID="b3aeab68a5913163936ab01d8f628039248b9f5339d79e193fe8dbc90c820672" Dec 03 13:05:10 crc kubenswrapper[5052]: I1203 13:05:10.806079 5052 scope.go:117] "RemoveContainer" containerID="1389f06196ce48311cde262c298f4468cd48b7d3b3e406e9a3dcc008f6f715a3" Dec 03 13:05:10 crc kubenswrapper[5052]: E1203 13:05:10.806473 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1389f06196ce48311cde262c298f4468cd48b7d3b3e406e9a3dcc008f6f715a3\": container with ID starting with 1389f06196ce48311cde262c298f4468cd48b7d3b3e406e9a3dcc008f6f715a3 not found: ID does not exist" containerID="1389f06196ce48311cde262c298f4468cd48b7d3b3e406e9a3dcc008f6f715a3" Dec 03 13:05:10 crc kubenswrapper[5052]: I1203 13:05:10.806498 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1389f06196ce48311cde262c298f4468cd48b7d3b3e406e9a3dcc008f6f715a3"} err="failed to get container status \"1389f06196ce48311cde262c298f4468cd48b7d3b3e406e9a3dcc008f6f715a3\": rpc error: code = NotFound desc = could not find container \"1389f06196ce48311cde262c298f4468cd48b7d3b3e406e9a3dcc008f6f715a3\": container with ID starting with 1389f06196ce48311cde262c298f4468cd48b7d3b3e406e9a3dcc008f6f715a3 not found: ID does not exist" Dec 03 13:05:10 crc kubenswrapper[5052]: I1203 13:05:10.806585 5052 scope.go:117] "RemoveContainer" containerID="9695bf67bf79d16d036f595c37796668bbc491961ac5c497f389fe969c830b2b" Dec 03 13:05:10 crc kubenswrapper[5052]: E1203 13:05:10.806817 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9695bf67bf79d16d036f595c37796668bbc491961ac5c497f389fe969c830b2b\": container with ID starting with 9695bf67bf79d16d036f595c37796668bbc491961ac5c497f389fe969c830b2b not found: ID does not exist" containerID="9695bf67bf79d16d036f595c37796668bbc491961ac5c497f389fe969c830b2b" Dec 03 13:05:10 crc kubenswrapper[5052]: I1203 13:05:10.806854 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9695bf67bf79d16d036f595c37796668bbc491961ac5c497f389fe969c830b2b"} err="failed to get container status \"9695bf67bf79d16d036f595c37796668bbc491961ac5c497f389fe969c830b2b\": rpc error: code = NotFound desc = could not find container \"9695bf67bf79d16d036f595c37796668bbc491961ac5c497f389fe969c830b2b\": container with ID starting with 9695bf67bf79d16d036f595c37796668bbc491961ac5c497f389fe969c830b2b not found: ID does not exist" Dec 03 13:05:10 crc kubenswrapper[5052]: I1203 13:05:10.806867 5052 scope.go:117] "RemoveContainer" containerID="b3aeab68a5913163936ab01d8f628039248b9f5339d79e193fe8dbc90c820672" Dec 03 13:05:10 crc kubenswrapper[5052]: E1203 13:05:10.807107 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b3aeab68a5913163936ab01d8f628039248b9f5339d79e193fe8dbc90c820672\": container with ID starting with b3aeab68a5913163936ab01d8f628039248b9f5339d79e193fe8dbc90c820672 not found: ID does not exist" containerID="b3aeab68a5913163936ab01d8f628039248b9f5339d79e193fe8dbc90c820672" Dec 03 13:05:10 crc kubenswrapper[5052]: I1203 13:05:10.807128 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b3aeab68a5913163936ab01d8f628039248b9f5339d79e193fe8dbc90c820672"} err="failed to get container status \"b3aeab68a5913163936ab01d8f628039248b9f5339d79e193fe8dbc90c820672\": rpc error: code = NotFound desc = could not find container \"b3aeab68a5913163936ab01d8f628039248b9f5339d79e193fe8dbc90c820672\": container with ID starting with b3aeab68a5913163936ab01d8f628039248b9f5339d79e193fe8dbc90c820672 not found: ID does not exist" Dec 03 13:05:11 crc kubenswrapper[5052]: I1203 13:05:11.080187 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["crc-storage/crc-storage-crc-f8jwz"] Dec 03 13:05:11 crc kubenswrapper[5052]: I1203 13:05:11.087087 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["crc-storage/crc-storage-crc-f8jwz"] Dec 03 13:05:11 crc kubenswrapper[5052]: I1203 13:05:11.248579 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["crc-storage/crc-storage-crc-k6vh5"] Dec 03 13:05:11 crc kubenswrapper[5052]: E1203 13:05:11.248979 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="271ab8d3-4460-4b19-82c1-e75a4fc488aa" containerName="registry-server" Dec 03 13:05:11 crc kubenswrapper[5052]: I1203 13:05:11.248999 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="271ab8d3-4460-4b19-82c1-e75a4fc488aa" containerName="registry-server" Dec 03 13:05:11 crc kubenswrapper[5052]: E1203 13:05:11.249026 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="271ab8d3-4460-4b19-82c1-e75a4fc488aa" containerName="extract-utilities" Dec 03 13:05:11 crc kubenswrapper[5052]: I1203 13:05:11.249035 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="271ab8d3-4460-4b19-82c1-e75a4fc488aa" containerName="extract-utilities" Dec 03 13:05:11 crc kubenswrapper[5052]: E1203 13:05:11.249054 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="271ab8d3-4460-4b19-82c1-e75a4fc488aa" containerName="extract-content" Dec 03 13:05:11 crc kubenswrapper[5052]: I1203 13:05:11.249062 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="271ab8d3-4460-4b19-82c1-e75a4fc488aa" containerName="extract-content" Dec 03 13:05:11 crc kubenswrapper[5052]: I1203 13:05:11.249232 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="271ab8d3-4460-4b19-82c1-e75a4fc488aa" containerName="registry-server" Dec 03 13:05:11 crc kubenswrapper[5052]: I1203 13:05:11.249950 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="crc-storage/crc-storage-crc-k6vh5" Dec 03 13:05:11 crc kubenswrapper[5052]: I1203 13:05:11.252634 5052 reflector.go:368] Caches populated for *v1.Secret from object-"crc-storage"/"crc-storage-dockercfg-mf76g" Dec 03 13:05:11 crc kubenswrapper[5052]: I1203 13:05:11.252999 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"crc-storage"/"crc-storage" Dec 03 13:05:11 crc kubenswrapper[5052]: I1203 13:05:11.254795 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"crc-storage"/"openshift-service-ca.crt" Dec 03 13:05:11 crc kubenswrapper[5052]: I1203 13:05:11.255152 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"crc-storage"/"kube-root-ca.crt" Dec 03 13:05:11 crc kubenswrapper[5052]: I1203 13:05:11.257799 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["crc-storage/crc-storage-crc-k6vh5"] Dec 03 13:05:11 crc kubenswrapper[5052]: I1203 13:05:11.344728 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dlvz8\" (UniqueName: \"kubernetes.io/projected/7c9645a0-5703-4231-b9e9-2d0be6b0eec5-kube-api-access-dlvz8\") pod \"crc-storage-crc-k6vh5\" (UID: \"7c9645a0-5703-4231-b9e9-2d0be6b0eec5\") " pod="crc-storage/crc-storage-crc-k6vh5" Dec 03 13:05:11 crc kubenswrapper[5052]: I1203 13:05:11.344947 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/7c9645a0-5703-4231-b9e9-2d0be6b0eec5-node-mnt\") pod \"crc-storage-crc-k6vh5\" (UID: \"7c9645a0-5703-4231-b9e9-2d0be6b0eec5\") " pod="crc-storage/crc-storage-crc-k6vh5" Dec 03 13:05:11 crc kubenswrapper[5052]: I1203 13:05:11.345137 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/7c9645a0-5703-4231-b9e9-2d0be6b0eec5-crc-storage\") pod \"crc-storage-crc-k6vh5\" (UID: \"7c9645a0-5703-4231-b9e9-2d0be6b0eec5\") " pod="crc-storage/crc-storage-crc-k6vh5" Dec 03 13:05:11 crc kubenswrapper[5052]: I1203 13:05:11.446491 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-dlvz8\" (UniqueName: \"kubernetes.io/projected/7c9645a0-5703-4231-b9e9-2d0be6b0eec5-kube-api-access-dlvz8\") pod \"crc-storage-crc-k6vh5\" (UID: \"7c9645a0-5703-4231-b9e9-2d0be6b0eec5\") " pod="crc-storage/crc-storage-crc-k6vh5" Dec 03 13:05:11 crc kubenswrapper[5052]: I1203 13:05:11.446957 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/7c9645a0-5703-4231-b9e9-2d0be6b0eec5-node-mnt\") pod \"crc-storage-crc-k6vh5\" (UID: \"7c9645a0-5703-4231-b9e9-2d0be6b0eec5\") " pod="crc-storage/crc-storage-crc-k6vh5" Dec 03 13:05:11 crc kubenswrapper[5052]: I1203 13:05:11.447248 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/7c9645a0-5703-4231-b9e9-2d0be6b0eec5-node-mnt\") pod \"crc-storage-crc-k6vh5\" (UID: \"7c9645a0-5703-4231-b9e9-2d0be6b0eec5\") " pod="crc-storage/crc-storage-crc-k6vh5" Dec 03 13:05:11 crc kubenswrapper[5052]: I1203 13:05:11.447311 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/7c9645a0-5703-4231-b9e9-2d0be6b0eec5-crc-storage\") pod \"crc-storage-crc-k6vh5\" (UID: \"7c9645a0-5703-4231-b9e9-2d0be6b0eec5\") " pod="crc-storage/crc-storage-crc-k6vh5" Dec 03 13:05:11 crc kubenswrapper[5052]: I1203 13:05:11.447953 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/7c9645a0-5703-4231-b9e9-2d0be6b0eec5-crc-storage\") pod \"crc-storage-crc-k6vh5\" (UID: \"7c9645a0-5703-4231-b9e9-2d0be6b0eec5\") " pod="crc-storage/crc-storage-crc-k6vh5" Dec 03 13:05:11 crc kubenswrapper[5052]: I1203 13:05:11.464823 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-dlvz8\" (UniqueName: \"kubernetes.io/projected/7c9645a0-5703-4231-b9e9-2d0be6b0eec5-kube-api-access-dlvz8\") pod \"crc-storage-crc-k6vh5\" (UID: \"7c9645a0-5703-4231-b9e9-2d0be6b0eec5\") " pod="crc-storage/crc-storage-crc-k6vh5" Dec 03 13:05:11 crc kubenswrapper[5052]: I1203 13:05:11.566862 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="crc-storage/crc-storage-crc-k6vh5" Dec 03 13:05:11 crc kubenswrapper[5052]: I1203 13:05:11.730542 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="271ab8d3-4460-4b19-82c1-e75a4fc488aa" path="/var/lib/kubelet/pods/271ab8d3-4460-4b19-82c1-e75a4fc488aa/volumes" Dec 03 13:05:11 crc kubenswrapper[5052]: I1203 13:05:11.731396 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e7db9821-37e8-440b-961b-2191b4f548ce" path="/var/lib/kubelet/pods/e7db9821-37e8-440b-961b-2191b4f548ce/volumes" Dec 03 13:05:12 crc kubenswrapper[5052]: I1203 13:05:12.151041 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["crc-storage/crc-storage-crc-k6vh5"] Dec 03 13:05:12 crc kubenswrapper[5052]: I1203 13:05:12.749579 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="crc-storage/crc-storage-crc-k6vh5" event={"ID":"7c9645a0-5703-4231-b9e9-2d0be6b0eec5","Type":"ContainerStarted","Data":"2ce67803c95bbdf7754887845a4782acd00127cf82857340aac4341b1139660e"} Dec 03 13:05:13 crc kubenswrapper[5052]: I1203 13:05:13.601459 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-4s9xh" Dec 03 13:05:13 crc kubenswrapper[5052]: I1203 13:05:13.664620 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-67886899f9-xgth2"] Dec 03 13:05:13 crc kubenswrapper[5052]: I1203 13:05:13.665159 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-67886899f9-xgth2" podUID="c70a070b-903b-4c81-b43a-e129647efea7" containerName="dnsmasq-dns" containerID="cri-o://db6c7734ee1af3a31951a50895f1410bc81d07a980f8bb934b75e98b775aa33f" gracePeriod=10 Dec 03 13:05:13 crc kubenswrapper[5052]: I1203 13:05:13.773985 5052 generic.go:334] "Generic (PLEG): container finished" podID="7c9645a0-5703-4231-b9e9-2d0be6b0eec5" containerID="72c60c8ec14bec08d197cc124437fb923cfe931408d815351ba64acc2169da26" exitCode=0 Dec 03 13:05:13 crc kubenswrapper[5052]: I1203 13:05:13.774038 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="crc-storage/crc-storage-crc-k6vh5" event={"ID":"7c9645a0-5703-4231-b9e9-2d0be6b0eec5","Type":"ContainerDied","Data":"72c60c8ec14bec08d197cc124437fb923cfe931408d815351ba64acc2169da26"} Dec 03 13:05:14 crc kubenswrapper[5052]: I1203 13:05:14.065911 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-67886899f9-xgth2" Dec 03 13:05:14 crc kubenswrapper[5052]: I1203 13:05:14.243600 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t97q4\" (UniqueName: \"kubernetes.io/projected/c70a070b-903b-4c81-b43a-e129647efea7-kube-api-access-t97q4\") pod \"c70a070b-903b-4c81-b43a-e129647efea7\" (UID: \"c70a070b-903b-4c81-b43a-e129647efea7\") " Dec 03 13:05:14 crc kubenswrapper[5052]: I1203 13:05:14.243678 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"edpm-compute-beta-nodeset\" (UniqueName: \"kubernetes.io/configmap/c70a070b-903b-4c81-b43a-e129647efea7-edpm-compute-beta-nodeset\") pod \"c70a070b-903b-4c81-b43a-e129647efea7\" (UID: \"c70a070b-903b-4c81-b43a-e129647efea7\") " Dec 03 13:05:14 crc kubenswrapper[5052]: I1203 13:05:14.243706 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"edpm-compute-no-nodes\" (UniqueName: \"kubernetes.io/configmap/c70a070b-903b-4c81-b43a-e129647efea7-edpm-compute-no-nodes\") pod \"c70a070b-903b-4c81-b43a-e129647efea7\" (UID: \"c70a070b-903b-4c81-b43a-e129647efea7\") " Dec 03 13:05:14 crc kubenswrapper[5052]: I1203 13:05:14.243730 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/c70a070b-903b-4c81-b43a-e129647efea7-dnsmasq-svc\") pod \"c70a070b-903b-4c81-b43a-e129647efea7\" (UID: \"c70a070b-903b-4c81-b43a-e129647efea7\") " Dec 03 13:05:14 crc kubenswrapper[5052]: I1203 13:05:14.243805 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c70a070b-903b-4c81-b43a-e129647efea7-config\") pod \"c70a070b-903b-4c81-b43a-e129647efea7\" (UID: \"c70a070b-903b-4c81-b43a-e129647efea7\") " Dec 03 13:05:14 crc kubenswrapper[5052]: I1203 13:05:14.249880 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c70a070b-903b-4c81-b43a-e129647efea7-kube-api-access-t97q4" (OuterVolumeSpecName: "kube-api-access-t97q4") pod "c70a070b-903b-4c81-b43a-e129647efea7" (UID: "c70a070b-903b-4c81-b43a-e129647efea7"). InnerVolumeSpecName "kube-api-access-t97q4". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:05:14 crc kubenswrapper[5052]: I1203 13:05:14.286621 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c70a070b-903b-4c81-b43a-e129647efea7-edpm-compute-beta-nodeset" (OuterVolumeSpecName: "edpm-compute-beta-nodeset") pod "c70a070b-903b-4c81-b43a-e129647efea7" (UID: "c70a070b-903b-4c81-b43a-e129647efea7"). InnerVolumeSpecName "edpm-compute-beta-nodeset". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:05:14 crc kubenswrapper[5052]: I1203 13:05:14.287606 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c70a070b-903b-4c81-b43a-e129647efea7-edpm-compute-no-nodes" (OuterVolumeSpecName: "edpm-compute-no-nodes") pod "c70a070b-903b-4c81-b43a-e129647efea7" (UID: "c70a070b-903b-4c81-b43a-e129647efea7"). InnerVolumeSpecName "edpm-compute-no-nodes". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:05:14 crc kubenswrapper[5052]: I1203 13:05:14.289616 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c70a070b-903b-4c81-b43a-e129647efea7-dnsmasq-svc" (OuterVolumeSpecName: "dnsmasq-svc") pod "c70a070b-903b-4c81-b43a-e129647efea7" (UID: "c70a070b-903b-4c81-b43a-e129647efea7"). InnerVolumeSpecName "dnsmasq-svc". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:05:14 crc kubenswrapper[5052]: I1203 13:05:14.292213 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c70a070b-903b-4c81-b43a-e129647efea7-config" (OuterVolumeSpecName: "config") pod "c70a070b-903b-4c81-b43a-e129647efea7" (UID: "c70a070b-903b-4c81-b43a-e129647efea7"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:05:14 crc kubenswrapper[5052]: I1203 13:05:14.346192 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-t97q4\" (UniqueName: \"kubernetes.io/projected/c70a070b-903b-4c81-b43a-e129647efea7-kube-api-access-t97q4\") on node \"crc\" DevicePath \"\"" Dec 03 13:05:14 crc kubenswrapper[5052]: I1203 13:05:14.346493 5052 reconciler_common.go:293] "Volume detached for volume \"edpm-compute-beta-nodeset\" (UniqueName: \"kubernetes.io/configmap/c70a070b-903b-4c81-b43a-e129647efea7-edpm-compute-beta-nodeset\") on node \"crc\" DevicePath \"\"" Dec 03 13:05:14 crc kubenswrapper[5052]: I1203 13:05:14.346623 5052 reconciler_common.go:293] "Volume detached for volume \"edpm-compute-no-nodes\" (UniqueName: \"kubernetes.io/configmap/c70a070b-903b-4c81-b43a-e129647efea7-edpm-compute-no-nodes\") on node \"crc\" DevicePath \"\"" Dec 03 13:05:14 crc kubenswrapper[5052]: I1203 13:05:14.346718 5052 reconciler_common.go:293] "Volume detached for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/c70a070b-903b-4c81-b43a-e129647efea7-dnsmasq-svc\") on node \"crc\" DevicePath \"\"" Dec 03 13:05:14 crc kubenswrapper[5052]: I1203 13:05:14.346788 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c70a070b-903b-4c81-b43a-e129647efea7-config\") on node \"crc\" DevicePath \"\"" Dec 03 13:05:14 crc kubenswrapper[5052]: I1203 13:05:14.783881 5052 generic.go:334] "Generic (PLEG): container finished" podID="c70a070b-903b-4c81-b43a-e129647efea7" containerID="db6c7734ee1af3a31951a50895f1410bc81d07a980f8bb934b75e98b775aa33f" exitCode=0 Dec 03 13:05:14 crc kubenswrapper[5052]: I1203 13:05:14.783982 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-67886899f9-xgth2" event={"ID":"c70a070b-903b-4c81-b43a-e129647efea7","Type":"ContainerDied","Data":"db6c7734ee1af3a31951a50895f1410bc81d07a980f8bb934b75e98b775aa33f"} Dec 03 13:05:14 crc kubenswrapper[5052]: I1203 13:05:14.783993 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-67886899f9-xgth2" Dec 03 13:05:14 crc kubenswrapper[5052]: I1203 13:05:14.784036 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-67886899f9-xgth2" event={"ID":"c70a070b-903b-4c81-b43a-e129647efea7","Type":"ContainerDied","Data":"ef52a8ef8e7ebbe9f4814fc966418199c2f75b2d505b6b1be94f319da8d194d4"} Dec 03 13:05:14 crc kubenswrapper[5052]: I1203 13:05:14.784048 5052 scope.go:117] "RemoveContainer" containerID="db6c7734ee1af3a31951a50895f1410bc81d07a980f8bb934b75e98b775aa33f" Dec 03 13:05:14 crc kubenswrapper[5052]: I1203 13:05:14.828621 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-67886899f9-xgth2"] Dec 03 13:05:14 crc kubenswrapper[5052]: I1203 13:05:14.842480 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-67886899f9-xgth2"] Dec 03 13:05:14 crc kubenswrapper[5052]: I1203 13:05:14.848099 5052 scope.go:117] "RemoveContainer" containerID="d631e8d62045cb6cb73a9f326f7b93e176e0760e4fc5e3871604750f68942881" Dec 03 13:05:14 crc kubenswrapper[5052]: I1203 13:05:14.880464 5052 scope.go:117] "RemoveContainer" containerID="db6c7734ee1af3a31951a50895f1410bc81d07a980f8bb934b75e98b775aa33f" Dec 03 13:05:14 crc kubenswrapper[5052]: E1203 13:05:14.884733 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"db6c7734ee1af3a31951a50895f1410bc81d07a980f8bb934b75e98b775aa33f\": container with ID starting with db6c7734ee1af3a31951a50895f1410bc81d07a980f8bb934b75e98b775aa33f not found: ID does not exist" containerID="db6c7734ee1af3a31951a50895f1410bc81d07a980f8bb934b75e98b775aa33f" Dec 03 13:05:14 crc kubenswrapper[5052]: I1203 13:05:14.884790 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"db6c7734ee1af3a31951a50895f1410bc81d07a980f8bb934b75e98b775aa33f"} err="failed to get container status \"db6c7734ee1af3a31951a50895f1410bc81d07a980f8bb934b75e98b775aa33f\": rpc error: code = NotFound desc = could not find container \"db6c7734ee1af3a31951a50895f1410bc81d07a980f8bb934b75e98b775aa33f\": container with ID starting with db6c7734ee1af3a31951a50895f1410bc81d07a980f8bb934b75e98b775aa33f not found: ID does not exist" Dec 03 13:05:14 crc kubenswrapper[5052]: I1203 13:05:14.884827 5052 scope.go:117] "RemoveContainer" containerID="d631e8d62045cb6cb73a9f326f7b93e176e0760e4fc5e3871604750f68942881" Dec 03 13:05:14 crc kubenswrapper[5052]: E1203 13:05:14.885425 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d631e8d62045cb6cb73a9f326f7b93e176e0760e4fc5e3871604750f68942881\": container with ID starting with d631e8d62045cb6cb73a9f326f7b93e176e0760e4fc5e3871604750f68942881 not found: ID does not exist" containerID="d631e8d62045cb6cb73a9f326f7b93e176e0760e4fc5e3871604750f68942881" Dec 03 13:05:14 crc kubenswrapper[5052]: I1203 13:05:14.885454 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d631e8d62045cb6cb73a9f326f7b93e176e0760e4fc5e3871604750f68942881"} err="failed to get container status \"d631e8d62045cb6cb73a9f326f7b93e176e0760e4fc5e3871604750f68942881\": rpc error: code = NotFound desc = could not find container \"d631e8d62045cb6cb73a9f326f7b93e176e0760e4fc5e3871604750f68942881\": container with ID starting with d631e8d62045cb6cb73a9f326f7b93e176e0760e4fc5e3871604750f68942881 not found: ID does not exist" Dec 03 13:05:15 crc kubenswrapper[5052]: I1203 13:05:15.134771 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="crc-storage/crc-storage-crc-k6vh5" Dec 03 13:05:15 crc kubenswrapper[5052]: I1203 13:05:15.265799 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/7c9645a0-5703-4231-b9e9-2d0be6b0eec5-node-mnt\") pod \"7c9645a0-5703-4231-b9e9-2d0be6b0eec5\" (UID: \"7c9645a0-5703-4231-b9e9-2d0be6b0eec5\") " Dec 03 13:05:15 crc kubenswrapper[5052]: I1203 13:05:15.265854 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/7c9645a0-5703-4231-b9e9-2d0be6b0eec5-crc-storage\") pod \"7c9645a0-5703-4231-b9e9-2d0be6b0eec5\" (UID: \"7c9645a0-5703-4231-b9e9-2d0be6b0eec5\") " Dec 03 13:05:15 crc kubenswrapper[5052]: I1203 13:05:15.265919 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/7c9645a0-5703-4231-b9e9-2d0be6b0eec5-node-mnt" (OuterVolumeSpecName: "node-mnt") pod "7c9645a0-5703-4231-b9e9-2d0be6b0eec5" (UID: "7c9645a0-5703-4231-b9e9-2d0be6b0eec5"). InnerVolumeSpecName "node-mnt". PluginName "kubernetes.io/host-path", VolumeGidValue "" Dec 03 13:05:15 crc kubenswrapper[5052]: I1203 13:05:15.265957 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dlvz8\" (UniqueName: \"kubernetes.io/projected/7c9645a0-5703-4231-b9e9-2d0be6b0eec5-kube-api-access-dlvz8\") pod \"7c9645a0-5703-4231-b9e9-2d0be6b0eec5\" (UID: \"7c9645a0-5703-4231-b9e9-2d0be6b0eec5\") " Dec 03 13:05:15 crc kubenswrapper[5052]: I1203 13:05:15.266422 5052 reconciler_common.go:293] "Volume detached for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/7c9645a0-5703-4231-b9e9-2d0be6b0eec5-node-mnt\") on node \"crc\" DevicePath \"\"" Dec 03 13:05:15 crc kubenswrapper[5052]: I1203 13:05:15.270718 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7c9645a0-5703-4231-b9e9-2d0be6b0eec5-kube-api-access-dlvz8" (OuterVolumeSpecName: "kube-api-access-dlvz8") pod "7c9645a0-5703-4231-b9e9-2d0be6b0eec5" (UID: "7c9645a0-5703-4231-b9e9-2d0be6b0eec5"). InnerVolumeSpecName "kube-api-access-dlvz8". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:05:15 crc kubenswrapper[5052]: I1203 13:05:15.287038 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7c9645a0-5703-4231-b9e9-2d0be6b0eec5-crc-storage" (OuterVolumeSpecName: "crc-storage") pod "7c9645a0-5703-4231-b9e9-2d0be6b0eec5" (UID: "7c9645a0-5703-4231-b9e9-2d0be6b0eec5"). InnerVolumeSpecName "crc-storage". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:05:15 crc kubenswrapper[5052]: I1203 13:05:15.368122 5052 reconciler_common.go:293] "Volume detached for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/7c9645a0-5703-4231-b9e9-2d0be6b0eec5-crc-storage\") on node \"crc\" DevicePath \"\"" Dec 03 13:05:15 crc kubenswrapper[5052]: I1203 13:05:15.368163 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-dlvz8\" (UniqueName: \"kubernetes.io/projected/7c9645a0-5703-4231-b9e9-2d0be6b0eec5-kube-api-access-dlvz8\") on node \"crc\" DevicePath \"\"" Dec 03 13:05:15 crc kubenswrapper[5052]: I1203 13:05:15.682044 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c70a070b-903b-4c81-b43a-e129647efea7" path="/var/lib/kubelet/pods/c70a070b-903b-4c81-b43a-e129647efea7/volumes" Dec 03 13:05:15 crc kubenswrapper[5052]: I1203 13:05:15.795278 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="crc-storage/crc-storage-crc-k6vh5" event={"ID":"7c9645a0-5703-4231-b9e9-2d0be6b0eec5","Type":"ContainerDied","Data":"2ce67803c95bbdf7754887845a4782acd00127cf82857340aac4341b1139660e"} Dec 03 13:05:15 crc kubenswrapper[5052]: I1203 13:05:15.795319 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2ce67803c95bbdf7754887845a4782acd00127cf82857340aac4341b1139660e" Dec 03 13:05:15 crc kubenswrapper[5052]: I1203 13:05:15.796654 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="crc-storage/crc-storage-crc-k6vh5" Dec 03 13:05:19 crc kubenswrapper[5052]: I1203 13:05:19.041771 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["crc-storage/crc-storage-crc-k6vh5"] Dec 03 13:05:19 crc kubenswrapper[5052]: I1203 13:05:19.047632 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["crc-storage/crc-storage-crc-k6vh5"] Dec 03 13:05:19 crc kubenswrapper[5052]: I1203 13:05:19.223166 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["crc-storage/crc-storage-crc-skgwc"] Dec 03 13:05:19 crc kubenswrapper[5052]: E1203 13:05:19.223591 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="7c9645a0-5703-4231-b9e9-2d0be6b0eec5" containerName="storage" Dec 03 13:05:19 crc kubenswrapper[5052]: I1203 13:05:19.223614 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c9645a0-5703-4231-b9e9-2d0be6b0eec5" containerName="storage" Dec 03 13:05:19 crc kubenswrapper[5052]: E1203 13:05:19.223639 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="c70a070b-903b-4c81-b43a-e129647efea7" containerName="dnsmasq-dns" Dec 03 13:05:19 crc kubenswrapper[5052]: I1203 13:05:19.223647 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="c70a070b-903b-4c81-b43a-e129647efea7" containerName="dnsmasq-dns" Dec 03 13:05:19 crc kubenswrapper[5052]: E1203 13:05:19.223669 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="c70a070b-903b-4c81-b43a-e129647efea7" containerName="init" Dec 03 13:05:19 crc kubenswrapper[5052]: I1203 13:05:19.223679 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="c70a070b-903b-4c81-b43a-e129647efea7" containerName="init" Dec 03 13:05:19 crc kubenswrapper[5052]: I1203 13:05:19.223864 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="c70a070b-903b-4c81-b43a-e129647efea7" containerName="dnsmasq-dns" Dec 03 13:05:19 crc kubenswrapper[5052]: I1203 13:05:19.223884 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="7c9645a0-5703-4231-b9e9-2d0be6b0eec5" containerName="storage" Dec 03 13:05:19 crc kubenswrapper[5052]: I1203 13:05:19.225071 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="crc-storage/crc-storage-crc-skgwc" Dec 03 13:05:19 crc kubenswrapper[5052]: I1203 13:05:19.227327 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"crc-storage"/"crc-storage" Dec 03 13:05:19 crc kubenswrapper[5052]: I1203 13:05:19.227773 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"crc-storage"/"openshift-service-ca.crt" Dec 03 13:05:19 crc kubenswrapper[5052]: I1203 13:05:19.227893 5052 reflector.go:368] Caches populated for *v1.Secret from object-"crc-storage"/"crc-storage-dockercfg-mf76g" Dec 03 13:05:19 crc kubenswrapper[5052]: I1203 13:05:19.228268 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"crc-storage"/"kube-root-ca.crt" Dec 03 13:05:19 crc kubenswrapper[5052]: I1203 13:05:19.231145 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["crc-storage/crc-storage-crc-skgwc"] Dec 03 13:05:19 crc kubenswrapper[5052]: I1203 13:05:19.362369 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/39a41b2f-5948-4e46-88d5-cf7a6364a8e8-crc-storage\") pod \"crc-storage-crc-skgwc\" (UID: \"39a41b2f-5948-4e46-88d5-cf7a6364a8e8\") " pod="crc-storage/crc-storage-crc-skgwc" Dec 03 13:05:19 crc kubenswrapper[5052]: I1203 13:05:19.362476 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/39a41b2f-5948-4e46-88d5-cf7a6364a8e8-node-mnt\") pod \"crc-storage-crc-skgwc\" (UID: \"39a41b2f-5948-4e46-88d5-cf7a6364a8e8\") " pod="crc-storage/crc-storage-crc-skgwc" Dec 03 13:05:19 crc kubenswrapper[5052]: I1203 13:05:19.362519 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d8b7k\" (UniqueName: \"kubernetes.io/projected/39a41b2f-5948-4e46-88d5-cf7a6364a8e8-kube-api-access-d8b7k\") pod \"crc-storage-crc-skgwc\" (UID: \"39a41b2f-5948-4e46-88d5-cf7a6364a8e8\") " pod="crc-storage/crc-storage-crc-skgwc" Dec 03 13:05:19 crc kubenswrapper[5052]: I1203 13:05:19.464663 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/39a41b2f-5948-4e46-88d5-cf7a6364a8e8-node-mnt\") pod \"crc-storage-crc-skgwc\" (UID: \"39a41b2f-5948-4e46-88d5-cf7a6364a8e8\") " pod="crc-storage/crc-storage-crc-skgwc" Dec 03 13:05:19 crc kubenswrapper[5052]: I1203 13:05:19.464713 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-d8b7k\" (UniqueName: \"kubernetes.io/projected/39a41b2f-5948-4e46-88d5-cf7a6364a8e8-kube-api-access-d8b7k\") pod \"crc-storage-crc-skgwc\" (UID: \"39a41b2f-5948-4e46-88d5-cf7a6364a8e8\") " pod="crc-storage/crc-storage-crc-skgwc" Dec 03 13:05:19 crc kubenswrapper[5052]: I1203 13:05:19.464767 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/39a41b2f-5948-4e46-88d5-cf7a6364a8e8-crc-storage\") pod \"crc-storage-crc-skgwc\" (UID: \"39a41b2f-5948-4e46-88d5-cf7a6364a8e8\") " pod="crc-storage/crc-storage-crc-skgwc" Dec 03 13:05:19 crc kubenswrapper[5052]: I1203 13:05:19.465442 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/39a41b2f-5948-4e46-88d5-cf7a6364a8e8-crc-storage\") pod \"crc-storage-crc-skgwc\" (UID: \"39a41b2f-5948-4e46-88d5-cf7a6364a8e8\") " pod="crc-storage/crc-storage-crc-skgwc" Dec 03 13:05:19 crc kubenswrapper[5052]: I1203 13:05:19.465868 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/39a41b2f-5948-4e46-88d5-cf7a6364a8e8-node-mnt\") pod \"crc-storage-crc-skgwc\" (UID: \"39a41b2f-5948-4e46-88d5-cf7a6364a8e8\") " pod="crc-storage/crc-storage-crc-skgwc" Dec 03 13:05:19 crc kubenswrapper[5052]: I1203 13:05:19.490445 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-d8b7k\" (UniqueName: \"kubernetes.io/projected/39a41b2f-5948-4e46-88d5-cf7a6364a8e8-kube-api-access-d8b7k\") pod \"crc-storage-crc-skgwc\" (UID: \"39a41b2f-5948-4e46-88d5-cf7a6364a8e8\") " pod="crc-storage/crc-storage-crc-skgwc" Dec 03 13:05:19 crc kubenswrapper[5052]: I1203 13:05:19.542588 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="crc-storage/crc-storage-crc-skgwc" Dec 03 13:05:19 crc kubenswrapper[5052]: I1203 13:05:19.692495 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7c9645a0-5703-4231-b9e9-2d0be6b0eec5" path="/var/lib/kubelet/pods/7c9645a0-5703-4231-b9e9-2d0be6b0eec5/volumes" Dec 03 13:05:19 crc kubenswrapper[5052]: I1203 13:05:19.986157 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["crc-storage/crc-storage-crc-skgwc"] Dec 03 13:05:19 crc kubenswrapper[5052]: W1203 13:05:19.988187 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod39a41b2f_5948_4e46_88d5_cf7a6364a8e8.slice/crio-7835a0a4b3bdcbe4b5aad1faa565f4f9262d15b97380f54dda9993fce628ceef WatchSource:0}: Error finding container 7835a0a4b3bdcbe4b5aad1faa565f4f9262d15b97380f54dda9993fce628ceef: Status 404 returned error can't find the container with id 7835a0a4b3bdcbe4b5aad1faa565f4f9262d15b97380f54dda9993fce628ceef Dec 03 13:05:20 crc kubenswrapper[5052]: I1203 13:05:20.850049 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="crc-storage/crc-storage-crc-skgwc" event={"ID":"39a41b2f-5948-4e46-88d5-cf7a6364a8e8","Type":"ContainerStarted","Data":"7835a0a4b3bdcbe4b5aad1faa565f4f9262d15b97380f54dda9993fce628ceef"} Dec 03 13:05:21 crc kubenswrapper[5052]: I1203 13:05:21.860012 5052 generic.go:334] "Generic (PLEG): container finished" podID="39a41b2f-5948-4e46-88d5-cf7a6364a8e8" containerID="b2159e0375630515208289b10bff2d1ff4c8f6302f4feeb712b83c151354ccde" exitCode=0 Dec 03 13:05:21 crc kubenswrapper[5052]: I1203 13:05:21.860215 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="crc-storage/crc-storage-crc-skgwc" event={"ID":"39a41b2f-5948-4e46-88d5-cf7a6364a8e8","Type":"ContainerDied","Data":"b2159e0375630515208289b10bff2d1ff4c8f6302f4feeb712b83c151354ccde"} Dec 03 13:05:23 crc kubenswrapper[5052]: I1203 13:05:23.144345 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="crc-storage/crc-storage-crc-skgwc" Dec 03 13:05:23 crc kubenswrapper[5052]: I1203 13:05:23.329786 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/39a41b2f-5948-4e46-88d5-cf7a6364a8e8-node-mnt\") pod \"39a41b2f-5948-4e46-88d5-cf7a6364a8e8\" (UID: \"39a41b2f-5948-4e46-88d5-cf7a6364a8e8\") " Dec 03 13:05:23 crc kubenswrapper[5052]: I1203 13:05:23.329856 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d8b7k\" (UniqueName: \"kubernetes.io/projected/39a41b2f-5948-4e46-88d5-cf7a6364a8e8-kube-api-access-d8b7k\") pod \"39a41b2f-5948-4e46-88d5-cf7a6364a8e8\" (UID: \"39a41b2f-5948-4e46-88d5-cf7a6364a8e8\") " Dec 03 13:05:23 crc kubenswrapper[5052]: I1203 13:05:23.329952 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/39a41b2f-5948-4e46-88d5-cf7a6364a8e8-crc-storage\") pod \"39a41b2f-5948-4e46-88d5-cf7a6364a8e8\" (UID: \"39a41b2f-5948-4e46-88d5-cf7a6364a8e8\") " Dec 03 13:05:23 crc kubenswrapper[5052]: I1203 13:05:23.329983 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/39a41b2f-5948-4e46-88d5-cf7a6364a8e8-node-mnt" (OuterVolumeSpecName: "node-mnt") pod "39a41b2f-5948-4e46-88d5-cf7a6364a8e8" (UID: "39a41b2f-5948-4e46-88d5-cf7a6364a8e8"). InnerVolumeSpecName "node-mnt". PluginName "kubernetes.io/host-path", VolumeGidValue "" Dec 03 13:05:23 crc kubenswrapper[5052]: I1203 13:05:23.330179 5052 reconciler_common.go:293] "Volume detached for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/39a41b2f-5948-4e46-88d5-cf7a6364a8e8-node-mnt\") on node \"crc\" DevicePath \"\"" Dec 03 13:05:23 crc kubenswrapper[5052]: I1203 13:05:23.335990 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/39a41b2f-5948-4e46-88d5-cf7a6364a8e8-kube-api-access-d8b7k" (OuterVolumeSpecName: "kube-api-access-d8b7k") pod "39a41b2f-5948-4e46-88d5-cf7a6364a8e8" (UID: "39a41b2f-5948-4e46-88d5-cf7a6364a8e8"). InnerVolumeSpecName "kube-api-access-d8b7k". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:05:23 crc kubenswrapper[5052]: I1203 13:05:23.366136 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/39a41b2f-5948-4e46-88d5-cf7a6364a8e8-crc-storage" (OuterVolumeSpecName: "crc-storage") pod "39a41b2f-5948-4e46-88d5-cf7a6364a8e8" (UID: "39a41b2f-5948-4e46-88d5-cf7a6364a8e8"). InnerVolumeSpecName "crc-storage". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:05:23 crc kubenswrapper[5052]: I1203 13:05:23.432181 5052 reconciler_common.go:293] "Volume detached for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/39a41b2f-5948-4e46-88d5-cf7a6364a8e8-crc-storage\") on node \"crc\" DevicePath \"\"" Dec 03 13:05:23 crc kubenswrapper[5052]: I1203 13:05:23.432316 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-d8b7k\" (UniqueName: \"kubernetes.io/projected/39a41b2f-5948-4e46-88d5-cf7a6364a8e8-kube-api-access-d8b7k\") on node \"crc\" DevicePath \"\"" Dec 03 13:05:23 crc kubenswrapper[5052]: I1203 13:05:23.882317 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="crc-storage/crc-storage-crc-skgwc" event={"ID":"39a41b2f-5948-4e46-88d5-cf7a6364a8e8","Type":"ContainerDied","Data":"7835a0a4b3bdcbe4b5aad1faa565f4f9262d15b97380f54dda9993fce628ceef"} Dec 03 13:05:23 crc kubenswrapper[5052]: I1203 13:05:23.882356 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="crc-storage/crc-storage-crc-skgwc" Dec 03 13:05:23 crc kubenswrapper[5052]: I1203 13:05:23.882368 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7835a0a4b3bdcbe4b5aad1faa565f4f9262d15b97380f54dda9993fce628ceef" Dec 03 13:05:26 crc kubenswrapper[5052]: I1203 13:05:26.641494 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-78c7b787f5-zr2sv"] Dec 03 13:05:26 crc kubenswrapper[5052]: E1203 13:05:26.642124 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="39a41b2f-5948-4e46-88d5-cf7a6364a8e8" containerName="storage" Dec 03 13:05:26 crc kubenswrapper[5052]: I1203 13:05:26.642137 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="39a41b2f-5948-4e46-88d5-cf7a6364a8e8" containerName="storage" Dec 03 13:05:26 crc kubenswrapper[5052]: I1203 13:05:26.642329 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="39a41b2f-5948-4e46-88d5-cf7a6364a8e8" containerName="storage" Dec 03 13:05:26 crc kubenswrapper[5052]: I1203 13:05:26.643161 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-78c7b787f5-zr2sv" Dec 03 13:05:26 crc kubenswrapper[5052]: I1203 13:05:26.644955 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"openstack-edpm-tls" Dec 03 13:05:26 crc kubenswrapper[5052]: I1203 13:05:26.667951 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-78c7b787f5-zr2sv"] Dec 03 13:05:26 crc kubenswrapper[5052]: I1203 13:05:26.712476 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-78c7b787f5-zr2sv"] Dec 03 13:05:26 crc kubenswrapper[5052]: E1203 13:05:26.712980 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[config dnsmasq-svc kube-api-access-pqkp9 openstack-edpm-tls], unattached volumes=[], failed to process volumes=[]: context canceled" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-78c7b787f5-zr2sv" podUID="0ba752ed-f76e-406d-a116-8f2dd724d5f2" Dec 03 13:05:26 crc kubenswrapper[5052]: I1203 13:05:26.747031 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-79cc674687-46mrq"] Dec 03 13:05:26 crc kubenswrapper[5052]: I1203 13:05:26.748417 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-79cc674687-46mrq" Dec 03 13:05:26 crc kubenswrapper[5052]: I1203 13:05:26.762125 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-79cc674687-46mrq"] Dec 03 13:05:26 crc kubenswrapper[5052]: I1203 13:05:26.782908 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/0ba752ed-f76e-406d-a116-8f2dd724d5f2-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-78c7b787f5-zr2sv\" (UID: \"0ba752ed-f76e-406d-a116-8f2dd724d5f2\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-78c7b787f5-zr2sv" Dec 03 13:05:26 crc kubenswrapper[5052]: I1203 13:05:26.783027 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pqkp9\" (UniqueName: \"kubernetes.io/projected/0ba752ed-f76e-406d-a116-8f2dd724d5f2-kube-api-access-pqkp9\") pod \"dnsmasq-dnsmasq-78c7b787f5-zr2sv\" (UID: \"0ba752ed-f76e-406d-a116-8f2dd724d5f2\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-78c7b787f5-zr2sv" Dec 03 13:05:26 crc kubenswrapper[5052]: I1203 13:05:26.783052 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0ba752ed-f76e-406d-a116-8f2dd724d5f2-config\") pod \"dnsmasq-dnsmasq-78c7b787f5-zr2sv\" (UID: \"0ba752ed-f76e-406d-a116-8f2dd724d5f2\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-78c7b787f5-zr2sv" Dec 03 13:05:26 crc kubenswrapper[5052]: I1203 13:05:26.783108 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openstack-edpm-tls\" (UniqueName: \"kubernetes.io/configmap/0ba752ed-f76e-406d-a116-8f2dd724d5f2-openstack-edpm-tls\") pod \"dnsmasq-dnsmasq-78c7b787f5-zr2sv\" (UID: \"0ba752ed-f76e-406d-a116-8f2dd724d5f2\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-78c7b787f5-zr2sv" Dec 03 13:05:26 crc kubenswrapper[5052]: I1203 13:05:26.884096 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/0ba752ed-f76e-406d-a116-8f2dd724d5f2-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-78c7b787f5-zr2sv\" (UID: \"0ba752ed-f76e-406d-a116-8f2dd724d5f2\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-78c7b787f5-zr2sv" Dec 03 13:05:26 crc kubenswrapper[5052]: I1203 13:05:26.885023 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/0ba752ed-f76e-406d-a116-8f2dd724d5f2-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-78c7b787f5-zr2sv\" (UID: \"0ba752ed-f76e-406d-a116-8f2dd724d5f2\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-78c7b787f5-zr2sv" Dec 03 13:05:26 crc kubenswrapper[5052]: I1203 13:05:26.885142 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d6mbs\" (UniqueName: \"kubernetes.io/projected/ff4169c4-e917-42c4-9ca1-b4f9d62be591-kube-api-access-d6mbs\") pod \"dnsmasq-dnsmasq-79cc674687-46mrq\" (UID: \"ff4169c4-e917-42c4-9ca1-b4f9d62be591\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-79cc674687-46mrq" Dec 03 13:05:26 crc kubenswrapper[5052]: I1203 13:05:26.885294 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openstack-edpm-tls\" (UniqueName: \"kubernetes.io/configmap/ff4169c4-e917-42c4-9ca1-b4f9d62be591-openstack-edpm-tls\") pod \"dnsmasq-dnsmasq-79cc674687-46mrq\" (UID: \"ff4169c4-e917-42c4-9ca1-b4f9d62be591\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-79cc674687-46mrq" Dec 03 13:05:26 crc kubenswrapper[5052]: I1203 13:05:26.885357 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-pqkp9\" (UniqueName: \"kubernetes.io/projected/0ba752ed-f76e-406d-a116-8f2dd724d5f2-kube-api-access-pqkp9\") pod \"dnsmasq-dnsmasq-78c7b787f5-zr2sv\" (UID: \"0ba752ed-f76e-406d-a116-8f2dd724d5f2\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-78c7b787f5-zr2sv" Dec 03 13:05:26 crc kubenswrapper[5052]: I1203 13:05:26.885384 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0ba752ed-f76e-406d-a116-8f2dd724d5f2-config\") pod \"dnsmasq-dnsmasq-78c7b787f5-zr2sv\" (UID: \"0ba752ed-f76e-406d-a116-8f2dd724d5f2\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-78c7b787f5-zr2sv" Dec 03 13:05:26 crc kubenswrapper[5052]: I1203 13:05:26.885402 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/ff4169c4-e917-42c4-9ca1-b4f9d62be591-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-79cc674687-46mrq\" (UID: \"ff4169c4-e917-42c4-9ca1-b4f9d62be591\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-79cc674687-46mrq" Dec 03 13:05:26 crc kubenswrapper[5052]: I1203 13:05:26.885449 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"openstack-edpm-tls\" (UniqueName: \"kubernetes.io/configmap/0ba752ed-f76e-406d-a116-8f2dd724d5f2-openstack-edpm-tls\") pod \"dnsmasq-dnsmasq-78c7b787f5-zr2sv\" (UID: \"0ba752ed-f76e-406d-a116-8f2dd724d5f2\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-78c7b787f5-zr2sv" Dec 03 13:05:26 crc kubenswrapper[5052]: I1203 13:05:26.885486 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ff4169c4-e917-42c4-9ca1-b4f9d62be591-config\") pod \"dnsmasq-dnsmasq-79cc674687-46mrq\" (UID: \"ff4169c4-e917-42c4-9ca1-b4f9d62be591\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-79cc674687-46mrq" Dec 03 13:05:26 crc kubenswrapper[5052]: I1203 13:05:26.886232 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0ba752ed-f76e-406d-a116-8f2dd724d5f2-config\") pod \"dnsmasq-dnsmasq-78c7b787f5-zr2sv\" (UID: \"0ba752ed-f76e-406d-a116-8f2dd724d5f2\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-78c7b787f5-zr2sv" Dec 03 13:05:26 crc kubenswrapper[5052]: I1203 13:05:26.886346 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"openstack-edpm-tls\" (UniqueName: \"kubernetes.io/configmap/0ba752ed-f76e-406d-a116-8f2dd724d5f2-openstack-edpm-tls\") pod \"dnsmasq-dnsmasq-78c7b787f5-zr2sv\" (UID: \"0ba752ed-f76e-406d-a116-8f2dd724d5f2\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-78c7b787f5-zr2sv" Dec 03 13:05:26 crc kubenswrapper[5052]: I1203 13:05:26.910028 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-pqkp9\" (UniqueName: \"kubernetes.io/projected/0ba752ed-f76e-406d-a116-8f2dd724d5f2-kube-api-access-pqkp9\") pod \"dnsmasq-dnsmasq-78c7b787f5-zr2sv\" (UID: \"0ba752ed-f76e-406d-a116-8f2dd724d5f2\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-78c7b787f5-zr2sv" Dec 03 13:05:26 crc kubenswrapper[5052]: I1203 13:05:26.911545 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-78c7b787f5-zr2sv" Dec 03 13:05:26 crc kubenswrapper[5052]: I1203 13:05:26.943701 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-78c7b787f5-zr2sv" Dec 03 13:05:26 crc kubenswrapper[5052]: I1203 13:05:26.986776 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0ba752ed-f76e-406d-a116-8f2dd724d5f2-config\") pod \"0ba752ed-f76e-406d-a116-8f2dd724d5f2\" (UID: \"0ba752ed-f76e-406d-a116-8f2dd724d5f2\") " Dec 03 13:05:26 crc kubenswrapper[5052]: I1203 13:05:26.986872 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"openstack-edpm-tls\" (UniqueName: \"kubernetes.io/configmap/0ba752ed-f76e-406d-a116-8f2dd724d5f2-openstack-edpm-tls\") pod \"0ba752ed-f76e-406d-a116-8f2dd724d5f2\" (UID: \"0ba752ed-f76e-406d-a116-8f2dd724d5f2\") " Dec 03 13:05:26 crc kubenswrapper[5052]: I1203 13:05:26.986936 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/0ba752ed-f76e-406d-a116-8f2dd724d5f2-dnsmasq-svc\") pod \"0ba752ed-f76e-406d-a116-8f2dd724d5f2\" (UID: \"0ba752ed-f76e-406d-a116-8f2dd724d5f2\") " Dec 03 13:05:26 crc kubenswrapper[5052]: I1203 13:05:26.986968 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pqkp9\" (UniqueName: \"kubernetes.io/projected/0ba752ed-f76e-406d-a116-8f2dd724d5f2-kube-api-access-pqkp9\") pod \"0ba752ed-f76e-406d-a116-8f2dd724d5f2\" (UID: \"0ba752ed-f76e-406d-a116-8f2dd724d5f2\") " Dec 03 13:05:26 crc kubenswrapper[5052]: I1203 13:05:26.987053 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ff4169c4-e917-42c4-9ca1-b4f9d62be591-config\") pod \"dnsmasq-dnsmasq-79cc674687-46mrq\" (UID: \"ff4169c4-e917-42c4-9ca1-b4f9d62be591\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-79cc674687-46mrq" Dec 03 13:05:26 crc kubenswrapper[5052]: I1203 13:05:26.987111 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-d6mbs\" (UniqueName: \"kubernetes.io/projected/ff4169c4-e917-42c4-9ca1-b4f9d62be591-kube-api-access-d6mbs\") pod \"dnsmasq-dnsmasq-79cc674687-46mrq\" (UID: \"ff4169c4-e917-42c4-9ca1-b4f9d62be591\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-79cc674687-46mrq" Dec 03 13:05:26 crc kubenswrapper[5052]: I1203 13:05:26.987151 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"openstack-edpm-tls\" (UniqueName: \"kubernetes.io/configmap/ff4169c4-e917-42c4-9ca1-b4f9d62be591-openstack-edpm-tls\") pod \"dnsmasq-dnsmasq-79cc674687-46mrq\" (UID: \"ff4169c4-e917-42c4-9ca1-b4f9d62be591\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-79cc674687-46mrq" Dec 03 13:05:26 crc kubenswrapper[5052]: I1203 13:05:26.987176 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/ff4169c4-e917-42c4-9ca1-b4f9d62be591-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-79cc674687-46mrq\" (UID: \"ff4169c4-e917-42c4-9ca1-b4f9d62be591\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-79cc674687-46mrq" Dec 03 13:05:26 crc kubenswrapper[5052]: I1203 13:05:26.987353 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0ba752ed-f76e-406d-a116-8f2dd724d5f2-config" (OuterVolumeSpecName: "config") pod "0ba752ed-f76e-406d-a116-8f2dd724d5f2" (UID: "0ba752ed-f76e-406d-a116-8f2dd724d5f2"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:05:26 crc kubenswrapper[5052]: I1203 13:05:26.987821 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0ba752ed-f76e-406d-a116-8f2dd724d5f2-dnsmasq-svc" (OuterVolumeSpecName: "dnsmasq-svc") pod "0ba752ed-f76e-406d-a116-8f2dd724d5f2" (UID: "0ba752ed-f76e-406d-a116-8f2dd724d5f2"). InnerVolumeSpecName "dnsmasq-svc". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:05:26 crc kubenswrapper[5052]: I1203 13:05:26.988190 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/ff4169c4-e917-42c4-9ca1-b4f9d62be591-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-79cc674687-46mrq\" (UID: \"ff4169c4-e917-42c4-9ca1-b4f9d62be591\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-79cc674687-46mrq" Dec 03 13:05:26 crc kubenswrapper[5052]: I1203 13:05:26.988270 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"openstack-edpm-tls\" (UniqueName: \"kubernetes.io/configmap/ff4169c4-e917-42c4-9ca1-b4f9d62be591-openstack-edpm-tls\") pod \"dnsmasq-dnsmasq-79cc674687-46mrq\" (UID: \"ff4169c4-e917-42c4-9ca1-b4f9d62be591\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-79cc674687-46mrq" Dec 03 13:05:26 crc kubenswrapper[5052]: I1203 13:05:26.988484 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0ba752ed-f76e-406d-a116-8f2dd724d5f2-openstack-edpm-tls" (OuterVolumeSpecName: "openstack-edpm-tls") pod "0ba752ed-f76e-406d-a116-8f2dd724d5f2" (UID: "0ba752ed-f76e-406d-a116-8f2dd724d5f2"). InnerVolumeSpecName "openstack-edpm-tls". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:05:26 crc kubenswrapper[5052]: I1203 13:05:26.989431 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ff4169c4-e917-42c4-9ca1-b4f9d62be591-config\") pod \"dnsmasq-dnsmasq-79cc674687-46mrq\" (UID: \"ff4169c4-e917-42c4-9ca1-b4f9d62be591\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-79cc674687-46mrq" Dec 03 13:05:26 crc kubenswrapper[5052]: I1203 13:05:26.990608 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0ba752ed-f76e-406d-a116-8f2dd724d5f2-kube-api-access-pqkp9" (OuterVolumeSpecName: "kube-api-access-pqkp9") pod "0ba752ed-f76e-406d-a116-8f2dd724d5f2" (UID: "0ba752ed-f76e-406d-a116-8f2dd724d5f2"). InnerVolumeSpecName "kube-api-access-pqkp9". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:05:27 crc kubenswrapper[5052]: I1203 13:05:27.008001 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-d6mbs\" (UniqueName: \"kubernetes.io/projected/ff4169c4-e917-42c4-9ca1-b4f9d62be591-kube-api-access-d6mbs\") pod \"dnsmasq-dnsmasq-79cc674687-46mrq\" (UID: \"ff4169c4-e917-42c4-9ca1-b4f9d62be591\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-79cc674687-46mrq" Dec 03 13:05:27 crc kubenswrapper[5052]: I1203 13:05:27.074869 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-79cc674687-46mrq" Dec 03 13:05:27 crc kubenswrapper[5052]: I1203 13:05:27.088598 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0ba752ed-f76e-406d-a116-8f2dd724d5f2-config\") on node \"crc\" DevicePath \"\"" Dec 03 13:05:27 crc kubenswrapper[5052]: I1203 13:05:27.088665 5052 reconciler_common.go:293] "Volume detached for volume \"openstack-edpm-tls\" (UniqueName: \"kubernetes.io/configmap/0ba752ed-f76e-406d-a116-8f2dd724d5f2-openstack-edpm-tls\") on node \"crc\" DevicePath \"\"" Dec 03 13:05:27 crc kubenswrapper[5052]: I1203 13:05:27.088690 5052 reconciler_common.go:293] "Volume detached for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/0ba752ed-f76e-406d-a116-8f2dd724d5f2-dnsmasq-svc\") on node \"crc\" DevicePath \"\"" Dec 03 13:05:27 crc kubenswrapper[5052]: I1203 13:05:27.088713 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-pqkp9\" (UniqueName: \"kubernetes.io/projected/0ba752ed-f76e-406d-a116-8f2dd724d5f2-kube-api-access-pqkp9\") on node \"crc\" DevicePath \"\"" Dec 03 13:05:27 crc kubenswrapper[5052]: I1203 13:05:27.559087 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-79cc674687-46mrq"] Dec 03 13:05:27 crc kubenswrapper[5052]: W1203 13:05:27.567371 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podff4169c4_e917_42c4_9ca1_b4f9d62be591.slice/crio-b1531a85bc812722feb1cb93d8bdb578e7b87369252aee63dc97076efc16936b WatchSource:0}: Error finding container b1531a85bc812722feb1cb93d8bdb578e7b87369252aee63dc97076efc16936b: Status 404 returned error can't find the container with id b1531a85bc812722feb1cb93d8bdb578e7b87369252aee63dc97076efc16936b Dec 03 13:05:27 crc kubenswrapper[5052]: I1203 13:05:27.921412 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-79cc674687-46mrq" event={"ID":"ff4169c4-e917-42c4-9ca1-b4f9d62be591","Type":"ContainerStarted","Data":"b1531a85bc812722feb1cb93d8bdb578e7b87369252aee63dc97076efc16936b"} Dec 03 13:05:27 crc kubenswrapper[5052]: I1203 13:05:27.921443 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-78c7b787f5-zr2sv" Dec 03 13:05:27 crc kubenswrapper[5052]: I1203 13:05:27.979277 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-78c7b787f5-zr2sv"] Dec 03 13:05:27 crc kubenswrapper[5052]: I1203 13:05:27.991549 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-78c7b787f5-zr2sv"] Dec 03 13:05:28 crc kubenswrapper[5052]: I1203 13:05:28.936991 5052 generic.go:334] "Generic (PLEG): container finished" podID="ff4169c4-e917-42c4-9ca1-b4f9d62be591" containerID="f104d4304f9bdd12c76ee231cc308e06758abe0908de3165985bc5bac2b531d5" exitCode=0 Dec 03 13:05:28 crc kubenswrapper[5052]: I1203 13:05:28.937103 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-79cc674687-46mrq" event={"ID":"ff4169c4-e917-42c4-9ca1-b4f9d62be591","Type":"ContainerDied","Data":"f104d4304f9bdd12c76ee231cc308e06758abe0908de3165985bc5bac2b531d5"} Dec 03 13:05:29 crc kubenswrapper[5052]: I1203 13:05:29.682623 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0ba752ed-f76e-406d-a116-8f2dd724d5f2" path="/var/lib/kubelet/pods/0ba752ed-f76e-406d-a116-8f2dd724d5f2/volumes" Dec 03 13:05:29 crc kubenswrapper[5052]: I1203 13:05:29.948279 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-79cc674687-46mrq" event={"ID":"ff4169c4-e917-42c4-9ca1-b4f9d62be591","Type":"ContainerStarted","Data":"b32b57428120953190e507dec8a34a893da0817c1268d08aabd67cd56b506473"} Dec 03 13:05:29 crc kubenswrapper[5052]: I1203 13:05:29.948409 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-79cc674687-46mrq" Dec 03 13:05:29 crc kubenswrapper[5052]: I1203 13:05:29.970594 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-79cc674687-46mrq" podStartSLOduration=3.970575528 podStartE2EDuration="3.970575528s" podCreationTimestamp="2025-12-03 13:05:26 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 13:05:29.96189678 +0000 UTC m=+6320.550138544" watchObservedRunningTime="2025-12-03 13:05:29.970575528 +0000 UTC m=+6320.558817292" Dec 03 13:05:37 crc kubenswrapper[5052]: I1203 13:05:37.076738 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-79cc674687-46mrq" Dec 03 13:05:37 crc kubenswrapper[5052]: I1203 13:05:37.128540 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-4s9xh"] Dec 03 13:05:37 crc kubenswrapper[5052]: I1203 13:05:37.131426 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-4s9xh" podUID="030b2288-f819-48fd-91a2-e08b1130e648" containerName="dnsmasq-dns" containerID="cri-o://307141c01a745c9de3c9354a528ed4b1de125eb377db1905522e2c657216d4ef" gracePeriod=10 Dec 03 13:05:37 crc kubenswrapper[5052]: I1203 13:05:37.514287 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-4s9xh" Dec 03 13:05:37 crc kubenswrapper[5052]: I1203 13:05:37.650380 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/030b2288-f819-48fd-91a2-e08b1130e648-config\") pod \"030b2288-f819-48fd-91a2-e08b1130e648\" (UID: \"030b2288-f819-48fd-91a2-e08b1130e648\") " Dec 03 13:05:37 crc kubenswrapper[5052]: I1203 13:05:37.650483 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kszbj\" (UniqueName: \"kubernetes.io/projected/030b2288-f819-48fd-91a2-e08b1130e648-kube-api-access-kszbj\") pod \"030b2288-f819-48fd-91a2-e08b1130e648\" (UID: \"030b2288-f819-48fd-91a2-e08b1130e648\") " Dec 03 13:05:37 crc kubenswrapper[5052]: I1203 13:05:37.650586 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/030b2288-f819-48fd-91a2-e08b1130e648-dnsmasq-svc\") pod \"030b2288-f819-48fd-91a2-e08b1130e648\" (UID: \"030b2288-f819-48fd-91a2-e08b1130e648\") " Dec 03 13:05:37 crc kubenswrapper[5052]: I1203 13:05:37.655960 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/030b2288-f819-48fd-91a2-e08b1130e648-kube-api-access-kszbj" (OuterVolumeSpecName: "kube-api-access-kszbj") pod "030b2288-f819-48fd-91a2-e08b1130e648" (UID: "030b2288-f819-48fd-91a2-e08b1130e648"). InnerVolumeSpecName "kube-api-access-kszbj". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:05:37 crc kubenswrapper[5052]: I1203 13:05:37.701111 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/030b2288-f819-48fd-91a2-e08b1130e648-config" (OuterVolumeSpecName: "config") pod "030b2288-f819-48fd-91a2-e08b1130e648" (UID: "030b2288-f819-48fd-91a2-e08b1130e648"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:05:37 crc kubenswrapper[5052]: I1203 13:05:37.702222 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/030b2288-f819-48fd-91a2-e08b1130e648-dnsmasq-svc" (OuterVolumeSpecName: "dnsmasq-svc") pod "030b2288-f819-48fd-91a2-e08b1130e648" (UID: "030b2288-f819-48fd-91a2-e08b1130e648"). InnerVolumeSpecName "dnsmasq-svc". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:05:37 crc kubenswrapper[5052]: I1203 13:05:37.753278 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-kszbj\" (UniqueName: \"kubernetes.io/projected/030b2288-f819-48fd-91a2-e08b1130e648-kube-api-access-kszbj\") on node \"crc\" DevicePath \"\"" Dec 03 13:05:37 crc kubenswrapper[5052]: I1203 13:05:37.753320 5052 reconciler_common.go:293] "Volume detached for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/030b2288-f819-48fd-91a2-e08b1130e648-dnsmasq-svc\") on node \"crc\" DevicePath \"\"" Dec 03 13:05:37 crc kubenswrapper[5052]: I1203 13:05:37.753334 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/030b2288-f819-48fd-91a2-e08b1130e648-config\") on node \"crc\" DevicePath \"\"" Dec 03 13:05:38 crc kubenswrapper[5052]: I1203 13:05:38.020565 5052 generic.go:334] "Generic (PLEG): container finished" podID="030b2288-f819-48fd-91a2-e08b1130e648" containerID="307141c01a745c9de3c9354a528ed4b1de125eb377db1905522e2c657216d4ef" exitCode=0 Dec 03 13:05:38 crc kubenswrapper[5052]: I1203 13:05:38.020610 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-4s9xh" event={"ID":"030b2288-f819-48fd-91a2-e08b1130e648","Type":"ContainerDied","Data":"307141c01a745c9de3c9354a528ed4b1de125eb377db1905522e2c657216d4ef"} Dec 03 13:05:38 crc kubenswrapper[5052]: I1203 13:05:38.020622 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-4s9xh" Dec 03 13:05:38 crc kubenswrapper[5052]: I1203 13:05:38.020647 5052 scope.go:117] "RemoveContainer" containerID="307141c01a745c9de3c9354a528ed4b1de125eb377db1905522e2c657216d4ef" Dec 03 13:05:38 crc kubenswrapper[5052]: I1203 13:05:38.020636 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-4s9xh" event={"ID":"030b2288-f819-48fd-91a2-e08b1130e648","Type":"ContainerDied","Data":"7bce5b84beacec4a6e859f708432283096ff43aa0ac5aebd15f1ac5dc49f681d"} Dec 03 13:05:38 crc kubenswrapper[5052]: I1203 13:05:38.049100 5052 scope.go:117] "RemoveContainer" containerID="b9f57165a0c4c083b6315e54f4620632e3db22e75d5ebfcd465b1fe0467252a9" Dec 03 13:05:38 crc kubenswrapper[5052]: I1203 13:05:38.053789 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-4s9xh"] Dec 03 13:05:38 crc kubenswrapper[5052]: I1203 13:05:38.068277 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-4s9xh"] Dec 03 13:05:38 crc kubenswrapper[5052]: I1203 13:05:38.092354 5052 scope.go:117] "RemoveContainer" containerID="307141c01a745c9de3c9354a528ed4b1de125eb377db1905522e2c657216d4ef" Dec 03 13:05:38 crc kubenswrapper[5052]: E1203 13:05:38.093000 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"307141c01a745c9de3c9354a528ed4b1de125eb377db1905522e2c657216d4ef\": container with ID starting with 307141c01a745c9de3c9354a528ed4b1de125eb377db1905522e2c657216d4ef not found: ID does not exist" containerID="307141c01a745c9de3c9354a528ed4b1de125eb377db1905522e2c657216d4ef" Dec 03 13:05:38 crc kubenswrapper[5052]: I1203 13:05:38.093034 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"307141c01a745c9de3c9354a528ed4b1de125eb377db1905522e2c657216d4ef"} err="failed to get container status \"307141c01a745c9de3c9354a528ed4b1de125eb377db1905522e2c657216d4ef\": rpc error: code = NotFound desc = could not find container \"307141c01a745c9de3c9354a528ed4b1de125eb377db1905522e2c657216d4ef\": container with ID starting with 307141c01a745c9de3c9354a528ed4b1de125eb377db1905522e2c657216d4ef not found: ID does not exist" Dec 03 13:05:38 crc kubenswrapper[5052]: I1203 13:05:38.093060 5052 scope.go:117] "RemoveContainer" containerID="b9f57165a0c4c083b6315e54f4620632e3db22e75d5ebfcd465b1fe0467252a9" Dec 03 13:05:38 crc kubenswrapper[5052]: E1203 13:05:38.094165 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b9f57165a0c4c083b6315e54f4620632e3db22e75d5ebfcd465b1fe0467252a9\": container with ID starting with b9f57165a0c4c083b6315e54f4620632e3db22e75d5ebfcd465b1fe0467252a9 not found: ID does not exist" containerID="b9f57165a0c4c083b6315e54f4620632e3db22e75d5ebfcd465b1fe0467252a9" Dec 03 13:05:38 crc kubenswrapper[5052]: I1203 13:05:38.094692 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b9f57165a0c4c083b6315e54f4620632e3db22e75d5ebfcd465b1fe0467252a9"} err="failed to get container status \"b9f57165a0c4c083b6315e54f4620632e3db22e75d5ebfcd465b1fe0467252a9\": rpc error: code = NotFound desc = could not find container \"b9f57165a0c4c083b6315e54f4620632e3db22e75d5ebfcd465b1fe0467252a9\": container with ID starting with b9f57165a0c4c083b6315e54f4620632e3db22e75d5ebfcd465b1fe0467252a9 not found: ID does not exist" Dec 03 13:05:39 crc kubenswrapper[5052]: I1203 13:05:39.668202 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/install-certs-ovrd-openstack-edpm-tls-openstack-edpm-tls-ts7gt"] Dec 03 13:05:39 crc kubenswrapper[5052]: E1203 13:05:39.669427 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="030b2288-f819-48fd-91a2-e08b1130e648" containerName="init" Dec 03 13:05:39 crc kubenswrapper[5052]: I1203 13:05:39.669632 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="030b2288-f819-48fd-91a2-e08b1130e648" containerName="init" Dec 03 13:05:39 crc kubenswrapper[5052]: E1203 13:05:39.669749 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="030b2288-f819-48fd-91a2-e08b1130e648" containerName="dnsmasq-dns" Dec 03 13:05:39 crc kubenswrapper[5052]: I1203 13:05:39.669814 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="030b2288-f819-48fd-91a2-e08b1130e648" containerName="dnsmasq-dns" Dec 03 13:05:39 crc kubenswrapper[5052]: I1203 13:05:39.670037 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="030b2288-f819-48fd-91a2-e08b1130e648" containerName="dnsmasq-dns" Dec 03 13:05:39 crc kubenswrapper[5052]: I1203 13:05:39.670770 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/install-certs-ovrd-openstack-edpm-tls-openstack-edpm-tls-ts7gt" Dec 03 13:05:39 crc kubenswrapper[5052]: I1203 13:05:39.674598 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"openstack-edpm-tls-dockercfg-kz8sq" Dec 03 13:05:39 crc kubenswrapper[5052]: I1203 13:05:39.675053 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"openstack-edpm-tls-tls-dnsnames-default-certs-0" Dec 03 13:05:39 crc kubenswrapper[5052]: I1203 13:05:39.675175 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"combined-ca-bundle" Dec 03 13:05:39 crc kubenswrapper[5052]: I1203 13:05:39.675272 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"openstack-aee-default-env" Dec 03 13:05:39 crc kubenswrapper[5052]: I1203 13:05:39.675609 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"dataplane-ansible-ssh-private-key-secret" Dec 03 13:05:39 crc kubenswrapper[5052]: I1203 13:05:39.675761 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"dataplanenodeset-openstack-edpm-tls" Dec 03 13:05:39 crc kubenswrapper[5052]: I1203 13:05:39.675634 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"openstack-edpm-tls-tls-dnsnames-second-certs-0" Dec 03 13:05:39 crc kubenswrapper[5052]: I1203 13:05:39.682851 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="030b2288-f819-48fd-91a2-e08b1130e648" path="/var/lib/kubelet/pods/030b2288-f819-48fd-91a2-e08b1130e648/volumes" Dec 03 13:05:39 crc kubenswrapper[5052]: I1203 13:05:39.683772 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/install-certs-ovrd-openstack-edpm-tls-openstack-edpm-tls-ts7gt"] Dec 03 13:05:39 crc kubenswrapper[5052]: I1203 13:05:39.781401 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openstack-edpm-tls-tls-dnsnames-default-certs-0\" (UniqueName: \"kubernetes.io/projected/2ec739d1-4d9b-4327-aa7d-d31a55fddb55-openstack-edpm-tls-tls-dnsnames-default-certs-0\") pod \"install-certs-ovrd-openstack-edpm-tls-openstack-edpm-tls-ts7gt\" (UID: \"2ec739d1-4d9b-4327-aa7d-d31a55fddb55\") " pod="openstack-kuttl-tests/install-certs-ovrd-openstack-edpm-tls-openstack-edpm-tls-ts7gt" Dec 03 13:05:39 crc kubenswrapper[5052]: I1203 13:05:39.781449 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/2ec739d1-4d9b-4327-aa7d-d31a55fddb55-ssh-key\") pod \"install-certs-ovrd-openstack-edpm-tls-openstack-edpm-tls-ts7gt\" (UID: \"2ec739d1-4d9b-4327-aa7d-d31a55fddb55\") " pod="openstack-kuttl-tests/install-certs-ovrd-openstack-edpm-tls-openstack-edpm-tls-ts7gt" Dec 03 13:05:39 crc kubenswrapper[5052]: I1203 13:05:39.781473 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-dnsnames-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2ec739d1-4d9b-4327-aa7d-d31a55fddb55-tls-dnsnames-combined-ca-bundle\") pod \"install-certs-ovrd-openstack-edpm-tls-openstack-edpm-tls-ts7gt\" (UID: \"2ec739d1-4d9b-4327-aa7d-d31a55fddb55\") " pod="openstack-kuttl-tests/install-certs-ovrd-openstack-edpm-tls-openstack-edpm-tls-ts7gt" Dec 03 13:05:39 crc kubenswrapper[5052]: I1203 13:05:39.781492 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/2ec739d1-4d9b-4327-aa7d-d31a55fddb55-inventory\") pod \"install-certs-ovrd-openstack-edpm-tls-openstack-edpm-tls-ts7gt\" (UID: \"2ec739d1-4d9b-4327-aa7d-d31a55fddb55\") " pod="openstack-kuttl-tests/install-certs-ovrd-openstack-edpm-tls-openstack-edpm-tls-ts7gt" Dec 03 13:05:39 crc kubenswrapper[5052]: I1203 13:05:39.781531 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zpz4f\" (UniqueName: \"kubernetes.io/projected/2ec739d1-4d9b-4327-aa7d-d31a55fddb55-kube-api-access-zpz4f\") pod \"install-certs-ovrd-openstack-edpm-tls-openstack-edpm-tls-ts7gt\" (UID: \"2ec739d1-4d9b-4327-aa7d-d31a55fddb55\") " pod="openstack-kuttl-tests/install-certs-ovrd-openstack-edpm-tls-openstack-edpm-tls-ts7gt" Dec 03 13:05:39 crc kubenswrapper[5052]: I1203 13:05:39.781556 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"install-certs-ovrd-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2ec739d1-4d9b-4327-aa7d-d31a55fddb55-install-certs-ovrd-combined-ca-bundle\") pod \"install-certs-ovrd-openstack-edpm-tls-openstack-edpm-tls-ts7gt\" (UID: \"2ec739d1-4d9b-4327-aa7d-d31a55fddb55\") " pod="openstack-kuttl-tests/install-certs-ovrd-openstack-edpm-tls-openstack-edpm-tls-ts7gt" Dec 03 13:05:39 crc kubenswrapper[5052]: I1203 13:05:39.781605 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openstack-edpm-tls-tls-dnsnames-second-certs-0\" (UniqueName: \"kubernetes.io/projected/2ec739d1-4d9b-4327-aa7d-d31a55fddb55-openstack-edpm-tls-tls-dnsnames-second-certs-0\") pod \"install-certs-ovrd-openstack-edpm-tls-openstack-edpm-tls-ts7gt\" (UID: \"2ec739d1-4d9b-4327-aa7d-d31a55fddb55\") " pod="openstack-kuttl-tests/install-certs-ovrd-openstack-edpm-tls-openstack-edpm-tls-ts7gt" Dec 03 13:05:39 crc kubenswrapper[5052]: I1203 13:05:39.883482 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"openstack-edpm-tls-tls-dnsnames-second-certs-0\" (UniqueName: \"kubernetes.io/projected/2ec739d1-4d9b-4327-aa7d-d31a55fddb55-openstack-edpm-tls-tls-dnsnames-second-certs-0\") pod \"install-certs-ovrd-openstack-edpm-tls-openstack-edpm-tls-ts7gt\" (UID: \"2ec739d1-4d9b-4327-aa7d-d31a55fddb55\") " pod="openstack-kuttl-tests/install-certs-ovrd-openstack-edpm-tls-openstack-edpm-tls-ts7gt" Dec 03 13:05:39 crc kubenswrapper[5052]: I1203 13:05:39.883610 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"openstack-edpm-tls-tls-dnsnames-default-certs-0\" (UniqueName: \"kubernetes.io/projected/2ec739d1-4d9b-4327-aa7d-d31a55fddb55-openstack-edpm-tls-tls-dnsnames-default-certs-0\") pod \"install-certs-ovrd-openstack-edpm-tls-openstack-edpm-tls-ts7gt\" (UID: \"2ec739d1-4d9b-4327-aa7d-d31a55fddb55\") " pod="openstack-kuttl-tests/install-certs-ovrd-openstack-edpm-tls-openstack-edpm-tls-ts7gt" Dec 03 13:05:39 crc kubenswrapper[5052]: I1203 13:05:39.883640 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/2ec739d1-4d9b-4327-aa7d-d31a55fddb55-ssh-key\") pod \"install-certs-ovrd-openstack-edpm-tls-openstack-edpm-tls-ts7gt\" (UID: \"2ec739d1-4d9b-4327-aa7d-d31a55fddb55\") " pod="openstack-kuttl-tests/install-certs-ovrd-openstack-edpm-tls-openstack-edpm-tls-ts7gt" Dec 03 13:05:39 crc kubenswrapper[5052]: I1203 13:05:39.883666 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"tls-dnsnames-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2ec739d1-4d9b-4327-aa7d-d31a55fddb55-tls-dnsnames-combined-ca-bundle\") pod \"install-certs-ovrd-openstack-edpm-tls-openstack-edpm-tls-ts7gt\" (UID: \"2ec739d1-4d9b-4327-aa7d-d31a55fddb55\") " pod="openstack-kuttl-tests/install-certs-ovrd-openstack-edpm-tls-openstack-edpm-tls-ts7gt" Dec 03 13:05:39 crc kubenswrapper[5052]: I1203 13:05:39.883698 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/2ec739d1-4d9b-4327-aa7d-d31a55fddb55-inventory\") pod \"install-certs-ovrd-openstack-edpm-tls-openstack-edpm-tls-ts7gt\" (UID: \"2ec739d1-4d9b-4327-aa7d-d31a55fddb55\") " pod="openstack-kuttl-tests/install-certs-ovrd-openstack-edpm-tls-openstack-edpm-tls-ts7gt" Dec 03 13:05:39 crc kubenswrapper[5052]: I1203 13:05:39.883736 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-zpz4f\" (UniqueName: \"kubernetes.io/projected/2ec739d1-4d9b-4327-aa7d-d31a55fddb55-kube-api-access-zpz4f\") pod \"install-certs-ovrd-openstack-edpm-tls-openstack-edpm-tls-ts7gt\" (UID: \"2ec739d1-4d9b-4327-aa7d-d31a55fddb55\") " pod="openstack-kuttl-tests/install-certs-ovrd-openstack-edpm-tls-openstack-edpm-tls-ts7gt" Dec 03 13:05:39 crc kubenswrapper[5052]: I1203 13:05:39.883770 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"install-certs-ovrd-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2ec739d1-4d9b-4327-aa7d-d31a55fddb55-install-certs-ovrd-combined-ca-bundle\") pod \"install-certs-ovrd-openstack-edpm-tls-openstack-edpm-tls-ts7gt\" (UID: \"2ec739d1-4d9b-4327-aa7d-d31a55fddb55\") " pod="openstack-kuttl-tests/install-certs-ovrd-openstack-edpm-tls-openstack-edpm-tls-ts7gt" Dec 03 13:05:39 crc kubenswrapper[5052]: I1203 13:05:39.888669 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"openstack-edpm-tls-tls-dnsnames-second-certs-0\" (UniqueName: \"kubernetes.io/projected/2ec739d1-4d9b-4327-aa7d-d31a55fddb55-openstack-edpm-tls-tls-dnsnames-second-certs-0\") pod \"install-certs-ovrd-openstack-edpm-tls-openstack-edpm-tls-ts7gt\" (UID: \"2ec739d1-4d9b-4327-aa7d-d31a55fddb55\") " pod="openstack-kuttl-tests/install-certs-ovrd-openstack-edpm-tls-openstack-edpm-tls-ts7gt" Dec 03 13:05:39 crc kubenswrapper[5052]: I1203 13:05:39.888688 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/2ec739d1-4d9b-4327-aa7d-d31a55fddb55-inventory\") pod \"install-certs-ovrd-openstack-edpm-tls-openstack-edpm-tls-ts7gt\" (UID: \"2ec739d1-4d9b-4327-aa7d-d31a55fddb55\") " pod="openstack-kuttl-tests/install-certs-ovrd-openstack-edpm-tls-openstack-edpm-tls-ts7gt" Dec 03 13:05:39 crc kubenswrapper[5052]: I1203 13:05:39.889372 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/2ec739d1-4d9b-4327-aa7d-d31a55fddb55-ssh-key\") pod \"install-certs-ovrd-openstack-edpm-tls-openstack-edpm-tls-ts7gt\" (UID: \"2ec739d1-4d9b-4327-aa7d-d31a55fddb55\") " pod="openstack-kuttl-tests/install-certs-ovrd-openstack-edpm-tls-openstack-edpm-tls-ts7gt" Dec 03 13:05:39 crc kubenswrapper[5052]: I1203 13:05:39.889648 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"openstack-edpm-tls-tls-dnsnames-default-certs-0\" (UniqueName: \"kubernetes.io/projected/2ec739d1-4d9b-4327-aa7d-d31a55fddb55-openstack-edpm-tls-tls-dnsnames-default-certs-0\") pod \"install-certs-ovrd-openstack-edpm-tls-openstack-edpm-tls-ts7gt\" (UID: \"2ec739d1-4d9b-4327-aa7d-d31a55fddb55\") " pod="openstack-kuttl-tests/install-certs-ovrd-openstack-edpm-tls-openstack-edpm-tls-ts7gt" Dec 03 13:05:39 crc kubenswrapper[5052]: I1203 13:05:39.890217 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"install-certs-ovrd-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2ec739d1-4d9b-4327-aa7d-d31a55fddb55-install-certs-ovrd-combined-ca-bundle\") pod \"install-certs-ovrd-openstack-edpm-tls-openstack-edpm-tls-ts7gt\" (UID: \"2ec739d1-4d9b-4327-aa7d-d31a55fddb55\") " pod="openstack-kuttl-tests/install-certs-ovrd-openstack-edpm-tls-openstack-edpm-tls-ts7gt" Dec 03 13:05:39 crc kubenswrapper[5052]: I1203 13:05:39.894640 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"tls-dnsnames-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2ec739d1-4d9b-4327-aa7d-d31a55fddb55-tls-dnsnames-combined-ca-bundle\") pod \"install-certs-ovrd-openstack-edpm-tls-openstack-edpm-tls-ts7gt\" (UID: \"2ec739d1-4d9b-4327-aa7d-d31a55fddb55\") " pod="openstack-kuttl-tests/install-certs-ovrd-openstack-edpm-tls-openstack-edpm-tls-ts7gt" Dec 03 13:05:39 crc kubenswrapper[5052]: I1203 13:05:39.900663 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-zpz4f\" (UniqueName: \"kubernetes.io/projected/2ec739d1-4d9b-4327-aa7d-d31a55fddb55-kube-api-access-zpz4f\") pod \"install-certs-ovrd-openstack-edpm-tls-openstack-edpm-tls-ts7gt\" (UID: \"2ec739d1-4d9b-4327-aa7d-d31a55fddb55\") " pod="openstack-kuttl-tests/install-certs-ovrd-openstack-edpm-tls-openstack-edpm-tls-ts7gt" Dec 03 13:05:39 crc kubenswrapper[5052]: I1203 13:05:39.990934 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/install-certs-ovrd-openstack-edpm-tls-openstack-edpm-tls-ts7gt" Dec 03 13:05:40 crc kubenswrapper[5052]: I1203 13:05:40.451953 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/install-certs-ovrd-openstack-edpm-tls-openstack-edpm-tls-ts7gt"] Dec 03 13:05:41 crc kubenswrapper[5052]: I1203 13:05:41.050414 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/install-certs-ovrd-openstack-edpm-tls-openstack-edpm-tls-ts7gt" event={"ID":"2ec739d1-4d9b-4327-aa7d-d31a55fddb55","Type":"ContainerStarted","Data":"52768cbf729b86d7c2ed018548785a3c3989a36d4866e79c83d0b71b8e93dd57"} Dec 03 13:05:41 crc kubenswrapper[5052]: I1203 13:05:41.628625 5052 scope.go:117] "RemoveContainer" containerID="4359c78028877a7402ffc505753075779053ce031be99a55254ae4756213eff6" Dec 03 13:05:41 crc kubenswrapper[5052]: I1203 13:05:41.661128 5052 scope.go:117] "RemoveContainer" containerID="69e36da60df181c814dcb00f2d0f2b1ce08d4683088cf9c43ddd1deeb7ba65a0" Dec 03 13:05:41 crc kubenswrapper[5052]: I1203 13:05:41.699614 5052 scope.go:117] "RemoveContainer" containerID="f0f5f5c3f2418588d3b27ec433b9cab51b14cc2c52c0e59202556c9b4642bab5" Dec 03 13:05:41 crc kubenswrapper[5052]: I1203 13:05:41.729767 5052 scope.go:117] "RemoveContainer" containerID="0b6c95d92f1b094a11ce62df4323bfac2aba1e665a8e25685952ebf1c6912b04" Dec 03 13:05:41 crc kubenswrapper[5052]: I1203 13:05:41.760018 5052 scope.go:117] "RemoveContainer" containerID="150deb6c290fbd42f50cc667850b46c58753348c6b8763611a69647692127bce" Dec 03 13:05:41 crc kubenswrapper[5052]: I1203 13:05:41.787280 5052 scope.go:117] "RemoveContainer" containerID="c2929449b63508dbdd5ff4b0cb9bdb1f65cf2a3423afd1a23614e1e61951b61c" Dec 03 13:05:41 crc kubenswrapper[5052]: I1203 13:05:41.819459 5052 scope.go:117] "RemoveContainer" containerID="6f57ee592c6c48f09e795dcd724baf8ca6f694926935f16a3ebe7fe88f7a2520" Dec 03 13:05:41 crc kubenswrapper[5052]: I1203 13:05:41.849995 5052 scope.go:117] "RemoveContainer" containerID="305ee98c14899a5167ed67bbb374adfee664b811ba63d26bfab5af7f0bbc880f" Dec 03 13:05:41 crc kubenswrapper[5052]: I1203 13:05:41.889743 5052 scope.go:117] "RemoveContainer" containerID="d86a514852064d3f9a1460f1d0c69f1d65af69f5fef6f94d0a52686d3a5d8055" Dec 03 13:05:41 crc kubenswrapper[5052]: I1203 13:05:41.914026 5052 scope.go:117] "RemoveContainer" containerID="351eb338913ef40dcaca07a18d88b7d7bee6b0144cace109e505631f705e3ed3" Dec 03 13:05:42 crc kubenswrapper[5052]: I1203 13:05:42.064554 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/install-certs-ovrd-openstack-edpm-tls-openstack-edpm-tls-ts7gt" event={"ID":"2ec739d1-4d9b-4327-aa7d-d31a55fddb55","Type":"ContainerStarted","Data":"3f41b3b842480bdfc1ae1546f4e2858dffe1059dff604526e61f5cdacd5a2155"} Dec 03 13:05:42 crc kubenswrapper[5052]: I1203 13:05:42.086240 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/install-certs-ovrd-openstack-edpm-tls-openstack-edpm-tls-ts7gt" podStartSLOduration=2.558608686 podStartE2EDuration="3.086214244s" podCreationTimestamp="2025-12-03 13:05:39 +0000 UTC" firstStartedPulling="2025-12-03 13:05:40.458772652 +0000 UTC m=+6331.047014416" lastFinishedPulling="2025-12-03 13:05:40.9863782 +0000 UTC m=+6331.574619974" observedRunningTime="2025-12-03 13:05:42.078254776 +0000 UTC m=+6332.666496540" watchObservedRunningTime="2025-12-03 13:05:42.086214244 +0000 UTC m=+6332.674456008" Dec 03 13:05:45 crc kubenswrapper[5052]: I1203 13:05:45.107394 5052 generic.go:334] "Generic (PLEG): container finished" podID="2ec739d1-4d9b-4327-aa7d-d31a55fddb55" containerID="3f41b3b842480bdfc1ae1546f4e2858dffe1059dff604526e61f5cdacd5a2155" exitCode=0 Dec 03 13:05:45 crc kubenswrapper[5052]: I1203 13:05:45.107468 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/install-certs-ovrd-openstack-edpm-tls-openstack-edpm-tls-ts7gt" event={"ID":"2ec739d1-4d9b-4327-aa7d-d31a55fddb55","Type":"ContainerDied","Data":"3f41b3b842480bdfc1ae1546f4e2858dffe1059dff604526e61f5cdacd5a2155"} Dec 03 13:05:46 crc kubenswrapper[5052]: I1203 13:05:46.422396 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/install-certs-ovrd-openstack-edpm-tls-openstack-edpm-tls-ts7gt" Dec 03 13:05:46 crc kubenswrapper[5052]: I1203 13:05:46.483578 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zpz4f\" (UniqueName: \"kubernetes.io/projected/2ec739d1-4d9b-4327-aa7d-d31a55fddb55-kube-api-access-zpz4f\") pod \"2ec739d1-4d9b-4327-aa7d-d31a55fddb55\" (UID: \"2ec739d1-4d9b-4327-aa7d-d31a55fddb55\") " Dec 03 13:05:46 crc kubenswrapper[5052]: I1203 13:05:46.483638 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"openstack-edpm-tls-tls-dnsnames-default-certs-0\" (UniqueName: \"kubernetes.io/projected/2ec739d1-4d9b-4327-aa7d-d31a55fddb55-openstack-edpm-tls-tls-dnsnames-default-certs-0\") pod \"2ec739d1-4d9b-4327-aa7d-d31a55fddb55\" (UID: \"2ec739d1-4d9b-4327-aa7d-d31a55fddb55\") " Dec 03 13:05:46 crc kubenswrapper[5052]: I1203 13:05:46.483689 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"openstack-edpm-tls-tls-dnsnames-second-certs-0\" (UniqueName: \"kubernetes.io/projected/2ec739d1-4d9b-4327-aa7d-d31a55fddb55-openstack-edpm-tls-tls-dnsnames-second-certs-0\") pod \"2ec739d1-4d9b-4327-aa7d-d31a55fddb55\" (UID: \"2ec739d1-4d9b-4327-aa7d-d31a55fddb55\") " Dec 03 13:05:46 crc kubenswrapper[5052]: I1203 13:05:46.483739 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/2ec739d1-4d9b-4327-aa7d-d31a55fddb55-ssh-key\") pod \"2ec739d1-4d9b-4327-aa7d-d31a55fddb55\" (UID: \"2ec739d1-4d9b-4327-aa7d-d31a55fddb55\") " Dec 03 13:05:46 crc kubenswrapper[5052]: I1203 13:05:46.483805 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"tls-dnsnames-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2ec739d1-4d9b-4327-aa7d-d31a55fddb55-tls-dnsnames-combined-ca-bundle\") pod \"2ec739d1-4d9b-4327-aa7d-d31a55fddb55\" (UID: \"2ec739d1-4d9b-4327-aa7d-d31a55fddb55\") " Dec 03 13:05:46 crc kubenswrapper[5052]: I1203 13:05:46.483847 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"install-certs-ovrd-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2ec739d1-4d9b-4327-aa7d-d31a55fddb55-install-certs-ovrd-combined-ca-bundle\") pod \"2ec739d1-4d9b-4327-aa7d-d31a55fddb55\" (UID: \"2ec739d1-4d9b-4327-aa7d-d31a55fddb55\") " Dec 03 13:05:46 crc kubenswrapper[5052]: I1203 13:05:46.483892 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/2ec739d1-4d9b-4327-aa7d-d31a55fddb55-inventory\") pod \"2ec739d1-4d9b-4327-aa7d-d31a55fddb55\" (UID: \"2ec739d1-4d9b-4327-aa7d-d31a55fddb55\") " Dec 03 13:05:46 crc kubenswrapper[5052]: I1203 13:05:46.491093 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2ec739d1-4d9b-4327-aa7d-d31a55fddb55-kube-api-access-zpz4f" (OuterVolumeSpecName: "kube-api-access-zpz4f") pod "2ec739d1-4d9b-4327-aa7d-d31a55fddb55" (UID: "2ec739d1-4d9b-4327-aa7d-d31a55fddb55"). InnerVolumeSpecName "kube-api-access-zpz4f". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:05:46 crc kubenswrapper[5052]: I1203 13:05:46.491442 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2ec739d1-4d9b-4327-aa7d-d31a55fddb55-tls-dnsnames-combined-ca-bundle" (OuterVolumeSpecName: "tls-dnsnames-combined-ca-bundle") pod "2ec739d1-4d9b-4327-aa7d-d31a55fddb55" (UID: "2ec739d1-4d9b-4327-aa7d-d31a55fddb55"). InnerVolumeSpecName "tls-dnsnames-combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:05:46 crc kubenswrapper[5052]: I1203 13:05:46.491741 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2ec739d1-4d9b-4327-aa7d-d31a55fddb55-openstack-edpm-tls-tls-dnsnames-default-certs-0" (OuterVolumeSpecName: "openstack-edpm-tls-tls-dnsnames-default-certs-0") pod "2ec739d1-4d9b-4327-aa7d-d31a55fddb55" (UID: "2ec739d1-4d9b-4327-aa7d-d31a55fddb55"). InnerVolumeSpecName "openstack-edpm-tls-tls-dnsnames-default-certs-0". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:05:46 crc kubenswrapper[5052]: I1203 13:05:46.495818 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2ec739d1-4d9b-4327-aa7d-d31a55fddb55-install-certs-ovrd-combined-ca-bundle" (OuterVolumeSpecName: "install-certs-ovrd-combined-ca-bundle") pod "2ec739d1-4d9b-4327-aa7d-d31a55fddb55" (UID: "2ec739d1-4d9b-4327-aa7d-d31a55fddb55"). InnerVolumeSpecName "install-certs-ovrd-combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:05:46 crc kubenswrapper[5052]: I1203 13:05:46.498856 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2ec739d1-4d9b-4327-aa7d-d31a55fddb55-openstack-edpm-tls-tls-dnsnames-second-certs-0" (OuterVolumeSpecName: "openstack-edpm-tls-tls-dnsnames-second-certs-0") pod "2ec739d1-4d9b-4327-aa7d-d31a55fddb55" (UID: "2ec739d1-4d9b-4327-aa7d-d31a55fddb55"). InnerVolumeSpecName "openstack-edpm-tls-tls-dnsnames-second-certs-0". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:05:46 crc kubenswrapper[5052]: I1203 13:05:46.507442 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2ec739d1-4d9b-4327-aa7d-d31a55fddb55-inventory" (OuterVolumeSpecName: "inventory") pod "2ec739d1-4d9b-4327-aa7d-d31a55fddb55" (UID: "2ec739d1-4d9b-4327-aa7d-d31a55fddb55"). InnerVolumeSpecName "inventory". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:05:46 crc kubenswrapper[5052]: I1203 13:05:46.508295 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2ec739d1-4d9b-4327-aa7d-d31a55fddb55-ssh-key" (OuterVolumeSpecName: "ssh-key") pod "2ec739d1-4d9b-4327-aa7d-d31a55fddb55" (UID: "2ec739d1-4d9b-4327-aa7d-d31a55fddb55"). InnerVolumeSpecName "ssh-key". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:05:46 crc kubenswrapper[5052]: I1203 13:05:46.585144 5052 reconciler_common.go:293] "Volume detached for volume \"tls-dnsnames-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2ec739d1-4d9b-4327-aa7d-d31a55fddb55-tls-dnsnames-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 13:05:46 crc kubenswrapper[5052]: I1203 13:05:46.585183 5052 reconciler_common.go:293] "Volume detached for volume \"install-certs-ovrd-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2ec739d1-4d9b-4327-aa7d-d31a55fddb55-install-certs-ovrd-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 13:05:46 crc kubenswrapper[5052]: I1203 13:05:46.585197 5052 reconciler_common.go:293] "Volume detached for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/2ec739d1-4d9b-4327-aa7d-d31a55fddb55-inventory\") on node \"crc\" DevicePath \"\"" Dec 03 13:05:46 crc kubenswrapper[5052]: I1203 13:05:46.585213 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-zpz4f\" (UniqueName: \"kubernetes.io/projected/2ec739d1-4d9b-4327-aa7d-d31a55fddb55-kube-api-access-zpz4f\") on node \"crc\" DevicePath \"\"" Dec 03 13:05:46 crc kubenswrapper[5052]: I1203 13:05:46.585225 5052 reconciler_common.go:293] "Volume detached for volume \"openstack-edpm-tls-tls-dnsnames-default-certs-0\" (UniqueName: \"kubernetes.io/projected/2ec739d1-4d9b-4327-aa7d-d31a55fddb55-openstack-edpm-tls-tls-dnsnames-default-certs-0\") on node \"crc\" DevicePath \"\"" Dec 03 13:05:46 crc kubenswrapper[5052]: I1203 13:05:46.585236 5052 reconciler_common.go:293] "Volume detached for volume \"openstack-edpm-tls-tls-dnsnames-second-certs-0\" (UniqueName: \"kubernetes.io/projected/2ec739d1-4d9b-4327-aa7d-d31a55fddb55-openstack-edpm-tls-tls-dnsnames-second-certs-0\") on node \"crc\" DevicePath \"\"" Dec 03 13:05:46 crc kubenswrapper[5052]: I1203 13:05:46.585246 5052 reconciler_common.go:293] "Volume detached for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/2ec739d1-4d9b-4327-aa7d-d31a55fddb55-ssh-key\") on node \"crc\" DevicePath \"\"" Dec 03 13:05:47 crc kubenswrapper[5052]: I1203 13:05:47.129034 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/install-certs-ovrd-openstack-edpm-tls-openstack-edpm-tls-ts7gt" event={"ID":"2ec739d1-4d9b-4327-aa7d-d31a55fddb55","Type":"ContainerDied","Data":"52768cbf729b86d7c2ed018548785a3c3989a36d4866e79c83d0b71b8e93dd57"} Dec 03 13:05:47 crc kubenswrapper[5052]: I1203 13:05:47.129083 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="52768cbf729b86d7c2ed018548785a3c3989a36d4866e79c83d0b71b8e93dd57" Dec 03 13:05:47 crc kubenswrapper[5052]: I1203 13:05:47.129146 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/install-certs-ovrd-openstack-edpm-tls-openstack-edpm-tls-ts7gt" Dec 03 13:05:47 crc kubenswrapper[5052]: I1203 13:05:47.209044 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/tls-dnsnames-openstack-edpm-tls-openstack-edpm-tls-mn9jk"] Dec 03 13:05:47 crc kubenswrapper[5052]: E1203 13:05:47.209759 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="2ec739d1-4d9b-4327-aa7d-d31a55fddb55" containerName="install-certs-ovrd-openstack-edpm-tls-openstack-edpm-tls" Dec 03 13:05:47 crc kubenswrapper[5052]: I1203 13:05:47.209868 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="2ec739d1-4d9b-4327-aa7d-d31a55fddb55" containerName="install-certs-ovrd-openstack-edpm-tls-openstack-edpm-tls" Dec 03 13:05:47 crc kubenswrapper[5052]: I1203 13:05:47.210147 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="2ec739d1-4d9b-4327-aa7d-d31a55fddb55" containerName="install-certs-ovrd-openstack-edpm-tls-openstack-edpm-tls" Dec 03 13:05:47 crc kubenswrapper[5052]: I1203 13:05:47.210941 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/tls-dnsnames-openstack-edpm-tls-openstack-edpm-tls-mn9jk" Dec 03 13:05:47 crc kubenswrapper[5052]: I1203 13:05:47.213917 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"openstack-edpm-tls-dockercfg-kz8sq" Dec 03 13:05:47 crc kubenswrapper[5052]: I1203 13:05:47.214297 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"combined-ca-bundle" Dec 03 13:05:47 crc kubenswrapper[5052]: I1203 13:05:47.214504 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"dataplanenodeset-openstack-edpm-tls" Dec 03 13:05:47 crc kubenswrapper[5052]: I1203 13:05:47.215060 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"openstack-aee-default-env" Dec 03 13:05:47 crc kubenswrapper[5052]: I1203 13:05:47.215254 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"dataplane-ansible-ssh-private-key-secret" Dec 03 13:05:47 crc kubenswrapper[5052]: I1203 13:05:47.227041 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/tls-dnsnames-openstack-edpm-tls-openstack-edpm-tls-mn9jk"] Dec 03 13:05:47 crc kubenswrapper[5052]: I1203 13:05:47.297111 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-dnsnames-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3d2991cb-ab4a-482e-913e-1d4110032258-tls-dnsnames-combined-ca-bundle\") pod \"tls-dnsnames-openstack-edpm-tls-openstack-edpm-tls-mn9jk\" (UID: \"3d2991cb-ab4a-482e-913e-1d4110032258\") " pod="openstack-kuttl-tests/tls-dnsnames-openstack-edpm-tls-openstack-edpm-tls-mn9jk" Dec 03 13:05:47 crc kubenswrapper[5052]: I1203 13:05:47.297205 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bzh5j\" (UniqueName: \"kubernetes.io/projected/3d2991cb-ab4a-482e-913e-1d4110032258-kube-api-access-bzh5j\") pod \"tls-dnsnames-openstack-edpm-tls-openstack-edpm-tls-mn9jk\" (UID: \"3d2991cb-ab4a-482e-913e-1d4110032258\") " pod="openstack-kuttl-tests/tls-dnsnames-openstack-edpm-tls-openstack-edpm-tls-mn9jk" Dec 03 13:05:47 crc kubenswrapper[5052]: I1203 13:05:47.297303 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/3d2991cb-ab4a-482e-913e-1d4110032258-inventory\") pod \"tls-dnsnames-openstack-edpm-tls-openstack-edpm-tls-mn9jk\" (UID: \"3d2991cb-ab4a-482e-913e-1d4110032258\") " pod="openstack-kuttl-tests/tls-dnsnames-openstack-edpm-tls-openstack-edpm-tls-mn9jk" Dec 03 13:05:47 crc kubenswrapper[5052]: I1203 13:05:47.297359 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/3d2991cb-ab4a-482e-913e-1d4110032258-ssh-key\") pod \"tls-dnsnames-openstack-edpm-tls-openstack-edpm-tls-mn9jk\" (UID: \"3d2991cb-ab4a-482e-913e-1d4110032258\") " pod="openstack-kuttl-tests/tls-dnsnames-openstack-edpm-tls-openstack-edpm-tls-mn9jk" Dec 03 13:05:47 crc kubenswrapper[5052]: I1203 13:05:47.398905 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/3d2991cb-ab4a-482e-913e-1d4110032258-inventory\") pod \"tls-dnsnames-openstack-edpm-tls-openstack-edpm-tls-mn9jk\" (UID: \"3d2991cb-ab4a-482e-913e-1d4110032258\") " pod="openstack-kuttl-tests/tls-dnsnames-openstack-edpm-tls-openstack-edpm-tls-mn9jk" Dec 03 13:05:47 crc kubenswrapper[5052]: I1203 13:05:47.399048 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/3d2991cb-ab4a-482e-913e-1d4110032258-ssh-key\") pod \"tls-dnsnames-openstack-edpm-tls-openstack-edpm-tls-mn9jk\" (UID: \"3d2991cb-ab4a-482e-913e-1d4110032258\") " pod="openstack-kuttl-tests/tls-dnsnames-openstack-edpm-tls-openstack-edpm-tls-mn9jk" Dec 03 13:05:47 crc kubenswrapper[5052]: I1203 13:05:47.399151 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"tls-dnsnames-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3d2991cb-ab4a-482e-913e-1d4110032258-tls-dnsnames-combined-ca-bundle\") pod \"tls-dnsnames-openstack-edpm-tls-openstack-edpm-tls-mn9jk\" (UID: \"3d2991cb-ab4a-482e-913e-1d4110032258\") " pod="openstack-kuttl-tests/tls-dnsnames-openstack-edpm-tls-openstack-edpm-tls-mn9jk" Dec 03 13:05:47 crc kubenswrapper[5052]: I1203 13:05:47.399195 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-bzh5j\" (UniqueName: \"kubernetes.io/projected/3d2991cb-ab4a-482e-913e-1d4110032258-kube-api-access-bzh5j\") pod \"tls-dnsnames-openstack-edpm-tls-openstack-edpm-tls-mn9jk\" (UID: \"3d2991cb-ab4a-482e-913e-1d4110032258\") " pod="openstack-kuttl-tests/tls-dnsnames-openstack-edpm-tls-openstack-edpm-tls-mn9jk" Dec 03 13:05:47 crc kubenswrapper[5052]: I1203 13:05:47.403923 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/3d2991cb-ab4a-482e-913e-1d4110032258-ssh-key\") pod \"tls-dnsnames-openstack-edpm-tls-openstack-edpm-tls-mn9jk\" (UID: \"3d2991cb-ab4a-482e-913e-1d4110032258\") " pod="openstack-kuttl-tests/tls-dnsnames-openstack-edpm-tls-openstack-edpm-tls-mn9jk" Dec 03 13:05:47 crc kubenswrapper[5052]: I1203 13:05:47.404379 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"tls-dnsnames-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3d2991cb-ab4a-482e-913e-1d4110032258-tls-dnsnames-combined-ca-bundle\") pod \"tls-dnsnames-openstack-edpm-tls-openstack-edpm-tls-mn9jk\" (UID: \"3d2991cb-ab4a-482e-913e-1d4110032258\") " pod="openstack-kuttl-tests/tls-dnsnames-openstack-edpm-tls-openstack-edpm-tls-mn9jk" Dec 03 13:05:47 crc kubenswrapper[5052]: I1203 13:05:47.404964 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/3d2991cb-ab4a-482e-913e-1d4110032258-inventory\") pod \"tls-dnsnames-openstack-edpm-tls-openstack-edpm-tls-mn9jk\" (UID: \"3d2991cb-ab4a-482e-913e-1d4110032258\") " pod="openstack-kuttl-tests/tls-dnsnames-openstack-edpm-tls-openstack-edpm-tls-mn9jk" Dec 03 13:05:47 crc kubenswrapper[5052]: I1203 13:05:47.419188 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-bzh5j\" (UniqueName: \"kubernetes.io/projected/3d2991cb-ab4a-482e-913e-1d4110032258-kube-api-access-bzh5j\") pod \"tls-dnsnames-openstack-edpm-tls-openstack-edpm-tls-mn9jk\" (UID: \"3d2991cb-ab4a-482e-913e-1d4110032258\") " pod="openstack-kuttl-tests/tls-dnsnames-openstack-edpm-tls-openstack-edpm-tls-mn9jk" Dec 03 13:05:47 crc kubenswrapper[5052]: I1203 13:05:47.543585 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/tls-dnsnames-openstack-edpm-tls-openstack-edpm-tls-mn9jk" Dec 03 13:05:47 crc kubenswrapper[5052]: I1203 13:05:47.987485 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/tls-dnsnames-openstack-edpm-tls-openstack-edpm-tls-mn9jk"] Dec 03 13:05:48 crc kubenswrapper[5052]: I1203 13:05:48.138256 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/tls-dnsnames-openstack-edpm-tls-openstack-edpm-tls-mn9jk" event={"ID":"3d2991cb-ab4a-482e-913e-1d4110032258","Type":"ContainerStarted","Data":"db769c391916a8f2d824ca4c20b05e7130e30ecedea820313e9ac8eeb9e57d41"} Dec 03 13:05:50 crc kubenswrapper[5052]: I1203 13:05:50.162482 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/tls-dnsnames-openstack-edpm-tls-openstack-edpm-tls-mn9jk" event={"ID":"3d2991cb-ab4a-482e-913e-1d4110032258","Type":"ContainerStarted","Data":"c75090bc5d8deae6a758cdfee457a3599e34c37592925eda56118f4d25373a96"} Dec 03 13:05:50 crc kubenswrapper[5052]: I1203 13:05:50.184157 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/tls-dnsnames-openstack-edpm-tls-openstack-edpm-tls-mn9jk" podStartSLOduration=2.053796405 podStartE2EDuration="3.18413461s" podCreationTimestamp="2025-12-03 13:05:47 +0000 UTC" firstStartedPulling="2025-12-03 13:05:47.996190589 +0000 UTC m=+6338.584432343" lastFinishedPulling="2025-12-03 13:05:49.126528784 +0000 UTC m=+6339.714770548" observedRunningTime="2025-12-03 13:05:50.178954652 +0000 UTC m=+6340.767196426" watchObservedRunningTime="2025-12-03 13:05:50.18413461 +0000 UTC m=+6340.772376374" Dec 03 13:05:52 crc kubenswrapper[5052]: I1203 13:05:52.196257 5052 generic.go:334] "Generic (PLEG): container finished" podID="3d2991cb-ab4a-482e-913e-1d4110032258" containerID="c75090bc5d8deae6a758cdfee457a3599e34c37592925eda56118f4d25373a96" exitCode=0 Dec 03 13:05:52 crc kubenswrapper[5052]: I1203 13:05:52.196374 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/tls-dnsnames-openstack-edpm-tls-openstack-edpm-tls-mn9jk" event={"ID":"3d2991cb-ab4a-482e-913e-1d4110032258","Type":"ContainerDied","Data":"c75090bc5d8deae6a758cdfee457a3599e34c37592925eda56118f4d25373a96"} Dec 03 13:05:53 crc kubenswrapper[5052]: I1203 13:05:53.501904 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/tls-dnsnames-openstack-edpm-tls-openstack-edpm-tls-mn9jk" Dec 03 13:05:53 crc kubenswrapper[5052]: I1203 13:05:53.601426 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/3d2991cb-ab4a-482e-913e-1d4110032258-ssh-key\") pod \"3d2991cb-ab4a-482e-913e-1d4110032258\" (UID: \"3d2991cb-ab4a-482e-913e-1d4110032258\") " Dec 03 13:05:53 crc kubenswrapper[5052]: I1203 13:05:53.601571 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"tls-dnsnames-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3d2991cb-ab4a-482e-913e-1d4110032258-tls-dnsnames-combined-ca-bundle\") pod \"3d2991cb-ab4a-482e-913e-1d4110032258\" (UID: \"3d2991cb-ab4a-482e-913e-1d4110032258\") " Dec 03 13:05:53 crc kubenswrapper[5052]: I1203 13:05:53.601702 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bzh5j\" (UniqueName: \"kubernetes.io/projected/3d2991cb-ab4a-482e-913e-1d4110032258-kube-api-access-bzh5j\") pod \"3d2991cb-ab4a-482e-913e-1d4110032258\" (UID: \"3d2991cb-ab4a-482e-913e-1d4110032258\") " Dec 03 13:05:53 crc kubenswrapper[5052]: I1203 13:05:53.601744 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/3d2991cb-ab4a-482e-913e-1d4110032258-inventory\") pod \"3d2991cb-ab4a-482e-913e-1d4110032258\" (UID: \"3d2991cb-ab4a-482e-913e-1d4110032258\") " Dec 03 13:05:53 crc kubenswrapper[5052]: I1203 13:05:53.611972 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3d2991cb-ab4a-482e-913e-1d4110032258-tls-dnsnames-combined-ca-bundle" (OuterVolumeSpecName: "tls-dnsnames-combined-ca-bundle") pod "3d2991cb-ab4a-482e-913e-1d4110032258" (UID: "3d2991cb-ab4a-482e-913e-1d4110032258"). InnerVolumeSpecName "tls-dnsnames-combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:05:53 crc kubenswrapper[5052]: I1203 13:05:53.612445 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3d2991cb-ab4a-482e-913e-1d4110032258-kube-api-access-bzh5j" (OuterVolumeSpecName: "kube-api-access-bzh5j") pod "3d2991cb-ab4a-482e-913e-1d4110032258" (UID: "3d2991cb-ab4a-482e-913e-1d4110032258"). InnerVolumeSpecName "kube-api-access-bzh5j". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:05:53 crc kubenswrapper[5052]: E1203 13:05:53.624692 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3d2991cb-ab4a-482e-913e-1d4110032258-inventory podName:3d2991cb-ab4a-482e-913e-1d4110032258 nodeName:}" failed. No retries permitted until 2025-12-03 13:05:54.124652253 +0000 UTC m=+6344.712894017 (durationBeforeRetry 500ms). Error: error cleaning subPath mounts for volume "inventory" (UniqueName: "kubernetes.io/secret/3d2991cb-ab4a-482e-913e-1d4110032258-inventory") pod "3d2991cb-ab4a-482e-913e-1d4110032258" (UID: "3d2991cb-ab4a-482e-913e-1d4110032258") : error deleting /var/lib/kubelet/pods/3d2991cb-ab4a-482e-913e-1d4110032258/volume-subpaths: remove /var/lib/kubelet/pods/3d2991cb-ab4a-482e-913e-1d4110032258/volume-subpaths: no such file or directory Dec 03 13:05:53 crc kubenswrapper[5052]: I1203 13:05:53.627360 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3d2991cb-ab4a-482e-913e-1d4110032258-ssh-key" (OuterVolumeSpecName: "ssh-key") pod "3d2991cb-ab4a-482e-913e-1d4110032258" (UID: "3d2991cb-ab4a-482e-913e-1d4110032258"). InnerVolumeSpecName "ssh-key". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:05:53 crc kubenswrapper[5052]: I1203 13:05:53.703540 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-bzh5j\" (UniqueName: \"kubernetes.io/projected/3d2991cb-ab4a-482e-913e-1d4110032258-kube-api-access-bzh5j\") on node \"crc\" DevicePath \"\"" Dec 03 13:05:53 crc kubenswrapper[5052]: I1203 13:05:53.704307 5052 reconciler_common.go:293] "Volume detached for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/3d2991cb-ab4a-482e-913e-1d4110032258-ssh-key\") on node \"crc\" DevicePath \"\"" Dec 03 13:05:53 crc kubenswrapper[5052]: I1203 13:05:53.704327 5052 reconciler_common.go:293] "Volume detached for volume \"tls-dnsnames-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/3d2991cb-ab4a-482e-913e-1d4110032258-tls-dnsnames-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 13:05:54 crc kubenswrapper[5052]: I1203 13:05:54.210661 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/3d2991cb-ab4a-482e-913e-1d4110032258-inventory\") pod \"3d2991cb-ab4a-482e-913e-1d4110032258\" (UID: \"3d2991cb-ab4a-482e-913e-1d4110032258\") " Dec 03 13:05:54 crc kubenswrapper[5052]: I1203 13:05:54.215614 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3d2991cb-ab4a-482e-913e-1d4110032258-inventory" (OuterVolumeSpecName: "inventory") pod "3d2991cb-ab4a-482e-913e-1d4110032258" (UID: "3d2991cb-ab4a-482e-913e-1d4110032258"). InnerVolumeSpecName "inventory". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:05:54 crc kubenswrapper[5052]: I1203 13:05:54.218960 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/tls-dnsnames-openstack-edpm-tls-openstack-edpm-tls-mn9jk" event={"ID":"3d2991cb-ab4a-482e-913e-1d4110032258","Type":"ContainerDied","Data":"db769c391916a8f2d824ca4c20b05e7130e30ecedea820313e9ac8eeb9e57d41"} Dec 03 13:05:54 crc kubenswrapper[5052]: I1203 13:05:54.219117 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="db769c391916a8f2d824ca4c20b05e7130e30ecedea820313e9ac8eeb9e57d41" Dec 03 13:05:54 crc kubenswrapper[5052]: I1203 13:05:54.219040 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/tls-dnsnames-openstack-edpm-tls-openstack-edpm-tls-mn9jk" Dec 03 13:05:54 crc kubenswrapper[5052]: I1203 13:05:54.312492 5052 reconciler_common.go:293] "Volume detached for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/3d2991cb-ab4a-482e-913e-1d4110032258-inventory\") on node \"crc\" DevicePath \"\"" Dec 03 13:05:56 crc kubenswrapper[5052]: I1203 13:05:56.654742 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/install-certs-ovrd-openstack-edpm-tls-ovrd-openstack-edpm-l6wm4"] Dec 03 13:05:56 crc kubenswrapper[5052]: E1203 13:05:56.655134 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="3d2991cb-ab4a-482e-913e-1d4110032258" containerName="tls-dnsnames-openstack-edpm-tls-openstack-edpm-tls" Dec 03 13:05:56 crc kubenswrapper[5052]: I1203 13:05:56.655148 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="3d2991cb-ab4a-482e-913e-1d4110032258" containerName="tls-dnsnames-openstack-edpm-tls-openstack-edpm-tls" Dec 03 13:05:56 crc kubenswrapper[5052]: I1203 13:05:56.655319 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="3d2991cb-ab4a-482e-913e-1d4110032258" containerName="tls-dnsnames-openstack-edpm-tls-openstack-edpm-tls" Dec 03 13:05:56 crc kubenswrapper[5052]: I1203 13:05:56.655884 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/install-certs-ovrd-openstack-edpm-tls-ovrd-openstack-edpm-l6wm4" Dec 03 13:05:56 crc kubenswrapper[5052]: I1203 13:05:56.657714 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"openstack-aee-default-env" Dec 03 13:05:56 crc kubenswrapper[5052]: I1203 13:05:56.659245 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"dataplanenodeset-openstack-edpm-tls" Dec 03 13:05:56 crc kubenswrapper[5052]: I1203 13:05:56.659337 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"openstack-edpm-tls-tls-dns-ips-default-certs-0" Dec 03 13:05:56 crc kubenswrapper[5052]: I1203 13:05:56.659419 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"openstack-edpm-tls-dockercfg-kz8sq" Dec 03 13:05:56 crc kubenswrapper[5052]: I1203 13:05:56.659548 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"openstack-edpm-tls-custom-tls-dns-default-certs-0" Dec 03 13:05:56 crc kubenswrapper[5052]: I1203 13:05:56.659809 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"dataplane-ansible-ssh-private-key-secret" Dec 03 13:05:56 crc kubenswrapper[5052]: I1203 13:05:56.660941 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"combined-ca-bundle" Dec 03 13:05:56 crc kubenswrapper[5052]: I1203 13:05:56.673172 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/install-certs-ovrd-openstack-edpm-tls-ovrd-openstack-edpm-l6wm4"] Dec 03 13:05:56 crc kubenswrapper[5052]: I1203 13:05:56.748018 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/be554003-60cb-4ea9-9cce-d5a0b6f3dbaa-ssh-key\") pod \"install-certs-ovrd-openstack-edpm-tls-ovrd-openstack-edpm-l6wm4\" (UID: \"be554003-60cb-4ea9-9cce-d5a0b6f3dbaa\") " pod="openstack-kuttl-tests/install-certs-ovrd-openstack-edpm-tls-ovrd-openstack-edpm-l6wm4" Dec 03 13:05:56 crc kubenswrapper[5052]: I1203 13:05:56.748071 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"custom-tls-dns-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/be554003-60cb-4ea9-9cce-d5a0b6f3dbaa-custom-tls-dns-combined-ca-bundle\") pod \"install-certs-ovrd-openstack-edpm-tls-ovrd-openstack-edpm-l6wm4\" (UID: \"be554003-60cb-4ea9-9cce-d5a0b6f3dbaa\") " pod="openstack-kuttl-tests/install-certs-ovrd-openstack-edpm-tls-ovrd-openstack-edpm-l6wm4" Dec 03 13:05:56 crc kubenswrapper[5052]: I1203 13:05:56.748224 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openstack-edpm-tls-tls-dns-ips-default-certs-0\" (UniqueName: \"kubernetes.io/projected/be554003-60cb-4ea9-9cce-d5a0b6f3dbaa-openstack-edpm-tls-tls-dns-ips-default-certs-0\") pod \"install-certs-ovrd-openstack-edpm-tls-ovrd-openstack-edpm-l6wm4\" (UID: \"be554003-60cb-4ea9-9cce-d5a0b6f3dbaa\") " pod="openstack-kuttl-tests/install-certs-ovrd-openstack-edpm-tls-ovrd-openstack-edpm-l6wm4" Dec 03 13:05:56 crc kubenswrapper[5052]: I1203 13:05:56.748439 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-npltc\" (UniqueName: \"kubernetes.io/projected/be554003-60cb-4ea9-9cce-d5a0b6f3dbaa-kube-api-access-npltc\") pod \"install-certs-ovrd-openstack-edpm-tls-ovrd-openstack-edpm-l6wm4\" (UID: \"be554003-60cb-4ea9-9cce-d5a0b6f3dbaa\") " pod="openstack-kuttl-tests/install-certs-ovrd-openstack-edpm-tls-ovrd-openstack-edpm-l6wm4" Dec 03 13:05:56 crc kubenswrapper[5052]: I1203 13:05:56.748526 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-dns-ips-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/be554003-60cb-4ea9-9cce-d5a0b6f3dbaa-tls-dns-ips-combined-ca-bundle\") pod \"install-certs-ovrd-openstack-edpm-tls-ovrd-openstack-edpm-l6wm4\" (UID: \"be554003-60cb-4ea9-9cce-d5a0b6f3dbaa\") " pod="openstack-kuttl-tests/install-certs-ovrd-openstack-edpm-tls-ovrd-openstack-edpm-l6wm4" Dec 03 13:05:56 crc kubenswrapper[5052]: I1203 13:05:56.748589 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/be554003-60cb-4ea9-9cce-d5a0b6f3dbaa-inventory\") pod \"install-certs-ovrd-openstack-edpm-tls-ovrd-openstack-edpm-l6wm4\" (UID: \"be554003-60cb-4ea9-9cce-d5a0b6f3dbaa\") " pod="openstack-kuttl-tests/install-certs-ovrd-openstack-edpm-tls-ovrd-openstack-edpm-l6wm4" Dec 03 13:05:56 crc kubenswrapper[5052]: I1203 13:05:56.748692 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openstack-edpm-tls-custom-tls-dns-default-certs-0\" (UniqueName: \"kubernetes.io/projected/be554003-60cb-4ea9-9cce-d5a0b6f3dbaa-openstack-edpm-tls-custom-tls-dns-default-certs-0\") pod \"install-certs-ovrd-openstack-edpm-tls-ovrd-openstack-edpm-l6wm4\" (UID: \"be554003-60cb-4ea9-9cce-d5a0b6f3dbaa\") " pod="openstack-kuttl-tests/install-certs-ovrd-openstack-edpm-tls-ovrd-openstack-edpm-l6wm4" Dec 03 13:05:56 crc kubenswrapper[5052]: I1203 13:05:56.748752 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"install-certs-ovrd-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/be554003-60cb-4ea9-9cce-d5a0b6f3dbaa-install-certs-ovrd-combined-ca-bundle\") pod \"install-certs-ovrd-openstack-edpm-tls-ovrd-openstack-edpm-l6wm4\" (UID: \"be554003-60cb-4ea9-9cce-d5a0b6f3dbaa\") " pod="openstack-kuttl-tests/install-certs-ovrd-openstack-edpm-tls-ovrd-openstack-edpm-l6wm4" Dec 03 13:05:56 crc kubenswrapper[5052]: I1203 13:05:56.850137 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-npltc\" (UniqueName: \"kubernetes.io/projected/be554003-60cb-4ea9-9cce-d5a0b6f3dbaa-kube-api-access-npltc\") pod \"install-certs-ovrd-openstack-edpm-tls-ovrd-openstack-edpm-l6wm4\" (UID: \"be554003-60cb-4ea9-9cce-d5a0b6f3dbaa\") " pod="openstack-kuttl-tests/install-certs-ovrd-openstack-edpm-tls-ovrd-openstack-edpm-l6wm4" Dec 03 13:05:56 crc kubenswrapper[5052]: I1203 13:05:56.850236 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"tls-dns-ips-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/be554003-60cb-4ea9-9cce-d5a0b6f3dbaa-tls-dns-ips-combined-ca-bundle\") pod \"install-certs-ovrd-openstack-edpm-tls-ovrd-openstack-edpm-l6wm4\" (UID: \"be554003-60cb-4ea9-9cce-d5a0b6f3dbaa\") " pod="openstack-kuttl-tests/install-certs-ovrd-openstack-edpm-tls-ovrd-openstack-edpm-l6wm4" Dec 03 13:05:56 crc kubenswrapper[5052]: I1203 13:05:56.850289 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/be554003-60cb-4ea9-9cce-d5a0b6f3dbaa-inventory\") pod \"install-certs-ovrd-openstack-edpm-tls-ovrd-openstack-edpm-l6wm4\" (UID: \"be554003-60cb-4ea9-9cce-d5a0b6f3dbaa\") " pod="openstack-kuttl-tests/install-certs-ovrd-openstack-edpm-tls-ovrd-openstack-edpm-l6wm4" Dec 03 13:05:56 crc kubenswrapper[5052]: I1203 13:05:56.850342 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"openstack-edpm-tls-custom-tls-dns-default-certs-0\" (UniqueName: \"kubernetes.io/projected/be554003-60cb-4ea9-9cce-d5a0b6f3dbaa-openstack-edpm-tls-custom-tls-dns-default-certs-0\") pod \"install-certs-ovrd-openstack-edpm-tls-ovrd-openstack-edpm-l6wm4\" (UID: \"be554003-60cb-4ea9-9cce-d5a0b6f3dbaa\") " pod="openstack-kuttl-tests/install-certs-ovrd-openstack-edpm-tls-ovrd-openstack-edpm-l6wm4" Dec 03 13:05:56 crc kubenswrapper[5052]: I1203 13:05:56.850381 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"install-certs-ovrd-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/be554003-60cb-4ea9-9cce-d5a0b6f3dbaa-install-certs-ovrd-combined-ca-bundle\") pod \"install-certs-ovrd-openstack-edpm-tls-ovrd-openstack-edpm-l6wm4\" (UID: \"be554003-60cb-4ea9-9cce-d5a0b6f3dbaa\") " pod="openstack-kuttl-tests/install-certs-ovrd-openstack-edpm-tls-ovrd-openstack-edpm-l6wm4" Dec 03 13:05:56 crc kubenswrapper[5052]: I1203 13:05:56.850439 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/be554003-60cb-4ea9-9cce-d5a0b6f3dbaa-ssh-key\") pod \"install-certs-ovrd-openstack-edpm-tls-ovrd-openstack-edpm-l6wm4\" (UID: \"be554003-60cb-4ea9-9cce-d5a0b6f3dbaa\") " pod="openstack-kuttl-tests/install-certs-ovrd-openstack-edpm-tls-ovrd-openstack-edpm-l6wm4" Dec 03 13:05:56 crc kubenswrapper[5052]: I1203 13:05:56.850470 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"custom-tls-dns-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/be554003-60cb-4ea9-9cce-d5a0b6f3dbaa-custom-tls-dns-combined-ca-bundle\") pod \"install-certs-ovrd-openstack-edpm-tls-ovrd-openstack-edpm-l6wm4\" (UID: \"be554003-60cb-4ea9-9cce-d5a0b6f3dbaa\") " pod="openstack-kuttl-tests/install-certs-ovrd-openstack-edpm-tls-ovrd-openstack-edpm-l6wm4" Dec 03 13:05:56 crc kubenswrapper[5052]: I1203 13:05:56.850528 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"openstack-edpm-tls-tls-dns-ips-default-certs-0\" (UniqueName: \"kubernetes.io/projected/be554003-60cb-4ea9-9cce-d5a0b6f3dbaa-openstack-edpm-tls-tls-dns-ips-default-certs-0\") pod \"install-certs-ovrd-openstack-edpm-tls-ovrd-openstack-edpm-l6wm4\" (UID: \"be554003-60cb-4ea9-9cce-d5a0b6f3dbaa\") " pod="openstack-kuttl-tests/install-certs-ovrd-openstack-edpm-tls-ovrd-openstack-edpm-l6wm4" Dec 03 13:05:56 crc kubenswrapper[5052]: I1203 13:05:56.855649 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/be554003-60cb-4ea9-9cce-d5a0b6f3dbaa-ssh-key\") pod \"install-certs-ovrd-openstack-edpm-tls-ovrd-openstack-edpm-l6wm4\" (UID: \"be554003-60cb-4ea9-9cce-d5a0b6f3dbaa\") " pod="openstack-kuttl-tests/install-certs-ovrd-openstack-edpm-tls-ovrd-openstack-edpm-l6wm4" Dec 03 13:05:56 crc kubenswrapper[5052]: I1203 13:05:56.858081 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/be554003-60cb-4ea9-9cce-d5a0b6f3dbaa-inventory\") pod \"install-certs-ovrd-openstack-edpm-tls-ovrd-openstack-edpm-l6wm4\" (UID: \"be554003-60cb-4ea9-9cce-d5a0b6f3dbaa\") " pod="openstack-kuttl-tests/install-certs-ovrd-openstack-edpm-tls-ovrd-openstack-edpm-l6wm4" Dec 03 13:05:56 crc kubenswrapper[5052]: I1203 13:05:56.858291 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"openstack-edpm-tls-custom-tls-dns-default-certs-0\" (UniqueName: \"kubernetes.io/projected/be554003-60cb-4ea9-9cce-d5a0b6f3dbaa-openstack-edpm-tls-custom-tls-dns-default-certs-0\") pod \"install-certs-ovrd-openstack-edpm-tls-ovrd-openstack-edpm-l6wm4\" (UID: \"be554003-60cb-4ea9-9cce-d5a0b6f3dbaa\") " pod="openstack-kuttl-tests/install-certs-ovrd-openstack-edpm-tls-ovrd-openstack-edpm-l6wm4" Dec 03 13:05:56 crc kubenswrapper[5052]: I1203 13:05:56.858401 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"custom-tls-dns-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/be554003-60cb-4ea9-9cce-d5a0b6f3dbaa-custom-tls-dns-combined-ca-bundle\") pod \"install-certs-ovrd-openstack-edpm-tls-ovrd-openstack-edpm-l6wm4\" (UID: \"be554003-60cb-4ea9-9cce-d5a0b6f3dbaa\") " pod="openstack-kuttl-tests/install-certs-ovrd-openstack-edpm-tls-ovrd-openstack-edpm-l6wm4" Dec 03 13:05:56 crc kubenswrapper[5052]: I1203 13:05:56.859079 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"install-certs-ovrd-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/be554003-60cb-4ea9-9cce-d5a0b6f3dbaa-install-certs-ovrd-combined-ca-bundle\") pod \"install-certs-ovrd-openstack-edpm-tls-ovrd-openstack-edpm-l6wm4\" (UID: \"be554003-60cb-4ea9-9cce-d5a0b6f3dbaa\") " pod="openstack-kuttl-tests/install-certs-ovrd-openstack-edpm-tls-ovrd-openstack-edpm-l6wm4" Dec 03 13:05:56 crc kubenswrapper[5052]: I1203 13:05:56.859279 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"tls-dns-ips-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/be554003-60cb-4ea9-9cce-d5a0b6f3dbaa-tls-dns-ips-combined-ca-bundle\") pod \"install-certs-ovrd-openstack-edpm-tls-ovrd-openstack-edpm-l6wm4\" (UID: \"be554003-60cb-4ea9-9cce-d5a0b6f3dbaa\") " pod="openstack-kuttl-tests/install-certs-ovrd-openstack-edpm-tls-ovrd-openstack-edpm-l6wm4" Dec 03 13:05:56 crc kubenswrapper[5052]: I1203 13:05:56.859576 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"openstack-edpm-tls-tls-dns-ips-default-certs-0\" (UniqueName: \"kubernetes.io/projected/be554003-60cb-4ea9-9cce-d5a0b6f3dbaa-openstack-edpm-tls-tls-dns-ips-default-certs-0\") pod \"install-certs-ovrd-openstack-edpm-tls-ovrd-openstack-edpm-l6wm4\" (UID: \"be554003-60cb-4ea9-9cce-d5a0b6f3dbaa\") " pod="openstack-kuttl-tests/install-certs-ovrd-openstack-edpm-tls-ovrd-openstack-edpm-l6wm4" Dec 03 13:05:56 crc kubenswrapper[5052]: I1203 13:05:56.876772 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-npltc\" (UniqueName: \"kubernetes.io/projected/be554003-60cb-4ea9-9cce-d5a0b6f3dbaa-kube-api-access-npltc\") pod \"install-certs-ovrd-openstack-edpm-tls-ovrd-openstack-edpm-l6wm4\" (UID: \"be554003-60cb-4ea9-9cce-d5a0b6f3dbaa\") " pod="openstack-kuttl-tests/install-certs-ovrd-openstack-edpm-tls-ovrd-openstack-edpm-l6wm4" Dec 03 13:05:56 crc kubenswrapper[5052]: I1203 13:05:56.975546 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/install-certs-ovrd-openstack-edpm-tls-ovrd-openstack-edpm-l6wm4" Dec 03 13:05:57 crc kubenswrapper[5052]: I1203 13:05:57.999138 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/install-certs-ovrd-openstack-edpm-tls-ovrd-openstack-edpm-l6wm4"] Dec 03 13:05:58 crc kubenswrapper[5052]: I1203 13:05:58.261105 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/install-certs-ovrd-openstack-edpm-tls-ovrd-openstack-edpm-l6wm4" event={"ID":"be554003-60cb-4ea9-9cce-d5a0b6f3dbaa","Type":"ContainerStarted","Data":"7bb35b246d31e28092163cc805b8d65f89eeb4499b20ccc0546f68c702c4e8bf"} Dec 03 13:05:59 crc kubenswrapper[5052]: I1203 13:05:59.272070 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/install-certs-ovrd-openstack-edpm-tls-ovrd-openstack-edpm-l6wm4" event={"ID":"be554003-60cb-4ea9-9cce-d5a0b6f3dbaa","Type":"ContainerStarted","Data":"c87489961ab06bb10e01de5fc388749bce75be1a64c04c0510bbfab5ab2f2a46"} Dec 03 13:05:59 crc kubenswrapper[5052]: I1203 13:05:59.301724 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/install-certs-ovrd-openstack-edpm-tls-ovrd-openstack-edpm-l6wm4" podStartSLOduration=2.773693346 podStartE2EDuration="3.301706626s" podCreationTimestamp="2025-12-03 13:05:56 +0000 UTC" firstStartedPulling="2025-12-03 13:05:58.01469134 +0000 UTC m=+6348.602933114" lastFinishedPulling="2025-12-03 13:05:58.54270463 +0000 UTC m=+6349.130946394" observedRunningTime="2025-12-03 13:05:59.30009954 +0000 UTC m=+6349.888341304" watchObservedRunningTime="2025-12-03 13:05:59.301706626 +0000 UTC m=+6349.889948390" Dec 03 13:05:59 crc kubenswrapper[5052]: I1203 13:05:59.379831 5052 patch_prober.go:28] interesting pod/machine-config-daemon-74bj6 container/machine-config-daemon namespace/openshift-machine-config-operator: Liveness probe status=failure output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" start-of-body= Dec 03 13:05:59 crc kubenswrapper[5052]: I1203 13:05:59.379902 5052 prober.go:107] "Probe failed" probeType="Liveness" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" containerName="machine-config-daemon" probeResult="failure" output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" Dec 03 13:06:02 crc kubenswrapper[5052]: I1203 13:06:02.299160 5052 generic.go:334] "Generic (PLEG): container finished" podID="be554003-60cb-4ea9-9cce-d5a0b6f3dbaa" containerID="c87489961ab06bb10e01de5fc388749bce75be1a64c04c0510bbfab5ab2f2a46" exitCode=0 Dec 03 13:06:02 crc kubenswrapper[5052]: I1203 13:06:02.299250 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/install-certs-ovrd-openstack-edpm-tls-ovrd-openstack-edpm-l6wm4" event={"ID":"be554003-60cb-4ea9-9cce-d5a0b6f3dbaa","Type":"ContainerDied","Data":"c87489961ab06bb10e01de5fc388749bce75be1a64c04c0510bbfab5ab2f2a46"} Dec 03 13:06:03 crc kubenswrapper[5052]: I1203 13:06:03.587569 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/install-certs-ovrd-openstack-edpm-tls-ovrd-openstack-edpm-l6wm4" Dec 03 13:06:03 crc kubenswrapper[5052]: I1203 13:06:03.788468 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/be554003-60cb-4ea9-9cce-d5a0b6f3dbaa-inventory\") pod \"be554003-60cb-4ea9-9cce-d5a0b6f3dbaa\" (UID: \"be554003-60cb-4ea9-9cce-d5a0b6f3dbaa\") " Dec 03 13:06:03 crc kubenswrapper[5052]: I1203 13:06:03.788560 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"install-certs-ovrd-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/be554003-60cb-4ea9-9cce-d5a0b6f3dbaa-install-certs-ovrd-combined-ca-bundle\") pod \"be554003-60cb-4ea9-9cce-d5a0b6f3dbaa\" (UID: \"be554003-60cb-4ea9-9cce-d5a0b6f3dbaa\") " Dec 03 13:06:03 crc kubenswrapper[5052]: I1203 13:06:03.788601 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"tls-dns-ips-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/be554003-60cb-4ea9-9cce-d5a0b6f3dbaa-tls-dns-ips-combined-ca-bundle\") pod \"be554003-60cb-4ea9-9cce-d5a0b6f3dbaa\" (UID: \"be554003-60cb-4ea9-9cce-d5a0b6f3dbaa\") " Dec 03 13:06:03 crc kubenswrapper[5052]: I1203 13:06:03.788630 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/be554003-60cb-4ea9-9cce-d5a0b6f3dbaa-ssh-key\") pod \"be554003-60cb-4ea9-9cce-d5a0b6f3dbaa\" (UID: \"be554003-60cb-4ea9-9cce-d5a0b6f3dbaa\") " Dec 03 13:06:03 crc kubenswrapper[5052]: I1203 13:06:03.788652 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"openstack-edpm-tls-custom-tls-dns-default-certs-0\" (UniqueName: \"kubernetes.io/projected/be554003-60cb-4ea9-9cce-d5a0b6f3dbaa-openstack-edpm-tls-custom-tls-dns-default-certs-0\") pod \"be554003-60cb-4ea9-9cce-d5a0b6f3dbaa\" (UID: \"be554003-60cb-4ea9-9cce-d5a0b6f3dbaa\") " Dec 03 13:06:03 crc kubenswrapper[5052]: I1203 13:06:03.788700 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"openstack-edpm-tls-tls-dns-ips-default-certs-0\" (UniqueName: \"kubernetes.io/projected/be554003-60cb-4ea9-9cce-d5a0b6f3dbaa-openstack-edpm-tls-tls-dns-ips-default-certs-0\") pod \"be554003-60cb-4ea9-9cce-d5a0b6f3dbaa\" (UID: \"be554003-60cb-4ea9-9cce-d5a0b6f3dbaa\") " Dec 03 13:06:03 crc kubenswrapper[5052]: I1203 13:06:03.788729 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-npltc\" (UniqueName: \"kubernetes.io/projected/be554003-60cb-4ea9-9cce-d5a0b6f3dbaa-kube-api-access-npltc\") pod \"be554003-60cb-4ea9-9cce-d5a0b6f3dbaa\" (UID: \"be554003-60cb-4ea9-9cce-d5a0b6f3dbaa\") " Dec 03 13:06:03 crc kubenswrapper[5052]: I1203 13:06:03.788771 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"custom-tls-dns-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/be554003-60cb-4ea9-9cce-d5a0b6f3dbaa-custom-tls-dns-combined-ca-bundle\") pod \"be554003-60cb-4ea9-9cce-d5a0b6f3dbaa\" (UID: \"be554003-60cb-4ea9-9cce-d5a0b6f3dbaa\") " Dec 03 13:06:03 crc kubenswrapper[5052]: I1203 13:06:03.794384 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/be554003-60cb-4ea9-9cce-d5a0b6f3dbaa-custom-tls-dns-combined-ca-bundle" (OuterVolumeSpecName: "custom-tls-dns-combined-ca-bundle") pod "be554003-60cb-4ea9-9cce-d5a0b6f3dbaa" (UID: "be554003-60cb-4ea9-9cce-d5a0b6f3dbaa"). InnerVolumeSpecName "custom-tls-dns-combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:06:03 crc kubenswrapper[5052]: I1203 13:06:03.794691 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/be554003-60cb-4ea9-9cce-d5a0b6f3dbaa-openstack-edpm-tls-custom-tls-dns-default-certs-0" (OuterVolumeSpecName: "openstack-edpm-tls-custom-tls-dns-default-certs-0") pod "be554003-60cb-4ea9-9cce-d5a0b6f3dbaa" (UID: "be554003-60cb-4ea9-9cce-d5a0b6f3dbaa"). InnerVolumeSpecName "openstack-edpm-tls-custom-tls-dns-default-certs-0". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:06:03 crc kubenswrapper[5052]: I1203 13:06:03.794743 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/be554003-60cb-4ea9-9cce-d5a0b6f3dbaa-install-certs-ovrd-combined-ca-bundle" (OuterVolumeSpecName: "install-certs-ovrd-combined-ca-bundle") pod "be554003-60cb-4ea9-9cce-d5a0b6f3dbaa" (UID: "be554003-60cb-4ea9-9cce-d5a0b6f3dbaa"). InnerVolumeSpecName "install-certs-ovrd-combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:06:03 crc kubenswrapper[5052]: I1203 13:06:03.795042 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/be554003-60cb-4ea9-9cce-d5a0b6f3dbaa-kube-api-access-npltc" (OuterVolumeSpecName: "kube-api-access-npltc") pod "be554003-60cb-4ea9-9cce-d5a0b6f3dbaa" (UID: "be554003-60cb-4ea9-9cce-d5a0b6f3dbaa"). InnerVolumeSpecName "kube-api-access-npltc". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:06:03 crc kubenswrapper[5052]: I1203 13:06:03.799785 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/be554003-60cb-4ea9-9cce-d5a0b6f3dbaa-openstack-edpm-tls-tls-dns-ips-default-certs-0" (OuterVolumeSpecName: "openstack-edpm-tls-tls-dns-ips-default-certs-0") pod "be554003-60cb-4ea9-9cce-d5a0b6f3dbaa" (UID: "be554003-60cb-4ea9-9cce-d5a0b6f3dbaa"). InnerVolumeSpecName "openstack-edpm-tls-tls-dns-ips-default-certs-0". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:06:03 crc kubenswrapper[5052]: I1203 13:06:03.806398 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/be554003-60cb-4ea9-9cce-d5a0b6f3dbaa-tls-dns-ips-combined-ca-bundle" (OuterVolumeSpecName: "tls-dns-ips-combined-ca-bundle") pod "be554003-60cb-4ea9-9cce-d5a0b6f3dbaa" (UID: "be554003-60cb-4ea9-9cce-d5a0b6f3dbaa"). InnerVolumeSpecName "tls-dns-ips-combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:06:03 crc kubenswrapper[5052]: I1203 13:06:03.827994 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/be554003-60cb-4ea9-9cce-d5a0b6f3dbaa-inventory" (OuterVolumeSpecName: "inventory") pod "be554003-60cb-4ea9-9cce-d5a0b6f3dbaa" (UID: "be554003-60cb-4ea9-9cce-d5a0b6f3dbaa"). InnerVolumeSpecName "inventory". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:06:03 crc kubenswrapper[5052]: I1203 13:06:03.828729 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/be554003-60cb-4ea9-9cce-d5a0b6f3dbaa-ssh-key" (OuterVolumeSpecName: "ssh-key") pod "be554003-60cb-4ea9-9cce-d5a0b6f3dbaa" (UID: "be554003-60cb-4ea9-9cce-d5a0b6f3dbaa"). InnerVolumeSpecName "ssh-key". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:06:03 crc kubenswrapper[5052]: I1203 13:06:03.890893 5052 reconciler_common.go:293] "Volume detached for volume \"openstack-edpm-tls-tls-dns-ips-default-certs-0\" (UniqueName: \"kubernetes.io/projected/be554003-60cb-4ea9-9cce-d5a0b6f3dbaa-openstack-edpm-tls-tls-dns-ips-default-certs-0\") on node \"crc\" DevicePath \"\"" Dec 03 13:06:03 crc kubenswrapper[5052]: I1203 13:06:03.890939 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-npltc\" (UniqueName: \"kubernetes.io/projected/be554003-60cb-4ea9-9cce-d5a0b6f3dbaa-kube-api-access-npltc\") on node \"crc\" DevicePath \"\"" Dec 03 13:06:03 crc kubenswrapper[5052]: I1203 13:06:03.890954 5052 reconciler_common.go:293] "Volume detached for volume \"custom-tls-dns-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/be554003-60cb-4ea9-9cce-d5a0b6f3dbaa-custom-tls-dns-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 13:06:03 crc kubenswrapper[5052]: I1203 13:06:03.890968 5052 reconciler_common.go:293] "Volume detached for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/be554003-60cb-4ea9-9cce-d5a0b6f3dbaa-inventory\") on node \"crc\" DevicePath \"\"" Dec 03 13:06:03 crc kubenswrapper[5052]: I1203 13:06:03.890979 5052 reconciler_common.go:293] "Volume detached for volume \"install-certs-ovrd-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/be554003-60cb-4ea9-9cce-d5a0b6f3dbaa-install-certs-ovrd-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 13:06:03 crc kubenswrapper[5052]: I1203 13:06:03.890992 5052 reconciler_common.go:293] "Volume detached for volume \"tls-dns-ips-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/be554003-60cb-4ea9-9cce-d5a0b6f3dbaa-tls-dns-ips-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 13:06:03 crc kubenswrapper[5052]: I1203 13:06:03.891003 5052 reconciler_common.go:293] "Volume detached for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/be554003-60cb-4ea9-9cce-d5a0b6f3dbaa-ssh-key\") on node \"crc\" DevicePath \"\"" Dec 03 13:06:03 crc kubenswrapper[5052]: I1203 13:06:03.891016 5052 reconciler_common.go:293] "Volume detached for volume \"openstack-edpm-tls-custom-tls-dns-default-certs-0\" (UniqueName: \"kubernetes.io/projected/be554003-60cb-4ea9-9cce-d5a0b6f3dbaa-openstack-edpm-tls-custom-tls-dns-default-certs-0\") on node \"crc\" DevicePath \"\"" Dec 03 13:06:04 crc kubenswrapper[5052]: I1203 13:06:04.325623 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/install-certs-ovrd-openstack-edpm-tls-ovrd-openstack-edpm-l6wm4" event={"ID":"be554003-60cb-4ea9-9cce-d5a0b6f3dbaa","Type":"ContainerDied","Data":"7bb35b246d31e28092163cc805b8d65f89eeb4499b20ccc0546f68c702c4e8bf"} Dec 03 13:06:04 crc kubenswrapper[5052]: I1203 13:06:04.325671 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7bb35b246d31e28092163cc805b8d65f89eeb4499b20ccc0546f68c702c4e8bf" Dec 03 13:06:04 crc kubenswrapper[5052]: I1203 13:06:04.325732 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/install-certs-ovrd-openstack-edpm-tls-ovrd-openstack-edpm-l6wm4" Dec 03 13:06:04 crc kubenswrapper[5052]: I1203 13:06:04.393527 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/tls-dns-ips-openstack-edpm-tls-ovrd-openstack-edpm-tls-tf7jr"] Dec 03 13:06:04 crc kubenswrapper[5052]: E1203 13:06:04.393902 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="be554003-60cb-4ea9-9cce-d5a0b6f3dbaa" containerName="install-certs-ovrd-openstack-edpm-tls-ovrd-openstack-edpm-tls" Dec 03 13:06:04 crc kubenswrapper[5052]: I1203 13:06:04.393922 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="be554003-60cb-4ea9-9cce-d5a0b6f3dbaa" containerName="install-certs-ovrd-openstack-edpm-tls-ovrd-openstack-edpm-tls" Dec 03 13:06:04 crc kubenswrapper[5052]: I1203 13:06:04.394104 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="be554003-60cb-4ea9-9cce-d5a0b6f3dbaa" containerName="install-certs-ovrd-openstack-edpm-tls-ovrd-openstack-edpm-tls" Dec 03 13:06:04 crc kubenswrapper[5052]: I1203 13:06:04.394755 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/tls-dns-ips-openstack-edpm-tls-ovrd-openstack-edpm-tls-tf7jr" Dec 03 13:06:04 crc kubenswrapper[5052]: I1203 13:06:04.396694 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"combined-ca-bundle" Dec 03 13:06:04 crc kubenswrapper[5052]: I1203 13:06:04.396851 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"openstack-aee-default-env" Dec 03 13:06:04 crc kubenswrapper[5052]: I1203 13:06:04.397085 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"dataplanenodeset-openstack-edpm-tls" Dec 03 13:06:04 crc kubenswrapper[5052]: I1203 13:06:04.397266 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"openstack-edpm-tls-dockercfg-kz8sq" Dec 03 13:06:04 crc kubenswrapper[5052]: I1203 13:06:04.398351 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"dataplane-ansible-ssh-private-key-secret" Dec 03 13:06:04 crc kubenswrapper[5052]: I1203 13:06:04.401859 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/tls-dns-ips-openstack-edpm-tls-ovrd-openstack-edpm-tls-tf7jr"] Dec 03 13:06:04 crc kubenswrapper[5052]: I1203 13:06:04.497756 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-28xhs\" (UniqueName: \"kubernetes.io/projected/89695efc-592a-44ac-9aeb-ddff36af1791-kube-api-access-28xhs\") pod \"tls-dns-ips-openstack-edpm-tls-ovrd-openstack-edpm-tls-tf7jr\" (UID: \"89695efc-592a-44ac-9aeb-ddff36af1791\") " pod="openstack-kuttl-tests/tls-dns-ips-openstack-edpm-tls-ovrd-openstack-edpm-tls-tf7jr" Dec 03 13:06:04 crc kubenswrapper[5052]: I1203 13:06:04.497919 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/89695efc-592a-44ac-9aeb-ddff36af1791-inventory\") pod \"tls-dns-ips-openstack-edpm-tls-ovrd-openstack-edpm-tls-tf7jr\" (UID: \"89695efc-592a-44ac-9aeb-ddff36af1791\") " pod="openstack-kuttl-tests/tls-dns-ips-openstack-edpm-tls-ovrd-openstack-edpm-tls-tf7jr" Dec 03 13:06:04 crc kubenswrapper[5052]: I1203 13:06:04.497976 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-dns-ips-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/89695efc-592a-44ac-9aeb-ddff36af1791-tls-dns-ips-combined-ca-bundle\") pod \"tls-dns-ips-openstack-edpm-tls-ovrd-openstack-edpm-tls-tf7jr\" (UID: \"89695efc-592a-44ac-9aeb-ddff36af1791\") " pod="openstack-kuttl-tests/tls-dns-ips-openstack-edpm-tls-ovrd-openstack-edpm-tls-tf7jr" Dec 03 13:06:04 crc kubenswrapper[5052]: I1203 13:06:04.498008 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/89695efc-592a-44ac-9aeb-ddff36af1791-ssh-key\") pod \"tls-dns-ips-openstack-edpm-tls-ovrd-openstack-edpm-tls-tf7jr\" (UID: \"89695efc-592a-44ac-9aeb-ddff36af1791\") " pod="openstack-kuttl-tests/tls-dns-ips-openstack-edpm-tls-ovrd-openstack-edpm-tls-tf7jr" Dec 03 13:06:04 crc kubenswrapper[5052]: I1203 13:06:04.599719 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/89695efc-592a-44ac-9aeb-ddff36af1791-inventory\") pod \"tls-dns-ips-openstack-edpm-tls-ovrd-openstack-edpm-tls-tf7jr\" (UID: \"89695efc-592a-44ac-9aeb-ddff36af1791\") " pod="openstack-kuttl-tests/tls-dns-ips-openstack-edpm-tls-ovrd-openstack-edpm-tls-tf7jr" Dec 03 13:06:04 crc kubenswrapper[5052]: I1203 13:06:04.599825 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"tls-dns-ips-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/89695efc-592a-44ac-9aeb-ddff36af1791-tls-dns-ips-combined-ca-bundle\") pod \"tls-dns-ips-openstack-edpm-tls-ovrd-openstack-edpm-tls-tf7jr\" (UID: \"89695efc-592a-44ac-9aeb-ddff36af1791\") " pod="openstack-kuttl-tests/tls-dns-ips-openstack-edpm-tls-ovrd-openstack-edpm-tls-tf7jr" Dec 03 13:06:04 crc kubenswrapper[5052]: I1203 13:06:04.599860 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/89695efc-592a-44ac-9aeb-ddff36af1791-ssh-key\") pod \"tls-dns-ips-openstack-edpm-tls-ovrd-openstack-edpm-tls-tf7jr\" (UID: \"89695efc-592a-44ac-9aeb-ddff36af1791\") " pod="openstack-kuttl-tests/tls-dns-ips-openstack-edpm-tls-ovrd-openstack-edpm-tls-tf7jr" Dec 03 13:06:04 crc kubenswrapper[5052]: I1203 13:06:04.599909 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-28xhs\" (UniqueName: \"kubernetes.io/projected/89695efc-592a-44ac-9aeb-ddff36af1791-kube-api-access-28xhs\") pod \"tls-dns-ips-openstack-edpm-tls-ovrd-openstack-edpm-tls-tf7jr\" (UID: \"89695efc-592a-44ac-9aeb-ddff36af1791\") " pod="openstack-kuttl-tests/tls-dns-ips-openstack-edpm-tls-ovrd-openstack-edpm-tls-tf7jr" Dec 03 13:06:04 crc kubenswrapper[5052]: I1203 13:06:04.603884 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/89695efc-592a-44ac-9aeb-ddff36af1791-ssh-key\") pod \"tls-dns-ips-openstack-edpm-tls-ovrd-openstack-edpm-tls-tf7jr\" (UID: \"89695efc-592a-44ac-9aeb-ddff36af1791\") " pod="openstack-kuttl-tests/tls-dns-ips-openstack-edpm-tls-ovrd-openstack-edpm-tls-tf7jr" Dec 03 13:06:04 crc kubenswrapper[5052]: I1203 13:06:04.603915 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/89695efc-592a-44ac-9aeb-ddff36af1791-inventory\") pod \"tls-dns-ips-openstack-edpm-tls-ovrd-openstack-edpm-tls-tf7jr\" (UID: \"89695efc-592a-44ac-9aeb-ddff36af1791\") " pod="openstack-kuttl-tests/tls-dns-ips-openstack-edpm-tls-ovrd-openstack-edpm-tls-tf7jr" Dec 03 13:06:04 crc kubenswrapper[5052]: I1203 13:06:04.604048 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"tls-dns-ips-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/89695efc-592a-44ac-9aeb-ddff36af1791-tls-dns-ips-combined-ca-bundle\") pod \"tls-dns-ips-openstack-edpm-tls-ovrd-openstack-edpm-tls-tf7jr\" (UID: \"89695efc-592a-44ac-9aeb-ddff36af1791\") " pod="openstack-kuttl-tests/tls-dns-ips-openstack-edpm-tls-ovrd-openstack-edpm-tls-tf7jr" Dec 03 13:06:04 crc kubenswrapper[5052]: I1203 13:06:04.616949 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-28xhs\" (UniqueName: \"kubernetes.io/projected/89695efc-592a-44ac-9aeb-ddff36af1791-kube-api-access-28xhs\") pod \"tls-dns-ips-openstack-edpm-tls-ovrd-openstack-edpm-tls-tf7jr\" (UID: \"89695efc-592a-44ac-9aeb-ddff36af1791\") " pod="openstack-kuttl-tests/tls-dns-ips-openstack-edpm-tls-ovrd-openstack-edpm-tls-tf7jr" Dec 03 13:06:04 crc kubenswrapper[5052]: I1203 13:06:04.717819 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/tls-dns-ips-openstack-edpm-tls-ovrd-openstack-edpm-tls-tf7jr" Dec 03 13:06:05 crc kubenswrapper[5052]: I1203 13:06:05.144993 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/tls-dns-ips-openstack-edpm-tls-ovrd-openstack-edpm-tls-tf7jr"] Dec 03 13:06:05 crc kubenswrapper[5052]: I1203 13:06:05.337429 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/tls-dns-ips-openstack-edpm-tls-ovrd-openstack-edpm-tls-tf7jr" event={"ID":"89695efc-592a-44ac-9aeb-ddff36af1791","Type":"ContainerStarted","Data":"98b66ad63f94a7a12e43c5f80dea25eb69dc4d5a9fd77ee51f42646d0225db77"} Dec 03 13:06:06 crc kubenswrapper[5052]: I1203 13:06:06.349757 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/tls-dns-ips-openstack-edpm-tls-ovrd-openstack-edpm-tls-tf7jr" event={"ID":"89695efc-592a-44ac-9aeb-ddff36af1791","Type":"ContainerStarted","Data":"1a3bf6cdb4478c9febde2b88d59cad00f4ed9b1e5a42911b9ae8bdf2360d5785"} Dec 03 13:06:06 crc kubenswrapper[5052]: I1203 13:06:06.377741 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/tls-dns-ips-openstack-edpm-tls-ovrd-openstack-edpm-tls-tf7jr" podStartSLOduration=1.924554597 podStartE2EDuration="2.377722567s" podCreationTimestamp="2025-12-03 13:06:04 +0000 UTC" firstStartedPulling="2025-12-03 13:06:05.154858946 +0000 UTC m=+6355.743100710" lastFinishedPulling="2025-12-03 13:06:05.608026916 +0000 UTC m=+6356.196268680" observedRunningTime="2025-12-03 13:06:06.366465395 +0000 UTC m=+6356.954707169" watchObservedRunningTime="2025-12-03 13:06:06.377722567 +0000 UTC m=+6356.965964331" Dec 03 13:06:09 crc kubenswrapper[5052]: I1203 13:06:09.392250 5052 generic.go:334] "Generic (PLEG): container finished" podID="89695efc-592a-44ac-9aeb-ddff36af1791" containerID="1a3bf6cdb4478c9febde2b88d59cad00f4ed9b1e5a42911b9ae8bdf2360d5785" exitCode=0 Dec 03 13:06:09 crc kubenswrapper[5052]: I1203 13:06:09.392330 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/tls-dns-ips-openstack-edpm-tls-ovrd-openstack-edpm-tls-tf7jr" event={"ID":"89695efc-592a-44ac-9aeb-ddff36af1791","Type":"ContainerDied","Data":"1a3bf6cdb4478c9febde2b88d59cad00f4ed9b1e5a42911b9ae8bdf2360d5785"} Dec 03 13:06:10 crc kubenswrapper[5052]: I1203 13:06:10.677310 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/tls-dns-ips-openstack-edpm-tls-ovrd-openstack-edpm-tls-tf7jr" Dec 03 13:06:10 crc kubenswrapper[5052]: I1203 13:06:10.798209 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-28xhs\" (UniqueName: \"kubernetes.io/projected/89695efc-592a-44ac-9aeb-ddff36af1791-kube-api-access-28xhs\") pod \"89695efc-592a-44ac-9aeb-ddff36af1791\" (UID: \"89695efc-592a-44ac-9aeb-ddff36af1791\") " Dec 03 13:06:10 crc kubenswrapper[5052]: I1203 13:06:10.798275 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"tls-dns-ips-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/89695efc-592a-44ac-9aeb-ddff36af1791-tls-dns-ips-combined-ca-bundle\") pod \"89695efc-592a-44ac-9aeb-ddff36af1791\" (UID: \"89695efc-592a-44ac-9aeb-ddff36af1791\") " Dec 03 13:06:10 crc kubenswrapper[5052]: I1203 13:06:10.798309 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/89695efc-592a-44ac-9aeb-ddff36af1791-inventory\") pod \"89695efc-592a-44ac-9aeb-ddff36af1791\" (UID: \"89695efc-592a-44ac-9aeb-ddff36af1791\") " Dec 03 13:06:10 crc kubenswrapper[5052]: I1203 13:06:10.798373 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/89695efc-592a-44ac-9aeb-ddff36af1791-ssh-key\") pod \"89695efc-592a-44ac-9aeb-ddff36af1791\" (UID: \"89695efc-592a-44ac-9aeb-ddff36af1791\") " Dec 03 13:06:10 crc kubenswrapper[5052]: I1203 13:06:10.803770 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/89695efc-592a-44ac-9aeb-ddff36af1791-kube-api-access-28xhs" (OuterVolumeSpecName: "kube-api-access-28xhs") pod "89695efc-592a-44ac-9aeb-ddff36af1791" (UID: "89695efc-592a-44ac-9aeb-ddff36af1791"). InnerVolumeSpecName "kube-api-access-28xhs". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:06:10 crc kubenswrapper[5052]: I1203 13:06:10.803858 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/89695efc-592a-44ac-9aeb-ddff36af1791-tls-dns-ips-combined-ca-bundle" (OuterVolumeSpecName: "tls-dns-ips-combined-ca-bundle") pod "89695efc-592a-44ac-9aeb-ddff36af1791" (UID: "89695efc-592a-44ac-9aeb-ddff36af1791"). InnerVolumeSpecName "tls-dns-ips-combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:06:10 crc kubenswrapper[5052]: I1203 13:06:10.822661 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/89695efc-592a-44ac-9aeb-ddff36af1791-ssh-key" (OuterVolumeSpecName: "ssh-key") pod "89695efc-592a-44ac-9aeb-ddff36af1791" (UID: "89695efc-592a-44ac-9aeb-ddff36af1791"). InnerVolumeSpecName "ssh-key". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:06:10 crc kubenswrapper[5052]: I1203 13:06:10.823923 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/89695efc-592a-44ac-9aeb-ddff36af1791-inventory" (OuterVolumeSpecName: "inventory") pod "89695efc-592a-44ac-9aeb-ddff36af1791" (UID: "89695efc-592a-44ac-9aeb-ddff36af1791"). InnerVolumeSpecName "inventory". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:06:10 crc kubenswrapper[5052]: I1203 13:06:10.900132 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-28xhs\" (UniqueName: \"kubernetes.io/projected/89695efc-592a-44ac-9aeb-ddff36af1791-kube-api-access-28xhs\") on node \"crc\" DevicePath \"\"" Dec 03 13:06:10 crc kubenswrapper[5052]: I1203 13:06:10.900170 5052 reconciler_common.go:293] "Volume detached for volume \"tls-dns-ips-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/89695efc-592a-44ac-9aeb-ddff36af1791-tls-dns-ips-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 13:06:10 crc kubenswrapper[5052]: I1203 13:06:10.900182 5052 reconciler_common.go:293] "Volume detached for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/89695efc-592a-44ac-9aeb-ddff36af1791-inventory\") on node \"crc\" DevicePath \"\"" Dec 03 13:06:10 crc kubenswrapper[5052]: I1203 13:06:10.900197 5052 reconciler_common.go:293] "Volume detached for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/89695efc-592a-44ac-9aeb-ddff36af1791-ssh-key\") on node \"crc\" DevicePath \"\"" Dec 03 13:06:11 crc kubenswrapper[5052]: I1203 13:06:11.409305 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/tls-dns-ips-openstack-edpm-tls-ovrd-openstack-edpm-tls-tf7jr" event={"ID":"89695efc-592a-44ac-9aeb-ddff36af1791","Type":"ContainerDied","Data":"98b66ad63f94a7a12e43c5f80dea25eb69dc4d5a9fd77ee51f42646d0225db77"} Dec 03 13:06:11 crc kubenswrapper[5052]: I1203 13:06:11.409355 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="98b66ad63f94a7a12e43c5f80dea25eb69dc4d5a9fd77ee51f42646d0225db77" Dec 03 13:06:11 crc kubenswrapper[5052]: I1203 13:06:11.409407 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/tls-dns-ips-openstack-edpm-tls-ovrd-openstack-edpm-tls-tf7jr" Dec 03 13:06:11 crc kubenswrapper[5052]: I1203 13:06:11.477312 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/custom-tls-dns-openstack-edpm-tls-ovrd-openstack-edpm-tls-96xgg"] Dec 03 13:06:11 crc kubenswrapper[5052]: E1203 13:06:11.477800 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="89695efc-592a-44ac-9aeb-ddff36af1791" containerName="tls-dns-ips-openstack-edpm-tls-ovrd-openstack-edpm-tls" Dec 03 13:06:11 crc kubenswrapper[5052]: I1203 13:06:11.477827 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="89695efc-592a-44ac-9aeb-ddff36af1791" containerName="tls-dns-ips-openstack-edpm-tls-ovrd-openstack-edpm-tls" Dec 03 13:06:11 crc kubenswrapper[5052]: I1203 13:06:11.477988 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="89695efc-592a-44ac-9aeb-ddff36af1791" containerName="tls-dns-ips-openstack-edpm-tls-ovrd-openstack-edpm-tls" Dec 03 13:06:11 crc kubenswrapper[5052]: I1203 13:06:11.478763 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/custom-tls-dns-openstack-edpm-tls-ovrd-openstack-edpm-tls-96xgg" Dec 03 13:06:11 crc kubenswrapper[5052]: I1203 13:06:11.481374 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"combined-ca-bundle" Dec 03 13:06:11 crc kubenswrapper[5052]: I1203 13:06:11.481385 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"openstack-aee-default-env" Dec 03 13:06:11 crc kubenswrapper[5052]: I1203 13:06:11.484431 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"dataplane-ansible-ssh-private-key-secret" Dec 03 13:06:11 crc kubenswrapper[5052]: I1203 13:06:11.484770 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"openstack-edpm-tls-dockercfg-kz8sq" Dec 03 13:06:11 crc kubenswrapper[5052]: I1203 13:06:11.485759 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"dataplanenodeset-openstack-edpm-tls" Dec 03 13:06:11 crc kubenswrapper[5052]: I1203 13:06:11.489808 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/custom-tls-dns-openstack-edpm-tls-ovrd-openstack-edpm-tls-96xgg"] Dec 03 13:06:11 crc kubenswrapper[5052]: I1203 13:06:11.506731 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/0a23beab-7ac9-4645-8c48-13b9d02e9659-inventory\") pod \"custom-tls-dns-openstack-edpm-tls-ovrd-openstack-edpm-tls-96xgg\" (UID: \"0a23beab-7ac9-4645-8c48-13b9d02e9659\") " pod="openstack-kuttl-tests/custom-tls-dns-openstack-edpm-tls-ovrd-openstack-edpm-tls-96xgg" Dec 03 13:06:11 crc kubenswrapper[5052]: I1203 13:06:11.506797 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/0a23beab-7ac9-4645-8c48-13b9d02e9659-ssh-key\") pod \"custom-tls-dns-openstack-edpm-tls-ovrd-openstack-edpm-tls-96xgg\" (UID: \"0a23beab-7ac9-4645-8c48-13b9d02e9659\") " pod="openstack-kuttl-tests/custom-tls-dns-openstack-edpm-tls-ovrd-openstack-edpm-tls-96xgg" Dec 03 13:06:11 crc kubenswrapper[5052]: I1203 13:06:11.506828 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zxjjd\" (UniqueName: \"kubernetes.io/projected/0a23beab-7ac9-4645-8c48-13b9d02e9659-kube-api-access-zxjjd\") pod \"custom-tls-dns-openstack-edpm-tls-ovrd-openstack-edpm-tls-96xgg\" (UID: \"0a23beab-7ac9-4645-8c48-13b9d02e9659\") " pod="openstack-kuttl-tests/custom-tls-dns-openstack-edpm-tls-ovrd-openstack-edpm-tls-96xgg" Dec 03 13:06:11 crc kubenswrapper[5052]: I1203 13:06:11.506864 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"custom-tls-dns-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0a23beab-7ac9-4645-8c48-13b9d02e9659-custom-tls-dns-combined-ca-bundle\") pod \"custom-tls-dns-openstack-edpm-tls-ovrd-openstack-edpm-tls-96xgg\" (UID: \"0a23beab-7ac9-4645-8c48-13b9d02e9659\") " pod="openstack-kuttl-tests/custom-tls-dns-openstack-edpm-tls-ovrd-openstack-edpm-tls-96xgg" Dec 03 13:06:11 crc kubenswrapper[5052]: I1203 13:06:11.607754 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/0a23beab-7ac9-4645-8c48-13b9d02e9659-inventory\") pod \"custom-tls-dns-openstack-edpm-tls-ovrd-openstack-edpm-tls-96xgg\" (UID: \"0a23beab-7ac9-4645-8c48-13b9d02e9659\") " pod="openstack-kuttl-tests/custom-tls-dns-openstack-edpm-tls-ovrd-openstack-edpm-tls-96xgg" Dec 03 13:06:11 crc kubenswrapper[5052]: I1203 13:06:11.608080 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/0a23beab-7ac9-4645-8c48-13b9d02e9659-ssh-key\") pod \"custom-tls-dns-openstack-edpm-tls-ovrd-openstack-edpm-tls-96xgg\" (UID: \"0a23beab-7ac9-4645-8c48-13b9d02e9659\") " pod="openstack-kuttl-tests/custom-tls-dns-openstack-edpm-tls-ovrd-openstack-edpm-tls-96xgg" Dec 03 13:06:11 crc kubenswrapper[5052]: I1203 13:06:11.608133 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-zxjjd\" (UniqueName: \"kubernetes.io/projected/0a23beab-7ac9-4645-8c48-13b9d02e9659-kube-api-access-zxjjd\") pod \"custom-tls-dns-openstack-edpm-tls-ovrd-openstack-edpm-tls-96xgg\" (UID: \"0a23beab-7ac9-4645-8c48-13b9d02e9659\") " pod="openstack-kuttl-tests/custom-tls-dns-openstack-edpm-tls-ovrd-openstack-edpm-tls-96xgg" Dec 03 13:06:11 crc kubenswrapper[5052]: I1203 13:06:11.608176 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"custom-tls-dns-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0a23beab-7ac9-4645-8c48-13b9d02e9659-custom-tls-dns-combined-ca-bundle\") pod \"custom-tls-dns-openstack-edpm-tls-ovrd-openstack-edpm-tls-96xgg\" (UID: \"0a23beab-7ac9-4645-8c48-13b9d02e9659\") " pod="openstack-kuttl-tests/custom-tls-dns-openstack-edpm-tls-ovrd-openstack-edpm-tls-96xgg" Dec 03 13:06:11 crc kubenswrapper[5052]: I1203 13:06:11.613225 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"custom-tls-dns-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0a23beab-7ac9-4645-8c48-13b9d02e9659-custom-tls-dns-combined-ca-bundle\") pod \"custom-tls-dns-openstack-edpm-tls-ovrd-openstack-edpm-tls-96xgg\" (UID: \"0a23beab-7ac9-4645-8c48-13b9d02e9659\") " pod="openstack-kuttl-tests/custom-tls-dns-openstack-edpm-tls-ovrd-openstack-edpm-tls-96xgg" Dec 03 13:06:11 crc kubenswrapper[5052]: I1203 13:06:11.616329 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/0a23beab-7ac9-4645-8c48-13b9d02e9659-ssh-key\") pod \"custom-tls-dns-openstack-edpm-tls-ovrd-openstack-edpm-tls-96xgg\" (UID: \"0a23beab-7ac9-4645-8c48-13b9d02e9659\") " pod="openstack-kuttl-tests/custom-tls-dns-openstack-edpm-tls-ovrd-openstack-edpm-tls-96xgg" Dec 03 13:06:11 crc kubenswrapper[5052]: I1203 13:06:11.621995 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/0a23beab-7ac9-4645-8c48-13b9d02e9659-inventory\") pod \"custom-tls-dns-openstack-edpm-tls-ovrd-openstack-edpm-tls-96xgg\" (UID: \"0a23beab-7ac9-4645-8c48-13b9d02e9659\") " pod="openstack-kuttl-tests/custom-tls-dns-openstack-edpm-tls-ovrd-openstack-edpm-tls-96xgg" Dec 03 13:06:11 crc kubenswrapper[5052]: I1203 13:06:11.628762 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-zxjjd\" (UniqueName: \"kubernetes.io/projected/0a23beab-7ac9-4645-8c48-13b9d02e9659-kube-api-access-zxjjd\") pod \"custom-tls-dns-openstack-edpm-tls-ovrd-openstack-edpm-tls-96xgg\" (UID: \"0a23beab-7ac9-4645-8c48-13b9d02e9659\") " pod="openstack-kuttl-tests/custom-tls-dns-openstack-edpm-tls-ovrd-openstack-edpm-tls-96xgg" Dec 03 13:06:11 crc kubenswrapper[5052]: I1203 13:06:11.797126 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/custom-tls-dns-openstack-edpm-tls-ovrd-openstack-edpm-tls-96xgg" Dec 03 13:06:12 crc kubenswrapper[5052]: I1203 13:06:12.210337 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/custom-tls-dns-openstack-edpm-tls-ovrd-openstack-edpm-tls-96xgg"] Dec 03 13:06:12 crc kubenswrapper[5052]: I1203 13:06:12.418965 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/custom-tls-dns-openstack-edpm-tls-ovrd-openstack-edpm-tls-96xgg" event={"ID":"0a23beab-7ac9-4645-8c48-13b9d02e9659","Type":"ContainerStarted","Data":"3a0e081f5ade4c6317de12f60d856840eca823b8a3f2e23fcd31121d1f6c8589"} Dec 03 13:06:13 crc kubenswrapper[5052]: I1203 13:06:13.427786 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/custom-tls-dns-openstack-edpm-tls-ovrd-openstack-edpm-tls-96xgg" event={"ID":"0a23beab-7ac9-4645-8c48-13b9d02e9659","Type":"ContainerStarted","Data":"9a69414fae786f9e02976dca01bc969c527b49236d5729e3a66b4a354a79dacd"} Dec 03 13:06:13 crc kubenswrapper[5052]: I1203 13:06:13.445692 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/custom-tls-dns-openstack-edpm-tls-ovrd-openstack-edpm-tls-96xgg" podStartSLOduration=1.990950093 podStartE2EDuration="2.445678308s" podCreationTimestamp="2025-12-03 13:06:11 +0000 UTC" firstStartedPulling="2025-12-03 13:06:12.215759755 +0000 UTC m=+6362.804001519" lastFinishedPulling="2025-12-03 13:06:12.67048797 +0000 UTC m=+6363.258729734" observedRunningTime="2025-12-03 13:06:13.441996993 +0000 UTC m=+6364.030238757" watchObservedRunningTime="2025-12-03 13:06:13.445678308 +0000 UTC m=+6364.033920072" Dec 03 13:06:16 crc kubenswrapper[5052]: I1203 13:06:16.459972 5052 generic.go:334] "Generic (PLEG): container finished" podID="0a23beab-7ac9-4645-8c48-13b9d02e9659" containerID="9a69414fae786f9e02976dca01bc969c527b49236d5729e3a66b4a354a79dacd" exitCode=0 Dec 03 13:06:16 crc kubenswrapper[5052]: I1203 13:06:16.460086 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/custom-tls-dns-openstack-edpm-tls-ovrd-openstack-edpm-tls-96xgg" event={"ID":"0a23beab-7ac9-4645-8c48-13b9d02e9659","Type":"ContainerDied","Data":"9a69414fae786f9e02976dca01bc969c527b49236d5729e3a66b4a354a79dacd"} Dec 03 13:06:17 crc kubenswrapper[5052]: I1203 13:06:17.938671 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/custom-tls-dns-openstack-edpm-tls-ovrd-openstack-edpm-tls-96xgg" Dec 03 13:06:18 crc kubenswrapper[5052]: I1203 13:06:18.116761 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/0a23beab-7ac9-4645-8c48-13b9d02e9659-inventory\") pod \"0a23beab-7ac9-4645-8c48-13b9d02e9659\" (UID: \"0a23beab-7ac9-4645-8c48-13b9d02e9659\") " Dec 03 13:06:18 crc kubenswrapper[5052]: I1203 13:06:18.116843 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zxjjd\" (UniqueName: \"kubernetes.io/projected/0a23beab-7ac9-4645-8c48-13b9d02e9659-kube-api-access-zxjjd\") pod \"0a23beab-7ac9-4645-8c48-13b9d02e9659\" (UID: \"0a23beab-7ac9-4645-8c48-13b9d02e9659\") " Dec 03 13:06:18 crc kubenswrapper[5052]: I1203 13:06:18.116925 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"custom-tls-dns-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0a23beab-7ac9-4645-8c48-13b9d02e9659-custom-tls-dns-combined-ca-bundle\") pod \"0a23beab-7ac9-4645-8c48-13b9d02e9659\" (UID: \"0a23beab-7ac9-4645-8c48-13b9d02e9659\") " Dec 03 13:06:18 crc kubenswrapper[5052]: I1203 13:06:18.116992 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/0a23beab-7ac9-4645-8c48-13b9d02e9659-ssh-key\") pod \"0a23beab-7ac9-4645-8c48-13b9d02e9659\" (UID: \"0a23beab-7ac9-4645-8c48-13b9d02e9659\") " Dec 03 13:06:18 crc kubenswrapper[5052]: I1203 13:06:18.126666 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0a23beab-7ac9-4645-8c48-13b9d02e9659-custom-tls-dns-combined-ca-bundle" (OuterVolumeSpecName: "custom-tls-dns-combined-ca-bundle") pod "0a23beab-7ac9-4645-8c48-13b9d02e9659" (UID: "0a23beab-7ac9-4645-8c48-13b9d02e9659"). InnerVolumeSpecName "custom-tls-dns-combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:06:18 crc kubenswrapper[5052]: I1203 13:06:18.129270 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0a23beab-7ac9-4645-8c48-13b9d02e9659-kube-api-access-zxjjd" (OuterVolumeSpecName: "kube-api-access-zxjjd") pod "0a23beab-7ac9-4645-8c48-13b9d02e9659" (UID: "0a23beab-7ac9-4645-8c48-13b9d02e9659"). InnerVolumeSpecName "kube-api-access-zxjjd". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:06:18 crc kubenswrapper[5052]: I1203 13:06:18.139997 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0a23beab-7ac9-4645-8c48-13b9d02e9659-ssh-key" (OuterVolumeSpecName: "ssh-key") pod "0a23beab-7ac9-4645-8c48-13b9d02e9659" (UID: "0a23beab-7ac9-4645-8c48-13b9d02e9659"). InnerVolumeSpecName "ssh-key". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:06:18 crc kubenswrapper[5052]: I1203 13:06:18.141664 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0a23beab-7ac9-4645-8c48-13b9d02e9659-inventory" (OuterVolumeSpecName: "inventory") pod "0a23beab-7ac9-4645-8c48-13b9d02e9659" (UID: "0a23beab-7ac9-4645-8c48-13b9d02e9659"). InnerVolumeSpecName "inventory". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:06:18 crc kubenswrapper[5052]: I1203 13:06:18.218803 5052 reconciler_common.go:293] "Volume detached for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/0a23beab-7ac9-4645-8c48-13b9d02e9659-inventory\") on node \"crc\" DevicePath \"\"" Dec 03 13:06:18 crc kubenswrapper[5052]: I1203 13:06:18.218864 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-zxjjd\" (UniqueName: \"kubernetes.io/projected/0a23beab-7ac9-4645-8c48-13b9d02e9659-kube-api-access-zxjjd\") on node \"crc\" DevicePath \"\"" Dec 03 13:06:18 crc kubenswrapper[5052]: I1203 13:06:18.218882 5052 reconciler_common.go:293] "Volume detached for volume \"custom-tls-dns-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0a23beab-7ac9-4645-8c48-13b9d02e9659-custom-tls-dns-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 13:06:18 crc kubenswrapper[5052]: I1203 13:06:18.218894 5052 reconciler_common.go:293] "Volume detached for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/0a23beab-7ac9-4645-8c48-13b9d02e9659-ssh-key\") on node \"crc\" DevicePath \"\"" Dec 03 13:06:18 crc kubenswrapper[5052]: I1203 13:06:18.503824 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/custom-tls-dns-openstack-edpm-tls-ovrd-openstack-edpm-tls-96xgg" event={"ID":"0a23beab-7ac9-4645-8c48-13b9d02e9659","Type":"ContainerDied","Data":"3a0e081f5ade4c6317de12f60d856840eca823b8a3f2e23fcd31121d1f6c8589"} Dec 03 13:06:18 crc kubenswrapper[5052]: I1203 13:06:18.503882 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3a0e081f5ade4c6317de12f60d856840eca823b8a3f2e23fcd31121d1f6c8589" Dec 03 13:06:18 crc kubenswrapper[5052]: I1203 13:06:18.503913 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/custom-tls-dns-openstack-edpm-tls-ovrd-openstack-edpm-tls-96xgg" Dec 03 13:06:19 crc kubenswrapper[5052]: I1203 13:06:19.855623 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5"] Dec 03 13:06:19 crc kubenswrapper[5052]: E1203 13:06:19.856324 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="0a23beab-7ac9-4645-8c48-13b9d02e9659" containerName="custom-tls-dns-openstack-edpm-tls-ovrd-openstack-edpm-tls" Dec 03 13:06:19 crc kubenswrapper[5052]: I1203 13:06:19.856343 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a23beab-7ac9-4645-8c48-13b9d02e9659" containerName="custom-tls-dns-openstack-edpm-tls-ovrd-openstack-edpm-tls" Dec 03 13:06:19 crc kubenswrapper[5052]: I1203 13:06:19.856550 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="0a23beab-7ac9-4645-8c48-13b9d02e9659" containerName="custom-tls-dns-openstack-edpm-tls-ovrd-openstack-edpm-tls" Dec 03 13:06:19 crc kubenswrapper[5052]: I1203 13:06:19.857206 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5" Dec 03 13:06:19 crc kubenswrapper[5052]: I1203 13:06:19.860254 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"dataplane-ansible-ssh-private-key-secret" Dec 03 13:06:19 crc kubenswrapper[5052]: I1203 13:06:19.860420 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"openstack-edpm-tls-custom-tls-dns-default-certs-0" Dec 03 13:06:19 crc kubenswrapper[5052]: I1203 13:06:19.861080 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"dataplanenodeset-openstack-edpm-tls" Dec 03 13:06:19 crc kubenswrapper[5052]: I1203 13:06:19.864568 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"combined-ca-bundle" Dec 03 13:06:19 crc kubenswrapper[5052]: I1203 13:06:19.864770 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"openstack-aee-default-env" Dec 03 13:06:19 crc kubenswrapper[5052]: I1203 13:06:19.866189 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"openstack-edpm-tls-dockercfg-kz8sq" Dec 03 13:06:19 crc kubenswrapper[5052]: I1203 13:06:19.867172 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"openstack-edpm-tls-tls-dns-ips-default-certs-0" Dec 03 13:06:19 crc kubenswrapper[5052]: I1203 13:06:19.868832 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5"] Dec 03 13:06:20 crc kubenswrapper[5052]: I1203 13:06:20.043372 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openstack-edpm-tls-custom-tls-dns-default-certs-0\" (UniqueName: \"kubernetes.io/projected/30e59ea6-cd2f-4e47-b4dc-7ff82bfce2b7-openstack-edpm-tls-custom-tls-dns-default-certs-0\") pod \"install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5\" (UID: \"30e59ea6-cd2f-4e47-b4dc-7ff82bfce2b7\") " pod="openstack-kuttl-tests/install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5" Dec 03 13:06:20 crc kubenswrapper[5052]: I1203 13:06:20.043613 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"install-certs-ovrd-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/30e59ea6-cd2f-4e47-b4dc-7ff82bfce2b7-install-certs-ovrd-combined-ca-bundle\") pod \"install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5\" (UID: \"30e59ea6-cd2f-4e47-b4dc-7ff82bfce2b7\") " pod="openstack-kuttl-tests/install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5" Dec 03 13:06:20 crc kubenswrapper[5052]: I1203 13:06:20.043660 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openstack-edpm-tls-tls-dns-ips-default-certs-0\" (UniqueName: \"kubernetes.io/projected/30e59ea6-cd2f-4e47-b4dc-7ff82bfce2b7-openstack-edpm-tls-tls-dns-ips-default-certs-0\") pod \"install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5\" (UID: \"30e59ea6-cd2f-4e47-b4dc-7ff82bfce2b7\") " pod="openstack-kuttl-tests/install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5" Dec 03 13:06:20 crc kubenswrapper[5052]: I1203 13:06:20.043686 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-dns-ips-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/30e59ea6-cd2f-4e47-b4dc-7ff82bfce2b7-tls-dns-ips-combined-ca-bundle\") pod \"install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5\" (UID: \"30e59ea6-cd2f-4e47-b4dc-7ff82bfce2b7\") " pod="openstack-kuttl-tests/install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5" Dec 03 13:06:20 crc kubenswrapper[5052]: I1203 13:06:20.043717 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rww9c\" (UniqueName: \"kubernetes.io/projected/30e59ea6-cd2f-4e47-b4dc-7ff82bfce2b7-kube-api-access-rww9c\") pod \"install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5\" (UID: \"30e59ea6-cd2f-4e47-b4dc-7ff82bfce2b7\") " pod="openstack-kuttl-tests/install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5" Dec 03 13:06:20 crc kubenswrapper[5052]: I1203 13:06:20.043781 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/30e59ea6-cd2f-4e47-b4dc-7ff82bfce2b7-ssh-key\") pod \"install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5\" (UID: \"30e59ea6-cd2f-4e47-b4dc-7ff82bfce2b7\") " pod="openstack-kuttl-tests/install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5" Dec 03 13:06:20 crc kubenswrapper[5052]: I1203 13:06:20.043808 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/30e59ea6-cd2f-4e47-b4dc-7ff82bfce2b7-inventory\") pod \"install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5\" (UID: \"30e59ea6-cd2f-4e47-b4dc-7ff82bfce2b7\") " pod="openstack-kuttl-tests/install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5" Dec 03 13:06:20 crc kubenswrapper[5052]: I1203 13:06:20.043828 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"custom-tls-dns-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/30e59ea6-cd2f-4e47-b4dc-7ff82bfce2b7-custom-tls-dns-combined-ca-bundle\") pod \"install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5\" (UID: \"30e59ea6-cd2f-4e47-b4dc-7ff82bfce2b7\") " pod="openstack-kuttl-tests/install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5" Dec 03 13:06:20 crc kubenswrapper[5052]: I1203 13:06:20.145306 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"install-certs-ovrd-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/30e59ea6-cd2f-4e47-b4dc-7ff82bfce2b7-install-certs-ovrd-combined-ca-bundle\") pod \"install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5\" (UID: \"30e59ea6-cd2f-4e47-b4dc-7ff82bfce2b7\") " pod="openstack-kuttl-tests/install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5" Dec 03 13:06:20 crc kubenswrapper[5052]: I1203 13:06:20.145357 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"openstack-edpm-tls-tls-dns-ips-default-certs-0\" (UniqueName: \"kubernetes.io/projected/30e59ea6-cd2f-4e47-b4dc-7ff82bfce2b7-openstack-edpm-tls-tls-dns-ips-default-certs-0\") pod \"install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5\" (UID: \"30e59ea6-cd2f-4e47-b4dc-7ff82bfce2b7\") " pod="openstack-kuttl-tests/install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5" Dec 03 13:06:20 crc kubenswrapper[5052]: I1203 13:06:20.145380 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"tls-dns-ips-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/30e59ea6-cd2f-4e47-b4dc-7ff82bfce2b7-tls-dns-ips-combined-ca-bundle\") pod \"install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5\" (UID: \"30e59ea6-cd2f-4e47-b4dc-7ff82bfce2b7\") " pod="openstack-kuttl-tests/install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5" Dec 03 13:06:20 crc kubenswrapper[5052]: I1203 13:06:20.145408 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-rww9c\" (UniqueName: \"kubernetes.io/projected/30e59ea6-cd2f-4e47-b4dc-7ff82bfce2b7-kube-api-access-rww9c\") pod \"install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5\" (UID: \"30e59ea6-cd2f-4e47-b4dc-7ff82bfce2b7\") " pod="openstack-kuttl-tests/install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5" Dec 03 13:06:20 crc kubenswrapper[5052]: I1203 13:06:20.145449 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/30e59ea6-cd2f-4e47-b4dc-7ff82bfce2b7-ssh-key\") pod \"install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5\" (UID: \"30e59ea6-cd2f-4e47-b4dc-7ff82bfce2b7\") " pod="openstack-kuttl-tests/install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5" Dec 03 13:06:20 crc kubenswrapper[5052]: I1203 13:06:20.145468 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/30e59ea6-cd2f-4e47-b4dc-7ff82bfce2b7-inventory\") pod \"install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5\" (UID: \"30e59ea6-cd2f-4e47-b4dc-7ff82bfce2b7\") " pod="openstack-kuttl-tests/install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5" Dec 03 13:06:20 crc kubenswrapper[5052]: I1203 13:06:20.145628 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"custom-tls-dns-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/30e59ea6-cd2f-4e47-b4dc-7ff82bfce2b7-custom-tls-dns-combined-ca-bundle\") pod \"install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5\" (UID: \"30e59ea6-cd2f-4e47-b4dc-7ff82bfce2b7\") " pod="openstack-kuttl-tests/install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5" Dec 03 13:06:20 crc kubenswrapper[5052]: I1203 13:06:20.145670 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"openstack-edpm-tls-custom-tls-dns-default-certs-0\" (UniqueName: \"kubernetes.io/projected/30e59ea6-cd2f-4e47-b4dc-7ff82bfce2b7-openstack-edpm-tls-custom-tls-dns-default-certs-0\") pod \"install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5\" (UID: \"30e59ea6-cd2f-4e47-b4dc-7ff82bfce2b7\") " pod="openstack-kuttl-tests/install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5" Dec 03 13:06:20 crc kubenswrapper[5052]: I1203 13:06:20.151007 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/30e59ea6-cd2f-4e47-b4dc-7ff82bfce2b7-inventory\") pod \"install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5\" (UID: \"30e59ea6-cd2f-4e47-b4dc-7ff82bfce2b7\") " pod="openstack-kuttl-tests/install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5" Dec 03 13:06:20 crc kubenswrapper[5052]: I1203 13:06:20.151025 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"custom-tls-dns-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/30e59ea6-cd2f-4e47-b4dc-7ff82bfce2b7-custom-tls-dns-combined-ca-bundle\") pod \"install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5\" (UID: \"30e59ea6-cd2f-4e47-b4dc-7ff82bfce2b7\") " pod="openstack-kuttl-tests/install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5" Dec 03 13:06:20 crc kubenswrapper[5052]: I1203 13:06:20.151437 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"openstack-edpm-tls-custom-tls-dns-default-certs-0\" (UniqueName: \"kubernetes.io/projected/30e59ea6-cd2f-4e47-b4dc-7ff82bfce2b7-openstack-edpm-tls-custom-tls-dns-default-certs-0\") pod \"install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5\" (UID: \"30e59ea6-cd2f-4e47-b4dc-7ff82bfce2b7\") " pod="openstack-kuttl-tests/install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5" Dec 03 13:06:20 crc kubenswrapper[5052]: I1203 13:06:20.151532 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"openstack-edpm-tls-tls-dns-ips-default-certs-0\" (UniqueName: \"kubernetes.io/projected/30e59ea6-cd2f-4e47-b4dc-7ff82bfce2b7-openstack-edpm-tls-tls-dns-ips-default-certs-0\") pod \"install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5\" (UID: \"30e59ea6-cd2f-4e47-b4dc-7ff82bfce2b7\") " pod="openstack-kuttl-tests/install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5" Dec 03 13:06:20 crc kubenswrapper[5052]: I1203 13:06:20.151937 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/30e59ea6-cd2f-4e47-b4dc-7ff82bfce2b7-ssh-key\") pod \"install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5\" (UID: \"30e59ea6-cd2f-4e47-b4dc-7ff82bfce2b7\") " pod="openstack-kuttl-tests/install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5" Dec 03 13:06:20 crc kubenswrapper[5052]: I1203 13:06:20.152257 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"tls-dns-ips-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/30e59ea6-cd2f-4e47-b4dc-7ff82bfce2b7-tls-dns-ips-combined-ca-bundle\") pod \"install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5\" (UID: \"30e59ea6-cd2f-4e47-b4dc-7ff82bfce2b7\") " pod="openstack-kuttl-tests/install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5" Dec 03 13:06:20 crc kubenswrapper[5052]: I1203 13:06:20.157075 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"install-certs-ovrd-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/30e59ea6-cd2f-4e47-b4dc-7ff82bfce2b7-install-certs-ovrd-combined-ca-bundle\") pod \"install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5\" (UID: \"30e59ea6-cd2f-4e47-b4dc-7ff82bfce2b7\") " pod="openstack-kuttl-tests/install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5" Dec 03 13:06:20 crc kubenswrapper[5052]: I1203 13:06:20.165261 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-rww9c\" (UniqueName: \"kubernetes.io/projected/30e59ea6-cd2f-4e47-b4dc-7ff82bfce2b7-kube-api-access-rww9c\") pod \"install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5\" (UID: \"30e59ea6-cd2f-4e47-b4dc-7ff82bfce2b7\") " pod="openstack-kuttl-tests/install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5" Dec 03 13:06:20 crc kubenswrapper[5052]: I1203 13:06:20.186271 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5" Dec 03 13:06:20 crc kubenswrapper[5052]: I1203 13:06:20.631025 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5"] Dec 03 13:06:21 crc kubenswrapper[5052]: I1203 13:06:21.536653 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5" event={"ID":"30e59ea6-cd2f-4e47-b4dc-7ff82bfce2b7","Type":"ContainerStarted","Data":"dc9bd993b4d6db1d054ab0a7f486584db56e37d1e6559b303531ddc0f825d402"} Dec 03 13:06:21 crc kubenswrapper[5052]: I1203 13:06:21.536725 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5" event={"ID":"30e59ea6-cd2f-4e47-b4dc-7ff82bfce2b7","Type":"ContainerStarted","Data":"dedbab00e5960b10da448caf910bdff96790aba930e02de42c5d1f3b90b94393"} Dec 03 13:06:21 crc kubenswrapper[5052]: I1203 13:06:21.557581 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5" podStartSLOduration=2.033546057 podStartE2EDuration="2.557560943s" podCreationTimestamp="2025-12-03 13:06:19 +0000 UTC" firstStartedPulling="2025-12-03 13:06:20.655685931 +0000 UTC m=+6371.243927695" lastFinishedPulling="2025-12-03 13:06:21.179700817 +0000 UTC m=+6371.767942581" observedRunningTime="2025-12-03 13:06:21.554392253 +0000 UTC m=+6372.142634017" watchObservedRunningTime="2025-12-03 13:06:21.557560943 +0000 UTC m=+6372.145802707" Dec 03 13:06:24 crc kubenswrapper[5052]: I1203 13:06:24.567097 5052 generic.go:334] "Generic (PLEG): container finished" podID="30e59ea6-cd2f-4e47-b4dc-7ff82bfce2b7" containerID="dc9bd993b4d6db1d054ab0a7f486584db56e37d1e6559b303531ddc0f825d402" exitCode=0 Dec 03 13:06:24 crc kubenswrapper[5052]: I1203 13:06:24.567210 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5" event={"ID":"30e59ea6-cd2f-4e47-b4dc-7ff82bfce2b7","Type":"ContainerDied","Data":"dc9bd993b4d6db1d054ab0a7f486584db56e37d1e6559b303531ddc0f825d402"} Dec 03 13:06:25 crc kubenswrapper[5052]: I1203 13:06:25.862708 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5" Dec 03 13:06:26 crc kubenswrapper[5052]: I1203 13:06:26.037198 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"custom-tls-dns-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/30e59ea6-cd2f-4e47-b4dc-7ff82bfce2b7-custom-tls-dns-combined-ca-bundle\") pod \"30e59ea6-cd2f-4e47-b4dc-7ff82bfce2b7\" (UID: \"30e59ea6-cd2f-4e47-b4dc-7ff82bfce2b7\") " Dec 03 13:06:26 crc kubenswrapper[5052]: I1203 13:06:26.037271 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rww9c\" (UniqueName: \"kubernetes.io/projected/30e59ea6-cd2f-4e47-b4dc-7ff82bfce2b7-kube-api-access-rww9c\") pod \"30e59ea6-cd2f-4e47-b4dc-7ff82bfce2b7\" (UID: \"30e59ea6-cd2f-4e47-b4dc-7ff82bfce2b7\") " Dec 03 13:06:26 crc kubenswrapper[5052]: I1203 13:06:26.037344 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/30e59ea6-cd2f-4e47-b4dc-7ff82bfce2b7-inventory\") pod \"30e59ea6-cd2f-4e47-b4dc-7ff82bfce2b7\" (UID: \"30e59ea6-cd2f-4e47-b4dc-7ff82bfce2b7\") " Dec 03 13:06:26 crc kubenswrapper[5052]: I1203 13:06:26.037369 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"openstack-edpm-tls-tls-dns-ips-default-certs-0\" (UniqueName: \"kubernetes.io/projected/30e59ea6-cd2f-4e47-b4dc-7ff82bfce2b7-openstack-edpm-tls-tls-dns-ips-default-certs-0\") pod \"30e59ea6-cd2f-4e47-b4dc-7ff82bfce2b7\" (UID: \"30e59ea6-cd2f-4e47-b4dc-7ff82bfce2b7\") " Dec 03 13:06:26 crc kubenswrapper[5052]: I1203 13:06:26.037412 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"tls-dns-ips-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/30e59ea6-cd2f-4e47-b4dc-7ff82bfce2b7-tls-dns-ips-combined-ca-bundle\") pod \"30e59ea6-cd2f-4e47-b4dc-7ff82bfce2b7\" (UID: \"30e59ea6-cd2f-4e47-b4dc-7ff82bfce2b7\") " Dec 03 13:06:26 crc kubenswrapper[5052]: I1203 13:06:26.037471 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"install-certs-ovrd-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/30e59ea6-cd2f-4e47-b4dc-7ff82bfce2b7-install-certs-ovrd-combined-ca-bundle\") pod \"30e59ea6-cd2f-4e47-b4dc-7ff82bfce2b7\" (UID: \"30e59ea6-cd2f-4e47-b4dc-7ff82bfce2b7\") " Dec 03 13:06:26 crc kubenswrapper[5052]: I1203 13:06:26.037546 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/30e59ea6-cd2f-4e47-b4dc-7ff82bfce2b7-ssh-key\") pod \"30e59ea6-cd2f-4e47-b4dc-7ff82bfce2b7\" (UID: \"30e59ea6-cd2f-4e47-b4dc-7ff82bfce2b7\") " Dec 03 13:06:26 crc kubenswrapper[5052]: I1203 13:06:26.037583 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"openstack-edpm-tls-custom-tls-dns-default-certs-0\" (UniqueName: \"kubernetes.io/projected/30e59ea6-cd2f-4e47-b4dc-7ff82bfce2b7-openstack-edpm-tls-custom-tls-dns-default-certs-0\") pod \"30e59ea6-cd2f-4e47-b4dc-7ff82bfce2b7\" (UID: \"30e59ea6-cd2f-4e47-b4dc-7ff82bfce2b7\") " Dec 03 13:06:26 crc kubenswrapper[5052]: I1203 13:06:26.042961 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/30e59ea6-cd2f-4e47-b4dc-7ff82bfce2b7-tls-dns-ips-combined-ca-bundle" (OuterVolumeSpecName: "tls-dns-ips-combined-ca-bundle") pod "30e59ea6-cd2f-4e47-b4dc-7ff82bfce2b7" (UID: "30e59ea6-cd2f-4e47-b4dc-7ff82bfce2b7"). InnerVolumeSpecName "tls-dns-ips-combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:06:26 crc kubenswrapper[5052]: I1203 13:06:26.044575 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/30e59ea6-cd2f-4e47-b4dc-7ff82bfce2b7-custom-tls-dns-combined-ca-bundle" (OuterVolumeSpecName: "custom-tls-dns-combined-ca-bundle") pod "30e59ea6-cd2f-4e47-b4dc-7ff82bfce2b7" (UID: "30e59ea6-cd2f-4e47-b4dc-7ff82bfce2b7"). InnerVolumeSpecName "custom-tls-dns-combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:06:26 crc kubenswrapper[5052]: I1203 13:06:26.044620 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/30e59ea6-cd2f-4e47-b4dc-7ff82bfce2b7-openstack-edpm-tls-tls-dns-ips-default-certs-0" (OuterVolumeSpecName: "openstack-edpm-tls-tls-dns-ips-default-certs-0") pod "30e59ea6-cd2f-4e47-b4dc-7ff82bfce2b7" (UID: "30e59ea6-cd2f-4e47-b4dc-7ff82bfce2b7"). InnerVolumeSpecName "openstack-edpm-tls-tls-dns-ips-default-certs-0". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:06:26 crc kubenswrapper[5052]: I1203 13:06:26.044624 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/30e59ea6-cd2f-4e47-b4dc-7ff82bfce2b7-install-certs-ovrd-combined-ca-bundle" (OuterVolumeSpecName: "install-certs-ovrd-combined-ca-bundle") pod "30e59ea6-cd2f-4e47-b4dc-7ff82bfce2b7" (UID: "30e59ea6-cd2f-4e47-b4dc-7ff82bfce2b7"). InnerVolumeSpecName "install-certs-ovrd-combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:06:26 crc kubenswrapper[5052]: I1203 13:06:26.044996 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/30e59ea6-cd2f-4e47-b4dc-7ff82bfce2b7-kube-api-access-rww9c" (OuterVolumeSpecName: "kube-api-access-rww9c") pod "30e59ea6-cd2f-4e47-b4dc-7ff82bfce2b7" (UID: "30e59ea6-cd2f-4e47-b4dc-7ff82bfce2b7"). InnerVolumeSpecName "kube-api-access-rww9c". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:06:26 crc kubenswrapper[5052]: I1203 13:06:26.045953 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/30e59ea6-cd2f-4e47-b4dc-7ff82bfce2b7-openstack-edpm-tls-custom-tls-dns-default-certs-0" (OuterVolumeSpecName: "openstack-edpm-tls-custom-tls-dns-default-certs-0") pod "30e59ea6-cd2f-4e47-b4dc-7ff82bfce2b7" (UID: "30e59ea6-cd2f-4e47-b4dc-7ff82bfce2b7"). InnerVolumeSpecName "openstack-edpm-tls-custom-tls-dns-default-certs-0". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:06:26 crc kubenswrapper[5052]: I1203 13:06:26.063544 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/30e59ea6-cd2f-4e47-b4dc-7ff82bfce2b7-inventory" (OuterVolumeSpecName: "inventory") pod "30e59ea6-cd2f-4e47-b4dc-7ff82bfce2b7" (UID: "30e59ea6-cd2f-4e47-b4dc-7ff82bfce2b7"). InnerVolumeSpecName "inventory". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:06:26 crc kubenswrapper[5052]: I1203 13:06:26.063921 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/30e59ea6-cd2f-4e47-b4dc-7ff82bfce2b7-ssh-key" (OuterVolumeSpecName: "ssh-key") pod "30e59ea6-cd2f-4e47-b4dc-7ff82bfce2b7" (UID: "30e59ea6-cd2f-4e47-b4dc-7ff82bfce2b7"). InnerVolumeSpecName "ssh-key". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:06:26 crc kubenswrapper[5052]: I1203 13:06:26.139107 5052 reconciler_common.go:293] "Volume detached for volume \"tls-dns-ips-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/30e59ea6-cd2f-4e47-b4dc-7ff82bfce2b7-tls-dns-ips-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 13:06:26 crc kubenswrapper[5052]: I1203 13:06:26.139415 5052 reconciler_common.go:293] "Volume detached for volume \"install-certs-ovrd-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/30e59ea6-cd2f-4e47-b4dc-7ff82bfce2b7-install-certs-ovrd-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 13:06:26 crc kubenswrapper[5052]: I1203 13:06:26.139476 5052 reconciler_common.go:293] "Volume detached for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/30e59ea6-cd2f-4e47-b4dc-7ff82bfce2b7-ssh-key\") on node \"crc\" DevicePath \"\"" Dec 03 13:06:26 crc kubenswrapper[5052]: I1203 13:06:26.139546 5052 reconciler_common.go:293] "Volume detached for volume \"openstack-edpm-tls-custom-tls-dns-default-certs-0\" (UniqueName: \"kubernetes.io/projected/30e59ea6-cd2f-4e47-b4dc-7ff82bfce2b7-openstack-edpm-tls-custom-tls-dns-default-certs-0\") on node \"crc\" DevicePath \"\"" Dec 03 13:06:26 crc kubenswrapper[5052]: I1203 13:06:26.139602 5052 reconciler_common.go:293] "Volume detached for volume \"custom-tls-dns-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/30e59ea6-cd2f-4e47-b4dc-7ff82bfce2b7-custom-tls-dns-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 13:06:26 crc kubenswrapper[5052]: I1203 13:06:26.139674 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-rww9c\" (UniqueName: \"kubernetes.io/projected/30e59ea6-cd2f-4e47-b4dc-7ff82bfce2b7-kube-api-access-rww9c\") on node \"crc\" DevicePath \"\"" Dec 03 13:06:26 crc kubenswrapper[5052]: I1203 13:06:26.139729 5052 reconciler_common.go:293] "Volume detached for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/30e59ea6-cd2f-4e47-b4dc-7ff82bfce2b7-inventory\") on node \"crc\" DevicePath \"\"" Dec 03 13:06:26 crc kubenswrapper[5052]: I1203 13:06:26.139779 5052 reconciler_common.go:293] "Volume detached for volume \"openstack-edpm-tls-tls-dns-ips-default-certs-0\" (UniqueName: \"kubernetes.io/projected/30e59ea6-cd2f-4e47-b4dc-7ff82bfce2b7-openstack-edpm-tls-tls-dns-ips-default-certs-0\") on node \"crc\" DevicePath \"\"" Dec 03 13:06:26 crc kubenswrapper[5052]: I1203 13:06:26.540216 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5-debug-bm6z8"] Dec 03 13:06:26 crc kubenswrapper[5052]: E1203 13:06:26.540621 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="30e59ea6-cd2f-4e47-b4dc-7ff82bfce2b7" containerName="install-certs-ovrd-certs-refresh-openstack-edpm-tls" Dec 03 13:06:26 crc kubenswrapper[5052]: I1203 13:06:26.540634 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="30e59ea6-cd2f-4e47-b4dc-7ff82bfce2b7" containerName="install-certs-ovrd-certs-refresh-openstack-edpm-tls" Dec 03 13:06:26 crc kubenswrapper[5052]: I1203 13:06:26.540774 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="30e59ea6-cd2f-4e47-b4dc-7ff82bfce2b7" containerName="install-certs-ovrd-certs-refresh-openstack-edpm-tls" Dec 03 13:06:26 crc kubenswrapper[5052]: I1203 13:06:26.541342 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5-debug-bm6z8" Dec 03 13:06:26 crc kubenswrapper[5052]: I1203 13:06:26.586240 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5-debug-bm6z8"] Dec 03 13:06:26 crc kubenswrapper[5052]: I1203 13:06:26.602048 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5" event={"ID":"30e59ea6-cd2f-4e47-b4dc-7ff82bfce2b7","Type":"ContainerDied","Data":"dedbab00e5960b10da448caf910bdff96790aba930e02de42c5d1f3b90b94393"} Dec 03 13:06:26 crc kubenswrapper[5052]: I1203 13:06:26.602099 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="dedbab00e5960b10da448caf910bdff96790aba930e02de42c5d1f3b90b94393" Dec 03 13:06:26 crc kubenswrapper[5052]: I1203 13:06:26.602232 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5" Dec 03 13:06:26 crc kubenswrapper[5052]: I1203 13:06:26.647674 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"custom-tls-dns-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/dfd64e45-b1eb-43cb-88f4-db4d99d7ec15-custom-tls-dns-combined-ca-bundle\") pod \"install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5-debug-bm6z8\" (UID: \"dfd64e45-b1eb-43cb-88f4-db4d99d7ec15\") " pod="openstack-kuttl-tests/install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5-debug-bm6z8" Dec 03 13:06:26 crc kubenswrapper[5052]: I1203 13:06:26.647741 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openstack-edpm-tls-tls-dns-ips-default-certs-0\" (UniqueName: \"kubernetes.io/projected/dfd64e45-b1eb-43cb-88f4-db4d99d7ec15-openstack-edpm-tls-tls-dns-ips-default-certs-0\") pod \"install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5-debug-bm6z8\" (UID: \"dfd64e45-b1eb-43cb-88f4-db4d99d7ec15\") " pod="openstack-kuttl-tests/install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5-debug-bm6z8" Dec 03 13:06:26 crc kubenswrapper[5052]: I1203 13:06:26.647768 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/dfd64e45-b1eb-43cb-88f4-db4d99d7ec15-inventory\") pod \"install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5-debug-bm6z8\" (UID: \"dfd64e45-b1eb-43cb-88f4-db4d99d7ec15\") " pod="openstack-kuttl-tests/install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5-debug-bm6z8" Dec 03 13:06:26 crc kubenswrapper[5052]: I1203 13:06:26.647841 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-dns-ips-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/dfd64e45-b1eb-43cb-88f4-db4d99d7ec15-tls-dns-ips-combined-ca-bundle\") pod \"install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5-debug-bm6z8\" (UID: \"dfd64e45-b1eb-43cb-88f4-db4d99d7ec15\") " pod="openstack-kuttl-tests/install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5-debug-bm6z8" Dec 03 13:06:26 crc kubenswrapper[5052]: I1203 13:06:26.647878 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/dfd64e45-b1eb-43cb-88f4-db4d99d7ec15-ssh-key\") pod \"install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5-debug-bm6z8\" (UID: \"dfd64e45-b1eb-43cb-88f4-db4d99d7ec15\") " pod="openstack-kuttl-tests/install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5-debug-bm6z8" Dec 03 13:06:26 crc kubenswrapper[5052]: I1203 13:06:26.647896 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openstack-edpm-tls-custom-tls-dns-default-certs-0\" (UniqueName: \"kubernetes.io/projected/dfd64e45-b1eb-43cb-88f4-db4d99d7ec15-openstack-edpm-tls-custom-tls-dns-default-certs-0\") pod \"install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5-debug-bm6z8\" (UID: \"dfd64e45-b1eb-43cb-88f4-db4d99d7ec15\") " pod="openstack-kuttl-tests/install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5-debug-bm6z8" Dec 03 13:06:26 crc kubenswrapper[5052]: I1203 13:06:26.647922 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"install-certs-ovrd-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/dfd64e45-b1eb-43cb-88f4-db4d99d7ec15-install-certs-ovrd-combined-ca-bundle\") pod \"install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5-debug-bm6z8\" (UID: \"dfd64e45-b1eb-43cb-88f4-db4d99d7ec15\") " pod="openstack-kuttl-tests/install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5-debug-bm6z8" Dec 03 13:06:26 crc kubenswrapper[5052]: I1203 13:06:26.647952 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rww9c\" (UniqueName: \"kubernetes.io/projected/dfd64e45-b1eb-43cb-88f4-db4d99d7ec15-kube-api-access-rww9c\") pod \"install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5-debug-bm6z8\" (UID: \"dfd64e45-b1eb-43cb-88f4-db4d99d7ec15\") " pod="openstack-kuttl-tests/install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5-debug-bm6z8" Dec 03 13:06:26 crc kubenswrapper[5052]: I1203 13:06:26.649483 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/tls-dns-ips-certs-refresh-openstack-edpm-tls-lbttg"] Dec 03 13:06:26 crc kubenswrapper[5052]: I1203 13:06:26.650528 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/tls-dns-ips-certs-refresh-openstack-edpm-tls-lbttg" Dec 03 13:06:26 crc kubenswrapper[5052]: I1203 13:06:26.659111 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/tls-dns-ips-certs-refresh-openstack-edpm-tls-lbttg"] Dec 03 13:06:26 crc kubenswrapper[5052]: I1203 13:06:26.749084 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/900d3fa2-4088-4bbd-ae8c-b0630237b304-ssh-key\") pod \"tls-dns-ips-certs-refresh-openstack-edpm-tls-lbttg\" (UID: \"900d3fa2-4088-4bbd-ae8c-b0630237b304\") " pod="openstack-kuttl-tests/tls-dns-ips-certs-refresh-openstack-edpm-tls-lbttg" Dec 03 13:06:26 crc kubenswrapper[5052]: I1203 13:06:26.749152 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-dns-ips-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/900d3fa2-4088-4bbd-ae8c-b0630237b304-tls-dns-ips-combined-ca-bundle\") pod \"tls-dns-ips-certs-refresh-openstack-edpm-tls-lbttg\" (UID: \"900d3fa2-4088-4bbd-ae8c-b0630237b304\") " pod="openstack-kuttl-tests/tls-dns-ips-certs-refresh-openstack-edpm-tls-lbttg" Dec 03 13:06:26 crc kubenswrapper[5052]: I1203 13:06:26.749210 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"openstack-edpm-tls-tls-dns-ips-default-certs-0\" (UniqueName: \"kubernetes.io/projected/dfd64e45-b1eb-43cb-88f4-db4d99d7ec15-openstack-edpm-tls-tls-dns-ips-default-certs-0\") pod \"install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5-debug-bm6z8\" (UID: \"dfd64e45-b1eb-43cb-88f4-db4d99d7ec15\") " pod="openstack-kuttl-tests/install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5-debug-bm6z8" Dec 03 13:06:26 crc kubenswrapper[5052]: I1203 13:06:26.749234 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lcwx6\" (UniqueName: \"kubernetes.io/projected/900d3fa2-4088-4bbd-ae8c-b0630237b304-kube-api-access-lcwx6\") pod \"tls-dns-ips-certs-refresh-openstack-edpm-tls-lbttg\" (UID: \"900d3fa2-4088-4bbd-ae8c-b0630237b304\") " pod="openstack-kuttl-tests/tls-dns-ips-certs-refresh-openstack-edpm-tls-lbttg" Dec 03 13:06:26 crc kubenswrapper[5052]: I1203 13:06:26.749260 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/dfd64e45-b1eb-43cb-88f4-db4d99d7ec15-inventory\") pod \"install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5-debug-bm6z8\" (UID: \"dfd64e45-b1eb-43cb-88f4-db4d99d7ec15\") " pod="openstack-kuttl-tests/install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5-debug-bm6z8" Dec 03 13:06:26 crc kubenswrapper[5052]: I1203 13:06:26.749435 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"tls-dns-ips-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/dfd64e45-b1eb-43cb-88f4-db4d99d7ec15-tls-dns-ips-combined-ca-bundle\") pod \"install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5-debug-bm6z8\" (UID: \"dfd64e45-b1eb-43cb-88f4-db4d99d7ec15\") " pod="openstack-kuttl-tests/install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5-debug-bm6z8" Dec 03 13:06:26 crc kubenswrapper[5052]: I1203 13:06:26.749474 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/dfd64e45-b1eb-43cb-88f4-db4d99d7ec15-ssh-key\") pod \"install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5-debug-bm6z8\" (UID: \"dfd64e45-b1eb-43cb-88f4-db4d99d7ec15\") " pod="openstack-kuttl-tests/install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5-debug-bm6z8" Dec 03 13:06:26 crc kubenswrapper[5052]: I1203 13:06:26.749494 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"openstack-edpm-tls-custom-tls-dns-default-certs-0\" (UniqueName: \"kubernetes.io/projected/dfd64e45-b1eb-43cb-88f4-db4d99d7ec15-openstack-edpm-tls-custom-tls-dns-default-certs-0\") pod \"install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5-debug-bm6z8\" (UID: \"dfd64e45-b1eb-43cb-88f4-db4d99d7ec15\") " pod="openstack-kuttl-tests/install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5-debug-bm6z8" Dec 03 13:06:26 crc kubenswrapper[5052]: I1203 13:06:26.749595 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"install-certs-ovrd-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/dfd64e45-b1eb-43cb-88f4-db4d99d7ec15-install-certs-ovrd-combined-ca-bundle\") pod \"install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5-debug-bm6z8\" (UID: \"dfd64e45-b1eb-43cb-88f4-db4d99d7ec15\") " pod="openstack-kuttl-tests/install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5-debug-bm6z8" Dec 03 13:06:26 crc kubenswrapper[5052]: I1203 13:06:26.749638 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-rww9c\" (UniqueName: \"kubernetes.io/projected/dfd64e45-b1eb-43cb-88f4-db4d99d7ec15-kube-api-access-rww9c\") pod \"install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5-debug-bm6z8\" (UID: \"dfd64e45-b1eb-43cb-88f4-db4d99d7ec15\") " pod="openstack-kuttl-tests/install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5-debug-bm6z8" Dec 03 13:06:26 crc kubenswrapper[5052]: I1203 13:06:26.749677 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/900d3fa2-4088-4bbd-ae8c-b0630237b304-inventory\") pod \"tls-dns-ips-certs-refresh-openstack-edpm-tls-lbttg\" (UID: \"900d3fa2-4088-4bbd-ae8c-b0630237b304\") " pod="openstack-kuttl-tests/tls-dns-ips-certs-refresh-openstack-edpm-tls-lbttg" Dec 03 13:06:26 crc kubenswrapper[5052]: I1203 13:06:26.749710 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"custom-tls-dns-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/dfd64e45-b1eb-43cb-88f4-db4d99d7ec15-custom-tls-dns-combined-ca-bundle\") pod \"install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5-debug-bm6z8\" (UID: \"dfd64e45-b1eb-43cb-88f4-db4d99d7ec15\") " pod="openstack-kuttl-tests/install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5-debug-bm6z8" Dec 03 13:06:26 crc kubenswrapper[5052]: I1203 13:06:26.754313 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"openstack-edpm-tls-tls-dns-ips-default-certs-0\" (UniqueName: \"kubernetes.io/projected/dfd64e45-b1eb-43cb-88f4-db4d99d7ec15-openstack-edpm-tls-tls-dns-ips-default-certs-0\") pod \"install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5-debug-bm6z8\" (UID: \"dfd64e45-b1eb-43cb-88f4-db4d99d7ec15\") " pod="openstack-kuttl-tests/install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5-debug-bm6z8" Dec 03 13:06:26 crc kubenswrapper[5052]: I1203 13:06:26.754368 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"tls-dns-ips-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/dfd64e45-b1eb-43cb-88f4-db4d99d7ec15-tls-dns-ips-combined-ca-bundle\") pod \"install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5-debug-bm6z8\" (UID: \"dfd64e45-b1eb-43cb-88f4-db4d99d7ec15\") " pod="openstack-kuttl-tests/install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5-debug-bm6z8" Dec 03 13:06:26 crc kubenswrapper[5052]: I1203 13:06:26.754677 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"openstack-edpm-tls-custom-tls-dns-default-certs-0\" (UniqueName: \"kubernetes.io/projected/dfd64e45-b1eb-43cb-88f4-db4d99d7ec15-openstack-edpm-tls-custom-tls-dns-default-certs-0\") pod \"install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5-debug-bm6z8\" (UID: \"dfd64e45-b1eb-43cb-88f4-db4d99d7ec15\") " pod="openstack-kuttl-tests/install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5-debug-bm6z8" Dec 03 13:06:26 crc kubenswrapper[5052]: I1203 13:06:26.755189 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"install-certs-ovrd-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/dfd64e45-b1eb-43cb-88f4-db4d99d7ec15-install-certs-ovrd-combined-ca-bundle\") pod \"install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5-debug-bm6z8\" (UID: \"dfd64e45-b1eb-43cb-88f4-db4d99d7ec15\") " pod="openstack-kuttl-tests/install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5-debug-bm6z8" Dec 03 13:06:26 crc kubenswrapper[5052]: I1203 13:06:26.755744 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/dfd64e45-b1eb-43cb-88f4-db4d99d7ec15-ssh-key\") pod \"install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5-debug-bm6z8\" (UID: \"dfd64e45-b1eb-43cb-88f4-db4d99d7ec15\") " pod="openstack-kuttl-tests/install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5-debug-bm6z8" Dec 03 13:06:26 crc kubenswrapper[5052]: I1203 13:06:26.756306 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"custom-tls-dns-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/dfd64e45-b1eb-43cb-88f4-db4d99d7ec15-custom-tls-dns-combined-ca-bundle\") pod \"install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5-debug-bm6z8\" (UID: \"dfd64e45-b1eb-43cb-88f4-db4d99d7ec15\") " pod="openstack-kuttl-tests/install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5-debug-bm6z8" Dec 03 13:06:26 crc kubenswrapper[5052]: I1203 13:06:26.757058 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/dfd64e45-b1eb-43cb-88f4-db4d99d7ec15-inventory\") pod \"install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5-debug-bm6z8\" (UID: \"dfd64e45-b1eb-43cb-88f4-db4d99d7ec15\") " pod="openstack-kuttl-tests/install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5-debug-bm6z8" Dec 03 13:06:26 crc kubenswrapper[5052]: I1203 13:06:26.767903 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-rww9c\" (UniqueName: \"kubernetes.io/projected/dfd64e45-b1eb-43cb-88f4-db4d99d7ec15-kube-api-access-rww9c\") pod \"install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5-debug-bm6z8\" (UID: \"dfd64e45-b1eb-43cb-88f4-db4d99d7ec15\") " pod="openstack-kuttl-tests/install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5-debug-bm6z8" Dec 03 13:06:26 crc kubenswrapper[5052]: I1203 13:06:26.851784 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-lcwx6\" (UniqueName: \"kubernetes.io/projected/900d3fa2-4088-4bbd-ae8c-b0630237b304-kube-api-access-lcwx6\") pod \"tls-dns-ips-certs-refresh-openstack-edpm-tls-lbttg\" (UID: \"900d3fa2-4088-4bbd-ae8c-b0630237b304\") " pod="openstack-kuttl-tests/tls-dns-ips-certs-refresh-openstack-edpm-tls-lbttg" Dec 03 13:06:26 crc kubenswrapper[5052]: I1203 13:06:26.852368 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/900d3fa2-4088-4bbd-ae8c-b0630237b304-inventory\") pod \"tls-dns-ips-certs-refresh-openstack-edpm-tls-lbttg\" (UID: \"900d3fa2-4088-4bbd-ae8c-b0630237b304\") " pod="openstack-kuttl-tests/tls-dns-ips-certs-refresh-openstack-edpm-tls-lbttg" Dec 03 13:06:26 crc kubenswrapper[5052]: I1203 13:06:26.852423 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/900d3fa2-4088-4bbd-ae8c-b0630237b304-ssh-key\") pod \"tls-dns-ips-certs-refresh-openstack-edpm-tls-lbttg\" (UID: \"900d3fa2-4088-4bbd-ae8c-b0630237b304\") " pod="openstack-kuttl-tests/tls-dns-ips-certs-refresh-openstack-edpm-tls-lbttg" Dec 03 13:06:26 crc kubenswrapper[5052]: I1203 13:06:26.852450 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"tls-dns-ips-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/900d3fa2-4088-4bbd-ae8c-b0630237b304-tls-dns-ips-combined-ca-bundle\") pod \"tls-dns-ips-certs-refresh-openstack-edpm-tls-lbttg\" (UID: \"900d3fa2-4088-4bbd-ae8c-b0630237b304\") " pod="openstack-kuttl-tests/tls-dns-ips-certs-refresh-openstack-edpm-tls-lbttg" Dec 03 13:06:26 crc kubenswrapper[5052]: I1203 13:06:26.856327 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/900d3fa2-4088-4bbd-ae8c-b0630237b304-inventory\") pod \"tls-dns-ips-certs-refresh-openstack-edpm-tls-lbttg\" (UID: \"900d3fa2-4088-4bbd-ae8c-b0630237b304\") " pod="openstack-kuttl-tests/tls-dns-ips-certs-refresh-openstack-edpm-tls-lbttg" Dec 03 13:06:26 crc kubenswrapper[5052]: I1203 13:06:26.856973 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5-debug-bm6z8" Dec 03 13:06:26 crc kubenswrapper[5052]: E1203 13:06:26.857044 5052 kubelet_pods.go:538] "Hostname for pod was too long, truncated it" podName="install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5-debug-bm6z8" hostnameMaxLen=63 truncatedHostname="install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5-debug" Dec 03 13:06:26 crc kubenswrapper[5052]: I1203 13:06:26.871406 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"tls-dns-ips-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/900d3fa2-4088-4bbd-ae8c-b0630237b304-tls-dns-ips-combined-ca-bundle\") pod \"tls-dns-ips-certs-refresh-openstack-edpm-tls-lbttg\" (UID: \"900d3fa2-4088-4bbd-ae8c-b0630237b304\") " pod="openstack-kuttl-tests/tls-dns-ips-certs-refresh-openstack-edpm-tls-lbttg" Dec 03 13:06:26 crc kubenswrapper[5052]: I1203 13:06:26.872057 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/900d3fa2-4088-4bbd-ae8c-b0630237b304-ssh-key\") pod \"tls-dns-ips-certs-refresh-openstack-edpm-tls-lbttg\" (UID: \"900d3fa2-4088-4bbd-ae8c-b0630237b304\") " pod="openstack-kuttl-tests/tls-dns-ips-certs-refresh-openstack-edpm-tls-lbttg" Dec 03 13:06:26 crc kubenswrapper[5052]: I1203 13:06:26.884320 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-lcwx6\" (UniqueName: \"kubernetes.io/projected/900d3fa2-4088-4bbd-ae8c-b0630237b304-kube-api-access-lcwx6\") pod \"tls-dns-ips-certs-refresh-openstack-edpm-tls-lbttg\" (UID: \"900d3fa2-4088-4bbd-ae8c-b0630237b304\") " pod="openstack-kuttl-tests/tls-dns-ips-certs-refresh-openstack-edpm-tls-lbttg" Dec 03 13:06:26 crc kubenswrapper[5052]: I1203 13:06:26.972085 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/tls-dns-ips-certs-refresh-openstack-edpm-tls-lbttg" Dec 03 13:06:27 crc kubenswrapper[5052]: I1203 13:06:27.295563 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5-debug-bm6z8"] Dec 03 13:06:27 crc kubenswrapper[5052]: E1203 13:06:27.300662 5052 kubelet_pods.go:538] "Hostname for pod was too long, truncated it" podName="install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5-debug-bm6z8" hostnameMaxLen=63 truncatedHostname="install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5-debug" Dec 03 13:06:27 crc kubenswrapper[5052]: I1203 13:06:27.393216 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/tls-dns-ips-certs-refresh-openstack-edpm-tls-lbttg"] Dec 03 13:06:27 crc kubenswrapper[5052]: W1203 13:06:27.395183 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod900d3fa2_4088_4bbd_ae8c_b0630237b304.slice/crio-0246c016f7866eea97ceebb7b19ca5cf8b7d9deda4cb0a0a79620ced7ed15993 WatchSource:0}: Error finding container 0246c016f7866eea97ceebb7b19ca5cf8b7d9deda4cb0a0a79620ced7ed15993: Status 404 returned error can't find the container with id 0246c016f7866eea97ceebb7b19ca5cf8b7d9deda4cb0a0a79620ced7ed15993 Dec 03 13:06:27 crc kubenswrapper[5052]: I1203 13:06:27.611830 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/tls-dns-ips-certs-refresh-openstack-edpm-tls-lbttg" event={"ID":"900d3fa2-4088-4bbd-ae8c-b0630237b304","Type":"ContainerStarted","Data":"0246c016f7866eea97ceebb7b19ca5cf8b7d9deda4cb0a0a79620ced7ed15993"} Dec 03 13:06:27 crc kubenswrapper[5052]: I1203 13:06:27.613407 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5-debug-bm6z8" event={"ID":"dfd64e45-b1eb-43cb-88f4-db4d99d7ec15","Type":"ContainerStarted","Data":"fcce0544421c6b5058dca134af72021e23f68732c44784f761726ad064d62e33"} Dec 03 13:06:27 crc kubenswrapper[5052]: E1203 13:06:27.903820 5052 kubelet_pods.go:538] "Hostname for pod was too long, truncated it" podName="install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5-debug-bm6z8" hostnameMaxLen=63 truncatedHostname="install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5-debug" Dec 03 13:06:28 crc kubenswrapper[5052]: I1203 13:06:28.633185 5052 generic.go:334] "Generic (PLEG): container finished" podID="dfd64e45-b1eb-43cb-88f4-db4d99d7ec15" containerID="fd20510142842c76468886580985329724397e31de0bb39c1f850edfe65f60aa" exitCode=0 Dec 03 13:06:28 crc kubenswrapper[5052]: I1203 13:06:28.633241 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5-debug-bm6z8" event={"ID":"dfd64e45-b1eb-43cb-88f4-db4d99d7ec15","Type":"ContainerDied","Data":"fd20510142842c76468886580985329724397e31de0bb39c1f850edfe65f60aa"} Dec 03 13:06:28 crc kubenswrapper[5052]: I1203 13:06:28.635977 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/tls-dns-ips-certs-refresh-openstack-edpm-tls-lbttg" event={"ID":"900d3fa2-4088-4bbd-ae8c-b0630237b304","Type":"ContainerStarted","Data":"cef1bc7a06434159cd29dfdbf2e3083b4d8c6668507619974f0b6b08c96991e3"} Dec 03 13:06:28 crc kubenswrapper[5052]: I1203 13:06:28.676305 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/tls-dns-ips-certs-refresh-openstack-edpm-tls-lbttg" podStartSLOduration=2.21657382 podStartE2EDuration="2.676285336s" podCreationTimestamp="2025-12-03 13:06:26 +0000 UTC" firstStartedPulling="2025-12-03 13:06:27.397965889 +0000 UTC m=+6377.986207653" lastFinishedPulling="2025-12-03 13:06:27.857677395 +0000 UTC m=+6378.445919169" observedRunningTime="2025-12-03 13:06:28.670253823 +0000 UTC m=+6379.258495587" watchObservedRunningTime="2025-12-03 13:06:28.676285336 +0000 UTC m=+6379.264527110" Dec 03 13:06:28 crc kubenswrapper[5052]: I1203 13:06:28.693727 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5-debug-bm6z8"] Dec 03 13:06:28 crc kubenswrapper[5052]: I1203 13:06:28.702001 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5-debug-bm6z8"] Dec 03 13:06:29 crc kubenswrapper[5052]: I1203 13:06:29.030929 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5-debug-pcs86"] Dec 03 13:06:29 crc kubenswrapper[5052]: E1203 13:06:29.031372 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="dfd64e45-b1eb-43cb-88f4-db4d99d7ec15" containerName="install-certs-ovrd-certs-refresh-openstack-edpm-tls" Dec 03 13:06:29 crc kubenswrapper[5052]: I1203 13:06:29.031389 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="dfd64e45-b1eb-43cb-88f4-db4d99d7ec15" containerName="install-certs-ovrd-certs-refresh-openstack-edpm-tls" Dec 03 13:06:29 crc kubenswrapper[5052]: I1203 13:06:29.031583 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="dfd64e45-b1eb-43cb-88f4-db4d99d7ec15" containerName="install-certs-ovrd-certs-refresh-openstack-edpm-tls" Dec 03 13:06:29 crc kubenswrapper[5052]: I1203 13:06:29.032185 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5-debug-pcs86" Dec 03 13:06:29 crc kubenswrapper[5052]: I1203 13:06:29.100630 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openstack-edpm-tls-tls-dns-ips-default-certs-0\" (UniqueName: \"kubernetes.io/projected/381fd12d-de34-40b8-a720-e6400b673a54-openstack-edpm-tls-tls-dns-ips-default-certs-0\") pod \"install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5-debug-pcs86\" (UID: \"381fd12d-de34-40b8-a720-e6400b673a54\") " pod="openstack-kuttl-tests/install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5-debug-pcs86" Dec 03 13:06:29 crc kubenswrapper[5052]: I1203 13:06:29.100747 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/381fd12d-de34-40b8-a720-e6400b673a54-inventory\") pod \"install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5-debug-pcs86\" (UID: \"381fd12d-de34-40b8-a720-e6400b673a54\") " pod="openstack-kuttl-tests/install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5-debug-pcs86" Dec 03 13:06:29 crc kubenswrapper[5052]: I1203 13:06:29.100867 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openstack-edpm-tls-custom-tls-dns-default-certs-0\" (UniqueName: \"kubernetes.io/projected/381fd12d-de34-40b8-a720-e6400b673a54-openstack-edpm-tls-custom-tls-dns-default-certs-0\") pod \"install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5-debug-pcs86\" (UID: \"381fd12d-de34-40b8-a720-e6400b673a54\") " pod="openstack-kuttl-tests/install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5-debug-pcs86" Dec 03 13:06:29 crc kubenswrapper[5052]: I1203 13:06:29.100964 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-dns-ips-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/381fd12d-de34-40b8-a720-e6400b673a54-tls-dns-ips-combined-ca-bundle\") pod \"install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5-debug-pcs86\" (UID: \"381fd12d-de34-40b8-a720-e6400b673a54\") " pod="openstack-kuttl-tests/install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5-debug-pcs86" Dec 03 13:06:29 crc kubenswrapper[5052]: I1203 13:06:29.101056 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/381fd12d-de34-40b8-a720-e6400b673a54-ssh-key\") pod \"install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5-debug-pcs86\" (UID: \"381fd12d-de34-40b8-a720-e6400b673a54\") " pod="openstack-kuttl-tests/install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5-debug-pcs86" Dec 03 13:06:29 crc kubenswrapper[5052]: I1203 13:06:29.101102 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"custom-tls-dns-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/381fd12d-de34-40b8-a720-e6400b673a54-custom-tls-dns-combined-ca-bundle\") pod \"install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5-debug-pcs86\" (UID: \"381fd12d-de34-40b8-a720-e6400b673a54\") " pod="openstack-kuttl-tests/install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5-debug-pcs86" Dec 03 13:06:29 crc kubenswrapper[5052]: I1203 13:06:29.101177 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rww9c\" (UniqueName: \"kubernetes.io/projected/381fd12d-de34-40b8-a720-e6400b673a54-kube-api-access-rww9c\") pod \"install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5-debug-pcs86\" (UID: \"381fd12d-de34-40b8-a720-e6400b673a54\") " pod="openstack-kuttl-tests/install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5-debug-pcs86" Dec 03 13:06:29 crc kubenswrapper[5052]: I1203 13:06:29.101403 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"install-certs-ovrd-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/381fd12d-de34-40b8-a720-e6400b673a54-install-certs-ovrd-combined-ca-bundle\") pod \"install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5-debug-pcs86\" (UID: \"381fd12d-de34-40b8-a720-e6400b673a54\") " pod="openstack-kuttl-tests/install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5-debug-pcs86" Dec 03 13:06:29 crc kubenswrapper[5052]: I1203 13:06:29.101747 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5-debug-pcs86"] Dec 03 13:06:29 crc kubenswrapper[5052]: I1203 13:06:29.202414 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"custom-tls-dns-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/381fd12d-de34-40b8-a720-e6400b673a54-custom-tls-dns-combined-ca-bundle\") pod \"install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5-debug-pcs86\" (UID: \"381fd12d-de34-40b8-a720-e6400b673a54\") " pod="openstack-kuttl-tests/install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5-debug-pcs86" Dec 03 13:06:29 crc kubenswrapper[5052]: I1203 13:06:29.202467 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-rww9c\" (UniqueName: \"kubernetes.io/projected/381fd12d-de34-40b8-a720-e6400b673a54-kube-api-access-rww9c\") pod \"install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5-debug-pcs86\" (UID: \"381fd12d-de34-40b8-a720-e6400b673a54\") " pod="openstack-kuttl-tests/install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5-debug-pcs86" Dec 03 13:06:29 crc kubenswrapper[5052]: I1203 13:06:29.202572 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"install-certs-ovrd-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/381fd12d-de34-40b8-a720-e6400b673a54-install-certs-ovrd-combined-ca-bundle\") pod \"install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5-debug-pcs86\" (UID: \"381fd12d-de34-40b8-a720-e6400b673a54\") " pod="openstack-kuttl-tests/install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5-debug-pcs86" Dec 03 13:06:29 crc kubenswrapper[5052]: I1203 13:06:29.202603 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"openstack-edpm-tls-tls-dns-ips-default-certs-0\" (UniqueName: \"kubernetes.io/projected/381fd12d-de34-40b8-a720-e6400b673a54-openstack-edpm-tls-tls-dns-ips-default-certs-0\") pod \"install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5-debug-pcs86\" (UID: \"381fd12d-de34-40b8-a720-e6400b673a54\") " pod="openstack-kuttl-tests/install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5-debug-pcs86" Dec 03 13:06:29 crc kubenswrapper[5052]: I1203 13:06:29.202626 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/381fd12d-de34-40b8-a720-e6400b673a54-inventory\") pod \"install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5-debug-pcs86\" (UID: \"381fd12d-de34-40b8-a720-e6400b673a54\") " pod="openstack-kuttl-tests/install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5-debug-pcs86" Dec 03 13:06:29 crc kubenswrapper[5052]: I1203 13:06:29.202656 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"openstack-edpm-tls-custom-tls-dns-default-certs-0\" (UniqueName: \"kubernetes.io/projected/381fd12d-de34-40b8-a720-e6400b673a54-openstack-edpm-tls-custom-tls-dns-default-certs-0\") pod \"install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5-debug-pcs86\" (UID: \"381fd12d-de34-40b8-a720-e6400b673a54\") " pod="openstack-kuttl-tests/install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5-debug-pcs86" Dec 03 13:06:29 crc kubenswrapper[5052]: I1203 13:06:29.202681 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"tls-dns-ips-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/381fd12d-de34-40b8-a720-e6400b673a54-tls-dns-ips-combined-ca-bundle\") pod \"install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5-debug-pcs86\" (UID: \"381fd12d-de34-40b8-a720-e6400b673a54\") " pod="openstack-kuttl-tests/install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5-debug-pcs86" Dec 03 13:06:29 crc kubenswrapper[5052]: I1203 13:06:29.202711 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/381fd12d-de34-40b8-a720-e6400b673a54-ssh-key\") pod \"install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5-debug-pcs86\" (UID: \"381fd12d-de34-40b8-a720-e6400b673a54\") " pod="openstack-kuttl-tests/install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5-debug-pcs86" Dec 03 13:06:29 crc kubenswrapper[5052]: I1203 13:06:29.207418 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"custom-tls-dns-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/381fd12d-de34-40b8-a720-e6400b673a54-custom-tls-dns-combined-ca-bundle\") pod \"install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5-debug-pcs86\" (UID: \"381fd12d-de34-40b8-a720-e6400b673a54\") " pod="openstack-kuttl-tests/install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5-debug-pcs86" Dec 03 13:06:29 crc kubenswrapper[5052]: I1203 13:06:29.207429 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/381fd12d-de34-40b8-a720-e6400b673a54-inventory\") pod \"install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5-debug-pcs86\" (UID: \"381fd12d-de34-40b8-a720-e6400b673a54\") " pod="openstack-kuttl-tests/install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5-debug-pcs86" Dec 03 13:06:29 crc kubenswrapper[5052]: I1203 13:06:29.207984 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"install-certs-ovrd-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/381fd12d-de34-40b8-a720-e6400b673a54-install-certs-ovrd-combined-ca-bundle\") pod \"install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5-debug-pcs86\" (UID: \"381fd12d-de34-40b8-a720-e6400b673a54\") " pod="openstack-kuttl-tests/install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5-debug-pcs86" Dec 03 13:06:29 crc kubenswrapper[5052]: I1203 13:06:29.208808 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"openstack-edpm-tls-custom-tls-dns-default-certs-0\" (UniqueName: \"kubernetes.io/projected/381fd12d-de34-40b8-a720-e6400b673a54-openstack-edpm-tls-custom-tls-dns-default-certs-0\") pod \"install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5-debug-pcs86\" (UID: \"381fd12d-de34-40b8-a720-e6400b673a54\") " pod="openstack-kuttl-tests/install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5-debug-pcs86" Dec 03 13:06:29 crc kubenswrapper[5052]: I1203 13:06:29.208887 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"openstack-edpm-tls-tls-dns-ips-default-certs-0\" (UniqueName: \"kubernetes.io/projected/381fd12d-de34-40b8-a720-e6400b673a54-openstack-edpm-tls-tls-dns-ips-default-certs-0\") pod \"install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5-debug-pcs86\" (UID: \"381fd12d-de34-40b8-a720-e6400b673a54\") " pod="openstack-kuttl-tests/install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5-debug-pcs86" Dec 03 13:06:29 crc kubenswrapper[5052]: I1203 13:06:29.208916 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"tls-dns-ips-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/381fd12d-de34-40b8-a720-e6400b673a54-tls-dns-ips-combined-ca-bundle\") pod \"install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5-debug-pcs86\" (UID: \"381fd12d-de34-40b8-a720-e6400b673a54\") " pod="openstack-kuttl-tests/install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5-debug-pcs86" Dec 03 13:06:29 crc kubenswrapper[5052]: I1203 13:06:29.209065 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/381fd12d-de34-40b8-a720-e6400b673a54-ssh-key\") pod \"install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5-debug-pcs86\" (UID: \"381fd12d-de34-40b8-a720-e6400b673a54\") " pod="openstack-kuttl-tests/install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5-debug-pcs86" Dec 03 13:06:29 crc kubenswrapper[5052]: I1203 13:06:29.219853 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-rww9c\" (UniqueName: \"kubernetes.io/projected/381fd12d-de34-40b8-a720-e6400b673a54-kube-api-access-rww9c\") pod \"install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5-debug-pcs86\" (UID: \"381fd12d-de34-40b8-a720-e6400b673a54\") " pod="openstack-kuttl-tests/install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5-debug-pcs86" Dec 03 13:06:29 crc kubenswrapper[5052]: I1203 13:06:29.379086 5052 patch_prober.go:28] interesting pod/machine-config-daemon-74bj6 container/machine-config-daemon namespace/openshift-machine-config-operator: Liveness probe status=failure output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" start-of-body= Dec 03 13:06:29 crc kubenswrapper[5052]: I1203 13:06:29.379386 5052 prober.go:107] "Probe failed" probeType="Liveness" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" containerName="machine-config-daemon" probeResult="failure" output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" Dec 03 13:06:29 crc kubenswrapper[5052]: I1203 13:06:29.408667 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5-debug-pcs86" Dec 03 13:06:29 crc kubenswrapper[5052]: E1203 13:06:29.408778 5052 kubelet_pods.go:538] "Hostname for pod was too long, truncated it" podName="install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5-debug-pcs86" hostnameMaxLen=63 truncatedHostname="install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5-debug" Dec 03 13:06:29 crc kubenswrapper[5052]: I1203 13:06:29.815277 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5-debug-pcs86"] Dec 03 13:06:29 crc kubenswrapper[5052]: W1203 13:06:29.821754 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod381fd12d_de34_40b8_a720_e6400b673a54.slice/crio-8ac751e8dd43a59aacd074b9562ed30c0db70c5aa2abd9b8f920252a153ce6f9 WatchSource:0}: Error finding container 8ac751e8dd43a59aacd074b9562ed30c0db70c5aa2abd9b8f920252a153ce6f9: Status 404 returned error can't find the container with id 8ac751e8dd43a59aacd074b9562ed30c0db70c5aa2abd9b8f920252a153ce6f9 Dec 03 13:06:29 crc kubenswrapper[5052]: E1203 13:06:29.822194 5052 kubelet_pods.go:538] "Hostname for pod was too long, truncated it" podName="install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5-debug-pcs86" hostnameMaxLen=63 truncatedHostname="install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5-debug" Dec 03 13:06:29 crc kubenswrapper[5052]: I1203 13:06:29.864341 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5-debug-bm6z8" Dec 03 13:06:29 crc kubenswrapper[5052]: I1203 13:06:29.914506 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rww9c\" (UniqueName: \"kubernetes.io/projected/dfd64e45-b1eb-43cb-88f4-db4d99d7ec15-kube-api-access-rww9c\") pod \"dfd64e45-b1eb-43cb-88f4-db4d99d7ec15\" (UID: \"dfd64e45-b1eb-43cb-88f4-db4d99d7ec15\") " Dec 03 13:06:29 crc kubenswrapper[5052]: I1203 13:06:29.914591 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"openstack-edpm-tls-tls-dns-ips-default-certs-0\" (UniqueName: \"kubernetes.io/projected/dfd64e45-b1eb-43cb-88f4-db4d99d7ec15-openstack-edpm-tls-tls-dns-ips-default-certs-0\") pod \"dfd64e45-b1eb-43cb-88f4-db4d99d7ec15\" (UID: \"dfd64e45-b1eb-43cb-88f4-db4d99d7ec15\") " Dec 03 13:06:29 crc kubenswrapper[5052]: I1203 13:06:29.914646 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"openstack-edpm-tls-custom-tls-dns-default-certs-0\" (UniqueName: \"kubernetes.io/projected/dfd64e45-b1eb-43cb-88f4-db4d99d7ec15-openstack-edpm-tls-custom-tls-dns-default-certs-0\") pod \"dfd64e45-b1eb-43cb-88f4-db4d99d7ec15\" (UID: \"dfd64e45-b1eb-43cb-88f4-db4d99d7ec15\") " Dec 03 13:06:29 crc kubenswrapper[5052]: I1203 13:06:29.914667 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"install-certs-ovrd-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/dfd64e45-b1eb-43cb-88f4-db4d99d7ec15-install-certs-ovrd-combined-ca-bundle\") pod \"dfd64e45-b1eb-43cb-88f4-db4d99d7ec15\" (UID: \"dfd64e45-b1eb-43cb-88f4-db4d99d7ec15\") " Dec 03 13:06:29 crc kubenswrapper[5052]: I1203 13:06:29.914713 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/dfd64e45-b1eb-43cb-88f4-db4d99d7ec15-ssh-key\") pod \"dfd64e45-b1eb-43cb-88f4-db4d99d7ec15\" (UID: \"dfd64e45-b1eb-43cb-88f4-db4d99d7ec15\") " Dec 03 13:06:29 crc kubenswrapper[5052]: I1203 13:06:29.914732 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"custom-tls-dns-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/dfd64e45-b1eb-43cb-88f4-db4d99d7ec15-custom-tls-dns-combined-ca-bundle\") pod \"dfd64e45-b1eb-43cb-88f4-db4d99d7ec15\" (UID: \"dfd64e45-b1eb-43cb-88f4-db4d99d7ec15\") " Dec 03 13:06:29 crc kubenswrapper[5052]: I1203 13:06:29.914774 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/dfd64e45-b1eb-43cb-88f4-db4d99d7ec15-inventory\") pod \"dfd64e45-b1eb-43cb-88f4-db4d99d7ec15\" (UID: \"dfd64e45-b1eb-43cb-88f4-db4d99d7ec15\") " Dec 03 13:06:29 crc kubenswrapper[5052]: I1203 13:06:29.914794 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"tls-dns-ips-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/dfd64e45-b1eb-43cb-88f4-db4d99d7ec15-tls-dns-ips-combined-ca-bundle\") pod \"dfd64e45-b1eb-43cb-88f4-db4d99d7ec15\" (UID: \"dfd64e45-b1eb-43cb-88f4-db4d99d7ec15\") " Dec 03 13:06:29 crc kubenswrapper[5052]: I1203 13:06:29.920051 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dfd64e45-b1eb-43cb-88f4-db4d99d7ec15-install-certs-ovrd-combined-ca-bundle" (OuterVolumeSpecName: "install-certs-ovrd-combined-ca-bundle") pod "dfd64e45-b1eb-43cb-88f4-db4d99d7ec15" (UID: "dfd64e45-b1eb-43cb-88f4-db4d99d7ec15"). InnerVolumeSpecName "install-certs-ovrd-combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:06:29 crc kubenswrapper[5052]: I1203 13:06:29.920149 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dfd64e45-b1eb-43cb-88f4-db4d99d7ec15-openstack-edpm-tls-custom-tls-dns-default-certs-0" (OuterVolumeSpecName: "openstack-edpm-tls-custom-tls-dns-default-certs-0") pod "dfd64e45-b1eb-43cb-88f4-db4d99d7ec15" (UID: "dfd64e45-b1eb-43cb-88f4-db4d99d7ec15"). InnerVolumeSpecName "openstack-edpm-tls-custom-tls-dns-default-certs-0". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:06:29 crc kubenswrapper[5052]: I1203 13:06:29.920196 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dfd64e45-b1eb-43cb-88f4-db4d99d7ec15-openstack-edpm-tls-tls-dns-ips-default-certs-0" (OuterVolumeSpecName: "openstack-edpm-tls-tls-dns-ips-default-certs-0") pod "dfd64e45-b1eb-43cb-88f4-db4d99d7ec15" (UID: "dfd64e45-b1eb-43cb-88f4-db4d99d7ec15"). InnerVolumeSpecName "openstack-edpm-tls-tls-dns-ips-default-certs-0". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:06:29 crc kubenswrapper[5052]: I1203 13:06:29.920193 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dfd64e45-b1eb-43cb-88f4-db4d99d7ec15-custom-tls-dns-combined-ca-bundle" (OuterVolumeSpecName: "custom-tls-dns-combined-ca-bundle") pod "dfd64e45-b1eb-43cb-88f4-db4d99d7ec15" (UID: "dfd64e45-b1eb-43cb-88f4-db4d99d7ec15"). InnerVolumeSpecName "custom-tls-dns-combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:06:29 crc kubenswrapper[5052]: I1203 13:06:29.920449 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dfd64e45-b1eb-43cb-88f4-db4d99d7ec15-kube-api-access-rww9c" (OuterVolumeSpecName: "kube-api-access-rww9c") pod "dfd64e45-b1eb-43cb-88f4-db4d99d7ec15" (UID: "dfd64e45-b1eb-43cb-88f4-db4d99d7ec15"). InnerVolumeSpecName "kube-api-access-rww9c". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:06:29 crc kubenswrapper[5052]: I1203 13:06:29.921038 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dfd64e45-b1eb-43cb-88f4-db4d99d7ec15-tls-dns-ips-combined-ca-bundle" (OuterVolumeSpecName: "tls-dns-ips-combined-ca-bundle") pod "dfd64e45-b1eb-43cb-88f4-db4d99d7ec15" (UID: "dfd64e45-b1eb-43cb-88f4-db4d99d7ec15"). InnerVolumeSpecName "tls-dns-ips-combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:06:29 crc kubenswrapper[5052]: I1203 13:06:29.935678 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dfd64e45-b1eb-43cb-88f4-db4d99d7ec15-inventory" (OuterVolumeSpecName: "inventory") pod "dfd64e45-b1eb-43cb-88f4-db4d99d7ec15" (UID: "dfd64e45-b1eb-43cb-88f4-db4d99d7ec15"). InnerVolumeSpecName "inventory". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:06:29 crc kubenswrapper[5052]: I1203 13:06:29.939012 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dfd64e45-b1eb-43cb-88f4-db4d99d7ec15-ssh-key" (OuterVolumeSpecName: "ssh-key") pod "dfd64e45-b1eb-43cb-88f4-db4d99d7ec15" (UID: "dfd64e45-b1eb-43cb-88f4-db4d99d7ec15"). InnerVolumeSpecName "ssh-key". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:06:30 crc kubenswrapper[5052]: I1203 13:06:30.027032 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-rww9c\" (UniqueName: \"kubernetes.io/projected/dfd64e45-b1eb-43cb-88f4-db4d99d7ec15-kube-api-access-rww9c\") on node \"crc\" DevicePath \"\"" Dec 03 13:06:30 crc kubenswrapper[5052]: I1203 13:06:30.027265 5052 reconciler_common.go:293] "Volume detached for volume \"openstack-edpm-tls-tls-dns-ips-default-certs-0\" (UniqueName: \"kubernetes.io/projected/dfd64e45-b1eb-43cb-88f4-db4d99d7ec15-openstack-edpm-tls-tls-dns-ips-default-certs-0\") on node \"crc\" DevicePath \"\"" Dec 03 13:06:30 crc kubenswrapper[5052]: I1203 13:06:30.027334 5052 reconciler_common.go:293] "Volume detached for volume \"openstack-edpm-tls-custom-tls-dns-default-certs-0\" (UniqueName: \"kubernetes.io/projected/dfd64e45-b1eb-43cb-88f4-db4d99d7ec15-openstack-edpm-tls-custom-tls-dns-default-certs-0\") on node \"crc\" DevicePath \"\"" Dec 03 13:06:30 crc kubenswrapper[5052]: I1203 13:06:30.027426 5052 reconciler_common.go:293] "Volume detached for volume \"install-certs-ovrd-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/dfd64e45-b1eb-43cb-88f4-db4d99d7ec15-install-certs-ovrd-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 13:06:30 crc kubenswrapper[5052]: I1203 13:06:30.027487 5052 reconciler_common.go:293] "Volume detached for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/dfd64e45-b1eb-43cb-88f4-db4d99d7ec15-ssh-key\") on node \"crc\" DevicePath \"\"" Dec 03 13:06:30 crc kubenswrapper[5052]: I1203 13:06:30.027585 5052 reconciler_common.go:293] "Volume detached for volume \"custom-tls-dns-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/dfd64e45-b1eb-43cb-88f4-db4d99d7ec15-custom-tls-dns-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 13:06:30 crc kubenswrapper[5052]: I1203 13:06:30.027703 5052 reconciler_common.go:293] "Volume detached for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/dfd64e45-b1eb-43cb-88f4-db4d99d7ec15-inventory\") on node \"crc\" DevicePath \"\"" Dec 03 13:06:30 crc kubenswrapper[5052]: I1203 13:06:30.027798 5052 reconciler_common.go:293] "Volume detached for volume \"tls-dns-ips-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/dfd64e45-b1eb-43cb-88f4-db4d99d7ec15-tls-dns-ips-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 13:06:30 crc kubenswrapper[5052]: E1203 13:06:30.291149 5052 kubelet_pods.go:538] "Hostname for pod was too long, truncated it" podName="install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5-debug-pcs86" hostnameMaxLen=63 truncatedHostname="install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5-debug" Dec 03 13:06:30 crc kubenswrapper[5052]: I1203 13:06:30.652413 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5-debug-pcs86" event={"ID":"381fd12d-de34-40b8-a720-e6400b673a54","Type":"ContainerStarted","Data":"2cc90b07e9c5d356befcb31dfd88b1afe512c689fb82ca3214afc9db1aafdce0"} Dec 03 13:06:30 crc kubenswrapper[5052]: I1203 13:06:30.653691 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5-debug-pcs86" event={"ID":"381fd12d-de34-40b8-a720-e6400b673a54","Type":"ContainerStarted","Data":"8ac751e8dd43a59aacd074b9562ed30c0db70c5aa2abd9b8f920252a153ce6f9"} Dec 03 13:06:30 crc kubenswrapper[5052]: I1203 13:06:30.654198 5052 scope.go:117] "RemoveContainer" containerID="fd20510142842c76468886580985329724397e31de0bb39c1f850edfe65f60aa" Dec 03 13:06:30 crc kubenswrapper[5052]: I1203 13:06:30.654296 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5-debug-bm6z8" Dec 03 13:06:31 crc kubenswrapper[5052]: I1203 13:06:31.668870 5052 generic.go:334] "Generic (PLEG): container finished" podID="381fd12d-de34-40b8-a720-e6400b673a54" containerID="2cc90b07e9c5d356befcb31dfd88b1afe512c689fb82ca3214afc9db1aafdce0" exitCode=0 Dec 03 13:06:31 crc kubenswrapper[5052]: I1203 13:06:31.668996 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5-debug-pcs86" event={"ID":"381fd12d-de34-40b8-a720-e6400b673a54","Type":"ContainerDied","Data":"2cc90b07e9c5d356befcb31dfd88b1afe512c689fb82ca3214afc9db1aafdce0"} Dec 03 13:06:31 crc kubenswrapper[5052]: I1203 13:06:31.673827 5052 generic.go:334] "Generic (PLEG): container finished" podID="900d3fa2-4088-4bbd-ae8c-b0630237b304" containerID="cef1bc7a06434159cd29dfdbf2e3083b4d8c6668507619974f0b6b08c96991e3" exitCode=0 Dec 03 13:06:31 crc kubenswrapper[5052]: I1203 13:06:31.689731 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dfd64e45-b1eb-43cb-88f4-db4d99d7ec15" path="/var/lib/kubelet/pods/dfd64e45-b1eb-43cb-88f4-db4d99d7ec15/volumes" Dec 03 13:06:31 crc kubenswrapper[5052]: I1203 13:06:31.690292 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/tls-dns-ips-certs-refresh-openstack-edpm-tls-lbttg" event={"ID":"900d3fa2-4088-4bbd-ae8c-b0630237b304","Type":"ContainerDied","Data":"cef1bc7a06434159cd29dfdbf2e3083b4d8c6668507619974f0b6b08c96991e3"} Dec 03 13:06:31 crc kubenswrapper[5052]: I1203 13:06:31.712494 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5-debug-pcs86"] Dec 03 13:06:31 crc kubenswrapper[5052]: I1203 13:06:31.719290 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5-debug-pcs86"] Dec 03 13:06:31 crc kubenswrapper[5052]: I1203 13:06:31.821279 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/install-certs-ovrd-openstack-edpm-tls-ovrd-openstack-edpm-l6wm4"] Dec 03 13:06:31 crc kubenswrapper[5052]: I1203 13:06:31.834625 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/tls-dns-ips-openstack-edpm-tls-ovrd-openstack-edpm-tls-tf7jr"] Dec 03 13:06:31 crc kubenswrapper[5052]: I1203 13:06:31.849877 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/tls-dns-ips-certs-refresh-openstack-edpm-tls-lbttg"] Dec 03 13:06:31 crc kubenswrapper[5052]: I1203 13:06:31.861863 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/custom-tls-dns-openstack-edpm-tls-ovrd-openstack-edpm-tls-96xgg"] Dec 03 13:06:31 crc kubenswrapper[5052]: I1203 13:06:31.874443 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5"] Dec 03 13:06:31 crc kubenswrapper[5052]: I1203 13:06:31.880484 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5"] Dec 03 13:06:31 crc kubenswrapper[5052]: I1203 13:06:31.886767 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/install-certs-ovrd-openstack-edpm-tls-ovrd-openstack-edpm-l6wm4"] Dec 03 13:06:31 crc kubenswrapper[5052]: I1203 13:06:31.893030 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/tls-dns-ips-openstack-edpm-tls-ovrd-openstack-edpm-tls-tf7jr"] Dec 03 13:06:31 crc kubenswrapper[5052]: I1203 13:06:31.899096 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/custom-tls-dns-openstack-edpm-tls-ovrd-openstack-edpm-tls-96xgg"] Dec 03 13:06:31 crc kubenswrapper[5052]: I1203 13:06:31.904798 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/install-certs-ovrd-openstack-edpm-tls-openstack-edpm-tls-ts7gt"] Dec 03 13:06:31 crc kubenswrapper[5052]: I1203 13:06:31.911402 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/tls-dnsnames-openstack-edpm-tls-openstack-edpm-tls-mn9jk"] Dec 03 13:06:31 crc kubenswrapper[5052]: I1203 13:06:31.919871 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/tls-dnsnames-openstack-edpm-tls-openstack-edpm-tls-mn9jk"] Dec 03 13:06:31 crc kubenswrapper[5052]: I1203 13:06:31.928434 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/install-certs-ovrd-openstack-edpm-tls-openstack-edpm-tls-ts7gt"] Dec 03 13:06:31 crc kubenswrapper[5052]: I1203 13:06:31.944358 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-n5w2j"] Dec 03 13:06:31 crc kubenswrapper[5052]: E1203 13:06:31.944796 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="381fd12d-de34-40b8-a720-e6400b673a54" containerName="install-certs-ovrd-certs-refresh-openstack-edpm-tls" Dec 03 13:06:31 crc kubenswrapper[5052]: I1203 13:06:31.944821 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="381fd12d-de34-40b8-a720-e6400b673a54" containerName="install-certs-ovrd-certs-refresh-openstack-edpm-tls" Dec 03 13:06:31 crc kubenswrapper[5052]: I1203 13:06:31.944956 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="381fd12d-de34-40b8-a720-e6400b673a54" containerName="install-certs-ovrd-certs-refresh-openstack-edpm-tls" Dec 03 13:06:31 crc kubenswrapper[5052]: I1203 13:06:31.945741 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-n5w2j" Dec 03 13:06:31 crc kubenswrapper[5052]: I1203 13:06:31.952711 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-n5w2j"] Dec 03 13:06:32 crc kubenswrapper[5052]: I1203 13:06:32.054810 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f7ddd39d-65bc-46a4-8938-20ac4f6a2078-config\") pod \"dnsmasq-dnsmasq-84b9f45d47-n5w2j\" (UID: \"f7ddd39d-65bc-46a4-8938-20ac4f6a2078\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-n5w2j" Dec 03 13:06:32 crc kubenswrapper[5052]: I1203 13:06:32.054945 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gbv8z\" (UniqueName: \"kubernetes.io/projected/f7ddd39d-65bc-46a4-8938-20ac4f6a2078-kube-api-access-gbv8z\") pod \"dnsmasq-dnsmasq-84b9f45d47-n5w2j\" (UID: \"f7ddd39d-65bc-46a4-8938-20ac4f6a2078\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-n5w2j" Dec 03 13:06:32 crc kubenswrapper[5052]: I1203 13:06:32.054986 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/f7ddd39d-65bc-46a4-8938-20ac4f6a2078-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-84b9f45d47-n5w2j\" (UID: \"f7ddd39d-65bc-46a4-8938-20ac4f6a2078\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-n5w2j" Dec 03 13:06:32 crc kubenswrapper[5052]: I1203 13:06:32.156616 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/f7ddd39d-65bc-46a4-8938-20ac4f6a2078-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-84b9f45d47-n5w2j\" (UID: \"f7ddd39d-65bc-46a4-8938-20ac4f6a2078\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-n5w2j" Dec 03 13:06:32 crc kubenswrapper[5052]: I1203 13:06:32.156831 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f7ddd39d-65bc-46a4-8938-20ac4f6a2078-config\") pod \"dnsmasq-dnsmasq-84b9f45d47-n5w2j\" (UID: \"f7ddd39d-65bc-46a4-8938-20ac4f6a2078\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-n5w2j" Dec 03 13:06:32 crc kubenswrapper[5052]: I1203 13:06:32.156958 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-gbv8z\" (UniqueName: \"kubernetes.io/projected/f7ddd39d-65bc-46a4-8938-20ac4f6a2078-kube-api-access-gbv8z\") pod \"dnsmasq-dnsmasq-84b9f45d47-n5w2j\" (UID: \"f7ddd39d-65bc-46a4-8938-20ac4f6a2078\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-n5w2j" Dec 03 13:06:32 crc kubenswrapper[5052]: I1203 13:06:32.157701 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/f7ddd39d-65bc-46a4-8938-20ac4f6a2078-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-84b9f45d47-n5w2j\" (UID: \"f7ddd39d-65bc-46a4-8938-20ac4f6a2078\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-n5w2j" Dec 03 13:06:32 crc kubenswrapper[5052]: I1203 13:06:32.157732 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f7ddd39d-65bc-46a4-8938-20ac4f6a2078-config\") pod \"dnsmasq-dnsmasq-84b9f45d47-n5w2j\" (UID: \"f7ddd39d-65bc-46a4-8938-20ac4f6a2078\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-n5w2j" Dec 03 13:06:32 crc kubenswrapper[5052]: I1203 13:06:32.177780 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-gbv8z\" (UniqueName: \"kubernetes.io/projected/f7ddd39d-65bc-46a4-8938-20ac4f6a2078-kube-api-access-gbv8z\") pod \"dnsmasq-dnsmasq-84b9f45d47-n5w2j\" (UID: \"f7ddd39d-65bc-46a4-8938-20ac4f6a2078\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-n5w2j" Dec 03 13:06:32 crc kubenswrapper[5052]: I1203 13:06:32.266715 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-n5w2j" Dec 03 13:06:32 crc kubenswrapper[5052]: I1203 13:06:32.688202 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-n5w2j"] Dec 03 13:06:32 crc kubenswrapper[5052]: I1203 13:06:32.932425 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/tls-dns-ips-certs-refresh-openstack-edpm-tls-lbttg" Dec 03 13:06:32 crc kubenswrapper[5052]: I1203 13:06:32.967721 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lcwx6\" (UniqueName: \"kubernetes.io/projected/900d3fa2-4088-4bbd-ae8c-b0630237b304-kube-api-access-lcwx6\") pod \"900d3fa2-4088-4bbd-ae8c-b0630237b304\" (UID: \"900d3fa2-4088-4bbd-ae8c-b0630237b304\") " Dec 03 13:06:32 crc kubenswrapper[5052]: I1203 13:06:32.967791 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/900d3fa2-4088-4bbd-ae8c-b0630237b304-ssh-key\") pod \"900d3fa2-4088-4bbd-ae8c-b0630237b304\" (UID: \"900d3fa2-4088-4bbd-ae8c-b0630237b304\") " Dec 03 13:06:32 crc kubenswrapper[5052]: I1203 13:06:32.967868 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"tls-dns-ips-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/900d3fa2-4088-4bbd-ae8c-b0630237b304-tls-dns-ips-combined-ca-bundle\") pod \"900d3fa2-4088-4bbd-ae8c-b0630237b304\" (UID: \"900d3fa2-4088-4bbd-ae8c-b0630237b304\") " Dec 03 13:06:32 crc kubenswrapper[5052]: I1203 13:06:32.967917 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/900d3fa2-4088-4bbd-ae8c-b0630237b304-inventory\") pod \"900d3fa2-4088-4bbd-ae8c-b0630237b304\" (UID: \"900d3fa2-4088-4bbd-ae8c-b0630237b304\") " Dec 03 13:06:32 crc kubenswrapper[5052]: I1203 13:06:32.972409 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/900d3fa2-4088-4bbd-ae8c-b0630237b304-kube-api-access-lcwx6" (OuterVolumeSpecName: "kube-api-access-lcwx6") pod "900d3fa2-4088-4bbd-ae8c-b0630237b304" (UID: "900d3fa2-4088-4bbd-ae8c-b0630237b304"). InnerVolumeSpecName "kube-api-access-lcwx6". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:06:32 crc kubenswrapper[5052]: I1203 13:06:32.972683 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/900d3fa2-4088-4bbd-ae8c-b0630237b304-tls-dns-ips-combined-ca-bundle" (OuterVolumeSpecName: "tls-dns-ips-combined-ca-bundle") pod "900d3fa2-4088-4bbd-ae8c-b0630237b304" (UID: "900d3fa2-4088-4bbd-ae8c-b0630237b304"). InnerVolumeSpecName "tls-dns-ips-combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:06:32 crc kubenswrapper[5052]: I1203 13:06:32.994025 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/900d3fa2-4088-4bbd-ae8c-b0630237b304-ssh-key" (OuterVolumeSpecName: "ssh-key") pod "900d3fa2-4088-4bbd-ae8c-b0630237b304" (UID: "900d3fa2-4088-4bbd-ae8c-b0630237b304"). InnerVolumeSpecName "ssh-key". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:06:32 crc kubenswrapper[5052]: I1203 13:06:32.995155 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/900d3fa2-4088-4bbd-ae8c-b0630237b304-inventory" (OuterVolumeSpecName: "inventory") pod "900d3fa2-4088-4bbd-ae8c-b0630237b304" (UID: "900d3fa2-4088-4bbd-ae8c-b0630237b304"). InnerVolumeSpecName "inventory". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:06:33 crc kubenswrapper[5052]: I1203 13:06:33.031689 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5-debug-pcs86" Dec 03 13:06:33 crc kubenswrapper[5052]: I1203 13:06:33.068817 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"tls-dns-ips-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/381fd12d-de34-40b8-a720-e6400b673a54-tls-dns-ips-combined-ca-bundle\") pod \"381fd12d-de34-40b8-a720-e6400b673a54\" (UID: \"381fd12d-de34-40b8-a720-e6400b673a54\") " Dec 03 13:06:33 crc kubenswrapper[5052]: I1203 13:06:33.068899 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"openstack-edpm-tls-tls-dns-ips-default-certs-0\" (UniqueName: \"kubernetes.io/projected/381fd12d-de34-40b8-a720-e6400b673a54-openstack-edpm-tls-tls-dns-ips-default-certs-0\") pod \"381fd12d-de34-40b8-a720-e6400b673a54\" (UID: \"381fd12d-de34-40b8-a720-e6400b673a54\") " Dec 03 13:06:33 crc kubenswrapper[5052]: I1203 13:06:33.068952 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"openstack-edpm-tls-custom-tls-dns-default-certs-0\" (UniqueName: \"kubernetes.io/projected/381fd12d-de34-40b8-a720-e6400b673a54-openstack-edpm-tls-custom-tls-dns-default-certs-0\") pod \"381fd12d-de34-40b8-a720-e6400b673a54\" (UID: \"381fd12d-de34-40b8-a720-e6400b673a54\") " Dec 03 13:06:33 crc kubenswrapper[5052]: I1203 13:06:33.069018 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rww9c\" (UniqueName: \"kubernetes.io/projected/381fd12d-de34-40b8-a720-e6400b673a54-kube-api-access-rww9c\") pod \"381fd12d-de34-40b8-a720-e6400b673a54\" (UID: \"381fd12d-de34-40b8-a720-e6400b673a54\") " Dec 03 13:06:33 crc kubenswrapper[5052]: I1203 13:06:33.069074 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/381fd12d-de34-40b8-a720-e6400b673a54-inventory\") pod \"381fd12d-de34-40b8-a720-e6400b673a54\" (UID: \"381fd12d-de34-40b8-a720-e6400b673a54\") " Dec 03 13:06:33 crc kubenswrapper[5052]: I1203 13:06:33.069100 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/381fd12d-de34-40b8-a720-e6400b673a54-ssh-key\") pod \"381fd12d-de34-40b8-a720-e6400b673a54\" (UID: \"381fd12d-de34-40b8-a720-e6400b673a54\") " Dec 03 13:06:33 crc kubenswrapper[5052]: I1203 13:06:33.069126 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"custom-tls-dns-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/381fd12d-de34-40b8-a720-e6400b673a54-custom-tls-dns-combined-ca-bundle\") pod \"381fd12d-de34-40b8-a720-e6400b673a54\" (UID: \"381fd12d-de34-40b8-a720-e6400b673a54\") " Dec 03 13:06:33 crc kubenswrapper[5052]: I1203 13:06:33.069148 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"install-certs-ovrd-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/381fd12d-de34-40b8-a720-e6400b673a54-install-certs-ovrd-combined-ca-bundle\") pod \"381fd12d-de34-40b8-a720-e6400b673a54\" (UID: \"381fd12d-de34-40b8-a720-e6400b673a54\") " Dec 03 13:06:33 crc kubenswrapper[5052]: I1203 13:06:33.069569 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-lcwx6\" (UniqueName: \"kubernetes.io/projected/900d3fa2-4088-4bbd-ae8c-b0630237b304-kube-api-access-lcwx6\") on node \"crc\" DevicePath \"\"" Dec 03 13:06:33 crc kubenswrapper[5052]: I1203 13:06:33.069587 5052 reconciler_common.go:293] "Volume detached for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/900d3fa2-4088-4bbd-ae8c-b0630237b304-ssh-key\") on node \"crc\" DevicePath \"\"" Dec 03 13:06:33 crc kubenswrapper[5052]: I1203 13:06:33.069598 5052 reconciler_common.go:293] "Volume detached for volume \"tls-dns-ips-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/900d3fa2-4088-4bbd-ae8c-b0630237b304-tls-dns-ips-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 13:06:33 crc kubenswrapper[5052]: I1203 13:06:33.069609 5052 reconciler_common.go:293] "Volume detached for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/900d3fa2-4088-4bbd-ae8c-b0630237b304-inventory\") on node \"crc\" DevicePath \"\"" Dec 03 13:06:33 crc kubenswrapper[5052]: I1203 13:06:33.072193 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/381fd12d-de34-40b8-a720-e6400b673a54-openstack-edpm-tls-tls-dns-ips-default-certs-0" (OuterVolumeSpecName: "openstack-edpm-tls-tls-dns-ips-default-certs-0") pod "381fd12d-de34-40b8-a720-e6400b673a54" (UID: "381fd12d-de34-40b8-a720-e6400b673a54"). InnerVolumeSpecName "openstack-edpm-tls-tls-dns-ips-default-certs-0". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:06:33 crc kubenswrapper[5052]: I1203 13:06:33.072302 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/381fd12d-de34-40b8-a720-e6400b673a54-tls-dns-ips-combined-ca-bundle" (OuterVolumeSpecName: "tls-dns-ips-combined-ca-bundle") pod "381fd12d-de34-40b8-a720-e6400b673a54" (UID: "381fd12d-de34-40b8-a720-e6400b673a54"). InnerVolumeSpecName "tls-dns-ips-combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:06:33 crc kubenswrapper[5052]: I1203 13:06:33.072772 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/381fd12d-de34-40b8-a720-e6400b673a54-kube-api-access-rww9c" (OuterVolumeSpecName: "kube-api-access-rww9c") pod "381fd12d-de34-40b8-a720-e6400b673a54" (UID: "381fd12d-de34-40b8-a720-e6400b673a54"). InnerVolumeSpecName "kube-api-access-rww9c". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:06:33 crc kubenswrapper[5052]: I1203 13:06:33.072794 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/381fd12d-de34-40b8-a720-e6400b673a54-openstack-edpm-tls-custom-tls-dns-default-certs-0" (OuterVolumeSpecName: "openstack-edpm-tls-custom-tls-dns-default-certs-0") pod "381fd12d-de34-40b8-a720-e6400b673a54" (UID: "381fd12d-de34-40b8-a720-e6400b673a54"). InnerVolumeSpecName "openstack-edpm-tls-custom-tls-dns-default-certs-0". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:06:33 crc kubenswrapper[5052]: I1203 13:06:33.073571 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/381fd12d-de34-40b8-a720-e6400b673a54-install-certs-ovrd-combined-ca-bundle" (OuterVolumeSpecName: "install-certs-ovrd-combined-ca-bundle") pod "381fd12d-de34-40b8-a720-e6400b673a54" (UID: "381fd12d-de34-40b8-a720-e6400b673a54"). InnerVolumeSpecName "install-certs-ovrd-combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:06:33 crc kubenswrapper[5052]: I1203 13:06:33.073864 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/381fd12d-de34-40b8-a720-e6400b673a54-custom-tls-dns-combined-ca-bundle" (OuterVolumeSpecName: "custom-tls-dns-combined-ca-bundle") pod "381fd12d-de34-40b8-a720-e6400b673a54" (UID: "381fd12d-de34-40b8-a720-e6400b673a54"). InnerVolumeSpecName "custom-tls-dns-combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:06:33 crc kubenswrapper[5052]: I1203 13:06:33.089960 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/381fd12d-de34-40b8-a720-e6400b673a54-inventory" (OuterVolumeSpecName: "inventory") pod "381fd12d-de34-40b8-a720-e6400b673a54" (UID: "381fd12d-de34-40b8-a720-e6400b673a54"). InnerVolumeSpecName "inventory". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:06:33 crc kubenswrapper[5052]: I1203 13:06:33.090729 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/381fd12d-de34-40b8-a720-e6400b673a54-ssh-key" (OuterVolumeSpecName: "ssh-key") pod "381fd12d-de34-40b8-a720-e6400b673a54" (UID: "381fd12d-de34-40b8-a720-e6400b673a54"). InnerVolumeSpecName "ssh-key". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:06:33 crc kubenswrapper[5052]: I1203 13:06:33.171006 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-rww9c\" (UniqueName: \"kubernetes.io/projected/381fd12d-de34-40b8-a720-e6400b673a54-kube-api-access-rww9c\") on node \"crc\" DevicePath \"\"" Dec 03 13:06:33 crc kubenswrapper[5052]: I1203 13:06:33.171056 5052 reconciler_common.go:293] "Volume detached for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/381fd12d-de34-40b8-a720-e6400b673a54-inventory\") on node \"crc\" DevicePath \"\"" Dec 03 13:06:33 crc kubenswrapper[5052]: I1203 13:06:33.171067 5052 reconciler_common.go:293] "Volume detached for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/381fd12d-de34-40b8-a720-e6400b673a54-ssh-key\") on node \"crc\" DevicePath \"\"" Dec 03 13:06:33 crc kubenswrapper[5052]: I1203 13:06:33.171076 5052 reconciler_common.go:293] "Volume detached for volume \"custom-tls-dns-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/381fd12d-de34-40b8-a720-e6400b673a54-custom-tls-dns-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 13:06:33 crc kubenswrapper[5052]: I1203 13:06:33.171088 5052 reconciler_common.go:293] "Volume detached for volume \"install-certs-ovrd-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/381fd12d-de34-40b8-a720-e6400b673a54-install-certs-ovrd-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 13:06:33 crc kubenswrapper[5052]: I1203 13:06:33.171098 5052 reconciler_common.go:293] "Volume detached for volume \"tls-dns-ips-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/381fd12d-de34-40b8-a720-e6400b673a54-tls-dns-ips-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 13:06:33 crc kubenswrapper[5052]: I1203 13:06:33.171107 5052 reconciler_common.go:293] "Volume detached for volume \"openstack-edpm-tls-tls-dns-ips-default-certs-0\" (UniqueName: \"kubernetes.io/projected/381fd12d-de34-40b8-a720-e6400b673a54-openstack-edpm-tls-tls-dns-ips-default-certs-0\") on node \"crc\" DevicePath \"\"" Dec 03 13:06:33 crc kubenswrapper[5052]: I1203 13:06:33.171120 5052 reconciler_common.go:293] "Volume detached for volume \"openstack-edpm-tls-custom-tls-dns-default-certs-0\" (UniqueName: \"kubernetes.io/projected/381fd12d-de34-40b8-a720-e6400b673a54-openstack-edpm-tls-custom-tls-dns-default-certs-0\") on node \"crc\" DevicePath \"\"" Dec 03 13:06:33 crc kubenswrapper[5052]: I1203 13:06:33.681870 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0a23beab-7ac9-4645-8c48-13b9d02e9659" path="/var/lib/kubelet/pods/0a23beab-7ac9-4645-8c48-13b9d02e9659/volumes" Dec 03 13:06:33 crc kubenswrapper[5052]: I1203 13:06:33.682632 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2ec739d1-4d9b-4327-aa7d-d31a55fddb55" path="/var/lib/kubelet/pods/2ec739d1-4d9b-4327-aa7d-d31a55fddb55/volumes" Dec 03 13:06:33 crc kubenswrapper[5052]: I1203 13:06:33.683244 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="30e59ea6-cd2f-4e47-b4dc-7ff82bfce2b7" path="/var/lib/kubelet/pods/30e59ea6-cd2f-4e47-b4dc-7ff82bfce2b7/volumes" Dec 03 13:06:33 crc kubenswrapper[5052]: I1203 13:06:33.683863 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="381fd12d-de34-40b8-a720-e6400b673a54" path="/var/lib/kubelet/pods/381fd12d-de34-40b8-a720-e6400b673a54/volumes" Dec 03 13:06:33 crc kubenswrapper[5052]: I1203 13:06:33.684946 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3d2991cb-ab4a-482e-913e-1d4110032258" path="/var/lib/kubelet/pods/3d2991cb-ab4a-482e-913e-1d4110032258/volumes" Dec 03 13:06:33 crc kubenswrapper[5052]: I1203 13:06:33.685560 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="89695efc-592a-44ac-9aeb-ddff36af1791" path="/var/lib/kubelet/pods/89695efc-592a-44ac-9aeb-ddff36af1791/volumes" Dec 03 13:06:33 crc kubenswrapper[5052]: I1203 13:06:33.686112 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="be554003-60cb-4ea9-9cce-d5a0b6f3dbaa" path="/var/lib/kubelet/pods/be554003-60cb-4ea9-9cce-d5a0b6f3dbaa/volumes" Dec 03 13:06:33 crc kubenswrapper[5052]: I1203 13:06:33.706916 5052 generic.go:334] "Generic (PLEG): container finished" podID="f7ddd39d-65bc-46a4-8938-20ac4f6a2078" containerID="a39a3886f97f5afd814ec6a1c24e3f7c85e19c621d9c90559f3a49074ecb8907" exitCode=0 Dec 03 13:06:33 crc kubenswrapper[5052]: I1203 13:06:33.706996 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-n5w2j" event={"ID":"f7ddd39d-65bc-46a4-8938-20ac4f6a2078","Type":"ContainerDied","Data":"a39a3886f97f5afd814ec6a1c24e3f7c85e19c621d9c90559f3a49074ecb8907"} Dec 03 13:06:33 crc kubenswrapper[5052]: I1203 13:06:33.707028 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-n5w2j" event={"ID":"f7ddd39d-65bc-46a4-8938-20ac4f6a2078","Type":"ContainerStarted","Data":"46b03680de82643b00804a1b184e64f7f680e2d1f3a0287d13bd07cfadaec639"} Dec 03 13:06:33 crc kubenswrapper[5052]: I1203 13:06:33.709451 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/install-certs-ovrd-certs-refresh-openstack-edpm-tls-bxrg5-debug-pcs86" Dec 03 13:06:33 crc kubenswrapper[5052]: I1203 13:06:33.709497 5052 scope.go:117] "RemoveContainer" containerID="2cc90b07e9c5d356befcb31dfd88b1afe512c689fb82ca3214afc9db1aafdce0" Dec 03 13:06:33 crc kubenswrapper[5052]: I1203 13:06:33.711460 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/tls-dns-ips-certs-refresh-openstack-edpm-tls-lbttg" event={"ID":"900d3fa2-4088-4bbd-ae8c-b0630237b304","Type":"ContainerDied","Data":"0246c016f7866eea97ceebb7b19ca5cf8b7d9deda4cb0a0a79620ced7ed15993"} Dec 03 13:06:33 crc kubenswrapper[5052]: I1203 13:06:33.711487 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0246c016f7866eea97ceebb7b19ca5cf8b7d9deda4cb0a0a79620ced7ed15993" Dec 03 13:06:33 crc kubenswrapper[5052]: I1203 13:06:33.711540 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/tls-dns-ips-certs-refresh-openstack-edpm-tls-lbttg" Dec 03 13:06:33 crc kubenswrapper[5052]: I1203 13:06:33.887766 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/tls-dns-ips-certs-refresh-openstack-edpm-tls-lbttg"] Dec 03 13:06:33 crc kubenswrapper[5052]: I1203 13:06:33.893936 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/tls-dns-ips-certs-refresh-openstack-edpm-tls-lbttg"] Dec 03 13:06:34 crc kubenswrapper[5052]: I1203 13:06:34.723145 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-n5w2j" event={"ID":"f7ddd39d-65bc-46a4-8938-20ac4f6a2078","Type":"ContainerStarted","Data":"0b3afaae2cbac154f4abc313f1ea0fcf8c553fbd1b0e6b72c914c2cba00c89e8"} Dec 03 13:06:34 crc kubenswrapper[5052]: I1203 13:06:34.723290 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-n5w2j" Dec 03 13:06:34 crc kubenswrapper[5052]: I1203 13:06:34.741640 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-n5w2j" podStartSLOduration=3.741620684 podStartE2EDuration="3.741620684s" podCreationTimestamp="2025-12-03 13:06:31 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 13:06:34.736358863 +0000 UTC m=+6385.324600637" watchObservedRunningTime="2025-12-03 13:06:34.741620684 +0000 UTC m=+6385.329862458" Dec 03 13:06:35 crc kubenswrapper[5052]: I1203 13:06:35.683265 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="900d3fa2-4088-4bbd-ae8c-b0630237b304" path="/var/lib/kubelet/pods/900d3fa2-4088-4bbd-ae8c-b0630237b304/volumes" Dec 03 13:06:40 crc kubenswrapper[5052]: I1203 13:06:40.149068 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["crc-storage/crc-storage-crc-skgwc"] Dec 03 13:06:40 crc kubenswrapper[5052]: I1203 13:06:40.153987 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["crc-storage/crc-storage-crc-skgwc"] Dec 03 13:06:40 crc kubenswrapper[5052]: I1203 13:06:40.303639 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["crc-storage/crc-storage-crc-9nprf"] Dec 03 13:06:40 crc kubenswrapper[5052]: E1203 13:06:40.304312 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="900d3fa2-4088-4bbd-ae8c-b0630237b304" containerName="tls-dns-ips-certs-refresh-openstack-edpm-tls" Dec 03 13:06:40 crc kubenswrapper[5052]: I1203 13:06:40.304339 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="900d3fa2-4088-4bbd-ae8c-b0630237b304" containerName="tls-dns-ips-certs-refresh-openstack-edpm-tls" Dec 03 13:06:40 crc kubenswrapper[5052]: I1203 13:06:40.304666 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="900d3fa2-4088-4bbd-ae8c-b0630237b304" containerName="tls-dns-ips-certs-refresh-openstack-edpm-tls" Dec 03 13:06:40 crc kubenswrapper[5052]: I1203 13:06:40.306247 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="crc-storage/crc-storage-crc-9nprf" Dec 03 13:06:40 crc kubenswrapper[5052]: I1203 13:06:40.308395 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"crc-storage"/"crc-storage" Dec 03 13:06:40 crc kubenswrapper[5052]: I1203 13:06:40.308769 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"crc-storage"/"openshift-service-ca.crt" Dec 03 13:06:40 crc kubenswrapper[5052]: I1203 13:06:40.308833 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"crc-storage"/"kube-root-ca.crt" Dec 03 13:06:40 crc kubenswrapper[5052]: I1203 13:06:40.309567 5052 reflector.go:368] Caches populated for *v1.Secret from object-"crc-storage"/"crc-storage-dockercfg-mf76g" Dec 03 13:06:40 crc kubenswrapper[5052]: I1203 13:06:40.309970 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["crc-storage/crc-storage-crc-9nprf"] Dec 03 13:06:40 crc kubenswrapper[5052]: I1203 13:06:40.382006 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-krsl2\" (UniqueName: \"kubernetes.io/projected/75b0171a-f626-444b-888f-80131592dbfe-kube-api-access-krsl2\") pod \"crc-storage-crc-9nprf\" (UID: \"75b0171a-f626-444b-888f-80131592dbfe\") " pod="crc-storage/crc-storage-crc-9nprf" Dec 03 13:06:40 crc kubenswrapper[5052]: I1203 13:06:40.382381 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/75b0171a-f626-444b-888f-80131592dbfe-crc-storage\") pod \"crc-storage-crc-9nprf\" (UID: \"75b0171a-f626-444b-888f-80131592dbfe\") " pod="crc-storage/crc-storage-crc-9nprf" Dec 03 13:06:40 crc kubenswrapper[5052]: I1203 13:06:40.382457 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/75b0171a-f626-444b-888f-80131592dbfe-node-mnt\") pod \"crc-storage-crc-9nprf\" (UID: \"75b0171a-f626-444b-888f-80131592dbfe\") " pod="crc-storage/crc-storage-crc-9nprf" Dec 03 13:06:40 crc kubenswrapper[5052]: I1203 13:06:40.484711 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/75b0171a-f626-444b-888f-80131592dbfe-crc-storage\") pod \"crc-storage-crc-9nprf\" (UID: \"75b0171a-f626-444b-888f-80131592dbfe\") " pod="crc-storage/crc-storage-crc-9nprf" Dec 03 13:06:40 crc kubenswrapper[5052]: I1203 13:06:40.484782 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/75b0171a-f626-444b-888f-80131592dbfe-node-mnt\") pod \"crc-storage-crc-9nprf\" (UID: \"75b0171a-f626-444b-888f-80131592dbfe\") " pod="crc-storage/crc-storage-crc-9nprf" Dec 03 13:06:40 crc kubenswrapper[5052]: I1203 13:06:40.484826 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-krsl2\" (UniqueName: \"kubernetes.io/projected/75b0171a-f626-444b-888f-80131592dbfe-kube-api-access-krsl2\") pod \"crc-storage-crc-9nprf\" (UID: \"75b0171a-f626-444b-888f-80131592dbfe\") " pod="crc-storage/crc-storage-crc-9nprf" Dec 03 13:06:40 crc kubenswrapper[5052]: I1203 13:06:40.485169 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/75b0171a-f626-444b-888f-80131592dbfe-node-mnt\") pod \"crc-storage-crc-9nprf\" (UID: \"75b0171a-f626-444b-888f-80131592dbfe\") " pod="crc-storage/crc-storage-crc-9nprf" Dec 03 13:06:40 crc kubenswrapper[5052]: I1203 13:06:40.485804 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/75b0171a-f626-444b-888f-80131592dbfe-crc-storage\") pod \"crc-storage-crc-9nprf\" (UID: \"75b0171a-f626-444b-888f-80131592dbfe\") " pod="crc-storage/crc-storage-crc-9nprf" Dec 03 13:06:40 crc kubenswrapper[5052]: I1203 13:06:40.511228 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-krsl2\" (UniqueName: \"kubernetes.io/projected/75b0171a-f626-444b-888f-80131592dbfe-kube-api-access-krsl2\") pod \"crc-storage-crc-9nprf\" (UID: \"75b0171a-f626-444b-888f-80131592dbfe\") " pod="crc-storage/crc-storage-crc-9nprf" Dec 03 13:06:40 crc kubenswrapper[5052]: I1203 13:06:40.624668 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="crc-storage/crc-storage-crc-9nprf" Dec 03 13:06:41 crc kubenswrapper[5052]: I1203 13:06:41.058643 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["crc-storage/crc-storage-crc-9nprf"] Dec 03 13:06:41 crc kubenswrapper[5052]: I1203 13:06:41.681967 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="39a41b2f-5948-4e46-88d5-cf7a6364a8e8" path="/var/lib/kubelet/pods/39a41b2f-5948-4e46-88d5-cf7a6364a8e8/volumes" Dec 03 13:06:41 crc kubenswrapper[5052]: I1203 13:06:41.782799 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="crc-storage/crc-storage-crc-9nprf" event={"ID":"75b0171a-f626-444b-888f-80131592dbfe","Type":"ContainerStarted","Data":"eb12bb55c47ef6befbd0d1361b7469ff407633f09880d5a41daddcc0abb743a1"} Dec 03 13:06:42 crc kubenswrapper[5052]: I1203 13:06:42.148018 5052 scope.go:117] "RemoveContainer" containerID="03433a1bf72d3b1f122b0c525ee8d41ba085889d0887543f5c99e28ea7edcd7e" Dec 03 13:06:42 crc kubenswrapper[5052]: I1203 13:06:42.170361 5052 scope.go:117] "RemoveContainer" containerID="f7c4412f149d80155c8075f51b8dc50b33bf84ecc7229551f39e61369acbf604" Dec 03 13:06:42 crc kubenswrapper[5052]: I1203 13:06:42.207078 5052 scope.go:117] "RemoveContainer" containerID="440638980aef4abb6f03a0c5400993e44ef78d9662a44adbdc33cec51bb52dd4" Dec 03 13:06:42 crc kubenswrapper[5052]: I1203 13:06:42.246229 5052 scope.go:117] "RemoveContainer" containerID="5a57d728d1afb3e7fc17f47c0c7e311b0f2422272ae2681700e97b1ccd03fa1d" Dec 03 13:06:42 crc kubenswrapper[5052]: I1203 13:06:42.269049 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-n5w2j" Dec 03 13:06:42 crc kubenswrapper[5052]: I1203 13:06:42.324123 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-79cc674687-46mrq"] Dec 03 13:06:42 crc kubenswrapper[5052]: I1203 13:06:42.324349 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-79cc674687-46mrq" podUID="ff4169c4-e917-42c4-9ca1-b4f9d62be591" containerName="dnsmasq-dns" containerID="cri-o://b32b57428120953190e507dec8a34a893da0817c1268d08aabd67cd56b506473" gracePeriod=10 Dec 03 13:06:42 crc kubenswrapper[5052]: I1203 13:06:42.666391 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-79cc674687-46mrq" Dec 03 13:06:42 crc kubenswrapper[5052]: I1203 13:06:42.739889 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d6mbs\" (UniqueName: \"kubernetes.io/projected/ff4169c4-e917-42c4-9ca1-b4f9d62be591-kube-api-access-d6mbs\") pod \"ff4169c4-e917-42c4-9ca1-b4f9d62be591\" (UID: \"ff4169c4-e917-42c4-9ca1-b4f9d62be591\") " Dec 03 13:06:42 crc kubenswrapper[5052]: I1203 13:06:42.740035 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/ff4169c4-e917-42c4-9ca1-b4f9d62be591-dnsmasq-svc\") pod \"ff4169c4-e917-42c4-9ca1-b4f9d62be591\" (UID: \"ff4169c4-e917-42c4-9ca1-b4f9d62be591\") " Dec 03 13:06:42 crc kubenswrapper[5052]: I1203 13:06:42.740126 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ff4169c4-e917-42c4-9ca1-b4f9d62be591-config\") pod \"ff4169c4-e917-42c4-9ca1-b4f9d62be591\" (UID: \"ff4169c4-e917-42c4-9ca1-b4f9d62be591\") " Dec 03 13:06:42 crc kubenswrapper[5052]: I1203 13:06:42.740183 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"openstack-edpm-tls\" (UniqueName: \"kubernetes.io/configmap/ff4169c4-e917-42c4-9ca1-b4f9d62be591-openstack-edpm-tls\") pod \"ff4169c4-e917-42c4-9ca1-b4f9d62be591\" (UID: \"ff4169c4-e917-42c4-9ca1-b4f9d62be591\") " Dec 03 13:06:42 crc kubenswrapper[5052]: I1203 13:06:42.745586 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ff4169c4-e917-42c4-9ca1-b4f9d62be591-kube-api-access-d6mbs" (OuterVolumeSpecName: "kube-api-access-d6mbs") pod "ff4169c4-e917-42c4-9ca1-b4f9d62be591" (UID: "ff4169c4-e917-42c4-9ca1-b4f9d62be591"). InnerVolumeSpecName "kube-api-access-d6mbs". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:06:42 crc kubenswrapper[5052]: I1203 13:06:42.780862 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ff4169c4-e917-42c4-9ca1-b4f9d62be591-dnsmasq-svc" (OuterVolumeSpecName: "dnsmasq-svc") pod "ff4169c4-e917-42c4-9ca1-b4f9d62be591" (UID: "ff4169c4-e917-42c4-9ca1-b4f9d62be591"). InnerVolumeSpecName "dnsmasq-svc". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:06:42 crc kubenswrapper[5052]: I1203 13:06:42.781868 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ff4169c4-e917-42c4-9ca1-b4f9d62be591-openstack-edpm-tls" (OuterVolumeSpecName: "openstack-edpm-tls") pod "ff4169c4-e917-42c4-9ca1-b4f9d62be591" (UID: "ff4169c4-e917-42c4-9ca1-b4f9d62be591"). InnerVolumeSpecName "openstack-edpm-tls". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:06:42 crc kubenswrapper[5052]: I1203 13:06:42.790907 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ff4169c4-e917-42c4-9ca1-b4f9d62be591-config" (OuterVolumeSpecName: "config") pod "ff4169c4-e917-42c4-9ca1-b4f9d62be591" (UID: "ff4169c4-e917-42c4-9ca1-b4f9d62be591"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:06:42 crc kubenswrapper[5052]: I1203 13:06:42.795256 5052 generic.go:334] "Generic (PLEG): container finished" podID="ff4169c4-e917-42c4-9ca1-b4f9d62be591" containerID="b32b57428120953190e507dec8a34a893da0817c1268d08aabd67cd56b506473" exitCode=0 Dec 03 13:06:42 crc kubenswrapper[5052]: I1203 13:06:42.795339 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-79cc674687-46mrq" Dec 03 13:06:42 crc kubenswrapper[5052]: I1203 13:06:42.795360 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-79cc674687-46mrq" event={"ID":"ff4169c4-e917-42c4-9ca1-b4f9d62be591","Type":"ContainerDied","Data":"b32b57428120953190e507dec8a34a893da0817c1268d08aabd67cd56b506473"} Dec 03 13:06:42 crc kubenswrapper[5052]: I1203 13:06:42.795384 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-79cc674687-46mrq" event={"ID":"ff4169c4-e917-42c4-9ca1-b4f9d62be591","Type":"ContainerDied","Data":"b1531a85bc812722feb1cb93d8bdb578e7b87369252aee63dc97076efc16936b"} Dec 03 13:06:42 crc kubenswrapper[5052]: I1203 13:06:42.795402 5052 scope.go:117] "RemoveContainer" containerID="b32b57428120953190e507dec8a34a893da0817c1268d08aabd67cd56b506473" Dec 03 13:06:42 crc kubenswrapper[5052]: I1203 13:06:42.799779 5052 generic.go:334] "Generic (PLEG): container finished" podID="75b0171a-f626-444b-888f-80131592dbfe" containerID="a19133637f8911d3fba7f2b47eb8e794f144864c729a5d01025e2b35f2d03fc8" exitCode=0 Dec 03 13:06:42 crc kubenswrapper[5052]: I1203 13:06:42.799829 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="crc-storage/crc-storage-crc-9nprf" event={"ID":"75b0171a-f626-444b-888f-80131592dbfe","Type":"ContainerDied","Data":"a19133637f8911d3fba7f2b47eb8e794f144864c729a5d01025e2b35f2d03fc8"} Dec 03 13:06:42 crc kubenswrapper[5052]: I1203 13:06:42.842293 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-d6mbs\" (UniqueName: \"kubernetes.io/projected/ff4169c4-e917-42c4-9ca1-b4f9d62be591-kube-api-access-d6mbs\") on node \"crc\" DevicePath \"\"" Dec 03 13:06:42 crc kubenswrapper[5052]: I1203 13:06:42.842658 5052 reconciler_common.go:293] "Volume detached for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/ff4169c4-e917-42c4-9ca1-b4f9d62be591-dnsmasq-svc\") on node \"crc\" DevicePath \"\"" Dec 03 13:06:42 crc kubenswrapper[5052]: I1203 13:06:42.842784 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ff4169c4-e917-42c4-9ca1-b4f9d62be591-config\") on node \"crc\" DevicePath \"\"" Dec 03 13:06:42 crc kubenswrapper[5052]: I1203 13:06:42.842801 5052 reconciler_common.go:293] "Volume detached for volume \"openstack-edpm-tls\" (UniqueName: \"kubernetes.io/configmap/ff4169c4-e917-42c4-9ca1-b4f9d62be591-openstack-edpm-tls\") on node \"crc\" DevicePath \"\"" Dec 03 13:06:42 crc kubenswrapper[5052]: I1203 13:06:42.844813 5052 scope.go:117] "RemoveContainer" containerID="f104d4304f9bdd12c76ee231cc308e06758abe0908de3165985bc5bac2b531d5" Dec 03 13:06:42 crc kubenswrapper[5052]: I1203 13:06:42.856779 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-79cc674687-46mrq"] Dec 03 13:06:42 crc kubenswrapper[5052]: I1203 13:06:42.862969 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-79cc674687-46mrq"] Dec 03 13:06:42 crc kubenswrapper[5052]: I1203 13:06:42.882923 5052 scope.go:117] "RemoveContainer" containerID="b32b57428120953190e507dec8a34a893da0817c1268d08aabd67cd56b506473" Dec 03 13:06:42 crc kubenswrapper[5052]: E1203 13:06:42.883392 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b32b57428120953190e507dec8a34a893da0817c1268d08aabd67cd56b506473\": container with ID starting with b32b57428120953190e507dec8a34a893da0817c1268d08aabd67cd56b506473 not found: ID does not exist" containerID="b32b57428120953190e507dec8a34a893da0817c1268d08aabd67cd56b506473" Dec 03 13:06:42 crc kubenswrapper[5052]: I1203 13:06:42.883426 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b32b57428120953190e507dec8a34a893da0817c1268d08aabd67cd56b506473"} err="failed to get container status \"b32b57428120953190e507dec8a34a893da0817c1268d08aabd67cd56b506473\": rpc error: code = NotFound desc = could not find container \"b32b57428120953190e507dec8a34a893da0817c1268d08aabd67cd56b506473\": container with ID starting with b32b57428120953190e507dec8a34a893da0817c1268d08aabd67cd56b506473 not found: ID does not exist" Dec 03 13:06:42 crc kubenswrapper[5052]: I1203 13:06:42.883448 5052 scope.go:117] "RemoveContainer" containerID="f104d4304f9bdd12c76ee231cc308e06758abe0908de3165985bc5bac2b531d5" Dec 03 13:06:42 crc kubenswrapper[5052]: E1203 13:06:42.883944 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f104d4304f9bdd12c76ee231cc308e06758abe0908de3165985bc5bac2b531d5\": container with ID starting with f104d4304f9bdd12c76ee231cc308e06758abe0908de3165985bc5bac2b531d5 not found: ID does not exist" containerID="f104d4304f9bdd12c76ee231cc308e06758abe0908de3165985bc5bac2b531d5" Dec 03 13:06:42 crc kubenswrapper[5052]: I1203 13:06:42.884005 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f104d4304f9bdd12c76ee231cc308e06758abe0908de3165985bc5bac2b531d5"} err="failed to get container status \"f104d4304f9bdd12c76ee231cc308e06758abe0908de3165985bc5bac2b531d5\": rpc error: code = NotFound desc = could not find container \"f104d4304f9bdd12c76ee231cc308e06758abe0908de3165985bc5bac2b531d5\": container with ID starting with f104d4304f9bdd12c76ee231cc308e06758abe0908de3165985bc5bac2b531d5 not found: ID does not exist" Dec 03 13:06:42 crc kubenswrapper[5052]: E1203 13:06:42.960036 5052 cadvisor_stats_provider.go:516] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podff4169c4_e917_42c4_9ca1_b4f9d62be591.slice\": RecentStats: unable to find data in memory cache]" Dec 03 13:06:43 crc kubenswrapper[5052]: I1203 13:06:43.685668 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ff4169c4-e917-42c4-9ca1-b4f9d62be591" path="/var/lib/kubelet/pods/ff4169c4-e917-42c4-9ca1-b4f9d62be591/volumes" Dec 03 13:06:44 crc kubenswrapper[5052]: I1203 13:06:44.090352 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="crc-storage/crc-storage-crc-9nprf" Dec 03 13:06:44 crc kubenswrapper[5052]: I1203 13:06:44.164888 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/75b0171a-f626-444b-888f-80131592dbfe-node-mnt\") pod \"75b0171a-f626-444b-888f-80131592dbfe\" (UID: \"75b0171a-f626-444b-888f-80131592dbfe\") " Dec 03 13:06:44 crc kubenswrapper[5052]: I1203 13:06:44.164974 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/75b0171a-f626-444b-888f-80131592dbfe-crc-storage\") pod \"75b0171a-f626-444b-888f-80131592dbfe\" (UID: \"75b0171a-f626-444b-888f-80131592dbfe\") " Dec 03 13:06:44 crc kubenswrapper[5052]: I1203 13:06:44.165044 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/75b0171a-f626-444b-888f-80131592dbfe-node-mnt" (OuterVolumeSpecName: "node-mnt") pod "75b0171a-f626-444b-888f-80131592dbfe" (UID: "75b0171a-f626-444b-888f-80131592dbfe"). InnerVolumeSpecName "node-mnt". PluginName "kubernetes.io/host-path", VolumeGidValue "" Dec 03 13:06:44 crc kubenswrapper[5052]: I1203 13:06:44.165087 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-krsl2\" (UniqueName: \"kubernetes.io/projected/75b0171a-f626-444b-888f-80131592dbfe-kube-api-access-krsl2\") pod \"75b0171a-f626-444b-888f-80131592dbfe\" (UID: \"75b0171a-f626-444b-888f-80131592dbfe\") " Dec 03 13:06:44 crc kubenswrapper[5052]: I1203 13:06:44.165484 5052 reconciler_common.go:293] "Volume detached for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/75b0171a-f626-444b-888f-80131592dbfe-node-mnt\") on node \"crc\" DevicePath \"\"" Dec 03 13:06:44 crc kubenswrapper[5052]: I1203 13:06:44.171769 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/75b0171a-f626-444b-888f-80131592dbfe-kube-api-access-krsl2" (OuterVolumeSpecName: "kube-api-access-krsl2") pod "75b0171a-f626-444b-888f-80131592dbfe" (UID: "75b0171a-f626-444b-888f-80131592dbfe"). InnerVolumeSpecName "kube-api-access-krsl2". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:06:44 crc kubenswrapper[5052]: I1203 13:06:44.184340 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/75b0171a-f626-444b-888f-80131592dbfe-crc-storage" (OuterVolumeSpecName: "crc-storage") pod "75b0171a-f626-444b-888f-80131592dbfe" (UID: "75b0171a-f626-444b-888f-80131592dbfe"). InnerVolumeSpecName "crc-storage". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:06:44 crc kubenswrapper[5052]: I1203 13:06:44.267100 5052 reconciler_common.go:293] "Volume detached for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/75b0171a-f626-444b-888f-80131592dbfe-crc-storage\") on node \"crc\" DevicePath \"\"" Dec 03 13:06:44 crc kubenswrapper[5052]: I1203 13:06:44.267146 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-krsl2\" (UniqueName: \"kubernetes.io/projected/75b0171a-f626-444b-888f-80131592dbfe-kube-api-access-krsl2\") on node \"crc\" DevicePath \"\"" Dec 03 13:06:44 crc kubenswrapper[5052]: I1203 13:06:44.824834 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="crc-storage/crc-storage-crc-9nprf" event={"ID":"75b0171a-f626-444b-888f-80131592dbfe","Type":"ContainerDied","Data":"eb12bb55c47ef6befbd0d1361b7469ff407633f09880d5a41daddcc0abb743a1"} Dec 03 13:06:44 crc kubenswrapper[5052]: I1203 13:06:44.824887 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="eb12bb55c47ef6befbd0d1361b7469ff407633f09880d5a41daddcc0abb743a1" Dec 03 13:06:44 crc kubenswrapper[5052]: I1203 13:06:44.824913 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="crc-storage/crc-storage-crc-9nprf" Dec 03 13:06:47 crc kubenswrapper[5052]: I1203 13:06:47.790209 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["crc-storage/crc-storage-crc-9nprf"] Dec 03 13:06:47 crc kubenswrapper[5052]: I1203 13:06:47.797339 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["crc-storage/crc-storage-crc-9nprf"] Dec 03 13:06:47 crc kubenswrapper[5052]: I1203 13:06:47.921212 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["crc-storage/crc-storage-crc-v74gs"] Dec 03 13:06:47 crc kubenswrapper[5052]: E1203 13:06:47.921644 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="75b0171a-f626-444b-888f-80131592dbfe" containerName="storage" Dec 03 13:06:47 crc kubenswrapper[5052]: I1203 13:06:47.921673 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="75b0171a-f626-444b-888f-80131592dbfe" containerName="storage" Dec 03 13:06:47 crc kubenswrapper[5052]: E1203 13:06:47.921700 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="ff4169c4-e917-42c4-9ca1-b4f9d62be591" containerName="dnsmasq-dns" Dec 03 13:06:47 crc kubenswrapper[5052]: I1203 13:06:47.921706 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="ff4169c4-e917-42c4-9ca1-b4f9d62be591" containerName="dnsmasq-dns" Dec 03 13:06:47 crc kubenswrapper[5052]: E1203 13:06:47.921723 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="ff4169c4-e917-42c4-9ca1-b4f9d62be591" containerName="init" Dec 03 13:06:47 crc kubenswrapper[5052]: I1203 13:06:47.921729 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="ff4169c4-e917-42c4-9ca1-b4f9d62be591" containerName="init" Dec 03 13:06:47 crc kubenswrapper[5052]: I1203 13:06:47.921879 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="75b0171a-f626-444b-888f-80131592dbfe" containerName="storage" Dec 03 13:06:47 crc kubenswrapper[5052]: I1203 13:06:47.921890 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="ff4169c4-e917-42c4-9ca1-b4f9d62be591" containerName="dnsmasq-dns" Dec 03 13:06:47 crc kubenswrapper[5052]: I1203 13:06:47.923111 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="crc-storage/crc-storage-crc-v74gs" Dec 03 13:06:47 crc kubenswrapper[5052]: I1203 13:06:47.927054 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"crc-storage"/"openshift-service-ca.crt" Dec 03 13:06:47 crc kubenswrapper[5052]: I1203 13:06:47.927057 5052 reflector.go:368] Caches populated for *v1.Secret from object-"crc-storage"/"crc-storage-dockercfg-mf76g" Dec 03 13:06:47 crc kubenswrapper[5052]: I1203 13:06:47.927141 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"crc-storage"/"crc-storage" Dec 03 13:06:47 crc kubenswrapper[5052]: I1203 13:06:47.927264 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"crc-storage"/"kube-root-ca.crt" Dec 03 13:06:47 crc kubenswrapper[5052]: I1203 13:06:47.927431 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["crc-storage/crc-storage-crc-v74gs"] Dec 03 13:06:48 crc kubenswrapper[5052]: I1203 13:06:48.024582 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/3eb2a1e9-07d5-4220-84b2-55c704ba4aae-crc-storage\") pod \"crc-storage-crc-v74gs\" (UID: \"3eb2a1e9-07d5-4220-84b2-55c704ba4aae\") " pod="crc-storage/crc-storage-crc-v74gs" Dec 03 13:06:48 crc kubenswrapper[5052]: I1203 13:06:48.024885 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hdk74\" (UniqueName: \"kubernetes.io/projected/3eb2a1e9-07d5-4220-84b2-55c704ba4aae-kube-api-access-hdk74\") pod \"crc-storage-crc-v74gs\" (UID: \"3eb2a1e9-07d5-4220-84b2-55c704ba4aae\") " pod="crc-storage/crc-storage-crc-v74gs" Dec 03 13:06:48 crc kubenswrapper[5052]: I1203 13:06:48.025096 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/3eb2a1e9-07d5-4220-84b2-55c704ba4aae-node-mnt\") pod \"crc-storage-crc-v74gs\" (UID: \"3eb2a1e9-07d5-4220-84b2-55c704ba4aae\") " pod="crc-storage/crc-storage-crc-v74gs" Dec 03 13:06:48 crc kubenswrapper[5052]: I1203 13:06:48.127757 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/3eb2a1e9-07d5-4220-84b2-55c704ba4aae-crc-storage\") pod \"crc-storage-crc-v74gs\" (UID: \"3eb2a1e9-07d5-4220-84b2-55c704ba4aae\") " pod="crc-storage/crc-storage-crc-v74gs" Dec 03 13:06:48 crc kubenswrapper[5052]: I1203 13:06:48.127859 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-hdk74\" (UniqueName: \"kubernetes.io/projected/3eb2a1e9-07d5-4220-84b2-55c704ba4aae-kube-api-access-hdk74\") pod \"crc-storage-crc-v74gs\" (UID: \"3eb2a1e9-07d5-4220-84b2-55c704ba4aae\") " pod="crc-storage/crc-storage-crc-v74gs" Dec 03 13:06:48 crc kubenswrapper[5052]: I1203 13:06:48.127913 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/3eb2a1e9-07d5-4220-84b2-55c704ba4aae-node-mnt\") pod \"crc-storage-crc-v74gs\" (UID: \"3eb2a1e9-07d5-4220-84b2-55c704ba4aae\") " pod="crc-storage/crc-storage-crc-v74gs" Dec 03 13:06:48 crc kubenswrapper[5052]: I1203 13:06:48.128361 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/3eb2a1e9-07d5-4220-84b2-55c704ba4aae-node-mnt\") pod \"crc-storage-crc-v74gs\" (UID: \"3eb2a1e9-07d5-4220-84b2-55c704ba4aae\") " pod="crc-storage/crc-storage-crc-v74gs" Dec 03 13:06:48 crc kubenswrapper[5052]: I1203 13:06:48.128723 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/3eb2a1e9-07d5-4220-84b2-55c704ba4aae-crc-storage\") pod \"crc-storage-crc-v74gs\" (UID: \"3eb2a1e9-07d5-4220-84b2-55c704ba4aae\") " pod="crc-storage/crc-storage-crc-v74gs" Dec 03 13:06:48 crc kubenswrapper[5052]: I1203 13:06:48.150784 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-hdk74\" (UniqueName: \"kubernetes.io/projected/3eb2a1e9-07d5-4220-84b2-55c704ba4aae-kube-api-access-hdk74\") pod \"crc-storage-crc-v74gs\" (UID: \"3eb2a1e9-07d5-4220-84b2-55c704ba4aae\") " pod="crc-storage/crc-storage-crc-v74gs" Dec 03 13:06:48 crc kubenswrapper[5052]: I1203 13:06:48.249334 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="crc-storage/crc-storage-crc-v74gs" Dec 03 13:06:48 crc kubenswrapper[5052]: I1203 13:06:48.656924 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["crc-storage/crc-storage-crc-v74gs"] Dec 03 13:06:48 crc kubenswrapper[5052]: I1203 13:06:48.876834 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="crc-storage/crc-storage-crc-v74gs" event={"ID":"3eb2a1e9-07d5-4220-84b2-55c704ba4aae","Type":"ContainerStarted","Data":"ef0424564de10c3198294a5c89abd6e2cd4ec490b32d62cb546c0372db63b846"} Dec 03 13:06:49 crc kubenswrapper[5052]: I1203 13:06:49.681667 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="75b0171a-f626-444b-888f-80131592dbfe" path="/var/lib/kubelet/pods/75b0171a-f626-444b-888f-80131592dbfe/volumes" Dec 03 13:06:49 crc kubenswrapper[5052]: I1203 13:06:49.887405 5052 generic.go:334] "Generic (PLEG): container finished" podID="3eb2a1e9-07d5-4220-84b2-55c704ba4aae" containerID="a47bc2d2848c1474f67abcd4b147864bcc3995d3a9a6ebf5e5fb621627f4d548" exitCode=0 Dec 03 13:06:49 crc kubenswrapper[5052]: I1203 13:06:49.887455 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="crc-storage/crc-storage-crc-v74gs" event={"ID":"3eb2a1e9-07d5-4220-84b2-55c704ba4aae","Type":"ContainerDied","Data":"a47bc2d2848c1474f67abcd4b147864bcc3995d3a9a6ebf5e5fb621627f4d548"} Dec 03 13:06:51 crc kubenswrapper[5052]: I1203 13:06:51.178367 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="crc-storage/crc-storage-crc-v74gs" Dec 03 13:06:51 crc kubenswrapper[5052]: I1203 13:06:51.277149 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/3eb2a1e9-07d5-4220-84b2-55c704ba4aae-node-mnt\") pod \"3eb2a1e9-07d5-4220-84b2-55c704ba4aae\" (UID: \"3eb2a1e9-07d5-4220-84b2-55c704ba4aae\") " Dec 03 13:06:51 crc kubenswrapper[5052]: I1203 13:06:51.277192 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/3eb2a1e9-07d5-4220-84b2-55c704ba4aae-crc-storage\") pod \"3eb2a1e9-07d5-4220-84b2-55c704ba4aae\" (UID: \"3eb2a1e9-07d5-4220-84b2-55c704ba4aae\") " Dec 03 13:06:51 crc kubenswrapper[5052]: I1203 13:06:51.277244 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/3eb2a1e9-07d5-4220-84b2-55c704ba4aae-node-mnt" (OuterVolumeSpecName: "node-mnt") pod "3eb2a1e9-07d5-4220-84b2-55c704ba4aae" (UID: "3eb2a1e9-07d5-4220-84b2-55c704ba4aae"). InnerVolumeSpecName "node-mnt". PluginName "kubernetes.io/host-path", VolumeGidValue "" Dec 03 13:06:51 crc kubenswrapper[5052]: I1203 13:06:51.277261 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hdk74\" (UniqueName: \"kubernetes.io/projected/3eb2a1e9-07d5-4220-84b2-55c704ba4aae-kube-api-access-hdk74\") pod \"3eb2a1e9-07d5-4220-84b2-55c704ba4aae\" (UID: \"3eb2a1e9-07d5-4220-84b2-55c704ba4aae\") " Dec 03 13:06:51 crc kubenswrapper[5052]: I1203 13:06:51.277911 5052 reconciler_common.go:293] "Volume detached for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/3eb2a1e9-07d5-4220-84b2-55c704ba4aae-node-mnt\") on node \"crc\" DevicePath \"\"" Dec 03 13:06:51 crc kubenswrapper[5052]: I1203 13:06:51.281749 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3eb2a1e9-07d5-4220-84b2-55c704ba4aae-kube-api-access-hdk74" (OuterVolumeSpecName: "kube-api-access-hdk74") pod "3eb2a1e9-07d5-4220-84b2-55c704ba4aae" (UID: "3eb2a1e9-07d5-4220-84b2-55c704ba4aae"). InnerVolumeSpecName "kube-api-access-hdk74". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:06:51 crc kubenswrapper[5052]: I1203 13:06:51.298865 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3eb2a1e9-07d5-4220-84b2-55c704ba4aae-crc-storage" (OuterVolumeSpecName: "crc-storage") pod "3eb2a1e9-07d5-4220-84b2-55c704ba4aae" (UID: "3eb2a1e9-07d5-4220-84b2-55c704ba4aae"). InnerVolumeSpecName "crc-storage". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:06:51 crc kubenswrapper[5052]: I1203 13:06:51.378946 5052 reconciler_common.go:293] "Volume detached for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/3eb2a1e9-07d5-4220-84b2-55c704ba4aae-crc-storage\") on node \"crc\" DevicePath \"\"" Dec 03 13:06:51 crc kubenswrapper[5052]: I1203 13:06:51.378989 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-hdk74\" (UniqueName: \"kubernetes.io/projected/3eb2a1e9-07d5-4220-84b2-55c704ba4aae-kube-api-access-hdk74\") on node \"crc\" DevicePath \"\"" Dec 03 13:06:51 crc kubenswrapper[5052]: I1203 13:06:51.909061 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="crc-storage/crc-storage-crc-v74gs" event={"ID":"3eb2a1e9-07d5-4220-84b2-55c704ba4aae","Type":"ContainerDied","Data":"ef0424564de10c3198294a5c89abd6e2cd4ec490b32d62cb546c0372db63b846"} Dec 03 13:06:51 crc kubenswrapper[5052]: I1203 13:06:51.909745 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ef0424564de10c3198294a5c89abd6e2cd4ec490b32d62cb546c0372db63b846" Dec 03 13:06:51 crc kubenswrapper[5052]: I1203 13:06:51.909118 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="crc-storage/crc-storage-crc-v74gs" Dec 03 13:06:54 crc kubenswrapper[5052]: I1203 13:06:54.633458 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-76cd9645f5-hrmzf"] Dec 03 13:06:54 crc kubenswrapper[5052]: E1203 13:06:54.634081 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="3eb2a1e9-07d5-4220-84b2-55c704ba4aae" containerName="storage" Dec 03 13:06:54 crc kubenswrapper[5052]: I1203 13:06:54.634097 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="3eb2a1e9-07d5-4220-84b2-55c704ba4aae" containerName="storage" Dec 03 13:06:54 crc kubenswrapper[5052]: I1203 13:06:54.634300 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="3eb2a1e9-07d5-4220-84b2-55c704ba4aae" containerName="storage" Dec 03 13:06:54 crc kubenswrapper[5052]: I1203 13:06:54.635265 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-76cd9645f5-hrmzf" Dec 03 13:06:54 crc kubenswrapper[5052]: I1203 13:06:54.638748 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"edpm-extramounts" Dec 03 13:06:54 crc kubenswrapper[5052]: I1203 13:06:54.663986 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-76cd9645f5-hrmzf"] Dec 03 13:06:54 crc kubenswrapper[5052]: I1203 13:06:54.722918 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openshift-marketplace/redhat-marketplace-zjjng"] Dec 03 13:06:54 crc kubenswrapper[5052]: I1203 13:06:54.724593 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/redhat-marketplace-zjjng" Dec 03 13:06:54 crc kubenswrapper[5052]: I1203 13:06:54.727263 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jqhml\" (UniqueName: \"kubernetes.io/projected/946e2885-6643-4b9e-931c-4c8b45fb0910-kube-api-access-jqhml\") pod \"dnsmasq-dnsmasq-76cd9645f5-hrmzf\" (UID: \"946e2885-6643-4b9e-931c-4c8b45fb0910\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-76cd9645f5-hrmzf" Dec 03 13:06:54 crc kubenswrapper[5052]: I1203 13:06:54.727448 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/946e2885-6643-4b9e-931c-4c8b45fb0910-config\") pod \"dnsmasq-dnsmasq-76cd9645f5-hrmzf\" (UID: \"946e2885-6643-4b9e-931c-4c8b45fb0910\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-76cd9645f5-hrmzf" Dec 03 13:06:54 crc kubenswrapper[5052]: I1203 13:06:54.727591 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"edpm-extramounts\" (UniqueName: \"kubernetes.io/configmap/946e2885-6643-4b9e-931c-4c8b45fb0910-edpm-extramounts\") pod \"dnsmasq-dnsmasq-76cd9645f5-hrmzf\" (UID: \"946e2885-6643-4b9e-931c-4c8b45fb0910\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-76cd9645f5-hrmzf" Dec 03 13:06:54 crc kubenswrapper[5052]: I1203 13:06:54.727724 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/946e2885-6643-4b9e-931c-4c8b45fb0910-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-76cd9645f5-hrmzf\" (UID: \"946e2885-6643-4b9e-931c-4c8b45fb0910\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-76cd9645f5-hrmzf" Dec 03 13:06:54 crc kubenswrapper[5052]: I1203 13:06:54.732503 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/redhat-marketplace-zjjng"] Dec 03 13:06:54 crc kubenswrapper[5052]: I1203 13:06:54.828889 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/2b154bce-7c2d-4f43-a304-1241a1355ceb-catalog-content\") pod \"redhat-marketplace-zjjng\" (UID: \"2b154bce-7c2d-4f43-a304-1241a1355ceb\") " pod="openshift-marketplace/redhat-marketplace-zjjng" Dec 03 13:06:54 crc kubenswrapper[5052]: I1203 13:06:54.828968 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-jqhml\" (UniqueName: \"kubernetes.io/projected/946e2885-6643-4b9e-931c-4c8b45fb0910-kube-api-access-jqhml\") pod \"dnsmasq-dnsmasq-76cd9645f5-hrmzf\" (UID: \"946e2885-6643-4b9e-931c-4c8b45fb0910\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-76cd9645f5-hrmzf" Dec 03 13:06:54 crc kubenswrapper[5052]: I1203 13:06:54.828992 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/946e2885-6643-4b9e-931c-4c8b45fb0910-config\") pod \"dnsmasq-dnsmasq-76cd9645f5-hrmzf\" (UID: \"946e2885-6643-4b9e-931c-4c8b45fb0910\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-76cd9645f5-hrmzf" Dec 03 13:06:54 crc kubenswrapper[5052]: I1203 13:06:54.829008 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"edpm-extramounts\" (UniqueName: \"kubernetes.io/configmap/946e2885-6643-4b9e-931c-4c8b45fb0910-edpm-extramounts\") pod \"dnsmasq-dnsmasq-76cd9645f5-hrmzf\" (UID: \"946e2885-6643-4b9e-931c-4c8b45fb0910\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-76cd9645f5-hrmzf" Dec 03 13:06:54 crc kubenswrapper[5052]: I1203 13:06:54.829038 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/2b154bce-7c2d-4f43-a304-1241a1355ceb-utilities\") pod \"redhat-marketplace-zjjng\" (UID: \"2b154bce-7c2d-4f43-a304-1241a1355ceb\") " pod="openshift-marketplace/redhat-marketplace-zjjng" Dec 03 13:06:54 crc kubenswrapper[5052]: I1203 13:06:54.829060 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/946e2885-6643-4b9e-931c-4c8b45fb0910-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-76cd9645f5-hrmzf\" (UID: \"946e2885-6643-4b9e-931c-4c8b45fb0910\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-76cd9645f5-hrmzf" Dec 03 13:06:54 crc kubenswrapper[5052]: I1203 13:06:54.829098 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f4kvh\" (UniqueName: \"kubernetes.io/projected/2b154bce-7c2d-4f43-a304-1241a1355ceb-kube-api-access-f4kvh\") pod \"redhat-marketplace-zjjng\" (UID: \"2b154bce-7c2d-4f43-a304-1241a1355ceb\") " pod="openshift-marketplace/redhat-marketplace-zjjng" Dec 03 13:06:54 crc kubenswrapper[5052]: I1203 13:06:54.830672 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/946e2885-6643-4b9e-931c-4c8b45fb0910-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-76cd9645f5-hrmzf\" (UID: \"946e2885-6643-4b9e-931c-4c8b45fb0910\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-76cd9645f5-hrmzf" Dec 03 13:06:54 crc kubenswrapper[5052]: I1203 13:06:54.831234 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/946e2885-6643-4b9e-931c-4c8b45fb0910-config\") pod \"dnsmasq-dnsmasq-76cd9645f5-hrmzf\" (UID: \"946e2885-6643-4b9e-931c-4c8b45fb0910\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-76cd9645f5-hrmzf" Dec 03 13:06:54 crc kubenswrapper[5052]: I1203 13:06:54.831664 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"edpm-extramounts\" (UniqueName: \"kubernetes.io/configmap/946e2885-6643-4b9e-931c-4c8b45fb0910-edpm-extramounts\") pod \"dnsmasq-dnsmasq-76cd9645f5-hrmzf\" (UID: \"946e2885-6643-4b9e-931c-4c8b45fb0910\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-76cd9645f5-hrmzf" Dec 03 13:06:54 crc kubenswrapper[5052]: I1203 13:06:54.857498 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-jqhml\" (UniqueName: \"kubernetes.io/projected/946e2885-6643-4b9e-931c-4c8b45fb0910-kube-api-access-jqhml\") pod \"dnsmasq-dnsmasq-76cd9645f5-hrmzf\" (UID: \"946e2885-6643-4b9e-931c-4c8b45fb0910\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-76cd9645f5-hrmzf" Dec 03 13:06:54 crc kubenswrapper[5052]: I1203 13:06:54.930439 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/2b154bce-7c2d-4f43-a304-1241a1355ceb-catalog-content\") pod \"redhat-marketplace-zjjng\" (UID: \"2b154bce-7c2d-4f43-a304-1241a1355ceb\") " pod="openshift-marketplace/redhat-marketplace-zjjng" Dec 03 13:06:54 crc kubenswrapper[5052]: I1203 13:06:54.930549 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/2b154bce-7c2d-4f43-a304-1241a1355ceb-utilities\") pod \"redhat-marketplace-zjjng\" (UID: \"2b154bce-7c2d-4f43-a304-1241a1355ceb\") " pod="openshift-marketplace/redhat-marketplace-zjjng" Dec 03 13:06:54 crc kubenswrapper[5052]: I1203 13:06:54.930608 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-f4kvh\" (UniqueName: \"kubernetes.io/projected/2b154bce-7c2d-4f43-a304-1241a1355ceb-kube-api-access-f4kvh\") pod \"redhat-marketplace-zjjng\" (UID: \"2b154bce-7c2d-4f43-a304-1241a1355ceb\") " pod="openshift-marketplace/redhat-marketplace-zjjng" Dec 03 13:06:54 crc kubenswrapper[5052]: I1203 13:06:54.931159 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/2b154bce-7c2d-4f43-a304-1241a1355ceb-catalog-content\") pod \"redhat-marketplace-zjjng\" (UID: \"2b154bce-7c2d-4f43-a304-1241a1355ceb\") " pod="openshift-marketplace/redhat-marketplace-zjjng" Dec 03 13:06:54 crc kubenswrapper[5052]: I1203 13:06:54.931377 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/2b154bce-7c2d-4f43-a304-1241a1355ceb-utilities\") pod \"redhat-marketplace-zjjng\" (UID: \"2b154bce-7c2d-4f43-a304-1241a1355ceb\") " pod="openshift-marketplace/redhat-marketplace-zjjng" Dec 03 13:06:54 crc kubenswrapper[5052]: I1203 13:06:54.950097 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-f4kvh\" (UniqueName: \"kubernetes.io/projected/2b154bce-7c2d-4f43-a304-1241a1355ceb-kube-api-access-f4kvh\") pod \"redhat-marketplace-zjjng\" (UID: \"2b154bce-7c2d-4f43-a304-1241a1355ceb\") " pod="openshift-marketplace/redhat-marketplace-zjjng" Dec 03 13:06:54 crc kubenswrapper[5052]: I1203 13:06:54.959847 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-76cd9645f5-hrmzf" Dec 03 13:06:55 crc kubenswrapper[5052]: I1203 13:06:55.046107 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/redhat-marketplace-zjjng" Dec 03 13:06:55 crc kubenswrapper[5052]: I1203 13:06:55.438429 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-76cd9645f5-hrmzf"] Dec 03 13:06:55 crc kubenswrapper[5052]: W1203 13:06:55.442252 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod946e2885_6643_4b9e_931c_4c8b45fb0910.slice/crio-f89d7c95588eb98dac05c70d1bb8ad5cc0538f4a253903a685117438c781fc51 WatchSource:0}: Error finding container f89d7c95588eb98dac05c70d1bb8ad5cc0538f4a253903a685117438c781fc51: Status 404 returned error can't find the container with id f89d7c95588eb98dac05c70d1bb8ad5cc0538f4a253903a685117438c781fc51 Dec 03 13:06:55 crc kubenswrapper[5052]: I1203 13:06:55.541175 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/redhat-marketplace-zjjng"] Dec 03 13:06:55 crc kubenswrapper[5052]: W1203 13:06:55.547041 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b154bce_7c2d_4f43_a304_1241a1355ceb.slice/crio-6b6cef13ebed3fe160229c48358528b101a471c91a8568d02535b72014d20e00 WatchSource:0}: Error finding container 6b6cef13ebed3fe160229c48358528b101a471c91a8568d02535b72014d20e00: Status 404 returned error can't find the container with id 6b6cef13ebed3fe160229c48358528b101a471c91a8568d02535b72014d20e00 Dec 03 13:06:55 crc kubenswrapper[5052]: I1203 13:06:55.944815 5052 generic.go:334] "Generic (PLEG): container finished" podID="946e2885-6643-4b9e-931c-4c8b45fb0910" containerID="2b7ead942a36e5ffede2d552fd91ac0a9dc6c7efa4575c46f4c19659c703d70d" exitCode=0 Dec 03 13:06:55 crc kubenswrapper[5052]: I1203 13:06:55.944859 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-76cd9645f5-hrmzf" event={"ID":"946e2885-6643-4b9e-931c-4c8b45fb0910","Type":"ContainerDied","Data":"2b7ead942a36e5ffede2d552fd91ac0a9dc6c7efa4575c46f4c19659c703d70d"} Dec 03 13:06:55 crc kubenswrapper[5052]: I1203 13:06:55.945135 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-76cd9645f5-hrmzf" event={"ID":"946e2885-6643-4b9e-931c-4c8b45fb0910","Type":"ContainerStarted","Data":"f89d7c95588eb98dac05c70d1bb8ad5cc0538f4a253903a685117438c781fc51"} Dec 03 13:06:55 crc kubenswrapper[5052]: I1203 13:06:55.947230 5052 generic.go:334] "Generic (PLEG): container finished" podID="2b154bce-7c2d-4f43-a304-1241a1355ceb" containerID="4c09e2e7a4b0ceeb31bd1b0399dd860d7f51c9d05dd45ceefdbe7a2c2d9d548b" exitCode=0 Dec 03 13:06:55 crc kubenswrapper[5052]: I1203 13:06:55.947265 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-marketplace-zjjng" event={"ID":"2b154bce-7c2d-4f43-a304-1241a1355ceb","Type":"ContainerDied","Data":"4c09e2e7a4b0ceeb31bd1b0399dd860d7f51c9d05dd45ceefdbe7a2c2d9d548b"} Dec 03 13:06:55 crc kubenswrapper[5052]: I1203 13:06:55.947320 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-marketplace-zjjng" event={"ID":"2b154bce-7c2d-4f43-a304-1241a1355ceb","Type":"ContainerStarted","Data":"6b6cef13ebed3fe160229c48358528b101a471c91a8568d02535b72014d20e00"} Dec 03 13:06:56 crc kubenswrapper[5052]: I1203 13:06:56.956504 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-76cd9645f5-hrmzf" event={"ID":"946e2885-6643-4b9e-931c-4c8b45fb0910","Type":"ContainerStarted","Data":"096df24192ce1de338b52a0bf35e409355f5ed81a274ef42637003e28d70ee76"} Dec 03 13:06:56 crc kubenswrapper[5052]: I1203 13:06:56.956885 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-76cd9645f5-hrmzf" Dec 03 13:06:56 crc kubenswrapper[5052]: I1203 13:06:56.972890 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-76cd9645f5-hrmzf" podStartSLOduration=2.972868518 podStartE2EDuration="2.972868518s" podCreationTimestamp="2025-12-03 13:06:54 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 13:06:56.971442778 +0000 UTC m=+6407.559684562" watchObservedRunningTime="2025-12-03 13:06:56.972868518 +0000 UTC m=+6407.561110292" Dec 03 13:06:57 crc kubenswrapper[5052]: I1203 13:06:57.966110 5052 generic.go:334] "Generic (PLEG): container finished" podID="2b154bce-7c2d-4f43-a304-1241a1355ceb" containerID="2fd516d3058824e2255080f2540e6dc0318d1c750252c6b4ef4e3211d6a49d90" exitCode=0 Dec 03 13:06:57 crc kubenswrapper[5052]: I1203 13:06:57.966345 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-marketplace-zjjng" event={"ID":"2b154bce-7c2d-4f43-a304-1241a1355ceb","Type":"ContainerDied","Data":"2fd516d3058824e2255080f2540e6dc0318d1c750252c6b4ef4e3211d6a49d90"} Dec 03 13:06:58 crc kubenswrapper[5052]: I1203 13:06:58.977966 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-marketplace-zjjng" event={"ID":"2b154bce-7c2d-4f43-a304-1241a1355ceb","Type":"ContainerStarted","Data":"1041a464a64241fa4ce8b8724582c2002ae228bea93bf7910b913eeef8ca4080"} Dec 03 13:06:58 crc kubenswrapper[5052]: I1203 13:06:58.996094 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-marketplace/redhat-marketplace-zjjng" podStartSLOduration=2.417049273 podStartE2EDuration="4.996054258s" podCreationTimestamp="2025-12-03 13:06:54 +0000 UTC" firstStartedPulling="2025-12-03 13:06:55.948752321 +0000 UTC m=+6406.536994085" lastFinishedPulling="2025-12-03 13:06:58.527757306 +0000 UTC m=+6409.115999070" observedRunningTime="2025-12-03 13:06:58.993450554 +0000 UTC m=+6409.581692318" watchObservedRunningTime="2025-12-03 13:06:58.996054258 +0000 UTC m=+6409.584296022" Dec 03 13:06:59 crc kubenswrapper[5052]: I1203 13:06:59.379860 5052 patch_prober.go:28] interesting pod/machine-config-daemon-74bj6 container/machine-config-daemon namespace/openshift-machine-config-operator: Liveness probe status=failure output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" start-of-body= Dec 03 13:06:59 crc kubenswrapper[5052]: I1203 13:06:59.379914 5052 prober.go:107] "Probe failed" probeType="Liveness" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" containerName="machine-config-daemon" probeResult="failure" output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" Dec 03 13:06:59 crc kubenswrapper[5052]: I1203 13:06:59.379951 5052 kubelet.go:2542] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" Dec 03 13:06:59 crc kubenswrapper[5052]: I1203 13:06:59.380596 5052 kuberuntime_manager.go:1027] "Message for Container of pod" containerName="machine-config-daemon" containerStatusID={"Type":"cri-o","ID":"dbc100c8081ea1a2f5487429cc9d67de6bc8ba11282b64f248c71275375b0e16"} pod="openshift-machine-config-operator/machine-config-daemon-74bj6" containerMessage="Container machine-config-daemon failed liveness probe, will be restarted" Dec 03 13:06:59 crc kubenswrapper[5052]: I1203 13:06:59.380655 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" containerName="machine-config-daemon" containerID="cri-o://dbc100c8081ea1a2f5487429cc9d67de6bc8ba11282b64f248c71275375b0e16" gracePeriod=600 Dec 03 13:06:59 crc kubenswrapper[5052]: E1203 13:06:59.507299 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 13:06:59 crc kubenswrapper[5052]: I1203 13:06:59.989015 5052 generic.go:334] "Generic (PLEG): container finished" podID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" containerID="dbc100c8081ea1a2f5487429cc9d67de6bc8ba11282b64f248c71275375b0e16" exitCode=0 Dec 03 13:06:59 crc kubenswrapper[5052]: I1203 13:06:59.989082 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" event={"ID":"10c237bc-72e7-4da3-9f2c-b1688395bf6e","Type":"ContainerDied","Data":"dbc100c8081ea1a2f5487429cc9d67de6bc8ba11282b64f248c71275375b0e16"} Dec 03 13:06:59 crc kubenswrapper[5052]: I1203 13:06:59.989122 5052 scope.go:117] "RemoveContainer" containerID="c2cab872e68bfd02259363286d891f85a8e365964ddb0da9f72bb76c86e2d3d2" Dec 03 13:06:59 crc kubenswrapper[5052]: I1203 13:06:59.989944 5052 scope.go:117] "RemoveContainer" containerID="dbc100c8081ea1a2f5487429cc9d67de6bc8ba11282b64f248c71275375b0e16" Dec 03 13:06:59 crc kubenswrapper[5052]: E1203 13:06:59.990206 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 13:07:04 crc kubenswrapper[5052]: I1203 13:07:04.961664 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-76cd9645f5-hrmzf" Dec 03 13:07:05 crc kubenswrapper[5052]: I1203 13:07:05.011550 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-n5w2j"] Dec 03 13:07:05 crc kubenswrapper[5052]: I1203 13:07:05.011773 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-n5w2j" podUID="f7ddd39d-65bc-46a4-8938-20ac4f6a2078" containerName="dnsmasq-dns" containerID="cri-o://0b3afaae2cbac154f4abc313f1ea0fcf8c553fbd1b0e6b72c914c2cba00c89e8" gracePeriod=10 Dec 03 13:07:05 crc kubenswrapper[5052]: I1203 13:07:05.046593 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openshift-marketplace/redhat-marketplace-zjjng" Dec 03 13:07:05 crc kubenswrapper[5052]: I1203 13:07:05.048092 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-marketplace/redhat-marketplace-zjjng" Dec 03 13:07:05 crc kubenswrapper[5052]: I1203 13:07:05.098328 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-marketplace/redhat-marketplace-zjjng" Dec 03 13:07:05 crc kubenswrapper[5052]: I1203 13:07:05.920643 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-n5w2j" Dec 03 13:07:05 crc kubenswrapper[5052]: I1203 13:07:05.999645 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gbv8z\" (UniqueName: \"kubernetes.io/projected/f7ddd39d-65bc-46a4-8938-20ac4f6a2078-kube-api-access-gbv8z\") pod \"f7ddd39d-65bc-46a4-8938-20ac4f6a2078\" (UID: \"f7ddd39d-65bc-46a4-8938-20ac4f6a2078\") " Dec 03 13:07:06 crc kubenswrapper[5052]: I1203 13:07:05.999725 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/f7ddd39d-65bc-46a4-8938-20ac4f6a2078-dnsmasq-svc\") pod \"f7ddd39d-65bc-46a4-8938-20ac4f6a2078\" (UID: \"f7ddd39d-65bc-46a4-8938-20ac4f6a2078\") " Dec 03 13:07:06 crc kubenswrapper[5052]: I1203 13:07:05.999814 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f7ddd39d-65bc-46a4-8938-20ac4f6a2078-config\") pod \"f7ddd39d-65bc-46a4-8938-20ac4f6a2078\" (UID: \"f7ddd39d-65bc-46a4-8938-20ac4f6a2078\") " Dec 03 13:07:06 crc kubenswrapper[5052]: I1203 13:07:06.004905 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f7ddd39d-65bc-46a4-8938-20ac4f6a2078-kube-api-access-gbv8z" (OuterVolumeSpecName: "kube-api-access-gbv8z") pod "f7ddd39d-65bc-46a4-8938-20ac4f6a2078" (UID: "f7ddd39d-65bc-46a4-8938-20ac4f6a2078"). InnerVolumeSpecName "kube-api-access-gbv8z". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:07:06 crc kubenswrapper[5052]: I1203 13:07:06.055283 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f7ddd39d-65bc-46a4-8938-20ac4f6a2078-config" (OuterVolumeSpecName: "config") pod "f7ddd39d-65bc-46a4-8938-20ac4f6a2078" (UID: "f7ddd39d-65bc-46a4-8938-20ac4f6a2078"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:07:06 crc kubenswrapper[5052]: I1203 13:07:06.071155 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f7ddd39d-65bc-46a4-8938-20ac4f6a2078-dnsmasq-svc" (OuterVolumeSpecName: "dnsmasq-svc") pod "f7ddd39d-65bc-46a4-8938-20ac4f6a2078" (UID: "f7ddd39d-65bc-46a4-8938-20ac4f6a2078"). InnerVolumeSpecName "dnsmasq-svc". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:07:06 crc kubenswrapper[5052]: I1203 13:07:06.080701 5052 generic.go:334] "Generic (PLEG): container finished" podID="f7ddd39d-65bc-46a4-8938-20ac4f6a2078" containerID="0b3afaae2cbac154f4abc313f1ea0fcf8c553fbd1b0e6b72c914c2cba00c89e8" exitCode=0 Dec 03 13:07:06 crc kubenswrapper[5052]: I1203 13:07:06.081675 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-n5w2j" Dec 03 13:07:06 crc kubenswrapper[5052]: I1203 13:07:06.081931 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-n5w2j" event={"ID":"f7ddd39d-65bc-46a4-8938-20ac4f6a2078","Type":"ContainerDied","Data":"0b3afaae2cbac154f4abc313f1ea0fcf8c553fbd1b0e6b72c914c2cba00c89e8"} Dec 03 13:07:06 crc kubenswrapper[5052]: I1203 13:07:06.081981 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-n5w2j" event={"ID":"f7ddd39d-65bc-46a4-8938-20ac4f6a2078","Type":"ContainerDied","Data":"46b03680de82643b00804a1b184e64f7f680e2d1f3a0287d13bd07cfadaec639"} Dec 03 13:07:06 crc kubenswrapper[5052]: I1203 13:07:06.082004 5052 scope.go:117] "RemoveContainer" containerID="0b3afaae2cbac154f4abc313f1ea0fcf8c553fbd1b0e6b72c914c2cba00c89e8" Dec 03 13:07:06 crc kubenswrapper[5052]: I1203 13:07:06.103488 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-gbv8z\" (UniqueName: \"kubernetes.io/projected/f7ddd39d-65bc-46a4-8938-20ac4f6a2078-kube-api-access-gbv8z\") on node \"crc\" DevicePath \"\"" Dec 03 13:07:06 crc kubenswrapper[5052]: I1203 13:07:06.103556 5052 reconciler_common.go:293] "Volume detached for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/f7ddd39d-65bc-46a4-8938-20ac4f6a2078-dnsmasq-svc\") on node \"crc\" DevicePath \"\"" Dec 03 13:07:06 crc kubenswrapper[5052]: I1203 13:07:06.103570 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f7ddd39d-65bc-46a4-8938-20ac4f6a2078-config\") on node \"crc\" DevicePath \"\"" Dec 03 13:07:06 crc kubenswrapper[5052]: I1203 13:07:06.135038 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-n5w2j"] Dec 03 13:07:06 crc kubenswrapper[5052]: I1203 13:07:06.156299 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-n5w2j"] Dec 03 13:07:06 crc kubenswrapper[5052]: I1203 13:07:06.158696 5052 scope.go:117] "RemoveContainer" containerID="a39a3886f97f5afd814ec6a1c24e3f7c85e19c621d9c90559f3a49074ecb8907" Dec 03 13:07:06 crc kubenswrapper[5052]: I1203 13:07:06.214924 5052 scope.go:117] "RemoveContainer" containerID="0b3afaae2cbac154f4abc313f1ea0fcf8c553fbd1b0e6b72c914c2cba00c89e8" Dec 03 13:07:06 crc kubenswrapper[5052]: E1203 13:07:06.215440 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0b3afaae2cbac154f4abc313f1ea0fcf8c553fbd1b0e6b72c914c2cba00c89e8\": container with ID starting with 0b3afaae2cbac154f4abc313f1ea0fcf8c553fbd1b0e6b72c914c2cba00c89e8 not found: ID does not exist" containerID="0b3afaae2cbac154f4abc313f1ea0fcf8c553fbd1b0e6b72c914c2cba00c89e8" Dec 03 13:07:06 crc kubenswrapper[5052]: I1203 13:07:06.215482 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0b3afaae2cbac154f4abc313f1ea0fcf8c553fbd1b0e6b72c914c2cba00c89e8"} err="failed to get container status \"0b3afaae2cbac154f4abc313f1ea0fcf8c553fbd1b0e6b72c914c2cba00c89e8\": rpc error: code = NotFound desc = could not find container \"0b3afaae2cbac154f4abc313f1ea0fcf8c553fbd1b0e6b72c914c2cba00c89e8\": container with ID starting with 0b3afaae2cbac154f4abc313f1ea0fcf8c553fbd1b0e6b72c914c2cba00c89e8 not found: ID does not exist" Dec 03 13:07:06 crc kubenswrapper[5052]: I1203 13:07:06.215526 5052 scope.go:117] "RemoveContainer" containerID="a39a3886f97f5afd814ec6a1c24e3f7c85e19c621d9c90559f3a49074ecb8907" Dec 03 13:07:06 crc kubenswrapper[5052]: I1203 13:07:06.215676 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-marketplace/redhat-marketplace-zjjng" Dec 03 13:07:06 crc kubenswrapper[5052]: E1203 13:07:06.216137 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a39a3886f97f5afd814ec6a1c24e3f7c85e19c621d9c90559f3a49074ecb8907\": container with ID starting with a39a3886f97f5afd814ec6a1c24e3f7c85e19c621d9c90559f3a49074ecb8907 not found: ID does not exist" containerID="a39a3886f97f5afd814ec6a1c24e3f7c85e19c621d9c90559f3a49074ecb8907" Dec 03 13:07:06 crc kubenswrapper[5052]: I1203 13:07:06.216157 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a39a3886f97f5afd814ec6a1c24e3f7c85e19c621d9c90559f3a49074ecb8907"} err="failed to get container status \"a39a3886f97f5afd814ec6a1c24e3f7c85e19c621d9c90559f3a49074ecb8907\": rpc error: code = NotFound desc = could not find container \"a39a3886f97f5afd814ec6a1c24e3f7c85e19c621d9c90559f3a49074ecb8907\": container with ID starting with a39a3886f97f5afd814ec6a1c24e3f7c85e19c621d9c90559f3a49074ecb8907 not found: ID does not exist" Dec 03 13:07:06 crc kubenswrapper[5052]: I1203 13:07:06.265691 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openshift-marketplace/redhat-marketplace-zjjng"] Dec 03 13:07:07 crc kubenswrapper[5052]: I1203 13:07:07.681581 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f7ddd39d-65bc-46a4-8938-20ac4f6a2078" path="/var/lib/kubelet/pods/f7ddd39d-65bc-46a4-8938-20ac4f6a2078/volumes" Dec 03 13:07:08 crc kubenswrapper[5052]: I1203 13:07:08.100051 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openshift-marketplace/redhat-marketplace-zjjng" podUID="2b154bce-7c2d-4f43-a304-1241a1355ceb" containerName="registry-server" containerID="cri-o://1041a464a64241fa4ce8b8724582c2002ae228bea93bf7910b913eeef8ca4080" gracePeriod=2 Dec 03 13:07:08 crc kubenswrapper[5052]: I1203 13:07:08.564978 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/redhat-marketplace-zjjng" Dec 03 13:07:08 crc kubenswrapper[5052]: I1203 13:07:08.647229 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/2b154bce-7c2d-4f43-a304-1241a1355ceb-utilities\") pod \"2b154bce-7c2d-4f43-a304-1241a1355ceb\" (UID: \"2b154bce-7c2d-4f43-a304-1241a1355ceb\") " Dec 03 13:07:08 crc kubenswrapper[5052]: I1203 13:07:08.647355 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f4kvh\" (UniqueName: \"kubernetes.io/projected/2b154bce-7c2d-4f43-a304-1241a1355ceb-kube-api-access-f4kvh\") pod \"2b154bce-7c2d-4f43-a304-1241a1355ceb\" (UID: \"2b154bce-7c2d-4f43-a304-1241a1355ceb\") " Dec 03 13:07:08 crc kubenswrapper[5052]: I1203 13:07:08.647419 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/2b154bce-7c2d-4f43-a304-1241a1355ceb-catalog-content\") pod \"2b154bce-7c2d-4f43-a304-1241a1355ceb\" (UID: \"2b154bce-7c2d-4f43-a304-1241a1355ceb\") " Dec 03 13:07:08 crc kubenswrapper[5052]: I1203 13:07:08.648345 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2b154bce-7c2d-4f43-a304-1241a1355ceb-utilities" (OuterVolumeSpecName: "utilities") pod "2b154bce-7c2d-4f43-a304-1241a1355ceb" (UID: "2b154bce-7c2d-4f43-a304-1241a1355ceb"). InnerVolumeSpecName "utilities". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 13:07:08 crc kubenswrapper[5052]: I1203 13:07:08.652361 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2b154bce-7c2d-4f43-a304-1241a1355ceb-kube-api-access-f4kvh" (OuterVolumeSpecName: "kube-api-access-f4kvh") pod "2b154bce-7c2d-4f43-a304-1241a1355ceb" (UID: "2b154bce-7c2d-4f43-a304-1241a1355ceb"). InnerVolumeSpecName "kube-api-access-f4kvh". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:07:08 crc kubenswrapper[5052]: I1203 13:07:08.668933 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2b154bce-7c2d-4f43-a304-1241a1355ceb-catalog-content" (OuterVolumeSpecName: "catalog-content") pod "2b154bce-7c2d-4f43-a304-1241a1355ceb" (UID: "2b154bce-7c2d-4f43-a304-1241a1355ceb"). InnerVolumeSpecName "catalog-content". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 13:07:08 crc kubenswrapper[5052]: I1203 13:07:08.749304 5052 reconciler_common.go:293] "Volume detached for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/2b154bce-7c2d-4f43-a304-1241a1355ceb-catalog-content\") on node \"crc\" DevicePath \"\"" Dec 03 13:07:08 crc kubenswrapper[5052]: I1203 13:07:08.750123 5052 reconciler_common.go:293] "Volume detached for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/2b154bce-7c2d-4f43-a304-1241a1355ceb-utilities\") on node \"crc\" DevicePath \"\"" Dec 03 13:07:08 crc kubenswrapper[5052]: I1203 13:07:08.750168 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-f4kvh\" (UniqueName: \"kubernetes.io/projected/2b154bce-7c2d-4f43-a304-1241a1355ceb-kube-api-access-f4kvh\") on node \"crc\" DevicePath \"\"" Dec 03 13:07:09 crc kubenswrapper[5052]: I1203 13:07:09.112457 5052 generic.go:334] "Generic (PLEG): container finished" podID="2b154bce-7c2d-4f43-a304-1241a1355ceb" containerID="1041a464a64241fa4ce8b8724582c2002ae228bea93bf7910b913eeef8ca4080" exitCode=0 Dec 03 13:07:09 crc kubenswrapper[5052]: I1203 13:07:09.112527 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-marketplace-zjjng" event={"ID":"2b154bce-7c2d-4f43-a304-1241a1355ceb","Type":"ContainerDied","Data":"1041a464a64241fa4ce8b8724582c2002ae228bea93bf7910b913eeef8ca4080"} Dec 03 13:07:09 crc kubenswrapper[5052]: I1203 13:07:09.112544 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/redhat-marketplace-zjjng" Dec 03 13:07:09 crc kubenswrapper[5052]: I1203 13:07:09.112567 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-marketplace-zjjng" event={"ID":"2b154bce-7c2d-4f43-a304-1241a1355ceb","Type":"ContainerDied","Data":"6b6cef13ebed3fe160229c48358528b101a471c91a8568d02535b72014d20e00"} Dec 03 13:07:09 crc kubenswrapper[5052]: I1203 13:07:09.112611 5052 scope.go:117] "RemoveContainer" containerID="1041a464a64241fa4ce8b8724582c2002ae228bea93bf7910b913eeef8ca4080" Dec 03 13:07:09 crc kubenswrapper[5052]: I1203 13:07:09.143699 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openshift-marketplace/redhat-marketplace-zjjng"] Dec 03 13:07:09 crc kubenswrapper[5052]: I1203 13:07:09.144159 5052 scope.go:117] "RemoveContainer" containerID="2fd516d3058824e2255080f2540e6dc0318d1c750252c6b4ef4e3211d6a49d90" Dec 03 13:07:09 crc kubenswrapper[5052]: I1203 13:07:09.152089 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openshift-marketplace/redhat-marketplace-zjjng"] Dec 03 13:07:09 crc kubenswrapper[5052]: I1203 13:07:09.178307 5052 scope.go:117] "RemoveContainer" containerID="4c09e2e7a4b0ceeb31bd1b0399dd860d7f51c9d05dd45ceefdbe7a2c2d9d548b" Dec 03 13:07:09 crc kubenswrapper[5052]: I1203 13:07:09.203407 5052 scope.go:117] "RemoveContainer" containerID="1041a464a64241fa4ce8b8724582c2002ae228bea93bf7910b913eeef8ca4080" Dec 03 13:07:09 crc kubenswrapper[5052]: E1203 13:07:09.204234 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1041a464a64241fa4ce8b8724582c2002ae228bea93bf7910b913eeef8ca4080\": container with ID starting with 1041a464a64241fa4ce8b8724582c2002ae228bea93bf7910b913eeef8ca4080 not found: ID does not exist" containerID="1041a464a64241fa4ce8b8724582c2002ae228bea93bf7910b913eeef8ca4080" Dec 03 13:07:09 crc kubenswrapper[5052]: I1203 13:07:09.204279 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1041a464a64241fa4ce8b8724582c2002ae228bea93bf7910b913eeef8ca4080"} err="failed to get container status \"1041a464a64241fa4ce8b8724582c2002ae228bea93bf7910b913eeef8ca4080\": rpc error: code = NotFound desc = could not find container \"1041a464a64241fa4ce8b8724582c2002ae228bea93bf7910b913eeef8ca4080\": container with ID starting with 1041a464a64241fa4ce8b8724582c2002ae228bea93bf7910b913eeef8ca4080 not found: ID does not exist" Dec 03 13:07:09 crc kubenswrapper[5052]: I1203 13:07:09.204307 5052 scope.go:117] "RemoveContainer" containerID="2fd516d3058824e2255080f2540e6dc0318d1c750252c6b4ef4e3211d6a49d90" Dec 03 13:07:09 crc kubenswrapper[5052]: E1203 13:07:09.204715 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2fd516d3058824e2255080f2540e6dc0318d1c750252c6b4ef4e3211d6a49d90\": container with ID starting with 2fd516d3058824e2255080f2540e6dc0318d1c750252c6b4ef4e3211d6a49d90 not found: ID does not exist" containerID="2fd516d3058824e2255080f2540e6dc0318d1c750252c6b4ef4e3211d6a49d90" Dec 03 13:07:09 crc kubenswrapper[5052]: I1203 13:07:09.204748 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2fd516d3058824e2255080f2540e6dc0318d1c750252c6b4ef4e3211d6a49d90"} err="failed to get container status \"2fd516d3058824e2255080f2540e6dc0318d1c750252c6b4ef4e3211d6a49d90\": rpc error: code = NotFound desc = could not find container \"2fd516d3058824e2255080f2540e6dc0318d1c750252c6b4ef4e3211d6a49d90\": container with ID starting with 2fd516d3058824e2255080f2540e6dc0318d1c750252c6b4ef4e3211d6a49d90 not found: ID does not exist" Dec 03 13:07:09 crc kubenswrapper[5052]: I1203 13:07:09.204765 5052 scope.go:117] "RemoveContainer" containerID="4c09e2e7a4b0ceeb31bd1b0399dd860d7f51c9d05dd45ceefdbe7a2c2d9d548b" Dec 03 13:07:09 crc kubenswrapper[5052]: E1203 13:07:09.205171 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4c09e2e7a4b0ceeb31bd1b0399dd860d7f51c9d05dd45ceefdbe7a2c2d9d548b\": container with ID starting with 4c09e2e7a4b0ceeb31bd1b0399dd860d7f51c9d05dd45ceefdbe7a2c2d9d548b not found: ID does not exist" containerID="4c09e2e7a4b0ceeb31bd1b0399dd860d7f51c9d05dd45ceefdbe7a2c2d9d548b" Dec 03 13:07:09 crc kubenswrapper[5052]: I1203 13:07:09.205205 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4c09e2e7a4b0ceeb31bd1b0399dd860d7f51c9d05dd45ceefdbe7a2c2d9d548b"} err="failed to get container status \"4c09e2e7a4b0ceeb31bd1b0399dd860d7f51c9d05dd45ceefdbe7a2c2d9d548b\": rpc error: code = NotFound desc = could not find container \"4c09e2e7a4b0ceeb31bd1b0399dd860d7f51c9d05dd45ceefdbe7a2c2d9d548b\": container with ID starting with 4c09e2e7a4b0ceeb31bd1b0399dd860d7f51c9d05dd45ceefdbe7a2c2d9d548b not found: ID does not exist" Dec 03 13:07:09 crc kubenswrapper[5052]: I1203 13:07:09.707596 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2b154bce-7c2d-4f43-a304-1241a1355ceb" path="/var/lib/kubelet/pods/2b154bce-7c2d-4f43-a304-1241a1355ceb/volumes" Dec 03 13:07:09 crc kubenswrapper[5052]: I1203 13:07:09.782722 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-wnkqw"] Dec 03 13:07:09 crc kubenswrapper[5052]: E1203 13:07:09.783076 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="2b154bce-7c2d-4f43-a304-1241a1355ceb" containerName="extract-utilities" Dec 03 13:07:09 crc kubenswrapper[5052]: I1203 13:07:09.783093 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="2b154bce-7c2d-4f43-a304-1241a1355ceb" containerName="extract-utilities" Dec 03 13:07:09 crc kubenswrapper[5052]: E1203 13:07:09.783104 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="f7ddd39d-65bc-46a4-8938-20ac4f6a2078" containerName="dnsmasq-dns" Dec 03 13:07:09 crc kubenswrapper[5052]: I1203 13:07:09.783111 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7ddd39d-65bc-46a4-8938-20ac4f6a2078" containerName="dnsmasq-dns" Dec 03 13:07:09 crc kubenswrapper[5052]: E1203 13:07:09.783129 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="2b154bce-7c2d-4f43-a304-1241a1355ceb" containerName="extract-content" Dec 03 13:07:09 crc kubenswrapper[5052]: I1203 13:07:09.783135 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="2b154bce-7c2d-4f43-a304-1241a1355ceb" containerName="extract-content" Dec 03 13:07:09 crc kubenswrapper[5052]: E1203 13:07:09.783160 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="f7ddd39d-65bc-46a4-8938-20ac4f6a2078" containerName="init" Dec 03 13:07:09 crc kubenswrapper[5052]: I1203 13:07:09.783165 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7ddd39d-65bc-46a4-8938-20ac4f6a2078" containerName="init" Dec 03 13:07:09 crc kubenswrapper[5052]: E1203 13:07:09.783174 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="2b154bce-7c2d-4f43-a304-1241a1355ceb" containerName="registry-server" Dec 03 13:07:09 crc kubenswrapper[5052]: I1203 13:07:09.783181 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="2b154bce-7c2d-4f43-a304-1241a1355ceb" containerName="registry-server" Dec 03 13:07:09 crc kubenswrapper[5052]: I1203 13:07:09.783314 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="f7ddd39d-65bc-46a4-8938-20ac4f6a2078" containerName="dnsmasq-dns" Dec 03 13:07:09 crc kubenswrapper[5052]: I1203 13:07:09.783325 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="2b154bce-7c2d-4f43-a304-1241a1355ceb" containerName="registry-server" Dec 03 13:07:09 crc kubenswrapper[5052]: I1203 13:07:09.784351 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-wnkqw" Dec 03 13:07:09 crc kubenswrapper[5052]: I1203 13:07:09.792740 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-wnkqw"] Dec 03 13:07:09 crc kubenswrapper[5052]: I1203 13:07:09.867424 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k7c2q\" (UniqueName: \"kubernetes.io/projected/ef40807a-35ce-4af5-9996-bc2f5b2fa3e4-kube-api-access-k7c2q\") pod \"dnsmasq-dnsmasq-84b9f45d47-wnkqw\" (UID: \"ef40807a-35ce-4af5-9996-bc2f5b2fa3e4\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-wnkqw" Dec 03 13:07:09 crc kubenswrapper[5052]: I1203 13:07:09.867493 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ef40807a-35ce-4af5-9996-bc2f5b2fa3e4-config\") pod \"dnsmasq-dnsmasq-84b9f45d47-wnkqw\" (UID: \"ef40807a-35ce-4af5-9996-bc2f5b2fa3e4\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-wnkqw" Dec 03 13:07:09 crc kubenswrapper[5052]: I1203 13:07:09.867558 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/ef40807a-35ce-4af5-9996-bc2f5b2fa3e4-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-84b9f45d47-wnkqw\" (UID: \"ef40807a-35ce-4af5-9996-bc2f5b2fa3e4\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-wnkqw" Dec 03 13:07:09 crc kubenswrapper[5052]: I1203 13:07:09.969114 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/ef40807a-35ce-4af5-9996-bc2f5b2fa3e4-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-84b9f45d47-wnkqw\" (UID: \"ef40807a-35ce-4af5-9996-bc2f5b2fa3e4\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-wnkqw" Dec 03 13:07:09 crc kubenswrapper[5052]: I1203 13:07:09.969256 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-k7c2q\" (UniqueName: \"kubernetes.io/projected/ef40807a-35ce-4af5-9996-bc2f5b2fa3e4-kube-api-access-k7c2q\") pod \"dnsmasq-dnsmasq-84b9f45d47-wnkqw\" (UID: \"ef40807a-35ce-4af5-9996-bc2f5b2fa3e4\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-wnkqw" Dec 03 13:07:09 crc kubenswrapper[5052]: I1203 13:07:09.969360 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ef40807a-35ce-4af5-9996-bc2f5b2fa3e4-config\") pod \"dnsmasq-dnsmasq-84b9f45d47-wnkqw\" (UID: \"ef40807a-35ce-4af5-9996-bc2f5b2fa3e4\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-wnkqw" Dec 03 13:07:09 crc kubenswrapper[5052]: I1203 13:07:09.970095 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/ef40807a-35ce-4af5-9996-bc2f5b2fa3e4-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-84b9f45d47-wnkqw\" (UID: \"ef40807a-35ce-4af5-9996-bc2f5b2fa3e4\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-wnkqw" Dec 03 13:07:09 crc kubenswrapper[5052]: I1203 13:07:09.970125 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ef40807a-35ce-4af5-9996-bc2f5b2fa3e4-config\") pod \"dnsmasq-dnsmasq-84b9f45d47-wnkqw\" (UID: \"ef40807a-35ce-4af5-9996-bc2f5b2fa3e4\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-wnkqw" Dec 03 13:07:09 crc kubenswrapper[5052]: I1203 13:07:09.992929 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-k7c2q\" (UniqueName: \"kubernetes.io/projected/ef40807a-35ce-4af5-9996-bc2f5b2fa3e4-kube-api-access-k7c2q\") pod \"dnsmasq-dnsmasq-84b9f45d47-wnkqw\" (UID: \"ef40807a-35ce-4af5-9996-bc2f5b2fa3e4\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-wnkqw" Dec 03 13:07:10 crc kubenswrapper[5052]: I1203 13:07:10.128883 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-wnkqw" Dec 03 13:07:10 crc kubenswrapper[5052]: I1203 13:07:10.560122 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-wnkqw"] Dec 03 13:07:10 crc kubenswrapper[5052]: W1203 13:07:10.570792 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podef40807a_35ce_4af5_9996_bc2f5b2fa3e4.slice/crio-7929be9fecdc702de97a668f70e5bb31736a5de2644ac0ac3763ca53f49d3e08 WatchSource:0}: Error finding container 7929be9fecdc702de97a668f70e5bb31736a5de2644ac0ac3763ca53f49d3e08: Status 404 returned error can't find the container with id 7929be9fecdc702de97a668f70e5bb31736a5de2644ac0ac3763ca53f49d3e08 Dec 03 13:07:11 crc kubenswrapper[5052]: I1203 13:07:11.141024 5052 generic.go:334] "Generic (PLEG): container finished" podID="ef40807a-35ce-4af5-9996-bc2f5b2fa3e4" containerID="96148fe6fe8d9fa1905c70170e7143057f38d106aceaef2bf07e1425b8756491" exitCode=0 Dec 03 13:07:11 crc kubenswrapper[5052]: I1203 13:07:11.141075 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-wnkqw" event={"ID":"ef40807a-35ce-4af5-9996-bc2f5b2fa3e4","Type":"ContainerDied","Data":"96148fe6fe8d9fa1905c70170e7143057f38d106aceaef2bf07e1425b8756491"} Dec 03 13:07:11 crc kubenswrapper[5052]: I1203 13:07:11.141407 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-wnkqw" event={"ID":"ef40807a-35ce-4af5-9996-bc2f5b2fa3e4","Type":"ContainerStarted","Data":"7929be9fecdc702de97a668f70e5bb31736a5de2644ac0ac3763ca53f49d3e08"} Dec 03 13:07:12 crc kubenswrapper[5052]: I1203 13:07:12.154253 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-wnkqw" event={"ID":"ef40807a-35ce-4af5-9996-bc2f5b2fa3e4","Type":"ContainerStarted","Data":"ec1bccc641595bfeab1754722642ceced12f7f3334c65a4375ce9af254d6ff29"} Dec 03 13:07:12 crc kubenswrapper[5052]: I1203 13:07:12.154627 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-wnkqw" Dec 03 13:07:12 crc kubenswrapper[5052]: I1203 13:07:12.177036 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-wnkqw" podStartSLOduration=3.17701693 podStartE2EDuration="3.17701693s" podCreationTimestamp="2025-12-03 13:07:09 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 13:07:12.173754907 +0000 UTC m=+6422.761996671" watchObservedRunningTime="2025-12-03 13:07:12.17701693 +0000 UTC m=+6422.765258694" Dec 03 13:07:12 crc kubenswrapper[5052]: I1203 13:07:12.673303 5052 scope.go:117] "RemoveContainer" containerID="dbc100c8081ea1a2f5487429cc9d67de6bc8ba11282b64f248c71275375b0e16" Dec 03 13:07:12 crc kubenswrapper[5052]: E1203 13:07:12.673530 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 13:07:18 crc kubenswrapper[5052]: I1203 13:07:18.225437 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["crc-storage/crc-storage-crc-v74gs"] Dec 03 13:07:18 crc kubenswrapper[5052]: I1203 13:07:18.236060 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["crc-storage/crc-storage-crc-v74gs"] Dec 03 13:07:18 crc kubenswrapper[5052]: I1203 13:07:18.385925 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["crc-storage/crc-storage-crc-4nhrr"] Dec 03 13:07:18 crc kubenswrapper[5052]: I1203 13:07:18.387005 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="crc-storage/crc-storage-crc-4nhrr" Dec 03 13:07:18 crc kubenswrapper[5052]: I1203 13:07:18.393098 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"crc-storage"/"openshift-service-ca.crt" Dec 03 13:07:18 crc kubenswrapper[5052]: I1203 13:07:18.393214 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"crc-storage"/"crc-storage" Dec 03 13:07:18 crc kubenswrapper[5052]: I1203 13:07:18.393487 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"crc-storage"/"kube-root-ca.crt" Dec 03 13:07:18 crc kubenswrapper[5052]: I1203 13:07:18.393591 5052 reflector.go:368] Caches populated for *v1.Secret from object-"crc-storage"/"crc-storage-dockercfg-mf76g" Dec 03 13:07:18 crc kubenswrapper[5052]: I1203 13:07:18.396675 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["crc-storage/crc-storage-crc-4nhrr"] Dec 03 13:07:18 crc kubenswrapper[5052]: I1203 13:07:18.504039 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5jkvq\" (UniqueName: \"kubernetes.io/projected/f8c70421-29c1-41f1-927d-fb922e5bd88b-kube-api-access-5jkvq\") pod \"crc-storage-crc-4nhrr\" (UID: \"f8c70421-29c1-41f1-927d-fb922e5bd88b\") " pod="crc-storage/crc-storage-crc-4nhrr" Dec 03 13:07:18 crc kubenswrapper[5052]: I1203 13:07:18.504108 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/f8c70421-29c1-41f1-927d-fb922e5bd88b-crc-storage\") pod \"crc-storage-crc-4nhrr\" (UID: \"f8c70421-29c1-41f1-927d-fb922e5bd88b\") " pod="crc-storage/crc-storage-crc-4nhrr" Dec 03 13:07:18 crc kubenswrapper[5052]: I1203 13:07:18.504236 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/f8c70421-29c1-41f1-927d-fb922e5bd88b-node-mnt\") pod \"crc-storage-crc-4nhrr\" (UID: \"f8c70421-29c1-41f1-927d-fb922e5bd88b\") " pod="crc-storage/crc-storage-crc-4nhrr" Dec 03 13:07:18 crc kubenswrapper[5052]: I1203 13:07:18.605903 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/f8c70421-29c1-41f1-927d-fb922e5bd88b-node-mnt\") pod \"crc-storage-crc-4nhrr\" (UID: \"f8c70421-29c1-41f1-927d-fb922e5bd88b\") " pod="crc-storage/crc-storage-crc-4nhrr" Dec 03 13:07:18 crc kubenswrapper[5052]: I1203 13:07:18.605967 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-5jkvq\" (UniqueName: \"kubernetes.io/projected/f8c70421-29c1-41f1-927d-fb922e5bd88b-kube-api-access-5jkvq\") pod \"crc-storage-crc-4nhrr\" (UID: \"f8c70421-29c1-41f1-927d-fb922e5bd88b\") " pod="crc-storage/crc-storage-crc-4nhrr" Dec 03 13:07:18 crc kubenswrapper[5052]: I1203 13:07:18.606029 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/f8c70421-29c1-41f1-927d-fb922e5bd88b-crc-storage\") pod \"crc-storage-crc-4nhrr\" (UID: \"f8c70421-29c1-41f1-927d-fb922e5bd88b\") " pod="crc-storage/crc-storage-crc-4nhrr" Dec 03 13:07:18 crc kubenswrapper[5052]: I1203 13:07:18.606345 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/f8c70421-29c1-41f1-927d-fb922e5bd88b-node-mnt\") pod \"crc-storage-crc-4nhrr\" (UID: \"f8c70421-29c1-41f1-927d-fb922e5bd88b\") " pod="crc-storage/crc-storage-crc-4nhrr" Dec 03 13:07:18 crc kubenswrapper[5052]: I1203 13:07:18.607191 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/f8c70421-29c1-41f1-927d-fb922e5bd88b-crc-storage\") pod \"crc-storage-crc-4nhrr\" (UID: \"f8c70421-29c1-41f1-927d-fb922e5bd88b\") " pod="crc-storage/crc-storage-crc-4nhrr" Dec 03 13:07:18 crc kubenswrapper[5052]: I1203 13:07:18.638587 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-5jkvq\" (UniqueName: \"kubernetes.io/projected/f8c70421-29c1-41f1-927d-fb922e5bd88b-kube-api-access-5jkvq\") pod \"crc-storage-crc-4nhrr\" (UID: \"f8c70421-29c1-41f1-927d-fb922e5bd88b\") " pod="crc-storage/crc-storage-crc-4nhrr" Dec 03 13:07:18 crc kubenswrapper[5052]: I1203 13:07:18.709306 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="crc-storage/crc-storage-crc-4nhrr" Dec 03 13:07:19 crc kubenswrapper[5052]: I1203 13:07:19.175098 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["crc-storage/crc-storage-crc-4nhrr"] Dec 03 13:07:19 crc kubenswrapper[5052]: I1203 13:07:19.213421 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="crc-storage/crc-storage-crc-4nhrr" event={"ID":"f8c70421-29c1-41f1-927d-fb922e5bd88b","Type":"ContainerStarted","Data":"7b5de1f0836d06c5666ba91795cb9ed76555e88e9402066f4ea15e6df4325bc4"} Dec 03 13:07:19 crc kubenswrapper[5052]: I1203 13:07:19.686368 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3eb2a1e9-07d5-4220-84b2-55c704ba4aae" path="/var/lib/kubelet/pods/3eb2a1e9-07d5-4220-84b2-55c704ba4aae/volumes" Dec 03 13:07:20 crc kubenswrapper[5052]: I1203 13:07:20.130708 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-wnkqw" Dec 03 13:07:20 crc kubenswrapper[5052]: I1203 13:07:20.220538 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-76cd9645f5-hrmzf"] Dec 03 13:07:20 crc kubenswrapper[5052]: I1203 13:07:20.220753 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-76cd9645f5-hrmzf" podUID="946e2885-6643-4b9e-931c-4c8b45fb0910" containerName="dnsmasq-dns" containerID="cri-o://096df24192ce1de338b52a0bf35e409355f5ed81a274ef42637003e28d70ee76" gracePeriod=10 Dec 03 13:07:20 crc kubenswrapper[5052]: I1203 13:07:20.236738 5052 generic.go:334] "Generic (PLEG): container finished" podID="f8c70421-29c1-41f1-927d-fb922e5bd88b" containerID="87a2769f559683c40211370d84da6bc23e30db34d7d335f2eb47ac47add4668d" exitCode=0 Dec 03 13:07:20 crc kubenswrapper[5052]: I1203 13:07:20.236781 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="crc-storage/crc-storage-crc-4nhrr" event={"ID":"f8c70421-29c1-41f1-927d-fb922e5bd88b","Type":"ContainerDied","Data":"87a2769f559683c40211370d84da6bc23e30db34d7d335f2eb47ac47add4668d"} Dec 03 13:07:20 crc kubenswrapper[5052]: I1203 13:07:20.705547 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-76cd9645f5-hrmzf" Dec 03 13:07:20 crc kubenswrapper[5052]: I1203 13:07:20.746219 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jqhml\" (UniqueName: \"kubernetes.io/projected/946e2885-6643-4b9e-931c-4c8b45fb0910-kube-api-access-jqhml\") pod \"946e2885-6643-4b9e-931c-4c8b45fb0910\" (UID: \"946e2885-6643-4b9e-931c-4c8b45fb0910\") " Dec 03 13:07:20 crc kubenswrapper[5052]: I1203 13:07:20.746619 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/946e2885-6643-4b9e-931c-4c8b45fb0910-config\") pod \"946e2885-6643-4b9e-931c-4c8b45fb0910\" (UID: \"946e2885-6643-4b9e-931c-4c8b45fb0910\") " Dec 03 13:07:20 crc kubenswrapper[5052]: I1203 13:07:20.746673 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"edpm-extramounts\" (UniqueName: \"kubernetes.io/configmap/946e2885-6643-4b9e-931c-4c8b45fb0910-edpm-extramounts\") pod \"946e2885-6643-4b9e-931c-4c8b45fb0910\" (UID: \"946e2885-6643-4b9e-931c-4c8b45fb0910\") " Dec 03 13:07:20 crc kubenswrapper[5052]: I1203 13:07:20.746818 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/946e2885-6643-4b9e-931c-4c8b45fb0910-dnsmasq-svc\") pod \"946e2885-6643-4b9e-931c-4c8b45fb0910\" (UID: \"946e2885-6643-4b9e-931c-4c8b45fb0910\") " Dec 03 13:07:20 crc kubenswrapper[5052]: I1203 13:07:20.758736 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/946e2885-6643-4b9e-931c-4c8b45fb0910-kube-api-access-jqhml" (OuterVolumeSpecName: "kube-api-access-jqhml") pod "946e2885-6643-4b9e-931c-4c8b45fb0910" (UID: "946e2885-6643-4b9e-931c-4c8b45fb0910"). InnerVolumeSpecName "kube-api-access-jqhml". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:07:20 crc kubenswrapper[5052]: I1203 13:07:20.788390 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/946e2885-6643-4b9e-931c-4c8b45fb0910-dnsmasq-svc" (OuterVolumeSpecName: "dnsmasq-svc") pod "946e2885-6643-4b9e-931c-4c8b45fb0910" (UID: "946e2885-6643-4b9e-931c-4c8b45fb0910"). InnerVolumeSpecName "dnsmasq-svc". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:07:20 crc kubenswrapper[5052]: I1203 13:07:20.792837 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/946e2885-6643-4b9e-931c-4c8b45fb0910-config" (OuterVolumeSpecName: "config") pod "946e2885-6643-4b9e-931c-4c8b45fb0910" (UID: "946e2885-6643-4b9e-931c-4c8b45fb0910"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:07:20 crc kubenswrapper[5052]: I1203 13:07:20.801371 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/946e2885-6643-4b9e-931c-4c8b45fb0910-edpm-extramounts" (OuterVolumeSpecName: "edpm-extramounts") pod "946e2885-6643-4b9e-931c-4c8b45fb0910" (UID: "946e2885-6643-4b9e-931c-4c8b45fb0910"). InnerVolumeSpecName "edpm-extramounts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:07:20 crc kubenswrapper[5052]: I1203 13:07:20.858410 5052 reconciler_common.go:293] "Volume detached for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/946e2885-6643-4b9e-931c-4c8b45fb0910-dnsmasq-svc\") on node \"crc\" DevicePath \"\"" Dec 03 13:07:20 crc kubenswrapper[5052]: I1203 13:07:20.858450 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-jqhml\" (UniqueName: \"kubernetes.io/projected/946e2885-6643-4b9e-931c-4c8b45fb0910-kube-api-access-jqhml\") on node \"crc\" DevicePath \"\"" Dec 03 13:07:20 crc kubenswrapper[5052]: I1203 13:07:20.858464 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/946e2885-6643-4b9e-931c-4c8b45fb0910-config\") on node \"crc\" DevicePath \"\"" Dec 03 13:07:20 crc kubenswrapper[5052]: I1203 13:07:20.858473 5052 reconciler_common.go:293] "Volume detached for volume \"edpm-extramounts\" (UniqueName: \"kubernetes.io/configmap/946e2885-6643-4b9e-931c-4c8b45fb0910-edpm-extramounts\") on node \"crc\" DevicePath \"\"" Dec 03 13:07:21 crc kubenswrapper[5052]: I1203 13:07:21.247959 5052 generic.go:334] "Generic (PLEG): container finished" podID="946e2885-6643-4b9e-931c-4c8b45fb0910" containerID="096df24192ce1de338b52a0bf35e409355f5ed81a274ef42637003e28d70ee76" exitCode=0 Dec 03 13:07:21 crc kubenswrapper[5052]: I1203 13:07:21.248014 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-76cd9645f5-hrmzf" event={"ID":"946e2885-6643-4b9e-931c-4c8b45fb0910","Type":"ContainerDied","Data":"096df24192ce1de338b52a0bf35e409355f5ed81a274ef42637003e28d70ee76"} Dec 03 13:07:21 crc kubenswrapper[5052]: I1203 13:07:21.248042 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-76cd9645f5-hrmzf" Dec 03 13:07:21 crc kubenswrapper[5052]: I1203 13:07:21.248074 5052 scope.go:117] "RemoveContainer" containerID="096df24192ce1de338b52a0bf35e409355f5ed81a274ef42637003e28d70ee76" Dec 03 13:07:21 crc kubenswrapper[5052]: I1203 13:07:21.248059 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-76cd9645f5-hrmzf" event={"ID":"946e2885-6643-4b9e-931c-4c8b45fb0910","Type":"ContainerDied","Data":"f89d7c95588eb98dac05c70d1bb8ad5cc0538f4a253903a685117438c781fc51"} Dec 03 13:07:21 crc kubenswrapper[5052]: I1203 13:07:21.274746 5052 scope.go:117] "RemoveContainer" containerID="2b7ead942a36e5ffede2d552fd91ac0a9dc6c7efa4575c46f4c19659c703d70d" Dec 03 13:07:21 crc kubenswrapper[5052]: I1203 13:07:21.284926 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-76cd9645f5-hrmzf"] Dec 03 13:07:21 crc kubenswrapper[5052]: I1203 13:07:21.293399 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-76cd9645f5-hrmzf"] Dec 03 13:07:21 crc kubenswrapper[5052]: I1203 13:07:21.502718 5052 scope.go:117] "RemoveContainer" containerID="096df24192ce1de338b52a0bf35e409355f5ed81a274ef42637003e28d70ee76" Dec 03 13:07:21 crc kubenswrapper[5052]: E1203 13:07:21.503379 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"096df24192ce1de338b52a0bf35e409355f5ed81a274ef42637003e28d70ee76\": container with ID starting with 096df24192ce1de338b52a0bf35e409355f5ed81a274ef42637003e28d70ee76 not found: ID does not exist" containerID="096df24192ce1de338b52a0bf35e409355f5ed81a274ef42637003e28d70ee76" Dec 03 13:07:21 crc kubenswrapper[5052]: I1203 13:07:21.503415 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"096df24192ce1de338b52a0bf35e409355f5ed81a274ef42637003e28d70ee76"} err="failed to get container status \"096df24192ce1de338b52a0bf35e409355f5ed81a274ef42637003e28d70ee76\": rpc error: code = NotFound desc = could not find container \"096df24192ce1de338b52a0bf35e409355f5ed81a274ef42637003e28d70ee76\": container with ID starting with 096df24192ce1de338b52a0bf35e409355f5ed81a274ef42637003e28d70ee76 not found: ID does not exist" Dec 03 13:07:21 crc kubenswrapper[5052]: I1203 13:07:21.503465 5052 scope.go:117] "RemoveContainer" containerID="2b7ead942a36e5ffede2d552fd91ac0a9dc6c7efa4575c46f4c19659c703d70d" Dec 03 13:07:21 crc kubenswrapper[5052]: E1203 13:07:21.503910 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2b7ead942a36e5ffede2d552fd91ac0a9dc6c7efa4575c46f4c19659c703d70d\": container with ID starting with 2b7ead942a36e5ffede2d552fd91ac0a9dc6c7efa4575c46f4c19659c703d70d not found: ID does not exist" containerID="2b7ead942a36e5ffede2d552fd91ac0a9dc6c7efa4575c46f4c19659c703d70d" Dec 03 13:07:21 crc kubenswrapper[5052]: I1203 13:07:21.503937 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2b7ead942a36e5ffede2d552fd91ac0a9dc6c7efa4575c46f4c19659c703d70d"} err="failed to get container status \"2b7ead942a36e5ffede2d552fd91ac0a9dc6c7efa4575c46f4c19659c703d70d\": rpc error: code = NotFound desc = could not find container \"2b7ead942a36e5ffede2d552fd91ac0a9dc6c7efa4575c46f4c19659c703d70d\": container with ID starting with 2b7ead942a36e5ffede2d552fd91ac0a9dc6c7efa4575c46f4c19659c703d70d not found: ID does not exist" Dec 03 13:07:21 crc kubenswrapper[5052]: I1203 13:07:21.667785 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="crc-storage/crc-storage-crc-4nhrr" Dec 03 13:07:21 crc kubenswrapper[5052]: I1203 13:07:21.687425 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="946e2885-6643-4b9e-931c-4c8b45fb0910" path="/var/lib/kubelet/pods/946e2885-6643-4b9e-931c-4c8b45fb0910/volumes" Dec 03 13:07:21 crc kubenswrapper[5052]: I1203 13:07:21.808385 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/f8c70421-29c1-41f1-927d-fb922e5bd88b-crc-storage\") pod \"f8c70421-29c1-41f1-927d-fb922e5bd88b\" (UID: \"f8c70421-29c1-41f1-927d-fb922e5bd88b\") " Dec 03 13:07:21 crc kubenswrapper[5052]: I1203 13:07:21.808441 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5jkvq\" (UniqueName: \"kubernetes.io/projected/f8c70421-29c1-41f1-927d-fb922e5bd88b-kube-api-access-5jkvq\") pod \"f8c70421-29c1-41f1-927d-fb922e5bd88b\" (UID: \"f8c70421-29c1-41f1-927d-fb922e5bd88b\") " Dec 03 13:07:21 crc kubenswrapper[5052]: I1203 13:07:21.808469 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/f8c70421-29c1-41f1-927d-fb922e5bd88b-node-mnt\") pod \"f8c70421-29c1-41f1-927d-fb922e5bd88b\" (UID: \"f8c70421-29c1-41f1-927d-fb922e5bd88b\") " Dec 03 13:07:21 crc kubenswrapper[5052]: I1203 13:07:21.809275 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/f8c70421-29c1-41f1-927d-fb922e5bd88b-node-mnt" (OuterVolumeSpecName: "node-mnt") pod "f8c70421-29c1-41f1-927d-fb922e5bd88b" (UID: "f8c70421-29c1-41f1-927d-fb922e5bd88b"). InnerVolumeSpecName "node-mnt". PluginName "kubernetes.io/host-path", VolumeGidValue "" Dec 03 13:07:21 crc kubenswrapper[5052]: I1203 13:07:21.814821 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f8c70421-29c1-41f1-927d-fb922e5bd88b-kube-api-access-5jkvq" (OuterVolumeSpecName: "kube-api-access-5jkvq") pod "f8c70421-29c1-41f1-927d-fb922e5bd88b" (UID: "f8c70421-29c1-41f1-927d-fb922e5bd88b"). InnerVolumeSpecName "kube-api-access-5jkvq". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:07:21 crc kubenswrapper[5052]: I1203 13:07:21.831643 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f8c70421-29c1-41f1-927d-fb922e5bd88b-crc-storage" (OuterVolumeSpecName: "crc-storage") pod "f8c70421-29c1-41f1-927d-fb922e5bd88b" (UID: "f8c70421-29c1-41f1-927d-fb922e5bd88b"). InnerVolumeSpecName "crc-storage". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:07:21 crc kubenswrapper[5052]: I1203 13:07:21.910763 5052 reconciler_common.go:293] "Volume detached for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/f8c70421-29c1-41f1-927d-fb922e5bd88b-crc-storage\") on node \"crc\" DevicePath \"\"" Dec 03 13:07:21 crc kubenswrapper[5052]: I1203 13:07:21.910816 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-5jkvq\" (UniqueName: \"kubernetes.io/projected/f8c70421-29c1-41f1-927d-fb922e5bd88b-kube-api-access-5jkvq\") on node \"crc\" DevicePath \"\"" Dec 03 13:07:21 crc kubenswrapper[5052]: I1203 13:07:21.910830 5052 reconciler_common.go:293] "Volume detached for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/f8c70421-29c1-41f1-927d-fb922e5bd88b-node-mnt\") on node \"crc\" DevicePath \"\"" Dec 03 13:07:22 crc kubenswrapper[5052]: I1203 13:07:22.273766 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="crc-storage/crc-storage-crc-4nhrr" event={"ID":"f8c70421-29c1-41f1-927d-fb922e5bd88b","Type":"ContainerDied","Data":"7b5de1f0836d06c5666ba91795cb9ed76555e88e9402066f4ea15e6df4325bc4"} Dec 03 13:07:22 crc kubenswrapper[5052]: I1203 13:07:22.273814 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7b5de1f0836d06c5666ba91795cb9ed76555e88e9402066f4ea15e6df4325bc4" Dec 03 13:07:22 crc kubenswrapper[5052]: I1203 13:07:22.273831 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="crc-storage/crc-storage-crc-4nhrr" Dec 03 13:07:25 crc kubenswrapper[5052]: I1203 13:07:25.447696 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["crc-storage/crc-storage-crc-4nhrr"] Dec 03 13:07:25 crc kubenswrapper[5052]: I1203 13:07:25.454794 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["crc-storage/crc-storage-crc-4nhrr"] Dec 03 13:07:25 crc kubenswrapper[5052]: I1203 13:07:25.629378 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["crc-storage/crc-storage-crc-rzt2l"] Dec 03 13:07:25 crc kubenswrapper[5052]: E1203 13:07:25.629698 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="946e2885-6643-4b9e-931c-4c8b45fb0910" containerName="init" Dec 03 13:07:25 crc kubenswrapper[5052]: I1203 13:07:25.629716 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="946e2885-6643-4b9e-931c-4c8b45fb0910" containerName="init" Dec 03 13:07:25 crc kubenswrapper[5052]: E1203 13:07:25.629748 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="946e2885-6643-4b9e-931c-4c8b45fb0910" containerName="dnsmasq-dns" Dec 03 13:07:25 crc kubenswrapper[5052]: I1203 13:07:25.629756 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="946e2885-6643-4b9e-931c-4c8b45fb0910" containerName="dnsmasq-dns" Dec 03 13:07:25 crc kubenswrapper[5052]: E1203 13:07:25.629785 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="f8c70421-29c1-41f1-927d-fb922e5bd88b" containerName="storage" Dec 03 13:07:25 crc kubenswrapper[5052]: I1203 13:07:25.629794 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="f8c70421-29c1-41f1-927d-fb922e5bd88b" containerName="storage" Dec 03 13:07:25 crc kubenswrapper[5052]: I1203 13:07:25.629995 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="f8c70421-29c1-41f1-927d-fb922e5bd88b" containerName="storage" Dec 03 13:07:25 crc kubenswrapper[5052]: I1203 13:07:25.630026 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="946e2885-6643-4b9e-931c-4c8b45fb0910" containerName="dnsmasq-dns" Dec 03 13:07:25 crc kubenswrapper[5052]: I1203 13:07:25.630585 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="crc-storage/crc-storage-crc-rzt2l" Dec 03 13:07:25 crc kubenswrapper[5052]: I1203 13:07:25.633454 5052 reflector.go:368] Caches populated for *v1.Secret from object-"crc-storage"/"crc-storage-dockercfg-mf76g" Dec 03 13:07:25 crc kubenswrapper[5052]: I1203 13:07:25.635744 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"crc-storage"/"openshift-service-ca.crt" Dec 03 13:07:25 crc kubenswrapper[5052]: I1203 13:07:25.635893 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"crc-storage"/"kube-root-ca.crt" Dec 03 13:07:25 crc kubenswrapper[5052]: I1203 13:07:25.635936 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"crc-storage"/"crc-storage" Dec 03 13:07:25 crc kubenswrapper[5052]: I1203 13:07:25.643043 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["crc-storage/crc-storage-crc-rzt2l"] Dec 03 13:07:25 crc kubenswrapper[5052]: I1203 13:07:25.682069 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f8c70421-29c1-41f1-927d-fb922e5bd88b" path="/var/lib/kubelet/pods/f8c70421-29c1-41f1-927d-fb922e5bd88b/volumes" Dec 03 13:07:25 crc kubenswrapper[5052]: I1203 13:07:25.684958 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/46a3997f-7536-4c23-a718-2e891bcbbd6f-crc-storage\") pod \"crc-storage-crc-rzt2l\" (UID: \"46a3997f-7536-4c23-a718-2e891bcbbd6f\") " pod="crc-storage/crc-storage-crc-rzt2l" Dec 03 13:07:25 crc kubenswrapper[5052]: I1203 13:07:25.685082 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/46a3997f-7536-4c23-a718-2e891bcbbd6f-node-mnt\") pod \"crc-storage-crc-rzt2l\" (UID: \"46a3997f-7536-4c23-a718-2e891bcbbd6f\") " pod="crc-storage/crc-storage-crc-rzt2l" Dec 03 13:07:25 crc kubenswrapper[5052]: I1203 13:07:25.685262 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4672v\" (UniqueName: \"kubernetes.io/projected/46a3997f-7536-4c23-a718-2e891bcbbd6f-kube-api-access-4672v\") pod \"crc-storage-crc-rzt2l\" (UID: \"46a3997f-7536-4c23-a718-2e891bcbbd6f\") " pod="crc-storage/crc-storage-crc-rzt2l" Dec 03 13:07:25 crc kubenswrapper[5052]: I1203 13:07:25.787001 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/46a3997f-7536-4c23-a718-2e891bcbbd6f-crc-storage\") pod \"crc-storage-crc-rzt2l\" (UID: \"46a3997f-7536-4c23-a718-2e891bcbbd6f\") " pod="crc-storage/crc-storage-crc-rzt2l" Dec 03 13:07:25 crc kubenswrapper[5052]: I1203 13:07:25.787091 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/46a3997f-7536-4c23-a718-2e891bcbbd6f-node-mnt\") pod \"crc-storage-crc-rzt2l\" (UID: \"46a3997f-7536-4c23-a718-2e891bcbbd6f\") " pod="crc-storage/crc-storage-crc-rzt2l" Dec 03 13:07:25 crc kubenswrapper[5052]: I1203 13:07:25.787158 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-4672v\" (UniqueName: \"kubernetes.io/projected/46a3997f-7536-4c23-a718-2e891bcbbd6f-kube-api-access-4672v\") pod \"crc-storage-crc-rzt2l\" (UID: \"46a3997f-7536-4c23-a718-2e891bcbbd6f\") " pod="crc-storage/crc-storage-crc-rzt2l" Dec 03 13:07:25 crc kubenswrapper[5052]: I1203 13:07:25.787913 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/46a3997f-7536-4c23-a718-2e891bcbbd6f-node-mnt\") pod \"crc-storage-crc-rzt2l\" (UID: \"46a3997f-7536-4c23-a718-2e891bcbbd6f\") " pod="crc-storage/crc-storage-crc-rzt2l" Dec 03 13:07:25 crc kubenswrapper[5052]: I1203 13:07:25.788603 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/46a3997f-7536-4c23-a718-2e891bcbbd6f-crc-storage\") pod \"crc-storage-crc-rzt2l\" (UID: \"46a3997f-7536-4c23-a718-2e891bcbbd6f\") " pod="crc-storage/crc-storage-crc-rzt2l" Dec 03 13:07:25 crc kubenswrapper[5052]: I1203 13:07:25.811476 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-4672v\" (UniqueName: \"kubernetes.io/projected/46a3997f-7536-4c23-a718-2e891bcbbd6f-kube-api-access-4672v\") pod \"crc-storage-crc-rzt2l\" (UID: \"46a3997f-7536-4c23-a718-2e891bcbbd6f\") " pod="crc-storage/crc-storage-crc-rzt2l" Dec 03 13:07:25 crc kubenswrapper[5052]: I1203 13:07:25.951565 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="crc-storage/crc-storage-crc-rzt2l" Dec 03 13:07:26 crc kubenswrapper[5052]: I1203 13:07:26.378120 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["crc-storage/crc-storage-crc-rzt2l"] Dec 03 13:07:27 crc kubenswrapper[5052]: I1203 13:07:27.337317 5052 generic.go:334] "Generic (PLEG): container finished" podID="46a3997f-7536-4c23-a718-2e891bcbbd6f" containerID="a866cc5f3fce03eb0ddb16849ed57f2f2fe63a3b094aba775cd1bfddebfc177c" exitCode=0 Dec 03 13:07:27 crc kubenswrapper[5052]: I1203 13:07:27.337363 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="crc-storage/crc-storage-crc-rzt2l" event={"ID":"46a3997f-7536-4c23-a718-2e891bcbbd6f","Type":"ContainerDied","Data":"a866cc5f3fce03eb0ddb16849ed57f2f2fe63a3b094aba775cd1bfddebfc177c"} Dec 03 13:07:27 crc kubenswrapper[5052]: I1203 13:07:27.337407 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="crc-storage/crc-storage-crc-rzt2l" event={"ID":"46a3997f-7536-4c23-a718-2e891bcbbd6f","Type":"ContainerStarted","Data":"8b41e772a01d9158a934dda75106080ff9af3c0d1208ec3b142d0e25c412b620"} Dec 03 13:07:27 crc kubenswrapper[5052]: I1203 13:07:27.672681 5052 scope.go:117] "RemoveContainer" containerID="dbc100c8081ea1a2f5487429cc9d67de6bc8ba11282b64f248c71275375b0e16" Dec 03 13:07:27 crc kubenswrapper[5052]: E1203 13:07:27.673193 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 13:07:28 crc kubenswrapper[5052]: I1203 13:07:28.601241 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="crc-storage/crc-storage-crc-rzt2l" Dec 03 13:07:28 crc kubenswrapper[5052]: I1203 13:07:28.732371 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4672v\" (UniqueName: \"kubernetes.io/projected/46a3997f-7536-4c23-a718-2e891bcbbd6f-kube-api-access-4672v\") pod \"46a3997f-7536-4c23-a718-2e891bcbbd6f\" (UID: \"46a3997f-7536-4c23-a718-2e891bcbbd6f\") " Dec 03 13:07:28 crc kubenswrapper[5052]: I1203 13:07:28.732581 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/46a3997f-7536-4c23-a718-2e891bcbbd6f-crc-storage\") pod \"46a3997f-7536-4c23-a718-2e891bcbbd6f\" (UID: \"46a3997f-7536-4c23-a718-2e891bcbbd6f\") " Dec 03 13:07:28 crc kubenswrapper[5052]: I1203 13:07:28.732647 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/46a3997f-7536-4c23-a718-2e891bcbbd6f-node-mnt\") pod \"46a3997f-7536-4c23-a718-2e891bcbbd6f\" (UID: \"46a3997f-7536-4c23-a718-2e891bcbbd6f\") " Dec 03 13:07:28 crc kubenswrapper[5052]: I1203 13:07:28.733266 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/46a3997f-7536-4c23-a718-2e891bcbbd6f-node-mnt" (OuterVolumeSpecName: "node-mnt") pod "46a3997f-7536-4c23-a718-2e891bcbbd6f" (UID: "46a3997f-7536-4c23-a718-2e891bcbbd6f"). InnerVolumeSpecName "node-mnt". PluginName "kubernetes.io/host-path", VolumeGidValue "" Dec 03 13:07:28 crc kubenswrapper[5052]: I1203 13:07:28.734667 5052 reconciler_common.go:293] "Volume detached for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/46a3997f-7536-4c23-a718-2e891bcbbd6f-node-mnt\") on node \"crc\" DevicePath \"\"" Dec 03 13:07:28 crc kubenswrapper[5052]: I1203 13:07:28.739607 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/46a3997f-7536-4c23-a718-2e891bcbbd6f-kube-api-access-4672v" (OuterVolumeSpecName: "kube-api-access-4672v") pod "46a3997f-7536-4c23-a718-2e891bcbbd6f" (UID: "46a3997f-7536-4c23-a718-2e891bcbbd6f"). InnerVolumeSpecName "kube-api-access-4672v". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:07:28 crc kubenswrapper[5052]: I1203 13:07:28.755687 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/46a3997f-7536-4c23-a718-2e891bcbbd6f-crc-storage" (OuterVolumeSpecName: "crc-storage") pod "46a3997f-7536-4c23-a718-2e891bcbbd6f" (UID: "46a3997f-7536-4c23-a718-2e891bcbbd6f"). InnerVolumeSpecName "crc-storage". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:07:28 crc kubenswrapper[5052]: I1203 13:07:28.836167 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-4672v\" (UniqueName: \"kubernetes.io/projected/46a3997f-7536-4c23-a718-2e891bcbbd6f-kube-api-access-4672v\") on node \"crc\" DevicePath \"\"" Dec 03 13:07:28 crc kubenswrapper[5052]: I1203 13:07:28.836561 5052 reconciler_common.go:293] "Volume detached for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/46a3997f-7536-4c23-a718-2e891bcbbd6f-crc-storage\") on node \"crc\" DevicePath \"\"" Dec 03 13:07:29 crc kubenswrapper[5052]: I1203 13:07:29.358049 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="crc-storage/crc-storage-crc-rzt2l" event={"ID":"46a3997f-7536-4c23-a718-2e891bcbbd6f","Type":"ContainerDied","Data":"8b41e772a01d9158a934dda75106080ff9af3c0d1208ec3b142d0e25c412b620"} Dec 03 13:07:29 crc kubenswrapper[5052]: I1203 13:07:29.358089 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8b41e772a01d9158a934dda75106080ff9af3c0d1208ec3b142d0e25c412b620" Dec 03 13:07:29 crc kubenswrapper[5052]: I1203 13:07:29.358127 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="crc-storage/crc-storage-crc-rzt2l" Dec 03 13:07:32 crc kubenswrapper[5052]: I1203 13:07:32.261306 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-58854494b5-pjqqv"] Dec 03 13:07:32 crc kubenswrapper[5052]: E1203 13:07:32.261728 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="46a3997f-7536-4c23-a718-2e891bcbbd6f" containerName="storage" Dec 03 13:07:32 crc kubenswrapper[5052]: I1203 13:07:32.261746 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="46a3997f-7536-4c23-a718-2e891bcbbd6f" containerName="storage" Dec 03 13:07:32 crc kubenswrapper[5052]: I1203 13:07:32.261945 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="46a3997f-7536-4c23-a718-2e891bcbbd6f" containerName="storage" Dec 03 13:07:32 crc kubenswrapper[5052]: I1203 13:07:32.262965 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-58854494b5-pjqqv" Dec 03 13:07:32 crc kubenswrapper[5052]: I1203 13:07:32.265502 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"openstack-edpm-multinode" Dec 03 13:07:32 crc kubenswrapper[5052]: I1203 13:07:32.295737 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-58854494b5-pjqqv"] Dec 03 13:07:32 crc kubenswrapper[5052]: I1203 13:07:32.397832 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/f598fbaf-216d-45d5-a432-e45ed1406aea-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-58854494b5-pjqqv\" (UID: \"f598fbaf-216d-45d5-a432-e45ed1406aea\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-58854494b5-pjqqv" Dec 03 13:07:32 crc kubenswrapper[5052]: I1203 13:07:32.397896 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f598fbaf-216d-45d5-a432-e45ed1406aea-config\") pod \"dnsmasq-dnsmasq-58854494b5-pjqqv\" (UID: \"f598fbaf-216d-45d5-a432-e45ed1406aea\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-58854494b5-pjqqv" Dec 03 13:07:32 crc kubenswrapper[5052]: I1203 13:07:32.397952 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openstack-edpm-multinode\" (UniqueName: \"kubernetes.io/configmap/f598fbaf-216d-45d5-a432-e45ed1406aea-openstack-edpm-multinode\") pod \"dnsmasq-dnsmasq-58854494b5-pjqqv\" (UID: \"f598fbaf-216d-45d5-a432-e45ed1406aea\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-58854494b5-pjqqv" Dec 03 13:07:32 crc kubenswrapper[5052]: I1203 13:07:32.398084 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l5zzt\" (UniqueName: \"kubernetes.io/projected/f598fbaf-216d-45d5-a432-e45ed1406aea-kube-api-access-l5zzt\") pod \"dnsmasq-dnsmasq-58854494b5-pjqqv\" (UID: \"f598fbaf-216d-45d5-a432-e45ed1406aea\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-58854494b5-pjqqv" Dec 03 13:07:32 crc kubenswrapper[5052]: I1203 13:07:32.499092 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-l5zzt\" (UniqueName: \"kubernetes.io/projected/f598fbaf-216d-45d5-a432-e45ed1406aea-kube-api-access-l5zzt\") pod \"dnsmasq-dnsmasq-58854494b5-pjqqv\" (UID: \"f598fbaf-216d-45d5-a432-e45ed1406aea\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-58854494b5-pjqqv" Dec 03 13:07:32 crc kubenswrapper[5052]: I1203 13:07:32.499172 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/f598fbaf-216d-45d5-a432-e45ed1406aea-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-58854494b5-pjqqv\" (UID: \"f598fbaf-216d-45d5-a432-e45ed1406aea\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-58854494b5-pjqqv" Dec 03 13:07:32 crc kubenswrapper[5052]: I1203 13:07:32.499200 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f598fbaf-216d-45d5-a432-e45ed1406aea-config\") pod \"dnsmasq-dnsmasq-58854494b5-pjqqv\" (UID: \"f598fbaf-216d-45d5-a432-e45ed1406aea\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-58854494b5-pjqqv" Dec 03 13:07:32 crc kubenswrapper[5052]: I1203 13:07:32.499223 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"openstack-edpm-multinode\" (UniqueName: \"kubernetes.io/configmap/f598fbaf-216d-45d5-a432-e45ed1406aea-openstack-edpm-multinode\") pod \"dnsmasq-dnsmasq-58854494b5-pjqqv\" (UID: \"f598fbaf-216d-45d5-a432-e45ed1406aea\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-58854494b5-pjqqv" Dec 03 13:07:32 crc kubenswrapper[5052]: I1203 13:07:32.500121 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"openstack-edpm-multinode\" (UniqueName: \"kubernetes.io/configmap/f598fbaf-216d-45d5-a432-e45ed1406aea-openstack-edpm-multinode\") pod \"dnsmasq-dnsmasq-58854494b5-pjqqv\" (UID: \"f598fbaf-216d-45d5-a432-e45ed1406aea\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-58854494b5-pjqqv" Dec 03 13:07:32 crc kubenswrapper[5052]: I1203 13:07:32.500265 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/f598fbaf-216d-45d5-a432-e45ed1406aea-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-58854494b5-pjqqv\" (UID: \"f598fbaf-216d-45d5-a432-e45ed1406aea\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-58854494b5-pjqqv" Dec 03 13:07:32 crc kubenswrapper[5052]: I1203 13:07:32.500293 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f598fbaf-216d-45d5-a432-e45ed1406aea-config\") pod \"dnsmasq-dnsmasq-58854494b5-pjqqv\" (UID: \"f598fbaf-216d-45d5-a432-e45ed1406aea\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-58854494b5-pjqqv" Dec 03 13:07:32 crc kubenswrapper[5052]: I1203 13:07:32.517186 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-l5zzt\" (UniqueName: \"kubernetes.io/projected/f598fbaf-216d-45d5-a432-e45ed1406aea-kube-api-access-l5zzt\") pod \"dnsmasq-dnsmasq-58854494b5-pjqqv\" (UID: \"f598fbaf-216d-45d5-a432-e45ed1406aea\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-58854494b5-pjqqv" Dec 03 13:07:32 crc kubenswrapper[5052]: I1203 13:07:32.581067 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-58854494b5-pjqqv" Dec 03 13:07:33 crc kubenswrapper[5052]: I1203 13:07:33.077946 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-58854494b5-pjqqv"] Dec 03 13:07:33 crc kubenswrapper[5052]: W1203 13:07:33.088062 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf598fbaf_216d_45d5_a432_e45ed1406aea.slice/crio-e87e4725839f5d90d6577c1e4c69c171e71322fb9c385bed589a96502b54d1cc WatchSource:0}: Error finding container e87e4725839f5d90d6577c1e4c69c171e71322fb9c385bed589a96502b54d1cc: Status 404 returned error can't find the container with id e87e4725839f5d90d6577c1e4c69c171e71322fb9c385bed589a96502b54d1cc Dec 03 13:07:33 crc kubenswrapper[5052]: I1203 13:07:33.399917 5052 generic.go:334] "Generic (PLEG): container finished" podID="f598fbaf-216d-45d5-a432-e45ed1406aea" containerID="57d7b31bfaf447f6238101640b82c00377ae0af452d9a64800393470d2516956" exitCode=0 Dec 03 13:07:33 crc kubenswrapper[5052]: I1203 13:07:33.400021 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-58854494b5-pjqqv" event={"ID":"f598fbaf-216d-45d5-a432-e45ed1406aea","Type":"ContainerDied","Data":"57d7b31bfaf447f6238101640b82c00377ae0af452d9a64800393470d2516956"} Dec 03 13:07:33 crc kubenswrapper[5052]: I1203 13:07:33.400224 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-58854494b5-pjqqv" event={"ID":"f598fbaf-216d-45d5-a432-e45ed1406aea","Type":"ContainerStarted","Data":"e87e4725839f5d90d6577c1e4c69c171e71322fb9c385bed589a96502b54d1cc"} Dec 03 13:07:34 crc kubenswrapper[5052]: I1203 13:07:34.410966 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-58854494b5-pjqqv" event={"ID":"f598fbaf-216d-45d5-a432-e45ed1406aea","Type":"ContainerStarted","Data":"d53d8576eea3c007e4aa4de14d49102afa730cd6f654fa476363f15a72a71c8d"} Dec 03 13:07:34 crc kubenswrapper[5052]: I1203 13:07:34.411409 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-58854494b5-pjqqv" Dec 03 13:07:34 crc kubenswrapper[5052]: I1203 13:07:34.433621 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-58854494b5-pjqqv" podStartSLOduration=2.433597399 podStartE2EDuration="2.433597399s" podCreationTimestamp="2025-12-03 13:07:32 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 13:07:34.424931361 +0000 UTC m=+6445.013173155" watchObservedRunningTime="2025-12-03 13:07:34.433597399 +0000 UTC m=+6445.021839173" Dec 03 13:07:41 crc kubenswrapper[5052]: I1203 13:07:41.674915 5052 scope.go:117] "RemoveContainer" containerID="dbc100c8081ea1a2f5487429cc9d67de6bc8ba11282b64f248c71275375b0e16" Dec 03 13:07:41 crc kubenswrapper[5052]: E1203 13:07:41.675593 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 13:07:42 crc kubenswrapper[5052]: I1203 13:07:42.421381 5052 scope.go:117] "RemoveContainer" containerID="64c56e75b7425465f6d086bf57aa1a1f81c6dd61a33a86f60578eb2ddf258c12" Dec 03 13:07:42 crc kubenswrapper[5052]: I1203 13:07:42.468824 5052 scope.go:117] "RemoveContainer" containerID="c9486e62f27a2d8c73beccf2e07b988ff983ae8ccc8c27855fbf685b2e59bbdc" Dec 03 13:07:42 crc kubenswrapper[5052]: I1203 13:07:42.509689 5052 scope.go:117] "RemoveContainer" containerID="aeb2368acbb7a77c33586bb17253462cc38025c7383d25dc21d52cfee3b961e1" Dec 03 13:07:42 crc kubenswrapper[5052]: I1203 13:07:42.533276 5052 scope.go:117] "RemoveContainer" containerID="c2ebf07b50f31375a46146921445745cfa84a1f04033cfe3691aef9bc2a5e576" Dec 03 13:07:42 crc kubenswrapper[5052]: I1203 13:07:42.569310 5052 scope.go:117] "RemoveContainer" containerID="adbfb99d0ce82b3e624c45e32aac97991f21de1f08612bf6ef1b89ddee3aafcd" Dec 03 13:07:42 crc kubenswrapper[5052]: I1203 13:07:42.583793 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-58854494b5-pjqqv" Dec 03 13:07:42 crc kubenswrapper[5052]: I1203 13:07:42.613655 5052 scope.go:117] "RemoveContainer" containerID="67b390a0268304ca7c8077e64e8d2a96fd60a9d43f440830c178191f3ee252a3" Dec 03 13:07:42 crc kubenswrapper[5052]: I1203 13:07:42.650978 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-wnkqw"] Dec 03 13:07:42 crc kubenswrapper[5052]: I1203 13:07:42.651493 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-wnkqw" podUID="ef40807a-35ce-4af5-9996-bc2f5b2fa3e4" containerName="dnsmasq-dns" containerID="cri-o://ec1bccc641595bfeab1754722642ceced12f7f3334c65a4375ce9af254d6ff29" gracePeriod=10 Dec 03 13:07:42 crc kubenswrapper[5052]: I1203 13:07:42.799697 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-59887957c5-st5lz"] Dec 03 13:07:42 crc kubenswrapper[5052]: I1203 13:07:42.801207 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-59887957c5-st5lz" Dec 03 13:07:42 crc kubenswrapper[5052]: I1203 13:07:42.820716 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-59887957c5-st5lz"] Dec 03 13:07:42 crc kubenswrapper[5052]: I1203 13:07:42.992259 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/16ade9c5-f16d-4698-8105-513ea6a3d495-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-59887957c5-st5lz\" (UID: \"16ade9c5-f16d-4698-8105-513ea6a3d495\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-59887957c5-st5lz" Dec 03 13:07:42 crc kubenswrapper[5052]: I1203 13:07:42.992727 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5rqld\" (UniqueName: \"kubernetes.io/projected/16ade9c5-f16d-4698-8105-513ea6a3d495-kube-api-access-5rqld\") pod \"dnsmasq-dnsmasq-59887957c5-st5lz\" (UID: \"16ade9c5-f16d-4698-8105-513ea6a3d495\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-59887957c5-st5lz" Dec 03 13:07:42 crc kubenswrapper[5052]: I1203 13:07:42.992804 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openstack-edpm-multinode\" (UniqueName: \"kubernetes.io/configmap/16ade9c5-f16d-4698-8105-513ea6a3d495-openstack-edpm-multinode\") pod \"dnsmasq-dnsmasq-59887957c5-st5lz\" (UID: \"16ade9c5-f16d-4698-8105-513ea6a3d495\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-59887957c5-st5lz" Dec 03 13:07:42 crc kubenswrapper[5052]: I1203 13:07:42.992835 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/16ade9c5-f16d-4698-8105-513ea6a3d495-config\") pod \"dnsmasq-dnsmasq-59887957c5-st5lz\" (UID: \"16ade9c5-f16d-4698-8105-513ea6a3d495\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-59887957c5-st5lz" Dec 03 13:07:43 crc kubenswrapper[5052]: I1203 13:07:43.058406 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-wnkqw" Dec 03 13:07:43 crc kubenswrapper[5052]: I1203 13:07:43.096334 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/16ade9c5-f16d-4698-8105-513ea6a3d495-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-59887957c5-st5lz\" (UID: \"16ade9c5-f16d-4698-8105-513ea6a3d495\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-59887957c5-st5lz" Dec 03 13:07:43 crc kubenswrapper[5052]: I1203 13:07:43.096419 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-5rqld\" (UniqueName: \"kubernetes.io/projected/16ade9c5-f16d-4698-8105-513ea6a3d495-kube-api-access-5rqld\") pod \"dnsmasq-dnsmasq-59887957c5-st5lz\" (UID: \"16ade9c5-f16d-4698-8105-513ea6a3d495\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-59887957c5-st5lz" Dec 03 13:07:43 crc kubenswrapper[5052]: I1203 13:07:43.096549 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"openstack-edpm-multinode\" (UniqueName: \"kubernetes.io/configmap/16ade9c5-f16d-4698-8105-513ea6a3d495-openstack-edpm-multinode\") pod \"dnsmasq-dnsmasq-59887957c5-st5lz\" (UID: \"16ade9c5-f16d-4698-8105-513ea6a3d495\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-59887957c5-st5lz" Dec 03 13:07:43 crc kubenswrapper[5052]: I1203 13:07:43.096593 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/16ade9c5-f16d-4698-8105-513ea6a3d495-config\") pod \"dnsmasq-dnsmasq-59887957c5-st5lz\" (UID: \"16ade9c5-f16d-4698-8105-513ea6a3d495\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-59887957c5-st5lz" Dec 03 13:07:43 crc kubenswrapper[5052]: I1203 13:07:43.097756 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"openstack-edpm-multinode\" (UniqueName: \"kubernetes.io/configmap/16ade9c5-f16d-4698-8105-513ea6a3d495-openstack-edpm-multinode\") pod \"dnsmasq-dnsmasq-59887957c5-st5lz\" (UID: \"16ade9c5-f16d-4698-8105-513ea6a3d495\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-59887957c5-st5lz" Dec 03 13:07:43 crc kubenswrapper[5052]: I1203 13:07:43.097846 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/16ade9c5-f16d-4698-8105-513ea6a3d495-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-59887957c5-st5lz\" (UID: \"16ade9c5-f16d-4698-8105-513ea6a3d495\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-59887957c5-st5lz" Dec 03 13:07:43 crc kubenswrapper[5052]: I1203 13:07:43.097917 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/16ade9c5-f16d-4698-8105-513ea6a3d495-config\") pod \"dnsmasq-dnsmasq-59887957c5-st5lz\" (UID: \"16ade9c5-f16d-4698-8105-513ea6a3d495\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-59887957c5-st5lz" Dec 03 13:07:43 crc kubenswrapper[5052]: I1203 13:07:43.126457 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-5rqld\" (UniqueName: \"kubernetes.io/projected/16ade9c5-f16d-4698-8105-513ea6a3d495-kube-api-access-5rqld\") pod \"dnsmasq-dnsmasq-59887957c5-st5lz\" (UID: \"16ade9c5-f16d-4698-8105-513ea6a3d495\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-59887957c5-st5lz" Dec 03 13:07:43 crc kubenswrapper[5052]: I1203 13:07:43.164070 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-59887957c5-st5lz" Dec 03 13:07:43 crc kubenswrapper[5052]: I1203 13:07:43.197833 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k7c2q\" (UniqueName: \"kubernetes.io/projected/ef40807a-35ce-4af5-9996-bc2f5b2fa3e4-kube-api-access-k7c2q\") pod \"ef40807a-35ce-4af5-9996-bc2f5b2fa3e4\" (UID: \"ef40807a-35ce-4af5-9996-bc2f5b2fa3e4\") " Dec 03 13:07:43 crc kubenswrapper[5052]: I1203 13:07:43.198735 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/ef40807a-35ce-4af5-9996-bc2f5b2fa3e4-dnsmasq-svc\") pod \"ef40807a-35ce-4af5-9996-bc2f5b2fa3e4\" (UID: \"ef40807a-35ce-4af5-9996-bc2f5b2fa3e4\") " Dec 03 13:07:43 crc kubenswrapper[5052]: I1203 13:07:43.198776 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ef40807a-35ce-4af5-9996-bc2f5b2fa3e4-config\") pod \"ef40807a-35ce-4af5-9996-bc2f5b2fa3e4\" (UID: \"ef40807a-35ce-4af5-9996-bc2f5b2fa3e4\") " Dec 03 13:07:43 crc kubenswrapper[5052]: I1203 13:07:43.202456 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ef40807a-35ce-4af5-9996-bc2f5b2fa3e4-kube-api-access-k7c2q" (OuterVolumeSpecName: "kube-api-access-k7c2q") pod "ef40807a-35ce-4af5-9996-bc2f5b2fa3e4" (UID: "ef40807a-35ce-4af5-9996-bc2f5b2fa3e4"). InnerVolumeSpecName "kube-api-access-k7c2q". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:07:43 crc kubenswrapper[5052]: I1203 13:07:43.235243 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ef40807a-35ce-4af5-9996-bc2f5b2fa3e4-config" (OuterVolumeSpecName: "config") pod "ef40807a-35ce-4af5-9996-bc2f5b2fa3e4" (UID: "ef40807a-35ce-4af5-9996-bc2f5b2fa3e4"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:07:43 crc kubenswrapper[5052]: I1203 13:07:43.237354 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ef40807a-35ce-4af5-9996-bc2f5b2fa3e4-dnsmasq-svc" (OuterVolumeSpecName: "dnsmasq-svc") pod "ef40807a-35ce-4af5-9996-bc2f5b2fa3e4" (UID: "ef40807a-35ce-4af5-9996-bc2f5b2fa3e4"). InnerVolumeSpecName "dnsmasq-svc". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:07:43 crc kubenswrapper[5052]: I1203 13:07:43.301618 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-k7c2q\" (UniqueName: \"kubernetes.io/projected/ef40807a-35ce-4af5-9996-bc2f5b2fa3e4-kube-api-access-k7c2q\") on node \"crc\" DevicePath \"\"" Dec 03 13:07:43 crc kubenswrapper[5052]: I1203 13:07:43.301656 5052 reconciler_common.go:293] "Volume detached for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/ef40807a-35ce-4af5-9996-bc2f5b2fa3e4-dnsmasq-svc\") on node \"crc\" DevicePath \"\"" Dec 03 13:07:43 crc kubenswrapper[5052]: I1203 13:07:43.301672 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ef40807a-35ce-4af5-9996-bc2f5b2fa3e4-config\") on node \"crc\" DevicePath \"\"" Dec 03 13:07:43 crc kubenswrapper[5052]: I1203 13:07:43.499107 5052 generic.go:334] "Generic (PLEG): container finished" podID="ef40807a-35ce-4af5-9996-bc2f5b2fa3e4" containerID="ec1bccc641595bfeab1754722642ceced12f7f3334c65a4375ce9af254d6ff29" exitCode=0 Dec 03 13:07:43 crc kubenswrapper[5052]: I1203 13:07:43.499210 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-wnkqw" Dec 03 13:07:43 crc kubenswrapper[5052]: I1203 13:07:43.499207 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-wnkqw" event={"ID":"ef40807a-35ce-4af5-9996-bc2f5b2fa3e4","Type":"ContainerDied","Data":"ec1bccc641595bfeab1754722642ceced12f7f3334c65a4375ce9af254d6ff29"} Dec 03 13:07:43 crc kubenswrapper[5052]: I1203 13:07:43.499615 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-wnkqw" event={"ID":"ef40807a-35ce-4af5-9996-bc2f5b2fa3e4","Type":"ContainerDied","Data":"7929be9fecdc702de97a668f70e5bb31736a5de2644ac0ac3763ca53f49d3e08"} Dec 03 13:07:43 crc kubenswrapper[5052]: I1203 13:07:43.499649 5052 scope.go:117] "RemoveContainer" containerID="ec1bccc641595bfeab1754722642ceced12f7f3334c65a4375ce9af254d6ff29" Dec 03 13:07:43 crc kubenswrapper[5052]: I1203 13:07:43.529584 5052 scope.go:117] "RemoveContainer" containerID="96148fe6fe8d9fa1905c70170e7143057f38d106aceaef2bf07e1425b8756491" Dec 03 13:07:43 crc kubenswrapper[5052]: I1203 13:07:43.534052 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-wnkqw"] Dec 03 13:07:43 crc kubenswrapper[5052]: I1203 13:07:43.544676 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-wnkqw"] Dec 03 13:07:43 crc kubenswrapper[5052]: I1203 13:07:43.567615 5052 scope.go:117] "RemoveContainer" containerID="ec1bccc641595bfeab1754722642ceced12f7f3334c65a4375ce9af254d6ff29" Dec 03 13:07:43 crc kubenswrapper[5052]: E1203 13:07:43.568183 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ec1bccc641595bfeab1754722642ceced12f7f3334c65a4375ce9af254d6ff29\": container with ID starting with ec1bccc641595bfeab1754722642ceced12f7f3334c65a4375ce9af254d6ff29 not found: ID does not exist" containerID="ec1bccc641595bfeab1754722642ceced12f7f3334c65a4375ce9af254d6ff29" Dec 03 13:07:43 crc kubenswrapper[5052]: I1203 13:07:43.568229 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ec1bccc641595bfeab1754722642ceced12f7f3334c65a4375ce9af254d6ff29"} err="failed to get container status \"ec1bccc641595bfeab1754722642ceced12f7f3334c65a4375ce9af254d6ff29\": rpc error: code = NotFound desc = could not find container \"ec1bccc641595bfeab1754722642ceced12f7f3334c65a4375ce9af254d6ff29\": container with ID starting with ec1bccc641595bfeab1754722642ceced12f7f3334c65a4375ce9af254d6ff29 not found: ID does not exist" Dec 03 13:07:43 crc kubenswrapper[5052]: I1203 13:07:43.568258 5052 scope.go:117] "RemoveContainer" containerID="96148fe6fe8d9fa1905c70170e7143057f38d106aceaef2bf07e1425b8756491" Dec 03 13:07:43 crc kubenswrapper[5052]: E1203 13:07:43.568601 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"96148fe6fe8d9fa1905c70170e7143057f38d106aceaef2bf07e1425b8756491\": container with ID starting with 96148fe6fe8d9fa1905c70170e7143057f38d106aceaef2bf07e1425b8756491 not found: ID does not exist" containerID="96148fe6fe8d9fa1905c70170e7143057f38d106aceaef2bf07e1425b8756491" Dec 03 13:07:43 crc kubenswrapper[5052]: I1203 13:07:43.568634 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"96148fe6fe8d9fa1905c70170e7143057f38d106aceaef2bf07e1425b8756491"} err="failed to get container status \"96148fe6fe8d9fa1905c70170e7143057f38d106aceaef2bf07e1425b8756491\": rpc error: code = NotFound desc = could not find container \"96148fe6fe8d9fa1905c70170e7143057f38d106aceaef2bf07e1425b8756491\": container with ID starting with 96148fe6fe8d9fa1905c70170e7143057f38d106aceaef2bf07e1425b8756491 not found: ID does not exist" Dec 03 13:07:43 crc kubenswrapper[5052]: I1203 13:07:43.586859 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-59887957c5-st5lz"] Dec 03 13:07:43 crc kubenswrapper[5052]: I1203 13:07:43.687144 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ef40807a-35ce-4af5-9996-bc2f5b2fa3e4" path="/var/lib/kubelet/pods/ef40807a-35ce-4af5-9996-bc2f5b2fa3e4/volumes" Dec 03 13:07:44 crc kubenswrapper[5052]: I1203 13:07:44.513235 5052 generic.go:334] "Generic (PLEG): container finished" podID="16ade9c5-f16d-4698-8105-513ea6a3d495" containerID="510b8f680c512b67c07165ba716d514a12a5a1dfcf58b0e98ca76e5290a10596" exitCode=0 Dec 03 13:07:44 crc kubenswrapper[5052]: I1203 13:07:44.513275 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-59887957c5-st5lz" event={"ID":"16ade9c5-f16d-4698-8105-513ea6a3d495","Type":"ContainerDied","Data":"510b8f680c512b67c07165ba716d514a12a5a1dfcf58b0e98ca76e5290a10596"} Dec 03 13:07:44 crc kubenswrapper[5052]: I1203 13:07:44.513666 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-59887957c5-st5lz" event={"ID":"16ade9c5-f16d-4698-8105-513ea6a3d495","Type":"ContainerStarted","Data":"5ab9bfe7aac9b7cc6a1a4ebf7436626758ce1f03a245ec30752e29c32774ffd7"} Dec 03 13:07:45 crc kubenswrapper[5052]: I1203 13:07:45.532715 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-59887957c5-st5lz" event={"ID":"16ade9c5-f16d-4698-8105-513ea6a3d495","Type":"ContainerStarted","Data":"d9be5849a3d1aae73a6827bedb94929106cb8767d1d3accbbab988da91bf6e9f"} Dec 03 13:07:45 crc kubenswrapper[5052]: I1203 13:07:45.533034 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-59887957c5-st5lz" Dec 03 13:07:45 crc kubenswrapper[5052]: I1203 13:07:45.555402 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-59887957c5-st5lz" podStartSLOduration=3.555377503 podStartE2EDuration="3.555377503s" podCreationTimestamp="2025-12-03 13:07:42 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 13:07:45.550167574 +0000 UTC m=+6456.138409338" watchObservedRunningTime="2025-12-03 13:07:45.555377503 +0000 UTC m=+6456.143619277" Dec 03 13:07:52 crc kubenswrapper[5052]: I1203 13:07:52.673058 5052 scope.go:117] "RemoveContainer" containerID="dbc100c8081ea1a2f5487429cc9d67de6bc8ba11282b64f248c71275375b0e16" Dec 03 13:07:52 crc kubenswrapper[5052]: E1203 13:07:52.673872 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 13:07:53 crc kubenswrapper[5052]: I1203 13:07:53.166332 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-59887957c5-st5lz" Dec 03 13:07:53 crc kubenswrapper[5052]: I1203 13:07:53.233540 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-58854494b5-pjqqv"] Dec 03 13:07:53 crc kubenswrapper[5052]: I1203 13:07:53.233823 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-58854494b5-pjqqv" podUID="f598fbaf-216d-45d5-a432-e45ed1406aea" containerName="dnsmasq-dns" containerID="cri-o://d53d8576eea3c007e4aa4de14d49102afa730cd6f654fa476363f15a72a71c8d" gracePeriod=10 Dec 03 13:07:53 crc kubenswrapper[5052]: I1203 13:07:53.605777 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-58854494b5-pjqqv" Dec 03 13:07:53 crc kubenswrapper[5052]: I1203 13:07:53.607438 5052 generic.go:334] "Generic (PLEG): container finished" podID="f598fbaf-216d-45d5-a432-e45ed1406aea" containerID="d53d8576eea3c007e4aa4de14d49102afa730cd6f654fa476363f15a72a71c8d" exitCode=0 Dec 03 13:07:53 crc kubenswrapper[5052]: I1203 13:07:53.607478 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-58854494b5-pjqqv" event={"ID":"f598fbaf-216d-45d5-a432-e45ed1406aea","Type":"ContainerDied","Data":"d53d8576eea3c007e4aa4de14d49102afa730cd6f654fa476363f15a72a71c8d"} Dec 03 13:07:53 crc kubenswrapper[5052]: I1203 13:07:53.607508 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-58854494b5-pjqqv" event={"ID":"f598fbaf-216d-45d5-a432-e45ed1406aea","Type":"ContainerDied","Data":"e87e4725839f5d90d6577c1e4c69c171e71322fb9c385bed589a96502b54d1cc"} Dec 03 13:07:53 crc kubenswrapper[5052]: I1203 13:07:53.607546 5052 scope.go:117] "RemoveContainer" containerID="d53d8576eea3c007e4aa4de14d49102afa730cd6f654fa476363f15a72a71c8d" Dec 03 13:07:53 crc kubenswrapper[5052]: I1203 13:07:53.626563 5052 scope.go:117] "RemoveContainer" containerID="57d7b31bfaf447f6238101640b82c00377ae0af452d9a64800393470d2516956" Dec 03 13:07:53 crc kubenswrapper[5052]: I1203 13:07:53.671891 5052 scope.go:117] "RemoveContainer" containerID="d53d8576eea3c007e4aa4de14d49102afa730cd6f654fa476363f15a72a71c8d" Dec 03 13:07:53 crc kubenswrapper[5052]: E1203 13:07:53.672503 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d53d8576eea3c007e4aa4de14d49102afa730cd6f654fa476363f15a72a71c8d\": container with ID starting with d53d8576eea3c007e4aa4de14d49102afa730cd6f654fa476363f15a72a71c8d not found: ID does not exist" containerID="d53d8576eea3c007e4aa4de14d49102afa730cd6f654fa476363f15a72a71c8d" Dec 03 13:07:53 crc kubenswrapper[5052]: I1203 13:07:53.672563 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d53d8576eea3c007e4aa4de14d49102afa730cd6f654fa476363f15a72a71c8d"} err="failed to get container status \"d53d8576eea3c007e4aa4de14d49102afa730cd6f654fa476363f15a72a71c8d\": rpc error: code = NotFound desc = could not find container \"d53d8576eea3c007e4aa4de14d49102afa730cd6f654fa476363f15a72a71c8d\": container with ID starting with d53d8576eea3c007e4aa4de14d49102afa730cd6f654fa476363f15a72a71c8d not found: ID does not exist" Dec 03 13:07:53 crc kubenswrapper[5052]: I1203 13:07:53.672590 5052 scope.go:117] "RemoveContainer" containerID="57d7b31bfaf447f6238101640b82c00377ae0af452d9a64800393470d2516956" Dec 03 13:07:53 crc kubenswrapper[5052]: E1203 13:07:53.673069 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"57d7b31bfaf447f6238101640b82c00377ae0af452d9a64800393470d2516956\": container with ID starting with 57d7b31bfaf447f6238101640b82c00377ae0af452d9a64800393470d2516956 not found: ID does not exist" containerID="57d7b31bfaf447f6238101640b82c00377ae0af452d9a64800393470d2516956" Dec 03 13:07:53 crc kubenswrapper[5052]: I1203 13:07:53.673091 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"57d7b31bfaf447f6238101640b82c00377ae0af452d9a64800393470d2516956"} err="failed to get container status \"57d7b31bfaf447f6238101640b82c00377ae0af452d9a64800393470d2516956\": rpc error: code = NotFound desc = could not find container \"57d7b31bfaf447f6238101640b82c00377ae0af452d9a64800393470d2516956\": container with ID starting with 57d7b31bfaf447f6238101640b82c00377ae0af452d9a64800393470d2516956 not found: ID does not exist" Dec 03 13:07:53 crc kubenswrapper[5052]: I1203 13:07:53.773328 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/f598fbaf-216d-45d5-a432-e45ed1406aea-dnsmasq-svc\") pod \"f598fbaf-216d-45d5-a432-e45ed1406aea\" (UID: \"f598fbaf-216d-45d5-a432-e45ed1406aea\") " Dec 03 13:07:53 crc kubenswrapper[5052]: I1203 13:07:53.773420 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"openstack-edpm-multinode\" (UniqueName: \"kubernetes.io/configmap/f598fbaf-216d-45d5-a432-e45ed1406aea-openstack-edpm-multinode\") pod \"f598fbaf-216d-45d5-a432-e45ed1406aea\" (UID: \"f598fbaf-216d-45d5-a432-e45ed1406aea\") " Dec 03 13:07:53 crc kubenswrapper[5052]: I1203 13:07:53.773467 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f598fbaf-216d-45d5-a432-e45ed1406aea-config\") pod \"f598fbaf-216d-45d5-a432-e45ed1406aea\" (UID: \"f598fbaf-216d-45d5-a432-e45ed1406aea\") " Dec 03 13:07:53 crc kubenswrapper[5052]: I1203 13:07:53.773572 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l5zzt\" (UniqueName: \"kubernetes.io/projected/f598fbaf-216d-45d5-a432-e45ed1406aea-kube-api-access-l5zzt\") pod \"f598fbaf-216d-45d5-a432-e45ed1406aea\" (UID: \"f598fbaf-216d-45d5-a432-e45ed1406aea\") " Dec 03 13:07:53 crc kubenswrapper[5052]: I1203 13:07:53.782027 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f598fbaf-216d-45d5-a432-e45ed1406aea-kube-api-access-l5zzt" (OuterVolumeSpecName: "kube-api-access-l5zzt") pod "f598fbaf-216d-45d5-a432-e45ed1406aea" (UID: "f598fbaf-216d-45d5-a432-e45ed1406aea"). InnerVolumeSpecName "kube-api-access-l5zzt". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:07:53 crc kubenswrapper[5052]: I1203 13:07:53.831618 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f598fbaf-216d-45d5-a432-e45ed1406aea-openstack-edpm-multinode" (OuterVolumeSpecName: "openstack-edpm-multinode") pod "f598fbaf-216d-45d5-a432-e45ed1406aea" (UID: "f598fbaf-216d-45d5-a432-e45ed1406aea"). InnerVolumeSpecName "openstack-edpm-multinode". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:07:53 crc kubenswrapper[5052]: I1203 13:07:53.834330 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f598fbaf-216d-45d5-a432-e45ed1406aea-config" (OuterVolumeSpecName: "config") pod "f598fbaf-216d-45d5-a432-e45ed1406aea" (UID: "f598fbaf-216d-45d5-a432-e45ed1406aea"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:07:53 crc kubenswrapper[5052]: I1203 13:07:53.851042 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f598fbaf-216d-45d5-a432-e45ed1406aea-dnsmasq-svc" (OuterVolumeSpecName: "dnsmasq-svc") pod "f598fbaf-216d-45d5-a432-e45ed1406aea" (UID: "f598fbaf-216d-45d5-a432-e45ed1406aea"). InnerVolumeSpecName "dnsmasq-svc". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:07:53 crc kubenswrapper[5052]: I1203 13:07:53.874040 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-5d55f47b6c-xpgvj"] Dec 03 13:07:53 crc kubenswrapper[5052]: E1203 13:07:53.874387 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="f598fbaf-216d-45d5-a432-e45ed1406aea" containerName="init" Dec 03 13:07:53 crc kubenswrapper[5052]: I1203 13:07:53.874415 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="f598fbaf-216d-45d5-a432-e45ed1406aea" containerName="init" Dec 03 13:07:53 crc kubenswrapper[5052]: E1203 13:07:53.874449 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="ef40807a-35ce-4af5-9996-bc2f5b2fa3e4" containerName="dnsmasq-dns" Dec 03 13:07:53 crc kubenswrapper[5052]: I1203 13:07:53.874459 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="ef40807a-35ce-4af5-9996-bc2f5b2fa3e4" containerName="dnsmasq-dns" Dec 03 13:07:53 crc kubenswrapper[5052]: E1203 13:07:53.874484 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="f598fbaf-216d-45d5-a432-e45ed1406aea" containerName="dnsmasq-dns" Dec 03 13:07:53 crc kubenswrapper[5052]: I1203 13:07:53.874497 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="f598fbaf-216d-45d5-a432-e45ed1406aea" containerName="dnsmasq-dns" Dec 03 13:07:53 crc kubenswrapper[5052]: E1203 13:07:53.874592 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="ef40807a-35ce-4af5-9996-bc2f5b2fa3e4" containerName="init" Dec 03 13:07:53 crc kubenswrapper[5052]: I1203 13:07:53.874602 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="ef40807a-35ce-4af5-9996-bc2f5b2fa3e4" containerName="init" Dec 03 13:07:53 crc kubenswrapper[5052]: I1203 13:07:53.874949 5052 reconciler_common.go:293] "Volume detached for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/f598fbaf-216d-45d5-a432-e45ed1406aea-dnsmasq-svc\") on node \"crc\" DevicePath \"\"" Dec 03 13:07:53 crc kubenswrapper[5052]: I1203 13:07:53.874975 5052 reconciler_common.go:293] "Volume detached for volume \"openstack-edpm-multinode\" (UniqueName: \"kubernetes.io/configmap/f598fbaf-216d-45d5-a432-e45ed1406aea-openstack-edpm-multinode\") on node \"crc\" DevicePath \"\"" Dec 03 13:07:53 crc kubenswrapper[5052]: I1203 13:07:53.874990 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f598fbaf-216d-45d5-a432-e45ed1406aea-config\") on node \"crc\" DevicePath \"\"" Dec 03 13:07:53 crc kubenswrapper[5052]: I1203 13:07:53.875005 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-l5zzt\" (UniqueName: \"kubernetes.io/projected/f598fbaf-216d-45d5-a432-e45ed1406aea-kube-api-access-l5zzt\") on node \"crc\" DevicePath \"\"" Dec 03 13:07:53 crc kubenswrapper[5052]: I1203 13:07:53.875015 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="ef40807a-35ce-4af5-9996-bc2f5b2fa3e4" containerName="dnsmasq-dns" Dec 03 13:07:53 crc kubenswrapper[5052]: I1203 13:07:53.875040 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="f598fbaf-216d-45d5-a432-e45ed1406aea" containerName="dnsmasq-dns" Dec 03 13:07:53 crc kubenswrapper[5052]: I1203 13:07:53.875854 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-5d55f47b6c-xpgvj" Dec 03 13:07:53 crc kubenswrapper[5052]: I1203 13:07:53.893282 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-5d55f47b6c-xpgvj"] Dec 03 13:07:53 crc kubenswrapper[5052]: I1203 13:07:53.976558 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c02a623d-feec-4c7b-92c2-3fed73e09d06-config\") pod \"dnsmasq-dnsmasq-5d55f47b6c-xpgvj\" (UID: \"c02a623d-feec-4c7b-92c2-3fed73e09d06\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-5d55f47b6c-xpgvj" Dec 03 13:07:53 crc kubenswrapper[5052]: I1203 13:07:53.977100 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/c02a623d-feec-4c7b-92c2-3fed73e09d06-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-5d55f47b6c-xpgvj\" (UID: \"c02a623d-feec-4c7b-92c2-3fed73e09d06\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-5d55f47b6c-xpgvj" Dec 03 13:07:53 crc kubenswrapper[5052]: I1203 13:07:53.977345 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fqtr8\" (UniqueName: \"kubernetes.io/projected/c02a623d-feec-4c7b-92c2-3fed73e09d06-kube-api-access-fqtr8\") pod \"dnsmasq-dnsmasq-5d55f47b6c-xpgvj\" (UID: \"c02a623d-feec-4c7b-92c2-3fed73e09d06\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-5d55f47b6c-xpgvj" Dec 03 13:07:53 crc kubenswrapper[5052]: I1203 13:07:53.977579 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openstack-edpm-multinode\" (UniqueName: \"kubernetes.io/configmap/c02a623d-feec-4c7b-92c2-3fed73e09d06-openstack-edpm-multinode\") pod \"dnsmasq-dnsmasq-5d55f47b6c-xpgvj\" (UID: \"c02a623d-feec-4c7b-92c2-3fed73e09d06\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-5d55f47b6c-xpgvj" Dec 03 13:07:54 crc kubenswrapper[5052]: I1203 13:07:54.078753 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-fqtr8\" (UniqueName: \"kubernetes.io/projected/c02a623d-feec-4c7b-92c2-3fed73e09d06-kube-api-access-fqtr8\") pod \"dnsmasq-dnsmasq-5d55f47b6c-xpgvj\" (UID: \"c02a623d-feec-4c7b-92c2-3fed73e09d06\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-5d55f47b6c-xpgvj" Dec 03 13:07:54 crc kubenswrapper[5052]: I1203 13:07:54.078814 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"openstack-edpm-multinode\" (UniqueName: \"kubernetes.io/configmap/c02a623d-feec-4c7b-92c2-3fed73e09d06-openstack-edpm-multinode\") pod \"dnsmasq-dnsmasq-5d55f47b6c-xpgvj\" (UID: \"c02a623d-feec-4c7b-92c2-3fed73e09d06\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-5d55f47b6c-xpgvj" Dec 03 13:07:54 crc kubenswrapper[5052]: I1203 13:07:54.078856 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c02a623d-feec-4c7b-92c2-3fed73e09d06-config\") pod \"dnsmasq-dnsmasq-5d55f47b6c-xpgvj\" (UID: \"c02a623d-feec-4c7b-92c2-3fed73e09d06\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-5d55f47b6c-xpgvj" Dec 03 13:07:54 crc kubenswrapper[5052]: I1203 13:07:54.078889 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/c02a623d-feec-4c7b-92c2-3fed73e09d06-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-5d55f47b6c-xpgvj\" (UID: \"c02a623d-feec-4c7b-92c2-3fed73e09d06\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-5d55f47b6c-xpgvj" Dec 03 13:07:54 crc kubenswrapper[5052]: I1203 13:07:54.079698 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c02a623d-feec-4c7b-92c2-3fed73e09d06-config\") pod \"dnsmasq-dnsmasq-5d55f47b6c-xpgvj\" (UID: \"c02a623d-feec-4c7b-92c2-3fed73e09d06\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-5d55f47b6c-xpgvj" Dec 03 13:07:54 crc kubenswrapper[5052]: I1203 13:07:54.079700 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"openstack-edpm-multinode\" (UniqueName: \"kubernetes.io/configmap/c02a623d-feec-4c7b-92c2-3fed73e09d06-openstack-edpm-multinode\") pod \"dnsmasq-dnsmasq-5d55f47b6c-xpgvj\" (UID: \"c02a623d-feec-4c7b-92c2-3fed73e09d06\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-5d55f47b6c-xpgvj" Dec 03 13:07:54 crc kubenswrapper[5052]: I1203 13:07:54.079952 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/c02a623d-feec-4c7b-92c2-3fed73e09d06-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-5d55f47b6c-xpgvj\" (UID: \"c02a623d-feec-4c7b-92c2-3fed73e09d06\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-5d55f47b6c-xpgvj" Dec 03 13:07:54 crc kubenswrapper[5052]: I1203 13:07:54.094872 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-fqtr8\" (UniqueName: \"kubernetes.io/projected/c02a623d-feec-4c7b-92c2-3fed73e09d06-kube-api-access-fqtr8\") pod \"dnsmasq-dnsmasq-5d55f47b6c-xpgvj\" (UID: \"c02a623d-feec-4c7b-92c2-3fed73e09d06\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-5d55f47b6c-xpgvj" Dec 03 13:07:54 crc kubenswrapper[5052]: I1203 13:07:54.196474 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-5d55f47b6c-xpgvj" Dec 03 13:07:54 crc kubenswrapper[5052]: I1203 13:07:54.619722 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-5d55f47b6c-xpgvj"] Dec 03 13:07:54 crc kubenswrapper[5052]: I1203 13:07:54.624140 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-58854494b5-pjqqv" Dec 03 13:07:54 crc kubenswrapper[5052]: I1203 13:07:54.732700 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-58854494b5-pjqqv"] Dec 03 13:07:54 crc kubenswrapper[5052]: I1203 13:07:54.741247 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-58854494b5-pjqqv"] Dec 03 13:07:55 crc kubenswrapper[5052]: I1203 13:07:55.636164 5052 generic.go:334] "Generic (PLEG): container finished" podID="c02a623d-feec-4c7b-92c2-3fed73e09d06" containerID="62f55f44a90a0f4177317b916d9a2e872cb27a3fdd28948ab0132e59dec35e2f" exitCode=0 Dec 03 13:07:55 crc kubenswrapper[5052]: I1203 13:07:55.636248 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-5d55f47b6c-xpgvj" event={"ID":"c02a623d-feec-4c7b-92c2-3fed73e09d06","Type":"ContainerDied","Data":"62f55f44a90a0f4177317b916d9a2e872cb27a3fdd28948ab0132e59dec35e2f"} Dec 03 13:07:55 crc kubenswrapper[5052]: I1203 13:07:55.636911 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-5d55f47b6c-xpgvj" event={"ID":"c02a623d-feec-4c7b-92c2-3fed73e09d06","Type":"ContainerStarted","Data":"8524251498b3d4431566f1869808e1059ddcd6826c8b888493813187faf53d66"} Dec 03 13:07:55 crc kubenswrapper[5052]: I1203 13:07:55.688410 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f598fbaf-216d-45d5-a432-e45ed1406aea" path="/var/lib/kubelet/pods/f598fbaf-216d-45d5-a432-e45ed1406aea/volumes" Dec 03 13:07:56 crc kubenswrapper[5052]: I1203 13:07:56.656980 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-5d55f47b6c-xpgvj" event={"ID":"c02a623d-feec-4c7b-92c2-3fed73e09d06","Type":"ContainerStarted","Data":"ec095b0a2ce4b448d9a3ef1c01d89dee455e0b69d5e95ae270355b86b7ae3e8c"} Dec 03 13:07:56 crc kubenswrapper[5052]: I1203 13:07:56.657161 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-5d55f47b6c-xpgvj" Dec 03 13:07:56 crc kubenswrapper[5052]: I1203 13:07:56.679605 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-5d55f47b6c-xpgvj" podStartSLOduration=3.679583595 podStartE2EDuration="3.679583595s" podCreationTimestamp="2025-12-03 13:07:53 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 13:07:56.677400782 +0000 UTC m=+6467.265642546" watchObservedRunningTime="2025-12-03 13:07:56.679583595 +0000 UTC m=+6467.267825349" Dec 03 13:08:03 crc kubenswrapper[5052]: I1203 13:08:03.672883 5052 scope.go:117] "RemoveContainer" containerID="dbc100c8081ea1a2f5487429cc9d67de6bc8ba11282b64f248c71275375b0e16" Dec 03 13:08:03 crc kubenswrapper[5052]: E1203 13:08:03.673406 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 13:08:04 crc kubenswrapper[5052]: I1203 13:08:04.198786 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-5d55f47b6c-xpgvj" Dec 03 13:08:04 crc kubenswrapper[5052]: I1203 13:08:04.268416 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-59887957c5-st5lz"] Dec 03 13:08:04 crc kubenswrapper[5052]: I1203 13:08:04.268982 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-59887957c5-st5lz" podUID="16ade9c5-f16d-4698-8105-513ea6a3d495" containerName="dnsmasq-dns" containerID="cri-o://d9be5849a3d1aae73a6827bedb94929106cb8767d1d3accbbab988da91bf6e9f" gracePeriod=10 Dec 03 13:08:04 crc kubenswrapper[5052]: I1203 13:08:04.494065 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-964c896d7-ckkr8"] Dec 03 13:08:04 crc kubenswrapper[5052]: I1203 13:08:04.496640 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-964c896d7-ckkr8" Dec 03 13:08:04 crc kubenswrapper[5052]: I1203 13:08:04.529054 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-964c896d7-ckkr8"] Dec 03 13:08:04 crc kubenswrapper[5052]: I1203 13:08:04.655881 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openstack-edpm-multinode\" (UniqueName: \"kubernetes.io/configmap/a370691c-0245-4503-809a-7037fd5359c3-openstack-edpm-multinode\") pod \"dnsmasq-dnsmasq-964c896d7-ckkr8\" (UID: \"a370691c-0245-4503-809a-7037fd5359c3\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-964c896d7-ckkr8" Dec 03 13:08:04 crc kubenswrapper[5052]: I1203 13:08:04.656581 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/a370691c-0245-4503-809a-7037fd5359c3-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-964c896d7-ckkr8\" (UID: \"a370691c-0245-4503-809a-7037fd5359c3\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-964c896d7-ckkr8" Dec 03 13:08:04 crc kubenswrapper[5052]: I1203 13:08:04.656658 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x7jx8\" (UniqueName: \"kubernetes.io/projected/a370691c-0245-4503-809a-7037fd5359c3-kube-api-access-x7jx8\") pod \"dnsmasq-dnsmasq-964c896d7-ckkr8\" (UID: \"a370691c-0245-4503-809a-7037fd5359c3\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-964c896d7-ckkr8" Dec 03 13:08:04 crc kubenswrapper[5052]: I1203 13:08:04.656704 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a370691c-0245-4503-809a-7037fd5359c3-config\") pod \"dnsmasq-dnsmasq-964c896d7-ckkr8\" (UID: \"a370691c-0245-4503-809a-7037fd5359c3\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-964c896d7-ckkr8" Dec 03 13:08:04 crc kubenswrapper[5052]: I1203 13:08:04.704355 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-59887957c5-st5lz" Dec 03 13:08:04 crc kubenswrapper[5052]: I1203 13:08:04.733116 5052 generic.go:334] "Generic (PLEG): container finished" podID="16ade9c5-f16d-4698-8105-513ea6a3d495" containerID="d9be5849a3d1aae73a6827bedb94929106cb8767d1d3accbbab988da91bf6e9f" exitCode=0 Dec 03 13:08:04 crc kubenswrapper[5052]: I1203 13:08:04.733171 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-59887957c5-st5lz" event={"ID":"16ade9c5-f16d-4698-8105-513ea6a3d495","Type":"ContainerDied","Data":"d9be5849a3d1aae73a6827bedb94929106cb8767d1d3accbbab988da91bf6e9f"} Dec 03 13:08:04 crc kubenswrapper[5052]: I1203 13:08:04.733203 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-59887957c5-st5lz" event={"ID":"16ade9c5-f16d-4698-8105-513ea6a3d495","Type":"ContainerDied","Data":"5ab9bfe7aac9b7cc6a1a4ebf7436626758ce1f03a245ec30752e29c32774ffd7"} Dec 03 13:08:04 crc kubenswrapper[5052]: I1203 13:08:04.733224 5052 scope.go:117] "RemoveContainer" containerID="d9be5849a3d1aae73a6827bedb94929106cb8767d1d3accbbab988da91bf6e9f" Dec 03 13:08:04 crc kubenswrapper[5052]: I1203 13:08:04.733368 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-59887957c5-st5lz" Dec 03 13:08:04 crc kubenswrapper[5052]: I1203 13:08:04.758418 5052 scope.go:117] "RemoveContainer" containerID="510b8f680c512b67c07165ba716d514a12a5a1dfcf58b0e98ca76e5290a10596" Dec 03 13:08:04 crc kubenswrapper[5052]: I1203 13:08:04.758918 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-x7jx8\" (UniqueName: \"kubernetes.io/projected/a370691c-0245-4503-809a-7037fd5359c3-kube-api-access-x7jx8\") pod \"dnsmasq-dnsmasq-964c896d7-ckkr8\" (UID: \"a370691c-0245-4503-809a-7037fd5359c3\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-964c896d7-ckkr8" Dec 03 13:08:04 crc kubenswrapper[5052]: I1203 13:08:04.758967 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a370691c-0245-4503-809a-7037fd5359c3-config\") pod \"dnsmasq-dnsmasq-964c896d7-ckkr8\" (UID: \"a370691c-0245-4503-809a-7037fd5359c3\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-964c896d7-ckkr8" Dec 03 13:08:04 crc kubenswrapper[5052]: I1203 13:08:04.759034 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"openstack-edpm-multinode\" (UniqueName: \"kubernetes.io/configmap/a370691c-0245-4503-809a-7037fd5359c3-openstack-edpm-multinode\") pod \"dnsmasq-dnsmasq-964c896d7-ckkr8\" (UID: \"a370691c-0245-4503-809a-7037fd5359c3\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-964c896d7-ckkr8" Dec 03 13:08:04 crc kubenswrapper[5052]: I1203 13:08:04.759073 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/a370691c-0245-4503-809a-7037fd5359c3-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-964c896d7-ckkr8\" (UID: \"a370691c-0245-4503-809a-7037fd5359c3\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-964c896d7-ckkr8" Dec 03 13:08:04 crc kubenswrapper[5052]: I1203 13:08:04.760655 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a370691c-0245-4503-809a-7037fd5359c3-config\") pod \"dnsmasq-dnsmasq-964c896d7-ckkr8\" (UID: \"a370691c-0245-4503-809a-7037fd5359c3\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-964c896d7-ckkr8" Dec 03 13:08:04 crc kubenswrapper[5052]: I1203 13:08:04.761153 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"openstack-edpm-multinode\" (UniqueName: \"kubernetes.io/configmap/a370691c-0245-4503-809a-7037fd5359c3-openstack-edpm-multinode\") pod \"dnsmasq-dnsmasq-964c896d7-ckkr8\" (UID: \"a370691c-0245-4503-809a-7037fd5359c3\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-964c896d7-ckkr8" Dec 03 13:08:04 crc kubenswrapper[5052]: I1203 13:08:04.761678 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/a370691c-0245-4503-809a-7037fd5359c3-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-964c896d7-ckkr8\" (UID: \"a370691c-0245-4503-809a-7037fd5359c3\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-964c896d7-ckkr8" Dec 03 13:08:04 crc kubenswrapper[5052]: I1203 13:08:04.781547 5052 scope.go:117] "RemoveContainer" containerID="d9be5849a3d1aae73a6827bedb94929106cb8767d1d3accbbab988da91bf6e9f" Dec 03 13:08:04 crc kubenswrapper[5052]: E1203 13:08:04.782061 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d9be5849a3d1aae73a6827bedb94929106cb8767d1d3accbbab988da91bf6e9f\": container with ID starting with d9be5849a3d1aae73a6827bedb94929106cb8767d1d3accbbab988da91bf6e9f not found: ID does not exist" containerID="d9be5849a3d1aae73a6827bedb94929106cb8767d1d3accbbab988da91bf6e9f" Dec 03 13:08:04 crc kubenswrapper[5052]: I1203 13:08:04.782096 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d9be5849a3d1aae73a6827bedb94929106cb8767d1d3accbbab988da91bf6e9f"} err="failed to get container status \"d9be5849a3d1aae73a6827bedb94929106cb8767d1d3accbbab988da91bf6e9f\": rpc error: code = NotFound desc = could not find container \"d9be5849a3d1aae73a6827bedb94929106cb8767d1d3accbbab988da91bf6e9f\": container with ID starting with d9be5849a3d1aae73a6827bedb94929106cb8767d1d3accbbab988da91bf6e9f not found: ID does not exist" Dec 03 13:08:04 crc kubenswrapper[5052]: I1203 13:08:04.782122 5052 scope.go:117] "RemoveContainer" containerID="510b8f680c512b67c07165ba716d514a12a5a1dfcf58b0e98ca76e5290a10596" Dec 03 13:08:04 crc kubenswrapper[5052]: E1203 13:08:04.782450 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"510b8f680c512b67c07165ba716d514a12a5a1dfcf58b0e98ca76e5290a10596\": container with ID starting with 510b8f680c512b67c07165ba716d514a12a5a1dfcf58b0e98ca76e5290a10596 not found: ID does not exist" containerID="510b8f680c512b67c07165ba716d514a12a5a1dfcf58b0e98ca76e5290a10596" Dec 03 13:08:04 crc kubenswrapper[5052]: I1203 13:08:04.782479 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"510b8f680c512b67c07165ba716d514a12a5a1dfcf58b0e98ca76e5290a10596"} err="failed to get container status \"510b8f680c512b67c07165ba716d514a12a5a1dfcf58b0e98ca76e5290a10596\": rpc error: code = NotFound desc = could not find container \"510b8f680c512b67c07165ba716d514a12a5a1dfcf58b0e98ca76e5290a10596\": container with ID starting with 510b8f680c512b67c07165ba716d514a12a5a1dfcf58b0e98ca76e5290a10596 not found: ID does not exist" Dec 03 13:08:04 crc kubenswrapper[5052]: I1203 13:08:04.783041 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-x7jx8\" (UniqueName: \"kubernetes.io/projected/a370691c-0245-4503-809a-7037fd5359c3-kube-api-access-x7jx8\") pod \"dnsmasq-dnsmasq-964c896d7-ckkr8\" (UID: \"a370691c-0245-4503-809a-7037fd5359c3\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-964c896d7-ckkr8" Dec 03 13:08:04 crc kubenswrapper[5052]: I1203 13:08:04.824820 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-964c896d7-ckkr8" Dec 03 13:08:04 crc kubenswrapper[5052]: I1203 13:08:04.860173 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"openstack-edpm-multinode\" (UniqueName: \"kubernetes.io/configmap/16ade9c5-f16d-4698-8105-513ea6a3d495-openstack-edpm-multinode\") pod \"16ade9c5-f16d-4698-8105-513ea6a3d495\" (UID: \"16ade9c5-f16d-4698-8105-513ea6a3d495\") " Dec 03 13:08:04 crc kubenswrapper[5052]: I1203 13:08:04.860287 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5rqld\" (UniqueName: \"kubernetes.io/projected/16ade9c5-f16d-4698-8105-513ea6a3d495-kube-api-access-5rqld\") pod \"16ade9c5-f16d-4698-8105-513ea6a3d495\" (UID: \"16ade9c5-f16d-4698-8105-513ea6a3d495\") " Dec 03 13:08:04 crc kubenswrapper[5052]: I1203 13:08:04.860377 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/16ade9c5-f16d-4698-8105-513ea6a3d495-config\") pod \"16ade9c5-f16d-4698-8105-513ea6a3d495\" (UID: \"16ade9c5-f16d-4698-8105-513ea6a3d495\") " Dec 03 13:08:04 crc kubenswrapper[5052]: I1203 13:08:04.860466 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/16ade9c5-f16d-4698-8105-513ea6a3d495-dnsmasq-svc\") pod \"16ade9c5-f16d-4698-8105-513ea6a3d495\" (UID: \"16ade9c5-f16d-4698-8105-513ea6a3d495\") " Dec 03 13:08:04 crc kubenswrapper[5052]: I1203 13:08:04.864189 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/16ade9c5-f16d-4698-8105-513ea6a3d495-kube-api-access-5rqld" (OuterVolumeSpecName: "kube-api-access-5rqld") pod "16ade9c5-f16d-4698-8105-513ea6a3d495" (UID: "16ade9c5-f16d-4698-8105-513ea6a3d495"). InnerVolumeSpecName "kube-api-access-5rqld". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:08:04 crc kubenswrapper[5052]: I1203 13:08:04.898283 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/16ade9c5-f16d-4698-8105-513ea6a3d495-dnsmasq-svc" (OuterVolumeSpecName: "dnsmasq-svc") pod "16ade9c5-f16d-4698-8105-513ea6a3d495" (UID: "16ade9c5-f16d-4698-8105-513ea6a3d495"). InnerVolumeSpecName "dnsmasq-svc". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:08:04 crc kubenswrapper[5052]: I1203 13:08:04.907967 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/16ade9c5-f16d-4698-8105-513ea6a3d495-config" (OuterVolumeSpecName: "config") pod "16ade9c5-f16d-4698-8105-513ea6a3d495" (UID: "16ade9c5-f16d-4698-8105-513ea6a3d495"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:08:04 crc kubenswrapper[5052]: I1203 13:08:04.914869 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/16ade9c5-f16d-4698-8105-513ea6a3d495-openstack-edpm-multinode" (OuterVolumeSpecName: "openstack-edpm-multinode") pod "16ade9c5-f16d-4698-8105-513ea6a3d495" (UID: "16ade9c5-f16d-4698-8105-513ea6a3d495"). InnerVolumeSpecName "openstack-edpm-multinode". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:08:04 crc kubenswrapper[5052]: I1203 13:08:04.962890 5052 reconciler_common.go:293] "Volume detached for volume \"openstack-edpm-multinode\" (UniqueName: \"kubernetes.io/configmap/16ade9c5-f16d-4698-8105-513ea6a3d495-openstack-edpm-multinode\") on node \"crc\" DevicePath \"\"" Dec 03 13:08:04 crc kubenswrapper[5052]: I1203 13:08:04.963566 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-5rqld\" (UniqueName: \"kubernetes.io/projected/16ade9c5-f16d-4698-8105-513ea6a3d495-kube-api-access-5rqld\") on node \"crc\" DevicePath \"\"" Dec 03 13:08:04 crc kubenswrapper[5052]: I1203 13:08:04.963585 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/16ade9c5-f16d-4698-8105-513ea6a3d495-config\") on node \"crc\" DevicePath \"\"" Dec 03 13:08:04 crc kubenswrapper[5052]: I1203 13:08:04.963597 5052 reconciler_common.go:293] "Volume detached for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/16ade9c5-f16d-4698-8105-513ea6a3d495-dnsmasq-svc\") on node \"crc\" DevicePath \"\"" Dec 03 13:08:05 crc kubenswrapper[5052]: I1203 13:08:05.069280 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-59887957c5-st5lz"] Dec 03 13:08:05 crc kubenswrapper[5052]: I1203 13:08:05.078935 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-59887957c5-st5lz"] Dec 03 13:08:05 crc kubenswrapper[5052]: I1203 13:08:05.258625 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-964c896d7-ckkr8"] Dec 03 13:08:05 crc kubenswrapper[5052]: I1203 13:08:05.683674 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="16ade9c5-f16d-4698-8105-513ea6a3d495" path="/var/lib/kubelet/pods/16ade9c5-f16d-4698-8105-513ea6a3d495/volumes" Dec 03 13:08:05 crc kubenswrapper[5052]: I1203 13:08:05.742961 5052 generic.go:334] "Generic (PLEG): container finished" podID="a370691c-0245-4503-809a-7037fd5359c3" containerID="22d5be4016a4643c23661779b5d09c69180dbe313c3d4a9b7f212f6c180902cd" exitCode=0 Dec 03 13:08:05 crc kubenswrapper[5052]: I1203 13:08:05.743226 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-964c896d7-ckkr8" event={"ID":"a370691c-0245-4503-809a-7037fd5359c3","Type":"ContainerDied","Data":"22d5be4016a4643c23661779b5d09c69180dbe313c3d4a9b7f212f6c180902cd"} Dec 03 13:08:05 crc kubenswrapper[5052]: I1203 13:08:05.743671 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-964c896d7-ckkr8" event={"ID":"a370691c-0245-4503-809a-7037fd5359c3","Type":"ContainerStarted","Data":"2df392067093a7583eba5e59cc8a3bef83d09e77c42521749b73bd6030bd3dec"} Dec 03 13:08:06 crc kubenswrapper[5052]: I1203 13:08:06.756299 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-964c896d7-ckkr8" event={"ID":"a370691c-0245-4503-809a-7037fd5359c3","Type":"ContainerStarted","Data":"2b38eb5ba3951b0647676e20f7400ab1178dfbe830d47f2c8baa2007f1d09b01"} Dec 03 13:08:06 crc kubenswrapper[5052]: I1203 13:08:06.756477 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-964c896d7-ckkr8" Dec 03 13:08:06 crc kubenswrapper[5052]: I1203 13:08:06.780501 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-964c896d7-ckkr8" podStartSLOduration=2.780480572 podStartE2EDuration="2.780480572s" podCreationTimestamp="2025-12-03 13:08:04 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 13:08:06.773704088 +0000 UTC m=+6477.361945862" watchObservedRunningTime="2025-12-03 13:08:06.780480572 +0000 UTC m=+6477.368722336" Dec 03 13:08:14 crc kubenswrapper[5052]: I1203 13:08:14.827245 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-964c896d7-ckkr8" Dec 03 13:08:14 crc kubenswrapper[5052]: I1203 13:08:14.895965 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-5d55f47b6c-xpgvj"] Dec 03 13:08:14 crc kubenswrapper[5052]: I1203 13:08:14.896352 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-5d55f47b6c-xpgvj" podUID="c02a623d-feec-4c7b-92c2-3fed73e09d06" containerName="dnsmasq-dns" containerID="cri-o://ec095b0a2ce4b448d9a3ef1c01d89dee455e0b69d5e95ae270355b86b7ae3e8c" gracePeriod=10 Dec 03 13:08:15 crc kubenswrapper[5052]: I1203 13:08:15.430506 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-74ckd"] Dec 03 13:08:15 crc kubenswrapper[5052]: E1203 13:08:15.431213 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="16ade9c5-f16d-4698-8105-513ea6a3d495" containerName="dnsmasq-dns" Dec 03 13:08:15 crc kubenswrapper[5052]: I1203 13:08:15.431237 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="16ade9c5-f16d-4698-8105-513ea6a3d495" containerName="dnsmasq-dns" Dec 03 13:08:15 crc kubenswrapper[5052]: E1203 13:08:15.431264 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="16ade9c5-f16d-4698-8105-513ea6a3d495" containerName="init" Dec 03 13:08:15 crc kubenswrapper[5052]: I1203 13:08:15.431272 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="16ade9c5-f16d-4698-8105-513ea6a3d495" containerName="init" Dec 03 13:08:15 crc kubenswrapper[5052]: I1203 13:08:15.431689 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="16ade9c5-f16d-4698-8105-513ea6a3d495" containerName="dnsmasq-dns" Dec 03 13:08:15 crc kubenswrapper[5052]: I1203 13:08:15.432709 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-74ckd" Dec 03 13:08:15 crc kubenswrapper[5052]: I1203 13:08:15.436862 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/62a2ca08-9a05-4d01-a792-05cedf6a1139-config\") pod \"dnsmasq-dnsmasq-84b9f45d47-74ckd\" (UID: \"62a2ca08-9a05-4d01-a792-05cedf6a1139\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-74ckd" Dec 03 13:08:15 crc kubenswrapper[5052]: I1203 13:08:15.436963 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b229d\" (UniqueName: \"kubernetes.io/projected/62a2ca08-9a05-4d01-a792-05cedf6a1139-kube-api-access-b229d\") pod \"dnsmasq-dnsmasq-84b9f45d47-74ckd\" (UID: \"62a2ca08-9a05-4d01-a792-05cedf6a1139\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-74ckd" Dec 03 13:08:15 crc kubenswrapper[5052]: I1203 13:08:15.437002 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/62a2ca08-9a05-4d01-a792-05cedf6a1139-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-84b9f45d47-74ckd\" (UID: \"62a2ca08-9a05-4d01-a792-05cedf6a1139\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-74ckd" Dec 03 13:08:15 crc kubenswrapper[5052]: I1203 13:08:15.446030 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-74ckd"] Dec 03 13:08:15 crc kubenswrapper[5052]: I1203 13:08:15.487631 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-5d55f47b6c-xpgvj" Dec 03 13:08:15 crc kubenswrapper[5052]: I1203 13:08:15.537431 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/c02a623d-feec-4c7b-92c2-3fed73e09d06-dnsmasq-svc\") pod \"c02a623d-feec-4c7b-92c2-3fed73e09d06\" (UID: \"c02a623d-feec-4c7b-92c2-3fed73e09d06\") " Dec 03 13:08:15 crc kubenswrapper[5052]: I1203 13:08:15.537529 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c02a623d-feec-4c7b-92c2-3fed73e09d06-config\") pod \"c02a623d-feec-4c7b-92c2-3fed73e09d06\" (UID: \"c02a623d-feec-4c7b-92c2-3fed73e09d06\") " Dec 03 13:08:15 crc kubenswrapper[5052]: I1203 13:08:15.537556 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fqtr8\" (UniqueName: \"kubernetes.io/projected/c02a623d-feec-4c7b-92c2-3fed73e09d06-kube-api-access-fqtr8\") pod \"c02a623d-feec-4c7b-92c2-3fed73e09d06\" (UID: \"c02a623d-feec-4c7b-92c2-3fed73e09d06\") " Dec 03 13:08:15 crc kubenswrapper[5052]: I1203 13:08:15.537633 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"openstack-edpm-multinode\" (UniqueName: \"kubernetes.io/configmap/c02a623d-feec-4c7b-92c2-3fed73e09d06-openstack-edpm-multinode\") pod \"c02a623d-feec-4c7b-92c2-3fed73e09d06\" (UID: \"c02a623d-feec-4c7b-92c2-3fed73e09d06\") " Dec 03 13:08:15 crc kubenswrapper[5052]: I1203 13:08:15.537732 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-b229d\" (UniqueName: \"kubernetes.io/projected/62a2ca08-9a05-4d01-a792-05cedf6a1139-kube-api-access-b229d\") pod \"dnsmasq-dnsmasq-84b9f45d47-74ckd\" (UID: \"62a2ca08-9a05-4d01-a792-05cedf6a1139\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-74ckd" Dec 03 13:08:15 crc kubenswrapper[5052]: I1203 13:08:15.537766 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/62a2ca08-9a05-4d01-a792-05cedf6a1139-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-84b9f45d47-74ckd\" (UID: \"62a2ca08-9a05-4d01-a792-05cedf6a1139\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-74ckd" Dec 03 13:08:15 crc kubenswrapper[5052]: I1203 13:08:15.537800 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/62a2ca08-9a05-4d01-a792-05cedf6a1139-config\") pod \"dnsmasq-dnsmasq-84b9f45d47-74ckd\" (UID: \"62a2ca08-9a05-4d01-a792-05cedf6a1139\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-74ckd" Dec 03 13:08:15 crc kubenswrapper[5052]: I1203 13:08:15.539231 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/62a2ca08-9a05-4d01-a792-05cedf6a1139-config\") pod \"dnsmasq-dnsmasq-84b9f45d47-74ckd\" (UID: \"62a2ca08-9a05-4d01-a792-05cedf6a1139\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-74ckd" Dec 03 13:08:15 crc kubenswrapper[5052]: I1203 13:08:15.541262 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/62a2ca08-9a05-4d01-a792-05cedf6a1139-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-84b9f45d47-74ckd\" (UID: \"62a2ca08-9a05-4d01-a792-05cedf6a1139\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-74ckd" Dec 03 13:08:15 crc kubenswrapper[5052]: I1203 13:08:15.558454 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c02a623d-feec-4c7b-92c2-3fed73e09d06-kube-api-access-fqtr8" (OuterVolumeSpecName: "kube-api-access-fqtr8") pod "c02a623d-feec-4c7b-92c2-3fed73e09d06" (UID: "c02a623d-feec-4c7b-92c2-3fed73e09d06"). InnerVolumeSpecName "kube-api-access-fqtr8". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:08:15 crc kubenswrapper[5052]: I1203 13:08:15.566574 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-b229d\" (UniqueName: \"kubernetes.io/projected/62a2ca08-9a05-4d01-a792-05cedf6a1139-kube-api-access-b229d\") pod \"dnsmasq-dnsmasq-84b9f45d47-74ckd\" (UID: \"62a2ca08-9a05-4d01-a792-05cedf6a1139\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-74ckd" Dec 03 13:08:15 crc kubenswrapper[5052]: I1203 13:08:15.580401 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c02a623d-feec-4c7b-92c2-3fed73e09d06-dnsmasq-svc" (OuterVolumeSpecName: "dnsmasq-svc") pod "c02a623d-feec-4c7b-92c2-3fed73e09d06" (UID: "c02a623d-feec-4c7b-92c2-3fed73e09d06"). InnerVolumeSpecName "dnsmasq-svc". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:08:15 crc kubenswrapper[5052]: I1203 13:08:15.580561 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c02a623d-feec-4c7b-92c2-3fed73e09d06-config" (OuterVolumeSpecName: "config") pod "c02a623d-feec-4c7b-92c2-3fed73e09d06" (UID: "c02a623d-feec-4c7b-92c2-3fed73e09d06"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:08:15 crc kubenswrapper[5052]: I1203 13:08:15.587891 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c02a623d-feec-4c7b-92c2-3fed73e09d06-openstack-edpm-multinode" (OuterVolumeSpecName: "openstack-edpm-multinode") pod "c02a623d-feec-4c7b-92c2-3fed73e09d06" (UID: "c02a623d-feec-4c7b-92c2-3fed73e09d06"). InnerVolumeSpecName "openstack-edpm-multinode". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:08:15 crc kubenswrapper[5052]: I1203 13:08:15.638296 5052 reconciler_common.go:293] "Volume detached for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/c02a623d-feec-4c7b-92c2-3fed73e09d06-dnsmasq-svc\") on node \"crc\" DevicePath \"\"" Dec 03 13:08:15 crc kubenswrapper[5052]: I1203 13:08:15.638331 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c02a623d-feec-4c7b-92c2-3fed73e09d06-config\") on node \"crc\" DevicePath \"\"" Dec 03 13:08:15 crc kubenswrapper[5052]: I1203 13:08:15.638343 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-fqtr8\" (UniqueName: \"kubernetes.io/projected/c02a623d-feec-4c7b-92c2-3fed73e09d06-kube-api-access-fqtr8\") on node \"crc\" DevicePath \"\"" Dec 03 13:08:15 crc kubenswrapper[5052]: I1203 13:08:15.638355 5052 reconciler_common.go:293] "Volume detached for volume \"openstack-edpm-multinode\" (UniqueName: \"kubernetes.io/configmap/c02a623d-feec-4c7b-92c2-3fed73e09d06-openstack-edpm-multinode\") on node \"crc\" DevicePath \"\"" Dec 03 13:08:15 crc kubenswrapper[5052]: I1203 13:08:15.673247 5052 scope.go:117] "RemoveContainer" containerID="dbc100c8081ea1a2f5487429cc9d67de6bc8ba11282b64f248c71275375b0e16" Dec 03 13:08:15 crc kubenswrapper[5052]: E1203 13:08:15.673672 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 13:08:15 crc kubenswrapper[5052]: I1203 13:08:15.800593 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-74ckd" Dec 03 13:08:15 crc kubenswrapper[5052]: I1203 13:08:15.840056 5052 generic.go:334] "Generic (PLEG): container finished" podID="c02a623d-feec-4c7b-92c2-3fed73e09d06" containerID="ec095b0a2ce4b448d9a3ef1c01d89dee455e0b69d5e95ae270355b86b7ae3e8c" exitCode=0 Dec 03 13:08:15 crc kubenswrapper[5052]: I1203 13:08:15.840100 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-5d55f47b6c-xpgvj" event={"ID":"c02a623d-feec-4c7b-92c2-3fed73e09d06","Type":"ContainerDied","Data":"ec095b0a2ce4b448d9a3ef1c01d89dee455e0b69d5e95ae270355b86b7ae3e8c"} Dec 03 13:08:15 crc kubenswrapper[5052]: I1203 13:08:15.840132 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-5d55f47b6c-xpgvj" event={"ID":"c02a623d-feec-4c7b-92c2-3fed73e09d06","Type":"ContainerDied","Data":"8524251498b3d4431566f1869808e1059ddcd6826c8b888493813187faf53d66"} Dec 03 13:08:15 crc kubenswrapper[5052]: I1203 13:08:15.840151 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-5d55f47b6c-xpgvj" Dec 03 13:08:15 crc kubenswrapper[5052]: I1203 13:08:15.840157 5052 scope.go:117] "RemoveContainer" containerID="ec095b0a2ce4b448d9a3ef1c01d89dee455e0b69d5e95ae270355b86b7ae3e8c" Dec 03 13:08:15 crc kubenswrapper[5052]: I1203 13:08:15.869296 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-5d55f47b6c-xpgvj"] Dec 03 13:08:15 crc kubenswrapper[5052]: I1203 13:08:15.870362 5052 scope.go:117] "RemoveContainer" containerID="62f55f44a90a0f4177317b916d9a2e872cb27a3fdd28948ab0132e59dec35e2f" Dec 03 13:08:15 crc kubenswrapper[5052]: I1203 13:08:15.879741 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-5d55f47b6c-xpgvj"] Dec 03 13:08:15 crc kubenswrapper[5052]: I1203 13:08:15.894418 5052 scope.go:117] "RemoveContainer" containerID="ec095b0a2ce4b448d9a3ef1c01d89dee455e0b69d5e95ae270355b86b7ae3e8c" Dec 03 13:08:15 crc kubenswrapper[5052]: E1203 13:08:15.898233 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ec095b0a2ce4b448d9a3ef1c01d89dee455e0b69d5e95ae270355b86b7ae3e8c\": container with ID starting with ec095b0a2ce4b448d9a3ef1c01d89dee455e0b69d5e95ae270355b86b7ae3e8c not found: ID does not exist" containerID="ec095b0a2ce4b448d9a3ef1c01d89dee455e0b69d5e95ae270355b86b7ae3e8c" Dec 03 13:08:15 crc kubenswrapper[5052]: I1203 13:08:15.898299 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ec095b0a2ce4b448d9a3ef1c01d89dee455e0b69d5e95ae270355b86b7ae3e8c"} err="failed to get container status \"ec095b0a2ce4b448d9a3ef1c01d89dee455e0b69d5e95ae270355b86b7ae3e8c\": rpc error: code = NotFound desc = could not find container \"ec095b0a2ce4b448d9a3ef1c01d89dee455e0b69d5e95ae270355b86b7ae3e8c\": container with ID starting with ec095b0a2ce4b448d9a3ef1c01d89dee455e0b69d5e95ae270355b86b7ae3e8c not found: ID does not exist" Dec 03 13:08:15 crc kubenswrapper[5052]: I1203 13:08:15.898336 5052 scope.go:117] "RemoveContainer" containerID="62f55f44a90a0f4177317b916d9a2e872cb27a3fdd28948ab0132e59dec35e2f" Dec 03 13:08:15 crc kubenswrapper[5052]: E1203 13:08:15.899767 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"62f55f44a90a0f4177317b916d9a2e872cb27a3fdd28948ab0132e59dec35e2f\": container with ID starting with 62f55f44a90a0f4177317b916d9a2e872cb27a3fdd28948ab0132e59dec35e2f not found: ID does not exist" containerID="62f55f44a90a0f4177317b916d9a2e872cb27a3fdd28948ab0132e59dec35e2f" Dec 03 13:08:15 crc kubenswrapper[5052]: I1203 13:08:15.899800 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"62f55f44a90a0f4177317b916d9a2e872cb27a3fdd28948ab0132e59dec35e2f"} err="failed to get container status \"62f55f44a90a0f4177317b916d9a2e872cb27a3fdd28948ab0132e59dec35e2f\": rpc error: code = NotFound desc = could not find container \"62f55f44a90a0f4177317b916d9a2e872cb27a3fdd28948ab0132e59dec35e2f\": container with ID starting with 62f55f44a90a0f4177317b916d9a2e872cb27a3fdd28948ab0132e59dec35e2f not found: ID does not exist" Dec 03 13:08:16 crc kubenswrapper[5052]: I1203 13:08:16.277615 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-74ckd"] Dec 03 13:08:16 crc kubenswrapper[5052]: W1203 13:08:16.284092 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod62a2ca08_9a05_4d01_a792_05cedf6a1139.slice/crio-58402d06843f109aea0e1cc6799b40383f767e9d18d59d0cb46cfa243a6d3c3c WatchSource:0}: Error finding container 58402d06843f109aea0e1cc6799b40383f767e9d18d59d0cb46cfa243a6d3c3c: Status 404 returned error can't find the container with id 58402d06843f109aea0e1cc6799b40383f767e9d18d59d0cb46cfa243a6d3c3c Dec 03 13:08:16 crc kubenswrapper[5052]: I1203 13:08:16.850953 5052 generic.go:334] "Generic (PLEG): container finished" podID="62a2ca08-9a05-4d01-a792-05cedf6a1139" containerID="37a1e360eb14306d138db9df18d06ca38c3f1d7d827caa6fb4f25ba1f4327857" exitCode=0 Dec 03 13:08:16 crc kubenswrapper[5052]: I1203 13:08:16.851011 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-74ckd" event={"ID":"62a2ca08-9a05-4d01-a792-05cedf6a1139","Type":"ContainerDied","Data":"37a1e360eb14306d138db9df18d06ca38c3f1d7d827caa6fb4f25ba1f4327857"} Dec 03 13:08:16 crc kubenswrapper[5052]: I1203 13:08:16.851291 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-74ckd" event={"ID":"62a2ca08-9a05-4d01-a792-05cedf6a1139","Type":"ContainerStarted","Data":"58402d06843f109aea0e1cc6799b40383f767e9d18d59d0cb46cfa243a6d3c3c"} Dec 03 13:08:17 crc kubenswrapper[5052]: I1203 13:08:17.683205 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c02a623d-feec-4c7b-92c2-3fed73e09d06" path="/var/lib/kubelet/pods/c02a623d-feec-4c7b-92c2-3fed73e09d06/volumes" Dec 03 13:08:17 crc kubenswrapper[5052]: I1203 13:08:17.864679 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-74ckd" event={"ID":"62a2ca08-9a05-4d01-a792-05cedf6a1139","Type":"ContainerStarted","Data":"b19c24563e2056610e60d5e194ca03d09ea40ab146c7738c775ce3f22d67bdf1"} Dec 03 13:08:17 crc kubenswrapper[5052]: I1203 13:08:17.864860 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-74ckd" Dec 03 13:08:17 crc kubenswrapper[5052]: I1203 13:08:17.885493 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-74ckd" podStartSLOduration=2.885454224 podStartE2EDuration="2.885454224s" podCreationTimestamp="2025-12-03 13:08:15 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 13:08:17.881350767 +0000 UTC m=+6488.469592531" watchObservedRunningTime="2025-12-03 13:08:17.885454224 +0000 UTC m=+6488.473695988" Dec 03 13:08:24 crc kubenswrapper[5052]: I1203 13:08:24.635721 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["crc-storage/crc-storage-crc-rzt2l"] Dec 03 13:08:24 crc kubenswrapper[5052]: I1203 13:08:24.641727 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["crc-storage/crc-storage-crc-rzt2l"] Dec 03 13:08:24 crc kubenswrapper[5052]: I1203 13:08:24.806708 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["crc-storage/crc-storage-crc-6rrhm"] Dec 03 13:08:24 crc kubenswrapper[5052]: E1203 13:08:24.807152 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="c02a623d-feec-4c7b-92c2-3fed73e09d06" containerName="init" Dec 03 13:08:24 crc kubenswrapper[5052]: I1203 13:08:24.807173 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="c02a623d-feec-4c7b-92c2-3fed73e09d06" containerName="init" Dec 03 13:08:24 crc kubenswrapper[5052]: E1203 13:08:24.807191 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="c02a623d-feec-4c7b-92c2-3fed73e09d06" containerName="dnsmasq-dns" Dec 03 13:08:24 crc kubenswrapper[5052]: I1203 13:08:24.807200 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="c02a623d-feec-4c7b-92c2-3fed73e09d06" containerName="dnsmasq-dns" Dec 03 13:08:24 crc kubenswrapper[5052]: I1203 13:08:24.807384 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="c02a623d-feec-4c7b-92c2-3fed73e09d06" containerName="dnsmasq-dns" Dec 03 13:08:24 crc kubenswrapper[5052]: I1203 13:08:24.807993 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="crc-storage/crc-storage-crc-6rrhm" Dec 03 13:08:24 crc kubenswrapper[5052]: I1203 13:08:24.810365 5052 reflector.go:368] Caches populated for *v1.Secret from object-"crc-storage"/"crc-storage-dockercfg-mf76g" Dec 03 13:08:24 crc kubenswrapper[5052]: I1203 13:08:24.810374 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"crc-storage"/"openshift-service-ca.crt" Dec 03 13:08:24 crc kubenswrapper[5052]: I1203 13:08:24.810849 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"crc-storage"/"crc-storage" Dec 03 13:08:24 crc kubenswrapper[5052]: I1203 13:08:24.810941 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"crc-storage"/"kube-root-ca.crt" Dec 03 13:08:24 crc kubenswrapper[5052]: I1203 13:08:24.815563 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["crc-storage/crc-storage-crc-6rrhm"] Dec 03 13:08:24 crc kubenswrapper[5052]: I1203 13:08:24.971273 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b595q\" (UniqueName: \"kubernetes.io/projected/cc2b6ea4-2e38-4fc5-8dfa-2ca1c147a947-kube-api-access-b595q\") pod \"crc-storage-crc-6rrhm\" (UID: \"cc2b6ea4-2e38-4fc5-8dfa-2ca1c147a947\") " pod="crc-storage/crc-storage-crc-6rrhm" Dec 03 13:08:24 crc kubenswrapper[5052]: I1203 13:08:24.971362 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/cc2b6ea4-2e38-4fc5-8dfa-2ca1c147a947-crc-storage\") pod \"crc-storage-crc-6rrhm\" (UID: \"cc2b6ea4-2e38-4fc5-8dfa-2ca1c147a947\") " pod="crc-storage/crc-storage-crc-6rrhm" Dec 03 13:08:24 crc kubenswrapper[5052]: I1203 13:08:24.971412 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/cc2b6ea4-2e38-4fc5-8dfa-2ca1c147a947-node-mnt\") pod \"crc-storage-crc-6rrhm\" (UID: \"cc2b6ea4-2e38-4fc5-8dfa-2ca1c147a947\") " pod="crc-storage/crc-storage-crc-6rrhm" Dec 03 13:08:25 crc kubenswrapper[5052]: I1203 13:08:25.072432 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-b595q\" (UniqueName: \"kubernetes.io/projected/cc2b6ea4-2e38-4fc5-8dfa-2ca1c147a947-kube-api-access-b595q\") pod \"crc-storage-crc-6rrhm\" (UID: \"cc2b6ea4-2e38-4fc5-8dfa-2ca1c147a947\") " pod="crc-storage/crc-storage-crc-6rrhm" Dec 03 13:08:25 crc kubenswrapper[5052]: I1203 13:08:25.072690 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/cc2b6ea4-2e38-4fc5-8dfa-2ca1c147a947-crc-storage\") pod \"crc-storage-crc-6rrhm\" (UID: \"cc2b6ea4-2e38-4fc5-8dfa-2ca1c147a947\") " pod="crc-storage/crc-storage-crc-6rrhm" Dec 03 13:08:25 crc kubenswrapper[5052]: I1203 13:08:25.072754 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/cc2b6ea4-2e38-4fc5-8dfa-2ca1c147a947-node-mnt\") pod \"crc-storage-crc-6rrhm\" (UID: \"cc2b6ea4-2e38-4fc5-8dfa-2ca1c147a947\") " pod="crc-storage/crc-storage-crc-6rrhm" Dec 03 13:08:25 crc kubenswrapper[5052]: I1203 13:08:25.073045 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/cc2b6ea4-2e38-4fc5-8dfa-2ca1c147a947-node-mnt\") pod \"crc-storage-crc-6rrhm\" (UID: \"cc2b6ea4-2e38-4fc5-8dfa-2ca1c147a947\") " pod="crc-storage/crc-storage-crc-6rrhm" Dec 03 13:08:25 crc kubenswrapper[5052]: I1203 13:08:25.075247 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/cc2b6ea4-2e38-4fc5-8dfa-2ca1c147a947-crc-storage\") pod \"crc-storage-crc-6rrhm\" (UID: \"cc2b6ea4-2e38-4fc5-8dfa-2ca1c147a947\") " pod="crc-storage/crc-storage-crc-6rrhm" Dec 03 13:08:25 crc kubenswrapper[5052]: I1203 13:08:25.092472 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-b595q\" (UniqueName: \"kubernetes.io/projected/cc2b6ea4-2e38-4fc5-8dfa-2ca1c147a947-kube-api-access-b595q\") pod \"crc-storage-crc-6rrhm\" (UID: \"cc2b6ea4-2e38-4fc5-8dfa-2ca1c147a947\") " pod="crc-storage/crc-storage-crc-6rrhm" Dec 03 13:08:25 crc kubenswrapper[5052]: I1203 13:08:25.132355 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="crc-storage/crc-storage-crc-6rrhm" Dec 03 13:08:25 crc kubenswrapper[5052]: I1203 13:08:25.566853 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["crc-storage/crc-storage-crc-6rrhm"] Dec 03 13:08:25 crc kubenswrapper[5052]: I1203 13:08:25.575580 5052 provider.go:102] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Dec 03 13:08:25 crc kubenswrapper[5052]: I1203 13:08:25.683397 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="46a3997f-7536-4c23-a718-2e891bcbbd6f" path="/var/lib/kubelet/pods/46a3997f-7536-4c23-a718-2e891bcbbd6f/volumes" Dec 03 13:08:25 crc kubenswrapper[5052]: I1203 13:08:25.802505 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-74ckd" Dec 03 13:08:25 crc kubenswrapper[5052]: I1203 13:08:25.855780 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-964c896d7-ckkr8"] Dec 03 13:08:25 crc kubenswrapper[5052]: I1203 13:08:25.856057 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-964c896d7-ckkr8" podUID="a370691c-0245-4503-809a-7037fd5359c3" containerName="dnsmasq-dns" containerID="cri-o://2b38eb5ba3951b0647676e20f7400ab1178dfbe830d47f2c8baa2007f1d09b01" gracePeriod=10 Dec 03 13:08:25 crc kubenswrapper[5052]: I1203 13:08:25.945764 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="crc-storage/crc-storage-crc-6rrhm" event={"ID":"cc2b6ea4-2e38-4fc5-8dfa-2ca1c147a947","Type":"ContainerStarted","Data":"62ef7e9ffd4c96c70364867bf18ca53d3da98c04a2f9b3b56719dc20702944c2"} Dec 03 13:08:26 crc kubenswrapper[5052]: I1203 13:08:26.785276 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-964c896d7-ckkr8" Dec 03 13:08:26 crc kubenswrapper[5052]: I1203 13:08:26.899889 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/a370691c-0245-4503-809a-7037fd5359c3-dnsmasq-svc\") pod \"a370691c-0245-4503-809a-7037fd5359c3\" (UID: \"a370691c-0245-4503-809a-7037fd5359c3\") " Dec 03 13:08:26 crc kubenswrapper[5052]: I1203 13:08:26.899998 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a370691c-0245-4503-809a-7037fd5359c3-config\") pod \"a370691c-0245-4503-809a-7037fd5359c3\" (UID: \"a370691c-0245-4503-809a-7037fd5359c3\") " Dec 03 13:08:26 crc kubenswrapper[5052]: I1203 13:08:26.900040 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x7jx8\" (UniqueName: \"kubernetes.io/projected/a370691c-0245-4503-809a-7037fd5359c3-kube-api-access-x7jx8\") pod \"a370691c-0245-4503-809a-7037fd5359c3\" (UID: \"a370691c-0245-4503-809a-7037fd5359c3\") " Dec 03 13:08:26 crc kubenswrapper[5052]: I1203 13:08:26.900109 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"openstack-edpm-multinode\" (UniqueName: \"kubernetes.io/configmap/a370691c-0245-4503-809a-7037fd5359c3-openstack-edpm-multinode\") pod \"a370691c-0245-4503-809a-7037fd5359c3\" (UID: \"a370691c-0245-4503-809a-7037fd5359c3\") " Dec 03 13:08:26 crc kubenswrapper[5052]: I1203 13:08:26.906194 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a370691c-0245-4503-809a-7037fd5359c3-kube-api-access-x7jx8" (OuterVolumeSpecName: "kube-api-access-x7jx8") pod "a370691c-0245-4503-809a-7037fd5359c3" (UID: "a370691c-0245-4503-809a-7037fd5359c3"). InnerVolumeSpecName "kube-api-access-x7jx8". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:08:26 crc kubenswrapper[5052]: I1203 13:08:26.938707 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a370691c-0245-4503-809a-7037fd5359c3-dnsmasq-svc" (OuterVolumeSpecName: "dnsmasq-svc") pod "a370691c-0245-4503-809a-7037fd5359c3" (UID: "a370691c-0245-4503-809a-7037fd5359c3"). InnerVolumeSpecName "dnsmasq-svc". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:08:26 crc kubenswrapper[5052]: I1203 13:08:26.939271 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a370691c-0245-4503-809a-7037fd5359c3-openstack-edpm-multinode" (OuterVolumeSpecName: "openstack-edpm-multinode") pod "a370691c-0245-4503-809a-7037fd5359c3" (UID: "a370691c-0245-4503-809a-7037fd5359c3"). InnerVolumeSpecName "openstack-edpm-multinode". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:08:26 crc kubenswrapper[5052]: I1203 13:08:26.939445 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a370691c-0245-4503-809a-7037fd5359c3-config" (OuterVolumeSpecName: "config") pod "a370691c-0245-4503-809a-7037fd5359c3" (UID: "a370691c-0245-4503-809a-7037fd5359c3"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:08:26 crc kubenswrapper[5052]: I1203 13:08:26.957861 5052 generic.go:334] "Generic (PLEG): container finished" podID="cc2b6ea4-2e38-4fc5-8dfa-2ca1c147a947" containerID="0b7a53ad8d1342040ea33fe43e227663b52d1086c7041a72d7d45693dec92e68" exitCode=0 Dec 03 13:08:26 crc kubenswrapper[5052]: I1203 13:08:26.957919 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="crc-storage/crc-storage-crc-6rrhm" event={"ID":"cc2b6ea4-2e38-4fc5-8dfa-2ca1c147a947","Type":"ContainerDied","Data":"0b7a53ad8d1342040ea33fe43e227663b52d1086c7041a72d7d45693dec92e68"} Dec 03 13:08:26 crc kubenswrapper[5052]: I1203 13:08:26.960348 5052 generic.go:334] "Generic (PLEG): container finished" podID="a370691c-0245-4503-809a-7037fd5359c3" containerID="2b38eb5ba3951b0647676e20f7400ab1178dfbe830d47f2c8baa2007f1d09b01" exitCode=0 Dec 03 13:08:26 crc kubenswrapper[5052]: I1203 13:08:26.960381 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-964c896d7-ckkr8" event={"ID":"a370691c-0245-4503-809a-7037fd5359c3","Type":"ContainerDied","Data":"2b38eb5ba3951b0647676e20f7400ab1178dfbe830d47f2c8baa2007f1d09b01"} Dec 03 13:08:26 crc kubenswrapper[5052]: I1203 13:08:26.960405 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-964c896d7-ckkr8" event={"ID":"a370691c-0245-4503-809a-7037fd5359c3","Type":"ContainerDied","Data":"2df392067093a7583eba5e59cc8a3bef83d09e77c42521749b73bd6030bd3dec"} Dec 03 13:08:26 crc kubenswrapper[5052]: I1203 13:08:26.960419 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-964c896d7-ckkr8" Dec 03 13:08:26 crc kubenswrapper[5052]: I1203 13:08:26.960429 5052 scope.go:117] "RemoveContainer" containerID="2b38eb5ba3951b0647676e20f7400ab1178dfbe830d47f2c8baa2007f1d09b01" Dec 03 13:08:26 crc kubenswrapper[5052]: I1203 13:08:26.984478 5052 scope.go:117] "RemoveContainer" containerID="22d5be4016a4643c23661779b5d09c69180dbe313c3d4a9b7f212f6c180902cd" Dec 03 13:08:26 crc kubenswrapper[5052]: I1203 13:08:26.995875 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-964c896d7-ckkr8"] Dec 03 13:08:27 crc kubenswrapper[5052]: I1203 13:08:27.001440 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a370691c-0245-4503-809a-7037fd5359c3-config\") on node \"crc\" DevicePath \"\"" Dec 03 13:08:27 crc kubenswrapper[5052]: I1203 13:08:27.001481 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-x7jx8\" (UniqueName: \"kubernetes.io/projected/a370691c-0245-4503-809a-7037fd5359c3-kube-api-access-x7jx8\") on node \"crc\" DevicePath \"\"" Dec 03 13:08:27 crc kubenswrapper[5052]: I1203 13:08:27.001493 5052 reconciler_common.go:293] "Volume detached for volume \"openstack-edpm-multinode\" (UniqueName: \"kubernetes.io/configmap/a370691c-0245-4503-809a-7037fd5359c3-openstack-edpm-multinode\") on node \"crc\" DevicePath \"\"" Dec 03 13:08:27 crc kubenswrapper[5052]: I1203 13:08:27.001918 5052 reconciler_common.go:293] "Volume detached for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/a370691c-0245-4503-809a-7037fd5359c3-dnsmasq-svc\") on node \"crc\" DevicePath \"\"" Dec 03 13:08:27 crc kubenswrapper[5052]: I1203 13:08:27.003976 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-964c896d7-ckkr8"] Dec 03 13:08:27 crc kubenswrapper[5052]: I1203 13:08:27.021152 5052 scope.go:117] "RemoveContainer" containerID="2b38eb5ba3951b0647676e20f7400ab1178dfbe830d47f2c8baa2007f1d09b01" Dec 03 13:08:27 crc kubenswrapper[5052]: E1203 13:08:27.021839 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2b38eb5ba3951b0647676e20f7400ab1178dfbe830d47f2c8baa2007f1d09b01\": container with ID starting with 2b38eb5ba3951b0647676e20f7400ab1178dfbe830d47f2c8baa2007f1d09b01 not found: ID does not exist" containerID="2b38eb5ba3951b0647676e20f7400ab1178dfbe830d47f2c8baa2007f1d09b01" Dec 03 13:08:27 crc kubenswrapper[5052]: I1203 13:08:27.021884 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2b38eb5ba3951b0647676e20f7400ab1178dfbe830d47f2c8baa2007f1d09b01"} err="failed to get container status \"2b38eb5ba3951b0647676e20f7400ab1178dfbe830d47f2c8baa2007f1d09b01\": rpc error: code = NotFound desc = could not find container \"2b38eb5ba3951b0647676e20f7400ab1178dfbe830d47f2c8baa2007f1d09b01\": container with ID starting with 2b38eb5ba3951b0647676e20f7400ab1178dfbe830d47f2c8baa2007f1d09b01 not found: ID does not exist" Dec 03 13:08:27 crc kubenswrapper[5052]: I1203 13:08:27.021911 5052 scope.go:117] "RemoveContainer" containerID="22d5be4016a4643c23661779b5d09c69180dbe313c3d4a9b7f212f6c180902cd" Dec 03 13:08:27 crc kubenswrapper[5052]: E1203 13:08:27.022437 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"22d5be4016a4643c23661779b5d09c69180dbe313c3d4a9b7f212f6c180902cd\": container with ID starting with 22d5be4016a4643c23661779b5d09c69180dbe313c3d4a9b7f212f6c180902cd not found: ID does not exist" containerID="22d5be4016a4643c23661779b5d09c69180dbe313c3d4a9b7f212f6c180902cd" Dec 03 13:08:27 crc kubenswrapper[5052]: I1203 13:08:27.022483 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"22d5be4016a4643c23661779b5d09c69180dbe313c3d4a9b7f212f6c180902cd"} err="failed to get container status \"22d5be4016a4643c23661779b5d09c69180dbe313c3d4a9b7f212f6c180902cd\": rpc error: code = NotFound desc = could not find container \"22d5be4016a4643c23661779b5d09c69180dbe313c3d4a9b7f212f6c180902cd\": container with ID starting with 22d5be4016a4643c23661779b5d09c69180dbe313c3d4a9b7f212f6c180902cd not found: ID does not exist" Dec 03 13:08:27 crc kubenswrapper[5052]: I1203 13:08:27.684221 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a370691c-0245-4503-809a-7037fd5359c3" path="/var/lib/kubelet/pods/a370691c-0245-4503-809a-7037fd5359c3/volumes" Dec 03 13:08:28 crc kubenswrapper[5052]: I1203 13:08:28.306870 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="crc-storage/crc-storage-crc-6rrhm" Dec 03 13:08:28 crc kubenswrapper[5052]: I1203 13:08:28.423434 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/cc2b6ea4-2e38-4fc5-8dfa-2ca1c147a947-crc-storage\") pod \"cc2b6ea4-2e38-4fc5-8dfa-2ca1c147a947\" (UID: \"cc2b6ea4-2e38-4fc5-8dfa-2ca1c147a947\") " Dec 03 13:08:28 crc kubenswrapper[5052]: I1203 13:08:28.423580 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b595q\" (UniqueName: \"kubernetes.io/projected/cc2b6ea4-2e38-4fc5-8dfa-2ca1c147a947-kube-api-access-b595q\") pod \"cc2b6ea4-2e38-4fc5-8dfa-2ca1c147a947\" (UID: \"cc2b6ea4-2e38-4fc5-8dfa-2ca1c147a947\") " Dec 03 13:08:28 crc kubenswrapper[5052]: I1203 13:08:28.423678 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/cc2b6ea4-2e38-4fc5-8dfa-2ca1c147a947-node-mnt\") pod \"cc2b6ea4-2e38-4fc5-8dfa-2ca1c147a947\" (UID: \"cc2b6ea4-2e38-4fc5-8dfa-2ca1c147a947\") " Dec 03 13:08:28 crc kubenswrapper[5052]: I1203 13:08:28.423926 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/cc2b6ea4-2e38-4fc5-8dfa-2ca1c147a947-node-mnt" (OuterVolumeSpecName: "node-mnt") pod "cc2b6ea4-2e38-4fc5-8dfa-2ca1c147a947" (UID: "cc2b6ea4-2e38-4fc5-8dfa-2ca1c147a947"). InnerVolumeSpecName "node-mnt". PluginName "kubernetes.io/host-path", VolumeGidValue "" Dec 03 13:08:28 crc kubenswrapper[5052]: I1203 13:08:28.427601 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cc2b6ea4-2e38-4fc5-8dfa-2ca1c147a947-kube-api-access-b595q" (OuterVolumeSpecName: "kube-api-access-b595q") pod "cc2b6ea4-2e38-4fc5-8dfa-2ca1c147a947" (UID: "cc2b6ea4-2e38-4fc5-8dfa-2ca1c147a947"). InnerVolumeSpecName "kube-api-access-b595q". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:08:28 crc kubenswrapper[5052]: I1203 13:08:28.446082 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cc2b6ea4-2e38-4fc5-8dfa-2ca1c147a947-crc-storage" (OuterVolumeSpecName: "crc-storage") pod "cc2b6ea4-2e38-4fc5-8dfa-2ca1c147a947" (UID: "cc2b6ea4-2e38-4fc5-8dfa-2ca1c147a947"). InnerVolumeSpecName "crc-storage". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:08:28 crc kubenswrapper[5052]: I1203 13:08:28.525325 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-b595q\" (UniqueName: \"kubernetes.io/projected/cc2b6ea4-2e38-4fc5-8dfa-2ca1c147a947-kube-api-access-b595q\") on node \"crc\" DevicePath \"\"" Dec 03 13:08:28 crc kubenswrapper[5052]: I1203 13:08:28.525368 5052 reconciler_common.go:293] "Volume detached for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/cc2b6ea4-2e38-4fc5-8dfa-2ca1c147a947-node-mnt\") on node \"crc\" DevicePath \"\"" Dec 03 13:08:28 crc kubenswrapper[5052]: I1203 13:08:28.525379 5052 reconciler_common.go:293] "Volume detached for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/cc2b6ea4-2e38-4fc5-8dfa-2ca1c147a947-crc-storage\") on node \"crc\" DevicePath \"\"" Dec 03 13:08:28 crc kubenswrapper[5052]: I1203 13:08:28.980071 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="crc-storage/crc-storage-crc-6rrhm" event={"ID":"cc2b6ea4-2e38-4fc5-8dfa-2ca1c147a947","Type":"ContainerDied","Data":"62ef7e9ffd4c96c70364867bf18ca53d3da98c04a2f9b3b56719dc20702944c2"} Dec 03 13:08:28 crc kubenswrapper[5052]: I1203 13:08:28.980386 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="62ef7e9ffd4c96c70364867bf18ca53d3da98c04a2f9b3b56719dc20702944c2" Dec 03 13:08:28 crc kubenswrapper[5052]: I1203 13:08:28.980142 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="crc-storage/crc-storage-crc-6rrhm" Dec 03 13:08:29 crc kubenswrapper[5052]: I1203 13:08:29.679109 5052 scope.go:117] "RemoveContainer" containerID="dbc100c8081ea1a2f5487429cc9d67de6bc8ba11282b64f248c71275375b0e16" Dec 03 13:08:29 crc kubenswrapper[5052]: E1203 13:08:29.679318 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 13:08:32 crc kubenswrapper[5052]: I1203 13:08:32.114423 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["crc-storage/crc-storage-crc-6rrhm"] Dec 03 13:08:32 crc kubenswrapper[5052]: I1203 13:08:32.120242 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["crc-storage/crc-storage-crc-6rrhm"] Dec 03 13:08:32 crc kubenswrapper[5052]: I1203 13:08:32.269868 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["crc-storage/crc-storage-crc-q7km4"] Dec 03 13:08:32 crc kubenswrapper[5052]: E1203 13:08:32.270249 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="cc2b6ea4-2e38-4fc5-8dfa-2ca1c147a947" containerName="storage" Dec 03 13:08:32 crc kubenswrapper[5052]: I1203 13:08:32.270274 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="cc2b6ea4-2e38-4fc5-8dfa-2ca1c147a947" containerName="storage" Dec 03 13:08:32 crc kubenswrapper[5052]: E1203 13:08:32.270297 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="a370691c-0245-4503-809a-7037fd5359c3" containerName="init" Dec 03 13:08:32 crc kubenswrapper[5052]: I1203 13:08:32.270308 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="a370691c-0245-4503-809a-7037fd5359c3" containerName="init" Dec 03 13:08:32 crc kubenswrapper[5052]: E1203 13:08:32.270334 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="a370691c-0245-4503-809a-7037fd5359c3" containerName="dnsmasq-dns" Dec 03 13:08:32 crc kubenswrapper[5052]: I1203 13:08:32.270343 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="a370691c-0245-4503-809a-7037fd5359c3" containerName="dnsmasq-dns" Dec 03 13:08:32 crc kubenswrapper[5052]: I1203 13:08:32.270540 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="a370691c-0245-4503-809a-7037fd5359c3" containerName="dnsmasq-dns" Dec 03 13:08:32 crc kubenswrapper[5052]: I1203 13:08:32.270565 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="cc2b6ea4-2e38-4fc5-8dfa-2ca1c147a947" containerName="storage" Dec 03 13:08:32 crc kubenswrapper[5052]: I1203 13:08:32.271254 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="crc-storage/crc-storage-crc-q7km4" Dec 03 13:08:32 crc kubenswrapper[5052]: I1203 13:08:32.275019 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"crc-storage"/"openshift-service-ca.crt" Dec 03 13:08:32 crc kubenswrapper[5052]: I1203 13:08:32.275075 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"crc-storage"/"kube-root-ca.crt" Dec 03 13:08:32 crc kubenswrapper[5052]: I1203 13:08:32.275045 5052 reflector.go:368] Caches populated for *v1.Secret from object-"crc-storage"/"crc-storage-dockercfg-mf76g" Dec 03 13:08:32 crc kubenswrapper[5052]: I1203 13:08:32.275718 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"crc-storage"/"crc-storage" Dec 03 13:08:32 crc kubenswrapper[5052]: I1203 13:08:32.280525 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/d23254f9-a1c0-45eb-bd6d-12cc273e6300-node-mnt\") pod \"crc-storage-crc-q7km4\" (UID: \"d23254f9-a1c0-45eb-bd6d-12cc273e6300\") " pod="crc-storage/crc-storage-crc-q7km4" Dec 03 13:08:32 crc kubenswrapper[5052]: I1203 13:08:32.280605 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6bn9g\" (UniqueName: \"kubernetes.io/projected/d23254f9-a1c0-45eb-bd6d-12cc273e6300-kube-api-access-6bn9g\") pod \"crc-storage-crc-q7km4\" (UID: \"d23254f9-a1c0-45eb-bd6d-12cc273e6300\") " pod="crc-storage/crc-storage-crc-q7km4" Dec 03 13:08:32 crc kubenswrapper[5052]: I1203 13:08:32.280646 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/d23254f9-a1c0-45eb-bd6d-12cc273e6300-crc-storage\") pod \"crc-storage-crc-q7km4\" (UID: \"d23254f9-a1c0-45eb-bd6d-12cc273e6300\") " pod="crc-storage/crc-storage-crc-q7km4" Dec 03 13:08:32 crc kubenswrapper[5052]: I1203 13:08:32.282998 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["crc-storage/crc-storage-crc-q7km4"] Dec 03 13:08:32 crc kubenswrapper[5052]: I1203 13:08:32.382029 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/d23254f9-a1c0-45eb-bd6d-12cc273e6300-crc-storage\") pod \"crc-storage-crc-q7km4\" (UID: \"d23254f9-a1c0-45eb-bd6d-12cc273e6300\") " pod="crc-storage/crc-storage-crc-q7km4" Dec 03 13:08:32 crc kubenswrapper[5052]: I1203 13:08:32.382389 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/d23254f9-a1c0-45eb-bd6d-12cc273e6300-node-mnt\") pod \"crc-storage-crc-q7km4\" (UID: \"d23254f9-a1c0-45eb-bd6d-12cc273e6300\") " pod="crc-storage/crc-storage-crc-q7km4" Dec 03 13:08:32 crc kubenswrapper[5052]: I1203 13:08:32.382767 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-6bn9g\" (UniqueName: \"kubernetes.io/projected/d23254f9-a1c0-45eb-bd6d-12cc273e6300-kube-api-access-6bn9g\") pod \"crc-storage-crc-q7km4\" (UID: \"d23254f9-a1c0-45eb-bd6d-12cc273e6300\") " pod="crc-storage/crc-storage-crc-q7km4" Dec 03 13:08:32 crc kubenswrapper[5052]: I1203 13:08:32.383174 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/d23254f9-a1c0-45eb-bd6d-12cc273e6300-crc-storage\") pod \"crc-storage-crc-q7km4\" (UID: \"d23254f9-a1c0-45eb-bd6d-12cc273e6300\") " pod="crc-storage/crc-storage-crc-q7km4" Dec 03 13:08:32 crc kubenswrapper[5052]: I1203 13:08:32.382701 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/d23254f9-a1c0-45eb-bd6d-12cc273e6300-node-mnt\") pod \"crc-storage-crc-q7km4\" (UID: \"d23254f9-a1c0-45eb-bd6d-12cc273e6300\") " pod="crc-storage/crc-storage-crc-q7km4" Dec 03 13:08:32 crc kubenswrapper[5052]: I1203 13:08:32.404288 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-6bn9g\" (UniqueName: \"kubernetes.io/projected/d23254f9-a1c0-45eb-bd6d-12cc273e6300-kube-api-access-6bn9g\") pod \"crc-storage-crc-q7km4\" (UID: \"d23254f9-a1c0-45eb-bd6d-12cc273e6300\") " pod="crc-storage/crc-storage-crc-q7km4" Dec 03 13:08:32 crc kubenswrapper[5052]: I1203 13:08:32.592375 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="crc-storage/crc-storage-crc-q7km4" Dec 03 13:08:33 crc kubenswrapper[5052]: I1203 13:08:33.034505 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["crc-storage/crc-storage-crc-q7km4"] Dec 03 13:08:33 crc kubenswrapper[5052]: W1203 13:08:33.036974 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd23254f9_a1c0_45eb_bd6d_12cc273e6300.slice/crio-1f0ee76b39b2eef274ebdf528cdf6f3e7f167afc28f8c83abaa54b2472b86f1f WatchSource:0}: Error finding container 1f0ee76b39b2eef274ebdf528cdf6f3e7f167afc28f8c83abaa54b2472b86f1f: Status 404 returned error can't find the container with id 1f0ee76b39b2eef274ebdf528cdf6f3e7f167afc28f8c83abaa54b2472b86f1f Dec 03 13:08:33 crc kubenswrapper[5052]: I1203 13:08:33.686124 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cc2b6ea4-2e38-4fc5-8dfa-2ca1c147a947" path="/var/lib/kubelet/pods/cc2b6ea4-2e38-4fc5-8dfa-2ca1c147a947/volumes" Dec 03 13:08:34 crc kubenswrapper[5052]: I1203 13:08:34.031300 5052 generic.go:334] "Generic (PLEG): container finished" podID="d23254f9-a1c0-45eb-bd6d-12cc273e6300" containerID="6019fc94d6b5f73e8d9826bdfe024871298a635aa8224470991ea4f8f488a20a" exitCode=0 Dec 03 13:08:34 crc kubenswrapper[5052]: I1203 13:08:34.031441 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="crc-storage/crc-storage-crc-q7km4" event={"ID":"d23254f9-a1c0-45eb-bd6d-12cc273e6300","Type":"ContainerDied","Data":"6019fc94d6b5f73e8d9826bdfe024871298a635aa8224470991ea4f8f488a20a"} Dec 03 13:08:34 crc kubenswrapper[5052]: I1203 13:08:34.031481 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="crc-storage/crc-storage-crc-q7km4" event={"ID":"d23254f9-a1c0-45eb-bd6d-12cc273e6300","Type":"ContainerStarted","Data":"1f0ee76b39b2eef274ebdf528cdf6f3e7f167afc28f8c83abaa54b2472b86f1f"} Dec 03 13:08:35 crc kubenswrapper[5052]: I1203 13:08:35.419327 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="crc-storage/crc-storage-crc-q7km4" Dec 03 13:08:35 crc kubenswrapper[5052]: I1203 13:08:35.527896 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/d23254f9-a1c0-45eb-bd6d-12cc273e6300-crc-storage\") pod \"d23254f9-a1c0-45eb-bd6d-12cc273e6300\" (UID: \"d23254f9-a1c0-45eb-bd6d-12cc273e6300\") " Dec 03 13:08:35 crc kubenswrapper[5052]: I1203 13:08:35.527985 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6bn9g\" (UniqueName: \"kubernetes.io/projected/d23254f9-a1c0-45eb-bd6d-12cc273e6300-kube-api-access-6bn9g\") pod \"d23254f9-a1c0-45eb-bd6d-12cc273e6300\" (UID: \"d23254f9-a1c0-45eb-bd6d-12cc273e6300\") " Dec 03 13:08:35 crc kubenswrapper[5052]: I1203 13:08:35.528066 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/d23254f9-a1c0-45eb-bd6d-12cc273e6300-node-mnt\") pod \"d23254f9-a1c0-45eb-bd6d-12cc273e6300\" (UID: \"d23254f9-a1c0-45eb-bd6d-12cc273e6300\") " Dec 03 13:08:35 crc kubenswrapper[5052]: I1203 13:08:35.528322 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/d23254f9-a1c0-45eb-bd6d-12cc273e6300-node-mnt" (OuterVolumeSpecName: "node-mnt") pod "d23254f9-a1c0-45eb-bd6d-12cc273e6300" (UID: "d23254f9-a1c0-45eb-bd6d-12cc273e6300"). InnerVolumeSpecName "node-mnt". PluginName "kubernetes.io/host-path", VolumeGidValue "" Dec 03 13:08:35 crc kubenswrapper[5052]: I1203 13:08:35.533650 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d23254f9-a1c0-45eb-bd6d-12cc273e6300-kube-api-access-6bn9g" (OuterVolumeSpecName: "kube-api-access-6bn9g") pod "d23254f9-a1c0-45eb-bd6d-12cc273e6300" (UID: "d23254f9-a1c0-45eb-bd6d-12cc273e6300"). InnerVolumeSpecName "kube-api-access-6bn9g". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:08:35 crc kubenswrapper[5052]: I1203 13:08:35.551593 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d23254f9-a1c0-45eb-bd6d-12cc273e6300-crc-storage" (OuterVolumeSpecName: "crc-storage") pod "d23254f9-a1c0-45eb-bd6d-12cc273e6300" (UID: "d23254f9-a1c0-45eb-bd6d-12cc273e6300"). InnerVolumeSpecName "crc-storage". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:08:35 crc kubenswrapper[5052]: I1203 13:08:35.630990 5052 reconciler_common.go:293] "Volume detached for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/d23254f9-a1c0-45eb-bd6d-12cc273e6300-node-mnt\") on node \"crc\" DevicePath \"\"" Dec 03 13:08:35 crc kubenswrapper[5052]: I1203 13:08:35.631076 5052 reconciler_common.go:293] "Volume detached for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/d23254f9-a1c0-45eb-bd6d-12cc273e6300-crc-storage\") on node \"crc\" DevicePath \"\"" Dec 03 13:08:35 crc kubenswrapper[5052]: I1203 13:08:35.631118 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-6bn9g\" (UniqueName: \"kubernetes.io/projected/d23254f9-a1c0-45eb-bd6d-12cc273e6300-kube-api-access-6bn9g\") on node \"crc\" DevicePath \"\"" Dec 03 13:08:36 crc kubenswrapper[5052]: I1203 13:08:36.048844 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="crc-storage/crc-storage-crc-q7km4" event={"ID":"d23254f9-a1c0-45eb-bd6d-12cc273e6300","Type":"ContainerDied","Data":"1f0ee76b39b2eef274ebdf528cdf6f3e7f167afc28f8c83abaa54b2472b86f1f"} Dec 03 13:08:36 crc kubenswrapper[5052]: I1203 13:08:36.048912 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1f0ee76b39b2eef274ebdf528cdf6f3e7f167afc28f8c83abaa54b2472b86f1f" Dec 03 13:08:36 crc kubenswrapper[5052]: I1203 13:08:36.048929 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="crc-storage/crc-storage-crc-q7km4" Dec 03 13:08:39 crc kubenswrapper[5052]: I1203 13:08:39.016221 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-64864b6d57-qj2vk"] Dec 03 13:08:39 crc kubenswrapper[5052]: E1203 13:08:39.017033 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="d23254f9-a1c0-45eb-bd6d-12cc273e6300" containerName="storage" Dec 03 13:08:39 crc kubenswrapper[5052]: I1203 13:08:39.017053 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="d23254f9-a1c0-45eb-bd6d-12cc273e6300" containerName="storage" Dec 03 13:08:39 crc kubenswrapper[5052]: I1203 13:08:39.017266 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="d23254f9-a1c0-45eb-bd6d-12cc273e6300" containerName="storage" Dec 03 13:08:39 crc kubenswrapper[5052]: I1203 13:08:39.018301 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-64864b6d57-qj2vk" Dec 03 13:08:39 crc kubenswrapper[5052]: I1203 13:08:39.021159 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"edpm-compute-no-nodes" Dec 03 13:08:39 crc kubenswrapper[5052]: I1203 13:08:39.029807 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-64864b6d57-qj2vk"] Dec 03 13:08:39 crc kubenswrapper[5052]: I1203 13:08:39.079340 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"edpm-compute-no-nodes\" (UniqueName: \"kubernetes.io/configmap/cd12a65f-6e18-400d-a3d0-ca62e2afd313-edpm-compute-no-nodes\") pod \"dnsmasq-dnsmasq-64864b6d57-qj2vk\" (UID: \"cd12a65f-6e18-400d-a3d0-ca62e2afd313\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-64864b6d57-qj2vk" Dec 03 13:08:39 crc kubenswrapper[5052]: I1203 13:08:39.079504 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c6fmd\" (UniqueName: \"kubernetes.io/projected/cd12a65f-6e18-400d-a3d0-ca62e2afd313-kube-api-access-c6fmd\") pod \"dnsmasq-dnsmasq-64864b6d57-qj2vk\" (UID: \"cd12a65f-6e18-400d-a3d0-ca62e2afd313\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-64864b6d57-qj2vk" Dec 03 13:08:39 crc kubenswrapper[5052]: I1203 13:08:39.079553 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/cd12a65f-6e18-400d-a3d0-ca62e2afd313-config\") pod \"dnsmasq-dnsmasq-64864b6d57-qj2vk\" (UID: \"cd12a65f-6e18-400d-a3d0-ca62e2afd313\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-64864b6d57-qj2vk" Dec 03 13:08:39 crc kubenswrapper[5052]: I1203 13:08:39.079583 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/cd12a65f-6e18-400d-a3d0-ca62e2afd313-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-64864b6d57-qj2vk\" (UID: \"cd12a65f-6e18-400d-a3d0-ca62e2afd313\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-64864b6d57-qj2vk" Dec 03 13:08:39 crc kubenswrapper[5052]: I1203 13:08:39.180737 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-c6fmd\" (UniqueName: \"kubernetes.io/projected/cd12a65f-6e18-400d-a3d0-ca62e2afd313-kube-api-access-c6fmd\") pod \"dnsmasq-dnsmasq-64864b6d57-qj2vk\" (UID: \"cd12a65f-6e18-400d-a3d0-ca62e2afd313\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-64864b6d57-qj2vk" Dec 03 13:08:39 crc kubenswrapper[5052]: I1203 13:08:39.180785 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/cd12a65f-6e18-400d-a3d0-ca62e2afd313-config\") pod \"dnsmasq-dnsmasq-64864b6d57-qj2vk\" (UID: \"cd12a65f-6e18-400d-a3d0-ca62e2afd313\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-64864b6d57-qj2vk" Dec 03 13:08:39 crc kubenswrapper[5052]: I1203 13:08:39.180813 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/cd12a65f-6e18-400d-a3d0-ca62e2afd313-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-64864b6d57-qj2vk\" (UID: \"cd12a65f-6e18-400d-a3d0-ca62e2afd313\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-64864b6d57-qj2vk" Dec 03 13:08:39 crc kubenswrapper[5052]: I1203 13:08:39.180857 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"edpm-compute-no-nodes\" (UniqueName: \"kubernetes.io/configmap/cd12a65f-6e18-400d-a3d0-ca62e2afd313-edpm-compute-no-nodes\") pod \"dnsmasq-dnsmasq-64864b6d57-qj2vk\" (UID: \"cd12a65f-6e18-400d-a3d0-ca62e2afd313\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-64864b6d57-qj2vk" Dec 03 13:08:39 crc kubenswrapper[5052]: I1203 13:08:39.181820 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"edpm-compute-no-nodes\" (UniqueName: \"kubernetes.io/configmap/cd12a65f-6e18-400d-a3d0-ca62e2afd313-edpm-compute-no-nodes\") pod \"dnsmasq-dnsmasq-64864b6d57-qj2vk\" (UID: \"cd12a65f-6e18-400d-a3d0-ca62e2afd313\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-64864b6d57-qj2vk" Dec 03 13:08:39 crc kubenswrapper[5052]: I1203 13:08:39.181886 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/cd12a65f-6e18-400d-a3d0-ca62e2afd313-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-64864b6d57-qj2vk\" (UID: \"cd12a65f-6e18-400d-a3d0-ca62e2afd313\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-64864b6d57-qj2vk" Dec 03 13:08:39 crc kubenswrapper[5052]: I1203 13:08:39.181903 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/cd12a65f-6e18-400d-a3d0-ca62e2afd313-config\") pod \"dnsmasq-dnsmasq-64864b6d57-qj2vk\" (UID: \"cd12a65f-6e18-400d-a3d0-ca62e2afd313\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-64864b6d57-qj2vk" Dec 03 13:08:39 crc kubenswrapper[5052]: I1203 13:08:39.200333 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-c6fmd\" (UniqueName: \"kubernetes.io/projected/cd12a65f-6e18-400d-a3d0-ca62e2afd313-kube-api-access-c6fmd\") pod \"dnsmasq-dnsmasq-64864b6d57-qj2vk\" (UID: \"cd12a65f-6e18-400d-a3d0-ca62e2afd313\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-64864b6d57-qj2vk" Dec 03 13:08:39 crc kubenswrapper[5052]: I1203 13:08:39.336811 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-64864b6d57-qj2vk" Dec 03 13:08:39 crc kubenswrapper[5052]: I1203 13:08:39.762769 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-64864b6d57-qj2vk"] Dec 03 13:08:40 crc kubenswrapper[5052]: I1203 13:08:40.084834 5052 generic.go:334] "Generic (PLEG): container finished" podID="cd12a65f-6e18-400d-a3d0-ca62e2afd313" containerID="dcd7fe0d5c55c396aef5fa88fdea31a3c34320a7640846776d865a58a8e00703" exitCode=0 Dec 03 13:08:40 crc kubenswrapper[5052]: I1203 13:08:40.084886 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-64864b6d57-qj2vk" event={"ID":"cd12a65f-6e18-400d-a3d0-ca62e2afd313","Type":"ContainerDied","Data":"dcd7fe0d5c55c396aef5fa88fdea31a3c34320a7640846776d865a58a8e00703"} Dec 03 13:08:40 crc kubenswrapper[5052]: I1203 13:08:40.085111 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-64864b6d57-qj2vk" event={"ID":"cd12a65f-6e18-400d-a3d0-ca62e2afd313","Type":"ContainerStarted","Data":"d9f7e73590128800b86f079ab763186da42eed45f88227a6467b4451d8d99b08"} Dec 03 13:08:40 crc kubenswrapper[5052]: I1203 13:08:40.672914 5052 scope.go:117] "RemoveContainer" containerID="dbc100c8081ea1a2f5487429cc9d67de6bc8ba11282b64f248c71275375b0e16" Dec 03 13:08:40 crc kubenswrapper[5052]: E1203 13:08:40.674478 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 13:08:41 crc kubenswrapper[5052]: I1203 13:08:41.095294 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-64864b6d57-qj2vk" event={"ID":"cd12a65f-6e18-400d-a3d0-ca62e2afd313","Type":"ContainerStarted","Data":"3879a9cd7f2731f88cc0512216043312d0f06d7f6c58684086bec12bb9fe4e6c"} Dec 03 13:08:41 crc kubenswrapper[5052]: I1203 13:08:41.095672 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-64864b6d57-qj2vk" Dec 03 13:08:41 crc kubenswrapper[5052]: I1203 13:08:41.115616 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-64864b6d57-qj2vk" podStartSLOduration=3.115596392 podStartE2EDuration="3.115596392s" podCreationTimestamp="2025-12-03 13:08:38 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 13:08:41.109908119 +0000 UTC m=+6511.698149883" watchObservedRunningTime="2025-12-03 13:08:41.115596392 +0000 UTC m=+6511.703838156" Dec 03 13:08:42 crc kubenswrapper[5052]: I1203 13:08:42.892780 5052 scope.go:117] "RemoveContainer" containerID="0ed1179c896e0ce5137ca77e2d09904242ece4c5dbb5564773a0003d6228bcd4" Dec 03 13:08:42 crc kubenswrapper[5052]: I1203 13:08:42.921251 5052 scope.go:117] "RemoveContainer" containerID="851473ffe16e2f7016240764347d2fcc40e389321b7a88e89031ad0e41410688" Dec 03 13:08:42 crc kubenswrapper[5052]: I1203 13:08:42.947772 5052 scope.go:117] "RemoveContainer" containerID="dfeed030359f149998b1eacfc553f122f272079d2f91b4885c2f7c259d4e916c" Dec 03 13:08:42 crc kubenswrapper[5052]: I1203 13:08:42.978626 5052 scope.go:117] "RemoveContainer" containerID="7a70ad232f06ff17ca3e174b43c18a67b0887a52348825e075c4580fe817962a" Dec 03 13:08:43 crc kubenswrapper[5052]: I1203 13:08:43.014884 5052 scope.go:117] "RemoveContainer" containerID="01a7317e32f22df48f58620df8fc79333f9f7e4dcae549076b81833d21183f92" Dec 03 13:08:43 crc kubenswrapper[5052]: I1203 13:08:43.043411 5052 scope.go:117] "RemoveContainer" containerID="a704128af20876fd571a1aed7e617e6f9b02413124b08ae38ffede0cbff7229b" Dec 03 13:08:43 crc kubenswrapper[5052]: I1203 13:08:43.070899 5052 scope.go:117] "RemoveContainer" containerID="c06935f9a7da4acc8f1b8e6b1c55344bff27f44a9b2cb91d2d0103498ab214fd" Dec 03 13:08:43 crc kubenswrapper[5052]: I1203 13:08:43.092849 5052 scope.go:117] "RemoveContainer" containerID="7590a7faf49819a0d4e82cdb779be314368b5b03ac7d8c6d4aabf90175203276" Dec 03 13:08:43 crc kubenswrapper[5052]: I1203 13:08:43.115503 5052 scope.go:117] "RemoveContainer" containerID="30828179aa61724ce0355188958eaf87142cd62b593ab9385d1292b37635d56f" Dec 03 13:08:43 crc kubenswrapper[5052]: I1203 13:08:43.135165 5052 scope.go:117] "RemoveContainer" containerID="5ed07a085c6c74321a783b17fed82c0431e5be607bd95c4d2220c8b12554163d" Dec 03 13:08:43 crc kubenswrapper[5052]: I1203 13:08:43.157379 5052 scope.go:117] "RemoveContainer" containerID="9e4253f194f13ba34b48aaa8ed2cf4fcb7b95dfb4cbf26c38f1e61ae9c3e3b4f" Dec 03 13:08:43 crc kubenswrapper[5052]: I1203 13:08:43.181793 5052 scope.go:117] "RemoveContainer" containerID="3d0f2fc795dd0265b63af29b876dbeb43989b9b0b175be2319f1c0399846fb7e" Dec 03 13:08:49 crc kubenswrapper[5052]: I1203 13:08:49.337704 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-64864b6d57-qj2vk" Dec 03 13:08:49 crc kubenswrapper[5052]: I1203 13:08:49.424428 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-74ckd"] Dec 03 13:08:49 crc kubenswrapper[5052]: I1203 13:08:49.424830 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-74ckd" podUID="62a2ca08-9a05-4d01-a792-05cedf6a1139" containerName="dnsmasq-dns" containerID="cri-o://b19c24563e2056610e60d5e194ca03d09ea40ab146c7738c775ce3f22d67bdf1" gracePeriod=10 Dec 03 13:08:49 crc kubenswrapper[5052]: I1203 13:08:49.863812 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-74ckd" Dec 03 13:08:49 crc kubenswrapper[5052]: I1203 13:08:49.998972 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b229d\" (UniqueName: \"kubernetes.io/projected/62a2ca08-9a05-4d01-a792-05cedf6a1139-kube-api-access-b229d\") pod \"62a2ca08-9a05-4d01-a792-05cedf6a1139\" (UID: \"62a2ca08-9a05-4d01-a792-05cedf6a1139\") " Dec 03 13:08:49 crc kubenswrapper[5052]: I1203 13:08:49.999041 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/62a2ca08-9a05-4d01-a792-05cedf6a1139-config\") pod \"62a2ca08-9a05-4d01-a792-05cedf6a1139\" (UID: \"62a2ca08-9a05-4d01-a792-05cedf6a1139\") " Dec 03 13:08:49 crc kubenswrapper[5052]: I1203 13:08:49.999081 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/62a2ca08-9a05-4d01-a792-05cedf6a1139-dnsmasq-svc\") pod \"62a2ca08-9a05-4d01-a792-05cedf6a1139\" (UID: \"62a2ca08-9a05-4d01-a792-05cedf6a1139\") " Dec 03 13:08:50 crc kubenswrapper[5052]: I1203 13:08:50.004890 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/62a2ca08-9a05-4d01-a792-05cedf6a1139-kube-api-access-b229d" (OuterVolumeSpecName: "kube-api-access-b229d") pod "62a2ca08-9a05-4d01-a792-05cedf6a1139" (UID: "62a2ca08-9a05-4d01-a792-05cedf6a1139"). InnerVolumeSpecName "kube-api-access-b229d". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:08:50 crc kubenswrapper[5052]: I1203 13:08:50.038320 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/62a2ca08-9a05-4d01-a792-05cedf6a1139-dnsmasq-svc" (OuterVolumeSpecName: "dnsmasq-svc") pod "62a2ca08-9a05-4d01-a792-05cedf6a1139" (UID: "62a2ca08-9a05-4d01-a792-05cedf6a1139"). InnerVolumeSpecName "dnsmasq-svc". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:08:50 crc kubenswrapper[5052]: I1203 13:08:50.043196 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/62a2ca08-9a05-4d01-a792-05cedf6a1139-config" (OuterVolumeSpecName: "config") pod "62a2ca08-9a05-4d01-a792-05cedf6a1139" (UID: "62a2ca08-9a05-4d01-a792-05cedf6a1139"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:08:50 crc kubenswrapper[5052]: I1203 13:08:50.100455 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-b229d\" (UniqueName: \"kubernetes.io/projected/62a2ca08-9a05-4d01-a792-05cedf6a1139-kube-api-access-b229d\") on node \"crc\" DevicePath \"\"" Dec 03 13:08:50 crc kubenswrapper[5052]: I1203 13:08:50.100493 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/62a2ca08-9a05-4d01-a792-05cedf6a1139-config\") on node \"crc\" DevicePath \"\"" Dec 03 13:08:50 crc kubenswrapper[5052]: I1203 13:08:50.100507 5052 reconciler_common.go:293] "Volume detached for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/62a2ca08-9a05-4d01-a792-05cedf6a1139-dnsmasq-svc\") on node \"crc\" DevicePath \"\"" Dec 03 13:08:50 crc kubenswrapper[5052]: I1203 13:08:50.188006 5052 generic.go:334] "Generic (PLEG): container finished" podID="62a2ca08-9a05-4d01-a792-05cedf6a1139" containerID="b19c24563e2056610e60d5e194ca03d09ea40ab146c7738c775ce3f22d67bdf1" exitCode=0 Dec 03 13:08:50 crc kubenswrapper[5052]: I1203 13:08:50.188063 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-74ckd" Dec 03 13:08:50 crc kubenswrapper[5052]: I1203 13:08:50.188061 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-74ckd" event={"ID":"62a2ca08-9a05-4d01-a792-05cedf6a1139","Type":"ContainerDied","Data":"b19c24563e2056610e60d5e194ca03d09ea40ab146c7738c775ce3f22d67bdf1"} Dec 03 13:08:50 crc kubenswrapper[5052]: I1203 13:08:50.188186 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-74ckd" event={"ID":"62a2ca08-9a05-4d01-a792-05cedf6a1139","Type":"ContainerDied","Data":"58402d06843f109aea0e1cc6799b40383f767e9d18d59d0cb46cfa243a6d3c3c"} Dec 03 13:08:50 crc kubenswrapper[5052]: I1203 13:08:50.188215 5052 scope.go:117] "RemoveContainer" containerID="b19c24563e2056610e60d5e194ca03d09ea40ab146c7738c775ce3f22d67bdf1" Dec 03 13:08:50 crc kubenswrapper[5052]: I1203 13:08:50.208546 5052 scope.go:117] "RemoveContainer" containerID="37a1e360eb14306d138db9df18d06ca38c3f1d7d827caa6fb4f25ba1f4327857" Dec 03 13:08:50 crc kubenswrapper[5052]: I1203 13:08:50.220559 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-74ckd"] Dec 03 13:08:50 crc kubenswrapper[5052]: I1203 13:08:50.226657 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-74ckd"] Dec 03 13:08:50 crc kubenswrapper[5052]: I1203 13:08:50.248968 5052 scope.go:117] "RemoveContainer" containerID="b19c24563e2056610e60d5e194ca03d09ea40ab146c7738c775ce3f22d67bdf1" Dec 03 13:08:50 crc kubenswrapper[5052]: E1203 13:08:50.249521 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b19c24563e2056610e60d5e194ca03d09ea40ab146c7738c775ce3f22d67bdf1\": container with ID starting with b19c24563e2056610e60d5e194ca03d09ea40ab146c7738c775ce3f22d67bdf1 not found: ID does not exist" containerID="b19c24563e2056610e60d5e194ca03d09ea40ab146c7738c775ce3f22d67bdf1" Dec 03 13:08:50 crc kubenswrapper[5052]: I1203 13:08:50.249566 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b19c24563e2056610e60d5e194ca03d09ea40ab146c7738c775ce3f22d67bdf1"} err="failed to get container status \"b19c24563e2056610e60d5e194ca03d09ea40ab146c7738c775ce3f22d67bdf1\": rpc error: code = NotFound desc = could not find container \"b19c24563e2056610e60d5e194ca03d09ea40ab146c7738c775ce3f22d67bdf1\": container with ID starting with b19c24563e2056610e60d5e194ca03d09ea40ab146c7738c775ce3f22d67bdf1 not found: ID does not exist" Dec 03 13:08:50 crc kubenswrapper[5052]: I1203 13:08:50.249598 5052 scope.go:117] "RemoveContainer" containerID="37a1e360eb14306d138db9df18d06ca38c3f1d7d827caa6fb4f25ba1f4327857" Dec 03 13:08:50 crc kubenswrapper[5052]: E1203 13:08:50.250149 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"37a1e360eb14306d138db9df18d06ca38c3f1d7d827caa6fb4f25ba1f4327857\": container with ID starting with 37a1e360eb14306d138db9df18d06ca38c3f1d7d827caa6fb4f25ba1f4327857 not found: ID does not exist" containerID="37a1e360eb14306d138db9df18d06ca38c3f1d7d827caa6fb4f25ba1f4327857" Dec 03 13:08:50 crc kubenswrapper[5052]: I1203 13:08:50.250209 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"37a1e360eb14306d138db9df18d06ca38c3f1d7d827caa6fb4f25ba1f4327857"} err="failed to get container status \"37a1e360eb14306d138db9df18d06ca38c3f1d7d827caa6fb4f25ba1f4327857\": rpc error: code = NotFound desc = could not find container \"37a1e360eb14306d138db9df18d06ca38c3f1d7d827caa6fb4f25ba1f4327857\": container with ID starting with 37a1e360eb14306d138db9df18d06ca38c3f1d7d827caa6fb4f25ba1f4327857 not found: ID does not exist" Dec 03 13:08:51 crc kubenswrapper[5052]: I1203 13:08:51.682504 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="62a2ca08-9a05-4d01-a792-05cedf6a1139" path="/var/lib/kubelet/pods/62a2ca08-9a05-4d01-a792-05cedf6a1139/volumes" Dec 03 13:08:53 crc kubenswrapper[5052]: I1203 13:08:53.959181 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/kuttl-service-edpm-compute-no-nodes-edpm-compute-no-nodes-48h7r"] Dec 03 13:08:53 crc kubenswrapper[5052]: E1203 13:08:53.959974 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="62a2ca08-9a05-4d01-a792-05cedf6a1139" containerName="dnsmasq-dns" Dec 03 13:08:53 crc kubenswrapper[5052]: I1203 13:08:53.959992 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="62a2ca08-9a05-4d01-a792-05cedf6a1139" containerName="dnsmasq-dns" Dec 03 13:08:53 crc kubenswrapper[5052]: E1203 13:08:53.960023 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="62a2ca08-9a05-4d01-a792-05cedf6a1139" containerName="init" Dec 03 13:08:53 crc kubenswrapper[5052]: I1203 13:08:53.960032 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="62a2ca08-9a05-4d01-a792-05cedf6a1139" containerName="init" Dec 03 13:08:53 crc kubenswrapper[5052]: I1203 13:08:53.960219 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="62a2ca08-9a05-4d01-a792-05cedf6a1139" containerName="dnsmasq-dns" Dec 03 13:08:53 crc kubenswrapper[5052]: I1203 13:08:53.960915 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/kuttl-service-edpm-compute-no-nodes-edpm-compute-no-nodes-48h7r" Dec 03 13:08:53 crc kubenswrapper[5052]: I1203 13:08:53.963307 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"openstack-aee-default-env" Dec 03 13:08:53 crc kubenswrapper[5052]: I1203 13:08:53.963343 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"combined-ca-bundle" Dec 03 13:08:53 crc kubenswrapper[5052]: I1203 13:08:53.963595 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"dataplane-ansible-ssh-private-key-secret" Dec 03 13:08:53 crc kubenswrapper[5052]: I1203 13:08:53.963616 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"kuttl-service-cm-0" Dec 03 13:08:53 crc kubenswrapper[5052]: I1203 13:08:53.963833 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"dataplanenodeset-edpm-compute-no-nodes" Dec 03 13:08:53 crc kubenswrapper[5052]: I1203 13:08:53.963942 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"kuttl-service-cm-2" Dec 03 13:08:53 crc kubenswrapper[5052]: I1203 13:08:53.964120 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"edpm-compute-no-nodes-dockercfg-v82jw" Dec 03 13:08:53 crc kubenswrapper[5052]: I1203 13:08:53.964220 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"kuttl-service-cm-1" Dec 03 13:08:53 crc kubenswrapper[5052]: I1203 13:08:53.978850 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/kuttl-service-edpm-compute-no-nodes-edpm-compute-no-nodes-48h7r"] Dec 03 13:08:54 crc kubenswrapper[5052]: I1203 13:08:54.060965 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kuttl-service-cm-0-0\" (UniqueName: \"kubernetes.io/configmap/117725d5-3490-4470-8278-caa18557de03-kuttl-service-cm-0-0\") pod \"kuttl-service-edpm-compute-no-nodes-edpm-compute-no-nodes-48h7r\" (UID: \"117725d5-3490-4470-8278-caa18557de03\") " pod="openstack-kuttl-tests/kuttl-service-edpm-compute-no-nodes-edpm-compute-no-nodes-48h7r" Dec 03 13:08:54 crc kubenswrapper[5052]: I1203 13:08:54.061039 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kuttl-service-cm-1-1\" (UniqueName: \"kubernetes.io/configmap/117725d5-3490-4470-8278-caa18557de03-kuttl-service-cm-1-1\") pod \"kuttl-service-edpm-compute-no-nodes-edpm-compute-no-nodes-48h7r\" (UID: \"117725d5-3490-4470-8278-caa18557de03\") " pod="openstack-kuttl-tests/kuttl-service-edpm-compute-no-nodes-edpm-compute-no-nodes-48h7r" Dec 03 13:08:54 crc kubenswrapper[5052]: I1203 13:08:54.061066 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kuttl-service-cm-1-2\" (UniqueName: \"kubernetes.io/configmap/117725d5-3490-4470-8278-caa18557de03-kuttl-service-cm-1-2\") pod \"kuttl-service-edpm-compute-no-nodes-edpm-compute-no-nodes-48h7r\" (UID: \"117725d5-3490-4470-8278-caa18557de03\") " pod="openstack-kuttl-tests/kuttl-service-edpm-compute-no-nodes-edpm-compute-no-nodes-48h7r" Dec 03 13:08:54 crc kubenswrapper[5052]: I1203 13:08:54.061117 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kuttl-service-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/117725d5-3490-4470-8278-caa18557de03-kuttl-service-combined-ca-bundle\") pod \"kuttl-service-edpm-compute-no-nodes-edpm-compute-no-nodes-48h7r\" (UID: \"117725d5-3490-4470-8278-caa18557de03\") " pod="openstack-kuttl-tests/kuttl-service-edpm-compute-no-nodes-edpm-compute-no-nodes-48h7r" Dec 03 13:08:54 crc kubenswrapper[5052]: I1203 13:08:54.061143 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kuttl-service-cm-1-0\" (UniqueName: \"kubernetes.io/configmap/117725d5-3490-4470-8278-caa18557de03-kuttl-service-cm-1-0\") pod \"kuttl-service-edpm-compute-no-nodes-edpm-compute-no-nodes-48h7r\" (UID: \"117725d5-3490-4470-8278-caa18557de03\") " pod="openstack-kuttl-tests/kuttl-service-edpm-compute-no-nodes-edpm-compute-no-nodes-48h7r" Dec 03 13:08:54 crc kubenswrapper[5052]: I1203 13:08:54.061161 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kuttl-service-cm-2-0\" (UniqueName: \"kubernetes.io/configmap/117725d5-3490-4470-8278-caa18557de03-kuttl-service-cm-2-0\") pod \"kuttl-service-edpm-compute-no-nodes-edpm-compute-no-nodes-48h7r\" (UID: \"117725d5-3490-4470-8278-caa18557de03\") " pod="openstack-kuttl-tests/kuttl-service-edpm-compute-no-nodes-edpm-compute-no-nodes-48h7r" Dec 03 13:08:54 crc kubenswrapper[5052]: I1203 13:08:54.061183 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/117725d5-3490-4470-8278-caa18557de03-inventory\") pod \"kuttl-service-edpm-compute-no-nodes-edpm-compute-no-nodes-48h7r\" (UID: \"117725d5-3490-4470-8278-caa18557de03\") " pod="openstack-kuttl-tests/kuttl-service-edpm-compute-no-nodes-edpm-compute-no-nodes-48h7r" Dec 03 13:08:54 crc kubenswrapper[5052]: I1203 13:08:54.061235 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vzk6x\" (UniqueName: \"kubernetes.io/projected/117725d5-3490-4470-8278-caa18557de03-kube-api-access-vzk6x\") pod \"kuttl-service-edpm-compute-no-nodes-edpm-compute-no-nodes-48h7r\" (UID: \"117725d5-3490-4470-8278-caa18557de03\") " pod="openstack-kuttl-tests/kuttl-service-edpm-compute-no-nodes-edpm-compute-no-nodes-48h7r" Dec 03 13:08:54 crc kubenswrapper[5052]: I1203 13:08:54.061259 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kuttl-service-cm-0-1\" (UniqueName: \"kubernetes.io/configmap/117725d5-3490-4470-8278-caa18557de03-kuttl-service-cm-0-1\") pod \"kuttl-service-edpm-compute-no-nodes-edpm-compute-no-nodes-48h7r\" (UID: \"117725d5-3490-4470-8278-caa18557de03\") " pod="openstack-kuttl-tests/kuttl-service-edpm-compute-no-nodes-edpm-compute-no-nodes-48h7r" Dec 03 13:08:54 crc kubenswrapper[5052]: I1203 13:08:54.061369 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/117725d5-3490-4470-8278-caa18557de03-ssh-key\") pod \"kuttl-service-edpm-compute-no-nodes-edpm-compute-no-nodes-48h7r\" (UID: \"117725d5-3490-4470-8278-caa18557de03\") " pod="openstack-kuttl-tests/kuttl-service-edpm-compute-no-nodes-edpm-compute-no-nodes-48h7r" Dec 03 13:08:54 crc kubenswrapper[5052]: I1203 13:08:54.061446 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kuttl-service-cm-0-2\" (UniqueName: \"kubernetes.io/configmap/117725d5-3490-4470-8278-caa18557de03-kuttl-service-cm-0-2\") pod \"kuttl-service-edpm-compute-no-nodes-edpm-compute-no-nodes-48h7r\" (UID: \"117725d5-3490-4470-8278-caa18557de03\") " pod="openstack-kuttl-tests/kuttl-service-edpm-compute-no-nodes-edpm-compute-no-nodes-48h7r" Dec 03 13:08:54 crc kubenswrapper[5052]: I1203 13:08:54.162552 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/117725d5-3490-4470-8278-caa18557de03-ssh-key\") pod \"kuttl-service-edpm-compute-no-nodes-edpm-compute-no-nodes-48h7r\" (UID: \"117725d5-3490-4470-8278-caa18557de03\") " pod="openstack-kuttl-tests/kuttl-service-edpm-compute-no-nodes-edpm-compute-no-nodes-48h7r" Dec 03 13:08:54 crc kubenswrapper[5052]: I1203 13:08:54.162608 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kuttl-service-cm-0-2\" (UniqueName: \"kubernetes.io/configmap/117725d5-3490-4470-8278-caa18557de03-kuttl-service-cm-0-2\") pod \"kuttl-service-edpm-compute-no-nodes-edpm-compute-no-nodes-48h7r\" (UID: \"117725d5-3490-4470-8278-caa18557de03\") " pod="openstack-kuttl-tests/kuttl-service-edpm-compute-no-nodes-edpm-compute-no-nodes-48h7r" Dec 03 13:08:54 crc kubenswrapper[5052]: I1203 13:08:54.162643 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kuttl-service-cm-0-0\" (UniqueName: \"kubernetes.io/configmap/117725d5-3490-4470-8278-caa18557de03-kuttl-service-cm-0-0\") pod \"kuttl-service-edpm-compute-no-nodes-edpm-compute-no-nodes-48h7r\" (UID: \"117725d5-3490-4470-8278-caa18557de03\") " pod="openstack-kuttl-tests/kuttl-service-edpm-compute-no-nodes-edpm-compute-no-nodes-48h7r" Dec 03 13:08:54 crc kubenswrapper[5052]: I1203 13:08:54.162689 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kuttl-service-cm-1-1\" (UniqueName: \"kubernetes.io/configmap/117725d5-3490-4470-8278-caa18557de03-kuttl-service-cm-1-1\") pod \"kuttl-service-edpm-compute-no-nodes-edpm-compute-no-nodes-48h7r\" (UID: \"117725d5-3490-4470-8278-caa18557de03\") " pod="openstack-kuttl-tests/kuttl-service-edpm-compute-no-nodes-edpm-compute-no-nodes-48h7r" Dec 03 13:08:54 crc kubenswrapper[5052]: I1203 13:08:54.162714 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kuttl-service-cm-1-2\" (UniqueName: \"kubernetes.io/configmap/117725d5-3490-4470-8278-caa18557de03-kuttl-service-cm-1-2\") pod \"kuttl-service-edpm-compute-no-nodes-edpm-compute-no-nodes-48h7r\" (UID: \"117725d5-3490-4470-8278-caa18557de03\") " pod="openstack-kuttl-tests/kuttl-service-edpm-compute-no-nodes-edpm-compute-no-nodes-48h7r" Dec 03 13:08:54 crc kubenswrapper[5052]: I1203 13:08:54.162745 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kuttl-service-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/117725d5-3490-4470-8278-caa18557de03-kuttl-service-combined-ca-bundle\") pod \"kuttl-service-edpm-compute-no-nodes-edpm-compute-no-nodes-48h7r\" (UID: \"117725d5-3490-4470-8278-caa18557de03\") " pod="openstack-kuttl-tests/kuttl-service-edpm-compute-no-nodes-edpm-compute-no-nodes-48h7r" Dec 03 13:08:54 crc kubenswrapper[5052]: I1203 13:08:54.162769 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kuttl-service-cm-1-0\" (UniqueName: \"kubernetes.io/configmap/117725d5-3490-4470-8278-caa18557de03-kuttl-service-cm-1-0\") pod \"kuttl-service-edpm-compute-no-nodes-edpm-compute-no-nodes-48h7r\" (UID: \"117725d5-3490-4470-8278-caa18557de03\") " pod="openstack-kuttl-tests/kuttl-service-edpm-compute-no-nodes-edpm-compute-no-nodes-48h7r" Dec 03 13:08:54 crc kubenswrapper[5052]: I1203 13:08:54.162787 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kuttl-service-cm-2-0\" (UniqueName: \"kubernetes.io/configmap/117725d5-3490-4470-8278-caa18557de03-kuttl-service-cm-2-0\") pod \"kuttl-service-edpm-compute-no-nodes-edpm-compute-no-nodes-48h7r\" (UID: \"117725d5-3490-4470-8278-caa18557de03\") " pod="openstack-kuttl-tests/kuttl-service-edpm-compute-no-nodes-edpm-compute-no-nodes-48h7r" Dec 03 13:08:54 crc kubenswrapper[5052]: I1203 13:08:54.162806 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/117725d5-3490-4470-8278-caa18557de03-inventory\") pod \"kuttl-service-edpm-compute-no-nodes-edpm-compute-no-nodes-48h7r\" (UID: \"117725d5-3490-4470-8278-caa18557de03\") " pod="openstack-kuttl-tests/kuttl-service-edpm-compute-no-nodes-edpm-compute-no-nodes-48h7r" Dec 03 13:08:54 crc kubenswrapper[5052]: I1203 13:08:54.162850 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-vzk6x\" (UniqueName: \"kubernetes.io/projected/117725d5-3490-4470-8278-caa18557de03-kube-api-access-vzk6x\") pod \"kuttl-service-edpm-compute-no-nodes-edpm-compute-no-nodes-48h7r\" (UID: \"117725d5-3490-4470-8278-caa18557de03\") " pod="openstack-kuttl-tests/kuttl-service-edpm-compute-no-nodes-edpm-compute-no-nodes-48h7r" Dec 03 13:08:54 crc kubenswrapper[5052]: I1203 13:08:54.162871 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kuttl-service-cm-0-1\" (UniqueName: \"kubernetes.io/configmap/117725d5-3490-4470-8278-caa18557de03-kuttl-service-cm-0-1\") pod \"kuttl-service-edpm-compute-no-nodes-edpm-compute-no-nodes-48h7r\" (UID: \"117725d5-3490-4470-8278-caa18557de03\") " pod="openstack-kuttl-tests/kuttl-service-edpm-compute-no-nodes-edpm-compute-no-nodes-48h7r" Dec 03 13:08:54 crc kubenswrapper[5052]: I1203 13:08:54.164053 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kuttl-service-cm-1-1\" (UniqueName: \"kubernetes.io/configmap/117725d5-3490-4470-8278-caa18557de03-kuttl-service-cm-1-1\") pod \"kuttl-service-edpm-compute-no-nodes-edpm-compute-no-nodes-48h7r\" (UID: \"117725d5-3490-4470-8278-caa18557de03\") " pod="openstack-kuttl-tests/kuttl-service-edpm-compute-no-nodes-edpm-compute-no-nodes-48h7r" Dec 03 13:08:54 crc kubenswrapper[5052]: I1203 13:08:54.164106 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kuttl-service-cm-0-1\" (UniqueName: \"kubernetes.io/configmap/117725d5-3490-4470-8278-caa18557de03-kuttl-service-cm-0-1\") pod \"kuttl-service-edpm-compute-no-nodes-edpm-compute-no-nodes-48h7r\" (UID: \"117725d5-3490-4470-8278-caa18557de03\") " pod="openstack-kuttl-tests/kuttl-service-edpm-compute-no-nodes-edpm-compute-no-nodes-48h7r" Dec 03 13:08:54 crc kubenswrapper[5052]: I1203 13:08:54.164240 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kuttl-service-cm-0-0\" (UniqueName: \"kubernetes.io/configmap/117725d5-3490-4470-8278-caa18557de03-kuttl-service-cm-0-0\") pod \"kuttl-service-edpm-compute-no-nodes-edpm-compute-no-nodes-48h7r\" (UID: \"117725d5-3490-4470-8278-caa18557de03\") " pod="openstack-kuttl-tests/kuttl-service-edpm-compute-no-nodes-edpm-compute-no-nodes-48h7r" Dec 03 13:08:54 crc kubenswrapper[5052]: I1203 13:08:54.164355 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kuttl-service-cm-2-0\" (UniqueName: \"kubernetes.io/configmap/117725d5-3490-4470-8278-caa18557de03-kuttl-service-cm-2-0\") pod \"kuttl-service-edpm-compute-no-nodes-edpm-compute-no-nodes-48h7r\" (UID: \"117725d5-3490-4470-8278-caa18557de03\") " pod="openstack-kuttl-tests/kuttl-service-edpm-compute-no-nodes-edpm-compute-no-nodes-48h7r" Dec 03 13:08:54 crc kubenswrapper[5052]: I1203 13:08:54.164395 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kuttl-service-cm-1-2\" (UniqueName: \"kubernetes.io/configmap/117725d5-3490-4470-8278-caa18557de03-kuttl-service-cm-1-2\") pod \"kuttl-service-edpm-compute-no-nodes-edpm-compute-no-nodes-48h7r\" (UID: \"117725d5-3490-4470-8278-caa18557de03\") " pod="openstack-kuttl-tests/kuttl-service-edpm-compute-no-nodes-edpm-compute-no-nodes-48h7r" Dec 03 13:08:54 crc kubenswrapper[5052]: I1203 13:08:54.164447 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kuttl-service-cm-0-2\" (UniqueName: \"kubernetes.io/configmap/117725d5-3490-4470-8278-caa18557de03-kuttl-service-cm-0-2\") pod \"kuttl-service-edpm-compute-no-nodes-edpm-compute-no-nodes-48h7r\" (UID: \"117725d5-3490-4470-8278-caa18557de03\") " pod="openstack-kuttl-tests/kuttl-service-edpm-compute-no-nodes-edpm-compute-no-nodes-48h7r" Dec 03 13:08:54 crc kubenswrapper[5052]: I1203 13:08:54.164448 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kuttl-service-cm-1-0\" (UniqueName: \"kubernetes.io/configmap/117725d5-3490-4470-8278-caa18557de03-kuttl-service-cm-1-0\") pod \"kuttl-service-edpm-compute-no-nodes-edpm-compute-no-nodes-48h7r\" (UID: \"117725d5-3490-4470-8278-caa18557de03\") " pod="openstack-kuttl-tests/kuttl-service-edpm-compute-no-nodes-edpm-compute-no-nodes-48h7r" Dec 03 13:08:54 crc kubenswrapper[5052]: I1203 13:08:54.168531 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/117725d5-3490-4470-8278-caa18557de03-ssh-key\") pod \"kuttl-service-edpm-compute-no-nodes-edpm-compute-no-nodes-48h7r\" (UID: \"117725d5-3490-4470-8278-caa18557de03\") " pod="openstack-kuttl-tests/kuttl-service-edpm-compute-no-nodes-edpm-compute-no-nodes-48h7r" Dec 03 13:08:54 crc kubenswrapper[5052]: I1203 13:08:54.177372 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kuttl-service-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/117725d5-3490-4470-8278-caa18557de03-kuttl-service-combined-ca-bundle\") pod \"kuttl-service-edpm-compute-no-nodes-edpm-compute-no-nodes-48h7r\" (UID: \"117725d5-3490-4470-8278-caa18557de03\") " pod="openstack-kuttl-tests/kuttl-service-edpm-compute-no-nodes-edpm-compute-no-nodes-48h7r" Dec 03 13:08:54 crc kubenswrapper[5052]: I1203 13:08:54.177411 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/117725d5-3490-4470-8278-caa18557de03-inventory\") pod \"kuttl-service-edpm-compute-no-nodes-edpm-compute-no-nodes-48h7r\" (UID: \"117725d5-3490-4470-8278-caa18557de03\") " pod="openstack-kuttl-tests/kuttl-service-edpm-compute-no-nodes-edpm-compute-no-nodes-48h7r" Dec 03 13:08:54 crc kubenswrapper[5052]: I1203 13:08:54.179184 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-vzk6x\" (UniqueName: \"kubernetes.io/projected/117725d5-3490-4470-8278-caa18557de03-kube-api-access-vzk6x\") pod \"kuttl-service-edpm-compute-no-nodes-edpm-compute-no-nodes-48h7r\" (UID: \"117725d5-3490-4470-8278-caa18557de03\") " pod="openstack-kuttl-tests/kuttl-service-edpm-compute-no-nodes-edpm-compute-no-nodes-48h7r" Dec 03 13:08:54 crc kubenswrapper[5052]: I1203 13:08:54.283408 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/kuttl-service-edpm-compute-no-nodes-edpm-compute-no-nodes-48h7r" Dec 03 13:08:54 crc kubenswrapper[5052]: I1203 13:08:54.772108 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/kuttl-service-edpm-compute-no-nodes-edpm-compute-no-nodes-48h7r"] Dec 03 13:08:55 crc kubenswrapper[5052]: I1203 13:08:55.226994 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/kuttl-service-edpm-compute-no-nodes-edpm-compute-no-nodes-48h7r" event={"ID":"117725d5-3490-4470-8278-caa18557de03","Type":"ContainerStarted","Data":"8a558ecc93b6d28c8a4c94ea0ff1d039785064a9a79b57df7468fc221f708da5"} Dec 03 13:08:55 crc kubenswrapper[5052]: I1203 13:08:55.673818 5052 scope.go:117] "RemoveContainer" containerID="dbc100c8081ea1a2f5487429cc9d67de6bc8ba11282b64f248c71275375b0e16" Dec 03 13:08:55 crc kubenswrapper[5052]: E1203 13:08:55.673993 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 13:08:58 crc kubenswrapper[5052]: I1203 13:08:58.251792 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/kuttl-service-edpm-compute-no-nodes-edpm-compute-no-nodes-48h7r" event={"ID":"117725d5-3490-4470-8278-caa18557de03","Type":"ContainerStarted","Data":"745ceeadda889b09b88ee267ee88537143aa04cc500e41c122373ca94b5bc0e9"} Dec 03 13:08:58 crc kubenswrapper[5052]: I1203 13:08:58.271892 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/kuttl-service-edpm-compute-no-nodes-edpm-compute-no-nodes-48h7r" podStartSLOduration=3.001941764 podStartE2EDuration="5.27186975s" podCreationTimestamp="2025-12-03 13:08:53 +0000 UTC" firstStartedPulling="2025-12-03 13:08:54.771160136 +0000 UTC m=+6525.359401900" lastFinishedPulling="2025-12-03 13:08:57.041088122 +0000 UTC m=+6527.629329886" observedRunningTime="2025-12-03 13:08:58.267216377 +0000 UTC m=+6528.855458141" watchObservedRunningTime="2025-12-03 13:08:58.27186975 +0000 UTC m=+6528.860111534" Dec 03 13:09:00 crc kubenswrapper[5052]: I1203 13:09:00.098273 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/kuttl-service-edpm-compute-no-nodes-edpm-compute-no-nodes-48h7r"] Dec 03 13:09:00 crc kubenswrapper[5052]: I1203 13:09:00.172721 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-xkzkn"] Dec 03 13:09:00 crc kubenswrapper[5052]: I1203 13:09:00.174142 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-xkzkn" Dec 03 13:09:00 crc kubenswrapper[5052]: I1203 13:09:00.179249 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7k6cb\" (UniqueName: \"kubernetes.io/projected/81271079-f185-486e-9c3e-f14df3ee0183-kube-api-access-7k6cb\") pod \"dnsmasq-dnsmasq-84b9f45d47-xkzkn\" (UID: \"81271079-f185-486e-9c3e-f14df3ee0183\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-xkzkn" Dec 03 13:09:00 crc kubenswrapper[5052]: I1203 13:09:00.179313 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/81271079-f185-486e-9c3e-f14df3ee0183-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-84b9f45d47-xkzkn\" (UID: \"81271079-f185-486e-9c3e-f14df3ee0183\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-xkzkn" Dec 03 13:09:00 crc kubenswrapper[5052]: I1203 13:09:00.179510 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/81271079-f185-486e-9c3e-f14df3ee0183-config\") pod \"dnsmasq-dnsmasq-84b9f45d47-xkzkn\" (UID: \"81271079-f185-486e-9c3e-f14df3ee0183\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-xkzkn" Dec 03 13:09:00 crc kubenswrapper[5052]: I1203 13:09:00.189939 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-xkzkn"] Dec 03 13:09:00 crc kubenswrapper[5052]: I1203 13:09:00.268475 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/kuttl-service-edpm-compute-no-nodes-edpm-compute-no-nodes-48h7r" podUID="117725d5-3490-4470-8278-caa18557de03" containerName="kuttl-service-edpm-compute-no-nodes-edpm-compute-no-nodes" containerID="cri-o://745ceeadda889b09b88ee267ee88537143aa04cc500e41c122373ca94b5bc0e9" gracePeriod=30 Dec 03 13:09:00 crc kubenswrapper[5052]: I1203 13:09:00.281174 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/81271079-f185-486e-9c3e-f14df3ee0183-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-84b9f45d47-xkzkn\" (UID: \"81271079-f185-486e-9c3e-f14df3ee0183\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-xkzkn" Dec 03 13:09:00 crc kubenswrapper[5052]: I1203 13:09:00.281277 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/81271079-f185-486e-9c3e-f14df3ee0183-config\") pod \"dnsmasq-dnsmasq-84b9f45d47-xkzkn\" (UID: \"81271079-f185-486e-9c3e-f14df3ee0183\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-xkzkn" Dec 03 13:09:00 crc kubenswrapper[5052]: I1203 13:09:00.281372 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-7k6cb\" (UniqueName: \"kubernetes.io/projected/81271079-f185-486e-9c3e-f14df3ee0183-kube-api-access-7k6cb\") pod \"dnsmasq-dnsmasq-84b9f45d47-xkzkn\" (UID: \"81271079-f185-486e-9c3e-f14df3ee0183\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-xkzkn" Dec 03 13:09:00 crc kubenswrapper[5052]: I1203 13:09:00.282250 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/81271079-f185-486e-9c3e-f14df3ee0183-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-84b9f45d47-xkzkn\" (UID: \"81271079-f185-486e-9c3e-f14df3ee0183\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-xkzkn" Dec 03 13:09:00 crc kubenswrapper[5052]: I1203 13:09:00.282250 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/81271079-f185-486e-9c3e-f14df3ee0183-config\") pod \"dnsmasq-dnsmasq-84b9f45d47-xkzkn\" (UID: \"81271079-f185-486e-9c3e-f14df3ee0183\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-xkzkn" Dec 03 13:09:00 crc kubenswrapper[5052]: I1203 13:09:00.300633 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-7k6cb\" (UniqueName: \"kubernetes.io/projected/81271079-f185-486e-9c3e-f14df3ee0183-kube-api-access-7k6cb\") pod \"dnsmasq-dnsmasq-84b9f45d47-xkzkn\" (UID: \"81271079-f185-486e-9c3e-f14df3ee0183\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-xkzkn" Dec 03 13:09:00 crc kubenswrapper[5052]: I1203 13:09:00.493100 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-xkzkn" Dec 03 13:09:00 crc kubenswrapper[5052]: I1203 13:09:00.976305 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-xkzkn"] Dec 03 13:09:00 crc kubenswrapper[5052]: W1203 13:09:00.980847 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod81271079_f185_486e_9c3e_f14df3ee0183.slice/crio-be239e67866efa697fe7dd954f9f198d082f656bc2c1e9265b42d216aedbe868 WatchSource:0}: Error finding container be239e67866efa697fe7dd954f9f198d082f656bc2c1e9265b42d216aedbe868: Status 404 returned error can't find the container with id be239e67866efa697fe7dd954f9f198d082f656bc2c1e9265b42d216aedbe868 Dec 03 13:09:01 crc kubenswrapper[5052]: I1203 13:09:01.279941 5052 generic.go:334] "Generic (PLEG): container finished" podID="81271079-f185-486e-9c3e-f14df3ee0183" containerID="ca89665a5e1fd6408c4ffd75fa1509f48241c6743e731c94c825e2f6dd2079b3" exitCode=0 Dec 03 13:09:01 crc kubenswrapper[5052]: I1203 13:09:01.280003 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-xkzkn" event={"ID":"81271079-f185-486e-9c3e-f14df3ee0183","Type":"ContainerDied","Data":"ca89665a5e1fd6408c4ffd75fa1509f48241c6743e731c94c825e2f6dd2079b3"} Dec 03 13:09:01 crc kubenswrapper[5052]: I1203 13:09:01.280249 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-xkzkn" event={"ID":"81271079-f185-486e-9c3e-f14df3ee0183","Type":"ContainerStarted","Data":"be239e67866efa697fe7dd954f9f198d082f656bc2c1e9265b42d216aedbe868"} Dec 03 13:09:02 crc kubenswrapper[5052]: I1203 13:09:02.291368 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-xkzkn" event={"ID":"81271079-f185-486e-9c3e-f14df3ee0183","Type":"ContainerStarted","Data":"5a23cdcb537efb9a57b1623406603895c39ab50e8e6028abec71b5f4e6e4a886"} Dec 03 13:09:02 crc kubenswrapper[5052]: I1203 13:09:02.291850 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-xkzkn" Dec 03 13:09:02 crc kubenswrapper[5052]: I1203 13:09:02.311975 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-xkzkn" podStartSLOduration=2.31195816 podStartE2EDuration="2.31195816s" podCreationTimestamp="2025-12-03 13:09:00 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 13:09:02.306794492 +0000 UTC m=+6532.895036256" watchObservedRunningTime="2025-12-03 13:09:02.31195816 +0000 UTC m=+6532.900199924" Dec 03 13:09:06 crc kubenswrapper[5052]: I1203 13:09:06.673508 5052 scope.go:117] "RemoveContainer" containerID="dbc100c8081ea1a2f5487429cc9d67de6bc8ba11282b64f248c71275375b0e16" Dec 03 13:09:06 crc kubenswrapper[5052]: E1203 13:09:06.674234 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 13:09:08 crc kubenswrapper[5052]: I1203 13:09:08.252651 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["crc-storage/crc-storage-crc-q7km4"] Dec 03 13:09:08 crc kubenswrapper[5052]: I1203 13:09:08.272725 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["crc-storage/crc-storage-crc-q7km4"] Dec 03 13:09:08 crc kubenswrapper[5052]: I1203 13:09:08.342791 5052 generic.go:334] "Generic (PLEG): container finished" podID="117725d5-3490-4470-8278-caa18557de03" containerID="745ceeadda889b09b88ee267ee88537143aa04cc500e41c122373ca94b5bc0e9" exitCode=254 Dec 03 13:09:08 crc kubenswrapper[5052]: I1203 13:09:08.342843 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/kuttl-service-edpm-compute-no-nodes-edpm-compute-no-nodes-48h7r" event={"ID":"117725d5-3490-4470-8278-caa18557de03","Type":"ContainerDied","Data":"745ceeadda889b09b88ee267ee88537143aa04cc500e41c122373ca94b5bc0e9"} Dec 03 13:09:08 crc kubenswrapper[5052]: I1203 13:09:08.402741 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["crc-storage/crc-storage-crc-9ctxf"] Dec 03 13:09:08 crc kubenswrapper[5052]: I1203 13:09:08.403776 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="crc-storage/crc-storage-crc-9ctxf" Dec 03 13:09:08 crc kubenswrapper[5052]: I1203 13:09:08.406326 5052 reflector.go:368] Caches populated for *v1.Secret from object-"crc-storage"/"crc-storage-dockercfg-mf76g" Dec 03 13:09:08 crc kubenswrapper[5052]: I1203 13:09:08.411795 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["crc-storage/crc-storage-crc-9ctxf"] Dec 03 13:09:08 crc kubenswrapper[5052]: I1203 13:09:08.412329 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/71d2103c-ea23-404c-80b0-d8c54569ffd5-node-mnt\") pod \"crc-storage-crc-9ctxf\" (UID: \"71d2103c-ea23-404c-80b0-d8c54569ffd5\") " pod="crc-storage/crc-storage-crc-9ctxf" Dec 03 13:09:08 crc kubenswrapper[5052]: I1203 13:09:08.412376 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zzzpc\" (UniqueName: \"kubernetes.io/projected/71d2103c-ea23-404c-80b0-d8c54569ffd5-kube-api-access-zzzpc\") pod \"crc-storage-crc-9ctxf\" (UID: \"71d2103c-ea23-404c-80b0-d8c54569ffd5\") " pod="crc-storage/crc-storage-crc-9ctxf" Dec 03 13:09:08 crc kubenswrapper[5052]: I1203 13:09:08.412440 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/71d2103c-ea23-404c-80b0-d8c54569ffd5-crc-storage\") pod \"crc-storage-crc-9ctxf\" (UID: \"71d2103c-ea23-404c-80b0-d8c54569ffd5\") " pod="crc-storage/crc-storage-crc-9ctxf" Dec 03 13:09:08 crc kubenswrapper[5052]: I1203 13:09:08.412894 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"crc-storage"/"kube-root-ca.crt" Dec 03 13:09:08 crc kubenswrapper[5052]: I1203 13:09:08.413132 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"crc-storage"/"openshift-service-ca.crt" Dec 03 13:09:08 crc kubenswrapper[5052]: I1203 13:09:08.413259 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"crc-storage"/"crc-storage" Dec 03 13:09:08 crc kubenswrapper[5052]: I1203 13:09:08.516481 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-zzzpc\" (UniqueName: \"kubernetes.io/projected/71d2103c-ea23-404c-80b0-d8c54569ffd5-kube-api-access-zzzpc\") pod \"crc-storage-crc-9ctxf\" (UID: \"71d2103c-ea23-404c-80b0-d8c54569ffd5\") " pod="crc-storage/crc-storage-crc-9ctxf" Dec 03 13:09:08 crc kubenswrapper[5052]: I1203 13:09:08.516874 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/71d2103c-ea23-404c-80b0-d8c54569ffd5-crc-storage\") pod \"crc-storage-crc-9ctxf\" (UID: \"71d2103c-ea23-404c-80b0-d8c54569ffd5\") " pod="crc-storage/crc-storage-crc-9ctxf" Dec 03 13:09:08 crc kubenswrapper[5052]: I1203 13:09:08.516917 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/71d2103c-ea23-404c-80b0-d8c54569ffd5-node-mnt\") pod \"crc-storage-crc-9ctxf\" (UID: \"71d2103c-ea23-404c-80b0-d8c54569ffd5\") " pod="crc-storage/crc-storage-crc-9ctxf" Dec 03 13:09:08 crc kubenswrapper[5052]: I1203 13:09:08.517167 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/71d2103c-ea23-404c-80b0-d8c54569ffd5-node-mnt\") pod \"crc-storage-crc-9ctxf\" (UID: \"71d2103c-ea23-404c-80b0-d8c54569ffd5\") " pod="crc-storage/crc-storage-crc-9ctxf" Dec 03 13:09:08 crc kubenswrapper[5052]: I1203 13:09:08.517758 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/71d2103c-ea23-404c-80b0-d8c54569ffd5-crc-storage\") pod \"crc-storage-crc-9ctxf\" (UID: \"71d2103c-ea23-404c-80b0-d8c54569ffd5\") " pod="crc-storage/crc-storage-crc-9ctxf" Dec 03 13:09:08 crc kubenswrapper[5052]: I1203 13:09:08.535785 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-zzzpc\" (UniqueName: \"kubernetes.io/projected/71d2103c-ea23-404c-80b0-d8c54569ffd5-kube-api-access-zzzpc\") pod \"crc-storage-crc-9ctxf\" (UID: \"71d2103c-ea23-404c-80b0-d8c54569ffd5\") " pod="crc-storage/crc-storage-crc-9ctxf" Dec 03 13:09:08 crc kubenswrapper[5052]: I1203 13:09:08.612268 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/kuttl-service-edpm-compute-no-nodes-edpm-compute-no-nodes-48h7r" Dec 03 13:09:08 crc kubenswrapper[5052]: I1203 13:09:08.719082 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kuttl-service-cm-0-0\" (UniqueName: \"kubernetes.io/configmap/117725d5-3490-4470-8278-caa18557de03-kuttl-service-cm-0-0\") pod \"117725d5-3490-4470-8278-caa18557de03\" (UID: \"117725d5-3490-4470-8278-caa18557de03\") " Dec 03 13:09:08 crc kubenswrapper[5052]: I1203 13:09:08.719115 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kuttl-service-cm-0-2\" (UniqueName: \"kubernetes.io/configmap/117725d5-3490-4470-8278-caa18557de03-kuttl-service-cm-0-2\") pod \"117725d5-3490-4470-8278-caa18557de03\" (UID: \"117725d5-3490-4470-8278-caa18557de03\") " Dec 03 13:09:08 crc kubenswrapper[5052]: I1203 13:09:08.719156 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kuttl-service-cm-2-0\" (UniqueName: \"kubernetes.io/configmap/117725d5-3490-4470-8278-caa18557de03-kuttl-service-cm-2-0\") pod \"117725d5-3490-4470-8278-caa18557de03\" (UID: \"117725d5-3490-4470-8278-caa18557de03\") " Dec 03 13:09:08 crc kubenswrapper[5052]: I1203 13:09:08.719176 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kuttl-service-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/117725d5-3490-4470-8278-caa18557de03-kuttl-service-combined-ca-bundle\") pod \"117725d5-3490-4470-8278-caa18557de03\" (UID: \"117725d5-3490-4470-8278-caa18557de03\") " Dec 03 13:09:08 crc kubenswrapper[5052]: I1203 13:09:08.719202 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vzk6x\" (UniqueName: \"kubernetes.io/projected/117725d5-3490-4470-8278-caa18557de03-kube-api-access-vzk6x\") pod \"117725d5-3490-4470-8278-caa18557de03\" (UID: \"117725d5-3490-4470-8278-caa18557de03\") " Dec 03 13:09:08 crc kubenswrapper[5052]: I1203 13:09:08.719228 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/117725d5-3490-4470-8278-caa18557de03-inventory\") pod \"117725d5-3490-4470-8278-caa18557de03\" (UID: \"117725d5-3490-4470-8278-caa18557de03\") " Dec 03 13:09:08 crc kubenswrapper[5052]: I1203 13:09:08.719264 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kuttl-service-cm-0-1\" (UniqueName: \"kubernetes.io/configmap/117725d5-3490-4470-8278-caa18557de03-kuttl-service-cm-0-1\") pod \"117725d5-3490-4470-8278-caa18557de03\" (UID: \"117725d5-3490-4470-8278-caa18557de03\") " Dec 03 13:09:08 crc kubenswrapper[5052]: I1203 13:09:08.719284 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kuttl-service-cm-1-0\" (UniqueName: \"kubernetes.io/configmap/117725d5-3490-4470-8278-caa18557de03-kuttl-service-cm-1-0\") pod \"117725d5-3490-4470-8278-caa18557de03\" (UID: \"117725d5-3490-4470-8278-caa18557de03\") " Dec 03 13:09:08 crc kubenswrapper[5052]: I1203 13:09:08.719305 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kuttl-service-cm-1-1\" (UniqueName: \"kubernetes.io/configmap/117725d5-3490-4470-8278-caa18557de03-kuttl-service-cm-1-1\") pod \"117725d5-3490-4470-8278-caa18557de03\" (UID: \"117725d5-3490-4470-8278-caa18557de03\") " Dec 03 13:09:08 crc kubenswrapper[5052]: I1203 13:09:08.719326 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/117725d5-3490-4470-8278-caa18557de03-ssh-key\") pod \"117725d5-3490-4470-8278-caa18557de03\" (UID: \"117725d5-3490-4470-8278-caa18557de03\") " Dec 03 13:09:08 crc kubenswrapper[5052]: I1203 13:09:08.719353 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kuttl-service-cm-1-2\" (UniqueName: \"kubernetes.io/configmap/117725d5-3490-4470-8278-caa18557de03-kuttl-service-cm-1-2\") pod \"117725d5-3490-4470-8278-caa18557de03\" (UID: \"117725d5-3490-4470-8278-caa18557de03\") " Dec 03 13:09:08 crc kubenswrapper[5052]: I1203 13:09:08.723630 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/117725d5-3490-4470-8278-caa18557de03-kube-api-access-vzk6x" (OuterVolumeSpecName: "kube-api-access-vzk6x") pod "117725d5-3490-4470-8278-caa18557de03" (UID: "117725d5-3490-4470-8278-caa18557de03"). InnerVolumeSpecName "kube-api-access-vzk6x". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:09:08 crc kubenswrapper[5052]: I1203 13:09:08.726796 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/117725d5-3490-4470-8278-caa18557de03-kuttl-service-combined-ca-bundle" (OuterVolumeSpecName: "kuttl-service-combined-ca-bundle") pod "117725d5-3490-4470-8278-caa18557de03" (UID: "117725d5-3490-4470-8278-caa18557de03"). InnerVolumeSpecName "kuttl-service-combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:09:08 crc kubenswrapper[5052]: I1203 13:09:08.739757 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/117725d5-3490-4470-8278-caa18557de03-kuttl-service-cm-2-0" (OuterVolumeSpecName: "kuttl-service-cm-2-0") pod "117725d5-3490-4470-8278-caa18557de03" (UID: "117725d5-3490-4470-8278-caa18557de03"). InnerVolumeSpecName "kuttl-service-cm-2-0". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:09:08 crc kubenswrapper[5052]: I1203 13:09:08.740346 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/117725d5-3490-4470-8278-caa18557de03-kuttl-service-cm-0-1" (OuterVolumeSpecName: "kuttl-service-cm-0-1") pod "117725d5-3490-4470-8278-caa18557de03" (UID: "117725d5-3490-4470-8278-caa18557de03"). InnerVolumeSpecName "kuttl-service-cm-0-1". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:09:08 crc kubenswrapper[5052]: I1203 13:09:08.740890 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/117725d5-3490-4470-8278-caa18557de03-kuttl-service-cm-0-2" (OuterVolumeSpecName: "kuttl-service-cm-0-2") pod "117725d5-3490-4470-8278-caa18557de03" (UID: "117725d5-3490-4470-8278-caa18557de03"). InnerVolumeSpecName "kuttl-service-cm-0-2". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:09:08 crc kubenswrapper[5052]: I1203 13:09:08.740918 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="crc-storage/crc-storage-crc-9ctxf" Dec 03 13:09:08 crc kubenswrapper[5052]: I1203 13:09:08.743200 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/117725d5-3490-4470-8278-caa18557de03-ssh-key" (OuterVolumeSpecName: "ssh-key") pod "117725d5-3490-4470-8278-caa18557de03" (UID: "117725d5-3490-4470-8278-caa18557de03"). InnerVolumeSpecName "ssh-key". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:09:08 crc kubenswrapper[5052]: I1203 13:09:08.746972 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/117725d5-3490-4470-8278-caa18557de03-kuttl-service-cm-1-1" (OuterVolumeSpecName: "kuttl-service-cm-1-1") pod "117725d5-3490-4470-8278-caa18557de03" (UID: "117725d5-3490-4470-8278-caa18557de03"). InnerVolumeSpecName "kuttl-service-cm-1-1". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:09:08 crc kubenswrapper[5052]: I1203 13:09:08.750691 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/117725d5-3490-4470-8278-caa18557de03-kuttl-service-cm-0-0" (OuterVolumeSpecName: "kuttl-service-cm-0-0") pod "117725d5-3490-4470-8278-caa18557de03" (UID: "117725d5-3490-4470-8278-caa18557de03"). InnerVolumeSpecName "kuttl-service-cm-0-0". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:09:08 crc kubenswrapper[5052]: I1203 13:09:08.753757 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/117725d5-3490-4470-8278-caa18557de03-kuttl-service-cm-1-2" (OuterVolumeSpecName: "kuttl-service-cm-1-2") pod "117725d5-3490-4470-8278-caa18557de03" (UID: "117725d5-3490-4470-8278-caa18557de03"). InnerVolumeSpecName "kuttl-service-cm-1-2". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:09:08 crc kubenswrapper[5052]: I1203 13:09:08.760353 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/117725d5-3490-4470-8278-caa18557de03-inventory" (OuterVolumeSpecName: "inventory") pod "117725d5-3490-4470-8278-caa18557de03" (UID: "117725d5-3490-4470-8278-caa18557de03"). InnerVolumeSpecName "inventory". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:09:08 crc kubenswrapper[5052]: I1203 13:09:08.762708 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/117725d5-3490-4470-8278-caa18557de03-kuttl-service-cm-1-0" (OuterVolumeSpecName: "kuttl-service-cm-1-0") pod "117725d5-3490-4470-8278-caa18557de03" (UID: "117725d5-3490-4470-8278-caa18557de03"). InnerVolumeSpecName "kuttl-service-cm-1-0". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:09:08 crc kubenswrapper[5052]: I1203 13:09:08.820369 5052 reconciler_common.go:293] "Volume detached for volume \"kuttl-service-cm-1-2\" (UniqueName: \"kubernetes.io/configmap/117725d5-3490-4470-8278-caa18557de03-kuttl-service-cm-1-2\") on node \"crc\" DevicePath \"\"" Dec 03 13:09:08 crc kubenswrapper[5052]: I1203 13:09:08.820396 5052 reconciler_common.go:293] "Volume detached for volume \"kuttl-service-cm-0-0\" (UniqueName: \"kubernetes.io/configmap/117725d5-3490-4470-8278-caa18557de03-kuttl-service-cm-0-0\") on node \"crc\" DevicePath \"\"" Dec 03 13:09:08 crc kubenswrapper[5052]: I1203 13:09:08.820406 5052 reconciler_common.go:293] "Volume detached for volume \"kuttl-service-cm-0-2\" (UniqueName: \"kubernetes.io/configmap/117725d5-3490-4470-8278-caa18557de03-kuttl-service-cm-0-2\") on node \"crc\" DevicePath \"\"" Dec 03 13:09:08 crc kubenswrapper[5052]: I1203 13:09:08.820414 5052 reconciler_common.go:293] "Volume detached for volume \"kuttl-service-cm-2-0\" (UniqueName: \"kubernetes.io/configmap/117725d5-3490-4470-8278-caa18557de03-kuttl-service-cm-2-0\") on node \"crc\" DevicePath \"\"" Dec 03 13:09:08 crc kubenswrapper[5052]: I1203 13:09:08.820424 5052 reconciler_common.go:293] "Volume detached for volume \"kuttl-service-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/117725d5-3490-4470-8278-caa18557de03-kuttl-service-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 13:09:08 crc kubenswrapper[5052]: I1203 13:09:08.820436 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-vzk6x\" (UniqueName: \"kubernetes.io/projected/117725d5-3490-4470-8278-caa18557de03-kube-api-access-vzk6x\") on node \"crc\" DevicePath \"\"" Dec 03 13:09:08 crc kubenswrapper[5052]: I1203 13:09:08.820446 5052 reconciler_common.go:293] "Volume detached for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/117725d5-3490-4470-8278-caa18557de03-inventory\") on node \"crc\" DevicePath \"\"" Dec 03 13:09:08 crc kubenswrapper[5052]: I1203 13:09:08.820455 5052 reconciler_common.go:293] "Volume detached for volume \"kuttl-service-cm-0-1\" (UniqueName: \"kubernetes.io/configmap/117725d5-3490-4470-8278-caa18557de03-kuttl-service-cm-0-1\") on node \"crc\" DevicePath \"\"" Dec 03 13:09:08 crc kubenswrapper[5052]: I1203 13:09:08.820463 5052 reconciler_common.go:293] "Volume detached for volume \"kuttl-service-cm-1-0\" (UniqueName: \"kubernetes.io/configmap/117725d5-3490-4470-8278-caa18557de03-kuttl-service-cm-1-0\") on node \"crc\" DevicePath \"\"" Dec 03 13:09:08 crc kubenswrapper[5052]: I1203 13:09:08.820471 5052 reconciler_common.go:293] "Volume detached for volume \"kuttl-service-cm-1-1\" (UniqueName: \"kubernetes.io/configmap/117725d5-3490-4470-8278-caa18557de03-kuttl-service-cm-1-1\") on node \"crc\" DevicePath \"\"" Dec 03 13:09:08 crc kubenswrapper[5052]: I1203 13:09:08.820482 5052 reconciler_common.go:293] "Volume detached for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/117725d5-3490-4470-8278-caa18557de03-ssh-key\") on node \"crc\" DevicePath \"\"" Dec 03 13:09:08 crc kubenswrapper[5052]: I1203 13:09:08.976722 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["crc-storage/crc-storage-crc-9ctxf"] Dec 03 13:09:09 crc kubenswrapper[5052]: I1203 13:09:09.176412 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openshift-marketplace/redhat-operators-cb8xq"] Dec 03 13:09:09 crc kubenswrapper[5052]: E1203 13:09:09.176961 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="117725d5-3490-4470-8278-caa18557de03" containerName="kuttl-service-edpm-compute-no-nodes-edpm-compute-no-nodes" Dec 03 13:09:09 crc kubenswrapper[5052]: I1203 13:09:09.176987 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="117725d5-3490-4470-8278-caa18557de03" containerName="kuttl-service-edpm-compute-no-nodes-edpm-compute-no-nodes" Dec 03 13:09:09 crc kubenswrapper[5052]: I1203 13:09:09.177480 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="117725d5-3490-4470-8278-caa18557de03" containerName="kuttl-service-edpm-compute-no-nodes-edpm-compute-no-nodes" Dec 03 13:09:09 crc kubenswrapper[5052]: I1203 13:09:09.178754 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/redhat-operators-cb8xq" Dec 03 13:09:09 crc kubenswrapper[5052]: I1203 13:09:09.183894 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/redhat-operators-cb8xq"] Dec 03 13:09:09 crc kubenswrapper[5052]: I1203 13:09:09.227836 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/16ba3f3b-bf6a-463c-b385-2c9f372652ae-utilities\") pod \"redhat-operators-cb8xq\" (UID: \"16ba3f3b-bf6a-463c-b385-2c9f372652ae\") " pod="openshift-marketplace/redhat-operators-cb8xq" Dec 03 13:09:09 crc kubenswrapper[5052]: I1203 13:09:09.227963 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qtmxs\" (UniqueName: \"kubernetes.io/projected/16ba3f3b-bf6a-463c-b385-2c9f372652ae-kube-api-access-qtmxs\") pod \"redhat-operators-cb8xq\" (UID: \"16ba3f3b-bf6a-463c-b385-2c9f372652ae\") " pod="openshift-marketplace/redhat-operators-cb8xq" Dec 03 13:09:09 crc kubenswrapper[5052]: I1203 13:09:09.228027 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/16ba3f3b-bf6a-463c-b385-2c9f372652ae-catalog-content\") pod \"redhat-operators-cb8xq\" (UID: \"16ba3f3b-bf6a-463c-b385-2c9f372652ae\") " pod="openshift-marketplace/redhat-operators-cb8xq" Dec 03 13:09:09 crc kubenswrapper[5052]: I1203 13:09:09.329181 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-qtmxs\" (UniqueName: \"kubernetes.io/projected/16ba3f3b-bf6a-463c-b385-2c9f372652ae-kube-api-access-qtmxs\") pod \"redhat-operators-cb8xq\" (UID: \"16ba3f3b-bf6a-463c-b385-2c9f372652ae\") " pod="openshift-marketplace/redhat-operators-cb8xq" Dec 03 13:09:09 crc kubenswrapper[5052]: I1203 13:09:09.330143 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/16ba3f3b-bf6a-463c-b385-2c9f372652ae-catalog-content\") pod \"redhat-operators-cb8xq\" (UID: \"16ba3f3b-bf6a-463c-b385-2c9f372652ae\") " pod="openshift-marketplace/redhat-operators-cb8xq" Dec 03 13:09:09 crc kubenswrapper[5052]: I1203 13:09:09.330256 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/16ba3f3b-bf6a-463c-b385-2c9f372652ae-utilities\") pod \"redhat-operators-cb8xq\" (UID: \"16ba3f3b-bf6a-463c-b385-2c9f372652ae\") " pod="openshift-marketplace/redhat-operators-cb8xq" Dec 03 13:09:09 crc kubenswrapper[5052]: I1203 13:09:09.330847 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/16ba3f3b-bf6a-463c-b385-2c9f372652ae-utilities\") pod \"redhat-operators-cb8xq\" (UID: \"16ba3f3b-bf6a-463c-b385-2c9f372652ae\") " pod="openshift-marketplace/redhat-operators-cb8xq" Dec 03 13:09:09 crc kubenswrapper[5052]: I1203 13:09:09.331125 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/16ba3f3b-bf6a-463c-b385-2c9f372652ae-catalog-content\") pod \"redhat-operators-cb8xq\" (UID: \"16ba3f3b-bf6a-463c-b385-2c9f372652ae\") " pod="openshift-marketplace/redhat-operators-cb8xq" Dec 03 13:09:09 crc kubenswrapper[5052]: I1203 13:09:09.349001 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-qtmxs\" (UniqueName: \"kubernetes.io/projected/16ba3f3b-bf6a-463c-b385-2c9f372652ae-kube-api-access-qtmxs\") pod \"redhat-operators-cb8xq\" (UID: \"16ba3f3b-bf6a-463c-b385-2c9f372652ae\") " pod="openshift-marketplace/redhat-operators-cb8xq" Dec 03 13:09:09 crc kubenswrapper[5052]: I1203 13:09:09.352772 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/kuttl-service-edpm-compute-no-nodes-edpm-compute-no-nodes-48h7r" event={"ID":"117725d5-3490-4470-8278-caa18557de03","Type":"ContainerDied","Data":"8a558ecc93b6d28c8a4c94ea0ff1d039785064a9a79b57df7468fc221f708da5"} Dec 03 13:09:09 crc kubenswrapper[5052]: I1203 13:09:09.352801 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/kuttl-service-edpm-compute-no-nodes-edpm-compute-no-nodes-48h7r" Dec 03 13:09:09 crc kubenswrapper[5052]: I1203 13:09:09.352809 5052 scope.go:117] "RemoveContainer" containerID="745ceeadda889b09b88ee267ee88537143aa04cc500e41c122373ca94b5bc0e9" Dec 03 13:09:09 crc kubenswrapper[5052]: I1203 13:09:09.354691 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="crc-storage/crc-storage-crc-9ctxf" event={"ID":"71d2103c-ea23-404c-80b0-d8c54569ffd5","Type":"ContainerStarted","Data":"16cf9a59ddda2ced482f80d4b5c38637833cf28f56c64a6bd300d829d986b087"} Dec 03 13:09:09 crc kubenswrapper[5052]: I1203 13:09:09.404782 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/kuttl-service-edpm-compute-no-nodes-edpm-compute-no-nodes-48h7r"] Dec 03 13:09:09 crc kubenswrapper[5052]: I1203 13:09:09.409753 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/kuttl-service-edpm-compute-no-nodes-edpm-compute-no-nodes-48h7r"] Dec 03 13:09:09 crc kubenswrapper[5052]: I1203 13:09:09.508584 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/redhat-operators-cb8xq" Dec 03 13:09:09 crc kubenswrapper[5052]: I1203 13:09:09.704648 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="117725d5-3490-4470-8278-caa18557de03" path="/var/lib/kubelet/pods/117725d5-3490-4470-8278-caa18557de03/volumes" Dec 03 13:09:09 crc kubenswrapper[5052]: I1203 13:09:09.706667 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d23254f9-a1c0-45eb-bd6d-12cc273e6300" path="/var/lib/kubelet/pods/d23254f9-a1c0-45eb-bd6d-12cc273e6300/volumes" Dec 03 13:09:09 crc kubenswrapper[5052]: I1203 13:09:09.762288 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/redhat-operators-cb8xq"] Dec 03 13:09:09 crc kubenswrapper[5052]: W1203 13:09:09.765184 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16ba3f3b_bf6a_463c_b385_2c9f372652ae.slice/crio-a770b147c0ab6b647499cca5812b9460719d12b8cde936d9b02b5bad1a17fa61 WatchSource:0}: Error finding container a770b147c0ab6b647499cca5812b9460719d12b8cde936d9b02b5bad1a17fa61: Status 404 returned error can't find the container with id a770b147c0ab6b647499cca5812b9460719d12b8cde936d9b02b5bad1a17fa61 Dec 03 13:09:10 crc kubenswrapper[5052]: I1203 13:09:10.363145 5052 generic.go:334] "Generic (PLEG): container finished" podID="16ba3f3b-bf6a-463c-b385-2c9f372652ae" containerID="97a4de12823b55904bf11efe29d72bb17224f53120506c2652845de5a9b0106c" exitCode=0 Dec 03 13:09:10 crc kubenswrapper[5052]: I1203 13:09:10.363248 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-operators-cb8xq" event={"ID":"16ba3f3b-bf6a-463c-b385-2c9f372652ae","Type":"ContainerDied","Data":"97a4de12823b55904bf11efe29d72bb17224f53120506c2652845de5a9b0106c"} Dec 03 13:09:10 crc kubenswrapper[5052]: I1203 13:09:10.363301 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-operators-cb8xq" event={"ID":"16ba3f3b-bf6a-463c-b385-2c9f372652ae","Type":"ContainerStarted","Data":"a770b147c0ab6b647499cca5812b9460719d12b8cde936d9b02b5bad1a17fa61"} Dec 03 13:09:10 crc kubenswrapper[5052]: I1203 13:09:10.369349 5052 generic.go:334] "Generic (PLEG): container finished" podID="71d2103c-ea23-404c-80b0-d8c54569ffd5" containerID="fff599f59ff0eda492ccce8b28da608718929bfdbc7bbc4f7db9703ffc681ebb" exitCode=0 Dec 03 13:09:10 crc kubenswrapper[5052]: I1203 13:09:10.369391 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="crc-storage/crc-storage-crc-9ctxf" event={"ID":"71d2103c-ea23-404c-80b0-d8c54569ffd5","Type":"ContainerDied","Data":"fff599f59ff0eda492ccce8b28da608718929bfdbc7bbc4f7db9703ffc681ebb"} Dec 03 13:09:10 crc kubenswrapper[5052]: I1203 13:09:10.494812 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-xkzkn" Dec 03 13:09:10 crc kubenswrapper[5052]: I1203 13:09:10.548905 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-64864b6d57-qj2vk"] Dec 03 13:09:10 crc kubenswrapper[5052]: I1203 13:09:10.549316 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-64864b6d57-qj2vk" podUID="cd12a65f-6e18-400d-a3d0-ca62e2afd313" containerName="dnsmasq-dns" containerID="cri-o://3879a9cd7f2731f88cc0512216043312d0f06d7f6c58684086bec12bb9fe4e6c" gracePeriod=10 Dec 03 13:09:10 crc kubenswrapper[5052]: I1203 13:09:10.954992 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-64864b6d57-qj2vk" Dec 03 13:09:11 crc kubenswrapper[5052]: I1203 13:09:11.057053 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/cd12a65f-6e18-400d-a3d0-ca62e2afd313-dnsmasq-svc\") pod \"cd12a65f-6e18-400d-a3d0-ca62e2afd313\" (UID: \"cd12a65f-6e18-400d-a3d0-ca62e2afd313\") " Dec 03 13:09:11 crc kubenswrapper[5052]: I1203 13:09:11.057247 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"edpm-compute-no-nodes\" (UniqueName: \"kubernetes.io/configmap/cd12a65f-6e18-400d-a3d0-ca62e2afd313-edpm-compute-no-nodes\") pod \"cd12a65f-6e18-400d-a3d0-ca62e2afd313\" (UID: \"cd12a65f-6e18-400d-a3d0-ca62e2afd313\") " Dec 03 13:09:11 crc kubenswrapper[5052]: I1203 13:09:11.057309 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c6fmd\" (UniqueName: \"kubernetes.io/projected/cd12a65f-6e18-400d-a3d0-ca62e2afd313-kube-api-access-c6fmd\") pod \"cd12a65f-6e18-400d-a3d0-ca62e2afd313\" (UID: \"cd12a65f-6e18-400d-a3d0-ca62e2afd313\") " Dec 03 13:09:11 crc kubenswrapper[5052]: I1203 13:09:11.057453 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/cd12a65f-6e18-400d-a3d0-ca62e2afd313-config\") pod \"cd12a65f-6e18-400d-a3d0-ca62e2afd313\" (UID: \"cd12a65f-6e18-400d-a3d0-ca62e2afd313\") " Dec 03 13:09:11 crc kubenswrapper[5052]: I1203 13:09:11.062895 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cd12a65f-6e18-400d-a3d0-ca62e2afd313-kube-api-access-c6fmd" (OuterVolumeSpecName: "kube-api-access-c6fmd") pod "cd12a65f-6e18-400d-a3d0-ca62e2afd313" (UID: "cd12a65f-6e18-400d-a3d0-ca62e2afd313"). InnerVolumeSpecName "kube-api-access-c6fmd". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:09:11 crc kubenswrapper[5052]: I1203 13:09:11.104080 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cd12a65f-6e18-400d-a3d0-ca62e2afd313-dnsmasq-svc" (OuterVolumeSpecName: "dnsmasq-svc") pod "cd12a65f-6e18-400d-a3d0-ca62e2afd313" (UID: "cd12a65f-6e18-400d-a3d0-ca62e2afd313"). InnerVolumeSpecName "dnsmasq-svc". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:09:11 crc kubenswrapper[5052]: I1203 13:09:11.104364 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cd12a65f-6e18-400d-a3d0-ca62e2afd313-config" (OuterVolumeSpecName: "config") pod "cd12a65f-6e18-400d-a3d0-ca62e2afd313" (UID: "cd12a65f-6e18-400d-a3d0-ca62e2afd313"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:09:11 crc kubenswrapper[5052]: I1203 13:09:11.106914 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cd12a65f-6e18-400d-a3d0-ca62e2afd313-edpm-compute-no-nodes" (OuterVolumeSpecName: "edpm-compute-no-nodes") pod "cd12a65f-6e18-400d-a3d0-ca62e2afd313" (UID: "cd12a65f-6e18-400d-a3d0-ca62e2afd313"). InnerVolumeSpecName "edpm-compute-no-nodes". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:09:11 crc kubenswrapper[5052]: I1203 13:09:11.160015 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/cd12a65f-6e18-400d-a3d0-ca62e2afd313-config\") on node \"crc\" DevicePath \"\"" Dec 03 13:09:11 crc kubenswrapper[5052]: I1203 13:09:11.160067 5052 reconciler_common.go:293] "Volume detached for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/cd12a65f-6e18-400d-a3d0-ca62e2afd313-dnsmasq-svc\") on node \"crc\" DevicePath \"\"" Dec 03 13:09:11 crc kubenswrapper[5052]: I1203 13:09:11.160091 5052 reconciler_common.go:293] "Volume detached for volume \"edpm-compute-no-nodes\" (UniqueName: \"kubernetes.io/configmap/cd12a65f-6e18-400d-a3d0-ca62e2afd313-edpm-compute-no-nodes\") on node \"crc\" DevicePath \"\"" Dec 03 13:09:11 crc kubenswrapper[5052]: I1203 13:09:11.160107 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-c6fmd\" (UniqueName: \"kubernetes.io/projected/cd12a65f-6e18-400d-a3d0-ca62e2afd313-kube-api-access-c6fmd\") on node \"crc\" DevicePath \"\"" Dec 03 13:09:11 crc kubenswrapper[5052]: I1203 13:09:11.379273 5052 generic.go:334] "Generic (PLEG): container finished" podID="cd12a65f-6e18-400d-a3d0-ca62e2afd313" containerID="3879a9cd7f2731f88cc0512216043312d0f06d7f6c58684086bec12bb9fe4e6c" exitCode=0 Dec 03 13:09:11 crc kubenswrapper[5052]: I1203 13:09:11.379363 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-64864b6d57-qj2vk" Dec 03 13:09:11 crc kubenswrapper[5052]: I1203 13:09:11.379347 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-64864b6d57-qj2vk" event={"ID":"cd12a65f-6e18-400d-a3d0-ca62e2afd313","Type":"ContainerDied","Data":"3879a9cd7f2731f88cc0512216043312d0f06d7f6c58684086bec12bb9fe4e6c"} Dec 03 13:09:11 crc kubenswrapper[5052]: I1203 13:09:11.379453 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-64864b6d57-qj2vk" event={"ID":"cd12a65f-6e18-400d-a3d0-ca62e2afd313","Type":"ContainerDied","Data":"d9f7e73590128800b86f079ab763186da42eed45f88227a6467b4451d8d99b08"} Dec 03 13:09:11 crc kubenswrapper[5052]: I1203 13:09:11.379500 5052 scope.go:117] "RemoveContainer" containerID="3879a9cd7f2731f88cc0512216043312d0f06d7f6c58684086bec12bb9fe4e6c" Dec 03 13:09:11 crc kubenswrapper[5052]: I1203 13:09:11.383367 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-operators-cb8xq" event={"ID":"16ba3f3b-bf6a-463c-b385-2c9f372652ae","Type":"ContainerStarted","Data":"a50ba4c665480a53110493524d3112700e961e95a4906183bc42251fce5618f6"} Dec 03 13:09:11 crc kubenswrapper[5052]: I1203 13:09:11.422466 5052 scope.go:117] "RemoveContainer" containerID="dcd7fe0d5c55c396aef5fa88fdea31a3c34320a7640846776d865a58a8e00703" Dec 03 13:09:11 crc kubenswrapper[5052]: I1203 13:09:11.424949 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-64864b6d57-qj2vk"] Dec 03 13:09:11 crc kubenswrapper[5052]: I1203 13:09:11.431568 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-64864b6d57-qj2vk"] Dec 03 13:09:11 crc kubenswrapper[5052]: I1203 13:09:11.443649 5052 scope.go:117] "RemoveContainer" containerID="3879a9cd7f2731f88cc0512216043312d0f06d7f6c58684086bec12bb9fe4e6c" Dec 03 13:09:11 crc kubenswrapper[5052]: E1203 13:09:11.444089 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3879a9cd7f2731f88cc0512216043312d0f06d7f6c58684086bec12bb9fe4e6c\": container with ID starting with 3879a9cd7f2731f88cc0512216043312d0f06d7f6c58684086bec12bb9fe4e6c not found: ID does not exist" containerID="3879a9cd7f2731f88cc0512216043312d0f06d7f6c58684086bec12bb9fe4e6c" Dec 03 13:09:11 crc kubenswrapper[5052]: I1203 13:09:11.444120 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3879a9cd7f2731f88cc0512216043312d0f06d7f6c58684086bec12bb9fe4e6c"} err="failed to get container status \"3879a9cd7f2731f88cc0512216043312d0f06d7f6c58684086bec12bb9fe4e6c\": rpc error: code = NotFound desc = could not find container \"3879a9cd7f2731f88cc0512216043312d0f06d7f6c58684086bec12bb9fe4e6c\": container with ID starting with 3879a9cd7f2731f88cc0512216043312d0f06d7f6c58684086bec12bb9fe4e6c not found: ID does not exist" Dec 03 13:09:11 crc kubenswrapper[5052]: I1203 13:09:11.444141 5052 scope.go:117] "RemoveContainer" containerID="dcd7fe0d5c55c396aef5fa88fdea31a3c34320a7640846776d865a58a8e00703" Dec 03 13:09:11 crc kubenswrapper[5052]: E1203 13:09:11.444395 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dcd7fe0d5c55c396aef5fa88fdea31a3c34320a7640846776d865a58a8e00703\": container with ID starting with dcd7fe0d5c55c396aef5fa88fdea31a3c34320a7640846776d865a58a8e00703 not found: ID does not exist" containerID="dcd7fe0d5c55c396aef5fa88fdea31a3c34320a7640846776d865a58a8e00703" Dec 03 13:09:11 crc kubenswrapper[5052]: I1203 13:09:11.444421 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dcd7fe0d5c55c396aef5fa88fdea31a3c34320a7640846776d865a58a8e00703"} err="failed to get container status \"dcd7fe0d5c55c396aef5fa88fdea31a3c34320a7640846776d865a58a8e00703\": rpc error: code = NotFound desc = could not find container \"dcd7fe0d5c55c396aef5fa88fdea31a3c34320a7640846776d865a58a8e00703\": container with ID starting with dcd7fe0d5c55c396aef5fa88fdea31a3c34320a7640846776d865a58a8e00703 not found: ID does not exist" Dec 03 13:09:11 crc kubenswrapper[5052]: I1203 13:09:11.622641 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="crc-storage/crc-storage-crc-9ctxf" Dec 03 13:09:11 crc kubenswrapper[5052]: I1203 13:09:11.667388 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/71d2103c-ea23-404c-80b0-d8c54569ffd5-crc-storage\") pod \"71d2103c-ea23-404c-80b0-d8c54569ffd5\" (UID: \"71d2103c-ea23-404c-80b0-d8c54569ffd5\") " Dec 03 13:09:11 crc kubenswrapper[5052]: I1203 13:09:11.667454 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zzzpc\" (UniqueName: \"kubernetes.io/projected/71d2103c-ea23-404c-80b0-d8c54569ffd5-kube-api-access-zzzpc\") pod \"71d2103c-ea23-404c-80b0-d8c54569ffd5\" (UID: \"71d2103c-ea23-404c-80b0-d8c54569ffd5\") " Dec 03 13:09:11 crc kubenswrapper[5052]: I1203 13:09:11.667617 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/71d2103c-ea23-404c-80b0-d8c54569ffd5-node-mnt\") pod \"71d2103c-ea23-404c-80b0-d8c54569ffd5\" (UID: \"71d2103c-ea23-404c-80b0-d8c54569ffd5\") " Dec 03 13:09:11 crc kubenswrapper[5052]: I1203 13:09:11.667809 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/71d2103c-ea23-404c-80b0-d8c54569ffd5-node-mnt" (OuterVolumeSpecName: "node-mnt") pod "71d2103c-ea23-404c-80b0-d8c54569ffd5" (UID: "71d2103c-ea23-404c-80b0-d8c54569ffd5"). InnerVolumeSpecName "node-mnt". PluginName "kubernetes.io/host-path", VolumeGidValue "" Dec 03 13:09:11 crc kubenswrapper[5052]: I1203 13:09:11.668169 5052 reconciler_common.go:293] "Volume detached for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/71d2103c-ea23-404c-80b0-d8c54569ffd5-node-mnt\") on node \"crc\" DevicePath \"\"" Dec 03 13:09:11 crc kubenswrapper[5052]: I1203 13:09:11.672061 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/71d2103c-ea23-404c-80b0-d8c54569ffd5-kube-api-access-zzzpc" (OuterVolumeSpecName: "kube-api-access-zzzpc") pod "71d2103c-ea23-404c-80b0-d8c54569ffd5" (UID: "71d2103c-ea23-404c-80b0-d8c54569ffd5"). InnerVolumeSpecName "kube-api-access-zzzpc". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:09:11 crc kubenswrapper[5052]: I1203 13:09:11.681332 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cd12a65f-6e18-400d-a3d0-ca62e2afd313" path="/var/lib/kubelet/pods/cd12a65f-6e18-400d-a3d0-ca62e2afd313/volumes" Dec 03 13:09:11 crc kubenswrapper[5052]: I1203 13:09:11.688183 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/71d2103c-ea23-404c-80b0-d8c54569ffd5-crc-storage" (OuterVolumeSpecName: "crc-storage") pod "71d2103c-ea23-404c-80b0-d8c54569ffd5" (UID: "71d2103c-ea23-404c-80b0-d8c54569ffd5"). InnerVolumeSpecName "crc-storage". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:09:11 crc kubenswrapper[5052]: I1203 13:09:11.769809 5052 reconciler_common.go:293] "Volume detached for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/71d2103c-ea23-404c-80b0-d8c54569ffd5-crc-storage\") on node \"crc\" DevicePath \"\"" Dec 03 13:09:11 crc kubenswrapper[5052]: I1203 13:09:11.770097 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-zzzpc\" (UniqueName: \"kubernetes.io/projected/71d2103c-ea23-404c-80b0-d8c54569ffd5-kube-api-access-zzzpc\") on node \"crc\" DevicePath \"\"" Dec 03 13:09:12 crc kubenswrapper[5052]: I1203 13:09:12.392795 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="crc-storage/crc-storage-crc-9ctxf" event={"ID":"71d2103c-ea23-404c-80b0-d8c54569ffd5","Type":"ContainerDied","Data":"16cf9a59ddda2ced482f80d4b5c38637833cf28f56c64a6bd300d829d986b087"} Dec 03 13:09:12 crc kubenswrapper[5052]: I1203 13:09:12.392824 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="crc-storage/crc-storage-crc-9ctxf" Dec 03 13:09:12 crc kubenswrapper[5052]: I1203 13:09:12.392838 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="16cf9a59ddda2ced482f80d4b5c38637833cf28f56c64a6bd300d829d986b087" Dec 03 13:09:12 crc kubenswrapper[5052]: I1203 13:09:12.396097 5052 generic.go:334] "Generic (PLEG): container finished" podID="16ba3f3b-bf6a-463c-b385-2c9f372652ae" containerID="a50ba4c665480a53110493524d3112700e961e95a4906183bc42251fce5618f6" exitCode=0 Dec 03 13:09:12 crc kubenswrapper[5052]: I1203 13:09:12.396171 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-operators-cb8xq" event={"ID":"16ba3f3b-bf6a-463c-b385-2c9f372652ae","Type":"ContainerDied","Data":"a50ba4c665480a53110493524d3112700e961e95a4906183bc42251fce5618f6"} Dec 03 13:09:13 crc kubenswrapper[5052]: I1203 13:09:13.407591 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-operators-cb8xq" event={"ID":"16ba3f3b-bf6a-463c-b385-2c9f372652ae","Type":"ContainerStarted","Data":"2606cfdef118ca2753a1a4025b43c953a9eb0a08ce88e4697e6ed36962037910"} Dec 03 13:09:13 crc kubenswrapper[5052]: I1203 13:09:13.432086 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-marketplace/redhat-operators-cb8xq" podStartSLOduration=1.963908312 podStartE2EDuration="4.432055353s" podCreationTimestamp="2025-12-03 13:09:09 +0000 UTC" firstStartedPulling="2025-12-03 13:09:10.365629788 +0000 UTC m=+6540.953871552" lastFinishedPulling="2025-12-03 13:09:12.833776829 +0000 UTC m=+6543.422018593" observedRunningTime="2025-12-03 13:09:13.424584159 +0000 UTC m=+6544.012825923" watchObservedRunningTime="2025-12-03 13:09:13.432055353 +0000 UTC m=+6544.020297117" Dec 03 13:09:15 crc kubenswrapper[5052]: I1203 13:09:15.147321 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["crc-storage/crc-storage-crc-9ctxf"] Dec 03 13:09:15 crc kubenswrapper[5052]: I1203 13:09:15.153422 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["crc-storage/crc-storage-crc-9ctxf"] Dec 03 13:09:15 crc kubenswrapper[5052]: I1203 13:09:15.285488 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["crc-storage/crc-storage-crc-72b2q"] Dec 03 13:09:15 crc kubenswrapper[5052]: E1203 13:09:15.285835 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="cd12a65f-6e18-400d-a3d0-ca62e2afd313" containerName="init" Dec 03 13:09:15 crc kubenswrapper[5052]: I1203 13:09:15.285857 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="cd12a65f-6e18-400d-a3d0-ca62e2afd313" containerName="init" Dec 03 13:09:15 crc kubenswrapper[5052]: E1203 13:09:15.285869 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="cd12a65f-6e18-400d-a3d0-ca62e2afd313" containerName="dnsmasq-dns" Dec 03 13:09:15 crc kubenswrapper[5052]: I1203 13:09:15.285876 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="cd12a65f-6e18-400d-a3d0-ca62e2afd313" containerName="dnsmasq-dns" Dec 03 13:09:15 crc kubenswrapper[5052]: E1203 13:09:15.285892 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="71d2103c-ea23-404c-80b0-d8c54569ffd5" containerName="storage" Dec 03 13:09:15 crc kubenswrapper[5052]: I1203 13:09:15.285898 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="71d2103c-ea23-404c-80b0-d8c54569ffd5" containerName="storage" Dec 03 13:09:15 crc kubenswrapper[5052]: I1203 13:09:15.286030 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="cd12a65f-6e18-400d-a3d0-ca62e2afd313" containerName="dnsmasq-dns" Dec 03 13:09:15 crc kubenswrapper[5052]: I1203 13:09:15.286050 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="71d2103c-ea23-404c-80b0-d8c54569ffd5" containerName="storage" Dec 03 13:09:15 crc kubenswrapper[5052]: I1203 13:09:15.286548 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="crc-storage/crc-storage-crc-72b2q" Dec 03 13:09:15 crc kubenswrapper[5052]: I1203 13:09:15.289548 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"crc-storage"/"crc-storage" Dec 03 13:09:15 crc kubenswrapper[5052]: I1203 13:09:15.289588 5052 reflector.go:368] Caches populated for *v1.Secret from object-"crc-storage"/"crc-storage-dockercfg-mf76g" Dec 03 13:09:15 crc kubenswrapper[5052]: I1203 13:09:15.289931 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"crc-storage"/"kube-root-ca.crt" Dec 03 13:09:15 crc kubenswrapper[5052]: I1203 13:09:15.291083 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"crc-storage"/"openshift-service-ca.crt" Dec 03 13:09:15 crc kubenswrapper[5052]: I1203 13:09:15.295078 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["crc-storage/crc-storage-crc-72b2q"] Dec 03 13:09:15 crc kubenswrapper[5052]: I1203 13:09:15.423402 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/d0a9f232-bc59-4925-b554-00382b313497-crc-storage\") pod \"crc-storage-crc-72b2q\" (UID: \"d0a9f232-bc59-4925-b554-00382b313497\") " pod="crc-storage/crc-storage-crc-72b2q" Dec 03 13:09:15 crc kubenswrapper[5052]: I1203 13:09:15.423498 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xrjr6\" (UniqueName: \"kubernetes.io/projected/d0a9f232-bc59-4925-b554-00382b313497-kube-api-access-xrjr6\") pod \"crc-storage-crc-72b2q\" (UID: \"d0a9f232-bc59-4925-b554-00382b313497\") " pod="crc-storage/crc-storage-crc-72b2q" Dec 03 13:09:15 crc kubenswrapper[5052]: I1203 13:09:15.423585 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/d0a9f232-bc59-4925-b554-00382b313497-node-mnt\") pod \"crc-storage-crc-72b2q\" (UID: \"d0a9f232-bc59-4925-b554-00382b313497\") " pod="crc-storage/crc-storage-crc-72b2q" Dec 03 13:09:15 crc kubenswrapper[5052]: I1203 13:09:15.525479 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/d0a9f232-bc59-4925-b554-00382b313497-crc-storage\") pod \"crc-storage-crc-72b2q\" (UID: \"d0a9f232-bc59-4925-b554-00382b313497\") " pod="crc-storage/crc-storage-crc-72b2q" Dec 03 13:09:15 crc kubenswrapper[5052]: I1203 13:09:15.525612 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-xrjr6\" (UniqueName: \"kubernetes.io/projected/d0a9f232-bc59-4925-b554-00382b313497-kube-api-access-xrjr6\") pod \"crc-storage-crc-72b2q\" (UID: \"d0a9f232-bc59-4925-b554-00382b313497\") " pod="crc-storage/crc-storage-crc-72b2q" Dec 03 13:09:15 crc kubenswrapper[5052]: I1203 13:09:15.525684 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/d0a9f232-bc59-4925-b554-00382b313497-node-mnt\") pod \"crc-storage-crc-72b2q\" (UID: \"d0a9f232-bc59-4925-b554-00382b313497\") " pod="crc-storage/crc-storage-crc-72b2q" Dec 03 13:09:15 crc kubenswrapper[5052]: I1203 13:09:15.525958 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/d0a9f232-bc59-4925-b554-00382b313497-node-mnt\") pod \"crc-storage-crc-72b2q\" (UID: \"d0a9f232-bc59-4925-b554-00382b313497\") " pod="crc-storage/crc-storage-crc-72b2q" Dec 03 13:09:15 crc kubenswrapper[5052]: I1203 13:09:15.526237 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/d0a9f232-bc59-4925-b554-00382b313497-crc-storage\") pod \"crc-storage-crc-72b2q\" (UID: \"d0a9f232-bc59-4925-b554-00382b313497\") " pod="crc-storage/crc-storage-crc-72b2q" Dec 03 13:09:15 crc kubenswrapper[5052]: I1203 13:09:15.554241 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-xrjr6\" (UniqueName: \"kubernetes.io/projected/d0a9f232-bc59-4925-b554-00382b313497-kube-api-access-xrjr6\") pod \"crc-storage-crc-72b2q\" (UID: \"d0a9f232-bc59-4925-b554-00382b313497\") " pod="crc-storage/crc-storage-crc-72b2q" Dec 03 13:09:15 crc kubenswrapper[5052]: I1203 13:09:15.602327 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="crc-storage/crc-storage-crc-72b2q" Dec 03 13:09:15 crc kubenswrapper[5052]: I1203 13:09:15.683181 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="71d2103c-ea23-404c-80b0-d8c54569ffd5" path="/var/lib/kubelet/pods/71d2103c-ea23-404c-80b0-d8c54569ffd5/volumes" Dec 03 13:09:16 crc kubenswrapper[5052]: I1203 13:09:16.044579 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["crc-storage/crc-storage-crc-72b2q"] Dec 03 13:09:16 crc kubenswrapper[5052]: I1203 13:09:16.438712 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="crc-storage/crc-storage-crc-72b2q" event={"ID":"d0a9f232-bc59-4925-b554-00382b313497","Type":"ContainerStarted","Data":"db0249b3e79705347af1e5b3671fc45875b0e77ad8888a5a3dccf104cc0ec426"} Dec 03 13:09:17 crc kubenswrapper[5052]: I1203 13:09:17.448260 5052 generic.go:334] "Generic (PLEG): container finished" podID="d0a9f232-bc59-4925-b554-00382b313497" containerID="9f1d7d94a704f0496350535cc5fb961eee1de686802f6da0f1f72f5c4cbc204b" exitCode=0 Dec 03 13:09:17 crc kubenswrapper[5052]: I1203 13:09:17.448298 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="crc-storage/crc-storage-crc-72b2q" event={"ID":"d0a9f232-bc59-4925-b554-00382b313497","Type":"ContainerDied","Data":"9f1d7d94a704f0496350535cc5fb961eee1de686802f6da0f1f72f5c4cbc204b"} Dec 03 13:09:18 crc kubenswrapper[5052]: I1203 13:09:18.674228 5052 scope.go:117] "RemoveContainer" containerID="dbc100c8081ea1a2f5487429cc9d67de6bc8ba11282b64f248c71275375b0e16" Dec 03 13:09:18 crc kubenswrapper[5052]: E1203 13:09:18.674801 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 13:09:18 crc kubenswrapper[5052]: I1203 13:09:18.740131 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="crc-storage/crc-storage-crc-72b2q" Dec 03 13:09:18 crc kubenswrapper[5052]: I1203 13:09:18.877754 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/d0a9f232-bc59-4925-b554-00382b313497-node-mnt\") pod \"d0a9f232-bc59-4925-b554-00382b313497\" (UID: \"d0a9f232-bc59-4925-b554-00382b313497\") " Dec 03 13:09:18 crc kubenswrapper[5052]: I1203 13:09:18.878302 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/d0a9f232-bc59-4925-b554-00382b313497-crc-storage\") pod \"d0a9f232-bc59-4925-b554-00382b313497\" (UID: \"d0a9f232-bc59-4925-b554-00382b313497\") " Dec 03 13:09:18 crc kubenswrapper[5052]: I1203 13:09:18.878233 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/d0a9f232-bc59-4925-b554-00382b313497-node-mnt" (OuterVolumeSpecName: "node-mnt") pod "d0a9f232-bc59-4925-b554-00382b313497" (UID: "d0a9f232-bc59-4925-b554-00382b313497"). InnerVolumeSpecName "node-mnt". PluginName "kubernetes.io/host-path", VolumeGidValue "" Dec 03 13:09:18 crc kubenswrapper[5052]: I1203 13:09:18.879232 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xrjr6\" (UniqueName: \"kubernetes.io/projected/d0a9f232-bc59-4925-b554-00382b313497-kube-api-access-xrjr6\") pod \"d0a9f232-bc59-4925-b554-00382b313497\" (UID: \"d0a9f232-bc59-4925-b554-00382b313497\") " Dec 03 13:09:18 crc kubenswrapper[5052]: I1203 13:09:18.879544 5052 reconciler_common.go:293] "Volume detached for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/d0a9f232-bc59-4925-b554-00382b313497-node-mnt\") on node \"crc\" DevicePath \"\"" Dec 03 13:09:18 crc kubenswrapper[5052]: I1203 13:09:18.887392 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d0a9f232-bc59-4925-b554-00382b313497-kube-api-access-xrjr6" (OuterVolumeSpecName: "kube-api-access-xrjr6") pod "d0a9f232-bc59-4925-b554-00382b313497" (UID: "d0a9f232-bc59-4925-b554-00382b313497"). InnerVolumeSpecName "kube-api-access-xrjr6". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:09:18 crc kubenswrapper[5052]: I1203 13:09:18.903280 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d0a9f232-bc59-4925-b554-00382b313497-crc-storage" (OuterVolumeSpecName: "crc-storage") pod "d0a9f232-bc59-4925-b554-00382b313497" (UID: "d0a9f232-bc59-4925-b554-00382b313497"). InnerVolumeSpecName "crc-storage". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:09:18 crc kubenswrapper[5052]: I1203 13:09:18.982006 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-xrjr6\" (UniqueName: \"kubernetes.io/projected/d0a9f232-bc59-4925-b554-00382b313497-kube-api-access-xrjr6\") on node \"crc\" DevicePath \"\"" Dec 03 13:09:18 crc kubenswrapper[5052]: I1203 13:09:18.982055 5052 reconciler_common.go:293] "Volume detached for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/d0a9f232-bc59-4925-b554-00382b313497-crc-storage\") on node \"crc\" DevicePath \"\"" Dec 03 13:09:19 crc kubenswrapper[5052]: I1203 13:09:19.466599 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="crc-storage/crc-storage-crc-72b2q" event={"ID":"d0a9f232-bc59-4925-b554-00382b313497","Type":"ContainerDied","Data":"db0249b3e79705347af1e5b3671fc45875b0e77ad8888a5a3dccf104cc0ec426"} Dec 03 13:09:19 crc kubenswrapper[5052]: I1203 13:09:19.466646 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="db0249b3e79705347af1e5b3671fc45875b0e77ad8888a5a3dccf104cc0ec426" Dec 03 13:09:19 crc kubenswrapper[5052]: I1203 13:09:19.466643 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="crc-storage/crc-storage-crc-72b2q" Dec 03 13:09:19 crc kubenswrapper[5052]: I1203 13:09:19.509265 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openshift-marketplace/redhat-operators-cb8xq" Dec 03 13:09:19 crc kubenswrapper[5052]: I1203 13:09:19.509316 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-marketplace/redhat-operators-cb8xq" Dec 03 13:09:19 crc kubenswrapper[5052]: I1203 13:09:19.547637 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-marketplace/redhat-operators-cb8xq" Dec 03 13:09:20 crc kubenswrapper[5052]: I1203 13:09:20.516711 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-marketplace/redhat-operators-cb8xq" Dec 03 13:09:20 crc kubenswrapper[5052]: I1203 13:09:20.568660 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openshift-marketplace/redhat-operators-cb8xq"] Dec 03 13:09:22 crc kubenswrapper[5052]: I1203 13:09:22.411118 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-6fdb84cf7c-dqtv8"] Dec 03 13:09:22 crc kubenswrapper[5052]: E1203 13:09:22.411473 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="d0a9f232-bc59-4925-b554-00382b313497" containerName="storage" Dec 03 13:09:22 crc kubenswrapper[5052]: I1203 13:09:22.411491 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="d0a9f232-bc59-4925-b554-00382b313497" containerName="storage" Dec 03 13:09:22 crc kubenswrapper[5052]: I1203 13:09:22.411718 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="d0a9f232-bc59-4925-b554-00382b313497" containerName="storage" Dec 03 13:09:22 crc kubenswrapper[5052]: I1203 13:09:22.412678 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-6fdb84cf7c-dqtv8" Dec 03 13:09:22 crc kubenswrapper[5052]: I1203 13:09:22.414757 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"edpm-no-nodes-custom-svc" Dec 03 13:09:22 crc kubenswrapper[5052]: I1203 13:09:22.421526 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-6fdb84cf7c-dqtv8"] Dec 03 13:09:22 crc kubenswrapper[5052]: I1203 13:09:22.489938 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openshift-marketplace/redhat-operators-cb8xq" podUID="16ba3f3b-bf6a-463c-b385-2c9f372652ae" containerName="registry-server" containerID="cri-o://2606cfdef118ca2753a1a4025b43c953a9eb0a08ce88e4697e6ed36962037910" gracePeriod=2 Dec 03 13:09:22 crc kubenswrapper[5052]: I1203 13:09:22.535306 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"edpm-no-nodes-custom-svc\" (UniqueName: \"kubernetes.io/configmap/30170bd3-66b5-4a6e-9d51-6ae38864e52d-edpm-no-nodes-custom-svc\") pod \"dnsmasq-dnsmasq-6fdb84cf7c-dqtv8\" (UID: \"30170bd3-66b5-4a6e-9d51-6ae38864e52d\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-6fdb84cf7c-dqtv8" Dec 03 13:09:22 crc kubenswrapper[5052]: I1203 13:09:22.535530 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/30170bd3-66b5-4a6e-9d51-6ae38864e52d-config\") pod \"dnsmasq-dnsmasq-6fdb84cf7c-dqtv8\" (UID: \"30170bd3-66b5-4a6e-9d51-6ae38864e52d\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-6fdb84cf7c-dqtv8" Dec 03 13:09:22 crc kubenswrapper[5052]: I1203 13:09:22.535674 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2jwqk\" (UniqueName: \"kubernetes.io/projected/30170bd3-66b5-4a6e-9d51-6ae38864e52d-kube-api-access-2jwqk\") pod \"dnsmasq-dnsmasq-6fdb84cf7c-dqtv8\" (UID: \"30170bd3-66b5-4a6e-9d51-6ae38864e52d\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-6fdb84cf7c-dqtv8" Dec 03 13:09:22 crc kubenswrapper[5052]: I1203 13:09:22.535850 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/30170bd3-66b5-4a6e-9d51-6ae38864e52d-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-6fdb84cf7c-dqtv8\" (UID: \"30170bd3-66b5-4a6e-9d51-6ae38864e52d\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-6fdb84cf7c-dqtv8" Dec 03 13:09:22 crc kubenswrapper[5052]: I1203 13:09:22.637056 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"edpm-no-nodes-custom-svc\" (UniqueName: \"kubernetes.io/configmap/30170bd3-66b5-4a6e-9d51-6ae38864e52d-edpm-no-nodes-custom-svc\") pod \"dnsmasq-dnsmasq-6fdb84cf7c-dqtv8\" (UID: \"30170bd3-66b5-4a6e-9d51-6ae38864e52d\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-6fdb84cf7c-dqtv8" Dec 03 13:09:22 crc kubenswrapper[5052]: I1203 13:09:22.637450 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/30170bd3-66b5-4a6e-9d51-6ae38864e52d-config\") pod \"dnsmasq-dnsmasq-6fdb84cf7c-dqtv8\" (UID: \"30170bd3-66b5-4a6e-9d51-6ae38864e52d\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-6fdb84cf7c-dqtv8" Dec 03 13:09:22 crc kubenswrapper[5052]: I1203 13:09:22.637484 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-2jwqk\" (UniqueName: \"kubernetes.io/projected/30170bd3-66b5-4a6e-9d51-6ae38864e52d-kube-api-access-2jwqk\") pod \"dnsmasq-dnsmasq-6fdb84cf7c-dqtv8\" (UID: \"30170bd3-66b5-4a6e-9d51-6ae38864e52d\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-6fdb84cf7c-dqtv8" Dec 03 13:09:22 crc kubenswrapper[5052]: I1203 13:09:22.637541 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/30170bd3-66b5-4a6e-9d51-6ae38864e52d-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-6fdb84cf7c-dqtv8\" (UID: \"30170bd3-66b5-4a6e-9d51-6ae38864e52d\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-6fdb84cf7c-dqtv8" Dec 03 13:09:22 crc kubenswrapper[5052]: I1203 13:09:22.638212 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"edpm-no-nodes-custom-svc\" (UniqueName: \"kubernetes.io/configmap/30170bd3-66b5-4a6e-9d51-6ae38864e52d-edpm-no-nodes-custom-svc\") pod \"dnsmasq-dnsmasq-6fdb84cf7c-dqtv8\" (UID: \"30170bd3-66b5-4a6e-9d51-6ae38864e52d\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-6fdb84cf7c-dqtv8" Dec 03 13:09:22 crc kubenswrapper[5052]: I1203 13:09:22.638285 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/30170bd3-66b5-4a6e-9d51-6ae38864e52d-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-6fdb84cf7c-dqtv8\" (UID: \"30170bd3-66b5-4a6e-9d51-6ae38864e52d\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-6fdb84cf7c-dqtv8" Dec 03 13:09:22 crc kubenswrapper[5052]: I1203 13:09:22.638285 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/30170bd3-66b5-4a6e-9d51-6ae38864e52d-config\") pod \"dnsmasq-dnsmasq-6fdb84cf7c-dqtv8\" (UID: \"30170bd3-66b5-4a6e-9d51-6ae38864e52d\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-6fdb84cf7c-dqtv8" Dec 03 13:09:22 crc kubenswrapper[5052]: I1203 13:09:22.656574 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-2jwqk\" (UniqueName: \"kubernetes.io/projected/30170bd3-66b5-4a6e-9d51-6ae38864e52d-kube-api-access-2jwqk\") pod \"dnsmasq-dnsmasq-6fdb84cf7c-dqtv8\" (UID: \"30170bd3-66b5-4a6e-9d51-6ae38864e52d\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-6fdb84cf7c-dqtv8" Dec 03 13:09:22 crc kubenswrapper[5052]: I1203 13:09:22.730611 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-6fdb84cf7c-dqtv8" Dec 03 13:09:23 crc kubenswrapper[5052]: I1203 13:09:23.762998 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-6fdb84cf7c-dqtv8"] Dec 03 13:09:24 crc kubenswrapper[5052]: I1203 13:09:24.103363 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/redhat-operators-cb8xq" Dec 03 13:09:24 crc kubenswrapper[5052]: I1203 13:09:24.262797 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qtmxs\" (UniqueName: \"kubernetes.io/projected/16ba3f3b-bf6a-463c-b385-2c9f372652ae-kube-api-access-qtmxs\") pod \"16ba3f3b-bf6a-463c-b385-2c9f372652ae\" (UID: \"16ba3f3b-bf6a-463c-b385-2c9f372652ae\") " Dec 03 13:09:24 crc kubenswrapper[5052]: I1203 13:09:24.262902 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/16ba3f3b-bf6a-463c-b385-2c9f372652ae-utilities\") pod \"16ba3f3b-bf6a-463c-b385-2c9f372652ae\" (UID: \"16ba3f3b-bf6a-463c-b385-2c9f372652ae\") " Dec 03 13:09:24 crc kubenswrapper[5052]: I1203 13:09:24.262965 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/16ba3f3b-bf6a-463c-b385-2c9f372652ae-catalog-content\") pod \"16ba3f3b-bf6a-463c-b385-2c9f372652ae\" (UID: \"16ba3f3b-bf6a-463c-b385-2c9f372652ae\") " Dec 03 13:09:24 crc kubenswrapper[5052]: I1203 13:09:24.264314 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/16ba3f3b-bf6a-463c-b385-2c9f372652ae-utilities" (OuterVolumeSpecName: "utilities") pod "16ba3f3b-bf6a-463c-b385-2c9f372652ae" (UID: "16ba3f3b-bf6a-463c-b385-2c9f372652ae"). InnerVolumeSpecName "utilities". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 13:09:24 crc kubenswrapper[5052]: I1203 13:09:24.268279 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/16ba3f3b-bf6a-463c-b385-2c9f372652ae-kube-api-access-qtmxs" (OuterVolumeSpecName: "kube-api-access-qtmxs") pod "16ba3f3b-bf6a-463c-b385-2c9f372652ae" (UID: "16ba3f3b-bf6a-463c-b385-2c9f372652ae"). InnerVolumeSpecName "kube-api-access-qtmxs". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:09:24 crc kubenswrapper[5052]: I1203 13:09:24.364990 5052 reconciler_common.go:293] "Volume detached for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/16ba3f3b-bf6a-463c-b385-2c9f372652ae-utilities\") on node \"crc\" DevicePath \"\"" Dec 03 13:09:24 crc kubenswrapper[5052]: I1203 13:09:24.365050 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-qtmxs\" (UniqueName: \"kubernetes.io/projected/16ba3f3b-bf6a-463c-b385-2c9f372652ae-kube-api-access-qtmxs\") on node \"crc\" DevicePath \"\"" Dec 03 13:09:24 crc kubenswrapper[5052]: I1203 13:09:24.384657 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/16ba3f3b-bf6a-463c-b385-2c9f372652ae-catalog-content" (OuterVolumeSpecName: "catalog-content") pod "16ba3f3b-bf6a-463c-b385-2c9f372652ae" (UID: "16ba3f3b-bf6a-463c-b385-2c9f372652ae"). InnerVolumeSpecName "catalog-content". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 13:09:24 crc kubenswrapper[5052]: I1203 13:09:24.466324 5052 reconciler_common.go:293] "Volume detached for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/16ba3f3b-bf6a-463c-b385-2c9f372652ae-catalog-content\") on node \"crc\" DevicePath \"\"" Dec 03 13:09:24 crc kubenswrapper[5052]: I1203 13:09:24.514384 5052 generic.go:334] "Generic (PLEG): container finished" podID="16ba3f3b-bf6a-463c-b385-2c9f372652ae" containerID="2606cfdef118ca2753a1a4025b43c953a9eb0a08ce88e4697e6ed36962037910" exitCode=0 Dec 03 13:09:24 crc kubenswrapper[5052]: I1203 13:09:24.514453 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/redhat-operators-cb8xq" Dec 03 13:09:24 crc kubenswrapper[5052]: I1203 13:09:24.514464 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-operators-cb8xq" event={"ID":"16ba3f3b-bf6a-463c-b385-2c9f372652ae","Type":"ContainerDied","Data":"2606cfdef118ca2753a1a4025b43c953a9eb0a08ce88e4697e6ed36962037910"} Dec 03 13:09:24 crc kubenswrapper[5052]: I1203 13:09:24.514589 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-operators-cb8xq" event={"ID":"16ba3f3b-bf6a-463c-b385-2c9f372652ae","Type":"ContainerDied","Data":"a770b147c0ab6b647499cca5812b9460719d12b8cde936d9b02b5bad1a17fa61"} Dec 03 13:09:24 crc kubenswrapper[5052]: I1203 13:09:24.514620 5052 scope.go:117] "RemoveContainer" containerID="2606cfdef118ca2753a1a4025b43c953a9eb0a08ce88e4697e6ed36962037910" Dec 03 13:09:24 crc kubenswrapper[5052]: I1203 13:09:24.516274 5052 generic.go:334] "Generic (PLEG): container finished" podID="30170bd3-66b5-4a6e-9d51-6ae38864e52d" containerID="442b33aafe04c47d1ab26e703a34facb9ad461a925ac6d6c11a29639412607b9" exitCode=0 Dec 03 13:09:24 crc kubenswrapper[5052]: I1203 13:09:24.516299 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-6fdb84cf7c-dqtv8" event={"ID":"30170bd3-66b5-4a6e-9d51-6ae38864e52d","Type":"ContainerDied","Data":"442b33aafe04c47d1ab26e703a34facb9ad461a925ac6d6c11a29639412607b9"} Dec 03 13:09:24 crc kubenswrapper[5052]: I1203 13:09:24.516364 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-6fdb84cf7c-dqtv8" event={"ID":"30170bd3-66b5-4a6e-9d51-6ae38864e52d","Type":"ContainerStarted","Data":"a45e1429c3a90578c6800692fa1ab6a612e151f830b3f60483783ea0f4f3a1bf"} Dec 03 13:09:24 crc kubenswrapper[5052]: I1203 13:09:24.534786 5052 scope.go:117] "RemoveContainer" containerID="a50ba4c665480a53110493524d3112700e961e95a4906183bc42251fce5618f6" Dec 03 13:09:24 crc kubenswrapper[5052]: I1203 13:09:24.562500 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openshift-marketplace/redhat-operators-cb8xq"] Dec 03 13:09:24 crc kubenswrapper[5052]: I1203 13:09:24.570358 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openshift-marketplace/redhat-operators-cb8xq"] Dec 03 13:09:24 crc kubenswrapper[5052]: I1203 13:09:24.572882 5052 scope.go:117] "RemoveContainer" containerID="97a4de12823b55904bf11efe29d72bb17224f53120506c2652845de5a9b0106c" Dec 03 13:09:24 crc kubenswrapper[5052]: I1203 13:09:24.601305 5052 scope.go:117] "RemoveContainer" containerID="2606cfdef118ca2753a1a4025b43c953a9eb0a08ce88e4697e6ed36962037910" Dec 03 13:09:24 crc kubenswrapper[5052]: E1203 13:09:24.601856 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2606cfdef118ca2753a1a4025b43c953a9eb0a08ce88e4697e6ed36962037910\": container with ID starting with 2606cfdef118ca2753a1a4025b43c953a9eb0a08ce88e4697e6ed36962037910 not found: ID does not exist" containerID="2606cfdef118ca2753a1a4025b43c953a9eb0a08ce88e4697e6ed36962037910" Dec 03 13:09:24 crc kubenswrapper[5052]: I1203 13:09:24.601897 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2606cfdef118ca2753a1a4025b43c953a9eb0a08ce88e4697e6ed36962037910"} err="failed to get container status \"2606cfdef118ca2753a1a4025b43c953a9eb0a08ce88e4697e6ed36962037910\": rpc error: code = NotFound desc = could not find container \"2606cfdef118ca2753a1a4025b43c953a9eb0a08ce88e4697e6ed36962037910\": container with ID starting with 2606cfdef118ca2753a1a4025b43c953a9eb0a08ce88e4697e6ed36962037910 not found: ID does not exist" Dec 03 13:09:24 crc kubenswrapper[5052]: I1203 13:09:24.601923 5052 scope.go:117] "RemoveContainer" containerID="a50ba4c665480a53110493524d3112700e961e95a4906183bc42251fce5618f6" Dec 03 13:09:24 crc kubenswrapper[5052]: E1203 13:09:24.602429 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a50ba4c665480a53110493524d3112700e961e95a4906183bc42251fce5618f6\": container with ID starting with a50ba4c665480a53110493524d3112700e961e95a4906183bc42251fce5618f6 not found: ID does not exist" containerID="a50ba4c665480a53110493524d3112700e961e95a4906183bc42251fce5618f6" Dec 03 13:09:24 crc kubenswrapper[5052]: I1203 13:09:24.602473 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a50ba4c665480a53110493524d3112700e961e95a4906183bc42251fce5618f6"} err="failed to get container status \"a50ba4c665480a53110493524d3112700e961e95a4906183bc42251fce5618f6\": rpc error: code = NotFound desc = could not find container \"a50ba4c665480a53110493524d3112700e961e95a4906183bc42251fce5618f6\": container with ID starting with a50ba4c665480a53110493524d3112700e961e95a4906183bc42251fce5618f6 not found: ID does not exist" Dec 03 13:09:24 crc kubenswrapper[5052]: I1203 13:09:24.602548 5052 scope.go:117] "RemoveContainer" containerID="97a4de12823b55904bf11efe29d72bb17224f53120506c2652845de5a9b0106c" Dec 03 13:09:24 crc kubenswrapper[5052]: E1203 13:09:24.602882 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"97a4de12823b55904bf11efe29d72bb17224f53120506c2652845de5a9b0106c\": container with ID starting with 97a4de12823b55904bf11efe29d72bb17224f53120506c2652845de5a9b0106c not found: ID does not exist" containerID="97a4de12823b55904bf11efe29d72bb17224f53120506c2652845de5a9b0106c" Dec 03 13:09:24 crc kubenswrapper[5052]: I1203 13:09:24.602909 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"97a4de12823b55904bf11efe29d72bb17224f53120506c2652845de5a9b0106c"} err="failed to get container status \"97a4de12823b55904bf11efe29d72bb17224f53120506c2652845de5a9b0106c\": rpc error: code = NotFound desc = could not find container \"97a4de12823b55904bf11efe29d72bb17224f53120506c2652845de5a9b0106c\": container with ID starting with 97a4de12823b55904bf11efe29d72bb17224f53120506c2652845de5a9b0106c not found: ID does not exist" Dec 03 13:09:25 crc kubenswrapper[5052]: I1203 13:09:25.528623 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-6fdb84cf7c-dqtv8" event={"ID":"30170bd3-66b5-4a6e-9d51-6ae38864e52d","Type":"ContainerStarted","Data":"d69be154a58b0971a3117672b73db8634b347161ec0e7bb4c89964f8ce1be9f8"} Dec 03 13:09:25 crc kubenswrapper[5052]: I1203 13:09:25.532676 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-6fdb84cf7c-dqtv8" Dec 03 13:09:25 crc kubenswrapper[5052]: I1203 13:09:25.558903 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-6fdb84cf7c-dqtv8" podStartSLOduration=3.558878247 podStartE2EDuration="3.558878247s" podCreationTimestamp="2025-12-03 13:09:22 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 13:09:25.55237101 +0000 UTC m=+6556.140612784" watchObservedRunningTime="2025-12-03 13:09:25.558878247 +0000 UTC m=+6556.147120031" Dec 03 13:09:25 crc kubenswrapper[5052]: I1203 13:09:25.683256 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="16ba3f3b-bf6a-463c-b385-2c9f372652ae" path="/var/lib/kubelet/pods/16ba3f3b-bf6a-463c-b385-2c9f372652ae/volumes" Dec 03 13:09:31 crc kubenswrapper[5052]: I1203 13:09:31.673829 5052 scope.go:117] "RemoveContainer" containerID="dbc100c8081ea1a2f5487429cc9d67de6bc8ba11282b64f248c71275375b0e16" Dec 03 13:09:31 crc kubenswrapper[5052]: E1203 13:09:31.674640 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 13:09:32 crc kubenswrapper[5052]: I1203 13:09:32.732752 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-6fdb84cf7c-dqtv8" Dec 03 13:09:32 crc kubenswrapper[5052]: I1203 13:09:32.802401 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-xkzkn"] Dec 03 13:09:32 crc kubenswrapper[5052]: I1203 13:09:32.802696 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-xkzkn" podUID="81271079-f185-486e-9c3e-f14df3ee0183" containerName="dnsmasq-dns" containerID="cri-o://5a23cdcb537efb9a57b1623406603895c39ab50e8e6028abec71b5f4e6e4a886" gracePeriod=10 Dec 03 13:09:33 crc kubenswrapper[5052]: I1203 13:09:33.440572 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-xkzkn" Dec 03 13:09:33 crc kubenswrapper[5052]: I1203 13:09:33.605317 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7k6cb\" (UniqueName: \"kubernetes.io/projected/81271079-f185-486e-9c3e-f14df3ee0183-kube-api-access-7k6cb\") pod \"81271079-f185-486e-9c3e-f14df3ee0183\" (UID: \"81271079-f185-486e-9c3e-f14df3ee0183\") " Dec 03 13:09:33 crc kubenswrapper[5052]: I1203 13:09:33.605476 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/81271079-f185-486e-9c3e-f14df3ee0183-config\") pod \"81271079-f185-486e-9c3e-f14df3ee0183\" (UID: \"81271079-f185-486e-9c3e-f14df3ee0183\") " Dec 03 13:09:33 crc kubenswrapper[5052]: I1203 13:09:33.605541 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/81271079-f185-486e-9c3e-f14df3ee0183-dnsmasq-svc\") pod \"81271079-f185-486e-9c3e-f14df3ee0183\" (UID: \"81271079-f185-486e-9c3e-f14df3ee0183\") " Dec 03 13:09:33 crc kubenswrapper[5052]: I1203 13:09:33.616566 5052 generic.go:334] "Generic (PLEG): container finished" podID="81271079-f185-486e-9c3e-f14df3ee0183" containerID="5a23cdcb537efb9a57b1623406603895c39ab50e8e6028abec71b5f4e6e4a886" exitCode=0 Dec 03 13:09:33 crc kubenswrapper[5052]: I1203 13:09:33.616624 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-xkzkn" event={"ID":"81271079-f185-486e-9c3e-f14df3ee0183","Type":"ContainerDied","Data":"5a23cdcb537efb9a57b1623406603895c39ab50e8e6028abec71b5f4e6e4a886"} Dec 03 13:09:33 crc kubenswrapper[5052]: I1203 13:09:33.616663 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-xkzkn" event={"ID":"81271079-f185-486e-9c3e-f14df3ee0183","Type":"ContainerDied","Data":"be239e67866efa697fe7dd954f9f198d082f656bc2c1e9265b42d216aedbe868"} Dec 03 13:09:33 crc kubenswrapper[5052]: I1203 13:09:33.616684 5052 scope.go:117] "RemoveContainer" containerID="5a23cdcb537efb9a57b1623406603895c39ab50e8e6028abec71b5f4e6e4a886" Dec 03 13:09:33 crc kubenswrapper[5052]: I1203 13:09:33.616878 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-xkzkn" Dec 03 13:09:33 crc kubenswrapper[5052]: I1203 13:09:33.620348 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/81271079-f185-486e-9c3e-f14df3ee0183-kube-api-access-7k6cb" (OuterVolumeSpecName: "kube-api-access-7k6cb") pod "81271079-f185-486e-9c3e-f14df3ee0183" (UID: "81271079-f185-486e-9c3e-f14df3ee0183"). InnerVolumeSpecName "kube-api-access-7k6cb". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:09:33 crc kubenswrapper[5052]: I1203 13:09:33.652977 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/81271079-f185-486e-9c3e-f14df3ee0183-dnsmasq-svc" (OuterVolumeSpecName: "dnsmasq-svc") pod "81271079-f185-486e-9c3e-f14df3ee0183" (UID: "81271079-f185-486e-9c3e-f14df3ee0183"). InnerVolumeSpecName "dnsmasq-svc". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:09:33 crc kubenswrapper[5052]: I1203 13:09:33.654741 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/81271079-f185-486e-9c3e-f14df3ee0183-config" (OuterVolumeSpecName: "config") pod "81271079-f185-486e-9c3e-f14df3ee0183" (UID: "81271079-f185-486e-9c3e-f14df3ee0183"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:09:33 crc kubenswrapper[5052]: I1203 13:09:33.702872 5052 scope.go:117] "RemoveContainer" containerID="ca89665a5e1fd6408c4ffd75fa1509f48241c6743e731c94c825e2f6dd2079b3" Dec 03 13:09:33 crc kubenswrapper[5052]: I1203 13:09:33.707915 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/81271079-f185-486e-9c3e-f14df3ee0183-config\") on node \"crc\" DevicePath \"\"" Dec 03 13:09:33 crc kubenswrapper[5052]: I1203 13:09:33.707963 5052 reconciler_common.go:293] "Volume detached for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/81271079-f185-486e-9c3e-f14df3ee0183-dnsmasq-svc\") on node \"crc\" DevicePath \"\"" Dec 03 13:09:33 crc kubenswrapper[5052]: I1203 13:09:33.707989 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-7k6cb\" (UniqueName: \"kubernetes.io/projected/81271079-f185-486e-9c3e-f14df3ee0183-kube-api-access-7k6cb\") on node \"crc\" DevicePath \"\"" Dec 03 13:09:33 crc kubenswrapper[5052]: I1203 13:09:33.737098 5052 scope.go:117] "RemoveContainer" containerID="5a23cdcb537efb9a57b1623406603895c39ab50e8e6028abec71b5f4e6e4a886" Dec 03 13:09:33 crc kubenswrapper[5052]: E1203 13:09:33.737660 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5a23cdcb537efb9a57b1623406603895c39ab50e8e6028abec71b5f4e6e4a886\": container with ID starting with 5a23cdcb537efb9a57b1623406603895c39ab50e8e6028abec71b5f4e6e4a886 not found: ID does not exist" containerID="5a23cdcb537efb9a57b1623406603895c39ab50e8e6028abec71b5f4e6e4a886" Dec 03 13:09:33 crc kubenswrapper[5052]: I1203 13:09:33.737697 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5a23cdcb537efb9a57b1623406603895c39ab50e8e6028abec71b5f4e6e4a886"} err="failed to get container status \"5a23cdcb537efb9a57b1623406603895c39ab50e8e6028abec71b5f4e6e4a886\": rpc error: code = NotFound desc = could not find container \"5a23cdcb537efb9a57b1623406603895c39ab50e8e6028abec71b5f4e6e4a886\": container with ID starting with 5a23cdcb537efb9a57b1623406603895c39ab50e8e6028abec71b5f4e6e4a886 not found: ID does not exist" Dec 03 13:09:33 crc kubenswrapper[5052]: I1203 13:09:33.737725 5052 scope.go:117] "RemoveContainer" containerID="ca89665a5e1fd6408c4ffd75fa1509f48241c6743e731c94c825e2f6dd2079b3" Dec 03 13:09:33 crc kubenswrapper[5052]: E1203 13:09:33.738167 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ca89665a5e1fd6408c4ffd75fa1509f48241c6743e731c94c825e2f6dd2079b3\": container with ID starting with ca89665a5e1fd6408c4ffd75fa1509f48241c6743e731c94c825e2f6dd2079b3 not found: ID does not exist" containerID="ca89665a5e1fd6408c4ffd75fa1509f48241c6743e731c94c825e2f6dd2079b3" Dec 03 13:09:33 crc kubenswrapper[5052]: I1203 13:09:33.738198 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ca89665a5e1fd6408c4ffd75fa1509f48241c6743e731c94c825e2f6dd2079b3"} err="failed to get container status \"ca89665a5e1fd6408c4ffd75fa1509f48241c6743e731c94c825e2f6dd2079b3\": rpc error: code = NotFound desc = could not find container \"ca89665a5e1fd6408c4ffd75fa1509f48241c6743e731c94c825e2f6dd2079b3\": container with ID starting with ca89665a5e1fd6408c4ffd75fa1509f48241c6743e731c94c825e2f6dd2079b3 not found: ID does not exist" Dec 03 13:09:33 crc kubenswrapper[5052]: I1203 13:09:33.938709 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-xkzkn"] Dec 03 13:09:33 crc kubenswrapper[5052]: I1203 13:09:33.947325 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-xkzkn"] Dec 03 13:09:35 crc kubenswrapper[5052]: I1203 13:09:35.684006 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="81271079-f185-486e-9c3e-f14df3ee0183" path="/var/lib/kubelet/pods/81271079-f185-486e-9c3e-f14df3ee0183/volumes" Dec 03 13:09:37 crc kubenswrapper[5052]: I1203 13:09:37.365441 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/custom-img-svc-edpm-compute-no-nodes-edpm-no-nodes-custom-pfc9q"] Dec 03 13:09:37 crc kubenswrapper[5052]: E1203 13:09:37.366165 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="81271079-f185-486e-9c3e-f14df3ee0183" containerName="dnsmasq-dns" Dec 03 13:09:37 crc kubenswrapper[5052]: I1203 13:09:37.366181 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="81271079-f185-486e-9c3e-f14df3ee0183" containerName="dnsmasq-dns" Dec 03 13:09:37 crc kubenswrapper[5052]: E1203 13:09:37.366209 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="16ba3f3b-bf6a-463c-b385-2c9f372652ae" containerName="extract-utilities" Dec 03 13:09:37 crc kubenswrapper[5052]: I1203 13:09:37.366218 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="16ba3f3b-bf6a-463c-b385-2c9f372652ae" containerName="extract-utilities" Dec 03 13:09:37 crc kubenswrapper[5052]: E1203 13:09:37.366227 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="16ba3f3b-bf6a-463c-b385-2c9f372652ae" containerName="extract-content" Dec 03 13:09:37 crc kubenswrapper[5052]: I1203 13:09:37.366235 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="16ba3f3b-bf6a-463c-b385-2c9f372652ae" containerName="extract-content" Dec 03 13:09:37 crc kubenswrapper[5052]: E1203 13:09:37.366260 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="81271079-f185-486e-9c3e-f14df3ee0183" containerName="init" Dec 03 13:09:37 crc kubenswrapper[5052]: I1203 13:09:37.366267 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="81271079-f185-486e-9c3e-f14df3ee0183" containerName="init" Dec 03 13:09:37 crc kubenswrapper[5052]: E1203 13:09:37.366285 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="16ba3f3b-bf6a-463c-b385-2c9f372652ae" containerName="registry-server" Dec 03 13:09:37 crc kubenswrapper[5052]: I1203 13:09:37.366293 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="16ba3f3b-bf6a-463c-b385-2c9f372652ae" containerName="registry-server" Dec 03 13:09:37 crc kubenswrapper[5052]: I1203 13:09:37.366486 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="16ba3f3b-bf6a-463c-b385-2c9f372652ae" containerName="registry-server" Dec 03 13:09:37 crc kubenswrapper[5052]: I1203 13:09:37.366502 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="81271079-f185-486e-9c3e-f14df3ee0183" containerName="dnsmasq-dns" Dec 03 13:09:37 crc kubenswrapper[5052]: I1203 13:09:37.367129 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/custom-img-svc-edpm-compute-no-nodes-edpm-no-nodes-custom-pfc9q" Dec 03 13:09:37 crc kubenswrapper[5052]: I1203 13:09:37.371875 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"edpm-no-nodes-custom-svc-dockercfg-6zkph" Dec 03 13:09:37 crc kubenswrapper[5052]: I1203 13:09:37.372330 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"dataplanenodeset-edpm-no-nodes-custom-svc" Dec 03 13:09:37 crc kubenswrapper[5052]: I1203 13:09:37.372608 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"combined-ca-bundle" Dec 03 13:09:37 crc kubenswrapper[5052]: I1203 13:09:37.372763 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"openstack-aee-default-env" Dec 03 13:09:37 crc kubenswrapper[5052]: I1203 13:09:37.373394 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"dataplane-ansible-ssh-private-key-secret" Dec 03 13:09:37 crc kubenswrapper[5052]: I1203 13:09:37.379393 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/custom-img-svc-edpm-compute-no-nodes-edpm-no-nodes-custom-pfc9q"] Dec 03 13:09:37 crc kubenswrapper[5052]: I1203 13:09:37.541962 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/custom-img-svc-edpm-compute-no-nodes-edpm-no-nodes-custom-pfc9q"] Dec 03 13:09:37 crc kubenswrapper[5052]: E1203 13:09:37.543631 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[custom-img-svc-combined-ca-bundle inventory kube-api-access-lrfgw ssh-key], unattached volumes=[], failed to process volumes=[]: context canceled" pod="openstack-kuttl-tests/custom-img-svc-edpm-compute-no-nodes-edpm-no-nodes-custom-pfc9q" podUID="30ecde4c-7c99-481c-9c4e-7d09a9efd171" Dec 03 13:09:37 crc kubenswrapper[5052]: I1203 13:09:37.564403 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/30ecde4c-7c99-481c-9c4e-7d09a9efd171-inventory\") pod \"custom-img-svc-edpm-compute-no-nodes-edpm-no-nodes-custom-pfc9q\" (UID: \"30ecde4c-7c99-481c-9c4e-7d09a9efd171\") " pod="openstack-kuttl-tests/custom-img-svc-edpm-compute-no-nodes-edpm-no-nodes-custom-pfc9q" Dec 03 13:09:37 crc kubenswrapper[5052]: I1203 13:09:37.564547 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/30ecde4c-7c99-481c-9c4e-7d09a9efd171-ssh-key\") pod \"custom-img-svc-edpm-compute-no-nodes-edpm-no-nodes-custom-pfc9q\" (UID: \"30ecde4c-7c99-481c-9c4e-7d09a9efd171\") " pod="openstack-kuttl-tests/custom-img-svc-edpm-compute-no-nodes-edpm-no-nodes-custom-pfc9q" Dec 03 13:09:37 crc kubenswrapper[5052]: I1203 13:09:37.564753 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"custom-img-svc-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/30ecde4c-7c99-481c-9c4e-7d09a9efd171-custom-img-svc-combined-ca-bundle\") pod \"custom-img-svc-edpm-compute-no-nodes-edpm-no-nodes-custom-pfc9q\" (UID: \"30ecde4c-7c99-481c-9c4e-7d09a9efd171\") " pod="openstack-kuttl-tests/custom-img-svc-edpm-compute-no-nodes-edpm-no-nodes-custom-pfc9q" Dec 03 13:09:37 crc kubenswrapper[5052]: I1203 13:09:37.564831 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lrfgw\" (UniqueName: \"kubernetes.io/projected/30ecde4c-7c99-481c-9c4e-7d09a9efd171-kube-api-access-lrfgw\") pod \"custom-img-svc-edpm-compute-no-nodes-edpm-no-nodes-custom-pfc9q\" (UID: \"30ecde4c-7c99-481c-9c4e-7d09a9efd171\") " pod="openstack-kuttl-tests/custom-img-svc-edpm-compute-no-nodes-edpm-no-nodes-custom-pfc9q" Dec 03 13:09:37 crc kubenswrapper[5052]: I1203 13:09:37.617875 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-ngbbh"] Dec 03 13:09:37 crc kubenswrapper[5052]: I1203 13:09:37.619505 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-ngbbh" Dec 03 13:09:37 crc kubenswrapper[5052]: I1203 13:09:37.636309 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-ngbbh"] Dec 03 13:09:37 crc kubenswrapper[5052]: I1203 13:09:37.655288 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/custom-img-svc-edpm-compute-no-nodes-edpm-no-nodes-custom-pfc9q" Dec 03 13:09:37 crc kubenswrapper[5052]: I1203 13:09:37.666692 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8f6cn\" (UniqueName: \"kubernetes.io/projected/db571454-c5ad-46f4-bc1e-9c29cc32d921-kube-api-access-8f6cn\") pod \"dnsmasq-dnsmasq-84b9f45d47-ngbbh\" (UID: \"db571454-c5ad-46f4-bc1e-9c29cc32d921\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-ngbbh" Dec 03 13:09:37 crc kubenswrapper[5052]: I1203 13:09:37.666752 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/30ecde4c-7c99-481c-9c4e-7d09a9efd171-inventory\") pod \"custom-img-svc-edpm-compute-no-nodes-edpm-no-nodes-custom-pfc9q\" (UID: \"30ecde4c-7c99-481c-9c4e-7d09a9efd171\") " pod="openstack-kuttl-tests/custom-img-svc-edpm-compute-no-nodes-edpm-no-nodes-custom-pfc9q" Dec 03 13:09:37 crc kubenswrapper[5052]: I1203 13:09:37.666810 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/30ecde4c-7c99-481c-9c4e-7d09a9efd171-ssh-key\") pod \"custom-img-svc-edpm-compute-no-nodes-edpm-no-nodes-custom-pfc9q\" (UID: \"30ecde4c-7c99-481c-9c4e-7d09a9efd171\") " pod="openstack-kuttl-tests/custom-img-svc-edpm-compute-no-nodes-edpm-no-nodes-custom-pfc9q" Dec 03 13:09:37 crc kubenswrapper[5052]: I1203 13:09:37.666854 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"custom-img-svc-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/30ecde4c-7c99-481c-9c4e-7d09a9efd171-custom-img-svc-combined-ca-bundle\") pod \"custom-img-svc-edpm-compute-no-nodes-edpm-no-nodes-custom-pfc9q\" (UID: \"30ecde4c-7c99-481c-9c4e-7d09a9efd171\") " pod="openstack-kuttl-tests/custom-img-svc-edpm-compute-no-nodes-edpm-no-nodes-custom-pfc9q" Dec 03 13:09:37 crc kubenswrapper[5052]: I1203 13:09:37.666886 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-lrfgw\" (UniqueName: \"kubernetes.io/projected/30ecde4c-7c99-481c-9c4e-7d09a9efd171-kube-api-access-lrfgw\") pod \"custom-img-svc-edpm-compute-no-nodes-edpm-no-nodes-custom-pfc9q\" (UID: \"30ecde4c-7c99-481c-9c4e-7d09a9efd171\") " pod="openstack-kuttl-tests/custom-img-svc-edpm-compute-no-nodes-edpm-no-nodes-custom-pfc9q" Dec 03 13:09:37 crc kubenswrapper[5052]: I1203 13:09:37.666922 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/db571454-c5ad-46f4-bc1e-9c29cc32d921-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-84b9f45d47-ngbbh\" (UID: \"db571454-c5ad-46f4-bc1e-9c29cc32d921\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-ngbbh" Dec 03 13:09:37 crc kubenswrapper[5052]: I1203 13:09:37.666982 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/db571454-c5ad-46f4-bc1e-9c29cc32d921-config\") pod \"dnsmasq-dnsmasq-84b9f45d47-ngbbh\" (UID: \"db571454-c5ad-46f4-bc1e-9c29cc32d921\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-ngbbh" Dec 03 13:09:37 crc kubenswrapper[5052]: E1203 13:09:37.667147 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/dataplanenodeset-edpm-no-nodes-custom-svc: secret "dataplanenodeset-edpm-no-nodes-custom-svc" not found Dec 03 13:09:37 crc kubenswrapper[5052]: E1203 13:09:37.667202 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/30ecde4c-7c99-481c-9c4e-7d09a9efd171-inventory podName:30ecde4c-7c99-481c-9c4e-7d09a9efd171 nodeName:}" failed. No retries permitted until 2025-12-03 13:09:38.16718213 +0000 UTC m=+6568.755423894 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "inventory" (UniqueName: "kubernetes.io/secret/30ecde4c-7c99-481c-9c4e-7d09a9efd171-inventory") pod "custom-img-svc-edpm-compute-no-nodes-edpm-no-nodes-custom-pfc9q" (UID: "30ecde4c-7c99-481c-9c4e-7d09a9efd171") : secret "dataplanenodeset-edpm-no-nodes-custom-svc" not found Dec 03 13:09:37 crc kubenswrapper[5052]: I1203 13:09:37.672298 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/custom-img-svc-edpm-compute-no-nodes-edpm-no-nodes-custom-pfc9q" Dec 03 13:09:37 crc kubenswrapper[5052]: E1203 13:09:37.674159 5052 projected.go:194] Error preparing data for projected volume kube-api-access-lrfgw for pod openstack-kuttl-tests/custom-img-svc-edpm-compute-no-nodes-edpm-no-nodes-custom-pfc9q: failed to fetch token: serviceaccounts "edpm-no-nodes-custom-svc" not found Dec 03 13:09:37 crc kubenswrapper[5052]: E1203 13:09:37.674248 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/30ecde4c-7c99-481c-9c4e-7d09a9efd171-kube-api-access-lrfgw podName:30ecde4c-7c99-481c-9c4e-7d09a9efd171 nodeName:}" failed. No retries permitted until 2025-12-03 13:09:38.174219601 +0000 UTC m=+6568.762461365 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-lrfgw" (UniqueName: "kubernetes.io/projected/30ecde4c-7c99-481c-9c4e-7d09a9efd171-kube-api-access-lrfgw") pod "custom-img-svc-edpm-compute-no-nodes-edpm-no-nodes-custom-pfc9q" (UID: "30ecde4c-7c99-481c-9c4e-7d09a9efd171") : failed to fetch token: serviceaccounts "edpm-no-nodes-custom-svc" not found Dec 03 13:09:37 crc kubenswrapper[5052]: I1203 13:09:37.674372 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"custom-img-svc-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/30ecde4c-7c99-481c-9c4e-7d09a9efd171-custom-img-svc-combined-ca-bundle\") pod \"custom-img-svc-edpm-compute-no-nodes-edpm-no-nodes-custom-pfc9q\" (UID: \"30ecde4c-7c99-481c-9c4e-7d09a9efd171\") " pod="openstack-kuttl-tests/custom-img-svc-edpm-compute-no-nodes-edpm-no-nodes-custom-pfc9q" Dec 03 13:09:37 crc kubenswrapper[5052]: I1203 13:09:37.681464 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/30ecde4c-7c99-481c-9c4e-7d09a9efd171-ssh-key\") pod \"custom-img-svc-edpm-compute-no-nodes-edpm-no-nodes-custom-pfc9q\" (UID: \"30ecde4c-7c99-481c-9c4e-7d09a9efd171\") " pod="openstack-kuttl-tests/custom-img-svc-edpm-compute-no-nodes-edpm-no-nodes-custom-pfc9q" Dec 03 13:09:37 crc kubenswrapper[5052]: I1203 13:09:37.767253 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/30ecde4c-7c99-481c-9c4e-7d09a9efd171-ssh-key\") pod \"30ecde4c-7c99-481c-9c4e-7d09a9efd171\" (UID: \"30ecde4c-7c99-481c-9c4e-7d09a9efd171\") " Dec 03 13:09:37 crc kubenswrapper[5052]: I1203 13:09:37.767309 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"custom-img-svc-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/30ecde4c-7c99-481c-9c4e-7d09a9efd171-custom-img-svc-combined-ca-bundle\") pod \"30ecde4c-7c99-481c-9c4e-7d09a9efd171\" (UID: \"30ecde4c-7c99-481c-9c4e-7d09a9efd171\") " Dec 03 13:09:37 crc kubenswrapper[5052]: I1203 13:09:37.767494 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/db571454-c5ad-46f4-bc1e-9c29cc32d921-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-84b9f45d47-ngbbh\" (UID: \"db571454-c5ad-46f4-bc1e-9c29cc32d921\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-ngbbh" Dec 03 13:09:37 crc kubenswrapper[5052]: I1203 13:09:37.767548 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/db571454-c5ad-46f4-bc1e-9c29cc32d921-config\") pod \"dnsmasq-dnsmasq-84b9f45d47-ngbbh\" (UID: \"db571454-c5ad-46f4-bc1e-9c29cc32d921\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-ngbbh" Dec 03 13:09:37 crc kubenswrapper[5052]: I1203 13:09:37.767593 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-8f6cn\" (UniqueName: \"kubernetes.io/projected/db571454-c5ad-46f4-bc1e-9c29cc32d921-kube-api-access-8f6cn\") pod \"dnsmasq-dnsmasq-84b9f45d47-ngbbh\" (UID: \"db571454-c5ad-46f4-bc1e-9c29cc32d921\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-ngbbh" Dec 03 13:09:37 crc kubenswrapper[5052]: I1203 13:09:37.768613 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/db571454-c5ad-46f4-bc1e-9c29cc32d921-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-84b9f45d47-ngbbh\" (UID: \"db571454-c5ad-46f4-bc1e-9c29cc32d921\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-ngbbh" Dec 03 13:09:37 crc kubenswrapper[5052]: I1203 13:09:37.770543 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/30ecde4c-7c99-481c-9c4e-7d09a9efd171-ssh-key" (OuterVolumeSpecName: "ssh-key") pod "30ecde4c-7c99-481c-9c4e-7d09a9efd171" (UID: "30ecde4c-7c99-481c-9c4e-7d09a9efd171"). InnerVolumeSpecName "ssh-key". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:09:37 crc kubenswrapper[5052]: I1203 13:09:37.770620 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/30ecde4c-7c99-481c-9c4e-7d09a9efd171-custom-img-svc-combined-ca-bundle" (OuterVolumeSpecName: "custom-img-svc-combined-ca-bundle") pod "30ecde4c-7c99-481c-9c4e-7d09a9efd171" (UID: "30ecde4c-7c99-481c-9c4e-7d09a9efd171"). InnerVolumeSpecName "custom-img-svc-combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:09:37 crc kubenswrapper[5052]: I1203 13:09:37.770884 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/db571454-c5ad-46f4-bc1e-9c29cc32d921-config\") pod \"dnsmasq-dnsmasq-84b9f45d47-ngbbh\" (UID: \"db571454-c5ad-46f4-bc1e-9c29cc32d921\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-ngbbh" Dec 03 13:09:37 crc kubenswrapper[5052]: I1203 13:09:37.789726 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-8f6cn\" (UniqueName: \"kubernetes.io/projected/db571454-c5ad-46f4-bc1e-9c29cc32d921-kube-api-access-8f6cn\") pod \"dnsmasq-dnsmasq-84b9f45d47-ngbbh\" (UID: \"db571454-c5ad-46f4-bc1e-9c29cc32d921\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-ngbbh" Dec 03 13:09:37 crc kubenswrapper[5052]: I1203 13:09:37.869893 5052 reconciler_common.go:293] "Volume detached for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/30ecde4c-7c99-481c-9c4e-7d09a9efd171-ssh-key\") on node \"crc\" DevicePath \"\"" Dec 03 13:09:37 crc kubenswrapper[5052]: I1203 13:09:37.869967 5052 reconciler_common.go:293] "Volume detached for volume \"custom-img-svc-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/30ecde4c-7c99-481c-9c4e-7d09a9efd171-custom-img-svc-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 13:09:37 crc kubenswrapper[5052]: I1203 13:09:37.935880 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-ngbbh" Dec 03 13:09:38 crc kubenswrapper[5052]: I1203 13:09:38.174845 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/30ecde4c-7c99-481c-9c4e-7d09a9efd171-inventory\") pod \"custom-img-svc-edpm-compute-no-nodes-edpm-no-nodes-custom-pfc9q\" (UID: \"30ecde4c-7c99-481c-9c4e-7d09a9efd171\") " pod="openstack-kuttl-tests/custom-img-svc-edpm-compute-no-nodes-edpm-no-nodes-custom-pfc9q" Dec 03 13:09:38 crc kubenswrapper[5052]: I1203 13:09:38.175262 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-lrfgw\" (UniqueName: \"kubernetes.io/projected/30ecde4c-7c99-481c-9c4e-7d09a9efd171-kube-api-access-lrfgw\") pod \"custom-img-svc-edpm-compute-no-nodes-edpm-no-nodes-custom-pfc9q\" (UID: \"30ecde4c-7c99-481c-9c4e-7d09a9efd171\") " pod="openstack-kuttl-tests/custom-img-svc-edpm-compute-no-nodes-edpm-no-nodes-custom-pfc9q" Dec 03 13:09:38 crc kubenswrapper[5052]: E1203 13:09:38.174981 5052 secret.go:188] Couldn't get secret openstack-kuttl-tests/dataplanenodeset-edpm-no-nodes-custom-svc: secret "dataplanenodeset-edpm-no-nodes-custom-svc" not found Dec 03 13:09:38 crc kubenswrapper[5052]: E1203 13:09:38.175355 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/30ecde4c-7c99-481c-9c4e-7d09a9efd171-inventory podName:30ecde4c-7c99-481c-9c4e-7d09a9efd171 nodeName:}" failed. No retries permitted until 2025-12-03 13:09:39.175333062 +0000 UTC m=+6569.763574826 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "inventory" (UniqueName: "kubernetes.io/secret/30ecde4c-7c99-481c-9c4e-7d09a9efd171-inventory") pod "custom-img-svc-edpm-compute-no-nodes-edpm-no-nodes-custom-pfc9q" (UID: "30ecde4c-7c99-481c-9c4e-7d09a9efd171") : secret "dataplanenodeset-edpm-no-nodes-custom-svc" not found Dec 03 13:09:38 crc kubenswrapper[5052]: E1203 13:09:38.179983 5052 projected.go:194] Error preparing data for projected volume kube-api-access-lrfgw for pod openstack-kuttl-tests/custom-img-svc-edpm-compute-no-nodes-edpm-no-nodes-custom-pfc9q: failed to fetch token: serviceaccounts "edpm-no-nodes-custom-svc" not found Dec 03 13:09:38 crc kubenswrapper[5052]: E1203 13:09:38.180077 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/30ecde4c-7c99-481c-9c4e-7d09a9efd171-kube-api-access-lrfgw podName:30ecde4c-7c99-481c-9c4e-7d09a9efd171 nodeName:}" failed. No retries permitted until 2025-12-03 13:09:39.180050497 +0000 UTC m=+6569.768292331 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-lrfgw" (UniqueName: "kubernetes.io/projected/30ecde4c-7c99-481c-9c4e-7d09a9efd171-kube-api-access-lrfgw") pod "custom-img-svc-edpm-compute-no-nodes-edpm-no-nodes-custom-pfc9q" (UID: "30ecde4c-7c99-481c-9c4e-7d09a9efd171") : failed to fetch token: serviceaccounts "edpm-no-nodes-custom-svc" not found Dec 03 13:09:38 crc kubenswrapper[5052]: I1203 13:09:38.413919 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-ngbbh"] Dec 03 13:09:38 crc kubenswrapper[5052]: W1203 13:09:38.419437 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddb571454_c5ad_46f4_bc1e_9c29cc32d921.slice/crio-9e73f8b3334441b2fa3e0e8a053db389076e57ea5c1d622c54d00b5c0ff8a35c WatchSource:0}: Error finding container 9e73f8b3334441b2fa3e0e8a053db389076e57ea5c1d622c54d00b5c0ff8a35c: Status 404 returned error can't find the container with id 9e73f8b3334441b2fa3e0e8a053db389076e57ea5c1d622c54d00b5c0ff8a35c Dec 03 13:09:38 crc kubenswrapper[5052]: I1203 13:09:38.666142 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/custom-img-svc-edpm-compute-no-nodes-edpm-no-nodes-custom-pfc9q" Dec 03 13:09:38 crc kubenswrapper[5052]: I1203 13:09:38.666727 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-ngbbh" event={"ID":"db571454-c5ad-46f4-bc1e-9c29cc32d921","Type":"ContainerStarted","Data":"9e73f8b3334441b2fa3e0e8a053db389076e57ea5c1d622c54d00b5c0ff8a35c"} Dec 03 13:09:38 crc kubenswrapper[5052]: I1203 13:09:38.823657 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/custom-img-svc-edpm-compute-no-nodes-edpm-no-nodes-custom-pfc9q"] Dec 03 13:09:38 crc kubenswrapper[5052]: I1203 13:09:38.829558 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/custom-img-svc-edpm-compute-no-nodes-edpm-no-nodes-custom-pfc9q"] Dec 03 13:09:38 crc kubenswrapper[5052]: I1203 13:09:38.893256 5052 reconciler_common.go:293] "Volume detached for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/30ecde4c-7c99-481c-9c4e-7d09a9efd171-inventory\") on node \"crc\" DevicePath \"\"" Dec 03 13:09:38 crc kubenswrapper[5052]: I1203 13:09:38.893355 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-lrfgw\" (UniqueName: \"kubernetes.io/projected/30ecde4c-7c99-481c-9c4e-7d09a9efd171-kube-api-access-lrfgw\") on node \"crc\" DevicePath \"\"" Dec 03 13:09:39 crc kubenswrapper[5052]: I1203 13:09:39.680952 5052 generic.go:334] "Generic (PLEG): container finished" podID="db571454-c5ad-46f4-bc1e-9c29cc32d921" containerID="cf085488899352e2835e041c566a78d8e9c159f55f8ce843fe2a8639160bbb2a" exitCode=0 Dec 03 13:09:39 crc kubenswrapper[5052]: I1203 13:09:39.684622 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="30ecde4c-7c99-481c-9c4e-7d09a9efd171" path="/var/lib/kubelet/pods/30ecde4c-7c99-481c-9c4e-7d09a9efd171/volumes" Dec 03 13:09:39 crc kubenswrapper[5052]: I1203 13:09:39.685051 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-ngbbh" event={"ID":"db571454-c5ad-46f4-bc1e-9c29cc32d921","Type":"ContainerDied","Data":"cf085488899352e2835e041c566a78d8e9c159f55f8ce843fe2a8639160bbb2a"} Dec 03 13:09:40 crc kubenswrapper[5052]: I1203 13:09:40.690110 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-ngbbh" event={"ID":"db571454-c5ad-46f4-bc1e-9c29cc32d921","Type":"ContainerStarted","Data":"1fe185d294d533560cf7c402a020f267cfea0c63d5f4a4e848fb594451b7c726"} Dec 03 13:09:40 crc kubenswrapper[5052]: I1203 13:09:40.690260 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-ngbbh" Dec 03 13:09:40 crc kubenswrapper[5052]: I1203 13:09:40.709306 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-ngbbh" podStartSLOduration=3.709285461 podStartE2EDuration="3.709285461s" podCreationTimestamp="2025-12-03 13:09:37 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 13:09:40.707172471 +0000 UTC m=+6571.295414235" watchObservedRunningTime="2025-12-03 13:09:40.709285461 +0000 UTC m=+6571.297527225" Dec 03 13:09:43 crc kubenswrapper[5052]: I1203 13:09:43.429851 5052 scope.go:117] "RemoveContainer" containerID="abc46c07438d7d2c153343e71e0c8ac8fc43f50b3e36dfc8d5e8cb94fbdadce4" Dec 03 13:09:43 crc kubenswrapper[5052]: I1203 13:09:43.466839 5052 scope.go:117] "RemoveContainer" containerID="74675321570d68686f0bf483f4f778e02335c2f21afd4d74029a906af20aca11" Dec 03 13:09:43 crc kubenswrapper[5052]: I1203 13:09:43.489491 5052 scope.go:117] "RemoveContainer" containerID="3af3678b7ccde3d4dd555b0b72ae1e5848a1e1bdcb3e281e44e225bb38528da2" Dec 03 13:09:43 crc kubenswrapper[5052]: I1203 13:09:43.520081 5052 scope.go:117] "RemoveContainer" containerID="4d9cf1e8cdc4000660f6348bc391570c43d1810c852fce65e112085af5c919fc" Dec 03 13:09:44 crc kubenswrapper[5052]: I1203 13:09:44.673122 5052 scope.go:117] "RemoveContainer" containerID="dbc100c8081ea1a2f5487429cc9d67de6bc8ba11282b64f248c71275375b0e16" Dec 03 13:09:44 crc kubenswrapper[5052]: E1203 13:09:44.673371 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 13:09:45 crc kubenswrapper[5052]: I1203 13:09:45.973253 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["crc-storage/crc-storage-crc-72b2q"] Dec 03 13:09:45 crc kubenswrapper[5052]: I1203 13:09:45.982591 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["crc-storage/crc-storage-crc-72b2q"] Dec 03 13:09:46 crc kubenswrapper[5052]: I1203 13:09:46.106790 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["crc-storage/crc-storage-crc-gcqvf"] Dec 03 13:09:46 crc kubenswrapper[5052]: I1203 13:09:46.107988 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="crc-storage/crc-storage-crc-gcqvf" Dec 03 13:09:46 crc kubenswrapper[5052]: I1203 13:09:46.110208 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"crc-storage"/"openshift-service-ca.crt" Dec 03 13:09:46 crc kubenswrapper[5052]: I1203 13:09:46.110323 5052 reflector.go:368] Caches populated for *v1.Secret from object-"crc-storage"/"crc-storage-dockercfg-mf76g" Dec 03 13:09:46 crc kubenswrapper[5052]: I1203 13:09:46.110446 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"crc-storage"/"kube-root-ca.crt" Dec 03 13:09:46 crc kubenswrapper[5052]: I1203 13:09:46.111242 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"crc-storage"/"crc-storage" Dec 03 13:09:46 crc kubenswrapper[5052]: I1203 13:09:46.114593 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["crc-storage/crc-storage-crc-gcqvf"] Dec 03 13:09:46 crc kubenswrapper[5052]: I1203 13:09:46.205143 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wxlp2\" (UniqueName: \"kubernetes.io/projected/e6630042-d7a9-46ff-8960-5b9ad3c8444c-kube-api-access-wxlp2\") pod \"crc-storage-crc-gcqvf\" (UID: \"e6630042-d7a9-46ff-8960-5b9ad3c8444c\") " pod="crc-storage/crc-storage-crc-gcqvf" Dec 03 13:09:46 crc kubenswrapper[5052]: I1203 13:09:46.205271 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/e6630042-d7a9-46ff-8960-5b9ad3c8444c-node-mnt\") pod \"crc-storage-crc-gcqvf\" (UID: \"e6630042-d7a9-46ff-8960-5b9ad3c8444c\") " pod="crc-storage/crc-storage-crc-gcqvf" Dec 03 13:09:46 crc kubenswrapper[5052]: I1203 13:09:46.205371 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/e6630042-d7a9-46ff-8960-5b9ad3c8444c-crc-storage\") pod \"crc-storage-crc-gcqvf\" (UID: \"e6630042-d7a9-46ff-8960-5b9ad3c8444c\") " pod="crc-storage/crc-storage-crc-gcqvf" Dec 03 13:09:46 crc kubenswrapper[5052]: I1203 13:09:46.306450 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/e6630042-d7a9-46ff-8960-5b9ad3c8444c-crc-storage\") pod \"crc-storage-crc-gcqvf\" (UID: \"e6630042-d7a9-46ff-8960-5b9ad3c8444c\") " pod="crc-storage/crc-storage-crc-gcqvf" Dec 03 13:09:46 crc kubenswrapper[5052]: I1203 13:09:46.306590 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-wxlp2\" (UniqueName: \"kubernetes.io/projected/e6630042-d7a9-46ff-8960-5b9ad3c8444c-kube-api-access-wxlp2\") pod \"crc-storage-crc-gcqvf\" (UID: \"e6630042-d7a9-46ff-8960-5b9ad3c8444c\") " pod="crc-storage/crc-storage-crc-gcqvf" Dec 03 13:09:46 crc kubenswrapper[5052]: I1203 13:09:46.307033 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/e6630042-d7a9-46ff-8960-5b9ad3c8444c-node-mnt\") pod \"crc-storage-crc-gcqvf\" (UID: \"e6630042-d7a9-46ff-8960-5b9ad3c8444c\") " pod="crc-storage/crc-storage-crc-gcqvf" Dec 03 13:09:46 crc kubenswrapper[5052]: I1203 13:09:46.307309 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/e6630042-d7a9-46ff-8960-5b9ad3c8444c-node-mnt\") pod \"crc-storage-crc-gcqvf\" (UID: \"e6630042-d7a9-46ff-8960-5b9ad3c8444c\") " pod="crc-storage/crc-storage-crc-gcqvf" Dec 03 13:09:46 crc kubenswrapper[5052]: I1203 13:09:46.308093 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/e6630042-d7a9-46ff-8960-5b9ad3c8444c-crc-storage\") pod \"crc-storage-crc-gcqvf\" (UID: \"e6630042-d7a9-46ff-8960-5b9ad3c8444c\") " pod="crc-storage/crc-storage-crc-gcqvf" Dec 03 13:09:46 crc kubenswrapper[5052]: I1203 13:09:46.326006 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-wxlp2\" (UniqueName: \"kubernetes.io/projected/e6630042-d7a9-46ff-8960-5b9ad3c8444c-kube-api-access-wxlp2\") pod \"crc-storage-crc-gcqvf\" (UID: \"e6630042-d7a9-46ff-8960-5b9ad3c8444c\") " pod="crc-storage/crc-storage-crc-gcqvf" Dec 03 13:09:46 crc kubenswrapper[5052]: I1203 13:09:46.433078 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="crc-storage/crc-storage-crc-gcqvf" Dec 03 13:09:46 crc kubenswrapper[5052]: I1203 13:09:46.653999 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["crc-storage/crc-storage-crc-gcqvf"] Dec 03 13:09:46 crc kubenswrapper[5052]: W1203 13:09:46.655740 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode6630042_d7a9_46ff_8960_5b9ad3c8444c.slice/crio-68158460711da034734262bad0ffd9a68a1342e6c4ac6aef917806abf9991561 WatchSource:0}: Error finding container 68158460711da034734262bad0ffd9a68a1342e6c4ac6aef917806abf9991561: Status 404 returned error can't find the container with id 68158460711da034734262bad0ffd9a68a1342e6c4ac6aef917806abf9991561 Dec 03 13:09:46 crc kubenswrapper[5052]: I1203 13:09:46.743296 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="crc-storage/crc-storage-crc-gcqvf" event={"ID":"e6630042-d7a9-46ff-8960-5b9ad3c8444c","Type":"ContainerStarted","Data":"68158460711da034734262bad0ffd9a68a1342e6c4ac6aef917806abf9991561"} Dec 03 13:09:47 crc kubenswrapper[5052]: I1203 13:09:47.683077 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d0a9f232-bc59-4925-b554-00382b313497" path="/var/lib/kubelet/pods/d0a9f232-bc59-4925-b554-00382b313497/volumes" Dec 03 13:09:47 crc kubenswrapper[5052]: I1203 13:09:47.937697 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-ngbbh" Dec 03 13:09:47 crc kubenswrapper[5052]: I1203 13:09:47.990097 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-6fdb84cf7c-dqtv8"] Dec 03 13:09:47 crc kubenswrapper[5052]: I1203 13:09:47.990376 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-6fdb84cf7c-dqtv8" podUID="30170bd3-66b5-4a6e-9d51-6ae38864e52d" containerName="dnsmasq-dns" containerID="cri-o://d69be154a58b0971a3117672b73db8634b347161ec0e7bb4c89964f8ce1be9f8" gracePeriod=10 Dec 03 13:09:48 crc kubenswrapper[5052]: I1203 13:09:48.436696 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-6fdb84cf7c-dqtv8" Dec 03 13:09:48 crc kubenswrapper[5052]: I1203 13:09:48.559099 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/30170bd3-66b5-4a6e-9d51-6ae38864e52d-dnsmasq-svc\") pod \"30170bd3-66b5-4a6e-9d51-6ae38864e52d\" (UID: \"30170bd3-66b5-4a6e-9d51-6ae38864e52d\") " Dec 03 13:09:48 crc kubenswrapper[5052]: I1203 13:09:48.559175 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2jwqk\" (UniqueName: \"kubernetes.io/projected/30170bd3-66b5-4a6e-9d51-6ae38864e52d-kube-api-access-2jwqk\") pod \"30170bd3-66b5-4a6e-9d51-6ae38864e52d\" (UID: \"30170bd3-66b5-4a6e-9d51-6ae38864e52d\") " Dec 03 13:09:48 crc kubenswrapper[5052]: I1203 13:09:48.559314 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"edpm-no-nodes-custom-svc\" (UniqueName: \"kubernetes.io/configmap/30170bd3-66b5-4a6e-9d51-6ae38864e52d-edpm-no-nodes-custom-svc\") pod \"30170bd3-66b5-4a6e-9d51-6ae38864e52d\" (UID: \"30170bd3-66b5-4a6e-9d51-6ae38864e52d\") " Dec 03 13:09:48 crc kubenswrapper[5052]: I1203 13:09:48.559815 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/30170bd3-66b5-4a6e-9d51-6ae38864e52d-config\") pod \"30170bd3-66b5-4a6e-9d51-6ae38864e52d\" (UID: \"30170bd3-66b5-4a6e-9d51-6ae38864e52d\") " Dec 03 13:09:48 crc kubenswrapper[5052]: I1203 13:09:48.564209 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/30170bd3-66b5-4a6e-9d51-6ae38864e52d-kube-api-access-2jwqk" (OuterVolumeSpecName: "kube-api-access-2jwqk") pod "30170bd3-66b5-4a6e-9d51-6ae38864e52d" (UID: "30170bd3-66b5-4a6e-9d51-6ae38864e52d"). InnerVolumeSpecName "kube-api-access-2jwqk". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:09:48 crc kubenswrapper[5052]: I1203 13:09:48.595142 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/30170bd3-66b5-4a6e-9d51-6ae38864e52d-config" (OuterVolumeSpecName: "config") pod "30170bd3-66b5-4a6e-9d51-6ae38864e52d" (UID: "30170bd3-66b5-4a6e-9d51-6ae38864e52d"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:09:48 crc kubenswrapper[5052]: I1203 13:09:48.595573 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/30170bd3-66b5-4a6e-9d51-6ae38864e52d-edpm-no-nodes-custom-svc" (OuterVolumeSpecName: "edpm-no-nodes-custom-svc") pod "30170bd3-66b5-4a6e-9d51-6ae38864e52d" (UID: "30170bd3-66b5-4a6e-9d51-6ae38864e52d"). InnerVolumeSpecName "edpm-no-nodes-custom-svc". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:09:48 crc kubenswrapper[5052]: I1203 13:09:48.597766 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/30170bd3-66b5-4a6e-9d51-6ae38864e52d-dnsmasq-svc" (OuterVolumeSpecName: "dnsmasq-svc") pod "30170bd3-66b5-4a6e-9d51-6ae38864e52d" (UID: "30170bd3-66b5-4a6e-9d51-6ae38864e52d"). InnerVolumeSpecName "dnsmasq-svc". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:09:48 crc kubenswrapper[5052]: I1203 13:09:48.661267 5052 reconciler_common.go:293] "Volume detached for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/30170bd3-66b5-4a6e-9d51-6ae38864e52d-dnsmasq-svc\") on node \"crc\" DevicePath \"\"" Dec 03 13:09:48 crc kubenswrapper[5052]: I1203 13:09:48.661323 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-2jwqk\" (UniqueName: \"kubernetes.io/projected/30170bd3-66b5-4a6e-9d51-6ae38864e52d-kube-api-access-2jwqk\") on node \"crc\" DevicePath \"\"" Dec 03 13:09:48 crc kubenswrapper[5052]: I1203 13:09:48.661336 5052 reconciler_common.go:293] "Volume detached for volume \"edpm-no-nodes-custom-svc\" (UniqueName: \"kubernetes.io/configmap/30170bd3-66b5-4a6e-9d51-6ae38864e52d-edpm-no-nodes-custom-svc\") on node \"crc\" DevicePath \"\"" Dec 03 13:09:48 crc kubenswrapper[5052]: I1203 13:09:48.661378 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/30170bd3-66b5-4a6e-9d51-6ae38864e52d-config\") on node \"crc\" DevicePath \"\"" Dec 03 13:09:48 crc kubenswrapper[5052]: I1203 13:09:48.777331 5052 generic.go:334] "Generic (PLEG): container finished" podID="30170bd3-66b5-4a6e-9d51-6ae38864e52d" containerID="d69be154a58b0971a3117672b73db8634b347161ec0e7bb4c89964f8ce1be9f8" exitCode=0 Dec 03 13:09:48 crc kubenswrapper[5052]: I1203 13:09:48.777449 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-6fdb84cf7c-dqtv8" event={"ID":"30170bd3-66b5-4a6e-9d51-6ae38864e52d","Type":"ContainerDied","Data":"d69be154a58b0971a3117672b73db8634b347161ec0e7bb4c89964f8ce1be9f8"} Dec 03 13:09:48 crc kubenswrapper[5052]: I1203 13:09:48.777457 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-6fdb84cf7c-dqtv8" Dec 03 13:09:48 crc kubenswrapper[5052]: I1203 13:09:48.777479 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-6fdb84cf7c-dqtv8" event={"ID":"30170bd3-66b5-4a6e-9d51-6ae38864e52d","Type":"ContainerDied","Data":"a45e1429c3a90578c6800692fa1ab6a612e151f830b3f60483783ea0f4f3a1bf"} Dec 03 13:09:48 crc kubenswrapper[5052]: I1203 13:09:48.777497 5052 scope.go:117] "RemoveContainer" containerID="d69be154a58b0971a3117672b73db8634b347161ec0e7bb4c89964f8ce1be9f8" Dec 03 13:09:48 crc kubenswrapper[5052]: I1203 13:09:48.782567 5052 generic.go:334] "Generic (PLEG): container finished" podID="e6630042-d7a9-46ff-8960-5b9ad3c8444c" containerID="9b097a226f82480c23c807d5b63ec486754d4f12840b676166e91363220ac17c" exitCode=0 Dec 03 13:09:48 crc kubenswrapper[5052]: I1203 13:09:48.782632 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="crc-storage/crc-storage-crc-gcqvf" event={"ID":"e6630042-d7a9-46ff-8960-5b9ad3c8444c","Type":"ContainerDied","Data":"9b097a226f82480c23c807d5b63ec486754d4f12840b676166e91363220ac17c"} Dec 03 13:09:48 crc kubenswrapper[5052]: I1203 13:09:48.823041 5052 scope.go:117] "RemoveContainer" containerID="442b33aafe04c47d1ab26e703a34facb9ad461a925ac6d6c11a29639412607b9" Dec 03 13:09:48 crc kubenswrapper[5052]: I1203 13:09:48.827031 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-6fdb84cf7c-dqtv8"] Dec 03 13:09:48 crc kubenswrapper[5052]: I1203 13:09:48.834639 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-6fdb84cf7c-dqtv8"] Dec 03 13:09:48 crc kubenswrapper[5052]: I1203 13:09:48.852627 5052 scope.go:117] "RemoveContainer" containerID="d69be154a58b0971a3117672b73db8634b347161ec0e7bb4c89964f8ce1be9f8" Dec 03 13:09:48 crc kubenswrapper[5052]: E1203 13:09:48.853074 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d69be154a58b0971a3117672b73db8634b347161ec0e7bb4c89964f8ce1be9f8\": container with ID starting with d69be154a58b0971a3117672b73db8634b347161ec0e7bb4c89964f8ce1be9f8 not found: ID does not exist" containerID="d69be154a58b0971a3117672b73db8634b347161ec0e7bb4c89964f8ce1be9f8" Dec 03 13:09:48 crc kubenswrapper[5052]: I1203 13:09:48.853116 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d69be154a58b0971a3117672b73db8634b347161ec0e7bb4c89964f8ce1be9f8"} err="failed to get container status \"d69be154a58b0971a3117672b73db8634b347161ec0e7bb4c89964f8ce1be9f8\": rpc error: code = NotFound desc = could not find container \"d69be154a58b0971a3117672b73db8634b347161ec0e7bb4c89964f8ce1be9f8\": container with ID starting with d69be154a58b0971a3117672b73db8634b347161ec0e7bb4c89964f8ce1be9f8 not found: ID does not exist" Dec 03 13:09:48 crc kubenswrapper[5052]: I1203 13:09:48.853139 5052 scope.go:117] "RemoveContainer" containerID="442b33aafe04c47d1ab26e703a34facb9ad461a925ac6d6c11a29639412607b9" Dec 03 13:09:48 crc kubenswrapper[5052]: E1203 13:09:48.853480 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"442b33aafe04c47d1ab26e703a34facb9ad461a925ac6d6c11a29639412607b9\": container with ID starting with 442b33aafe04c47d1ab26e703a34facb9ad461a925ac6d6c11a29639412607b9 not found: ID does not exist" containerID="442b33aafe04c47d1ab26e703a34facb9ad461a925ac6d6c11a29639412607b9" Dec 03 13:09:48 crc kubenswrapper[5052]: I1203 13:09:48.853521 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"442b33aafe04c47d1ab26e703a34facb9ad461a925ac6d6c11a29639412607b9"} err="failed to get container status \"442b33aafe04c47d1ab26e703a34facb9ad461a925ac6d6c11a29639412607b9\": rpc error: code = NotFound desc = could not find container \"442b33aafe04c47d1ab26e703a34facb9ad461a925ac6d6c11a29639412607b9\": container with ID starting with 442b33aafe04c47d1ab26e703a34facb9ad461a925ac6d6c11a29639412607b9 not found: ID does not exist" Dec 03 13:09:49 crc kubenswrapper[5052]: I1203 13:09:49.681845 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="30170bd3-66b5-4a6e-9d51-6ae38864e52d" path="/var/lib/kubelet/pods/30170bd3-66b5-4a6e-9d51-6ae38864e52d/volumes" Dec 03 13:09:50 crc kubenswrapper[5052]: I1203 13:09:50.112333 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="crc-storage/crc-storage-crc-gcqvf" Dec 03 13:09:50 crc kubenswrapper[5052]: I1203 13:09:50.290401 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wxlp2\" (UniqueName: \"kubernetes.io/projected/e6630042-d7a9-46ff-8960-5b9ad3c8444c-kube-api-access-wxlp2\") pod \"e6630042-d7a9-46ff-8960-5b9ad3c8444c\" (UID: \"e6630042-d7a9-46ff-8960-5b9ad3c8444c\") " Dec 03 13:09:50 crc kubenswrapper[5052]: I1203 13:09:50.290469 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/e6630042-d7a9-46ff-8960-5b9ad3c8444c-node-mnt\") pod \"e6630042-d7a9-46ff-8960-5b9ad3c8444c\" (UID: \"e6630042-d7a9-46ff-8960-5b9ad3c8444c\") " Dec 03 13:09:50 crc kubenswrapper[5052]: I1203 13:09:50.290545 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/e6630042-d7a9-46ff-8960-5b9ad3c8444c-crc-storage\") pod \"e6630042-d7a9-46ff-8960-5b9ad3c8444c\" (UID: \"e6630042-d7a9-46ff-8960-5b9ad3c8444c\") " Dec 03 13:09:50 crc kubenswrapper[5052]: I1203 13:09:50.290544 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/e6630042-d7a9-46ff-8960-5b9ad3c8444c-node-mnt" (OuterVolumeSpecName: "node-mnt") pod "e6630042-d7a9-46ff-8960-5b9ad3c8444c" (UID: "e6630042-d7a9-46ff-8960-5b9ad3c8444c"). InnerVolumeSpecName "node-mnt". PluginName "kubernetes.io/host-path", VolumeGidValue "" Dec 03 13:09:50 crc kubenswrapper[5052]: I1203 13:09:50.291017 5052 reconciler_common.go:293] "Volume detached for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/e6630042-d7a9-46ff-8960-5b9ad3c8444c-node-mnt\") on node \"crc\" DevicePath \"\"" Dec 03 13:09:50 crc kubenswrapper[5052]: I1203 13:09:50.296761 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e6630042-d7a9-46ff-8960-5b9ad3c8444c-kube-api-access-wxlp2" (OuterVolumeSpecName: "kube-api-access-wxlp2") pod "e6630042-d7a9-46ff-8960-5b9ad3c8444c" (UID: "e6630042-d7a9-46ff-8960-5b9ad3c8444c"). InnerVolumeSpecName "kube-api-access-wxlp2". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:09:50 crc kubenswrapper[5052]: I1203 13:09:50.312107 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e6630042-d7a9-46ff-8960-5b9ad3c8444c-crc-storage" (OuterVolumeSpecName: "crc-storage") pod "e6630042-d7a9-46ff-8960-5b9ad3c8444c" (UID: "e6630042-d7a9-46ff-8960-5b9ad3c8444c"). InnerVolumeSpecName "crc-storage". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:09:50 crc kubenswrapper[5052]: I1203 13:09:50.392500 5052 reconciler_common.go:293] "Volume detached for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/e6630042-d7a9-46ff-8960-5b9ad3c8444c-crc-storage\") on node \"crc\" DevicePath \"\"" Dec 03 13:09:50 crc kubenswrapper[5052]: I1203 13:09:50.392578 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-wxlp2\" (UniqueName: \"kubernetes.io/projected/e6630042-d7a9-46ff-8960-5b9ad3c8444c-kube-api-access-wxlp2\") on node \"crc\" DevicePath \"\"" Dec 03 13:09:50 crc kubenswrapper[5052]: I1203 13:09:50.809427 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="crc-storage/crc-storage-crc-gcqvf" event={"ID":"e6630042-d7a9-46ff-8960-5b9ad3c8444c","Type":"ContainerDied","Data":"68158460711da034734262bad0ffd9a68a1342e6c4ac6aef917806abf9991561"} Dec 03 13:09:50 crc kubenswrapper[5052]: I1203 13:09:50.809467 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="68158460711da034734262bad0ffd9a68a1342e6c4ac6aef917806abf9991561" Dec 03 13:09:50 crc kubenswrapper[5052]: I1203 13:09:50.809663 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="crc-storage/crc-storage-crc-gcqvf" Dec 03 13:09:53 crc kubenswrapper[5052]: I1203 13:09:53.818154 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["crc-storage/crc-storage-crc-gcqvf"] Dec 03 13:09:53 crc kubenswrapper[5052]: I1203 13:09:53.823824 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["crc-storage/crc-storage-crc-gcqvf"] Dec 03 13:09:53 crc kubenswrapper[5052]: I1203 13:09:53.965608 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["crc-storage/crc-storage-crc-qkz7w"] Dec 03 13:09:53 crc kubenswrapper[5052]: E1203 13:09:53.965999 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="30170bd3-66b5-4a6e-9d51-6ae38864e52d" containerName="init" Dec 03 13:09:53 crc kubenswrapper[5052]: I1203 13:09:53.966020 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="30170bd3-66b5-4a6e-9d51-6ae38864e52d" containerName="init" Dec 03 13:09:53 crc kubenswrapper[5052]: E1203 13:09:53.966050 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="e6630042-d7a9-46ff-8960-5b9ad3c8444c" containerName="storage" Dec 03 13:09:53 crc kubenswrapper[5052]: I1203 13:09:53.966058 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="e6630042-d7a9-46ff-8960-5b9ad3c8444c" containerName="storage" Dec 03 13:09:53 crc kubenswrapper[5052]: E1203 13:09:53.966068 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="30170bd3-66b5-4a6e-9d51-6ae38864e52d" containerName="dnsmasq-dns" Dec 03 13:09:53 crc kubenswrapper[5052]: I1203 13:09:53.966075 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="30170bd3-66b5-4a6e-9d51-6ae38864e52d" containerName="dnsmasq-dns" Dec 03 13:09:53 crc kubenswrapper[5052]: I1203 13:09:53.966242 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="30170bd3-66b5-4a6e-9d51-6ae38864e52d" containerName="dnsmasq-dns" Dec 03 13:09:53 crc kubenswrapper[5052]: I1203 13:09:53.966253 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="e6630042-d7a9-46ff-8960-5b9ad3c8444c" containerName="storage" Dec 03 13:09:53 crc kubenswrapper[5052]: I1203 13:09:53.966911 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="crc-storage/crc-storage-crc-qkz7w" Dec 03 13:09:53 crc kubenswrapper[5052]: I1203 13:09:53.968358 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"crc-storage"/"crc-storage" Dec 03 13:09:53 crc kubenswrapper[5052]: I1203 13:09:53.968906 5052 reflector.go:368] Caches populated for *v1.Secret from object-"crc-storage"/"crc-storage-dockercfg-mf76g" Dec 03 13:09:53 crc kubenswrapper[5052]: I1203 13:09:53.969152 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"crc-storage"/"kube-root-ca.crt" Dec 03 13:09:53 crc kubenswrapper[5052]: I1203 13:09:53.971180 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"crc-storage"/"openshift-service-ca.crt" Dec 03 13:09:53 crc kubenswrapper[5052]: I1203 13:09:53.984049 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["crc-storage/crc-storage-crc-qkz7w"] Dec 03 13:09:54 crc kubenswrapper[5052]: I1203 13:09:54.169301 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/2a898663-87ce-4ae9-9046-42e6ec649028-crc-storage\") pod \"crc-storage-crc-qkz7w\" (UID: \"2a898663-87ce-4ae9-9046-42e6ec649028\") " pod="crc-storage/crc-storage-crc-qkz7w" Dec 03 13:09:54 crc kubenswrapper[5052]: I1203 13:09:54.169617 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/2a898663-87ce-4ae9-9046-42e6ec649028-node-mnt\") pod \"crc-storage-crc-qkz7w\" (UID: \"2a898663-87ce-4ae9-9046-42e6ec649028\") " pod="crc-storage/crc-storage-crc-qkz7w" Dec 03 13:09:54 crc kubenswrapper[5052]: I1203 13:09:54.169763 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tbzxs\" (UniqueName: \"kubernetes.io/projected/2a898663-87ce-4ae9-9046-42e6ec649028-kube-api-access-tbzxs\") pod \"crc-storage-crc-qkz7w\" (UID: \"2a898663-87ce-4ae9-9046-42e6ec649028\") " pod="crc-storage/crc-storage-crc-qkz7w" Dec 03 13:09:54 crc kubenswrapper[5052]: I1203 13:09:54.270768 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/2a898663-87ce-4ae9-9046-42e6ec649028-crc-storage\") pod \"crc-storage-crc-qkz7w\" (UID: \"2a898663-87ce-4ae9-9046-42e6ec649028\") " pod="crc-storage/crc-storage-crc-qkz7w" Dec 03 13:09:54 crc kubenswrapper[5052]: I1203 13:09:54.271150 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/2a898663-87ce-4ae9-9046-42e6ec649028-node-mnt\") pod \"crc-storage-crc-qkz7w\" (UID: \"2a898663-87ce-4ae9-9046-42e6ec649028\") " pod="crc-storage/crc-storage-crc-qkz7w" Dec 03 13:09:54 crc kubenswrapper[5052]: I1203 13:09:54.271324 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-tbzxs\" (UniqueName: \"kubernetes.io/projected/2a898663-87ce-4ae9-9046-42e6ec649028-kube-api-access-tbzxs\") pod \"crc-storage-crc-qkz7w\" (UID: \"2a898663-87ce-4ae9-9046-42e6ec649028\") " pod="crc-storage/crc-storage-crc-qkz7w" Dec 03 13:09:54 crc kubenswrapper[5052]: I1203 13:09:54.271676 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/2a898663-87ce-4ae9-9046-42e6ec649028-crc-storage\") pod \"crc-storage-crc-qkz7w\" (UID: \"2a898663-87ce-4ae9-9046-42e6ec649028\") " pod="crc-storage/crc-storage-crc-qkz7w" Dec 03 13:09:54 crc kubenswrapper[5052]: I1203 13:09:54.271957 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/2a898663-87ce-4ae9-9046-42e6ec649028-node-mnt\") pod \"crc-storage-crc-qkz7w\" (UID: \"2a898663-87ce-4ae9-9046-42e6ec649028\") " pod="crc-storage/crc-storage-crc-qkz7w" Dec 03 13:09:54 crc kubenswrapper[5052]: I1203 13:09:54.298496 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-tbzxs\" (UniqueName: \"kubernetes.io/projected/2a898663-87ce-4ae9-9046-42e6ec649028-kube-api-access-tbzxs\") pod \"crc-storage-crc-qkz7w\" (UID: \"2a898663-87ce-4ae9-9046-42e6ec649028\") " pod="crc-storage/crc-storage-crc-qkz7w" Dec 03 13:09:54 crc kubenswrapper[5052]: I1203 13:09:54.585651 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="crc-storage/crc-storage-crc-qkz7w" Dec 03 13:09:55 crc kubenswrapper[5052]: I1203 13:09:55.002616 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["crc-storage/crc-storage-crc-qkz7w"] Dec 03 13:09:55 crc kubenswrapper[5052]: I1203 13:09:55.683696 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e6630042-d7a9-46ff-8960-5b9ad3c8444c" path="/var/lib/kubelet/pods/e6630042-d7a9-46ff-8960-5b9ad3c8444c/volumes" Dec 03 13:09:55 crc kubenswrapper[5052]: I1203 13:09:55.856753 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="crc-storage/crc-storage-crc-qkz7w" event={"ID":"2a898663-87ce-4ae9-9046-42e6ec649028","Type":"ContainerStarted","Data":"c0e82c5c6fbd2bf28732700eac5f15ed98a699ada7872b209cfad32e08032207"} Dec 03 13:09:56 crc kubenswrapper[5052]: I1203 13:09:56.672490 5052 scope.go:117] "RemoveContainer" containerID="dbc100c8081ea1a2f5487429cc9d67de6bc8ba11282b64f248c71275375b0e16" Dec 03 13:09:56 crc kubenswrapper[5052]: E1203 13:09:56.673006 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 13:09:56 crc kubenswrapper[5052]: I1203 13:09:56.869555 5052 generic.go:334] "Generic (PLEG): container finished" podID="2a898663-87ce-4ae9-9046-42e6ec649028" containerID="33e0c0374a2870f420c1450f629b436c5a778189032e066202c7a4ea56dec46c" exitCode=0 Dec 03 13:09:56 crc kubenswrapper[5052]: I1203 13:09:56.869621 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="crc-storage/crc-storage-crc-qkz7w" event={"ID":"2a898663-87ce-4ae9-9046-42e6ec649028","Type":"ContainerDied","Data":"33e0c0374a2870f420c1450f629b436c5a778189032e066202c7a4ea56dec46c"} Dec 03 13:09:58 crc kubenswrapper[5052]: I1203 13:09:58.167129 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="crc-storage/crc-storage-crc-qkz7w" Dec 03 13:09:58 crc kubenswrapper[5052]: I1203 13:09:58.331088 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/2a898663-87ce-4ae9-9046-42e6ec649028-node-mnt\") pod \"2a898663-87ce-4ae9-9046-42e6ec649028\" (UID: \"2a898663-87ce-4ae9-9046-42e6ec649028\") " Dec 03 13:09:58 crc kubenswrapper[5052]: I1203 13:09:58.331158 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/2a898663-87ce-4ae9-9046-42e6ec649028-node-mnt" (OuterVolumeSpecName: "node-mnt") pod "2a898663-87ce-4ae9-9046-42e6ec649028" (UID: "2a898663-87ce-4ae9-9046-42e6ec649028"). InnerVolumeSpecName "node-mnt". PluginName "kubernetes.io/host-path", VolumeGidValue "" Dec 03 13:09:58 crc kubenswrapper[5052]: I1203 13:09:58.331265 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/2a898663-87ce-4ae9-9046-42e6ec649028-crc-storage\") pod \"2a898663-87ce-4ae9-9046-42e6ec649028\" (UID: \"2a898663-87ce-4ae9-9046-42e6ec649028\") " Dec 03 13:09:58 crc kubenswrapper[5052]: I1203 13:09:58.331315 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tbzxs\" (UniqueName: \"kubernetes.io/projected/2a898663-87ce-4ae9-9046-42e6ec649028-kube-api-access-tbzxs\") pod \"2a898663-87ce-4ae9-9046-42e6ec649028\" (UID: \"2a898663-87ce-4ae9-9046-42e6ec649028\") " Dec 03 13:09:58 crc kubenswrapper[5052]: I1203 13:09:58.331607 5052 reconciler_common.go:293] "Volume detached for volume \"node-mnt\" (UniqueName: \"kubernetes.io/host-path/2a898663-87ce-4ae9-9046-42e6ec649028-node-mnt\") on node \"crc\" DevicePath \"\"" Dec 03 13:09:58 crc kubenswrapper[5052]: I1203 13:09:58.341750 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2a898663-87ce-4ae9-9046-42e6ec649028-kube-api-access-tbzxs" (OuterVolumeSpecName: "kube-api-access-tbzxs") pod "2a898663-87ce-4ae9-9046-42e6ec649028" (UID: "2a898663-87ce-4ae9-9046-42e6ec649028"). InnerVolumeSpecName "kube-api-access-tbzxs". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:09:58 crc kubenswrapper[5052]: I1203 13:09:58.352111 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2a898663-87ce-4ae9-9046-42e6ec649028-crc-storage" (OuterVolumeSpecName: "crc-storage") pod "2a898663-87ce-4ae9-9046-42e6ec649028" (UID: "2a898663-87ce-4ae9-9046-42e6ec649028"). InnerVolumeSpecName "crc-storage". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:09:58 crc kubenswrapper[5052]: I1203 13:09:58.432531 5052 reconciler_common.go:293] "Volume detached for volume \"crc-storage\" (UniqueName: \"kubernetes.io/configmap/2a898663-87ce-4ae9-9046-42e6ec649028-crc-storage\") on node \"crc\" DevicePath \"\"" Dec 03 13:09:58 crc kubenswrapper[5052]: I1203 13:09:58.432567 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-tbzxs\" (UniqueName: \"kubernetes.io/projected/2a898663-87ce-4ae9-9046-42e6ec649028-kube-api-access-tbzxs\") on node \"crc\" DevicePath \"\"" Dec 03 13:09:58 crc kubenswrapper[5052]: I1203 13:09:58.889419 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="crc-storage/crc-storage-crc-qkz7w" event={"ID":"2a898663-87ce-4ae9-9046-42e6ec649028","Type":"ContainerDied","Data":"c0e82c5c6fbd2bf28732700eac5f15ed98a699ada7872b209cfad32e08032207"} Dec 03 13:09:58 crc kubenswrapper[5052]: I1203 13:09:58.889478 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="crc-storage/crc-storage-crc-qkz7w" Dec 03 13:09:58 crc kubenswrapper[5052]: I1203 13:09:58.889501 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c0e82c5c6fbd2bf28732700eac5f15ed98a699ada7872b209cfad32e08032207" Dec 03 13:10:01 crc kubenswrapper[5052]: I1203 13:10:01.700150 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-64864b6d57-wjkrf"] Dec 03 13:10:01 crc kubenswrapper[5052]: E1203 13:10:01.701097 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="2a898663-87ce-4ae9-9046-42e6ec649028" containerName="storage" Dec 03 13:10:01 crc kubenswrapper[5052]: I1203 13:10:01.701114 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="2a898663-87ce-4ae9-9046-42e6ec649028" containerName="storage" Dec 03 13:10:01 crc kubenswrapper[5052]: I1203 13:10:01.701281 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="2a898663-87ce-4ae9-9046-42e6ec649028" containerName="storage" Dec 03 13:10:01 crc kubenswrapper[5052]: I1203 13:10:01.702261 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-64864b6d57-wjkrf" Dec 03 13:10:01 crc kubenswrapper[5052]: I1203 13:10:01.704449 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"edpm-compute-no-nodes" Dec 03 13:10:01 crc kubenswrapper[5052]: I1203 13:10:01.714055 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-64864b6d57-wjkrf"] Dec 03 13:10:01 crc kubenswrapper[5052]: I1203 13:10:01.775237 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"edpm-compute-no-nodes\" (UniqueName: \"kubernetes.io/configmap/cde663cf-e7e2-42b1-9581-ea7a27f43481-edpm-compute-no-nodes\") pod \"dnsmasq-dnsmasq-64864b6d57-wjkrf\" (UID: \"cde663cf-e7e2-42b1-9581-ea7a27f43481\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-64864b6d57-wjkrf" Dec 03 13:10:01 crc kubenswrapper[5052]: I1203 13:10:01.776218 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dlrnr\" (UniqueName: \"kubernetes.io/projected/cde663cf-e7e2-42b1-9581-ea7a27f43481-kube-api-access-dlrnr\") pod \"dnsmasq-dnsmasq-64864b6d57-wjkrf\" (UID: \"cde663cf-e7e2-42b1-9581-ea7a27f43481\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-64864b6d57-wjkrf" Dec 03 13:10:01 crc kubenswrapper[5052]: I1203 13:10:01.776259 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/cde663cf-e7e2-42b1-9581-ea7a27f43481-config\") pod \"dnsmasq-dnsmasq-64864b6d57-wjkrf\" (UID: \"cde663cf-e7e2-42b1-9581-ea7a27f43481\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-64864b6d57-wjkrf" Dec 03 13:10:01 crc kubenswrapper[5052]: I1203 13:10:01.776304 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/cde663cf-e7e2-42b1-9581-ea7a27f43481-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-64864b6d57-wjkrf\" (UID: \"cde663cf-e7e2-42b1-9581-ea7a27f43481\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-64864b6d57-wjkrf" Dec 03 13:10:01 crc kubenswrapper[5052]: I1203 13:10:01.877544 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"edpm-compute-no-nodes\" (UniqueName: \"kubernetes.io/configmap/cde663cf-e7e2-42b1-9581-ea7a27f43481-edpm-compute-no-nodes\") pod \"dnsmasq-dnsmasq-64864b6d57-wjkrf\" (UID: \"cde663cf-e7e2-42b1-9581-ea7a27f43481\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-64864b6d57-wjkrf" Dec 03 13:10:01 crc kubenswrapper[5052]: I1203 13:10:01.877623 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-dlrnr\" (UniqueName: \"kubernetes.io/projected/cde663cf-e7e2-42b1-9581-ea7a27f43481-kube-api-access-dlrnr\") pod \"dnsmasq-dnsmasq-64864b6d57-wjkrf\" (UID: \"cde663cf-e7e2-42b1-9581-ea7a27f43481\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-64864b6d57-wjkrf" Dec 03 13:10:01 crc kubenswrapper[5052]: I1203 13:10:01.877647 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/cde663cf-e7e2-42b1-9581-ea7a27f43481-config\") pod \"dnsmasq-dnsmasq-64864b6d57-wjkrf\" (UID: \"cde663cf-e7e2-42b1-9581-ea7a27f43481\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-64864b6d57-wjkrf" Dec 03 13:10:01 crc kubenswrapper[5052]: I1203 13:10:01.877679 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/cde663cf-e7e2-42b1-9581-ea7a27f43481-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-64864b6d57-wjkrf\" (UID: \"cde663cf-e7e2-42b1-9581-ea7a27f43481\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-64864b6d57-wjkrf" Dec 03 13:10:01 crc kubenswrapper[5052]: I1203 13:10:01.878616 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/cde663cf-e7e2-42b1-9581-ea7a27f43481-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-64864b6d57-wjkrf\" (UID: \"cde663cf-e7e2-42b1-9581-ea7a27f43481\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-64864b6d57-wjkrf" Dec 03 13:10:01 crc kubenswrapper[5052]: I1203 13:10:01.878692 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/cde663cf-e7e2-42b1-9581-ea7a27f43481-config\") pod \"dnsmasq-dnsmasq-64864b6d57-wjkrf\" (UID: \"cde663cf-e7e2-42b1-9581-ea7a27f43481\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-64864b6d57-wjkrf" Dec 03 13:10:01 crc kubenswrapper[5052]: I1203 13:10:01.878864 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"edpm-compute-no-nodes\" (UniqueName: \"kubernetes.io/configmap/cde663cf-e7e2-42b1-9581-ea7a27f43481-edpm-compute-no-nodes\") pod \"dnsmasq-dnsmasq-64864b6d57-wjkrf\" (UID: \"cde663cf-e7e2-42b1-9581-ea7a27f43481\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-64864b6d57-wjkrf" Dec 03 13:10:01 crc kubenswrapper[5052]: I1203 13:10:01.901133 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-dlrnr\" (UniqueName: \"kubernetes.io/projected/cde663cf-e7e2-42b1-9581-ea7a27f43481-kube-api-access-dlrnr\") pod \"dnsmasq-dnsmasq-64864b6d57-wjkrf\" (UID: \"cde663cf-e7e2-42b1-9581-ea7a27f43481\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-64864b6d57-wjkrf" Dec 03 13:10:02 crc kubenswrapper[5052]: I1203 13:10:02.065940 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-64864b6d57-wjkrf" Dec 03 13:10:02 crc kubenswrapper[5052]: I1203 13:10:02.510020 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-64864b6d57-wjkrf"] Dec 03 13:10:02 crc kubenswrapper[5052]: I1203 13:10:02.922177 5052 generic.go:334] "Generic (PLEG): container finished" podID="cde663cf-e7e2-42b1-9581-ea7a27f43481" containerID="0d231e87cdac7a1441d49513b13649a70fdacb9619a3e661990497f875952595" exitCode=0 Dec 03 13:10:02 crc kubenswrapper[5052]: I1203 13:10:02.922230 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-64864b6d57-wjkrf" event={"ID":"cde663cf-e7e2-42b1-9581-ea7a27f43481","Type":"ContainerDied","Data":"0d231e87cdac7a1441d49513b13649a70fdacb9619a3e661990497f875952595"} Dec 03 13:10:02 crc kubenswrapper[5052]: I1203 13:10:02.922286 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-64864b6d57-wjkrf" event={"ID":"cde663cf-e7e2-42b1-9581-ea7a27f43481","Type":"ContainerStarted","Data":"b21d4232aee992bb16d83c0aac5a2d0722d84c925a5f4eebe000444a287fb4d0"} Dec 03 13:10:03 crc kubenswrapper[5052]: I1203 13:10:03.574561 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openshift-marketplace/certified-operators-dkf2f"] Dec 03 13:10:03 crc kubenswrapper[5052]: I1203 13:10:03.576483 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/certified-operators-dkf2f" Dec 03 13:10:03 crc kubenswrapper[5052]: I1203 13:10:03.587867 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/certified-operators-dkf2f"] Dec 03 13:10:03 crc kubenswrapper[5052]: I1203 13:10:03.704947 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/919f8203-0bfd-4470-8311-76d24f152b46-utilities\") pod \"certified-operators-dkf2f\" (UID: \"919f8203-0bfd-4470-8311-76d24f152b46\") " pod="openshift-marketplace/certified-operators-dkf2f" Dec 03 13:10:03 crc kubenswrapper[5052]: I1203 13:10:03.705042 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-74cps\" (UniqueName: \"kubernetes.io/projected/919f8203-0bfd-4470-8311-76d24f152b46-kube-api-access-74cps\") pod \"certified-operators-dkf2f\" (UID: \"919f8203-0bfd-4470-8311-76d24f152b46\") " pod="openshift-marketplace/certified-operators-dkf2f" Dec 03 13:10:03 crc kubenswrapper[5052]: I1203 13:10:03.705187 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/919f8203-0bfd-4470-8311-76d24f152b46-catalog-content\") pod \"certified-operators-dkf2f\" (UID: \"919f8203-0bfd-4470-8311-76d24f152b46\") " pod="openshift-marketplace/certified-operators-dkf2f" Dec 03 13:10:03 crc kubenswrapper[5052]: I1203 13:10:03.806983 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/919f8203-0bfd-4470-8311-76d24f152b46-utilities\") pod \"certified-operators-dkf2f\" (UID: \"919f8203-0bfd-4470-8311-76d24f152b46\") " pod="openshift-marketplace/certified-operators-dkf2f" Dec 03 13:10:03 crc kubenswrapper[5052]: I1203 13:10:03.807048 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-74cps\" (UniqueName: \"kubernetes.io/projected/919f8203-0bfd-4470-8311-76d24f152b46-kube-api-access-74cps\") pod \"certified-operators-dkf2f\" (UID: \"919f8203-0bfd-4470-8311-76d24f152b46\") " pod="openshift-marketplace/certified-operators-dkf2f" Dec 03 13:10:03 crc kubenswrapper[5052]: I1203 13:10:03.807141 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/919f8203-0bfd-4470-8311-76d24f152b46-catalog-content\") pod \"certified-operators-dkf2f\" (UID: \"919f8203-0bfd-4470-8311-76d24f152b46\") " pod="openshift-marketplace/certified-operators-dkf2f" Dec 03 13:10:03 crc kubenswrapper[5052]: I1203 13:10:03.807605 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/919f8203-0bfd-4470-8311-76d24f152b46-utilities\") pod \"certified-operators-dkf2f\" (UID: \"919f8203-0bfd-4470-8311-76d24f152b46\") " pod="openshift-marketplace/certified-operators-dkf2f" Dec 03 13:10:03 crc kubenswrapper[5052]: I1203 13:10:03.807812 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/919f8203-0bfd-4470-8311-76d24f152b46-catalog-content\") pod \"certified-operators-dkf2f\" (UID: \"919f8203-0bfd-4470-8311-76d24f152b46\") " pod="openshift-marketplace/certified-operators-dkf2f" Dec 03 13:10:03 crc kubenswrapper[5052]: I1203 13:10:03.827009 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-74cps\" (UniqueName: \"kubernetes.io/projected/919f8203-0bfd-4470-8311-76d24f152b46-kube-api-access-74cps\") pod \"certified-operators-dkf2f\" (UID: \"919f8203-0bfd-4470-8311-76d24f152b46\") " pod="openshift-marketplace/certified-operators-dkf2f" Dec 03 13:10:03 crc kubenswrapper[5052]: I1203 13:10:03.898143 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/certified-operators-dkf2f" Dec 03 13:10:03 crc kubenswrapper[5052]: I1203 13:10:03.940394 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-64864b6d57-wjkrf" event={"ID":"cde663cf-e7e2-42b1-9581-ea7a27f43481","Type":"ContainerStarted","Data":"d5863db69cdfdebc8e10ce524ecd40405cc19e2ff443d5a3b4bfd94d1146db1c"} Dec 03 13:10:03 crc kubenswrapper[5052]: I1203 13:10:03.941194 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-64864b6d57-wjkrf" Dec 03 13:10:03 crc kubenswrapper[5052]: I1203 13:10:03.965621 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-64864b6d57-wjkrf" podStartSLOduration=2.965600138 podStartE2EDuration="2.965600138s" podCreationTimestamp="2025-12-03 13:10:01 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 13:10:03.964781935 +0000 UTC m=+6594.553023699" watchObservedRunningTime="2025-12-03 13:10:03.965600138 +0000 UTC m=+6594.553841902" Dec 03 13:10:04 crc kubenswrapper[5052]: I1203 13:10:04.411961 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/certified-operators-dkf2f"] Dec 03 13:10:04 crc kubenswrapper[5052]: I1203 13:10:04.950712 5052 generic.go:334] "Generic (PLEG): container finished" podID="919f8203-0bfd-4470-8311-76d24f152b46" containerID="289c9aa2a69d8523eb8b7a8c71085942c6e95d27bb323140d5a40f257af90a61" exitCode=0 Dec 03 13:10:04 crc kubenswrapper[5052]: I1203 13:10:04.950765 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/certified-operators-dkf2f" event={"ID":"919f8203-0bfd-4470-8311-76d24f152b46","Type":"ContainerDied","Data":"289c9aa2a69d8523eb8b7a8c71085942c6e95d27bb323140d5a40f257af90a61"} Dec 03 13:10:04 crc kubenswrapper[5052]: I1203 13:10:04.950807 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/certified-operators-dkf2f" event={"ID":"919f8203-0bfd-4470-8311-76d24f152b46","Type":"ContainerStarted","Data":"0f4984a6b7a3d8e65680f12d0c28c5a6acc080bc0678acadad852f1cd57343f7"} Dec 03 13:10:05 crc kubenswrapper[5052]: I1203 13:10:05.962613 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/certified-operators-dkf2f" event={"ID":"919f8203-0bfd-4470-8311-76d24f152b46","Type":"ContainerStarted","Data":"3d7ea20aa643bda689d03143d9c83ca4168d5d49b9523b3d1cd87949afc2fe42"} Dec 03 13:10:06 crc kubenswrapper[5052]: I1203 13:10:06.988632 5052 generic.go:334] "Generic (PLEG): container finished" podID="919f8203-0bfd-4470-8311-76d24f152b46" containerID="3d7ea20aa643bda689d03143d9c83ca4168d5d49b9523b3d1cd87949afc2fe42" exitCode=0 Dec 03 13:10:06 crc kubenswrapper[5052]: I1203 13:10:06.988691 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/certified-operators-dkf2f" event={"ID":"919f8203-0bfd-4470-8311-76d24f152b46","Type":"ContainerDied","Data":"3d7ea20aa643bda689d03143d9c83ca4168d5d49b9523b3d1cd87949afc2fe42"} Dec 03 13:10:07 crc kubenswrapper[5052]: I1203 13:10:07.999622 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/certified-operators-dkf2f" event={"ID":"919f8203-0bfd-4470-8311-76d24f152b46","Type":"ContainerStarted","Data":"16303c3f95d4ec20aba16cf2e1070ba56108f371c04557baf90f23e48a86eb94"} Dec 03 13:10:08 crc kubenswrapper[5052]: I1203 13:10:08.022840 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-marketplace/certified-operators-dkf2f" podStartSLOduration=2.348708097 podStartE2EDuration="5.022821307s" podCreationTimestamp="2025-12-03 13:10:03 +0000 UTC" firstStartedPulling="2025-12-03 13:10:04.952289375 +0000 UTC m=+6595.540531149" lastFinishedPulling="2025-12-03 13:10:07.626402595 +0000 UTC m=+6598.214644359" observedRunningTime="2025-12-03 13:10:08.018137713 +0000 UTC m=+6598.606379477" watchObservedRunningTime="2025-12-03 13:10:08.022821307 +0000 UTC m=+6598.611063071" Dec 03 13:10:08 crc kubenswrapper[5052]: I1203 13:10:08.673902 5052 scope.go:117] "RemoveContainer" containerID="dbc100c8081ea1a2f5487429cc9d67de6bc8ba11282b64f248c71275375b0e16" Dec 03 13:10:08 crc kubenswrapper[5052]: E1203 13:10:08.674125 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 13:10:12 crc kubenswrapper[5052]: I1203 13:10:12.074712 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-64864b6d57-wjkrf" Dec 03 13:10:12 crc kubenswrapper[5052]: I1203 13:10:12.144154 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-ngbbh"] Dec 03 13:10:12 crc kubenswrapper[5052]: I1203 13:10:12.144424 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-ngbbh" podUID="db571454-c5ad-46f4-bc1e-9c29cc32d921" containerName="dnsmasq-dns" containerID="cri-o://1fe185d294d533560cf7c402a020f267cfea0c63d5f4a4e848fb594451b7c726" gracePeriod=10 Dec 03 13:10:13 crc kubenswrapper[5052]: I1203 13:10:13.044008 5052 generic.go:334] "Generic (PLEG): container finished" podID="db571454-c5ad-46f4-bc1e-9c29cc32d921" containerID="1fe185d294d533560cf7c402a020f267cfea0c63d5f4a4e848fb594451b7c726" exitCode=0 Dec 03 13:10:13 crc kubenswrapper[5052]: I1203 13:10:13.044344 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-ngbbh" event={"ID":"db571454-c5ad-46f4-bc1e-9c29cc32d921","Type":"ContainerDied","Data":"1fe185d294d533560cf7c402a020f267cfea0c63d5f4a4e848fb594451b7c726"} Dec 03 13:10:13 crc kubenswrapper[5052]: I1203 13:10:13.044377 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-ngbbh" event={"ID":"db571454-c5ad-46f4-bc1e-9c29cc32d921","Type":"ContainerDied","Data":"9e73f8b3334441b2fa3e0e8a053db389076e57ea5c1d622c54d00b5c0ff8a35c"} Dec 03 13:10:13 crc kubenswrapper[5052]: I1203 13:10:13.044391 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9e73f8b3334441b2fa3e0e8a053db389076e57ea5c1d622c54d00b5c0ff8a35c" Dec 03 13:10:13 crc kubenswrapper[5052]: I1203 13:10:13.065672 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-ngbbh" Dec 03 13:10:13 crc kubenswrapper[5052]: I1203 13:10:13.245543 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/db571454-c5ad-46f4-bc1e-9c29cc32d921-config\") pod \"db571454-c5ad-46f4-bc1e-9c29cc32d921\" (UID: \"db571454-c5ad-46f4-bc1e-9c29cc32d921\") " Dec 03 13:10:13 crc kubenswrapper[5052]: I1203 13:10:13.245675 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8f6cn\" (UniqueName: \"kubernetes.io/projected/db571454-c5ad-46f4-bc1e-9c29cc32d921-kube-api-access-8f6cn\") pod \"db571454-c5ad-46f4-bc1e-9c29cc32d921\" (UID: \"db571454-c5ad-46f4-bc1e-9c29cc32d921\") " Dec 03 13:10:13 crc kubenswrapper[5052]: I1203 13:10:13.245750 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/db571454-c5ad-46f4-bc1e-9c29cc32d921-dnsmasq-svc\") pod \"db571454-c5ad-46f4-bc1e-9c29cc32d921\" (UID: \"db571454-c5ad-46f4-bc1e-9c29cc32d921\") " Dec 03 13:10:13 crc kubenswrapper[5052]: I1203 13:10:13.251129 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/db571454-c5ad-46f4-bc1e-9c29cc32d921-kube-api-access-8f6cn" (OuterVolumeSpecName: "kube-api-access-8f6cn") pod "db571454-c5ad-46f4-bc1e-9c29cc32d921" (UID: "db571454-c5ad-46f4-bc1e-9c29cc32d921"). InnerVolumeSpecName "kube-api-access-8f6cn". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:10:13 crc kubenswrapper[5052]: I1203 13:10:13.287217 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/db571454-c5ad-46f4-bc1e-9c29cc32d921-dnsmasq-svc" (OuterVolumeSpecName: "dnsmasq-svc") pod "db571454-c5ad-46f4-bc1e-9c29cc32d921" (UID: "db571454-c5ad-46f4-bc1e-9c29cc32d921"). InnerVolumeSpecName "dnsmasq-svc". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:10:13 crc kubenswrapper[5052]: I1203 13:10:13.296152 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/db571454-c5ad-46f4-bc1e-9c29cc32d921-config" (OuterVolumeSpecName: "config") pod "db571454-c5ad-46f4-bc1e-9c29cc32d921" (UID: "db571454-c5ad-46f4-bc1e-9c29cc32d921"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:10:13 crc kubenswrapper[5052]: I1203 13:10:13.347028 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-8f6cn\" (UniqueName: \"kubernetes.io/projected/db571454-c5ad-46f4-bc1e-9c29cc32d921-kube-api-access-8f6cn\") on node \"crc\" DevicePath \"\"" Dec 03 13:10:13 crc kubenswrapper[5052]: I1203 13:10:13.347064 5052 reconciler_common.go:293] "Volume detached for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/db571454-c5ad-46f4-bc1e-9c29cc32d921-dnsmasq-svc\") on node \"crc\" DevicePath \"\"" Dec 03 13:10:13 crc kubenswrapper[5052]: I1203 13:10:13.347081 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/db571454-c5ad-46f4-bc1e-9c29cc32d921-config\") on node \"crc\" DevicePath \"\"" Dec 03 13:10:13 crc kubenswrapper[5052]: I1203 13:10:13.898931 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-marketplace/certified-operators-dkf2f" Dec 03 13:10:13 crc kubenswrapper[5052]: I1203 13:10:13.898987 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openshift-marketplace/certified-operators-dkf2f" Dec 03 13:10:13 crc kubenswrapper[5052]: I1203 13:10:13.947062 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-marketplace/certified-operators-dkf2f" Dec 03 13:10:14 crc kubenswrapper[5052]: I1203 13:10:14.054023 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-ngbbh" Dec 03 13:10:14 crc kubenswrapper[5052]: I1203 13:10:14.077961 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-ngbbh"] Dec 03 13:10:14 crc kubenswrapper[5052]: I1203 13:10:14.085132 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-ngbbh"] Dec 03 13:10:14 crc kubenswrapper[5052]: I1203 13:10:14.106630 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-marketplace/certified-operators-dkf2f" Dec 03 13:10:14 crc kubenswrapper[5052]: I1203 13:10:14.185299 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openshift-marketplace/certified-operators-dkf2f"] Dec 03 13:10:15 crc kubenswrapper[5052]: I1203 13:10:15.683818 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="db571454-c5ad-46f4-bc1e-9c29cc32d921" path="/var/lib/kubelet/pods/db571454-c5ad-46f4-bc1e-9c29cc32d921/volumes" Dec 03 13:10:16 crc kubenswrapper[5052]: I1203 13:10:16.073471 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openshift-marketplace/certified-operators-dkf2f" podUID="919f8203-0bfd-4470-8311-76d24f152b46" containerName="registry-server" containerID="cri-o://16303c3f95d4ec20aba16cf2e1070ba56108f371c04557baf90f23e48a86eb94" gracePeriod=2 Dec 03 13:10:16 crc kubenswrapper[5052]: I1203 13:10:16.652855 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/failed-service-edpm-compute-no-nodes-edpm-compute-no-nodesr47df"] Dec 03 13:10:16 crc kubenswrapper[5052]: E1203 13:10:16.653206 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="db571454-c5ad-46f4-bc1e-9c29cc32d921" containerName="dnsmasq-dns" Dec 03 13:10:16 crc kubenswrapper[5052]: I1203 13:10:16.653219 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="db571454-c5ad-46f4-bc1e-9c29cc32d921" containerName="dnsmasq-dns" Dec 03 13:10:16 crc kubenswrapper[5052]: E1203 13:10:16.653243 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="db571454-c5ad-46f4-bc1e-9c29cc32d921" containerName="init" Dec 03 13:10:16 crc kubenswrapper[5052]: I1203 13:10:16.653249 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="db571454-c5ad-46f4-bc1e-9c29cc32d921" containerName="init" Dec 03 13:10:16 crc kubenswrapper[5052]: I1203 13:10:16.653424 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="db571454-c5ad-46f4-bc1e-9c29cc32d921" containerName="dnsmasq-dns" Dec 03 13:10:16 crc kubenswrapper[5052]: I1203 13:10:16.654051 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/failed-service-edpm-compute-no-nodes-edpm-compute-no-nodesr47df" Dec 03 13:10:16 crc kubenswrapper[5052]: I1203 13:10:16.656158 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"combined-ca-bundle" Dec 03 13:10:16 crc kubenswrapper[5052]: I1203 13:10:16.657700 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"dataplane-ansible-ssh-private-key-secret" Dec 03 13:10:16 crc kubenswrapper[5052]: I1203 13:10:16.657963 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"openstack-aee-default-env" Dec 03 13:10:16 crc kubenswrapper[5052]: I1203 13:10:16.661242 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"dataplanenodeset-edpm-compute-no-nodes" Dec 03 13:10:16 crc kubenswrapper[5052]: I1203 13:10:16.663347 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"edpm-compute-no-nodes-dockercfg-7spm4" Dec 03 13:10:16 crc kubenswrapper[5052]: I1203 13:10:16.674311 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/failed-service-edpm-compute-no-nodes-edpm-compute-no-nodesr47df"] Dec 03 13:10:16 crc kubenswrapper[5052]: I1203 13:10:16.801242 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/b9427361-e875-4d16-b3a6-cf4629dc9d00-ssh-key\") pod \"failed-service-edpm-compute-no-nodes-edpm-compute-no-nodesr47df\" (UID: \"b9427361-e875-4d16-b3a6-cf4629dc9d00\") " pod="openstack-kuttl-tests/failed-service-edpm-compute-no-nodes-edpm-compute-no-nodesr47df" Dec 03 13:10:16 crc kubenswrapper[5052]: I1203 13:10:16.801371 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"failed-service-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b9427361-e875-4d16-b3a6-cf4629dc9d00-failed-service-combined-ca-bundle\") pod \"failed-service-edpm-compute-no-nodes-edpm-compute-no-nodesr47df\" (UID: \"b9427361-e875-4d16-b3a6-cf4629dc9d00\") " pod="openstack-kuttl-tests/failed-service-edpm-compute-no-nodes-edpm-compute-no-nodesr47df" Dec 03 13:10:16 crc kubenswrapper[5052]: I1203 13:10:16.801570 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/b9427361-e875-4d16-b3a6-cf4629dc9d00-inventory\") pod \"failed-service-edpm-compute-no-nodes-edpm-compute-no-nodesr47df\" (UID: \"b9427361-e875-4d16-b3a6-cf4629dc9d00\") " pod="openstack-kuttl-tests/failed-service-edpm-compute-no-nodes-edpm-compute-no-nodesr47df" Dec 03 13:10:16 crc kubenswrapper[5052]: I1203 13:10:16.801714 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-twcmr\" (UniqueName: \"kubernetes.io/projected/b9427361-e875-4d16-b3a6-cf4629dc9d00-kube-api-access-twcmr\") pod \"failed-service-edpm-compute-no-nodes-edpm-compute-no-nodesr47df\" (UID: \"b9427361-e875-4d16-b3a6-cf4629dc9d00\") " pod="openstack-kuttl-tests/failed-service-edpm-compute-no-nodes-edpm-compute-no-nodesr47df" Dec 03 13:10:16 crc kubenswrapper[5052]: I1203 13:10:16.903657 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-twcmr\" (UniqueName: \"kubernetes.io/projected/b9427361-e875-4d16-b3a6-cf4629dc9d00-kube-api-access-twcmr\") pod \"failed-service-edpm-compute-no-nodes-edpm-compute-no-nodesr47df\" (UID: \"b9427361-e875-4d16-b3a6-cf4629dc9d00\") " pod="openstack-kuttl-tests/failed-service-edpm-compute-no-nodes-edpm-compute-no-nodesr47df" Dec 03 13:10:16 crc kubenswrapper[5052]: I1203 13:10:16.904072 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/b9427361-e875-4d16-b3a6-cf4629dc9d00-ssh-key\") pod \"failed-service-edpm-compute-no-nodes-edpm-compute-no-nodesr47df\" (UID: \"b9427361-e875-4d16-b3a6-cf4629dc9d00\") " pod="openstack-kuttl-tests/failed-service-edpm-compute-no-nodes-edpm-compute-no-nodesr47df" Dec 03 13:10:16 crc kubenswrapper[5052]: I1203 13:10:16.904126 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"failed-service-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b9427361-e875-4d16-b3a6-cf4629dc9d00-failed-service-combined-ca-bundle\") pod \"failed-service-edpm-compute-no-nodes-edpm-compute-no-nodesr47df\" (UID: \"b9427361-e875-4d16-b3a6-cf4629dc9d00\") " pod="openstack-kuttl-tests/failed-service-edpm-compute-no-nodes-edpm-compute-no-nodesr47df" Dec 03 13:10:16 crc kubenswrapper[5052]: I1203 13:10:16.904216 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/b9427361-e875-4d16-b3a6-cf4629dc9d00-inventory\") pod \"failed-service-edpm-compute-no-nodes-edpm-compute-no-nodesr47df\" (UID: \"b9427361-e875-4d16-b3a6-cf4629dc9d00\") " pod="openstack-kuttl-tests/failed-service-edpm-compute-no-nodes-edpm-compute-no-nodesr47df" Dec 03 13:10:16 crc kubenswrapper[5052]: I1203 13:10:16.910606 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/b9427361-e875-4d16-b3a6-cf4629dc9d00-inventory\") pod \"failed-service-edpm-compute-no-nodes-edpm-compute-no-nodesr47df\" (UID: \"b9427361-e875-4d16-b3a6-cf4629dc9d00\") " pod="openstack-kuttl-tests/failed-service-edpm-compute-no-nodes-edpm-compute-no-nodesr47df" Dec 03 13:10:16 crc kubenswrapper[5052]: I1203 13:10:16.910632 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/b9427361-e875-4d16-b3a6-cf4629dc9d00-ssh-key\") pod \"failed-service-edpm-compute-no-nodes-edpm-compute-no-nodesr47df\" (UID: \"b9427361-e875-4d16-b3a6-cf4629dc9d00\") " pod="openstack-kuttl-tests/failed-service-edpm-compute-no-nodes-edpm-compute-no-nodesr47df" Dec 03 13:10:16 crc kubenswrapper[5052]: I1203 13:10:16.911238 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"failed-service-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b9427361-e875-4d16-b3a6-cf4629dc9d00-failed-service-combined-ca-bundle\") pod \"failed-service-edpm-compute-no-nodes-edpm-compute-no-nodesr47df\" (UID: \"b9427361-e875-4d16-b3a6-cf4629dc9d00\") " pod="openstack-kuttl-tests/failed-service-edpm-compute-no-nodes-edpm-compute-no-nodesr47df" Dec 03 13:10:16 crc kubenswrapper[5052]: I1203 13:10:16.927211 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-twcmr\" (UniqueName: \"kubernetes.io/projected/b9427361-e875-4d16-b3a6-cf4629dc9d00-kube-api-access-twcmr\") pod \"failed-service-edpm-compute-no-nodes-edpm-compute-no-nodesr47df\" (UID: \"b9427361-e875-4d16-b3a6-cf4629dc9d00\") " pod="openstack-kuttl-tests/failed-service-edpm-compute-no-nodes-edpm-compute-no-nodesr47df" Dec 03 13:10:16 crc kubenswrapper[5052]: I1203 13:10:16.975800 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/failed-service-edpm-compute-no-nodes-edpm-compute-no-nodesr47df" Dec 03 13:10:17 crc kubenswrapper[5052]: I1203 13:10:17.004494 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/certified-operators-dkf2f" Dec 03 13:10:17 crc kubenswrapper[5052]: I1203 13:10:17.086878 5052 generic.go:334] "Generic (PLEG): container finished" podID="919f8203-0bfd-4470-8311-76d24f152b46" containerID="16303c3f95d4ec20aba16cf2e1070ba56108f371c04557baf90f23e48a86eb94" exitCode=0 Dec 03 13:10:17 crc kubenswrapper[5052]: I1203 13:10:17.086961 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/certified-operators-dkf2f" event={"ID":"919f8203-0bfd-4470-8311-76d24f152b46","Type":"ContainerDied","Data":"16303c3f95d4ec20aba16cf2e1070ba56108f371c04557baf90f23e48a86eb94"} Dec 03 13:10:17 crc kubenswrapper[5052]: I1203 13:10:17.087226 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/certified-operators-dkf2f" event={"ID":"919f8203-0bfd-4470-8311-76d24f152b46","Type":"ContainerDied","Data":"0f4984a6b7a3d8e65680f12d0c28c5a6acc080bc0678acadad852f1cd57343f7"} Dec 03 13:10:17 crc kubenswrapper[5052]: I1203 13:10:17.086975 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/certified-operators-dkf2f" Dec 03 13:10:17 crc kubenswrapper[5052]: I1203 13:10:17.087244 5052 scope.go:117] "RemoveContainer" containerID="16303c3f95d4ec20aba16cf2e1070ba56108f371c04557baf90f23e48a86eb94" Dec 03 13:10:17 crc kubenswrapper[5052]: I1203 13:10:17.113405 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/919f8203-0bfd-4470-8311-76d24f152b46-utilities\") pod \"919f8203-0bfd-4470-8311-76d24f152b46\" (UID: \"919f8203-0bfd-4470-8311-76d24f152b46\") " Dec 03 13:10:17 crc kubenswrapper[5052]: I1203 13:10:17.113600 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/919f8203-0bfd-4470-8311-76d24f152b46-catalog-content\") pod \"919f8203-0bfd-4470-8311-76d24f152b46\" (UID: \"919f8203-0bfd-4470-8311-76d24f152b46\") " Dec 03 13:10:17 crc kubenswrapper[5052]: I1203 13:10:17.113722 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-74cps\" (UniqueName: \"kubernetes.io/projected/919f8203-0bfd-4470-8311-76d24f152b46-kube-api-access-74cps\") pod \"919f8203-0bfd-4470-8311-76d24f152b46\" (UID: \"919f8203-0bfd-4470-8311-76d24f152b46\") " Dec 03 13:10:17 crc kubenswrapper[5052]: I1203 13:10:17.114746 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/919f8203-0bfd-4470-8311-76d24f152b46-utilities" (OuterVolumeSpecName: "utilities") pod "919f8203-0bfd-4470-8311-76d24f152b46" (UID: "919f8203-0bfd-4470-8311-76d24f152b46"). InnerVolumeSpecName "utilities". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 13:10:17 crc kubenswrapper[5052]: I1203 13:10:17.118558 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/919f8203-0bfd-4470-8311-76d24f152b46-kube-api-access-74cps" (OuterVolumeSpecName: "kube-api-access-74cps") pod "919f8203-0bfd-4470-8311-76d24f152b46" (UID: "919f8203-0bfd-4470-8311-76d24f152b46"). InnerVolumeSpecName "kube-api-access-74cps". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:10:17 crc kubenswrapper[5052]: I1203 13:10:17.124366 5052 scope.go:117] "RemoveContainer" containerID="3d7ea20aa643bda689d03143d9c83ca4168d5d49b9523b3d1cd87949afc2fe42" Dec 03 13:10:17 crc kubenswrapper[5052]: I1203 13:10:17.157063 5052 scope.go:117] "RemoveContainer" containerID="289c9aa2a69d8523eb8b7a8c71085942c6e95d27bb323140d5a40f257af90a61" Dec 03 13:10:17 crc kubenswrapper[5052]: I1203 13:10:17.178158 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/919f8203-0bfd-4470-8311-76d24f152b46-catalog-content" (OuterVolumeSpecName: "catalog-content") pod "919f8203-0bfd-4470-8311-76d24f152b46" (UID: "919f8203-0bfd-4470-8311-76d24f152b46"). InnerVolumeSpecName "catalog-content". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 13:10:17 crc kubenswrapper[5052]: I1203 13:10:17.203000 5052 scope.go:117] "RemoveContainer" containerID="16303c3f95d4ec20aba16cf2e1070ba56108f371c04557baf90f23e48a86eb94" Dec 03 13:10:17 crc kubenswrapper[5052]: E1203 13:10:17.203577 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"16303c3f95d4ec20aba16cf2e1070ba56108f371c04557baf90f23e48a86eb94\": container with ID starting with 16303c3f95d4ec20aba16cf2e1070ba56108f371c04557baf90f23e48a86eb94 not found: ID does not exist" containerID="16303c3f95d4ec20aba16cf2e1070ba56108f371c04557baf90f23e48a86eb94" Dec 03 13:10:17 crc kubenswrapper[5052]: I1203 13:10:17.203618 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"16303c3f95d4ec20aba16cf2e1070ba56108f371c04557baf90f23e48a86eb94"} err="failed to get container status \"16303c3f95d4ec20aba16cf2e1070ba56108f371c04557baf90f23e48a86eb94\": rpc error: code = NotFound desc = could not find container \"16303c3f95d4ec20aba16cf2e1070ba56108f371c04557baf90f23e48a86eb94\": container with ID starting with 16303c3f95d4ec20aba16cf2e1070ba56108f371c04557baf90f23e48a86eb94 not found: ID does not exist" Dec 03 13:10:17 crc kubenswrapper[5052]: I1203 13:10:17.203652 5052 scope.go:117] "RemoveContainer" containerID="3d7ea20aa643bda689d03143d9c83ca4168d5d49b9523b3d1cd87949afc2fe42" Dec 03 13:10:17 crc kubenswrapper[5052]: E1203 13:10:17.203880 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3d7ea20aa643bda689d03143d9c83ca4168d5d49b9523b3d1cd87949afc2fe42\": container with ID starting with 3d7ea20aa643bda689d03143d9c83ca4168d5d49b9523b3d1cd87949afc2fe42 not found: ID does not exist" containerID="3d7ea20aa643bda689d03143d9c83ca4168d5d49b9523b3d1cd87949afc2fe42" Dec 03 13:10:17 crc kubenswrapper[5052]: I1203 13:10:17.203910 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3d7ea20aa643bda689d03143d9c83ca4168d5d49b9523b3d1cd87949afc2fe42"} err="failed to get container status \"3d7ea20aa643bda689d03143d9c83ca4168d5d49b9523b3d1cd87949afc2fe42\": rpc error: code = NotFound desc = could not find container \"3d7ea20aa643bda689d03143d9c83ca4168d5d49b9523b3d1cd87949afc2fe42\": container with ID starting with 3d7ea20aa643bda689d03143d9c83ca4168d5d49b9523b3d1cd87949afc2fe42 not found: ID does not exist" Dec 03 13:10:17 crc kubenswrapper[5052]: I1203 13:10:17.203928 5052 scope.go:117] "RemoveContainer" containerID="289c9aa2a69d8523eb8b7a8c71085942c6e95d27bb323140d5a40f257af90a61" Dec 03 13:10:17 crc kubenswrapper[5052]: E1203 13:10:17.204180 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"289c9aa2a69d8523eb8b7a8c71085942c6e95d27bb323140d5a40f257af90a61\": container with ID starting with 289c9aa2a69d8523eb8b7a8c71085942c6e95d27bb323140d5a40f257af90a61 not found: ID does not exist" containerID="289c9aa2a69d8523eb8b7a8c71085942c6e95d27bb323140d5a40f257af90a61" Dec 03 13:10:17 crc kubenswrapper[5052]: I1203 13:10:17.204201 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"289c9aa2a69d8523eb8b7a8c71085942c6e95d27bb323140d5a40f257af90a61"} err="failed to get container status \"289c9aa2a69d8523eb8b7a8c71085942c6e95d27bb323140d5a40f257af90a61\": rpc error: code = NotFound desc = could not find container \"289c9aa2a69d8523eb8b7a8c71085942c6e95d27bb323140d5a40f257af90a61\": container with ID starting with 289c9aa2a69d8523eb8b7a8c71085942c6e95d27bb323140d5a40f257af90a61 not found: ID does not exist" Dec 03 13:10:17 crc kubenswrapper[5052]: I1203 13:10:17.214883 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-74cps\" (UniqueName: \"kubernetes.io/projected/919f8203-0bfd-4470-8311-76d24f152b46-kube-api-access-74cps\") on node \"crc\" DevicePath \"\"" Dec 03 13:10:17 crc kubenswrapper[5052]: I1203 13:10:17.214926 5052 reconciler_common.go:293] "Volume detached for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/919f8203-0bfd-4470-8311-76d24f152b46-utilities\") on node \"crc\" DevicePath \"\"" Dec 03 13:10:17 crc kubenswrapper[5052]: I1203 13:10:17.214944 5052 reconciler_common.go:293] "Volume detached for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/919f8203-0bfd-4470-8311-76d24f152b46-catalog-content\") on node \"crc\" DevicePath \"\"" Dec 03 13:10:17 crc kubenswrapper[5052]: I1203 13:10:17.423937 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openshift-marketplace/certified-operators-dkf2f"] Dec 03 13:10:17 crc kubenswrapper[5052]: I1203 13:10:17.430785 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openshift-marketplace/certified-operators-dkf2f"] Dec 03 13:10:17 crc kubenswrapper[5052]: I1203 13:10:17.474375 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/failed-service-edpm-compute-no-nodes-edpm-compute-no-nodesr47df"] Dec 03 13:10:17 crc kubenswrapper[5052]: I1203 13:10:17.686391 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="919f8203-0bfd-4470-8311-76d24f152b46" path="/var/lib/kubelet/pods/919f8203-0bfd-4470-8311-76d24f152b46/volumes" Dec 03 13:10:17 crc kubenswrapper[5052]: I1203 13:10:17.937481 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-ngbbh" podUID="db571454-c5ad-46f4-bc1e-9c29cc32d921" containerName="dnsmasq-dns" probeResult="failure" output="dial tcp 10.217.1.112:5353: i/o timeout" Dec 03 13:10:18 crc kubenswrapper[5052]: I1203 13:10:18.098924 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/failed-service-edpm-compute-no-nodes-edpm-compute-no-nodesr47df" event={"ID":"b9427361-e875-4d16-b3a6-cf4629dc9d00","Type":"ContainerStarted","Data":"4ca80fe41e48c4d6a838710b3baa2c10617784dfe8c1eb21ee441c0e971cbc43"} Dec 03 13:10:20 crc kubenswrapper[5052]: I1203 13:10:20.152740 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/failed-service-edpm-compute-no-nodes-edpm-compute-no-nodesr47df" event={"ID":"b9427361-e875-4d16-b3a6-cf4629dc9d00","Type":"ContainerStarted","Data":"8d155c44a415d1221f170deb5a3083c00cf41d47f75364b5404f3376474ddcfb"} Dec 03 13:10:20 crc kubenswrapper[5052]: I1203 13:10:20.187191 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/failed-service-edpm-compute-no-nodes-edpm-compute-no-nodesr47df" podStartSLOduration=2.8473761890000002 podStartE2EDuration="4.187165378s" podCreationTimestamp="2025-12-03 13:10:16 +0000 UTC" firstStartedPulling="2025-12-03 13:10:17.480572027 +0000 UTC m=+6608.068813791" lastFinishedPulling="2025-12-03 13:10:18.820361216 +0000 UTC m=+6609.408602980" observedRunningTime="2025-12-03 13:10:20.181166586 +0000 UTC m=+6610.769408370" watchObservedRunningTime="2025-12-03 13:10:20.187165378 +0000 UTC m=+6610.775407142" Dec 03 13:10:21 crc kubenswrapper[5052]: I1203 13:10:21.169862 5052 generic.go:334] "Generic (PLEG): container finished" podID="b9427361-e875-4d16-b3a6-cf4629dc9d00" containerID="8d155c44a415d1221f170deb5a3083c00cf41d47f75364b5404f3376474ddcfb" exitCode=2 Dec 03 13:10:21 crc kubenswrapper[5052]: I1203 13:10:21.169946 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/failed-service-edpm-compute-no-nodes-edpm-compute-no-nodesr47df" event={"ID":"b9427361-e875-4d16-b3a6-cf4629dc9d00","Type":"ContainerDied","Data":"8d155c44a415d1221f170deb5a3083c00cf41d47f75364b5404f3376474ddcfb"} Dec 03 13:10:22 crc kubenswrapper[5052]: I1203 13:10:22.488398 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/failed-service-edpm-compute-no-nodes-edpm-compute-no-nodesr47df" Dec 03 13:10:22 crc kubenswrapper[5052]: I1203 13:10:22.607327 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/b9427361-e875-4d16-b3a6-cf4629dc9d00-ssh-key\") pod \"b9427361-e875-4d16-b3a6-cf4629dc9d00\" (UID: \"b9427361-e875-4d16-b3a6-cf4629dc9d00\") " Dec 03 13:10:22 crc kubenswrapper[5052]: I1203 13:10:22.607793 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"failed-service-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b9427361-e875-4d16-b3a6-cf4629dc9d00-failed-service-combined-ca-bundle\") pod \"b9427361-e875-4d16-b3a6-cf4629dc9d00\" (UID: \"b9427361-e875-4d16-b3a6-cf4629dc9d00\") " Dec 03 13:10:22 crc kubenswrapper[5052]: I1203 13:10:22.607902 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-twcmr\" (UniqueName: \"kubernetes.io/projected/b9427361-e875-4d16-b3a6-cf4629dc9d00-kube-api-access-twcmr\") pod \"b9427361-e875-4d16-b3a6-cf4629dc9d00\" (UID: \"b9427361-e875-4d16-b3a6-cf4629dc9d00\") " Dec 03 13:10:22 crc kubenswrapper[5052]: I1203 13:10:22.607934 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/b9427361-e875-4d16-b3a6-cf4629dc9d00-inventory\") pod \"b9427361-e875-4d16-b3a6-cf4629dc9d00\" (UID: \"b9427361-e875-4d16-b3a6-cf4629dc9d00\") " Dec 03 13:10:22 crc kubenswrapper[5052]: I1203 13:10:22.614717 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b9427361-e875-4d16-b3a6-cf4629dc9d00-failed-service-combined-ca-bundle" (OuterVolumeSpecName: "failed-service-combined-ca-bundle") pod "b9427361-e875-4d16-b3a6-cf4629dc9d00" (UID: "b9427361-e875-4d16-b3a6-cf4629dc9d00"). InnerVolumeSpecName "failed-service-combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:10:22 crc kubenswrapper[5052]: I1203 13:10:22.614793 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b9427361-e875-4d16-b3a6-cf4629dc9d00-kube-api-access-twcmr" (OuterVolumeSpecName: "kube-api-access-twcmr") pod "b9427361-e875-4d16-b3a6-cf4629dc9d00" (UID: "b9427361-e875-4d16-b3a6-cf4629dc9d00"). InnerVolumeSpecName "kube-api-access-twcmr". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:10:22 crc kubenswrapper[5052]: I1203 13:10:22.633248 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b9427361-e875-4d16-b3a6-cf4629dc9d00-inventory" (OuterVolumeSpecName: "inventory") pod "b9427361-e875-4d16-b3a6-cf4629dc9d00" (UID: "b9427361-e875-4d16-b3a6-cf4629dc9d00"). InnerVolumeSpecName "inventory". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:10:22 crc kubenswrapper[5052]: I1203 13:10:22.633945 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b9427361-e875-4d16-b3a6-cf4629dc9d00-ssh-key" (OuterVolumeSpecName: "ssh-key") pod "b9427361-e875-4d16-b3a6-cf4629dc9d00" (UID: "b9427361-e875-4d16-b3a6-cf4629dc9d00"). InnerVolumeSpecName "ssh-key". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:10:22 crc kubenswrapper[5052]: I1203 13:10:22.674230 5052 scope.go:117] "RemoveContainer" containerID="dbc100c8081ea1a2f5487429cc9d67de6bc8ba11282b64f248c71275375b0e16" Dec 03 13:10:22 crc kubenswrapper[5052]: E1203 13:10:22.674674 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 13:10:22 crc kubenswrapper[5052]: I1203 13:10:22.710327 5052 reconciler_common.go:293] "Volume detached for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/b9427361-e875-4d16-b3a6-cf4629dc9d00-ssh-key\") on node \"crc\" DevicePath \"\"" Dec 03 13:10:22 crc kubenswrapper[5052]: I1203 13:10:22.710384 5052 reconciler_common.go:293] "Volume detached for volume \"failed-service-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b9427361-e875-4d16-b3a6-cf4629dc9d00-failed-service-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 13:10:22 crc kubenswrapper[5052]: I1203 13:10:22.710404 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-twcmr\" (UniqueName: \"kubernetes.io/projected/b9427361-e875-4d16-b3a6-cf4629dc9d00-kube-api-access-twcmr\") on node \"crc\" DevicePath \"\"" Dec 03 13:10:22 crc kubenswrapper[5052]: I1203 13:10:22.710432 5052 reconciler_common.go:293] "Volume detached for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/b9427361-e875-4d16-b3a6-cf4629dc9d00-inventory\") on node \"crc\" DevicePath \"\"" Dec 03 13:10:23 crc kubenswrapper[5052]: I1203 13:10:23.191422 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/failed-service-edpm-compute-no-nodes-edpm-compute-no-nodesr47df" event={"ID":"b9427361-e875-4d16-b3a6-cf4629dc9d00","Type":"ContainerDied","Data":"4ca80fe41e48c4d6a838710b3baa2c10617784dfe8c1eb21ee441c0e971cbc43"} Dec 03 13:10:23 crc kubenswrapper[5052]: I1203 13:10:23.191467 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4ca80fe41e48c4d6a838710b3baa2c10617784dfe8c1eb21ee441c0e971cbc43" Dec 03 13:10:23 crc kubenswrapper[5052]: I1203 13:10:23.192248 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/failed-service-edpm-compute-no-nodes-edpm-compute-no-nodesr47df" Dec 03 13:10:31 crc kubenswrapper[5052]: I1203 13:10:31.021446 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/failed-service-edpm-compute-no-nodes-edpm-compute-no-nodes6wb5q"] Dec 03 13:10:31 crc kubenswrapper[5052]: E1203 13:10:31.022378 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="919f8203-0bfd-4470-8311-76d24f152b46" containerName="extract-utilities" Dec 03 13:10:31 crc kubenswrapper[5052]: I1203 13:10:31.022395 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="919f8203-0bfd-4470-8311-76d24f152b46" containerName="extract-utilities" Dec 03 13:10:31 crc kubenswrapper[5052]: E1203 13:10:31.022422 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="919f8203-0bfd-4470-8311-76d24f152b46" containerName="registry-server" Dec 03 13:10:31 crc kubenswrapper[5052]: I1203 13:10:31.022431 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="919f8203-0bfd-4470-8311-76d24f152b46" containerName="registry-server" Dec 03 13:10:31 crc kubenswrapper[5052]: E1203 13:10:31.022463 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="919f8203-0bfd-4470-8311-76d24f152b46" containerName="extract-content" Dec 03 13:10:31 crc kubenswrapper[5052]: I1203 13:10:31.022474 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="919f8203-0bfd-4470-8311-76d24f152b46" containerName="extract-content" Dec 03 13:10:31 crc kubenswrapper[5052]: E1203 13:10:31.022485 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="b9427361-e875-4d16-b3a6-cf4629dc9d00" containerName="failed-service-edpm-compute-no-nodes-edpm-compute-no-nodes" Dec 03 13:10:31 crc kubenswrapper[5052]: I1203 13:10:31.022494 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="b9427361-e875-4d16-b3a6-cf4629dc9d00" containerName="failed-service-edpm-compute-no-nodes-edpm-compute-no-nodes" Dec 03 13:10:31 crc kubenswrapper[5052]: I1203 13:10:31.022699 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="b9427361-e875-4d16-b3a6-cf4629dc9d00" containerName="failed-service-edpm-compute-no-nodes-edpm-compute-no-nodes" Dec 03 13:10:31 crc kubenswrapper[5052]: I1203 13:10:31.022729 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="919f8203-0bfd-4470-8311-76d24f152b46" containerName="registry-server" Dec 03 13:10:31 crc kubenswrapper[5052]: I1203 13:10:31.023406 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/failed-service-edpm-compute-no-nodes-edpm-compute-no-nodes6wb5q" Dec 03 13:10:31 crc kubenswrapper[5052]: I1203 13:10:31.025786 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"dataplanenodeset-edpm-compute-no-nodes" Dec 03 13:10:31 crc kubenswrapper[5052]: I1203 13:10:31.026248 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"dataplane-ansible-ssh-private-key-secret" Dec 03 13:10:31 crc kubenswrapper[5052]: I1203 13:10:31.026271 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"openstack-aee-default-env" Dec 03 13:10:31 crc kubenswrapper[5052]: I1203 13:10:31.026258 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"edpm-compute-no-nodes-dockercfg-7spm4" Dec 03 13:10:31 crc kubenswrapper[5052]: I1203 13:10:31.026295 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"combined-ca-bundle" Dec 03 13:10:31 crc kubenswrapper[5052]: I1203 13:10:31.035135 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"failed-service-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e3636dea-6b99-4621-a2c1-edb24bcb0990-failed-service-combined-ca-bundle\") pod \"failed-service-edpm-compute-no-nodes-edpm-compute-no-nodes6wb5q\" (UID: \"e3636dea-6b99-4621-a2c1-edb24bcb0990\") " pod="openstack-kuttl-tests/failed-service-edpm-compute-no-nodes-edpm-compute-no-nodes6wb5q" Dec 03 13:10:31 crc kubenswrapper[5052]: I1203 13:10:31.035234 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/e3636dea-6b99-4621-a2c1-edb24bcb0990-ssh-key\") pod \"failed-service-edpm-compute-no-nodes-edpm-compute-no-nodes6wb5q\" (UID: \"e3636dea-6b99-4621-a2c1-edb24bcb0990\") " pod="openstack-kuttl-tests/failed-service-edpm-compute-no-nodes-edpm-compute-no-nodes6wb5q" Dec 03 13:10:31 crc kubenswrapper[5052]: I1203 13:10:31.035266 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xj5g7\" (UniqueName: \"kubernetes.io/projected/e3636dea-6b99-4621-a2c1-edb24bcb0990-kube-api-access-xj5g7\") pod \"failed-service-edpm-compute-no-nodes-edpm-compute-no-nodes6wb5q\" (UID: \"e3636dea-6b99-4621-a2c1-edb24bcb0990\") " pod="openstack-kuttl-tests/failed-service-edpm-compute-no-nodes-edpm-compute-no-nodes6wb5q" Dec 03 13:10:31 crc kubenswrapper[5052]: I1203 13:10:31.035300 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/e3636dea-6b99-4621-a2c1-edb24bcb0990-inventory\") pod \"failed-service-edpm-compute-no-nodes-edpm-compute-no-nodes6wb5q\" (UID: \"e3636dea-6b99-4621-a2c1-edb24bcb0990\") " pod="openstack-kuttl-tests/failed-service-edpm-compute-no-nodes-edpm-compute-no-nodes6wb5q" Dec 03 13:10:31 crc kubenswrapper[5052]: I1203 13:10:31.035482 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/failed-service-edpm-compute-no-nodes-edpm-compute-no-nodes6wb5q"] Dec 03 13:10:31 crc kubenswrapper[5052]: I1203 13:10:31.136721 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"failed-service-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e3636dea-6b99-4621-a2c1-edb24bcb0990-failed-service-combined-ca-bundle\") pod \"failed-service-edpm-compute-no-nodes-edpm-compute-no-nodes6wb5q\" (UID: \"e3636dea-6b99-4621-a2c1-edb24bcb0990\") " pod="openstack-kuttl-tests/failed-service-edpm-compute-no-nodes-edpm-compute-no-nodes6wb5q" Dec 03 13:10:31 crc kubenswrapper[5052]: I1203 13:10:31.136921 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/e3636dea-6b99-4621-a2c1-edb24bcb0990-ssh-key\") pod \"failed-service-edpm-compute-no-nodes-edpm-compute-no-nodes6wb5q\" (UID: \"e3636dea-6b99-4621-a2c1-edb24bcb0990\") " pod="openstack-kuttl-tests/failed-service-edpm-compute-no-nodes-edpm-compute-no-nodes6wb5q" Dec 03 13:10:31 crc kubenswrapper[5052]: I1203 13:10:31.136984 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-xj5g7\" (UniqueName: \"kubernetes.io/projected/e3636dea-6b99-4621-a2c1-edb24bcb0990-kube-api-access-xj5g7\") pod \"failed-service-edpm-compute-no-nodes-edpm-compute-no-nodes6wb5q\" (UID: \"e3636dea-6b99-4621-a2c1-edb24bcb0990\") " pod="openstack-kuttl-tests/failed-service-edpm-compute-no-nodes-edpm-compute-no-nodes6wb5q" Dec 03 13:10:31 crc kubenswrapper[5052]: I1203 13:10:31.137027 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/e3636dea-6b99-4621-a2c1-edb24bcb0990-inventory\") pod \"failed-service-edpm-compute-no-nodes-edpm-compute-no-nodes6wb5q\" (UID: \"e3636dea-6b99-4621-a2c1-edb24bcb0990\") " pod="openstack-kuttl-tests/failed-service-edpm-compute-no-nodes-edpm-compute-no-nodes6wb5q" Dec 03 13:10:31 crc kubenswrapper[5052]: I1203 13:10:31.143723 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/e3636dea-6b99-4621-a2c1-edb24bcb0990-ssh-key\") pod \"failed-service-edpm-compute-no-nodes-edpm-compute-no-nodes6wb5q\" (UID: \"e3636dea-6b99-4621-a2c1-edb24bcb0990\") " pod="openstack-kuttl-tests/failed-service-edpm-compute-no-nodes-edpm-compute-no-nodes6wb5q" Dec 03 13:10:31 crc kubenswrapper[5052]: I1203 13:10:31.143724 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/e3636dea-6b99-4621-a2c1-edb24bcb0990-inventory\") pod \"failed-service-edpm-compute-no-nodes-edpm-compute-no-nodes6wb5q\" (UID: \"e3636dea-6b99-4621-a2c1-edb24bcb0990\") " pod="openstack-kuttl-tests/failed-service-edpm-compute-no-nodes-edpm-compute-no-nodes6wb5q" Dec 03 13:10:31 crc kubenswrapper[5052]: I1203 13:10:31.155250 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"failed-service-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e3636dea-6b99-4621-a2c1-edb24bcb0990-failed-service-combined-ca-bundle\") pod \"failed-service-edpm-compute-no-nodes-edpm-compute-no-nodes6wb5q\" (UID: \"e3636dea-6b99-4621-a2c1-edb24bcb0990\") " pod="openstack-kuttl-tests/failed-service-edpm-compute-no-nodes-edpm-compute-no-nodes6wb5q" Dec 03 13:10:31 crc kubenswrapper[5052]: I1203 13:10:31.158169 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-xj5g7\" (UniqueName: \"kubernetes.io/projected/e3636dea-6b99-4621-a2c1-edb24bcb0990-kube-api-access-xj5g7\") pod \"failed-service-edpm-compute-no-nodes-edpm-compute-no-nodes6wb5q\" (UID: \"e3636dea-6b99-4621-a2c1-edb24bcb0990\") " pod="openstack-kuttl-tests/failed-service-edpm-compute-no-nodes-edpm-compute-no-nodes6wb5q" Dec 03 13:10:31 crc kubenswrapper[5052]: I1203 13:10:31.399219 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/failed-service-edpm-compute-no-nodes-edpm-compute-no-nodes6wb5q" Dec 03 13:10:31 crc kubenswrapper[5052]: I1203 13:10:31.865257 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/failed-service-edpm-compute-no-nodes-edpm-compute-no-nodes6wb5q"] Dec 03 13:10:32 crc kubenswrapper[5052]: I1203 13:10:32.276410 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/failed-service-edpm-compute-no-nodes-edpm-compute-no-nodes6wb5q" event={"ID":"e3636dea-6b99-4621-a2c1-edb24bcb0990","Type":"ContainerStarted","Data":"690032f7ee86c00b8cbe2bb978babba1d3fddb8c2d5d8fbe66dc6077c66aa124"} Dec 03 13:10:33 crc kubenswrapper[5052]: I1203 13:10:33.286489 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/failed-service-edpm-compute-no-nodes-edpm-compute-no-nodes6wb5q" event={"ID":"e3636dea-6b99-4621-a2c1-edb24bcb0990","Type":"ContainerStarted","Data":"40acab817908dc89ffbb3d9a51b07484f7a5fbb642a88c1eb9f7403ecd646051"} Dec 03 13:10:33 crc kubenswrapper[5052]: I1203 13:10:33.312682 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/failed-service-edpm-compute-no-nodes-edpm-compute-no-nodes6wb5q" podStartSLOduration=1.660959319 podStartE2EDuration="2.312659352s" podCreationTimestamp="2025-12-03 13:10:31 +0000 UTC" firstStartedPulling="2025-12-03 13:10:31.872964312 +0000 UTC m=+6622.461206076" lastFinishedPulling="2025-12-03 13:10:32.524664345 +0000 UTC m=+6623.112906109" observedRunningTime="2025-12-03 13:10:33.302997755 +0000 UTC m=+6623.891239519" watchObservedRunningTime="2025-12-03 13:10:33.312659352 +0000 UTC m=+6623.900901116" Dec 03 13:10:33 crc kubenswrapper[5052]: I1203 13:10:33.674100 5052 scope.go:117] "RemoveContainer" containerID="dbc100c8081ea1a2f5487429cc9d67de6bc8ba11282b64f248c71275375b0e16" Dec 03 13:10:33 crc kubenswrapper[5052]: E1203 13:10:33.674396 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 13:10:35 crc kubenswrapper[5052]: I1203 13:10:35.304255 5052 generic.go:334] "Generic (PLEG): container finished" podID="e3636dea-6b99-4621-a2c1-edb24bcb0990" containerID="40acab817908dc89ffbb3d9a51b07484f7a5fbb642a88c1eb9f7403ecd646051" exitCode=2 Dec 03 13:10:35 crc kubenswrapper[5052]: I1203 13:10:35.304351 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/failed-service-edpm-compute-no-nodes-edpm-compute-no-nodes6wb5q" event={"ID":"e3636dea-6b99-4621-a2c1-edb24bcb0990","Type":"ContainerDied","Data":"40acab817908dc89ffbb3d9a51b07484f7a5fbb642a88c1eb9f7403ecd646051"} Dec 03 13:10:36 crc kubenswrapper[5052]: I1203 13:10:36.585756 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/failed-service-edpm-compute-no-nodes-edpm-compute-no-nodes6wb5q" Dec 03 13:10:36 crc kubenswrapper[5052]: I1203 13:10:36.724586 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"failed-service-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e3636dea-6b99-4621-a2c1-edb24bcb0990-failed-service-combined-ca-bundle\") pod \"e3636dea-6b99-4621-a2c1-edb24bcb0990\" (UID: \"e3636dea-6b99-4621-a2c1-edb24bcb0990\") " Dec 03 13:10:36 crc kubenswrapper[5052]: I1203 13:10:36.724715 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xj5g7\" (UniqueName: \"kubernetes.io/projected/e3636dea-6b99-4621-a2c1-edb24bcb0990-kube-api-access-xj5g7\") pod \"e3636dea-6b99-4621-a2c1-edb24bcb0990\" (UID: \"e3636dea-6b99-4621-a2c1-edb24bcb0990\") " Dec 03 13:10:36 crc kubenswrapper[5052]: I1203 13:10:36.724956 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/e3636dea-6b99-4621-a2c1-edb24bcb0990-ssh-key\") pod \"e3636dea-6b99-4621-a2c1-edb24bcb0990\" (UID: \"e3636dea-6b99-4621-a2c1-edb24bcb0990\") " Dec 03 13:10:36 crc kubenswrapper[5052]: I1203 13:10:36.724996 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/e3636dea-6b99-4621-a2c1-edb24bcb0990-inventory\") pod \"e3636dea-6b99-4621-a2c1-edb24bcb0990\" (UID: \"e3636dea-6b99-4621-a2c1-edb24bcb0990\") " Dec 03 13:10:36 crc kubenswrapper[5052]: I1203 13:10:36.731114 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e3636dea-6b99-4621-a2c1-edb24bcb0990-failed-service-combined-ca-bundle" (OuterVolumeSpecName: "failed-service-combined-ca-bundle") pod "e3636dea-6b99-4621-a2c1-edb24bcb0990" (UID: "e3636dea-6b99-4621-a2c1-edb24bcb0990"). InnerVolumeSpecName "failed-service-combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:10:36 crc kubenswrapper[5052]: I1203 13:10:36.731552 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e3636dea-6b99-4621-a2c1-edb24bcb0990-kube-api-access-xj5g7" (OuterVolumeSpecName: "kube-api-access-xj5g7") pod "e3636dea-6b99-4621-a2c1-edb24bcb0990" (UID: "e3636dea-6b99-4621-a2c1-edb24bcb0990"). InnerVolumeSpecName "kube-api-access-xj5g7". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:10:36 crc kubenswrapper[5052]: I1203 13:10:36.753770 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e3636dea-6b99-4621-a2c1-edb24bcb0990-ssh-key" (OuterVolumeSpecName: "ssh-key") pod "e3636dea-6b99-4621-a2c1-edb24bcb0990" (UID: "e3636dea-6b99-4621-a2c1-edb24bcb0990"). InnerVolumeSpecName "ssh-key". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:10:36 crc kubenswrapper[5052]: I1203 13:10:36.753996 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e3636dea-6b99-4621-a2c1-edb24bcb0990-inventory" (OuterVolumeSpecName: "inventory") pod "e3636dea-6b99-4621-a2c1-edb24bcb0990" (UID: "e3636dea-6b99-4621-a2c1-edb24bcb0990"). InnerVolumeSpecName "inventory". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:10:36 crc kubenswrapper[5052]: I1203 13:10:36.826276 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-xj5g7\" (UniqueName: \"kubernetes.io/projected/e3636dea-6b99-4621-a2c1-edb24bcb0990-kube-api-access-xj5g7\") on node \"crc\" DevicePath \"\"" Dec 03 13:10:36 crc kubenswrapper[5052]: I1203 13:10:36.826322 5052 reconciler_common.go:293] "Volume detached for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/e3636dea-6b99-4621-a2c1-edb24bcb0990-ssh-key\") on node \"crc\" DevicePath \"\"" Dec 03 13:10:36 crc kubenswrapper[5052]: I1203 13:10:36.826333 5052 reconciler_common.go:293] "Volume detached for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/e3636dea-6b99-4621-a2c1-edb24bcb0990-inventory\") on node \"crc\" DevicePath \"\"" Dec 03 13:10:36 crc kubenswrapper[5052]: I1203 13:10:36.826345 5052 reconciler_common.go:293] "Volume detached for volume \"failed-service-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e3636dea-6b99-4621-a2c1-edb24bcb0990-failed-service-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 13:10:37 crc kubenswrapper[5052]: I1203 13:10:37.324451 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/failed-service-edpm-compute-no-nodes-edpm-compute-no-nodes6wb5q" event={"ID":"e3636dea-6b99-4621-a2c1-edb24bcb0990","Type":"ContainerDied","Data":"690032f7ee86c00b8cbe2bb978babba1d3fddb8c2d5d8fbe66dc6077c66aa124"} Dec 03 13:10:37 crc kubenswrapper[5052]: I1203 13:10:37.324527 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="690032f7ee86c00b8cbe2bb978babba1d3fddb8c2d5d8fbe66dc6077c66aa124" Dec 03 13:10:37 crc kubenswrapper[5052]: I1203 13:10:37.324576 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/failed-service-edpm-compute-no-nodes-edpm-compute-no-nodes6wb5q" Dec 03 13:10:43 crc kubenswrapper[5052]: I1203 13:10:43.703994 5052 scope.go:117] "RemoveContainer" containerID="e38a3ae7079d380dc875cb9151781bdf35b9d38f31c9f182c28fb0c90661bbe5" Dec 03 13:10:43 crc kubenswrapper[5052]: I1203 13:10:43.737349 5052 scope.go:117] "RemoveContainer" containerID="f1f0febd18f3eaa033195f2f7e941a98439672c1a789fc20f5e73209a2dc90d8" Dec 03 13:10:43 crc kubenswrapper[5052]: I1203 13:10:43.773740 5052 scope.go:117] "RemoveContainer" containerID="29cabd7861943ebda29291906beab8165e925c6aaf78701fe45479cfc3e5387e" Dec 03 13:10:43 crc kubenswrapper[5052]: I1203 13:10:43.809850 5052 scope.go:117] "RemoveContainer" containerID="b3ba42f32af8ad8ea170cf0a0523a777860aef8c269e25c986888ef8bf7698c4" Dec 03 13:10:43 crc kubenswrapper[5052]: I1203 13:10:43.842900 5052 scope.go:117] "RemoveContainer" containerID="27e39a5e00dde973d1e50ee5f6e7b7601cbc03765f47c2836b5134dc9e48ab01" Dec 03 13:10:43 crc kubenswrapper[5052]: I1203 13:10:43.876271 5052 scope.go:117] "RemoveContainer" containerID="c13e3f2c1e7922f82cf6b655ebfa96b329f60e10c37d64b31a8eb3d4b17c0e34" Dec 03 13:10:43 crc kubenswrapper[5052]: I1203 13:10:43.914275 5052 scope.go:117] "RemoveContainer" containerID="3f496c994d26bdb760c4659da1eaac2a9c50c8691b6b7b40bfd130a5a5621039" Dec 03 13:10:43 crc kubenswrapper[5052]: I1203 13:10:43.944800 5052 scope.go:117] "RemoveContainer" containerID="e15c56c4973e2554de21d4b43c45463964918fa85401b7c3ae40228c1566f441" Dec 03 13:10:43 crc kubenswrapper[5052]: I1203 13:10:43.971221 5052 scope.go:117] "RemoveContainer" containerID="feb5b0cb00c367705dfda4372f7ee49d4b7b78b30ec4edcee55d1df737e21b21" Dec 03 13:10:44 crc kubenswrapper[5052]: I1203 13:10:44.001627 5052 scope.go:117] "RemoveContainer" containerID="e5342dec4985872b7d7ce685e6185d0580bf743b2ef488b1180b66491c91c7cf" Dec 03 13:10:44 crc kubenswrapper[5052]: I1203 13:10:44.029795 5052 scope.go:117] "RemoveContainer" containerID="2a430f54ad25c239aaffcfc2cd168c7a1018a6d91ff915b9917a846608c9c8ae" Dec 03 13:10:44 crc kubenswrapper[5052]: I1203 13:10:44.066775 5052 scope.go:117] "RemoveContainer" containerID="e2645a93f6a091a31daccb5c7df437f199acb3979d9c509e0996643c8eed095b" Dec 03 13:10:44 crc kubenswrapper[5052]: I1203 13:10:44.674383 5052 scope.go:117] "RemoveContainer" containerID="dbc100c8081ea1a2f5487429cc9d67de6bc8ba11282b64f248c71275375b0e16" Dec 03 13:10:44 crc kubenswrapper[5052]: E1203 13:10:44.674670 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 13:10:54 crc kubenswrapper[5052]: I1203 13:10:54.036969 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/failed-service-edpm-compute-no-nodes-edpm-compute-no-nodesz9xr7"] Dec 03 13:10:54 crc kubenswrapper[5052]: E1203 13:10:54.039761 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="e3636dea-6b99-4621-a2c1-edb24bcb0990" containerName="failed-service-edpm-compute-no-nodes-edpm-compute-no-nodes" Dec 03 13:10:54 crc kubenswrapper[5052]: I1203 13:10:54.039797 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="e3636dea-6b99-4621-a2c1-edb24bcb0990" containerName="failed-service-edpm-compute-no-nodes-edpm-compute-no-nodes" Dec 03 13:10:54 crc kubenswrapper[5052]: I1203 13:10:54.040034 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="e3636dea-6b99-4621-a2c1-edb24bcb0990" containerName="failed-service-edpm-compute-no-nodes-edpm-compute-no-nodes" Dec 03 13:10:54 crc kubenswrapper[5052]: I1203 13:10:54.040831 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/failed-service-edpm-compute-no-nodes-edpm-compute-no-nodesz9xr7" Dec 03 13:10:54 crc kubenswrapper[5052]: I1203 13:10:54.045062 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"combined-ca-bundle" Dec 03 13:10:54 crc kubenswrapper[5052]: I1203 13:10:54.046362 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"dataplane-ansible-ssh-private-key-secret" Dec 03 13:10:54 crc kubenswrapper[5052]: I1203 13:10:54.046474 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"openstack-aee-default-env" Dec 03 13:10:54 crc kubenswrapper[5052]: I1203 13:10:54.046719 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"dataplanenodeset-edpm-compute-no-nodes" Dec 03 13:10:54 crc kubenswrapper[5052]: I1203 13:10:54.047708 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"edpm-compute-no-nodes-dockercfg-7spm4" Dec 03 13:10:54 crc kubenswrapper[5052]: I1203 13:10:54.050045 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/failed-service-edpm-compute-no-nodes-edpm-compute-no-nodesz9xr7"] Dec 03 13:10:54 crc kubenswrapper[5052]: I1203 13:10:54.192770 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q48gw\" (UniqueName: \"kubernetes.io/projected/bde7c5d5-7a99-43aa-92eb-357d00206cd7-kube-api-access-q48gw\") pod \"failed-service-edpm-compute-no-nodes-edpm-compute-no-nodesz9xr7\" (UID: \"bde7c5d5-7a99-43aa-92eb-357d00206cd7\") " pod="openstack-kuttl-tests/failed-service-edpm-compute-no-nodes-edpm-compute-no-nodesz9xr7" Dec 03 13:10:54 crc kubenswrapper[5052]: I1203 13:10:54.192843 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"failed-service-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/bde7c5d5-7a99-43aa-92eb-357d00206cd7-failed-service-combined-ca-bundle\") pod \"failed-service-edpm-compute-no-nodes-edpm-compute-no-nodesz9xr7\" (UID: \"bde7c5d5-7a99-43aa-92eb-357d00206cd7\") " pod="openstack-kuttl-tests/failed-service-edpm-compute-no-nodes-edpm-compute-no-nodesz9xr7" Dec 03 13:10:54 crc kubenswrapper[5052]: I1203 13:10:54.192973 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/bde7c5d5-7a99-43aa-92eb-357d00206cd7-ssh-key\") pod \"failed-service-edpm-compute-no-nodes-edpm-compute-no-nodesz9xr7\" (UID: \"bde7c5d5-7a99-43aa-92eb-357d00206cd7\") " pod="openstack-kuttl-tests/failed-service-edpm-compute-no-nodes-edpm-compute-no-nodesz9xr7" Dec 03 13:10:54 crc kubenswrapper[5052]: I1203 13:10:54.193003 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/bde7c5d5-7a99-43aa-92eb-357d00206cd7-inventory\") pod \"failed-service-edpm-compute-no-nodes-edpm-compute-no-nodesz9xr7\" (UID: \"bde7c5d5-7a99-43aa-92eb-357d00206cd7\") " pod="openstack-kuttl-tests/failed-service-edpm-compute-no-nodes-edpm-compute-no-nodesz9xr7" Dec 03 13:10:54 crc kubenswrapper[5052]: I1203 13:10:54.294904 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/bde7c5d5-7a99-43aa-92eb-357d00206cd7-ssh-key\") pod \"failed-service-edpm-compute-no-nodes-edpm-compute-no-nodesz9xr7\" (UID: \"bde7c5d5-7a99-43aa-92eb-357d00206cd7\") " pod="openstack-kuttl-tests/failed-service-edpm-compute-no-nodes-edpm-compute-no-nodesz9xr7" Dec 03 13:10:54 crc kubenswrapper[5052]: I1203 13:10:54.294976 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/bde7c5d5-7a99-43aa-92eb-357d00206cd7-inventory\") pod \"failed-service-edpm-compute-no-nodes-edpm-compute-no-nodesz9xr7\" (UID: \"bde7c5d5-7a99-43aa-92eb-357d00206cd7\") " pod="openstack-kuttl-tests/failed-service-edpm-compute-no-nodes-edpm-compute-no-nodesz9xr7" Dec 03 13:10:54 crc kubenswrapper[5052]: I1203 13:10:54.295084 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-q48gw\" (UniqueName: \"kubernetes.io/projected/bde7c5d5-7a99-43aa-92eb-357d00206cd7-kube-api-access-q48gw\") pod \"failed-service-edpm-compute-no-nodes-edpm-compute-no-nodesz9xr7\" (UID: \"bde7c5d5-7a99-43aa-92eb-357d00206cd7\") " pod="openstack-kuttl-tests/failed-service-edpm-compute-no-nodes-edpm-compute-no-nodesz9xr7" Dec 03 13:10:54 crc kubenswrapper[5052]: I1203 13:10:54.295125 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"failed-service-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/bde7c5d5-7a99-43aa-92eb-357d00206cd7-failed-service-combined-ca-bundle\") pod \"failed-service-edpm-compute-no-nodes-edpm-compute-no-nodesz9xr7\" (UID: \"bde7c5d5-7a99-43aa-92eb-357d00206cd7\") " pod="openstack-kuttl-tests/failed-service-edpm-compute-no-nodes-edpm-compute-no-nodesz9xr7" Dec 03 13:10:54 crc kubenswrapper[5052]: I1203 13:10:54.300813 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/bde7c5d5-7a99-43aa-92eb-357d00206cd7-inventory\") pod \"failed-service-edpm-compute-no-nodes-edpm-compute-no-nodesz9xr7\" (UID: \"bde7c5d5-7a99-43aa-92eb-357d00206cd7\") " pod="openstack-kuttl-tests/failed-service-edpm-compute-no-nodes-edpm-compute-no-nodesz9xr7" Dec 03 13:10:54 crc kubenswrapper[5052]: I1203 13:10:54.301191 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"failed-service-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/bde7c5d5-7a99-43aa-92eb-357d00206cd7-failed-service-combined-ca-bundle\") pod \"failed-service-edpm-compute-no-nodes-edpm-compute-no-nodesz9xr7\" (UID: \"bde7c5d5-7a99-43aa-92eb-357d00206cd7\") " pod="openstack-kuttl-tests/failed-service-edpm-compute-no-nodes-edpm-compute-no-nodesz9xr7" Dec 03 13:10:54 crc kubenswrapper[5052]: I1203 13:10:54.302321 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/bde7c5d5-7a99-43aa-92eb-357d00206cd7-ssh-key\") pod \"failed-service-edpm-compute-no-nodes-edpm-compute-no-nodesz9xr7\" (UID: \"bde7c5d5-7a99-43aa-92eb-357d00206cd7\") " pod="openstack-kuttl-tests/failed-service-edpm-compute-no-nodes-edpm-compute-no-nodesz9xr7" Dec 03 13:10:54 crc kubenswrapper[5052]: I1203 13:10:54.314054 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-q48gw\" (UniqueName: \"kubernetes.io/projected/bde7c5d5-7a99-43aa-92eb-357d00206cd7-kube-api-access-q48gw\") pod \"failed-service-edpm-compute-no-nodes-edpm-compute-no-nodesz9xr7\" (UID: \"bde7c5d5-7a99-43aa-92eb-357d00206cd7\") " pod="openstack-kuttl-tests/failed-service-edpm-compute-no-nodes-edpm-compute-no-nodesz9xr7" Dec 03 13:10:54 crc kubenswrapper[5052]: I1203 13:10:54.380105 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/failed-service-edpm-compute-no-nodes-edpm-compute-no-nodesz9xr7" Dec 03 13:10:54 crc kubenswrapper[5052]: I1203 13:10:54.852325 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/failed-service-edpm-compute-no-nodes-edpm-compute-no-nodesz9xr7"] Dec 03 13:10:55 crc kubenswrapper[5052]: I1203 13:10:55.488011 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/failed-service-edpm-compute-no-nodes-edpm-compute-no-nodesz9xr7" event={"ID":"bde7c5d5-7a99-43aa-92eb-357d00206cd7","Type":"ContainerStarted","Data":"21675fefbfdb5ab881e4176082c139cecfc37deea31f7f15f9eb543ed17c6099"} Dec 03 13:10:56 crc kubenswrapper[5052]: I1203 13:10:56.500308 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/failed-service-edpm-compute-no-nodes-edpm-compute-no-nodesz9xr7" event={"ID":"bde7c5d5-7a99-43aa-92eb-357d00206cd7","Type":"ContainerStarted","Data":"3f9790aa10f8b223320a934c6473ecbc82c9128a418a4c84445a169a85afddab"} Dec 03 13:10:56 crc kubenswrapper[5052]: I1203 13:10:56.522484 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/failed-service-edpm-compute-no-nodes-edpm-compute-no-nodesz9xr7" podStartSLOduration=1.791036732 podStartE2EDuration="2.522462848s" podCreationTimestamp="2025-12-03 13:10:54 +0000 UTC" firstStartedPulling="2025-12-03 13:10:54.859816594 +0000 UTC m=+6645.448058358" lastFinishedPulling="2025-12-03 13:10:55.59124271 +0000 UTC m=+6646.179484474" observedRunningTime="2025-12-03 13:10:56.519452781 +0000 UTC m=+6647.107694545" watchObservedRunningTime="2025-12-03 13:10:56.522462848 +0000 UTC m=+6647.110704612" Dec 03 13:10:58 crc kubenswrapper[5052]: I1203 13:10:58.523642 5052 generic.go:334] "Generic (PLEG): container finished" podID="bde7c5d5-7a99-43aa-92eb-357d00206cd7" containerID="3f9790aa10f8b223320a934c6473ecbc82c9128a418a4c84445a169a85afddab" exitCode=2 Dec 03 13:10:58 crc kubenswrapper[5052]: I1203 13:10:58.523713 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/failed-service-edpm-compute-no-nodes-edpm-compute-no-nodesz9xr7" event={"ID":"bde7c5d5-7a99-43aa-92eb-357d00206cd7","Type":"ContainerDied","Data":"3f9790aa10f8b223320a934c6473ecbc82c9128a418a4c84445a169a85afddab"} Dec 03 13:10:59 crc kubenswrapper[5052]: I1203 13:10:59.681484 5052 scope.go:117] "RemoveContainer" containerID="dbc100c8081ea1a2f5487429cc9d67de6bc8ba11282b64f248c71275375b0e16" Dec 03 13:10:59 crc kubenswrapper[5052]: E1203 13:10:59.682077 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 13:10:59 crc kubenswrapper[5052]: I1203 13:10:59.839991 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/failed-service-edpm-compute-no-nodes-edpm-compute-no-nodesz9xr7" Dec 03 13:10:59 crc kubenswrapper[5052]: I1203 13:10:59.982076 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/bde7c5d5-7a99-43aa-92eb-357d00206cd7-ssh-key\") pod \"bde7c5d5-7a99-43aa-92eb-357d00206cd7\" (UID: \"bde7c5d5-7a99-43aa-92eb-357d00206cd7\") " Dec 03 13:10:59 crc kubenswrapper[5052]: I1203 13:10:59.982198 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/bde7c5d5-7a99-43aa-92eb-357d00206cd7-inventory\") pod \"bde7c5d5-7a99-43aa-92eb-357d00206cd7\" (UID: \"bde7c5d5-7a99-43aa-92eb-357d00206cd7\") " Dec 03 13:10:59 crc kubenswrapper[5052]: I1203 13:10:59.982295 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"failed-service-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/bde7c5d5-7a99-43aa-92eb-357d00206cd7-failed-service-combined-ca-bundle\") pod \"bde7c5d5-7a99-43aa-92eb-357d00206cd7\" (UID: \"bde7c5d5-7a99-43aa-92eb-357d00206cd7\") " Dec 03 13:10:59 crc kubenswrapper[5052]: I1203 13:10:59.982375 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q48gw\" (UniqueName: \"kubernetes.io/projected/bde7c5d5-7a99-43aa-92eb-357d00206cd7-kube-api-access-q48gw\") pod \"bde7c5d5-7a99-43aa-92eb-357d00206cd7\" (UID: \"bde7c5d5-7a99-43aa-92eb-357d00206cd7\") " Dec 03 13:10:59 crc kubenswrapper[5052]: I1203 13:10:59.988754 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bde7c5d5-7a99-43aa-92eb-357d00206cd7-failed-service-combined-ca-bundle" (OuterVolumeSpecName: "failed-service-combined-ca-bundle") pod "bde7c5d5-7a99-43aa-92eb-357d00206cd7" (UID: "bde7c5d5-7a99-43aa-92eb-357d00206cd7"). InnerVolumeSpecName "failed-service-combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:10:59 crc kubenswrapper[5052]: I1203 13:10:59.988862 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bde7c5d5-7a99-43aa-92eb-357d00206cd7-kube-api-access-q48gw" (OuterVolumeSpecName: "kube-api-access-q48gw") pod "bde7c5d5-7a99-43aa-92eb-357d00206cd7" (UID: "bde7c5d5-7a99-43aa-92eb-357d00206cd7"). InnerVolumeSpecName "kube-api-access-q48gw". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:11:00 crc kubenswrapper[5052]: I1203 13:11:00.003382 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bde7c5d5-7a99-43aa-92eb-357d00206cd7-inventory" (OuterVolumeSpecName: "inventory") pod "bde7c5d5-7a99-43aa-92eb-357d00206cd7" (UID: "bde7c5d5-7a99-43aa-92eb-357d00206cd7"). InnerVolumeSpecName "inventory". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:11:00 crc kubenswrapper[5052]: I1203 13:11:00.004893 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bde7c5d5-7a99-43aa-92eb-357d00206cd7-ssh-key" (OuterVolumeSpecName: "ssh-key") pod "bde7c5d5-7a99-43aa-92eb-357d00206cd7" (UID: "bde7c5d5-7a99-43aa-92eb-357d00206cd7"). InnerVolumeSpecName "ssh-key". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:11:00 crc kubenswrapper[5052]: I1203 13:11:00.084369 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-q48gw\" (UniqueName: \"kubernetes.io/projected/bde7c5d5-7a99-43aa-92eb-357d00206cd7-kube-api-access-q48gw\") on node \"crc\" DevicePath \"\"" Dec 03 13:11:00 crc kubenswrapper[5052]: I1203 13:11:00.084415 5052 reconciler_common.go:293] "Volume detached for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/bde7c5d5-7a99-43aa-92eb-357d00206cd7-ssh-key\") on node \"crc\" DevicePath \"\"" Dec 03 13:11:00 crc kubenswrapper[5052]: I1203 13:11:00.084431 5052 reconciler_common.go:293] "Volume detached for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/bde7c5d5-7a99-43aa-92eb-357d00206cd7-inventory\") on node \"crc\" DevicePath \"\"" Dec 03 13:11:00 crc kubenswrapper[5052]: I1203 13:11:00.084444 5052 reconciler_common.go:293] "Volume detached for volume \"failed-service-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/bde7c5d5-7a99-43aa-92eb-357d00206cd7-failed-service-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 13:11:00 crc kubenswrapper[5052]: I1203 13:11:00.547634 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/failed-service-edpm-compute-no-nodes-edpm-compute-no-nodesz9xr7" event={"ID":"bde7c5d5-7a99-43aa-92eb-357d00206cd7","Type":"ContainerDied","Data":"21675fefbfdb5ab881e4176082c139cecfc37deea31f7f15f9eb543ed17c6099"} Dec 03 13:11:00 crc kubenswrapper[5052]: I1203 13:11:00.547678 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="21675fefbfdb5ab881e4176082c139cecfc37deea31f7f15f9eb543ed17c6099" Dec 03 13:11:00 crc kubenswrapper[5052]: I1203 13:11:00.547731 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/failed-service-edpm-compute-no-nodes-edpm-compute-no-nodesz9xr7" Dec 03 13:11:12 crc kubenswrapper[5052]: I1203 13:11:12.673483 5052 scope.go:117] "RemoveContainer" containerID="dbc100c8081ea1a2f5487429cc9d67de6bc8ba11282b64f248c71275375b0e16" Dec 03 13:11:12 crc kubenswrapper[5052]: E1203 13:11:12.674315 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 13:11:26 crc kubenswrapper[5052]: I1203 13:11:26.673297 5052 scope.go:117] "RemoveContainer" containerID="dbc100c8081ea1a2f5487429cc9d67de6bc8ba11282b64f248c71275375b0e16" Dec 03 13:11:26 crc kubenswrapper[5052]: E1203 13:11:26.674104 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 13:11:37 crc kubenswrapper[5052]: I1203 13:11:37.028812 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/failed-service-edpm-compute-no-nodes-edpm-compute-no-nodes5hk6k"] Dec 03 13:11:37 crc kubenswrapper[5052]: E1203 13:11:37.029902 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="bde7c5d5-7a99-43aa-92eb-357d00206cd7" containerName="failed-service-edpm-compute-no-nodes-edpm-compute-no-nodes" Dec 03 13:11:37 crc kubenswrapper[5052]: I1203 13:11:37.029933 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="bde7c5d5-7a99-43aa-92eb-357d00206cd7" containerName="failed-service-edpm-compute-no-nodes-edpm-compute-no-nodes" Dec 03 13:11:37 crc kubenswrapper[5052]: I1203 13:11:37.030133 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="bde7c5d5-7a99-43aa-92eb-357d00206cd7" containerName="failed-service-edpm-compute-no-nodes-edpm-compute-no-nodes" Dec 03 13:11:37 crc kubenswrapper[5052]: I1203 13:11:37.031795 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/failed-service-edpm-compute-no-nodes-edpm-compute-no-nodes5hk6k" Dec 03 13:11:37 crc kubenswrapper[5052]: I1203 13:11:37.042386 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"combined-ca-bundle" Dec 03 13:11:37 crc kubenswrapper[5052]: I1203 13:11:37.042811 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"dataplane-ansible-ssh-private-key-secret" Dec 03 13:11:37 crc kubenswrapper[5052]: I1203 13:11:37.042555 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"dataplanenodeset-edpm-compute-no-nodes" Dec 03 13:11:37 crc kubenswrapper[5052]: I1203 13:11:37.042713 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-kuttl-tests"/"openstack-aee-default-env" Dec 03 13:11:37 crc kubenswrapper[5052]: I1203 13:11:37.043285 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-kuttl-tests"/"edpm-compute-no-nodes-dockercfg-7spm4" Dec 03 13:11:37 crc kubenswrapper[5052]: I1203 13:11:37.081082 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/failed-service-edpm-compute-no-nodes-edpm-compute-no-nodes5hk6k"] Dec 03 13:11:37 crc kubenswrapper[5052]: I1203 13:11:37.169462 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wldlv\" (UniqueName: \"kubernetes.io/projected/adcccc87-3638-4486-aec4-884510a4dc63-kube-api-access-wldlv\") pod \"failed-service-edpm-compute-no-nodes-edpm-compute-no-nodes5hk6k\" (UID: \"adcccc87-3638-4486-aec4-884510a4dc63\") " pod="openstack-kuttl-tests/failed-service-edpm-compute-no-nodes-edpm-compute-no-nodes5hk6k" Dec 03 13:11:37 crc kubenswrapper[5052]: I1203 13:11:37.169569 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/adcccc87-3638-4486-aec4-884510a4dc63-inventory\") pod \"failed-service-edpm-compute-no-nodes-edpm-compute-no-nodes5hk6k\" (UID: \"adcccc87-3638-4486-aec4-884510a4dc63\") " pod="openstack-kuttl-tests/failed-service-edpm-compute-no-nodes-edpm-compute-no-nodes5hk6k" Dec 03 13:11:37 crc kubenswrapper[5052]: I1203 13:11:37.169800 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/adcccc87-3638-4486-aec4-884510a4dc63-ssh-key\") pod \"failed-service-edpm-compute-no-nodes-edpm-compute-no-nodes5hk6k\" (UID: \"adcccc87-3638-4486-aec4-884510a4dc63\") " pod="openstack-kuttl-tests/failed-service-edpm-compute-no-nodes-edpm-compute-no-nodes5hk6k" Dec 03 13:11:37 crc kubenswrapper[5052]: I1203 13:11:37.169940 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"failed-service-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/adcccc87-3638-4486-aec4-884510a4dc63-failed-service-combined-ca-bundle\") pod \"failed-service-edpm-compute-no-nodes-edpm-compute-no-nodes5hk6k\" (UID: \"adcccc87-3638-4486-aec4-884510a4dc63\") " pod="openstack-kuttl-tests/failed-service-edpm-compute-no-nodes-edpm-compute-no-nodes5hk6k" Dec 03 13:11:37 crc kubenswrapper[5052]: I1203 13:11:37.271127 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-wldlv\" (UniqueName: \"kubernetes.io/projected/adcccc87-3638-4486-aec4-884510a4dc63-kube-api-access-wldlv\") pod \"failed-service-edpm-compute-no-nodes-edpm-compute-no-nodes5hk6k\" (UID: \"adcccc87-3638-4486-aec4-884510a4dc63\") " pod="openstack-kuttl-tests/failed-service-edpm-compute-no-nodes-edpm-compute-no-nodes5hk6k" Dec 03 13:11:37 crc kubenswrapper[5052]: I1203 13:11:37.271227 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/adcccc87-3638-4486-aec4-884510a4dc63-inventory\") pod \"failed-service-edpm-compute-no-nodes-edpm-compute-no-nodes5hk6k\" (UID: \"adcccc87-3638-4486-aec4-884510a4dc63\") " pod="openstack-kuttl-tests/failed-service-edpm-compute-no-nodes-edpm-compute-no-nodes5hk6k" Dec 03 13:11:37 crc kubenswrapper[5052]: I1203 13:11:37.271280 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/adcccc87-3638-4486-aec4-884510a4dc63-ssh-key\") pod \"failed-service-edpm-compute-no-nodes-edpm-compute-no-nodes5hk6k\" (UID: \"adcccc87-3638-4486-aec4-884510a4dc63\") " pod="openstack-kuttl-tests/failed-service-edpm-compute-no-nodes-edpm-compute-no-nodes5hk6k" Dec 03 13:11:37 crc kubenswrapper[5052]: I1203 13:11:37.271341 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"failed-service-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/adcccc87-3638-4486-aec4-884510a4dc63-failed-service-combined-ca-bundle\") pod \"failed-service-edpm-compute-no-nodes-edpm-compute-no-nodes5hk6k\" (UID: \"adcccc87-3638-4486-aec4-884510a4dc63\") " pod="openstack-kuttl-tests/failed-service-edpm-compute-no-nodes-edpm-compute-no-nodes5hk6k" Dec 03 13:11:37 crc kubenswrapper[5052]: I1203 13:11:37.279277 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"failed-service-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/adcccc87-3638-4486-aec4-884510a4dc63-failed-service-combined-ca-bundle\") pod \"failed-service-edpm-compute-no-nodes-edpm-compute-no-nodes5hk6k\" (UID: \"adcccc87-3638-4486-aec4-884510a4dc63\") " pod="openstack-kuttl-tests/failed-service-edpm-compute-no-nodes-edpm-compute-no-nodes5hk6k" Dec 03 13:11:37 crc kubenswrapper[5052]: I1203 13:11:37.279925 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/adcccc87-3638-4486-aec4-884510a4dc63-inventory\") pod \"failed-service-edpm-compute-no-nodes-edpm-compute-no-nodes5hk6k\" (UID: \"adcccc87-3638-4486-aec4-884510a4dc63\") " pod="openstack-kuttl-tests/failed-service-edpm-compute-no-nodes-edpm-compute-no-nodes5hk6k" Dec 03 13:11:37 crc kubenswrapper[5052]: I1203 13:11:37.281445 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/adcccc87-3638-4486-aec4-884510a4dc63-ssh-key\") pod \"failed-service-edpm-compute-no-nodes-edpm-compute-no-nodes5hk6k\" (UID: \"adcccc87-3638-4486-aec4-884510a4dc63\") " pod="openstack-kuttl-tests/failed-service-edpm-compute-no-nodes-edpm-compute-no-nodes5hk6k" Dec 03 13:11:37 crc kubenswrapper[5052]: I1203 13:11:37.293357 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-wldlv\" (UniqueName: \"kubernetes.io/projected/adcccc87-3638-4486-aec4-884510a4dc63-kube-api-access-wldlv\") pod \"failed-service-edpm-compute-no-nodes-edpm-compute-no-nodes5hk6k\" (UID: \"adcccc87-3638-4486-aec4-884510a4dc63\") " pod="openstack-kuttl-tests/failed-service-edpm-compute-no-nodes-edpm-compute-no-nodes5hk6k" Dec 03 13:11:37 crc kubenswrapper[5052]: I1203 13:11:37.388889 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/failed-service-edpm-compute-no-nodes-edpm-compute-no-nodes5hk6k" Dec 03 13:11:37 crc kubenswrapper[5052]: I1203 13:11:37.864598 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/failed-service-edpm-compute-no-nodes-edpm-compute-no-nodes5hk6k"] Dec 03 13:11:38 crc kubenswrapper[5052]: I1203 13:11:38.889528 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/failed-service-edpm-compute-no-nodes-edpm-compute-no-nodes5hk6k" event={"ID":"adcccc87-3638-4486-aec4-884510a4dc63","Type":"ContainerStarted","Data":"89e73c0c215bbb5887aeba8a7aac0436aeada46d17549f59a4621f95d65b2357"} Dec 03 13:11:38 crc kubenswrapper[5052]: I1203 13:11:38.889879 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/failed-service-edpm-compute-no-nodes-edpm-compute-no-nodes5hk6k" event={"ID":"adcccc87-3638-4486-aec4-884510a4dc63","Type":"ContainerStarted","Data":"565fa0bbd9c5ed9b1abde7b0c7925e884a1abb3818fea77f51201e1ed57b59ea"} Dec 03 13:11:38 crc kubenswrapper[5052]: I1203 13:11:38.922896 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/failed-service-edpm-compute-no-nodes-edpm-compute-no-nodes5hk6k" podStartSLOduration=1.342230011 podStartE2EDuration="1.92287045s" podCreationTimestamp="2025-12-03 13:11:37 +0000 UTC" firstStartedPulling="2025-12-03 13:11:37.874446116 +0000 UTC m=+6688.462687880" lastFinishedPulling="2025-12-03 13:11:38.455086555 +0000 UTC m=+6689.043328319" observedRunningTime="2025-12-03 13:11:38.911997809 +0000 UTC m=+6689.500239573" watchObservedRunningTime="2025-12-03 13:11:38.92287045 +0000 UTC m=+6689.511112214" Dec 03 13:11:40 crc kubenswrapper[5052]: I1203 13:11:40.672969 5052 scope.go:117] "RemoveContainer" containerID="dbc100c8081ea1a2f5487429cc9d67de6bc8ba11282b64f248c71275375b0e16" Dec 03 13:11:40 crc kubenswrapper[5052]: E1203 13:11:40.673766 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 13:11:40 crc kubenswrapper[5052]: I1203 13:11:40.909607 5052 generic.go:334] "Generic (PLEG): container finished" podID="adcccc87-3638-4486-aec4-884510a4dc63" containerID="89e73c0c215bbb5887aeba8a7aac0436aeada46d17549f59a4621f95d65b2357" exitCode=2 Dec 03 13:11:40 crc kubenswrapper[5052]: I1203 13:11:40.909662 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/failed-service-edpm-compute-no-nodes-edpm-compute-no-nodes5hk6k" event={"ID":"adcccc87-3638-4486-aec4-884510a4dc63","Type":"ContainerDied","Data":"89e73c0c215bbb5887aeba8a7aac0436aeada46d17549f59a4621f95d65b2357"} Dec 03 13:11:42 crc kubenswrapper[5052]: I1203 13:11:42.222899 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/failed-service-edpm-compute-no-nodes-edpm-compute-no-nodes5hk6k" Dec 03 13:11:42 crc kubenswrapper[5052]: I1203 13:11:42.351770 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"failed-service-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/adcccc87-3638-4486-aec4-884510a4dc63-failed-service-combined-ca-bundle\") pod \"adcccc87-3638-4486-aec4-884510a4dc63\" (UID: \"adcccc87-3638-4486-aec4-884510a4dc63\") " Dec 03 13:11:42 crc kubenswrapper[5052]: I1203 13:11:42.351885 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wldlv\" (UniqueName: \"kubernetes.io/projected/adcccc87-3638-4486-aec4-884510a4dc63-kube-api-access-wldlv\") pod \"adcccc87-3638-4486-aec4-884510a4dc63\" (UID: \"adcccc87-3638-4486-aec4-884510a4dc63\") " Dec 03 13:11:42 crc kubenswrapper[5052]: I1203 13:11:42.351958 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/adcccc87-3638-4486-aec4-884510a4dc63-inventory\") pod \"adcccc87-3638-4486-aec4-884510a4dc63\" (UID: \"adcccc87-3638-4486-aec4-884510a4dc63\") " Dec 03 13:11:42 crc kubenswrapper[5052]: I1203 13:11:42.352036 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/adcccc87-3638-4486-aec4-884510a4dc63-ssh-key\") pod \"adcccc87-3638-4486-aec4-884510a4dc63\" (UID: \"adcccc87-3638-4486-aec4-884510a4dc63\") " Dec 03 13:11:42 crc kubenswrapper[5052]: I1203 13:11:42.357792 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/adcccc87-3638-4486-aec4-884510a4dc63-kube-api-access-wldlv" (OuterVolumeSpecName: "kube-api-access-wldlv") pod "adcccc87-3638-4486-aec4-884510a4dc63" (UID: "adcccc87-3638-4486-aec4-884510a4dc63"). InnerVolumeSpecName "kube-api-access-wldlv". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:11:42 crc kubenswrapper[5052]: I1203 13:11:42.359672 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/adcccc87-3638-4486-aec4-884510a4dc63-failed-service-combined-ca-bundle" (OuterVolumeSpecName: "failed-service-combined-ca-bundle") pod "adcccc87-3638-4486-aec4-884510a4dc63" (UID: "adcccc87-3638-4486-aec4-884510a4dc63"). InnerVolumeSpecName "failed-service-combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:11:42 crc kubenswrapper[5052]: I1203 13:11:42.373944 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/adcccc87-3638-4486-aec4-884510a4dc63-ssh-key" (OuterVolumeSpecName: "ssh-key") pod "adcccc87-3638-4486-aec4-884510a4dc63" (UID: "adcccc87-3638-4486-aec4-884510a4dc63"). InnerVolumeSpecName "ssh-key". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:11:42 crc kubenswrapper[5052]: I1203 13:11:42.386716 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/adcccc87-3638-4486-aec4-884510a4dc63-inventory" (OuterVolumeSpecName: "inventory") pod "adcccc87-3638-4486-aec4-884510a4dc63" (UID: "adcccc87-3638-4486-aec4-884510a4dc63"). InnerVolumeSpecName "inventory". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:11:42 crc kubenswrapper[5052]: I1203 13:11:42.453730 5052 reconciler_common.go:293] "Volume detached for volume \"failed-service-combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/adcccc87-3638-4486-aec4-884510a4dc63-failed-service-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 13:11:42 crc kubenswrapper[5052]: I1203 13:11:42.453778 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-wldlv\" (UniqueName: \"kubernetes.io/projected/adcccc87-3638-4486-aec4-884510a4dc63-kube-api-access-wldlv\") on node \"crc\" DevicePath \"\"" Dec 03 13:11:42 crc kubenswrapper[5052]: I1203 13:11:42.453793 5052 reconciler_common.go:293] "Volume detached for volume \"inventory\" (UniqueName: \"kubernetes.io/secret/adcccc87-3638-4486-aec4-884510a4dc63-inventory\") on node \"crc\" DevicePath \"\"" Dec 03 13:11:42 crc kubenswrapper[5052]: I1203 13:11:42.453804 5052 reconciler_common.go:293] "Volume detached for volume \"ssh-key\" (UniqueName: \"kubernetes.io/secret/adcccc87-3638-4486-aec4-884510a4dc63-ssh-key\") on node \"crc\" DevicePath \"\"" Dec 03 13:11:42 crc kubenswrapper[5052]: I1203 13:11:42.929056 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/failed-service-edpm-compute-no-nodes-edpm-compute-no-nodes5hk6k" event={"ID":"adcccc87-3638-4486-aec4-884510a4dc63","Type":"ContainerDied","Data":"565fa0bbd9c5ed9b1abde7b0c7925e884a1abb3818fea77f51201e1ed57b59ea"} Dec 03 13:11:42 crc kubenswrapper[5052]: I1203 13:11:42.929370 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="565fa0bbd9c5ed9b1abde7b0c7925e884a1abb3818fea77f51201e1ed57b59ea" Dec 03 13:11:42 crc kubenswrapper[5052]: I1203 13:11:42.929125 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/failed-service-edpm-compute-no-nodes-edpm-compute-no-nodes5hk6k" Dec 03 13:11:43 crc kubenswrapper[5052]: I1203 13:11:43.805815 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/failed-service-edpm-compute-no-nodes-edpm-compute-no-nodes6wb5q"] Dec 03 13:11:43 crc kubenswrapper[5052]: I1203 13:11:43.814214 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/failed-service-edpm-compute-no-nodes-edpm-compute-no-nodesr47df"] Dec 03 13:11:43 crc kubenswrapper[5052]: I1203 13:11:43.823107 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/failed-service-edpm-compute-no-nodes-edpm-compute-no-nodes5hk6k"] Dec 03 13:11:43 crc kubenswrapper[5052]: I1203 13:11:43.830968 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/failed-service-edpm-compute-no-nodes-edpm-compute-no-nodesz9xr7"] Dec 03 13:11:43 crc kubenswrapper[5052]: I1203 13:11:43.838054 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/failed-service-edpm-compute-no-nodes-edpm-compute-no-nodesz9xr7"] Dec 03 13:11:43 crc kubenswrapper[5052]: I1203 13:11:43.847218 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/failed-service-edpm-compute-no-nodes-edpm-compute-no-nodes6wb5q"] Dec 03 13:11:43 crc kubenswrapper[5052]: I1203 13:11:43.857957 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/failed-service-edpm-compute-no-nodes-edpm-compute-no-nodes5hk6k"] Dec 03 13:11:43 crc kubenswrapper[5052]: I1203 13:11:43.866243 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/failed-service-edpm-compute-no-nodes-edpm-compute-no-nodesr47df"] Dec 03 13:11:43 crc kubenswrapper[5052]: I1203 13:11:43.895645 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-d8nh7"] Dec 03 13:11:43 crc kubenswrapper[5052]: E1203 13:11:43.896069 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="adcccc87-3638-4486-aec4-884510a4dc63" containerName="failed-service-edpm-compute-no-nodes-edpm-compute-no-nodes" Dec 03 13:11:43 crc kubenswrapper[5052]: I1203 13:11:43.896103 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="adcccc87-3638-4486-aec4-884510a4dc63" containerName="failed-service-edpm-compute-no-nodes-edpm-compute-no-nodes" Dec 03 13:11:43 crc kubenswrapper[5052]: I1203 13:11:43.896324 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="adcccc87-3638-4486-aec4-884510a4dc63" containerName="failed-service-edpm-compute-no-nodes-edpm-compute-no-nodes" Dec 03 13:11:43 crc kubenswrapper[5052]: I1203 13:11:43.897392 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-d8nh7" Dec 03 13:11:43 crc kubenswrapper[5052]: I1203 13:11:43.910030 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-d8nh7"] Dec 03 13:11:43 crc kubenswrapper[5052]: I1203 13:11:43.974578 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/84b4fa9a-0498-4663-8ed2-2609ba3e2f88-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-84b9f45d47-d8nh7\" (UID: \"84b4fa9a-0498-4663-8ed2-2609ba3e2f88\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-d8nh7" Dec 03 13:11:43 crc kubenswrapper[5052]: I1203 13:11:43.974632 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/84b4fa9a-0498-4663-8ed2-2609ba3e2f88-config\") pod \"dnsmasq-dnsmasq-84b9f45d47-d8nh7\" (UID: \"84b4fa9a-0498-4663-8ed2-2609ba3e2f88\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-d8nh7" Dec 03 13:11:43 crc kubenswrapper[5052]: I1203 13:11:43.974741 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v9fwl\" (UniqueName: \"kubernetes.io/projected/84b4fa9a-0498-4663-8ed2-2609ba3e2f88-kube-api-access-v9fwl\") pod \"dnsmasq-dnsmasq-84b9f45d47-d8nh7\" (UID: \"84b4fa9a-0498-4663-8ed2-2609ba3e2f88\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-d8nh7" Dec 03 13:11:44 crc kubenswrapper[5052]: I1203 13:11:44.076082 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/84b4fa9a-0498-4663-8ed2-2609ba3e2f88-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-84b9f45d47-d8nh7\" (UID: \"84b4fa9a-0498-4663-8ed2-2609ba3e2f88\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-d8nh7" Dec 03 13:11:44 crc kubenswrapper[5052]: I1203 13:11:44.076128 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/84b4fa9a-0498-4663-8ed2-2609ba3e2f88-config\") pod \"dnsmasq-dnsmasq-84b9f45d47-d8nh7\" (UID: \"84b4fa9a-0498-4663-8ed2-2609ba3e2f88\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-d8nh7" Dec 03 13:11:44 crc kubenswrapper[5052]: I1203 13:11:44.076208 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-v9fwl\" (UniqueName: \"kubernetes.io/projected/84b4fa9a-0498-4663-8ed2-2609ba3e2f88-kube-api-access-v9fwl\") pod \"dnsmasq-dnsmasq-84b9f45d47-d8nh7\" (UID: \"84b4fa9a-0498-4663-8ed2-2609ba3e2f88\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-d8nh7" Dec 03 13:11:44 crc kubenswrapper[5052]: I1203 13:11:44.077163 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/84b4fa9a-0498-4663-8ed2-2609ba3e2f88-dnsmasq-svc\") pod \"dnsmasq-dnsmasq-84b9f45d47-d8nh7\" (UID: \"84b4fa9a-0498-4663-8ed2-2609ba3e2f88\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-d8nh7" Dec 03 13:11:44 crc kubenswrapper[5052]: I1203 13:11:44.077361 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/84b4fa9a-0498-4663-8ed2-2609ba3e2f88-config\") pod \"dnsmasq-dnsmasq-84b9f45d47-d8nh7\" (UID: \"84b4fa9a-0498-4663-8ed2-2609ba3e2f88\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-d8nh7" Dec 03 13:11:44 crc kubenswrapper[5052]: I1203 13:11:44.099633 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-v9fwl\" (UniqueName: \"kubernetes.io/projected/84b4fa9a-0498-4663-8ed2-2609ba3e2f88-kube-api-access-v9fwl\") pod \"dnsmasq-dnsmasq-84b9f45d47-d8nh7\" (UID: \"84b4fa9a-0498-4663-8ed2-2609ba3e2f88\") " pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-d8nh7" Dec 03 13:11:44 crc kubenswrapper[5052]: I1203 13:11:44.239874 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-d8nh7" Dec 03 13:11:44 crc kubenswrapper[5052]: I1203 13:11:44.310952 5052 scope.go:117] "RemoveContainer" containerID="3f41b3b842480bdfc1ae1546f4e2858dffe1059dff604526e61f5cdacd5a2155" Dec 03 13:11:44 crc kubenswrapper[5052]: I1203 13:11:44.376876 5052 scope.go:117] "RemoveContainer" containerID="b2159e0375630515208289b10bff2d1ff4c8f6302f4feeb712b83c151354ccde" Dec 03 13:11:44 crc kubenswrapper[5052]: I1203 13:11:44.403977 5052 scope.go:117] "RemoveContainer" containerID="72c60c8ec14bec08d197cc124437fb923cfe931408d815351ba64acc2169da26" Dec 03 13:11:44 crc kubenswrapper[5052]: I1203 13:11:44.442864 5052 scope.go:117] "RemoveContainer" containerID="5fbbcf4e8261ee635f3df3bdf47a7c2e7675875364098a4863ec002d31248f1f" Dec 03 13:11:44 crc kubenswrapper[5052]: I1203 13:11:44.477112 5052 scope.go:117] "RemoveContainer" containerID="df668bb734666f2293cb0286d21d224f05868ec6e232b49bbfa1d595aec11db4" Dec 03 13:11:44 crc kubenswrapper[5052]: I1203 13:11:44.510584 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-d8nh7"] Dec 03 13:11:44 crc kubenswrapper[5052]: I1203 13:11:44.513386 5052 scope.go:117] "RemoveContainer" containerID="91930adc08bdb71680d2e02f8ac8466bfa248f6135ebeeb75dbf410e50719566" Dec 03 13:11:44 crc kubenswrapper[5052]: I1203 13:11:44.954080 5052 generic.go:334] "Generic (PLEG): container finished" podID="84b4fa9a-0498-4663-8ed2-2609ba3e2f88" containerID="4935bedc610bfa9d2532d0663da830e623b4b3ef3e4e3a033e7fecfae8c7945f" exitCode=0 Dec 03 13:11:44 crc kubenswrapper[5052]: I1203 13:11:44.954134 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-d8nh7" event={"ID":"84b4fa9a-0498-4663-8ed2-2609ba3e2f88","Type":"ContainerDied","Data":"4935bedc610bfa9d2532d0663da830e623b4b3ef3e4e3a033e7fecfae8c7945f"} Dec 03 13:11:44 crc kubenswrapper[5052]: I1203 13:11:44.954188 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-d8nh7" event={"ID":"84b4fa9a-0498-4663-8ed2-2609ba3e2f88","Type":"ContainerStarted","Data":"6c7269c8a774f6557603ebdd41645b8666a093da7f72e3e2283441e91d50de78"} Dec 03 13:11:45 crc kubenswrapper[5052]: I1203 13:11:45.683789 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="adcccc87-3638-4486-aec4-884510a4dc63" path="/var/lib/kubelet/pods/adcccc87-3638-4486-aec4-884510a4dc63/volumes" Dec 03 13:11:45 crc kubenswrapper[5052]: I1203 13:11:45.685008 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b9427361-e875-4d16-b3a6-cf4629dc9d00" path="/var/lib/kubelet/pods/b9427361-e875-4d16-b3a6-cf4629dc9d00/volumes" Dec 03 13:11:45 crc kubenswrapper[5052]: I1203 13:11:45.686074 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bde7c5d5-7a99-43aa-92eb-357d00206cd7" path="/var/lib/kubelet/pods/bde7c5d5-7a99-43aa-92eb-357d00206cd7/volumes" Dec 03 13:11:45 crc kubenswrapper[5052]: I1203 13:11:45.687287 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e3636dea-6b99-4621-a2c1-edb24bcb0990" path="/var/lib/kubelet/pods/e3636dea-6b99-4621-a2c1-edb24bcb0990/volumes" Dec 03 13:11:45 crc kubenswrapper[5052]: I1203 13:11:45.964160 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-d8nh7" event={"ID":"84b4fa9a-0498-4663-8ed2-2609ba3e2f88","Type":"ContainerStarted","Data":"1036dd7c5c2657c919572246ddc67dc56064933c479d2d9da80a91500712e419"} Dec 03 13:11:45 crc kubenswrapper[5052]: I1203 13:11:45.964341 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-d8nh7" Dec 03 13:11:45 crc kubenswrapper[5052]: I1203 13:11:45.987158 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-d8nh7" podStartSLOduration=2.987137145 podStartE2EDuration="2.987137145s" podCreationTimestamp="2025-12-03 13:11:43 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 13:11:45.980032521 +0000 UTC m=+6696.568274285" watchObservedRunningTime="2025-12-03 13:11:45.987137145 +0000 UTC m=+6696.575378919" Dec 03 13:11:47 crc kubenswrapper[5052]: I1203 13:11:47.617321 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-operators/nova-operator-controller-manager-697bc559fc-gcj5d"] Dec 03 13:11:47 crc kubenswrapper[5052]: I1203 13:11:47.617631 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-operators/nova-operator-controller-manager-697bc559fc-gcj5d" podUID="3e6bc3f6-1f1a-4b99-8835-98252369d364" containerName="manager" containerID="cri-o://21bac89f97ac17aa04e4ad5d34fe06b056702b1896eabe8f69dc6ad5ebbc42bf" gracePeriod=10 Dec 03 13:11:47 crc kubenswrapper[5052]: I1203 13:11:47.618047 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-operators/nova-operator-controller-manager-697bc559fc-gcj5d" podUID="3e6bc3f6-1f1a-4b99-8835-98252369d364" containerName="kube-rbac-proxy" containerID="cri-o://a7e939b334d0e92f65588cb0c22e7536f6716a1e32472cd5033b456da2b36a68" gracePeriod=10 Dec 03 13:11:47 crc kubenswrapper[5052]: I1203 13:11:47.625191 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-operators/octavia-operator-controller-manager-998648c74-z6cw9"] Dec 03 13:11:47 crc kubenswrapper[5052]: I1203 13:11:47.625464 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-operators/octavia-operator-controller-manager-998648c74-z6cw9" podUID="50c6e918-7f1f-4d6f-ba5f-940e5f485825" containerName="manager" containerID="cri-o://829ed8e10d7aad4792ee147f6891ccdb7a056f605c04a4c0349d94bfa248bff9" gracePeriod=10 Dec 03 13:11:47 crc kubenswrapper[5052]: I1203 13:11:47.625529 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-operators/octavia-operator-controller-manager-998648c74-z6cw9" podUID="50c6e918-7f1f-4d6f-ba5f-940e5f485825" containerName="kube-rbac-proxy" containerID="cri-o://c55bdca1c305eb9768bdfbe023291651a872e6e10fe50e1b32182c071f21b870" gracePeriod=10 Dec 03 13:11:47 crc kubenswrapper[5052]: I1203 13:11:47.640597 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-operators/rabbitmq-cluster-operator-manager-668c99d594-ksw2p"] Dec 03 13:11:47 crc kubenswrapper[5052]: I1203 13:11:47.640858 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-operators/rabbitmq-cluster-operator-manager-668c99d594-ksw2p" podUID="459f5555-efae-4c19-84d0-2af917b9560f" containerName="operator" containerID="cri-o://9f7f47624da378a4c9e182afc6f0245cb94a2c07414a47ce8f104bcb13883f17" gracePeriod=10 Dec 03 13:11:47 crc kubenswrapper[5052]: I1203 13:11:47.647946 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-operators/telemetry-operator-controller-manager-76cc84c6bb-zl7px"] Dec 03 13:11:47 crc kubenswrapper[5052]: I1203 13:11:47.648167 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-operators/telemetry-operator-controller-manager-76cc84c6bb-zl7px" podUID="3894016b-8242-4b12-8407-bcf78af00851" containerName="kube-rbac-proxy" containerID="cri-o://97920a5e6bb5c96fa08c020aae2055f6d6c6ddf72d91e1b35081548101fd59bc" gracePeriod=10 Dec 03 13:11:47 crc kubenswrapper[5052]: I1203 13:11:47.648243 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-operators/telemetry-operator-controller-manager-76cc84c6bb-zl7px" podUID="3894016b-8242-4b12-8407-bcf78af00851" containerName="manager" containerID="cri-o://3015e5486b47a528197db8ba47bf3513191f52877b648e0cf08565573ad9202b" gracePeriod=10 Dec 03 13:11:47 crc kubenswrapper[5052]: I1203 13:11:47.668865 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-operators/designate-operator-controller-manager-78b4bc895b-874g2"] Dec 03 13:11:47 crc kubenswrapper[5052]: I1203 13:11:47.669111 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-operators/designate-operator-controller-manager-78b4bc895b-874g2" podUID="2e2dc077-c02f-4f98-8fa6-9f72890faeaf" containerName="manager" containerID="cri-o://bc9d68e8694514b249866a594a571232d780cb8f8315ced546035025828c5930" gracePeriod=10 Dec 03 13:11:47 crc kubenswrapper[5052]: I1203 13:11:47.669675 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-operators/designate-operator-controller-manager-78b4bc895b-874g2" podUID="2e2dc077-c02f-4f98-8fa6-9f72890faeaf" containerName="kube-rbac-proxy" containerID="cri-o://c38ccb2125893d9107a9365db7e805b861125cfe34c07ee20dd8158df83fc99c" gracePeriod=10 Dec 03 13:11:47 crc kubenswrapper[5052]: I1203 13:11:47.693563 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-operators/manila-operator-controller-manager-7c79b5df47-ccbqq"] Dec 03 13:11:47 crc kubenswrapper[5052]: I1203 13:11:47.693594 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-operators/neutron-operator-controller-manager-5fdfd5b6b5-q48nd"] Dec 03 13:11:47 crc kubenswrapper[5052]: I1203 13:11:47.693794 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-operators/neutron-operator-controller-manager-5fdfd5b6b5-q48nd" podUID="53c88ca3-39f7-4a81-b305-40047bcaeb88" containerName="kube-rbac-proxy" containerID="cri-o://5216edc450cbbbdc9be723de0d23e9c4c7f79734c44ce55e5ee6afa4b2ca1bd2" gracePeriod=10 Dec 03 13:11:47 crc kubenswrapper[5052]: I1203 13:11:47.693924 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-operators/neutron-operator-controller-manager-5fdfd5b6b5-q48nd" podUID="53c88ca3-39f7-4a81-b305-40047bcaeb88" containerName="manager" containerID="cri-o://efce01fd34c9aaa82b7beeb80cae30909d6a58a957f233a93543d43ab86d6552" gracePeriod=10 Dec 03 13:11:47 crc kubenswrapper[5052]: I1203 13:11:47.693928 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-operators/manila-operator-controller-manager-7c79b5df47-ccbqq" podUID="ea1e91de-7e7d-498a-a17b-551f2bee325a" containerName="manager" containerID="cri-o://0add256c4e87d7b91cc63837e4958d34bfbf4128a08af08316deec77be2a75eb" gracePeriod=10 Dec 03 13:11:47 crc kubenswrapper[5052]: I1203 13:11:47.694073 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-operators/manila-operator-controller-manager-7c79b5df47-ccbqq" podUID="ea1e91de-7e7d-498a-a17b-551f2bee325a" containerName="kube-rbac-proxy" containerID="cri-o://c8a22e6613869e6f1219b539f8d24e9e232f41001d8d1d96f9c514c62fd7214c" gracePeriod=10 Dec 03 13:11:47 crc kubenswrapper[5052]: I1203 13:11:47.723644 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-operators/mariadb-operator-controller-manager-56bbcc9d85-sr7zw"] Dec 03 13:11:47 crc kubenswrapper[5052]: I1203 13:11:47.723946 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-operators/mariadb-operator-controller-manager-56bbcc9d85-sr7zw" podUID="f0e70c0d-10fe-43b7-a08c-0ce411f223a6" containerName="manager" containerID="cri-o://c16b4916d85027f28775414e1e9e061796b804441bf4aded0b2a1af11134a874" gracePeriod=10 Dec 03 13:11:47 crc kubenswrapper[5052]: I1203 13:11:47.724143 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-operators/mariadb-operator-controller-manager-56bbcc9d85-sr7zw" podUID="f0e70c0d-10fe-43b7-a08c-0ce411f223a6" containerName="kube-rbac-proxy" containerID="cri-o://7a38fc46fa1de916c4dcdb66fb009b9e798bf1d5b7c4bb72dc6db4bde166741c" gracePeriod=10 Dec 03 13:11:47 crc kubenswrapper[5052]: I1203 13:11:47.765575 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-operators/ironic-operator-controller-manager-6c548fd776-pqfnd"] Dec 03 13:11:47 crc kubenswrapper[5052]: I1203 13:11:47.765812 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-operators/ironic-operator-controller-manager-6c548fd776-pqfnd" podUID="7017b3f7-1eb9-4dfe-8ef3-6db27cf3a1f1" containerName="manager" containerID="cri-o://acbb799df7a125d48309f76c7ac211fd06dfd6580980ca517793b3d574553f2e" gracePeriod=10 Dec 03 13:11:47 crc kubenswrapper[5052]: I1203 13:11:47.765955 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-operators/ironic-operator-controller-manager-6c548fd776-pqfnd" podUID="7017b3f7-1eb9-4dfe-8ef3-6db27cf3a1f1" containerName="kube-rbac-proxy" containerID="cri-o://f4171f259274e4008a6e5b98254b34066d3fc61afa68cf052421761cd5affdfb" gracePeriod=10 Dec 03 13:11:47 crc kubenswrapper[5052]: I1203 13:11:47.803784 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-operators/keystone-operator-controller-manager-7765d96ddf-nlrl9"] Dec 03 13:11:47 crc kubenswrapper[5052]: I1203 13:11:47.804092 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-operators/keystone-operator-controller-manager-7765d96ddf-nlrl9" podUID="95b50980-fd5a-4a19-b1cd-13df98d257fe" containerName="kube-rbac-proxy" containerID="cri-o://97782ec29d2e9e3193fca38c78d6d80a51ddf23167b6089a154cabfc32ab7307" gracePeriod=10 Dec 03 13:11:47 crc kubenswrapper[5052]: I1203 13:11:47.804687 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-operators/keystone-operator-controller-manager-7765d96ddf-nlrl9" podUID="95b50980-fd5a-4a19-b1cd-13df98d257fe" containerName="manager" containerID="cri-o://61b7e037d8a6c427bda841efbbb994e108ee2daac74c44aca5542d608eed3910" gracePeriod=10 Dec 03 13:11:47 crc kubenswrapper[5052]: I1203 13:11:47.838327 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-operators/horizon-operator-controller-manager-68c6d99b8f-mjdvh"] Dec 03 13:11:47 crc kubenswrapper[5052]: I1203 13:11:47.838648 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-operators/horizon-operator-controller-manager-68c6d99b8f-mjdvh" podUID="ce3d34f6-e545-4eb2-bcc5-999377931601" containerName="manager" containerID="cri-o://202df0b6bac6f793c5ac016ef55a754be22068b68cff50a0616b5de880db9a3c" gracePeriod=10 Dec 03 13:11:47 crc kubenswrapper[5052]: I1203 13:11:47.839075 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-operators/horizon-operator-controller-manager-68c6d99b8f-mjdvh" podUID="ce3d34f6-e545-4eb2-bcc5-999377931601" containerName="kube-rbac-proxy" containerID="cri-o://6ba995fc9ab1c19d638faf3e62a10a01ba4220c74844ad63f599ace4affd6e49" gracePeriod=10 Dec 03 13:11:47 crc kubenswrapper[5052]: I1203 13:11:47.870087 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-operators/cinder-operator-controller-manager-859b6ccc6-vt9d2"] Dec 03 13:11:47 crc kubenswrapper[5052]: I1203 13:11:47.870331 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-operators/cinder-operator-controller-manager-859b6ccc6-vt9d2" podUID="5968d273-33dd-45cc-aab0-e83628bcd15c" containerName="manager" containerID="cri-o://e020c88c64940fc2399454d010fff5531ddc1320fccf47d9835db05c4424f1b2" gracePeriod=10 Dec 03 13:11:47 crc kubenswrapper[5052]: I1203 13:11:47.870753 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-operators/cinder-operator-controller-manager-859b6ccc6-vt9d2" podUID="5968d273-33dd-45cc-aab0-e83628bcd15c" containerName="kube-rbac-proxy" containerID="cri-o://cd51be9345abb5eafe858033c68a1ced7d4100915afaf161d165bcac5ef6661e" gracePeriod=10 Dec 03 13:11:47 crc kubenswrapper[5052]: I1203 13:11:47.893317 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-operators/test-operator-controller-manager-5854674fcc-jltb6"] Dec 03 13:11:47 crc kubenswrapper[5052]: I1203 13:11:47.893639 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-operators/test-operator-controller-manager-5854674fcc-jltb6" podUID="1cb3d739-e4d6-4058-9456-a3c2e1c09130" containerName="manager" containerID="cri-o://7f760d28d8a9a27373a0d324cb077f28fc6b0310ae54ade63b76d7f19979ccad" gracePeriod=10 Dec 03 13:11:47 crc kubenswrapper[5052]: I1203 13:11:47.897291 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-operators/test-operator-controller-manager-5854674fcc-jltb6" podUID="1cb3d739-e4d6-4058-9456-a3c2e1c09130" containerName="kube-rbac-proxy" containerID="cri-o://dabaf7e10aa7d0b0ec06b6b8e0023ae509392bd5567d994a850b73988ff68ec4" gracePeriod=10 Dec 03 13:11:47 crc kubenswrapper[5052]: I1203 13:11:47.908903 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-operators/barbican-operator-controller-manager-7d9dfd778-cqfdk"] Dec 03 13:11:47 crc kubenswrapper[5052]: I1203 13:11:47.909192 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-operators/barbican-operator-controller-manager-7d9dfd778-cqfdk" podUID="8dbd25a1-9de9-431e-be37-d239e3f64728" containerName="manager" containerID="cri-o://5025f6eb197bd7384ff4dad6b52992f6489ac18de5c05c2535a9026ee37b6f24" gracePeriod=10 Dec 03 13:11:47 crc kubenswrapper[5052]: I1203 13:11:47.909377 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-operators/barbican-operator-controller-manager-7d9dfd778-cqfdk" podUID="8dbd25a1-9de9-431e-be37-d239e3f64728" containerName="kube-rbac-proxy" containerID="cri-o://754a05a1ad89a85b548ec6034299b8041b571878c20c15a1d29f8a87a0d19577" gracePeriod=10 Dec 03 13:11:47 crc kubenswrapper[5052]: I1203 13:11:47.927059 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-operators/heat-operator-controller-manager-5f64f6f8bb-48fnb"] Dec 03 13:11:47 crc kubenswrapper[5052]: I1203 13:11:47.927396 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-operators/heat-operator-controller-manager-5f64f6f8bb-48fnb" podUID="0cbd234c-ecb3-4e54-af86-a2b15fdd94a3" containerName="manager" containerID="cri-o://05da1e86576897a06ee62b48ffa3acc05751be603b91af013f2fc3b98094a039" gracePeriod=10 Dec 03 13:11:47 crc kubenswrapper[5052]: I1203 13:11:47.927739 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-operators/heat-operator-controller-manager-5f64f6f8bb-48fnb" podUID="0cbd234c-ecb3-4e54-af86-a2b15fdd94a3" containerName="kube-rbac-proxy" containerID="cri-o://ac6e6a21862aaede93ccb524e481d0b5e6a4f1f2e70966ac0cb19106016bbeef" gracePeriod=10 Dec 03 13:11:47 crc kubenswrapper[5052]: I1203 13:11:47.955188 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-operators/swift-operator-controller-manager-5f8c65bbfc-2cgdz"] Dec 03 13:11:47 crc kubenswrapper[5052]: I1203 13:11:47.955527 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-operators/swift-operator-controller-manager-5f8c65bbfc-2cgdz" podUID="8fd08931-7f0b-4d5b-bfb1-f27980648561" containerName="manager" containerID="cri-o://a075d6f4a44381ae20a16b8fa5bb2d8c0561bce325ee8a9d99cee59abaad79c8" gracePeriod=10 Dec 03 13:11:47 crc kubenswrapper[5052]: I1203 13:11:47.956048 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-operators/swift-operator-controller-manager-5f8c65bbfc-2cgdz" podUID="8fd08931-7f0b-4d5b-bfb1-f27980648561" containerName="kube-rbac-proxy" containerID="cri-o://0c828acdac4251182a50609e1c01b6a1bb38eb6449736b7bbd7aef20f9dc2f09" gracePeriod=10 Dec 03 13:11:47 crc kubenswrapper[5052]: I1203 13:11:47.987159 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-operators/placement-operator-controller-manager-78f8948974-j62hd"] Dec 03 13:11:47 crc kubenswrapper[5052]: I1203 13:11:47.987460 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-operators/placement-operator-controller-manager-78f8948974-j62hd" podUID="37ab421b-0ad3-4d9d-bf47-8bf908c4aa57" containerName="manager" containerID="cri-o://e3943739460d5bb48bf27023494d0f23c5ff3cd0f58f04e59d49d0dbc05a7830" gracePeriod=10 Dec 03 13:11:47 crc kubenswrapper[5052]: I1203 13:11:47.987830 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-operators/placement-operator-controller-manager-78f8948974-j62hd" podUID="37ab421b-0ad3-4d9d-bf47-8bf908c4aa57" containerName="kube-rbac-proxy" containerID="cri-o://2dcc207b577f508a2a51aeed06eda37f7a37cdaa339bc4bd5b4f2712d43ec227" gracePeriod=10 Dec 03 13:11:48 crc kubenswrapper[5052]: I1203 13:11:48.009039 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-operators/openstack-operator-controller-manager-9f56fc979-b8jv8"] Dec 03 13:11:48 crc kubenswrapper[5052]: I1203 13:11:48.009286 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-operators/openstack-operator-controller-manager-9f56fc979-b8jv8" podUID="527bcd50-6d8d-4d4f-bc11-5382eb9adfc2" containerName="manager" containerID="cri-o://d1bb936db1c98c58f4e109008a5dc7d293c9a04b310fd4b2ab5bb2e70449c5aa" gracePeriod=10 Dec 03 13:11:48 crc kubenswrapper[5052]: I1203 13:11:48.028353 5052 generic.go:334] "Generic (PLEG): container finished" podID="3894016b-8242-4b12-8407-bcf78af00851" containerID="3015e5486b47a528197db8ba47bf3513191f52877b648e0cf08565573ad9202b" exitCode=0 Dec 03 13:11:48 crc kubenswrapper[5052]: I1203 13:11:48.028385 5052 generic.go:334] "Generic (PLEG): container finished" podID="3894016b-8242-4b12-8407-bcf78af00851" containerID="97920a5e6bb5c96fa08c020aae2055f6d6c6ddf72d91e1b35081548101fd59bc" exitCode=0 Dec 03 13:11:48 crc kubenswrapper[5052]: I1203 13:11:48.028438 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/telemetry-operator-controller-manager-76cc84c6bb-zl7px" event={"ID":"3894016b-8242-4b12-8407-bcf78af00851","Type":"ContainerDied","Data":"3015e5486b47a528197db8ba47bf3513191f52877b648e0cf08565573ad9202b"} Dec 03 13:11:48 crc kubenswrapper[5052]: I1203 13:11:48.028468 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/telemetry-operator-controller-manager-76cc84c6bb-zl7px" event={"ID":"3894016b-8242-4b12-8407-bcf78af00851","Type":"ContainerDied","Data":"97920a5e6bb5c96fa08c020aae2055f6d6c6ddf72d91e1b35081548101fd59bc"} Dec 03 13:11:48 crc kubenswrapper[5052]: I1203 13:11:48.044955 5052 generic.go:334] "Generic (PLEG): container finished" podID="53c88ca3-39f7-4a81-b305-40047bcaeb88" containerID="5216edc450cbbbdc9be723de0d23e9c4c7f79734c44ce55e5ee6afa4b2ca1bd2" exitCode=0 Dec 03 13:11:48 crc kubenswrapper[5052]: I1203 13:11:48.045016 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/neutron-operator-controller-manager-5fdfd5b6b5-q48nd" event={"ID":"53c88ca3-39f7-4a81-b305-40047bcaeb88","Type":"ContainerDied","Data":"5216edc450cbbbdc9be723de0d23e9c4c7f79734c44ce55e5ee6afa4b2ca1bd2"} Dec 03 13:11:48 crc kubenswrapper[5052]: I1203 13:11:48.052339 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-operators/infra-operator-controller-manager-57548d458d-9fkll"] Dec 03 13:11:48 crc kubenswrapper[5052]: I1203 13:11:48.052719 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-operators/infra-operator-controller-manager-57548d458d-9fkll" podUID="81f5f8e1-015a-4b45-83e6-13e88efb8e80" containerName="manager" containerID="cri-o://43ae682827f3b8fe92dd61ac021bde418f76e0a34a72497c150c64083536410d" gracePeriod=10 Dec 03 13:11:48 crc kubenswrapper[5052]: I1203 13:11:48.053295 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-operators/infra-operator-controller-manager-57548d458d-9fkll" podUID="81f5f8e1-015a-4b45-83e6-13e88efb8e80" containerName="kube-rbac-proxy" containerID="cri-o://01a4046ea27a459acec0b9a6a2a20c352ddcb26e0f79bddb80874932462702e9" gracePeriod=10 Dec 03 13:11:48 crc kubenswrapper[5052]: I1203 13:11:48.079055 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-operators/ovn-operator-controller-manager-b6456fdb6-sggpb"] Dec 03 13:11:48 crc kubenswrapper[5052]: I1203 13:11:48.079249 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-operators/ovn-operator-controller-manager-b6456fdb6-sggpb" podUID="a605d632-d9a5-4ce0-9449-e3d57105df1b" containerName="manager" containerID="cri-o://2944fe91598c6c5f038d1de9c8551c8352a4c6b145016c6151bfc37b330edddd" gracePeriod=10 Dec 03 13:11:48 crc kubenswrapper[5052]: I1203 13:11:48.079596 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-operators/ovn-operator-controller-manager-b6456fdb6-sggpb" podUID="a605d632-d9a5-4ce0-9449-e3d57105df1b" containerName="kube-rbac-proxy" containerID="cri-o://6e810659ae074c724982667dc34e29af08df25d8aa94eea774ad9e2b2ba3b241" gracePeriod=10 Dec 03 13:11:48 crc kubenswrapper[5052]: I1203 13:11:48.096843 5052 generic.go:334] "Generic (PLEG): container finished" podID="50c6e918-7f1f-4d6f-ba5f-940e5f485825" containerID="c55bdca1c305eb9768bdfbe023291651a872e6e10fe50e1b32182c071f21b870" exitCode=0 Dec 03 13:11:48 crc kubenswrapper[5052]: I1203 13:11:48.096875 5052 generic.go:334] "Generic (PLEG): container finished" podID="50c6e918-7f1f-4d6f-ba5f-940e5f485825" containerID="829ed8e10d7aad4792ee147f6891ccdb7a056f605c04a4c0349d94bfa248bff9" exitCode=0 Dec 03 13:11:48 crc kubenswrapper[5052]: I1203 13:11:48.096939 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/octavia-operator-controller-manager-998648c74-z6cw9" event={"ID":"50c6e918-7f1f-4d6f-ba5f-940e5f485825","Type":"ContainerDied","Data":"c55bdca1c305eb9768bdfbe023291651a872e6e10fe50e1b32182c071f21b870"} Dec 03 13:11:48 crc kubenswrapper[5052]: I1203 13:11:48.096965 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/octavia-operator-controller-manager-998648c74-z6cw9" event={"ID":"50c6e918-7f1f-4d6f-ba5f-940e5f485825","Type":"ContainerDied","Data":"829ed8e10d7aad4792ee147f6891ccdb7a056f605c04a4c0349d94bfa248bff9"} Dec 03 13:11:48 crc kubenswrapper[5052]: I1203 13:11:48.101245 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-operators/watcher-operator-controller-manager-769dc69bc-7rhg6"] Dec 03 13:11:48 crc kubenswrapper[5052]: I1203 13:11:48.101457 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-operators/watcher-operator-controller-manager-769dc69bc-7rhg6" podUID="6bea5f42-82da-4148-9996-dc05121212f7" containerName="manager" containerID="cri-o://af5bcb24f3ea207a68cadd9479e8bc16e424b9d9fac437c1e5ea1ef07af38d6d" gracePeriod=10 Dec 03 13:11:48 crc kubenswrapper[5052]: I1203 13:11:48.101567 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-operators/watcher-operator-controller-manager-769dc69bc-7rhg6" podUID="6bea5f42-82da-4148-9996-dc05121212f7" containerName="kube-rbac-proxy" containerID="cri-o://7145c8d45908da236459c9313b6926c516ea98b7ba253967ef2007aca4397a66" gracePeriod=10 Dec 03 13:11:48 crc kubenswrapper[5052]: I1203 13:11:48.129965 5052 generic.go:334] "Generic (PLEG): container finished" podID="95b50980-fd5a-4a19-b1cd-13df98d257fe" containerID="97782ec29d2e9e3193fca38c78d6d80a51ddf23167b6089a154cabfc32ab7307" exitCode=0 Dec 03 13:11:48 crc kubenswrapper[5052]: I1203 13:11:48.130057 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/keystone-operator-controller-manager-7765d96ddf-nlrl9" event={"ID":"95b50980-fd5a-4a19-b1cd-13df98d257fe","Type":"ContainerDied","Data":"97782ec29d2e9e3193fca38c78d6d80a51ddf23167b6089a154cabfc32ab7307"} Dec 03 13:11:48 crc kubenswrapper[5052]: I1203 13:11:48.138866 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-operators/glance-operator-controller-manager-77987cd8cd-btn2l"] Dec 03 13:11:48 crc kubenswrapper[5052]: I1203 13:11:48.139181 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-operators/glance-operator-controller-manager-77987cd8cd-btn2l" podUID="d429f7ab-9a88-4411-a7a5-88bc8ab76285" containerName="manager" containerID="cri-o://27a2c0f47320ac8e0e4f631d691ed2469ebf4f27f0dad96adcf0fcb2e224d156" gracePeriod=10 Dec 03 13:11:48 crc kubenswrapper[5052]: I1203 13:11:48.139321 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-operators/glance-operator-controller-manager-77987cd8cd-btn2l" podUID="d429f7ab-9a88-4411-a7a5-88bc8ab76285" containerName="kube-rbac-proxy" containerID="cri-o://cd378be1b828198fb7e23f2fd50570c21a7bd0e958e5f4f92497cb7ebe420226" gracePeriod=10 Dec 03 13:11:48 crc kubenswrapper[5052]: I1203 13:11:48.170821 5052 generic.go:334] "Generic (PLEG): container finished" podID="459f5555-efae-4c19-84d0-2af917b9560f" containerID="9f7f47624da378a4c9e182afc6f0245cb94a2c07414a47ce8f104bcb13883f17" exitCode=0 Dec 03 13:11:48 crc kubenswrapper[5052]: I1203 13:11:48.170942 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/rabbitmq-cluster-operator-manager-668c99d594-ksw2p" event={"ID":"459f5555-efae-4c19-84d0-2af917b9560f","Type":"ContainerDied","Data":"9f7f47624da378a4c9e182afc6f0245cb94a2c07414a47ce8f104bcb13883f17"} Dec 03 13:11:48 crc kubenswrapper[5052]: I1203 13:11:48.180559 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-operators/openstack-baremetal-operator-controller-manager-55d86b6686f2mft"] Dec 03 13:11:48 crc kubenswrapper[5052]: I1203 13:11:48.180801 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-operators/openstack-baremetal-operator-controller-manager-55d86b6686f2mft" podUID="e10da887-70df-43aa-aa79-4a14231e5ec9" containerName="manager" containerID="cri-o://0f961febcff89cbc4c5cd57e2535ba7fe24e5f74a9bf78788782f9c8e2c423d5" gracePeriod=10 Dec 03 13:11:48 crc kubenswrapper[5052]: I1203 13:11:48.180927 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-operators/openstack-baremetal-operator-controller-manager-55d86b6686f2mft" podUID="e10da887-70df-43aa-aa79-4a14231e5ec9" containerName="kube-rbac-proxy" containerID="cri-o://0f00b4f3db71336942e4ffb2969c9f402b3aa40103a5ee65081fd36ca18be57d" gracePeriod=10 Dec 03 13:11:48 crc kubenswrapper[5052]: I1203 13:11:48.244402 5052 generic.go:334] "Generic (PLEG): container finished" podID="2e2dc077-c02f-4f98-8fa6-9f72890faeaf" containerID="c38ccb2125893d9107a9365db7e805b861125cfe34c07ee20dd8158df83fc99c" exitCode=0 Dec 03 13:11:48 crc kubenswrapper[5052]: I1203 13:11:48.244733 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/designate-operator-controller-manager-78b4bc895b-874g2" event={"ID":"2e2dc077-c02f-4f98-8fa6-9f72890faeaf","Type":"ContainerDied","Data":"c38ccb2125893d9107a9365db7e805b861125cfe34c07ee20dd8158df83fc99c"} Dec 03 13:11:48 crc kubenswrapper[5052]: I1203 13:11:48.289967 5052 generic.go:334] "Generic (PLEG): container finished" podID="3e6bc3f6-1f1a-4b99-8835-98252369d364" containerID="a7e939b334d0e92f65588cb0c22e7536f6716a1e32472cd5033b456da2b36a68" exitCode=0 Dec 03 13:11:48 crc kubenswrapper[5052]: I1203 13:11:48.290002 5052 generic.go:334] "Generic (PLEG): container finished" podID="3e6bc3f6-1f1a-4b99-8835-98252369d364" containerID="21bac89f97ac17aa04e4ad5d34fe06b056702b1896eabe8f69dc6ad5ebbc42bf" exitCode=0 Dec 03 13:11:48 crc kubenswrapper[5052]: I1203 13:11:48.290078 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/nova-operator-controller-manager-697bc559fc-gcj5d" event={"ID":"3e6bc3f6-1f1a-4b99-8835-98252369d364","Type":"ContainerDied","Data":"a7e939b334d0e92f65588cb0c22e7536f6716a1e32472cd5033b456da2b36a68"} Dec 03 13:11:48 crc kubenswrapper[5052]: I1203 13:11:48.290110 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/nova-operator-controller-manager-697bc559fc-gcj5d" event={"ID":"3e6bc3f6-1f1a-4b99-8835-98252369d364","Type":"ContainerDied","Data":"21bac89f97ac17aa04e4ad5d34fe06b056702b1896eabe8f69dc6ad5ebbc42bf"} Dec 03 13:11:48 crc kubenswrapper[5052]: I1203 13:11:48.344604 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-operators/openstack-operator-controller-operator-7dd5c7bb7c-rjflp"] Dec 03 13:11:48 crc kubenswrapper[5052]: I1203 13:11:48.344931 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-operators/openstack-operator-controller-operator-7dd5c7bb7c-rjflp" podUID="cd2c5c4c-7ea8-4f95-9556-58b87d14a2be" containerName="operator" containerID="cri-o://5a4c5c6a86ebb9b257e5e311d545f7f9899b5f426c04b2d6f5217d3cce6845e8" gracePeriod=10 Dec 03 13:11:48 crc kubenswrapper[5052]: I1203 13:11:48.406550 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-operators/openstack-operator-index-x6glm"] Dec 03 13:11:48 crc kubenswrapper[5052]: I1203 13:11:48.407064 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-operators/openstack-operator-index-x6glm" podUID="6fe52b82-e00b-4c40-8ba1-bb52e7897667" containerName="registry-server" containerID="cri-o://409bd09ed8d8c06816bb54f36a222437bc97be11de3a87273fc8a642cd7acdb8" gracePeriod=30 Dec 03 13:11:48 crc kubenswrapper[5052]: I1203 13:11:48.414378 5052 generic.go:334] "Generic (PLEG): container finished" podID="f0e70c0d-10fe-43b7-a08c-0ce411f223a6" containerID="7a38fc46fa1de916c4dcdb66fb009b9e798bf1d5b7c4bb72dc6db4bde166741c" exitCode=0 Dec 03 13:11:48 crc kubenswrapper[5052]: I1203 13:11:48.414549 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/mariadb-operator-controller-manager-56bbcc9d85-sr7zw" event={"ID":"f0e70c0d-10fe-43b7-a08c-0ce411f223a6","Type":"ContainerDied","Data":"7a38fc46fa1de916c4dcdb66fb009b9e798bf1d5b7c4bb72dc6db4bde166741c"} Dec 03 13:11:48 crc kubenswrapper[5052]: I1203 13:11:48.461573 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-operators/98dc3bd0b5c63de8bc52e3558b9d3e72fafafb6fd127fd2510d2206864t722l"] Dec 03 13:11:48 crc kubenswrapper[5052]: I1203 13:11:48.464274 5052 generic.go:334] "Generic (PLEG): container finished" podID="7017b3f7-1eb9-4dfe-8ef3-6db27cf3a1f1" containerID="f4171f259274e4008a6e5b98254b34066d3fc61afa68cf052421761cd5affdfb" exitCode=0 Dec 03 13:11:48 crc kubenswrapper[5052]: I1203 13:11:48.464316 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/ironic-operator-controller-manager-6c548fd776-pqfnd" event={"ID":"7017b3f7-1eb9-4dfe-8ef3-6db27cf3a1f1","Type":"ContainerDied","Data":"f4171f259274e4008a6e5b98254b34066d3fc61afa68cf052421761cd5affdfb"} Dec 03 13:11:48 crc kubenswrapper[5052]: I1203 13:11:48.492361 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-operators/98dc3bd0b5c63de8bc52e3558b9d3e72fafafb6fd127fd2510d2206864t722l"] Dec 03 13:11:49 crc kubenswrapper[5052]: I1203 13:11:49.104926 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-operators/openstack-baremetal-operator-controller-manager-55d86b6686f2mft" podUID="e10da887-70df-43aa-aa79-4a14231e5ec9" containerName="manager" probeResult="failure" output="Get \"http://10.217.0.86:8081/readyz\": dial tcp 10.217.0.86:8081: connect: connection refused" Dec 03 13:11:49 crc kubenswrapper[5052]: E1203 13:11:49.155319 5052 cadvisor_stats_provider.go:516] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0e70c0d_10fe_43b7_a08c_0ce411f223a6.slice/crio-conmon-7a38fc46fa1de916c4dcdb66fb009b9e798bf1d5b7c4bb72dc6db4bde166741c.scope\": RecentStats: unable to find data in memory cache]" Dec 03 13:11:49 crc kubenswrapper[5052]: I1203 13:11:49.437267 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-operators/manila-operator-controller-manager-7c79b5df47-ccbqq" Dec 03 13:11:49 crc kubenswrapper[5052]: I1203 13:11:49.447637 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-operators/octavia-operator-controller-manager-998648c74-z6cw9" Dec 03 13:11:49 crc kubenswrapper[5052]: I1203 13:11:49.459492 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-operators/keystone-operator-controller-manager-7765d96ddf-nlrl9" Dec 03 13:11:49 crc kubenswrapper[5052]: I1203 13:11:49.480555 5052 generic.go:334] "Generic (PLEG): container finished" podID="cd2c5c4c-7ea8-4f95-9556-58b87d14a2be" containerID="5a4c5c6a86ebb9b257e5e311d545f7f9899b5f426c04b2d6f5217d3cce6845e8" exitCode=0 Dec 03 13:11:49 crc kubenswrapper[5052]: I1203 13:11:49.480634 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/openstack-operator-controller-operator-7dd5c7bb7c-rjflp" event={"ID":"cd2c5c4c-7ea8-4f95-9556-58b87d14a2be","Type":"ContainerDied","Data":"5a4c5c6a86ebb9b257e5e311d545f7f9899b5f426c04b2d6f5217d3cce6845e8"} Dec 03 13:11:49 crc kubenswrapper[5052]: I1203 13:11:49.480644 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-operators/neutron-operator-controller-manager-5fdfd5b6b5-q48nd" Dec 03 13:11:49 crc kubenswrapper[5052]: E1203 13:11:49.483645 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of 409bd09ed8d8c06816bb54f36a222437bc97be11de3a87273fc8a642cd7acdb8 is running failed: container process not found" containerID="409bd09ed8d8c06816bb54f36a222437bc97be11de3a87273fc8a642cd7acdb8" cmd=["grpc_health_probe","-addr=:50051"] Dec 03 13:11:49 crc kubenswrapper[5052]: E1203 13:11:49.486785 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of 409bd09ed8d8c06816bb54f36a222437bc97be11de3a87273fc8a642cd7acdb8 is running failed: container process not found" containerID="409bd09ed8d8c06816bb54f36a222437bc97be11de3a87273fc8a642cd7acdb8" cmd=["grpc_health_probe","-addr=:50051"] Dec 03 13:11:49 crc kubenswrapper[5052]: I1203 13:11:49.488888 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/octavia-operator-controller-manager-998648c74-z6cw9" event={"ID":"50c6e918-7f1f-4d6f-ba5f-940e5f485825","Type":"ContainerDied","Data":"3365351e01a48c03cdca7119bd0ecf92f9b4476a8f54c2cb8418e4e17d437293"} Dec 03 13:11:49 crc kubenswrapper[5052]: I1203 13:11:49.488934 5052 scope.go:117] "RemoveContainer" containerID="c55bdca1c305eb9768bdfbe023291651a872e6e10fe50e1b32182c071f21b870" Dec 03 13:11:49 crc kubenswrapper[5052]: I1203 13:11:49.489007 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-operators/octavia-operator-controller-manager-998648c74-z6cw9" Dec 03 13:11:49 crc kubenswrapper[5052]: E1203 13:11:49.488882 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of 409bd09ed8d8c06816bb54f36a222437bc97be11de3a87273fc8a642cd7acdb8 is running failed: container process not found" containerID="409bd09ed8d8c06816bb54f36a222437bc97be11de3a87273fc8a642cd7acdb8" cmd=["grpc_health_probe","-addr=:50051"] Dec 03 13:11:49 crc kubenswrapper[5052]: E1203 13:11:49.489395 5052 prober.go:104] "Probe errored" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of 409bd09ed8d8c06816bb54f36a222437bc97be11de3a87273fc8a642cd7acdb8 is running failed: container process not found" probeType="Readiness" pod="openstack-operators/openstack-operator-index-x6glm" podUID="6fe52b82-e00b-4c40-8ba1-bb52e7897667" containerName="registry-server" Dec 03 13:11:49 crc kubenswrapper[5052]: I1203 13:11:49.490049 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-operators/designate-operator-controller-manager-78b4bc895b-874g2" Dec 03 13:11:49 crc kubenswrapper[5052]: I1203 13:11:49.499096 5052 generic.go:334] "Generic (PLEG): container finished" podID="6bea5f42-82da-4148-9996-dc05121212f7" containerID="7145c8d45908da236459c9313b6926c516ea98b7ba253967ef2007aca4397a66" exitCode=0 Dec 03 13:11:49 crc kubenswrapper[5052]: I1203 13:11:49.499134 5052 generic.go:334] "Generic (PLEG): container finished" podID="6bea5f42-82da-4148-9996-dc05121212f7" containerID="af5bcb24f3ea207a68cadd9479e8bc16e424b9d9fac437c1e5ea1ef07af38d6d" exitCode=0 Dec 03 13:11:49 crc kubenswrapper[5052]: I1203 13:11:49.499184 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/watcher-operator-controller-manager-769dc69bc-7rhg6" event={"ID":"6bea5f42-82da-4148-9996-dc05121212f7","Type":"ContainerDied","Data":"7145c8d45908da236459c9313b6926c516ea98b7ba253967ef2007aca4397a66"} Dec 03 13:11:49 crc kubenswrapper[5052]: I1203 13:11:49.499213 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/watcher-operator-controller-manager-769dc69bc-7rhg6" event={"ID":"6bea5f42-82da-4148-9996-dc05121212f7","Type":"ContainerDied","Data":"af5bcb24f3ea207a68cadd9479e8bc16e424b9d9fac437c1e5ea1ef07af38d6d"} Dec 03 13:11:49 crc kubenswrapper[5052]: I1203 13:11:49.501879 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-operators/ironic-operator-controller-manager-6c548fd776-pqfnd" Dec 03 13:11:49 crc kubenswrapper[5052]: I1203 13:11:49.502277 5052 generic.go:334] "Generic (PLEG): container finished" podID="37ab421b-0ad3-4d9d-bf47-8bf908c4aa57" containerID="2dcc207b577f508a2a51aeed06eda37f7a37cdaa339bc4bd5b4f2712d43ec227" exitCode=0 Dec 03 13:11:49 crc kubenswrapper[5052]: I1203 13:11:49.502305 5052 generic.go:334] "Generic (PLEG): container finished" podID="37ab421b-0ad3-4d9d-bf47-8bf908c4aa57" containerID="e3943739460d5bb48bf27023494d0f23c5ff3cd0f58f04e59d49d0dbc05a7830" exitCode=0 Dec 03 13:11:49 crc kubenswrapper[5052]: I1203 13:11:49.502356 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/placement-operator-controller-manager-78f8948974-j62hd" event={"ID":"37ab421b-0ad3-4d9d-bf47-8bf908c4aa57","Type":"ContainerDied","Data":"2dcc207b577f508a2a51aeed06eda37f7a37cdaa339bc4bd5b4f2712d43ec227"} Dec 03 13:11:49 crc kubenswrapper[5052]: I1203 13:11:49.502381 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/placement-operator-controller-manager-78f8948974-j62hd" event={"ID":"37ab421b-0ad3-4d9d-bf47-8bf908c4aa57","Type":"ContainerDied","Data":"e3943739460d5bb48bf27023494d0f23c5ff3cd0f58f04e59d49d0dbc05a7830"} Dec 03 13:11:49 crc kubenswrapper[5052]: I1203 13:11:49.506862 5052 generic.go:334] "Generic (PLEG): container finished" podID="8dbd25a1-9de9-431e-be37-d239e3f64728" containerID="754a05a1ad89a85b548ec6034299b8041b571878c20c15a1d29f8a87a0d19577" exitCode=0 Dec 03 13:11:49 crc kubenswrapper[5052]: I1203 13:11:49.506895 5052 generic.go:334] "Generic (PLEG): container finished" podID="8dbd25a1-9de9-431e-be37-d239e3f64728" containerID="5025f6eb197bd7384ff4dad6b52992f6489ac18de5c05c2535a9026ee37b6f24" exitCode=0 Dec 03 13:11:49 crc kubenswrapper[5052]: I1203 13:11:49.506938 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/barbican-operator-controller-manager-7d9dfd778-cqfdk" event={"ID":"8dbd25a1-9de9-431e-be37-d239e3f64728","Type":"ContainerDied","Data":"754a05a1ad89a85b548ec6034299b8041b571878c20c15a1d29f8a87a0d19577"} Dec 03 13:11:49 crc kubenswrapper[5052]: I1203 13:11:49.506964 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/barbican-operator-controller-manager-7d9dfd778-cqfdk" event={"ID":"8dbd25a1-9de9-431e-be37-d239e3f64728","Type":"ContainerDied","Data":"5025f6eb197bd7384ff4dad6b52992f6489ac18de5c05c2535a9026ee37b6f24"} Dec 03 13:11:49 crc kubenswrapper[5052]: I1203 13:11:49.508826 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-operators/horizon-operator-controller-manager-68c6d99b8f-mjdvh" Dec 03 13:11:49 crc kubenswrapper[5052]: I1203 13:11:49.509273 5052 generic.go:334] "Generic (PLEG): container finished" podID="5968d273-33dd-45cc-aab0-e83628bcd15c" containerID="cd51be9345abb5eafe858033c68a1ced7d4100915afaf161d165bcac5ef6661e" exitCode=0 Dec 03 13:11:49 crc kubenswrapper[5052]: I1203 13:11:49.509293 5052 generic.go:334] "Generic (PLEG): container finished" podID="5968d273-33dd-45cc-aab0-e83628bcd15c" containerID="e020c88c64940fc2399454d010fff5531ddc1320fccf47d9835db05c4424f1b2" exitCode=0 Dec 03 13:11:49 crc kubenswrapper[5052]: I1203 13:11:49.509352 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/cinder-operator-controller-manager-859b6ccc6-vt9d2" event={"ID":"5968d273-33dd-45cc-aab0-e83628bcd15c","Type":"ContainerDied","Data":"cd51be9345abb5eafe858033c68a1ced7d4100915afaf161d165bcac5ef6661e"} Dec 03 13:11:49 crc kubenswrapper[5052]: I1203 13:11:49.509374 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/cinder-operator-controller-manager-859b6ccc6-vt9d2" event={"ID":"5968d273-33dd-45cc-aab0-e83628bcd15c","Type":"ContainerDied","Data":"e020c88c64940fc2399454d010fff5531ddc1320fccf47d9835db05c4424f1b2"} Dec 03 13:11:49 crc kubenswrapper[5052]: I1203 13:11:49.515697 5052 generic.go:334] "Generic (PLEG): container finished" podID="527bcd50-6d8d-4d4f-bc11-5382eb9adfc2" containerID="d1bb936db1c98c58f4e109008a5dc7d293c9a04b310fd4b2ab5bb2e70449c5aa" exitCode=0 Dec 03 13:11:49 crc kubenswrapper[5052]: I1203 13:11:49.515792 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/openstack-operator-controller-manager-9f56fc979-b8jv8" event={"ID":"527bcd50-6d8d-4d4f-bc11-5382eb9adfc2","Type":"ContainerDied","Data":"d1bb936db1c98c58f4e109008a5dc7d293c9a04b310fd4b2ab5bb2e70449c5aa"} Dec 03 13:11:49 crc kubenswrapper[5052]: I1203 13:11:49.518115 5052 generic.go:334] "Generic (PLEG): container finished" podID="7017b3f7-1eb9-4dfe-8ef3-6db27cf3a1f1" containerID="acbb799df7a125d48309f76c7ac211fd06dfd6580980ca517793b3d574553f2e" exitCode=0 Dec 03 13:11:49 crc kubenswrapper[5052]: I1203 13:11:49.518173 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/ironic-operator-controller-manager-6c548fd776-pqfnd" event={"ID":"7017b3f7-1eb9-4dfe-8ef3-6db27cf3a1f1","Type":"ContainerDied","Data":"acbb799df7a125d48309f76c7ac211fd06dfd6580980ca517793b3d574553f2e"} Dec 03 13:11:49 crc kubenswrapper[5052]: I1203 13:11:49.518202 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/ironic-operator-controller-manager-6c548fd776-pqfnd" event={"ID":"7017b3f7-1eb9-4dfe-8ef3-6db27cf3a1f1","Type":"ContainerDied","Data":"30b3e66e0e29044a9783b97379a11cf5b95d8e7a61c3e8f54beb16a7be1e4d3b"} Dec 03 13:11:49 crc kubenswrapper[5052]: I1203 13:11:49.518264 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-operators/ironic-operator-controller-manager-6c548fd776-pqfnd" Dec 03 13:11:49 crc kubenswrapper[5052]: I1203 13:11:49.519944 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/telemetry-operator-controller-manager-76cc84c6bb-zl7px" event={"ID":"3894016b-8242-4b12-8407-bcf78af00851","Type":"ContainerDied","Data":"0f99870dc82861c793dbad4149f80170c0c9dc2378102d4f80a980b605b8485f"} Dec 03 13:11:49 crc kubenswrapper[5052]: I1203 13:11:49.519977 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0f99870dc82861c793dbad4149f80170c0c9dc2378102d4f80a980b605b8485f" Dec 03 13:11:49 crc kubenswrapper[5052]: I1203 13:11:49.520593 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-operators/swift-operator-controller-manager-5f8c65bbfc-2cgdz" Dec 03 13:11:49 crc kubenswrapper[5052]: I1203 13:11:49.521187 5052 generic.go:334] "Generic (PLEG): container finished" podID="d429f7ab-9a88-4411-a7a5-88bc8ab76285" containerID="cd378be1b828198fb7e23f2fd50570c21a7bd0e958e5f4f92497cb7ebe420226" exitCode=0 Dec 03 13:11:49 crc kubenswrapper[5052]: I1203 13:11:49.521205 5052 generic.go:334] "Generic (PLEG): container finished" podID="d429f7ab-9a88-4411-a7a5-88bc8ab76285" containerID="27a2c0f47320ac8e0e4f631d691ed2469ebf4f27f0dad96adcf0fcb2e224d156" exitCode=0 Dec 03 13:11:49 crc kubenswrapper[5052]: I1203 13:11:49.521259 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/glance-operator-controller-manager-77987cd8cd-btn2l" event={"ID":"d429f7ab-9a88-4411-a7a5-88bc8ab76285","Type":"ContainerDied","Data":"cd378be1b828198fb7e23f2fd50570c21a7bd0e958e5f4f92497cb7ebe420226"} Dec 03 13:11:49 crc kubenswrapper[5052]: I1203 13:11:49.521279 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/glance-operator-controller-manager-77987cd8cd-btn2l" event={"ID":"d429f7ab-9a88-4411-a7a5-88bc8ab76285","Type":"ContainerDied","Data":"27a2c0f47320ac8e0e4f631d691ed2469ebf4f27f0dad96adcf0fcb2e224d156"} Dec 03 13:11:49 crc kubenswrapper[5052]: I1203 13:11:49.522494 5052 generic.go:334] "Generic (PLEG): container finished" podID="a605d632-d9a5-4ce0-9449-e3d57105df1b" containerID="6e810659ae074c724982667dc34e29af08df25d8aa94eea774ad9e2b2ba3b241" exitCode=0 Dec 03 13:11:49 crc kubenswrapper[5052]: I1203 13:11:49.522536 5052 generic.go:334] "Generic (PLEG): container finished" podID="a605d632-d9a5-4ce0-9449-e3d57105df1b" containerID="2944fe91598c6c5f038d1de9c8551c8352a4c6b145016c6151bfc37b330edddd" exitCode=0 Dec 03 13:11:49 crc kubenswrapper[5052]: I1203 13:11:49.522589 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/ovn-operator-controller-manager-b6456fdb6-sggpb" event={"ID":"a605d632-d9a5-4ce0-9449-e3d57105df1b","Type":"ContainerDied","Data":"6e810659ae074c724982667dc34e29af08df25d8aa94eea774ad9e2b2ba3b241"} Dec 03 13:11:49 crc kubenswrapper[5052]: I1203 13:11:49.522609 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/ovn-operator-controller-manager-b6456fdb6-sggpb" event={"ID":"a605d632-d9a5-4ce0-9449-e3d57105df1b","Type":"ContainerDied","Data":"2944fe91598c6c5f038d1de9c8551c8352a4c6b145016c6151bfc37b330edddd"} Dec 03 13:11:49 crc kubenswrapper[5052]: I1203 13:11:49.523795 5052 generic.go:334] "Generic (PLEG): container finished" podID="81f5f8e1-015a-4b45-83e6-13e88efb8e80" containerID="01a4046ea27a459acec0b9a6a2a20c352ddcb26e0f79bddb80874932462702e9" exitCode=0 Dec 03 13:11:49 crc kubenswrapper[5052]: I1203 13:11:49.523812 5052 generic.go:334] "Generic (PLEG): container finished" podID="81f5f8e1-015a-4b45-83e6-13e88efb8e80" containerID="43ae682827f3b8fe92dd61ac021bde418f76e0a34a72497c150c64083536410d" exitCode=0 Dec 03 13:11:49 crc kubenswrapper[5052]: I1203 13:11:49.523842 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/infra-operator-controller-manager-57548d458d-9fkll" event={"ID":"81f5f8e1-015a-4b45-83e6-13e88efb8e80","Type":"ContainerDied","Data":"01a4046ea27a459acec0b9a6a2a20c352ddcb26e0f79bddb80874932462702e9"} Dec 03 13:11:49 crc kubenswrapper[5052]: I1203 13:11:49.523859 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/infra-operator-controller-manager-57548d458d-9fkll" event={"ID":"81f5f8e1-015a-4b45-83e6-13e88efb8e80","Type":"ContainerDied","Data":"43ae682827f3b8fe92dd61ac021bde418f76e0a34a72497c150c64083536410d"} Dec 03 13:11:49 crc kubenswrapper[5052]: I1203 13:11:49.525252 5052 generic.go:334] "Generic (PLEG): container finished" podID="ea1e91de-7e7d-498a-a17b-551f2bee325a" containerID="c8a22e6613869e6f1219b539f8d24e9e232f41001d8d1d96f9c514c62fd7214c" exitCode=0 Dec 03 13:11:49 crc kubenswrapper[5052]: I1203 13:11:49.525278 5052 generic.go:334] "Generic (PLEG): container finished" podID="ea1e91de-7e7d-498a-a17b-551f2bee325a" containerID="0add256c4e87d7b91cc63837e4958d34bfbf4128a08af08316deec77be2a75eb" exitCode=0 Dec 03 13:11:49 crc kubenswrapper[5052]: I1203 13:11:49.525313 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/manila-operator-controller-manager-7c79b5df47-ccbqq" event={"ID":"ea1e91de-7e7d-498a-a17b-551f2bee325a","Type":"ContainerDied","Data":"c8a22e6613869e6f1219b539f8d24e9e232f41001d8d1d96f9c514c62fd7214c"} Dec 03 13:11:49 crc kubenswrapper[5052]: I1203 13:11:49.525333 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/manila-operator-controller-manager-7c79b5df47-ccbqq" event={"ID":"ea1e91de-7e7d-498a-a17b-551f2bee325a","Type":"ContainerDied","Data":"0add256c4e87d7b91cc63837e4958d34bfbf4128a08af08316deec77be2a75eb"} Dec 03 13:11:49 crc kubenswrapper[5052]: I1203 13:11:49.525346 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/manila-operator-controller-manager-7c79b5df47-ccbqq" event={"ID":"ea1e91de-7e7d-498a-a17b-551f2bee325a","Type":"ContainerDied","Data":"de9266c7e68b4ad9a6760a4efe0699ed57cc7eaba3f772d4b1d0f34a3589e5a0"} Dec 03 13:11:49 crc kubenswrapper[5052]: I1203 13:11:49.525405 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-operators/manila-operator-controller-manager-7c79b5df47-ccbqq" Dec 03 13:11:49 crc kubenswrapper[5052]: I1203 13:11:49.530653 5052 generic.go:334] "Generic (PLEG): container finished" podID="1cb3d739-e4d6-4058-9456-a3c2e1c09130" containerID="dabaf7e10aa7d0b0ec06b6b8e0023ae509392bd5567d994a850b73988ff68ec4" exitCode=0 Dec 03 13:11:49 crc kubenswrapper[5052]: I1203 13:11:49.530685 5052 generic.go:334] "Generic (PLEG): container finished" podID="1cb3d739-e4d6-4058-9456-a3c2e1c09130" containerID="7f760d28d8a9a27373a0d324cb077f28fc6b0310ae54ade63b76d7f19979ccad" exitCode=0 Dec 03 13:11:49 crc kubenswrapper[5052]: I1203 13:11:49.530733 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/test-operator-controller-manager-5854674fcc-jltb6" event={"ID":"1cb3d739-e4d6-4058-9456-a3c2e1c09130","Type":"ContainerDied","Data":"dabaf7e10aa7d0b0ec06b6b8e0023ae509392bd5567d994a850b73988ff68ec4"} Dec 03 13:11:49 crc kubenswrapper[5052]: I1203 13:11:49.530766 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/test-operator-controller-manager-5854674fcc-jltb6" event={"ID":"1cb3d739-e4d6-4058-9456-a3c2e1c09130","Type":"ContainerDied","Data":"7f760d28d8a9a27373a0d324cb077f28fc6b0310ae54ade63b76d7f19979ccad"} Dec 03 13:11:49 crc kubenswrapper[5052]: I1203 13:11:49.542760 5052 scope.go:117] "RemoveContainer" containerID="829ed8e10d7aad4792ee147f6891ccdb7a056f605c04a4c0349d94bfa248bff9" Dec 03 13:11:49 crc kubenswrapper[5052]: I1203 13:11:49.546963 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f44vw\" (UniqueName: \"kubernetes.io/projected/50c6e918-7f1f-4d6f-ba5f-940e5f485825-kube-api-access-f44vw\") pod \"50c6e918-7f1f-4d6f-ba5f-940e5f485825\" (UID: \"50c6e918-7f1f-4d6f-ba5f-940e5f485825\") " Dec 03 13:11:49 crc kubenswrapper[5052]: I1203 13:11:49.547045 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7shls\" (UniqueName: \"kubernetes.io/projected/ce3d34f6-e545-4eb2-bcc5-999377931601-kube-api-access-7shls\") pod \"ce3d34f6-e545-4eb2-bcc5-999377931601\" (UID: \"ce3d34f6-e545-4eb2-bcc5-999377931601\") " Dec 03 13:11:49 crc kubenswrapper[5052]: I1203 13:11:49.547083 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j8lx5\" (UniqueName: \"kubernetes.io/projected/ea1e91de-7e7d-498a-a17b-551f2bee325a-kube-api-access-j8lx5\") pod \"ea1e91de-7e7d-498a-a17b-551f2bee325a\" (UID: \"ea1e91de-7e7d-498a-a17b-551f2bee325a\") " Dec 03 13:11:49 crc kubenswrapper[5052]: I1203 13:11:49.547098 5052 generic.go:334] "Generic (PLEG): container finished" podID="f0e70c0d-10fe-43b7-a08c-0ce411f223a6" containerID="c16b4916d85027f28775414e1e9e061796b804441bf4aded0b2a1af11134a874" exitCode=0 Dec 03 13:11:49 crc kubenswrapper[5052]: I1203 13:11:49.547145 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/mariadb-operator-controller-manager-56bbcc9d85-sr7zw" event={"ID":"f0e70c0d-10fe-43b7-a08c-0ce411f223a6","Type":"ContainerDied","Data":"c16b4916d85027f28775414e1e9e061796b804441bf4aded0b2a1af11134a874"} Dec 03 13:11:49 crc kubenswrapper[5052]: I1203 13:11:49.547158 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h6smm\" (UniqueName: \"kubernetes.io/projected/7017b3f7-1eb9-4dfe-8ef3-6db27cf3a1f1-kube-api-access-h6smm\") pod \"7017b3f7-1eb9-4dfe-8ef3-6db27cf3a1f1\" (UID: \"7017b3f7-1eb9-4dfe-8ef3-6db27cf3a1f1\") " Dec 03 13:11:49 crc kubenswrapper[5052]: I1203 13:11:49.547171 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/mariadb-operator-controller-manager-56bbcc9d85-sr7zw" event={"ID":"f0e70c0d-10fe-43b7-a08c-0ce411f223a6","Type":"ContainerDied","Data":"63b967be8542405d76ece84234f3c81710dfda97bca61633bc0f8bb8e114ed85"} Dec 03 13:11:49 crc kubenswrapper[5052]: I1203 13:11:49.547185 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="63b967be8542405d76ece84234f3c81710dfda97bca61633bc0f8bb8e114ed85" Dec 03 13:11:49 crc kubenswrapper[5052]: I1203 13:11:49.547195 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j2rjq\" (UniqueName: \"kubernetes.io/projected/53c88ca3-39f7-4a81-b305-40047bcaeb88-kube-api-access-j2rjq\") pod \"53c88ca3-39f7-4a81-b305-40047bcaeb88\" (UID: \"53c88ca3-39f7-4a81-b305-40047bcaeb88\") " Dec 03 13:11:49 crc kubenswrapper[5052]: I1203 13:11:49.547232 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vpllc\" (UniqueName: \"kubernetes.io/projected/8fd08931-7f0b-4d5b-bfb1-f27980648561-kube-api-access-vpllc\") pod \"8fd08931-7f0b-4d5b-bfb1-f27980648561\" (UID: \"8fd08931-7f0b-4d5b-bfb1-f27980648561\") " Dec 03 13:11:49 crc kubenswrapper[5052]: I1203 13:11:49.547257 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sw44h\" (UniqueName: \"kubernetes.io/projected/95b50980-fd5a-4a19-b1cd-13df98d257fe-kube-api-access-sw44h\") pod \"95b50980-fd5a-4a19-b1cd-13df98d257fe\" (UID: \"95b50980-fd5a-4a19-b1cd-13df98d257fe\") " Dec 03 13:11:49 crc kubenswrapper[5052]: I1203 13:11:49.547304 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qtl9v\" (UniqueName: \"kubernetes.io/projected/2e2dc077-c02f-4f98-8fa6-9f72890faeaf-kube-api-access-qtl9v\") pod \"2e2dc077-c02f-4f98-8fa6-9f72890faeaf\" (UID: \"2e2dc077-c02f-4f98-8fa6-9f72890faeaf\") " Dec 03 13:11:49 crc kubenswrapper[5052]: I1203 13:11:49.557310 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/50c6e918-7f1f-4d6f-ba5f-940e5f485825-kube-api-access-f44vw" (OuterVolumeSpecName: "kube-api-access-f44vw") pod "50c6e918-7f1f-4d6f-ba5f-940e5f485825" (UID: "50c6e918-7f1f-4d6f-ba5f-940e5f485825"). InnerVolumeSpecName "kube-api-access-f44vw". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:11:49 crc kubenswrapper[5052]: I1203 13:11:49.560475 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/95b50980-fd5a-4a19-b1cd-13df98d257fe-kube-api-access-sw44h" (OuterVolumeSpecName: "kube-api-access-sw44h") pod "95b50980-fd5a-4a19-b1cd-13df98d257fe" (UID: "95b50980-fd5a-4a19-b1cd-13df98d257fe"). InnerVolumeSpecName "kube-api-access-sw44h". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:11:49 crc kubenswrapper[5052]: I1203 13:11:49.561263 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-operators/mariadb-operator-controller-manager-56bbcc9d85-sr7zw" Dec 03 13:11:49 crc kubenswrapper[5052]: I1203 13:11:49.562759 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/53c88ca3-39f7-4a81-b305-40047bcaeb88-kube-api-access-j2rjq" (OuterVolumeSpecName: "kube-api-access-j2rjq") pod "53c88ca3-39f7-4a81-b305-40047bcaeb88" (UID: "53c88ca3-39f7-4a81-b305-40047bcaeb88"). InnerVolumeSpecName "kube-api-access-j2rjq". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:11:49 crc kubenswrapper[5052]: I1203 13:11:49.569052 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2e2dc077-c02f-4f98-8fa6-9f72890faeaf-kube-api-access-qtl9v" (OuterVolumeSpecName: "kube-api-access-qtl9v") pod "2e2dc077-c02f-4f98-8fa6-9f72890faeaf" (UID: "2e2dc077-c02f-4f98-8fa6-9f72890faeaf"). InnerVolumeSpecName "kube-api-access-qtl9v". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:11:49 crc kubenswrapper[5052]: I1203 13:11:49.569934 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ce3d34f6-e545-4eb2-bcc5-999377931601-kube-api-access-7shls" (OuterVolumeSpecName: "kube-api-access-7shls") pod "ce3d34f6-e545-4eb2-bcc5-999377931601" (UID: "ce3d34f6-e545-4eb2-bcc5-999377931601"). InnerVolumeSpecName "kube-api-access-7shls". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:11:49 crc kubenswrapper[5052]: I1203 13:11:49.571493 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7017b3f7-1eb9-4dfe-8ef3-6db27cf3a1f1-kube-api-access-h6smm" (OuterVolumeSpecName: "kube-api-access-h6smm") pod "7017b3f7-1eb9-4dfe-8ef3-6db27cf3a1f1" (UID: "7017b3f7-1eb9-4dfe-8ef3-6db27cf3a1f1"). InnerVolumeSpecName "kube-api-access-h6smm". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:11:49 crc kubenswrapper[5052]: I1203 13:11:49.573111 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8fd08931-7f0b-4d5b-bfb1-f27980648561-kube-api-access-vpllc" (OuterVolumeSpecName: "kube-api-access-vpllc") pod "8fd08931-7f0b-4d5b-bfb1-f27980648561" (UID: "8fd08931-7f0b-4d5b-bfb1-f27980648561"). InnerVolumeSpecName "kube-api-access-vpllc". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:11:49 crc kubenswrapper[5052]: I1203 13:11:49.574017 5052 generic.go:334] "Generic (PLEG): container finished" podID="53c88ca3-39f7-4a81-b305-40047bcaeb88" containerID="efce01fd34c9aaa82b7beeb80cae30909d6a58a957f233a93543d43ab86d6552" exitCode=0 Dec 03 13:11:49 crc kubenswrapper[5052]: I1203 13:11:49.574081 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/neutron-operator-controller-manager-5fdfd5b6b5-q48nd" event={"ID":"53c88ca3-39f7-4a81-b305-40047bcaeb88","Type":"ContainerDied","Data":"efce01fd34c9aaa82b7beeb80cae30909d6a58a957f233a93543d43ab86d6552"} Dec 03 13:11:49 crc kubenswrapper[5052]: I1203 13:11:49.574110 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/neutron-operator-controller-manager-5fdfd5b6b5-q48nd" event={"ID":"53c88ca3-39f7-4a81-b305-40047bcaeb88","Type":"ContainerDied","Data":"40332f3aff533656bb056f489cc49291efeba64208111acbf8513a4f9ecbfd55"} Dec 03 13:11:49 crc kubenswrapper[5052]: I1203 13:11:49.574162 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-operators/neutron-operator-controller-manager-5fdfd5b6b5-q48nd" Dec 03 13:11:49 crc kubenswrapper[5052]: I1203 13:11:49.574399 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ea1e91de-7e7d-498a-a17b-551f2bee325a-kube-api-access-j8lx5" (OuterVolumeSpecName: "kube-api-access-j8lx5") pod "ea1e91de-7e7d-498a-a17b-551f2bee325a" (UID: "ea1e91de-7e7d-498a-a17b-551f2bee325a"). InnerVolumeSpecName "kube-api-access-j8lx5". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:11:49 crc kubenswrapper[5052]: I1203 13:11:49.580222 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-operators/telemetry-operator-controller-manager-76cc84c6bb-zl7px" Dec 03 13:11:49 crc kubenswrapper[5052]: I1203 13:11:49.609330 5052 generic.go:334] "Generic (PLEG): container finished" podID="95b50980-fd5a-4a19-b1cd-13df98d257fe" containerID="61b7e037d8a6c427bda841efbbb994e108ee2daac74c44aca5542d608eed3910" exitCode=0 Dec 03 13:11:49 crc kubenswrapper[5052]: I1203 13:11:49.609646 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/keystone-operator-controller-manager-7765d96ddf-nlrl9" event={"ID":"95b50980-fd5a-4a19-b1cd-13df98d257fe","Type":"ContainerDied","Data":"61b7e037d8a6c427bda841efbbb994e108ee2daac74c44aca5542d608eed3910"} Dec 03 13:11:49 crc kubenswrapper[5052]: I1203 13:11:49.609741 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/keystone-operator-controller-manager-7765d96ddf-nlrl9" event={"ID":"95b50980-fd5a-4a19-b1cd-13df98d257fe","Type":"ContainerDied","Data":"4e2b8f936690d204398735706497822d2ac5c4c0d87f2b484844cdaa49396523"} Dec 03 13:11:49 crc kubenswrapper[5052]: I1203 13:11:49.609874 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-operators/keystone-operator-controller-manager-7765d96ddf-nlrl9" Dec 03 13:11:49 crc kubenswrapper[5052]: I1203 13:11:49.634195 5052 scope.go:117] "RemoveContainer" containerID="f4171f259274e4008a6e5b98254b34066d3fc61afa68cf052421761cd5affdfb" Dec 03 13:11:49 crc kubenswrapper[5052]: I1203 13:11:49.650785 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pmdm6\" (UniqueName: \"kubernetes.io/projected/3894016b-8242-4b12-8407-bcf78af00851-kube-api-access-pmdm6\") pod \"3894016b-8242-4b12-8407-bcf78af00851\" (UID: \"3894016b-8242-4b12-8407-bcf78af00851\") " Dec 03 13:11:49 crc kubenswrapper[5052]: I1203 13:11:49.650973 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cpczm\" (UniqueName: \"kubernetes.io/projected/f0e70c0d-10fe-43b7-a08c-0ce411f223a6-kube-api-access-cpczm\") pod \"f0e70c0d-10fe-43b7-a08c-0ce411f223a6\" (UID: \"f0e70c0d-10fe-43b7-a08c-0ce411f223a6\") " Dec 03 13:11:49 crc kubenswrapper[5052]: I1203 13:11:49.651238 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-7shls\" (UniqueName: \"kubernetes.io/projected/ce3d34f6-e545-4eb2-bcc5-999377931601-kube-api-access-7shls\") on node \"crc\" DevicePath \"\"" Dec 03 13:11:49 crc kubenswrapper[5052]: I1203 13:11:49.651250 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-j8lx5\" (UniqueName: \"kubernetes.io/projected/ea1e91de-7e7d-498a-a17b-551f2bee325a-kube-api-access-j8lx5\") on node \"crc\" DevicePath \"\"" Dec 03 13:11:49 crc kubenswrapper[5052]: I1203 13:11:49.651260 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-h6smm\" (UniqueName: \"kubernetes.io/projected/7017b3f7-1eb9-4dfe-8ef3-6db27cf3a1f1-kube-api-access-h6smm\") on node \"crc\" DevicePath \"\"" Dec 03 13:11:49 crc kubenswrapper[5052]: I1203 13:11:49.651269 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-j2rjq\" (UniqueName: \"kubernetes.io/projected/53c88ca3-39f7-4a81-b305-40047bcaeb88-kube-api-access-j2rjq\") on node \"crc\" DevicePath \"\"" Dec 03 13:11:49 crc kubenswrapper[5052]: I1203 13:11:49.651279 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-vpllc\" (UniqueName: \"kubernetes.io/projected/8fd08931-7f0b-4d5b-bfb1-f27980648561-kube-api-access-vpllc\") on node \"crc\" DevicePath \"\"" Dec 03 13:11:49 crc kubenswrapper[5052]: I1203 13:11:49.651288 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-sw44h\" (UniqueName: \"kubernetes.io/projected/95b50980-fd5a-4a19-b1cd-13df98d257fe-kube-api-access-sw44h\") on node \"crc\" DevicePath \"\"" Dec 03 13:11:49 crc kubenswrapper[5052]: I1203 13:11:49.651297 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-qtl9v\" (UniqueName: \"kubernetes.io/projected/2e2dc077-c02f-4f98-8fa6-9f72890faeaf-kube-api-access-qtl9v\") on node \"crc\" DevicePath \"\"" Dec 03 13:11:49 crc kubenswrapper[5052]: I1203 13:11:49.651307 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-f44vw\" (UniqueName: \"kubernetes.io/projected/50c6e918-7f1f-4d6f-ba5f-940e5f485825-kube-api-access-f44vw\") on node \"crc\" DevicePath \"\"" Dec 03 13:11:49 crc kubenswrapper[5052]: I1203 13:11:49.654011 5052 generic.go:334] "Generic (PLEG): container finished" podID="e10da887-70df-43aa-aa79-4a14231e5ec9" containerID="0f00b4f3db71336942e4ffb2969c9f402b3aa40103a5ee65081fd36ca18be57d" exitCode=0 Dec 03 13:11:49 crc kubenswrapper[5052]: I1203 13:11:49.654044 5052 generic.go:334] "Generic (PLEG): container finished" podID="e10da887-70df-43aa-aa79-4a14231e5ec9" containerID="0f961febcff89cbc4c5cd57e2535ba7fe24e5f74a9bf78788782f9c8e2c423d5" exitCode=0 Dec 03 13:11:49 crc kubenswrapper[5052]: I1203 13:11:49.654126 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/openstack-baremetal-operator-controller-manager-55d86b6686f2mft" event={"ID":"e10da887-70df-43aa-aa79-4a14231e5ec9","Type":"ContainerDied","Data":"0f00b4f3db71336942e4ffb2969c9f402b3aa40103a5ee65081fd36ca18be57d"} Dec 03 13:11:49 crc kubenswrapper[5052]: I1203 13:11:49.654155 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/openstack-baremetal-operator-controller-manager-55d86b6686f2mft" event={"ID":"e10da887-70df-43aa-aa79-4a14231e5ec9","Type":"ContainerDied","Data":"0f961febcff89cbc4c5cd57e2535ba7fe24e5f74a9bf78788782f9c8e2c423d5"} Dec 03 13:11:49 crc kubenswrapper[5052]: I1203 13:11:49.678992 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f0e70c0d-10fe-43b7-a08c-0ce411f223a6-kube-api-access-cpczm" (OuterVolumeSpecName: "kube-api-access-cpczm") pod "f0e70c0d-10fe-43b7-a08c-0ce411f223a6" (UID: "f0e70c0d-10fe-43b7-a08c-0ce411f223a6"). InnerVolumeSpecName "kube-api-access-cpczm". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:11:49 crc kubenswrapper[5052]: I1203 13:11:49.679001 5052 generic.go:334] "Generic (PLEG): container finished" podID="ce3d34f6-e545-4eb2-bcc5-999377931601" containerID="6ba995fc9ab1c19d638faf3e62a10a01ba4220c74844ad63f599ace4affd6e49" exitCode=0 Dec 03 13:11:49 crc kubenswrapper[5052]: I1203 13:11:49.679077 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-operators/horizon-operator-controller-manager-68c6d99b8f-mjdvh" Dec 03 13:11:49 crc kubenswrapper[5052]: I1203 13:11:49.679083 5052 generic.go:334] "Generic (PLEG): container finished" podID="ce3d34f6-e545-4eb2-bcc5-999377931601" containerID="202df0b6bac6f793c5ac016ef55a754be22068b68cff50a0616b5de880db9a3c" exitCode=0 Dec 03 13:11:49 crc kubenswrapper[5052]: I1203 13:11:49.697009 5052 generic.go:334] "Generic (PLEG): container finished" podID="0cbd234c-ecb3-4e54-af86-a2b15fdd94a3" containerID="ac6e6a21862aaede93ccb524e481d0b5e6a4f1f2e70966ac0cb19106016bbeef" exitCode=0 Dec 03 13:11:49 crc kubenswrapper[5052]: I1203 13:11:49.697044 5052 generic.go:334] "Generic (PLEG): container finished" podID="0cbd234c-ecb3-4e54-af86-a2b15fdd94a3" containerID="05da1e86576897a06ee62b48ffa3acc05751be603b91af013f2fc3b98094a039" exitCode=0 Dec 03 13:11:49 crc kubenswrapper[5052]: I1203 13:11:49.703039 5052 generic.go:334] "Generic (PLEG): container finished" podID="8fd08931-7f0b-4d5b-bfb1-f27980648561" containerID="0c828acdac4251182a50609e1c01b6a1bb38eb6449736b7bbd7aef20f9dc2f09" exitCode=0 Dec 03 13:11:49 crc kubenswrapper[5052]: I1203 13:11:49.703075 5052 generic.go:334] "Generic (PLEG): container finished" podID="8fd08931-7f0b-4d5b-bfb1-f27980648561" containerID="a075d6f4a44381ae20a16b8fa5bb2d8c0561bce325ee8a9d99cee59abaad79c8" exitCode=0 Dec 03 13:11:49 crc kubenswrapper[5052]: I1203 13:11:49.703171 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-operators/swift-operator-controller-manager-5f8c65bbfc-2cgdz" Dec 03 13:11:49 crc kubenswrapper[5052]: I1203 13:11:49.714704 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3894016b-8242-4b12-8407-bcf78af00851-kube-api-access-pmdm6" (OuterVolumeSpecName: "kube-api-access-pmdm6") pod "3894016b-8242-4b12-8407-bcf78af00851" (UID: "3894016b-8242-4b12-8407-bcf78af00851"). InnerVolumeSpecName "kube-api-access-pmdm6". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:11:49 crc kubenswrapper[5052]: I1203 13:11:49.716561 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8ad91c8d-10f1-465d-b835-74904f775d32" path="/var/lib/kubelet/pods/8ad91c8d-10f1-465d-b835-74904f775d32/volumes" Dec 03 13:11:49 crc kubenswrapper[5052]: I1203 13:11:49.728564 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/horizon-operator-controller-manager-68c6d99b8f-mjdvh" event={"ID":"ce3d34f6-e545-4eb2-bcc5-999377931601","Type":"ContainerDied","Data":"6ba995fc9ab1c19d638faf3e62a10a01ba4220c74844ad63f599ace4affd6e49"} Dec 03 13:11:49 crc kubenswrapper[5052]: I1203 13:11:49.728672 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/horizon-operator-controller-manager-68c6d99b8f-mjdvh" event={"ID":"ce3d34f6-e545-4eb2-bcc5-999377931601","Type":"ContainerDied","Data":"202df0b6bac6f793c5ac016ef55a754be22068b68cff50a0616b5de880db9a3c"} Dec 03 13:11:49 crc kubenswrapper[5052]: I1203 13:11:49.728691 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/horizon-operator-controller-manager-68c6d99b8f-mjdvh" event={"ID":"ce3d34f6-e545-4eb2-bcc5-999377931601","Type":"ContainerDied","Data":"c1394d759658941f9b2cdd49562466fdf93b8e45f0f6eb82dea4b803e222fd83"} Dec 03 13:11:49 crc kubenswrapper[5052]: I1203 13:11:49.728705 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-operators/keystone-operator-controller-manager-7765d96ddf-nlrl9"] Dec 03 13:11:49 crc kubenswrapper[5052]: I1203 13:11:49.728724 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/heat-operator-controller-manager-5f64f6f8bb-48fnb" event={"ID":"0cbd234c-ecb3-4e54-af86-a2b15fdd94a3","Type":"ContainerDied","Data":"ac6e6a21862aaede93ccb524e481d0b5e6a4f1f2e70966ac0cb19106016bbeef"} Dec 03 13:11:49 crc kubenswrapper[5052]: I1203 13:11:49.728743 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/heat-operator-controller-manager-5f64f6f8bb-48fnb" event={"ID":"0cbd234c-ecb3-4e54-af86-a2b15fdd94a3","Type":"ContainerDied","Data":"05da1e86576897a06ee62b48ffa3acc05751be603b91af013f2fc3b98094a039"} Dec 03 13:11:49 crc kubenswrapper[5052]: I1203 13:11:49.728755 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/swift-operator-controller-manager-5f8c65bbfc-2cgdz" event={"ID":"8fd08931-7f0b-4d5b-bfb1-f27980648561","Type":"ContainerDied","Data":"0c828acdac4251182a50609e1c01b6a1bb38eb6449736b7bbd7aef20f9dc2f09"} Dec 03 13:11:49 crc kubenswrapper[5052]: I1203 13:11:49.728770 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/swift-operator-controller-manager-5f8c65bbfc-2cgdz" event={"ID":"8fd08931-7f0b-4d5b-bfb1-f27980648561","Type":"ContainerDied","Data":"a075d6f4a44381ae20a16b8fa5bb2d8c0561bce325ee8a9d99cee59abaad79c8"} Dec 03 13:11:49 crc kubenswrapper[5052]: I1203 13:11:49.728784 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/swift-operator-controller-manager-5f8c65bbfc-2cgdz" event={"ID":"8fd08931-7f0b-4d5b-bfb1-f27980648561","Type":"ContainerDied","Data":"28d85cfd295d39693c46611fbf3d445b3491e1c5f0e900a55bc02fd874da005e"} Dec 03 13:11:49 crc kubenswrapper[5052]: I1203 13:11:49.735807 5052 generic.go:334] "Generic (PLEG): container finished" podID="6fe52b82-e00b-4c40-8ba1-bb52e7897667" containerID="409bd09ed8d8c06816bb54f36a222437bc97be11de3a87273fc8a642cd7acdb8" exitCode=0 Dec 03 13:11:49 crc kubenswrapper[5052]: I1203 13:11:49.735892 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/openstack-operator-index-x6glm" event={"ID":"6fe52b82-e00b-4c40-8ba1-bb52e7897667","Type":"ContainerDied","Data":"409bd09ed8d8c06816bb54f36a222437bc97be11de3a87273fc8a642cd7acdb8"} Dec 03 13:11:49 crc kubenswrapper[5052]: I1203 13:11:49.745251 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-operators/keystone-operator-controller-manager-7765d96ddf-nlrl9"] Dec 03 13:11:49 crc kubenswrapper[5052]: I1203 13:11:49.745458 5052 scope.go:117] "RemoveContainer" containerID="acbb799df7a125d48309f76c7ac211fd06dfd6580980ca517793b3d574553f2e" Dec 03 13:11:49 crc kubenswrapper[5052]: I1203 13:11:49.750295 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-operators/heat-operator-controller-manager-5f64f6f8bb-48fnb" Dec 03 13:11:49 crc kubenswrapper[5052]: I1203 13:11:49.752351 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-cpczm\" (UniqueName: \"kubernetes.io/projected/f0e70c0d-10fe-43b7-a08c-0ce411f223a6-kube-api-access-cpczm\") on node \"crc\" DevicePath \"\"" Dec 03 13:11:49 crc kubenswrapper[5052]: I1203 13:11:49.752376 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-pmdm6\" (UniqueName: \"kubernetes.io/projected/3894016b-8242-4b12-8407-bcf78af00851-kube-api-access-pmdm6\") on node \"crc\" DevicePath \"\"" Dec 03 13:11:49 crc kubenswrapper[5052]: I1203 13:11:49.784581 5052 generic.go:334] "Generic (PLEG): container finished" podID="2e2dc077-c02f-4f98-8fa6-9f72890faeaf" containerID="bc9d68e8694514b249866a594a571232d780cb8f8315ced546035025828c5930" exitCode=0 Dec 03 13:11:49 crc kubenswrapper[5052]: I1203 13:11:49.784631 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/designate-operator-controller-manager-78b4bc895b-874g2" event={"ID":"2e2dc077-c02f-4f98-8fa6-9f72890faeaf","Type":"ContainerDied","Data":"bc9d68e8694514b249866a594a571232d780cb8f8315ced546035025828c5930"} Dec 03 13:11:49 crc kubenswrapper[5052]: I1203 13:11:49.784661 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/designate-operator-controller-manager-78b4bc895b-874g2" event={"ID":"2e2dc077-c02f-4f98-8fa6-9f72890faeaf","Type":"ContainerDied","Data":"e122802b4e64823d6136a13ba845b5b5991723cebe321d42e23c65d78aa51ace"} Dec 03 13:11:49 crc kubenswrapper[5052]: I1203 13:11:49.784732 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-operators/designate-operator-controller-manager-78b4bc895b-874g2" Dec 03 13:11:49 crc kubenswrapper[5052]: I1203 13:11:49.789334 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-operators/neutron-operator-controller-manager-5fdfd5b6b5-q48nd"] Dec 03 13:11:49 crc kubenswrapper[5052]: I1203 13:11:49.795699 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-operators/neutron-operator-controller-manager-5fdfd5b6b5-q48nd"] Dec 03 13:11:49 crc kubenswrapper[5052]: I1203 13:11:49.854207 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wgp7x\" (UniqueName: \"kubernetes.io/projected/0cbd234c-ecb3-4e54-af86-a2b15fdd94a3-kube-api-access-wgp7x\") pod \"0cbd234c-ecb3-4e54-af86-a2b15fdd94a3\" (UID: \"0cbd234c-ecb3-4e54-af86-a2b15fdd94a3\") " Dec 03 13:11:49 crc kubenswrapper[5052]: I1203 13:11:49.861955 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0cbd234c-ecb3-4e54-af86-a2b15fdd94a3-kube-api-access-wgp7x" (OuterVolumeSpecName: "kube-api-access-wgp7x") pod "0cbd234c-ecb3-4e54-af86-a2b15fdd94a3" (UID: "0cbd234c-ecb3-4e54-af86-a2b15fdd94a3"). InnerVolumeSpecName "kube-api-access-wgp7x". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:11:49 crc kubenswrapper[5052]: I1203 13:11:49.956147 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-wgp7x\" (UniqueName: \"kubernetes.io/projected/0cbd234c-ecb3-4e54-af86-a2b15fdd94a3-kube-api-access-wgp7x\") on node \"crc\" DevicePath \"\"" Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.090852 5052 scope.go:117] "RemoveContainer" containerID="f4171f259274e4008a6e5b98254b34066d3fc61afa68cf052421761cd5affdfb" Dec 03 13:11:50 crc kubenswrapper[5052]: E1203 13:11:50.091747 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f4171f259274e4008a6e5b98254b34066d3fc61afa68cf052421761cd5affdfb\": container with ID starting with f4171f259274e4008a6e5b98254b34066d3fc61afa68cf052421761cd5affdfb not found: ID does not exist" containerID="f4171f259274e4008a6e5b98254b34066d3fc61afa68cf052421761cd5affdfb" Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.091788 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f4171f259274e4008a6e5b98254b34066d3fc61afa68cf052421761cd5affdfb"} err="failed to get container status \"f4171f259274e4008a6e5b98254b34066d3fc61afa68cf052421761cd5affdfb\": rpc error: code = NotFound desc = could not find container \"f4171f259274e4008a6e5b98254b34066d3fc61afa68cf052421761cd5affdfb\": container with ID starting with f4171f259274e4008a6e5b98254b34066d3fc61afa68cf052421761cd5affdfb not found: ID does not exist" Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.091813 5052 scope.go:117] "RemoveContainer" containerID="acbb799df7a125d48309f76c7ac211fd06dfd6580980ca517793b3d574553f2e" Dec 03 13:11:50 crc kubenswrapper[5052]: E1203 13:11:50.092173 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"acbb799df7a125d48309f76c7ac211fd06dfd6580980ca517793b3d574553f2e\": container with ID starting with acbb799df7a125d48309f76c7ac211fd06dfd6580980ca517793b3d574553f2e not found: ID does not exist" containerID="acbb799df7a125d48309f76c7ac211fd06dfd6580980ca517793b3d574553f2e" Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.092216 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"acbb799df7a125d48309f76c7ac211fd06dfd6580980ca517793b3d574553f2e"} err="failed to get container status \"acbb799df7a125d48309f76c7ac211fd06dfd6580980ca517793b3d574553f2e\": rpc error: code = NotFound desc = could not find container \"acbb799df7a125d48309f76c7ac211fd06dfd6580980ca517793b3d574553f2e\": container with ID starting with acbb799df7a125d48309f76c7ac211fd06dfd6580980ca517793b3d574553f2e not found: ID does not exist" Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.092245 5052 scope.go:117] "RemoveContainer" containerID="c8a22e6613869e6f1219b539f8d24e9e232f41001d8d1d96f9c514c62fd7214c" Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.127356 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-operators/placement-operator-controller-manager-78f8948974-j62hd" Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.142365 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-operators/rabbitmq-cluster-operator-manager-668c99d594-ksw2p" Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.147765 5052 scope.go:117] "RemoveContainer" containerID="0add256c4e87d7b91cc63837e4958d34bfbf4128a08af08316deec77be2a75eb" Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.157911 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-operators/horizon-operator-controller-manager-68c6d99b8f-mjdvh"] Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.164518 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-operators/barbican-operator-controller-manager-7d9dfd778-cqfdk" Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.166830 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-operators/horizon-operator-controller-manager-68c6d99b8f-mjdvh"] Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.181057 5052 scope.go:117] "RemoveContainer" containerID="c8a22e6613869e6f1219b539f8d24e9e232f41001d8d1d96f9c514c62fd7214c" Dec 03 13:11:50 crc kubenswrapper[5052]: E1203 13:11:50.184832 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c8a22e6613869e6f1219b539f8d24e9e232f41001d8d1d96f9c514c62fd7214c\": container with ID starting with c8a22e6613869e6f1219b539f8d24e9e232f41001d8d1d96f9c514c62fd7214c not found: ID does not exist" containerID="c8a22e6613869e6f1219b539f8d24e9e232f41001d8d1d96f9c514c62fd7214c" Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.184905 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c8a22e6613869e6f1219b539f8d24e9e232f41001d8d1d96f9c514c62fd7214c"} err="failed to get container status \"c8a22e6613869e6f1219b539f8d24e9e232f41001d8d1d96f9c514c62fd7214c\": rpc error: code = NotFound desc = could not find container \"c8a22e6613869e6f1219b539f8d24e9e232f41001d8d1d96f9c514c62fd7214c\": container with ID starting with c8a22e6613869e6f1219b539f8d24e9e232f41001d8d1d96f9c514c62fd7214c not found: ID does not exist" Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.184992 5052 scope.go:117] "RemoveContainer" containerID="0add256c4e87d7b91cc63837e4958d34bfbf4128a08af08316deec77be2a75eb" Dec 03 13:11:50 crc kubenswrapper[5052]: E1203 13:11:50.185846 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0add256c4e87d7b91cc63837e4958d34bfbf4128a08af08316deec77be2a75eb\": container with ID starting with 0add256c4e87d7b91cc63837e4958d34bfbf4128a08af08316deec77be2a75eb not found: ID does not exist" containerID="0add256c4e87d7b91cc63837e4958d34bfbf4128a08af08316deec77be2a75eb" Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.185906 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0add256c4e87d7b91cc63837e4958d34bfbf4128a08af08316deec77be2a75eb"} err="failed to get container status \"0add256c4e87d7b91cc63837e4958d34bfbf4128a08af08316deec77be2a75eb\": rpc error: code = NotFound desc = could not find container \"0add256c4e87d7b91cc63837e4958d34bfbf4128a08af08316deec77be2a75eb\": container with ID starting with 0add256c4e87d7b91cc63837e4958d34bfbf4128a08af08316deec77be2a75eb not found: ID does not exist" Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.185926 5052 scope.go:117] "RemoveContainer" containerID="c8a22e6613869e6f1219b539f8d24e9e232f41001d8d1d96f9c514c62fd7214c" Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.189845 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c8a22e6613869e6f1219b539f8d24e9e232f41001d8d1d96f9c514c62fd7214c"} err="failed to get container status \"c8a22e6613869e6f1219b539f8d24e9e232f41001d8d1d96f9c514c62fd7214c\": rpc error: code = NotFound desc = could not find container \"c8a22e6613869e6f1219b539f8d24e9e232f41001d8d1d96f9c514c62fd7214c\": container with ID starting with c8a22e6613869e6f1219b539f8d24e9e232f41001d8d1d96f9c514c62fd7214c not found: ID does not exist" Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.189907 5052 scope.go:117] "RemoveContainer" containerID="0add256c4e87d7b91cc63837e4958d34bfbf4128a08af08316deec77be2a75eb" Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.192639 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0add256c4e87d7b91cc63837e4958d34bfbf4128a08af08316deec77be2a75eb"} err="failed to get container status \"0add256c4e87d7b91cc63837e4958d34bfbf4128a08af08316deec77be2a75eb\": rpc error: code = NotFound desc = could not find container \"0add256c4e87d7b91cc63837e4958d34bfbf4128a08af08316deec77be2a75eb\": container with ID starting with 0add256c4e87d7b91cc63837e4958d34bfbf4128a08af08316deec77be2a75eb not found: ID does not exist" Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.192716 5052 scope.go:117] "RemoveContainer" containerID="efce01fd34c9aaa82b7beeb80cae30909d6a58a957f233a93543d43ab86d6552" Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.193814 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-operators/openstack-baremetal-operator-controller-manager-55d86b6686f2mft" Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.213724 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-operators/openstack-operator-index-x6glm" Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.224528 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-operators/infra-operator-controller-manager-57548d458d-9fkll" Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.224808 5052 scope.go:117] "RemoveContainer" containerID="5216edc450cbbbdc9be723de0d23e9c4c7f79734c44ce55e5ee6afa4b2ca1bd2" Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.238474 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-operators/openstack-operator-controller-operator-7dd5c7bb7c-rjflp" Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.241557 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-operators/glance-operator-controller-manager-77987cd8cd-btn2l" Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.249347 5052 scope.go:117] "RemoveContainer" containerID="efce01fd34c9aaa82b7beeb80cae30909d6a58a957f233a93543d43ab86d6552" Dec 03 13:11:50 crc kubenswrapper[5052]: E1203 13:11:50.249839 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"efce01fd34c9aaa82b7beeb80cae30909d6a58a957f233a93543d43ab86d6552\": container with ID starting with efce01fd34c9aaa82b7beeb80cae30909d6a58a957f233a93543d43ab86d6552 not found: ID does not exist" containerID="efce01fd34c9aaa82b7beeb80cae30909d6a58a957f233a93543d43ab86d6552" Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.249880 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"efce01fd34c9aaa82b7beeb80cae30909d6a58a957f233a93543d43ab86d6552"} err="failed to get container status \"efce01fd34c9aaa82b7beeb80cae30909d6a58a957f233a93543d43ab86d6552\": rpc error: code = NotFound desc = could not find container \"efce01fd34c9aaa82b7beeb80cae30909d6a58a957f233a93543d43ab86d6552\": container with ID starting with efce01fd34c9aaa82b7beeb80cae30909d6a58a957f233a93543d43ab86d6552 not found: ID does not exist" Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.249908 5052 scope.go:117] "RemoveContainer" containerID="5216edc450cbbbdc9be723de0d23e9c4c7f79734c44ce55e5ee6afa4b2ca1bd2" Dec 03 13:11:50 crc kubenswrapper[5052]: E1203 13:11:50.250221 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5216edc450cbbbdc9be723de0d23e9c4c7f79734c44ce55e5ee6afa4b2ca1bd2\": container with ID starting with 5216edc450cbbbdc9be723de0d23e9c4c7f79734c44ce55e5ee6afa4b2ca1bd2 not found: ID does not exist" containerID="5216edc450cbbbdc9be723de0d23e9c4c7f79734c44ce55e5ee6afa4b2ca1bd2" Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.250252 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5216edc450cbbbdc9be723de0d23e9c4c7f79734c44ce55e5ee6afa4b2ca1bd2"} err="failed to get container status \"5216edc450cbbbdc9be723de0d23e9c4c7f79734c44ce55e5ee6afa4b2ca1bd2\": rpc error: code = NotFound desc = could not find container \"5216edc450cbbbdc9be723de0d23e9c4c7f79734c44ce55e5ee6afa4b2ca1bd2\": container with ID starting with 5216edc450cbbbdc9be723de0d23e9c4c7f79734c44ce55e5ee6afa4b2ca1bd2 not found: ID does not exist" Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.250276 5052 scope.go:117] "RemoveContainer" containerID="61b7e037d8a6c427bda841efbbb994e108ee2daac74c44aca5542d608eed3910" Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.251443 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-operators/nova-operator-controller-manager-697bc559fc-gcj5d" Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.260633 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7dk26\" (UniqueName: \"kubernetes.io/projected/459f5555-efae-4c19-84d0-2af917b9560f-kube-api-access-7dk26\") pod \"459f5555-efae-4c19-84d0-2af917b9560f\" (UID: \"459f5555-efae-4c19-84d0-2af917b9560f\") " Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.262802 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bpxzs\" (UniqueName: \"kubernetes.io/projected/37ab421b-0ad3-4d9d-bf47-8bf908c4aa57-kube-api-access-bpxzs\") pod \"37ab421b-0ad3-4d9d-bf47-8bf908c4aa57\" (UID: \"37ab421b-0ad3-4d9d-bf47-8bf908c4aa57\") " Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.269361 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/37ab421b-0ad3-4d9d-bf47-8bf908c4aa57-kube-api-access-bpxzs" (OuterVolumeSpecName: "kube-api-access-bpxzs") pod "37ab421b-0ad3-4d9d-bf47-8bf908c4aa57" (UID: "37ab421b-0ad3-4d9d-bf47-8bf908c4aa57"). InnerVolumeSpecName "kube-api-access-bpxzs". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.269694 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-operators/watcher-operator-controller-manager-769dc69bc-7rhg6" Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.271134 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/459f5555-efae-4c19-84d0-2af917b9560f-kube-api-access-7dk26" (OuterVolumeSpecName: "kube-api-access-7dk26") pod "459f5555-efae-4c19-84d0-2af917b9560f" (UID: "459f5555-efae-4c19-84d0-2af917b9560f"). InnerVolumeSpecName "kube-api-access-7dk26". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.283139 5052 scope.go:117] "RemoveContainer" containerID="97782ec29d2e9e3193fca38c78d6d80a51ddf23167b6089a154cabfc32ab7307" Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.286860 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-operators/swift-operator-controller-manager-5f8c65bbfc-2cgdz"] Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.295898 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-operators/swift-operator-controller-manager-5f8c65bbfc-2cgdz"] Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.304309 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-operators/ironic-operator-controller-manager-6c548fd776-pqfnd"] Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.312937 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-operators/ironic-operator-controller-manager-6c548fd776-pqfnd"] Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.319477 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-operators/octavia-operator-controller-manager-998648c74-z6cw9"] Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.328573 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-operators/octavia-operator-controller-manager-998648c74-z6cw9"] Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.333283 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-operators/designate-operator-controller-manager-78b4bc895b-874g2"] Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.338093 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-operators/designate-operator-controller-manager-78b4bc895b-874g2"] Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.345921 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-operators/cinder-operator-controller-manager-859b6ccc6-vt9d2" Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.366031 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e10da887-70df-43aa-aa79-4a14231e5ec9-cert\") pod \"e10da887-70df-43aa-aa79-4a14231e5ec9\" (UID: \"e10da887-70df-43aa-aa79-4a14231e5ec9\") " Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.366144 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2cfz6\" (UniqueName: \"kubernetes.io/projected/e10da887-70df-43aa-aa79-4a14231e5ec9-kube-api-access-2cfz6\") pod \"e10da887-70df-43aa-aa79-4a14231e5ec9\" (UID: \"e10da887-70df-43aa-aa79-4a14231e5ec9\") " Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.366219 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-42lr8\" (UniqueName: \"kubernetes.io/projected/6fe52b82-e00b-4c40-8ba1-bb52e7897667-kube-api-access-42lr8\") pod \"6fe52b82-e00b-4c40-8ba1-bb52e7897667\" (UID: \"6fe52b82-e00b-4c40-8ba1-bb52e7897667\") " Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.366236 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8vjcl\" (UniqueName: \"kubernetes.io/projected/8dbd25a1-9de9-431e-be37-d239e3f64728-kube-api-access-8vjcl\") pod \"8dbd25a1-9de9-431e-be37-d239e3f64728\" (UID: \"8dbd25a1-9de9-431e-be37-d239e3f64728\") " Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.366807 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kvnlb\" (UniqueName: \"kubernetes.io/projected/d429f7ab-9a88-4411-a7a5-88bc8ab76285-kube-api-access-kvnlb\") pod \"d429f7ab-9a88-4411-a7a5-88bc8ab76285\" (UID: \"d429f7ab-9a88-4411-a7a5-88bc8ab76285\") " Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.366869 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-658xp\" (UniqueName: \"kubernetes.io/projected/cd2c5c4c-7ea8-4f95-9556-58b87d14a2be-kube-api-access-658xp\") pod \"cd2c5c4c-7ea8-4f95-9556-58b87d14a2be\" (UID: \"cd2c5c4c-7ea8-4f95-9556-58b87d14a2be\") " Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.366903 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6pqvt\" (UniqueName: \"kubernetes.io/projected/3e6bc3f6-1f1a-4b99-8835-98252369d364-kube-api-access-6pqvt\") pod \"3e6bc3f6-1f1a-4b99-8835-98252369d364\" (UID: \"3e6bc3f6-1f1a-4b99-8835-98252369d364\") " Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.366943 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-klcr8\" (UniqueName: \"kubernetes.io/projected/81f5f8e1-015a-4b45-83e6-13e88efb8e80-kube-api-access-klcr8\") pod \"81f5f8e1-015a-4b45-83e6-13e88efb8e80\" (UID: \"81f5f8e1-015a-4b45-83e6-13e88efb8e80\") " Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.366978 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/81f5f8e1-015a-4b45-83e6-13e88efb8e80-cert\") pod \"81f5f8e1-015a-4b45-83e6-13e88efb8e80\" (UID: \"81f5f8e1-015a-4b45-83e6-13e88efb8e80\") " Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.367458 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-7dk26\" (UniqueName: \"kubernetes.io/projected/459f5555-efae-4c19-84d0-2af917b9560f-kube-api-access-7dk26\") on node \"crc\" DevicePath \"\"" Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.367581 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-bpxzs\" (UniqueName: \"kubernetes.io/projected/37ab421b-0ad3-4d9d-bf47-8bf908c4aa57-kube-api-access-bpxzs\") on node \"crc\" DevicePath \"\"" Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.372179 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e10da887-70df-43aa-aa79-4a14231e5ec9-kube-api-access-2cfz6" (OuterVolumeSpecName: "kube-api-access-2cfz6") pod "e10da887-70df-43aa-aa79-4a14231e5ec9" (UID: "e10da887-70df-43aa-aa79-4a14231e5ec9"). InnerVolumeSpecName "kube-api-access-2cfz6". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.373661 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/81f5f8e1-015a-4b45-83e6-13e88efb8e80-kube-api-access-klcr8" (OuterVolumeSpecName: "kube-api-access-klcr8") pod "81f5f8e1-015a-4b45-83e6-13e88efb8e80" (UID: "81f5f8e1-015a-4b45-83e6-13e88efb8e80"). InnerVolumeSpecName "kube-api-access-klcr8". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.373846 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3e6bc3f6-1f1a-4b99-8835-98252369d364-kube-api-access-6pqvt" (OuterVolumeSpecName: "kube-api-access-6pqvt") pod "3e6bc3f6-1f1a-4b99-8835-98252369d364" (UID: "3e6bc3f6-1f1a-4b99-8835-98252369d364"). InnerVolumeSpecName "kube-api-access-6pqvt". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.375260 5052 scope.go:117] "RemoveContainer" containerID="61b7e037d8a6c427bda841efbbb994e108ee2daac74c44aca5542d608eed3910" Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.375649 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e10da887-70df-43aa-aa79-4a14231e5ec9-cert" (OuterVolumeSpecName: "cert") pod "e10da887-70df-43aa-aa79-4a14231e5ec9" (UID: "e10da887-70df-43aa-aa79-4a14231e5ec9"). InnerVolumeSpecName "cert". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.375876 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cd2c5c4c-7ea8-4f95-9556-58b87d14a2be-kube-api-access-658xp" (OuterVolumeSpecName: "kube-api-access-658xp") pod "cd2c5c4c-7ea8-4f95-9556-58b87d14a2be" (UID: "cd2c5c4c-7ea8-4f95-9556-58b87d14a2be"). InnerVolumeSpecName "kube-api-access-658xp". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.377129 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/81f5f8e1-015a-4b45-83e6-13e88efb8e80-cert" (OuterVolumeSpecName: "cert") pod "81f5f8e1-015a-4b45-83e6-13e88efb8e80" (UID: "81f5f8e1-015a-4b45-83e6-13e88efb8e80"). InnerVolumeSpecName "cert". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.377831 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6fe52b82-e00b-4c40-8ba1-bb52e7897667-kube-api-access-42lr8" (OuterVolumeSpecName: "kube-api-access-42lr8") pod "6fe52b82-e00b-4c40-8ba1-bb52e7897667" (UID: "6fe52b82-e00b-4c40-8ba1-bb52e7897667"). InnerVolumeSpecName "kube-api-access-42lr8". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:11:50 crc kubenswrapper[5052]: E1203 13:11:50.378831 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"61b7e037d8a6c427bda841efbbb994e108ee2daac74c44aca5542d608eed3910\": container with ID starting with 61b7e037d8a6c427bda841efbbb994e108ee2daac74c44aca5542d608eed3910 not found: ID does not exist" containerID="61b7e037d8a6c427bda841efbbb994e108ee2daac74c44aca5542d608eed3910" Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.378883 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"61b7e037d8a6c427bda841efbbb994e108ee2daac74c44aca5542d608eed3910"} err="failed to get container status \"61b7e037d8a6c427bda841efbbb994e108ee2daac74c44aca5542d608eed3910\": rpc error: code = NotFound desc = could not find container \"61b7e037d8a6c427bda841efbbb994e108ee2daac74c44aca5542d608eed3910\": container with ID starting with 61b7e037d8a6c427bda841efbbb994e108ee2daac74c44aca5542d608eed3910 not found: ID does not exist" Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.378940 5052 scope.go:117] "RemoveContainer" containerID="97782ec29d2e9e3193fca38c78d6d80a51ddf23167b6089a154cabfc32ab7307" Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.380503 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d429f7ab-9a88-4411-a7a5-88bc8ab76285-kube-api-access-kvnlb" (OuterVolumeSpecName: "kube-api-access-kvnlb") pod "d429f7ab-9a88-4411-a7a5-88bc8ab76285" (UID: "d429f7ab-9a88-4411-a7a5-88bc8ab76285"). InnerVolumeSpecName "kube-api-access-kvnlb". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.381036 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-operators/test-operator-controller-manager-5854674fcc-jltb6" Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.381133 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8dbd25a1-9de9-431e-be37-d239e3f64728-kube-api-access-8vjcl" (OuterVolumeSpecName: "kube-api-access-8vjcl") pod "8dbd25a1-9de9-431e-be37-d239e3f64728" (UID: "8dbd25a1-9de9-431e-be37-d239e3f64728"). InnerVolumeSpecName "kube-api-access-8vjcl". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:11:50 crc kubenswrapper[5052]: E1203 13:11:50.383653 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"97782ec29d2e9e3193fca38c78d6d80a51ddf23167b6089a154cabfc32ab7307\": container with ID starting with 97782ec29d2e9e3193fca38c78d6d80a51ddf23167b6089a154cabfc32ab7307 not found: ID does not exist" containerID="97782ec29d2e9e3193fca38c78d6d80a51ddf23167b6089a154cabfc32ab7307" Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.383690 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"97782ec29d2e9e3193fca38c78d6d80a51ddf23167b6089a154cabfc32ab7307"} err="failed to get container status \"97782ec29d2e9e3193fca38c78d6d80a51ddf23167b6089a154cabfc32ab7307\": rpc error: code = NotFound desc = could not find container \"97782ec29d2e9e3193fca38c78d6d80a51ddf23167b6089a154cabfc32ab7307\": container with ID starting with 97782ec29d2e9e3193fca38c78d6d80a51ddf23167b6089a154cabfc32ab7307 not found: ID does not exist" Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.383717 5052 scope.go:117] "RemoveContainer" containerID="6ba995fc9ab1c19d638faf3e62a10a01ba4220c74844ad63f599ace4affd6e49" Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.426296 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-operators/openstack-operator-controller-manager-9f56fc979-b8jv8" Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.433886 5052 scope.go:117] "RemoveContainer" containerID="202df0b6bac6f793c5ac016ef55a754be22068b68cff50a0616b5de880db9a3c" Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.441327 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-operators/ovn-operator-controller-manager-b6456fdb6-sggpb" Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.469019 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gxjqd\" (UniqueName: \"kubernetes.io/projected/6bea5f42-82da-4148-9996-dc05121212f7-kube-api-access-gxjqd\") pod \"6bea5f42-82da-4148-9996-dc05121212f7\" (UID: \"6bea5f42-82da-4148-9996-dc05121212f7\") " Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.469148 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fkq98\" (UniqueName: \"kubernetes.io/projected/5968d273-33dd-45cc-aab0-e83628bcd15c-kube-api-access-fkq98\") pod \"5968d273-33dd-45cc-aab0-e83628bcd15c\" (UID: \"5968d273-33dd-45cc-aab0-e83628bcd15c\") " Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.469538 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-klcr8\" (UniqueName: \"kubernetes.io/projected/81f5f8e1-015a-4b45-83e6-13e88efb8e80-kube-api-access-klcr8\") on node \"crc\" DevicePath \"\"" Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.469557 5052 reconciler_common.go:293] "Volume detached for volume \"cert\" (UniqueName: \"kubernetes.io/secret/81f5f8e1-015a-4b45-83e6-13e88efb8e80-cert\") on node \"crc\" DevicePath \"\"" Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.469570 5052 reconciler_common.go:293] "Volume detached for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e10da887-70df-43aa-aa79-4a14231e5ec9-cert\") on node \"crc\" DevicePath \"\"" Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.469584 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-2cfz6\" (UniqueName: \"kubernetes.io/projected/e10da887-70df-43aa-aa79-4a14231e5ec9-kube-api-access-2cfz6\") on node \"crc\" DevicePath \"\"" Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.469595 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-42lr8\" (UniqueName: \"kubernetes.io/projected/6fe52b82-e00b-4c40-8ba1-bb52e7897667-kube-api-access-42lr8\") on node \"crc\" DevicePath \"\"" Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.469609 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-8vjcl\" (UniqueName: \"kubernetes.io/projected/8dbd25a1-9de9-431e-be37-d239e3f64728-kube-api-access-8vjcl\") on node \"crc\" DevicePath \"\"" Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.469620 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-kvnlb\" (UniqueName: \"kubernetes.io/projected/d429f7ab-9a88-4411-a7a5-88bc8ab76285-kube-api-access-kvnlb\") on node \"crc\" DevicePath \"\"" Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.469632 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-658xp\" (UniqueName: \"kubernetes.io/projected/cd2c5c4c-7ea8-4f95-9556-58b87d14a2be-kube-api-access-658xp\") on node \"crc\" DevicePath \"\"" Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.469643 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-6pqvt\" (UniqueName: \"kubernetes.io/projected/3e6bc3f6-1f1a-4b99-8835-98252369d364-kube-api-access-6pqvt\") on node \"crc\" DevicePath \"\"" Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.472677 5052 scope.go:117] "RemoveContainer" containerID="6ba995fc9ab1c19d638faf3e62a10a01ba4220c74844ad63f599ace4affd6e49" Dec 03 13:11:50 crc kubenswrapper[5052]: E1203 13:11:50.473281 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6ba995fc9ab1c19d638faf3e62a10a01ba4220c74844ad63f599ace4affd6e49\": container with ID starting with 6ba995fc9ab1c19d638faf3e62a10a01ba4220c74844ad63f599ace4affd6e49 not found: ID does not exist" containerID="6ba995fc9ab1c19d638faf3e62a10a01ba4220c74844ad63f599ace4affd6e49" Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.473324 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6ba995fc9ab1c19d638faf3e62a10a01ba4220c74844ad63f599ace4affd6e49"} err="failed to get container status \"6ba995fc9ab1c19d638faf3e62a10a01ba4220c74844ad63f599ace4affd6e49\": rpc error: code = NotFound desc = could not find container \"6ba995fc9ab1c19d638faf3e62a10a01ba4220c74844ad63f599ace4affd6e49\": container with ID starting with 6ba995fc9ab1c19d638faf3e62a10a01ba4220c74844ad63f599ace4affd6e49 not found: ID does not exist" Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.473353 5052 scope.go:117] "RemoveContainer" containerID="202df0b6bac6f793c5ac016ef55a754be22068b68cff50a0616b5de880db9a3c" Dec 03 13:11:50 crc kubenswrapper[5052]: E1203 13:11:50.474018 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"202df0b6bac6f793c5ac016ef55a754be22068b68cff50a0616b5de880db9a3c\": container with ID starting with 202df0b6bac6f793c5ac016ef55a754be22068b68cff50a0616b5de880db9a3c not found: ID does not exist" containerID="202df0b6bac6f793c5ac016ef55a754be22068b68cff50a0616b5de880db9a3c" Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.474075 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"202df0b6bac6f793c5ac016ef55a754be22068b68cff50a0616b5de880db9a3c"} err="failed to get container status \"202df0b6bac6f793c5ac016ef55a754be22068b68cff50a0616b5de880db9a3c\": rpc error: code = NotFound desc = could not find container \"202df0b6bac6f793c5ac016ef55a754be22068b68cff50a0616b5de880db9a3c\": container with ID starting with 202df0b6bac6f793c5ac016ef55a754be22068b68cff50a0616b5de880db9a3c not found: ID does not exist" Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.474091 5052 scope.go:117] "RemoveContainer" containerID="6ba995fc9ab1c19d638faf3e62a10a01ba4220c74844ad63f599ace4affd6e49" Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.474388 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6ba995fc9ab1c19d638faf3e62a10a01ba4220c74844ad63f599ace4affd6e49"} err="failed to get container status \"6ba995fc9ab1c19d638faf3e62a10a01ba4220c74844ad63f599ace4affd6e49\": rpc error: code = NotFound desc = could not find container \"6ba995fc9ab1c19d638faf3e62a10a01ba4220c74844ad63f599ace4affd6e49\": container with ID starting with 6ba995fc9ab1c19d638faf3e62a10a01ba4220c74844ad63f599ace4affd6e49 not found: ID does not exist" Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.474437 5052 scope.go:117] "RemoveContainer" containerID="202df0b6bac6f793c5ac016ef55a754be22068b68cff50a0616b5de880db9a3c" Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.474878 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"202df0b6bac6f793c5ac016ef55a754be22068b68cff50a0616b5de880db9a3c"} err="failed to get container status \"202df0b6bac6f793c5ac016ef55a754be22068b68cff50a0616b5de880db9a3c\": rpc error: code = NotFound desc = could not find container \"202df0b6bac6f793c5ac016ef55a754be22068b68cff50a0616b5de880db9a3c\": container with ID starting with 202df0b6bac6f793c5ac016ef55a754be22068b68cff50a0616b5de880db9a3c not found: ID does not exist" Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.474900 5052 scope.go:117] "RemoveContainer" containerID="0c828acdac4251182a50609e1c01b6a1bb38eb6449736b7bbd7aef20f9dc2f09" Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.476884 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6bea5f42-82da-4148-9996-dc05121212f7-kube-api-access-gxjqd" (OuterVolumeSpecName: "kube-api-access-gxjqd") pod "6bea5f42-82da-4148-9996-dc05121212f7" (UID: "6bea5f42-82da-4148-9996-dc05121212f7"). InnerVolumeSpecName "kube-api-access-gxjqd". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.478324 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5968d273-33dd-45cc-aab0-e83628bcd15c-kube-api-access-fkq98" (OuterVolumeSpecName: "kube-api-access-fkq98") pod "5968d273-33dd-45cc-aab0-e83628bcd15c" (UID: "5968d273-33dd-45cc-aab0-e83628bcd15c"). InnerVolumeSpecName "kube-api-access-fkq98". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.504158 5052 scope.go:117] "RemoveContainer" containerID="a075d6f4a44381ae20a16b8fa5bb2d8c0561bce325ee8a9d99cee59abaad79c8" Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.552065 5052 scope.go:117] "RemoveContainer" containerID="0c828acdac4251182a50609e1c01b6a1bb38eb6449736b7bbd7aef20f9dc2f09" Dec 03 13:11:50 crc kubenswrapper[5052]: E1203 13:11:50.553155 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0c828acdac4251182a50609e1c01b6a1bb38eb6449736b7bbd7aef20f9dc2f09\": container with ID starting with 0c828acdac4251182a50609e1c01b6a1bb38eb6449736b7bbd7aef20f9dc2f09 not found: ID does not exist" containerID="0c828acdac4251182a50609e1c01b6a1bb38eb6449736b7bbd7aef20f9dc2f09" Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.553215 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0c828acdac4251182a50609e1c01b6a1bb38eb6449736b7bbd7aef20f9dc2f09"} err="failed to get container status \"0c828acdac4251182a50609e1c01b6a1bb38eb6449736b7bbd7aef20f9dc2f09\": rpc error: code = NotFound desc = could not find container \"0c828acdac4251182a50609e1c01b6a1bb38eb6449736b7bbd7aef20f9dc2f09\": container with ID starting with 0c828acdac4251182a50609e1c01b6a1bb38eb6449736b7bbd7aef20f9dc2f09 not found: ID does not exist" Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.553249 5052 scope.go:117] "RemoveContainer" containerID="a075d6f4a44381ae20a16b8fa5bb2d8c0561bce325ee8a9d99cee59abaad79c8" Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.553163 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="openstack-operators/openstack-operator-controller-manager-9f56fc979-b8jv8" podUID="527bcd50-6d8d-4d4f-bc11-5382eb9adfc2" containerName="manager" probeResult="failure" output="Get \"http://10.217.0.94:8081/readyz\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Dec 03 13:11:50 crc kubenswrapper[5052]: E1203 13:11:50.554197 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a075d6f4a44381ae20a16b8fa5bb2d8c0561bce325ee8a9d99cee59abaad79c8\": container with ID starting with a075d6f4a44381ae20a16b8fa5bb2d8c0561bce325ee8a9d99cee59abaad79c8 not found: ID does not exist" containerID="a075d6f4a44381ae20a16b8fa5bb2d8c0561bce325ee8a9d99cee59abaad79c8" Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.554231 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a075d6f4a44381ae20a16b8fa5bb2d8c0561bce325ee8a9d99cee59abaad79c8"} err="failed to get container status \"a075d6f4a44381ae20a16b8fa5bb2d8c0561bce325ee8a9d99cee59abaad79c8\": rpc error: code = NotFound desc = could not find container \"a075d6f4a44381ae20a16b8fa5bb2d8c0561bce325ee8a9d99cee59abaad79c8\": container with ID starting with a075d6f4a44381ae20a16b8fa5bb2d8c0561bce325ee8a9d99cee59abaad79c8 not found: ID does not exist" Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.554251 5052 scope.go:117] "RemoveContainer" containerID="0c828acdac4251182a50609e1c01b6a1bb38eb6449736b7bbd7aef20f9dc2f09" Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.555187 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0c828acdac4251182a50609e1c01b6a1bb38eb6449736b7bbd7aef20f9dc2f09"} err="failed to get container status \"0c828acdac4251182a50609e1c01b6a1bb38eb6449736b7bbd7aef20f9dc2f09\": rpc error: code = NotFound desc = could not find container \"0c828acdac4251182a50609e1c01b6a1bb38eb6449736b7bbd7aef20f9dc2f09\": container with ID starting with 0c828acdac4251182a50609e1c01b6a1bb38eb6449736b7bbd7aef20f9dc2f09 not found: ID does not exist" Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.555223 5052 scope.go:117] "RemoveContainer" containerID="a075d6f4a44381ae20a16b8fa5bb2d8c0561bce325ee8a9d99cee59abaad79c8" Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.566319 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a075d6f4a44381ae20a16b8fa5bb2d8c0561bce325ee8a9d99cee59abaad79c8"} err="failed to get container status \"a075d6f4a44381ae20a16b8fa5bb2d8c0561bce325ee8a9d99cee59abaad79c8\": rpc error: code = NotFound desc = could not find container \"a075d6f4a44381ae20a16b8fa5bb2d8c0561bce325ee8a9d99cee59abaad79c8\": container with ID starting with a075d6f4a44381ae20a16b8fa5bb2d8c0561bce325ee8a9d99cee59abaad79c8 not found: ID does not exist" Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.566377 5052 scope.go:117] "RemoveContainer" containerID="c38ccb2125893d9107a9365db7e805b861125cfe34c07ee20dd8158df83fc99c" Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.570604 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rwsmp\" (UniqueName: \"kubernetes.io/projected/527bcd50-6d8d-4d4f-bc11-5382eb9adfc2-kube-api-access-rwsmp\") pod \"527bcd50-6d8d-4d4f-bc11-5382eb9adfc2\" (UID: \"527bcd50-6d8d-4d4f-bc11-5382eb9adfc2\") " Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.570678 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dsx9h\" (UniqueName: \"kubernetes.io/projected/1cb3d739-e4d6-4058-9456-a3c2e1c09130-kube-api-access-dsx9h\") pod \"1cb3d739-e4d6-4058-9456-a3c2e1c09130\" (UID: \"1cb3d739-e4d6-4058-9456-a3c2e1c09130\") " Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.570737 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/527bcd50-6d8d-4d4f-bc11-5382eb9adfc2-metrics-certs\") pod \"527bcd50-6d8d-4d4f-bc11-5382eb9adfc2\" (UID: \"527bcd50-6d8d-4d4f-bc11-5382eb9adfc2\") " Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.570762 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6jkbw\" (UniqueName: \"kubernetes.io/projected/a605d632-d9a5-4ce0-9449-e3d57105df1b-kube-api-access-6jkbw\") pod \"a605d632-d9a5-4ce0-9449-e3d57105df1b\" (UID: \"a605d632-d9a5-4ce0-9449-e3d57105df1b\") " Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.570816 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"webhook-certs\" (UniqueName: \"kubernetes.io/secret/527bcd50-6d8d-4d4f-bc11-5382eb9adfc2-webhook-certs\") pod \"527bcd50-6d8d-4d4f-bc11-5382eb9adfc2\" (UID: \"527bcd50-6d8d-4d4f-bc11-5382eb9adfc2\") " Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.571168 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-gxjqd\" (UniqueName: \"kubernetes.io/projected/6bea5f42-82da-4148-9996-dc05121212f7-kube-api-access-gxjqd\") on node \"crc\" DevicePath \"\"" Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.571193 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-fkq98\" (UniqueName: \"kubernetes.io/projected/5968d273-33dd-45cc-aab0-e83628bcd15c-kube-api-access-fkq98\") on node \"crc\" DevicePath \"\"" Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.577791 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/527bcd50-6d8d-4d4f-bc11-5382eb9adfc2-kube-api-access-rwsmp" (OuterVolumeSpecName: "kube-api-access-rwsmp") pod "527bcd50-6d8d-4d4f-bc11-5382eb9adfc2" (UID: "527bcd50-6d8d-4d4f-bc11-5382eb9adfc2"). InnerVolumeSpecName "kube-api-access-rwsmp". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.577964 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a605d632-d9a5-4ce0-9449-e3d57105df1b-kube-api-access-6jkbw" (OuterVolumeSpecName: "kube-api-access-6jkbw") pod "a605d632-d9a5-4ce0-9449-e3d57105df1b" (UID: "a605d632-d9a5-4ce0-9449-e3d57105df1b"). InnerVolumeSpecName "kube-api-access-6jkbw". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.578378 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/527bcd50-6d8d-4d4f-bc11-5382eb9adfc2-webhook-certs" (OuterVolumeSpecName: "webhook-certs") pod "527bcd50-6d8d-4d4f-bc11-5382eb9adfc2" (UID: "527bcd50-6d8d-4d4f-bc11-5382eb9adfc2"). InnerVolumeSpecName "webhook-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.580127 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/527bcd50-6d8d-4d4f-bc11-5382eb9adfc2-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "527bcd50-6d8d-4d4f-bc11-5382eb9adfc2" (UID: "527bcd50-6d8d-4d4f-bc11-5382eb9adfc2"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.584990 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1cb3d739-e4d6-4058-9456-a3c2e1c09130-kube-api-access-dsx9h" (OuterVolumeSpecName: "kube-api-access-dsx9h") pod "1cb3d739-e4d6-4058-9456-a3c2e1c09130" (UID: "1cb3d739-e4d6-4058-9456-a3c2e1c09130"). InnerVolumeSpecName "kube-api-access-dsx9h". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.598697 5052 scope.go:117] "RemoveContainer" containerID="bc9d68e8694514b249866a594a571232d780cb8f8315ced546035025828c5930" Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.622240 5052 scope.go:117] "RemoveContainer" containerID="c38ccb2125893d9107a9365db7e805b861125cfe34c07ee20dd8158df83fc99c" Dec 03 13:11:50 crc kubenswrapper[5052]: E1203 13:11:50.623406 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c38ccb2125893d9107a9365db7e805b861125cfe34c07ee20dd8158df83fc99c\": container with ID starting with c38ccb2125893d9107a9365db7e805b861125cfe34c07ee20dd8158df83fc99c not found: ID does not exist" containerID="c38ccb2125893d9107a9365db7e805b861125cfe34c07ee20dd8158df83fc99c" Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.623461 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c38ccb2125893d9107a9365db7e805b861125cfe34c07ee20dd8158df83fc99c"} err="failed to get container status \"c38ccb2125893d9107a9365db7e805b861125cfe34c07ee20dd8158df83fc99c\": rpc error: code = NotFound desc = could not find container \"c38ccb2125893d9107a9365db7e805b861125cfe34c07ee20dd8158df83fc99c\": container with ID starting with c38ccb2125893d9107a9365db7e805b861125cfe34c07ee20dd8158df83fc99c not found: ID does not exist" Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.623528 5052 scope.go:117] "RemoveContainer" containerID="bc9d68e8694514b249866a594a571232d780cb8f8315ced546035025828c5930" Dec 03 13:11:50 crc kubenswrapper[5052]: E1203 13:11:50.623930 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bc9d68e8694514b249866a594a571232d780cb8f8315ced546035025828c5930\": container with ID starting with bc9d68e8694514b249866a594a571232d780cb8f8315ced546035025828c5930 not found: ID does not exist" containerID="bc9d68e8694514b249866a594a571232d780cb8f8315ced546035025828c5930" Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.623969 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bc9d68e8694514b249866a594a571232d780cb8f8315ced546035025828c5930"} err="failed to get container status \"bc9d68e8694514b249866a594a571232d780cb8f8315ced546035025828c5930\": rpc error: code = NotFound desc = could not find container \"bc9d68e8694514b249866a594a571232d780cb8f8315ced546035025828c5930\": container with ID starting with bc9d68e8694514b249866a594a571232d780cb8f8315ced546035025828c5930 not found: ID does not exist" Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.672696 5052 reconciler_common.go:293] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/527bcd50-6d8d-4d4f-bc11-5382eb9adfc2-metrics-certs\") on node \"crc\" DevicePath \"\"" Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.673445 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-6jkbw\" (UniqueName: \"kubernetes.io/projected/a605d632-d9a5-4ce0-9449-e3d57105df1b-kube-api-access-6jkbw\") on node \"crc\" DevicePath \"\"" Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.673627 5052 reconciler_common.go:293] "Volume detached for volume \"webhook-certs\" (UniqueName: \"kubernetes.io/secret/527bcd50-6d8d-4d4f-bc11-5382eb9adfc2-webhook-certs\") on node \"crc\" DevicePath \"\"" Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.673647 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-rwsmp\" (UniqueName: \"kubernetes.io/projected/527bcd50-6d8d-4d4f-bc11-5382eb9adfc2-kube-api-access-rwsmp\") on node \"crc\" DevicePath \"\"" Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.673662 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-dsx9h\" (UniqueName: \"kubernetes.io/projected/1cb3d739-e4d6-4058-9456-a3c2e1c09130-kube-api-access-dsx9h\") on node \"crc\" DevicePath \"\"" Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.798636 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-operators/openstack-operator-controller-manager-9f56fc979-b8jv8" Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.798652 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/openstack-operator-controller-manager-9f56fc979-b8jv8" event={"ID":"527bcd50-6d8d-4d4f-bc11-5382eb9adfc2","Type":"ContainerDied","Data":"539dfe13c0fa9889c9112a3f1df72e1767185682a73bbc446b193911a1924168"} Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.798732 5052 scope.go:117] "RemoveContainer" containerID="d1bb936db1c98c58f4e109008a5dc7d293c9a04b310fd4b2ab5bb2e70449c5aa" Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.805943 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/watcher-operator-controller-manager-769dc69bc-7rhg6" event={"ID":"6bea5f42-82da-4148-9996-dc05121212f7","Type":"ContainerDied","Data":"1fd4785ca7f9f26faf2ff9815a8f586e287d376cfb561cb9cd6dc321b0639c61"} Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.806056 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-operators/watcher-operator-controller-manager-769dc69bc-7rhg6" Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.812007 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-operators/openstack-baremetal-operator-controller-manager-55d86b6686f2mft" Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.812024 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/openstack-baremetal-operator-controller-manager-55d86b6686f2mft" event={"ID":"e10da887-70df-43aa-aa79-4a14231e5ec9","Type":"ContainerDied","Data":"7e31de0787f492c6adfc9a9e6a17a8682798e05059001ae1d2b2235f6c963d16"} Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.816814 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-operators/ovn-operator-controller-manager-b6456fdb6-sggpb" Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.817144 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/ovn-operator-controller-manager-b6456fdb6-sggpb" event={"ID":"a605d632-d9a5-4ce0-9449-e3d57105df1b","Type":"ContainerDied","Data":"1a9cebed54caff6e9a56f1b8ace88fd42d7b461be12f6905f9f8e7708d196205"} Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.818986 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-operators/nova-operator-controller-manager-697bc559fc-gcj5d" Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.818983 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/nova-operator-controller-manager-697bc559fc-gcj5d" event={"ID":"3e6bc3f6-1f1a-4b99-8835-98252369d364","Type":"ContainerDied","Data":"7045ff1c243a7bb725866b1b3e3247264c4aacef017a1c00ffd6fbcd148e25d4"} Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.822707 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/infra-operator-controller-manager-57548d458d-9fkll" event={"ID":"81f5f8e1-015a-4b45-83e6-13e88efb8e80","Type":"ContainerDied","Data":"9e9b4a8947d10fcefa968eaeff05f6115cfd21c0b57f83374c28eb0d6a26a01b"} Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.822735 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-operators/infra-operator-controller-manager-57548d458d-9fkll" Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.825764 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/glance-operator-controller-manager-77987cd8cd-btn2l" event={"ID":"d429f7ab-9a88-4411-a7a5-88bc8ab76285","Type":"ContainerDied","Data":"5a62234bd4ac90688320976093ff7b70fc1912755385a234ae0df8cfe617705e"} Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.825780 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-operators/glance-operator-controller-manager-77987cd8cd-btn2l" Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.831267 5052 scope.go:117] "RemoveContainer" containerID="7145c8d45908da236459c9313b6926c516ea98b7ba253967ef2007aca4397a66" Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.831807 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/rabbitmq-cluster-operator-manager-668c99d594-ksw2p" event={"ID":"459f5555-efae-4c19-84d0-2af917b9560f","Type":"ContainerDied","Data":"07c1349b2f26f45310447aa0bc4ba69d47ba46de5abeb65a7707c18910c76842"} Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.831917 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-operators/rabbitmq-cluster-operator-manager-668c99d594-ksw2p" Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.845439 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-operators/openstack-operator-controller-manager-9f56fc979-b8jv8"] Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.847295 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/barbican-operator-controller-manager-7d9dfd778-cqfdk" event={"ID":"8dbd25a1-9de9-431e-be37-d239e3f64728","Type":"ContainerDied","Data":"1a2e5fff53f0ca0a4391f1595e6092df6dd19f5f6bd8eafc64d716bf92687cc8"} Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.847394 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-operators/barbican-operator-controller-manager-7d9dfd778-cqfdk" Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.852460 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-operators/openstack-operator-controller-manager-9f56fc979-b8jv8"] Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.852662 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-operators/openstack-operator-controller-operator-7dd5c7bb7c-rjflp" Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.852885 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/openstack-operator-controller-operator-7dd5c7bb7c-rjflp" event={"ID":"cd2c5c4c-7ea8-4f95-9556-58b87d14a2be","Type":"ContainerDied","Data":"a34afd81a3a9a587cceb4cb7cd37efc7470c49f3210edaf5369224a9cae78e67"} Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.865558 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/cinder-operator-controller-manager-859b6ccc6-vt9d2" event={"ID":"5968d273-33dd-45cc-aab0-e83628bcd15c","Type":"ContainerDied","Data":"4767fb81ab0a91c75523ffdadd4d3c055b86bd200dce767ffc908e79f3bc07b6"} Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.865674 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-operators/cinder-operator-controller-manager-859b6ccc6-vt9d2" Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.879882 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-operators/watcher-operator-controller-manager-769dc69bc-7rhg6"] Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.880911 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-operators/test-operator-controller-manager-5854674fcc-jltb6" Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.881108 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/test-operator-controller-manager-5854674fcc-jltb6" event={"ID":"1cb3d739-e4d6-4058-9456-a3c2e1c09130","Type":"ContainerDied","Data":"b3f6f7a69f6afa8cf29e3a2449acca972f8066a53aa83808f7f32f9d8477b2e9"} Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.883812 5052 scope.go:117] "RemoveContainer" containerID="af5bcb24f3ea207a68cadd9479e8bc16e424b9d9fac437c1e5ea1ef07af38d6d" Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.885804 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/placement-operator-controller-manager-78f8948974-j62hd" event={"ID":"37ab421b-0ad3-4d9d-bf47-8bf908c4aa57","Type":"ContainerDied","Data":"b30889a9adcb74865c1c9d647128b6159f40569f2b1f1e64751965e5079962e2"} Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.886008 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-operators/placement-operator-controller-manager-78f8948974-j62hd" Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.896447 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-operators/watcher-operator-controller-manager-769dc69bc-7rhg6"] Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.896916 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-operators/openstack-operator-index-x6glm" Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.896983 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/openstack-operator-index-x6glm" event={"ID":"6fe52b82-e00b-4c40-8ba1-bb52e7897667","Type":"ContainerDied","Data":"ec3b6d0b8463f153bb7a7b108aac3ac84c5a218111f60f535adc1948f70e1e20"} Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.908498 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-operators/heat-operator-controller-manager-5f64f6f8bb-48fnb" Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.908559 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/heat-operator-controller-manager-5f64f6f8bb-48fnb" event={"ID":"0cbd234c-ecb3-4e54-af86-a2b15fdd94a3","Type":"ContainerDied","Data":"64418c91a2ae67f0561b20b32b426ab61c7e64d86df77e00d7495ca6dec54ebb"} Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.935020 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-operators/telemetry-operator-controller-manager-76cc84c6bb-zl7px" Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.937126 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-operators/mariadb-operator-controller-manager-56bbcc9d85-sr7zw" Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.959054 5052 scope.go:117] "RemoveContainer" containerID="0f00b4f3db71336942e4ffb2969c9f402b3aa40103a5ee65081fd36ca18be57d" Dec 03 13:11:50 crc kubenswrapper[5052]: I1203 13:11:50.978275 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-operators/ovn-operator-controller-manager-b6456fdb6-sggpb"] Dec 03 13:11:51 crc kubenswrapper[5052]: I1203 13:11:51.002394 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-operators/ovn-operator-controller-manager-b6456fdb6-sggpb"] Dec 03 13:11:51 crc kubenswrapper[5052]: I1203 13:11:51.005334 5052 scope.go:117] "RemoveContainer" containerID="0f961febcff89cbc4c5cd57e2535ba7fe24e5f74a9bf78788782f9c8e2c423d5" Dec 03 13:11:51 crc kubenswrapper[5052]: I1203 13:11:51.015215 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-operators/nova-operator-controller-manager-697bc559fc-gcj5d"] Dec 03 13:11:51 crc kubenswrapper[5052]: I1203 13:11:51.025007 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-operators/nova-operator-controller-manager-697bc559fc-gcj5d"] Dec 03 13:11:51 crc kubenswrapper[5052]: I1203 13:11:51.031846 5052 scope.go:117] "RemoveContainer" containerID="6e810659ae074c724982667dc34e29af08df25d8aa94eea774ad9e2b2ba3b241" Dec 03 13:11:51 crc kubenswrapper[5052]: I1203 13:11:51.038736 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-operators/test-operator-controller-manager-5854674fcc-jltb6"] Dec 03 13:11:51 crc kubenswrapper[5052]: I1203 13:11:51.047717 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-operators/test-operator-controller-manager-5854674fcc-jltb6"] Dec 03 13:11:51 crc kubenswrapper[5052]: I1203 13:11:51.054597 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-operators/openstack-operator-index-x6glm"] Dec 03 13:11:51 crc kubenswrapper[5052]: I1203 13:11:51.061325 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-operators/openstack-operator-index-x6glm"] Dec 03 13:11:51 crc kubenswrapper[5052]: I1203 13:11:51.062558 5052 scope.go:117] "RemoveContainer" containerID="2944fe91598c6c5f038d1de9c8551c8352a4c6b145016c6151bfc37b330edddd" Dec 03 13:11:51 crc kubenswrapper[5052]: I1203 13:11:51.069180 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-operators/openstack-operator-controller-operator-7dd5c7bb7c-rjflp"] Dec 03 13:11:51 crc kubenswrapper[5052]: I1203 13:11:51.076155 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-operators/openstack-operator-controller-operator-7dd5c7bb7c-rjflp"] Dec 03 13:11:51 crc kubenswrapper[5052]: I1203 13:11:51.080148 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-operators/glance-operator-controller-manager-77987cd8cd-btn2l"] Dec 03 13:11:51 crc kubenswrapper[5052]: I1203 13:11:51.086448 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-operators/glance-operator-controller-manager-77987cd8cd-btn2l"] Dec 03 13:11:51 crc kubenswrapper[5052]: I1203 13:11:51.093322 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-operators/infra-operator-controller-manager-57548d458d-9fkll"] Dec 03 13:11:51 crc kubenswrapper[5052]: I1203 13:11:51.098802 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-operators/infra-operator-controller-manager-57548d458d-9fkll"] Dec 03 13:11:51 crc kubenswrapper[5052]: I1203 13:11:51.101802 5052 scope.go:117] "RemoveContainer" containerID="a7e939b334d0e92f65588cb0c22e7536f6716a1e32472cd5033b456da2b36a68" Dec 03 13:11:51 crc kubenswrapper[5052]: I1203 13:11:51.107165 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-operators/cinder-operator-controller-manager-859b6ccc6-vt9d2"] Dec 03 13:11:51 crc kubenswrapper[5052]: I1203 13:11:51.114924 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-operators/cinder-operator-controller-manager-859b6ccc6-vt9d2"] Dec 03 13:11:51 crc kubenswrapper[5052]: I1203 13:11:51.123487 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-operators/barbican-operator-controller-manager-7d9dfd778-cqfdk"] Dec 03 13:11:51 crc kubenswrapper[5052]: I1203 13:11:51.130397 5052 scope.go:117] "RemoveContainer" containerID="21bac89f97ac17aa04e4ad5d34fe06b056702b1896eabe8f69dc6ad5ebbc42bf" Dec 03 13:11:51 crc kubenswrapper[5052]: I1203 13:11:51.135601 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-operators/barbican-operator-controller-manager-7d9dfd778-cqfdk"] Dec 03 13:11:51 crc kubenswrapper[5052]: I1203 13:11:51.142490 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-operators/openstack-baremetal-operator-controller-manager-55d86b6686f2mft"] Dec 03 13:11:51 crc kubenswrapper[5052]: I1203 13:11:51.148319 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-operators/openstack-baremetal-operator-controller-manager-55d86b6686f2mft"] Dec 03 13:11:51 crc kubenswrapper[5052]: I1203 13:11:51.155182 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-operators/mariadb-operator-controller-manager-56bbcc9d85-sr7zw"] Dec 03 13:11:51 crc kubenswrapper[5052]: I1203 13:11:51.156017 5052 scope.go:117] "RemoveContainer" containerID="01a4046ea27a459acec0b9a6a2a20c352ddcb26e0f79bddb80874932462702e9" Dec 03 13:11:51 crc kubenswrapper[5052]: I1203 13:11:51.162675 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-operators/mariadb-operator-controller-manager-56bbcc9d85-sr7zw"] Dec 03 13:11:51 crc kubenswrapper[5052]: I1203 13:11:51.169915 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-operators/telemetry-operator-controller-manager-76cc84c6bb-zl7px"] Dec 03 13:11:51 crc kubenswrapper[5052]: I1203 13:11:51.181526 5052 scope.go:117] "RemoveContainer" containerID="43ae682827f3b8fe92dd61ac021bde418f76e0a34a72497c150c64083536410d" Dec 03 13:11:51 crc kubenswrapper[5052]: I1203 13:11:51.181547 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-operators/telemetry-operator-controller-manager-76cc84c6bb-zl7px"] Dec 03 13:11:51 crc kubenswrapper[5052]: I1203 13:11:51.191382 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-operators/placement-operator-controller-manager-78f8948974-j62hd"] Dec 03 13:11:51 crc kubenswrapper[5052]: I1203 13:11:51.201834 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-operators/placement-operator-controller-manager-78f8948974-j62hd"] Dec 03 13:11:51 crc kubenswrapper[5052]: I1203 13:11:51.204166 5052 scope.go:117] "RemoveContainer" containerID="cd378be1b828198fb7e23f2fd50570c21a7bd0e958e5f4f92497cb7ebe420226" Dec 03 13:11:51 crc kubenswrapper[5052]: I1203 13:11:51.204205 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-operators/rabbitmq-cluster-operator-manager-668c99d594-ksw2p"] Dec 03 13:11:51 crc kubenswrapper[5052]: I1203 13:11:51.209602 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-operators/rabbitmq-cluster-operator-manager-668c99d594-ksw2p"] Dec 03 13:11:51 crc kubenswrapper[5052]: I1203 13:11:51.214635 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-operators/heat-operator-controller-manager-5f64f6f8bb-48fnb"] Dec 03 13:11:51 crc kubenswrapper[5052]: I1203 13:11:51.220192 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-operators/heat-operator-controller-manager-5f64f6f8bb-48fnb"] Dec 03 13:11:51 crc kubenswrapper[5052]: I1203 13:11:51.222553 5052 scope.go:117] "RemoveContainer" containerID="27a2c0f47320ac8e0e4f631d691ed2469ebf4f27f0dad96adcf0fcb2e224d156" Dec 03 13:11:51 crc kubenswrapper[5052]: I1203 13:11:51.242056 5052 scope.go:117] "RemoveContainer" containerID="9f7f47624da378a4c9e182afc6f0245cb94a2c07414a47ce8f104bcb13883f17" Dec 03 13:11:51 crc kubenswrapper[5052]: I1203 13:11:51.264919 5052 scope.go:117] "RemoveContainer" containerID="754a05a1ad89a85b548ec6034299b8041b571878c20c15a1d29f8a87a0d19577" Dec 03 13:11:51 crc kubenswrapper[5052]: I1203 13:11:51.288244 5052 scope.go:117] "RemoveContainer" containerID="5025f6eb197bd7384ff4dad6b52992f6489ac18de5c05c2535a9026ee37b6f24" Dec 03 13:11:51 crc kubenswrapper[5052]: I1203 13:11:51.312414 5052 scope.go:117] "RemoveContainer" containerID="5a4c5c6a86ebb9b257e5e311d545f7f9899b5f426c04b2d6f5217d3cce6845e8" Dec 03 13:11:51 crc kubenswrapper[5052]: I1203 13:11:51.352096 5052 scope.go:117] "RemoveContainer" containerID="cd51be9345abb5eafe858033c68a1ced7d4100915afaf161d165bcac5ef6661e" Dec 03 13:11:51 crc kubenswrapper[5052]: I1203 13:11:51.372179 5052 scope.go:117] "RemoveContainer" containerID="e020c88c64940fc2399454d010fff5531ddc1320fccf47d9835db05c4424f1b2" Dec 03 13:11:51 crc kubenswrapper[5052]: I1203 13:11:51.393012 5052 scope.go:117] "RemoveContainer" containerID="dabaf7e10aa7d0b0ec06b6b8e0023ae509392bd5567d994a850b73988ff68ec4" Dec 03 13:11:51 crc kubenswrapper[5052]: I1203 13:11:51.412899 5052 scope.go:117] "RemoveContainer" containerID="7f760d28d8a9a27373a0d324cb077f28fc6b0310ae54ade63b76d7f19979ccad" Dec 03 13:11:51 crc kubenswrapper[5052]: I1203 13:11:51.434422 5052 scope.go:117] "RemoveContainer" containerID="2dcc207b577f508a2a51aeed06eda37f7a37cdaa339bc4bd5b4f2712d43ec227" Dec 03 13:11:51 crc kubenswrapper[5052]: I1203 13:11:51.457035 5052 scope.go:117] "RemoveContainer" containerID="e3943739460d5bb48bf27023494d0f23c5ff3cd0f58f04e59d49d0dbc05a7830" Dec 03 13:11:51 crc kubenswrapper[5052]: I1203 13:11:51.477088 5052 scope.go:117] "RemoveContainer" containerID="409bd09ed8d8c06816bb54f36a222437bc97be11de3a87273fc8a642cd7acdb8" Dec 03 13:11:51 crc kubenswrapper[5052]: I1203 13:11:51.502735 5052 scope.go:117] "RemoveContainer" containerID="ac6e6a21862aaede93ccb524e481d0b5e6a4f1f2e70966ac0cb19106016bbeef" Dec 03 13:11:51 crc kubenswrapper[5052]: I1203 13:11:51.535727 5052 scope.go:117] "RemoveContainer" containerID="05da1e86576897a06ee62b48ffa3acc05751be603b91af013f2fc3b98094a039" Dec 03 13:11:51 crc kubenswrapper[5052]: I1203 13:11:51.673545 5052 scope.go:117] "RemoveContainer" containerID="dbc100c8081ea1a2f5487429cc9d67de6bc8ba11282b64f248c71275375b0e16" Dec 03 13:11:51 crc kubenswrapper[5052]: E1203 13:11:51.673832 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 13:11:51 crc kubenswrapper[5052]: I1203 13:11:51.684231 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0cbd234c-ecb3-4e54-af86-a2b15fdd94a3" path="/var/lib/kubelet/pods/0cbd234c-ecb3-4e54-af86-a2b15fdd94a3/volumes" Dec 03 13:11:51 crc kubenswrapper[5052]: I1203 13:11:51.684873 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1cb3d739-e4d6-4058-9456-a3c2e1c09130" path="/var/lib/kubelet/pods/1cb3d739-e4d6-4058-9456-a3c2e1c09130/volumes" Dec 03 13:11:51 crc kubenswrapper[5052]: I1203 13:11:51.685346 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2e2dc077-c02f-4f98-8fa6-9f72890faeaf" path="/var/lib/kubelet/pods/2e2dc077-c02f-4f98-8fa6-9f72890faeaf/volumes" Dec 03 13:11:51 crc kubenswrapper[5052]: I1203 13:11:51.690182 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="37ab421b-0ad3-4d9d-bf47-8bf908c4aa57" path="/var/lib/kubelet/pods/37ab421b-0ad3-4d9d-bf47-8bf908c4aa57/volumes" Dec 03 13:11:51 crc kubenswrapper[5052]: I1203 13:11:51.691777 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3894016b-8242-4b12-8407-bcf78af00851" path="/var/lib/kubelet/pods/3894016b-8242-4b12-8407-bcf78af00851/volumes" Dec 03 13:11:51 crc kubenswrapper[5052]: I1203 13:11:51.692876 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3e6bc3f6-1f1a-4b99-8835-98252369d364" path="/var/lib/kubelet/pods/3e6bc3f6-1f1a-4b99-8835-98252369d364/volumes" Dec 03 13:11:51 crc kubenswrapper[5052]: I1203 13:11:51.695832 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="459f5555-efae-4c19-84d0-2af917b9560f" path="/var/lib/kubelet/pods/459f5555-efae-4c19-84d0-2af917b9560f/volumes" Dec 03 13:11:51 crc kubenswrapper[5052]: I1203 13:11:51.698063 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="50c6e918-7f1f-4d6f-ba5f-940e5f485825" path="/var/lib/kubelet/pods/50c6e918-7f1f-4d6f-ba5f-940e5f485825/volumes" Dec 03 13:11:51 crc kubenswrapper[5052]: I1203 13:11:51.699212 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="527bcd50-6d8d-4d4f-bc11-5382eb9adfc2" path="/var/lib/kubelet/pods/527bcd50-6d8d-4d4f-bc11-5382eb9adfc2/volumes" Dec 03 13:11:51 crc kubenswrapper[5052]: I1203 13:11:51.700412 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="53c88ca3-39f7-4a81-b305-40047bcaeb88" path="/var/lib/kubelet/pods/53c88ca3-39f7-4a81-b305-40047bcaeb88/volumes" Dec 03 13:11:51 crc kubenswrapper[5052]: I1203 13:11:51.702002 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5968d273-33dd-45cc-aab0-e83628bcd15c" path="/var/lib/kubelet/pods/5968d273-33dd-45cc-aab0-e83628bcd15c/volumes" Dec 03 13:11:51 crc kubenswrapper[5052]: I1203 13:11:51.704711 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6bea5f42-82da-4148-9996-dc05121212f7" path="/var/lib/kubelet/pods/6bea5f42-82da-4148-9996-dc05121212f7/volumes" Dec 03 13:11:51 crc kubenswrapper[5052]: I1203 13:11:51.706849 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6fe52b82-e00b-4c40-8ba1-bb52e7897667" path="/var/lib/kubelet/pods/6fe52b82-e00b-4c40-8ba1-bb52e7897667/volumes" Dec 03 13:11:51 crc kubenswrapper[5052]: I1203 13:11:51.708052 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7017b3f7-1eb9-4dfe-8ef3-6db27cf3a1f1" path="/var/lib/kubelet/pods/7017b3f7-1eb9-4dfe-8ef3-6db27cf3a1f1/volumes" Dec 03 13:11:51 crc kubenswrapper[5052]: I1203 13:11:51.710225 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="81f5f8e1-015a-4b45-83e6-13e88efb8e80" path="/var/lib/kubelet/pods/81f5f8e1-015a-4b45-83e6-13e88efb8e80/volumes" Dec 03 13:11:51 crc kubenswrapper[5052]: I1203 13:11:51.710819 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8dbd25a1-9de9-431e-be37-d239e3f64728" path="/var/lib/kubelet/pods/8dbd25a1-9de9-431e-be37-d239e3f64728/volumes" Dec 03 13:11:51 crc kubenswrapper[5052]: I1203 13:11:51.712279 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8fd08931-7f0b-4d5b-bfb1-f27980648561" path="/var/lib/kubelet/pods/8fd08931-7f0b-4d5b-bfb1-f27980648561/volumes" Dec 03 13:11:51 crc kubenswrapper[5052]: I1203 13:11:51.714984 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="95b50980-fd5a-4a19-b1cd-13df98d257fe" path="/var/lib/kubelet/pods/95b50980-fd5a-4a19-b1cd-13df98d257fe/volumes" Dec 03 13:11:51 crc kubenswrapper[5052]: I1203 13:11:51.716270 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a605d632-d9a5-4ce0-9449-e3d57105df1b" path="/var/lib/kubelet/pods/a605d632-d9a5-4ce0-9449-e3d57105df1b/volumes" Dec 03 13:11:51 crc kubenswrapper[5052]: I1203 13:11:51.716863 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cd2c5c4c-7ea8-4f95-9556-58b87d14a2be" path="/var/lib/kubelet/pods/cd2c5c4c-7ea8-4f95-9556-58b87d14a2be/volumes" Dec 03 13:11:51 crc kubenswrapper[5052]: I1203 13:11:51.719271 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ce3d34f6-e545-4eb2-bcc5-999377931601" path="/var/lib/kubelet/pods/ce3d34f6-e545-4eb2-bcc5-999377931601/volumes" Dec 03 13:11:51 crc kubenswrapper[5052]: I1203 13:11:51.719951 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d429f7ab-9a88-4411-a7a5-88bc8ab76285" path="/var/lib/kubelet/pods/d429f7ab-9a88-4411-a7a5-88bc8ab76285/volumes" Dec 03 13:11:51 crc kubenswrapper[5052]: I1203 13:11:51.720534 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e10da887-70df-43aa-aa79-4a14231e5ec9" path="/var/lib/kubelet/pods/e10da887-70df-43aa-aa79-4a14231e5ec9/volumes" Dec 03 13:11:51 crc kubenswrapper[5052]: I1203 13:11:51.722021 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f0e70c0d-10fe-43b7-a08c-0ce411f223a6" path="/var/lib/kubelet/pods/f0e70c0d-10fe-43b7-a08c-0ce411f223a6/volumes" Dec 03 13:11:54 crc kubenswrapper[5052]: I1203 13:11:54.242811 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-84b9f45d47-d8nh7" Dec 03 13:11:54 crc kubenswrapper[5052]: I1203 13:11:54.308504 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-64864b6d57-wjkrf"] Dec 03 13:11:54 crc kubenswrapper[5052]: I1203 13:11:54.309189 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-64864b6d57-wjkrf" podUID="cde663cf-e7e2-42b1-9581-ea7a27f43481" containerName="dnsmasq-dns" containerID="cri-o://d5863db69cdfdebc8e10ce524ecd40405cc19e2ff443d5a3b4bfd94d1146db1c" gracePeriod=10 Dec 03 13:11:54 crc kubenswrapper[5052]: I1203 13:11:54.713360 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-64864b6d57-wjkrf" Dec 03 13:11:54 crc kubenswrapper[5052]: I1203 13:11:54.855736 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/cde663cf-e7e2-42b1-9581-ea7a27f43481-dnsmasq-svc\") pod \"cde663cf-e7e2-42b1-9581-ea7a27f43481\" (UID: \"cde663cf-e7e2-42b1-9581-ea7a27f43481\") " Dec 03 13:11:54 crc kubenswrapper[5052]: I1203 13:11:54.855846 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dlrnr\" (UniqueName: \"kubernetes.io/projected/cde663cf-e7e2-42b1-9581-ea7a27f43481-kube-api-access-dlrnr\") pod \"cde663cf-e7e2-42b1-9581-ea7a27f43481\" (UID: \"cde663cf-e7e2-42b1-9581-ea7a27f43481\") " Dec 03 13:11:54 crc kubenswrapper[5052]: I1203 13:11:54.855958 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/cde663cf-e7e2-42b1-9581-ea7a27f43481-config\") pod \"cde663cf-e7e2-42b1-9581-ea7a27f43481\" (UID: \"cde663cf-e7e2-42b1-9581-ea7a27f43481\") " Dec 03 13:11:54 crc kubenswrapper[5052]: I1203 13:11:54.856012 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"edpm-compute-no-nodes\" (UniqueName: \"kubernetes.io/configmap/cde663cf-e7e2-42b1-9581-ea7a27f43481-edpm-compute-no-nodes\") pod \"cde663cf-e7e2-42b1-9581-ea7a27f43481\" (UID: \"cde663cf-e7e2-42b1-9581-ea7a27f43481\") " Dec 03 13:11:54 crc kubenswrapper[5052]: I1203 13:11:54.878753 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cde663cf-e7e2-42b1-9581-ea7a27f43481-kube-api-access-dlrnr" (OuterVolumeSpecName: "kube-api-access-dlrnr") pod "cde663cf-e7e2-42b1-9581-ea7a27f43481" (UID: "cde663cf-e7e2-42b1-9581-ea7a27f43481"). InnerVolumeSpecName "kube-api-access-dlrnr". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:11:54 crc kubenswrapper[5052]: I1203 13:11:54.911128 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cde663cf-e7e2-42b1-9581-ea7a27f43481-config" (OuterVolumeSpecName: "config") pod "cde663cf-e7e2-42b1-9581-ea7a27f43481" (UID: "cde663cf-e7e2-42b1-9581-ea7a27f43481"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:11:54 crc kubenswrapper[5052]: I1203 13:11:54.939932 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cde663cf-e7e2-42b1-9581-ea7a27f43481-edpm-compute-no-nodes" (OuterVolumeSpecName: "edpm-compute-no-nodes") pod "cde663cf-e7e2-42b1-9581-ea7a27f43481" (UID: "cde663cf-e7e2-42b1-9581-ea7a27f43481"). InnerVolumeSpecName "edpm-compute-no-nodes". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:11:54 crc kubenswrapper[5052]: I1203 13:11:54.942889 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cde663cf-e7e2-42b1-9581-ea7a27f43481-dnsmasq-svc" (OuterVolumeSpecName: "dnsmasq-svc") pod "cde663cf-e7e2-42b1-9581-ea7a27f43481" (UID: "cde663cf-e7e2-42b1-9581-ea7a27f43481"). InnerVolumeSpecName "dnsmasq-svc". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:11:54 crc kubenswrapper[5052]: I1203 13:11:54.958456 5052 reconciler_common.go:293] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/cde663cf-e7e2-42b1-9581-ea7a27f43481-config\") on node \"crc\" DevicePath \"\"" Dec 03 13:11:54 crc kubenswrapper[5052]: I1203 13:11:54.958529 5052 reconciler_common.go:293] "Volume detached for volume \"edpm-compute-no-nodes\" (UniqueName: \"kubernetes.io/configmap/cde663cf-e7e2-42b1-9581-ea7a27f43481-edpm-compute-no-nodes\") on node \"crc\" DevicePath \"\"" Dec 03 13:11:54 crc kubenswrapper[5052]: I1203 13:11:54.958545 5052 reconciler_common.go:293] "Volume detached for volume \"dnsmasq-svc\" (UniqueName: \"kubernetes.io/configmap/cde663cf-e7e2-42b1-9581-ea7a27f43481-dnsmasq-svc\") on node \"crc\" DevicePath \"\"" Dec 03 13:11:54 crc kubenswrapper[5052]: I1203 13:11:54.958558 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-dlrnr\" (UniqueName: \"kubernetes.io/projected/cde663cf-e7e2-42b1-9581-ea7a27f43481-kube-api-access-dlrnr\") on node \"crc\" DevicePath \"\"" Dec 03 13:11:54 crc kubenswrapper[5052]: I1203 13:11:54.989624 5052 generic.go:334] "Generic (PLEG): container finished" podID="cde663cf-e7e2-42b1-9581-ea7a27f43481" containerID="d5863db69cdfdebc8e10ce524ecd40405cc19e2ff443d5a3b4bfd94d1146db1c" exitCode=0 Dec 03 13:11:54 crc kubenswrapper[5052]: I1203 13:11:54.989674 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-64864b6d57-wjkrf" event={"ID":"cde663cf-e7e2-42b1-9581-ea7a27f43481","Type":"ContainerDied","Data":"d5863db69cdfdebc8e10ce524ecd40405cc19e2ff443d5a3b4bfd94d1146db1c"} Dec 03 13:11:54 crc kubenswrapper[5052]: I1203 13:11:54.989685 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-64864b6d57-wjkrf" Dec 03 13:11:54 crc kubenswrapper[5052]: I1203 13:11:54.989702 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-kuttl-tests/dnsmasq-dnsmasq-64864b6d57-wjkrf" event={"ID":"cde663cf-e7e2-42b1-9581-ea7a27f43481","Type":"ContainerDied","Data":"b21d4232aee992bb16d83c0aac5a2d0722d84c925a5f4eebe000444a287fb4d0"} Dec 03 13:11:54 crc kubenswrapper[5052]: I1203 13:11:54.989719 5052 scope.go:117] "RemoveContainer" containerID="d5863db69cdfdebc8e10ce524ecd40405cc19e2ff443d5a3b4bfd94d1146db1c" Dec 03 13:11:55 crc kubenswrapper[5052]: I1203 13:11:55.021273 5052 scope.go:117] "RemoveContainer" containerID="0d231e87cdac7a1441d49513b13649a70fdacb9619a3e661990497f875952595" Dec 03 13:11:55 crc kubenswrapper[5052]: I1203 13:11:55.039623 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-64864b6d57-wjkrf"] Dec 03 13:11:55 crc kubenswrapper[5052]: I1203 13:11:55.044450 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-kuttl-tests/dnsmasq-dnsmasq-64864b6d57-wjkrf"] Dec 03 13:11:55 crc kubenswrapper[5052]: I1203 13:11:55.072596 5052 scope.go:117] "RemoveContainer" containerID="d5863db69cdfdebc8e10ce524ecd40405cc19e2ff443d5a3b4bfd94d1146db1c" Dec 03 13:11:55 crc kubenswrapper[5052]: E1203 13:11:55.073223 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d5863db69cdfdebc8e10ce524ecd40405cc19e2ff443d5a3b4bfd94d1146db1c\": container with ID starting with d5863db69cdfdebc8e10ce524ecd40405cc19e2ff443d5a3b4bfd94d1146db1c not found: ID does not exist" containerID="d5863db69cdfdebc8e10ce524ecd40405cc19e2ff443d5a3b4bfd94d1146db1c" Dec 03 13:11:55 crc kubenswrapper[5052]: I1203 13:11:55.073280 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d5863db69cdfdebc8e10ce524ecd40405cc19e2ff443d5a3b4bfd94d1146db1c"} err="failed to get container status \"d5863db69cdfdebc8e10ce524ecd40405cc19e2ff443d5a3b4bfd94d1146db1c\": rpc error: code = NotFound desc = could not find container \"d5863db69cdfdebc8e10ce524ecd40405cc19e2ff443d5a3b4bfd94d1146db1c\": container with ID starting with d5863db69cdfdebc8e10ce524ecd40405cc19e2ff443d5a3b4bfd94d1146db1c not found: ID does not exist" Dec 03 13:11:55 crc kubenswrapper[5052]: I1203 13:11:55.073308 5052 scope.go:117] "RemoveContainer" containerID="0d231e87cdac7a1441d49513b13649a70fdacb9619a3e661990497f875952595" Dec 03 13:11:55 crc kubenswrapper[5052]: E1203 13:11:55.073771 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0d231e87cdac7a1441d49513b13649a70fdacb9619a3e661990497f875952595\": container with ID starting with 0d231e87cdac7a1441d49513b13649a70fdacb9619a3e661990497f875952595 not found: ID does not exist" containerID="0d231e87cdac7a1441d49513b13649a70fdacb9619a3e661990497f875952595" Dec 03 13:11:55 crc kubenswrapper[5052]: I1203 13:11:55.073936 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0d231e87cdac7a1441d49513b13649a70fdacb9619a3e661990497f875952595"} err="failed to get container status \"0d231e87cdac7a1441d49513b13649a70fdacb9619a3e661990497f875952595\": rpc error: code = NotFound desc = could not find container \"0d231e87cdac7a1441d49513b13649a70fdacb9619a3e661990497f875952595\": container with ID starting with 0d231e87cdac7a1441d49513b13649a70fdacb9619a3e661990497f875952595 not found: ID does not exist" Dec 03 13:11:55 crc kubenswrapper[5052]: I1203 13:11:55.680926 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cde663cf-e7e2-42b1-9581-ea7a27f43481" path="/var/lib/kubelet/pods/cde663cf-e7e2-42b1-9581-ea7a27f43481/volumes" Dec 03 13:12:02 crc kubenswrapper[5052]: I1203 13:12:02.672559 5052 scope.go:117] "RemoveContainer" containerID="dbc100c8081ea1a2f5487429cc9d67de6bc8ba11282b64f248c71275375b0e16" Dec 03 13:12:03 crc kubenswrapper[5052]: I1203 13:12:03.068451 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" event={"ID":"10c237bc-72e7-4da3-9f2c-b1688395bf6e","Type":"ContainerStarted","Data":"059812316dc85d1da8d022c96bdd889e4f16ae597afc7241b51949522378de3a"} Dec 03 13:12:20 crc kubenswrapper[5052]: I1203 13:12:20.181263 5052 pod_container_manager_linux.go:210] "Failed to delete cgroup paths" cgroupName=["kubepods","burstable","podea1e91de-7e7d-498a-a17b-551f2bee325a"] err="unable to destroy cgroup paths for cgroup [kubepods burstable podea1e91de-7e7d-498a-a17b-551f2bee325a] : Timed out while waiting for systemd to remove kubepods-burstable-podea1e91de_7e7d_498a_a17b_551f2bee325a.slice" Dec 03 13:12:20 crc kubenswrapper[5052]: E1203 13:12:20.182626 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to delete cgroup paths for [kubepods burstable podea1e91de-7e7d-498a-a17b-551f2bee325a] : unable to destroy cgroup paths for cgroup [kubepods burstable podea1e91de-7e7d-498a-a17b-551f2bee325a] : Timed out while waiting for systemd to remove kubepods-burstable-podea1e91de_7e7d_498a_a17b_551f2bee325a.slice" pod="openstack-operators/manila-operator-controller-manager-7c79b5df47-ccbqq" podUID="ea1e91de-7e7d-498a-a17b-551f2bee325a" Dec 03 13:12:20 crc kubenswrapper[5052]: I1203 13:12:20.197539 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-operators/manila-operator-controller-manager-7c79b5df47-ccbqq" Dec 03 13:12:20 crc kubenswrapper[5052]: I1203 13:12:20.220256 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-operators/manila-operator-controller-manager-7c79b5df47-ccbqq"] Dec 03 13:12:20 crc kubenswrapper[5052]: I1203 13:12:20.224839 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-operators/manila-operator-controller-manager-7c79b5df47-ccbqq"] Dec 03 13:12:21 crc kubenswrapper[5052]: I1203 13:12:21.683762 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ea1e91de-7e7d-498a-a17b-551f2bee325a" path="/var/lib/kubelet/pods/ea1e91de-7e7d-498a-a17b-551f2bee325a/volumes" Dec 03 13:12:36 crc kubenswrapper[5052]: I1203 13:12:36.917203 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-operators/mariadb-operator-index-sl85p"] Dec 03 13:12:36 crc kubenswrapper[5052]: E1203 13:12:36.918643 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="50c6e918-7f1f-4d6f-ba5f-940e5f485825" containerName="manager" Dec 03 13:12:36 crc kubenswrapper[5052]: I1203 13:12:36.918669 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="50c6e918-7f1f-4d6f-ba5f-940e5f485825" containerName="manager" Dec 03 13:12:36 crc kubenswrapper[5052]: E1203 13:12:36.918709 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="ea1e91de-7e7d-498a-a17b-551f2bee325a" containerName="kube-rbac-proxy" Dec 03 13:12:36 crc kubenswrapper[5052]: I1203 13:12:36.918719 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="ea1e91de-7e7d-498a-a17b-551f2bee325a" containerName="kube-rbac-proxy" Dec 03 13:12:36 crc kubenswrapper[5052]: E1203 13:12:36.918730 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="81f5f8e1-015a-4b45-83e6-13e88efb8e80" containerName="kube-rbac-proxy" Dec 03 13:12:36 crc kubenswrapper[5052]: I1203 13:12:36.918738 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="81f5f8e1-015a-4b45-83e6-13e88efb8e80" containerName="kube-rbac-proxy" Dec 03 13:12:36 crc kubenswrapper[5052]: E1203 13:12:36.918752 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="8fd08931-7f0b-4d5b-bfb1-f27980648561" containerName="manager" Dec 03 13:12:36 crc kubenswrapper[5052]: I1203 13:12:36.918762 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="8fd08931-7f0b-4d5b-bfb1-f27980648561" containerName="manager" Dec 03 13:12:36 crc kubenswrapper[5052]: E1203 13:12:36.918778 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="8fd08931-7f0b-4d5b-bfb1-f27980648561" containerName="kube-rbac-proxy" Dec 03 13:12:36 crc kubenswrapper[5052]: I1203 13:12:36.918786 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="8fd08931-7f0b-4d5b-bfb1-f27980648561" containerName="kube-rbac-proxy" Dec 03 13:12:36 crc kubenswrapper[5052]: E1203 13:12:36.918804 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="e10da887-70df-43aa-aa79-4a14231e5ec9" containerName="kube-rbac-proxy" Dec 03 13:12:36 crc kubenswrapper[5052]: I1203 13:12:36.918812 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="e10da887-70df-43aa-aa79-4a14231e5ec9" containerName="kube-rbac-proxy" Dec 03 13:12:36 crc kubenswrapper[5052]: E1203 13:12:36.918823 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="2e2dc077-c02f-4f98-8fa6-9f72890faeaf" containerName="kube-rbac-proxy" Dec 03 13:12:36 crc kubenswrapper[5052]: I1203 13:12:36.918830 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="2e2dc077-c02f-4f98-8fa6-9f72890faeaf" containerName="kube-rbac-proxy" Dec 03 13:12:36 crc kubenswrapper[5052]: E1203 13:12:36.918845 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="0cbd234c-ecb3-4e54-af86-a2b15fdd94a3" containerName="manager" Dec 03 13:12:36 crc kubenswrapper[5052]: I1203 13:12:36.918853 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="0cbd234c-ecb3-4e54-af86-a2b15fdd94a3" containerName="manager" Dec 03 13:12:36 crc kubenswrapper[5052]: E1203 13:12:36.918867 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="ea1e91de-7e7d-498a-a17b-551f2bee325a" containerName="manager" Dec 03 13:12:36 crc kubenswrapper[5052]: I1203 13:12:36.918875 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="ea1e91de-7e7d-498a-a17b-551f2bee325a" containerName="manager" Dec 03 13:12:36 crc kubenswrapper[5052]: E1203 13:12:36.918887 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="7017b3f7-1eb9-4dfe-8ef3-6db27cf3a1f1" containerName="manager" Dec 03 13:12:36 crc kubenswrapper[5052]: I1203 13:12:36.918895 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="7017b3f7-1eb9-4dfe-8ef3-6db27cf3a1f1" containerName="manager" Dec 03 13:12:36 crc kubenswrapper[5052]: E1203 13:12:36.918907 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="ce3d34f6-e545-4eb2-bcc5-999377931601" containerName="manager" Dec 03 13:12:36 crc kubenswrapper[5052]: I1203 13:12:36.918914 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="ce3d34f6-e545-4eb2-bcc5-999377931601" containerName="manager" Dec 03 13:12:36 crc kubenswrapper[5052]: E1203 13:12:36.918932 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="3894016b-8242-4b12-8407-bcf78af00851" containerName="manager" Dec 03 13:12:36 crc kubenswrapper[5052]: I1203 13:12:36.918941 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="3894016b-8242-4b12-8407-bcf78af00851" containerName="manager" Dec 03 13:12:36 crc kubenswrapper[5052]: E1203 13:12:36.918952 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="3894016b-8242-4b12-8407-bcf78af00851" containerName="kube-rbac-proxy" Dec 03 13:12:36 crc kubenswrapper[5052]: I1203 13:12:36.918960 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="3894016b-8242-4b12-8407-bcf78af00851" containerName="kube-rbac-proxy" Dec 03 13:12:36 crc kubenswrapper[5052]: E1203 13:12:36.918970 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="53c88ca3-39f7-4a81-b305-40047bcaeb88" containerName="kube-rbac-proxy" Dec 03 13:12:36 crc kubenswrapper[5052]: I1203 13:12:36.918979 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="53c88ca3-39f7-4a81-b305-40047bcaeb88" containerName="kube-rbac-proxy" Dec 03 13:12:36 crc kubenswrapper[5052]: E1203 13:12:36.918987 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="6fe52b82-e00b-4c40-8ba1-bb52e7897667" containerName="registry-server" Dec 03 13:12:36 crc kubenswrapper[5052]: I1203 13:12:36.918994 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="6fe52b82-e00b-4c40-8ba1-bb52e7897667" containerName="registry-server" Dec 03 13:12:36 crc kubenswrapper[5052]: E1203 13:12:36.919004 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="f0e70c0d-10fe-43b7-a08c-0ce411f223a6" containerName="kube-rbac-proxy" Dec 03 13:12:36 crc kubenswrapper[5052]: I1203 13:12:36.919012 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="f0e70c0d-10fe-43b7-a08c-0ce411f223a6" containerName="kube-rbac-proxy" Dec 03 13:12:36 crc kubenswrapper[5052]: E1203 13:12:36.919026 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="a605d632-d9a5-4ce0-9449-e3d57105df1b" containerName="manager" Dec 03 13:12:36 crc kubenswrapper[5052]: I1203 13:12:36.919035 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="a605d632-d9a5-4ce0-9449-e3d57105df1b" containerName="manager" Dec 03 13:12:36 crc kubenswrapper[5052]: E1203 13:12:36.919047 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="cde663cf-e7e2-42b1-9581-ea7a27f43481" containerName="init" Dec 03 13:12:36 crc kubenswrapper[5052]: I1203 13:12:36.919056 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="cde663cf-e7e2-42b1-9581-ea7a27f43481" containerName="init" Dec 03 13:12:36 crc kubenswrapper[5052]: E1203 13:12:36.919065 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="6bea5f42-82da-4148-9996-dc05121212f7" containerName="manager" Dec 03 13:12:36 crc kubenswrapper[5052]: I1203 13:12:36.919073 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="6bea5f42-82da-4148-9996-dc05121212f7" containerName="manager" Dec 03 13:12:36 crc kubenswrapper[5052]: E1203 13:12:36.919086 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="6bea5f42-82da-4148-9996-dc05121212f7" containerName="kube-rbac-proxy" Dec 03 13:12:36 crc kubenswrapper[5052]: I1203 13:12:36.919095 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="6bea5f42-82da-4148-9996-dc05121212f7" containerName="kube-rbac-proxy" Dec 03 13:12:36 crc kubenswrapper[5052]: E1203 13:12:36.919105 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="81f5f8e1-015a-4b45-83e6-13e88efb8e80" containerName="manager" Dec 03 13:12:36 crc kubenswrapper[5052]: I1203 13:12:36.919113 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="81f5f8e1-015a-4b45-83e6-13e88efb8e80" containerName="manager" Dec 03 13:12:36 crc kubenswrapper[5052]: E1203 13:12:36.919125 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="95b50980-fd5a-4a19-b1cd-13df98d257fe" containerName="kube-rbac-proxy" Dec 03 13:12:36 crc kubenswrapper[5052]: I1203 13:12:36.919134 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="95b50980-fd5a-4a19-b1cd-13df98d257fe" containerName="kube-rbac-proxy" Dec 03 13:12:36 crc kubenswrapper[5052]: E1203 13:12:36.919148 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="37ab421b-0ad3-4d9d-bf47-8bf908c4aa57" containerName="kube-rbac-proxy" Dec 03 13:12:36 crc kubenswrapper[5052]: I1203 13:12:36.919156 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="37ab421b-0ad3-4d9d-bf47-8bf908c4aa57" containerName="kube-rbac-proxy" Dec 03 13:12:36 crc kubenswrapper[5052]: E1203 13:12:36.919167 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="2e2dc077-c02f-4f98-8fa6-9f72890faeaf" containerName="manager" Dec 03 13:12:36 crc kubenswrapper[5052]: I1203 13:12:36.919174 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="2e2dc077-c02f-4f98-8fa6-9f72890faeaf" containerName="manager" Dec 03 13:12:36 crc kubenswrapper[5052]: E1203 13:12:36.919184 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="ce3d34f6-e545-4eb2-bcc5-999377931601" containerName="kube-rbac-proxy" Dec 03 13:12:36 crc kubenswrapper[5052]: I1203 13:12:36.919193 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="ce3d34f6-e545-4eb2-bcc5-999377931601" containerName="kube-rbac-proxy" Dec 03 13:12:36 crc kubenswrapper[5052]: E1203 13:12:36.919208 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="0cbd234c-ecb3-4e54-af86-a2b15fdd94a3" containerName="kube-rbac-proxy" Dec 03 13:12:36 crc kubenswrapper[5052]: I1203 13:12:36.919216 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="0cbd234c-ecb3-4e54-af86-a2b15fdd94a3" containerName="kube-rbac-proxy" Dec 03 13:12:36 crc kubenswrapper[5052]: E1203 13:12:36.919230 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="f0e70c0d-10fe-43b7-a08c-0ce411f223a6" containerName="manager" Dec 03 13:12:36 crc kubenswrapper[5052]: I1203 13:12:36.919238 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="f0e70c0d-10fe-43b7-a08c-0ce411f223a6" containerName="manager" Dec 03 13:12:36 crc kubenswrapper[5052]: E1203 13:12:36.919247 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="1cb3d739-e4d6-4058-9456-a3c2e1c09130" containerName="kube-rbac-proxy" Dec 03 13:12:36 crc kubenswrapper[5052]: I1203 13:12:36.919254 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="1cb3d739-e4d6-4058-9456-a3c2e1c09130" containerName="kube-rbac-proxy" Dec 03 13:12:36 crc kubenswrapper[5052]: E1203 13:12:36.919267 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="5968d273-33dd-45cc-aab0-e83628bcd15c" containerName="kube-rbac-proxy" Dec 03 13:12:36 crc kubenswrapper[5052]: I1203 13:12:36.919276 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="5968d273-33dd-45cc-aab0-e83628bcd15c" containerName="kube-rbac-proxy" Dec 03 13:12:36 crc kubenswrapper[5052]: E1203 13:12:36.919287 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="3e6bc3f6-1f1a-4b99-8835-98252369d364" containerName="kube-rbac-proxy" Dec 03 13:12:36 crc kubenswrapper[5052]: I1203 13:12:36.919294 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="3e6bc3f6-1f1a-4b99-8835-98252369d364" containerName="kube-rbac-proxy" Dec 03 13:12:36 crc kubenswrapper[5052]: E1203 13:12:36.919306 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="5968d273-33dd-45cc-aab0-e83628bcd15c" containerName="manager" Dec 03 13:12:36 crc kubenswrapper[5052]: I1203 13:12:36.919315 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="5968d273-33dd-45cc-aab0-e83628bcd15c" containerName="manager" Dec 03 13:12:36 crc kubenswrapper[5052]: E1203 13:12:36.919325 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="3e6bc3f6-1f1a-4b99-8835-98252369d364" containerName="manager" Dec 03 13:12:36 crc kubenswrapper[5052]: I1203 13:12:36.919333 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="3e6bc3f6-1f1a-4b99-8835-98252369d364" containerName="manager" Dec 03 13:12:36 crc kubenswrapper[5052]: E1203 13:12:36.919346 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="d429f7ab-9a88-4411-a7a5-88bc8ab76285" containerName="kube-rbac-proxy" Dec 03 13:12:36 crc kubenswrapper[5052]: I1203 13:12:36.919354 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="d429f7ab-9a88-4411-a7a5-88bc8ab76285" containerName="kube-rbac-proxy" Dec 03 13:12:36 crc kubenswrapper[5052]: E1203 13:12:36.919367 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="8dbd25a1-9de9-431e-be37-d239e3f64728" containerName="manager" Dec 03 13:12:36 crc kubenswrapper[5052]: I1203 13:12:36.919374 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="8dbd25a1-9de9-431e-be37-d239e3f64728" containerName="manager" Dec 03 13:12:36 crc kubenswrapper[5052]: E1203 13:12:36.919388 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="cde663cf-e7e2-42b1-9581-ea7a27f43481" containerName="dnsmasq-dns" Dec 03 13:12:36 crc kubenswrapper[5052]: I1203 13:12:36.919397 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="cde663cf-e7e2-42b1-9581-ea7a27f43481" containerName="dnsmasq-dns" Dec 03 13:12:36 crc kubenswrapper[5052]: E1203 13:12:36.919412 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="e10da887-70df-43aa-aa79-4a14231e5ec9" containerName="manager" Dec 03 13:12:36 crc kubenswrapper[5052]: I1203 13:12:36.919420 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="e10da887-70df-43aa-aa79-4a14231e5ec9" containerName="manager" Dec 03 13:12:36 crc kubenswrapper[5052]: E1203 13:12:36.919433 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="53c88ca3-39f7-4a81-b305-40047bcaeb88" containerName="manager" Dec 03 13:12:36 crc kubenswrapper[5052]: I1203 13:12:36.919441 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="53c88ca3-39f7-4a81-b305-40047bcaeb88" containerName="manager" Dec 03 13:12:36 crc kubenswrapper[5052]: E1203 13:12:36.919450 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="cd2c5c4c-7ea8-4f95-9556-58b87d14a2be" containerName="operator" Dec 03 13:12:36 crc kubenswrapper[5052]: I1203 13:12:36.919457 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="cd2c5c4c-7ea8-4f95-9556-58b87d14a2be" containerName="operator" Dec 03 13:12:36 crc kubenswrapper[5052]: E1203 13:12:36.919465 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="7017b3f7-1eb9-4dfe-8ef3-6db27cf3a1f1" containerName="kube-rbac-proxy" Dec 03 13:12:36 crc kubenswrapper[5052]: I1203 13:12:36.919472 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="7017b3f7-1eb9-4dfe-8ef3-6db27cf3a1f1" containerName="kube-rbac-proxy" Dec 03 13:12:36 crc kubenswrapper[5052]: E1203 13:12:36.919485 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="459f5555-efae-4c19-84d0-2af917b9560f" containerName="operator" Dec 03 13:12:36 crc kubenswrapper[5052]: I1203 13:12:36.919495 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="459f5555-efae-4c19-84d0-2af917b9560f" containerName="operator" Dec 03 13:12:36 crc kubenswrapper[5052]: E1203 13:12:36.919513 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="50c6e918-7f1f-4d6f-ba5f-940e5f485825" containerName="kube-rbac-proxy" Dec 03 13:12:36 crc kubenswrapper[5052]: I1203 13:12:36.919521 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="50c6e918-7f1f-4d6f-ba5f-940e5f485825" containerName="kube-rbac-proxy" Dec 03 13:12:36 crc kubenswrapper[5052]: E1203 13:12:36.919551 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="37ab421b-0ad3-4d9d-bf47-8bf908c4aa57" containerName="manager" Dec 03 13:12:36 crc kubenswrapper[5052]: I1203 13:12:36.919560 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="37ab421b-0ad3-4d9d-bf47-8bf908c4aa57" containerName="manager" Dec 03 13:12:36 crc kubenswrapper[5052]: E1203 13:12:36.919588 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="d429f7ab-9a88-4411-a7a5-88bc8ab76285" containerName="manager" Dec 03 13:12:36 crc kubenswrapper[5052]: I1203 13:12:36.919620 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="d429f7ab-9a88-4411-a7a5-88bc8ab76285" containerName="manager" Dec 03 13:12:36 crc kubenswrapper[5052]: E1203 13:12:36.919652 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="95b50980-fd5a-4a19-b1cd-13df98d257fe" containerName="manager" Dec 03 13:12:36 crc kubenswrapper[5052]: I1203 13:12:36.919663 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="95b50980-fd5a-4a19-b1cd-13df98d257fe" containerName="manager" Dec 03 13:12:36 crc kubenswrapper[5052]: E1203 13:12:36.919677 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="1cb3d739-e4d6-4058-9456-a3c2e1c09130" containerName="manager" Dec 03 13:12:36 crc kubenswrapper[5052]: I1203 13:12:36.919685 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="1cb3d739-e4d6-4058-9456-a3c2e1c09130" containerName="manager" Dec 03 13:12:36 crc kubenswrapper[5052]: E1203 13:12:36.919697 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="a605d632-d9a5-4ce0-9449-e3d57105df1b" containerName="kube-rbac-proxy" Dec 03 13:12:36 crc kubenswrapper[5052]: I1203 13:12:36.919705 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="a605d632-d9a5-4ce0-9449-e3d57105df1b" containerName="kube-rbac-proxy" Dec 03 13:12:36 crc kubenswrapper[5052]: E1203 13:12:36.919718 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="8dbd25a1-9de9-431e-be37-d239e3f64728" containerName="kube-rbac-proxy" Dec 03 13:12:36 crc kubenswrapper[5052]: I1203 13:12:36.919731 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="8dbd25a1-9de9-431e-be37-d239e3f64728" containerName="kube-rbac-proxy" Dec 03 13:12:36 crc kubenswrapper[5052]: E1203 13:12:36.919743 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="527bcd50-6d8d-4d4f-bc11-5382eb9adfc2" containerName="manager" Dec 03 13:12:36 crc kubenswrapper[5052]: I1203 13:12:36.919752 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="527bcd50-6d8d-4d4f-bc11-5382eb9adfc2" containerName="manager" Dec 03 13:12:36 crc kubenswrapper[5052]: I1203 13:12:36.919944 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="6fe52b82-e00b-4c40-8ba1-bb52e7897667" containerName="registry-server" Dec 03 13:12:36 crc kubenswrapper[5052]: I1203 13:12:36.919962 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="8fd08931-7f0b-4d5b-bfb1-f27980648561" containerName="kube-rbac-proxy" Dec 03 13:12:36 crc kubenswrapper[5052]: I1203 13:12:36.919977 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="f0e70c0d-10fe-43b7-a08c-0ce411f223a6" containerName="manager" Dec 03 13:12:36 crc kubenswrapper[5052]: I1203 13:12:36.919990 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="53c88ca3-39f7-4a81-b305-40047bcaeb88" containerName="manager" Dec 03 13:12:36 crc kubenswrapper[5052]: I1203 13:12:36.920000 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="3894016b-8242-4b12-8407-bcf78af00851" containerName="manager" Dec 03 13:12:36 crc kubenswrapper[5052]: I1203 13:12:36.920009 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="53c88ca3-39f7-4a81-b305-40047bcaeb88" containerName="kube-rbac-proxy" Dec 03 13:12:36 crc kubenswrapper[5052]: I1203 13:12:36.920021 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="5968d273-33dd-45cc-aab0-e83628bcd15c" containerName="kube-rbac-proxy" Dec 03 13:12:36 crc kubenswrapper[5052]: I1203 13:12:36.920033 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="1cb3d739-e4d6-4058-9456-a3c2e1c09130" containerName="kube-rbac-proxy" Dec 03 13:12:36 crc kubenswrapper[5052]: I1203 13:12:36.920043 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="8dbd25a1-9de9-431e-be37-d239e3f64728" containerName="manager" Dec 03 13:12:36 crc kubenswrapper[5052]: I1203 13:12:36.920051 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="7017b3f7-1eb9-4dfe-8ef3-6db27cf3a1f1" containerName="manager" Dec 03 13:12:36 crc kubenswrapper[5052]: I1203 13:12:36.920064 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="8fd08931-7f0b-4d5b-bfb1-f27980648561" containerName="manager" Dec 03 13:12:36 crc kubenswrapper[5052]: I1203 13:12:36.920076 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="1cb3d739-e4d6-4058-9456-a3c2e1c09130" containerName="manager" Dec 03 13:12:36 crc kubenswrapper[5052]: I1203 13:12:36.920092 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="50c6e918-7f1f-4d6f-ba5f-940e5f485825" containerName="manager" Dec 03 13:12:36 crc kubenswrapper[5052]: I1203 13:12:36.920101 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="50c6e918-7f1f-4d6f-ba5f-940e5f485825" containerName="kube-rbac-proxy" Dec 03 13:12:36 crc kubenswrapper[5052]: I1203 13:12:36.920114 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="95b50980-fd5a-4a19-b1cd-13df98d257fe" containerName="kube-rbac-proxy" Dec 03 13:12:36 crc kubenswrapper[5052]: I1203 13:12:36.920125 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="6bea5f42-82da-4148-9996-dc05121212f7" containerName="kube-rbac-proxy" Dec 03 13:12:36 crc kubenswrapper[5052]: I1203 13:12:36.920137 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="ce3d34f6-e545-4eb2-bcc5-999377931601" containerName="manager" Dec 03 13:12:36 crc kubenswrapper[5052]: I1203 13:12:36.920149 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="3e6bc3f6-1f1a-4b99-8835-98252369d364" containerName="manager" Dec 03 13:12:36 crc kubenswrapper[5052]: I1203 13:12:36.920161 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="95b50980-fd5a-4a19-b1cd-13df98d257fe" containerName="manager" Dec 03 13:12:36 crc kubenswrapper[5052]: I1203 13:12:36.920172 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="37ab421b-0ad3-4d9d-bf47-8bf908c4aa57" containerName="manager" Dec 03 13:12:36 crc kubenswrapper[5052]: I1203 13:12:36.920182 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="0cbd234c-ecb3-4e54-af86-a2b15fdd94a3" containerName="manager" Dec 03 13:12:36 crc kubenswrapper[5052]: I1203 13:12:36.920191 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="e10da887-70df-43aa-aa79-4a14231e5ec9" containerName="manager" Dec 03 13:12:36 crc kubenswrapper[5052]: I1203 13:12:36.920204 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="a605d632-d9a5-4ce0-9449-e3d57105df1b" containerName="kube-rbac-proxy" Dec 03 13:12:36 crc kubenswrapper[5052]: I1203 13:12:36.920219 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="5968d273-33dd-45cc-aab0-e83628bcd15c" containerName="manager" Dec 03 13:12:36 crc kubenswrapper[5052]: I1203 13:12:36.920233 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="d429f7ab-9a88-4411-a7a5-88bc8ab76285" containerName="kube-rbac-proxy" Dec 03 13:12:36 crc kubenswrapper[5052]: I1203 13:12:36.920242 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="459f5555-efae-4c19-84d0-2af917b9560f" containerName="operator" Dec 03 13:12:36 crc kubenswrapper[5052]: I1203 13:12:36.920256 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="ea1e91de-7e7d-498a-a17b-551f2bee325a" containerName="kube-rbac-proxy" Dec 03 13:12:36 crc kubenswrapper[5052]: I1203 13:12:36.920265 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="ea1e91de-7e7d-498a-a17b-551f2bee325a" containerName="manager" Dec 03 13:12:36 crc kubenswrapper[5052]: I1203 13:12:36.920276 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="a605d632-d9a5-4ce0-9449-e3d57105df1b" containerName="manager" Dec 03 13:12:36 crc kubenswrapper[5052]: I1203 13:12:36.920287 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="6bea5f42-82da-4148-9996-dc05121212f7" containerName="manager" Dec 03 13:12:36 crc kubenswrapper[5052]: I1203 13:12:36.920299 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="ce3d34f6-e545-4eb2-bcc5-999377931601" containerName="kube-rbac-proxy" Dec 03 13:12:36 crc kubenswrapper[5052]: I1203 13:12:36.920309 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="3894016b-8242-4b12-8407-bcf78af00851" containerName="kube-rbac-proxy" Dec 03 13:12:36 crc kubenswrapper[5052]: I1203 13:12:36.920319 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="f0e70c0d-10fe-43b7-a08c-0ce411f223a6" containerName="kube-rbac-proxy" Dec 03 13:12:36 crc kubenswrapper[5052]: I1203 13:12:36.920330 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="7017b3f7-1eb9-4dfe-8ef3-6db27cf3a1f1" containerName="kube-rbac-proxy" Dec 03 13:12:36 crc kubenswrapper[5052]: I1203 13:12:36.920340 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="8dbd25a1-9de9-431e-be37-d239e3f64728" containerName="kube-rbac-proxy" Dec 03 13:12:36 crc kubenswrapper[5052]: I1203 13:12:36.920349 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="527bcd50-6d8d-4d4f-bc11-5382eb9adfc2" containerName="manager" Dec 03 13:12:36 crc kubenswrapper[5052]: I1203 13:12:36.920358 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="d429f7ab-9a88-4411-a7a5-88bc8ab76285" containerName="manager" Dec 03 13:12:36 crc kubenswrapper[5052]: I1203 13:12:36.920368 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="cde663cf-e7e2-42b1-9581-ea7a27f43481" containerName="dnsmasq-dns" Dec 03 13:12:36 crc kubenswrapper[5052]: I1203 13:12:36.920393 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="cd2c5c4c-7ea8-4f95-9556-58b87d14a2be" containerName="operator" Dec 03 13:12:36 crc kubenswrapper[5052]: I1203 13:12:36.920408 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="3e6bc3f6-1f1a-4b99-8835-98252369d364" containerName="kube-rbac-proxy" Dec 03 13:12:36 crc kubenswrapper[5052]: I1203 13:12:36.920422 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="e10da887-70df-43aa-aa79-4a14231e5ec9" containerName="kube-rbac-proxy" Dec 03 13:12:36 crc kubenswrapper[5052]: I1203 13:12:36.920435 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="0cbd234c-ecb3-4e54-af86-a2b15fdd94a3" containerName="kube-rbac-proxy" Dec 03 13:12:36 crc kubenswrapper[5052]: I1203 13:12:36.920450 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="37ab421b-0ad3-4d9d-bf47-8bf908c4aa57" containerName="kube-rbac-proxy" Dec 03 13:12:36 crc kubenswrapper[5052]: I1203 13:12:36.920462 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="81f5f8e1-015a-4b45-83e6-13e88efb8e80" containerName="kube-rbac-proxy" Dec 03 13:12:36 crc kubenswrapper[5052]: I1203 13:12:36.920473 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="2e2dc077-c02f-4f98-8fa6-9f72890faeaf" containerName="kube-rbac-proxy" Dec 03 13:12:36 crc kubenswrapper[5052]: I1203 13:12:36.920485 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="2e2dc077-c02f-4f98-8fa6-9f72890faeaf" containerName="manager" Dec 03 13:12:36 crc kubenswrapper[5052]: I1203 13:12:36.920498 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="81f5f8e1-015a-4b45-83e6-13e88efb8e80" containerName="manager" Dec 03 13:12:36 crc kubenswrapper[5052]: I1203 13:12:36.921174 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-operators/mariadb-operator-index-sl85p" Dec 03 13:12:36 crc kubenswrapper[5052]: I1203 13:12:36.929046 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-operators"/"openshift-service-ca.crt" Dec 03 13:12:36 crc kubenswrapper[5052]: I1203 13:12:36.929180 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-operators"/"mariadb-operator-index-dockercfg-s4zjb" Dec 03 13:12:36 crc kubenswrapper[5052]: I1203 13:12:36.929377 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-operators"/"kube-root-ca.crt" Dec 03 13:12:36 crc kubenswrapper[5052]: I1203 13:12:36.948389 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-operators/mariadb-operator-index-sl85p"] Dec 03 13:12:36 crc kubenswrapper[5052]: I1203 13:12:36.969758 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6qr8b\" (UniqueName: \"kubernetes.io/projected/ac315a77-e67f-45e0-8813-d8d3ed044944-kube-api-access-6qr8b\") pod \"mariadb-operator-index-sl85p\" (UID: \"ac315a77-e67f-45e0-8813-d8d3ed044944\") " pod="openstack-operators/mariadb-operator-index-sl85p" Dec 03 13:12:37 crc kubenswrapper[5052]: I1203 13:12:37.071442 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-6qr8b\" (UniqueName: \"kubernetes.io/projected/ac315a77-e67f-45e0-8813-d8d3ed044944-kube-api-access-6qr8b\") pod \"mariadb-operator-index-sl85p\" (UID: \"ac315a77-e67f-45e0-8813-d8d3ed044944\") " pod="openstack-operators/mariadb-operator-index-sl85p" Dec 03 13:12:37 crc kubenswrapper[5052]: I1203 13:12:37.096919 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-6qr8b\" (UniqueName: \"kubernetes.io/projected/ac315a77-e67f-45e0-8813-d8d3ed044944-kube-api-access-6qr8b\") pod \"mariadb-operator-index-sl85p\" (UID: \"ac315a77-e67f-45e0-8813-d8d3ed044944\") " pod="openstack-operators/mariadb-operator-index-sl85p" Dec 03 13:12:37 crc kubenswrapper[5052]: I1203 13:12:37.254329 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-operators/mariadb-operator-index-sl85p" Dec 03 13:12:37 crc kubenswrapper[5052]: I1203 13:12:37.296919 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-operators/mariadb-operator-index-sl85p"] Dec 03 13:12:37 crc kubenswrapper[5052]: I1203 13:12:37.507954 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-operators/mariadb-operator-index-jbcj6"] Dec 03 13:12:37 crc kubenswrapper[5052]: I1203 13:12:37.509289 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-operators/mariadb-operator-index-jbcj6" Dec 03 13:12:37 crc kubenswrapper[5052]: I1203 13:12:37.527385 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-operators/mariadb-operator-index-jbcj6"] Dec 03 13:12:37 crc kubenswrapper[5052]: I1203 13:12:37.541109 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-operators/mariadb-operator-index-sl85p"] Dec 03 13:12:37 crc kubenswrapper[5052]: I1203 13:12:37.586248 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bxch2\" (UniqueName: \"kubernetes.io/projected/f7ca8ea0-92d6-4274-96f7-8297170883fc-kube-api-access-bxch2\") pod \"mariadb-operator-index-jbcj6\" (UID: \"f7ca8ea0-92d6-4274-96f7-8297170883fc\") " pod="openstack-operators/mariadb-operator-index-jbcj6" Dec 03 13:12:37 crc kubenswrapper[5052]: I1203 13:12:37.688108 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-bxch2\" (UniqueName: \"kubernetes.io/projected/f7ca8ea0-92d6-4274-96f7-8297170883fc-kube-api-access-bxch2\") pod \"mariadb-operator-index-jbcj6\" (UID: \"f7ca8ea0-92d6-4274-96f7-8297170883fc\") " pod="openstack-operators/mariadb-operator-index-jbcj6" Dec 03 13:12:37 crc kubenswrapper[5052]: I1203 13:12:37.708380 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-bxch2\" (UniqueName: \"kubernetes.io/projected/f7ca8ea0-92d6-4274-96f7-8297170883fc-kube-api-access-bxch2\") pod \"mariadb-operator-index-jbcj6\" (UID: \"f7ca8ea0-92d6-4274-96f7-8297170883fc\") " pod="openstack-operators/mariadb-operator-index-jbcj6" Dec 03 13:12:37 crc kubenswrapper[5052]: I1203 13:12:37.842406 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-operators/mariadb-operator-index-jbcj6" Dec 03 13:12:38 crc kubenswrapper[5052]: I1203 13:12:38.071802 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-operators/mariadb-operator-index-jbcj6"] Dec 03 13:12:38 crc kubenswrapper[5052]: I1203 13:12:38.335986 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/mariadb-operator-index-jbcj6" event={"ID":"f7ca8ea0-92d6-4274-96f7-8297170883fc","Type":"ContainerStarted","Data":"f491e7b203095b1a58705d93ae6c6200836e6180c6a29211cfff9043b6b5b29d"} Dec 03 13:12:38 crc kubenswrapper[5052]: I1203 13:12:38.347767 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/mariadb-operator-index-sl85p" event={"ID":"ac315a77-e67f-45e0-8813-d8d3ed044944","Type":"ContainerStarted","Data":"95e6672973073f714414950027a0b9fea1f44066d23e108ffee3db87468bd6d9"} Dec 03 13:12:40 crc kubenswrapper[5052]: I1203 13:12:40.364986 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/mariadb-operator-index-sl85p" event={"ID":"ac315a77-e67f-45e0-8813-d8d3ed044944","Type":"ContainerStarted","Data":"9ebcac7a37e57c22a46c5cfbb771a1501b69066b9650a302b7087c0f0fd7139e"} Dec 03 13:12:40 crc kubenswrapper[5052]: I1203 13:12:40.365068 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-operators/mariadb-operator-index-sl85p" podUID="ac315a77-e67f-45e0-8813-d8d3ed044944" containerName="registry-server" containerID="cri-o://9ebcac7a37e57c22a46c5cfbb771a1501b69066b9650a302b7087c0f0fd7139e" gracePeriod=2 Dec 03 13:12:40 crc kubenswrapper[5052]: I1203 13:12:40.369201 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/mariadb-operator-index-jbcj6" event={"ID":"f7ca8ea0-92d6-4274-96f7-8297170883fc","Type":"ContainerStarted","Data":"cb36d6b31264eb861a891fb9a33d6ae809a253d3427f4bda6dfaa99d86e000b1"} Dec 03 13:12:40 crc kubenswrapper[5052]: I1203 13:12:40.401995 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-operators/mariadb-operator-index-jbcj6" podStartSLOduration=2.35266328 podStartE2EDuration="3.40197193s" podCreationTimestamp="2025-12-03 13:12:37 +0000 UTC" firstStartedPulling="2025-12-03 13:12:38.087476469 +0000 UTC m=+6748.675718233" lastFinishedPulling="2025-12-03 13:12:39.136785099 +0000 UTC m=+6749.725026883" observedRunningTime="2025-12-03 13:12:40.395387442 +0000 UTC m=+6750.983629206" watchObservedRunningTime="2025-12-03 13:12:40.40197193 +0000 UTC m=+6750.990213694" Dec 03 13:12:40 crc kubenswrapper[5052]: I1203 13:12:40.403944 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-operators/mariadb-operator-index-sl85p" podStartSLOduration=2.8237637639999997 podStartE2EDuration="4.403937427s" podCreationTimestamp="2025-12-03 13:12:36 +0000 UTC" firstStartedPulling="2025-12-03 13:12:37.556635827 +0000 UTC m=+6748.144877601" lastFinishedPulling="2025-12-03 13:12:39.1368095 +0000 UTC m=+6749.725051264" observedRunningTime="2025-12-03 13:12:40.381567596 +0000 UTC m=+6750.969809380" watchObservedRunningTime="2025-12-03 13:12:40.403937427 +0000 UTC m=+6750.992179191" Dec 03 13:12:40 crc kubenswrapper[5052]: I1203 13:12:40.878545 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-operators/mariadb-operator-index-sl85p" Dec 03 13:12:40 crc kubenswrapper[5052]: I1203 13:12:40.963333 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6qr8b\" (UniqueName: \"kubernetes.io/projected/ac315a77-e67f-45e0-8813-d8d3ed044944-kube-api-access-6qr8b\") pod \"ac315a77-e67f-45e0-8813-d8d3ed044944\" (UID: \"ac315a77-e67f-45e0-8813-d8d3ed044944\") " Dec 03 13:12:40 crc kubenswrapper[5052]: I1203 13:12:40.969719 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ac315a77-e67f-45e0-8813-d8d3ed044944-kube-api-access-6qr8b" (OuterVolumeSpecName: "kube-api-access-6qr8b") pod "ac315a77-e67f-45e0-8813-d8d3ed044944" (UID: "ac315a77-e67f-45e0-8813-d8d3ed044944"). InnerVolumeSpecName "kube-api-access-6qr8b". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:12:41 crc kubenswrapper[5052]: I1203 13:12:41.064949 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-6qr8b\" (UniqueName: \"kubernetes.io/projected/ac315a77-e67f-45e0-8813-d8d3ed044944-kube-api-access-6qr8b\") on node \"crc\" DevicePath \"\"" Dec 03 13:12:41 crc kubenswrapper[5052]: I1203 13:12:41.379297 5052 generic.go:334] "Generic (PLEG): container finished" podID="ac315a77-e67f-45e0-8813-d8d3ed044944" containerID="9ebcac7a37e57c22a46c5cfbb771a1501b69066b9650a302b7087c0f0fd7139e" exitCode=0 Dec 03 13:12:41 crc kubenswrapper[5052]: I1203 13:12:41.379364 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-operators/mariadb-operator-index-sl85p" Dec 03 13:12:41 crc kubenswrapper[5052]: I1203 13:12:41.379379 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/mariadb-operator-index-sl85p" event={"ID":"ac315a77-e67f-45e0-8813-d8d3ed044944","Type":"ContainerDied","Data":"9ebcac7a37e57c22a46c5cfbb771a1501b69066b9650a302b7087c0f0fd7139e"} Dec 03 13:12:41 crc kubenswrapper[5052]: I1203 13:12:41.379455 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/mariadb-operator-index-sl85p" event={"ID":"ac315a77-e67f-45e0-8813-d8d3ed044944","Type":"ContainerDied","Data":"95e6672973073f714414950027a0b9fea1f44066d23e108ffee3db87468bd6d9"} Dec 03 13:12:41 crc kubenswrapper[5052]: I1203 13:12:41.379479 5052 scope.go:117] "RemoveContainer" containerID="9ebcac7a37e57c22a46c5cfbb771a1501b69066b9650a302b7087c0f0fd7139e" Dec 03 13:12:41 crc kubenswrapper[5052]: I1203 13:12:41.399874 5052 scope.go:117] "RemoveContainer" containerID="9ebcac7a37e57c22a46c5cfbb771a1501b69066b9650a302b7087c0f0fd7139e" Dec 03 13:12:41 crc kubenswrapper[5052]: E1203 13:12:41.402436 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9ebcac7a37e57c22a46c5cfbb771a1501b69066b9650a302b7087c0f0fd7139e\": container with ID starting with 9ebcac7a37e57c22a46c5cfbb771a1501b69066b9650a302b7087c0f0fd7139e not found: ID does not exist" containerID="9ebcac7a37e57c22a46c5cfbb771a1501b69066b9650a302b7087c0f0fd7139e" Dec 03 13:12:41 crc kubenswrapper[5052]: I1203 13:12:41.402499 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9ebcac7a37e57c22a46c5cfbb771a1501b69066b9650a302b7087c0f0fd7139e"} err="failed to get container status \"9ebcac7a37e57c22a46c5cfbb771a1501b69066b9650a302b7087c0f0fd7139e\": rpc error: code = NotFound desc = could not find container \"9ebcac7a37e57c22a46c5cfbb771a1501b69066b9650a302b7087c0f0fd7139e\": container with ID starting with 9ebcac7a37e57c22a46c5cfbb771a1501b69066b9650a302b7087c0f0fd7139e not found: ID does not exist" Dec 03 13:12:41 crc kubenswrapper[5052]: I1203 13:12:41.409309 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-operators/mariadb-operator-index-sl85p"] Dec 03 13:12:41 crc kubenswrapper[5052]: I1203 13:12:41.414351 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-operators/mariadb-operator-index-sl85p"] Dec 03 13:12:41 crc kubenswrapper[5052]: I1203 13:12:41.683139 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ac315a77-e67f-45e0-8813-d8d3ed044944" path="/var/lib/kubelet/pods/ac315a77-e67f-45e0-8813-d8d3ed044944/volumes" Dec 03 13:12:44 crc kubenswrapper[5052]: I1203 13:12:44.663693 5052 scope.go:117] "RemoveContainer" containerID="c87489961ab06bb10e01de5fc388749bce75be1a64c04c0510bbfab5ab2f2a46" Dec 03 13:12:44 crc kubenswrapper[5052]: I1203 13:12:44.693474 5052 scope.go:117] "RemoveContainer" containerID="68d019501383a405411c007622c2abbfc555ddefac71b2db656127bfbb35778d" Dec 03 13:12:44 crc kubenswrapper[5052]: I1203 13:12:44.723706 5052 scope.go:117] "RemoveContainer" containerID="c16b4916d85027f28775414e1e9e061796b804441bf4aded0b2a1af11134a874" Dec 03 13:12:44 crc kubenswrapper[5052]: I1203 13:12:44.755055 5052 scope.go:117] "RemoveContainer" containerID="a19133637f8911d3fba7f2b47eb8e794f144864c729a5d01025e2b35f2d03fc8" Dec 03 13:12:44 crc kubenswrapper[5052]: I1203 13:12:44.777388 5052 scope.go:117] "RemoveContainer" containerID="aef3df5f8fff4b308899303eaa804c7403a302b0281ea9d04dd205a1a36bc5e6" Dec 03 13:12:44 crc kubenswrapper[5052]: I1203 13:12:44.812823 5052 scope.go:117] "RemoveContainer" containerID="dc9bd993b4d6db1d054ab0a7f486584db56e37d1e6559b303531ddc0f825d402" Dec 03 13:12:44 crc kubenswrapper[5052]: I1203 13:12:44.836784 5052 scope.go:117] "RemoveContainer" containerID="c75090bc5d8deae6a758cdfee457a3599e34c37592925eda56118f4d25373a96" Dec 03 13:12:44 crc kubenswrapper[5052]: I1203 13:12:44.867843 5052 scope.go:117] "RemoveContainer" containerID="7a38fc46fa1de916c4dcdb66fb009b9e798bf1d5b7c4bb72dc6db4bde166741c" Dec 03 13:12:44 crc kubenswrapper[5052]: I1203 13:12:44.888818 5052 scope.go:117] "RemoveContainer" containerID="9a69414fae786f9e02976dca01bc969c527b49236d5729e3a66b4a354a79dacd" Dec 03 13:12:44 crc kubenswrapper[5052]: I1203 13:12:44.915361 5052 scope.go:117] "RemoveContainer" containerID="4581166bdfee51112c24884b2492e7341e87ef27c9a5126a5746e43910a66aa4" Dec 03 13:12:44 crc kubenswrapper[5052]: I1203 13:12:44.935022 5052 scope.go:117] "RemoveContainer" containerID="cef1bc7a06434159cd29dfdbf2e3083b4d8c6668507619974f0b6b08c96991e3" Dec 03 13:12:44 crc kubenswrapper[5052]: I1203 13:12:44.962143 5052 scope.go:117] "RemoveContainer" containerID="1a3bf6cdb4478c9febde2b88d59cad00f4ed9b1e5a42911b9ae8bdf2360d5785" Dec 03 13:12:45 crc kubenswrapper[5052]: I1203 13:12:45.003444 5052 scope.go:117] "RemoveContainer" containerID="3015e5486b47a528197db8ba47bf3513191f52877b648e0cf08565573ad9202b" Dec 03 13:12:45 crc kubenswrapper[5052]: I1203 13:12:45.024616 5052 scope.go:117] "RemoveContainer" containerID="97920a5e6bb5c96fa08c020aae2055f6d6c6ddf72d91e1b35081548101fd59bc" Dec 03 13:12:47 crc kubenswrapper[5052]: I1203 13:12:47.843485 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-operators/mariadb-operator-index-jbcj6" Dec 03 13:12:47 crc kubenswrapper[5052]: I1203 13:12:47.843560 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-operators/mariadb-operator-index-jbcj6" Dec 03 13:12:47 crc kubenswrapper[5052]: I1203 13:12:47.881623 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-operators/mariadb-operator-index-jbcj6" Dec 03 13:12:48 crc kubenswrapper[5052]: I1203 13:12:48.459766 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-operators/mariadb-operator-index-jbcj6" Dec 03 13:12:54 crc kubenswrapper[5052]: I1203 13:12:54.542990 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-operators/55a3886cc1ed42812df4eab61c7a6033dc924d195539e8545c8f175f61g5jxg"] Dec 03 13:12:54 crc kubenswrapper[5052]: E1203 13:12:54.543758 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="ac315a77-e67f-45e0-8813-d8d3ed044944" containerName="registry-server" Dec 03 13:12:54 crc kubenswrapper[5052]: I1203 13:12:54.543771 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="ac315a77-e67f-45e0-8813-d8d3ed044944" containerName="registry-server" Dec 03 13:12:54 crc kubenswrapper[5052]: I1203 13:12:54.543908 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="ac315a77-e67f-45e0-8813-d8d3ed044944" containerName="registry-server" Dec 03 13:12:54 crc kubenswrapper[5052]: I1203 13:12:54.544925 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-operators/55a3886cc1ed42812df4eab61c7a6033dc924d195539e8545c8f175f61g5jxg" Dec 03 13:12:54 crc kubenswrapper[5052]: I1203 13:12:54.547282 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-operators"/"default-dockercfg-7fklr" Dec 03 13:12:54 crc kubenswrapper[5052]: I1203 13:12:54.554378 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-operators/55a3886cc1ed42812df4eab61c7a6033dc924d195539e8545c8f175f61g5jxg"] Dec 03 13:12:54 crc kubenswrapper[5052]: I1203 13:12:54.555429 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c8e42363-9fc0-49b1-a9d4-44a4eaa2f093-bundle\") pod \"55a3886cc1ed42812df4eab61c7a6033dc924d195539e8545c8f175f61g5jxg\" (UID: \"c8e42363-9fc0-49b1-a9d4-44a4eaa2f093\") " pod="openstack-operators/55a3886cc1ed42812df4eab61c7a6033dc924d195539e8545c8f175f61g5jxg" Dec 03 13:12:54 crc kubenswrapper[5052]: I1203 13:12:54.555528 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c8e42363-9fc0-49b1-a9d4-44a4eaa2f093-util\") pod \"55a3886cc1ed42812df4eab61c7a6033dc924d195539e8545c8f175f61g5jxg\" (UID: \"c8e42363-9fc0-49b1-a9d4-44a4eaa2f093\") " pod="openstack-operators/55a3886cc1ed42812df4eab61c7a6033dc924d195539e8545c8f175f61g5jxg" Dec 03 13:12:54 crc kubenswrapper[5052]: I1203 13:12:54.555568 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6v7pz\" (UniqueName: \"kubernetes.io/projected/c8e42363-9fc0-49b1-a9d4-44a4eaa2f093-kube-api-access-6v7pz\") pod \"55a3886cc1ed42812df4eab61c7a6033dc924d195539e8545c8f175f61g5jxg\" (UID: \"c8e42363-9fc0-49b1-a9d4-44a4eaa2f093\") " pod="openstack-operators/55a3886cc1ed42812df4eab61c7a6033dc924d195539e8545c8f175f61g5jxg" Dec 03 13:12:54 crc kubenswrapper[5052]: I1203 13:12:54.658499 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c8e42363-9fc0-49b1-a9d4-44a4eaa2f093-bundle\") pod \"55a3886cc1ed42812df4eab61c7a6033dc924d195539e8545c8f175f61g5jxg\" (UID: \"c8e42363-9fc0-49b1-a9d4-44a4eaa2f093\") " pod="openstack-operators/55a3886cc1ed42812df4eab61c7a6033dc924d195539e8545c8f175f61g5jxg" Dec 03 13:12:54 crc kubenswrapper[5052]: I1203 13:12:54.658620 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c8e42363-9fc0-49b1-a9d4-44a4eaa2f093-util\") pod \"55a3886cc1ed42812df4eab61c7a6033dc924d195539e8545c8f175f61g5jxg\" (UID: \"c8e42363-9fc0-49b1-a9d4-44a4eaa2f093\") " pod="openstack-operators/55a3886cc1ed42812df4eab61c7a6033dc924d195539e8545c8f175f61g5jxg" Dec 03 13:12:54 crc kubenswrapper[5052]: I1203 13:12:54.658656 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-6v7pz\" (UniqueName: \"kubernetes.io/projected/c8e42363-9fc0-49b1-a9d4-44a4eaa2f093-kube-api-access-6v7pz\") pod \"55a3886cc1ed42812df4eab61c7a6033dc924d195539e8545c8f175f61g5jxg\" (UID: \"c8e42363-9fc0-49b1-a9d4-44a4eaa2f093\") " pod="openstack-operators/55a3886cc1ed42812df4eab61c7a6033dc924d195539e8545c8f175f61g5jxg" Dec 03 13:12:54 crc kubenswrapper[5052]: I1203 13:12:54.659120 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c8e42363-9fc0-49b1-a9d4-44a4eaa2f093-bundle\") pod \"55a3886cc1ed42812df4eab61c7a6033dc924d195539e8545c8f175f61g5jxg\" (UID: \"c8e42363-9fc0-49b1-a9d4-44a4eaa2f093\") " pod="openstack-operators/55a3886cc1ed42812df4eab61c7a6033dc924d195539e8545c8f175f61g5jxg" Dec 03 13:12:54 crc kubenswrapper[5052]: I1203 13:12:54.659220 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c8e42363-9fc0-49b1-a9d4-44a4eaa2f093-util\") pod \"55a3886cc1ed42812df4eab61c7a6033dc924d195539e8545c8f175f61g5jxg\" (UID: \"c8e42363-9fc0-49b1-a9d4-44a4eaa2f093\") " pod="openstack-operators/55a3886cc1ed42812df4eab61c7a6033dc924d195539e8545c8f175f61g5jxg" Dec 03 13:12:54 crc kubenswrapper[5052]: I1203 13:12:54.690539 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-6v7pz\" (UniqueName: \"kubernetes.io/projected/c8e42363-9fc0-49b1-a9d4-44a4eaa2f093-kube-api-access-6v7pz\") pod \"55a3886cc1ed42812df4eab61c7a6033dc924d195539e8545c8f175f61g5jxg\" (UID: \"c8e42363-9fc0-49b1-a9d4-44a4eaa2f093\") " pod="openstack-operators/55a3886cc1ed42812df4eab61c7a6033dc924d195539e8545c8f175f61g5jxg" Dec 03 13:12:54 crc kubenswrapper[5052]: I1203 13:12:54.875124 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-operators/55a3886cc1ed42812df4eab61c7a6033dc924d195539e8545c8f175f61g5jxg" Dec 03 13:12:55 crc kubenswrapper[5052]: I1203 13:12:55.132954 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-operators/55a3886cc1ed42812df4eab61c7a6033dc924d195539e8545c8f175f61g5jxg"] Dec 03 13:12:55 crc kubenswrapper[5052]: I1203 13:12:55.485761 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/55a3886cc1ed42812df4eab61c7a6033dc924d195539e8545c8f175f61g5jxg" event={"ID":"c8e42363-9fc0-49b1-a9d4-44a4eaa2f093","Type":"ContainerStarted","Data":"bc20387fa50eb6a9597e4423e26aef13c7f844e105dc15dd899d9f9df12373b7"} Dec 03 13:12:55 crc kubenswrapper[5052]: I1203 13:12:55.486221 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/55a3886cc1ed42812df4eab61c7a6033dc924d195539e8545c8f175f61g5jxg" event={"ID":"c8e42363-9fc0-49b1-a9d4-44a4eaa2f093","Type":"ContainerStarted","Data":"25d1972b5ba5a31971804b2de53f969beba166cf42b79f9e8b62296463185df7"} Dec 03 13:12:56 crc kubenswrapper[5052]: I1203 13:12:56.493603 5052 generic.go:334] "Generic (PLEG): container finished" podID="c8e42363-9fc0-49b1-a9d4-44a4eaa2f093" containerID="bc20387fa50eb6a9597e4423e26aef13c7f844e105dc15dd899d9f9df12373b7" exitCode=0 Dec 03 13:12:56 crc kubenswrapper[5052]: I1203 13:12:56.493661 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/55a3886cc1ed42812df4eab61c7a6033dc924d195539e8545c8f175f61g5jxg" event={"ID":"c8e42363-9fc0-49b1-a9d4-44a4eaa2f093","Type":"ContainerDied","Data":"bc20387fa50eb6a9597e4423e26aef13c7f844e105dc15dd899d9f9df12373b7"} Dec 03 13:12:57 crc kubenswrapper[5052]: I1203 13:12:57.505856 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/55a3886cc1ed42812df4eab61c7a6033dc924d195539e8545c8f175f61g5jxg" event={"ID":"c8e42363-9fc0-49b1-a9d4-44a4eaa2f093","Type":"ContainerStarted","Data":"1918a3a17b37d4280c7e0df4b3ef36de0a518ce5732f596e41092dd4c5c7f007"} Dec 03 13:12:58 crc kubenswrapper[5052]: I1203 13:12:58.525441 5052 generic.go:334] "Generic (PLEG): container finished" podID="c8e42363-9fc0-49b1-a9d4-44a4eaa2f093" containerID="1918a3a17b37d4280c7e0df4b3ef36de0a518ce5732f596e41092dd4c5c7f007" exitCode=0 Dec 03 13:12:58 crc kubenswrapper[5052]: I1203 13:12:58.525556 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/55a3886cc1ed42812df4eab61c7a6033dc924d195539e8545c8f175f61g5jxg" event={"ID":"c8e42363-9fc0-49b1-a9d4-44a4eaa2f093","Type":"ContainerDied","Data":"1918a3a17b37d4280c7e0df4b3ef36de0a518ce5732f596e41092dd4c5c7f007"} Dec 03 13:12:59 crc kubenswrapper[5052]: I1203 13:12:59.536155 5052 generic.go:334] "Generic (PLEG): container finished" podID="c8e42363-9fc0-49b1-a9d4-44a4eaa2f093" containerID="2b0e197aa5bd3ab0ed35a7a9a8cfd5a398150425b1a75056bf68e75c6bf3e7d3" exitCode=0 Dec 03 13:12:59 crc kubenswrapper[5052]: I1203 13:12:59.536203 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/55a3886cc1ed42812df4eab61c7a6033dc924d195539e8545c8f175f61g5jxg" event={"ID":"c8e42363-9fc0-49b1-a9d4-44a4eaa2f093","Type":"ContainerDied","Data":"2b0e197aa5bd3ab0ed35a7a9a8cfd5a398150425b1a75056bf68e75c6bf3e7d3"} Dec 03 13:13:00 crc kubenswrapper[5052]: I1203 13:13:00.848296 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-operators/55a3886cc1ed42812df4eab61c7a6033dc924d195539e8545c8f175f61g5jxg" Dec 03 13:13:00 crc kubenswrapper[5052]: I1203 13:13:00.961147 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6v7pz\" (UniqueName: \"kubernetes.io/projected/c8e42363-9fc0-49b1-a9d4-44a4eaa2f093-kube-api-access-6v7pz\") pod \"c8e42363-9fc0-49b1-a9d4-44a4eaa2f093\" (UID: \"c8e42363-9fc0-49b1-a9d4-44a4eaa2f093\") " Dec 03 13:13:00 crc kubenswrapper[5052]: I1203 13:13:00.961225 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c8e42363-9fc0-49b1-a9d4-44a4eaa2f093-util\") pod \"c8e42363-9fc0-49b1-a9d4-44a4eaa2f093\" (UID: \"c8e42363-9fc0-49b1-a9d4-44a4eaa2f093\") " Dec 03 13:13:00 crc kubenswrapper[5052]: I1203 13:13:00.961301 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c8e42363-9fc0-49b1-a9d4-44a4eaa2f093-bundle\") pod \"c8e42363-9fc0-49b1-a9d4-44a4eaa2f093\" (UID: \"c8e42363-9fc0-49b1-a9d4-44a4eaa2f093\") " Dec 03 13:13:00 crc kubenswrapper[5052]: I1203 13:13:00.962872 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c8e42363-9fc0-49b1-a9d4-44a4eaa2f093-bundle" (OuterVolumeSpecName: "bundle") pod "c8e42363-9fc0-49b1-a9d4-44a4eaa2f093" (UID: "c8e42363-9fc0-49b1-a9d4-44a4eaa2f093"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 13:13:00 crc kubenswrapper[5052]: I1203 13:13:00.968600 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c8e42363-9fc0-49b1-a9d4-44a4eaa2f093-kube-api-access-6v7pz" (OuterVolumeSpecName: "kube-api-access-6v7pz") pod "c8e42363-9fc0-49b1-a9d4-44a4eaa2f093" (UID: "c8e42363-9fc0-49b1-a9d4-44a4eaa2f093"). InnerVolumeSpecName "kube-api-access-6v7pz". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:13:00 crc kubenswrapper[5052]: I1203 13:13:00.976041 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c8e42363-9fc0-49b1-a9d4-44a4eaa2f093-util" (OuterVolumeSpecName: "util") pod "c8e42363-9fc0-49b1-a9d4-44a4eaa2f093" (UID: "c8e42363-9fc0-49b1-a9d4-44a4eaa2f093"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 13:13:01 crc kubenswrapper[5052]: I1203 13:13:01.063026 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-6v7pz\" (UniqueName: \"kubernetes.io/projected/c8e42363-9fc0-49b1-a9d4-44a4eaa2f093-kube-api-access-6v7pz\") on node \"crc\" DevicePath \"\"" Dec 03 13:13:01 crc kubenswrapper[5052]: I1203 13:13:01.063068 5052 reconciler_common.go:293] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c8e42363-9fc0-49b1-a9d4-44a4eaa2f093-util\") on node \"crc\" DevicePath \"\"" Dec 03 13:13:01 crc kubenswrapper[5052]: I1203 13:13:01.063080 5052 reconciler_common.go:293] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c8e42363-9fc0-49b1-a9d4-44a4eaa2f093-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 13:13:01 crc kubenswrapper[5052]: I1203 13:13:01.551185 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/55a3886cc1ed42812df4eab61c7a6033dc924d195539e8545c8f175f61g5jxg" event={"ID":"c8e42363-9fc0-49b1-a9d4-44a4eaa2f093","Type":"ContainerDied","Data":"25d1972b5ba5a31971804b2de53f969beba166cf42b79f9e8b62296463185df7"} Dec 03 13:13:01 crc kubenswrapper[5052]: I1203 13:13:01.551229 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="25d1972b5ba5a31971804b2de53f969beba166cf42b79f9e8b62296463185df7" Dec 03 13:13:01 crc kubenswrapper[5052]: I1203 13:13:01.551257 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-operators/55a3886cc1ed42812df4eab61c7a6033dc924d195539e8545c8f175f61g5jxg" Dec 03 13:13:07 crc kubenswrapper[5052]: I1203 13:13:07.598213 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-operators/mariadb-operator-controller-manager-7595db4997-skrx7"] Dec 03 13:13:07 crc kubenswrapper[5052]: E1203 13:13:07.599091 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="c8e42363-9fc0-49b1-a9d4-44a4eaa2f093" containerName="util" Dec 03 13:13:07 crc kubenswrapper[5052]: I1203 13:13:07.599106 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="c8e42363-9fc0-49b1-a9d4-44a4eaa2f093" containerName="util" Dec 03 13:13:07 crc kubenswrapper[5052]: E1203 13:13:07.599137 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="c8e42363-9fc0-49b1-a9d4-44a4eaa2f093" containerName="pull" Dec 03 13:13:07 crc kubenswrapper[5052]: I1203 13:13:07.599145 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="c8e42363-9fc0-49b1-a9d4-44a4eaa2f093" containerName="pull" Dec 03 13:13:07 crc kubenswrapper[5052]: E1203 13:13:07.599173 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="c8e42363-9fc0-49b1-a9d4-44a4eaa2f093" containerName="extract" Dec 03 13:13:07 crc kubenswrapper[5052]: I1203 13:13:07.599182 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="c8e42363-9fc0-49b1-a9d4-44a4eaa2f093" containerName="extract" Dec 03 13:13:07 crc kubenswrapper[5052]: I1203 13:13:07.599322 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="c8e42363-9fc0-49b1-a9d4-44a4eaa2f093" containerName="extract" Dec 03 13:13:07 crc kubenswrapper[5052]: I1203 13:13:07.599903 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-operators/mariadb-operator-controller-manager-7595db4997-skrx7" Dec 03 13:13:07 crc kubenswrapper[5052]: I1203 13:13:07.602220 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-operators"/"mariadb-operator-controller-manager-service-cert" Dec 03 13:13:07 crc kubenswrapper[5052]: I1203 13:13:07.602899 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-operators"/"webhook-server-cert" Dec 03 13:13:07 crc kubenswrapper[5052]: I1203 13:13:07.606034 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-operators"/"mariadb-operator-controller-manager-dockercfg-kdc9p" Dec 03 13:13:07 crc kubenswrapper[5052]: I1203 13:13:07.617190 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-operators/mariadb-operator-controller-manager-7595db4997-skrx7"] Dec 03 13:13:07 crc kubenswrapper[5052]: I1203 13:13:07.754011 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/568b316e-dee2-4a2a-967b-730a5f03de09-webhook-cert\") pod \"mariadb-operator-controller-manager-7595db4997-skrx7\" (UID: \"568b316e-dee2-4a2a-967b-730a5f03de09\") " pod="openstack-operators/mariadb-operator-controller-manager-7595db4997-skrx7" Dec 03 13:13:07 crc kubenswrapper[5052]: I1203 13:13:07.754116 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/568b316e-dee2-4a2a-967b-730a5f03de09-apiservice-cert\") pod \"mariadb-operator-controller-manager-7595db4997-skrx7\" (UID: \"568b316e-dee2-4a2a-967b-730a5f03de09\") " pod="openstack-operators/mariadb-operator-controller-manager-7595db4997-skrx7" Dec 03 13:13:07 crc kubenswrapper[5052]: I1203 13:13:07.754151 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7cvgp\" (UniqueName: \"kubernetes.io/projected/568b316e-dee2-4a2a-967b-730a5f03de09-kube-api-access-7cvgp\") pod \"mariadb-operator-controller-manager-7595db4997-skrx7\" (UID: \"568b316e-dee2-4a2a-967b-730a5f03de09\") " pod="openstack-operators/mariadb-operator-controller-manager-7595db4997-skrx7" Dec 03 13:13:07 crc kubenswrapper[5052]: I1203 13:13:07.856727 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/568b316e-dee2-4a2a-967b-730a5f03de09-webhook-cert\") pod \"mariadb-operator-controller-manager-7595db4997-skrx7\" (UID: \"568b316e-dee2-4a2a-967b-730a5f03de09\") " pod="openstack-operators/mariadb-operator-controller-manager-7595db4997-skrx7" Dec 03 13:13:07 crc kubenswrapper[5052]: I1203 13:13:07.857165 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/568b316e-dee2-4a2a-967b-730a5f03de09-apiservice-cert\") pod \"mariadb-operator-controller-manager-7595db4997-skrx7\" (UID: \"568b316e-dee2-4a2a-967b-730a5f03de09\") " pod="openstack-operators/mariadb-operator-controller-manager-7595db4997-skrx7" Dec 03 13:13:07 crc kubenswrapper[5052]: I1203 13:13:07.857199 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-7cvgp\" (UniqueName: \"kubernetes.io/projected/568b316e-dee2-4a2a-967b-730a5f03de09-kube-api-access-7cvgp\") pod \"mariadb-operator-controller-manager-7595db4997-skrx7\" (UID: \"568b316e-dee2-4a2a-967b-730a5f03de09\") " pod="openstack-operators/mariadb-operator-controller-manager-7595db4997-skrx7" Dec 03 13:13:07 crc kubenswrapper[5052]: I1203 13:13:07.875669 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/568b316e-dee2-4a2a-967b-730a5f03de09-apiservice-cert\") pod \"mariadb-operator-controller-manager-7595db4997-skrx7\" (UID: \"568b316e-dee2-4a2a-967b-730a5f03de09\") " pod="openstack-operators/mariadb-operator-controller-manager-7595db4997-skrx7" Dec 03 13:13:07 crc kubenswrapper[5052]: I1203 13:13:07.885426 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-7cvgp\" (UniqueName: \"kubernetes.io/projected/568b316e-dee2-4a2a-967b-730a5f03de09-kube-api-access-7cvgp\") pod \"mariadb-operator-controller-manager-7595db4997-skrx7\" (UID: \"568b316e-dee2-4a2a-967b-730a5f03de09\") " pod="openstack-operators/mariadb-operator-controller-manager-7595db4997-skrx7" Dec 03 13:13:07 crc kubenswrapper[5052]: I1203 13:13:07.893305 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/568b316e-dee2-4a2a-967b-730a5f03de09-webhook-cert\") pod \"mariadb-operator-controller-manager-7595db4997-skrx7\" (UID: \"568b316e-dee2-4a2a-967b-730a5f03de09\") " pod="openstack-operators/mariadb-operator-controller-manager-7595db4997-skrx7" Dec 03 13:13:07 crc kubenswrapper[5052]: I1203 13:13:07.926659 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-operators/mariadb-operator-controller-manager-7595db4997-skrx7" Dec 03 13:13:08 crc kubenswrapper[5052]: I1203 13:13:08.373352 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-operators/mariadb-operator-controller-manager-7595db4997-skrx7"] Dec 03 13:13:08 crc kubenswrapper[5052]: I1203 13:13:08.615564 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/mariadb-operator-controller-manager-7595db4997-skrx7" event={"ID":"568b316e-dee2-4a2a-967b-730a5f03de09","Type":"ContainerStarted","Data":"259cb507b5536b21cde88922e40cc69650c04a6771e7243954ae6811adade8a0"} Dec 03 13:13:11 crc kubenswrapper[5052]: I1203 13:13:11.643771 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/mariadb-operator-controller-manager-7595db4997-skrx7" event={"ID":"568b316e-dee2-4a2a-967b-730a5f03de09","Type":"ContainerStarted","Data":"151cb2f59f27897dd419d8881ee3dbbaf6cfece15d5d067e9de2130be92940a7"} Dec 03 13:13:11 crc kubenswrapper[5052]: I1203 13:13:11.644424 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-operators/mariadb-operator-controller-manager-7595db4997-skrx7" Dec 03 13:13:11 crc kubenswrapper[5052]: I1203 13:13:11.666275 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-operators/mariadb-operator-controller-manager-7595db4997-skrx7" podStartSLOduration=1.70762613 podStartE2EDuration="4.666257489s" podCreationTimestamp="2025-12-03 13:13:07 +0000 UTC" firstStartedPulling="2025-12-03 13:13:08.387931224 +0000 UTC m=+6778.976172988" lastFinishedPulling="2025-12-03 13:13:11.346562583 +0000 UTC m=+6781.934804347" observedRunningTime="2025-12-03 13:13:11.66386932 +0000 UTC m=+6782.252111084" watchObservedRunningTime="2025-12-03 13:13:11.666257489 +0000 UTC m=+6782.254499263" Dec 03 13:13:17 crc kubenswrapper[5052]: I1203 13:13:17.933358 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-operators/mariadb-operator-controller-manager-7595db4997-skrx7" Dec 03 13:13:31 crc kubenswrapper[5052]: I1203 13:13:31.455798 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-operators/infra-operator-index-mhrwx"] Dec 03 13:13:31 crc kubenswrapper[5052]: I1203 13:13:31.457161 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-operators/infra-operator-index-mhrwx" Dec 03 13:13:31 crc kubenswrapper[5052]: I1203 13:13:31.460850 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-operators"/"infra-operator-index-dockercfg-njfqb" Dec 03 13:13:31 crc kubenswrapper[5052]: I1203 13:13:31.469882 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-operators/infra-operator-index-mhrwx"] Dec 03 13:13:31 crc kubenswrapper[5052]: I1203 13:13:31.616162 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z9snr\" (UniqueName: \"kubernetes.io/projected/fd9c88e5-c614-42c3-a252-e8a0600210a7-kube-api-access-z9snr\") pod \"infra-operator-index-mhrwx\" (UID: \"fd9c88e5-c614-42c3-a252-e8a0600210a7\") " pod="openstack-operators/infra-operator-index-mhrwx" Dec 03 13:13:31 crc kubenswrapper[5052]: I1203 13:13:31.719340 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-z9snr\" (UniqueName: \"kubernetes.io/projected/fd9c88e5-c614-42c3-a252-e8a0600210a7-kube-api-access-z9snr\") pod \"infra-operator-index-mhrwx\" (UID: \"fd9c88e5-c614-42c3-a252-e8a0600210a7\") " pod="openstack-operators/infra-operator-index-mhrwx" Dec 03 13:13:31 crc kubenswrapper[5052]: I1203 13:13:31.742829 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-z9snr\" (UniqueName: \"kubernetes.io/projected/fd9c88e5-c614-42c3-a252-e8a0600210a7-kube-api-access-z9snr\") pod \"infra-operator-index-mhrwx\" (UID: \"fd9c88e5-c614-42c3-a252-e8a0600210a7\") " pod="openstack-operators/infra-operator-index-mhrwx" Dec 03 13:13:31 crc kubenswrapper[5052]: I1203 13:13:31.775963 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-operators/infra-operator-index-mhrwx" Dec 03 13:13:32 crc kubenswrapper[5052]: I1203 13:13:32.205974 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-operators/infra-operator-index-mhrwx"] Dec 03 13:13:32 crc kubenswrapper[5052]: W1203 13:13:32.211428 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd9c88e5_c614_42c3_a252_e8a0600210a7.slice/crio-37cc2172abe1f801bb661c98a931f03d0b8e87e027ed2c31fed08f704ff318e5 WatchSource:0}: Error finding container 37cc2172abe1f801bb661c98a931f03d0b8e87e027ed2c31fed08f704ff318e5: Status 404 returned error can't find the container with id 37cc2172abe1f801bb661c98a931f03d0b8e87e027ed2c31fed08f704ff318e5 Dec 03 13:13:32 crc kubenswrapper[5052]: I1203 13:13:32.214459 5052 provider.go:102] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Dec 03 13:13:32 crc kubenswrapper[5052]: I1203 13:13:32.805162 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/infra-operator-index-mhrwx" event={"ID":"fd9c88e5-c614-42c3-a252-e8a0600210a7","Type":"ContainerStarted","Data":"37cc2172abe1f801bb661c98a931f03d0b8e87e027ed2c31fed08f704ff318e5"} Dec 03 13:13:41 crc kubenswrapper[5052]: I1203 13:13:41.882169 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/infra-operator-index-mhrwx" event={"ID":"fd9c88e5-c614-42c3-a252-e8a0600210a7","Type":"ContainerStarted","Data":"dca482a55ee1a10de30189b059d80aa7ebe62669d226bfa13c6621dbaf4686a5"} Dec 03 13:13:41 crc kubenswrapper[5052]: I1203 13:13:41.902473 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-operators/infra-operator-index-mhrwx" podStartSLOduration=1.567713658 podStartE2EDuration="10.902457033s" podCreationTimestamp="2025-12-03 13:13:31 +0000 UTC" firstStartedPulling="2025-12-03 13:13:32.214179374 +0000 UTC m=+6802.802421138" lastFinishedPulling="2025-12-03 13:13:41.548922749 +0000 UTC m=+6812.137164513" observedRunningTime="2025-12-03 13:13:41.900190408 +0000 UTC m=+6812.488432172" watchObservedRunningTime="2025-12-03 13:13:41.902457033 +0000 UTC m=+6812.490698797" Dec 03 13:13:45 crc kubenswrapper[5052]: I1203 13:13:45.677214 5052 scope.go:117] "RemoveContainer" containerID="a47bc2d2848c1474f67abcd4b147864bcc3995d3a9a6ebf5e5fb621627f4d548" Dec 03 13:13:45 crc kubenswrapper[5052]: I1203 13:13:45.701880 5052 scope.go:117] "RemoveContainer" containerID="87a2769f559683c40211370d84da6bc23e30db34d7d335f2eb47ac47add4668d" Dec 03 13:13:45 crc kubenswrapper[5052]: I1203 13:13:45.725740 5052 scope.go:117] "RemoveContainer" containerID="a866cc5f3fce03eb0ddb16849ed57f2f2fe63a3b094aba775cd1bfddebfc177c" Dec 03 13:13:46 crc kubenswrapper[5052]: I1203 13:13:46.014397 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["barbican-kuttl-tests/openstack-galera-0"] Dec 03 13:13:46 crc kubenswrapper[5052]: I1203 13:13:46.016069 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="barbican-kuttl-tests/openstack-galera-0" Dec 03 13:13:46 crc kubenswrapper[5052]: I1203 13:13:46.019891 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"barbican-kuttl-tests"/"openshift-service-ca.crt" Dec 03 13:13:46 crc kubenswrapper[5052]: I1203 13:13:46.022805 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"barbican-kuttl-tests"/"kube-root-ca.crt" Dec 03 13:13:46 crc kubenswrapper[5052]: I1203 13:13:46.023426 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"barbican-kuttl-tests"/"openstack-config-data" Dec 03 13:13:46 crc kubenswrapper[5052]: I1203 13:13:46.023564 5052 reflector.go:368] Caches populated for *v1.Secret from object-"barbican-kuttl-tests"/"galera-openstack-dockercfg-jrdn2" Dec 03 13:13:46 crc kubenswrapper[5052]: I1203 13:13:46.024122 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"barbican-kuttl-tests"/"openstack-scripts" Dec 03 13:13:46 crc kubenswrapper[5052]: I1203 13:13:46.034133 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["barbican-kuttl-tests/openstack-galera-0"] Dec 03 13:13:46 crc kubenswrapper[5052]: I1203 13:13:46.039634 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["barbican-kuttl-tests/openstack-galera-2"] Dec 03 13:13:46 crc kubenswrapper[5052]: I1203 13:13:46.040702 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="barbican-kuttl-tests/openstack-galera-2" Dec 03 13:13:46 crc kubenswrapper[5052]: I1203 13:13:46.060933 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["barbican-kuttl-tests/openstack-galera-1"] Dec 03 13:13:46 crc kubenswrapper[5052]: I1203 13:13:46.062504 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="barbican-kuttl-tests/openstack-galera-1" Dec 03 13:13:46 crc kubenswrapper[5052]: I1203 13:13:46.075823 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["barbican-kuttl-tests/openstack-galera-2"] Dec 03 13:13:46 crc kubenswrapper[5052]: I1203 13:13:46.092019 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["barbican-kuttl-tests/openstack-galera-1"] Dec 03 13:13:46 crc kubenswrapper[5052]: I1203 13:13:46.146442 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6hsr6\" (UniqueName: \"kubernetes.io/projected/61edc25e-298b-4f43-ad10-5457259b6dc0-kube-api-access-6hsr6\") pod \"openstack-galera-2\" (UID: \"61edc25e-298b-4f43-ad10-5457259b6dc0\") " pod="barbican-kuttl-tests/openstack-galera-2" Dec 03 13:13:46 crc kubenswrapper[5052]: I1203 13:13:46.146500 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/26dcf610-b5bf-4d7c-86c4-169ef42fb0e7-config-data-default\") pod \"openstack-galera-0\" (UID: \"26dcf610-b5bf-4d7c-86c4-169ef42fb0e7\") " pod="barbican-kuttl-tests/openstack-galera-0" Dec 03 13:13:46 crc kubenswrapper[5052]: I1203 13:13:46.146545 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/61edc25e-298b-4f43-ad10-5457259b6dc0-config-data-generated\") pod \"openstack-galera-2\" (UID: \"61edc25e-298b-4f43-ad10-5457259b6dc0\") " pod="barbican-kuttl-tests/openstack-galera-2" Dec 03 13:13:46 crc kubenswrapper[5052]: I1203 13:13:46.146588 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2pfv6\" (UniqueName: \"kubernetes.io/projected/26dcf610-b5bf-4d7c-86c4-169ef42fb0e7-kube-api-access-2pfv6\") pod \"openstack-galera-0\" (UID: \"26dcf610-b5bf-4d7c-86c4-169ef42fb0e7\") " pod="barbican-kuttl-tests/openstack-galera-0" Dec 03 13:13:46 crc kubenswrapper[5052]: I1203 13:13:46.146816 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/26dcf610-b5bf-4d7c-86c4-169ef42fb0e7-config-data-generated\") pod \"openstack-galera-0\" (UID: \"26dcf610-b5bf-4d7c-86c4-169ef42fb0e7\") " pod="barbican-kuttl-tests/openstack-galera-0" Dec 03 13:13:46 crc kubenswrapper[5052]: I1203 13:13:46.146998 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage09-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage09-crc\") pod \"openstack-galera-2\" (UID: \"61edc25e-298b-4f43-ad10-5457259b6dc0\") " pod="barbican-kuttl-tests/openstack-galera-2" Dec 03 13:13:46 crc kubenswrapper[5052]: I1203 13:13:46.147135 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/61edc25e-298b-4f43-ad10-5457259b6dc0-config-data-default\") pod \"openstack-galera-2\" (UID: \"61edc25e-298b-4f43-ad10-5457259b6dc0\") " pod="barbican-kuttl-tests/openstack-galera-2" Dec 03 13:13:46 crc kubenswrapper[5052]: I1203 13:13:46.147263 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/26dcf610-b5bf-4d7c-86c4-169ef42fb0e7-kolla-config\") pod \"openstack-galera-0\" (UID: \"26dcf610-b5bf-4d7c-86c4-169ef42fb0e7\") " pod="barbican-kuttl-tests/openstack-galera-0" Dec 03 13:13:46 crc kubenswrapper[5052]: I1203 13:13:46.147448 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/26dcf610-b5bf-4d7c-86c4-169ef42fb0e7-operator-scripts\") pod \"openstack-galera-0\" (UID: \"26dcf610-b5bf-4d7c-86c4-169ef42fb0e7\") " pod="barbican-kuttl-tests/openstack-galera-0" Dec 03 13:13:46 crc kubenswrapper[5052]: I1203 13:13:46.147539 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/61edc25e-298b-4f43-ad10-5457259b6dc0-operator-scripts\") pod \"openstack-galera-2\" (UID: \"61edc25e-298b-4f43-ad10-5457259b6dc0\") " pod="barbican-kuttl-tests/openstack-galera-2" Dec 03 13:13:46 crc kubenswrapper[5052]: I1203 13:13:46.147576 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage17-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage17-crc\") pod \"openstack-galera-0\" (UID: \"26dcf610-b5bf-4d7c-86c4-169ef42fb0e7\") " pod="barbican-kuttl-tests/openstack-galera-0" Dec 03 13:13:46 crc kubenswrapper[5052]: I1203 13:13:46.147605 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/61edc25e-298b-4f43-ad10-5457259b6dc0-kolla-config\") pod \"openstack-galera-2\" (UID: \"61edc25e-298b-4f43-ad10-5457259b6dc0\") " pod="barbican-kuttl-tests/openstack-galera-2" Dec 03 13:13:46 crc kubenswrapper[5052]: I1203 13:13:46.249286 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/61edc25e-298b-4f43-ad10-5457259b6dc0-config-data-default\") pod \"openstack-galera-2\" (UID: \"61edc25e-298b-4f43-ad10-5457259b6dc0\") " pod="barbican-kuttl-tests/openstack-galera-2" Dec 03 13:13:46 crc kubenswrapper[5052]: I1203 13:13:46.249351 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bcrcd\" (UniqueName: \"kubernetes.io/projected/317b3c95-8146-4310-862e-950f6f4c8f3c-kube-api-access-bcrcd\") pod \"openstack-galera-1\" (UID: \"317b3c95-8146-4310-862e-950f6f4c8f3c\") " pod="barbican-kuttl-tests/openstack-galera-1" Dec 03 13:13:46 crc kubenswrapper[5052]: I1203 13:13:46.249400 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/26dcf610-b5bf-4d7c-86c4-169ef42fb0e7-kolla-config\") pod \"openstack-galera-0\" (UID: \"26dcf610-b5bf-4d7c-86c4-169ef42fb0e7\") " pod="barbican-kuttl-tests/openstack-galera-0" Dec 03 13:13:46 crc kubenswrapper[5052]: I1203 13:13:46.249423 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/317b3c95-8146-4310-862e-950f6f4c8f3c-config-data-generated\") pod \"openstack-galera-1\" (UID: \"317b3c95-8146-4310-862e-950f6f4c8f3c\") " pod="barbican-kuttl-tests/openstack-galera-1" Dec 03 13:13:46 crc kubenswrapper[5052]: I1203 13:13:46.249445 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage19-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage19-crc\") pod \"openstack-galera-1\" (UID: \"317b3c95-8146-4310-862e-950f6f4c8f3c\") " pod="barbican-kuttl-tests/openstack-galera-1" Dec 03 13:13:46 crc kubenswrapper[5052]: I1203 13:13:46.249473 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/26dcf610-b5bf-4d7c-86c4-169ef42fb0e7-operator-scripts\") pod \"openstack-galera-0\" (UID: \"26dcf610-b5bf-4d7c-86c4-169ef42fb0e7\") " pod="barbican-kuttl-tests/openstack-galera-0" Dec 03 13:13:46 crc kubenswrapper[5052]: I1203 13:13:46.249492 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/317b3c95-8146-4310-862e-950f6f4c8f3c-config-data-default\") pod \"openstack-galera-1\" (UID: \"317b3c95-8146-4310-862e-950f6f4c8f3c\") " pod="barbican-kuttl-tests/openstack-galera-1" Dec 03 13:13:46 crc kubenswrapper[5052]: I1203 13:13:46.249512 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/61edc25e-298b-4f43-ad10-5457259b6dc0-operator-scripts\") pod \"openstack-galera-2\" (UID: \"61edc25e-298b-4f43-ad10-5457259b6dc0\") " pod="barbican-kuttl-tests/openstack-galera-2" Dec 03 13:13:46 crc kubenswrapper[5052]: I1203 13:13:46.249554 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage17-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage17-crc\") pod \"openstack-galera-0\" (UID: \"26dcf610-b5bf-4d7c-86c4-169ef42fb0e7\") " pod="barbican-kuttl-tests/openstack-galera-0" Dec 03 13:13:46 crc kubenswrapper[5052]: I1203 13:13:46.249607 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/61edc25e-298b-4f43-ad10-5457259b6dc0-kolla-config\") pod \"openstack-galera-2\" (UID: \"61edc25e-298b-4f43-ad10-5457259b6dc0\") " pod="barbican-kuttl-tests/openstack-galera-2" Dec 03 13:13:46 crc kubenswrapper[5052]: I1203 13:13:46.249634 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-6hsr6\" (UniqueName: \"kubernetes.io/projected/61edc25e-298b-4f43-ad10-5457259b6dc0-kube-api-access-6hsr6\") pod \"openstack-galera-2\" (UID: \"61edc25e-298b-4f43-ad10-5457259b6dc0\") " pod="barbican-kuttl-tests/openstack-galera-2" Dec 03 13:13:46 crc kubenswrapper[5052]: I1203 13:13:46.249660 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/26dcf610-b5bf-4d7c-86c4-169ef42fb0e7-config-data-default\") pod \"openstack-galera-0\" (UID: \"26dcf610-b5bf-4d7c-86c4-169ef42fb0e7\") " pod="barbican-kuttl-tests/openstack-galera-0" Dec 03 13:13:46 crc kubenswrapper[5052]: I1203 13:13:46.249681 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/61edc25e-298b-4f43-ad10-5457259b6dc0-config-data-generated\") pod \"openstack-galera-2\" (UID: \"61edc25e-298b-4f43-ad10-5457259b6dc0\") " pod="barbican-kuttl-tests/openstack-galera-2" Dec 03 13:13:46 crc kubenswrapper[5052]: I1203 13:13:46.249707 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-2pfv6\" (UniqueName: \"kubernetes.io/projected/26dcf610-b5bf-4d7c-86c4-169ef42fb0e7-kube-api-access-2pfv6\") pod \"openstack-galera-0\" (UID: \"26dcf610-b5bf-4d7c-86c4-169ef42fb0e7\") " pod="barbican-kuttl-tests/openstack-galera-0" Dec 03 13:13:46 crc kubenswrapper[5052]: I1203 13:13:46.249734 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/26dcf610-b5bf-4d7c-86c4-169ef42fb0e7-config-data-generated\") pod \"openstack-galera-0\" (UID: \"26dcf610-b5bf-4d7c-86c4-169ef42fb0e7\") " pod="barbican-kuttl-tests/openstack-galera-0" Dec 03 13:13:46 crc kubenswrapper[5052]: I1203 13:13:46.249765 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/317b3c95-8146-4310-862e-950f6f4c8f3c-operator-scripts\") pod \"openstack-galera-1\" (UID: \"317b3c95-8146-4310-862e-950f6f4c8f3c\") " pod="barbican-kuttl-tests/openstack-galera-1" Dec 03 13:13:46 crc kubenswrapper[5052]: I1203 13:13:46.249795 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/317b3c95-8146-4310-862e-950f6f4c8f3c-kolla-config\") pod \"openstack-galera-1\" (UID: \"317b3c95-8146-4310-862e-950f6f4c8f3c\") " pod="barbican-kuttl-tests/openstack-galera-1" Dec 03 13:13:46 crc kubenswrapper[5052]: I1203 13:13:46.249821 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage09-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage09-crc\") pod \"openstack-galera-2\" (UID: \"61edc25e-298b-4f43-ad10-5457259b6dc0\") " pod="barbican-kuttl-tests/openstack-galera-2" Dec 03 13:13:46 crc kubenswrapper[5052]: I1203 13:13:46.250162 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage09-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage09-crc\") pod \"openstack-galera-2\" (UID: \"61edc25e-298b-4f43-ad10-5457259b6dc0\") device mount path \"/mnt/openstack/pv09\"" pod="barbican-kuttl-tests/openstack-galera-2" Dec 03 13:13:46 crc kubenswrapper[5052]: I1203 13:13:46.250362 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage17-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage17-crc\") pod \"openstack-galera-0\" (UID: \"26dcf610-b5bf-4d7c-86c4-169ef42fb0e7\") device mount path \"/mnt/openstack/pv17\"" pod="barbican-kuttl-tests/openstack-galera-0" Dec 03 13:13:46 crc kubenswrapper[5052]: I1203 13:13:46.251438 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/61edc25e-298b-4f43-ad10-5457259b6dc0-config-data-default\") pod \"openstack-galera-2\" (UID: \"61edc25e-298b-4f43-ad10-5457259b6dc0\") " pod="barbican-kuttl-tests/openstack-galera-2" Dec 03 13:13:46 crc kubenswrapper[5052]: I1203 13:13:46.251429 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/26dcf610-b5bf-4d7c-86c4-169ef42fb0e7-config-data-default\") pod \"openstack-galera-0\" (UID: \"26dcf610-b5bf-4d7c-86c4-169ef42fb0e7\") " pod="barbican-kuttl-tests/openstack-galera-0" Dec 03 13:13:46 crc kubenswrapper[5052]: I1203 13:13:46.251505 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/26dcf610-b5bf-4d7c-86c4-169ef42fb0e7-kolla-config\") pod \"openstack-galera-0\" (UID: \"26dcf610-b5bf-4d7c-86c4-169ef42fb0e7\") " pod="barbican-kuttl-tests/openstack-galera-0" Dec 03 13:13:46 crc kubenswrapper[5052]: I1203 13:13:46.251292 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/26dcf610-b5bf-4d7c-86c4-169ef42fb0e7-config-data-generated\") pod \"openstack-galera-0\" (UID: \"26dcf610-b5bf-4d7c-86c4-169ef42fb0e7\") " pod="barbican-kuttl-tests/openstack-galera-0" Dec 03 13:13:46 crc kubenswrapper[5052]: I1203 13:13:46.251952 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/61edc25e-298b-4f43-ad10-5457259b6dc0-kolla-config\") pod \"openstack-galera-2\" (UID: \"61edc25e-298b-4f43-ad10-5457259b6dc0\") " pod="barbican-kuttl-tests/openstack-galera-2" Dec 03 13:13:46 crc kubenswrapper[5052]: I1203 13:13:46.252226 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/61edc25e-298b-4f43-ad10-5457259b6dc0-config-data-generated\") pod \"openstack-galera-2\" (UID: \"61edc25e-298b-4f43-ad10-5457259b6dc0\") " pod="barbican-kuttl-tests/openstack-galera-2" Dec 03 13:13:46 crc kubenswrapper[5052]: I1203 13:13:46.252748 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/61edc25e-298b-4f43-ad10-5457259b6dc0-operator-scripts\") pod \"openstack-galera-2\" (UID: \"61edc25e-298b-4f43-ad10-5457259b6dc0\") " pod="barbican-kuttl-tests/openstack-galera-2" Dec 03 13:13:46 crc kubenswrapper[5052]: I1203 13:13:46.253051 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/26dcf610-b5bf-4d7c-86c4-169ef42fb0e7-operator-scripts\") pod \"openstack-galera-0\" (UID: \"26dcf610-b5bf-4d7c-86c4-169ef42fb0e7\") " pod="barbican-kuttl-tests/openstack-galera-0" Dec 03 13:13:46 crc kubenswrapper[5052]: I1203 13:13:46.269964 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-2pfv6\" (UniqueName: \"kubernetes.io/projected/26dcf610-b5bf-4d7c-86c4-169ef42fb0e7-kube-api-access-2pfv6\") pod \"openstack-galera-0\" (UID: \"26dcf610-b5bf-4d7c-86c4-169ef42fb0e7\") " pod="barbican-kuttl-tests/openstack-galera-0" Dec 03 13:13:46 crc kubenswrapper[5052]: I1203 13:13:46.270145 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage17-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage17-crc\") pod \"openstack-galera-0\" (UID: \"26dcf610-b5bf-4d7c-86c4-169ef42fb0e7\") " pod="barbican-kuttl-tests/openstack-galera-0" Dec 03 13:13:46 crc kubenswrapper[5052]: I1203 13:13:46.272860 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-6hsr6\" (UniqueName: \"kubernetes.io/projected/61edc25e-298b-4f43-ad10-5457259b6dc0-kube-api-access-6hsr6\") pod \"openstack-galera-2\" (UID: \"61edc25e-298b-4f43-ad10-5457259b6dc0\") " pod="barbican-kuttl-tests/openstack-galera-2" Dec 03 13:13:46 crc kubenswrapper[5052]: I1203 13:13:46.273218 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage09-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage09-crc\") pod \"openstack-galera-2\" (UID: \"61edc25e-298b-4f43-ad10-5457259b6dc0\") " pod="barbican-kuttl-tests/openstack-galera-2" Dec 03 13:13:46 crc kubenswrapper[5052]: I1203 13:13:46.342494 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="barbican-kuttl-tests/openstack-galera-0" Dec 03 13:13:46 crc kubenswrapper[5052]: I1203 13:13:46.351830 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-bcrcd\" (UniqueName: \"kubernetes.io/projected/317b3c95-8146-4310-862e-950f6f4c8f3c-kube-api-access-bcrcd\") pod \"openstack-galera-1\" (UID: \"317b3c95-8146-4310-862e-950f6f4c8f3c\") " pod="barbican-kuttl-tests/openstack-galera-1" Dec 03 13:13:46 crc kubenswrapper[5052]: I1203 13:13:46.351953 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/317b3c95-8146-4310-862e-950f6f4c8f3c-config-data-generated\") pod \"openstack-galera-1\" (UID: \"317b3c95-8146-4310-862e-950f6f4c8f3c\") " pod="barbican-kuttl-tests/openstack-galera-1" Dec 03 13:13:46 crc kubenswrapper[5052]: I1203 13:13:46.351985 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage19-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage19-crc\") pod \"openstack-galera-1\" (UID: \"317b3c95-8146-4310-862e-950f6f4c8f3c\") " pod="barbican-kuttl-tests/openstack-galera-1" Dec 03 13:13:46 crc kubenswrapper[5052]: I1203 13:13:46.352031 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/317b3c95-8146-4310-862e-950f6f4c8f3c-config-data-default\") pod \"openstack-galera-1\" (UID: \"317b3c95-8146-4310-862e-950f6f4c8f3c\") " pod="barbican-kuttl-tests/openstack-galera-1" Dec 03 13:13:46 crc kubenswrapper[5052]: I1203 13:13:46.352109 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/317b3c95-8146-4310-862e-950f6f4c8f3c-operator-scripts\") pod \"openstack-galera-1\" (UID: \"317b3c95-8146-4310-862e-950f6f4c8f3c\") " pod="barbican-kuttl-tests/openstack-galera-1" Dec 03 13:13:46 crc kubenswrapper[5052]: I1203 13:13:46.352144 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/317b3c95-8146-4310-862e-950f6f4c8f3c-kolla-config\") pod \"openstack-galera-1\" (UID: \"317b3c95-8146-4310-862e-950f6f4c8f3c\") " pod="barbican-kuttl-tests/openstack-galera-1" Dec 03 13:13:46 crc kubenswrapper[5052]: I1203 13:13:46.352357 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/317b3c95-8146-4310-862e-950f6f4c8f3c-config-data-generated\") pod \"openstack-galera-1\" (UID: \"317b3c95-8146-4310-862e-950f6f4c8f3c\") " pod="barbican-kuttl-tests/openstack-galera-1" Dec 03 13:13:46 crc kubenswrapper[5052]: I1203 13:13:46.353077 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/317b3c95-8146-4310-862e-950f6f4c8f3c-config-data-default\") pod \"openstack-galera-1\" (UID: \"317b3c95-8146-4310-862e-950f6f4c8f3c\") " pod="barbican-kuttl-tests/openstack-galera-1" Dec 03 13:13:46 crc kubenswrapper[5052]: I1203 13:13:46.353107 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/317b3c95-8146-4310-862e-950f6f4c8f3c-kolla-config\") pod \"openstack-galera-1\" (UID: \"317b3c95-8146-4310-862e-950f6f4c8f3c\") " pod="barbican-kuttl-tests/openstack-galera-1" Dec 03 13:13:46 crc kubenswrapper[5052]: I1203 13:13:46.353158 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage19-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage19-crc\") pod \"openstack-galera-1\" (UID: \"317b3c95-8146-4310-862e-950f6f4c8f3c\") device mount path \"/mnt/openstack/pv19\"" pod="barbican-kuttl-tests/openstack-galera-1" Dec 03 13:13:46 crc kubenswrapper[5052]: I1203 13:13:46.354049 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/317b3c95-8146-4310-862e-950f6f4c8f3c-operator-scripts\") pod \"openstack-galera-1\" (UID: \"317b3c95-8146-4310-862e-950f6f4c8f3c\") " pod="barbican-kuttl-tests/openstack-galera-1" Dec 03 13:13:46 crc kubenswrapper[5052]: I1203 13:13:46.362234 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="barbican-kuttl-tests/openstack-galera-2" Dec 03 13:13:46 crc kubenswrapper[5052]: I1203 13:13:46.370307 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-bcrcd\" (UniqueName: \"kubernetes.io/projected/317b3c95-8146-4310-862e-950f6f4c8f3c-kube-api-access-bcrcd\") pod \"openstack-galera-1\" (UID: \"317b3c95-8146-4310-862e-950f6f4c8f3c\") " pod="barbican-kuttl-tests/openstack-galera-1" Dec 03 13:13:46 crc kubenswrapper[5052]: I1203 13:13:46.371888 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage19-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage19-crc\") pod \"openstack-galera-1\" (UID: \"317b3c95-8146-4310-862e-950f6f4c8f3c\") " pod="barbican-kuttl-tests/openstack-galera-1" Dec 03 13:13:46 crc kubenswrapper[5052]: I1203 13:13:46.408141 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="barbican-kuttl-tests/openstack-galera-1" Dec 03 13:13:46 crc kubenswrapper[5052]: I1203 13:13:46.738122 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["barbican-kuttl-tests/openstack-galera-1"] Dec 03 13:13:46 crc kubenswrapper[5052]: W1203 13:13:46.747713 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod317b3c95_8146_4310_862e_950f6f4c8f3c.slice/crio-54fd3935c0852102209f69b1f36d2047f3a38ffa9cf7991666d43b316843072c WatchSource:0}: Error finding container 54fd3935c0852102209f69b1f36d2047f3a38ffa9cf7991666d43b316843072c: Status 404 returned error can't find the container with id 54fd3935c0852102209f69b1f36d2047f3a38ffa9cf7991666d43b316843072c Dec 03 13:13:46 crc kubenswrapper[5052]: W1203 13:13:46.808696 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod26dcf610_b5bf_4d7c_86c4_169ef42fb0e7.slice/crio-f051410aa43ae943f86e6ccbeda741b77e989c4b8f69e52a08b4c0c4df4e790b WatchSource:0}: Error finding container f051410aa43ae943f86e6ccbeda741b77e989c4b8f69e52a08b4c0c4df4e790b: Status 404 returned error can't find the container with id f051410aa43ae943f86e6ccbeda741b77e989c4b8f69e52a08b4c0c4df4e790b Dec 03 13:13:46 crc kubenswrapper[5052]: I1203 13:13:46.811792 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["barbican-kuttl-tests/openstack-galera-0"] Dec 03 13:13:46 crc kubenswrapper[5052]: I1203 13:13:46.879498 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["barbican-kuttl-tests/openstack-galera-2"] Dec 03 13:13:46 crc kubenswrapper[5052]: I1203 13:13:46.928907 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="barbican-kuttl-tests/openstack-galera-0" event={"ID":"26dcf610-b5bf-4d7c-86c4-169ef42fb0e7","Type":"ContainerStarted","Data":"f051410aa43ae943f86e6ccbeda741b77e989c4b8f69e52a08b4c0c4df4e790b"} Dec 03 13:13:46 crc kubenswrapper[5052]: I1203 13:13:46.932684 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="barbican-kuttl-tests/openstack-galera-1" event={"ID":"317b3c95-8146-4310-862e-950f6f4c8f3c","Type":"ContainerStarted","Data":"54fd3935c0852102209f69b1f36d2047f3a38ffa9cf7991666d43b316843072c"} Dec 03 13:13:46 crc kubenswrapper[5052]: I1203 13:13:46.935621 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="barbican-kuttl-tests/openstack-galera-2" event={"ID":"61edc25e-298b-4f43-ad10-5457259b6dc0","Type":"ContainerStarted","Data":"a44708890f8809fdec72dcbfec88ae5c6af9700e88c0572d47003c7efec9f6bc"} Dec 03 13:13:47 crc kubenswrapper[5052]: I1203 13:13:47.946324 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="barbican-kuttl-tests/openstack-galera-2" event={"ID":"61edc25e-298b-4f43-ad10-5457259b6dc0","Type":"ContainerStarted","Data":"dcb879b68854d0520cab523c4e6ee01c37924d1f9f4d49a5514c260629ab703d"} Dec 03 13:13:47 crc kubenswrapper[5052]: I1203 13:13:47.950824 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="barbican-kuttl-tests/openstack-galera-0" event={"ID":"26dcf610-b5bf-4d7c-86c4-169ef42fb0e7","Type":"ContainerStarted","Data":"3722162af199440174441c0d30eadecfe9d0f035426d3ccb3ca353885c8ba797"} Dec 03 13:13:47 crc kubenswrapper[5052]: I1203 13:13:47.953637 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="barbican-kuttl-tests/openstack-galera-1" event={"ID":"317b3c95-8146-4310-862e-950f6f4c8f3c","Type":"ContainerStarted","Data":"420e9c019a410f22173fb0b09e2ee22a65b964d18dc45c57ba6125760b4e7f1d"} Dec 03 13:13:50 crc kubenswrapper[5052]: I1203 13:13:50.979813 5052 generic.go:334] "Generic (PLEG): container finished" podID="317b3c95-8146-4310-862e-950f6f4c8f3c" containerID="420e9c019a410f22173fb0b09e2ee22a65b964d18dc45c57ba6125760b4e7f1d" exitCode=0 Dec 03 13:13:50 crc kubenswrapper[5052]: I1203 13:13:50.980026 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="barbican-kuttl-tests/openstack-galera-1" event={"ID":"317b3c95-8146-4310-862e-950f6f4c8f3c","Type":"ContainerDied","Data":"420e9c019a410f22173fb0b09e2ee22a65b964d18dc45c57ba6125760b4e7f1d"} Dec 03 13:13:50 crc kubenswrapper[5052]: I1203 13:13:50.982153 5052 generic.go:334] "Generic (PLEG): container finished" podID="61edc25e-298b-4f43-ad10-5457259b6dc0" containerID="dcb879b68854d0520cab523c4e6ee01c37924d1f9f4d49a5514c260629ab703d" exitCode=0 Dec 03 13:13:50 crc kubenswrapper[5052]: I1203 13:13:50.982212 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="barbican-kuttl-tests/openstack-galera-2" event={"ID":"61edc25e-298b-4f43-ad10-5457259b6dc0","Type":"ContainerDied","Data":"dcb879b68854d0520cab523c4e6ee01c37924d1f9f4d49a5514c260629ab703d"} Dec 03 13:13:50 crc kubenswrapper[5052]: I1203 13:13:50.984887 5052 generic.go:334] "Generic (PLEG): container finished" podID="26dcf610-b5bf-4d7c-86c4-169ef42fb0e7" containerID="3722162af199440174441c0d30eadecfe9d0f035426d3ccb3ca353885c8ba797" exitCode=0 Dec 03 13:13:50 crc kubenswrapper[5052]: I1203 13:13:50.984935 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="barbican-kuttl-tests/openstack-galera-0" event={"ID":"26dcf610-b5bf-4d7c-86c4-169ef42fb0e7","Type":"ContainerDied","Data":"3722162af199440174441c0d30eadecfe9d0f035426d3ccb3ca353885c8ba797"} Dec 03 13:13:51 crc kubenswrapper[5052]: I1203 13:13:51.776586 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-operators/infra-operator-index-mhrwx" Dec 03 13:13:51 crc kubenswrapper[5052]: I1203 13:13:51.776935 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-operators/infra-operator-index-mhrwx" Dec 03 13:13:51 crc kubenswrapper[5052]: I1203 13:13:51.808143 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-operators/infra-operator-index-mhrwx" Dec 03 13:13:51 crc kubenswrapper[5052]: I1203 13:13:51.994703 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="barbican-kuttl-tests/openstack-galera-1" event={"ID":"317b3c95-8146-4310-862e-950f6f4c8f3c","Type":"ContainerStarted","Data":"a2568d7c893958b9c781cf905acf82ee5d1934b31c497db120efcc60318b22b2"} Dec 03 13:13:51 crc kubenswrapper[5052]: I1203 13:13:51.997647 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="barbican-kuttl-tests/openstack-galera-2" event={"ID":"61edc25e-298b-4f43-ad10-5457259b6dc0","Type":"ContainerStarted","Data":"ebb0211d789390961e1a49efd60d7c01d10b4b8a4382ac1b33810fef21c52b38"} Dec 03 13:13:51 crc kubenswrapper[5052]: I1203 13:13:51.999539 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="barbican-kuttl-tests/openstack-galera-0" event={"ID":"26dcf610-b5bf-4d7c-86c4-169ef42fb0e7","Type":"ContainerStarted","Data":"23c3dc6c480f2bb3c4438a36768474fa482ce0b0b929335a5d9baf9d865615ae"} Dec 03 13:13:52 crc kubenswrapper[5052]: I1203 13:13:52.023989 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="barbican-kuttl-tests/openstack-galera-1" podStartSLOduration=7.02396109 podStartE2EDuration="7.02396109s" podCreationTimestamp="2025-12-03 13:13:45 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 13:13:52.017404772 +0000 UTC m=+6822.605646536" watchObservedRunningTime="2025-12-03 13:13:52.02396109 +0000 UTC m=+6822.612202854" Dec 03 13:13:52 crc kubenswrapper[5052]: I1203 13:13:52.036335 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-operators/infra-operator-index-mhrwx" Dec 03 13:13:52 crc kubenswrapper[5052]: I1203 13:13:52.044095 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="barbican-kuttl-tests/openstack-galera-0" podStartSLOduration=8.044073556 podStartE2EDuration="8.044073556s" podCreationTimestamp="2025-12-03 13:13:44 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 13:13:52.037953031 +0000 UTC m=+6822.626194795" watchObservedRunningTime="2025-12-03 13:13:52.044073556 +0000 UTC m=+6822.632315320" Dec 03 13:13:52 crc kubenswrapper[5052]: I1203 13:13:52.064692 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="barbican-kuttl-tests/openstack-galera-2" podStartSLOduration=7.064672636 podStartE2EDuration="7.064672636s" podCreationTimestamp="2025-12-03 13:13:45 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 13:13:52.057096219 +0000 UTC m=+6822.645337983" watchObservedRunningTime="2025-12-03 13:13:52.064672636 +0000 UTC m=+6822.652914400" Dec 03 13:13:56 crc kubenswrapper[5052]: I1203 13:13:56.342975 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="barbican-kuttl-tests/openstack-galera-0" Dec 03 13:13:56 crc kubenswrapper[5052]: I1203 13:13:56.343543 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="barbican-kuttl-tests/openstack-galera-0" Dec 03 13:13:56 crc kubenswrapper[5052]: I1203 13:13:56.362939 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="barbican-kuttl-tests/openstack-galera-2" Dec 03 13:13:56 crc kubenswrapper[5052]: I1203 13:13:56.363238 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="barbican-kuttl-tests/openstack-galera-2" Dec 03 13:13:56 crc kubenswrapper[5052]: I1203 13:13:56.409455 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="barbican-kuttl-tests/openstack-galera-1" Dec 03 13:13:56 crc kubenswrapper[5052]: I1203 13:13:56.409508 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="barbican-kuttl-tests/openstack-galera-1" Dec 03 13:14:01 crc kubenswrapper[5052]: I1203 13:14:01.109382 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-operators/5d473c3169f40b179d14921c90af2c8546b7b757fe551b7dba7d903f5dc2k4q"] Dec 03 13:14:01 crc kubenswrapper[5052]: I1203 13:14:01.111338 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-operators/5d473c3169f40b179d14921c90af2c8546b7b757fe551b7dba7d903f5dc2k4q" Dec 03 13:14:01 crc kubenswrapper[5052]: I1203 13:14:01.113757 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-operators"/"default-dockercfg-7fklr" Dec 03 13:14:01 crc kubenswrapper[5052]: I1203 13:14:01.124576 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-operators/5d473c3169f40b179d14921c90af2c8546b7b757fe551b7dba7d903f5dc2k4q"] Dec 03 13:14:01 crc kubenswrapper[5052]: I1203 13:14:01.211325 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d5ee6ef6-8fe3-450b-ac6d-e0db3e409540-util\") pod \"5d473c3169f40b179d14921c90af2c8546b7b757fe551b7dba7d903f5dc2k4q\" (UID: \"d5ee6ef6-8fe3-450b-ac6d-e0db3e409540\") " pod="openstack-operators/5d473c3169f40b179d14921c90af2c8546b7b757fe551b7dba7d903f5dc2k4q" Dec 03 13:14:01 crc kubenswrapper[5052]: I1203 13:14:01.211465 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w4c9p\" (UniqueName: \"kubernetes.io/projected/d5ee6ef6-8fe3-450b-ac6d-e0db3e409540-kube-api-access-w4c9p\") pod \"5d473c3169f40b179d14921c90af2c8546b7b757fe551b7dba7d903f5dc2k4q\" (UID: \"d5ee6ef6-8fe3-450b-ac6d-e0db3e409540\") " pod="openstack-operators/5d473c3169f40b179d14921c90af2c8546b7b757fe551b7dba7d903f5dc2k4q" Dec 03 13:14:01 crc kubenswrapper[5052]: I1203 13:14:01.211488 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d5ee6ef6-8fe3-450b-ac6d-e0db3e409540-bundle\") pod \"5d473c3169f40b179d14921c90af2c8546b7b757fe551b7dba7d903f5dc2k4q\" (UID: \"d5ee6ef6-8fe3-450b-ac6d-e0db3e409540\") " pod="openstack-operators/5d473c3169f40b179d14921c90af2c8546b7b757fe551b7dba7d903f5dc2k4q" Dec 03 13:14:01 crc kubenswrapper[5052]: I1203 13:14:01.313338 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d5ee6ef6-8fe3-450b-ac6d-e0db3e409540-util\") pod \"5d473c3169f40b179d14921c90af2c8546b7b757fe551b7dba7d903f5dc2k4q\" (UID: \"d5ee6ef6-8fe3-450b-ac6d-e0db3e409540\") " pod="openstack-operators/5d473c3169f40b179d14921c90af2c8546b7b757fe551b7dba7d903f5dc2k4q" Dec 03 13:14:01 crc kubenswrapper[5052]: I1203 13:14:01.313543 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-w4c9p\" (UniqueName: \"kubernetes.io/projected/d5ee6ef6-8fe3-450b-ac6d-e0db3e409540-kube-api-access-w4c9p\") pod \"5d473c3169f40b179d14921c90af2c8546b7b757fe551b7dba7d903f5dc2k4q\" (UID: \"d5ee6ef6-8fe3-450b-ac6d-e0db3e409540\") " pod="openstack-operators/5d473c3169f40b179d14921c90af2c8546b7b757fe551b7dba7d903f5dc2k4q" Dec 03 13:14:01 crc kubenswrapper[5052]: I1203 13:14:01.313575 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d5ee6ef6-8fe3-450b-ac6d-e0db3e409540-bundle\") pod \"5d473c3169f40b179d14921c90af2c8546b7b757fe551b7dba7d903f5dc2k4q\" (UID: \"d5ee6ef6-8fe3-450b-ac6d-e0db3e409540\") " pod="openstack-operators/5d473c3169f40b179d14921c90af2c8546b7b757fe551b7dba7d903f5dc2k4q" Dec 03 13:14:01 crc kubenswrapper[5052]: I1203 13:14:01.313926 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d5ee6ef6-8fe3-450b-ac6d-e0db3e409540-util\") pod \"5d473c3169f40b179d14921c90af2c8546b7b757fe551b7dba7d903f5dc2k4q\" (UID: \"d5ee6ef6-8fe3-450b-ac6d-e0db3e409540\") " pod="openstack-operators/5d473c3169f40b179d14921c90af2c8546b7b757fe551b7dba7d903f5dc2k4q" Dec 03 13:14:01 crc kubenswrapper[5052]: I1203 13:14:01.314092 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d5ee6ef6-8fe3-450b-ac6d-e0db3e409540-bundle\") pod \"5d473c3169f40b179d14921c90af2c8546b7b757fe551b7dba7d903f5dc2k4q\" (UID: \"d5ee6ef6-8fe3-450b-ac6d-e0db3e409540\") " pod="openstack-operators/5d473c3169f40b179d14921c90af2c8546b7b757fe551b7dba7d903f5dc2k4q" Dec 03 13:14:01 crc kubenswrapper[5052]: I1203 13:14:01.336352 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-w4c9p\" (UniqueName: \"kubernetes.io/projected/d5ee6ef6-8fe3-450b-ac6d-e0db3e409540-kube-api-access-w4c9p\") pod \"5d473c3169f40b179d14921c90af2c8546b7b757fe551b7dba7d903f5dc2k4q\" (UID: \"d5ee6ef6-8fe3-450b-ac6d-e0db3e409540\") " pod="openstack-operators/5d473c3169f40b179d14921c90af2c8546b7b757fe551b7dba7d903f5dc2k4q" Dec 03 13:14:01 crc kubenswrapper[5052]: I1203 13:14:01.427307 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-operators/5d473c3169f40b179d14921c90af2c8546b7b757fe551b7dba7d903f5dc2k4q" Dec 03 13:14:01 crc kubenswrapper[5052]: I1203 13:14:01.651006 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-operators/5d473c3169f40b179d14921c90af2c8546b7b757fe551b7dba7d903f5dc2k4q"] Dec 03 13:14:02 crc kubenswrapper[5052]: I1203 13:14:02.069314 5052 generic.go:334] "Generic (PLEG): container finished" podID="d5ee6ef6-8fe3-450b-ac6d-e0db3e409540" containerID="1ea1cbc762aeca1fb318ff705889f4f8e8d1ed822f4fa40f9b9269799265366b" exitCode=0 Dec 03 13:14:02 crc kubenswrapper[5052]: I1203 13:14:02.069356 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/5d473c3169f40b179d14921c90af2c8546b7b757fe551b7dba7d903f5dc2k4q" event={"ID":"d5ee6ef6-8fe3-450b-ac6d-e0db3e409540","Type":"ContainerDied","Data":"1ea1cbc762aeca1fb318ff705889f4f8e8d1ed822f4fa40f9b9269799265366b"} Dec 03 13:14:02 crc kubenswrapper[5052]: I1203 13:14:02.069382 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/5d473c3169f40b179d14921c90af2c8546b7b757fe551b7dba7d903f5dc2k4q" event={"ID":"d5ee6ef6-8fe3-450b-ac6d-e0db3e409540","Type":"ContainerStarted","Data":"d002599b8e693a25d6b6fd16ed56edc544a637d51653ae5237a5a01c17734208"} Dec 03 13:14:03 crc kubenswrapper[5052]: E1203 13:14:03.299764 5052 upgradeaware.go:441] Error proxying data from backend to client: writeto tcp 38.129.56.252:57884->38.129.56.252:34849: read tcp 38.129.56.252:57884->38.129.56.252:34849: read: connection reset by peer Dec 03 13:14:04 crc kubenswrapper[5052]: I1203 13:14:04.085498 5052 generic.go:334] "Generic (PLEG): container finished" podID="d5ee6ef6-8fe3-450b-ac6d-e0db3e409540" containerID="29bff1b69fb756f4688efc0a60f108814d3e91d0ce26942157ef0c61fb757c45" exitCode=0 Dec 03 13:14:04 crc kubenswrapper[5052]: I1203 13:14:04.085547 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/5d473c3169f40b179d14921c90af2c8546b7b757fe551b7dba7d903f5dc2k4q" event={"ID":"d5ee6ef6-8fe3-450b-ac6d-e0db3e409540","Type":"ContainerDied","Data":"29bff1b69fb756f4688efc0a60f108814d3e91d0ce26942157ef0c61fb757c45"} Dec 03 13:14:04 crc kubenswrapper[5052]: I1203 13:14:04.543532 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="barbican-kuttl-tests/openstack-galera-2" Dec 03 13:14:04 crc kubenswrapper[5052]: I1203 13:14:04.611456 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="barbican-kuttl-tests/openstack-galera-2" Dec 03 13:14:04 crc kubenswrapper[5052]: E1203 13:14:04.727013 5052 upgradeaware.go:441] Error proxying data from backend to client: writeto tcp 38.129.56.252:47158->38.129.56.252:34849: read tcp 38.129.56.252:47158->38.129.56.252:34849: read: connection reset by peer Dec 03 13:14:05 crc kubenswrapper[5052]: I1203 13:14:05.094930 5052 generic.go:334] "Generic (PLEG): container finished" podID="d5ee6ef6-8fe3-450b-ac6d-e0db3e409540" containerID="a36b5fe28d987ac737b31620a308b4c87b943989ef9a1150ae5273524d39fe57" exitCode=0 Dec 03 13:14:05 crc kubenswrapper[5052]: I1203 13:14:05.095116 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/5d473c3169f40b179d14921c90af2c8546b7b757fe551b7dba7d903f5dc2k4q" event={"ID":"d5ee6ef6-8fe3-450b-ac6d-e0db3e409540","Type":"ContainerDied","Data":"a36b5fe28d987ac737b31620a308b4c87b943989ef9a1150ae5273524d39fe57"} Dec 03 13:14:06 crc kubenswrapper[5052]: I1203 13:14:06.340285 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-operators/5d473c3169f40b179d14921c90af2c8546b7b757fe551b7dba7d903f5dc2k4q" Dec 03 13:14:06 crc kubenswrapper[5052]: I1203 13:14:06.497233 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d5ee6ef6-8fe3-450b-ac6d-e0db3e409540-util\") pod \"d5ee6ef6-8fe3-450b-ac6d-e0db3e409540\" (UID: \"d5ee6ef6-8fe3-450b-ac6d-e0db3e409540\") " Dec 03 13:14:06 crc kubenswrapper[5052]: I1203 13:14:06.497310 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d5ee6ef6-8fe3-450b-ac6d-e0db3e409540-bundle\") pod \"d5ee6ef6-8fe3-450b-ac6d-e0db3e409540\" (UID: \"d5ee6ef6-8fe3-450b-ac6d-e0db3e409540\") " Dec 03 13:14:06 crc kubenswrapper[5052]: I1203 13:14:06.497371 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w4c9p\" (UniqueName: \"kubernetes.io/projected/d5ee6ef6-8fe3-450b-ac6d-e0db3e409540-kube-api-access-w4c9p\") pod \"d5ee6ef6-8fe3-450b-ac6d-e0db3e409540\" (UID: \"d5ee6ef6-8fe3-450b-ac6d-e0db3e409540\") " Dec 03 13:14:06 crc kubenswrapper[5052]: I1203 13:14:06.499287 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d5ee6ef6-8fe3-450b-ac6d-e0db3e409540-bundle" (OuterVolumeSpecName: "bundle") pod "d5ee6ef6-8fe3-450b-ac6d-e0db3e409540" (UID: "d5ee6ef6-8fe3-450b-ac6d-e0db3e409540"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 13:14:06 crc kubenswrapper[5052]: I1203 13:14:06.504013 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d5ee6ef6-8fe3-450b-ac6d-e0db3e409540-kube-api-access-w4c9p" (OuterVolumeSpecName: "kube-api-access-w4c9p") pod "d5ee6ef6-8fe3-450b-ac6d-e0db3e409540" (UID: "d5ee6ef6-8fe3-450b-ac6d-e0db3e409540"). InnerVolumeSpecName "kube-api-access-w4c9p". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:14:06 crc kubenswrapper[5052]: I1203 13:14:06.519043 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d5ee6ef6-8fe3-450b-ac6d-e0db3e409540-util" (OuterVolumeSpecName: "util") pod "d5ee6ef6-8fe3-450b-ac6d-e0db3e409540" (UID: "d5ee6ef6-8fe3-450b-ac6d-e0db3e409540"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 13:14:06 crc kubenswrapper[5052]: I1203 13:14:06.598609 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-w4c9p\" (UniqueName: \"kubernetes.io/projected/d5ee6ef6-8fe3-450b-ac6d-e0db3e409540-kube-api-access-w4c9p\") on node \"crc\" DevicePath \"\"" Dec 03 13:14:06 crc kubenswrapper[5052]: I1203 13:14:06.598652 5052 reconciler_common.go:293] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d5ee6ef6-8fe3-450b-ac6d-e0db3e409540-util\") on node \"crc\" DevicePath \"\"" Dec 03 13:14:06 crc kubenswrapper[5052]: I1203 13:14:06.598662 5052 reconciler_common.go:293] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d5ee6ef6-8fe3-450b-ac6d-e0db3e409540-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 13:14:07 crc kubenswrapper[5052]: I1203 13:14:07.109133 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/5d473c3169f40b179d14921c90af2c8546b7b757fe551b7dba7d903f5dc2k4q" event={"ID":"d5ee6ef6-8fe3-450b-ac6d-e0db3e409540","Type":"ContainerDied","Data":"d002599b8e693a25d6b6fd16ed56edc544a637d51653ae5237a5a01c17734208"} Dec 03 13:14:07 crc kubenswrapper[5052]: I1203 13:14:07.109470 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d002599b8e693a25d6b6fd16ed56edc544a637d51653ae5237a5a01c17734208" Dec 03 13:14:07 crc kubenswrapper[5052]: I1203 13:14:07.109196 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-operators/5d473c3169f40b179d14921c90af2c8546b7b757fe551b7dba7d903f5dc2k4q" Dec 03 13:14:13 crc kubenswrapper[5052]: I1203 13:14:13.532128 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="barbican-kuttl-tests/openstack-galera-0" Dec 03 13:14:13 crc kubenswrapper[5052]: I1203 13:14:13.625495 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="barbican-kuttl-tests/openstack-galera-0" Dec 03 13:14:19 crc kubenswrapper[5052]: I1203 13:14:19.059357 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="barbican-kuttl-tests/openstack-galera-1" Dec 03 13:14:19 crc kubenswrapper[5052]: I1203 13:14:19.136420 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="barbican-kuttl-tests/openstack-galera-1" Dec 03 13:14:20 crc kubenswrapper[5052]: I1203 13:14:20.140781 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-operators/infra-operator-controller-manager-58dbf89689-bnlrt"] Dec 03 13:14:20 crc kubenswrapper[5052]: E1203 13:14:20.141361 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="d5ee6ef6-8fe3-450b-ac6d-e0db3e409540" containerName="extract" Dec 03 13:14:20 crc kubenswrapper[5052]: I1203 13:14:20.141374 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="d5ee6ef6-8fe3-450b-ac6d-e0db3e409540" containerName="extract" Dec 03 13:14:20 crc kubenswrapper[5052]: E1203 13:14:20.141393 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="d5ee6ef6-8fe3-450b-ac6d-e0db3e409540" containerName="util" Dec 03 13:14:20 crc kubenswrapper[5052]: I1203 13:14:20.141399 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="d5ee6ef6-8fe3-450b-ac6d-e0db3e409540" containerName="util" Dec 03 13:14:20 crc kubenswrapper[5052]: E1203 13:14:20.141421 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="d5ee6ef6-8fe3-450b-ac6d-e0db3e409540" containerName="pull" Dec 03 13:14:20 crc kubenswrapper[5052]: I1203 13:14:20.141428 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="d5ee6ef6-8fe3-450b-ac6d-e0db3e409540" containerName="pull" Dec 03 13:14:20 crc kubenswrapper[5052]: I1203 13:14:20.141581 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="d5ee6ef6-8fe3-450b-ac6d-e0db3e409540" containerName="extract" Dec 03 13:14:20 crc kubenswrapper[5052]: I1203 13:14:20.142307 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-operators/infra-operator-controller-manager-58dbf89689-bnlrt" Dec 03 13:14:20 crc kubenswrapper[5052]: I1203 13:14:20.145103 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-operators"/"infra-operator-controller-manager-service-cert" Dec 03 13:14:20 crc kubenswrapper[5052]: I1203 13:14:20.145132 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-operators"/"infra-operator-controller-manager-dockercfg-xpnfv" Dec 03 13:14:20 crc kubenswrapper[5052]: I1203 13:14:20.165405 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-operators/infra-operator-controller-manager-58dbf89689-bnlrt"] Dec 03 13:14:20 crc kubenswrapper[5052]: I1203 13:14:20.210219 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5f441756-e059-4db0-943e-eafb2d48e668-webhook-cert\") pod \"infra-operator-controller-manager-58dbf89689-bnlrt\" (UID: \"5f441756-e059-4db0-943e-eafb2d48e668\") " pod="openstack-operators/infra-operator-controller-manager-58dbf89689-bnlrt" Dec 03 13:14:20 crc kubenswrapper[5052]: I1203 13:14:20.210300 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/5f441756-e059-4db0-943e-eafb2d48e668-apiservice-cert\") pod \"infra-operator-controller-manager-58dbf89689-bnlrt\" (UID: \"5f441756-e059-4db0-943e-eafb2d48e668\") " pod="openstack-operators/infra-operator-controller-manager-58dbf89689-bnlrt" Dec 03 13:14:20 crc kubenswrapper[5052]: I1203 13:14:20.210338 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-66lhs\" (UniqueName: \"kubernetes.io/projected/5f441756-e059-4db0-943e-eafb2d48e668-kube-api-access-66lhs\") pod \"infra-operator-controller-manager-58dbf89689-bnlrt\" (UID: \"5f441756-e059-4db0-943e-eafb2d48e668\") " pod="openstack-operators/infra-operator-controller-manager-58dbf89689-bnlrt" Dec 03 13:14:20 crc kubenswrapper[5052]: I1203 13:14:20.311957 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5f441756-e059-4db0-943e-eafb2d48e668-webhook-cert\") pod \"infra-operator-controller-manager-58dbf89689-bnlrt\" (UID: \"5f441756-e059-4db0-943e-eafb2d48e668\") " pod="openstack-operators/infra-operator-controller-manager-58dbf89689-bnlrt" Dec 03 13:14:20 crc kubenswrapper[5052]: I1203 13:14:20.312033 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/5f441756-e059-4db0-943e-eafb2d48e668-apiservice-cert\") pod \"infra-operator-controller-manager-58dbf89689-bnlrt\" (UID: \"5f441756-e059-4db0-943e-eafb2d48e668\") " pod="openstack-operators/infra-operator-controller-manager-58dbf89689-bnlrt" Dec 03 13:14:20 crc kubenswrapper[5052]: I1203 13:14:20.312075 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-66lhs\" (UniqueName: \"kubernetes.io/projected/5f441756-e059-4db0-943e-eafb2d48e668-kube-api-access-66lhs\") pod \"infra-operator-controller-manager-58dbf89689-bnlrt\" (UID: \"5f441756-e059-4db0-943e-eafb2d48e668\") " pod="openstack-operators/infra-operator-controller-manager-58dbf89689-bnlrt" Dec 03 13:14:20 crc kubenswrapper[5052]: I1203 13:14:20.320251 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/5f441756-e059-4db0-943e-eafb2d48e668-apiservice-cert\") pod \"infra-operator-controller-manager-58dbf89689-bnlrt\" (UID: \"5f441756-e059-4db0-943e-eafb2d48e668\") " pod="openstack-operators/infra-operator-controller-manager-58dbf89689-bnlrt" Dec 03 13:14:20 crc kubenswrapper[5052]: I1203 13:14:20.321094 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5f441756-e059-4db0-943e-eafb2d48e668-webhook-cert\") pod \"infra-operator-controller-manager-58dbf89689-bnlrt\" (UID: \"5f441756-e059-4db0-943e-eafb2d48e668\") " pod="openstack-operators/infra-operator-controller-manager-58dbf89689-bnlrt" Dec 03 13:14:20 crc kubenswrapper[5052]: I1203 13:14:20.349693 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-66lhs\" (UniqueName: \"kubernetes.io/projected/5f441756-e059-4db0-943e-eafb2d48e668-kube-api-access-66lhs\") pod \"infra-operator-controller-manager-58dbf89689-bnlrt\" (UID: \"5f441756-e059-4db0-943e-eafb2d48e668\") " pod="openstack-operators/infra-operator-controller-manager-58dbf89689-bnlrt" Dec 03 13:14:20 crc kubenswrapper[5052]: I1203 13:14:20.463315 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-operators/infra-operator-controller-manager-58dbf89689-bnlrt" Dec 03 13:14:20 crc kubenswrapper[5052]: I1203 13:14:20.802940 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-operators/infra-operator-controller-manager-58dbf89689-bnlrt"] Dec 03 13:14:21 crc kubenswrapper[5052]: I1203 13:14:21.206747 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/infra-operator-controller-manager-58dbf89689-bnlrt" event={"ID":"5f441756-e059-4db0-943e-eafb2d48e668","Type":"ContainerStarted","Data":"b25eafbee44eb19fc61d68c549818ea4ebc9cc256b939273da70c1836a1a40b2"} Dec 03 13:14:21 crc kubenswrapper[5052]: I1203 13:14:21.207074 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-operators/infra-operator-controller-manager-58dbf89689-bnlrt" Dec 03 13:14:21 crc kubenswrapper[5052]: I1203 13:14:21.207085 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/infra-operator-controller-manager-58dbf89689-bnlrt" event={"ID":"5f441756-e059-4db0-943e-eafb2d48e668","Type":"ContainerStarted","Data":"75dfc7b94c9bc6c8c51aeb2593ca52f48a46a8ea64b1eb3540d4ca2bc6dd492e"} Dec 03 13:14:21 crc kubenswrapper[5052]: I1203 13:14:21.207094 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/infra-operator-controller-manager-58dbf89689-bnlrt" event={"ID":"5f441756-e059-4db0-943e-eafb2d48e668","Type":"ContainerStarted","Data":"d289f264cd7289e4d87674831df826d8924fdb666dad9b363be016783449ff13"} Dec 03 13:14:21 crc kubenswrapper[5052]: I1203 13:14:21.233584 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-operators/infra-operator-controller-manager-58dbf89689-bnlrt" podStartSLOduration=1.233564336 podStartE2EDuration="1.233564336s" podCreationTimestamp="2025-12-03 13:14:20 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 13:14:21.230282272 +0000 UTC m=+6851.818524036" watchObservedRunningTime="2025-12-03 13:14:21.233564336 +0000 UTC m=+6851.821806100" Dec 03 13:14:29 crc kubenswrapper[5052]: I1203 13:14:29.379971 5052 patch_prober.go:28] interesting pod/machine-config-daemon-74bj6 container/machine-config-daemon namespace/openshift-machine-config-operator: Liveness probe status=failure output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" start-of-body= Dec 03 13:14:29 crc kubenswrapper[5052]: I1203 13:14:29.380546 5052 prober.go:107] "Probe failed" probeType="Liveness" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" containerName="machine-config-daemon" probeResult="failure" output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" Dec 03 13:14:30 crc kubenswrapper[5052]: I1203 13:14:30.468544 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-operators/infra-operator-controller-manager-58dbf89689-bnlrt" Dec 03 13:14:38 crc kubenswrapper[5052]: I1203 13:14:38.259894 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-operators/rabbitmq-cluster-operator-index-xzbt6"] Dec 03 13:14:38 crc kubenswrapper[5052]: I1203 13:14:38.261495 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-operators/rabbitmq-cluster-operator-index-xzbt6" Dec 03 13:14:38 crc kubenswrapper[5052]: I1203 13:14:38.263800 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-operators"/"rabbitmq-cluster-operator-index-dockercfg-db7mc" Dec 03 13:14:38 crc kubenswrapper[5052]: I1203 13:14:38.270305 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-operators/rabbitmq-cluster-operator-index-xzbt6"] Dec 03 13:14:38 crc kubenswrapper[5052]: I1203 13:14:38.396969 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ph7wd\" (UniqueName: \"kubernetes.io/projected/0c10ada3-f983-43bc-a010-8a8d88893838-kube-api-access-ph7wd\") pod \"rabbitmq-cluster-operator-index-xzbt6\" (UID: \"0c10ada3-f983-43bc-a010-8a8d88893838\") " pod="openstack-operators/rabbitmq-cluster-operator-index-xzbt6" Dec 03 13:14:38 crc kubenswrapper[5052]: I1203 13:14:38.498779 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-ph7wd\" (UniqueName: \"kubernetes.io/projected/0c10ada3-f983-43bc-a010-8a8d88893838-kube-api-access-ph7wd\") pod \"rabbitmq-cluster-operator-index-xzbt6\" (UID: \"0c10ada3-f983-43bc-a010-8a8d88893838\") " pod="openstack-operators/rabbitmq-cluster-operator-index-xzbt6" Dec 03 13:14:38 crc kubenswrapper[5052]: I1203 13:14:38.520758 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-ph7wd\" (UniqueName: \"kubernetes.io/projected/0c10ada3-f983-43bc-a010-8a8d88893838-kube-api-access-ph7wd\") pod \"rabbitmq-cluster-operator-index-xzbt6\" (UID: \"0c10ada3-f983-43bc-a010-8a8d88893838\") " pod="openstack-operators/rabbitmq-cluster-operator-index-xzbt6" Dec 03 13:14:38 crc kubenswrapper[5052]: I1203 13:14:38.622150 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-operators/rabbitmq-cluster-operator-index-xzbt6" Dec 03 13:14:39 crc kubenswrapper[5052]: I1203 13:14:39.052590 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-operators/rabbitmq-cluster-operator-index-xzbt6"] Dec 03 13:14:39 crc kubenswrapper[5052]: I1203 13:14:39.346796 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/rabbitmq-cluster-operator-index-xzbt6" event={"ID":"0c10ada3-f983-43bc-a010-8a8d88893838","Type":"ContainerStarted","Data":"7e9c45609aebe87a8e52ec85abe07307c9a4ea6f6698cd5fc781c0fadd5bf67d"} Dec 03 13:14:43 crc kubenswrapper[5052]: I1203 13:14:43.051863 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-operators/rabbitmq-cluster-operator-index-xzbt6"] Dec 03 13:14:43 crc kubenswrapper[5052]: I1203 13:14:43.859909 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-operators/rabbitmq-cluster-operator-index-49tjq"] Dec 03 13:14:43 crc kubenswrapper[5052]: I1203 13:14:43.863624 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-operators/rabbitmq-cluster-operator-index-49tjq" Dec 03 13:14:43 crc kubenswrapper[5052]: I1203 13:14:43.870355 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-operators/rabbitmq-cluster-operator-index-49tjq"] Dec 03 13:14:43 crc kubenswrapper[5052]: I1203 13:14:43.911238 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g54gh\" (UniqueName: \"kubernetes.io/projected/0abdde9a-bebd-4ece-9fb6-0cf23ba9a11a-kube-api-access-g54gh\") pod \"rabbitmq-cluster-operator-index-49tjq\" (UID: \"0abdde9a-bebd-4ece-9fb6-0cf23ba9a11a\") " pod="openstack-operators/rabbitmq-cluster-operator-index-49tjq" Dec 03 13:14:44 crc kubenswrapper[5052]: I1203 13:14:44.013748 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-g54gh\" (UniqueName: \"kubernetes.io/projected/0abdde9a-bebd-4ece-9fb6-0cf23ba9a11a-kube-api-access-g54gh\") pod \"rabbitmq-cluster-operator-index-49tjq\" (UID: \"0abdde9a-bebd-4ece-9fb6-0cf23ba9a11a\") " pod="openstack-operators/rabbitmq-cluster-operator-index-49tjq" Dec 03 13:14:44 crc kubenswrapper[5052]: I1203 13:14:44.034948 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-g54gh\" (UniqueName: \"kubernetes.io/projected/0abdde9a-bebd-4ece-9fb6-0cf23ba9a11a-kube-api-access-g54gh\") pod \"rabbitmq-cluster-operator-index-49tjq\" (UID: \"0abdde9a-bebd-4ece-9fb6-0cf23ba9a11a\") " pod="openstack-operators/rabbitmq-cluster-operator-index-49tjq" Dec 03 13:14:44 crc kubenswrapper[5052]: I1203 13:14:44.188973 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-operators/rabbitmq-cluster-operator-index-49tjq" Dec 03 13:14:44 crc kubenswrapper[5052]: I1203 13:14:44.419370 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/rabbitmq-cluster-operator-index-xzbt6" event={"ID":"0c10ada3-f983-43bc-a010-8a8d88893838","Type":"ContainerStarted","Data":"2387e2e2e3309d882ea29e2b21fe9d0f494c7ccc7e2664018c29ad1ce2123d0d"} Dec 03 13:14:44 crc kubenswrapper[5052]: I1203 13:14:44.419712 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-operators/rabbitmq-cluster-operator-index-xzbt6" podUID="0c10ada3-f983-43bc-a010-8a8d88893838" containerName="registry-server" containerID="cri-o://2387e2e2e3309d882ea29e2b21fe9d0f494c7ccc7e2664018c29ad1ce2123d0d" gracePeriod=2 Dec 03 13:14:44 crc kubenswrapper[5052]: I1203 13:14:44.446036 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-operators/rabbitmq-cluster-operator-index-xzbt6" podStartSLOduration=1.845123147 podStartE2EDuration="6.446014216s" podCreationTimestamp="2025-12-03 13:14:38 +0000 UTC" firstStartedPulling="2025-12-03 13:14:39.060160857 +0000 UTC m=+6869.648402621" lastFinishedPulling="2025-12-03 13:14:43.661051926 +0000 UTC m=+6874.249293690" observedRunningTime="2025-12-03 13:14:44.437125871 +0000 UTC m=+6875.025367645" watchObservedRunningTime="2025-12-03 13:14:44.446014216 +0000 UTC m=+6875.034256000" Dec 03 13:14:44 crc kubenswrapper[5052]: I1203 13:14:44.606241 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-operators/rabbitmq-cluster-operator-index-49tjq"] Dec 03 13:14:44 crc kubenswrapper[5052]: W1203 13:14:44.613010 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0abdde9a_bebd_4ece_9fb6_0cf23ba9a11a.slice/crio-95345410d6c07e824f9c799487121152ca64915b0aac177dbdc7807c478d3e3c WatchSource:0}: Error finding container 95345410d6c07e824f9c799487121152ca64915b0aac177dbdc7807c478d3e3c: Status 404 returned error can't find the container with id 95345410d6c07e824f9c799487121152ca64915b0aac177dbdc7807c478d3e3c Dec 03 13:14:45 crc kubenswrapper[5052]: I1203 13:14:45.431416 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/rabbitmq-cluster-operator-index-49tjq" event={"ID":"0abdde9a-bebd-4ece-9fb6-0cf23ba9a11a","Type":"ContainerStarted","Data":"95345410d6c07e824f9c799487121152ca64915b0aac177dbdc7807c478d3e3c"} Dec 03 13:14:45 crc kubenswrapper[5052]: I1203 13:14:45.434367 5052 generic.go:334] "Generic (PLEG): container finished" podID="0c10ada3-f983-43bc-a010-8a8d88893838" containerID="2387e2e2e3309d882ea29e2b21fe9d0f494c7ccc7e2664018c29ad1ce2123d0d" exitCode=0 Dec 03 13:14:45 crc kubenswrapper[5052]: I1203 13:14:45.434415 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/rabbitmq-cluster-operator-index-xzbt6" event={"ID":"0c10ada3-f983-43bc-a010-8a8d88893838","Type":"ContainerDied","Data":"2387e2e2e3309d882ea29e2b21fe9d0f494c7ccc7e2664018c29ad1ce2123d0d"} Dec 03 13:14:45 crc kubenswrapper[5052]: I1203 13:14:45.807416 5052 scope.go:117] "RemoveContainer" containerID="6019fc94d6b5f73e8d9826bdfe024871298a635aa8224470991ea4f8f488a20a" Dec 03 13:14:45 crc kubenswrapper[5052]: I1203 13:14:45.833903 5052 scope.go:117] "RemoveContainer" containerID="0b7a53ad8d1342040ea33fe43e227663b52d1086c7041a72d7d45693dec92e68" Dec 03 13:14:45 crc kubenswrapper[5052]: I1203 13:14:45.933043 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-operators/rabbitmq-cluster-operator-index-xzbt6" Dec 03 13:14:46 crc kubenswrapper[5052]: I1203 13:14:46.040085 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ph7wd\" (UniqueName: \"kubernetes.io/projected/0c10ada3-f983-43bc-a010-8a8d88893838-kube-api-access-ph7wd\") pod \"0c10ada3-f983-43bc-a010-8a8d88893838\" (UID: \"0c10ada3-f983-43bc-a010-8a8d88893838\") " Dec 03 13:14:46 crc kubenswrapper[5052]: I1203 13:14:46.046790 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0c10ada3-f983-43bc-a010-8a8d88893838-kube-api-access-ph7wd" (OuterVolumeSpecName: "kube-api-access-ph7wd") pod "0c10ada3-f983-43bc-a010-8a8d88893838" (UID: "0c10ada3-f983-43bc-a010-8a8d88893838"). InnerVolumeSpecName "kube-api-access-ph7wd". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:14:46 crc kubenswrapper[5052]: I1203 13:14:46.142380 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-ph7wd\" (UniqueName: \"kubernetes.io/projected/0c10ada3-f983-43bc-a010-8a8d88893838-kube-api-access-ph7wd\") on node \"crc\" DevicePath \"\"" Dec 03 13:14:46 crc kubenswrapper[5052]: I1203 13:14:46.444893 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/rabbitmq-cluster-operator-index-49tjq" event={"ID":"0abdde9a-bebd-4ece-9fb6-0cf23ba9a11a","Type":"ContainerStarted","Data":"de5964c1bb69ca89e1f609bdd5b508f4c04326b4ef02da9f9683bc08a72d7ef3"} Dec 03 13:14:46 crc kubenswrapper[5052]: I1203 13:14:46.455722 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/rabbitmq-cluster-operator-index-xzbt6" event={"ID":"0c10ada3-f983-43bc-a010-8a8d88893838","Type":"ContainerDied","Data":"7e9c45609aebe87a8e52ec85abe07307c9a4ea6f6698cd5fc781c0fadd5bf67d"} Dec 03 13:14:46 crc kubenswrapper[5052]: I1203 13:14:46.455780 5052 scope.go:117] "RemoveContainer" containerID="2387e2e2e3309d882ea29e2b21fe9d0f494c7ccc7e2664018c29ad1ce2123d0d" Dec 03 13:14:46 crc kubenswrapper[5052]: I1203 13:14:46.455954 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-operators/rabbitmq-cluster-operator-index-xzbt6" Dec 03 13:14:46 crc kubenswrapper[5052]: I1203 13:14:46.464096 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-operators/rabbitmq-cluster-operator-index-49tjq" podStartSLOduration=2.591897472 podStartE2EDuration="3.464077078s" podCreationTimestamp="2025-12-03 13:14:43 +0000 UTC" firstStartedPulling="2025-12-03 13:14:44.618695891 +0000 UTC m=+6875.206937655" lastFinishedPulling="2025-12-03 13:14:45.490875497 +0000 UTC m=+6876.079117261" observedRunningTime="2025-12-03 13:14:46.462678318 +0000 UTC m=+6877.050920092" watchObservedRunningTime="2025-12-03 13:14:46.464077078 +0000 UTC m=+6877.052318842" Dec 03 13:14:46 crc kubenswrapper[5052]: I1203 13:14:46.498324 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-operators/rabbitmq-cluster-operator-index-xzbt6"] Dec 03 13:14:46 crc kubenswrapper[5052]: I1203 13:14:46.503760 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-operators/rabbitmq-cluster-operator-index-xzbt6"] Dec 03 13:14:47 crc kubenswrapper[5052]: I1203 13:14:47.681638 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0c10ada3-f983-43bc-a010-8a8d88893838" path="/var/lib/kubelet/pods/0c10ada3-f983-43bc-a010-8a8d88893838/volumes" Dec 03 13:14:54 crc kubenswrapper[5052]: I1203 13:14:54.189538 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-operators/rabbitmq-cluster-operator-index-49tjq" Dec 03 13:14:54 crc kubenswrapper[5052]: I1203 13:14:54.190712 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-operators/rabbitmq-cluster-operator-index-49tjq" Dec 03 13:14:54 crc kubenswrapper[5052]: I1203 13:14:54.218435 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-operators/rabbitmq-cluster-operator-index-49tjq" Dec 03 13:14:54 crc kubenswrapper[5052]: I1203 13:14:54.545573 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-operators/rabbitmq-cluster-operator-index-49tjq" Dec 03 13:14:55 crc kubenswrapper[5052]: I1203 13:14:55.421239 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["barbican-kuttl-tests/memcached-0"] Dec 03 13:14:55 crc kubenswrapper[5052]: E1203 13:14:55.421616 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="0c10ada3-f983-43bc-a010-8a8d88893838" containerName="registry-server" Dec 03 13:14:55 crc kubenswrapper[5052]: I1203 13:14:55.421630 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="0c10ada3-f983-43bc-a010-8a8d88893838" containerName="registry-server" Dec 03 13:14:55 crc kubenswrapper[5052]: I1203 13:14:55.421790 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="0c10ada3-f983-43bc-a010-8a8d88893838" containerName="registry-server" Dec 03 13:14:55 crc kubenswrapper[5052]: I1203 13:14:55.422247 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="barbican-kuttl-tests/memcached-0" Dec 03 13:14:55 crc kubenswrapper[5052]: I1203 13:14:55.424391 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"barbican-kuttl-tests"/"memcached-config-data" Dec 03 13:14:55 crc kubenswrapper[5052]: I1203 13:14:55.424563 5052 reflector.go:368] Caches populated for *v1.Secret from object-"barbican-kuttl-tests"/"memcached-memcached-dockercfg-54slp" Dec 03 13:14:55 crc kubenswrapper[5052]: I1203 13:14:55.441859 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["barbican-kuttl-tests/memcached-0"] Dec 03 13:14:55 crc kubenswrapper[5052]: I1203 13:14:55.487012 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/6ccef89f-58bd-4e50-9d8c-d5f076df3505-kolla-config\") pod \"memcached-0\" (UID: \"6ccef89f-58bd-4e50-9d8c-d5f076df3505\") " pod="barbican-kuttl-tests/memcached-0" Dec 03 13:14:55 crc kubenswrapper[5052]: I1203 13:14:55.487178 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/6ccef89f-58bd-4e50-9d8c-d5f076df3505-config-data\") pod \"memcached-0\" (UID: \"6ccef89f-58bd-4e50-9d8c-d5f076df3505\") " pod="barbican-kuttl-tests/memcached-0" Dec 03 13:14:55 crc kubenswrapper[5052]: I1203 13:14:55.487251 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vvm2j\" (UniqueName: \"kubernetes.io/projected/6ccef89f-58bd-4e50-9d8c-d5f076df3505-kube-api-access-vvm2j\") pod \"memcached-0\" (UID: \"6ccef89f-58bd-4e50-9d8c-d5f076df3505\") " pod="barbican-kuttl-tests/memcached-0" Dec 03 13:14:55 crc kubenswrapper[5052]: I1203 13:14:55.589302 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/6ccef89f-58bd-4e50-9d8c-d5f076df3505-kolla-config\") pod \"memcached-0\" (UID: \"6ccef89f-58bd-4e50-9d8c-d5f076df3505\") " pod="barbican-kuttl-tests/memcached-0" Dec 03 13:14:55 crc kubenswrapper[5052]: I1203 13:14:55.589351 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/6ccef89f-58bd-4e50-9d8c-d5f076df3505-config-data\") pod \"memcached-0\" (UID: \"6ccef89f-58bd-4e50-9d8c-d5f076df3505\") " pod="barbican-kuttl-tests/memcached-0" Dec 03 13:14:55 crc kubenswrapper[5052]: I1203 13:14:55.589375 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-vvm2j\" (UniqueName: \"kubernetes.io/projected/6ccef89f-58bd-4e50-9d8c-d5f076df3505-kube-api-access-vvm2j\") pod \"memcached-0\" (UID: \"6ccef89f-58bd-4e50-9d8c-d5f076df3505\") " pod="barbican-kuttl-tests/memcached-0" Dec 03 13:14:55 crc kubenswrapper[5052]: I1203 13:14:55.590153 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/6ccef89f-58bd-4e50-9d8c-d5f076df3505-kolla-config\") pod \"memcached-0\" (UID: \"6ccef89f-58bd-4e50-9d8c-d5f076df3505\") " pod="barbican-kuttl-tests/memcached-0" Dec 03 13:14:55 crc kubenswrapper[5052]: I1203 13:14:55.590383 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/6ccef89f-58bd-4e50-9d8c-d5f076df3505-config-data\") pod \"memcached-0\" (UID: \"6ccef89f-58bd-4e50-9d8c-d5f076df3505\") " pod="barbican-kuttl-tests/memcached-0" Dec 03 13:14:55 crc kubenswrapper[5052]: I1203 13:14:55.610606 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-vvm2j\" (UniqueName: \"kubernetes.io/projected/6ccef89f-58bd-4e50-9d8c-d5f076df3505-kube-api-access-vvm2j\") pod \"memcached-0\" (UID: \"6ccef89f-58bd-4e50-9d8c-d5f076df3505\") " pod="barbican-kuttl-tests/memcached-0" Dec 03 13:14:55 crc kubenswrapper[5052]: I1203 13:14:55.741212 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="barbican-kuttl-tests/memcached-0" Dec 03 13:14:56 crc kubenswrapper[5052]: I1203 13:14:56.189385 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["barbican-kuttl-tests/memcached-0"] Dec 03 13:14:56 crc kubenswrapper[5052]: W1203 13:14:56.192653 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6ccef89f_58bd_4e50_9d8c_d5f076df3505.slice/crio-7d4e966dd4615fabff47530d8b49c16e88d16eace928aaa59144c2eb3a54c27c WatchSource:0}: Error finding container 7d4e966dd4615fabff47530d8b49c16e88d16eace928aaa59144c2eb3a54c27c: Status 404 returned error can't find the container with id 7d4e966dd4615fabff47530d8b49c16e88d16eace928aaa59144c2eb3a54c27c Dec 03 13:14:56 crc kubenswrapper[5052]: I1203 13:14:56.530704 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="barbican-kuttl-tests/memcached-0" event={"ID":"6ccef89f-58bd-4e50-9d8c-d5f076df3505","Type":"ContainerStarted","Data":"7d4e966dd4615fabff47530d8b49c16e88d16eace928aaa59144c2eb3a54c27c"} Dec 03 13:14:59 crc kubenswrapper[5052]: I1203 13:14:59.379485 5052 patch_prober.go:28] interesting pod/machine-config-daemon-74bj6 container/machine-config-daemon namespace/openshift-machine-config-operator: Liveness probe status=failure output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" start-of-body= Dec 03 13:14:59 crc kubenswrapper[5052]: I1203 13:14:59.379836 5052 prober.go:107] "Probe failed" probeType="Liveness" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" containerName="machine-config-daemon" probeResult="failure" output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" Dec 03 13:15:00 crc kubenswrapper[5052]: I1203 13:15:00.142758 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openshift-operator-lifecycle-manager/collect-profiles-29412795-zq67h"] Dec 03 13:15:00 crc kubenswrapper[5052]: I1203 13:15:00.145472 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-operator-lifecycle-manager/collect-profiles-29412795-zq67h" Dec 03 13:15:00 crc kubenswrapper[5052]: I1203 13:15:00.147978 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-operator-lifecycle-manager"/"collect-profiles-dockercfg-kzf4t" Dec 03 13:15:00 crc kubenswrapper[5052]: I1203 13:15:00.148931 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-operator-lifecycle-manager"/"collect-profiles-config" Dec 03 13:15:00 crc kubenswrapper[5052]: I1203 13:15:00.150467 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-operator-lifecycle-manager/collect-profiles-29412795-zq67h"] Dec 03 13:15:00 crc kubenswrapper[5052]: I1203 13:15:00.261278 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-volume\" (UniqueName: \"kubernetes.io/secret/7f92038f-ab2b-4ce3-86ab-fa83e4d889a9-secret-volume\") pod \"collect-profiles-29412795-zq67h\" (UID: \"7f92038f-ab2b-4ce3-86ab-fa83e4d889a9\") " pod="openshift-operator-lifecycle-manager/collect-profiles-29412795-zq67h" Dec 03 13:15:00 crc kubenswrapper[5052]: I1203 13:15:00.261382 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/7f92038f-ab2b-4ce3-86ab-fa83e4d889a9-config-volume\") pod \"collect-profiles-29412795-zq67h\" (UID: \"7f92038f-ab2b-4ce3-86ab-fa83e4d889a9\") " pod="openshift-operator-lifecycle-manager/collect-profiles-29412795-zq67h" Dec 03 13:15:00 crc kubenswrapper[5052]: I1203 13:15:00.261416 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5nk4m\" (UniqueName: \"kubernetes.io/projected/7f92038f-ab2b-4ce3-86ab-fa83e4d889a9-kube-api-access-5nk4m\") pod \"collect-profiles-29412795-zq67h\" (UID: \"7f92038f-ab2b-4ce3-86ab-fa83e4d889a9\") " pod="openshift-operator-lifecycle-manager/collect-profiles-29412795-zq67h" Dec 03 13:15:00 crc kubenswrapper[5052]: I1203 13:15:00.362830 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"secret-volume\" (UniqueName: \"kubernetes.io/secret/7f92038f-ab2b-4ce3-86ab-fa83e4d889a9-secret-volume\") pod \"collect-profiles-29412795-zq67h\" (UID: \"7f92038f-ab2b-4ce3-86ab-fa83e4d889a9\") " pod="openshift-operator-lifecycle-manager/collect-profiles-29412795-zq67h" Dec 03 13:15:00 crc kubenswrapper[5052]: I1203 13:15:00.362960 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/7f92038f-ab2b-4ce3-86ab-fa83e4d889a9-config-volume\") pod \"collect-profiles-29412795-zq67h\" (UID: \"7f92038f-ab2b-4ce3-86ab-fa83e4d889a9\") " pod="openshift-operator-lifecycle-manager/collect-profiles-29412795-zq67h" Dec 03 13:15:00 crc kubenswrapper[5052]: I1203 13:15:00.363003 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-5nk4m\" (UniqueName: \"kubernetes.io/projected/7f92038f-ab2b-4ce3-86ab-fa83e4d889a9-kube-api-access-5nk4m\") pod \"collect-profiles-29412795-zq67h\" (UID: \"7f92038f-ab2b-4ce3-86ab-fa83e4d889a9\") " pod="openshift-operator-lifecycle-manager/collect-profiles-29412795-zq67h" Dec 03 13:15:00 crc kubenswrapper[5052]: I1203 13:15:00.365091 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/7f92038f-ab2b-4ce3-86ab-fa83e4d889a9-config-volume\") pod \"collect-profiles-29412795-zq67h\" (UID: \"7f92038f-ab2b-4ce3-86ab-fa83e4d889a9\") " pod="openshift-operator-lifecycle-manager/collect-profiles-29412795-zq67h" Dec 03 13:15:00 crc kubenswrapper[5052]: I1203 13:15:00.371256 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"secret-volume\" (UniqueName: \"kubernetes.io/secret/7f92038f-ab2b-4ce3-86ab-fa83e4d889a9-secret-volume\") pod \"collect-profiles-29412795-zq67h\" (UID: \"7f92038f-ab2b-4ce3-86ab-fa83e4d889a9\") " pod="openshift-operator-lifecycle-manager/collect-profiles-29412795-zq67h" Dec 03 13:15:00 crc kubenswrapper[5052]: I1203 13:15:00.383366 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-5nk4m\" (UniqueName: \"kubernetes.io/projected/7f92038f-ab2b-4ce3-86ab-fa83e4d889a9-kube-api-access-5nk4m\") pod \"collect-profiles-29412795-zq67h\" (UID: \"7f92038f-ab2b-4ce3-86ab-fa83e4d889a9\") " pod="openshift-operator-lifecycle-manager/collect-profiles-29412795-zq67h" Dec 03 13:15:00 crc kubenswrapper[5052]: I1203 13:15:00.477782 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-operator-lifecycle-manager/collect-profiles-29412795-zq67h" Dec 03 13:15:03 crc kubenswrapper[5052]: I1203 13:15:03.463067 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-operator-lifecycle-manager/collect-profiles-29412795-zq67h"] Dec 03 13:15:03 crc kubenswrapper[5052]: I1203 13:15:03.585469 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="barbican-kuttl-tests/memcached-0" event={"ID":"6ccef89f-58bd-4e50-9d8c-d5f076df3505","Type":"ContainerStarted","Data":"c321459e91c2cd07d0f5294f7e8373b33fa3a1bebfd2f83df4b9a87fc3058238"} Dec 03 13:15:03 crc kubenswrapper[5052]: I1203 13:15:03.585916 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="barbican-kuttl-tests/memcached-0" Dec 03 13:15:03 crc kubenswrapper[5052]: I1203 13:15:03.587262 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-operator-lifecycle-manager/collect-profiles-29412795-zq67h" event={"ID":"7f92038f-ab2b-4ce3-86ab-fa83e4d889a9","Type":"ContainerStarted","Data":"dd7d7e550b9a8fc4f4694533de0c8ae75a083a4a622c910bb8e5b026f2db9adb"} Dec 03 13:15:03 crc kubenswrapper[5052]: I1203 13:15:03.611749 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="barbican-kuttl-tests/memcached-0" podStartSLOduration=1.631357197 podStartE2EDuration="8.611717018s" podCreationTimestamp="2025-12-03 13:14:55 +0000 UTC" firstStartedPulling="2025-12-03 13:14:56.194420394 +0000 UTC m=+6886.782662168" lastFinishedPulling="2025-12-03 13:15:03.174780225 +0000 UTC m=+6893.763021989" observedRunningTime="2025-12-03 13:15:03.606590741 +0000 UTC m=+6894.194832505" watchObservedRunningTime="2025-12-03 13:15:03.611717018 +0000 UTC m=+6894.199958792" Dec 03 13:15:04 crc kubenswrapper[5052]: I1203 13:15:04.597235 5052 generic.go:334] "Generic (PLEG): container finished" podID="7f92038f-ab2b-4ce3-86ab-fa83e4d889a9" containerID="b1827ad080ae04322bd6bcc322215be51cca6cfa3f880d717657cd150aa81838" exitCode=0 Dec 03 13:15:04 crc kubenswrapper[5052]: I1203 13:15:04.597303 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-operator-lifecycle-manager/collect-profiles-29412795-zq67h" event={"ID":"7f92038f-ab2b-4ce3-86ab-fa83e4d889a9","Type":"ContainerDied","Data":"b1827ad080ae04322bd6bcc322215be51cca6cfa3f880d717657cd150aa81838"} Dec 03 13:15:05 crc kubenswrapper[5052]: I1203 13:15:05.890903 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-operator-lifecycle-manager/collect-profiles-29412795-zq67h" Dec 03 13:15:05 crc kubenswrapper[5052]: I1203 13:15:05.956422 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"secret-volume\" (UniqueName: \"kubernetes.io/secret/7f92038f-ab2b-4ce3-86ab-fa83e4d889a9-secret-volume\") pod \"7f92038f-ab2b-4ce3-86ab-fa83e4d889a9\" (UID: \"7f92038f-ab2b-4ce3-86ab-fa83e4d889a9\") " Dec 03 13:15:05 crc kubenswrapper[5052]: I1203 13:15:05.956861 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5nk4m\" (UniqueName: \"kubernetes.io/projected/7f92038f-ab2b-4ce3-86ab-fa83e4d889a9-kube-api-access-5nk4m\") pod \"7f92038f-ab2b-4ce3-86ab-fa83e4d889a9\" (UID: \"7f92038f-ab2b-4ce3-86ab-fa83e4d889a9\") " Dec 03 13:15:05 crc kubenswrapper[5052]: I1203 13:15:05.957121 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/7f92038f-ab2b-4ce3-86ab-fa83e4d889a9-config-volume\") pod \"7f92038f-ab2b-4ce3-86ab-fa83e4d889a9\" (UID: \"7f92038f-ab2b-4ce3-86ab-fa83e4d889a9\") " Dec 03 13:15:05 crc kubenswrapper[5052]: I1203 13:15:05.958231 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7f92038f-ab2b-4ce3-86ab-fa83e4d889a9-config-volume" (OuterVolumeSpecName: "config-volume") pod "7f92038f-ab2b-4ce3-86ab-fa83e4d889a9" (UID: "7f92038f-ab2b-4ce3-86ab-fa83e4d889a9"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:15:05 crc kubenswrapper[5052]: I1203 13:15:05.964787 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7f92038f-ab2b-4ce3-86ab-fa83e4d889a9-secret-volume" (OuterVolumeSpecName: "secret-volume") pod "7f92038f-ab2b-4ce3-86ab-fa83e4d889a9" (UID: "7f92038f-ab2b-4ce3-86ab-fa83e4d889a9"). InnerVolumeSpecName "secret-volume". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:15:05 crc kubenswrapper[5052]: I1203 13:15:05.964880 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7f92038f-ab2b-4ce3-86ab-fa83e4d889a9-kube-api-access-5nk4m" (OuterVolumeSpecName: "kube-api-access-5nk4m") pod "7f92038f-ab2b-4ce3-86ab-fa83e4d889a9" (UID: "7f92038f-ab2b-4ce3-86ab-fa83e4d889a9"). InnerVolumeSpecName "kube-api-access-5nk4m". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:15:06 crc kubenswrapper[5052]: I1203 13:15:06.059703 5052 reconciler_common.go:293] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/7f92038f-ab2b-4ce3-86ab-fa83e4d889a9-config-volume\") on node \"crc\" DevicePath \"\"" Dec 03 13:15:06 crc kubenswrapper[5052]: I1203 13:15:06.059748 5052 reconciler_common.go:293] "Volume detached for volume \"secret-volume\" (UniqueName: \"kubernetes.io/secret/7f92038f-ab2b-4ce3-86ab-fa83e4d889a9-secret-volume\") on node \"crc\" DevicePath \"\"" Dec 03 13:15:06 crc kubenswrapper[5052]: I1203 13:15:06.059760 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-5nk4m\" (UniqueName: \"kubernetes.io/projected/7f92038f-ab2b-4ce3-86ab-fa83e4d889a9-kube-api-access-5nk4m\") on node \"crc\" DevicePath \"\"" Dec 03 13:15:06 crc kubenswrapper[5052]: I1203 13:15:06.612504 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-operator-lifecycle-manager/collect-profiles-29412795-zq67h" event={"ID":"7f92038f-ab2b-4ce3-86ab-fa83e4d889a9","Type":"ContainerDied","Data":"dd7d7e550b9a8fc4f4694533de0c8ae75a083a4a622c910bb8e5b026f2db9adb"} Dec 03 13:15:06 crc kubenswrapper[5052]: I1203 13:15:06.612571 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="dd7d7e550b9a8fc4f4694533de0c8ae75a083a4a622c910bb8e5b026f2db9adb" Dec 03 13:15:06 crc kubenswrapper[5052]: I1203 13:15:06.612570 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-operator-lifecycle-manager/collect-profiles-29412795-zq67h" Dec 03 13:15:06 crc kubenswrapper[5052]: I1203 13:15:06.971036 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openshift-operator-lifecycle-manager/collect-profiles-29412750-42nq6"] Dec 03 13:15:06 crc kubenswrapper[5052]: I1203 13:15:06.977572 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openshift-operator-lifecycle-manager/collect-profiles-29412750-42nq6"] Dec 03 13:15:07 crc kubenswrapper[5052]: I1203 13:15:07.683217 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6a77ebe2-ac70-4620-acef-bda957db5766" path="/var/lib/kubelet/pods/6a77ebe2-ac70-4620-acef-bda957db5766/volumes" Dec 03 13:15:09 crc kubenswrapper[5052]: I1203 13:15:09.713580 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-operators/9704761d240e56fb98655ffd81084895b33a73ec711f4dcdef0450e590xlm56"] Dec 03 13:15:09 crc kubenswrapper[5052]: E1203 13:15:09.714203 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="7f92038f-ab2b-4ce3-86ab-fa83e4d889a9" containerName="collect-profiles" Dec 03 13:15:09 crc kubenswrapper[5052]: I1203 13:15:09.714225 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f92038f-ab2b-4ce3-86ab-fa83e4d889a9" containerName="collect-profiles" Dec 03 13:15:09 crc kubenswrapper[5052]: I1203 13:15:09.714420 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="7f92038f-ab2b-4ce3-86ab-fa83e4d889a9" containerName="collect-profiles" Dec 03 13:15:09 crc kubenswrapper[5052]: I1203 13:15:09.715479 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-operators/9704761d240e56fb98655ffd81084895b33a73ec711f4dcdef0450e590xlm56" Dec 03 13:15:09 crc kubenswrapper[5052]: I1203 13:15:09.721826 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-operators"/"default-dockercfg-7fklr" Dec 03 13:15:09 crc kubenswrapper[5052]: I1203 13:15:09.723068 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-operators/9704761d240e56fb98655ffd81084895b33a73ec711f4dcdef0450e590xlm56"] Dec 03 13:15:09 crc kubenswrapper[5052]: I1203 13:15:09.822237 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5701f773-f548-4133-867d-dca86f9bed69-util\") pod \"9704761d240e56fb98655ffd81084895b33a73ec711f4dcdef0450e590xlm56\" (UID: \"5701f773-f548-4133-867d-dca86f9bed69\") " pod="openstack-operators/9704761d240e56fb98655ffd81084895b33a73ec711f4dcdef0450e590xlm56" Dec 03 13:15:09 crc kubenswrapper[5052]: I1203 13:15:09.822337 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2svtd\" (UniqueName: \"kubernetes.io/projected/5701f773-f548-4133-867d-dca86f9bed69-kube-api-access-2svtd\") pod \"9704761d240e56fb98655ffd81084895b33a73ec711f4dcdef0450e590xlm56\" (UID: \"5701f773-f548-4133-867d-dca86f9bed69\") " pod="openstack-operators/9704761d240e56fb98655ffd81084895b33a73ec711f4dcdef0450e590xlm56" Dec 03 13:15:09 crc kubenswrapper[5052]: I1203 13:15:09.822553 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5701f773-f548-4133-867d-dca86f9bed69-bundle\") pod \"9704761d240e56fb98655ffd81084895b33a73ec711f4dcdef0450e590xlm56\" (UID: \"5701f773-f548-4133-867d-dca86f9bed69\") " pod="openstack-operators/9704761d240e56fb98655ffd81084895b33a73ec711f4dcdef0450e590xlm56" Dec 03 13:15:09 crc kubenswrapper[5052]: I1203 13:15:09.923703 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5701f773-f548-4133-867d-dca86f9bed69-bundle\") pod \"9704761d240e56fb98655ffd81084895b33a73ec711f4dcdef0450e590xlm56\" (UID: \"5701f773-f548-4133-867d-dca86f9bed69\") " pod="openstack-operators/9704761d240e56fb98655ffd81084895b33a73ec711f4dcdef0450e590xlm56" Dec 03 13:15:09 crc kubenswrapper[5052]: I1203 13:15:09.923767 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5701f773-f548-4133-867d-dca86f9bed69-util\") pod \"9704761d240e56fb98655ffd81084895b33a73ec711f4dcdef0450e590xlm56\" (UID: \"5701f773-f548-4133-867d-dca86f9bed69\") " pod="openstack-operators/9704761d240e56fb98655ffd81084895b33a73ec711f4dcdef0450e590xlm56" Dec 03 13:15:09 crc kubenswrapper[5052]: I1203 13:15:09.923812 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-2svtd\" (UniqueName: \"kubernetes.io/projected/5701f773-f548-4133-867d-dca86f9bed69-kube-api-access-2svtd\") pod \"9704761d240e56fb98655ffd81084895b33a73ec711f4dcdef0450e590xlm56\" (UID: \"5701f773-f548-4133-867d-dca86f9bed69\") " pod="openstack-operators/9704761d240e56fb98655ffd81084895b33a73ec711f4dcdef0450e590xlm56" Dec 03 13:15:09 crc kubenswrapper[5052]: I1203 13:15:09.924788 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5701f773-f548-4133-867d-dca86f9bed69-bundle\") pod \"9704761d240e56fb98655ffd81084895b33a73ec711f4dcdef0450e590xlm56\" (UID: \"5701f773-f548-4133-867d-dca86f9bed69\") " pod="openstack-operators/9704761d240e56fb98655ffd81084895b33a73ec711f4dcdef0450e590xlm56" Dec 03 13:15:09 crc kubenswrapper[5052]: I1203 13:15:09.924864 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5701f773-f548-4133-867d-dca86f9bed69-util\") pod \"9704761d240e56fb98655ffd81084895b33a73ec711f4dcdef0450e590xlm56\" (UID: \"5701f773-f548-4133-867d-dca86f9bed69\") " pod="openstack-operators/9704761d240e56fb98655ffd81084895b33a73ec711f4dcdef0450e590xlm56" Dec 03 13:15:09 crc kubenswrapper[5052]: I1203 13:15:09.952239 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-2svtd\" (UniqueName: \"kubernetes.io/projected/5701f773-f548-4133-867d-dca86f9bed69-kube-api-access-2svtd\") pod \"9704761d240e56fb98655ffd81084895b33a73ec711f4dcdef0450e590xlm56\" (UID: \"5701f773-f548-4133-867d-dca86f9bed69\") " pod="openstack-operators/9704761d240e56fb98655ffd81084895b33a73ec711f4dcdef0450e590xlm56" Dec 03 13:15:10 crc kubenswrapper[5052]: I1203 13:15:10.033486 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-operators/9704761d240e56fb98655ffd81084895b33a73ec711f4dcdef0450e590xlm56" Dec 03 13:15:11 crc kubenswrapper[5052]: I1203 13:15:10.743360 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="barbican-kuttl-tests/memcached-0" Dec 03 13:15:11 crc kubenswrapper[5052]: I1203 13:15:11.477224 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-operators/9704761d240e56fb98655ffd81084895b33a73ec711f4dcdef0450e590xlm56"] Dec 03 13:15:11 crc kubenswrapper[5052]: I1203 13:15:11.654580 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/9704761d240e56fb98655ffd81084895b33a73ec711f4dcdef0450e590xlm56" event={"ID":"5701f773-f548-4133-867d-dca86f9bed69","Type":"ContainerStarted","Data":"9d1a5effba76c4d0977a6218732b082145188053dcb92d3cfae1b59f60bf745d"} Dec 03 13:15:12 crc kubenswrapper[5052]: I1203 13:15:12.663170 5052 generic.go:334] "Generic (PLEG): container finished" podID="5701f773-f548-4133-867d-dca86f9bed69" containerID="a42119ef503a1dddabc103a0657953fdebd8d75d56eba979880480d403dd5012" exitCode=0 Dec 03 13:15:12 crc kubenswrapper[5052]: I1203 13:15:12.663238 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/9704761d240e56fb98655ffd81084895b33a73ec711f4dcdef0450e590xlm56" event={"ID":"5701f773-f548-4133-867d-dca86f9bed69","Type":"ContainerDied","Data":"a42119ef503a1dddabc103a0657953fdebd8d75d56eba979880480d403dd5012"} Dec 03 13:15:14 crc kubenswrapper[5052]: I1203 13:15:14.678760 5052 generic.go:334] "Generic (PLEG): container finished" podID="5701f773-f548-4133-867d-dca86f9bed69" containerID="bc72d7795dfc75a429a7be5f27349a239f919e01278ceec74d94c753a4d344bf" exitCode=0 Dec 03 13:15:14 crc kubenswrapper[5052]: I1203 13:15:14.678854 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/9704761d240e56fb98655ffd81084895b33a73ec711f4dcdef0450e590xlm56" event={"ID":"5701f773-f548-4133-867d-dca86f9bed69","Type":"ContainerDied","Data":"bc72d7795dfc75a429a7be5f27349a239f919e01278ceec74d94c753a4d344bf"} Dec 03 13:15:15 crc kubenswrapper[5052]: I1203 13:15:15.688562 5052 generic.go:334] "Generic (PLEG): container finished" podID="5701f773-f548-4133-867d-dca86f9bed69" containerID="6d2fbf27b742c9ad335064ab874e1e65c7ad76481322449d44ccbd07b89f6085" exitCode=0 Dec 03 13:15:15 crc kubenswrapper[5052]: I1203 13:15:15.688635 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/9704761d240e56fb98655ffd81084895b33a73ec711f4dcdef0450e590xlm56" event={"ID":"5701f773-f548-4133-867d-dca86f9bed69","Type":"ContainerDied","Data":"6d2fbf27b742c9ad335064ab874e1e65c7ad76481322449d44ccbd07b89f6085"} Dec 03 13:15:17 crc kubenswrapper[5052]: I1203 13:15:17.002958 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-operators/9704761d240e56fb98655ffd81084895b33a73ec711f4dcdef0450e590xlm56" Dec 03 13:15:17 crc kubenswrapper[5052]: I1203 13:15:17.133610 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5701f773-f548-4133-867d-dca86f9bed69-util\") pod \"5701f773-f548-4133-867d-dca86f9bed69\" (UID: \"5701f773-f548-4133-867d-dca86f9bed69\") " Dec 03 13:15:17 crc kubenswrapper[5052]: I1203 13:15:17.133661 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5701f773-f548-4133-867d-dca86f9bed69-bundle\") pod \"5701f773-f548-4133-867d-dca86f9bed69\" (UID: \"5701f773-f548-4133-867d-dca86f9bed69\") " Dec 03 13:15:17 crc kubenswrapper[5052]: I1203 13:15:17.133777 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2svtd\" (UniqueName: \"kubernetes.io/projected/5701f773-f548-4133-867d-dca86f9bed69-kube-api-access-2svtd\") pod \"5701f773-f548-4133-867d-dca86f9bed69\" (UID: \"5701f773-f548-4133-867d-dca86f9bed69\") " Dec 03 13:15:17 crc kubenswrapper[5052]: I1203 13:15:17.134971 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5701f773-f548-4133-867d-dca86f9bed69-bundle" (OuterVolumeSpecName: "bundle") pod "5701f773-f548-4133-867d-dca86f9bed69" (UID: "5701f773-f548-4133-867d-dca86f9bed69"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 13:15:17 crc kubenswrapper[5052]: I1203 13:15:17.139556 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5701f773-f548-4133-867d-dca86f9bed69-kube-api-access-2svtd" (OuterVolumeSpecName: "kube-api-access-2svtd") pod "5701f773-f548-4133-867d-dca86f9bed69" (UID: "5701f773-f548-4133-867d-dca86f9bed69"). InnerVolumeSpecName "kube-api-access-2svtd". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:15:17 crc kubenswrapper[5052]: I1203 13:15:17.235123 5052 reconciler_common.go:293] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5701f773-f548-4133-867d-dca86f9bed69-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 13:15:17 crc kubenswrapper[5052]: I1203 13:15:17.235164 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-2svtd\" (UniqueName: \"kubernetes.io/projected/5701f773-f548-4133-867d-dca86f9bed69-kube-api-access-2svtd\") on node \"crc\" DevicePath \"\"" Dec 03 13:15:17 crc kubenswrapper[5052]: I1203 13:15:17.309606 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5701f773-f548-4133-867d-dca86f9bed69-util" (OuterVolumeSpecName: "util") pod "5701f773-f548-4133-867d-dca86f9bed69" (UID: "5701f773-f548-4133-867d-dca86f9bed69"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 13:15:17 crc kubenswrapper[5052]: I1203 13:15:17.336077 5052 reconciler_common.go:293] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5701f773-f548-4133-867d-dca86f9bed69-util\") on node \"crc\" DevicePath \"\"" Dec 03 13:15:17 crc kubenswrapper[5052]: I1203 13:15:17.703098 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/9704761d240e56fb98655ffd81084895b33a73ec711f4dcdef0450e590xlm56" event={"ID":"5701f773-f548-4133-867d-dca86f9bed69","Type":"ContainerDied","Data":"9d1a5effba76c4d0977a6218732b082145188053dcb92d3cfae1b59f60bf745d"} Dec 03 13:15:17 crc kubenswrapper[5052]: I1203 13:15:17.703139 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9d1a5effba76c4d0977a6218732b082145188053dcb92d3cfae1b59f60bf745d" Dec 03 13:15:17 crc kubenswrapper[5052]: I1203 13:15:17.703143 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-operators/9704761d240e56fb98655ffd81084895b33a73ec711f4dcdef0450e590xlm56" Dec 03 13:15:24 crc kubenswrapper[5052]: I1203 13:15:24.296560 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-operators/rabbitmq-cluster-operator-779fc9694b-2vdrl"] Dec 03 13:15:24 crc kubenswrapper[5052]: E1203 13:15:24.297462 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="5701f773-f548-4133-867d-dca86f9bed69" containerName="extract" Dec 03 13:15:24 crc kubenswrapper[5052]: I1203 13:15:24.297476 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="5701f773-f548-4133-867d-dca86f9bed69" containerName="extract" Dec 03 13:15:24 crc kubenswrapper[5052]: E1203 13:15:24.297497 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="5701f773-f548-4133-867d-dca86f9bed69" containerName="util" Dec 03 13:15:24 crc kubenswrapper[5052]: I1203 13:15:24.297523 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="5701f773-f548-4133-867d-dca86f9bed69" containerName="util" Dec 03 13:15:24 crc kubenswrapper[5052]: E1203 13:15:24.297556 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="5701f773-f548-4133-867d-dca86f9bed69" containerName="pull" Dec 03 13:15:24 crc kubenswrapper[5052]: I1203 13:15:24.297564 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="5701f773-f548-4133-867d-dca86f9bed69" containerName="pull" Dec 03 13:15:24 crc kubenswrapper[5052]: I1203 13:15:24.297725 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="5701f773-f548-4133-867d-dca86f9bed69" containerName="extract" Dec 03 13:15:24 crc kubenswrapper[5052]: I1203 13:15:24.298300 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-operators/rabbitmq-cluster-operator-779fc9694b-2vdrl" Dec 03 13:15:24 crc kubenswrapper[5052]: I1203 13:15:24.304475 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-operators"/"rabbitmq-cluster-operator-dockercfg-f8ldt" Dec 03 13:15:24 crc kubenswrapper[5052]: I1203 13:15:24.353991 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-operators/rabbitmq-cluster-operator-779fc9694b-2vdrl"] Dec 03 13:15:24 crc kubenswrapper[5052]: I1203 13:15:24.454035 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-569g9\" (UniqueName: \"kubernetes.io/projected/9bef396c-7ed5-46b5-b57d-f4cd7df5c872-kube-api-access-569g9\") pod \"rabbitmq-cluster-operator-779fc9694b-2vdrl\" (UID: \"9bef396c-7ed5-46b5-b57d-f4cd7df5c872\") " pod="openstack-operators/rabbitmq-cluster-operator-779fc9694b-2vdrl" Dec 03 13:15:24 crc kubenswrapper[5052]: I1203 13:15:24.555822 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-569g9\" (UniqueName: \"kubernetes.io/projected/9bef396c-7ed5-46b5-b57d-f4cd7df5c872-kube-api-access-569g9\") pod \"rabbitmq-cluster-operator-779fc9694b-2vdrl\" (UID: \"9bef396c-7ed5-46b5-b57d-f4cd7df5c872\") " pod="openstack-operators/rabbitmq-cluster-operator-779fc9694b-2vdrl" Dec 03 13:15:24 crc kubenswrapper[5052]: I1203 13:15:24.574221 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-569g9\" (UniqueName: \"kubernetes.io/projected/9bef396c-7ed5-46b5-b57d-f4cd7df5c872-kube-api-access-569g9\") pod \"rabbitmq-cluster-operator-779fc9694b-2vdrl\" (UID: \"9bef396c-7ed5-46b5-b57d-f4cd7df5c872\") " pod="openstack-operators/rabbitmq-cluster-operator-779fc9694b-2vdrl" Dec 03 13:15:24 crc kubenswrapper[5052]: I1203 13:15:24.618402 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-operators/rabbitmq-cluster-operator-779fc9694b-2vdrl" Dec 03 13:15:25 crc kubenswrapper[5052]: I1203 13:15:25.075459 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-operators/rabbitmq-cluster-operator-779fc9694b-2vdrl"] Dec 03 13:15:25 crc kubenswrapper[5052]: I1203 13:15:25.781383 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/rabbitmq-cluster-operator-779fc9694b-2vdrl" event={"ID":"9bef396c-7ed5-46b5-b57d-f4cd7df5c872","Type":"ContainerStarted","Data":"44620fbcf5c8209527c08cfab1c00bbdbef1fc4ba9bc3c160aecbaf8ddce18fb"} Dec 03 13:15:27 crc kubenswrapper[5052]: I1203 13:15:27.795946 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/rabbitmq-cluster-operator-779fc9694b-2vdrl" event={"ID":"9bef396c-7ed5-46b5-b57d-f4cd7df5c872","Type":"ContainerStarted","Data":"8a0e209c01fd705fa336c0ddc5fe285ff3876ea628c51c4d228852bb47da3169"} Dec 03 13:15:27 crc kubenswrapper[5052]: I1203 13:15:27.810845 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-operators/rabbitmq-cluster-operator-779fc9694b-2vdrl" podStartSLOduration=3.810824534 podStartE2EDuration="3.810824534s" podCreationTimestamp="2025-12-03 13:15:24 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 13:15:27.809384852 +0000 UTC m=+6918.397626616" watchObservedRunningTime="2025-12-03 13:15:27.810824534 +0000 UTC m=+6918.399066298" Dec 03 13:15:29 crc kubenswrapper[5052]: I1203 13:15:29.379337 5052 patch_prober.go:28] interesting pod/machine-config-daemon-74bj6 container/machine-config-daemon namespace/openshift-machine-config-operator: Liveness probe status=failure output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" start-of-body= Dec 03 13:15:29 crc kubenswrapper[5052]: I1203 13:15:29.379403 5052 prober.go:107] "Probe failed" probeType="Liveness" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" containerName="machine-config-daemon" probeResult="failure" output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" Dec 03 13:15:29 crc kubenswrapper[5052]: I1203 13:15:29.379455 5052 kubelet.go:2542] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" Dec 03 13:15:29 crc kubenswrapper[5052]: I1203 13:15:29.380167 5052 kuberuntime_manager.go:1027] "Message for Container of pod" containerName="machine-config-daemon" containerStatusID={"Type":"cri-o","ID":"059812316dc85d1da8d022c96bdd889e4f16ae597afc7241b51949522378de3a"} pod="openshift-machine-config-operator/machine-config-daemon-74bj6" containerMessage="Container machine-config-daemon failed liveness probe, will be restarted" Dec 03 13:15:29 crc kubenswrapper[5052]: I1203 13:15:29.380227 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" containerName="machine-config-daemon" containerID="cri-o://059812316dc85d1da8d022c96bdd889e4f16ae597afc7241b51949522378de3a" gracePeriod=600 Dec 03 13:15:29 crc kubenswrapper[5052]: I1203 13:15:29.811158 5052 generic.go:334] "Generic (PLEG): container finished" podID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" containerID="059812316dc85d1da8d022c96bdd889e4f16ae597afc7241b51949522378de3a" exitCode=0 Dec 03 13:15:29 crc kubenswrapper[5052]: I1203 13:15:29.811198 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" event={"ID":"10c237bc-72e7-4da3-9f2c-b1688395bf6e","Type":"ContainerDied","Data":"059812316dc85d1da8d022c96bdd889e4f16ae597afc7241b51949522378de3a"} Dec 03 13:15:29 crc kubenswrapper[5052]: I1203 13:15:29.811545 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" event={"ID":"10c237bc-72e7-4da3-9f2c-b1688395bf6e","Type":"ContainerStarted","Data":"12a8b1f470f6d59d7700e376f2a48eb36363f91f9823a35ce534062c0974c10c"} Dec 03 13:15:29 crc kubenswrapper[5052]: I1203 13:15:29.811568 5052 scope.go:117] "RemoveContainer" containerID="dbc100c8081ea1a2f5487429cc9d67de6bc8ba11282b64f248c71275375b0e16" Dec 03 13:15:34 crc kubenswrapper[5052]: I1203 13:15:34.065486 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-operators/keystone-operator-index-ltzwq"] Dec 03 13:15:34 crc kubenswrapper[5052]: I1203 13:15:34.066874 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-operators/keystone-operator-index-ltzwq" Dec 03 13:15:34 crc kubenswrapper[5052]: I1203 13:15:34.074023 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-operators"/"keystone-operator-index-dockercfg-6hlpx" Dec 03 13:15:34 crc kubenswrapper[5052]: I1203 13:15:34.078194 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-operators/keystone-operator-index-ltzwq"] Dec 03 13:15:34 crc kubenswrapper[5052]: I1203 13:15:34.138013 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7lndw\" (UniqueName: \"kubernetes.io/projected/291ddb97-2aa4-4e12-9aad-9a8aacd4f66d-kube-api-access-7lndw\") pod \"keystone-operator-index-ltzwq\" (UID: \"291ddb97-2aa4-4e12-9aad-9a8aacd4f66d\") " pod="openstack-operators/keystone-operator-index-ltzwq" Dec 03 13:15:34 crc kubenswrapper[5052]: I1203 13:15:34.239532 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-7lndw\" (UniqueName: \"kubernetes.io/projected/291ddb97-2aa4-4e12-9aad-9a8aacd4f66d-kube-api-access-7lndw\") pod \"keystone-operator-index-ltzwq\" (UID: \"291ddb97-2aa4-4e12-9aad-9a8aacd4f66d\") " pod="openstack-operators/keystone-operator-index-ltzwq" Dec 03 13:15:34 crc kubenswrapper[5052]: I1203 13:15:34.260864 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-7lndw\" (UniqueName: \"kubernetes.io/projected/291ddb97-2aa4-4e12-9aad-9a8aacd4f66d-kube-api-access-7lndw\") pod \"keystone-operator-index-ltzwq\" (UID: \"291ddb97-2aa4-4e12-9aad-9a8aacd4f66d\") " pod="openstack-operators/keystone-operator-index-ltzwq" Dec 03 13:15:34 crc kubenswrapper[5052]: I1203 13:15:34.395407 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-operators/keystone-operator-index-ltzwq" Dec 03 13:15:34 crc kubenswrapper[5052]: I1203 13:15:34.804705 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-operators/keystone-operator-index-ltzwq"] Dec 03 13:15:34 crc kubenswrapper[5052]: I1203 13:15:34.850031 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/keystone-operator-index-ltzwq" event={"ID":"291ddb97-2aa4-4e12-9aad-9a8aacd4f66d","Type":"ContainerStarted","Data":"0d452f83563e6fec7fc666382eaf7effb4c87352e4f64397b393615e63e20660"} Dec 03 13:15:36 crc kubenswrapper[5052]: I1203 13:15:36.865862 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/keystone-operator-index-ltzwq" event={"ID":"291ddb97-2aa4-4e12-9aad-9a8aacd4f66d","Type":"ContainerStarted","Data":"f3129f2cd2ac043dc79c419a0bf1e47c231e44e15e880331cea1f8a8d30c284a"} Dec 03 13:15:36 crc kubenswrapper[5052]: I1203 13:15:36.880196 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-operators/keystone-operator-index-ltzwq" podStartSLOduration=1.9800846920000001 podStartE2EDuration="2.880178438s" podCreationTimestamp="2025-12-03 13:15:34 +0000 UTC" firstStartedPulling="2025-12-03 13:15:34.816254423 +0000 UTC m=+6925.404496187" lastFinishedPulling="2025-12-03 13:15:35.716348179 +0000 UTC m=+6926.304589933" observedRunningTime="2025-12-03 13:15:36.877866652 +0000 UTC m=+6927.466108416" watchObservedRunningTime="2025-12-03 13:15:36.880178438 +0000 UTC m=+6927.468420202" Dec 03 13:15:38 crc kubenswrapper[5052]: I1203 13:15:38.258746 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-operators/keystone-operator-index-ltzwq"] Dec 03 13:15:38 crc kubenswrapper[5052]: I1203 13:15:38.863230 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-operators/keystone-operator-index-ntwzz"] Dec 03 13:15:38 crc kubenswrapper[5052]: I1203 13:15:38.864319 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-operators/keystone-operator-index-ntwzz" Dec 03 13:15:38 crc kubenswrapper[5052]: I1203 13:15:38.870693 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-operators/keystone-operator-index-ntwzz"] Dec 03 13:15:38 crc kubenswrapper[5052]: I1203 13:15:38.880009 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-operators/keystone-operator-index-ltzwq" podUID="291ddb97-2aa4-4e12-9aad-9a8aacd4f66d" containerName="registry-server" containerID="cri-o://f3129f2cd2ac043dc79c419a0bf1e47c231e44e15e880331cea1f8a8d30c284a" gracePeriod=2 Dec 03 13:15:38 crc kubenswrapper[5052]: I1203 13:15:38.907656 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pq87d\" (UniqueName: \"kubernetes.io/projected/a93ce5ec-f168-4024-9cad-d6886d0421cb-kube-api-access-pq87d\") pod \"keystone-operator-index-ntwzz\" (UID: \"a93ce5ec-f168-4024-9cad-d6886d0421cb\") " pod="openstack-operators/keystone-operator-index-ntwzz" Dec 03 13:15:39 crc kubenswrapper[5052]: I1203 13:15:39.010925 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-pq87d\" (UniqueName: \"kubernetes.io/projected/a93ce5ec-f168-4024-9cad-d6886d0421cb-kube-api-access-pq87d\") pod \"keystone-operator-index-ntwzz\" (UID: \"a93ce5ec-f168-4024-9cad-d6886d0421cb\") " pod="openstack-operators/keystone-operator-index-ntwzz" Dec 03 13:15:39 crc kubenswrapper[5052]: I1203 13:15:39.032262 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-pq87d\" (UniqueName: \"kubernetes.io/projected/a93ce5ec-f168-4024-9cad-d6886d0421cb-kube-api-access-pq87d\") pod \"keystone-operator-index-ntwzz\" (UID: \"a93ce5ec-f168-4024-9cad-d6886d0421cb\") " pod="openstack-operators/keystone-operator-index-ntwzz" Dec 03 13:15:39 crc kubenswrapper[5052]: I1203 13:15:39.186801 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-operators/keystone-operator-index-ntwzz" Dec 03 13:15:39 crc kubenswrapper[5052]: I1203 13:15:39.277220 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-operators/keystone-operator-index-ltzwq" Dec 03 13:15:39 crc kubenswrapper[5052]: I1203 13:15:39.441106 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7lndw\" (UniqueName: \"kubernetes.io/projected/291ddb97-2aa4-4e12-9aad-9a8aacd4f66d-kube-api-access-7lndw\") pod \"291ddb97-2aa4-4e12-9aad-9a8aacd4f66d\" (UID: \"291ddb97-2aa4-4e12-9aad-9a8aacd4f66d\") " Dec 03 13:15:39 crc kubenswrapper[5052]: I1203 13:15:39.467152 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/291ddb97-2aa4-4e12-9aad-9a8aacd4f66d-kube-api-access-7lndw" (OuterVolumeSpecName: "kube-api-access-7lndw") pod "291ddb97-2aa4-4e12-9aad-9a8aacd4f66d" (UID: "291ddb97-2aa4-4e12-9aad-9a8aacd4f66d"). InnerVolumeSpecName "kube-api-access-7lndw". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:15:39 crc kubenswrapper[5052]: I1203 13:15:39.543017 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-7lndw\" (UniqueName: \"kubernetes.io/projected/291ddb97-2aa4-4e12-9aad-9a8aacd4f66d-kube-api-access-7lndw\") on node \"crc\" DevicePath \"\"" Dec 03 13:15:39 crc kubenswrapper[5052]: W1203 13:15:39.680847 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda93ce5ec_f168_4024_9cad_d6886d0421cb.slice/crio-3528dd53de4f3b1651bfc571314f2648d37776f662e925ca6f7b537e9fa1d6a0 WatchSource:0}: Error finding container 3528dd53de4f3b1651bfc571314f2648d37776f662e925ca6f7b537e9fa1d6a0: Status 404 returned error can't find the container with id 3528dd53de4f3b1651bfc571314f2648d37776f662e925ca6f7b537e9fa1d6a0 Dec 03 13:15:39 crc kubenswrapper[5052]: I1203 13:15:39.684918 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-operators/keystone-operator-index-ntwzz"] Dec 03 13:15:39 crc kubenswrapper[5052]: I1203 13:15:39.902428 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/keystone-operator-index-ntwzz" event={"ID":"a93ce5ec-f168-4024-9cad-d6886d0421cb","Type":"ContainerStarted","Data":"3528dd53de4f3b1651bfc571314f2648d37776f662e925ca6f7b537e9fa1d6a0"} Dec 03 13:15:39 crc kubenswrapper[5052]: I1203 13:15:39.903864 5052 generic.go:334] "Generic (PLEG): container finished" podID="291ddb97-2aa4-4e12-9aad-9a8aacd4f66d" containerID="f3129f2cd2ac043dc79c419a0bf1e47c231e44e15e880331cea1f8a8d30c284a" exitCode=0 Dec 03 13:15:39 crc kubenswrapper[5052]: I1203 13:15:39.903893 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/keystone-operator-index-ltzwq" event={"ID":"291ddb97-2aa4-4e12-9aad-9a8aacd4f66d","Type":"ContainerDied","Data":"f3129f2cd2ac043dc79c419a0bf1e47c231e44e15e880331cea1f8a8d30c284a"} Dec 03 13:15:39 crc kubenswrapper[5052]: I1203 13:15:39.903938 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/keystone-operator-index-ltzwq" event={"ID":"291ddb97-2aa4-4e12-9aad-9a8aacd4f66d","Type":"ContainerDied","Data":"0d452f83563e6fec7fc666382eaf7effb4c87352e4f64397b393615e63e20660"} Dec 03 13:15:39 crc kubenswrapper[5052]: I1203 13:15:39.903960 5052 scope.go:117] "RemoveContainer" containerID="f3129f2cd2ac043dc79c419a0bf1e47c231e44e15e880331cea1f8a8d30c284a" Dec 03 13:15:39 crc kubenswrapper[5052]: I1203 13:15:39.904102 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-operators/keystone-operator-index-ltzwq" Dec 03 13:15:39 crc kubenswrapper[5052]: I1203 13:15:39.927730 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-operators/keystone-operator-index-ltzwq"] Dec 03 13:15:39 crc kubenswrapper[5052]: I1203 13:15:39.928472 5052 scope.go:117] "RemoveContainer" containerID="f3129f2cd2ac043dc79c419a0bf1e47c231e44e15e880331cea1f8a8d30c284a" Dec 03 13:15:39 crc kubenswrapper[5052]: E1203 13:15:39.929048 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f3129f2cd2ac043dc79c419a0bf1e47c231e44e15e880331cea1f8a8d30c284a\": container with ID starting with f3129f2cd2ac043dc79c419a0bf1e47c231e44e15e880331cea1f8a8d30c284a not found: ID does not exist" containerID="f3129f2cd2ac043dc79c419a0bf1e47c231e44e15e880331cea1f8a8d30c284a" Dec 03 13:15:39 crc kubenswrapper[5052]: I1203 13:15:39.929084 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f3129f2cd2ac043dc79c419a0bf1e47c231e44e15e880331cea1f8a8d30c284a"} err="failed to get container status \"f3129f2cd2ac043dc79c419a0bf1e47c231e44e15e880331cea1f8a8d30c284a\": rpc error: code = NotFound desc = could not find container \"f3129f2cd2ac043dc79c419a0bf1e47c231e44e15e880331cea1f8a8d30c284a\": container with ID starting with f3129f2cd2ac043dc79c419a0bf1e47c231e44e15e880331cea1f8a8d30c284a not found: ID does not exist" Dec 03 13:15:39 crc kubenswrapper[5052]: I1203 13:15:39.933379 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-operators/keystone-operator-index-ltzwq"] Dec 03 13:15:40 crc kubenswrapper[5052]: I1203 13:15:40.913523 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/keystone-operator-index-ntwzz" event={"ID":"a93ce5ec-f168-4024-9cad-d6886d0421cb","Type":"ContainerStarted","Data":"cd6a5b8d53361b1f39a40fa101eac92a94f36bf2c94105882839a39103e9d2b0"} Dec 03 13:15:40 crc kubenswrapper[5052]: I1203 13:15:40.935426 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-operators/keystone-operator-index-ntwzz" podStartSLOduration=2.126402142 podStartE2EDuration="2.93540082s" podCreationTimestamp="2025-12-03 13:15:38 +0000 UTC" firstStartedPulling="2025-12-03 13:15:39.686232976 +0000 UTC m=+6930.274474740" lastFinishedPulling="2025-12-03 13:15:40.495231654 +0000 UTC m=+6931.083473418" observedRunningTime="2025-12-03 13:15:40.928530873 +0000 UTC m=+6931.516772637" watchObservedRunningTime="2025-12-03 13:15:40.93540082 +0000 UTC m=+6931.523642584" Dec 03 13:15:41 crc kubenswrapper[5052]: I1203 13:15:41.680933 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="291ddb97-2aa4-4e12-9aad-9a8aacd4f66d" path="/var/lib/kubelet/pods/291ddb97-2aa4-4e12-9aad-9a8aacd4f66d/volumes" Dec 03 13:15:45 crc kubenswrapper[5052]: I1203 13:15:45.892021 5052 scope.go:117] "RemoveContainer" containerID="9f1d7d94a704f0496350535cc5fb961eee1de686802f6da0f1f72f5c4cbc204b" Dec 03 13:15:45 crc kubenswrapper[5052]: I1203 13:15:45.912582 5052 scope.go:117] "RemoveContainer" containerID="a9e5b3a74a64e4b357cabfb3951a5add7c42865e635bf5406c566aff3b0049b6" Dec 03 13:15:45 crc kubenswrapper[5052]: I1203 13:15:45.947031 5052 scope.go:117] "RemoveContainer" containerID="cf085488899352e2835e041c566a78d8e9c159f55f8ce843fe2a8639160bbb2a" Dec 03 13:15:45 crc kubenswrapper[5052]: I1203 13:15:45.974734 5052 scope.go:117] "RemoveContainer" containerID="fff599f59ff0eda492ccce8b28da608718929bfdbc7bbc4f7db9703ffc681ebb" Dec 03 13:15:46 crc kubenswrapper[5052]: I1203 13:15:46.009178 5052 scope.go:117] "RemoveContainer" containerID="1fe185d294d533560cf7c402a020f267cfea0c63d5f4a4e848fb594451b7c726" Dec 03 13:15:49 crc kubenswrapper[5052]: I1203 13:15:49.186994 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-operators/keystone-operator-index-ntwzz" Dec 03 13:15:49 crc kubenswrapper[5052]: I1203 13:15:49.187581 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-operators/keystone-operator-index-ntwzz" Dec 03 13:15:49 crc kubenswrapper[5052]: I1203 13:15:49.215248 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-operators/keystone-operator-index-ntwzz" Dec 03 13:15:50 crc kubenswrapper[5052]: I1203 13:15:50.011068 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-operators/keystone-operator-index-ntwzz" Dec 03 13:15:51 crc kubenswrapper[5052]: I1203 13:15:51.939490 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-operators/49c083020ae5dfe237b73a6c6b807501660a323f061d7879268c43a121bn4xq"] Dec 03 13:15:51 crc kubenswrapper[5052]: E1203 13:15:51.940904 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="291ddb97-2aa4-4e12-9aad-9a8aacd4f66d" containerName="registry-server" Dec 03 13:15:51 crc kubenswrapper[5052]: I1203 13:15:51.941032 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="291ddb97-2aa4-4e12-9aad-9a8aacd4f66d" containerName="registry-server" Dec 03 13:15:51 crc kubenswrapper[5052]: I1203 13:15:51.941301 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="291ddb97-2aa4-4e12-9aad-9a8aacd4f66d" containerName="registry-server" Dec 03 13:15:51 crc kubenswrapper[5052]: I1203 13:15:51.942460 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-operators/49c083020ae5dfe237b73a6c6b807501660a323f061d7879268c43a121bn4xq" Dec 03 13:15:51 crc kubenswrapper[5052]: I1203 13:15:51.946180 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-operators"/"default-dockercfg-7fklr" Dec 03 13:15:51 crc kubenswrapper[5052]: I1203 13:15:51.978313 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-operators/49c083020ae5dfe237b73a6c6b807501660a323f061d7879268c43a121bn4xq"] Dec 03 13:15:52 crc kubenswrapper[5052]: I1203 13:15:52.139775 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4903ac07-9e1a-41f9-9d62-23ebfb24f0a2-util\") pod \"49c083020ae5dfe237b73a6c6b807501660a323f061d7879268c43a121bn4xq\" (UID: \"4903ac07-9e1a-41f9-9d62-23ebfb24f0a2\") " pod="openstack-operators/49c083020ae5dfe237b73a6c6b807501660a323f061d7879268c43a121bn4xq" Dec 03 13:15:52 crc kubenswrapper[5052]: I1203 13:15:52.139849 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-84ng8\" (UniqueName: \"kubernetes.io/projected/4903ac07-9e1a-41f9-9d62-23ebfb24f0a2-kube-api-access-84ng8\") pod \"49c083020ae5dfe237b73a6c6b807501660a323f061d7879268c43a121bn4xq\" (UID: \"4903ac07-9e1a-41f9-9d62-23ebfb24f0a2\") " pod="openstack-operators/49c083020ae5dfe237b73a6c6b807501660a323f061d7879268c43a121bn4xq" Dec 03 13:15:52 crc kubenswrapper[5052]: I1203 13:15:52.139929 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4903ac07-9e1a-41f9-9d62-23ebfb24f0a2-bundle\") pod \"49c083020ae5dfe237b73a6c6b807501660a323f061d7879268c43a121bn4xq\" (UID: \"4903ac07-9e1a-41f9-9d62-23ebfb24f0a2\") " pod="openstack-operators/49c083020ae5dfe237b73a6c6b807501660a323f061d7879268c43a121bn4xq" Dec 03 13:15:52 crc kubenswrapper[5052]: I1203 13:15:52.241212 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4903ac07-9e1a-41f9-9d62-23ebfb24f0a2-util\") pod \"49c083020ae5dfe237b73a6c6b807501660a323f061d7879268c43a121bn4xq\" (UID: \"4903ac07-9e1a-41f9-9d62-23ebfb24f0a2\") " pod="openstack-operators/49c083020ae5dfe237b73a6c6b807501660a323f061d7879268c43a121bn4xq" Dec 03 13:15:52 crc kubenswrapper[5052]: I1203 13:15:52.241270 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-84ng8\" (UniqueName: \"kubernetes.io/projected/4903ac07-9e1a-41f9-9d62-23ebfb24f0a2-kube-api-access-84ng8\") pod \"49c083020ae5dfe237b73a6c6b807501660a323f061d7879268c43a121bn4xq\" (UID: \"4903ac07-9e1a-41f9-9d62-23ebfb24f0a2\") " pod="openstack-operators/49c083020ae5dfe237b73a6c6b807501660a323f061d7879268c43a121bn4xq" Dec 03 13:15:52 crc kubenswrapper[5052]: I1203 13:15:52.241348 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4903ac07-9e1a-41f9-9d62-23ebfb24f0a2-bundle\") pod \"49c083020ae5dfe237b73a6c6b807501660a323f061d7879268c43a121bn4xq\" (UID: \"4903ac07-9e1a-41f9-9d62-23ebfb24f0a2\") " pod="openstack-operators/49c083020ae5dfe237b73a6c6b807501660a323f061d7879268c43a121bn4xq" Dec 03 13:15:52 crc kubenswrapper[5052]: I1203 13:15:52.241870 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4903ac07-9e1a-41f9-9d62-23ebfb24f0a2-util\") pod \"49c083020ae5dfe237b73a6c6b807501660a323f061d7879268c43a121bn4xq\" (UID: \"4903ac07-9e1a-41f9-9d62-23ebfb24f0a2\") " pod="openstack-operators/49c083020ae5dfe237b73a6c6b807501660a323f061d7879268c43a121bn4xq" Dec 03 13:15:52 crc kubenswrapper[5052]: I1203 13:15:52.241887 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4903ac07-9e1a-41f9-9d62-23ebfb24f0a2-bundle\") pod \"49c083020ae5dfe237b73a6c6b807501660a323f061d7879268c43a121bn4xq\" (UID: \"4903ac07-9e1a-41f9-9d62-23ebfb24f0a2\") " pod="openstack-operators/49c083020ae5dfe237b73a6c6b807501660a323f061d7879268c43a121bn4xq" Dec 03 13:15:52 crc kubenswrapper[5052]: I1203 13:15:52.265094 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-84ng8\" (UniqueName: \"kubernetes.io/projected/4903ac07-9e1a-41f9-9d62-23ebfb24f0a2-kube-api-access-84ng8\") pod \"49c083020ae5dfe237b73a6c6b807501660a323f061d7879268c43a121bn4xq\" (UID: \"4903ac07-9e1a-41f9-9d62-23ebfb24f0a2\") " pod="openstack-operators/49c083020ae5dfe237b73a6c6b807501660a323f061d7879268c43a121bn4xq" Dec 03 13:15:52 crc kubenswrapper[5052]: I1203 13:15:52.561657 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-operators/49c083020ae5dfe237b73a6c6b807501660a323f061d7879268c43a121bn4xq" Dec 03 13:15:52 crc kubenswrapper[5052]: I1203 13:15:52.834017 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-operators/49c083020ae5dfe237b73a6c6b807501660a323f061d7879268c43a121bn4xq"] Dec 03 13:15:53 crc kubenswrapper[5052]: I1203 13:15:53.018556 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/49c083020ae5dfe237b73a6c6b807501660a323f061d7879268c43a121bn4xq" event={"ID":"4903ac07-9e1a-41f9-9d62-23ebfb24f0a2","Type":"ContainerStarted","Data":"50231d2ab95b75afdf907ccf83edd321b011b7620171da6993993bc7ffc713a9"} Dec 03 13:15:54 crc kubenswrapper[5052]: I1203 13:15:54.030089 5052 generic.go:334] "Generic (PLEG): container finished" podID="4903ac07-9e1a-41f9-9d62-23ebfb24f0a2" containerID="ed7869e5c41a6f93a71e760064518e708db2a09a0bfcceea88578d8c7aef5b48" exitCode=0 Dec 03 13:15:54 crc kubenswrapper[5052]: I1203 13:15:54.030144 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/49c083020ae5dfe237b73a6c6b807501660a323f061d7879268c43a121bn4xq" event={"ID":"4903ac07-9e1a-41f9-9d62-23ebfb24f0a2","Type":"ContainerDied","Data":"ed7869e5c41a6f93a71e760064518e708db2a09a0bfcceea88578d8c7aef5b48"} Dec 03 13:15:54 crc kubenswrapper[5052]: I1203 13:15:54.470933 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openshift-marketplace/community-operators-cs7vl"] Dec 03 13:15:54 crc kubenswrapper[5052]: I1203 13:15:54.472541 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/community-operators-cs7vl" Dec 03 13:15:54 crc kubenswrapper[5052]: I1203 13:15:54.484098 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9t2bd\" (UniqueName: \"kubernetes.io/projected/f7eebd92-c92d-4184-a171-15d7106fbc1e-kube-api-access-9t2bd\") pod \"community-operators-cs7vl\" (UID: \"f7eebd92-c92d-4184-a171-15d7106fbc1e\") " pod="openshift-marketplace/community-operators-cs7vl" Dec 03 13:15:54 crc kubenswrapper[5052]: I1203 13:15:54.484146 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/f7eebd92-c92d-4184-a171-15d7106fbc1e-catalog-content\") pod \"community-operators-cs7vl\" (UID: \"f7eebd92-c92d-4184-a171-15d7106fbc1e\") " pod="openshift-marketplace/community-operators-cs7vl" Dec 03 13:15:54 crc kubenswrapper[5052]: I1203 13:15:54.484177 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/f7eebd92-c92d-4184-a171-15d7106fbc1e-utilities\") pod \"community-operators-cs7vl\" (UID: \"f7eebd92-c92d-4184-a171-15d7106fbc1e\") " pod="openshift-marketplace/community-operators-cs7vl" Dec 03 13:15:54 crc kubenswrapper[5052]: I1203 13:15:54.489401 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/community-operators-cs7vl"] Dec 03 13:15:54 crc kubenswrapper[5052]: I1203 13:15:54.585358 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-9t2bd\" (UniqueName: \"kubernetes.io/projected/f7eebd92-c92d-4184-a171-15d7106fbc1e-kube-api-access-9t2bd\") pod \"community-operators-cs7vl\" (UID: \"f7eebd92-c92d-4184-a171-15d7106fbc1e\") " pod="openshift-marketplace/community-operators-cs7vl" Dec 03 13:15:54 crc kubenswrapper[5052]: I1203 13:15:54.585415 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/f7eebd92-c92d-4184-a171-15d7106fbc1e-catalog-content\") pod \"community-operators-cs7vl\" (UID: \"f7eebd92-c92d-4184-a171-15d7106fbc1e\") " pod="openshift-marketplace/community-operators-cs7vl" Dec 03 13:15:54 crc kubenswrapper[5052]: I1203 13:15:54.585458 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/f7eebd92-c92d-4184-a171-15d7106fbc1e-utilities\") pod \"community-operators-cs7vl\" (UID: \"f7eebd92-c92d-4184-a171-15d7106fbc1e\") " pod="openshift-marketplace/community-operators-cs7vl" Dec 03 13:15:54 crc kubenswrapper[5052]: I1203 13:15:54.586204 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/f7eebd92-c92d-4184-a171-15d7106fbc1e-catalog-content\") pod \"community-operators-cs7vl\" (UID: \"f7eebd92-c92d-4184-a171-15d7106fbc1e\") " pod="openshift-marketplace/community-operators-cs7vl" Dec 03 13:15:54 crc kubenswrapper[5052]: I1203 13:15:54.586246 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/f7eebd92-c92d-4184-a171-15d7106fbc1e-utilities\") pod \"community-operators-cs7vl\" (UID: \"f7eebd92-c92d-4184-a171-15d7106fbc1e\") " pod="openshift-marketplace/community-operators-cs7vl" Dec 03 13:15:54 crc kubenswrapper[5052]: I1203 13:15:54.623593 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-9t2bd\" (UniqueName: \"kubernetes.io/projected/f7eebd92-c92d-4184-a171-15d7106fbc1e-kube-api-access-9t2bd\") pod \"community-operators-cs7vl\" (UID: \"f7eebd92-c92d-4184-a171-15d7106fbc1e\") " pod="openshift-marketplace/community-operators-cs7vl" Dec 03 13:15:54 crc kubenswrapper[5052]: I1203 13:15:54.888797 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/community-operators-cs7vl" Dec 03 13:15:55 crc kubenswrapper[5052]: I1203 13:15:55.203602 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/community-operators-cs7vl"] Dec 03 13:15:56 crc kubenswrapper[5052]: I1203 13:15:56.052875 5052 generic.go:334] "Generic (PLEG): container finished" podID="4903ac07-9e1a-41f9-9d62-23ebfb24f0a2" containerID="70b90ec2bd7bf33637c4f77fd83ed552d859433d48e56f89379d0ab0cef6b4f4" exitCode=0 Dec 03 13:15:56 crc kubenswrapper[5052]: I1203 13:15:56.053076 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/49c083020ae5dfe237b73a6c6b807501660a323f061d7879268c43a121bn4xq" event={"ID":"4903ac07-9e1a-41f9-9d62-23ebfb24f0a2","Type":"ContainerDied","Data":"70b90ec2bd7bf33637c4f77fd83ed552d859433d48e56f89379d0ab0cef6b4f4"} Dec 03 13:15:56 crc kubenswrapper[5052]: I1203 13:15:56.056818 5052 generic.go:334] "Generic (PLEG): container finished" podID="f7eebd92-c92d-4184-a171-15d7106fbc1e" containerID="e94191eba67dfa682dee955e2d1dbc7b54af8cf1fec07aaa1bf038dc762da2e2" exitCode=0 Dec 03 13:15:56 crc kubenswrapper[5052]: I1203 13:15:56.056861 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/community-operators-cs7vl" event={"ID":"f7eebd92-c92d-4184-a171-15d7106fbc1e","Type":"ContainerDied","Data":"e94191eba67dfa682dee955e2d1dbc7b54af8cf1fec07aaa1bf038dc762da2e2"} Dec 03 13:15:56 crc kubenswrapper[5052]: I1203 13:15:56.056886 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/community-operators-cs7vl" event={"ID":"f7eebd92-c92d-4184-a171-15d7106fbc1e","Type":"ContainerStarted","Data":"6690115f093e170953a0096c1474e465df59e3559909d2a5bb7993a1521166e4"} Dec 03 13:15:57 crc kubenswrapper[5052]: I1203 13:15:57.065906 5052 generic.go:334] "Generic (PLEG): container finished" podID="f7eebd92-c92d-4184-a171-15d7106fbc1e" containerID="b0867e196cd9a52872b0257e046b86e35981914382e8873fcf59eb54af7729ec" exitCode=0 Dec 03 13:15:57 crc kubenswrapper[5052]: I1203 13:15:57.065978 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/community-operators-cs7vl" event={"ID":"f7eebd92-c92d-4184-a171-15d7106fbc1e","Type":"ContainerDied","Data":"b0867e196cd9a52872b0257e046b86e35981914382e8873fcf59eb54af7729ec"} Dec 03 13:15:57 crc kubenswrapper[5052]: I1203 13:15:57.070740 5052 generic.go:334] "Generic (PLEG): container finished" podID="4903ac07-9e1a-41f9-9d62-23ebfb24f0a2" containerID="e7a4da6499c8365eb070c94f4a88cd1c2a40f56b2ad8397b4c8ecc9e35e5d297" exitCode=0 Dec 03 13:15:57 crc kubenswrapper[5052]: I1203 13:15:57.070779 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/49c083020ae5dfe237b73a6c6b807501660a323f061d7879268c43a121bn4xq" event={"ID":"4903ac07-9e1a-41f9-9d62-23ebfb24f0a2","Type":"ContainerDied","Data":"e7a4da6499c8365eb070c94f4a88cd1c2a40f56b2ad8397b4c8ecc9e35e5d297"} Dec 03 13:15:58 crc kubenswrapper[5052]: I1203 13:15:58.080861 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/community-operators-cs7vl" event={"ID":"f7eebd92-c92d-4184-a171-15d7106fbc1e","Type":"ContainerStarted","Data":"5ea958fb5306423ea057d1946c99896611162e111fd2e3a9bdd856dddb8c7618"} Dec 03 13:15:58 crc kubenswrapper[5052]: I1203 13:15:58.109612 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-marketplace/community-operators-cs7vl" podStartSLOduration=2.663888637 podStartE2EDuration="4.109591478s" podCreationTimestamp="2025-12-03 13:15:54 +0000 UTC" firstStartedPulling="2025-12-03 13:15:56.058274174 +0000 UTC m=+6946.646515938" lastFinishedPulling="2025-12-03 13:15:57.503977015 +0000 UTC m=+6948.092218779" observedRunningTime="2025-12-03 13:15:58.105032878 +0000 UTC m=+6948.693274642" watchObservedRunningTime="2025-12-03 13:15:58.109591478 +0000 UTC m=+6948.697833242" Dec 03 13:15:58 crc kubenswrapper[5052]: I1203 13:15:58.464767 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-operators/49c083020ae5dfe237b73a6c6b807501660a323f061d7879268c43a121bn4xq" Dec 03 13:15:58 crc kubenswrapper[5052]: I1203 13:15:58.549038 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4903ac07-9e1a-41f9-9d62-23ebfb24f0a2-util\") pod \"4903ac07-9e1a-41f9-9d62-23ebfb24f0a2\" (UID: \"4903ac07-9e1a-41f9-9d62-23ebfb24f0a2\") " Dec 03 13:15:58 crc kubenswrapper[5052]: I1203 13:15:58.549142 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-84ng8\" (UniqueName: \"kubernetes.io/projected/4903ac07-9e1a-41f9-9d62-23ebfb24f0a2-kube-api-access-84ng8\") pod \"4903ac07-9e1a-41f9-9d62-23ebfb24f0a2\" (UID: \"4903ac07-9e1a-41f9-9d62-23ebfb24f0a2\") " Dec 03 13:15:58 crc kubenswrapper[5052]: I1203 13:15:58.549269 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4903ac07-9e1a-41f9-9d62-23ebfb24f0a2-bundle\") pod \"4903ac07-9e1a-41f9-9d62-23ebfb24f0a2\" (UID: \"4903ac07-9e1a-41f9-9d62-23ebfb24f0a2\") " Dec 03 13:15:58 crc kubenswrapper[5052]: I1203 13:15:58.550118 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4903ac07-9e1a-41f9-9d62-23ebfb24f0a2-bundle" (OuterVolumeSpecName: "bundle") pod "4903ac07-9e1a-41f9-9d62-23ebfb24f0a2" (UID: "4903ac07-9e1a-41f9-9d62-23ebfb24f0a2"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 13:15:58 crc kubenswrapper[5052]: I1203 13:15:58.555407 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4903ac07-9e1a-41f9-9d62-23ebfb24f0a2-kube-api-access-84ng8" (OuterVolumeSpecName: "kube-api-access-84ng8") pod "4903ac07-9e1a-41f9-9d62-23ebfb24f0a2" (UID: "4903ac07-9e1a-41f9-9d62-23ebfb24f0a2"). InnerVolumeSpecName "kube-api-access-84ng8". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:15:58 crc kubenswrapper[5052]: I1203 13:15:58.572228 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4903ac07-9e1a-41f9-9d62-23ebfb24f0a2-util" (OuterVolumeSpecName: "util") pod "4903ac07-9e1a-41f9-9d62-23ebfb24f0a2" (UID: "4903ac07-9e1a-41f9-9d62-23ebfb24f0a2"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 13:15:58 crc kubenswrapper[5052]: I1203 13:15:58.651810 5052 reconciler_common.go:293] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4903ac07-9e1a-41f9-9d62-23ebfb24f0a2-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 13:15:58 crc kubenswrapper[5052]: I1203 13:15:58.652258 5052 reconciler_common.go:293] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4903ac07-9e1a-41f9-9d62-23ebfb24f0a2-util\") on node \"crc\" DevicePath \"\"" Dec 03 13:15:58 crc kubenswrapper[5052]: I1203 13:15:58.652377 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-84ng8\" (UniqueName: \"kubernetes.io/projected/4903ac07-9e1a-41f9-9d62-23ebfb24f0a2-kube-api-access-84ng8\") on node \"crc\" DevicePath \"\"" Dec 03 13:15:59 crc kubenswrapper[5052]: I1203 13:15:59.091102 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/49c083020ae5dfe237b73a6c6b807501660a323f061d7879268c43a121bn4xq" event={"ID":"4903ac07-9e1a-41f9-9d62-23ebfb24f0a2","Type":"ContainerDied","Data":"50231d2ab95b75afdf907ccf83edd321b011b7620171da6993993bc7ffc713a9"} Dec 03 13:15:59 crc kubenswrapper[5052]: I1203 13:15:59.091164 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-operators/49c083020ae5dfe237b73a6c6b807501660a323f061d7879268c43a121bn4xq" Dec 03 13:15:59 crc kubenswrapper[5052]: I1203 13:15:59.091178 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="50231d2ab95b75afdf907ccf83edd321b011b7620171da6993993bc7ffc713a9" Dec 03 13:15:59 crc kubenswrapper[5052]: I1203 13:15:59.522168 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["barbican-kuttl-tests/rabbitmq-server-0"] Dec 03 13:15:59 crc kubenswrapper[5052]: E1203 13:15:59.522740 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="4903ac07-9e1a-41f9-9d62-23ebfb24f0a2" containerName="util" Dec 03 13:15:59 crc kubenswrapper[5052]: I1203 13:15:59.522760 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="4903ac07-9e1a-41f9-9d62-23ebfb24f0a2" containerName="util" Dec 03 13:15:59 crc kubenswrapper[5052]: E1203 13:15:59.522773 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="4903ac07-9e1a-41f9-9d62-23ebfb24f0a2" containerName="extract" Dec 03 13:15:59 crc kubenswrapper[5052]: I1203 13:15:59.522783 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="4903ac07-9e1a-41f9-9d62-23ebfb24f0a2" containerName="extract" Dec 03 13:15:59 crc kubenswrapper[5052]: E1203 13:15:59.522798 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="4903ac07-9e1a-41f9-9d62-23ebfb24f0a2" containerName="pull" Dec 03 13:15:59 crc kubenswrapper[5052]: I1203 13:15:59.522806 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="4903ac07-9e1a-41f9-9d62-23ebfb24f0a2" containerName="pull" Dec 03 13:15:59 crc kubenswrapper[5052]: I1203 13:15:59.522951 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="4903ac07-9e1a-41f9-9d62-23ebfb24f0a2" containerName="extract" Dec 03 13:15:59 crc kubenswrapper[5052]: I1203 13:15:59.523798 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="barbican-kuttl-tests/rabbitmq-server-0" Dec 03 13:15:59 crc kubenswrapper[5052]: I1203 13:15:59.528935 5052 reflector.go:368] Caches populated for *v1.Secret from object-"barbican-kuttl-tests"/"rabbitmq-erlang-cookie" Dec 03 13:15:59 crc kubenswrapper[5052]: I1203 13:15:59.529203 5052 reflector.go:368] Caches populated for *v1.Secret from object-"barbican-kuttl-tests"/"rabbitmq-server-dockercfg-wprwx" Dec 03 13:15:59 crc kubenswrapper[5052]: I1203 13:15:59.529427 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"barbican-kuttl-tests"/"rabbitmq-server-conf" Dec 03 13:15:59 crc kubenswrapper[5052]: I1203 13:15:59.529927 5052 reflector.go:368] Caches populated for *v1.Secret from object-"barbican-kuttl-tests"/"rabbitmq-default-user" Dec 03 13:15:59 crc kubenswrapper[5052]: I1203 13:15:59.536924 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"barbican-kuttl-tests"/"rabbitmq-plugins-conf" Dec 03 13:15:59 crc kubenswrapper[5052]: I1203 13:15:59.548060 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["barbican-kuttl-tests/rabbitmq-server-0"] Dec 03 13:15:59 crc kubenswrapper[5052]: I1203 13:15:59.677262 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"pvc-309aff0f-7126-4483-9f08-540649d86541\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-309aff0f-7126-4483-9f08-540649d86541\") pod \"rabbitmq-server-0\" (UID: \"caf5c2cd-0156-4996-b20a-e5b5b9255ddc\") " pod="barbican-kuttl-tests/rabbitmq-server-0" Dec 03 13:15:59 crc kubenswrapper[5052]: I1203 13:15:59.677331 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"rabbitmq-confd\" (UniqueName: \"kubernetes.io/projected/caf5c2cd-0156-4996-b20a-e5b5b9255ddc-rabbitmq-confd\") pod \"rabbitmq-server-0\" (UID: \"caf5c2cd-0156-4996-b20a-e5b5b9255ddc\") " pod="barbican-kuttl-tests/rabbitmq-server-0" Dec 03 13:15:59 crc kubenswrapper[5052]: I1203 13:15:59.677364 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q7mvk\" (UniqueName: \"kubernetes.io/projected/caf5c2cd-0156-4996-b20a-e5b5b9255ddc-kube-api-access-q7mvk\") pod \"rabbitmq-server-0\" (UID: \"caf5c2cd-0156-4996-b20a-e5b5b9255ddc\") " pod="barbican-kuttl-tests/rabbitmq-server-0" Dec 03 13:15:59 crc kubenswrapper[5052]: I1203 13:15:59.677414 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"pod-info\" (UniqueName: \"kubernetes.io/downward-api/caf5c2cd-0156-4996-b20a-e5b5b9255ddc-pod-info\") pod \"rabbitmq-server-0\" (UID: \"caf5c2cd-0156-4996-b20a-e5b5b9255ddc\") " pod="barbican-kuttl-tests/rabbitmq-server-0" Dec 03 13:15:59 crc kubenswrapper[5052]: I1203 13:15:59.677470 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"erlang-cookie-secret\" (UniqueName: \"kubernetes.io/secret/caf5c2cd-0156-4996-b20a-e5b5b9255ddc-erlang-cookie-secret\") pod \"rabbitmq-server-0\" (UID: \"caf5c2cd-0156-4996-b20a-e5b5b9255ddc\") " pod="barbican-kuttl-tests/rabbitmq-server-0" Dec 03 13:15:59 crc kubenswrapper[5052]: I1203 13:15:59.677547 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-conf\" (UniqueName: \"kubernetes.io/configmap/caf5c2cd-0156-4996-b20a-e5b5b9255ddc-plugins-conf\") pod \"rabbitmq-server-0\" (UID: \"caf5c2cd-0156-4996-b20a-e5b5b9255ddc\") " pod="barbican-kuttl-tests/rabbitmq-server-0" Dec 03 13:15:59 crc kubenswrapper[5052]: I1203 13:15:59.677575 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"rabbitmq-erlang-cookie\" (UniqueName: \"kubernetes.io/empty-dir/caf5c2cd-0156-4996-b20a-e5b5b9255ddc-rabbitmq-erlang-cookie\") pod \"rabbitmq-server-0\" (UID: \"caf5c2cd-0156-4996-b20a-e5b5b9255ddc\") " pod="barbican-kuttl-tests/rabbitmq-server-0" Dec 03 13:15:59 crc kubenswrapper[5052]: I1203 13:15:59.677607 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"rabbitmq-plugins\" (UniqueName: \"kubernetes.io/empty-dir/caf5c2cd-0156-4996-b20a-e5b5b9255ddc-rabbitmq-plugins\") pod \"rabbitmq-server-0\" (UID: \"caf5c2cd-0156-4996-b20a-e5b5b9255ddc\") " pod="barbican-kuttl-tests/rabbitmq-server-0" Dec 03 13:15:59 crc kubenswrapper[5052]: I1203 13:15:59.778844 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"pvc-309aff0f-7126-4483-9f08-540649d86541\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-309aff0f-7126-4483-9f08-540649d86541\") pod \"rabbitmq-server-0\" (UID: \"caf5c2cd-0156-4996-b20a-e5b5b9255ddc\") " pod="barbican-kuttl-tests/rabbitmq-server-0" Dec 03 13:15:59 crc kubenswrapper[5052]: I1203 13:15:59.778894 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"rabbitmq-confd\" (UniqueName: \"kubernetes.io/projected/caf5c2cd-0156-4996-b20a-e5b5b9255ddc-rabbitmq-confd\") pod \"rabbitmq-server-0\" (UID: \"caf5c2cd-0156-4996-b20a-e5b5b9255ddc\") " pod="barbican-kuttl-tests/rabbitmq-server-0" Dec 03 13:15:59 crc kubenswrapper[5052]: I1203 13:15:59.778919 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-q7mvk\" (UniqueName: \"kubernetes.io/projected/caf5c2cd-0156-4996-b20a-e5b5b9255ddc-kube-api-access-q7mvk\") pod \"rabbitmq-server-0\" (UID: \"caf5c2cd-0156-4996-b20a-e5b5b9255ddc\") " pod="barbican-kuttl-tests/rabbitmq-server-0" Dec 03 13:15:59 crc kubenswrapper[5052]: I1203 13:15:59.778957 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"pod-info\" (UniqueName: \"kubernetes.io/downward-api/caf5c2cd-0156-4996-b20a-e5b5b9255ddc-pod-info\") pod \"rabbitmq-server-0\" (UID: \"caf5c2cd-0156-4996-b20a-e5b5b9255ddc\") " pod="barbican-kuttl-tests/rabbitmq-server-0" Dec 03 13:15:59 crc kubenswrapper[5052]: I1203 13:15:59.779014 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"erlang-cookie-secret\" (UniqueName: \"kubernetes.io/secret/caf5c2cd-0156-4996-b20a-e5b5b9255ddc-erlang-cookie-secret\") pod \"rabbitmq-server-0\" (UID: \"caf5c2cd-0156-4996-b20a-e5b5b9255ddc\") " pod="barbican-kuttl-tests/rabbitmq-server-0" Dec 03 13:15:59 crc kubenswrapper[5052]: I1203 13:15:59.779096 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"plugins-conf\" (UniqueName: \"kubernetes.io/configmap/caf5c2cd-0156-4996-b20a-e5b5b9255ddc-plugins-conf\") pod \"rabbitmq-server-0\" (UID: \"caf5c2cd-0156-4996-b20a-e5b5b9255ddc\") " pod="barbican-kuttl-tests/rabbitmq-server-0" Dec 03 13:15:59 crc kubenswrapper[5052]: I1203 13:15:59.779117 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"rabbitmq-erlang-cookie\" (UniqueName: \"kubernetes.io/empty-dir/caf5c2cd-0156-4996-b20a-e5b5b9255ddc-rabbitmq-erlang-cookie\") pod \"rabbitmq-server-0\" (UID: \"caf5c2cd-0156-4996-b20a-e5b5b9255ddc\") " pod="barbican-kuttl-tests/rabbitmq-server-0" Dec 03 13:15:59 crc kubenswrapper[5052]: I1203 13:15:59.779142 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"rabbitmq-plugins\" (UniqueName: \"kubernetes.io/empty-dir/caf5c2cd-0156-4996-b20a-e5b5b9255ddc-rabbitmq-plugins\") pod \"rabbitmq-server-0\" (UID: \"caf5c2cd-0156-4996-b20a-e5b5b9255ddc\") " pod="barbican-kuttl-tests/rabbitmq-server-0" Dec 03 13:15:59 crc kubenswrapper[5052]: I1203 13:15:59.780086 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"rabbitmq-erlang-cookie\" (UniqueName: \"kubernetes.io/empty-dir/caf5c2cd-0156-4996-b20a-e5b5b9255ddc-rabbitmq-erlang-cookie\") pod \"rabbitmq-server-0\" (UID: \"caf5c2cd-0156-4996-b20a-e5b5b9255ddc\") " pod="barbican-kuttl-tests/rabbitmq-server-0" Dec 03 13:15:59 crc kubenswrapper[5052]: I1203 13:15:59.780471 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"rabbitmq-plugins\" (UniqueName: \"kubernetes.io/empty-dir/caf5c2cd-0156-4996-b20a-e5b5b9255ddc-rabbitmq-plugins\") pod \"rabbitmq-server-0\" (UID: \"caf5c2cd-0156-4996-b20a-e5b5b9255ddc\") " pod="barbican-kuttl-tests/rabbitmq-server-0" Dec 03 13:15:59 crc kubenswrapper[5052]: I1203 13:15:59.780501 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"plugins-conf\" (UniqueName: \"kubernetes.io/configmap/caf5c2cd-0156-4996-b20a-e5b5b9255ddc-plugins-conf\") pod \"rabbitmq-server-0\" (UID: \"caf5c2cd-0156-4996-b20a-e5b5b9255ddc\") " pod="barbican-kuttl-tests/rabbitmq-server-0" Dec 03 13:15:59 crc kubenswrapper[5052]: I1203 13:15:59.784543 5052 csi_attacher.go:380] kubernetes.io/csi: attacher.MountDevice STAGE_UNSTAGE_VOLUME capability not set. Skipping MountDevice... Dec 03 13:15:59 crc kubenswrapper[5052]: I1203 13:15:59.784577 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"pvc-309aff0f-7126-4483-9f08-540649d86541\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-309aff0f-7126-4483-9f08-540649d86541\") pod \"rabbitmq-server-0\" (UID: \"caf5c2cd-0156-4996-b20a-e5b5b9255ddc\") device mount path \"/var/lib/kubelet/plugins/kubernetes.io/csi/kubevirt.io.hostpath-provisioner/ec6a7d17763d84b77dff5f7094bc980f2dc77527ca6d65b5967b533201e98d84/globalmount\"" pod="barbican-kuttl-tests/rabbitmq-server-0" Dec 03 13:15:59 crc kubenswrapper[5052]: I1203 13:15:59.785963 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"pod-info\" (UniqueName: \"kubernetes.io/downward-api/caf5c2cd-0156-4996-b20a-e5b5b9255ddc-pod-info\") pod \"rabbitmq-server-0\" (UID: \"caf5c2cd-0156-4996-b20a-e5b5b9255ddc\") " pod="barbican-kuttl-tests/rabbitmq-server-0" Dec 03 13:15:59 crc kubenswrapper[5052]: I1203 13:15:59.786017 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"rabbitmq-confd\" (UniqueName: \"kubernetes.io/projected/caf5c2cd-0156-4996-b20a-e5b5b9255ddc-rabbitmq-confd\") pod \"rabbitmq-server-0\" (UID: \"caf5c2cd-0156-4996-b20a-e5b5b9255ddc\") " pod="barbican-kuttl-tests/rabbitmq-server-0" Dec 03 13:15:59 crc kubenswrapper[5052]: I1203 13:15:59.787850 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"erlang-cookie-secret\" (UniqueName: \"kubernetes.io/secret/caf5c2cd-0156-4996-b20a-e5b5b9255ddc-erlang-cookie-secret\") pod \"rabbitmq-server-0\" (UID: \"caf5c2cd-0156-4996-b20a-e5b5b9255ddc\") " pod="barbican-kuttl-tests/rabbitmq-server-0" Dec 03 13:15:59 crc kubenswrapper[5052]: I1203 13:15:59.796095 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-q7mvk\" (UniqueName: \"kubernetes.io/projected/caf5c2cd-0156-4996-b20a-e5b5b9255ddc-kube-api-access-q7mvk\") pod \"rabbitmq-server-0\" (UID: \"caf5c2cd-0156-4996-b20a-e5b5b9255ddc\") " pod="barbican-kuttl-tests/rabbitmq-server-0" Dec 03 13:15:59 crc kubenswrapper[5052]: I1203 13:15:59.812830 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"pvc-309aff0f-7126-4483-9f08-540649d86541\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-309aff0f-7126-4483-9f08-540649d86541\") pod \"rabbitmq-server-0\" (UID: \"caf5c2cd-0156-4996-b20a-e5b5b9255ddc\") " pod="barbican-kuttl-tests/rabbitmq-server-0" Dec 03 13:15:59 crc kubenswrapper[5052]: I1203 13:15:59.842035 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="barbican-kuttl-tests/rabbitmq-server-0" Dec 03 13:16:00 crc kubenswrapper[5052]: I1203 13:16:00.241119 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["barbican-kuttl-tests/rabbitmq-server-0"] Dec 03 13:16:00 crc kubenswrapper[5052]: W1203 13:16:00.252401 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcaf5c2cd_0156_4996_b20a_e5b5b9255ddc.slice/crio-8f32b7ae471678447346e896dc4c9d3ef5039195cbc8276a4a43e1c308a474ea WatchSource:0}: Error finding container 8f32b7ae471678447346e896dc4c9d3ef5039195cbc8276a4a43e1c308a474ea: Status 404 returned error can't find the container with id 8f32b7ae471678447346e896dc4c9d3ef5039195cbc8276a4a43e1c308a474ea Dec 03 13:16:01 crc kubenswrapper[5052]: I1203 13:16:01.109221 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="barbican-kuttl-tests/rabbitmq-server-0" event={"ID":"caf5c2cd-0156-4996-b20a-e5b5b9255ddc","Type":"ContainerStarted","Data":"8f32b7ae471678447346e896dc4c9d3ef5039195cbc8276a4a43e1c308a474ea"} Dec 03 13:16:04 crc kubenswrapper[5052]: I1203 13:16:04.889466 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-marketplace/community-operators-cs7vl" Dec 03 13:16:04 crc kubenswrapper[5052]: I1203 13:16:04.890097 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openshift-marketplace/community-operators-cs7vl" Dec 03 13:16:04 crc kubenswrapper[5052]: I1203 13:16:04.940231 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-marketplace/community-operators-cs7vl" Dec 03 13:16:05 crc kubenswrapper[5052]: I1203 13:16:05.204166 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-marketplace/community-operators-cs7vl" Dec 03 13:16:06 crc kubenswrapper[5052]: I1203 13:16:06.467921 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openshift-marketplace/community-operators-cs7vl"] Dec 03 13:16:07 crc kubenswrapper[5052]: I1203 13:16:07.166880 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openshift-marketplace/community-operators-cs7vl" podUID="f7eebd92-c92d-4184-a171-15d7106fbc1e" containerName="registry-server" containerID="cri-o://5ea958fb5306423ea057d1946c99896611162e111fd2e3a9bdd856dddb8c7618" gracePeriod=2 Dec 03 13:16:08 crc kubenswrapper[5052]: I1203 13:16:08.871528 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/community-operators-cs7vl" Dec 03 13:16:09 crc kubenswrapper[5052]: I1203 13:16:09.064338 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9t2bd\" (UniqueName: \"kubernetes.io/projected/f7eebd92-c92d-4184-a171-15d7106fbc1e-kube-api-access-9t2bd\") pod \"f7eebd92-c92d-4184-a171-15d7106fbc1e\" (UID: \"f7eebd92-c92d-4184-a171-15d7106fbc1e\") " Dec 03 13:16:09 crc kubenswrapper[5052]: I1203 13:16:09.064999 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/f7eebd92-c92d-4184-a171-15d7106fbc1e-utilities\") pod \"f7eebd92-c92d-4184-a171-15d7106fbc1e\" (UID: \"f7eebd92-c92d-4184-a171-15d7106fbc1e\") " Dec 03 13:16:09 crc kubenswrapper[5052]: I1203 13:16:09.065069 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/f7eebd92-c92d-4184-a171-15d7106fbc1e-catalog-content\") pod \"f7eebd92-c92d-4184-a171-15d7106fbc1e\" (UID: \"f7eebd92-c92d-4184-a171-15d7106fbc1e\") " Dec 03 13:16:09 crc kubenswrapper[5052]: I1203 13:16:09.065806 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7eebd92-c92d-4184-a171-15d7106fbc1e-utilities" (OuterVolumeSpecName: "utilities") pod "f7eebd92-c92d-4184-a171-15d7106fbc1e" (UID: "f7eebd92-c92d-4184-a171-15d7106fbc1e"). InnerVolumeSpecName "utilities". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 13:16:09 crc kubenswrapper[5052]: I1203 13:16:09.094311 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f7eebd92-c92d-4184-a171-15d7106fbc1e-kube-api-access-9t2bd" (OuterVolumeSpecName: "kube-api-access-9t2bd") pod "f7eebd92-c92d-4184-a171-15d7106fbc1e" (UID: "f7eebd92-c92d-4184-a171-15d7106fbc1e"). InnerVolumeSpecName "kube-api-access-9t2bd". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:16:09 crc kubenswrapper[5052]: I1203 13:16:09.141616 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7eebd92-c92d-4184-a171-15d7106fbc1e-catalog-content" (OuterVolumeSpecName: "catalog-content") pod "f7eebd92-c92d-4184-a171-15d7106fbc1e" (UID: "f7eebd92-c92d-4184-a171-15d7106fbc1e"). InnerVolumeSpecName "catalog-content". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 13:16:09 crc kubenswrapper[5052]: I1203 13:16:09.167311 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-9t2bd\" (UniqueName: \"kubernetes.io/projected/f7eebd92-c92d-4184-a171-15d7106fbc1e-kube-api-access-9t2bd\") on node \"crc\" DevicePath \"\"" Dec 03 13:16:09 crc kubenswrapper[5052]: I1203 13:16:09.167363 5052 reconciler_common.go:293] "Volume detached for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/f7eebd92-c92d-4184-a171-15d7106fbc1e-utilities\") on node \"crc\" DevicePath \"\"" Dec 03 13:16:09 crc kubenswrapper[5052]: I1203 13:16:09.167375 5052 reconciler_common.go:293] "Volume detached for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/f7eebd92-c92d-4184-a171-15d7106fbc1e-catalog-content\") on node \"crc\" DevicePath \"\"" Dec 03 13:16:09 crc kubenswrapper[5052]: I1203 13:16:09.182212 5052 generic.go:334] "Generic (PLEG): container finished" podID="f7eebd92-c92d-4184-a171-15d7106fbc1e" containerID="5ea958fb5306423ea057d1946c99896611162e111fd2e3a9bdd856dddb8c7618" exitCode=0 Dec 03 13:16:09 crc kubenswrapper[5052]: I1203 13:16:09.182254 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/community-operators-cs7vl" Dec 03 13:16:09 crc kubenswrapper[5052]: I1203 13:16:09.182258 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/community-operators-cs7vl" event={"ID":"f7eebd92-c92d-4184-a171-15d7106fbc1e","Type":"ContainerDied","Data":"5ea958fb5306423ea057d1946c99896611162e111fd2e3a9bdd856dddb8c7618"} Dec 03 13:16:09 crc kubenswrapper[5052]: I1203 13:16:09.182291 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/community-operators-cs7vl" event={"ID":"f7eebd92-c92d-4184-a171-15d7106fbc1e","Type":"ContainerDied","Data":"6690115f093e170953a0096c1474e465df59e3559909d2a5bb7993a1521166e4"} Dec 03 13:16:09 crc kubenswrapper[5052]: I1203 13:16:09.182311 5052 scope.go:117] "RemoveContainer" containerID="5ea958fb5306423ea057d1946c99896611162e111fd2e3a9bdd856dddb8c7618" Dec 03 13:16:09 crc kubenswrapper[5052]: I1203 13:16:09.214223 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openshift-marketplace/community-operators-cs7vl"] Dec 03 13:16:09 crc kubenswrapper[5052]: I1203 13:16:09.218723 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openshift-marketplace/community-operators-cs7vl"] Dec 03 13:16:09 crc kubenswrapper[5052]: I1203 13:16:09.686482 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f7eebd92-c92d-4184-a171-15d7106fbc1e" path="/var/lib/kubelet/pods/f7eebd92-c92d-4184-a171-15d7106fbc1e/volumes" Dec 03 13:16:09 crc kubenswrapper[5052]: I1203 13:16:09.700589 5052 scope.go:117] "RemoveContainer" containerID="b0867e196cd9a52872b0257e046b86e35981914382e8873fcf59eb54af7729ec" Dec 03 13:16:09 crc kubenswrapper[5052]: I1203 13:16:09.752782 5052 scope.go:117] "RemoveContainer" containerID="e94191eba67dfa682dee955e2d1dbc7b54af8cf1fec07aaa1bf038dc762da2e2" Dec 03 13:16:09 crc kubenswrapper[5052]: I1203 13:16:09.839604 5052 scope.go:117] "RemoveContainer" containerID="5ea958fb5306423ea057d1946c99896611162e111fd2e3a9bdd856dddb8c7618" Dec 03 13:16:09 crc kubenswrapper[5052]: E1203 13:16:09.840116 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5ea958fb5306423ea057d1946c99896611162e111fd2e3a9bdd856dddb8c7618\": container with ID starting with 5ea958fb5306423ea057d1946c99896611162e111fd2e3a9bdd856dddb8c7618 not found: ID does not exist" containerID="5ea958fb5306423ea057d1946c99896611162e111fd2e3a9bdd856dddb8c7618" Dec 03 13:16:09 crc kubenswrapper[5052]: I1203 13:16:09.840189 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5ea958fb5306423ea057d1946c99896611162e111fd2e3a9bdd856dddb8c7618"} err="failed to get container status \"5ea958fb5306423ea057d1946c99896611162e111fd2e3a9bdd856dddb8c7618\": rpc error: code = NotFound desc = could not find container \"5ea958fb5306423ea057d1946c99896611162e111fd2e3a9bdd856dddb8c7618\": container with ID starting with 5ea958fb5306423ea057d1946c99896611162e111fd2e3a9bdd856dddb8c7618 not found: ID does not exist" Dec 03 13:16:09 crc kubenswrapper[5052]: I1203 13:16:09.840218 5052 scope.go:117] "RemoveContainer" containerID="b0867e196cd9a52872b0257e046b86e35981914382e8873fcf59eb54af7729ec" Dec 03 13:16:09 crc kubenswrapper[5052]: E1203 13:16:09.840663 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b0867e196cd9a52872b0257e046b86e35981914382e8873fcf59eb54af7729ec\": container with ID starting with b0867e196cd9a52872b0257e046b86e35981914382e8873fcf59eb54af7729ec not found: ID does not exist" containerID="b0867e196cd9a52872b0257e046b86e35981914382e8873fcf59eb54af7729ec" Dec 03 13:16:09 crc kubenswrapper[5052]: I1203 13:16:09.840698 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b0867e196cd9a52872b0257e046b86e35981914382e8873fcf59eb54af7729ec"} err="failed to get container status \"b0867e196cd9a52872b0257e046b86e35981914382e8873fcf59eb54af7729ec\": rpc error: code = NotFound desc = could not find container \"b0867e196cd9a52872b0257e046b86e35981914382e8873fcf59eb54af7729ec\": container with ID starting with b0867e196cd9a52872b0257e046b86e35981914382e8873fcf59eb54af7729ec not found: ID does not exist" Dec 03 13:16:09 crc kubenswrapper[5052]: I1203 13:16:09.840724 5052 scope.go:117] "RemoveContainer" containerID="e94191eba67dfa682dee955e2d1dbc7b54af8cf1fec07aaa1bf038dc762da2e2" Dec 03 13:16:09 crc kubenswrapper[5052]: E1203 13:16:09.840962 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e94191eba67dfa682dee955e2d1dbc7b54af8cf1fec07aaa1bf038dc762da2e2\": container with ID starting with e94191eba67dfa682dee955e2d1dbc7b54af8cf1fec07aaa1bf038dc762da2e2 not found: ID does not exist" containerID="e94191eba67dfa682dee955e2d1dbc7b54af8cf1fec07aaa1bf038dc762da2e2" Dec 03 13:16:09 crc kubenswrapper[5052]: I1203 13:16:09.840999 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e94191eba67dfa682dee955e2d1dbc7b54af8cf1fec07aaa1bf038dc762da2e2"} err="failed to get container status \"e94191eba67dfa682dee955e2d1dbc7b54af8cf1fec07aaa1bf038dc762da2e2\": rpc error: code = NotFound desc = could not find container \"e94191eba67dfa682dee955e2d1dbc7b54af8cf1fec07aaa1bf038dc762da2e2\": container with ID starting with e94191eba67dfa682dee955e2d1dbc7b54af8cf1fec07aaa1bf038dc762da2e2 not found: ID does not exist" Dec 03 13:16:10 crc kubenswrapper[5052]: I1203 13:16:10.918366 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-operators/keystone-operator-controller-manager-5dfbb47f6f-jmkcm"] Dec 03 13:16:10 crc kubenswrapper[5052]: E1203 13:16:10.918982 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="f7eebd92-c92d-4184-a171-15d7106fbc1e" containerName="extract-utilities" Dec 03 13:16:10 crc kubenswrapper[5052]: I1203 13:16:10.918995 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7eebd92-c92d-4184-a171-15d7106fbc1e" containerName="extract-utilities" Dec 03 13:16:10 crc kubenswrapper[5052]: E1203 13:16:10.919006 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="f7eebd92-c92d-4184-a171-15d7106fbc1e" containerName="registry-server" Dec 03 13:16:10 crc kubenswrapper[5052]: I1203 13:16:10.919011 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7eebd92-c92d-4184-a171-15d7106fbc1e" containerName="registry-server" Dec 03 13:16:10 crc kubenswrapper[5052]: E1203 13:16:10.919055 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="f7eebd92-c92d-4184-a171-15d7106fbc1e" containerName="extract-content" Dec 03 13:16:10 crc kubenswrapper[5052]: I1203 13:16:10.919064 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7eebd92-c92d-4184-a171-15d7106fbc1e" containerName="extract-content" Dec 03 13:16:10 crc kubenswrapper[5052]: I1203 13:16:10.919224 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="f7eebd92-c92d-4184-a171-15d7106fbc1e" containerName="registry-server" Dec 03 13:16:10 crc kubenswrapper[5052]: I1203 13:16:10.919710 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-operators/keystone-operator-controller-manager-5dfbb47f6f-jmkcm" Dec 03 13:16:10 crc kubenswrapper[5052]: I1203 13:16:10.921356 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-operators"/"keystone-operator-controller-manager-dockercfg-cmrkh" Dec 03 13:16:10 crc kubenswrapper[5052]: I1203 13:16:10.921573 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-operators"/"keystone-operator-controller-manager-service-cert" Dec 03 13:16:10 crc kubenswrapper[5052]: I1203 13:16:10.928151 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-operators/keystone-operator-controller-manager-5dfbb47f6f-jmkcm"] Dec 03 13:16:11 crc kubenswrapper[5052]: I1203 13:16:11.000139 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/995e0ea0-f1df-4bf5-8d59-1d75f9861688-webhook-cert\") pod \"keystone-operator-controller-manager-5dfbb47f6f-jmkcm\" (UID: \"995e0ea0-f1df-4bf5-8d59-1d75f9861688\") " pod="openstack-operators/keystone-operator-controller-manager-5dfbb47f6f-jmkcm" Dec 03 13:16:11 crc kubenswrapper[5052]: I1203 13:16:11.000204 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/995e0ea0-f1df-4bf5-8d59-1d75f9861688-apiservice-cert\") pod \"keystone-operator-controller-manager-5dfbb47f6f-jmkcm\" (UID: \"995e0ea0-f1df-4bf5-8d59-1d75f9861688\") " pod="openstack-operators/keystone-operator-controller-manager-5dfbb47f6f-jmkcm" Dec 03 13:16:11 crc kubenswrapper[5052]: I1203 13:16:11.000266 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zms57\" (UniqueName: \"kubernetes.io/projected/995e0ea0-f1df-4bf5-8d59-1d75f9861688-kube-api-access-zms57\") pod \"keystone-operator-controller-manager-5dfbb47f6f-jmkcm\" (UID: \"995e0ea0-f1df-4bf5-8d59-1d75f9861688\") " pod="openstack-operators/keystone-operator-controller-manager-5dfbb47f6f-jmkcm" Dec 03 13:16:11 crc kubenswrapper[5052]: I1203 13:16:11.103122 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/995e0ea0-f1df-4bf5-8d59-1d75f9861688-webhook-cert\") pod \"keystone-operator-controller-manager-5dfbb47f6f-jmkcm\" (UID: \"995e0ea0-f1df-4bf5-8d59-1d75f9861688\") " pod="openstack-operators/keystone-operator-controller-manager-5dfbb47f6f-jmkcm" Dec 03 13:16:11 crc kubenswrapper[5052]: I1203 13:16:11.103173 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/995e0ea0-f1df-4bf5-8d59-1d75f9861688-apiservice-cert\") pod \"keystone-operator-controller-manager-5dfbb47f6f-jmkcm\" (UID: \"995e0ea0-f1df-4bf5-8d59-1d75f9861688\") " pod="openstack-operators/keystone-operator-controller-manager-5dfbb47f6f-jmkcm" Dec 03 13:16:11 crc kubenswrapper[5052]: I1203 13:16:11.103233 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-zms57\" (UniqueName: \"kubernetes.io/projected/995e0ea0-f1df-4bf5-8d59-1d75f9861688-kube-api-access-zms57\") pod \"keystone-operator-controller-manager-5dfbb47f6f-jmkcm\" (UID: \"995e0ea0-f1df-4bf5-8d59-1d75f9861688\") " pod="openstack-operators/keystone-operator-controller-manager-5dfbb47f6f-jmkcm" Dec 03 13:16:11 crc kubenswrapper[5052]: I1203 13:16:11.108258 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/995e0ea0-f1df-4bf5-8d59-1d75f9861688-webhook-cert\") pod \"keystone-operator-controller-manager-5dfbb47f6f-jmkcm\" (UID: \"995e0ea0-f1df-4bf5-8d59-1d75f9861688\") " pod="openstack-operators/keystone-operator-controller-manager-5dfbb47f6f-jmkcm" Dec 03 13:16:11 crc kubenswrapper[5052]: I1203 13:16:11.108258 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/995e0ea0-f1df-4bf5-8d59-1d75f9861688-apiservice-cert\") pod \"keystone-operator-controller-manager-5dfbb47f6f-jmkcm\" (UID: \"995e0ea0-f1df-4bf5-8d59-1d75f9861688\") " pod="openstack-operators/keystone-operator-controller-manager-5dfbb47f6f-jmkcm" Dec 03 13:16:11 crc kubenswrapper[5052]: I1203 13:16:11.127401 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-zms57\" (UniqueName: \"kubernetes.io/projected/995e0ea0-f1df-4bf5-8d59-1d75f9861688-kube-api-access-zms57\") pod \"keystone-operator-controller-manager-5dfbb47f6f-jmkcm\" (UID: \"995e0ea0-f1df-4bf5-8d59-1d75f9861688\") " pod="openstack-operators/keystone-operator-controller-manager-5dfbb47f6f-jmkcm" Dec 03 13:16:11 crc kubenswrapper[5052]: I1203 13:16:11.199355 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="barbican-kuttl-tests/rabbitmq-server-0" event={"ID":"caf5c2cd-0156-4996-b20a-e5b5b9255ddc","Type":"ContainerStarted","Data":"78245ad00f303fd209e0f5933535394975e3fb1bf7c4663687be9ecf89b8800a"} Dec 03 13:16:11 crc kubenswrapper[5052]: I1203 13:16:11.239212 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-operators/keystone-operator-controller-manager-5dfbb47f6f-jmkcm" Dec 03 13:16:11 crc kubenswrapper[5052]: I1203 13:16:11.682445 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-operators/keystone-operator-controller-manager-5dfbb47f6f-jmkcm"] Dec 03 13:16:12 crc kubenswrapper[5052]: I1203 13:16:12.208384 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/keystone-operator-controller-manager-5dfbb47f6f-jmkcm" event={"ID":"995e0ea0-f1df-4bf5-8d59-1d75f9861688","Type":"ContainerStarted","Data":"a66cc6a12dc29455332fa1c0b95ceb0eae29e48533b2008cb042482d96cdf018"} Dec 03 13:16:13 crc kubenswrapper[5052]: I1203 13:16:13.218728 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/keystone-operator-controller-manager-5dfbb47f6f-jmkcm" event={"ID":"995e0ea0-f1df-4bf5-8d59-1d75f9861688","Type":"ContainerStarted","Data":"d3fab3a005806e264fd0127bc2b5e2af93aa6096567f55cdb094588d5174801b"} Dec 03 13:16:13 crc kubenswrapper[5052]: I1203 13:16:13.218935 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-operators/keystone-operator-controller-manager-5dfbb47f6f-jmkcm" Dec 03 13:16:13 crc kubenswrapper[5052]: I1203 13:16:13.247111 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-operators/keystone-operator-controller-manager-5dfbb47f6f-jmkcm" podStartSLOduration=3.247088973 podStartE2EDuration="3.247088973s" podCreationTimestamp="2025-12-03 13:16:10 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 13:16:13.238167587 +0000 UTC m=+6963.826409371" watchObservedRunningTime="2025-12-03 13:16:13.247088973 +0000 UTC m=+6963.835330737" Dec 03 13:16:21 crc kubenswrapper[5052]: I1203 13:16:21.654810 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-operators/keystone-operator-controller-manager-5dfbb47f6f-jmkcm" Dec 03 13:16:28 crc kubenswrapper[5052]: I1203 13:16:28.866723 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-operators/barbican-operator-index-27fzr"] Dec 03 13:16:28 crc kubenswrapper[5052]: I1203 13:16:28.868323 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-operators/barbican-operator-index-27fzr" Dec 03 13:16:28 crc kubenswrapper[5052]: I1203 13:16:28.870734 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-operators"/"barbican-operator-index-dockercfg-4lkt6" Dec 03 13:16:28 crc kubenswrapper[5052]: I1203 13:16:28.873435 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-operators/barbican-operator-index-27fzr"] Dec 03 13:16:28 crc kubenswrapper[5052]: I1203 13:16:28.970437 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rnndk\" (UniqueName: \"kubernetes.io/projected/42d5fccb-f24d-4707-8167-30eeae73378e-kube-api-access-rnndk\") pod \"barbican-operator-index-27fzr\" (UID: \"42d5fccb-f24d-4707-8167-30eeae73378e\") " pod="openstack-operators/barbican-operator-index-27fzr" Dec 03 13:16:29 crc kubenswrapper[5052]: I1203 13:16:29.071947 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-rnndk\" (UniqueName: \"kubernetes.io/projected/42d5fccb-f24d-4707-8167-30eeae73378e-kube-api-access-rnndk\") pod \"barbican-operator-index-27fzr\" (UID: \"42d5fccb-f24d-4707-8167-30eeae73378e\") " pod="openstack-operators/barbican-operator-index-27fzr" Dec 03 13:16:29 crc kubenswrapper[5052]: I1203 13:16:29.091811 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-rnndk\" (UniqueName: \"kubernetes.io/projected/42d5fccb-f24d-4707-8167-30eeae73378e-kube-api-access-rnndk\") pod \"barbican-operator-index-27fzr\" (UID: \"42d5fccb-f24d-4707-8167-30eeae73378e\") " pod="openstack-operators/barbican-operator-index-27fzr" Dec 03 13:16:29 crc kubenswrapper[5052]: I1203 13:16:29.188115 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-operators/barbican-operator-index-27fzr" Dec 03 13:16:29 crc kubenswrapper[5052]: I1203 13:16:29.632729 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-operators/barbican-operator-index-27fzr"] Dec 03 13:16:29 crc kubenswrapper[5052]: W1203 13:16:29.633720 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42d5fccb_f24d_4707_8167_30eeae73378e.slice/crio-3ac16256a126a72b395395510628778e7f28a21d38ee102ef12dfc83d2615d96 WatchSource:0}: Error finding container 3ac16256a126a72b395395510628778e7f28a21d38ee102ef12dfc83d2615d96: Status 404 returned error can't find the container with id 3ac16256a126a72b395395510628778e7f28a21d38ee102ef12dfc83d2615d96 Dec 03 13:16:30 crc kubenswrapper[5052]: I1203 13:16:30.355730 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/barbican-operator-index-27fzr" event={"ID":"42d5fccb-f24d-4707-8167-30eeae73378e","Type":"ContainerStarted","Data":"3ac16256a126a72b395395510628778e7f28a21d38ee102ef12dfc83d2615d96"} Dec 03 13:16:31 crc kubenswrapper[5052]: I1203 13:16:31.362760 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/barbican-operator-index-27fzr" event={"ID":"42d5fccb-f24d-4707-8167-30eeae73378e","Type":"ContainerStarted","Data":"049d108774c46990f3b5856dbd4a31bee08fe0b5f5aa400c1f16208b8fe4f161"} Dec 03 13:16:31 crc kubenswrapper[5052]: I1203 13:16:31.385915 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-operators/barbican-operator-index-27fzr" podStartSLOduration=2.46909027 podStartE2EDuration="3.385896026s" podCreationTimestamp="2025-12-03 13:16:28 +0000 UTC" firstStartedPulling="2025-12-03 13:16:29.636844756 +0000 UTC m=+6980.225086530" lastFinishedPulling="2025-12-03 13:16:30.553650522 +0000 UTC m=+6981.141892286" observedRunningTime="2025-12-03 13:16:31.378305418 +0000 UTC m=+6981.966547182" watchObservedRunningTime="2025-12-03 13:16:31.385896026 +0000 UTC m=+6981.974137790" Dec 03 13:16:39 crc kubenswrapper[5052]: I1203 13:16:39.188656 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-operators/barbican-operator-index-27fzr" Dec 03 13:16:39 crc kubenswrapper[5052]: I1203 13:16:39.189088 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-operators/barbican-operator-index-27fzr" Dec 03 13:16:39 crc kubenswrapper[5052]: I1203 13:16:39.221645 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-operators/barbican-operator-index-27fzr" Dec 03 13:16:39 crc kubenswrapper[5052]: I1203 13:16:39.450545 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-operators/barbican-operator-index-27fzr" Dec 03 13:16:42 crc kubenswrapper[5052]: I1203 13:16:42.178721 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-operators/ee8a7e2925b38108919a6c4a37131c62bef9ed0b2003ea2b2120dc9476kfhs5"] Dec 03 13:16:42 crc kubenswrapper[5052]: I1203 13:16:42.180572 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-operators/ee8a7e2925b38108919a6c4a37131c62bef9ed0b2003ea2b2120dc9476kfhs5" Dec 03 13:16:42 crc kubenswrapper[5052]: I1203 13:16:42.182932 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-operators"/"default-dockercfg-7fklr" Dec 03 13:16:42 crc kubenswrapper[5052]: I1203 13:16:42.203309 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-operators/ee8a7e2925b38108919a6c4a37131c62bef9ed0b2003ea2b2120dc9476kfhs5"] Dec 03 13:16:42 crc kubenswrapper[5052]: I1203 13:16:42.271980 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b7eb0ffc-ed97-4e2e-aa46-5959beae46aa-bundle\") pod \"ee8a7e2925b38108919a6c4a37131c62bef9ed0b2003ea2b2120dc9476kfhs5\" (UID: \"b7eb0ffc-ed97-4e2e-aa46-5959beae46aa\") " pod="openstack-operators/ee8a7e2925b38108919a6c4a37131c62bef9ed0b2003ea2b2120dc9476kfhs5" Dec 03 13:16:42 crc kubenswrapper[5052]: I1203 13:16:42.272068 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b7eb0ffc-ed97-4e2e-aa46-5959beae46aa-util\") pod \"ee8a7e2925b38108919a6c4a37131c62bef9ed0b2003ea2b2120dc9476kfhs5\" (UID: \"b7eb0ffc-ed97-4e2e-aa46-5959beae46aa\") " pod="openstack-operators/ee8a7e2925b38108919a6c4a37131c62bef9ed0b2003ea2b2120dc9476kfhs5" Dec 03 13:16:42 crc kubenswrapper[5052]: I1203 13:16:42.272133 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-smll8\" (UniqueName: \"kubernetes.io/projected/b7eb0ffc-ed97-4e2e-aa46-5959beae46aa-kube-api-access-smll8\") pod \"ee8a7e2925b38108919a6c4a37131c62bef9ed0b2003ea2b2120dc9476kfhs5\" (UID: \"b7eb0ffc-ed97-4e2e-aa46-5959beae46aa\") " pod="openstack-operators/ee8a7e2925b38108919a6c4a37131c62bef9ed0b2003ea2b2120dc9476kfhs5" Dec 03 13:16:42 crc kubenswrapper[5052]: I1203 13:16:42.373052 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b7eb0ffc-ed97-4e2e-aa46-5959beae46aa-bundle\") pod \"ee8a7e2925b38108919a6c4a37131c62bef9ed0b2003ea2b2120dc9476kfhs5\" (UID: \"b7eb0ffc-ed97-4e2e-aa46-5959beae46aa\") " pod="openstack-operators/ee8a7e2925b38108919a6c4a37131c62bef9ed0b2003ea2b2120dc9476kfhs5" Dec 03 13:16:42 crc kubenswrapper[5052]: I1203 13:16:42.373127 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b7eb0ffc-ed97-4e2e-aa46-5959beae46aa-util\") pod \"ee8a7e2925b38108919a6c4a37131c62bef9ed0b2003ea2b2120dc9476kfhs5\" (UID: \"b7eb0ffc-ed97-4e2e-aa46-5959beae46aa\") " pod="openstack-operators/ee8a7e2925b38108919a6c4a37131c62bef9ed0b2003ea2b2120dc9476kfhs5" Dec 03 13:16:42 crc kubenswrapper[5052]: I1203 13:16:42.373200 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-smll8\" (UniqueName: \"kubernetes.io/projected/b7eb0ffc-ed97-4e2e-aa46-5959beae46aa-kube-api-access-smll8\") pod \"ee8a7e2925b38108919a6c4a37131c62bef9ed0b2003ea2b2120dc9476kfhs5\" (UID: \"b7eb0ffc-ed97-4e2e-aa46-5959beae46aa\") " pod="openstack-operators/ee8a7e2925b38108919a6c4a37131c62bef9ed0b2003ea2b2120dc9476kfhs5" Dec 03 13:16:42 crc kubenswrapper[5052]: I1203 13:16:42.373712 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b7eb0ffc-ed97-4e2e-aa46-5959beae46aa-bundle\") pod \"ee8a7e2925b38108919a6c4a37131c62bef9ed0b2003ea2b2120dc9476kfhs5\" (UID: \"b7eb0ffc-ed97-4e2e-aa46-5959beae46aa\") " pod="openstack-operators/ee8a7e2925b38108919a6c4a37131c62bef9ed0b2003ea2b2120dc9476kfhs5" Dec 03 13:16:42 crc kubenswrapper[5052]: I1203 13:16:42.373748 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b7eb0ffc-ed97-4e2e-aa46-5959beae46aa-util\") pod \"ee8a7e2925b38108919a6c4a37131c62bef9ed0b2003ea2b2120dc9476kfhs5\" (UID: \"b7eb0ffc-ed97-4e2e-aa46-5959beae46aa\") " pod="openstack-operators/ee8a7e2925b38108919a6c4a37131c62bef9ed0b2003ea2b2120dc9476kfhs5" Dec 03 13:16:42 crc kubenswrapper[5052]: I1203 13:16:42.392199 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-smll8\" (UniqueName: \"kubernetes.io/projected/b7eb0ffc-ed97-4e2e-aa46-5959beae46aa-kube-api-access-smll8\") pod \"ee8a7e2925b38108919a6c4a37131c62bef9ed0b2003ea2b2120dc9476kfhs5\" (UID: \"b7eb0ffc-ed97-4e2e-aa46-5959beae46aa\") " pod="openstack-operators/ee8a7e2925b38108919a6c4a37131c62bef9ed0b2003ea2b2120dc9476kfhs5" Dec 03 13:16:42 crc kubenswrapper[5052]: I1203 13:16:42.511939 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-operators/ee8a7e2925b38108919a6c4a37131c62bef9ed0b2003ea2b2120dc9476kfhs5" Dec 03 13:16:42 crc kubenswrapper[5052]: I1203 13:16:42.929990 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-operators/ee8a7e2925b38108919a6c4a37131c62bef9ed0b2003ea2b2120dc9476kfhs5"] Dec 03 13:16:43 crc kubenswrapper[5052]: I1203 13:16:43.459351 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/ee8a7e2925b38108919a6c4a37131c62bef9ed0b2003ea2b2120dc9476kfhs5" event={"ID":"b7eb0ffc-ed97-4e2e-aa46-5959beae46aa","Type":"ContainerStarted","Data":"e9aed9fe7507a94a5809c8dd6af846f9f2d041cdcaa300fb732127399c90fe77"} Dec 03 13:16:44 crc kubenswrapper[5052]: I1203 13:16:44.467044 5052 generic.go:334] "Generic (PLEG): container finished" podID="caf5c2cd-0156-4996-b20a-e5b5b9255ddc" containerID="78245ad00f303fd209e0f5933535394975e3fb1bf7c4663687be9ecf89b8800a" exitCode=0 Dec 03 13:16:44 crc kubenswrapper[5052]: I1203 13:16:44.467503 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="barbican-kuttl-tests/rabbitmq-server-0" event={"ID":"caf5c2cd-0156-4996-b20a-e5b5b9255ddc","Type":"ContainerDied","Data":"78245ad00f303fd209e0f5933535394975e3fb1bf7c4663687be9ecf89b8800a"} Dec 03 13:16:44 crc kubenswrapper[5052]: I1203 13:16:44.472766 5052 generic.go:334] "Generic (PLEG): container finished" podID="b7eb0ffc-ed97-4e2e-aa46-5959beae46aa" containerID="88b36946effde7a166f55150a328e073b1f02cf790e86bea202520684cdaf41c" exitCode=0 Dec 03 13:16:44 crc kubenswrapper[5052]: I1203 13:16:44.472799 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/ee8a7e2925b38108919a6c4a37131c62bef9ed0b2003ea2b2120dc9476kfhs5" event={"ID":"b7eb0ffc-ed97-4e2e-aa46-5959beae46aa","Type":"ContainerDied","Data":"88b36946effde7a166f55150a328e073b1f02cf790e86bea202520684cdaf41c"} Dec 03 13:16:45 crc kubenswrapper[5052]: I1203 13:16:45.482400 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="barbican-kuttl-tests/rabbitmq-server-0" event={"ID":"caf5c2cd-0156-4996-b20a-e5b5b9255ddc","Type":"ContainerStarted","Data":"121a8279cdacc589caa6d7117b9d018c151fe58e70cc62342f8278ecd590fbce"} Dec 03 13:16:45 crc kubenswrapper[5052]: I1203 13:16:45.483607 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="barbican-kuttl-tests/rabbitmq-server-0" Dec 03 13:16:46 crc kubenswrapper[5052]: I1203 13:16:46.125953 5052 scope.go:117] "RemoveContainer" containerID="9b097a226f82480c23c807d5b63ec486754d4f12840b676166e91363220ac17c" Dec 03 13:16:46 crc kubenswrapper[5052]: I1203 13:16:46.144400 5052 scope.go:117] "RemoveContainer" containerID="40acab817908dc89ffbb3d9a51b07484f7a5fbb642a88c1eb9f7403ecd646051" Dec 03 13:16:46 crc kubenswrapper[5052]: I1203 13:16:46.199053 5052 scope.go:117] "RemoveContainer" containerID="8d155c44a415d1221f170deb5a3083c00cf41d47f75364b5404f3376474ddcfb" Dec 03 13:16:46 crc kubenswrapper[5052]: I1203 13:16:46.492370 5052 generic.go:334] "Generic (PLEG): container finished" podID="b7eb0ffc-ed97-4e2e-aa46-5959beae46aa" containerID="969b3a09d6c12b5b9cee46d7bed7eabe1e6cdfab75e304e9f87fa15d482c3ea0" exitCode=0 Dec 03 13:16:46 crc kubenswrapper[5052]: I1203 13:16:46.492484 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/ee8a7e2925b38108919a6c4a37131c62bef9ed0b2003ea2b2120dc9476kfhs5" event={"ID":"b7eb0ffc-ed97-4e2e-aa46-5959beae46aa","Type":"ContainerDied","Data":"969b3a09d6c12b5b9cee46d7bed7eabe1e6cdfab75e304e9f87fa15d482c3ea0"} Dec 03 13:16:46 crc kubenswrapper[5052]: I1203 13:16:46.519761 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="barbican-kuttl-tests/rabbitmq-server-0" podStartSLOduration=38.854521654 podStartE2EDuration="48.519742316s" podCreationTimestamp="2025-12-03 13:15:58 +0000 UTC" firstStartedPulling="2025-12-03 13:16:00.256802969 +0000 UTC m=+6950.845044723" lastFinishedPulling="2025-12-03 13:16:09.922023501 +0000 UTC m=+6960.510265385" observedRunningTime="2025-12-03 13:16:45.516722711 +0000 UTC m=+6996.104964485" watchObservedRunningTime="2025-12-03 13:16:46.519742316 +0000 UTC m=+6997.107984080" Dec 03 13:16:47 crc kubenswrapper[5052]: I1203 13:16:47.248847 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["barbican-kuttl-tests/keystone-44d2-account-create-update-vczdq"] Dec 03 13:16:47 crc kubenswrapper[5052]: I1203 13:16:47.249834 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="barbican-kuttl-tests/keystone-44d2-account-create-update-vczdq" Dec 03 13:16:47 crc kubenswrapper[5052]: I1203 13:16:47.251662 5052 reflector.go:368] Caches populated for *v1.Secret from object-"barbican-kuttl-tests"/"keystone-db-secret" Dec 03 13:16:47 crc kubenswrapper[5052]: I1203 13:16:47.261009 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["barbican-kuttl-tests/keystone-44d2-account-create-update-vczdq"] Dec 03 13:16:47 crc kubenswrapper[5052]: I1203 13:16:47.339335 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["barbican-kuttl-tests/keystone-db-create-rhh8p"] Dec 03 13:16:47 crc kubenswrapper[5052]: I1203 13:16:47.340479 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="barbican-kuttl-tests/keystone-db-create-rhh8p" Dec 03 13:16:47 crc kubenswrapper[5052]: I1203 13:16:47.348261 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/1ab950dd-0324-4c92-a979-dae6cd28feb4-operator-scripts\") pod \"keystone-44d2-account-create-update-vczdq\" (UID: \"1ab950dd-0324-4c92-a979-dae6cd28feb4\") " pod="barbican-kuttl-tests/keystone-44d2-account-create-update-vczdq" Dec 03 13:16:47 crc kubenswrapper[5052]: I1203 13:16:47.348357 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5bpd8\" (UniqueName: \"kubernetes.io/projected/1ab950dd-0324-4c92-a979-dae6cd28feb4-kube-api-access-5bpd8\") pod \"keystone-44d2-account-create-update-vczdq\" (UID: \"1ab950dd-0324-4c92-a979-dae6cd28feb4\") " pod="barbican-kuttl-tests/keystone-44d2-account-create-update-vczdq" Dec 03 13:16:47 crc kubenswrapper[5052]: I1203 13:16:47.349450 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["barbican-kuttl-tests/keystone-db-create-rhh8p"] Dec 03 13:16:47 crc kubenswrapper[5052]: I1203 13:16:47.449745 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/1ab950dd-0324-4c92-a979-dae6cd28feb4-operator-scripts\") pod \"keystone-44d2-account-create-update-vczdq\" (UID: \"1ab950dd-0324-4c92-a979-dae6cd28feb4\") " pod="barbican-kuttl-tests/keystone-44d2-account-create-update-vczdq" Dec 03 13:16:47 crc kubenswrapper[5052]: I1203 13:16:47.449819 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vh6gm\" (UniqueName: \"kubernetes.io/projected/e6a63dba-1ce4-452f-bc4d-7c4e8a34bf7d-kube-api-access-vh6gm\") pod \"keystone-db-create-rhh8p\" (UID: \"e6a63dba-1ce4-452f-bc4d-7c4e8a34bf7d\") " pod="barbican-kuttl-tests/keystone-db-create-rhh8p" Dec 03 13:16:47 crc kubenswrapper[5052]: I1203 13:16:47.449883 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/e6a63dba-1ce4-452f-bc4d-7c4e8a34bf7d-operator-scripts\") pod \"keystone-db-create-rhh8p\" (UID: \"e6a63dba-1ce4-452f-bc4d-7c4e8a34bf7d\") " pod="barbican-kuttl-tests/keystone-db-create-rhh8p" Dec 03 13:16:47 crc kubenswrapper[5052]: I1203 13:16:47.449950 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-5bpd8\" (UniqueName: \"kubernetes.io/projected/1ab950dd-0324-4c92-a979-dae6cd28feb4-kube-api-access-5bpd8\") pod \"keystone-44d2-account-create-update-vczdq\" (UID: \"1ab950dd-0324-4c92-a979-dae6cd28feb4\") " pod="barbican-kuttl-tests/keystone-44d2-account-create-update-vczdq" Dec 03 13:16:47 crc kubenswrapper[5052]: I1203 13:16:47.450645 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/1ab950dd-0324-4c92-a979-dae6cd28feb4-operator-scripts\") pod \"keystone-44d2-account-create-update-vczdq\" (UID: \"1ab950dd-0324-4c92-a979-dae6cd28feb4\") " pod="barbican-kuttl-tests/keystone-44d2-account-create-update-vczdq" Dec 03 13:16:47 crc kubenswrapper[5052]: I1203 13:16:47.469768 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-5bpd8\" (UniqueName: \"kubernetes.io/projected/1ab950dd-0324-4c92-a979-dae6cd28feb4-kube-api-access-5bpd8\") pod \"keystone-44d2-account-create-update-vczdq\" (UID: \"1ab950dd-0324-4c92-a979-dae6cd28feb4\") " pod="barbican-kuttl-tests/keystone-44d2-account-create-update-vczdq" Dec 03 13:16:47 crc kubenswrapper[5052]: I1203 13:16:47.502821 5052 generic.go:334] "Generic (PLEG): container finished" podID="b7eb0ffc-ed97-4e2e-aa46-5959beae46aa" containerID="dd50b8915948090dfae0c0215b7cda1b0f1f8beadca40b4106938fda7400ba5d" exitCode=0 Dec 03 13:16:47 crc kubenswrapper[5052]: I1203 13:16:47.502862 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/ee8a7e2925b38108919a6c4a37131c62bef9ed0b2003ea2b2120dc9476kfhs5" event={"ID":"b7eb0ffc-ed97-4e2e-aa46-5959beae46aa","Type":"ContainerDied","Data":"dd50b8915948090dfae0c0215b7cda1b0f1f8beadca40b4106938fda7400ba5d"} Dec 03 13:16:47 crc kubenswrapper[5052]: I1203 13:16:47.551150 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/e6a63dba-1ce4-452f-bc4d-7c4e8a34bf7d-operator-scripts\") pod \"keystone-db-create-rhh8p\" (UID: \"e6a63dba-1ce4-452f-bc4d-7c4e8a34bf7d\") " pod="barbican-kuttl-tests/keystone-db-create-rhh8p" Dec 03 13:16:47 crc kubenswrapper[5052]: I1203 13:16:47.551285 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-vh6gm\" (UniqueName: \"kubernetes.io/projected/e6a63dba-1ce4-452f-bc4d-7c4e8a34bf7d-kube-api-access-vh6gm\") pod \"keystone-db-create-rhh8p\" (UID: \"e6a63dba-1ce4-452f-bc4d-7c4e8a34bf7d\") " pod="barbican-kuttl-tests/keystone-db-create-rhh8p" Dec 03 13:16:47 crc kubenswrapper[5052]: I1203 13:16:47.552334 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/e6a63dba-1ce4-452f-bc4d-7c4e8a34bf7d-operator-scripts\") pod \"keystone-db-create-rhh8p\" (UID: \"e6a63dba-1ce4-452f-bc4d-7c4e8a34bf7d\") " pod="barbican-kuttl-tests/keystone-db-create-rhh8p" Dec 03 13:16:47 crc kubenswrapper[5052]: I1203 13:16:47.565257 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="barbican-kuttl-tests/keystone-44d2-account-create-update-vczdq" Dec 03 13:16:47 crc kubenswrapper[5052]: I1203 13:16:47.569690 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-vh6gm\" (UniqueName: \"kubernetes.io/projected/e6a63dba-1ce4-452f-bc4d-7c4e8a34bf7d-kube-api-access-vh6gm\") pod \"keystone-db-create-rhh8p\" (UID: \"e6a63dba-1ce4-452f-bc4d-7c4e8a34bf7d\") " pod="barbican-kuttl-tests/keystone-db-create-rhh8p" Dec 03 13:16:47 crc kubenswrapper[5052]: I1203 13:16:47.658078 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="barbican-kuttl-tests/keystone-db-create-rhh8p" Dec 03 13:16:47 crc kubenswrapper[5052]: I1203 13:16:47.985610 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["barbican-kuttl-tests/keystone-44d2-account-create-update-vczdq"] Dec 03 13:16:48 crc kubenswrapper[5052]: I1203 13:16:48.119101 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["barbican-kuttl-tests/keystone-db-create-rhh8p"] Dec 03 13:16:48 crc kubenswrapper[5052]: W1203 13:16:48.125942 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode6a63dba_1ce4_452f_bc4d_7c4e8a34bf7d.slice/crio-d85f5ab34dce7ce45c7feeb4ad1451c48ffe582bd2c801c3b60341a2c13d703d WatchSource:0}: Error finding container d85f5ab34dce7ce45c7feeb4ad1451c48ffe582bd2c801c3b60341a2c13d703d: Status 404 returned error can't find the container with id d85f5ab34dce7ce45c7feeb4ad1451c48ffe582bd2c801c3b60341a2c13d703d Dec 03 13:16:48 crc kubenswrapper[5052]: I1203 13:16:48.512023 5052 generic.go:334] "Generic (PLEG): container finished" podID="e6a63dba-1ce4-452f-bc4d-7c4e8a34bf7d" containerID="3ca414320048cba59bb177a28c61f8981b44327362df587c989d552ef06b54cf" exitCode=0 Dec 03 13:16:48 crc kubenswrapper[5052]: I1203 13:16:48.512083 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="barbican-kuttl-tests/keystone-db-create-rhh8p" event={"ID":"e6a63dba-1ce4-452f-bc4d-7c4e8a34bf7d","Type":"ContainerDied","Data":"3ca414320048cba59bb177a28c61f8981b44327362df587c989d552ef06b54cf"} Dec 03 13:16:48 crc kubenswrapper[5052]: I1203 13:16:48.512108 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="barbican-kuttl-tests/keystone-db-create-rhh8p" event={"ID":"e6a63dba-1ce4-452f-bc4d-7c4e8a34bf7d","Type":"ContainerStarted","Data":"d85f5ab34dce7ce45c7feeb4ad1451c48ffe582bd2c801c3b60341a2c13d703d"} Dec 03 13:16:48 crc kubenswrapper[5052]: I1203 13:16:48.514161 5052 generic.go:334] "Generic (PLEG): container finished" podID="1ab950dd-0324-4c92-a979-dae6cd28feb4" containerID="cd7ffff87a56771879dc7fa9b0357427929bcf93b368901d07fa90c727f4e1fc" exitCode=0 Dec 03 13:16:48 crc kubenswrapper[5052]: I1203 13:16:48.514382 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="barbican-kuttl-tests/keystone-44d2-account-create-update-vczdq" event={"ID":"1ab950dd-0324-4c92-a979-dae6cd28feb4","Type":"ContainerDied","Data":"cd7ffff87a56771879dc7fa9b0357427929bcf93b368901d07fa90c727f4e1fc"} Dec 03 13:16:48 crc kubenswrapper[5052]: I1203 13:16:48.514407 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="barbican-kuttl-tests/keystone-44d2-account-create-update-vczdq" event={"ID":"1ab950dd-0324-4c92-a979-dae6cd28feb4","Type":"ContainerStarted","Data":"20533ddfbf20e2f3609a684b03592081d3a21d97d56e65e2c7a22373d4567908"} Dec 03 13:16:48 crc kubenswrapper[5052]: I1203 13:16:48.786768 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-operators/ee8a7e2925b38108919a6c4a37131c62bef9ed0b2003ea2b2120dc9476kfhs5" Dec 03 13:16:48 crc kubenswrapper[5052]: I1203 13:16:48.870935 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b7eb0ffc-ed97-4e2e-aa46-5959beae46aa-bundle\") pod \"b7eb0ffc-ed97-4e2e-aa46-5959beae46aa\" (UID: \"b7eb0ffc-ed97-4e2e-aa46-5959beae46aa\") " Dec 03 13:16:48 crc kubenswrapper[5052]: I1203 13:16:48.871005 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b7eb0ffc-ed97-4e2e-aa46-5959beae46aa-util\") pod \"b7eb0ffc-ed97-4e2e-aa46-5959beae46aa\" (UID: \"b7eb0ffc-ed97-4e2e-aa46-5959beae46aa\") " Dec 03 13:16:48 crc kubenswrapper[5052]: I1203 13:16:48.871116 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-smll8\" (UniqueName: \"kubernetes.io/projected/b7eb0ffc-ed97-4e2e-aa46-5959beae46aa-kube-api-access-smll8\") pod \"b7eb0ffc-ed97-4e2e-aa46-5959beae46aa\" (UID: \"b7eb0ffc-ed97-4e2e-aa46-5959beae46aa\") " Dec 03 13:16:48 crc kubenswrapper[5052]: I1203 13:16:48.872197 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b7eb0ffc-ed97-4e2e-aa46-5959beae46aa-bundle" (OuterVolumeSpecName: "bundle") pod "b7eb0ffc-ed97-4e2e-aa46-5959beae46aa" (UID: "b7eb0ffc-ed97-4e2e-aa46-5959beae46aa"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 13:16:48 crc kubenswrapper[5052]: I1203 13:16:48.876904 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b7eb0ffc-ed97-4e2e-aa46-5959beae46aa-kube-api-access-smll8" (OuterVolumeSpecName: "kube-api-access-smll8") pod "b7eb0ffc-ed97-4e2e-aa46-5959beae46aa" (UID: "b7eb0ffc-ed97-4e2e-aa46-5959beae46aa"). InnerVolumeSpecName "kube-api-access-smll8". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:16:48 crc kubenswrapper[5052]: I1203 13:16:48.973193 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-smll8\" (UniqueName: \"kubernetes.io/projected/b7eb0ffc-ed97-4e2e-aa46-5959beae46aa-kube-api-access-smll8\") on node \"crc\" DevicePath \"\"" Dec 03 13:16:48 crc kubenswrapper[5052]: I1203 13:16:48.973568 5052 reconciler_common.go:293] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b7eb0ffc-ed97-4e2e-aa46-5959beae46aa-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 13:16:48 crc kubenswrapper[5052]: I1203 13:16:48.975276 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b7eb0ffc-ed97-4e2e-aa46-5959beae46aa-util" (OuterVolumeSpecName: "util") pod "b7eb0ffc-ed97-4e2e-aa46-5959beae46aa" (UID: "b7eb0ffc-ed97-4e2e-aa46-5959beae46aa"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 13:16:49 crc kubenswrapper[5052]: I1203 13:16:49.074862 5052 reconciler_common.go:293] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b7eb0ffc-ed97-4e2e-aa46-5959beae46aa-util\") on node \"crc\" DevicePath \"\"" Dec 03 13:16:49 crc kubenswrapper[5052]: I1203 13:16:49.523412 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-operators/ee8a7e2925b38108919a6c4a37131c62bef9ed0b2003ea2b2120dc9476kfhs5" Dec 03 13:16:49 crc kubenswrapper[5052]: I1203 13:16:49.523628 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/ee8a7e2925b38108919a6c4a37131c62bef9ed0b2003ea2b2120dc9476kfhs5" event={"ID":"b7eb0ffc-ed97-4e2e-aa46-5959beae46aa","Type":"ContainerDied","Data":"e9aed9fe7507a94a5809c8dd6af846f9f2d041cdcaa300fb732127399c90fe77"} Dec 03 13:16:49 crc kubenswrapper[5052]: I1203 13:16:49.523739 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e9aed9fe7507a94a5809c8dd6af846f9f2d041cdcaa300fb732127399c90fe77" Dec 03 13:16:49 crc kubenswrapper[5052]: I1203 13:16:49.818272 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="barbican-kuttl-tests/keystone-db-create-rhh8p" Dec 03 13:16:49 crc kubenswrapper[5052]: I1203 13:16:49.884410 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="barbican-kuttl-tests/keystone-44d2-account-create-update-vczdq" Dec 03 13:16:49 crc kubenswrapper[5052]: I1203 13:16:49.885476 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/e6a63dba-1ce4-452f-bc4d-7c4e8a34bf7d-operator-scripts\") pod \"e6a63dba-1ce4-452f-bc4d-7c4e8a34bf7d\" (UID: \"e6a63dba-1ce4-452f-bc4d-7c4e8a34bf7d\") " Dec 03 13:16:49 crc kubenswrapper[5052]: I1203 13:16:49.885582 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vh6gm\" (UniqueName: \"kubernetes.io/projected/e6a63dba-1ce4-452f-bc4d-7c4e8a34bf7d-kube-api-access-vh6gm\") pod \"e6a63dba-1ce4-452f-bc4d-7c4e8a34bf7d\" (UID: \"e6a63dba-1ce4-452f-bc4d-7c4e8a34bf7d\") " Dec 03 13:16:49 crc kubenswrapper[5052]: I1203 13:16:49.886434 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e6a63dba-1ce4-452f-bc4d-7c4e8a34bf7d-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "e6a63dba-1ce4-452f-bc4d-7c4e8a34bf7d" (UID: "e6a63dba-1ce4-452f-bc4d-7c4e8a34bf7d"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:16:49 crc kubenswrapper[5052]: I1203 13:16:49.889706 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e6a63dba-1ce4-452f-bc4d-7c4e8a34bf7d-kube-api-access-vh6gm" (OuterVolumeSpecName: "kube-api-access-vh6gm") pod "e6a63dba-1ce4-452f-bc4d-7c4e8a34bf7d" (UID: "e6a63dba-1ce4-452f-bc4d-7c4e8a34bf7d"). InnerVolumeSpecName "kube-api-access-vh6gm". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:16:49 crc kubenswrapper[5052]: I1203 13:16:49.987879 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5bpd8\" (UniqueName: \"kubernetes.io/projected/1ab950dd-0324-4c92-a979-dae6cd28feb4-kube-api-access-5bpd8\") pod \"1ab950dd-0324-4c92-a979-dae6cd28feb4\" (UID: \"1ab950dd-0324-4c92-a979-dae6cd28feb4\") " Dec 03 13:16:49 crc kubenswrapper[5052]: I1203 13:16:49.987981 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/1ab950dd-0324-4c92-a979-dae6cd28feb4-operator-scripts\") pod \"1ab950dd-0324-4c92-a979-dae6cd28feb4\" (UID: \"1ab950dd-0324-4c92-a979-dae6cd28feb4\") " Dec 03 13:16:49 crc kubenswrapper[5052]: I1203 13:16:49.988345 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/e6a63dba-1ce4-452f-bc4d-7c4e8a34bf7d-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 13:16:49 crc kubenswrapper[5052]: I1203 13:16:49.988364 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-vh6gm\" (UniqueName: \"kubernetes.io/projected/e6a63dba-1ce4-452f-bc4d-7c4e8a34bf7d-kube-api-access-vh6gm\") on node \"crc\" DevicePath \"\"" Dec 03 13:16:49 crc kubenswrapper[5052]: I1203 13:16:49.988597 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1ab950dd-0324-4c92-a979-dae6cd28feb4-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "1ab950dd-0324-4c92-a979-dae6cd28feb4" (UID: "1ab950dd-0324-4c92-a979-dae6cd28feb4"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:16:49 crc kubenswrapper[5052]: I1203 13:16:49.991184 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1ab950dd-0324-4c92-a979-dae6cd28feb4-kube-api-access-5bpd8" (OuterVolumeSpecName: "kube-api-access-5bpd8") pod "1ab950dd-0324-4c92-a979-dae6cd28feb4" (UID: "1ab950dd-0324-4c92-a979-dae6cd28feb4"). InnerVolumeSpecName "kube-api-access-5bpd8". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:16:50 crc kubenswrapper[5052]: I1203 13:16:50.090282 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-5bpd8\" (UniqueName: \"kubernetes.io/projected/1ab950dd-0324-4c92-a979-dae6cd28feb4-kube-api-access-5bpd8\") on node \"crc\" DevicePath \"\"" Dec 03 13:16:50 crc kubenswrapper[5052]: I1203 13:16:50.090536 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/1ab950dd-0324-4c92-a979-dae6cd28feb4-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 13:16:50 crc kubenswrapper[5052]: I1203 13:16:50.537367 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="barbican-kuttl-tests/keystone-44d2-account-create-update-vczdq" event={"ID":"1ab950dd-0324-4c92-a979-dae6cd28feb4","Type":"ContainerDied","Data":"20533ddfbf20e2f3609a684b03592081d3a21d97d56e65e2c7a22373d4567908"} Dec 03 13:16:50 crc kubenswrapper[5052]: I1203 13:16:50.537407 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="20533ddfbf20e2f3609a684b03592081d3a21d97d56e65e2c7a22373d4567908" Dec 03 13:16:50 crc kubenswrapper[5052]: I1203 13:16:50.537459 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="barbican-kuttl-tests/keystone-44d2-account-create-update-vczdq" Dec 03 13:16:50 crc kubenswrapper[5052]: I1203 13:16:50.542259 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="barbican-kuttl-tests/keystone-db-create-rhh8p" event={"ID":"e6a63dba-1ce4-452f-bc4d-7c4e8a34bf7d","Type":"ContainerDied","Data":"d85f5ab34dce7ce45c7feeb4ad1451c48ffe582bd2c801c3b60341a2c13d703d"} Dec 03 13:16:50 crc kubenswrapper[5052]: I1203 13:16:50.542329 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d85f5ab34dce7ce45c7feeb4ad1451c48ffe582bd2c801c3b60341a2c13d703d" Dec 03 13:16:50 crc kubenswrapper[5052]: I1203 13:16:50.542415 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="barbican-kuttl-tests/keystone-db-create-rhh8p" Dec 03 13:16:59 crc kubenswrapper[5052]: I1203 13:16:59.845279 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="barbican-kuttl-tests/rabbitmq-server-0" Dec 03 13:17:00 crc kubenswrapper[5052]: I1203 13:17:00.430494 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["barbican-kuttl-tests/keystone-db-sync-mvw4c"] Dec 03 13:17:00 crc kubenswrapper[5052]: E1203 13:17:00.430831 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="b7eb0ffc-ed97-4e2e-aa46-5959beae46aa" containerName="extract" Dec 03 13:17:00 crc kubenswrapper[5052]: I1203 13:17:00.430851 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="b7eb0ffc-ed97-4e2e-aa46-5959beae46aa" containerName="extract" Dec 03 13:17:00 crc kubenswrapper[5052]: E1203 13:17:00.430875 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="b7eb0ffc-ed97-4e2e-aa46-5959beae46aa" containerName="util" Dec 03 13:17:00 crc kubenswrapper[5052]: I1203 13:17:00.430884 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="b7eb0ffc-ed97-4e2e-aa46-5959beae46aa" containerName="util" Dec 03 13:17:00 crc kubenswrapper[5052]: E1203 13:17:00.430896 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="b7eb0ffc-ed97-4e2e-aa46-5959beae46aa" containerName="pull" Dec 03 13:17:00 crc kubenswrapper[5052]: I1203 13:17:00.430903 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="b7eb0ffc-ed97-4e2e-aa46-5959beae46aa" containerName="pull" Dec 03 13:17:00 crc kubenswrapper[5052]: E1203 13:17:00.430913 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="e6a63dba-1ce4-452f-bc4d-7c4e8a34bf7d" containerName="mariadb-database-create" Dec 03 13:17:00 crc kubenswrapper[5052]: I1203 13:17:00.430922 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="e6a63dba-1ce4-452f-bc4d-7c4e8a34bf7d" containerName="mariadb-database-create" Dec 03 13:17:00 crc kubenswrapper[5052]: E1203 13:17:00.430941 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="1ab950dd-0324-4c92-a979-dae6cd28feb4" containerName="mariadb-account-create-update" Dec 03 13:17:00 crc kubenswrapper[5052]: I1203 13:17:00.430948 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="1ab950dd-0324-4c92-a979-dae6cd28feb4" containerName="mariadb-account-create-update" Dec 03 13:17:00 crc kubenswrapper[5052]: I1203 13:17:00.431108 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="1ab950dd-0324-4c92-a979-dae6cd28feb4" containerName="mariadb-account-create-update" Dec 03 13:17:00 crc kubenswrapper[5052]: I1203 13:17:00.431139 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="b7eb0ffc-ed97-4e2e-aa46-5959beae46aa" containerName="extract" Dec 03 13:17:00 crc kubenswrapper[5052]: I1203 13:17:00.431155 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="e6a63dba-1ce4-452f-bc4d-7c4e8a34bf7d" containerName="mariadb-database-create" Dec 03 13:17:00 crc kubenswrapper[5052]: I1203 13:17:00.431739 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="barbican-kuttl-tests/keystone-db-sync-mvw4c" Dec 03 13:17:00 crc kubenswrapper[5052]: I1203 13:17:00.434237 5052 reflector.go:368] Caches populated for *v1.Secret from object-"barbican-kuttl-tests"/"keystone-scripts" Dec 03 13:17:00 crc kubenswrapper[5052]: I1203 13:17:00.435959 5052 reflector.go:368] Caches populated for *v1.Secret from object-"barbican-kuttl-tests"/"keystone" Dec 03 13:17:00 crc kubenswrapper[5052]: I1203 13:17:00.436259 5052 reflector.go:368] Caches populated for *v1.Secret from object-"barbican-kuttl-tests"/"keystone-keystone-dockercfg-rxpzr" Dec 03 13:17:00 crc kubenswrapper[5052]: I1203 13:17:00.444270 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["barbican-kuttl-tests/keystone-db-sync-mvw4c"] Dec 03 13:17:00 crc kubenswrapper[5052]: I1203 13:17:00.444301 5052 reflector.go:368] Caches populated for *v1.Secret from object-"barbican-kuttl-tests"/"keystone-config-data" Dec 03 13:17:00 crc kubenswrapper[5052]: I1203 13:17:00.543927 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/27911c95-b1e3-4d8c-8bca-03d8cc76e76b-config-data\") pod \"keystone-db-sync-mvw4c\" (UID: \"27911c95-b1e3-4d8c-8bca-03d8cc76e76b\") " pod="barbican-kuttl-tests/keystone-db-sync-mvw4c" Dec 03 13:17:00 crc kubenswrapper[5052]: I1203 13:17:00.544052 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qwzgk\" (UniqueName: \"kubernetes.io/projected/27911c95-b1e3-4d8c-8bca-03d8cc76e76b-kube-api-access-qwzgk\") pod \"keystone-db-sync-mvw4c\" (UID: \"27911c95-b1e3-4d8c-8bca-03d8cc76e76b\") " pod="barbican-kuttl-tests/keystone-db-sync-mvw4c" Dec 03 13:17:00 crc kubenswrapper[5052]: I1203 13:17:00.645308 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/27911c95-b1e3-4d8c-8bca-03d8cc76e76b-config-data\") pod \"keystone-db-sync-mvw4c\" (UID: \"27911c95-b1e3-4d8c-8bca-03d8cc76e76b\") " pod="barbican-kuttl-tests/keystone-db-sync-mvw4c" Dec 03 13:17:00 crc kubenswrapper[5052]: I1203 13:17:00.645434 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-qwzgk\" (UniqueName: \"kubernetes.io/projected/27911c95-b1e3-4d8c-8bca-03d8cc76e76b-kube-api-access-qwzgk\") pod \"keystone-db-sync-mvw4c\" (UID: \"27911c95-b1e3-4d8c-8bca-03d8cc76e76b\") " pod="barbican-kuttl-tests/keystone-db-sync-mvw4c" Dec 03 13:17:00 crc kubenswrapper[5052]: I1203 13:17:00.652502 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/27911c95-b1e3-4d8c-8bca-03d8cc76e76b-config-data\") pod \"keystone-db-sync-mvw4c\" (UID: \"27911c95-b1e3-4d8c-8bca-03d8cc76e76b\") " pod="barbican-kuttl-tests/keystone-db-sync-mvw4c" Dec 03 13:17:00 crc kubenswrapper[5052]: I1203 13:17:00.670725 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-qwzgk\" (UniqueName: \"kubernetes.io/projected/27911c95-b1e3-4d8c-8bca-03d8cc76e76b-kube-api-access-qwzgk\") pod \"keystone-db-sync-mvw4c\" (UID: \"27911c95-b1e3-4d8c-8bca-03d8cc76e76b\") " pod="barbican-kuttl-tests/keystone-db-sync-mvw4c" Dec 03 13:17:00 crc kubenswrapper[5052]: I1203 13:17:00.752138 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="barbican-kuttl-tests/keystone-db-sync-mvw4c" Dec 03 13:17:01 crc kubenswrapper[5052]: I1203 13:17:01.174868 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["barbican-kuttl-tests/keystone-db-sync-mvw4c"] Dec 03 13:17:01 crc kubenswrapper[5052]: W1203 13:17:01.181634 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod27911c95_b1e3_4d8c_8bca_03d8cc76e76b.slice/crio-8b8e37093d0a3242aaf31ed612d843bb8893c52ad6174a9b574e32dd01e5e307 WatchSource:0}: Error finding container 8b8e37093d0a3242aaf31ed612d843bb8893c52ad6174a9b574e32dd01e5e307: Status 404 returned error can't find the container with id 8b8e37093d0a3242aaf31ed612d843bb8893c52ad6174a9b574e32dd01e5e307 Dec 03 13:17:01 crc kubenswrapper[5052]: I1203 13:17:01.628010 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="barbican-kuttl-tests/keystone-db-sync-mvw4c" event={"ID":"27911c95-b1e3-4d8c-8bca-03d8cc76e76b","Type":"ContainerStarted","Data":"8b8e37093d0a3242aaf31ed612d843bb8893c52ad6174a9b574e32dd01e5e307"} Dec 03 13:17:01 crc kubenswrapper[5052]: I1203 13:17:01.719795 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-operators/barbican-operator-controller-manager-6bfcb8b657-bbhsb"] Dec 03 13:17:01 crc kubenswrapper[5052]: I1203 13:17:01.720923 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-operators/barbican-operator-controller-manager-6bfcb8b657-bbhsb" Dec 03 13:17:01 crc kubenswrapper[5052]: I1203 13:17:01.726853 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-operators"/"barbican-operator-controller-manager-dockercfg-zx486" Dec 03 13:17:01 crc kubenswrapper[5052]: I1203 13:17:01.726895 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-operators"/"barbican-operator-controller-manager-service-cert" Dec 03 13:17:01 crc kubenswrapper[5052]: I1203 13:17:01.751193 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-operators/barbican-operator-controller-manager-6bfcb8b657-bbhsb"] Dec 03 13:17:01 crc kubenswrapper[5052]: I1203 13:17:01.865186 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/dd5d90ba-19d0-40a0-91ef-71f560226194-apiservice-cert\") pod \"barbican-operator-controller-manager-6bfcb8b657-bbhsb\" (UID: \"dd5d90ba-19d0-40a0-91ef-71f560226194\") " pod="openstack-operators/barbican-operator-controller-manager-6bfcb8b657-bbhsb" Dec 03 13:17:01 crc kubenswrapper[5052]: I1203 13:17:01.865585 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vvk49\" (UniqueName: \"kubernetes.io/projected/dd5d90ba-19d0-40a0-91ef-71f560226194-kube-api-access-vvk49\") pod \"barbican-operator-controller-manager-6bfcb8b657-bbhsb\" (UID: \"dd5d90ba-19d0-40a0-91ef-71f560226194\") " pod="openstack-operators/barbican-operator-controller-manager-6bfcb8b657-bbhsb" Dec 03 13:17:01 crc kubenswrapper[5052]: I1203 13:17:01.865656 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/dd5d90ba-19d0-40a0-91ef-71f560226194-webhook-cert\") pod \"barbican-operator-controller-manager-6bfcb8b657-bbhsb\" (UID: \"dd5d90ba-19d0-40a0-91ef-71f560226194\") " pod="openstack-operators/barbican-operator-controller-manager-6bfcb8b657-bbhsb" Dec 03 13:17:01 crc kubenswrapper[5052]: I1203 13:17:01.967382 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/dd5d90ba-19d0-40a0-91ef-71f560226194-apiservice-cert\") pod \"barbican-operator-controller-manager-6bfcb8b657-bbhsb\" (UID: \"dd5d90ba-19d0-40a0-91ef-71f560226194\") " pod="openstack-operators/barbican-operator-controller-manager-6bfcb8b657-bbhsb" Dec 03 13:17:01 crc kubenswrapper[5052]: I1203 13:17:01.967631 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-vvk49\" (UniqueName: \"kubernetes.io/projected/dd5d90ba-19d0-40a0-91ef-71f560226194-kube-api-access-vvk49\") pod \"barbican-operator-controller-manager-6bfcb8b657-bbhsb\" (UID: \"dd5d90ba-19d0-40a0-91ef-71f560226194\") " pod="openstack-operators/barbican-operator-controller-manager-6bfcb8b657-bbhsb" Dec 03 13:17:01 crc kubenswrapper[5052]: I1203 13:17:01.967961 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/dd5d90ba-19d0-40a0-91ef-71f560226194-webhook-cert\") pod \"barbican-operator-controller-manager-6bfcb8b657-bbhsb\" (UID: \"dd5d90ba-19d0-40a0-91ef-71f560226194\") " pod="openstack-operators/barbican-operator-controller-manager-6bfcb8b657-bbhsb" Dec 03 13:17:01 crc kubenswrapper[5052]: I1203 13:17:01.981203 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/dd5d90ba-19d0-40a0-91ef-71f560226194-webhook-cert\") pod \"barbican-operator-controller-manager-6bfcb8b657-bbhsb\" (UID: \"dd5d90ba-19d0-40a0-91ef-71f560226194\") " pod="openstack-operators/barbican-operator-controller-manager-6bfcb8b657-bbhsb" Dec 03 13:17:01 crc kubenswrapper[5052]: I1203 13:17:01.988241 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/dd5d90ba-19d0-40a0-91ef-71f560226194-apiservice-cert\") pod \"barbican-operator-controller-manager-6bfcb8b657-bbhsb\" (UID: \"dd5d90ba-19d0-40a0-91ef-71f560226194\") " pod="openstack-operators/barbican-operator-controller-manager-6bfcb8b657-bbhsb" Dec 03 13:17:01 crc kubenswrapper[5052]: I1203 13:17:01.994291 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-vvk49\" (UniqueName: \"kubernetes.io/projected/dd5d90ba-19d0-40a0-91ef-71f560226194-kube-api-access-vvk49\") pod \"barbican-operator-controller-manager-6bfcb8b657-bbhsb\" (UID: \"dd5d90ba-19d0-40a0-91ef-71f560226194\") " pod="openstack-operators/barbican-operator-controller-manager-6bfcb8b657-bbhsb" Dec 03 13:17:02 crc kubenswrapper[5052]: I1203 13:17:02.049207 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-operators/barbican-operator-controller-manager-6bfcb8b657-bbhsb" Dec 03 13:17:02 crc kubenswrapper[5052]: I1203 13:17:02.368220 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-operators/barbican-operator-controller-manager-6bfcb8b657-bbhsb"] Dec 03 13:17:02 crc kubenswrapper[5052]: W1203 13:17:02.370779 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd5d90ba_19d0_40a0_91ef_71f560226194.slice/crio-e79f20138944a3f8e14ea26127e60f49917f580d1a38a0deb101c46903325d62 WatchSource:0}: Error finding container e79f20138944a3f8e14ea26127e60f49917f580d1a38a0deb101c46903325d62: Status 404 returned error can't find the container with id e79f20138944a3f8e14ea26127e60f49917f580d1a38a0deb101c46903325d62 Dec 03 13:17:02 crc kubenswrapper[5052]: I1203 13:17:02.637716 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="barbican-kuttl-tests/keystone-db-sync-mvw4c" event={"ID":"27911c95-b1e3-4d8c-8bca-03d8cc76e76b","Type":"ContainerStarted","Data":"6d6fee6ddf6dbb6b2db73045b174da8bbbf13b8aadb20644253355996e832c5d"} Dec 03 13:17:02 crc kubenswrapper[5052]: I1203 13:17:02.639633 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/barbican-operator-controller-manager-6bfcb8b657-bbhsb" event={"ID":"dd5d90ba-19d0-40a0-91ef-71f560226194","Type":"ContainerStarted","Data":"d0ecd6e9cce151a40a1c6f4c92dc2927b712f54d3838d50f77721c2a8fe0fbee"} Dec 03 13:17:02 crc kubenswrapper[5052]: I1203 13:17:02.639681 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/barbican-operator-controller-manager-6bfcb8b657-bbhsb" event={"ID":"dd5d90ba-19d0-40a0-91ef-71f560226194","Type":"ContainerStarted","Data":"e79f20138944a3f8e14ea26127e60f49917f580d1a38a0deb101c46903325d62"} Dec 03 13:17:02 crc kubenswrapper[5052]: I1203 13:17:02.653986 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="barbican-kuttl-tests/keystone-db-sync-mvw4c" podStartSLOduration=2.031463004 podStartE2EDuration="2.653969801s" podCreationTimestamp="2025-12-03 13:17:00 +0000 UTC" firstStartedPulling="2025-12-03 13:17:01.183652935 +0000 UTC m=+7011.771894699" lastFinishedPulling="2025-12-03 13:17:01.806159732 +0000 UTC m=+7012.394401496" observedRunningTime="2025-12-03 13:17:02.652150599 +0000 UTC m=+7013.240392363" watchObservedRunningTime="2025-12-03 13:17:02.653969801 +0000 UTC m=+7013.242211565" Dec 03 13:17:02 crc kubenswrapper[5052]: I1203 13:17:02.668764 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-operators/barbican-operator-controller-manager-6bfcb8b657-bbhsb" podStartSLOduration=1.668749015 podStartE2EDuration="1.668749015s" podCreationTimestamp="2025-12-03 13:17:01 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 13:17:02.66755068 +0000 UTC m=+7013.255792444" watchObservedRunningTime="2025-12-03 13:17:02.668749015 +0000 UTC m=+7013.256990769" Dec 03 13:17:03 crc kubenswrapper[5052]: I1203 13:17:03.646588 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-operators/barbican-operator-controller-manager-6bfcb8b657-bbhsb" Dec 03 13:17:09 crc kubenswrapper[5052]: I1203 13:17:09.688520 5052 generic.go:334] "Generic (PLEG): container finished" podID="27911c95-b1e3-4d8c-8bca-03d8cc76e76b" containerID="6d6fee6ddf6dbb6b2db73045b174da8bbbf13b8aadb20644253355996e832c5d" exitCode=0 Dec 03 13:17:09 crc kubenswrapper[5052]: I1203 13:17:09.688582 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="barbican-kuttl-tests/keystone-db-sync-mvw4c" event={"ID":"27911c95-b1e3-4d8c-8bca-03d8cc76e76b","Type":"ContainerDied","Data":"6d6fee6ddf6dbb6b2db73045b174da8bbbf13b8aadb20644253355996e832c5d"} Dec 03 13:17:11 crc kubenswrapper[5052]: I1203 13:17:11.008212 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="barbican-kuttl-tests/keystone-db-sync-mvw4c" Dec 03 13:17:11 crc kubenswrapper[5052]: I1203 13:17:11.106502 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qwzgk\" (UniqueName: \"kubernetes.io/projected/27911c95-b1e3-4d8c-8bca-03d8cc76e76b-kube-api-access-qwzgk\") pod \"27911c95-b1e3-4d8c-8bca-03d8cc76e76b\" (UID: \"27911c95-b1e3-4d8c-8bca-03d8cc76e76b\") " Dec 03 13:17:11 crc kubenswrapper[5052]: I1203 13:17:11.106708 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/27911c95-b1e3-4d8c-8bca-03d8cc76e76b-config-data\") pod \"27911c95-b1e3-4d8c-8bca-03d8cc76e76b\" (UID: \"27911c95-b1e3-4d8c-8bca-03d8cc76e76b\") " Dec 03 13:17:11 crc kubenswrapper[5052]: I1203 13:17:11.113605 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/27911c95-b1e3-4d8c-8bca-03d8cc76e76b-kube-api-access-qwzgk" (OuterVolumeSpecName: "kube-api-access-qwzgk") pod "27911c95-b1e3-4d8c-8bca-03d8cc76e76b" (UID: "27911c95-b1e3-4d8c-8bca-03d8cc76e76b"). InnerVolumeSpecName "kube-api-access-qwzgk". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:17:11 crc kubenswrapper[5052]: I1203 13:17:11.159214 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/27911c95-b1e3-4d8c-8bca-03d8cc76e76b-config-data" (OuterVolumeSpecName: "config-data") pod "27911c95-b1e3-4d8c-8bca-03d8cc76e76b" (UID: "27911c95-b1e3-4d8c-8bca-03d8cc76e76b"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:17:11 crc kubenswrapper[5052]: I1203 13:17:11.209441 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/27911c95-b1e3-4d8c-8bca-03d8cc76e76b-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 13:17:11 crc kubenswrapper[5052]: I1203 13:17:11.209696 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-qwzgk\" (UniqueName: \"kubernetes.io/projected/27911c95-b1e3-4d8c-8bca-03d8cc76e76b-kube-api-access-qwzgk\") on node \"crc\" DevicePath \"\"" Dec 03 13:17:11 crc kubenswrapper[5052]: I1203 13:17:11.704715 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="barbican-kuttl-tests/keystone-db-sync-mvw4c" event={"ID":"27911c95-b1e3-4d8c-8bca-03d8cc76e76b","Type":"ContainerDied","Data":"8b8e37093d0a3242aaf31ed612d843bb8893c52ad6174a9b574e32dd01e5e307"} Dec 03 13:17:11 crc kubenswrapper[5052]: I1203 13:17:11.705113 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8b8e37093d0a3242aaf31ed612d843bb8893c52ad6174a9b574e32dd01e5e307" Dec 03 13:17:11 crc kubenswrapper[5052]: I1203 13:17:11.704883 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="barbican-kuttl-tests/keystone-db-sync-mvw4c" Dec 03 13:17:11 crc kubenswrapper[5052]: I1203 13:17:11.888629 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["barbican-kuttl-tests/keystone-bootstrap-kqb5l"] Dec 03 13:17:11 crc kubenswrapper[5052]: E1203 13:17:11.889283 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="27911c95-b1e3-4d8c-8bca-03d8cc76e76b" containerName="keystone-db-sync" Dec 03 13:17:11 crc kubenswrapper[5052]: I1203 13:17:11.889303 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="27911c95-b1e3-4d8c-8bca-03d8cc76e76b" containerName="keystone-db-sync" Dec 03 13:17:11 crc kubenswrapper[5052]: I1203 13:17:11.890003 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="27911c95-b1e3-4d8c-8bca-03d8cc76e76b" containerName="keystone-db-sync" Dec 03 13:17:11 crc kubenswrapper[5052]: I1203 13:17:11.890981 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="barbican-kuttl-tests/keystone-bootstrap-kqb5l" Dec 03 13:17:11 crc kubenswrapper[5052]: I1203 13:17:11.894548 5052 reflector.go:368] Caches populated for *v1.Secret from object-"barbican-kuttl-tests"/"keystone" Dec 03 13:17:11 crc kubenswrapper[5052]: I1203 13:17:11.894630 5052 reflector.go:368] Caches populated for *v1.Secret from object-"barbican-kuttl-tests"/"keystone-keystone-dockercfg-rxpzr" Dec 03 13:17:11 crc kubenswrapper[5052]: I1203 13:17:11.894709 5052 reflector.go:368] Caches populated for *v1.Secret from object-"barbican-kuttl-tests"/"keystone-scripts" Dec 03 13:17:11 crc kubenswrapper[5052]: I1203 13:17:11.894732 5052 reflector.go:368] Caches populated for *v1.Secret from object-"barbican-kuttl-tests"/"osp-secret" Dec 03 13:17:11 crc kubenswrapper[5052]: I1203 13:17:11.894870 5052 reflector.go:368] Caches populated for *v1.Secret from object-"barbican-kuttl-tests"/"keystone-config-data" Dec 03 13:17:11 crc kubenswrapper[5052]: I1203 13:17:11.904157 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["barbican-kuttl-tests/keystone-bootstrap-kqb5l"] Dec 03 13:17:12 crc kubenswrapper[5052]: I1203 13:17:12.023439 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/3d02d92e-1cee-476f-82ab-ceb335c1a247-fernet-keys\") pod \"keystone-bootstrap-kqb5l\" (UID: \"3d02d92e-1cee-476f-82ab-ceb335c1a247\") " pod="barbican-kuttl-tests/keystone-bootstrap-kqb5l" Dec 03 13:17:12 crc kubenswrapper[5052]: I1203 13:17:12.023493 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/3d02d92e-1cee-476f-82ab-ceb335c1a247-credential-keys\") pod \"keystone-bootstrap-kqb5l\" (UID: \"3d02d92e-1cee-476f-82ab-ceb335c1a247\") " pod="barbican-kuttl-tests/keystone-bootstrap-kqb5l" Dec 03 13:17:12 crc kubenswrapper[5052]: I1203 13:17:12.023565 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bfb2z\" (UniqueName: \"kubernetes.io/projected/3d02d92e-1cee-476f-82ab-ceb335c1a247-kube-api-access-bfb2z\") pod \"keystone-bootstrap-kqb5l\" (UID: \"3d02d92e-1cee-476f-82ab-ceb335c1a247\") " pod="barbican-kuttl-tests/keystone-bootstrap-kqb5l" Dec 03 13:17:12 crc kubenswrapper[5052]: I1203 13:17:12.023674 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/3d02d92e-1cee-476f-82ab-ceb335c1a247-config-data\") pod \"keystone-bootstrap-kqb5l\" (UID: \"3d02d92e-1cee-476f-82ab-ceb335c1a247\") " pod="barbican-kuttl-tests/keystone-bootstrap-kqb5l" Dec 03 13:17:12 crc kubenswrapper[5052]: I1203 13:17:12.023814 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/3d02d92e-1cee-476f-82ab-ceb335c1a247-scripts\") pod \"keystone-bootstrap-kqb5l\" (UID: \"3d02d92e-1cee-476f-82ab-ceb335c1a247\") " pod="barbican-kuttl-tests/keystone-bootstrap-kqb5l" Dec 03 13:17:12 crc kubenswrapper[5052]: I1203 13:17:12.055740 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-operators/barbican-operator-controller-manager-6bfcb8b657-bbhsb" Dec 03 13:17:12 crc kubenswrapper[5052]: I1203 13:17:12.129915 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/3d02d92e-1cee-476f-82ab-ceb335c1a247-config-data\") pod \"keystone-bootstrap-kqb5l\" (UID: \"3d02d92e-1cee-476f-82ab-ceb335c1a247\") " pod="barbican-kuttl-tests/keystone-bootstrap-kqb5l" Dec 03 13:17:12 crc kubenswrapper[5052]: I1203 13:17:12.129960 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/3d02d92e-1cee-476f-82ab-ceb335c1a247-scripts\") pod \"keystone-bootstrap-kqb5l\" (UID: \"3d02d92e-1cee-476f-82ab-ceb335c1a247\") " pod="barbican-kuttl-tests/keystone-bootstrap-kqb5l" Dec 03 13:17:12 crc kubenswrapper[5052]: I1203 13:17:12.130030 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/3d02d92e-1cee-476f-82ab-ceb335c1a247-fernet-keys\") pod \"keystone-bootstrap-kqb5l\" (UID: \"3d02d92e-1cee-476f-82ab-ceb335c1a247\") " pod="barbican-kuttl-tests/keystone-bootstrap-kqb5l" Dec 03 13:17:12 crc kubenswrapper[5052]: I1203 13:17:12.130054 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/3d02d92e-1cee-476f-82ab-ceb335c1a247-credential-keys\") pod \"keystone-bootstrap-kqb5l\" (UID: \"3d02d92e-1cee-476f-82ab-ceb335c1a247\") " pod="barbican-kuttl-tests/keystone-bootstrap-kqb5l" Dec 03 13:17:12 crc kubenswrapper[5052]: I1203 13:17:12.130100 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-bfb2z\" (UniqueName: \"kubernetes.io/projected/3d02d92e-1cee-476f-82ab-ceb335c1a247-kube-api-access-bfb2z\") pod \"keystone-bootstrap-kqb5l\" (UID: \"3d02d92e-1cee-476f-82ab-ceb335c1a247\") " pod="barbican-kuttl-tests/keystone-bootstrap-kqb5l" Dec 03 13:17:12 crc kubenswrapper[5052]: I1203 13:17:12.134813 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/3d02d92e-1cee-476f-82ab-ceb335c1a247-config-data\") pod \"keystone-bootstrap-kqb5l\" (UID: \"3d02d92e-1cee-476f-82ab-ceb335c1a247\") " pod="barbican-kuttl-tests/keystone-bootstrap-kqb5l" Dec 03 13:17:12 crc kubenswrapper[5052]: I1203 13:17:12.136905 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/3d02d92e-1cee-476f-82ab-ceb335c1a247-scripts\") pod \"keystone-bootstrap-kqb5l\" (UID: \"3d02d92e-1cee-476f-82ab-ceb335c1a247\") " pod="barbican-kuttl-tests/keystone-bootstrap-kqb5l" Dec 03 13:17:12 crc kubenswrapper[5052]: I1203 13:17:12.137006 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/3d02d92e-1cee-476f-82ab-ceb335c1a247-fernet-keys\") pod \"keystone-bootstrap-kqb5l\" (UID: \"3d02d92e-1cee-476f-82ab-ceb335c1a247\") " pod="barbican-kuttl-tests/keystone-bootstrap-kqb5l" Dec 03 13:17:12 crc kubenswrapper[5052]: I1203 13:17:12.137865 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/3d02d92e-1cee-476f-82ab-ceb335c1a247-credential-keys\") pod \"keystone-bootstrap-kqb5l\" (UID: \"3d02d92e-1cee-476f-82ab-ceb335c1a247\") " pod="barbican-kuttl-tests/keystone-bootstrap-kqb5l" Dec 03 13:17:12 crc kubenswrapper[5052]: I1203 13:17:12.150288 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-bfb2z\" (UniqueName: \"kubernetes.io/projected/3d02d92e-1cee-476f-82ab-ceb335c1a247-kube-api-access-bfb2z\") pod \"keystone-bootstrap-kqb5l\" (UID: \"3d02d92e-1cee-476f-82ab-ceb335c1a247\") " pod="barbican-kuttl-tests/keystone-bootstrap-kqb5l" Dec 03 13:17:12 crc kubenswrapper[5052]: I1203 13:17:12.211798 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="barbican-kuttl-tests/keystone-bootstrap-kqb5l" Dec 03 13:17:12 crc kubenswrapper[5052]: I1203 13:17:12.619299 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["barbican-kuttl-tests/keystone-bootstrap-kqb5l"] Dec 03 13:17:12 crc kubenswrapper[5052]: W1203 13:17:12.626025 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3d02d92e_1cee_476f_82ab_ceb335c1a247.slice/crio-2b3d34850a2c07733af6048fb5c8085b5bef3dab5ae6a6377cffca358c390c2a WatchSource:0}: Error finding container 2b3d34850a2c07733af6048fb5c8085b5bef3dab5ae6a6377cffca358c390c2a: Status 404 returned error can't find the container with id 2b3d34850a2c07733af6048fb5c8085b5bef3dab5ae6a6377cffca358c390c2a Dec 03 13:17:12 crc kubenswrapper[5052]: I1203 13:17:12.712957 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="barbican-kuttl-tests/keystone-bootstrap-kqb5l" event={"ID":"3d02d92e-1cee-476f-82ab-ceb335c1a247","Type":"ContainerStarted","Data":"2b3d34850a2c07733af6048fb5c8085b5bef3dab5ae6a6377cffca358c390c2a"} Dec 03 13:17:13 crc kubenswrapper[5052]: I1203 13:17:13.721960 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="barbican-kuttl-tests/keystone-bootstrap-kqb5l" event={"ID":"3d02d92e-1cee-476f-82ab-ceb335c1a247","Type":"ContainerStarted","Data":"3adb3ffb576dce9dbe075283d6512dbd645ee60e83ca7fc9eafde6d7cbb01e38"} Dec 03 13:17:13 crc kubenswrapper[5052]: I1203 13:17:13.750451 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="barbican-kuttl-tests/keystone-bootstrap-kqb5l" podStartSLOduration=2.750433869 podStartE2EDuration="2.750433869s" podCreationTimestamp="2025-12-03 13:17:11 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 13:17:13.749368388 +0000 UTC m=+7024.337610152" watchObservedRunningTime="2025-12-03 13:17:13.750433869 +0000 UTC m=+7024.338675633" Dec 03 13:17:17 crc kubenswrapper[5052]: I1203 13:17:17.748481 5052 generic.go:334] "Generic (PLEG): container finished" podID="3d02d92e-1cee-476f-82ab-ceb335c1a247" containerID="3adb3ffb576dce9dbe075283d6512dbd645ee60e83ca7fc9eafde6d7cbb01e38" exitCode=0 Dec 03 13:17:17 crc kubenswrapper[5052]: I1203 13:17:17.748565 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="barbican-kuttl-tests/keystone-bootstrap-kqb5l" event={"ID":"3d02d92e-1cee-476f-82ab-ceb335c1a247","Type":"ContainerDied","Data":"3adb3ffb576dce9dbe075283d6512dbd645ee60e83ca7fc9eafde6d7cbb01e38"} Dec 03 13:17:19 crc kubenswrapper[5052]: I1203 13:17:19.066884 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="barbican-kuttl-tests/keystone-bootstrap-kqb5l" Dec 03 13:17:19 crc kubenswrapper[5052]: I1203 13:17:19.241025 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/3d02d92e-1cee-476f-82ab-ceb335c1a247-credential-keys\") pod \"3d02d92e-1cee-476f-82ab-ceb335c1a247\" (UID: \"3d02d92e-1cee-476f-82ab-ceb335c1a247\") " Dec 03 13:17:19 crc kubenswrapper[5052]: I1203 13:17:19.241103 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/3d02d92e-1cee-476f-82ab-ceb335c1a247-scripts\") pod \"3d02d92e-1cee-476f-82ab-ceb335c1a247\" (UID: \"3d02d92e-1cee-476f-82ab-ceb335c1a247\") " Dec 03 13:17:19 crc kubenswrapper[5052]: I1203 13:17:19.241165 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/3d02d92e-1cee-476f-82ab-ceb335c1a247-fernet-keys\") pod \"3d02d92e-1cee-476f-82ab-ceb335c1a247\" (UID: \"3d02d92e-1cee-476f-82ab-ceb335c1a247\") " Dec 03 13:17:19 crc kubenswrapper[5052]: I1203 13:17:19.241209 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/3d02d92e-1cee-476f-82ab-ceb335c1a247-config-data\") pod \"3d02d92e-1cee-476f-82ab-ceb335c1a247\" (UID: \"3d02d92e-1cee-476f-82ab-ceb335c1a247\") " Dec 03 13:17:19 crc kubenswrapper[5052]: I1203 13:17:19.241309 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bfb2z\" (UniqueName: \"kubernetes.io/projected/3d02d92e-1cee-476f-82ab-ceb335c1a247-kube-api-access-bfb2z\") pod \"3d02d92e-1cee-476f-82ab-ceb335c1a247\" (UID: \"3d02d92e-1cee-476f-82ab-ceb335c1a247\") " Dec 03 13:17:19 crc kubenswrapper[5052]: I1203 13:17:19.249802 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3d02d92e-1cee-476f-82ab-ceb335c1a247-credential-keys" (OuterVolumeSpecName: "credential-keys") pod "3d02d92e-1cee-476f-82ab-ceb335c1a247" (UID: "3d02d92e-1cee-476f-82ab-ceb335c1a247"). InnerVolumeSpecName "credential-keys". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:17:19 crc kubenswrapper[5052]: I1203 13:17:19.249831 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3d02d92e-1cee-476f-82ab-ceb335c1a247-kube-api-access-bfb2z" (OuterVolumeSpecName: "kube-api-access-bfb2z") pod "3d02d92e-1cee-476f-82ab-ceb335c1a247" (UID: "3d02d92e-1cee-476f-82ab-ceb335c1a247"). InnerVolumeSpecName "kube-api-access-bfb2z". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:17:19 crc kubenswrapper[5052]: I1203 13:17:19.249835 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3d02d92e-1cee-476f-82ab-ceb335c1a247-fernet-keys" (OuterVolumeSpecName: "fernet-keys") pod "3d02d92e-1cee-476f-82ab-ceb335c1a247" (UID: "3d02d92e-1cee-476f-82ab-ceb335c1a247"). InnerVolumeSpecName "fernet-keys". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:17:19 crc kubenswrapper[5052]: I1203 13:17:19.252763 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3d02d92e-1cee-476f-82ab-ceb335c1a247-scripts" (OuterVolumeSpecName: "scripts") pod "3d02d92e-1cee-476f-82ab-ceb335c1a247" (UID: "3d02d92e-1cee-476f-82ab-ceb335c1a247"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:17:19 crc kubenswrapper[5052]: I1203 13:17:19.268725 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3d02d92e-1cee-476f-82ab-ceb335c1a247-config-data" (OuterVolumeSpecName: "config-data") pod "3d02d92e-1cee-476f-82ab-ceb335c1a247" (UID: "3d02d92e-1cee-476f-82ab-ceb335c1a247"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:17:19 crc kubenswrapper[5052]: I1203 13:17:19.343218 5052 reconciler_common.go:293] "Volume detached for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/3d02d92e-1cee-476f-82ab-ceb335c1a247-credential-keys\") on node \"crc\" DevicePath \"\"" Dec 03 13:17:19 crc kubenswrapper[5052]: I1203 13:17:19.343467 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/3d02d92e-1cee-476f-82ab-ceb335c1a247-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 13:17:19 crc kubenswrapper[5052]: I1203 13:17:19.343580 5052 reconciler_common.go:293] "Volume detached for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/3d02d92e-1cee-476f-82ab-ceb335c1a247-fernet-keys\") on node \"crc\" DevicePath \"\"" Dec 03 13:17:19 crc kubenswrapper[5052]: I1203 13:17:19.343669 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/3d02d92e-1cee-476f-82ab-ceb335c1a247-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 13:17:19 crc kubenswrapper[5052]: I1203 13:17:19.343742 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-bfb2z\" (UniqueName: \"kubernetes.io/projected/3d02d92e-1cee-476f-82ab-ceb335c1a247-kube-api-access-bfb2z\") on node \"crc\" DevicePath \"\"" Dec 03 13:17:19 crc kubenswrapper[5052]: I1203 13:17:19.765405 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="barbican-kuttl-tests/keystone-bootstrap-kqb5l" event={"ID":"3d02d92e-1cee-476f-82ab-ceb335c1a247","Type":"ContainerDied","Data":"2b3d34850a2c07733af6048fb5c8085b5bef3dab5ae6a6377cffca358c390c2a"} Dec 03 13:17:19 crc kubenswrapper[5052]: I1203 13:17:19.765707 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2b3d34850a2c07733af6048fb5c8085b5bef3dab5ae6a6377cffca358c390c2a" Dec 03 13:17:19 crc kubenswrapper[5052]: I1203 13:17:19.765480 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="barbican-kuttl-tests/keystone-bootstrap-kqb5l" Dec 03 13:17:19 crc kubenswrapper[5052]: I1203 13:17:19.857119 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["barbican-kuttl-tests/keystone-67d5bf7497-2t6vk"] Dec 03 13:17:19 crc kubenswrapper[5052]: E1203 13:17:19.857454 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="3d02d92e-1cee-476f-82ab-ceb335c1a247" containerName="keystone-bootstrap" Dec 03 13:17:19 crc kubenswrapper[5052]: I1203 13:17:19.857473 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="3d02d92e-1cee-476f-82ab-ceb335c1a247" containerName="keystone-bootstrap" Dec 03 13:17:19 crc kubenswrapper[5052]: I1203 13:17:19.857665 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="3d02d92e-1cee-476f-82ab-ceb335c1a247" containerName="keystone-bootstrap" Dec 03 13:17:19 crc kubenswrapper[5052]: I1203 13:17:19.858133 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="barbican-kuttl-tests/keystone-67d5bf7497-2t6vk" Dec 03 13:17:19 crc kubenswrapper[5052]: I1203 13:17:19.862819 5052 reflector.go:368] Caches populated for *v1.Secret from object-"barbican-kuttl-tests"/"keystone" Dec 03 13:17:19 crc kubenswrapper[5052]: I1203 13:17:19.862951 5052 reflector.go:368] Caches populated for *v1.Secret from object-"barbican-kuttl-tests"/"keystone-scripts" Dec 03 13:17:19 crc kubenswrapper[5052]: I1203 13:17:19.864903 5052 reflector.go:368] Caches populated for *v1.Secret from object-"barbican-kuttl-tests"/"keystone-config-data" Dec 03 13:17:19 crc kubenswrapper[5052]: I1203 13:17:19.865242 5052 reflector.go:368] Caches populated for *v1.Secret from object-"barbican-kuttl-tests"/"keystone-keystone-dockercfg-rxpzr" Dec 03 13:17:19 crc kubenswrapper[5052]: I1203 13:17:19.874543 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["barbican-kuttl-tests/keystone-67d5bf7497-2t6vk"] Dec 03 13:17:19 crc kubenswrapper[5052]: I1203 13:17:19.953702 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wzsms\" (UniqueName: \"kubernetes.io/projected/c8797321-9192-42b6-a077-6c057a12de0c-kube-api-access-wzsms\") pod \"keystone-67d5bf7497-2t6vk\" (UID: \"c8797321-9192-42b6-a077-6c057a12de0c\") " pod="barbican-kuttl-tests/keystone-67d5bf7497-2t6vk" Dec 03 13:17:19 crc kubenswrapper[5052]: I1203 13:17:19.953794 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/c8797321-9192-42b6-a077-6c057a12de0c-credential-keys\") pod \"keystone-67d5bf7497-2t6vk\" (UID: \"c8797321-9192-42b6-a077-6c057a12de0c\") " pod="barbican-kuttl-tests/keystone-67d5bf7497-2t6vk" Dec 03 13:17:19 crc kubenswrapper[5052]: I1203 13:17:19.953909 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/c8797321-9192-42b6-a077-6c057a12de0c-config-data\") pod \"keystone-67d5bf7497-2t6vk\" (UID: \"c8797321-9192-42b6-a077-6c057a12de0c\") " pod="barbican-kuttl-tests/keystone-67d5bf7497-2t6vk" Dec 03 13:17:19 crc kubenswrapper[5052]: I1203 13:17:19.953951 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/c8797321-9192-42b6-a077-6c057a12de0c-fernet-keys\") pod \"keystone-67d5bf7497-2t6vk\" (UID: \"c8797321-9192-42b6-a077-6c057a12de0c\") " pod="barbican-kuttl-tests/keystone-67d5bf7497-2t6vk" Dec 03 13:17:19 crc kubenswrapper[5052]: I1203 13:17:19.954023 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/c8797321-9192-42b6-a077-6c057a12de0c-scripts\") pod \"keystone-67d5bf7497-2t6vk\" (UID: \"c8797321-9192-42b6-a077-6c057a12de0c\") " pod="barbican-kuttl-tests/keystone-67d5bf7497-2t6vk" Dec 03 13:17:20 crc kubenswrapper[5052]: I1203 13:17:20.055102 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/c8797321-9192-42b6-a077-6c057a12de0c-credential-keys\") pod \"keystone-67d5bf7497-2t6vk\" (UID: \"c8797321-9192-42b6-a077-6c057a12de0c\") " pod="barbican-kuttl-tests/keystone-67d5bf7497-2t6vk" Dec 03 13:17:20 crc kubenswrapper[5052]: I1203 13:17:20.055235 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/c8797321-9192-42b6-a077-6c057a12de0c-config-data\") pod \"keystone-67d5bf7497-2t6vk\" (UID: \"c8797321-9192-42b6-a077-6c057a12de0c\") " pod="barbican-kuttl-tests/keystone-67d5bf7497-2t6vk" Dec 03 13:17:20 crc kubenswrapper[5052]: I1203 13:17:20.055272 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/c8797321-9192-42b6-a077-6c057a12de0c-fernet-keys\") pod \"keystone-67d5bf7497-2t6vk\" (UID: \"c8797321-9192-42b6-a077-6c057a12de0c\") " pod="barbican-kuttl-tests/keystone-67d5bf7497-2t6vk" Dec 03 13:17:20 crc kubenswrapper[5052]: I1203 13:17:20.055316 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/c8797321-9192-42b6-a077-6c057a12de0c-scripts\") pod \"keystone-67d5bf7497-2t6vk\" (UID: \"c8797321-9192-42b6-a077-6c057a12de0c\") " pod="barbican-kuttl-tests/keystone-67d5bf7497-2t6vk" Dec 03 13:17:20 crc kubenswrapper[5052]: I1203 13:17:20.055493 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-wzsms\" (UniqueName: \"kubernetes.io/projected/c8797321-9192-42b6-a077-6c057a12de0c-kube-api-access-wzsms\") pod \"keystone-67d5bf7497-2t6vk\" (UID: \"c8797321-9192-42b6-a077-6c057a12de0c\") " pod="barbican-kuttl-tests/keystone-67d5bf7497-2t6vk" Dec 03 13:17:20 crc kubenswrapper[5052]: I1203 13:17:20.059945 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/c8797321-9192-42b6-a077-6c057a12de0c-scripts\") pod \"keystone-67d5bf7497-2t6vk\" (UID: \"c8797321-9192-42b6-a077-6c057a12de0c\") " pod="barbican-kuttl-tests/keystone-67d5bf7497-2t6vk" Dec 03 13:17:20 crc kubenswrapper[5052]: I1203 13:17:20.060227 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/c8797321-9192-42b6-a077-6c057a12de0c-credential-keys\") pod \"keystone-67d5bf7497-2t6vk\" (UID: \"c8797321-9192-42b6-a077-6c057a12de0c\") " pod="barbican-kuttl-tests/keystone-67d5bf7497-2t6vk" Dec 03 13:17:20 crc kubenswrapper[5052]: I1203 13:17:20.062043 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/c8797321-9192-42b6-a077-6c057a12de0c-fernet-keys\") pod \"keystone-67d5bf7497-2t6vk\" (UID: \"c8797321-9192-42b6-a077-6c057a12de0c\") " pod="barbican-kuttl-tests/keystone-67d5bf7497-2t6vk" Dec 03 13:17:20 crc kubenswrapper[5052]: I1203 13:17:20.062228 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/c8797321-9192-42b6-a077-6c057a12de0c-config-data\") pod \"keystone-67d5bf7497-2t6vk\" (UID: \"c8797321-9192-42b6-a077-6c057a12de0c\") " pod="barbican-kuttl-tests/keystone-67d5bf7497-2t6vk" Dec 03 13:17:20 crc kubenswrapper[5052]: I1203 13:17:20.073440 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-wzsms\" (UniqueName: \"kubernetes.io/projected/c8797321-9192-42b6-a077-6c057a12de0c-kube-api-access-wzsms\") pod \"keystone-67d5bf7497-2t6vk\" (UID: \"c8797321-9192-42b6-a077-6c057a12de0c\") " pod="barbican-kuttl-tests/keystone-67d5bf7497-2t6vk" Dec 03 13:17:20 crc kubenswrapper[5052]: I1203 13:17:20.176821 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="barbican-kuttl-tests/keystone-67d5bf7497-2t6vk" Dec 03 13:17:20 crc kubenswrapper[5052]: I1203 13:17:20.600258 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["barbican-kuttl-tests/keystone-67d5bf7497-2t6vk"] Dec 03 13:17:20 crc kubenswrapper[5052]: W1203 13:17:20.607504 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8797321_9192_42b6_a077_6c057a12de0c.slice/crio-6da85fd53d7a2f0184c4b3d60e6c191c54a86da7433ea676a1b135cbd1d9ec64 WatchSource:0}: Error finding container 6da85fd53d7a2f0184c4b3d60e6c191c54a86da7433ea676a1b135cbd1d9ec64: Status 404 returned error can't find the container with id 6da85fd53d7a2f0184c4b3d60e6c191c54a86da7433ea676a1b135cbd1d9ec64 Dec 03 13:17:20 crc kubenswrapper[5052]: I1203 13:17:20.774936 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="barbican-kuttl-tests/keystone-67d5bf7497-2t6vk" event={"ID":"c8797321-9192-42b6-a077-6c057a12de0c","Type":"ContainerStarted","Data":"6da85fd53d7a2f0184c4b3d60e6c191c54a86da7433ea676a1b135cbd1d9ec64"} Dec 03 13:17:21 crc kubenswrapper[5052]: I1203 13:17:21.784866 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="barbican-kuttl-tests/keystone-67d5bf7497-2t6vk" event={"ID":"c8797321-9192-42b6-a077-6c057a12de0c","Type":"ContainerStarted","Data":"957b6e8f140e175739368e4e1e2c09516ca0ffc214563f78f23a59f8af879c2c"} Dec 03 13:17:21 crc kubenswrapper[5052]: I1203 13:17:21.786212 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="barbican-kuttl-tests/keystone-67d5bf7497-2t6vk" Dec 03 13:17:29 crc kubenswrapper[5052]: I1203 13:17:29.379617 5052 patch_prober.go:28] interesting pod/machine-config-daemon-74bj6 container/machine-config-daemon namespace/openshift-machine-config-operator: Liveness probe status=failure output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" start-of-body= Dec 03 13:17:29 crc kubenswrapper[5052]: I1203 13:17:29.380214 5052 prober.go:107] "Probe failed" probeType="Liveness" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" containerName="machine-config-daemon" probeResult="failure" output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" Dec 03 13:17:35 crc kubenswrapper[5052]: I1203 13:17:35.521393 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="barbican-kuttl-tests/keystone-67d5bf7497-2t6vk" podStartSLOduration=16.521376297 podStartE2EDuration="16.521376297s" podCreationTimestamp="2025-12-03 13:17:19 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 13:17:21.806582298 +0000 UTC m=+7032.394824062" watchObservedRunningTime="2025-12-03 13:17:35.521376297 +0000 UTC m=+7046.109618061" Dec 03 13:17:35 crc kubenswrapper[5052]: I1203 13:17:35.522906 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["barbican-kuttl-tests/barbican-db-create-dln98"] Dec 03 13:17:35 crc kubenswrapper[5052]: I1203 13:17:35.523798 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="barbican-kuttl-tests/barbican-db-create-dln98" Dec 03 13:17:35 crc kubenswrapper[5052]: I1203 13:17:35.531338 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["barbican-kuttl-tests/barbican-db-create-dln98"] Dec 03 13:17:35 crc kubenswrapper[5052]: I1203 13:17:35.635854 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["barbican-kuttl-tests/barbican-c550-account-create-update-mfpdv"] Dec 03 13:17:35 crc kubenswrapper[5052]: I1203 13:17:35.636853 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="barbican-kuttl-tests/barbican-c550-account-create-update-mfpdv" Dec 03 13:17:35 crc kubenswrapper[5052]: I1203 13:17:35.641213 5052 reflector.go:368] Caches populated for *v1.Secret from object-"barbican-kuttl-tests"/"barbican-db-secret" Dec 03 13:17:35 crc kubenswrapper[5052]: I1203 13:17:35.648623 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["barbican-kuttl-tests/barbican-c550-account-create-update-mfpdv"] Dec 03 13:17:35 crc kubenswrapper[5052]: I1203 13:17:35.698826 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cn7sd\" (UniqueName: \"kubernetes.io/projected/711275ad-0d01-45b9-a19c-93d45b52ec17-kube-api-access-cn7sd\") pod \"barbican-db-create-dln98\" (UID: \"711275ad-0d01-45b9-a19c-93d45b52ec17\") " pod="barbican-kuttl-tests/barbican-db-create-dln98" Dec 03 13:17:35 crc kubenswrapper[5052]: I1203 13:17:35.698899 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/711275ad-0d01-45b9-a19c-93d45b52ec17-operator-scripts\") pod \"barbican-db-create-dln98\" (UID: \"711275ad-0d01-45b9-a19c-93d45b52ec17\") " pod="barbican-kuttl-tests/barbican-db-create-dln98" Dec 03 13:17:35 crc kubenswrapper[5052]: I1203 13:17:35.799926 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x8g4d\" (UniqueName: \"kubernetes.io/projected/da4f9b2e-4f41-4ddd-8059-1f9bad922873-kube-api-access-x8g4d\") pod \"barbican-c550-account-create-update-mfpdv\" (UID: \"da4f9b2e-4f41-4ddd-8059-1f9bad922873\") " pod="barbican-kuttl-tests/barbican-c550-account-create-update-mfpdv" Dec 03 13:17:35 crc kubenswrapper[5052]: I1203 13:17:35.800303 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-cn7sd\" (UniqueName: \"kubernetes.io/projected/711275ad-0d01-45b9-a19c-93d45b52ec17-kube-api-access-cn7sd\") pod \"barbican-db-create-dln98\" (UID: \"711275ad-0d01-45b9-a19c-93d45b52ec17\") " pod="barbican-kuttl-tests/barbican-db-create-dln98" Dec 03 13:17:35 crc kubenswrapper[5052]: I1203 13:17:35.800382 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/da4f9b2e-4f41-4ddd-8059-1f9bad922873-operator-scripts\") pod \"barbican-c550-account-create-update-mfpdv\" (UID: \"da4f9b2e-4f41-4ddd-8059-1f9bad922873\") " pod="barbican-kuttl-tests/barbican-c550-account-create-update-mfpdv" Dec 03 13:17:35 crc kubenswrapper[5052]: I1203 13:17:35.800412 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/711275ad-0d01-45b9-a19c-93d45b52ec17-operator-scripts\") pod \"barbican-db-create-dln98\" (UID: \"711275ad-0d01-45b9-a19c-93d45b52ec17\") " pod="barbican-kuttl-tests/barbican-db-create-dln98" Dec 03 13:17:35 crc kubenswrapper[5052]: I1203 13:17:35.801743 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/711275ad-0d01-45b9-a19c-93d45b52ec17-operator-scripts\") pod \"barbican-db-create-dln98\" (UID: \"711275ad-0d01-45b9-a19c-93d45b52ec17\") " pod="barbican-kuttl-tests/barbican-db-create-dln98" Dec 03 13:17:35 crc kubenswrapper[5052]: I1203 13:17:35.826284 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-cn7sd\" (UniqueName: \"kubernetes.io/projected/711275ad-0d01-45b9-a19c-93d45b52ec17-kube-api-access-cn7sd\") pod \"barbican-db-create-dln98\" (UID: \"711275ad-0d01-45b9-a19c-93d45b52ec17\") " pod="barbican-kuttl-tests/barbican-db-create-dln98" Dec 03 13:17:35 crc kubenswrapper[5052]: I1203 13:17:35.858838 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="barbican-kuttl-tests/barbican-db-create-dln98" Dec 03 13:17:35 crc kubenswrapper[5052]: I1203 13:17:35.901415 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-x8g4d\" (UniqueName: \"kubernetes.io/projected/da4f9b2e-4f41-4ddd-8059-1f9bad922873-kube-api-access-x8g4d\") pod \"barbican-c550-account-create-update-mfpdv\" (UID: \"da4f9b2e-4f41-4ddd-8059-1f9bad922873\") " pod="barbican-kuttl-tests/barbican-c550-account-create-update-mfpdv" Dec 03 13:17:35 crc kubenswrapper[5052]: I1203 13:17:35.901597 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/da4f9b2e-4f41-4ddd-8059-1f9bad922873-operator-scripts\") pod \"barbican-c550-account-create-update-mfpdv\" (UID: \"da4f9b2e-4f41-4ddd-8059-1f9bad922873\") " pod="barbican-kuttl-tests/barbican-c550-account-create-update-mfpdv" Dec 03 13:17:35 crc kubenswrapper[5052]: I1203 13:17:35.902307 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/da4f9b2e-4f41-4ddd-8059-1f9bad922873-operator-scripts\") pod \"barbican-c550-account-create-update-mfpdv\" (UID: \"da4f9b2e-4f41-4ddd-8059-1f9bad922873\") " pod="barbican-kuttl-tests/barbican-c550-account-create-update-mfpdv" Dec 03 13:17:35 crc kubenswrapper[5052]: I1203 13:17:35.924485 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-x8g4d\" (UniqueName: \"kubernetes.io/projected/da4f9b2e-4f41-4ddd-8059-1f9bad922873-kube-api-access-x8g4d\") pod \"barbican-c550-account-create-update-mfpdv\" (UID: \"da4f9b2e-4f41-4ddd-8059-1f9bad922873\") " pod="barbican-kuttl-tests/barbican-c550-account-create-update-mfpdv" Dec 03 13:17:35 crc kubenswrapper[5052]: I1203 13:17:35.952185 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="barbican-kuttl-tests/barbican-c550-account-create-update-mfpdv" Dec 03 13:17:36 crc kubenswrapper[5052]: I1203 13:17:36.100909 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["barbican-kuttl-tests/barbican-db-create-dln98"] Dec 03 13:17:36 crc kubenswrapper[5052]: I1203 13:17:36.223672 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["barbican-kuttl-tests/barbican-c550-account-create-update-mfpdv"] Dec 03 13:17:36 crc kubenswrapper[5052]: W1203 13:17:36.229332 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podda4f9b2e_4f41_4ddd_8059_1f9bad922873.slice/crio-36f128417e56ee8558dc147be9217fcf856a5f3a0d85a34c70cd12b203e277ec WatchSource:0}: Error finding container 36f128417e56ee8558dc147be9217fcf856a5f3a0d85a34c70cd12b203e277ec: Status 404 returned error can't find the container with id 36f128417e56ee8558dc147be9217fcf856a5f3a0d85a34c70cd12b203e277ec Dec 03 13:17:36 crc kubenswrapper[5052]: I1203 13:17:36.936927 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="barbican-kuttl-tests/barbican-c550-account-create-update-mfpdv" event={"ID":"da4f9b2e-4f41-4ddd-8059-1f9bad922873","Type":"ContainerStarted","Data":"36f128417e56ee8558dc147be9217fcf856a5f3a0d85a34c70cd12b203e277ec"} Dec 03 13:17:36 crc kubenswrapper[5052]: I1203 13:17:36.938297 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="barbican-kuttl-tests/barbican-db-create-dln98" event={"ID":"711275ad-0d01-45b9-a19c-93d45b52ec17","Type":"ContainerStarted","Data":"18ccf153be72b84cb4c2c50bf2fd1e207d7f7e5bb95a26a02088f39e35d1dea5"} Dec 03 13:17:37 crc kubenswrapper[5052]: I1203 13:17:37.947811 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="barbican-kuttl-tests/barbican-db-create-dln98" event={"ID":"711275ad-0d01-45b9-a19c-93d45b52ec17","Type":"ContainerStarted","Data":"4a7bf8ce71f62b3d07d290f5b183f960f0b2e5bea1d2acf9c5ff15f79e1c42af"} Dec 03 13:17:37 crc kubenswrapper[5052]: I1203 13:17:37.949691 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="barbican-kuttl-tests/barbican-c550-account-create-update-mfpdv" event={"ID":"da4f9b2e-4f41-4ddd-8059-1f9bad922873","Type":"ContainerStarted","Data":"caa9eaa1c00b2cf8d364eb84b3a9fa7ed49cf8b3f8beb87790fca5594d6ff0f0"} Dec 03 13:17:37 crc kubenswrapper[5052]: I1203 13:17:37.964774 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="barbican-kuttl-tests/barbican-db-create-dln98" podStartSLOduration=2.964753849 podStartE2EDuration="2.964753849s" podCreationTimestamp="2025-12-03 13:17:35 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 13:17:37.961010612 +0000 UTC m=+7048.549252376" watchObservedRunningTime="2025-12-03 13:17:37.964753849 +0000 UTC m=+7048.552995613" Dec 03 13:17:37 crc kubenswrapper[5052]: I1203 13:17:37.978829 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="barbican-kuttl-tests/barbican-c550-account-create-update-mfpdv" podStartSLOduration=2.978809252 podStartE2EDuration="2.978809252s" podCreationTimestamp="2025-12-03 13:17:35 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 13:17:37.974427657 +0000 UTC m=+7048.562669421" watchObservedRunningTime="2025-12-03 13:17:37.978809252 +0000 UTC m=+7048.567051016" Dec 03 13:17:38 crc kubenswrapper[5052]: I1203 13:17:38.958901 5052 generic.go:334] "Generic (PLEG): container finished" podID="711275ad-0d01-45b9-a19c-93d45b52ec17" containerID="4a7bf8ce71f62b3d07d290f5b183f960f0b2e5bea1d2acf9c5ff15f79e1c42af" exitCode=0 Dec 03 13:17:38 crc kubenswrapper[5052]: I1203 13:17:38.958948 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="barbican-kuttl-tests/barbican-db-create-dln98" event={"ID":"711275ad-0d01-45b9-a19c-93d45b52ec17","Type":"ContainerDied","Data":"4a7bf8ce71f62b3d07d290f5b183f960f0b2e5bea1d2acf9c5ff15f79e1c42af"} Dec 03 13:17:39 crc kubenswrapper[5052]: I1203 13:17:39.967794 5052 generic.go:334] "Generic (PLEG): container finished" podID="da4f9b2e-4f41-4ddd-8059-1f9bad922873" containerID="caa9eaa1c00b2cf8d364eb84b3a9fa7ed49cf8b3f8beb87790fca5594d6ff0f0" exitCode=0 Dec 03 13:17:39 crc kubenswrapper[5052]: I1203 13:17:39.967893 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="barbican-kuttl-tests/barbican-c550-account-create-update-mfpdv" event={"ID":"da4f9b2e-4f41-4ddd-8059-1f9bad922873","Type":"ContainerDied","Data":"caa9eaa1c00b2cf8d364eb84b3a9fa7ed49cf8b3f8beb87790fca5594d6ff0f0"} Dec 03 13:17:40 crc kubenswrapper[5052]: I1203 13:17:40.256704 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="barbican-kuttl-tests/barbican-db-create-dln98" Dec 03 13:17:40 crc kubenswrapper[5052]: I1203 13:17:40.385146 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/711275ad-0d01-45b9-a19c-93d45b52ec17-operator-scripts\") pod \"711275ad-0d01-45b9-a19c-93d45b52ec17\" (UID: \"711275ad-0d01-45b9-a19c-93d45b52ec17\") " Dec 03 13:17:40 crc kubenswrapper[5052]: I1203 13:17:40.385479 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cn7sd\" (UniqueName: \"kubernetes.io/projected/711275ad-0d01-45b9-a19c-93d45b52ec17-kube-api-access-cn7sd\") pod \"711275ad-0d01-45b9-a19c-93d45b52ec17\" (UID: \"711275ad-0d01-45b9-a19c-93d45b52ec17\") " Dec 03 13:17:40 crc kubenswrapper[5052]: I1203 13:17:40.386040 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/711275ad-0d01-45b9-a19c-93d45b52ec17-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "711275ad-0d01-45b9-a19c-93d45b52ec17" (UID: "711275ad-0d01-45b9-a19c-93d45b52ec17"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:17:40 crc kubenswrapper[5052]: I1203 13:17:40.391433 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/711275ad-0d01-45b9-a19c-93d45b52ec17-kube-api-access-cn7sd" (OuterVolumeSpecName: "kube-api-access-cn7sd") pod "711275ad-0d01-45b9-a19c-93d45b52ec17" (UID: "711275ad-0d01-45b9-a19c-93d45b52ec17"). InnerVolumeSpecName "kube-api-access-cn7sd". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:17:40 crc kubenswrapper[5052]: I1203 13:17:40.487806 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/711275ad-0d01-45b9-a19c-93d45b52ec17-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 13:17:40 crc kubenswrapper[5052]: I1203 13:17:40.487855 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-cn7sd\" (UniqueName: \"kubernetes.io/projected/711275ad-0d01-45b9-a19c-93d45b52ec17-kube-api-access-cn7sd\") on node \"crc\" DevicePath \"\"" Dec 03 13:17:40 crc kubenswrapper[5052]: I1203 13:17:40.976797 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="barbican-kuttl-tests/barbican-db-create-dln98" event={"ID":"711275ad-0d01-45b9-a19c-93d45b52ec17","Type":"ContainerDied","Data":"18ccf153be72b84cb4c2c50bf2fd1e207d7f7e5bb95a26a02088f39e35d1dea5"} Dec 03 13:17:40 crc kubenswrapper[5052]: I1203 13:17:40.976867 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="18ccf153be72b84cb4c2c50bf2fd1e207d7f7e5bb95a26a02088f39e35d1dea5" Dec 03 13:17:40 crc kubenswrapper[5052]: I1203 13:17:40.976825 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="barbican-kuttl-tests/barbican-db-create-dln98" Dec 03 13:17:41 crc kubenswrapper[5052]: I1203 13:17:41.249234 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="barbican-kuttl-tests/barbican-c550-account-create-update-mfpdv" Dec 03 13:17:41 crc kubenswrapper[5052]: I1203 13:17:41.401389 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x8g4d\" (UniqueName: \"kubernetes.io/projected/da4f9b2e-4f41-4ddd-8059-1f9bad922873-kube-api-access-x8g4d\") pod \"da4f9b2e-4f41-4ddd-8059-1f9bad922873\" (UID: \"da4f9b2e-4f41-4ddd-8059-1f9bad922873\") " Dec 03 13:17:41 crc kubenswrapper[5052]: I1203 13:17:41.401437 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/da4f9b2e-4f41-4ddd-8059-1f9bad922873-operator-scripts\") pod \"da4f9b2e-4f41-4ddd-8059-1f9bad922873\" (UID: \"da4f9b2e-4f41-4ddd-8059-1f9bad922873\") " Dec 03 13:17:41 crc kubenswrapper[5052]: I1203 13:17:41.402410 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/da4f9b2e-4f41-4ddd-8059-1f9bad922873-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "da4f9b2e-4f41-4ddd-8059-1f9bad922873" (UID: "da4f9b2e-4f41-4ddd-8059-1f9bad922873"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:17:41 crc kubenswrapper[5052]: I1203 13:17:41.406386 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/da4f9b2e-4f41-4ddd-8059-1f9bad922873-kube-api-access-x8g4d" (OuterVolumeSpecName: "kube-api-access-x8g4d") pod "da4f9b2e-4f41-4ddd-8059-1f9bad922873" (UID: "da4f9b2e-4f41-4ddd-8059-1f9bad922873"). InnerVolumeSpecName "kube-api-access-x8g4d". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:17:41 crc kubenswrapper[5052]: I1203 13:17:41.502871 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-x8g4d\" (UniqueName: \"kubernetes.io/projected/da4f9b2e-4f41-4ddd-8059-1f9bad922873-kube-api-access-x8g4d\") on node \"crc\" DevicePath \"\"" Dec 03 13:17:41 crc kubenswrapper[5052]: I1203 13:17:41.502908 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/da4f9b2e-4f41-4ddd-8059-1f9bad922873-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 13:17:41 crc kubenswrapper[5052]: I1203 13:17:41.985669 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="barbican-kuttl-tests/barbican-c550-account-create-update-mfpdv" event={"ID":"da4f9b2e-4f41-4ddd-8059-1f9bad922873","Type":"ContainerDied","Data":"36f128417e56ee8558dc147be9217fcf856a5f3a0d85a34c70cd12b203e277ec"} Dec 03 13:17:41 crc kubenswrapper[5052]: I1203 13:17:41.985711 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="36f128417e56ee8558dc147be9217fcf856a5f3a0d85a34c70cd12b203e277ec" Dec 03 13:17:41 crc kubenswrapper[5052]: I1203 13:17:41.985762 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="barbican-kuttl-tests/barbican-c550-account-create-update-mfpdv" Dec 03 13:17:46 crc kubenswrapper[5052]: I1203 13:17:46.000847 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["barbican-kuttl-tests/barbican-db-sync-tnw45"] Dec 03 13:17:46 crc kubenswrapper[5052]: E1203 13:17:46.001142 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="da4f9b2e-4f41-4ddd-8059-1f9bad922873" containerName="mariadb-account-create-update" Dec 03 13:17:46 crc kubenswrapper[5052]: I1203 13:17:46.001156 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="da4f9b2e-4f41-4ddd-8059-1f9bad922873" containerName="mariadb-account-create-update" Dec 03 13:17:46 crc kubenswrapper[5052]: E1203 13:17:46.001178 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="711275ad-0d01-45b9-a19c-93d45b52ec17" containerName="mariadb-database-create" Dec 03 13:17:46 crc kubenswrapper[5052]: I1203 13:17:46.001184 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="711275ad-0d01-45b9-a19c-93d45b52ec17" containerName="mariadb-database-create" Dec 03 13:17:46 crc kubenswrapper[5052]: I1203 13:17:46.001303 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="da4f9b2e-4f41-4ddd-8059-1f9bad922873" containerName="mariadb-account-create-update" Dec 03 13:17:46 crc kubenswrapper[5052]: I1203 13:17:46.001325 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="711275ad-0d01-45b9-a19c-93d45b52ec17" containerName="mariadb-database-create" Dec 03 13:17:46 crc kubenswrapper[5052]: I1203 13:17:46.001837 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="barbican-kuttl-tests/barbican-db-sync-tnw45" Dec 03 13:17:46 crc kubenswrapper[5052]: I1203 13:17:46.003966 5052 reflector.go:368] Caches populated for *v1.Secret from object-"barbican-kuttl-tests"/"barbican-config-data" Dec 03 13:17:46 crc kubenswrapper[5052]: I1203 13:17:46.009420 5052 reflector.go:368] Caches populated for *v1.Secret from object-"barbican-kuttl-tests"/"barbican-barbican-dockercfg-spcqc" Dec 03 13:17:46 crc kubenswrapper[5052]: I1203 13:17:46.017327 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["barbican-kuttl-tests/barbican-db-sync-tnw45"] Dec 03 13:17:46 crc kubenswrapper[5052]: I1203 13:17:46.179351 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-87brx\" (UniqueName: \"kubernetes.io/projected/a5e071a4-f78a-46b3-b8a6-5880f4eecd2a-kube-api-access-87brx\") pod \"barbican-db-sync-tnw45\" (UID: \"a5e071a4-f78a-46b3-b8a6-5880f4eecd2a\") " pod="barbican-kuttl-tests/barbican-db-sync-tnw45" Dec 03 13:17:46 crc kubenswrapper[5052]: I1203 13:17:46.179540 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/a5e071a4-f78a-46b3-b8a6-5880f4eecd2a-db-sync-config-data\") pod \"barbican-db-sync-tnw45\" (UID: \"a5e071a4-f78a-46b3-b8a6-5880f4eecd2a\") " pod="barbican-kuttl-tests/barbican-db-sync-tnw45" Dec 03 13:17:46 crc kubenswrapper[5052]: I1203 13:17:46.280792 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/a5e071a4-f78a-46b3-b8a6-5880f4eecd2a-db-sync-config-data\") pod \"barbican-db-sync-tnw45\" (UID: \"a5e071a4-f78a-46b3-b8a6-5880f4eecd2a\") " pod="barbican-kuttl-tests/barbican-db-sync-tnw45" Dec 03 13:17:46 crc kubenswrapper[5052]: I1203 13:17:46.280901 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-87brx\" (UniqueName: \"kubernetes.io/projected/a5e071a4-f78a-46b3-b8a6-5880f4eecd2a-kube-api-access-87brx\") pod \"barbican-db-sync-tnw45\" (UID: \"a5e071a4-f78a-46b3-b8a6-5880f4eecd2a\") " pod="barbican-kuttl-tests/barbican-db-sync-tnw45" Dec 03 13:17:46 crc kubenswrapper[5052]: I1203 13:17:46.289701 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/a5e071a4-f78a-46b3-b8a6-5880f4eecd2a-db-sync-config-data\") pod \"barbican-db-sync-tnw45\" (UID: \"a5e071a4-f78a-46b3-b8a6-5880f4eecd2a\") " pod="barbican-kuttl-tests/barbican-db-sync-tnw45" Dec 03 13:17:46 crc kubenswrapper[5052]: I1203 13:17:46.294481 5052 scope.go:117] "RemoveContainer" containerID="3f9790aa10f8b223320a934c6473ecbc82c9128a418a4c84445a169a85afddab" Dec 03 13:17:46 crc kubenswrapper[5052]: I1203 13:17:46.320235 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-87brx\" (UniqueName: \"kubernetes.io/projected/a5e071a4-f78a-46b3-b8a6-5880f4eecd2a-kube-api-access-87brx\") pod \"barbican-db-sync-tnw45\" (UID: \"a5e071a4-f78a-46b3-b8a6-5880f4eecd2a\") " pod="barbican-kuttl-tests/barbican-db-sync-tnw45" Dec 03 13:17:46 crc kubenswrapper[5052]: I1203 13:17:46.321898 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="barbican-kuttl-tests/barbican-db-sync-tnw45" Dec 03 13:17:46 crc kubenswrapper[5052]: I1203 13:17:46.419790 5052 scope.go:117] "RemoveContainer" containerID="89e73c0c215bbb5887aeba8a7aac0436aeada46d17549f59a4621f95d65b2357" Dec 03 13:17:46 crc kubenswrapper[5052]: I1203 13:17:46.876622 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["barbican-kuttl-tests/barbican-db-sync-tnw45"] Dec 03 13:17:47 crc kubenswrapper[5052]: I1203 13:17:47.021701 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="barbican-kuttl-tests/barbican-db-sync-tnw45" event={"ID":"a5e071a4-f78a-46b3-b8a6-5880f4eecd2a","Type":"ContainerStarted","Data":"f6ed7bb991f72d04528dd5706bfdef7f09dadf31e90f4843a525ec3108b66914"} Dec 03 13:17:48 crc kubenswrapper[5052]: I1203 13:17:48.032312 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="barbican-kuttl-tests/barbican-db-sync-tnw45" event={"ID":"a5e071a4-f78a-46b3-b8a6-5880f4eecd2a","Type":"ContainerStarted","Data":"ce9c06b6087aafd38af61e1b86904aad0dbe372e78c8e94652ef1add52ada691"} Dec 03 13:17:48 crc kubenswrapper[5052]: I1203 13:17:48.052072 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="barbican-kuttl-tests/barbican-db-sync-tnw45" podStartSLOduration=2.3161915730000002 podStartE2EDuration="3.052051956s" podCreationTimestamp="2025-12-03 13:17:45 +0000 UTC" firstStartedPulling="2025-12-03 13:17:46.883597674 +0000 UTC m=+7057.471839438" lastFinishedPulling="2025-12-03 13:17:47.619458057 +0000 UTC m=+7058.207699821" observedRunningTime="2025-12-03 13:17:48.048535425 +0000 UTC m=+7058.636777209" watchObservedRunningTime="2025-12-03 13:17:48.052051956 +0000 UTC m=+7058.640293720" Dec 03 13:17:51 crc kubenswrapper[5052]: I1203 13:17:51.721355 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="barbican-kuttl-tests/keystone-67d5bf7497-2t6vk" Dec 03 13:17:52 crc kubenswrapper[5052]: I1203 13:17:52.064373 5052 generic.go:334] "Generic (PLEG): container finished" podID="a5e071a4-f78a-46b3-b8a6-5880f4eecd2a" containerID="ce9c06b6087aafd38af61e1b86904aad0dbe372e78c8e94652ef1add52ada691" exitCode=0 Dec 03 13:17:52 crc kubenswrapper[5052]: I1203 13:17:52.064425 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="barbican-kuttl-tests/barbican-db-sync-tnw45" event={"ID":"a5e071a4-f78a-46b3-b8a6-5880f4eecd2a","Type":"ContainerDied","Data":"ce9c06b6087aafd38af61e1b86904aad0dbe372e78c8e94652ef1add52ada691"} Dec 03 13:17:53 crc kubenswrapper[5052]: I1203 13:17:53.335592 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="barbican-kuttl-tests/barbican-db-sync-tnw45" Dec 03 13:17:53 crc kubenswrapper[5052]: I1203 13:17:53.489419 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/a5e071a4-f78a-46b3-b8a6-5880f4eecd2a-db-sync-config-data\") pod \"a5e071a4-f78a-46b3-b8a6-5880f4eecd2a\" (UID: \"a5e071a4-f78a-46b3-b8a6-5880f4eecd2a\") " Dec 03 13:17:53 crc kubenswrapper[5052]: I1203 13:17:53.489524 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-87brx\" (UniqueName: \"kubernetes.io/projected/a5e071a4-f78a-46b3-b8a6-5880f4eecd2a-kube-api-access-87brx\") pod \"a5e071a4-f78a-46b3-b8a6-5880f4eecd2a\" (UID: \"a5e071a4-f78a-46b3-b8a6-5880f4eecd2a\") " Dec 03 13:17:53 crc kubenswrapper[5052]: I1203 13:17:53.496471 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a5e071a4-f78a-46b3-b8a6-5880f4eecd2a-db-sync-config-data" (OuterVolumeSpecName: "db-sync-config-data") pod "a5e071a4-f78a-46b3-b8a6-5880f4eecd2a" (UID: "a5e071a4-f78a-46b3-b8a6-5880f4eecd2a"). InnerVolumeSpecName "db-sync-config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:17:53 crc kubenswrapper[5052]: I1203 13:17:53.496492 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a5e071a4-f78a-46b3-b8a6-5880f4eecd2a-kube-api-access-87brx" (OuterVolumeSpecName: "kube-api-access-87brx") pod "a5e071a4-f78a-46b3-b8a6-5880f4eecd2a" (UID: "a5e071a4-f78a-46b3-b8a6-5880f4eecd2a"). InnerVolumeSpecName "kube-api-access-87brx". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:17:53 crc kubenswrapper[5052]: I1203 13:17:53.591548 5052 reconciler_common.go:293] "Volume detached for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/a5e071a4-f78a-46b3-b8a6-5880f4eecd2a-db-sync-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 13:17:53 crc kubenswrapper[5052]: I1203 13:17:53.591591 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-87brx\" (UniqueName: \"kubernetes.io/projected/a5e071a4-f78a-46b3-b8a6-5880f4eecd2a-kube-api-access-87brx\") on node \"crc\" DevicePath \"\"" Dec 03 13:17:54 crc kubenswrapper[5052]: I1203 13:17:54.081129 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="barbican-kuttl-tests/barbican-db-sync-tnw45" event={"ID":"a5e071a4-f78a-46b3-b8a6-5880f4eecd2a","Type":"ContainerDied","Data":"f6ed7bb991f72d04528dd5706bfdef7f09dadf31e90f4843a525ec3108b66914"} Dec 03 13:17:54 crc kubenswrapper[5052]: I1203 13:17:54.081172 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f6ed7bb991f72d04528dd5706bfdef7f09dadf31e90f4843a525ec3108b66914" Dec 03 13:17:54 crc kubenswrapper[5052]: I1203 13:17:54.081195 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="barbican-kuttl-tests/barbican-db-sync-tnw45" Dec 03 13:17:54 crc kubenswrapper[5052]: I1203 13:17:54.312767 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["barbican-kuttl-tests/barbican-keystone-listener-66b47d8764-7bflh"] Dec 03 13:17:54 crc kubenswrapper[5052]: E1203 13:17:54.313159 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="a5e071a4-f78a-46b3-b8a6-5880f4eecd2a" containerName="barbican-db-sync" Dec 03 13:17:54 crc kubenswrapper[5052]: I1203 13:17:54.313182 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="a5e071a4-f78a-46b3-b8a6-5880f4eecd2a" containerName="barbican-db-sync" Dec 03 13:17:54 crc kubenswrapper[5052]: I1203 13:17:54.313385 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="a5e071a4-f78a-46b3-b8a6-5880f4eecd2a" containerName="barbican-db-sync" Dec 03 13:17:54 crc kubenswrapper[5052]: I1203 13:17:54.314339 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="barbican-kuttl-tests/barbican-keystone-listener-66b47d8764-7bflh" Dec 03 13:17:54 crc kubenswrapper[5052]: I1203 13:17:54.316796 5052 reflector.go:368] Caches populated for *v1.Secret from object-"barbican-kuttl-tests"/"barbican-keystone-listener-config-data" Dec 03 13:17:54 crc kubenswrapper[5052]: I1203 13:17:54.316969 5052 reflector.go:368] Caches populated for *v1.Secret from object-"barbican-kuttl-tests"/"barbican-barbican-dockercfg-spcqc" Dec 03 13:17:54 crc kubenswrapper[5052]: I1203 13:17:54.317001 5052 reflector.go:368] Caches populated for *v1.Secret from object-"barbican-kuttl-tests"/"barbican-config-data" Dec 03 13:17:54 crc kubenswrapper[5052]: I1203 13:17:54.323415 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["barbican-kuttl-tests/barbican-worker-74467844f5-w7kwd"] Dec 03 13:17:54 crc kubenswrapper[5052]: I1203 13:17:54.324887 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="barbican-kuttl-tests/barbican-worker-74467844f5-w7kwd" Dec 03 13:17:54 crc kubenswrapper[5052]: I1203 13:17:54.335272 5052 reflector.go:368] Caches populated for *v1.Secret from object-"barbican-kuttl-tests"/"barbican-worker-config-data" Dec 03 13:17:54 crc kubenswrapper[5052]: I1203 13:17:54.342147 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["barbican-kuttl-tests/barbican-keystone-listener-66b47d8764-7bflh"] Dec 03 13:17:54 crc kubenswrapper[5052]: I1203 13:17:54.371554 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["barbican-kuttl-tests/barbican-worker-74467844f5-w7kwd"] Dec 03 13:17:54 crc kubenswrapper[5052]: I1203 13:17:54.411960 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cw9tq\" (UniqueName: \"kubernetes.io/projected/3a71e182-1787-48f5-a4f8-60b9fb56f603-kube-api-access-cw9tq\") pod \"barbican-keystone-listener-66b47d8764-7bflh\" (UID: \"3a71e182-1787-48f5-a4f8-60b9fb56f603\") " pod="barbican-kuttl-tests/barbican-keystone-listener-66b47d8764-7bflh" Dec 03 13:17:54 crc kubenswrapper[5052]: I1203 13:17:54.412103 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/72c65803-7d22-4b27-9cfc-b8a246a0ba7a-config-data\") pod \"barbican-worker-74467844f5-w7kwd\" (UID: \"72c65803-7d22-4b27-9cfc-b8a246a0ba7a\") " pod="barbican-kuttl-tests/barbican-worker-74467844f5-w7kwd" Dec 03 13:17:54 crc kubenswrapper[5052]: I1203 13:17:54.412141 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/72c65803-7d22-4b27-9cfc-b8a246a0ba7a-config-data-custom\") pod \"barbican-worker-74467844f5-w7kwd\" (UID: \"72c65803-7d22-4b27-9cfc-b8a246a0ba7a\") " pod="barbican-kuttl-tests/barbican-worker-74467844f5-w7kwd" Dec 03 13:17:54 crc kubenswrapper[5052]: I1203 13:17:54.412217 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/72c65803-7d22-4b27-9cfc-b8a246a0ba7a-logs\") pod \"barbican-worker-74467844f5-w7kwd\" (UID: \"72c65803-7d22-4b27-9cfc-b8a246a0ba7a\") " pod="barbican-kuttl-tests/barbican-worker-74467844f5-w7kwd" Dec 03 13:17:54 crc kubenswrapper[5052]: I1203 13:17:54.412245 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/3a71e182-1787-48f5-a4f8-60b9fb56f603-config-data-custom\") pod \"barbican-keystone-listener-66b47d8764-7bflh\" (UID: \"3a71e182-1787-48f5-a4f8-60b9fb56f603\") " pod="barbican-kuttl-tests/barbican-keystone-listener-66b47d8764-7bflh" Dec 03 13:17:54 crc kubenswrapper[5052]: I1203 13:17:54.412293 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4glrf\" (UniqueName: \"kubernetes.io/projected/72c65803-7d22-4b27-9cfc-b8a246a0ba7a-kube-api-access-4glrf\") pod \"barbican-worker-74467844f5-w7kwd\" (UID: \"72c65803-7d22-4b27-9cfc-b8a246a0ba7a\") " pod="barbican-kuttl-tests/barbican-worker-74467844f5-w7kwd" Dec 03 13:17:54 crc kubenswrapper[5052]: I1203 13:17:54.412336 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/3a71e182-1787-48f5-a4f8-60b9fb56f603-config-data\") pod \"barbican-keystone-listener-66b47d8764-7bflh\" (UID: \"3a71e182-1787-48f5-a4f8-60b9fb56f603\") " pod="barbican-kuttl-tests/barbican-keystone-listener-66b47d8764-7bflh" Dec 03 13:17:54 crc kubenswrapper[5052]: I1203 13:17:54.412360 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/3a71e182-1787-48f5-a4f8-60b9fb56f603-logs\") pod \"barbican-keystone-listener-66b47d8764-7bflh\" (UID: \"3a71e182-1787-48f5-a4f8-60b9fb56f603\") " pod="barbican-kuttl-tests/barbican-keystone-listener-66b47d8764-7bflh" Dec 03 13:17:54 crc kubenswrapper[5052]: I1203 13:17:54.514650 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/3a71e182-1787-48f5-a4f8-60b9fb56f603-config-data-custom\") pod \"barbican-keystone-listener-66b47d8764-7bflh\" (UID: \"3a71e182-1787-48f5-a4f8-60b9fb56f603\") " pod="barbican-kuttl-tests/barbican-keystone-listener-66b47d8764-7bflh" Dec 03 13:17:54 crc kubenswrapper[5052]: I1203 13:17:54.515309 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-4glrf\" (UniqueName: \"kubernetes.io/projected/72c65803-7d22-4b27-9cfc-b8a246a0ba7a-kube-api-access-4glrf\") pod \"barbican-worker-74467844f5-w7kwd\" (UID: \"72c65803-7d22-4b27-9cfc-b8a246a0ba7a\") " pod="barbican-kuttl-tests/barbican-worker-74467844f5-w7kwd" Dec 03 13:17:54 crc kubenswrapper[5052]: I1203 13:17:54.515370 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/3a71e182-1787-48f5-a4f8-60b9fb56f603-config-data\") pod \"barbican-keystone-listener-66b47d8764-7bflh\" (UID: \"3a71e182-1787-48f5-a4f8-60b9fb56f603\") " pod="barbican-kuttl-tests/barbican-keystone-listener-66b47d8764-7bflh" Dec 03 13:17:54 crc kubenswrapper[5052]: I1203 13:17:54.515404 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/3a71e182-1787-48f5-a4f8-60b9fb56f603-logs\") pod \"barbican-keystone-listener-66b47d8764-7bflh\" (UID: \"3a71e182-1787-48f5-a4f8-60b9fb56f603\") " pod="barbican-kuttl-tests/barbican-keystone-listener-66b47d8764-7bflh" Dec 03 13:17:54 crc kubenswrapper[5052]: I1203 13:17:54.515551 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-cw9tq\" (UniqueName: \"kubernetes.io/projected/3a71e182-1787-48f5-a4f8-60b9fb56f603-kube-api-access-cw9tq\") pod \"barbican-keystone-listener-66b47d8764-7bflh\" (UID: \"3a71e182-1787-48f5-a4f8-60b9fb56f603\") " pod="barbican-kuttl-tests/barbican-keystone-listener-66b47d8764-7bflh" Dec 03 13:17:54 crc kubenswrapper[5052]: I1203 13:17:54.515654 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/72c65803-7d22-4b27-9cfc-b8a246a0ba7a-config-data\") pod \"barbican-worker-74467844f5-w7kwd\" (UID: \"72c65803-7d22-4b27-9cfc-b8a246a0ba7a\") " pod="barbican-kuttl-tests/barbican-worker-74467844f5-w7kwd" Dec 03 13:17:54 crc kubenswrapper[5052]: I1203 13:17:54.515702 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/72c65803-7d22-4b27-9cfc-b8a246a0ba7a-config-data-custom\") pod \"barbican-worker-74467844f5-w7kwd\" (UID: \"72c65803-7d22-4b27-9cfc-b8a246a0ba7a\") " pod="barbican-kuttl-tests/barbican-worker-74467844f5-w7kwd" Dec 03 13:17:54 crc kubenswrapper[5052]: I1203 13:17:54.515783 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/72c65803-7d22-4b27-9cfc-b8a246a0ba7a-logs\") pod \"barbican-worker-74467844f5-w7kwd\" (UID: \"72c65803-7d22-4b27-9cfc-b8a246a0ba7a\") " pod="barbican-kuttl-tests/barbican-worker-74467844f5-w7kwd" Dec 03 13:17:54 crc kubenswrapper[5052]: I1203 13:17:54.515935 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/3a71e182-1787-48f5-a4f8-60b9fb56f603-logs\") pod \"barbican-keystone-listener-66b47d8764-7bflh\" (UID: \"3a71e182-1787-48f5-a4f8-60b9fb56f603\") " pod="barbican-kuttl-tests/barbican-keystone-listener-66b47d8764-7bflh" Dec 03 13:17:54 crc kubenswrapper[5052]: I1203 13:17:54.516353 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/72c65803-7d22-4b27-9cfc-b8a246a0ba7a-logs\") pod \"barbican-worker-74467844f5-w7kwd\" (UID: \"72c65803-7d22-4b27-9cfc-b8a246a0ba7a\") " pod="barbican-kuttl-tests/barbican-worker-74467844f5-w7kwd" Dec 03 13:17:54 crc kubenswrapper[5052]: I1203 13:17:54.523669 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/3a71e182-1787-48f5-a4f8-60b9fb56f603-config-data-custom\") pod \"barbican-keystone-listener-66b47d8764-7bflh\" (UID: \"3a71e182-1787-48f5-a4f8-60b9fb56f603\") " pod="barbican-kuttl-tests/barbican-keystone-listener-66b47d8764-7bflh" Dec 03 13:17:54 crc kubenswrapper[5052]: I1203 13:17:54.524213 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/72c65803-7d22-4b27-9cfc-b8a246a0ba7a-config-data\") pod \"barbican-worker-74467844f5-w7kwd\" (UID: \"72c65803-7d22-4b27-9cfc-b8a246a0ba7a\") " pod="barbican-kuttl-tests/barbican-worker-74467844f5-w7kwd" Dec 03 13:17:54 crc kubenswrapper[5052]: I1203 13:17:54.527977 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/3a71e182-1787-48f5-a4f8-60b9fb56f603-config-data\") pod \"barbican-keystone-listener-66b47d8764-7bflh\" (UID: \"3a71e182-1787-48f5-a4f8-60b9fb56f603\") " pod="barbican-kuttl-tests/barbican-keystone-listener-66b47d8764-7bflh" Dec 03 13:17:54 crc kubenswrapper[5052]: I1203 13:17:54.527978 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/72c65803-7d22-4b27-9cfc-b8a246a0ba7a-config-data-custom\") pod \"barbican-worker-74467844f5-w7kwd\" (UID: \"72c65803-7d22-4b27-9cfc-b8a246a0ba7a\") " pod="barbican-kuttl-tests/barbican-worker-74467844f5-w7kwd" Dec 03 13:17:54 crc kubenswrapper[5052]: I1203 13:17:54.555348 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-cw9tq\" (UniqueName: \"kubernetes.io/projected/3a71e182-1787-48f5-a4f8-60b9fb56f603-kube-api-access-cw9tq\") pod \"barbican-keystone-listener-66b47d8764-7bflh\" (UID: \"3a71e182-1787-48f5-a4f8-60b9fb56f603\") " pod="barbican-kuttl-tests/barbican-keystone-listener-66b47d8764-7bflh" Dec 03 13:17:54 crc kubenswrapper[5052]: I1203 13:17:54.559255 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-4glrf\" (UniqueName: \"kubernetes.io/projected/72c65803-7d22-4b27-9cfc-b8a246a0ba7a-kube-api-access-4glrf\") pod \"barbican-worker-74467844f5-w7kwd\" (UID: \"72c65803-7d22-4b27-9cfc-b8a246a0ba7a\") " pod="barbican-kuttl-tests/barbican-worker-74467844f5-w7kwd" Dec 03 13:17:54 crc kubenswrapper[5052]: I1203 13:17:54.634082 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="barbican-kuttl-tests/barbican-keystone-listener-66b47d8764-7bflh" Dec 03 13:17:54 crc kubenswrapper[5052]: I1203 13:17:54.661894 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="barbican-kuttl-tests/barbican-worker-74467844f5-w7kwd" Dec 03 13:17:54 crc kubenswrapper[5052]: I1203 13:17:54.779126 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["barbican-kuttl-tests/barbican-api-75495b98c8-4vgmd"] Dec 03 13:17:54 crc kubenswrapper[5052]: I1203 13:17:54.780395 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="barbican-kuttl-tests/barbican-api-75495b98c8-4vgmd" Dec 03 13:17:54 crc kubenswrapper[5052]: I1203 13:17:54.783423 5052 reflector.go:368] Caches populated for *v1.Secret from object-"barbican-kuttl-tests"/"barbican-api-config-data" Dec 03 13:17:54 crc kubenswrapper[5052]: I1203 13:17:54.791707 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["barbican-kuttl-tests/barbican-api-75495b98c8-4vgmd"] Dec 03 13:17:54 crc kubenswrapper[5052]: I1203 13:17:54.921873 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/47809cbf-0aa5-4d38-8b9f-95ce55763ed3-logs\") pod \"barbican-api-75495b98c8-4vgmd\" (UID: \"47809cbf-0aa5-4d38-8b9f-95ce55763ed3\") " pod="barbican-kuttl-tests/barbican-api-75495b98c8-4vgmd" Dec 03 13:17:54 crc kubenswrapper[5052]: I1203 13:17:54.922293 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mnwwp\" (UniqueName: \"kubernetes.io/projected/47809cbf-0aa5-4d38-8b9f-95ce55763ed3-kube-api-access-mnwwp\") pod \"barbican-api-75495b98c8-4vgmd\" (UID: \"47809cbf-0aa5-4d38-8b9f-95ce55763ed3\") " pod="barbican-kuttl-tests/barbican-api-75495b98c8-4vgmd" Dec 03 13:17:54 crc kubenswrapper[5052]: I1203 13:17:54.922373 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/47809cbf-0aa5-4d38-8b9f-95ce55763ed3-config-data-custom\") pod \"barbican-api-75495b98c8-4vgmd\" (UID: \"47809cbf-0aa5-4d38-8b9f-95ce55763ed3\") " pod="barbican-kuttl-tests/barbican-api-75495b98c8-4vgmd" Dec 03 13:17:54 crc kubenswrapper[5052]: I1203 13:17:54.922409 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/47809cbf-0aa5-4d38-8b9f-95ce55763ed3-config-data\") pod \"barbican-api-75495b98c8-4vgmd\" (UID: \"47809cbf-0aa5-4d38-8b9f-95ce55763ed3\") " pod="barbican-kuttl-tests/barbican-api-75495b98c8-4vgmd" Dec 03 13:17:55 crc kubenswrapper[5052]: I1203 13:17:55.023950 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/47809cbf-0aa5-4d38-8b9f-95ce55763ed3-config-data\") pod \"barbican-api-75495b98c8-4vgmd\" (UID: \"47809cbf-0aa5-4d38-8b9f-95ce55763ed3\") " pod="barbican-kuttl-tests/barbican-api-75495b98c8-4vgmd" Dec 03 13:17:55 crc kubenswrapper[5052]: I1203 13:17:55.024098 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/47809cbf-0aa5-4d38-8b9f-95ce55763ed3-logs\") pod \"barbican-api-75495b98c8-4vgmd\" (UID: \"47809cbf-0aa5-4d38-8b9f-95ce55763ed3\") " pod="barbican-kuttl-tests/barbican-api-75495b98c8-4vgmd" Dec 03 13:17:55 crc kubenswrapper[5052]: I1203 13:17:55.024173 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-mnwwp\" (UniqueName: \"kubernetes.io/projected/47809cbf-0aa5-4d38-8b9f-95ce55763ed3-kube-api-access-mnwwp\") pod \"barbican-api-75495b98c8-4vgmd\" (UID: \"47809cbf-0aa5-4d38-8b9f-95ce55763ed3\") " pod="barbican-kuttl-tests/barbican-api-75495b98c8-4vgmd" Dec 03 13:17:55 crc kubenswrapper[5052]: I1203 13:17:55.024246 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/47809cbf-0aa5-4d38-8b9f-95ce55763ed3-config-data-custom\") pod \"barbican-api-75495b98c8-4vgmd\" (UID: \"47809cbf-0aa5-4d38-8b9f-95ce55763ed3\") " pod="barbican-kuttl-tests/barbican-api-75495b98c8-4vgmd" Dec 03 13:17:55 crc kubenswrapper[5052]: I1203 13:17:55.026359 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/47809cbf-0aa5-4d38-8b9f-95ce55763ed3-logs\") pod \"barbican-api-75495b98c8-4vgmd\" (UID: \"47809cbf-0aa5-4d38-8b9f-95ce55763ed3\") " pod="barbican-kuttl-tests/barbican-api-75495b98c8-4vgmd" Dec 03 13:17:55 crc kubenswrapper[5052]: I1203 13:17:55.032603 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/47809cbf-0aa5-4d38-8b9f-95ce55763ed3-config-data-custom\") pod \"barbican-api-75495b98c8-4vgmd\" (UID: \"47809cbf-0aa5-4d38-8b9f-95ce55763ed3\") " pod="barbican-kuttl-tests/barbican-api-75495b98c8-4vgmd" Dec 03 13:17:55 crc kubenswrapper[5052]: I1203 13:17:55.041721 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/47809cbf-0aa5-4d38-8b9f-95ce55763ed3-config-data\") pod \"barbican-api-75495b98c8-4vgmd\" (UID: \"47809cbf-0aa5-4d38-8b9f-95ce55763ed3\") " pod="barbican-kuttl-tests/barbican-api-75495b98c8-4vgmd" Dec 03 13:17:55 crc kubenswrapper[5052]: I1203 13:17:55.056714 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-mnwwp\" (UniqueName: \"kubernetes.io/projected/47809cbf-0aa5-4d38-8b9f-95ce55763ed3-kube-api-access-mnwwp\") pod \"barbican-api-75495b98c8-4vgmd\" (UID: \"47809cbf-0aa5-4d38-8b9f-95ce55763ed3\") " pod="barbican-kuttl-tests/barbican-api-75495b98c8-4vgmd" Dec 03 13:17:55 crc kubenswrapper[5052]: I1203 13:17:55.075354 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["barbican-kuttl-tests/barbican-keystone-listener-66b47d8764-7bflh"] Dec 03 13:17:55 crc kubenswrapper[5052]: I1203 13:17:55.099490 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="barbican-kuttl-tests/barbican-api-75495b98c8-4vgmd" Dec 03 13:17:55 crc kubenswrapper[5052]: I1203 13:17:55.143392 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["barbican-kuttl-tests/barbican-worker-74467844f5-w7kwd"] Dec 03 13:17:55 crc kubenswrapper[5052]: I1203 13:17:55.207177 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["barbican-kuttl-tests/barbican-api-75495b98c8-6rdjk"] Dec 03 13:17:55 crc kubenswrapper[5052]: I1203 13:17:55.209014 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="barbican-kuttl-tests/barbican-api-75495b98c8-6rdjk" Dec 03 13:17:55 crc kubenswrapper[5052]: I1203 13:17:55.225659 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["barbican-kuttl-tests/barbican-api-75495b98c8-6rdjk"] Dec 03 13:17:55 crc kubenswrapper[5052]: I1203 13:17:55.329063 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/82893f5c-2185-4a6a-8b4a-a3d4c4dd92b5-config-data\") pod \"barbican-api-75495b98c8-6rdjk\" (UID: \"82893f5c-2185-4a6a-8b4a-a3d4c4dd92b5\") " pod="barbican-kuttl-tests/barbican-api-75495b98c8-6rdjk" Dec 03 13:17:55 crc kubenswrapper[5052]: I1203 13:17:55.329127 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7kkkq\" (UniqueName: \"kubernetes.io/projected/82893f5c-2185-4a6a-8b4a-a3d4c4dd92b5-kube-api-access-7kkkq\") pod \"barbican-api-75495b98c8-6rdjk\" (UID: \"82893f5c-2185-4a6a-8b4a-a3d4c4dd92b5\") " pod="barbican-kuttl-tests/barbican-api-75495b98c8-6rdjk" Dec 03 13:17:55 crc kubenswrapper[5052]: I1203 13:17:55.329288 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/82893f5c-2185-4a6a-8b4a-a3d4c4dd92b5-config-data-custom\") pod \"barbican-api-75495b98c8-6rdjk\" (UID: \"82893f5c-2185-4a6a-8b4a-a3d4c4dd92b5\") " pod="barbican-kuttl-tests/barbican-api-75495b98c8-6rdjk" Dec 03 13:17:55 crc kubenswrapper[5052]: I1203 13:17:55.329342 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/82893f5c-2185-4a6a-8b4a-a3d4c4dd92b5-logs\") pod \"barbican-api-75495b98c8-6rdjk\" (UID: \"82893f5c-2185-4a6a-8b4a-a3d4c4dd92b5\") " pod="barbican-kuttl-tests/barbican-api-75495b98c8-6rdjk" Dec 03 13:17:55 crc kubenswrapper[5052]: I1203 13:17:55.348670 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["barbican-kuttl-tests/barbican-api-75495b98c8-4vgmd"] Dec 03 13:17:55 crc kubenswrapper[5052]: W1203 13:17:55.351042 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod47809cbf_0aa5_4d38_8b9f_95ce55763ed3.slice/crio-264156498b352d98a23b875a3e46d95ac0b1a0134bf5ae9579feba3d3ffbf1ab WatchSource:0}: Error finding container 264156498b352d98a23b875a3e46d95ac0b1a0134bf5ae9579feba3d3ffbf1ab: Status 404 returned error can't find the container with id 264156498b352d98a23b875a3e46d95ac0b1a0134bf5ae9579feba3d3ffbf1ab Dec 03 13:17:55 crc kubenswrapper[5052]: I1203 13:17:55.401737 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["barbican-kuttl-tests/barbican-keystone-listener-66b47d8764-wbbbf"] Dec 03 13:17:55 crc kubenswrapper[5052]: I1203 13:17:55.403394 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="barbican-kuttl-tests/barbican-keystone-listener-66b47d8764-wbbbf" Dec 03 13:17:55 crc kubenswrapper[5052]: I1203 13:17:55.413988 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["barbican-kuttl-tests/barbican-keystone-listener-66b47d8764-wbbbf"] Dec 03 13:17:55 crc kubenswrapper[5052]: I1203 13:17:55.431651 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/82893f5c-2185-4a6a-8b4a-a3d4c4dd92b5-config-data-custom\") pod \"barbican-api-75495b98c8-6rdjk\" (UID: \"82893f5c-2185-4a6a-8b4a-a3d4c4dd92b5\") " pod="barbican-kuttl-tests/barbican-api-75495b98c8-6rdjk" Dec 03 13:17:55 crc kubenswrapper[5052]: I1203 13:17:55.431730 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/82893f5c-2185-4a6a-8b4a-a3d4c4dd92b5-logs\") pod \"barbican-api-75495b98c8-6rdjk\" (UID: \"82893f5c-2185-4a6a-8b4a-a3d4c4dd92b5\") " pod="barbican-kuttl-tests/barbican-api-75495b98c8-6rdjk" Dec 03 13:17:55 crc kubenswrapper[5052]: I1203 13:17:55.431766 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/82893f5c-2185-4a6a-8b4a-a3d4c4dd92b5-config-data\") pod \"barbican-api-75495b98c8-6rdjk\" (UID: \"82893f5c-2185-4a6a-8b4a-a3d4c4dd92b5\") " pod="barbican-kuttl-tests/barbican-api-75495b98c8-6rdjk" Dec 03 13:17:55 crc kubenswrapper[5052]: I1203 13:17:55.431792 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-7kkkq\" (UniqueName: \"kubernetes.io/projected/82893f5c-2185-4a6a-8b4a-a3d4c4dd92b5-kube-api-access-7kkkq\") pod \"barbican-api-75495b98c8-6rdjk\" (UID: \"82893f5c-2185-4a6a-8b4a-a3d4c4dd92b5\") " pod="barbican-kuttl-tests/barbican-api-75495b98c8-6rdjk" Dec 03 13:17:55 crc kubenswrapper[5052]: I1203 13:17:55.433403 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/82893f5c-2185-4a6a-8b4a-a3d4c4dd92b5-logs\") pod \"barbican-api-75495b98c8-6rdjk\" (UID: \"82893f5c-2185-4a6a-8b4a-a3d4c4dd92b5\") " pod="barbican-kuttl-tests/barbican-api-75495b98c8-6rdjk" Dec 03 13:17:55 crc kubenswrapper[5052]: I1203 13:17:55.438002 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/82893f5c-2185-4a6a-8b4a-a3d4c4dd92b5-config-data-custom\") pod \"barbican-api-75495b98c8-6rdjk\" (UID: \"82893f5c-2185-4a6a-8b4a-a3d4c4dd92b5\") " pod="barbican-kuttl-tests/barbican-api-75495b98c8-6rdjk" Dec 03 13:17:55 crc kubenswrapper[5052]: I1203 13:17:55.438820 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/82893f5c-2185-4a6a-8b4a-a3d4c4dd92b5-config-data\") pod \"barbican-api-75495b98c8-6rdjk\" (UID: \"82893f5c-2185-4a6a-8b4a-a3d4c4dd92b5\") " pod="barbican-kuttl-tests/barbican-api-75495b98c8-6rdjk" Dec 03 13:17:55 crc kubenswrapper[5052]: I1203 13:17:55.460104 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-7kkkq\" (UniqueName: \"kubernetes.io/projected/82893f5c-2185-4a6a-8b4a-a3d4c4dd92b5-kube-api-access-7kkkq\") pod \"barbican-api-75495b98c8-6rdjk\" (UID: \"82893f5c-2185-4a6a-8b4a-a3d4c4dd92b5\") " pod="barbican-kuttl-tests/barbican-api-75495b98c8-6rdjk" Dec 03 13:17:55 crc kubenswrapper[5052]: I1203 13:17:55.534935 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/3676d121-81bf-431a-b180-39c950731a2f-config-data\") pod \"barbican-keystone-listener-66b47d8764-wbbbf\" (UID: \"3676d121-81bf-431a-b180-39c950731a2f\") " pod="barbican-kuttl-tests/barbican-keystone-listener-66b47d8764-wbbbf" Dec 03 13:17:55 crc kubenswrapper[5052]: I1203 13:17:55.535025 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/3676d121-81bf-431a-b180-39c950731a2f-config-data-custom\") pod \"barbican-keystone-listener-66b47d8764-wbbbf\" (UID: \"3676d121-81bf-431a-b180-39c950731a2f\") " pod="barbican-kuttl-tests/barbican-keystone-listener-66b47d8764-wbbbf" Dec 03 13:17:55 crc kubenswrapper[5052]: I1203 13:17:55.535120 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jxrdv\" (UniqueName: \"kubernetes.io/projected/3676d121-81bf-431a-b180-39c950731a2f-kube-api-access-jxrdv\") pod \"barbican-keystone-listener-66b47d8764-wbbbf\" (UID: \"3676d121-81bf-431a-b180-39c950731a2f\") " pod="barbican-kuttl-tests/barbican-keystone-listener-66b47d8764-wbbbf" Dec 03 13:17:55 crc kubenswrapper[5052]: I1203 13:17:55.535142 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/3676d121-81bf-431a-b180-39c950731a2f-logs\") pod \"barbican-keystone-listener-66b47d8764-wbbbf\" (UID: \"3676d121-81bf-431a-b180-39c950731a2f\") " pod="barbican-kuttl-tests/barbican-keystone-listener-66b47d8764-wbbbf" Dec 03 13:17:55 crc kubenswrapper[5052]: I1203 13:17:55.539416 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="barbican-kuttl-tests/barbican-api-75495b98c8-6rdjk" Dec 03 13:17:55 crc kubenswrapper[5052]: I1203 13:17:55.627012 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["barbican-kuttl-tests/barbican-worker-74467844f5-llwv8"] Dec 03 13:17:55 crc kubenswrapper[5052]: I1203 13:17:55.630640 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="barbican-kuttl-tests/barbican-worker-74467844f5-llwv8" Dec 03 13:17:55 crc kubenswrapper[5052]: I1203 13:17:55.637326 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/3676d121-81bf-431a-b180-39c950731a2f-config-data\") pod \"barbican-keystone-listener-66b47d8764-wbbbf\" (UID: \"3676d121-81bf-431a-b180-39c950731a2f\") " pod="barbican-kuttl-tests/barbican-keystone-listener-66b47d8764-wbbbf" Dec 03 13:17:55 crc kubenswrapper[5052]: I1203 13:17:55.637433 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/3676d121-81bf-431a-b180-39c950731a2f-config-data-custom\") pod \"barbican-keystone-listener-66b47d8764-wbbbf\" (UID: \"3676d121-81bf-431a-b180-39c950731a2f\") " pod="barbican-kuttl-tests/barbican-keystone-listener-66b47d8764-wbbbf" Dec 03 13:17:55 crc kubenswrapper[5052]: I1203 13:17:55.637497 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-jxrdv\" (UniqueName: \"kubernetes.io/projected/3676d121-81bf-431a-b180-39c950731a2f-kube-api-access-jxrdv\") pod \"barbican-keystone-listener-66b47d8764-wbbbf\" (UID: \"3676d121-81bf-431a-b180-39c950731a2f\") " pod="barbican-kuttl-tests/barbican-keystone-listener-66b47d8764-wbbbf" Dec 03 13:17:55 crc kubenswrapper[5052]: I1203 13:17:55.637530 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/3676d121-81bf-431a-b180-39c950731a2f-logs\") pod \"barbican-keystone-listener-66b47d8764-wbbbf\" (UID: \"3676d121-81bf-431a-b180-39c950731a2f\") " pod="barbican-kuttl-tests/barbican-keystone-listener-66b47d8764-wbbbf" Dec 03 13:17:55 crc kubenswrapper[5052]: I1203 13:17:55.638941 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/3676d121-81bf-431a-b180-39c950731a2f-logs\") pod \"barbican-keystone-listener-66b47d8764-wbbbf\" (UID: \"3676d121-81bf-431a-b180-39c950731a2f\") " pod="barbican-kuttl-tests/barbican-keystone-listener-66b47d8764-wbbbf" Dec 03 13:17:55 crc kubenswrapper[5052]: I1203 13:17:55.643909 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/3676d121-81bf-431a-b180-39c950731a2f-config-data\") pod \"barbican-keystone-listener-66b47d8764-wbbbf\" (UID: \"3676d121-81bf-431a-b180-39c950731a2f\") " pod="barbican-kuttl-tests/barbican-keystone-listener-66b47d8764-wbbbf" Dec 03 13:17:55 crc kubenswrapper[5052]: I1203 13:17:55.645329 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/3676d121-81bf-431a-b180-39c950731a2f-config-data-custom\") pod \"barbican-keystone-listener-66b47d8764-wbbbf\" (UID: \"3676d121-81bf-431a-b180-39c950731a2f\") " pod="barbican-kuttl-tests/barbican-keystone-listener-66b47d8764-wbbbf" Dec 03 13:17:55 crc kubenswrapper[5052]: I1203 13:17:55.651089 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["barbican-kuttl-tests/barbican-worker-74467844f5-llwv8"] Dec 03 13:17:55 crc kubenswrapper[5052]: I1203 13:17:55.667560 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-jxrdv\" (UniqueName: \"kubernetes.io/projected/3676d121-81bf-431a-b180-39c950731a2f-kube-api-access-jxrdv\") pod \"barbican-keystone-listener-66b47d8764-wbbbf\" (UID: \"3676d121-81bf-431a-b180-39c950731a2f\") " pod="barbican-kuttl-tests/barbican-keystone-listener-66b47d8764-wbbbf" Dec 03 13:17:55 crc kubenswrapper[5052]: I1203 13:17:55.739052 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/e033a810-2874-4fb9-ab5b-654fb638cf18-logs\") pod \"barbican-worker-74467844f5-llwv8\" (UID: \"e033a810-2874-4fb9-ab5b-654fb638cf18\") " pod="barbican-kuttl-tests/barbican-worker-74467844f5-llwv8" Dec 03 13:17:55 crc kubenswrapper[5052]: I1203 13:17:55.739116 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zshmv\" (UniqueName: \"kubernetes.io/projected/e033a810-2874-4fb9-ab5b-654fb638cf18-kube-api-access-zshmv\") pod \"barbican-worker-74467844f5-llwv8\" (UID: \"e033a810-2874-4fb9-ab5b-654fb638cf18\") " pod="barbican-kuttl-tests/barbican-worker-74467844f5-llwv8" Dec 03 13:17:55 crc kubenswrapper[5052]: I1203 13:17:55.739556 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/e033a810-2874-4fb9-ab5b-654fb638cf18-config-data-custom\") pod \"barbican-worker-74467844f5-llwv8\" (UID: \"e033a810-2874-4fb9-ab5b-654fb638cf18\") " pod="barbican-kuttl-tests/barbican-worker-74467844f5-llwv8" Dec 03 13:17:55 crc kubenswrapper[5052]: I1203 13:17:55.739732 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/e033a810-2874-4fb9-ab5b-654fb638cf18-config-data\") pod \"barbican-worker-74467844f5-llwv8\" (UID: \"e033a810-2874-4fb9-ab5b-654fb638cf18\") " pod="barbican-kuttl-tests/barbican-worker-74467844f5-llwv8" Dec 03 13:17:55 crc kubenswrapper[5052]: I1203 13:17:55.758068 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="barbican-kuttl-tests/barbican-keystone-listener-66b47d8764-wbbbf" Dec 03 13:17:55 crc kubenswrapper[5052]: I1203 13:17:55.841631 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/e033a810-2874-4fb9-ab5b-654fb638cf18-logs\") pod \"barbican-worker-74467844f5-llwv8\" (UID: \"e033a810-2874-4fb9-ab5b-654fb638cf18\") " pod="barbican-kuttl-tests/barbican-worker-74467844f5-llwv8" Dec 03 13:17:55 crc kubenswrapper[5052]: I1203 13:17:55.842014 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-zshmv\" (UniqueName: \"kubernetes.io/projected/e033a810-2874-4fb9-ab5b-654fb638cf18-kube-api-access-zshmv\") pod \"barbican-worker-74467844f5-llwv8\" (UID: \"e033a810-2874-4fb9-ab5b-654fb638cf18\") " pod="barbican-kuttl-tests/barbican-worker-74467844f5-llwv8" Dec 03 13:17:55 crc kubenswrapper[5052]: I1203 13:17:55.842155 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/e033a810-2874-4fb9-ab5b-654fb638cf18-config-data-custom\") pod \"barbican-worker-74467844f5-llwv8\" (UID: \"e033a810-2874-4fb9-ab5b-654fb638cf18\") " pod="barbican-kuttl-tests/barbican-worker-74467844f5-llwv8" Dec 03 13:17:55 crc kubenswrapper[5052]: I1203 13:17:55.842206 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/e033a810-2874-4fb9-ab5b-654fb638cf18-config-data\") pod \"barbican-worker-74467844f5-llwv8\" (UID: \"e033a810-2874-4fb9-ab5b-654fb638cf18\") " pod="barbican-kuttl-tests/barbican-worker-74467844f5-llwv8" Dec 03 13:17:55 crc kubenswrapper[5052]: I1203 13:17:55.851630 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/e033a810-2874-4fb9-ab5b-654fb638cf18-logs\") pod \"barbican-worker-74467844f5-llwv8\" (UID: \"e033a810-2874-4fb9-ab5b-654fb638cf18\") " pod="barbican-kuttl-tests/barbican-worker-74467844f5-llwv8" Dec 03 13:17:55 crc kubenswrapper[5052]: I1203 13:17:55.858161 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/e033a810-2874-4fb9-ab5b-654fb638cf18-config-data\") pod \"barbican-worker-74467844f5-llwv8\" (UID: \"e033a810-2874-4fb9-ab5b-654fb638cf18\") " pod="barbican-kuttl-tests/barbican-worker-74467844f5-llwv8" Dec 03 13:17:55 crc kubenswrapper[5052]: I1203 13:17:55.859351 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/e033a810-2874-4fb9-ab5b-654fb638cf18-config-data-custom\") pod \"barbican-worker-74467844f5-llwv8\" (UID: \"e033a810-2874-4fb9-ab5b-654fb638cf18\") " pod="barbican-kuttl-tests/barbican-worker-74467844f5-llwv8" Dec 03 13:17:55 crc kubenswrapper[5052]: I1203 13:17:55.877458 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-zshmv\" (UniqueName: \"kubernetes.io/projected/e033a810-2874-4fb9-ab5b-654fb638cf18-kube-api-access-zshmv\") pod \"barbican-worker-74467844f5-llwv8\" (UID: \"e033a810-2874-4fb9-ab5b-654fb638cf18\") " pod="barbican-kuttl-tests/barbican-worker-74467844f5-llwv8" Dec 03 13:17:55 crc kubenswrapper[5052]: I1203 13:17:55.917094 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="barbican-kuttl-tests/barbican-worker-74467844f5-llwv8" Dec 03 13:17:56 crc kubenswrapper[5052]: I1203 13:17:56.116051 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["barbican-kuttl-tests/barbican-api-75495b98c8-6rdjk"] Dec 03 13:17:56 crc kubenswrapper[5052]: I1203 13:17:56.124351 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="barbican-kuttl-tests/barbican-keystone-listener-66b47d8764-7bflh" event={"ID":"3a71e182-1787-48f5-a4f8-60b9fb56f603","Type":"ContainerStarted","Data":"77fdae1edc081d92e84e48b069e63e28b031ef998432a493107b3427449baa2d"} Dec 03 13:17:56 crc kubenswrapper[5052]: I1203 13:17:56.124398 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="barbican-kuttl-tests/barbican-keystone-listener-66b47d8764-7bflh" event={"ID":"3a71e182-1787-48f5-a4f8-60b9fb56f603","Type":"ContainerStarted","Data":"e6af7006ba7fafb6119a764132ef9d62ff06ccbc771e754f553f0acd3a7ab348"} Dec 03 13:17:56 crc kubenswrapper[5052]: W1203 13:17:56.136285 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod82893f5c_2185_4a6a_8b4a_a3d4c4dd92b5.slice/crio-2c978c6a9e9bb40e9a017cbca9f2673b198ce71de6e8243ccd0bf06db1017796 WatchSource:0}: Error finding container 2c978c6a9e9bb40e9a017cbca9f2673b198ce71de6e8243ccd0bf06db1017796: Status 404 returned error can't find the container with id 2c978c6a9e9bb40e9a017cbca9f2673b198ce71de6e8243ccd0bf06db1017796 Dec 03 13:17:56 crc kubenswrapper[5052]: I1203 13:17:56.136891 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="barbican-kuttl-tests/barbican-api-75495b98c8-4vgmd" event={"ID":"47809cbf-0aa5-4d38-8b9f-95ce55763ed3","Type":"ContainerStarted","Data":"c741ab94362e9e80b0eebbdccb461275bde7eb198357602ecf738959164f75cd"} Dec 03 13:17:56 crc kubenswrapper[5052]: I1203 13:17:56.136927 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="barbican-kuttl-tests/barbican-api-75495b98c8-4vgmd" event={"ID":"47809cbf-0aa5-4d38-8b9f-95ce55763ed3","Type":"ContainerStarted","Data":"ea7dfeda9cfa835bba5e01e59ae1970fffc8bed2ac1b9377849a24e4e02b17ec"} Dec 03 13:17:56 crc kubenswrapper[5052]: I1203 13:17:56.136936 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="barbican-kuttl-tests/barbican-api-75495b98c8-4vgmd" event={"ID":"47809cbf-0aa5-4d38-8b9f-95ce55763ed3","Type":"ContainerStarted","Data":"264156498b352d98a23b875a3e46d95ac0b1a0134bf5ae9579feba3d3ffbf1ab"} Dec 03 13:17:56 crc kubenswrapper[5052]: I1203 13:17:56.137633 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="barbican-kuttl-tests/barbican-api-75495b98c8-4vgmd" Dec 03 13:17:56 crc kubenswrapper[5052]: I1203 13:17:56.137684 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="barbican-kuttl-tests/barbican-api-75495b98c8-4vgmd" Dec 03 13:17:56 crc kubenswrapper[5052]: I1203 13:17:56.139739 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="barbican-kuttl-tests/barbican-worker-74467844f5-w7kwd" event={"ID":"72c65803-7d22-4b27-9cfc-b8a246a0ba7a","Type":"ContainerStarted","Data":"1a6e83c31c16b733e0a9dddbbfeffb1a8033e40fe0d2444509c93a09efb4dcf5"} Dec 03 13:17:56 crc kubenswrapper[5052]: I1203 13:17:56.139783 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="barbican-kuttl-tests/barbican-worker-74467844f5-w7kwd" event={"ID":"72c65803-7d22-4b27-9cfc-b8a246a0ba7a","Type":"ContainerStarted","Data":"d2f2f65968f7b9706a4750c08370f44252c83ac459cdcabb00e2ca4e8ab28287"} Dec 03 13:17:56 crc kubenswrapper[5052]: I1203 13:17:56.170268 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="barbican-kuttl-tests/barbican-api-75495b98c8-4vgmd" podStartSLOduration=2.170249225 podStartE2EDuration="2.170249225s" podCreationTimestamp="2025-12-03 13:17:54 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 13:17:56.160041102 +0000 UTC m=+7066.748282926" watchObservedRunningTime="2025-12-03 13:17:56.170249225 +0000 UTC m=+7066.758490989" Dec 03 13:17:56 crc kubenswrapper[5052]: I1203 13:17:56.223385 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["barbican-kuttl-tests/barbican-keystone-listener-66b47d8764-wbbbf"] Dec 03 13:17:56 crc kubenswrapper[5052]: I1203 13:17:56.398265 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["barbican-kuttl-tests/barbican-worker-74467844f5-llwv8"] Dec 03 13:17:56 crc kubenswrapper[5052]: I1203 13:17:56.934557 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["barbican-kuttl-tests/barbican-api-75495b98c8-6rdjk"] Dec 03 13:17:57 crc kubenswrapper[5052]: I1203 13:17:57.123595 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["barbican-kuttl-tests/barbican-keystone-listener-66b47d8764-wbbbf"] Dec 03 13:17:57 crc kubenswrapper[5052]: I1203 13:17:57.148955 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="barbican-kuttl-tests/barbican-worker-74467844f5-w7kwd" event={"ID":"72c65803-7d22-4b27-9cfc-b8a246a0ba7a","Type":"ContainerStarted","Data":"9002ed94299a72447f2ba07c9dff1796e735f445645d52ddc8ab402fc97b4dce"} Dec 03 13:17:57 crc kubenswrapper[5052]: I1203 13:17:57.152981 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="barbican-kuttl-tests/barbican-worker-74467844f5-llwv8" event={"ID":"e033a810-2874-4fb9-ab5b-654fb638cf18","Type":"ContainerStarted","Data":"5572642d7c888f6ff77ddabfc745ab3a1eafa82c40c49357c61e77d9a54c3aef"} Dec 03 13:17:57 crc kubenswrapper[5052]: I1203 13:17:57.153021 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="barbican-kuttl-tests/barbican-worker-74467844f5-llwv8" event={"ID":"e033a810-2874-4fb9-ab5b-654fb638cf18","Type":"ContainerStarted","Data":"b5bb49d9158f4c1812ee278785430bfc1cb627406b76f99d62ed896e546bc0b9"} Dec 03 13:17:57 crc kubenswrapper[5052]: I1203 13:17:57.153030 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="barbican-kuttl-tests/barbican-worker-74467844f5-llwv8" event={"ID":"e033a810-2874-4fb9-ab5b-654fb638cf18","Type":"ContainerStarted","Data":"b7afb54f639942d174ed5592fdc147498de1e6077ea9f4cbbedffcabf9a3260e"} Dec 03 13:17:57 crc kubenswrapper[5052]: I1203 13:17:57.156079 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="barbican-kuttl-tests/barbican-keystone-listener-66b47d8764-7bflh" event={"ID":"3a71e182-1787-48f5-a4f8-60b9fb56f603","Type":"ContainerStarted","Data":"216a92996bad8313803338c34e597edfe8e9388bfe6f7735837c64532f3c7f37"} Dec 03 13:17:57 crc kubenswrapper[5052]: I1203 13:17:57.159858 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="barbican-kuttl-tests/barbican-api-75495b98c8-6rdjk" event={"ID":"82893f5c-2185-4a6a-8b4a-a3d4c4dd92b5","Type":"ContainerStarted","Data":"029a009230f1ba16341785fd2ffd862ac155d705c88fe1b90a91d80376a7b91e"} Dec 03 13:17:57 crc kubenswrapper[5052]: I1203 13:17:57.159928 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="barbican-kuttl-tests/barbican-api-75495b98c8-6rdjk" event={"ID":"82893f5c-2185-4a6a-8b4a-a3d4c4dd92b5","Type":"ContainerStarted","Data":"6040073ae04bab570414042d48ee0cdd54b0c398e72aa0ee54835dc801126797"} Dec 03 13:17:57 crc kubenswrapper[5052]: I1203 13:17:57.159943 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="barbican-kuttl-tests/barbican-api-75495b98c8-6rdjk" event={"ID":"82893f5c-2185-4a6a-8b4a-a3d4c4dd92b5","Type":"ContainerStarted","Data":"2c978c6a9e9bb40e9a017cbca9f2673b198ce71de6e8243ccd0bf06db1017796"} Dec 03 13:17:57 crc kubenswrapper[5052]: I1203 13:17:57.160586 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="barbican-kuttl-tests/barbican-api-75495b98c8-6rdjk" Dec 03 13:17:57 crc kubenswrapper[5052]: I1203 13:17:57.160627 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="barbican-kuttl-tests/barbican-api-75495b98c8-6rdjk" Dec 03 13:17:57 crc kubenswrapper[5052]: I1203 13:17:57.163720 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="barbican-kuttl-tests/barbican-keystone-listener-66b47d8764-wbbbf" event={"ID":"3676d121-81bf-431a-b180-39c950731a2f","Type":"ContainerStarted","Data":"4ba3d3e87a26e0d6d7240b66b881b51bccdf1a55ddfece30f153f6571cb9deb5"} Dec 03 13:17:57 crc kubenswrapper[5052]: I1203 13:17:57.163763 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="barbican-kuttl-tests/barbican-keystone-listener-66b47d8764-wbbbf" event={"ID":"3676d121-81bf-431a-b180-39c950731a2f","Type":"ContainerStarted","Data":"13ad56c24c7d9b6fbee10e8632bad5f404915f724a561319e8f88cbcaa76b786"} Dec 03 13:17:57 crc kubenswrapper[5052]: I1203 13:17:57.163779 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="barbican-kuttl-tests/barbican-keystone-listener-66b47d8764-wbbbf" event={"ID":"3676d121-81bf-431a-b180-39c950731a2f","Type":"ContainerStarted","Data":"d45055c6ef7825aad860064b9f9481e3ce3d665c71b2f5632d8c044bb106ec80"} Dec 03 13:17:57 crc kubenswrapper[5052]: I1203 13:17:57.182124 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="barbican-kuttl-tests/barbican-worker-74467844f5-w7kwd" podStartSLOduration=2.671278884 podStartE2EDuration="3.182105132s" podCreationTimestamp="2025-12-03 13:17:54 +0000 UTC" firstStartedPulling="2025-12-03 13:17:55.167944041 +0000 UTC m=+7065.756185805" lastFinishedPulling="2025-12-03 13:17:55.678770289 +0000 UTC m=+7066.267012053" observedRunningTime="2025-12-03 13:17:57.173420013 +0000 UTC m=+7067.761661777" watchObservedRunningTime="2025-12-03 13:17:57.182105132 +0000 UTC m=+7067.770346896" Dec 03 13:17:57 crc kubenswrapper[5052]: I1203 13:17:57.209446 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="barbican-kuttl-tests/barbican-api-75495b98c8-6rdjk" podStartSLOduration=2.209420374 podStartE2EDuration="2.209420374s" podCreationTimestamp="2025-12-03 13:17:55 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 13:17:57.203422602 +0000 UTC m=+7067.791664386" watchObservedRunningTime="2025-12-03 13:17:57.209420374 +0000 UTC m=+7067.797662138" Dec 03 13:17:57 crc kubenswrapper[5052]: I1203 13:17:57.233045 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="barbican-kuttl-tests/barbican-keystone-listener-66b47d8764-wbbbf" podStartSLOduration=2.23302131 podStartE2EDuration="2.23302131s" podCreationTimestamp="2025-12-03 13:17:55 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 13:17:57.230383975 +0000 UTC m=+7067.818625749" watchObservedRunningTime="2025-12-03 13:17:57.23302131 +0000 UTC m=+7067.821263074" Dec 03 13:17:57 crc kubenswrapper[5052]: I1203 13:17:57.258279 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="barbican-kuttl-tests/barbican-worker-74467844f5-llwv8" podStartSLOduration=2.258256903 podStartE2EDuration="2.258256903s" podCreationTimestamp="2025-12-03 13:17:55 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 13:17:57.253840006 +0000 UTC m=+7067.842081770" watchObservedRunningTime="2025-12-03 13:17:57.258256903 +0000 UTC m=+7067.846498667" Dec 03 13:17:57 crc kubenswrapper[5052]: I1203 13:17:57.274166 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="barbican-kuttl-tests/barbican-keystone-listener-66b47d8764-7bflh" podStartSLOduration=2.7590648570000003 podStartE2EDuration="3.274149498s" podCreationTimestamp="2025-12-03 13:17:54 +0000 UTC" firstStartedPulling="2025-12-03 13:17:55.096049972 +0000 UTC m=+7065.684291726" lastFinishedPulling="2025-12-03 13:17:55.611134603 +0000 UTC m=+7066.199376367" observedRunningTime="2025-12-03 13:17:57.273090508 +0000 UTC m=+7067.861332272" watchObservedRunningTime="2025-12-03 13:17:57.274149498 +0000 UTC m=+7067.862391262" Dec 03 13:17:57 crc kubenswrapper[5052]: I1203 13:17:57.336718 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["barbican-kuttl-tests/barbican-worker-74467844f5-w7kwd"] Dec 03 13:17:58 crc kubenswrapper[5052]: I1203 13:17:58.171725 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="barbican-kuttl-tests/barbican-api-75495b98c8-6rdjk" podUID="82893f5c-2185-4a6a-8b4a-a3d4c4dd92b5" containerName="barbican-api-log" containerID="cri-o://6040073ae04bab570414042d48ee0cdd54b0c398e72aa0ee54835dc801126797" gracePeriod=30 Dec 03 13:17:58 crc kubenswrapper[5052]: I1203 13:17:58.173296 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="barbican-kuttl-tests/barbican-keystone-listener-66b47d8764-wbbbf" podUID="3676d121-81bf-431a-b180-39c950731a2f" containerName="barbican-keystone-listener-log" containerID="cri-o://13ad56c24c7d9b6fbee10e8632bad5f404915f724a561319e8f88cbcaa76b786" gracePeriod=30 Dec 03 13:17:58 crc kubenswrapper[5052]: I1203 13:17:58.173414 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="barbican-kuttl-tests/barbican-api-75495b98c8-6rdjk" podUID="82893f5c-2185-4a6a-8b4a-a3d4c4dd92b5" containerName="barbican-api" containerID="cri-o://029a009230f1ba16341785fd2ffd862ac155d705c88fe1b90a91d80376a7b91e" gracePeriod=30 Dec 03 13:17:58 crc kubenswrapper[5052]: I1203 13:17:58.173627 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="barbican-kuttl-tests/barbican-keystone-listener-66b47d8764-wbbbf" podUID="3676d121-81bf-431a-b180-39c950731a2f" containerName="barbican-keystone-listener" containerID="cri-o://4ba3d3e87a26e0d6d7240b66b881b51bccdf1a55ddfece30f153f6571cb9deb5" gracePeriod=30 Dec 03 13:17:58 crc kubenswrapper[5052]: I1203 13:17:58.504586 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["barbican-kuttl-tests/barbican-api-75495b98c8-4vgmd"] Dec 03 13:17:58 crc kubenswrapper[5052]: I1203 13:17:58.771958 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["barbican-kuttl-tests/barbican-keystone-listener-66b47d8764-7bflh"] Dec 03 13:17:58 crc kubenswrapper[5052]: I1203 13:17:58.875253 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["barbican-kuttl-tests/barbican-worker-74467844f5-llwv8"] Dec 03 13:17:59 crc kubenswrapper[5052]: I1203 13:17:59.170767 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="barbican-kuttl-tests/barbican-api-75495b98c8-6rdjk" Dec 03 13:17:59 crc kubenswrapper[5052]: I1203 13:17:59.181502 5052 generic.go:334] "Generic (PLEG): container finished" podID="82893f5c-2185-4a6a-8b4a-a3d4c4dd92b5" containerID="029a009230f1ba16341785fd2ffd862ac155d705c88fe1b90a91d80376a7b91e" exitCode=0 Dec 03 13:17:59 crc kubenswrapper[5052]: I1203 13:17:59.181562 5052 generic.go:334] "Generic (PLEG): container finished" podID="82893f5c-2185-4a6a-8b4a-a3d4c4dd92b5" containerID="6040073ae04bab570414042d48ee0cdd54b0c398e72aa0ee54835dc801126797" exitCode=143 Dec 03 13:17:59 crc kubenswrapper[5052]: I1203 13:17:59.181579 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="barbican-kuttl-tests/barbican-api-75495b98c8-6rdjk" event={"ID":"82893f5c-2185-4a6a-8b4a-a3d4c4dd92b5","Type":"ContainerDied","Data":"029a009230f1ba16341785fd2ffd862ac155d705c88fe1b90a91d80376a7b91e"} Dec 03 13:17:59 crc kubenswrapper[5052]: I1203 13:17:59.181647 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="barbican-kuttl-tests/barbican-api-75495b98c8-6rdjk" event={"ID":"82893f5c-2185-4a6a-8b4a-a3d4c4dd92b5","Type":"ContainerDied","Data":"6040073ae04bab570414042d48ee0cdd54b0c398e72aa0ee54835dc801126797"} Dec 03 13:17:59 crc kubenswrapper[5052]: I1203 13:17:59.181666 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="barbican-kuttl-tests/barbican-api-75495b98c8-6rdjk" event={"ID":"82893f5c-2185-4a6a-8b4a-a3d4c4dd92b5","Type":"ContainerDied","Data":"2c978c6a9e9bb40e9a017cbca9f2673b198ce71de6e8243ccd0bf06db1017796"} Dec 03 13:17:59 crc kubenswrapper[5052]: I1203 13:17:59.181654 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="barbican-kuttl-tests/barbican-api-75495b98c8-6rdjk" Dec 03 13:17:59 crc kubenswrapper[5052]: I1203 13:17:59.181688 5052 scope.go:117] "RemoveContainer" containerID="029a009230f1ba16341785fd2ffd862ac155d705c88fe1b90a91d80376a7b91e" Dec 03 13:17:59 crc kubenswrapper[5052]: I1203 13:17:59.183366 5052 generic.go:334] "Generic (PLEG): container finished" podID="3676d121-81bf-431a-b180-39c950731a2f" containerID="13ad56c24c7d9b6fbee10e8632bad5f404915f724a561319e8f88cbcaa76b786" exitCode=143 Dec 03 13:17:59 crc kubenswrapper[5052]: I1203 13:17:59.183489 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="barbican-kuttl-tests/barbican-keystone-listener-66b47d8764-wbbbf" event={"ID":"3676d121-81bf-431a-b180-39c950731a2f","Type":"ContainerDied","Data":"13ad56c24c7d9b6fbee10e8632bad5f404915f724a561319e8f88cbcaa76b786"} Dec 03 13:17:59 crc kubenswrapper[5052]: I1203 13:17:59.183595 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="barbican-kuttl-tests/barbican-worker-74467844f5-llwv8" podUID="e033a810-2874-4fb9-ab5b-654fb638cf18" containerName="barbican-worker-log" containerID="cri-o://b5bb49d9158f4c1812ee278785430bfc1cb627406b76f99d62ed896e546bc0b9" gracePeriod=30 Dec 03 13:17:59 crc kubenswrapper[5052]: I1203 13:17:59.183680 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="barbican-kuttl-tests/barbican-worker-74467844f5-llwv8" podUID="e033a810-2874-4fb9-ab5b-654fb638cf18" containerName="barbican-worker" containerID="cri-o://5572642d7c888f6ff77ddabfc745ab3a1eafa82c40c49357c61e77d9a54c3aef" gracePeriod=30 Dec 03 13:17:59 crc kubenswrapper[5052]: I1203 13:17:59.183966 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="barbican-kuttl-tests/barbican-api-75495b98c8-4vgmd" podUID="47809cbf-0aa5-4d38-8b9f-95ce55763ed3" containerName="barbican-api-log" containerID="cri-o://ea7dfeda9cfa835bba5e01e59ae1970fffc8bed2ac1b9377849a24e4e02b17ec" gracePeriod=30 Dec 03 13:17:59 crc kubenswrapper[5052]: I1203 13:17:59.184093 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="barbican-kuttl-tests/barbican-api-75495b98c8-4vgmd" podUID="47809cbf-0aa5-4d38-8b9f-95ce55763ed3" containerName="barbican-api" containerID="cri-o://c741ab94362e9e80b0eebbdccb461275bde7eb198357602ecf738959164f75cd" gracePeriod=30 Dec 03 13:17:59 crc kubenswrapper[5052]: I1203 13:17:59.184186 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="barbican-kuttl-tests/barbican-worker-74467844f5-w7kwd" podUID="72c65803-7d22-4b27-9cfc-b8a246a0ba7a" containerName="barbican-worker-log" containerID="cri-o://1a6e83c31c16b733e0a9dddbbfeffb1a8033e40fe0d2444509c93a09efb4dcf5" gracePeriod=30 Dec 03 13:17:59 crc kubenswrapper[5052]: I1203 13:17:59.184230 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="barbican-kuttl-tests/barbican-worker-74467844f5-w7kwd" podUID="72c65803-7d22-4b27-9cfc-b8a246a0ba7a" containerName="barbican-worker" containerID="cri-o://9002ed94299a72447f2ba07c9dff1796e735f445645d52ddc8ab402fc97b4dce" gracePeriod=30 Dec 03 13:17:59 crc kubenswrapper[5052]: I1203 13:17:59.184287 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="barbican-kuttl-tests/barbican-keystone-listener-66b47d8764-7bflh" podUID="3a71e182-1787-48f5-a4f8-60b9fb56f603" containerName="barbican-keystone-listener-log" containerID="cri-o://77fdae1edc081d92e84e48b069e63e28b031ef998432a493107b3427449baa2d" gracePeriod=30 Dec 03 13:17:59 crc kubenswrapper[5052]: I1203 13:17:59.184296 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="barbican-kuttl-tests/barbican-keystone-listener-66b47d8764-7bflh" podUID="3a71e182-1787-48f5-a4f8-60b9fb56f603" containerName="barbican-keystone-listener" containerID="cri-o://216a92996bad8313803338c34e597edfe8e9388bfe6f7735837c64532f3c7f37" gracePeriod=30 Dec 03 13:17:59 crc kubenswrapper[5052]: I1203 13:17:59.232078 5052 scope.go:117] "RemoveContainer" containerID="6040073ae04bab570414042d48ee0cdd54b0c398e72aa0ee54835dc801126797" Dec 03 13:17:59 crc kubenswrapper[5052]: I1203 13:17:59.250419 5052 scope.go:117] "RemoveContainer" containerID="029a009230f1ba16341785fd2ffd862ac155d705c88fe1b90a91d80376a7b91e" Dec 03 13:17:59 crc kubenswrapper[5052]: E1203 13:17:59.250941 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"029a009230f1ba16341785fd2ffd862ac155d705c88fe1b90a91d80376a7b91e\": container with ID starting with 029a009230f1ba16341785fd2ffd862ac155d705c88fe1b90a91d80376a7b91e not found: ID does not exist" containerID="029a009230f1ba16341785fd2ffd862ac155d705c88fe1b90a91d80376a7b91e" Dec 03 13:17:59 crc kubenswrapper[5052]: I1203 13:17:59.250988 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"029a009230f1ba16341785fd2ffd862ac155d705c88fe1b90a91d80376a7b91e"} err="failed to get container status \"029a009230f1ba16341785fd2ffd862ac155d705c88fe1b90a91d80376a7b91e\": rpc error: code = NotFound desc = could not find container \"029a009230f1ba16341785fd2ffd862ac155d705c88fe1b90a91d80376a7b91e\": container with ID starting with 029a009230f1ba16341785fd2ffd862ac155d705c88fe1b90a91d80376a7b91e not found: ID does not exist" Dec 03 13:17:59 crc kubenswrapper[5052]: I1203 13:17:59.251008 5052 scope.go:117] "RemoveContainer" containerID="6040073ae04bab570414042d48ee0cdd54b0c398e72aa0ee54835dc801126797" Dec 03 13:17:59 crc kubenswrapper[5052]: E1203 13:17:59.251192 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6040073ae04bab570414042d48ee0cdd54b0c398e72aa0ee54835dc801126797\": container with ID starting with 6040073ae04bab570414042d48ee0cdd54b0c398e72aa0ee54835dc801126797 not found: ID does not exist" containerID="6040073ae04bab570414042d48ee0cdd54b0c398e72aa0ee54835dc801126797" Dec 03 13:17:59 crc kubenswrapper[5052]: I1203 13:17:59.251223 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6040073ae04bab570414042d48ee0cdd54b0c398e72aa0ee54835dc801126797"} err="failed to get container status \"6040073ae04bab570414042d48ee0cdd54b0c398e72aa0ee54835dc801126797\": rpc error: code = NotFound desc = could not find container \"6040073ae04bab570414042d48ee0cdd54b0c398e72aa0ee54835dc801126797\": container with ID starting with 6040073ae04bab570414042d48ee0cdd54b0c398e72aa0ee54835dc801126797 not found: ID does not exist" Dec 03 13:17:59 crc kubenswrapper[5052]: I1203 13:17:59.251237 5052 scope.go:117] "RemoveContainer" containerID="029a009230f1ba16341785fd2ffd862ac155d705c88fe1b90a91d80376a7b91e" Dec 03 13:17:59 crc kubenswrapper[5052]: I1203 13:17:59.251421 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"029a009230f1ba16341785fd2ffd862ac155d705c88fe1b90a91d80376a7b91e"} err="failed to get container status \"029a009230f1ba16341785fd2ffd862ac155d705c88fe1b90a91d80376a7b91e\": rpc error: code = NotFound desc = could not find container \"029a009230f1ba16341785fd2ffd862ac155d705c88fe1b90a91d80376a7b91e\": container with ID starting with 029a009230f1ba16341785fd2ffd862ac155d705c88fe1b90a91d80376a7b91e not found: ID does not exist" Dec 03 13:17:59 crc kubenswrapper[5052]: I1203 13:17:59.251459 5052 scope.go:117] "RemoveContainer" containerID="6040073ae04bab570414042d48ee0cdd54b0c398e72aa0ee54835dc801126797" Dec 03 13:17:59 crc kubenswrapper[5052]: I1203 13:17:59.251868 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6040073ae04bab570414042d48ee0cdd54b0c398e72aa0ee54835dc801126797"} err="failed to get container status \"6040073ae04bab570414042d48ee0cdd54b0c398e72aa0ee54835dc801126797\": rpc error: code = NotFound desc = could not find container \"6040073ae04bab570414042d48ee0cdd54b0c398e72aa0ee54835dc801126797\": container with ID starting with 6040073ae04bab570414042d48ee0cdd54b0c398e72aa0ee54835dc801126797 not found: ID does not exist" Dec 03 13:17:59 crc kubenswrapper[5052]: I1203 13:17:59.302364 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/82893f5c-2185-4a6a-8b4a-a3d4c4dd92b5-logs\") pod \"82893f5c-2185-4a6a-8b4a-a3d4c4dd92b5\" (UID: \"82893f5c-2185-4a6a-8b4a-a3d4c4dd92b5\") " Dec 03 13:17:59 crc kubenswrapper[5052]: I1203 13:17:59.302592 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/82893f5c-2185-4a6a-8b4a-a3d4c4dd92b5-config-data\") pod \"82893f5c-2185-4a6a-8b4a-a3d4c4dd92b5\" (UID: \"82893f5c-2185-4a6a-8b4a-a3d4c4dd92b5\") " Dec 03 13:17:59 crc kubenswrapper[5052]: I1203 13:17:59.302622 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/82893f5c-2185-4a6a-8b4a-a3d4c4dd92b5-config-data-custom\") pod \"82893f5c-2185-4a6a-8b4a-a3d4c4dd92b5\" (UID: \"82893f5c-2185-4a6a-8b4a-a3d4c4dd92b5\") " Dec 03 13:17:59 crc kubenswrapper[5052]: I1203 13:17:59.302757 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7kkkq\" (UniqueName: \"kubernetes.io/projected/82893f5c-2185-4a6a-8b4a-a3d4c4dd92b5-kube-api-access-7kkkq\") pod \"82893f5c-2185-4a6a-8b4a-a3d4c4dd92b5\" (UID: \"82893f5c-2185-4a6a-8b4a-a3d4c4dd92b5\") " Dec 03 13:17:59 crc kubenswrapper[5052]: I1203 13:17:59.304456 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/82893f5c-2185-4a6a-8b4a-a3d4c4dd92b5-logs" (OuterVolumeSpecName: "logs") pod "82893f5c-2185-4a6a-8b4a-a3d4c4dd92b5" (UID: "82893f5c-2185-4a6a-8b4a-a3d4c4dd92b5"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 13:17:59 crc kubenswrapper[5052]: I1203 13:17:59.313403 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/82893f5c-2185-4a6a-8b4a-a3d4c4dd92b5-kube-api-access-7kkkq" (OuterVolumeSpecName: "kube-api-access-7kkkq") pod "82893f5c-2185-4a6a-8b4a-a3d4c4dd92b5" (UID: "82893f5c-2185-4a6a-8b4a-a3d4c4dd92b5"). InnerVolumeSpecName "kube-api-access-7kkkq". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:17:59 crc kubenswrapper[5052]: I1203 13:17:59.322961 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/82893f5c-2185-4a6a-8b4a-a3d4c4dd92b5-config-data-custom" (OuterVolumeSpecName: "config-data-custom") pod "82893f5c-2185-4a6a-8b4a-a3d4c4dd92b5" (UID: "82893f5c-2185-4a6a-8b4a-a3d4c4dd92b5"). InnerVolumeSpecName "config-data-custom". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:17:59 crc kubenswrapper[5052]: I1203 13:17:59.354794 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/82893f5c-2185-4a6a-8b4a-a3d4c4dd92b5-config-data" (OuterVolumeSpecName: "config-data") pod "82893f5c-2185-4a6a-8b4a-a3d4c4dd92b5" (UID: "82893f5c-2185-4a6a-8b4a-a3d4c4dd92b5"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:17:59 crc kubenswrapper[5052]: I1203 13:17:59.379584 5052 patch_prober.go:28] interesting pod/machine-config-daemon-74bj6 container/machine-config-daemon namespace/openshift-machine-config-operator: Liveness probe status=failure output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" start-of-body= Dec 03 13:17:59 crc kubenswrapper[5052]: I1203 13:17:59.379635 5052 prober.go:107] "Probe failed" probeType="Liveness" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" containerName="machine-config-daemon" probeResult="failure" output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" Dec 03 13:17:59 crc kubenswrapper[5052]: I1203 13:17:59.404802 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-7kkkq\" (UniqueName: \"kubernetes.io/projected/82893f5c-2185-4a6a-8b4a-a3d4c4dd92b5-kube-api-access-7kkkq\") on node \"crc\" DevicePath \"\"" Dec 03 13:17:59 crc kubenswrapper[5052]: I1203 13:17:59.404847 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/82893f5c-2185-4a6a-8b4a-a3d4c4dd92b5-logs\") on node \"crc\" DevicePath \"\"" Dec 03 13:17:59 crc kubenswrapper[5052]: I1203 13:17:59.404861 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/82893f5c-2185-4a6a-8b4a-a3d4c4dd92b5-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 13:17:59 crc kubenswrapper[5052]: I1203 13:17:59.404877 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/82893f5c-2185-4a6a-8b4a-a3d4c4dd92b5-config-data-custom\") on node \"crc\" DevicePath \"\"" Dec 03 13:17:59 crc kubenswrapper[5052]: I1203 13:17:59.732059 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["barbican-kuttl-tests/barbican-api-75495b98c8-6rdjk"] Dec 03 13:17:59 crc kubenswrapper[5052]: I1203 13:17:59.754057 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["barbican-kuttl-tests/barbican-api-75495b98c8-6rdjk"] Dec 03 13:17:59 crc kubenswrapper[5052]: I1203 13:17:59.775715 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="barbican-kuttl-tests/barbican-api-75495b98c8-4vgmd" Dec 03 13:17:59 crc kubenswrapper[5052]: I1203 13:17:59.923143 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mnwwp\" (UniqueName: \"kubernetes.io/projected/47809cbf-0aa5-4d38-8b9f-95ce55763ed3-kube-api-access-mnwwp\") pod \"47809cbf-0aa5-4d38-8b9f-95ce55763ed3\" (UID: \"47809cbf-0aa5-4d38-8b9f-95ce55763ed3\") " Dec 03 13:17:59 crc kubenswrapper[5052]: I1203 13:17:59.923320 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/47809cbf-0aa5-4d38-8b9f-95ce55763ed3-logs\") pod \"47809cbf-0aa5-4d38-8b9f-95ce55763ed3\" (UID: \"47809cbf-0aa5-4d38-8b9f-95ce55763ed3\") " Dec 03 13:17:59 crc kubenswrapper[5052]: I1203 13:17:59.923352 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/47809cbf-0aa5-4d38-8b9f-95ce55763ed3-config-data\") pod \"47809cbf-0aa5-4d38-8b9f-95ce55763ed3\" (UID: \"47809cbf-0aa5-4d38-8b9f-95ce55763ed3\") " Dec 03 13:17:59 crc kubenswrapper[5052]: I1203 13:17:59.923374 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/47809cbf-0aa5-4d38-8b9f-95ce55763ed3-config-data-custom\") pod \"47809cbf-0aa5-4d38-8b9f-95ce55763ed3\" (UID: \"47809cbf-0aa5-4d38-8b9f-95ce55763ed3\") " Dec 03 13:17:59 crc kubenswrapper[5052]: I1203 13:17:59.924523 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/47809cbf-0aa5-4d38-8b9f-95ce55763ed3-logs" (OuterVolumeSpecName: "logs") pod "47809cbf-0aa5-4d38-8b9f-95ce55763ed3" (UID: "47809cbf-0aa5-4d38-8b9f-95ce55763ed3"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 13:17:59 crc kubenswrapper[5052]: I1203 13:17:59.928842 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/47809cbf-0aa5-4d38-8b9f-95ce55763ed3-config-data-custom" (OuterVolumeSpecName: "config-data-custom") pod "47809cbf-0aa5-4d38-8b9f-95ce55763ed3" (UID: "47809cbf-0aa5-4d38-8b9f-95ce55763ed3"). InnerVolumeSpecName "config-data-custom". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:17:59 crc kubenswrapper[5052]: I1203 13:17:59.928849 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/47809cbf-0aa5-4d38-8b9f-95ce55763ed3-kube-api-access-mnwwp" (OuterVolumeSpecName: "kube-api-access-mnwwp") pod "47809cbf-0aa5-4d38-8b9f-95ce55763ed3" (UID: "47809cbf-0aa5-4d38-8b9f-95ce55763ed3"). InnerVolumeSpecName "kube-api-access-mnwwp". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:17:59 crc kubenswrapper[5052]: I1203 13:17:59.947302 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="barbican-kuttl-tests/barbican-keystone-listener-66b47d8764-7bflh" Dec 03 13:17:59 crc kubenswrapper[5052]: I1203 13:17:59.971480 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="barbican-kuttl-tests/barbican-worker-74467844f5-llwv8" Dec 03 13:17:59 crc kubenswrapper[5052]: I1203 13:17:59.975772 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/47809cbf-0aa5-4d38-8b9f-95ce55763ed3-config-data" (OuterVolumeSpecName: "config-data") pod "47809cbf-0aa5-4d38-8b9f-95ce55763ed3" (UID: "47809cbf-0aa5-4d38-8b9f-95ce55763ed3"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:18:00 crc kubenswrapper[5052]: I1203 13:18:00.024762 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/3a71e182-1787-48f5-a4f8-60b9fb56f603-logs\") pod \"3a71e182-1787-48f5-a4f8-60b9fb56f603\" (UID: \"3a71e182-1787-48f5-a4f8-60b9fb56f603\") " Dec 03 13:18:00 crc kubenswrapper[5052]: I1203 13:18:00.024838 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/3a71e182-1787-48f5-a4f8-60b9fb56f603-config-data\") pod \"3a71e182-1787-48f5-a4f8-60b9fb56f603\" (UID: \"3a71e182-1787-48f5-a4f8-60b9fb56f603\") " Dec 03 13:18:00 crc kubenswrapper[5052]: I1203 13:18:00.024909 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cw9tq\" (UniqueName: \"kubernetes.io/projected/3a71e182-1787-48f5-a4f8-60b9fb56f603-kube-api-access-cw9tq\") pod \"3a71e182-1787-48f5-a4f8-60b9fb56f603\" (UID: \"3a71e182-1787-48f5-a4f8-60b9fb56f603\") " Dec 03 13:18:00 crc kubenswrapper[5052]: I1203 13:18:00.025059 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/3a71e182-1787-48f5-a4f8-60b9fb56f603-config-data-custom\") pod \"3a71e182-1787-48f5-a4f8-60b9fb56f603\" (UID: \"3a71e182-1787-48f5-a4f8-60b9fb56f603\") " Dec 03 13:18:00 crc kubenswrapper[5052]: I1203 13:18:00.025505 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-mnwwp\" (UniqueName: \"kubernetes.io/projected/47809cbf-0aa5-4d38-8b9f-95ce55763ed3-kube-api-access-mnwwp\") on node \"crc\" DevicePath \"\"" Dec 03 13:18:00 crc kubenswrapper[5052]: I1203 13:18:00.025539 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/47809cbf-0aa5-4d38-8b9f-95ce55763ed3-logs\") on node \"crc\" DevicePath \"\"" Dec 03 13:18:00 crc kubenswrapper[5052]: I1203 13:18:00.025553 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/47809cbf-0aa5-4d38-8b9f-95ce55763ed3-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 13:18:00 crc kubenswrapper[5052]: I1203 13:18:00.025562 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/47809cbf-0aa5-4d38-8b9f-95ce55763ed3-config-data-custom\") on node \"crc\" DevicePath \"\"" Dec 03 13:18:00 crc kubenswrapper[5052]: I1203 13:18:00.026122 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3a71e182-1787-48f5-a4f8-60b9fb56f603-logs" (OuterVolumeSpecName: "logs") pod "3a71e182-1787-48f5-a4f8-60b9fb56f603" (UID: "3a71e182-1787-48f5-a4f8-60b9fb56f603"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 13:18:00 crc kubenswrapper[5052]: I1203 13:18:00.031325 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3a71e182-1787-48f5-a4f8-60b9fb56f603-config-data-custom" (OuterVolumeSpecName: "config-data-custom") pod "3a71e182-1787-48f5-a4f8-60b9fb56f603" (UID: "3a71e182-1787-48f5-a4f8-60b9fb56f603"). InnerVolumeSpecName "config-data-custom". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:18:00 crc kubenswrapper[5052]: I1203 13:18:00.031427 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3a71e182-1787-48f5-a4f8-60b9fb56f603-kube-api-access-cw9tq" (OuterVolumeSpecName: "kube-api-access-cw9tq") pod "3a71e182-1787-48f5-a4f8-60b9fb56f603" (UID: "3a71e182-1787-48f5-a4f8-60b9fb56f603"). InnerVolumeSpecName "kube-api-access-cw9tq". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:18:00 crc kubenswrapper[5052]: I1203 13:18:00.072181 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openshift-marketplace/redhat-marketplace-w8xnr"] Dec 03 13:18:00 crc kubenswrapper[5052]: E1203 13:18:00.072594 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="47809cbf-0aa5-4d38-8b9f-95ce55763ed3" containerName="barbican-api-log" Dec 03 13:18:00 crc kubenswrapper[5052]: I1203 13:18:00.072612 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="47809cbf-0aa5-4d38-8b9f-95ce55763ed3" containerName="barbican-api-log" Dec 03 13:18:00 crc kubenswrapper[5052]: E1203 13:18:00.072629 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="47809cbf-0aa5-4d38-8b9f-95ce55763ed3" containerName="barbican-api" Dec 03 13:18:00 crc kubenswrapper[5052]: I1203 13:18:00.072637 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="47809cbf-0aa5-4d38-8b9f-95ce55763ed3" containerName="barbican-api" Dec 03 13:18:00 crc kubenswrapper[5052]: E1203 13:18:00.072656 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="3a71e182-1787-48f5-a4f8-60b9fb56f603" containerName="barbican-keystone-listener" Dec 03 13:18:00 crc kubenswrapper[5052]: I1203 13:18:00.072661 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="3a71e182-1787-48f5-a4f8-60b9fb56f603" containerName="barbican-keystone-listener" Dec 03 13:18:00 crc kubenswrapper[5052]: E1203 13:18:00.072684 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="82893f5c-2185-4a6a-8b4a-a3d4c4dd92b5" containerName="barbican-api-log" Dec 03 13:18:00 crc kubenswrapper[5052]: I1203 13:18:00.072691 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="82893f5c-2185-4a6a-8b4a-a3d4c4dd92b5" containerName="barbican-api-log" Dec 03 13:18:00 crc kubenswrapper[5052]: E1203 13:18:00.072704 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="e033a810-2874-4fb9-ab5b-654fb638cf18" containerName="barbican-worker" Dec 03 13:18:00 crc kubenswrapper[5052]: I1203 13:18:00.072711 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="e033a810-2874-4fb9-ab5b-654fb638cf18" containerName="barbican-worker" Dec 03 13:18:00 crc kubenswrapper[5052]: E1203 13:18:00.072729 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="3a71e182-1787-48f5-a4f8-60b9fb56f603" containerName="barbican-keystone-listener-log" Dec 03 13:18:00 crc kubenswrapper[5052]: I1203 13:18:00.072736 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="3a71e182-1787-48f5-a4f8-60b9fb56f603" containerName="barbican-keystone-listener-log" Dec 03 13:18:00 crc kubenswrapper[5052]: E1203 13:18:00.072745 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="82893f5c-2185-4a6a-8b4a-a3d4c4dd92b5" containerName="barbican-api" Dec 03 13:18:00 crc kubenswrapper[5052]: I1203 13:18:00.072752 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="82893f5c-2185-4a6a-8b4a-a3d4c4dd92b5" containerName="barbican-api" Dec 03 13:18:00 crc kubenswrapper[5052]: E1203 13:18:00.072762 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="e033a810-2874-4fb9-ab5b-654fb638cf18" containerName="barbican-worker-log" Dec 03 13:18:00 crc kubenswrapper[5052]: I1203 13:18:00.072770 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="e033a810-2874-4fb9-ab5b-654fb638cf18" containerName="barbican-worker-log" Dec 03 13:18:00 crc kubenswrapper[5052]: I1203 13:18:00.072903 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="e033a810-2874-4fb9-ab5b-654fb638cf18" containerName="barbican-worker" Dec 03 13:18:00 crc kubenswrapper[5052]: I1203 13:18:00.072916 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="82893f5c-2185-4a6a-8b4a-a3d4c4dd92b5" containerName="barbican-api" Dec 03 13:18:00 crc kubenswrapper[5052]: I1203 13:18:00.072928 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="47809cbf-0aa5-4d38-8b9f-95ce55763ed3" containerName="barbican-api" Dec 03 13:18:00 crc kubenswrapper[5052]: I1203 13:18:00.072944 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="e033a810-2874-4fb9-ab5b-654fb638cf18" containerName="barbican-worker-log" Dec 03 13:18:00 crc kubenswrapper[5052]: I1203 13:18:00.072950 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="3a71e182-1787-48f5-a4f8-60b9fb56f603" containerName="barbican-keystone-listener-log" Dec 03 13:18:00 crc kubenswrapper[5052]: I1203 13:18:00.072960 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="82893f5c-2185-4a6a-8b4a-a3d4c4dd92b5" containerName="barbican-api-log" Dec 03 13:18:00 crc kubenswrapper[5052]: I1203 13:18:00.072966 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="3a71e182-1787-48f5-a4f8-60b9fb56f603" containerName="barbican-keystone-listener" Dec 03 13:18:00 crc kubenswrapper[5052]: I1203 13:18:00.072978 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="47809cbf-0aa5-4d38-8b9f-95ce55763ed3" containerName="barbican-api-log" Dec 03 13:18:00 crc kubenswrapper[5052]: I1203 13:18:00.074235 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/redhat-marketplace-w8xnr" Dec 03 13:18:00 crc kubenswrapper[5052]: I1203 13:18:00.079624 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3a71e182-1787-48f5-a4f8-60b9fb56f603-config-data" (OuterVolumeSpecName: "config-data") pod "3a71e182-1787-48f5-a4f8-60b9fb56f603" (UID: "3a71e182-1787-48f5-a4f8-60b9fb56f603"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:18:00 crc kubenswrapper[5052]: I1203 13:18:00.090435 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/redhat-marketplace-w8xnr"] Dec 03 13:18:00 crc kubenswrapper[5052]: I1203 13:18:00.126391 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/e033a810-2874-4fb9-ab5b-654fb638cf18-config-data-custom\") pod \"e033a810-2874-4fb9-ab5b-654fb638cf18\" (UID: \"e033a810-2874-4fb9-ab5b-654fb638cf18\") " Dec 03 13:18:00 crc kubenswrapper[5052]: I1203 13:18:00.126485 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zshmv\" (UniqueName: \"kubernetes.io/projected/e033a810-2874-4fb9-ab5b-654fb638cf18-kube-api-access-zshmv\") pod \"e033a810-2874-4fb9-ab5b-654fb638cf18\" (UID: \"e033a810-2874-4fb9-ab5b-654fb638cf18\") " Dec 03 13:18:00 crc kubenswrapper[5052]: I1203 13:18:00.126567 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/e033a810-2874-4fb9-ab5b-654fb638cf18-logs\") pod \"e033a810-2874-4fb9-ab5b-654fb638cf18\" (UID: \"e033a810-2874-4fb9-ab5b-654fb638cf18\") " Dec 03 13:18:00 crc kubenswrapper[5052]: I1203 13:18:00.126617 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/e033a810-2874-4fb9-ab5b-654fb638cf18-config-data\") pod \"e033a810-2874-4fb9-ab5b-654fb638cf18\" (UID: \"e033a810-2874-4fb9-ab5b-654fb638cf18\") " Dec 03 13:18:00 crc kubenswrapper[5052]: I1203 13:18:00.126892 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/3a71e182-1787-48f5-a4f8-60b9fb56f603-config-data-custom\") on node \"crc\" DevicePath \"\"" Dec 03 13:18:00 crc kubenswrapper[5052]: I1203 13:18:00.126906 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/3a71e182-1787-48f5-a4f8-60b9fb56f603-logs\") on node \"crc\" DevicePath \"\"" Dec 03 13:18:00 crc kubenswrapper[5052]: I1203 13:18:00.126914 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/3a71e182-1787-48f5-a4f8-60b9fb56f603-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 13:18:00 crc kubenswrapper[5052]: I1203 13:18:00.126924 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-cw9tq\" (UniqueName: \"kubernetes.io/projected/3a71e182-1787-48f5-a4f8-60b9fb56f603-kube-api-access-cw9tq\") on node \"crc\" DevicePath \"\"" Dec 03 13:18:00 crc kubenswrapper[5052]: I1203 13:18:00.128806 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e033a810-2874-4fb9-ab5b-654fb638cf18-logs" (OuterVolumeSpecName: "logs") pod "e033a810-2874-4fb9-ab5b-654fb638cf18" (UID: "e033a810-2874-4fb9-ab5b-654fb638cf18"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 13:18:00 crc kubenswrapper[5052]: I1203 13:18:00.137910 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e033a810-2874-4fb9-ab5b-654fb638cf18-config-data-custom" (OuterVolumeSpecName: "config-data-custom") pod "e033a810-2874-4fb9-ab5b-654fb638cf18" (UID: "e033a810-2874-4fb9-ab5b-654fb638cf18"). InnerVolumeSpecName "config-data-custom". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:18:00 crc kubenswrapper[5052]: I1203 13:18:00.138198 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e033a810-2874-4fb9-ab5b-654fb638cf18-kube-api-access-zshmv" (OuterVolumeSpecName: "kube-api-access-zshmv") pod "e033a810-2874-4fb9-ab5b-654fb638cf18" (UID: "e033a810-2874-4fb9-ab5b-654fb638cf18"). InnerVolumeSpecName "kube-api-access-zshmv". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:18:00 crc kubenswrapper[5052]: I1203 13:18:00.194290 5052 generic.go:334] "Generic (PLEG): container finished" podID="e033a810-2874-4fb9-ab5b-654fb638cf18" containerID="5572642d7c888f6ff77ddabfc745ab3a1eafa82c40c49357c61e77d9a54c3aef" exitCode=0 Dec 03 13:18:00 crc kubenswrapper[5052]: I1203 13:18:00.194323 5052 generic.go:334] "Generic (PLEG): container finished" podID="e033a810-2874-4fb9-ab5b-654fb638cf18" containerID="b5bb49d9158f4c1812ee278785430bfc1cb627406b76f99d62ed896e546bc0b9" exitCode=143 Dec 03 13:18:00 crc kubenswrapper[5052]: I1203 13:18:00.194367 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="barbican-kuttl-tests/barbican-worker-74467844f5-llwv8" event={"ID":"e033a810-2874-4fb9-ab5b-654fb638cf18","Type":"ContainerDied","Data":"5572642d7c888f6ff77ddabfc745ab3a1eafa82c40c49357c61e77d9a54c3aef"} Dec 03 13:18:00 crc kubenswrapper[5052]: I1203 13:18:00.194395 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="barbican-kuttl-tests/barbican-worker-74467844f5-llwv8" event={"ID":"e033a810-2874-4fb9-ab5b-654fb638cf18","Type":"ContainerDied","Data":"b5bb49d9158f4c1812ee278785430bfc1cb627406b76f99d62ed896e546bc0b9"} Dec 03 13:18:00 crc kubenswrapper[5052]: I1203 13:18:00.194408 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="barbican-kuttl-tests/barbican-worker-74467844f5-llwv8" event={"ID":"e033a810-2874-4fb9-ab5b-654fb638cf18","Type":"ContainerDied","Data":"b7afb54f639942d174ed5592fdc147498de1e6077ea9f4cbbedffcabf9a3260e"} Dec 03 13:18:00 crc kubenswrapper[5052]: I1203 13:18:00.194426 5052 scope.go:117] "RemoveContainer" containerID="5572642d7c888f6ff77ddabfc745ab3a1eafa82c40c49357c61e77d9a54c3aef" Dec 03 13:18:00 crc kubenswrapper[5052]: I1203 13:18:00.194606 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="barbican-kuttl-tests/barbican-worker-74467844f5-llwv8" Dec 03 13:18:00 crc kubenswrapper[5052]: I1203 13:18:00.200640 5052 generic.go:334] "Generic (PLEG): container finished" podID="3a71e182-1787-48f5-a4f8-60b9fb56f603" containerID="216a92996bad8313803338c34e597edfe8e9388bfe6f7735837c64532f3c7f37" exitCode=0 Dec 03 13:18:00 crc kubenswrapper[5052]: I1203 13:18:00.200670 5052 generic.go:334] "Generic (PLEG): container finished" podID="3a71e182-1787-48f5-a4f8-60b9fb56f603" containerID="77fdae1edc081d92e84e48b069e63e28b031ef998432a493107b3427449baa2d" exitCode=143 Dec 03 13:18:00 crc kubenswrapper[5052]: I1203 13:18:00.200733 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="barbican-kuttl-tests/barbican-keystone-listener-66b47d8764-7bflh" event={"ID":"3a71e182-1787-48f5-a4f8-60b9fb56f603","Type":"ContainerDied","Data":"216a92996bad8313803338c34e597edfe8e9388bfe6f7735837c64532f3c7f37"} Dec 03 13:18:00 crc kubenswrapper[5052]: I1203 13:18:00.200761 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="barbican-kuttl-tests/barbican-keystone-listener-66b47d8764-7bflh" event={"ID":"3a71e182-1787-48f5-a4f8-60b9fb56f603","Type":"ContainerDied","Data":"77fdae1edc081d92e84e48b069e63e28b031ef998432a493107b3427449baa2d"} Dec 03 13:18:00 crc kubenswrapper[5052]: I1203 13:18:00.200773 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="barbican-kuttl-tests/barbican-keystone-listener-66b47d8764-7bflh" event={"ID":"3a71e182-1787-48f5-a4f8-60b9fb56f603","Type":"ContainerDied","Data":"e6af7006ba7fafb6119a764132ef9d62ff06ccbc771e754f553f0acd3a7ab348"} Dec 03 13:18:00 crc kubenswrapper[5052]: I1203 13:18:00.200836 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="barbican-kuttl-tests/barbican-keystone-listener-66b47d8764-7bflh" Dec 03 13:18:00 crc kubenswrapper[5052]: I1203 13:18:00.210478 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["barbican-kuttl-tests/barbican-db-sync-tnw45"] Dec 03 13:18:00 crc kubenswrapper[5052]: I1203 13:18:00.216236 5052 generic.go:334] "Generic (PLEG): container finished" podID="47809cbf-0aa5-4d38-8b9f-95ce55763ed3" containerID="c741ab94362e9e80b0eebbdccb461275bde7eb198357602ecf738959164f75cd" exitCode=0 Dec 03 13:18:00 crc kubenswrapper[5052]: I1203 13:18:00.216275 5052 generic.go:334] "Generic (PLEG): container finished" podID="47809cbf-0aa5-4d38-8b9f-95ce55763ed3" containerID="ea7dfeda9cfa835bba5e01e59ae1970fffc8bed2ac1b9377849a24e4e02b17ec" exitCode=143 Dec 03 13:18:00 crc kubenswrapper[5052]: I1203 13:18:00.216323 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="barbican-kuttl-tests/barbican-api-75495b98c8-4vgmd" event={"ID":"47809cbf-0aa5-4d38-8b9f-95ce55763ed3","Type":"ContainerDied","Data":"c741ab94362e9e80b0eebbdccb461275bde7eb198357602ecf738959164f75cd"} Dec 03 13:18:00 crc kubenswrapper[5052]: I1203 13:18:00.216353 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="barbican-kuttl-tests/barbican-api-75495b98c8-4vgmd" event={"ID":"47809cbf-0aa5-4d38-8b9f-95ce55763ed3","Type":"ContainerDied","Data":"ea7dfeda9cfa835bba5e01e59ae1970fffc8bed2ac1b9377849a24e4e02b17ec"} Dec 03 13:18:00 crc kubenswrapper[5052]: I1203 13:18:00.216367 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="barbican-kuttl-tests/barbican-api-75495b98c8-4vgmd" event={"ID":"47809cbf-0aa5-4d38-8b9f-95ce55763ed3","Type":"ContainerDied","Data":"264156498b352d98a23b875a3e46d95ac0b1a0134bf5ae9579feba3d3ffbf1ab"} Dec 03 13:18:00 crc kubenswrapper[5052]: I1203 13:18:00.216449 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="barbican-kuttl-tests/barbican-api-75495b98c8-4vgmd" Dec 03 13:18:00 crc kubenswrapper[5052]: I1203 13:18:00.218265 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e033a810-2874-4fb9-ab5b-654fb638cf18-config-data" (OuterVolumeSpecName: "config-data") pod "e033a810-2874-4fb9-ab5b-654fb638cf18" (UID: "e033a810-2874-4fb9-ab5b-654fb638cf18"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:18:00 crc kubenswrapper[5052]: I1203 13:18:00.218457 5052 scope.go:117] "RemoveContainer" containerID="b5bb49d9158f4c1812ee278785430bfc1cb627406b76f99d62ed896e546bc0b9" Dec 03 13:18:00 crc kubenswrapper[5052]: I1203 13:18:00.219952 5052 generic.go:334] "Generic (PLEG): container finished" podID="72c65803-7d22-4b27-9cfc-b8a246a0ba7a" containerID="1a6e83c31c16b733e0a9dddbbfeffb1a8033e40fe0d2444509c93a09efb4dcf5" exitCode=143 Dec 03 13:18:00 crc kubenswrapper[5052]: I1203 13:18:00.220070 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="barbican-kuttl-tests/barbican-worker-74467844f5-w7kwd" event={"ID":"72c65803-7d22-4b27-9cfc-b8a246a0ba7a","Type":"ContainerDied","Data":"1a6e83c31c16b733e0a9dddbbfeffb1a8033e40fe0d2444509c93a09efb4dcf5"} Dec 03 13:18:00 crc kubenswrapper[5052]: I1203 13:18:00.227754 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/411fcfda-357c-4967-b59f-54cb173c3ce5-catalog-content\") pod \"redhat-marketplace-w8xnr\" (UID: \"411fcfda-357c-4967-b59f-54cb173c3ce5\") " pod="openshift-marketplace/redhat-marketplace-w8xnr" Dec 03 13:18:00 crc kubenswrapper[5052]: I1203 13:18:00.227810 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4hhvn\" (UniqueName: \"kubernetes.io/projected/411fcfda-357c-4967-b59f-54cb173c3ce5-kube-api-access-4hhvn\") pod \"redhat-marketplace-w8xnr\" (UID: \"411fcfda-357c-4967-b59f-54cb173c3ce5\") " pod="openshift-marketplace/redhat-marketplace-w8xnr" Dec 03 13:18:00 crc kubenswrapper[5052]: I1203 13:18:00.227951 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/411fcfda-357c-4967-b59f-54cb173c3ce5-utilities\") pod \"redhat-marketplace-w8xnr\" (UID: \"411fcfda-357c-4967-b59f-54cb173c3ce5\") " pod="openshift-marketplace/redhat-marketplace-w8xnr" Dec 03 13:18:00 crc kubenswrapper[5052]: I1203 13:18:00.228019 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/e033a810-2874-4fb9-ab5b-654fb638cf18-config-data-custom\") on node \"crc\" DevicePath \"\"" Dec 03 13:18:00 crc kubenswrapper[5052]: I1203 13:18:00.228042 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-zshmv\" (UniqueName: \"kubernetes.io/projected/e033a810-2874-4fb9-ab5b-654fb638cf18-kube-api-access-zshmv\") on node \"crc\" DevicePath \"\"" Dec 03 13:18:00 crc kubenswrapper[5052]: I1203 13:18:00.228057 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/e033a810-2874-4fb9-ab5b-654fb638cf18-logs\") on node \"crc\" DevicePath \"\"" Dec 03 13:18:00 crc kubenswrapper[5052]: I1203 13:18:00.228071 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/e033a810-2874-4fb9-ab5b-654fb638cf18-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 13:18:00 crc kubenswrapper[5052]: I1203 13:18:00.228235 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["barbican-kuttl-tests/barbican-db-sync-tnw45"] Dec 03 13:18:00 crc kubenswrapper[5052]: I1203 13:18:00.246575 5052 scope.go:117] "RemoveContainer" containerID="5572642d7c888f6ff77ddabfc745ab3a1eafa82c40c49357c61e77d9a54c3aef" Dec 03 13:18:00 crc kubenswrapper[5052]: E1203 13:18:00.248117 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5572642d7c888f6ff77ddabfc745ab3a1eafa82c40c49357c61e77d9a54c3aef\": container with ID starting with 5572642d7c888f6ff77ddabfc745ab3a1eafa82c40c49357c61e77d9a54c3aef not found: ID does not exist" containerID="5572642d7c888f6ff77ddabfc745ab3a1eafa82c40c49357c61e77d9a54c3aef" Dec 03 13:18:00 crc kubenswrapper[5052]: I1203 13:18:00.248176 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5572642d7c888f6ff77ddabfc745ab3a1eafa82c40c49357c61e77d9a54c3aef"} err="failed to get container status \"5572642d7c888f6ff77ddabfc745ab3a1eafa82c40c49357c61e77d9a54c3aef\": rpc error: code = NotFound desc = could not find container \"5572642d7c888f6ff77ddabfc745ab3a1eafa82c40c49357c61e77d9a54c3aef\": container with ID starting with 5572642d7c888f6ff77ddabfc745ab3a1eafa82c40c49357c61e77d9a54c3aef not found: ID does not exist" Dec 03 13:18:00 crc kubenswrapper[5052]: I1203 13:18:00.248207 5052 scope.go:117] "RemoveContainer" containerID="b5bb49d9158f4c1812ee278785430bfc1cb627406b76f99d62ed896e546bc0b9" Dec 03 13:18:00 crc kubenswrapper[5052]: E1203 13:18:00.250012 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b5bb49d9158f4c1812ee278785430bfc1cb627406b76f99d62ed896e546bc0b9\": container with ID starting with b5bb49d9158f4c1812ee278785430bfc1cb627406b76f99d62ed896e546bc0b9 not found: ID does not exist" containerID="b5bb49d9158f4c1812ee278785430bfc1cb627406b76f99d62ed896e546bc0b9" Dec 03 13:18:00 crc kubenswrapper[5052]: I1203 13:18:00.250130 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b5bb49d9158f4c1812ee278785430bfc1cb627406b76f99d62ed896e546bc0b9"} err="failed to get container status \"b5bb49d9158f4c1812ee278785430bfc1cb627406b76f99d62ed896e546bc0b9\": rpc error: code = NotFound desc = could not find container \"b5bb49d9158f4c1812ee278785430bfc1cb627406b76f99d62ed896e546bc0b9\": container with ID starting with b5bb49d9158f4c1812ee278785430bfc1cb627406b76f99d62ed896e546bc0b9 not found: ID does not exist" Dec 03 13:18:00 crc kubenswrapper[5052]: I1203 13:18:00.250224 5052 scope.go:117] "RemoveContainer" containerID="5572642d7c888f6ff77ddabfc745ab3a1eafa82c40c49357c61e77d9a54c3aef" Dec 03 13:18:00 crc kubenswrapper[5052]: I1203 13:18:00.250703 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5572642d7c888f6ff77ddabfc745ab3a1eafa82c40c49357c61e77d9a54c3aef"} err="failed to get container status \"5572642d7c888f6ff77ddabfc745ab3a1eafa82c40c49357c61e77d9a54c3aef\": rpc error: code = NotFound desc = could not find container \"5572642d7c888f6ff77ddabfc745ab3a1eafa82c40c49357c61e77d9a54c3aef\": container with ID starting with 5572642d7c888f6ff77ddabfc745ab3a1eafa82c40c49357c61e77d9a54c3aef not found: ID does not exist" Dec 03 13:18:00 crc kubenswrapper[5052]: I1203 13:18:00.250780 5052 scope.go:117] "RemoveContainer" containerID="b5bb49d9158f4c1812ee278785430bfc1cb627406b76f99d62ed896e546bc0b9" Dec 03 13:18:00 crc kubenswrapper[5052]: I1203 13:18:00.251919 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b5bb49d9158f4c1812ee278785430bfc1cb627406b76f99d62ed896e546bc0b9"} err="failed to get container status \"b5bb49d9158f4c1812ee278785430bfc1cb627406b76f99d62ed896e546bc0b9\": rpc error: code = NotFound desc = could not find container \"b5bb49d9158f4c1812ee278785430bfc1cb627406b76f99d62ed896e546bc0b9\": container with ID starting with b5bb49d9158f4c1812ee278785430bfc1cb627406b76f99d62ed896e546bc0b9 not found: ID does not exist" Dec 03 13:18:00 crc kubenswrapper[5052]: I1203 13:18:00.251948 5052 scope.go:117] "RemoveContainer" containerID="216a92996bad8313803338c34e597edfe8e9388bfe6f7735837c64532f3c7f37" Dec 03 13:18:00 crc kubenswrapper[5052]: I1203 13:18:00.274719 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["barbican-kuttl-tests/barbican-keystone-listener-66b47d8764-7bflh"] Dec 03 13:18:00 crc kubenswrapper[5052]: I1203 13:18:00.296361 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["barbican-kuttl-tests/barbican-keystone-listener-66b47d8764-7bflh"] Dec 03 13:18:00 crc kubenswrapper[5052]: I1203 13:18:00.296709 5052 scope.go:117] "RemoveContainer" containerID="77fdae1edc081d92e84e48b069e63e28b031ef998432a493107b3427449baa2d" Dec 03 13:18:00 crc kubenswrapper[5052]: I1203 13:18:00.314565 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["barbican-kuttl-tests/barbican-api-75495b98c8-4vgmd"] Dec 03 13:18:00 crc kubenswrapper[5052]: I1203 13:18:00.324631 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["barbican-kuttl-tests/barbican-api-75495b98c8-4vgmd"] Dec 03 13:18:00 crc kubenswrapper[5052]: I1203 13:18:00.328937 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/411fcfda-357c-4967-b59f-54cb173c3ce5-utilities\") pod \"redhat-marketplace-w8xnr\" (UID: \"411fcfda-357c-4967-b59f-54cb173c3ce5\") " pod="openshift-marketplace/redhat-marketplace-w8xnr" Dec 03 13:18:00 crc kubenswrapper[5052]: I1203 13:18:00.328991 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/411fcfda-357c-4967-b59f-54cb173c3ce5-catalog-content\") pod \"redhat-marketplace-w8xnr\" (UID: \"411fcfda-357c-4967-b59f-54cb173c3ce5\") " pod="openshift-marketplace/redhat-marketplace-w8xnr" Dec 03 13:18:00 crc kubenswrapper[5052]: I1203 13:18:00.329012 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-4hhvn\" (UniqueName: \"kubernetes.io/projected/411fcfda-357c-4967-b59f-54cb173c3ce5-kube-api-access-4hhvn\") pod \"redhat-marketplace-w8xnr\" (UID: \"411fcfda-357c-4967-b59f-54cb173c3ce5\") " pod="openshift-marketplace/redhat-marketplace-w8xnr" Dec 03 13:18:00 crc kubenswrapper[5052]: I1203 13:18:00.329859 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/411fcfda-357c-4967-b59f-54cb173c3ce5-utilities\") pod \"redhat-marketplace-w8xnr\" (UID: \"411fcfda-357c-4967-b59f-54cb173c3ce5\") " pod="openshift-marketplace/redhat-marketplace-w8xnr" Dec 03 13:18:00 crc kubenswrapper[5052]: I1203 13:18:00.330115 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/411fcfda-357c-4967-b59f-54cb173c3ce5-catalog-content\") pod \"redhat-marketplace-w8xnr\" (UID: \"411fcfda-357c-4967-b59f-54cb173c3ce5\") " pod="openshift-marketplace/redhat-marketplace-w8xnr" Dec 03 13:18:00 crc kubenswrapper[5052]: I1203 13:18:00.332954 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["barbican-kuttl-tests/barbicanc550-account-delete-lxl47"] Dec 03 13:18:00 crc kubenswrapper[5052]: I1203 13:18:00.334630 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="barbican-kuttl-tests/barbicanc550-account-delete-lxl47" Dec 03 13:18:00 crc kubenswrapper[5052]: I1203 13:18:00.340967 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["barbican-kuttl-tests/barbicanc550-account-delete-lxl47"] Dec 03 13:18:00 crc kubenswrapper[5052]: I1203 13:18:00.357781 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-4hhvn\" (UniqueName: \"kubernetes.io/projected/411fcfda-357c-4967-b59f-54cb173c3ce5-kube-api-access-4hhvn\") pod \"redhat-marketplace-w8xnr\" (UID: \"411fcfda-357c-4967-b59f-54cb173c3ce5\") " pod="openshift-marketplace/redhat-marketplace-w8xnr" Dec 03 13:18:00 crc kubenswrapper[5052]: I1203 13:18:00.399900 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/redhat-marketplace-w8xnr" Dec 03 13:18:00 crc kubenswrapper[5052]: I1203 13:18:00.430627 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fdb2f\" (UniqueName: \"kubernetes.io/projected/53dce915-20c6-40ce-91ca-6d1499d9165f-kube-api-access-fdb2f\") pod \"barbicanc550-account-delete-lxl47\" (UID: \"53dce915-20c6-40ce-91ca-6d1499d9165f\") " pod="barbican-kuttl-tests/barbicanc550-account-delete-lxl47" Dec 03 13:18:00 crc kubenswrapper[5052]: I1203 13:18:00.430721 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/53dce915-20c6-40ce-91ca-6d1499d9165f-operator-scripts\") pod \"barbicanc550-account-delete-lxl47\" (UID: \"53dce915-20c6-40ce-91ca-6d1499d9165f\") " pod="barbican-kuttl-tests/barbicanc550-account-delete-lxl47" Dec 03 13:18:00 crc kubenswrapper[5052]: I1203 13:18:00.512931 5052 scope.go:117] "RemoveContainer" containerID="216a92996bad8313803338c34e597edfe8e9388bfe6f7735837c64532f3c7f37" Dec 03 13:18:00 crc kubenswrapper[5052]: E1203 13:18:00.516651 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"216a92996bad8313803338c34e597edfe8e9388bfe6f7735837c64532f3c7f37\": container with ID starting with 216a92996bad8313803338c34e597edfe8e9388bfe6f7735837c64532f3c7f37 not found: ID does not exist" containerID="216a92996bad8313803338c34e597edfe8e9388bfe6f7735837c64532f3c7f37" Dec 03 13:18:00 crc kubenswrapper[5052]: I1203 13:18:00.516693 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"216a92996bad8313803338c34e597edfe8e9388bfe6f7735837c64532f3c7f37"} err="failed to get container status \"216a92996bad8313803338c34e597edfe8e9388bfe6f7735837c64532f3c7f37\": rpc error: code = NotFound desc = could not find container \"216a92996bad8313803338c34e597edfe8e9388bfe6f7735837c64532f3c7f37\": container with ID starting with 216a92996bad8313803338c34e597edfe8e9388bfe6f7735837c64532f3c7f37 not found: ID does not exist" Dec 03 13:18:00 crc kubenswrapper[5052]: I1203 13:18:00.516720 5052 scope.go:117] "RemoveContainer" containerID="77fdae1edc081d92e84e48b069e63e28b031ef998432a493107b3427449baa2d" Dec 03 13:18:00 crc kubenswrapper[5052]: E1203 13:18:00.531032 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"77fdae1edc081d92e84e48b069e63e28b031ef998432a493107b3427449baa2d\": container with ID starting with 77fdae1edc081d92e84e48b069e63e28b031ef998432a493107b3427449baa2d not found: ID does not exist" containerID="77fdae1edc081d92e84e48b069e63e28b031ef998432a493107b3427449baa2d" Dec 03 13:18:00 crc kubenswrapper[5052]: I1203 13:18:00.531078 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"77fdae1edc081d92e84e48b069e63e28b031ef998432a493107b3427449baa2d"} err="failed to get container status \"77fdae1edc081d92e84e48b069e63e28b031ef998432a493107b3427449baa2d\": rpc error: code = NotFound desc = could not find container \"77fdae1edc081d92e84e48b069e63e28b031ef998432a493107b3427449baa2d\": container with ID starting with 77fdae1edc081d92e84e48b069e63e28b031ef998432a493107b3427449baa2d not found: ID does not exist" Dec 03 13:18:00 crc kubenswrapper[5052]: I1203 13:18:00.531103 5052 scope.go:117] "RemoveContainer" containerID="216a92996bad8313803338c34e597edfe8e9388bfe6f7735837c64532f3c7f37" Dec 03 13:18:00 crc kubenswrapper[5052]: I1203 13:18:00.532121 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/53dce915-20c6-40ce-91ca-6d1499d9165f-operator-scripts\") pod \"barbicanc550-account-delete-lxl47\" (UID: \"53dce915-20c6-40ce-91ca-6d1499d9165f\") " pod="barbican-kuttl-tests/barbicanc550-account-delete-lxl47" Dec 03 13:18:00 crc kubenswrapper[5052]: I1203 13:18:00.532267 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-fdb2f\" (UniqueName: \"kubernetes.io/projected/53dce915-20c6-40ce-91ca-6d1499d9165f-kube-api-access-fdb2f\") pod \"barbicanc550-account-delete-lxl47\" (UID: \"53dce915-20c6-40ce-91ca-6d1499d9165f\") " pod="barbican-kuttl-tests/barbicanc550-account-delete-lxl47" Dec 03 13:18:00 crc kubenswrapper[5052]: I1203 13:18:00.536899 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/53dce915-20c6-40ce-91ca-6d1499d9165f-operator-scripts\") pod \"barbicanc550-account-delete-lxl47\" (UID: \"53dce915-20c6-40ce-91ca-6d1499d9165f\") " pod="barbican-kuttl-tests/barbicanc550-account-delete-lxl47" Dec 03 13:18:00 crc kubenswrapper[5052]: I1203 13:18:00.539780 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"216a92996bad8313803338c34e597edfe8e9388bfe6f7735837c64532f3c7f37"} err="failed to get container status \"216a92996bad8313803338c34e597edfe8e9388bfe6f7735837c64532f3c7f37\": rpc error: code = NotFound desc = could not find container \"216a92996bad8313803338c34e597edfe8e9388bfe6f7735837c64532f3c7f37\": container with ID starting with 216a92996bad8313803338c34e597edfe8e9388bfe6f7735837c64532f3c7f37 not found: ID does not exist" Dec 03 13:18:00 crc kubenswrapper[5052]: I1203 13:18:00.539828 5052 scope.go:117] "RemoveContainer" containerID="77fdae1edc081d92e84e48b069e63e28b031ef998432a493107b3427449baa2d" Dec 03 13:18:00 crc kubenswrapper[5052]: I1203 13:18:00.547438 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["barbican-kuttl-tests/barbican-worker-74467844f5-llwv8"] Dec 03 13:18:00 crc kubenswrapper[5052]: I1203 13:18:00.548085 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"77fdae1edc081d92e84e48b069e63e28b031ef998432a493107b3427449baa2d"} err="failed to get container status \"77fdae1edc081d92e84e48b069e63e28b031ef998432a493107b3427449baa2d\": rpc error: code = NotFound desc = could not find container \"77fdae1edc081d92e84e48b069e63e28b031ef998432a493107b3427449baa2d\": container with ID starting with 77fdae1edc081d92e84e48b069e63e28b031ef998432a493107b3427449baa2d not found: ID does not exist" Dec 03 13:18:00 crc kubenswrapper[5052]: I1203 13:18:00.548114 5052 scope.go:117] "RemoveContainer" containerID="c741ab94362e9e80b0eebbdccb461275bde7eb198357602ecf738959164f75cd" Dec 03 13:18:00 crc kubenswrapper[5052]: I1203 13:18:00.555725 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["barbican-kuttl-tests/barbican-worker-74467844f5-llwv8"] Dec 03 13:18:00 crc kubenswrapper[5052]: I1203 13:18:00.568461 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-fdb2f\" (UniqueName: \"kubernetes.io/projected/53dce915-20c6-40ce-91ca-6d1499d9165f-kube-api-access-fdb2f\") pod \"barbicanc550-account-delete-lxl47\" (UID: \"53dce915-20c6-40ce-91ca-6d1499d9165f\") " pod="barbican-kuttl-tests/barbicanc550-account-delete-lxl47" Dec 03 13:18:00 crc kubenswrapper[5052]: I1203 13:18:00.588378 5052 scope.go:117] "RemoveContainer" containerID="ea7dfeda9cfa835bba5e01e59ae1970fffc8bed2ac1b9377849a24e4e02b17ec" Dec 03 13:18:00 crc kubenswrapper[5052]: I1203 13:18:00.660500 5052 scope.go:117] "RemoveContainer" containerID="c741ab94362e9e80b0eebbdccb461275bde7eb198357602ecf738959164f75cd" Dec 03 13:18:00 crc kubenswrapper[5052]: E1203 13:18:00.662736 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c741ab94362e9e80b0eebbdccb461275bde7eb198357602ecf738959164f75cd\": container with ID starting with c741ab94362e9e80b0eebbdccb461275bde7eb198357602ecf738959164f75cd not found: ID does not exist" containerID="c741ab94362e9e80b0eebbdccb461275bde7eb198357602ecf738959164f75cd" Dec 03 13:18:00 crc kubenswrapper[5052]: I1203 13:18:00.662774 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c741ab94362e9e80b0eebbdccb461275bde7eb198357602ecf738959164f75cd"} err="failed to get container status \"c741ab94362e9e80b0eebbdccb461275bde7eb198357602ecf738959164f75cd\": rpc error: code = NotFound desc = could not find container \"c741ab94362e9e80b0eebbdccb461275bde7eb198357602ecf738959164f75cd\": container with ID starting with c741ab94362e9e80b0eebbdccb461275bde7eb198357602ecf738959164f75cd not found: ID does not exist" Dec 03 13:18:00 crc kubenswrapper[5052]: I1203 13:18:00.662802 5052 scope.go:117] "RemoveContainer" containerID="ea7dfeda9cfa835bba5e01e59ae1970fffc8bed2ac1b9377849a24e4e02b17ec" Dec 03 13:18:00 crc kubenswrapper[5052]: E1203 13:18:00.663206 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ea7dfeda9cfa835bba5e01e59ae1970fffc8bed2ac1b9377849a24e4e02b17ec\": container with ID starting with ea7dfeda9cfa835bba5e01e59ae1970fffc8bed2ac1b9377849a24e4e02b17ec not found: ID does not exist" containerID="ea7dfeda9cfa835bba5e01e59ae1970fffc8bed2ac1b9377849a24e4e02b17ec" Dec 03 13:18:00 crc kubenswrapper[5052]: I1203 13:18:00.663253 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ea7dfeda9cfa835bba5e01e59ae1970fffc8bed2ac1b9377849a24e4e02b17ec"} err="failed to get container status \"ea7dfeda9cfa835bba5e01e59ae1970fffc8bed2ac1b9377849a24e4e02b17ec\": rpc error: code = NotFound desc = could not find container \"ea7dfeda9cfa835bba5e01e59ae1970fffc8bed2ac1b9377849a24e4e02b17ec\": container with ID starting with ea7dfeda9cfa835bba5e01e59ae1970fffc8bed2ac1b9377849a24e4e02b17ec not found: ID does not exist" Dec 03 13:18:00 crc kubenswrapper[5052]: I1203 13:18:00.663289 5052 scope.go:117] "RemoveContainer" containerID="c741ab94362e9e80b0eebbdccb461275bde7eb198357602ecf738959164f75cd" Dec 03 13:18:00 crc kubenswrapper[5052]: I1203 13:18:00.665899 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c741ab94362e9e80b0eebbdccb461275bde7eb198357602ecf738959164f75cd"} err="failed to get container status \"c741ab94362e9e80b0eebbdccb461275bde7eb198357602ecf738959164f75cd\": rpc error: code = NotFound desc = could not find container \"c741ab94362e9e80b0eebbdccb461275bde7eb198357602ecf738959164f75cd\": container with ID starting with c741ab94362e9e80b0eebbdccb461275bde7eb198357602ecf738959164f75cd not found: ID does not exist" Dec 03 13:18:00 crc kubenswrapper[5052]: I1203 13:18:00.665940 5052 scope.go:117] "RemoveContainer" containerID="ea7dfeda9cfa835bba5e01e59ae1970fffc8bed2ac1b9377849a24e4e02b17ec" Dec 03 13:18:00 crc kubenswrapper[5052]: I1203 13:18:00.666821 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ea7dfeda9cfa835bba5e01e59ae1970fffc8bed2ac1b9377849a24e4e02b17ec"} err="failed to get container status \"ea7dfeda9cfa835bba5e01e59ae1970fffc8bed2ac1b9377849a24e4e02b17ec\": rpc error: code = NotFound desc = could not find container \"ea7dfeda9cfa835bba5e01e59ae1970fffc8bed2ac1b9377849a24e4e02b17ec\": container with ID starting with ea7dfeda9cfa835bba5e01e59ae1970fffc8bed2ac1b9377849a24e4e02b17ec not found: ID does not exist" Dec 03 13:18:00 crc kubenswrapper[5052]: I1203 13:18:00.797662 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="barbican-kuttl-tests/barbican-worker-74467844f5-w7kwd" Dec 03 13:18:00 crc kubenswrapper[5052]: I1203 13:18:00.812295 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="barbican-kuttl-tests/barbicanc550-account-delete-lxl47" Dec 03 13:18:00 crc kubenswrapper[5052]: I1203 13:18:00.937489 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/72c65803-7d22-4b27-9cfc-b8a246a0ba7a-logs\") pod \"72c65803-7d22-4b27-9cfc-b8a246a0ba7a\" (UID: \"72c65803-7d22-4b27-9cfc-b8a246a0ba7a\") " Dec 03 13:18:00 crc kubenswrapper[5052]: I1203 13:18:00.937598 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/72c65803-7d22-4b27-9cfc-b8a246a0ba7a-config-data\") pod \"72c65803-7d22-4b27-9cfc-b8a246a0ba7a\" (UID: \"72c65803-7d22-4b27-9cfc-b8a246a0ba7a\") " Dec 03 13:18:00 crc kubenswrapper[5052]: I1203 13:18:00.937652 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/72c65803-7d22-4b27-9cfc-b8a246a0ba7a-config-data-custom\") pod \"72c65803-7d22-4b27-9cfc-b8a246a0ba7a\" (UID: \"72c65803-7d22-4b27-9cfc-b8a246a0ba7a\") " Dec 03 13:18:00 crc kubenswrapper[5052]: I1203 13:18:00.937691 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4glrf\" (UniqueName: \"kubernetes.io/projected/72c65803-7d22-4b27-9cfc-b8a246a0ba7a-kube-api-access-4glrf\") pod \"72c65803-7d22-4b27-9cfc-b8a246a0ba7a\" (UID: \"72c65803-7d22-4b27-9cfc-b8a246a0ba7a\") " Dec 03 13:18:00 crc kubenswrapper[5052]: I1203 13:18:00.938114 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/72c65803-7d22-4b27-9cfc-b8a246a0ba7a-logs" (OuterVolumeSpecName: "logs") pod "72c65803-7d22-4b27-9cfc-b8a246a0ba7a" (UID: "72c65803-7d22-4b27-9cfc-b8a246a0ba7a"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 13:18:00 crc kubenswrapper[5052]: I1203 13:18:00.943374 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/72c65803-7d22-4b27-9cfc-b8a246a0ba7a-config-data-custom" (OuterVolumeSpecName: "config-data-custom") pod "72c65803-7d22-4b27-9cfc-b8a246a0ba7a" (UID: "72c65803-7d22-4b27-9cfc-b8a246a0ba7a"). InnerVolumeSpecName "config-data-custom". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:18:00 crc kubenswrapper[5052]: I1203 13:18:00.949414 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/72c65803-7d22-4b27-9cfc-b8a246a0ba7a-kube-api-access-4glrf" (OuterVolumeSpecName: "kube-api-access-4glrf") pod "72c65803-7d22-4b27-9cfc-b8a246a0ba7a" (UID: "72c65803-7d22-4b27-9cfc-b8a246a0ba7a"). InnerVolumeSpecName "kube-api-access-4glrf". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:18:00 crc kubenswrapper[5052]: I1203 13:18:00.949963 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/72c65803-7d22-4b27-9cfc-b8a246a0ba7a-logs\") on node \"crc\" DevicePath \"\"" Dec 03 13:18:00 crc kubenswrapper[5052]: I1203 13:18:00.949981 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/72c65803-7d22-4b27-9cfc-b8a246a0ba7a-config-data-custom\") on node \"crc\" DevicePath \"\"" Dec 03 13:18:00 crc kubenswrapper[5052]: I1203 13:18:00.949992 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-4glrf\" (UniqueName: \"kubernetes.io/projected/72c65803-7d22-4b27-9cfc-b8a246a0ba7a-kube-api-access-4glrf\") on node \"crc\" DevicePath \"\"" Dec 03 13:18:00 crc kubenswrapper[5052]: I1203 13:18:00.990198 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/72c65803-7d22-4b27-9cfc-b8a246a0ba7a-config-data" (OuterVolumeSpecName: "config-data") pod "72c65803-7d22-4b27-9cfc-b8a246a0ba7a" (UID: "72c65803-7d22-4b27-9cfc-b8a246a0ba7a"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:18:01 crc kubenswrapper[5052]: I1203 13:18:01.014266 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/redhat-marketplace-w8xnr"] Dec 03 13:18:01 crc kubenswrapper[5052]: I1203 13:18:01.050924 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/72c65803-7d22-4b27-9cfc-b8a246a0ba7a-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 13:18:01 crc kubenswrapper[5052]: I1203 13:18:01.231928 5052 generic.go:334] "Generic (PLEG): container finished" podID="411fcfda-357c-4967-b59f-54cb173c3ce5" containerID="61f1e8f662aa9d093bfafe8ab44ac5d48ac59a5f8a06613bc37e62a1f287c9ef" exitCode=0 Dec 03 13:18:01 crc kubenswrapper[5052]: I1203 13:18:01.232041 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-marketplace-w8xnr" event={"ID":"411fcfda-357c-4967-b59f-54cb173c3ce5","Type":"ContainerDied","Data":"61f1e8f662aa9d093bfafe8ab44ac5d48ac59a5f8a06613bc37e62a1f287c9ef"} Dec 03 13:18:01 crc kubenswrapper[5052]: I1203 13:18:01.232261 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-marketplace-w8xnr" event={"ID":"411fcfda-357c-4967-b59f-54cb173c3ce5","Type":"ContainerStarted","Data":"fceecf4a4b59f96e3c69167dcf160a070a862172ff4ebab2eef490575eae60a4"} Dec 03 13:18:01 crc kubenswrapper[5052]: I1203 13:18:01.235629 5052 generic.go:334] "Generic (PLEG): container finished" podID="72c65803-7d22-4b27-9cfc-b8a246a0ba7a" containerID="9002ed94299a72447f2ba07c9dff1796e735f445645d52ddc8ab402fc97b4dce" exitCode=0 Dec 03 13:18:01 crc kubenswrapper[5052]: I1203 13:18:01.235661 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="barbican-kuttl-tests/barbican-worker-74467844f5-w7kwd" event={"ID":"72c65803-7d22-4b27-9cfc-b8a246a0ba7a","Type":"ContainerDied","Data":"9002ed94299a72447f2ba07c9dff1796e735f445645d52ddc8ab402fc97b4dce"} Dec 03 13:18:01 crc kubenswrapper[5052]: I1203 13:18:01.235708 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="barbican-kuttl-tests/barbican-worker-74467844f5-w7kwd" event={"ID":"72c65803-7d22-4b27-9cfc-b8a246a0ba7a","Type":"ContainerDied","Data":"d2f2f65968f7b9706a4750c08370f44252c83ac459cdcabb00e2ca4e8ab28287"} Dec 03 13:18:01 crc kubenswrapper[5052]: I1203 13:18:01.235730 5052 scope.go:117] "RemoveContainer" containerID="9002ed94299a72447f2ba07c9dff1796e735f445645d52ddc8ab402fc97b4dce" Dec 03 13:18:01 crc kubenswrapper[5052]: I1203 13:18:01.235663 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="barbican-kuttl-tests/barbican-worker-74467844f5-w7kwd" Dec 03 13:18:01 crc kubenswrapper[5052]: I1203 13:18:01.259913 5052 scope.go:117] "RemoveContainer" containerID="1a6e83c31c16b733e0a9dddbbfeffb1a8033e40fe0d2444509c93a09efb4dcf5" Dec 03 13:18:01 crc kubenswrapper[5052]: I1203 13:18:01.274994 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["barbican-kuttl-tests/barbican-worker-74467844f5-w7kwd"] Dec 03 13:18:01 crc kubenswrapper[5052]: I1203 13:18:01.287679 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["barbican-kuttl-tests/barbican-worker-74467844f5-w7kwd"] Dec 03 13:18:01 crc kubenswrapper[5052]: I1203 13:18:01.287698 5052 scope.go:117] "RemoveContainer" containerID="9002ed94299a72447f2ba07c9dff1796e735f445645d52ddc8ab402fc97b4dce" Dec 03 13:18:01 crc kubenswrapper[5052]: E1203 13:18:01.288217 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9002ed94299a72447f2ba07c9dff1796e735f445645d52ddc8ab402fc97b4dce\": container with ID starting with 9002ed94299a72447f2ba07c9dff1796e735f445645d52ddc8ab402fc97b4dce not found: ID does not exist" containerID="9002ed94299a72447f2ba07c9dff1796e735f445645d52ddc8ab402fc97b4dce" Dec 03 13:18:01 crc kubenswrapper[5052]: I1203 13:18:01.288262 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9002ed94299a72447f2ba07c9dff1796e735f445645d52ddc8ab402fc97b4dce"} err="failed to get container status \"9002ed94299a72447f2ba07c9dff1796e735f445645d52ddc8ab402fc97b4dce\": rpc error: code = NotFound desc = could not find container \"9002ed94299a72447f2ba07c9dff1796e735f445645d52ddc8ab402fc97b4dce\": container with ID starting with 9002ed94299a72447f2ba07c9dff1796e735f445645d52ddc8ab402fc97b4dce not found: ID does not exist" Dec 03 13:18:01 crc kubenswrapper[5052]: I1203 13:18:01.288289 5052 scope.go:117] "RemoveContainer" containerID="1a6e83c31c16b733e0a9dddbbfeffb1a8033e40fe0d2444509c93a09efb4dcf5" Dec 03 13:18:01 crc kubenswrapper[5052]: E1203 13:18:01.288835 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1a6e83c31c16b733e0a9dddbbfeffb1a8033e40fe0d2444509c93a09efb4dcf5\": container with ID starting with 1a6e83c31c16b733e0a9dddbbfeffb1a8033e40fe0d2444509c93a09efb4dcf5 not found: ID does not exist" containerID="1a6e83c31c16b733e0a9dddbbfeffb1a8033e40fe0d2444509c93a09efb4dcf5" Dec 03 13:18:01 crc kubenswrapper[5052]: I1203 13:18:01.288875 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1a6e83c31c16b733e0a9dddbbfeffb1a8033e40fe0d2444509c93a09efb4dcf5"} err="failed to get container status \"1a6e83c31c16b733e0a9dddbbfeffb1a8033e40fe0d2444509c93a09efb4dcf5\": rpc error: code = NotFound desc = could not find container \"1a6e83c31c16b733e0a9dddbbfeffb1a8033e40fe0d2444509c93a09efb4dcf5\": container with ID starting with 1a6e83c31c16b733e0a9dddbbfeffb1a8033e40fe0d2444509c93a09efb4dcf5 not found: ID does not exist" Dec 03 13:18:01 crc kubenswrapper[5052]: W1203 13:18:01.292761 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod53dce915_20c6_40ce_91ca_6d1499d9165f.slice/crio-6672a364f4d98db63279cbf9cff1b36f6cdff376415d54d182ff9a2b0f5e9ec3 WatchSource:0}: Error finding container 6672a364f4d98db63279cbf9cff1b36f6cdff376415d54d182ff9a2b0f5e9ec3: Status 404 returned error can't find the container with id 6672a364f4d98db63279cbf9cff1b36f6cdff376415d54d182ff9a2b0f5e9ec3 Dec 03 13:18:01 crc kubenswrapper[5052]: I1203 13:18:01.294761 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["barbican-kuttl-tests/barbicanc550-account-delete-lxl47"] Dec 03 13:18:01 crc kubenswrapper[5052]: I1203 13:18:01.684148 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3a71e182-1787-48f5-a4f8-60b9fb56f603" path="/var/lib/kubelet/pods/3a71e182-1787-48f5-a4f8-60b9fb56f603/volumes" Dec 03 13:18:01 crc kubenswrapper[5052]: I1203 13:18:01.685635 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="47809cbf-0aa5-4d38-8b9f-95ce55763ed3" path="/var/lib/kubelet/pods/47809cbf-0aa5-4d38-8b9f-95ce55763ed3/volumes" Dec 03 13:18:01 crc kubenswrapper[5052]: I1203 13:18:01.686568 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="72c65803-7d22-4b27-9cfc-b8a246a0ba7a" path="/var/lib/kubelet/pods/72c65803-7d22-4b27-9cfc-b8a246a0ba7a/volumes" Dec 03 13:18:01 crc kubenswrapper[5052]: I1203 13:18:01.687821 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="82893f5c-2185-4a6a-8b4a-a3d4c4dd92b5" path="/var/lib/kubelet/pods/82893f5c-2185-4a6a-8b4a-a3d4c4dd92b5/volumes" Dec 03 13:18:01 crc kubenswrapper[5052]: I1203 13:18:01.688615 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a5e071a4-f78a-46b3-b8a6-5880f4eecd2a" path="/var/lib/kubelet/pods/a5e071a4-f78a-46b3-b8a6-5880f4eecd2a/volumes" Dec 03 13:18:01 crc kubenswrapper[5052]: I1203 13:18:01.689301 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e033a810-2874-4fb9-ab5b-654fb638cf18" path="/var/lib/kubelet/pods/e033a810-2874-4fb9-ab5b-654fb638cf18/volumes" Dec 03 13:18:02 crc kubenswrapper[5052]: I1203 13:18:02.246126 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-marketplace-w8xnr" event={"ID":"411fcfda-357c-4967-b59f-54cb173c3ce5","Type":"ContainerStarted","Data":"a892789735abbd055df01cf8ac89215cb376bd65004bc2af80955a24c37a6a95"} Dec 03 13:18:02 crc kubenswrapper[5052]: I1203 13:18:02.248591 5052 generic.go:334] "Generic (PLEG): container finished" podID="53dce915-20c6-40ce-91ca-6d1499d9165f" containerID="72266a7bb1fe568dab23c67dcac94727a3bd4a60e5dd4652c1b945ef1e716bc2" exitCode=0 Dec 03 13:18:02 crc kubenswrapper[5052]: I1203 13:18:02.248664 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="barbican-kuttl-tests/barbicanc550-account-delete-lxl47" event={"ID":"53dce915-20c6-40ce-91ca-6d1499d9165f","Type":"ContainerDied","Data":"72266a7bb1fe568dab23c67dcac94727a3bd4a60e5dd4652c1b945ef1e716bc2"} Dec 03 13:18:02 crc kubenswrapper[5052]: I1203 13:18:02.248696 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="barbican-kuttl-tests/barbicanc550-account-delete-lxl47" event={"ID":"53dce915-20c6-40ce-91ca-6d1499d9165f","Type":"ContainerStarted","Data":"6672a364f4d98db63279cbf9cff1b36f6cdff376415d54d182ff9a2b0f5e9ec3"} Dec 03 13:18:03 crc kubenswrapper[5052]: I1203 13:18:03.263862 5052 generic.go:334] "Generic (PLEG): container finished" podID="411fcfda-357c-4967-b59f-54cb173c3ce5" containerID="a892789735abbd055df01cf8ac89215cb376bd65004bc2af80955a24c37a6a95" exitCode=0 Dec 03 13:18:03 crc kubenswrapper[5052]: I1203 13:18:03.263903 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-marketplace-w8xnr" event={"ID":"411fcfda-357c-4967-b59f-54cb173c3ce5","Type":"ContainerDied","Data":"a892789735abbd055df01cf8ac89215cb376bd65004bc2af80955a24c37a6a95"} Dec 03 13:18:03 crc kubenswrapper[5052]: I1203 13:18:03.674117 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="barbican-kuttl-tests/barbicanc550-account-delete-lxl47" Dec 03 13:18:03 crc kubenswrapper[5052]: I1203 13:18:03.798900 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fdb2f\" (UniqueName: \"kubernetes.io/projected/53dce915-20c6-40ce-91ca-6d1499d9165f-kube-api-access-fdb2f\") pod \"53dce915-20c6-40ce-91ca-6d1499d9165f\" (UID: \"53dce915-20c6-40ce-91ca-6d1499d9165f\") " Dec 03 13:18:03 crc kubenswrapper[5052]: I1203 13:18:03.798987 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/53dce915-20c6-40ce-91ca-6d1499d9165f-operator-scripts\") pod \"53dce915-20c6-40ce-91ca-6d1499d9165f\" (UID: \"53dce915-20c6-40ce-91ca-6d1499d9165f\") " Dec 03 13:18:03 crc kubenswrapper[5052]: I1203 13:18:03.799759 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/53dce915-20c6-40ce-91ca-6d1499d9165f-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "53dce915-20c6-40ce-91ca-6d1499d9165f" (UID: "53dce915-20c6-40ce-91ca-6d1499d9165f"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:18:03 crc kubenswrapper[5052]: I1203 13:18:03.812162 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/53dce915-20c6-40ce-91ca-6d1499d9165f-kube-api-access-fdb2f" (OuterVolumeSpecName: "kube-api-access-fdb2f") pod "53dce915-20c6-40ce-91ca-6d1499d9165f" (UID: "53dce915-20c6-40ce-91ca-6d1499d9165f"). InnerVolumeSpecName "kube-api-access-fdb2f". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:18:03 crc kubenswrapper[5052]: I1203 13:18:03.900820 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/53dce915-20c6-40ce-91ca-6d1499d9165f-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 13:18:03 crc kubenswrapper[5052]: I1203 13:18:03.900860 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-fdb2f\" (UniqueName: \"kubernetes.io/projected/53dce915-20c6-40ce-91ca-6d1499d9165f-kube-api-access-fdb2f\") on node \"crc\" DevicePath \"\"" Dec 03 13:18:04 crc kubenswrapper[5052]: I1203 13:18:04.275176 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-marketplace-w8xnr" event={"ID":"411fcfda-357c-4967-b59f-54cb173c3ce5","Type":"ContainerStarted","Data":"ae8cda6642051c25f353e5437b371c9e818873d754d74886089536dced67ee9b"} Dec 03 13:18:04 crc kubenswrapper[5052]: I1203 13:18:04.277140 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="barbican-kuttl-tests/barbicanc550-account-delete-lxl47" event={"ID":"53dce915-20c6-40ce-91ca-6d1499d9165f","Type":"ContainerDied","Data":"6672a364f4d98db63279cbf9cff1b36f6cdff376415d54d182ff9a2b0f5e9ec3"} Dec 03 13:18:04 crc kubenswrapper[5052]: I1203 13:18:04.277188 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6672a364f4d98db63279cbf9cff1b36f6cdff376415d54d182ff9a2b0f5e9ec3" Dec 03 13:18:04 crc kubenswrapper[5052]: I1203 13:18:04.277233 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="barbican-kuttl-tests/barbicanc550-account-delete-lxl47" Dec 03 13:18:04 crc kubenswrapper[5052]: I1203 13:18:04.312373 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-marketplace/redhat-marketplace-w8xnr" podStartSLOduration=1.740452732 podStartE2EDuration="4.312349845s" podCreationTimestamp="2025-12-03 13:18:00 +0000 UTC" firstStartedPulling="2025-12-03 13:18:01.23335882 +0000 UTC m=+7071.821600584" lastFinishedPulling="2025-12-03 13:18:03.805255943 +0000 UTC m=+7074.393497697" observedRunningTime="2025-12-03 13:18:04.305810548 +0000 UTC m=+7074.894052312" watchObservedRunningTime="2025-12-03 13:18:04.312349845 +0000 UTC m=+7074.900591609" Dec 03 13:18:05 crc kubenswrapper[5052]: I1203 13:18:05.323290 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["barbican-kuttl-tests/barbican-db-create-dln98"] Dec 03 13:18:05 crc kubenswrapper[5052]: I1203 13:18:05.332626 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["barbican-kuttl-tests/barbican-db-create-dln98"] Dec 03 13:18:05 crc kubenswrapper[5052]: I1203 13:18:05.340879 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["barbican-kuttl-tests/barbicanc550-account-delete-lxl47"] Dec 03 13:18:05 crc kubenswrapper[5052]: I1203 13:18:05.348160 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["barbican-kuttl-tests/barbican-c550-account-create-update-mfpdv"] Dec 03 13:18:05 crc kubenswrapper[5052]: I1203 13:18:05.355404 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["barbican-kuttl-tests/barbican-c550-account-create-update-mfpdv"] Dec 03 13:18:05 crc kubenswrapper[5052]: I1203 13:18:05.363668 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["barbican-kuttl-tests/barbicanc550-account-delete-lxl47"] Dec 03 13:18:05 crc kubenswrapper[5052]: I1203 13:18:05.419691 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["barbican-kuttl-tests/barbican-db-create-kk889"] Dec 03 13:18:05 crc kubenswrapper[5052]: E1203 13:18:05.420005 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="72c65803-7d22-4b27-9cfc-b8a246a0ba7a" containerName="barbican-worker-log" Dec 03 13:18:05 crc kubenswrapper[5052]: I1203 13:18:05.420042 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="72c65803-7d22-4b27-9cfc-b8a246a0ba7a" containerName="barbican-worker-log" Dec 03 13:18:05 crc kubenswrapper[5052]: E1203 13:18:05.420059 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="72c65803-7d22-4b27-9cfc-b8a246a0ba7a" containerName="barbican-worker" Dec 03 13:18:05 crc kubenswrapper[5052]: I1203 13:18:05.420067 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="72c65803-7d22-4b27-9cfc-b8a246a0ba7a" containerName="barbican-worker" Dec 03 13:18:05 crc kubenswrapper[5052]: E1203 13:18:05.420075 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="53dce915-20c6-40ce-91ca-6d1499d9165f" containerName="mariadb-account-delete" Dec 03 13:18:05 crc kubenswrapper[5052]: I1203 13:18:05.420081 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="53dce915-20c6-40ce-91ca-6d1499d9165f" containerName="mariadb-account-delete" Dec 03 13:18:05 crc kubenswrapper[5052]: I1203 13:18:05.420216 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="53dce915-20c6-40ce-91ca-6d1499d9165f" containerName="mariadb-account-delete" Dec 03 13:18:05 crc kubenswrapper[5052]: I1203 13:18:05.420238 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="72c65803-7d22-4b27-9cfc-b8a246a0ba7a" containerName="barbican-worker" Dec 03 13:18:05 crc kubenswrapper[5052]: I1203 13:18:05.420249 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="72c65803-7d22-4b27-9cfc-b8a246a0ba7a" containerName="barbican-worker-log" Dec 03 13:18:05 crc kubenswrapper[5052]: I1203 13:18:05.420832 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="barbican-kuttl-tests/barbican-db-create-kk889" Dec 03 13:18:05 crc kubenswrapper[5052]: I1203 13:18:05.429573 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["barbican-kuttl-tests/barbican-db-create-kk889"] Dec 03 13:18:05 crc kubenswrapper[5052]: I1203 13:18:05.523834 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2ptw4\" (UniqueName: \"kubernetes.io/projected/1d6fbb80-d818-42ba-9bf7-caf7148a7a19-kube-api-access-2ptw4\") pod \"barbican-db-create-kk889\" (UID: \"1d6fbb80-d818-42ba-9bf7-caf7148a7a19\") " pod="barbican-kuttl-tests/barbican-db-create-kk889" Dec 03 13:18:05 crc kubenswrapper[5052]: I1203 13:18:05.523906 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/1d6fbb80-d818-42ba-9bf7-caf7148a7a19-operator-scripts\") pod \"barbican-db-create-kk889\" (UID: \"1d6fbb80-d818-42ba-9bf7-caf7148a7a19\") " pod="barbican-kuttl-tests/barbican-db-create-kk889" Dec 03 13:18:05 crc kubenswrapper[5052]: I1203 13:18:05.525469 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["barbican-kuttl-tests/barbican-a311-account-create-update-c5mpp"] Dec 03 13:18:05 crc kubenswrapper[5052]: I1203 13:18:05.527327 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="barbican-kuttl-tests/barbican-a311-account-create-update-c5mpp" Dec 03 13:18:05 crc kubenswrapper[5052]: I1203 13:18:05.530015 5052 reflector.go:368] Caches populated for *v1.Secret from object-"barbican-kuttl-tests"/"barbican-db-secret" Dec 03 13:18:05 crc kubenswrapper[5052]: I1203 13:18:05.534077 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["barbican-kuttl-tests/barbican-a311-account-create-update-c5mpp"] Dec 03 13:18:05 crc kubenswrapper[5052]: I1203 13:18:05.625863 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-2ptw4\" (UniqueName: \"kubernetes.io/projected/1d6fbb80-d818-42ba-9bf7-caf7148a7a19-kube-api-access-2ptw4\") pod \"barbican-db-create-kk889\" (UID: \"1d6fbb80-d818-42ba-9bf7-caf7148a7a19\") " pod="barbican-kuttl-tests/barbican-db-create-kk889" Dec 03 13:18:05 crc kubenswrapper[5052]: I1203 13:18:05.625969 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/1d6fbb80-d818-42ba-9bf7-caf7148a7a19-operator-scripts\") pod \"barbican-db-create-kk889\" (UID: \"1d6fbb80-d818-42ba-9bf7-caf7148a7a19\") " pod="barbican-kuttl-tests/barbican-db-create-kk889" Dec 03 13:18:05 crc kubenswrapper[5052]: I1203 13:18:05.626036 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/f1847d2e-1709-40fe-ae2b-97c1295a186e-operator-scripts\") pod \"barbican-a311-account-create-update-c5mpp\" (UID: \"f1847d2e-1709-40fe-ae2b-97c1295a186e\") " pod="barbican-kuttl-tests/barbican-a311-account-create-update-c5mpp" Dec 03 13:18:05 crc kubenswrapper[5052]: I1203 13:18:05.626101 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xtnbk\" (UniqueName: \"kubernetes.io/projected/f1847d2e-1709-40fe-ae2b-97c1295a186e-kube-api-access-xtnbk\") pod \"barbican-a311-account-create-update-c5mpp\" (UID: \"f1847d2e-1709-40fe-ae2b-97c1295a186e\") " pod="barbican-kuttl-tests/barbican-a311-account-create-update-c5mpp" Dec 03 13:18:05 crc kubenswrapper[5052]: I1203 13:18:05.626948 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/1d6fbb80-d818-42ba-9bf7-caf7148a7a19-operator-scripts\") pod \"barbican-db-create-kk889\" (UID: \"1d6fbb80-d818-42ba-9bf7-caf7148a7a19\") " pod="barbican-kuttl-tests/barbican-db-create-kk889" Dec 03 13:18:05 crc kubenswrapper[5052]: I1203 13:18:05.653478 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-2ptw4\" (UniqueName: \"kubernetes.io/projected/1d6fbb80-d818-42ba-9bf7-caf7148a7a19-kube-api-access-2ptw4\") pod \"barbican-db-create-kk889\" (UID: \"1d6fbb80-d818-42ba-9bf7-caf7148a7a19\") " pod="barbican-kuttl-tests/barbican-db-create-kk889" Dec 03 13:18:05 crc kubenswrapper[5052]: I1203 13:18:05.684238 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="53dce915-20c6-40ce-91ca-6d1499d9165f" path="/var/lib/kubelet/pods/53dce915-20c6-40ce-91ca-6d1499d9165f/volumes" Dec 03 13:18:05 crc kubenswrapper[5052]: I1203 13:18:05.685063 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="711275ad-0d01-45b9-a19c-93d45b52ec17" path="/var/lib/kubelet/pods/711275ad-0d01-45b9-a19c-93d45b52ec17/volumes" Dec 03 13:18:05 crc kubenswrapper[5052]: I1203 13:18:05.685951 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="da4f9b2e-4f41-4ddd-8059-1f9bad922873" path="/var/lib/kubelet/pods/da4f9b2e-4f41-4ddd-8059-1f9bad922873/volumes" Dec 03 13:18:05 crc kubenswrapper[5052]: I1203 13:18:05.727382 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/f1847d2e-1709-40fe-ae2b-97c1295a186e-operator-scripts\") pod \"barbican-a311-account-create-update-c5mpp\" (UID: \"f1847d2e-1709-40fe-ae2b-97c1295a186e\") " pod="barbican-kuttl-tests/barbican-a311-account-create-update-c5mpp" Dec 03 13:18:05 crc kubenswrapper[5052]: I1203 13:18:05.727497 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-xtnbk\" (UniqueName: \"kubernetes.io/projected/f1847d2e-1709-40fe-ae2b-97c1295a186e-kube-api-access-xtnbk\") pod \"barbican-a311-account-create-update-c5mpp\" (UID: \"f1847d2e-1709-40fe-ae2b-97c1295a186e\") " pod="barbican-kuttl-tests/barbican-a311-account-create-update-c5mpp" Dec 03 13:18:05 crc kubenswrapper[5052]: I1203 13:18:05.728350 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/f1847d2e-1709-40fe-ae2b-97c1295a186e-operator-scripts\") pod \"barbican-a311-account-create-update-c5mpp\" (UID: \"f1847d2e-1709-40fe-ae2b-97c1295a186e\") " pod="barbican-kuttl-tests/barbican-a311-account-create-update-c5mpp" Dec 03 13:18:05 crc kubenswrapper[5052]: I1203 13:18:05.739173 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="barbican-kuttl-tests/barbican-db-create-kk889" Dec 03 13:18:05 crc kubenswrapper[5052]: I1203 13:18:05.748921 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-xtnbk\" (UniqueName: \"kubernetes.io/projected/f1847d2e-1709-40fe-ae2b-97c1295a186e-kube-api-access-xtnbk\") pod \"barbican-a311-account-create-update-c5mpp\" (UID: \"f1847d2e-1709-40fe-ae2b-97c1295a186e\") " pod="barbican-kuttl-tests/barbican-a311-account-create-update-c5mpp" Dec 03 13:18:05 crc kubenswrapper[5052]: I1203 13:18:05.848366 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="barbican-kuttl-tests/barbican-a311-account-create-update-c5mpp" Dec 03 13:18:06 crc kubenswrapper[5052]: W1203 13:18:06.198979 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1d6fbb80_d818_42ba_9bf7_caf7148a7a19.slice/crio-eea323e42c33f419c676d1030dde67c534cc4095db69a4a6f82d30012b862e4a WatchSource:0}: Error finding container eea323e42c33f419c676d1030dde67c534cc4095db69a4a6f82d30012b862e4a: Status 404 returned error can't find the container with id eea323e42c33f419c676d1030dde67c534cc4095db69a4a6f82d30012b862e4a Dec 03 13:18:06 crc kubenswrapper[5052]: I1203 13:18:06.207027 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["barbican-kuttl-tests/barbican-db-create-kk889"] Dec 03 13:18:06 crc kubenswrapper[5052]: I1203 13:18:06.303389 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["barbican-kuttl-tests/barbican-a311-account-create-update-c5mpp"] Dec 03 13:18:06 crc kubenswrapper[5052]: I1203 13:18:06.305713 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="barbican-kuttl-tests/barbican-db-create-kk889" event={"ID":"1d6fbb80-d818-42ba-9bf7-caf7148a7a19","Type":"ContainerStarted","Data":"eea323e42c33f419c676d1030dde67c534cc4095db69a4a6f82d30012b862e4a"} Dec 03 13:18:06 crc kubenswrapper[5052]: W1203 13:18:06.315684 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf1847d2e_1709_40fe_ae2b_97c1295a186e.slice/crio-d2262248c634addbc39b3f79d11a7828bbf9a41aa0ac75dc84fefa55578c6c66 WatchSource:0}: Error finding container d2262248c634addbc39b3f79d11a7828bbf9a41aa0ac75dc84fefa55578c6c66: Status 404 returned error can't find the container with id d2262248c634addbc39b3f79d11a7828bbf9a41aa0ac75dc84fefa55578c6c66 Dec 03 13:18:07 crc kubenswrapper[5052]: I1203 13:18:07.315244 5052 generic.go:334] "Generic (PLEG): container finished" podID="f1847d2e-1709-40fe-ae2b-97c1295a186e" containerID="de9f481ae76794b2e2c23743632c15a110f79b63254a4044d0999c1775b41e20" exitCode=0 Dec 03 13:18:07 crc kubenswrapper[5052]: I1203 13:18:07.315292 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="barbican-kuttl-tests/barbican-a311-account-create-update-c5mpp" event={"ID":"f1847d2e-1709-40fe-ae2b-97c1295a186e","Type":"ContainerDied","Data":"de9f481ae76794b2e2c23743632c15a110f79b63254a4044d0999c1775b41e20"} Dec 03 13:18:07 crc kubenswrapper[5052]: I1203 13:18:07.315617 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="barbican-kuttl-tests/barbican-a311-account-create-update-c5mpp" event={"ID":"f1847d2e-1709-40fe-ae2b-97c1295a186e","Type":"ContainerStarted","Data":"d2262248c634addbc39b3f79d11a7828bbf9a41aa0ac75dc84fefa55578c6c66"} Dec 03 13:18:07 crc kubenswrapper[5052]: I1203 13:18:07.318409 5052 generic.go:334] "Generic (PLEG): container finished" podID="1d6fbb80-d818-42ba-9bf7-caf7148a7a19" containerID="c84ad400a7a634dd4ee8e626c400c2c08de9f53fbccffb6eb453f6b48b379e9a" exitCode=0 Dec 03 13:18:07 crc kubenswrapper[5052]: I1203 13:18:07.318453 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="barbican-kuttl-tests/barbican-db-create-kk889" event={"ID":"1d6fbb80-d818-42ba-9bf7-caf7148a7a19","Type":"ContainerDied","Data":"c84ad400a7a634dd4ee8e626c400c2c08de9f53fbccffb6eb453f6b48b379e9a"} Dec 03 13:18:08 crc kubenswrapper[5052]: I1203 13:18:08.717085 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="barbican-kuttl-tests/barbican-a311-account-create-update-c5mpp" Dec 03 13:18:08 crc kubenswrapper[5052]: I1203 13:18:08.723586 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="barbican-kuttl-tests/barbican-db-create-kk889" Dec 03 13:18:08 crc kubenswrapper[5052]: I1203 13:18:08.872831 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2ptw4\" (UniqueName: \"kubernetes.io/projected/1d6fbb80-d818-42ba-9bf7-caf7148a7a19-kube-api-access-2ptw4\") pod \"1d6fbb80-d818-42ba-9bf7-caf7148a7a19\" (UID: \"1d6fbb80-d818-42ba-9bf7-caf7148a7a19\") " Dec 03 13:18:08 crc kubenswrapper[5052]: I1203 13:18:08.872954 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xtnbk\" (UniqueName: \"kubernetes.io/projected/f1847d2e-1709-40fe-ae2b-97c1295a186e-kube-api-access-xtnbk\") pod \"f1847d2e-1709-40fe-ae2b-97c1295a186e\" (UID: \"f1847d2e-1709-40fe-ae2b-97c1295a186e\") " Dec 03 13:18:08 crc kubenswrapper[5052]: I1203 13:18:08.873027 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/f1847d2e-1709-40fe-ae2b-97c1295a186e-operator-scripts\") pod \"f1847d2e-1709-40fe-ae2b-97c1295a186e\" (UID: \"f1847d2e-1709-40fe-ae2b-97c1295a186e\") " Dec 03 13:18:08 crc kubenswrapper[5052]: I1203 13:18:08.873091 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/1d6fbb80-d818-42ba-9bf7-caf7148a7a19-operator-scripts\") pod \"1d6fbb80-d818-42ba-9bf7-caf7148a7a19\" (UID: \"1d6fbb80-d818-42ba-9bf7-caf7148a7a19\") " Dec 03 13:18:08 crc kubenswrapper[5052]: I1203 13:18:08.873903 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f1847d2e-1709-40fe-ae2b-97c1295a186e-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "f1847d2e-1709-40fe-ae2b-97c1295a186e" (UID: "f1847d2e-1709-40fe-ae2b-97c1295a186e"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:18:08 crc kubenswrapper[5052]: I1203 13:18:08.873938 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1d6fbb80-d818-42ba-9bf7-caf7148a7a19-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "1d6fbb80-d818-42ba-9bf7-caf7148a7a19" (UID: "1d6fbb80-d818-42ba-9bf7-caf7148a7a19"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:18:08 crc kubenswrapper[5052]: I1203 13:18:08.878732 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f1847d2e-1709-40fe-ae2b-97c1295a186e-kube-api-access-xtnbk" (OuterVolumeSpecName: "kube-api-access-xtnbk") pod "f1847d2e-1709-40fe-ae2b-97c1295a186e" (UID: "f1847d2e-1709-40fe-ae2b-97c1295a186e"). InnerVolumeSpecName "kube-api-access-xtnbk". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:18:08 crc kubenswrapper[5052]: I1203 13:18:08.878797 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1d6fbb80-d818-42ba-9bf7-caf7148a7a19-kube-api-access-2ptw4" (OuterVolumeSpecName: "kube-api-access-2ptw4") pod "1d6fbb80-d818-42ba-9bf7-caf7148a7a19" (UID: "1d6fbb80-d818-42ba-9bf7-caf7148a7a19"). InnerVolumeSpecName "kube-api-access-2ptw4". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:18:08 crc kubenswrapper[5052]: I1203 13:18:08.975089 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/1d6fbb80-d818-42ba-9bf7-caf7148a7a19-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 13:18:08 crc kubenswrapper[5052]: I1203 13:18:08.975126 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-2ptw4\" (UniqueName: \"kubernetes.io/projected/1d6fbb80-d818-42ba-9bf7-caf7148a7a19-kube-api-access-2ptw4\") on node \"crc\" DevicePath \"\"" Dec 03 13:18:08 crc kubenswrapper[5052]: I1203 13:18:08.975138 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-xtnbk\" (UniqueName: \"kubernetes.io/projected/f1847d2e-1709-40fe-ae2b-97c1295a186e-kube-api-access-xtnbk\") on node \"crc\" DevicePath \"\"" Dec 03 13:18:08 crc kubenswrapper[5052]: I1203 13:18:08.975147 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/f1847d2e-1709-40fe-ae2b-97c1295a186e-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 13:18:09 crc kubenswrapper[5052]: I1203 13:18:09.334123 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="barbican-kuttl-tests/barbican-db-create-kk889" event={"ID":"1d6fbb80-d818-42ba-9bf7-caf7148a7a19","Type":"ContainerDied","Data":"eea323e42c33f419c676d1030dde67c534cc4095db69a4a6f82d30012b862e4a"} Dec 03 13:18:09 crc kubenswrapper[5052]: I1203 13:18:09.334473 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="eea323e42c33f419c676d1030dde67c534cc4095db69a4a6f82d30012b862e4a" Dec 03 13:18:09 crc kubenswrapper[5052]: I1203 13:18:09.334148 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="barbican-kuttl-tests/barbican-db-create-kk889" Dec 03 13:18:09 crc kubenswrapper[5052]: I1203 13:18:09.336453 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="barbican-kuttl-tests/barbican-a311-account-create-update-c5mpp" event={"ID":"f1847d2e-1709-40fe-ae2b-97c1295a186e","Type":"ContainerDied","Data":"d2262248c634addbc39b3f79d11a7828bbf9a41aa0ac75dc84fefa55578c6c66"} Dec 03 13:18:09 crc kubenswrapper[5052]: I1203 13:18:09.336482 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d2262248c634addbc39b3f79d11a7828bbf9a41aa0ac75dc84fefa55578c6c66" Dec 03 13:18:09 crc kubenswrapper[5052]: I1203 13:18:09.336488 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="barbican-kuttl-tests/barbican-a311-account-create-update-c5mpp" Dec 03 13:18:10 crc kubenswrapper[5052]: I1203 13:18:10.401990 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openshift-marketplace/redhat-marketplace-w8xnr" Dec 03 13:18:10 crc kubenswrapper[5052]: I1203 13:18:10.402047 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-marketplace/redhat-marketplace-w8xnr" Dec 03 13:18:10 crc kubenswrapper[5052]: I1203 13:18:10.444749 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-marketplace/redhat-marketplace-w8xnr" Dec 03 13:18:10 crc kubenswrapper[5052]: I1203 13:18:10.783986 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["barbican-kuttl-tests/barbican-db-sync-2dm58"] Dec 03 13:18:10 crc kubenswrapper[5052]: E1203 13:18:10.784834 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="f1847d2e-1709-40fe-ae2b-97c1295a186e" containerName="mariadb-account-create-update" Dec 03 13:18:10 crc kubenswrapper[5052]: I1203 13:18:10.784921 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="f1847d2e-1709-40fe-ae2b-97c1295a186e" containerName="mariadb-account-create-update" Dec 03 13:18:10 crc kubenswrapper[5052]: E1203 13:18:10.785053 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="1d6fbb80-d818-42ba-9bf7-caf7148a7a19" containerName="mariadb-database-create" Dec 03 13:18:10 crc kubenswrapper[5052]: I1203 13:18:10.785224 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="1d6fbb80-d818-42ba-9bf7-caf7148a7a19" containerName="mariadb-database-create" Dec 03 13:18:10 crc kubenswrapper[5052]: I1203 13:18:10.785479 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="1d6fbb80-d818-42ba-9bf7-caf7148a7a19" containerName="mariadb-database-create" Dec 03 13:18:10 crc kubenswrapper[5052]: I1203 13:18:10.785586 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="f1847d2e-1709-40fe-ae2b-97c1295a186e" containerName="mariadb-account-create-update" Dec 03 13:18:10 crc kubenswrapper[5052]: I1203 13:18:10.786225 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="barbican-kuttl-tests/barbican-db-sync-2dm58" Dec 03 13:18:10 crc kubenswrapper[5052]: I1203 13:18:10.788094 5052 reflector.go:368] Caches populated for *v1.Secret from object-"barbican-kuttl-tests"/"barbican-barbican-dockercfg-mrz7d" Dec 03 13:18:10 crc kubenswrapper[5052]: I1203 13:18:10.788359 5052 reflector.go:368] Caches populated for *v1.Secret from object-"barbican-kuttl-tests"/"combined-ca-bundle" Dec 03 13:18:10 crc kubenswrapper[5052]: I1203 13:18:10.793798 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["barbican-kuttl-tests/barbican-db-sync-2dm58"] Dec 03 13:18:10 crc kubenswrapper[5052]: I1203 13:18:10.905655 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/62780bef-5fe0-4e1a-b6a1-62e6d5566cc0-db-sync-config-data\") pod \"barbican-db-sync-2dm58\" (UID: \"62780bef-5fe0-4e1a-b6a1-62e6d5566cc0\") " pod="barbican-kuttl-tests/barbican-db-sync-2dm58" Dec 03 13:18:10 crc kubenswrapper[5052]: I1203 13:18:10.905762 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/62780bef-5fe0-4e1a-b6a1-62e6d5566cc0-combined-ca-bundle\") pod \"barbican-db-sync-2dm58\" (UID: \"62780bef-5fe0-4e1a-b6a1-62e6d5566cc0\") " pod="barbican-kuttl-tests/barbican-db-sync-2dm58" Dec 03 13:18:10 crc kubenswrapper[5052]: I1203 13:18:10.905862 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vlgtn\" (UniqueName: \"kubernetes.io/projected/62780bef-5fe0-4e1a-b6a1-62e6d5566cc0-kube-api-access-vlgtn\") pod \"barbican-db-sync-2dm58\" (UID: \"62780bef-5fe0-4e1a-b6a1-62e6d5566cc0\") " pod="barbican-kuttl-tests/barbican-db-sync-2dm58" Dec 03 13:18:11 crc kubenswrapper[5052]: I1203 13:18:11.007454 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-vlgtn\" (UniqueName: \"kubernetes.io/projected/62780bef-5fe0-4e1a-b6a1-62e6d5566cc0-kube-api-access-vlgtn\") pod \"barbican-db-sync-2dm58\" (UID: \"62780bef-5fe0-4e1a-b6a1-62e6d5566cc0\") " pod="barbican-kuttl-tests/barbican-db-sync-2dm58" Dec 03 13:18:11 crc kubenswrapper[5052]: I1203 13:18:11.007536 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/62780bef-5fe0-4e1a-b6a1-62e6d5566cc0-db-sync-config-data\") pod \"barbican-db-sync-2dm58\" (UID: \"62780bef-5fe0-4e1a-b6a1-62e6d5566cc0\") " pod="barbican-kuttl-tests/barbican-db-sync-2dm58" Dec 03 13:18:11 crc kubenswrapper[5052]: I1203 13:18:11.007580 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/62780bef-5fe0-4e1a-b6a1-62e6d5566cc0-combined-ca-bundle\") pod \"barbican-db-sync-2dm58\" (UID: \"62780bef-5fe0-4e1a-b6a1-62e6d5566cc0\") " pod="barbican-kuttl-tests/barbican-db-sync-2dm58" Dec 03 13:18:11 crc kubenswrapper[5052]: I1203 13:18:11.025563 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/62780bef-5fe0-4e1a-b6a1-62e6d5566cc0-db-sync-config-data\") pod \"barbican-db-sync-2dm58\" (UID: \"62780bef-5fe0-4e1a-b6a1-62e6d5566cc0\") " pod="barbican-kuttl-tests/barbican-db-sync-2dm58" Dec 03 13:18:11 crc kubenswrapper[5052]: I1203 13:18:11.025635 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/62780bef-5fe0-4e1a-b6a1-62e6d5566cc0-combined-ca-bundle\") pod \"barbican-db-sync-2dm58\" (UID: \"62780bef-5fe0-4e1a-b6a1-62e6d5566cc0\") " pod="barbican-kuttl-tests/barbican-db-sync-2dm58" Dec 03 13:18:11 crc kubenswrapper[5052]: I1203 13:18:11.036810 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-vlgtn\" (UniqueName: \"kubernetes.io/projected/62780bef-5fe0-4e1a-b6a1-62e6d5566cc0-kube-api-access-vlgtn\") pod \"barbican-db-sync-2dm58\" (UID: \"62780bef-5fe0-4e1a-b6a1-62e6d5566cc0\") " pod="barbican-kuttl-tests/barbican-db-sync-2dm58" Dec 03 13:18:11 crc kubenswrapper[5052]: I1203 13:18:11.104479 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="barbican-kuttl-tests/barbican-db-sync-2dm58" Dec 03 13:18:11 crc kubenswrapper[5052]: I1203 13:18:11.393987 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-marketplace/redhat-marketplace-w8xnr" Dec 03 13:18:11 crc kubenswrapper[5052]: I1203 13:18:11.441201 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openshift-marketplace/redhat-marketplace-w8xnr"] Dec 03 13:18:11 crc kubenswrapper[5052]: I1203 13:18:11.542991 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["barbican-kuttl-tests/barbican-db-sync-2dm58"] Dec 03 13:18:12 crc kubenswrapper[5052]: I1203 13:18:12.360283 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="barbican-kuttl-tests/barbican-db-sync-2dm58" event={"ID":"62780bef-5fe0-4e1a-b6a1-62e6d5566cc0","Type":"ContainerStarted","Data":"d507079afcc0267ce43d476785ec8f34ff653c173761201507cb26592443c0fd"} Dec 03 13:18:12 crc kubenswrapper[5052]: I1203 13:18:12.360477 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="barbican-kuttl-tests/barbican-db-sync-2dm58" event={"ID":"62780bef-5fe0-4e1a-b6a1-62e6d5566cc0","Type":"ContainerStarted","Data":"34e1f47a0df17822b3d50bfa4bb47522b2b9ff49353626dd3e7a17d90bcc83b9"} Dec 03 13:18:12 crc kubenswrapper[5052]: I1203 13:18:12.373754 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="barbican-kuttl-tests/barbican-db-sync-2dm58" podStartSLOduration=2.373739085 podStartE2EDuration="2.373739085s" podCreationTimestamp="2025-12-03 13:18:10 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 13:18:12.372061897 +0000 UTC m=+7082.960303661" watchObservedRunningTime="2025-12-03 13:18:12.373739085 +0000 UTC m=+7082.961980839" Dec 03 13:18:13 crc kubenswrapper[5052]: I1203 13:18:13.370949 5052 generic.go:334] "Generic (PLEG): container finished" podID="62780bef-5fe0-4e1a-b6a1-62e6d5566cc0" containerID="d507079afcc0267ce43d476785ec8f34ff653c173761201507cb26592443c0fd" exitCode=0 Dec 03 13:18:13 crc kubenswrapper[5052]: I1203 13:18:13.371016 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="barbican-kuttl-tests/barbican-db-sync-2dm58" event={"ID":"62780bef-5fe0-4e1a-b6a1-62e6d5566cc0","Type":"ContainerDied","Data":"d507079afcc0267ce43d476785ec8f34ff653c173761201507cb26592443c0fd"} Dec 03 13:18:13 crc kubenswrapper[5052]: I1203 13:18:13.372345 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openshift-marketplace/redhat-marketplace-w8xnr" podUID="411fcfda-357c-4967-b59f-54cb173c3ce5" containerName="registry-server" containerID="cri-o://ae8cda6642051c25f353e5437b371c9e818873d754d74886089536dced67ee9b" gracePeriod=2 Dec 03 13:18:13 crc kubenswrapper[5052]: I1203 13:18:13.810629 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/redhat-marketplace-w8xnr" Dec 03 13:18:13 crc kubenswrapper[5052]: I1203 13:18:13.955367 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/411fcfda-357c-4967-b59f-54cb173c3ce5-catalog-content\") pod \"411fcfda-357c-4967-b59f-54cb173c3ce5\" (UID: \"411fcfda-357c-4967-b59f-54cb173c3ce5\") " Dec 03 13:18:13 crc kubenswrapper[5052]: I1203 13:18:13.955475 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/411fcfda-357c-4967-b59f-54cb173c3ce5-utilities\") pod \"411fcfda-357c-4967-b59f-54cb173c3ce5\" (UID: \"411fcfda-357c-4967-b59f-54cb173c3ce5\") " Dec 03 13:18:13 crc kubenswrapper[5052]: I1203 13:18:13.955574 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4hhvn\" (UniqueName: \"kubernetes.io/projected/411fcfda-357c-4967-b59f-54cb173c3ce5-kube-api-access-4hhvn\") pod \"411fcfda-357c-4967-b59f-54cb173c3ce5\" (UID: \"411fcfda-357c-4967-b59f-54cb173c3ce5\") " Dec 03 13:18:13 crc kubenswrapper[5052]: I1203 13:18:13.956651 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/411fcfda-357c-4967-b59f-54cb173c3ce5-utilities" (OuterVolumeSpecName: "utilities") pod "411fcfda-357c-4967-b59f-54cb173c3ce5" (UID: "411fcfda-357c-4967-b59f-54cb173c3ce5"). InnerVolumeSpecName "utilities". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 13:18:13 crc kubenswrapper[5052]: I1203 13:18:13.962770 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/411fcfda-357c-4967-b59f-54cb173c3ce5-kube-api-access-4hhvn" (OuterVolumeSpecName: "kube-api-access-4hhvn") pod "411fcfda-357c-4967-b59f-54cb173c3ce5" (UID: "411fcfda-357c-4967-b59f-54cb173c3ce5"). InnerVolumeSpecName "kube-api-access-4hhvn". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:18:13 crc kubenswrapper[5052]: I1203 13:18:13.975164 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/411fcfda-357c-4967-b59f-54cb173c3ce5-catalog-content" (OuterVolumeSpecName: "catalog-content") pod "411fcfda-357c-4967-b59f-54cb173c3ce5" (UID: "411fcfda-357c-4967-b59f-54cb173c3ce5"). InnerVolumeSpecName "catalog-content". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 13:18:14 crc kubenswrapper[5052]: I1203 13:18:14.057680 5052 reconciler_common.go:293] "Volume detached for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/411fcfda-357c-4967-b59f-54cb173c3ce5-utilities\") on node \"crc\" DevicePath \"\"" Dec 03 13:18:14 crc kubenswrapper[5052]: I1203 13:18:14.057715 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-4hhvn\" (UniqueName: \"kubernetes.io/projected/411fcfda-357c-4967-b59f-54cb173c3ce5-kube-api-access-4hhvn\") on node \"crc\" DevicePath \"\"" Dec 03 13:18:14 crc kubenswrapper[5052]: I1203 13:18:14.057727 5052 reconciler_common.go:293] "Volume detached for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/411fcfda-357c-4967-b59f-54cb173c3ce5-catalog-content\") on node \"crc\" DevicePath \"\"" Dec 03 13:18:14 crc kubenswrapper[5052]: I1203 13:18:14.388008 5052 generic.go:334] "Generic (PLEG): container finished" podID="411fcfda-357c-4967-b59f-54cb173c3ce5" containerID="ae8cda6642051c25f353e5437b371c9e818873d754d74886089536dced67ee9b" exitCode=0 Dec 03 13:18:14 crc kubenswrapper[5052]: I1203 13:18:14.388254 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/redhat-marketplace-w8xnr" Dec 03 13:18:14 crc kubenswrapper[5052]: I1203 13:18:14.390668 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-marketplace-w8xnr" event={"ID":"411fcfda-357c-4967-b59f-54cb173c3ce5","Type":"ContainerDied","Data":"ae8cda6642051c25f353e5437b371c9e818873d754d74886089536dced67ee9b"} Dec 03 13:18:14 crc kubenswrapper[5052]: I1203 13:18:14.390790 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-marketplace-w8xnr" event={"ID":"411fcfda-357c-4967-b59f-54cb173c3ce5","Type":"ContainerDied","Data":"fceecf4a4b59f96e3c69167dcf160a070a862172ff4ebab2eef490575eae60a4"} Dec 03 13:18:14 crc kubenswrapper[5052]: I1203 13:18:14.390828 5052 scope.go:117] "RemoveContainer" containerID="ae8cda6642051c25f353e5437b371c9e818873d754d74886089536dced67ee9b" Dec 03 13:18:14 crc kubenswrapper[5052]: I1203 13:18:14.420947 5052 scope.go:117] "RemoveContainer" containerID="a892789735abbd055df01cf8ac89215cb376bd65004bc2af80955a24c37a6a95" Dec 03 13:18:14 crc kubenswrapper[5052]: I1203 13:18:14.431690 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openshift-marketplace/redhat-marketplace-w8xnr"] Dec 03 13:18:14 crc kubenswrapper[5052]: I1203 13:18:14.447467 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openshift-marketplace/redhat-marketplace-w8xnr"] Dec 03 13:18:14 crc kubenswrapper[5052]: I1203 13:18:14.469945 5052 scope.go:117] "RemoveContainer" containerID="61f1e8f662aa9d093bfafe8ab44ac5d48ac59a5f8a06613bc37e62a1f287c9ef" Dec 03 13:18:14 crc kubenswrapper[5052]: I1203 13:18:14.529079 5052 scope.go:117] "RemoveContainer" containerID="ae8cda6642051c25f353e5437b371c9e818873d754d74886089536dced67ee9b" Dec 03 13:18:14 crc kubenswrapper[5052]: E1203 13:18:14.529576 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ae8cda6642051c25f353e5437b371c9e818873d754d74886089536dced67ee9b\": container with ID starting with ae8cda6642051c25f353e5437b371c9e818873d754d74886089536dced67ee9b not found: ID does not exist" containerID="ae8cda6642051c25f353e5437b371c9e818873d754d74886089536dced67ee9b" Dec 03 13:18:14 crc kubenswrapper[5052]: I1203 13:18:14.529608 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ae8cda6642051c25f353e5437b371c9e818873d754d74886089536dced67ee9b"} err="failed to get container status \"ae8cda6642051c25f353e5437b371c9e818873d754d74886089536dced67ee9b\": rpc error: code = NotFound desc = could not find container \"ae8cda6642051c25f353e5437b371c9e818873d754d74886089536dced67ee9b\": container with ID starting with ae8cda6642051c25f353e5437b371c9e818873d754d74886089536dced67ee9b not found: ID does not exist" Dec 03 13:18:14 crc kubenswrapper[5052]: I1203 13:18:14.529631 5052 scope.go:117] "RemoveContainer" containerID="a892789735abbd055df01cf8ac89215cb376bd65004bc2af80955a24c37a6a95" Dec 03 13:18:14 crc kubenswrapper[5052]: E1203 13:18:14.529830 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a892789735abbd055df01cf8ac89215cb376bd65004bc2af80955a24c37a6a95\": container with ID starting with a892789735abbd055df01cf8ac89215cb376bd65004bc2af80955a24c37a6a95 not found: ID does not exist" containerID="a892789735abbd055df01cf8ac89215cb376bd65004bc2af80955a24c37a6a95" Dec 03 13:18:14 crc kubenswrapper[5052]: I1203 13:18:14.529869 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a892789735abbd055df01cf8ac89215cb376bd65004bc2af80955a24c37a6a95"} err="failed to get container status \"a892789735abbd055df01cf8ac89215cb376bd65004bc2af80955a24c37a6a95\": rpc error: code = NotFound desc = could not find container \"a892789735abbd055df01cf8ac89215cb376bd65004bc2af80955a24c37a6a95\": container with ID starting with a892789735abbd055df01cf8ac89215cb376bd65004bc2af80955a24c37a6a95 not found: ID does not exist" Dec 03 13:18:14 crc kubenswrapper[5052]: I1203 13:18:14.529884 5052 scope.go:117] "RemoveContainer" containerID="61f1e8f662aa9d093bfafe8ab44ac5d48ac59a5f8a06613bc37e62a1f287c9ef" Dec 03 13:18:14 crc kubenswrapper[5052]: E1203 13:18:14.543901 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"61f1e8f662aa9d093bfafe8ab44ac5d48ac59a5f8a06613bc37e62a1f287c9ef\": container with ID starting with 61f1e8f662aa9d093bfafe8ab44ac5d48ac59a5f8a06613bc37e62a1f287c9ef not found: ID does not exist" containerID="61f1e8f662aa9d093bfafe8ab44ac5d48ac59a5f8a06613bc37e62a1f287c9ef" Dec 03 13:18:14 crc kubenswrapper[5052]: I1203 13:18:14.543953 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"61f1e8f662aa9d093bfafe8ab44ac5d48ac59a5f8a06613bc37e62a1f287c9ef"} err="failed to get container status \"61f1e8f662aa9d093bfafe8ab44ac5d48ac59a5f8a06613bc37e62a1f287c9ef\": rpc error: code = NotFound desc = could not find container \"61f1e8f662aa9d093bfafe8ab44ac5d48ac59a5f8a06613bc37e62a1f287c9ef\": container with ID starting with 61f1e8f662aa9d093bfafe8ab44ac5d48ac59a5f8a06613bc37e62a1f287c9ef not found: ID does not exist" Dec 03 13:18:14 crc kubenswrapper[5052]: I1203 13:18:14.791200 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="barbican-kuttl-tests/barbican-db-sync-2dm58" Dec 03 13:18:14 crc kubenswrapper[5052]: I1203 13:18:14.873375 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vlgtn\" (UniqueName: \"kubernetes.io/projected/62780bef-5fe0-4e1a-b6a1-62e6d5566cc0-kube-api-access-vlgtn\") pod \"62780bef-5fe0-4e1a-b6a1-62e6d5566cc0\" (UID: \"62780bef-5fe0-4e1a-b6a1-62e6d5566cc0\") " Dec 03 13:18:14 crc kubenswrapper[5052]: I1203 13:18:14.873546 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/62780bef-5fe0-4e1a-b6a1-62e6d5566cc0-db-sync-config-data\") pod \"62780bef-5fe0-4e1a-b6a1-62e6d5566cc0\" (UID: \"62780bef-5fe0-4e1a-b6a1-62e6d5566cc0\") " Dec 03 13:18:14 crc kubenswrapper[5052]: I1203 13:18:14.873588 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/62780bef-5fe0-4e1a-b6a1-62e6d5566cc0-combined-ca-bundle\") pod \"62780bef-5fe0-4e1a-b6a1-62e6d5566cc0\" (UID: \"62780bef-5fe0-4e1a-b6a1-62e6d5566cc0\") " Dec 03 13:18:14 crc kubenswrapper[5052]: I1203 13:18:14.878111 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/62780bef-5fe0-4e1a-b6a1-62e6d5566cc0-db-sync-config-data" (OuterVolumeSpecName: "db-sync-config-data") pod "62780bef-5fe0-4e1a-b6a1-62e6d5566cc0" (UID: "62780bef-5fe0-4e1a-b6a1-62e6d5566cc0"). InnerVolumeSpecName "db-sync-config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:18:14 crc kubenswrapper[5052]: I1203 13:18:14.879709 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/62780bef-5fe0-4e1a-b6a1-62e6d5566cc0-kube-api-access-vlgtn" (OuterVolumeSpecName: "kube-api-access-vlgtn") pod "62780bef-5fe0-4e1a-b6a1-62e6d5566cc0" (UID: "62780bef-5fe0-4e1a-b6a1-62e6d5566cc0"). InnerVolumeSpecName "kube-api-access-vlgtn". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:18:14 crc kubenswrapper[5052]: I1203 13:18:14.896235 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/62780bef-5fe0-4e1a-b6a1-62e6d5566cc0-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "62780bef-5fe0-4e1a-b6a1-62e6d5566cc0" (UID: "62780bef-5fe0-4e1a-b6a1-62e6d5566cc0"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:18:14 crc kubenswrapper[5052]: I1203 13:18:14.975619 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-vlgtn\" (UniqueName: \"kubernetes.io/projected/62780bef-5fe0-4e1a-b6a1-62e6d5566cc0-kube-api-access-vlgtn\") on node \"crc\" DevicePath \"\"" Dec 03 13:18:14 crc kubenswrapper[5052]: I1203 13:18:14.975667 5052 reconciler_common.go:293] "Volume detached for volume \"db-sync-config-data\" (UniqueName: \"kubernetes.io/secret/62780bef-5fe0-4e1a-b6a1-62e6d5566cc0-db-sync-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 13:18:14 crc kubenswrapper[5052]: I1203 13:18:14.975677 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/62780bef-5fe0-4e1a-b6a1-62e6d5566cc0-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 13:18:15 crc kubenswrapper[5052]: I1203 13:18:15.404404 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="barbican-kuttl-tests/barbican-db-sync-2dm58" event={"ID":"62780bef-5fe0-4e1a-b6a1-62e6d5566cc0","Type":"ContainerDied","Data":"34e1f47a0df17822b3d50bfa4bb47522b2b9ff49353626dd3e7a17d90bcc83b9"} Dec 03 13:18:15 crc kubenswrapper[5052]: I1203 13:18:15.404455 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="34e1f47a0df17822b3d50bfa4bb47522b2b9ff49353626dd3e7a17d90bcc83b9" Dec 03 13:18:15 crc kubenswrapper[5052]: I1203 13:18:15.404457 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="barbican-kuttl-tests/barbican-db-sync-2dm58" Dec 03 13:18:15 crc kubenswrapper[5052]: I1203 13:18:15.605066 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["barbican-kuttl-tests/barbican-worker-79c7d8f77-zbb42"] Dec 03 13:18:15 crc kubenswrapper[5052]: E1203 13:18:15.605384 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="411fcfda-357c-4967-b59f-54cb173c3ce5" containerName="registry-server" Dec 03 13:18:15 crc kubenswrapper[5052]: I1203 13:18:15.605397 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="411fcfda-357c-4967-b59f-54cb173c3ce5" containerName="registry-server" Dec 03 13:18:15 crc kubenswrapper[5052]: E1203 13:18:15.605413 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="411fcfda-357c-4967-b59f-54cb173c3ce5" containerName="extract-content" Dec 03 13:18:15 crc kubenswrapper[5052]: I1203 13:18:15.605419 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="411fcfda-357c-4967-b59f-54cb173c3ce5" containerName="extract-content" Dec 03 13:18:15 crc kubenswrapper[5052]: E1203 13:18:15.605434 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="411fcfda-357c-4967-b59f-54cb173c3ce5" containerName="extract-utilities" Dec 03 13:18:15 crc kubenswrapper[5052]: I1203 13:18:15.605440 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="411fcfda-357c-4967-b59f-54cb173c3ce5" containerName="extract-utilities" Dec 03 13:18:15 crc kubenswrapper[5052]: E1203 13:18:15.605452 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="62780bef-5fe0-4e1a-b6a1-62e6d5566cc0" containerName="barbican-db-sync" Dec 03 13:18:15 crc kubenswrapper[5052]: I1203 13:18:15.605458 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="62780bef-5fe0-4e1a-b6a1-62e6d5566cc0" containerName="barbican-db-sync" Dec 03 13:18:15 crc kubenswrapper[5052]: I1203 13:18:15.605616 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="411fcfda-357c-4967-b59f-54cb173c3ce5" containerName="registry-server" Dec 03 13:18:15 crc kubenswrapper[5052]: I1203 13:18:15.605631 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="62780bef-5fe0-4e1a-b6a1-62e6d5566cc0" containerName="barbican-db-sync" Dec 03 13:18:15 crc kubenswrapper[5052]: I1203 13:18:15.606571 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="barbican-kuttl-tests/barbican-worker-79c7d8f77-zbb42" Dec 03 13:18:15 crc kubenswrapper[5052]: I1203 13:18:15.609838 5052 reflector.go:368] Caches populated for *v1.Secret from object-"barbican-kuttl-tests"/"barbican-barbican-dockercfg-mrz7d" Dec 03 13:18:15 crc kubenswrapper[5052]: I1203 13:18:15.610142 5052 reflector.go:368] Caches populated for *v1.Secret from object-"barbican-kuttl-tests"/"barbican-worker-config-data" Dec 03 13:18:15 crc kubenswrapper[5052]: I1203 13:18:15.610364 5052 reflector.go:368] Caches populated for *v1.Secret from object-"barbican-kuttl-tests"/"combined-ca-bundle" Dec 03 13:18:15 crc kubenswrapper[5052]: I1203 13:18:15.626877 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["barbican-kuttl-tests/barbican-worker-79c7d8f77-zbb42"] Dec 03 13:18:15 crc kubenswrapper[5052]: I1203 13:18:15.655325 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["barbican-kuttl-tests/barbican-keystone-listener-94cbbc8d6-wxwcq"] Dec 03 13:18:15 crc kubenswrapper[5052]: I1203 13:18:15.657060 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="barbican-kuttl-tests/barbican-keystone-listener-94cbbc8d6-wxwcq" Dec 03 13:18:15 crc kubenswrapper[5052]: I1203 13:18:15.686017 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s68rf\" (UniqueName: \"kubernetes.io/projected/9dfe9635-56c2-4a66-9776-57eb16259e44-kube-api-access-s68rf\") pod \"barbican-worker-79c7d8f77-zbb42\" (UID: \"9dfe9635-56c2-4a66-9776-57eb16259e44\") " pod="barbican-kuttl-tests/barbican-worker-79c7d8f77-zbb42" Dec 03 13:18:15 crc kubenswrapper[5052]: I1203 13:18:15.686107 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9dfe9635-56c2-4a66-9776-57eb16259e44-combined-ca-bundle\") pod \"barbican-worker-79c7d8f77-zbb42\" (UID: \"9dfe9635-56c2-4a66-9776-57eb16259e44\") " pod="barbican-kuttl-tests/barbican-worker-79c7d8f77-zbb42" Dec 03 13:18:15 crc kubenswrapper[5052]: I1203 13:18:15.686387 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/9dfe9635-56c2-4a66-9776-57eb16259e44-config-data\") pod \"barbican-worker-79c7d8f77-zbb42\" (UID: \"9dfe9635-56c2-4a66-9776-57eb16259e44\") " pod="barbican-kuttl-tests/barbican-worker-79c7d8f77-zbb42" Dec 03 13:18:15 crc kubenswrapper[5052]: I1203 13:18:15.686599 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="411fcfda-357c-4967-b59f-54cb173c3ce5" path="/var/lib/kubelet/pods/411fcfda-357c-4967-b59f-54cb173c3ce5/volumes" Dec 03 13:18:15 crc kubenswrapper[5052]: I1203 13:18:15.686687 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/9dfe9635-56c2-4a66-9776-57eb16259e44-logs\") pod \"barbican-worker-79c7d8f77-zbb42\" (UID: \"9dfe9635-56c2-4a66-9776-57eb16259e44\") " pod="barbican-kuttl-tests/barbican-worker-79c7d8f77-zbb42" Dec 03 13:18:15 crc kubenswrapper[5052]: I1203 13:18:15.686720 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/9dfe9635-56c2-4a66-9776-57eb16259e44-config-data-custom\") pod \"barbican-worker-79c7d8f77-zbb42\" (UID: \"9dfe9635-56c2-4a66-9776-57eb16259e44\") " pod="barbican-kuttl-tests/barbican-worker-79c7d8f77-zbb42" Dec 03 13:18:15 crc kubenswrapper[5052]: I1203 13:18:15.687693 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["barbican-kuttl-tests/barbican-keystone-listener-94cbbc8d6-wxwcq"] Dec 03 13:18:15 crc kubenswrapper[5052]: I1203 13:18:15.755720 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["barbican-kuttl-tests/barbican-api-6d67464bc8-bqnm6"] Dec 03 13:18:15 crc kubenswrapper[5052]: I1203 13:18:15.757186 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="barbican-kuttl-tests/barbican-api-6d67464bc8-bqnm6" Dec 03 13:18:15 crc kubenswrapper[5052]: I1203 13:18:15.799205 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/9dfe9635-56c2-4a66-9776-57eb16259e44-logs\") pod \"barbican-worker-79c7d8f77-zbb42\" (UID: \"9dfe9635-56c2-4a66-9776-57eb16259e44\") " pod="barbican-kuttl-tests/barbican-worker-79c7d8f77-zbb42" Dec 03 13:18:15 crc kubenswrapper[5052]: I1203 13:18:15.799245 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/9dfe9635-56c2-4a66-9776-57eb16259e44-config-data-custom\") pod \"barbican-worker-79c7d8f77-zbb42\" (UID: \"9dfe9635-56c2-4a66-9776-57eb16259e44\") " pod="barbican-kuttl-tests/barbican-worker-79c7d8f77-zbb42" Dec 03 13:18:15 crc kubenswrapper[5052]: I1203 13:18:15.799311 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ql7ql\" (UniqueName: \"kubernetes.io/projected/2b90d104-bcfc-4c8c-b5a0-16f68747ec01-kube-api-access-ql7ql\") pod \"barbican-keystone-listener-94cbbc8d6-wxwcq\" (UID: \"2b90d104-bcfc-4c8c-b5a0-16f68747ec01\") " pod="barbican-kuttl-tests/barbican-keystone-listener-94cbbc8d6-wxwcq" Dec 03 13:18:15 crc kubenswrapper[5052]: I1203 13:18:15.799336 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-s68rf\" (UniqueName: \"kubernetes.io/projected/9dfe9635-56c2-4a66-9776-57eb16259e44-kube-api-access-s68rf\") pod \"barbican-worker-79c7d8f77-zbb42\" (UID: \"9dfe9635-56c2-4a66-9776-57eb16259e44\") " pod="barbican-kuttl-tests/barbican-worker-79c7d8f77-zbb42" Dec 03 13:18:15 crc kubenswrapper[5052]: I1203 13:18:15.799362 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/2b90d104-bcfc-4c8c-b5a0-16f68747ec01-config-data-custom\") pod \"barbican-keystone-listener-94cbbc8d6-wxwcq\" (UID: \"2b90d104-bcfc-4c8c-b5a0-16f68747ec01\") " pod="barbican-kuttl-tests/barbican-keystone-listener-94cbbc8d6-wxwcq" Dec 03 13:18:15 crc kubenswrapper[5052]: I1203 13:18:15.799401 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/2b90d104-bcfc-4c8c-b5a0-16f68747ec01-config-data\") pod \"barbican-keystone-listener-94cbbc8d6-wxwcq\" (UID: \"2b90d104-bcfc-4c8c-b5a0-16f68747ec01\") " pod="barbican-kuttl-tests/barbican-keystone-listener-94cbbc8d6-wxwcq" Dec 03 13:18:15 crc kubenswrapper[5052]: I1203 13:18:15.799429 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9dfe9635-56c2-4a66-9776-57eb16259e44-combined-ca-bundle\") pod \"barbican-worker-79c7d8f77-zbb42\" (UID: \"9dfe9635-56c2-4a66-9776-57eb16259e44\") " pod="barbican-kuttl-tests/barbican-worker-79c7d8f77-zbb42" Dec 03 13:18:15 crc kubenswrapper[5052]: I1203 13:18:15.799455 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/2b90d104-bcfc-4c8c-b5a0-16f68747ec01-logs\") pod \"barbican-keystone-listener-94cbbc8d6-wxwcq\" (UID: \"2b90d104-bcfc-4c8c-b5a0-16f68747ec01\") " pod="barbican-kuttl-tests/barbican-keystone-listener-94cbbc8d6-wxwcq" Dec 03 13:18:15 crc kubenswrapper[5052]: I1203 13:18:15.799473 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2b90d104-bcfc-4c8c-b5a0-16f68747ec01-combined-ca-bundle\") pod \"barbican-keystone-listener-94cbbc8d6-wxwcq\" (UID: \"2b90d104-bcfc-4c8c-b5a0-16f68747ec01\") " pod="barbican-kuttl-tests/barbican-keystone-listener-94cbbc8d6-wxwcq" Dec 03 13:18:15 crc kubenswrapper[5052]: I1203 13:18:15.799490 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/9dfe9635-56c2-4a66-9776-57eb16259e44-config-data\") pod \"barbican-worker-79c7d8f77-zbb42\" (UID: \"9dfe9635-56c2-4a66-9776-57eb16259e44\") " pod="barbican-kuttl-tests/barbican-worker-79c7d8f77-zbb42" Dec 03 13:18:15 crc kubenswrapper[5052]: I1203 13:18:15.800199 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/9dfe9635-56c2-4a66-9776-57eb16259e44-logs\") pod \"barbican-worker-79c7d8f77-zbb42\" (UID: \"9dfe9635-56c2-4a66-9776-57eb16259e44\") " pod="barbican-kuttl-tests/barbican-worker-79c7d8f77-zbb42" Dec 03 13:18:15 crc kubenswrapper[5052]: I1203 13:18:15.804157 5052 reflector.go:368] Caches populated for *v1.Secret from object-"barbican-kuttl-tests"/"barbican-api-config-data" Dec 03 13:18:15 crc kubenswrapper[5052]: I1203 13:18:15.804378 5052 reflector.go:368] Caches populated for *v1.Secret from object-"barbican-kuttl-tests"/"cert-barbican-public-svc" Dec 03 13:18:15 crc kubenswrapper[5052]: I1203 13:18:15.804587 5052 reflector.go:368] Caches populated for *v1.Secret from object-"barbican-kuttl-tests"/"cert-barbican-internal-svc" Dec 03 13:18:15 crc kubenswrapper[5052]: I1203 13:18:15.805108 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/9dfe9635-56c2-4a66-9776-57eb16259e44-config-data-custom\") pod \"barbican-worker-79c7d8f77-zbb42\" (UID: \"9dfe9635-56c2-4a66-9776-57eb16259e44\") " pod="barbican-kuttl-tests/barbican-worker-79c7d8f77-zbb42" Dec 03 13:18:15 crc kubenswrapper[5052]: I1203 13:18:15.805803 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/9dfe9635-56c2-4a66-9776-57eb16259e44-config-data\") pod \"barbican-worker-79c7d8f77-zbb42\" (UID: \"9dfe9635-56c2-4a66-9776-57eb16259e44\") " pod="barbican-kuttl-tests/barbican-worker-79c7d8f77-zbb42" Dec 03 13:18:15 crc kubenswrapper[5052]: I1203 13:18:15.833877 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9dfe9635-56c2-4a66-9776-57eb16259e44-combined-ca-bundle\") pod \"barbican-worker-79c7d8f77-zbb42\" (UID: \"9dfe9635-56c2-4a66-9776-57eb16259e44\") " pod="barbican-kuttl-tests/barbican-worker-79c7d8f77-zbb42" Dec 03 13:18:15 crc kubenswrapper[5052]: I1203 13:18:15.838266 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-s68rf\" (UniqueName: \"kubernetes.io/projected/9dfe9635-56c2-4a66-9776-57eb16259e44-kube-api-access-s68rf\") pod \"barbican-worker-79c7d8f77-zbb42\" (UID: \"9dfe9635-56c2-4a66-9776-57eb16259e44\") " pod="barbican-kuttl-tests/barbican-worker-79c7d8f77-zbb42" Dec 03 13:18:15 crc kubenswrapper[5052]: I1203 13:18:15.841579 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["barbican-kuttl-tests/barbican-api-6d67464bc8-bqnm6"] Dec 03 13:18:15 crc kubenswrapper[5052]: I1203 13:18:15.900516 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1cd1c126-8f6c-4d19-b6ba-f34b0cbd37b1-combined-ca-bundle\") pod \"barbican-api-6d67464bc8-bqnm6\" (UID: \"1cd1c126-8f6c-4d19-b6ba-f34b0cbd37b1\") " pod="barbican-kuttl-tests/barbican-api-6d67464bc8-bqnm6" Dec 03 13:18:15 crc kubenswrapper[5052]: I1203 13:18:15.900805 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/1cd1c126-8f6c-4d19-b6ba-f34b0cbd37b1-internal-tls-certs\") pod \"barbican-api-6d67464bc8-bqnm6\" (UID: \"1cd1c126-8f6c-4d19-b6ba-f34b0cbd37b1\") " pod="barbican-kuttl-tests/barbican-api-6d67464bc8-bqnm6" Dec 03 13:18:15 crc kubenswrapper[5052]: I1203 13:18:15.900836 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ccw2r\" (UniqueName: \"kubernetes.io/projected/1cd1c126-8f6c-4d19-b6ba-f34b0cbd37b1-kube-api-access-ccw2r\") pod \"barbican-api-6d67464bc8-bqnm6\" (UID: \"1cd1c126-8f6c-4d19-b6ba-f34b0cbd37b1\") " pod="barbican-kuttl-tests/barbican-api-6d67464bc8-bqnm6" Dec 03 13:18:15 crc kubenswrapper[5052]: I1203 13:18:15.900864 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/1cd1c126-8f6c-4d19-b6ba-f34b0cbd37b1-config-data-custom\") pod \"barbican-api-6d67464bc8-bqnm6\" (UID: \"1cd1c126-8f6c-4d19-b6ba-f34b0cbd37b1\") " pod="barbican-kuttl-tests/barbican-api-6d67464bc8-bqnm6" Dec 03 13:18:15 crc kubenswrapper[5052]: I1203 13:18:15.900905 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-ql7ql\" (UniqueName: \"kubernetes.io/projected/2b90d104-bcfc-4c8c-b5a0-16f68747ec01-kube-api-access-ql7ql\") pod \"barbican-keystone-listener-94cbbc8d6-wxwcq\" (UID: \"2b90d104-bcfc-4c8c-b5a0-16f68747ec01\") " pod="barbican-kuttl-tests/barbican-keystone-listener-94cbbc8d6-wxwcq" Dec 03 13:18:15 crc kubenswrapper[5052]: I1203 13:18:15.900935 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/2b90d104-bcfc-4c8c-b5a0-16f68747ec01-config-data-custom\") pod \"barbican-keystone-listener-94cbbc8d6-wxwcq\" (UID: \"2b90d104-bcfc-4c8c-b5a0-16f68747ec01\") " pod="barbican-kuttl-tests/barbican-keystone-listener-94cbbc8d6-wxwcq" Dec 03 13:18:15 crc kubenswrapper[5052]: I1203 13:18:15.900965 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/1cd1c126-8f6c-4d19-b6ba-f34b0cbd37b1-config-data\") pod \"barbican-api-6d67464bc8-bqnm6\" (UID: \"1cd1c126-8f6c-4d19-b6ba-f34b0cbd37b1\") " pod="barbican-kuttl-tests/barbican-api-6d67464bc8-bqnm6" Dec 03 13:18:15 crc kubenswrapper[5052]: I1203 13:18:15.900995 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/1cd1c126-8f6c-4d19-b6ba-f34b0cbd37b1-public-tls-certs\") pod \"barbican-api-6d67464bc8-bqnm6\" (UID: \"1cd1c126-8f6c-4d19-b6ba-f34b0cbd37b1\") " pod="barbican-kuttl-tests/barbican-api-6d67464bc8-bqnm6" Dec 03 13:18:15 crc kubenswrapper[5052]: I1203 13:18:15.901015 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/1cd1c126-8f6c-4d19-b6ba-f34b0cbd37b1-logs\") pod \"barbican-api-6d67464bc8-bqnm6\" (UID: \"1cd1c126-8f6c-4d19-b6ba-f34b0cbd37b1\") " pod="barbican-kuttl-tests/barbican-api-6d67464bc8-bqnm6" Dec 03 13:18:15 crc kubenswrapper[5052]: I1203 13:18:15.901037 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/2b90d104-bcfc-4c8c-b5a0-16f68747ec01-config-data\") pod \"barbican-keystone-listener-94cbbc8d6-wxwcq\" (UID: \"2b90d104-bcfc-4c8c-b5a0-16f68747ec01\") " pod="barbican-kuttl-tests/barbican-keystone-listener-94cbbc8d6-wxwcq" Dec 03 13:18:15 crc kubenswrapper[5052]: I1203 13:18:15.901064 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/2b90d104-bcfc-4c8c-b5a0-16f68747ec01-logs\") pod \"barbican-keystone-listener-94cbbc8d6-wxwcq\" (UID: \"2b90d104-bcfc-4c8c-b5a0-16f68747ec01\") " pod="barbican-kuttl-tests/barbican-keystone-listener-94cbbc8d6-wxwcq" Dec 03 13:18:15 crc kubenswrapper[5052]: I1203 13:18:15.901086 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2b90d104-bcfc-4c8c-b5a0-16f68747ec01-combined-ca-bundle\") pod \"barbican-keystone-listener-94cbbc8d6-wxwcq\" (UID: \"2b90d104-bcfc-4c8c-b5a0-16f68747ec01\") " pod="barbican-kuttl-tests/barbican-keystone-listener-94cbbc8d6-wxwcq" Dec 03 13:18:15 crc kubenswrapper[5052]: I1203 13:18:15.901709 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/2b90d104-bcfc-4c8c-b5a0-16f68747ec01-logs\") pod \"barbican-keystone-listener-94cbbc8d6-wxwcq\" (UID: \"2b90d104-bcfc-4c8c-b5a0-16f68747ec01\") " pod="barbican-kuttl-tests/barbican-keystone-listener-94cbbc8d6-wxwcq" Dec 03 13:18:15 crc kubenswrapper[5052]: I1203 13:18:15.905631 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/2b90d104-bcfc-4c8c-b5a0-16f68747ec01-config-data\") pod \"barbican-keystone-listener-94cbbc8d6-wxwcq\" (UID: \"2b90d104-bcfc-4c8c-b5a0-16f68747ec01\") " pod="barbican-kuttl-tests/barbican-keystone-listener-94cbbc8d6-wxwcq" Dec 03 13:18:15 crc kubenswrapper[5052]: I1203 13:18:15.906010 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2b90d104-bcfc-4c8c-b5a0-16f68747ec01-combined-ca-bundle\") pod \"barbican-keystone-listener-94cbbc8d6-wxwcq\" (UID: \"2b90d104-bcfc-4c8c-b5a0-16f68747ec01\") " pod="barbican-kuttl-tests/barbican-keystone-listener-94cbbc8d6-wxwcq" Dec 03 13:18:15 crc kubenswrapper[5052]: I1203 13:18:15.906600 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/2b90d104-bcfc-4c8c-b5a0-16f68747ec01-config-data-custom\") pod \"barbican-keystone-listener-94cbbc8d6-wxwcq\" (UID: \"2b90d104-bcfc-4c8c-b5a0-16f68747ec01\") " pod="barbican-kuttl-tests/barbican-keystone-listener-94cbbc8d6-wxwcq" Dec 03 13:18:15 crc kubenswrapper[5052]: I1203 13:18:15.916393 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-ql7ql\" (UniqueName: \"kubernetes.io/projected/2b90d104-bcfc-4c8c-b5a0-16f68747ec01-kube-api-access-ql7ql\") pod \"barbican-keystone-listener-94cbbc8d6-wxwcq\" (UID: \"2b90d104-bcfc-4c8c-b5a0-16f68747ec01\") " pod="barbican-kuttl-tests/barbican-keystone-listener-94cbbc8d6-wxwcq" Dec 03 13:18:15 crc kubenswrapper[5052]: I1203 13:18:15.922571 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="barbican-kuttl-tests/barbican-worker-79c7d8f77-zbb42" Dec 03 13:18:15 crc kubenswrapper[5052]: I1203 13:18:15.974909 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="barbican-kuttl-tests/barbican-keystone-listener-94cbbc8d6-wxwcq" Dec 03 13:18:16 crc kubenswrapper[5052]: I1203 13:18:16.002100 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/1cd1c126-8f6c-4d19-b6ba-f34b0cbd37b1-config-data\") pod \"barbican-api-6d67464bc8-bqnm6\" (UID: \"1cd1c126-8f6c-4d19-b6ba-f34b0cbd37b1\") " pod="barbican-kuttl-tests/barbican-api-6d67464bc8-bqnm6" Dec 03 13:18:16 crc kubenswrapper[5052]: I1203 13:18:16.002161 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/1cd1c126-8f6c-4d19-b6ba-f34b0cbd37b1-public-tls-certs\") pod \"barbican-api-6d67464bc8-bqnm6\" (UID: \"1cd1c126-8f6c-4d19-b6ba-f34b0cbd37b1\") " pod="barbican-kuttl-tests/barbican-api-6d67464bc8-bqnm6" Dec 03 13:18:16 crc kubenswrapper[5052]: I1203 13:18:16.002185 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/1cd1c126-8f6c-4d19-b6ba-f34b0cbd37b1-logs\") pod \"barbican-api-6d67464bc8-bqnm6\" (UID: \"1cd1c126-8f6c-4d19-b6ba-f34b0cbd37b1\") " pod="barbican-kuttl-tests/barbican-api-6d67464bc8-bqnm6" Dec 03 13:18:16 crc kubenswrapper[5052]: I1203 13:18:16.002285 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1cd1c126-8f6c-4d19-b6ba-f34b0cbd37b1-combined-ca-bundle\") pod \"barbican-api-6d67464bc8-bqnm6\" (UID: \"1cd1c126-8f6c-4d19-b6ba-f34b0cbd37b1\") " pod="barbican-kuttl-tests/barbican-api-6d67464bc8-bqnm6" Dec 03 13:18:16 crc kubenswrapper[5052]: I1203 13:18:16.002344 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/1cd1c126-8f6c-4d19-b6ba-f34b0cbd37b1-internal-tls-certs\") pod \"barbican-api-6d67464bc8-bqnm6\" (UID: \"1cd1c126-8f6c-4d19-b6ba-f34b0cbd37b1\") " pod="barbican-kuttl-tests/barbican-api-6d67464bc8-bqnm6" Dec 03 13:18:16 crc kubenswrapper[5052]: I1203 13:18:16.002375 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-ccw2r\" (UniqueName: \"kubernetes.io/projected/1cd1c126-8f6c-4d19-b6ba-f34b0cbd37b1-kube-api-access-ccw2r\") pod \"barbican-api-6d67464bc8-bqnm6\" (UID: \"1cd1c126-8f6c-4d19-b6ba-f34b0cbd37b1\") " pod="barbican-kuttl-tests/barbican-api-6d67464bc8-bqnm6" Dec 03 13:18:16 crc kubenswrapper[5052]: I1203 13:18:16.002405 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/1cd1c126-8f6c-4d19-b6ba-f34b0cbd37b1-config-data-custom\") pod \"barbican-api-6d67464bc8-bqnm6\" (UID: \"1cd1c126-8f6c-4d19-b6ba-f34b0cbd37b1\") " pod="barbican-kuttl-tests/barbican-api-6d67464bc8-bqnm6" Dec 03 13:18:16 crc kubenswrapper[5052]: I1203 13:18:16.006209 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/1cd1c126-8f6c-4d19-b6ba-f34b0cbd37b1-config-data-custom\") pod \"barbican-api-6d67464bc8-bqnm6\" (UID: \"1cd1c126-8f6c-4d19-b6ba-f34b0cbd37b1\") " pod="barbican-kuttl-tests/barbican-api-6d67464bc8-bqnm6" Dec 03 13:18:16 crc kubenswrapper[5052]: I1203 13:18:16.006238 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/1cd1c126-8f6c-4d19-b6ba-f34b0cbd37b1-logs\") pod \"barbican-api-6d67464bc8-bqnm6\" (UID: \"1cd1c126-8f6c-4d19-b6ba-f34b0cbd37b1\") " pod="barbican-kuttl-tests/barbican-api-6d67464bc8-bqnm6" Dec 03 13:18:16 crc kubenswrapper[5052]: I1203 13:18:16.009259 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1cd1c126-8f6c-4d19-b6ba-f34b0cbd37b1-combined-ca-bundle\") pod \"barbican-api-6d67464bc8-bqnm6\" (UID: \"1cd1c126-8f6c-4d19-b6ba-f34b0cbd37b1\") " pod="barbican-kuttl-tests/barbican-api-6d67464bc8-bqnm6" Dec 03 13:18:16 crc kubenswrapper[5052]: I1203 13:18:16.009505 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/1cd1c126-8f6c-4d19-b6ba-f34b0cbd37b1-public-tls-certs\") pod \"barbican-api-6d67464bc8-bqnm6\" (UID: \"1cd1c126-8f6c-4d19-b6ba-f34b0cbd37b1\") " pod="barbican-kuttl-tests/barbican-api-6d67464bc8-bqnm6" Dec 03 13:18:16 crc kubenswrapper[5052]: I1203 13:18:16.012017 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/1cd1c126-8f6c-4d19-b6ba-f34b0cbd37b1-internal-tls-certs\") pod \"barbican-api-6d67464bc8-bqnm6\" (UID: \"1cd1c126-8f6c-4d19-b6ba-f34b0cbd37b1\") " pod="barbican-kuttl-tests/barbican-api-6d67464bc8-bqnm6" Dec 03 13:18:16 crc kubenswrapper[5052]: I1203 13:18:16.017548 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/1cd1c126-8f6c-4d19-b6ba-f34b0cbd37b1-config-data\") pod \"barbican-api-6d67464bc8-bqnm6\" (UID: \"1cd1c126-8f6c-4d19-b6ba-f34b0cbd37b1\") " pod="barbican-kuttl-tests/barbican-api-6d67464bc8-bqnm6" Dec 03 13:18:16 crc kubenswrapper[5052]: I1203 13:18:16.035332 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-ccw2r\" (UniqueName: \"kubernetes.io/projected/1cd1c126-8f6c-4d19-b6ba-f34b0cbd37b1-kube-api-access-ccw2r\") pod \"barbican-api-6d67464bc8-bqnm6\" (UID: \"1cd1c126-8f6c-4d19-b6ba-f34b0cbd37b1\") " pod="barbican-kuttl-tests/barbican-api-6d67464bc8-bqnm6" Dec 03 13:18:16 crc kubenswrapper[5052]: I1203 13:18:16.199197 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="barbican-kuttl-tests/barbican-api-6d67464bc8-bqnm6" Dec 03 13:18:16 crc kubenswrapper[5052]: I1203 13:18:16.403135 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["barbican-kuttl-tests/barbican-worker-79c7d8f77-zbb42"] Dec 03 13:18:16 crc kubenswrapper[5052]: I1203 13:18:16.471632 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["barbican-kuttl-tests/barbican-api-6d67464bc8-bqnm6"] Dec 03 13:18:16 crc kubenswrapper[5052]: I1203 13:18:16.484833 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["barbican-kuttl-tests/barbican-keystone-listener-94cbbc8d6-wxwcq"] Dec 03 13:18:16 crc kubenswrapper[5052]: W1203 13:18:16.491612 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2b90d104_bcfc_4c8c_b5a0_16f68747ec01.slice/crio-057f0d58ca500095e475548b41e1a5d86d4549ee50d17af39efaaca94e9ab335 WatchSource:0}: Error finding container 057f0d58ca500095e475548b41e1a5d86d4549ee50d17af39efaaca94e9ab335: Status 404 returned error can't find the container with id 057f0d58ca500095e475548b41e1a5d86d4549ee50d17af39efaaca94e9ab335 Dec 03 13:18:16 crc kubenswrapper[5052]: I1203 13:18:16.508907 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["barbican-kuttl-tests/barbican-db-sync-2dm58"] Dec 03 13:18:16 crc kubenswrapper[5052]: I1203 13:18:16.530321 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["barbican-kuttl-tests/barbican-db-sync-2dm58"] Dec 03 13:18:16 crc kubenswrapper[5052]: I1203 13:18:16.556699 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["barbican-kuttl-tests/barbicana311-account-delete-lggsm"] Dec 03 13:18:16 crc kubenswrapper[5052]: I1203 13:18:16.558590 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="barbican-kuttl-tests/barbicana311-account-delete-lggsm" Dec 03 13:18:16 crc kubenswrapper[5052]: I1203 13:18:16.577288 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["barbican-kuttl-tests/barbicana311-account-delete-lggsm"] Dec 03 13:18:16 crc kubenswrapper[5052]: I1203 13:18:16.593759 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["barbican-kuttl-tests/barbican-keystone-listener-94cbbc8d6-wxwcq"] Dec 03 13:18:16 crc kubenswrapper[5052]: I1203 13:18:16.600946 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["barbican-kuttl-tests/barbican-worker-79c7d8f77-zbb42"] Dec 03 13:18:16 crc kubenswrapper[5052]: I1203 13:18:16.607989 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["barbican-kuttl-tests/barbican-api-6d67464bc8-bqnm6"] Dec 03 13:18:16 crc kubenswrapper[5052]: I1203 13:18:16.715351 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/406d0ab4-b286-41c6-b080-5b3e41f6c3eb-operator-scripts\") pod \"barbicana311-account-delete-lggsm\" (UID: \"406d0ab4-b286-41c6-b080-5b3e41f6c3eb\") " pod="barbican-kuttl-tests/barbicana311-account-delete-lggsm" Dec 03 13:18:16 crc kubenswrapper[5052]: I1203 13:18:16.715476 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zpdw4\" (UniqueName: \"kubernetes.io/projected/406d0ab4-b286-41c6-b080-5b3e41f6c3eb-kube-api-access-zpdw4\") pod \"barbicana311-account-delete-lggsm\" (UID: \"406d0ab4-b286-41c6-b080-5b3e41f6c3eb\") " pod="barbican-kuttl-tests/barbicana311-account-delete-lggsm" Dec 03 13:18:16 crc kubenswrapper[5052]: I1203 13:18:16.817470 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-zpdw4\" (UniqueName: \"kubernetes.io/projected/406d0ab4-b286-41c6-b080-5b3e41f6c3eb-kube-api-access-zpdw4\") pod \"barbicana311-account-delete-lggsm\" (UID: \"406d0ab4-b286-41c6-b080-5b3e41f6c3eb\") " pod="barbican-kuttl-tests/barbicana311-account-delete-lggsm" Dec 03 13:18:16 crc kubenswrapper[5052]: I1203 13:18:16.817637 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/406d0ab4-b286-41c6-b080-5b3e41f6c3eb-operator-scripts\") pod \"barbicana311-account-delete-lggsm\" (UID: \"406d0ab4-b286-41c6-b080-5b3e41f6c3eb\") " pod="barbican-kuttl-tests/barbicana311-account-delete-lggsm" Dec 03 13:18:16 crc kubenswrapper[5052]: I1203 13:18:16.818545 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/406d0ab4-b286-41c6-b080-5b3e41f6c3eb-operator-scripts\") pod \"barbicana311-account-delete-lggsm\" (UID: \"406d0ab4-b286-41c6-b080-5b3e41f6c3eb\") " pod="barbican-kuttl-tests/barbicana311-account-delete-lggsm" Dec 03 13:18:16 crc kubenswrapper[5052]: I1203 13:18:16.834996 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-zpdw4\" (UniqueName: \"kubernetes.io/projected/406d0ab4-b286-41c6-b080-5b3e41f6c3eb-kube-api-access-zpdw4\") pod \"barbicana311-account-delete-lggsm\" (UID: \"406d0ab4-b286-41c6-b080-5b3e41f6c3eb\") " pod="barbican-kuttl-tests/barbicana311-account-delete-lggsm" Dec 03 13:18:16 crc kubenswrapper[5052]: I1203 13:18:16.892241 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="barbican-kuttl-tests/barbicana311-account-delete-lggsm" Dec 03 13:18:17 crc kubenswrapper[5052]: I1203 13:18:17.367759 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["barbican-kuttl-tests/barbicana311-account-delete-lggsm"] Dec 03 13:18:17 crc kubenswrapper[5052]: W1203 13:18:17.370430 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod406d0ab4_b286_41c6_b080_5b3e41f6c3eb.slice/crio-7cdb9851e7c850140f4b04daaa0ee00241ed22c465e2064c38e2f68f49b7f432 WatchSource:0}: Error finding container 7cdb9851e7c850140f4b04daaa0ee00241ed22c465e2064c38e2f68f49b7f432: Status 404 returned error can't find the container with id 7cdb9851e7c850140f4b04daaa0ee00241ed22c465e2064c38e2f68f49b7f432 Dec 03 13:18:17 crc kubenswrapper[5052]: I1203 13:18:17.435855 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="barbican-kuttl-tests/barbican-api-6d67464bc8-bqnm6" event={"ID":"1cd1c126-8f6c-4d19-b6ba-f34b0cbd37b1","Type":"ContainerStarted","Data":"9ffeca7b60e2b08a162a2bd0fd67835cd41e4e6861bbce88300d98fbc5680138"} Dec 03 13:18:17 crc kubenswrapper[5052]: I1203 13:18:17.435909 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="barbican-kuttl-tests/barbican-api-6d67464bc8-bqnm6" event={"ID":"1cd1c126-8f6c-4d19-b6ba-f34b0cbd37b1","Type":"ContainerStarted","Data":"5bf996ccb4dcc6fb7b692b007ba9c41573ab80bb3f058993d340190772e4c9c2"} Dec 03 13:18:17 crc kubenswrapper[5052]: I1203 13:18:17.435923 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="barbican-kuttl-tests/barbican-api-6d67464bc8-bqnm6" event={"ID":"1cd1c126-8f6c-4d19-b6ba-f34b0cbd37b1","Type":"ContainerStarted","Data":"1bdb45b3dff5b148b5815192bbbd8a2a5102db2c56239d44354dd3424cc445e3"} Dec 03 13:18:17 crc kubenswrapper[5052]: I1203 13:18:17.435992 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="barbican-kuttl-tests/barbican-api-6d67464bc8-bqnm6" Dec 03 13:18:17 crc kubenswrapper[5052]: I1203 13:18:17.435994 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="barbican-kuttl-tests/barbican-api-6d67464bc8-bqnm6" podUID="1cd1c126-8f6c-4d19-b6ba-f34b0cbd37b1" containerName="barbican-api-log" containerID="cri-o://5bf996ccb4dcc6fb7b692b007ba9c41573ab80bb3f058993d340190772e4c9c2" gracePeriod=30 Dec 03 13:18:17 crc kubenswrapper[5052]: I1203 13:18:17.436026 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="barbican-kuttl-tests/barbican-api-6d67464bc8-bqnm6" Dec 03 13:18:17 crc kubenswrapper[5052]: I1203 13:18:17.436074 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="barbican-kuttl-tests/barbican-api-6d67464bc8-bqnm6" podUID="1cd1c126-8f6c-4d19-b6ba-f34b0cbd37b1" containerName="barbican-api" containerID="cri-o://9ffeca7b60e2b08a162a2bd0fd67835cd41e4e6861bbce88300d98fbc5680138" gracePeriod=30 Dec 03 13:18:17 crc kubenswrapper[5052]: I1203 13:18:17.437595 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="barbican-kuttl-tests/barbicana311-account-delete-lggsm" event={"ID":"406d0ab4-b286-41c6-b080-5b3e41f6c3eb","Type":"ContainerStarted","Data":"7cdb9851e7c850140f4b04daaa0ee00241ed22c465e2064c38e2f68f49b7f432"} Dec 03 13:18:17 crc kubenswrapper[5052]: I1203 13:18:17.439173 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="barbican-kuttl-tests/barbican-keystone-listener-94cbbc8d6-wxwcq" event={"ID":"2b90d104-bcfc-4c8c-b5a0-16f68747ec01","Type":"ContainerStarted","Data":"459a240a4da7270c5ba3e7487932749b68c6ccf78c76547f1fdb9f2d15d3d309"} Dec 03 13:18:17 crc kubenswrapper[5052]: I1203 13:18:17.439201 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="barbican-kuttl-tests/barbican-keystone-listener-94cbbc8d6-wxwcq" event={"ID":"2b90d104-bcfc-4c8c-b5a0-16f68747ec01","Type":"ContainerStarted","Data":"b81c650b76790d5d29bbe5d40e759d35d38cbade3ae8dadaab978f7bce1037ef"} Dec 03 13:18:17 crc kubenswrapper[5052]: I1203 13:18:17.439217 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="barbican-kuttl-tests/barbican-keystone-listener-94cbbc8d6-wxwcq" event={"ID":"2b90d104-bcfc-4c8c-b5a0-16f68747ec01","Type":"ContainerStarted","Data":"057f0d58ca500095e475548b41e1a5d86d4549ee50d17af39efaaca94e9ab335"} Dec 03 13:18:17 crc kubenswrapper[5052]: I1203 13:18:17.439324 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="barbican-kuttl-tests/barbican-keystone-listener-94cbbc8d6-wxwcq" podUID="2b90d104-bcfc-4c8c-b5a0-16f68747ec01" containerName="barbican-keystone-listener-log" containerID="cri-o://b81c650b76790d5d29bbe5d40e759d35d38cbade3ae8dadaab978f7bce1037ef" gracePeriod=30 Dec 03 13:18:17 crc kubenswrapper[5052]: I1203 13:18:17.439438 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="barbican-kuttl-tests/barbican-keystone-listener-94cbbc8d6-wxwcq" podUID="2b90d104-bcfc-4c8c-b5a0-16f68747ec01" containerName="barbican-keystone-listener" containerID="cri-o://459a240a4da7270c5ba3e7487932749b68c6ccf78c76547f1fdb9f2d15d3d309" gracePeriod=30 Dec 03 13:18:17 crc kubenswrapper[5052]: I1203 13:18:17.445059 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="barbican-kuttl-tests/barbican-worker-79c7d8f77-zbb42" event={"ID":"9dfe9635-56c2-4a66-9776-57eb16259e44","Type":"ContainerStarted","Data":"1a67c592af264993789211f0bcac258e7194d8c6ffd07517dfc11274d84ccaaa"} Dec 03 13:18:17 crc kubenswrapper[5052]: I1203 13:18:17.445102 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="barbican-kuttl-tests/barbican-worker-79c7d8f77-zbb42" event={"ID":"9dfe9635-56c2-4a66-9776-57eb16259e44","Type":"ContainerStarted","Data":"27059cd5c285465956cfe0704daf188e0d826a352ab19da31389942f31ceb8da"} Dec 03 13:18:17 crc kubenswrapper[5052]: I1203 13:18:17.445113 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="barbican-kuttl-tests/barbican-worker-79c7d8f77-zbb42" event={"ID":"9dfe9635-56c2-4a66-9776-57eb16259e44","Type":"ContainerStarted","Data":"093df003b028fa9b0e731cfad1a5916ed8a6e59489550858dd0bc4aa611b8c72"} Dec 03 13:18:17 crc kubenswrapper[5052]: I1203 13:18:17.445182 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="barbican-kuttl-tests/barbican-worker-79c7d8f77-zbb42" podUID="9dfe9635-56c2-4a66-9776-57eb16259e44" containerName="barbican-worker-log" containerID="cri-o://27059cd5c285465956cfe0704daf188e0d826a352ab19da31389942f31ceb8da" gracePeriod=30 Dec 03 13:18:17 crc kubenswrapper[5052]: I1203 13:18:17.445280 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="barbican-kuttl-tests/barbican-worker-79c7d8f77-zbb42" podUID="9dfe9635-56c2-4a66-9776-57eb16259e44" containerName="barbican-worker" containerID="cri-o://1a67c592af264993789211f0bcac258e7194d8c6ffd07517dfc11274d84ccaaa" gracePeriod=30 Dec 03 13:18:17 crc kubenswrapper[5052]: I1203 13:18:17.461202 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="barbican-kuttl-tests/barbican-api-6d67464bc8-bqnm6" podStartSLOduration=2.461184097 podStartE2EDuration="2.461184097s" podCreationTimestamp="2025-12-03 13:18:15 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 13:18:17.45710933 +0000 UTC m=+7088.045351094" watchObservedRunningTime="2025-12-03 13:18:17.461184097 +0000 UTC m=+7088.049425861" Dec 03 13:18:17 crc kubenswrapper[5052]: I1203 13:18:17.483343 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="barbican-kuttl-tests/barbican-keystone-listener-94cbbc8d6-wxwcq" podStartSLOduration=2.48332093 podStartE2EDuration="2.48332093s" podCreationTimestamp="2025-12-03 13:18:15 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 13:18:17.479274635 +0000 UTC m=+7088.067516399" watchObservedRunningTime="2025-12-03 13:18:17.48332093 +0000 UTC m=+7088.071562694" Dec 03 13:18:17 crc kubenswrapper[5052]: I1203 13:18:17.496210 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="barbican-kuttl-tests/barbican-worker-79c7d8f77-zbb42" podStartSLOduration=2.496193889 podStartE2EDuration="2.496193889s" podCreationTimestamp="2025-12-03 13:18:15 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 13:18:17.494063208 +0000 UTC m=+7088.082304972" watchObservedRunningTime="2025-12-03 13:18:17.496193889 +0000 UTC m=+7088.084435653" Dec 03 13:18:17 crc kubenswrapper[5052]: I1203 13:18:17.682099 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="62780bef-5fe0-4e1a-b6a1-62e6d5566cc0" path="/var/lib/kubelet/pods/62780bef-5fe0-4e1a-b6a1-62e6d5566cc0/volumes" Dec 03 13:18:17 crc kubenswrapper[5052]: I1203 13:18:17.954183 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="barbican-kuttl-tests/barbican-api-6d67464bc8-bqnm6" Dec 03 13:18:18 crc kubenswrapper[5052]: I1203 13:18:18.034971 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/1cd1c126-8f6c-4d19-b6ba-f34b0cbd37b1-config-data-custom\") pod \"1cd1c126-8f6c-4d19-b6ba-f34b0cbd37b1\" (UID: \"1cd1c126-8f6c-4d19-b6ba-f34b0cbd37b1\") " Dec 03 13:18:18 crc kubenswrapper[5052]: I1203 13:18:18.035092 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/1cd1c126-8f6c-4d19-b6ba-f34b0cbd37b1-internal-tls-certs\") pod \"1cd1c126-8f6c-4d19-b6ba-f34b0cbd37b1\" (UID: \"1cd1c126-8f6c-4d19-b6ba-f34b0cbd37b1\") " Dec 03 13:18:18 crc kubenswrapper[5052]: I1203 13:18:18.035125 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1cd1c126-8f6c-4d19-b6ba-f34b0cbd37b1-combined-ca-bundle\") pod \"1cd1c126-8f6c-4d19-b6ba-f34b0cbd37b1\" (UID: \"1cd1c126-8f6c-4d19-b6ba-f34b0cbd37b1\") " Dec 03 13:18:18 crc kubenswrapper[5052]: I1203 13:18:18.035166 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/1cd1c126-8f6c-4d19-b6ba-f34b0cbd37b1-public-tls-certs\") pod \"1cd1c126-8f6c-4d19-b6ba-f34b0cbd37b1\" (UID: \"1cd1c126-8f6c-4d19-b6ba-f34b0cbd37b1\") " Dec 03 13:18:18 crc kubenswrapper[5052]: I1203 13:18:18.035212 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ccw2r\" (UniqueName: \"kubernetes.io/projected/1cd1c126-8f6c-4d19-b6ba-f34b0cbd37b1-kube-api-access-ccw2r\") pod \"1cd1c126-8f6c-4d19-b6ba-f34b0cbd37b1\" (UID: \"1cd1c126-8f6c-4d19-b6ba-f34b0cbd37b1\") " Dec 03 13:18:18 crc kubenswrapper[5052]: I1203 13:18:18.035271 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/1cd1c126-8f6c-4d19-b6ba-f34b0cbd37b1-logs\") pod \"1cd1c126-8f6c-4d19-b6ba-f34b0cbd37b1\" (UID: \"1cd1c126-8f6c-4d19-b6ba-f34b0cbd37b1\") " Dec 03 13:18:18 crc kubenswrapper[5052]: I1203 13:18:18.035296 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/1cd1c126-8f6c-4d19-b6ba-f34b0cbd37b1-config-data\") pod \"1cd1c126-8f6c-4d19-b6ba-f34b0cbd37b1\" (UID: \"1cd1c126-8f6c-4d19-b6ba-f34b0cbd37b1\") " Dec 03 13:18:18 crc kubenswrapper[5052]: I1203 13:18:18.036734 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1cd1c126-8f6c-4d19-b6ba-f34b0cbd37b1-logs" (OuterVolumeSpecName: "logs") pod "1cd1c126-8f6c-4d19-b6ba-f34b0cbd37b1" (UID: "1cd1c126-8f6c-4d19-b6ba-f34b0cbd37b1"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 13:18:18 crc kubenswrapper[5052]: I1203 13:18:18.040811 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1cd1c126-8f6c-4d19-b6ba-f34b0cbd37b1-kube-api-access-ccw2r" (OuterVolumeSpecName: "kube-api-access-ccw2r") pod "1cd1c126-8f6c-4d19-b6ba-f34b0cbd37b1" (UID: "1cd1c126-8f6c-4d19-b6ba-f34b0cbd37b1"). InnerVolumeSpecName "kube-api-access-ccw2r". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:18:18 crc kubenswrapper[5052]: I1203 13:18:18.046819 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1cd1c126-8f6c-4d19-b6ba-f34b0cbd37b1-config-data-custom" (OuterVolumeSpecName: "config-data-custom") pod "1cd1c126-8f6c-4d19-b6ba-f34b0cbd37b1" (UID: "1cd1c126-8f6c-4d19-b6ba-f34b0cbd37b1"). InnerVolumeSpecName "config-data-custom". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:18:18 crc kubenswrapper[5052]: I1203 13:18:18.071675 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1cd1c126-8f6c-4d19-b6ba-f34b0cbd37b1-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "1cd1c126-8f6c-4d19-b6ba-f34b0cbd37b1" (UID: "1cd1c126-8f6c-4d19-b6ba-f34b0cbd37b1"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:18:18 crc kubenswrapper[5052]: I1203 13:18:18.075903 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1cd1c126-8f6c-4d19-b6ba-f34b0cbd37b1-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "1cd1c126-8f6c-4d19-b6ba-f34b0cbd37b1" (UID: "1cd1c126-8f6c-4d19-b6ba-f34b0cbd37b1"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:18:18 crc kubenswrapper[5052]: I1203 13:18:18.089769 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1cd1c126-8f6c-4d19-b6ba-f34b0cbd37b1-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "1cd1c126-8f6c-4d19-b6ba-f34b0cbd37b1" (UID: "1cd1c126-8f6c-4d19-b6ba-f34b0cbd37b1"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:18:18 crc kubenswrapper[5052]: I1203 13:18:18.093540 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1cd1c126-8f6c-4d19-b6ba-f34b0cbd37b1-config-data" (OuterVolumeSpecName: "config-data") pod "1cd1c126-8f6c-4d19-b6ba-f34b0cbd37b1" (UID: "1cd1c126-8f6c-4d19-b6ba-f34b0cbd37b1"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:18:18 crc kubenswrapper[5052]: I1203 13:18:18.137481 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/1cd1c126-8f6c-4d19-b6ba-f34b0cbd37b1-logs\") on node \"crc\" DevicePath \"\"" Dec 03 13:18:18 crc kubenswrapper[5052]: I1203 13:18:18.137529 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/1cd1c126-8f6c-4d19-b6ba-f34b0cbd37b1-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 13:18:18 crc kubenswrapper[5052]: I1203 13:18:18.137547 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/1cd1c126-8f6c-4d19-b6ba-f34b0cbd37b1-config-data-custom\") on node \"crc\" DevicePath \"\"" Dec 03 13:18:18 crc kubenswrapper[5052]: I1203 13:18:18.137560 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/1cd1c126-8f6c-4d19-b6ba-f34b0cbd37b1-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 13:18:18 crc kubenswrapper[5052]: I1203 13:18:18.137573 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1cd1c126-8f6c-4d19-b6ba-f34b0cbd37b1-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 13:18:18 crc kubenswrapper[5052]: I1203 13:18:18.137586 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/1cd1c126-8f6c-4d19-b6ba-f34b0cbd37b1-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 13:18:18 crc kubenswrapper[5052]: I1203 13:18:18.137600 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-ccw2r\" (UniqueName: \"kubernetes.io/projected/1cd1c126-8f6c-4d19-b6ba-f34b0cbd37b1-kube-api-access-ccw2r\") on node \"crc\" DevicePath \"\"" Dec 03 13:18:18 crc kubenswrapper[5052]: I1203 13:18:18.461029 5052 generic.go:334] "Generic (PLEG): container finished" podID="2b90d104-bcfc-4c8c-b5a0-16f68747ec01" containerID="b81c650b76790d5d29bbe5d40e759d35d38cbade3ae8dadaab978f7bce1037ef" exitCode=143 Dec 03 13:18:18 crc kubenswrapper[5052]: I1203 13:18:18.461104 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="barbican-kuttl-tests/barbican-keystone-listener-94cbbc8d6-wxwcq" event={"ID":"2b90d104-bcfc-4c8c-b5a0-16f68747ec01","Type":"ContainerDied","Data":"b81c650b76790d5d29bbe5d40e759d35d38cbade3ae8dadaab978f7bce1037ef"} Dec 03 13:18:18 crc kubenswrapper[5052]: I1203 13:18:18.463369 5052 generic.go:334] "Generic (PLEG): container finished" podID="9dfe9635-56c2-4a66-9776-57eb16259e44" containerID="1a67c592af264993789211f0bcac258e7194d8c6ffd07517dfc11274d84ccaaa" exitCode=1 Dec 03 13:18:18 crc kubenswrapper[5052]: I1203 13:18:18.463405 5052 generic.go:334] "Generic (PLEG): container finished" podID="9dfe9635-56c2-4a66-9776-57eb16259e44" containerID="27059cd5c285465956cfe0704daf188e0d826a352ab19da31389942f31ceb8da" exitCode=143 Dec 03 13:18:18 crc kubenswrapper[5052]: I1203 13:18:18.463441 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="barbican-kuttl-tests/barbican-worker-79c7d8f77-zbb42" event={"ID":"9dfe9635-56c2-4a66-9776-57eb16259e44","Type":"ContainerDied","Data":"1a67c592af264993789211f0bcac258e7194d8c6ffd07517dfc11274d84ccaaa"} Dec 03 13:18:18 crc kubenswrapper[5052]: I1203 13:18:18.463483 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="barbican-kuttl-tests/barbican-worker-79c7d8f77-zbb42" event={"ID":"9dfe9635-56c2-4a66-9776-57eb16259e44","Type":"ContainerDied","Data":"27059cd5c285465956cfe0704daf188e0d826a352ab19da31389942f31ceb8da"} Dec 03 13:18:18 crc kubenswrapper[5052]: I1203 13:18:18.465205 5052 generic.go:334] "Generic (PLEG): container finished" podID="1cd1c126-8f6c-4d19-b6ba-f34b0cbd37b1" containerID="9ffeca7b60e2b08a162a2bd0fd67835cd41e4e6861bbce88300d98fbc5680138" exitCode=0 Dec 03 13:18:18 crc kubenswrapper[5052]: I1203 13:18:18.465230 5052 generic.go:334] "Generic (PLEG): container finished" podID="1cd1c126-8f6c-4d19-b6ba-f34b0cbd37b1" containerID="5bf996ccb4dcc6fb7b692b007ba9c41573ab80bb3f058993d340190772e4c9c2" exitCode=143 Dec 03 13:18:18 crc kubenswrapper[5052]: I1203 13:18:18.465292 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="barbican-kuttl-tests/barbican-api-6d67464bc8-bqnm6" event={"ID":"1cd1c126-8f6c-4d19-b6ba-f34b0cbd37b1","Type":"ContainerDied","Data":"9ffeca7b60e2b08a162a2bd0fd67835cd41e4e6861bbce88300d98fbc5680138"} Dec 03 13:18:18 crc kubenswrapper[5052]: I1203 13:18:18.465279 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="barbican-kuttl-tests/barbican-api-6d67464bc8-bqnm6" Dec 03 13:18:18 crc kubenswrapper[5052]: I1203 13:18:18.465314 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="barbican-kuttl-tests/barbican-api-6d67464bc8-bqnm6" event={"ID":"1cd1c126-8f6c-4d19-b6ba-f34b0cbd37b1","Type":"ContainerDied","Data":"5bf996ccb4dcc6fb7b692b007ba9c41573ab80bb3f058993d340190772e4c9c2"} Dec 03 13:18:18 crc kubenswrapper[5052]: I1203 13:18:18.465328 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="barbican-kuttl-tests/barbican-api-6d67464bc8-bqnm6" event={"ID":"1cd1c126-8f6c-4d19-b6ba-f34b0cbd37b1","Type":"ContainerDied","Data":"1bdb45b3dff5b148b5815192bbbd8a2a5102db2c56239d44354dd3424cc445e3"} Dec 03 13:18:18 crc kubenswrapper[5052]: I1203 13:18:18.465340 5052 scope.go:117] "RemoveContainer" containerID="9ffeca7b60e2b08a162a2bd0fd67835cd41e4e6861bbce88300d98fbc5680138" Dec 03 13:18:18 crc kubenswrapper[5052]: I1203 13:18:18.469410 5052 generic.go:334] "Generic (PLEG): container finished" podID="406d0ab4-b286-41c6-b080-5b3e41f6c3eb" containerID="6b9bfacd4508cedeeb65c806b3b42b4c837645204ac98fec3e0d29a3ee0f2971" exitCode=0 Dec 03 13:18:18 crc kubenswrapper[5052]: I1203 13:18:18.469448 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="barbican-kuttl-tests/barbicana311-account-delete-lggsm" event={"ID":"406d0ab4-b286-41c6-b080-5b3e41f6c3eb","Type":"ContainerDied","Data":"6b9bfacd4508cedeeb65c806b3b42b4c837645204ac98fec3e0d29a3ee0f2971"} Dec 03 13:18:18 crc kubenswrapper[5052]: I1203 13:18:18.503034 5052 scope.go:117] "RemoveContainer" containerID="5bf996ccb4dcc6fb7b692b007ba9c41573ab80bb3f058993d340190772e4c9c2" Dec 03 13:18:18 crc kubenswrapper[5052]: I1203 13:18:18.531888 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["barbican-kuttl-tests/barbican-api-6d67464bc8-bqnm6"] Dec 03 13:18:18 crc kubenswrapper[5052]: I1203 13:18:18.538478 5052 scope.go:117] "RemoveContainer" containerID="9ffeca7b60e2b08a162a2bd0fd67835cd41e4e6861bbce88300d98fbc5680138" Dec 03 13:18:18 crc kubenswrapper[5052]: I1203 13:18:18.538630 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["barbican-kuttl-tests/barbican-api-6d67464bc8-bqnm6"] Dec 03 13:18:18 crc kubenswrapper[5052]: E1203 13:18:18.539709 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9ffeca7b60e2b08a162a2bd0fd67835cd41e4e6861bbce88300d98fbc5680138\": container with ID starting with 9ffeca7b60e2b08a162a2bd0fd67835cd41e4e6861bbce88300d98fbc5680138 not found: ID does not exist" containerID="9ffeca7b60e2b08a162a2bd0fd67835cd41e4e6861bbce88300d98fbc5680138" Dec 03 13:18:18 crc kubenswrapper[5052]: I1203 13:18:18.539814 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9ffeca7b60e2b08a162a2bd0fd67835cd41e4e6861bbce88300d98fbc5680138"} err="failed to get container status \"9ffeca7b60e2b08a162a2bd0fd67835cd41e4e6861bbce88300d98fbc5680138\": rpc error: code = NotFound desc = could not find container \"9ffeca7b60e2b08a162a2bd0fd67835cd41e4e6861bbce88300d98fbc5680138\": container with ID starting with 9ffeca7b60e2b08a162a2bd0fd67835cd41e4e6861bbce88300d98fbc5680138 not found: ID does not exist" Dec 03 13:18:18 crc kubenswrapper[5052]: I1203 13:18:18.539875 5052 scope.go:117] "RemoveContainer" containerID="5bf996ccb4dcc6fb7b692b007ba9c41573ab80bb3f058993d340190772e4c9c2" Dec 03 13:18:18 crc kubenswrapper[5052]: E1203 13:18:18.540444 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5bf996ccb4dcc6fb7b692b007ba9c41573ab80bb3f058993d340190772e4c9c2\": container with ID starting with 5bf996ccb4dcc6fb7b692b007ba9c41573ab80bb3f058993d340190772e4c9c2 not found: ID does not exist" containerID="5bf996ccb4dcc6fb7b692b007ba9c41573ab80bb3f058993d340190772e4c9c2" Dec 03 13:18:18 crc kubenswrapper[5052]: I1203 13:18:18.540493 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5bf996ccb4dcc6fb7b692b007ba9c41573ab80bb3f058993d340190772e4c9c2"} err="failed to get container status \"5bf996ccb4dcc6fb7b692b007ba9c41573ab80bb3f058993d340190772e4c9c2\": rpc error: code = NotFound desc = could not find container \"5bf996ccb4dcc6fb7b692b007ba9c41573ab80bb3f058993d340190772e4c9c2\": container with ID starting with 5bf996ccb4dcc6fb7b692b007ba9c41573ab80bb3f058993d340190772e4c9c2 not found: ID does not exist" Dec 03 13:18:18 crc kubenswrapper[5052]: I1203 13:18:18.540588 5052 scope.go:117] "RemoveContainer" containerID="9ffeca7b60e2b08a162a2bd0fd67835cd41e4e6861bbce88300d98fbc5680138" Dec 03 13:18:18 crc kubenswrapper[5052]: I1203 13:18:18.541192 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9ffeca7b60e2b08a162a2bd0fd67835cd41e4e6861bbce88300d98fbc5680138"} err="failed to get container status \"9ffeca7b60e2b08a162a2bd0fd67835cd41e4e6861bbce88300d98fbc5680138\": rpc error: code = NotFound desc = could not find container \"9ffeca7b60e2b08a162a2bd0fd67835cd41e4e6861bbce88300d98fbc5680138\": container with ID starting with 9ffeca7b60e2b08a162a2bd0fd67835cd41e4e6861bbce88300d98fbc5680138 not found: ID does not exist" Dec 03 13:18:18 crc kubenswrapper[5052]: I1203 13:18:18.541226 5052 scope.go:117] "RemoveContainer" containerID="5bf996ccb4dcc6fb7b692b007ba9c41573ab80bb3f058993d340190772e4c9c2" Dec 03 13:18:18 crc kubenswrapper[5052]: I1203 13:18:18.543158 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5bf996ccb4dcc6fb7b692b007ba9c41573ab80bb3f058993d340190772e4c9c2"} err="failed to get container status \"5bf996ccb4dcc6fb7b692b007ba9c41573ab80bb3f058993d340190772e4c9c2\": rpc error: code = NotFound desc = could not find container \"5bf996ccb4dcc6fb7b692b007ba9c41573ab80bb3f058993d340190772e4c9c2\": container with ID starting with 5bf996ccb4dcc6fb7b692b007ba9c41573ab80bb3f058993d340190772e4c9c2 not found: ID does not exist" Dec 03 13:18:18 crc kubenswrapper[5052]: I1203 13:18:18.627441 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="barbican-kuttl-tests/barbican-worker-79c7d8f77-zbb42" Dec 03 13:18:18 crc kubenswrapper[5052]: I1203 13:18:18.747265 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/9dfe9635-56c2-4a66-9776-57eb16259e44-config-data-custom\") pod \"9dfe9635-56c2-4a66-9776-57eb16259e44\" (UID: \"9dfe9635-56c2-4a66-9776-57eb16259e44\") " Dec 03 13:18:18 crc kubenswrapper[5052]: I1203 13:18:18.747375 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s68rf\" (UniqueName: \"kubernetes.io/projected/9dfe9635-56c2-4a66-9776-57eb16259e44-kube-api-access-s68rf\") pod \"9dfe9635-56c2-4a66-9776-57eb16259e44\" (UID: \"9dfe9635-56c2-4a66-9776-57eb16259e44\") " Dec 03 13:18:18 crc kubenswrapper[5052]: I1203 13:18:18.747638 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/9dfe9635-56c2-4a66-9776-57eb16259e44-config-data\") pod \"9dfe9635-56c2-4a66-9776-57eb16259e44\" (UID: \"9dfe9635-56c2-4a66-9776-57eb16259e44\") " Dec 03 13:18:18 crc kubenswrapper[5052]: I1203 13:18:18.747750 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/9dfe9635-56c2-4a66-9776-57eb16259e44-logs\") pod \"9dfe9635-56c2-4a66-9776-57eb16259e44\" (UID: \"9dfe9635-56c2-4a66-9776-57eb16259e44\") " Dec 03 13:18:18 crc kubenswrapper[5052]: I1203 13:18:18.747776 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9dfe9635-56c2-4a66-9776-57eb16259e44-combined-ca-bundle\") pod \"9dfe9635-56c2-4a66-9776-57eb16259e44\" (UID: \"9dfe9635-56c2-4a66-9776-57eb16259e44\") " Dec 03 13:18:18 crc kubenswrapper[5052]: I1203 13:18:18.750452 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9dfe9635-56c2-4a66-9776-57eb16259e44-logs" (OuterVolumeSpecName: "logs") pod "9dfe9635-56c2-4a66-9776-57eb16259e44" (UID: "9dfe9635-56c2-4a66-9776-57eb16259e44"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 13:18:18 crc kubenswrapper[5052]: I1203 13:18:18.752317 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9dfe9635-56c2-4a66-9776-57eb16259e44-config-data-custom" (OuterVolumeSpecName: "config-data-custom") pod "9dfe9635-56c2-4a66-9776-57eb16259e44" (UID: "9dfe9635-56c2-4a66-9776-57eb16259e44"). InnerVolumeSpecName "config-data-custom". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:18:18 crc kubenswrapper[5052]: I1203 13:18:18.752451 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9dfe9635-56c2-4a66-9776-57eb16259e44-kube-api-access-s68rf" (OuterVolumeSpecName: "kube-api-access-s68rf") pod "9dfe9635-56c2-4a66-9776-57eb16259e44" (UID: "9dfe9635-56c2-4a66-9776-57eb16259e44"). InnerVolumeSpecName "kube-api-access-s68rf". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:18:18 crc kubenswrapper[5052]: I1203 13:18:18.776541 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9dfe9635-56c2-4a66-9776-57eb16259e44-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "9dfe9635-56c2-4a66-9776-57eb16259e44" (UID: "9dfe9635-56c2-4a66-9776-57eb16259e44"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:18:18 crc kubenswrapper[5052]: I1203 13:18:18.786309 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9dfe9635-56c2-4a66-9776-57eb16259e44-config-data" (OuterVolumeSpecName: "config-data") pod "9dfe9635-56c2-4a66-9776-57eb16259e44" (UID: "9dfe9635-56c2-4a66-9776-57eb16259e44"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:18:18 crc kubenswrapper[5052]: I1203 13:18:18.849826 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/9dfe9635-56c2-4a66-9776-57eb16259e44-logs\") on node \"crc\" DevicePath \"\"" Dec 03 13:18:18 crc kubenswrapper[5052]: I1203 13:18:18.850352 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9dfe9635-56c2-4a66-9776-57eb16259e44-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 13:18:18 crc kubenswrapper[5052]: I1203 13:18:18.850429 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/9dfe9635-56c2-4a66-9776-57eb16259e44-config-data-custom\") on node \"crc\" DevicePath \"\"" Dec 03 13:18:18 crc kubenswrapper[5052]: I1203 13:18:18.850502 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-s68rf\" (UniqueName: \"kubernetes.io/projected/9dfe9635-56c2-4a66-9776-57eb16259e44-kube-api-access-s68rf\") on node \"crc\" DevicePath \"\"" Dec 03 13:18:18 crc kubenswrapper[5052]: I1203 13:18:18.850614 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/9dfe9635-56c2-4a66-9776-57eb16259e44-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 13:18:19 crc kubenswrapper[5052]: I1203 13:18:19.045980 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="barbican-kuttl-tests/barbican-keystone-listener-94cbbc8d6-wxwcq" Dec 03 13:18:19 crc kubenswrapper[5052]: I1203 13:18:19.155401 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/2b90d104-bcfc-4c8c-b5a0-16f68747ec01-config-data\") pod \"2b90d104-bcfc-4c8c-b5a0-16f68747ec01\" (UID: \"2b90d104-bcfc-4c8c-b5a0-16f68747ec01\") " Dec 03 13:18:19 crc kubenswrapper[5052]: I1203 13:18:19.155459 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2b90d104-bcfc-4c8c-b5a0-16f68747ec01-combined-ca-bundle\") pod \"2b90d104-bcfc-4c8c-b5a0-16f68747ec01\" (UID: \"2b90d104-bcfc-4c8c-b5a0-16f68747ec01\") " Dec 03 13:18:19 crc kubenswrapper[5052]: I1203 13:18:19.155590 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ql7ql\" (UniqueName: \"kubernetes.io/projected/2b90d104-bcfc-4c8c-b5a0-16f68747ec01-kube-api-access-ql7ql\") pod \"2b90d104-bcfc-4c8c-b5a0-16f68747ec01\" (UID: \"2b90d104-bcfc-4c8c-b5a0-16f68747ec01\") " Dec 03 13:18:19 crc kubenswrapper[5052]: I1203 13:18:19.155668 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/2b90d104-bcfc-4c8c-b5a0-16f68747ec01-logs\") pod \"2b90d104-bcfc-4c8c-b5a0-16f68747ec01\" (UID: \"2b90d104-bcfc-4c8c-b5a0-16f68747ec01\") " Dec 03 13:18:19 crc kubenswrapper[5052]: I1203 13:18:19.155712 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/2b90d104-bcfc-4c8c-b5a0-16f68747ec01-config-data-custom\") pod \"2b90d104-bcfc-4c8c-b5a0-16f68747ec01\" (UID: \"2b90d104-bcfc-4c8c-b5a0-16f68747ec01\") " Dec 03 13:18:19 crc kubenswrapper[5052]: I1203 13:18:19.156212 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2b90d104-bcfc-4c8c-b5a0-16f68747ec01-logs" (OuterVolumeSpecName: "logs") pod "2b90d104-bcfc-4c8c-b5a0-16f68747ec01" (UID: "2b90d104-bcfc-4c8c-b5a0-16f68747ec01"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 13:18:19 crc kubenswrapper[5052]: I1203 13:18:19.160478 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2b90d104-bcfc-4c8c-b5a0-16f68747ec01-config-data-custom" (OuterVolumeSpecName: "config-data-custom") pod "2b90d104-bcfc-4c8c-b5a0-16f68747ec01" (UID: "2b90d104-bcfc-4c8c-b5a0-16f68747ec01"). InnerVolumeSpecName "config-data-custom". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:18:19 crc kubenswrapper[5052]: I1203 13:18:19.160540 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2b90d104-bcfc-4c8c-b5a0-16f68747ec01-kube-api-access-ql7ql" (OuterVolumeSpecName: "kube-api-access-ql7ql") pod "2b90d104-bcfc-4c8c-b5a0-16f68747ec01" (UID: "2b90d104-bcfc-4c8c-b5a0-16f68747ec01"). InnerVolumeSpecName "kube-api-access-ql7ql". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:18:19 crc kubenswrapper[5052]: I1203 13:18:19.173822 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2b90d104-bcfc-4c8c-b5a0-16f68747ec01-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "2b90d104-bcfc-4c8c-b5a0-16f68747ec01" (UID: "2b90d104-bcfc-4c8c-b5a0-16f68747ec01"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:18:19 crc kubenswrapper[5052]: I1203 13:18:19.192277 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2b90d104-bcfc-4c8c-b5a0-16f68747ec01-config-data" (OuterVolumeSpecName: "config-data") pod "2b90d104-bcfc-4c8c-b5a0-16f68747ec01" (UID: "2b90d104-bcfc-4c8c-b5a0-16f68747ec01"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:18:19 crc kubenswrapper[5052]: I1203 13:18:19.256960 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/2b90d104-bcfc-4c8c-b5a0-16f68747ec01-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 13:18:19 crc kubenswrapper[5052]: I1203 13:18:19.257001 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2b90d104-bcfc-4c8c-b5a0-16f68747ec01-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 13:18:19 crc kubenswrapper[5052]: I1203 13:18:19.257015 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-ql7ql\" (UniqueName: \"kubernetes.io/projected/2b90d104-bcfc-4c8c-b5a0-16f68747ec01-kube-api-access-ql7ql\") on node \"crc\" DevicePath \"\"" Dec 03 13:18:19 crc kubenswrapper[5052]: I1203 13:18:19.257025 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/2b90d104-bcfc-4c8c-b5a0-16f68747ec01-logs\") on node \"crc\" DevicePath \"\"" Dec 03 13:18:19 crc kubenswrapper[5052]: I1203 13:18:19.257033 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/2b90d104-bcfc-4c8c-b5a0-16f68747ec01-config-data-custom\") on node \"crc\" DevicePath \"\"" Dec 03 13:18:19 crc kubenswrapper[5052]: I1203 13:18:19.479743 5052 generic.go:334] "Generic (PLEG): container finished" podID="2b90d104-bcfc-4c8c-b5a0-16f68747ec01" containerID="459a240a4da7270c5ba3e7487932749b68c6ccf78c76547f1fdb9f2d15d3d309" exitCode=1 Dec 03 13:18:19 crc kubenswrapper[5052]: I1203 13:18:19.479816 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="barbican-kuttl-tests/barbican-keystone-listener-94cbbc8d6-wxwcq" Dec 03 13:18:19 crc kubenswrapper[5052]: I1203 13:18:19.479808 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="barbican-kuttl-tests/barbican-keystone-listener-94cbbc8d6-wxwcq" event={"ID":"2b90d104-bcfc-4c8c-b5a0-16f68747ec01","Type":"ContainerDied","Data":"459a240a4da7270c5ba3e7487932749b68c6ccf78c76547f1fdb9f2d15d3d309"} Dec 03 13:18:19 crc kubenswrapper[5052]: I1203 13:18:19.480204 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="barbican-kuttl-tests/barbican-keystone-listener-94cbbc8d6-wxwcq" event={"ID":"2b90d104-bcfc-4c8c-b5a0-16f68747ec01","Type":"ContainerDied","Data":"057f0d58ca500095e475548b41e1a5d86d4549ee50d17af39efaaca94e9ab335"} Dec 03 13:18:19 crc kubenswrapper[5052]: I1203 13:18:19.480226 5052 scope.go:117] "RemoveContainer" containerID="459a240a4da7270c5ba3e7487932749b68c6ccf78c76547f1fdb9f2d15d3d309" Dec 03 13:18:19 crc kubenswrapper[5052]: I1203 13:18:19.482183 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="barbican-kuttl-tests/barbican-worker-79c7d8f77-zbb42" Dec 03 13:18:19 crc kubenswrapper[5052]: I1203 13:18:19.482183 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="barbican-kuttl-tests/barbican-worker-79c7d8f77-zbb42" event={"ID":"9dfe9635-56c2-4a66-9776-57eb16259e44","Type":"ContainerDied","Data":"093df003b028fa9b0e731cfad1a5916ed8a6e59489550858dd0bc4aa611b8c72"} Dec 03 13:18:19 crc kubenswrapper[5052]: I1203 13:18:19.515291 5052 scope.go:117] "RemoveContainer" containerID="b81c650b76790d5d29bbe5d40e759d35d38cbade3ae8dadaab978f7bce1037ef" Dec 03 13:18:19 crc kubenswrapper[5052]: I1203 13:18:19.515595 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["barbican-kuttl-tests/barbican-keystone-listener-94cbbc8d6-wxwcq"] Dec 03 13:18:19 crc kubenswrapper[5052]: I1203 13:18:19.528432 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["barbican-kuttl-tests/barbican-keystone-listener-94cbbc8d6-wxwcq"] Dec 03 13:18:19 crc kubenswrapper[5052]: I1203 13:18:19.533715 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["barbican-kuttl-tests/barbican-worker-79c7d8f77-zbb42"] Dec 03 13:18:19 crc kubenswrapper[5052]: I1203 13:18:19.536416 5052 scope.go:117] "RemoveContainer" containerID="459a240a4da7270c5ba3e7487932749b68c6ccf78c76547f1fdb9f2d15d3d309" Dec 03 13:18:19 crc kubenswrapper[5052]: E1203 13:18:19.536916 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"459a240a4da7270c5ba3e7487932749b68c6ccf78c76547f1fdb9f2d15d3d309\": container with ID starting with 459a240a4da7270c5ba3e7487932749b68c6ccf78c76547f1fdb9f2d15d3d309 not found: ID does not exist" containerID="459a240a4da7270c5ba3e7487932749b68c6ccf78c76547f1fdb9f2d15d3d309" Dec 03 13:18:19 crc kubenswrapper[5052]: I1203 13:18:19.536960 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"459a240a4da7270c5ba3e7487932749b68c6ccf78c76547f1fdb9f2d15d3d309"} err="failed to get container status \"459a240a4da7270c5ba3e7487932749b68c6ccf78c76547f1fdb9f2d15d3d309\": rpc error: code = NotFound desc = could not find container \"459a240a4da7270c5ba3e7487932749b68c6ccf78c76547f1fdb9f2d15d3d309\": container with ID starting with 459a240a4da7270c5ba3e7487932749b68c6ccf78c76547f1fdb9f2d15d3d309 not found: ID does not exist" Dec 03 13:18:19 crc kubenswrapper[5052]: I1203 13:18:19.536987 5052 scope.go:117] "RemoveContainer" containerID="b81c650b76790d5d29bbe5d40e759d35d38cbade3ae8dadaab978f7bce1037ef" Dec 03 13:18:19 crc kubenswrapper[5052]: E1203 13:18:19.537769 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b81c650b76790d5d29bbe5d40e759d35d38cbade3ae8dadaab978f7bce1037ef\": container with ID starting with b81c650b76790d5d29bbe5d40e759d35d38cbade3ae8dadaab978f7bce1037ef not found: ID does not exist" containerID="b81c650b76790d5d29bbe5d40e759d35d38cbade3ae8dadaab978f7bce1037ef" Dec 03 13:18:19 crc kubenswrapper[5052]: I1203 13:18:19.537801 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b81c650b76790d5d29bbe5d40e759d35d38cbade3ae8dadaab978f7bce1037ef"} err="failed to get container status \"b81c650b76790d5d29bbe5d40e759d35d38cbade3ae8dadaab978f7bce1037ef\": rpc error: code = NotFound desc = could not find container \"b81c650b76790d5d29bbe5d40e759d35d38cbade3ae8dadaab978f7bce1037ef\": container with ID starting with b81c650b76790d5d29bbe5d40e759d35d38cbade3ae8dadaab978f7bce1037ef not found: ID does not exist" Dec 03 13:18:19 crc kubenswrapper[5052]: I1203 13:18:19.537829 5052 scope.go:117] "RemoveContainer" containerID="1a67c592af264993789211f0bcac258e7194d8c6ffd07517dfc11274d84ccaaa" Dec 03 13:18:19 crc kubenswrapper[5052]: I1203 13:18:19.540328 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["barbican-kuttl-tests/barbican-worker-79c7d8f77-zbb42"] Dec 03 13:18:19 crc kubenswrapper[5052]: I1203 13:18:19.560774 5052 scope.go:117] "RemoveContainer" containerID="27059cd5c285465956cfe0704daf188e0d826a352ab19da31389942f31ceb8da" Dec 03 13:18:19 crc kubenswrapper[5052]: I1203 13:18:19.682953 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1cd1c126-8f6c-4d19-b6ba-f34b0cbd37b1" path="/var/lib/kubelet/pods/1cd1c126-8f6c-4d19-b6ba-f34b0cbd37b1/volumes" Dec 03 13:18:19 crc kubenswrapper[5052]: I1203 13:18:19.683543 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2b90d104-bcfc-4c8c-b5a0-16f68747ec01" path="/var/lib/kubelet/pods/2b90d104-bcfc-4c8c-b5a0-16f68747ec01/volumes" Dec 03 13:18:19 crc kubenswrapper[5052]: I1203 13:18:19.684119 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9dfe9635-56c2-4a66-9776-57eb16259e44" path="/var/lib/kubelet/pods/9dfe9635-56c2-4a66-9776-57eb16259e44/volumes" Dec 03 13:18:19 crc kubenswrapper[5052]: I1203 13:18:19.709146 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="barbican-kuttl-tests/barbicana311-account-delete-lggsm" Dec 03 13:18:19 crc kubenswrapper[5052]: I1203 13:18:19.865703 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/406d0ab4-b286-41c6-b080-5b3e41f6c3eb-operator-scripts\") pod \"406d0ab4-b286-41c6-b080-5b3e41f6c3eb\" (UID: \"406d0ab4-b286-41c6-b080-5b3e41f6c3eb\") " Dec 03 13:18:19 crc kubenswrapper[5052]: I1203 13:18:19.866041 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zpdw4\" (UniqueName: \"kubernetes.io/projected/406d0ab4-b286-41c6-b080-5b3e41f6c3eb-kube-api-access-zpdw4\") pod \"406d0ab4-b286-41c6-b080-5b3e41f6c3eb\" (UID: \"406d0ab4-b286-41c6-b080-5b3e41f6c3eb\") " Dec 03 13:18:19 crc kubenswrapper[5052]: I1203 13:18:19.866708 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/406d0ab4-b286-41c6-b080-5b3e41f6c3eb-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "406d0ab4-b286-41c6-b080-5b3e41f6c3eb" (UID: "406d0ab4-b286-41c6-b080-5b3e41f6c3eb"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:18:19 crc kubenswrapper[5052]: I1203 13:18:19.871075 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/406d0ab4-b286-41c6-b080-5b3e41f6c3eb-kube-api-access-zpdw4" (OuterVolumeSpecName: "kube-api-access-zpdw4") pod "406d0ab4-b286-41c6-b080-5b3e41f6c3eb" (UID: "406d0ab4-b286-41c6-b080-5b3e41f6c3eb"). InnerVolumeSpecName "kube-api-access-zpdw4". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:18:19 crc kubenswrapper[5052]: I1203 13:18:19.967573 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/406d0ab4-b286-41c6-b080-5b3e41f6c3eb-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 13:18:19 crc kubenswrapper[5052]: I1203 13:18:19.967613 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-zpdw4\" (UniqueName: \"kubernetes.io/projected/406d0ab4-b286-41c6-b080-5b3e41f6c3eb-kube-api-access-zpdw4\") on node \"crc\" DevicePath \"\"" Dec 03 13:18:20 crc kubenswrapper[5052]: I1203 13:18:20.497214 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="barbican-kuttl-tests/barbicana311-account-delete-lggsm" event={"ID":"406d0ab4-b286-41c6-b080-5b3e41f6c3eb","Type":"ContainerDied","Data":"7cdb9851e7c850140f4b04daaa0ee00241ed22c465e2064c38e2f68f49b7f432"} Dec 03 13:18:20 crc kubenswrapper[5052]: I1203 13:18:20.497259 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7cdb9851e7c850140f4b04daaa0ee00241ed22c465e2064c38e2f68f49b7f432" Dec 03 13:18:20 crc kubenswrapper[5052]: I1203 13:18:20.497231 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="barbican-kuttl-tests/barbicana311-account-delete-lggsm" Dec 03 13:18:21 crc kubenswrapper[5052]: I1203 13:18:21.545634 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["barbican-kuttl-tests/barbican-db-create-kk889"] Dec 03 13:18:21 crc kubenswrapper[5052]: I1203 13:18:21.554313 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["barbican-kuttl-tests/barbican-db-create-kk889"] Dec 03 13:18:21 crc kubenswrapper[5052]: I1203 13:18:21.562224 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["barbican-kuttl-tests/barbican-a311-account-create-update-c5mpp"] Dec 03 13:18:21 crc kubenswrapper[5052]: I1203 13:18:21.567885 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["barbican-kuttl-tests/barbicana311-account-delete-lggsm"] Dec 03 13:18:21 crc kubenswrapper[5052]: I1203 13:18:21.573090 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["barbican-kuttl-tests/barbican-a311-account-create-update-c5mpp"] Dec 03 13:18:21 crc kubenswrapper[5052]: I1203 13:18:21.581198 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["barbican-kuttl-tests/barbicana311-account-delete-lggsm"] Dec 03 13:18:21 crc kubenswrapper[5052]: I1203 13:18:21.681473 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1d6fbb80-d818-42ba-9bf7-caf7148a7a19" path="/var/lib/kubelet/pods/1d6fbb80-d818-42ba-9bf7-caf7148a7a19/volumes" Dec 03 13:18:21 crc kubenswrapper[5052]: I1203 13:18:21.682038 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="406d0ab4-b286-41c6-b080-5b3e41f6c3eb" path="/var/lib/kubelet/pods/406d0ab4-b286-41c6-b080-5b3e41f6c3eb/volumes" Dec 03 13:18:21 crc kubenswrapper[5052]: I1203 13:18:21.682490 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f1847d2e-1709-40fe-ae2b-97c1295a186e" path="/var/lib/kubelet/pods/f1847d2e-1709-40fe-ae2b-97c1295a186e/volumes" Dec 03 13:18:28 crc kubenswrapper[5052]: I1203 13:18:28.560365 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="barbican-kuttl-tests/barbican-keystone-listener-66b47d8764-wbbbf" Dec 03 13:18:28 crc kubenswrapper[5052]: I1203 13:18:28.561321 5052 generic.go:334] "Generic (PLEG): container finished" podID="3676d121-81bf-431a-b180-39c950731a2f" containerID="4ba3d3e87a26e0d6d7240b66b881b51bccdf1a55ddfece30f153f6571cb9deb5" exitCode=137 Dec 03 13:18:28 crc kubenswrapper[5052]: I1203 13:18:28.561363 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="barbican-kuttl-tests/barbican-keystone-listener-66b47d8764-wbbbf" event={"ID":"3676d121-81bf-431a-b180-39c950731a2f","Type":"ContainerDied","Data":"4ba3d3e87a26e0d6d7240b66b881b51bccdf1a55ddfece30f153f6571cb9deb5"} Dec 03 13:18:28 crc kubenswrapper[5052]: I1203 13:18:28.561395 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="barbican-kuttl-tests/barbican-keystone-listener-66b47d8764-wbbbf" event={"ID":"3676d121-81bf-431a-b180-39c950731a2f","Type":"ContainerDied","Data":"d45055c6ef7825aad860064b9f9481e3ce3d665c71b2f5632d8c044bb106ec80"} Dec 03 13:18:28 crc kubenswrapper[5052]: I1203 13:18:28.561415 5052 scope.go:117] "RemoveContainer" containerID="4ba3d3e87a26e0d6d7240b66b881b51bccdf1a55ddfece30f153f6571cb9deb5" Dec 03 13:18:28 crc kubenswrapper[5052]: I1203 13:18:28.595883 5052 scope.go:117] "RemoveContainer" containerID="13ad56c24c7d9b6fbee10e8632bad5f404915f724a561319e8f88cbcaa76b786" Dec 03 13:18:28 crc kubenswrapper[5052]: I1203 13:18:28.617095 5052 scope.go:117] "RemoveContainer" containerID="4ba3d3e87a26e0d6d7240b66b881b51bccdf1a55ddfece30f153f6571cb9deb5" Dec 03 13:18:28 crc kubenswrapper[5052]: E1203 13:18:28.617541 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4ba3d3e87a26e0d6d7240b66b881b51bccdf1a55ddfece30f153f6571cb9deb5\": container with ID starting with 4ba3d3e87a26e0d6d7240b66b881b51bccdf1a55ddfece30f153f6571cb9deb5 not found: ID does not exist" containerID="4ba3d3e87a26e0d6d7240b66b881b51bccdf1a55ddfece30f153f6571cb9deb5" Dec 03 13:18:28 crc kubenswrapper[5052]: I1203 13:18:28.617590 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4ba3d3e87a26e0d6d7240b66b881b51bccdf1a55ddfece30f153f6571cb9deb5"} err="failed to get container status \"4ba3d3e87a26e0d6d7240b66b881b51bccdf1a55ddfece30f153f6571cb9deb5\": rpc error: code = NotFound desc = could not find container \"4ba3d3e87a26e0d6d7240b66b881b51bccdf1a55ddfece30f153f6571cb9deb5\": container with ID starting with 4ba3d3e87a26e0d6d7240b66b881b51bccdf1a55ddfece30f153f6571cb9deb5 not found: ID does not exist" Dec 03 13:18:28 crc kubenswrapper[5052]: I1203 13:18:28.617621 5052 scope.go:117] "RemoveContainer" containerID="13ad56c24c7d9b6fbee10e8632bad5f404915f724a561319e8f88cbcaa76b786" Dec 03 13:18:28 crc kubenswrapper[5052]: E1203 13:18:28.617928 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"13ad56c24c7d9b6fbee10e8632bad5f404915f724a561319e8f88cbcaa76b786\": container with ID starting with 13ad56c24c7d9b6fbee10e8632bad5f404915f724a561319e8f88cbcaa76b786 not found: ID does not exist" containerID="13ad56c24c7d9b6fbee10e8632bad5f404915f724a561319e8f88cbcaa76b786" Dec 03 13:18:28 crc kubenswrapper[5052]: I1203 13:18:28.617956 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"13ad56c24c7d9b6fbee10e8632bad5f404915f724a561319e8f88cbcaa76b786"} err="failed to get container status \"13ad56c24c7d9b6fbee10e8632bad5f404915f724a561319e8f88cbcaa76b786\": rpc error: code = NotFound desc = could not find container \"13ad56c24c7d9b6fbee10e8632bad5f404915f724a561319e8f88cbcaa76b786\": container with ID starting with 13ad56c24c7d9b6fbee10e8632bad5f404915f724a561319e8f88cbcaa76b786 not found: ID does not exist" Dec 03 13:18:28 crc kubenswrapper[5052]: I1203 13:18:28.695748 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/3676d121-81bf-431a-b180-39c950731a2f-logs\") pod \"3676d121-81bf-431a-b180-39c950731a2f\" (UID: \"3676d121-81bf-431a-b180-39c950731a2f\") " Dec 03 13:18:28 crc kubenswrapper[5052]: I1203 13:18:28.695794 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/3676d121-81bf-431a-b180-39c950731a2f-config-data\") pod \"3676d121-81bf-431a-b180-39c950731a2f\" (UID: \"3676d121-81bf-431a-b180-39c950731a2f\") " Dec 03 13:18:28 crc kubenswrapper[5052]: I1203 13:18:28.695867 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/3676d121-81bf-431a-b180-39c950731a2f-config-data-custom\") pod \"3676d121-81bf-431a-b180-39c950731a2f\" (UID: \"3676d121-81bf-431a-b180-39c950731a2f\") " Dec 03 13:18:28 crc kubenswrapper[5052]: I1203 13:18:28.695926 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jxrdv\" (UniqueName: \"kubernetes.io/projected/3676d121-81bf-431a-b180-39c950731a2f-kube-api-access-jxrdv\") pod \"3676d121-81bf-431a-b180-39c950731a2f\" (UID: \"3676d121-81bf-431a-b180-39c950731a2f\") " Dec 03 13:18:28 crc kubenswrapper[5052]: I1203 13:18:28.696525 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3676d121-81bf-431a-b180-39c950731a2f-logs" (OuterVolumeSpecName: "logs") pod "3676d121-81bf-431a-b180-39c950731a2f" (UID: "3676d121-81bf-431a-b180-39c950731a2f"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 13:18:28 crc kubenswrapper[5052]: I1203 13:18:28.701276 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3676d121-81bf-431a-b180-39c950731a2f-config-data-custom" (OuterVolumeSpecName: "config-data-custom") pod "3676d121-81bf-431a-b180-39c950731a2f" (UID: "3676d121-81bf-431a-b180-39c950731a2f"). InnerVolumeSpecName "config-data-custom". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:18:28 crc kubenswrapper[5052]: I1203 13:18:28.701381 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3676d121-81bf-431a-b180-39c950731a2f-kube-api-access-jxrdv" (OuterVolumeSpecName: "kube-api-access-jxrdv") pod "3676d121-81bf-431a-b180-39c950731a2f" (UID: "3676d121-81bf-431a-b180-39c950731a2f"). InnerVolumeSpecName "kube-api-access-jxrdv". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:18:28 crc kubenswrapper[5052]: I1203 13:18:28.730838 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3676d121-81bf-431a-b180-39c950731a2f-config-data" (OuterVolumeSpecName: "config-data") pod "3676d121-81bf-431a-b180-39c950731a2f" (UID: "3676d121-81bf-431a-b180-39c950731a2f"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:18:28 crc kubenswrapper[5052]: I1203 13:18:28.798079 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/3676d121-81bf-431a-b180-39c950731a2f-logs\") on node \"crc\" DevicePath \"\"" Dec 03 13:18:28 crc kubenswrapper[5052]: I1203 13:18:28.798123 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/3676d121-81bf-431a-b180-39c950731a2f-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 13:18:28 crc kubenswrapper[5052]: I1203 13:18:28.798139 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-custom\" (UniqueName: \"kubernetes.io/secret/3676d121-81bf-431a-b180-39c950731a2f-config-data-custom\") on node \"crc\" DevicePath \"\"" Dec 03 13:18:28 crc kubenswrapper[5052]: I1203 13:18:28.798156 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-jxrdv\" (UniqueName: \"kubernetes.io/projected/3676d121-81bf-431a-b180-39c950731a2f-kube-api-access-jxrdv\") on node \"crc\" DevicePath \"\"" Dec 03 13:18:29 crc kubenswrapper[5052]: I1203 13:18:29.379929 5052 patch_prober.go:28] interesting pod/machine-config-daemon-74bj6 container/machine-config-daemon namespace/openshift-machine-config-operator: Liveness probe status=failure output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" start-of-body= Dec 03 13:18:29 crc kubenswrapper[5052]: I1203 13:18:29.380243 5052 prober.go:107] "Probe failed" probeType="Liveness" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" containerName="machine-config-daemon" probeResult="failure" output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" Dec 03 13:18:29 crc kubenswrapper[5052]: I1203 13:18:29.380287 5052 kubelet.go:2542] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" Dec 03 13:18:29 crc kubenswrapper[5052]: I1203 13:18:29.381029 5052 kuberuntime_manager.go:1027] "Message for Container of pod" containerName="machine-config-daemon" containerStatusID={"Type":"cri-o","ID":"12a8b1f470f6d59d7700e376f2a48eb36363f91f9823a35ce534062c0974c10c"} pod="openshift-machine-config-operator/machine-config-daemon-74bj6" containerMessage="Container machine-config-daemon failed liveness probe, will be restarted" Dec 03 13:18:29 crc kubenswrapper[5052]: I1203 13:18:29.381101 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" containerName="machine-config-daemon" containerID="cri-o://12a8b1f470f6d59d7700e376f2a48eb36363f91f9823a35ce534062c0974c10c" gracePeriod=600 Dec 03 13:18:29 crc kubenswrapper[5052]: E1203 13:18:29.501392 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 13:18:29 crc kubenswrapper[5052]: I1203 13:18:29.568247 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="barbican-kuttl-tests/barbican-keystone-listener-66b47d8764-wbbbf" Dec 03 13:18:29 crc kubenswrapper[5052]: I1203 13:18:29.573253 5052 generic.go:334] "Generic (PLEG): container finished" podID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" containerID="12a8b1f470f6d59d7700e376f2a48eb36363f91f9823a35ce534062c0974c10c" exitCode=0 Dec 03 13:18:29 crc kubenswrapper[5052]: I1203 13:18:29.573291 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" event={"ID":"10c237bc-72e7-4da3-9f2c-b1688395bf6e","Type":"ContainerDied","Data":"12a8b1f470f6d59d7700e376f2a48eb36363f91f9823a35ce534062c0974c10c"} Dec 03 13:18:29 crc kubenswrapper[5052]: I1203 13:18:29.573325 5052 scope.go:117] "RemoveContainer" containerID="059812316dc85d1da8d022c96bdd889e4f16ae597afc7241b51949522378de3a" Dec 03 13:18:29 crc kubenswrapper[5052]: I1203 13:18:29.573816 5052 scope.go:117] "RemoveContainer" containerID="12a8b1f470f6d59d7700e376f2a48eb36363f91f9823a35ce534062c0974c10c" Dec 03 13:18:29 crc kubenswrapper[5052]: E1203 13:18:29.574031 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 13:18:29 crc kubenswrapper[5052]: I1203 13:18:29.614828 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["barbican-kuttl-tests/barbican-keystone-listener-66b47d8764-wbbbf"] Dec 03 13:18:29 crc kubenswrapper[5052]: I1203 13:18:29.620987 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["barbican-kuttl-tests/barbican-keystone-listener-66b47d8764-wbbbf"] Dec 03 13:18:29 crc kubenswrapper[5052]: I1203 13:18:29.680774 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3676d121-81bf-431a-b180-39c950731a2f" path="/var/lib/kubelet/pods/3676d121-81bf-431a-b180-39c950731a2f/volumes" Dec 03 13:18:36 crc kubenswrapper[5052]: I1203 13:18:36.045991 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["barbican-kuttl-tests/keystone-db-sync-mvw4c"] Dec 03 13:18:36 crc kubenswrapper[5052]: I1203 13:18:36.057872 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["barbican-kuttl-tests/keystone-bootstrap-kqb5l"] Dec 03 13:18:36 crc kubenswrapper[5052]: I1203 13:18:36.064743 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["barbican-kuttl-tests/keystone-db-sync-mvw4c"] Dec 03 13:18:36 crc kubenswrapper[5052]: I1203 13:18:36.072167 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["barbican-kuttl-tests/keystone-bootstrap-kqb5l"] Dec 03 13:18:36 crc kubenswrapper[5052]: I1203 13:18:36.089634 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["barbican-kuttl-tests/keystone-67d5bf7497-2t6vk"] Dec 03 13:18:36 crc kubenswrapper[5052]: I1203 13:18:36.089928 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="barbican-kuttl-tests/keystone-67d5bf7497-2t6vk" podUID="c8797321-9192-42b6-a077-6c057a12de0c" containerName="keystone-api" containerID="cri-o://957b6e8f140e175739368e4e1e2c09516ca0ffc214563f78f23a59f8af879c2c" gracePeriod=30 Dec 03 13:18:36 crc kubenswrapper[5052]: I1203 13:18:36.135162 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["barbican-kuttl-tests/keystone44d2-account-delete-5ccw6"] Dec 03 13:18:36 crc kubenswrapper[5052]: E1203 13:18:36.135529 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="1cd1c126-8f6c-4d19-b6ba-f34b0cbd37b1" containerName="barbican-api-log" Dec 03 13:18:36 crc kubenswrapper[5052]: I1203 13:18:36.135659 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="1cd1c126-8f6c-4d19-b6ba-f34b0cbd37b1" containerName="barbican-api-log" Dec 03 13:18:36 crc kubenswrapper[5052]: E1203 13:18:36.135673 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="1cd1c126-8f6c-4d19-b6ba-f34b0cbd37b1" containerName="barbican-api" Dec 03 13:18:36 crc kubenswrapper[5052]: I1203 13:18:36.135680 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="1cd1c126-8f6c-4d19-b6ba-f34b0cbd37b1" containerName="barbican-api" Dec 03 13:18:36 crc kubenswrapper[5052]: E1203 13:18:36.135712 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="2b90d104-bcfc-4c8c-b5a0-16f68747ec01" containerName="barbican-keystone-listener" Dec 03 13:18:36 crc kubenswrapper[5052]: I1203 13:18:36.135719 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="2b90d104-bcfc-4c8c-b5a0-16f68747ec01" containerName="barbican-keystone-listener" Dec 03 13:18:36 crc kubenswrapper[5052]: E1203 13:18:36.135733 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="2b90d104-bcfc-4c8c-b5a0-16f68747ec01" containerName="barbican-keystone-listener-log" Dec 03 13:18:36 crc kubenswrapper[5052]: I1203 13:18:36.135738 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="2b90d104-bcfc-4c8c-b5a0-16f68747ec01" containerName="barbican-keystone-listener-log" Dec 03 13:18:36 crc kubenswrapper[5052]: E1203 13:18:36.135753 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="3676d121-81bf-431a-b180-39c950731a2f" containerName="barbican-keystone-listener" Dec 03 13:18:36 crc kubenswrapper[5052]: I1203 13:18:36.135759 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="3676d121-81bf-431a-b180-39c950731a2f" containerName="barbican-keystone-listener" Dec 03 13:18:36 crc kubenswrapper[5052]: E1203 13:18:36.135767 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="9dfe9635-56c2-4a66-9776-57eb16259e44" containerName="barbican-worker" Dec 03 13:18:36 crc kubenswrapper[5052]: I1203 13:18:36.135772 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="9dfe9635-56c2-4a66-9776-57eb16259e44" containerName="barbican-worker" Dec 03 13:18:36 crc kubenswrapper[5052]: E1203 13:18:36.135783 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="3676d121-81bf-431a-b180-39c950731a2f" containerName="barbican-keystone-listener-log" Dec 03 13:18:36 crc kubenswrapper[5052]: I1203 13:18:36.135789 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="3676d121-81bf-431a-b180-39c950731a2f" containerName="barbican-keystone-listener-log" Dec 03 13:18:36 crc kubenswrapper[5052]: E1203 13:18:36.135796 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="9dfe9635-56c2-4a66-9776-57eb16259e44" containerName="barbican-worker-log" Dec 03 13:18:36 crc kubenswrapper[5052]: I1203 13:18:36.135804 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="9dfe9635-56c2-4a66-9776-57eb16259e44" containerName="barbican-worker-log" Dec 03 13:18:36 crc kubenswrapper[5052]: E1203 13:18:36.135822 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="406d0ab4-b286-41c6-b080-5b3e41f6c3eb" containerName="mariadb-account-delete" Dec 03 13:18:36 crc kubenswrapper[5052]: I1203 13:18:36.135828 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="406d0ab4-b286-41c6-b080-5b3e41f6c3eb" containerName="mariadb-account-delete" Dec 03 13:18:36 crc kubenswrapper[5052]: I1203 13:18:36.135943 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="9dfe9635-56c2-4a66-9776-57eb16259e44" containerName="barbican-worker" Dec 03 13:18:36 crc kubenswrapper[5052]: I1203 13:18:36.135953 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="3676d121-81bf-431a-b180-39c950731a2f" containerName="barbican-keystone-listener" Dec 03 13:18:36 crc kubenswrapper[5052]: I1203 13:18:36.135967 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="2b90d104-bcfc-4c8c-b5a0-16f68747ec01" containerName="barbican-keystone-listener-log" Dec 03 13:18:36 crc kubenswrapper[5052]: I1203 13:18:36.135983 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="2b90d104-bcfc-4c8c-b5a0-16f68747ec01" containerName="barbican-keystone-listener" Dec 03 13:18:36 crc kubenswrapper[5052]: I1203 13:18:36.135995 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="9dfe9635-56c2-4a66-9776-57eb16259e44" containerName="barbican-worker-log" Dec 03 13:18:36 crc kubenswrapper[5052]: I1203 13:18:36.136010 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="1cd1c126-8f6c-4d19-b6ba-f34b0cbd37b1" containerName="barbican-api-log" Dec 03 13:18:36 crc kubenswrapper[5052]: I1203 13:18:36.136020 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="406d0ab4-b286-41c6-b080-5b3e41f6c3eb" containerName="mariadb-account-delete" Dec 03 13:18:36 crc kubenswrapper[5052]: I1203 13:18:36.136029 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="3676d121-81bf-431a-b180-39c950731a2f" containerName="barbican-keystone-listener-log" Dec 03 13:18:36 crc kubenswrapper[5052]: I1203 13:18:36.136040 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="1cd1c126-8f6c-4d19-b6ba-f34b0cbd37b1" containerName="barbican-api" Dec 03 13:18:36 crc kubenswrapper[5052]: I1203 13:18:36.136562 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="barbican-kuttl-tests/keystone44d2-account-delete-5ccw6" Dec 03 13:18:36 crc kubenswrapper[5052]: I1203 13:18:36.158234 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["barbican-kuttl-tests/keystone44d2-account-delete-5ccw6"] Dec 03 13:18:36 crc kubenswrapper[5052]: I1203 13:18:36.211046 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p4n8w\" (UniqueName: \"kubernetes.io/projected/e2e3b100-677d-4b22-af35-1563bbda4b2a-kube-api-access-p4n8w\") pod \"keystone44d2-account-delete-5ccw6\" (UID: \"e2e3b100-677d-4b22-af35-1563bbda4b2a\") " pod="barbican-kuttl-tests/keystone44d2-account-delete-5ccw6" Dec 03 13:18:36 crc kubenswrapper[5052]: I1203 13:18:36.211202 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/e2e3b100-677d-4b22-af35-1563bbda4b2a-operator-scripts\") pod \"keystone44d2-account-delete-5ccw6\" (UID: \"e2e3b100-677d-4b22-af35-1563bbda4b2a\") " pod="barbican-kuttl-tests/keystone44d2-account-delete-5ccw6" Dec 03 13:18:36 crc kubenswrapper[5052]: I1203 13:18:36.313034 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/e2e3b100-677d-4b22-af35-1563bbda4b2a-operator-scripts\") pod \"keystone44d2-account-delete-5ccw6\" (UID: \"e2e3b100-677d-4b22-af35-1563bbda4b2a\") " pod="barbican-kuttl-tests/keystone44d2-account-delete-5ccw6" Dec 03 13:18:36 crc kubenswrapper[5052]: I1203 13:18:36.313126 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-p4n8w\" (UniqueName: \"kubernetes.io/projected/e2e3b100-677d-4b22-af35-1563bbda4b2a-kube-api-access-p4n8w\") pod \"keystone44d2-account-delete-5ccw6\" (UID: \"e2e3b100-677d-4b22-af35-1563bbda4b2a\") " pod="barbican-kuttl-tests/keystone44d2-account-delete-5ccw6" Dec 03 13:18:36 crc kubenswrapper[5052]: I1203 13:18:36.314012 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/e2e3b100-677d-4b22-af35-1563bbda4b2a-operator-scripts\") pod \"keystone44d2-account-delete-5ccw6\" (UID: \"e2e3b100-677d-4b22-af35-1563bbda4b2a\") " pod="barbican-kuttl-tests/keystone44d2-account-delete-5ccw6" Dec 03 13:18:36 crc kubenswrapper[5052]: I1203 13:18:36.341606 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-p4n8w\" (UniqueName: \"kubernetes.io/projected/e2e3b100-677d-4b22-af35-1563bbda4b2a-kube-api-access-p4n8w\") pod \"keystone44d2-account-delete-5ccw6\" (UID: \"e2e3b100-677d-4b22-af35-1563bbda4b2a\") " pod="barbican-kuttl-tests/keystone44d2-account-delete-5ccw6" Dec 03 13:18:36 crc kubenswrapper[5052]: I1203 13:18:36.456782 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="barbican-kuttl-tests/keystone44d2-account-delete-5ccw6" Dec 03 13:18:36 crc kubenswrapper[5052]: I1203 13:18:36.820942 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["barbican-kuttl-tests/openstack-galera-1"] Dec 03 13:18:36 crc kubenswrapper[5052]: I1203 13:18:36.830124 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["barbican-kuttl-tests/openstack-galera-0"] Dec 03 13:18:36 crc kubenswrapper[5052]: I1203 13:18:36.839355 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["barbican-kuttl-tests/openstack-galera-2"] Dec 03 13:18:36 crc kubenswrapper[5052]: I1203 13:18:36.868405 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["barbican-kuttl-tests/keystone44d2-account-delete-5ccw6"] Dec 03 13:18:36 crc kubenswrapper[5052]: W1203 13:18:36.879473 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode2e3b100_677d_4b22_af35_1563bbda4b2a.slice/crio-ec192752d51b7d71ac4100758b379df4fd3183fa8bd20334fb10eee50704c5d0 WatchSource:0}: Error finding container ec192752d51b7d71ac4100758b379df4fd3183fa8bd20334fb10eee50704c5d0: Status 404 returned error can't find the container with id ec192752d51b7d71ac4100758b379df4fd3183fa8bd20334fb10eee50704c5d0 Dec 03 13:18:36 crc kubenswrapper[5052]: I1203 13:18:36.982805 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="barbican-kuttl-tests/openstack-galera-2" podUID="61edc25e-298b-4f43-ad10-5457259b6dc0" containerName="galera" containerID="cri-o://ebb0211d789390961e1a49efd60d7c01d10b4b8a4382ac1b33810fef21c52b38" gracePeriod=30 Dec 03 13:18:37 crc kubenswrapper[5052]: I1203 13:18:37.650028 5052 generic.go:334] "Generic (PLEG): container finished" podID="61edc25e-298b-4f43-ad10-5457259b6dc0" containerID="ebb0211d789390961e1a49efd60d7c01d10b4b8a4382ac1b33810fef21c52b38" exitCode=0 Dec 03 13:18:37 crc kubenswrapper[5052]: I1203 13:18:37.650112 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="barbican-kuttl-tests/openstack-galera-2" event={"ID":"61edc25e-298b-4f43-ad10-5457259b6dc0","Type":"ContainerDied","Data":"ebb0211d789390961e1a49efd60d7c01d10b4b8a4382ac1b33810fef21c52b38"} Dec 03 13:18:37 crc kubenswrapper[5052]: I1203 13:18:37.655377 5052 generic.go:334] "Generic (PLEG): container finished" podID="e2e3b100-677d-4b22-af35-1563bbda4b2a" containerID="c86b8850faaeca689a5438c43c2ccdc76ef556e6e449812a88e50f6c3b6b3d55" exitCode=1 Dec 03 13:18:37 crc kubenswrapper[5052]: I1203 13:18:37.655430 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="barbican-kuttl-tests/keystone44d2-account-delete-5ccw6" event={"ID":"e2e3b100-677d-4b22-af35-1563bbda4b2a","Type":"ContainerDied","Data":"c86b8850faaeca689a5438c43c2ccdc76ef556e6e449812a88e50f6c3b6b3d55"} Dec 03 13:18:37 crc kubenswrapper[5052]: I1203 13:18:37.655459 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="barbican-kuttl-tests/keystone44d2-account-delete-5ccw6" event={"ID":"e2e3b100-677d-4b22-af35-1563bbda4b2a","Type":"ContainerStarted","Data":"ec192752d51b7d71ac4100758b379df4fd3183fa8bd20334fb10eee50704c5d0"} Dec 03 13:18:37 crc kubenswrapper[5052]: I1203 13:18:37.655955 5052 kubelet_pods.go:1007] "Unable to retrieve pull secret, the image pull may not succeed." pod="barbican-kuttl-tests/keystone44d2-account-delete-5ccw6" secret="" err="secret \"galera-openstack-dockercfg-jrdn2\" not found" Dec 03 13:18:37 crc kubenswrapper[5052]: I1203 13:18:37.655990 5052 scope.go:117] "RemoveContainer" containerID="c86b8850faaeca689a5438c43c2ccdc76ef556e6e449812a88e50f6c3b6b3d55" Dec 03 13:18:37 crc kubenswrapper[5052]: I1203 13:18:37.662851 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["barbican-kuttl-tests/memcached-0"] Dec 03 13:18:37 crc kubenswrapper[5052]: I1203 13:18:37.663066 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="barbican-kuttl-tests/memcached-0" podUID="6ccef89f-58bd-4e50-9d8c-d5f076df3505" containerName="memcached" containerID="cri-o://c321459e91c2cd07d0f5294f7e8373b33fa3a1bebfd2f83df4b9a87fc3058238" gracePeriod=30 Dec 03 13:18:37 crc kubenswrapper[5052]: I1203 13:18:37.684072 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="27911c95-b1e3-4d8c-8bca-03d8cc76e76b" path="/var/lib/kubelet/pods/27911c95-b1e3-4d8c-8bca-03d8cc76e76b/volumes" Dec 03 13:18:37 crc kubenswrapper[5052]: I1203 13:18:37.684619 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3d02d92e-1cee-476f-82ab-ceb335c1a247" path="/var/lib/kubelet/pods/3d02d92e-1cee-476f-82ab-ceb335c1a247/volumes" Dec 03 13:18:37 crc kubenswrapper[5052]: E1203 13:18:37.737758 5052 configmap.go:193] Couldn't get configMap barbican-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 13:18:37 crc kubenswrapper[5052]: E1203 13:18:37.737870 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e2e3b100-677d-4b22-af35-1563bbda4b2a-operator-scripts podName:e2e3b100-677d-4b22-af35-1563bbda4b2a nodeName:}" failed. No retries permitted until 2025-12-03 13:18:38.237852168 +0000 UTC m=+7108.826093932 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/e2e3b100-677d-4b22-af35-1563bbda4b2a-operator-scripts") pod "keystone44d2-account-delete-5ccw6" (UID: "e2e3b100-677d-4b22-af35-1563bbda4b2a") : configmap "openstack-scripts" not found Dec 03 13:18:37 crc kubenswrapper[5052]: I1203 13:18:37.916931 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="barbican-kuttl-tests/openstack-galera-2" Dec 03 13:18:38 crc kubenswrapper[5052]: I1203 13:18:38.041688 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/61edc25e-298b-4f43-ad10-5457259b6dc0-config-data-generated\") pod \"61edc25e-298b-4f43-ad10-5457259b6dc0\" (UID: \"61edc25e-298b-4f43-ad10-5457259b6dc0\") " Dec 03 13:18:38 crc kubenswrapper[5052]: I1203 13:18:38.041761 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/61edc25e-298b-4f43-ad10-5457259b6dc0-kolla-config\") pod \"61edc25e-298b-4f43-ad10-5457259b6dc0\" (UID: \"61edc25e-298b-4f43-ad10-5457259b6dc0\") " Dec 03 13:18:38 crc kubenswrapper[5052]: I1203 13:18:38.041859 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"mysql-db\" (UniqueName: \"kubernetes.io/local-volume/local-storage09-crc\") pod \"61edc25e-298b-4f43-ad10-5457259b6dc0\" (UID: \"61edc25e-298b-4f43-ad10-5457259b6dc0\") " Dec 03 13:18:38 crc kubenswrapper[5052]: I1203 13:18:38.041949 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/61edc25e-298b-4f43-ad10-5457259b6dc0-config-data-default\") pod \"61edc25e-298b-4f43-ad10-5457259b6dc0\" (UID: \"61edc25e-298b-4f43-ad10-5457259b6dc0\") " Dec 03 13:18:38 crc kubenswrapper[5052]: I1203 13:18:38.041971 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6hsr6\" (UniqueName: \"kubernetes.io/projected/61edc25e-298b-4f43-ad10-5457259b6dc0-kube-api-access-6hsr6\") pod \"61edc25e-298b-4f43-ad10-5457259b6dc0\" (UID: \"61edc25e-298b-4f43-ad10-5457259b6dc0\") " Dec 03 13:18:38 crc kubenswrapper[5052]: I1203 13:18:38.041996 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/61edc25e-298b-4f43-ad10-5457259b6dc0-operator-scripts\") pod \"61edc25e-298b-4f43-ad10-5457259b6dc0\" (UID: \"61edc25e-298b-4f43-ad10-5457259b6dc0\") " Dec 03 13:18:38 crc kubenswrapper[5052]: I1203 13:18:38.042085 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/61edc25e-298b-4f43-ad10-5457259b6dc0-config-data-generated" (OuterVolumeSpecName: "config-data-generated") pod "61edc25e-298b-4f43-ad10-5457259b6dc0" (UID: "61edc25e-298b-4f43-ad10-5457259b6dc0"). InnerVolumeSpecName "config-data-generated". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 13:18:38 crc kubenswrapper[5052]: I1203 13:18:38.042365 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/61edc25e-298b-4f43-ad10-5457259b6dc0-config-data-generated\") on node \"crc\" DevicePath \"\"" Dec 03 13:18:38 crc kubenswrapper[5052]: I1203 13:18:38.042368 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/61edc25e-298b-4f43-ad10-5457259b6dc0-kolla-config" (OuterVolumeSpecName: "kolla-config") pod "61edc25e-298b-4f43-ad10-5457259b6dc0" (UID: "61edc25e-298b-4f43-ad10-5457259b6dc0"). InnerVolumeSpecName "kolla-config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:18:38 crc kubenswrapper[5052]: I1203 13:18:38.043120 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/61edc25e-298b-4f43-ad10-5457259b6dc0-config-data-default" (OuterVolumeSpecName: "config-data-default") pod "61edc25e-298b-4f43-ad10-5457259b6dc0" (UID: "61edc25e-298b-4f43-ad10-5457259b6dc0"). InnerVolumeSpecName "config-data-default". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:18:38 crc kubenswrapper[5052]: I1203 13:18:38.043263 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/61edc25e-298b-4f43-ad10-5457259b6dc0-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "61edc25e-298b-4f43-ad10-5457259b6dc0" (UID: "61edc25e-298b-4f43-ad10-5457259b6dc0"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:18:38 crc kubenswrapper[5052]: I1203 13:18:38.048113 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/61edc25e-298b-4f43-ad10-5457259b6dc0-kube-api-access-6hsr6" (OuterVolumeSpecName: "kube-api-access-6hsr6") pod "61edc25e-298b-4f43-ad10-5457259b6dc0" (UID: "61edc25e-298b-4f43-ad10-5457259b6dc0"). InnerVolumeSpecName "kube-api-access-6hsr6". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:18:38 crc kubenswrapper[5052]: I1203 13:18:38.053951 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage09-crc" (OuterVolumeSpecName: "mysql-db") pod "61edc25e-298b-4f43-ad10-5457259b6dc0" (UID: "61edc25e-298b-4f43-ad10-5457259b6dc0"). InnerVolumeSpecName "local-storage09-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 13:18:38 crc kubenswrapper[5052]: I1203 13:18:38.104080 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["barbican-kuttl-tests/rabbitmq-server-0"] Dec 03 13:18:38 crc kubenswrapper[5052]: I1203 13:18:38.143815 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage09-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage09-crc\") on node \"crc\" " Dec 03 13:18:38 crc kubenswrapper[5052]: I1203 13:18:38.143881 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/61edc25e-298b-4f43-ad10-5457259b6dc0-config-data-default\") on node \"crc\" DevicePath \"\"" Dec 03 13:18:38 crc kubenswrapper[5052]: I1203 13:18:38.143900 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-6hsr6\" (UniqueName: \"kubernetes.io/projected/61edc25e-298b-4f43-ad10-5457259b6dc0-kube-api-access-6hsr6\") on node \"crc\" DevicePath \"\"" Dec 03 13:18:38 crc kubenswrapper[5052]: I1203 13:18:38.143943 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/61edc25e-298b-4f43-ad10-5457259b6dc0-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 13:18:38 crc kubenswrapper[5052]: I1203 13:18:38.143956 5052 reconciler_common.go:293] "Volume detached for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/61edc25e-298b-4f43-ad10-5457259b6dc0-kolla-config\") on node \"crc\" DevicePath \"\"" Dec 03 13:18:38 crc kubenswrapper[5052]: I1203 13:18:38.157679 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage09-crc" (UniqueName: "kubernetes.io/local-volume/local-storage09-crc") on node "crc" Dec 03 13:18:38 crc kubenswrapper[5052]: I1203 13:18:38.245340 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage09-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage09-crc\") on node \"crc\" DevicePath \"\"" Dec 03 13:18:38 crc kubenswrapper[5052]: E1203 13:18:38.245406 5052 configmap.go:193] Couldn't get configMap barbican-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 13:18:38 crc kubenswrapper[5052]: E1203 13:18:38.245461 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e2e3b100-677d-4b22-af35-1563bbda4b2a-operator-scripts podName:e2e3b100-677d-4b22-af35-1563bbda4b2a nodeName:}" failed. No retries permitted until 2025-12-03 13:18:39.245444424 +0000 UTC m=+7109.833686188 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/e2e3b100-677d-4b22-af35-1563bbda4b2a-operator-scripts") pod "keystone44d2-account-delete-5ccw6" (UID: "e2e3b100-677d-4b22-af35-1563bbda4b2a") : configmap "openstack-scripts" not found Dec 03 13:18:38 crc kubenswrapper[5052]: I1203 13:18:38.490394 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["barbican-kuttl-tests/rabbitmq-server-0"] Dec 03 13:18:38 crc kubenswrapper[5052]: I1203 13:18:38.681550 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="barbican-kuttl-tests/openstack-galera-2" event={"ID":"61edc25e-298b-4f43-ad10-5457259b6dc0","Type":"ContainerDied","Data":"a44708890f8809fdec72dcbfec88ae5c6af9700e88c0572d47003c7efec9f6bc"} Dec 03 13:18:38 crc kubenswrapper[5052]: I1203 13:18:38.681624 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="barbican-kuttl-tests/openstack-galera-2" Dec 03 13:18:38 crc kubenswrapper[5052]: I1203 13:18:38.681678 5052 scope.go:117] "RemoveContainer" containerID="ebb0211d789390961e1a49efd60d7c01d10b4b8a4382ac1b33810fef21c52b38" Dec 03 13:18:38 crc kubenswrapper[5052]: I1203 13:18:38.683498 5052 generic.go:334] "Generic (PLEG): container finished" podID="e2e3b100-677d-4b22-af35-1563bbda4b2a" containerID="800011904d206316b2f7719e7cd9492c8eb1600fb64dbf64a3ee5450ed831771" exitCode=1 Dec 03 13:18:38 crc kubenswrapper[5052]: I1203 13:18:38.683766 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="barbican-kuttl-tests/keystone44d2-account-delete-5ccw6" event={"ID":"e2e3b100-677d-4b22-af35-1563bbda4b2a","Type":"ContainerDied","Data":"800011904d206316b2f7719e7cd9492c8eb1600fb64dbf64a3ee5450ed831771"} Dec 03 13:18:38 crc kubenswrapper[5052]: I1203 13:18:38.685844 5052 kubelet_pods.go:1007] "Unable to retrieve pull secret, the image pull may not succeed." pod="barbican-kuttl-tests/keystone44d2-account-delete-5ccw6" secret="" err="secret \"galera-openstack-dockercfg-jrdn2\" not found" Dec 03 13:18:38 crc kubenswrapper[5052]: I1203 13:18:38.685900 5052 scope.go:117] "RemoveContainer" containerID="800011904d206316b2f7719e7cd9492c8eb1600fb64dbf64a3ee5450ed831771" Dec 03 13:18:38 crc kubenswrapper[5052]: E1203 13:18:38.686587 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"mariadb-account-delete\" with CrashLoopBackOff: \"back-off 10s restarting failed container=mariadb-account-delete pod=keystone44d2-account-delete-5ccw6_barbican-kuttl-tests(e2e3b100-677d-4b22-af35-1563bbda4b2a)\"" pod="barbican-kuttl-tests/keystone44d2-account-delete-5ccw6" podUID="e2e3b100-677d-4b22-af35-1563bbda4b2a" Dec 03 13:18:38 crc kubenswrapper[5052]: I1203 13:18:38.718048 5052 scope.go:117] "RemoveContainer" containerID="dcb879b68854d0520cab523c4e6ee01c37924d1f9f4d49a5514c260629ab703d" Dec 03 13:18:38 crc kubenswrapper[5052]: I1203 13:18:38.723772 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["barbican-kuttl-tests/openstack-galera-2"] Dec 03 13:18:38 crc kubenswrapper[5052]: I1203 13:18:38.724951 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="barbican-kuttl-tests/rabbitmq-server-0" podUID="caf5c2cd-0156-4996-b20a-e5b5b9255ddc" containerName="rabbitmq" containerID="cri-o://121a8279cdacc589caa6d7117b9d018c151fe58e70cc62342f8278ecd590fbce" gracePeriod=604800 Dec 03 13:18:38 crc kubenswrapper[5052]: I1203 13:18:38.729903 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["barbican-kuttl-tests/openstack-galera-2"] Dec 03 13:18:38 crc kubenswrapper[5052]: I1203 13:18:38.750377 5052 scope.go:117] "RemoveContainer" containerID="c86b8850faaeca689a5438c43c2ccdc76ef556e6e449812a88e50f6c3b6b3d55" Dec 03 13:18:39 crc kubenswrapper[5052]: I1203 13:18:38.998660 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="barbican-kuttl-tests/openstack-galera-1" podUID="317b3c95-8146-4310-862e-950f6f4c8f3c" containerName="galera" containerID="cri-o://a2568d7c893958b9c781cf905acf82ee5d1934b31c497db120efcc60318b22b2" gracePeriod=28 Dec 03 13:18:39 crc kubenswrapper[5052]: I1203 13:18:39.216616 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="barbican-kuttl-tests/memcached-0" Dec 03 13:18:39 crc kubenswrapper[5052]: I1203 13:18:39.281012 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/6ccef89f-58bd-4e50-9d8c-d5f076df3505-kolla-config\") pod \"6ccef89f-58bd-4e50-9d8c-d5f076df3505\" (UID: \"6ccef89f-58bd-4e50-9d8c-d5f076df3505\") " Dec 03 13:18:39 crc kubenswrapper[5052]: I1203 13:18:39.281112 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/6ccef89f-58bd-4e50-9d8c-d5f076df3505-config-data\") pod \"6ccef89f-58bd-4e50-9d8c-d5f076df3505\" (UID: \"6ccef89f-58bd-4e50-9d8c-d5f076df3505\") " Dec 03 13:18:39 crc kubenswrapper[5052]: I1203 13:18:39.281274 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vvm2j\" (UniqueName: \"kubernetes.io/projected/6ccef89f-58bd-4e50-9d8c-d5f076df3505-kube-api-access-vvm2j\") pod \"6ccef89f-58bd-4e50-9d8c-d5f076df3505\" (UID: \"6ccef89f-58bd-4e50-9d8c-d5f076df3505\") " Dec 03 13:18:39 crc kubenswrapper[5052]: E1203 13:18:39.281716 5052 configmap.go:193] Couldn't get configMap barbican-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 13:18:39 crc kubenswrapper[5052]: E1203 13:18:39.281799 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e2e3b100-677d-4b22-af35-1563bbda4b2a-operator-scripts podName:e2e3b100-677d-4b22-af35-1563bbda4b2a nodeName:}" failed. No retries permitted until 2025-12-03 13:18:41.281777703 +0000 UTC m=+7111.870019527 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/e2e3b100-677d-4b22-af35-1563bbda4b2a-operator-scripts") pod "keystone44d2-account-delete-5ccw6" (UID: "e2e3b100-677d-4b22-af35-1563bbda4b2a") : configmap "openstack-scripts" not found Dec 03 13:18:39 crc kubenswrapper[5052]: I1203 13:18:39.281914 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6ccef89f-58bd-4e50-9d8c-d5f076df3505-config-data" (OuterVolumeSpecName: "config-data") pod "6ccef89f-58bd-4e50-9d8c-d5f076df3505" (UID: "6ccef89f-58bd-4e50-9d8c-d5f076df3505"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:18:39 crc kubenswrapper[5052]: I1203 13:18:39.282346 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6ccef89f-58bd-4e50-9d8c-d5f076df3505-kolla-config" (OuterVolumeSpecName: "kolla-config") pod "6ccef89f-58bd-4e50-9d8c-d5f076df3505" (UID: "6ccef89f-58bd-4e50-9d8c-d5f076df3505"). InnerVolumeSpecName "kolla-config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:18:39 crc kubenswrapper[5052]: I1203 13:18:39.288125 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6ccef89f-58bd-4e50-9d8c-d5f076df3505-kube-api-access-vvm2j" (OuterVolumeSpecName: "kube-api-access-vvm2j") pod "6ccef89f-58bd-4e50-9d8c-d5f076df3505" (UID: "6ccef89f-58bd-4e50-9d8c-d5f076df3505"). InnerVolumeSpecName "kube-api-access-vvm2j". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:18:39 crc kubenswrapper[5052]: I1203 13:18:39.383183 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-vvm2j\" (UniqueName: \"kubernetes.io/projected/6ccef89f-58bd-4e50-9d8c-d5f076df3505-kube-api-access-vvm2j\") on node \"crc\" DevicePath \"\"" Dec 03 13:18:39 crc kubenswrapper[5052]: I1203 13:18:39.383216 5052 reconciler_common.go:293] "Volume detached for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/6ccef89f-58bd-4e50-9d8c-d5f076df3505-kolla-config\") on node \"crc\" DevicePath \"\"" Dec 03 13:18:39 crc kubenswrapper[5052]: I1203 13:18:39.383227 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/6ccef89f-58bd-4e50-9d8c-d5f076df3505-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 13:18:39 crc kubenswrapper[5052]: I1203 13:18:39.477906 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-operators/barbican-operator-controller-manager-6bfcb8b657-bbhsb"] Dec 03 13:18:39 crc kubenswrapper[5052]: I1203 13:18:39.479409 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-operators/barbican-operator-controller-manager-6bfcb8b657-bbhsb" podUID="dd5d90ba-19d0-40a0-91ef-71f560226194" containerName="manager" containerID="cri-o://d0ecd6e9cce151a40a1c6f4c92dc2927b712f54d3838d50f77721c2a8fe0fbee" gracePeriod=10 Dec 03 13:18:39 crc kubenswrapper[5052]: I1203 13:18:39.687497 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="61edc25e-298b-4f43-ad10-5457259b6dc0" path="/var/lib/kubelet/pods/61edc25e-298b-4f43-ad10-5457259b6dc0/volumes" Dec 03 13:18:39 crc kubenswrapper[5052]: I1203 13:18:39.703333 5052 generic.go:334] "Generic (PLEG): container finished" podID="c8797321-9192-42b6-a077-6c057a12de0c" containerID="957b6e8f140e175739368e4e1e2c09516ca0ffc214563f78f23a59f8af879c2c" exitCode=0 Dec 03 13:18:39 crc kubenswrapper[5052]: I1203 13:18:39.703441 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="barbican-kuttl-tests/keystone-67d5bf7497-2t6vk" event={"ID":"c8797321-9192-42b6-a077-6c057a12de0c","Type":"ContainerDied","Data":"957b6e8f140e175739368e4e1e2c09516ca0ffc214563f78f23a59f8af879c2c"} Dec 03 13:18:39 crc kubenswrapper[5052]: I1203 13:18:39.708059 5052 generic.go:334] "Generic (PLEG): container finished" podID="6ccef89f-58bd-4e50-9d8c-d5f076df3505" containerID="c321459e91c2cd07d0f5294f7e8373b33fa3a1bebfd2f83df4b9a87fc3058238" exitCode=0 Dec 03 13:18:39 crc kubenswrapper[5052]: I1203 13:18:39.708165 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="barbican-kuttl-tests/memcached-0" Dec 03 13:18:39 crc kubenswrapper[5052]: I1203 13:18:39.708203 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="barbican-kuttl-tests/memcached-0" event={"ID":"6ccef89f-58bd-4e50-9d8c-d5f076df3505","Type":"ContainerDied","Data":"c321459e91c2cd07d0f5294f7e8373b33fa3a1bebfd2f83df4b9a87fc3058238"} Dec 03 13:18:39 crc kubenswrapper[5052]: I1203 13:18:39.708261 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="barbican-kuttl-tests/memcached-0" event={"ID":"6ccef89f-58bd-4e50-9d8c-d5f076df3505","Type":"ContainerDied","Data":"7d4e966dd4615fabff47530d8b49c16e88d16eace928aaa59144c2eb3a54c27c"} Dec 03 13:18:39 crc kubenswrapper[5052]: I1203 13:18:39.708282 5052 scope.go:117] "RemoveContainer" containerID="c321459e91c2cd07d0f5294f7e8373b33fa3a1bebfd2f83df4b9a87fc3058238" Dec 03 13:18:39 crc kubenswrapper[5052]: I1203 13:18:39.714547 5052 generic.go:334] "Generic (PLEG): container finished" podID="dd5d90ba-19d0-40a0-91ef-71f560226194" containerID="d0ecd6e9cce151a40a1c6f4c92dc2927b712f54d3838d50f77721c2a8fe0fbee" exitCode=0 Dec 03 13:18:39 crc kubenswrapper[5052]: I1203 13:18:39.714614 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/barbican-operator-controller-manager-6bfcb8b657-bbhsb" event={"ID":"dd5d90ba-19d0-40a0-91ef-71f560226194","Type":"ContainerDied","Data":"d0ecd6e9cce151a40a1c6f4c92dc2927b712f54d3838d50f77721c2a8fe0fbee"} Dec 03 13:18:39 crc kubenswrapper[5052]: I1203 13:18:39.720780 5052 kubelet_pods.go:1007] "Unable to retrieve pull secret, the image pull may not succeed." pod="barbican-kuttl-tests/keystone44d2-account-delete-5ccw6" secret="" err="secret \"galera-openstack-dockercfg-jrdn2\" not found" Dec 03 13:18:39 crc kubenswrapper[5052]: I1203 13:18:39.720843 5052 scope.go:117] "RemoveContainer" containerID="800011904d206316b2f7719e7cd9492c8eb1600fb64dbf64a3ee5450ed831771" Dec 03 13:18:39 crc kubenswrapper[5052]: E1203 13:18:39.721081 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"mariadb-account-delete\" with CrashLoopBackOff: \"back-off 10s restarting failed container=mariadb-account-delete pod=keystone44d2-account-delete-5ccw6_barbican-kuttl-tests(e2e3b100-677d-4b22-af35-1563bbda4b2a)\"" pod="barbican-kuttl-tests/keystone44d2-account-delete-5ccw6" podUID="e2e3b100-677d-4b22-af35-1563bbda4b2a" Dec 03 13:18:39 crc kubenswrapper[5052]: I1203 13:18:39.738505 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["barbican-kuttl-tests/memcached-0"] Dec 03 13:18:39 crc kubenswrapper[5052]: I1203 13:18:39.742566 5052 scope.go:117] "RemoveContainer" containerID="c321459e91c2cd07d0f5294f7e8373b33fa3a1bebfd2f83df4b9a87fc3058238" Dec 03 13:18:39 crc kubenswrapper[5052]: E1203 13:18:39.743375 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c321459e91c2cd07d0f5294f7e8373b33fa3a1bebfd2f83df4b9a87fc3058238\": container with ID starting with c321459e91c2cd07d0f5294f7e8373b33fa3a1bebfd2f83df4b9a87fc3058238 not found: ID does not exist" containerID="c321459e91c2cd07d0f5294f7e8373b33fa3a1bebfd2f83df4b9a87fc3058238" Dec 03 13:18:39 crc kubenswrapper[5052]: I1203 13:18:39.743407 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c321459e91c2cd07d0f5294f7e8373b33fa3a1bebfd2f83df4b9a87fc3058238"} err="failed to get container status \"c321459e91c2cd07d0f5294f7e8373b33fa3a1bebfd2f83df4b9a87fc3058238\": rpc error: code = NotFound desc = could not find container \"c321459e91c2cd07d0f5294f7e8373b33fa3a1bebfd2f83df4b9a87fc3058238\": container with ID starting with c321459e91c2cd07d0f5294f7e8373b33fa3a1bebfd2f83df4b9a87fc3058238 not found: ID does not exist" Dec 03 13:18:39 crc kubenswrapper[5052]: I1203 13:18:39.757212 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["barbican-kuttl-tests/memcached-0"] Dec 03 13:18:39 crc kubenswrapper[5052]: I1203 13:18:39.781279 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-operators/barbican-operator-index-27fzr"] Dec 03 13:18:39 crc kubenswrapper[5052]: I1203 13:18:39.781533 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-operators/barbican-operator-index-27fzr" podUID="42d5fccb-f24d-4707-8167-30eeae73378e" containerName="registry-server" containerID="cri-o://049d108774c46990f3b5856dbd4a31bee08fe0b5f5aa400c1f16208b8fe4f161" gracePeriod=30 Dec 03 13:18:39 crc kubenswrapper[5052]: I1203 13:18:39.834586 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-operators/ee8a7e2925b38108919a6c4a37131c62bef9ed0b2003ea2b2120dc9476kfhs5"] Dec 03 13:18:39 crc kubenswrapper[5052]: I1203 13:18:39.845476 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="barbican-kuttl-tests/rabbitmq-server-0" podUID="caf5c2cd-0156-4996-b20a-e5b5b9255ddc" containerName="rabbitmq" probeResult="failure" output="dial tcp 10.217.1.142:5672: connect: connection refused" Dec 03 13:18:39 crc kubenswrapper[5052]: I1203 13:18:39.850141 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-operators/ee8a7e2925b38108919a6c4a37131c62bef9ed0b2003ea2b2120dc9476kfhs5"] Dec 03 13:18:40 crc kubenswrapper[5052]: I1203 13:18:40.152563 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-operators/barbican-operator-controller-manager-6bfcb8b657-bbhsb" Dec 03 13:18:40 crc kubenswrapper[5052]: I1203 13:18:40.205460 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/dd5d90ba-19d0-40a0-91ef-71f560226194-apiservice-cert\") pod \"dd5d90ba-19d0-40a0-91ef-71f560226194\" (UID: \"dd5d90ba-19d0-40a0-91ef-71f560226194\") " Dec 03 13:18:40 crc kubenswrapper[5052]: I1203 13:18:40.205570 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/dd5d90ba-19d0-40a0-91ef-71f560226194-webhook-cert\") pod \"dd5d90ba-19d0-40a0-91ef-71f560226194\" (UID: \"dd5d90ba-19d0-40a0-91ef-71f560226194\") " Dec 03 13:18:40 crc kubenswrapper[5052]: I1203 13:18:40.205648 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vvk49\" (UniqueName: \"kubernetes.io/projected/dd5d90ba-19d0-40a0-91ef-71f560226194-kube-api-access-vvk49\") pod \"dd5d90ba-19d0-40a0-91ef-71f560226194\" (UID: \"dd5d90ba-19d0-40a0-91ef-71f560226194\") " Dec 03 13:18:40 crc kubenswrapper[5052]: I1203 13:18:40.211569 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dd5d90ba-19d0-40a0-91ef-71f560226194-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "dd5d90ba-19d0-40a0-91ef-71f560226194" (UID: "dd5d90ba-19d0-40a0-91ef-71f560226194"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:18:40 crc kubenswrapper[5052]: I1203 13:18:40.211652 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dd5d90ba-19d0-40a0-91ef-71f560226194-apiservice-cert" (OuterVolumeSpecName: "apiservice-cert") pod "dd5d90ba-19d0-40a0-91ef-71f560226194" (UID: "dd5d90ba-19d0-40a0-91ef-71f560226194"). InnerVolumeSpecName "apiservice-cert". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:18:40 crc kubenswrapper[5052]: I1203 13:18:40.211803 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dd5d90ba-19d0-40a0-91ef-71f560226194-kube-api-access-vvk49" (OuterVolumeSpecName: "kube-api-access-vvk49") pod "dd5d90ba-19d0-40a0-91ef-71f560226194" (UID: "dd5d90ba-19d0-40a0-91ef-71f560226194"). InnerVolumeSpecName "kube-api-access-vvk49". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:18:40 crc kubenswrapper[5052]: I1203 13:18:40.309393 5052 reconciler_common.go:293] "Volume detached for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/dd5d90ba-19d0-40a0-91ef-71f560226194-apiservice-cert\") on node \"crc\" DevicePath \"\"" Dec 03 13:18:40 crc kubenswrapper[5052]: I1203 13:18:40.309456 5052 reconciler_common.go:293] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/dd5d90ba-19d0-40a0-91ef-71f560226194-webhook-cert\") on node \"crc\" DevicePath \"\"" Dec 03 13:18:40 crc kubenswrapper[5052]: I1203 13:18:40.309477 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-vvk49\" (UniqueName: \"kubernetes.io/projected/dd5d90ba-19d0-40a0-91ef-71f560226194-kube-api-access-vvk49\") on node \"crc\" DevicePath \"\"" Dec 03 13:18:40 crc kubenswrapper[5052]: I1203 13:18:40.338084 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-operators/barbican-operator-index-27fzr" Dec 03 13:18:40 crc kubenswrapper[5052]: I1203 13:18:40.410574 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rnndk\" (UniqueName: \"kubernetes.io/projected/42d5fccb-f24d-4707-8167-30eeae73378e-kube-api-access-rnndk\") pod \"42d5fccb-f24d-4707-8167-30eeae73378e\" (UID: \"42d5fccb-f24d-4707-8167-30eeae73378e\") " Dec 03 13:18:40 crc kubenswrapper[5052]: I1203 13:18:40.414979 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/42d5fccb-f24d-4707-8167-30eeae73378e-kube-api-access-rnndk" (OuterVolumeSpecName: "kube-api-access-rnndk") pod "42d5fccb-f24d-4707-8167-30eeae73378e" (UID: "42d5fccb-f24d-4707-8167-30eeae73378e"). InnerVolumeSpecName "kube-api-access-rnndk". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:18:40 crc kubenswrapper[5052]: I1203 13:18:40.424904 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="barbican-kuttl-tests/keystone-67d5bf7497-2t6vk" Dec 03 13:18:40 crc kubenswrapper[5052]: I1203 13:18:40.462125 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="barbican-kuttl-tests/rabbitmq-server-0" Dec 03 13:18:40 crc kubenswrapper[5052]: I1203 13:18:40.511425 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"pod-info\" (UniqueName: \"kubernetes.io/downward-api/caf5c2cd-0156-4996-b20a-e5b5b9255ddc-pod-info\") pod \"caf5c2cd-0156-4996-b20a-e5b5b9255ddc\" (UID: \"caf5c2cd-0156-4996-b20a-e5b5b9255ddc\") " Dec 03 13:18:40 crc kubenswrapper[5052]: I1203 13:18:40.511482 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"rabbitmq-plugins\" (UniqueName: \"kubernetes.io/empty-dir/caf5c2cd-0156-4996-b20a-e5b5b9255ddc-rabbitmq-plugins\") pod \"caf5c2cd-0156-4996-b20a-e5b5b9255ddc\" (UID: \"caf5c2cd-0156-4996-b20a-e5b5b9255ddc\") " Dec 03 13:18:40 crc kubenswrapper[5052]: I1203 13:18:40.511538 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/c8797321-9192-42b6-a077-6c057a12de0c-credential-keys\") pod \"c8797321-9192-42b6-a077-6c057a12de0c\" (UID: \"c8797321-9192-42b6-a077-6c057a12de0c\") " Dec 03 13:18:40 crc kubenswrapper[5052]: I1203 13:18:40.511590 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wzsms\" (UniqueName: \"kubernetes.io/projected/c8797321-9192-42b6-a077-6c057a12de0c-kube-api-access-wzsms\") pod \"c8797321-9192-42b6-a077-6c057a12de0c\" (UID: \"c8797321-9192-42b6-a077-6c057a12de0c\") " Dec 03 13:18:40 crc kubenswrapper[5052]: I1203 13:18:40.511631 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"rabbitmq-erlang-cookie\" (UniqueName: \"kubernetes.io/empty-dir/caf5c2cd-0156-4996-b20a-e5b5b9255ddc-rabbitmq-erlang-cookie\") pod \"caf5c2cd-0156-4996-b20a-e5b5b9255ddc\" (UID: \"caf5c2cd-0156-4996-b20a-e5b5b9255ddc\") " Dec 03 13:18:40 crc kubenswrapper[5052]: I1203 13:18:40.511646 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/c8797321-9192-42b6-a077-6c057a12de0c-config-data\") pod \"c8797321-9192-42b6-a077-6c057a12de0c\" (UID: \"c8797321-9192-42b6-a077-6c057a12de0c\") " Dec 03 13:18:40 crc kubenswrapper[5052]: I1203 13:18:40.511671 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q7mvk\" (UniqueName: \"kubernetes.io/projected/caf5c2cd-0156-4996-b20a-e5b5b9255ddc-kube-api-access-q7mvk\") pod \"caf5c2cd-0156-4996-b20a-e5b5b9255ddc\" (UID: \"caf5c2cd-0156-4996-b20a-e5b5b9255ddc\") " Dec 03 13:18:40 crc kubenswrapper[5052]: I1203 13:18:40.511704 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/c8797321-9192-42b6-a077-6c057a12de0c-fernet-keys\") pod \"c8797321-9192-42b6-a077-6c057a12de0c\" (UID: \"c8797321-9192-42b6-a077-6c057a12de0c\") " Dec 03 13:18:40 crc kubenswrapper[5052]: I1203 13:18:40.511769 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"erlang-cookie-secret\" (UniqueName: \"kubernetes.io/secret/caf5c2cd-0156-4996-b20a-e5b5b9255ddc-erlang-cookie-secret\") pod \"caf5c2cd-0156-4996-b20a-e5b5b9255ddc\" (UID: \"caf5c2cd-0156-4996-b20a-e5b5b9255ddc\") " Dec 03 13:18:40 crc kubenswrapper[5052]: I1203 13:18:40.511790 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"rabbitmq-confd\" (UniqueName: \"kubernetes.io/projected/caf5c2cd-0156-4996-b20a-e5b5b9255ddc-rabbitmq-confd\") pod \"caf5c2cd-0156-4996-b20a-e5b5b9255ddc\" (UID: \"caf5c2cd-0156-4996-b20a-e5b5b9255ddc\") " Dec 03 13:18:40 crc kubenswrapper[5052]: I1203 13:18:40.511951 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"persistence\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-309aff0f-7126-4483-9f08-540649d86541\") pod \"caf5c2cd-0156-4996-b20a-e5b5b9255ddc\" (UID: \"caf5c2cd-0156-4996-b20a-e5b5b9255ddc\") " Dec 03 13:18:40 crc kubenswrapper[5052]: I1203 13:18:40.511986 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/c8797321-9192-42b6-a077-6c057a12de0c-scripts\") pod \"c8797321-9192-42b6-a077-6c057a12de0c\" (UID: \"c8797321-9192-42b6-a077-6c057a12de0c\") " Dec 03 13:18:40 crc kubenswrapper[5052]: I1203 13:18:40.512019 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"plugins-conf\" (UniqueName: \"kubernetes.io/configmap/caf5c2cd-0156-4996-b20a-e5b5b9255ddc-plugins-conf\") pod \"caf5c2cd-0156-4996-b20a-e5b5b9255ddc\" (UID: \"caf5c2cd-0156-4996-b20a-e5b5b9255ddc\") " Dec 03 13:18:40 crc kubenswrapper[5052]: I1203 13:18:40.512352 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-rnndk\" (UniqueName: \"kubernetes.io/projected/42d5fccb-f24d-4707-8167-30eeae73378e-kube-api-access-rnndk\") on node \"crc\" DevicePath \"\"" Dec 03 13:18:40 crc kubenswrapper[5052]: I1203 13:18:40.512372 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/caf5c2cd-0156-4996-b20a-e5b5b9255ddc-rabbitmq-plugins" (OuterVolumeSpecName: "rabbitmq-plugins") pod "caf5c2cd-0156-4996-b20a-e5b5b9255ddc" (UID: "caf5c2cd-0156-4996-b20a-e5b5b9255ddc"). InnerVolumeSpecName "rabbitmq-plugins". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 13:18:40 crc kubenswrapper[5052]: I1203 13:18:40.512899 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/caf5c2cd-0156-4996-b20a-e5b5b9255ddc-plugins-conf" (OuterVolumeSpecName: "plugins-conf") pod "caf5c2cd-0156-4996-b20a-e5b5b9255ddc" (UID: "caf5c2cd-0156-4996-b20a-e5b5b9255ddc"). InnerVolumeSpecName "plugins-conf". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:18:40 crc kubenswrapper[5052]: I1203 13:18:40.513791 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/caf5c2cd-0156-4996-b20a-e5b5b9255ddc-rabbitmq-erlang-cookie" (OuterVolumeSpecName: "rabbitmq-erlang-cookie") pod "caf5c2cd-0156-4996-b20a-e5b5b9255ddc" (UID: "caf5c2cd-0156-4996-b20a-e5b5b9255ddc"). InnerVolumeSpecName "rabbitmq-erlang-cookie". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 13:18:40 crc kubenswrapper[5052]: I1203 13:18:40.515369 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/caf5c2cd-0156-4996-b20a-e5b5b9255ddc-erlang-cookie-secret" (OuterVolumeSpecName: "erlang-cookie-secret") pod "caf5c2cd-0156-4996-b20a-e5b5b9255ddc" (UID: "caf5c2cd-0156-4996-b20a-e5b5b9255ddc"). InnerVolumeSpecName "erlang-cookie-secret". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:18:40 crc kubenswrapper[5052]: I1203 13:18:40.515374 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c8797321-9192-42b6-a077-6c057a12de0c-scripts" (OuterVolumeSpecName: "scripts") pod "c8797321-9192-42b6-a077-6c057a12de0c" (UID: "c8797321-9192-42b6-a077-6c057a12de0c"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:18:40 crc kubenswrapper[5052]: I1203 13:18:40.515710 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/caf5c2cd-0156-4996-b20a-e5b5b9255ddc-pod-info" (OuterVolumeSpecName: "pod-info") pod "caf5c2cd-0156-4996-b20a-e5b5b9255ddc" (UID: "caf5c2cd-0156-4996-b20a-e5b5b9255ddc"). InnerVolumeSpecName "pod-info". PluginName "kubernetes.io/downward-api", VolumeGidValue "" Dec 03 13:18:40 crc kubenswrapper[5052]: I1203 13:18:40.515823 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/caf5c2cd-0156-4996-b20a-e5b5b9255ddc-kube-api-access-q7mvk" (OuterVolumeSpecName: "kube-api-access-q7mvk") pod "caf5c2cd-0156-4996-b20a-e5b5b9255ddc" (UID: "caf5c2cd-0156-4996-b20a-e5b5b9255ddc"). InnerVolumeSpecName "kube-api-access-q7mvk". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:18:40 crc kubenswrapper[5052]: I1203 13:18:40.515852 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c8797321-9192-42b6-a077-6c057a12de0c-fernet-keys" (OuterVolumeSpecName: "fernet-keys") pod "c8797321-9192-42b6-a077-6c057a12de0c" (UID: "c8797321-9192-42b6-a077-6c057a12de0c"). InnerVolumeSpecName "fernet-keys". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:18:40 crc kubenswrapper[5052]: I1203 13:18:40.516254 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c8797321-9192-42b6-a077-6c057a12de0c-kube-api-access-wzsms" (OuterVolumeSpecName: "kube-api-access-wzsms") pod "c8797321-9192-42b6-a077-6c057a12de0c" (UID: "c8797321-9192-42b6-a077-6c057a12de0c"). InnerVolumeSpecName "kube-api-access-wzsms". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:18:40 crc kubenswrapper[5052]: I1203 13:18:40.516376 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c8797321-9192-42b6-a077-6c057a12de0c-credential-keys" (OuterVolumeSpecName: "credential-keys") pod "c8797321-9192-42b6-a077-6c057a12de0c" (UID: "c8797321-9192-42b6-a077-6c057a12de0c"). InnerVolumeSpecName "credential-keys". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:18:40 crc kubenswrapper[5052]: I1203 13:18:40.524696 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-309aff0f-7126-4483-9f08-540649d86541" (OuterVolumeSpecName: "persistence") pod "caf5c2cd-0156-4996-b20a-e5b5b9255ddc" (UID: "caf5c2cd-0156-4996-b20a-e5b5b9255ddc"). InnerVolumeSpecName "pvc-309aff0f-7126-4483-9f08-540649d86541". PluginName "kubernetes.io/csi", VolumeGidValue "" Dec 03 13:18:40 crc kubenswrapper[5052]: I1203 13:18:40.531947 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c8797321-9192-42b6-a077-6c057a12de0c-config-data" (OuterVolumeSpecName: "config-data") pod "c8797321-9192-42b6-a077-6c057a12de0c" (UID: "c8797321-9192-42b6-a077-6c057a12de0c"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:18:40 crc kubenswrapper[5052]: I1203 13:18:40.592838 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/caf5c2cd-0156-4996-b20a-e5b5b9255ddc-rabbitmq-confd" (OuterVolumeSpecName: "rabbitmq-confd") pod "caf5c2cd-0156-4996-b20a-e5b5b9255ddc" (UID: "caf5c2cd-0156-4996-b20a-e5b5b9255ddc"). InnerVolumeSpecName "rabbitmq-confd". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:18:40 crc kubenswrapper[5052]: I1203 13:18:40.613658 5052 reconciler_common.go:293] "Volume detached for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/c8797321-9192-42b6-a077-6c057a12de0c-credential-keys\") on node \"crc\" DevicePath \"\"" Dec 03 13:18:40 crc kubenswrapper[5052]: I1203 13:18:40.613692 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-wzsms\" (UniqueName: \"kubernetes.io/projected/c8797321-9192-42b6-a077-6c057a12de0c-kube-api-access-wzsms\") on node \"crc\" DevicePath \"\"" Dec 03 13:18:40 crc kubenswrapper[5052]: I1203 13:18:40.613706 5052 reconciler_common.go:293] "Volume detached for volume \"rabbitmq-erlang-cookie\" (UniqueName: \"kubernetes.io/empty-dir/caf5c2cd-0156-4996-b20a-e5b5b9255ddc-rabbitmq-erlang-cookie\") on node \"crc\" DevicePath \"\"" Dec 03 13:18:40 crc kubenswrapper[5052]: I1203 13:18:40.613716 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/c8797321-9192-42b6-a077-6c057a12de0c-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 13:18:40 crc kubenswrapper[5052]: I1203 13:18:40.613727 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-q7mvk\" (UniqueName: \"kubernetes.io/projected/caf5c2cd-0156-4996-b20a-e5b5b9255ddc-kube-api-access-q7mvk\") on node \"crc\" DevicePath \"\"" Dec 03 13:18:40 crc kubenswrapper[5052]: I1203 13:18:40.613739 5052 reconciler_common.go:293] "Volume detached for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/c8797321-9192-42b6-a077-6c057a12de0c-fernet-keys\") on node \"crc\" DevicePath \"\"" Dec 03 13:18:40 crc kubenswrapper[5052]: I1203 13:18:40.613750 5052 reconciler_common.go:293] "Volume detached for volume \"erlang-cookie-secret\" (UniqueName: \"kubernetes.io/secret/caf5c2cd-0156-4996-b20a-e5b5b9255ddc-erlang-cookie-secret\") on node \"crc\" DevicePath \"\"" Dec 03 13:18:40 crc kubenswrapper[5052]: I1203 13:18:40.613762 5052 reconciler_common.go:293] "Volume detached for volume \"rabbitmq-confd\" (UniqueName: \"kubernetes.io/projected/caf5c2cd-0156-4996-b20a-e5b5b9255ddc-rabbitmq-confd\") on node \"crc\" DevicePath \"\"" Dec 03 13:18:40 crc kubenswrapper[5052]: I1203 13:18:40.613804 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"pvc-309aff0f-7126-4483-9f08-540649d86541\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-309aff0f-7126-4483-9f08-540649d86541\") on node \"crc\" " Dec 03 13:18:40 crc kubenswrapper[5052]: I1203 13:18:40.613818 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/c8797321-9192-42b6-a077-6c057a12de0c-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 13:18:40 crc kubenswrapper[5052]: I1203 13:18:40.613831 5052 reconciler_common.go:293] "Volume detached for volume \"plugins-conf\" (UniqueName: \"kubernetes.io/configmap/caf5c2cd-0156-4996-b20a-e5b5b9255ddc-plugins-conf\") on node \"crc\" DevicePath \"\"" Dec 03 13:18:40 crc kubenswrapper[5052]: I1203 13:18:40.613844 5052 reconciler_common.go:293] "Volume detached for volume \"pod-info\" (UniqueName: \"kubernetes.io/downward-api/caf5c2cd-0156-4996-b20a-e5b5b9255ddc-pod-info\") on node \"crc\" DevicePath \"\"" Dec 03 13:18:40 crc kubenswrapper[5052]: I1203 13:18:40.613855 5052 reconciler_common.go:293] "Volume detached for volume \"rabbitmq-plugins\" (UniqueName: \"kubernetes.io/empty-dir/caf5c2cd-0156-4996-b20a-e5b5b9255ddc-rabbitmq-plugins\") on node \"crc\" DevicePath \"\"" Dec 03 13:18:40 crc kubenswrapper[5052]: I1203 13:18:40.629858 5052 csi_attacher.go:630] kubernetes.io/csi: attacher.UnmountDevice STAGE_UNSTAGE_VOLUME capability not set. Skipping UnmountDevice... Dec 03 13:18:40 crc kubenswrapper[5052]: I1203 13:18:40.630076 5052 operation_generator.go:917] UnmountDevice succeeded for volume "pvc-309aff0f-7126-4483-9f08-540649d86541" (UniqueName: "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-309aff0f-7126-4483-9f08-540649d86541") on node "crc" Dec 03 13:18:40 crc kubenswrapper[5052]: I1203 13:18:40.715097 5052 reconciler_common.go:293] "Volume detached for volume \"pvc-309aff0f-7126-4483-9f08-540649d86541\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-309aff0f-7126-4483-9f08-540649d86541\") on node \"crc\" DevicePath \"\"" Dec 03 13:18:40 crc kubenswrapper[5052]: I1203 13:18:40.728019 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="barbican-kuttl-tests/keystone-67d5bf7497-2t6vk" event={"ID":"c8797321-9192-42b6-a077-6c057a12de0c","Type":"ContainerDied","Data":"6da85fd53d7a2f0184c4b3d60e6c191c54a86da7433ea676a1b135cbd1d9ec64"} Dec 03 13:18:40 crc kubenswrapper[5052]: I1203 13:18:40.728071 5052 scope.go:117] "RemoveContainer" containerID="957b6e8f140e175739368e4e1e2c09516ca0ffc214563f78f23a59f8af879c2c" Dec 03 13:18:40 crc kubenswrapper[5052]: I1203 13:18:40.728150 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="barbican-kuttl-tests/keystone-67d5bf7497-2t6vk" Dec 03 13:18:40 crc kubenswrapper[5052]: I1203 13:18:40.731995 5052 generic.go:334] "Generic (PLEG): container finished" podID="caf5c2cd-0156-4996-b20a-e5b5b9255ddc" containerID="121a8279cdacc589caa6d7117b9d018c151fe58e70cc62342f8278ecd590fbce" exitCode=0 Dec 03 13:18:40 crc kubenswrapper[5052]: I1203 13:18:40.732031 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="barbican-kuttl-tests/rabbitmq-server-0" event={"ID":"caf5c2cd-0156-4996-b20a-e5b5b9255ddc","Type":"ContainerDied","Data":"121a8279cdacc589caa6d7117b9d018c151fe58e70cc62342f8278ecd590fbce"} Dec 03 13:18:40 crc kubenswrapper[5052]: I1203 13:18:40.732069 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="barbican-kuttl-tests/rabbitmq-server-0" Dec 03 13:18:40 crc kubenswrapper[5052]: I1203 13:18:40.732099 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="barbican-kuttl-tests/rabbitmq-server-0" event={"ID":"caf5c2cd-0156-4996-b20a-e5b5b9255ddc","Type":"ContainerDied","Data":"8f32b7ae471678447346e896dc4c9d3ef5039195cbc8276a4a43e1c308a474ea"} Dec 03 13:18:40 crc kubenswrapper[5052]: I1203 13:18:40.737068 5052 generic.go:334] "Generic (PLEG): container finished" podID="317b3c95-8146-4310-862e-950f6f4c8f3c" containerID="a2568d7c893958b9c781cf905acf82ee5d1934b31c497db120efcc60318b22b2" exitCode=0 Dec 03 13:18:40 crc kubenswrapper[5052]: I1203 13:18:40.737127 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="barbican-kuttl-tests/openstack-galera-1" event={"ID":"317b3c95-8146-4310-862e-950f6f4c8f3c","Type":"ContainerDied","Data":"a2568d7c893958b9c781cf905acf82ee5d1934b31c497db120efcc60318b22b2"} Dec 03 13:18:40 crc kubenswrapper[5052]: I1203 13:18:40.738879 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/barbican-operator-controller-manager-6bfcb8b657-bbhsb" event={"ID":"dd5d90ba-19d0-40a0-91ef-71f560226194","Type":"ContainerDied","Data":"e79f20138944a3f8e14ea26127e60f49917f580d1a38a0deb101c46903325d62"} Dec 03 13:18:40 crc kubenswrapper[5052]: I1203 13:18:40.738900 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-operators/barbican-operator-controller-manager-6bfcb8b657-bbhsb" Dec 03 13:18:40 crc kubenswrapper[5052]: I1203 13:18:40.747437 5052 generic.go:334] "Generic (PLEG): container finished" podID="42d5fccb-f24d-4707-8167-30eeae73378e" containerID="049d108774c46990f3b5856dbd4a31bee08fe0b5f5aa400c1f16208b8fe4f161" exitCode=0 Dec 03 13:18:40 crc kubenswrapper[5052]: I1203 13:18:40.747530 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-operators/barbican-operator-index-27fzr" Dec 03 13:18:40 crc kubenswrapper[5052]: I1203 13:18:40.747549 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/barbican-operator-index-27fzr" event={"ID":"42d5fccb-f24d-4707-8167-30eeae73378e","Type":"ContainerDied","Data":"049d108774c46990f3b5856dbd4a31bee08fe0b5f5aa400c1f16208b8fe4f161"} Dec 03 13:18:40 crc kubenswrapper[5052]: I1203 13:18:40.749899 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/barbican-operator-index-27fzr" event={"ID":"42d5fccb-f24d-4707-8167-30eeae73378e","Type":"ContainerDied","Data":"3ac16256a126a72b395395510628778e7f28a21d38ee102ef12dfc83d2615d96"} Dec 03 13:18:40 crc kubenswrapper[5052]: I1203 13:18:40.824027 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-operators/barbican-operator-controller-manager-6bfcb8b657-bbhsb"] Dec 03 13:18:40 crc kubenswrapper[5052]: I1203 13:18:40.849182 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-operators/barbican-operator-controller-manager-6bfcb8b657-bbhsb"] Dec 03 13:18:40 crc kubenswrapper[5052]: I1203 13:18:40.854900 5052 scope.go:117] "RemoveContainer" containerID="121a8279cdacc589caa6d7117b9d018c151fe58e70cc62342f8278ecd590fbce" Dec 03 13:18:40 crc kubenswrapper[5052]: I1203 13:18:40.877785 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["barbican-kuttl-tests/rabbitmq-server-0"] Dec 03 13:18:40 crc kubenswrapper[5052]: I1203 13:18:40.885535 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["barbican-kuttl-tests/rabbitmq-server-0"] Dec 03 13:18:40 crc kubenswrapper[5052]: I1203 13:18:40.905568 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["barbican-kuttl-tests/keystone-67d5bf7497-2t6vk"] Dec 03 13:18:40 crc kubenswrapper[5052]: I1203 13:18:40.914742 5052 scope.go:117] "RemoveContainer" containerID="78245ad00f303fd209e0f5933535394975e3fb1bf7c4663687be9ecf89b8800a" Dec 03 13:18:40 crc kubenswrapper[5052]: I1203 13:18:40.916341 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["barbican-kuttl-tests/keystone-67d5bf7497-2t6vk"] Dec 03 13:18:40 crc kubenswrapper[5052]: I1203 13:18:40.925803 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-operators/barbican-operator-index-27fzr"] Dec 03 13:18:40 crc kubenswrapper[5052]: I1203 13:18:40.938917 5052 scope.go:117] "RemoveContainer" containerID="121a8279cdacc589caa6d7117b9d018c151fe58e70cc62342f8278ecd590fbce" Dec 03 13:18:40 crc kubenswrapper[5052]: E1203 13:18:40.939466 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"121a8279cdacc589caa6d7117b9d018c151fe58e70cc62342f8278ecd590fbce\": container with ID starting with 121a8279cdacc589caa6d7117b9d018c151fe58e70cc62342f8278ecd590fbce not found: ID does not exist" containerID="121a8279cdacc589caa6d7117b9d018c151fe58e70cc62342f8278ecd590fbce" Dec 03 13:18:40 crc kubenswrapper[5052]: I1203 13:18:40.939498 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"121a8279cdacc589caa6d7117b9d018c151fe58e70cc62342f8278ecd590fbce"} err="failed to get container status \"121a8279cdacc589caa6d7117b9d018c151fe58e70cc62342f8278ecd590fbce\": rpc error: code = NotFound desc = could not find container \"121a8279cdacc589caa6d7117b9d018c151fe58e70cc62342f8278ecd590fbce\": container with ID starting with 121a8279cdacc589caa6d7117b9d018c151fe58e70cc62342f8278ecd590fbce not found: ID does not exist" Dec 03 13:18:40 crc kubenswrapper[5052]: I1203 13:18:40.939544 5052 scope.go:117] "RemoveContainer" containerID="78245ad00f303fd209e0f5933535394975e3fb1bf7c4663687be9ecf89b8800a" Dec 03 13:18:40 crc kubenswrapper[5052]: E1203 13:18:40.940047 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"78245ad00f303fd209e0f5933535394975e3fb1bf7c4663687be9ecf89b8800a\": container with ID starting with 78245ad00f303fd209e0f5933535394975e3fb1bf7c4663687be9ecf89b8800a not found: ID does not exist" containerID="78245ad00f303fd209e0f5933535394975e3fb1bf7c4663687be9ecf89b8800a" Dec 03 13:18:40 crc kubenswrapper[5052]: I1203 13:18:40.940074 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"78245ad00f303fd209e0f5933535394975e3fb1bf7c4663687be9ecf89b8800a"} err="failed to get container status \"78245ad00f303fd209e0f5933535394975e3fb1bf7c4663687be9ecf89b8800a\": rpc error: code = NotFound desc = could not find container \"78245ad00f303fd209e0f5933535394975e3fb1bf7c4663687be9ecf89b8800a\": container with ID starting with 78245ad00f303fd209e0f5933535394975e3fb1bf7c4663687be9ecf89b8800a not found: ID does not exist" Dec 03 13:18:40 crc kubenswrapper[5052]: I1203 13:18:40.940092 5052 scope.go:117] "RemoveContainer" containerID="d0ecd6e9cce151a40a1c6f4c92dc2927b712f54d3838d50f77721c2a8fe0fbee" Dec 03 13:18:40 crc kubenswrapper[5052]: I1203 13:18:40.942054 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-operators/barbican-operator-index-27fzr"] Dec 03 13:18:40 crc kubenswrapper[5052]: I1203 13:18:40.983917 5052 scope.go:117] "RemoveContainer" containerID="049d108774c46990f3b5856dbd4a31bee08fe0b5f5aa400c1f16208b8fe4f161" Dec 03 13:18:41 crc kubenswrapper[5052]: I1203 13:18:41.010370 5052 scope.go:117] "RemoveContainer" containerID="049d108774c46990f3b5856dbd4a31bee08fe0b5f5aa400c1f16208b8fe4f161" Dec 03 13:18:41 crc kubenswrapper[5052]: E1203 13:18:41.011007 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"049d108774c46990f3b5856dbd4a31bee08fe0b5f5aa400c1f16208b8fe4f161\": container with ID starting with 049d108774c46990f3b5856dbd4a31bee08fe0b5f5aa400c1f16208b8fe4f161 not found: ID does not exist" containerID="049d108774c46990f3b5856dbd4a31bee08fe0b5f5aa400c1f16208b8fe4f161" Dec 03 13:18:41 crc kubenswrapper[5052]: I1203 13:18:41.011050 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"049d108774c46990f3b5856dbd4a31bee08fe0b5f5aa400c1f16208b8fe4f161"} err="failed to get container status \"049d108774c46990f3b5856dbd4a31bee08fe0b5f5aa400c1f16208b8fe4f161\": rpc error: code = NotFound desc = could not find container \"049d108774c46990f3b5856dbd4a31bee08fe0b5f5aa400c1f16208b8fe4f161\": container with ID starting with 049d108774c46990f3b5856dbd4a31bee08fe0b5f5aa400c1f16208b8fe4f161 not found: ID does not exist" Dec 03 13:18:41 crc kubenswrapper[5052]: I1203 13:18:41.052400 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="barbican-kuttl-tests/openstack-galera-0" podUID="26dcf610-b5bf-4d7c-86c4-169ef42fb0e7" containerName="galera" containerID="cri-o://23c3dc6c480f2bb3c4438a36768474fa482ce0b0b929335a5d9baf9d865615ae" gracePeriod=26 Dec 03 13:18:41 crc kubenswrapper[5052]: I1203 13:18:41.097191 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="barbican-kuttl-tests/openstack-galera-1" Dec 03 13:18:41 crc kubenswrapper[5052]: I1203 13:18:41.169472 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["barbican-kuttl-tests/keystone44d2-account-delete-5ccw6"] Dec 03 13:18:41 crc kubenswrapper[5052]: I1203 13:18:41.177687 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["barbican-kuttl-tests/keystone-44d2-account-create-update-vczdq"] Dec 03 13:18:41 crc kubenswrapper[5052]: I1203 13:18:41.189853 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["barbican-kuttl-tests/keystone-db-create-rhh8p"] Dec 03 13:18:41 crc kubenswrapper[5052]: I1203 13:18:41.193606 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["barbican-kuttl-tests/keystone-db-create-rhh8p"] Dec 03 13:18:41 crc kubenswrapper[5052]: I1203 13:18:41.198022 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["barbican-kuttl-tests/keystone-44d2-account-create-update-vczdq"] Dec 03 13:18:41 crc kubenswrapper[5052]: I1203 13:18:41.244121 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/317b3c95-8146-4310-862e-950f6f4c8f3c-config-data-generated\") pod \"317b3c95-8146-4310-862e-950f6f4c8f3c\" (UID: \"317b3c95-8146-4310-862e-950f6f4c8f3c\") " Dec 03 13:18:41 crc kubenswrapper[5052]: I1203 13:18:41.244502 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bcrcd\" (UniqueName: \"kubernetes.io/projected/317b3c95-8146-4310-862e-950f6f4c8f3c-kube-api-access-bcrcd\") pod \"317b3c95-8146-4310-862e-950f6f4c8f3c\" (UID: \"317b3c95-8146-4310-862e-950f6f4c8f3c\") " Dec 03 13:18:41 crc kubenswrapper[5052]: I1203 13:18:41.244570 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/317b3c95-8146-4310-862e-950f6f4c8f3c-config-data-default\") pod \"317b3c95-8146-4310-862e-950f6f4c8f3c\" (UID: \"317b3c95-8146-4310-862e-950f6f4c8f3c\") " Dec 03 13:18:41 crc kubenswrapper[5052]: I1203 13:18:41.244648 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/317b3c95-8146-4310-862e-950f6f4c8f3c-kolla-config\") pod \"317b3c95-8146-4310-862e-950f6f4c8f3c\" (UID: \"317b3c95-8146-4310-862e-950f6f4c8f3c\") " Dec 03 13:18:41 crc kubenswrapper[5052]: I1203 13:18:41.244713 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"mysql-db\" (UniqueName: \"kubernetes.io/local-volume/local-storage19-crc\") pod \"317b3c95-8146-4310-862e-950f6f4c8f3c\" (UID: \"317b3c95-8146-4310-862e-950f6f4c8f3c\") " Dec 03 13:18:41 crc kubenswrapper[5052]: I1203 13:18:41.244734 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/317b3c95-8146-4310-862e-950f6f4c8f3c-config-data-generated" (OuterVolumeSpecName: "config-data-generated") pod "317b3c95-8146-4310-862e-950f6f4c8f3c" (UID: "317b3c95-8146-4310-862e-950f6f4c8f3c"). InnerVolumeSpecName "config-data-generated". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 13:18:41 crc kubenswrapper[5052]: I1203 13:18:41.244762 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/317b3c95-8146-4310-862e-950f6f4c8f3c-operator-scripts\") pod \"317b3c95-8146-4310-862e-950f6f4c8f3c\" (UID: \"317b3c95-8146-4310-862e-950f6f4c8f3c\") " Dec 03 13:18:41 crc kubenswrapper[5052]: I1203 13:18:41.245105 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/317b3c95-8146-4310-862e-950f6f4c8f3c-config-data-generated\") on node \"crc\" DevicePath \"\"" Dec 03 13:18:41 crc kubenswrapper[5052]: I1203 13:18:41.245115 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/317b3c95-8146-4310-862e-950f6f4c8f3c-kolla-config" (OuterVolumeSpecName: "kolla-config") pod "317b3c95-8146-4310-862e-950f6f4c8f3c" (UID: "317b3c95-8146-4310-862e-950f6f4c8f3c"). InnerVolumeSpecName "kolla-config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:18:41 crc kubenswrapper[5052]: I1203 13:18:41.245550 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/317b3c95-8146-4310-862e-950f6f4c8f3c-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "317b3c95-8146-4310-862e-950f6f4c8f3c" (UID: "317b3c95-8146-4310-862e-950f6f4c8f3c"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:18:41 crc kubenswrapper[5052]: I1203 13:18:41.245580 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/317b3c95-8146-4310-862e-950f6f4c8f3c-config-data-default" (OuterVolumeSpecName: "config-data-default") pod "317b3c95-8146-4310-862e-950f6f4c8f3c" (UID: "317b3c95-8146-4310-862e-950f6f4c8f3c"). InnerVolumeSpecName "config-data-default". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:18:41 crc kubenswrapper[5052]: I1203 13:18:41.249683 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/317b3c95-8146-4310-862e-950f6f4c8f3c-kube-api-access-bcrcd" (OuterVolumeSpecName: "kube-api-access-bcrcd") pod "317b3c95-8146-4310-862e-950f6f4c8f3c" (UID: "317b3c95-8146-4310-862e-950f6f4c8f3c"). InnerVolumeSpecName "kube-api-access-bcrcd". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:18:41 crc kubenswrapper[5052]: I1203 13:18:41.257699 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage19-crc" (OuterVolumeSpecName: "mysql-db") pod "317b3c95-8146-4310-862e-950f6f4c8f3c" (UID: "317b3c95-8146-4310-862e-950f6f4c8f3c"). InnerVolumeSpecName "local-storage19-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 13:18:41 crc kubenswrapper[5052]: I1203 13:18:41.346507 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-bcrcd\" (UniqueName: \"kubernetes.io/projected/317b3c95-8146-4310-862e-950f6f4c8f3c-kube-api-access-bcrcd\") on node \"crc\" DevicePath \"\"" Dec 03 13:18:41 crc kubenswrapper[5052]: I1203 13:18:41.346572 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/317b3c95-8146-4310-862e-950f6f4c8f3c-config-data-default\") on node \"crc\" DevicePath \"\"" Dec 03 13:18:41 crc kubenswrapper[5052]: I1203 13:18:41.346586 5052 reconciler_common.go:293] "Volume detached for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/317b3c95-8146-4310-862e-950f6f4c8f3c-kolla-config\") on node \"crc\" DevicePath \"\"" Dec 03 13:18:41 crc kubenswrapper[5052]: I1203 13:18:41.346643 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage19-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage19-crc\") on node \"crc\" " Dec 03 13:18:41 crc kubenswrapper[5052]: I1203 13:18:41.346659 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/317b3c95-8146-4310-862e-950f6f4c8f3c-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 13:18:41 crc kubenswrapper[5052]: E1203 13:18:41.346829 5052 configmap.go:193] Couldn't get configMap barbican-kuttl-tests/openstack-scripts: configmap "openstack-scripts" not found Dec 03 13:18:41 crc kubenswrapper[5052]: E1203 13:18:41.346949 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e2e3b100-677d-4b22-af35-1563bbda4b2a-operator-scripts podName:e2e3b100-677d-4b22-af35-1563bbda4b2a nodeName:}" failed. No retries permitted until 2025-12-03 13:18:45.346902393 +0000 UTC m=+7115.935144157 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "operator-scripts" (UniqueName: "kubernetes.io/configmap/e2e3b100-677d-4b22-af35-1563bbda4b2a-operator-scripts") pod "keystone44d2-account-delete-5ccw6" (UID: "e2e3b100-677d-4b22-af35-1563bbda4b2a") : configmap "openstack-scripts" not found Dec 03 13:18:41 crc kubenswrapper[5052]: I1203 13:18:41.371650 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-operators/keystone-operator-controller-manager-5dfbb47f6f-jmkcm"] Dec 03 13:18:41 crc kubenswrapper[5052]: I1203 13:18:41.371851 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-operators/keystone-operator-controller-manager-5dfbb47f6f-jmkcm" podUID="995e0ea0-f1df-4bf5-8d59-1d75f9861688" containerName="manager" containerID="cri-o://d3fab3a005806e264fd0127bc2b5e2af93aa6096567f55cdb094588d5174801b" gracePeriod=10 Dec 03 13:18:41 crc kubenswrapper[5052]: I1203 13:18:41.372032 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage19-crc" (UniqueName: "kubernetes.io/local-volume/local-storage19-crc") on node "crc" Dec 03 13:18:41 crc kubenswrapper[5052]: I1203 13:18:41.448396 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage19-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage19-crc\") on node \"crc\" DevicePath \"\"" Dec 03 13:18:41 crc kubenswrapper[5052]: I1203 13:18:41.502115 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="barbican-kuttl-tests/keystone44d2-account-delete-5ccw6" Dec 03 13:18:41 crc kubenswrapper[5052]: I1203 13:18:41.651847 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/e2e3b100-677d-4b22-af35-1563bbda4b2a-operator-scripts\") pod \"e2e3b100-677d-4b22-af35-1563bbda4b2a\" (UID: \"e2e3b100-677d-4b22-af35-1563bbda4b2a\") " Dec 03 13:18:41 crc kubenswrapper[5052]: I1203 13:18:41.651946 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p4n8w\" (UniqueName: \"kubernetes.io/projected/e2e3b100-677d-4b22-af35-1563bbda4b2a-kube-api-access-p4n8w\") pod \"e2e3b100-677d-4b22-af35-1563bbda4b2a\" (UID: \"e2e3b100-677d-4b22-af35-1563bbda4b2a\") " Dec 03 13:18:41 crc kubenswrapper[5052]: I1203 13:18:41.656621 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e2e3b100-677d-4b22-af35-1563bbda4b2a-kube-api-access-p4n8w" (OuterVolumeSpecName: "kube-api-access-p4n8w") pod "e2e3b100-677d-4b22-af35-1563bbda4b2a" (UID: "e2e3b100-677d-4b22-af35-1563bbda4b2a"). InnerVolumeSpecName "kube-api-access-p4n8w". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:18:41 crc kubenswrapper[5052]: I1203 13:18:41.656701 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e2e3b100-677d-4b22-af35-1563bbda4b2a-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "e2e3b100-677d-4b22-af35-1563bbda4b2a" (UID: "e2e3b100-677d-4b22-af35-1563bbda4b2a"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:18:41 crc kubenswrapper[5052]: I1203 13:18:41.671115 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-operators/keystone-operator-index-ntwzz"] Dec 03 13:18:41 crc kubenswrapper[5052]: I1203 13:18:41.671336 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-operators/keystone-operator-index-ntwzz" podUID="a93ce5ec-f168-4024-9cad-d6886d0421cb" containerName="registry-server" containerID="cri-o://cd6a5b8d53361b1f39a40fa101eac92a94f36bf2c94105882839a39103e9d2b0" gracePeriod=30 Dec 03 13:18:41 crc kubenswrapper[5052]: I1203 13:18:41.676282 5052 scope.go:117] "RemoveContainer" containerID="12a8b1f470f6d59d7700e376f2a48eb36363f91f9823a35ce534062c0974c10c" Dec 03 13:18:41 crc kubenswrapper[5052]: E1203 13:18:41.676573 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 13:18:41 crc kubenswrapper[5052]: I1203 13:18:41.705829 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1ab950dd-0324-4c92-a979-dae6cd28feb4" path="/var/lib/kubelet/pods/1ab950dd-0324-4c92-a979-dae6cd28feb4/volumes" Dec 03 13:18:41 crc kubenswrapper[5052]: I1203 13:18:41.706279 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="42d5fccb-f24d-4707-8167-30eeae73378e" path="/var/lib/kubelet/pods/42d5fccb-f24d-4707-8167-30eeae73378e/volumes" Dec 03 13:18:41 crc kubenswrapper[5052]: I1203 13:18:41.706793 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6ccef89f-58bd-4e50-9d8c-d5f076df3505" path="/var/lib/kubelet/pods/6ccef89f-58bd-4e50-9d8c-d5f076df3505/volumes" Dec 03 13:18:41 crc kubenswrapper[5052]: I1203 13:18:41.707239 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b7eb0ffc-ed97-4e2e-aa46-5959beae46aa" path="/var/lib/kubelet/pods/b7eb0ffc-ed97-4e2e-aa46-5959beae46aa/volumes" Dec 03 13:18:41 crc kubenswrapper[5052]: I1203 13:18:41.708449 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c8797321-9192-42b6-a077-6c057a12de0c" path="/var/lib/kubelet/pods/c8797321-9192-42b6-a077-6c057a12de0c/volumes" Dec 03 13:18:41 crc kubenswrapper[5052]: I1203 13:18:41.709073 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="caf5c2cd-0156-4996-b20a-e5b5b9255ddc" path="/var/lib/kubelet/pods/caf5c2cd-0156-4996-b20a-e5b5b9255ddc/volumes" Dec 03 13:18:41 crc kubenswrapper[5052]: I1203 13:18:41.710115 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dd5d90ba-19d0-40a0-91ef-71f560226194" path="/var/lib/kubelet/pods/dd5d90ba-19d0-40a0-91ef-71f560226194/volumes" Dec 03 13:18:41 crc kubenswrapper[5052]: I1203 13:18:41.710613 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e6a63dba-1ce4-452f-bc4d-7c4e8a34bf7d" path="/var/lib/kubelet/pods/e6a63dba-1ce4-452f-bc4d-7c4e8a34bf7d/volumes" Dec 03 13:18:41 crc kubenswrapper[5052]: I1203 13:18:41.749789 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-operators/49c083020ae5dfe237b73a6c6b807501660a323f061d7879268c43a121bn4xq"] Dec 03 13:18:41 crc kubenswrapper[5052]: I1203 13:18:41.755220 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/e2e3b100-677d-4b22-af35-1563bbda4b2a-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 13:18:41 crc kubenswrapper[5052]: I1203 13:18:41.755253 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-p4n8w\" (UniqueName: \"kubernetes.io/projected/e2e3b100-677d-4b22-af35-1563bbda4b2a-kube-api-access-p4n8w\") on node \"crc\" DevicePath \"\"" Dec 03 13:18:41 crc kubenswrapper[5052]: I1203 13:18:41.771157 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="barbican-kuttl-tests/openstack-galera-1" event={"ID":"317b3c95-8146-4310-862e-950f6f4c8f3c","Type":"ContainerDied","Data":"54fd3935c0852102209f69b1f36d2047f3a38ffa9cf7991666d43b316843072c"} Dec 03 13:18:41 crc kubenswrapper[5052]: I1203 13:18:41.771219 5052 scope.go:117] "RemoveContainer" containerID="a2568d7c893958b9c781cf905acf82ee5d1934b31c497db120efcc60318b22b2" Dec 03 13:18:41 crc kubenswrapper[5052]: I1203 13:18:41.771381 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="barbican-kuttl-tests/openstack-galera-1" Dec 03 13:18:41 crc kubenswrapper[5052]: I1203 13:18:41.772600 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-operators/49c083020ae5dfe237b73a6c6b807501660a323f061d7879268c43a121bn4xq"] Dec 03 13:18:41 crc kubenswrapper[5052]: I1203 13:18:41.782649 5052 generic.go:334] "Generic (PLEG): container finished" podID="995e0ea0-f1df-4bf5-8d59-1d75f9861688" containerID="d3fab3a005806e264fd0127bc2b5e2af93aa6096567f55cdb094588d5174801b" exitCode=0 Dec 03 13:18:41 crc kubenswrapper[5052]: I1203 13:18:41.782713 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/keystone-operator-controller-manager-5dfbb47f6f-jmkcm" event={"ID":"995e0ea0-f1df-4bf5-8d59-1d75f9861688","Type":"ContainerDied","Data":"d3fab3a005806e264fd0127bc2b5e2af93aa6096567f55cdb094588d5174801b"} Dec 03 13:18:41 crc kubenswrapper[5052]: I1203 13:18:41.787116 5052 generic.go:334] "Generic (PLEG): container finished" podID="26dcf610-b5bf-4d7c-86c4-169ef42fb0e7" containerID="23c3dc6c480f2bb3c4438a36768474fa482ce0b0b929335a5d9baf9d865615ae" exitCode=0 Dec 03 13:18:41 crc kubenswrapper[5052]: I1203 13:18:41.787161 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="barbican-kuttl-tests/openstack-galera-0" event={"ID":"26dcf610-b5bf-4d7c-86c4-169ef42fb0e7","Type":"ContainerDied","Data":"23c3dc6c480f2bb3c4438a36768474fa482ce0b0b929335a5d9baf9d865615ae"} Dec 03 13:18:41 crc kubenswrapper[5052]: I1203 13:18:41.789075 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="barbican-kuttl-tests/keystone44d2-account-delete-5ccw6" event={"ID":"e2e3b100-677d-4b22-af35-1563bbda4b2a","Type":"ContainerDied","Data":"ec192752d51b7d71ac4100758b379df4fd3183fa8bd20334fb10eee50704c5d0"} Dec 03 13:18:41 crc kubenswrapper[5052]: I1203 13:18:41.789144 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="barbican-kuttl-tests/keystone44d2-account-delete-5ccw6" Dec 03 13:18:41 crc kubenswrapper[5052]: I1203 13:18:41.862261 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-operators/keystone-operator-controller-manager-5dfbb47f6f-jmkcm" Dec 03 13:18:41 crc kubenswrapper[5052]: I1203 13:18:41.872391 5052 scope.go:117] "RemoveContainer" containerID="420e9c019a410f22173fb0b09e2ee22a65b964d18dc45c57ba6125760b4e7f1d" Dec 03 13:18:41 crc kubenswrapper[5052]: I1203 13:18:41.900756 5052 scope.go:117] "RemoveContainer" containerID="800011904d206316b2f7719e7cd9492c8eb1600fb64dbf64a3ee5450ed831771" Dec 03 13:18:41 crc kubenswrapper[5052]: I1203 13:18:41.906307 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["barbican-kuttl-tests/keystone44d2-account-delete-5ccw6"] Dec 03 13:18:41 crc kubenswrapper[5052]: I1203 13:18:41.915647 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["barbican-kuttl-tests/keystone44d2-account-delete-5ccw6"] Dec 03 13:18:41 crc kubenswrapper[5052]: I1203 13:18:41.923362 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["barbican-kuttl-tests/openstack-galera-1"] Dec 03 13:18:41 crc kubenswrapper[5052]: I1203 13:18:41.935282 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["barbican-kuttl-tests/openstack-galera-1"] Dec 03 13:18:41 crc kubenswrapper[5052]: I1203 13:18:41.958974 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/995e0ea0-f1df-4bf5-8d59-1d75f9861688-webhook-cert\") pod \"995e0ea0-f1df-4bf5-8d59-1d75f9861688\" (UID: \"995e0ea0-f1df-4bf5-8d59-1d75f9861688\") " Dec 03 13:18:41 crc kubenswrapper[5052]: I1203 13:18:41.959449 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/995e0ea0-f1df-4bf5-8d59-1d75f9861688-apiservice-cert\") pod \"995e0ea0-f1df-4bf5-8d59-1d75f9861688\" (UID: \"995e0ea0-f1df-4bf5-8d59-1d75f9861688\") " Dec 03 13:18:41 crc kubenswrapper[5052]: I1203 13:18:41.959501 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zms57\" (UniqueName: \"kubernetes.io/projected/995e0ea0-f1df-4bf5-8d59-1d75f9861688-kube-api-access-zms57\") pod \"995e0ea0-f1df-4bf5-8d59-1d75f9861688\" (UID: \"995e0ea0-f1df-4bf5-8d59-1d75f9861688\") " Dec 03 13:18:41 crc kubenswrapper[5052]: I1203 13:18:41.967108 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/995e0ea0-f1df-4bf5-8d59-1d75f9861688-apiservice-cert" (OuterVolumeSpecName: "apiservice-cert") pod "995e0ea0-f1df-4bf5-8d59-1d75f9861688" (UID: "995e0ea0-f1df-4bf5-8d59-1d75f9861688"). InnerVolumeSpecName "apiservice-cert". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:18:41 crc kubenswrapper[5052]: I1203 13:18:41.967227 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/995e0ea0-f1df-4bf5-8d59-1d75f9861688-kube-api-access-zms57" (OuterVolumeSpecName: "kube-api-access-zms57") pod "995e0ea0-f1df-4bf5-8d59-1d75f9861688" (UID: "995e0ea0-f1df-4bf5-8d59-1d75f9861688"). InnerVolumeSpecName "kube-api-access-zms57". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:18:41 crc kubenswrapper[5052]: I1203 13:18:41.967411 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/995e0ea0-f1df-4bf5-8d59-1d75f9861688-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "995e0ea0-f1df-4bf5-8d59-1d75f9861688" (UID: "995e0ea0-f1df-4bf5-8d59-1d75f9861688"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:18:42 crc kubenswrapper[5052]: I1203 13:18:42.055022 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="barbican-kuttl-tests/openstack-galera-0" Dec 03 13:18:42 crc kubenswrapper[5052]: I1203 13:18:42.061803 5052 reconciler_common.go:293] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/995e0ea0-f1df-4bf5-8d59-1d75f9861688-webhook-cert\") on node \"crc\" DevicePath \"\"" Dec 03 13:18:42 crc kubenswrapper[5052]: I1203 13:18:42.061837 5052 reconciler_common.go:293] "Volume detached for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/995e0ea0-f1df-4bf5-8d59-1d75f9861688-apiservice-cert\") on node \"crc\" DevicePath \"\"" Dec 03 13:18:42 crc kubenswrapper[5052]: I1203 13:18:42.061854 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-zms57\" (UniqueName: \"kubernetes.io/projected/995e0ea0-f1df-4bf5-8d59-1d75f9861688-kube-api-access-zms57\") on node \"crc\" DevicePath \"\"" Dec 03 13:18:42 crc kubenswrapper[5052]: I1203 13:18:42.121941 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-operators/keystone-operator-index-ntwzz" Dec 03 13:18:42 crc kubenswrapper[5052]: I1203 13:18:42.162606 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/26dcf610-b5bf-4d7c-86c4-169ef42fb0e7-config-data-default\") pod \"26dcf610-b5bf-4d7c-86c4-169ef42fb0e7\" (UID: \"26dcf610-b5bf-4d7c-86c4-169ef42fb0e7\") " Dec 03 13:18:42 crc kubenswrapper[5052]: I1203 13:18:42.162706 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/26dcf610-b5bf-4d7c-86c4-169ef42fb0e7-operator-scripts\") pod \"26dcf610-b5bf-4d7c-86c4-169ef42fb0e7\" (UID: \"26dcf610-b5bf-4d7c-86c4-169ef42fb0e7\") " Dec 03 13:18:42 crc kubenswrapper[5052]: I1203 13:18:42.162760 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/26dcf610-b5bf-4d7c-86c4-169ef42fb0e7-kolla-config\") pod \"26dcf610-b5bf-4d7c-86c4-169ef42fb0e7\" (UID: \"26dcf610-b5bf-4d7c-86c4-169ef42fb0e7\") " Dec 03 13:18:42 crc kubenswrapper[5052]: I1203 13:18:42.162782 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"mysql-db\" (UniqueName: \"kubernetes.io/local-volume/local-storage17-crc\") pod \"26dcf610-b5bf-4d7c-86c4-169ef42fb0e7\" (UID: \"26dcf610-b5bf-4d7c-86c4-169ef42fb0e7\") " Dec 03 13:18:42 crc kubenswrapper[5052]: I1203 13:18:42.162826 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2pfv6\" (UniqueName: \"kubernetes.io/projected/26dcf610-b5bf-4d7c-86c4-169ef42fb0e7-kube-api-access-2pfv6\") pod \"26dcf610-b5bf-4d7c-86c4-169ef42fb0e7\" (UID: \"26dcf610-b5bf-4d7c-86c4-169ef42fb0e7\") " Dec 03 13:18:42 crc kubenswrapper[5052]: I1203 13:18:42.162854 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/26dcf610-b5bf-4d7c-86c4-169ef42fb0e7-config-data-generated\") pod \"26dcf610-b5bf-4d7c-86c4-169ef42fb0e7\" (UID: \"26dcf610-b5bf-4d7c-86c4-169ef42fb0e7\") " Dec 03 13:18:42 crc kubenswrapper[5052]: I1203 13:18:42.163468 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/26dcf610-b5bf-4d7c-86c4-169ef42fb0e7-config-data-generated" (OuterVolumeSpecName: "config-data-generated") pod "26dcf610-b5bf-4d7c-86c4-169ef42fb0e7" (UID: "26dcf610-b5bf-4d7c-86c4-169ef42fb0e7"). InnerVolumeSpecName "config-data-generated". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 13:18:42 crc kubenswrapper[5052]: I1203 13:18:42.163855 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/26dcf610-b5bf-4d7c-86c4-169ef42fb0e7-config-data-default" (OuterVolumeSpecName: "config-data-default") pod "26dcf610-b5bf-4d7c-86c4-169ef42fb0e7" (UID: "26dcf610-b5bf-4d7c-86c4-169ef42fb0e7"). InnerVolumeSpecName "config-data-default". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:18:42 crc kubenswrapper[5052]: I1203 13:18:42.164444 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/26dcf610-b5bf-4d7c-86c4-169ef42fb0e7-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "26dcf610-b5bf-4d7c-86c4-169ef42fb0e7" (UID: "26dcf610-b5bf-4d7c-86c4-169ef42fb0e7"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:18:42 crc kubenswrapper[5052]: I1203 13:18:42.164749 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/26dcf610-b5bf-4d7c-86c4-169ef42fb0e7-kolla-config" (OuterVolumeSpecName: "kolla-config") pod "26dcf610-b5bf-4d7c-86c4-169ef42fb0e7" (UID: "26dcf610-b5bf-4d7c-86c4-169ef42fb0e7"). InnerVolumeSpecName "kolla-config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:18:42 crc kubenswrapper[5052]: I1203 13:18:42.169848 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/26dcf610-b5bf-4d7c-86c4-169ef42fb0e7-kube-api-access-2pfv6" (OuterVolumeSpecName: "kube-api-access-2pfv6") pod "26dcf610-b5bf-4d7c-86c4-169ef42fb0e7" (UID: "26dcf610-b5bf-4d7c-86c4-169ef42fb0e7"). InnerVolumeSpecName "kube-api-access-2pfv6". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:18:42 crc kubenswrapper[5052]: I1203 13:18:42.173036 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage17-crc" (OuterVolumeSpecName: "mysql-db") pod "26dcf610-b5bf-4d7c-86c4-169ef42fb0e7" (UID: "26dcf610-b5bf-4d7c-86c4-169ef42fb0e7"). InnerVolumeSpecName "local-storage17-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 13:18:42 crc kubenswrapper[5052]: I1203 13:18:42.263662 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pq87d\" (UniqueName: \"kubernetes.io/projected/a93ce5ec-f168-4024-9cad-d6886d0421cb-kube-api-access-pq87d\") pod \"a93ce5ec-f168-4024-9cad-d6886d0421cb\" (UID: \"a93ce5ec-f168-4024-9cad-d6886d0421cb\") " Dec 03 13:18:42 crc kubenswrapper[5052]: I1203 13:18:42.264618 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-2pfv6\" (UniqueName: \"kubernetes.io/projected/26dcf610-b5bf-4d7c-86c4-169ef42fb0e7-kube-api-access-2pfv6\") on node \"crc\" DevicePath \"\"" Dec 03 13:18:42 crc kubenswrapper[5052]: I1203 13:18:42.264648 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/26dcf610-b5bf-4d7c-86c4-169ef42fb0e7-config-data-generated\") on node \"crc\" DevicePath \"\"" Dec 03 13:18:42 crc kubenswrapper[5052]: I1203 13:18:42.264663 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/26dcf610-b5bf-4d7c-86c4-169ef42fb0e7-config-data-default\") on node \"crc\" DevicePath \"\"" Dec 03 13:18:42 crc kubenswrapper[5052]: I1203 13:18:42.264676 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/26dcf610-b5bf-4d7c-86c4-169ef42fb0e7-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 13:18:42 crc kubenswrapper[5052]: I1203 13:18:42.264688 5052 reconciler_common.go:293] "Volume detached for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/26dcf610-b5bf-4d7c-86c4-169ef42fb0e7-kolla-config\") on node \"crc\" DevicePath \"\"" Dec 03 13:18:42 crc kubenswrapper[5052]: I1203 13:18:42.264720 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage17-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage17-crc\") on node \"crc\" " Dec 03 13:18:42 crc kubenswrapper[5052]: I1203 13:18:42.266389 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a93ce5ec-f168-4024-9cad-d6886d0421cb-kube-api-access-pq87d" (OuterVolumeSpecName: "kube-api-access-pq87d") pod "a93ce5ec-f168-4024-9cad-d6886d0421cb" (UID: "a93ce5ec-f168-4024-9cad-d6886d0421cb"). InnerVolumeSpecName "kube-api-access-pq87d". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:18:42 crc kubenswrapper[5052]: I1203 13:18:42.278737 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage17-crc" (UniqueName: "kubernetes.io/local-volume/local-storage17-crc") on node "crc" Dec 03 13:18:42 crc kubenswrapper[5052]: I1203 13:18:42.366010 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage17-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage17-crc\") on node \"crc\" DevicePath \"\"" Dec 03 13:18:42 crc kubenswrapper[5052]: I1203 13:18:42.366048 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-pq87d\" (UniqueName: \"kubernetes.io/projected/a93ce5ec-f168-4024-9cad-d6886d0421cb-kube-api-access-pq87d\") on node \"crc\" DevicePath \"\"" Dec 03 13:18:42 crc kubenswrapper[5052]: I1203 13:18:42.804044 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-operators/keystone-operator-controller-manager-5dfbb47f6f-jmkcm" Dec 03 13:18:42 crc kubenswrapper[5052]: I1203 13:18:42.804037 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/keystone-operator-controller-manager-5dfbb47f6f-jmkcm" event={"ID":"995e0ea0-f1df-4bf5-8d59-1d75f9861688","Type":"ContainerDied","Data":"a66cc6a12dc29455332fa1c0b95ceb0eae29e48533b2008cb042482d96cdf018"} Dec 03 13:18:42 crc kubenswrapper[5052]: I1203 13:18:42.804104 5052 scope.go:117] "RemoveContainer" containerID="d3fab3a005806e264fd0127bc2b5e2af93aa6096567f55cdb094588d5174801b" Dec 03 13:18:42 crc kubenswrapper[5052]: I1203 13:18:42.805838 5052 generic.go:334] "Generic (PLEG): container finished" podID="a93ce5ec-f168-4024-9cad-d6886d0421cb" containerID="cd6a5b8d53361b1f39a40fa101eac92a94f36bf2c94105882839a39103e9d2b0" exitCode=0 Dec 03 13:18:42 crc kubenswrapper[5052]: I1203 13:18:42.805890 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-operators/keystone-operator-index-ntwzz" Dec 03 13:18:42 crc kubenswrapper[5052]: I1203 13:18:42.805895 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/keystone-operator-index-ntwzz" event={"ID":"a93ce5ec-f168-4024-9cad-d6886d0421cb","Type":"ContainerDied","Data":"cd6a5b8d53361b1f39a40fa101eac92a94f36bf2c94105882839a39103e9d2b0"} Dec 03 13:18:42 crc kubenswrapper[5052]: I1203 13:18:42.805935 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/keystone-operator-index-ntwzz" event={"ID":"a93ce5ec-f168-4024-9cad-d6886d0421cb","Type":"ContainerDied","Data":"3528dd53de4f3b1651bfc571314f2648d37776f662e925ca6f7b537e9fa1d6a0"} Dec 03 13:18:42 crc kubenswrapper[5052]: I1203 13:18:42.807648 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="barbican-kuttl-tests/openstack-galera-0" Dec 03 13:18:42 crc kubenswrapper[5052]: I1203 13:18:42.807665 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="barbican-kuttl-tests/openstack-galera-0" event={"ID":"26dcf610-b5bf-4d7c-86c4-169ef42fb0e7","Type":"ContainerDied","Data":"f051410aa43ae943f86e6ccbeda741b77e989c4b8f69e52a08b4c0c4df4e790b"} Dec 03 13:18:42 crc kubenswrapper[5052]: I1203 13:18:42.821633 5052 scope.go:117] "RemoveContainer" containerID="cd6a5b8d53361b1f39a40fa101eac92a94f36bf2c94105882839a39103e9d2b0" Dec 03 13:18:42 crc kubenswrapper[5052]: I1203 13:18:42.845295 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-operators/keystone-operator-controller-manager-5dfbb47f6f-jmkcm"] Dec 03 13:18:42 crc kubenswrapper[5052]: I1203 13:18:42.846942 5052 scope.go:117] "RemoveContainer" containerID="cd6a5b8d53361b1f39a40fa101eac92a94f36bf2c94105882839a39103e9d2b0" Dec 03 13:18:42 crc kubenswrapper[5052]: E1203 13:18:42.851711 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cd6a5b8d53361b1f39a40fa101eac92a94f36bf2c94105882839a39103e9d2b0\": container with ID starting with cd6a5b8d53361b1f39a40fa101eac92a94f36bf2c94105882839a39103e9d2b0 not found: ID does not exist" containerID="cd6a5b8d53361b1f39a40fa101eac92a94f36bf2c94105882839a39103e9d2b0" Dec 03 13:18:42 crc kubenswrapper[5052]: I1203 13:18:42.851772 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cd6a5b8d53361b1f39a40fa101eac92a94f36bf2c94105882839a39103e9d2b0"} err="failed to get container status \"cd6a5b8d53361b1f39a40fa101eac92a94f36bf2c94105882839a39103e9d2b0\": rpc error: code = NotFound desc = could not find container \"cd6a5b8d53361b1f39a40fa101eac92a94f36bf2c94105882839a39103e9d2b0\": container with ID starting with cd6a5b8d53361b1f39a40fa101eac92a94f36bf2c94105882839a39103e9d2b0 not found: ID does not exist" Dec 03 13:18:42 crc kubenswrapper[5052]: I1203 13:18:42.851801 5052 scope.go:117] "RemoveContainer" containerID="23c3dc6c480f2bb3c4438a36768474fa482ce0b0b929335a5d9baf9d865615ae" Dec 03 13:18:42 crc kubenswrapper[5052]: I1203 13:18:42.855012 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-operators/keystone-operator-controller-manager-5dfbb47f6f-jmkcm"] Dec 03 13:18:42 crc kubenswrapper[5052]: I1203 13:18:42.861979 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-operators/keystone-operator-index-ntwzz"] Dec 03 13:18:42 crc kubenswrapper[5052]: I1203 13:18:42.867175 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-operators/keystone-operator-index-ntwzz"] Dec 03 13:18:42 crc kubenswrapper[5052]: I1203 13:18:42.871977 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["barbican-kuttl-tests/openstack-galera-0"] Dec 03 13:18:42 crc kubenswrapper[5052]: I1203 13:18:42.873849 5052 scope.go:117] "RemoveContainer" containerID="3722162af199440174441c0d30eadecfe9d0f035426d3ccb3ca353885c8ba797" Dec 03 13:18:42 crc kubenswrapper[5052]: I1203 13:18:42.876933 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["barbican-kuttl-tests/openstack-galera-0"] Dec 03 13:18:43 crc kubenswrapper[5052]: I1203 13:18:43.633130 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-operators/rabbitmq-cluster-operator-779fc9694b-2vdrl"] Dec 03 13:18:43 crc kubenswrapper[5052]: I1203 13:18:43.633330 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-operators/rabbitmq-cluster-operator-779fc9694b-2vdrl" podUID="9bef396c-7ed5-46b5-b57d-f4cd7df5c872" containerName="operator" containerID="cri-o://8a0e209c01fd705fa336c0ddc5fe285ff3876ea628c51c4d228852bb47da3169" gracePeriod=10 Dec 03 13:18:43 crc kubenswrapper[5052]: I1203 13:18:43.683395 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="26dcf610-b5bf-4d7c-86c4-169ef42fb0e7" path="/var/lib/kubelet/pods/26dcf610-b5bf-4d7c-86c4-169ef42fb0e7/volumes" Dec 03 13:18:43 crc kubenswrapper[5052]: I1203 13:18:43.684683 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="317b3c95-8146-4310-862e-950f6f4c8f3c" path="/var/lib/kubelet/pods/317b3c95-8146-4310-862e-950f6f4c8f3c/volumes" Dec 03 13:18:43 crc kubenswrapper[5052]: I1203 13:18:43.685225 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4903ac07-9e1a-41f9-9d62-23ebfb24f0a2" path="/var/lib/kubelet/pods/4903ac07-9e1a-41f9-9d62-23ebfb24f0a2/volumes" Dec 03 13:18:43 crc kubenswrapper[5052]: I1203 13:18:43.686324 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="995e0ea0-f1df-4bf5-8d59-1d75f9861688" path="/var/lib/kubelet/pods/995e0ea0-f1df-4bf5-8d59-1d75f9861688/volumes" Dec 03 13:18:43 crc kubenswrapper[5052]: I1203 13:18:43.686771 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a93ce5ec-f168-4024-9cad-d6886d0421cb" path="/var/lib/kubelet/pods/a93ce5ec-f168-4024-9cad-d6886d0421cb/volumes" Dec 03 13:18:43 crc kubenswrapper[5052]: I1203 13:18:43.687196 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e2e3b100-677d-4b22-af35-1563bbda4b2a" path="/var/lib/kubelet/pods/e2e3b100-677d-4b22-af35-1563bbda4b2a/volumes" Dec 03 13:18:43 crc kubenswrapper[5052]: I1203 13:18:43.823134 5052 generic.go:334] "Generic (PLEG): container finished" podID="9bef396c-7ed5-46b5-b57d-f4cd7df5c872" containerID="8a0e209c01fd705fa336c0ddc5fe285ff3876ea628c51c4d228852bb47da3169" exitCode=0 Dec 03 13:18:43 crc kubenswrapper[5052]: I1203 13:18:43.823184 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/rabbitmq-cluster-operator-779fc9694b-2vdrl" event={"ID":"9bef396c-7ed5-46b5-b57d-f4cd7df5c872","Type":"ContainerDied","Data":"8a0e209c01fd705fa336c0ddc5fe285ff3876ea628c51c4d228852bb47da3169"} Dec 03 13:18:43 crc kubenswrapper[5052]: I1203 13:18:43.935426 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-operators/rabbitmq-cluster-operator-index-49tjq"] Dec 03 13:18:43 crc kubenswrapper[5052]: I1203 13:18:43.939655 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-operators/rabbitmq-cluster-operator-index-49tjq" podUID="0abdde9a-bebd-4ece-9fb6-0cf23ba9a11a" containerName="registry-server" containerID="cri-o://de5964c1bb69ca89e1f609bdd5b508f4c04326b4ef02da9f9683bc08a72d7ef3" gracePeriod=30 Dec 03 13:18:43 crc kubenswrapper[5052]: I1203 13:18:43.980270 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-operators/9704761d240e56fb98655ffd81084895b33a73ec711f4dcdef0450e590xlm56"] Dec 03 13:18:43 crc kubenswrapper[5052]: I1203 13:18:43.988920 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-operators/9704761d240e56fb98655ffd81084895b33a73ec711f4dcdef0450e590xlm56"] Dec 03 13:18:44 crc kubenswrapper[5052]: I1203 13:18:44.167098 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-operators/rabbitmq-cluster-operator-779fc9694b-2vdrl" Dec 03 13:18:44 crc kubenswrapper[5052]: E1203 13:18:44.190661 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of de5964c1bb69ca89e1f609bdd5b508f4c04326b4ef02da9f9683bc08a72d7ef3 is running failed: container process not found" containerID="de5964c1bb69ca89e1f609bdd5b508f4c04326b4ef02da9f9683bc08a72d7ef3" cmd=["grpc_health_probe","-addr=:50051"] Dec 03 13:18:44 crc kubenswrapper[5052]: E1203 13:18:44.191962 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of de5964c1bb69ca89e1f609bdd5b508f4c04326b4ef02da9f9683bc08a72d7ef3 is running failed: container process not found" containerID="de5964c1bb69ca89e1f609bdd5b508f4c04326b4ef02da9f9683bc08a72d7ef3" cmd=["grpc_health_probe","-addr=:50051"] Dec 03 13:18:44 crc kubenswrapper[5052]: E1203 13:18:44.192195 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of de5964c1bb69ca89e1f609bdd5b508f4c04326b4ef02da9f9683bc08a72d7ef3 is running failed: container process not found" containerID="de5964c1bb69ca89e1f609bdd5b508f4c04326b4ef02da9f9683bc08a72d7ef3" cmd=["grpc_health_probe","-addr=:50051"] Dec 03 13:18:44 crc kubenswrapper[5052]: E1203 13:18:44.192216 5052 prober.go:104] "Probe errored" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of de5964c1bb69ca89e1f609bdd5b508f4c04326b4ef02da9f9683bc08a72d7ef3 is running failed: container process not found" probeType="Readiness" pod="openstack-operators/rabbitmq-cluster-operator-index-49tjq" podUID="0abdde9a-bebd-4ece-9fb6-0cf23ba9a11a" containerName="registry-server" Dec 03 13:18:44 crc kubenswrapper[5052]: I1203 13:18:44.298472 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-operators/rabbitmq-cluster-operator-index-49tjq" Dec 03 13:18:44 crc kubenswrapper[5052]: I1203 13:18:44.303075 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-569g9\" (UniqueName: \"kubernetes.io/projected/9bef396c-7ed5-46b5-b57d-f4cd7df5c872-kube-api-access-569g9\") pod \"9bef396c-7ed5-46b5-b57d-f4cd7df5c872\" (UID: \"9bef396c-7ed5-46b5-b57d-f4cd7df5c872\") " Dec 03 13:18:44 crc kubenswrapper[5052]: I1203 13:18:44.310796 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9bef396c-7ed5-46b5-b57d-f4cd7df5c872-kube-api-access-569g9" (OuterVolumeSpecName: "kube-api-access-569g9") pod "9bef396c-7ed5-46b5-b57d-f4cd7df5c872" (UID: "9bef396c-7ed5-46b5-b57d-f4cd7df5c872"). InnerVolumeSpecName "kube-api-access-569g9". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:18:44 crc kubenswrapper[5052]: I1203 13:18:44.404499 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g54gh\" (UniqueName: \"kubernetes.io/projected/0abdde9a-bebd-4ece-9fb6-0cf23ba9a11a-kube-api-access-g54gh\") pod \"0abdde9a-bebd-4ece-9fb6-0cf23ba9a11a\" (UID: \"0abdde9a-bebd-4ece-9fb6-0cf23ba9a11a\") " Dec 03 13:18:44 crc kubenswrapper[5052]: I1203 13:18:44.405139 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-569g9\" (UniqueName: \"kubernetes.io/projected/9bef396c-7ed5-46b5-b57d-f4cd7df5c872-kube-api-access-569g9\") on node \"crc\" DevicePath \"\"" Dec 03 13:18:44 crc kubenswrapper[5052]: I1203 13:18:44.408125 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0abdde9a-bebd-4ece-9fb6-0cf23ba9a11a-kube-api-access-g54gh" (OuterVolumeSpecName: "kube-api-access-g54gh") pod "0abdde9a-bebd-4ece-9fb6-0cf23ba9a11a" (UID: "0abdde9a-bebd-4ece-9fb6-0cf23ba9a11a"). InnerVolumeSpecName "kube-api-access-g54gh". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:18:44 crc kubenswrapper[5052]: I1203 13:18:44.507046 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-g54gh\" (UniqueName: \"kubernetes.io/projected/0abdde9a-bebd-4ece-9fb6-0cf23ba9a11a-kube-api-access-g54gh\") on node \"crc\" DevicePath \"\"" Dec 03 13:18:44 crc kubenswrapper[5052]: I1203 13:18:44.833201 5052 generic.go:334] "Generic (PLEG): container finished" podID="0abdde9a-bebd-4ece-9fb6-0cf23ba9a11a" containerID="de5964c1bb69ca89e1f609bdd5b508f4c04326b4ef02da9f9683bc08a72d7ef3" exitCode=0 Dec 03 13:18:44 crc kubenswrapper[5052]: I1203 13:18:44.833265 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/rabbitmq-cluster-operator-index-49tjq" event={"ID":"0abdde9a-bebd-4ece-9fb6-0cf23ba9a11a","Type":"ContainerDied","Data":"de5964c1bb69ca89e1f609bdd5b508f4c04326b4ef02da9f9683bc08a72d7ef3"} Dec 03 13:18:44 crc kubenswrapper[5052]: I1203 13:18:44.833271 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-operators/rabbitmq-cluster-operator-index-49tjq" Dec 03 13:18:44 crc kubenswrapper[5052]: I1203 13:18:44.833290 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/rabbitmq-cluster-operator-index-49tjq" event={"ID":"0abdde9a-bebd-4ece-9fb6-0cf23ba9a11a","Type":"ContainerDied","Data":"95345410d6c07e824f9c799487121152ca64915b0aac177dbdc7807c478d3e3c"} Dec 03 13:18:44 crc kubenswrapper[5052]: I1203 13:18:44.833305 5052 scope.go:117] "RemoveContainer" containerID="de5964c1bb69ca89e1f609bdd5b508f4c04326b4ef02da9f9683bc08a72d7ef3" Dec 03 13:18:44 crc kubenswrapper[5052]: I1203 13:18:44.835816 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/rabbitmq-cluster-operator-779fc9694b-2vdrl" event={"ID":"9bef396c-7ed5-46b5-b57d-f4cd7df5c872","Type":"ContainerDied","Data":"44620fbcf5c8209527c08cfab1c00bbdbef1fc4ba9bc3c160aecbaf8ddce18fb"} Dec 03 13:18:44 crc kubenswrapper[5052]: I1203 13:18:44.835919 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-operators/rabbitmq-cluster-operator-779fc9694b-2vdrl" Dec 03 13:18:44 crc kubenswrapper[5052]: I1203 13:18:44.852891 5052 scope.go:117] "RemoveContainer" containerID="de5964c1bb69ca89e1f609bdd5b508f4c04326b4ef02da9f9683bc08a72d7ef3" Dec 03 13:18:44 crc kubenswrapper[5052]: E1203 13:18:44.853380 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"de5964c1bb69ca89e1f609bdd5b508f4c04326b4ef02da9f9683bc08a72d7ef3\": container with ID starting with de5964c1bb69ca89e1f609bdd5b508f4c04326b4ef02da9f9683bc08a72d7ef3 not found: ID does not exist" containerID="de5964c1bb69ca89e1f609bdd5b508f4c04326b4ef02da9f9683bc08a72d7ef3" Dec 03 13:18:44 crc kubenswrapper[5052]: I1203 13:18:44.853412 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"de5964c1bb69ca89e1f609bdd5b508f4c04326b4ef02da9f9683bc08a72d7ef3"} err="failed to get container status \"de5964c1bb69ca89e1f609bdd5b508f4c04326b4ef02da9f9683bc08a72d7ef3\": rpc error: code = NotFound desc = could not find container \"de5964c1bb69ca89e1f609bdd5b508f4c04326b4ef02da9f9683bc08a72d7ef3\": container with ID starting with de5964c1bb69ca89e1f609bdd5b508f4c04326b4ef02da9f9683bc08a72d7ef3 not found: ID does not exist" Dec 03 13:18:44 crc kubenswrapper[5052]: I1203 13:18:44.853433 5052 scope.go:117] "RemoveContainer" containerID="8a0e209c01fd705fa336c0ddc5fe285ff3876ea628c51c4d228852bb47da3169" Dec 03 13:18:44 crc kubenswrapper[5052]: I1203 13:18:44.870371 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-operators/rabbitmq-cluster-operator-index-49tjq"] Dec 03 13:18:44 crc kubenswrapper[5052]: I1203 13:18:44.877565 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-operators/rabbitmq-cluster-operator-index-49tjq"] Dec 03 13:18:44 crc kubenswrapper[5052]: I1203 13:18:44.888056 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-operators/rabbitmq-cluster-operator-779fc9694b-2vdrl"] Dec 03 13:18:44 crc kubenswrapper[5052]: I1203 13:18:44.893324 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-operators/rabbitmq-cluster-operator-779fc9694b-2vdrl"] Dec 03 13:18:45 crc kubenswrapper[5052]: I1203 13:18:45.681241 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0abdde9a-bebd-4ece-9fb6-0cf23ba9a11a" path="/var/lib/kubelet/pods/0abdde9a-bebd-4ece-9fb6-0cf23ba9a11a/volumes" Dec 03 13:18:45 crc kubenswrapper[5052]: I1203 13:18:45.681830 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5701f773-f548-4133-867d-dca86f9bed69" path="/var/lib/kubelet/pods/5701f773-f548-4133-867d-dca86f9bed69/volumes" Dec 03 13:18:45 crc kubenswrapper[5052]: I1203 13:18:45.682371 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9bef396c-7ed5-46b5-b57d-f4cd7df5c872" path="/var/lib/kubelet/pods/9bef396c-7ed5-46b5-b57d-f4cd7df5c872/volumes" Dec 03 13:18:47 crc kubenswrapper[5052]: I1203 13:18:47.126156 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-operators/infra-operator-controller-manager-58dbf89689-bnlrt"] Dec 03 13:18:47 crc kubenswrapper[5052]: I1203 13:18:47.126718 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-operators/infra-operator-controller-manager-58dbf89689-bnlrt" podUID="5f441756-e059-4db0-943e-eafb2d48e668" containerName="kube-rbac-proxy" containerID="cri-o://b25eafbee44eb19fc61d68c549818ea4ebc9cc256b939273da70c1836a1a40b2" gracePeriod=10 Dec 03 13:18:47 crc kubenswrapper[5052]: I1203 13:18:47.126875 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-operators/infra-operator-controller-manager-58dbf89689-bnlrt" podUID="5f441756-e059-4db0-943e-eafb2d48e668" containerName="manager" containerID="cri-o://75dfc7b94c9bc6c8c51aeb2593ca52f48a46a8ea64b1eb3540d4ca2bc6dd492e" gracePeriod=10 Dec 03 13:18:47 crc kubenswrapper[5052]: I1203 13:18:47.427666 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-operators/infra-operator-index-mhrwx"] Dec 03 13:18:47 crc kubenswrapper[5052]: I1203 13:18:47.427909 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-operators/infra-operator-index-mhrwx" podUID="fd9c88e5-c614-42c3-a252-e8a0600210a7" containerName="registry-server" containerID="cri-o://dca482a55ee1a10de30189b059d80aa7ebe62669d226bfa13c6621dbaf4686a5" gracePeriod=30 Dec 03 13:18:47 crc kubenswrapper[5052]: I1203 13:18:47.468144 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-operators/5d473c3169f40b179d14921c90af2c8546b7b757fe551b7dba7d903f5dc2k4q"] Dec 03 13:18:47 crc kubenswrapper[5052]: I1203 13:18:47.475048 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-operators/5d473c3169f40b179d14921c90af2c8546b7b757fe551b7dba7d903f5dc2k4q"] Dec 03 13:18:47 crc kubenswrapper[5052]: I1203 13:18:47.661617 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-operators/infra-operator-controller-manager-58dbf89689-bnlrt" Dec 03 13:18:47 crc kubenswrapper[5052]: I1203 13:18:47.684387 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d5ee6ef6-8fe3-450b-ac6d-e0db3e409540" path="/var/lib/kubelet/pods/d5ee6ef6-8fe3-450b-ac6d-e0db3e409540/volumes" Dec 03 13:18:47 crc kubenswrapper[5052]: I1203 13:18:47.748134 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5f441756-e059-4db0-943e-eafb2d48e668-webhook-cert\") pod \"5f441756-e059-4db0-943e-eafb2d48e668\" (UID: \"5f441756-e059-4db0-943e-eafb2d48e668\") " Dec 03 13:18:47 crc kubenswrapper[5052]: I1203 13:18:47.748225 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/5f441756-e059-4db0-943e-eafb2d48e668-apiservice-cert\") pod \"5f441756-e059-4db0-943e-eafb2d48e668\" (UID: \"5f441756-e059-4db0-943e-eafb2d48e668\") " Dec 03 13:18:47 crc kubenswrapper[5052]: I1203 13:18:47.748260 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-66lhs\" (UniqueName: \"kubernetes.io/projected/5f441756-e059-4db0-943e-eafb2d48e668-kube-api-access-66lhs\") pod \"5f441756-e059-4db0-943e-eafb2d48e668\" (UID: \"5f441756-e059-4db0-943e-eafb2d48e668\") " Dec 03 13:18:47 crc kubenswrapper[5052]: I1203 13:18:47.756569 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5f441756-e059-4db0-943e-eafb2d48e668-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "5f441756-e059-4db0-943e-eafb2d48e668" (UID: "5f441756-e059-4db0-943e-eafb2d48e668"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:18:47 crc kubenswrapper[5052]: I1203 13:18:47.756637 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5f441756-e059-4db0-943e-eafb2d48e668-kube-api-access-66lhs" (OuterVolumeSpecName: "kube-api-access-66lhs") pod "5f441756-e059-4db0-943e-eafb2d48e668" (UID: "5f441756-e059-4db0-943e-eafb2d48e668"). InnerVolumeSpecName "kube-api-access-66lhs". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:18:47 crc kubenswrapper[5052]: I1203 13:18:47.760675 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5f441756-e059-4db0-943e-eafb2d48e668-apiservice-cert" (OuterVolumeSpecName: "apiservice-cert") pod "5f441756-e059-4db0-943e-eafb2d48e668" (UID: "5f441756-e059-4db0-943e-eafb2d48e668"). InnerVolumeSpecName "apiservice-cert". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:18:47 crc kubenswrapper[5052]: I1203 13:18:47.849882 5052 reconciler_common.go:293] "Volume detached for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/5f441756-e059-4db0-943e-eafb2d48e668-apiservice-cert\") on node \"crc\" DevicePath \"\"" Dec 03 13:18:47 crc kubenswrapper[5052]: I1203 13:18:47.849934 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-66lhs\" (UniqueName: \"kubernetes.io/projected/5f441756-e059-4db0-943e-eafb2d48e668-kube-api-access-66lhs\") on node \"crc\" DevicePath \"\"" Dec 03 13:18:47 crc kubenswrapper[5052]: I1203 13:18:47.849950 5052 reconciler_common.go:293] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5f441756-e059-4db0-943e-eafb2d48e668-webhook-cert\") on node \"crc\" DevicePath \"\"" Dec 03 13:18:47 crc kubenswrapper[5052]: I1203 13:18:47.869811 5052 generic.go:334] "Generic (PLEG): container finished" podID="fd9c88e5-c614-42c3-a252-e8a0600210a7" containerID="dca482a55ee1a10de30189b059d80aa7ebe62669d226bfa13c6621dbaf4686a5" exitCode=0 Dec 03 13:18:47 crc kubenswrapper[5052]: I1203 13:18:47.869904 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/infra-operator-index-mhrwx" event={"ID":"fd9c88e5-c614-42c3-a252-e8a0600210a7","Type":"ContainerDied","Data":"dca482a55ee1a10de30189b059d80aa7ebe62669d226bfa13c6621dbaf4686a5"} Dec 03 13:18:47 crc kubenswrapper[5052]: I1203 13:18:47.874028 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-operators/infra-operator-index-mhrwx" Dec 03 13:18:47 crc kubenswrapper[5052]: I1203 13:18:47.874263 5052 generic.go:334] "Generic (PLEG): container finished" podID="5f441756-e059-4db0-943e-eafb2d48e668" containerID="b25eafbee44eb19fc61d68c549818ea4ebc9cc256b939273da70c1836a1a40b2" exitCode=0 Dec 03 13:18:47 crc kubenswrapper[5052]: I1203 13:18:47.874299 5052 generic.go:334] "Generic (PLEG): container finished" podID="5f441756-e059-4db0-943e-eafb2d48e668" containerID="75dfc7b94c9bc6c8c51aeb2593ca52f48a46a8ea64b1eb3540d4ca2bc6dd492e" exitCode=0 Dec 03 13:18:47 crc kubenswrapper[5052]: I1203 13:18:47.874324 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-operators/infra-operator-controller-manager-58dbf89689-bnlrt" Dec 03 13:18:47 crc kubenswrapper[5052]: I1203 13:18:47.874327 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/infra-operator-controller-manager-58dbf89689-bnlrt" event={"ID":"5f441756-e059-4db0-943e-eafb2d48e668","Type":"ContainerDied","Data":"b25eafbee44eb19fc61d68c549818ea4ebc9cc256b939273da70c1836a1a40b2"} Dec 03 13:18:47 crc kubenswrapper[5052]: I1203 13:18:47.874681 5052 scope.go:117] "RemoveContainer" containerID="b25eafbee44eb19fc61d68c549818ea4ebc9cc256b939273da70c1836a1a40b2" Dec 03 13:18:47 crc kubenswrapper[5052]: I1203 13:18:47.874650 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/infra-operator-controller-manager-58dbf89689-bnlrt" event={"ID":"5f441756-e059-4db0-943e-eafb2d48e668","Type":"ContainerDied","Data":"75dfc7b94c9bc6c8c51aeb2593ca52f48a46a8ea64b1eb3540d4ca2bc6dd492e"} Dec 03 13:18:47 crc kubenswrapper[5052]: I1203 13:18:47.874969 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/infra-operator-controller-manager-58dbf89689-bnlrt" event={"ID":"5f441756-e059-4db0-943e-eafb2d48e668","Type":"ContainerDied","Data":"d289f264cd7289e4d87674831df826d8924fdb666dad9b363be016783449ff13"} Dec 03 13:18:47 crc kubenswrapper[5052]: I1203 13:18:47.901641 5052 scope.go:117] "RemoveContainer" containerID="75dfc7b94c9bc6c8c51aeb2593ca52f48a46a8ea64b1eb3540d4ca2bc6dd492e" Dec 03 13:18:47 crc kubenswrapper[5052]: I1203 13:18:47.908959 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-operators/infra-operator-controller-manager-58dbf89689-bnlrt"] Dec 03 13:18:47 crc kubenswrapper[5052]: I1203 13:18:47.918010 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-operators/infra-operator-controller-manager-58dbf89689-bnlrt"] Dec 03 13:18:47 crc kubenswrapper[5052]: I1203 13:18:47.949061 5052 scope.go:117] "RemoveContainer" containerID="b25eafbee44eb19fc61d68c549818ea4ebc9cc256b939273da70c1836a1a40b2" Dec 03 13:18:47 crc kubenswrapper[5052]: E1203 13:18:47.949763 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b25eafbee44eb19fc61d68c549818ea4ebc9cc256b939273da70c1836a1a40b2\": container with ID starting with b25eafbee44eb19fc61d68c549818ea4ebc9cc256b939273da70c1836a1a40b2 not found: ID does not exist" containerID="b25eafbee44eb19fc61d68c549818ea4ebc9cc256b939273da70c1836a1a40b2" Dec 03 13:18:47 crc kubenswrapper[5052]: I1203 13:18:47.949841 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b25eafbee44eb19fc61d68c549818ea4ebc9cc256b939273da70c1836a1a40b2"} err="failed to get container status \"b25eafbee44eb19fc61d68c549818ea4ebc9cc256b939273da70c1836a1a40b2\": rpc error: code = NotFound desc = could not find container \"b25eafbee44eb19fc61d68c549818ea4ebc9cc256b939273da70c1836a1a40b2\": container with ID starting with b25eafbee44eb19fc61d68c549818ea4ebc9cc256b939273da70c1836a1a40b2 not found: ID does not exist" Dec 03 13:18:47 crc kubenswrapper[5052]: I1203 13:18:47.949871 5052 scope.go:117] "RemoveContainer" containerID="75dfc7b94c9bc6c8c51aeb2593ca52f48a46a8ea64b1eb3540d4ca2bc6dd492e" Dec 03 13:18:47 crc kubenswrapper[5052]: E1203 13:18:47.952015 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"75dfc7b94c9bc6c8c51aeb2593ca52f48a46a8ea64b1eb3540d4ca2bc6dd492e\": container with ID starting with 75dfc7b94c9bc6c8c51aeb2593ca52f48a46a8ea64b1eb3540d4ca2bc6dd492e not found: ID does not exist" containerID="75dfc7b94c9bc6c8c51aeb2593ca52f48a46a8ea64b1eb3540d4ca2bc6dd492e" Dec 03 13:18:47 crc kubenswrapper[5052]: I1203 13:18:47.952048 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"75dfc7b94c9bc6c8c51aeb2593ca52f48a46a8ea64b1eb3540d4ca2bc6dd492e"} err="failed to get container status \"75dfc7b94c9bc6c8c51aeb2593ca52f48a46a8ea64b1eb3540d4ca2bc6dd492e\": rpc error: code = NotFound desc = could not find container \"75dfc7b94c9bc6c8c51aeb2593ca52f48a46a8ea64b1eb3540d4ca2bc6dd492e\": container with ID starting with 75dfc7b94c9bc6c8c51aeb2593ca52f48a46a8ea64b1eb3540d4ca2bc6dd492e not found: ID does not exist" Dec 03 13:18:47 crc kubenswrapper[5052]: I1203 13:18:47.952069 5052 scope.go:117] "RemoveContainer" containerID="b25eafbee44eb19fc61d68c549818ea4ebc9cc256b939273da70c1836a1a40b2" Dec 03 13:18:47 crc kubenswrapper[5052]: I1203 13:18:47.954109 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z9snr\" (UniqueName: \"kubernetes.io/projected/fd9c88e5-c614-42c3-a252-e8a0600210a7-kube-api-access-z9snr\") pod \"fd9c88e5-c614-42c3-a252-e8a0600210a7\" (UID: \"fd9c88e5-c614-42c3-a252-e8a0600210a7\") " Dec 03 13:18:47 crc kubenswrapper[5052]: I1203 13:18:47.955889 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b25eafbee44eb19fc61d68c549818ea4ebc9cc256b939273da70c1836a1a40b2"} err="failed to get container status \"b25eafbee44eb19fc61d68c549818ea4ebc9cc256b939273da70c1836a1a40b2\": rpc error: code = NotFound desc = could not find container \"b25eafbee44eb19fc61d68c549818ea4ebc9cc256b939273da70c1836a1a40b2\": container with ID starting with b25eafbee44eb19fc61d68c549818ea4ebc9cc256b939273da70c1836a1a40b2 not found: ID does not exist" Dec 03 13:18:47 crc kubenswrapper[5052]: I1203 13:18:47.956619 5052 scope.go:117] "RemoveContainer" containerID="75dfc7b94c9bc6c8c51aeb2593ca52f48a46a8ea64b1eb3540d4ca2bc6dd492e" Dec 03 13:18:47 crc kubenswrapper[5052]: I1203 13:18:47.957280 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fd9c88e5-c614-42c3-a252-e8a0600210a7-kube-api-access-z9snr" (OuterVolumeSpecName: "kube-api-access-z9snr") pod "fd9c88e5-c614-42c3-a252-e8a0600210a7" (UID: "fd9c88e5-c614-42c3-a252-e8a0600210a7"). InnerVolumeSpecName "kube-api-access-z9snr". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:18:47 crc kubenswrapper[5052]: I1203 13:18:47.962699 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"75dfc7b94c9bc6c8c51aeb2593ca52f48a46a8ea64b1eb3540d4ca2bc6dd492e"} err="failed to get container status \"75dfc7b94c9bc6c8c51aeb2593ca52f48a46a8ea64b1eb3540d4ca2bc6dd492e\": rpc error: code = NotFound desc = could not find container \"75dfc7b94c9bc6c8c51aeb2593ca52f48a46a8ea64b1eb3540d4ca2bc6dd492e\": container with ID starting with 75dfc7b94c9bc6c8c51aeb2593ca52f48a46a8ea64b1eb3540d4ca2bc6dd492e not found: ID does not exist" Dec 03 13:18:48 crc kubenswrapper[5052]: I1203 13:18:48.055456 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-z9snr\" (UniqueName: \"kubernetes.io/projected/fd9c88e5-c614-42c3-a252-e8a0600210a7-kube-api-access-z9snr\") on node \"crc\" DevicePath \"\"" Dec 03 13:18:48 crc kubenswrapper[5052]: I1203 13:18:48.884258 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/infra-operator-index-mhrwx" event={"ID":"fd9c88e5-c614-42c3-a252-e8a0600210a7","Type":"ContainerDied","Data":"37cc2172abe1f801bb661c98a931f03d0b8e87e027ed2c31fed08f704ff318e5"} Dec 03 13:18:48 crc kubenswrapper[5052]: I1203 13:18:48.884646 5052 scope.go:117] "RemoveContainer" containerID="dca482a55ee1a10de30189b059d80aa7ebe62669d226bfa13c6621dbaf4686a5" Dec 03 13:18:48 crc kubenswrapper[5052]: I1203 13:18:48.884304 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-operators/infra-operator-index-mhrwx" Dec 03 13:18:48 crc kubenswrapper[5052]: I1203 13:18:48.915414 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-operators/infra-operator-index-mhrwx"] Dec 03 13:18:48 crc kubenswrapper[5052]: I1203 13:18:48.920324 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-operators/infra-operator-index-mhrwx"] Dec 03 13:18:49 crc kubenswrapper[5052]: I1203 13:18:49.018626 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-operators/mariadb-operator-controller-manager-7595db4997-skrx7"] Dec 03 13:18:49 crc kubenswrapper[5052]: I1203 13:18:49.018895 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-operators/mariadb-operator-controller-manager-7595db4997-skrx7" podUID="568b316e-dee2-4a2a-967b-730a5f03de09" containerName="manager" containerID="cri-o://151cb2f59f27897dd419d8881ee3dbbaf6cfece15d5d067e9de2130be92940a7" gracePeriod=10 Dec 03 13:18:49 crc kubenswrapper[5052]: I1203 13:18:49.340467 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-operators/mariadb-operator-index-jbcj6"] Dec 03 13:18:49 crc kubenswrapper[5052]: I1203 13:18:49.340698 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-operators/mariadb-operator-index-jbcj6" podUID="f7ca8ea0-92d6-4274-96f7-8297170883fc" containerName="registry-server" containerID="cri-o://cb36d6b31264eb861a891fb9a33d6ae809a253d3427f4bda6dfaa99d86e000b1" gracePeriod=30 Dec 03 13:18:49 crc kubenswrapper[5052]: I1203 13:18:49.382979 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-operators/55a3886cc1ed42812df4eab61c7a6033dc924d195539e8545c8f175f61g5jxg"] Dec 03 13:18:49 crc kubenswrapper[5052]: I1203 13:18:49.398669 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-operators/55a3886cc1ed42812df4eab61c7a6033dc924d195539e8545c8f175f61g5jxg"] Dec 03 13:18:49 crc kubenswrapper[5052]: I1203 13:18:49.526921 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-operators/mariadb-operator-controller-manager-7595db4997-skrx7" Dec 03 13:18:49 crc kubenswrapper[5052]: I1203 13:18:49.682977 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5f441756-e059-4db0-943e-eafb2d48e668" path="/var/lib/kubelet/pods/5f441756-e059-4db0-943e-eafb2d48e668/volumes" Dec 03 13:18:49 crc kubenswrapper[5052]: I1203 13:18:49.683672 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c8e42363-9fc0-49b1-a9d4-44a4eaa2f093" path="/var/lib/kubelet/pods/c8e42363-9fc0-49b1-a9d4-44a4eaa2f093/volumes" Dec 03 13:18:49 crc kubenswrapper[5052]: I1203 13:18:49.684655 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fd9c88e5-c614-42c3-a252-e8a0600210a7" path="/var/lib/kubelet/pods/fd9c88e5-c614-42c3-a252-e8a0600210a7/volumes" Dec 03 13:18:49 crc kubenswrapper[5052]: I1203 13:18:49.687393 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/568b316e-dee2-4a2a-967b-730a5f03de09-apiservice-cert\") pod \"568b316e-dee2-4a2a-967b-730a5f03de09\" (UID: \"568b316e-dee2-4a2a-967b-730a5f03de09\") " Dec 03 13:18:49 crc kubenswrapper[5052]: I1203 13:18:49.687494 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/568b316e-dee2-4a2a-967b-730a5f03de09-webhook-cert\") pod \"568b316e-dee2-4a2a-967b-730a5f03de09\" (UID: \"568b316e-dee2-4a2a-967b-730a5f03de09\") " Dec 03 13:18:49 crc kubenswrapper[5052]: I1203 13:18:49.687552 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7cvgp\" (UniqueName: \"kubernetes.io/projected/568b316e-dee2-4a2a-967b-730a5f03de09-kube-api-access-7cvgp\") pod \"568b316e-dee2-4a2a-967b-730a5f03de09\" (UID: \"568b316e-dee2-4a2a-967b-730a5f03de09\") " Dec 03 13:18:49 crc kubenswrapper[5052]: I1203 13:18:49.693631 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/568b316e-dee2-4a2a-967b-730a5f03de09-apiservice-cert" (OuterVolumeSpecName: "apiservice-cert") pod "568b316e-dee2-4a2a-967b-730a5f03de09" (UID: "568b316e-dee2-4a2a-967b-730a5f03de09"). InnerVolumeSpecName "apiservice-cert". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:18:49 crc kubenswrapper[5052]: I1203 13:18:49.694301 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/568b316e-dee2-4a2a-967b-730a5f03de09-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "568b316e-dee2-4a2a-967b-730a5f03de09" (UID: "568b316e-dee2-4a2a-967b-730a5f03de09"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:18:49 crc kubenswrapper[5052]: I1203 13:18:49.695220 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/568b316e-dee2-4a2a-967b-730a5f03de09-kube-api-access-7cvgp" (OuterVolumeSpecName: "kube-api-access-7cvgp") pod "568b316e-dee2-4a2a-967b-730a5f03de09" (UID: "568b316e-dee2-4a2a-967b-730a5f03de09"). InnerVolumeSpecName "kube-api-access-7cvgp". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:18:49 crc kubenswrapper[5052]: I1203 13:18:49.791438 5052 reconciler_common.go:293] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/568b316e-dee2-4a2a-967b-730a5f03de09-webhook-cert\") on node \"crc\" DevicePath \"\"" Dec 03 13:18:49 crc kubenswrapper[5052]: I1203 13:18:49.791489 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-7cvgp\" (UniqueName: \"kubernetes.io/projected/568b316e-dee2-4a2a-967b-730a5f03de09-kube-api-access-7cvgp\") on node \"crc\" DevicePath \"\"" Dec 03 13:18:49 crc kubenswrapper[5052]: I1203 13:18:49.791506 5052 reconciler_common.go:293] "Volume detached for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/568b316e-dee2-4a2a-967b-730a5f03de09-apiservice-cert\") on node \"crc\" DevicePath \"\"" Dec 03 13:18:49 crc kubenswrapper[5052]: I1203 13:18:49.874305 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-operators/mariadb-operator-index-jbcj6" Dec 03 13:18:49 crc kubenswrapper[5052]: I1203 13:18:49.893094 5052 generic.go:334] "Generic (PLEG): container finished" podID="568b316e-dee2-4a2a-967b-730a5f03de09" containerID="151cb2f59f27897dd419d8881ee3dbbaf6cfece15d5d067e9de2130be92940a7" exitCode=0 Dec 03 13:18:49 crc kubenswrapper[5052]: I1203 13:18:49.893168 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-operators/mariadb-operator-controller-manager-7595db4997-skrx7" Dec 03 13:18:49 crc kubenswrapper[5052]: I1203 13:18:49.893180 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/mariadb-operator-controller-manager-7595db4997-skrx7" event={"ID":"568b316e-dee2-4a2a-967b-730a5f03de09","Type":"ContainerDied","Data":"151cb2f59f27897dd419d8881ee3dbbaf6cfece15d5d067e9de2130be92940a7"} Dec 03 13:18:49 crc kubenswrapper[5052]: I1203 13:18:49.893232 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/mariadb-operator-controller-manager-7595db4997-skrx7" event={"ID":"568b316e-dee2-4a2a-967b-730a5f03de09","Type":"ContainerDied","Data":"259cb507b5536b21cde88922e40cc69650c04a6771e7243954ae6811adade8a0"} Dec 03 13:18:49 crc kubenswrapper[5052]: I1203 13:18:49.893255 5052 scope.go:117] "RemoveContainer" containerID="151cb2f59f27897dd419d8881ee3dbbaf6cfece15d5d067e9de2130be92940a7" Dec 03 13:18:49 crc kubenswrapper[5052]: I1203 13:18:49.895123 5052 generic.go:334] "Generic (PLEG): container finished" podID="f7ca8ea0-92d6-4274-96f7-8297170883fc" containerID="cb36d6b31264eb861a891fb9a33d6ae809a253d3427f4bda6dfaa99d86e000b1" exitCode=0 Dec 03 13:18:49 crc kubenswrapper[5052]: I1203 13:18:49.895164 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/mariadb-operator-index-jbcj6" event={"ID":"f7ca8ea0-92d6-4274-96f7-8297170883fc","Type":"ContainerDied","Data":"cb36d6b31264eb861a891fb9a33d6ae809a253d3427f4bda6dfaa99d86e000b1"} Dec 03 13:18:49 crc kubenswrapper[5052]: I1203 13:18:49.895220 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/mariadb-operator-index-jbcj6" event={"ID":"f7ca8ea0-92d6-4274-96f7-8297170883fc","Type":"ContainerDied","Data":"f491e7b203095b1a58705d93ae6c6200836e6180c6a29211cfff9043b6b5b29d"} Dec 03 13:18:49 crc kubenswrapper[5052]: I1203 13:18:49.895181 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-operators/mariadb-operator-index-jbcj6" Dec 03 13:18:49 crc kubenswrapper[5052]: I1203 13:18:49.915757 5052 scope.go:117] "RemoveContainer" containerID="151cb2f59f27897dd419d8881ee3dbbaf6cfece15d5d067e9de2130be92940a7" Dec 03 13:18:49 crc kubenswrapper[5052]: E1203 13:18:49.916602 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"151cb2f59f27897dd419d8881ee3dbbaf6cfece15d5d067e9de2130be92940a7\": container with ID starting with 151cb2f59f27897dd419d8881ee3dbbaf6cfece15d5d067e9de2130be92940a7 not found: ID does not exist" containerID="151cb2f59f27897dd419d8881ee3dbbaf6cfece15d5d067e9de2130be92940a7" Dec 03 13:18:49 crc kubenswrapper[5052]: I1203 13:18:49.916654 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"151cb2f59f27897dd419d8881ee3dbbaf6cfece15d5d067e9de2130be92940a7"} err="failed to get container status \"151cb2f59f27897dd419d8881ee3dbbaf6cfece15d5d067e9de2130be92940a7\": rpc error: code = NotFound desc = could not find container \"151cb2f59f27897dd419d8881ee3dbbaf6cfece15d5d067e9de2130be92940a7\": container with ID starting with 151cb2f59f27897dd419d8881ee3dbbaf6cfece15d5d067e9de2130be92940a7 not found: ID does not exist" Dec 03 13:18:49 crc kubenswrapper[5052]: I1203 13:18:49.916723 5052 scope.go:117] "RemoveContainer" containerID="cb36d6b31264eb861a891fb9a33d6ae809a253d3427f4bda6dfaa99d86e000b1" Dec 03 13:18:49 crc kubenswrapper[5052]: I1203 13:18:49.930353 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-operators/mariadb-operator-controller-manager-7595db4997-skrx7"] Dec 03 13:18:49 crc kubenswrapper[5052]: I1203 13:18:49.952474 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-operators/mariadb-operator-controller-manager-7595db4997-skrx7"] Dec 03 13:18:49 crc kubenswrapper[5052]: I1203 13:18:49.993698 5052 scope.go:117] "RemoveContainer" containerID="cb36d6b31264eb861a891fb9a33d6ae809a253d3427f4bda6dfaa99d86e000b1" Dec 03 13:18:49 crc kubenswrapper[5052]: I1203 13:18:49.995036 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bxch2\" (UniqueName: \"kubernetes.io/projected/f7ca8ea0-92d6-4274-96f7-8297170883fc-kube-api-access-bxch2\") pod \"f7ca8ea0-92d6-4274-96f7-8297170883fc\" (UID: \"f7ca8ea0-92d6-4274-96f7-8297170883fc\") " Dec 03 13:18:49 crc kubenswrapper[5052]: E1203 13:18:49.996115 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cb36d6b31264eb861a891fb9a33d6ae809a253d3427f4bda6dfaa99d86e000b1\": container with ID starting with cb36d6b31264eb861a891fb9a33d6ae809a253d3427f4bda6dfaa99d86e000b1 not found: ID does not exist" containerID="cb36d6b31264eb861a891fb9a33d6ae809a253d3427f4bda6dfaa99d86e000b1" Dec 03 13:18:49 crc kubenswrapper[5052]: I1203 13:18:49.996160 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cb36d6b31264eb861a891fb9a33d6ae809a253d3427f4bda6dfaa99d86e000b1"} err="failed to get container status \"cb36d6b31264eb861a891fb9a33d6ae809a253d3427f4bda6dfaa99d86e000b1\": rpc error: code = NotFound desc = could not find container \"cb36d6b31264eb861a891fb9a33d6ae809a253d3427f4bda6dfaa99d86e000b1\": container with ID starting with cb36d6b31264eb861a891fb9a33d6ae809a253d3427f4bda6dfaa99d86e000b1 not found: ID does not exist" Dec 03 13:18:50 crc kubenswrapper[5052]: I1203 13:18:49.999122 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f7ca8ea0-92d6-4274-96f7-8297170883fc-kube-api-access-bxch2" (OuterVolumeSpecName: "kube-api-access-bxch2") pod "f7ca8ea0-92d6-4274-96f7-8297170883fc" (UID: "f7ca8ea0-92d6-4274-96f7-8297170883fc"). InnerVolumeSpecName "kube-api-access-bxch2". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:18:50 crc kubenswrapper[5052]: I1203 13:18:50.096161 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-bxch2\" (UniqueName: \"kubernetes.io/projected/f7ca8ea0-92d6-4274-96f7-8297170883fc-kube-api-access-bxch2\") on node \"crc\" DevicePath \"\"" Dec 03 13:18:50 crc kubenswrapper[5052]: I1203 13:18:50.230703 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-operators/mariadb-operator-index-jbcj6"] Dec 03 13:18:50 crc kubenswrapper[5052]: I1203 13:18:50.234909 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-operators/mariadb-operator-index-jbcj6"] Dec 03 13:18:51 crc kubenswrapper[5052]: I1203 13:18:51.682371 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="568b316e-dee2-4a2a-967b-730a5f03de09" path="/var/lib/kubelet/pods/568b316e-dee2-4a2a-967b-730a5f03de09/volumes" Dec 03 13:18:51 crc kubenswrapper[5052]: I1203 13:18:51.683061 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f7ca8ea0-92d6-4274-96f7-8297170883fc" path="/var/lib/kubelet/pods/f7ca8ea0-92d6-4274-96f7-8297170883fc/volumes" Dec 03 13:18:55 crc kubenswrapper[5052]: I1203 13:18:55.673546 5052 scope.go:117] "RemoveContainer" containerID="12a8b1f470f6d59d7700e376f2a48eb36363f91f9823a35ce534062c0974c10c" Dec 03 13:18:55 crc kubenswrapper[5052]: E1203 13:18:55.674892 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 13:19:07 crc kubenswrapper[5052]: I1203 13:19:07.673364 5052 scope.go:117] "RemoveContainer" containerID="12a8b1f470f6d59d7700e376f2a48eb36363f91f9823a35ce534062c0974c10c" Dec 03 13:19:07 crc kubenswrapper[5052]: E1203 13:19:07.674426 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 13:19:11 crc kubenswrapper[5052]: I1203 13:19:11.845830 5052 pod_container_manager_linux.go:210] "Failed to delete cgroup paths" cgroupName=["kubepods","besteffort","pode2e3b100-677d-4b22-af35-1563bbda4b2a"] err="unable to destroy cgroup paths for cgroup [kubepods besteffort pode2e3b100-677d-4b22-af35-1563bbda4b2a] : Timed out while waiting for systemd to remove kubepods-besteffort-pode2e3b100_677d_4b22_af35_1563bbda4b2a.slice" Dec 03 13:19:18 crc kubenswrapper[5052]: I1203 13:19:18.673392 5052 scope.go:117] "RemoveContainer" containerID="12a8b1f470f6d59d7700e376f2a48eb36363f91f9823a35ce534062c0974c10c" Dec 03 13:19:18 crc kubenswrapper[5052]: E1203 13:19:18.674822 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 13:19:33 crc kubenswrapper[5052]: I1203 13:19:33.672636 5052 scope.go:117] "RemoveContainer" containerID="12a8b1f470f6d59d7700e376f2a48eb36363f91f9823a35ce534062c0974c10c" Dec 03 13:19:33 crc kubenswrapper[5052]: E1203 13:19:33.673439 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 13:19:43 crc kubenswrapper[5052]: I1203 13:19:43.659163 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-operators/mariadb-operator-index-h5gkb"] Dec 03 13:19:43 crc kubenswrapper[5052]: E1203 13:19:43.659956 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="fd9c88e5-c614-42c3-a252-e8a0600210a7" containerName="registry-server" Dec 03 13:19:43 crc kubenswrapper[5052]: I1203 13:19:43.659971 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="fd9c88e5-c614-42c3-a252-e8a0600210a7" containerName="registry-server" Dec 03 13:19:43 crc kubenswrapper[5052]: E1203 13:19:43.659987 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="9bef396c-7ed5-46b5-b57d-f4cd7df5c872" containerName="operator" Dec 03 13:19:43 crc kubenswrapper[5052]: I1203 13:19:43.659992 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="9bef396c-7ed5-46b5-b57d-f4cd7df5c872" containerName="operator" Dec 03 13:19:43 crc kubenswrapper[5052]: E1203 13:19:43.660006 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="caf5c2cd-0156-4996-b20a-e5b5b9255ddc" containerName="rabbitmq" Dec 03 13:19:43 crc kubenswrapper[5052]: I1203 13:19:43.660012 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="caf5c2cd-0156-4996-b20a-e5b5b9255ddc" containerName="rabbitmq" Dec 03 13:19:43 crc kubenswrapper[5052]: E1203 13:19:43.660022 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="61edc25e-298b-4f43-ad10-5457259b6dc0" containerName="mysql-bootstrap" Dec 03 13:19:43 crc kubenswrapper[5052]: I1203 13:19:43.660027 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="61edc25e-298b-4f43-ad10-5457259b6dc0" containerName="mysql-bootstrap" Dec 03 13:19:43 crc kubenswrapper[5052]: E1203 13:19:43.660037 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="5f441756-e059-4db0-943e-eafb2d48e668" containerName="kube-rbac-proxy" Dec 03 13:19:43 crc kubenswrapper[5052]: I1203 13:19:43.660043 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f441756-e059-4db0-943e-eafb2d48e668" containerName="kube-rbac-proxy" Dec 03 13:19:43 crc kubenswrapper[5052]: E1203 13:19:43.660055 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="61edc25e-298b-4f43-ad10-5457259b6dc0" containerName="galera" Dec 03 13:19:43 crc kubenswrapper[5052]: I1203 13:19:43.660060 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="61edc25e-298b-4f43-ad10-5457259b6dc0" containerName="galera" Dec 03 13:19:43 crc kubenswrapper[5052]: E1203 13:19:43.660067 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="e2e3b100-677d-4b22-af35-1563bbda4b2a" containerName="mariadb-account-delete" Dec 03 13:19:43 crc kubenswrapper[5052]: I1203 13:19:43.660073 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="e2e3b100-677d-4b22-af35-1563bbda4b2a" containerName="mariadb-account-delete" Dec 03 13:19:43 crc kubenswrapper[5052]: E1203 13:19:43.660082 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="caf5c2cd-0156-4996-b20a-e5b5b9255ddc" containerName="setup-container" Dec 03 13:19:43 crc kubenswrapper[5052]: I1203 13:19:43.660087 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="caf5c2cd-0156-4996-b20a-e5b5b9255ddc" containerName="setup-container" Dec 03 13:19:43 crc kubenswrapper[5052]: E1203 13:19:43.660094 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="6ccef89f-58bd-4e50-9d8c-d5f076df3505" containerName="memcached" Dec 03 13:19:43 crc kubenswrapper[5052]: I1203 13:19:43.660102 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="6ccef89f-58bd-4e50-9d8c-d5f076df3505" containerName="memcached" Dec 03 13:19:43 crc kubenswrapper[5052]: E1203 13:19:43.660114 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="a93ce5ec-f168-4024-9cad-d6886d0421cb" containerName="registry-server" Dec 03 13:19:43 crc kubenswrapper[5052]: I1203 13:19:43.660120 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="a93ce5ec-f168-4024-9cad-d6886d0421cb" containerName="registry-server" Dec 03 13:19:43 crc kubenswrapper[5052]: E1203 13:19:43.660130 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="26dcf610-b5bf-4d7c-86c4-169ef42fb0e7" containerName="mysql-bootstrap" Dec 03 13:19:43 crc kubenswrapper[5052]: I1203 13:19:43.660135 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="26dcf610-b5bf-4d7c-86c4-169ef42fb0e7" containerName="mysql-bootstrap" Dec 03 13:19:43 crc kubenswrapper[5052]: E1203 13:19:43.660144 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="f7ca8ea0-92d6-4274-96f7-8297170883fc" containerName="registry-server" Dec 03 13:19:43 crc kubenswrapper[5052]: I1203 13:19:43.660149 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7ca8ea0-92d6-4274-96f7-8297170883fc" containerName="registry-server" Dec 03 13:19:43 crc kubenswrapper[5052]: E1203 13:19:43.660158 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="42d5fccb-f24d-4707-8167-30eeae73378e" containerName="registry-server" Dec 03 13:19:43 crc kubenswrapper[5052]: I1203 13:19:43.660164 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="42d5fccb-f24d-4707-8167-30eeae73378e" containerName="registry-server" Dec 03 13:19:43 crc kubenswrapper[5052]: E1203 13:19:43.660178 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="5f441756-e059-4db0-943e-eafb2d48e668" containerName="manager" Dec 03 13:19:43 crc kubenswrapper[5052]: I1203 13:19:43.660184 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f441756-e059-4db0-943e-eafb2d48e668" containerName="manager" Dec 03 13:19:43 crc kubenswrapper[5052]: E1203 13:19:43.660192 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="e2e3b100-677d-4b22-af35-1563bbda4b2a" containerName="mariadb-account-delete" Dec 03 13:19:43 crc kubenswrapper[5052]: I1203 13:19:43.660199 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="e2e3b100-677d-4b22-af35-1563bbda4b2a" containerName="mariadb-account-delete" Dec 03 13:19:43 crc kubenswrapper[5052]: E1203 13:19:43.660206 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="c8797321-9192-42b6-a077-6c057a12de0c" containerName="keystone-api" Dec 03 13:19:43 crc kubenswrapper[5052]: I1203 13:19:43.660212 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="c8797321-9192-42b6-a077-6c057a12de0c" containerName="keystone-api" Dec 03 13:19:43 crc kubenswrapper[5052]: E1203 13:19:43.660220 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="995e0ea0-f1df-4bf5-8d59-1d75f9861688" containerName="manager" Dec 03 13:19:43 crc kubenswrapper[5052]: I1203 13:19:43.660225 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="995e0ea0-f1df-4bf5-8d59-1d75f9861688" containerName="manager" Dec 03 13:19:43 crc kubenswrapper[5052]: E1203 13:19:43.660236 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="26dcf610-b5bf-4d7c-86c4-169ef42fb0e7" containerName="galera" Dec 03 13:19:43 crc kubenswrapper[5052]: I1203 13:19:43.660242 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="26dcf610-b5bf-4d7c-86c4-169ef42fb0e7" containerName="galera" Dec 03 13:19:43 crc kubenswrapper[5052]: E1203 13:19:43.660248 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="568b316e-dee2-4a2a-967b-730a5f03de09" containerName="manager" Dec 03 13:19:43 crc kubenswrapper[5052]: I1203 13:19:43.660254 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="568b316e-dee2-4a2a-967b-730a5f03de09" containerName="manager" Dec 03 13:19:43 crc kubenswrapper[5052]: E1203 13:19:43.660264 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="317b3c95-8146-4310-862e-950f6f4c8f3c" containerName="galera" Dec 03 13:19:43 crc kubenswrapper[5052]: I1203 13:19:43.660269 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="317b3c95-8146-4310-862e-950f6f4c8f3c" containerName="galera" Dec 03 13:19:43 crc kubenswrapper[5052]: E1203 13:19:43.660275 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="317b3c95-8146-4310-862e-950f6f4c8f3c" containerName="mysql-bootstrap" Dec 03 13:19:43 crc kubenswrapper[5052]: I1203 13:19:43.660282 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="317b3c95-8146-4310-862e-950f6f4c8f3c" containerName="mysql-bootstrap" Dec 03 13:19:43 crc kubenswrapper[5052]: E1203 13:19:43.660289 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="dd5d90ba-19d0-40a0-91ef-71f560226194" containerName="manager" Dec 03 13:19:43 crc kubenswrapper[5052]: I1203 13:19:43.660295 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="dd5d90ba-19d0-40a0-91ef-71f560226194" containerName="manager" Dec 03 13:19:43 crc kubenswrapper[5052]: E1203 13:19:43.660303 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="0abdde9a-bebd-4ece-9fb6-0cf23ba9a11a" containerName="registry-server" Dec 03 13:19:43 crc kubenswrapper[5052]: I1203 13:19:43.660308 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="0abdde9a-bebd-4ece-9fb6-0cf23ba9a11a" containerName="registry-server" Dec 03 13:19:43 crc kubenswrapper[5052]: I1203 13:19:43.660401 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="5f441756-e059-4db0-943e-eafb2d48e668" containerName="manager" Dec 03 13:19:43 crc kubenswrapper[5052]: I1203 13:19:43.660419 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="9bef396c-7ed5-46b5-b57d-f4cd7df5c872" containerName="operator" Dec 03 13:19:43 crc kubenswrapper[5052]: I1203 13:19:43.660439 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="317b3c95-8146-4310-862e-950f6f4c8f3c" containerName="galera" Dec 03 13:19:43 crc kubenswrapper[5052]: I1203 13:19:43.660447 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="42d5fccb-f24d-4707-8167-30eeae73378e" containerName="registry-server" Dec 03 13:19:43 crc kubenswrapper[5052]: I1203 13:19:43.660453 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="26dcf610-b5bf-4d7c-86c4-169ef42fb0e7" containerName="galera" Dec 03 13:19:43 crc kubenswrapper[5052]: I1203 13:19:43.660461 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="568b316e-dee2-4a2a-967b-730a5f03de09" containerName="manager" Dec 03 13:19:43 crc kubenswrapper[5052]: I1203 13:19:43.660470 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="6ccef89f-58bd-4e50-9d8c-d5f076df3505" containerName="memcached" Dec 03 13:19:43 crc kubenswrapper[5052]: I1203 13:19:43.660479 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="caf5c2cd-0156-4996-b20a-e5b5b9255ddc" containerName="rabbitmq" Dec 03 13:19:43 crc kubenswrapper[5052]: I1203 13:19:43.660485 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="5f441756-e059-4db0-943e-eafb2d48e668" containerName="kube-rbac-proxy" Dec 03 13:19:43 crc kubenswrapper[5052]: I1203 13:19:43.660495 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="61edc25e-298b-4f43-ad10-5457259b6dc0" containerName="galera" Dec 03 13:19:43 crc kubenswrapper[5052]: I1203 13:19:43.660503 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="a93ce5ec-f168-4024-9cad-d6886d0421cb" containerName="registry-server" Dec 03 13:19:43 crc kubenswrapper[5052]: I1203 13:19:43.660554 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="dd5d90ba-19d0-40a0-91ef-71f560226194" containerName="manager" Dec 03 13:19:43 crc kubenswrapper[5052]: I1203 13:19:43.660570 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="fd9c88e5-c614-42c3-a252-e8a0600210a7" containerName="registry-server" Dec 03 13:19:43 crc kubenswrapper[5052]: I1203 13:19:43.660582 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="f7ca8ea0-92d6-4274-96f7-8297170883fc" containerName="registry-server" Dec 03 13:19:43 crc kubenswrapper[5052]: I1203 13:19:43.660590 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="0abdde9a-bebd-4ece-9fb6-0cf23ba9a11a" containerName="registry-server" Dec 03 13:19:43 crc kubenswrapper[5052]: I1203 13:19:43.660599 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="e2e3b100-677d-4b22-af35-1563bbda4b2a" containerName="mariadb-account-delete" Dec 03 13:19:43 crc kubenswrapper[5052]: I1203 13:19:43.660609 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="c8797321-9192-42b6-a077-6c057a12de0c" containerName="keystone-api" Dec 03 13:19:43 crc kubenswrapper[5052]: I1203 13:19:43.660620 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="995e0ea0-f1df-4bf5-8d59-1d75f9861688" containerName="manager" Dec 03 13:19:43 crc kubenswrapper[5052]: I1203 13:19:43.660628 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="e2e3b100-677d-4b22-af35-1563bbda4b2a" containerName="mariadb-account-delete" Dec 03 13:19:43 crc kubenswrapper[5052]: I1203 13:19:43.661039 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-operators/mariadb-operator-index-h5gkb" Dec 03 13:19:43 crc kubenswrapper[5052]: I1203 13:19:43.663032 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-operators"/"mariadb-operator-index-dockercfg-cm6kt" Dec 03 13:19:43 crc kubenswrapper[5052]: I1203 13:19:43.663725 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-operators"/"kube-root-ca.crt" Dec 03 13:19:43 crc kubenswrapper[5052]: I1203 13:19:43.666183 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-operators"/"openshift-service-ca.crt" Dec 03 13:19:43 crc kubenswrapper[5052]: I1203 13:19:43.672558 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-operators/mariadb-operator-index-h5gkb"] Dec 03 13:19:43 crc kubenswrapper[5052]: I1203 13:19:43.804657 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wn795\" (UniqueName: \"kubernetes.io/projected/9e850982-751d-4fb2-afdb-8673d925c8e5-kube-api-access-wn795\") pod \"mariadb-operator-index-h5gkb\" (UID: \"9e850982-751d-4fb2-afdb-8673d925c8e5\") " pod="openstack-operators/mariadb-operator-index-h5gkb" Dec 03 13:19:43 crc kubenswrapper[5052]: I1203 13:19:43.906481 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-wn795\" (UniqueName: \"kubernetes.io/projected/9e850982-751d-4fb2-afdb-8673d925c8e5-kube-api-access-wn795\") pod \"mariadb-operator-index-h5gkb\" (UID: \"9e850982-751d-4fb2-afdb-8673d925c8e5\") " pod="openstack-operators/mariadb-operator-index-h5gkb" Dec 03 13:19:43 crc kubenswrapper[5052]: I1203 13:19:43.923907 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-wn795\" (UniqueName: \"kubernetes.io/projected/9e850982-751d-4fb2-afdb-8673d925c8e5-kube-api-access-wn795\") pod \"mariadb-operator-index-h5gkb\" (UID: \"9e850982-751d-4fb2-afdb-8673d925c8e5\") " pod="openstack-operators/mariadb-operator-index-h5gkb" Dec 03 13:19:43 crc kubenswrapper[5052]: I1203 13:19:43.984937 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-operators/mariadb-operator-index-h5gkb" Dec 03 13:19:44 crc kubenswrapper[5052]: I1203 13:19:44.033039 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-operators/mariadb-operator-index-h5gkb"] Dec 03 13:19:44 crc kubenswrapper[5052]: I1203 13:19:44.247777 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-operators/mariadb-operator-index-lg4mj"] Dec 03 13:19:44 crc kubenswrapper[5052]: I1203 13:19:44.249012 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-operators/mariadb-operator-index-lg4mj" Dec 03 13:19:44 crc kubenswrapper[5052]: I1203 13:19:44.253152 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-operators/mariadb-operator-index-lg4mj"] Dec 03 13:19:44 crc kubenswrapper[5052]: I1203 13:19:44.310760 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k9lgt\" (UniqueName: \"kubernetes.io/projected/8cb04567-42af-44a4-a4ed-acfd1d7027bd-kube-api-access-k9lgt\") pod \"mariadb-operator-index-lg4mj\" (UID: \"8cb04567-42af-44a4-a4ed-acfd1d7027bd\") " pod="openstack-operators/mariadb-operator-index-lg4mj" Dec 03 13:19:44 crc kubenswrapper[5052]: I1203 13:19:44.401051 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-operators/mariadb-operator-index-h5gkb"] Dec 03 13:19:44 crc kubenswrapper[5052]: I1203 13:19:44.408626 5052 provider.go:102] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Dec 03 13:19:44 crc kubenswrapper[5052]: I1203 13:19:44.412366 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-k9lgt\" (UniqueName: \"kubernetes.io/projected/8cb04567-42af-44a4-a4ed-acfd1d7027bd-kube-api-access-k9lgt\") pod \"mariadb-operator-index-lg4mj\" (UID: \"8cb04567-42af-44a4-a4ed-acfd1d7027bd\") " pod="openstack-operators/mariadb-operator-index-lg4mj" Dec 03 13:19:44 crc kubenswrapper[5052]: I1203 13:19:44.431348 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-k9lgt\" (UniqueName: \"kubernetes.io/projected/8cb04567-42af-44a4-a4ed-acfd1d7027bd-kube-api-access-k9lgt\") pod \"mariadb-operator-index-lg4mj\" (UID: \"8cb04567-42af-44a4-a4ed-acfd1d7027bd\") " pod="openstack-operators/mariadb-operator-index-lg4mj" Dec 03 13:19:44 crc kubenswrapper[5052]: I1203 13:19:44.571873 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-operators/mariadb-operator-index-lg4mj" Dec 03 13:19:44 crc kubenswrapper[5052]: I1203 13:19:44.761370 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-operators/mariadb-operator-index-lg4mj"] Dec 03 13:19:44 crc kubenswrapper[5052]: W1203 13:19:44.768175 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8cb04567_42af_44a4_a4ed_acfd1d7027bd.slice/crio-0b452ac9a3d24fdcaf95cc93172de891ed82333bd974639962a332a120f23063 WatchSource:0}: Error finding container 0b452ac9a3d24fdcaf95cc93172de891ed82333bd974639962a332a120f23063: Status 404 returned error can't find the container with id 0b452ac9a3d24fdcaf95cc93172de891ed82333bd974639962a332a120f23063 Dec 03 13:19:45 crc kubenswrapper[5052]: I1203 13:19:45.311298 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/mariadb-operator-index-lg4mj" event={"ID":"8cb04567-42af-44a4-a4ed-acfd1d7027bd","Type":"ContainerStarted","Data":"0b452ac9a3d24fdcaf95cc93172de891ed82333bd974639962a332a120f23063"} Dec 03 13:19:45 crc kubenswrapper[5052]: I1203 13:19:45.312388 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/mariadb-operator-index-h5gkb" event={"ID":"9e850982-751d-4fb2-afdb-8673d925c8e5","Type":"ContainerStarted","Data":"45768985fc28acc075548ee6ca5e78ffe49b11fc65ee9ce57e35e2d2c032b64d"} Dec 03 13:19:46 crc kubenswrapper[5052]: I1203 13:19:46.321758 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-operators/mariadb-operator-index-h5gkb" podUID="9e850982-751d-4fb2-afdb-8673d925c8e5" containerName="registry-server" containerID="cri-o://2f995ffdf90a3b07ea03eb2c6f6f31b2d61ec696cb77ab3b9f1ce8f4f7cbddda" gracePeriod=2 Dec 03 13:19:46 crc kubenswrapper[5052]: I1203 13:19:46.322334 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/mariadb-operator-index-h5gkb" event={"ID":"9e850982-751d-4fb2-afdb-8673d925c8e5","Type":"ContainerStarted","Data":"2f995ffdf90a3b07ea03eb2c6f6f31b2d61ec696cb77ab3b9f1ce8f4f7cbddda"} Dec 03 13:19:46 crc kubenswrapper[5052]: I1203 13:19:46.324005 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/mariadb-operator-index-lg4mj" event={"ID":"8cb04567-42af-44a4-a4ed-acfd1d7027bd","Type":"ContainerStarted","Data":"78e5b6948d0c0fec3187be4ca25bc145dbcde6fc92d4ef02b69bf5faf93ed27e"} Dec 03 13:19:46 crc kubenswrapper[5052]: I1203 13:19:46.338762 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-operators/mariadb-operator-index-h5gkb" podStartSLOduration=1.652624754 podStartE2EDuration="3.33873704s" podCreationTimestamp="2025-12-03 13:19:43 +0000 UTC" firstStartedPulling="2025-12-03 13:19:44.408433971 +0000 UTC m=+7174.996675735" lastFinishedPulling="2025-12-03 13:19:46.094546257 +0000 UTC m=+7176.682788021" observedRunningTime="2025-12-03 13:19:46.337800243 +0000 UTC m=+7176.926042017" watchObservedRunningTime="2025-12-03 13:19:46.33873704 +0000 UTC m=+7176.926978804" Dec 03 13:19:46 crc kubenswrapper[5052]: I1203 13:19:46.352338 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-operators/mariadb-operator-index-lg4mj" podStartSLOduration=1.0933481760000001 podStartE2EDuration="2.352316919s" podCreationTimestamp="2025-12-03 13:19:44 +0000 UTC" firstStartedPulling="2025-12-03 13:19:44.770126419 +0000 UTC m=+7175.358368183" lastFinishedPulling="2025-12-03 13:19:46.029095152 +0000 UTC m=+7176.617336926" observedRunningTime="2025-12-03 13:19:46.352067312 +0000 UTC m=+7176.940309086" watchObservedRunningTime="2025-12-03 13:19:46.352316919 +0000 UTC m=+7176.940558713" Dec 03 13:19:46 crc kubenswrapper[5052]: I1203 13:19:46.686829 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-operators/mariadb-operator-index-h5gkb" Dec 03 13:19:46 crc kubenswrapper[5052]: I1203 13:19:46.844556 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wn795\" (UniqueName: \"kubernetes.io/projected/9e850982-751d-4fb2-afdb-8673d925c8e5-kube-api-access-wn795\") pod \"9e850982-751d-4fb2-afdb-8673d925c8e5\" (UID: \"9e850982-751d-4fb2-afdb-8673d925c8e5\") " Dec 03 13:19:46 crc kubenswrapper[5052]: I1203 13:19:46.850424 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9e850982-751d-4fb2-afdb-8673d925c8e5-kube-api-access-wn795" (OuterVolumeSpecName: "kube-api-access-wn795") pod "9e850982-751d-4fb2-afdb-8673d925c8e5" (UID: "9e850982-751d-4fb2-afdb-8673d925c8e5"). InnerVolumeSpecName "kube-api-access-wn795". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:19:46 crc kubenswrapper[5052]: I1203 13:19:46.939153 5052 scope.go:117] "RemoveContainer" containerID="2b0e197aa5bd3ab0ed35a7a9a8cfd5a398150425b1a75056bf68e75c6bf3e7d3" Dec 03 13:19:46 crc kubenswrapper[5052]: I1203 13:19:46.945893 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-wn795\" (UniqueName: \"kubernetes.io/projected/9e850982-751d-4fb2-afdb-8673d925c8e5-kube-api-access-wn795\") on node \"crc\" DevicePath \"\"" Dec 03 13:19:46 crc kubenswrapper[5052]: I1203 13:19:46.968301 5052 scope.go:117] "RemoveContainer" containerID="bc20387fa50eb6a9597e4423e26aef13c7f844e105dc15dd899d9f9df12373b7" Dec 03 13:19:46 crc kubenswrapper[5052]: I1203 13:19:46.985358 5052 scope.go:117] "RemoveContainer" containerID="1918a3a17b37d4280c7e0df4b3ef36de0a518ce5732f596e41092dd4c5c7f007" Dec 03 13:19:47 crc kubenswrapper[5052]: I1203 13:19:47.332816 5052 generic.go:334] "Generic (PLEG): container finished" podID="9e850982-751d-4fb2-afdb-8673d925c8e5" containerID="2f995ffdf90a3b07ea03eb2c6f6f31b2d61ec696cb77ab3b9f1ce8f4f7cbddda" exitCode=0 Dec 03 13:19:47 crc kubenswrapper[5052]: I1203 13:19:47.333568 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-operators/mariadb-operator-index-h5gkb" Dec 03 13:19:47 crc kubenswrapper[5052]: I1203 13:19:47.338575 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/mariadb-operator-index-h5gkb" event={"ID":"9e850982-751d-4fb2-afdb-8673d925c8e5","Type":"ContainerDied","Data":"2f995ffdf90a3b07ea03eb2c6f6f31b2d61ec696cb77ab3b9f1ce8f4f7cbddda"} Dec 03 13:19:47 crc kubenswrapper[5052]: I1203 13:19:47.338644 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/mariadb-operator-index-h5gkb" event={"ID":"9e850982-751d-4fb2-afdb-8673d925c8e5","Type":"ContainerDied","Data":"45768985fc28acc075548ee6ca5e78ffe49b11fc65ee9ce57e35e2d2c032b64d"} Dec 03 13:19:47 crc kubenswrapper[5052]: I1203 13:19:47.338704 5052 scope.go:117] "RemoveContainer" containerID="2f995ffdf90a3b07ea03eb2c6f6f31b2d61ec696cb77ab3b9f1ce8f4f7cbddda" Dec 03 13:19:47 crc kubenswrapper[5052]: I1203 13:19:47.358758 5052 scope.go:117] "RemoveContainer" containerID="2f995ffdf90a3b07ea03eb2c6f6f31b2d61ec696cb77ab3b9f1ce8f4f7cbddda" Dec 03 13:19:47 crc kubenswrapper[5052]: E1203 13:19:47.359243 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2f995ffdf90a3b07ea03eb2c6f6f31b2d61ec696cb77ab3b9f1ce8f4f7cbddda\": container with ID starting with 2f995ffdf90a3b07ea03eb2c6f6f31b2d61ec696cb77ab3b9f1ce8f4f7cbddda not found: ID does not exist" containerID="2f995ffdf90a3b07ea03eb2c6f6f31b2d61ec696cb77ab3b9f1ce8f4f7cbddda" Dec 03 13:19:47 crc kubenswrapper[5052]: I1203 13:19:47.359275 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2f995ffdf90a3b07ea03eb2c6f6f31b2d61ec696cb77ab3b9f1ce8f4f7cbddda"} err="failed to get container status \"2f995ffdf90a3b07ea03eb2c6f6f31b2d61ec696cb77ab3b9f1ce8f4f7cbddda\": rpc error: code = NotFound desc = could not find container \"2f995ffdf90a3b07ea03eb2c6f6f31b2d61ec696cb77ab3b9f1ce8f4f7cbddda\": container with ID starting with 2f995ffdf90a3b07ea03eb2c6f6f31b2d61ec696cb77ab3b9f1ce8f4f7cbddda not found: ID does not exist" Dec 03 13:19:47 crc kubenswrapper[5052]: I1203 13:19:47.364659 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-operators/mariadb-operator-index-h5gkb"] Dec 03 13:19:47 crc kubenswrapper[5052]: I1203 13:19:47.370382 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-operators/mariadb-operator-index-h5gkb"] Dec 03 13:19:47 crc kubenswrapper[5052]: I1203 13:19:47.682385 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9e850982-751d-4fb2-afdb-8673d925c8e5" path="/var/lib/kubelet/pods/9e850982-751d-4fb2-afdb-8673d925c8e5/volumes" Dec 03 13:19:48 crc kubenswrapper[5052]: I1203 13:19:48.673423 5052 scope.go:117] "RemoveContainer" containerID="12a8b1f470f6d59d7700e376f2a48eb36363f91f9823a35ce534062c0974c10c" Dec 03 13:19:48 crc kubenswrapper[5052]: E1203 13:19:48.673780 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 13:19:50 crc kubenswrapper[5052]: I1203 13:19:50.678253 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-operators/55a3886cc1ed42812df4eab61c7a6033dc924d195539e8545c8f175f6182z8b"] Dec 03 13:19:50 crc kubenswrapper[5052]: E1203 13:19:50.678881 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="9e850982-751d-4fb2-afdb-8673d925c8e5" containerName="registry-server" Dec 03 13:19:50 crc kubenswrapper[5052]: I1203 13:19:50.678895 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="9e850982-751d-4fb2-afdb-8673d925c8e5" containerName="registry-server" Dec 03 13:19:50 crc kubenswrapper[5052]: I1203 13:19:50.679016 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="9e850982-751d-4fb2-afdb-8673d925c8e5" containerName="registry-server" Dec 03 13:19:50 crc kubenswrapper[5052]: I1203 13:19:50.680035 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-operators/55a3886cc1ed42812df4eab61c7a6033dc924d195539e8545c8f175f6182z8b" Dec 03 13:19:50 crc kubenswrapper[5052]: I1203 13:19:50.682301 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-operators"/"default-dockercfg-7fklr" Dec 03 13:19:50 crc kubenswrapper[5052]: I1203 13:19:50.687751 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-operators/55a3886cc1ed42812df4eab61c7a6033dc924d195539e8545c8f175f6182z8b"] Dec 03 13:19:50 crc kubenswrapper[5052]: I1203 13:19:50.792771 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a48b1dcb-27f2-4c5b-b735-cc1de58a1384-bundle\") pod \"55a3886cc1ed42812df4eab61c7a6033dc924d195539e8545c8f175f6182z8b\" (UID: \"a48b1dcb-27f2-4c5b-b735-cc1de58a1384\") " pod="openstack-operators/55a3886cc1ed42812df4eab61c7a6033dc924d195539e8545c8f175f6182z8b" Dec 03 13:19:50 crc kubenswrapper[5052]: I1203 13:19:50.793057 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a48b1dcb-27f2-4c5b-b735-cc1de58a1384-util\") pod \"55a3886cc1ed42812df4eab61c7a6033dc924d195539e8545c8f175f6182z8b\" (UID: \"a48b1dcb-27f2-4c5b-b735-cc1de58a1384\") " pod="openstack-operators/55a3886cc1ed42812df4eab61c7a6033dc924d195539e8545c8f175f6182z8b" Dec 03 13:19:50 crc kubenswrapper[5052]: I1203 13:19:50.793225 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gfs4p\" (UniqueName: \"kubernetes.io/projected/a48b1dcb-27f2-4c5b-b735-cc1de58a1384-kube-api-access-gfs4p\") pod \"55a3886cc1ed42812df4eab61c7a6033dc924d195539e8545c8f175f6182z8b\" (UID: \"a48b1dcb-27f2-4c5b-b735-cc1de58a1384\") " pod="openstack-operators/55a3886cc1ed42812df4eab61c7a6033dc924d195539e8545c8f175f6182z8b" Dec 03 13:19:50 crc kubenswrapper[5052]: I1203 13:19:50.894013 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a48b1dcb-27f2-4c5b-b735-cc1de58a1384-bundle\") pod \"55a3886cc1ed42812df4eab61c7a6033dc924d195539e8545c8f175f6182z8b\" (UID: \"a48b1dcb-27f2-4c5b-b735-cc1de58a1384\") " pod="openstack-operators/55a3886cc1ed42812df4eab61c7a6033dc924d195539e8545c8f175f6182z8b" Dec 03 13:19:50 crc kubenswrapper[5052]: I1203 13:19:50.894064 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a48b1dcb-27f2-4c5b-b735-cc1de58a1384-util\") pod \"55a3886cc1ed42812df4eab61c7a6033dc924d195539e8545c8f175f6182z8b\" (UID: \"a48b1dcb-27f2-4c5b-b735-cc1de58a1384\") " pod="openstack-operators/55a3886cc1ed42812df4eab61c7a6033dc924d195539e8545c8f175f6182z8b" Dec 03 13:19:50 crc kubenswrapper[5052]: I1203 13:19:50.894133 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-gfs4p\" (UniqueName: \"kubernetes.io/projected/a48b1dcb-27f2-4c5b-b735-cc1de58a1384-kube-api-access-gfs4p\") pod \"55a3886cc1ed42812df4eab61c7a6033dc924d195539e8545c8f175f6182z8b\" (UID: \"a48b1dcb-27f2-4c5b-b735-cc1de58a1384\") " pod="openstack-operators/55a3886cc1ed42812df4eab61c7a6033dc924d195539e8545c8f175f6182z8b" Dec 03 13:19:50 crc kubenswrapper[5052]: I1203 13:19:50.894627 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a48b1dcb-27f2-4c5b-b735-cc1de58a1384-bundle\") pod \"55a3886cc1ed42812df4eab61c7a6033dc924d195539e8545c8f175f6182z8b\" (UID: \"a48b1dcb-27f2-4c5b-b735-cc1de58a1384\") " pod="openstack-operators/55a3886cc1ed42812df4eab61c7a6033dc924d195539e8545c8f175f6182z8b" Dec 03 13:19:50 crc kubenswrapper[5052]: I1203 13:19:50.894722 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a48b1dcb-27f2-4c5b-b735-cc1de58a1384-util\") pod \"55a3886cc1ed42812df4eab61c7a6033dc924d195539e8545c8f175f6182z8b\" (UID: \"a48b1dcb-27f2-4c5b-b735-cc1de58a1384\") " pod="openstack-operators/55a3886cc1ed42812df4eab61c7a6033dc924d195539e8545c8f175f6182z8b" Dec 03 13:19:50 crc kubenswrapper[5052]: I1203 13:19:50.923562 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-gfs4p\" (UniqueName: \"kubernetes.io/projected/a48b1dcb-27f2-4c5b-b735-cc1de58a1384-kube-api-access-gfs4p\") pod \"55a3886cc1ed42812df4eab61c7a6033dc924d195539e8545c8f175f6182z8b\" (UID: \"a48b1dcb-27f2-4c5b-b735-cc1de58a1384\") " pod="openstack-operators/55a3886cc1ed42812df4eab61c7a6033dc924d195539e8545c8f175f6182z8b" Dec 03 13:19:50 crc kubenswrapper[5052]: I1203 13:19:50.998560 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-operators/55a3886cc1ed42812df4eab61c7a6033dc924d195539e8545c8f175f6182z8b" Dec 03 13:19:51 crc kubenswrapper[5052]: I1203 13:19:51.382698 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-operators/55a3886cc1ed42812df4eab61c7a6033dc924d195539e8545c8f175f6182z8b"] Dec 03 13:19:52 crc kubenswrapper[5052]: I1203 13:19:52.371826 5052 generic.go:334] "Generic (PLEG): container finished" podID="a48b1dcb-27f2-4c5b-b735-cc1de58a1384" containerID="7f88dc0ba48b4207d557f5ae6a680b60bb1a94eee39236bcbc1b4e9b30d4d4b1" exitCode=0 Dec 03 13:19:52 crc kubenswrapper[5052]: I1203 13:19:52.371970 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/55a3886cc1ed42812df4eab61c7a6033dc924d195539e8545c8f175f6182z8b" event={"ID":"a48b1dcb-27f2-4c5b-b735-cc1de58a1384","Type":"ContainerDied","Data":"7f88dc0ba48b4207d557f5ae6a680b60bb1a94eee39236bcbc1b4e9b30d4d4b1"} Dec 03 13:19:52 crc kubenswrapper[5052]: I1203 13:19:52.372779 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/55a3886cc1ed42812df4eab61c7a6033dc924d195539e8545c8f175f6182z8b" event={"ID":"a48b1dcb-27f2-4c5b-b735-cc1de58a1384","Type":"ContainerStarted","Data":"c2cf01fb297b919adc2fbdc44a629c1cd07ed38f81845e9a5c318bb78a0549b7"} Dec 03 13:19:53 crc kubenswrapper[5052]: I1203 13:19:53.382296 5052 generic.go:334] "Generic (PLEG): container finished" podID="a48b1dcb-27f2-4c5b-b735-cc1de58a1384" containerID="fb7b00ce91c676ce96b647fb7ab7b5541879f301a362cd7af3c7252cd2acfc00" exitCode=0 Dec 03 13:19:53 crc kubenswrapper[5052]: I1203 13:19:53.382492 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/55a3886cc1ed42812df4eab61c7a6033dc924d195539e8545c8f175f6182z8b" event={"ID":"a48b1dcb-27f2-4c5b-b735-cc1de58a1384","Type":"ContainerDied","Data":"fb7b00ce91c676ce96b647fb7ab7b5541879f301a362cd7af3c7252cd2acfc00"} Dec 03 13:19:54 crc kubenswrapper[5052]: I1203 13:19:54.394859 5052 generic.go:334] "Generic (PLEG): container finished" podID="a48b1dcb-27f2-4c5b-b735-cc1de58a1384" containerID="3f059c4beaaddb8bf0a200dd1eee0146df44f5a9adbac4ed4b87510ffd01762f" exitCode=0 Dec 03 13:19:54 crc kubenswrapper[5052]: I1203 13:19:54.394962 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/55a3886cc1ed42812df4eab61c7a6033dc924d195539e8545c8f175f6182z8b" event={"ID":"a48b1dcb-27f2-4c5b-b735-cc1de58a1384","Type":"ContainerDied","Data":"3f059c4beaaddb8bf0a200dd1eee0146df44f5a9adbac4ed4b87510ffd01762f"} Dec 03 13:19:54 crc kubenswrapper[5052]: I1203 13:19:54.573599 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-operators/mariadb-operator-index-lg4mj" Dec 03 13:19:54 crc kubenswrapper[5052]: I1203 13:19:54.573717 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-operators/mariadb-operator-index-lg4mj" Dec 03 13:19:54 crc kubenswrapper[5052]: I1203 13:19:54.605593 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-operators/mariadb-operator-index-lg4mj" Dec 03 13:19:55 crc kubenswrapper[5052]: I1203 13:19:55.428164 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-operators/mariadb-operator-index-lg4mj" Dec 03 13:19:55 crc kubenswrapper[5052]: I1203 13:19:55.673503 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-operators/55a3886cc1ed42812df4eab61c7a6033dc924d195539e8545c8f175f6182z8b" Dec 03 13:19:55 crc kubenswrapper[5052]: I1203 13:19:55.859351 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a48b1dcb-27f2-4c5b-b735-cc1de58a1384-util\") pod \"a48b1dcb-27f2-4c5b-b735-cc1de58a1384\" (UID: \"a48b1dcb-27f2-4c5b-b735-cc1de58a1384\") " Dec 03 13:19:55 crc kubenswrapper[5052]: I1203 13:19:55.859422 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a48b1dcb-27f2-4c5b-b735-cc1de58a1384-bundle\") pod \"a48b1dcb-27f2-4c5b-b735-cc1de58a1384\" (UID: \"a48b1dcb-27f2-4c5b-b735-cc1de58a1384\") " Dec 03 13:19:55 crc kubenswrapper[5052]: I1203 13:19:55.859473 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gfs4p\" (UniqueName: \"kubernetes.io/projected/a48b1dcb-27f2-4c5b-b735-cc1de58a1384-kube-api-access-gfs4p\") pod \"a48b1dcb-27f2-4c5b-b735-cc1de58a1384\" (UID: \"a48b1dcb-27f2-4c5b-b735-cc1de58a1384\") " Dec 03 13:19:55 crc kubenswrapper[5052]: I1203 13:19:55.861392 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a48b1dcb-27f2-4c5b-b735-cc1de58a1384-bundle" (OuterVolumeSpecName: "bundle") pod "a48b1dcb-27f2-4c5b-b735-cc1de58a1384" (UID: "a48b1dcb-27f2-4c5b-b735-cc1de58a1384"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 13:19:55 crc kubenswrapper[5052]: I1203 13:19:55.864932 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a48b1dcb-27f2-4c5b-b735-cc1de58a1384-kube-api-access-gfs4p" (OuterVolumeSpecName: "kube-api-access-gfs4p") pod "a48b1dcb-27f2-4c5b-b735-cc1de58a1384" (UID: "a48b1dcb-27f2-4c5b-b735-cc1de58a1384"). InnerVolumeSpecName "kube-api-access-gfs4p". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:19:55 crc kubenswrapper[5052]: I1203 13:19:55.891435 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a48b1dcb-27f2-4c5b-b735-cc1de58a1384-util" (OuterVolumeSpecName: "util") pod "a48b1dcb-27f2-4c5b-b735-cc1de58a1384" (UID: "a48b1dcb-27f2-4c5b-b735-cc1de58a1384"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 13:19:55 crc kubenswrapper[5052]: I1203 13:19:55.960690 5052 reconciler_common.go:293] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a48b1dcb-27f2-4c5b-b735-cc1de58a1384-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 13:19:55 crc kubenswrapper[5052]: I1203 13:19:55.960732 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-gfs4p\" (UniqueName: \"kubernetes.io/projected/a48b1dcb-27f2-4c5b-b735-cc1de58a1384-kube-api-access-gfs4p\") on node \"crc\" DevicePath \"\"" Dec 03 13:19:55 crc kubenswrapper[5052]: I1203 13:19:55.960742 5052 reconciler_common.go:293] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a48b1dcb-27f2-4c5b-b735-cc1de58a1384-util\") on node \"crc\" DevicePath \"\"" Dec 03 13:19:56 crc kubenswrapper[5052]: I1203 13:19:56.412316 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-operators/55a3886cc1ed42812df4eab61c7a6033dc924d195539e8545c8f175f6182z8b" Dec 03 13:19:56 crc kubenswrapper[5052]: I1203 13:19:56.412306 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/55a3886cc1ed42812df4eab61c7a6033dc924d195539e8545c8f175f6182z8b" event={"ID":"a48b1dcb-27f2-4c5b-b735-cc1de58a1384","Type":"ContainerDied","Data":"c2cf01fb297b919adc2fbdc44a629c1cd07ed38f81845e9a5c318bb78a0549b7"} Dec 03 13:19:56 crc kubenswrapper[5052]: I1203 13:19:56.412439 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c2cf01fb297b919adc2fbdc44a629c1cd07ed38f81845e9a5c318bb78a0549b7" Dec 03 13:19:59 crc kubenswrapper[5052]: I1203 13:19:59.677325 5052 scope.go:117] "RemoveContainer" containerID="12a8b1f470f6d59d7700e376f2a48eb36363f91f9823a35ce534062c0974c10c" Dec 03 13:19:59 crc kubenswrapper[5052]: E1203 13:19:59.678005 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 13:20:11 crc kubenswrapper[5052]: I1203 13:20:11.719936 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-operators/mariadb-operator-controller-manager-759d8858f9-9bhzf"] Dec 03 13:20:11 crc kubenswrapper[5052]: E1203 13:20:11.720914 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="a48b1dcb-27f2-4c5b-b735-cc1de58a1384" containerName="extract" Dec 03 13:20:11 crc kubenswrapper[5052]: I1203 13:20:11.720934 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="a48b1dcb-27f2-4c5b-b735-cc1de58a1384" containerName="extract" Dec 03 13:20:11 crc kubenswrapper[5052]: E1203 13:20:11.720970 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="a48b1dcb-27f2-4c5b-b735-cc1de58a1384" containerName="pull" Dec 03 13:20:11 crc kubenswrapper[5052]: I1203 13:20:11.720978 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="a48b1dcb-27f2-4c5b-b735-cc1de58a1384" containerName="pull" Dec 03 13:20:11 crc kubenswrapper[5052]: E1203 13:20:11.720992 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="a48b1dcb-27f2-4c5b-b735-cc1de58a1384" containerName="util" Dec 03 13:20:11 crc kubenswrapper[5052]: I1203 13:20:11.721000 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="a48b1dcb-27f2-4c5b-b735-cc1de58a1384" containerName="util" Dec 03 13:20:11 crc kubenswrapper[5052]: I1203 13:20:11.721144 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="a48b1dcb-27f2-4c5b-b735-cc1de58a1384" containerName="extract" Dec 03 13:20:11 crc kubenswrapper[5052]: I1203 13:20:11.721738 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-operators/mariadb-operator-controller-manager-759d8858f9-9bhzf" Dec 03 13:20:11 crc kubenswrapper[5052]: I1203 13:20:11.725444 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-operators"/"mariadb-operator-controller-manager-service-cert" Dec 03 13:20:11 crc kubenswrapper[5052]: I1203 13:20:11.730835 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-operators"/"mariadb-operator-controller-manager-dockercfg-sqjjx" Dec 03 13:20:11 crc kubenswrapper[5052]: I1203 13:20:11.731045 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-operators"/"webhook-server-cert" Dec 03 13:20:11 crc kubenswrapper[5052]: I1203 13:20:11.737149 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-operators/mariadb-operator-controller-manager-759d8858f9-9bhzf"] Dec 03 13:20:11 crc kubenswrapper[5052]: I1203 13:20:11.783838 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cgtqq\" (UniqueName: \"kubernetes.io/projected/44402132-c7a3-4a0b-a87e-eea057961099-kube-api-access-cgtqq\") pod \"mariadb-operator-controller-manager-759d8858f9-9bhzf\" (UID: \"44402132-c7a3-4a0b-a87e-eea057961099\") " pod="openstack-operators/mariadb-operator-controller-manager-759d8858f9-9bhzf" Dec 03 13:20:11 crc kubenswrapper[5052]: I1203 13:20:11.783963 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/44402132-c7a3-4a0b-a87e-eea057961099-webhook-cert\") pod \"mariadb-operator-controller-manager-759d8858f9-9bhzf\" (UID: \"44402132-c7a3-4a0b-a87e-eea057961099\") " pod="openstack-operators/mariadb-operator-controller-manager-759d8858f9-9bhzf" Dec 03 13:20:11 crc kubenswrapper[5052]: I1203 13:20:11.784204 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/44402132-c7a3-4a0b-a87e-eea057961099-apiservice-cert\") pod \"mariadb-operator-controller-manager-759d8858f9-9bhzf\" (UID: \"44402132-c7a3-4a0b-a87e-eea057961099\") " pod="openstack-operators/mariadb-operator-controller-manager-759d8858f9-9bhzf" Dec 03 13:20:11 crc kubenswrapper[5052]: I1203 13:20:11.885039 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/44402132-c7a3-4a0b-a87e-eea057961099-webhook-cert\") pod \"mariadb-operator-controller-manager-759d8858f9-9bhzf\" (UID: \"44402132-c7a3-4a0b-a87e-eea057961099\") " pod="openstack-operators/mariadb-operator-controller-manager-759d8858f9-9bhzf" Dec 03 13:20:11 crc kubenswrapper[5052]: I1203 13:20:11.885115 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/44402132-c7a3-4a0b-a87e-eea057961099-apiservice-cert\") pod \"mariadb-operator-controller-manager-759d8858f9-9bhzf\" (UID: \"44402132-c7a3-4a0b-a87e-eea057961099\") " pod="openstack-operators/mariadb-operator-controller-manager-759d8858f9-9bhzf" Dec 03 13:20:11 crc kubenswrapper[5052]: I1203 13:20:11.885160 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-cgtqq\" (UniqueName: \"kubernetes.io/projected/44402132-c7a3-4a0b-a87e-eea057961099-kube-api-access-cgtqq\") pod \"mariadb-operator-controller-manager-759d8858f9-9bhzf\" (UID: \"44402132-c7a3-4a0b-a87e-eea057961099\") " pod="openstack-operators/mariadb-operator-controller-manager-759d8858f9-9bhzf" Dec 03 13:20:11 crc kubenswrapper[5052]: I1203 13:20:11.890806 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/44402132-c7a3-4a0b-a87e-eea057961099-apiservice-cert\") pod \"mariadb-operator-controller-manager-759d8858f9-9bhzf\" (UID: \"44402132-c7a3-4a0b-a87e-eea057961099\") " pod="openstack-operators/mariadb-operator-controller-manager-759d8858f9-9bhzf" Dec 03 13:20:11 crc kubenswrapper[5052]: I1203 13:20:11.898232 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/44402132-c7a3-4a0b-a87e-eea057961099-webhook-cert\") pod \"mariadb-operator-controller-manager-759d8858f9-9bhzf\" (UID: \"44402132-c7a3-4a0b-a87e-eea057961099\") " pod="openstack-operators/mariadb-operator-controller-manager-759d8858f9-9bhzf" Dec 03 13:20:11 crc kubenswrapper[5052]: I1203 13:20:11.900849 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-cgtqq\" (UniqueName: \"kubernetes.io/projected/44402132-c7a3-4a0b-a87e-eea057961099-kube-api-access-cgtqq\") pod \"mariadb-operator-controller-manager-759d8858f9-9bhzf\" (UID: \"44402132-c7a3-4a0b-a87e-eea057961099\") " pod="openstack-operators/mariadb-operator-controller-manager-759d8858f9-9bhzf" Dec 03 13:20:12 crc kubenswrapper[5052]: I1203 13:20:12.048675 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-operators/mariadb-operator-controller-manager-759d8858f9-9bhzf" Dec 03 13:20:12 crc kubenswrapper[5052]: I1203 13:20:12.450349 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-operators/mariadb-operator-controller-manager-759d8858f9-9bhzf"] Dec 03 13:20:12 crc kubenswrapper[5052]: W1203 13:20:12.454377 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44402132_c7a3_4a0b_a87e_eea057961099.slice/crio-7ca5429362c6e00d430dd7065a8c0ad514c63681f9fdeafca1d1e07e52ad06d8 WatchSource:0}: Error finding container 7ca5429362c6e00d430dd7065a8c0ad514c63681f9fdeafca1d1e07e52ad06d8: Status 404 returned error can't find the container with id 7ca5429362c6e00d430dd7065a8c0ad514c63681f9fdeafca1d1e07e52ad06d8 Dec 03 13:20:12 crc kubenswrapper[5052]: I1203 13:20:12.531997 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/mariadb-operator-controller-manager-759d8858f9-9bhzf" event={"ID":"44402132-c7a3-4a0b-a87e-eea057961099","Type":"ContainerStarted","Data":"7ca5429362c6e00d430dd7065a8c0ad514c63681f9fdeafca1d1e07e52ad06d8"} Dec 03 13:20:13 crc kubenswrapper[5052]: I1203 13:20:13.549692 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/mariadb-operator-controller-manager-759d8858f9-9bhzf" event={"ID":"44402132-c7a3-4a0b-a87e-eea057961099","Type":"ContainerStarted","Data":"8bcd92e03839ee1b954a6b77c88cef613ce2e0ffcf2f824f7e2db180bc50f14d"} Dec 03 13:20:13 crc kubenswrapper[5052]: I1203 13:20:13.551496 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-operators/mariadb-operator-controller-manager-759d8858f9-9bhzf" Dec 03 13:20:13 crc kubenswrapper[5052]: I1203 13:20:13.582282 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-operators/mariadb-operator-controller-manager-759d8858f9-9bhzf" podStartSLOduration=2.582260331 podStartE2EDuration="2.582260331s" podCreationTimestamp="2025-12-03 13:20:11 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 13:20:13.569194057 +0000 UTC m=+7204.157435821" watchObservedRunningTime="2025-12-03 13:20:13.582260331 +0000 UTC m=+7204.170502095" Dec 03 13:20:14 crc kubenswrapper[5052]: I1203 13:20:14.673049 5052 scope.go:117] "RemoveContainer" containerID="12a8b1f470f6d59d7700e376f2a48eb36363f91f9823a35ce534062c0974c10c" Dec 03 13:20:14 crc kubenswrapper[5052]: E1203 13:20:14.673602 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 13:20:22 crc kubenswrapper[5052]: I1203 13:20:22.054002 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-operators/mariadb-operator-controller-manager-759d8858f9-9bhzf" Dec 03 13:20:26 crc kubenswrapper[5052]: I1203 13:20:26.068964 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-operators/infra-operator-index-zbc9s"] Dec 03 13:20:26 crc kubenswrapper[5052]: I1203 13:20:26.071605 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-operators/infra-operator-index-zbc9s" Dec 03 13:20:26 crc kubenswrapper[5052]: I1203 13:20:26.074239 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-operators"/"infra-operator-index-dockercfg-mrnbl" Dec 03 13:20:26 crc kubenswrapper[5052]: I1203 13:20:26.077871 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-operators/infra-operator-index-zbc9s"] Dec 03 13:20:26 crc kubenswrapper[5052]: I1203 13:20:26.157776 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xksft\" (UniqueName: \"kubernetes.io/projected/f6d49863-5cbf-4793-804b-dc8ea2a3c853-kube-api-access-xksft\") pod \"infra-operator-index-zbc9s\" (UID: \"f6d49863-5cbf-4793-804b-dc8ea2a3c853\") " pod="openstack-operators/infra-operator-index-zbc9s" Dec 03 13:20:26 crc kubenswrapper[5052]: I1203 13:20:26.259755 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-xksft\" (UniqueName: \"kubernetes.io/projected/f6d49863-5cbf-4793-804b-dc8ea2a3c853-kube-api-access-xksft\") pod \"infra-operator-index-zbc9s\" (UID: \"f6d49863-5cbf-4793-804b-dc8ea2a3c853\") " pod="openstack-operators/infra-operator-index-zbc9s" Dec 03 13:20:26 crc kubenswrapper[5052]: I1203 13:20:26.284563 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-xksft\" (UniqueName: \"kubernetes.io/projected/f6d49863-5cbf-4793-804b-dc8ea2a3c853-kube-api-access-xksft\") pod \"infra-operator-index-zbc9s\" (UID: \"f6d49863-5cbf-4793-804b-dc8ea2a3c853\") " pod="openstack-operators/infra-operator-index-zbc9s" Dec 03 13:20:26 crc kubenswrapper[5052]: I1203 13:20:26.409010 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-operators/infra-operator-index-zbc9s" Dec 03 13:20:26 crc kubenswrapper[5052]: I1203 13:20:26.843620 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-operators/infra-operator-index-zbc9s"] Dec 03 13:20:26 crc kubenswrapper[5052]: W1203 13:20:26.851735 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6d49863_5cbf_4793_804b_dc8ea2a3c853.slice/crio-31009f98eca56e4e806d47ab48f48503abb3676477939328caac75d34243c4bf WatchSource:0}: Error finding container 31009f98eca56e4e806d47ab48f48503abb3676477939328caac75d34243c4bf: Status 404 returned error can't find the container with id 31009f98eca56e4e806d47ab48f48503abb3676477939328caac75d34243c4bf Dec 03 13:20:27 crc kubenswrapper[5052]: I1203 13:20:27.850837 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/infra-operator-index-zbc9s" event={"ID":"f6d49863-5cbf-4793-804b-dc8ea2a3c853","Type":"ContainerStarted","Data":"686719df992387cbb21764a83eb165ccdc0ed1c12420db888a3c62bd9e89a73f"} Dec 03 13:20:27 crc kubenswrapper[5052]: I1203 13:20:27.851184 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/infra-operator-index-zbc9s" event={"ID":"f6d49863-5cbf-4793-804b-dc8ea2a3c853","Type":"ContainerStarted","Data":"31009f98eca56e4e806d47ab48f48503abb3676477939328caac75d34243c4bf"} Dec 03 13:20:27 crc kubenswrapper[5052]: I1203 13:20:27.868429 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-operators/infra-operator-index-zbc9s" podStartSLOduration=1.192067904 podStartE2EDuration="1.868411883s" podCreationTimestamp="2025-12-03 13:20:26 +0000 UTC" firstStartedPulling="2025-12-03 13:20:26.85426276 +0000 UTC m=+7217.442504524" lastFinishedPulling="2025-12-03 13:20:27.530606739 +0000 UTC m=+7218.118848503" observedRunningTime="2025-12-03 13:20:27.863610366 +0000 UTC m=+7218.451852150" watchObservedRunningTime="2025-12-03 13:20:27.868411883 +0000 UTC m=+7218.456653647" Dec 03 13:20:29 crc kubenswrapper[5052]: I1203 13:20:29.676344 5052 scope.go:117] "RemoveContainer" containerID="12a8b1f470f6d59d7700e376f2a48eb36363f91f9823a35ce534062c0974c10c" Dec 03 13:20:29 crc kubenswrapper[5052]: E1203 13:20:29.676641 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 13:20:29 crc kubenswrapper[5052]: I1203 13:20:29.847425 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-operators/infra-operator-index-zbc9s"] Dec 03 13:20:29 crc kubenswrapper[5052]: I1203 13:20:29.863766 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-operators/infra-operator-index-zbc9s" podUID="f6d49863-5cbf-4793-804b-dc8ea2a3c853" containerName="registry-server" containerID="cri-o://686719df992387cbb21764a83eb165ccdc0ed1c12420db888a3c62bd9e89a73f" gracePeriod=2 Dec 03 13:20:30 crc kubenswrapper[5052]: I1203 13:20:30.241187 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-operators/infra-operator-index-zbc9s" Dec 03 13:20:30 crc kubenswrapper[5052]: I1203 13:20:30.315100 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xksft\" (UniqueName: \"kubernetes.io/projected/f6d49863-5cbf-4793-804b-dc8ea2a3c853-kube-api-access-xksft\") pod \"f6d49863-5cbf-4793-804b-dc8ea2a3c853\" (UID: \"f6d49863-5cbf-4793-804b-dc8ea2a3c853\") " Dec 03 13:20:30 crc kubenswrapper[5052]: I1203 13:20:30.319756 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f6d49863-5cbf-4793-804b-dc8ea2a3c853-kube-api-access-xksft" (OuterVolumeSpecName: "kube-api-access-xksft") pod "f6d49863-5cbf-4793-804b-dc8ea2a3c853" (UID: "f6d49863-5cbf-4793-804b-dc8ea2a3c853"). InnerVolumeSpecName "kube-api-access-xksft". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:20:30 crc kubenswrapper[5052]: I1203 13:20:30.416765 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-xksft\" (UniqueName: \"kubernetes.io/projected/f6d49863-5cbf-4793-804b-dc8ea2a3c853-kube-api-access-xksft\") on node \"crc\" DevicePath \"\"" Dec 03 13:20:30 crc kubenswrapper[5052]: I1203 13:20:30.656784 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-operators/infra-operator-index-2dtcx"] Dec 03 13:20:30 crc kubenswrapper[5052]: E1203 13:20:30.657125 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="f6d49863-5cbf-4793-804b-dc8ea2a3c853" containerName="registry-server" Dec 03 13:20:30 crc kubenswrapper[5052]: I1203 13:20:30.657145 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="f6d49863-5cbf-4793-804b-dc8ea2a3c853" containerName="registry-server" Dec 03 13:20:30 crc kubenswrapper[5052]: I1203 13:20:30.657294 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="f6d49863-5cbf-4793-804b-dc8ea2a3c853" containerName="registry-server" Dec 03 13:20:30 crc kubenswrapper[5052]: I1203 13:20:30.657751 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-operators/infra-operator-index-2dtcx" Dec 03 13:20:30 crc kubenswrapper[5052]: I1203 13:20:30.666650 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-operators/infra-operator-index-2dtcx"] Dec 03 13:20:30 crc kubenswrapper[5052]: I1203 13:20:30.720249 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r4h8w\" (UniqueName: \"kubernetes.io/projected/75a90895-a53c-4c1f-8d5b-fd859d61a681-kube-api-access-r4h8w\") pod \"infra-operator-index-2dtcx\" (UID: \"75a90895-a53c-4c1f-8d5b-fd859d61a681\") " pod="openstack-operators/infra-operator-index-2dtcx" Dec 03 13:20:30 crc kubenswrapper[5052]: I1203 13:20:30.821948 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-r4h8w\" (UniqueName: \"kubernetes.io/projected/75a90895-a53c-4c1f-8d5b-fd859d61a681-kube-api-access-r4h8w\") pod \"infra-operator-index-2dtcx\" (UID: \"75a90895-a53c-4c1f-8d5b-fd859d61a681\") " pod="openstack-operators/infra-operator-index-2dtcx" Dec 03 13:20:30 crc kubenswrapper[5052]: I1203 13:20:30.838488 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-r4h8w\" (UniqueName: \"kubernetes.io/projected/75a90895-a53c-4c1f-8d5b-fd859d61a681-kube-api-access-r4h8w\") pod \"infra-operator-index-2dtcx\" (UID: \"75a90895-a53c-4c1f-8d5b-fd859d61a681\") " pod="openstack-operators/infra-operator-index-2dtcx" Dec 03 13:20:30 crc kubenswrapper[5052]: I1203 13:20:30.871936 5052 generic.go:334] "Generic (PLEG): container finished" podID="f6d49863-5cbf-4793-804b-dc8ea2a3c853" containerID="686719df992387cbb21764a83eb165ccdc0ed1c12420db888a3c62bd9e89a73f" exitCode=0 Dec 03 13:20:30 crc kubenswrapper[5052]: I1203 13:20:30.871972 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/infra-operator-index-zbc9s" event={"ID":"f6d49863-5cbf-4793-804b-dc8ea2a3c853","Type":"ContainerDied","Data":"686719df992387cbb21764a83eb165ccdc0ed1c12420db888a3c62bd9e89a73f"} Dec 03 13:20:30 crc kubenswrapper[5052]: I1203 13:20:30.872000 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/infra-operator-index-zbc9s" event={"ID":"f6d49863-5cbf-4793-804b-dc8ea2a3c853","Type":"ContainerDied","Data":"31009f98eca56e4e806d47ab48f48503abb3676477939328caac75d34243c4bf"} Dec 03 13:20:30 crc kubenswrapper[5052]: I1203 13:20:30.871997 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-operators/infra-operator-index-zbc9s" Dec 03 13:20:30 crc kubenswrapper[5052]: I1203 13:20:30.872014 5052 scope.go:117] "RemoveContainer" containerID="686719df992387cbb21764a83eb165ccdc0ed1c12420db888a3c62bd9e89a73f" Dec 03 13:20:30 crc kubenswrapper[5052]: I1203 13:20:30.888987 5052 scope.go:117] "RemoveContainer" containerID="686719df992387cbb21764a83eb165ccdc0ed1c12420db888a3c62bd9e89a73f" Dec 03 13:20:30 crc kubenswrapper[5052]: E1203 13:20:30.889590 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"686719df992387cbb21764a83eb165ccdc0ed1c12420db888a3c62bd9e89a73f\": container with ID starting with 686719df992387cbb21764a83eb165ccdc0ed1c12420db888a3c62bd9e89a73f not found: ID does not exist" containerID="686719df992387cbb21764a83eb165ccdc0ed1c12420db888a3c62bd9e89a73f" Dec 03 13:20:30 crc kubenswrapper[5052]: I1203 13:20:30.889619 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"686719df992387cbb21764a83eb165ccdc0ed1c12420db888a3c62bd9e89a73f"} err="failed to get container status \"686719df992387cbb21764a83eb165ccdc0ed1c12420db888a3c62bd9e89a73f\": rpc error: code = NotFound desc = could not find container \"686719df992387cbb21764a83eb165ccdc0ed1c12420db888a3c62bd9e89a73f\": container with ID starting with 686719df992387cbb21764a83eb165ccdc0ed1c12420db888a3c62bd9e89a73f not found: ID does not exist" Dec 03 13:20:30 crc kubenswrapper[5052]: I1203 13:20:30.914207 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-operators/infra-operator-index-zbc9s"] Dec 03 13:20:30 crc kubenswrapper[5052]: I1203 13:20:30.918951 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-operators/infra-operator-index-zbc9s"] Dec 03 13:20:30 crc kubenswrapper[5052]: I1203 13:20:30.974893 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-operators/infra-operator-index-2dtcx" Dec 03 13:20:31 crc kubenswrapper[5052]: I1203 13:20:31.387225 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-operators/infra-operator-index-2dtcx"] Dec 03 13:20:31 crc kubenswrapper[5052]: I1203 13:20:31.682251 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f6d49863-5cbf-4793-804b-dc8ea2a3c853" path="/var/lib/kubelet/pods/f6d49863-5cbf-4793-804b-dc8ea2a3c853/volumes" Dec 03 13:20:31 crc kubenswrapper[5052]: I1203 13:20:31.880272 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/infra-operator-index-2dtcx" event={"ID":"75a90895-a53c-4c1f-8d5b-fd859d61a681","Type":"ContainerStarted","Data":"2a55ec0d603e2c7bf8960f5e2103d87a4dcc4355900639ea4d2f614536e13871"} Dec 03 13:20:32 crc kubenswrapper[5052]: I1203 13:20:32.707111 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["keystone-kuttl-tests/openstack-galera-0"] Dec 03 13:20:32 crc kubenswrapper[5052]: I1203 13:20:32.708304 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="keystone-kuttl-tests/openstack-galera-0" Dec 03 13:20:32 crc kubenswrapper[5052]: I1203 13:20:32.710910 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"keystone-kuttl-tests"/"openstack-scripts" Dec 03 13:20:32 crc kubenswrapper[5052]: I1203 13:20:32.711188 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"keystone-kuttl-tests"/"openstack-config-data" Dec 03 13:20:32 crc kubenswrapper[5052]: I1203 13:20:32.714544 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"keystone-kuttl-tests"/"openshift-service-ca.crt" Dec 03 13:20:32 crc kubenswrapper[5052]: I1203 13:20:32.714971 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"keystone-kuttl-tests"/"kube-root-ca.crt" Dec 03 13:20:32 crc kubenswrapper[5052]: I1203 13:20:32.715196 5052 reflector.go:368] Caches populated for *v1.Secret from object-"keystone-kuttl-tests"/"galera-openstack-dockercfg-7c78s" Dec 03 13:20:32 crc kubenswrapper[5052]: I1203 13:20:32.717226 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["keystone-kuttl-tests/openstack-galera-0"] Dec 03 13:20:32 crc kubenswrapper[5052]: I1203 13:20:32.723561 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["keystone-kuttl-tests/openstack-galera-1"] Dec 03 13:20:32 crc kubenswrapper[5052]: I1203 13:20:32.724866 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="keystone-kuttl-tests/openstack-galera-1" Dec 03 13:20:32 crc kubenswrapper[5052]: I1203 13:20:32.728424 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["keystone-kuttl-tests/openstack-galera-2"] Dec 03 13:20:32 crc kubenswrapper[5052]: I1203 13:20:32.729434 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="keystone-kuttl-tests/openstack-galera-2" Dec 03 13:20:32 crc kubenswrapper[5052]: I1203 13:20:32.756009 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["keystone-kuttl-tests/openstack-galera-1"] Dec 03 13:20:32 crc kubenswrapper[5052]: I1203 13:20:32.764063 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["keystone-kuttl-tests/openstack-galera-2"] Dec 03 13:20:32 crc kubenswrapper[5052]: I1203 13:20:32.849622 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/2e771f67-f156-4dec-a77f-88b4735f298b-kolla-config\") pod \"openstack-galera-2\" (UID: \"2e771f67-f156-4dec-a77f-88b4735f298b\") " pod="keystone-kuttl-tests/openstack-galera-2" Dec 03 13:20:32 crc kubenswrapper[5052]: I1203 13:20:32.849683 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage14-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage14-crc\") pod \"openstack-galera-2\" (UID: \"2e771f67-f156-4dec-a77f-88b4735f298b\") " pod="keystone-kuttl-tests/openstack-galera-2" Dec 03 13:20:32 crc kubenswrapper[5052]: I1203 13:20:32.849704 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/36e1b611-408e-4cc3-8531-03adcac84683-operator-scripts\") pod \"openstack-galera-0\" (UID: \"36e1b611-408e-4cc3-8531-03adcac84683\") " pod="keystone-kuttl-tests/openstack-galera-0" Dec 03 13:20:32 crc kubenswrapper[5052]: I1203 13:20:32.849728 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-27nwd\" (UniqueName: \"kubernetes.io/projected/0ea9b78b-2beb-4bdb-852c-1b92b9638551-kube-api-access-27nwd\") pod \"openstack-galera-1\" (UID: \"0ea9b78b-2beb-4bdb-852c-1b92b9638551\") " pod="keystone-kuttl-tests/openstack-galera-1" Dec 03 13:20:32 crc kubenswrapper[5052]: I1203 13:20:32.849780 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/36e1b611-408e-4cc3-8531-03adcac84683-config-data-default\") pod \"openstack-galera-0\" (UID: \"36e1b611-408e-4cc3-8531-03adcac84683\") " pod="keystone-kuttl-tests/openstack-galera-0" Dec 03 13:20:32 crc kubenswrapper[5052]: I1203 13:20:32.849802 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/0ea9b78b-2beb-4bdb-852c-1b92b9638551-operator-scripts\") pod \"openstack-galera-1\" (UID: \"0ea9b78b-2beb-4bdb-852c-1b92b9638551\") " pod="keystone-kuttl-tests/openstack-galera-1" Dec 03 13:20:32 crc kubenswrapper[5052]: I1203 13:20:32.849826 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/0ea9b78b-2beb-4bdb-852c-1b92b9638551-config-data-generated\") pod \"openstack-galera-1\" (UID: \"0ea9b78b-2beb-4bdb-852c-1b92b9638551\") " pod="keystone-kuttl-tests/openstack-galera-1" Dec 03 13:20:32 crc kubenswrapper[5052]: I1203 13:20:32.849853 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/2e771f67-f156-4dec-a77f-88b4735f298b-operator-scripts\") pod \"openstack-galera-2\" (UID: \"2e771f67-f156-4dec-a77f-88b4735f298b\") " pod="keystone-kuttl-tests/openstack-galera-2" Dec 03 13:20:32 crc kubenswrapper[5052]: I1203 13:20:32.849881 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xzkxh\" (UniqueName: \"kubernetes.io/projected/2e771f67-f156-4dec-a77f-88b4735f298b-kube-api-access-xzkxh\") pod \"openstack-galera-2\" (UID: \"2e771f67-f156-4dec-a77f-88b4735f298b\") " pod="keystone-kuttl-tests/openstack-galera-2" Dec 03 13:20:32 crc kubenswrapper[5052]: I1203 13:20:32.849900 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/0ea9b78b-2beb-4bdb-852c-1b92b9638551-config-data-default\") pod \"openstack-galera-1\" (UID: \"0ea9b78b-2beb-4bdb-852c-1b92b9638551\") " pod="keystone-kuttl-tests/openstack-galera-1" Dec 03 13:20:32 crc kubenswrapper[5052]: I1203 13:20:32.849923 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2lf8x\" (UniqueName: \"kubernetes.io/projected/36e1b611-408e-4cc3-8531-03adcac84683-kube-api-access-2lf8x\") pod \"openstack-galera-0\" (UID: \"36e1b611-408e-4cc3-8531-03adcac84683\") " pod="keystone-kuttl-tests/openstack-galera-0" Dec 03 13:20:32 crc kubenswrapper[5052]: I1203 13:20:32.849941 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/2e771f67-f156-4dec-a77f-88b4735f298b-config-data-generated\") pod \"openstack-galera-2\" (UID: \"2e771f67-f156-4dec-a77f-88b4735f298b\") " pod="keystone-kuttl-tests/openstack-galera-2" Dec 03 13:20:32 crc kubenswrapper[5052]: I1203 13:20:32.849960 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage13-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage13-crc\") pod \"openstack-galera-0\" (UID: \"36e1b611-408e-4cc3-8531-03adcac84683\") " pod="keystone-kuttl-tests/openstack-galera-0" Dec 03 13:20:32 crc kubenswrapper[5052]: I1203 13:20:32.849978 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/36e1b611-408e-4cc3-8531-03adcac84683-config-data-generated\") pod \"openstack-galera-0\" (UID: \"36e1b611-408e-4cc3-8531-03adcac84683\") " pod="keystone-kuttl-tests/openstack-galera-0" Dec 03 13:20:32 crc kubenswrapper[5052]: I1203 13:20:32.849994 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/0ea9b78b-2beb-4bdb-852c-1b92b9638551-kolla-config\") pod \"openstack-galera-1\" (UID: \"0ea9b78b-2beb-4bdb-852c-1b92b9638551\") " pod="keystone-kuttl-tests/openstack-galera-1" Dec 03 13:20:32 crc kubenswrapper[5052]: I1203 13:20:32.850011 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage01-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage01-crc\") pod \"openstack-galera-1\" (UID: \"0ea9b78b-2beb-4bdb-852c-1b92b9638551\") " pod="keystone-kuttl-tests/openstack-galera-1" Dec 03 13:20:32 crc kubenswrapper[5052]: I1203 13:20:32.850036 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/2e771f67-f156-4dec-a77f-88b4735f298b-config-data-default\") pod \"openstack-galera-2\" (UID: \"2e771f67-f156-4dec-a77f-88b4735f298b\") " pod="keystone-kuttl-tests/openstack-galera-2" Dec 03 13:20:32 crc kubenswrapper[5052]: I1203 13:20:32.850061 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/36e1b611-408e-4cc3-8531-03adcac84683-kolla-config\") pod \"openstack-galera-0\" (UID: \"36e1b611-408e-4cc3-8531-03adcac84683\") " pod="keystone-kuttl-tests/openstack-galera-0" Dec 03 13:20:32 crc kubenswrapper[5052]: I1203 13:20:32.889170 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/infra-operator-index-2dtcx" event={"ID":"75a90895-a53c-4c1f-8d5b-fd859d61a681","Type":"ContainerStarted","Data":"36bea13f9e88148b806bf93382abfb0f19013d1939bd07876c9afc32b9057387"} Dec 03 13:20:32 crc kubenswrapper[5052]: I1203 13:20:32.906756 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-operators/infra-operator-index-2dtcx" podStartSLOduration=2.441280799 podStartE2EDuration="2.906735289s" podCreationTimestamp="2025-12-03 13:20:30 +0000 UTC" firstStartedPulling="2025-12-03 13:20:31.395537672 +0000 UTC m=+7221.983779426" lastFinishedPulling="2025-12-03 13:20:31.860992142 +0000 UTC m=+7222.449233916" observedRunningTime="2025-12-03 13:20:32.900864901 +0000 UTC m=+7223.489106685" watchObservedRunningTime="2025-12-03 13:20:32.906735289 +0000 UTC m=+7223.494977053" Dec 03 13:20:32 crc kubenswrapper[5052]: I1203 13:20:32.951567 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/2e771f67-f156-4dec-a77f-88b4735f298b-kolla-config\") pod \"openstack-galera-2\" (UID: \"2e771f67-f156-4dec-a77f-88b4735f298b\") " pod="keystone-kuttl-tests/openstack-galera-2" Dec 03 13:20:32 crc kubenswrapper[5052]: I1203 13:20:32.951630 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage14-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage14-crc\") pod \"openstack-galera-2\" (UID: \"2e771f67-f156-4dec-a77f-88b4735f298b\") " pod="keystone-kuttl-tests/openstack-galera-2" Dec 03 13:20:32 crc kubenswrapper[5052]: I1203 13:20:32.951663 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/36e1b611-408e-4cc3-8531-03adcac84683-operator-scripts\") pod \"openstack-galera-0\" (UID: \"36e1b611-408e-4cc3-8531-03adcac84683\") " pod="keystone-kuttl-tests/openstack-galera-0" Dec 03 13:20:32 crc kubenswrapper[5052]: I1203 13:20:32.951695 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-27nwd\" (UniqueName: \"kubernetes.io/projected/0ea9b78b-2beb-4bdb-852c-1b92b9638551-kube-api-access-27nwd\") pod \"openstack-galera-1\" (UID: \"0ea9b78b-2beb-4bdb-852c-1b92b9638551\") " pod="keystone-kuttl-tests/openstack-galera-1" Dec 03 13:20:32 crc kubenswrapper[5052]: I1203 13:20:32.951726 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/36e1b611-408e-4cc3-8531-03adcac84683-config-data-default\") pod \"openstack-galera-0\" (UID: \"36e1b611-408e-4cc3-8531-03adcac84683\") " pod="keystone-kuttl-tests/openstack-galera-0" Dec 03 13:20:32 crc kubenswrapper[5052]: I1203 13:20:32.951775 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/0ea9b78b-2beb-4bdb-852c-1b92b9638551-operator-scripts\") pod \"openstack-galera-1\" (UID: \"0ea9b78b-2beb-4bdb-852c-1b92b9638551\") " pod="keystone-kuttl-tests/openstack-galera-1" Dec 03 13:20:32 crc kubenswrapper[5052]: I1203 13:20:32.951804 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/0ea9b78b-2beb-4bdb-852c-1b92b9638551-config-data-generated\") pod \"openstack-galera-1\" (UID: \"0ea9b78b-2beb-4bdb-852c-1b92b9638551\") " pod="keystone-kuttl-tests/openstack-galera-1" Dec 03 13:20:32 crc kubenswrapper[5052]: I1203 13:20:32.951851 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/2e771f67-f156-4dec-a77f-88b4735f298b-operator-scripts\") pod \"openstack-galera-2\" (UID: \"2e771f67-f156-4dec-a77f-88b4735f298b\") " pod="keystone-kuttl-tests/openstack-galera-2" Dec 03 13:20:32 crc kubenswrapper[5052]: I1203 13:20:32.951886 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-xzkxh\" (UniqueName: \"kubernetes.io/projected/2e771f67-f156-4dec-a77f-88b4735f298b-kube-api-access-xzkxh\") pod \"openstack-galera-2\" (UID: \"2e771f67-f156-4dec-a77f-88b4735f298b\") " pod="keystone-kuttl-tests/openstack-galera-2" Dec 03 13:20:32 crc kubenswrapper[5052]: I1203 13:20:32.951910 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/0ea9b78b-2beb-4bdb-852c-1b92b9638551-config-data-default\") pod \"openstack-galera-1\" (UID: \"0ea9b78b-2beb-4bdb-852c-1b92b9638551\") " pod="keystone-kuttl-tests/openstack-galera-1" Dec 03 13:20:32 crc kubenswrapper[5052]: I1203 13:20:32.951937 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-2lf8x\" (UniqueName: \"kubernetes.io/projected/36e1b611-408e-4cc3-8531-03adcac84683-kube-api-access-2lf8x\") pod \"openstack-galera-0\" (UID: \"36e1b611-408e-4cc3-8531-03adcac84683\") " pod="keystone-kuttl-tests/openstack-galera-0" Dec 03 13:20:32 crc kubenswrapper[5052]: I1203 13:20:32.951961 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/2e771f67-f156-4dec-a77f-88b4735f298b-config-data-generated\") pod \"openstack-galera-2\" (UID: \"2e771f67-f156-4dec-a77f-88b4735f298b\") " pod="keystone-kuttl-tests/openstack-galera-2" Dec 03 13:20:32 crc kubenswrapper[5052]: I1203 13:20:32.951983 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage13-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage13-crc\") pod \"openstack-galera-0\" (UID: \"36e1b611-408e-4cc3-8531-03adcac84683\") " pod="keystone-kuttl-tests/openstack-galera-0" Dec 03 13:20:32 crc kubenswrapper[5052]: I1203 13:20:32.952011 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/36e1b611-408e-4cc3-8531-03adcac84683-config-data-generated\") pod \"openstack-galera-0\" (UID: \"36e1b611-408e-4cc3-8531-03adcac84683\") " pod="keystone-kuttl-tests/openstack-galera-0" Dec 03 13:20:32 crc kubenswrapper[5052]: I1203 13:20:32.952037 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/0ea9b78b-2beb-4bdb-852c-1b92b9638551-kolla-config\") pod \"openstack-galera-1\" (UID: \"0ea9b78b-2beb-4bdb-852c-1b92b9638551\") " pod="keystone-kuttl-tests/openstack-galera-1" Dec 03 13:20:32 crc kubenswrapper[5052]: I1203 13:20:32.952061 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage01-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage01-crc\") pod \"openstack-galera-1\" (UID: \"0ea9b78b-2beb-4bdb-852c-1b92b9638551\") " pod="keystone-kuttl-tests/openstack-galera-1" Dec 03 13:20:32 crc kubenswrapper[5052]: I1203 13:20:32.952096 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/2e771f67-f156-4dec-a77f-88b4735f298b-config-data-default\") pod \"openstack-galera-2\" (UID: \"2e771f67-f156-4dec-a77f-88b4735f298b\") " pod="keystone-kuttl-tests/openstack-galera-2" Dec 03 13:20:32 crc kubenswrapper[5052]: I1203 13:20:32.952125 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/36e1b611-408e-4cc3-8531-03adcac84683-kolla-config\") pod \"openstack-galera-0\" (UID: \"36e1b611-408e-4cc3-8531-03adcac84683\") " pod="keystone-kuttl-tests/openstack-galera-0" Dec 03 13:20:32 crc kubenswrapper[5052]: I1203 13:20:32.952395 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage14-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage14-crc\") pod \"openstack-galera-2\" (UID: \"2e771f67-f156-4dec-a77f-88b4735f298b\") device mount path \"/mnt/openstack/pv14\"" pod="keystone-kuttl-tests/openstack-galera-2" Dec 03 13:20:32 crc kubenswrapper[5052]: I1203 13:20:32.952491 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/2e771f67-f156-4dec-a77f-88b4735f298b-kolla-config\") pod \"openstack-galera-2\" (UID: \"2e771f67-f156-4dec-a77f-88b4735f298b\") " pod="keystone-kuttl-tests/openstack-galera-2" Dec 03 13:20:32 crc kubenswrapper[5052]: I1203 13:20:32.952685 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage13-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage13-crc\") pod \"openstack-galera-0\" (UID: \"36e1b611-408e-4cc3-8531-03adcac84683\") device mount path \"/mnt/openstack/pv13\"" pod="keystone-kuttl-tests/openstack-galera-0" Dec 03 13:20:32 crc kubenswrapper[5052]: I1203 13:20:32.952820 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/36e1b611-408e-4cc3-8531-03adcac84683-kolla-config\") pod \"openstack-galera-0\" (UID: \"36e1b611-408e-4cc3-8531-03adcac84683\") " pod="keystone-kuttl-tests/openstack-galera-0" Dec 03 13:20:32 crc kubenswrapper[5052]: I1203 13:20:32.953156 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/36e1b611-408e-4cc3-8531-03adcac84683-config-data-generated\") pod \"openstack-galera-0\" (UID: \"36e1b611-408e-4cc3-8531-03adcac84683\") " pod="keystone-kuttl-tests/openstack-galera-0" Dec 03 13:20:32 crc kubenswrapper[5052]: I1203 13:20:32.953287 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/36e1b611-408e-4cc3-8531-03adcac84683-operator-scripts\") pod \"openstack-galera-0\" (UID: \"36e1b611-408e-4cc3-8531-03adcac84683\") " pod="keystone-kuttl-tests/openstack-galera-0" Dec 03 13:20:32 crc kubenswrapper[5052]: I1203 13:20:32.953546 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/0ea9b78b-2beb-4bdb-852c-1b92b9638551-config-data-default\") pod \"openstack-galera-1\" (UID: \"0ea9b78b-2beb-4bdb-852c-1b92b9638551\") " pod="keystone-kuttl-tests/openstack-galera-1" Dec 03 13:20:32 crc kubenswrapper[5052]: I1203 13:20:32.953691 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/0ea9b78b-2beb-4bdb-852c-1b92b9638551-kolla-config\") pod \"openstack-galera-1\" (UID: \"0ea9b78b-2beb-4bdb-852c-1b92b9638551\") " pod="keystone-kuttl-tests/openstack-galera-1" Dec 03 13:20:32 crc kubenswrapper[5052]: I1203 13:20:32.953750 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/36e1b611-408e-4cc3-8531-03adcac84683-config-data-default\") pod \"openstack-galera-0\" (UID: \"36e1b611-408e-4cc3-8531-03adcac84683\") " pod="keystone-kuttl-tests/openstack-galera-0" Dec 03 13:20:32 crc kubenswrapper[5052]: I1203 13:20:32.953791 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage01-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage01-crc\") pod \"openstack-galera-1\" (UID: \"0ea9b78b-2beb-4bdb-852c-1b92b9638551\") device mount path \"/mnt/openstack/pv01\"" pod="keystone-kuttl-tests/openstack-galera-1" Dec 03 13:20:32 crc kubenswrapper[5052]: I1203 13:20:32.953989 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/0ea9b78b-2beb-4bdb-852c-1b92b9638551-config-data-generated\") pod \"openstack-galera-1\" (UID: \"0ea9b78b-2beb-4bdb-852c-1b92b9638551\") " pod="keystone-kuttl-tests/openstack-galera-1" Dec 03 13:20:32 crc kubenswrapper[5052]: I1203 13:20:32.954077 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/2e771f67-f156-4dec-a77f-88b4735f298b-config-data-generated\") pod \"openstack-galera-2\" (UID: \"2e771f67-f156-4dec-a77f-88b4735f298b\") " pod="keystone-kuttl-tests/openstack-galera-2" Dec 03 13:20:32 crc kubenswrapper[5052]: I1203 13:20:32.954640 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/2e771f67-f156-4dec-a77f-88b4735f298b-config-data-default\") pod \"openstack-galera-2\" (UID: \"2e771f67-f156-4dec-a77f-88b4735f298b\") " pod="keystone-kuttl-tests/openstack-galera-2" Dec 03 13:20:32 crc kubenswrapper[5052]: I1203 13:20:32.954993 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/0ea9b78b-2beb-4bdb-852c-1b92b9638551-operator-scripts\") pod \"openstack-galera-1\" (UID: \"0ea9b78b-2beb-4bdb-852c-1b92b9638551\") " pod="keystone-kuttl-tests/openstack-galera-1" Dec 03 13:20:32 crc kubenswrapper[5052]: I1203 13:20:32.955189 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/2e771f67-f156-4dec-a77f-88b4735f298b-operator-scripts\") pod \"openstack-galera-2\" (UID: \"2e771f67-f156-4dec-a77f-88b4735f298b\") " pod="keystone-kuttl-tests/openstack-galera-2" Dec 03 13:20:32 crc kubenswrapper[5052]: I1203 13:20:32.972469 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-27nwd\" (UniqueName: \"kubernetes.io/projected/0ea9b78b-2beb-4bdb-852c-1b92b9638551-kube-api-access-27nwd\") pod \"openstack-galera-1\" (UID: \"0ea9b78b-2beb-4bdb-852c-1b92b9638551\") " pod="keystone-kuttl-tests/openstack-galera-1" Dec 03 13:20:32 crc kubenswrapper[5052]: I1203 13:20:32.973547 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage01-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage01-crc\") pod \"openstack-galera-1\" (UID: \"0ea9b78b-2beb-4bdb-852c-1b92b9638551\") " pod="keystone-kuttl-tests/openstack-galera-1" Dec 03 13:20:32 crc kubenswrapper[5052]: I1203 13:20:32.974914 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage14-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage14-crc\") pod \"openstack-galera-2\" (UID: \"2e771f67-f156-4dec-a77f-88b4735f298b\") " pod="keystone-kuttl-tests/openstack-galera-2" Dec 03 13:20:32 crc kubenswrapper[5052]: I1203 13:20:32.975197 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-2lf8x\" (UniqueName: \"kubernetes.io/projected/36e1b611-408e-4cc3-8531-03adcac84683-kube-api-access-2lf8x\") pod \"openstack-galera-0\" (UID: \"36e1b611-408e-4cc3-8531-03adcac84683\") " pod="keystone-kuttl-tests/openstack-galera-0" Dec 03 13:20:32 crc kubenswrapper[5052]: I1203 13:20:32.975404 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage13-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage13-crc\") pod \"openstack-galera-0\" (UID: \"36e1b611-408e-4cc3-8531-03adcac84683\") " pod="keystone-kuttl-tests/openstack-galera-0" Dec 03 13:20:32 crc kubenswrapper[5052]: I1203 13:20:32.980173 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-xzkxh\" (UniqueName: \"kubernetes.io/projected/2e771f67-f156-4dec-a77f-88b4735f298b-kube-api-access-xzkxh\") pod \"openstack-galera-2\" (UID: \"2e771f67-f156-4dec-a77f-88b4735f298b\") " pod="keystone-kuttl-tests/openstack-galera-2" Dec 03 13:20:33 crc kubenswrapper[5052]: I1203 13:20:33.023737 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="keystone-kuttl-tests/openstack-galera-0" Dec 03 13:20:33 crc kubenswrapper[5052]: I1203 13:20:33.042034 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="keystone-kuttl-tests/openstack-galera-1" Dec 03 13:20:33 crc kubenswrapper[5052]: I1203 13:20:33.051969 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="keystone-kuttl-tests/openstack-galera-2" Dec 03 13:20:33 crc kubenswrapper[5052]: I1203 13:20:33.472090 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["keystone-kuttl-tests/openstack-galera-0"] Dec 03 13:20:33 crc kubenswrapper[5052]: I1203 13:20:33.531735 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["keystone-kuttl-tests/openstack-galera-2"] Dec 03 13:20:33 crc kubenswrapper[5052]: I1203 13:20:33.549621 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["keystone-kuttl-tests/openstack-galera-1"] Dec 03 13:20:33 crc kubenswrapper[5052]: I1203 13:20:33.900331 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="keystone-kuttl-tests/openstack-galera-0" event={"ID":"36e1b611-408e-4cc3-8531-03adcac84683","Type":"ContainerStarted","Data":"fbba02a1d00fb616f73bb42e9d24b9e28e4932a3a8df0503d4d09b3d0f37d171"} Dec 03 13:20:33 crc kubenswrapper[5052]: I1203 13:20:33.900723 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="keystone-kuttl-tests/openstack-galera-0" event={"ID":"36e1b611-408e-4cc3-8531-03adcac84683","Type":"ContainerStarted","Data":"9d071802138ee9f0d7d75122d8fa0e3d571d088f32c0815f4cb6cb7b2dcf2c2a"} Dec 03 13:20:33 crc kubenswrapper[5052]: I1203 13:20:33.901918 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="keystone-kuttl-tests/openstack-galera-1" event={"ID":"0ea9b78b-2beb-4bdb-852c-1b92b9638551","Type":"ContainerStarted","Data":"4c382ef0474c9229a0a4e17621c93a48f914490ac61a39d27ed7408ea6100c9f"} Dec 03 13:20:33 crc kubenswrapper[5052]: I1203 13:20:33.901941 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="keystone-kuttl-tests/openstack-galera-1" event={"ID":"0ea9b78b-2beb-4bdb-852c-1b92b9638551","Type":"ContainerStarted","Data":"1182de3d450f8cac1f36dfe05e9f539509003033d9b4de58373f045a1bb7ea2f"} Dec 03 13:20:33 crc kubenswrapper[5052]: I1203 13:20:33.903899 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="keystone-kuttl-tests/openstack-galera-2" event={"ID":"2e771f67-f156-4dec-a77f-88b4735f298b","Type":"ContainerStarted","Data":"58affb7d9a0f5d58e2c49f7f718b13fdf9ae2fbe26ffd445be1a013018c36651"} Dec 03 13:20:33 crc kubenswrapper[5052]: I1203 13:20:33.903933 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="keystone-kuttl-tests/openstack-galera-2" event={"ID":"2e771f67-f156-4dec-a77f-88b4735f298b","Type":"ContainerStarted","Data":"de5dad1917376303577d9774889bfe76e15007d2b0a03fcc56189936a91a8bd7"} Dec 03 13:20:37 crc kubenswrapper[5052]: I1203 13:20:37.930380 5052 generic.go:334] "Generic (PLEG): container finished" podID="0ea9b78b-2beb-4bdb-852c-1b92b9638551" containerID="4c382ef0474c9229a0a4e17621c93a48f914490ac61a39d27ed7408ea6100c9f" exitCode=0 Dec 03 13:20:37 crc kubenswrapper[5052]: I1203 13:20:37.930422 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="keystone-kuttl-tests/openstack-galera-1" event={"ID":"0ea9b78b-2beb-4bdb-852c-1b92b9638551","Type":"ContainerDied","Data":"4c382ef0474c9229a0a4e17621c93a48f914490ac61a39d27ed7408ea6100c9f"} Dec 03 13:20:37 crc kubenswrapper[5052]: I1203 13:20:37.932839 5052 generic.go:334] "Generic (PLEG): container finished" podID="2e771f67-f156-4dec-a77f-88b4735f298b" containerID="58affb7d9a0f5d58e2c49f7f718b13fdf9ae2fbe26ffd445be1a013018c36651" exitCode=0 Dec 03 13:20:37 crc kubenswrapper[5052]: I1203 13:20:37.932907 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="keystone-kuttl-tests/openstack-galera-2" event={"ID":"2e771f67-f156-4dec-a77f-88b4735f298b","Type":"ContainerDied","Data":"58affb7d9a0f5d58e2c49f7f718b13fdf9ae2fbe26ffd445be1a013018c36651"} Dec 03 13:20:37 crc kubenswrapper[5052]: I1203 13:20:37.935497 5052 generic.go:334] "Generic (PLEG): container finished" podID="36e1b611-408e-4cc3-8531-03adcac84683" containerID="fbba02a1d00fb616f73bb42e9d24b9e28e4932a3a8df0503d4d09b3d0f37d171" exitCode=0 Dec 03 13:20:37 crc kubenswrapper[5052]: I1203 13:20:37.935537 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="keystone-kuttl-tests/openstack-galera-0" event={"ID":"36e1b611-408e-4cc3-8531-03adcac84683","Type":"ContainerDied","Data":"fbba02a1d00fb616f73bb42e9d24b9e28e4932a3a8df0503d4d09b3d0f37d171"} Dec 03 13:20:38 crc kubenswrapper[5052]: I1203 13:20:38.945415 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="keystone-kuttl-tests/openstack-galera-0" event={"ID":"36e1b611-408e-4cc3-8531-03adcac84683","Type":"ContainerStarted","Data":"97588a63cef8ea1ec768dbe56c144e30ff801fcf2bf63f638e5f9b770d79693d"} Dec 03 13:20:38 crc kubenswrapper[5052]: I1203 13:20:38.950606 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="keystone-kuttl-tests/openstack-galera-1" event={"ID":"0ea9b78b-2beb-4bdb-852c-1b92b9638551","Type":"ContainerStarted","Data":"2ebd4f7e1d8ef53cddce7b2e5ce181643f18fc1958462225a56999f9b3633b6b"} Dec 03 13:20:38 crc kubenswrapper[5052]: I1203 13:20:38.952396 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="keystone-kuttl-tests/openstack-galera-2" event={"ID":"2e771f67-f156-4dec-a77f-88b4735f298b","Type":"ContainerStarted","Data":"9fb2554ac99ea6b2010070c8f5d4dd0dec16b79118170eabf332947ede384598"} Dec 03 13:20:38 crc kubenswrapper[5052]: I1203 13:20:38.962623 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="keystone-kuttl-tests/openstack-galera-0" podStartSLOduration=7.962606154 podStartE2EDuration="7.962606154s" podCreationTimestamp="2025-12-03 13:20:31 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 13:20:38.961127172 +0000 UTC m=+7229.549368936" watchObservedRunningTime="2025-12-03 13:20:38.962606154 +0000 UTC m=+7229.550847918" Dec 03 13:20:38 crc kubenswrapper[5052]: I1203 13:20:38.986590 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="keystone-kuttl-tests/openstack-galera-2" podStartSLOduration=7.986572681 podStartE2EDuration="7.986572681s" podCreationTimestamp="2025-12-03 13:20:31 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 13:20:38.981439004 +0000 UTC m=+7229.569680768" watchObservedRunningTime="2025-12-03 13:20:38.986572681 +0000 UTC m=+7229.574814445" Dec 03 13:20:39 crc kubenswrapper[5052]: I1203 13:20:39.012154 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="keystone-kuttl-tests/openstack-galera-1" podStartSLOduration=8.012136373 podStartE2EDuration="8.012136373s" podCreationTimestamp="2025-12-03 13:20:31 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 13:20:39.009472357 +0000 UTC m=+7229.597714111" watchObservedRunningTime="2025-12-03 13:20:39.012136373 +0000 UTC m=+7229.600378137" Dec 03 13:20:40 crc kubenswrapper[5052]: I1203 13:20:40.976187 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-operators/infra-operator-index-2dtcx" Dec 03 13:20:40 crc kubenswrapper[5052]: I1203 13:20:40.976551 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-operators/infra-operator-index-2dtcx" Dec 03 13:20:41 crc kubenswrapper[5052]: I1203 13:20:41.003812 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-operators/infra-operator-index-2dtcx" Dec 03 13:20:41 crc kubenswrapper[5052]: I1203 13:20:41.997549 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-operators/infra-operator-index-2dtcx" Dec 03 13:20:42 crc kubenswrapper[5052]: I1203 13:20:42.673114 5052 scope.go:117] "RemoveContainer" containerID="12a8b1f470f6d59d7700e376f2a48eb36363f91f9823a35ce534062c0974c10c" Dec 03 13:20:42 crc kubenswrapper[5052]: E1203 13:20:42.673427 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 13:20:43 crc kubenswrapper[5052]: I1203 13:20:43.024333 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="keystone-kuttl-tests/openstack-galera-0" Dec 03 13:20:43 crc kubenswrapper[5052]: I1203 13:20:43.024391 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="keystone-kuttl-tests/openstack-galera-0" Dec 03 13:20:43 crc kubenswrapper[5052]: I1203 13:20:43.042709 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="keystone-kuttl-tests/openstack-galera-1" Dec 03 13:20:43 crc kubenswrapper[5052]: I1203 13:20:43.042970 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="keystone-kuttl-tests/openstack-galera-1" Dec 03 13:20:43 crc kubenswrapper[5052]: I1203 13:20:43.052617 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="keystone-kuttl-tests/openstack-galera-2" Dec 03 13:20:43 crc kubenswrapper[5052]: I1203 13:20:43.052685 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="keystone-kuttl-tests/openstack-galera-2" Dec 03 13:20:47 crc kubenswrapper[5052]: I1203 13:20:47.071427 5052 scope.go:117] "RemoveContainer" containerID="1ea1cbc762aeca1fb318ff705889f4f8e8d1ed822f4fa40f9b9269799265366b" Dec 03 13:20:47 crc kubenswrapper[5052]: I1203 13:20:47.091302 5052 scope.go:117] "RemoveContainer" containerID="a36b5fe28d987ac737b31620a308b4c87b943989ef9a1150ae5273524d39fe57" Dec 03 13:20:47 crc kubenswrapper[5052]: I1203 13:20:47.117202 5052 scope.go:117] "RemoveContainer" containerID="29bff1b69fb756f4688efc0a60f108814d3e91d0ce26942157ef0c61fb757c45" Dec 03 13:20:47 crc kubenswrapper[5052]: I1203 13:20:47.123172 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="keystone-kuttl-tests/openstack-galera-2" Dec 03 13:20:47 crc kubenswrapper[5052]: I1203 13:20:47.196092 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="keystone-kuttl-tests/openstack-galera-2" Dec 03 13:20:50 crc kubenswrapper[5052]: I1203 13:20:50.268235 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openshift-marketplace/certified-operators-jzfhg"] Dec 03 13:20:50 crc kubenswrapper[5052]: I1203 13:20:50.270889 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/certified-operators-jzfhg" Dec 03 13:20:50 crc kubenswrapper[5052]: I1203 13:20:50.279309 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/certified-operators-jzfhg"] Dec 03 13:20:50 crc kubenswrapper[5052]: I1203 13:20:50.310920 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hk7fc\" (UniqueName: \"kubernetes.io/projected/643f418e-7979-4b37-93b6-c571cf467745-kube-api-access-hk7fc\") pod \"certified-operators-jzfhg\" (UID: \"643f418e-7979-4b37-93b6-c571cf467745\") " pod="openshift-marketplace/certified-operators-jzfhg" Dec 03 13:20:50 crc kubenswrapper[5052]: I1203 13:20:50.311206 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/643f418e-7979-4b37-93b6-c571cf467745-utilities\") pod \"certified-operators-jzfhg\" (UID: \"643f418e-7979-4b37-93b6-c571cf467745\") " pod="openshift-marketplace/certified-operators-jzfhg" Dec 03 13:20:50 crc kubenswrapper[5052]: I1203 13:20:50.311364 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/643f418e-7979-4b37-93b6-c571cf467745-catalog-content\") pod \"certified-operators-jzfhg\" (UID: \"643f418e-7979-4b37-93b6-c571cf467745\") " pod="openshift-marketplace/certified-operators-jzfhg" Dec 03 13:20:50 crc kubenswrapper[5052]: I1203 13:20:50.413452 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/643f418e-7979-4b37-93b6-c571cf467745-catalog-content\") pod \"certified-operators-jzfhg\" (UID: \"643f418e-7979-4b37-93b6-c571cf467745\") " pod="openshift-marketplace/certified-operators-jzfhg" Dec 03 13:20:50 crc kubenswrapper[5052]: I1203 13:20:50.413574 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-hk7fc\" (UniqueName: \"kubernetes.io/projected/643f418e-7979-4b37-93b6-c571cf467745-kube-api-access-hk7fc\") pod \"certified-operators-jzfhg\" (UID: \"643f418e-7979-4b37-93b6-c571cf467745\") " pod="openshift-marketplace/certified-operators-jzfhg" Dec 03 13:20:50 crc kubenswrapper[5052]: I1203 13:20:50.413640 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/643f418e-7979-4b37-93b6-c571cf467745-utilities\") pod \"certified-operators-jzfhg\" (UID: \"643f418e-7979-4b37-93b6-c571cf467745\") " pod="openshift-marketplace/certified-operators-jzfhg" Dec 03 13:20:50 crc kubenswrapper[5052]: I1203 13:20:50.414723 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/643f418e-7979-4b37-93b6-c571cf467745-catalog-content\") pod \"certified-operators-jzfhg\" (UID: \"643f418e-7979-4b37-93b6-c571cf467745\") " pod="openshift-marketplace/certified-operators-jzfhg" Dec 03 13:20:50 crc kubenswrapper[5052]: I1203 13:20:50.414757 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/643f418e-7979-4b37-93b6-c571cf467745-utilities\") pod \"certified-operators-jzfhg\" (UID: \"643f418e-7979-4b37-93b6-c571cf467745\") " pod="openshift-marketplace/certified-operators-jzfhg" Dec 03 13:20:50 crc kubenswrapper[5052]: I1203 13:20:50.449699 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-hk7fc\" (UniqueName: \"kubernetes.io/projected/643f418e-7979-4b37-93b6-c571cf467745-kube-api-access-hk7fc\") pod \"certified-operators-jzfhg\" (UID: \"643f418e-7979-4b37-93b6-c571cf467745\") " pod="openshift-marketplace/certified-operators-jzfhg" Dec 03 13:20:50 crc kubenswrapper[5052]: I1203 13:20:50.589848 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/certified-operators-jzfhg" Dec 03 13:20:50 crc kubenswrapper[5052]: I1203 13:20:50.713032 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-operators/5d473c3169f40b179d14921c90af2c8546b7b757fe551b7dba7d903f5d28bdz"] Dec 03 13:20:50 crc kubenswrapper[5052]: I1203 13:20:50.714657 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-operators/5d473c3169f40b179d14921c90af2c8546b7b757fe551b7dba7d903f5d28bdz" Dec 03 13:20:50 crc kubenswrapper[5052]: I1203 13:20:50.718051 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-operators"/"default-dockercfg-7fklr" Dec 03 13:20:50 crc kubenswrapper[5052]: I1203 13:20:50.729969 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-operators/5d473c3169f40b179d14921c90af2c8546b7b757fe551b7dba7d903f5d28bdz"] Dec 03 13:20:50 crc kubenswrapper[5052]: I1203 13:20:50.818612 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/862171d5-fa42-4b46-85d7-369b796daae6-util\") pod \"5d473c3169f40b179d14921c90af2c8546b7b757fe551b7dba7d903f5d28bdz\" (UID: \"862171d5-fa42-4b46-85d7-369b796daae6\") " pod="openstack-operators/5d473c3169f40b179d14921c90af2c8546b7b757fe551b7dba7d903f5d28bdz" Dec 03 13:20:50 crc kubenswrapper[5052]: I1203 13:20:50.818691 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jb4jg\" (UniqueName: \"kubernetes.io/projected/862171d5-fa42-4b46-85d7-369b796daae6-kube-api-access-jb4jg\") pod \"5d473c3169f40b179d14921c90af2c8546b7b757fe551b7dba7d903f5d28bdz\" (UID: \"862171d5-fa42-4b46-85d7-369b796daae6\") " pod="openstack-operators/5d473c3169f40b179d14921c90af2c8546b7b757fe551b7dba7d903f5d28bdz" Dec 03 13:20:50 crc kubenswrapper[5052]: I1203 13:20:50.818712 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/862171d5-fa42-4b46-85d7-369b796daae6-bundle\") pod \"5d473c3169f40b179d14921c90af2c8546b7b757fe551b7dba7d903f5d28bdz\" (UID: \"862171d5-fa42-4b46-85d7-369b796daae6\") " pod="openstack-operators/5d473c3169f40b179d14921c90af2c8546b7b757fe551b7dba7d903f5d28bdz" Dec 03 13:20:50 crc kubenswrapper[5052]: I1203 13:20:50.920601 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/862171d5-fa42-4b46-85d7-369b796daae6-util\") pod \"5d473c3169f40b179d14921c90af2c8546b7b757fe551b7dba7d903f5d28bdz\" (UID: \"862171d5-fa42-4b46-85d7-369b796daae6\") " pod="openstack-operators/5d473c3169f40b179d14921c90af2c8546b7b757fe551b7dba7d903f5d28bdz" Dec 03 13:20:50 crc kubenswrapper[5052]: I1203 13:20:50.920678 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-jb4jg\" (UniqueName: \"kubernetes.io/projected/862171d5-fa42-4b46-85d7-369b796daae6-kube-api-access-jb4jg\") pod \"5d473c3169f40b179d14921c90af2c8546b7b757fe551b7dba7d903f5d28bdz\" (UID: \"862171d5-fa42-4b46-85d7-369b796daae6\") " pod="openstack-operators/5d473c3169f40b179d14921c90af2c8546b7b757fe551b7dba7d903f5d28bdz" Dec 03 13:20:50 crc kubenswrapper[5052]: I1203 13:20:50.920710 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/862171d5-fa42-4b46-85d7-369b796daae6-bundle\") pod \"5d473c3169f40b179d14921c90af2c8546b7b757fe551b7dba7d903f5d28bdz\" (UID: \"862171d5-fa42-4b46-85d7-369b796daae6\") " pod="openstack-operators/5d473c3169f40b179d14921c90af2c8546b7b757fe551b7dba7d903f5d28bdz" Dec 03 13:20:50 crc kubenswrapper[5052]: I1203 13:20:50.921203 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/862171d5-fa42-4b46-85d7-369b796daae6-util\") pod \"5d473c3169f40b179d14921c90af2c8546b7b757fe551b7dba7d903f5d28bdz\" (UID: \"862171d5-fa42-4b46-85d7-369b796daae6\") " pod="openstack-operators/5d473c3169f40b179d14921c90af2c8546b7b757fe551b7dba7d903f5d28bdz" Dec 03 13:20:50 crc kubenswrapper[5052]: I1203 13:20:50.922299 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/862171d5-fa42-4b46-85d7-369b796daae6-bundle\") pod \"5d473c3169f40b179d14921c90af2c8546b7b757fe551b7dba7d903f5d28bdz\" (UID: \"862171d5-fa42-4b46-85d7-369b796daae6\") " pod="openstack-operators/5d473c3169f40b179d14921c90af2c8546b7b757fe551b7dba7d903f5d28bdz" Dec 03 13:20:50 crc kubenswrapper[5052]: I1203 13:20:50.934638 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/certified-operators-jzfhg"] Dec 03 13:20:50 crc kubenswrapper[5052]: W1203 13:20:50.935614 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod643f418e_7979_4b37_93b6_c571cf467745.slice/crio-6e3952b08881e106759c7817754d50229698f8c288f49f5331c8b48cd6da9d58 WatchSource:0}: Error finding container 6e3952b08881e106759c7817754d50229698f8c288f49f5331c8b48cd6da9d58: Status 404 returned error can't find the container with id 6e3952b08881e106759c7817754d50229698f8c288f49f5331c8b48cd6da9d58 Dec 03 13:20:50 crc kubenswrapper[5052]: I1203 13:20:50.949203 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-jb4jg\" (UniqueName: \"kubernetes.io/projected/862171d5-fa42-4b46-85d7-369b796daae6-kube-api-access-jb4jg\") pod \"5d473c3169f40b179d14921c90af2c8546b7b757fe551b7dba7d903f5d28bdz\" (UID: \"862171d5-fa42-4b46-85d7-369b796daae6\") " pod="openstack-operators/5d473c3169f40b179d14921c90af2c8546b7b757fe551b7dba7d903f5d28bdz" Dec 03 13:20:51 crc kubenswrapper[5052]: I1203 13:20:51.032415 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-operators/5d473c3169f40b179d14921c90af2c8546b7b757fe551b7dba7d903f5d28bdz" Dec 03 13:20:51 crc kubenswrapper[5052]: I1203 13:20:51.037843 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/certified-operators-jzfhg" event={"ID":"643f418e-7979-4b37-93b6-c571cf467745","Type":"ContainerStarted","Data":"6e3952b08881e106759c7817754d50229698f8c288f49f5331c8b48cd6da9d58"} Dec 03 13:20:51 crc kubenswrapper[5052]: I1203 13:20:51.452446 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-operators/5d473c3169f40b179d14921c90af2c8546b7b757fe551b7dba7d903f5d28bdz"] Dec 03 13:20:52 crc kubenswrapper[5052]: I1203 13:20:52.049074 5052 generic.go:334] "Generic (PLEG): container finished" podID="643f418e-7979-4b37-93b6-c571cf467745" containerID="de3a7509dd205c7228e339f5d8060bd9de272bcded2ecf6301a8fa5a473a5dbd" exitCode=0 Dec 03 13:20:52 crc kubenswrapper[5052]: I1203 13:20:52.049179 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/certified-operators-jzfhg" event={"ID":"643f418e-7979-4b37-93b6-c571cf467745","Type":"ContainerDied","Data":"de3a7509dd205c7228e339f5d8060bd9de272bcded2ecf6301a8fa5a473a5dbd"} Dec 03 13:20:52 crc kubenswrapper[5052]: I1203 13:20:52.052167 5052 generic.go:334] "Generic (PLEG): container finished" podID="862171d5-fa42-4b46-85d7-369b796daae6" containerID="fedb14f501ff028d683c77ffefa608766dad89bb6094768270196099685e2f28" exitCode=0 Dec 03 13:20:52 crc kubenswrapper[5052]: I1203 13:20:52.052253 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/5d473c3169f40b179d14921c90af2c8546b7b757fe551b7dba7d903f5d28bdz" event={"ID":"862171d5-fa42-4b46-85d7-369b796daae6","Type":"ContainerDied","Data":"fedb14f501ff028d683c77ffefa608766dad89bb6094768270196099685e2f28"} Dec 03 13:20:52 crc kubenswrapper[5052]: I1203 13:20:52.052387 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/5d473c3169f40b179d14921c90af2c8546b7b757fe551b7dba7d903f5d28bdz" event={"ID":"862171d5-fa42-4b46-85d7-369b796daae6","Type":"ContainerStarted","Data":"0e35f7d67b8476709889ea1c6652ee0921a749e4c4ef0e5fbb82db49be35d989"} Dec 03 13:20:53 crc kubenswrapper[5052]: I1203 13:20:53.063753 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/5d473c3169f40b179d14921c90af2c8546b7b757fe551b7dba7d903f5d28bdz" event={"ID":"862171d5-fa42-4b46-85d7-369b796daae6","Type":"ContainerStarted","Data":"bbd7f06d451dffd6aa1fb53ba54bfac0479837de5dc87cc768f048d6bf9c838d"} Dec 03 13:20:53 crc kubenswrapper[5052]: I1203 13:20:53.132554 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="keystone-kuttl-tests/openstack-galera-2" podUID="2e771f67-f156-4dec-a77f-88b4735f298b" containerName="galera" probeResult="failure" output=< Dec 03 13:20:53 crc kubenswrapper[5052]: wsrep_local_state_comment (Donor/Desynced) differs from Synced Dec 03 13:20:53 crc kubenswrapper[5052]: > Dec 03 13:20:54 crc kubenswrapper[5052]: I1203 13:20:54.074111 5052 generic.go:334] "Generic (PLEG): container finished" podID="862171d5-fa42-4b46-85d7-369b796daae6" containerID="bbd7f06d451dffd6aa1fb53ba54bfac0479837de5dc87cc768f048d6bf9c838d" exitCode=0 Dec 03 13:20:54 crc kubenswrapper[5052]: I1203 13:20:54.074312 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/5d473c3169f40b179d14921c90af2c8546b7b757fe551b7dba7d903f5d28bdz" event={"ID":"862171d5-fa42-4b46-85d7-369b796daae6","Type":"ContainerDied","Data":"bbd7f06d451dffd6aa1fb53ba54bfac0479837de5dc87cc768f048d6bf9c838d"} Dec 03 13:20:54 crc kubenswrapper[5052]: I1203 13:20:54.077006 5052 generic.go:334] "Generic (PLEG): container finished" podID="643f418e-7979-4b37-93b6-c571cf467745" containerID="889f704dcafa2c465f2fe8782eeab807c69a7a4b7c5c8e08b5ffca85430efe56" exitCode=0 Dec 03 13:20:54 crc kubenswrapper[5052]: I1203 13:20:54.077034 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/certified-operators-jzfhg" event={"ID":"643f418e-7979-4b37-93b6-c571cf467745","Type":"ContainerDied","Data":"889f704dcafa2c465f2fe8782eeab807c69a7a4b7c5c8e08b5ffca85430efe56"} Dec 03 13:20:54 crc kubenswrapper[5052]: I1203 13:20:54.673499 5052 scope.go:117] "RemoveContainer" containerID="12a8b1f470f6d59d7700e376f2a48eb36363f91f9823a35ce534062c0974c10c" Dec 03 13:20:54 crc kubenswrapper[5052]: E1203 13:20:54.674330 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 13:20:55 crc kubenswrapper[5052]: I1203 13:20:55.087729 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/certified-operators-jzfhg" event={"ID":"643f418e-7979-4b37-93b6-c571cf467745","Type":"ContainerStarted","Data":"cb7f5a14340387fd6a9c034e75b80c4faf5db3d921c1e6f99b1f724ccacd41aa"} Dec 03 13:20:55 crc kubenswrapper[5052]: I1203 13:20:55.090200 5052 generic.go:334] "Generic (PLEG): container finished" podID="862171d5-fa42-4b46-85d7-369b796daae6" containerID="3e0fb88e579785672f0a6cbbd960e331f0d6d07ddf21d7be9104243f2d460a69" exitCode=0 Dec 03 13:20:55 crc kubenswrapper[5052]: I1203 13:20:55.090253 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/5d473c3169f40b179d14921c90af2c8546b7b757fe551b7dba7d903f5d28bdz" event={"ID":"862171d5-fa42-4b46-85d7-369b796daae6","Type":"ContainerDied","Data":"3e0fb88e579785672f0a6cbbd960e331f0d6d07ddf21d7be9104243f2d460a69"} Dec 03 13:20:55 crc kubenswrapper[5052]: I1203 13:20:55.115449 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-marketplace/certified-operators-jzfhg" podStartSLOduration=2.568249868 podStartE2EDuration="5.115427513s" podCreationTimestamp="2025-12-03 13:20:50 +0000 UTC" firstStartedPulling="2025-12-03 13:20:52.053013673 +0000 UTC m=+7242.641255437" lastFinishedPulling="2025-12-03 13:20:54.600191308 +0000 UTC m=+7245.188433082" observedRunningTime="2025-12-03 13:20:55.10799891 +0000 UTC m=+7245.696240694" watchObservedRunningTime="2025-12-03 13:20:55.115427513 +0000 UTC m=+7245.703669277" Dec 03 13:20:56 crc kubenswrapper[5052]: I1203 13:20:56.370073 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-operators/5d473c3169f40b179d14921c90af2c8546b7b757fe551b7dba7d903f5d28bdz" Dec 03 13:20:56 crc kubenswrapper[5052]: I1203 13:20:56.415423 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/862171d5-fa42-4b46-85d7-369b796daae6-bundle\") pod \"862171d5-fa42-4b46-85d7-369b796daae6\" (UID: \"862171d5-fa42-4b46-85d7-369b796daae6\") " Dec 03 13:20:56 crc kubenswrapper[5052]: I1203 13:20:56.415485 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/862171d5-fa42-4b46-85d7-369b796daae6-util\") pod \"862171d5-fa42-4b46-85d7-369b796daae6\" (UID: \"862171d5-fa42-4b46-85d7-369b796daae6\") " Dec 03 13:20:56 crc kubenswrapper[5052]: I1203 13:20:56.415575 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jb4jg\" (UniqueName: \"kubernetes.io/projected/862171d5-fa42-4b46-85d7-369b796daae6-kube-api-access-jb4jg\") pod \"862171d5-fa42-4b46-85d7-369b796daae6\" (UID: \"862171d5-fa42-4b46-85d7-369b796daae6\") " Dec 03 13:20:56 crc kubenswrapper[5052]: I1203 13:20:56.417945 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/862171d5-fa42-4b46-85d7-369b796daae6-bundle" (OuterVolumeSpecName: "bundle") pod "862171d5-fa42-4b46-85d7-369b796daae6" (UID: "862171d5-fa42-4b46-85d7-369b796daae6"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 13:20:56 crc kubenswrapper[5052]: I1203 13:20:56.422372 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/862171d5-fa42-4b46-85d7-369b796daae6-kube-api-access-jb4jg" (OuterVolumeSpecName: "kube-api-access-jb4jg") pod "862171d5-fa42-4b46-85d7-369b796daae6" (UID: "862171d5-fa42-4b46-85d7-369b796daae6"). InnerVolumeSpecName "kube-api-access-jb4jg". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:20:56 crc kubenswrapper[5052]: I1203 13:20:56.429114 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/862171d5-fa42-4b46-85d7-369b796daae6-util" (OuterVolumeSpecName: "util") pod "862171d5-fa42-4b46-85d7-369b796daae6" (UID: "862171d5-fa42-4b46-85d7-369b796daae6"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 13:20:56 crc kubenswrapper[5052]: I1203 13:20:56.518479 5052 reconciler_common.go:293] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/862171d5-fa42-4b46-85d7-369b796daae6-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 13:20:56 crc kubenswrapper[5052]: I1203 13:20:56.518962 5052 reconciler_common.go:293] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/862171d5-fa42-4b46-85d7-369b796daae6-util\") on node \"crc\" DevicePath \"\"" Dec 03 13:20:56 crc kubenswrapper[5052]: I1203 13:20:56.519080 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-jb4jg\" (UniqueName: \"kubernetes.io/projected/862171d5-fa42-4b46-85d7-369b796daae6-kube-api-access-jb4jg\") on node \"crc\" DevicePath \"\"" Dec 03 13:20:57 crc kubenswrapper[5052]: I1203 13:20:57.105995 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/5d473c3169f40b179d14921c90af2c8546b7b757fe551b7dba7d903f5d28bdz" event={"ID":"862171d5-fa42-4b46-85d7-369b796daae6","Type":"ContainerDied","Data":"0e35f7d67b8476709889ea1c6652ee0921a749e4c4ef0e5fbb82db49be35d989"} Dec 03 13:20:57 crc kubenswrapper[5052]: I1203 13:20:57.106338 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0e35f7d67b8476709889ea1c6652ee0921a749e4c4ef0e5fbb82db49be35d989" Dec 03 13:20:57 crc kubenswrapper[5052]: I1203 13:20:57.106200 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-operators/5d473c3169f40b179d14921c90af2c8546b7b757fe551b7dba7d903f5d28bdz" Dec 03 13:21:00 crc kubenswrapper[5052]: I1203 13:21:00.517485 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="keystone-kuttl-tests/openstack-galera-0" Dec 03 13:21:00 crc kubenswrapper[5052]: I1203 13:21:00.590178 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-marketplace/certified-operators-jzfhg" Dec 03 13:21:00 crc kubenswrapper[5052]: I1203 13:21:00.590244 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openshift-marketplace/certified-operators-jzfhg" Dec 03 13:21:00 crc kubenswrapper[5052]: I1203 13:21:00.598886 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="keystone-kuttl-tests/openstack-galera-0" Dec 03 13:21:00 crc kubenswrapper[5052]: I1203 13:21:00.642040 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-marketplace/certified-operators-jzfhg" Dec 03 13:21:01 crc kubenswrapper[5052]: I1203 13:21:01.179647 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-marketplace/certified-operators-jzfhg" Dec 03 13:21:01 crc kubenswrapper[5052]: I1203 13:21:01.710851 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="keystone-kuttl-tests/openstack-galera-1" Dec 03 13:21:01 crc kubenswrapper[5052]: I1203 13:21:01.794612 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="keystone-kuttl-tests/openstack-galera-1" Dec 03 13:21:04 crc kubenswrapper[5052]: I1203 13:21:04.251911 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openshift-marketplace/certified-operators-jzfhg"] Dec 03 13:21:04 crc kubenswrapper[5052]: I1203 13:21:04.252961 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openshift-marketplace/certified-operators-jzfhg" podUID="643f418e-7979-4b37-93b6-c571cf467745" containerName="registry-server" containerID="cri-o://cb7f5a14340387fd6a9c034e75b80c4faf5db3d921c1e6f99b1f724ccacd41aa" gracePeriod=2 Dec 03 13:21:04 crc kubenswrapper[5052]: I1203 13:21:04.619728 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/certified-operators-jzfhg" Dec 03 13:21:04 crc kubenswrapper[5052]: I1203 13:21:04.739474 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hk7fc\" (UniqueName: \"kubernetes.io/projected/643f418e-7979-4b37-93b6-c571cf467745-kube-api-access-hk7fc\") pod \"643f418e-7979-4b37-93b6-c571cf467745\" (UID: \"643f418e-7979-4b37-93b6-c571cf467745\") " Dec 03 13:21:04 crc kubenswrapper[5052]: I1203 13:21:04.739614 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/643f418e-7979-4b37-93b6-c571cf467745-catalog-content\") pod \"643f418e-7979-4b37-93b6-c571cf467745\" (UID: \"643f418e-7979-4b37-93b6-c571cf467745\") " Dec 03 13:21:04 crc kubenswrapper[5052]: I1203 13:21:04.739640 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/643f418e-7979-4b37-93b6-c571cf467745-utilities\") pod \"643f418e-7979-4b37-93b6-c571cf467745\" (UID: \"643f418e-7979-4b37-93b6-c571cf467745\") " Dec 03 13:21:04 crc kubenswrapper[5052]: I1203 13:21:04.740672 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/643f418e-7979-4b37-93b6-c571cf467745-utilities" (OuterVolumeSpecName: "utilities") pod "643f418e-7979-4b37-93b6-c571cf467745" (UID: "643f418e-7979-4b37-93b6-c571cf467745"). InnerVolumeSpecName "utilities". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 13:21:04 crc kubenswrapper[5052]: I1203 13:21:04.746551 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/643f418e-7979-4b37-93b6-c571cf467745-kube-api-access-hk7fc" (OuterVolumeSpecName: "kube-api-access-hk7fc") pod "643f418e-7979-4b37-93b6-c571cf467745" (UID: "643f418e-7979-4b37-93b6-c571cf467745"). InnerVolumeSpecName "kube-api-access-hk7fc". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:21:04 crc kubenswrapper[5052]: I1203 13:21:04.801816 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/643f418e-7979-4b37-93b6-c571cf467745-catalog-content" (OuterVolumeSpecName: "catalog-content") pod "643f418e-7979-4b37-93b6-c571cf467745" (UID: "643f418e-7979-4b37-93b6-c571cf467745"). InnerVolumeSpecName "catalog-content". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 13:21:04 crc kubenswrapper[5052]: I1203 13:21:04.841796 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-hk7fc\" (UniqueName: \"kubernetes.io/projected/643f418e-7979-4b37-93b6-c571cf467745-kube-api-access-hk7fc\") on node \"crc\" DevicePath \"\"" Dec 03 13:21:04 crc kubenswrapper[5052]: I1203 13:21:04.841849 5052 reconciler_common.go:293] "Volume detached for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/643f418e-7979-4b37-93b6-c571cf467745-catalog-content\") on node \"crc\" DevicePath \"\"" Dec 03 13:21:04 crc kubenswrapper[5052]: I1203 13:21:04.841864 5052 reconciler_common.go:293] "Volume detached for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/643f418e-7979-4b37-93b6-c571cf467745-utilities\") on node \"crc\" DevicePath \"\"" Dec 03 13:21:05 crc kubenswrapper[5052]: I1203 13:21:05.163727 5052 generic.go:334] "Generic (PLEG): container finished" podID="643f418e-7979-4b37-93b6-c571cf467745" containerID="cb7f5a14340387fd6a9c034e75b80c4faf5db3d921c1e6f99b1f724ccacd41aa" exitCode=0 Dec 03 13:21:05 crc kubenswrapper[5052]: I1203 13:21:05.163800 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/certified-operators-jzfhg" Dec 03 13:21:05 crc kubenswrapper[5052]: I1203 13:21:05.163782 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/certified-operators-jzfhg" event={"ID":"643f418e-7979-4b37-93b6-c571cf467745","Type":"ContainerDied","Data":"cb7f5a14340387fd6a9c034e75b80c4faf5db3d921c1e6f99b1f724ccacd41aa"} Dec 03 13:21:05 crc kubenswrapper[5052]: I1203 13:21:05.164219 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/certified-operators-jzfhg" event={"ID":"643f418e-7979-4b37-93b6-c571cf467745","Type":"ContainerDied","Data":"6e3952b08881e106759c7817754d50229698f8c288f49f5331c8b48cd6da9d58"} Dec 03 13:21:05 crc kubenswrapper[5052]: I1203 13:21:05.164244 5052 scope.go:117] "RemoveContainer" containerID="cb7f5a14340387fd6a9c034e75b80c4faf5db3d921c1e6f99b1f724ccacd41aa" Dec 03 13:21:05 crc kubenswrapper[5052]: I1203 13:21:05.181967 5052 scope.go:117] "RemoveContainer" containerID="889f704dcafa2c465f2fe8782eeab807c69a7a4b7c5c8e08b5ffca85430efe56" Dec 03 13:21:05 crc kubenswrapper[5052]: I1203 13:21:05.194128 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openshift-marketplace/certified-operators-jzfhg"] Dec 03 13:21:05 crc kubenswrapper[5052]: I1203 13:21:05.199587 5052 scope.go:117] "RemoveContainer" containerID="de3a7509dd205c7228e339f5d8060bd9de272bcded2ecf6301a8fa5a473a5dbd" Dec 03 13:21:05 crc kubenswrapper[5052]: I1203 13:21:05.204911 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openshift-marketplace/certified-operators-jzfhg"] Dec 03 13:21:05 crc kubenswrapper[5052]: I1203 13:21:05.228978 5052 scope.go:117] "RemoveContainer" containerID="cb7f5a14340387fd6a9c034e75b80c4faf5db3d921c1e6f99b1f724ccacd41aa" Dec 03 13:21:05 crc kubenswrapper[5052]: E1203 13:21:05.229394 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cb7f5a14340387fd6a9c034e75b80c4faf5db3d921c1e6f99b1f724ccacd41aa\": container with ID starting with cb7f5a14340387fd6a9c034e75b80c4faf5db3d921c1e6f99b1f724ccacd41aa not found: ID does not exist" containerID="cb7f5a14340387fd6a9c034e75b80c4faf5db3d921c1e6f99b1f724ccacd41aa" Dec 03 13:21:05 crc kubenswrapper[5052]: I1203 13:21:05.229434 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cb7f5a14340387fd6a9c034e75b80c4faf5db3d921c1e6f99b1f724ccacd41aa"} err="failed to get container status \"cb7f5a14340387fd6a9c034e75b80c4faf5db3d921c1e6f99b1f724ccacd41aa\": rpc error: code = NotFound desc = could not find container \"cb7f5a14340387fd6a9c034e75b80c4faf5db3d921c1e6f99b1f724ccacd41aa\": container with ID starting with cb7f5a14340387fd6a9c034e75b80c4faf5db3d921c1e6f99b1f724ccacd41aa not found: ID does not exist" Dec 03 13:21:05 crc kubenswrapper[5052]: I1203 13:21:05.229460 5052 scope.go:117] "RemoveContainer" containerID="889f704dcafa2c465f2fe8782eeab807c69a7a4b7c5c8e08b5ffca85430efe56" Dec 03 13:21:05 crc kubenswrapper[5052]: E1203 13:21:05.229871 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"889f704dcafa2c465f2fe8782eeab807c69a7a4b7c5c8e08b5ffca85430efe56\": container with ID starting with 889f704dcafa2c465f2fe8782eeab807c69a7a4b7c5c8e08b5ffca85430efe56 not found: ID does not exist" containerID="889f704dcafa2c465f2fe8782eeab807c69a7a4b7c5c8e08b5ffca85430efe56" Dec 03 13:21:05 crc kubenswrapper[5052]: I1203 13:21:05.229946 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"889f704dcafa2c465f2fe8782eeab807c69a7a4b7c5c8e08b5ffca85430efe56"} err="failed to get container status \"889f704dcafa2c465f2fe8782eeab807c69a7a4b7c5c8e08b5ffca85430efe56\": rpc error: code = NotFound desc = could not find container \"889f704dcafa2c465f2fe8782eeab807c69a7a4b7c5c8e08b5ffca85430efe56\": container with ID starting with 889f704dcafa2c465f2fe8782eeab807c69a7a4b7c5c8e08b5ffca85430efe56 not found: ID does not exist" Dec 03 13:21:05 crc kubenswrapper[5052]: I1203 13:21:05.230021 5052 scope.go:117] "RemoveContainer" containerID="de3a7509dd205c7228e339f5d8060bd9de272bcded2ecf6301a8fa5a473a5dbd" Dec 03 13:21:05 crc kubenswrapper[5052]: E1203 13:21:05.230309 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"de3a7509dd205c7228e339f5d8060bd9de272bcded2ecf6301a8fa5a473a5dbd\": container with ID starting with de3a7509dd205c7228e339f5d8060bd9de272bcded2ecf6301a8fa5a473a5dbd not found: ID does not exist" containerID="de3a7509dd205c7228e339f5d8060bd9de272bcded2ecf6301a8fa5a473a5dbd" Dec 03 13:21:05 crc kubenswrapper[5052]: I1203 13:21:05.230393 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"de3a7509dd205c7228e339f5d8060bd9de272bcded2ecf6301a8fa5a473a5dbd"} err="failed to get container status \"de3a7509dd205c7228e339f5d8060bd9de272bcded2ecf6301a8fa5a473a5dbd\": rpc error: code = NotFound desc = could not find container \"de3a7509dd205c7228e339f5d8060bd9de272bcded2ecf6301a8fa5a473a5dbd\": container with ID starting with de3a7509dd205c7228e339f5d8060bd9de272bcded2ecf6301a8fa5a473a5dbd not found: ID does not exist" Dec 03 13:21:05 crc kubenswrapper[5052]: I1203 13:21:05.682752 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="643f418e-7979-4b37-93b6-c571cf467745" path="/var/lib/kubelet/pods/643f418e-7979-4b37-93b6-c571cf467745/volumes" Dec 03 13:21:06 crc kubenswrapper[5052]: I1203 13:21:06.672750 5052 scope.go:117] "RemoveContainer" containerID="12a8b1f470f6d59d7700e376f2a48eb36363f91f9823a35ce534062c0974c10c" Dec 03 13:21:06 crc kubenswrapper[5052]: E1203 13:21:06.673161 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 13:21:07 crc kubenswrapper[5052]: I1203 13:21:07.186808 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-operators/infra-operator-controller-manager-6f9d567ccb-lth55"] Dec 03 13:21:07 crc kubenswrapper[5052]: E1203 13:21:07.187144 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="862171d5-fa42-4b46-85d7-369b796daae6" containerName="pull" Dec 03 13:21:07 crc kubenswrapper[5052]: I1203 13:21:07.187159 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="862171d5-fa42-4b46-85d7-369b796daae6" containerName="pull" Dec 03 13:21:07 crc kubenswrapper[5052]: E1203 13:21:07.187175 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="643f418e-7979-4b37-93b6-c571cf467745" containerName="extract-content" Dec 03 13:21:07 crc kubenswrapper[5052]: I1203 13:21:07.187187 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="643f418e-7979-4b37-93b6-c571cf467745" containerName="extract-content" Dec 03 13:21:07 crc kubenswrapper[5052]: E1203 13:21:07.187204 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="643f418e-7979-4b37-93b6-c571cf467745" containerName="extract-utilities" Dec 03 13:21:07 crc kubenswrapper[5052]: I1203 13:21:07.187212 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="643f418e-7979-4b37-93b6-c571cf467745" containerName="extract-utilities" Dec 03 13:21:07 crc kubenswrapper[5052]: E1203 13:21:07.187226 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="862171d5-fa42-4b46-85d7-369b796daae6" containerName="extract" Dec 03 13:21:07 crc kubenswrapper[5052]: I1203 13:21:07.187233 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="862171d5-fa42-4b46-85d7-369b796daae6" containerName="extract" Dec 03 13:21:07 crc kubenswrapper[5052]: E1203 13:21:07.187252 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="643f418e-7979-4b37-93b6-c571cf467745" containerName="registry-server" Dec 03 13:21:07 crc kubenswrapper[5052]: I1203 13:21:07.187259 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="643f418e-7979-4b37-93b6-c571cf467745" containerName="registry-server" Dec 03 13:21:07 crc kubenswrapper[5052]: E1203 13:21:07.187276 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="862171d5-fa42-4b46-85d7-369b796daae6" containerName="util" Dec 03 13:21:07 crc kubenswrapper[5052]: I1203 13:21:07.187284 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="862171d5-fa42-4b46-85d7-369b796daae6" containerName="util" Dec 03 13:21:07 crc kubenswrapper[5052]: I1203 13:21:07.187431 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="862171d5-fa42-4b46-85d7-369b796daae6" containerName="extract" Dec 03 13:21:07 crc kubenswrapper[5052]: I1203 13:21:07.187452 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="643f418e-7979-4b37-93b6-c571cf467745" containerName="registry-server" Dec 03 13:21:07 crc kubenswrapper[5052]: I1203 13:21:07.188361 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-operators/infra-operator-controller-manager-6f9d567ccb-lth55" Dec 03 13:21:07 crc kubenswrapper[5052]: I1203 13:21:07.191644 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-operators"/"infra-operator-controller-manager-service-cert" Dec 03 13:21:07 crc kubenswrapper[5052]: I1203 13:21:07.192094 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-operators"/"infra-operator-controller-manager-dockercfg-78vmf" Dec 03 13:21:07 crc kubenswrapper[5052]: I1203 13:21:07.206312 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-operators/infra-operator-controller-manager-6f9d567ccb-lth55"] Dec 03 13:21:07 crc kubenswrapper[5052]: I1203 13:21:07.276364 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/4dd99d7d-b7ff-4ba7-bf73-5e66e3e500b7-apiservice-cert\") pod \"infra-operator-controller-manager-6f9d567ccb-lth55\" (UID: \"4dd99d7d-b7ff-4ba7-bf73-5e66e3e500b7\") " pod="openstack-operators/infra-operator-controller-manager-6f9d567ccb-lth55" Dec 03 13:21:07 crc kubenswrapper[5052]: I1203 13:21:07.276522 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hc6qm\" (UniqueName: \"kubernetes.io/projected/4dd99d7d-b7ff-4ba7-bf73-5e66e3e500b7-kube-api-access-hc6qm\") pod \"infra-operator-controller-manager-6f9d567ccb-lth55\" (UID: \"4dd99d7d-b7ff-4ba7-bf73-5e66e3e500b7\") " pod="openstack-operators/infra-operator-controller-manager-6f9d567ccb-lth55" Dec 03 13:21:07 crc kubenswrapper[5052]: I1203 13:21:07.276549 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4dd99d7d-b7ff-4ba7-bf73-5e66e3e500b7-webhook-cert\") pod \"infra-operator-controller-manager-6f9d567ccb-lth55\" (UID: \"4dd99d7d-b7ff-4ba7-bf73-5e66e3e500b7\") " pod="openstack-operators/infra-operator-controller-manager-6f9d567ccb-lth55" Dec 03 13:21:07 crc kubenswrapper[5052]: I1203 13:21:07.377689 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-hc6qm\" (UniqueName: \"kubernetes.io/projected/4dd99d7d-b7ff-4ba7-bf73-5e66e3e500b7-kube-api-access-hc6qm\") pod \"infra-operator-controller-manager-6f9d567ccb-lth55\" (UID: \"4dd99d7d-b7ff-4ba7-bf73-5e66e3e500b7\") " pod="openstack-operators/infra-operator-controller-manager-6f9d567ccb-lth55" Dec 03 13:21:07 crc kubenswrapper[5052]: I1203 13:21:07.377739 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4dd99d7d-b7ff-4ba7-bf73-5e66e3e500b7-webhook-cert\") pod \"infra-operator-controller-manager-6f9d567ccb-lth55\" (UID: \"4dd99d7d-b7ff-4ba7-bf73-5e66e3e500b7\") " pod="openstack-operators/infra-operator-controller-manager-6f9d567ccb-lth55" Dec 03 13:21:07 crc kubenswrapper[5052]: I1203 13:21:07.377796 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/4dd99d7d-b7ff-4ba7-bf73-5e66e3e500b7-apiservice-cert\") pod \"infra-operator-controller-manager-6f9d567ccb-lth55\" (UID: \"4dd99d7d-b7ff-4ba7-bf73-5e66e3e500b7\") " pod="openstack-operators/infra-operator-controller-manager-6f9d567ccb-lth55" Dec 03 13:21:07 crc kubenswrapper[5052]: I1203 13:21:07.382391 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/4dd99d7d-b7ff-4ba7-bf73-5e66e3e500b7-apiservice-cert\") pod \"infra-operator-controller-manager-6f9d567ccb-lth55\" (UID: \"4dd99d7d-b7ff-4ba7-bf73-5e66e3e500b7\") " pod="openstack-operators/infra-operator-controller-manager-6f9d567ccb-lth55" Dec 03 13:21:07 crc kubenswrapper[5052]: I1203 13:21:07.389002 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4dd99d7d-b7ff-4ba7-bf73-5e66e3e500b7-webhook-cert\") pod \"infra-operator-controller-manager-6f9d567ccb-lth55\" (UID: \"4dd99d7d-b7ff-4ba7-bf73-5e66e3e500b7\") " pod="openstack-operators/infra-operator-controller-manager-6f9d567ccb-lth55" Dec 03 13:21:07 crc kubenswrapper[5052]: I1203 13:21:07.399556 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-hc6qm\" (UniqueName: \"kubernetes.io/projected/4dd99d7d-b7ff-4ba7-bf73-5e66e3e500b7-kube-api-access-hc6qm\") pod \"infra-operator-controller-manager-6f9d567ccb-lth55\" (UID: \"4dd99d7d-b7ff-4ba7-bf73-5e66e3e500b7\") " pod="openstack-operators/infra-operator-controller-manager-6f9d567ccb-lth55" Dec 03 13:21:07 crc kubenswrapper[5052]: I1203 13:21:07.510462 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-operators/infra-operator-controller-manager-6f9d567ccb-lth55" Dec 03 13:21:07 crc kubenswrapper[5052]: I1203 13:21:07.777130 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-operators/infra-operator-controller-manager-6f9d567ccb-lth55"] Dec 03 13:21:08 crc kubenswrapper[5052]: I1203 13:21:08.188652 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/infra-operator-controller-manager-6f9d567ccb-lth55" event={"ID":"4dd99d7d-b7ff-4ba7-bf73-5e66e3e500b7","Type":"ContainerStarted","Data":"8e4b737cdfe5a62b1eea4be10d7ae8408ed9d5db9eb9e377b74b41e783c9bec7"} Dec 03 13:21:08 crc kubenswrapper[5052]: I1203 13:21:08.188969 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/infra-operator-controller-manager-6f9d567ccb-lth55" event={"ID":"4dd99d7d-b7ff-4ba7-bf73-5e66e3e500b7","Type":"ContainerStarted","Data":"ff1e3219f320766fd15e0d3660ebbf002fdc2bc78689c5ca0227b84da981ed38"} Dec 03 13:21:09 crc kubenswrapper[5052]: I1203 13:21:09.198451 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/infra-operator-controller-manager-6f9d567ccb-lth55" event={"ID":"4dd99d7d-b7ff-4ba7-bf73-5e66e3e500b7","Type":"ContainerStarted","Data":"152450c0accaa40ece4dc43da5d14b974063c658a420e0a00dae85acf98002de"} Dec 03 13:21:09 crc kubenswrapper[5052]: I1203 13:21:09.198853 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-operators/infra-operator-controller-manager-6f9d567ccb-lth55" Dec 03 13:21:17 crc kubenswrapper[5052]: I1203 13:21:17.517005 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-operators/infra-operator-controller-manager-6f9d567ccb-lth55" Dec 03 13:21:17 crc kubenswrapper[5052]: I1203 13:21:17.548848 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-operators/infra-operator-controller-manager-6f9d567ccb-lth55" podStartSLOduration=10.548822353 podStartE2EDuration="10.548822353s" podCreationTimestamp="2025-12-03 13:21:07 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 13:21:09.224683608 +0000 UTC m=+7259.812925372" watchObservedRunningTime="2025-12-03 13:21:17.548822353 +0000 UTC m=+7268.137064117" Dec 03 13:21:20 crc kubenswrapper[5052]: I1203 13:21:20.672718 5052 scope.go:117] "RemoveContainer" containerID="12a8b1f470f6d59d7700e376f2a48eb36363f91f9823a35ce534062c0974c10c" Dec 03 13:21:20 crc kubenswrapper[5052]: E1203 13:21:20.673765 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 13:21:23 crc kubenswrapper[5052]: I1203 13:21:23.057765 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-operators/rabbitmq-cluster-operator-index-7qf66"] Dec 03 13:21:23 crc kubenswrapper[5052]: I1203 13:21:23.059354 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-operators/rabbitmq-cluster-operator-index-7qf66" Dec 03 13:21:23 crc kubenswrapper[5052]: I1203 13:21:23.062798 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-operators"/"rabbitmq-cluster-operator-index-dockercfg-cb9nk" Dec 03 13:21:23 crc kubenswrapper[5052]: I1203 13:21:23.064880 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-operators/rabbitmq-cluster-operator-index-7qf66"] Dec 03 13:21:23 crc kubenswrapper[5052]: I1203 13:21:23.155338 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vr6fx\" (UniqueName: \"kubernetes.io/projected/0bae405f-384d-45f9-8261-685ad5804551-kube-api-access-vr6fx\") pod \"rabbitmq-cluster-operator-index-7qf66\" (UID: \"0bae405f-384d-45f9-8261-685ad5804551\") " pod="openstack-operators/rabbitmq-cluster-operator-index-7qf66" Dec 03 13:21:23 crc kubenswrapper[5052]: I1203 13:21:23.258058 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-vr6fx\" (UniqueName: \"kubernetes.io/projected/0bae405f-384d-45f9-8261-685ad5804551-kube-api-access-vr6fx\") pod \"rabbitmq-cluster-operator-index-7qf66\" (UID: \"0bae405f-384d-45f9-8261-685ad5804551\") " pod="openstack-operators/rabbitmq-cluster-operator-index-7qf66" Dec 03 13:21:23 crc kubenswrapper[5052]: I1203 13:21:23.278553 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-vr6fx\" (UniqueName: \"kubernetes.io/projected/0bae405f-384d-45f9-8261-685ad5804551-kube-api-access-vr6fx\") pod \"rabbitmq-cluster-operator-index-7qf66\" (UID: \"0bae405f-384d-45f9-8261-685ad5804551\") " pod="openstack-operators/rabbitmq-cluster-operator-index-7qf66" Dec 03 13:21:23 crc kubenswrapper[5052]: I1203 13:21:23.394912 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-operators/rabbitmq-cluster-operator-index-7qf66" Dec 03 13:21:23 crc kubenswrapper[5052]: I1203 13:21:23.644272 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-operators/rabbitmq-cluster-operator-index-7qf66"] Dec 03 13:21:24 crc kubenswrapper[5052]: I1203 13:21:24.330914 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/rabbitmq-cluster-operator-index-7qf66" event={"ID":"0bae405f-384d-45f9-8261-685ad5804551","Type":"ContainerStarted","Data":"9f4253fd294b4a245149dd9313ada3ad5a467f0fb3f82ccb36c2a9558661a95f"} Dec 03 13:21:25 crc kubenswrapper[5052]: I1203 13:21:25.341905 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/rabbitmq-cluster-operator-index-7qf66" event={"ID":"0bae405f-384d-45f9-8261-685ad5804551","Type":"ContainerStarted","Data":"aa8acc3c56d6c3e5bb6a62cf6429806fdf196e6914f29e81fb07207514b0d2e1"} Dec 03 13:21:25 crc kubenswrapper[5052]: I1203 13:21:25.364421 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-operators/rabbitmq-cluster-operator-index-7qf66" podStartSLOduration=1.7243757849999999 podStartE2EDuration="2.364402004s" podCreationTimestamp="2025-12-03 13:21:23 +0000 UTC" firstStartedPulling="2025-12-03 13:21:23.652015024 +0000 UTC m=+7274.240256778" lastFinishedPulling="2025-12-03 13:21:24.292041233 +0000 UTC m=+7274.880282997" observedRunningTime="2025-12-03 13:21:25.359463792 +0000 UTC m=+7275.947705566" watchObservedRunningTime="2025-12-03 13:21:25.364402004 +0000 UTC m=+7275.952643768" Dec 03 13:21:28 crc kubenswrapper[5052]: I1203 13:21:28.783866 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["keystone-kuttl-tests/memcached-0"] Dec 03 13:21:28 crc kubenswrapper[5052]: I1203 13:21:28.785040 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="keystone-kuttl-tests/memcached-0" Dec 03 13:21:28 crc kubenswrapper[5052]: I1203 13:21:28.787742 5052 reflector.go:368] Caches populated for *v1.Secret from object-"keystone-kuttl-tests"/"memcached-memcached-dockercfg-qllsn" Dec 03 13:21:28 crc kubenswrapper[5052]: I1203 13:21:28.791603 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"keystone-kuttl-tests"/"memcached-config-data" Dec 03 13:21:28 crc kubenswrapper[5052]: I1203 13:21:28.802531 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["keystone-kuttl-tests/memcached-0"] Dec 03 13:21:28 crc kubenswrapper[5052]: I1203 13:21:28.938990 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/f72c36e0-b6bc-412c-887f-3e1394cf6c1f-kolla-config\") pod \"memcached-0\" (UID: \"f72c36e0-b6bc-412c-887f-3e1394cf6c1f\") " pod="keystone-kuttl-tests/memcached-0" Dec 03 13:21:28 crc kubenswrapper[5052]: I1203 13:21:28.939066 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-968v7\" (UniqueName: \"kubernetes.io/projected/f72c36e0-b6bc-412c-887f-3e1394cf6c1f-kube-api-access-968v7\") pod \"memcached-0\" (UID: \"f72c36e0-b6bc-412c-887f-3e1394cf6c1f\") " pod="keystone-kuttl-tests/memcached-0" Dec 03 13:21:28 crc kubenswrapper[5052]: I1203 13:21:28.939129 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/f72c36e0-b6bc-412c-887f-3e1394cf6c1f-config-data\") pod \"memcached-0\" (UID: \"f72c36e0-b6bc-412c-887f-3e1394cf6c1f\") " pod="keystone-kuttl-tests/memcached-0" Dec 03 13:21:29 crc kubenswrapper[5052]: I1203 13:21:29.040503 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/f72c36e0-b6bc-412c-887f-3e1394cf6c1f-kolla-config\") pod \"memcached-0\" (UID: \"f72c36e0-b6bc-412c-887f-3e1394cf6c1f\") " pod="keystone-kuttl-tests/memcached-0" Dec 03 13:21:29 crc kubenswrapper[5052]: I1203 13:21:29.040572 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-968v7\" (UniqueName: \"kubernetes.io/projected/f72c36e0-b6bc-412c-887f-3e1394cf6c1f-kube-api-access-968v7\") pod \"memcached-0\" (UID: \"f72c36e0-b6bc-412c-887f-3e1394cf6c1f\") " pod="keystone-kuttl-tests/memcached-0" Dec 03 13:21:29 crc kubenswrapper[5052]: I1203 13:21:29.040605 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/f72c36e0-b6bc-412c-887f-3e1394cf6c1f-config-data\") pod \"memcached-0\" (UID: \"f72c36e0-b6bc-412c-887f-3e1394cf6c1f\") " pod="keystone-kuttl-tests/memcached-0" Dec 03 13:21:29 crc kubenswrapper[5052]: I1203 13:21:29.041471 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/f72c36e0-b6bc-412c-887f-3e1394cf6c1f-config-data\") pod \"memcached-0\" (UID: \"f72c36e0-b6bc-412c-887f-3e1394cf6c1f\") " pod="keystone-kuttl-tests/memcached-0" Dec 03 13:21:29 crc kubenswrapper[5052]: I1203 13:21:29.041608 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/f72c36e0-b6bc-412c-887f-3e1394cf6c1f-kolla-config\") pod \"memcached-0\" (UID: \"f72c36e0-b6bc-412c-887f-3e1394cf6c1f\") " pod="keystone-kuttl-tests/memcached-0" Dec 03 13:21:29 crc kubenswrapper[5052]: I1203 13:21:29.062036 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-968v7\" (UniqueName: \"kubernetes.io/projected/f72c36e0-b6bc-412c-887f-3e1394cf6c1f-kube-api-access-968v7\") pod \"memcached-0\" (UID: \"f72c36e0-b6bc-412c-887f-3e1394cf6c1f\") " pod="keystone-kuttl-tests/memcached-0" Dec 03 13:21:29 crc kubenswrapper[5052]: I1203 13:21:29.105262 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="keystone-kuttl-tests/memcached-0" Dec 03 13:21:29 crc kubenswrapper[5052]: I1203 13:21:29.363105 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["keystone-kuttl-tests/memcached-0"] Dec 03 13:21:29 crc kubenswrapper[5052]: W1203 13:21:29.368218 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf72c36e0_b6bc_412c_887f_3e1394cf6c1f.slice/crio-cfc681750910beccb769ac932e3168898930981b97f1a39969af9884b064168d WatchSource:0}: Error finding container cfc681750910beccb769ac932e3168898930981b97f1a39969af9884b064168d: Status 404 returned error can't find the container with id cfc681750910beccb769ac932e3168898930981b97f1a39969af9884b064168d Dec 03 13:21:30 crc kubenswrapper[5052]: I1203 13:21:30.376199 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="keystone-kuttl-tests/memcached-0" event={"ID":"f72c36e0-b6bc-412c-887f-3e1394cf6c1f","Type":"ContainerStarted","Data":"e957ae3dd15df2d129c0d9b9587d9be204eec37250b8e16dd64286c8ed6109e8"} Dec 03 13:21:30 crc kubenswrapper[5052]: I1203 13:21:30.377534 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="keystone-kuttl-tests/memcached-0" event={"ID":"f72c36e0-b6bc-412c-887f-3e1394cf6c1f","Type":"ContainerStarted","Data":"cfc681750910beccb769ac932e3168898930981b97f1a39969af9884b064168d"} Dec 03 13:21:30 crc kubenswrapper[5052]: I1203 13:21:30.377663 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="keystone-kuttl-tests/memcached-0" Dec 03 13:21:30 crc kubenswrapper[5052]: I1203 13:21:30.398415 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="keystone-kuttl-tests/memcached-0" podStartSLOduration=2.398396045 podStartE2EDuration="2.398396045s" podCreationTimestamp="2025-12-03 13:21:28 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 13:21:30.393843725 +0000 UTC m=+7280.982085499" watchObservedRunningTime="2025-12-03 13:21:30.398396045 +0000 UTC m=+7280.986637819" Dec 03 13:21:33 crc kubenswrapper[5052]: I1203 13:21:33.395494 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-operators/rabbitmq-cluster-operator-index-7qf66" Dec 03 13:21:33 crc kubenswrapper[5052]: I1203 13:21:33.396052 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-operators/rabbitmq-cluster-operator-index-7qf66" Dec 03 13:21:33 crc kubenswrapper[5052]: I1203 13:21:33.433028 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-operators/rabbitmq-cluster-operator-index-7qf66" Dec 03 13:21:34 crc kubenswrapper[5052]: I1203 13:21:34.106737 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="keystone-kuttl-tests/memcached-0" Dec 03 13:21:34 crc kubenswrapper[5052]: I1203 13:21:34.432010 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-operators/rabbitmq-cluster-operator-index-7qf66" Dec 03 13:21:35 crc kubenswrapper[5052]: I1203 13:21:35.673427 5052 scope.go:117] "RemoveContainer" containerID="12a8b1f470f6d59d7700e376f2a48eb36363f91f9823a35ce534062c0974c10c" Dec 03 13:21:35 crc kubenswrapper[5052]: E1203 13:21:35.674279 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 13:21:41 crc kubenswrapper[5052]: I1203 13:21:41.939806 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-operators/9704761d240e56fb98655ffd81084895b33a73ec711f4dcdef0450e5907pw26"] Dec 03 13:21:41 crc kubenswrapper[5052]: I1203 13:21:41.941770 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-operators/9704761d240e56fb98655ffd81084895b33a73ec711f4dcdef0450e5907pw26" Dec 03 13:21:41 crc kubenswrapper[5052]: I1203 13:21:41.946448 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-operators"/"default-dockercfg-7fklr" Dec 03 13:21:41 crc kubenswrapper[5052]: I1203 13:21:41.960295 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-operators/9704761d240e56fb98655ffd81084895b33a73ec711f4dcdef0450e5907pw26"] Dec 03 13:21:42 crc kubenswrapper[5052]: I1203 13:21:42.060066 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/372ee9c5-9cbd-40f9-83ca-abf8b5b01a5c-bundle\") pod \"9704761d240e56fb98655ffd81084895b33a73ec711f4dcdef0450e5907pw26\" (UID: \"372ee9c5-9cbd-40f9-83ca-abf8b5b01a5c\") " pod="openstack-operators/9704761d240e56fb98655ffd81084895b33a73ec711f4dcdef0450e5907pw26" Dec 03 13:21:42 crc kubenswrapper[5052]: I1203 13:21:42.060332 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dmzf6\" (UniqueName: \"kubernetes.io/projected/372ee9c5-9cbd-40f9-83ca-abf8b5b01a5c-kube-api-access-dmzf6\") pod \"9704761d240e56fb98655ffd81084895b33a73ec711f4dcdef0450e5907pw26\" (UID: \"372ee9c5-9cbd-40f9-83ca-abf8b5b01a5c\") " pod="openstack-operators/9704761d240e56fb98655ffd81084895b33a73ec711f4dcdef0450e5907pw26" Dec 03 13:21:42 crc kubenswrapper[5052]: I1203 13:21:42.060456 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/372ee9c5-9cbd-40f9-83ca-abf8b5b01a5c-util\") pod \"9704761d240e56fb98655ffd81084895b33a73ec711f4dcdef0450e5907pw26\" (UID: \"372ee9c5-9cbd-40f9-83ca-abf8b5b01a5c\") " pod="openstack-operators/9704761d240e56fb98655ffd81084895b33a73ec711f4dcdef0450e5907pw26" Dec 03 13:21:42 crc kubenswrapper[5052]: I1203 13:21:42.161831 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/372ee9c5-9cbd-40f9-83ca-abf8b5b01a5c-bundle\") pod \"9704761d240e56fb98655ffd81084895b33a73ec711f4dcdef0450e5907pw26\" (UID: \"372ee9c5-9cbd-40f9-83ca-abf8b5b01a5c\") " pod="openstack-operators/9704761d240e56fb98655ffd81084895b33a73ec711f4dcdef0450e5907pw26" Dec 03 13:21:42 crc kubenswrapper[5052]: I1203 13:21:42.162190 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-dmzf6\" (UniqueName: \"kubernetes.io/projected/372ee9c5-9cbd-40f9-83ca-abf8b5b01a5c-kube-api-access-dmzf6\") pod \"9704761d240e56fb98655ffd81084895b33a73ec711f4dcdef0450e5907pw26\" (UID: \"372ee9c5-9cbd-40f9-83ca-abf8b5b01a5c\") " pod="openstack-operators/9704761d240e56fb98655ffd81084895b33a73ec711f4dcdef0450e5907pw26" Dec 03 13:21:42 crc kubenswrapper[5052]: I1203 13:21:42.162310 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/372ee9c5-9cbd-40f9-83ca-abf8b5b01a5c-util\") pod \"9704761d240e56fb98655ffd81084895b33a73ec711f4dcdef0450e5907pw26\" (UID: \"372ee9c5-9cbd-40f9-83ca-abf8b5b01a5c\") " pod="openstack-operators/9704761d240e56fb98655ffd81084895b33a73ec711f4dcdef0450e5907pw26" Dec 03 13:21:42 crc kubenswrapper[5052]: I1203 13:21:42.162477 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/372ee9c5-9cbd-40f9-83ca-abf8b5b01a5c-bundle\") pod \"9704761d240e56fb98655ffd81084895b33a73ec711f4dcdef0450e5907pw26\" (UID: \"372ee9c5-9cbd-40f9-83ca-abf8b5b01a5c\") " pod="openstack-operators/9704761d240e56fb98655ffd81084895b33a73ec711f4dcdef0450e5907pw26" Dec 03 13:21:42 crc kubenswrapper[5052]: I1203 13:21:42.162813 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/372ee9c5-9cbd-40f9-83ca-abf8b5b01a5c-util\") pod \"9704761d240e56fb98655ffd81084895b33a73ec711f4dcdef0450e5907pw26\" (UID: \"372ee9c5-9cbd-40f9-83ca-abf8b5b01a5c\") " pod="openstack-operators/9704761d240e56fb98655ffd81084895b33a73ec711f4dcdef0450e5907pw26" Dec 03 13:21:42 crc kubenswrapper[5052]: I1203 13:21:42.180896 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-dmzf6\" (UniqueName: \"kubernetes.io/projected/372ee9c5-9cbd-40f9-83ca-abf8b5b01a5c-kube-api-access-dmzf6\") pod \"9704761d240e56fb98655ffd81084895b33a73ec711f4dcdef0450e5907pw26\" (UID: \"372ee9c5-9cbd-40f9-83ca-abf8b5b01a5c\") " pod="openstack-operators/9704761d240e56fb98655ffd81084895b33a73ec711f4dcdef0450e5907pw26" Dec 03 13:21:42 crc kubenswrapper[5052]: I1203 13:21:42.264462 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-operators/9704761d240e56fb98655ffd81084895b33a73ec711f4dcdef0450e5907pw26" Dec 03 13:21:42 crc kubenswrapper[5052]: I1203 13:21:42.662017 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-operators/9704761d240e56fb98655ffd81084895b33a73ec711f4dcdef0450e5907pw26"] Dec 03 13:21:43 crc kubenswrapper[5052]: I1203 13:21:43.486750 5052 generic.go:334] "Generic (PLEG): container finished" podID="372ee9c5-9cbd-40f9-83ca-abf8b5b01a5c" containerID="7cb4c042c3262ef46295f0a30d5bc2bf6d49a22c5ee6c0af14ded068615ffc7d" exitCode=0 Dec 03 13:21:43 crc kubenswrapper[5052]: I1203 13:21:43.486823 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/9704761d240e56fb98655ffd81084895b33a73ec711f4dcdef0450e5907pw26" event={"ID":"372ee9c5-9cbd-40f9-83ca-abf8b5b01a5c","Type":"ContainerDied","Data":"7cb4c042c3262ef46295f0a30d5bc2bf6d49a22c5ee6c0af14ded068615ffc7d"} Dec 03 13:21:43 crc kubenswrapper[5052]: I1203 13:21:43.487025 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/9704761d240e56fb98655ffd81084895b33a73ec711f4dcdef0450e5907pw26" event={"ID":"372ee9c5-9cbd-40f9-83ca-abf8b5b01a5c","Type":"ContainerStarted","Data":"ec104fe3e43159b3df944f6d328bf3ff59c4089809928497f8e16d651b57238f"} Dec 03 13:21:44 crc kubenswrapper[5052]: I1203 13:21:44.497999 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/9704761d240e56fb98655ffd81084895b33a73ec711f4dcdef0450e5907pw26" event={"ID":"372ee9c5-9cbd-40f9-83ca-abf8b5b01a5c","Type":"ContainerStarted","Data":"4e9f96da9e8199ccd0b09b27ffd95748a6ab4cc99eb521c45ab48d2e34d494a1"} Dec 03 13:21:45 crc kubenswrapper[5052]: I1203 13:21:45.508212 5052 generic.go:334] "Generic (PLEG): container finished" podID="372ee9c5-9cbd-40f9-83ca-abf8b5b01a5c" containerID="4e9f96da9e8199ccd0b09b27ffd95748a6ab4cc99eb521c45ab48d2e34d494a1" exitCode=0 Dec 03 13:21:45 crc kubenswrapper[5052]: I1203 13:21:45.508328 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/9704761d240e56fb98655ffd81084895b33a73ec711f4dcdef0450e5907pw26" event={"ID":"372ee9c5-9cbd-40f9-83ca-abf8b5b01a5c","Type":"ContainerDied","Data":"4e9f96da9e8199ccd0b09b27ffd95748a6ab4cc99eb521c45ab48d2e34d494a1"} Dec 03 13:21:46 crc kubenswrapper[5052]: I1203 13:21:46.518970 5052 generic.go:334] "Generic (PLEG): container finished" podID="372ee9c5-9cbd-40f9-83ca-abf8b5b01a5c" containerID="c772ba41bb82907750b2d521f5db102f1d8c6b06a1c98e8c2a13bdd2fdedd5e4" exitCode=0 Dec 03 13:21:46 crc kubenswrapper[5052]: I1203 13:21:46.519055 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/9704761d240e56fb98655ffd81084895b33a73ec711f4dcdef0450e5907pw26" event={"ID":"372ee9c5-9cbd-40f9-83ca-abf8b5b01a5c","Type":"ContainerDied","Data":"c772ba41bb82907750b2d521f5db102f1d8c6b06a1c98e8c2a13bdd2fdedd5e4"} Dec 03 13:21:47 crc kubenswrapper[5052]: I1203 13:21:47.187097 5052 scope.go:117] "RemoveContainer" containerID="a42119ef503a1dddabc103a0657953fdebd8d75d56eba979880480d403dd5012" Dec 03 13:21:47 crc kubenswrapper[5052]: I1203 13:21:47.211786 5052 scope.go:117] "RemoveContainer" containerID="6d2fbf27b742c9ad335064ab874e1e65c7ad76481322449d44ccbd07b89f6085" Dec 03 13:21:47 crc kubenswrapper[5052]: I1203 13:21:47.248990 5052 scope.go:117] "RemoveContainer" containerID="bc72d7795dfc75a429a7be5f27349a239f919e01278ceec74d94c753a4d344bf" Dec 03 13:21:47 crc kubenswrapper[5052]: I1203 13:21:47.673559 5052 scope.go:117] "RemoveContainer" containerID="12a8b1f470f6d59d7700e376f2a48eb36363f91f9823a35ce534062c0974c10c" Dec 03 13:21:47 crc kubenswrapper[5052]: E1203 13:21:47.673885 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 13:21:47 crc kubenswrapper[5052]: I1203 13:21:47.776647 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-operators/9704761d240e56fb98655ffd81084895b33a73ec711f4dcdef0450e5907pw26" Dec 03 13:21:47 crc kubenswrapper[5052]: I1203 13:21:47.947379 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/372ee9c5-9cbd-40f9-83ca-abf8b5b01a5c-bundle\") pod \"372ee9c5-9cbd-40f9-83ca-abf8b5b01a5c\" (UID: \"372ee9c5-9cbd-40f9-83ca-abf8b5b01a5c\") " Dec 03 13:21:47 crc kubenswrapper[5052]: I1203 13:21:47.947588 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/372ee9c5-9cbd-40f9-83ca-abf8b5b01a5c-util\") pod \"372ee9c5-9cbd-40f9-83ca-abf8b5b01a5c\" (UID: \"372ee9c5-9cbd-40f9-83ca-abf8b5b01a5c\") " Dec 03 13:21:47 crc kubenswrapper[5052]: I1203 13:21:47.947638 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dmzf6\" (UniqueName: \"kubernetes.io/projected/372ee9c5-9cbd-40f9-83ca-abf8b5b01a5c-kube-api-access-dmzf6\") pod \"372ee9c5-9cbd-40f9-83ca-abf8b5b01a5c\" (UID: \"372ee9c5-9cbd-40f9-83ca-abf8b5b01a5c\") " Dec 03 13:21:47 crc kubenswrapper[5052]: I1203 13:21:47.948640 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/372ee9c5-9cbd-40f9-83ca-abf8b5b01a5c-bundle" (OuterVolumeSpecName: "bundle") pod "372ee9c5-9cbd-40f9-83ca-abf8b5b01a5c" (UID: "372ee9c5-9cbd-40f9-83ca-abf8b5b01a5c"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 13:21:47 crc kubenswrapper[5052]: I1203 13:21:47.952598 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/372ee9c5-9cbd-40f9-83ca-abf8b5b01a5c-kube-api-access-dmzf6" (OuterVolumeSpecName: "kube-api-access-dmzf6") pod "372ee9c5-9cbd-40f9-83ca-abf8b5b01a5c" (UID: "372ee9c5-9cbd-40f9-83ca-abf8b5b01a5c"). InnerVolumeSpecName "kube-api-access-dmzf6". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:21:47 crc kubenswrapper[5052]: I1203 13:21:47.962246 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/372ee9c5-9cbd-40f9-83ca-abf8b5b01a5c-util" (OuterVolumeSpecName: "util") pod "372ee9c5-9cbd-40f9-83ca-abf8b5b01a5c" (UID: "372ee9c5-9cbd-40f9-83ca-abf8b5b01a5c"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 13:21:48 crc kubenswrapper[5052]: I1203 13:21:48.048999 5052 reconciler_common.go:293] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/372ee9c5-9cbd-40f9-83ca-abf8b5b01a5c-util\") on node \"crc\" DevicePath \"\"" Dec 03 13:21:48 crc kubenswrapper[5052]: I1203 13:21:48.049044 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-dmzf6\" (UniqueName: \"kubernetes.io/projected/372ee9c5-9cbd-40f9-83ca-abf8b5b01a5c-kube-api-access-dmzf6\") on node \"crc\" DevicePath \"\"" Dec 03 13:21:48 crc kubenswrapper[5052]: I1203 13:21:48.049058 5052 reconciler_common.go:293] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/372ee9c5-9cbd-40f9-83ca-abf8b5b01a5c-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 13:21:48 crc kubenswrapper[5052]: I1203 13:21:48.533559 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/9704761d240e56fb98655ffd81084895b33a73ec711f4dcdef0450e5907pw26" event={"ID":"372ee9c5-9cbd-40f9-83ca-abf8b5b01a5c","Type":"ContainerDied","Data":"ec104fe3e43159b3df944f6d328bf3ff59c4089809928497f8e16d651b57238f"} Dec 03 13:21:48 crc kubenswrapper[5052]: I1203 13:21:48.533605 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ec104fe3e43159b3df944f6d328bf3ff59c4089809928497f8e16d651b57238f" Dec 03 13:21:48 crc kubenswrapper[5052]: I1203 13:21:48.533662 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-operators/9704761d240e56fb98655ffd81084895b33a73ec711f4dcdef0450e5907pw26" Dec 03 13:21:57 crc kubenswrapper[5052]: I1203 13:21:57.750932 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-operators/rabbitmq-cluster-operator-779fc9694b-g9mr6"] Dec 03 13:21:57 crc kubenswrapper[5052]: E1203 13:21:57.752017 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="372ee9c5-9cbd-40f9-83ca-abf8b5b01a5c" containerName="util" Dec 03 13:21:57 crc kubenswrapper[5052]: I1203 13:21:57.752037 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="372ee9c5-9cbd-40f9-83ca-abf8b5b01a5c" containerName="util" Dec 03 13:21:57 crc kubenswrapper[5052]: E1203 13:21:57.752095 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="372ee9c5-9cbd-40f9-83ca-abf8b5b01a5c" containerName="pull" Dec 03 13:21:57 crc kubenswrapper[5052]: I1203 13:21:57.752105 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="372ee9c5-9cbd-40f9-83ca-abf8b5b01a5c" containerName="pull" Dec 03 13:21:57 crc kubenswrapper[5052]: E1203 13:21:57.752126 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="372ee9c5-9cbd-40f9-83ca-abf8b5b01a5c" containerName="extract" Dec 03 13:21:57 crc kubenswrapper[5052]: I1203 13:21:57.752134 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="372ee9c5-9cbd-40f9-83ca-abf8b5b01a5c" containerName="extract" Dec 03 13:21:57 crc kubenswrapper[5052]: I1203 13:21:57.752300 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="372ee9c5-9cbd-40f9-83ca-abf8b5b01a5c" containerName="extract" Dec 03 13:21:57 crc kubenswrapper[5052]: I1203 13:21:57.752979 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-operators/rabbitmq-cluster-operator-779fc9694b-g9mr6" Dec 03 13:21:57 crc kubenswrapper[5052]: I1203 13:21:57.755018 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-operators"/"rabbitmq-cluster-operator-dockercfg-rzwfw" Dec 03 13:21:57 crc kubenswrapper[5052]: I1203 13:21:57.789905 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-operators/rabbitmq-cluster-operator-779fc9694b-g9mr6"] Dec 03 13:21:57 crc kubenswrapper[5052]: I1203 13:21:57.918991 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8nksz\" (UniqueName: \"kubernetes.io/projected/933b8ff7-751e-4329-9c00-1d6c086677de-kube-api-access-8nksz\") pod \"rabbitmq-cluster-operator-779fc9694b-g9mr6\" (UID: \"933b8ff7-751e-4329-9c00-1d6c086677de\") " pod="openstack-operators/rabbitmq-cluster-operator-779fc9694b-g9mr6" Dec 03 13:21:58 crc kubenswrapper[5052]: I1203 13:21:58.020986 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-8nksz\" (UniqueName: \"kubernetes.io/projected/933b8ff7-751e-4329-9c00-1d6c086677de-kube-api-access-8nksz\") pod \"rabbitmq-cluster-operator-779fc9694b-g9mr6\" (UID: \"933b8ff7-751e-4329-9c00-1d6c086677de\") " pod="openstack-operators/rabbitmq-cluster-operator-779fc9694b-g9mr6" Dec 03 13:21:58 crc kubenswrapper[5052]: I1203 13:21:58.040263 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-8nksz\" (UniqueName: \"kubernetes.io/projected/933b8ff7-751e-4329-9c00-1d6c086677de-kube-api-access-8nksz\") pod \"rabbitmq-cluster-operator-779fc9694b-g9mr6\" (UID: \"933b8ff7-751e-4329-9c00-1d6c086677de\") " pod="openstack-operators/rabbitmq-cluster-operator-779fc9694b-g9mr6" Dec 03 13:21:58 crc kubenswrapper[5052]: I1203 13:21:58.073785 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-operators/rabbitmq-cluster-operator-779fc9694b-g9mr6" Dec 03 13:21:58 crc kubenswrapper[5052]: I1203 13:21:58.406629 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-operators/rabbitmq-cluster-operator-779fc9694b-g9mr6"] Dec 03 13:21:58 crc kubenswrapper[5052]: I1203 13:21:58.607260 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/rabbitmq-cluster-operator-779fc9694b-g9mr6" event={"ID":"933b8ff7-751e-4329-9c00-1d6c086677de","Type":"ContainerStarted","Data":"7b3c3a7e185f8a38af4532a8085004a12d20a74385d43faea1c3603c23042e44"} Dec 03 13:21:59 crc kubenswrapper[5052]: I1203 13:21:59.619258 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/rabbitmq-cluster-operator-779fc9694b-g9mr6" event={"ID":"933b8ff7-751e-4329-9c00-1d6c086677de","Type":"ContainerStarted","Data":"b94cbf9acefffbed6c9cf42ee7a481524b7998828fdcd3f8a9882f97f66bc832"} Dec 03 13:21:59 crc kubenswrapper[5052]: I1203 13:21:59.635601 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-operators/rabbitmq-cluster-operator-779fc9694b-g9mr6" podStartSLOduration=2.635586342 podStartE2EDuration="2.635586342s" podCreationTimestamp="2025-12-03 13:21:57 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 13:21:59.635282943 +0000 UTC m=+7310.223524707" watchObservedRunningTime="2025-12-03 13:21:59.635586342 +0000 UTC m=+7310.223828106" Dec 03 13:22:00 crc kubenswrapper[5052]: I1203 13:22:00.673447 5052 scope.go:117] "RemoveContainer" containerID="12a8b1f470f6d59d7700e376f2a48eb36363f91f9823a35ce534062c0974c10c" Dec 03 13:22:00 crc kubenswrapper[5052]: E1203 13:22:00.673842 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 13:22:06 crc kubenswrapper[5052]: I1203 13:22:06.654942 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-operators/keystone-operator-index-9ddkd"] Dec 03 13:22:06 crc kubenswrapper[5052]: I1203 13:22:06.656094 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-operators/keystone-operator-index-9ddkd" Dec 03 13:22:06 crc kubenswrapper[5052]: I1203 13:22:06.660885 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-operators"/"keystone-operator-index-dockercfg-25g6p" Dec 03 13:22:06 crc kubenswrapper[5052]: I1203 13:22:06.668759 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-operators/keystone-operator-index-9ddkd"] Dec 03 13:22:06 crc kubenswrapper[5052]: I1203 13:22:06.772659 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qknd7\" (UniqueName: \"kubernetes.io/projected/da70419b-4f9e-405d-8fe7-7dbca0ca315e-kube-api-access-qknd7\") pod \"keystone-operator-index-9ddkd\" (UID: \"da70419b-4f9e-405d-8fe7-7dbca0ca315e\") " pod="openstack-operators/keystone-operator-index-9ddkd" Dec 03 13:22:06 crc kubenswrapper[5052]: I1203 13:22:06.874283 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-qknd7\" (UniqueName: \"kubernetes.io/projected/da70419b-4f9e-405d-8fe7-7dbca0ca315e-kube-api-access-qknd7\") pod \"keystone-operator-index-9ddkd\" (UID: \"da70419b-4f9e-405d-8fe7-7dbca0ca315e\") " pod="openstack-operators/keystone-operator-index-9ddkd" Dec 03 13:22:06 crc kubenswrapper[5052]: I1203 13:22:06.899813 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-qknd7\" (UniqueName: \"kubernetes.io/projected/da70419b-4f9e-405d-8fe7-7dbca0ca315e-kube-api-access-qknd7\") pod \"keystone-operator-index-9ddkd\" (UID: \"da70419b-4f9e-405d-8fe7-7dbca0ca315e\") " pod="openstack-operators/keystone-operator-index-9ddkd" Dec 03 13:22:06 crc kubenswrapper[5052]: I1203 13:22:06.974502 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-operators/keystone-operator-index-9ddkd" Dec 03 13:22:07 crc kubenswrapper[5052]: I1203 13:22:07.220658 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-operators/keystone-operator-index-9ddkd"] Dec 03 13:22:07 crc kubenswrapper[5052]: I1203 13:22:07.684719 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/keystone-operator-index-9ddkd" event={"ID":"da70419b-4f9e-405d-8fe7-7dbca0ca315e","Type":"ContainerStarted","Data":"344127d9109e2627417aeec60b2a8b1ccd36c41d207950c72c645e3b00415152"} Dec 03 13:22:08 crc kubenswrapper[5052]: I1203 13:22:08.700418 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/keystone-operator-index-9ddkd" event={"ID":"da70419b-4f9e-405d-8fe7-7dbca0ca315e","Type":"ContainerStarted","Data":"2d6aaf797a5a705bce7467f9106b54a4a7ecb16c29f11a578ba19225c543f84d"} Dec 03 13:22:08 crc kubenswrapper[5052]: I1203 13:22:08.723394 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-operators/keystone-operator-index-9ddkd" podStartSLOduration=1.910020173 podStartE2EDuration="2.723356715s" podCreationTimestamp="2025-12-03 13:22:06 +0000 UTC" firstStartedPulling="2025-12-03 13:22:07.235842985 +0000 UTC m=+7317.824084749" lastFinishedPulling="2025-12-03 13:22:08.049179527 +0000 UTC m=+7318.637421291" observedRunningTime="2025-12-03 13:22:08.717414994 +0000 UTC m=+7319.305656758" watchObservedRunningTime="2025-12-03 13:22:08.723356715 +0000 UTC m=+7319.311598479" Dec 03 13:22:12 crc kubenswrapper[5052]: I1203 13:22:12.673421 5052 scope.go:117] "RemoveContainer" containerID="12a8b1f470f6d59d7700e376f2a48eb36363f91f9823a35ce534062c0974c10c" Dec 03 13:22:12 crc kubenswrapper[5052]: E1203 13:22:12.673946 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 13:22:16 crc kubenswrapper[5052]: I1203 13:22:16.975299 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-operators/keystone-operator-index-9ddkd" Dec 03 13:22:16 crc kubenswrapper[5052]: I1203 13:22:16.975687 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-operators/keystone-operator-index-9ddkd" Dec 03 13:22:17 crc kubenswrapper[5052]: I1203 13:22:17.007420 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-operators/keystone-operator-index-9ddkd" Dec 03 13:22:17 crc kubenswrapper[5052]: I1203 13:22:17.863895 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-operators/keystone-operator-index-9ddkd" Dec 03 13:22:19 crc kubenswrapper[5052]: I1203 13:22:19.655302 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["keystone-kuttl-tests/rabbitmq-server-0"] Dec 03 13:22:19 crc kubenswrapper[5052]: I1203 13:22:19.657277 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="keystone-kuttl-tests/rabbitmq-server-0" Dec 03 13:22:19 crc kubenswrapper[5052]: I1203 13:22:19.659530 5052 reflector.go:368] Caches populated for *v1.Secret from object-"keystone-kuttl-tests"/"rabbitmq-erlang-cookie" Dec 03 13:22:19 crc kubenswrapper[5052]: I1203 13:22:19.660334 5052 reflector.go:368] Caches populated for *v1.Secret from object-"keystone-kuttl-tests"/"rabbitmq-default-user" Dec 03 13:22:19 crc kubenswrapper[5052]: I1203 13:22:19.661281 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"keystone-kuttl-tests"/"rabbitmq-server-conf" Dec 03 13:22:19 crc kubenswrapper[5052]: I1203 13:22:19.661674 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"keystone-kuttl-tests"/"rabbitmq-plugins-conf" Dec 03 13:22:19 crc kubenswrapper[5052]: I1203 13:22:19.662367 5052 reflector.go:368] Caches populated for *v1.Secret from object-"keystone-kuttl-tests"/"rabbitmq-server-dockercfg-msnrd" Dec 03 13:22:19 crc kubenswrapper[5052]: I1203 13:22:19.668808 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["keystone-kuttl-tests/rabbitmq-server-0"] Dec 03 13:22:19 crc kubenswrapper[5052]: I1203 13:22:19.772177 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"rabbitmq-confd\" (UniqueName: \"kubernetes.io/projected/9d2e4b3b-6b0f-4e92-a6bb-fa0df37feb28-rabbitmq-confd\") pod \"rabbitmq-server-0\" (UID: \"9d2e4b3b-6b0f-4e92-a6bb-fa0df37feb28\") " pod="keystone-kuttl-tests/rabbitmq-server-0" Dec 03 13:22:19 crc kubenswrapper[5052]: I1203 13:22:19.772239 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"erlang-cookie-secret\" (UniqueName: \"kubernetes.io/secret/9d2e4b3b-6b0f-4e92-a6bb-fa0df37feb28-erlang-cookie-secret\") pod \"rabbitmq-server-0\" (UID: \"9d2e4b3b-6b0f-4e92-a6bb-fa0df37feb28\") " pod="keystone-kuttl-tests/rabbitmq-server-0" Dec 03 13:22:19 crc kubenswrapper[5052]: I1203 13:22:19.772305 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hzlgb\" (UniqueName: \"kubernetes.io/projected/9d2e4b3b-6b0f-4e92-a6bb-fa0df37feb28-kube-api-access-hzlgb\") pod \"rabbitmq-server-0\" (UID: \"9d2e4b3b-6b0f-4e92-a6bb-fa0df37feb28\") " pod="keystone-kuttl-tests/rabbitmq-server-0" Dec 03 13:22:19 crc kubenswrapper[5052]: I1203 13:22:19.772332 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"pvc-10491a20-0ef4-41c6-b028-50667b3c5171\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-10491a20-0ef4-41c6-b028-50667b3c5171\") pod \"rabbitmq-server-0\" (UID: \"9d2e4b3b-6b0f-4e92-a6bb-fa0df37feb28\") " pod="keystone-kuttl-tests/rabbitmq-server-0" Dec 03 13:22:19 crc kubenswrapper[5052]: I1203 13:22:19.772366 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-conf\" (UniqueName: \"kubernetes.io/configmap/9d2e4b3b-6b0f-4e92-a6bb-fa0df37feb28-plugins-conf\") pod \"rabbitmq-server-0\" (UID: \"9d2e4b3b-6b0f-4e92-a6bb-fa0df37feb28\") " pod="keystone-kuttl-tests/rabbitmq-server-0" Dec 03 13:22:19 crc kubenswrapper[5052]: I1203 13:22:19.772435 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"rabbitmq-erlang-cookie\" (UniqueName: \"kubernetes.io/empty-dir/9d2e4b3b-6b0f-4e92-a6bb-fa0df37feb28-rabbitmq-erlang-cookie\") pod \"rabbitmq-server-0\" (UID: \"9d2e4b3b-6b0f-4e92-a6bb-fa0df37feb28\") " pod="keystone-kuttl-tests/rabbitmq-server-0" Dec 03 13:22:19 crc kubenswrapper[5052]: I1203 13:22:19.772564 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"rabbitmq-plugins\" (UniqueName: \"kubernetes.io/empty-dir/9d2e4b3b-6b0f-4e92-a6bb-fa0df37feb28-rabbitmq-plugins\") pod \"rabbitmq-server-0\" (UID: \"9d2e4b3b-6b0f-4e92-a6bb-fa0df37feb28\") " pod="keystone-kuttl-tests/rabbitmq-server-0" Dec 03 13:22:19 crc kubenswrapper[5052]: I1203 13:22:19.772593 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"pod-info\" (UniqueName: \"kubernetes.io/downward-api/9d2e4b3b-6b0f-4e92-a6bb-fa0df37feb28-pod-info\") pod \"rabbitmq-server-0\" (UID: \"9d2e4b3b-6b0f-4e92-a6bb-fa0df37feb28\") " pod="keystone-kuttl-tests/rabbitmq-server-0" Dec 03 13:22:19 crc kubenswrapper[5052]: I1203 13:22:19.873449 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-hzlgb\" (UniqueName: \"kubernetes.io/projected/9d2e4b3b-6b0f-4e92-a6bb-fa0df37feb28-kube-api-access-hzlgb\") pod \"rabbitmq-server-0\" (UID: \"9d2e4b3b-6b0f-4e92-a6bb-fa0df37feb28\") " pod="keystone-kuttl-tests/rabbitmq-server-0" Dec 03 13:22:19 crc kubenswrapper[5052]: I1203 13:22:19.873523 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"pvc-10491a20-0ef4-41c6-b028-50667b3c5171\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-10491a20-0ef4-41c6-b028-50667b3c5171\") pod \"rabbitmq-server-0\" (UID: \"9d2e4b3b-6b0f-4e92-a6bb-fa0df37feb28\") " pod="keystone-kuttl-tests/rabbitmq-server-0" Dec 03 13:22:19 crc kubenswrapper[5052]: I1203 13:22:19.873559 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"plugins-conf\" (UniqueName: \"kubernetes.io/configmap/9d2e4b3b-6b0f-4e92-a6bb-fa0df37feb28-plugins-conf\") pod \"rabbitmq-server-0\" (UID: \"9d2e4b3b-6b0f-4e92-a6bb-fa0df37feb28\") " pod="keystone-kuttl-tests/rabbitmq-server-0" Dec 03 13:22:19 crc kubenswrapper[5052]: I1203 13:22:19.873599 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"rabbitmq-erlang-cookie\" (UniqueName: \"kubernetes.io/empty-dir/9d2e4b3b-6b0f-4e92-a6bb-fa0df37feb28-rabbitmq-erlang-cookie\") pod \"rabbitmq-server-0\" (UID: \"9d2e4b3b-6b0f-4e92-a6bb-fa0df37feb28\") " pod="keystone-kuttl-tests/rabbitmq-server-0" Dec 03 13:22:19 crc kubenswrapper[5052]: I1203 13:22:19.873658 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"rabbitmq-plugins\" (UniqueName: \"kubernetes.io/empty-dir/9d2e4b3b-6b0f-4e92-a6bb-fa0df37feb28-rabbitmq-plugins\") pod \"rabbitmq-server-0\" (UID: \"9d2e4b3b-6b0f-4e92-a6bb-fa0df37feb28\") " pod="keystone-kuttl-tests/rabbitmq-server-0" Dec 03 13:22:19 crc kubenswrapper[5052]: I1203 13:22:19.873683 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"pod-info\" (UniqueName: \"kubernetes.io/downward-api/9d2e4b3b-6b0f-4e92-a6bb-fa0df37feb28-pod-info\") pod \"rabbitmq-server-0\" (UID: \"9d2e4b3b-6b0f-4e92-a6bb-fa0df37feb28\") " pod="keystone-kuttl-tests/rabbitmq-server-0" Dec 03 13:22:19 crc kubenswrapper[5052]: I1203 13:22:19.873718 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"rabbitmq-confd\" (UniqueName: \"kubernetes.io/projected/9d2e4b3b-6b0f-4e92-a6bb-fa0df37feb28-rabbitmq-confd\") pod \"rabbitmq-server-0\" (UID: \"9d2e4b3b-6b0f-4e92-a6bb-fa0df37feb28\") " pod="keystone-kuttl-tests/rabbitmq-server-0" Dec 03 13:22:19 crc kubenswrapper[5052]: I1203 13:22:19.873751 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"erlang-cookie-secret\" (UniqueName: \"kubernetes.io/secret/9d2e4b3b-6b0f-4e92-a6bb-fa0df37feb28-erlang-cookie-secret\") pod \"rabbitmq-server-0\" (UID: \"9d2e4b3b-6b0f-4e92-a6bb-fa0df37feb28\") " pod="keystone-kuttl-tests/rabbitmq-server-0" Dec 03 13:22:19 crc kubenswrapper[5052]: I1203 13:22:19.874158 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"rabbitmq-plugins\" (UniqueName: \"kubernetes.io/empty-dir/9d2e4b3b-6b0f-4e92-a6bb-fa0df37feb28-rabbitmq-plugins\") pod \"rabbitmq-server-0\" (UID: \"9d2e4b3b-6b0f-4e92-a6bb-fa0df37feb28\") " pod="keystone-kuttl-tests/rabbitmq-server-0" Dec 03 13:22:19 crc kubenswrapper[5052]: I1203 13:22:19.874466 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"rabbitmq-erlang-cookie\" (UniqueName: \"kubernetes.io/empty-dir/9d2e4b3b-6b0f-4e92-a6bb-fa0df37feb28-rabbitmq-erlang-cookie\") pod \"rabbitmq-server-0\" (UID: \"9d2e4b3b-6b0f-4e92-a6bb-fa0df37feb28\") " pod="keystone-kuttl-tests/rabbitmq-server-0" Dec 03 13:22:19 crc kubenswrapper[5052]: I1203 13:22:19.874926 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"plugins-conf\" (UniqueName: \"kubernetes.io/configmap/9d2e4b3b-6b0f-4e92-a6bb-fa0df37feb28-plugins-conf\") pod \"rabbitmq-server-0\" (UID: \"9d2e4b3b-6b0f-4e92-a6bb-fa0df37feb28\") " pod="keystone-kuttl-tests/rabbitmq-server-0" Dec 03 13:22:19 crc kubenswrapper[5052]: I1203 13:22:19.876314 5052 csi_attacher.go:380] kubernetes.io/csi: attacher.MountDevice STAGE_UNSTAGE_VOLUME capability not set. Skipping MountDevice... Dec 03 13:22:19 crc kubenswrapper[5052]: I1203 13:22:19.876343 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"pvc-10491a20-0ef4-41c6-b028-50667b3c5171\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-10491a20-0ef4-41c6-b028-50667b3c5171\") pod \"rabbitmq-server-0\" (UID: \"9d2e4b3b-6b0f-4e92-a6bb-fa0df37feb28\") device mount path \"/var/lib/kubelet/plugins/kubernetes.io/csi/kubevirt.io.hostpath-provisioner/7e517517a120c2c6343724cb202f37d8d1832e6b6bd6acb50c7df2c68be306e8/globalmount\"" pod="keystone-kuttl-tests/rabbitmq-server-0" Dec 03 13:22:19 crc kubenswrapper[5052]: I1203 13:22:19.879798 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"pod-info\" (UniqueName: \"kubernetes.io/downward-api/9d2e4b3b-6b0f-4e92-a6bb-fa0df37feb28-pod-info\") pod \"rabbitmq-server-0\" (UID: \"9d2e4b3b-6b0f-4e92-a6bb-fa0df37feb28\") " pod="keystone-kuttl-tests/rabbitmq-server-0" Dec 03 13:22:19 crc kubenswrapper[5052]: I1203 13:22:19.885562 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"rabbitmq-confd\" (UniqueName: \"kubernetes.io/projected/9d2e4b3b-6b0f-4e92-a6bb-fa0df37feb28-rabbitmq-confd\") pod \"rabbitmq-server-0\" (UID: \"9d2e4b3b-6b0f-4e92-a6bb-fa0df37feb28\") " pod="keystone-kuttl-tests/rabbitmq-server-0" Dec 03 13:22:19 crc kubenswrapper[5052]: I1203 13:22:19.886328 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"erlang-cookie-secret\" (UniqueName: \"kubernetes.io/secret/9d2e4b3b-6b0f-4e92-a6bb-fa0df37feb28-erlang-cookie-secret\") pod \"rabbitmq-server-0\" (UID: \"9d2e4b3b-6b0f-4e92-a6bb-fa0df37feb28\") " pod="keystone-kuttl-tests/rabbitmq-server-0" Dec 03 13:22:19 crc kubenswrapper[5052]: I1203 13:22:19.893735 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-hzlgb\" (UniqueName: \"kubernetes.io/projected/9d2e4b3b-6b0f-4e92-a6bb-fa0df37feb28-kube-api-access-hzlgb\") pod \"rabbitmq-server-0\" (UID: \"9d2e4b3b-6b0f-4e92-a6bb-fa0df37feb28\") " pod="keystone-kuttl-tests/rabbitmq-server-0" Dec 03 13:22:19 crc kubenswrapper[5052]: I1203 13:22:19.910721 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"pvc-10491a20-0ef4-41c6-b028-50667b3c5171\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-10491a20-0ef4-41c6-b028-50667b3c5171\") pod \"rabbitmq-server-0\" (UID: \"9d2e4b3b-6b0f-4e92-a6bb-fa0df37feb28\") " pod="keystone-kuttl-tests/rabbitmq-server-0" Dec 03 13:22:19 crc kubenswrapper[5052]: I1203 13:22:19.992081 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="keystone-kuttl-tests/rabbitmq-server-0" Dec 03 13:22:20 crc kubenswrapper[5052]: I1203 13:22:20.431050 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["keystone-kuttl-tests/rabbitmq-server-0"] Dec 03 13:22:20 crc kubenswrapper[5052]: I1203 13:22:20.794116 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="keystone-kuttl-tests/rabbitmq-server-0" event={"ID":"9d2e4b3b-6b0f-4e92-a6bb-fa0df37feb28","Type":"ContainerStarted","Data":"ed2fc97665e06cef2aa063ab759091835ea52d8037ffdfdf0f95ee224cb95dd8"} Dec 03 13:22:22 crc kubenswrapper[5052]: I1203 13:22:22.809858 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="keystone-kuttl-tests/rabbitmq-server-0" event={"ID":"9d2e4b3b-6b0f-4e92-a6bb-fa0df37feb28","Type":"ContainerStarted","Data":"3807b3fde4c670a420fbb539c30bdd9519a330175de6eda8aac8288b9c2663b4"} Dec 03 13:22:24 crc kubenswrapper[5052]: I1203 13:22:24.912216 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-operators/49c083020ae5dfe237b73a6c6b807501660a323f061d7879268c43a121jj5pq"] Dec 03 13:22:24 crc kubenswrapper[5052]: I1203 13:22:24.914127 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-operators/49c083020ae5dfe237b73a6c6b807501660a323f061d7879268c43a121jj5pq" Dec 03 13:22:24 crc kubenswrapper[5052]: I1203 13:22:24.922356 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-operators"/"default-dockercfg-7fklr" Dec 03 13:22:24 crc kubenswrapper[5052]: I1203 13:22:24.923410 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-operators/49c083020ae5dfe237b73a6c6b807501660a323f061d7879268c43a121jj5pq"] Dec 03 13:22:25 crc kubenswrapper[5052]: I1203 13:22:25.056075 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6a7ab332-9572-45d1-bec6-0dab8e148eb0-bundle\") pod \"49c083020ae5dfe237b73a6c6b807501660a323f061d7879268c43a121jj5pq\" (UID: \"6a7ab332-9572-45d1-bec6-0dab8e148eb0\") " pod="openstack-operators/49c083020ae5dfe237b73a6c6b807501660a323f061d7879268c43a121jj5pq" Dec 03 13:22:25 crc kubenswrapper[5052]: I1203 13:22:25.056439 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cchfm\" (UniqueName: \"kubernetes.io/projected/6a7ab332-9572-45d1-bec6-0dab8e148eb0-kube-api-access-cchfm\") pod \"49c083020ae5dfe237b73a6c6b807501660a323f061d7879268c43a121jj5pq\" (UID: \"6a7ab332-9572-45d1-bec6-0dab8e148eb0\") " pod="openstack-operators/49c083020ae5dfe237b73a6c6b807501660a323f061d7879268c43a121jj5pq" Dec 03 13:22:25 crc kubenswrapper[5052]: I1203 13:22:25.056586 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6a7ab332-9572-45d1-bec6-0dab8e148eb0-util\") pod \"49c083020ae5dfe237b73a6c6b807501660a323f061d7879268c43a121jj5pq\" (UID: \"6a7ab332-9572-45d1-bec6-0dab8e148eb0\") " pod="openstack-operators/49c083020ae5dfe237b73a6c6b807501660a323f061d7879268c43a121jj5pq" Dec 03 13:22:25 crc kubenswrapper[5052]: I1203 13:22:25.157808 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6a7ab332-9572-45d1-bec6-0dab8e148eb0-bundle\") pod \"49c083020ae5dfe237b73a6c6b807501660a323f061d7879268c43a121jj5pq\" (UID: \"6a7ab332-9572-45d1-bec6-0dab8e148eb0\") " pod="openstack-operators/49c083020ae5dfe237b73a6c6b807501660a323f061d7879268c43a121jj5pq" Dec 03 13:22:25 crc kubenswrapper[5052]: I1203 13:22:25.157902 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-cchfm\" (UniqueName: \"kubernetes.io/projected/6a7ab332-9572-45d1-bec6-0dab8e148eb0-kube-api-access-cchfm\") pod \"49c083020ae5dfe237b73a6c6b807501660a323f061d7879268c43a121jj5pq\" (UID: \"6a7ab332-9572-45d1-bec6-0dab8e148eb0\") " pod="openstack-operators/49c083020ae5dfe237b73a6c6b807501660a323f061d7879268c43a121jj5pq" Dec 03 13:22:25 crc kubenswrapper[5052]: I1203 13:22:25.157939 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6a7ab332-9572-45d1-bec6-0dab8e148eb0-util\") pod \"49c083020ae5dfe237b73a6c6b807501660a323f061d7879268c43a121jj5pq\" (UID: \"6a7ab332-9572-45d1-bec6-0dab8e148eb0\") " pod="openstack-operators/49c083020ae5dfe237b73a6c6b807501660a323f061d7879268c43a121jj5pq" Dec 03 13:22:25 crc kubenswrapper[5052]: I1203 13:22:25.158337 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6a7ab332-9572-45d1-bec6-0dab8e148eb0-bundle\") pod \"49c083020ae5dfe237b73a6c6b807501660a323f061d7879268c43a121jj5pq\" (UID: \"6a7ab332-9572-45d1-bec6-0dab8e148eb0\") " pod="openstack-operators/49c083020ae5dfe237b73a6c6b807501660a323f061d7879268c43a121jj5pq" Dec 03 13:22:25 crc kubenswrapper[5052]: I1203 13:22:25.158398 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6a7ab332-9572-45d1-bec6-0dab8e148eb0-util\") pod \"49c083020ae5dfe237b73a6c6b807501660a323f061d7879268c43a121jj5pq\" (UID: \"6a7ab332-9572-45d1-bec6-0dab8e148eb0\") " pod="openstack-operators/49c083020ae5dfe237b73a6c6b807501660a323f061d7879268c43a121jj5pq" Dec 03 13:22:25 crc kubenswrapper[5052]: I1203 13:22:25.183344 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-cchfm\" (UniqueName: \"kubernetes.io/projected/6a7ab332-9572-45d1-bec6-0dab8e148eb0-kube-api-access-cchfm\") pod \"49c083020ae5dfe237b73a6c6b807501660a323f061d7879268c43a121jj5pq\" (UID: \"6a7ab332-9572-45d1-bec6-0dab8e148eb0\") " pod="openstack-operators/49c083020ae5dfe237b73a6c6b807501660a323f061d7879268c43a121jj5pq" Dec 03 13:22:25 crc kubenswrapper[5052]: I1203 13:22:25.246659 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-operators/49c083020ae5dfe237b73a6c6b807501660a323f061d7879268c43a121jj5pq" Dec 03 13:22:25 crc kubenswrapper[5052]: I1203 13:22:25.689067 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-operators/49c083020ae5dfe237b73a6c6b807501660a323f061d7879268c43a121jj5pq"] Dec 03 13:22:25 crc kubenswrapper[5052]: I1203 13:22:25.836886 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/49c083020ae5dfe237b73a6c6b807501660a323f061d7879268c43a121jj5pq" event={"ID":"6a7ab332-9572-45d1-bec6-0dab8e148eb0","Type":"ContainerStarted","Data":"10fe8fd770d9f97ed962e4bb391499680377b3301a6e31c137a7b58e2d031301"} Dec 03 13:22:26 crc kubenswrapper[5052]: I1203 13:22:26.673873 5052 scope.go:117] "RemoveContainer" containerID="12a8b1f470f6d59d7700e376f2a48eb36363f91f9823a35ce534062c0974c10c" Dec 03 13:22:26 crc kubenswrapper[5052]: E1203 13:22:26.674394 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 13:22:26 crc kubenswrapper[5052]: I1203 13:22:26.846857 5052 generic.go:334] "Generic (PLEG): container finished" podID="6a7ab332-9572-45d1-bec6-0dab8e148eb0" containerID="ba99885028205208f9fcdb8293b68a3b0bcb51bede04b7a8197d8286cc8e79b5" exitCode=0 Dec 03 13:22:26 crc kubenswrapper[5052]: I1203 13:22:26.846905 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/49c083020ae5dfe237b73a6c6b807501660a323f061d7879268c43a121jj5pq" event={"ID":"6a7ab332-9572-45d1-bec6-0dab8e148eb0","Type":"ContainerDied","Data":"ba99885028205208f9fcdb8293b68a3b0bcb51bede04b7a8197d8286cc8e79b5"} Dec 03 13:22:27 crc kubenswrapper[5052]: I1203 13:22:27.858097 5052 generic.go:334] "Generic (PLEG): container finished" podID="6a7ab332-9572-45d1-bec6-0dab8e148eb0" containerID="6c963b738a30433ba59b5eb6bb7791cca4717b29ee25a4df081130697ce887f2" exitCode=0 Dec 03 13:22:27 crc kubenswrapper[5052]: I1203 13:22:27.858149 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/49c083020ae5dfe237b73a6c6b807501660a323f061d7879268c43a121jj5pq" event={"ID":"6a7ab332-9572-45d1-bec6-0dab8e148eb0","Type":"ContainerDied","Data":"6c963b738a30433ba59b5eb6bb7791cca4717b29ee25a4df081130697ce887f2"} Dec 03 13:22:28 crc kubenswrapper[5052]: I1203 13:22:28.871786 5052 generic.go:334] "Generic (PLEG): container finished" podID="6a7ab332-9572-45d1-bec6-0dab8e148eb0" containerID="e061fd78a790655f73443f302c63c58b18336d6d8a5985a5f8ba3ceae2993a4c" exitCode=0 Dec 03 13:22:28 crc kubenswrapper[5052]: I1203 13:22:28.871859 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/49c083020ae5dfe237b73a6c6b807501660a323f061d7879268c43a121jj5pq" event={"ID":"6a7ab332-9572-45d1-bec6-0dab8e148eb0","Type":"ContainerDied","Data":"e061fd78a790655f73443f302c63c58b18336d6d8a5985a5f8ba3ceae2993a4c"} Dec 03 13:22:30 crc kubenswrapper[5052]: I1203 13:22:30.178151 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-operators/49c083020ae5dfe237b73a6c6b807501660a323f061d7879268c43a121jj5pq" Dec 03 13:22:30 crc kubenswrapper[5052]: I1203 13:22:30.245406 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6a7ab332-9572-45d1-bec6-0dab8e148eb0-bundle\") pod \"6a7ab332-9572-45d1-bec6-0dab8e148eb0\" (UID: \"6a7ab332-9572-45d1-bec6-0dab8e148eb0\") " Dec 03 13:22:30 crc kubenswrapper[5052]: I1203 13:22:30.245500 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6a7ab332-9572-45d1-bec6-0dab8e148eb0-util\") pod \"6a7ab332-9572-45d1-bec6-0dab8e148eb0\" (UID: \"6a7ab332-9572-45d1-bec6-0dab8e148eb0\") " Dec 03 13:22:30 crc kubenswrapper[5052]: I1203 13:22:30.245612 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cchfm\" (UniqueName: \"kubernetes.io/projected/6a7ab332-9572-45d1-bec6-0dab8e148eb0-kube-api-access-cchfm\") pod \"6a7ab332-9572-45d1-bec6-0dab8e148eb0\" (UID: \"6a7ab332-9572-45d1-bec6-0dab8e148eb0\") " Dec 03 13:22:30 crc kubenswrapper[5052]: I1203 13:22:30.247061 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a7ab332-9572-45d1-bec6-0dab8e148eb0-bundle" (OuterVolumeSpecName: "bundle") pod "6a7ab332-9572-45d1-bec6-0dab8e148eb0" (UID: "6a7ab332-9572-45d1-bec6-0dab8e148eb0"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 13:22:30 crc kubenswrapper[5052]: I1203 13:22:30.252909 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6a7ab332-9572-45d1-bec6-0dab8e148eb0-kube-api-access-cchfm" (OuterVolumeSpecName: "kube-api-access-cchfm") pod "6a7ab332-9572-45d1-bec6-0dab8e148eb0" (UID: "6a7ab332-9572-45d1-bec6-0dab8e148eb0"). InnerVolumeSpecName "kube-api-access-cchfm". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:22:30 crc kubenswrapper[5052]: I1203 13:22:30.261370 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a7ab332-9572-45d1-bec6-0dab8e148eb0-util" (OuterVolumeSpecName: "util") pod "6a7ab332-9572-45d1-bec6-0dab8e148eb0" (UID: "6a7ab332-9572-45d1-bec6-0dab8e148eb0"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 13:22:30 crc kubenswrapper[5052]: I1203 13:22:30.347101 5052 reconciler_common.go:293] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6a7ab332-9572-45d1-bec6-0dab8e148eb0-util\") on node \"crc\" DevicePath \"\"" Dec 03 13:22:30 crc kubenswrapper[5052]: I1203 13:22:30.347154 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-cchfm\" (UniqueName: \"kubernetes.io/projected/6a7ab332-9572-45d1-bec6-0dab8e148eb0-kube-api-access-cchfm\") on node \"crc\" DevicePath \"\"" Dec 03 13:22:30 crc kubenswrapper[5052]: I1203 13:22:30.347166 5052 reconciler_common.go:293] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6a7ab332-9572-45d1-bec6-0dab8e148eb0-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 13:22:30 crc kubenswrapper[5052]: I1203 13:22:30.894349 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/49c083020ae5dfe237b73a6c6b807501660a323f061d7879268c43a121jj5pq" event={"ID":"6a7ab332-9572-45d1-bec6-0dab8e148eb0","Type":"ContainerDied","Data":"10fe8fd770d9f97ed962e4bb391499680377b3301a6e31c137a7b58e2d031301"} Dec 03 13:22:30 crc kubenswrapper[5052]: I1203 13:22:30.894396 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="10fe8fd770d9f97ed962e4bb391499680377b3301a6e31c137a7b58e2d031301" Dec 03 13:22:30 crc kubenswrapper[5052]: I1203 13:22:30.894456 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-operators/49c083020ae5dfe237b73a6c6b807501660a323f061d7879268c43a121jj5pq" Dec 03 13:22:41 crc kubenswrapper[5052]: I1203 13:22:41.673827 5052 scope.go:117] "RemoveContainer" containerID="12a8b1f470f6d59d7700e376f2a48eb36363f91f9823a35ce534062c0974c10c" Dec 03 13:22:41 crc kubenswrapper[5052]: E1203 13:22:41.675276 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 13:22:42 crc kubenswrapper[5052]: I1203 13:22:42.554790 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-operators/keystone-operator-controller-manager-6f4cbcd5b5-8gmln"] Dec 03 13:22:42 crc kubenswrapper[5052]: E1203 13:22:42.555171 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="6a7ab332-9572-45d1-bec6-0dab8e148eb0" containerName="util" Dec 03 13:22:42 crc kubenswrapper[5052]: I1203 13:22:42.555189 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="6a7ab332-9572-45d1-bec6-0dab8e148eb0" containerName="util" Dec 03 13:22:42 crc kubenswrapper[5052]: E1203 13:22:42.555219 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="6a7ab332-9572-45d1-bec6-0dab8e148eb0" containerName="pull" Dec 03 13:22:42 crc kubenswrapper[5052]: I1203 13:22:42.555226 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="6a7ab332-9572-45d1-bec6-0dab8e148eb0" containerName="pull" Dec 03 13:22:42 crc kubenswrapper[5052]: E1203 13:22:42.555244 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="6a7ab332-9572-45d1-bec6-0dab8e148eb0" containerName="extract" Dec 03 13:22:42 crc kubenswrapper[5052]: I1203 13:22:42.555251 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="6a7ab332-9572-45d1-bec6-0dab8e148eb0" containerName="extract" Dec 03 13:22:42 crc kubenswrapper[5052]: I1203 13:22:42.555391 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="6a7ab332-9572-45d1-bec6-0dab8e148eb0" containerName="extract" Dec 03 13:22:42 crc kubenswrapper[5052]: I1203 13:22:42.556003 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-operators/keystone-operator-controller-manager-6f4cbcd5b5-8gmln" Dec 03 13:22:42 crc kubenswrapper[5052]: I1203 13:22:42.558658 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-operators"/"keystone-operator-controller-manager-service-cert" Dec 03 13:22:42 crc kubenswrapper[5052]: I1203 13:22:42.558843 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-operators"/"keystone-operator-controller-manager-dockercfg-vwr4s" Dec 03 13:22:42 crc kubenswrapper[5052]: I1203 13:22:42.590459 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-operators/keystone-operator-controller-manager-6f4cbcd5b5-8gmln"] Dec 03 13:22:42 crc kubenswrapper[5052]: I1203 13:22:42.651759 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ae81dcbe-7c90-4de6-94bd-014eb4e9557f-webhook-cert\") pod \"keystone-operator-controller-manager-6f4cbcd5b5-8gmln\" (UID: \"ae81dcbe-7c90-4de6-94bd-014eb4e9557f\") " pod="openstack-operators/keystone-operator-controller-manager-6f4cbcd5b5-8gmln" Dec 03 13:22:42 crc kubenswrapper[5052]: I1203 13:22:42.651942 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ae81dcbe-7c90-4de6-94bd-014eb4e9557f-apiservice-cert\") pod \"keystone-operator-controller-manager-6f4cbcd5b5-8gmln\" (UID: \"ae81dcbe-7c90-4de6-94bd-014eb4e9557f\") " pod="openstack-operators/keystone-operator-controller-manager-6f4cbcd5b5-8gmln" Dec 03 13:22:42 crc kubenswrapper[5052]: I1203 13:22:42.652019 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8stn5\" (UniqueName: \"kubernetes.io/projected/ae81dcbe-7c90-4de6-94bd-014eb4e9557f-kube-api-access-8stn5\") pod \"keystone-operator-controller-manager-6f4cbcd5b5-8gmln\" (UID: \"ae81dcbe-7c90-4de6-94bd-014eb4e9557f\") " pod="openstack-operators/keystone-operator-controller-manager-6f4cbcd5b5-8gmln" Dec 03 13:22:42 crc kubenswrapper[5052]: I1203 13:22:42.753744 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ae81dcbe-7c90-4de6-94bd-014eb4e9557f-webhook-cert\") pod \"keystone-operator-controller-manager-6f4cbcd5b5-8gmln\" (UID: \"ae81dcbe-7c90-4de6-94bd-014eb4e9557f\") " pod="openstack-operators/keystone-operator-controller-manager-6f4cbcd5b5-8gmln" Dec 03 13:22:42 crc kubenswrapper[5052]: I1203 13:22:42.753911 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ae81dcbe-7c90-4de6-94bd-014eb4e9557f-apiservice-cert\") pod \"keystone-operator-controller-manager-6f4cbcd5b5-8gmln\" (UID: \"ae81dcbe-7c90-4de6-94bd-014eb4e9557f\") " pod="openstack-operators/keystone-operator-controller-manager-6f4cbcd5b5-8gmln" Dec 03 13:22:42 crc kubenswrapper[5052]: I1203 13:22:42.754933 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-8stn5\" (UniqueName: \"kubernetes.io/projected/ae81dcbe-7c90-4de6-94bd-014eb4e9557f-kube-api-access-8stn5\") pod \"keystone-operator-controller-manager-6f4cbcd5b5-8gmln\" (UID: \"ae81dcbe-7c90-4de6-94bd-014eb4e9557f\") " pod="openstack-operators/keystone-operator-controller-manager-6f4cbcd5b5-8gmln" Dec 03 13:22:42 crc kubenswrapper[5052]: I1203 13:22:42.774462 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ae81dcbe-7c90-4de6-94bd-014eb4e9557f-apiservice-cert\") pod \"keystone-operator-controller-manager-6f4cbcd5b5-8gmln\" (UID: \"ae81dcbe-7c90-4de6-94bd-014eb4e9557f\") " pod="openstack-operators/keystone-operator-controller-manager-6f4cbcd5b5-8gmln" Dec 03 13:22:42 crc kubenswrapper[5052]: I1203 13:22:42.774768 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ae81dcbe-7c90-4de6-94bd-014eb4e9557f-webhook-cert\") pod \"keystone-operator-controller-manager-6f4cbcd5b5-8gmln\" (UID: \"ae81dcbe-7c90-4de6-94bd-014eb4e9557f\") " pod="openstack-operators/keystone-operator-controller-manager-6f4cbcd5b5-8gmln" Dec 03 13:22:42 crc kubenswrapper[5052]: I1203 13:22:42.784415 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-8stn5\" (UniqueName: \"kubernetes.io/projected/ae81dcbe-7c90-4de6-94bd-014eb4e9557f-kube-api-access-8stn5\") pod \"keystone-operator-controller-manager-6f4cbcd5b5-8gmln\" (UID: \"ae81dcbe-7c90-4de6-94bd-014eb4e9557f\") " pod="openstack-operators/keystone-operator-controller-manager-6f4cbcd5b5-8gmln" Dec 03 13:22:42 crc kubenswrapper[5052]: I1203 13:22:42.877866 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-operators/keystone-operator-controller-manager-6f4cbcd5b5-8gmln" Dec 03 13:22:43 crc kubenswrapper[5052]: I1203 13:22:43.315720 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-operators/keystone-operator-controller-manager-6f4cbcd5b5-8gmln"] Dec 03 13:22:44 crc kubenswrapper[5052]: I1203 13:22:44.021243 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/keystone-operator-controller-manager-6f4cbcd5b5-8gmln" event={"ID":"ae81dcbe-7c90-4de6-94bd-014eb4e9557f","Type":"ContainerStarted","Data":"deb1f298b409739ff6b92cb257ecf5946876b50d914e7262651ac96341518d6e"} Dec 03 13:22:44 crc kubenswrapper[5052]: I1203 13:22:44.021602 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/keystone-operator-controller-manager-6f4cbcd5b5-8gmln" event={"ID":"ae81dcbe-7c90-4de6-94bd-014eb4e9557f","Type":"ContainerStarted","Data":"c40fff398775ad8ba60b54de93d6068e174d54988001c53e16b6e2c6387e0579"} Dec 03 13:22:44 crc kubenswrapper[5052]: I1203 13:22:44.021623 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-operators/keystone-operator-controller-manager-6f4cbcd5b5-8gmln" Dec 03 13:22:44 crc kubenswrapper[5052]: I1203 13:22:44.060382 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-operators/keystone-operator-controller-manager-6f4cbcd5b5-8gmln" podStartSLOduration=2.060359416 podStartE2EDuration="2.060359416s" podCreationTimestamp="2025-12-03 13:22:42 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 13:22:44.046935611 +0000 UTC m=+7354.635177385" watchObservedRunningTime="2025-12-03 13:22:44.060359416 +0000 UTC m=+7354.648601190" Dec 03 13:22:47 crc kubenswrapper[5052]: I1203 13:22:47.333168 5052 scope.go:117] "RemoveContainer" containerID="ed7869e5c41a6f93a71e760064518e708db2a09a0bfcceea88578d8c7aef5b48" Dec 03 13:22:47 crc kubenswrapper[5052]: I1203 13:22:47.365561 5052 scope.go:117] "RemoveContainer" containerID="e7a4da6499c8365eb070c94f4a88cd1c2a40f56b2ad8397b4c8ecc9e35e5d297" Dec 03 13:22:47 crc kubenswrapper[5052]: I1203 13:22:47.392262 5052 scope.go:117] "RemoveContainer" containerID="88b36946effde7a166f55150a328e073b1f02cf790e86bea202520684cdaf41c" Dec 03 13:22:47 crc kubenswrapper[5052]: I1203 13:22:47.416758 5052 scope.go:117] "RemoveContainer" containerID="969b3a09d6c12b5b9cee46d7bed7eabe1e6cdfab75e304e9f87fa15d482c3ea0" Dec 03 13:22:47 crc kubenswrapper[5052]: I1203 13:22:47.445259 5052 scope.go:117] "RemoveContainer" containerID="dd50b8915948090dfae0c0215b7cda1b0f1f8beadca40b4106938fda7400ba5d" Dec 03 13:22:47 crc kubenswrapper[5052]: I1203 13:22:47.476282 5052 scope.go:117] "RemoveContainer" containerID="70b90ec2bd7bf33637c4f77fd83ed552d859433d48e56f89379d0ab0cef6b4f4" Dec 03 13:22:52 crc kubenswrapper[5052]: I1203 13:22:52.882367 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-operators/keystone-operator-controller-manager-6f4cbcd5b5-8gmln" Dec 03 13:22:53 crc kubenswrapper[5052]: I1203 13:22:53.673792 5052 scope.go:117] "RemoveContainer" containerID="12a8b1f470f6d59d7700e376f2a48eb36363f91f9823a35ce534062c0974c10c" Dec 03 13:22:53 crc kubenswrapper[5052]: E1203 13:22:53.674175 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 13:22:54 crc kubenswrapper[5052]: I1203 13:22:54.097245 5052 generic.go:334] "Generic (PLEG): container finished" podID="9d2e4b3b-6b0f-4e92-a6bb-fa0df37feb28" containerID="3807b3fde4c670a420fbb539c30bdd9519a330175de6eda8aac8288b9c2663b4" exitCode=0 Dec 03 13:22:54 crc kubenswrapper[5052]: I1203 13:22:54.097363 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="keystone-kuttl-tests/rabbitmq-server-0" event={"ID":"9d2e4b3b-6b0f-4e92-a6bb-fa0df37feb28","Type":"ContainerDied","Data":"3807b3fde4c670a420fbb539c30bdd9519a330175de6eda8aac8288b9c2663b4"} Dec 03 13:22:55 crc kubenswrapper[5052]: I1203 13:22:55.106319 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="keystone-kuttl-tests/rabbitmq-server-0" event={"ID":"9d2e4b3b-6b0f-4e92-a6bb-fa0df37feb28","Type":"ContainerStarted","Data":"64166ff7fe7a645af18c176c3f4902178d3aa89fc21f02ad0a4020f06c1200e0"} Dec 03 13:22:55 crc kubenswrapper[5052]: I1203 13:22:55.107055 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="keystone-kuttl-tests/rabbitmq-server-0" Dec 03 13:22:55 crc kubenswrapper[5052]: I1203 13:22:55.129968 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="keystone-kuttl-tests/rabbitmq-server-0" podStartSLOduration=37.129948873 podStartE2EDuration="37.129948873s" podCreationTimestamp="2025-12-03 13:22:18 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 13:22:55.127168113 +0000 UTC m=+7365.715409917" watchObservedRunningTime="2025-12-03 13:22:55.129948873 +0000 UTC m=+7365.718190637" Dec 03 13:23:01 crc kubenswrapper[5052]: I1203 13:23:01.657437 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["keystone-kuttl-tests/keystone-236c-account-create-update-vjwmh"] Dec 03 13:23:01 crc kubenswrapper[5052]: I1203 13:23:01.659276 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="keystone-kuttl-tests/keystone-236c-account-create-update-vjwmh" Dec 03 13:23:01 crc kubenswrapper[5052]: I1203 13:23:01.663090 5052 reflector.go:368] Caches populated for *v1.Secret from object-"keystone-kuttl-tests"/"keystone-db-secret" Dec 03 13:23:01 crc kubenswrapper[5052]: I1203 13:23:01.668712 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["keystone-kuttl-tests/keystone-236c-account-create-update-vjwmh"] Dec 03 13:23:01 crc kubenswrapper[5052]: I1203 13:23:01.731408 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/57a8e2c4-46ec-4f92-9f60-2a6af9c546ff-operator-scripts\") pod \"keystone-236c-account-create-update-vjwmh\" (UID: \"57a8e2c4-46ec-4f92-9f60-2a6af9c546ff\") " pod="keystone-kuttl-tests/keystone-236c-account-create-update-vjwmh" Dec 03 13:23:01 crc kubenswrapper[5052]: I1203 13:23:01.731497 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-whtql\" (UniqueName: \"kubernetes.io/projected/57a8e2c4-46ec-4f92-9f60-2a6af9c546ff-kube-api-access-whtql\") pod \"keystone-236c-account-create-update-vjwmh\" (UID: \"57a8e2c4-46ec-4f92-9f60-2a6af9c546ff\") " pod="keystone-kuttl-tests/keystone-236c-account-create-update-vjwmh" Dec 03 13:23:01 crc kubenswrapper[5052]: I1203 13:23:01.744538 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["keystone-kuttl-tests/keystone-db-create-gnvhm"] Dec 03 13:23:01 crc kubenswrapper[5052]: I1203 13:23:01.747325 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="keystone-kuttl-tests/keystone-db-create-gnvhm" Dec 03 13:23:01 crc kubenswrapper[5052]: I1203 13:23:01.757032 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["keystone-kuttl-tests/keystone-db-create-gnvhm"] Dec 03 13:23:01 crc kubenswrapper[5052]: I1203 13:23:01.832655 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/57a8e2c4-46ec-4f92-9f60-2a6af9c546ff-operator-scripts\") pod \"keystone-236c-account-create-update-vjwmh\" (UID: \"57a8e2c4-46ec-4f92-9f60-2a6af9c546ff\") " pod="keystone-kuttl-tests/keystone-236c-account-create-update-vjwmh" Dec 03 13:23:01 crc kubenswrapper[5052]: I1203 13:23:01.832765 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-whtql\" (UniqueName: \"kubernetes.io/projected/57a8e2c4-46ec-4f92-9f60-2a6af9c546ff-kube-api-access-whtql\") pod \"keystone-236c-account-create-update-vjwmh\" (UID: \"57a8e2c4-46ec-4f92-9f60-2a6af9c546ff\") " pod="keystone-kuttl-tests/keystone-236c-account-create-update-vjwmh" Dec 03 13:23:01 crc kubenswrapper[5052]: I1203 13:23:01.833203 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v6bbs\" (UniqueName: \"kubernetes.io/projected/8f5a0f03-b75b-41b7-8f6a-695cf4c58a22-kube-api-access-v6bbs\") pod \"keystone-db-create-gnvhm\" (UID: \"8f5a0f03-b75b-41b7-8f6a-695cf4c58a22\") " pod="keystone-kuttl-tests/keystone-db-create-gnvhm" Dec 03 13:23:01 crc kubenswrapper[5052]: I1203 13:23:01.833296 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/8f5a0f03-b75b-41b7-8f6a-695cf4c58a22-operator-scripts\") pod \"keystone-db-create-gnvhm\" (UID: \"8f5a0f03-b75b-41b7-8f6a-695cf4c58a22\") " pod="keystone-kuttl-tests/keystone-db-create-gnvhm" Dec 03 13:23:01 crc kubenswrapper[5052]: I1203 13:23:01.833643 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/57a8e2c4-46ec-4f92-9f60-2a6af9c546ff-operator-scripts\") pod \"keystone-236c-account-create-update-vjwmh\" (UID: \"57a8e2c4-46ec-4f92-9f60-2a6af9c546ff\") " pod="keystone-kuttl-tests/keystone-236c-account-create-update-vjwmh" Dec 03 13:23:01 crc kubenswrapper[5052]: I1203 13:23:01.852335 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-whtql\" (UniqueName: \"kubernetes.io/projected/57a8e2c4-46ec-4f92-9f60-2a6af9c546ff-kube-api-access-whtql\") pod \"keystone-236c-account-create-update-vjwmh\" (UID: \"57a8e2c4-46ec-4f92-9f60-2a6af9c546ff\") " pod="keystone-kuttl-tests/keystone-236c-account-create-update-vjwmh" Dec 03 13:23:01 crc kubenswrapper[5052]: I1203 13:23:01.934751 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-v6bbs\" (UniqueName: \"kubernetes.io/projected/8f5a0f03-b75b-41b7-8f6a-695cf4c58a22-kube-api-access-v6bbs\") pod \"keystone-db-create-gnvhm\" (UID: \"8f5a0f03-b75b-41b7-8f6a-695cf4c58a22\") " pod="keystone-kuttl-tests/keystone-db-create-gnvhm" Dec 03 13:23:01 crc kubenswrapper[5052]: I1203 13:23:01.934832 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/8f5a0f03-b75b-41b7-8f6a-695cf4c58a22-operator-scripts\") pod \"keystone-db-create-gnvhm\" (UID: \"8f5a0f03-b75b-41b7-8f6a-695cf4c58a22\") " pod="keystone-kuttl-tests/keystone-db-create-gnvhm" Dec 03 13:23:01 crc kubenswrapper[5052]: I1203 13:23:01.935570 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/8f5a0f03-b75b-41b7-8f6a-695cf4c58a22-operator-scripts\") pod \"keystone-db-create-gnvhm\" (UID: \"8f5a0f03-b75b-41b7-8f6a-695cf4c58a22\") " pod="keystone-kuttl-tests/keystone-db-create-gnvhm" Dec 03 13:23:01 crc kubenswrapper[5052]: I1203 13:23:01.951931 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-v6bbs\" (UniqueName: \"kubernetes.io/projected/8f5a0f03-b75b-41b7-8f6a-695cf4c58a22-kube-api-access-v6bbs\") pod \"keystone-db-create-gnvhm\" (UID: \"8f5a0f03-b75b-41b7-8f6a-695cf4c58a22\") " pod="keystone-kuttl-tests/keystone-db-create-gnvhm" Dec 03 13:23:01 crc kubenswrapper[5052]: I1203 13:23:01.985989 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="keystone-kuttl-tests/keystone-236c-account-create-update-vjwmh" Dec 03 13:23:02 crc kubenswrapper[5052]: I1203 13:23:02.065466 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="keystone-kuttl-tests/keystone-db-create-gnvhm" Dec 03 13:23:02 crc kubenswrapper[5052]: I1203 13:23:02.411864 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["keystone-kuttl-tests/keystone-236c-account-create-update-vjwmh"] Dec 03 13:23:02 crc kubenswrapper[5052]: W1203 13:23:02.413976 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod57a8e2c4_46ec_4f92_9f60_2a6af9c546ff.slice/crio-af06e4c50ba312b6699173c533267e39d2dbbb58c7e63a1c016453b687db2302 WatchSource:0}: Error finding container af06e4c50ba312b6699173c533267e39d2dbbb58c7e63a1c016453b687db2302: Status 404 returned error can't find the container with id af06e4c50ba312b6699173c533267e39d2dbbb58c7e63a1c016453b687db2302 Dec 03 13:23:02 crc kubenswrapper[5052]: I1203 13:23:02.535449 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["keystone-kuttl-tests/keystone-db-create-gnvhm"] Dec 03 13:23:02 crc kubenswrapper[5052]: W1203 13:23:02.540735 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8f5a0f03_b75b_41b7_8f6a_695cf4c58a22.slice/crio-f76a223d1949c0a7b0198b8fc5ee6027468125a3f22d7b4daa6881491d4bb91e WatchSource:0}: Error finding container f76a223d1949c0a7b0198b8fc5ee6027468125a3f22d7b4daa6881491d4bb91e: Status 404 returned error can't find the container with id f76a223d1949c0a7b0198b8fc5ee6027468125a3f22d7b4daa6881491d4bb91e Dec 03 13:23:03 crc kubenswrapper[5052]: I1203 13:23:03.185488 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="keystone-kuttl-tests/keystone-236c-account-create-update-vjwmh" event={"ID":"57a8e2c4-46ec-4f92-9f60-2a6af9c546ff","Type":"ContainerStarted","Data":"e6eb2354b7b82fbf58211b1ade674871f298871d3dde2b23f2b3cda4d4628594"} Dec 03 13:23:03 crc kubenswrapper[5052]: I1203 13:23:03.185558 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="keystone-kuttl-tests/keystone-236c-account-create-update-vjwmh" event={"ID":"57a8e2c4-46ec-4f92-9f60-2a6af9c546ff","Type":"ContainerStarted","Data":"af06e4c50ba312b6699173c533267e39d2dbbb58c7e63a1c016453b687db2302"} Dec 03 13:23:03 crc kubenswrapper[5052]: I1203 13:23:03.188360 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="keystone-kuttl-tests/keystone-db-create-gnvhm" event={"ID":"8f5a0f03-b75b-41b7-8f6a-695cf4c58a22","Type":"ContainerStarted","Data":"5c25cf982fe9f2774b23e40fa8b85755d117d93a53b1f6ffe4553bf32d68b53c"} Dec 03 13:23:03 crc kubenswrapper[5052]: I1203 13:23:03.188405 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="keystone-kuttl-tests/keystone-db-create-gnvhm" event={"ID":"8f5a0f03-b75b-41b7-8f6a-695cf4c58a22","Type":"ContainerStarted","Data":"f76a223d1949c0a7b0198b8fc5ee6027468125a3f22d7b4daa6881491d4bb91e"} Dec 03 13:23:03 crc kubenswrapper[5052]: I1203 13:23:03.209043 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="keystone-kuttl-tests/keystone-236c-account-create-update-vjwmh" podStartSLOduration=2.209020407 podStartE2EDuration="2.209020407s" podCreationTimestamp="2025-12-03 13:23:01 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 13:23:03.20315688 +0000 UTC m=+7373.791398644" watchObservedRunningTime="2025-12-03 13:23:03.209020407 +0000 UTC m=+7373.797262171" Dec 03 13:23:03 crc kubenswrapper[5052]: I1203 13:23:03.218870 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="keystone-kuttl-tests/keystone-db-create-gnvhm" podStartSLOduration=2.218849119 podStartE2EDuration="2.218849119s" podCreationTimestamp="2025-12-03 13:23:01 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 13:23:03.217320305 +0000 UTC m=+7373.805562079" watchObservedRunningTime="2025-12-03 13:23:03.218849119 +0000 UTC m=+7373.807090883" Dec 03 13:23:04 crc kubenswrapper[5052]: I1203 13:23:04.196956 5052 generic.go:334] "Generic (PLEG): container finished" podID="57a8e2c4-46ec-4f92-9f60-2a6af9c546ff" containerID="e6eb2354b7b82fbf58211b1ade674871f298871d3dde2b23f2b3cda4d4628594" exitCode=0 Dec 03 13:23:04 crc kubenswrapper[5052]: I1203 13:23:04.197046 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="keystone-kuttl-tests/keystone-236c-account-create-update-vjwmh" event={"ID":"57a8e2c4-46ec-4f92-9f60-2a6af9c546ff","Type":"ContainerDied","Data":"e6eb2354b7b82fbf58211b1ade674871f298871d3dde2b23f2b3cda4d4628594"} Dec 03 13:23:04 crc kubenswrapper[5052]: I1203 13:23:04.201211 5052 generic.go:334] "Generic (PLEG): container finished" podID="8f5a0f03-b75b-41b7-8f6a-695cf4c58a22" containerID="5c25cf982fe9f2774b23e40fa8b85755d117d93a53b1f6ffe4553bf32d68b53c" exitCode=0 Dec 03 13:23:04 crc kubenswrapper[5052]: I1203 13:23:04.201261 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="keystone-kuttl-tests/keystone-db-create-gnvhm" event={"ID":"8f5a0f03-b75b-41b7-8f6a-695cf4c58a22","Type":"ContainerDied","Data":"5c25cf982fe9f2774b23e40fa8b85755d117d93a53b1f6ffe4553bf32d68b53c"} Dec 03 13:23:04 crc kubenswrapper[5052]: I1203 13:23:04.674016 5052 scope.go:117] "RemoveContainer" containerID="12a8b1f470f6d59d7700e376f2a48eb36363f91f9823a35ce534062c0974c10c" Dec 03 13:23:04 crc kubenswrapper[5052]: E1203 13:23:04.674768 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 13:23:05 crc kubenswrapper[5052]: I1203 13:23:05.554022 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="keystone-kuttl-tests/keystone-236c-account-create-update-vjwmh" Dec 03 13:23:05 crc kubenswrapper[5052]: I1203 13:23:05.561547 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="keystone-kuttl-tests/keystone-db-create-gnvhm" Dec 03 13:23:05 crc kubenswrapper[5052]: I1203 13:23:05.699852 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v6bbs\" (UniqueName: \"kubernetes.io/projected/8f5a0f03-b75b-41b7-8f6a-695cf4c58a22-kube-api-access-v6bbs\") pod \"8f5a0f03-b75b-41b7-8f6a-695cf4c58a22\" (UID: \"8f5a0f03-b75b-41b7-8f6a-695cf4c58a22\") " Dec 03 13:23:05 crc kubenswrapper[5052]: I1203 13:23:05.699945 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/8f5a0f03-b75b-41b7-8f6a-695cf4c58a22-operator-scripts\") pod \"8f5a0f03-b75b-41b7-8f6a-695cf4c58a22\" (UID: \"8f5a0f03-b75b-41b7-8f6a-695cf4c58a22\") " Dec 03 13:23:05 crc kubenswrapper[5052]: I1203 13:23:05.700010 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-whtql\" (UniqueName: \"kubernetes.io/projected/57a8e2c4-46ec-4f92-9f60-2a6af9c546ff-kube-api-access-whtql\") pod \"57a8e2c4-46ec-4f92-9f60-2a6af9c546ff\" (UID: \"57a8e2c4-46ec-4f92-9f60-2a6af9c546ff\") " Dec 03 13:23:05 crc kubenswrapper[5052]: I1203 13:23:05.700305 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/57a8e2c4-46ec-4f92-9f60-2a6af9c546ff-operator-scripts\") pod \"57a8e2c4-46ec-4f92-9f60-2a6af9c546ff\" (UID: \"57a8e2c4-46ec-4f92-9f60-2a6af9c546ff\") " Dec 03 13:23:05 crc kubenswrapper[5052]: I1203 13:23:05.701111 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8f5a0f03-b75b-41b7-8f6a-695cf4c58a22-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "8f5a0f03-b75b-41b7-8f6a-695cf4c58a22" (UID: "8f5a0f03-b75b-41b7-8f6a-695cf4c58a22"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:23:05 crc kubenswrapper[5052]: I1203 13:23:05.701260 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/57a8e2c4-46ec-4f92-9f60-2a6af9c546ff-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "57a8e2c4-46ec-4f92-9f60-2a6af9c546ff" (UID: "57a8e2c4-46ec-4f92-9f60-2a6af9c546ff"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:23:05 crc kubenswrapper[5052]: I1203 13:23:05.705739 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/57a8e2c4-46ec-4f92-9f60-2a6af9c546ff-kube-api-access-whtql" (OuterVolumeSpecName: "kube-api-access-whtql") pod "57a8e2c4-46ec-4f92-9f60-2a6af9c546ff" (UID: "57a8e2c4-46ec-4f92-9f60-2a6af9c546ff"). InnerVolumeSpecName "kube-api-access-whtql". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:23:05 crc kubenswrapper[5052]: I1203 13:23:05.705925 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8f5a0f03-b75b-41b7-8f6a-695cf4c58a22-kube-api-access-v6bbs" (OuterVolumeSpecName: "kube-api-access-v6bbs") pod "8f5a0f03-b75b-41b7-8f6a-695cf4c58a22" (UID: "8f5a0f03-b75b-41b7-8f6a-695cf4c58a22"). InnerVolumeSpecName "kube-api-access-v6bbs". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:23:05 crc kubenswrapper[5052]: I1203 13:23:05.803379 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/57a8e2c4-46ec-4f92-9f60-2a6af9c546ff-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 13:23:05 crc kubenswrapper[5052]: I1203 13:23:05.803835 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-v6bbs\" (UniqueName: \"kubernetes.io/projected/8f5a0f03-b75b-41b7-8f6a-695cf4c58a22-kube-api-access-v6bbs\") on node \"crc\" DevicePath \"\"" Dec 03 13:23:05 crc kubenswrapper[5052]: I1203 13:23:05.803849 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/8f5a0f03-b75b-41b7-8f6a-695cf4c58a22-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 13:23:05 crc kubenswrapper[5052]: I1203 13:23:05.803861 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-whtql\" (UniqueName: \"kubernetes.io/projected/57a8e2c4-46ec-4f92-9f60-2a6af9c546ff-kube-api-access-whtql\") on node \"crc\" DevicePath \"\"" Dec 03 13:23:06 crc kubenswrapper[5052]: I1203 13:23:06.217173 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="keystone-kuttl-tests/keystone-236c-account-create-update-vjwmh" Dec 03 13:23:06 crc kubenswrapper[5052]: I1203 13:23:06.217160 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="keystone-kuttl-tests/keystone-236c-account-create-update-vjwmh" event={"ID":"57a8e2c4-46ec-4f92-9f60-2a6af9c546ff","Type":"ContainerDied","Data":"af06e4c50ba312b6699173c533267e39d2dbbb58c7e63a1c016453b687db2302"} Dec 03 13:23:06 crc kubenswrapper[5052]: I1203 13:23:06.217608 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="af06e4c50ba312b6699173c533267e39d2dbbb58c7e63a1c016453b687db2302" Dec 03 13:23:06 crc kubenswrapper[5052]: I1203 13:23:06.219069 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="keystone-kuttl-tests/keystone-db-create-gnvhm" event={"ID":"8f5a0f03-b75b-41b7-8f6a-695cf4c58a22","Type":"ContainerDied","Data":"f76a223d1949c0a7b0198b8fc5ee6027468125a3f22d7b4daa6881491d4bb91e"} Dec 03 13:23:06 crc kubenswrapper[5052]: I1203 13:23:06.219115 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f76a223d1949c0a7b0198b8fc5ee6027468125a3f22d7b4daa6881491d4bb91e" Dec 03 13:23:06 crc kubenswrapper[5052]: I1203 13:23:06.219180 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="keystone-kuttl-tests/keystone-db-create-gnvhm" Dec 03 13:23:09 crc kubenswrapper[5052]: I1203 13:23:09.995779 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="keystone-kuttl-tests/rabbitmq-server-0" Dec 03 13:23:10 crc kubenswrapper[5052]: I1203 13:23:10.571824 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["keystone-kuttl-tests/keystone-db-sync-64k2t"] Dec 03 13:23:10 crc kubenswrapper[5052]: E1203 13:23:10.572106 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="57a8e2c4-46ec-4f92-9f60-2a6af9c546ff" containerName="mariadb-account-create-update" Dec 03 13:23:10 crc kubenswrapper[5052]: I1203 13:23:10.572118 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="57a8e2c4-46ec-4f92-9f60-2a6af9c546ff" containerName="mariadb-account-create-update" Dec 03 13:23:10 crc kubenswrapper[5052]: E1203 13:23:10.572141 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="8f5a0f03-b75b-41b7-8f6a-695cf4c58a22" containerName="mariadb-database-create" Dec 03 13:23:10 crc kubenswrapper[5052]: I1203 13:23:10.572148 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="8f5a0f03-b75b-41b7-8f6a-695cf4c58a22" containerName="mariadb-database-create" Dec 03 13:23:10 crc kubenswrapper[5052]: I1203 13:23:10.572333 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="57a8e2c4-46ec-4f92-9f60-2a6af9c546ff" containerName="mariadb-account-create-update" Dec 03 13:23:10 crc kubenswrapper[5052]: I1203 13:23:10.572345 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="8f5a0f03-b75b-41b7-8f6a-695cf4c58a22" containerName="mariadb-database-create" Dec 03 13:23:10 crc kubenswrapper[5052]: I1203 13:23:10.572826 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="keystone-kuttl-tests/keystone-db-sync-64k2t" Dec 03 13:23:10 crc kubenswrapper[5052]: I1203 13:23:10.575314 5052 reflector.go:368] Caches populated for *v1.Secret from object-"keystone-kuttl-tests"/"keystone-config-data" Dec 03 13:23:10 crc kubenswrapper[5052]: I1203 13:23:10.576015 5052 reflector.go:368] Caches populated for *v1.Secret from object-"keystone-kuttl-tests"/"keystone-scripts" Dec 03 13:23:10 crc kubenswrapper[5052]: I1203 13:23:10.576157 5052 reflector.go:368] Caches populated for *v1.Secret from object-"keystone-kuttl-tests"/"keystone" Dec 03 13:23:10 crc kubenswrapper[5052]: I1203 13:23:10.577418 5052 reflector.go:368] Caches populated for *v1.Secret from object-"keystone-kuttl-tests"/"keystone-keystone-dockercfg-nxrlv" Dec 03 13:23:10 crc kubenswrapper[5052]: I1203 13:23:10.592290 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["keystone-kuttl-tests/keystone-db-sync-64k2t"] Dec 03 13:23:10 crc kubenswrapper[5052]: I1203 13:23:10.772017 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/bc879d39-a3db-4c0f-b200-cc7b1a27169c-config-data\") pod \"keystone-db-sync-64k2t\" (UID: \"bc879d39-a3db-4c0f-b200-cc7b1a27169c\") " pod="keystone-kuttl-tests/keystone-db-sync-64k2t" Dec 03 13:23:10 crc kubenswrapper[5052]: I1203 13:23:10.772236 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vkwh8\" (UniqueName: \"kubernetes.io/projected/bc879d39-a3db-4c0f-b200-cc7b1a27169c-kube-api-access-vkwh8\") pod \"keystone-db-sync-64k2t\" (UID: \"bc879d39-a3db-4c0f-b200-cc7b1a27169c\") " pod="keystone-kuttl-tests/keystone-db-sync-64k2t" Dec 03 13:23:10 crc kubenswrapper[5052]: I1203 13:23:10.873370 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-vkwh8\" (UniqueName: \"kubernetes.io/projected/bc879d39-a3db-4c0f-b200-cc7b1a27169c-kube-api-access-vkwh8\") pod \"keystone-db-sync-64k2t\" (UID: \"bc879d39-a3db-4c0f-b200-cc7b1a27169c\") " pod="keystone-kuttl-tests/keystone-db-sync-64k2t" Dec 03 13:23:10 crc kubenswrapper[5052]: I1203 13:23:10.873470 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/bc879d39-a3db-4c0f-b200-cc7b1a27169c-config-data\") pod \"keystone-db-sync-64k2t\" (UID: \"bc879d39-a3db-4c0f-b200-cc7b1a27169c\") " pod="keystone-kuttl-tests/keystone-db-sync-64k2t" Dec 03 13:23:10 crc kubenswrapper[5052]: I1203 13:23:10.882210 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/bc879d39-a3db-4c0f-b200-cc7b1a27169c-config-data\") pod \"keystone-db-sync-64k2t\" (UID: \"bc879d39-a3db-4c0f-b200-cc7b1a27169c\") " pod="keystone-kuttl-tests/keystone-db-sync-64k2t" Dec 03 13:23:10 crc kubenswrapper[5052]: I1203 13:23:10.896366 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-vkwh8\" (UniqueName: \"kubernetes.io/projected/bc879d39-a3db-4c0f-b200-cc7b1a27169c-kube-api-access-vkwh8\") pod \"keystone-db-sync-64k2t\" (UID: \"bc879d39-a3db-4c0f-b200-cc7b1a27169c\") " pod="keystone-kuttl-tests/keystone-db-sync-64k2t" Dec 03 13:23:10 crc kubenswrapper[5052]: I1203 13:23:10.930665 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="keystone-kuttl-tests/keystone-db-sync-64k2t" Dec 03 13:23:11 crc kubenswrapper[5052]: I1203 13:23:11.421156 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["keystone-kuttl-tests/keystone-db-sync-64k2t"] Dec 03 13:23:11 crc kubenswrapper[5052]: W1203 13:23:11.425853 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbc879d39_a3db_4c0f_b200_cc7b1a27169c.slice/crio-c8c72ea74fbd7b4828e064c84e638bdbd214d54fde2ecb497b5b351aa662f58f WatchSource:0}: Error finding container c8c72ea74fbd7b4828e064c84e638bdbd214d54fde2ecb497b5b351aa662f58f: Status 404 returned error can't find the container with id c8c72ea74fbd7b4828e064c84e638bdbd214d54fde2ecb497b5b351aa662f58f Dec 03 13:23:12 crc kubenswrapper[5052]: I1203 13:23:12.273734 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="keystone-kuttl-tests/keystone-db-sync-64k2t" event={"ID":"bc879d39-a3db-4c0f-b200-cc7b1a27169c","Type":"ContainerStarted","Data":"c8c72ea74fbd7b4828e064c84e638bdbd214d54fde2ecb497b5b351aa662f58f"} Dec 03 13:23:13 crc kubenswrapper[5052]: I1203 13:23:13.282211 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="keystone-kuttl-tests/keystone-db-sync-64k2t" event={"ID":"bc879d39-a3db-4c0f-b200-cc7b1a27169c","Type":"ContainerStarted","Data":"276fd58ee69d0b6333d60f17fdb07e4b7cc6c2ad653cdd45d3f2534d88c59b8e"} Dec 03 13:23:13 crc kubenswrapper[5052]: I1203 13:23:13.301738 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="keystone-kuttl-tests/keystone-db-sync-64k2t" podStartSLOduration=3.301707879 podStartE2EDuration="3.301707879s" podCreationTimestamp="2025-12-03 13:23:10 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 13:23:13.299762123 +0000 UTC m=+7383.888003897" watchObservedRunningTime="2025-12-03 13:23:13.301707879 +0000 UTC m=+7383.889949643" Dec 03 13:23:15 crc kubenswrapper[5052]: I1203 13:23:15.673144 5052 scope.go:117] "RemoveContainer" containerID="12a8b1f470f6d59d7700e376f2a48eb36363f91f9823a35ce534062c0974c10c" Dec 03 13:23:15 crc kubenswrapper[5052]: E1203 13:23:15.673888 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 13:23:19 crc kubenswrapper[5052]: I1203 13:23:19.328288 5052 generic.go:334] "Generic (PLEG): container finished" podID="bc879d39-a3db-4c0f-b200-cc7b1a27169c" containerID="276fd58ee69d0b6333d60f17fdb07e4b7cc6c2ad653cdd45d3f2534d88c59b8e" exitCode=0 Dec 03 13:23:19 crc kubenswrapper[5052]: I1203 13:23:19.328382 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="keystone-kuttl-tests/keystone-db-sync-64k2t" event={"ID":"bc879d39-a3db-4c0f-b200-cc7b1a27169c","Type":"ContainerDied","Data":"276fd58ee69d0b6333d60f17fdb07e4b7cc6c2ad653cdd45d3f2534d88c59b8e"} Dec 03 13:23:20 crc kubenswrapper[5052]: I1203 13:23:20.591616 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="keystone-kuttl-tests/keystone-db-sync-64k2t" Dec 03 13:23:20 crc kubenswrapper[5052]: I1203 13:23:20.635616 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/bc879d39-a3db-4c0f-b200-cc7b1a27169c-config-data\") pod \"bc879d39-a3db-4c0f-b200-cc7b1a27169c\" (UID: \"bc879d39-a3db-4c0f-b200-cc7b1a27169c\") " Dec 03 13:23:20 crc kubenswrapper[5052]: I1203 13:23:20.636141 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vkwh8\" (UniqueName: \"kubernetes.io/projected/bc879d39-a3db-4c0f-b200-cc7b1a27169c-kube-api-access-vkwh8\") pod \"bc879d39-a3db-4c0f-b200-cc7b1a27169c\" (UID: \"bc879d39-a3db-4c0f-b200-cc7b1a27169c\") " Dec 03 13:23:20 crc kubenswrapper[5052]: I1203 13:23:20.645790 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bc879d39-a3db-4c0f-b200-cc7b1a27169c-kube-api-access-vkwh8" (OuterVolumeSpecName: "kube-api-access-vkwh8") pod "bc879d39-a3db-4c0f-b200-cc7b1a27169c" (UID: "bc879d39-a3db-4c0f-b200-cc7b1a27169c"). InnerVolumeSpecName "kube-api-access-vkwh8". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:23:20 crc kubenswrapper[5052]: I1203 13:23:20.676258 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bc879d39-a3db-4c0f-b200-cc7b1a27169c-config-data" (OuterVolumeSpecName: "config-data") pod "bc879d39-a3db-4c0f-b200-cc7b1a27169c" (UID: "bc879d39-a3db-4c0f-b200-cc7b1a27169c"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:23:20 crc kubenswrapper[5052]: I1203 13:23:20.738054 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-vkwh8\" (UniqueName: \"kubernetes.io/projected/bc879d39-a3db-4c0f-b200-cc7b1a27169c-kube-api-access-vkwh8\") on node \"crc\" DevicePath \"\"" Dec 03 13:23:20 crc kubenswrapper[5052]: I1203 13:23:20.738342 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/bc879d39-a3db-4c0f-b200-cc7b1a27169c-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 13:23:21 crc kubenswrapper[5052]: I1203 13:23:21.345419 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="keystone-kuttl-tests/keystone-db-sync-64k2t" event={"ID":"bc879d39-a3db-4c0f-b200-cc7b1a27169c","Type":"ContainerDied","Data":"c8c72ea74fbd7b4828e064c84e638bdbd214d54fde2ecb497b5b351aa662f58f"} Dec 03 13:23:21 crc kubenswrapper[5052]: I1203 13:23:21.345468 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c8c72ea74fbd7b4828e064c84e638bdbd214d54fde2ecb497b5b351aa662f58f" Dec 03 13:23:21 crc kubenswrapper[5052]: I1203 13:23:21.345536 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="keystone-kuttl-tests/keystone-db-sync-64k2t" Dec 03 13:23:21 crc kubenswrapper[5052]: I1203 13:23:21.582336 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["keystone-kuttl-tests/keystone-bootstrap-698w8"] Dec 03 13:23:21 crc kubenswrapper[5052]: E1203 13:23:21.583120 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="bc879d39-a3db-4c0f-b200-cc7b1a27169c" containerName="keystone-db-sync" Dec 03 13:23:21 crc kubenswrapper[5052]: I1203 13:23:21.583141 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="bc879d39-a3db-4c0f-b200-cc7b1a27169c" containerName="keystone-db-sync" Dec 03 13:23:21 crc kubenswrapper[5052]: I1203 13:23:21.583337 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="bc879d39-a3db-4c0f-b200-cc7b1a27169c" containerName="keystone-db-sync" Dec 03 13:23:21 crc kubenswrapper[5052]: I1203 13:23:21.584000 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="keystone-kuttl-tests/keystone-bootstrap-698w8" Dec 03 13:23:21 crc kubenswrapper[5052]: I1203 13:23:21.589391 5052 reflector.go:368] Caches populated for *v1.Secret from object-"keystone-kuttl-tests"/"keystone" Dec 03 13:23:21 crc kubenswrapper[5052]: I1203 13:23:21.589671 5052 reflector.go:368] Caches populated for *v1.Secret from object-"keystone-kuttl-tests"/"keystone-keystone-dockercfg-nxrlv" Dec 03 13:23:21 crc kubenswrapper[5052]: I1203 13:23:21.589801 5052 reflector.go:368] Caches populated for *v1.Secret from object-"keystone-kuttl-tests"/"keystone-config-data" Dec 03 13:23:21 crc kubenswrapper[5052]: I1203 13:23:21.589919 5052 reflector.go:368] Caches populated for *v1.Secret from object-"keystone-kuttl-tests"/"osp-secret" Dec 03 13:23:21 crc kubenswrapper[5052]: I1203 13:23:21.590012 5052 reflector.go:368] Caches populated for *v1.Secret from object-"keystone-kuttl-tests"/"keystone-scripts" Dec 03 13:23:21 crc kubenswrapper[5052]: I1203 13:23:21.595996 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["keystone-kuttl-tests/keystone-bootstrap-698w8"] Dec 03 13:23:21 crc kubenswrapper[5052]: I1203 13:23:21.750675 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/f0106b94-4b62-4e42-8edb-7ee5f0bfd8be-credential-keys\") pod \"keystone-bootstrap-698w8\" (UID: \"f0106b94-4b62-4e42-8edb-7ee5f0bfd8be\") " pod="keystone-kuttl-tests/keystone-bootstrap-698w8" Dec 03 13:23:21 crc kubenswrapper[5052]: I1203 13:23:21.750744 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/f0106b94-4b62-4e42-8edb-7ee5f0bfd8be-fernet-keys\") pod \"keystone-bootstrap-698w8\" (UID: \"f0106b94-4b62-4e42-8edb-7ee5f0bfd8be\") " pod="keystone-kuttl-tests/keystone-bootstrap-698w8" Dec 03 13:23:21 crc kubenswrapper[5052]: I1203 13:23:21.750784 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pk67h\" (UniqueName: \"kubernetes.io/projected/f0106b94-4b62-4e42-8edb-7ee5f0bfd8be-kube-api-access-pk67h\") pod \"keystone-bootstrap-698w8\" (UID: \"f0106b94-4b62-4e42-8edb-7ee5f0bfd8be\") " pod="keystone-kuttl-tests/keystone-bootstrap-698w8" Dec 03 13:23:21 crc kubenswrapper[5052]: I1203 13:23:21.750828 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/f0106b94-4b62-4e42-8edb-7ee5f0bfd8be-config-data\") pod \"keystone-bootstrap-698w8\" (UID: \"f0106b94-4b62-4e42-8edb-7ee5f0bfd8be\") " pod="keystone-kuttl-tests/keystone-bootstrap-698w8" Dec 03 13:23:21 crc kubenswrapper[5052]: I1203 13:23:21.750908 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/f0106b94-4b62-4e42-8edb-7ee5f0bfd8be-scripts\") pod \"keystone-bootstrap-698w8\" (UID: \"f0106b94-4b62-4e42-8edb-7ee5f0bfd8be\") " pod="keystone-kuttl-tests/keystone-bootstrap-698w8" Dec 03 13:23:21 crc kubenswrapper[5052]: I1203 13:23:21.851893 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/f0106b94-4b62-4e42-8edb-7ee5f0bfd8be-credential-keys\") pod \"keystone-bootstrap-698w8\" (UID: \"f0106b94-4b62-4e42-8edb-7ee5f0bfd8be\") " pod="keystone-kuttl-tests/keystone-bootstrap-698w8" Dec 03 13:23:21 crc kubenswrapper[5052]: I1203 13:23:21.851959 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/f0106b94-4b62-4e42-8edb-7ee5f0bfd8be-fernet-keys\") pod \"keystone-bootstrap-698w8\" (UID: \"f0106b94-4b62-4e42-8edb-7ee5f0bfd8be\") " pod="keystone-kuttl-tests/keystone-bootstrap-698w8" Dec 03 13:23:21 crc kubenswrapper[5052]: I1203 13:23:21.851998 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-pk67h\" (UniqueName: \"kubernetes.io/projected/f0106b94-4b62-4e42-8edb-7ee5f0bfd8be-kube-api-access-pk67h\") pod \"keystone-bootstrap-698w8\" (UID: \"f0106b94-4b62-4e42-8edb-7ee5f0bfd8be\") " pod="keystone-kuttl-tests/keystone-bootstrap-698w8" Dec 03 13:23:21 crc kubenswrapper[5052]: I1203 13:23:21.852041 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/f0106b94-4b62-4e42-8edb-7ee5f0bfd8be-config-data\") pod \"keystone-bootstrap-698w8\" (UID: \"f0106b94-4b62-4e42-8edb-7ee5f0bfd8be\") " pod="keystone-kuttl-tests/keystone-bootstrap-698w8" Dec 03 13:23:21 crc kubenswrapper[5052]: I1203 13:23:21.852098 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/f0106b94-4b62-4e42-8edb-7ee5f0bfd8be-scripts\") pod \"keystone-bootstrap-698w8\" (UID: \"f0106b94-4b62-4e42-8edb-7ee5f0bfd8be\") " pod="keystone-kuttl-tests/keystone-bootstrap-698w8" Dec 03 13:23:21 crc kubenswrapper[5052]: I1203 13:23:21.856129 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/f0106b94-4b62-4e42-8edb-7ee5f0bfd8be-scripts\") pod \"keystone-bootstrap-698w8\" (UID: \"f0106b94-4b62-4e42-8edb-7ee5f0bfd8be\") " pod="keystone-kuttl-tests/keystone-bootstrap-698w8" Dec 03 13:23:21 crc kubenswrapper[5052]: I1203 13:23:21.856325 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/f0106b94-4b62-4e42-8edb-7ee5f0bfd8be-credential-keys\") pod \"keystone-bootstrap-698w8\" (UID: \"f0106b94-4b62-4e42-8edb-7ee5f0bfd8be\") " pod="keystone-kuttl-tests/keystone-bootstrap-698w8" Dec 03 13:23:21 crc kubenswrapper[5052]: I1203 13:23:21.856477 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/f0106b94-4b62-4e42-8edb-7ee5f0bfd8be-fernet-keys\") pod \"keystone-bootstrap-698w8\" (UID: \"f0106b94-4b62-4e42-8edb-7ee5f0bfd8be\") " pod="keystone-kuttl-tests/keystone-bootstrap-698w8" Dec 03 13:23:21 crc kubenswrapper[5052]: I1203 13:23:21.865243 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/f0106b94-4b62-4e42-8edb-7ee5f0bfd8be-config-data\") pod \"keystone-bootstrap-698w8\" (UID: \"f0106b94-4b62-4e42-8edb-7ee5f0bfd8be\") " pod="keystone-kuttl-tests/keystone-bootstrap-698w8" Dec 03 13:23:21 crc kubenswrapper[5052]: I1203 13:23:21.870243 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-pk67h\" (UniqueName: \"kubernetes.io/projected/f0106b94-4b62-4e42-8edb-7ee5f0bfd8be-kube-api-access-pk67h\") pod \"keystone-bootstrap-698w8\" (UID: \"f0106b94-4b62-4e42-8edb-7ee5f0bfd8be\") " pod="keystone-kuttl-tests/keystone-bootstrap-698w8" Dec 03 13:23:21 crc kubenswrapper[5052]: I1203 13:23:21.906258 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="keystone-kuttl-tests/keystone-bootstrap-698w8" Dec 03 13:23:22 crc kubenswrapper[5052]: I1203 13:23:22.311045 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["keystone-kuttl-tests/keystone-bootstrap-698w8"] Dec 03 13:23:22 crc kubenswrapper[5052]: I1203 13:23:22.356323 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="keystone-kuttl-tests/keystone-bootstrap-698w8" event={"ID":"f0106b94-4b62-4e42-8edb-7ee5f0bfd8be","Type":"ContainerStarted","Data":"d65e5c30dd7a7bde224c4c159eb57b072ed84c383131b98a3fb950865f19b039"} Dec 03 13:23:23 crc kubenswrapper[5052]: I1203 13:23:23.365538 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="keystone-kuttl-tests/keystone-bootstrap-698w8" event={"ID":"f0106b94-4b62-4e42-8edb-7ee5f0bfd8be","Type":"ContainerStarted","Data":"777ec4f26345b1716eff3fdf7d0c3ccea9c1431897696a7ce94f5adae5fdd623"} Dec 03 13:23:23 crc kubenswrapper[5052]: I1203 13:23:23.388081 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="keystone-kuttl-tests/keystone-bootstrap-698w8" podStartSLOduration=2.388064219 podStartE2EDuration="2.388064219s" podCreationTimestamp="2025-12-03 13:23:21 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 13:23:23.381944183 +0000 UTC m=+7393.970185957" watchObservedRunningTime="2025-12-03 13:23:23.388064219 +0000 UTC m=+7393.976305983" Dec 03 13:23:26 crc kubenswrapper[5052]: I1203 13:23:26.387410 5052 generic.go:334] "Generic (PLEG): container finished" podID="f0106b94-4b62-4e42-8edb-7ee5f0bfd8be" containerID="777ec4f26345b1716eff3fdf7d0c3ccea9c1431897696a7ce94f5adae5fdd623" exitCode=0 Dec 03 13:23:26 crc kubenswrapper[5052]: I1203 13:23:26.387748 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="keystone-kuttl-tests/keystone-bootstrap-698w8" event={"ID":"f0106b94-4b62-4e42-8edb-7ee5f0bfd8be","Type":"ContainerDied","Data":"777ec4f26345b1716eff3fdf7d0c3ccea9c1431897696a7ce94f5adae5fdd623"} Dec 03 13:23:27 crc kubenswrapper[5052]: I1203 13:23:27.689156 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="keystone-kuttl-tests/keystone-bootstrap-698w8" Dec 03 13:23:27 crc kubenswrapper[5052]: I1203 13:23:27.848179 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/f0106b94-4b62-4e42-8edb-7ee5f0bfd8be-fernet-keys\") pod \"f0106b94-4b62-4e42-8edb-7ee5f0bfd8be\" (UID: \"f0106b94-4b62-4e42-8edb-7ee5f0bfd8be\") " Dec 03 13:23:27 crc kubenswrapper[5052]: I1203 13:23:27.848478 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/f0106b94-4b62-4e42-8edb-7ee5f0bfd8be-credential-keys\") pod \"f0106b94-4b62-4e42-8edb-7ee5f0bfd8be\" (UID: \"f0106b94-4b62-4e42-8edb-7ee5f0bfd8be\") " Dec 03 13:23:27 crc kubenswrapper[5052]: I1203 13:23:27.848550 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pk67h\" (UniqueName: \"kubernetes.io/projected/f0106b94-4b62-4e42-8edb-7ee5f0bfd8be-kube-api-access-pk67h\") pod \"f0106b94-4b62-4e42-8edb-7ee5f0bfd8be\" (UID: \"f0106b94-4b62-4e42-8edb-7ee5f0bfd8be\") " Dec 03 13:23:27 crc kubenswrapper[5052]: I1203 13:23:27.848624 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/f0106b94-4b62-4e42-8edb-7ee5f0bfd8be-config-data\") pod \"f0106b94-4b62-4e42-8edb-7ee5f0bfd8be\" (UID: \"f0106b94-4b62-4e42-8edb-7ee5f0bfd8be\") " Dec 03 13:23:27 crc kubenswrapper[5052]: I1203 13:23:27.848678 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/f0106b94-4b62-4e42-8edb-7ee5f0bfd8be-scripts\") pod \"f0106b94-4b62-4e42-8edb-7ee5f0bfd8be\" (UID: \"f0106b94-4b62-4e42-8edb-7ee5f0bfd8be\") " Dec 03 13:23:27 crc kubenswrapper[5052]: I1203 13:23:27.857534 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f0106b94-4b62-4e42-8edb-7ee5f0bfd8be-fernet-keys" (OuterVolumeSpecName: "fernet-keys") pod "f0106b94-4b62-4e42-8edb-7ee5f0bfd8be" (UID: "f0106b94-4b62-4e42-8edb-7ee5f0bfd8be"). InnerVolumeSpecName "fernet-keys". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:23:27 crc kubenswrapper[5052]: I1203 13:23:27.857862 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f0106b94-4b62-4e42-8edb-7ee5f0bfd8be-credential-keys" (OuterVolumeSpecName: "credential-keys") pod "f0106b94-4b62-4e42-8edb-7ee5f0bfd8be" (UID: "f0106b94-4b62-4e42-8edb-7ee5f0bfd8be"). InnerVolumeSpecName "credential-keys". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:23:27 crc kubenswrapper[5052]: I1203 13:23:27.858546 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f0106b94-4b62-4e42-8edb-7ee5f0bfd8be-kube-api-access-pk67h" (OuterVolumeSpecName: "kube-api-access-pk67h") pod "f0106b94-4b62-4e42-8edb-7ee5f0bfd8be" (UID: "f0106b94-4b62-4e42-8edb-7ee5f0bfd8be"). InnerVolumeSpecName "kube-api-access-pk67h". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:23:27 crc kubenswrapper[5052]: I1203 13:23:27.858621 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f0106b94-4b62-4e42-8edb-7ee5f0bfd8be-scripts" (OuterVolumeSpecName: "scripts") pod "f0106b94-4b62-4e42-8edb-7ee5f0bfd8be" (UID: "f0106b94-4b62-4e42-8edb-7ee5f0bfd8be"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:23:27 crc kubenswrapper[5052]: I1203 13:23:27.881444 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f0106b94-4b62-4e42-8edb-7ee5f0bfd8be-config-data" (OuterVolumeSpecName: "config-data") pod "f0106b94-4b62-4e42-8edb-7ee5f0bfd8be" (UID: "f0106b94-4b62-4e42-8edb-7ee5f0bfd8be"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:23:27 crc kubenswrapper[5052]: I1203 13:23:27.950055 5052 reconciler_common.go:293] "Volume detached for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/f0106b94-4b62-4e42-8edb-7ee5f0bfd8be-credential-keys\") on node \"crc\" DevicePath \"\"" Dec 03 13:23:27 crc kubenswrapper[5052]: I1203 13:23:27.950090 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-pk67h\" (UniqueName: \"kubernetes.io/projected/f0106b94-4b62-4e42-8edb-7ee5f0bfd8be-kube-api-access-pk67h\") on node \"crc\" DevicePath \"\"" Dec 03 13:23:27 crc kubenswrapper[5052]: I1203 13:23:27.950102 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/f0106b94-4b62-4e42-8edb-7ee5f0bfd8be-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 13:23:27 crc kubenswrapper[5052]: I1203 13:23:27.950111 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/f0106b94-4b62-4e42-8edb-7ee5f0bfd8be-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 13:23:27 crc kubenswrapper[5052]: I1203 13:23:27.950135 5052 reconciler_common.go:293] "Volume detached for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/f0106b94-4b62-4e42-8edb-7ee5f0bfd8be-fernet-keys\") on node \"crc\" DevicePath \"\"" Dec 03 13:23:28 crc kubenswrapper[5052]: I1203 13:23:28.404816 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="keystone-kuttl-tests/keystone-bootstrap-698w8" event={"ID":"f0106b94-4b62-4e42-8edb-7ee5f0bfd8be","Type":"ContainerDied","Data":"d65e5c30dd7a7bde224c4c159eb57b072ed84c383131b98a3fb950865f19b039"} Dec 03 13:23:28 crc kubenswrapper[5052]: I1203 13:23:28.404851 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d65e5c30dd7a7bde224c4c159eb57b072ed84c383131b98a3fb950865f19b039" Dec 03 13:23:28 crc kubenswrapper[5052]: I1203 13:23:28.404933 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="keystone-kuttl-tests/keystone-bootstrap-698w8" Dec 03 13:23:28 crc kubenswrapper[5052]: I1203 13:23:28.577693 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["keystone-kuttl-tests/keystone-6b45c857fc-fnlw6"] Dec 03 13:23:28 crc kubenswrapper[5052]: E1203 13:23:28.578080 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="f0106b94-4b62-4e42-8edb-7ee5f0bfd8be" containerName="keystone-bootstrap" Dec 03 13:23:28 crc kubenswrapper[5052]: I1203 13:23:28.578109 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="f0106b94-4b62-4e42-8edb-7ee5f0bfd8be" containerName="keystone-bootstrap" Dec 03 13:23:28 crc kubenswrapper[5052]: I1203 13:23:28.578294 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="f0106b94-4b62-4e42-8edb-7ee5f0bfd8be" containerName="keystone-bootstrap" Dec 03 13:23:28 crc kubenswrapper[5052]: I1203 13:23:28.578931 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="keystone-kuttl-tests/keystone-6b45c857fc-fnlw6" Dec 03 13:23:28 crc kubenswrapper[5052]: I1203 13:23:28.587137 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["keystone-kuttl-tests/keystone-6b45c857fc-fnlw6"] Dec 03 13:23:28 crc kubenswrapper[5052]: I1203 13:23:28.587539 5052 reflector.go:368] Caches populated for *v1.Secret from object-"keystone-kuttl-tests"/"keystone" Dec 03 13:23:28 crc kubenswrapper[5052]: I1203 13:23:28.587643 5052 reflector.go:368] Caches populated for *v1.Secret from object-"keystone-kuttl-tests"/"keystone-keystone-dockercfg-nxrlv" Dec 03 13:23:28 crc kubenswrapper[5052]: I1203 13:23:28.587650 5052 reflector.go:368] Caches populated for *v1.Secret from object-"keystone-kuttl-tests"/"keystone-config-data" Dec 03 13:23:28 crc kubenswrapper[5052]: I1203 13:23:28.587602 5052 reflector.go:368] Caches populated for *v1.Secret from object-"keystone-kuttl-tests"/"keystone-scripts" Dec 03 13:23:28 crc kubenswrapper[5052]: I1203 13:23:28.661705 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/eac56493-7a8a-4685-9bb1-451c679368c7-fernet-keys\") pod \"keystone-6b45c857fc-fnlw6\" (UID: \"eac56493-7a8a-4685-9bb1-451c679368c7\") " pod="keystone-kuttl-tests/keystone-6b45c857fc-fnlw6" Dec 03 13:23:28 crc kubenswrapper[5052]: I1203 13:23:28.661776 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/eac56493-7a8a-4685-9bb1-451c679368c7-credential-keys\") pod \"keystone-6b45c857fc-fnlw6\" (UID: \"eac56493-7a8a-4685-9bb1-451c679368c7\") " pod="keystone-kuttl-tests/keystone-6b45c857fc-fnlw6" Dec 03 13:23:28 crc kubenswrapper[5052]: I1203 13:23:28.661796 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/eac56493-7a8a-4685-9bb1-451c679368c7-config-data\") pod \"keystone-6b45c857fc-fnlw6\" (UID: \"eac56493-7a8a-4685-9bb1-451c679368c7\") " pod="keystone-kuttl-tests/keystone-6b45c857fc-fnlw6" Dec 03 13:23:28 crc kubenswrapper[5052]: I1203 13:23:28.661891 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/eac56493-7a8a-4685-9bb1-451c679368c7-scripts\") pod \"keystone-6b45c857fc-fnlw6\" (UID: \"eac56493-7a8a-4685-9bb1-451c679368c7\") " pod="keystone-kuttl-tests/keystone-6b45c857fc-fnlw6" Dec 03 13:23:28 crc kubenswrapper[5052]: I1203 13:23:28.661916 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pmg2h\" (UniqueName: \"kubernetes.io/projected/eac56493-7a8a-4685-9bb1-451c679368c7-kube-api-access-pmg2h\") pod \"keystone-6b45c857fc-fnlw6\" (UID: \"eac56493-7a8a-4685-9bb1-451c679368c7\") " pod="keystone-kuttl-tests/keystone-6b45c857fc-fnlw6" Dec 03 13:23:28 crc kubenswrapper[5052]: I1203 13:23:28.763175 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/eac56493-7a8a-4685-9bb1-451c679368c7-scripts\") pod \"keystone-6b45c857fc-fnlw6\" (UID: \"eac56493-7a8a-4685-9bb1-451c679368c7\") " pod="keystone-kuttl-tests/keystone-6b45c857fc-fnlw6" Dec 03 13:23:28 crc kubenswrapper[5052]: I1203 13:23:28.763395 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-pmg2h\" (UniqueName: \"kubernetes.io/projected/eac56493-7a8a-4685-9bb1-451c679368c7-kube-api-access-pmg2h\") pod \"keystone-6b45c857fc-fnlw6\" (UID: \"eac56493-7a8a-4685-9bb1-451c679368c7\") " pod="keystone-kuttl-tests/keystone-6b45c857fc-fnlw6" Dec 03 13:23:28 crc kubenswrapper[5052]: I1203 13:23:28.763455 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/eac56493-7a8a-4685-9bb1-451c679368c7-fernet-keys\") pod \"keystone-6b45c857fc-fnlw6\" (UID: \"eac56493-7a8a-4685-9bb1-451c679368c7\") " pod="keystone-kuttl-tests/keystone-6b45c857fc-fnlw6" Dec 03 13:23:28 crc kubenswrapper[5052]: I1203 13:23:28.763486 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/eac56493-7a8a-4685-9bb1-451c679368c7-credential-keys\") pod \"keystone-6b45c857fc-fnlw6\" (UID: \"eac56493-7a8a-4685-9bb1-451c679368c7\") " pod="keystone-kuttl-tests/keystone-6b45c857fc-fnlw6" Dec 03 13:23:28 crc kubenswrapper[5052]: I1203 13:23:28.763507 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/eac56493-7a8a-4685-9bb1-451c679368c7-config-data\") pod \"keystone-6b45c857fc-fnlw6\" (UID: \"eac56493-7a8a-4685-9bb1-451c679368c7\") " pod="keystone-kuttl-tests/keystone-6b45c857fc-fnlw6" Dec 03 13:23:28 crc kubenswrapper[5052]: I1203 13:23:28.772083 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/eac56493-7a8a-4685-9bb1-451c679368c7-scripts\") pod \"keystone-6b45c857fc-fnlw6\" (UID: \"eac56493-7a8a-4685-9bb1-451c679368c7\") " pod="keystone-kuttl-tests/keystone-6b45c857fc-fnlw6" Dec 03 13:23:28 crc kubenswrapper[5052]: I1203 13:23:28.772246 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/eac56493-7a8a-4685-9bb1-451c679368c7-credential-keys\") pod \"keystone-6b45c857fc-fnlw6\" (UID: \"eac56493-7a8a-4685-9bb1-451c679368c7\") " pod="keystone-kuttl-tests/keystone-6b45c857fc-fnlw6" Dec 03 13:23:28 crc kubenswrapper[5052]: I1203 13:23:28.772394 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/eac56493-7a8a-4685-9bb1-451c679368c7-fernet-keys\") pod \"keystone-6b45c857fc-fnlw6\" (UID: \"eac56493-7a8a-4685-9bb1-451c679368c7\") " pod="keystone-kuttl-tests/keystone-6b45c857fc-fnlw6" Dec 03 13:23:28 crc kubenswrapper[5052]: I1203 13:23:28.772693 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/eac56493-7a8a-4685-9bb1-451c679368c7-config-data\") pod \"keystone-6b45c857fc-fnlw6\" (UID: \"eac56493-7a8a-4685-9bb1-451c679368c7\") " pod="keystone-kuttl-tests/keystone-6b45c857fc-fnlw6" Dec 03 13:23:28 crc kubenswrapper[5052]: I1203 13:23:28.781222 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-pmg2h\" (UniqueName: \"kubernetes.io/projected/eac56493-7a8a-4685-9bb1-451c679368c7-kube-api-access-pmg2h\") pod \"keystone-6b45c857fc-fnlw6\" (UID: \"eac56493-7a8a-4685-9bb1-451c679368c7\") " pod="keystone-kuttl-tests/keystone-6b45c857fc-fnlw6" Dec 03 13:23:28 crc kubenswrapper[5052]: I1203 13:23:28.896089 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="keystone-kuttl-tests/keystone-6b45c857fc-fnlw6" Dec 03 13:23:29 crc kubenswrapper[5052]: I1203 13:23:29.324261 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["keystone-kuttl-tests/keystone-6b45c857fc-fnlw6"] Dec 03 13:23:29 crc kubenswrapper[5052]: I1203 13:23:29.419046 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="keystone-kuttl-tests/keystone-6b45c857fc-fnlw6" event={"ID":"eac56493-7a8a-4685-9bb1-451c679368c7","Type":"ContainerStarted","Data":"b861aa598c89a177f431f4fea53742f60cc40a0b633bdbadf4df800a231fd74f"} Dec 03 13:23:29 crc kubenswrapper[5052]: I1203 13:23:29.681109 5052 scope.go:117] "RemoveContainer" containerID="12a8b1f470f6d59d7700e376f2a48eb36363f91f9823a35ce534062c0974c10c" Dec 03 13:23:30 crc kubenswrapper[5052]: I1203 13:23:30.433132 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" event={"ID":"10c237bc-72e7-4da3-9f2c-b1688395bf6e","Type":"ContainerStarted","Data":"6f2eaa045b10c8c10abc2278895801064c7c776bf5ef0e26c0544a891311f82f"} Dec 03 13:23:30 crc kubenswrapper[5052]: I1203 13:23:30.435800 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="keystone-kuttl-tests/keystone-6b45c857fc-fnlw6" event={"ID":"eac56493-7a8a-4685-9bb1-451c679368c7","Type":"ContainerStarted","Data":"c0dff97041961e0c5e747fef8e278e2e4ef0fa6a416b4f064a69f71f61f32f9d"} Dec 03 13:23:30 crc kubenswrapper[5052]: I1203 13:23:30.436358 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="keystone-kuttl-tests/keystone-6b45c857fc-fnlw6" Dec 03 13:23:30 crc kubenswrapper[5052]: I1203 13:23:30.472286 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="keystone-kuttl-tests/keystone-6b45c857fc-fnlw6" podStartSLOduration=2.472268743 podStartE2EDuration="2.472268743s" podCreationTimestamp="2025-12-03 13:23:28 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 13:23:30.467004522 +0000 UTC m=+7401.055246296" watchObservedRunningTime="2025-12-03 13:23:30.472268743 +0000 UTC m=+7401.060510507" Dec 03 13:23:47 crc kubenswrapper[5052]: I1203 13:23:47.543183 5052 scope.go:117] "RemoveContainer" containerID="caa9eaa1c00b2cf8d364eb84b3a9fa7ed49cf8b3f8beb87790fca5594d6ff0f0" Dec 03 13:23:47 crc kubenswrapper[5052]: I1203 13:23:47.568645 5052 scope.go:117] "RemoveContainer" containerID="6d6fee6ddf6dbb6b2db73045b174da8bbbf13b8aadb20644253355996e832c5d" Dec 03 13:23:47 crc kubenswrapper[5052]: I1203 13:23:47.631484 5052 scope.go:117] "RemoveContainer" containerID="3adb3ffb576dce9dbe075283d6512dbd645ee60e83ca7fc9eafde6d7cbb01e38" Dec 03 13:23:47 crc kubenswrapper[5052]: I1203 13:23:47.665207 5052 scope.go:117] "RemoveContainer" containerID="cd7ffff87a56771879dc7fa9b0357427929bcf93b368901d07fa90c727f4e1fc" Dec 03 13:23:47 crc kubenswrapper[5052]: I1203 13:23:47.686362 5052 scope.go:117] "RemoveContainer" containerID="4a7bf8ce71f62b3d07d290f5b183f960f0b2e5bea1d2acf9c5ff15f79e1c42af" Dec 03 13:23:47 crc kubenswrapper[5052]: I1203 13:23:47.712134 5052 scope.go:117] "RemoveContainer" containerID="3ca414320048cba59bb177a28c61f8981b44327362df587c989d552ef06b54cf" Dec 03 13:24:00 crc kubenswrapper[5052]: I1203 13:24:00.466881 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="keystone-kuttl-tests/keystone-6b45c857fc-fnlw6" Dec 03 13:24:01 crc kubenswrapper[5052]: E1203 13:24:01.441219 5052 log.go:32] "Failed when writing line to log file" err="http2: stream closed" path="/var/log/pods/keystone-kuttl-tests_keystone-6b45c857fc-fnlw6_eac56493-7a8a-4685-9bb1-451c679368c7/keystone-api/0.log" line={} Dec 03 13:24:01 crc kubenswrapper[5052]: I1203 13:24:01.798469 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["keystone-kuttl-tests/keystone-5d496f47f5-mf5lg"] Dec 03 13:24:01 crc kubenswrapper[5052]: I1203 13:24:01.807006 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="keystone-kuttl-tests/keystone-5d496f47f5-mf5lg" Dec 03 13:24:01 crc kubenswrapper[5052]: I1203 13:24:01.814873 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["keystone-kuttl-tests/keystone-5d496f47f5-mf5lg"] Dec 03 13:24:01 crc kubenswrapper[5052]: I1203 13:24:01.894820 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/24607acd-2ccb-487b-b383-9601ca4f3d80-scripts\") pod \"keystone-5d496f47f5-mf5lg\" (UID: \"24607acd-2ccb-487b-b383-9601ca4f3d80\") " pod="keystone-kuttl-tests/keystone-5d496f47f5-mf5lg" Dec 03 13:24:01 crc kubenswrapper[5052]: I1203 13:24:01.895045 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/24607acd-2ccb-487b-b383-9601ca4f3d80-credential-keys\") pod \"keystone-5d496f47f5-mf5lg\" (UID: \"24607acd-2ccb-487b-b383-9601ca4f3d80\") " pod="keystone-kuttl-tests/keystone-5d496f47f5-mf5lg" Dec 03 13:24:01 crc kubenswrapper[5052]: I1203 13:24:01.895101 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/24607acd-2ccb-487b-b383-9601ca4f3d80-fernet-keys\") pod \"keystone-5d496f47f5-mf5lg\" (UID: \"24607acd-2ccb-487b-b383-9601ca4f3d80\") " pod="keystone-kuttl-tests/keystone-5d496f47f5-mf5lg" Dec 03 13:24:01 crc kubenswrapper[5052]: I1203 13:24:01.895175 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kzrj6\" (UniqueName: \"kubernetes.io/projected/24607acd-2ccb-487b-b383-9601ca4f3d80-kube-api-access-kzrj6\") pod \"keystone-5d496f47f5-mf5lg\" (UID: \"24607acd-2ccb-487b-b383-9601ca4f3d80\") " pod="keystone-kuttl-tests/keystone-5d496f47f5-mf5lg" Dec 03 13:24:01 crc kubenswrapper[5052]: I1203 13:24:01.895212 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/24607acd-2ccb-487b-b383-9601ca4f3d80-config-data\") pod \"keystone-5d496f47f5-mf5lg\" (UID: \"24607acd-2ccb-487b-b383-9601ca4f3d80\") " pod="keystone-kuttl-tests/keystone-5d496f47f5-mf5lg" Dec 03 13:24:01 crc kubenswrapper[5052]: I1203 13:24:01.996627 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/24607acd-2ccb-487b-b383-9601ca4f3d80-credential-keys\") pod \"keystone-5d496f47f5-mf5lg\" (UID: \"24607acd-2ccb-487b-b383-9601ca4f3d80\") " pod="keystone-kuttl-tests/keystone-5d496f47f5-mf5lg" Dec 03 13:24:01 crc kubenswrapper[5052]: I1203 13:24:01.997008 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/24607acd-2ccb-487b-b383-9601ca4f3d80-fernet-keys\") pod \"keystone-5d496f47f5-mf5lg\" (UID: \"24607acd-2ccb-487b-b383-9601ca4f3d80\") " pod="keystone-kuttl-tests/keystone-5d496f47f5-mf5lg" Dec 03 13:24:01 crc kubenswrapper[5052]: I1203 13:24:01.997065 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-kzrj6\" (UniqueName: \"kubernetes.io/projected/24607acd-2ccb-487b-b383-9601ca4f3d80-kube-api-access-kzrj6\") pod \"keystone-5d496f47f5-mf5lg\" (UID: \"24607acd-2ccb-487b-b383-9601ca4f3d80\") " pod="keystone-kuttl-tests/keystone-5d496f47f5-mf5lg" Dec 03 13:24:01 crc kubenswrapper[5052]: I1203 13:24:01.997090 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/24607acd-2ccb-487b-b383-9601ca4f3d80-config-data\") pod \"keystone-5d496f47f5-mf5lg\" (UID: \"24607acd-2ccb-487b-b383-9601ca4f3d80\") " pod="keystone-kuttl-tests/keystone-5d496f47f5-mf5lg" Dec 03 13:24:01 crc kubenswrapper[5052]: I1203 13:24:01.997126 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/24607acd-2ccb-487b-b383-9601ca4f3d80-scripts\") pod \"keystone-5d496f47f5-mf5lg\" (UID: \"24607acd-2ccb-487b-b383-9601ca4f3d80\") " pod="keystone-kuttl-tests/keystone-5d496f47f5-mf5lg" Dec 03 13:24:02 crc kubenswrapper[5052]: I1203 13:24:02.002548 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/24607acd-2ccb-487b-b383-9601ca4f3d80-scripts\") pod \"keystone-5d496f47f5-mf5lg\" (UID: \"24607acd-2ccb-487b-b383-9601ca4f3d80\") " pod="keystone-kuttl-tests/keystone-5d496f47f5-mf5lg" Dec 03 13:24:02 crc kubenswrapper[5052]: I1203 13:24:02.004003 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/24607acd-2ccb-487b-b383-9601ca4f3d80-credential-keys\") pod \"keystone-5d496f47f5-mf5lg\" (UID: \"24607acd-2ccb-487b-b383-9601ca4f3d80\") " pod="keystone-kuttl-tests/keystone-5d496f47f5-mf5lg" Dec 03 13:24:02 crc kubenswrapper[5052]: I1203 13:24:02.004126 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/24607acd-2ccb-487b-b383-9601ca4f3d80-config-data\") pod \"keystone-5d496f47f5-mf5lg\" (UID: \"24607acd-2ccb-487b-b383-9601ca4f3d80\") " pod="keystone-kuttl-tests/keystone-5d496f47f5-mf5lg" Dec 03 13:24:02 crc kubenswrapper[5052]: I1203 13:24:02.004909 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/24607acd-2ccb-487b-b383-9601ca4f3d80-fernet-keys\") pod \"keystone-5d496f47f5-mf5lg\" (UID: \"24607acd-2ccb-487b-b383-9601ca4f3d80\") " pod="keystone-kuttl-tests/keystone-5d496f47f5-mf5lg" Dec 03 13:24:02 crc kubenswrapper[5052]: I1203 13:24:02.029119 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-kzrj6\" (UniqueName: \"kubernetes.io/projected/24607acd-2ccb-487b-b383-9601ca4f3d80-kube-api-access-kzrj6\") pod \"keystone-5d496f47f5-mf5lg\" (UID: \"24607acd-2ccb-487b-b383-9601ca4f3d80\") " pod="keystone-kuttl-tests/keystone-5d496f47f5-mf5lg" Dec 03 13:24:02 crc kubenswrapper[5052]: I1203 13:24:02.048897 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["keystone-kuttl-tests/keystone-db-sync-64k2t"] Dec 03 13:24:02 crc kubenswrapper[5052]: I1203 13:24:02.061434 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["keystone-kuttl-tests/keystone-db-sync-64k2t"] Dec 03 13:24:02 crc kubenswrapper[5052]: I1203 13:24:02.073792 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["keystone-kuttl-tests/keystone-bootstrap-698w8"] Dec 03 13:24:02 crc kubenswrapper[5052]: I1203 13:24:02.083524 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["keystone-kuttl-tests/keystone-bootstrap-698w8"] Dec 03 13:24:02 crc kubenswrapper[5052]: I1203 13:24:02.090855 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["keystone-kuttl-tests/keystone-6b45c857fc-fnlw6"] Dec 03 13:24:02 crc kubenswrapper[5052]: I1203 13:24:02.091116 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="keystone-kuttl-tests/keystone-6b45c857fc-fnlw6" podUID="eac56493-7a8a-4685-9bb1-451c679368c7" containerName="keystone-api" containerID="cri-o://c0dff97041961e0c5e747fef8e278e2e4ef0fa6a416b4f064a69f71f61f32f9d" gracePeriod=30 Dec 03 13:24:02 crc kubenswrapper[5052]: I1203 13:24:02.104968 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["keystone-kuttl-tests/keystone-5d496f47f5-mf5lg"] Dec 03 13:24:02 crc kubenswrapper[5052]: I1203 13:24:02.105813 5052 kubelet_pods.go:1007] "Unable to retrieve pull secret, the image pull may not succeed." pod="keystone-kuttl-tests/keystone-5d496f47f5-mf5lg" secret="" err="secret \"keystone-keystone-dockercfg-nxrlv\" not found" Dec 03 13:24:02 crc kubenswrapper[5052]: I1203 13:24:02.105886 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="keystone-kuttl-tests/keystone-5d496f47f5-mf5lg" Dec 03 13:24:02 crc kubenswrapper[5052]: I1203 13:24:02.129378 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["keystone-kuttl-tests/keystone236c-account-delete-qwbfp"] Dec 03 13:24:02 crc kubenswrapper[5052]: I1203 13:24:02.130641 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="keystone-kuttl-tests/keystone236c-account-delete-qwbfp" Dec 03 13:24:02 crc kubenswrapper[5052]: I1203 13:24:02.137122 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["keystone-kuttl-tests/keystone236c-account-delete-qwbfp"] Dec 03 13:24:02 crc kubenswrapper[5052]: I1203 13:24:02.204842 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-24668\" (UniqueName: \"kubernetes.io/projected/43950713-65ca-4855-89b5-de8d245df9c8-kube-api-access-24668\") pod \"keystone236c-account-delete-qwbfp\" (UID: \"43950713-65ca-4855-89b5-de8d245df9c8\") " pod="keystone-kuttl-tests/keystone236c-account-delete-qwbfp" Dec 03 13:24:02 crc kubenswrapper[5052]: I1203 13:24:02.204947 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/43950713-65ca-4855-89b5-de8d245df9c8-operator-scripts\") pod \"keystone236c-account-delete-qwbfp\" (UID: \"43950713-65ca-4855-89b5-de8d245df9c8\") " pod="keystone-kuttl-tests/keystone236c-account-delete-qwbfp" Dec 03 13:24:02 crc kubenswrapper[5052]: I1203 13:24:02.306807 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/43950713-65ca-4855-89b5-de8d245df9c8-operator-scripts\") pod \"keystone236c-account-delete-qwbfp\" (UID: \"43950713-65ca-4855-89b5-de8d245df9c8\") " pod="keystone-kuttl-tests/keystone236c-account-delete-qwbfp" Dec 03 13:24:02 crc kubenswrapper[5052]: I1203 13:24:02.306905 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-24668\" (UniqueName: \"kubernetes.io/projected/43950713-65ca-4855-89b5-de8d245df9c8-kube-api-access-24668\") pod \"keystone236c-account-delete-qwbfp\" (UID: \"43950713-65ca-4855-89b5-de8d245df9c8\") " pod="keystone-kuttl-tests/keystone236c-account-delete-qwbfp" Dec 03 13:24:02 crc kubenswrapper[5052]: I1203 13:24:02.307776 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/43950713-65ca-4855-89b5-de8d245df9c8-operator-scripts\") pod \"keystone236c-account-delete-qwbfp\" (UID: \"43950713-65ca-4855-89b5-de8d245df9c8\") " pod="keystone-kuttl-tests/keystone236c-account-delete-qwbfp" Dec 03 13:24:02 crc kubenswrapper[5052]: I1203 13:24:02.329993 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-24668\" (UniqueName: \"kubernetes.io/projected/43950713-65ca-4855-89b5-de8d245df9c8-kube-api-access-24668\") pod \"keystone236c-account-delete-qwbfp\" (UID: \"43950713-65ca-4855-89b5-de8d245df9c8\") " pod="keystone-kuttl-tests/keystone236c-account-delete-qwbfp" Dec 03 13:24:02 crc kubenswrapper[5052]: I1203 13:24:02.492903 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="keystone-kuttl-tests/keystone236c-account-delete-qwbfp" Dec 03 13:24:02 crc kubenswrapper[5052]: I1203 13:24:02.546014 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["keystone-kuttl-tests/keystone-5d496f47f5-mf5lg"] Dec 03 13:24:02 crc kubenswrapper[5052]: I1203 13:24:02.714936 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="keystone-kuttl-tests/keystone-5d496f47f5-mf5lg" event={"ID":"24607acd-2ccb-487b-b383-9601ca4f3d80","Type":"ContainerStarted","Data":"b3ac2efa4a801f1dc3b9185ffa67e31282bf1dafde1300ab915d2e250b9b3555"} Dec 03 13:24:02 crc kubenswrapper[5052]: I1203 13:24:02.889720 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["keystone-kuttl-tests/keystone236c-account-delete-qwbfp"] Dec 03 13:24:02 crc kubenswrapper[5052]: W1203 13:24:02.896674 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod43950713_65ca_4855_89b5_de8d245df9c8.slice/crio-ba111ae36cc35a9443f7ac0884ac62d0d8ebae3d927976edea4c49e0b1f2f827 WatchSource:0}: Error finding container ba111ae36cc35a9443f7ac0884ac62d0d8ebae3d927976edea4c49e0b1f2f827: Status 404 returned error can't find the container with id ba111ae36cc35a9443f7ac0884ac62d0d8ebae3d927976edea4c49e0b1f2f827 Dec 03 13:24:03 crc kubenswrapper[5052]: I1203 13:24:03.682809 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bc879d39-a3db-4c0f-b200-cc7b1a27169c" path="/var/lib/kubelet/pods/bc879d39-a3db-4c0f-b200-cc7b1a27169c/volumes" Dec 03 13:24:03 crc kubenswrapper[5052]: I1203 13:24:03.683794 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f0106b94-4b62-4e42-8edb-7ee5f0bfd8be" path="/var/lib/kubelet/pods/f0106b94-4b62-4e42-8edb-7ee5f0bfd8be/volumes" Dec 03 13:24:03 crc kubenswrapper[5052]: I1203 13:24:03.730966 5052 generic.go:334] "Generic (PLEG): container finished" podID="43950713-65ca-4855-89b5-de8d245df9c8" containerID="b082d4fab31821909ca8acfed207b5720e05d5fdeb6d37090657dfe881f3b038" exitCode=0 Dec 03 13:24:03 crc kubenswrapper[5052]: I1203 13:24:03.731024 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="keystone-kuttl-tests/keystone236c-account-delete-qwbfp" event={"ID":"43950713-65ca-4855-89b5-de8d245df9c8","Type":"ContainerDied","Data":"b082d4fab31821909ca8acfed207b5720e05d5fdeb6d37090657dfe881f3b038"} Dec 03 13:24:03 crc kubenswrapper[5052]: I1203 13:24:03.731049 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="keystone-kuttl-tests/keystone236c-account-delete-qwbfp" event={"ID":"43950713-65ca-4855-89b5-de8d245df9c8","Type":"ContainerStarted","Data":"ba111ae36cc35a9443f7ac0884ac62d0d8ebae3d927976edea4c49e0b1f2f827"} Dec 03 13:24:03 crc kubenswrapper[5052]: I1203 13:24:03.733756 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="keystone-kuttl-tests/keystone-5d496f47f5-mf5lg" event={"ID":"24607acd-2ccb-487b-b383-9601ca4f3d80","Type":"ContainerStarted","Data":"8ac8307ef501f8d1944ef7596381e031e5a0ddfa843d945e8058aacd95d71b07"} Dec 03 13:24:03 crc kubenswrapper[5052]: I1203 13:24:03.733900 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="keystone-kuttl-tests/keystone-5d496f47f5-mf5lg" Dec 03 13:24:03 crc kubenswrapper[5052]: I1203 13:24:03.733893 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="keystone-kuttl-tests/keystone-5d496f47f5-mf5lg" podUID="24607acd-2ccb-487b-b383-9601ca4f3d80" containerName="keystone-api" containerID="cri-o://8ac8307ef501f8d1944ef7596381e031e5a0ddfa843d945e8058aacd95d71b07" gracePeriod=30 Dec 03 13:24:03 crc kubenswrapper[5052]: I1203 13:24:03.765431 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="keystone-kuttl-tests/keystone-5d496f47f5-mf5lg" podStartSLOduration=2.765413301 podStartE2EDuration="2.765413301s" podCreationTimestamp="2025-12-03 13:24:01 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 13:24:03.764445753 +0000 UTC m=+7434.352687517" watchObservedRunningTime="2025-12-03 13:24:03.765413301 +0000 UTC m=+7434.353655065" Dec 03 13:24:04 crc kubenswrapper[5052]: I1203 13:24:04.172081 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="keystone-kuttl-tests/keystone-5d496f47f5-mf5lg" Dec 03 13:24:04 crc kubenswrapper[5052]: I1203 13:24:04.342937 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/24607acd-2ccb-487b-b383-9601ca4f3d80-scripts\") pod \"24607acd-2ccb-487b-b383-9601ca4f3d80\" (UID: \"24607acd-2ccb-487b-b383-9601ca4f3d80\") " Dec 03 13:24:04 crc kubenswrapper[5052]: I1203 13:24:04.342980 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kzrj6\" (UniqueName: \"kubernetes.io/projected/24607acd-2ccb-487b-b383-9601ca4f3d80-kube-api-access-kzrj6\") pod \"24607acd-2ccb-487b-b383-9601ca4f3d80\" (UID: \"24607acd-2ccb-487b-b383-9601ca4f3d80\") " Dec 03 13:24:04 crc kubenswrapper[5052]: I1203 13:24:04.343036 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/24607acd-2ccb-487b-b383-9601ca4f3d80-fernet-keys\") pod \"24607acd-2ccb-487b-b383-9601ca4f3d80\" (UID: \"24607acd-2ccb-487b-b383-9601ca4f3d80\") " Dec 03 13:24:04 crc kubenswrapper[5052]: I1203 13:24:04.343095 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/24607acd-2ccb-487b-b383-9601ca4f3d80-config-data\") pod \"24607acd-2ccb-487b-b383-9601ca4f3d80\" (UID: \"24607acd-2ccb-487b-b383-9601ca4f3d80\") " Dec 03 13:24:04 crc kubenswrapper[5052]: I1203 13:24:04.343114 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/24607acd-2ccb-487b-b383-9601ca4f3d80-credential-keys\") pod \"24607acd-2ccb-487b-b383-9601ca4f3d80\" (UID: \"24607acd-2ccb-487b-b383-9601ca4f3d80\") " Dec 03 13:24:04 crc kubenswrapper[5052]: I1203 13:24:04.349374 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/24607acd-2ccb-487b-b383-9601ca4f3d80-credential-keys" (OuterVolumeSpecName: "credential-keys") pod "24607acd-2ccb-487b-b383-9601ca4f3d80" (UID: "24607acd-2ccb-487b-b383-9601ca4f3d80"). InnerVolumeSpecName "credential-keys". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:24:04 crc kubenswrapper[5052]: I1203 13:24:04.349875 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/24607acd-2ccb-487b-b383-9601ca4f3d80-kube-api-access-kzrj6" (OuterVolumeSpecName: "kube-api-access-kzrj6") pod "24607acd-2ccb-487b-b383-9601ca4f3d80" (UID: "24607acd-2ccb-487b-b383-9601ca4f3d80"). InnerVolumeSpecName "kube-api-access-kzrj6". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:24:04 crc kubenswrapper[5052]: I1203 13:24:04.350172 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/24607acd-2ccb-487b-b383-9601ca4f3d80-fernet-keys" (OuterVolumeSpecName: "fernet-keys") pod "24607acd-2ccb-487b-b383-9601ca4f3d80" (UID: "24607acd-2ccb-487b-b383-9601ca4f3d80"). InnerVolumeSpecName "fernet-keys". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:24:04 crc kubenswrapper[5052]: I1203 13:24:04.350199 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/24607acd-2ccb-487b-b383-9601ca4f3d80-scripts" (OuterVolumeSpecName: "scripts") pod "24607acd-2ccb-487b-b383-9601ca4f3d80" (UID: "24607acd-2ccb-487b-b383-9601ca4f3d80"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:24:04 crc kubenswrapper[5052]: I1203 13:24:04.367916 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/24607acd-2ccb-487b-b383-9601ca4f3d80-config-data" (OuterVolumeSpecName: "config-data") pod "24607acd-2ccb-487b-b383-9601ca4f3d80" (UID: "24607acd-2ccb-487b-b383-9601ca4f3d80"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:24:04 crc kubenswrapper[5052]: I1203 13:24:04.444710 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/24607acd-2ccb-487b-b383-9601ca4f3d80-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 13:24:04 crc kubenswrapper[5052]: I1203 13:24:04.444767 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-kzrj6\" (UniqueName: \"kubernetes.io/projected/24607acd-2ccb-487b-b383-9601ca4f3d80-kube-api-access-kzrj6\") on node \"crc\" DevicePath \"\"" Dec 03 13:24:04 crc kubenswrapper[5052]: I1203 13:24:04.444781 5052 reconciler_common.go:293] "Volume detached for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/24607acd-2ccb-487b-b383-9601ca4f3d80-fernet-keys\") on node \"crc\" DevicePath \"\"" Dec 03 13:24:04 crc kubenswrapper[5052]: I1203 13:24:04.444793 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/24607acd-2ccb-487b-b383-9601ca4f3d80-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 13:24:04 crc kubenswrapper[5052]: I1203 13:24:04.444808 5052 reconciler_common.go:293] "Volume detached for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/24607acd-2ccb-487b-b383-9601ca4f3d80-credential-keys\") on node \"crc\" DevicePath \"\"" Dec 03 13:24:04 crc kubenswrapper[5052]: I1203 13:24:04.742963 5052 generic.go:334] "Generic (PLEG): container finished" podID="24607acd-2ccb-487b-b383-9601ca4f3d80" containerID="8ac8307ef501f8d1944ef7596381e031e5a0ddfa843d945e8058aacd95d71b07" exitCode=0 Dec 03 13:24:04 crc kubenswrapper[5052]: I1203 13:24:04.743008 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="keystone-kuttl-tests/keystone-5d496f47f5-mf5lg" Dec 03 13:24:04 crc kubenswrapper[5052]: I1203 13:24:04.743024 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="keystone-kuttl-tests/keystone-5d496f47f5-mf5lg" event={"ID":"24607acd-2ccb-487b-b383-9601ca4f3d80","Type":"ContainerDied","Data":"8ac8307ef501f8d1944ef7596381e031e5a0ddfa843d945e8058aacd95d71b07"} Dec 03 13:24:04 crc kubenswrapper[5052]: I1203 13:24:04.743435 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="keystone-kuttl-tests/keystone-5d496f47f5-mf5lg" event={"ID":"24607acd-2ccb-487b-b383-9601ca4f3d80","Type":"ContainerDied","Data":"b3ac2efa4a801f1dc3b9185ffa67e31282bf1dafde1300ab915d2e250b9b3555"} Dec 03 13:24:04 crc kubenswrapper[5052]: I1203 13:24:04.743466 5052 scope.go:117] "RemoveContainer" containerID="8ac8307ef501f8d1944ef7596381e031e5a0ddfa843d945e8058aacd95d71b07" Dec 03 13:24:04 crc kubenswrapper[5052]: I1203 13:24:04.770179 5052 scope.go:117] "RemoveContainer" containerID="8ac8307ef501f8d1944ef7596381e031e5a0ddfa843d945e8058aacd95d71b07" Dec 03 13:24:04 crc kubenswrapper[5052]: E1203 13:24:04.770662 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8ac8307ef501f8d1944ef7596381e031e5a0ddfa843d945e8058aacd95d71b07\": container with ID starting with 8ac8307ef501f8d1944ef7596381e031e5a0ddfa843d945e8058aacd95d71b07 not found: ID does not exist" containerID="8ac8307ef501f8d1944ef7596381e031e5a0ddfa843d945e8058aacd95d71b07" Dec 03 13:24:04 crc kubenswrapper[5052]: I1203 13:24:04.770715 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8ac8307ef501f8d1944ef7596381e031e5a0ddfa843d945e8058aacd95d71b07"} err="failed to get container status \"8ac8307ef501f8d1944ef7596381e031e5a0ddfa843d945e8058aacd95d71b07\": rpc error: code = NotFound desc = could not find container \"8ac8307ef501f8d1944ef7596381e031e5a0ddfa843d945e8058aacd95d71b07\": container with ID starting with 8ac8307ef501f8d1944ef7596381e031e5a0ddfa843d945e8058aacd95d71b07 not found: ID does not exist" Dec 03 13:24:04 crc kubenswrapper[5052]: I1203 13:24:04.772362 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["keystone-kuttl-tests/keystone-5d496f47f5-mf5lg"] Dec 03 13:24:04 crc kubenswrapper[5052]: I1203 13:24:04.780458 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["keystone-kuttl-tests/keystone-5d496f47f5-mf5lg"] Dec 03 13:24:04 crc kubenswrapper[5052]: I1203 13:24:04.994859 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="keystone-kuttl-tests/keystone236c-account-delete-qwbfp" Dec 03 13:24:05 crc kubenswrapper[5052]: I1203 13:24:05.157478 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-24668\" (UniqueName: \"kubernetes.io/projected/43950713-65ca-4855-89b5-de8d245df9c8-kube-api-access-24668\") pod \"43950713-65ca-4855-89b5-de8d245df9c8\" (UID: \"43950713-65ca-4855-89b5-de8d245df9c8\") " Dec 03 13:24:05 crc kubenswrapper[5052]: I1203 13:24:05.157721 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/43950713-65ca-4855-89b5-de8d245df9c8-operator-scripts\") pod \"43950713-65ca-4855-89b5-de8d245df9c8\" (UID: \"43950713-65ca-4855-89b5-de8d245df9c8\") " Dec 03 13:24:05 crc kubenswrapper[5052]: I1203 13:24:05.159094 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/43950713-65ca-4855-89b5-de8d245df9c8-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "43950713-65ca-4855-89b5-de8d245df9c8" (UID: "43950713-65ca-4855-89b5-de8d245df9c8"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:24:05 crc kubenswrapper[5052]: I1203 13:24:05.168863 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/43950713-65ca-4855-89b5-de8d245df9c8-kube-api-access-24668" (OuterVolumeSpecName: "kube-api-access-24668") pod "43950713-65ca-4855-89b5-de8d245df9c8" (UID: "43950713-65ca-4855-89b5-de8d245df9c8"). InnerVolumeSpecName "kube-api-access-24668". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:24:05 crc kubenswrapper[5052]: I1203 13:24:05.260549 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/43950713-65ca-4855-89b5-de8d245df9c8-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 13:24:05 crc kubenswrapper[5052]: I1203 13:24:05.260619 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-24668\" (UniqueName: \"kubernetes.io/projected/43950713-65ca-4855-89b5-de8d245df9c8-kube-api-access-24668\") on node \"crc\" DevicePath \"\"" Dec 03 13:24:05 crc kubenswrapper[5052]: I1203 13:24:05.634268 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="keystone-kuttl-tests/keystone-6b45c857fc-fnlw6" Dec 03 13:24:05 crc kubenswrapper[5052]: I1203 13:24:05.683507 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="24607acd-2ccb-487b-b383-9601ca4f3d80" path="/var/lib/kubelet/pods/24607acd-2ccb-487b-b383-9601ca4f3d80/volumes" Dec 03 13:24:05 crc kubenswrapper[5052]: I1203 13:24:05.755017 5052 generic.go:334] "Generic (PLEG): container finished" podID="eac56493-7a8a-4685-9bb1-451c679368c7" containerID="c0dff97041961e0c5e747fef8e278e2e4ef0fa6a416b4f064a69f71f61f32f9d" exitCode=0 Dec 03 13:24:05 crc kubenswrapper[5052]: I1203 13:24:05.755121 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="keystone-kuttl-tests/keystone-6b45c857fc-fnlw6" Dec 03 13:24:05 crc kubenswrapper[5052]: I1203 13:24:05.755170 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="keystone-kuttl-tests/keystone-6b45c857fc-fnlw6" event={"ID":"eac56493-7a8a-4685-9bb1-451c679368c7","Type":"ContainerDied","Data":"c0dff97041961e0c5e747fef8e278e2e4ef0fa6a416b4f064a69f71f61f32f9d"} Dec 03 13:24:05 crc kubenswrapper[5052]: I1203 13:24:05.755240 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="keystone-kuttl-tests/keystone-6b45c857fc-fnlw6" event={"ID":"eac56493-7a8a-4685-9bb1-451c679368c7","Type":"ContainerDied","Data":"b861aa598c89a177f431f4fea53742f60cc40a0b633bdbadf4df800a231fd74f"} Dec 03 13:24:05 crc kubenswrapper[5052]: I1203 13:24:05.755269 5052 scope.go:117] "RemoveContainer" containerID="c0dff97041961e0c5e747fef8e278e2e4ef0fa6a416b4f064a69f71f61f32f9d" Dec 03 13:24:05 crc kubenswrapper[5052]: I1203 13:24:05.757556 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="keystone-kuttl-tests/keystone236c-account-delete-qwbfp" Dec 03 13:24:05 crc kubenswrapper[5052]: I1203 13:24:05.757541 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="keystone-kuttl-tests/keystone236c-account-delete-qwbfp" event={"ID":"43950713-65ca-4855-89b5-de8d245df9c8","Type":"ContainerDied","Data":"ba111ae36cc35a9443f7ac0884ac62d0d8ebae3d927976edea4c49e0b1f2f827"} Dec 03 13:24:05 crc kubenswrapper[5052]: I1203 13:24:05.757604 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ba111ae36cc35a9443f7ac0884ac62d0d8ebae3d927976edea4c49e0b1f2f827" Dec 03 13:24:05 crc kubenswrapper[5052]: I1203 13:24:05.768909 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/eac56493-7a8a-4685-9bb1-451c679368c7-credential-keys\") pod \"eac56493-7a8a-4685-9bb1-451c679368c7\" (UID: \"eac56493-7a8a-4685-9bb1-451c679368c7\") " Dec 03 13:24:05 crc kubenswrapper[5052]: I1203 13:24:05.768963 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pmg2h\" (UniqueName: \"kubernetes.io/projected/eac56493-7a8a-4685-9bb1-451c679368c7-kube-api-access-pmg2h\") pod \"eac56493-7a8a-4685-9bb1-451c679368c7\" (UID: \"eac56493-7a8a-4685-9bb1-451c679368c7\") " Dec 03 13:24:05 crc kubenswrapper[5052]: I1203 13:24:05.769006 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/eac56493-7a8a-4685-9bb1-451c679368c7-scripts\") pod \"eac56493-7a8a-4685-9bb1-451c679368c7\" (UID: \"eac56493-7a8a-4685-9bb1-451c679368c7\") " Dec 03 13:24:05 crc kubenswrapper[5052]: I1203 13:24:05.769034 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/eac56493-7a8a-4685-9bb1-451c679368c7-fernet-keys\") pod \"eac56493-7a8a-4685-9bb1-451c679368c7\" (UID: \"eac56493-7a8a-4685-9bb1-451c679368c7\") " Dec 03 13:24:05 crc kubenswrapper[5052]: I1203 13:24:05.769100 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/eac56493-7a8a-4685-9bb1-451c679368c7-config-data\") pod \"eac56493-7a8a-4685-9bb1-451c679368c7\" (UID: \"eac56493-7a8a-4685-9bb1-451c679368c7\") " Dec 03 13:24:05 crc kubenswrapper[5052]: I1203 13:24:05.772694 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eac56493-7a8a-4685-9bb1-451c679368c7-credential-keys" (OuterVolumeSpecName: "credential-keys") pod "eac56493-7a8a-4685-9bb1-451c679368c7" (UID: "eac56493-7a8a-4685-9bb1-451c679368c7"). InnerVolumeSpecName "credential-keys". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:24:05 crc kubenswrapper[5052]: I1203 13:24:05.773197 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eac56493-7a8a-4685-9bb1-451c679368c7-fernet-keys" (OuterVolumeSpecName: "fernet-keys") pod "eac56493-7a8a-4685-9bb1-451c679368c7" (UID: "eac56493-7a8a-4685-9bb1-451c679368c7"). InnerVolumeSpecName "fernet-keys". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:24:05 crc kubenswrapper[5052]: I1203 13:24:05.773479 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eac56493-7a8a-4685-9bb1-451c679368c7-kube-api-access-pmg2h" (OuterVolumeSpecName: "kube-api-access-pmg2h") pod "eac56493-7a8a-4685-9bb1-451c679368c7" (UID: "eac56493-7a8a-4685-9bb1-451c679368c7"). InnerVolumeSpecName "kube-api-access-pmg2h". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:24:05 crc kubenswrapper[5052]: I1203 13:24:05.773644 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eac56493-7a8a-4685-9bb1-451c679368c7-scripts" (OuterVolumeSpecName: "scripts") pod "eac56493-7a8a-4685-9bb1-451c679368c7" (UID: "eac56493-7a8a-4685-9bb1-451c679368c7"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:24:05 crc kubenswrapper[5052]: I1203 13:24:05.778262 5052 scope.go:117] "RemoveContainer" containerID="c0dff97041961e0c5e747fef8e278e2e4ef0fa6a416b4f064a69f71f61f32f9d" Dec 03 13:24:05 crc kubenswrapper[5052]: E1203 13:24:05.778884 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c0dff97041961e0c5e747fef8e278e2e4ef0fa6a416b4f064a69f71f61f32f9d\": container with ID starting with c0dff97041961e0c5e747fef8e278e2e4ef0fa6a416b4f064a69f71f61f32f9d not found: ID does not exist" containerID="c0dff97041961e0c5e747fef8e278e2e4ef0fa6a416b4f064a69f71f61f32f9d" Dec 03 13:24:05 crc kubenswrapper[5052]: I1203 13:24:05.778923 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c0dff97041961e0c5e747fef8e278e2e4ef0fa6a416b4f064a69f71f61f32f9d"} err="failed to get container status \"c0dff97041961e0c5e747fef8e278e2e4ef0fa6a416b4f064a69f71f61f32f9d\": rpc error: code = NotFound desc = could not find container \"c0dff97041961e0c5e747fef8e278e2e4ef0fa6a416b4f064a69f71f61f32f9d\": container with ID starting with c0dff97041961e0c5e747fef8e278e2e4ef0fa6a416b4f064a69f71f61f32f9d not found: ID does not exist" Dec 03 13:24:05 crc kubenswrapper[5052]: I1203 13:24:05.789245 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eac56493-7a8a-4685-9bb1-451c679368c7-config-data" (OuterVolumeSpecName: "config-data") pod "eac56493-7a8a-4685-9bb1-451c679368c7" (UID: "eac56493-7a8a-4685-9bb1-451c679368c7"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:24:05 crc kubenswrapper[5052]: I1203 13:24:05.870967 5052 reconciler_common.go:293] "Volume detached for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/eac56493-7a8a-4685-9bb1-451c679368c7-credential-keys\") on node \"crc\" DevicePath \"\"" Dec 03 13:24:05 crc kubenswrapper[5052]: I1203 13:24:05.871003 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-pmg2h\" (UniqueName: \"kubernetes.io/projected/eac56493-7a8a-4685-9bb1-451c679368c7-kube-api-access-pmg2h\") on node \"crc\" DevicePath \"\"" Dec 03 13:24:05 crc kubenswrapper[5052]: I1203 13:24:05.871014 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/eac56493-7a8a-4685-9bb1-451c679368c7-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 13:24:05 crc kubenswrapper[5052]: I1203 13:24:05.871022 5052 reconciler_common.go:293] "Volume detached for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/eac56493-7a8a-4685-9bb1-451c679368c7-fernet-keys\") on node \"crc\" DevicePath \"\"" Dec 03 13:24:05 crc kubenswrapper[5052]: I1203 13:24:05.871031 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/eac56493-7a8a-4685-9bb1-451c679368c7-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 13:24:06 crc kubenswrapper[5052]: I1203 13:24:06.086322 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["keystone-kuttl-tests/keystone-6b45c857fc-fnlw6"] Dec 03 13:24:06 crc kubenswrapper[5052]: I1203 13:24:06.091719 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["keystone-kuttl-tests/keystone-6b45c857fc-fnlw6"] Dec 03 13:24:07 crc kubenswrapper[5052]: I1203 13:24:07.104578 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["keystone-kuttl-tests/keystone-db-create-gnvhm"] Dec 03 13:24:07 crc kubenswrapper[5052]: I1203 13:24:07.111113 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["keystone-kuttl-tests/keystone-db-create-gnvhm"] Dec 03 13:24:07 crc kubenswrapper[5052]: I1203 13:24:07.121017 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["keystone-kuttl-tests/keystone236c-account-delete-qwbfp"] Dec 03 13:24:07 crc kubenswrapper[5052]: I1203 13:24:07.127679 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["keystone-kuttl-tests/keystone-236c-account-create-update-vjwmh"] Dec 03 13:24:07 crc kubenswrapper[5052]: I1203 13:24:07.133314 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["keystone-kuttl-tests/keystone-236c-account-create-update-vjwmh"] Dec 03 13:24:07 crc kubenswrapper[5052]: I1203 13:24:07.138785 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["keystone-kuttl-tests/keystone236c-account-delete-qwbfp"] Dec 03 13:24:07 crc kubenswrapper[5052]: I1203 13:24:07.192795 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["keystone-kuttl-tests/keystone-db-create-zkhpv"] Dec 03 13:24:07 crc kubenswrapper[5052]: E1203 13:24:07.193342 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="43950713-65ca-4855-89b5-de8d245df9c8" containerName="mariadb-account-delete" Dec 03 13:24:07 crc kubenswrapper[5052]: I1203 13:24:07.193416 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="43950713-65ca-4855-89b5-de8d245df9c8" containerName="mariadb-account-delete" Dec 03 13:24:07 crc kubenswrapper[5052]: E1203 13:24:07.193504 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="eac56493-7a8a-4685-9bb1-451c679368c7" containerName="keystone-api" Dec 03 13:24:07 crc kubenswrapper[5052]: I1203 13:24:07.193572 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="eac56493-7a8a-4685-9bb1-451c679368c7" containerName="keystone-api" Dec 03 13:24:07 crc kubenswrapper[5052]: E1203 13:24:07.193630 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="24607acd-2ccb-487b-b383-9601ca4f3d80" containerName="keystone-api" Dec 03 13:24:07 crc kubenswrapper[5052]: I1203 13:24:07.193676 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="24607acd-2ccb-487b-b383-9601ca4f3d80" containerName="keystone-api" Dec 03 13:24:07 crc kubenswrapper[5052]: I1203 13:24:07.193895 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="43950713-65ca-4855-89b5-de8d245df9c8" containerName="mariadb-account-delete" Dec 03 13:24:07 crc kubenswrapper[5052]: I1203 13:24:07.193985 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="eac56493-7a8a-4685-9bb1-451c679368c7" containerName="keystone-api" Dec 03 13:24:07 crc kubenswrapper[5052]: I1203 13:24:07.194067 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="24607acd-2ccb-487b-b383-9601ca4f3d80" containerName="keystone-api" Dec 03 13:24:07 crc kubenswrapper[5052]: I1203 13:24:07.194735 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="keystone-kuttl-tests/keystone-db-create-zkhpv" Dec 03 13:24:07 crc kubenswrapper[5052]: I1203 13:24:07.202132 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["keystone-kuttl-tests/keystone-db-create-zkhpv"] Dec 03 13:24:07 crc kubenswrapper[5052]: I1203 13:24:07.292428 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c9gkp\" (UniqueName: \"kubernetes.io/projected/d6ae75e1-39bf-4fdf-a0b9-231b3ce16f9c-kube-api-access-c9gkp\") pod \"keystone-db-create-zkhpv\" (UID: \"d6ae75e1-39bf-4fdf-a0b9-231b3ce16f9c\") " pod="keystone-kuttl-tests/keystone-db-create-zkhpv" Dec 03 13:24:07 crc kubenswrapper[5052]: I1203 13:24:07.292523 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/d6ae75e1-39bf-4fdf-a0b9-231b3ce16f9c-operator-scripts\") pod \"keystone-db-create-zkhpv\" (UID: \"d6ae75e1-39bf-4fdf-a0b9-231b3ce16f9c\") " pod="keystone-kuttl-tests/keystone-db-create-zkhpv" Dec 03 13:24:07 crc kubenswrapper[5052]: I1203 13:24:07.296846 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["keystone-kuttl-tests/keystone-3828-account-create-update-ggk67"] Dec 03 13:24:07 crc kubenswrapper[5052]: I1203 13:24:07.297879 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="keystone-kuttl-tests/keystone-3828-account-create-update-ggk67" Dec 03 13:24:07 crc kubenswrapper[5052]: I1203 13:24:07.301289 5052 reflector.go:368] Caches populated for *v1.Secret from object-"keystone-kuttl-tests"/"keystone-db-secret" Dec 03 13:24:07 crc kubenswrapper[5052]: I1203 13:24:07.307415 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["keystone-kuttl-tests/keystone-3828-account-create-update-ggk67"] Dec 03 13:24:07 crc kubenswrapper[5052]: I1203 13:24:07.394136 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mgmq4\" (UniqueName: \"kubernetes.io/projected/31c262f6-ecf6-4e0f-9c98-187f2c3e814a-kube-api-access-mgmq4\") pod \"keystone-3828-account-create-update-ggk67\" (UID: \"31c262f6-ecf6-4e0f-9c98-187f2c3e814a\") " pod="keystone-kuttl-tests/keystone-3828-account-create-update-ggk67" Dec 03 13:24:07 crc kubenswrapper[5052]: I1203 13:24:07.394444 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-c9gkp\" (UniqueName: \"kubernetes.io/projected/d6ae75e1-39bf-4fdf-a0b9-231b3ce16f9c-kube-api-access-c9gkp\") pod \"keystone-db-create-zkhpv\" (UID: \"d6ae75e1-39bf-4fdf-a0b9-231b3ce16f9c\") " pod="keystone-kuttl-tests/keystone-db-create-zkhpv" Dec 03 13:24:07 crc kubenswrapper[5052]: I1203 13:24:07.394562 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/d6ae75e1-39bf-4fdf-a0b9-231b3ce16f9c-operator-scripts\") pod \"keystone-db-create-zkhpv\" (UID: \"d6ae75e1-39bf-4fdf-a0b9-231b3ce16f9c\") " pod="keystone-kuttl-tests/keystone-db-create-zkhpv" Dec 03 13:24:07 crc kubenswrapper[5052]: I1203 13:24:07.394704 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/31c262f6-ecf6-4e0f-9c98-187f2c3e814a-operator-scripts\") pod \"keystone-3828-account-create-update-ggk67\" (UID: \"31c262f6-ecf6-4e0f-9c98-187f2c3e814a\") " pod="keystone-kuttl-tests/keystone-3828-account-create-update-ggk67" Dec 03 13:24:07 crc kubenswrapper[5052]: I1203 13:24:07.395864 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/d6ae75e1-39bf-4fdf-a0b9-231b3ce16f9c-operator-scripts\") pod \"keystone-db-create-zkhpv\" (UID: \"d6ae75e1-39bf-4fdf-a0b9-231b3ce16f9c\") " pod="keystone-kuttl-tests/keystone-db-create-zkhpv" Dec 03 13:24:07 crc kubenswrapper[5052]: I1203 13:24:07.415358 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-c9gkp\" (UniqueName: \"kubernetes.io/projected/d6ae75e1-39bf-4fdf-a0b9-231b3ce16f9c-kube-api-access-c9gkp\") pod \"keystone-db-create-zkhpv\" (UID: \"d6ae75e1-39bf-4fdf-a0b9-231b3ce16f9c\") " pod="keystone-kuttl-tests/keystone-db-create-zkhpv" Dec 03 13:24:07 crc kubenswrapper[5052]: I1203 13:24:07.496006 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/31c262f6-ecf6-4e0f-9c98-187f2c3e814a-operator-scripts\") pod \"keystone-3828-account-create-update-ggk67\" (UID: \"31c262f6-ecf6-4e0f-9c98-187f2c3e814a\") " pod="keystone-kuttl-tests/keystone-3828-account-create-update-ggk67" Dec 03 13:24:07 crc kubenswrapper[5052]: I1203 13:24:07.496087 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-mgmq4\" (UniqueName: \"kubernetes.io/projected/31c262f6-ecf6-4e0f-9c98-187f2c3e814a-kube-api-access-mgmq4\") pod \"keystone-3828-account-create-update-ggk67\" (UID: \"31c262f6-ecf6-4e0f-9c98-187f2c3e814a\") " pod="keystone-kuttl-tests/keystone-3828-account-create-update-ggk67" Dec 03 13:24:07 crc kubenswrapper[5052]: I1203 13:24:07.497204 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/31c262f6-ecf6-4e0f-9c98-187f2c3e814a-operator-scripts\") pod \"keystone-3828-account-create-update-ggk67\" (UID: \"31c262f6-ecf6-4e0f-9c98-187f2c3e814a\") " pod="keystone-kuttl-tests/keystone-3828-account-create-update-ggk67" Dec 03 13:24:07 crc kubenswrapper[5052]: I1203 13:24:07.512914 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="keystone-kuttl-tests/keystone-db-create-zkhpv" Dec 03 13:24:07 crc kubenswrapper[5052]: I1203 13:24:07.519719 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-mgmq4\" (UniqueName: \"kubernetes.io/projected/31c262f6-ecf6-4e0f-9c98-187f2c3e814a-kube-api-access-mgmq4\") pod \"keystone-3828-account-create-update-ggk67\" (UID: \"31c262f6-ecf6-4e0f-9c98-187f2c3e814a\") " pod="keystone-kuttl-tests/keystone-3828-account-create-update-ggk67" Dec 03 13:24:07 crc kubenswrapper[5052]: I1203 13:24:07.615224 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="keystone-kuttl-tests/keystone-3828-account-create-update-ggk67" Dec 03 13:24:07 crc kubenswrapper[5052]: I1203 13:24:07.682336 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="43950713-65ca-4855-89b5-de8d245df9c8" path="/var/lib/kubelet/pods/43950713-65ca-4855-89b5-de8d245df9c8/volumes" Dec 03 13:24:07 crc kubenswrapper[5052]: I1203 13:24:07.683467 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="57a8e2c4-46ec-4f92-9f60-2a6af9c546ff" path="/var/lib/kubelet/pods/57a8e2c4-46ec-4f92-9f60-2a6af9c546ff/volumes" Dec 03 13:24:07 crc kubenswrapper[5052]: I1203 13:24:07.684356 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8f5a0f03-b75b-41b7-8f6a-695cf4c58a22" path="/var/lib/kubelet/pods/8f5a0f03-b75b-41b7-8f6a-695cf4c58a22/volumes" Dec 03 13:24:07 crc kubenswrapper[5052]: I1203 13:24:07.684961 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eac56493-7a8a-4685-9bb1-451c679368c7" path="/var/lib/kubelet/pods/eac56493-7a8a-4685-9bb1-451c679368c7/volumes" Dec 03 13:24:07 crc kubenswrapper[5052]: I1203 13:24:07.934697 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["keystone-kuttl-tests/keystone-db-create-zkhpv"] Dec 03 13:24:07 crc kubenswrapper[5052]: W1203 13:24:07.938295 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd6ae75e1_39bf_4fdf_a0b9_231b3ce16f9c.slice/crio-d6e7b47045cab2fb07a4bd513fbe24bcc1cbd55ac892eb26639be290ef467165 WatchSource:0}: Error finding container d6e7b47045cab2fb07a4bd513fbe24bcc1cbd55ac892eb26639be290ef467165: Status 404 returned error can't find the container with id d6e7b47045cab2fb07a4bd513fbe24bcc1cbd55ac892eb26639be290ef467165 Dec 03 13:24:08 crc kubenswrapper[5052]: I1203 13:24:08.052204 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["keystone-kuttl-tests/keystone-3828-account-create-update-ggk67"] Dec 03 13:24:08 crc kubenswrapper[5052]: W1203 13:24:08.077473 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod31c262f6_ecf6_4e0f_9c98_187f2c3e814a.slice/crio-13a7b5d8eb977cb5ab5b437a8161c4f23fe7bd8fe7fcf0946fcd3a9e7e38fcc1 WatchSource:0}: Error finding container 13a7b5d8eb977cb5ab5b437a8161c4f23fe7bd8fe7fcf0946fcd3a9e7e38fcc1: Status 404 returned error can't find the container with id 13a7b5d8eb977cb5ab5b437a8161c4f23fe7bd8fe7fcf0946fcd3a9e7e38fcc1 Dec 03 13:24:08 crc kubenswrapper[5052]: I1203 13:24:08.784358 5052 generic.go:334] "Generic (PLEG): container finished" podID="31c262f6-ecf6-4e0f-9c98-187f2c3e814a" containerID="099eaf13ff9f270a1f63f166fcf530c4a80c2dbee77e267ce7612003cad35e12" exitCode=0 Dec 03 13:24:08 crc kubenswrapper[5052]: I1203 13:24:08.784443 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="keystone-kuttl-tests/keystone-3828-account-create-update-ggk67" event={"ID":"31c262f6-ecf6-4e0f-9c98-187f2c3e814a","Type":"ContainerDied","Data":"099eaf13ff9f270a1f63f166fcf530c4a80c2dbee77e267ce7612003cad35e12"} Dec 03 13:24:08 crc kubenswrapper[5052]: I1203 13:24:08.785013 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="keystone-kuttl-tests/keystone-3828-account-create-update-ggk67" event={"ID":"31c262f6-ecf6-4e0f-9c98-187f2c3e814a","Type":"ContainerStarted","Data":"13a7b5d8eb977cb5ab5b437a8161c4f23fe7bd8fe7fcf0946fcd3a9e7e38fcc1"} Dec 03 13:24:08 crc kubenswrapper[5052]: I1203 13:24:08.786076 5052 generic.go:334] "Generic (PLEG): container finished" podID="d6ae75e1-39bf-4fdf-a0b9-231b3ce16f9c" containerID="646a267de36c895038df3d44efc515c5d638e303a5b0b303acb5bc3f5fc4bc54" exitCode=0 Dec 03 13:24:08 crc kubenswrapper[5052]: I1203 13:24:08.786133 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="keystone-kuttl-tests/keystone-db-create-zkhpv" event={"ID":"d6ae75e1-39bf-4fdf-a0b9-231b3ce16f9c","Type":"ContainerDied","Data":"646a267de36c895038df3d44efc515c5d638e303a5b0b303acb5bc3f5fc4bc54"} Dec 03 13:24:08 crc kubenswrapper[5052]: I1203 13:24:08.786175 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="keystone-kuttl-tests/keystone-db-create-zkhpv" event={"ID":"d6ae75e1-39bf-4fdf-a0b9-231b3ce16f9c","Type":"ContainerStarted","Data":"d6e7b47045cab2fb07a4bd513fbe24bcc1cbd55ac892eb26639be290ef467165"} Dec 03 13:24:10 crc kubenswrapper[5052]: I1203 13:24:10.106710 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="keystone-kuttl-tests/keystone-db-create-zkhpv" Dec 03 13:24:10 crc kubenswrapper[5052]: I1203 13:24:10.112980 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="keystone-kuttl-tests/keystone-3828-account-create-update-ggk67" Dec 03 13:24:10 crc kubenswrapper[5052]: I1203 13:24:10.247253 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/31c262f6-ecf6-4e0f-9c98-187f2c3e814a-operator-scripts\") pod \"31c262f6-ecf6-4e0f-9c98-187f2c3e814a\" (UID: \"31c262f6-ecf6-4e0f-9c98-187f2c3e814a\") " Dec 03 13:24:10 crc kubenswrapper[5052]: I1203 13:24:10.247345 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/d6ae75e1-39bf-4fdf-a0b9-231b3ce16f9c-operator-scripts\") pod \"d6ae75e1-39bf-4fdf-a0b9-231b3ce16f9c\" (UID: \"d6ae75e1-39bf-4fdf-a0b9-231b3ce16f9c\") " Dec 03 13:24:10 crc kubenswrapper[5052]: I1203 13:24:10.247377 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c9gkp\" (UniqueName: \"kubernetes.io/projected/d6ae75e1-39bf-4fdf-a0b9-231b3ce16f9c-kube-api-access-c9gkp\") pod \"d6ae75e1-39bf-4fdf-a0b9-231b3ce16f9c\" (UID: \"d6ae75e1-39bf-4fdf-a0b9-231b3ce16f9c\") " Dec 03 13:24:10 crc kubenswrapper[5052]: I1203 13:24:10.247453 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mgmq4\" (UniqueName: \"kubernetes.io/projected/31c262f6-ecf6-4e0f-9c98-187f2c3e814a-kube-api-access-mgmq4\") pod \"31c262f6-ecf6-4e0f-9c98-187f2c3e814a\" (UID: \"31c262f6-ecf6-4e0f-9c98-187f2c3e814a\") " Dec 03 13:24:10 crc kubenswrapper[5052]: I1203 13:24:10.248245 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d6ae75e1-39bf-4fdf-a0b9-231b3ce16f9c-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "d6ae75e1-39bf-4fdf-a0b9-231b3ce16f9c" (UID: "d6ae75e1-39bf-4fdf-a0b9-231b3ce16f9c"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:24:10 crc kubenswrapper[5052]: I1203 13:24:10.248583 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/31c262f6-ecf6-4e0f-9c98-187f2c3e814a-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "31c262f6-ecf6-4e0f-9c98-187f2c3e814a" (UID: "31c262f6-ecf6-4e0f-9c98-187f2c3e814a"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:24:10 crc kubenswrapper[5052]: I1203 13:24:10.252256 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/31c262f6-ecf6-4e0f-9c98-187f2c3e814a-kube-api-access-mgmq4" (OuterVolumeSpecName: "kube-api-access-mgmq4") pod "31c262f6-ecf6-4e0f-9c98-187f2c3e814a" (UID: "31c262f6-ecf6-4e0f-9c98-187f2c3e814a"). InnerVolumeSpecName "kube-api-access-mgmq4". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:24:10 crc kubenswrapper[5052]: I1203 13:24:10.252290 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d6ae75e1-39bf-4fdf-a0b9-231b3ce16f9c-kube-api-access-c9gkp" (OuterVolumeSpecName: "kube-api-access-c9gkp") pod "d6ae75e1-39bf-4fdf-a0b9-231b3ce16f9c" (UID: "d6ae75e1-39bf-4fdf-a0b9-231b3ce16f9c"). InnerVolumeSpecName "kube-api-access-c9gkp". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:24:10 crc kubenswrapper[5052]: I1203 13:24:10.349398 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/31c262f6-ecf6-4e0f-9c98-187f2c3e814a-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 13:24:10 crc kubenswrapper[5052]: I1203 13:24:10.349451 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/d6ae75e1-39bf-4fdf-a0b9-231b3ce16f9c-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 13:24:10 crc kubenswrapper[5052]: I1203 13:24:10.349471 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-c9gkp\" (UniqueName: \"kubernetes.io/projected/d6ae75e1-39bf-4fdf-a0b9-231b3ce16f9c-kube-api-access-c9gkp\") on node \"crc\" DevicePath \"\"" Dec 03 13:24:10 crc kubenswrapper[5052]: I1203 13:24:10.349487 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-mgmq4\" (UniqueName: \"kubernetes.io/projected/31c262f6-ecf6-4e0f-9c98-187f2c3e814a-kube-api-access-mgmq4\") on node \"crc\" DevicePath \"\"" Dec 03 13:24:10 crc kubenswrapper[5052]: I1203 13:24:10.802383 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="keystone-kuttl-tests/keystone-db-create-zkhpv" Dec 03 13:24:10 crc kubenswrapper[5052]: I1203 13:24:10.802395 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="keystone-kuttl-tests/keystone-db-create-zkhpv" event={"ID":"d6ae75e1-39bf-4fdf-a0b9-231b3ce16f9c","Type":"ContainerDied","Data":"d6e7b47045cab2fb07a4bd513fbe24bcc1cbd55ac892eb26639be290ef467165"} Dec 03 13:24:10 crc kubenswrapper[5052]: I1203 13:24:10.802434 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d6e7b47045cab2fb07a4bd513fbe24bcc1cbd55ac892eb26639be290ef467165" Dec 03 13:24:10 crc kubenswrapper[5052]: I1203 13:24:10.804187 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="keystone-kuttl-tests/keystone-3828-account-create-update-ggk67" event={"ID":"31c262f6-ecf6-4e0f-9c98-187f2c3e814a","Type":"ContainerDied","Data":"13a7b5d8eb977cb5ab5b437a8161c4f23fe7bd8fe7fcf0946fcd3a9e7e38fcc1"} Dec 03 13:24:10 crc kubenswrapper[5052]: I1203 13:24:10.804222 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="13a7b5d8eb977cb5ab5b437a8161c4f23fe7bd8fe7fcf0946fcd3a9e7e38fcc1" Dec 03 13:24:10 crc kubenswrapper[5052]: I1203 13:24:10.804233 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="keystone-kuttl-tests/keystone-3828-account-create-update-ggk67" Dec 03 13:24:12 crc kubenswrapper[5052]: I1203 13:24:12.856432 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["keystone-kuttl-tests/keystone-db-sync-2jjgq"] Dec 03 13:24:12 crc kubenswrapper[5052]: E1203 13:24:12.857348 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="d6ae75e1-39bf-4fdf-a0b9-231b3ce16f9c" containerName="mariadb-database-create" Dec 03 13:24:12 crc kubenswrapper[5052]: I1203 13:24:12.857380 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="d6ae75e1-39bf-4fdf-a0b9-231b3ce16f9c" containerName="mariadb-database-create" Dec 03 13:24:12 crc kubenswrapper[5052]: E1203 13:24:12.857409 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="31c262f6-ecf6-4e0f-9c98-187f2c3e814a" containerName="mariadb-account-create-update" Dec 03 13:24:12 crc kubenswrapper[5052]: I1203 13:24:12.857421 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="31c262f6-ecf6-4e0f-9c98-187f2c3e814a" containerName="mariadb-account-create-update" Dec 03 13:24:12 crc kubenswrapper[5052]: I1203 13:24:12.857596 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="d6ae75e1-39bf-4fdf-a0b9-231b3ce16f9c" containerName="mariadb-database-create" Dec 03 13:24:12 crc kubenswrapper[5052]: I1203 13:24:12.857612 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="31c262f6-ecf6-4e0f-9c98-187f2c3e814a" containerName="mariadb-account-create-update" Dec 03 13:24:12 crc kubenswrapper[5052]: I1203 13:24:12.858115 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="keystone-kuttl-tests/keystone-db-sync-2jjgq" Dec 03 13:24:12 crc kubenswrapper[5052]: I1203 13:24:12.861987 5052 reflector.go:368] Caches populated for *v1.Secret from object-"keystone-kuttl-tests"/"keystone" Dec 03 13:24:12 crc kubenswrapper[5052]: I1203 13:24:12.862286 5052 reflector.go:368] Caches populated for *v1.Secret from object-"keystone-kuttl-tests"/"combined-ca-bundle" Dec 03 13:24:12 crc kubenswrapper[5052]: I1203 13:24:12.862497 5052 reflector.go:368] Caches populated for *v1.Secret from object-"keystone-kuttl-tests"/"keystone-scripts" Dec 03 13:24:12 crc kubenswrapper[5052]: I1203 13:24:12.862780 5052 reflector.go:368] Caches populated for *v1.Secret from object-"keystone-kuttl-tests"/"keystone-keystone-dockercfg-gjxdd" Dec 03 13:24:12 crc kubenswrapper[5052]: I1203 13:24:12.863343 5052 reflector.go:368] Caches populated for *v1.Secret from object-"keystone-kuttl-tests"/"keystone-config-data" Dec 03 13:24:12 crc kubenswrapper[5052]: I1203 13:24:12.864152 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["keystone-kuttl-tests/keystone-db-sync-2jjgq"] Dec 03 13:24:13 crc kubenswrapper[5052]: I1203 13:24:13.003353 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5742d593-c183-48fa-b69f-e074cfb3ebaa-combined-ca-bundle\") pod \"keystone-db-sync-2jjgq\" (UID: \"5742d593-c183-48fa-b69f-e074cfb3ebaa\") " pod="keystone-kuttl-tests/keystone-db-sync-2jjgq" Dec 03 13:24:13 crc kubenswrapper[5052]: I1203 13:24:13.003394 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gvppq\" (UniqueName: \"kubernetes.io/projected/5742d593-c183-48fa-b69f-e074cfb3ebaa-kube-api-access-gvppq\") pod \"keystone-db-sync-2jjgq\" (UID: \"5742d593-c183-48fa-b69f-e074cfb3ebaa\") " pod="keystone-kuttl-tests/keystone-db-sync-2jjgq" Dec 03 13:24:13 crc kubenswrapper[5052]: I1203 13:24:13.003460 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/5742d593-c183-48fa-b69f-e074cfb3ebaa-config-data\") pod \"keystone-db-sync-2jjgq\" (UID: \"5742d593-c183-48fa-b69f-e074cfb3ebaa\") " pod="keystone-kuttl-tests/keystone-db-sync-2jjgq" Dec 03 13:24:13 crc kubenswrapper[5052]: I1203 13:24:13.105255 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/5742d593-c183-48fa-b69f-e074cfb3ebaa-config-data\") pod \"keystone-db-sync-2jjgq\" (UID: \"5742d593-c183-48fa-b69f-e074cfb3ebaa\") " pod="keystone-kuttl-tests/keystone-db-sync-2jjgq" Dec 03 13:24:13 crc kubenswrapper[5052]: I1203 13:24:13.106393 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-gvppq\" (UniqueName: \"kubernetes.io/projected/5742d593-c183-48fa-b69f-e074cfb3ebaa-kube-api-access-gvppq\") pod \"keystone-db-sync-2jjgq\" (UID: \"5742d593-c183-48fa-b69f-e074cfb3ebaa\") " pod="keystone-kuttl-tests/keystone-db-sync-2jjgq" Dec 03 13:24:13 crc kubenswrapper[5052]: I1203 13:24:13.106429 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5742d593-c183-48fa-b69f-e074cfb3ebaa-combined-ca-bundle\") pod \"keystone-db-sync-2jjgq\" (UID: \"5742d593-c183-48fa-b69f-e074cfb3ebaa\") " pod="keystone-kuttl-tests/keystone-db-sync-2jjgq" Dec 03 13:24:13 crc kubenswrapper[5052]: I1203 13:24:13.110032 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5742d593-c183-48fa-b69f-e074cfb3ebaa-combined-ca-bundle\") pod \"keystone-db-sync-2jjgq\" (UID: \"5742d593-c183-48fa-b69f-e074cfb3ebaa\") " pod="keystone-kuttl-tests/keystone-db-sync-2jjgq" Dec 03 13:24:13 crc kubenswrapper[5052]: I1203 13:24:13.113218 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/5742d593-c183-48fa-b69f-e074cfb3ebaa-config-data\") pod \"keystone-db-sync-2jjgq\" (UID: \"5742d593-c183-48fa-b69f-e074cfb3ebaa\") " pod="keystone-kuttl-tests/keystone-db-sync-2jjgq" Dec 03 13:24:13 crc kubenswrapper[5052]: I1203 13:24:13.126777 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-gvppq\" (UniqueName: \"kubernetes.io/projected/5742d593-c183-48fa-b69f-e074cfb3ebaa-kube-api-access-gvppq\") pod \"keystone-db-sync-2jjgq\" (UID: \"5742d593-c183-48fa-b69f-e074cfb3ebaa\") " pod="keystone-kuttl-tests/keystone-db-sync-2jjgq" Dec 03 13:24:13 crc kubenswrapper[5052]: I1203 13:24:13.212660 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="keystone-kuttl-tests/keystone-db-sync-2jjgq" Dec 03 13:24:13 crc kubenswrapper[5052]: I1203 13:24:13.642554 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["keystone-kuttl-tests/keystone-db-sync-2jjgq"] Dec 03 13:24:13 crc kubenswrapper[5052]: I1203 13:24:13.831687 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="keystone-kuttl-tests/keystone-db-sync-2jjgq" event={"ID":"5742d593-c183-48fa-b69f-e074cfb3ebaa","Type":"ContainerStarted","Data":"041258df7df51ef69e0aa8df4ebfe15bf23915b3cae94d446e2e20673ac91cf7"} Dec 03 13:24:14 crc kubenswrapper[5052]: I1203 13:24:14.840651 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="keystone-kuttl-tests/keystone-db-sync-2jjgq" event={"ID":"5742d593-c183-48fa-b69f-e074cfb3ebaa","Type":"ContainerStarted","Data":"afc7258a68d4ed2a4b7cff8e1f795eeb19f0ce27b182320ca9cfd851d51e629e"} Dec 03 13:24:14 crc kubenswrapper[5052]: I1203 13:24:14.857056 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="keystone-kuttl-tests/keystone-db-sync-2jjgq" podStartSLOduration=2.8569929590000003 podStartE2EDuration="2.856992959s" podCreationTimestamp="2025-12-03 13:24:12 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 13:24:14.854975721 +0000 UTC m=+7445.443217485" watchObservedRunningTime="2025-12-03 13:24:14.856992959 +0000 UTC m=+7445.445234723" Dec 03 13:24:15 crc kubenswrapper[5052]: I1203 13:24:15.853557 5052 generic.go:334] "Generic (PLEG): container finished" podID="5742d593-c183-48fa-b69f-e074cfb3ebaa" containerID="afc7258a68d4ed2a4b7cff8e1f795eeb19f0ce27b182320ca9cfd851d51e629e" exitCode=0 Dec 03 13:24:15 crc kubenswrapper[5052]: I1203 13:24:15.853680 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="keystone-kuttl-tests/keystone-db-sync-2jjgq" event={"ID":"5742d593-c183-48fa-b69f-e074cfb3ebaa","Type":"ContainerDied","Data":"afc7258a68d4ed2a4b7cff8e1f795eeb19f0ce27b182320ca9cfd851d51e629e"} Dec 03 13:24:17 crc kubenswrapper[5052]: I1203 13:24:17.213383 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="keystone-kuttl-tests/keystone-db-sync-2jjgq" Dec 03 13:24:17 crc kubenswrapper[5052]: I1203 13:24:17.369415 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gvppq\" (UniqueName: \"kubernetes.io/projected/5742d593-c183-48fa-b69f-e074cfb3ebaa-kube-api-access-gvppq\") pod \"5742d593-c183-48fa-b69f-e074cfb3ebaa\" (UID: \"5742d593-c183-48fa-b69f-e074cfb3ebaa\") " Dec 03 13:24:17 crc kubenswrapper[5052]: I1203 13:24:17.369487 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5742d593-c183-48fa-b69f-e074cfb3ebaa-combined-ca-bundle\") pod \"5742d593-c183-48fa-b69f-e074cfb3ebaa\" (UID: \"5742d593-c183-48fa-b69f-e074cfb3ebaa\") " Dec 03 13:24:17 crc kubenswrapper[5052]: I1203 13:24:17.369718 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/5742d593-c183-48fa-b69f-e074cfb3ebaa-config-data\") pod \"5742d593-c183-48fa-b69f-e074cfb3ebaa\" (UID: \"5742d593-c183-48fa-b69f-e074cfb3ebaa\") " Dec 03 13:24:17 crc kubenswrapper[5052]: I1203 13:24:17.383577 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5742d593-c183-48fa-b69f-e074cfb3ebaa-kube-api-access-gvppq" (OuterVolumeSpecName: "kube-api-access-gvppq") pod "5742d593-c183-48fa-b69f-e074cfb3ebaa" (UID: "5742d593-c183-48fa-b69f-e074cfb3ebaa"). InnerVolumeSpecName "kube-api-access-gvppq". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:24:17 crc kubenswrapper[5052]: I1203 13:24:17.394364 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5742d593-c183-48fa-b69f-e074cfb3ebaa-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "5742d593-c183-48fa-b69f-e074cfb3ebaa" (UID: "5742d593-c183-48fa-b69f-e074cfb3ebaa"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:24:17 crc kubenswrapper[5052]: I1203 13:24:17.415554 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5742d593-c183-48fa-b69f-e074cfb3ebaa-config-data" (OuterVolumeSpecName: "config-data") pod "5742d593-c183-48fa-b69f-e074cfb3ebaa" (UID: "5742d593-c183-48fa-b69f-e074cfb3ebaa"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:24:17 crc kubenswrapper[5052]: I1203 13:24:17.471077 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/5742d593-c183-48fa-b69f-e074cfb3ebaa-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 13:24:17 crc kubenswrapper[5052]: I1203 13:24:17.471120 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-gvppq\" (UniqueName: \"kubernetes.io/projected/5742d593-c183-48fa-b69f-e074cfb3ebaa-kube-api-access-gvppq\") on node \"crc\" DevicePath \"\"" Dec 03 13:24:17 crc kubenswrapper[5052]: I1203 13:24:17.471134 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5742d593-c183-48fa-b69f-e074cfb3ebaa-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 13:24:17 crc kubenswrapper[5052]: I1203 13:24:17.874629 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="keystone-kuttl-tests/keystone-db-sync-2jjgq" event={"ID":"5742d593-c183-48fa-b69f-e074cfb3ebaa","Type":"ContainerDied","Data":"041258df7df51ef69e0aa8df4ebfe15bf23915b3cae94d446e2e20673ac91cf7"} Dec 03 13:24:17 crc kubenswrapper[5052]: I1203 13:24:17.874692 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="041258df7df51ef69e0aa8df4ebfe15bf23915b3cae94d446e2e20673ac91cf7" Dec 03 13:24:17 crc kubenswrapper[5052]: I1203 13:24:17.874737 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="keystone-kuttl-tests/keystone-db-sync-2jjgq" Dec 03 13:24:18 crc kubenswrapper[5052]: I1203 13:24:18.019096 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["keystone-kuttl-tests/keystone-bootstrap-mq5st"] Dec 03 13:24:18 crc kubenswrapper[5052]: E1203 13:24:18.019561 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="5742d593-c183-48fa-b69f-e074cfb3ebaa" containerName="keystone-db-sync" Dec 03 13:24:18 crc kubenswrapper[5052]: I1203 13:24:18.019591 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="5742d593-c183-48fa-b69f-e074cfb3ebaa" containerName="keystone-db-sync" Dec 03 13:24:18 crc kubenswrapper[5052]: I1203 13:24:18.019778 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="5742d593-c183-48fa-b69f-e074cfb3ebaa" containerName="keystone-db-sync" Dec 03 13:24:18 crc kubenswrapper[5052]: I1203 13:24:18.020393 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="keystone-kuttl-tests/keystone-bootstrap-mq5st" Dec 03 13:24:18 crc kubenswrapper[5052]: I1203 13:24:18.027943 5052 reflector.go:368] Caches populated for *v1.Secret from object-"keystone-kuttl-tests"/"combined-ca-bundle" Dec 03 13:24:18 crc kubenswrapper[5052]: I1203 13:24:18.028492 5052 reflector.go:368] Caches populated for *v1.Secret from object-"keystone-kuttl-tests"/"keystone-scripts" Dec 03 13:24:18 crc kubenswrapper[5052]: I1203 13:24:18.029248 5052 reflector.go:368] Caches populated for *v1.Secret from object-"keystone-kuttl-tests"/"keystone" Dec 03 13:24:18 crc kubenswrapper[5052]: I1203 13:24:18.029422 5052 reflector.go:368] Caches populated for *v1.Secret from object-"keystone-kuttl-tests"/"osp-secret" Dec 03 13:24:18 crc kubenswrapper[5052]: I1203 13:24:18.029676 5052 reflector.go:368] Caches populated for *v1.Secret from object-"keystone-kuttl-tests"/"keystone-config-data" Dec 03 13:24:18 crc kubenswrapper[5052]: I1203 13:24:18.029815 5052 reflector.go:368] Caches populated for *v1.Secret from object-"keystone-kuttl-tests"/"keystone-keystone-dockercfg-gjxdd" Dec 03 13:24:18 crc kubenswrapper[5052]: I1203 13:24:18.035061 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["keystone-kuttl-tests/keystone-bootstrap-mq5st"] Dec 03 13:24:18 crc kubenswrapper[5052]: I1203 13:24:18.181864 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/b72867db-9193-4ca4-9829-253609425890-credential-keys\") pod \"keystone-bootstrap-mq5st\" (UID: \"b72867db-9193-4ca4-9829-253609425890\") " pod="keystone-kuttl-tests/keystone-bootstrap-mq5st" Dec 03 13:24:18 crc kubenswrapper[5052]: I1203 13:24:18.181921 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zrfgj\" (UniqueName: \"kubernetes.io/projected/b72867db-9193-4ca4-9829-253609425890-kube-api-access-zrfgj\") pod \"keystone-bootstrap-mq5st\" (UID: \"b72867db-9193-4ca4-9829-253609425890\") " pod="keystone-kuttl-tests/keystone-bootstrap-mq5st" Dec 03 13:24:18 crc kubenswrapper[5052]: I1203 13:24:18.181967 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/b72867db-9193-4ca4-9829-253609425890-scripts\") pod \"keystone-bootstrap-mq5st\" (UID: \"b72867db-9193-4ca4-9829-253609425890\") " pod="keystone-kuttl-tests/keystone-bootstrap-mq5st" Dec 03 13:24:18 crc kubenswrapper[5052]: I1203 13:24:18.182063 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b72867db-9193-4ca4-9829-253609425890-combined-ca-bundle\") pod \"keystone-bootstrap-mq5st\" (UID: \"b72867db-9193-4ca4-9829-253609425890\") " pod="keystone-kuttl-tests/keystone-bootstrap-mq5st" Dec 03 13:24:18 crc kubenswrapper[5052]: I1203 13:24:18.182103 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/b72867db-9193-4ca4-9829-253609425890-fernet-keys\") pod \"keystone-bootstrap-mq5st\" (UID: \"b72867db-9193-4ca4-9829-253609425890\") " pod="keystone-kuttl-tests/keystone-bootstrap-mq5st" Dec 03 13:24:18 crc kubenswrapper[5052]: I1203 13:24:18.182124 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/b72867db-9193-4ca4-9829-253609425890-config-data\") pod \"keystone-bootstrap-mq5st\" (UID: \"b72867db-9193-4ca4-9829-253609425890\") " pod="keystone-kuttl-tests/keystone-bootstrap-mq5st" Dec 03 13:24:18 crc kubenswrapper[5052]: I1203 13:24:18.284007 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/b72867db-9193-4ca4-9829-253609425890-credential-keys\") pod \"keystone-bootstrap-mq5st\" (UID: \"b72867db-9193-4ca4-9829-253609425890\") " pod="keystone-kuttl-tests/keystone-bootstrap-mq5st" Dec 03 13:24:18 crc kubenswrapper[5052]: I1203 13:24:18.284089 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-zrfgj\" (UniqueName: \"kubernetes.io/projected/b72867db-9193-4ca4-9829-253609425890-kube-api-access-zrfgj\") pod \"keystone-bootstrap-mq5st\" (UID: \"b72867db-9193-4ca4-9829-253609425890\") " pod="keystone-kuttl-tests/keystone-bootstrap-mq5st" Dec 03 13:24:18 crc kubenswrapper[5052]: I1203 13:24:18.284140 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/b72867db-9193-4ca4-9829-253609425890-scripts\") pod \"keystone-bootstrap-mq5st\" (UID: \"b72867db-9193-4ca4-9829-253609425890\") " pod="keystone-kuttl-tests/keystone-bootstrap-mq5st" Dec 03 13:24:18 crc kubenswrapper[5052]: I1203 13:24:18.284193 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b72867db-9193-4ca4-9829-253609425890-combined-ca-bundle\") pod \"keystone-bootstrap-mq5st\" (UID: \"b72867db-9193-4ca4-9829-253609425890\") " pod="keystone-kuttl-tests/keystone-bootstrap-mq5st" Dec 03 13:24:18 crc kubenswrapper[5052]: I1203 13:24:18.284218 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/b72867db-9193-4ca4-9829-253609425890-fernet-keys\") pod \"keystone-bootstrap-mq5st\" (UID: \"b72867db-9193-4ca4-9829-253609425890\") " pod="keystone-kuttl-tests/keystone-bootstrap-mq5st" Dec 03 13:24:18 crc kubenswrapper[5052]: I1203 13:24:18.284240 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/b72867db-9193-4ca4-9829-253609425890-config-data\") pod \"keystone-bootstrap-mq5st\" (UID: \"b72867db-9193-4ca4-9829-253609425890\") " pod="keystone-kuttl-tests/keystone-bootstrap-mq5st" Dec 03 13:24:18 crc kubenswrapper[5052]: I1203 13:24:18.288536 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b72867db-9193-4ca4-9829-253609425890-combined-ca-bundle\") pod \"keystone-bootstrap-mq5st\" (UID: \"b72867db-9193-4ca4-9829-253609425890\") " pod="keystone-kuttl-tests/keystone-bootstrap-mq5st" Dec 03 13:24:18 crc kubenswrapper[5052]: I1203 13:24:18.288923 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/b72867db-9193-4ca4-9829-253609425890-config-data\") pod \"keystone-bootstrap-mq5st\" (UID: \"b72867db-9193-4ca4-9829-253609425890\") " pod="keystone-kuttl-tests/keystone-bootstrap-mq5st" Dec 03 13:24:18 crc kubenswrapper[5052]: I1203 13:24:18.290427 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/b72867db-9193-4ca4-9829-253609425890-fernet-keys\") pod \"keystone-bootstrap-mq5st\" (UID: \"b72867db-9193-4ca4-9829-253609425890\") " pod="keystone-kuttl-tests/keystone-bootstrap-mq5st" Dec 03 13:24:18 crc kubenswrapper[5052]: I1203 13:24:18.290814 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/b72867db-9193-4ca4-9829-253609425890-credential-keys\") pod \"keystone-bootstrap-mq5st\" (UID: \"b72867db-9193-4ca4-9829-253609425890\") " pod="keystone-kuttl-tests/keystone-bootstrap-mq5st" Dec 03 13:24:18 crc kubenswrapper[5052]: I1203 13:24:18.291253 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/b72867db-9193-4ca4-9829-253609425890-scripts\") pod \"keystone-bootstrap-mq5st\" (UID: \"b72867db-9193-4ca4-9829-253609425890\") " pod="keystone-kuttl-tests/keystone-bootstrap-mq5st" Dec 03 13:24:18 crc kubenswrapper[5052]: I1203 13:24:18.309627 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-zrfgj\" (UniqueName: \"kubernetes.io/projected/b72867db-9193-4ca4-9829-253609425890-kube-api-access-zrfgj\") pod \"keystone-bootstrap-mq5st\" (UID: \"b72867db-9193-4ca4-9829-253609425890\") " pod="keystone-kuttl-tests/keystone-bootstrap-mq5st" Dec 03 13:24:18 crc kubenswrapper[5052]: I1203 13:24:18.343206 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="keystone-kuttl-tests/keystone-bootstrap-mq5st" Dec 03 13:24:18 crc kubenswrapper[5052]: I1203 13:24:18.790315 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["keystone-kuttl-tests/keystone-bootstrap-mq5st"] Dec 03 13:24:18 crc kubenswrapper[5052]: I1203 13:24:18.888813 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="keystone-kuttl-tests/keystone-bootstrap-mq5st" event={"ID":"b72867db-9193-4ca4-9829-253609425890","Type":"ContainerStarted","Data":"86385738f07a37dd54b09755f6e8393ee44336325f22959ebc1430bf56ff1c85"} Dec 03 13:24:19 crc kubenswrapper[5052]: I1203 13:24:19.899807 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="keystone-kuttl-tests/keystone-bootstrap-mq5st" event={"ID":"b72867db-9193-4ca4-9829-253609425890","Type":"ContainerStarted","Data":"1d45929c6ce39bf4ba7e60b055c3cb3b4cf00eb1c74796eee6b556fd8acbbd30"} Dec 03 13:24:19 crc kubenswrapper[5052]: I1203 13:24:19.917024 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="keystone-kuttl-tests/keystone-bootstrap-mq5st" podStartSLOduration=2.917003576 podStartE2EDuration="2.917003576s" podCreationTimestamp="2025-12-03 13:24:17 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 13:24:19.916225364 +0000 UTC m=+7450.504467128" watchObservedRunningTime="2025-12-03 13:24:19.917003576 +0000 UTC m=+7450.505245350" Dec 03 13:24:22 crc kubenswrapper[5052]: E1203 13:24:22.252203 5052 cadvisor_stats_provider.go:516] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb72867db_9193_4ca4_9829_253609425890.slice/crio-1d45929c6ce39bf4ba7e60b055c3cb3b4cf00eb1c74796eee6b556fd8acbbd30.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb72867db_9193_4ca4_9829_253609425890.slice/crio-conmon-1d45929c6ce39bf4ba7e60b055c3cb3b4cf00eb1c74796eee6b556fd8acbbd30.scope\": RecentStats: unable to find data in memory cache]" Dec 03 13:24:22 crc kubenswrapper[5052]: I1203 13:24:22.920018 5052 generic.go:334] "Generic (PLEG): container finished" podID="b72867db-9193-4ca4-9829-253609425890" containerID="1d45929c6ce39bf4ba7e60b055c3cb3b4cf00eb1c74796eee6b556fd8acbbd30" exitCode=0 Dec 03 13:24:22 crc kubenswrapper[5052]: I1203 13:24:22.920067 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="keystone-kuttl-tests/keystone-bootstrap-mq5st" event={"ID":"b72867db-9193-4ca4-9829-253609425890","Type":"ContainerDied","Data":"1d45929c6ce39bf4ba7e60b055c3cb3b4cf00eb1c74796eee6b556fd8acbbd30"} Dec 03 13:24:24 crc kubenswrapper[5052]: I1203 13:24:24.237907 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="keystone-kuttl-tests/keystone-bootstrap-mq5st" Dec 03 13:24:24 crc kubenswrapper[5052]: I1203 13:24:24.297730 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/b72867db-9193-4ca4-9829-253609425890-config-data\") pod \"b72867db-9193-4ca4-9829-253609425890\" (UID: \"b72867db-9193-4ca4-9829-253609425890\") " Dec 03 13:24:24 crc kubenswrapper[5052]: I1203 13:24:24.297824 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zrfgj\" (UniqueName: \"kubernetes.io/projected/b72867db-9193-4ca4-9829-253609425890-kube-api-access-zrfgj\") pod \"b72867db-9193-4ca4-9829-253609425890\" (UID: \"b72867db-9193-4ca4-9829-253609425890\") " Dec 03 13:24:24 crc kubenswrapper[5052]: I1203 13:24:24.297858 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/b72867db-9193-4ca4-9829-253609425890-fernet-keys\") pod \"b72867db-9193-4ca4-9829-253609425890\" (UID: \"b72867db-9193-4ca4-9829-253609425890\") " Dec 03 13:24:24 crc kubenswrapper[5052]: I1203 13:24:24.297889 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b72867db-9193-4ca4-9829-253609425890-combined-ca-bundle\") pod \"b72867db-9193-4ca4-9829-253609425890\" (UID: \"b72867db-9193-4ca4-9829-253609425890\") " Dec 03 13:24:24 crc kubenswrapper[5052]: I1203 13:24:24.297934 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/b72867db-9193-4ca4-9829-253609425890-credential-keys\") pod \"b72867db-9193-4ca4-9829-253609425890\" (UID: \"b72867db-9193-4ca4-9829-253609425890\") " Dec 03 13:24:24 crc kubenswrapper[5052]: I1203 13:24:24.297973 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/b72867db-9193-4ca4-9829-253609425890-scripts\") pod \"b72867db-9193-4ca4-9829-253609425890\" (UID: \"b72867db-9193-4ca4-9829-253609425890\") " Dec 03 13:24:24 crc kubenswrapper[5052]: I1203 13:24:24.304351 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b72867db-9193-4ca4-9829-253609425890-kube-api-access-zrfgj" (OuterVolumeSpecName: "kube-api-access-zrfgj") pod "b72867db-9193-4ca4-9829-253609425890" (UID: "b72867db-9193-4ca4-9829-253609425890"). InnerVolumeSpecName "kube-api-access-zrfgj". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:24:24 crc kubenswrapper[5052]: I1203 13:24:24.304500 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b72867db-9193-4ca4-9829-253609425890-fernet-keys" (OuterVolumeSpecName: "fernet-keys") pod "b72867db-9193-4ca4-9829-253609425890" (UID: "b72867db-9193-4ca4-9829-253609425890"). InnerVolumeSpecName "fernet-keys". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:24:24 crc kubenswrapper[5052]: I1203 13:24:24.310104 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b72867db-9193-4ca4-9829-253609425890-credential-keys" (OuterVolumeSpecName: "credential-keys") pod "b72867db-9193-4ca4-9829-253609425890" (UID: "b72867db-9193-4ca4-9829-253609425890"). InnerVolumeSpecName "credential-keys". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:24:24 crc kubenswrapper[5052]: I1203 13:24:24.310491 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b72867db-9193-4ca4-9829-253609425890-scripts" (OuterVolumeSpecName: "scripts") pod "b72867db-9193-4ca4-9829-253609425890" (UID: "b72867db-9193-4ca4-9829-253609425890"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:24:24 crc kubenswrapper[5052]: I1203 13:24:24.322629 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b72867db-9193-4ca4-9829-253609425890-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "b72867db-9193-4ca4-9829-253609425890" (UID: "b72867db-9193-4ca4-9829-253609425890"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:24:24 crc kubenswrapper[5052]: I1203 13:24:24.323951 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b72867db-9193-4ca4-9829-253609425890-config-data" (OuterVolumeSpecName: "config-data") pod "b72867db-9193-4ca4-9829-253609425890" (UID: "b72867db-9193-4ca4-9829-253609425890"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:24:24 crc kubenswrapper[5052]: I1203 13:24:24.399712 5052 reconciler_common.go:293] "Volume detached for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/b72867db-9193-4ca4-9829-253609425890-credential-keys\") on node \"crc\" DevicePath \"\"" Dec 03 13:24:24 crc kubenswrapper[5052]: I1203 13:24:24.400054 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/b72867db-9193-4ca4-9829-253609425890-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 13:24:24 crc kubenswrapper[5052]: I1203 13:24:24.400068 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/b72867db-9193-4ca4-9829-253609425890-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 13:24:24 crc kubenswrapper[5052]: I1203 13:24:24.400086 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-zrfgj\" (UniqueName: \"kubernetes.io/projected/b72867db-9193-4ca4-9829-253609425890-kube-api-access-zrfgj\") on node \"crc\" DevicePath \"\"" Dec 03 13:24:24 crc kubenswrapper[5052]: I1203 13:24:24.400101 5052 reconciler_common.go:293] "Volume detached for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/b72867db-9193-4ca4-9829-253609425890-fernet-keys\") on node \"crc\" DevicePath \"\"" Dec 03 13:24:24 crc kubenswrapper[5052]: I1203 13:24:24.400115 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b72867db-9193-4ca4-9829-253609425890-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 13:24:24 crc kubenswrapper[5052]: I1203 13:24:24.936020 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="keystone-kuttl-tests/keystone-bootstrap-mq5st" event={"ID":"b72867db-9193-4ca4-9829-253609425890","Type":"ContainerDied","Data":"86385738f07a37dd54b09755f6e8393ee44336325f22959ebc1430bf56ff1c85"} Dec 03 13:24:24 crc kubenswrapper[5052]: I1203 13:24:24.936068 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="keystone-kuttl-tests/keystone-bootstrap-mq5st" Dec 03 13:24:24 crc kubenswrapper[5052]: I1203 13:24:24.936073 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="86385738f07a37dd54b09755f6e8393ee44336325f22959ebc1430bf56ff1c85" Dec 03 13:24:25 crc kubenswrapper[5052]: I1203 13:24:25.111866 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["keystone-kuttl-tests/keystone-db495cbcd-dkqhp"] Dec 03 13:24:25 crc kubenswrapper[5052]: E1203 13:24:25.112700 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="b72867db-9193-4ca4-9829-253609425890" containerName="keystone-bootstrap" Dec 03 13:24:25 crc kubenswrapper[5052]: I1203 13:24:25.112718 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="b72867db-9193-4ca4-9829-253609425890" containerName="keystone-bootstrap" Dec 03 13:24:25 crc kubenswrapper[5052]: I1203 13:24:25.113020 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="b72867db-9193-4ca4-9829-253609425890" containerName="keystone-bootstrap" Dec 03 13:24:25 crc kubenswrapper[5052]: I1203 13:24:25.113794 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="keystone-kuttl-tests/keystone-db495cbcd-dkqhp" Dec 03 13:24:25 crc kubenswrapper[5052]: I1203 13:24:25.127258 5052 reflector.go:368] Caches populated for *v1.Secret from object-"keystone-kuttl-tests"/"keystone" Dec 03 13:24:25 crc kubenswrapper[5052]: I1203 13:24:25.127635 5052 reflector.go:368] Caches populated for *v1.Secret from object-"keystone-kuttl-tests"/"cert-keystone-internal-svc" Dec 03 13:24:25 crc kubenswrapper[5052]: I1203 13:24:25.129113 5052 reflector.go:368] Caches populated for *v1.Secret from object-"keystone-kuttl-tests"/"keystone-config-data" Dec 03 13:24:25 crc kubenswrapper[5052]: I1203 13:24:25.129409 5052 reflector.go:368] Caches populated for *v1.Secret from object-"keystone-kuttl-tests"/"keystone-keystone-dockercfg-gjxdd" Dec 03 13:24:25 crc kubenswrapper[5052]: I1203 13:24:25.129726 5052 reflector.go:368] Caches populated for *v1.Secret from object-"keystone-kuttl-tests"/"combined-ca-bundle" Dec 03 13:24:25 crc kubenswrapper[5052]: I1203 13:24:25.130443 5052 reflector.go:368] Caches populated for *v1.Secret from object-"keystone-kuttl-tests"/"cert-keystone-public-svc" Dec 03 13:24:25 crc kubenswrapper[5052]: I1203 13:24:25.132062 5052 reflector.go:368] Caches populated for *v1.Secret from object-"keystone-kuttl-tests"/"keystone-scripts" Dec 03 13:24:25 crc kubenswrapper[5052]: I1203 13:24:25.139647 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["keystone-kuttl-tests/keystone-db495cbcd-dkqhp"] Dec 03 13:24:25 crc kubenswrapper[5052]: I1203 13:24:25.217775 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/eb6725e8-d540-42f8-b20e-0dbdf461c097-internal-tls-certs\") pod \"keystone-db495cbcd-dkqhp\" (UID: \"eb6725e8-d540-42f8-b20e-0dbdf461c097\") " pod="keystone-kuttl-tests/keystone-db495cbcd-dkqhp" Dec 03 13:24:25 crc kubenswrapper[5052]: I1203 13:24:25.217859 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qbjph\" (UniqueName: \"kubernetes.io/projected/eb6725e8-d540-42f8-b20e-0dbdf461c097-kube-api-access-qbjph\") pod \"keystone-db495cbcd-dkqhp\" (UID: \"eb6725e8-d540-42f8-b20e-0dbdf461c097\") " pod="keystone-kuttl-tests/keystone-db495cbcd-dkqhp" Dec 03 13:24:25 crc kubenswrapper[5052]: I1203 13:24:25.217904 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/eb6725e8-d540-42f8-b20e-0dbdf461c097-public-tls-certs\") pod \"keystone-db495cbcd-dkqhp\" (UID: \"eb6725e8-d540-42f8-b20e-0dbdf461c097\") " pod="keystone-kuttl-tests/keystone-db495cbcd-dkqhp" Dec 03 13:24:25 crc kubenswrapper[5052]: I1203 13:24:25.217953 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/eb6725e8-d540-42f8-b20e-0dbdf461c097-scripts\") pod \"keystone-db495cbcd-dkqhp\" (UID: \"eb6725e8-d540-42f8-b20e-0dbdf461c097\") " pod="keystone-kuttl-tests/keystone-db495cbcd-dkqhp" Dec 03 13:24:25 crc kubenswrapper[5052]: I1203 13:24:25.217990 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/eb6725e8-d540-42f8-b20e-0dbdf461c097-fernet-keys\") pod \"keystone-db495cbcd-dkqhp\" (UID: \"eb6725e8-d540-42f8-b20e-0dbdf461c097\") " pod="keystone-kuttl-tests/keystone-db495cbcd-dkqhp" Dec 03 13:24:25 crc kubenswrapper[5052]: I1203 13:24:25.218010 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/eb6725e8-d540-42f8-b20e-0dbdf461c097-credential-keys\") pod \"keystone-db495cbcd-dkqhp\" (UID: \"eb6725e8-d540-42f8-b20e-0dbdf461c097\") " pod="keystone-kuttl-tests/keystone-db495cbcd-dkqhp" Dec 03 13:24:25 crc kubenswrapper[5052]: I1203 13:24:25.218080 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/eb6725e8-d540-42f8-b20e-0dbdf461c097-combined-ca-bundle\") pod \"keystone-db495cbcd-dkqhp\" (UID: \"eb6725e8-d540-42f8-b20e-0dbdf461c097\") " pod="keystone-kuttl-tests/keystone-db495cbcd-dkqhp" Dec 03 13:24:25 crc kubenswrapper[5052]: I1203 13:24:25.218179 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/eb6725e8-d540-42f8-b20e-0dbdf461c097-config-data\") pod \"keystone-db495cbcd-dkqhp\" (UID: \"eb6725e8-d540-42f8-b20e-0dbdf461c097\") " pod="keystone-kuttl-tests/keystone-db495cbcd-dkqhp" Dec 03 13:24:25 crc kubenswrapper[5052]: I1203 13:24:25.319109 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/eb6725e8-d540-42f8-b20e-0dbdf461c097-combined-ca-bundle\") pod \"keystone-db495cbcd-dkqhp\" (UID: \"eb6725e8-d540-42f8-b20e-0dbdf461c097\") " pod="keystone-kuttl-tests/keystone-db495cbcd-dkqhp" Dec 03 13:24:25 crc kubenswrapper[5052]: I1203 13:24:25.319159 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/eb6725e8-d540-42f8-b20e-0dbdf461c097-config-data\") pod \"keystone-db495cbcd-dkqhp\" (UID: \"eb6725e8-d540-42f8-b20e-0dbdf461c097\") " pod="keystone-kuttl-tests/keystone-db495cbcd-dkqhp" Dec 03 13:24:25 crc kubenswrapper[5052]: I1203 13:24:25.319185 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/eb6725e8-d540-42f8-b20e-0dbdf461c097-internal-tls-certs\") pod \"keystone-db495cbcd-dkqhp\" (UID: \"eb6725e8-d540-42f8-b20e-0dbdf461c097\") " pod="keystone-kuttl-tests/keystone-db495cbcd-dkqhp" Dec 03 13:24:25 crc kubenswrapper[5052]: I1203 13:24:25.319212 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-qbjph\" (UniqueName: \"kubernetes.io/projected/eb6725e8-d540-42f8-b20e-0dbdf461c097-kube-api-access-qbjph\") pod \"keystone-db495cbcd-dkqhp\" (UID: \"eb6725e8-d540-42f8-b20e-0dbdf461c097\") " pod="keystone-kuttl-tests/keystone-db495cbcd-dkqhp" Dec 03 13:24:25 crc kubenswrapper[5052]: I1203 13:24:25.319229 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/eb6725e8-d540-42f8-b20e-0dbdf461c097-public-tls-certs\") pod \"keystone-db495cbcd-dkqhp\" (UID: \"eb6725e8-d540-42f8-b20e-0dbdf461c097\") " pod="keystone-kuttl-tests/keystone-db495cbcd-dkqhp" Dec 03 13:24:25 crc kubenswrapper[5052]: I1203 13:24:25.319264 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/eb6725e8-d540-42f8-b20e-0dbdf461c097-scripts\") pod \"keystone-db495cbcd-dkqhp\" (UID: \"eb6725e8-d540-42f8-b20e-0dbdf461c097\") " pod="keystone-kuttl-tests/keystone-db495cbcd-dkqhp" Dec 03 13:24:25 crc kubenswrapper[5052]: I1203 13:24:25.319315 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/eb6725e8-d540-42f8-b20e-0dbdf461c097-fernet-keys\") pod \"keystone-db495cbcd-dkqhp\" (UID: \"eb6725e8-d540-42f8-b20e-0dbdf461c097\") " pod="keystone-kuttl-tests/keystone-db495cbcd-dkqhp" Dec 03 13:24:25 crc kubenswrapper[5052]: I1203 13:24:25.319332 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/eb6725e8-d540-42f8-b20e-0dbdf461c097-credential-keys\") pod \"keystone-db495cbcd-dkqhp\" (UID: \"eb6725e8-d540-42f8-b20e-0dbdf461c097\") " pod="keystone-kuttl-tests/keystone-db495cbcd-dkqhp" Dec 03 13:24:25 crc kubenswrapper[5052]: I1203 13:24:25.325377 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/eb6725e8-d540-42f8-b20e-0dbdf461c097-scripts\") pod \"keystone-db495cbcd-dkqhp\" (UID: \"eb6725e8-d540-42f8-b20e-0dbdf461c097\") " pod="keystone-kuttl-tests/keystone-db495cbcd-dkqhp" Dec 03 13:24:25 crc kubenswrapper[5052]: I1203 13:24:25.325477 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/eb6725e8-d540-42f8-b20e-0dbdf461c097-combined-ca-bundle\") pod \"keystone-db495cbcd-dkqhp\" (UID: \"eb6725e8-d540-42f8-b20e-0dbdf461c097\") " pod="keystone-kuttl-tests/keystone-db495cbcd-dkqhp" Dec 03 13:24:25 crc kubenswrapper[5052]: I1203 13:24:25.328995 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/eb6725e8-d540-42f8-b20e-0dbdf461c097-public-tls-certs\") pod \"keystone-db495cbcd-dkqhp\" (UID: \"eb6725e8-d540-42f8-b20e-0dbdf461c097\") " pod="keystone-kuttl-tests/keystone-db495cbcd-dkqhp" Dec 03 13:24:25 crc kubenswrapper[5052]: I1203 13:24:25.329753 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/eb6725e8-d540-42f8-b20e-0dbdf461c097-internal-tls-certs\") pod \"keystone-db495cbcd-dkqhp\" (UID: \"eb6725e8-d540-42f8-b20e-0dbdf461c097\") " pod="keystone-kuttl-tests/keystone-db495cbcd-dkqhp" Dec 03 13:24:25 crc kubenswrapper[5052]: I1203 13:24:25.330131 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/eb6725e8-d540-42f8-b20e-0dbdf461c097-credential-keys\") pod \"keystone-db495cbcd-dkqhp\" (UID: \"eb6725e8-d540-42f8-b20e-0dbdf461c097\") " pod="keystone-kuttl-tests/keystone-db495cbcd-dkqhp" Dec 03 13:24:25 crc kubenswrapper[5052]: I1203 13:24:25.333068 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/eb6725e8-d540-42f8-b20e-0dbdf461c097-config-data\") pod \"keystone-db495cbcd-dkqhp\" (UID: \"eb6725e8-d540-42f8-b20e-0dbdf461c097\") " pod="keystone-kuttl-tests/keystone-db495cbcd-dkqhp" Dec 03 13:24:25 crc kubenswrapper[5052]: I1203 13:24:25.337053 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/eb6725e8-d540-42f8-b20e-0dbdf461c097-fernet-keys\") pod \"keystone-db495cbcd-dkqhp\" (UID: \"eb6725e8-d540-42f8-b20e-0dbdf461c097\") " pod="keystone-kuttl-tests/keystone-db495cbcd-dkqhp" Dec 03 13:24:25 crc kubenswrapper[5052]: I1203 13:24:25.343662 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-qbjph\" (UniqueName: \"kubernetes.io/projected/eb6725e8-d540-42f8-b20e-0dbdf461c097-kube-api-access-qbjph\") pod \"keystone-db495cbcd-dkqhp\" (UID: \"eb6725e8-d540-42f8-b20e-0dbdf461c097\") " pod="keystone-kuttl-tests/keystone-db495cbcd-dkqhp" Dec 03 13:24:25 crc kubenswrapper[5052]: I1203 13:24:25.445165 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="keystone-kuttl-tests/keystone-db495cbcd-dkqhp" Dec 03 13:24:26 crc kubenswrapper[5052]: I1203 13:24:26.089687 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["keystone-kuttl-tests/keystone-db495cbcd-dkqhp"] Dec 03 13:24:26 crc kubenswrapper[5052]: I1203 13:24:26.952710 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="keystone-kuttl-tests/keystone-db495cbcd-dkqhp" event={"ID":"eb6725e8-d540-42f8-b20e-0dbdf461c097","Type":"ContainerStarted","Data":"3cbdf79fe420cf9422c3d88ccf18cff6578d9314bb2d2bad0e5920d53509dcd9"} Dec 03 13:24:26 crc kubenswrapper[5052]: I1203 13:24:26.953089 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="keystone-kuttl-tests/keystone-db495cbcd-dkqhp" event={"ID":"eb6725e8-d540-42f8-b20e-0dbdf461c097","Type":"ContainerStarted","Data":"8c5f683a2bb43dff524ad5963bd9cf8f8cd7fec73d2259684b52ce1d4afc8c6b"} Dec 03 13:24:26 crc kubenswrapper[5052]: I1203 13:24:26.953134 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="keystone-kuttl-tests/keystone-db495cbcd-dkqhp" Dec 03 13:24:27 crc kubenswrapper[5052]: I1203 13:24:27.049416 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="keystone-kuttl-tests/keystone-db495cbcd-dkqhp" podStartSLOduration=2.049396061 podStartE2EDuration="2.049396061s" podCreationTimestamp="2025-12-03 13:24:25 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 13:24:27.044961994 +0000 UTC m=+7457.633203768" watchObservedRunningTime="2025-12-03 13:24:27.049396061 +0000 UTC m=+7457.637637835" Dec 03 13:24:27 crc kubenswrapper[5052]: I1203 13:24:27.119874 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openshift-marketplace/redhat-operators-5scnq"] Dec 03 13:24:27 crc kubenswrapper[5052]: I1203 13:24:27.121288 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/redhat-operators-5scnq" Dec 03 13:24:27 crc kubenswrapper[5052]: I1203 13:24:27.149575 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/redhat-operators-5scnq"] Dec 03 13:24:27 crc kubenswrapper[5052]: I1203 13:24:27.269616 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/b0f8697a-378c-47e5-a2f6-20b921c21b3b-catalog-content\") pod \"redhat-operators-5scnq\" (UID: \"b0f8697a-378c-47e5-a2f6-20b921c21b3b\") " pod="openshift-marketplace/redhat-operators-5scnq" Dec 03 13:24:27 crc kubenswrapper[5052]: I1203 13:24:27.269763 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/b0f8697a-378c-47e5-a2f6-20b921c21b3b-utilities\") pod \"redhat-operators-5scnq\" (UID: \"b0f8697a-378c-47e5-a2f6-20b921c21b3b\") " pod="openshift-marketplace/redhat-operators-5scnq" Dec 03 13:24:27 crc kubenswrapper[5052]: I1203 13:24:27.269792 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8rdwv\" (UniqueName: \"kubernetes.io/projected/b0f8697a-378c-47e5-a2f6-20b921c21b3b-kube-api-access-8rdwv\") pod \"redhat-operators-5scnq\" (UID: \"b0f8697a-378c-47e5-a2f6-20b921c21b3b\") " pod="openshift-marketplace/redhat-operators-5scnq" Dec 03 13:24:27 crc kubenswrapper[5052]: I1203 13:24:27.371189 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/b0f8697a-378c-47e5-a2f6-20b921c21b3b-utilities\") pod \"redhat-operators-5scnq\" (UID: \"b0f8697a-378c-47e5-a2f6-20b921c21b3b\") " pod="openshift-marketplace/redhat-operators-5scnq" Dec 03 13:24:27 crc kubenswrapper[5052]: I1203 13:24:27.371247 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-8rdwv\" (UniqueName: \"kubernetes.io/projected/b0f8697a-378c-47e5-a2f6-20b921c21b3b-kube-api-access-8rdwv\") pod \"redhat-operators-5scnq\" (UID: \"b0f8697a-378c-47e5-a2f6-20b921c21b3b\") " pod="openshift-marketplace/redhat-operators-5scnq" Dec 03 13:24:27 crc kubenswrapper[5052]: I1203 13:24:27.371323 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/b0f8697a-378c-47e5-a2f6-20b921c21b3b-catalog-content\") pod \"redhat-operators-5scnq\" (UID: \"b0f8697a-378c-47e5-a2f6-20b921c21b3b\") " pod="openshift-marketplace/redhat-operators-5scnq" Dec 03 13:24:27 crc kubenswrapper[5052]: I1203 13:24:27.371715 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/b0f8697a-378c-47e5-a2f6-20b921c21b3b-utilities\") pod \"redhat-operators-5scnq\" (UID: \"b0f8697a-378c-47e5-a2f6-20b921c21b3b\") " pod="openshift-marketplace/redhat-operators-5scnq" Dec 03 13:24:27 crc kubenswrapper[5052]: I1203 13:24:27.371817 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/b0f8697a-378c-47e5-a2f6-20b921c21b3b-catalog-content\") pod \"redhat-operators-5scnq\" (UID: \"b0f8697a-378c-47e5-a2f6-20b921c21b3b\") " pod="openshift-marketplace/redhat-operators-5scnq" Dec 03 13:24:27 crc kubenswrapper[5052]: I1203 13:24:27.392745 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-8rdwv\" (UniqueName: \"kubernetes.io/projected/b0f8697a-378c-47e5-a2f6-20b921c21b3b-kube-api-access-8rdwv\") pod \"redhat-operators-5scnq\" (UID: \"b0f8697a-378c-47e5-a2f6-20b921c21b3b\") " pod="openshift-marketplace/redhat-operators-5scnq" Dec 03 13:24:27 crc kubenswrapper[5052]: I1203 13:24:27.440829 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/redhat-operators-5scnq" Dec 03 13:24:28 crc kubenswrapper[5052]: I1203 13:24:28.112052 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/redhat-operators-5scnq"] Dec 03 13:24:28 crc kubenswrapper[5052]: W1203 13:24:28.117640 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb0f8697a_378c_47e5_a2f6_20b921c21b3b.slice/crio-305ee04e85c5a5eb32e13eee260eed003a4b70df0185aaea1c381692c3077a16 WatchSource:0}: Error finding container 305ee04e85c5a5eb32e13eee260eed003a4b70df0185aaea1c381692c3077a16: Status 404 returned error can't find the container with id 305ee04e85c5a5eb32e13eee260eed003a4b70df0185aaea1c381692c3077a16 Dec 03 13:24:28 crc kubenswrapper[5052]: I1203 13:24:28.967489 5052 generic.go:334] "Generic (PLEG): container finished" podID="b0f8697a-378c-47e5-a2f6-20b921c21b3b" containerID="2ad51961e603a113221071a7d411818fd16e630e094b1802d4650e2f9eac6c7b" exitCode=0 Dec 03 13:24:28 crc kubenswrapper[5052]: I1203 13:24:28.967561 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-operators-5scnq" event={"ID":"b0f8697a-378c-47e5-a2f6-20b921c21b3b","Type":"ContainerDied","Data":"2ad51961e603a113221071a7d411818fd16e630e094b1802d4650e2f9eac6c7b"} Dec 03 13:24:28 crc kubenswrapper[5052]: I1203 13:24:28.967873 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-operators-5scnq" event={"ID":"b0f8697a-378c-47e5-a2f6-20b921c21b3b","Type":"ContainerStarted","Data":"305ee04e85c5a5eb32e13eee260eed003a4b70df0185aaea1c381692c3077a16"} Dec 03 13:24:30 crc kubenswrapper[5052]: I1203 13:24:30.989208 5052 generic.go:334] "Generic (PLEG): container finished" podID="b0f8697a-378c-47e5-a2f6-20b921c21b3b" containerID="dca86579f240d6df55194364f49e46a0f57833bc375c37a4e7d51c0036706d02" exitCode=0 Dec 03 13:24:30 crc kubenswrapper[5052]: I1203 13:24:30.989342 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-operators-5scnq" event={"ID":"b0f8697a-378c-47e5-a2f6-20b921c21b3b","Type":"ContainerDied","Data":"dca86579f240d6df55194364f49e46a0f57833bc375c37a4e7d51c0036706d02"} Dec 03 13:24:32 crc kubenswrapper[5052]: I1203 13:24:31.999962 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-operators-5scnq" event={"ID":"b0f8697a-378c-47e5-a2f6-20b921c21b3b","Type":"ContainerStarted","Data":"fb8aa2466ecceca7b6bc087cf6e4afc757f21fd13022a9c01ff8d7434d34dcb8"} Dec 03 13:24:32 crc kubenswrapper[5052]: I1203 13:24:32.028273 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-marketplace/redhat-operators-5scnq" podStartSLOduration=2.499925736 podStartE2EDuration="5.028250942s" podCreationTimestamp="2025-12-03 13:24:27 +0000 UTC" firstStartedPulling="2025-12-03 13:24:28.968738385 +0000 UTC m=+7459.556980149" lastFinishedPulling="2025-12-03 13:24:31.497063591 +0000 UTC m=+7462.085305355" observedRunningTime="2025-12-03 13:24:32.019879803 +0000 UTC m=+7462.608121577" watchObservedRunningTime="2025-12-03 13:24:32.028250942 +0000 UTC m=+7462.616492706" Dec 03 13:24:37 crc kubenswrapper[5052]: I1203 13:24:37.441542 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-marketplace/redhat-operators-5scnq" Dec 03 13:24:37 crc kubenswrapper[5052]: I1203 13:24:37.441890 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openshift-marketplace/redhat-operators-5scnq" Dec 03 13:24:37 crc kubenswrapper[5052]: I1203 13:24:37.492657 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-marketplace/redhat-operators-5scnq" Dec 03 13:24:38 crc kubenswrapper[5052]: I1203 13:24:38.102479 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-marketplace/redhat-operators-5scnq" Dec 03 13:24:38 crc kubenswrapper[5052]: I1203 13:24:38.159438 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openshift-marketplace/redhat-operators-5scnq"] Dec 03 13:24:40 crc kubenswrapper[5052]: I1203 13:24:40.062948 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openshift-marketplace/redhat-operators-5scnq" podUID="b0f8697a-378c-47e5-a2f6-20b921c21b3b" containerName="registry-server" containerID="cri-o://fb8aa2466ecceca7b6bc087cf6e4afc757f21fd13022a9c01ff8d7434d34dcb8" gracePeriod=2 Dec 03 13:24:40 crc kubenswrapper[5052]: I1203 13:24:40.581138 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/redhat-operators-5scnq" Dec 03 13:24:40 crc kubenswrapper[5052]: I1203 13:24:40.778058 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/b0f8697a-378c-47e5-a2f6-20b921c21b3b-catalog-content\") pod \"b0f8697a-378c-47e5-a2f6-20b921c21b3b\" (UID: \"b0f8697a-378c-47e5-a2f6-20b921c21b3b\") " Dec 03 13:24:40 crc kubenswrapper[5052]: I1203 13:24:40.778239 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/b0f8697a-378c-47e5-a2f6-20b921c21b3b-utilities\") pod \"b0f8697a-378c-47e5-a2f6-20b921c21b3b\" (UID: \"b0f8697a-378c-47e5-a2f6-20b921c21b3b\") " Dec 03 13:24:40 crc kubenswrapper[5052]: I1203 13:24:40.778370 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8rdwv\" (UniqueName: \"kubernetes.io/projected/b0f8697a-378c-47e5-a2f6-20b921c21b3b-kube-api-access-8rdwv\") pod \"b0f8697a-378c-47e5-a2f6-20b921c21b3b\" (UID: \"b0f8697a-378c-47e5-a2f6-20b921c21b3b\") " Dec 03 13:24:40 crc kubenswrapper[5052]: I1203 13:24:40.779106 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b0f8697a-378c-47e5-a2f6-20b921c21b3b-utilities" (OuterVolumeSpecName: "utilities") pod "b0f8697a-378c-47e5-a2f6-20b921c21b3b" (UID: "b0f8697a-378c-47e5-a2f6-20b921c21b3b"). InnerVolumeSpecName "utilities". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 13:24:40 crc kubenswrapper[5052]: I1203 13:24:40.787697 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b0f8697a-378c-47e5-a2f6-20b921c21b3b-kube-api-access-8rdwv" (OuterVolumeSpecName: "kube-api-access-8rdwv") pod "b0f8697a-378c-47e5-a2f6-20b921c21b3b" (UID: "b0f8697a-378c-47e5-a2f6-20b921c21b3b"). InnerVolumeSpecName "kube-api-access-8rdwv". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:24:40 crc kubenswrapper[5052]: I1203 13:24:40.879986 5052 reconciler_common.go:293] "Volume detached for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/b0f8697a-378c-47e5-a2f6-20b921c21b3b-utilities\") on node \"crc\" DevicePath \"\"" Dec 03 13:24:40 crc kubenswrapper[5052]: I1203 13:24:40.880035 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-8rdwv\" (UniqueName: \"kubernetes.io/projected/b0f8697a-378c-47e5-a2f6-20b921c21b3b-kube-api-access-8rdwv\") on node \"crc\" DevicePath \"\"" Dec 03 13:24:41 crc kubenswrapper[5052]: I1203 13:24:41.073433 5052 generic.go:334] "Generic (PLEG): container finished" podID="b0f8697a-378c-47e5-a2f6-20b921c21b3b" containerID="fb8aa2466ecceca7b6bc087cf6e4afc757f21fd13022a9c01ff8d7434d34dcb8" exitCode=0 Dec 03 13:24:41 crc kubenswrapper[5052]: I1203 13:24:41.073477 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-operators-5scnq" event={"ID":"b0f8697a-378c-47e5-a2f6-20b921c21b3b","Type":"ContainerDied","Data":"fb8aa2466ecceca7b6bc087cf6e4afc757f21fd13022a9c01ff8d7434d34dcb8"} Dec 03 13:24:41 crc kubenswrapper[5052]: I1203 13:24:41.073507 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-operators-5scnq" event={"ID":"b0f8697a-378c-47e5-a2f6-20b921c21b3b","Type":"ContainerDied","Data":"305ee04e85c5a5eb32e13eee260eed003a4b70df0185aaea1c381692c3077a16"} Dec 03 13:24:41 crc kubenswrapper[5052]: I1203 13:24:41.073563 5052 scope.go:117] "RemoveContainer" containerID="fb8aa2466ecceca7b6bc087cf6e4afc757f21fd13022a9c01ff8d7434d34dcb8" Dec 03 13:24:41 crc kubenswrapper[5052]: I1203 13:24:41.073735 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/redhat-operators-5scnq" Dec 03 13:24:41 crc kubenswrapper[5052]: I1203 13:24:41.092018 5052 scope.go:117] "RemoveContainer" containerID="dca86579f240d6df55194364f49e46a0f57833bc375c37a4e7d51c0036706d02" Dec 03 13:24:41 crc kubenswrapper[5052]: I1203 13:24:41.110820 5052 scope.go:117] "RemoveContainer" containerID="2ad51961e603a113221071a7d411818fd16e630e094b1802d4650e2f9eac6c7b" Dec 03 13:24:41 crc kubenswrapper[5052]: I1203 13:24:41.143444 5052 scope.go:117] "RemoveContainer" containerID="fb8aa2466ecceca7b6bc087cf6e4afc757f21fd13022a9c01ff8d7434d34dcb8" Dec 03 13:24:41 crc kubenswrapper[5052]: E1203 13:24:41.144049 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fb8aa2466ecceca7b6bc087cf6e4afc757f21fd13022a9c01ff8d7434d34dcb8\": container with ID starting with fb8aa2466ecceca7b6bc087cf6e4afc757f21fd13022a9c01ff8d7434d34dcb8 not found: ID does not exist" containerID="fb8aa2466ecceca7b6bc087cf6e4afc757f21fd13022a9c01ff8d7434d34dcb8" Dec 03 13:24:41 crc kubenswrapper[5052]: I1203 13:24:41.144115 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fb8aa2466ecceca7b6bc087cf6e4afc757f21fd13022a9c01ff8d7434d34dcb8"} err="failed to get container status \"fb8aa2466ecceca7b6bc087cf6e4afc757f21fd13022a9c01ff8d7434d34dcb8\": rpc error: code = NotFound desc = could not find container \"fb8aa2466ecceca7b6bc087cf6e4afc757f21fd13022a9c01ff8d7434d34dcb8\": container with ID starting with fb8aa2466ecceca7b6bc087cf6e4afc757f21fd13022a9c01ff8d7434d34dcb8 not found: ID does not exist" Dec 03 13:24:41 crc kubenswrapper[5052]: I1203 13:24:41.144150 5052 scope.go:117] "RemoveContainer" containerID="dca86579f240d6df55194364f49e46a0f57833bc375c37a4e7d51c0036706d02" Dec 03 13:24:41 crc kubenswrapper[5052]: E1203 13:24:41.144648 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dca86579f240d6df55194364f49e46a0f57833bc375c37a4e7d51c0036706d02\": container with ID starting with dca86579f240d6df55194364f49e46a0f57833bc375c37a4e7d51c0036706d02 not found: ID does not exist" containerID="dca86579f240d6df55194364f49e46a0f57833bc375c37a4e7d51c0036706d02" Dec 03 13:24:41 crc kubenswrapper[5052]: I1203 13:24:41.144698 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dca86579f240d6df55194364f49e46a0f57833bc375c37a4e7d51c0036706d02"} err="failed to get container status \"dca86579f240d6df55194364f49e46a0f57833bc375c37a4e7d51c0036706d02\": rpc error: code = NotFound desc = could not find container \"dca86579f240d6df55194364f49e46a0f57833bc375c37a4e7d51c0036706d02\": container with ID starting with dca86579f240d6df55194364f49e46a0f57833bc375c37a4e7d51c0036706d02 not found: ID does not exist" Dec 03 13:24:41 crc kubenswrapper[5052]: I1203 13:24:41.144729 5052 scope.go:117] "RemoveContainer" containerID="2ad51961e603a113221071a7d411818fd16e630e094b1802d4650e2f9eac6c7b" Dec 03 13:24:41 crc kubenswrapper[5052]: E1203 13:24:41.145134 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2ad51961e603a113221071a7d411818fd16e630e094b1802d4650e2f9eac6c7b\": container with ID starting with 2ad51961e603a113221071a7d411818fd16e630e094b1802d4650e2f9eac6c7b not found: ID does not exist" containerID="2ad51961e603a113221071a7d411818fd16e630e094b1802d4650e2f9eac6c7b" Dec 03 13:24:41 crc kubenswrapper[5052]: I1203 13:24:41.145159 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2ad51961e603a113221071a7d411818fd16e630e094b1802d4650e2f9eac6c7b"} err="failed to get container status \"2ad51961e603a113221071a7d411818fd16e630e094b1802d4650e2f9eac6c7b\": rpc error: code = NotFound desc = could not find container \"2ad51961e603a113221071a7d411818fd16e630e094b1802d4650e2f9eac6c7b\": container with ID starting with 2ad51961e603a113221071a7d411818fd16e630e094b1802d4650e2f9eac6c7b not found: ID does not exist" Dec 03 13:24:42 crc kubenswrapper[5052]: I1203 13:24:42.187037 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b0f8697a-378c-47e5-a2f6-20b921c21b3b-catalog-content" (OuterVolumeSpecName: "catalog-content") pod "b0f8697a-378c-47e5-a2f6-20b921c21b3b" (UID: "b0f8697a-378c-47e5-a2f6-20b921c21b3b"). InnerVolumeSpecName "catalog-content". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 13:24:42 crc kubenswrapper[5052]: I1203 13:24:42.197674 5052 reconciler_common.go:293] "Volume detached for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/b0f8697a-378c-47e5-a2f6-20b921c21b3b-catalog-content\") on node \"crc\" DevicePath \"\"" Dec 03 13:24:42 crc kubenswrapper[5052]: I1203 13:24:42.309923 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openshift-marketplace/redhat-operators-5scnq"] Dec 03 13:24:42 crc kubenswrapper[5052]: I1203 13:24:42.317105 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openshift-marketplace/redhat-operators-5scnq"] Dec 03 13:24:43 crc kubenswrapper[5052]: I1203 13:24:43.683272 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b0f8697a-378c-47e5-a2f6-20b921c21b3b" path="/var/lib/kubelet/pods/b0f8697a-378c-47e5-a2f6-20b921c21b3b/volumes" Dec 03 13:24:47 crc kubenswrapper[5052]: I1203 13:24:47.844823 5052 scope.go:117] "RemoveContainer" containerID="d507079afcc0267ce43d476785ec8f34ff653c173761201507cb26592443c0fd" Dec 03 13:24:47 crc kubenswrapper[5052]: I1203 13:24:47.880727 5052 scope.go:117] "RemoveContainer" containerID="c84ad400a7a634dd4ee8e626c400c2c08de9f53fbccffb6eb453f6b48b379e9a" Dec 03 13:24:47 crc kubenswrapper[5052]: I1203 13:24:47.901276 5052 scope.go:117] "RemoveContainer" containerID="72266a7bb1fe568dab23c67dcac94727a3bd4a60e5dd4652c1b945ef1e716bc2" Dec 03 13:24:47 crc kubenswrapper[5052]: I1203 13:24:47.929753 5052 scope.go:117] "RemoveContainer" containerID="de9f481ae76794b2e2c23743632c15a110f79b63254a4044d0999c1775b41e20" Dec 03 13:24:47 crc kubenswrapper[5052]: I1203 13:24:47.965282 5052 scope.go:117] "RemoveContainer" containerID="6b9bfacd4508cedeeb65c806b3b42b4c837645204ac98fec3e0d29a3ee0f2971" Dec 03 13:24:47 crc kubenswrapper[5052]: I1203 13:24:47.990021 5052 scope.go:117] "RemoveContainer" containerID="ce9c06b6087aafd38af61e1b86904aad0dbe372e78c8e94652ef1add52ada691" Dec 03 13:24:57 crc kubenswrapper[5052]: I1203 13:24:57.080641 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="keystone-kuttl-tests/keystone-db495cbcd-dkqhp" Dec 03 13:24:58 crc kubenswrapper[5052]: I1203 13:24:58.332601 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["keystone-kuttl-tests/keystone-bootstrap-mq5st"] Dec 03 13:24:58 crc kubenswrapper[5052]: I1203 13:24:58.346972 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["keystone-kuttl-tests/keystone-db-sync-2jjgq"] Dec 03 13:24:58 crc kubenswrapper[5052]: I1203 13:24:58.352683 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["keystone-kuttl-tests/keystone-db-sync-2jjgq"] Dec 03 13:24:58 crc kubenswrapper[5052]: I1203 13:24:58.361569 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["keystone-kuttl-tests/keystone-bootstrap-mq5st"] Dec 03 13:24:58 crc kubenswrapper[5052]: I1203 13:24:58.379203 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["keystone-kuttl-tests/keystone-db495cbcd-dkqhp"] Dec 03 13:24:58 crc kubenswrapper[5052]: I1203 13:24:58.379425 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="keystone-kuttl-tests/keystone-db495cbcd-dkqhp" podUID="eb6725e8-d540-42f8-b20e-0dbdf461c097" containerName="keystone-api" containerID="cri-o://3cbdf79fe420cf9422c3d88ccf18cff6578d9314bb2d2bad0e5920d53509dcd9" gracePeriod=30 Dec 03 13:24:58 crc kubenswrapper[5052]: I1203 13:24:58.425301 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["keystone-kuttl-tests/keystone3828-account-delete-57qhp"] Dec 03 13:24:58 crc kubenswrapper[5052]: E1203 13:24:58.425713 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="b0f8697a-378c-47e5-a2f6-20b921c21b3b" containerName="extract-utilities" Dec 03 13:24:58 crc kubenswrapper[5052]: I1203 13:24:58.425735 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="b0f8697a-378c-47e5-a2f6-20b921c21b3b" containerName="extract-utilities" Dec 03 13:24:58 crc kubenswrapper[5052]: E1203 13:24:58.425754 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="b0f8697a-378c-47e5-a2f6-20b921c21b3b" containerName="registry-server" Dec 03 13:24:58 crc kubenswrapper[5052]: I1203 13:24:58.425761 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="b0f8697a-378c-47e5-a2f6-20b921c21b3b" containerName="registry-server" Dec 03 13:24:58 crc kubenswrapper[5052]: E1203 13:24:58.425793 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="b0f8697a-378c-47e5-a2f6-20b921c21b3b" containerName="extract-content" Dec 03 13:24:58 crc kubenswrapper[5052]: I1203 13:24:58.425799 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="b0f8697a-378c-47e5-a2f6-20b921c21b3b" containerName="extract-content" Dec 03 13:24:58 crc kubenswrapper[5052]: I1203 13:24:58.425969 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="b0f8697a-378c-47e5-a2f6-20b921c21b3b" containerName="registry-server" Dec 03 13:24:58 crc kubenswrapper[5052]: I1203 13:24:58.426626 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="keystone-kuttl-tests/keystone3828-account-delete-57qhp" Dec 03 13:24:58 crc kubenswrapper[5052]: I1203 13:24:58.435080 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["keystone-kuttl-tests/keystone3828-account-delete-57qhp"] Dec 03 13:24:58 crc kubenswrapper[5052]: I1203 13:24:58.531418 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cqkgg\" (UniqueName: \"kubernetes.io/projected/8fb5a2bb-5a02-487b-a695-d9bd88401a42-kube-api-access-cqkgg\") pod \"keystone3828-account-delete-57qhp\" (UID: \"8fb5a2bb-5a02-487b-a695-d9bd88401a42\") " pod="keystone-kuttl-tests/keystone3828-account-delete-57qhp" Dec 03 13:24:58 crc kubenswrapper[5052]: I1203 13:24:58.531477 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/8fb5a2bb-5a02-487b-a695-d9bd88401a42-operator-scripts\") pod \"keystone3828-account-delete-57qhp\" (UID: \"8fb5a2bb-5a02-487b-a695-d9bd88401a42\") " pod="keystone-kuttl-tests/keystone3828-account-delete-57qhp" Dec 03 13:24:58 crc kubenswrapper[5052]: I1203 13:24:58.633051 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-cqkgg\" (UniqueName: \"kubernetes.io/projected/8fb5a2bb-5a02-487b-a695-d9bd88401a42-kube-api-access-cqkgg\") pod \"keystone3828-account-delete-57qhp\" (UID: \"8fb5a2bb-5a02-487b-a695-d9bd88401a42\") " pod="keystone-kuttl-tests/keystone3828-account-delete-57qhp" Dec 03 13:24:58 crc kubenswrapper[5052]: I1203 13:24:58.633166 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/8fb5a2bb-5a02-487b-a695-d9bd88401a42-operator-scripts\") pod \"keystone3828-account-delete-57qhp\" (UID: \"8fb5a2bb-5a02-487b-a695-d9bd88401a42\") " pod="keystone-kuttl-tests/keystone3828-account-delete-57qhp" Dec 03 13:24:58 crc kubenswrapper[5052]: I1203 13:24:58.633956 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/8fb5a2bb-5a02-487b-a695-d9bd88401a42-operator-scripts\") pod \"keystone3828-account-delete-57qhp\" (UID: \"8fb5a2bb-5a02-487b-a695-d9bd88401a42\") " pod="keystone-kuttl-tests/keystone3828-account-delete-57qhp" Dec 03 13:24:58 crc kubenswrapper[5052]: I1203 13:24:58.654345 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-cqkgg\" (UniqueName: \"kubernetes.io/projected/8fb5a2bb-5a02-487b-a695-d9bd88401a42-kube-api-access-cqkgg\") pod \"keystone3828-account-delete-57qhp\" (UID: \"8fb5a2bb-5a02-487b-a695-d9bd88401a42\") " pod="keystone-kuttl-tests/keystone3828-account-delete-57qhp" Dec 03 13:24:58 crc kubenswrapper[5052]: I1203 13:24:58.748585 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="keystone-kuttl-tests/keystone3828-account-delete-57qhp" Dec 03 13:24:59 crc kubenswrapper[5052]: I1203 13:24:59.181564 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["keystone-kuttl-tests/keystone3828-account-delete-57qhp"] Dec 03 13:24:59 crc kubenswrapper[5052]: I1203 13:24:59.211121 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="keystone-kuttl-tests/keystone3828-account-delete-57qhp" event={"ID":"8fb5a2bb-5a02-487b-a695-d9bd88401a42","Type":"ContainerStarted","Data":"e317cacb055c02f56b9590fff65fbc9da0929abf2763753ed615217e2414989f"} Dec 03 13:24:59 crc kubenswrapper[5052]: I1203 13:24:59.683588 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5742d593-c183-48fa-b69f-e074cfb3ebaa" path="/var/lib/kubelet/pods/5742d593-c183-48fa-b69f-e074cfb3ebaa/volumes" Dec 03 13:24:59 crc kubenswrapper[5052]: I1203 13:24:59.684245 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b72867db-9193-4ca4-9829-253609425890" path="/var/lib/kubelet/pods/b72867db-9193-4ca4-9829-253609425890/volumes" Dec 03 13:25:00 crc kubenswrapper[5052]: I1203 13:25:00.218996 5052 generic.go:334] "Generic (PLEG): container finished" podID="8fb5a2bb-5a02-487b-a695-d9bd88401a42" containerID="f91deacc7890c7e8ae6b0f49bb0ee18463db2e86e796436b3082d2a76adb0308" exitCode=0 Dec 03 13:25:00 crc kubenswrapper[5052]: I1203 13:25:00.219062 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="keystone-kuttl-tests/keystone3828-account-delete-57qhp" event={"ID":"8fb5a2bb-5a02-487b-a695-d9bd88401a42","Type":"ContainerDied","Data":"f91deacc7890c7e8ae6b0f49bb0ee18463db2e86e796436b3082d2a76adb0308"} Dec 03 13:25:01 crc kubenswrapper[5052]: I1203 13:25:01.714644 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="keystone-kuttl-tests/keystone3828-account-delete-57qhp" Dec 03 13:25:01 crc kubenswrapper[5052]: I1203 13:25:01.886889 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cqkgg\" (UniqueName: \"kubernetes.io/projected/8fb5a2bb-5a02-487b-a695-d9bd88401a42-kube-api-access-cqkgg\") pod \"8fb5a2bb-5a02-487b-a695-d9bd88401a42\" (UID: \"8fb5a2bb-5a02-487b-a695-d9bd88401a42\") " Dec 03 13:25:01 crc kubenswrapper[5052]: I1203 13:25:01.887046 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/8fb5a2bb-5a02-487b-a695-d9bd88401a42-operator-scripts\") pod \"8fb5a2bb-5a02-487b-a695-d9bd88401a42\" (UID: \"8fb5a2bb-5a02-487b-a695-d9bd88401a42\") " Dec 03 13:25:01 crc kubenswrapper[5052]: I1203 13:25:01.889043 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8fb5a2bb-5a02-487b-a695-d9bd88401a42-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "8fb5a2bb-5a02-487b-a695-d9bd88401a42" (UID: "8fb5a2bb-5a02-487b-a695-d9bd88401a42"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:25:01 crc kubenswrapper[5052]: I1203 13:25:01.900761 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8fb5a2bb-5a02-487b-a695-d9bd88401a42-kube-api-access-cqkgg" (OuterVolumeSpecName: "kube-api-access-cqkgg") pod "8fb5a2bb-5a02-487b-a695-d9bd88401a42" (UID: "8fb5a2bb-5a02-487b-a695-d9bd88401a42"). InnerVolumeSpecName "kube-api-access-cqkgg". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:25:01 crc kubenswrapper[5052]: I1203 13:25:01.969564 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="keystone-kuttl-tests/keystone-db495cbcd-dkqhp" Dec 03 13:25:01 crc kubenswrapper[5052]: I1203 13:25:01.989876 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-cqkgg\" (UniqueName: \"kubernetes.io/projected/8fb5a2bb-5a02-487b-a695-d9bd88401a42-kube-api-access-cqkgg\") on node \"crc\" DevicePath \"\"" Dec 03 13:25:01 crc kubenswrapper[5052]: I1203 13:25:01.989920 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/8fb5a2bb-5a02-487b-a695-d9bd88401a42-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 13:25:02 crc kubenswrapper[5052]: I1203 13:25:02.091497 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qbjph\" (UniqueName: \"kubernetes.io/projected/eb6725e8-d540-42f8-b20e-0dbdf461c097-kube-api-access-qbjph\") pod \"eb6725e8-d540-42f8-b20e-0dbdf461c097\" (UID: \"eb6725e8-d540-42f8-b20e-0dbdf461c097\") " Dec 03 13:25:02 crc kubenswrapper[5052]: I1203 13:25:02.091691 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/eb6725e8-d540-42f8-b20e-0dbdf461c097-fernet-keys\") pod \"eb6725e8-d540-42f8-b20e-0dbdf461c097\" (UID: \"eb6725e8-d540-42f8-b20e-0dbdf461c097\") " Dec 03 13:25:02 crc kubenswrapper[5052]: I1203 13:25:02.091743 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/eb6725e8-d540-42f8-b20e-0dbdf461c097-scripts\") pod \"eb6725e8-d540-42f8-b20e-0dbdf461c097\" (UID: \"eb6725e8-d540-42f8-b20e-0dbdf461c097\") " Dec 03 13:25:02 crc kubenswrapper[5052]: I1203 13:25:02.091805 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/eb6725e8-d540-42f8-b20e-0dbdf461c097-public-tls-certs\") pod \"eb6725e8-d540-42f8-b20e-0dbdf461c097\" (UID: \"eb6725e8-d540-42f8-b20e-0dbdf461c097\") " Dec 03 13:25:02 crc kubenswrapper[5052]: I1203 13:25:02.091890 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/eb6725e8-d540-42f8-b20e-0dbdf461c097-combined-ca-bundle\") pod \"eb6725e8-d540-42f8-b20e-0dbdf461c097\" (UID: \"eb6725e8-d540-42f8-b20e-0dbdf461c097\") " Dec 03 13:25:02 crc kubenswrapper[5052]: I1203 13:25:02.091924 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/eb6725e8-d540-42f8-b20e-0dbdf461c097-internal-tls-certs\") pod \"eb6725e8-d540-42f8-b20e-0dbdf461c097\" (UID: \"eb6725e8-d540-42f8-b20e-0dbdf461c097\") " Dec 03 13:25:02 crc kubenswrapper[5052]: I1203 13:25:02.091965 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/eb6725e8-d540-42f8-b20e-0dbdf461c097-credential-keys\") pod \"eb6725e8-d540-42f8-b20e-0dbdf461c097\" (UID: \"eb6725e8-d540-42f8-b20e-0dbdf461c097\") " Dec 03 13:25:02 crc kubenswrapper[5052]: I1203 13:25:02.092012 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/eb6725e8-d540-42f8-b20e-0dbdf461c097-config-data\") pod \"eb6725e8-d540-42f8-b20e-0dbdf461c097\" (UID: \"eb6725e8-d540-42f8-b20e-0dbdf461c097\") " Dec 03 13:25:02 crc kubenswrapper[5052]: I1203 13:25:02.094858 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eb6725e8-d540-42f8-b20e-0dbdf461c097-scripts" (OuterVolumeSpecName: "scripts") pod "eb6725e8-d540-42f8-b20e-0dbdf461c097" (UID: "eb6725e8-d540-42f8-b20e-0dbdf461c097"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:25:02 crc kubenswrapper[5052]: I1203 13:25:02.094924 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eb6725e8-d540-42f8-b20e-0dbdf461c097-kube-api-access-qbjph" (OuterVolumeSpecName: "kube-api-access-qbjph") pod "eb6725e8-d540-42f8-b20e-0dbdf461c097" (UID: "eb6725e8-d540-42f8-b20e-0dbdf461c097"). InnerVolumeSpecName "kube-api-access-qbjph". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:25:02 crc kubenswrapper[5052]: I1203 13:25:02.095260 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eb6725e8-d540-42f8-b20e-0dbdf461c097-fernet-keys" (OuterVolumeSpecName: "fernet-keys") pod "eb6725e8-d540-42f8-b20e-0dbdf461c097" (UID: "eb6725e8-d540-42f8-b20e-0dbdf461c097"). InnerVolumeSpecName "fernet-keys". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:25:02 crc kubenswrapper[5052]: I1203 13:25:02.101817 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eb6725e8-d540-42f8-b20e-0dbdf461c097-credential-keys" (OuterVolumeSpecName: "credential-keys") pod "eb6725e8-d540-42f8-b20e-0dbdf461c097" (UID: "eb6725e8-d540-42f8-b20e-0dbdf461c097"). InnerVolumeSpecName "credential-keys". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:25:02 crc kubenswrapper[5052]: I1203 13:25:02.112278 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eb6725e8-d540-42f8-b20e-0dbdf461c097-config-data" (OuterVolumeSpecName: "config-data") pod "eb6725e8-d540-42f8-b20e-0dbdf461c097" (UID: "eb6725e8-d540-42f8-b20e-0dbdf461c097"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:25:02 crc kubenswrapper[5052]: I1203 13:25:02.112777 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eb6725e8-d540-42f8-b20e-0dbdf461c097-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "eb6725e8-d540-42f8-b20e-0dbdf461c097" (UID: "eb6725e8-d540-42f8-b20e-0dbdf461c097"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:25:02 crc kubenswrapper[5052]: I1203 13:25:02.129485 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eb6725e8-d540-42f8-b20e-0dbdf461c097-public-tls-certs" (OuterVolumeSpecName: "public-tls-certs") pod "eb6725e8-d540-42f8-b20e-0dbdf461c097" (UID: "eb6725e8-d540-42f8-b20e-0dbdf461c097"). InnerVolumeSpecName "public-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:25:02 crc kubenswrapper[5052]: I1203 13:25:02.132324 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eb6725e8-d540-42f8-b20e-0dbdf461c097-internal-tls-certs" (OuterVolumeSpecName: "internal-tls-certs") pod "eb6725e8-d540-42f8-b20e-0dbdf461c097" (UID: "eb6725e8-d540-42f8-b20e-0dbdf461c097"). InnerVolumeSpecName "internal-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:25:02 crc kubenswrapper[5052]: I1203 13:25:02.195090 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/eb6725e8-d540-42f8-b20e-0dbdf461c097-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 13:25:02 crc kubenswrapper[5052]: I1203 13:25:02.195141 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-qbjph\" (UniqueName: \"kubernetes.io/projected/eb6725e8-d540-42f8-b20e-0dbdf461c097-kube-api-access-qbjph\") on node \"crc\" DevicePath \"\"" Dec 03 13:25:02 crc kubenswrapper[5052]: I1203 13:25:02.195156 5052 reconciler_common.go:293] "Volume detached for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/eb6725e8-d540-42f8-b20e-0dbdf461c097-fernet-keys\") on node \"crc\" DevicePath \"\"" Dec 03 13:25:02 crc kubenswrapper[5052]: I1203 13:25:02.195166 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/eb6725e8-d540-42f8-b20e-0dbdf461c097-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 13:25:02 crc kubenswrapper[5052]: I1203 13:25:02.195175 5052 reconciler_common.go:293] "Volume detached for volume \"public-tls-certs\" (UniqueName: \"kubernetes.io/secret/eb6725e8-d540-42f8-b20e-0dbdf461c097-public-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 13:25:02 crc kubenswrapper[5052]: I1203 13:25:02.195186 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/eb6725e8-d540-42f8-b20e-0dbdf461c097-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 13:25:02 crc kubenswrapper[5052]: I1203 13:25:02.195194 5052 reconciler_common.go:293] "Volume detached for volume \"internal-tls-certs\" (UniqueName: \"kubernetes.io/secret/eb6725e8-d540-42f8-b20e-0dbdf461c097-internal-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 13:25:02 crc kubenswrapper[5052]: I1203 13:25:02.195202 5052 reconciler_common.go:293] "Volume detached for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/eb6725e8-d540-42f8-b20e-0dbdf461c097-credential-keys\") on node \"crc\" DevicePath \"\"" Dec 03 13:25:02 crc kubenswrapper[5052]: I1203 13:25:02.237700 5052 generic.go:334] "Generic (PLEG): container finished" podID="eb6725e8-d540-42f8-b20e-0dbdf461c097" containerID="3cbdf79fe420cf9422c3d88ccf18cff6578d9314bb2d2bad0e5920d53509dcd9" exitCode=0 Dec 03 13:25:02 crc kubenswrapper[5052]: I1203 13:25:02.237784 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="keystone-kuttl-tests/keystone-db495cbcd-dkqhp" event={"ID":"eb6725e8-d540-42f8-b20e-0dbdf461c097","Type":"ContainerDied","Data":"3cbdf79fe420cf9422c3d88ccf18cff6578d9314bb2d2bad0e5920d53509dcd9"} Dec 03 13:25:02 crc kubenswrapper[5052]: I1203 13:25:02.237818 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="keystone-kuttl-tests/keystone-db495cbcd-dkqhp" event={"ID":"eb6725e8-d540-42f8-b20e-0dbdf461c097","Type":"ContainerDied","Data":"8c5f683a2bb43dff524ad5963bd9cf8f8cd7fec73d2259684b52ce1d4afc8c6b"} Dec 03 13:25:02 crc kubenswrapper[5052]: I1203 13:25:02.237840 5052 scope.go:117] "RemoveContainer" containerID="3cbdf79fe420cf9422c3d88ccf18cff6578d9314bb2d2bad0e5920d53509dcd9" Dec 03 13:25:02 crc kubenswrapper[5052]: I1203 13:25:02.238190 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="keystone-kuttl-tests/keystone-db495cbcd-dkqhp" Dec 03 13:25:02 crc kubenswrapper[5052]: I1203 13:25:02.240857 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="keystone-kuttl-tests/keystone3828-account-delete-57qhp" event={"ID":"8fb5a2bb-5a02-487b-a695-d9bd88401a42","Type":"ContainerDied","Data":"e317cacb055c02f56b9590fff65fbc9da0929abf2763753ed615217e2414989f"} Dec 03 13:25:02 crc kubenswrapper[5052]: I1203 13:25:02.240897 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e317cacb055c02f56b9590fff65fbc9da0929abf2763753ed615217e2414989f" Dec 03 13:25:02 crc kubenswrapper[5052]: I1203 13:25:02.240929 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="keystone-kuttl-tests/keystone3828-account-delete-57qhp" Dec 03 13:25:02 crc kubenswrapper[5052]: I1203 13:25:02.267376 5052 scope.go:117] "RemoveContainer" containerID="3cbdf79fe420cf9422c3d88ccf18cff6578d9314bb2d2bad0e5920d53509dcd9" Dec 03 13:25:02 crc kubenswrapper[5052]: E1203 13:25:02.267786 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3cbdf79fe420cf9422c3d88ccf18cff6578d9314bb2d2bad0e5920d53509dcd9\": container with ID starting with 3cbdf79fe420cf9422c3d88ccf18cff6578d9314bb2d2bad0e5920d53509dcd9 not found: ID does not exist" containerID="3cbdf79fe420cf9422c3d88ccf18cff6578d9314bb2d2bad0e5920d53509dcd9" Dec 03 13:25:02 crc kubenswrapper[5052]: I1203 13:25:02.267825 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3cbdf79fe420cf9422c3d88ccf18cff6578d9314bb2d2bad0e5920d53509dcd9"} err="failed to get container status \"3cbdf79fe420cf9422c3d88ccf18cff6578d9314bb2d2bad0e5920d53509dcd9\": rpc error: code = NotFound desc = could not find container \"3cbdf79fe420cf9422c3d88ccf18cff6578d9314bb2d2bad0e5920d53509dcd9\": container with ID starting with 3cbdf79fe420cf9422c3d88ccf18cff6578d9314bb2d2bad0e5920d53509dcd9 not found: ID does not exist" Dec 03 13:25:02 crc kubenswrapper[5052]: I1203 13:25:02.276322 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["keystone-kuttl-tests/keystone-db495cbcd-dkqhp"] Dec 03 13:25:02 crc kubenswrapper[5052]: I1203 13:25:02.282270 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["keystone-kuttl-tests/keystone-db495cbcd-dkqhp"] Dec 03 13:25:03 crc kubenswrapper[5052]: I1203 13:25:03.444537 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["keystone-kuttl-tests/keystone-db-create-zkhpv"] Dec 03 13:25:03 crc kubenswrapper[5052]: I1203 13:25:03.451600 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["keystone-kuttl-tests/keystone-db-create-zkhpv"] Dec 03 13:25:03 crc kubenswrapper[5052]: I1203 13:25:03.460229 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["keystone-kuttl-tests/keystone-3828-account-create-update-ggk67"] Dec 03 13:25:03 crc kubenswrapper[5052]: I1203 13:25:03.467364 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["keystone-kuttl-tests/keystone3828-account-delete-57qhp"] Dec 03 13:25:03 crc kubenswrapper[5052]: I1203 13:25:03.473181 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["keystone-kuttl-tests/keystone3828-account-delete-57qhp"] Dec 03 13:25:03 crc kubenswrapper[5052]: I1203 13:25:03.478438 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["keystone-kuttl-tests/keystone-3828-account-create-update-ggk67"] Dec 03 13:25:03 crc kubenswrapper[5052]: I1203 13:25:03.545784 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["keystone-kuttl-tests/keystone-db-create-d4zvz"] Dec 03 13:25:03 crc kubenswrapper[5052]: E1203 13:25:03.546141 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="eb6725e8-d540-42f8-b20e-0dbdf461c097" containerName="keystone-api" Dec 03 13:25:03 crc kubenswrapper[5052]: I1203 13:25:03.546164 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="eb6725e8-d540-42f8-b20e-0dbdf461c097" containerName="keystone-api" Dec 03 13:25:03 crc kubenswrapper[5052]: E1203 13:25:03.546196 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="8fb5a2bb-5a02-487b-a695-d9bd88401a42" containerName="mariadb-account-delete" Dec 03 13:25:03 crc kubenswrapper[5052]: I1203 13:25:03.546205 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="8fb5a2bb-5a02-487b-a695-d9bd88401a42" containerName="mariadb-account-delete" Dec 03 13:25:03 crc kubenswrapper[5052]: I1203 13:25:03.546420 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="8fb5a2bb-5a02-487b-a695-d9bd88401a42" containerName="mariadb-account-delete" Dec 03 13:25:03 crc kubenswrapper[5052]: I1203 13:25:03.546441 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="eb6725e8-d540-42f8-b20e-0dbdf461c097" containerName="keystone-api" Dec 03 13:25:03 crc kubenswrapper[5052]: I1203 13:25:03.547110 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="keystone-kuttl-tests/keystone-db-create-d4zvz" Dec 03 13:25:03 crc kubenswrapper[5052]: I1203 13:25:03.561175 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["keystone-kuttl-tests/keystone-db-create-d4zvz"] Dec 03 13:25:03 crc kubenswrapper[5052]: I1203 13:25:03.642019 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["keystone-kuttl-tests/keystone-f47f-account-create-update-9bf9g"] Dec 03 13:25:03 crc kubenswrapper[5052]: I1203 13:25:03.642975 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="keystone-kuttl-tests/keystone-f47f-account-create-update-9bf9g" Dec 03 13:25:03 crc kubenswrapper[5052]: I1203 13:25:03.644866 5052 reflector.go:368] Caches populated for *v1.Secret from object-"keystone-kuttl-tests"/"keystone-db-secret" Dec 03 13:25:03 crc kubenswrapper[5052]: I1203 13:25:03.671162 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["keystone-kuttl-tests/keystone-f47f-account-create-update-9bf9g"] Dec 03 13:25:03 crc kubenswrapper[5052]: I1203 13:25:03.682012 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="31c262f6-ecf6-4e0f-9c98-187f2c3e814a" path="/var/lib/kubelet/pods/31c262f6-ecf6-4e0f-9c98-187f2c3e814a/volumes" Dec 03 13:25:03 crc kubenswrapper[5052]: I1203 13:25:03.682635 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8fb5a2bb-5a02-487b-a695-d9bd88401a42" path="/var/lib/kubelet/pods/8fb5a2bb-5a02-487b-a695-d9bd88401a42/volumes" Dec 03 13:25:03 crc kubenswrapper[5052]: I1203 13:25:03.683185 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d6ae75e1-39bf-4fdf-a0b9-231b3ce16f9c" path="/var/lib/kubelet/pods/d6ae75e1-39bf-4fdf-a0b9-231b3ce16f9c/volumes" Dec 03 13:25:03 crc kubenswrapper[5052]: I1203 13:25:03.683712 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eb6725e8-d540-42f8-b20e-0dbdf461c097" path="/var/lib/kubelet/pods/eb6725e8-d540-42f8-b20e-0dbdf461c097/volumes" Dec 03 13:25:03 crc kubenswrapper[5052]: I1203 13:25:03.721883 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n7tnt\" (UniqueName: \"kubernetes.io/projected/6397f36b-76b0-4f60-a546-3f9c2ca74bb8-kube-api-access-n7tnt\") pod \"keystone-db-create-d4zvz\" (UID: \"6397f36b-76b0-4f60-a546-3f9c2ca74bb8\") " pod="keystone-kuttl-tests/keystone-db-create-d4zvz" Dec 03 13:25:03 crc kubenswrapper[5052]: I1203 13:25:03.721992 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/6397f36b-76b0-4f60-a546-3f9c2ca74bb8-operator-scripts\") pod \"keystone-db-create-d4zvz\" (UID: \"6397f36b-76b0-4f60-a546-3f9c2ca74bb8\") " pod="keystone-kuttl-tests/keystone-db-create-d4zvz" Dec 03 13:25:03 crc kubenswrapper[5052]: I1203 13:25:03.823866 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/6397f36b-76b0-4f60-a546-3f9c2ca74bb8-operator-scripts\") pod \"keystone-db-create-d4zvz\" (UID: \"6397f36b-76b0-4f60-a546-3f9c2ca74bb8\") " pod="keystone-kuttl-tests/keystone-db-create-d4zvz" Dec 03 13:25:03 crc kubenswrapper[5052]: I1203 13:25:03.824726 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-km5qp\" (UniqueName: \"kubernetes.io/projected/9186bb8c-cb25-4bbb-9b54-b53b64a0c858-kube-api-access-km5qp\") pod \"keystone-f47f-account-create-update-9bf9g\" (UID: \"9186bb8c-cb25-4bbb-9b54-b53b64a0c858\") " pod="keystone-kuttl-tests/keystone-f47f-account-create-update-9bf9g" Dec 03 13:25:03 crc kubenswrapper[5052]: I1203 13:25:03.824952 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/9186bb8c-cb25-4bbb-9b54-b53b64a0c858-operator-scripts\") pod \"keystone-f47f-account-create-update-9bf9g\" (UID: \"9186bb8c-cb25-4bbb-9b54-b53b64a0c858\") " pod="keystone-kuttl-tests/keystone-f47f-account-create-update-9bf9g" Dec 03 13:25:03 crc kubenswrapper[5052]: I1203 13:25:03.825009 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-n7tnt\" (UniqueName: \"kubernetes.io/projected/6397f36b-76b0-4f60-a546-3f9c2ca74bb8-kube-api-access-n7tnt\") pod \"keystone-db-create-d4zvz\" (UID: \"6397f36b-76b0-4f60-a546-3f9c2ca74bb8\") " pod="keystone-kuttl-tests/keystone-db-create-d4zvz" Dec 03 13:25:03 crc kubenswrapper[5052]: I1203 13:25:03.825039 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/6397f36b-76b0-4f60-a546-3f9c2ca74bb8-operator-scripts\") pod \"keystone-db-create-d4zvz\" (UID: \"6397f36b-76b0-4f60-a546-3f9c2ca74bb8\") " pod="keystone-kuttl-tests/keystone-db-create-d4zvz" Dec 03 13:25:03 crc kubenswrapper[5052]: I1203 13:25:03.842795 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-n7tnt\" (UniqueName: \"kubernetes.io/projected/6397f36b-76b0-4f60-a546-3f9c2ca74bb8-kube-api-access-n7tnt\") pod \"keystone-db-create-d4zvz\" (UID: \"6397f36b-76b0-4f60-a546-3f9c2ca74bb8\") " pod="keystone-kuttl-tests/keystone-db-create-d4zvz" Dec 03 13:25:03 crc kubenswrapper[5052]: I1203 13:25:03.861696 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="keystone-kuttl-tests/keystone-db-create-d4zvz" Dec 03 13:25:03 crc kubenswrapper[5052]: I1203 13:25:03.925978 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/9186bb8c-cb25-4bbb-9b54-b53b64a0c858-operator-scripts\") pod \"keystone-f47f-account-create-update-9bf9g\" (UID: \"9186bb8c-cb25-4bbb-9b54-b53b64a0c858\") " pod="keystone-kuttl-tests/keystone-f47f-account-create-update-9bf9g" Dec 03 13:25:03 crc kubenswrapper[5052]: I1203 13:25:03.926074 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-km5qp\" (UniqueName: \"kubernetes.io/projected/9186bb8c-cb25-4bbb-9b54-b53b64a0c858-kube-api-access-km5qp\") pod \"keystone-f47f-account-create-update-9bf9g\" (UID: \"9186bb8c-cb25-4bbb-9b54-b53b64a0c858\") " pod="keystone-kuttl-tests/keystone-f47f-account-create-update-9bf9g" Dec 03 13:25:03 crc kubenswrapper[5052]: I1203 13:25:03.926819 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/9186bb8c-cb25-4bbb-9b54-b53b64a0c858-operator-scripts\") pod \"keystone-f47f-account-create-update-9bf9g\" (UID: \"9186bb8c-cb25-4bbb-9b54-b53b64a0c858\") " pod="keystone-kuttl-tests/keystone-f47f-account-create-update-9bf9g" Dec 03 13:25:03 crc kubenswrapper[5052]: I1203 13:25:03.947624 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-km5qp\" (UniqueName: \"kubernetes.io/projected/9186bb8c-cb25-4bbb-9b54-b53b64a0c858-kube-api-access-km5qp\") pod \"keystone-f47f-account-create-update-9bf9g\" (UID: \"9186bb8c-cb25-4bbb-9b54-b53b64a0c858\") " pod="keystone-kuttl-tests/keystone-f47f-account-create-update-9bf9g" Dec 03 13:25:03 crc kubenswrapper[5052]: I1203 13:25:03.965161 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="keystone-kuttl-tests/keystone-f47f-account-create-update-9bf9g" Dec 03 13:25:04 crc kubenswrapper[5052]: I1203 13:25:04.328179 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["keystone-kuttl-tests/keystone-db-create-d4zvz"] Dec 03 13:25:04 crc kubenswrapper[5052]: I1203 13:25:04.458270 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["keystone-kuttl-tests/keystone-f47f-account-create-update-9bf9g"] Dec 03 13:25:05 crc kubenswrapper[5052]: I1203 13:25:05.265574 5052 generic.go:334] "Generic (PLEG): container finished" podID="9186bb8c-cb25-4bbb-9b54-b53b64a0c858" containerID="96e2baa6a38f359a88096b36441f51880922a13bfca52434fffa726ba73e87a7" exitCode=0 Dec 03 13:25:05 crc kubenswrapper[5052]: I1203 13:25:05.265630 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="keystone-kuttl-tests/keystone-f47f-account-create-update-9bf9g" event={"ID":"9186bb8c-cb25-4bbb-9b54-b53b64a0c858","Type":"ContainerDied","Data":"96e2baa6a38f359a88096b36441f51880922a13bfca52434fffa726ba73e87a7"} Dec 03 13:25:05 crc kubenswrapper[5052]: I1203 13:25:05.265956 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="keystone-kuttl-tests/keystone-f47f-account-create-update-9bf9g" event={"ID":"9186bb8c-cb25-4bbb-9b54-b53b64a0c858","Type":"ContainerStarted","Data":"ff7d233c3b520b2b49868b77597b125a2ac3325d3ebeb6ee670e1189eab99652"} Dec 03 13:25:05 crc kubenswrapper[5052]: I1203 13:25:05.268800 5052 generic.go:334] "Generic (PLEG): container finished" podID="6397f36b-76b0-4f60-a546-3f9c2ca74bb8" containerID="601076b6926a51495b03f8a0a4221dc359b96fcb6bcdb02d3a1c3ebadd4a24a3" exitCode=0 Dec 03 13:25:05 crc kubenswrapper[5052]: I1203 13:25:05.268858 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="keystone-kuttl-tests/keystone-db-create-d4zvz" event={"ID":"6397f36b-76b0-4f60-a546-3f9c2ca74bb8","Type":"ContainerDied","Data":"601076b6926a51495b03f8a0a4221dc359b96fcb6bcdb02d3a1c3ebadd4a24a3"} Dec 03 13:25:05 crc kubenswrapper[5052]: I1203 13:25:05.268896 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="keystone-kuttl-tests/keystone-db-create-d4zvz" event={"ID":"6397f36b-76b0-4f60-a546-3f9c2ca74bb8","Type":"ContainerStarted","Data":"756a30372a06c6914c37c9a03490ccb5f7317c1948c33653a75ba5e8e93fb975"} Dec 03 13:25:06 crc kubenswrapper[5052]: I1203 13:25:06.679171 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="keystone-kuttl-tests/keystone-db-create-d4zvz" Dec 03 13:25:06 crc kubenswrapper[5052]: I1203 13:25:06.686299 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="keystone-kuttl-tests/keystone-f47f-account-create-update-9bf9g" Dec 03 13:25:06 crc kubenswrapper[5052]: I1203 13:25:06.874470 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/9186bb8c-cb25-4bbb-9b54-b53b64a0c858-operator-scripts\") pod \"9186bb8c-cb25-4bbb-9b54-b53b64a0c858\" (UID: \"9186bb8c-cb25-4bbb-9b54-b53b64a0c858\") " Dec 03 13:25:06 crc kubenswrapper[5052]: I1203 13:25:06.874675 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/6397f36b-76b0-4f60-a546-3f9c2ca74bb8-operator-scripts\") pod \"6397f36b-76b0-4f60-a546-3f9c2ca74bb8\" (UID: \"6397f36b-76b0-4f60-a546-3f9c2ca74bb8\") " Dec 03 13:25:06 crc kubenswrapper[5052]: I1203 13:25:06.874770 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-km5qp\" (UniqueName: \"kubernetes.io/projected/9186bb8c-cb25-4bbb-9b54-b53b64a0c858-kube-api-access-km5qp\") pod \"9186bb8c-cb25-4bbb-9b54-b53b64a0c858\" (UID: \"9186bb8c-cb25-4bbb-9b54-b53b64a0c858\") " Dec 03 13:25:06 crc kubenswrapper[5052]: I1203 13:25:06.874824 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n7tnt\" (UniqueName: \"kubernetes.io/projected/6397f36b-76b0-4f60-a546-3f9c2ca74bb8-kube-api-access-n7tnt\") pod \"6397f36b-76b0-4f60-a546-3f9c2ca74bb8\" (UID: \"6397f36b-76b0-4f60-a546-3f9c2ca74bb8\") " Dec 03 13:25:06 crc kubenswrapper[5052]: I1203 13:25:06.875294 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9186bb8c-cb25-4bbb-9b54-b53b64a0c858-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "9186bb8c-cb25-4bbb-9b54-b53b64a0c858" (UID: "9186bb8c-cb25-4bbb-9b54-b53b64a0c858"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:25:06 crc kubenswrapper[5052]: I1203 13:25:06.875453 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6397f36b-76b0-4f60-a546-3f9c2ca74bb8-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "6397f36b-76b0-4f60-a546-3f9c2ca74bb8" (UID: "6397f36b-76b0-4f60-a546-3f9c2ca74bb8"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:25:06 crc kubenswrapper[5052]: I1203 13:25:06.880998 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6397f36b-76b0-4f60-a546-3f9c2ca74bb8-kube-api-access-n7tnt" (OuterVolumeSpecName: "kube-api-access-n7tnt") pod "6397f36b-76b0-4f60-a546-3f9c2ca74bb8" (UID: "6397f36b-76b0-4f60-a546-3f9c2ca74bb8"). InnerVolumeSpecName "kube-api-access-n7tnt". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:25:06 crc kubenswrapper[5052]: I1203 13:25:06.881991 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9186bb8c-cb25-4bbb-9b54-b53b64a0c858-kube-api-access-km5qp" (OuterVolumeSpecName: "kube-api-access-km5qp") pod "9186bb8c-cb25-4bbb-9b54-b53b64a0c858" (UID: "9186bb8c-cb25-4bbb-9b54-b53b64a0c858"). InnerVolumeSpecName "kube-api-access-km5qp". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:25:06 crc kubenswrapper[5052]: I1203 13:25:06.976231 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/6397f36b-76b0-4f60-a546-3f9c2ca74bb8-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 13:25:06 crc kubenswrapper[5052]: I1203 13:25:06.976573 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-km5qp\" (UniqueName: \"kubernetes.io/projected/9186bb8c-cb25-4bbb-9b54-b53b64a0c858-kube-api-access-km5qp\") on node \"crc\" DevicePath \"\"" Dec 03 13:25:06 crc kubenswrapper[5052]: I1203 13:25:06.976585 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-n7tnt\" (UniqueName: \"kubernetes.io/projected/6397f36b-76b0-4f60-a546-3f9c2ca74bb8-kube-api-access-n7tnt\") on node \"crc\" DevicePath \"\"" Dec 03 13:25:06 crc kubenswrapper[5052]: I1203 13:25:06.976596 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/9186bb8c-cb25-4bbb-9b54-b53b64a0c858-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 13:25:07 crc kubenswrapper[5052]: I1203 13:25:07.289036 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="keystone-kuttl-tests/keystone-f47f-account-create-update-9bf9g" event={"ID":"9186bb8c-cb25-4bbb-9b54-b53b64a0c858","Type":"ContainerDied","Data":"ff7d233c3b520b2b49868b77597b125a2ac3325d3ebeb6ee670e1189eab99652"} Dec 03 13:25:07 crc kubenswrapper[5052]: I1203 13:25:07.289090 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ff7d233c3b520b2b49868b77597b125a2ac3325d3ebeb6ee670e1189eab99652" Dec 03 13:25:07 crc kubenswrapper[5052]: I1203 13:25:07.289566 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="keystone-kuttl-tests/keystone-f47f-account-create-update-9bf9g" Dec 03 13:25:07 crc kubenswrapper[5052]: I1203 13:25:07.291066 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="keystone-kuttl-tests/keystone-db-create-d4zvz" event={"ID":"6397f36b-76b0-4f60-a546-3f9c2ca74bb8","Type":"ContainerDied","Data":"756a30372a06c6914c37c9a03490ccb5f7317c1948c33653a75ba5e8e93fb975"} Dec 03 13:25:07 crc kubenswrapper[5052]: I1203 13:25:07.291209 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="756a30372a06c6914c37c9a03490ccb5f7317c1948c33653a75ba5e8e93fb975" Dec 03 13:25:07 crc kubenswrapper[5052]: I1203 13:25:07.291104 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="keystone-kuttl-tests/keystone-db-create-d4zvz" Dec 03 13:25:09 crc kubenswrapper[5052]: I1203 13:25:09.198821 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["keystone-kuttl-tests/keystone-db-sync-zfwhv"] Dec 03 13:25:09 crc kubenswrapper[5052]: E1203 13:25:09.199148 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="6397f36b-76b0-4f60-a546-3f9c2ca74bb8" containerName="mariadb-database-create" Dec 03 13:25:09 crc kubenswrapper[5052]: I1203 13:25:09.199171 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="6397f36b-76b0-4f60-a546-3f9c2ca74bb8" containerName="mariadb-database-create" Dec 03 13:25:09 crc kubenswrapper[5052]: E1203 13:25:09.199222 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="9186bb8c-cb25-4bbb-9b54-b53b64a0c858" containerName="mariadb-account-create-update" Dec 03 13:25:09 crc kubenswrapper[5052]: I1203 13:25:09.199231 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="9186bb8c-cb25-4bbb-9b54-b53b64a0c858" containerName="mariadb-account-create-update" Dec 03 13:25:09 crc kubenswrapper[5052]: I1203 13:25:09.199377 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="9186bb8c-cb25-4bbb-9b54-b53b64a0c858" containerName="mariadb-account-create-update" Dec 03 13:25:09 crc kubenswrapper[5052]: I1203 13:25:09.199400 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="6397f36b-76b0-4f60-a546-3f9c2ca74bb8" containerName="mariadb-database-create" Dec 03 13:25:09 crc kubenswrapper[5052]: I1203 13:25:09.200001 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="keystone-kuttl-tests/keystone-db-sync-zfwhv" Dec 03 13:25:09 crc kubenswrapper[5052]: I1203 13:25:09.205012 5052 reflector.go:368] Caches populated for *v1.Secret from object-"keystone-kuttl-tests"/"keystone-scripts" Dec 03 13:25:09 crc kubenswrapper[5052]: I1203 13:25:09.205151 5052 reflector.go:368] Caches populated for *v1.Secret from object-"keystone-kuttl-tests"/"keystone-config-data" Dec 03 13:25:09 crc kubenswrapper[5052]: I1203 13:25:09.205431 5052 reflector.go:368] Caches populated for *v1.Secret from object-"keystone-kuttl-tests"/"keystone" Dec 03 13:25:09 crc kubenswrapper[5052]: I1203 13:25:09.205444 5052 reflector.go:368] Caches populated for *v1.Secret from object-"keystone-kuttl-tests"/"keystone-keystone-dockercfg-4pnrz" Dec 03 13:25:09 crc kubenswrapper[5052]: I1203 13:25:09.211304 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["keystone-kuttl-tests/keystone-db-sync-zfwhv"] Dec 03 13:25:09 crc kubenswrapper[5052]: I1203 13:25:09.309144 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/dfa1f868-cd4f-4e2b-97fd-46e65188ad75-config-data\") pod \"keystone-db-sync-zfwhv\" (UID: \"dfa1f868-cd4f-4e2b-97fd-46e65188ad75\") " pod="keystone-kuttl-tests/keystone-db-sync-zfwhv" Dec 03 13:25:09 crc kubenswrapper[5052]: I1203 13:25:09.309799 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gb4st\" (UniqueName: \"kubernetes.io/projected/dfa1f868-cd4f-4e2b-97fd-46e65188ad75-kube-api-access-gb4st\") pod \"keystone-db-sync-zfwhv\" (UID: \"dfa1f868-cd4f-4e2b-97fd-46e65188ad75\") " pod="keystone-kuttl-tests/keystone-db-sync-zfwhv" Dec 03 13:25:09 crc kubenswrapper[5052]: I1203 13:25:09.411205 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-gb4st\" (UniqueName: \"kubernetes.io/projected/dfa1f868-cd4f-4e2b-97fd-46e65188ad75-kube-api-access-gb4st\") pod \"keystone-db-sync-zfwhv\" (UID: \"dfa1f868-cd4f-4e2b-97fd-46e65188ad75\") " pod="keystone-kuttl-tests/keystone-db-sync-zfwhv" Dec 03 13:25:09 crc kubenswrapper[5052]: I1203 13:25:09.411317 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/dfa1f868-cd4f-4e2b-97fd-46e65188ad75-config-data\") pod \"keystone-db-sync-zfwhv\" (UID: \"dfa1f868-cd4f-4e2b-97fd-46e65188ad75\") " pod="keystone-kuttl-tests/keystone-db-sync-zfwhv" Dec 03 13:25:09 crc kubenswrapper[5052]: I1203 13:25:09.415909 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/dfa1f868-cd4f-4e2b-97fd-46e65188ad75-config-data\") pod \"keystone-db-sync-zfwhv\" (UID: \"dfa1f868-cd4f-4e2b-97fd-46e65188ad75\") " pod="keystone-kuttl-tests/keystone-db-sync-zfwhv" Dec 03 13:25:09 crc kubenswrapper[5052]: I1203 13:25:09.427944 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-gb4st\" (UniqueName: \"kubernetes.io/projected/dfa1f868-cd4f-4e2b-97fd-46e65188ad75-kube-api-access-gb4st\") pod \"keystone-db-sync-zfwhv\" (UID: \"dfa1f868-cd4f-4e2b-97fd-46e65188ad75\") " pod="keystone-kuttl-tests/keystone-db-sync-zfwhv" Dec 03 13:25:09 crc kubenswrapper[5052]: I1203 13:25:09.572086 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="keystone-kuttl-tests/keystone-db-sync-zfwhv" Dec 03 13:25:10 crc kubenswrapper[5052]: I1203 13:25:10.037931 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["keystone-kuttl-tests/keystone-db-sync-zfwhv"] Dec 03 13:25:10 crc kubenswrapper[5052]: I1203 13:25:10.313392 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="keystone-kuttl-tests/keystone-db-sync-zfwhv" event={"ID":"dfa1f868-cd4f-4e2b-97fd-46e65188ad75","Type":"ContainerStarted","Data":"17082d2831f68dfa87f467d88de959bb9cf51c0169262a93d703df12813dea6f"} Dec 03 13:25:10 crc kubenswrapper[5052]: I1203 13:25:10.313814 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="keystone-kuttl-tests/keystone-db-sync-zfwhv" event={"ID":"dfa1f868-cd4f-4e2b-97fd-46e65188ad75","Type":"ContainerStarted","Data":"41d021021a4c1d2fc336a75f8e7b8ac793f49b56a60111793afe6f1411dfd116"} Dec 03 13:25:10 crc kubenswrapper[5052]: I1203 13:25:10.332909 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="keystone-kuttl-tests/keystone-db-sync-zfwhv" podStartSLOduration=1.332892668 podStartE2EDuration="1.332892668s" podCreationTimestamp="2025-12-03 13:25:09 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 13:25:10.328100051 +0000 UTC m=+7500.916341835" watchObservedRunningTime="2025-12-03 13:25:10.332892668 +0000 UTC m=+7500.921134432" Dec 03 13:25:12 crc kubenswrapper[5052]: I1203 13:25:12.337336 5052 generic.go:334] "Generic (PLEG): container finished" podID="dfa1f868-cd4f-4e2b-97fd-46e65188ad75" containerID="17082d2831f68dfa87f467d88de959bb9cf51c0169262a93d703df12813dea6f" exitCode=0 Dec 03 13:25:12 crc kubenswrapper[5052]: I1203 13:25:12.337388 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="keystone-kuttl-tests/keystone-db-sync-zfwhv" event={"ID":"dfa1f868-cd4f-4e2b-97fd-46e65188ad75","Type":"ContainerDied","Data":"17082d2831f68dfa87f467d88de959bb9cf51c0169262a93d703df12813dea6f"} Dec 03 13:25:13 crc kubenswrapper[5052]: I1203 13:25:13.670725 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="keystone-kuttl-tests/keystone-db-sync-zfwhv" Dec 03 13:25:13 crc kubenswrapper[5052]: I1203 13:25:13.795954 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/dfa1f868-cd4f-4e2b-97fd-46e65188ad75-config-data\") pod \"dfa1f868-cd4f-4e2b-97fd-46e65188ad75\" (UID: \"dfa1f868-cd4f-4e2b-97fd-46e65188ad75\") " Dec 03 13:25:13 crc kubenswrapper[5052]: I1203 13:25:13.796435 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gb4st\" (UniqueName: \"kubernetes.io/projected/dfa1f868-cd4f-4e2b-97fd-46e65188ad75-kube-api-access-gb4st\") pod \"dfa1f868-cd4f-4e2b-97fd-46e65188ad75\" (UID: \"dfa1f868-cd4f-4e2b-97fd-46e65188ad75\") " Dec 03 13:25:13 crc kubenswrapper[5052]: I1203 13:25:13.802656 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dfa1f868-cd4f-4e2b-97fd-46e65188ad75-kube-api-access-gb4st" (OuterVolumeSpecName: "kube-api-access-gb4st") pod "dfa1f868-cd4f-4e2b-97fd-46e65188ad75" (UID: "dfa1f868-cd4f-4e2b-97fd-46e65188ad75"). InnerVolumeSpecName "kube-api-access-gb4st". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:25:13 crc kubenswrapper[5052]: I1203 13:25:13.833077 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dfa1f868-cd4f-4e2b-97fd-46e65188ad75-config-data" (OuterVolumeSpecName: "config-data") pod "dfa1f868-cd4f-4e2b-97fd-46e65188ad75" (UID: "dfa1f868-cd4f-4e2b-97fd-46e65188ad75"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:25:13 crc kubenswrapper[5052]: I1203 13:25:13.899580 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/dfa1f868-cd4f-4e2b-97fd-46e65188ad75-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 13:25:13 crc kubenswrapper[5052]: I1203 13:25:13.899629 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-gb4st\" (UniqueName: \"kubernetes.io/projected/dfa1f868-cd4f-4e2b-97fd-46e65188ad75-kube-api-access-gb4st\") on node \"crc\" DevicePath \"\"" Dec 03 13:25:14 crc kubenswrapper[5052]: I1203 13:25:14.355774 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="keystone-kuttl-tests/keystone-db-sync-zfwhv" event={"ID":"dfa1f868-cd4f-4e2b-97fd-46e65188ad75","Type":"ContainerDied","Data":"41d021021a4c1d2fc336a75f8e7b8ac793f49b56a60111793afe6f1411dfd116"} Dec 03 13:25:14 crc kubenswrapper[5052]: I1203 13:25:14.355844 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="41d021021a4c1d2fc336a75f8e7b8ac793f49b56a60111793afe6f1411dfd116" Dec 03 13:25:14 crc kubenswrapper[5052]: I1203 13:25:14.355852 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="keystone-kuttl-tests/keystone-db-sync-zfwhv" Dec 03 13:25:14 crc kubenswrapper[5052]: I1203 13:25:14.528073 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["keystone-kuttl-tests/keystone-bootstrap-x5gg9"] Dec 03 13:25:14 crc kubenswrapper[5052]: E1203 13:25:14.528423 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="dfa1f868-cd4f-4e2b-97fd-46e65188ad75" containerName="keystone-db-sync" Dec 03 13:25:14 crc kubenswrapper[5052]: I1203 13:25:14.528445 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="dfa1f868-cd4f-4e2b-97fd-46e65188ad75" containerName="keystone-db-sync" Dec 03 13:25:14 crc kubenswrapper[5052]: I1203 13:25:14.528623 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="dfa1f868-cd4f-4e2b-97fd-46e65188ad75" containerName="keystone-db-sync" Dec 03 13:25:14 crc kubenswrapper[5052]: I1203 13:25:14.529231 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="keystone-kuttl-tests/keystone-bootstrap-x5gg9" Dec 03 13:25:14 crc kubenswrapper[5052]: I1203 13:25:14.533464 5052 reflector.go:368] Caches populated for *v1.Secret from object-"keystone-kuttl-tests"/"keystone" Dec 03 13:25:14 crc kubenswrapper[5052]: I1203 13:25:14.533819 5052 reflector.go:368] Caches populated for *v1.Secret from object-"keystone-kuttl-tests"/"osp-secret" Dec 03 13:25:14 crc kubenswrapper[5052]: I1203 13:25:14.534024 5052 reflector.go:368] Caches populated for *v1.Secret from object-"keystone-kuttl-tests"/"keystone-keystone-dockercfg-4pnrz" Dec 03 13:25:14 crc kubenswrapper[5052]: I1203 13:25:14.534924 5052 reflector.go:368] Caches populated for *v1.Secret from object-"keystone-kuttl-tests"/"keystone-scripts" Dec 03 13:25:14 crc kubenswrapper[5052]: I1203 13:25:14.535224 5052 reflector.go:368] Caches populated for *v1.Secret from object-"keystone-kuttl-tests"/"keystone-config-data" Dec 03 13:25:14 crc kubenswrapper[5052]: I1203 13:25:14.544457 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["keystone-kuttl-tests/keystone-bootstrap-x5gg9"] Dec 03 13:25:14 crc kubenswrapper[5052]: I1203 13:25:14.711410 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/11bc3563-46e9-4256-afa2-5ebc442af77e-fernet-keys\") pod \"keystone-bootstrap-x5gg9\" (UID: \"11bc3563-46e9-4256-afa2-5ebc442af77e\") " pod="keystone-kuttl-tests/keystone-bootstrap-x5gg9" Dec 03 13:25:14 crc kubenswrapper[5052]: I1203 13:25:14.711475 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bccbl\" (UniqueName: \"kubernetes.io/projected/11bc3563-46e9-4256-afa2-5ebc442af77e-kube-api-access-bccbl\") pod \"keystone-bootstrap-x5gg9\" (UID: \"11bc3563-46e9-4256-afa2-5ebc442af77e\") " pod="keystone-kuttl-tests/keystone-bootstrap-x5gg9" Dec 03 13:25:14 crc kubenswrapper[5052]: I1203 13:25:14.711571 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/11bc3563-46e9-4256-afa2-5ebc442af77e-credential-keys\") pod \"keystone-bootstrap-x5gg9\" (UID: \"11bc3563-46e9-4256-afa2-5ebc442af77e\") " pod="keystone-kuttl-tests/keystone-bootstrap-x5gg9" Dec 03 13:25:14 crc kubenswrapper[5052]: I1203 13:25:14.711898 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/11bc3563-46e9-4256-afa2-5ebc442af77e-scripts\") pod \"keystone-bootstrap-x5gg9\" (UID: \"11bc3563-46e9-4256-afa2-5ebc442af77e\") " pod="keystone-kuttl-tests/keystone-bootstrap-x5gg9" Dec 03 13:25:14 crc kubenswrapper[5052]: I1203 13:25:14.711970 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/11bc3563-46e9-4256-afa2-5ebc442af77e-config-data\") pod \"keystone-bootstrap-x5gg9\" (UID: \"11bc3563-46e9-4256-afa2-5ebc442af77e\") " pod="keystone-kuttl-tests/keystone-bootstrap-x5gg9" Dec 03 13:25:14 crc kubenswrapper[5052]: I1203 13:25:14.813336 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/11bc3563-46e9-4256-afa2-5ebc442af77e-scripts\") pod \"keystone-bootstrap-x5gg9\" (UID: \"11bc3563-46e9-4256-afa2-5ebc442af77e\") " pod="keystone-kuttl-tests/keystone-bootstrap-x5gg9" Dec 03 13:25:14 crc kubenswrapper[5052]: I1203 13:25:14.813391 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/11bc3563-46e9-4256-afa2-5ebc442af77e-config-data\") pod \"keystone-bootstrap-x5gg9\" (UID: \"11bc3563-46e9-4256-afa2-5ebc442af77e\") " pod="keystone-kuttl-tests/keystone-bootstrap-x5gg9" Dec 03 13:25:14 crc kubenswrapper[5052]: I1203 13:25:14.813423 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/11bc3563-46e9-4256-afa2-5ebc442af77e-fernet-keys\") pod \"keystone-bootstrap-x5gg9\" (UID: \"11bc3563-46e9-4256-afa2-5ebc442af77e\") " pod="keystone-kuttl-tests/keystone-bootstrap-x5gg9" Dec 03 13:25:14 crc kubenswrapper[5052]: I1203 13:25:14.813443 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-bccbl\" (UniqueName: \"kubernetes.io/projected/11bc3563-46e9-4256-afa2-5ebc442af77e-kube-api-access-bccbl\") pod \"keystone-bootstrap-x5gg9\" (UID: \"11bc3563-46e9-4256-afa2-5ebc442af77e\") " pod="keystone-kuttl-tests/keystone-bootstrap-x5gg9" Dec 03 13:25:14 crc kubenswrapper[5052]: I1203 13:25:14.813477 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/11bc3563-46e9-4256-afa2-5ebc442af77e-credential-keys\") pod \"keystone-bootstrap-x5gg9\" (UID: \"11bc3563-46e9-4256-afa2-5ebc442af77e\") " pod="keystone-kuttl-tests/keystone-bootstrap-x5gg9" Dec 03 13:25:14 crc kubenswrapper[5052]: I1203 13:25:14.817607 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/11bc3563-46e9-4256-afa2-5ebc442af77e-credential-keys\") pod \"keystone-bootstrap-x5gg9\" (UID: \"11bc3563-46e9-4256-afa2-5ebc442af77e\") " pod="keystone-kuttl-tests/keystone-bootstrap-x5gg9" Dec 03 13:25:14 crc kubenswrapper[5052]: I1203 13:25:14.817909 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/11bc3563-46e9-4256-afa2-5ebc442af77e-fernet-keys\") pod \"keystone-bootstrap-x5gg9\" (UID: \"11bc3563-46e9-4256-afa2-5ebc442af77e\") " pod="keystone-kuttl-tests/keystone-bootstrap-x5gg9" Dec 03 13:25:14 crc kubenswrapper[5052]: I1203 13:25:14.818063 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/11bc3563-46e9-4256-afa2-5ebc442af77e-config-data\") pod \"keystone-bootstrap-x5gg9\" (UID: \"11bc3563-46e9-4256-afa2-5ebc442af77e\") " pod="keystone-kuttl-tests/keystone-bootstrap-x5gg9" Dec 03 13:25:14 crc kubenswrapper[5052]: I1203 13:25:14.819698 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/11bc3563-46e9-4256-afa2-5ebc442af77e-scripts\") pod \"keystone-bootstrap-x5gg9\" (UID: \"11bc3563-46e9-4256-afa2-5ebc442af77e\") " pod="keystone-kuttl-tests/keystone-bootstrap-x5gg9" Dec 03 13:25:14 crc kubenswrapper[5052]: I1203 13:25:14.831674 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-bccbl\" (UniqueName: \"kubernetes.io/projected/11bc3563-46e9-4256-afa2-5ebc442af77e-kube-api-access-bccbl\") pod \"keystone-bootstrap-x5gg9\" (UID: \"11bc3563-46e9-4256-afa2-5ebc442af77e\") " pod="keystone-kuttl-tests/keystone-bootstrap-x5gg9" Dec 03 13:25:14 crc kubenswrapper[5052]: I1203 13:25:14.859149 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="keystone-kuttl-tests/keystone-bootstrap-x5gg9" Dec 03 13:25:15 crc kubenswrapper[5052]: I1203 13:25:15.285004 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["keystone-kuttl-tests/keystone-bootstrap-x5gg9"] Dec 03 13:25:15 crc kubenswrapper[5052]: I1203 13:25:15.368483 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="keystone-kuttl-tests/keystone-bootstrap-x5gg9" event={"ID":"11bc3563-46e9-4256-afa2-5ebc442af77e","Type":"ContainerStarted","Data":"04f90af43bcef87139ea1e450c88298a3bb84e8c54c35d1eb77d79b820b37fd6"} Dec 03 13:25:16 crc kubenswrapper[5052]: I1203 13:25:16.378912 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="keystone-kuttl-tests/keystone-bootstrap-x5gg9" event={"ID":"11bc3563-46e9-4256-afa2-5ebc442af77e","Type":"ContainerStarted","Data":"ccc93222bdd0ec8f4c393492fd2a2db39b61fc1a51c6abe7dc5153fe1fb0ec76"} Dec 03 13:25:16 crc kubenswrapper[5052]: I1203 13:25:16.418204 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="keystone-kuttl-tests/keystone-bootstrap-x5gg9" podStartSLOduration=2.418181556 podStartE2EDuration="2.418181556s" podCreationTimestamp="2025-12-03 13:25:14 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 13:25:16.392541601 +0000 UTC m=+7506.980783385" watchObservedRunningTime="2025-12-03 13:25:16.418181556 +0000 UTC m=+7507.006423320" Dec 03 13:25:18 crc kubenswrapper[5052]: I1203 13:25:18.398668 5052 generic.go:334] "Generic (PLEG): container finished" podID="11bc3563-46e9-4256-afa2-5ebc442af77e" containerID="ccc93222bdd0ec8f4c393492fd2a2db39b61fc1a51c6abe7dc5153fe1fb0ec76" exitCode=0 Dec 03 13:25:18 crc kubenswrapper[5052]: I1203 13:25:18.398748 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="keystone-kuttl-tests/keystone-bootstrap-x5gg9" event={"ID":"11bc3563-46e9-4256-afa2-5ebc442af77e","Type":"ContainerDied","Data":"ccc93222bdd0ec8f4c393492fd2a2db39b61fc1a51c6abe7dc5153fe1fb0ec76"} Dec 03 13:25:19 crc kubenswrapper[5052]: I1203 13:25:19.718082 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="keystone-kuttl-tests/keystone-bootstrap-x5gg9" Dec 03 13:25:19 crc kubenswrapper[5052]: I1203 13:25:19.902770 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/11bc3563-46e9-4256-afa2-5ebc442af77e-fernet-keys\") pod \"11bc3563-46e9-4256-afa2-5ebc442af77e\" (UID: \"11bc3563-46e9-4256-afa2-5ebc442af77e\") " Dec 03 13:25:19 crc kubenswrapper[5052]: I1203 13:25:19.902866 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/11bc3563-46e9-4256-afa2-5ebc442af77e-credential-keys\") pod \"11bc3563-46e9-4256-afa2-5ebc442af77e\" (UID: \"11bc3563-46e9-4256-afa2-5ebc442af77e\") " Dec 03 13:25:19 crc kubenswrapper[5052]: I1203 13:25:19.902903 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bccbl\" (UniqueName: \"kubernetes.io/projected/11bc3563-46e9-4256-afa2-5ebc442af77e-kube-api-access-bccbl\") pod \"11bc3563-46e9-4256-afa2-5ebc442af77e\" (UID: \"11bc3563-46e9-4256-afa2-5ebc442af77e\") " Dec 03 13:25:19 crc kubenswrapper[5052]: I1203 13:25:19.902939 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/11bc3563-46e9-4256-afa2-5ebc442af77e-scripts\") pod \"11bc3563-46e9-4256-afa2-5ebc442af77e\" (UID: \"11bc3563-46e9-4256-afa2-5ebc442af77e\") " Dec 03 13:25:19 crc kubenswrapper[5052]: I1203 13:25:19.902995 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/11bc3563-46e9-4256-afa2-5ebc442af77e-config-data\") pod \"11bc3563-46e9-4256-afa2-5ebc442af77e\" (UID: \"11bc3563-46e9-4256-afa2-5ebc442af77e\") " Dec 03 13:25:19 crc kubenswrapper[5052]: I1203 13:25:19.911203 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/11bc3563-46e9-4256-afa2-5ebc442af77e-credential-keys" (OuterVolumeSpecName: "credential-keys") pod "11bc3563-46e9-4256-afa2-5ebc442af77e" (UID: "11bc3563-46e9-4256-afa2-5ebc442af77e"). InnerVolumeSpecName "credential-keys". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:25:19 crc kubenswrapper[5052]: I1203 13:25:19.912000 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/11bc3563-46e9-4256-afa2-5ebc442af77e-scripts" (OuterVolumeSpecName: "scripts") pod "11bc3563-46e9-4256-afa2-5ebc442af77e" (UID: "11bc3563-46e9-4256-afa2-5ebc442af77e"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:25:19 crc kubenswrapper[5052]: I1203 13:25:19.912063 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/11bc3563-46e9-4256-afa2-5ebc442af77e-fernet-keys" (OuterVolumeSpecName: "fernet-keys") pod "11bc3563-46e9-4256-afa2-5ebc442af77e" (UID: "11bc3563-46e9-4256-afa2-5ebc442af77e"). InnerVolumeSpecName "fernet-keys". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:25:19 crc kubenswrapper[5052]: I1203 13:25:19.912175 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/11bc3563-46e9-4256-afa2-5ebc442af77e-kube-api-access-bccbl" (OuterVolumeSpecName: "kube-api-access-bccbl") pod "11bc3563-46e9-4256-afa2-5ebc442af77e" (UID: "11bc3563-46e9-4256-afa2-5ebc442af77e"). InnerVolumeSpecName "kube-api-access-bccbl". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:25:19 crc kubenswrapper[5052]: I1203 13:25:19.939779 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/11bc3563-46e9-4256-afa2-5ebc442af77e-config-data" (OuterVolumeSpecName: "config-data") pod "11bc3563-46e9-4256-afa2-5ebc442af77e" (UID: "11bc3563-46e9-4256-afa2-5ebc442af77e"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:25:20 crc kubenswrapper[5052]: I1203 13:25:20.004152 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/11bc3563-46e9-4256-afa2-5ebc442af77e-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 13:25:20 crc kubenswrapper[5052]: I1203 13:25:20.004202 5052 reconciler_common.go:293] "Volume detached for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/11bc3563-46e9-4256-afa2-5ebc442af77e-fernet-keys\") on node \"crc\" DevicePath \"\"" Dec 03 13:25:20 crc kubenswrapper[5052]: I1203 13:25:20.004211 5052 reconciler_common.go:293] "Volume detached for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/11bc3563-46e9-4256-afa2-5ebc442af77e-credential-keys\") on node \"crc\" DevicePath \"\"" Dec 03 13:25:20 crc kubenswrapper[5052]: I1203 13:25:20.004227 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-bccbl\" (UniqueName: \"kubernetes.io/projected/11bc3563-46e9-4256-afa2-5ebc442af77e-kube-api-access-bccbl\") on node \"crc\" DevicePath \"\"" Dec 03 13:25:20 crc kubenswrapper[5052]: I1203 13:25:20.004238 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/11bc3563-46e9-4256-afa2-5ebc442af77e-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 13:25:20 crc kubenswrapper[5052]: I1203 13:25:20.418411 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="keystone-kuttl-tests/keystone-bootstrap-x5gg9" event={"ID":"11bc3563-46e9-4256-afa2-5ebc442af77e","Type":"ContainerDied","Data":"04f90af43bcef87139ea1e450c88298a3bb84e8c54c35d1eb77d79b820b37fd6"} Dec 03 13:25:20 crc kubenswrapper[5052]: I1203 13:25:20.418454 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="04f90af43bcef87139ea1e450c88298a3bb84e8c54c35d1eb77d79b820b37fd6" Dec 03 13:25:20 crc kubenswrapper[5052]: I1203 13:25:20.418480 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="keystone-kuttl-tests/keystone-bootstrap-x5gg9" Dec 03 13:25:20 crc kubenswrapper[5052]: I1203 13:25:20.505463 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["keystone-kuttl-tests/keystone-6b94f77657-72gm8"] Dec 03 13:25:20 crc kubenswrapper[5052]: E1203 13:25:20.505900 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="11bc3563-46e9-4256-afa2-5ebc442af77e" containerName="keystone-bootstrap" Dec 03 13:25:20 crc kubenswrapper[5052]: I1203 13:25:20.505925 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="11bc3563-46e9-4256-afa2-5ebc442af77e" containerName="keystone-bootstrap" Dec 03 13:25:20 crc kubenswrapper[5052]: I1203 13:25:20.506102 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="11bc3563-46e9-4256-afa2-5ebc442af77e" containerName="keystone-bootstrap" Dec 03 13:25:20 crc kubenswrapper[5052]: I1203 13:25:20.506730 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="keystone-kuttl-tests/keystone-6b94f77657-72gm8" Dec 03 13:25:20 crc kubenswrapper[5052]: I1203 13:25:20.508818 5052 reflector.go:368] Caches populated for *v1.Secret from object-"keystone-kuttl-tests"/"keystone-keystone-dockercfg-4pnrz" Dec 03 13:25:20 crc kubenswrapper[5052]: I1203 13:25:20.508859 5052 reflector.go:368] Caches populated for *v1.Secret from object-"keystone-kuttl-tests"/"keystone-config-data" Dec 03 13:25:20 crc kubenswrapper[5052]: I1203 13:25:20.510352 5052 reflector.go:368] Caches populated for *v1.Secret from object-"keystone-kuttl-tests"/"keystone-scripts" Dec 03 13:25:20 crc kubenswrapper[5052]: I1203 13:25:20.511300 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/7bdc1a91-11f2-48ff-a2a5-f26606395613-credential-keys\") pod \"keystone-6b94f77657-72gm8\" (UID: \"7bdc1a91-11f2-48ff-a2a5-f26606395613\") " pod="keystone-kuttl-tests/keystone-6b94f77657-72gm8" Dec 03 13:25:20 crc kubenswrapper[5052]: I1203 13:25:20.511390 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-skwn9\" (UniqueName: \"kubernetes.io/projected/7bdc1a91-11f2-48ff-a2a5-f26606395613-kube-api-access-skwn9\") pod \"keystone-6b94f77657-72gm8\" (UID: \"7bdc1a91-11f2-48ff-a2a5-f26606395613\") " pod="keystone-kuttl-tests/keystone-6b94f77657-72gm8" Dec 03 13:25:20 crc kubenswrapper[5052]: I1203 13:25:20.511415 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/7bdc1a91-11f2-48ff-a2a5-f26606395613-config-data\") pod \"keystone-6b94f77657-72gm8\" (UID: \"7bdc1a91-11f2-48ff-a2a5-f26606395613\") " pod="keystone-kuttl-tests/keystone-6b94f77657-72gm8" Dec 03 13:25:20 crc kubenswrapper[5052]: I1203 13:25:20.511474 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/7bdc1a91-11f2-48ff-a2a5-f26606395613-scripts\") pod \"keystone-6b94f77657-72gm8\" (UID: \"7bdc1a91-11f2-48ff-a2a5-f26606395613\") " pod="keystone-kuttl-tests/keystone-6b94f77657-72gm8" Dec 03 13:25:20 crc kubenswrapper[5052]: I1203 13:25:20.511498 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/7bdc1a91-11f2-48ff-a2a5-f26606395613-fernet-keys\") pod \"keystone-6b94f77657-72gm8\" (UID: \"7bdc1a91-11f2-48ff-a2a5-f26606395613\") " pod="keystone-kuttl-tests/keystone-6b94f77657-72gm8" Dec 03 13:25:20 crc kubenswrapper[5052]: I1203 13:25:20.512451 5052 reflector.go:368] Caches populated for *v1.Secret from object-"keystone-kuttl-tests"/"keystone" Dec 03 13:25:20 crc kubenswrapper[5052]: I1203 13:25:20.522394 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["keystone-kuttl-tests/keystone-6b94f77657-72gm8"] Dec 03 13:25:20 crc kubenswrapper[5052]: I1203 13:25:20.612715 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-skwn9\" (UniqueName: \"kubernetes.io/projected/7bdc1a91-11f2-48ff-a2a5-f26606395613-kube-api-access-skwn9\") pod \"keystone-6b94f77657-72gm8\" (UID: \"7bdc1a91-11f2-48ff-a2a5-f26606395613\") " pod="keystone-kuttl-tests/keystone-6b94f77657-72gm8" Dec 03 13:25:20 crc kubenswrapper[5052]: I1203 13:25:20.612963 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/7bdc1a91-11f2-48ff-a2a5-f26606395613-config-data\") pod \"keystone-6b94f77657-72gm8\" (UID: \"7bdc1a91-11f2-48ff-a2a5-f26606395613\") " pod="keystone-kuttl-tests/keystone-6b94f77657-72gm8" Dec 03 13:25:20 crc kubenswrapper[5052]: I1203 13:25:20.613228 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/7bdc1a91-11f2-48ff-a2a5-f26606395613-scripts\") pod \"keystone-6b94f77657-72gm8\" (UID: \"7bdc1a91-11f2-48ff-a2a5-f26606395613\") " pod="keystone-kuttl-tests/keystone-6b94f77657-72gm8" Dec 03 13:25:20 crc kubenswrapper[5052]: I1203 13:25:20.613325 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/7bdc1a91-11f2-48ff-a2a5-f26606395613-fernet-keys\") pod \"keystone-6b94f77657-72gm8\" (UID: \"7bdc1a91-11f2-48ff-a2a5-f26606395613\") " pod="keystone-kuttl-tests/keystone-6b94f77657-72gm8" Dec 03 13:25:20 crc kubenswrapper[5052]: I1203 13:25:20.613442 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/7bdc1a91-11f2-48ff-a2a5-f26606395613-credential-keys\") pod \"keystone-6b94f77657-72gm8\" (UID: \"7bdc1a91-11f2-48ff-a2a5-f26606395613\") " pod="keystone-kuttl-tests/keystone-6b94f77657-72gm8" Dec 03 13:25:20 crc kubenswrapper[5052]: I1203 13:25:20.617362 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/7bdc1a91-11f2-48ff-a2a5-f26606395613-scripts\") pod \"keystone-6b94f77657-72gm8\" (UID: \"7bdc1a91-11f2-48ff-a2a5-f26606395613\") " pod="keystone-kuttl-tests/keystone-6b94f77657-72gm8" Dec 03 13:25:20 crc kubenswrapper[5052]: I1203 13:25:20.617499 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/7bdc1a91-11f2-48ff-a2a5-f26606395613-fernet-keys\") pod \"keystone-6b94f77657-72gm8\" (UID: \"7bdc1a91-11f2-48ff-a2a5-f26606395613\") " pod="keystone-kuttl-tests/keystone-6b94f77657-72gm8" Dec 03 13:25:20 crc kubenswrapper[5052]: I1203 13:25:20.617783 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/7bdc1a91-11f2-48ff-a2a5-f26606395613-config-data\") pod \"keystone-6b94f77657-72gm8\" (UID: \"7bdc1a91-11f2-48ff-a2a5-f26606395613\") " pod="keystone-kuttl-tests/keystone-6b94f77657-72gm8" Dec 03 13:25:20 crc kubenswrapper[5052]: I1203 13:25:20.619137 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/7bdc1a91-11f2-48ff-a2a5-f26606395613-credential-keys\") pod \"keystone-6b94f77657-72gm8\" (UID: \"7bdc1a91-11f2-48ff-a2a5-f26606395613\") " pod="keystone-kuttl-tests/keystone-6b94f77657-72gm8" Dec 03 13:25:20 crc kubenswrapper[5052]: I1203 13:25:20.632961 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-skwn9\" (UniqueName: \"kubernetes.io/projected/7bdc1a91-11f2-48ff-a2a5-f26606395613-kube-api-access-skwn9\") pod \"keystone-6b94f77657-72gm8\" (UID: \"7bdc1a91-11f2-48ff-a2a5-f26606395613\") " pod="keystone-kuttl-tests/keystone-6b94f77657-72gm8" Dec 03 13:25:20 crc kubenswrapper[5052]: I1203 13:25:20.826970 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="keystone-kuttl-tests/keystone-6b94f77657-72gm8" Dec 03 13:25:21 crc kubenswrapper[5052]: I1203 13:25:21.290507 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["keystone-kuttl-tests/keystone-6b94f77657-72gm8"] Dec 03 13:25:21 crc kubenswrapper[5052]: I1203 13:25:21.433157 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="keystone-kuttl-tests/keystone-6b94f77657-72gm8" event={"ID":"7bdc1a91-11f2-48ff-a2a5-f26606395613","Type":"ContainerStarted","Data":"a3260335673fd00780ee9021736de4a0b437b8d7bd92fde8abfeddded2167246"} Dec 03 13:25:22 crc kubenswrapper[5052]: I1203 13:25:22.442022 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="keystone-kuttl-tests/keystone-6b94f77657-72gm8" event={"ID":"7bdc1a91-11f2-48ff-a2a5-f26606395613","Type":"ContainerStarted","Data":"cf529eea53b2ebc69e317bc10d7f707894210ad8e2d39e0a77271d5c0aa80d48"} Dec 03 13:25:22 crc kubenswrapper[5052]: I1203 13:25:22.442398 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="keystone-kuttl-tests/keystone-6b94f77657-72gm8" Dec 03 13:25:22 crc kubenswrapper[5052]: I1203 13:25:22.470421 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="keystone-kuttl-tests/keystone-6b94f77657-72gm8" podStartSLOduration=2.470397157 podStartE2EDuration="2.470397157s" podCreationTimestamp="2025-12-03 13:25:20 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 13:25:22.464477628 +0000 UTC m=+7513.052719402" watchObservedRunningTime="2025-12-03 13:25:22.470397157 +0000 UTC m=+7513.058638921" Dec 03 13:25:29 crc kubenswrapper[5052]: I1203 13:25:29.379626 5052 patch_prober.go:28] interesting pod/machine-config-daemon-74bj6 container/machine-config-daemon namespace/openshift-machine-config-operator: Liveness probe status=failure output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" start-of-body= Dec 03 13:25:29 crc kubenswrapper[5052]: I1203 13:25:29.380035 5052 prober.go:107] "Probe failed" probeType="Liveness" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" containerName="machine-config-daemon" probeResult="failure" output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" Dec 03 13:25:52 crc kubenswrapper[5052]: I1203 13:25:52.381764 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="keystone-kuttl-tests/keystone-6b94f77657-72gm8" Dec 03 13:25:53 crc kubenswrapper[5052]: I1203 13:25:53.777774 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["keystone-kuttl-tests/keystone-6b94f77657-hqt2p"] Dec 03 13:25:53 crc kubenswrapper[5052]: I1203 13:25:53.778991 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="keystone-kuttl-tests/keystone-6b94f77657-hqt2p" Dec 03 13:25:53 crc kubenswrapper[5052]: I1203 13:25:53.785906 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["keystone-kuttl-tests/keystone-6b94f77657-fg5j2"] Dec 03 13:25:53 crc kubenswrapper[5052]: I1203 13:25:53.787147 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="keystone-kuttl-tests/keystone-6b94f77657-fg5j2" Dec 03 13:25:53 crc kubenswrapper[5052]: I1203 13:25:53.793605 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["keystone-kuttl-tests/keystone-6b94f77657-hqt2p"] Dec 03 13:25:53 crc kubenswrapper[5052]: I1203 13:25:53.801085 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["keystone-kuttl-tests/keystone-6b94f77657-fg5j2"] Dec 03 13:25:53 crc kubenswrapper[5052]: I1203 13:25:53.965740 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/c9e8b9fe-4bb0-42f2-aec5-47b93b283868-fernet-keys\") pod \"keystone-6b94f77657-fg5j2\" (UID: \"c9e8b9fe-4bb0-42f2-aec5-47b93b283868\") " pod="keystone-kuttl-tests/keystone-6b94f77657-fg5j2" Dec 03 13:25:53 crc kubenswrapper[5052]: I1203 13:25:53.966335 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/c9e8b9fe-4bb0-42f2-aec5-47b93b283868-scripts\") pod \"keystone-6b94f77657-fg5j2\" (UID: \"c9e8b9fe-4bb0-42f2-aec5-47b93b283868\") " pod="keystone-kuttl-tests/keystone-6b94f77657-fg5j2" Dec 03 13:25:53 crc kubenswrapper[5052]: I1203 13:25:53.966473 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/6844c5ca-9f10-46af-b462-f54319ff59ea-fernet-keys\") pod \"keystone-6b94f77657-hqt2p\" (UID: \"6844c5ca-9f10-46af-b462-f54319ff59ea\") " pod="keystone-kuttl-tests/keystone-6b94f77657-hqt2p" Dec 03 13:25:53 crc kubenswrapper[5052]: I1203 13:25:53.966608 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/c9e8b9fe-4bb0-42f2-aec5-47b93b283868-credential-keys\") pod \"keystone-6b94f77657-fg5j2\" (UID: \"c9e8b9fe-4bb0-42f2-aec5-47b93b283868\") " pod="keystone-kuttl-tests/keystone-6b94f77657-fg5j2" Dec 03 13:25:53 crc kubenswrapper[5052]: I1203 13:25:53.966760 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-98vt7\" (UniqueName: \"kubernetes.io/projected/6844c5ca-9f10-46af-b462-f54319ff59ea-kube-api-access-98vt7\") pod \"keystone-6b94f77657-hqt2p\" (UID: \"6844c5ca-9f10-46af-b462-f54319ff59ea\") " pod="keystone-kuttl-tests/keystone-6b94f77657-hqt2p" Dec 03 13:25:53 crc kubenswrapper[5052]: I1203 13:25:53.966823 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/c9e8b9fe-4bb0-42f2-aec5-47b93b283868-config-data\") pod \"keystone-6b94f77657-fg5j2\" (UID: \"c9e8b9fe-4bb0-42f2-aec5-47b93b283868\") " pod="keystone-kuttl-tests/keystone-6b94f77657-fg5j2" Dec 03 13:25:53 crc kubenswrapper[5052]: I1203 13:25:53.966862 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2wxlg\" (UniqueName: \"kubernetes.io/projected/c9e8b9fe-4bb0-42f2-aec5-47b93b283868-kube-api-access-2wxlg\") pod \"keystone-6b94f77657-fg5j2\" (UID: \"c9e8b9fe-4bb0-42f2-aec5-47b93b283868\") " pod="keystone-kuttl-tests/keystone-6b94f77657-fg5j2" Dec 03 13:25:53 crc kubenswrapper[5052]: I1203 13:25:53.966882 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/6844c5ca-9f10-46af-b462-f54319ff59ea-scripts\") pod \"keystone-6b94f77657-hqt2p\" (UID: \"6844c5ca-9f10-46af-b462-f54319ff59ea\") " pod="keystone-kuttl-tests/keystone-6b94f77657-hqt2p" Dec 03 13:25:53 crc kubenswrapper[5052]: I1203 13:25:53.966901 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6844c5ca-9f10-46af-b462-f54319ff59ea-config-data\") pod \"keystone-6b94f77657-hqt2p\" (UID: \"6844c5ca-9f10-46af-b462-f54319ff59ea\") " pod="keystone-kuttl-tests/keystone-6b94f77657-hqt2p" Dec 03 13:25:53 crc kubenswrapper[5052]: I1203 13:25:53.967009 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/6844c5ca-9f10-46af-b462-f54319ff59ea-credential-keys\") pod \"keystone-6b94f77657-hqt2p\" (UID: \"6844c5ca-9f10-46af-b462-f54319ff59ea\") " pod="keystone-kuttl-tests/keystone-6b94f77657-hqt2p" Dec 03 13:25:54 crc kubenswrapper[5052]: I1203 13:25:54.068160 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/c9e8b9fe-4bb0-42f2-aec5-47b93b283868-config-data\") pod \"keystone-6b94f77657-fg5j2\" (UID: \"c9e8b9fe-4bb0-42f2-aec5-47b93b283868\") " pod="keystone-kuttl-tests/keystone-6b94f77657-fg5j2" Dec 03 13:25:54 crc kubenswrapper[5052]: I1203 13:25:54.068249 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-2wxlg\" (UniqueName: \"kubernetes.io/projected/c9e8b9fe-4bb0-42f2-aec5-47b93b283868-kube-api-access-2wxlg\") pod \"keystone-6b94f77657-fg5j2\" (UID: \"c9e8b9fe-4bb0-42f2-aec5-47b93b283868\") " pod="keystone-kuttl-tests/keystone-6b94f77657-fg5j2" Dec 03 13:25:54 crc kubenswrapper[5052]: I1203 13:25:54.068275 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/6844c5ca-9f10-46af-b462-f54319ff59ea-scripts\") pod \"keystone-6b94f77657-hqt2p\" (UID: \"6844c5ca-9f10-46af-b462-f54319ff59ea\") " pod="keystone-kuttl-tests/keystone-6b94f77657-hqt2p" Dec 03 13:25:54 crc kubenswrapper[5052]: I1203 13:25:54.068299 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6844c5ca-9f10-46af-b462-f54319ff59ea-config-data\") pod \"keystone-6b94f77657-hqt2p\" (UID: \"6844c5ca-9f10-46af-b462-f54319ff59ea\") " pod="keystone-kuttl-tests/keystone-6b94f77657-hqt2p" Dec 03 13:25:54 crc kubenswrapper[5052]: I1203 13:25:54.068330 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/6844c5ca-9f10-46af-b462-f54319ff59ea-credential-keys\") pod \"keystone-6b94f77657-hqt2p\" (UID: \"6844c5ca-9f10-46af-b462-f54319ff59ea\") " pod="keystone-kuttl-tests/keystone-6b94f77657-hqt2p" Dec 03 13:25:54 crc kubenswrapper[5052]: I1203 13:25:54.068392 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/c9e8b9fe-4bb0-42f2-aec5-47b93b283868-fernet-keys\") pod \"keystone-6b94f77657-fg5j2\" (UID: \"c9e8b9fe-4bb0-42f2-aec5-47b93b283868\") " pod="keystone-kuttl-tests/keystone-6b94f77657-fg5j2" Dec 03 13:25:54 crc kubenswrapper[5052]: I1203 13:25:54.068415 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/c9e8b9fe-4bb0-42f2-aec5-47b93b283868-scripts\") pod \"keystone-6b94f77657-fg5j2\" (UID: \"c9e8b9fe-4bb0-42f2-aec5-47b93b283868\") " pod="keystone-kuttl-tests/keystone-6b94f77657-fg5j2" Dec 03 13:25:54 crc kubenswrapper[5052]: I1203 13:25:54.068457 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/6844c5ca-9f10-46af-b462-f54319ff59ea-fernet-keys\") pod \"keystone-6b94f77657-hqt2p\" (UID: \"6844c5ca-9f10-46af-b462-f54319ff59ea\") " pod="keystone-kuttl-tests/keystone-6b94f77657-hqt2p" Dec 03 13:25:54 crc kubenswrapper[5052]: I1203 13:25:54.068486 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/c9e8b9fe-4bb0-42f2-aec5-47b93b283868-credential-keys\") pod \"keystone-6b94f77657-fg5j2\" (UID: \"c9e8b9fe-4bb0-42f2-aec5-47b93b283868\") " pod="keystone-kuttl-tests/keystone-6b94f77657-fg5j2" Dec 03 13:25:54 crc kubenswrapper[5052]: I1203 13:25:54.068548 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-98vt7\" (UniqueName: \"kubernetes.io/projected/6844c5ca-9f10-46af-b462-f54319ff59ea-kube-api-access-98vt7\") pod \"keystone-6b94f77657-hqt2p\" (UID: \"6844c5ca-9f10-46af-b462-f54319ff59ea\") " pod="keystone-kuttl-tests/keystone-6b94f77657-hqt2p" Dec 03 13:25:54 crc kubenswrapper[5052]: I1203 13:25:54.076048 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/6844c5ca-9f10-46af-b462-f54319ff59ea-credential-keys\") pod \"keystone-6b94f77657-hqt2p\" (UID: \"6844c5ca-9f10-46af-b462-f54319ff59ea\") " pod="keystone-kuttl-tests/keystone-6b94f77657-hqt2p" Dec 03 13:25:54 crc kubenswrapper[5052]: I1203 13:25:54.076169 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/6844c5ca-9f10-46af-b462-f54319ff59ea-fernet-keys\") pod \"keystone-6b94f77657-hqt2p\" (UID: \"6844c5ca-9f10-46af-b462-f54319ff59ea\") " pod="keystone-kuttl-tests/keystone-6b94f77657-hqt2p" Dec 03 13:25:54 crc kubenswrapper[5052]: I1203 13:25:54.076237 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/c9e8b9fe-4bb0-42f2-aec5-47b93b283868-config-data\") pod \"keystone-6b94f77657-fg5j2\" (UID: \"c9e8b9fe-4bb0-42f2-aec5-47b93b283868\") " pod="keystone-kuttl-tests/keystone-6b94f77657-fg5j2" Dec 03 13:25:54 crc kubenswrapper[5052]: I1203 13:25:54.076498 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/c9e8b9fe-4bb0-42f2-aec5-47b93b283868-scripts\") pod \"keystone-6b94f77657-fg5j2\" (UID: \"c9e8b9fe-4bb0-42f2-aec5-47b93b283868\") " pod="keystone-kuttl-tests/keystone-6b94f77657-fg5j2" Dec 03 13:25:54 crc kubenswrapper[5052]: I1203 13:25:54.077439 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/c9e8b9fe-4bb0-42f2-aec5-47b93b283868-fernet-keys\") pod \"keystone-6b94f77657-fg5j2\" (UID: \"c9e8b9fe-4bb0-42f2-aec5-47b93b283868\") " pod="keystone-kuttl-tests/keystone-6b94f77657-fg5j2" Dec 03 13:25:54 crc kubenswrapper[5052]: I1203 13:25:54.077724 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6844c5ca-9f10-46af-b462-f54319ff59ea-config-data\") pod \"keystone-6b94f77657-hqt2p\" (UID: \"6844c5ca-9f10-46af-b462-f54319ff59ea\") " pod="keystone-kuttl-tests/keystone-6b94f77657-hqt2p" Dec 03 13:25:54 crc kubenswrapper[5052]: I1203 13:25:54.078586 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/c9e8b9fe-4bb0-42f2-aec5-47b93b283868-credential-keys\") pod \"keystone-6b94f77657-fg5j2\" (UID: \"c9e8b9fe-4bb0-42f2-aec5-47b93b283868\") " pod="keystone-kuttl-tests/keystone-6b94f77657-fg5j2" Dec 03 13:25:54 crc kubenswrapper[5052]: I1203 13:25:54.087599 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/6844c5ca-9f10-46af-b462-f54319ff59ea-scripts\") pod \"keystone-6b94f77657-hqt2p\" (UID: \"6844c5ca-9f10-46af-b462-f54319ff59ea\") " pod="keystone-kuttl-tests/keystone-6b94f77657-hqt2p" Dec 03 13:25:54 crc kubenswrapper[5052]: I1203 13:25:54.089494 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-98vt7\" (UniqueName: \"kubernetes.io/projected/6844c5ca-9f10-46af-b462-f54319ff59ea-kube-api-access-98vt7\") pod \"keystone-6b94f77657-hqt2p\" (UID: \"6844c5ca-9f10-46af-b462-f54319ff59ea\") " pod="keystone-kuttl-tests/keystone-6b94f77657-hqt2p" Dec 03 13:25:54 crc kubenswrapper[5052]: I1203 13:25:54.090018 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-2wxlg\" (UniqueName: \"kubernetes.io/projected/c9e8b9fe-4bb0-42f2-aec5-47b93b283868-kube-api-access-2wxlg\") pod \"keystone-6b94f77657-fg5j2\" (UID: \"c9e8b9fe-4bb0-42f2-aec5-47b93b283868\") " pod="keystone-kuttl-tests/keystone-6b94f77657-fg5j2" Dec 03 13:25:54 crc kubenswrapper[5052]: I1203 13:25:54.104392 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="keystone-kuttl-tests/keystone-6b94f77657-hqt2p" Dec 03 13:25:54 crc kubenswrapper[5052]: I1203 13:25:54.117992 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="keystone-kuttl-tests/keystone-6b94f77657-fg5j2" Dec 03 13:25:54 crc kubenswrapper[5052]: I1203 13:25:54.387597 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["keystone-kuttl-tests/keystone-6b94f77657-fg5j2"] Dec 03 13:25:54 crc kubenswrapper[5052]: W1203 13:25:54.579015 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6844c5ca_9f10_46af_b462_f54319ff59ea.slice/crio-c5d8a5e91232a83a4ef806bfceb5d1b25dd1bd53059d8c5f9d1cc19a44d243c9 WatchSource:0}: Error finding container c5d8a5e91232a83a4ef806bfceb5d1b25dd1bd53059d8c5f9d1cc19a44d243c9: Status 404 returned error can't find the container with id c5d8a5e91232a83a4ef806bfceb5d1b25dd1bd53059d8c5f9d1cc19a44d243c9 Dec 03 13:25:54 crc kubenswrapper[5052]: I1203 13:25:54.579123 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["keystone-kuttl-tests/keystone-6b94f77657-hqt2p"] Dec 03 13:25:54 crc kubenswrapper[5052]: I1203 13:25:54.721619 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="keystone-kuttl-tests/keystone-6b94f77657-hqt2p" event={"ID":"6844c5ca-9f10-46af-b462-f54319ff59ea","Type":"ContainerStarted","Data":"2ab9679d66c2bddc779455e79fda921b12f80e7c03e6109d792923508b0d0f9f"} Dec 03 13:25:54 crc kubenswrapper[5052]: I1203 13:25:54.721658 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="keystone-kuttl-tests/keystone-6b94f77657-hqt2p" event={"ID":"6844c5ca-9f10-46af-b462-f54319ff59ea","Type":"ContainerStarted","Data":"c5d8a5e91232a83a4ef806bfceb5d1b25dd1bd53059d8c5f9d1cc19a44d243c9"} Dec 03 13:25:54 crc kubenswrapper[5052]: I1203 13:25:54.722468 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="keystone-kuttl-tests/keystone-6b94f77657-hqt2p" Dec 03 13:25:54 crc kubenswrapper[5052]: I1203 13:25:54.724540 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="keystone-kuttl-tests/keystone-6b94f77657-fg5j2" event={"ID":"c9e8b9fe-4bb0-42f2-aec5-47b93b283868","Type":"ContainerStarted","Data":"8ea4edd4258fec190e0d96af0f029e657a89ae8516a0b72b3a4e210cf596038e"} Dec 03 13:25:54 crc kubenswrapper[5052]: I1203 13:25:54.724594 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="keystone-kuttl-tests/keystone-6b94f77657-fg5j2" event={"ID":"c9e8b9fe-4bb0-42f2-aec5-47b93b283868","Type":"ContainerStarted","Data":"207592b01bd0967733fbc1b8c446c555a320660540b1eac884e50d8ac280a607"} Dec 03 13:25:54 crc kubenswrapper[5052]: I1203 13:25:54.724798 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="keystone-kuttl-tests/keystone-6b94f77657-fg5j2" Dec 03 13:25:54 crc kubenswrapper[5052]: I1203 13:25:54.741417 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="keystone-kuttl-tests/keystone-6b94f77657-hqt2p" podStartSLOduration=1.741397815 podStartE2EDuration="1.741397815s" podCreationTimestamp="2025-12-03 13:25:53 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 13:25:54.740627533 +0000 UTC m=+7545.328869317" watchObservedRunningTime="2025-12-03 13:25:54.741397815 +0000 UTC m=+7545.329639579" Dec 03 13:25:54 crc kubenswrapper[5052]: I1203 13:25:54.765541 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="keystone-kuttl-tests/keystone-6b94f77657-fg5j2" podStartSLOduration=1.765502926 podStartE2EDuration="1.765502926s" podCreationTimestamp="2025-12-03 13:25:53 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 13:25:54.761587483 +0000 UTC m=+7545.349829247" watchObservedRunningTime="2025-12-03 13:25:54.765502926 +0000 UTC m=+7545.353744690" Dec 03 13:25:59 crc kubenswrapper[5052]: I1203 13:25:59.379890 5052 patch_prober.go:28] interesting pod/machine-config-daemon-74bj6 container/machine-config-daemon namespace/openshift-machine-config-operator: Liveness probe status=failure output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" start-of-body= Dec 03 13:25:59 crc kubenswrapper[5052]: I1203 13:25:59.380484 5052 prober.go:107] "Probe failed" probeType="Liveness" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" containerName="machine-config-daemon" probeResult="failure" output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" Dec 03 13:26:15 crc kubenswrapper[5052]: I1203 13:26:15.065503 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openshift-marketplace/community-operators-ntppl"] Dec 03 13:26:15 crc kubenswrapper[5052]: I1203 13:26:15.067859 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/community-operators-ntppl" Dec 03 13:26:15 crc kubenswrapper[5052]: I1203 13:26:15.081832 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/community-operators-ntppl"] Dec 03 13:26:15 crc kubenswrapper[5052]: I1203 13:26:15.149319 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/2d8df6a3-ab5e-4c69-9f03-3fa57c872ad9-utilities\") pod \"community-operators-ntppl\" (UID: \"2d8df6a3-ab5e-4c69-9f03-3fa57c872ad9\") " pod="openshift-marketplace/community-operators-ntppl" Dec 03 13:26:15 crc kubenswrapper[5052]: I1203 13:26:15.149410 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-46c82\" (UniqueName: \"kubernetes.io/projected/2d8df6a3-ab5e-4c69-9f03-3fa57c872ad9-kube-api-access-46c82\") pod \"community-operators-ntppl\" (UID: \"2d8df6a3-ab5e-4c69-9f03-3fa57c872ad9\") " pod="openshift-marketplace/community-operators-ntppl" Dec 03 13:26:15 crc kubenswrapper[5052]: I1203 13:26:15.149437 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/2d8df6a3-ab5e-4c69-9f03-3fa57c872ad9-catalog-content\") pod \"community-operators-ntppl\" (UID: \"2d8df6a3-ab5e-4c69-9f03-3fa57c872ad9\") " pod="openshift-marketplace/community-operators-ntppl" Dec 03 13:26:15 crc kubenswrapper[5052]: I1203 13:26:15.250568 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/2d8df6a3-ab5e-4c69-9f03-3fa57c872ad9-catalog-content\") pod \"community-operators-ntppl\" (UID: \"2d8df6a3-ab5e-4c69-9f03-3fa57c872ad9\") " pod="openshift-marketplace/community-operators-ntppl" Dec 03 13:26:15 crc kubenswrapper[5052]: I1203 13:26:15.250870 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-46c82\" (UniqueName: \"kubernetes.io/projected/2d8df6a3-ab5e-4c69-9f03-3fa57c872ad9-kube-api-access-46c82\") pod \"community-operators-ntppl\" (UID: \"2d8df6a3-ab5e-4c69-9f03-3fa57c872ad9\") " pod="openshift-marketplace/community-operators-ntppl" Dec 03 13:26:15 crc kubenswrapper[5052]: I1203 13:26:15.251067 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/2d8df6a3-ab5e-4c69-9f03-3fa57c872ad9-utilities\") pod \"community-operators-ntppl\" (UID: \"2d8df6a3-ab5e-4c69-9f03-3fa57c872ad9\") " pod="openshift-marketplace/community-operators-ntppl" Dec 03 13:26:15 crc kubenswrapper[5052]: I1203 13:26:15.251126 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/2d8df6a3-ab5e-4c69-9f03-3fa57c872ad9-catalog-content\") pod \"community-operators-ntppl\" (UID: \"2d8df6a3-ab5e-4c69-9f03-3fa57c872ad9\") " pod="openshift-marketplace/community-operators-ntppl" Dec 03 13:26:15 crc kubenswrapper[5052]: I1203 13:26:15.251440 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/2d8df6a3-ab5e-4c69-9f03-3fa57c872ad9-utilities\") pod \"community-operators-ntppl\" (UID: \"2d8df6a3-ab5e-4c69-9f03-3fa57c872ad9\") " pod="openshift-marketplace/community-operators-ntppl" Dec 03 13:26:15 crc kubenswrapper[5052]: I1203 13:26:15.276450 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-46c82\" (UniqueName: \"kubernetes.io/projected/2d8df6a3-ab5e-4c69-9f03-3fa57c872ad9-kube-api-access-46c82\") pod \"community-operators-ntppl\" (UID: \"2d8df6a3-ab5e-4c69-9f03-3fa57c872ad9\") " pod="openshift-marketplace/community-operators-ntppl" Dec 03 13:26:15 crc kubenswrapper[5052]: I1203 13:26:15.392560 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/community-operators-ntppl" Dec 03 13:26:15 crc kubenswrapper[5052]: I1203 13:26:15.886768 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/community-operators-ntppl"] Dec 03 13:26:15 crc kubenswrapper[5052]: W1203 13:26:15.896388 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d8df6a3_ab5e_4c69_9f03_3fa57c872ad9.slice/crio-aefb7e965e06938769edc8d1306eab48d628e2cb8ec73c2aad2b6f7aa13828bf WatchSource:0}: Error finding container aefb7e965e06938769edc8d1306eab48d628e2cb8ec73c2aad2b6f7aa13828bf: Status 404 returned error can't find the container with id aefb7e965e06938769edc8d1306eab48d628e2cb8ec73c2aad2b6f7aa13828bf Dec 03 13:26:15 crc kubenswrapper[5052]: I1203 13:26:15.907209 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/community-operators-ntppl" event={"ID":"2d8df6a3-ab5e-4c69-9f03-3fa57c872ad9","Type":"ContainerStarted","Data":"aefb7e965e06938769edc8d1306eab48d628e2cb8ec73c2aad2b6f7aa13828bf"} Dec 03 13:26:17 crc kubenswrapper[5052]: I1203 13:26:17.927467 5052 generic.go:334] "Generic (PLEG): container finished" podID="2d8df6a3-ab5e-4c69-9f03-3fa57c872ad9" containerID="e45224d4b7d8d7869bd7739cbdd643beeff51e6943ee3bdd365b97b107f5ba02" exitCode=0 Dec 03 13:26:17 crc kubenswrapper[5052]: I1203 13:26:17.927575 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/community-operators-ntppl" event={"ID":"2d8df6a3-ab5e-4c69-9f03-3fa57c872ad9","Type":"ContainerDied","Data":"e45224d4b7d8d7869bd7739cbdd643beeff51e6943ee3bdd365b97b107f5ba02"} Dec 03 13:26:17 crc kubenswrapper[5052]: I1203 13:26:17.929723 5052 provider.go:102] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Dec 03 13:26:23 crc kubenswrapper[5052]: I1203 13:26:23.981719 5052 generic.go:334] "Generic (PLEG): container finished" podID="2d8df6a3-ab5e-4c69-9f03-3fa57c872ad9" containerID="07e9239c1a3f4d71963a7c3d7ecb619ea1694e572e3eedd373a7c04997dd0e81" exitCode=0 Dec 03 13:26:23 crc kubenswrapper[5052]: I1203 13:26:23.981818 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/community-operators-ntppl" event={"ID":"2d8df6a3-ab5e-4c69-9f03-3fa57c872ad9","Type":"ContainerDied","Data":"07e9239c1a3f4d71963a7c3d7ecb619ea1694e572e3eedd373a7c04997dd0e81"} Dec 03 13:26:25 crc kubenswrapper[5052]: I1203 13:26:25.636455 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="keystone-kuttl-tests/keystone-6b94f77657-fg5j2" Dec 03 13:26:25 crc kubenswrapper[5052]: I1203 13:26:25.644561 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="keystone-kuttl-tests/keystone-6b94f77657-hqt2p" Dec 03 13:26:26 crc kubenswrapper[5052]: I1203 13:26:26.940447 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["keystone-kuttl-tests/keystone-6b94f77657-fg5j2"] Dec 03 13:26:26 crc kubenswrapper[5052]: I1203 13:26:26.941048 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="keystone-kuttl-tests/keystone-6b94f77657-fg5j2" podUID="c9e8b9fe-4bb0-42f2-aec5-47b93b283868" containerName="keystone-api" containerID="cri-o://8ea4edd4258fec190e0d96af0f029e657a89ae8516a0b72b3a4e210cf596038e" gracePeriod=30 Dec 03 13:26:26 crc kubenswrapper[5052]: I1203 13:26:26.948311 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["keystone-kuttl-tests/keystone-6b94f77657-hqt2p"] Dec 03 13:26:26 crc kubenswrapper[5052]: I1203 13:26:26.948556 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="keystone-kuttl-tests/keystone-6b94f77657-hqt2p" podUID="6844c5ca-9f10-46af-b462-f54319ff59ea" containerName="keystone-api" containerID="cri-o://2ab9679d66c2bddc779455e79fda921b12f80e7c03e6109d792923508b0d0f9f" gracePeriod=30 Dec 03 13:26:27 crc kubenswrapper[5052]: I1203 13:26:27.033226 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/community-operators-ntppl" event={"ID":"2d8df6a3-ab5e-4c69-9f03-3fa57c872ad9","Type":"ContainerStarted","Data":"db19da52fb74558cbc59eec3c9e5ae3fd96de82c2f4e2d69ccf5f202de9d17dd"} Dec 03 13:26:27 crc kubenswrapper[5052]: I1203 13:26:27.065138 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-marketplace/community-operators-ntppl" podStartSLOduration=4.204950349 podStartE2EDuration="12.065113497s" podCreationTimestamp="2025-12-03 13:26:15 +0000 UTC" firstStartedPulling="2025-12-03 13:26:17.929392041 +0000 UTC m=+7568.517633805" lastFinishedPulling="2025-12-03 13:26:25.789555189 +0000 UTC m=+7576.377796953" observedRunningTime="2025-12-03 13:26:27.057434817 +0000 UTC m=+7577.645676591" watchObservedRunningTime="2025-12-03 13:26:27.065113497 +0000 UTC m=+7577.653355261" Dec 03 13:26:28 crc kubenswrapper[5052]: I1203 13:26:28.173384 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["keystone-kuttl-tests/keystone-6b94f77657-72gm8"] Dec 03 13:26:28 crc kubenswrapper[5052]: I1203 13:26:28.173667 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="keystone-kuttl-tests/keystone-6b94f77657-72gm8" podUID="7bdc1a91-11f2-48ff-a2a5-f26606395613" containerName="keystone-api" containerID="cri-o://cf529eea53b2ebc69e317bc10d7f707894210ad8e2d39e0a77271d5c0aa80d48" gracePeriod=30 Dec 03 13:26:29 crc kubenswrapper[5052]: I1203 13:26:29.379345 5052 patch_prober.go:28] interesting pod/machine-config-daemon-74bj6 container/machine-config-daemon namespace/openshift-machine-config-operator: Liveness probe status=failure output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" start-of-body= Dec 03 13:26:29 crc kubenswrapper[5052]: I1203 13:26:29.379411 5052 prober.go:107] "Probe failed" probeType="Liveness" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" containerName="machine-config-daemon" probeResult="failure" output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" Dec 03 13:26:29 crc kubenswrapper[5052]: I1203 13:26:29.379460 5052 kubelet.go:2542] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" Dec 03 13:26:29 crc kubenswrapper[5052]: I1203 13:26:29.380181 5052 kuberuntime_manager.go:1027] "Message for Container of pod" containerName="machine-config-daemon" containerStatusID={"Type":"cri-o","ID":"6f2eaa045b10c8c10abc2278895801064c7c776bf5ef0e26c0544a891311f82f"} pod="openshift-machine-config-operator/machine-config-daemon-74bj6" containerMessage="Container machine-config-daemon failed liveness probe, will be restarted" Dec 03 13:26:29 crc kubenswrapper[5052]: I1203 13:26:29.380276 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" containerName="machine-config-daemon" containerID="cri-o://6f2eaa045b10c8c10abc2278895801064c7c776bf5ef0e26c0544a891311f82f" gracePeriod=600 Dec 03 13:26:32 crc kubenswrapper[5052]: I1203 13:26:32.072576 5052 generic.go:334] "Generic (PLEG): container finished" podID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" containerID="6f2eaa045b10c8c10abc2278895801064c7c776bf5ef0e26c0544a891311f82f" exitCode=0 Dec 03 13:26:32 crc kubenswrapper[5052]: I1203 13:26:32.072639 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" event={"ID":"10c237bc-72e7-4da3-9f2c-b1688395bf6e","Type":"ContainerDied","Data":"6f2eaa045b10c8c10abc2278895801064c7c776bf5ef0e26c0544a891311f82f"} Dec 03 13:26:32 crc kubenswrapper[5052]: I1203 13:26:32.073638 5052 scope.go:117] "RemoveContainer" containerID="12a8b1f470f6d59d7700e376f2a48eb36363f91f9823a35ce534062c0974c10c" Dec 03 13:26:32 crc kubenswrapper[5052]: I1203 13:26:32.075373 5052 generic.go:334] "Generic (PLEG): container finished" podID="6844c5ca-9f10-46af-b462-f54319ff59ea" containerID="2ab9679d66c2bddc779455e79fda921b12f80e7c03e6109d792923508b0d0f9f" exitCode=0 Dec 03 13:26:32 crc kubenswrapper[5052]: I1203 13:26:32.075444 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="keystone-kuttl-tests/keystone-6b94f77657-hqt2p" event={"ID":"6844c5ca-9f10-46af-b462-f54319ff59ea","Type":"ContainerDied","Data":"2ab9679d66c2bddc779455e79fda921b12f80e7c03e6109d792923508b0d0f9f"} Dec 03 13:26:32 crc kubenswrapper[5052]: I1203 13:26:32.077132 5052 generic.go:334] "Generic (PLEG): container finished" podID="7bdc1a91-11f2-48ff-a2a5-f26606395613" containerID="cf529eea53b2ebc69e317bc10d7f707894210ad8e2d39e0a77271d5c0aa80d48" exitCode=0 Dec 03 13:26:32 crc kubenswrapper[5052]: I1203 13:26:32.077278 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="keystone-kuttl-tests/keystone-6b94f77657-72gm8" event={"ID":"7bdc1a91-11f2-48ff-a2a5-f26606395613","Type":"ContainerDied","Data":"cf529eea53b2ebc69e317bc10d7f707894210ad8e2d39e0a77271d5c0aa80d48"} Dec 03 13:26:32 crc kubenswrapper[5052]: I1203 13:26:32.079045 5052 generic.go:334] "Generic (PLEG): container finished" podID="c9e8b9fe-4bb0-42f2-aec5-47b93b283868" containerID="8ea4edd4258fec190e0d96af0f029e657a89ae8516a0b72b3a4e210cf596038e" exitCode=0 Dec 03 13:26:32 crc kubenswrapper[5052]: I1203 13:26:32.079131 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="keystone-kuttl-tests/keystone-6b94f77657-fg5j2" event={"ID":"c9e8b9fe-4bb0-42f2-aec5-47b93b283868","Type":"ContainerDied","Data":"8ea4edd4258fec190e0d96af0f029e657a89ae8516a0b72b3a4e210cf596038e"} Dec 03 13:26:32 crc kubenswrapper[5052]: I1203 13:26:32.504009 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="keystone-kuttl-tests/keystone-6b94f77657-fg5j2" Dec 03 13:26:32 crc kubenswrapper[5052]: I1203 13:26:32.654151 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/c9e8b9fe-4bb0-42f2-aec5-47b93b283868-config-data\") pod \"c9e8b9fe-4bb0-42f2-aec5-47b93b283868\" (UID: \"c9e8b9fe-4bb0-42f2-aec5-47b93b283868\") " Dec 03 13:26:32 crc kubenswrapper[5052]: I1203 13:26:32.654243 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/c9e8b9fe-4bb0-42f2-aec5-47b93b283868-fernet-keys\") pod \"c9e8b9fe-4bb0-42f2-aec5-47b93b283868\" (UID: \"c9e8b9fe-4bb0-42f2-aec5-47b93b283868\") " Dec 03 13:26:32 crc kubenswrapper[5052]: I1203 13:26:32.654268 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/c9e8b9fe-4bb0-42f2-aec5-47b93b283868-credential-keys\") pod \"c9e8b9fe-4bb0-42f2-aec5-47b93b283868\" (UID: \"c9e8b9fe-4bb0-42f2-aec5-47b93b283868\") " Dec 03 13:26:32 crc kubenswrapper[5052]: I1203 13:26:32.654325 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/c9e8b9fe-4bb0-42f2-aec5-47b93b283868-scripts\") pod \"c9e8b9fe-4bb0-42f2-aec5-47b93b283868\" (UID: \"c9e8b9fe-4bb0-42f2-aec5-47b93b283868\") " Dec 03 13:26:32 crc kubenswrapper[5052]: I1203 13:26:32.654365 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2wxlg\" (UniqueName: \"kubernetes.io/projected/c9e8b9fe-4bb0-42f2-aec5-47b93b283868-kube-api-access-2wxlg\") pod \"c9e8b9fe-4bb0-42f2-aec5-47b93b283868\" (UID: \"c9e8b9fe-4bb0-42f2-aec5-47b93b283868\") " Dec 03 13:26:32 crc kubenswrapper[5052]: I1203 13:26:32.666413 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c9e8b9fe-4bb0-42f2-aec5-47b93b283868-fernet-keys" (OuterVolumeSpecName: "fernet-keys") pod "c9e8b9fe-4bb0-42f2-aec5-47b93b283868" (UID: "c9e8b9fe-4bb0-42f2-aec5-47b93b283868"). InnerVolumeSpecName "fernet-keys". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:26:32 crc kubenswrapper[5052]: I1203 13:26:32.666595 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c9e8b9fe-4bb0-42f2-aec5-47b93b283868-credential-keys" (OuterVolumeSpecName: "credential-keys") pod "c9e8b9fe-4bb0-42f2-aec5-47b93b283868" (UID: "c9e8b9fe-4bb0-42f2-aec5-47b93b283868"). InnerVolumeSpecName "credential-keys". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:26:32 crc kubenswrapper[5052]: I1203 13:26:32.666595 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c9e8b9fe-4bb0-42f2-aec5-47b93b283868-scripts" (OuterVolumeSpecName: "scripts") pod "c9e8b9fe-4bb0-42f2-aec5-47b93b283868" (UID: "c9e8b9fe-4bb0-42f2-aec5-47b93b283868"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:26:32 crc kubenswrapper[5052]: I1203 13:26:32.667745 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c9e8b9fe-4bb0-42f2-aec5-47b93b283868-kube-api-access-2wxlg" (OuterVolumeSpecName: "kube-api-access-2wxlg") pod "c9e8b9fe-4bb0-42f2-aec5-47b93b283868" (UID: "c9e8b9fe-4bb0-42f2-aec5-47b93b283868"). InnerVolumeSpecName "kube-api-access-2wxlg". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:26:32 crc kubenswrapper[5052]: I1203 13:26:32.695937 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c9e8b9fe-4bb0-42f2-aec5-47b93b283868-config-data" (OuterVolumeSpecName: "config-data") pod "c9e8b9fe-4bb0-42f2-aec5-47b93b283868" (UID: "c9e8b9fe-4bb0-42f2-aec5-47b93b283868"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:26:32 crc kubenswrapper[5052]: I1203 13:26:32.742298 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="keystone-kuttl-tests/keystone-6b94f77657-hqt2p" Dec 03 13:26:32 crc kubenswrapper[5052]: I1203 13:26:32.756156 5052 reconciler_common.go:293] "Volume detached for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/c9e8b9fe-4bb0-42f2-aec5-47b93b283868-fernet-keys\") on node \"crc\" DevicePath \"\"" Dec 03 13:26:32 crc kubenswrapper[5052]: I1203 13:26:32.756196 5052 reconciler_common.go:293] "Volume detached for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/c9e8b9fe-4bb0-42f2-aec5-47b93b283868-credential-keys\") on node \"crc\" DevicePath \"\"" Dec 03 13:26:32 crc kubenswrapper[5052]: I1203 13:26:32.756209 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/c9e8b9fe-4bb0-42f2-aec5-47b93b283868-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 13:26:32 crc kubenswrapper[5052]: I1203 13:26:32.756314 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-2wxlg\" (UniqueName: \"kubernetes.io/projected/c9e8b9fe-4bb0-42f2-aec5-47b93b283868-kube-api-access-2wxlg\") on node \"crc\" DevicePath \"\"" Dec 03 13:26:32 crc kubenswrapper[5052]: I1203 13:26:32.756331 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/c9e8b9fe-4bb0-42f2-aec5-47b93b283868-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 13:26:32 crc kubenswrapper[5052]: I1203 13:26:32.857254 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/6844c5ca-9f10-46af-b462-f54319ff59ea-credential-keys\") pod \"6844c5ca-9f10-46af-b462-f54319ff59ea\" (UID: \"6844c5ca-9f10-46af-b462-f54319ff59ea\") " Dec 03 13:26:32 crc kubenswrapper[5052]: I1203 13:26:32.858078 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6844c5ca-9f10-46af-b462-f54319ff59ea-config-data\") pod \"6844c5ca-9f10-46af-b462-f54319ff59ea\" (UID: \"6844c5ca-9f10-46af-b462-f54319ff59ea\") " Dec 03 13:26:32 crc kubenswrapper[5052]: I1203 13:26:32.858191 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-98vt7\" (UniqueName: \"kubernetes.io/projected/6844c5ca-9f10-46af-b462-f54319ff59ea-kube-api-access-98vt7\") pod \"6844c5ca-9f10-46af-b462-f54319ff59ea\" (UID: \"6844c5ca-9f10-46af-b462-f54319ff59ea\") " Dec 03 13:26:32 crc kubenswrapper[5052]: I1203 13:26:32.858255 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/6844c5ca-9f10-46af-b462-f54319ff59ea-fernet-keys\") pod \"6844c5ca-9f10-46af-b462-f54319ff59ea\" (UID: \"6844c5ca-9f10-46af-b462-f54319ff59ea\") " Dec 03 13:26:32 crc kubenswrapper[5052]: I1203 13:26:32.858389 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/6844c5ca-9f10-46af-b462-f54319ff59ea-scripts\") pod \"6844c5ca-9f10-46af-b462-f54319ff59ea\" (UID: \"6844c5ca-9f10-46af-b462-f54319ff59ea\") " Dec 03 13:26:32 crc kubenswrapper[5052]: I1203 13:26:32.861357 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6844c5ca-9f10-46af-b462-f54319ff59ea-fernet-keys" (OuterVolumeSpecName: "fernet-keys") pod "6844c5ca-9f10-46af-b462-f54319ff59ea" (UID: "6844c5ca-9f10-46af-b462-f54319ff59ea"). InnerVolumeSpecName "fernet-keys". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:26:32 crc kubenswrapper[5052]: I1203 13:26:32.861452 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6844c5ca-9f10-46af-b462-f54319ff59ea-kube-api-access-98vt7" (OuterVolumeSpecName: "kube-api-access-98vt7") pod "6844c5ca-9f10-46af-b462-f54319ff59ea" (UID: "6844c5ca-9f10-46af-b462-f54319ff59ea"). InnerVolumeSpecName "kube-api-access-98vt7". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:26:32 crc kubenswrapper[5052]: I1203 13:26:32.862329 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6844c5ca-9f10-46af-b462-f54319ff59ea-scripts" (OuterVolumeSpecName: "scripts") pod "6844c5ca-9f10-46af-b462-f54319ff59ea" (UID: "6844c5ca-9f10-46af-b462-f54319ff59ea"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:26:32 crc kubenswrapper[5052]: I1203 13:26:32.862449 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6844c5ca-9f10-46af-b462-f54319ff59ea-credential-keys" (OuterVolumeSpecName: "credential-keys") pod "6844c5ca-9f10-46af-b462-f54319ff59ea" (UID: "6844c5ca-9f10-46af-b462-f54319ff59ea"). InnerVolumeSpecName "credential-keys". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:26:32 crc kubenswrapper[5052]: I1203 13:26:32.887399 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6844c5ca-9f10-46af-b462-f54319ff59ea-config-data" (OuterVolumeSpecName: "config-data") pod "6844c5ca-9f10-46af-b462-f54319ff59ea" (UID: "6844c5ca-9f10-46af-b462-f54319ff59ea"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:26:32 crc kubenswrapper[5052]: I1203 13:26:32.960791 5052 reconciler_common.go:293] "Volume detached for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/6844c5ca-9f10-46af-b462-f54319ff59ea-fernet-keys\") on node \"crc\" DevicePath \"\"" Dec 03 13:26:32 crc kubenswrapper[5052]: I1203 13:26:32.960828 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/6844c5ca-9f10-46af-b462-f54319ff59ea-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 13:26:32 crc kubenswrapper[5052]: I1203 13:26:32.960838 5052 reconciler_common.go:293] "Volume detached for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/6844c5ca-9f10-46af-b462-f54319ff59ea-credential-keys\") on node \"crc\" DevicePath \"\"" Dec 03 13:26:32 crc kubenswrapper[5052]: I1203 13:26:32.960846 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6844c5ca-9f10-46af-b462-f54319ff59ea-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 13:26:32 crc kubenswrapper[5052]: I1203 13:26:32.960858 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-98vt7\" (UniqueName: \"kubernetes.io/projected/6844c5ca-9f10-46af-b462-f54319ff59ea-kube-api-access-98vt7\") on node \"crc\" DevicePath \"\"" Dec 03 13:26:33 crc kubenswrapper[5052]: I1203 13:26:33.014872 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="keystone-kuttl-tests/keystone-6b94f77657-72gm8" Dec 03 13:26:33 crc kubenswrapper[5052]: I1203 13:26:33.088732 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="keystone-kuttl-tests/keystone-6b94f77657-72gm8" event={"ID":"7bdc1a91-11f2-48ff-a2a5-f26606395613","Type":"ContainerDied","Data":"a3260335673fd00780ee9021736de4a0b437b8d7bd92fde8abfeddded2167246"} Dec 03 13:26:33 crc kubenswrapper[5052]: I1203 13:26:33.089070 5052 scope.go:117] "RemoveContainer" containerID="cf529eea53b2ebc69e317bc10d7f707894210ad8e2d39e0a77271d5c0aa80d48" Dec 03 13:26:33 crc kubenswrapper[5052]: I1203 13:26:33.089174 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="keystone-kuttl-tests/keystone-6b94f77657-72gm8" Dec 03 13:26:33 crc kubenswrapper[5052]: I1203 13:26:33.101943 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="keystone-kuttl-tests/keystone-6b94f77657-fg5j2" Dec 03 13:26:33 crc kubenswrapper[5052]: I1203 13:26:33.101937 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="keystone-kuttl-tests/keystone-6b94f77657-fg5j2" event={"ID":"c9e8b9fe-4bb0-42f2-aec5-47b93b283868","Type":"ContainerDied","Data":"207592b01bd0967733fbc1b8c446c555a320660540b1eac884e50d8ac280a607"} Dec 03 13:26:33 crc kubenswrapper[5052]: I1203 13:26:33.109897 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" event={"ID":"10c237bc-72e7-4da3-9f2c-b1688395bf6e","Type":"ContainerStarted","Data":"4dbf4cc5b784c22fc4eecbfecde10f7c7824875bd119667c218a37a8cbc71d15"} Dec 03 13:26:33 crc kubenswrapper[5052]: I1203 13:26:33.121631 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="keystone-kuttl-tests/keystone-6b94f77657-hqt2p" event={"ID":"6844c5ca-9f10-46af-b462-f54319ff59ea","Type":"ContainerDied","Data":"c5d8a5e91232a83a4ef806bfceb5d1b25dd1bd53059d8c5f9d1cc19a44d243c9"} Dec 03 13:26:33 crc kubenswrapper[5052]: I1203 13:26:33.121721 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="keystone-kuttl-tests/keystone-6b94f77657-hqt2p" Dec 03 13:26:33 crc kubenswrapper[5052]: I1203 13:26:33.132493 5052 scope.go:117] "RemoveContainer" containerID="8ea4edd4258fec190e0d96af0f029e657a89ae8516a0b72b3a4e210cf596038e" Dec 03 13:26:33 crc kubenswrapper[5052]: I1203 13:26:33.155041 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["keystone-kuttl-tests/keystone-6b94f77657-fg5j2"] Dec 03 13:26:33 crc kubenswrapper[5052]: I1203 13:26:33.161799 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["keystone-kuttl-tests/keystone-6b94f77657-fg5j2"] Dec 03 13:26:33 crc kubenswrapper[5052]: I1203 13:26:33.163448 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/7bdc1a91-11f2-48ff-a2a5-f26606395613-credential-keys\") pod \"7bdc1a91-11f2-48ff-a2a5-f26606395613\" (UID: \"7bdc1a91-11f2-48ff-a2a5-f26606395613\") " Dec 03 13:26:33 crc kubenswrapper[5052]: I1203 13:26:33.163577 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/7bdc1a91-11f2-48ff-a2a5-f26606395613-scripts\") pod \"7bdc1a91-11f2-48ff-a2a5-f26606395613\" (UID: \"7bdc1a91-11f2-48ff-a2a5-f26606395613\") " Dec 03 13:26:33 crc kubenswrapper[5052]: I1203 13:26:33.163615 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/7bdc1a91-11f2-48ff-a2a5-f26606395613-config-data\") pod \"7bdc1a91-11f2-48ff-a2a5-f26606395613\" (UID: \"7bdc1a91-11f2-48ff-a2a5-f26606395613\") " Dec 03 13:26:33 crc kubenswrapper[5052]: I1203 13:26:33.163684 5052 scope.go:117] "RemoveContainer" containerID="2ab9679d66c2bddc779455e79fda921b12f80e7c03e6109d792923508b0d0f9f" Dec 03 13:26:33 crc kubenswrapper[5052]: I1203 13:26:33.163691 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/7bdc1a91-11f2-48ff-a2a5-f26606395613-fernet-keys\") pod \"7bdc1a91-11f2-48ff-a2a5-f26606395613\" (UID: \"7bdc1a91-11f2-48ff-a2a5-f26606395613\") " Dec 03 13:26:33 crc kubenswrapper[5052]: I1203 13:26:33.163887 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-skwn9\" (UniqueName: \"kubernetes.io/projected/7bdc1a91-11f2-48ff-a2a5-f26606395613-kube-api-access-skwn9\") pod \"7bdc1a91-11f2-48ff-a2a5-f26606395613\" (UID: \"7bdc1a91-11f2-48ff-a2a5-f26606395613\") " Dec 03 13:26:33 crc kubenswrapper[5052]: I1203 13:26:33.167332 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["keystone-kuttl-tests/keystone-6b94f77657-hqt2p"] Dec 03 13:26:33 crc kubenswrapper[5052]: I1203 13:26:33.170496 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7bdc1a91-11f2-48ff-a2a5-f26606395613-credential-keys" (OuterVolumeSpecName: "credential-keys") pod "7bdc1a91-11f2-48ff-a2a5-f26606395613" (UID: "7bdc1a91-11f2-48ff-a2a5-f26606395613"). InnerVolumeSpecName "credential-keys". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:26:33 crc kubenswrapper[5052]: I1203 13:26:33.172497 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["keystone-kuttl-tests/keystone-6b94f77657-hqt2p"] Dec 03 13:26:33 crc kubenswrapper[5052]: I1203 13:26:33.172623 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7bdc1a91-11f2-48ff-a2a5-f26606395613-scripts" (OuterVolumeSpecName: "scripts") pod "7bdc1a91-11f2-48ff-a2a5-f26606395613" (UID: "7bdc1a91-11f2-48ff-a2a5-f26606395613"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:26:33 crc kubenswrapper[5052]: I1203 13:26:33.172825 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7bdc1a91-11f2-48ff-a2a5-f26606395613-kube-api-access-skwn9" (OuterVolumeSpecName: "kube-api-access-skwn9") pod "7bdc1a91-11f2-48ff-a2a5-f26606395613" (UID: "7bdc1a91-11f2-48ff-a2a5-f26606395613"). InnerVolumeSpecName "kube-api-access-skwn9". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:26:33 crc kubenswrapper[5052]: I1203 13:26:33.176420 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7bdc1a91-11f2-48ff-a2a5-f26606395613-fernet-keys" (OuterVolumeSpecName: "fernet-keys") pod "7bdc1a91-11f2-48ff-a2a5-f26606395613" (UID: "7bdc1a91-11f2-48ff-a2a5-f26606395613"). InnerVolumeSpecName "fernet-keys". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:26:33 crc kubenswrapper[5052]: I1203 13:26:33.188560 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7bdc1a91-11f2-48ff-a2a5-f26606395613-config-data" (OuterVolumeSpecName: "config-data") pod "7bdc1a91-11f2-48ff-a2a5-f26606395613" (UID: "7bdc1a91-11f2-48ff-a2a5-f26606395613"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:26:33 crc kubenswrapper[5052]: I1203 13:26:33.266982 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/7bdc1a91-11f2-48ff-a2a5-f26606395613-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 13:26:33 crc kubenswrapper[5052]: I1203 13:26:33.267034 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/7bdc1a91-11f2-48ff-a2a5-f26606395613-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 13:26:33 crc kubenswrapper[5052]: I1203 13:26:33.267047 5052 reconciler_common.go:293] "Volume detached for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/7bdc1a91-11f2-48ff-a2a5-f26606395613-fernet-keys\") on node \"crc\" DevicePath \"\"" Dec 03 13:26:33 crc kubenswrapper[5052]: I1203 13:26:33.267062 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-skwn9\" (UniqueName: \"kubernetes.io/projected/7bdc1a91-11f2-48ff-a2a5-f26606395613-kube-api-access-skwn9\") on node \"crc\" DevicePath \"\"" Dec 03 13:26:33 crc kubenswrapper[5052]: I1203 13:26:33.267074 5052 reconciler_common.go:293] "Volume detached for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/7bdc1a91-11f2-48ff-a2a5-f26606395613-credential-keys\") on node \"crc\" DevicePath \"\"" Dec 03 13:26:33 crc kubenswrapper[5052]: I1203 13:26:33.426280 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["keystone-kuttl-tests/keystone-6b94f77657-72gm8"] Dec 03 13:26:33 crc kubenswrapper[5052]: I1203 13:26:33.435146 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["keystone-kuttl-tests/keystone-6b94f77657-72gm8"] Dec 03 13:26:33 crc kubenswrapper[5052]: I1203 13:26:33.698885 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6844c5ca-9f10-46af-b462-f54319ff59ea" path="/var/lib/kubelet/pods/6844c5ca-9f10-46af-b462-f54319ff59ea/volumes" Dec 03 13:26:33 crc kubenswrapper[5052]: I1203 13:26:33.699940 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7bdc1a91-11f2-48ff-a2a5-f26606395613" path="/var/lib/kubelet/pods/7bdc1a91-11f2-48ff-a2a5-f26606395613/volumes" Dec 03 13:26:33 crc kubenswrapper[5052]: I1203 13:26:33.700885 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c9e8b9fe-4bb0-42f2-aec5-47b93b283868" path="/var/lib/kubelet/pods/c9e8b9fe-4bb0-42f2-aec5-47b93b283868/volumes" Dec 03 13:26:34 crc kubenswrapper[5052]: I1203 13:26:34.427496 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["keystone-kuttl-tests/keystone-bootstrap-x5gg9"] Dec 03 13:26:34 crc kubenswrapper[5052]: I1203 13:26:34.440192 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["keystone-kuttl-tests/keystone-db-sync-zfwhv"] Dec 03 13:26:34 crc kubenswrapper[5052]: I1203 13:26:34.447295 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["keystone-kuttl-tests/keystone-db-sync-zfwhv"] Dec 03 13:26:34 crc kubenswrapper[5052]: I1203 13:26:34.453223 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["keystone-kuttl-tests/keystone-bootstrap-x5gg9"] Dec 03 13:26:34 crc kubenswrapper[5052]: I1203 13:26:34.516285 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["keystone-kuttl-tests/keystonef47f-account-delete-mnx42"] Dec 03 13:26:34 crc kubenswrapper[5052]: E1203 13:26:34.517064 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="6844c5ca-9f10-46af-b462-f54319ff59ea" containerName="keystone-api" Dec 03 13:26:34 crc kubenswrapper[5052]: I1203 13:26:34.517091 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="6844c5ca-9f10-46af-b462-f54319ff59ea" containerName="keystone-api" Dec 03 13:26:34 crc kubenswrapper[5052]: E1203 13:26:34.517117 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="c9e8b9fe-4bb0-42f2-aec5-47b93b283868" containerName="keystone-api" Dec 03 13:26:34 crc kubenswrapper[5052]: I1203 13:26:34.517130 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="c9e8b9fe-4bb0-42f2-aec5-47b93b283868" containerName="keystone-api" Dec 03 13:26:34 crc kubenswrapper[5052]: E1203 13:26:34.517143 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="7bdc1a91-11f2-48ff-a2a5-f26606395613" containerName="keystone-api" Dec 03 13:26:34 crc kubenswrapper[5052]: I1203 13:26:34.517157 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="7bdc1a91-11f2-48ff-a2a5-f26606395613" containerName="keystone-api" Dec 03 13:26:34 crc kubenswrapper[5052]: I1203 13:26:34.517361 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="7bdc1a91-11f2-48ff-a2a5-f26606395613" containerName="keystone-api" Dec 03 13:26:34 crc kubenswrapper[5052]: I1203 13:26:34.517393 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="c9e8b9fe-4bb0-42f2-aec5-47b93b283868" containerName="keystone-api" Dec 03 13:26:34 crc kubenswrapper[5052]: I1203 13:26:34.517414 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="6844c5ca-9f10-46af-b462-f54319ff59ea" containerName="keystone-api" Dec 03 13:26:34 crc kubenswrapper[5052]: I1203 13:26:34.518420 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="keystone-kuttl-tests/keystonef47f-account-delete-mnx42" Dec 03 13:26:34 crc kubenswrapper[5052]: I1203 13:26:34.564661 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["keystone-kuttl-tests/keystonef47f-account-delete-mnx42"] Dec 03 13:26:34 crc kubenswrapper[5052]: I1203 13:26:34.590557 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gtdfh\" (UniqueName: \"kubernetes.io/projected/8c58eb7f-401b-474e-a5cd-eee60fe761b3-kube-api-access-gtdfh\") pod \"keystonef47f-account-delete-mnx42\" (UID: \"8c58eb7f-401b-474e-a5cd-eee60fe761b3\") " pod="keystone-kuttl-tests/keystonef47f-account-delete-mnx42" Dec 03 13:26:34 crc kubenswrapper[5052]: I1203 13:26:34.590892 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/8c58eb7f-401b-474e-a5cd-eee60fe761b3-operator-scripts\") pod \"keystonef47f-account-delete-mnx42\" (UID: \"8c58eb7f-401b-474e-a5cd-eee60fe761b3\") " pod="keystone-kuttl-tests/keystonef47f-account-delete-mnx42" Dec 03 13:26:34 crc kubenswrapper[5052]: I1203 13:26:34.695229 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-gtdfh\" (UniqueName: \"kubernetes.io/projected/8c58eb7f-401b-474e-a5cd-eee60fe761b3-kube-api-access-gtdfh\") pod \"keystonef47f-account-delete-mnx42\" (UID: \"8c58eb7f-401b-474e-a5cd-eee60fe761b3\") " pod="keystone-kuttl-tests/keystonef47f-account-delete-mnx42" Dec 03 13:26:34 crc kubenswrapper[5052]: I1203 13:26:34.695358 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/8c58eb7f-401b-474e-a5cd-eee60fe761b3-operator-scripts\") pod \"keystonef47f-account-delete-mnx42\" (UID: \"8c58eb7f-401b-474e-a5cd-eee60fe761b3\") " pod="keystone-kuttl-tests/keystonef47f-account-delete-mnx42" Dec 03 13:26:34 crc kubenswrapper[5052]: I1203 13:26:34.699663 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/8c58eb7f-401b-474e-a5cd-eee60fe761b3-operator-scripts\") pod \"keystonef47f-account-delete-mnx42\" (UID: \"8c58eb7f-401b-474e-a5cd-eee60fe761b3\") " pod="keystone-kuttl-tests/keystonef47f-account-delete-mnx42" Dec 03 13:26:34 crc kubenswrapper[5052]: I1203 13:26:34.729901 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-gtdfh\" (UniqueName: \"kubernetes.io/projected/8c58eb7f-401b-474e-a5cd-eee60fe761b3-kube-api-access-gtdfh\") pod \"keystonef47f-account-delete-mnx42\" (UID: \"8c58eb7f-401b-474e-a5cd-eee60fe761b3\") " pod="keystone-kuttl-tests/keystonef47f-account-delete-mnx42" Dec 03 13:26:34 crc kubenswrapper[5052]: I1203 13:26:34.873375 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="keystone-kuttl-tests/keystonef47f-account-delete-mnx42" Dec 03 13:26:35 crc kubenswrapper[5052]: I1203 13:26:35.087744 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["keystone-kuttl-tests/keystonef47f-account-delete-mnx42"] Dec 03 13:26:35 crc kubenswrapper[5052]: I1203 13:26:35.147023 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="keystone-kuttl-tests/keystonef47f-account-delete-mnx42" event={"ID":"8c58eb7f-401b-474e-a5cd-eee60fe761b3","Type":"ContainerStarted","Data":"7e5c48986619000af246da0ed43e7447e8c4eacae79d94047fc16377fefd7a35"} Dec 03 13:26:35 crc kubenswrapper[5052]: I1203 13:26:35.392893 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openshift-marketplace/community-operators-ntppl" Dec 03 13:26:35 crc kubenswrapper[5052]: I1203 13:26:35.392943 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-marketplace/community-operators-ntppl" Dec 03 13:26:35 crc kubenswrapper[5052]: I1203 13:26:35.457548 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-marketplace/community-operators-ntppl" Dec 03 13:26:35 crc kubenswrapper[5052]: I1203 13:26:35.683366 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="11bc3563-46e9-4256-afa2-5ebc442af77e" path="/var/lib/kubelet/pods/11bc3563-46e9-4256-afa2-5ebc442af77e/volumes" Dec 03 13:26:35 crc kubenswrapper[5052]: I1203 13:26:35.684356 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dfa1f868-cd4f-4e2b-97fd-46e65188ad75" path="/var/lib/kubelet/pods/dfa1f868-cd4f-4e2b-97fd-46e65188ad75/volumes" Dec 03 13:26:36 crc kubenswrapper[5052]: I1203 13:26:36.161330 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="keystone-kuttl-tests/keystonef47f-account-delete-mnx42" event={"ID":"8c58eb7f-401b-474e-a5cd-eee60fe761b3","Type":"ContainerStarted","Data":"ef77869d85af31266b483685c0df0c8353444fc238667ea89fed39c37d442c8e"} Dec 03 13:26:36 crc kubenswrapper[5052]: I1203 13:26:36.211724 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-marketplace/community-operators-ntppl" Dec 03 13:26:36 crc kubenswrapper[5052]: I1203 13:26:36.265787 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openshift-marketplace/community-operators-ntppl"] Dec 03 13:26:37 crc kubenswrapper[5052]: I1203 13:26:37.170257 5052 generic.go:334] "Generic (PLEG): container finished" podID="8c58eb7f-401b-474e-a5cd-eee60fe761b3" containerID="ef77869d85af31266b483685c0df0c8353444fc238667ea89fed39c37d442c8e" exitCode=0 Dec 03 13:26:37 crc kubenswrapper[5052]: I1203 13:26:37.170357 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="keystone-kuttl-tests/keystonef47f-account-delete-mnx42" event={"ID":"8c58eb7f-401b-474e-a5cd-eee60fe761b3","Type":"ContainerDied","Data":"ef77869d85af31266b483685c0df0c8353444fc238667ea89fed39c37d442c8e"} Dec 03 13:26:37 crc kubenswrapper[5052]: I1203 13:26:37.429368 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="keystone-kuttl-tests/keystonef47f-account-delete-mnx42" Dec 03 13:26:37 crc kubenswrapper[5052]: I1203 13:26:37.437116 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/8c58eb7f-401b-474e-a5cd-eee60fe761b3-operator-scripts\") pod \"8c58eb7f-401b-474e-a5cd-eee60fe761b3\" (UID: \"8c58eb7f-401b-474e-a5cd-eee60fe761b3\") " Dec 03 13:26:37 crc kubenswrapper[5052]: I1203 13:26:37.437233 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gtdfh\" (UniqueName: \"kubernetes.io/projected/8c58eb7f-401b-474e-a5cd-eee60fe761b3-kube-api-access-gtdfh\") pod \"8c58eb7f-401b-474e-a5cd-eee60fe761b3\" (UID: \"8c58eb7f-401b-474e-a5cd-eee60fe761b3\") " Dec 03 13:26:37 crc kubenswrapper[5052]: I1203 13:26:37.437780 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8c58eb7f-401b-474e-a5cd-eee60fe761b3-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "8c58eb7f-401b-474e-a5cd-eee60fe761b3" (UID: "8c58eb7f-401b-474e-a5cd-eee60fe761b3"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:26:37 crc kubenswrapper[5052]: I1203 13:26:37.442987 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8c58eb7f-401b-474e-a5cd-eee60fe761b3-kube-api-access-gtdfh" (OuterVolumeSpecName: "kube-api-access-gtdfh") pod "8c58eb7f-401b-474e-a5cd-eee60fe761b3" (UID: "8c58eb7f-401b-474e-a5cd-eee60fe761b3"). InnerVolumeSpecName "kube-api-access-gtdfh". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:26:37 crc kubenswrapper[5052]: I1203 13:26:37.538893 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/8c58eb7f-401b-474e-a5cd-eee60fe761b3-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 13:26:37 crc kubenswrapper[5052]: I1203 13:26:37.538946 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-gtdfh\" (UniqueName: \"kubernetes.io/projected/8c58eb7f-401b-474e-a5cd-eee60fe761b3-kube-api-access-gtdfh\") on node \"crc\" DevicePath \"\"" Dec 03 13:26:38 crc kubenswrapper[5052]: I1203 13:26:38.179331 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="keystone-kuttl-tests/keystonef47f-account-delete-mnx42" Dec 03 13:26:38 crc kubenswrapper[5052]: I1203 13:26:38.179378 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="keystone-kuttl-tests/keystonef47f-account-delete-mnx42" event={"ID":"8c58eb7f-401b-474e-a5cd-eee60fe761b3","Type":"ContainerDied","Data":"7e5c48986619000af246da0ed43e7447e8c4eacae79d94047fc16377fefd7a35"} Dec 03 13:26:38 crc kubenswrapper[5052]: I1203 13:26:38.179456 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7e5c48986619000af246da0ed43e7447e8c4eacae79d94047fc16377fefd7a35" Dec 03 13:26:38 crc kubenswrapper[5052]: I1203 13:26:38.179456 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openshift-marketplace/community-operators-ntppl" podUID="2d8df6a3-ab5e-4c69-9f03-3fa57c872ad9" containerName="registry-server" containerID="cri-o://db19da52fb74558cbc59eec3c9e5ae3fd96de82c2f4e2d69ccf5f202de9d17dd" gracePeriod=2 Dec 03 13:26:38 crc kubenswrapper[5052]: I1203 13:26:38.601521 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/community-operators-ntppl" Dec 03 13:26:38 crc kubenswrapper[5052]: I1203 13:26:38.756166 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-46c82\" (UniqueName: \"kubernetes.io/projected/2d8df6a3-ab5e-4c69-9f03-3fa57c872ad9-kube-api-access-46c82\") pod \"2d8df6a3-ab5e-4c69-9f03-3fa57c872ad9\" (UID: \"2d8df6a3-ab5e-4c69-9f03-3fa57c872ad9\") " Dec 03 13:26:38 crc kubenswrapper[5052]: I1203 13:26:38.756328 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/2d8df6a3-ab5e-4c69-9f03-3fa57c872ad9-catalog-content\") pod \"2d8df6a3-ab5e-4c69-9f03-3fa57c872ad9\" (UID: \"2d8df6a3-ab5e-4c69-9f03-3fa57c872ad9\") " Dec 03 13:26:38 crc kubenswrapper[5052]: I1203 13:26:38.756418 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/2d8df6a3-ab5e-4c69-9f03-3fa57c872ad9-utilities\") pod \"2d8df6a3-ab5e-4c69-9f03-3fa57c872ad9\" (UID: \"2d8df6a3-ab5e-4c69-9f03-3fa57c872ad9\") " Dec 03 13:26:38 crc kubenswrapper[5052]: I1203 13:26:38.757730 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2d8df6a3-ab5e-4c69-9f03-3fa57c872ad9-utilities" (OuterVolumeSpecName: "utilities") pod "2d8df6a3-ab5e-4c69-9f03-3fa57c872ad9" (UID: "2d8df6a3-ab5e-4c69-9f03-3fa57c872ad9"). InnerVolumeSpecName "utilities". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 13:26:38 crc kubenswrapper[5052]: I1203 13:26:38.760780 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2d8df6a3-ab5e-4c69-9f03-3fa57c872ad9-kube-api-access-46c82" (OuterVolumeSpecName: "kube-api-access-46c82") pod "2d8df6a3-ab5e-4c69-9f03-3fa57c872ad9" (UID: "2d8df6a3-ab5e-4c69-9f03-3fa57c872ad9"). InnerVolumeSpecName "kube-api-access-46c82". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:26:38 crc kubenswrapper[5052]: I1203 13:26:38.804342 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2d8df6a3-ab5e-4c69-9f03-3fa57c872ad9-catalog-content" (OuterVolumeSpecName: "catalog-content") pod "2d8df6a3-ab5e-4c69-9f03-3fa57c872ad9" (UID: "2d8df6a3-ab5e-4c69-9f03-3fa57c872ad9"). InnerVolumeSpecName "catalog-content". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 13:26:38 crc kubenswrapper[5052]: I1203 13:26:38.858132 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-46c82\" (UniqueName: \"kubernetes.io/projected/2d8df6a3-ab5e-4c69-9f03-3fa57c872ad9-kube-api-access-46c82\") on node \"crc\" DevicePath \"\"" Dec 03 13:26:38 crc kubenswrapper[5052]: I1203 13:26:38.858172 5052 reconciler_common.go:293] "Volume detached for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/2d8df6a3-ab5e-4c69-9f03-3fa57c872ad9-catalog-content\") on node \"crc\" DevicePath \"\"" Dec 03 13:26:38 crc kubenswrapper[5052]: I1203 13:26:38.858182 5052 reconciler_common.go:293] "Volume detached for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/2d8df6a3-ab5e-4c69-9f03-3fa57c872ad9-utilities\") on node \"crc\" DevicePath \"\"" Dec 03 13:26:39 crc kubenswrapper[5052]: I1203 13:26:39.188035 5052 generic.go:334] "Generic (PLEG): container finished" podID="2d8df6a3-ab5e-4c69-9f03-3fa57c872ad9" containerID="db19da52fb74558cbc59eec3c9e5ae3fd96de82c2f4e2d69ccf5f202de9d17dd" exitCode=0 Dec 03 13:26:39 crc kubenswrapper[5052]: I1203 13:26:39.188069 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/community-operators-ntppl" event={"ID":"2d8df6a3-ab5e-4c69-9f03-3fa57c872ad9","Type":"ContainerDied","Data":"db19da52fb74558cbc59eec3c9e5ae3fd96de82c2f4e2d69ccf5f202de9d17dd"} Dec 03 13:26:39 crc kubenswrapper[5052]: I1203 13:26:39.188116 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/community-operators-ntppl" event={"ID":"2d8df6a3-ab5e-4c69-9f03-3fa57c872ad9","Type":"ContainerDied","Data":"aefb7e965e06938769edc8d1306eab48d628e2cb8ec73c2aad2b6f7aa13828bf"} Dec 03 13:26:39 crc kubenswrapper[5052]: I1203 13:26:39.188140 5052 scope.go:117] "RemoveContainer" containerID="db19da52fb74558cbc59eec3c9e5ae3fd96de82c2f4e2d69ccf5f202de9d17dd" Dec 03 13:26:39 crc kubenswrapper[5052]: I1203 13:26:39.188731 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/community-operators-ntppl" Dec 03 13:26:39 crc kubenswrapper[5052]: I1203 13:26:39.205346 5052 scope.go:117] "RemoveContainer" containerID="07e9239c1a3f4d71963a7c3d7ecb619ea1694e572e3eedd373a7c04997dd0e81" Dec 03 13:26:39 crc kubenswrapper[5052]: I1203 13:26:39.242040 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openshift-marketplace/community-operators-ntppl"] Dec 03 13:26:39 crc kubenswrapper[5052]: I1203 13:26:39.242738 5052 scope.go:117] "RemoveContainer" containerID="e45224d4b7d8d7869bd7739cbdd643beeff51e6943ee3bdd365b97b107f5ba02" Dec 03 13:26:39 crc kubenswrapper[5052]: I1203 13:26:39.250856 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openshift-marketplace/community-operators-ntppl"] Dec 03 13:26:39 crc kubenswrapper[5052]: I1203 13:26:39.276281 5052 scope.go:117] "RemoveContainer" containerID="db19da52fb74558cbc59eec3c9e5ae3fd96de82c2f4e2d69ccf5f202de9d17dd" Dec 03 13:26:39 crc kubenswrapper[5052]: E1203 13:26:39.278349 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"db19da52fb74558cbc59eec3c9e5ae3fd96de82c2f4e2d69ccf5f202de9d17dd\": container with ID starting with db19da52fb74558cbc59eec3c9e5ae3fd96de82c2f4e2d69ccf5f202de9d17dd not found: ID does not exist" containerID="db19da52fb74558cbc59eec3c9e5ae3fd96de82c2f4e2d69ccf5f202de9d17dd" Dec 03 13:26:39 crc kubenswrapper[5052]: I1203 13:26:39.278425 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"db19da52fb74558cbc59eec3c9e5ae3fd96de82c2f4e2d69ccf5f202de9d17dd"} err="failed to get container status \"db19da52fb74558cbc59eec3c9e5ae3fd96de82c2f4e2d69ccf5f202de9d17dd\": rpc error: code = NotFound desc = could not find container \"db19da52fb74558cbc59eec3c9e5ae3fd96de82c2f4e2d69ccf5f202de9d17dd\": container with ID starting with db19da52fb74558cbc59eec3c9e5ae3fd96de82c2f4e2d69ccf5f202de9d17dd not found: ID does not exist" Dec 03 13:26:39 crc kubenswrapper[5052]: I1203 13:26:39.278478 5052 scope.go:117] "RemoveContainer" containerID="07e9239c1a3f4d71963a7c3d7ecb619ea1694e572e3eedd373a7c04997dd0e81" Dec 03 13:26:39 crc kubenswrapper[5052]: E1203 13:26:39.278970 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"07e9239c1a3f4d71963a7c3d7ecb619ea1694e572e3eedd373a7c04997dd0e81\": container with ID starting with 07e9239c1a3f4d71963a7c3d7ecb619ea1694e572e3eedd373a7c04997dd0e81 not found: ID does not exist" containerID="07e9239c1a3f4d71963a7c3d7ecb619ea1694e572e3eedd373a7c04997dd0e81" Dec 03 13:26:39 crc kubenswrapper[5052]: I1203 13:26:39.279028 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"07e9239c1a3f4d71963a7c3d7ecb619ea1694e572e3eedd373a7c04997dd0e81"} err="failed to get container status \"07e9239c1a3f4d71963a7c3d7ecb619ea1694e572e3eedd373a7c04997dd0e81\": rpc error: code = NotFound desc = could not find container \"07e9239c1a3f4d71963a7c3d7ecb619ea1694e572e3eedd373a7c04997dd0e81\": container with ID starting with 07e9239c1a3f4d71963a7c3d7ecb619ea1694e572e3eedd373a7c04997dd0e81 not found: ID does not exist" Dec 03 13:26:39 crc kubenswrapper[5052]: I1203 13:26:39.279062 5052 scope.go:117] "RemoveContainer" containerID="e45224d4b7d8d7869bd7739cbdd643beeff51e6943ee3bdd365b97b107f5ba02" Dec 03 13:26:39 crc kubenswrapper[5052]: E1203 13:26:39.281944 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e45224d4b7d8d7869bd7739cbdd643beeff51e6943ee3bdd365b97b107f5ba02\": container with ID starting with e45224d4b7d8d7869bd7739cbdd643beeff51e6943ee3bdd365b97b107f5ba02 not found: ID does not exist" containerID="e45224d4b7d8d7869bd7739cbdd643beeff51e6943ee3bdd365b97b107f5ba02" Dec 03 13:26:39 crc kubenswrapper[5052]: I1203 13:26:39.281975 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e45224d4b7d8d7869bd7739cbdd643beeff51e6943ee3bdd365b97b107f5ba02"} err="failed to get container status \"e45224d4b7d8d7869bd7739cbdd643beeff51e6943ee3bdd365b97b107f5ba02\": rpc error: code = NotFound desc = could not find container \"e45224d4b7d8d7869bd7739cbdd643beeff51e6943ee3bdd365b97b107f5ba02\": container with ID starting with e45224d4b7d8d7869bd7739cbdd643beeff51e6943ee3bdd365b97b107f5ba02 not found: ID does not exist" Dec 03 13:26:39 crc kubenswrapper[5052]: I1203 13:26:39.511989 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["keystone-kuttl-tests/keystone-db-create-d4zvz"] Dec 03 13:26:39 crc kubenswrapper[5052]: I1203 13:26:39.519288 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["keystone-kuttl-tests/keystone-db-create-d4zvz"] Dec 03 13:26:39 crc kubenswrapper[5052]: I1203 13:26:39.530896 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["keystone-kuttl-tests/keystonef47f-account-delete-mnx42"] Dec 03 13:26:39 crc kubenswrapper[5052]: I1203 13:26:39.536756 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["keystone-kuttl-tests/keystone-f47f-account-create-update-9bf9g"] Dec 03 13:26:39 crc kubenswrapper[5052]: I1203 13:26:39.542423 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["keystone-kuttl-tests/keystonef47f-account-delete-mnx42"] Dec 03 13:26:39 crc kubenswrapper[5052]: I1203 13:26:39.548576 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["keystone-kuttl-tests/keystone-f47f-account-create-update-9bf9g"] Dec 03 13:26:39 crc kubenswrapper[5052]: I1203 13:26:39.606108 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["keystone-kuttl-tests/keystone-db-create-g5klp"] Dec 03 13:26:39 crc kubenswrapper[5052]: E1203 13:26:39.606706 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="8c58eb7f-401b-474e-a5cd-eee60fe761b3" containerName="mariadb-account-delete" Dec 03 13:26:39 crc kubenswrapper[5052]: I1203 13:26:39.606738 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="8c58eb7f-401b-474e-a5cd-eee60fe761b3" containerName="mariadb-account-delete" Dec 03 13:26:39 crc kubenswrapper[5052]: E1203 13:26:39.606771 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="2d8df6a3-ab5e-4c69-9f03-3fa57c872ad9" containerName="extract-content" Dec 03 13:26:39 crc kubenswrapper[5052]: I1203 13:26:39.606782 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d8df6a3-ab5e-4c69-9f03-3fa57c872ad9" containerName="extract-content" Dec 03 13:26:39 crc kubenswrapper[5052]: E1203 13:26:39.606804 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="2d8df6a3-ab5e-4c69-9f03-3fa57c872ad9" containerName="extract-utilities" Dec 03 13:26:39 crc kubenswrapper[5052]: I1203 13:26:39.606814 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d8df6a3-ab5e-4c69-9f03-3fa57c872ad9" containerName="extract-utilities" Dec 03 13:26:39 crc kubenswrapper[5052]: E1203 13:26:39.606828 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="2d8df6a3-ab5e-4c69-9f03-3fa57c872ad9" containerName="registry-server" Dec 03 13:26:39 crc kubenswrapper[5052]: I1203 13:26:39.606838 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d8df6a3-ab5e-4c69-9f03-3fa57c872ad9" containerName="registry-server" Dec 03 13:26:39 crc kubenswrapper[5052]: I1203 13:26:39.607057 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="8c58eb7f-401b-474e-a5cd-eee60fe761b3" containerName="mariadb-account-delete" Dec 03 13:26:39 crc kubenswrapper[5052]: I1203 13:26:39.607080 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="2d8df6a3-ab5e-4c69-9f03-3fa57c872ad9" containerName="registry-server" Dec 03 13:26:39 crc kubenswrapper[5052]: I1203 13:26:39.608015 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="keystone-kuttl-tests/keystone-db-create-g5klp" Dec 03 13:26:39 crc kubenswrapper[5052]: I1203 13:26:39.612771 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["keystone-kuttl-tests/keystone-db-create-g5klp"] Dec 03 13:26:39 crc kubenswrapper[5052]: I1203 13:26:39.686787 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2d8df6a3-ab5e-4c69-9f03-3fa57c872ad9" path="/var/lib/kubelet/pods/2d8df6a3-ab5e-4c69-9f03-3fa57c872ad9/volumes" Dec 03 13:26:39 crc kubenswrapper[5052]: I1203 13:26:39.687898 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6397f36b-76b0-4f60-a546-3f9c2ca74bb8" path="/var/lib/kubelet/pods/6397f36b-76b0-4f60-a546-3f9c2ca74bb8/volumes" Dec 03 13:26:39 crc kubenswrapper[5052]: I1203 13:26:39.691914 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8c58eb7f-401b-474e-a5cd-eee60fe761b3" path="/var/lib/kubelet/pods/8c58eb7f-401b-474e-a5cd-eee60fe761b3/volumes" Dec 03 13:26:39 crc kubenswrapper[5052]: I1203 13:26:39.693610 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9186bb8c-cb25-4bbb-9b54-b53b64a0c858" path="/var/lib/kubelet/pods/9186bb8c-cb25-4bbb-9b54-b53b64a0c858/volumes" Dec 03 13:26:39 crc kubenswrapper[5052]: I1203 13:26:39.720633 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["keystone-kuttl-tests/keystone-052c-account-create-update-rjc4m"] Dec 03 13:26:39 crc kubenswrapper[5052]: I1203 13:26:39.721786 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="keystone-kuttl-tests/keystone-052c-account-create-update-rjc4m" Dec 03 13:26:39 crc kubenswrapper[5052]: I1203 13:26:39.724199 5052 reflector.go:368] Caches populated for *v1.Secret from object-"keystone-kuttl-tests"/"keystone-db-secret" Dec 03 13:26:39 crc kubenswrapper[5052]: I1203 13:26:39.728059 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["keystone-kuttl-tests/keystone-052c-account-create-update-rjc4m"] Dec 03 13:26:39 crc kubenswrapper[5052]: I1203 13:26:39.772636 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zpzrz\" (UniqueName: \"kubernetes.io/projected/90a0c638-12e3-40ed-9926-751d20df9208-kube-api-access-zpzrz\") pod \"keystone-db-create-g5klp\" (UID: \"90a0c638-12e3-40ed-9926-751d20df9208\") " pod="keystone-kuttl-tests/keystone-db-create-g5klp" Dec 03 13:26:39 crc kubenswrapper[5052]: I1203 13:26:39.772775 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/90a0c638-12e3-40ed-9926-751d20df9208-operator-scripts\") pod \"keystone-db-create-g5klp\" (UID: \"90a0c638-12e3-40ed-9926-751d20df9208\") " pod="keystone-kuttl-tests/keystone-db-create-g5klp" Dec 03 13:26:39 crc kubenswrapper[5052]: I1203 13:26:39.873963 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5kb2f\" (UniqueName: \"kubernetes.io/projected/d734fba5-735c-4a3a-828a-8a205069e17f-kube-api-access-5kb2f\") pod \"keystone-052c-account-create-update-rjc4m\" (UID: \"d734fba5-735c-4a3a-828a-8a205069e17f\") " pod="keystone-kuttl-tests/keystone-052c-account-create-update-rjc4m" Dec 03 13:26:39 crc kubenswrapper[5052]: I1203 13:26:39.874048 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/d734fba5-735c-4a3a-828a-8a205069e17f-operator-scripts\") pod \"keystone-052c-account-create-update-rjc4m\" (UID: \"d734fba5-735c-4a3a-828a-8a205069e17f\") " pod="keystone-kuttl-tests/keystone-052c-account-create-update-rjc4m" Dec 03 13:26:39 crc kubenswrapper[5052]: I1203 13:26:39.874187 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/90a0c638-12e3-40ed-9926-751d20df9208-operator-scripts\") pod \"keystone-db-create-g5klp\" (UID: \"90a0c638-12e3-40ed-9926-751d20df9208\") " pod="keystone-kuttl-tests/keystone-db-create-g5klp" Dec 03 13:26:39 crc kubenswrapper[5052]: I1203 13:26:39.874375 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-zpzrz\" (UniqueName: \"kubernetes.io/projected/90a0c638-12e3-40ed-9926-751d20df9208-kube-api-access-zpzrz\") pod \"keystone-db-create-g5klp\" (UID: \"90a0c638-12e3-40ed-9926-751d20df9208\") " pod="keystone-kuttl-tests/keystone-db-create-g5klp" Dec 03 13:26:39 crc kubenswrapper[5052]: I1203 13:26:39.874828 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/90a0c638-12e3-40ed-9926-751d20df9208-operator-scripts\") pod \"keystone-db-create-g5klp\" (UID: \"90a0c638-12e3-40ed-9926-751d20df9208\") " pod="keystone-kuttl-tests/keystone-db-create-g5klp" Dec 03 13:26:39 crc kubenswrapper[5052]: I1203 13:26:39.898732 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-zpzrz\" (UniqueName: \"kubernetes.io/projected/90a0c638-12e3-40ed-9926-751d20df9208-kube-api-access-zpzrz\") pod \"keystone-db-create-g5klp\" (UID: \"90a0c638-12e3-40ed-9926-751d20df9208\") " pod="keystone-kuttl-tests/keystone-db-create-g5klp" Dec 03 13:26:39 crc kubenswrapper[5052]: I1203 13:26:39.927818 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="keystone-kuttl-tests/keystone-db-create-g5klp" Dec 03 13:26:39 crc kubenswrapper[5052]: I1203 13:26:39.976626 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-5kb2f\" (UniqueName: \"kubernetes.io/projected/d734fba5-735c-4a3a-828a-8a205069e17f-kube-api-access-5kb2f\") pod \"keystone-052c-account-create-update-rjc4m\" (UID: \"d734fba5-735c-4a3a-828a-8a205069e17f\") " pod="keystone-kuttl-tests/keystone-052c-account-create-update-rjc4m" Dec 03 13:26:39 crc kubenswrapper[5052]: I1203 13:26:39.976698 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/d734fba5-735c-4a3a-828a-8a205069e17f-operator-scripts\") pod \"keystone-052c-account-create-update-rjc4m\" (UID: \"d734fba5-735c-4a3a-828a-8a205069e17f\") " pod="keystone-kuttl-tests/keystone-052c-account-create-update-rjc4m" Dec 03 13:26:39 crc kubenswrapper[5052]: I1203 13:26:39.977913 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/d734fba5-735c-4a3a-828a-8a205069e17f-operator-scripts\") pod \"keystone-052c-account-create-update-rjc4m\" (UID: \"d734fba5-735c-4a3a-828a-8a205069e17f\") " pod="keystone-kuttl-tests/keystone-052c-account-create-update-rjc4m" Dec 03 13:26:39 crc kubenswrapper[5052]: I1203 13:26:39.995236 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-5kb2f\" (UniqueName: \"kubernetes.io/projected/d734fba5-735c-4a3a-828a-8a205069e17f-kube-api-access-5kb2f\") pod \"keystone-052c-account-create-update-rjc4m\" (UID: \"d734fba5-735c-4a3a-828a-8a205069e17f\") " pod="keystone-kuttl-tests/keystone-052c-account-create-update-rjc4m" Dec 03 13:26:40 crc kubenswrapper[5052]: I1203 13:26:40.040467 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="keystone-kuttl-tests/keystone-052c-account-create-update-rjc4m" Dec 03 13:26:40 crc kubenswrapper[5052]: I1203 13:26:40.345433 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["keystone-kuttl-tests/keystone-db-create-g5klp"] Dec 03 13:26:40 crc kubenswrapper[5052]: W1203 13:26:40.347923 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod90a0c638_12e3_40ed_9926_751d20df9208.slice/crio-8c88143a1c3d3afa5a870c10b2df18029ade57109c763e00fc64b8ebbfbda5d7 WatchSource:0}: Error finding container 8c88143a1c3d3afa5a870c10b2df18029ade57109c763e00fc64b8ebbfbda5d7: Status 404 returned error can't find the container with id 8c88143a1c3d3afa5a870c10b2df18029ade57109c763e00fc64b8ebbfbda5d7 Dec 03 13:26:40 crc kubenswrapper[5052]: I1203 13:26:40.486396 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["keystone-kuttl-tests/keystone-052c-account-create-update-rjc4m"] Dec 03 13:26:40 crc kubenswrapper[5052]: W1203 13:26:40.494979 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd734fba5_735c_4a3a_828a_8a205069e17f.slice/crio-a62db994c1bae871dbe09d9a49095f7620f966d352c6d7e63bc63fabe7b12aca WatchSource:0}: Error finding container a62db994c1bae871dbe09d9a49095f7620f966d352c6d7e63bc63fabe7b12aca: Status 404 returned error can't find the container with id a62db994c1bae871dbe09d9a49095f7620f966d352c6d7e63bc63fabe7b12aca Dec 03 13:26:41 crc kubenswrapper[5052]: I1203 13:26:41.205946 5052 generic.go:334] "Generic (PLEG): container finished" podID="d734fba5-735c-4a3a-828a-8a205069e17f" containerID="2dd0532fc46c347cff5bc379a6f28c1b515400bdbd08af61a5b258613138c7ff" exitCode=0 Dec 03 13:26:41 crc kubenswrapper[5052]: I1203 13:26:41.206016 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="keystone-kuttl-tests/keystone-052c-account-create-update-rjc4m" event={"ID":"d734fba5-735c-4a3a-828a-8a205069e17f","Type":"ContainerDied","Data":"2dd0532fc46c347cff5bc379a6f28c1b515400bdbd08af61a5b258613138c7ff"} Dec 03 13:26:41 crc kubenswrapper[5052]: I1203 13:26:41.206753 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="keystone-kuttl-tests/keystone-052c-account-create-update-rjc4m" event={"ID":"d734fba5-735c-4a3a-828a-8a205069e17f","Type":"ContainerStarted","Data":"a62db994c1bae871dbe09d9a49095f7620f966d352c6d7e63bc63fabe7b12aca"} Dec 03 13:26:41 crc kubenswrapper[5052]: I1203 13:26:41.208627 5052 generic.go:334] "Generic (PLEG): container finished" podID="90a0c638-12e3-40ed-9926-751d20df9208" containerID="14b3a3fb2e82ee2d3772353eb2afdd3eec1e8b6a6efc3575f8cfcb36c98eb4da" exitCode=0 Dec 03 13:26:41 crc kubenswrapper[5052]: I1203 13:26:41.208656 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="keystone-kuttl-tests/keystone-db-create-g5klp" event={"ID":"90a0c638-12e3-40ed-9926-751d20df9208","Type":"ContainerDied","Data":"14b3a3fb2e82ee2d3772353eb2afdd3eec1e8b6a6efc3575f8cfcb36c98eb4da"} Dec 03 13:26:41 crc kubenswrapper[5052]: I1203 13:26:41.208692 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="keystone-kuttl-tests/keystone-db-create-g5klp" event={"ID":"90a0c638-12e3-40ed-9926-751d20df9208","Type":"ContainerStarted","Data":"8c88143a1c3d3afa5a870c10b2df18029ade57109c763e00fc64b8ebbfbda5d7"} Dec 03 13:26:42 crc kubenswrapper[5052]: I1203 13:26:42.596204 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="keystone-kuttl-tests/keystone-052c-account-create-update-rjc4m" Dec 03 13:26:42 crc kubenswrapper[5052]: I1203 13:26:42.604921 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="keystone-kuttl-tests/keystone-db-create-g5klp" Dec 03 13:26:42 crc kubenswrapper[5052]: I1203 13:26:42.737582 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zpzrz\" (UniqueName: \"kubernetes.io/projected/90a0c638-12e3-40ed-9926-751d20df9208-kube-api-access-zpzrz\") pod \"90a0c638-12e3-40ed-9926-751d20df9208\" (UID: \"90a0c638-12e3-40ed-9926-751d20df9208\") " Dec 03 13:26:42 crc kubenswrapper[5052]: I1203 13:26:42.737746 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5kb2f\" (UniqueName: \"kubernetes.io/projected/d734fba5-735c-4a3a-828a-8a205069e17f-kube-api-access-5kb2f\") pod \"d734fba5-735c-4a3a-828a-8a205069e17f\" (UID: \"d734fba5-735c-4a3a-828a-8a205069e17f\") " Dec 03 13:26:42 crc kubenswrapper[5052]: I1203 13:26:42.737808 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/90a0c638-12e3-40ed-9926-751d20df9208-operator-scripts\") pod \"90a0c638-12e3-40ed-9926-751d20df9208\" (UID: \"90a0c638-12e3-40ed-9926-751d20df9208\") " Dec 03 13:26:42 crc kubenswrapper[5052]: I1203 13:26:42.737846 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/d734fba5-735c-4a3a-828a-8a205069e17f-operator-scripts\") pod \"d734fba5-735c-4a3a-828a-8a205069e17f\" (UID: \"d734fba5-735c-4a3a-828a-8a205069e17f\") " Dec 03 13:26:42 crc kubenswrapper[5052]: I1203 13:26:42.738490 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d734fba5-735c-4a3a-828a-8a205069e17f-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "d734fba5-735c-4a3a-828a-8a205069e17f" (UID: "d734fba5-735c-4a3a-828a-8a205069e17f"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:26:42 crc kubenswrapper[5052]: I1203 13:26:42.738951 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/90a0c638-12e3-40ed-9926-751d20df9208-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "90a0c638-12e3-40ed-9926-751d20df9208" (UID: "90a0c638-12e3-40ed-9926-751d20df9208"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:26:42 crc kubenswrapper[5052]: I1203 13:26:42.743485 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d734fba5-735c-4a3a-828a-8a205069e17f-kube-api-access-5kb2f" (OuterVolumeSpecName: "kube-api-access-5kb2f") pod "d734fba5-735c-4a3a-828a-8a205069e17f" (UID: "d734fba5-735c-4a3a-828a-8a205069e17f"). InnerVolumeSpecName "kube-api-access-5kb2f". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:26:42 crc kubenswrapper[5052]: I1203 13:26:42.744592 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/90a0c638-12e3-40ed-9926-751d20df9208-kube-api-access-zpzrz" (OuterVolumeSpecName: "kube-api-access-zpzrz") pod "90a0c638-12e3-40ed-9926-751d20df9208" (UID: "90a0c638-12e3-40ed-9926-751d20df9208"). InnerVolumeSpecName "kube-api-access-zpzrz". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:26:42 crc kubenswrapper[5052]: I1203 13:26:42.840407 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-zpzrz\" (UniqueName: \"kubernetes.io/projected/90a0c638-12e3-40ed-9926-751d20df9208-kube-api-access-zpzrz\") on node \"crc\" DevicePath \"\"" Dec 03 13:26:42 crc kubenswrapper[5052]: I1203 13:26:42.840878 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-5kb2f\" (UniqueName: \"kubernetes.io/projected/d734fba5-735c-4a3a-828a-8a205069e17f-kube-api-access-5kb2f\") on node \"crc\" DevicePath \"\"" Dec 03 13:26:42 crc kubenswrapper[5052]: I1203 13:26:42.840910 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/90a0c638-12e3-40ed-9926-751d20df9208-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 13:26:42 crc kubenswrapper[5052]: I1203 13:26:42.840922 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/d734fba5-735c-4a3a-828a-8a205069e17f-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 13:26:43 crc kubenswrapper[5052]: I1203 13:26:43.224280 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="keystone-kuttl-tests/keystone-db-create-g5klp" event={"ID":"90a0c638-12e3-40ed-9926-751d20df9208","Type":"ContainerDied","Data":"8c88143a1c3d3afa5a870c10b2df18029ade57109c763e00fc64b8ebbfbda5d7"} Dec 03 13:26:43 crc kubenswrapper[5052]: I1203 13:26:43.224319 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="keystone-kuttl-tests/keystone-db-create-g5klp" Dec 03 13:26:43 crc kubenswrapper[5052]: I1203 13:26:43.224325 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8c88143a1c3d3afa5a870c10b2df18029ade57109c763e00fc64b8ebbfbda5d7" Dec 03 13:26:43 crc kubenswrapper[5052]: I1203 13:26:43.226314 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="keystone-kuttl-tests/keystone-052c-account-create-update-rjc4m" event={"ID":"d734fba5-735c-4a3a-828a-8a205069e17f","Type":"ContainerDied","Data":"a62db994c1bae871dbe09d9a49095f7620f966d352c6d7e63bc63fabe7b12aca"} Dec 03 13:26:43 crc kubenswrapper[5052]: I1203 13:26:43.226345 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a62db994c1bae871dbe09d9a49095f7620f966d352c6d7e63bc63fabe7b12aca" Dec 03 13:26:43 crc kubenswrapper[5052]: I1203 13:26:43.226541 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="keystone-kuttl-tests/keystone-052c-account-create-update-rjc4m" Dec 03 13:26:45 crc kubenswrapper[5052]: I1203 13:26:45.287039 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["keystone-kuttl-tests/keystone-db-sync-xnk4r"] Dec 03 13:26:45 crc kubenswrapper[5052]: E1203 13:26:45.287826 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="90a0c638-12e3-40ed-9926-751d20df9208" containerName="mariadb-database-create" Dec 03 13:26:45 crc kubenswrapper[5052]: I1203 13:26:45.287838 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="90a0c638-12e3-40ed-9926-751d20df9208" containerName="mariadb-database-create" Dec 03 13:26:45 crc kubenswrapper[5052]: E1203 13:26:45.287866 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="d734fba5-735c-4a3a-828a-8a205069e17f" containerName="mariadb-account-create-update" Dec 03 13:26:45 crc kubenswrapper[5052]: I1203 13:26:45.287872 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="d734fba5-735c-4a3a-828a-8a205069e17f" containerName="mariadb-account-create-update" Dec 03 13:26:45 crc kubenswrapper[5052]: I1203 13:26:45.288026 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="90a0c638-12e3-40ed-9926-751d20df9208" containerName="mariadb-database-create" Dec 03 13:26:45 crc kubenswrapper[5052]: I1203 13:26:45.288049 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="d734fba5-735c-4a3a-828a-8a205069e17f" containerName="mariadb-account-create-update" Dec 03 13:26:45 crc kubenswrapper[5052]: I1203 13:26:45.288655 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="keystone-kuttl-tests/keystone-db-sync-xnk4r" Dec 03 13:26:45 crc kubenswrapper[5052]: I1203 13:26:45.293126 5052 reflector.go:368] Caches populated for *v1.Secret from object-"keystone-kuttl-tests"/"keystone-scripts" Dec 03 13:26:45 crc kubenswrapper[5052]: I1203 13:26:45.293907 5052 reflector.go:368] Caches populated for *v1.Secret from object-"keystone-kuttl-tests"/"keystone-config-data" Dec 03 13:26:45 crc kubenswrapper[5052]: I1203 13:26:45.294163 5052 reflector.go:368] Caches populated for *v1.Secret from object-"keystone-kuttl-tests"/"keystone-keystone-dockercfg-6nzts" Dec 03 13:26:45 crc kubenswrapper[5052]: I1203 13:26:45.294502 5052 reflector.go:368] Caches populated for *v1.Secret from object-"keystone-kuttl-tests"/"keystone" Dec 03 13:26:45 crc kubenswrapper[5052]: I1203 13:26:45.296644 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["keystone-kuttl-tests/keystone-db-sync-xnk4r"] Dec 03 13:26:45 crc kubenswrapper[5052]: I1203 13:26:45.476572 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4tqnc\" (UniqueName: \"kubernetes.io/projected/6bc59279-5d0c-41d6-9d4b-c80b03620cb0-kube-api-access-4tqnc\") pod \"keystone-db-sync-xnk4r\" (UID: \"6bc59279-5d0c-41d6-9d4b-c80b03620cb0\") " pod="keystone-kuttl-tests/keystone-db-sync-xnk4r" Dec 03 13:26:45 crc kubenswrapper[5052]: I1203 13:26:45.476833 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6bc59279-5d0c-41d6-9d4b-c80b03620cb0-config-data\") pod \"keystone-db-sync-xnk4r\" (UID: \"6bc59279-5d0c-41d6-9d4b-c80b03620cb0\") " pod="keystone-kuttl-tests/keystone-db-sync-xnk4r" Dec 03 13:26:45 crc kubenswrapper[5052]: I1203 13:26:45.578305 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-4tqnc\" (UniqueName: \"kubernetes.io/projected/6bc59279-5d0c-41d6-9d4b-c80b03620cb0-kube-api-access-4tqnc\") pod \"keystone-db-sync-xnk4r\" (UID: \"6bc59279-5d0c-41d6-9d4b-c80b03620cb0\") " pod="keystone-kuttl-tests/keystone-db-sync-xnk4r" Dec 03 13:26:45 crc kubenswrapper[5052]: I1203 13:26:45.578445 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6bc59279-5d0c-41d6-9d4b-c80b03620cb0-config-data\") pod \"keystone-db-sync-xnk4r\" (UID: \"6bc59279-5d0c-41d6-9d4b-c80b03620cb0\") " pod="keystone-kuttl-tests/keystone-db-sync-xnk4r" Dec 03 13:26:45 crc kubenswrapper[5052]: I1203 13:26:45.585827 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6bc59279-5d0c-41d6-9d4b-c80b03620cb0-config-data\") pod \"keystone-db-sync-xnk4r\" (UID: \"6bc59279-5d0c-41d6-9d4b-c80b03620cb0\") " pod="keystone-kuttl-tests/keystone-db-sync-xnk4r" Dec 03 13:26:45 crc kubenswrapper[5052]: I1203 13:26:45.601215 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-4tqnc\" (UniqueName: \"kubernetes.io/projected/6bc59279-5d0c-41d6-9d4b-c80b03620cb0-kube-api-access-4tqnc\") pod \"keystone-db-sync-xnk4r\" (UID: \"6bc59279-5d0c-41d6-9d4b-c80b03620cb0\") " pod="keystone-kuttl-tests/keystone-db-sync-xnk4r" Dec 03 13:26:45 crc kubenswrapper[5052]: I1203 13:26:45.612162 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="keystone-kuttl-tests/keystone-db-sync-xnk4r" Dec 03 13:26:46 crc kubenswrapper[5052]: I1203 13:26:46.066401 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["keystone-kuttl-tests/keystone-db-sync-xnk4r"] Dec 03 13:26:46 crc kubenswrapper[5052]: I1203 13:26:46.267628 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="keystone-kuttl-tests/keystone-db-sync-xnk4r" event={"ID":"6bc59279-5d0c-41d6-9d4b-c80b03620cb0","Type":"ContainerStarted","Data":"ebbd9e155f51ffb015f7ba820bcc86ed1c54112a262104e4180e88e71517c408"} Dec 03 13:26:46 crc kubenswrapper[5052]: I1203 13:26:46.267687 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="keystone-kuttl-tests/keystone-db-sync-xnk4r" event={"ID":"6bc59279-5d0c-41d6-9d4b-c80b03620cb0","Type":"ContainerStarted","Data":"ab037c697c3d0e0d7cb64df33fe8a9452335011b2a5a5361be0ec36c49b9b16e"} Dec 03 13:26:46 crc kubenswrapper[5052]: I1203 13:26:46.299473 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="keystone-kuttl-tests/keystone-db-sync-xnk4r" podStartSLOduration=1.299444746 podStartE2EDuration="1.299444746s" podCreationTimestamp="2025-12-03 13:26:45 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 13:26:46.291980572 +0000 UTC m=+7596.880222336" watchObservedRunningTime="2025-12-03 13:26:46.299444746 +0000 UTC m=+7596.887686510" Dec 03 13:26:48 crc kubenswrapper[5052]: I1203 13:26:48.288921 5052 generic.go:334] "Generic (PLEG): container finished" podID="6bc59279-5d0c-41d6-9d4b-c80b03620cb0" containerID="ebbd9e155f51ffb015f7ba820bcc86ed1c54112a262104e4180e88e71517c408" exitCode=0 Dec 03 13:26:48 crc kubenswrapper[5052]: I1203 13:26:48.289065 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="keystone-kuttl-tests/keystone-db-sync-xnk4r" event={"ID":"6bc59279-5d0c-41d6-9d4b-c80b03620cb0","Type":"ContainerDied","Data":"ebbd9e155f51ffb015f7ba820bcc86ed1c54112a262104e4180e88e71517c408"} Dec 03 13:26:49 crc kubenswrapper[5052]: I1203 13:26:49.613286 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="keystone-kuttl-tests/keystone-db-sync-xnk4r" Dec 03 13:26:49 crc kubenswrapper[5052]: I1203 13:26:49.649729 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4tqnc\" (UniqueName: \"kubernetes.io/projected/6bc59279-5d0c-41d6-9d4b-c80b03620cb0-kube-api-access-4tqnc\") pod \"6bc59279-5d0c-41d6-9d4b-c80b03620cb0\" (UID: \"6bc59279-5d0c-41d6-9d4b-c80b03620cb0\") " Dec 03 13:26:49 crc kubenswrapper[5052]: I1203 13:26:49.649842 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6bc59279-5d0c-41d6-9d4b-c80b03620cb0-config-data\") pod \"6bc59279-5d0c-41d6-9d4b-c80b03620cb0\" (UID: \"6bc59279-5d0c-41d6-9d4b-c80b03620cb0\") " Dec 03 13:26:49 crc kubenswrapper[5052]: I1203 13:26:49.655704 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6bc59279-5d0c-41d6-9d4b-c80b03620cb0-kube-api-access-4tqnc" (OuterVolumeSpecName: "kube-api-access-4tqnc") pod "6bc59279-5d0c-41d6-9d4b-c80b03620cb0" (UID: "6bc59279-5d0c-41d6-9d4b-c80b03620cb0"). InnerVolumeSpecName "kube-api-access-4tqnc". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:26:49 crc kubenswrapper[5052]: I1203 13:26:49.699692 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6bc59279-5d0c-41d6-9d4b-c80b03620cb0-config-data" (OuterVolumeSpecName: "config-data") pod "6bc59279-5d0c-41d6-9d4b-c80b03620cb0" (UID: "6bc59279-5d0c-41d6-9d4b-c80b03620cb0"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:26:49 crc kubenswrapper[5052]: I1203 13:26:49.751942 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/6bc59279-5d0c-41d6-9d4b-c80b03620cb0-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 13:26:49 crc kubenswrapper[5052]: I1203 13:26:49.752015 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-4tqnc\" (UniqueName: \"kubernetes.io/projected/6bc59279-5d0c-41d6-9d4b-c80b03620cb0-kube-api-access-4tqnc\") on node \"crc\" DevicePath \"\"" Dec 03 13:26:50 crc kubenswrapper[5052]: I1203 13:26:50.314846 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="keystone-kuttl-tests/keystone-db-sync-xnk4r" event={"ID":"6bc59279-5d0c-41d6-9d4b-c80b03620cb0","Type":"ContainerDied","Data":"ab037c697c3d0e0d7cb64df33fe8a9452335011b2a5a5361be0ec36c49b9b16e"} Dec 03 13:26:50 crc kubenswrapper[5052]: I1203 13:26:50.314892 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ab037c697c3d0e0d7cb64df33fe8a9452335011b2a5a5361be0ec36c49b9b16e" Dec 03 13:26:50 crc kubenswrapper[5052]: I1203 13:26:50.314871 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="keystone-kuttl-tests/keystone-db-sync-xnk4r" Dec 03 13:26:50 crc kubenswrapper[5052]: I1203 13:26:50.458870 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["keystone-kuttl-tests/keystone-bootstrap-p7ld6"] Dec 03 13:26:50 crc kubenswrapper[5052]: E1203 13:26:50.459143 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="6bc59279-5d0c-41d6-9d4b-c80b03620cb0" containerName="keystone-db-sync" Dec 03 13:26:50 crc kubenswrapper[5052]: I1203 13:26:50.459157 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="6bc59279-5d0c-41d6-9d4b-c80b03620cb0" containerName="keystone-db-sync" Dec 03 13:26:50 crc kubenswrapper[5052]: I1203 13:26:50.459283 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="6bc59279-5d0c-41d6-9d4b-c80b03620cb0" containerName="keystone-db-sync" Dec 03 13:26:50 crc kubenswrapper[5052]: I1203 13:26:50.459790 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="keystone-kuttl-tests/keystone-bootstrap-p7ld6" Dec 03 13:26:50 crc kubenswrapper[5052]: I1203 13:26:50.464553 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lnbgc\" (UniqueName: \"kubernetes.io/projected/8b51a88e-2fe0-48a5-8ad3-6621ee91749d-kube-api-access-lnbgc\") pod \"keystone-bootstrap-p7ld6\" (UID: \"8b51a88e-2fe0-48a5-8ad3-6621ee91749d\") " pod="keystone-kuttl-tests/keystone-bootstrap-p7ld6" Dec 03 13:26:50 crc kubenswrapper[5052]: I1203 13:26:50.465025 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/8b51a88e-2fe0-48a5-8ad3-6621ee91749d-config-data\") pod \"keystone-bootstrap-p7ld6\" (UID: \"8b51a88e-2fe0-48a5-8ad3-6621ee91749d\") " pod="keystone-kuttl-tests/keystone-bootstrap-p7ld6" Dec 03 13:26:50 crc kubenswrapper[5052]: I1203 13:26:50.465082 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/8b51a88e-2fe0-48a5-8ad3-6621ee91749d-scripts\") pod \"keystone-bootstrap-p7ld6\" (UID: \"8b51a88e-2fe0-48a5-8ad3-6621ee91749d\") " pod="keystone-kuttl-tests/keystone-bootstrap-p7ld6" Dec 03 13:26:50 crc kubenswrapper[5052]: I1203 13:26:50.465140 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/8b51a88e-2fe0-48a5-8ad3-6621ee91749d-fernet-keys\") pod \"keystone-bootstrap-p7ld6\" (UID: \"8b51a88e-2fe0-48a5-8ad3-6621ee91749d\") " pod="keystone-kuttl-tests/keystone-bootstrap-p7ld6" Dec 03 13:26:50 crc kubenswrapper[5052]: I1203 13:26:50.465192 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/8b51a88e-2fe0-48a5-8ad3-6621ee91749d-credential-keys\") pod \"keystone-bootstrap-p7ld6\" (UID: \"8b51a88e-2fe0-48a5-8ad3-6621ee91749d\") " pod="keystone-kuttl-tests/keystone-bootstrap-p7ld6" Dec 03 13:26:50 crc kubenswrapper[5052]: I1203 13:26:50.464927 5052 reflector.go:368] Caches populated for *v1.Secret from object-"keystone-kuttl-tests"/"keystone-scripts" Dec 03 13:26:50 crc kubenswrapper[5052]: I1203 13:26:50.465302 5052 reflector.go:368] Caches populated for *v1.Secret from object-"keystone-kuttl-tests"/"keystone-config-data" Dec 03 13:26:50 crc kubenswrapper[5052]: I1203 13:26:50.465093 5052 reflector.go:368] Caches populated for *v1.Secret from object-"keystone-kuttl-tests"/"keystone-keystone-dockercfg-6nzts" Dec 03 13:26:50 crc kubenswrapper[5052]: I1203 13:26:50.465228 5052 reflector.go:368] Caches populated for *v1.Secret from object-"keystone-kuttl-tests"/"keystone" Dec 03 13:26:50 crc kubenswrapper[5052]: I1203 13:26:50.465269 5052 reflector.go:368] Caches populated for *v1.Secret from object-"keystone-kuttl-tests"/"osp-secret" Dec 03 13:26:50 crc kubenswrapper[5052]: I1203 13:26:50.488916 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["keystone-kuttl-tests/keystone-bootstrap-p7ld6"] Dec 03 13:26:50 crc kubenswrapper[5052]: I1203 13:26:50.566411 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/8b51a88e-2fe0-48a5-8ad3-6621ee91749d-fernet-keys\") pod \"keystone-bootstrap-p7ld6\" (UID: \"8b51a88e-2fe0-48a5-8ad3-6621ee91749d\") " pod="keystone-kuttl-tests/keystone-bootstrap-p7ld6" Dec 03 13:26:50 crc kubenswrapper[5052]: I1203 13:26:50.566805 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/8b51a88e-2fe0-48a5-8ad3-6621ee91749d-credential-keys\") pod \"keystone-bootstrap-p7ld6\" (UID: \"8b51a88e-2fe0-48a5-8ad3-6621ee91749d\") " pod="keystone-kuttl-tests/keystone-bootstrap-p7ld6" Dec 03 13:26:50 crc kubenswrapper[5052]: I1203 13:26:50.566947 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-lnbgc\" (UniqueName: \"kubernetes.io/projected/8b51a88e-2fe0-48a5-8ad3-6621ee91749d-kube-api-access-lnbgc\") pod \"keystone-bootstrap-p7ld6\" (UID: \"8b51a88e-2fe0-48a5-8ad3-6621ee91749d\") " pod="keystone-kuttl-tests/keystone-bootstrap-p7ld6" Dec 03 13:26:50 crc kubenswrapper[5052]: I1203 13:26:50.567025 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/8b51a88e-2fe0-48a5-8ad3-6621ee91749d-config-data\") pod \"keystone-bootstrap-p7ld6\" (UID: \"8b51a88e-2fe0-48a5-8ad3-6621ee91749d\") " pod="keystone-kuttl-tests/keystone-bootstrap-p7ld6" Dec 03 13:26:50 crc kubenswrapper[5052]: I1203 13:26:50.567190 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/8b51a88e-2fe0-48a5-8ad3-6621ee91749d-scripts\") pod \"keystone-bootstrap-p7ld6\" (UID: \"8b51a88e-2fe0-48a5-8ad3-6621ee91749d\") " pod="keystone-kuttl-tests/keystone-bootstrap-p7ld6" Dec 03 13:26:50 crc kubenswrapper[5052]: I1203 13:26:50.571450 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/8b51a88e-2fe0-48a5-8ad3-6621ee91749d-scripts\") pod \"keystone-bootstrap-p7ld6\" (UID: \"8b51a88e-2fe0-48a5-8ad3-6621ee91749d\") " pod="keystone-kuttl-tests/keystone-bootstrap-p7ld6" Dec 03 13:26:50 crc kubenswrapper[5052]: I1203 13:26:50.571771 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/8b51a88e-2fe0-48a5-8ad3-6621ee91749d-config-data\") pod \"keystone-bootstrap-p7ld6\" (UID: \"8b51a88e-2fe0-48a5-8ad3-6621ee91749d\") " pod="keystone-kuttl-tests/keystone-bootstrap-p7ld6" Dec 03 13:26:50 crc kubenswrapper[5052]: I1203 13:26:50.572218 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/8b51a88e-2fe0-48a5-8ad3-6621ee91749d-credential-keys\") pod \"keystone-bootstrap-p7ld6\" (UID: \"8b51a88e-2fe0-48a5-8ad3-6621ee91749d\") " pod="keystone-kuttl-tests/keystone-bootstrap-p7ld6" Dec 03 13:26:50 crc kubenswrapper[5052]: I1203 13:26:50.573659 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/8b51a88e-2fe0-48a5-8ad3-6621ee91749d-fernet-keys\") pod \"keystone-bootstrap-p7ld6\" (UID: \"8b51a88e-2fe0-48a5-8ad3-6621ee91749d\") " pod="keystone-kuttl-tests/keystone-bootstrap-p7ld6" Dec 03 13:26:50 crc kubenswrapper[5052]: I1203 13:26:50.589162 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-lnbgc\" (UniqueName: \"kubernetes.io/projected/8b51a88e-2fe0-48a5-8ad3-6621ee91749d-kube-api-access-lnbgc\") pod \"keystone-bootstrap-p7ld6\" (UID: \"8b51a88e-2fe0-48a5-8ad3-6621ee91749d\") " pod="keystone-kuttl-tests/keystone-bootstrap-p7ld6" Dec 03 13:26:50 crc kubenswrapper[5052]: I1203 13:26:50.789015 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="keystone-kuttl-tests/keystone-bootstrap-p7ld6" Dec 03 13:26:51 crc kubenswrapper[5052]: I1203 13:26:51.219389 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["keystone-kuttl-tests/keystone-bootstrap-p7ld6"] Dec 03 13:26:51 crc kubenswrapper[5052]: I1203 13:26:51.324052 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="keystone-kuttl-tests/keystone-bootstrap-p7ld6" event={"ID":"8b51a88e-2fe0-48a5-8ad3-6621ee91749d","Type":"ContainerStarted","Data":"b4253f21e1357e53e4b73e17ba17944413f9b87f6814859000be7364c7604317"} Dec 03 13:26:52 crc kubenswrapper[5052]: I1203 13:26:52.332407 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="keystone-kuttl-tests/keystone-bootstrap-p7ld6" event={"ID":"8b51a88e-2fe0-48a5-8ad3-6621ee91749d","Type":"ContainerStarted","Data":"1c1b136bf371a6589d221ca1ca0b4aceda61a8f8bd70b84409466c8903f6020a"} Dec 03 13:26:55 crc kubenswrapper[5052]: I1203 13:26:55.359321 5052 generic.go:334] "Generic (PLEG): container finished" podID="8b51a88e-2fe0-48a5-8ad3-6621ee91749d" containerID="1c1b136bf371a6589d221ca1ca0b4aceda61a8f8bd70b84409466c8903f6020a" exitCode=0 Dec 03 13:26:55 crc kubenswrapper[5052]: I1203 13:26:55.359489 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="keystone-kuttl-tests/keystone-bootstrap-p7ld6" event={"ID":"8b51a88e-2fe0-48a5-8ad3-6621ee91749d","Type":"ContainerDied","Data":"1c1b136bf371a6589d221ca1ca0b4aceda61a8f8bd70b84409466c8903f6020a"} Dec 03 13:26:56 crc kubenswrapper[5052]: I1203 13:26:56.673426 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="keystone-kuttl-tests/keystone-bootstrap-p7ld6" Dec 03 13:26:56 crc kubenswrapper[5052]: I1203 13:26:56.867175 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/8b51a88e-2fe0-48a5-8ad3-6621ee91749d-fernet-keys\") pod \"8b51a88e-2fe0-48a5-8ad3-6621ee91749d\" (UID: \"8b51a88e-2fe0-48a5-8ad3-6621ee91749d\") " Dec 03 13:26:56 crc kubenswrapper[5052]: I1203 13:26:56.867247 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/8b51a88e-2fe0-48a5-8ad3-6621ee91749d-config-data\") pod \"8b51a88e-2fe0-48a5-8ad3-6621ee91749d\" (UID: \"8b51a88e-2fe0-48a5-8ad3-6621ee91749d\") " Dec 03 13:26:56 crc kubenswrapper[5052]: I1203 13:26:56.867355 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/8b51a88e-2fe0-48a5-8ad3-6621ee91749d-credential-keys\") pod \"8b51a88e-2fe0-48a5-8ad3-6621ee91749d\" (UID: \"8b51a88e-2fe0-48a5-8ad3-6621ee91749d\") " Dec 03 13:26:56 crc kubenswrapper[5052]: I1203 13:26:56.867456 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lnbgc\" (UniqueName: \"kubernetes.io/projected/8b51a88e-2fe0-48a5-8ad3-6621ee91749d-kube-api-access-lnbgc\") pod \"8b51a88e-2fe0-48a5-8ad3-6621ee91749d\" (UID: \"8b51a88e-2fe0-48a5-8ad3-6621ee91749d\") " Dec 03 13:26:56 crc kubenswrapper[5052]: I1203 13:26:56.867491 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/8b51a88e-2fe0-48a5-8ad3-6621ee91749d-scripts\") pod \"8b51a88e-2fe0-48a5-8ad3-6621ee91749d\" (UID: \"8b51a88e-2fe0-48a5-8ad3-6621ee91749d\") " Dec 03 13:26:56 crc kubenswrapper[5052]: I1203 13:26:56.887641 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8b51a88e-2fe0-48a5-8ad3-6621ee91749d-credential-keys" (OuterVolumeSpecName: "credential-keys") pod "8b51a88e-2fe0-48a5-8ad3-6621ee91749d" (UID: "8b51a88e-2fe0-48a5-8ad3-6621ee91749d"). InnerVolumeSpecName "credential-keys". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:26:56 crc kubenswrapper[5052]: I1203 13:26:56.887854 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8b51a88e-2fe0-48a5-8ad3-6621ee91749d-kube-api-access-lnbgc" (OuterVolumeSpecName: "kube-api-access-lnbgc") pod "8b51a88e-2fe0-48a5-8ad3-6621ee91749d" (UID: "8b51a88e-2fe0-48a5-8ad3-6621ee91749d"). InnerVolumeSpecName "kube-api-access-lnbgc". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:26:56 crc kubenswrapper[5052]: I1203 13:26:56.887901 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8b51a88e-2fe0-48a5-8ad3-6621ee91749d-scripts" (OuterVolumeSpecName: "scripts") pod "8b51a88e-2fe0-48a5-8ad3-6621ee91749d" (UID: "8b51a88e-2fe0-48a5-8ad3-6621ee91749d"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:26:56 crc kubenswrapper[5052]: I1203 13:26:56.895503 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8b51a88e-2fe0-48a5-8ad3-6621ee91749d-fernet-keys" (OuterVolumeSpecName: "fernet-keys") pod "8b51a88e-2fe0-48a5-8ad3-6621ee91749d" (UID: "8b51a88e-2fe0-48a5-8ad3-6621ee91749d"). InnerVolumeSpecName "fernet-keys". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:26:56 crc kubenswrapper[5052]: I1203 13:26:56.908399 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8b51a88e-2fe0-48a5-8ad3-6621ee91749d-config-data" (OuterVolumeSpecName: "config-data") pod "8b51a88e-2fe0-48a5-8ad3-6621ee91749d" (UID: "8b51a88e-2fe0-48a5-8ad3-6621ee91749d"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:26:56 crc kubenswrapper[5052]: I1203 13:26:56.969547 5052 reconciler_common.go:293] "Volume detached for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/8b51a88e-2fe0-48a5-8ad3-6621ee91749d-fernet-keys\") on node \"crc\" DevicePath \"\"" Dec 03 13:26:56 crc kubenswrapper[5052]: I1203 13:26:56.969587 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/8b51a88e-2fe0-48a5-8ad3-6621ee91749d-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 13:26:56 crc kubenswrapper[5052]: I1203 13:26:56.969599 5052 reconciler_common.go:293] "Volume detached for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/8b51a88e-2fe0-48a5-8ad3-6621ee91749d-credential-keys\") on node \"crc\" DevicePath \"\"" Dec 03 13:26:56 crc kubenswrapper[5052]: I1203 13:26:56.969613 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-lnbgc\" (UniqueName: \"kubernetes.io/projected/8b51a88e-2fe0-48a5-8ad3-6621ee91749d-kube-api-access-lnbgc\") on node \"crc\" DevicePath \"\"" Dec 03 13:26:56 crc kubenswrapper[5052]: I1203 13:26:56.969623 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/8b51a88e-2fe0-48a5-8ad3-6621ee91749d-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 13:26:57 crc kubenswrapper[5052]: I1203 13:26:57.376520 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="keystone-kuttl-tests/keystone-bootstrap-p7ld6" event={"ID":"8b51a88e-2fe0-48a5-8ad3-6621ee91749d","Type":"ContainerDied","Data":"b4253f21e1357e53e4b73e17ba17944413f9b87f6814859000be7364c7604317"} Dec 03 13:26:57 crc kubenswrapper[5052]: I1203 13:26:57.376882 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b4253f21e1357e53e4b73e17ba17944413f9b87f6814859000be7364c7604317" Dec 03 13:26:57 crc kubenswrapper[5052]: I1203 13:26:57.376594 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="keystone-kuttl-tests/keystone-bootstrap-p7ld6" Dec 03 13:26:57 crc kubenswrapper[5052]: I1203 13:26:57.477948 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["keystone-kuttl-tests/keystone-7b88cf5568-rhltv"] Dec 03 13:26:57 crc kubenswrapper[5052]: E1203 13:26:57.478347 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="8b51a88e-2fe0-48a5-8ad3-6621ee91749d" containerName="keystone-bootstrap" Dec 03 13:26:57 crc kubenswrapper[5052]: I1203 13:26:57.478363 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="8b51a88e-2fe0-48a5-8ad3-6621ee91749d" containerName="keystone-bootstrap" Dec 03 13:26:57 crc kubenswrapper[5052]: I1203 13:26:57.478633 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="8b51a88e-2fe0-48a5-8ad3-6621ee91749d" containerName="keystone-bootstrap" Dec 03 13:26:57 crc kubenswrapper[5052]: I1203 13:26:57.479360 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="keystone-kuttl-tests/keystone-7b88cf5568-rhltv" Dec 03 13:26:57 crc kubenswrapper[5052]: I1203 13:26:57.482620 5052 reflector.go:368] Caches populated for *v1.Secret from object-"keystone-kuttl-tests"/"keystone" Dec 03 13:26:57 crc kubenswrapper[5052]: I1203 13:26:57.482667 5052 reflector.go:368] Caches populated for *v1.Secret from object-"keystone-kuttl-tests"/"keystone-keystone-dockercfg-6nzts" Dec 03 13:26:57 crc kubenswrapper[5052]: I1203 13:26:57.482620 5052 reflector.go:368] Caches populated for *v1.Secret from object-"keystone-kuttl-tests"/"keystone-scripts" Dec 03 13:26:57 crc kubenswrapper[5052]: I1203 13:26:57.482925 5052 reflector.go:368] Caches populated for *v1.Secret from object-"keystone-kuttl-tests"/"keystone-config-data" Dec 03 13:26:57 crc kubenswrapper[5052]: I1203 13:26:57.490490 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["keystone-kuttl-tests/keystone-7b88cf5568-rhltv"] Dec 03 13:26:57 crc kubenswrapper[5052]: I1203 13:26:57.678320 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/d0e65860-f04f-4cbf-9a3a-8c5824e9d26c-config-data\") pod \"keystone-7b88cf5568-rhltv\" (UID: \"d0e65860-f04f-4cbf-9a3a-8c5824e9d26c\") " pod="keystone-kuttl-tests/keystone-7b88cf5568-rhltv" Dec 03 13:26:57 crc kubenswrapper[5052]: I1203 13:26:57.678383 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/d0e65860-f04f-4cbf-9a3a-8c5824e9d26c-credential-keys\") pod \"keystone-7b88cf5568-rhltv\" (UID: \"d0e65860-f04f-4cbf-9a3a-8c5824e9d26c\") " pod="keystone-kuttl-tests/keystone-7b88cf5568-rhltv" Dec 03 13:26:57 crc kubenswrapper[5052]: I1203 13:26:57.678413 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xz9ff\" (UniqueName: \"kubernetes.io/projected/d0e65860-f04f-4cbf-9a3a-8c5824e9d26c-kube-api-access-xz9ff\") pod \"keystone-7b88cf5568-rhltv\" (UID: \"d0e65860-f04f-4cbf-9a3a-8c5824e9d26c\") " pod="keystone-kuttl-tests/keystone-7b88cf5568-rhltv" Dec 03 13:26:57 crc kubenswrapper[5052]: I1203 13:26:57.678441 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/d0e65860-f04f-4cbf-9a3a-8c5824e9d26c-scripts\") pod \"keystone-7b88cf5568-rhltv\" (UID: \"d0e65860-f04f-4cbf-9a3a-8c5824e9d26c\") " pod="keystone-kuttl-tests/keystone-7b88cf5568-rhltv" Dec 03 13:26:57 crc kubenswrapper[5052]: I1203 13:26:57.678489 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/d0e65860-f04f-4cbf-9a3a-8c5824e9d26c-fernet-keys\") pod \"keystone-7b88cf5568-rhltv\" (UID: \"d0e65860-f04f-4cbf-9a3a-8c5824e9d26c\") " pod="keystone-kuttl-tests/keystone-7b88cf5568-rhltv" Dec 03 13:26:57 crc kubenswrapper[5052]: I1203 13:26:57.780158 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/d0e65860-f04f-4cbf-9a3a-8c5824e9d26c-fernet-keys\") pod \"keystone-7b88cf5568-rhltv\" (UID: \"d0e65860-f04f-4cbf-9a3a-8c5824e9d26c\") " pod="keystone-kuttl-tests/keystone-7b88cf5568-rhltv" Dec 03 13:26:57 crc kubenswrapper[5052]: I1203 13:26:57.780767 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/d0e65860-f04f-4cbf-9a3a-8c5824e9d26c-config-data\") pod \"keystone-7b88cf5568-rhltv\" (UID: \"d0e65860-f04f-4cbf-9a3a-8c5824e9d26c\") " pod="keystone-kuttl-tests/keystone-7b88cf5568-rhltv" Dec 03 13:26:57 crc kubenswrapper[5052]: I1203 13:26:57.780837 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/d0e65860-f04f-4cbf-9a3a-8c5824e9d26c-credential-keys\") pod \"keystone-7b88cf5568-rhltv\" (UID: \"d0e65860-f04f-4cbf-9a3a-8c5824e9d26c\") " pod="keystone-kuttl-tests/keystone-7b88cf5568-rhltv" Dec 03 13:26:57 crc kubenswrapper[5052]: I1203 13:26:57.780911 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-xz9ff\" (UniqueName: \"kubernetes.io/projected/d0e65860-f04f-4cbf-9a3a-8c5824e9d26c-kube-api-access-xz9ff\") pod \"keystone-7b88cf5568-rhltv\" (UID: \"d0e65860-f04f-4cbf-9a3a-8c5824e9d26c\") " pod="keystone-kuttl-tests/keystone-7b88cf5568-rhltv" Dec 03 13:26:57 crc kubenswrapper[5052]: I1203 13:26:57.780957 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/d0e65860-f04f-4cbf-9a3a-8c5824e9d26c-scripts\") pod \"keystone-7b88cf5568-rhltv\" (UID: \"d0e65860-f04f-4cbf-9a3a-8c5824e9d26c\") " pod="keystone-kuttl-tests/keystone-7b88cf5568-rhltv" Dec 03 13:26:57 crc kubenswrapper[5052]: I1203 13:26:57.790429 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/d0e65860-f04f-4cbf-9a3a-8c5824e9d26c-scripts\") pod \"keystone-7b88cf5568-rhltv\" (UID: \"d0e65860-f04f-4cbf-9a3a-8c5824e9d26c\") " pod="keystone-kuttl-tests/keystone-7b88cf5568-rhltv" Dec 03 13:26:57 crc kubenswrapper[5052]: I1203 13:26:57.790597 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/d0e65860-f04f-4cbf-9a3a-8c5824e9d26c-credential-keys\") pod \"keystone-7b88cf5568-rhltv\" (UID: \"d0e65860-f04f-4cbf-9a3a-8c5824e9d26c\") " pod="keystone-kuttl-tests/keystone-7b88cf5568-rhltv" Dec 03 13:26:57 crc kubenswrapper[5052]: I1203 13:26:57.790881 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/d0e65860-f04f-4cbf-9a3a-8c5824e9d26c-fernet-keys\") pod \"keystone-7b88cf5568-rhltv\" (UID: \"d0e65860-f04f-4cbf-9a3a-8c5824e9d26c\") " pod="keystone-kuttl-tests/keystone-7b88cf5568-rhltv" Dec 03 13:26:57 crc kubenswrapper[5052]: I1203 13:26:57.802485 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/d0e65860-f04f-4cbf-9a3a-8c5824e9d26c-config-data\") pod \"keystone-7b88cf5568-rhltv\" (UID: \"d0e65860-f04f-4cbf-9a3a-8c5824e9d26c\") " pod="keystone-kuttl-tests/keystone-7b88cf5568-rhltv" Dec 03 13:26:57 crc kubenswrapper[5052]: I1203 13:26:57.807119 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-xz9ff\" (UniqueName: \"kubernetes.io/projected/d0e65860-f04f-4cbf-9a3a-8c5824e9d26c-kube-api-access-xz9ff\") pod \"keystone-7b88cf5568-rhltv\" (UID: \"d0e65860-f04f-4cbf-9a3a-8c5824e9d26c\") " pod="keystone-kuttl-tests/keystone-7b88cf5568-rhltv" Dec 03 13:26:58 crc kubenswrapper[5052]: I1203 13:26:58.105086 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="keystone-kuttl-tests/keystone-7b88cf5568-rhltv" Dec 03 13:26:58 crc kubenswrapper[5052]: I1203 13:26:58.582061 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["keystone-kuttl-tests/keystone-7b88cf5568-rhltv"] Dec 03 13:26:59 crc kubenswrapper[5052]: I1203 13:26:59.397311 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="keystone-kuttl-tests/keystone-7b88cf5568-rhltv" event={"ID":"d0e65860-f04f-4cbf-9a3a-8c5824e9d26c","Type":"ContainerStarted","Data":"9af9672e136fb08963a2a3b3e91f771d58bc7f70c7891f4d2017fd1047c95cd0"} Dec 03 13:26:59 crc kubenswrapper[5052]: I1203 13:26:59.397359 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="keystone-kuttl-tests/keystone-7b88cf5568-rhltv" event={"ID":"d0e65860-f04f-4cbf-9a3a-8c5824e9d26c","Type":"ContainerStarted","Data":"81b8cfd7db9dd8f63c2c3f074e527227c5ae62654b6b8b6d4ff791d33fe89445"} Dec 03 13:26:59 crc kubenswrapper[5052]: I1203 13:26:59.397715 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="keystone-kuttl-tests/keystone-7b88cf5568-rhltv" Dec 03 13:26:59 crc kubenswrapper[5052]: I1203 13:26:59.424923 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="keystone-kuttl-tests/keystone-7b88cf5568-rhltv" podStartSLOduration=2.424890023 podStartE2EDuration="2.424890023s" podCreationTimestamp="2025-12-03 13:26:57 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 13:26:59.423578735 +0000 UTC m=+7610.011820499" watchObservedRunningTime="2025-12-03 13:26:59.424890023 +0000 UTC m=+7610.013131787" Dec 03 13:27:29 crc kubenswrapper[5052]: I1203 13:27:29.778946 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="keystone-kuttl-tests/keystone-7b88cf5568-rhltv" Dec 03 13:27:49 crc kubenswrapper[5052]: I1203 13:27:49.484470 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["keystone-kuttl-tests/keystone-db-sync-xnk4r"] Dec 03 13:27:49 crc kubenswrapper[5052]: I1203 13:27:49.496858 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["keystone-kuttl-tests/keystone-db-sync-xnk4r"] Dec 03 13:27:49 crc kubenswrapper[5052]: I1203 13:27:49.510771 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["keystone-kuttl-tests/keystone-bootstrap-p7ld6"] Dec 03 13:27:49 crc kubenswrapper[5052]: I1203 13:27:49.521276 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["keystone-kuttl-tests/keystone-bootstrap-p7ld6"] Dec 03 13:27:49 crc kubenswrapper[5052]: I1203 13:27:49.532294 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["keystone-kuttl-tests/keystone-7b88cf5568-rhltv"] Dec 03 13:27:49 crc kubenswrapper[5052]: I1203 13:27:49.532744 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="keystone-kuttl-tests/keystone-7b88cf5568-rhltv" podUID="d0e65860-f04f-4cbf-9a3a-8c5824e9d26c" containerName="keystone-api" containerID="cri-o://9af9672e136fb08963a2a3b3e91f771d58bc7f70c7891f4d2017fd1047c95cd0" gracePeriod=30 Dec 03 13:27:49 crc kubenswrapper[5052]: I1203 13:27:49.560221 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["keystone-kuttl-tests/keystone052c-account-delete-9rt6l"] Dec 03 13:27:49 crc kubenswrapper[5052]: I1203 13:27:49.561706 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="keystone-kuttl-tests/keystone052c-account-delete-9rt6l" Dec 03 13:27:49 crc kubenswrapper[5052]: I1203 13:27:49.604478 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["keystone-kuttl-tests/keystone052c-account-delete-9rt6l"] Dec 03 13:27:49 crc kubenswrapper[5052]: I1203 13:27:49.675692 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p4gcd\" (UniqueName: \"kubernetes.io/projected/15e06664-c524-4b1b-87d5-d5a3d10efa9e-kube-api-access-p4gcd\") pod \"keystone052c-account-delete-9rt6l\" (UID: \"15e06664-c524-4b1b-87d5-d5a3d10efa9e\") " pod="keystone-kuttl-tests/keystone052c-account-delete-9rt6l" Dec 03 13:27:49 crc kubenswrapper[5052]: I1203 13:27:49.675801 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/15e06664-c524-4b1b-87d5-d5a3d10efa9e-operator-scripts\") pod \"keystone052c-account-delete-9rt6l\" (UID: \"15e06664-c524-4b1b-87d5-d5a3d10efa9e\") " pod="keystone-kuttl-tests/keystone052c-account-delete-9rt6l" Dec 03 13:27:49 crc kubenswrapper[5052]: I1203 13:27:49.686038 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6bc59279-5d0c-41d6-9d4b-c80b03620cb0" path="/var/lib/kubelet/pods/6bc59279-5d0c-41d6-9d4b-c80b03620cb0/volumes" Dec 03 13:27:49 crc kubenswrapper[5052]: I1203 13:27:49.687026 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8b51a88e-2fe0-48a5-8ad3-6621ee91749d" path="/var/lib/kubelet/pods/8b51a88e-2fe0-48a5-8ad3-6621ee91749d/volumes" Dec 03 13:27:49 crc kubenswrapper[5052]: I1203 13:27:49.777925 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/15e06664-c524-4b1b-87d5-d5a3d10efa9e-operator-scripts\") pod \"keystone052c-account-delete-9rt6l\" (UID: \"15e06664-c524-4b1b-87d5-d5a3d10efa9e\") " pod="keystone-kuttl-tests/keystone052c-account-delete-9rt6l" Dec 03 13:27:49 crc kubenswrapper[5052]: I1203 13:27:49.778894 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-p4gcd\" (UniqueName: \"kubernetes.io/projected/15e06664-c524-4b1b-87d5-d5a3d10efa9e-kube-api-access-p4gcd\") pod \"keystone052c-account-delete-9rt6l\" (UID: \"15e06664-c524-4b1b-87d5-d5a3d10efa9e\") " pod="keystone-kuttl-tests/keystone052c-account-delete-9rt6l" Dec 03 13:27:49 crc kubenswrapper[5052]: I1203 13:27:49.780811 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/15e06664-c524-4b1b-87d5-d5a3d10efa9e-operator-scripts\") pod \"keystone052c-account-delete-9rt6l\" (UID: \"15e06664-c524-4b1b-87d5-d5a3d10efa9e\") " pod="keystone-kuttl-tests/keystone052c-account-delete-9rt6l" Dec 03 13:27:49 crc kubenswrapper[5052]: I1203 13:27:49.813943 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-p4gcd\" (UniqueName: \"kubernetes.io/projected/15e06664-c524-4b1b-87d5-d5a3d10efa9e-kube-api-access-p4gcd\") pod \"keystone052c-account-delete-9rt6l\" (UID: \"15e06664-c524-4b1b-87d5-d5a3d10efa9e\") " pod="keystone-kuttl-tests/keystone052c-account-delete-9rt6l" Dec 03 13:27:49 crc kubenswrapper[5052]: I1203 13:27:49.896018 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="keystone-kuttl-tests/keystone052c-account-delete-9rt6l" Dec 03 13:27:50 crc kubenswrapper[5052]: I1203 13:27:50.394563 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["keystone-kuttl-tests/keystone052c-account-delete-9rt6l"] Dec 03 13:27:50 crc kubenswrapper[5052]: I1203 13:27:50.862084 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="keystone-kuttl-tests/keystone052c-account-delete-9rt6l" event={"ID":"15e06664-c524-4b1b-87d5-d5a3d10efa9e","Type":"ContainerStarted","Data":"2208e063fd9094a87558d34922ec0edd3bcd1611bb58c2eca33573658414fdb0"} Dec 03 13:27:51 crc kubenswrapper[5052]: I1203 13:27:51.873928 5052 generic.go:334] "Generic (PLEG): container finished" podID="15e06664-c524-4b1b-87d5-d5a3d10efa9e" containerID="23495a521d6c51f0ce23bdc259ec38320a9821ccf40b84f6ce800d03dc181084" exitCode=0 Dec 03 13:27:51 crc kubenswrapper[5052]: I1203 13:27:51.874064 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="keystone-kuttl-tests/keystone052c-account-delete-9rt6l" event={"ID":"15e06664-c524-4b1b-87d5-d5a3d10efa9e","Type":"ContainerDied","Data":"23495a521d6c51f0ce23bdc259ec38320a9821ccf40b84f6ce800d03dc181084"} Dec 03 13:27:52 crc kubenswrapper[5052]: I1203 13:27:52.885027 5052 generic.go:334] "Generic (PLEG): container finished" podID="d0e65860-f04f-4cbf-9a3a-8c5824e9d26c" containerID="9af9672e136fb08963a2a3b3e91f771d58bc7f70c7891f4d2017fd1047c95cd0" exitCode=0 Dec 03 13:27:52 crc kubenswrapper[5052]: I1203 13:27:52.885224 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="keystone-kuttl-tests/keystone-7b88cf5568-rhltv" event={"ID":"d0e65860-f04f-4cbf-9a3a-8c5824e9d26c","Type":"ContainerDied","Data":"9af9672e136fb08963a2a3b3e91f771d58bc7f70c7891f4d2017fd1047c95cd0"} Dec 03 13:27:53 crc kubenswrapper[5052]: I1203 13:27:53.172127 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="keystone-kuttl-tests/keystone052c-account-delete-9rt6l" Dec 03 13:27:53 crc kubenswrapper[5052]: I1203 13:27:53.302984 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/15e06664-c524-4b1b-87d5-d5a3d10efa9e-operator-scripts\") pod \"15e06664-c524-4b1b-87d5-d5a3d10efa9e\" (UID: \"15e06664-c524-4b1b-87d5-d5a3d10efa9e\") " Dec 03 13:27:53 crc kubenswrapper[5052]: I1203 13:27:53.303742 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p4gcd\" (UniqueName: \"kubernetes.io/projected/15e06664-c524-4b1b-87d5-d5a3d10efa9e-kube-api-access-p4gcd\") pod \"15e06664-c524-4b1b-87d5-d5a3d10efa9e\" (UID: \"15e06664-c524-4b1b-87d5-d5a3d10efa9e\") " Dec 03 13:27:53 crc kubenswrapper[5052]: I1203 13:27:53.305460 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/15e06664-c524-4b1b-87d5-d5a3d10efa9e-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "15e06664-c524-4b1b-87d5-d5a3d10efa9e" (UID: "15e06664-c524-4b1b-87d5-d5a3d10efa9e"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:27:53 crc kubenswrapper[5052]: I1203 13:27:53.310506 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/15e06664-c524-4b1b-87d5-d5a3d10efa9e-kube-api-access-p4gcd" (OuterVolumeSpecName: "kube-api-access-p4gcd") pod "15e06664-c524-4b1b-87d5-d5a3d10efa9e" (UID: "15e06664-c524-4b1b-87d5-d5a3d10efa9e"). InnerVolumeSpecName "kube-api-access-p4gcd". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:27:53 crc kubenswrapper[5052]: I1203 13:27:53.407163 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-p4gcd\" (UniqueName: \"kubernetes.io/projected/15e06664-c524-4b1b-87d5-d5a3d10efa9e-kube-api-access-p4gcd\") on node \"crc\" DevicePath \"\"" Dec 03 13:27:53 crc kubenswrapper[5052]: I1203 13:27:53.407583 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/15e06664-c524-4b1b-87d5-d5a3d10efa9e-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 13:27:53 crc kubenswrapper[5052]: I1203 13:27:53.543771 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="keystone-kuttl-tests/keystone-7b88cf5568-rhltv" Dec 03 13:27:53 crc kubenswrapper[5052]: I1203 13:27:53.610418 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/d0e65860-f04f-4cbf-9a3a-8c5824e9d26c-scripts\") pod \"d0e65860-f04f-4cbf-9a3a-8c5824e9d26c\" (UID: \"d0e65860-f04f-4cbf-9a3a-8c5824e9d26c\") " Dec 03 13:27:53 crc kubenswrapper[5052]: I1203 13:27:53.610478 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/d0e65860-f04f-4cbf-9a3a-8c5824e9d26c-config-data\") pod \"d0e65860-f04f-4cbf-9a3a-8c5824e9d26c\" (UID: \"d0e65860-f04f-4cbf-9a3a-8c5824e9d26c\") " Dec 03 13:27:53 crc kubenswrapper[5052]: I1203 13:27:53.610538 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/d0e65860-f04f-4cbf-9a3a-8c5824e9d26c-fernet-keys\") pod \"d0e65860-f04f-4cbf-9a3a-8c5824e9d26c\" (UID: \"d0e65860-f04f-4cbf-9a3a-8c5824e9d26c\") " Dec 03 13:27:53 crc kubenswrapper[5052]: I1203 13:27:53.610597 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xz9ff\" (UniqueName: \"kubernetes.io/projected/d0e65860-f04f-4cbf-9a3a-8c5824e9d26c-kube-api-access-xz9ff\") pod \"d0e65860-f04f-4cbf-9a3a-8c5824e9d26c\" (UID: \"d0e65860-f04f-4cbf-9a3a-8c5824e9d26c\") " Dec 03 13:27:53 crc kubenswrapper[5052]: I1203 13:27:53.610663 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/d0e65860-f04f-4cbf-9a3a-8c5824e9d26c-credential-keys\") pod \"d0e65860-f04f-4cbf-9a3a-8c5824e9d26c\" (UID: \"d0e65860-f04f-4cbf-9a3a-8c5824e9d26c\") " Dec 03 13:27:53 crc kubenswrapper[5052]: I1203 13:27:53.615723 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d0e65860-f04f-4cbf-9a3a-8c5824e9d26c-credential-keys" (OuterVolumeSpecName: "credential-keys") pod "d0e65860-f04f-4cbf-9a3a-8c5824e9d26c" (UID: "d0e65860-f04f-4cbf-9a3a-8c5824e9d26c"). InnerVolumeSpecName "credential-keys". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:27:53 crc kubenswrapper[5052]: I1203 13:27:53.615820 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d0e65860-f04f-4cbf-9a3a-8c5824e9d26c-kube-api-access-xz9ff" (OuterVolumeSpecName: "kube-api-access-xz9ff") pod "d0e65860-f04f-4cbf-9a3a-8c5824e9d26c" (UID: "d0e65860-f04f-4cbf-9a3a-8c5824e9d26c"). InnerVolumeSpecName "kube-api-access-xz9ff". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:27:53 crc kubenswrapper[5052]: I1203 13:27:53.616681 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d0e65860-f04f-4cbf-9a3a-8c5824e9d26c-scripts" (OuterVolumeSpecName: "scripts") pod "d0e65860-f04f-4cbf-9a3a-8c5824e9d26c" (UID: "d0e65860-f04f-4cbf-9a3a-8c5824e9d26c"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:27:53 crc kubenswrapper[5052]: I1203 13:27:53.616710 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d0e65860-f04f-4cbf-9a3a-8c5824e9d26c-fernet-keys" (OuterVolumeSpecName: "fernet-keys") pod "d0e65860-f04f-4cbf-9a3a-8c5824e9d26c" (UID: "d0e65860-f04f-4cbf-9a3a-8c5824e9d26c"). InnerVolumeSpecName "fernet-keys". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:27:53 crc kubenswrapper[5052]: I1203 13:27:53.633386 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d0e65860-f04f-4cbf-9a3a-8c5824e9d26c-config-data" (OuterVolumeSpecName: "config-data") pod "d0e65860-f04f-4cbf-9a3a-8c5824e9d26c" (UID: "d0e65860-f04f-4cbf-9a3a-8c5824e9d26c"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:27:53 crc kubenswrapper[5052]: I1203 13:27:53.713606 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/d0e65860-f04f-4cbf-9a3a-8c5824e9d26c-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 13:27:53 crc kubenswrapper[5052]: I1203 13:27:53.713638 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/d0e65860-f04f-4cbf-9a3a-8c5824e9d26c-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 13:27:53 crc kubenswrapper[5052]: I1203 13:27:53.713647 5052 reconciler_common.go:293] "Volume detached for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/d0e65860-f04f-4cbf-9a3a-8c5824e9d26c-fernet-keys\") on node \"crc\" DevicePath \"\"" Dec 03 13:27:53 crc kubenswrapper[5052]: I1203 13:27:53.713657 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-xz9ff\" (UniqueName: \"kubernetes.io/projected/d0e65860-f04f-4cbf-9a3a-8c5824e9d26c-kube-api-access-xz9ff\") on node \"crc\" DevicePath \"\"" Dec 03 13:27:53 crc kubenswrapper[5052]: I1203 13:27:53.713667 5052 reconciler_common.go:293] "Volume detached for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/d0e65860-f04f-4cbf-9a3a-8c5824e9d26c-credential-keys\") on node \"crc\" DevicePath \"\"" Dec 03 13:27:53 crc kubenswrapper[5052]: I1203 13:27:53.894913 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="keystone-kuttl-tests/keystone052c-account-delete-9rt6l" Dec 03 13:27:53 crc kubenswrapper[5052]: I1203 13:27:53.895601 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="keystone-kuttl-tests/keystone052c-account-delete-9rt6l" event={"ID":"15e06664-c524-4b1b-87d5-d5a3d10efa9e","Type":"ContainerDied","Data":"2208e063fd9094a87558d34922ec0edd3bcd1611bb58c2eca33573658414fdb0"} Dec 03 13:27:53 crc kubenswrapper[5052]: I1203 13:27:53.895651 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2208e063fd9094a87558d34922ec0edd3bcd1611bb58c2eca33573658414fdb0" Dec 03 13:27:53 crc kubenswrapper[5052]: I1203 13:27:53.903501 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="keystone-kuttl-tests/keystone-7b88cf5568-rhltv" event={"ID":"d0e65860-f04f-4cbf-9a3a-8c5824e9d26c","Type":"ContainerDied","Data":"81b8cfd7db9dd8f63c2c3f074e527227c5ae62654b6b8b6d4ff791d33fe89445"} Dec 03 13:27:53 crc kubenswrapper[5052]: I1203 13:27:53.903615 5052 scope.go:117] "RemoveContainer" containerID="9af9672e136fb08963a2a3b3e91f771d58bc7f70c7891f4d2017fd1047c95cd0" Dec 03 13:27:53 crc kubenswrapper[5052]: I1203 13:27:53.903997 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="keystone-kuttl-tests/keystone-7b88cf5568-rhltv" Dec 03 13:27:53 crc kubenswrapper[5052]: I1203 13:27:53.927492 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["keystone-kuttl-tests/keystone-7b88cf5568-rhltv"] Dec 03 13:27:53 crc kubenswrapper[5052]: I1203 13:27:53.937116 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["keystone-kuttl-tests/keystone-7b88cf5568-rhltv"] Dec 03 13:27:54 crc kubenswrapper[5052]: I1203 13:27:54.573267 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["keystone-kuttl-tests/keystone-db-create-g5klp"] Dec 03 13:27:54 crc kubenswrapper[5052]: I1203 13:27:54.583242 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["keystone-kuttl-tests/keystone-db-create-g5klp"] Dec 03 13:27:54 crc kubenswrapper[5052]: I1203 13:27:54.598057 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["keystone-kuttl-tests/keystone-052c-account-create-update-rjc4m"] Dec 03 13:27:54 crc kubenswrapper[5052]: I1203 13:27:54.609794 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["keystone-kuttl-tests/keystone052c-account-delete-9rt6l"] Dec 03 13:27:54 crc kubenswrapper[5052]: I1203 13:27:54.615613 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["keystone-kuttl-tests/keystone-052c-account-create-update-rjc4m"] Dec 03 13:27:54 crc kubenswrapper[5052]: I1203 13:27:54.641461 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["keystone-kuttl-tests/keystone052c-account-delete-9rt6l"] Dec 03 13:27:54 crc kubenswrapper[5052]: I1203 13:27:54.868864 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["keystone-kuttl-tests/keystone-db-create-q78p4"] Dec 03 13:27:54 crc kubenswrapper[5052]: E1203 13:27:54.869195 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="d0e65860-f04f-4cbf-9a3a-8c5824e9d26c" containerName="keystone-api" Dec 03 13:27:54 crc kubenswrapper[5052]: I1203 13:27:54.869214 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="d0e65860-f04f-4cbf-9a3a-8c5824e9d26c" containerName="keystone-api" Dec 03 13:27:54 crc kubenswrapper[5052]: E1203 13:27:54.869238 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="15e06664-c524-4b1b-87d5-d5a3d10efa9e" containerName="mariadb-account-delete" Dec 03 13:27:54 crc kubenswrapper[5052]: I1203 13:27:54.869246 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="15e06664-c524-4b1b-87d5-d5a3d10efa9e" containerName="mariadb-account-delete" Dec 03 13:27:54 crc kubenswrapper[5052]: I1203 13:27:54.869423 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="d0e65860-f04f-4cbf-9a3a-8c5824e9d26c" containerName="keystone-api" Dec 03 13:27:54 crc kubenswrapper[5052]: I1203 13:27:54.869444 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="15e06664-c524-4b1b-87d5-d5a3d10efa9e" containerName="mariadb-account-delete" Dec 03 13:27:54 crc kubenswrapper[5052]: I1203 13:27:54.870150 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="keystone-kuttl-tests/keystone-db-create-q78p4" Dec 03 13:27:54 crc kubenswrapper[5052]: I1203 13:27:54.876956 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["keystone-kuttl-tests/keystone-db-create-q78p4"] Dec 03 13:27:54 crc kubenswrapper[5052]: I1203 13:27:54.886353 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["keystone-kuttl-tests/keystone-4828-account-create-update-wn9s4"] Dec 03 13:27:54 crc kubenswrapper[5052]: I1203 13:27:54.887577 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="keystone-kuttl-tests/keystone-4828-account-create-update-wn9s4" Dec 03 13:27:54 crc kubenswrapper[5052]: I1203 13:27:54.893603 5052 reflector.go:368] Caches populated for *v1.Secret from object-"keystone-kuttl-tests"/"keystone-db-secret" Dec 03 13:27:54 crc kubenswrapper[5052]: I1203 13:27:54.907433 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["keystone-kuttl-tests/keystone-4828-account-create-update-wn9s4"] Dec 03 13:27:55 crc kubenswrapper[5052]: I1203 13:27:55.037083 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/a4ff3963-8b4c-4364-9aaf-3dfdc8434921-operator-scripts\") pod \"keystone-db-create-q78p4\" (UID: \"a4ff3963-8b4c-4364-9aaf-3dfdc8434921\") " pod="keystone-kuttl-tests/keystone-db-create-q78p4" Dec 03 13:27:55 crc kubenswrapper[5052]: I1203 13:27:55.037165 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ttz76\" (UniqueName: \"kubernetes.io/projected/6bc0cf1a-1224-48d1-b280-b6d0d612caa2-kube-api-access-ttz76\") pod \"keystone-4828-account-create-update-wn9s4\" (UID: \"6bc0cf1a-1224-48d1-b280-b6d0d612caa2\") " pod="keystone-kuttl-tests/keystone-4828-account-create-update-wn9s4" Dec 03 13:27:55 crc kubenswrapper[5052]: I1203 13:27:55.037212 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/6bc0cf1a-1224-48d1-b280-b6d0d612caa2-operator-scripts\") pod \"keystone-4828-account-create-update-wn9s4\" (UID: \"6bc0cf1a-1224-48d1-b280-b6d0d612caa2\") " pod="keystone-kuttl-tests/keystone-4828-account-create-update-wn9s4" Dec 03 13:27:55 crc kubenswrapper[5052]: I1203 13:27:55.037241 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gs9tk\" (UniqueName: \"kubernetes.io/projected/a4ff3963-8b4c-4364-9aaf-3dfdc8434921-kube-api-access-gs9tk\") pod \"keystone-db-create-q78p4\" (UID: \"a4ff3963-8b4c-4364-9aaf-3dfdc8434921\") " pod="keystone-kuttl-tests/keystone-db-create-q78p4" Dec 03 13:27:55 crc kubenswrapper[5052]: I1203 13:27:55.138492 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/a4ff3963-8b4c-4364-9aaf-3dfdc8434921-operator-scripts\") pod \"keystone-db-create-q78p4\" (UID: \"a4ff3963-8b4c-4364-9aaf-3dfdc8434921\") " pod="keystone-kuttl-tests/keystone-db-create-q78p4" Dec 03 13:27:55 crc kubenswrapper[5052]: I1203 13:27:55.138653 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-ttz76\" (UniqueName: \"kubernetes.io/projected/6bc0cf1a-1224-48d1-b280-b6d0d612caa2-kube-api-access-ttz76\") pod \"keystone-4828-account-create-update-wn9s4\" (UID: \"6bc0cf1a-1224-48d1-b280-b6d0d612caa2\") " pod="keystone-kuttl-tests/keystone-4828-account-create-update-wn9s4" Dec 03 13:27:55 crc kubenswrapper[5052]: I1203 13:27:55.138773 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/6bc0cf1a-1224-48d1-b280-b6d0d612caa2-operator-scripts\") pod \"keystone-4828-account-create-update-wn9s4\" (UID: \"6bc0cf1a-1224-48d1-b280-b6d0d612caa2\") " pod="keystone-kuttl-tests/keystone-4828-account-create-update-wn9s4" Dec 03 13:27:55 crc kubenswrapper[5052]: I1203 13:27:55.138822 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-gs9tk\" (UniqueName: \"kubernetes.io/projected/a4ff3963-8b4c-4364-9aaf-3dfdc8434921-kube-api-access-gs9tk\") pod \"keystone-db-create-q78p4\" (UID: \"a4ff3963-8b4c-4364-9aaf-3dfdc8434921\") " pod="keystone-kuttl-tests/keystone-db-create-q78p4" Dec 03 13:27:55 crc kubenswrapper[5052]: I1203 13:27:55.139616 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/a4ff3963-8b4c-4364-9aaf-3dfdc8434921-operator-scripts\") pod \"keystone-db-create-q78p4\" (UID: \"a4ff3963-8b4c-4364-9aaf-3dfdc8434921\") " pod="keystone-kuttl-tests/keystone-db-create-q78p4" Dec 03 13:27:55 crc kubenswrapper[5052]: I1203 13:27:55.139864 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/6bc0cf1a-1224-48d1-b280-b6d0d612caa2-operator-scripts\") pod \"keystone-4828-account-create-update-wn9s4\" (UID: \"6bc0cf1a-1224-48d1-b280-b6d0d612caa2\") " pod="keystone-kuttl-tests/keystone-4828-account-create-update-wn9s4" Dec 03 13:27:55 crc kubenswrapper[5052]: I1203 13:27:55.159035 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-ttz76\" (UniqueName: \"kubernetes.io/projected/6bc0cf1a-1224-48d1-b280-b6d0d612caa2-kube-api-access-ttz76\") pod \"keystone-4828-account-create-update-wn9s4\" (UID: \"6bc0cf1a-1224-48d1-b280-b6d0d612caa2\") " pod="keystone-kuttl-tests/keystone-4828-account-create-update-wn9s4" Dec 03 13:27:55 crc kubenswrapper[5052]: I1203 13:27:55.161777 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-gs9tk\" (UniqueName: \"kubernetes.io/projected/a4ff3963-8b4c-4364-9aaf-3dfdc8434921-kube-api-access-gs9tk\") pod \"keystone-db-create-q78p4\" (UID: \"a4ff3963-8b4c-4364-9aaf-3dfdc8434921\") " pod="keystone-kuttl-tests/keystone-db-create-q78p4" Dec 03 13:27:55 crc kubenswrapper[5052]: I1203 13:27:55.194271 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="keystone-kuttl-tests/keystone-db-create-q78p4" Dec 03 13:27:55 crc kubenswrapper[5052]: I1203 13:27:55.225034 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="keystone-kuttl-tests/keystone-4828-account-create-update-wn9s4" Dec 03 13:27:55 crc kubenswrapper[5052]: I1203 13:27:55.526462 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["keystone-kuttl-tests/keystone-db-create-q78p4"] Dec 03 13:27:55 crc kubenswrapper[5052]: I1203 13:27:55.574871 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["keystone-kuttl-tests/keystone-4828-account-create-update-wn9s4"] Dec 03 13:27:55 crc kubenswrapper[5052]: I1203 13:27:55.684196 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="15e06664-c524-4b1b-87d5-d5a3d10efa9e" path="/var/lib/kubelet/pods/15e06664-c524-4b1b-87d5-d5a3d10efa9e/volumes" Dec 03 13:27:55 crc kubenswrapper[5052]: I1203 13:27:55.685172 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="90a0c638-12e3-40ed-9926-751d20df9208" path="/var/lib/kubelet/pods/90a0c638-12e3-40ed-9926-751d20df9208/volumes" Dec 03 13:27:55 crc kubenswrapper[5052]: I1203 13:27:55.685863 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d0e65860-f04f-4cbf-9a3a-8c5824e9d26c" path="/var/lib/kubelet/pods/d0e65860-f04f-4cbf-9a3a-8c5824e9d26c/volumes" Dec 03 13:27:55 crc kubenswrapper[5052]: I1203 13:27:55.687384 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d734fba5-735c-4a3a-828a-8a205069e17f" path="/var/lib/kubelet/pods/d734fba5-735c-4a3a-828a-8a205069e17f/volumes" Dec 03 13:27:55 crc kubenswrapper[5052]: I1203 13:27:55.931300 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="keystone-kuttl-tests/keystone-4828-account-create-update-wn9s4" event={"ID":"6bc0cf1a-1224-48d1-b280-b6d0d612caa2","Type":"ContainerStarted","Data":"bf140143123c5c502f6c9cbf40b8b2a7bf5df0f0440f316c7f197f181d943498"} Dec 03 13:27:55 crc kubenswrapper[5052]: I1203 13:27:55.931387 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="keystone-kuttl-tests/keystone-4828-account-create-update-wn9s4" event={"ID":"6bc0cf1a-1224-48d1-b280-b6d0d612caa2","Type":"ContainerStarted","Data":"f8de0eaf85f484b2f56e30b54421eb6c4bacc840c552edbf4a803f556e998f4e"} Dec 03 13:27:55 crc kubenswrapper[5052]: I1203 13:27:55.938306 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="keystone-kuttl-tests/keystone-db-create-q78p4" event={"ID":"a4ff3963-8b4c-4364-9aaf-3dfdc8434921","Type":"ContainerStarted","Data":"486ef892c9c6611d7d60b6d2028a0cf4f6a796e746a75ced89976f69fbf64ec0"} Dec 03 13:27:55 crc kubenswrapper[5052]: I1203 13:27:55.938372 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="keystone-kuttl-tests/keystone-db-create-q78p4" event={"ID":"a4ff3963-8b4c-4364-9aaf-3dfdc8434921","Type":"ContainerStarted","Data":"06e6cbe18a33b47ed1d63e0bb9cedf6847ce303db4275e6b45661640a93ffe55"} Dec 03 13:27:55 crc kubenswrapper[5052]: I1203 13:27:55.959909 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="keystone-kuttl-tests/keystone-4828-account-create-update-wn9s4" podStartSLOduration=1.9598853470000002 podStartE2EDuration="1.959885347s" podCreationTimestamp="2025-12-03 13:27:54 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 13:27:55.952258248 +0000 UTC m=+7666.540500012" watchObservedRunningTime="2025-12-03 13:27:55.959885347 +0000 UTC m=+7666.548127111" Dec 03 13:27:55 crc kubenswrapper[5052]: I1203 13:27:55.974758 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="keystone-kuttl-tests/keystone-db-create-q78p4" podStartSLOduration=1.974736272 podStartE2EDuration="1.974736272s" podCreationTimestamp="2025-12-03 13:27:54 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 13:27:55.972680154 +0000 UTC m=+7666.560921918" watchObservedRunningTime="2025-12-03 13:27:55.974736272 +0000 UTC m=+7666.562978046" Dec 03 13:27:57 crc kubenswrapper[5052]: I1203 13:27:57.955393 5052 generic.go:334] "Generic (PLEG): container finished" podID="6bc0cf1a-1224-48d1-b280-b6d0d612caa2" containerID="bf140143123c5c502f6c9cbf40b8b2a7bf5df0f0440f316c7f197f181d943498" exitCode=0 Dec 03 13:27:57 crc kubenswrapper[5052]: I1203 13:27:57.955541 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="keystone-kuttl-tests/keystone-4828-account-create-update-wn9s4" event={"ID":"6bc0cf1a-1224-48d1-b280-b6d0d612caa2","Type":"ContainerDied","Data":"bf140143123c5c502f6c9cbf40b8b2a7bf5df0f0440f316c7f197f181d943498"} Dec 03 13:27:57 crc kubenswrapper[5052]: I1203 13:27:57.958273 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="keystone-kuttl-tests/keystone-db-create-q78p4" event={"ID":"a4ff3963-8b4c-4364-9aaf-3dfdc8434921","Type":"ContainerDied","Data":"486ef892c9c6611d7d60b6d2028a0cf4f6a796e746a75ced89976f69fbf64ec0"} Dec 03 13:27:57 crc kubenswrapper[5052]: I1203 13:27:57.958172 5052 generic.go:334] "Generic (PLEG): container finished" podID="a4ff3963-8b4c-4364-9aaf-3dfdc8434921" containerID="486ef892c9c6611d7d60b6d2028a0cf4f6a796e746a75ced89976f69fbf64ec0" exitCode=0 Dec 03 13:27:59 crc kubenswrapper[5052]: I1203 13:27:59.373861 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="keystone-kuttl-tests/keystone-4828-account-create-update-wn9s4" Dec 03 13:27:59 crc kubenswrapper[5052]: I1203 13:27:59.380740 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="keystone-kuttl-tests/keystone-db-create-q78p4" Dec 03 13:27:59 crc kubenswrapper[5052]: I1203 13:27:59.425197 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/a4ff3963-8b4c-4364-9aaf-3dfdc8434921-operator-scripts\") pod \"a4ff3963-8b4c-4364-9aaf-3dfdc8434921\" (UID: \"a4ff3963-8b4c-4364-9aaf-3dfdc8434921\") " Dec 03 13:27:59 crc kubenswrapper[5052]: I1203 13:27:59.425292 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gs9tk\" (UniqueName: \"kubernetes.io/projected/a4ff3963-8b4c-4364-9aaf-3dfdc8434921-kube-api-access-gs9tk\") pod \"a4ff3963-8b4c-4364-9aaf-3dfdc8434921\" (UID: \"a4ff3963-8b4c-4364-9aaf-3dfdc8434921\") " Dec 03 13:27:59 crc kubenswrapper[5052]: I1203 13:27:59.425360 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/6bc0cf1a-1224-48d1-b280-b6d0d612caa2-operator-scripts\") pod \"6bc0cf1a-1224-48d1-b280-b6d0d612caa2\" (UID: \"6bc0cf1a-1224-48d1-b280-b6d0d612caa2\") " Dec 03 13:27:59 crc kubenswrapper[5052]: I1203 13:27:59.425389 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ttz76\" (UniqueName: \"kubernetes.io/projected/6bc0cf1a-1224-48d1-b280-b6d0d612caa2-kube-api-access-ttz76\") pod \"6bc0cf1a-1224-48d1-b280-b6d0d612caa2\" (UID: \"6bc0cf1a-1224-48d1-b280-b6d0d612caa2\") " Dec 03 13:27:59 crc kubenswrapper[5052]: I1203 13:27:59.426101 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a4ff3963-8b4c-4364-9aaf-3dfdc8434921-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "a4ff3963-8b4c-4364-9aaf-3dfdc8434921" (UID: "a4ff3963-8b4c-4364-9aaf-3dfdc8434921"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:27:59 crc kubenswrapper[5052]: I1203 13:27:59.426260 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6bc0cf1a-1224-48d1-b280-b6d0d612caa2-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "6bc0cf1a-1224-48d1-b280-b6d0d612caa2" (UID: "6bc0cf1a-1224-48d1-b280-b6d0d612caa2"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:27:59 crc kubenswrapper[5052]: I1203 13:27:59.431054 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a4ff3963-8b4c-4364-9aaf-3dfdc8434921-kube-api-access-gs9tk" (OuterVolumeSpecName: "kube-api-access-gs9tk") pod "a4ff3963-8b4c-4364-9aaf-3dfdc8434921" (UID: "a4ff3963-8b4c-4364-9aaf-3dfdc8434921"). InnerVolumeSpecName "kube-api-access-gs9tk". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:27:59 crc kubenswrapper[5052]: I1203 13:27:59.437756 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6bc0cf1a-1224-48d1-b280-b6d0d612caa2-kube-api-access-ttz76" (OuterVolumeSpecName: "kube-api-access-ttz76") pod "6bc0cf1a-1224-48d1-b280-b6d0d612caa2" (UID: "6bc0cf1a-1224-48d1-b280-b6d0d612caa2"). InnerVolumeSpecName "kube-api-access-ttz76". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:27:59 crc kubenswrapper[5052]: I1203 13:27:59.527649 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/a4ff3963-8b4c-4364-9aaf-3dfdc8434921-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 13:27:59 crc kubenswrapper[5052]: I1203 13:27:59.527709 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-gs9tk\" (UniqueName: \"kubernetes.io/projected/a4ff3963-8b4c-4364-9aaf-3dfdc8434921-kube-api-access-gs9tk\") on node \"crc\" DevicePath \"\"" Dec 03 13:27:59 crc kubenswrapper[5052]: I1203 13:27:59.527722 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/6bc0cf1a-1224-48d1-b280-b6d0d612caa2-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 13:27:59 crc kubenswrapper[5052]: I1203 13:27:59.527732 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-ttz76\" (UniqueName: \"kubernetes.io/projected/6bc0cf1a-1224-48d1-b280-b6d0d612caa2-kube-api-access-ttz76\") on node \"crc\" DevicePath \"\"" Dec 03 13:27:59 crc kubenswrapper[5052]: I1203 13:27:59.978669 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="keystone-kuttl-tests/keystone-4828-account-create-update-wn9s4" Dec 03 13:27:59 crc kubenswrapper[5052]: I1203 13:27:59.978701 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="keystone-kuttl-tests/keystone-4828-account-create-update-wn9s4" event={"ID":"6bc0cf1a-1224-48d1-b280-b6d0d612caa2","Type":"ContainerDied","Data":"f8de0eaf85f484b2f56e30b54421eb6c4bacc840c552edbf4a803f556e998f4e"} Dec 03 13:27:59 crc kubenswrapper[5052]: I1203 13:27:59.978757 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f8de0eaf85f484b2f56e30b54421eb6c4bacc840c552edbf4a803f556e998f4e" Dec 03 13:27:59 crc kubenswrapper[5052]: I1203 13:27:59.982938 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="keystone-kuttl-tests/keystone-db-create-q78p4" event={"ID":"a4ff3963-8b4c-4364-9aaf-3dfdc8434921","Type":"ContainerDied","Data":"06e6cbe18a33b47ed1d63e0bb9cedf6847ce303db4275e6b45661640a93ffe55"} Dec 03 13:27:59 crc kubenswrapper[5052]: I1203 13:27:59.983011 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="06e6cbe18a33b47ed1d63e0bb9cedf6847ce303db4275e6b45661640a93ffe55" Dec 03 13:27:59 crc kubenswrapper[5052]: I1203 13:27:59.983073 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="keystone-kuttl-tests/keystone-db-create-q78p4" Dec 03 13:28:05 crc kubenswrapper[5052]: I1203 13:28:05.464897 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["keystone-kuttl-tests/keystone-db-sync-8tgzj"] Dec 03 13:28:05 crc kubenswrapper[5052]: E1203 13:28:05.468216 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="a4ff3963-8b4c-4364-9aaf-3dfdc8434921" containerName="mariadb-database-create" Dec 03 13:28:05 crc kubenswrapper[5052]: I1203 13:28:05.468325 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="a4ff3963-8b4c-4364-9aaf-3dfdc8434921" containerName="mariadb-database-create" Dec 03 13:28:05 crc kubenswrapper[5052]: E1203 13:28:05.468424 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="6bc0cf1a-1224-48d1-b280-b6d0d612caa2" containerName="mariadb-account-create-update" Dec 03 13:28:05 crc kubenswrapper[5052]: I1203 13:28:05.468479 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="6bc0cf1a-1224-48d1-b280-b6d0d612caa2" containerName="mariadb-account-create-update" Dec 03 13:28:05 crc kubenswrapper[5052]: I1203 13:28:05.468798 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="6bc0cf1a-1224-48d1-b280-b6d0d612caa2" containerName="mariadb-account-create-update" Dec 03 13:28:05 crc kubenswrapper[5052]: I1203 13:28:05.468911 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="a4ff3963-8b4c-4364-9aaf-3dfdc8434921" containerName="mariadb-database-create" Dec 03 13:28:05 crc kubenswrapper[5052]: I1203 13:28:05.469739 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="keystone-kuttl-tests/keystone-db-sync-8tgzj" Dec 03 13:28:05 crc kubenswrapper[5052]: I1203 13:28:05.472011 5052 reflector.go:368] Caches populated for *v1.Secret from object-"keystone-kuttl-tests"/"keystone-keystone-dockercfg-h96kl" Dec 03 13:28:05 crc kubenswrapper[5052]: I1203 13:28:05.472651 5052 reflector.go:368] Caches populated for *v1.Secret from object-"keystone-kuttl-tests"/"keystone-config-data" Dec 03 13:28:05 crc kubenswrapper[5052]: I1203 13:28:05.473058 5052 reflector.go:368] Caches populated for *v1.Secret from object-"keystone-kuttl-tests"/"keystone" Dec 03 13:28:05 crc kubenswrapper[5052]: I1203 13:28:05.473427 5052 reflector.go:368] Caches populated for *v1.Secret from object-"keystone-kuttl-tests"/"keystone-scripts" Dec 03 13:28:05 crc kubenswrapper[5052]: I1203 13:28:05.480886 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["keystone-kuttl-tests/keystone-db-sync-8tgzj"] Dec 03 13:28:05 crc kubenswrapper[5052]: I1203 13:28:05.533352 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/420a4d60-e580-4289-9a33-527b9203fca1-config-data\") pod \"keystone-db-sync-8tgzj\" (UID: \"420a4d60-e580-4289-9a33-527b9203fca1\") " pod="keystone-kuttl-tests/keystone-db-sync-8tgzj" Dec 03 13:28:05 crc kubenswrapper[5052]: I1203 13:28:05.533681 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mfztk\" (UniqueName: \"kubernetes.io/projected/420a4d60-e580-4289-9a33-527b9203fca1-kube-api-access-mfztk\") pod \"keystone-db-sync-8tgzj\" (UID: \"420a4d60-e580-4289-9a33-527b9203fca1\") " pod="keystone-kuttl-tests/keystone-db-sync-8tgzj" Dec 03 13:28:05 crc kubenswrapper[5052]: I1203 13:28:05.636062 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/420a4d60-e580-4289-9a33-527b9203fca1-config-data\") pod \"keystone-db-sync-8tgzj\" (UID: \"420a4d60-e580-4289-9a33-527b9203fca1\") " pod="keystone-kuttl-tests/keystone-db-sync-8tgzj" Dec 03 13:28:05 crc kubenswrapper[5052]: I1203 13:28:05.636156 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-mfztk\" (UniqueName: \"kubernetes.io/projected/420a4d60-e580-4289-9a33-527b9203fca1-kube-api-access-mfztk\") pod \"keystone-db-sync-8tgzj\" (UID: \"420a4d60-e580-4289-9a33-527b9203fca1\") " pod="keystone-kuttl-tests/keystone-db-sync-8tgzj" Dec 03 13:28:05 crc kubenswrapper[5052]: I1203 13:28:05.645343 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/420a4d60-e580-4289-9a33-527b9203fca1-config-data\") pod \"keystone-db-sync-8tgzj\" (UID: \"420a4d60-e580-4289-9a33-527b9203fca1\") " pod="keystone-kuttl-tests/keystone-db-sync-8tgzj" Dec 03 13:28:05 crc kubenswrapper[5052]: I1203 13:28:05.659213 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-mfztk\" (UniqueName: \"kubernetes.io/projected/420a4d60-e580-4289-9a33-527b9203fca1-kube-api-access-mfztk\") pod \"keystone-db-sync-8tgzj\" (UID: \"420a4d60-e580-4289-9a33-527b9203fca1\") " pod="keystone-kuttl-tests/keystone-db-sync-8tgzj" Dec 03 13:28:05 crc kubenswrapper[5052]: I1203 13:28:05.796650 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="keystone-kuttl-tests/keystone-db-sync-8tgzj" Dec 03 13:28:06 crc kubenswrapper[5052]: I1203 13:28:06.260284 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["keystone-kuttl-tests/keystone-db-sync-8tgzj"] Dec 03 13:28:07 crc kubenswrapper[5052]: I1203 13:28:07.058801 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="keystone-kuttl-tests/keystone-db-sync-8tgzj" event={"ID":"420a4d60-e580-4289-9a33-527b9203fca1","Type":"ContainerStarted","Data":"e7266642c6bd413f33aeaa06fe477903708f8f8e1b5f82a2dfdfef551cafbf10"} Dec 03 13:28:07 crc kubenswrapper[5052]: I1203 13:28:07.059823 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="keystone-kuttl-tests/keystone-db-sync-8tgzj" event={"ID":"420a4d60-e580-4289-9a33-527b9203fca1","Type":"ContainerStarted","Data":"7f4206e247e17a916e92ea34129f8b9930115656fa15fcc9e1fe8ff9703bd8d7"} Dec 03 13:28:07 crc kubenswrapper[5052]: I1203 13:28:07.089768 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="keystone-kuttl-tests/keystone-db-sync-8tgzj" podStartSLOduration=2.089741015 podStartE2EDuration="2.089741015s" podCreationTimestamp="2025-12-03 13:28:05 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 13:28:07.080117239 +0000 UTC m=+7677.668359013" watchObservedRunningTime="2025-12-03 13:28:07.089741015 +0000 UTC m=+7677.677982779" Dec 03 13:28:09 crc kubenswrapper[5052]: I1203 13:28:09.074547 5052 generic.go:334] "Generic (PLEG): container finished" podID="420a4d60-e580-4289-9a33-527b9203fca1" containerID="e7266642c6bd413f33aeaa06fe477903708f8f8e1b5f82a2dfdfef551cafbf10" exitCode=0 Dec 03 13:28:09 crc kubenswrapper[5052]: I1203 13:28:09.074643 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="keystone-kuttl-tests/keystone-db-sync-8tgzj" event={"ID":"420a4d60-e580-4289-9a33-527b9203fca1","Type":"ContainerDied","Data":"e7266642c6bd413f33aeaa06fe477903708f8f8e1b5f82a2dfdfef551cafbf10"} Dec 03 13:28:10 crc kubenswrapper[5052]: I1203 13:28:10.372999 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="keystone-kuttl-tests/keystone-db-sync-8tgzj" Dec 03 13:28:10 crc kubenswrapper[5052]: I1203 13:28:10.530218 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/420a4d60-e580-4289-9a33-527b9203fca1-config-data\") pod \"420a4d60-e580-4289-9a33-527b9203fca1\" (UID: \"420a4d60-e580-4289-9a33-527b9203fca1\") " Dec 03 13:28:10 crc kubenswrapper[5052]: I1203 13:28:10.531213 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mfztk\" (UniqueName: \"kubernetes.io/projected/420a4d60-e580-4289-9a33-527b9203fca1-kube-api-access-mfztk\") pod \"420a4d60-e580-4289-9a33-527b9203fca1\" (UID: \"420a4d60-e580-4289-9a33-527b9203fca1\") " Dec 03 13:28:10 crc kubenswrapper[5052]: I1203 13:28:10.543067 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/420a4d60-e580-4289-9a33-527b9203fca1-kube-api-access-mfztk" (OuterVolumeSpecName: "kube-api-access-mfztk") pod "420a4d60-e580-4289-9a33-527b9203fca1" (UID: "420a4d60-e580-4289-9a33-527b9203fca1"). InnerVolumeSpecName "kube-api-access-mfztk". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:28:10 crc kubenswrapper[5052]: I1203 13:28:10.579263 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/420a4d60-e580-4289-9a33-527b9203fca1-config-data" (OuterVolumeSpecName: "config-data") pod "420a4d60-e580-4289-9a33-527b9203fca1" (UID: "420a4d60-e580-4289-9a33-527b9203fca1"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:28:10 crc kubenswrapper[5052]: I1203 13:28:10.634194 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/420a4d60-e580-4289-9a33-527b9203fca1-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 13:28:10 crc kubenswrapper[5052]: I1203 13:28:10.634264 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-mfztk\" (UniqueName: \"kubernetes.io/projected/420a4d60-e580-4289-9a33-527b9203fca1-kube-api-access-mfztk\") on node \"crc\" DevicePath \"\"" Dec 03 13:28:11 crc kubenswrapper[5052]: I1203 13:28:11.096369 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="keystone-kuttl-tests/keystone-db-sync-8tgzj" event={"ID":"420a4d60-e580-4289-9a33-527b9203fca1","Type":"ContainerDied","Data":"7f4206e247e17a916e92ea34129f8b9930115656fa15fcc9e1fe8ff9703bd8d7"} Dec 03 13:28:11 crc kubenswrapper[5052]: I1203 13:28:11.096433 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7f4206e247e17a916e92ea34129f8b9930115656fa15fcc9e1fe8ff9703bd8d7" Dec 03 13:28:11 crc kubenswrapper[5052]: I1203 13:28:11.096495 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="keystone-kuttl-tests/keystone-db-sync-8tgzj" Dec 03 13:28:11 crc kubenswrapper[5052]: I1203 13:28:11.631749 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["keystone-kuttl-tests/keystone-bootstrap-gvh6k"] Dec 03 13:28:11 crc kubenswrapper[5052]: E1203 13:28:11.632235 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="420a4d60-e580-4289-9a33-527b9203fca1" containerName="keystone-db-sync" Dec 03 13:28:11 crc kubenswrapper[5052]: I1203 13:28:11.632251 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="420a4d60-e580-4289-9a33-527b9203fca1" containerName="keystone-db-sync" Dec 03 13:28:11 crc kubenswrapper[5052]: I1203 13:28:11.632423 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="420a4d60-e580-4289-9a33-527b9203fca1" containerName="keystone-db-sync" Dec 03 13:28:11 crc kubenswrapper[5052]: I1203 13:28:11.633205 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="keystone-kuttl-tests/keystone-bootstrap-gvh6k" Dec 03 13:28:11 crc kubenswrapper[5052]: I1203 13:28:11.635937 5052 reflector.go:368] Caches populated for *v1.Secret from object-"keystone-kuttl-tests"/"keystone" Dec 03 13:28:11 crc kubenswrapper[5052]: I1203 13:28:11.636094 5052 reflector.go:368] Caches populated for *v1.Secret from object-"keystone-kuttl-tests"/"keystone-keystone-dockercfg-h96kl" Dec 03 13:28:11 crc kubenswrapper[5052]: I1203 13:28:11.637272 5052 reflector.go:368] Caches populated for *v1.Secret from object-"keystone-kuttl-tests"/"osp-secret" Dec 03 13:28:11 crc kubenswrapper[5052]: I1203 13:28:11.637679 5052 reflector.go:368] Caches populated for *v1.Secret from object-"keystone-kuttl-tests"/"keystone-scripts" Dec 03 13:28:11 crc kubenswrapper[5052]: I1203 13:28:11.643076 5052 reflector.go:368] Caches populated for *v1.Secret from object-"keystone-kuttl-tests"/"keystone-config-data" Dec 03 13:28:11 crc kubenswrapper[5052]: I1203 13:28:11.664327 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["keystone-kuttl-tests/keystone-bootstrap-gvh6k"] Dec 03 13:28:11 crc kubenswrapper[5052]: I1203 13:28:11.763551 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/57b69c1e-a382-47ca-9ba1-e5a0f5fc1bb8-fernet-keys\") pod \"keystone-bootstrap-gvh6k\" (UID: \"57b69c1e-a382-47ca-9ba1-e5a0f5fc1bb8\") " pod="keystone-kuttl-tests/keystone-bootstrap-gvh6k" Dec 03 13:28:11 crc kubenswrapper[5052]: I1203 13:28:11.763808 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/57b69c1e-a382-47ca-9ba1-e5a0f5fc1bb8-scripts\") pod \"keystone-bootstrap-gvh6k\" (UID: \"57b69c1e-a382-47ca-9ba1-e5a0f5fc1bb8\") " pod="keystone-kuttl-tests/keystone-bootstrap-gvh6k" Dec 03 13:28:11 crc kubenswrapper[5052]: I1203 13:28:11.763937 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/57b69c1e-a382-47ca-9ba1-e5a0f5fc1bb8-credential-keys\") pod \"keystone-bootstrap-gvh6k\" (UID: \"57b69c1e-a382-47ca-9ba1-e5a0f5fc1bb8\") " pod="keystone-kuttl-tests/keystone-bootstrap-gvh6k" Dec 03 13:28:11 crc kubenswrapper[5052]: I1203 13:28:11.764117 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p8n4l\" (UniqueName: \"kubernetes.io/projected/57b69c1e-a382-47ca-9ba1-e5a0f5fc1bb8-kube-api-access-p8n4l\") pod \"keystone-bootstrap-gvh6k\" (UID: \"57b69c1e-a382-47ca-9ba1-e5a0f5fc1bb8\") " pod="keystone-kuttl-tests/keystone-bootstrap-gvh6k" Dec 03 13:28:11 crc kubenswrapper[5052]: I1203 13:28:11.764204 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/57b69c1e-a382-47ca-9ba1-e5a0f5fc1bb8-config-data\") pod \"keystone-bootstrap-gvh6k\" (UID: \"57b69c1e-a382-47ca-9ba1-e5a0f5fc1bb8\") " pod="keystone-kuttl-tests/keystone-bootstrap-gvh6k" Dec 03 13:28:11 crc kubenswrapper[5052]: I1203 13:28:11.865907 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/57b69c1e-a382-47ca-9ba1-e5a0f5fc1bb8-credential-keys\") pod \"keystone-bootstrap-gvh6k\" (UID: \"57b69c1e-a382-47ca-9ba1-e5a0f5fc1bb8\") " pod="keystone-kuttl-tests/keystone-bootstrap-gvh6k" Dec 03 13:28:11 crc kubenswrapper[5052]: I1203 13:28:11.866004 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-p8n4l\" (UniqueName: \"kubernetes.io/projected/57b69c1e-a382-47ca-9ba1-e5a0f5fc1bb8-kube-api-access-p8n4l\") pod \"keystone-bootstrap-gvh6k\" (UID: \"57b69c1e-a382-47ca-9ba1-e5a0f5fc1bb8\") " pod="keystone-kuttl-tests/keystone-bootstrap-gvh6k" Dec 03 13:28:11 crc kubenswrapper[5052]: I1203 13:28:11.866057 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/57b69c1e-a382-47ca-9ba1-e5a0f5fc1bb8-config-data\") pod \"keystone-bootstrap-gvh6k\" (UID: \"57b69c1e-a382-47ca-9ba1-e5a0f5fc1bb8\") " pod="keystone-kuttl-tests/keystone-bootstrap-gvh6k" Dec 03 13:28:11 crc kubenswrapper[5052]: I1203 13:28:11.866114 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/57b69c1e-a382-47ca-9ba1-e5a0f5fc1bb8-fernet-keys\") pod \"keystone-bootstrap-gvh6k\" (UID: \"57b69c1e-a382-47ca-9ba1-e5a0f5fc1bb8\") " pod="keystone-kuttl-tests/keystone-bootstrap-gvh6k" Dec 03 13:28:11 crc kubenswrapper[5052]: I1203 13:28:11.866164 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/57b69c1e-a382-47ca-9ba1-e5a0f5fc1bb8-scripts\") pod \"keystone-bootstrap-gvh6k\" (UID: \"57b69c1e-a382-47ca-9ba1-e5a0f5fc1bb8\") " pod="keystone-kuttl-tests/keystone-bootstrap-gvh6k" Dec 03 13:28:11 crc kubenswrapper[5052]: I1203 13:28:11.871180 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/57b69c1e-a382-47ca-9ba1-e5a0f5fc1bb8-scripts\") pod \"keystone-bootstrap-gvh6k\" (UID: \"57b69c1e-a382-47ca-9ba1-e5a0f5fc1bb8\") " pod="keystone-kuttl-tests/keystone-bootstrap-gvh6k" Dec 03 13:28:11 crc kubenswrapper[5052]: I1203 13:28:11.871187 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/57b69c1e-a382-47ca-9ba1-e5a0f5fc1bb8-fernet-keys\") pod \"keystone-bootstrap-gvh6k\" (UID: \"57b69c1e-a382-47ca-9ba1-e5a0f5fc1bb8\") " pod="keystone-kuttl-tests/keystone-bootstrap-gvh6k" Dec 03 13:28:11 crc kubenswrapper[5052]: I1203 13:28:11.871829 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/57b69c1e-a382-47ca-9ba1-e5a0f5fc1bb8-credential-keys\") pod \"keystone-bootstrap-gvh6k\" (UID: \"57b69c1e-a382-47ca-9ba1-e5a0f5fc1bb8\") " pod="keystone-kuttl-tests/keystone-bootstrap-gvh6k" Dec 03 13:28:11 crc kubenswrapper[5052]: I1203 13:28:11.873912 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/57b69c1e-a382-47ca-9ba1-e5a0f5fc1bb8-config-data\") pod \"keystone-bootstrap-gvh6k\" (UID: \"57b69c1e-a382-47ca-9ba1-e5a0f5fc1bb8\") " pod="keystone-kuttl-tests/keystone-bootstrap-gvh6k" Dec 03 13:28:11 crc kubenswrapper[5052]: I1203 13:28:11.886303 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-p8n4l\" (UniqueName: \"kubernetes.io/projected/57b69c1e-a382-47ca-9ba1-e5a0f5fc1bb8-kube-api-access-p8n4l\") pod \"keystone-bootstrap-gvh6k\" (UID: \"57b69c1e-a382-47ca-9ba1-e5a0f5fc1bb8\") " pod="keystone-kuttl-tests/keystone-bootstrap-gvh6k" Dec 03 13:28:12 crc kubenswrapper[5052]: I1203 13:28:12.015669 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="keystone-kuttl-tests/keystone-bootstrap-gvh6k" Dec 03 13:28:12 crc kubenswrapper[5052]: I1203 13:28:12.474247 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["keystone-kuttl-tests/keystone-bootstrap-gvh6k"] Dec 03 13:28:13 crc kubenswrapper[5052]: I1203 13:28:13.139835 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="keystone-kuttl-tests/keystone-bootstrap-gvh6k" event={"ID":"57b69c1e-a382-47ca-9ba1-e5a0f5fc1bb8","Type":"ContainerStarted","Data":"8f609298a2f431078988cc66a6f6635d4d096a14318a691ecacb9d76ebb0e7a4"} Dec 03 13:28:13 crc kubenswrapper[5052]: I1203 13:28:13.140458 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="keystone-kuttl-tests/keystone-bootstrap-gvh6k" event={"ID":"57b69c1e-a382-47ca-9ba1-e5a0f5fc1bb8","Type":"ContainerStarted","Data":"a45e4dba9d616fec29db22eebb195fb6b62e2caac09120d28e3374c261438bcf"} Dec 03 13:28:13 crc kubenswrapper[5052]: I1203 13:28:13.164595 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="keystone-kuttl-tests/keystone-bootstrap-gvh6k" podStartSLOduration=2.164557443 podStartE2EDuration="2.164557443s" podCreationTimestamp="2025-12-03 13:28:11 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 13:28:13.159703004 +0000 UTC m=+7683.747944788" watchObservedRunningTime="2025-12-03 13:28:13.164557443 +0000 UTC m=+7683.752799217" Dec 03 13:28:16 crc kubenswrapper[5052]: I1203 13:28:16.168035 5052 generic.go:334] "Generic (PLEG): container finished" podID="57b69c1e-a382-47ca-9ba1-e5a0f5fc1bb8" containerID="8f609298a2f431078988cc66a6f6635d4d096a14318a691ecacb9d76ebb0e7a4" exitCode=0 Dec 03 13:28:16 crc kubenswrapper[5052]: I1203 13:28:16.168161 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="keystone-kuttl-tests/keystone-bootstrap-gvh6k" event={"ID":"57b69c1e-a382-47ca-9ba1-e5a0f5fc1bb8","Type":"ContainerDied","Data":"8f609298a2f431078988cc66a6f6635d4d096a14318a691ecacb9d76ebb0e7a4"} Dec 03 13:28:17 crc kubenswrapper[5052]: I1203 13:28:17.521104 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="keystone-kuttl-tests/keystone-bootstrap-gvh6k" Dec 03 13:28:17 crc kubenswrapper[5052]: I1203 13:28:17.681060 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p8n4l\" (UniqueName: \"kubernetes.io/projected/57b69c1e-a382-47ca-9ba1-e5a0f5fc1bb8-kube-api-access-p8n4l\") pod \"57b69c1e-a382-47ca-9ba1-e5a0f5fc1bb8\" (UID: \"57b69c1e-a382-47ca-9ba1-e5a0f5fc1bb8\") " Dec 03 13:28:17 crc kubenswrapper[5052]: I1203 13:28:17.681186 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/57b69c1e-a382-47ca-9ba1-e5a0f5fc1bb8-config-data\") pod \"57b69c1e-a382-47ca-9ba1-e5a0f5fc1bb8\" (UID: \"57b69c1e-a382-47ca-9ba1-e5a0f5fc1bb8\") " Dec 03 13:28:17 crc kubenswrapper[5052]: I1203 13:28:17.681239 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/57b69c1e-a382-47ca-9ba1-e5a0f5fc1bb8-scripts\") pod \"57b69c1e-a382-47ca-9ba1-e5a0f5fc1bb8\" (UID: \"57b69c1e-a382-47ca-9ba1-e5a0f5fc1bb8\") " Dec 03 13:28:17 crc kubenswrapper[5052]: I1203 13:28:17.681438 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/57b69c1e-a382-47ca-9ba1-e5a0f5fc1bb8-fernet-keys\") pod \"57b69c1e-a382-47ca-9ba1-e5a0f5fc1bb8\" (UID: \"57b69c1e-a382-47ca-9ba1-e5a0f5fc1bb8\") " Dec 03 13:28:17 crc kubenswrapper[5052]: I1203 13:28:17.681477 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/57b69c1e-a382-47ca-9ba1-e5a0f5fc1bb8-credential-keys\") pod \"57b69c1e-a382-47ca-9ba1-e5a0f5fc1bb8\" (UID: \"57b69c1e-a382-47ca-9ba1-e5a0f5fc1bb8\") " Dec 03 13:28:17 crc kubenswrapper[5052]: I1203 13:28:17.690830 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/57b69c1e-a382-47ca-9ba1-e5a0f5fc1bb8-scripts" (OuterVolumeSpecName: "scripts") pod "57b69c1e-a382-47ca-9ba1-e5a0f5fc1bb8" (UID: "57b69c1e-a382-47ca-9ba1-e5a0f5fc1bb8"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:28:17 crc kubenswrapper[5052]: I1203 13:28:17.691384 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/57b69c1e-a382-47ca-9ba1-e5a0f5fc1bb8-kube-api-access-p8n4l" (OuterVolumeSpecName: "kube-api-access-p8n4l") pod "57b69c1e-a382-47ca-9ba1-e5a0f5fc1bb8" (UID: "57b69c1e-a382-47ca-9ba1-e5a0f5fc1bb8"). InnerVolumeSpecName "kube-api-access-p8n4l". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:28:17 crc kubenswrapper[5052]: I1203 13:28:17.710005 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/57b69c1e-a382-47ca-9ba1-e5a0f5fc1bb8-fernet-keys" (OuterVolumeSpecName: "fernet-keys") pod "57b69c1e-a382-47ca-9ba1-e5a0f5fc1bb8" (UID: "57b69c1e-a382-47ca-9ba1-e5a0f5fc1bb8"). InnerVolumeSpecName "fernet-keys". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:28:17 crc kubenswrapper[5052]: I1203 13:28:17.719847 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/57b69c1e-a382-47ca-9ba1-e5a0f5fc1bb8-config-data" (OuterVolumeSpecName: "config-data") pod "57b69c1e-a382-47ca-9ba1-e5a0f5fc1bb8" (UID: "57b69c1e-a382-47ca-9ba1-e5a0f5fc1bb8"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:28:17 crc kubenswrapper[5052]: I1203 13:28:17.721623 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/57b69c1e-a382-47ca-9ba1-e5a0f5fc1bb8-credential-keys" (OuterVolumeSpecName: "credential-keys") pod "57b69c1e-a382-47ca-9ba1-e5a0f5fc1bb8" (UID: "57b69c1e-a382-47ca-9ba1-e5a0f5fc1bb8"). InnerVolumeSpecName "credential-keys". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:28:17 crc kubenswrapper[5052]: I1203 13:28:17.787777 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/57b69c1e-a382-47ca-9ba1-e5a0f5fc1bb8-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 13:28:17 crc kubenswrapper[5052]: I1203 13:28:17.787825 5052 reconciler_common.go:293] "Volume detached for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/57b69c1e-a382-47ca-9ba1-e5a0f5fc1bb8-fernet-keys\") on node \"crc\" DevicePath \"\"" Dec 03 13:28:17 crc kubenswrapper[5052]: I1203 13:28:17.787874 5052 reconciler_common.go:293] "Volume detached for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/57b69c1e-a382-47ca-9ba1-e5a0f5fc1bb8-credential-keys\") on node \"crc\" DevicePath \"\"" Dec 03 13:28:17 crc kubenswrapper[5052]: I1203 13:28:17.787898 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-p8n4l\" (UniqueName: \"kubernetes.io/projected/57b69c1e-a382-47ca-9ba1-e5a0f5fc1bb8-kube-api-access-p8n4l\") on node \"crc\" DevicePath \"\"" Dec 03 13:28:17 crc kubenswrapper[5052]: I1203 13:28:17.787914 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/57b69c1e-a382-47ca-9ba1-e5a0f5fc1bb8-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 13:28:18 crc kubenswrapper[5052]: I1203 13:28:18.190671 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="keystone-kuttl-tests/keystone-bootstrap-gvh6k" Dec 03 13:28:18 crc kubenswrapper[5052]: I1203 13:28:18.190720 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="keystone-kuttl-tests/keystone-bootstrap-gvh6k" event={"ID":"57b69c1e-a382-47ca-9ba1-e5a0f5fc1bb8","Type":"ContainerDied","Data":"a45e4dba9d616fec29db22eebb195fb6b62e2caac09120d28e3374c261438bcf"} Dec 03 13:28:18 crc kubenswrapper[5052]: I1203 13:28:18.190806 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a45e4dba9d616fec29db22eebb195fb6b62e2caac09120d28e3374c261438bcf" Dec 03 13:28:18 crc kubenswrapper[5052]: I1203 13:28:18.393012 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["keystone-kuttl-tests/keystone-6c857bcbcd-4n6lw"] Dec 03 13:28:18 crc kubenswrapper[5052]: E1203 13:28:18.393456 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="57b69c1e-a382-47ca-9ba1-e5a0f5fc1bb8" containerName="keystone-bootstrap" Dec 03 13:28:18 crc kubenswrapper[5052]: I1203 13:28:18.393481 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="57b69c1e-a382-47ca-9ba1-e5a0f5fc1bb8" containerName="keystone-bootstrap" Dec 03 13:28:18 crc kubenswrapper[5052]: I1203 13:28:18.393687 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="57b69c1e-a382-47ca-9ba1-e5a0f5fc1bb8" containerName="keystone-bootstrap" Dec 03 13:28:18 crc kubenswrapper[5052]: I1203 13:28:18.394368 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="keystone-kuttl-tests/keystone-6c857bcbcd-4n6lw" Dec 03 13:28:18 crc kubenswrapper[5052]: I1203 13:28:18.396971 5052 reflector.go:368] Caches populated for *v1.Secret from object-"keystone-kuttl-tests"/"keystone-config-data" Dec 03 13:28:18 crc kubenswrapper[5052]: I1203 13:28:18.397222 5052 reflector.go:368] Caches populated for *v1.Secret from object-"keystone-kuttl-tests"/"keystone-keystone-dockercfg-h96kl" Dec 03 13:28:18 crc kubenswrapper[5052]: I1203 13:28:18.397385 5052 reflector.go:368] Caches populated for *v1.Secret from object-"keystone-kuttl-tests"/"keystone" Dec 03 13:28:18 crc kubenswrapper[5052]: I1203 13:28:18.399604 5052 reflector.go:368] Caches populated for *v1.Secret from object-"keystone-kuttl-tests"/"keystone-scripts" Dec 03 13:28:18 crc kubenswrapper[5052]: I1203 13:28:18.406883 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["keystone-kuttl-tests/keystone-6c857bcbcd-4n6lw"] Dec 03 13:28:18 crc kubenswrapper[5052]: I1203 13:28:18.500611 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hj6hs\" (UniqueName: \"kubernetes.io/projected/e7224fc6-4050-43bd-95ce-b6257be9ed06-kube-api-access-hj6hs\") pod \"keystone-6c857bcbcd-4n6lw\" (UID: \"e7224fc6-4050-43bd-95ce-b6257be9ed06\") " pod="keystone-kuttl-tests/keystone-6c857bcbcd-4n6lw" Dec 03 13:28:18 crc kubenswrapper[5052]: I1203 13:28:18.500676 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/e7224fc6-4050-43bd-95ce-b6257be9ed06-config-data\") pod \"keystone-6c857bcbcd-4n6lw\" (UID: \"e7224fc6-4050-43bd-95ce-b6257be9ed06\") " pod="keystone-kuttl-tests/keystone-6c857bcbcd-4n6lw" Dec 03 13:28:18 crc kubenswrapper[5052]: I1203 13:28:18.500700 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/e7224fc6-4050-43bd-95ce-b6257be9ed06-credential-keys\") pod \"keystone-6c857bcbcd-4n6lw\" (UID: \"e7224fc6-4050-43bd-95ce-b6257be9ed06\") " pod="keystone-kuttl-tests/keystone-6c857bcbcd-4n6lw" Dec 03 13:28:18 crc kubenswrapper[5052]: I1203 13:28:18.500746 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/e7224fc6-4050-43bd-95ce-b6257be9ed06-scripts\") pod \"keystone-6c857bcbcd-4n6lw\" (UID: \"e7224fc6-4050-43bd-95ce-b6257be9ed06\") " pod="keystone-kuttl-tests/keystone-6c857bcbcd-4n6lw" Dec 03 13:28:18 crc kubenswrapper[5052]: I1203 13:28:18.500764 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/e7224fc6-4050-43bd-95ce-b6257be9ed06-fernet-keys\") pod \"keystone-6c857bcbcd-4n6lw\" (UID: \"e7224fc6-4050-43bd-95ce-b6257be9ed06\") " pod="keystone-kuttl-tests/keystone-6c857bcbcd-4n6lw" Dec 03 13:28:18 crc kubenswrapper[5052]: I1203 13:28:18.604279 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/e7224fc6-4050-43bd-95ce-b6257be9ed06-scripts\") pod \"keystone-6c857bcbcd-4n6lw\" (UID: \"e7224fc6-4050-43bd-95ce-b6257be9ed06\") " pod="keystone-kuttl-tests/keystone-6c857bcbcd-4n6lw" Dec 03 13:28:18 crc kubenswrapper[5052]: I1203 13:28:18.604360 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/e7224fc6-4050-43bd-95ce-b6257be9ed06-fernet-keys\") pod \"keystone-6c857bcbcd-4n6lw\" (UID: \"e7224fc6-4050-43bd-95ce-b6257be9ed06\") " pod="keystone-kuttl-tests/keystone-6c857bcbcd-4n6lw" Dec 03 13:28:18 crc kubenswrapper[5052]: I1203 13:28:18.604489 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-hj6hs\" (UniqueName: \"kubernetes.io/projected/e7224fc6-4050-43bd-95ce-b6257be9ed06-kube-api-access-hj6hs\") pod \"keystone-6c857bcbcd-4n6lw\" (UID: \"e7224fc6-4050-43bd-95ce-b6257be9ed06\") " pod="keystone-kuttl-tests/keystone-6c857bcbcd-4n6lw" Dec 03 13:28:18 crc kubenswrapper[5052]: I1203 13:28:18.604549 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/e7224fc6-4050-43bd-95ce-b6257be9ed06-config-data\") pod \"keystone-6c857bcbcd-4n6lw\" (UID: \"e7224fc6-4050-43bd-95ce-b6257be9ed06\") " pod="keystone-kuttl-tests/keystone-6c857bcbcd-4n6lw" Dec 03 13:28:18 crc kubenswrapper[5052]: I1203 13:28:18.604602 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/e7224fc6-4050-43bd-95ce-b6257be9ed06-credential-keys\") pod \"keystone-6c857bcbcd-4n6lw\" (UID: \"e7224fc6-4050-43bd-95ce-b6257be9ed06\") " pod="keystone-kuttl-tests/keystone-6c857bcbcd-4n6lw" Dec 03 13:28:18 crc kubenswrapper[5052]: I1203 13:28:18.609160 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/e7224fc6-4050-43bd-95ce-b6257be9ed06-fernet-keys\") pod \"keystone-6c857bcbcd-4n6lw\" (UID: \"e7224fc6-4050-43bd-95ce-b6257be9ed06\") " pod="keystone-kuttl-tests/keystone-6c857bcbcd-4n6lw" Dec 03 13:28:18 crc kubenswrapper[5052]: I1203 13:28:18.609283 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/e7224fc6-4050-43bd-95ce-b6257be9ed06-credential-keys\") pod \"keystone-6c857bcbcd-4n6lw\" (UID: \"e7224fc6-4050-43bd-95ce-b6257be9ed06\") " pod="keystone-kuttl-tests/keystone-6c857bcbcd-4n6lw" Dec 03 13:28:18 crc kubenswrapper[5052]: I1203 13:28:18.609418 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/e7224fc6-4050-43bd-95ce-b6257be9ed06-config-data\") pod \"keystone-6c857bcbcd-4n6lw\" (UID: \"e7224fc6-4050-43bd-95ce-b6257be9ed06\") " pod="keystone-kuttl-tests/keystone-6c857bcbcd-4n6lw" Dec 03 13:28:18 crc kubenswrapper[5052]: I1203 13:28:18.610675 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/e7224fc6-4050-43bd-95ce-b6257be9ed06-scripts\") pod \"keystone-6c857bcbcd-4n6lw\" (UID: \"e7224fc6-4050-43bd-95ce-b6257be9ed06\") " pod="keystone-kuttl-tests/keystone-6c857bcbcd-4n6lw" Dec 03 13:28:18 crc kubenswrapper[5052]: I1203 13:28:18.629535 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-hj6hs\" (UniqueName: \"kubernetes.io/projected/e7224fc6-4050-43bd-95ce-b6257be9ed06-kube-api-access-hj6hs\") pod \"keystone-6c857bcbcd-4n6lw\" (UID: \"e7224fc6-4050-43bd-95ce-b6257be9ed06\") " pod="keystone-kuttl-tests/keystone-6c857bcbcd-4n6lw" Dec 03 13:28:18 crc kubenswrapper[5052]: I1203 13:28:18.724928 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="keystone-kuttl-tests/keystone-6c857bcbcd-4n6lw" Dec 03 13:28:18 crc kubenswrapper[5052]: I1203 13:28:18.966224 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["keystone-kuttl-tests/keystone-6c857bcbcd-4n6lw"] Dec 03 13:28:19 crc kubenswrapper[5052]: I1203 13:28:19.203781 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="keystone-kuttl-tests/keystone-6c857bcbcd-4n6lw" event={"ID":"e7224fc6-4050-43bd-95ce-b6257be9ed06","Type":"ContainerStarted","Data":"5343e1050e418b924cbc8dd6d94de00ba87a38833e47b7c24b33ef15a3be59b4"} Dec 03 13:28:19 crc kubenswrapper[5052]: I1203 13:28:19.204414 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="keystone-kuttl-tests/keystone-6c857bcbcd-4n6lw" Dec 03 13:28:19 crc kubenswrapper[5052]: I1203 13:28:19.204430 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="keystone-kuttl-tests/keystone-6c857bcbcd-4n6lw" event={"ID":"e7224fc6-4050-43bd-95ce-b6257be9ed06","Type":"ContainerStarted","Data":"ad1979e69a5c187e51317c6d8b117beb91a87bfaeed839e8f6b9203fa4438eaf"} Dec 03 13:28:19 crc kubenswrapper[5052]: I1203 13:28:19.228721 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="keystone-kuttl-tests/keystone-6c857bcbcd-4n6lw" podStartSLOduration=1.228678551 podStartE2EDuration="1.228678551s" podCreationTimestamp="2025-12-03 13:28:18 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 13:28:19.227165628 +0000 UTC m=+7689.815407392" watchObservedRunningTime="2025-12-03 13:28:19.228678551 +0000 UTC m=+7689.816920315" Dec 03 13:28:50 crc kubenswrapper[5052]: I1203 13:28:50.333205 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="keystone-kuttl-tests/keystone-6c857bcbcd-4n6lw" Dec 03 13:28:50 crc kubenswrapper[5052]: I1203 13:28:50.926460 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["keystone-kuttl-tests/openstackclient"] Dec 03 13:28:50 crc kubenswrapper[5052]: I1203 13:28:50.927877 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="keystone-kuttl-tests/openstackclient" Dec 03 13:28:50 crc kubenswrapper[5052]: I1203 13:28:50.929870 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"keystone-kuttl-tests"/"openstack-config" Dec 03 13:28:50 crc kubenswrapper[5052]: I1203 13:28:50.929897 5052 reflector.go:368] Caches populated for *v1.Secret from object-"keystone-kuttl-tests"/"openstack-config-secret" Dec 03 13:28:50 crc kubenswrapper[5052]: I1203 13:28:50.932389 5052 reflector.go:368] Caches populated for *v1.Secret from object-"keystone-kuttl-tests"/"default-dockercfg-4n42s" Dec 03 13:28:50 crc kubenswrapper[5052]: I1203 13:28:50.934069 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["keystone-kuttl-tests/openstackclient"] Dec 03 13:28:51 crc kubenswrapper[5052]: I1203 13:28:51.046237 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-phhqb\" (UniqueName: \"kubernetes.io/projected/973f3dde-e424-4de8-a9fa-3c267647e3de-kube-api-access-phhqb\") pod \"openstackclient\" (UID: \"973f3dde-e424-4de8-a9fa-3c267647e3de\") " pod="keystone-kuttl-tests/openstackclient" Dec 03 13:28:51 crc kubenswrapper[5052]: I1203 13:28:51.047394 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openstack-config-secret\" (UniqueName: \"kubernetes.io/secret/973f3dde-e424-4de8-a9fa-3c267647e3de-openstack-config-secret\") pod \"openstackclient\" (UID: \"973f3dde-e424-4de8-a9fa-3c267647e3de\") " pod="keystone-kuttl-tests/openstackclient" Dec 03 13:28:51 crc kubenswrapper[5052]: I1203 13:28:51.047544 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openstack-config\" (UniqueName: \"kubernetes.io/configmap/973f3dde-e424-4de8-a9fa-3c267647e3de-openstack-config\") pod \"openstackclient\" (UID: \"973f3dde-e424-4de8-a9fa-3c267647e3de\") " pod="keystone-kuttl-tests/openstackclient" Dec 03 13:28:51 crc kubenswrapper[5052]: I1203 13:28:51.149246 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"openstack-config\" (UniqueName: \"kubernetes.io/configmap/973f3dde-e424-4de8-a9fa-3c267647e3de-openstack-config\") pod \"openstackclient\" (UID: \"973f3dde-e424-4de8-a9fa-3c267647e3de\") " pod="keystone-kuttl-tests/openstackclient" Dec 03 13:28:51 crc kubenswrapper[5052]: I1203 13:28:51.149333 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-phhqb\" (UniqueName: \"kubernetes.io/projected/973f3dde-e424-4de8-a9fa-3c267647e3de-kube-api-access-phhqb\") pod \"openstackclient\" (UID: \"973f3dde-e424-4de8-a9fa-3c267647e3de\") " pod="keystone-kuttl-tests/openstackclient" Dec 03 13:28:51 crc kubenswrapper[5052]: I1203 13:28:51.149386 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"openstack-config-secret\" (UniqueName: \"kubernetes.io/secret/973f3dde-e424-4de8-a9fa-3c267647e3de-openstack-config-secret\") pod \"openstackclient\" (UID: \"973f3dde-e424-4de8-a9fa-3c267647e3de\") " pod="keystone-kuttl-tests/openstackclient" Dec 03 13:28:51 crc kubenswrapper[5052]: I1203 13:28:51.150125 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"openstack-config\" (UniqueName: \"kubernetes.io/configmap/973f3dde-e424-4de8-a9fa-3c267647e3de-openstack-config\") pod \"openstackclient\" (UID: \"973f3dde-e424-4de8-a9fa-3c267647e3de\") " pod="keystone-kuttl-tests/openstackclient" Dec 03 13:28:51 crc kubenswrapper[5052]: I1203 13:28:51.154849 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"openstack-config-secret\" (UniqueName: \"kubernetes.io/secret/973f3dde-e424-4de8-a9fa-3c267647e3de-openstack-config-secret\") pod \"openstackclient\" (UID: \"973f3dde-e424-4de8-a9fa-3c267647e3de\") " pod="keystone-kuttl-tests/openstackclient" Dec 03 13:28:51 crc kubenswrapper[5052]: I1203 13:28:51.166140 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-phhqb\" (UniqueName: \"kubernetes.io/projected/973f3dde-e424-4de8-a9fa-3c267647e3de-kube-api-access-phhqb\") pod \"openstackclient\" (UID: \"973f3dde-e424-4de8-a9fa-3c267647e3de\") " pod="keystone-kuttl-tests/openstackclient" Dec 03 13:28:51 crc kubenswrapper[5052]: I1203 13:28:51.252672 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="keystone-kuttl-tests/openstackclient" Dec 03 13:28:51 crc kubenswrapper[5052]: I1203 13:28:51.668045 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["keystone-kuttl-tests/openstackclient"] Dec 03 13:28:52 crc kubenswrapper[5052]: I1203 13:28:52.488112 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="keystone-kuttl-tests/openstackclient" event={"ID":"973f3dde-e424-4de8-a9fa-3c267647e3de","Type":"ContainerStarted","Data":"bc4acb938c8313fd21fbff043e2dc683926b4d15a3a2fe645f80ac6ea13382f5"} Dec 03 13:28:52 crc kubenswrapper[5052]: I1203 13:28:52.488458 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="keystone-kuttl-tests/openstackclient" event={"ID":"973f3dde-e424-4de8-a9fa-3c267647e3de","Type":"ContainerStarted","Data":"40c85982bff78eaa7774e362ba5b2f3ce542ec20a73c9b0801600e41a722a74d"} Dec 03 13:28:52 crc kubenswrapper[5052]: I1203 13:28:52.508380 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="keystone-kuttl-tests/openstackclient" podStartSLOduration=1.9664871590000002 podStartE2EDuration="2.508361799s" podCreationTimestamp="2025-12-03 13:28:50 +0000 UTC" firstStartedPulling="2025-12-03 13:28:51.674474245 +0000 UTC m=+7722.262716009" lastFinishedPulling="2025-12-03 13:28:52.216348885 +0000 UTC m=+7722.804590649" observedRunningTime="2025-12-03 13:28:52.506834485 +0000 UTC m=+7723.095076259" watchObservedRunningTime="2025-12-03 13:28:52.508361799 +0000 UTC m=+7723.096603573" Dec 03 13:28:59 crc kubenswrapper[5052]: I1203 13:28:59.379044 5052 patch_prober.go:28] interesting pod/machine-config-daemon-74bj6 container/machine-config-daemon namespace/openshift-machine-config-operator: Liveness probe status=failure output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" start-of-body= Dec 03 13:28:59 crc kubenswrapper[5052]: I1203 13:28:59.379709 5052 prober.go:107] "Probe failed" probeType="Liveness" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" containerName="machine-config-daemon" probeResult="failure" output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" Dec 03 13:29:29 crc kubenswrapper[5052]: I1203 13:29:29.380176 5052 patch_prober.go:28] interesting pod/machine-config-daemon-74bj6 container/machine-config-daemon namespace/openshift-machine-config-operator: Liveness probe status=failure output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" start-of-body= Dec 03 13:29:29 crc kubenswrapper[5052]: I1203 13:29:29.380806 5052 prober.go:107] "Probe failed" probeType="Liveness" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" containerName="machine-config-daemon" probeResult="failure" output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" Dec 03 13:29:48 crc kubenswrapper[5052]: I1203 13:29:48.303400 5052 scope.go:117] "RemoveContainer" containerID="777ec4f26345b1716eff3fdf7d0c3ccea9c1431897696a7ce94f5adae5fdd623" Dec 03 13:29:48 crc kubenswrapper[5052]: I1203 13:29:48.343757 5052 scope.go:117] "RemoveContainer" containerID="5c25cf982fe9f2774b23e40fa8b85755d117d93a53b1f6ffe4553bf32d68b53c" Dec 03 13:29:48 crc kubenswrapper[5052]: I1203 13:29:48.366357 5052 scope.go:117] "RemoveContainer" containerID="276fd58ee69d0b6333d60f17fdb07e4b7cc6c2ad653cdd45d3f2534d88c59b8e" Dec 03 13:29:48 crc kubenswrapper[5052]: I1203 13:29:48.406157 5052 scope.go:117] "RemoveContainer" containerID="e6eb2354b7b82fbf58211b1ade674871f298871d3dde2b23f2b3cda4d4628594" Dec 03 13:29:59 crc kubenswrapper[5052]: I1203 13:29:59.379580 5052 patch_prober.go:28] interesting pod/machine-config-daemon-74bj6 container/machine-config-daemon namespace/openshift-machine-config-operator: Liveness probe status=failure output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" start-of-body= Dec 03 13:29:59 crc kubenswrapper[5052]: I1203 13:29:59.380209 5052 prober.go:107] "Probe failed" probeType="Liveness" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" containerName="machine-config-daemon" probeResult="failure" output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" Dec 03 13:29:59 crc kubenswrapper[5052]: I1203 13:29:59.380268 5052 kubelet.go:2542] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" Dec 03 13:29:59 crc kubenswrapper[5052]: I1203 13:29:59.381163 5052 kuberuntime_manager.go:1027] "Message for Container of pod" containerName="machine-config-daemon" containerStatusID={"Type":"cri-o","ID":"4dbf4cc5b784c22fc4eecbfecde10f7c7824875bd119667c218a37a8cbc71d15"} pod="openshift-machine-config-operator/machine-config-daemon-74bj6" containerMessage="Container machine-config-daemon failed liveness probe, will be restarted" Dec 03 13:29:59 crc kubenswrapper[5052]: I1203 13:29:59.381247 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" containerName="machine-config-daemon" containerID="cri-o://4dbf4cc5b784c22fc4eecbfecde10f7c7824875bd119667c218a37a8cbc71d15" gracePeriod=600 Dec 03 13:29:59 crc kubenswrapper[5052]: E1203 13:29:59.642538 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 13:30:00 crc kubenswrapper[5052]: I1203 13:30:00.026693 5052 generic.go:334] "Generic (PLEG): container finished" podID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" containerID="4dbf4cc5b784c22fc4eecbfecde10f7c7824875bd119667c218a37a8cbc71d15" exitCode=0 Dec 03 13:30:00 crc kubenswrapper[5052]: I1203 13:30:00.026751 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" event={"ID":"10c237bc-72e7-4da3-9f2c-b1688395bf6e","Type":"ContainerDied","Data":"4dbf4cc5b784c22fc4eecbfecde10f7c7824875bd119667c218a37a8cbc71d15"} Dec 03 13:30:00 crc kubenswrapper[5052]: I1203 13:30:00.026798 5052 scope.go:117] "RemoveContainer" containerID="6f2eaa045b10c8c10abc2278895801064c7c776bf5ef0e26c0544a891311f82f" Dec 03 13:30:00 crc kubenswrapper[5052]: I1203 13:30:00.027668 5052 scope.go:117] "RemoveContainer" containerID="4dbf4cc5b784c22fc4eecbfecde10f7c7824875bd119667c218a37a8cbc71d15" Dec 03 13:30:00 crc kubenswrapper[5052]: E1203 13:30:00.028001 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 13:30:00 crc kubenswrapper[5052]: I1203 13:30:00.172493 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openshift-operator-lifecycle-manager/collect-profiles-29412810-4cw4z"] Dec 03 13:30:00 crc kubenswrapper[5052]: I1203 13:30:00.173604 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-operator-lifecycle-manager/collect-profiles-29412810-4cw4z" Dec 03 13:30:00 crc kubenswrapper[5052]: I1203 13:30:00.176160 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-operator-lifecycle-manager"/"collect-profiles-dockercfg-kzf4t" Dec 03 13:30:00 crc kubenswrapper[5052]: I1203 13:30:00.179121 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-operator-lifecycle-manager"/"collect-profiles-config" Dec 03 13:30:00 crc kubenswrapper[5052]: I1203 13:30:00.190326 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-operator-lifecycle-manager/collect-profiles-29412810-4cw4z"] Dec 03 13:30:00 crc kubenswrapper[5052]: I1203 13:30:00.237700 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/4ee42a38-5507-4b91-896a-96a64ac7ffc3-config-volume\") pod \"collect-profiles-29412810-4cw4z\" (UID: \"4ee42a38-5507-4b91-896a-96a64ac7ffc3\") " pod="openshift-operator-lifecycle-manager/collect-profiles-29412810-4cw4z" Dec 03 13:30:00 crc kubenswrapper[5052]: I1203 13:30:00.237811 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-volume\" (UniqueName: \"kubernetes.io/secret/4ee42a38-5507-4b91-896a-96a64ac7ffc3-secret-volume\") pod \"collect-profiles-29412810-4cw4z\" (UID: \"4ee42a38-5507-4b91-896a-96a64ac7ffc3\") " pod="openshift-operator-lifecycle-manager/collect-profiles-29412810-4cw4z" Dec 03 13:30:00 crc kubenswrapper[5052]: I1203 13:30:00.237869 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wkbhn\" (UniqueName: \"kubernetes.io/projected/4ee42a38-5507-4b91-896a-96a64ac7ffc3-kube-api-access-wkbhn\") pod \"collect-profiles-29412810-4cw4z\" (UID: \"4ee42a38-5507-4b91-896a-96a64ac7ffc3\") " pod="openshift-operator-lifecycle-manager/collect-profiles-29412810-4cw4z" Dec 03 13:30:00 crc kubenswrapper[5052]: I1203 13:30:00.339664 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"secret-volume\" (UniqueName: \"kubernetes.io/secret/4ee42a38-5507-4b91-896a-96a64ac7ffc3-secret-volume\") pod \"collect-profiles-29412810-4cw4z\" (UID: \"4ee42a38-5507-4b91-896a-96a64ac7ffc3\") " pod="openshift-operator-lifecycle-manager/collect-profiles-29412810-4cw4z" Dec 03 13:30:00 crc kubenswrapper[5052]: I1203 13:30:00.339736 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-wkbhn\" (UniqueName: \"kubernetes.io/projected/4ee42a38-5507-4b91-896a-96a64ac7ffc3-kube-api-access-wkbhn\") pod \"collect-profiles-29412810-4cw4z\" (UID: \"4ee42a38-5507-4b91-896a-96a64ac7ffc3\") " pod="openshift-operator-lifecycle-manager/collect-profiles-29412810-4cw4z" Dec 03 13:30:00 crc kubenswrapper[5052]: I1203 13:30:00.339846 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/4ee42a38-5507-4b91-896a-96a64ac7ffc3-config-volume\") pod \"collect-profiles-29412810-4cw4z\" (UID: \"4ee42a38-5507-4b91-896a-96a64ac7ffc3\") " pod="openshift-operator-lifecycle-manager/collect-profiles-29412810-4cw4z" Dec 03 13:30:00 crc kubenswrapper[5052]: I1203 13:30:00.340835 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/4ee42a38-5507-4b91-896a-96a64ac7ffc3-config-volume\") pod \"collect-profiles-29412810-4cw4z\" (UID: \"4ee42a38-5507-4b91-896a-96a64ac7ffc3\") " pod="openshift-operator-lifecycle-manager/collect-profiles-29412810-4cw4z" Dec 03 13:30:00 crc kubenswrapper[5052]: I1203 13:30:00.346935 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"secret-volume\" (UniqueName: \"kubernetes.io/secret/4ee42a38-5507-4b91-896a-96a64ac7ffc3-secret-volume\") pod \"collect-profiles-29412810-4cw4z\" (UID: \"4ee42a38-5507-4b91-896a-96a64ac7ffc3\") " pod="openshift-operator-lifecycle-manager/collect-profiles-29412810-4cw4z" Dec 03 13:30:00 crc kubenswrapper[5052]: I1203 13:30:00.362651 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-wkbhn\" (UniqueName: \"kubernetes.io/projected/4ee42a38-5507-4b91-896a-96a64ac7ffc3-kube-api-access-wkbhn\") pod \"collect-profiles-29412810-4cw4z\" (UID: \"4ee42a38-5507-4b91-896a-96a64ac7ffc3\") " pod="openshift-operator-lifecycle-manager/collect-profiles-29412810-4cw4z" Dec 03 13:30:00 crc kubenswrapper[5052]: I1203 13:30:00.497240 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-operator-lifecycle-manager/collect-profiles-29412810-4cw4z" Dec 03 13:30:01 crc kubenswrapper[5052]: I1203 13:30:01.002658 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-operator-lifecycle-manager/collect-profiles-29412810-4cw4z"] Dec 03 13:30:01 crc kubenswrapper[5052]: W1203 13:30:01.021707 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ee42a38_5507_4b91_896a_96a64ac7ffc3.slice/crio-c966352e1e6171b8d386314bb48aa5dd47ec330ed7d39487149bfb6449705501 WatchSource:0}: Error finding container c966352e1e6171b8d386314bb48aa5dd47ec330ed7d39487149bfb6449705501: Status 404 returned error can't find the container with id c966352e1e6171b8d386314bb48aa5dd47ec330ed7d39487149bfb6449705501 Dec 03 13:30:01 crc kubenswrapper[5052]: I1203 13:30:01.039493 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-operator-lifecycle-manager/collect-profiles-29412810-4cw4z" event={"ID":"4ee42a38-5507-4b91-896a-96a64ac7ffc3","Type":"ContainerStarted","Data":"c966352e1e6171b8d386314bb48aa5dd47ec330ed7d39487149bfb6449705501"} Dec 03 13:30:02 crc kubenswrapper[5052]: I1203 13:30:02.049548 5052 generic.go:334] "Generic (PLEG): container finished" podID="4ee42a38-5507-4b91-896a-96a64ac7ffc3" containerID="c1378929115995f43dc3acb5efc848f8c739c70e6c1e43999ad266f60fb3f4c9" exitCode=0 Dec 03 13:30:02 crc kubenswrapper[5052]: I1203 13:30:02.049751 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-operator-lifecycle-manager/collect-profiles-29412810-4cw4z" event={"ID":"4ee42a38-5507-4b91-896a-96a64ac7ffc3","Type":"ContainerDied","Data":"c1378929115995f43dc3acb5efc848f8c739c70e6c1e43999ad266f60fb3f4c9"} Dec 03 13:30:03 crc kubenswrapper[5052]: I1203 13:30:03.320326 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-operator-lifecycle-manager/collect-profiles-29412810-4cw4z" Dec 03 13:30:03 crc kubenswrapper[5052]: I1203 13:30:03.500107 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/4ee42a38-5507-4b91-896a-96a64ac7ffc3-config-volume\") pod \"4ee42a38-5507-4b91-896a-96a64ac7ffc3\" (UID: \"4ee42a38-5507-4b91-896a-96a64ac7ffc3\") " Dec 03 13:30:03 crc kubenswrapper[5052]: I1203 13:30:03.500240 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"secret-volume\" (UniqueName: \"kubernetes.io/secret/4ee42a38-5507-4b91-896a-96a64ac7ffc3-secret-volume\") pod \"4ee42a38-5507-4b91-896a-96a64ac7ffc3\" (UID: \"4ee42a38-5507-4b91-896a-96a64ac7ffc3\") " Dec 03 13:30:03 crc kubenswrapper[5052]: I1203 13:30:03.500341 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wkbhn\" (UniqueName: \"kubernetes.io/projected/4ee42a38-5507-4b91-896a-96a64ac7ffc3-kube-api-access-wkbhn\") pod \"4ee42a38-5507-4b91-896a-96a64ac7ffc3\" (UID: \"4ee42a38-5507-4b91-896a-96a64ac7ffc3\") " Dec 03 13:30:03 crc kubenswrapper[5052]: I1203 13:30:03.500869 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4ee42a38-5507-4b91-896a-96a64ac7ffc3-config-volume" (OuterVolumeSpecName: "config-volume") pod "4ee42a38-5507-4b91-896a-96a64ac7ffc3" (UID: "4ee42a38-5507-4b91-896a-96a64ac7ffc3"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:30:03 crc kubenswrapper[5052]: I1203 13:30:03.506471 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4ee42a38-5507-4b91-896a-96a64ac7ffc3-kube-api-access-wkbhn" (OuterVolumeSpecName: "kube-api-access-wkbhn") pod "4ee42a38-5507-4b91-896a-96a64ac7ffc3" (UID: "4ee42a38-5507-4b91-896a-96a64ac7ffc3"). InnerVolumeSpecName "kube-api-access-wkbhn". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:30:03 crc kubenswrapper[5052]: I1203 13:30:03.506567 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4ee42a38-5507-4b91-896a-96a64ac7ffc3-secret-volume" (OuterVolumeSpecName: "secret-volume") pod "4ee42a38-5507-4b91-896a-96a64ac7ffc3" (UID: "4ee42a38-5507-4b91-896a-96a64ac7ffc3"). InnerVolumeSpecName "secret-volume". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:30:03 crc kubenswrapper[5052]: I1203 13:30:03.602869 5052 reconciler_common.go:293] "Volume detached for volume \"secret-volume\" (UniqueName: \"kubernetes.io/secret/4ee42a38-5507-4b91-896a-96a64ac7ffc3-secret-volume\") on node \"crc\" DevicePath \"\"" Dec 03 13:30:03 crc kubenswrapper[5052]: I1203 13:30:03.602961 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-wkbhn\" (UniqueName: \"kubernetes.io/projected/4ee42a38-5507-4b91-896a-96a64ac7ffc3-kube-api-access-wkbhn\") on node \"crc\" DevicePath \"\"" Dec 03 13:30:03 crc kubenswrapper[5052]: I1203 13:30:03.602979 5052 reconciler_common.go:293] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/4ee42a38-5507-4b91-896a-96a64ac7ffc3-config-volume\") on node \"crc\" DevicePath \"\"" Dec 03 13:30:04 crc kubenswrapper[5052]: I1203 13:30:04.066846 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-operator-lifecycle-manager/collect-profiles-29412810-4cw4z" event={"ID":"4ee42a38-5507-4b91-896a-96a64ac7ffc3","Type":"ContainerDied","Data":"c966352e1e6171b8d386314bb48aa5dd47ec330ed7d39487149bfb6449705501"} Dec 03 13:30:04 crc kubenswrapper[5052]: I1203 13:30:04.066902 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c966352e1e6171b8d386314bb48aa5dd47ec330ed7d39487149bfb6449705501" Dec 03 13:30:04 crc kubenswrapper[5052]: I1203 13:30:04.066990 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-operator-lifecycle-manager/collect-profiles-29412810-4cw4z" Dec 03 13:30:04 crc kubenswrapper[5052]: I1203 13:30:04.416161 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openshift-operator-lifecycle-manager/collect-profiles-29412765-wndkq"] Dec 03 13:30:04 crc kubenswrapper[5052]: I1203 13:30:04.422344 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openshift-operator-lifecycle-manager/collect-profiles-29412765-wndkq"] Dec 03 13:30:05 crc kubenswrapper[5052]: I1203 13:30:05.687025 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="64cc53c7-bf05-4064-98ee-ae5c931896cd" path="/var/lib/kubelet/pods/64cc53c7-bf05-4064-98ee-ae5c931896cd/volumes" Dec 03 13:30:14 crc kubenswrapper[5052]: I1203 13:30:14.673558 5052 scope.go:117] "RemoveContainer" containerID="4dbf4cc5b784c22fc4eecbfecde10f7c7824875bd119667c218a37a8cbc71d15" Dec 03 13:30:14 crc kubenswrapper[5052]: E1203 13:30:14.674253 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 13:30:25 crc kubenswrapper[5052]: I1203 13:30:25.673356 5052 scope.go:117] "RemoveContainer" containerID="4dbf4cc5b784c22fc4eecbfecde10f7c7824875bd119667c218a37a8cbc71d15" Dec 03 13:30:25 crc kubenswrapper[5052]: E1203 13:30:25.675945 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 13:30:39 crc kubenswrapper[5052]: I1203 13:30:39.678235 5052 scope.go:117] "RemoveContainer" containerID="4dbf4cc5b784c22fc4eecbfecde10f7c7824875bd119667c218a37a8cbc71d15" Dec 03 13:30:39 crc kubenswrapper[5052]: E1203 13:30:39.679086 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 13:30:48 crc kubenswrapper[5052]: I1203 13:30:48.486693 5052 scope.go:117] "RemoveContainer" containerID="b082d4fab31821909ca8acfed207b5720e05d5fdeb6d37090657dfe881f3b038" Dec 03 13:30:48 crc kubenswrapper[5052]: I1203 13:30:48.525610 5052 scope.go:117] "RemoveContainer" containerID="646a267de36c895038df3d44efc515c5d638e303a5b0b303acb5bc3f5fc4bc54" Dec 03 13:30:48 crc kubenswrapper[5052]: I1203 13:30:48.568910 5052 scope.go:117] "RemoveContainer" containerID="099eaf13ff9f270a1f63f166fcf530c4a80c2dbee77e267ce7612003cad35e12" Dec 03 13:30:48 crc kubenswrapper[5052]: I1203 13:30:48.590612 5052 scope.go:117] "RemoveContainer" containerID="afc7258a68d4ed2a4b7cff8e1f795eeb19f0ce27b182320ca9cfd851d51e629e" Dec 03 13:30:48 crc kubenswrapper[5052]: I1203 13:30:48.637202 5052 scope.go:117] "RemoveContainer" containerID="1d45929c6ce39bf4ba7e60b055c3cb3b4cf00eb1c74796eee6b556fd8acbbd30" Dec 03 13:30:48 crc kubenswrapper[5052]: I1203 13:30:48.673014 5052 scope.go:117] "RemoveContainer" containerID="cb8cc9c892038369c63d4b0785205187517f29920130257f2326f188cdfb66d9" Dec 03 13:30:52 crc kubenswrapper[5052]: I1203 13:30:52.672708 5052 scope.go:117] "RemoveContainer" containerID="4dbf4cc5b784c22fc4eecbfecde10f7c7824875bd119667c218a37a8cbc71d15" Dec 03 13:30:52 crc kubenswrapper[5052]: E1203 13:30:52.673487 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 13:31:05 crc kubenswrapper[5052]: I1203 13:31:05.673304 5052 scope.go:117] "RemoveContainer" containerID="4dbf4cc5b784c22fc4eecbfecde10f7c7824875bd119667c218a37a8cbc71d15" Dec 03 13:31:05 crc kubenswrapper[5052]: E1203 13:31:05.674282 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 13:31:08 crc kubenswrapper[5052]: I1203 13:31:08.636136 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openshift-marketplace/certified-operators-tx7pt"] Dec 03 13:31:08 crc kubenswrapper[5052]: E1203 13:31:08.636904 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="4ee42a38-5507-4b91-896a-96a64ac7ffc3" containerName="collect-profiles" Dec 03 13:31:08 crc kubenswrapper[5052]: I1203 13:31:08.636919 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="4ee42a38-5507-4b91-896a-96a64ac7ffc3" containerName="collect-profiles" Dec 03 13:31:08 crc kubenswrapper[5052]: I1203 13:31:08.637058 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="4ee42a38-5507-4b91-896a-96a64ac7ffc3" containerName="collect-profiles" Dec 03 13:31:08 crc kubenswrapper[5052]: I1203 13:31:08.638267 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/certified-operators-tx7pt" Dec 03 13:31:08 crc kubenswrapper[5052]: I1203 13:31:08.669735 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/certified-operators-tx7pt"] Dec 03 13:31:08 crc kubenswrapper[5052]: I1203 13:31:08.738332 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dj8zm\" (UniqueName: \"kubernetes.io/projected/68c8fa22-b461-4ee9-b2d4-89b18459ac27-kube-api-access-dj8zm\") pod \"certified-operators-tx7pt\" (UID: \"68c8fa22-b461-4ee9-b2d4-89b18459ac27\") " pod="openshift-marketplace/certified-operators-tx7pt" Dec 03 13:31:08 crc kubenswrapper[5052]: I1203 13:31:08.738394 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/68c8fa22-b461-4ee9-b2d4-89b18459ac27-catalog-content\") pod \"certified-operators-tx7pt\" (UID: \"68c8fa22-b461-4ee9-b2d4-89b18459ac27\") " pod="openshift-marketplace/certified-operators-tx7pt" Dec 03 13:31:08 crc kubenswrapper[5052]: I1203 13:31:08.738560 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/68c8fa22-b461-4ee9-b2d4-89b18459ac27-utilities\") pod \"certified-operators-tx7pt\" (UID: \"68c8fa22-b461-4ee9-b2d4-89b18459ac27\") " pod="openshift-marketplace/certified-operators-tx7pt" Dec 03 13:31:08 crc kubenswrapper[5052]: I1203 13:31:08.839716 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/68c8fa22-b461-4ee9-b2d4-89b18459ac27-catalog-content\") pod \"certified-operators-tx7pt\" (UID: \"68c8fa22-b461-4ee9-b2d4-89b18459ac27\") " pod="openshift-marketplace/certified-operators-tx7pt" Dec 03 13:31:08 crc kubenswrapper[5052]: I1203 13:31:08.839833 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/68c8fa22-b461-4ee9-b2d4-89b18459ac27-utilities\") pod \"certified-operators-tx7pt\" (UID: \"68c8fa22-b461-4ee9-b2d4-89b18459ac27\") " pod="openshift-marketplace/certified-operators-tx7pt" Dec 03 13:31:08 crc kubenswrapper[5052]: I1203 13:31:08.840000 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-dj8zm\" (UniqueName: \"kubernetes.io/projected/68c8fa22-b461-4ee9-b2d4-89b18459ac27-kube-api-access-dj8zm\") pod \"certified-operators-tx7pt\" (UID: \"68c8fa22-b461-4ee9-b2d4-89b18459ac27\") " pod="openshift-marketplace/certified-operators-tx7pt" Dec 03 13:31:08 crc kubenswrapper[5052]: I1203 13:31:08.840630 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/68c8fa22-b461-4ee9-b2d4-89b18459ac27-catalog-content\") pod \"certified-operators-tx7pt\" (UID: \"68c8fa22-b461-4ee9-b2d4-89b18459ac27\") " pod="openshift-marketplace/certified-operators-tx7pt" Dec 03 13:31:08 crc kubenswrapper[5052]: I1203 13:31:08.841407 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/68c8fa22-b461-4ee9-b2d4-89b18459ac27-utilities\") pod \"certified-operators-tx7pt\" (UID: \"68c8fa22-b461-4ee9-b2d4-89b18459ac27\") " pod="openshift-marketplace/certified-operators-tx7pt" Dec 03 13:31:08 crc kubenswrapper[5052]: I1203 13:31:08.863857 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-dj8zm\" (UniqueName: \"kubernetes.io/projected/68c8fa22-b461-4ee9-b2d4-89b18459ac27-kube-api-access-dj8zm\") pod \"certified-operators-tx7pt\" (UID: \"68c8fa22-b461-4ee9-b2d4-89b18459ac27\") " pod="openshift-marketplace/certified-operators-tx7pt" Dec 03 13:31:09 crc kubenswrapper[5052]: I1203 13:31:09.027214 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/certified-operators-tx7pt" Dec 03 13:31:09 crc kubenswrapper[5052]: I1203 13:31:09.359808 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/certified-operators-tx7pt"] Dec 03 13:31:09 crc kubenswrapper[5052]: W1203 13:31:09.370975 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68c8fa22_b461_4ee9_b2d4_89b18459ac27.slice/crio-616ff7e1a91e664ce76b18bfc9fee7450aa4cbdd857dc430d6a2da3bc07dc1e9 WatchSource:0}: Error finding container 616ff7e1a91e664ce76b18bfc9fee7450aa4cbdd857dc430d6a2da3bc07dc1e9: Status 404 returned error can't find the container with id 616ff7e1a91e664ce76b18bfc9fee7450aa4cbdd857dc430d6a2da3bc07dc1e9 Dec 03 13:31:09 crc kubenswrapper[5052]: I1203 13:31:09.665457 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/certified-operators-tx7pt" event={"ID":"68c8fa22-b461-4ee9-b2d4-89b18459ac27","Type":"ContainerStarted","Data":"616ff7e1a91e664ce76b18bfc9fee7450aa4cbdd857dc430d6a2da3bc07dc1e9"} Dec 03 13:31:10 crc kubenswrapper[5052]: I1203 13:31:10.674479 5052 generic.go:334] "Generic (PLEG): container finished" podID="68c8fa22-b461-4ee9-b2d4-89b18459ac27" containerID="4b8d0857772ad34b60fbe6216eb669db4403278ec3c1fff536bc8126da9e077b" exitCode=0 Dec 03 13:31:10 crc kubenswrapper[5052]: I1203 13:31:10.674553 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/certified-operators-tx7pt" event={"ID":"68c8fa22-b461-4ee9-b2d4-89b18459ac27","Type":"ContainerDied","Data":"4b8d0857772ad34b60fbe6216eb669db4403278ec3c1fff536bc8126da9e077b"} Dec 03 13:31:12 crc kubenswrapper[5052]: I1203 13:31:12.690420 5052 generic.go:334] "Generic (PLEG): container finished" podID="68c8fa22-b461-4ee9-b2d4-89b18459ac27" containerID="dcfdbd679ccb44368a035156158cc57ff7876702ad68b25e0f7c65a6e6d2ee2e" exitCode=0 Dec 03 13:31:12 crc kubenswrapper[5052]: I1203 13:31:12.690461 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/certified-operators-tx7pt" event={"ID":"68c8fa22-b461-4ee9-b2d4-89b18459ac27","Type":"ContainerDied","Data":"dcfdbd679ccb44368a035156158cc57ff7876702ad68b25e0f7c65a6e6d2ee2e"} Dec 03 13:31:13 crc kubenswrapper[5052]: I1203 13:31:13.698501 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/certified-operators-tx7pt" event={"ID":"68c8fa22-b461-4ee9-b2d4-89b18459ac27","Type":"ContainerStarted","Data":"cad13e7812e3f23e48d3aab6cb800eca450132bc6ccb5e20a527cf94f00e1d2f"} Dec 03 13:31:13 crc kubenswrapper[5052]: I1203 13:31:13.723104 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-marketplace/certified-operators-tx7pt" podStartSLOduration=3.231070445 podStartE2EDuration="5.723085748s" podCreationTimestamp="2025-12-03 13:31:08 +0000 UTC" firstStartedPulling="2025-12-03 13:31:10.677649584 +0000 UTC m=+7861.265891358" lastFinishedPulling="2025-12-03 13:31:13.169664897 +0000 UTC m=+7863.757906661" observedRunningTime="2025-12-03 13:31:13.719053012 +0000 UTC m=+7864.307294776" watchObservedRunningTime="2025-12-03 13:31:13.723085748 +0000 UTC m=+7864.311327512" Dec 03 13:31:18 crc kubenswrapper[5052]: I1203 13:31:18.673112 5052 scope.go:117] "RemoveContainer" containerID="4dbf4cc5b784c22fc4eecbfecde10f7c7824875bd119667c218a37a8cbc71d15" Dec 03 13:31:18 crc kubenswrapper[5052]: E1203 13:31:18.673440 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 13:31:19 crc kubenswrapper[5052]: I1203 13:31:19.027999 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-marketplace/certified-operators-tx7pt" Dec 03 13:31:19 crc kubenswrapper[5052]: I1203 13:31:19.028063 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openshift-marketplace/certified-operators-tx7pt" Dec 03 13:31:19 crc kubenswrapper[5052]: I1203 13:31:19.072269 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-marketplace/certified-operators-tx7pt" Dec 03 13:31:19 crc kubenswrapper[5052]: I1203 13:31:19.790475 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-marketplace/certified-operators-tx7pt" Dec 03 13:31:19 crc kubenswrapper[5052]: I1203 13:31:19.837548 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openshift-marketplace/certified-operators-tx7pt"] Dec 03 13:31:21 crc kubenswrapper[5052]: I1203 13:31:21.761573 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openshift-marketplace/certified-operators-tx7pt" podUID="68c8fa22-b461-4ee9-b2d4-89b18459ac27" containerName="registry-server" containerID="cri-o://cad13e7812e3f23e48d3aab6cb800eca450132bc6ccb5e20a527cf94f00e1d2f" gracePeriod=2 Dec 03 13:31:23 crc kubenswrapper[5052]: I1203 13:31:23.778352 5052 generic.go:334] "Generic (PLEG): container finished" podID="68c8fa22-b461-4ee9-b2d4-89b18459ac27" containerID="cad13e7812e3f23e48d3aab6cb800eca450132bc6ccb5e20a527cf94f00e1d2f" exitCode=0 Dec 03 13:31:23 crc kubenswrapper[5052]: I1203 13:31:23.778581 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/certified-operators-tx7pt" event={"ID":"68c8fa22-b461-4ee9-b2d4-89b18459ac27","Type":"ContainerDied","Data":"cad13e7812e3f23e48d3aab6cb800eca450132bc6ccb5e20a527cf94f00e1d2f"} Dec 03 13:31:23 crc kubenswrapper[5052]: I1203 13:31:23.948911 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/certified-operators-tx7pt" Dec 03 13:31:24 crc kubenswrapper[5052]: I1203 13:31:24.096230 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dj8zm\" (UniqueName: \"kubernetes.io/projected/68c8fa22-b461-4ee9-b2d4-89b18459ac27-kube-api-access-dj8zm\") pod \"68c8fa22-b461-4ee9-b2d4-89b18459ac27\" (UID: \"68c8fa22-b461-4ee9-b2d4-89b18459ac27\") " Dec 03 13:31:24 crc kubenswrapper[5052]: I1203 13:31:24.096393 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/68c8fa22-b461-4ee9-b2d4-89b18459ac27-utilities\") pod \"68c8fa22-b461-4ee9-b2d4-89b18459ac27\" (UID: \"68c8fa22-b461-4ee9-b2d4-89b18459ac27\") " Dec 03 13:31:24 crc kubenswrapper[5052]: I1203 13:31:24.096473 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/68c8fa22-b461-4ee9-b2d4-89b18459ac27-catalog-content\") pod \"68c8fa22-b461-4ee9-b2d4-89b18459ac27\" (UID: \"68c8fa22-b461-4ee9-b2d4-89b18459ac27\") " Dec 03 13:31:24 crc kubenswrapper[5052]: I1203 13:31:24.097837 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/68c8fa22-b461-4ee9-b2d4-89b18459ac27-utilities" (OuterVolumeSpecName: "utilities") pod "68c8fa22-b461-4ee9-b2d4-89b18459ac27" (UID: "68c8fa22-b461-4ee9-b2d4-89b18459ac27"). InnerVolumeSpecName "utilities". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 13:31:24 crc kubenswrapper[5052]: I1203 13:31:24.098401 5052 reconciler_common.go:293] "Volume detached for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/68c8fa22-b461-4ee9-b2d4-89b18459ac27-utilities\") on node \"crc\" DevicePath \"\"" Dec 03 13:31:24 crc kubenswrapper[5052]: I1203 13:31:24.108765 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/68c8fa22-b461-4ee9-b2d4-89b18459ac27-kube-api-access-dj8zm" (OuterVolumeSpecName: "kube-api-access-dj8zm") pod "68c8fa22-b461-4ee9-b2d4-89b18459ac27" (UID: "68c8fa22-b461-4ee9-b2d4-89b18459ac27"). InnerVolumeSpecName "kube-api-access-dj8zm". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:31:24 crc kubenswrapper[5052]: I1203 13:31:24.144871 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/68c8fa22-b461-4ee9-b2d4-89b18459ac27-catalog-content" (OuterVolumeSpecName: "catalog-content") pod "68c8fa22-b461-4ee9-b2d4-89b18459ac27" (UID: "68c8fa22-b461-4ee9-b2d4-89b18459ac27"). InnerVolumeSpecName "catalog-content". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 13:31:24 crc kubenswrapper[5052]: I1203 13:31:24.200275 5052 reconciler_common.go:293] "Volume detached for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/68c8fa22-b461-4ee9-b2d4-89b18459ac27-catalog-content\") on node \"crc\" DevicePath \"\"" Dec 03 13:31:24 crc kubenswrapper[5052]: I1203 13:31:24.200332 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-dj8zm\" (UniqueName: \"kubernetes.io/projected/68c8fa22-b461-4ee9-b2d4-89b18459ac27-kube-api-access-dj8zm\") on node \"crc\" DevicePath \"\"" Dec 03 13:31:24 crc kubenswrapper[5052]: I1203 13:31:24.787772 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/certified-operators-tx7pt" event={"ID":"68c8fa22-b461-4ee9-b2d4-89b18459ac27","Type":"ContainerDied","Data":"616ff7e1a91e664ce76b18bfc9fee7450aa4cbdd857dc430d6a2da3bc07dc1e9"} Dec 03 13:31:24 crc kubenswrapper[5052]: I1203 13:31:24.787824 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/certified-operators-tx7pt" Dec 03 13:31:24 crc kubenswrapper[5052]: I1203 13:31:24.788085 5052 scope.go:117] "RemoveContainer" containerID="cad13e7812e3f23e48d3aab6cb800eca450132bc6ccb5e20a527cf94f00e1d2f" Dec 03 13:31:24 crc kubenswrapper[5052]: I1203 13:31:24.816103 5052 scope.go:117] "RemoveContainer" containerID="dcfdbd679ccb44368a035156158cc57ff7876702ad68b25e0f7c65a6e6d2ee2e" Dec 03 13:31:24 crc kubenswrapper[5052]: I1203 13:31:24.817555 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openshift-marketplace/certified-operators-tx7pt"] Dec 03 13:31:24 crc kubenswrapper[5052]: I1203 13:31:24.824422 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openshift-marketplace/certified-operators-tx7pt"] Dec 03 13:31:24 crc kubenswrapper[5052]: I1203 13:31:24.857574 5052 scope.go:117] "RemoveContainer" containerID="4b8d0857772ad34b60fbe6216eb669db4403278ec3c1fff536bc8126da9e077b" Dec 03 13:31:25 crc kubenswrapper[5052]: I1203 13:31:25.682656 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="68c8fa22-b461-4ee9-b2d4-89b18459ac27" path="/var/lib/kubelet/pods/68c8fa22-b461-4ee9-b2d4-89b18459ac27/volumes" Dec 03 13:31:32 crc kubenswrapper[5052]: I1203 13:31:32.672846 5052 scope.go:117] "RemoveContainer" containerID="4dbf4cc5b784c22fc4eecbfecde10f7c7824875bd119667c218a37a8cbc71d15" Dec 03 13:31:32 crc kubenswrapper[5052]: E1203 13:31:32.673733 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 13:31:45 crc kubenswrapper[5052]: I1203 13:31:45.186525 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openshift-marketplace/redhat-marketplace-rfcct"] Dec 03 13:31:45 crc kubenswrapper[5052]: E1203 13:31:45.187260 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="68c8fa22-b461-4ee9-b2d4-89b18459ac27" containerName="extract-content" Dec 03 13:31:45 crc kubenswrapper[5052]: I1203 13:31:45.187272 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="68c8fa22-b461-4ee9-b2d4-89b18459ac27" containerName="extract-content" Dec 03 13:31:45 crc kubenswrapper[5052]: E1203 13:31:45.187285 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="68c8fa22-b461-4ee9-b2d4-89b18459ac27" containerName="registry-server" Dec 03 13:31:45 crc kubenswrapper[5052]: I1203 13:31:45.187291 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="68c8fa22-b461-4ee9-b2d4-89b18459ac27" containerName="registry-server" Dec 03 13:31:45 crc kubenswrapper[5052]: E1203 13:31:45.187322 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="68c8fa22-b461-4ee9-b2d4-89b18459ac27" containerName="extract-utilities" Dec 03 13:31:45 crc kubenswrapper[5052]: I1203 13:31:45.187328 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="68c8fa22-b461-4ee9-b2d4-89b18459ac27" containerName="extract-utilities" Dec 03 13:31:45 crc kubenswrapper[5052]: I1203 13:31:45.187455 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="68c8fa22-b461-4ee9-b2d4-89b18459ac27" containerName="registry-server" Dec 03 13:31:45 crc kubenswrapper[5052]: I1203 13:31:45.188479 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/redhat-marketplace-rfcct" Dec 03 13:31:45 crc kubenswrapper[5052]: I1203 13:31:45.206296 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/redhat-marketplace-rfcct"] Dec 03 13:31:45 crc kubenswrapper[5052]: I1203 13:31:45.239035 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/559c277b-dd6a-4719-bfaa-ee68815faf85-utilities\") pod \"redhat-marketplace-rfcct\" (UID: \"559c277b-dd6a-4719-bfaa-ee68815faf85\") " pod="openshift-marketplace/redhat-marketplace-rfcct" Dec 03 13:31:45 crc kubenswrapper[5052]: I1203 13:31:45.239093 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nlmcj\" (UniqueName: \"kubernetes.io/projected/559c277b-dd6a-4719-bfaa-ee68815faf85-kube-api-access-nlmcj\") pod \"redhat-marketplace-rfcct\" (UID: \"559c277b-dd6a-4719-bfaa-ee68815faf85\") " pod="openshift-marketplace/redhat-marketplace-rfcct" Dec 03 13:31:45 crc kubenswrapper[5052]: I1203 13:31:45.239172 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/559c277b-dd6a-4719-bfaa-ee68815faf85-catalog-content\") pod \"redhat-marketplace-rfcct\" (UID: \"559c277b-dd6a-4719-bfaa-ee68815faf85\") " pod="openshift-marketplace/redhat-marketplace-rfcct" Dec 03 13:31:45 crc kubenswrapper[5052]: I1203 13:31:45.356039 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/559c277b-dd6a-4719-bfaa-ee68815faf85-utilities\") pod \"redhat-marketplace-rfcct\" (UID: \"559c277b-dd6a-4719-bfaa-ee68815faf85\") " pod="openshift-marketplace/redhat-marketplace-rfcct" Dec 03 13:31:45 crc kubenswrapper[5052]: I1203 13:31:45.356471 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-nlmcj\" (UniqueName: \"kubernetes.io/projected/559c277b-dd6a-4719-bfaa-ee68815faf85-kube-api-access-nlmcj\") pod \"redhat-marketplace-rfcct\" (UID: \"559c277b-dd6a-4719-bfaa-ee68815faf85\") " pod="openshift-marketplace/redhat-marketplace-rfcct" Dec 03 13:31:45 crc kubenswrapper[5052]: I1203 13:31:45.356582 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/559c277b-dd6a-4719-bfaa-ee68815faf85-catalog-content\") pod \"redhat-marketplace-rfcct\" (UID: \"559c277b-dd6a-4719-bfaa-ee68815faf85\") " pod="openshift-marketplace/redhat-marketplace-rfcct" Dec 03 13:31:45 crc kubenswrapper[5052]: I1203 13:31:45.356997 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/559c277b-dd6a-4719-bfaa-ee68815faf85-utilities\") pod \"redhat-marketplace-rfcct\" (UID: \"559c277b-dd6a-4719-bfaa-ee68815faf85\") " pod="openshift-marketplace/redhat-marketplace-rfcct" Dec 03 13:31:45 crc kubenswrapper[5052]: I1203 13:31:45.357114 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/559c277b-dd6a-4719-bfaa-ee68815faf85-catalog-content\") pod \"redhat-marketplace-rfcct\" (UID: \"559c277b-dd6a-4719-bfaa-ee68815faf85\") " pod="openshift-marketplace/redhat-marketplace-rfcct" Dec 03 13:31:45 crc kubenswrapper[5052]: I1203 13:31:45.382263 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-nlmcj\" (UniqueName: \"kubernetes.io/projected/559c277b-dd6a-4719-bfaa-ee68815faf85-kube-api-access-nlmcj\") pod \"redhat-marketplace-rfcct\" (UID: \"559c277b-dd6a-4719-bfaa-ee68815faf85\") " pod="openshift-marketplace/redhat-marketplace-rfcct" Dec 03 13:31:45 crc kubenswrapper[5052]: I1203 13:31:45.507991 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/redhat-marketplace-rfcct" Dec 03 13:31:45 crc kubenswrapper[5052]: I1203 13:31:45.762388 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/redhat-marketplace-rfcct"] Dec 03 13:31:45 crc kubenswrapper[5052]: I1203 13:31:45.951801 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-marketplace-rfcct" event={"ID":"559c277b-dd6a-4719-bfaa-ee68815faf85","Type":"ContainerStarted","Data":"c4a72c9cfbf5fa3e629c35eeff55156f07a1c43d2fb9e46bf6d253c2ab2295be"} Dec 03 13:31:46 crc kubenswrapper[5052]: I1203 13:31:46.672643 5052 scope.go:117] "RemoveContainer" containerID="4dbf4cc5b784c22fc4eecbfecde10f7c7824875bd119667c218a37a8cbc71d15" Dec 03 13:31:46 crc kubenswrapper[5052]: E1203 13:31:46.673189 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 13:31:46 crc kubenswrapper[5052]: I1203 13:31:46.962143 5052 generic.go:334] "Generic (PLEG): container finished" podID="559c277b-dd6a-4719-bfaa-ee68815faf85" containerID="7a86f7c82b58fde62475e0474560c9c931e60d35ddfea47e4d1af42454819207" exitCode=0 Dec 03 13:31:46 crc kubenswrapper[5052]: I1203 13:31:46.962197 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-marketplace-rfcct" event={"ID":"559c277b-dd6a-4719-bfaa-ee68815faf85","Type":"ContainerDied","Data":"7a86f7c82b58fde62475e0474560c9c931e60d35ddfea47e4d1af42454819207"} Dec 03 13:31:47 crc kubenswrapper[5052]: I1203 13:31:47.971945 5052 provider.go:102] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Dec 03 13:31:48 crc kubenswrapper[5052]: I1203 13:31:48.773278 5052 scope.go:117] "RemoveContainer" containerID="ccc93222bdd0ec8f4c393492fd2a2db39b61fc1a51c6abe7dc5153fe1fb0ec76" Dec 03 13:31:48 crc kubenswrapper[5052]: I1203 13:31:48.810861 5052 scope.go:117] "RemoveContainer" containerID="96e2baa6a38f359a88096b36441f51880922a13bfca52434fffa726ba73e87a7" Dec 03 13:31:48 crc kubenswrapper[5052]: I1203 13:31:48.831472 5052 scope.go:117] "RemoveContainer" containerID="601076b6926a51495b03f8a0a4221dc359b96fcb6bcdb02d3a1c3ebadd4a24a3" Dec 03 13:31:48 crc kubenswrapper[5052]: I1203 13:31:48.857489 5052 scope.go:117] "RemoveContainer" containerID="f91deacc7890c7e8ae6b0f49bb0ee18463db2e86e796436b3082d2a76adb0308" Dec 03 13:31:48 crc kubenswrapper[5052]: I1203 13:31:48.882545 5052 scope.go:117] "RemoveContainer" containerID="17082d2831f68dfa87f467d88de959bb9cf51c0169262a93d703df12813dea6f" Dec 03 13:31:52 crc kubenswrapper[5052]: I1203 13:31:52.025162 5052 generic.go:334] "Generic (PLEG): container finished" podID="559c277b-dd6a-4719-bfaa-ee68815faf85" containerID="50b004c671770e6363cf5d02bc68a84120ca97430e7be318d78d9b0225142c59" exitCode=0 Dec 03 13:31:52 crc kubenswrapper[5052]: I1203 13:31:52.025193 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-marketplace-rfcct" event={"ID":"559c277b-dd6a-4719-bfaa-ee68815faf85","Type":"ContainerDied","Data":"50b004c671770e6363cf5d02bc68a84120ca97430e7be318d78d9b0225142c59"} Dec 03 13:31:55 crc kubenswrapper[5052]: I1203 13:31:55.047939 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-marketplace-rfcct" event={"ID":"559c277b-dd6a-4719-bfaa-ee68815faf85","Type":"ContainerStarted","Data":"1be22dcd915a3fbc8bea26702eef1591ab25fe8e4d2934f158fed2b5665df307"} Dec 03 13:31:55 crc kubenswrapper[5052]: I1203 13:31:55.075292 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-marketplace/redhat-marketplace-rfcct" podStartSLOduration=4.266618354 podStartE2EDuration="10.075267718s" podCreationTimestamp="2025-12-03 13:31:45 +0000 UTC" firstStartedPulling="2025-12-03 13:31:47.971420122 +0000 UTC m=+7898.559661906" lastFinishedPulling="2025-12-03 13:31:53.780069506 +0000 UTC m=+7904.368311270" observedRunningTime="2025-12-03 13:31:55.068136674 +0000 UTC m=+7905.656378438" watchObservedRunningTime="2025-12-03 13:31:55.075267718 +0000 UTC m=+7905.663509482" Dec 03 13:31:55 crc kubenswrapper[5052]: I1203 13:31:55.508767 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openshift-marketplace/redhat-marketplace-rfcct" Dec 03 13:31:55 crc kubenswrapper[5052]: I1203 13:31:55.509133 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-marketplace/redhat-marketplace-rfcct" Dec 03 13:31:55 crc kubenswrapper[5052]: I1203 13:31:55.549769 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-marketplace/redhat-marketplace-rfcct" Dec 03 13:31:59 crc kubenswrapper[5052]: I1203 13:31:59.679794 5052 scope.go:117] "RemoveContainer" containerID="4dbf4cc5b784c22fc4eecbfecde10f7c7824875bd119667c218a37a8cbc71d15" Dec 03 13:31:59 crc kubenswrapper[5052]: E1203 13:31:59.680316 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 13:32:05 crc kubenswrapper[5052]: I1203 13:32:05.551778 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-marketplace/redhat-marketplace-rfcct" Dec 03 13:32:05 crc kubenswrapper[5052]: I1203 13:32:05.603285 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openshift-marketplace/redhat-marketplace-rfcct"] Dec 03 13:32:06 crc kubenswrapper[5052]: I1203 13:32:06.125778 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openshift-marketplace/redhat-marketplace-rfcct" podUID="559c277b-dd6a-4719-bfaa-ee68815faf85" containerName="registry-server" containerID="cri-o://1be22dcd915a3fbc8bea26702eef1591ab25fe8e4d2934f158fed2b5665df307" gracePeriod=2 Dec 03 13:32:07 crc kubenswrapper[5052]: I1203 13:32:07.049214 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/redhat-marketplace-rfcct" Dec 03 13:32:07 crc kubenswrapper[5052]: I1203 13:32:07.112729 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nlmcj\" (UniqueName: \"kubernetes.io/projected/559c277b-dd6a-4719-bfaa-ee68815faf85-kube-api-access-nlmcj\") pod \"559c277b-dd6a-4719-bfaa-ee68815faf85\" (UID: \"559c277b-dd6a-4719-bfaa-ee68815faf85\") " Dec 03 13:32:07 crc kubenswrapper[5052]: I1203 13:32:07.112781 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/559c277b-dd6a-4719-bfaa-ee68815faf85-utilities\") pod \"559c277b-dd6a-4719-bfaa-ee68815faf85\" (UID: \"559c277b-dd6a-4719-bfaa-ee68815faf85\") " Dec 03 13:32:07 crc kubenswrapper[5052]: I1203 13:32:07.112847 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/559c277b-dd6a-4719-bfaa-ee68815faf85-catalog-content\") pod \"559c277b-dd6a-4719-bfaa-ee68815faf85\" (UID: \"559c277b-dd6a-4719-bfaa-ee68815faf85\") " Dec 03 13:32:07 crc kubenswrapper[5052]: I1203 13:32:07.114460 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/559c277b-dd6a-4719-bfaa-ee68815faf85-utilities" (OuterVolumeSpecName: "utilities") pod "559c277b-dd6a-4719-bfaa-ee68815faf85" (UID: "559c277b-dd6a-4719-bfaa-ee68815faf85"). InnerVolumeSpecName "utilities". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 13:32:07 crc kubenswrapper[5052]: I1203 13:32:07.118974 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/559c277b-dd6a-4719-bfaa-ee68815faf85-kube-api-access-nlmcj" (OuterVolumeSpecName: "kube-api-access-nlmcj") pod "559c277b-dd6a-4719-bfaa-ee68815faf85" (UID: "559c277b-dd6a-4719-bfaa-ee68815faf85"). InnerVolumeSpecName "kube-api-access-nlmcj". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:32:07 crc kubenswrapper[5052]: I1203 13:32:07.135035 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/559c277b-dd6a-4719-bfaa-ee68815faf85-catalog-content" (OuterVolumeSpecName: "catalog-content") pod "559c277b-dd6a-4719-bfaa-ee68815faf85" (UID: "559c277b-dd6a-4719-bfaa-ee68815faf85"). InnerVolumeSpecName "catalog-content". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 13:32:07 crc kubenswrapper[5052]: I1203 13:32:07.139096 5052 generic.go:334] "Generic (PLEG): container finished" podID="559c277b-dd6a-4719-bfaa-ee68815faf85" containerID="1be22dcd915a3fbc8bea26702eef1591ab25fe8e4d2934f158fed2b5665df307" exitCode=0 Dec 03 13:32:07 crc kubenswrapper[5052]: I1203 13:32:07.139135 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-marketplace-rfcct" event={"ID":"559c277b-dd6a-4719-bfaa-ee68815faf85","Type":"ContainerDied","Data":"1be22dcd915a3fbc8bea26702eef1591ab25fe8e4d2934f158fed2b5665df307"} Dec 03 13:32:07 crc kubenswrapper[5052]: I1203 13:32:07.139165 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-marketplace-rfcct" event={"ID":"559c277b-dd6a-4719-bfaa-ee68815faf85","Type":"ContainerDied","Data":"c4a72c9cfbf5fa3e629c35eeff55156f07a1c43d2fb9e46bf6d253c2ab2295be"} Dec 03 13:32:07 crc kubenswrapper[5052]: I1203 13:32:07.139183 5052 scope.go:117] "RemoveContainer" containerID="1be22dcd915a3fbc8bea26702eef1591ab25fe8e4d2934f158fed2b5665df307" Dec 03 13:32:07 crc kubenswrapper[5052]: I1203 13:32:07.139217 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/redhat-marketplace-rfcct" Dec 03 13:32:07 crc kubenswrapper[5052]: I1203 13:32:07.162555 5052 scope.go:117] "RemoveContainer" containerID="50b004c671770e6363cf5d02bc68a84120ca97430e7be318d78d9b0225142c59" Dec 03 13:32:07 crc kubenswrapper[5052]: I1203 13:32:07.183720 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openshift-marketplace/redhat-marketplace-rfcct"] Dec 03 13:32:07 crc kubenswrapper[5052]: I1203 13:32:07.191178 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openshift-marketplace/redhat-marketplace-rfcct"] Dec 03 13:32:07 crc kubenswrapper[5052]: I1203 13:32:07.199173 5052 scope.go:117] "RemoveContainer" containerID="7a86f7c82b58fde62475e0474560c9c931e60d35ddfea47e4d1af42454819207" Dec 03 13:32:07 crc kubenswrapper[5052]: I1203 13:32:07.214634 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-nlmcj\" (UniqueName: \"kubernetes.io/projected/559c277b-dd6a-4719-bfaa-ee68815faf85-kube-api-access-nlmcj\") on node \"crc\" DevicePath \"\"" Dec 03 13:32:07 crc kubenswrapper[5052]: I1203 13:32:07.214676 5052 reconciler_common.go:293] "Volume detached for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/559c277b-dd6a-4719-bfaa-ee68815faf85-utilities\") on node \"crc\" DevicePath \"\"" Dec 03 13:32:07 crc kubenswrapper[5052]: I1203 13:32:07.214696 5052 reconciler_common.go:293] "Volume detached for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/559c277b-dd6a-4719-bfaa-ee68815faf85-catalog-content\") on node \"crc\" DevicePath \"\"" Dec 03 13:32:07 crc kubenswrapper[5052]: I1203 13:32:07.220370 5052 scope.go:117] "RemoveContainer" containerID="1be22dcd915a3fbc8bea26702eef1591ab25fe8e4d2934f158fed2b5665df307" Dec 03 13:32:07 crc kubenswrapper[5052]: E1203 13:32:07.220832 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1be22dcd915a3fbc8bea26702eef1591ab25fe8e4d2934f158fed2b5665df307\": container with ID starting with 1be22dcd915a3fbc8bea26702eef1591ab25fe8e4d2934f158fed2b5665df307 not found: ID does not exist" containerID="1be22dcd915a3fbc8bea26702eef1591ab25fe8e4d2934f158fed2b5665df307" Dec 03 13:32:07 crc kubenswrapper[5052]: I1203 13:32:07.220933 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1be22dcd915a3fbc8bea26702eef1591ab25fe8e4d2934f158fed2b5665df307"} err="failed to get container status \"1be22dcd915a3fbc8bea26702eef1591ab25fe8e4d2934f158fed2b5665df307\": rpc error: code = NotFound desc = could not find container \"1be22dcd915a3fbc8bea26702eef1591ab25fe8e4d2934f158fed2b5665df307\": container with ID starting with 1be22dcd915a3fbc8bea26702eef1591ab25fe8e4d2934f158fed2b5665df307 not found: ID does not exist" Dec 03 13:32:07 crc kubenswrapper[5052]: I1203 13:32:07.220968 5052 scope.go:117] "RemoveContainer" containerID="50b004c671770e6363cf5d02bc68a84120ca97430e7be318d78d9b0225142c59" Dec 03 13:32:07 crc kubenswrapper[5052]: E1203 13:32:07.221365 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"50b004c671770e6363cf5d02bc68a84120ca97430e7be318d78d9b0225142c59\": container with ID starting with 50b004c671770e6363cf5d02bc68a84120ca97430e7be318d78d9b0225142c59 not found: ID does not exist" containerID="50b004c671770e6363cf5d02bc68a84120ca97430e7be318d78d9b0225142c59" Dec 03 13:32:07 crc kubenswrapper[5052]: I1203 13:32:07.221400 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"50b004c671770e6363cf5d02bc68a84120ca97430e7be318d78d9b0225142c59"} err="failed to get container status \"50b004c671770e6363cf5d02bc68a84120ca97430e7be318d78d9b0225142c59\": rpc error: code = NotFound desc = could not find container \"50b004c671770e6363cf5d02bc68a84120ca97430e7be318d78d9b0225142c59\": container with ID starting with 50b004c671770e6363cf5d02bc68a84120ca97430e7be318d78d9b0225142c59 not found: ID does not exist" Dec 03 13:32:07 crc kubenswrapper[5052]: I1203 13:32:07.221442 5052 scope.go:117] "RemoveContainer" containerID="7a86f7c82b58fde62475e0474560c9c931e60d35ddfea47e4d1af42454819207" Dec 03 13:32:07 crc kubenswrapper[5052]: E1203 13:32:07.222160 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7a86f7c82b58fde62475e0474560c9c931e60d35ddfea47e4d1af42454819207\": container with ID starting with 7a86f7c82b58fde62475e0474560c9c931e60d35ddfea47e4d1af42454819207 not found: ID does not exist" containerID="7a86f7c82b58fde62475e0474560c9c931e60d35ddfea47e4d1af42454819207" Dec 03 13:32:07 crc kubenswrapper[5052]: I1203 13:32:07.222304 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7a86f7c82b58fde62475e0474560c9c931e60d35ddfea47e4d1af42454819207"} err="failed to get container status \"7a86f7c82b58fde62475e0474560c9c931e60d35ddfea47e4d1af42454819207\": rpc error: code = NotFound desc = could not find container \"7a86f7c82b58fde62475e0474560c9c931e60d35ddfea47e4d1af42454819207\": container with ID starting with 7a86f7c82b58fde62475e0474560c9c931e60d35ddfea47e4d1af42454819207 not found: ID does not exist" Dec 03 13:32:07 crc kubenswrapper[5052]: I1203 13:32:07.684048 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="559c277b-dd6a-4719-bfaa-ee68815faf85" path="/var/lib/kubelet/pods/559c277b-dd6a-4719-bfaa-ee68815faf85/volumes" Dec 03 13:32:10 crc kubenswrapper[5052]: I1203 13:32:10.673707 5052 scope.go:117] "RemoveContainer" containerID="4dbf4cc5b784c22fc4eecbfecde10f7c7824875bd119667c218a37a8cbc71d15" Dec 03 13:32:10 crc kubenswrapper[5052]: E1203 13:32:10.674397 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 13:32:24 crc kubenswrapper[5052]: I1203 13:32:24.673356 5052 scope.go:117] "RemoveContainer" containerID="4dbf4cc5b784c22fc4eecbfecde10f7c7824875bd119667c218a37a8cbc71d15" Dec 03 13:32:24 crc kubenswrapper[5052]: E1203 13:32:24.675512 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 13:32:39 crc kubenswrapper[5052]: I1203 13:32:39.677355 5052 scope.go:117] "RemoveContainer" containerID="4dbf4cc5b784c22fc4eecbfecde10f7c7824875bd119667c218a37a8cbc71d15" Dec 03 13:32:39 crc kubenswrapper[5052]: E1203 13:32:39.678786 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 13:32:49 crc kubenswrapper[5052]: I1203 13:32:49.012670 5052 scope.go:117] "RemoveContainer" containerID="2dd0532fc46c347cff5bc379a6f28c1b515400bdbd08af61a5b258613138c7ff" Dec 03 13:32:49 crc kubenswrapper[5052]: I1203 13:32:49.035399 5052 scope.go:117] "RemoveContainer" containerID="14b3a3fb2e82ee2d3772353eb2afdd3eec1e8b6a6efc3575f8cfcb36c98eb4da" Dec 03 13:32:49 crc kubenswrapper[5052]: I1203 13:32:49.069374 5052 scope.go:117] "RemoveContainer" containerID="ef77869d85af31266b483685c0df0c8353444fc238667ea89fed39c37d442c8e" Dec 03 13:32:49 crc kubenswrapper[5052]: I1203 13:32:49.101580 5052 scope.go:117] "RemoveContainer" containerID="ebbd9e155f51ffb015f7ba820bcc86ed1c54112a262104e4180e88e71517c408" Dec 03 13:32:54 crc kubenswrapper[5052]: I1203 13:32:54.672707 5052 scope.go:117] "RemoveContainer" containerID="4dbf4cc5b784c22fc4eecbfecde10f7c7824875bd119667c218a37a8cbc71d15" Dec 03 13:32:54 crc kubenswrapper[5052]: E1203 13:32:54.673566 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 13:33:08 crc kubenswrapper[5052]: I1203 13:33:08.673502 5052 scope.go:117] "RemoveContainer" containerID="4dbf4cc5b784c22fc4eecbfecde10f7c7824875bd119667c218a37a8cbc71d15" Dec 03 13:33:08 crc kubenswrapper[5052]: E1203 13:33:08.674348 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 13:33:20 crc kubenswrapper[5052]: I1203 13:33:20.673997 5052 scope.go:117] "RemoveContainer" containerID="4dbf4cc5b784c22fc4eecbfecde10f7c7824875bd119667c218a37a8cbc71d15" Dec 03 13:33:20 crc kubenswrapper[5052]: E1203 13:33:20.674981 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 13:33:33 crc kubenswrapper[5052]: I1203 13:33:33.674139 5052 scope.go:117] "RemoveContainer" containerID="4dbf4cc5b784c22fc4eecbfecde10f7c7824875bd119667c218a37a8cbc71d15" Dec 03 13:33:33 crc kubenswrapper[5052]: E1203 13:33:33.675160 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 13:33:48 crc kubenswrapper[5052]: I1203 13:33:48.672891 5052 scope.go:117] "RemoveContainer" containerID="4dbf4cc5b784c22fc4eecbfecde10f7c7824875bd119667c218a37a8cbc71d15" Dec 03 13:33:48 crc kubenswrapper[5052]: E1203 13:33:48.673757 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 13:33:49 crc kubenswrapper[5052]: I1203 13:33:49.208268 5052 scope.go:117] "RemoveContainer" containerID="1c1b136bf371a6589d221ca1ca0b4aceda61a8f8bd70b84409466c8903f6020a" Dec 03 13:34:02 crc kubenswrapper[5052]: I1203 13:34:02.674052 5052 scope.go:117] "RemoveContainer" containerID="4dbf4cc5b784c22fc4eecbfecde10f7c7824875bd119667c218a37a8cbc71d15" Dec 03 13:34:02 crc kubenswrapper[5052]: E1203 13:34:02.677666 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 13:34:17 crc kubenswrapper[5052]: I1203 13:34:17.673920 5052 scope.go:117] "RemoveContainer" containerID="4dbf4cc5b784c22fc4eecbfecde10f7c7824875bd119667c218a37a8cbc71d15" Dec 03 13:34:17 crc kubenswrapper[5052]: E1203 13:34:17.674770 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 13:34:28 crc kubenswrapper[5052]: I1203 13:34:28.673701 5052 scope.go:117] "RemoveContainer" containerID="4dbf4cc5b784c22fc4eecbfecde10f7c7824875bd119667c218a37a8cbc71d15" Dec 03 13:34:28 crc kubenswrapper[5052]: E1203 13:34:28.674579 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 13:34:36 crc kubenswrapper[5052]: I1203 13:34:36.339480 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openshift-marketplace/redhat-operators-v6rlr"] Dec 03 13:34:36 crc kubenswrapper[5052]: E1203 13:34:36.341726 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="559c277b-dd6a-4719-bfaa-ee68815faf85" containerName="extract-utilities" Dec 03 13:34:36 crc kubenswrapper[5052]: I1203 13:34:36.341843 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="559c277b-dd6a-4719-bfaa-ee68815faf85" containerName="extract-utilities" Dec 03 13:34:36 crc kubenswrapper[5052]: E1203 13:34:36.341962 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="559c277b-dd6a-4719-bfaa-ee68815faf85" containerName="registry-server" Dec 03 13:34:36 crc kubenswrapper[5052]: I1203 13:34:36.342046 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="559c277b-dd6a-4719-bfaa-ee68815faf85" containerName="registry-server" Dec 03 13:34:36 crc kubenswrapper[5052]: E1203 13:34:36.342138 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="559c277b-dd6a-4719-bfaa-ee68815faf85" containerName="extract-content" Dec 03 13:34:36 crc kubenswrapper[5052]: I1203 13:34:36.342223 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="559c277b-dd6a-4719-bfaa-ee68815faf85" containerName="extract-content" Dec 03 13:34:36 crc kubenswrapper[5052]: I1203 13:34:36.342439 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="559c277b-dd6a-4719-bfaa-ee68815faf85" containerName="registry-server" Dec 03 13:34:36 crc kubenswrapper[5052]: I1203 13:34:36.343676 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/redhat-operators-v6rlr" Dec 03 13:34:36 crc kubenswrapper[5052]: I1203 13:34:36.368446 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/redhat-operators-v6rlr"] Dec 03 13:34:36 crc kubenswrapper[5052]: I1203 13:34:36.388210 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c6bhd\" (UniqueName: \"kubernetes.io/projected/cf24d65b-a73d-4635-a0e6-9b1e69968040-kube-api-access-c6bhd\") pod \"redhat-operators-v6rlr\" (UID: \"cf24d65b-a73d-4635-a0e6-9b1e69968040\") " pod="openshift-marketplace/redhat-operators-v6rlr" Dec 03 13:34:36 crc kubenswrapper[5052]: I1203 13:34:36.388256 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/cf24d65b-a73d-4635-a0e6-9b1e69968040-catalog-content\") pod \"redhat-operators-v6rlr\" (UID: \"cf24d65b-a73d-4635-a0e6-9b1e69968040\") " pod="openshift-marketplace/redhat-operators-v6rlr" Dec 03 13:34:36 crc kubenswrapper[5052]: I1203 13:34:36.388279 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/cf24d65b-a73d-4635-a0e6-9b1e69968040-utilities\") pod \"redhat-operators-v6rlr\" (UID: \"cf24d65b-a73d-4635-a0e6-9b1e69968040\") " pod="openshift-marketplace/redhat-operators-v6rlr" Dec 03 13:34:36 crc kubenswrapper[5052]: I1203 13:34:36.489921 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-c6bhd\" (UniqueName: \"kubernetes.io/projected/cf24d65b-a73d-4635-a0e6-9b1e69968040-kube-api-access-c6bhd\") pod \"redhat-operators-v6rlr\" (UID: \"cf24d65b-a73d-4635-a0e6-9b1e69968040\") " pod="openshift-marketplace/redhat-operators-v6rlr" Dec 03 13:34:36 crc kubenswrapper[5052]: I1203 13:34:36.489994 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/cf24d65b-a73d-4635-a0e6-9b1e69968040-catalog-content\") pod \"redhat-operators-v6rlr\" (UID: \"cf24d65b-a73d-4635-a0e6-9b1e69968040\") " pod="openshift-marketplace/redhat-operators-v6rlr" Dec 03 13:34:36 crc kubenswrapper[5052]: I1203 13:34:36.490025 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/cf24d65b-a73d-4635-a0e6-9b1e69968040-utilities\") pod \"redhat-operators-v6rlr\" (UID: \"cf24d65b-a73d-4635-a0e6-9b1e69968040\") " pod="openshift-marketplace/redhat-operators-v6rlr" Dec 03 13:34:36 crc kubenswrapper[5052]: I1203 13:34:36.490694 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/cf24d65b-a73d-4635-a0e6-9b1e69968040-utilities\") pod \"redhat-operators-v6rlr\" (UID: \"cf24d65b-a73d-4635-a0e6-9b1e69968040\") " pod="openshift-marketplace/redhat-operators-v6rlr" Dec 03 13:34:36 crc kubenswrapper[5052]: I1203 13:34:36.490711 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/cf24d65b-a73d-4635-a0e6-9b1e69968040-catalog-content\") pod \"redhat-operators-v6rlr\" (UID: \"cf24d65b-a73d-4635-a0e6-9b1e69968040\") " pod="openshift-marketplace/redhat-operators-v6rlr" Dec 03 13:34:36 crc kubenswrapper[5052]: I1203 13:34:36.512056 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-c6bhd\" (UniqueName: \"kubernetes.io/projected/cf24d65b-a73d-4635-a0e6-9b1e69968040-kube-api-access-c6bhd\") pod \"redhat-operators-v6rlr\" (UID: \"cf24d65b-a73d-4635-a0e6-9b1e69968040\") " pod="openshift-marketplace/redhat-operators-v6rlr" Dec 03 13:34:36 crc kubenswrapper[5052]: I1203 13:34:36.679214 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/redhat-operators-v6rlr" Dec 03 13:34:37 crc kubenswrapper[5052]: I1203 13:34:37.153727 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/redhat-operators-v6rlr"] Dec 03 13:34:37 crc kubenswrapper[5052]: I1203 13:34:37.409206 5052 generic.go:334] "Generic (PLEG): container finished" podID="cf24d65b-a73d-4635-a0e6-9b1e69968040" containerID="f5c1ce0cf13eff6064ed96e0851b2f64d488f723bff44d7a4252d3935d67aab9" exitCode=0 Dec 03 13:34:37 crc kubenswrapper[5052]: I1203 13:34:37.409413 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-operators-v6rlr" event={"ID":"cf24d65b-a73d-4635-a0e6-9b1e69968040","Type":"ContainerDied","Data":"f5c1ce0cf13eff6064ed96e0851b2f64d488f723bff44d7a4252d3935d67aab9"} Dec 03 13:34:37 crc kubenswrapper[5052]: I1203 13:34:37.409675 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-operators-v6rlr" event={"ID":"cf24d65b-a73d-4635-a0e6-9b1e69968040","Type":"ContainerStarted","Data":"8c6f12f4bdd77913f2b1fcc9d0d1996b0f59b3aa2431913dbab7667818ab9b81"} Dec 03 13:34:38 crc kubenswrapper[5052]: I1203 13:34:38.421220 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-operators-v6rlr" event={"ID":"cf24d65b-a73d-4635-a0e6-9b1e69968040","Type":"ContainerStarted","Data":"c91042a93b3190018098c0dda26cc342313d3a61d7a5fb6b450536b5517e1701"} Dec 03 13:34:39 crc kubenswrapper[5052]: I1203 13:34:39.431189 5052 generic.go:334] "Generic (PLEG): container finished" podID="cf24d65b-a73d-4635-a0e6-9b1e69968040" containerID="c91042a93b3190018098c0dda26cc342313d3a61d7a5fb6b450536b5517e1701" exitCode=0 Dec 03 13:34:39 crc kubenswrapper[5052]: I1203 13:34:39.431245 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-operators-v6rlr" event={"ID":"cf24d65b-a73d-4635-a0e6-9b1e69968040","Type":"ContainerDied","Data":"c91042a93b3190018098c0dda26cc342313d3a61d7a5fb6b450536b5517e1701"} Dec 03 13:34:40 crc kubenswrapper[5052]: I1203 13:34:40.441634 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-operators-v6rlr" event={"ID":"cf24d65b-a73d-4635-a0e6-9b1e69968040","Type":"ContainerStarted","Data":"37e7e0fc067021b581379cb05bc636bc00906b44df9d9c4135e7ab1979d4e643"} Dec 03 13:34:40 crc kubenswrapper[5052]: I1203 13:34:40.464176 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-marketplace/redhat-operators-v6rlr" podStartSLOduration=1.9535980400000001 podStartE2EDuration="4.464145595s" podCreationTimestamp="2025-12-03 13:34:36 +0000 UTC" firstStartedPulling="2025-12-03 13:34:37.411825523 +0000 UTC m=+8068.000067287" lastFinishedPulling="2025-12-03 13:34:39.922373078 +0000 UTC m=+8070.510614842" observedRunningTime="2025-12-03 13:34:40.463023202 +0000 UTC m=+8071.051264966" watchObservedRunningTime="2025-12-03 13:34:40.464145595 +0000 UTC m=+8071.052387399" Dec 03 13:34:41 crc kubenswrapper[5052]: I1203 13:34:41.672924 5052 scope.go:117] "RemoveContainer" containerID="4dbf4cc5b784c22fc4eecbfecde10f7c7824875bd119667c218a37a8cbc71d15" Dec 03 13:34:41 crc kubenswrapper[5052]: E1203 13:34:41.673556 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 13:34:46 crc kubenswrapper[5052]: I1203 13:34:46.679333 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-marketplace/redhat-operators-v6rlr" Dec 03 13:34:46 crc kubenswrapper[5052]: I1203 13:34:46.679939 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openshift-marketplace/redhat-operators-v6rlr" Dec 03 13:34:46 crc kubenswrapper[5052]: I1203 13:34:46.719804 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-marketplace/redhat-operators-v6rlr" Dec 03 13:34:47 crc kubenswrapper[5052]: I1203 13:34:47.543787 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-marketplace/redhat-operators-v6rlr" Dec 03 13:34:47 crc kubenswrapper[5052]: I1203 13:34:47.592861 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openshift-marketplace/redhat-operators-v6rlr"] Dec 03 13:34:49 crc kubenswrapper[5052]: I1203 13:34:49.277174 5052 scope.go:117] "RemoveContainer" containerID="23495a521d6c51f0ce23bdc259ec38320a9821ccf40b84f6ce800d03dc181084" Dec 03 13:34:49 crc kubenswrapper[5052]: I1203 13:34:49.511585 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openshift-marketplace/redhat-operators-v6rlr" podUID="cf24d65b-a73d-4635-a0e6-9b1e69968040" containerName="registry-server" containerID="cri-o://37e7e0fc067021b581379cb05bc636bc00906b44df9d9c4135e7ab1979d4e643" gracePeriod=2 Dec 03 13:34:51 crc kubenswrapper[5052]: I1203 13:34:51.529314 5052 generic.go:334] "Generic (PLEG): container finished" podID="cf24d65b-a73d-4635-a0e6-9b1e69968040" containerID="37e7e0fc067021b581379cb05bc636bc00906b44df9d9c4135e7ab1979d4e643" exitCode=0 Dec 03 13:34:51 crc kubenswrapper[5052]: I1203 13:34:51.529469 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-operators-v6rlr" event={"ID":"cf24d65b-a73d-4635-a0e6-9b1e69968040","Type":"ContainerDied","Data":"37e7e0fc067021b581379cb05bc636bc00906b44df9d9c4135e7ab1979d4e643"} Dec 03 13:34:51 crc kubenswrapper[5052]: I1203 13:34:51.831561 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/redhat-operators-v6rlr" Dec 03 13:34:52 crc kubenswrapper[5052]: I1203 13:34:52.027886 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/cf24d65b-a73d-4635-a0e6-9b1e69968040-utilities\") pod \"cf24d65b-a73d-4635-a0e6-9b1e69968040\" (UID: \"cf24d65b-a73d-4635-a0e6-9b1e69968040\") " Dec 03 13:34:52 crc kubenswrapper[5052]: I1203 13:34:52.028220 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/cf24d65b-a73d-4635-a0e6-9b1e69968040-catalog-content\") pod \"cf24d65b-a73d-4635-a0e6-9b1e69968040\" (UID: \"cf24d65b-a73d-4635-a0e6-9b1e69968040\") " Dec 03 13:34:52 crc kubenswrapper[5052]: I1203 13:34:52.028299 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c6bhd\" (UniqueName: \"kubernetes.io/projected/cf24d65b-a73d-4635-a0e6-9b1e69968040-kube-api-access-c6bhd\") pod \"cf24d65b-a73d-4635-a0e6-9b1e69968040\" (UID: \"cf24d65b-a73d-4635-a0e6-9b1e69968040\") " Dec 03 13:34:52 crc kubenswrapper[5052]: I1203 13:34:52.028941 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf24d65b-a73d-4635-a0e6-9b1e69968040-utilities" (OuterVolumeSpecName: "utilities") pod "cf24d65b-a73d-4635-a0e6-9b1e69968040" (UID: "cf24d65b-a73d-4635-a0e6-9b1e69968040"). InnerVolumeSpecName "utilities". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 13:34:52 crc kubenswrapper[5052]: I1203 13:34:52.036271 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cf24d65b-a73d-4635-a0e6-9b1e69968040-kube-api-access-c6bhd" (OuterVolumeSpecName: "kube-api-access-c6bhd") pod "cf24d65b-a73d-4635-a0e6-9b1e69968040" (UID: "cf24d65b-a73d-4635-a0e6-9b1e69968040"). InnerVolumeSpecName "kube-api-access-c6bhd". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:34:52 crc kubenswrapper[5052]: I1203 13:34:52.129260 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf24d65b-a73d-4635-a0e6-9b1e69968040-catalog-content" (OuterVolumeSpecName: "catalog-content") pod "cf24d65b-a73d-4635-a0e6-9b1e69968040" (UID: "cf24d65b-a73d-4635-a0e6-9b1e69968040"). InnerVolumeSpecName "catalog-content". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 13:34:52 crc kubenswrapper[5052]: I1203 13:34:52.129982 5052 reconciler_common.go:293] "Volume detached for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/cf24d65b-a73d-4635-a0e6-9b1e69968040-catalog-content\") on node \"crc\" DevicePath \"\"" Dec 03 13:34:52 crc kubenswrapper[5052]: I1203 13:34:52.130032 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-c6bhd\" (UniqueName: \"kubernetes.io/projected/cf24d65b-a73d-4635-a0e6-9b1e69968040-kube-api-access-c6bhd\") on node \"crc\" DevicePath \"\"" Dec 03 13:34:52 crc kubenswrapper[5052]: I1203 13:34:52.130045 5052 reconciler_common.go:293] "Volume detached for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/cf24d65b-a73d-4635-a0e6-9b1e69968040-utilities\") on node \"crc\" DevicePath \"\"" Dec 03 13:34:52 crc kubenswrapper[5052]: I1203 13:34:52.539166 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-operators-v6rlr" event={"ID":"cf24d65b-a73d-4635-a0e6-9b1e69968040","Type":"ContainerDied","Data":"8c6f12f4bdd77913f2b1fcc9d0d1996b0f59b3aa2431913dbab7667818ab9b81"} Dec 03 13:34:52 crc kubenswrapper[5052]: I1203 13:34:52.539220 5052 scope.go:117] "RemoveContainer" containerID="37e7e0fc067021b581379cb05bc636bc00906b44df9d9c4135e7ab1979d4e643" Dec 03 13:34:52 crc kubenswrapper[5052]: I1203 13:34:52.539273 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/redhat-operators-v6rlr" Dec 03 13:34:52 crc kubenswrapper[5052]: I1203 13:34:52.561248 5052 scope.go:117] "RemoveContainer" containerID="c91042a93b3190018098c0dda26cc342313d3a61d7a5fb6b450536b5517e1701" Dec 03 13:34:52 crc kubenswrapper[5052]: I1203 13:34:52.586302 5052 scope.go:117] "RemoveContainer" containerID="f5c1ce0cf13eff6064ed96e0851b2f64d488f723bff44d7a4252d3935d67aab9" Dec 03 13:34:52 crc kubenswrapper[5052]: I1203 13:34:52.593098 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openshift-marketplace/redhat-operators-v6rlr"] Dec 03 13:34:52 crc kubenswrapper[5052]: I1203 13:34:52.599340 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openshift-marketplace/redhat-operators-v6rlr"] Dec 03 13:34:53 crc kubenswrapper[5052]: I1203 13:34:53.683337 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cf24d65b-a73d-4635-a0e6-9b1e69968040" path="/var/lib/kubelet/pods/cf24d65b-a73d-4635-a0e6-9b1e69968040/volumes" Dec 03 13:34:54 crc kubenswrapper[5052]: I1203 13:34:54.672803 5052 scope.go:117] "RemoveContainer" containerID="4dbf4cc5b784c22fc4eecbfecde10f7c7824875bd119667c218a37a8cbc71d15" Dec 03 13:34:54 crc kubenswrapper[5052]: E1203 13:34:54.673398 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 13:35:09 crc kubenswrapper[5052]: I1203 13:35:09.672735 5052 scope.go:117] "RemoveContainer" containerID="4dbf4cc5b784c22fc4eecbfecde10f7c7824875bd119667c218a37a8cbc71d15" Dec 03 13:35:10 crc kubenswrapper[5052]: I1203 13:35:10.713369 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" event={"ID":"10c237bc-72e7-4da3-9f2c-b1688395bf6e","Type":"ContainerStarted","Data":"c52249d2a9f574a7f765a482d66dd438e6d6ff79a446ffe0920be2784f98e6b8"} Dec 03 13:37:29 crc kubenswrapper[5052]: I1203 13:37:29.379906 5052 patch_prober.go:28] interesting pod/machine-config-daemon-74bj6 container/machine-config-daemon namespace/openshift-machine-config-operator: Liveness probe status=failure output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" start-of-body= Dec 03 13:37:29 crc kubenswrapper[5052]: I1203 13:37:29.380400 5052 prober.go:107] "Probe failed" probeType="Liveness" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" containerName="machine-config-daemon" probeResult="failure" output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" Dec 03 13:37:33 crc kubenswrapper[5052]: I1203 13:37:33.224374 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openshift-marketplace/community-operators-pxcms"] Dec 03 13:37:33 crc kubenswrapper[5052]: E1203 13:37:33.225135 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="cf24d65b-a73d-4635-a0e6-9b1e69968040" containerName="extract-utilities" Dec 03 13:37:33 crc kubenswrapper[5052]: I1203 13:37:33.225152 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="cf24d65b-a73d-4635-a0e6-9b1e69968040" containerName="extract-utilities" Dec 03 13:37:33 crc kubenswrapper[5052]: E1203 13:37:33.225162 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="cf24d65b-a73d-4635-a0e6-9b1e69968040" containerName="extract-content" Dec 03 13:37:33 crc kubenswrapper[5052]: I1203 13:37:33.225169 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="cf24d65b-a73d-4635-a0e6-9b1e69968040" containerName="extract-content" Dec 03 13:37:33 crc kubenswrapper[5052]: E1203 13:37:33.225179 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="cf24d65b-a73d-4635-a0e6-9b1e69968040" containerName="registry-server" Dec 03 13:37:33 crc kubenswrapper[5052]: I1203 13:37:33.225187 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="cf24d65b-a73d-4635-a0e6-9b1e69968040" containerName="registry-server" Dec 03 13:37:33 crc kubenswrapper[5052]: I1203 13:37:33.225367 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="cf24d65b-a73d-4635-a0e6-9b1e69968040" containerName="registry-server" Dec 03 13:37:33 crc kubenswrapper[5052]: I1203 13:37:33.226716 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/community-operators-pxcms" Dec 03 13:37:33 crc kubenswrapper[5052]: I1203 13:37:33.275669 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/community-operators-pxcms"] Dec 03 13:37:33 crc kubenswrapper[5052]: I1203 13:37:33.353480 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/1232c517-147e-407a-9167-afab87e26c94-utilities\") pod \"community-operators-pxcms\" (UID: \"1232c517-147e-407a-9167-afab87e26c94\") " pod="openshift-marketplace/community-operators-pxcms" Dec 03 13:37:33 crc kubenswrapper[5052]: I1203 13:37:33.353719 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/1232c517-147e-407a-9167-afab87e26c94-catalog-content\") pod \"community-operators-pxcms\" (UID: \"1232c517-147e-407a-9167-afab87e26c94\") " pod="openshift-marketplace/community-operators-pxcms" Dec 03 13:37:33 crc kubenswrapper[5052]: I1203 13:37:33.353906 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k9zl6\" (UniqueName: \"kubernetes.io/projected/1232c517-147e-407a-9167-afab87e26c94-kube-api-access-k9zl6\") pod \"community-operators-pxcms\" (UID: \"1232c517-147e-407a-9167-afab87e26c94\") " pod="openshift-marketplace/community-operators-pxcms" Dec 03 13:37:33 crc kubenswrapper[5052]: I1203 13:37:33.455035 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/1232c517-147e-407a-9167-afab87e26c94-utilities\") pod \"community-operators-pxcms\" (UID: \"1232c517-147e-407a-9167-afab87e26c94\") " pod="openshift-marketplace/community-operators-pxcms" Dec 03 13:37:33 crc kubenswrapper[5052]: I1203 13:37:33.455136 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/1232c517-147e-407a-9167-afab87e26c94-catalog-content\") pod \"community-operators-pxcms\" (UID: \"1232c517-147e-407a-9167-afab87e26c94\") " pod="openshift-marketplace/community-operators-pxcms" Dec 03 13:37:33 crc kubenswrapper[5052]: I1203 13:37:33.455201 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-k9zl6\" (UniqueName: \"kubernetes.io/projected/1232c517-147e-407a-9167-afab87e26c94-kube-api-access-k9zl6\") pod \"community-operators-pxcms\" (UID: \"1232c517-147e-407a-9167-afab87e26c94\") " pod="openshift-marketplace/community-operators-pxcms" Dec 03 13:37:33 crc kubenswrapper[5052]: I1203 13:37:33.455651 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/1232c517-147e-407a-9167-afab87e26c94-utilities\") pod \"community-operators-pxcms\" (UID: \"1232c517-147e-407a-9167-afab87e26c94\") " pod="openshift-marketplace/community-operators-pxcms" Dec 03 13:37:33 crc kubenswrapper[5052]: I1203 13:37:33.455783 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/1232c517-147e-407a-9167-afab87e26c94-catalog-content\") pod \"community-operators-pxcms\" (UID: \"1232c517-147e-407a-9167-afab87e26c94\") " pod="openshift-marketplace/community-operators-pxcms" Dec 03 13:37:33 crc kubenswrapper[5052]: I1203 13:37:33.476243 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-k9zl6\" (UniqueName: \"kubernetes.io/projected/1232c517-147e-407a-9167-afab87e26c94-kube-api-access-k9zl6\") pod \"community-operators-pxcms\" (UID: \"1232c517-147e-407a-9167-afab87e26c94\") " pod="openshift-marketplace/community-operators-pxcms" Dec 03 13:37:33 crc kubenswrapper[5052]: I1203 13:37:33.546348 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/community-operators-pxcms" Dec 03 13:37:33 crc kubenswrapper[5052]: I1203 13:37:33.820982 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/community-operators-pxcms"] Dec 03 13:37:34 crc kubenswrapper[5052]: I1203 13:37:34.774749 5052 generic.go:334] "Generic (PLEG): container finished" podID="1232c517-147e-407a-9167-afab87e26c94" containerID="a1cb9aebaab390eadb7705903d5724816e6eb71eeee747aee213f71287b44e41" exitCode=0 Dec 03 13:37:34 crc kubenswrapper[5052]: I1203 13:37:34.775123 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/community-operators-pxcms" event={"ID":"1232c517-147e-407a-9167-afab87e26c94","Type":"ContainerDied","Data":"a1cb9aebaab390eadb7705903d5724816e6eb71eeee747aee213f71287b44e41"} Dec 03 13:37:34 crc kubenswrapper[5052]: I1203 13:37:34.775161 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/community-operators-pxcms" event={"ID":"1232c517-147e-407a-9167-afab87e26c94","Type":"ContainerStarted","Data":"461a8f0e311bac3146ff1d498e0690789a3174698117b493f82a77aa1d773937"} Dec 03 13:37:34 crc kubenswrapper[5052]: I1203 13:37:34.778022 5052 provider.go:102] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Dec 03 13:37:35 crc kubenswrapper[5052]: I1203 13:37:35.785970 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/community-operators-pxcms" event={"ID":"1232c517-147e-407a-9167-afab87e26c94","Type":"ContainerStarted","Data":"386df539d13728f267e470445fc34e5a40c791f634d265137a2af1298b15d6af"} Dec 03 13:37:36 crc kubenswrapper[5052]: I1203 13:37:36.794842 5052 generic.go:334] "Generic (PLEG): container finished" podID="1232c517-147e-407a-9167-afab87e26c94" containerID="386df539d13728f267e470445fc34e5a40c791f634d265137a2af1298b15d6af" exitCode=0 Dec 03 13:37:36 crc kubenswrapper[5052]: I1203 13:37:36.794947 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/community-operators-pxcms" event={"ID":"1232c517-147e-407a-9167-afab87e26c94","Type":"ContainerDied","Data":"386df539d13728f267e470445fc34e5a40c791f634d265137a2af1298b15d6af"} Dec 03 13:37:40 crc kubenswrapper[5052]: I1203 13:37:40.831050 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/community-operators-pxcms" event={"ID":"1232c517-147e-407a-9167-afab87e26c94","Type":"ContainerStarted","Data":"7ff05962da079ce5adb75be8f3ee108dfbb61569bf71ae57720ca53e7a53431b"} Dec 03 13:37:40 crc kubenswrapper[5052]: I1203 13:37:40.850203 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-marketplace/community-operators-pxcms" podStartSLOduration=2.920005756 podStartE2EDuration="7.850183157s" podCreationTimestamp="2025-12-03 13:37:33 +0000 UTC" firstStartedPulling="2025-12-03 13:37:34.777636416 +0000 UTC m=+8245.365878190" lastFinishedPulling="2025-12-03 13:37:39.707813827 +0000 UTC m=+8250.296055591" observedRunningTime="2025-12-03 13:37:40.846777699 +0000 UTC m=+8251.435019483" watchObservedRunningTime="2025-12-03 13:37:40.850183157 +0000 UTC m=+8251.438424921" Dec 03 13:37:43 crc kubenswrapper[5052]: I1203 13:37:43.546778 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openshift-marketplace/community-operators-pxcms" Dec 03 13:37:43 crc kubenswrapper[5052]: I1203 13:37:43.547154 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-marketplace/community-operators-pxcms" Dec 03 13:37:43 crc kubenswrapper[5052]: I1203 13:37:43.590574 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-marketplace/community-operators-pxcms" Dec 03 13:37:53 crc kubenswrapper[5052]: I1203 13:37:53.593192 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-marketplace/community-operators-pxcms" Dec 03 13:37:53 crc kubenswrapper[5052]: I1203 13:37:53.666969 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openshift-marketplace/community-operators-pxcms"] Dec 03 13:37:53 crc kubenswrapper[5052]: I1203 13:37:53.925210 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openshift-marketplace/community-operators-pxcms" podUID="1232c517-147e-407a-9167-afab87e26c94" containerName="registry-server" containerID="cri-o://7ff05962da079ce5adb75be8f3ee108dfbb61569bf71ae57720ca53e7a53431b" gracePeriod=2 Dec 03 13:37:54 crc kubenswrapper[5052]: I1203 13:37:54.312119 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/community-operators-pxcms" Dec 03 13:37:54 crc kubenswrapper[5052]: I1203 13:37:54.374592 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/1232c517-147e-407a-9167-afab87e26c94-catalog-content\") pod \"1232c517-147e-407a-9167-afab87e26c94\" (UID: \"1232c517-147e-407a-9167-afab87e26c94\") " Dec 03 13:37:54 crc kubenswrapper[5052]: I1203 13:37:54.374944 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/1232c517-147e-407a-9167-afab87e26c94-utilities\") pod \"1232c517-147e-407a-9167-afab87e26c94\" (UID: \"1232c517-147e-407a-9167-afab87e26c94\") " Dec 03 13:37:54 crc kubenswrapper[5052]: I1203 13:37:54.375088 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k9zl6\" (UniqueName: \"kubernetes.io/projected/1232c517-147e-407a-9167-afab87e26c94-kube-api-access-k9zl6\") pod \"1232c517-147e-407a-9167-afab87e26c94\" (UID: \"1232c517-147e-407a-9167-afab87e26c94\") " Dec 03 13:37:54 crc kubenswrapper[5052]: I1203 13:37:54.375996 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1232c517-147e-407a-9167-afab87e26c94-utilities" (OuterVolumeSpecName: "utilities") pod "1232c517-147e-407a-9167-afab87e26c94" (UID: "1232c517-147e-407a-9167-afab87e26c94"). InnerVolumeSpecName "utilities". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 13:37:54 crc kubenswrapper[5052]: I1203 13:37:54.384133 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1232c517-147e-407a-9167-afab87e26c94-kube-api-access-k9zl6" (OuterVolumeSpecName: "kube-api-access-k9zl6") pod "1232c517-147e-407a-9167-afab87e26c94" (UID: "1232c517-147e-407a-9167-afab87e26c94"). InnerVolumeSpecName "kube-api-access-k9zl6". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:37:54 crc kubenswrapper[5052]: I1203 13:37:54.430923 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1232c517-147e-407a-9167-afab87e26c94-catalog-content" (OuterVolumeSpecName: "catalog-content") pod "1232c517-147e-407a-9167-afab87e26c94" (UID: "1232c517-147e-407a-9167-afab87e26c94"). InnerVolumeSpecName "catalog-content". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 13:37:54 crc kubenswrapper[5052]: I1203 13:37:54.477367 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-k9zl6\" (UniqueName: \"kubernetes.io/projected/1232c517-147e-407a-9167-afab87e26c94-kube-api-access-k9zl6\") on node \"crc\" DevicePath \"\"" Dec 03 13:37:54 crc kubenswrapper[5052]: I1203 13:37:54.477421 5052 reconciler_common.go:293] "Volume detached for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/1232c517-147e-407a-9167-afab87e26c94-catalog-content\") on node \"crc\" DevicePath \"\"" Dec 03 13:37:54 crc kubenswrapper[5052]: I1203 13:37:54.477433 5052 reconciler_common.go:293] "Volume detached for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/1232c517-147e-407a-9167-afab87e26c94-utilities\") on node \"crc\" DevicePath \"\"" Dec 03 13:37:54 crc kubenswrapper[5052]: I1203 13:37:54.935989 5052 generic.go:334] "Generic (PLEG): container finished" podID="1232c517-147e-407a-9167-afab87e26c94" containerID="7ff05962da079ce5adb75be8f3ee108dfbb61569bf71ae57720ca53e7a53431b" exitCode=0 Dec 03 13:37:54 crc kubenswrapper[5052]: I1203 13:37:54.936185 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/community-operators-pxcms" event={"ID":"1232c517-147e-407a-9167-afab87e26c94","Type":"ContainerDied","Data":"7ff05962da079ce5adb75be8f3ee108dfbb61569bf71ae57720ca53e7a53431b"} Dec 03 13:37:54 crc kubenswrapper[5052]: I1203 13:37:54.937812 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/community-operators-pxcms" event={"ID":"1232c517-147e-407a-9167-afab87e26c94","Type":"ContainerDied","Data":"461a8f0e311bac3146ff1d498e0690789a3174698117b493f82a77aa1d773937"} Dec 03 13:37:54 crc kubenswrapper[5052]: I1203 13:37:54.937860 5052 scope.go:117] "RemoveContainer" containerID="7ff05962da079ce5adb75be8f3ee108dfbb61569bf71ae57720ca53e7a53431b" Dec 03 13:37:54 crc kubenswrapper[5052]: I1203 13:37:54.936312 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/community-operators-pxcms" Dec 03 13:37:54 crc kubenswrapper[5052]: I1203 13:37:54.960594 5052 scope.go:117] "RemoveContainer" containerID="386df539d13728f267e470445fc34e5a40c791f634d265137a2af1298b15d6af" Dec 03 13:37:54 crc kubenswrapper[5052]: I1203 13:37:54.977957 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openshift-marketplace/community-operators-pxcms"] Dec 03 13:37:54 crc kubenswrapper[5052]: I1203 13:37:54.981205 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openshift-marketplace/community-operators-pxcms"] Dec 03 13:37:55 crc kubenswrapper[5052]: I1203 13:37:55.019554 5052 scope.go:117] "RemoveContainer" containerID="a1cb9aebaab390eadb7705903d5724816e6eb71eeee747aee213f71287b44e41" Dec 03 13:37:55 crc kubenswrapper[5052]: I1203 13:37:55.045295 5052 scope.go:117] "RemoveContainer" containerID="7ff05962da079ce5adb75be8f3ee108dfbb61569bf71ae57720ca53e7a53431b" Dec 03 13:37:55 crc kubenswrapper[5052]: E1203 13:37:55.045816 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7ff05962da079ce5adb75be8f3ee108dfbb61569bf71ae57720ca53e7a53431b\": container with ID starting with 7ff05962da079ce5adb75be8f3ee108dfbb61569bf71ae57720ca53e7a53431b not found: ID does not exist" containerID="7ff05962da079ce5adb75be8f3ee108dfbb61569bf71ae57720ca53e7a53431b" Dec 03 13:37:55 crc kubenswrapper[5052]: I1203 13:37:55.045855 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7ff05962da079ce5adb75be8f3ee108dfbb61569bf71ae57720ca53e7a53431b"} err="failed to get container status \"7ff05962da079ce5adb75be8f3ee108dfbb61569bf71ae57720ca53e7a53431b\": rpc error: code = NotFound desc = could not find container \"7ff05962da079ce5adb75be8f3ee108dfbb61569bf71ae57720ca53e7a53431b\": container with ID starting with 7ff05962da079ce5adb75be8f3ee108dfbb61569bf71ae57720ca53e7a53431b not found: ID does not exist" Dec 03 13:37:55 crc kubenswrapper[5052]: I1203 13:37:55.045883 5052 scope.go:117] "RemoveContainer" containerID="386df539d13728f267e470445fc34e5a40c791f634d265137a2af1298b15d6af" Dec 03 13:37:55 crc kubenswrapper[5052]: E1203 13:37:55.046134 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"386df539d13728f267e470445fc34e5a40c791f634d265137a2af1298b15d6af\": container with ID starting with 386df539d13728f267e470445fc34e5a40c791f634d265137a2af1298b15d6af not found: ID does not exist" containerID="386df539d13728f267e470445fc34e5a40c791f634d265137a2af1298b15d6af" Dec 03 13:37:55 crc kubenswrapper[5052]: I1203 13:37:55.046161 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"386df539d13728f267e470445fc34e5a40c791f634d265137a2af1298b15d6af"} err="failed to get container status \"386df539d13728f267e470445fc34e5a40c791f634d265137a2af1298b15d6af\": rpc error: code = NotFound desc = could not find container \"386df539d13728f267e470445fc34e5a40c791f634d265137a2af1298b15d6af\": container with ID starting with 386df539d13728f267e470445fc34e5a40c791f634d265137a2af1298b15d6af not found: ID does not exist" Dec 03 13:37:55 crc kubenswrapper[5052]: I1203 13:37:55.046177 5052 scope.go:117] "RemoveContainer" containerID="a1cb9aebaab390eadb7705903d5724816e6eb71eeee747aee213f71287b44e41" Dec 03 13:37:55 crc kubenswrapper[5052]: E1203 13:37:55.046461 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a1cb9aebaab390eadb7705903d5724816e6eb71eeee747aee213f71287b44e41\": container with ID starting with a1cb9aebaab390eadb7705903d5724816e6eb71eeee747aee213f71287b44e41 not found: ID does not exist" containerID="a1cb9aebaab390eadb7705903d5724816e6eb71eeee747aee213f71287b44e41" Dec 03 13:37:55 crc kubenswrapper[5052]: I1203 13:37:55.046484 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a1cb9aebaab390eadb7705903d5724816e6eb71eeee747aee213f71287b44e41"} err="failed to get container status \"a1cb9aebaab390eadb7705903d5724816e6eb71eeee747aee213f71287b44e41\": rpc error: code = NotFound desc = could not find container \"a1cb9aebaab390eadb7705903d5724816e6eb71eeee747aee213f71287b44e41\": container with ID starting with a1cb9aebaab390eadb7705903d5724816e6eb71eeee747aee213f71287b44e41 not found: ID does not exist" Dec 03 13:37:55 crc kubenswrapper[5052]: I1203 13:37:55.682380 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1232c517-147e-407a-9167-afab87e26c94" path="/var/lib/kubelet/pods/1232c517-147e-407a-9167-afab87e26c94/volumes" Dec 03 13:37:59 crc kubenswrapper[5052]: I1203 13:37:59.379983 5052 patch_prober.go:28] interesting pod/machine-config-daemon-74bj6 container/machine-config-daemon namespace/openshift-machine-config-operator: Liveness probe status=failure output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" start-of-body= Dec 03 13:37:59 crc kubenswrapper[5052]: I1203 13:37:59.380645 5052 prober.go:107] "Probe failed" probeType="Liveness" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" containerName="machine-config-daemon" probeResult="failure" output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" Dec 03 13:38:00 crc kubenswrapper[5052]: I1203 13:38:00.052864 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["keystone-kuttl-tests/keystone-4828-account-create-update-wn9s4"] Dec 03 13:38:00 crc kubenswrapper[5052]: I1203 13:38:00.058877 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["keystone-kuttl-tests/keystone-db-create-q78p4"] Dec 03 13:38:00 crc kubenswrapper[5052]: I1203 13:38:00.064407 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["keystone-kuttl-tests/keystone-4828-account-create-update-wn9s4"] Dec 03 13:38:00 crc kubenswrapper[5052]: I1203 13:38:00.069427 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["keystone-kuttl-tests/keystone-db-create-q78p4"] Dec 03 13:38:01 crc kubenswrapper[5052]: I1203 13:38:01.681642 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6bc0cf1a-1224-48d1-b280-b6d0d612caa2" path="/var/lib/kubelet/pods/6bc0cf1a-1224-48d1-b280-b6d0d612caa2/volumes" Dec 03 13:38:01 crc kubenswrapper[5052]: I1203 13:38:01.682555 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a4ff3963-8b4c-4364-9aaf-3dfdc8434921" path="/var/lib/kubelet/pods/a4ff3963-8b4c-4364-9aaf-3dfdc8434921/volumes" Dec 03 13:38:11 crc kubenswrapper[5052]: I1203 13:38:11.045056 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["keystone-kuttl-tests/keystone-db-sync-8tgzj"] Dec 03 13:38:11 crc kubenswrapper[5052]: I1203 13:38:11.056925 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["keystone-kuttl-tests/keystone-db-sync-8tgzj"] Dec 03 13:38:11 crc kubenswrapper[5052]: I1203 13:38:11.682760 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="420a4d60-e580-4289-9a33-527b9203fca1" path="/var/lib/kubelet/pods/420a4d60-e580-4289-9a33-527b9203fca1/volumes" Dec 03 13:38:18 crc kubenswrapper[5052]: I1203 13:38:18.024851 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["keystone-kuttl-tests/keystone-bootstrap-gvh6k"] Dec 03 13:38:18 crc kubenswrapper[5052]: I1203 13:38:18.031649 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["keystone-kuttl-tests/keystone-bootstrap-gvh6k"] Dec 03 13:38:19 crc kubenswrapper[5052]: I1203 13:38:19.682815 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="57b69c1e-a382-47ca-9ba1-e5a0f5fc1bb8" path="/var/lib/kubelet/pods/57b69c1e-a382-47ca-9ba1-e5a0f5fc1bb8/volumes" Dec 03 13:38:29 crc kubenswrapper[5052]: I1203 13:38:29.379597 5052 patch_prober.go:28] interesting pod/machine-config-daemon-74bj6 container/machine-config-daemon namespace/openshift-machine-config-operator: Liveness probe status=failure output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" start-of-body= Dec 03 13:38:29 crc kubenswrapper[5052]: I1203 13:38:29.381459 5052 prober.go:107] "Probe failed" probeType="Liveness" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" containerName="machine-config-daemon" probeResult="failure" output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" Dec 03 13:38:29 crc kubenswrapper[5052]: I1203 13:38:29.381812 5052 kubelet.go:2542] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" Dec 03 13:38:29 crc kubenswrapper[5052]: I1203 13:38:29.382561 5052 kuberuntime_manager.go:1027] "Message for Container of pod" containerName="machine-config-daemon" containerStatusID={"Type":"cri-o","ID":"c52249d2a9f574a7f765a482d66dd438e6d6ff79a446ffe0920be2784f98e6b8"} pod="openshift-machine-config-operator/machine-config-daemon-74bj6" containerMessage="Container machine-config-daemon failed liveness probe, will be restarted" Dec 03 13:38:29 crc kubenswrapper[5052]: I1203 13:38:29.382695 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" containerName="machine-config-daemon" containerID="cri-o://c52249d2a9f574a7f765a482d66dd438e6d6ff79a446ffe0920be2784f98e6b8" gracePeriod=600 Dec 03 13:38:30 crc kubenswrapper[5052]: I1203 13:38:30.219758 5052 generic.go:334] "Generic (PLEG): container finished" podID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" containerID="c52249d2a9f574a7f765a482d66dd438e6d6ff79a446ffe0920be2784f98e6b8" exitCode=0 Dec 03 13:38:30 crc kubenswrapper[5052]: I1203 13:38:30.220141 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" event={"ID":"10c237bc-72e7-4da3-9f2c-b1688395bf6e","Type":"ContainerDied","Data":"c52249d2a9f574a7f765a482d66dd438e6d6ff79a446ffe0920be2784f98e6b8"} Dec 03 13:38:30 crc kubenswrapper[5052]: I1203 13:38:30.220177 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" event={"ID":"10c237bc-72e7-4da3-9f2c-b1688395bf6e","Type":"ContainerStarted","Data":"2fbd293d6edef0fa059e91c119312e77b262534107c322feb7b59dbea3464526"} Dec 03 13:38:30 crc kubenswrapper[5052]: I1203 13:38:30.220196 5052 scope.go:117] "RemoveContainer" containerID="4dbf4cc5b784c22fc4eecbfecde10f7c7824875bd119667c218a37a8cbc71d15" Dec 03 13:38:49 crc kubenswrapper[5052]: I1203 13:38:49.407493 5052 scope.go:117] "RemoveContainer" containerID="8f609298a2f431078988cc66a6f6635d4d096a14318a691ecacb9d76ebb0e7a4" Dec 03 13:38:49 crc kubenswrapper[5052]: I1203 13:38:49.443497 5052 scope.go:117] "RemoveContainer" containerID="486ef892c9c6611d7d60b6d2028a0cf4f6a796e746a75ced89976f69fbf64ec0" Dec 03 13:38:49 crc kubenswrapper[5052]: I1203 13:38:49.473542 5052 scope.go:117] "RemoveContainer" containerID="e7266642c6bd413f33aeaa06fe477903708f8f8e1b5f82a2dfdfef551cafbf10" Dec 03 13:38:49 crc kubenswrapper[5052]: I1203 13:38:49.507109 5052 scope.go:117] "RemoveContainer" containerID="bf140143123c5c502f6c9cbf40b8b2a7bf5df0f0440f316c7f197f181d943498" Dec 03 13:38:58 crc kubenswrapper[5052]: I1203 13:38:58.091961 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["keystone-kuttl-tests/openstackclient"] Dec 03 13:38:58 crc kubenswrapper[5052]: I1203 13:38:58.092831 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="keystone-kuttl-tests/openstackclient" podUID="973f3dde-e424-4de8-a9fa-3c267647e3de" containerName="openstackclient" containerID="cri-o://bc4acb938c8313fd21fbff043e2dc683926b4d15a3a2fe645f80ac6ea13382f5" gracePeriod=30 Dec 03 13:38:58 crc kubenswrapper[5052]: I1203 13:38:58.427340 5052 generic.go:334] "Generic (PLEG): container finished" podID="973f3dde-e424-4de8-a9fa-3c267647e3de" containerID="bc4acb938c8313fd21fbff043e2dc683926b4d15a3a2fe645f80ac6ea13382f5" exitCode=143 Dec 03 13:38:58 crc kubenswrapper[5052]: I1203 13:38:58.427427 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="keystone-kuttl-tests/openstackclient" event={"ID":"973f3dde-e424-4de8-a9fa-3c267647e3de","Type":"ContainerDied","Data":"bc4acb938c8313fd21fbff043e2dc683926b4d15a3a2fe645f80ac6ea13382f5"} Dec 03 13:38:58 crc kubenswrapper[5052]: I1203 13:38:58.518034 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="keystone-kuttl-tests/openstackclient" Dec 03 13:38:58 crc kubenswrapper[5052]: I1203 13:38:58.671476 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-phhqb\" (UniqueName: \"kubernetes.io/projected/973f3dde-e424-4de8-a9fa-3c267647e3de-kube-api-access-phhqb\") pod \"973f3dde-e424-4de8-a9fa-3c267647e3de\" (UID: \"973f3dde-e424-4de8-a9fa-3c267647e3de\") " Dec 03 13:38:58 crc kubenswrapper[5052]: I1203 13:38:58.671858 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"openstack-config\" (UniqueName: \"kubernetes.io/configmap/973f3dde-e424-4de8-a9fa-3c267647e3de-openstack-config\") pod \"973f3dde-e424-4de8-a9fa-3c267647e3de\" (UID: \"973f3dde-e424-4de8-a9fa-3c267647e3de\") " Dec 03 13:38:58 crc kubenswrapper[5052]: I1203 13:38:58.671929 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"openstack-config-secret\" (UniqueName: \"kubernetes.io/secret/973f3dde-e424-4de8-a9fa-3c267647e3de-openstack-config-secret\") pod \"973f3dde-e424-4de8-a9fa-3c267647e3de\" (UID: \"973f3dde-e424-4de8-a9fa-3c267647e3de\") " Dec 03 13:38:58 crc kubenswrapper[5052]: I1203 13:38:58.678463 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/973f3dde-e424-4de8-a9fa-3c267647e3de-kube-api-access-phhqb" (OuterVolumeSpecName: "kube-api-access-phhqb") pod "973f3dde-e424-4de8-a9fa-3c267647e3de" (UID: "973f3dde-e424-4de8-a9fa-3c267647e3de"). InnerVolumeSpecName "kube-api-access-phhqb". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:38:58 crc kubenswrapper[5052]: I1203 13:38:58.691853 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/973f3dde-e424-4de8-a9fa-3c267647e3de-openstack-config" (OuterVolumeSpecName: "openstack-config") pod "973f3dde-e424-4de8-a9fa-3c267647e3de" (UID: "973f3dde-e424-4de8-a9fa-3c267647e3de"). InnerVolumeSpecName "openstack-config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:38:58 crc kubenswrapper[5052]: I1203 13:38:58.694783 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/973f3dde-e424-4de8-a9fa-3c267647e3de-openstack-config-secret" (OuterVolumeSpecName: "openstack-config-secret") pod "973f3dde-e424-4de8-a9fa-3c267647e3de" (UID: "973f3dde-e424-4de8-a9fa-3c267647e3de"). InnerVolumeSpecName "openstack-config-secret". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:38:58 crc kubenswrapper[5052]: I1203 13:38:58.773694 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-phhqb\" (UniqueName: \"kubernetes.io/projected/973f3dde-e424-4de8-a9fa-3c267647e3de-kube-api-access-phhqb\") on node \"crc\" DevicePath \"\"" Dec 03 13:38:58 crc kubenswrapper[5052]: I1203 13:38:58.773738 5052 reconciler_common.go:293] "Volume detached for volume \"openstack-config\" (UniqueName: \"kubernetes.io/configmap/973f3dde-e424-4de8-a9fa-3c267647e3de-openstack-config\") on node \"crc\" DevicePath \"\"" Dec 03 13:38:58 crc kubenswrapper[5052]: I1203 13:38:58.773748 5052 reconciler_common.go:293] "Volume detached for volume \"openstack-config-secret\" (UniqueName: \"kubernetes.io/secret/973f3dde-e424-4de8-a9fa-3c267647e3de-openstack-config-secret\") on node \"crc\" DevicePath \"\"" Dec 03 13:38:59 crc kubenswrapper[5052]: I1203 13:38:59.436737 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="keystone-kuttl-tests/openstackclient" event={"ID":"973f3dde-e424-4de8-a9fa-3c267647e3de","Type":"ContainerDied","Data":"40c85982bff78eaa7774e362ba5b2f3ce542ec20a73c9b0801600e41a722a74d"} Dec 03 13:38:59 crc kubenswrapper[5052]: I1203 13:38:59.436795 5052 scope.go:117] "RemoveContainer" containerID="bc4acb938c8313fd21fbff043e2dc683926b4d15a3a2fe645f80ac6ea13382f5" Dec 03 13:38:59 crc kubenswrapper[5052]: I1203 13:38:59.436842 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="keystone-kuttl-tests/openstackclient" Dec 03 13:38:59 crc kubenswrapper[5052]: I1203 13:38:59.470309 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["keystone-kuttl-tests/openstackclient"] Dec 03 13:38:59 crc kubenswrapper[5052]: I1203 13:38:59.476822 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["keystone-kuttl-tests/openstackclient"] Dec 03 13:38:59 crc kubenswrapper[5052]: I1203 13:38:59.650641 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["keystone-kuttl-tests/keystone-6c857bcbcd-4n6lw"] Dec 03 13:38:59 crc kubenswrapper[5052]: I1203 13:38:59.650905 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="keystone-kuttl-tests/keystone-6c857bcbcd-4n6lw" podUID="e7224fc6-4050-43bd-95ce-b6257be9ed06" containerName="keystone-api" containerID="cri-o://5343e1050e418b924cbc8dd6d94de00ba87a38833e47b7c24b33ef15a3be59b4" gracePeriod=30 Dec 03 13:38:59 crc kubenswrapper[5052]: I1203 13:38:59.671405 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["keystone-kuttl-tests/keystone4828-account-delete-nvh2w"] Dec 03 13:38:59 crc kubenswrapper[5052]: E1203 13:38:59.676482 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="1232c517-147e-407a-9167-afab87e26c94" containerName="extract-utilities" Dec 03 13:38:59 crc kubenswrapper[5052]: I1203 13:38:59.676534 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="1232c517-147e-407a-9167-afab87e26c94" containerName="extract-utilities" Dec 03 13:38:59 crc kubenswrapper[5052]: E1203 13:38:59.676550 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="973f3dde-e424-4de8-a9fa-3c267647e3de" containerName="openstackclient" Dec 03 13:38:59 crc kubenswrapper[5052]: I1203 13:38:59.676559 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="973f3dde-e424-4de8-a9fa-3c267647e3de" containerName="openstackclient" Dec 03 13:38:59 crc kubenswrapper[5052]: E1203 13:38:59.676586 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="1232c517-147e-407a-9167-afab87e26c94" containerName="extract-content" Dec 03 13:38:59 crc kubenswrapper[5052]: I1203 13:38:59.676595 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="1232c517-147e-407a-9167-afab87e26c94" containerName="extract-content" Dec 03 13:38:59 crc kubenswrapper[5052]: E1203 13:38:59.676607 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="1232c517-147e-407a-9167-afab87e26c94" containerName="registry-server" Dec 03 13:38:59 crc kubenswrapper[5052]: I1203 13:38:59.676639 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="1232c517-147e-407a-9167-afab87e26c94" containerName="registry-server" Dec 03 13:38:59 crc kubenswrapper[5052]: I1203 13:38:59.676807 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="973f3dde-e424-4de8-a9fa-3c267647e3de" containerName="openstackclient" Dec 03 13:38:59 crc kubenswrapper[5052]: I1203 13:38:59.676820 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="1232c517-147e-407a-9167-afab87e26c94" containerName="registry-server" Dec 03 13:38:59 crc kubenswrapper[5052]: I1203 13:38:59.679077 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="keystone-kuttl-tests/keystone4828-account-delete-nvh2w" Dec 03 13:38:59 crc kubenswrapper[5052]: I1203 13:38:59.690767 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="973f3dde-e424-4de8-a9fa-3c267647e3de" path="/var/lib/kubelet/pods/973f3dde-e424-4de8-a9fa-3c267647e3de/volumes" Dec 03 13:38:59 crc kubenswrapper[5052]: I1203 13:38:59.691250 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["keystone-kuttl-tests/keystone4828-account-delete-nvh2w"] Dec 03 13:38:59 crc kubenswrapper[5052]: I1203 13:38:59.789679 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r2qsk\" (UniqueName: \"kubernetes.io/projected/56410ea8-8901-4c3e-b21c-ec3b2f908c47-kube-api-access-r2qsk\") pod \"keystone4828-account-delete-nvh2w\" (UID: \"56410ea8-8901-4c3e-b21c-ec3b2f908c47\") " pod="keystone-kuttl-tests/keystone4828-account-delete-nvh2w" Dec 03 13:38:59 crc kubenswrapper[5052]: I1203 13:38:59.789742 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/56410ea8-8901-4c3e-b21c-ec3b2f908c47-operator-scripts\") pod \"keystone4828-account-delete-nvh2w\" (UID: \"56410ea8-8901-4c3e-b21c-ec3b2f908c47\") " pod="keystone-kuttl-tests/keystone4828-account-delete-nvh2w" Dec 03 13:38:59 crc kubenswrapper[5052]: I1203 13:38:59.891440 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-r2qsk\" (UniqueName: \"kubernetes.io/projected/56410ea8-8901-4c3e-b21c-ec3b2f908c47-kube-api-access-r2qsk\") pod \"keystone4828-account-delete-nvh2w\" (UID: \"56410ea8-8901-4c3e-b21c-ec3b2f908c47\") " pod="keystone-kuttl-tests/keystone4828-account-delete-nvh2w" Dec 03 13:38:59 crc kubenswrapper[5052]: I1203 13:38:59.891533 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/56410ea8-8901-4c3e-b21c-ec3b2f908c47-operator-scripts\") pod \"keystone4828-account-delete-nvh2w\" (UID: \"56410ea8-8901-4c3e-b21c-ec3b2f908c47\") " pod="keystone-kuttl-tests/keystone4828-account-delete-nvh2w" Dec 03 13:38:59 crc kubenswrapper[5052]: I1203 13:38:59.892349 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/56410ea8-8901-4c3e-b21c-ec3b2f908c47-operator-scripts\") pod \"keystone4828-account-delete-nvh2w\" (UID: \"56410ea8-8901-4c3e-b21c-ec3b2f908c47\") " pod="keystone-kuttl-tests/keystone4828-account-delete-nvh2w" Dec 03 13:38:59 crc kubenswrapper[5052]: I1203 13:38:59.912343 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-r2qsk\" (UniqueName: \"kubernetes.io/projected/56410ea8-8901-4c3e-b21c-ec3b2f908c47-kube-api-access-r2qsk\") pod \"keystone4828-account-delete-nvh2w\" (UID: \"56410ea8-8901-4c3e-b21c-ec3b2f908c47\") " pod="keystone-kuttl-tests/keystone4828-account-delete-nvh2w" Dec 03 13:39:00 crc kubenswrapper[5052]: I1203 13:39:00.001800 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="keystone-kuttl-tests/keystone4828-account-delete-nvh2w" Dec 03 13:39:00 crc kubenswrapper[5052]: I1203 13:39:00.409728 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["keystone-kuttl-tests/keystone4828-account-delete-nvh2w"] Dec 03 13:39:00 crc kubenswrapper[5052]: I1203 13:39:00.445190 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="keystone-kuttl-tests/keystone4828-account-delete-nvh2w" event={"ID":"56410ea8-8901-4c3e-b21c-ec3b2f908c47","Type":"ContainerStarted","Data":"6126162e5e3f24c26788d37431ba66b422378daf7b1b1933ba9f80e50ea02a0b"} Dec 03 13:39:01 crc kubenswrapper[5052]: I1203 13:39:01.454349 5052 generic.go:334] "Generic (PLEG): container finished" podID="56410ea8-8901-4c3e-b21c-ec3b2f908c47" containerID="3ac66b90ddd94a93b20370b1a1cd343faca7462f8609d6c9ef6bc35c293eebb2" exitCode=0 Dec 03 13:39:01 crc kubenswrapper[5052]: I1203 13:39:01.454387 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="keystone-kuttl-tests/keystone4828-account-delete-nvh2w" event={"ID":"56410ea8-8901-4c3e-b21c-ec3b2f908c47","Type":"ContainerDied","Data":"3ac66b90ddd94a93b20370b1a1cd343faca7462f8609d6c9ef6bc35c293eebb2"} Dec 03 13:39:02 crc kubenswrapper[5052]: I1203 13:39:02.932481 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="keystone-kuttl-tests/keystone4828-account-delete-nvh2w" Dec 03 13:39:03 crc kubenswrapper[5052]: I1203 13:39:03.042153 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/56410ea8-8901-4c3e-b21c-ec3b2f908c47-operator-scripts\") pod \"56410ea8-8901-4c3e-b21c-ec3b2f908c47\" (UID: \"56410ea8-8901-4c3e-b21c-ec3b2f908c47\") " Dec 03 13:39:03 crc kubenswrapper[5052]: I1203 13:39:03.042228 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r2qsk\" (UniqueName: \"kubernetes.io/projected/56410ea8-8901-4c3e-b21c-ec3b2f908c47-kube-api-access-r2qsk\") pod \"56410ea8-8901-4c3e-b21c-ec3b2f908c47\" (UID: \"56410ea8-8901-4c3e-b21c-ec3b2f908c47\") " Dec 03 13:39:03 crc kubenswrapper[5052]: I1203 13:39:03.042839 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/56410ea8-8901-4c3e-b21c-ec3b2f908c47-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "56410ea8-8901-4c3e-b21c-ec3b2f908c47" (UID: "56410ea8-8901-4c3e-b21c-ec3b2f908c47"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:39:03 crc kubenswrapper[5052]: I1203 13:39:03.048089 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/56410ea8-8901-4c3e-b21c-ec3b2f908c47-kube-api-access-r2qsk" (OuterVolumeSpecName: "kube-api-access-r2qsk") pod "56410ea8-8901-4c3e-b21c-ec3b2f908c47" (UID: "56410ea8-8901-4c3e-b21c-ec3b2f908c47"). InnerVolumeSpecName "kube-api-access-r2qsk". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:39:03 crc kubenswrapper[5052]: I1203 13:39:03.111505 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="keystone-kuttl-tests/keystone-6c857bcbcd-4n6lw" Dec 03 13:39:03 crc kubenswrapper[5052]: I1203 13:39:03.144102 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/56410ea8-8901-4c3e-b21c-ec3b2f908c47-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 13:39:03 crc kubenswrapper[5052]: I1203 13:39:03.144142 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-r2qsk\" (UniqueName: \"kubernetes.io/projected/56410ea8-8901-4c3e-b21c-ec3b2f908c47-kube-api-access-r2qsk\") on node \"crc\" DevicePath \"\"" Dec 03 13:39:03 crc kubenswrapper[5052]: I1203 13:39:03.244794 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/e7224fc6-4050-43bd-95ce-b6257be9ed06-scripts\") pod \"e7224fc6-4050-43bd-95ce-b6257be9ed06\" (UID: \"e7224fc6-4050-43bd-95ce-b6257be9ed06\") " Dec 03 13:39:03 crc kubenswrapper[5052]: I1203 13:39:03.244837 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/e7224fc6-4050-43bd-95ce-b6257be9ed06-fernet-keys\") pod \"e7224fc6-4050-43bd-95ce-b6257be9ed06\" (UID: \"e7224fc6-4050-43bd-95ce-b6257be9ed06\") " Dec 03 13:39:03 crc kubenswrapper[5052]: I1203 13:39:03.244921 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hj6hs\" (UniqueName: \"kubernetes.io/projected/e7224fc6-4050-43bd-95ce-b6257be9ed06-kube-api-access-hj6hs\") pod \"e7224fc6-4050-43bd-95ce-b6257be9ed06\" (UID: \"e7224fc6-4050-43bd-95ce-b6257be9ed06\") " Dec 03 13:39:03 crc kubenswrapper[5052]: I1203 13:39:03.244966 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/e7224fc6-4050-43bd-95ce-b6257be9ed06-config-data\") pod \"e7224fc6-4050-43bd-95ce-b6257be9ed06\" (UID: \"e7224fc6-4050-43bd-95ce-b6257be9ed06\") " Dec 03 13:39:03 crc kubenswrapper[5052]: I1203 13:39:03.245102 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/e7224fc6-4050-43bd-95ce-b6257be9ed06-credential-keys\") pod \"e7224fc6-4050-43bd-95ce-b6257be9ed06\" (UID: \"e7224fc6-4050-43bd-95ce-b6257be9ed06\") " Dec 03 13:39:03 crc kubenswrapper[5052]: I1203 13:39:03.248219 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e7224fc6-4050-43bd-95ce-b6257be9ed06-fernet-keys" (OuterVolumeSpecName: "fernet-keys") pod "e7224fc6-4050-43bd-95ce-b6257be9ed06" (UID: "e7224fc6-4050-43bd-95ce-b6257be9ed06"). InnerVolumeSpecName "fernet-keys". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:39:03 crc kubenswrapper[5052]: I1203 13:39:03.248238 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e7224fc6-4050-43bd-95ce-b6257be9ed06-scripts" (OuterVolumeSpecName: "scripts") pod "e7224fc6-4050-43bd-95ce-b6257be9ed06" (UID: "e7224fc6-4050-43bd-95ce-b6257be9ed06"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:39:03 crc kubenswrapper[5052]: I1203 13:39:03.248269 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e7224fc6-4050-43bd-95ce-b6257be9ed06-kube-api-access-hj6hs" (OuterVolumeSpecName: "kube-api-access-hj6hs") pod "e7224fc6-4050-43bd-95ce-b6257be9ed06" (UID: "e7224fc6-4050-43bd-95ce-b6257be9ed06"). InnerVolumeSpecName "kube-api-access-hj6hs". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:39:03 crc kubenswrapper[5052]: I1203 13:39:03.248965 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e7224fc6-4050-43bd-95ce-b6257be9ed06-credential-keys" (OuterVolumeSpecName: "credential-keys") pod "e7224fc6-4050-43bd-95ce-b6257be9ed06" (UID: "e7224fc6-4050-43bd-95ce-b6257be9ed06"). InnerVolumeSpecName "credential-keys". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:39:03 crc kubenswrapper[5052]: I1203 13:39:03.264698 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e7224fc6-4050-43bd-95ce-b6257be9ed06-config-data" (OuterVolumeSpecName: "config-data") pod "e7224fc6-4050-43bd-95ce-b6257be9ed06" (UID: "e7224fc6-4050-43bd-95ce-b6257be9ed06"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:39:03 crc kubenswrapper[5052]: I1203 13:39:03.347199 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/e7224fc6-4050-43bd-95ce-b6257be9ed06-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 13:39:03 crc kubenswrapper[5052]: I1203 13:39:03.347293 5052 reconciler_common.go:293] "Volume detached for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/e7224fc6-4050-43bd-95ce-b6257be9ed06-fernet-keys\") on node \"crc\" DevicePath \"\"" Dec 03 13:39:03 crc kubenswrapper[5052]: I1203 13:39:03.347315 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-hj6hs\" (UniqueName: \"kubernetes.io/projected/e7224fc6-4050-43bd-95ce-b6257be9ed06-kube-api-access-hj6hs\") on node \"crc\" DevicePath \"\"" Dec 03 13:39:03 crc kubenswrapper[5052]: I1203 13:39:03.347330 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/e7224fc6-4050-43bd-95ce-b6257be9ed06-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 13:39:03 crc kubenswrapper[5052]: I1203 13:39:03.347341 5052 reconciler_common.go:293] "Volume detached for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/e7224fc6-4050-43bd-95ce-b6257be9ed06-credential-keys\") on node \"crc\" DevicePath \"\"" Dec 03 13:39:03 crc kubenswrapper[5052]: I1203 13:39:03.472225 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="keystone-kuttl-tests/keystone4828-account-delete-nvh2w" Dec 03 13:39:03 crc kubenswrapper[5052]: I1203 13:39:03.472214 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="keystone-kuttl-tests/keystone4828-account-delete-nvh2w" event={"ID":"56410ea8-8901-4c3e-b21c-ec3b2f908c47","Type":"ContainerDied","Data":"6126162e5e3f24c26788d37431ba66b422378daf7b1b1933ba9f80e50ea02a0b"} Dec 03 13:39:03 crc kubenswrapper[5052]: I1203 13:39:03.472373 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6126162e5e3f24c26788d37431ba66b422378daf7b1b1933ba9f80e50ea02a0b" Dec 03 13:39:03 crc kubenswrapper[5052]: I1203 13:39:03.474348 5052 generic.go:334] "Generic (PLEG): container finished" podID="e7224fc6-4050-43bd-95ce-b6257be9ed06" containerID="5343e1050e418b924cbc8dd6d94de00ba87a38833e47b7c24b33ef15a3be59b4" exitCode=0 Dec 03 13:39:03 crc kubenswrapper[5052]: I1203 13:39:03.474369 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="keystone-kuttl-tests/keystone-6c857bcbcd-4n6lw" event={"ID":"e7224fc6-4050-43bd-95ce-b6257be9ed06","Type":"ContainerDied","Data":"5343e1050e418b924cbc8dd6d94de00ba87a38833e47b7c24b33ef15a3be59b4"} Dec 03 13:39:03 crc kubenswrapper[5052]: I1203 13:39:03.474404 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="keystone-kuttl-tests/keystone-6c857bcbcd-4n6lw" event={"ID":"e7224fc6-4050-43bd-95ce-b6257be9ed06","Type":"ContainerDied","Data":"ad1979e69a5c187e51317c6d8b117beb91a87bfaeed839e8f6b9203fa4438eaf"} Dec 03 13:39:03 crc kubenswrapper[5052]: I1203 13:39:03.474415 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="keystone-kuttl-tests/keystone-6c857bcbcd-4n6lw" Dec 03 13:39:03 crc kubenswrapper[5052]: I1203 13:39:03.474429 5052 scope.go:117] "RemoveContainer" containerID="5343e1050e418b924cbc8dd6d94de00ba87a38833e47b7c24b33ef15a3be59b4" Dec 03 13:39:03 crc kubenswrapper[5052]: I1203 13:39:03.501057 5052 scope.go:117] "RemoveContainer" containerID="5343e1050e418b924cbc8dd6d94de00ba87a38833e47b7c24b33ef15a3be59b4" Dec 03 13:39:03 crc kubenswrapper[5052]: E1203 13:39:03.501640 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5343e1050e418b924cbc8dd6d94de00ba87a38833e47b7c24b33ef15a3be59b4\": container with ID starting with 5343e1050e418b924cbc8dd6d94de00ba87a38833e47b7c24b33ef15a3be59b4 not found: ID does not exist" containerID="5343e1050e418b924cbc8dd6d94de00ba87a38833e47b7c24b33ef15a3be59b4" Dec 03 13:39:03 crc kubenswrapper[5052]: I1203 13:39:03.501683 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5343e1050e418b924cbc8dd6d94de00ba87a38833e47b7c24b33ef15a3be59b4"} err="failed to get container status \"5343e1050e418b924cbc8dd6d94de00ba87a38833e47b7c24b33ef15a3be59b4\": rpc error: code = NotFound desc = could not find container \"5343e1050e418b924cbc8dd6d94de00ba87a38833e47b7c24b33ef15a3be59b4\": container with ID starting with 5343e1050e418b924cbc8dd6d94de00ba87a38833e47b7c24b33ef15a3be59b4 not found: ID does not exist" Dec 03 13:39:03 crc kubenswrapper[5052]: I1203 13:39:03.517585 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["keystone-kuttl-tests/keystone-6c857bcbcd-4n6lw"] Dec 03 13:39:03 crc kubenswrapper[5052]: I1203 13:39:03.520617 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["keystone-kuttl-tests/keystone-6c857bcbcd-4n6lw"] Dec 03 13:39:03 crc kubenswrapper[5052]: I1203 13:39:03.681166 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e7224fc6-4050-43bd-95ce-b6257be9ed06" path="/var/lib/kubelet/pods/e7224fc6-4050-43bd-95ce-b6257be9ed06/volumes" Dec 03 13:39:04 crc kubenswrapper[5052]: I1203 13:39:04.708174 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["keystone-kuttl-tests/keystone4828-account-delete-nvh2w"] Dec 03 13:39:04 crc kubenswrapper[5052]: I1203 13:39:04.713608 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["keystone-kuttl-tests/keystone4828-account-delete-nvh2w"] Dec 03 13:39:05 crc kubenswrapper[5052]: I1203 13:39:05.680389 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="56410ea8-8901-4c3e-b21c-ec3b2f908c47" path="/var/lib/kubelet/pods/56410ea8-8901-4c3e-b21c-ec3b2f908c47/volumes" Dec 03 13:39:11 crc kubenswrapper[5052]: I1203 13:39:11.415626 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["keystone-kuttl-tests/openstack-galera-0"] Dec 03 13:39:11 crc kubenswrapper[5052]: I1203 13:39:11.421286 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["keystone-kuttl-tests/openstack-galera-1"] Dec 03 13:39:11 crc kubenswrapper[5052]: I1203 13:39:11.429499 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["keystone-kuttl-tests/openstack-galera-2"] Dec 03 13:39:11 crc kubenswrapper[5052]: I1203 13:39:11.556228 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="keystone-kuttl-tests/openstack-galera-2" podUID="2e771f67-f156-4dec-a77f-88b4735f298b" containerName="galera" containerID="cri-o://9fb2554ac99ea6b2010070c8f5d4dd0dec16b79118170eabf332947ede384598" gracePeriod=30 Dec 03 13:39:12 crc kubenswrapper[5052]: I1203 13:39:12.342683 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["keystone-kuttl-tests/memcached-0"] Dec 03 13:39:12 crc kubenswrapper[5052]: I1203 13:39:12.343221 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="keystone-kuttl-tests/memcached-0" podUID="f72c36e0-b6bc-412c-887f-3e1394cf6c1f" containerName="memcached" containerID="cri-o://e957ae3dd15df2d129c0d9b9587d9be204eec37250b8e16dd64286c8ed6109e8" gracePeriod=30 Dec 03 13:39:12 crc kubenswrapper[5052]: I1203 13:39:12.543157 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="keystone-kuttl-tests/openstack-galera-2" Dec 03 13:39:12 crc kubenswrapper[5052]: I1203 13:39:12.545376 5052 generic.go:334] "Generic (PLEG): container finished" podID="2e771f67-f156-4dec-a77f-88b4735f298b" containerID="9fb2554ac99ea6b2010070c8f5d4dd0dec16b79118170eabf332947ede384598" exitCode=0 Dec 03 13:39:12 crc kubenswrapper[5052]: I1203 13:39:12.545440 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="keystone-kuttl-tests/openstack-galera-2" event={"ID":"2e771f67-f156-4dec-a77f-88b4735f298b","Type":"ContainerDied","Data":"9fb2554ac99ea6b2010070c8f5d4dd0dec16b79118170eabf332947ede384598"} Dec 03 13:39:12 crc kubenswrapper[5052]: I1203 13:39:12.545504 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="keystone-kuttl-tests/openstack-galera-2" event={"ID":"2e771f67-f156-4dec-a77f-88b4735f298b","Type":"ContainerDied","Data":"de5dad1917376303577d9774889bfe76e15007d2b0a03fcc56189936a91a8bd7"} Dec 03 13:39:12 crc kubenswrapper[5052]: I1203 13:39:12.545611 5052 scope.go:117] "RemoveContainer" containerID="9fb2554ac99ea6b2010070c8f5d4dd0dec16b79118170eabf332947ede384598" Dec 03 13:39:12 crc kubenswrapper[5052]: I1203 13:39:12.569838 5052 scope.go:117] "RemoveContainer" containerID="58affb7d9a0f5d58e2c49f7f718b13fdf9ae2fbe26ffd445be1a013018c36651" Dec 03 13:39:12 crc kubenswrapper[5052]: I1203 13:39:12.586710 5052 scope.go:117] "RemoveContainer" containerID="9fb2554ac99ea6b2010070c8f5d4dd0dec16b79118170eabf332947ede384598" Dec 03 13:39:12 crc kubenswrapper[5052]: E1203 13:39:12.587116 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9fb2554ac99ea6b2010070c8f5d4dd0dec16b79118170eabf332947ede384598\": container with ID starting with 9fb2554ac99ea6b2010070c8f5d4dd0dec16b79118170eabf332947ede384598 not found: ID does not exist" containerID="9fb2554ac99ea6b2010070c8f5d4dd0dec16b79118170eabf332947ede384598" Dec 03 13:39:12 crc kubenswrapper[5052]: I1203 13:39:12.587149 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9fb2554ac99ea6b2010070c8f5d4dd0dec16b79118170eabf332947ede384598"} err="failed to get container status \"9fb2554ac99ea6b2010070c8f5d4dd0dec16b79118170eabf332947ede384598\": rpc error: code = NotFound desc = could not find container \"9fb2554ac99ea6b2010070c8f5d4dd0dec16b79118170eabf332947ede384598\": container with ID starting with 9fb2554ac99ea6b2010070c8f5d4dd0dec16b79118170eabf332947ede384598 not found: ID does not exist" Dec 03 13:39:12 crc kubenswrapper[5052]: I1203 13:39:12.587172 5052 scope.go:117] "RemoveContainer" containerID="58affb7d9a0f5d58e2c49f7f718b13fdf9ae2fbe26ffd445be1a013018c36651" Dec 03 13:39:12 crc kubenswrapper[5052]: E1203 13:39:12.587412 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"58affb7d9a0f5d58e2c49f7f718b13fdf9ae2fbe26ffd445be1a013018c36651\": container with ID starting with 58affb7d9a0f5d58e2c49f7f718b13fdf9ae2fbe26ffd445be1a013018c36651 not found: ID does not exist" containerID="58affb7d9a0f5d58e2c49f7f718b13fdf9ae2fbe26ffd445be1a013018c36651" Dec 03 13:39:12 crc kubenswrapper[5052]: I1203 13:39:12.587447 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"58affb7d9a0f5d58e2c49f7f718b13fdf9ae2fbe26ffd445be1a013018c36651"} err="failed to get container status \"58affb7d9a0f5d58e2c49f7f718b13fdf9ae2fbe26ffd445be1a013018c36651\": rpc error: code = NotFound desc = could not find container \"58affb7d9a0f5d58e2c49f7f718b13fdf9ae2fbe26ffd445be1a013018c36651\": container with ID starting with 58affb7d9a0f5d58e2c49f7f718b13fdf9ae2fbe26ffd445be1a013018c36651 not found: ID does not exist" Dec 03 13:39:12 crc kubenswrapper[5052]: I1203 13:39:12.702376 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xzkxh\" (UniqueName: \"kubernetes.io/projected/2e771f67-f156-4dec-a77f-88b4735f298b-kube-api-access-xzkxh\") pod \"2e771f67-f156-4dec-a77f-88b4735f298b\" (UID: \"2e771f67-f156-4dec-a77f-88b4735f298b\") " Dec 03 13:39:12 crc kubenswrapper[5052]: I1203 13:39:12.702438 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/2e771f67-f156-4dec-a77f-88b4735f298b-config-data-default\") pod \"2e771f67-f156-4dec-a77f-88b4735f298b\" (UID: \"2e771f67-f156-4dec-a77f-88b4735f298b\") " Dec 03 13:39:12 crc kubenswrapper[5052]: I1203 13:39:12.702465 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"mysql-db\" (UniqueName: \"kubernetes.io/local-volume/local-storage14-crc\") pod \"2e771f67-f156-4dec-a77f-88b4735f298b\" (UID: \"2e771f67-f156-4dec-a77f-88b4735f298b\") " Dec 03 13:39:12 crc kubenswrapper[5052]: I1203 13:39:12.702485 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/2e771f67-f156-4dec-a77f-88b4735f298b-operator-scripts\") pod \"2e771f67-f156-4dec-a77f-88b4735f298b\" (UID: \"2e771f67-f156-4dec-a77f-88b4735f298b\") " Dec 03 13:39:12 crc kubenswrapper[5052]: I1203 13:39:12.702535 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/2e771f67-f156-4dec-a77f-88b4735f298b-kolla-config\") pod \"2e771f67-f156-4dec-a77f-88b4735f298b\" (UID: \"2e771f67-f156-4dec-a77f-88b4735f298b\") " Dec 03 13:39:12 crc kubenswrapper[5052]: I1203 13:39:12.702607 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/2e771f67-f156-4dec-a77f-88b4735f298b-config-data-generated\") pod \"2e771f67-f156-4dec-a77f-88b4735f298b\" (UID: \"2e771f67-f156-4dec-a77f-88b4735f298b\") " Dec 03 13:39:12 crc kubenswrapper[5052]: I1203 13:39:12.704266 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2e771f67-f156-4dec-a77f-88b4735f298b-kolla-config" (OuterVolumeSpecName: "kolla-config") pod "2e771f67-f156-4dec-a77f-88b4735f298b" (UID: "2e771f67-f156-4dec-a77f-88b4735f298b"). InnerVolumeSpecName "kolla-config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:39:12 crc kubenswrapper[5052]: I1203 13:39:12.704275 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e771f67-f156-4dec-a77f-88b4735f298b-config-data-generated" (OuterVolumeSpecName: "config-data-generated") pod "2e771f67-f156-4dec-a77f-88b4735f298b" (UID: "2e771f67-f156-4dec-a77f-88b4735f298b"). InnerVolumeSpecName "config-data-generated". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 13:39:12 crc kubenswrapper[5052]: I1203 13:39:12.704347 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2e771f67-f156-4dec-a77f-88b4735f298b-config-data-default" (OuterVolumeSpecName: "config-data-default") pod "2e771f67-f156-4dec-a77f-88b4735f298b" (UID: "2e771f67-f156-4dec-a77f-88b4735f298b"). InnerVolumeSpecName "config-data-default". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:39:12 crc kubenswrapper[5052]: I1203 13:39:12.704429 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2e771f67-f156-4dec-a77f-88b4735f298b-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "2e771f67-f156-4dec-a77f-88b4735f298b" (UID: "2e771f67-f156-4dec-a77f-88b4735f298b"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:39:12 crc kubenswrapper[5052]: I1203 13:39:12.708997 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2e771f67-f156-4dec-a77f-88b4735f298b-kube-api-access-xzkxh" (OuterVolumeSpecName: "kube-api-access-xzkxh") pod "2e771f67-f156-4dec-a77f-88b4735f298b" (UID: "2e771f67-f156-4dec-a77f-88b4735f298b"). InnerVolumeSpecName "kube-api-access-xzkxh". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:39:12 crc kubenswrapper[5052]: I1203 13:39:12.714204 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage14-crc" (OuterVolumeSpecName: "mysql-db") pod "2e771f67-f156-4dec-a77f-88b4735f298b" (UID: "2e771f67-f156-4dec-a77f-88b4735f298b"). InnerVolumeSpecName "local-storage14-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 13:39:12 crc kubenswrapper[5052]: I1203 13:39:12.804274 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/2e771f67-f156-4dec-a77f-88b4735f298b-config-data-generated\") on node \"crc\" DevicePath \"\"" Dec 03 13:39:12 crc kubenswrapper[5052]: I1203 13:39:12.804314 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-xzkxh\" (UniqueName: \"kubernetes.io/projected/2e771f67-f156-4dec-a77f-88b4735f298b-kube-api-access-xzkxh\") on node \"crc\" DevicePath \"\"" Dec 03 13:39:12 crc kubenswrapper[5052]: I1203 13:39:12.804332 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/2e771f67-f156-4dec-a77f-88b4735f298b-config-data-default\") on node \"crc\" DevicePath \"\"" Dec 03 13:39:12 crc kubenswrapper[5052]: I1203 13:39:12.804357 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage14-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage14-crc\") on node \"crc\" " Dec 03 13:39:12 crc kubenswrapper[5052]: I1203 13:39:12.804372 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/2e771f67-f156-4dec-a77f-88b4735f298b-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 13:39:12 crc kubenswrapper[5052]: I1203 13:39:12.804384 5052 reconciler_common.go:293] "Volume detached for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/2e771f67-f156-4dec-a77f-88b4735f298b-kolla-config\") on node \"crc\" DevicePath \"\"" Dec 03 13:39:12 crc kubenswrapper[5052]: I1203 13:39:12.819345 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage14-crc" (UniqueName: "kubernetes.io/local-volume/local-storage14-crc") on node "crc" Dec 03 13:39:12 crc kubenswrapper[5052]: I1203 13:39:12.863812 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["keystone-kuttl-tests/rabbitmq-server-0"] Dec 03 13:39:12 crc kubenswrapper[5052]: I1203 13:39:12.906283 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage14-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage14-crc\") on node \"crc\" DevicePath \"\"" Dec 03 13:39:13 crc kubenswrapper[5052]: I1203 13:39:13.245387 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["keystone-kuttl-tests/rabbitmq-server-0"] Dec 03 13:39:13 crc kubenswrapper[5052]: I1203 13:39:13.553800 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="keystone-kuttl-tests/openstack-galera-2" Dec 03 13:39:13 crc kubenswrapper[5052]: I1203 13:39:13.584802 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="keystone-kuttl-tests/openstack-galera-1" podUID="0ea9b78b-2beb-4bdb-852c-1b92b9638551" containerName="galera" containerID="cri-o://2ebd4f7e1d8ef53cddce7b2e5ce181643f18fc1958462225a56999f9b3633b6b" gracePeriod=28 Dec 03 13:39:13 crc kubenswrapper[5052]: I1203 13:39:13.584932 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["keystone-kuttl-tests/openstack-galera-2"] Dec 03 13:39:13 crc kubenswrapper[5052]: I1203 13:39:13.589615 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["keystone-kuttl-tests/openstack-galera-2"] Dec 03 13:39:13 crc kubenswrapper[5052]: I1203 13:39:13.593492 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="keystone-kuttl-tests/rabbitmq-server-0" podUID="9d2e4b3b-6b0f-4e92-a6bb-fa0df37feb28" containerName="rabbitmq" containerID="cri-o://64166ff7fe7a645af18c176c3f4902178d3aa89fc21f02ad0a4020f06c1200e0" gracePeriod=604800 Dec 03 13:39:13 crc kubenswrapper[5052]: I1203 13:39:13.681993 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2e771f67-f156-4dec-a77f-88b4735f298b" path="/var/lib/kubelet/pods/2e771f67-f156-4dec-a77f-88b4735f298b/volumes" Dec 03 13:39:14 crc kubenswrapper[5052]: I1203 13:39:14.095336 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="keystone-kuttl-tests/memcached-0" Dec 03 13:39:14 crc kubenswrapper[5052]: I1203 13:39:14.231309 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/f72c36e0-b6bc-412c-887f-3e1394cf6c1f-config-data\") pod \"f72c36e0-b6bc-412c-887f-3e1394cf6c1f\" (UID: \"f72c36e0-b6bc-412c-887f-3e1394cf6c1f\") " Dec 03 13:39:14 crc kubenswrapper[5052]: I1203 13:39:14.231373 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/f72c36e0-b6bc-412c-887f-3e1394cf6c1f-kolla-config\") pod \"f72c36e0-b6bc-412c-887f-3e1394cf6c1f\" (UID: \"f72c36e0-b6bc-412c-887f-3e1394cf6c1f\") " Dec 03 13:39:14 crc kubenswrapper[5052]: I1203 13:39:14.231426 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-968v7\" (UniqueName: \"kubernetes.io/projected/f72c36e0-b6bc-412c-887f-3e1394cf6c1f-kube-api-access-968v7\") pod \"f72c36e0-b6bc-412c-887f-3e1394cf6c1f\" (UID: \"f72c36e0-b6bc-412c-887f-3e1394cf6c1f\") " Dec 03 13:39:14 crc kubenswrapper[5052]: I1203 13:39:14.232386 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f72c36e0-b6bc-412c-887f-3e1394cf6c1f-config-data" (OuterVolumeSpecName: "config-data") pod "f72c36e0-b6bc-412c-887f-3e1394cf6c1f" (UID: "f72c36e0-b6bc-412c-887f-3e1394cf6c1f"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:39:14 crc kubenswrapper[5052]: I1203 13:39:14.232807 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f72c36e0-b6bc-412c-887f-3e1394cf6c1f-kolla-config" (OuterVolumeSpecName: "kolla-config") pod "f72c36e0-b6bc-412c-887f-3e1394cf6c1f" (UID: "f72c36e0-b6bc-412c-887f-3e1394cf6c1f"). InnerVolumeSpecName "kolla-config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:39:14 crc kubenswrapper[5052]: I1203 13:39:14.273781 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-operators/keystone-operator-controller-manager-6f4cbcd5b5-8gmln"] Dec 03 13:39:14 crc kubenswrapper[5052]: I1203 13:39:14.273969 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-operators/keystone-operator-controller-manager-6f4cbcd5b5-8gmln" podUID="ae81dcbe-7c90-4de6-94bd-014eb4e9557f" containerName="manager" containerID="cri-o://deb1f298b409739ff6b92cb257ecf5946876b50d914e7262651ac96341518d6e" gracePeriod=10 Dec 03 13:39:14 crc kubenswrapper[5052]: I1203 13:39:14.275821 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f72c36e0-b6bc-412c-887f-3e1394cf6c1f-kube-api-access-968v7" (OuterVolumeSpecName: "kube-api-access-968v7") pod "f72c36e0-b6bc-412c-887f-3e1394cf6c1f" (UID: "f72c36e0-b6bc-412c-887f-3e1394cf6c1f"). InnerVolumeSpecName "kube-api-access-968v7". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:39:14 crc kubenswrapper[5052]: I1203 13:39:14.338385 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-968v7\" (UniqueName: \"kubernetes.io/projected/f72c36e0-b6bc-412c-887f-3e1394cf6c1f-kube-api-access-968v7\") on node \"crc\" DevicePath \"\"" Dec 03 13:39:14 crc kubenswrapper[5052]: I1203 13:39:14.338765 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/f72c36e0-b6bc-412c-887f-3e1394cf6c1f-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 13:39:14 crc kubenswrapper[5052]: I1203 13:39:14.338781 5052 reconciler_common.go:293] "Volume detached for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/f72c36e0-b6bc-412c-887f-3e1394cf6c1f-kolla-config\") on node \"crc\" DevicePath \"\"" Dec 03 13:39:14 crc kubenswrapper[5052]: I1203 13:39:14.497610 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-operators/keystone-operator-index-9ddkd"] Dec 03 13:39:14 crc kubenswrapper[5052]: I1203 13:39:14.497834 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-operators/keystone-operator-index-9ddkd" podUID="da70419b-4f9e-405d-8fe7-7dbca0ca315e" containerName="registry-server" containerID="cri-o://2d6aaf797a5a705bce7467f9106b54a4a7ecb16c29f11a578ba19225c543f84d" gracePeriod=30 Dec 03 13:39:14 crc kubenswrapper[5052]: I1203 13:39:14.590035 5052 generic.go:334] "Generic (PLEG): container finished" podID="ae81dcbe-7c90-4de6-94bd-014eb4e9557f" containerID="deb1f298b409739ff6b92cb257ecf5946876b50d914e7262651ac96341518d6e" exitCode=0 Dec 03 13:39:14 crc kubenswrapper[5052]: I1203 13:39:14.590155 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/keystone-operator-controller-manager-6f4cbcd5b5-8gmln" event={"ID":"ae81dcbe-7c90-4de6-94bd-014eb4e9557f","Type":"ContainerDied","Data":"deb1f298b409739ff6b92cb257ecf5946876b50d914e7262651ac96341518d6e"} Dec 03 13:39:14 crc kubenswrapper[5052]: I1203 13:39:14.606286 5052 generic.go:334] "Generic (PLEG): container finished" podID="f72c36e0-b6bc-412c-887f-3e1394cf6c1f" containerID="e957ae3dd15df2d129c0d9b9587d9be204eec37250b8e16dd64286c8ed6109e8" exitCode=0 Dec 03 13:39:14 crc kubenswrapper[5052]: I1203 13:39:14.606343 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="keystone-kuttl-tests/memcached-0" event={"ID":"f72c36e0-b6bc-412c-887f-3e1394cf6c1f","Type":"ContainerDied","Data":"e957ae3dd15df2d129c0d9b9587d9be204eec37250b8e16dd64286c8ed6109e8"} Dec 03 13:39:14 crc kubenswrapper[5052]: I1203 13:39:14.606369 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="keystone-kuttl-tests/memcached-0" event={"ID":"f72c36e0-b6bc-412c-887f-3e1394cf6c1f","Type":"ContainerDied","Data":"cfc681750910beccb769ac932e3168898930981b97f1a39969af9884b064168d"} Dec 03 13:39:14 crc kubenswrapper[5052]: I1203 13:39:14.606386 5052 scope.go:117] "RemoveContainer" containerID="e957ae3dd15df2d129c0d9b9587d9be204eec37250b8e16dd64286c8ed6109e8" Dec 03 13:39:14 crc kubenswrapper[5052]: I1203 13:39:14.606611 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="keystone-kuttl-tests/memcached-0" Dec 03 13:39:14 crc kubenswrapper[5052]: I1203 13:39:14.632149 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-operators/49c083020ae5dfe237b73a6c6b807501660a323f061d7879268c43a121jj5pq"] Dec 03 13:39:14 crc kubenswrapper[5052]: I1203 13:39:14.653961 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-operators/49c083020ae5dfe237b73a6c6b807501660a323f061d7879268c43a121jj5pq"] Dec 03 13:39:14 crc kubenswrapper[5052]: I1203 13:39:14.687464 5052 scope.go:117] "RemoveContainer" containerID="e957ae3dd15df2d129c0d9b9587d9be204eec37250b8e16dd64286c8ed6109e8" Dec 03 13:39:14 crc kubenswrapper[5052]: E1203 13:39:14.689946 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e957ae3dd15df2d129c0d9b9587d9be204eec37250b8e16dd64286c8ed6109e8\": container with ID starting with e957ae3dd15df2d129c0d9b9587d9be204eec37250b8e16dd64286c8ed6109e8 not found: ID does not exist" containerID="e957ae3dd15df2d129c0d9b9587d9be204eec37250b8e16dd64286c8ed6109e8" Dec 03 13:39:14 crc kubenswrapper[5052]: I1203 13:39:14.689983 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e957ae3dd15df2d129c0d9b9587d9be204eec37250b8e16dd64286c8ed6109e8"} err="failed to get container status \"e957ae3dd15df2d129c0d9b9587d9be204eec37250b8e16dd64286c8ed6109e8\": rpc error: code = NotFound desc = could not find container \"e957ae3dd15df2d129c0d9b9587d9be204eec37250b8e16dd64286c8ed6109e8\": container with ID starting with e957ae3dd15df2d129c0d9b9587d9be204eec37250b8e16dd64286c8ed6109e8 not found: ID does not exist" Dec 03 13:39:14 crc kubenswrapper[5052]: I1203 13:39:14.698170 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["keystone-kuttl-tests/memcached-0"] Dec 03 13:39:14 crc kubenswrapper[5052]: I1203 13:39:14.730227 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["keystone-kuttl-tests/memcached-0"] Dec 03 13:39:14 crc kubenswrapper[5052]: I1203 13:39:14.922100 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-operators/keystone-operator-controller-manager-6f4cbcd5b5-8gmln" Dec 03 13:39:15 crc kubenswrapper[5052]: I1203 13:39:15.058286 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ae81dcbe-7c90-4de6-94bd-014eb4e9557f-apiservice-cert\") pod \"ae81dcbe-7c90-4de6-94bd-014eb4e9557f\" (UID: \"ae81dcbe-7c90-4de6-94bd-014eb4e9557f\") " Dec 03 13:39:15 crc kubenswrapper[5052]: I1203 13:39:15.058439 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8stn5\" (UniqueName: \"kubernetes.io/projected/ae81dcbe-7c90-4de6-94bd-014eb4e9557f-kube-api-access-8stn5\") pod \"ae81dcbe-7c90-4de6-94bd-014eb4e9557f\" (UID: \"ae81dcbe-7c90-4de6-94bd-014eb4e9557f\") " Dec 03 13:39:15 crc kubenswrapper[5052]: I1203 13:39:15.058697 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ae81dcbe-7c90-4de6-94bd-014eb4e9557f-webhook-cert\") pod \"ae81dcbe-7c90-4de6-94bd-014eb4e9557f\" (UID: \"ae81dcbe-7c90-4de6-94bd-014eb4e9557f\") " Dec 03 13:39:15 crc kubenswrapper[5052]: I1203 13:39:15.065563 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ae81dcbe-7c90-4de6-94bd-014eb4e9557f-kube-api-access-8stn5" (OuterVolumeSpecName: "kube-api-access-8stn5") pod "ae81dcbe-7c90-4de6-94bd-014eb4e9557f" (UID: "ae81dcbe-7c90-4de6-94bd-014eb4e9557f"). InnerVolumeSpecName "kube-api-access-8stn5". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:39:15 crc kubenswrapper[5052]: I1203 13:39:15.065698 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ae81dcbe-7c90-4de6-94bd-014eb4e9557f-apiservice-cert" (OuterVolumeSpecName: "apiservice-cert") pod "ae81dcbe-7c90-4de6-94bd-014eb4e9557f" (UID: "ae81dcbe-7c90-4de6-94bd-014eb4e9557f"). InnerVolumeSpecName "apiservice-cert". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:39:15 crc kubenswrapper[5052]: I1203 13:39:15.066346 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ae81dcbe-7c90-4de6-94bd-014eb4e9557f-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "ae81dcbe-7c90-4de6-94bd-014eb4e9557f" (UID: "ae81dcbe-7c90-4de6-94bd-014eb4e9557f"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:39:15 crc kubenswrapper[5052]: I1203 13:39:15.068447 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-operators/keystone-operator-index-9ddkd" Dec 03 13:39:15 crc kubenswrapper[5052]: I1203 13:39:15.129203 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="keystone-kuttl-tests/rabbitmq-server-0" Dec 03 13:39:15 crc kubenswrapper[5052]: I1203 13:39:15.160811 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qknd7\" (UniqueName: \"kubernetes.io/projected/da70419b-4f9e-405d-8fe7-7dbca0ca315e-kube-api-access-qknd7\") pod \"da70419b-4f9e-405d-8fe7-7dbca0ca315e\" (UID: \"da70419b-4f9e-405d-8fe7-7dbca0ca315e\") " Dec 03 13:39:15 crc kubenswrapper[5052]: I1203 13:39:15.162298 5052 reconciler_common.go:293] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ae81dcbe-7c90-4de6-94bd-014eb4e9557f-webhook-cert\") on node \"crc\" DevicePath \"\"" Dec 03 13:39:15 crc kubenswrapper[5052]: I1203 13:39:15.162360 5052 reconciler_common.go:293] "Volume detached for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ae81dcbe-7c90-4de6-94bd-014eb4e9557f-apiservice-cert\") on node \"crc\" DevicePath \"\"" Dec 03 13:39:15 crc kubenswrapper[5052]: I1203 13:39:15.162382 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-8stn5\" (UniqueName: \"kubernetes.io/projected/ae81dcbe-7c90-4de6-94bd-014eb4e9557f-kube-api-access-8stn5\") on node \"crc\" DevicePath \"\"" Dec 03 13:39:15 crc kubenswrapper[5052]: I1203 13:39:15.164862 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/da70419b-4f9e-405d-8fe7-7dbca0ca315e-kube-api-access-qknd7" (OuterVolumeSpecName: "kube-api-access-qknd7") pod "da70419b-4f9e-405d-8fe7-7dbca0ca315e" (UID: "da70419b-4f9e-405d-8fe7-7dbca0ca315e"). InnerVolumeSpecName "kube-api-access-qknd7". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:39:15 crc kubenswrapper[5052]: I1203 13:39:15.263788 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"pod-info\" (UniqueName: \"kubernetes.io/downward-api/9d2e4b3b-6b0f-4e92-a6bb-fa0df37feb28-pod-info\") pod \"9d2e4b3b-6b0f-4e92-a6bb-fa0df37feb28\" (UID: \"9d2e4b3b-6b0f-4e92-a6bb-fa0df37feb28\") " Dec 03 13:39:15 crc kubenswrapper[5052]: I1203 13:39:15.264281 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"plugins-conf\" (UniqueName: \"kubernetes.io/configmap/9d2e4b3b-6b0f-4e92-a6bb-fa0df37feb28-plugins-conf\") pod \"9d2e4b3b-6b0f-4e92-a6bb-fa0df37feb28\" (UID: \"9d2e4b3b-6b0f-4e92-a6bb-fa0df37feb28\") " Dec 03 13:39:15 crc kubenswrapper[5052]: I1203 13:39:15.264345 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"rabbitmq-confd\" (UniqueName: \"kubernetes.io/projected/9d2e4b3b-6b0f-4e92-a6bb-fa0df37feb28-rabbitmq-confd\") pod \"9d2e4b3b-6b0f-4e92-a6bb-fa0df37feb28\" (UID: \"9d2e4b3b-6b0f-4e92-a6bb-fa0df37feb28\") " Dec 03 13:39:15 crc kubenswrapper[5052]: I1203 13:39:15.264526 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"persistence\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-10491a20-0ef4-41c6-b028-50667b3c5171\") pod \"9d2e4b3b-6b0f-4e92-a6bb-fa0df37feb28\" (UID: \"9d2e4b3b-6b0f-4e92-a6bb-fa0df37feb28\") " Dec 03 13:39:15 crc kubenswrapper[5052]: I1203 13:39:15.264568 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"erlang-cookie-secret\" (UniqueName: \"kubernetes.io/secret/9d2e4b3b-6b0f-4e92-a6bb-fa0df37feb28-erlang-cookie-secret\") pod \"9d2e4b3b-6b0f-4e92-a6bb-fa0df37feb28\" (UID: \"9d2e4b3b-6b0f-4e92-a6bb-fa0df37feb28\") " Dec 03 13:39:15 crc kubenswrapper[5052]: I1203 13:39:15.264597 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"rabbitmq-plugins\" (UniqueName: \"kubernetes.io/empty-dir/9d2e4b3b-6b0f-4e92-a6bb-fa0df37feb28-rabbitmq-plugins\") pod \"9d2e4b3b-6b0f-4e92-a6bb-fa0df37feb28\" (UID: \"9d2e4b3b-6b0f-4e92-a6bb-fa0df37feb28\") " Dec 03 13:39:15 crc kubenswrapper[5052]: I1203 13:39:15.264646 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"rabbitmq-erlang-cookie\" (UniqueName: \"kubernetes.io/empty-dir/9d2e4b3b-6b0f-4e92-a6bb-fa0df37feb28-rabbitmq-erlang-cookie\") pod \"9d2e4b3b-6b0f-4e92-a6bb-fa0df37feb28\" (UID: \"9d2e4b3b-6b0f-4e92-a6bb-fa0df37feb28\") " Dec 03 13:39:15 crc kubenswrapper[5052]: I1203 13:39:15.264676 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hzlgb\" (UniqueName: \"kubernetes.io/projected/9d2e4b3b-6b0f-4e92-a6bb-fa0df37feb28-kube-api-access-hzlgb\") pod \"9d2e4b3b-6b0f-4e92-a6bb-fa0df37feb28\" (UID: \"9d2e4b3b-6b0f-4e92-a6bb-fa0df37feb28\") " Dec 03 13:39:15 crc kubenswrapper[5052]: I1203 13:39:15.264808 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9d2e4b3b-6b0f-4e92-a6bb-fa0df37feb28-plugins-conf" (OuterVolumeSpecName: "plugins-conf") pod "9d2e4b3b-6b0f-4e92-a6bb-fa0df37feb28" (UID: "9d2e4b3b-6b0f-4e92-a6bb-fa0df37feb28"). InnerVolumeSpecName "plugins-conf". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:39:15 crc kubenswrapper[5052]: I1203 13:39:15.265012 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-qknd7\" (UniqueName: \"kubernetes.io/projected/da70419b-4f9e-405d-8fe7-7dbca0ca315e-kube-api-access-qknd7\") on node \"crc\" DevicePath \"\"" Dec 03 13:39:15 crc kubenswrapper[5052]: I1203 13:39:15.265035 5052 reconciler_common.go:293] "Volume detached for volume \"plugins-conf\" (UniqueName: \"kubernetes.io/configmap/9d2e4b3b-6b0f-4e92-a6bb-fa0df37feb28-plugins-conf\") on node \"crc\" DevicePath \"\"" Dec 03 13:39:15 crc kubenswrapper[5052]: I1203 13:39:15.265233 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d2e4b3b-6b0f-4e92-a6bb-fa0df37feb28-rabbitmq-plugins" (OuterVolumeSpecName: "rabbitmq-plugins") pod "9d2e4b3b-6b0f-4e92-a6bb-fa0df37feb28" (UID: "9d2e4b3b-6b0f-4e92-a6bb-fa0df37feb28"). InnerVolumeSpecName "rabbitmq-plugins". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 13:39:15 crc kubenswrapper[5052]: I1203 13:39:15.265435 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d2e4b3b-6b0f-4e92-a6bb-fa0df37feb28-rabbitmq-erlang-cookie" (OuterVolumeSpecName: "rabbitmq-erlang-cookie") pod "9d2e4b3b-6b0f-4e92-a6bb-fa0df37feb28" (UID: "9d2e4b3b-6b0f-4e92-a6bb-fa0df37feb28"). InnerVolumeSpecName "rabbitmq-erlang-cookie". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 13:39:15 crc kubenswrapper[5052]: I1203 13:39:15.267682 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9d2e4b3b-6b0f-4e92-a6bb-fa0df37feb28-pod-info" (OuterVolumeSpecName: "pod-info") pod "9d2e4b3b-6b0f-4e92-a6bb-fa0df37feb28" (UID: "9d2e4b3b-6b0f-4e92-a6bb-fa0df37feb28"). InnerVolumeSpecName "pod-info". PluginName "kubernetes.io/downward-api", VolumeGidValue "" Dec 03 13:39:15 crc kubenswrapper[5052]: I1203 13:39:15.268472 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9d2e4b3b-6b0f-4e92-a6bb-fa0df37feb28-kube-api-access-hzlgb" (OuterVolumeSpecName: "kube-api-access-hzlgb") pod "9d2e4b3b-6b0f-4e92-a6bb-fa0df37feb28" (UID: "9d2e4b3b-6b0f-4e92-a6bb-fa0df37feb28"). InnerVolumeSpecName "kube-api-access-hzlgb". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:39:15 crc kubenswrapper[5052]: I1203 13:39:15.269744 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9d2e4b3b-6b0f-4e92-a6bb-fa0df37feb28-erlang-cookie-secret" (OuterVolumeSpecName: "erlang-cookie-secret") pod "9d2e4b3b-6b0f-4e92-a6bb-fa0df37feb28" (UID: "9d2e4b3b-6b0f-4e92-a6bb-fa0df37feb28"). InnerVolumeSpecName "erlang-cookie-secret". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:39:15 crc kubenswrapper[5052]: I1203 13:39:15.274694 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-10491a20-0ef4-41c6-b028-50667b3c5171" (OuterVolumeSpecName: "persistence") pod "9d2e4b3b-6b0f-4e92-a6bb-fa0df37feb28" (UID: "9d2e4b3b-6b0f-4e92-a6bb-fa0df37feb28"). InnerVolumeSpecName "pvc-10491a20-0ef4-41c6-b028-50667b3c5171". PluginName "kubernetes.io/csi", VolumeGidValue "" Dec 03 13:39:15 crc kubenswrapper[5052]: I1203 13:39:15.329539 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9d2e4b3b-6b0f-4e92-a6bb-fa0df37feb28-rabbitmq-confd" (OuterVolumeSpecName: "rabbitmq-confd") pod "9d2e4b3b-6b0f-4e92-a6bb-fa0df37feb28" (UID: "9d2e4b3b-6b0f-4e92-a6bb-fa0df37feb28"). InnerVolumeSpecName "rabbitmq-confd". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:39:15 crc kubenswrapper[5052]: I1203 13:39:15.366586 5052 reconciler_common.go:293] "Volume detached for volume \"pod-info\" (UniqueName: \"kubernetes.io/downward-api/9d2e4b3b-6b0f-4e92-a6bb-fa0df37feb28-pod-info\") on node \"crc\" DevicePath \"\"" Dec 03 13:39:15 crc kubenswrapper[5052]: I1203 13:39:15.366618 5052 reconciler_common.go:293] "Volume detached for volume \"rabbitmq-confd\" (UniqueName: \"kubernetes.io/projected/9d2e4b3b-6b0f-4e92-a6bb-fa0df37feb28-rabbitmq-confd\") on node \"crc\" DevicePath \"\"" Dec 03 13:39:15 crc kubenswrapper[5052]: I1203 13:39:15.366668 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"pvc-10491a20-0ef4-41c6-b028-50667b3c5171\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-10491a20-0ef4-41c6-b028-50667b3c5171\") on node \"crc\" " Dec 03 13:39:15 crc kubenswrapper[5052]: I1203 13:39:15.366682 5052 reconciler_common.go:293] "Volume detached for volume \"erlang-cookie-secret\" (UniqueName: \"kubernetes.io/secret/9d2e4b3b-6b0f-4e92-a6bb-fa0df37feb28-erlang-cookie-secret\") on node \"crc\" DevicePath \"\"" Dec 03 13:39:15 crc kubenswrapper[5052]: I1203 13:39:15.366693 5052 reconciler_common.go:293] "Volume detached for volume \"rabbitmq-plugins\" (UniqueName: \"kubernetes.io/empty-dir/9d2e4b3b-6b0f-4e92-a6bb-fa0df37feb28-rabbitmq-plugins\") on node \"crc\" DevicePath \"\"" Dec 03 13:39:15 crc kubenswrapper[5052]: I1203 13:39:15.366702 5052 reconciler_common.go:293] "Volume detached for volume \"rabbitmq-erlang-cookie\" (UniqueName: \"kubernetes.io/empty-dir/9d2e4b3b-6b0f-4e92-a6bb-fa0df37feb28-rabbitmq-erlang-cookie\") on node \"crc\" DevicePath \"\"" Dec 03 13:39:15 crc kubenswrapper[5052]: I1203 13:39:15.366716 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-hzlgb\" (UniqueName: \"kubernetes.io/projected/9d2e4b3b-6b0f-4e92-a6bb-fa0df37feb28-kube-api-access-hzlgb\") on node \"crc\" DevicePath \"\"" Dec 03 13:39:15 crc kubenswrapper[5052]: I1203 13:39:15.382617 5052 csi_attacher.go:630] kubernetes.io/csi: attacher.UnmountDevice STAGE_UNSTAGE_VOLUME capability not set. Skipping UnmountDevice... Dec 03 13:39:15 crc kubenswrapper[5052]: I1203 13:39:15.382793 5052 operation_generator.go:917] UnmountDevice succeeded for volume "pvc-10491a20-0ef4-41c6-b028-50667b3c5171" (UniqueName: "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-10491a20-0ef4-41c6-b028-50667b3c5171") on node "crc" Dec 03 13:39:15 crc kubenswrapper[5052]: I1203 13:39:15.472392 5052 reconciler_common.go:293] "Volume detached for volume \"pvc-10491a20-0ef4-41c6-b028-50667b3c5171\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-10491a20-0ef4-41c6-b028-50667b3c5171\") on node \"crc\" DevicePath \"\"" Dec 03 13:39:15 crc kubenswrapper[5052]: I1203 13:39:15.599790 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="keystone-kuttl-tests/openstack-galera-0" podUID="36e1b611-408e-4cc3-8531-03adcac84683" containerName="galera" containerID="cri-o://97588a63cef8ea1ec768dbe56c144e30ff801fcf2bf63f638e5f9b770d79693d" gracePeriod=26 Dec 03 13:39:15 crc kubenswrapper[5052]: I1203 13:39:15.616786 5052 generic.go:334] "Generic (PLEG): container finished" podID="9d2e4b3b-6b0f-4e92-a6bb-fa0df37feb28" containerID="64166ff7fe7a645af18c176c3f4902178d3aa89fc21f02ad0a4020f06c1200e0" exitCode=0 Dec 03 13:39:15 crc kubenswrapper[5052]: I1203 13:39:15.616857 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="keystone-kuttl-tests/rabbitmq-server-0" event={"ID":"9d2e4b3b-6b0f-4e92-a6bb-fa0df37feb28","Type":"ContainerDied","Data":"64166ff7fe7a645af18c176c3f4902178d3aa89fc21f02ad0a4020f06c1200e0"} Dec 03 13:39:15 crc kubenswrapper[5052]: I1203 13:39:15.616893 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="keystone-kuttl-tests/rabbitmq-server-0" event={"ID":"9d2e4b3b-6b0f-4e92-a6bb-fa0df37feb28","Type":"ContainerDied","Data":"ed2fc97665e06cef2aa063ab759091835ea52d8037ffdfdf0f95ee224cb95dd8"} Dec 03 13:39:15 crc kubenswrapper[5052]: I1203 13:39:15.616914 5052 scope.go:117] "RemoveContainer" containerID="64166ff7fe7a645af18c176c3f4902178d3aa89fc21f02ad0a4020f06c1200e0" Dec 03 13:39:15 crc kubenswrapper[5052]: I1203 13:39:15.617035 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="keystone-kuttl-tests/rabbitmq-server-0" Dec 03 13:39:15 crc kubenswrapper[5052]: I1203 13:39:15.630476 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-operators/keystone-operator-controller-manager-6f4cbcd5b5-8gmln" Dec 03 13:39:15 crc kubenswrapper[5052]: I1203 13:39:15.630474 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/keystone-operator-controller-manager-6f4cbcd5b5-8gmln" event={"ID":"ae81dcbe-7c90-4de6-94bd-014eb4e9557f","Type":"ContainerDied","Data":"c40fff398775ad8ba60b54de93d6068e174d54988001c53e16b6e2c6387e0579"} Dec 03 13:39:15 crc kubenswrapper[5052]: I1203 13:39:15.639837 5052 generic.go:334] "Generic (PLEG): container finished" podID="da70419b-4f9e-405d-8fe7-7dbca0ca315e" containerID="2d6aaf797a5a705bce7467f9106b54a4a7ecb16c29f11a578ba19225c543f84d" exitCode=0 Dec 03 13:39:15 crc kubenswrapper[5052]: I1203 13:39:15.639882 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-operators/keystone-operator-index-9ddkd" Dec 03 13:39:15 crc kubenswrapper[5052]: I1203 13:39:15.639927 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/keystone-operator-index-9ddkd" event={"ID":"da70419b-4f9e-405d-8fe7-7dbca0ca315e","Type":"ContainerDied","Data":"2d6aaf797a5a705bce7467f9106b54a4a7ecb16c29f11a578ba19225c543f84d"} Dec 03 13:39:15 crc kubenswrapper[5052]: I1203 13:39:15.639982 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/keystone-operator-index-9ddkd" event={"ID":"da70419b-4f9e-405d-8fe7-7dbca0ca315e","Type":"ContainerDied","Data":"344127d9109e2627417aeec60b2a8b1ccd36c41d207950c72c645e3b00415152"} Dec 03 13:39:15 crc kubenswrapper[5052]: I1203 13:39:15.644853 5052 generic.go:334] "Generic (PLEG): container finished" podID="0ea9b78b-2beb-4bdb-852c-1b92b9638551" containerID="2ebd4f7e1d8ef53cddce7b2e5ce181643f18fc1958462225a56999f9b3633b6b" exitCode=0 Dec 03 13:39:15 crc kubenswrapper[5052]: I1203 13:39:15.644892 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="keystone-kuttl-tests/openstack-galera-1" event={"ID":"0ea9b78b-2beb-4bdb-852c-1b92b9638551","Type":"ContainerDied","Data":"2ebd4f7e1d8ef53cddce7b2e5ce181643f18fc1958462225a56999f9b3633b6b"} Dec 03 13:39:15 crc kubenswrapper[5052]: I1203 13:39:15.681989 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6a7ab332-9572-45d1-bec6-0dab8e148eb0" path="/var/lib/kubelet/pods/6a7ab332-9572-45d1-bec6-0dab8e148eb0/volumes" Dec 03 13:39:15 crc kubenswrapper[5052]: I1203 13:39:15.682924 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f72c36e0-b6bc-412c-887f-3e1394cf6c1f" path="/var/lib/kubelet/pods/f72c36e0-b6bc-412c-887f-3e1394cf6c1f/volumes" Dec 03 13:39:15 crc kubenswrapper[5052]: I1203 13:39:15.694871 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="keystone-kuttl-tests/openstack-galera-1" Dec 03 13:39:15 crc kubenswrapper[5052]: I1203 13:39:15.695627 5052 scope.go:117] "RemoveContainer" containerID="3807b3fde4c670a420fbb539c30bdd9519a330175de6eda8aac8288b9c2663b4" Dec 03 13:39:15 crc kubenswrapper[5052]: I1203 13:39:15.714040 5052 scope.go:117] "RemoveContainer" containerID="64166ff7fe7a645af18c176c3f4902178d3aa89fc21f02ad0a4020f06c1200e0" Dec 03 13:39:15 crc kubenswrapper[5052]: E1203 13:39:15.714547 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"64166ff7fe7a645af18c176c3f4902178d3aa89fc21f02ad0a4020f06c1200e0\": container with ID starting with 64166ff7fe7a645af18c176c3f4902178d3aa89fc21f02ad0a4020f06c1200e0 not found: ID does not exist" containerID="64166ff7fe7a645af18c176c3f4902178d3aa89fc21f02ad0a4020f06c1200e0" Dec 03 13:39:15 crc kubenswrapper[5052]: I1203 13:39:15.714585 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"64166ff7fe7a645af18c176c3f4902178d3aa89fc21f02ad0a4020f06c1200e0"} err="failed to get container status \"64166ff7fe7a645af18c176c3f4902178d3aa89fc21f02ad0a4020f06c1200e0\": rpc error: code = NotFound desc = could not find container \"64166ff7fe7a645af18c176c3f4902178d3aa89fc21f02ad0a4020f06c1200e0\": container with ID starting with 64166ff7fe7a645af18c176c3f4902178d3aa89fc21f02ad0a4020f06c1200e0 not found: ID does not exist" Dec 03 13:39:15 crc kubenswrapper[5052]: I1203 13:39:15.714616 5052 scope.go:117] "RemoveContainer" containerID="3807b3fde4c670a420fbb539c30bdd9519a330175de6eda8aac8288b9c2663b4" Dec 03 13:39:15 crc kubenswrapper[5052]: E1203 13:39:15.715180 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3807b3fde4c670a420fbb539c30bdd9519a330175de6eda8aac8288b9c2663b4\": container with ID starting with 3807b3fde4c670a420fbb539c30bdd9519a330175de6eda8aac8288b9c2663b4 not found: ID does not exist" containerID="3807b3fde4c670a420fbb539c30bdd9519a330175de6eda8aac8288b9c2663b4" Dec 03 13:39:15 crc kubenswrapper[5052]: I1203 13:39:15.715201 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3807b3fde4c670a420fbb539c30bdd9519a330175de6eda8aac8288b9c2663b4"} err="failed to get container status \"3807b3fde4c670a420fbb539c30bdd9519a330175de6eda8aac8288b9c2663b4\": rpc error: code = NotFound desc = could not find container \"3807b3fde4c670a420fbb539c30bdd9519a330175de6eda8aac8288b9c2663b4\": container with ID starting with 3807b3fde4c670a420fbb539c30bdd9519a330175de6eda8aac8288b9c2663b4 not found: ID does not exist" Dec 03 13:39:15 crc kubenswrapper[5052]: I1203 13:39:15.715215 5052 scope.go:117] "RemoveContainer" containerID="deb1f298b409739ff6b92cb257ecf5946876b50d914e7262651ac96341518d6e" Dec 03 13:39:15 crc kubenswrapper[5052]: I1203 13:39:15.754821 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["keystone-kuttl-tests/rabbitmq-server-0"] Dec 03 13:39:15 crc kubenswrapper[5052]: I1203 13:39:15.761327 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["keystone-kuttl-tests/rabbitmq-server-0"] Dec 03 13:39:15 crc kubenswrapper[5052]: I1203 13:39:15.764655 5052 scope.go:117] "RemoveContainer" containerID="2d6aaf797a5a705bce7467f9106b54a4a7ecb16c29f11a578ba19225c543f84d" Dec 03 13:39:15 crc kubenswrapper[5052]: I1203 13:39:15.774079 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-operators/keystone-operator-index-9ddkd"] Dec 03 13:39:15 crc kubenswrapper[5052]: I1203 13:39:15.777934 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/0ea9b78b-2beb-4bdb-852c-1b92b9638551-config-data-default\") pod \"0ea9b78b-2beb-4bdb-852c-1b92b9638551\" (UID: \"0ea9b78b-2beb-4bdb-852c-1b92b9638551\") " Dec 03 13:39:15 crc kubenswrapper[5052]: I1203 13:39:15.777981 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-27nwd\" (UniqueName: \"kubernetes.io/projected/0ea9b78b-2beb-4bdb-852c-1b92b9638551-kube-api-access-27nwd\") pod \"0ea9b78b-2beb-4bdb-852c-1b92b9638551\" (UID: \"0ea9b78b-2beb-4bdb-852c-1b92b9638551\") " Dec 03 13:39:15 crc kubenswrapper[5052]: I1203 13:39:15.778040 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/0ea9b78b-2beb-4bdb-852c-1b92b9638551-config-data-generated\") pod \"0ea9b78b-2beb-4bdb-852c-1b92b9638551\" (UID: \"0ea9b78b-2beb-4bdb-852c-1b92b9638551\") " Dec 03 13:39:15 crc kubenswrapper[5052]: I1203 13:39:15.778142 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/0ea9b78b-2beb-4bdb-852c-1b92b9638551-operator-scripts\") pod \"0ea9b78b-2beb-4bdb-852c-1b92b9638551\" (UID: \"0ea9b78b-2beb-4bdb-852c-1b92b9638551\") " Dec 03 13:39:15 crc kubenswrapper[5052]: I1203 13:39:15.778191 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"mysql-db\" (UniqueName: \"kubernetes.io/local-volume/local-storage01-crc\") pod \"0ea9b78b-2beb-4bdb-852c-1b92b9638551\" (UID: \"0ea9b78b-2beb-4bdb-852c-1b92b9638551\") " Dec 03 13:39:15 crc kubenswrapper[5052]: I1203 13:39:15.778227 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/0ea9b78b-2beb-4bdb-852c-1b92b9638551-kolla-config\") pod \"0ea9b78b-2beb-4bdb-852c-1b92b9638551\" (UID: \"0ea9b78b-2beb-4bdb-852c-1b92b9638551\") " Dec 03 13:39:15 crc kubenswrapper[5052]: I1203 13:39:15.779416 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0ea9b78b-2beb-4bdb-852c-1b92b9638551-kolla-config" (OuterVolumeSpecName: "kolla-config") pod "0ea9b78b-2beb-4bdb-852c-1b92b9638551" (UID: "0ea9b78b-2beb-4bdb-852c-1b92b9638551"). InnerVolumeSpecName "kolla-config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:39:15 crc kubenswrapper[5052]: I1203 13:39:15.779854 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0ea9b78b-2beb-4bdb-852c-1b92b9638551-config-data-default" (OuterVolumeSpecName: "config-data-default") pod "0ea9b78b-2beb-4bdb-852c-1b92b9638551" (UID: "0ea9b78b-2beb-4bdb-852c-1b92b9638551"). InnerVolumeSpecName "config-data-default". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:39:15 crc kubenswrapper[5052]: I1203 13:39:15.779906 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0ea9b78b-2beb-4bdb-852c-1b92b9638551-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "0ea9b78b-2beb-4bdb-852c-1b92b9638551" (UID: "0ea9b78b-2beb-4bdb-852c-1b92b9638551"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:39:15 crc kubenswrapper[5052]: I1203 13:39:15.779961 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ea9b78b-2beb-4bdb-852c-1b92b9638551-config-data-generated" (OuterVolumeSpecName: "config-data-generated") pod "0ea9b78b-2beb-4bdb-852c-1b92b9638551" (UID: "0ea9b78b-2beb-4bdb-852c-1b92b9638551"). InnerVolumeSpecName "config-data-generated". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 13:39:15 crc kubenswrapper[5052]: I1203 13:39:15.782317 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-operators/keystone-operator-index-9ddkd"] Dec 03 13:39:15 crc kubenswrapper[5052]: I1203 13:39:15.782649 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0ea9b78b-2beb-4bdb-852c-1b92b9638551-kube-api-access-27nwd" (OuterVolumeSpecName: "kube-api-access-27nwd") pod "0ea9b78b-2beb-4bdb-852c-1b92b9638551" (UID: "0ea9b78b-2beb-4bdb-852c-1b92b9638551"). InnerVolumeSpecName "kube-api-access-27nwd". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:39:15 crc kubenswrapper[5052]: I1203 13:39:15.786279 5052 scope.go:117] "RemoveContainer" containerID="2d6aaf797a5a705bce7467f9106b54a4a7ecb16c29f11a578ba19225c543f84d" Dec 03 13:39:15 crc kubenswrapper[5052]: I1203 13:39:15.786754 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-operators/keystone-operator-controller-manager-6f4cbcd5b5-8gmln"] Dec 03 13:39:15 crc kubenswrapper[5052]: E1203 13:39:15.786754 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2d6aaf797a5a705bce7467f9106b54a4a7ecb16c29f11a578ba19225c543f84d\": container with ID starting with 2d6aaf797a5a705bce7467f9106b54a4a7ecb16c29f11a578ba19225c543f84d not found: ID does not exist" containerID="2d6aaf797a5a705bce7467f9106b54a4a7ecb16c29f11a578ba19225c543f84d" Dec 03 13:39:15 crc kubenswrapper[5052]: I1203 13:39:15.786834 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d6aaf797a5a705bce7467f9106b54a4a7ecb16c29f11a578ba19225c543f84d"} err="failed to get container status \"2d6aaf797a5a705bce7467f9106b54a4a7ecb16c29f11a578ba19225c543f84d\": rpc error: code = NotFound desc = could not find container \"2d6aaf797a5a705bce7467f9106b54a4a7ecb16c29f11a578ba19225c543f84d\": container with ID starting with 2d6aaf797a5a705bce7467f9106b54a4a7ecb16c29f11a578ba19225c543f84d not found: ID does not exist" Dec 03 13:39:15 crc kubenswrapper[5052]: I1203 13:39:15.792005 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-operators/keystone-operator-controller-manager-6f4cbcd5b5-8gmln"] Dec 03 13:39:15 crc kubenswrapper[5052]: I1203 13:39:15.800320 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage01-crc" (OuterVolumeSpecName: "mysql-db") pod "0ea9b78b-2beb-4bdb-852c-1b92b9638551" (UID: "0ea9b78b-2beb-4bdb-852c-1b92b9638551"). InnerVolumeSpecName "local-storage01-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 13:39:15 crc kubenswrapper[5052]: I1203 13:39:15.880427 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/0ea9b78b-2beb-4bdb-852c-1b92b9638551-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 13:39:15 crc kubenswrapper[5052]: I1203 13:39:15.880504 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage01-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage01-crc\") on node \"crc\" " Dec 03 13:39:15 crc kubenswrapper[5052]: I1203 13:39:15.880545 5052 reconciler_common.go:293] "Volume detached for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/0ea9b78b-2beb-4bdb-852c-1b92b9638551-kolla-config\") on node \"crc\" DevicePath \"\"" Dec 03 13:39:15 crc kubenswrapper[5052]: I1203 13:39:15.880564 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/0ea9b78b-2beb-4bdb-852c-1b92b9638551-config-data-default\") on node \"crc\" DevicePath \"\"" Dec 03 13:39:15 crc kubenswrapper[5052]: I1203 13:39:15.880579 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-27nwd\" (UniqueName: \"kubernetes.io/projected/0ea9b78b-2beb-4bdb-852c-1b92b9638551-kube-api-access-27nwd\") on node \"crc\" DevicePath \"\"" Dec 03 13:39:15 crc kubenswrapper[5052]: I1203 13:39:15.880593 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/0ea9b78b-2beb-4bdb-852c-1b92b9638551-config-data-generated\") on node \"crc\" DevicePath \"\"" Dec 03 13:39:15 crc kubenswrapper[5052]: I1203 13:39:15.891456 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage01-crc" (UniqueName: "kubernetes.io/local-volume/local-storage01-crc") on node "crc" Dec 03 13:39:15 crc kubenswrapper[5052]: I1203 13:39:15.981667 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage01-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage01-crc\") on node \"crc\" DevicePath \"\"" Dec 03 13:39:16 crc kubenswrapper[5052]: I1203 13:39:16.241777 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="keystone-kuttl-tests/openstack-galera-0" Dec 03 13:39:16 crc kubenswrapper[5052]: I1203 13:39:16.389286 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/36e1b611-408e-4cc3-8531-03adcac84683-operator-scripts\") pod \"36e1b611-408e-4cc3-8531-03adcac84683\" (UID: \"36e1b611-408e-4cc3-8531-03adcac84683\") " Dec 03 13:39:16 crc kubenswrapper[5052]: I1203 13:39:16.389365 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/36e1b611-408e-4cc3-8531-03adcac84683-config-data-default\") pod \"36e1b611-408e-4cc3-8531-03adcac84683\" (UID: \"36e1b611-408e-4cc3-8531-03adcac84683\") " Dec 03 13:39:16 crc kubenswrapper[5052]: I1203 13:39:16.389392 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2lf8x\" (UniqueName: \"kubernetes.io/projected/36e1b611-408e-4cc3-8531-03adcac84683-kube-api-access-2lf8x\") pod \"36e1b611-408e-4cc3-8531-03adcac84683\" (UID: \"36e1b611-408e-4cc3-8531-03adcac84683\") " Dec 03 13:39:16 crc kubenswrapper[5052]: I1203 13:39:16.389411 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"mysql-db\" (UniqueName: \"kubernetes.io/local-volume/local-storage13-crc\") pod \"36e1b611-408e-4cc3-8531-03adcac84683\" (UID: \"36e1b611-408e-4cc3-8531-03adcac84683\") " Dec 03 13:39:16 crc kubenswrapper[5052]: I1203 13:39:16.389441 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/36e1b611-408e-4cc3-8531-03adcac84683-kolla-config\") pod \"36e1b611-408e-4cc3-8531-03adcac84683\" (UID: \"36e1b611-408e-4cc3-8531-03adcac84683\") " Dec 03 13:39:16 crc kubenswrapper[5052]: I1203 13:39:16.389471 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/36e1b611-408e-4cc3-8531-03adcac84683-config-data-generated\") pod \"36e1b611-408e-4cc3-8531-03adcac84683\" (UID: \"36e1b611-408e-4cc3-8531-03adcac84683\") " Dec 03 13:39:16 crc kubenswrapper[5052]: I1203 13:39:16.390122 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/36e1b611-408e-4cc3-8531-03adcac84683-kolla-config" (OuterVolumeSpecName: "kolla-config") pod "36e1b611-408e-4cc3-8531-03adcac84683" (UID: "36e1b611-408e-4cc3-8531-03adcac84683"). InnerVolumeSpecName "kolla-config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:39:16 crc kubenswrapper[5052]: I1203 13:39:16.390273 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36e1b611-408e-4cc3-8531-03adcac84683-config-data-generated" (OuterVolumeSpecName: "config-data-generated") pod "36e1b611-408e-4cc3-8531-03adcac84683" (UID: "36e1b611-408e-4cc3-8531-03adcac84683"). InnerVolumeSpecName "config-data-generated". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 13:39:16 crc kubenswrapper[5052]: I1203 13:39:16.390292 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/36e1b611-408e-4cc3-8531-03adcac84683-config-data-default" (OuterVolumeSpecName: "config-data-default") pod "36e1b611-408e-4cc3-8531-03adcac84683" (UID: "36e1b611-408e-4cc3-8531-03adcac84683"). InnerVolumeSpecName "config-data-default". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:39:16 crc kubenswrapper[5052]: I1203 13:39:16.390449 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/36e1b611-408e-4cc3-8531-03adcac84683-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "36e1b611-408e-4cc3-8531-03adcac84683" (UID: "36e1b611-408e-4cc3-8531-03adcac84683"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:39:16 crc kubenswrapper[5052]: I1203 13:39:16.393476 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/36e1b611-408e-4cc3-8531-03adcac84683-kube-api-access-2lf8x" (OuterVolumeSpecName: "kube-api-access-2lf8x") pod "36e1b611-408e-4cc3-8531-03adcac84683" (UID: "36e1b611-408e-4cc3-8531-03adcac84683"). InnerVolumeSpecName "kube-api-access-2lf8x". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:39:16 crc kubenswrapper[5052]: I1203 13:39:16.397993 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage13-crc" (OuterVolumeSpecName: "mysql-db") pod "36e1b611-408e-4cc3-8531-03adcac84683" (UID: "36e1b611-408e-4cc3-8531-03adcac84683"). InnerVolumeSpecName "local-storage13-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 13:39:16 crc kubenswrapper[5052]: I1203 13:39:16.491407 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/36e1b611-408e-4cc3-8531-03adcac84683-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 13:39:16 crc kubenswrapper[5052]: I1203 13:39:16.491801 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/36e1b611-408e-4cc3-8531-03adcac84683-config-data-default\") on node \"crc\" DevicePath \"\"" Dec 03 13:39:16 crc kubenswrapper[5052]: I1203 13:39:16.491820 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-2lf8x\" (UniqueName: \"kubernetes.io/projected/36e1b611-408e-4cc3-8531-03adcac84683-kube-api-access-2lf8x\") on node \"crc\" DevicePath \"\"" Dec 03 13:39:16 crc kubenswrapper[5052]: I1203 13:39:16.491862 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage13-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage13-crc\") on node \"crc\" " Dec 03 13:39:16 crc kubenswrapper[5052]: I1203 13:39:16.491876 5052 reconciler_common.go:293] "Volume detached for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/36e1b611-408e-4cc3-8531-03adcac84683-kolla-config\") on node \"crc\" DevicePath \"\"" Dec 03 13:39:16 crc kubenswrapper[5052]: I1203 13:39:16.491890 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/36e1b611-408e-4cc3-8531-03adcac84683-config-data-generated\") on node \"crc\" DevicePath \"\"" Dec 03 13:39:16 crc kubenswrapper[5052]: I1203 13:39:16.507858 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage13-crc" (UniqueName: "kubernetes.io/local-volume/local-storage13-crc") on node "crc" Dec 03 13:39:16 crc kubenswrapper[5052]: I1203 13:39:16.592969 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage13-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage13-crc\") on node \"crc\" DevicePath \"\"" Dec 03 13:39:16 crc kubenswrapper[5052]: I1203 13:39:16.653748 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="keystone-kuttl-tests/openstack-galera-1" event={"ID":"0ea9b78b-2beb-4bdb-852c-1b92b9638551","Type":"ContainerDied","Data":"1182de3d450f8cac1f36dfe05e9f539509003033d9b4de58373f045a1bb7ea2f"} Dec 03 13:39:16 crc kubenswrapper[5052]: I1203 13:39:16.653769 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="keystone-kuttl-tests/openstack-galera-1" Dec 03 13:39:16 crc kubenswrapper[5052]: I1203 13:39:16.653798 5052 scope.go:117] "RemoveContainer" containerID="2ebd4f7e1d8ef53cddce7b2e5ce181643f18fc1958462225a56999f9b3633b6b" Dec 03 13:39:16 crc kubenswrapper[5052]: I1203 13:39:16.658555 5052 generic.go:334] "Generic (PLEG): container finished" podID="36e1b611-408e-4cc3-8531-03adcac84683" containerID="97588a63cef8ea1ec768dbe56c144e30ff801fcf2bf63f638e5f9b770d79693d" exitCode=0 Dec 03 13:39:16 crc kubenswrapper[5052]: I1203 13:39:16.658596 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="keystone-kuttl-tests/openstack-galera-0" Dec 03 13:39:16 crc kubenswrapper[5052]: I1203 13:39:16.658611 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="keystone-kuttl-tests/openstack-galera-0" event={"ID":"36e1b611-408e-4cc3-8531-03adcac84683","Type":"ContainerDied","Data":"97588a63cef8ea1ec768dbe56c144e30ff801fcf2bf63f638e5f9b770d79693d"} Dec 03 13:39:16 crc kubenswrapper[5052]: I1203 13:39:16.658676 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="keystone-kuttl-tests/openstack-galera-0" event={"ID":"36e1b611-408e-4cc3-8531-03adcac84683","Type":"ContainerDied","Data":"9d071802138ee9f0d7d75122d8fa0e3d571d088f32c0815f4cb6cb7b2dcf2c2a"} Dec 03 13:39:16 crc kubenswrapper[5052]: I1203 13:39:16.676861 5052 scope.go:117] "RemoveContainer" containerID="4c382ef0474c9229a0a4e17621c93a48f914490ac61a39d27ed7408ea6100c9f" Dec 03 13:39:16 crc kubenswrapper[5052]: I1203 13:39:16.694169 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["keystone-kuttl-tests/openstack-galera-0"] Dec 03 13:39:16 crc kubenswrapper[5052]: I1203 13:39:16.702121 5052 scope.go:117] "RemoveContainer" containerID="97588a63cef8ea1ec768dbe56c144e30ff801fcf2bf63f638e5f9b770d79693d" Dec 03 13:39:16 crc kubenswrapper[5052]: I1203 13:39:16.703617 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["keystone-kuttl-tests/openstack-galera-0"] Dec 03 13:39:16 crc kubenswrapper[5052]: I1203 13:39:16.715610 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["keystone-kuttl-tests/openstack-galera-1"] Dec 03 13:39:16 crc kubenswrapper[5052]: I1203 13:39:16.740837 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["keystone-kuttl-tests/openstack-galera-1"] Dec 03 13:39:16 crc kubenswrapper[5052]: I1203 13:39:16.749192 5052 scope.go:117] "RemoveContainer" containerID="fbba02a1d00fb616f73bb42e9d24b9e28e4932a3a8df0503d4d09b3d0f37d171" Dec 03 13:39:16 crc kubenswrapper[5052]: I1203 13:39:16.773871 5052 scope.go:117] "RemoveContainer" containerID="97588a63cef8ea1ec768dbe56c144e30ff801fcf2bf63f638e5f9b770d79693d" Dec 03 13:39:16 crc kubenswrapper[5052]: E1203 13:39:16.774820 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"97588a63cef8ea1ec768dbe56c144e30ff801fcf2bf63f638e5f9b770d79693d\": container with ID starting with 97588a63cef8ea1ec768dbe56c144e30ff801fcf2bf63f638e5f9b770d79693d not found: ID does not exist" containerID="97588a63cef8ea1ec768dbe56c144e30ff801fcf2bf63f638e5f9b770d79693d" Dec 03 13:39:16 crc kubenswrapper[5052]: I1203 13:39:16.774889 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"97588a63cef8ea1ec768dbe56c144e30ff801fcf2bf63f638e5f9b770d79693d"} err="failed to get container status \"97588a63cef8ea1ec768dbe56c144e30ff801fcf2bf63f638e5f9b770d79693d\": rpc error: code = NotFound desc = could not find container \"97588a63cef8ea1ec768dbe56c144e30ff801fcf2bf63f638e5f9b770d79693d\": container with ID starting with 97588a63cef8ea1ec768dbe56c144e30ff801fcf2bf63f638e5f9b770d79693d not found: ID does not exist" Dec 03 13:39:16 crc kubenswrapper[5052]: I1203 13:39:16.774919 5052 scope.go:117] "RemoveContainer" containerID="fbba02a1d00fb616f73bb42e9d24b9e28e4932a3a8df0503d4d09b3d0f37d171" Dec 03 13:39:16 crc kubenswrapper[5052]: E1203 13:39:16.775725 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fbba02a1d00fb616f73bb42e9d24b9e28e4932a3a8df0503d4d09b3d0f37d171\": container with ID starting with fbba02a1d00fb616f73bb42e9d24b9e28e4932a3a8df0503d4d09b3d0f37d171 not found: ID does not exist" containerID="fbba02a1d00fb616f73bb42e9d24b9e28e4932a3a8df0503d4d09b3d0f37d171" Dec 03 13:39:16 crc kubenswrapper[5052]: I1203 13:39:16.775769 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fbba02a1d00fb616f73bb42e9d24b9e28e4932a3a8df0503d4d09b3d0f37d171"} err="failed to get container status \"fbba02a1d00fb616f73bb42e9d24b9e28e4932a3a8df0503d4d09b3d0f37d171\": rpc error: code = NotFound desc = could not find container \"fbba02a1d00fb616f73bb42e9d24b9e28e4932a3a8df0503d4d09b3d0f37d171\": container with ID starting with fbba02a1d00fb616f73bb42e9d24b9e28e4932a3a8df0503d4d09b3d0f37d171 not found: ID does not exist" Dec 03 13:39:17 crc kubenswrapper[5052]: I1203 13:39:17.014117 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-operators/infra-operator-controller-manager-6f9d567ccb-lth55"] Dec 03 13:39:17 crc kubenswrapper[5052]: I1203 13:39:17.014346 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-operators/infra-operator-controller-manager-6f9d567ccb-lth55" podUID="4dd99d7d-b7ff-4ba7-bf73-5e66e3e500b7" containerName="manager" containerID="cri-o://8e4b737cdfe5a62b1eea4be10d7ae8408ed9d5db9eb9e377b74b41e783c9bec7" gracePeriod=10 Dec 03 13:39:17 crc kubenswrapper[5052]: I1203 13:39:17.014449 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-operators/infra-operator-controller-manager-6f9d567ccb-lth55" podUID="4dd99d7d-b7ff-4ba7-bf73-5e66e3e500b7" containerName="kube-rbac-proxy" containerID="cri-o://152450c0accaa40ece4dc43da5d14b974063c658a420e0a00dae85acf98002de" gracePeriod=10 Dec 03 13:39:17 crc kubenswrapper[5052]: I1203 13:39:17.286032 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-operators/infra-operator-index-2dtcx"] Dec 03 13:39:17 crc kubenswrapper[5052]: I1203 13:39:17.286232 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-operators/infra-operator-index-2dtcx" podUID="75a90895-a53c-4c1f-8d5b-fd859d61a681" containerName="registry-server" containerID="cri-o://36bea13f9e88148b806bf93382abfb0f19013d1939bd07876c9afc32b9057387" gracePeriod=30 Dec 03 13:39:17 crc kubenswrapper[5052]: I1203 13:39:17.337443 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-operators/5d473c3169f40b179d14921c90af2c8546b7b757fe551b7dba7d903f5d28bdz"] Dec 03 13:39:17 crc kubenswrapper[5052]: I1203 13:39:17.358277 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-operators/5d473c3169f40b179d14921c90af2c8546b7b757fe551b7dba7d903f5d28bdz"] Dec 03 13:39:17 crc kubenswrapper[5052]: I1203 13:39:17.481746 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-operators/infra-operator-controller-manager-6f9d567ccb-lth55" Dec 03 13:39:17 crc kubenswrapper[5052]: I1203 13:39:17.618684 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4dd99d7d-b7ff-4ba7-bf73-5e66e3e500b7-webhook-cert\") pod \"4dd99d7d-b7ff-4ba7-bf73-5e66e3e500b7\" (UID: \"4dd99d7d-b7ff-4ba7-bf73-5e66e3e500b7\") " Dec 03 13:39:17 crc kubenswrapper[5052]: I1203 13:39:17.618758 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hc6qm\" (UniqueName: \"kubernetes.io/projected/4dd99d7d-b7ff-4ba7-bf73-5e66e3e500b7-kube-api-access-hc6qm\") pod \"4dd99d7d-b7ff-4ba7-bf73-5e66e3e500b7\" (UID: \"4dd99d7d-b7ff-4ba7-bf73-5e66e3e500b7\") " Dec 03 13:39:17 crc kubenswrapper[5052]: I1203 13:39:17.618866 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/4dd99d7d-b7ff-4ba7-bf73-5e66e3e500b7-apiservice-cert\") pod \"4dd99d7d-b7ff-4ba7-bf73-5e66e3e500b7\" (UID: \"4dd99d7d-b7ff-4ba7-bf73-5e66e3e500b7\") " Dec 03 13:39:17 crc kubenswrapper[5052]: I1203 13:39:17.625925 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4dd99d7d-b7ff-4ba7-bf73-5e66e3e500b7-apiservice-cert" (OuterVolumeSpecName: "apiservice-cert") pod "4dd99d7d-b7ff-4ba7-bf73-5e66e3e500b7" (UID: "4dd99d7d-b7ff-4ba7-bf73-5e66e3e500b7"). InnerVolumeSpecName "apiservice-cert". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:39:17 crc kubenswrapper[5052]: I1203 13:39:17.625908 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4dd99d7d-b7ff-4ba7-bf73-5e66e3e500b7-kube-api-access-hc6qm" (OuterVolumeSpecName: "kube-api-access-hc6qm") pod "4dd99d7d-b7ff-4ba7-bf73-5e66e3e500b7" (UID: "4dd99d7d-b7ff-4ba7-bf73-5e66e3e500b7"). InnerVolumeSpecName "kube-api-access-hc6qm". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:39:17 crc kubenswrapper[5052]: I1203 13:39:17.626030 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4dd99d7d-b7ff-4ba7-bf73-5e66e3e500b7-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "4dd99d7d-b7ff-4ba7-bf73-5e66e3e500b7" (UID: "4dd99d7d-b7ff-4ba7-bf73-5e66e3e500b7"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:39:17 crc kubenswrapper[5052]: I1203 13:39:17.670358 5052 generic.go:334] "Generic (PLEG): container finished" podID="4dd99d7d-b7ff-4ba7-bf73-5e66e3e500b7" containerID="152450c0accaa40ece4dc43da5d14b974063c658a420e0a00dae85acf98002de" exitCode=0 Dec 03 13:39:17 crc kubenswrapper[5052]: I1203 13:39:17.670423 5052 generic.go:334] "Generic (PLEG): container finished" podID="4dd99d7d-b7ff-4ba7-bf73-5e66e3e500b7" containerID="8e4b737cdfe5a62b1eea4be10d7ae8408ed9d5db9eb9e377b74b41e783c9bec7" exitCode=0 Dec 03 13:39:17 crc kubenswrapper[5052]: I1203 13:39:17.670461 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/infra-operator-controller-manager-6f9d567ccb-lth55" event={"ID":"4dd99d7d-b7ff-4ba7-bf73-5e66e3e500b7","Type":"ContainerDied","Data":"152450c0accaa40ece4dc43da5d14b974063c658a420e0a00dae85acf98002de"} Dec 03 13:39:17 crc kubenswrapper[5052]: I1203 13:39:17.670486 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/infra-operator-controller-manager-6f9d567ccb-lth55" event={"ID":"4dd99d7d-b7ff-4ba7-bf73-5e66e3e500b7","Type":"ContainerDied","Data":"8e4b737cdfe5a62b1eea4be10d7ae8408ed9d5db9eb9e377b74b41e783c9bec7"} Dec 03 13:39:17 crc kubenswrapper[5052]: I1203 13:39:17.670499 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/infra-operator-controller-manager-6f9d567ccb-lth55" event={"ID":"4dd99d7d-b7ff-4ba7-bf73-5e66e3e500b7","Type":"ContainerDied","Data":"ff1e3219f320766fd15e0d3660ebbf002fdc2bc78689c5ca0227b84da981ed38"} Dec 03 13:39:17 crc kubenswrapper[5052]: I1203 13:39:17.670531 5052 scope.go:117] "RemoveContainer" containerID="152450c0accaa40ece4dc43da5d14b974063c658a420e0a00dae85acf98002de" Dec 03 13:39:17 crc kubenswrapper[5052]: I1203 13:39:17.670653 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-operators/infra-operator-controller-manager-6f9d567ccb-lth55" Dec 03 13:39:17 crc kubenswrapper[5052]: I1203 13:39:17.672446 5052 generic.go:334] "Generic (PLEG): container finished" podID="75a90895-a53c-4c1f-8d5b-fd859d61a681" containerID="36bea13f9e88148b806bf93382abfb0f19013d1939bd07876c9afc32b9057387" exitCode=0 Dec 03 13:39:17 crc kubenswrapper[5052]: I1203 13:39:17.683296 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0ea9b78b-2beb-4bdb-852c-1b92b9638551" path="/var/lib/kubelet/pods/0ea9b78b-2beb-4bdb-852c-1b92b9638551/volumes" Dec 03 13:39:17 crc kubenswrapper[5052]: I1203 13:39:17.684283 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="36e1b611-408e-4cc3-8531-03adcac84683" path="/var/lib/kubelet/pods/36e1b611-408e-4cc3-8531-03adcac84683/volumes" Dec 03 13:39:17 crc kubenswrapper[5052]: I1203 13:39:17.685006 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="862171d5-fa42-4b46-85d7-369b796daae6" path="/var/lib/kubelet/pods/862171d5-fa42-4b46-85d7-369b796daae6/volumes" Dec 03 13:39:17 crc kubenswrapper[5052]: I1203 13:39:17.686287 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9d2e4b3b-6b0f-4e92-a6bb-fa0df37feb28" path="/var/lib/kubelet/pods/9d2e4b3b-6b0f-4e92-a6bb-fa0df37feb28/volumes" Dec 03 13:39:17 crc kubenswrapper[5052]: I1203 13:39:17.686859 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ae81dcbe-7c90-4de6-94bd-014eb4e9557f" path="/var/lib/kubelet/pods/ae81dcbe-7c90-4de6-94bd-014eb4e9557f/volumes" Dec 03 13:39:17 crc kubenswrapper[5052]: I1203 13:39:17.687802 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="da70419b-4f9e-405d-8fe7-7dbca0ca315e" path="/var/lib/kubelet/pods/da70419b-4f9e-405d-8fe7-7dbca0ca315e/volumes" Dec 03 13:39:17 crc kubenswrapper[5052]: I1203 13:39:17.688433 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/infra-operator-index-2dtcx" event={"ID":"75a90895-a53c-4c1f-8d5b-fd859d61a681","Type":"ContainerDied","Data":"36bea13f9e88148b806bf93382abfb0f19013d1939bd07876c9afc32b9057387"} Dec 03 13:39:17 crc kubenswrapper[5052]: I1203 13:39:17.688473 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/infra-operator-index-2dtcx" event={"ID":"75a90895-a53c-4c1f-8d5b-fd859d61a681","Type":"ContainerDied","Data":"2a55ec0d603e2c7bf8960f5e2103d87a4dcc4355900639ea4d2f614536e13871"} Dec 03 13:39:17 crc kubenswrapper[5052]: I1203 13:39:17.688489 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2a55ec0d603e2c7bf8960f5e2103d87a4dcc4355900639ea4d2f614536e13871" Dec 03 13:39:17 crc kubenswrapper[5052]: I1203 13:39:17.710630 5052 scope.go:117] "RemoveContainer" containerID="8e4b737cdfe5a62b1eea4be10d7ae8408ed9d5db9eb9e377b74b41e783c9bec7" Dec 03 13:39:17 crc kubenswrapper[5052]: I1203 13:39:17.710764 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-operators/infra-operator-index-2dtcx" Dec 03 13:39:17 crc kubenswrapper[5052]: I1203 13:39:17.724608 5052 reconciler_common.go:293] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4dd99d7d-b7ff-4ba7-bf73-5e66e3e500b7-webhook-cert\") on node \"crc\" DevicePath \"\"" Dec 03 13:39:17 crc kubenswrapper[5052]: I1203 13:39:17.724643 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-hc6qm\" (UniqueName: \"kubernetes.io/projected/4dd99d7d-b7ff-4ba7-bf73-5e66e3e500b7-kube-api-access-hc6qm\") on node \"crc\" DevicePath \"\"" Dec 03 13:39:17 crc kubenswrapper[5052]: I1203 13:39:17.724653 5052 reconciler_common.go:293] "Volume detached for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/4dd99d7d-b7ff-4ba7-bf73-5e66e3e500b7-apiservice-cert\") on node \"crc\" DevicePath \"\"" Dec 03 13:39:17 crc kubenswrapper[5052]: I1203 13:39:17.726277 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-operators/infra-operator-controller-manager-6f9d567ccb-lth55"] Dec 03 13:39:17 crc kubenswrapper[5052]: I1203 13:39:17.732199 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-operators/infra-operator-controller-manager-6f9d567ccb-lth55"] Dec 03 13:39:17 crc kubenswrapper[5052]: I1203 13:39:17.747466 5052 scope.go:117] "RemoveContainer" containerID="152450c0accaa40ece4dc43da5d14b974063c658a420e0a00dae85acf98002de" Dec 03 13:39:17 crc kubenswrapper[5052]: E1203 13:39:17.748180 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"152450c0accaa40ece4dc43da5d14b974063c658a420e0a00dae85acf98002de\": container with ID starting with 152450c0accaa40ece4dc43da5d14b974063c658a420e0a00dae85acf98002de not found: ID does not exist" containerID="152450c0accaa40ece4dc43da5d14b974063c658a420e0a00dae85acf98002de" Dec 03 13:39:17 crc kubenswrapper[5052]: I1203 13:39:17.748219 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"152450c0accaa40ece4dc43da5d14b974063c658a420e0a00dae85acf98002de"} err="failed to get container status \"152450c0accaa40ece4dc43da5d14b974063c658a420e0a00dae85acf98002de\": rpc error: code = NotFound desc = could not find container \"152450c0accaa40ece4dc43da5d14b974063c658a420e0a00dae85acf98002de\": container with ID starting with 152450c0accaa40ece4dc43da5d14b974063c658a420e0a00dae85acf98002de not found: ID does not exist" Dec 03 13:39:17 crc kubenswrapper[5052]: I1203 13:39:17.748246 5052 scope.go:117] "RemoveContainer" containerID="8e4b737cdfe5a62b1eea4be10d7ae8408ed9d5db9eb9e377b74b41e783c9bec7" Dec 03 13:39:17 crc kubenswrapper[5052]: E1203 13:39:17.748582 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8e4b737cdfe5a62b1eea4be10d7ae8408ed9d5db9eb9e377b74b41e783c9bec7\": container with ID starting with 8e4b737cdfe5a62b1eea4be10d7ae8408ed9d5db9eb9e377b74b41e783c9bec7 not found: ID does not exist" containerID="8e4b737cdfe5a62b1eea4be10d7ae8408ed9d5db9eb9e377b74b41e783c9bec7" Dec 03 13:39:17 crc kubenswrapper[5052]: I1203 13:39:17.748612 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8e4b737cdfe5a62b1eea4be10d7ae8408ed9d5db9eb9e377b74b41e783c9bec7"} err="failed to get container status \"8e4b737cdfe5a62b1eea4be10d7ae8408ed9d5db9eb9e377b74b41e783c9bec7\": rpc error: code = NotFound desc = could not find container \"8e4b737cdfe5a62b1eea4be10d7ae8408ed9d5db9eb9e377b74b41e783c9bec7\": container with ID starting with 8e4b737cdfe5a62b1eea4be10d7ae8408ed9d5db9eb9e377b74b41e783c9bec7 not found: ID does not exist" Dec 03 13:39:17 crc kubenswrapper[5052]: I1203 13:39:17.748625 5052 scope.go:117] "RemoveContainer" containerID="152450c0accaa40ece4dc43da5d14b974063c658a420e0a00dae85acf98002de" Dec 03 13:39:17 crc kubenswrapper[5052]: I1203 13:39:17.748863 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"152450c0accaa40ece4dc43da5d14b974063c658a420e0a00dae85acf98002de"} err="failed to get container status \"152450c0accaa40ece4dc43da5d14b974063c658a420e0a00dae85acf98002de\": rpc error: code = NotFound desc = could not find container \"152450c0accaa40ece4dc43da5d14b974063c658a420e0a00dae85acf98002de\": container with ID starting with 152450c0accaa40ece4dc43da5d14b974063c658a420e0a00dae85acf98002de not found: ID does not exist" Dec 03 13:39:17 crc kubenswrapper[5052]: I1203 13:39:17.748880 5052 scope.go:117] "RemoveContainer" containerID="8e4b737cdfe5a62b1eea4be10d7ae8408ed9d5db9eb9e377b74b41e783c9bec7" Dec 03 13:39:17 crc kubenswrapper[5052]: I1203 13:39:17.749092 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8e4b737cdfe5a62b1eea4be10d7ae8408ed9d5db9eb9e377b74b41e783c9bec7"} err="failed to get container status \"8e4b737cdfe5a62b1eea4be10d7ae8408ed9d5db9eb9e377b74b41e783c9bec7\": rpc error: code = NotFound desc = could not find container \"8e4b737cdfe5a62b1eea4be10d7ae8408ed9d5db9eb9e377b74b41e783c9bec7\": container with ID starting with 8e4b737cdfe5a62b1eea4be10d7ae8408ed9d5db9eb9e377b74b41e783c9bec7 not found: ID does not exist" Dec 03 13:39:17 crc kubenswrapper[5052]: I1203 13:39:17.825806 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r4h8w\" (UniqueName: \"kubernetes.io/projected/75a90895-a53c-4c1f-8d5b-fd859d61a681-kube-api-access-r4h8w\") pod \"75a90895-a53c-4c1f-8d5b-fd859d61a681\" (UID: \"75a90895-a53c-4c1f-8d5b-fd859d61a681\") " Dec 03 13:39:17 crc kubenswrapper[5052]: I1203 13:39:17.829191 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/75a90895-a53c-4c1f-8d5b-fd859d61a681-kube-api-access-r4h8w" (OuterVolumeSpecName: "kube-api-access-r4h8w") pod "75a90895-a53c-4c1f-8d5b-fd859d61a681" (UID: "75a90895-a53c-4c1f-8d5b-fd859d61a681"). InnerVolumeSpecName "kube-api-access-r4h8w". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:39:17 crc kubenswrapper[5052]: I1203 13:39:17.927569 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-r4h8w\" (UniqueName: \"kubernetes.io/projected/75a90895-a53c-4c1f-8d5b-fd859d61a681-kube-api-access-r4h8w\") on node \"crc\" DevicePath \"\"" Dec 03 13:39:18 crc kubenswrapper[5052]: I1203 13:39:18.680928 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-operators/infra-operator-index-2dtcx" Dec 03 13:39:18 crc kubenswrapper[5052]: I1203 13:39:18.732702 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-operators/infra-operator-index-2dtcx"] Dec 03 13:39:18 crc kubenswrapper[5052]: I1203 13:39:18.741359 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-operators/infra-operator-index-2dtcx"] Dec 03 13:39:18 crc kubenswrapper[5052]: I1203 13:39:18.752783 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-operators/mariadb-operator-controller-manager-759d8858f9-9bhzf"] Dec 03 13:39:18 crc kubenswrapper[5052]: I1203 13:39:18.753009 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-operators/mariadb-operator-controller-manager-759d8858f9-9bhzf" podUID="44402132-c7a3-4a0b-a87e-eea057961099" containerName="manager" containerID="cri-o://8bcd92e03839ee1b954a6b77c88cef613ce2e0ffcf2f824f7e2db180bc50f14d" gracePeriod=10 Dec 03 13:39:19 crc kubenswrapper[5052]: I1203 13:39:19.027847 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-operators/mariadb-operator-index-lg4mj"] Dec 03 13:39:19 crc kubenswrapper[5052]: I1203 13:39:19.028068 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-operators/mariadb-operator-index-lg4mj" podUID="8cb04567-42af-44a4-a4ed-acfd1d7027bd" containerName="registry-server" containerID="cri-o://78e5b6948d0c0fec3187be4ca25bc145dbcde6fc92d4ef02b69bf5faf93ed27e" gracePeriod=30 Dec 03 13:39:19 crc kubenswrapper[5052]: I1203 13:39:19.064824 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-operators/55a3886cc1ed42812df4eab61c7a6033dc924d195539e8545c8f175f6182z8b"] Dec 03 13:39:19 crc kubenswrapper[5052]: I1203 13:39:19.075039 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-operators/55a3886cc1ed42812df4eab61c7a6033dc924d195539e8545c8f175f6182z8b"] Dec 03 13:39:19 crc kubenswrapper[5052]: I1203 13:39:19.189714 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-operators/mariadb-operator-controller-manager-759d8858f9-9bhzf" Dec 03 13:39:19 crc kubenswrapper[5052]: I1203 13:39:19.363580 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cgtqq\" (UniqueName: \"kubernetes.io/projected/44402132-c7a3-4a0b-a87e-eea057961099-kube-api-access-cgtqq\") pod \"44402132-c7a3-4a0b-a87e-eea057961099\" (UID: \"44402132-c7a3-4a0b-a87e-eea057961099\") " Dec 03 13:39:19 crc kubenswrapper[5052]: I1203 13:39:19.365464 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/44402132-c7a3-4a0b-a87e-eea057961099-webhook-cert\") pod \"44402132-c7a3-4a0b-a87e-eea057961099\" (UID: \"44402132-c7a3-4a0b-a87e-eea057961099\") " Dec 03 13:39:19 crc kubenswrapper[5052]: I1203 13:39:19.365605 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/44402132-c7a3-4a0b-a87e-eea057961099-apiservice-cert\") pod \"44402132-c7a3-4a0b-a87e-eea057961099\" (UID: \"44402132-c7a3-4a0b-a87e-eea057961099\") " Dec 03 13:39:19 crc kubenswrapper[5052]: I1203 13:39:19.378883 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/44402132-c7a3-4a0b-a87e-eea057961099-kube-api-access-cgtqq" (OuterVolumeSpecName: "kube-api-access-cgtqq") pod "44402132-c7a3-4a0b-a87e-eea057961099" (UID: "44402132-c7a3-4a0b-a87e-eea057961099"). InnerVolumeSpecName "kube-api-access-cgtqq". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:39:19 crc kubenswrapper[5052]: I1203 13:39:19.381232 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/44402132-c7a3-4a0b-a87e-eea057961099-apiservice-cert" (OuterVolumeSpecName: "apiservice-cert") pod "44402132-c7a3-4a0b-a87e-eea057961099" (UID: "44402132-c7a3-4a0b-a87e-eea057961099"). InnerVolumeSpecName "apiservice-cert". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:39:19 crc kubenswrapper[5052]: I1203 13:39:19.381471 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/44402132-c7a3-4a0b-a87e-eea057961099-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "44402132-c7a3-4a0b-a87e-eea057961099" (UID: "44402132-c7a3-4a0b-a87e-eea057961099"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:39:19 crc kubenswrapper[5052]: I1203 13:39:19.432538 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-operators/mariadb-operator-index-lg4mj" Dec 03 13:39:19 crc kubenswrapper[5052]: I1203 13:39:19.467810 5052 reconciler_common.go:293] "Volume detached for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/44402132-c7a3-4a0b-a87e-eea057961099-apiservice-cert\") on node \"crc\" DevicePath \"\"" Dec 03 13:39:19 crc kubenswrapper[5052]: I1203 13:39:19.467859 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-cgtqq\" (UniqueName: \"kubernetes.io/projected/44402132-c7a3-4a0b-a87e-eea057961099-kube-api-access-cgtqq\") on node \"crc\" DevicePath \"\"" Dec 03 13:39:19 crc kubenswrapper[5052]: I1203 13:39:19.467876 5052 reconciler_common.go:293] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/44402132-c7a3-4a0b-a87e-eea057961099-webhook-cert\") on node \"crc\" DevicePath \"\"" Dec 03 13:39:19 crc kubenswrapper[5052]: I1203 13:39:19.569114 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k9lgt\" (UniqueName: \"kubernetes.io/projected/8cb04567-42af-44a4-a4ed-acfd1d7027bd-kube-api-access-k9lgt\") pod \"8cb04567-42af-44a4-a4ed-acfd1d7027bd\" (UID: \"8cb04567-42af-44a4-a4ed-acfd1d7027bd\") " Dec 03 13:39:19 crc kubenswrapper[5052]: I1203 13:39:19.572630 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8cb04567-42af-44a4-a4ed-acfd1d7027bd-kube-api-access-k9lgt" (OuterVolumeSpecName: "kube-api-access-k9lgt") pod "8cb04567-42af-44a4-a4ed-acfd1d7027bd" (UID: "8cb04567-42af-44a4-a4ed-acfd1d7027bd"). InnerVolumeSpecName "kube-api-access-k9lgt". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:39:19 crc kubenswrapper[5052]: I1203 13:39:19.671265 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-k9lgt\" (UniqueName: \"kubernetes.io/projected/8cb04567-42af-44a4-a4ed-acfd1d7027bd-kube-api-access-k9lgt\") on node \"crc\" DevicePath \"\"" Dec 03 13:39:19 crc kubenswrapper[5052]: I1203 13:39:19.685653 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4dd99d7d-b7ff-4ba7-bf73-5e66e3e500b7" path="/var/lib/kubelet/pods/4dd99d7d-b7ff-4ba7-bf73-5e66e3e500b7/volumes" Dec 03 13:39:19 crc kubenswrapper[5052]: I1203 13:39:19.686760 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="75a90895-a53c-4c1f-8d5b-fd859d61a681" path="/var/lib/kubelet/pods/75a90895-a53c-4c1f-8d5b-fd859d61a681/volumes" Dec 03 13:39:19 crc kubenswrapper[5052]: I1203 13:39:19.688044 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a48b1dcb-27f2-4c5b-b735-cc1de58a1384" path="/var/lib/kubelet/pods/a48b1dcb-27f2-4c5b-b735-cc1de58a1384/volumes" Dec 03 13:39:19 crc kubenswrapper[5052]: I1203 13:39:19.694247 5052 generic.go:334] "Generic (PLEG): container finished" podID="44402132-c7a3-4a0b-a87e-eea057961099" containerID="8bcd92e03839ee1b954a6b77c88cef613ce2e0ffcf2f824f7e2db180bc50f14d" exitCode=0 Dec 03 13:39:19 crc kubenswrapper[5052]: I1203 13:39:19.694318 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/mariadb-operator-controller-manager-759d8858f9-9bhzf" event={"ID":"44402132-c7a3-4a0b-a87e-eea057961099","Type":"ContainerDied","Data":"8bcd92e03839ee1b954a6b77c88cef613ce2e0ffcf2f824f7e2db180bc50f14d"} Dec 03 13:39:19 crc kubenswrapper[5052]: I1203 13:39:19.694349 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/mariadb-operator-controller-manager-759d8858f9-9bhzf" event={"ID":"44402132-c7a3-4a0b-a87e-eea057961099","Type":"ContainerDied","Data":"7ca5429362c6e00d430dd7065a8c0ad514c63681f9fdeafca1d1e07e52ad06d8"} Dec 03 13:39:19 crc kubenswrapper[5052]: I1203 13:39:19.694370 5052 scope.go:117] "RemoveContainer" containerID="8bcd92e03839ee1b954a6b77c88cef613ce2e0ffcf2f824f7e2db180bc50f14d" Dec 03 13:39:19 crc kubenswrapper[5052]: I1203 13:39:19.694532 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-operators/mariadb-operator-controller-manager-759d8858f9-9bhzf" Dec 03 13:39:19 crc kubenswrapper[5052]: I1203 13:39:19.698633 5052 generic.go:334] "Generic (PLEG): container finished" podID="8cb04567-42af-44a4-a4ed-acfd1d7027bd" containerID="78e5b6948d0c0fec3187be4ca25bc145dbcde6fc92d4ef02b69bf5faf93ed27e" exitCode=0 Dec 03 13:39:19 crc kubenswrapper[5052]: I1203 13:39:19.698675 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/mariadb-operator-index-lg4mj" event={"ID":"8cb04567-42af-44a4-a4ed-acfd1d7027bd","Type":"ContainerDied","Data":"78e5b6948d0c0fec3187be4ca25bc145dbcde6fc92d4ef02b69bf5faf93ed27e"} Dec 03 13:39:19 crc kubenswrapper[5052]: I1203 13:39:19.698701 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/mariadb-operator-index-lg4mj" event={"ID":"8cb04567-42af-44a4-a4ed-acfd1d7027bd","Type":"ContainerDied","Data":"0b452ac9a3d24fdcaf95cc93172de891ed82333bd974639962a332a120f23063"} Dec 03 13:39:19 crc kubenswrapper[5052]: I1203 13:39:19.698762 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-operators/mariadb-operator-index-lg4mj" Dec 03 13:39:19 crc kubenswrapper[5052]: I1203 13:39:19.723361 5052 scope.go:117] "RemoveContainer" containerID="8bcd92e03839ee1b954a6b77c88cef613ce2e0ffcf2f824f7e2db180bc50f14d" Dec 03 13:39:19 crc kubenswrapper[5052]: E1203 13:39:19.724024 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8bcd92e03839ee1b954a6b77c88cef613ce2e0ffcf2f824f7e2db180bc50f14d\": container with ID starting with 8bcd92e03839ee1b954a6b77c88cef613ce2e0ffcf2f824f7e2db180bc50f14d not found: ID does not exist" containerID="8bcd92e03839ee1b954a6b77c88cef613ce2e0ffcf2f824f7e2db180bc50f14d" Dec 03 13:39:19 crc kubenswrapper[5052]: I1203 13:39:19.724062 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8bcd92e03839ee1b954a6b77c88cef613ce2e0ffcf2f824f7e2db180bc50f14d"} err="failed to get container status \"8bcd92e03839ee1b954a6b77c88cef613ce2e0ffcf2f824f7e2db180bc50f14d\": rpc error: code = NotFound desc = could not find container \"8bcd92e03839ee1b954a6b77c88cef613ce2e0ffcf2f824f7e2db180bc50f14d\": container with ID starting with 8bcd92e03839ee1b954a6b77c88cef613ce2e0ffcf2f824f7e2db180bc50f14d not found: ID does not exist" Dec 03 13:39:19 crc kubenswrapper[5052]: I1203 13:39:19.724088 5052 scope.go:117] "RemoveContainer" containerID="78e5b6948d0c0fec3187be4ca25bc145dbcde6fc92d4ef02b69bf5faf93ed27e" Dec 03 13:39:19 crc kubenswrapper[5052]: I1203 13:39:19.737949 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-operators/mariadb-operator-controller-manager-759d8858f9-9bhzf"] Dec 03 13:39:19 crc kubenswrapper[5052]: I1203 13:39:19.759309 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-operators/mariadb-operator-controller-manager-759d8858f9-9bhzf"] Dec 03 13:39:19 crc kubenswrapper[5052]: I1203 13:39:19.763623 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-operators/mariadb-operator-index-lg4mj"] Dec 03 13:39:19 crc kubenswrapper[5052]: I1203 13:39:19.769011 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-operators/mariadb-operator-index-lg4mj"] Dec 03 13:39:19 crc kubenswrapper[5052]: I1203 13:39:19.774691 5052 scope.go:117] "RemoveContainer" containerID="78e5b6948d0c0fec3187be4ca25bc145dbcde6fc92d4ef02b69bf5faf93ed27e" Dec 03 13:39:19 crc kubenswrapper[5052]: E1203 13:39:19.775962 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"78e5b6948d0c0fec3187be4ca25bc145dbcde6fc92d4ef02b69bf5faf93ed27e\": container with ID starting with 78e5b6948d0c0fec3187be4ca25bc145dbcde6fc92d4ef02b69bf5faf93ed27e not found: ID does not exist" containerID="78e5b6948d0c0fec3187be4ca25bc145dbcde6fc92d4ef02b69bf5faf93ed27e" Dec 03 13:39:19 crc kubenswrapper[5052]: I1203 13:39:19.775994 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"78e5b6948d0c0fec3187be4ca25bc145dbcde6fc92d4ef02b69bf5faf93ed27e"} err="failed to get container status \"78e5b6948d0c0fec3187be4ca25bc145dbcde6fc92d4ef02b69bf5faf93ed27e\": rpc error: code = NotFound desc = could not find container \"78e5b6948d0c0fec3187be4ca25bc145dbcde6fc92d4ef02b69bf5faf93ed27e\": container with ID starting with 78e5b6948d0c0fec3187be4ca25bc145dbcde6fc92d4ef02b69bf5faf93ed27e not found: ID does not exist" Dec 03 13:39:21 crc kubenswrapper[5052]: I1203 13:39:21.681504 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="44402132-c7a3-4a0b-a87e-eea057961099" path="/var/lib/kubelet/pods/44402132-c7a3-4a0b-a87e-eea057961099/volumes" Dec 03 13:39:21 crc kubenswrapper[5052]: I1203 13:39:21.682406 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8cb04567-42af-44a4-a4ed-acfd1d7027bd" path="/var/lib/kubelet/pods/8cb04567-42af-44a4-a4ed-acfd1d7027bd/volumes" Dec 03 13:39:23 crc kubenswrapper[5052]: I1203 13:39:23.057441 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-operators/rabbitmq-cluster-operator-779fc9694b-g9mr6"] Dec 03 13:39:23 crc kubenswrapper[5052]: I1203 13:39:23.057675 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-operators/rabbitmq-cluster-operator-779fc9694b-g9mr6" podUID="933b8ff7-751e-4329-9c00-1d6c086677de" containerName="operator" containerID="cri-o://b94cbf9acefffbed6c9cf42ee7a481524b7998828fdcd3f8a9882f97f66bc832" gracePeriod=10 Dec 03 13:39:23 crc kubenswrapper[5052]: I1203 13:39:23.372226 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-operators/rabbitmq-cluster-operator-index-7qf66"] Dec 03 13:39:23 crc kubenswrapper[5052]: I1203 13:39:23.377640 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-operators/rabbitmq-cluster-operator-index-7qf66" podUID="0bae405f-384d-45f9-8261-685ad5804551" containerName="registry-server" containerID="cri-o://aa8acc3c56d6c3e5bb6a62cf6429806fdf196e6914f29e81fb07207514b0d2e1" gracePeriod=30 Dec 03 13:39:23 crc kubenswrapper[5052]: E1203 13:39:23.396985 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of aa8acc3c56d6c3e5bb6a62cf6429806fdf196e6914f29e81fb07207514b0d2e1 is running failed: container process not found" containerID="aa8acc3c56d6c3e5bb6a62cf6429806fdf196e6914f29e81fb07207514b0d2e1" cmd=["grpc_health_probe","-addr=:50051"] Dec 03 13:39:23 crc kubenswrapper[5052]: E1203 13:39:23.397431 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of aa8acc3c56d6c3e5bb6a62cf6429806fdf196e6914f29e81fb07207514b0d2e1 is running failed: container process not found" containerID="aa8acc3c56d6c3e5bb6a62cf6429806fdf196e6914f29e81fb07207514b0d2e1" cmd=["grpc_health_probe","-addr=:50051"] Dec 03 13:39:23 crc kubenswrapper[5052]: E1203 13:39:23.397759 5052 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of aa8acc3c56d6c3e5bb6a62cf6429806fdf196e6914f29e81fb07207514b0d2e1 is running failed: container process not found" containerID="aa8acc3c56d6c3e5bb6a62cf6429806fdf196e6914f29e81fb07207514b0d2e1" cmd=["grpc_health_probe","-addr=:50051"] Dec 03 13:39:23 crc kubenswrapper[5052]: E1203 13:39:23.397805 5052 prober.go:104] "Probe errored" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of aa8acc3c56d6c3e5bb6a62cf6429806fdf196e6914f29e81fb07207514b0d2e1 is running failed: container process not found" probeType="Readiness" pod="openstack-operators/rabbitmq-cluster-operator-index-7qf66" podUID="0bae405f-384d-45f9-8261-685ad5804551" containerName="registry-server" Dec 03 13:39:23 crc kubenswrapper[5052]: I1203 13:39:23.431047 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-operators/9704761d240e56fb98655ffd81084895b33a73ec711f4dcdef0450e5907pw26"] Dec 03 13:39:23 crc kubenswrapper[5052]: I1203 13:39:23.439199 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-operators/9704761d240e56fb98655ffd81084895b33a73ec711f4dcdef0450e5907pw26"] Dec 03 13:39:23 crc kubenswrapper[5052]: I1203 13:39:23.519471 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-operators/rabbitmq-cluster-operator-779fc9694b-g9mr6" Dec 03 13:39:23 crc kubenswrapper[5052]: I1203 13:39:23.681505 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="372ee9c5-9cbd-40f9-83ca-abf8b5b01a5c" path="/var/lib/kubelet/pods/372ee9c5-9cbd-40f9-83ca-abf8b5b01a5c/volumes" Dec 03 13:39:23 crc kubenswrapper[5052]: I1203 13:39:23.712815 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8nksz\" (UniqueName: \"kubernetes.io/projected/933b8ff7-751e-4329-9c00-1d6c086677de-kube-api-access-8nksz\") pod \"933b8ff7-751e-4329-9c00-1d6c086677de\" (UID: \"933b8ff7-751e-4329-9c00-1d6c086677de\") " Dec 03 13:39:23 crc kubenswrapper[5052]: I1203 13:39:23.720864 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/933b8ff7-751e-4329-9c00-1d6c086677de-kube-api-access-8nksz" (OuterVolumeSpecName: "kube-api-access-8nksz") pod "933b8ff7-751e-4329-9c00-1d6c086677de" (UID: "933b8ff7-751e-4329-9c00-1d6c086677de"). InnerVolumeSpecName "kube-api-access-8nksz". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:39:23 crc kubenswrapper[5052]: I1203 13:39:23.762982 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-operators/rabbitmq-cluster-operator-index-7qf66" Dec 03 13:39:23 crc kubenswrapper[5052]: I1203 13:39:23.814543 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-8nksz\" (UniqueName: \"kubernetes.io/projected/933b8ff7-751e-4329-9c00-1d6c086677de-kube-api-access-8nksz\") on node \"crc\" DevicePath \"\"" Dec 03 13:39:23 crc kubenswrapper[5052]: I1203 13:39:23.915589 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vr6fx\" (UniqueName: \"kubernetes.io/projected/0bae405f-384d-45f9-8261-685ad5804551-kube-api-access-vr6fx\") pod \"0bae405f-384d-45f9-8261-685ad5804551\" (UID: \"0bae405f-384d-45f9-8261-685ad5804551\") " Dec 03 13:39:23 crc kubenswrapper[5052]: I1203 13:39:23.918665 5052 generic.go:334] "Generic (PLEG): container finished" podID="0bae405f-384d-45f9-8261-685ad5804551" containerID="aa8acc3c56d6c3e5bb6a62cf6429806fdf196e6914f29e81fb07207514b0d2e1" exitCode=0 Dec 03 13:39:23 crc kubenswrapper[5052]: I1203 13:39:23.918746 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-operators/rabbitmq-cluster-operator-index-7qf66" Dec 03 13:39:23 crc kubenswrapper[5052]: I1203 13:39:23.919209 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0bae405f-384d-45f9-8261-685ad5804551-kube-api-access-vr6fx" (OuterVolumeSpecName: "kube-api-access-vr6fx") pod "0bae405f-384d-45f9-8261-685ad5804551" (UID: "0bae405f-384d-45f9-8261-685ad5804551"). InnerVolumeSpecName "kube-api-access-vr6fx". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:39:23 crc kubenswrapper[5052]: I1203 13:39:23.918774 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/rabbitmq-cluster-operator-index-7qf66" event={"ID":"0bae405f-384d-45f9-8261-685ad5804551","Type":"ContainerDied","Data":"aa8acc3c56d6c3e5bb6a62cf6429806fdf196e6914f29e81fb07207514b0d2e1"} Dec 03 13:39:23 crc kubenswrapper[5052]: I1203 13:39:23.919304 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/rabbitmq-cluster-operator-index-7qf66" event={"ID":"0bae405f-384d-45f9-8261-685ad5804551","Type":"ContainerDied","Data":"9f4253fd294b4a245149dd9313ada3ad5a467f0fb3f82ccb36c2a9558661a95f"} Dec 03 13:39:23 crc kubenswrapper[5052]: I1203 13:39:23.919379 5052 scope.go:117] "RemoveContainer" containerID="aa8acc3c56d6c3e5bb6a62cf6429806fdf196e6914f29e81fb07207514b0d2e1" Dec 03 13:39:23 crc kubenswrapper[5052]: I1203 13:39:23.924739 5052 generic.go:334] "Generic (PLEG): container finished" podID="933b8ff7-751e-4329-9c00-1d6c086677de" containerID="b94cbf9acefffbed6c9cf42ee7a481524b7998828fdcd3f8a9882f97f66bc832" exitCode=0 Dec 03 13:39:23 crc kubenswrapper[5052]: I1203 13:39:23.924773 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/rabbitmq-cluster-operator-779fc9694b-g9mr6" event={"ID":"933b8ff7-751e-4329-9c00-1d6c086677de","Type":"ContainerDied","Data":"b94cbf9acefffbed6c9cf42ee7a481524b7998828fdcd3f8a9882f97f66bc832"} Dec 03 13:39:23 crc kubenswrapper[5052]: I1203 13:39:23.924797 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/rabbitmq-cluster-operator-779fc9694b-g9mr6" event={"ID":"933b8ff7-751e-4329-9c00-1d6c086677de","Type":"ContainerDied","Data":"7b3c3a7e185f8a38af4532a8085004a12d20a74385d43faea1c3603c23042e44"} Dec 03 13:39:23 crc kubenswrapper[5052]: I1203 13:39:23.924824 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-operators/rabbitmq-cluster-operator-779fc9694b-g9mr6" Dec 03 13:39:23 crc kubenswrapper[5052]: I1203 13:39:23.942593 5052 scope.go:117] "RemoveContainer" containerID="aa8acc3c56d6c3e5bb6a62cf6429806fdf196e6914f29e81fb07207514b0d2e1" Dec 03 13:39:23 crc kubenswrapper[5052]: E1203 13:39:23.943446 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aa8acc3c56d6c3e5bb6a62cf6429806fdf196e6914f29e81fb07207514b0d2e1\": container with ID starting with aa8acc3c56d6c3e5bb6a62cf6429806fdf196e6914f29e81fb07207514b0d2e1 not found: ID does not exist" containerID="aa8acc3c56d6c3e5bb6a62cf6429806fdf196e6914f29e81fb07207514b0d2e1" Dec 03 13:39:23 crc kubenswrapper[5052]: I1203 13:39:23.943482 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aa8acc3c56d6c3e5bb6a62cf6429806fdf196e6914f29e81fb07207514b0d2e1"} err="failed to get container status \"aa8acc3c56d6c3e5bb6a62cf6429806fdf196e6914f29e81fb07207514b0d2e1\": rpc error: code = NotFound desc = could not find container \"aa8acc3c56d6c3e5bb6a62cf6429806fdf196e6914f29e81fb07207514b0d2e1\": container with ID starting with aa8acc3c56d6c3e5bb6a62cf6429806fdf196e6914f29e81fb07207514b0d2e1 not found: ID does not exist" Dec 03 13:39:23 crc kubenswrapper[5052]: I1203 13:39:23.943503 5052 scope.go:117] "RemoveContainer" containerID="b94cbf9acefffbed6c9cf42ee7a481524b7998828fdcd3f8a9882f97f66bc832" Dec 03 13:39:23 crc kubenswrapper[5052]: I1203 13:39:23.956126 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-operators/rabbitmq-cluster-operator-779fc9694b-g9mr6"] Dec 03 13:39:23 crc kubenswrapper[5052]: I1203 13:39:23.961786 5052 scope.go:117] "RemoveContainer" containerID="b94cbf9acefffbed6c9cf42ee7a481524b7998828fdcd3f8a9882f97f66bc832" Dec 03 13:39:23 crc kubenswrapper[5052]: E1203 13:39:23.962314 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b94cbf9acefffbed6c9cf42ee7a481524b7998828fdcd3f8a9882f97f66bc832\": container with ID starting with b94cbf9acefffbed6c9cf42ee7a481524b7998828fdcd3f8a9882f97f66bc832 not found: ID does not exist" containerID="b94cbf9acefffbed6c9cf42ee7a481524b7998828fdcd3f8a9882f97f66bc832" Dec 03 13:39:23 crc kubenswrapper[5052]: I1203 13:39:23.962346 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b94cbf9acefffbed6c9cf42ee7a481524b7998828fdcd3f8a9882f97f66bc832"} err="failed to get container status \"b94cbf9acefffbed6c9cf42ee7a481524b7998828fdcd3f8a9882f97f66bc832\": rpc error: code = NotFound desc = could not find container \"b94cbf9acefffbed6c9cf42ee7a481524b7998828fdcd3f8a9882f97f66bc832\": container with ID starting with b94cbf9acefffbed6c9cf42ee7a481524b7998828fdcd3f8a9882f97f66bc832 not found: ID does not exist" Dec 03 13:39:23 crc kubenswrapper[5052]: I1203 13:39:23.963100 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-operators/rabbitmq-cluster-operator-779fc9694b-g9mr6"] Dec 03 13:39:24 crc kubenswrapper[5052]: I1203 13:39:24.017734 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-vr6fx\" (UniqueName: \"kubernetes.io/projected/0bae405f-384d-45f9-8261-685ad5804551-kube-api-access-vr6fx\") on node \"crc\" DevicePath \"\"" Dec 03 13:39:24 crc kubenswrapper[5052]: I1203 13:39:24.264375 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-operators/rabbitmq-cluster-operator-index-7qf66"] Dec 03 13:39:24 crc kubenswrapper[5052]: I1203 13:39:24.271869 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-operators/rabbitmq-cluster-operator-index-7qf66"] Dec 03 13:39:25 crc kubenswrapper[5052]: I1203 13:39:25.681547 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0bae405f-384d-45f9-8261-685ad5804551" path="/var/lib/kubelet/pods/0bae405f-384d-45f9-8261-685ad5804551/volumes" Dec 03 13:39:25 crc kubenswrapper[5052]: I1203 13:39:25.682205 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="933b8ff7-751e-4329-9c00-1d6c086677de" path="/var/lib/kubelet/pods/933b8ff7-751e-4329-9c00-1d6c086677de/volumes" Dec 03 13:39:49 crc kubenswrapper[5052]: I1203 13:39:49.606645 5052 scope.go:117] "RemoveContainer" containerID="fedb14f501ff028d683c77ffefa608766dad89bb6094768270196099685e2f28" Dec 03 13:39:49 crc kubenswrapper[5052]: I1203 13:39:49.625984 5052 scope.go:117] "RemoveContainer" containerID="36bea13f9e88148b806bf93382abfb0f19013d1939bd07876c9afc32b9057387" Dec 03 13:39:49 crc kubenswrapper[5052]: I1203 13:39:49.653652 5052 scope.go:117] "RemoveContainer" containerID="3e0fb88e579785672f0a6cbbd960e331f0d6d07ddf21d7be9104243f2d460a69" Dec 03 13:39:49 crc kubenswrapper[5052]: I1203 13:39:49.672870 5052 scope.go:117] "RemoveContainer" containerID="7f88dc0ba48b4207d557f5ae6a680b60bb1a94eee39236bcbc1b4e9b30d4d4b1" Dec 03 13:39:49 crc kubenswrapper[5052]: I1203 13:39:49.701145 5052 scope.go:117] "RemoveContainer" containerID="ba99885028205208f9fcdb8293b68a3b0bcb51bede04b7a8197d8286cc8e79b5" Dec 03 13:39:49 crc kubenswrapper[5052]: I1203 13:39:49.720905 5052 scope.go:117] "RemoveContainer" containerID="3f059c4beaaddb8bf0a200dd1eee0146df44f5a9adbac4ed4b87510ffd01762f" Dec 03 13:39:49 crc kubenswrapper[5052]: I1203 13:39:49.746298 5052 scope.go:117] "RemoveContainer" containerID="7cb4c042c3262ef46295f0a30d5bc2bf6d49a22c5ee6c0af14ded068615ffc7d" Dec 03 13:39:49 crc kubenswrapper[5052]: I1203 13:39:49.768069 5052 scope.go:117] "RemoveContainer" containerID="6c963b738a30433ba59b5eb6bb7791cca4717b29ee25a4df081130697ce887f2" Dec 03 13:39:49 crc kubenswrapper[5052]: I1203 13:39:49.790664 5052 scope.go:117] "RemoveContainer" containerID="e061fd78a790655f73443f302c63c58b18336d6d8a5985a5f8ba3ceae2993a4c" Dec 03 13:39:49 crc kubenswrapper[5052]: I1203 13:39:49.805664 5052 scope.go:117] "RemoveContainer" containerID="c772ba41bb82907750b2d521f5db102f1d8c6b06a1c98e8c2a13bdd2fdedd5e4" Dec 03 13:39:49 crc kubenswrapper[5052]: I1203 13:39:49.821153 5052 scope.go:117] "RemoveContainer" containerID="4e9f96da9e8199ccd0b09b27ffd95748a6ab4cc99eb521c45ab48d2e34d494a1" Dec 03 13:39:49 crc kubenswrapper[5052]: I1203 13:39:49.839014 5052 scope.go:117] "RemoveContainer" containerID="bbd7f06d451dffd6aa1fb53ba54bfac0479837de5dc87cc768f048d6bf9c838d" Dec 03 13:39:49 crc kubenswrapper[5052]: I1203 13:39:49.858017 5052 scope.go:117] "RemoveContainer" containerID="fb7b00ce91c676ce96b647fb7ab7b5541879f301a362cd7af3c7252cd2acfc00" Dec 03 13:40:24 crc kubenswrapper[5052]: I1203 13:40:24.254387 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-operators/mariadb-operator-index-w76tp"] Dec 03 13:40:24 crc kubenswrapper[5052]: E1203 13:40:24.255759 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="da70419b-4f9e-405d-8fe7-7dbca0ca315e" containerName="registry-server" Dec 03 13:40:24 crc kubenswrapper[5052]: I1203 13:40:24.255779 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="da70419b-4f9e-405d-8fe7-7dbca0ca315e" containerName="registry-server" Dec 03 13:40:24 crc kubenswrapper[5052]: E1203 13:40:24.255791 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="2e771f67-f156-4dec-a77f-88b4735f298b" containerName="galera" Dec 03 13:40:24 crc kubenswrapper[5052]: I1203 13:40:24.255798 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="2e771f67-f156-4dec-a77f-88b4735f298b" containerName="galera" Dec 03 13:40:24 crc kubenswrapper[5052]: E1203 13:40:24.255825 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="e7224fc6-4050-43bd-95ce-b6257be9ed06" containerName="keystone-api" Dec 03 13:40:24 crc kubenswrapper[5052]: I1203 13:40:24.255833 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="e7224fc6-4050-43bd-95ce-b6257be9ed06" containerName="keystone-api" Dec 03 13:40:24 crc kubenswrapper[5052]: E1203 13:40:24.255864 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="44402132-c7a3-4a0b-a87e-eea057961099" containerName="manager" Dec 03 13:40:24 crc kubenswrapper[5052]: I1203 13:40:24.255871 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="44402132-c7a3-4a0b-a87e-eea057961099" containerName="manager" Dec 03 13:40:24 crc kubenswrapper[5052]: E1203 13:40:24.255885 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="9d2e4b3b-6b0f-4e92-a6bb-fa0df37feb28" containerName="setup-container" Dec 03 13:40:24 crc kubenswrapper[5052]: I1203 13:40:24.255891 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d2e4b3b-6b0f-4e92-a6bb-fa0df37feb28" containerName="setup-container" Dec 03 13:40:24 crc kubenswrapper[5052]: E1203 13:40:24.255902 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="0bae405f-384d-45f9-8261-685ad5804551" containerName="registry-server" Dec 03 13:40:24 crc kubenswrapper[5052]: I1203 13:40:24.255909 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="0bae405f-384d-45f9-8261-685ad5804551" containerName="registry-server" Dec 03 13:40:24 crc kubenswrapper[5052]: E1203 13:40:24.255918 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="f72c36e0-b6bc-412c-887f-3e1394cf6c1f" containerName="memcached" Dec 03 13:40:24 crc kubenswrapper[5052]: I1203 13:40:24.255925 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="f72c36e0-b6bc-412c-887f-3e1394cf6c1f" containerName="memcached" Dec 03 13:40:24 crc kubenswrapper[5052]: E1203 13:40:24.255933 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="4dd99d7d-b7ff-4ba7-bf73-5e66e3e500b7" containerName="kube-rbac-proxy" Dec 03 13:40:24 crc kubenswrapper[5052]: I1203 13:40:24.255939 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="4dd99d7d-b7ff-4ba7-bf73-5e66e3e500b7" containerName="kube-rbac-proxy" Dec 03 13:40:24 crc kubenswrapper[5052]: E1203 13:40:24.255953 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="75a90895-a53c-4c1f-8d5b-fd859d61a681" containerName="registry-server" Dec 03 13:40:24 crc kubenswrapper[5052]: I1203 13:40:24.255960 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="75a90895-a53c-4c1f-8d5b-fd859d61a681" containerName="registry-server" Dec 03 13:40:24 crc kubenswrapper[5052]: E1203 13:40:24.255970 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="9d2e4b3b-6b0f-4e92-a6bb-fa0df37feb28" containerName="rabbitmq" Dec 03 13:40:24 crc kubenswrapper[5052]: I1203 13:40:24.255977 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d2e4b3b-6b0f-4e92-a6bb-fa0df37feb28" containerName="rabbitmq" Dec 03 13:40:24 crc kubenswrapper[5052]: E1203 13:40:24.255989 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="ae81dcbe-7c90-4de6-94bd-014eb4e9557f" containerName="manager" Dec 03 13:40:24 crc kubenswrapper[5052]: I1203 13:40:24.255996 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae81dcbe-7c90-4de6-94bd-014eb4e9557f" containerName="manager" Dec 03 13:40:24 crc kubenswrapper[5052]: E1203 13:40:24.256009 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="0ea9b78b-2beb-4bdb-852c-1b92b9638551" containerName="galera" Dec 03 13:40:24 crc kubenswrapper[5052]: I1203 13:40:24.256021 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="0ea9b78b-2beb-4bdb-852c-1b92b9638551" containerName="galera" Dec 03 13:40:24 crc kubenswrapper[5052]: E1203 13:40:24.256029 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="0ea9b78b-2beb-4bdb-852c-1b92b9638551" containerName="mysql-bootstrap" Dec 03 13:40:24 crc kubenswrapper[5052]: I1203 13:40:24.256036 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="0ea9b78b-2beb-4bdb-852c-1b92b9638551" containerName="mysql-bootstrap" Dec 03 13:40:24 crc kubenswrapper[5052]: E1203 13:40:24.256046 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="4dd99d7d-b7ff-4ba7-bf73-5e66e3e500b7" containerName="manager" Dec 03 13:40:24 crc kubenswrapper[5052]: I1203 13:40:24.256053 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="4dd99d7d-b7ff-4ba7-bf73-5e66e3e500b7" containerName="manager" Dec 03 13:40:24 crc kubenswrapper[5052]: E1203 13:40:24.256061 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="36e1b611-408e-4cc3-8531-03adcac84683" containerName="mysql-bootstrap" Dec 03 13:40:24 crc kubenswrapper[5052]: I1203 13:40:24.256069 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="36e1b611-408e-4cc3-8531-03adcac84683" containerName="mysql-bootstrap" Dec 03 13:40:24 crc kubenswrapper[5052]: E1203 13:40:24.256076 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="933b8ff7-751e-4329-9c00-1d6c086677de" containerName="operator" Dec 03 13:40:24 crc kubenswrapper[5052]: I1203 13:40:24.256083 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="933b8ff7-751e-4329-9c00-1d6c086677de" containerName="operator" Dec 03 13:40:24 crc kubenswrapper[5052]: E1203 13:40:24.256092 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="56410ea8-8901-4c3e-b21c-ec3b2f908c47" containerName="mariadb-account-delete" Dec 03 13:40:24 crc kubenswrapper[5052]: I1203 13:40:24.256098 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="56410ea8-8901-4c3e-b21c-ec3b2f908c47" containerName="mariadb-account-delete" Dec 03 13:40:24 crc kubenswrapper[5052]: E1203 13:40:24.256112 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="8cb04567-42af-44a4-a4ed-acfd1d7027bd" containerName="registry-server" Dec 03 13:40:24 crc kubenswrapper[5052]: I1203 13:40:24.256118 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="8cb04567-42af-44a4-a4ed-acfd1d7027bd" containerName="registry-server" Dec 03 13:40:24 crc kubenswrapper[5052]: E1203 13:40:24.256126 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="36e1b611-408e-4cc3-8531-03adcac84683" containerName="galera" Dec 03 13:40:24 crc kubenswrapper[5052]: I1203 13:40:24.256133 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="36e1b611-408e-4cc3-8531-03adcac84683" containerName="galera" Dec 03 13:40:24 crc kubenswrapper[5052]: E1203 13:40:24.256144 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="2e771f67-f156-4dec-a77f-88b4735f298b" containerName="mysql-bootstrap" Dec 03 13:40:24 crc kubenswrapper[5052]: I1203 13:40:24.256151 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="2e771f67-f156-4dec-a77f-88b4735f298b" containerName="mysql-bootstrap" Dec 03 13:40:24 crc kubenswrapper[5052]: I1203 13:40:24.256328 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="e7224fc6-4050-43bd-95ce-b6257be9ed06" containerName="keystone-api" Dec 03 13:40:24 crc kubenswrapper[5052]: I1203 13:40:24.256347 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="56410ea8-8901-4c3e-b21c-ec3b2f908c47" containerName="mariadb-account-delete" Dec 03 13:40:24 crc kubenswrapper[5052]: I1203 13:40:24.256358 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="0ea9b78b-2beb-4bdb-852c-1b92b9638551" containerName="galera" Dec 03 13:40:24 crc kubenswrapper[5052]: I1203 13:40:24.256367 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="8cb04567-42af-44a4-a4ed-acfd1d7027bd" containerName="registry-server" Dec 03 13:40:24 crc kubenswrapper[5052]: I1203 13:40:24.256377 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="9d2e4b3b-6b0f-4e92-a6bb-fa0df37feb28" containerName="rabbitmq" Dec 03 13:40:24 crc kubenswrapper[5052]: I1203 13:40:24.256389 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="36e1b611-408e-4cc3-8531-03adcac84683" containerName="galera" Dec 03 13:40:24 crc kubenswrapper[5052]: I1203 13:40:24.256396 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="75a90895-a53c-4c1f-8d5b-fd859d61a681" containerName="registry-server" Dec 03 13:40:24 crc kubenswrapper[5052]: I1203 13:40:24.256405 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="4dd99d7d-b7ff-4ba7-bf73-5e66e3e500b7" containerName="manager" Dec 03 13:40:24 crc kubenswrapper[5052]: I1203 13:40:24.256412 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="f72c36e0-b6bc-412c-887f-3e1394cf6c1f" containerName="memcached" Dec 03 13:40:24 crc kubenswrapper[5052]: I1203 13:40:24.256420 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="933b8ff7-751e-4329-9c00-1d6c086677de" containerName="operator" Dec 03 13:40:24 crc kubenswrapper[5052]: I1203 13:40:24.256430 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="4dd99d7d-b7ff-4ba7-bf73-5e66e3e500b7" containerName="kube-rbac-proxy" Dec 03 13:40:24 crc kubenswrapper[5052]: I1203 13:40:24.256440 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="ae81dcbe-7c90-4de6-94bd-014eb4e9557f" containerName="manager" Dec 03 13:40:24 crc kubenswrapper[5052]: I1203 13:40:24.256446 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="0bae405f-384d-45f9-8261-685ad5804551" containerName="registry-server" Dec 03 13:40:24 crc kubenswrapper[5052]: I1203 13:40:24.256454 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="da70419b-4f9e-405d-8fe7-7dbca0ca315e" containerName="registry-server" Dec 03 13:40:24 crc kubenswrapper[5052]: I1203 13:40:24.256462 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="2e771f67-f156-4dec-a77f-88b4735f298b" containerName="galera" Dec 03 13:40:24 crc kubenswrapper[5052]: I1203 13:40:24.256471 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="44402132-c7a3-4a0b-a87e-eea057961099" containerName="manager" Dec 03 13:40:24 crc kubenswrapper[5052]: I1203 13:40:24.257178 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-operators/mariadb-operator-index-w76tp" Dec 03 13:40:24 crc kubenswrapper[5052]: I1203 13:40:24.261294 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-operators"/"openshift-service-ca.crt" Dec 03 13:40:24 crc kubenswrapper[5052]: I1203 13:40:24.262528 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-operators"/"mariadb-operator-index-dockercfg-xr6tx" Dec 03 13:40:24 crc kubenswrapper[5052]: I1203 13:40:24.264998 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openstack-operators"/"kube-root-ca.crt" Dec 03 13:40:24 crc kubenswrapper[5052]: I1203 13:40:24.272548 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-operators/mariadb-operator-index-w76tp"] Dec 03 13:40:24 crc kubenswrapper[5052]: I1203 13:40:24.363111 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lqjd2\" (UniqueName: \"kubernetes.io/projected/126387b4-22aa-4952-9293-0b9385c931bb-kube-api-access-lqjd2\") pod \"mariadb-operator-index-w76tp\" (UID: \"126387b4-22aa-4952-9293-0b9385c931bb\") " pod="openstack-operators/mariadb-operator-index-w76tp" Dec 03 13:40:24 crc kubenswrapper[5052]: I1203 13:40:24.464406 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-lqjd2\" (UniqueName: \"kubernetes.io/projected/126387b4-22aa-4952-9293-0b9385c931bb-kube-api-access-lqjd2\") pod \"mariadb-operator-index-w76tp\" (UID: \"126387b4-22aa-4952-9293-0b9385c931bb\") " pod="openstack-operators/mariadb-operator-index-w76tp" Dec 03 13:40:24 crc kubenswrapper[5052]: I1203 13:40:24.483703 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-lqjd2\" (UniqueName: \"kubernetes.io/projected/126387b4-22aa-4952-9293-0b9385c931bb-kube-api-access-lqjd2\") pod \"mariadb-operator-index-w76tp\" (UID: \"126387b4-22aa-4952-9293-0b9385c931bb\") " pod="openstack-operators/mariadb-operator-index-w76tp" Dec 03 13:40:24 crc kubenswrapper[5052]: I1203 13:40:24.577378 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-operators/mariadb-operator-index-w76tp" Dec 03 13:40:24 crc kubenswrapper[5052]: I1203 13:40:24.635076 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-operators/mariadb-operator-index-w76tp"] Dec 03 13:40:24 crc kubenswrapper[5052]: I1203 13:40:24.839599 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-operators/mariadb-operator-index-p2x5s"] Dec 03 13:40:24 crc kubenswrapper[5052]: I1203 13:40:24.841441 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-operators/mariadb-operator-index-p2x5s" Dec 03 13:40:24 crc kubenswrapper[5052]: I1203 13:40:24.849894 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-operators/mariadb-operator-index-p2x5s"] Dec 03 13:40:24 crc kubenswrapper[5052]: I1203 13:40:24.972407 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hr2xr\" (UniqueName: \"kubernetes.io/projected/f82e13e4-6fd4-4bf8-8d78-c00e7b5a402e-kube-api-access-hr2xr\") pod \"mariadb-operator-index-p2x5s\" (UID: \"f82e13e4-6fd4-4bf8-8d78-c00e7b5a402e\") " pod="openstack-operators/mariadb-operator-index-p2x5s" Dec 03 13:40:25 crc kubenswrapper[5052]: I1203 13:40:25.009613 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-operators/mariadb-operator-index-w76tp"] Dec 03 13:40:25 crc kubenswrapper[5052]: I1203 13:40:25.074769 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-hr2xr\" (UniqueName: \"kubernetes.io/projected/f82e13e4-6fd4-4bf8-8d78-c00e7b5a402e-kube-api-access-hr2xr\") pod \"mariadb-operator-index-p2x5s\" (UID: \"f82e13e4-6fd4-4bf8-8d78-c00e7b5a402e\") " pod="openstack-operators/mariadb-operator-index-p2x5s" Dec 03 13:40:25 crc kubenswrapper[5052]: I1203 13:40:25.099100 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-hr2xr\" (UniqueName: \"kubernetes.io/projected/f82e13e4-6fd4-4bf8-8d78-c00e7b5a402e-kube-api-access-hr2xr\") pod \"mariadb-operator-index-p2x5s\" (UID: \"f82e13e4-6fd4-4bf8-8d78-c00e7b5a402e\") " pod="openstack-operators/mariadb-operator-index-p2x5s" Dec 03 13:40:25 crc kubenswrapper[5052]: I1203 13:40:25.168234 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-operators/mariadb-operator-index-p2x5s" Dec 03 13:40:25 crc kubenswrapper[5052]: I1203 13:40:25.345664 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/mariadb-operator-index-w76tp" event={"ID":"126387b4-22aa-4952-9293-0b9385c931bb","Type":"ContainerStarted","Data":"ba5d1e17782ca277fa5d7f434c14a4c3184792fea85a3d226081dbb0d612d24d"} Dec 03 13:40:25 crc kubenswrapper[5052]: I1203 13:40:25.395444 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-operators/mariadb-operator-index-p2x5s"] Dec 03 13:40:26 crc kubenswrapper[5052]: I1203 13:40:26.355128 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/mariadb-operator-index-p2x5s" event={"ID":"f82e13e4-6fd4-4bf8-8d78-c00e7b5a402e","Type":"ContainerStarted","Data":"1980c94fffa6f6a1807de1ec07b99d9b54965f9d063a33252ff595d7c1d8ea26"} Dec 03 13:40:26 crc kubenswrapper[5052]: I1203 13:40:26.356890 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/mariadb-operator-index-w76tp" event={"ID":"126387b4-22aa-4952-9293-0b9385c931bb","Type":"ContainerStarted","Data":"cc4d42905b6e71c81cae84432de7ae2e8d457105d23fb74502b79fa527e29895"} Dec 03 13:40:26 crc kubenswrapper[5052]: I1203 13:40:26.357038 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-operators/mariadb-operator-index-w76tp" podUID="126387b4-22aa-4952-9293-0b9385c931bb" containerName="registry-server" containerID="cri-o://cc4d42905b6e71c81cae84432de7ae2e8d457105d23fb74502b79fa527e29895" gracePeriod=2 Dec 03 13:40:26 crc kubenswrapper[5052]: I1203 13:40:26.373272 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-operators/mariadb-operator-index-w76tp" podStartSLOduration=1.579309938 podStartE2EDuration="2.373245896s" podCreationTimestamp="2025-12-03 13:40:24 +0000 UTC" firstStartedPulling="2025-12-03 13:40:25.015670044 +0000 UTC m=+8415.603911808" lastFinishedPulling="2025-12-03 13:40:25.809605992 +0000 UTC m=+8416.397847766" observedRunningTime="2025-12-03 13:40:26.369394245 +0000 UTC m=+8416.957636009" watchObservedRunningTime="2025-12-03 13:40:26.373245896 +0000 UTC m=+8416.961487670" Dec 03 13:40:27 crc kubenswrapper[5052]: I1203 13:40:27.091891 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-operators/mariadb-operator-index-w76tp" Dec 03 13:40:27 crc kubenswrapper[5052]: I1203 13:40:27.210882 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lqjd2\" (UniqueName: \"kubernetes.io/projected/126387b4-22aa-4952-9293-0b9385c931bb-kube-api-access-lqjd2\") pod \"126387b4-22aa-4952-9293-0b9385c931bb\" (UID: \"126387b4-22aa-4952-9293-0b9385c931bb\") " Dec 03 13:40:27 crc kubenswrapper[5052]: I1203 13:40:27.216555 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/126387b4-22aa-4952-9293-0b9385c931bb-kube-api-access-lqjd2" (OuterVolumeSpecName: "kube-api-access-lqjd2") pod "126387b4-22aa-4952-9293-0b9385c931bb" (UID: "126387b4-22aa-4952-9293-0b9385c931bb"). InnerVolumeSpecName "kube-api-access-lqjd2". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:40:27 crc kubenswrapper[5052]: I1203 13:40:27.313245 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-lqjd2\" (UniqueName: \"kubernetes.io/projected/126387b4-22aa-4952-9293-0b9385c931bb-kube-api-access-lqjd2\") on node \"crc\" DevicePath \"\"" Dec 03 13:40:27 crc kubenswrapper[5052]: I1203 13:40:27.366231 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/mariadb-operator-index-p2x5s" event={"ID":"f82e13e4-6fd4-4bf8-8d78-c00e7b5a402e","Type":"ContainerStarted","Data":"29ad16f9f611514fc76240ac2c1cfa6a3a5cb4213b29155759c964ed5238361c"} Dec 03 13:40:27 crc kubenswrapper[5052]: I1203 13:40:27.368031 5052 generic.go:334] "Generic (PLEG): container finished" podID="126387b4-22aa-4952-9293-0b9385c931bb" containerID="cc4d42905b6e71c81cae84432de7ae2e8d457105d23fb74502b79fa527e29895" exitCode=0 Dec 03 13:40:27 crc kubenswrapper[5052]: I1203 13:40:27.368062 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/mariadb-operator-index-w76tp" event={"ID":"126387b4-22aa-4952-9293-0b9385c931bb","Type":"ContainerDied","Data":"cc4d42905b6e71c81cae84432de7ae2e8d457105d23fb74502b79fa527e29895"} Dec 03 13:40:27 crc kubenswrapper[5052]: I1203 13:40:27.368081 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/mariadb-operator-index-w76tp" event={"ID":"126387b4-22aa-4952-9293-0b9385c931bb","Type":"ContainerDied","Data":"ba5d1e17782ca277fa5d7f434c14a4c3184792fea85a3d226081dbb0d612d24d"} Dec 03 13:40:27 crc kubenswrapper[5052]: I1203 13:40:27.368091 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-operators/mariadb-operator-index-w76tp" Dec 03 13:40:27 crc kubenswrapper[5052]: I1203 13:40:27.368099 5052 scope.go:117] "RemoveContainer" containerID="cc4d42905b6e71c81cae84432de7ae2e8d457105d23fb74502b79fa527e29895" Dec 03 13:40:27 crc kubenswrapper[5052]: I1203 13:40:27.389721 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-operators/mariadb-operator-index-p2x5s" podStartSLOduration=2.5437990360000002 podStartE2EDuration="3.389701035s" podCreationTimestamp="2025-12-03 13:40:24 +0000 UTC" firstStartedPulling="2025-12-03 13:40:25.397103282 +0000 UTC m=+8415.985345046" lastFinishedPulling="2025-12-03 13:40:26.243005291 +0000 UTC m=+8416.831247045" observedRunningTime="2025-12-03 13:40:27.381208811 +0000 UTC m=+8417.969450575" watchObservedRunningTime="2025-12-03 13:40:27.389701035 +0000 UTC m=+8417.977942819" Dec 03 13:40:27 crc kubenswrapper[5052]: I1203 13:40:27.397483 5052 scope.go:117] "RemoveContainer" containerID="cc4d42905b6e71c81cae84432de7ae2e8d457105d23fb74502b79fa527e29895" Dec 03 13:40:27 crc kubenswrapper[5052]: I1203 13:40:27.399240 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-operators/mariadb-operator-index-w76tp"] Dec 03 13:40:27 crc kubenswrapper[5052]: E1203 13:40:27.400726 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cc4d42905b6e71c81cae84432de7ae2e8d457105d23fb74502b79fa527e29895\": container with ID starting with cc4d42905b6e71c81cae84432de7ae2e8d457105d23fb74502b79fa527e29895 not found: ID does not exist" containerID="cc4d42905b6e71c81cae84432de7ae2e8d457105d23fb74502b79fa527e29895" Dec 03 13:40:27 crc kubenswrapper[5052]: I1203 13:40:27.400763 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cc4d42905b6e71c81cae84432de7ae2e8d457105d23fb74502b79fa527e29895"} err="failed to get container status \"cc4d42905b6e71c81cae84432de7ae2e8d457105d23fb74502b79fa527e29895\": rpc error: code = NotFound desc = could not find container \"cc4d42905b6e71c81cae84432de7ae2e8d457105d23fb74502b79fa527e29895\": container with ID starting with cc4d42905b6e71c81cae84432de7ae2e8d457105d23fb74502b79fa527e29895 not found: ID does not exist" Dec 03 13:40:27 crc kubenswrapper[5052]: I1203 13:40:27.403559 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-operators/mariadb-operator-index-w76tp"] Dec 03 13:40:27 crc kubenswrapper[5052]: I1203 13:40:27.680740 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="126387b4-22aa-4952-9293-0b9385c931bb" path="/var/lib/kubelet/pods/126387b4-22aa-4952-9293-0b9385c931bb/volumes" Dec 03 13:40:29 crc kubenswrapper[5052]: I1203 13:40:29.379024 5052 patch_prober.go:28] interesting pod/machine-config-daemon-74bj6 container/machine-config-daemon namespace/openshift-machine-config-operator: Liveness probe status=failure output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" start-of-body= Dec 03 13:40:29 crc kubenswrapper[5052]: I1203 13:40:29.379076 5052 prober.go:107] "Probe failed" probeType="Liveness" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" containerName="machine-config-daemon" probeResult="failure" output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" Dec 03 13:40:31 crc kubenswrapper[5052]: I1203 13:40:31.292323 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-operators/55a3886cc1ed42812df4eab61c7a6033dc924d195539e8545c8f175f6194wgk"] Dec 03 13:40:31 crc kubenswrapper[5052]: E1203 13:40:31.292617 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="126387b4-22aa-4952-9293-0b9385c931bb" containerName="registry-server" Dec 03 13:40:31 crc kubenswrapper[5052]: I1203 13:40:31.292628 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="126387b4-22aa-4952-9293-0b9385c931bb" containerName="registry-server" Dec 03 13:40:31 crc kubenswrapper[5052]: I1203 13:40:31.292733 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="126387b4-22aa-4952-9293-0b9385c931bb" containerName="registry-server" Dec 03 13:40:31 crc kubenswrapper[5052]: I1203 13:40:31.293616 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-operators/55a3886cc1ed42812df4eab61c7a6033dc924d195539e8545c8f175f6194wgk" Dec 03 13:40:31 crc kubenswrapper[5052]: I1203 13:40:31.295734 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-operators"/"default-dockercfg-7fklr" Dec 03 13:40:31 crc kubenswrapper[5052]: I1203 13:40:31.304750 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-operators/55a3886cc1ed42812df4eab61c7a6033dc924d195539e8545c8f175f6194wgk"] Dec 03 13:40:31 crc kubenswrapper[5052]: I1203 13:40:31.370781 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/11c2bcbf-8d83-494a-8019-e2f34b7b1125-bundle\") pod \"55a3886cc1ed42812df4eab61c7a6033dc924d195539e8545c8f175f6194wgk\" (UID: \"11c2bcbf-8d83-494a-8019-e2f34b7b1125\") " pod="openstack-operators/55a3886cc1ed42812df4eab61c7a6033dc924d195539e8545c8f175f6194wgk" Dec 03 13:40:31 crc kubenswrapper[5052]: I1203 13:40:31.370845 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/11c2bcbf-8d83-494a-8019-e2f34b7b1125-util\") pod \"55a3886cc1ed42812df4eab61c7a6033dc924d195539e8545c8f175f6194wgk\" (UID: \"11c2bcbf-8d83-494a-8019-e2f34b7b1125\") " pod="openstack-operators/55a3886cc1ed42812df4eab61c7a6033dc924d195539e8545c8f175f6194wgk" Dec 03 13:40:31 crc kubenswrapper[5052]: I1203 13:40:31.370896 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s7qnq\" (UniqueName: \"kubernetes.io/projected/11c2bcbf-8d83-494a-8019-e2f34b7b1125-kube-api-access-s7qnq\") pod \"55a3886cc1ed42812df4eab61c7a6033dc924d195539e8545c8f175f6194wgk\" (UID: \"11c2bcbf-8d83-494a-8019-e2f34b7b1125\") " pod="openstack-operators/55a3886cc1ed42812df4eab61c7a6033dc924d195539e8545c8f175f6194wgk" Dec 03 13:40:31 crc kubenswrapper[5052]: I1203 13:40:31.471975 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/11c2bcbf-8d83-494a-8019-e2f34b7b1125-bundle\") pod \"55a3886cc1ed42812df4eab61c7a6033dc924d195539e8545c8f175f6194wgk\" (UID: \"11c2bcbf-8d83-494a-8019-e2f34b7b1125\") " pod="openstack-operators/55a3886cc1ed42812df4eab61c7a6033dc924d195539e8545c8f175f6194wgk" Dec 03 13:40:31 crc kubenswrapper[5052]: I1203 13:40:31.472039 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/11c2bcbf-8d83-494a-8019-e2f34b7b1125-util\") pod \"55a3886cc1ed42812df4eab61c7a6033dc924d195539e8545c8f175f6194wgk\" (UID: \"11c2bcbf-8d83-494a-8019-e2f34b7b1125\") " pod="openstack-operators/55a3886cc1ed42812df4eab61c7a6033dc924d195539e8545c8f175f6194wgk" Dec 03 13:40:31 crc kubenswrapper[5052]: I1203 13:40:31.472085 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-s7qnq\" (UniqueName: \"kubernetes.io/projected/11c2bcbf-8d83-494a-8019-e2f34b7b1125-kube-api-access-s7qnq\") pod \"55a3886cc1ed42812df4eab61c7a6033dc924d195539e8545c8f175f6194wgk\" (UID: \"11c2bcbf-8d83-494a-8019-e2f34b7b1125\") " pod="openstack-operators/55a3886cc1ed42812df4eab61c7a6033dc924d195539e8545c8f175f6194wgk" Dec 03 13:40:31 crc kubenswrapper[5052]: I1203 13:40:31.472701 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/11c2bcbf-8d83-494a-8019-e2f34b7b1125-bundle\") pod \"55a3886cc1ed42812df4eab61c7a6033dc924d195539e8545c8f175f6194wgk\" (UID: \"11c2bcbf-8d83-494a-8019-e2f34b7b1125\") " pod="openstack-operators/55a3886cc1ed42812df4eab61c7a6033dc924d195539e8545c8f175f6194wgk" Dec 03 13:40:31 crc kubenswrapper[5052]: I1203 13:40:31.472732 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/11c2bcbf-8d83-494a-8019-e2f34b7b1125-util\") pod \"55a3886cc1ed42812df4eab61c7a6033dc924d195539e8545c8f175f6194wgk\" (UID: \"11c2bcbf-8d83-494a-8019-e2f34b7b1125\") " pod="openstack-operators/55a3886cc1ed42812df4eab61c7a6033dc924d195539e8545c8f175f6194wgk" Dec 03 13:40:31 crc kubenswrapper[5052]: I1203 13:40:31.493142 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-s7qnq\" (UniqueName: \"kubernetes.io/projected/11c2bcbf-8d83-494a-8019-e2f34b7b1125-kube-api-access-s7qnq\") pod \"55a3886cc1ed42812df4eab61c7a6033dc924d195539e8545c8f175f6194wgk\" (UID: \"11c2bcbf-8d83-494a-8019-e2f34b7b1125\") " pod="openstack-operators/55a3886cc1ed42812df4eab61c7a6033dc924d195539e8545c8f175f6194wgk" Dec 03 13:40:31 crc kubenswrapper[5052]: I1203 13:40:31.611227 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-operators/55a3886cc1ed42812df4eab61c7a6033dc924d195539e8545c8f175f6194wgk" Dec 03 13:40:32 crc kubenswrapper[5052]: I1203 13:40:32.211803 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-operators/55a3886cc1ed42812df4eab61c7a6033dc924d195539e8545c8f175f6194wgk"] Dec 03 13:40:32 crc kubenswrapper[5052]: I1203 13:40:32.400889 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/55a3886cc1ed42812df4eab61c7a6033dc924d195539e8545c8f175f6194wgk" event={"ID":"11c2bcbf-8d83-494a-8019-e2f34b7b1125","Type":"ContainerStarted","Data":"7719385fe02b50ef2050bd44cc01ace4022f136cb0014df36a26664c6e047801"} Dec 03 13:40:33 crc kubenswrapper[5052]: I1203 13:40:33.408355 5052 generic.go:334] "Generic (PLEG): container finished" podID="11c2bcbf-8d83-494a-8019-e2f34b7b1125" containerID="389d7bcd41004f53ddb05546d5ccd89efddb3c3608f638ed464fec31dc8be00f" exitCode=0 Dec 03 13:40:33 crc kubenswrapper[5052]: I1203 13:40:33.408417 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/55a3886cc1ed42812df4eab61c7a6033dc924d195539e8545c8f175f6194wgk" event={"ID":"11c2bcbf-8d83-494a-8019-e2f34b7b1125","Type":"ContainerDied","Data":"389d7bcd41004f53ddb05546d5ccd89efddb3c3608f638ed464fec31dc8be00f"} Dec 03 13:40:34 crc kubenswrapper[5052]: I1203 13:40:34.440616 5052 generic.go:334] "Generic (PLEG): container finished" podID="11c2bcbf-8d83-494a-8019-e2f34b7b1125" containerID="c3d33d9edee4d4da19e64abf662598c36eceea009e210193291bdb4169fa0983" exitCode=0 Dec 03 13:40:34 crc kubenswrapper[5052]: I1203 13:40:34.441340 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/55a3886cc1ed42812df4eab61c7a6033dc924d195539e8545c8f175f6194wgk" event={"ID":"11c2bcbf-8d83-494a-8019-e2f34b7b1125","Type":"ContainerDied","Data":"c3d33d9edee4d4da19e64abf662598c36eceea009e210193291bdb4169fa0983"} Dec 03 13:40:35 crc kubenswrapper[5052]: I1203 13:40:35.169421 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-operators/mariadb-operator-index-p2x5s" Dec 03 13:40:35 crc kubenswrapper[5052]: I1203 13:40:35.169783 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-operators/mariadb-operator-index-p2x5s" Dec 03 13:40:35 crc kubenswrapper[5052]: I1203 13:40:35.199014 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-operators/mariadb-operator-index-p2x5s" Dec 03 13:40:35 crc kubenswrapper[5052]: I1203 13:40:35.450119 5052 generic.go:334] "Generic (PLEG): container finished" podID="11c2bcbf-8d83-494a-8019-e2f34b7b1125" containerID="bf666d027e71853bcf0130e438fe6020c28e6d75ee676b585d4aba6febc4bc3e" exitCode=0 Dec 03 13:40:35 crc kubenswrapper[5052]: I1203 13:40:35.450171 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/55a3886cc1ed42812df4eab61c7a6033dc924d195539e8545c8f175f6194wgk" event={"ID":"11c2bcbf-8d83-494a-8019-e2f34b7b1125","Type":"ContainerDied","Data":"bf666d027e71853bcf0130e438fe6020c28e6d75ee676b585d4aba6febc4bc3e"} Dec 03 13:40:35 crc kubenswrapper[5052]: I1203 13:40:35.479620 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-operators/mariadb-operator-index-p2x5s" Dec 03 13:40:36 crc kubenswrapper[5052]: I1203 13:40:36.702562 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-operators/55a3886cc1ed42812df4eab61c7a6033dc924d195539e8545c8f175f6194wgk" Dec 03 13:40:36 crc kubenswrapper[5052]: I1203 13:40:36.862704 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/11c2bcbf-8d83-494a-8019-e2f34b7b1125-bundle\") pod \"11c2bcbf-8d83-494a-8019-e2f34b7b1125\" (UID: \"11c2bcbf-8d83-494a-8019-e2f34b7b1125\") " Dec 03 13:40:36 crc kubenswrapper[5052]: I1203 13:40:36.862802 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s7qnq\" (UniqueName: \"kubernetes.io/projected/11c2bcbf-8d83-494a-8019-e2f34b7b1125-kube-api-access-s7qnq\") pod \"11c2bcbf-8d83-494a-8019-e2f34b7b1125\" (UID: \"11c2bcbf-8d83-494a-8019-e2f34b7b1125\") " Dec 03 13:40:36 crc kubenswrapper[5052]: I1203 13:40:36.862868 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/11c2bcbf-8d83-494a-8019-e2f34b7b1125-util\") pod \"11c2bcbf-8d83-494a-8019-e2f34b7b1125\" (UID: \"11c2bcbf-8d83-494a-8019-e2f34b7b1125\") " Dec 03 13:40:36 crc kubenswrapper[5052]: I1203 13:40:36.864207 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/11c2bcbf-8d83-494a-8019-e2f34b7b1125-bundle" (OuterVolumeSpecName: "bundle") pod "11c2bcbf-8d83-494a-8019-e2f34b7b1125" (UID: "11c2bcbf-8d83-494a-8019-e2f34b7b1125"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 13:40:36 crc kubenswrapper[5052]: I1203 13:40:36.869751 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/11c2bcbf-8d83-494a-8019-e2f34b7b1125-kube-api-access-s7qnq" (OuterVolumeSpecName: "kube-api-access-s7qnq") pod "11c2bcbf-8d83-494a-8019-e2f34b7b1125" (UID: "11c2bcbf-8d83-494a-8019-e2f34b7b1125"). InnerVolumeSpecName "kube-api-access-s7qnq". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:40:36 crc kubenswrapper[5052]: I1203 13:40:36.879306 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/11c2bcbf-8d83-494a-8019-e2f34b7b1125-util" (OuterVolumeSpecName: "util") pod "11c2bcbf-8d83-494a-8019-e2f34b7b1125" (UID: "11c2bcbf-8d83-494a-8019-e2f34b7b1125"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 13:40:36 crc kubenswrapper[5052]: I1203 13:40:36.965047 5052 reconciler_common.go:293] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/11c2bcbf-8d83-494a-8019-e2f34b7b1125-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 13:40:36 crc kubenswrapper[5052]: I1203 13:40:36.965095 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-s7qnq\" (UniqueName: \"kubernetes.io/projected/11c2bcbf-8d83-494a-8019-e2f34b7b1125-kube-api-access-s7qnq\") on node \"crc\" DevicePath \"\"" Dec 03 13:40:36 crc kubenswrapper[5052]: I1203 13:40:36.965112 5052 reconciler_common.go:293] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/11c2bcbf-8d83-494a-8019-e2f34b7b1125-util\") on node \"crc\" DevicePath \"\"" Dec 03 13:40:37 crc kubenswrapper[5052]: I1203 13:40:37.464453 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/55a3886cc1ed42812df4eab61c7a6033dc924d195539e8545c8f175f6194wgk" event={"ID":"11c2bcbf-8d83-494a-8019-e2f34b7b1125","Type":"ContainerDied","Data":"7719385fe02b50ef2050bd44cc01ace4022f136cb0014df36a26664c6e047801"} Dec 03 13:40:37 crc kubenswrapper[5052]: I1203 13:40:37.464496 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7719385fe02b50ef2050bd44cc01ace4022f136cb0014df36a26664c6e047801" Dec 03 13:40:37 crc kubenswrapper[5052]: I1203 13:40:37.464840 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-operators/55a3886cc1ed42812df4eab61c7a6033dc924d195539e8545c8f175f6194wgk" Dec 03 13:40:45 crc kubenswrapper[5052]: I1203 13:40:45.457940 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-operators/mariadb-operator-controller-manager-5bdcd87d9d-b7mh5"] Dec 03 13:40:45 crc kubenswrapper[5052]: E1203 13:40:45.458954 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="11c2bcbf-8d83-494a-8019-e2f34b7b1125" containerName="extract" Dec 03 13:40:45 crc kubenswrapper[5052]: I1203 13:40:45.458973 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="11c2bcbf-8d83-494a-8019-e2f34b7b1125" containerName="extract" Dec 03 13:40:45 crc kubenswrapper[5052]: E1203 13:40:45.458987 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="11c2bcbf-8d83-494a-8019-e2f34b7b1125" containerName="util" Dec 03 13:40:45 crc kubenswrapper[5052]: I1203 13:40:45.458995 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="11c2bcbf-8d83-494a-8019-e2f34b7b1125" containerName="util" Dec 03 13:40:45 crc kubenswrapper[5052]: E1203 13:40:45.459032 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="11c2bcbf-8d83-494a-8019-e2f34b7b1125" containerName="pull" Dec 03 13:40:45 crc kubenswrapper[5052]: I1203 13:40:45.459040 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="11c2bcbf-8d83-494a-8019-e2f34b7b1125" containerName="pull" Dec 03 13:40:45 crc kubenswrapper[5052]: I1203 13:40:45.459206 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="11c2bcbf-8d83-494a-8019-e2f34b7b1125" containerName="extract" Dec 03 13:40:45 crc kubenswrapper[5052]: I1203 13:40:45.459840 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-operators/mariadb-operator-controller-manager-5bdcd87d9d-b7mh5" Dec 03 13:40:45 crc kubenswrapper[5052]: I1203 13:40:45.461869 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-operators"/"webhook-server-cert" Dec 03 13:40:45 crc kubenswrapper[5052]: I1203 13:40:45.462124 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-operators"/"mariadb-operator-controller-manager-service-cert" Dec 03 13:40:45 crc kubenswrapper[5052]: I1203 13:40:45.464722 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-operators"/"mariadb-operator-controller-manager-dockercfg-zd5pm" Dec 03 13:40:45 crc kubenswrapper[5052]: I1203 13:40:45.470821 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-operators/mariadb-operator-controller-manager-5bdcd87d9d-b7mh5"] Dec 03 13:40:45 crc kubenswrapper[5052]: I1203 13:40:45.593528 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kkdx2\" (UniqueName: \"kubernetes.io/projected/8400a794-64c3-4172-93fa-ceb4acef604d-kube-api-access-kkdx2\") pod \"mariadb-operator-controller-manager-5bdcd87d9d-b7mh5\" (UID: \"8400a794-64c3-4172-93fa-ceb4acef604d\") " pod="openstack-operators/mariadb-operator-controller-manager-5bdcd87d9d-b7mh5" Dec 03 13:40:45 crc kubenswrapper[5052]: I1203 13:40:45.593877 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/8400a794-64c3-4172-93fa-ceb4acef604d-apiservice-cert\") pod \"mariadb-operator-controller-manager-5bdcd87d9d-b7mh5\" (UID: \"8400a794-64c3-4172-93fa-ceb4acef604d\") " pod="openstack-operators/mariadb-operator-controller-manager-5bdcd87d9d-b7mh5" Dec 03 13:40:45 crc kubenswrapper[5052]: I1203 13:40:45.593944 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8400a794-64c3-4172-93fa-ceb4acef604d-webhook-cert\") pod \"mariadb-operator-controller-manager-5bdcd87d9d-b7mh5\" (UID: \"8400a794-64c3-4172-93fa-ceb4acef604d\") " pod="openstack-operators/mariadb-operator-controller-manager-5bdcd87d9d-b7mh5" Dec 03 13:40:45 crc kubenswrapper[5052]: I1203 13:40:45.695209 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-kkdx2\" (UniqueName: \"kubernetes.io/projected/8400a794-64c3-4172-93fa-ceb4acef604d-kube-api-access-kkdx2\") pod \"mariadb-operator-controller-manager-5bdcd87d9d-b7mh5\" (UID: \"8400a794-64c3-4172-93fa-ceb4acef604d\") " pod="openstack-operators/mariadb-operator-controller-manager-5bdcd87d9d-b7mh5" Dec 03 13:40:45 crc kubenswrapper[5052]: I1203 13:40:45.695262 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/8400a794-64c3-4172-93fa-ceb4acef604d-apiservice-cert\") pod \"mariadb-operator-controller-manager-5bdcd87d9d-b7mh5\" (UID: \"8400a794-64c3-4172-93fa-ceb4acef604d\") " pod="openstack-operators/mariadb-operator-controller-manager-5bdcd87d9d-b7mh5" Dec 03 13:40:45 crc kubenswrapper[5052]: I1203 13:40:45.695324 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8400a794-64c3-4172-93fa-ceb4acef604d-webhook-cert\") pod \"mariadb-operator-controller-manager-5bdcd87d9d-b7mh5\" (UID: \"8400a794-64c3-4172-93fa-ceb4acef604d\") " pod="openstack-operators/mariadb-operator-controller-manager-5bdcd87d9d-b7mh5" Dec 03 13:40:45 crc kubenswrapper[5052]: I1203 13:40:45.702180 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8400a794-64c3-4172-93fa-ceb4acef604d-webhook-cert\") pod \"mariadb-operator-controller-manager-5bdcd87d9d-b7mh5\" (UID: \"8400a794-64c3-4172-93fa-ceb4acef604d\") " pod="openstack-operators/mariadb-operator-controller-manager-5bdcd87d9d-b7mh5" Dec 03 13:40:45 crc kubenswrapper[5052]: I1203 13:40:45.702187 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/8400a794-64c3-4172-93fa-ceb4acef604d-apiservice-cert\") pod \"mariadb-operator-controller-manager-5bdcd87d9d-b7mh5\" (UID: \"8400a794-64c3-4172-93fa-ceb4acef604d\") " pod="openstack-operators/mariadb-operator-controller-manager-5bdcd87d9d-b7mh5" Dec 03 13:40:45 crc kubenswrapper[5052]: I1203 13:40:45.714377 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-kkdx2\" (UniqueName: \"kubernetes.io/projected/8400a794-64c3-4172-93fa-ceb4acef604d-kube-api-access-kkdx2\") pod \"mariadb-operator-controller-manager-5bdcd87d9d-b7mh5\" (UID: \"8400a794-64c3-4172-93fa-ceb4acef604d\") " pod="openstack-operators/mariadb-operator-controller-manager-5bdcd87d9d-b7mh5" Dec 03 13:40:45 crc kubenswrapper[5052]: I1203 13:40:45.786788 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-operators/mariadb-operator-controller-manager-5bdcd87d9d-b7mh5" Dec 03 13:40:46 crc kubenswrapper[5052]: I1203 13:40:46.012660 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-operators/mariadb-operator-controller-manager-5bdcd87d9d-b7mh5"] Dec 03 13:40:46 crc kubenswrapper[5052]: W1203 13:40:46.017339 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8400a794_64c3_4172_93fa_ceb4acef604d.slice/crio-ed7aff77cd517c1dbb79af44d954a67ad4672631c50588bbc8f241f8f4b1b277 WatchSource:0}: Error finding container ed7aff77cd517c1dbb79af44d954a67ad4672631c50588bbc8f241f8f4b1b277: Status 404 returned error can't find the container with id ed7aff77cd517c1dbb79af44d954a67ad4672631c50588bbc8f241f8f4b1b277 Dec 03 13:40:46 crc kubenswrapper[5052]: I1203 13:40:46.521262 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/mariadb-operator-controller-manager-5bdcd87d9d-b7mh5" event={"ID":"8400a794-64c3-4172-93fa-ceb4acef604d","Type":"ContainerStarted","Data":"f09e2aed5955802de7fb6d5d419275a5a96d96c3647698e76456aabab1a980c9"} Dec 03 13:40:46 crc kubenswrapper[5052]: I1203 13:40:46.521316 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/mariadb-operator-controller-manager-5bdcd87d9d-b7mh5" event={"ID":"8400a794-64c3-4172-93fa-ceb4acef604d","Type":"ContainerStarted","Data":"ed7aff77cd517c1dbb79af44d954a67ad4672631c50588bbc8f241f8f4b1b277"} Dec 03 13:40:46 crc kubenswrapper[5052]: I1203 13:40:46.521554 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-operators/mariadb-operator-controller-manager-5bdcd87d9d-b7mh5" Dec 03 13:40:46 crc kubenswrapper[5052]: I1203 13:40:46.542201 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-operators/mariadb-operator-controller-manager-5bdcd87d9d-b7mh5" podStartSLOduration=1.5421801290000001 podStartE2EDuration="1.542180129s" podCreationTimestamp="2025-12-03 13:40:45 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 13:40:46.536099225 +0000 UTC m=+8437.124340999" watchObservedRunningTime="2025-12-03 13:40:46.542180129 +0000 UTC m=+8437.130421893" Dec 03 13:40:55 crc kubenswrapper[5052]: I1203 13:40:55.791123 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-operators/mariadb-operator-controller-manager-5bdcd87d9d-b7mh5" Dec 03 13:40:59 crc kubenswrapper[5052]: I1203 13:40:59.379794 5052 patch_prober.go:28] interesting pod/machine-config-daemon-74bj6 container/machine-config-daemon namespace/openshift-machine-config-operator: Liveness probe status=failure output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" start-of-body= Dec 03 13:40:59 crc kubenswrapper[5052]: I1203 13:40:59.380820 5052 prober.go:107] "Probe failed" probeType="Liveness" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" containerName="machine-config-daemon" probeResult="failure" output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" Dec 03 13:40:59 crc kubenswrapper[5052]: I1203 13:40:59.986052 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-operators/infra-operator-index-hvvnf"] Dec 03 13:40:59 crc kubenswrapper[5052]: I1203 13:40:59.987080 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-operators/infra-operator-index-hvvnf" Dec 03 13:40:59 crc kubenswrapper[5052]: I1203 13:40:59.990454 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-operators"/"infra-operator-index-dockercfg-lqwkn" Dec 03 13:40:59 crc kubenswrapper[5052]: I1203 13:40:59.997935 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-operators/infra-operator-index-hvvnf"] Dec 03 13:41:00 crc kubenswrapper[5052]: I1203 13:41:00.014440 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-msftx\" (UniqueName: \"kubernetes.io/projected/fb9f28a9-2859-4b61-9eb5-2d4712fab081-kube-api-access-msftx\") pod \"infra-operator-index-hvvnf\" (UID: \"fb9f28a9-2859-4b61-9eb5-2d4712fab081\") " pod="openstack-operators/infra-operator-index-hvvnf" Dec 03 13:41:00 crc kubenswrapper[5052]: I1203 13:41:00.116299 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-msftx\" (UniqueName: \"kubernetes.io/projected/fb9f28a9-2859-4b61-9eb5-2d4712fab081-kube-api-access-msftx\") pod \"infra-operator-index-hvvnf\" (UID: \"fb9f28a9-2859-4b61-9eb5-2d4712fab081\") " pod="openstack-operators/infra-operator-index-hvvnf" Dec 03 13:41:00 crc kubenswrapper[5052]: I1203 13:41:00.135396 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-msftx\" (UniqueName: \"kubernetes.io/projected/fb9f28a9-2859-4b61-9eb5-2d4712fab081-kube-api-access-msftx\") pod \"infra-operator-index-hvvnf\" (UID: \"fb9f28a9-2859-4b61-9eb5-2d4712fab081\") " pod="openstack-operators/infra-operator-index-hvvnf" Dec 03 13:41:00 crc kubenswrapper[5052]: I1203 13:41:00.308257 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-operators/infra-operator-index-hvvnf" Dec 03 13:41:00 crc kubenswrapper[5052]: I1203 13:41:00.718468 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-operators/infra-operator-index-hvvnf"] Dec 03 13:41:00 crc kubenswrapper[5052]: W1203 13:41:00.725332 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb9f28a9_2859_4b61_9eb5_2d4712fab081.slice/crio-a8d359b02843ea270015de9122e3a8d9c49698530d942df2448335adbaeaac6e WatchSource:0}: Error finding container a8d359b02843ea270015de9122e3a8d9c49698530d942df2448335adbaeaac6e: Status 404 returned error can't find the container with id a8d359b02843ea270015de9122e3a8d9c49698530d942df2448335adbaeaac6e Dec 03 13:41:01 crc kubenswrapper[5052]: I1203 13:41:01.622343 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/infra-operator-index-hvvnf" event={"ID":"fb9f28a9-2859-4b61-9eb5-2d4712fab081","Type":"ContainerStarted","Data":"6a375902732a2b70688a0c9badb55235a7a2f0502a7f947a9da4079db9ded40e"} Dec 03 13:41:01 crc kubenswrapper[5052]: I1203 13:41:01.622402 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/infra-operator-index-hvvnf" event={"ID":"fb9f28a9-2859-4b61-9eb5-2d4712fab081","Type":"ContainerStarted","Data":"a8d359b02843ea270015de9122e3a8d9c49698530d942df2448335adbaeaac6e"} Dec 03 13:41:01 crc kubenswrapper[5052]: I1203 13:41:01.640807 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-operators/infra-operator-index-hvvnf" podStartSLOduration=1.9352400379999999 podStartE2EDuration="2.640786201s" podCreationTimestamp="2025-12-03 13:40:59 +0000 UTC" firstStartedPulling="2025-12-03 13:41:00.728448948 +0000 UTC m=+8451.316690712" lastFinishedPulling="2025-12-03 13:41:01.433995111 +0000 UTC m=+8452.022236875" observedRunningTime="2025-12-03 13:41:01.637211689 +0000 UTC m=+8452.225453453" watchObservedRunningTime="2025-12-03 13:41:01.640786201 +0000 UTC m=+8452.229027975" Dec 03 13:41:04 crc kubenswrapper[5052]: I1203 13:41:04.162672 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-operators/infra-operator-index-hvvnf"] Dec 03 13:41:04 crc kubenswrapper[5052]: I1203 13:41:04.164067 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-operators/infra-operator-index-hvvnf" podUID="fb9f28a9-2859-4b61-9eb5-2d4712fab081" containerName="registry-server" containerID="cri-o://6a375902732a2b70688a0c9badb55235a7a2f0502a7f947a9da4079db9ded40e" gracePeriod=2 Dec 03 13:41:04 crc kubenswrapper[5052]: I1203 13:41:04.643759 5052 generic.go:334] "Generic (PLEG): container finished" podID="fb9f28a9-2859-4b61-9eb5-2d4712fab081" containerID="6a375902732a2b70688a0c9badb55235a7a2f0502a7f947a9da4079db9ded40e" exitCode=0 Dec 03 13:41:04 crc kubenswrapper[5052]: I1203 13:41:04.643802 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/infra-operator-index-hvvnf" event={"ID":"fb9f28a9-2859-4b61-9eb5-2d4712fab081","Type":"ContainerDied","Data":"6a375902732a2b70688a0c9badb55235a7a2f0502a7f947a9da4079db9ded40e"} Dec 03 13:41:04 crc kubenswrapper[5052]: I1203 13:41:04.765699 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-operators/infra-operator-index-5rfr5"] Dec 03 13:41:04 crc kubenswrapper[5052]: I1203 13:41:04.767014 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-operators/infra-operator-index-5rfr5" Dec 03 13:41:04 crc kubenswrapper[5052]: I1203 13:41:04.773249 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-operators/infra-operator-index-5rfr5"] Dec 03 13:41:04 crc kubenswrapper[5052]: I1203 13:41:04.815309 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qxwhk\" (UniqueName: \"kubernetes.io/projected/a8c2a4d6-1e91-40de-9edc-60b79b888d1f-kube-api-access-qxwhk\") pod \"infra-operator-index-5rfr5\" (UID: \"a8c2a4d6-1e91-40de-9edc-60b79b888d1f\") " pod="openstack-operators/infra-operator-index-5rfr5" Dec 03 13:41:04 crc kubenswrapper[5052]: I1203 13:41:04.916780 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-qxwhk\" (UniqueName: \"kubernetes.io/projected/a8c2a4d6-1e91-40de-9edc-60b79b888d1f-kube-api-access-qxwhk\") pod \"infra-operator-index-5rfr5\" (UID: \"a8c2a4d6-1e91-40de-9edc-60b79b888d1f\") " pod="openstack-operators/infra-operator-index-5rfr5" Dec 03 13:41:04 crc kubenswrapper[5052]: I1203 13:41:04.960324 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-qxwhk\" (UniqueName: \"kubernetes.io/projected/a8c2a4d6-1e91-40de-9edc-60b79b888d1f-kube-api-access-qxwhk\") pod \"infra-operator-index-5rfr5\" (UID: \"a8c2a4d6-1e91-40de-9edc-60b79b888d1f\") " pod="openstack-operators/infra-operator-index-5rfr5" Dec 03 13:41:05 crc kubenswrapper[5052]: I1203 13:41:05.135004 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-operators/infra-operator-index-5rfr5" Dec 03 13:41:05 crc kubenswrapper[5052]: I1203 13:41:05.197728 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-operators/infra-operator-index-hvvnf" Dec 03 13:41:05 crc kubenswrapper[5052]: I1203 13:41:05.326684 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-msftx\" (UniqueName: \"kubernetes.io/projected/fb9f28a9-2859-4b61-9eb5-2d4712fab081-kube-api-access-msftx\") pod \"fb9f28a9-2859-4b61-9eb5-2d4712fab081\" (UID: \"fb9f28a9-2859-4b61-9eb5-2d4712fab081\") " Dec 03 13:41:05 crc kubenswrapper[5052]: I1203 13:41:05.332805 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fb9f28a9-2859-4b61-9eb5-2d4712fab081-kube-api-access-msftx" (OuterVolumeSpecName: "kube-api-access-msftx") pod "fb9f28a9-2859-4b61-9eb5-2d4712fab081" (UID: "fb9f28a9-2859-4b61-9eb5-2d4712fab081"). InnerVolumeSpecName "kube-api-access-msftx". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:41:05 crc kubenswrapper[5052]: I1203 13:41:05.404034 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-operators/5d473c3169f40b179d14921c90af2c8546b7b757fe551b7dba7d903f5dr8sfn"] Dec 03 13:41:05 crc kubenswrapper[5052]: E1203 13:41:05.404368 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="fb9f28a9-2859-4b61-9eb5-2d4712fab081" containerName="registry-server" Dec 03 13:41:05 crc kubenswrapper[5052]: I1203 13:41:05.404390 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb9f28a9-2859-4b61-9eb5-2d4712fab081" containerName="registry-server" Dec 03 13:41:05 crc kubenswrapper[5052]: I1203 13:41:05.404588 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="fb9f28a9-2859-4b61-9eb5-2d4712fab081" containerName="registry-server" Dec 03 13:41:05 crc kubenswrapper[5052]: I1203 13:41:05.406658 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-operators/5d473c3169f40b179d14921c90af2c8546b7b757fe551b7dba7d903f5dr8sfn" Dec 03 13:41:05 crc kubenswrapper[5052]: I1203 13:41:05.409166 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-operators"/"default-dockercfg-7fklr" Dec 03 13:41:05 crc kubenswrapper[5052]: I1203 13:41:05.416152 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-operators/5d473c3169f40b179d14921c90af2c8546b7b757fe551b7dba7d903f5dr8sfn"] Dec 03 13:41:05 crc kubenswrapper[5052]: I1203 13:41:05.427963 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-msftx\" (UniqueName: \"kubernetes.io/projected/fb9f28a9-2859-4b61-9eb5-2d4712fab081-kube-api-access-msftx\") on node \"crc\" DevicePath \"\"" Dec 03 13:41:05 crc kubenswrapper[5052]: I1203 13:41:05.529354 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mzbl4\" (UniqueName: \"kubernetes.io/projected/2aaa3b0a-cd10-42b4-8c0d-fc9317d22547-kube-api-access-mzbl4\") pod \"5d473c3169f40b179d14921c90af2c8546b7b757fe551b7dba7d903f5dr8sfn\" (UID: \"2aaa3b0a-cd10-42b4-8c0d-fc9317d22547\") " pod="openstack-operators/5d473c3169f40b179d14921c90af2c8546b7b757fe551b7dba7d903f5dr8sfn" Dec 03 13:41:05 crc kubenswrapper[5052]: I1203 13:41:05.529400 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2aaa3b0a-cd10-42b4-8c0d-fc9317d22547-util\") pod \"5d473c3169f40b179d14921c90af2c8546b7b757fe551b7dba7d903f5dr8sfn\" (UID: \"2aaa3b0a-cd10-42b4-8c0d-fc9317d22547\") " pod="openstack-operators/5d473c3169f40b179d14921c90af2c8546b7b757fe551b7dba7d903f5dr8sfn" Dec 03 13:41:05 crc kubenswrapper[5052]: I1203 13:41:05.529442 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2aaa3b0a-cd10-42b4-8c0d-fc9317d22547-bundle\") pod \"5d473c3169f40b179d14921c90af2c8546b7b757fe551b7dba7d903f5dr8sfn\" (UID: \"2aaa3b0a-cd10-42b4-8c0d-fc9317d22547\") " pod="openstack-operators/5d473c3169f40b179d14921c90af2c8546b7b757fe551b7dba7d903f5dr8sfn" Dec 03 13:41:05 crc kubenswrapper[5052]: I1203 13:41:05.594294 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-operators/infra-operator-index-5rfr5"] Dec 03 13:41:05 crc kubenswrapper[5052]: W1203 13:41:05.600240 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8c2a4d6_1e91_40de_9edc_60b79b888d1f.slice/crio-02e30a400a6bcd15b0cc33f7903897d8c54556d43d11a15a3b0b3a7f9f4db5f9 WatchSource:0}: Error finding container 02e30a400a6bcd15b0cc33f7903897d8c54556d43d11a15a3b0b3a7f9f4db5f9: Status 404 returned error can't find the container with id 02e30a400a6bcd15b0cc33f7903897d8c54556d43d11a15a3b0b3a7f9f4db5f9 Dec 03 13:41:05 crc kubenswrapper[5052]: I1203 13:41:05.630541 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-mzbl4\" (UniqueName: \"kubernetes.io/projected/2aaa3b0a-cd10-42b4-8c0d-fc9317d22547-kube-api-access-mzbl4\") pod \"5d473c3169f40b179d14921c90af2c8546b7b757fe551b7dba7d903f5dr8sfn\" (UID: \"2aaa3b0a-cd10-42b4-8c0d-fc9317d22547\") " pod="openstack-operators/5d473c3169f40b179d14921c90af2c8546b7b757fe551b7dba7d903f5dr8sfn" Dec 03 13:41:05 crc kubenswrapper[5052]: I1203 13:41:05.630606 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2aaa3b0a-cd10-42b4-8c0d-fc9317d22547-util\") pod \"5d473c3169f40b179d14921c90af2c8546b7b757fe551b7dba7d903f5dr8sfn\" (UID: \"2aaa3b0a-cd10-42b4-8c0d-fc9317d22547\") " pod="openstack-operators/5d473c3169f40b179d14921c90af2c8546b7b757fe551b7dba7d903f5dr8sfn" Dec 03 13:41:05 crc kubenswrapper[5052]: I1203 13:41:05.630663 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2aaa3b0a-cd10-42b4-8c0d-fc9317d22547-bundle\") pod \"5d473c3169f40b179d14921c90af2c8546b7b757fe551b7dba7d903f5dr8sfn\" (UID: \"2aaa3b0a-cd10-42b4-8c0d-fc9317d22547\") " pod="openstack-operators/5d473c3169f40b179d14921c90af2c8546b7b757fe551b7dba7d903f5dr8sfn" Dec 03 13:41:05 crc kubenswrapper[5052]: I1203 13:41:05.631226 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2aaa3b0a-cd10-42b4-8c0d-fc9317d22547-bundle\") pod \"5d473c3169f40b179d14921c90af2c8546b7b757fe551b7dba7d903f5dr8sfn\" (UID: \"2aaa3b0a-cd10-42b4-8c0d-fc9317d22547\") " pod="openstack-operators/5d473c3169f40b179d14921c90af2c8546b7b757fe551b7dba7d903f5dr8sfn" Dec 03 13:41:05 crc kubenswrapper[5052]: I1203 13:41:05.631437 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2aaa3b0a-cd10-42b4-8c0d-fc9317d22547-util\") pod \"5d473c3169f40b179d14921c90af2c8546b7b757fe551b7dba7d903f5dr8sfn\" (UID: \"2aaa3b0a-cd10-42b4-8c0d-fc9317d22547\") " pod="openstack-operators/5d473c3169f40b179d14921c90af2c8546b7b757fe551b7dba7d903f5dr8sfn" Dec 03 13:41:05 crc kubenswrapper[5052]: I1203 13:41:05.649323 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-mzbl4\" (UniqueName: \"kubernetes.io/projected/2aaa3b0a-cd10-42b4-8c0d-fc9317d22547-kube-api-access-mzbl4\") pod \"5d473c3169f40b179d14921c90af2c8546b7b757fe551b7dba7d903f5dr8sfn\" (UID: \"2aaa3b0a-cd10-42b4-8c0d-fc9317d22547\") " pod="openstack-operators/5d473c3169f40b179d14921c90af2c8546b7b757fe551b7dba7d903f5dr8sfn" Dec 03 13:41:05 crc kubenswrapper[5052]: I1203 13:41:05.650806 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/infra-operator-index-hvvnf" event={"ID":"fb9f28a9-2859-4b61-9eb5-2d4712fab081","Type":"ContainerDied","Data":"a8d359b02843ea270015de9122e3a8d9c49698530d942df2448335adbaeaac6e"} Dec 03 13:41:05 crc kubenswrapper[5052]: I1203 13:41:05.650853 5052 scope.go:117] "RemoveContainer" containerID="6a375902732a2b70688a0c9badb55235a7a2f0502a7f947a9da4079db9ded40e" Dec 03 13:41:05 crc kubenswrapper[5052]: I1203 13:41:05.650863 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-operators/infra-operator-index-hvvnf" Dec 03 13:41:05 crc kubenswrapper[5052]: I1203 13:41:05.654756 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/infra-operator-index-5rfr5" event={"ID":"a8c2a4d6-1e91-40de-9edc-60b79b888d1f","Type":"ContainerStarted","Data":"02e30a400a6bcd15b0cc33f7903897d8c54556d43d11a15a3b0b3a7f9f4db5f9"} Dec 03 13:41:05 crc kubenswrapper[5052]: I1203 13:41:05.697404 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-operators/infra-operator-index-hvvnf"] Dec 03 13:41:05 crc kubenswrapper[5052]: I1203 13:41:05.702425 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-operators/infra-operator-index-hvvnf"] Dec 03 13:41:05 crc kubenswrapper[5052]: I1203 13:41:05.724062 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-operators/5d473c3169f40b179d14921c90af2c8546b7b757fe551b7dba7d903f5dr8sfn" Dec 03 13:41:05 crc kubenswrapper[5052]: I1203 13:41:05.950114 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-operators/5d473c3169f40b179d14921c90af2c8546b7b757fe551b7dba7d903f5dr8sfn"] Dec 03 13:41:05 crc kubenswrapper[5052]: W1203 13:41:05.954789 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2aaa3b0a_cd10_42b4_8c0d_fc9317d22547.slice/crio-632116d624d473974062234b1feecc9a845ff3c8ffec439ad06a4c6e5e5efef7 WatchSource:0}: Error finding container 632116d624d473974062234b1feecc9a845ff3c8ffec439ad06a4c6e5e5efef7: Status 404 returned error can't find the container with id 632116d624d473974062234b1feecc9a845ff3c8ffec439ad06a4c6e5e5efef7 Dec 03 13:41:06 crc kubenswrapper[5052]: I1203 13:41:06.671594 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/5d473c3169f40b179d14921c90af2c8546b7b757fe551b7dba7d903f5dr8sfn" event={"ID":"2aaa3b0a-cd10-42b4-8c0d-fc9317d22547","Type":"ContainerStarted","Data":"2b5c7cb691c57e605cca4635b3f50a70ac794369cf573145396a541e9dbf3452"} Dec 03 13:41:06 crc kubenswrapper[5052]: I1203 13:41:06.671947 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/5d473c3169f40b179d14921c90af2c8546b7b757fe551b7dba7d903f5dr8sfn" event={"ID":"2aaa3b0a-cd10-42b4-8c0d-fc9317d22547","Type":"ContainerStarted","Data":"632116d624d473974062234b1feecc9a845ff3c8ffec439ad06a4c6e5e5efef7"} Dec 03 13:41:06 crc kubenswrapper[5052]: I1203 13:41:06.914224 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["horizon-kuttl-tests/openstack-galera-0"] Dec 03 13:41:06 crc kubenswrapper[5052]: I1203 13:41:06.915708 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="horizon-kuttl-tests/openstack-galera-0" Dec 03 13:41:06 crc kubenswrapper[5052]: I1203 13:41:06.924318 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["horizon-kuttl-tests/openstack-galera-0"] Dec 03 13:41:06 crc kubenswrapper[5052]: I1203 13:41:06.962992 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"horizon-kuttl-tests"/"openstack-config-data" Dec 03 13:41:06 crc kubenswrapper[5052]: I1203 13:41:06.963665 5052 reflector.go:368] Caches populated for *v1.Secret from object-"horizon-kuttl-tests"/"galera-openstack-dockercfg-lpm7j" Dec 03 13:41:06 crc kubenswrapper[5052]: I1203 13:41:06.964125 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"horizon-kuttl-tests"/"openshift-service-ca.crt" Dec 03 13:41:06 crc kubenswrapper[5052]: I1203 13:41:06.964947 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"horizon-kuttl-tests"/"kube-root-ca.crt" Dec 03 13:41:06 crc kubenswrapper[5052]: I1203 13:41:06.965385 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"horizon-kuttl-tests"/"openstack-scripts" Dec 03 13:41:06 crc kubenswrapper[5052]: I1203 13:41:06.983028 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["horizon-kuttl-tests/openstack-galera-2"] Dec 03 13:41:06 crc kubenswrapper[5052]: I1203 13:41:06.985566 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="horizon-kuttl-tests/openstack-galera-2" Dec 03 13:41:06 crc kubenswrapper[5052]: I1203 13:41:06.991208 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["horizon-kuttl-tests/openstack-galera-2"] Dec 03 13:41:06 crc kubenswrapper[5052]: I1203 13:41:06.998214 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["horizon-kuttl-tests/openstack-galera-1"] Dec 03 13:41:06 crc kubenswrapper[5052]: I1203 13:41:06.999309 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="horizon-kuttl-tests/openstack-galera-1" Dec 03 13:41:07 crc kubenswrapper[5052]: I1203 13:41:07.019873 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["horizon-kuttl-tests/openstack-galera-1"] Dec 03 13:41:07 crc kubenswrapper[5052]: I1203 13:41:07.063693 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/a2db0de0-8930-4d21-ad37-58889377d33b-config-data-default\") pod \"openstack-galera-2\" (UID: \"a2db0de0-8930-4d21-ad37-58889377d33b\") " pod="horizon-kuttl-tests/openstack-galera-2" Dec 03 13:41:07 crc kubenswrapper[5052]: I1203 13:41:07.063761 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/5096dd99-719d-412e-804b-fc1452ee8b95-operator-scripts\") pod \"openstack-galera-0\" (UID: \"5096dd99-719d-412e-804b-fc1452ee8b95\") " pod="horizon-kuttl-tests/openstack-galera-0" Dec 03 13:41:07 crc kubenswrapper[5052]: I1203 13:41:07.063792 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/2e3a3ea5-4ff6-4c9f-a093-38754c10012a-config-data-default\") pod \"openstack-galera-1\" (UID: \"2e3a3ea5-4ff6-4c9f-a093-38754c10012a\") " pod="horizon-kuttl-tests/openstack-galera-1" Dec 03 13:41:07 crc kubenswrapper[5052]: I1203 13:41:07.063845 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/5096dd99-719d-412e-804b-fc1452ee8b95-kolla-config\") pod \"openstack-galera-0\" (UID: \"5096dd99-719d-412e-804b-fc1452ee8b95\") " pod="horizon-kuttl-tests/openstack-galera-0" Dec 03 13:41:07 crc kubenswrapper[5052]: I1203 13:41:07.063933 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bcjgn\" (UniqueName: \"kubernetes.io/projected/a2db0de0-8930-4d21-ad37-58889377d33b-kube-api-access-bcjgn\") pod \"openstack-galera-2\" (UID: \"a2db0de0-8930-4d21-ad37-58889377d33b\") " pod="horizon-kuttl-tests/openstack-galera-2" Dec 03 13:41:07 crc kubenswrapper[5052]: I1203 13:41:07.063972 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/a2db0de0-8930-4d21-ad37-58889377d33b-operator-scripts\") pod \"openstack-galera-2\" (UID: \"a2db0de0-8930-4d21-ad37-58889377d33b\") " pod="horizon-kuttl-tests/openstack-galera-2" Dec 03 13:41:07 crc kubenswrapper[5052]: I1203 13:41:07.064006 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage06-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage06-crc\") pod \"openstack-galera-2\" (UID: \"a2db0de0-8930-4d21-ad37-58889377d33b\") " pod="horizon-kuttl-tests/openstack-galera-2" Dec 03 13:41:07 crc kubenswrapper[5052]: I1203 13:41:07.064040 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qwt8c\" (UniqueName: \"kubernetes.io/projected/2e3a3ea5-4ff6-4c9f-a093-38754c10012a-kube-api-access-qwt8c\") pod \"openstack-galera-1\" (UID: \"2e3a3ea5-4ff6-4c9f-a093-38754c10012a\") " pod="horizon-kuttl-tests/openstack-galera-1" Dec 03 13:41:07 crc kubenswrapper[5052]: I1203 13:41:07.064059 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/2e3a3ea5-4ff6-4c9f-a093-38754c10012a-operator-scripts\") pod \"openstack-galera-1\" (UID: \"2e3a3ea5-4ff6-4c9f-a093-38754c10012a\") " pod="horizon-kuttl-tests/openstack-galera-1" Dec 03 13:41:07 crc kubenswrapper[5052]: I1203 13:41:07.064079 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage02-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage02-crc\") pod \"openstack-galera-0\" (UID: \"5096dd99-719d-412e-804b-fc1452ee8b95\") " pod="horizon-kuttl-tests/openstack-galera-0" Dec 03 13:41:07 crc kubenswrapper[5052]: I1203 13:41:07.064103 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/5096dd99-719d-412e-804b-fc1452ee8b95-config-data-generated\") pod \"openstack-galera-0\" (UID: \"5096dd99-719d-412e-804b-fc1452ee8b95\") " pod="horizon-kuttl-tests/openstack-galera-0" Dec 03 13:41:07 crc kubenswrapper[5052]: I1203 13:41:07.064149 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/2e3a3ea5-4ff6-4c9f-a093-38754c10012a-kolla-config\") pod \"openstack-galera-1\" (UID: \"2e3a3ea5-4ff6-4c9f-a093-38754c10012a\") " pod="horizon-kuttl-tests/openstack-galera-1" Dec 03 13:41:07 crc kubenswrapper[5052]: I1203 13:41:07.064173 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/2e3a3ea5-4ff6-4c9f-a093-38754c10012a-config-data-generated\") pod \"openstack-galera-1\" (UID: \"2e3a3ea5-4ff6-4c9f-a093-38754c10012a\") " pod="horizon-kuttl-tests/openstack-galera-1" Dec 03 13:41:07 crc kubenswrapper[5052]: I1203 13:41:07.064191 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/5096dd99-719d-412e-804b-fc1452ee8b95-config-data-default\") pod \"openstack-galera-0\" (UID: \"5096dd99-719d-412e-804b-fc1452ee8b95\") " pod="horizon-kuttl-tests/openstack-galera-0" Dec 03 13:41:07 crc kubenswrapper[5052]: I1203 13:41:07.064211 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-98wmd\" (UniqueName: \"kubernetes.io/projected/5096dd99-719d-412e-804b-fc1452ee8b95-kube-api-access-98wmd\") pod \"openstack-galera-0\" (UID: \"5096dd99-719d-412e-804b-fc1452ee8b95\") " pod="horizon-kuttl-tests/openstack-galera-0" Dec 03 13:41:07 crc kubenswrapper[5052]: I1203 13:41:07.064229 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/a2db0de0-8930-4d21-ad37-58889377d33b-kolla-config\") pod \"openstack-galera-2\" (UID: \"a2db0de0-8930-4d21-ad37-58889377d33b\") " pod="horizon-kuttl-tests/openstack-galera-2" Dec 03 13:41:07 crc kubenswrapper[5052]: I1203 13:41:07.064274 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/a2db0de0-8930-4d21-ad37-58889377d33b-config-data-generated\") pod \"openstack-galera-2\" (UID: \"a2db0de0-8930-4d21-ad37-58889377d33b\") " pod="horizon-kuttl-tests/openstack-galera-2" Dec 03 13:41:07 crc kubenswrapper[5052]: I1203 13:41:07.064294 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-storage18-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage18-crc\") pod \"openstack-galera-1\" (UID: \"2e3a3ea5-4ff6-4c9f-a093-38754c10012a\") " pod="horizon-kuttl-tests/openstack-galera-1" Dec 03 13:41:07 crc kubenswrapper[5052]: I1203 13:41:07.165377 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-bcjgn\" (UniqueName: \"kubernetes.io/projected/a2db0de0-8930-4d21-ad37-58889377d33b-kube-api-access-bcjgn\") pod \"openstack-galera-2\" (UID: \"a2db0de0-8930-4d21-ad37-58889377d33b\") " pod="horizon-kuttl-tests/openstack-galera-2" Dec 03 13:41:07 crc kubenswrapper[5052]: I1203 13:41:07.165427 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/a2db0de0-8930-4d21-ad37-58889377d33b-operator-scripts\") pod \"openstack-galera-2\" (UID: \"a2db0de0-8930-4d21-ad37-58889377d33b\") " pod="horizon-kuttl-tests/openstack-galera-2" Dec 03 13:41:07 crc kubenswrapper[5052]: I1203 13:41:07.165455 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage06-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage06-crc\") pod \"openstack-galera-2\" (UID: \"a2db0de0-8930-4d21-ad37-58889377d33b\") " pod="horizon-kuttl-tests/openstack-galera-2" Dec 03 13:41:07 crc kubenswrapper[5052]: I1203 13:41:07.165473 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-qwt8c\" (UniqueName: \"kubernetes.io/projected/2e3a3ea5-4ff6-4c9f-a093-38754c10012a-kube-api-access-qwt8c\") pod \"openstack-galera-1\" (UID: \"2e3a3ea5-4ff6-4c9f-a093-38754c10012a\") " pod="horizon-kuttl-tests/openstack-galera-1" Dec 03 13:41:07 crc kubenswrapper[5052]: I1203 13:41:07.165492 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/2e3a3ea5-4ff6-4c9f-a093-38754c10012a-operator-scripts\") pod \"openstack-galera-1\" (UID: \"2e3a3ea5-4ff6-4c9f-a093-38754c10012a\") " pod="horizon-kuttl-tests/openstack-galera-1" Dec 03 13:41:07 crc kubenswrapper[5052]: I1203 13:41:07.165521 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage02-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage02-crc\") pod \"openstack-galera-0\" (UID: \"5096dd99-719d-412e-804b-fc1452ee8b95\") " pod="horizon-kuttl-tests/openstack-galera-0" Dec 03 13:41:07 crc kubenswrapper[5052]: I1203 13:41:07.165540 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/5096dd99-719d-412e-804b-fc1452ee8b95-config-data-generated\") pod \"openstack-galera-0\" (UID: \"5096dd99-719d-412e-804b-fc1452ee8b95\") " pod="horizon-kuttl-tests/openstack-galera-0" Dec 03 13:41:07 crc kubenswrapper[5052]: I1203 13:41:07.165563 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/2e3a3ea5-4ff6-4c9f-a093-38754c10012a-kolla-config\") pod \"openstack-galera-1\" (UID: \"2e3a3ea5-4ff6-4c9f-a093-38754c10012a\") " pod="horizon-kuttl-tests/openstack-galera-1" Dec 03 13:41:07 crc kubenswrapper[5052]: I1203 13:41:07.165577 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/2e3a3ea5-4ff6-4c9f-a093-38754c10012a-config-data-generated\") pod \"openstack-galera-1\" (UID: \"2e3a3ea5-4ff6-4c9f-a093-38754c10012a\") " pod="horizon-kuttl-tests/openstack-galera-1" Dec 03 13:41:07 crc kubenswrapper[5052]: I1203 13:41:07.165595 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/5096dd99-719d-412e-804b-fc1452ee8b95-config-data-default\") pod \"openstack-galera-0\" (UID: \"5096dd99-719d-412e-804b-fc1452ee8b95\") " pod="horizon-kuttl-tests/openstack-galera-0" Dec 03 13:41:07 crc kubenswrapper[5052]: I1203 13:41:07.165613 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-98wmd\" (UniqueName: \"kubernetes.io/projected/5096dd99-719d-412e-804b-fc1452ee8b95-kube-api-access-98wmd\") pod \"openstack-galera-0\" (UID: \"5096dd99-719d-412e-804b-fc1452ee8b95\") " pod="horizon-kuttl-tests/openstack-galera-0" Dec 03 13:41:07 crc kubenswrapper[5052]: I1203 13:41:07.165634 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/a2db0de0-8930-4d21-ad37-58889377d33b-kolla-config\") pod \"openstack-galera-2\" (UID: \"a2db0de0-8930-4d21-ad37-58889377d33b\") " pod="horizon-kuttl-tests/openstack-galera-2" Dec 03 13:41:07 crc kubenswrapper[5052]: I1203 13:41:07.165658 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/a2db0de0-8930-4d21-ad37-58889377d33b-config-data-generated\") pod \"openstack-galera-2\" (UID: \"a2db0de0-8930-4d21-ad37-58889377d33b\") " pod="horizon-kuttl-tests/openstack-galera-2" Dec 03 13:41:07 crc kubenswrapper[5052]: I1203 13:41:07.165685 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"local-storage18-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage18-crc\") pod \"openstack-galera-1\" (UID: \"2e3a3ea5-4ff6-4c9f-a093-38754c10012a\") " pod="horizon-kuttl-tests/openstack-galera-1" Dec 03 13:41:07 crc kubenswrapper[5052]: I1203 13:41:07.165719 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/a2db0de0-8930-4d21-ad37-58889377d33b-config-data-default\") pod \"openstack-galera-2\" (UID: \"a2db0de0-8930-4d21-ad37-58889377d33b\") " pod="horizon-kuttl-tests/openstack-galera-2" Dec 03 13:41:07 crc kubenswrapper[5052]: I1203 13:41:07.165751 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/5096dd99-719d-412e-804b-fc1452ee8b95-operator-scripts\") pod \"openstack-galera-0\" (UID: \"5096dd99-719d-412e-804b-fc1452ee8b95\") " pod="horizon-kuttl-tests/openstack-galera-0" Dec 03 13:41:07 crc kubenswrapper[5052]: I1203 13:41:07.165768 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/2e3a3ea5-4ff6-4c9f-a093-38754c10012a-config-data-default\") pod \"openstack-galera-1\" (UID: \"2e3a3ea5-4ff6-4c9f-a093-38754c10012a\") " pod="horizon-kuttl-tests/openstack-galera-1" Dec 03 13:41:07 crc kubenswrapper[5052]: I1203 13:41:07.165789 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/5096dd99-719d-412e-804b-fc1452ee8b95-kolla-config\") pod \"openstack-galera-0\" (UID: \"5096dd99-719d-412e-804b-fc1452ee8b95\") " pod="horizon-kuttl-tests/openstack-galera-0" Dec 03 13:41:07 crc kubenswrapper[5052]: I1203 13:41:07.166115 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/a2db0de0-8930-4d21-ad37-58889377d33b-config-data-generated\") pod \"openstack-galera-2\" (UID: \"a2db0de0-8930-4d21-ad37-58889377d33b\") " pod="horizon-kuttl-tests/openstack-galera-2" Dec 03 13:41:07 crc kubenswrapper[5052]: I1203 13:41:07.166115 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/2e3a3ea5-4ff6-4c9f-a093-38754c10012a-config-data-generated\") pod \"openstack-galera-1\" (UID: \"2e3a3ea5-4ff6-4c9f-a093-38754c10012a\") " pod="horizon-kuttl-tests/openstack-galera-1" Dec 03 13:41:07 crc kubenswrapper[5052]: I1203 13:41:07.166176 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage02-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage02-crc\") pod \"openstack-galera-0\" (UID: \"5096dd99-719d-412e-804b-fc1452ee8b95\") device mount path \"/mnt/openstack/pv02\"" pod="horizon-kuttl-tests/openstack-galera-0" Dec 03 13:41:07 crc kubenswrapper[5052]: I1203 13:41:07.166267 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage18-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage18-crc\") pod \"openstack-galera-1\" (UID: \"2e3a3ea5-4ff6-4c9f-a093-38754c10012a\") device mount path \"/mnt/openstack/pv18\"" pod="horizon-kuttl-tests/openstack-galera-1" Dec 03 13:41:07 crc kubenswrapper[5052]: I1203 13:41:07.166563 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/5096dd99-719d-412e-804b-fc1452ee8b95-config-data-generated\") pod \"openstack-galera-0\" (UID: \"5096dd99-719d-412e-804b-fc1452ee8b95\") " pod="horizon-kuttl-tests/openstack-galera-0" Dec 03 13:41:07 crc kubenswrapper[5052]: I1203 13:41:07.166685 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/5096dd99-719d-412e-804b-fc1452ee8b95-kolla-config\") pod \"openstack-galera-0\" (UID: \"5096dd99-719d-412e-804b-fc1452ee8b95\") " pod="horizon-kuttl-tests/openstack-galera-0" Dec 03 13:41:07 crc kubenswrapper[5052]: I1203 13:41:07.166695 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/a2db0de0-8930-4d21-ad37-58889377d33b-kolla-config\") pod \"openstack-galera-2\" (UID: \"a2db0de0-8930-4d21-ad37-58889377d33b\") " pod="horizon-kuttl-tests/openstack-galera-2" Dec 03 13:41:07 crc kubenswrapper[5052]: I1203 13:41:07.166770 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"local-storage06-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage06-crc\") pod \"openstack-galera-2\" (UID: \"a2db0de0-8930-4d21-ad37-58889377d33b\") device mount path \"/mnt/openstack/pv06\"" pod="horizon-kuttl-tests/openstack-galera-2" Dec 03 13:41:07 crc kubenswrapper[5052]: I1203 13:41:07.166972 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/a2db0de0-8930-4d21-ad37-58889377d33b-config-data-default\") pod \"openstack-galera-2\" (UID: \"a2db0de0-8930-4d21-ad37-58889377d33b\") " pod="horizon-kuttl-tests/openstack-galera-2" Dec 03 13:41:07 crc kubenswrapper[5052]: I1203 13:41:07.167048 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/5096dd99-719d-412e-804b-fc1452ee8b95-config-data-default\") pod \"openstack-galera-0\" (UID: \"5096dd99-719d-412e-804b-fc1452ee8b95\") " pod="horizon-kuttl-tests/openstack-galera-0" Dec 03 13:41:07 crc kubenswrapper[5052]: I1203 13:41:07.167153 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/2e3a3ea5-4ff6-4c9f-a093-38754c10012a-config-data-default\") pod \"openstack-galera-1\" (UID: \"2e3a3ea5-4ff6-4c9f-a093-38754c10012a\") " pod="horizon-kuttl-tests/openstack-galera-1" Dec 03 13:41:07 crc kubenswrapper[5052]: I1203 13:41:07.167186 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/2e3a3ea5-4ff6-4c9f-a093-38754c10012a-kolla-config\") pod \"openstack-galera-1\" (UID: \"2e3a3ea5-4ff6-4c9f-a093-38754c10012a\") " pod="horizon-kuttl-tests/openstack-galera-1" Dec 03 13:41:07 crc kubenswrapper[5052]: I1203 13:41:07.167903 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/2e3a3ea5-4ff6-4c9f-a093-38754c10012a-operator-scripts\") pod \"openstack-galera-1\" (UID: \"2e3a3ea5-4ff6-4c9f-a093-38754c10012a\") " pod="horizon-kuttl-tests/openstack-galera-1" Dec 03 13:41:07 crc kubenswrapper[5052]: I1203 13:41:07.168367 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/5096dd99-719d-412e-804b-fc1452ee8b95-operator-scripts\") pod \"openstack-galera-0\" (UID: \"5096dd99-719d-412e-804b-fc1452ee8b95\") " pod="horizon-kuttl-tests/openstack-galera-0" Dec 03 13:41:07 crc kubenswrapper[5052]: I1203 13:41:07.170088 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/a2db0de0-8930-4d21-ad37-58889377d33b-operator-scripts\") pod \"openstack-galera-2\" (UID: \"a2db0de0-8930-4d21-ad37-58889377d33b\") " pod="horizon-kuttl-tests/openstack-galera-2" Dec 03 13:41:07 crc kubenswrapper[5052]: I1203 13:41:07.185848 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-98wmd\" (UniqueName: \"kubernetes.io/projected/5096dd99-719d-412e-804b-fc1452ee8b95-kube-api-access-98wmd\") pod \"openstack-galera-0\" (UID: \"5096dd99-719d-412e-804b-fc1452ee8b95\") " pod="horizon-kuttl-tests/openstack-galera-0" Dec 03 13:41:07 crc kubenswrapper[5052]: I1203 13:41:07.186930 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage06-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage06-crc\") pod \"openstack-galera-2\" (UID: \"a2db0de0-8930-4d21-ad37-58889377d33b\") " pod="horizon-kuttl-tests/openstack-galera-2" Dec 03 13:41:07 crc kubenswrapper[5052]: I1203 13:41:07.188456 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-bcjgn\" (UniqueName: \"kubernetes.io/projected/a2db0de0-8930-4d21-ad37-58889377d33b-kube-api-access-bcjgn\") pod \"openstack-galera-2\" (UID: \"a2db0de0-8930-4d21-ad37-58889377d33b\") " pod="horizon-kuttl-tests/openstack-galera-2" Dec 03 13:41:07 crc kubenswrapper[5052]: I1203 13:41:07.188765 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage18-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage18-crc\") pod \"openstack-galera-1\" (UID: \"2e3a3ea5-4ff6-4c9f-a093-38754c10012a\") " pod="horizon-kuttl-tests/openstack-galera-1" Dec 03 13:41:07 crc kubenswrapper[5052]: I1203 13:41:07.193853 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-qwt8c\" (UniqueName: \"kubernetes.io/projected/2e3a3ea5-4ff6-4c9f-a093-38754c10012a-kube-api-access-qwt8c\") pod \"openstack-galera-1\" (UID: \"2e3a3ea5-4ff6-4c9f-a093-38754c10012a\") " pod="horizon-kuttl-tests/openstack-galera-1" Dec 03 13:41:07 crc kubenswrapper[5052]: I1203 13:41:07.194777 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"local-storage02-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage02-crc\") pod \"openstack-galera-0\" (UID: \"5096dd99-719d-412e-804b-fc1452ee8b95\") " pod="horizon-kuttl-tests/openstack-galera-0" Dec 03 13:41:07 crc kubenswrapper[5052]: I1203 13:41:07.285611 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="horizon-kuttl-tests/openstack-galera-0" Dec 03 13:41:07 crc kubenswrapper[5052]: I1203 13:41:07.306320 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="horizon-kuttl-tests/openstack-galera-2" Dec 03 13:41:07 crc kubenswrapper[5052]: I1203 13:41:07.327502 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="horizon-kuttl-tests/openstack-galera-1" Dec 03 13:41:07 crc kubenswrapper[5052]: I1203 13:41:07.637114 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["horizon-kuttl-tests/openstack-galera-1"] Dec 03 13:41:07 crc kubenswrapper[5052]: I1203 13:41:07.687249 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fb9f28a9-2859-4b61-9eb5-2d4712fab081" path="/var/lib/kubelet/pods/fb9f28a9-2859-4b61-9eb5-2d4712fab081/volumes" Dec 03 13:41:07 crc kubenswrapper[5052]: I1203 13:41:07.687880 5052 generic.go:334] "Generic (PLEG): container finished" podID="2aaa3b0a-cd10-42b4-8c0d-fc9317d22547" containerID="2b5c7cb691c57e605cca4635b3f50a70ac794369cf573145396a541e9dbf3452" exitCode=0 Dec 03 13:41:07 crc kubenswrapper[5052]: I1203 13:41:07.687965 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/5d473c3169f40b179d14921c90af2c8546b7b757fe551b7dba7d903f5dr8sfn" event={"ID":"2aaa3b0a-cd10-42b4-8c0d-fc9317d22547","Type":"ContainerDied","Data":"2b5c7cb691c57e605cca4635b3f50a70ac794369cf573145396a541e9dbf3452"} Dec 03 13:41:07 crc kubenswrapper[5052]: I1203 13:41:07.687990 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["horizon-kuttl-tests/openstack-galera-0"] Dec 03 13:41:07 crc kubenswrapper[5052]: W1203 13:41:07.689948 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5096dd99_719d_412e_804b_fc1452ee8b95.slice/crio-a44911a627c7c251eb81c830124d8c2d245e843c16fad9c055b0d77736b3f2b5 WatchSource:0}: Error finding container a44911a627c7c251eb81c830124d8c2d245e843c16fad9c055b0d77736b3f2b5: Status 404 returned error can't find the container with id a44911a627c7c251eb81c830124d8c2d245e843c16fad9c055b0d77736b3f2b5 Dec 03 13:41:07 crc kubenswrapper[5052]: I1203 13:41:07.691657 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="horizon-kuttl-tests/openstack-galera-1" event={"ID":"2e3a3ea5-4ff6-4c9f-a093-38754c10012a","Type":"ContainerStarted","Data":"a89f203ce2e2eb97a77dbff1eeb25d1e0344ea80e27e5ab0ca177959b28fe31c"} Dec 03 13:41:07 crc kubenswrapper[5052]: I1203 13:41:07.788673 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["horizon-kuttl-tests/openstack-galera-2"] Dec 03 13:41:07 crc kubenswrapper[5052]: W1203 13:41:07.793714 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda2db0de0_8930_4d21_ad37_58889377d33b.slice/crio-7ad7f58856967988188b660fbc86a7f695b89a66430918f2fe18d1f92c2282c0 WatchSource:0}: Error finding container 7ad7f58856967988188b660fbc86a7f695b89a66430918f2fe18d1f92c2282c0: Status 404 returned error can't find the container with id 7ad7f58856967988188b660fbc86a7f695b89a66430918f2fe18d1f92c2282c0 Dec 03 13:41:08 crc kubenswrapper[5052]: I1203 13:41:08.698962 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="horizon-kuttl-tests/openstack-galera-2" event={"ID":"a2db0de0-8930-4d21-ad37-58889377d33b","Type":"ContainerStarted","Data":"e632e67c1bed917f6c8760138200ea096bc9a6b7c545fd437d184fea50809f96"} Dec 03 13:41:08 crc kubenswrapper[5052]: I1203 13:41:08.700362 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="horizon-kuttl-tests/openstack-galera-2" event={"ID":"a2db0de0-8930-4d21-ad37-58889377d33b","Type":"ContainerStarted","Data":"7ad7f58856967988188b660fbc86a7f695b89a66430918f2fe18d1f92c2282c0"} Dec 03 13:41:08 crc kubenswrapper[5052]: I1203 13:41:08.700481 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="horizon-kuttl-tests/openstack-galera-1" event={"ID":"2e3a3ea5-4ff6-4c9f-a093-38754c10012a","Type":"ContainerStarted","Data":"4a8f84ee4a2986347ca455ced4186fc482fec4ebfa9e3c68729d3ad41525166b"} Dec 03 13:41:08 crc kubenswrapper[5052]: I1203 13:41:08.702330 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="horizon-kuttl-tests/openstack-galera-0" event={"ID":"5096dd99-719d-412e-804b-fc1452ee8b95","Type":"ContainerStarted","Data":"09dbf2188eb534590a55f1a626c0cdf554c64b93ef01ea9824fcf16cd298c2c7"} Dec 03 13:41:08 crc kubenswrapper[5052]: I1203 13:41:08.702384 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="horizon-kuttl-tests/openstack-galera-0" event={"ID":"5096dd99-719d-412e-804b-fc1452ee8b95","Type":"ContainerStarted","Data":"a44911a627c7c251eb81c830124d8c2d245e843c16fad9c055b0d77736b3f2b5"} Dec 03 13:41:08 crc kubenswrapper[5052]: I1203 13:41:08.703819 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/infra-operator-index-5rfr5" event={"ID":"a8c2a4d6-1e91-40de-9edc-60b79b888d1f","Type":"ContainerStarted","Data":"bbe020afe41be5770ac45a3d0ff5e4a49b3a6871dca3b0cc331acc3c7df428b9"} Dec 03 13:41:08 crc kubenswrapper[5052]: I1203 13:41:08.735179 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-operators/infra-operator-index-5rfr5" podStartSLOduration=2.976637825 podStartE2EDuration="4.735162003s" podCreationTimestamp="2025-12-03 13:41:04 +0000 UTC" firstStartedPulling="2025-12-03 13:41:05.602606682 +0000 UTC m=+8456.190848446" lastFinishedPulling="2025-12-03 13:41:07.36113086 +0000 UTC m=+8457.949372624" observedRunningTime="2025-12-03 13:41:08.733341561 +0000 UTC m=+8459.321583325" watchObservedRunningTime="2025-12-03 13:41:08.735162003 +0000 UTC m=+8459.323403767" Dec 03 13:41:09 crc kubenswrapper[5052]: I1203 13:41:09.711967 5052 generic.go:334] "Generic (PLEG): container finished" podID="2aaa3b0a-cd10-42b4-8c0d-fc9317d22547" containerID="d16a0d5eefe4a10e9077ee9312a1eeaa8cbc2a0b41d3f9cece70e26a2bb2f117" exitCode=0 Dec 03 13:41:09 crc kubenswrapper[5052]: I1203 13:41:09.713320 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/5d473c3169f40b179d14921c90af2c8546b7b757fe551b7dba7d903f5dr8sfn" event={"ID":"2aaa3b0a-cd10-42b4-8c0d-fc9317d22547","Type":"ContainerDied","Data":"d16a0d5eefe4a10e9077ee9312a1eeaa8cbc2a0b41d3f9cece70e26a2bb2f117"} Dec 03 13:41:10 crc kubenswrapper[5052]: I1203 13:41:10.721285 5052 generic.go:334] "Generic (PLEG): container finished" podID="2aaa3b0a-cd10-42b4-8c0d-fc9317d22547" containerID="a0d9a069ce3a3dc919e542b0eff35e9e7d9b2ddf5b1ace2db265d276d2873513" exitCode=0 Dec 03 13:41:10 crc kubenswrapper[5052]: I1203 13:41:10.721356 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/5d473c3169f40b179d14921c90af2c8546b7b757fe551b7dba7d903f5dr8sfn" event={"ID":"2aaa3b0a-cd10-42b4-8c0d-fc9317d22547","Type":"ContainerDied","Data":"a0d9a069ce3a3dc919e542b0eff35e9e7d9b2ddf5b1ace2db265d276d2873513"} Dec 03 13:41:12 crc kubenswrapper[5052]: I1203 13:41:12.002783 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-operators/5d473c3169f40b179d14921c90af2c8546b7b757fe551b7dba7d903f5dr8sfn" Dec 03 13:41:12 crc kubenswrapper[5052]: I1203 13:41:12.064801 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mzbl4\" (UniqueName: \"kubernetes.io/projected/2aaa3b0a-cd10-42b4-8c0d-fc9317d22547-kube-api-access-mzbl4\") pod \"2aaa3b0a-cd10-42b4-8c0d-fc9317d22547\" (UID: \"2aaa3b0a-cd10-42b4-8c0d-fc9317d22547\") " Dec 03 13:41:12 crc kubenswrapper[5052]: I1203 13:41:12.064912 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2aaa3b0a-cd10-42b4-8c0d-fc9317d22547-bundle\") pod \"2aaa3b0a-cd10-42b4-8c0d-fc9317d22547\" (UID: \"2aaa3b0a-cd10-42b4-8c0d-fc9317d22547\") " Dec 03 13:41:12 crc kubenswrapper[5052]: I1203 13:41:12.064941 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2aaa3b0a-cd10-42b4-8c0d-fc9317d22547-util\") pod \"2aaa3b0a-cd10-42b4-8c0d-fc9317d22547\" (UID: \"2aaa3b0a-cd10-42b4-8c0d-fc9317d22547\") " Dec 03 13:41:12 crc kubenswrapper[5052]: I1203 13:41:12.066199 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2aaa3b0a-cd10-42b4-8c0d-fc9317d22547-bundle" (OuterVolumeSpecName: "bundle") pod "2aaa3b0a-cd10-42b4-8c0d-fc9317d22547" (UID: "2aaa3b0a-cd10-42b4-8c0d-fc9317d22547"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 13:41:12 crc kubenswrapper[5052]: I1203 13:41:12.070941 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2aaa3b0a-cd10-42b4-8c0d-fc9317d22547-kube-api-access-mzbl4" (OuterVolumeSpecName: "kube-api-access-mzbl4") pod "2aaa3b0a-cd10-42b4-8c0d-fc9317d22547" (UID: "2aaa3b0a-cd10-42b4-8c0d-fc9317d22547"). InnerVolumeSpecName "kube-api-access-mzbl4". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:41:12 crc kubenswrapper[5052]: I1203 13:41:12.078672 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2aaa3b0a-cd10-42b4-8c0d-fc9317d22547-util" (OuterVolumeSpecName: "util") pod "2aaa3b0a-cd10-42b4-8c0d-fc9317d22547" (UID: "2aaa3b0a-cd10-42b4-8c0d-fc9317d22547"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 13:41:12 crc kubenswrapper[5052]: I1203 13:41:12.166766 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-mzbl4\" (UniqueName: \"kubernetes.io/projected/2aaa3b0a-cd10-42b4-8c0d-fc9317d22547-kube-api-access-mzbl4\") on node \"crc\" DevicePath \"\"" Dec 03 13:41:12 crc kubenswrapper[5052]: I1203 13:41:12.166814 5052 reconciler_common.go:293] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2aaa3b0a-cd10-42b4-8c0d-fc9317d22547-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 13:41:12 crc kubenswrapper[5052]: I1203 13:41:12.166826 5052 reconciler_common.go:293] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2aaa3b0a-cd10-42b4-8c0d-fc9317d22547-util\") on node \"crc\" DevicePath \"\"" Dec 03 13:41:12 crc kubenswrapper[5052]: I1203 13:41:12.736078 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/5d473c3169f40b179d14921c90af2c8546b7b757fe551b7dba7d903f5dr8sfn" event={"ID":"2aaa3b0a-cd10-42b4-8c0d-fc9317d22547","Type":"ContainerDied","Data":"632116d624d473974062234b1feecc9a845ff3c8ffec439ad06a4c6e5e5efef7"} Dec 03 13:41:12 crc kubenswrapper[5052]: I1203 13:41:12.736407 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="632116d624d473974062234b1feecc9a845ff3c8ffec439ad06a4c6e5e5efef7" Dec 03 13:41:12 crc kubenswrapper[5052]: I1203 13:41:12.736138 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-operators/5d473c3169f40b179d14921c90af2c8546b7b757fe551b7dba7d903f5dr8sfn" Dec 03 13:41:13 crc kubenswrapper[5052]: I1203 13:41:13.743125 5052 generic.go:334] "Generic (PLEG): container finished" podID="2e3a3ea5-4ff6-4c9f-a093-38754c10012a" containerID="4a8f84ee4a2986347ca455ced4186fc482fec4ebfa9e3c68729d3ad41525166b" exitCode=0 Dec 03 13:41:13 crc kubenswrapper[5052]: I1203 13:41:13.743186 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="horizon-kuttl-tests/openstack-galera-1" event={"ID":"2e3a3ea5-4ff6-4c9f-a093-38754c10012a","Type":"ContainerDied","Data":"4a8f84ee4a2986347ca455ced4186fc482fec4ebfa9e3c68729d3ad41525166b"} Dec 03 13:41:13 crc kubenswrapper[5052]: I1203 13:41:13.744734 5052 generic.go:334] "Generic (PLEG): container finished" podID="5096dd99-719d-412e-804b-fc1452ee8b95" containerID="09dbf2188eb534590a55f1a626c0cdf554c64b93ef01ea9824fcf16cd298c2c7" exitCode=0 Dec 03 13:41:13 crc kubenswrapper[5052]: I1203 13:41:13.744834 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="horizon-kuttl-tests/openstack-galera-0" event={"ID":"5096dd99-719d-412e-804b-fc1452ee8b95","Type":"ContainerDied","Data":"09dbf2188eb534590a55f1a626c0cdf554c64b93ef01ea9824fcf16cd298c2c7"} Dec 03 13:41:13 crc kubenswrapper[5052]: I1203 13:41:13.749818 5052 generic.go:334] "Generic (PLEG): container finished" podID="a2db0de0-8930-4d21-ad37-58889377d33b" containerID="e632e67c1bed917f6c8760138200ea096bc9a6b7c545fd437d184fea50809f96" exitCode=0 Dec 03 13:41:13 crc kubenswrapper[5052]: I1203 13:41:13.749876 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="horizon-kuttl-tests/openstack-galera-2" event={"ID":"a2db0de0-8930-4d21-ad37-58889377d33b","Type":"ContainerDied","Data":"e632e67c1bed917f6c8760138200ea096bc9a6b7c545fd437d184fea50809f96"} Dec 03 13:41:14 crc kubenswrapper[5052]: I1203 13:41:14.759002 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="horizon-kuttl-tests/openstack-galera-1" event={"ID":"2e3a3ea5-4ff6-4c9f-a093-38754c10012a","Type":"ContainerStarted","Data":"13cf7bd27ade2aba6b22d0a54ef5527f62fa3398c10e1e0f8714b3adc70d7e83"} Dec 03 13:41:14 crc kubenswrapper[5052]: I1203 13:41:14.761841 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="horizon-kuttl-tests/openstack-galera-0" event={"ID":"5096dd99-719d-412e-804b-fc1452ee8b95","Type":"ContainerStarted","Data":"a302cf322d3d960edabc879fda7af5d1764ecc2cb32a3acf71f26c4b65d418d4"} Dec 03 13:41:14 crc kubenswrapper[5052]: I1203 13:41:14.764110 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="horizon-kuttl-tests/openstack-galera-2" event={"ID":"a2db0de0-8930-4d21-ad37-58889377d33b","Type":"ContainerStarted","Data":"fbc326fc54a520afef74ee80957e5d836345c10d4d03b2e32cd807930864490e"} Dec 03 13:41:14 crc kubenswrapper[5052]: I1203 13:41:14.780619 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="horizon-kuttl-tests/openstack-galera-1" podStartSLOduration=9.780600907 podStartE2EDuration="9.780600907s" podCreationTimestamp="2025-12-03 13:41:05 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 13:41:14.775567202 +0000 UTC m=+8465.363808986" watchObservedRunningTime="2025-12-03 13:41:14.780600907 +0000 UTC m=+8465.368842671" Dec 03 13:41:14 crc kubenswrapper[5052]: I1203 13:41:14.796923 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="horizon-kuttl-tests/openstack-galera-2" podStartSLOduration=9.796899444 podStartE2EDuration="9.796899444s" podCreationTimestamp="2025-12-03 13:41:05 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 13:41:14.793042674 +0000 UTC m=+8465.381284448" watchObservedRunningTime="2025-12-03 13:41:14.796899444 +0000 UTC m=+8465.385141218" Dec 03 13:41:14 crc kubenswrapper[5052]: I1203 13:41:14.823915 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="horizon-kuttl-tests/openstack-galera-0" podStartSLOduration=9.823894768 podStartE2EDuration="9.823894768s" podCreationTimestamp="2025-12-03 13:41:05 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 13:41:14.818433032 +0000 UTC m=+8465.406674796" watchObservedRunningTime="2025-12-03 13:41:14.823894768 +0000 UTC m=+8465.412136532" Dec 03 13:41:15 crc kubenswrapper[5052]: I1203 13:41:15.136329 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-operators/infra-operator-index-5rfr5" Dec 03 13:41:15 crc kubenswrapper[5052]: I1203 13:41:15.136387 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-operators/infra-operator-index-5rfr5" Dec 03 13:41:15 crc kubenswrapper[5052]: I1203 13:41:15.165782 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-operators/infra-operator-index-5rfr5" Dec 03 13:41:15 crc kubenswrapper[5052]: I1203 13:41:15.797293 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-operators/infra-operator-index-5rfr5" Dec 03 13:41:17 crc kubenswrapper[5052]: I1203 13:41:17.285847 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="horizon-kuttl-tests/openstack-galera-0" Dec 03 13:41:17 crc kubenswrapper[5052]: I1203 13:41:17.285909 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="horizon-kuttl-tests/openstack-galera-0" Dec 03 13:41:17 crc kubenswrapper[5052]: I1203 13:41:17.307258 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="horizon-kuttl-tests/openstack-galera-2" Dec 03 13:41:17 crc kubenswrapper[5052]: I1203 13:41:17.307330 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="horizon-kuttl-tests/openstack-galera-2" Dec 03 13:41:17 crc kubenswrapper[5052]: I1203 13:41:17.328199 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="horizon-kuttl-tests/openstack-galera-1" Dec 03 13:41:17 crc kubenswrapper[5052]: I1203 13:41:17.328246 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="horizon-kuttl-tests/openstack-galera-1" Dec 03 13:41:21 crc kubenswrapper[5052]: I1203 13:41:21.502803 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="horizon-kuttl-tests/openstack-galera-2" Dec 03 13:41:21 crc kubenswrapper[5052]: I1203 13:41:21.568294 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="horizon-kuttl-tests/openstack-galera-2" Dec 03 13:41:27 crc kubenswrapper[5052]: I1203 13:41:27.394891 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="horizon-kuttl-tests/openstack-galera-2" podUID="a2db0de0-8930-4d21-ad37-58889377d33b" containerName="galera" probeResult="failure" output=< Dec 03 13:41:27 crc kubenswrapper[5052]: wsrep_local_state_comment (Donor/Desynced) differs from Synced Dec 03 13:41:27 crc kubenswrapper[5052]: > Dec 03 13:41:27 crc kubenswrapper[5052]: I1203 13:41:27.543044 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-operators/infra-operator-controller-manager-598d599d5d-bp6rp"] Dec 03 13:41:27 crc kubenswrapper[5052]: E1203 13:41:27.543358 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="2aaa3b0a-cd10-42b4-8c0d-fc9317d22547" containerName="pull" Dec 03 13:41:27 crc kubenswrapper[5052]: I1203 13:41:27.543380 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="2aaa3b0a-cd10-42b4-8c0d-fc9317d22547" containerName="pull" Dec 03 13:41:27 crc kubenswrapper[5052]: E1203 13:41:27.543397 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="2aaa3b0a-cd10-42b4-8c0d-fc9317d22547" containerName="util" Dec 03 13:41:27 crc kubenswrapper[5052]: I1203 13:41:27.543405 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="2aaa3b0a-cd10-42b4-8c0d-fc9317d22547" containerName="util" Dec 03 13:41:27 crc kubenswrapper[5052]: E1203 13:41:27.543433 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="2aaa3b0a-cd10-42b4-8c0d-fc9317d22547" containerName="extract" Dec 03 13:41:27 crc kubenswrapper[5052]: I1203 13:41:27.543441 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="2aaa3b0a-cd10-42b4-8c0d-fc9317d22547" containerName="extract" Dec 03 13:41:27 crc kubenswrapper[5052]: I1203 13:41:27.543598 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="2aaa3b0a-cd10-42b4-8c0d-fc9317d22547" containerName="extract" Dec 03 13:41:27 crc kubenswrapper[5052]: I1203 13:41:27.544333 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-operators/infra-operator-controller-manager-598d599d5d-bp6rp" Dec 03 13:41:27 crc kubenswrapper[5052]: I1203 13:41:27.546121 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-operators"/"infra-operator-controller-manager-service-cert" Dec 03 13:41:27 crc kubenswrapper[5052]: I1203 13:41:27.547090 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-operators"/"infra-operator-controller-manager-dockercfg-pb52n" Dec 03 13:41:27 crc kubenswrapper[5052]: I1203 13:41:27.552205 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/afe4ff83-55b7-48a0-902c-4162f0f68ac5-apiservice-cert\") pod \"infra-operator-controller-manager-598d599d5d-bp6rp\" (UID: \"afe4ff83-55b7-48a0-902c-4162f0f68ac5\") " pod="openstack-operators/infra-operator-controller-manager-598d599d5d-bp6rp" Dec 03 13:41:27 crc kubenswrapper[5052]: I1203 13:41:27.552337 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jvcdg\" (UniqueName: \"kubernetes.io/projected/afe4ff83-55b7-48a0-902c-4162f0f68ac5-kube-api-access-jvcdg\") pod \"infra-operator-controller-manager-598d599d5d-bp6rp\" (UID: \"afe4ff83-55b7-48a0-902c-4162f0f68ac5\") " pod="openstack-operators/infra-operator-controller-manager-598d599d5d-bp6rp" Dec 03 13:41:27 crc kubenswrapper[5052]: I1203 13:41:27.552370 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/afe4ff83-55b7-48a0-902c-4162f0f68ac5-webhook-cert\") pod \"infra-operator-controller-manager-598d599d5d-bp6rp\" (UID: \"afe4ff83-55b7-48a0-902c-4162f0f68ac5\") " pod="openstack-operators/infra-operator-controller-manager-598d599d5d-bp6rp" Dec 03 13:41:27 crc kubenswrapper[5052]: I1203 13:41:27.572464 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-operators/infra-operator-controller-manager-598d599d5d-bp6rp"] Dec 03 13:41:27 crc kubenswrapper[5052]: I1203 13:41:27.652941 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-jvcdg\" (UniqueName: \"kubernetes.io/projected/afe4ff83-55b7-48a0-902c-4162f0f68ac5-kube-api-access-jvcdg\") pod \"infra-operator-controller-manager-598d599d5d-bp6rp\" (UID: \"afe4ff83-55b7-48a0-902c-4162f0f68ac5\") " pod="openstack-operators/infra-operator-controller-manager-598d599d5d-bp6rp" Dec 03 13:41:27 crc kubenswrapper[5052]: I1203 13:41:27.652999 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/afe4ff83-55b7-48a0-902c-4162f0f68ac5-webhook-cert\") pod \"infra-operator-controller-manager-598d599d5d-bp6rp\" (UID: \"afe4ff83-55b7-48a0-902c-4162f0f68ac5\") " pod="openstack-operators/infra-operator-controller-manager-598d599d5d-bp6rp" Dec 03 13:41:27 crc kubenswrapper[5052]: I1203 13:41:27.653043 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/afe4ff83-55b7-48a0-902c-4162f0f68ac5-apiservice-cert\") pod \"infra-operator-controller-manager-598d599d5d-bp6rp\" (UID: \"afe4ff83-55b7-48a0-902c-4162f0f68ac5\") " pod="openstack-operators/infra-operator-controller-manager-598d599d5d-bp6rp" Dec 03 13:41:27 crc kubenswrapper[5052]: I1203 13:41:27.661058 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/afe4ff83-55b7-48a0-902c-4162f0f68ac5-apiservice-cert\") pod \"infra-operator-controller-manager-598d599d5d-bp6rp\" (UID: \"afe4ff83-55b7-48a0-902c-4162f0f68ac5\") " pod="openstack-operators/infra-operator-controller-manager-598d599d5d-bp6rp" Dec 03 13:41:27 crc kubenswrapper[5052]: I1203 13:41:27.670438 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/afe4ff83-55b7-48a0-902c-4162f0f68ac5-webhook-cert\") pod \"infra-operator-controller-manager-598d599d5d-bp6rp\" (UID: \"afe4ff83-55b7-48a0-902c-4162f0f68ac5\") " pod="openstack-operators/infra-operator-controller-manager-598d599d5d-bp6rp" Dec 03 13:41:27 crc kubenswrapper[5052]: I1203 13:41:27.675930 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-jvcdg\" (UniqueName: \"kubernetes.io/projected/afe4ff83-55b7-48a0-902c-4162f0f68ac5-kube-api-access-jvcdg\") pod \"infra-operator-controller-manager-598d599d5d-bp6rp\" (UID: \"afe4ff83-55b7-48a0-902c-4162f0f68ac5\") " pod="openstack-operators/infra-operator-controller-manager-598d599d5d-bp6rp" Dec 03 13:41:27 crc kubenswrapper[5052]: I1203 13:41:27.869336 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-operators/infra-operator-controller-manager-598d599d5d-bp6rp" Dec 03 13:41:28 crc kubenswrapper[5052]: I1203 13:41:28.608382 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-operators/infra-operator-controller-manager-598d599d5d-bp6rp"] Dec 03 13:41:29 crc kubenswrapper[5052]: I1203 13:41:29.129946 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/infra-operator-controller-manager-598d599d5d-bp6rp" event={"ID":"afe4ff83-55b7-48a0-902c-4162f0f68ac5","Type":"ContainerStarted","Data":"54216bbfe5e666b89a444f7877f5ea60ca671124686b8f915233b4d21e590480"} Dec 03 13:41:29 crc kubenswrapper[5052]: I1203 13:41:29.130280 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/infra-operator-controller-manager-598d599d5d-bp6rp" event={"ID":"afe4ff83-55b7-48a0-902c-4162f0f68ac5","Type":"ContainerStarted","Data":"64f7ed9421435a11b729eee8e4a6e3db7a76f095f7066f7d1d2ba30a8b6da64f"} Dec 03 13:41:29 crc kubenswrapper[5052]: I1203 13:41:29.130298 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/infra-operator-controller-manager-598d599d5d-bp6rp" event={"ID":"afe4ff83-55b7-48a0-902c-4162f0f68ac5","Type":"ContainerStarted","Data":"134000982b2104e950f30f602bb879142ea99e91d212ec4378b7858698bcc651"} Dec 03 13:41:29 crc kubenswrapper[5052]: I1203 13:41:29.130344 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-operators/infra-operator-controller-manager-598d599d5d-bp6rp" Dec 03 13:41:29 crc kubenswrapper[5052]: I1203 13:41:29.151243 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-operators/infra-operator-controller-manager-598d599d5d-bp6rp" podStartSLOduration=2.151227672 podStartE2EDuration="2.151227672s" podCreationTimestamp="2025-12-03 13:41:27 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 13:41:29.150566154 +0000 UTC m=+8479.738807918" watchObservedRunningTime="2025-12-03 13:41:29.151227672 +0000 UTC m=+8479.739469436" Dec 03 13:41:29 crc kubenswrapper[5052]: I1203 13:41:29.379356 5052 patch_prober.go:28] interesting pod/machine-config-daemon-74bj6 container/machine-config-daemon namespace/openshift-machine-config-operator: Liveness probe status=failure output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" start-of-body= Dec 03 13:41:29 crc kubenswrapper[5052]: I1203 13:41:29.379423 5052 prober.go:107] "Probe failed" probeType="Liveness" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" containerName="machine-config-daemon" probeResult="failure" output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" Dec 03 13:41:29 crc kubenswrapper[5052]: I1203 13:41:29.379466 5052 kubelet.go:2542] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" Dec 03 13:41:29 crc kubenswrapper[5052]: I1203 13:41:29.380018 5052 kuberuntime_manager.go:1027] "Message for Container of pod" containerName="machine-config-daemon" containerStatusID={"Type":"cri-o","ID":"2fbd293d6edef0fa059e91c119312e77b262534107c322feb7b59dbea3464526"} pod="openshift-machine-config-operator/machine-config-daemon-74bj6" containerMessage="Container machine-config-daemon failed liveness probe, will be restarted" Dec 03 13:41:29 crc kubenswrapper[5052]: I1203 13:41:29.380076 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" containerName="machine-config-daemon" containerID="cri-o://2fbd293d6edef0fa059e91c119312e77b262534107c322feb7b59dbea3464526" gracePeriod=600 Dec 03 13:41:29 crc kubenswrapper[5052]: E1203 13:41:29.511112 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 13:41:30 crc kubenswrapper[5052]: I1203 13:41:30.138230 5052 generic.go:334] "Generic (PLEG): container finished" podID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" containerID="2fbd293d6edef0fa059e91c119312e77b262534107c322feb7b59dbea3464526" exitCode=0 Dec 03 13:41:30 crc kubenswrapper[5052]: I1203 13:41:30.138274 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" event={"ID":"10c237bc-72e7-4da3-9f2c-b1688395bf6e","Type":"ContainerDied","Data":"2fbd293d6edef0fa059e91c119312e77b262534107c322feb7b59dbea3464526"} Dec 03 13:41:30 crc kubenswrapper[5052]: I1203 13:41:30.138330 5052 scope.go:117] "RemoveContainer" containerID="c52249d2a9f574a7f765a482d66dd438e6d6ff79a446ffe0920be2784f98e6b8" Dec 03 13:41:30 crc kubenswrapper[5052]: I1203 13:41:30.138886 5052 scope.go:117] "RemoveContainer" containerID="2fbd293d6edef0fa059e91c119312e77b262534107c322feb7b59dbea3464526" Dec 03 13:41:30 crc kubenswrapper[5052]: E1203 13:41:30.139242 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 13:41:33 crc kubenswrapper[5052]: I1203 13:41:33.782306 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openshift-marketplace/certified-operators-6jp8g"] Dec 03 13:41:33 crc kubenswrapper[5052]: I1203 13:41:33.783983 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/certified-operators-6jp8g" Dec 03 13:41:33 crc kubenswrapper[5052]: I1203 13:41:33.844701 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/certified-operators-6jp8g"] Dec 03 13:41:33 crc kubenswrapper[5052]: I1203 13:41:33.945076 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tddxg\" (UniqueName: \"kubernetes.io/projected/3cfb23b9-370f-493f-9595-229867b9c138-kube-api-access-tddxg\") pod \"certified-operators-6jp8g\" (UID: \"3cfb23b9-370f-493f-9595-229867b9c138\") " pod="openshift-marketplace/certified-operators-6jp8g" Dec 03 13:41:33 crc kubenswrapper[5052]: I1203 13:41:33.945156 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/3cfb23b9-370f-493f-9595-229867b9c138-utilities\") pod \"certified-operators-6jp8g\" (UID: \"3cfb23b9-370f-493f-9595-229867b9c138\") " pod="openshift-marketplace/certified-operators-6jp8g" Dec 03 13:41:33 crc kubenswrapper[5052]: I1203 13:41:33.946351 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/3cfb23b9-370f-493f-9595-229867b9c138-catalog-content\") pod \"certified-operators-6jp8g\" (UID: \"3cfb23b9-370f-493f-9595-229867b9c138\") " pod="openshift-marketplace/certified-operators-6jp8g" Dec 03 13:41:34 crc kubenswrapper[5052]: I1203 13:41:34.027470 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="horizon-kuttl-tests/openstack-galera-1" Dec 03 13:41:34 crc kubenswrapper[5052]: I1203 13:41:34.047999 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/3cfb23b9-370f-493f-9595-229867b9c138-utilities\") pod \"certified-operators-6jp8g\" (UID: \"3cfb23b9-370f-493f-9595-229867b9c138\") " pod="openshift-marketplace/certified-operators-6jp8g" Dec 03 13:41:34 crc kubenswrapper[5052]: I1203 13:41:34.048066 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/3cfb23b9-370f-493f-9595-229867b9c138-catalog-content\") pod \"certified-operators-6jp8g\" (UID: \"3cfb23b9-370f-493f-9595-229867b9c138\") " pod="openshift-marketplace/certified-operators-6jp8g" Dec 03 13:41:34 crc kubenswrapper[5052]: I1203 13:41:34.048145 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-tddxg\" (UniqueName: \"kubernetes.io/projected/3cfb23b9-370f-493f-9595-229867b9c138-kube-api-access-tddxg\") pod \"certified-operators-6jp8g\" (UID: \"3cfb23b9-370f-493f-9595-229867b9c138\") " pod="openshift-marketplace/certified-operators-6jp8g" Dec 03 13:41:34 crc kubenswrapper[5052]: I1203 13:41:34.048504 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/3cfb23b9-370f-493f-9595-229867b9c138-utilities\") pod \"certified-operators-6jp8g\" (UID: \"3cfb23b9-370f-493f-9595-229867b9c138\") " pod="openshift-marketplace/certified-operators-6jp8g" Dec 03 13:41:34 crc kubenswrapper[5052]: I1203 13:41:34.048538 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/3cfb23b9-370f-493f-9595-229867b9c138-catalog-content\") pod \"certified-operators-6jp8g\" (UID: \"3cfb23b9-370f-493f-9595-229867b9c138\") " pod="openshift-marketplace/certified-operators-6jp8g" Dec 03 13:41:34 crc kubenswrapper[5052]: I1203 13:41:34.073481 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-tddxg\" (UniqueName: \"kubernetes.io/projected/3cfb23b9-370f-493f-9595-229867b9c138-kube-api-access-tddxg\") pod \"certified-operators-6jp8g\" (UID: \"3cfb23b9-370f-493f-9595-229867b9c138\") " pod="openshift-marketplace/certified-operators-6jp8g" Dec 03 13:41:34 crc kubenswrapper[5052]: I1203 13:41:34.106714 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/certified-operators-6jp8g" Dec 03 13:41:34 crc kubenswrapper[5052]: I1203 13:41:34.136665 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="horizon-kuttl-tests/openstack-galera-1" Dec 03 13:41:34 crc kubenswrapper[5052]: I1203 13:41:34.576874 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/certified-operators-6jp8g"] Dec 03 13:41:34 crc kubenswrapper[5052]: W1203 13:41:34.582734 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3cfb23b9_370f_493f_9595_229867b9c138.slice/crio-90893d952daa3286a6669a2a664a77d85d513dbcb2f40511d9a38779b5d6f1da WatchSource:0}: Error finding container 90893d952daa3286a6669a2a664a77d85d513dbcb2f40511d9a38779b5d6f1da: Status 404 returned error can't find the container with id 90893d952daa3286a6669a2a664a77d85d513dbcb2f40511d9a38779b5d6f1da Dec 03 13:41:35 crc kubenswrapper[5052]: I1203 13:41:35.178980 5052 generic.go:334] "Generic (PLEG): container finished" podID="3cfb23b9-370f-493f-9595-229867b9c138" containerID="18b905b4094c643d8df1fc7d464fe687bbd51e63ce09a43f85c4af4c01e34bc1" exitCode=0 Dec 03 13:41:35 crc kubenswrapper[5052]: I1203 13:41:35.179030 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/certified-operators-6jp8g" event={"ID":"3cfb23b9-370f-493f-9595-229867b9c138","Type":"ContainerDied","Data":"18b905b4094c643d8df1fc7d464fe687bbd51e63ce09a43f85c4af4c01e34bc1"} Dec 03 13:41:35 crc kubenswrapper[5052]: I1203 13:41:35.179258 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/certified-operators-6jp8g" event={"ID":"3cfb23b9-370f-493f-9595-229867b9c138","Type":"ContainerStarted","Data":"90893d952daa3286a6669a2a664a77d85d513dbcb2f40511d9a38779b5d6f1da"} Dec 03 13:41:35 crc kubenswrapper[5052]: I1203 13:41:35.855446 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="horizon-kuttl-tests/openstack-galera-0" Dec 03 13:41:35 crc kubenswrapper[5052]: I1203 13:41:35.993043 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="horizon-kuttl-tests/openstack-galera-0" Dec 03 13:41:36 crc kubenswrapper[5052]: I1203 13:41:36.188027 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/certified-operators-6jp8g" event={"ID":"3cfb23b9-370f-493f-9595-229867b9c138","Type":"ContainerStarted","Data":"84ae7999255ab0fcad07b1ee6430977c32d2b0d9c89a45131754c4dfd9935f0f"} Dec 03 13:41:37 crc kubenswrapper[5052]: I1203 13:41:37.196755 5052 generic.go:334] "Generic (PLEG): container finished" podID="3cfb23b9-370f-493f-9595-229867b9c138" containerID="84ae7999255ab0fcad07b1ee6430977c32d2b0d9c89a45131754c4dfd9935f0f" exitCode=0 Dec 03 13:41:37 crc kubenswrapper[5052]: I1203 13:41:37.196817 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/certified-operators-6jp8g" event={"ID":"3cfb23b9-370f-493f-9595-229867b9c138","Type":"ContainerDied","Data":"84ae7999255ab0fcad07b1ee6430977c32d2b0d9c89a45131754c4dfd9935f0f"} Dec 03 13:41:37 crc kubenswrapper[5052]: I1203 13:41:37.873838 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-operators/infra-operator-controller-manager-598d599d5d-bp6rp" Dec 03 13:41:38 crc kubenswrapper[5052]: I1203 13:41:38.205408 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/certified-operators-6jp8g" event={"ID":"3cfb23b9-370f-493f-9595-229867b9c138","Type":"ContainerStarted","Data":"e1bf00783028e79920479fbf87373fcb30a17688bfac601c062863d520ef931a"} Dec 03 13:41:38 crc kubenswrapper[5052]: I1203 13:41:38.221697 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-marketplace/certified-operators-6jp8g" podStartSLOduration=2.7748852 podStartE2EDuration="5.221675036s" podCreationTimestamp="2025-12-03 13:41:33 +0000 UTC" firstStartedPulling="2025-12-03 13:41:35.182289026 +0000 UTC m=+8485.770530790" lastFinishedPulling="2025-12-03 13:41:37.629078872 +0000 UTC m=+8488.217320626" observedRunningTime="2025-12-03 13:41:38.220466501 +0000 UTC m=+8488.808708265" watchObservedRunningTime="2025-12-03 13:41:38.221675036 +0000 UTC m=+8488.809916800" Dec 03 13:41:43 crc kubenswrapper[5052]: I1203 13:41:43.673068 5052 scope.go:117] "RemoveContainer" containerID="2fbd293d6edef0fa059e91c119312e77b262534107c322feb7b59dbea3464526" Dec 03 13:41:43 crc kubenswrapper[5052]: E1203 13:41:43.673868 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 13:41:44 crc kubenswrapper[5052]: I1203 13:41:44.107263 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-marketplace/certified-operators-6jp8g" Dec 03 13:41:44 crc kubenswrapper[5052]: I1203 13:41:44.107355 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openshift-marketplace/certified-operators-6jp8g" Dec 03 13:41:44 crc kubenswrapper[5052]: I1203 13:41:44.149026 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-marketplace/certified-operators-6jp8g" Dec 03 13:41:44 crc kubenswrapper[5052]: I1203 13:41:44.283249 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-marketplace/certified-operators-6jp8g" Dec 03 13:41:46 crc kubenswrapper[5052]: I1203 13:41:46.010232 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-operators/9704761d240e56fb98655ffd81084895b33a73ec711f4dcdef0450e590nfq89"] Dec 03 13:41:46 crc kubenswrapper[5052]: I1203 13:41:46.011780 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-operators/9704761d240e56fb98655ffd81084895b33a73ec711f4dcdef0450e590nfq89" Dec 03 13:41:46 crc kubenswrapper[5052]: I1203 13:41:46.014356 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-operators"/"default-dockercfg-7fklr" Dec 03 13:41:46 crc kubenswrapper[5052]: I1203 13:41:46.018973 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-operators/9704761d240e56fb98655ffd81084895b33a73ec711f4dcdef0450e590nfq89"] Dec 03 13:41:46 crc kubenswrapper[5052]: I1203 13:41:46.124503 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/845f4c5b-f54f-4cf4-8b78-17c73bdccfdc-util\") pod \"9704761d240e56fb98655ffd81084895b33a73ec711f4dcdef0450e590nfq89\" (UID: \"845f4c5b-f54f-4cf4-8b78-17c73bdccfdc\") " pod="openstack-operators/9704761d240e56fb98655ffd81084895b33a73ec711f4dcdef0450e590nfq89" Dec 03 13:41:46 crc kubenswrapper[5052]: I1203 13:41:46.124594 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tr7rg\" (UniqueName: \"kubernetes.io/projected/845f4c5b-f54f-4cf4-8b78-17c73bdccfdc-kube-api-access-tr7rg\") pod \"9704761d240e56fb98655ffd81084895b33a73ec711f4dcdef0450e590nfq89\" (UID: \"845f4c5b-f54f-4cf4-8b78-17c73bdccfdc\") " pod="openstack-operators/9704761d240e56fb98655ffd81084895b33a73ec711f4dcdef0450e590nfq89" Dec 03 13:41:46 crc kubenswrapper[5052]: I1203 13:41:46.124664 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/845f4c5b-f54f-4cf4-8b78-17c73bdccfdc-bundle\") pod \"9704761d240e56fb98655ffd81084895b33a73ec711f4dcdef0450e590nfq89\" (UID: \"845f4c5b-f54f-4cf4-8b78-17c73bdccfdc\") " pod="openstack-operators/9704761d240e56fb98655ffd81084895b33a73ec711f4dcdef0450e590nfq89" Dec 03 13:41:46 crc kubenswrapper[5052]: I1203 13:41:46.168577 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-operators/rabbitmq-cluster-operator-index-spsf8"] Dec 03 13:41:46 crc kubenswrapper[5052]: I1203 13:41:46.169837 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-operators/rabbitmq-cluster-operator-index-spsf8" Dec 03 13:41:46 crc kubenswrapper[5052]: I1203 13:41:46.171840 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-operators"/"rabbitmq-cluster-operator-index-dockercfg-6w6zb" Dec 03 13:41:46 crc kubenswrapper[5052]: I1203 13:41:46.179145 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-operators/rabbitmq-cluster-operator-index-spsf8"] Dec 03 13:41:46 crc kubenswrapper[5052]: I1203 13:41:46.226180 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/845f4c5b-f54f-4cf4-8b78-17c73bdccfdc-util\") pod \"9704761d240e56fb98655ffd81084895b33a73ec711f4dcdef0450e590nfq89\" (UID: \"845f4c5b-f54f-4cf4-8b78-17c73bdccfdc\") " pod="openstack-operators/9704761d240e56fb98655ffd81084895b33a73ec711f4dcdef0450e590nfq89" Dec 03 13:41:46 crc kubenswrapper[5052]: I1203 13:41:46.226252 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-tr7rg\" (UniqueName: \"kubernetes.io/projected/845f4c5b-f54f-4cf4-8b78-17c73bdccfdc-kube-api-access-tr7rg\") pod \"9704761d240e56fb98655ffd81084895b33a73ec711f4dcdef0450e590nfq89\" (UID: \"845f4c5b-f54f-4cf4-8b78-17c73bdccfdc\") " pod="openstack-operators/9704761d240e56fb98655ffd81084895b33a73ec711f4dcdef0450e590nfq89" Dec 03 13:41:46 crc kubenswrapper[5052]: I1203 13:41:46.226315 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/845f4c5b-f54f-4cf4-8b78-17c73bdccfdc-bundle\") pod \"9704761d240e56fb98655ffd81084895b33a73ec711f4dcdef0450e590nfq89\" (UID: \"845f4c5b-f54f-4cf4-8b78-17c73bdccfdc\") " pod="openstack-operators/9704761d240e56fb98655ffd81084895b33a73ec711f4dcdef0450e590nfq89" Dec 03 13:41:46 crc kubenswrapper[5052]: I1203 13:41:46.226904 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/845f4c5b-f54f-4cf4-8b78-17c73bdccfdc-util\") pod \"9704761d240e56fb98655ffd81084895b33a73ec711f4dcdef0450e590nfq89\" (UID: \"845f4c5b-f54f-4cf4-8b78-17c73bdccfdc\") " pod="openstack-operators/9704761d240e56fb98655ffd81084895b33a73ec711f4dcdef0450e590nfq89" Dec 03 13:41:46 crc kubenswrapper[5052]: I1203 13:41:46.226919 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/845f4c5b-f54f-4cf4-8b78-17c73bdccfdc-bundle\") pod \"9704761d240e56fb98655ffd81084895b33a73ec711f4dcdef0450e590nfq89\" (UID: \"845f4c5b-f54f-4cf4-8b78-17c73bdccfdc\") " pod="openstack-operators/9704761d240e56fb98655ffd81084895b33a73ec711f4dcdef0450e590nfq89" Dec 03 13:41:46 crc kubenswrapper[5052]: I1203 13:41:46.263004 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-tr7rg\" (UniqueName: \"kubernetes.io/projected/845f4c5b-f54f-4cf4-8b78-17c73bdccfdc-kube-api-access-tr7rg\") pod \"9704761d240e56fb98655ffd81084895b33a73ec711f4dcdef0450e590nfq89\" (UID: \"845f4c5b-f54f-4cf4-8b78-17c73bdccfdc\") " pod="openstack-operators/9704761d240e56fb98655ffd81084895b33a73ec711f4dcdef0450e590nfq89" Dec 03 13:41:46 crc kubenswrapper[5052]: I1203 13:41:46.327706 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qgjq4\" (UniqueName: \"kubernetes.io/projected/b70bf54e-974c-4721-8384-2c6bceaee23f-kube-api-access-qgjq4\") pod \"rabbitmq-cluster-operator-index-spsf8\" (UID: \"b70bf54e-974c-4721-8384-2c6bceaee23f\") " pod="openstack-operators/rabbitmq-cluster-operator-index-spsf8" Dec 03 13:41:46 crc kubenswrapper[5052]: I1203 13:41:46.331265 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-operators/9704761d240e56fb98655ffd81084895b33a73ec711f4dcdef0450e590nfq89" Dec 03 13:41:46 crc kubenswrapper[5052]: I1203 13:41:46.429119 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-qgjq4\" (UniqueName: \"kubernetes.io/projected/b70bf54e-974c-4721-8384-2c6bceaee23f-kube-api-access-qgjq4\") pod \"rabbitmq-cluster-operator-index-spsf8\" (UID: \"b70bf54e-974c-4721-8384-2c6bceaee23f\") " pod="openstack-operators/rabbitmq-cluster-operator-index-spsf8" Dec 03 13:41:46 crc kubenswrapper[5052]: I1203 13:41:46.448389 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-qgjq4\" (UniqueName: \"kubernetes.io/projected/b70bf54e-974c-4721-8384-2c6bceaee23f-kube-api-access-qgjq4\") pod \"rabbitmq-cluster-operator-index-spsf8\" (UID: \"b70bf54e-974c-4721-8384-2c6bceaee23f\") " pod="openstack-operators/rabbitmq-cluster-operator-index-spsf8" Dec 03 13:41:46 crc kubenswrapper[5052]: I1203 13:41:46.489800 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-operators/rabbitmq-cluster-operator-index-spsf8" Dec 03 13:41:46 crc kubenswrapper[5052]: I1203 13:41:46.554647 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["horizon-kuttl-tests/memcached-0"] Dec 03 13:41:46 crc kubenswrapper[5052]: I1203 13:41:46.556832 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="horizon-kuttl-tests/memcached-0" Dec 03 13:41:46 crc kubenswrapper[5052]: I1203 13:41:46.565010 5052 reflector.go:368] Caches populated for *v1.Secret from object-"horizon-kuttl-tests"/"memcached-memcached-dockercfg-2bvhc" Dec 03 13:41:46 crc kubenswrapper[5052]: I1203 13:41:46.565086 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"horizon-kuttl-tests"/"memcached-config-data" Dec 03 13:41:46 crc kubenswrapper[5052]: I1203 13:41:46.568306 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["horizon-kuttl-tests/memcached-0"] Dec 03 13:41:46 crc kubenswrapper[5052]: I1203 13:41:46.583292 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openshift-marketplace/certified-operators-6jp8g"] Dec 03 13:41:46 crc kubenswrapper[5052]: I1203 13:41:46.632321 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/9dfea40e-ca29-4e45-85ff-fde0ca025e75-config-data\") pod \"memcached-0\" (UID: \"9dfea40e-ca29-4e45-85ff-fde0ca025e75\") " pod="horizon-kuttl-tests/memcached-0" Dec 03 13:41:46 crc kubenswrapper[5052]: I1203 13:41:46.632392 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lj4j7\" (UniqueName: \"kubernetes.io/projected/9dfea40e-ca29-4e45-85ff-fde0ca025e75-kube-api-access-lj4j7\") pod \"memcached-0\" (UID: \"9dfea40e-ca29-4e45-85ff-fde0ca025e75\") " pod="horizon-kuttl-tests/memcached-0" Dec 03 13:41:46 crc kubenswrapper[5052]: I1203 13:41:46.632492 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/9dfea40e-ca29-4e45-85ff-fde0ca025e75-kolla-config\") pod \"memcached-0\" (UID: \"9dfea40e-ca29-4e45-85ff-fde0ca025e75\") " pod="horizon-kuttl-tests/memcached-0" Dec 03 13:41:46 crc kubenswrapper[5052]: I1203 13:41:46.734578 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/9dfea40e-ca29-4e45-85ff-fde0ca025e75-kolla-config\") pod \"memcached-0\" (UID: \"9dfea40e-ca29-4e45-85ff-fde0ca025e75\") " pod="horizon-kuttl-tests/memcached-0" Dec 03 13:41:46 crc kubenswrapper[5052]: I1203 13:41:46.734730 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/9dfea40e-ca29-4e45-85ff-fde0ca025e75-config-data\") pod \"memcached-0\" (UID: \"9dfea40e-ca29-4e45-85ff-fde0ca025e75\") " pod="horizon-kuttl-tests/memcached-0" Dec 03 13:41:46 crc kubenswrapper[5052]: I1203 13:41:46.734805 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-lj4j7\" (UniqueName: \"kubernetes.io/projected/9dfea40e-ca29-4e45-85ff-fde0ca025e75-kube-api-access-lj4j7\") pod \"memcached-0\" (UID: \"9dfea40e-ca29-4e45-85ff-fde0ca025e75\") " pod="horizon-kuttl-tests/memcached-0" Dec 03 13:41:46 crc kubenswrapper[5052]: I1203 13:41:46.735605 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/9dfea40e-ca29-4e45-85ff-fde0ca025e75-config-data\") pod \"memcached-0\" (UID: \"9dfea40e-ca29-4e45-85ff-fde0ca025e75\") " pod="horizon-kuttl-tests/memcached-0" Dec 03 13:41:46 crc kubenswrapper[5052]: I1203 13:41:46.736798 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/9dfea40e-ca29-4e45-85ff-fde0ca025e75-kolla-config\") pod \"memcached-0\" (UID: \"9dfea40e-ca29-4e45-85ff-fde0ca025e75\") " pod="horizon-kuttl-tests/memcached-0" Dec 03 13:41:46 crc kubenswrapper[5052]: I1203 13:41:46.753454 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-lj4j7\" (UniqueName: \"kubernetes.io/projected/9dfea40e-ca29-4e45-85ff-fde0ca025e75-kube-api-access-lj4j7\") pod \"memcached-0\" (UID: \"9dfea40e-ca29-4e45-85ff-fde0ca025e75\") " pod="horizon-kuttl-tests/memcached-0" Dec 03 13:41:46 crc kubenswrapper[5052]: I1203 13:41:46.771843 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-operators/9704761d240e56fb98655ffd81084895b33a73ec711f4dcdef0450e590nfq89"] Dec 03 13:41:46 crc kubenswrapper[5052]: W1203 13:41:46.779092 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod845f4c5b_f54f_4cf4_8b78_17c73bdccfdc.slice/crio-cf2d31c5efdb47d10073fe39fd953f89b99a7fccf0ab9f754de6141e663057f2 WatchSource:0}: Error finding container cf2d31c5efdb47d10073fe39fd953f89b99a7fccf0ab9f754de6141e663057f2: Status 404 returned error can't find the container with id cf2d31c5efdb47d10073fe39fd953f89b99a7fccf0ab9f754de6141e663057f2 Dec 03 13:41:46 crc kubenswrapper[5052]: I1203 13:41:46.881926 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="horizon-kuttl-tests/memcached-0" Dec 03 13:41:46 crc kubenswrapper[5052]: I1203 13:41:46.963246 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-operators/rabbitmq-cluster-operator-index-spsf8"] Dec 03 13:41:46 crc kubenswrapper[5052]: W1203 13:41:46.976890 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb70bf54e_974c_4721_8384_2c6bceaee23f.slice/crio-0afac7c1b1cfca5ac7751926324f2fa323ed32cb3638b4b6bd3bee2d0e23f7e6 WatchSource:0}: Error finding container 0afac7c1b1cfca5ac7751926324f2fa323ed32cb3638b4b6bd3bee2d0e23f7e6: Status 404 returned error can't find the container with id 0afac7c1b1cfca5ac7751926324f2fa323ed32cb3638b4b6bd3bee2d0e23f7e6 Dec 03 13:41:47 crc kubenswrapper[5052]: I1203 13:41:47.267605 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/9704761d240e56fb98655ffd81084895b33a73ec711f4dcdef0450e590nfq89" event={"ID":"845f4c5b-f54f-4cf4-8b78-17c73bdccfdc","Type":"ContainerStarted","Data":"cf2d31c5efdb47d10073fe39fd953f89b99a7fccf0ab9f754de6141e663057f2"} Dec 03 13:41:47 crc kubenswrapper[5052]: I1203 13:41:47.269098 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openshift-marketplace/certified-operators-6jp8g" podUID="3cfb23b9-370f-493f-9595-229867b9c138" containerName="registry-server" containerID="cri-o://e1bf00783028e79920479fbf87373fcb30a17688bfac601c062863d520ef931a" gracePeriod=2 Dec 03 13:41:47 crc kubenswrapper[5052]: I1203 13:41:47.269336 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/rabbitmq-cluster-operator-index-spsf8" event={"ID":"b70bf54e-974c-4721-8384-2c6bceaee23f","Type":"ContainerStarted","Data":"0afac7c1b1cfca5ac7751926324f2fa323ed32cb3638b4b6bd3bee2d0e23f7e6"} Dec 03 13:41:47 crc kubenswrapper[5052]: I1203 13:41:47.317016 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["horizon-kuttl-tests/memcached-0"] Dec 03 13:41:48 crc kubenswrapper[5052]: I1203 13:41:48.094831 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/certified-operators-6jp8g" Dec 03 13:41:48 crc kubenswrapper[5052]: I1203 13:41:48.261616 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/3cfb23b9-370f-493f-9595-229867b9c138-catalog-content\") pod \"3cfb23b9-370f-493f-9595-229867b9c138\" (UID: \"3cfb23b9-370f-493f-9595-229867b9c138\") " Dec 03 13:41:48 crc kubenswrapper[5052]: I1203 13:41:48.261688 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tddxg\" (UniqueName: \"kubernetes.io/projected/3cfb23b9-370f-493f-9595-229867b9c138-kube-api-access-tddxg\") pod \"3cfb23b9-370f-493f-9595-229867b9c138\" (UID: \"3cfb23b9-370f-493f-9595-229867b9c138\") " Dec 03 13:41:48 crc kubenswrapper[5052]: I1203 13:41:48.261705 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/3cfb23b9-370f-493f-9595-229867b9c138-utilities\") pod \"3cfb23b9-370f-493f-9595-229867b9c138\" (UID: \"3cfb23b9-370f-493f-9595-229867b9c138\") " Dec 03 13:41:48 crc kubenswrapper[5052]: I1203 13:41:48.262944 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3cfb23b9-370f-493f-9595-229867b9c138-utilities" (OuterVolumeSpecName: "utilities") pod "3cfb23b9-370f-493f-9595-229867b9c138" (UID: "3cfb23b9-370f-493f-9595-229867b9c138"). InnerVolumeSpecName "utilities". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 13:41:48 crc kubenswrapper[5052]: I1203 13:41:48.275595 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3cfb23b9-370f-493f-9595-229867b9c138-kube-api-access-tddxg" (OuterVolumeSpecName: "kube-api-access-tddxg") pod "3cfb23b9-370f-493f-9595-229867b9c138" (UID: "3cfb23b9-370f-493f-9595-229867b9c138"). InnerVolumeSpecName "kube-api-access-tddxg". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:41:48 crc kubenswrapper[5052]: I1203 13:41:48.277937 5052 generic.go:334] "Generic (PLEG): container finished" podID="3cfb23b9-370f-493f-9595-229867b9c138" containerID="e1bf00783028e79920479fbf87373fcb30a17688bfac601c062863d520ef931a" exitCode=0 Dec 03 13:41:48 crc kubenswrapper[5052]: I1203 13:41:48.278000 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/certified-operators-6jp8g" event={"ID":"3cfb23b9-370f-493f-9595-229867b9c138","Type":"ContainerDied","Data":"e1bf00783028e79920479fbf87373fcb30a17688bfac601c062863d520ef931a"} Dec 03 13:41:48 crc kubenswrapper[5052]: I1203 13:41:48.278027 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/certified-operators-6jp8g" Dec 03 13:41:48 crc kubenswrapper[5052]: I1203 13:41:48.278052 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/certified-operators-6jp8g" event={"ID":"3cfb23b9-370f-493f-9595-229867b9c138","Type":"ContainerDied","Data":"90893d952daa3286a6669a2a664a77d85d513dbcb2f40511d9a38779b5d6f1da"} Dec 03 13:41:48 crc kubenswrapper[5052]: I1203 13:41:48.278081 5052 scope.go:117] "RemoveContainer" containerID="e1bf00783028e79920479fbf87373fcb30a17688bfac601c062863d520ef931a" Dec 03 13:41:48 crc kubenswrapper[5052]: I1203 13:41:48.279889 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="horizon-kuttl-tests/memcached-0" event={"ID":"9dfea40e-ca29-4e45-85ff-fde0ca025e75","Type":"ContainerStarted","Data":"3da4c1b8183200dee64b40f8bf2a81811e6e3aa41058e945b5ba1277203b2410"} Dec 03 13:41:48 crc kubenswrapper[5052]: I1203 13:41:48.279942 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="horizon-kuttl-tests/memcached-0" event={"ID":"9dfea40e-ca29-4e45-85ff-fde0ca025e75","Type":"ContainerStarted","Data":"079d371f2803e67ed128a139ef7126f51323a9fb550c4d00094fd7db8be77b7b"} Dec 03 13:41:48 crc kubenswrapper[5052]: I1203 13:41:48.280734 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="horizon-kuttl-tests/memcached-0" Dec 03 13:41:48 crc kubenswrapper[5052]: I1203 13:41:48.284890 5052 generic.go:334] "Generic (PLEG): container finished" podID="845f4c5b-f54f-4cf4-8b78-17c73bdccfdc" containerID="f5f0bb0a00d163138a9bdba06050a73ef3becaa2960669339132343450137116" exitCode=0 Dec 03 13:41:48 crc kubenswrapper[5052]: I1203 13:41:48.284948 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/9704761d240e56fb98655ffd81084895b33a73ec711f4dcdef0450e590nfq89" event={"ID":"845f4c5b-f54f-4cf4-8b78-17c73bdccfdc","Type":"ContainerDied","Data":"f5f0bb0a00d163138a9bdba06050a73ef3becaa2960669339132343450137116"} Dec 03 13:41:48 crc kubenswrapper[5052]: I1203 13:41:48.286945 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/rabbitmq-cluster-operator-index-spsf8" event={"ID":"b70bf54e-974c-4721-8384-2c6bceaee23f","Type":"ContainerStarted","Data":"1fbf3c65821d10127aefc7f2256bb0068a8e464cb01a82c3b91a365cc1011a71"} Dec 03 13:41:48 crc kubenswrapper[5052]: I1203 13:41:48.320988 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3cfb23b9-370f-493f-9595-229867b9c138-catalog-content" (OuterVolumeSpecName: "catalog-content") pod "3cfb23b9-370f-493f-9595-229867b9c138" (UID: "3cfb23b9-370f-493f-9595-229867b9c138"). InnerVolumeSpecName "catalog-content". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 13:41:48 crc kubenswrapper[5052]: I1203 13:41:48.325585 5052 scope.go:117] "RemoveContainer" containerID="84ae7999255ab0fcad07b1ee6430977c32d2b0d9c89a45131754c4dfd9935f0f" Dec 03 13:41:48 crc kubenswrapper[5052]: I1203 13:41:48.331009 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="horizon-kuttl-tests/memcached-0" podStartSLOduration=2.330987969 podStartE2EDuration="2.330987969s" podCreationTimestamp="2025-12-03 13:41:46 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 13:41:48.31146524 +0000 UTC m=+8498.899707024" watchObservedRunningTime="2025-12-03 13:41:48.330987969 +0000 UTC m=+8498.919229723" Dec 03 13:41:48 crc kubenswrapper[5052]: I1203 13:41:48.350585 5052 scope.go:117] "RemoveContainer" containerID="18b905b4094c643d8df1fc7d464fe687bbd51e63ce09a43f85c4af4c01e34bc1" Dec 03 13:41:48 crc kubenswrapper[5052]: I1203 13:41:48.351502 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-operators/rabbitmq-cluster-operator-index-spsf8" podStartSLOduration=1.822917269 podStartE2EDuration="2.351482177s" podCreationTimestamp="2025-12-03 13:41:46 +0000 UTC" firstStartedPulling="2025-12-03 13:41:46.980399068 +0000 UTC m=+8497.568640852" lastFinishedPulling="2025-12-03 13:41:47.508963976 +0000 UTC m=+8498.097205760" observedRunningTime="2025-12-03 13:41:48.342711986 +0000 UTC m=+8498.930953760" watchObservedRunningTime="2025-12-03 13:41:48.351482177 +0000 UTC m=+8498.939723941" Dec 03 13:41:48 crc kubenswrapper[5052]: I1203 13:41:48.365717 5052 reconciler_common.go:293] "Volume detached for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/3cfb23b9-370f-493f-9595-229867b9c138-catalog-content\") on node \"crc\" DevicePath \"\"" Dec 03 13:41:48 crc kubenswrapper[5052]: I1203 13:41:48.365760 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-tddxg\" (UniqueName: \"kubernetes.io/projected/3cfb23b9-370f-493f-9595-229867b9c138-kube-api-access-tddxg\") on node \"crc\" DevicePath \"\"" Dec 03 13:41:48 crc kubenswrapper[5052]: I1203 13:41:48.365773 5052 reconciler_common.go:293] "Volume detached for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/3cfb23b9-370f-493f-9595-229867b9c138-utilities\") on node \"crc\" DevicePath \"\"" Dec 03 13:41:48 crc kubenswrapper[5052]: I1203 13:41:48.380774 5052 scope.go:117] "RemoveContainer" containerID="e1bf00783028e79920479fbf87373fcb30a17688bfac601c062863d520ef931a" Dec 03 13:41:48 crc kubenswrapper[5052]: E1203 13:41:48.381187 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e1bf00783028e79920479fbf87373fcb30a17688bfac601c062863d520ef931a\": container with ID starting with e1bf00783028e79920479fbf87373fcb30a17688bfac601c062863d520ef931a not found: ID does not exist" containerID="e1bf00783028e79920479fbf87373fcb30a17688bfac601c062863d520ef931a" Dec 03 13:41:48 crc kubenswrapper[5052]: I1203 13:41:48.381230 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e1bf00783028e79920479fbf87373fcb30a17688bfac601c062863d520ef931a"} err="failed to get container status \"e1bf00783028e79920479fbf87373fcb30a17688bfac601c062863d520ef931a\": rpc error: code = NotFound desc = could not find container \"e1bf00783028e79920479fbf87373fcb30a17688bfac601c062863d520ef931a\": container with ID starting with e1bf00783028e79920479fbf87373fcb30a17688bfac601c062863d520ef931a not found: ID does not exist" Dec 03 13:41:48 crc kubenswrapper[5052]: I1203 13:41:48.381254 5052 scope.go:117] "RemoveContainer" containerID="84ae7999255ab0fcad07b1ee6430977c32d2b0d9c89a45131754c4dfd9935f0f" Dec 03 13:41:48 crc kubenswrapper[5052]: E1203 13:41:48.381677 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"84ae7999255ab0fcad07b1ee6430977c32d2b0d9c89a45131754c4dfd9935f0f\": container with ID starting with 84ae7999255ab0fcad07b1ee6430977c32d2b0d9c89a45131754c4dfd9935f0f not found: ID does not exist" containerID="84ae7999255ab0fcad07b1ee6430977c32d2b0d9c89a45131754c4dfd9935f0f" Dec 03 13:41:48 crc kubenswrapper[5052]: I1203 13:41:48.381758 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"84ae7999255ab0fcad07b1ee6430977c32d2b0d9c89a45131754c4dfd9935f0f"} err="failed to get container status \"84ae7999255ab0fcad07b1ee6430977c32d2b0d9c89a45131754c4dfd9935f0f\": rpc error: code = NotFound desc = could not find container \"84ae7999255ab0fcad07b1ee6430977c32d2b0d9c89a45131754c4dfd9935f0f\": container with ID starting with 84ae7999255ab0fcad07b1ee6430977c32d2b0d9c89a45131754c4dfd9935f0f not found: ID does not exist" Dec 03 13:41:48 crc kubenswrapper[5052]: I1203 13:41:48.381801 5052 scope.go:117] "RemoveContainer" containerID="18b905b4094c643d8df1fc7d464fe687bbd51e63ce09a43f85c4af4c01e34bc1" Dec 03 13:41:48 crc kubenswrapper[5052]: E1203 13:41:48.382036 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"18b905b4094c643d8df1fc7d464fe687bbd51e63ce09a43f85c4af4c01e34bc1\": container with ID starting with 18b905b4094c643d8df1fc7d464fe687bbd51e63ce09a43f85c4af4c01e34bc1 not found: ID does not exist" containerID="18b905b4094c643d8df1fc7d464fe687bbd51e63ce09a43f85c4af4c01e34bc1" Dec 03 13:41:48 crc kubenswrapper[5052]: I1203 13:41:48.382064 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"18b905b4094c643d8df1fc7d464fe687bbd51e63ce09a43f85c4af4c01e34bc1"} err="failed to get container status \"18b905b4094c643d8df1fc7d464fe687bbd51e63ce09a43f85c4af4c01e34bc1\": rpc error: code = NotFound desc = could not find container \"18b905b4094c643d8df1fc7d464fe687bbd51e63ce09a43f85c4af4c01e34bc1\": container with ID starting with 18b905b4094c643d8df1fc7d464fe687bbd51e63ce09a43f85c4af4c01e34bc1 not found: ID does not exist" Dec 03 13:41:48 crc kubenswrapper[5052]: I1203 13:41:48.611379 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openshift-marketplace/certified-operators-6jp8g"] Dec 03 13:41:48 crc kubenswrapper[5052]: I1203 13:41:48.618339 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openshift-marketplace/certified-operators-6jp8g"] Dec 03 13:41:49 crc kubenswrapper[5052]: I1203 13:41:49.302490 5052 generic.go:334] "Generic (PLEG): container finished" podID="845f4c5b-f54f-4cf4-8b78-17c73bdccfdc" containerID="e13a734f46760dde236b0ad6acc81c9bd68e9050cb89d5e1a84ab2008f0ccb9c" exitCode=0 Dec 03 13:41:49 crc kubenswrapper[5052]: I1203 13:41:49.302565 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/9704761d240e56fb98655ffd81084895b33a73ec711f4dcdef0450e590nfq89" event={"ID":"845f4c5b-f54f-4cf4-8b78-17c73bdccfdc","Type":"ContainerDied","Data":"e13a734f46760dde236b0ad6acc81c9bd68e9050cb89d5e1a84ab2008f0ccb9c"} Dec 03 13:41:49 crc kubenswrapper[5052]: I1203 13:41:49.681051 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3cfb23b9-370f-493f-9595-229867b9c138" path="/var/lib/kubelet/pods/3cfb23b9-370f-493f-9595-229867b9c138/volumes" Dec 03 13:41:50 crc kubenswrapper[5052]: I1203 13:41:50.310444 5052 generic.go:334] "Generic (PLEG): container finished" podID="845f4c5b-f54f-4cf4-8b78-17c73bdccfdc" containerID="15c5bcdbe9f2bd77b8f6616dc862afa27d22a4b1d007a7072eab0bb3d373ea96" exitCode=0 Dec 03 13:41:50 crc kubenswrapper[5052]: I1203 13:41:50.310488 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/9704761d240e56fb98655ffd81084895b33a73ec711f4dcdef0450e590nfq89" event={"ID":"845f4c5b-f54f-4cf4-8b78-17c73bdccfdc","Type":"ContainerDied","Data":"15c5bcdbe9f2bd77b8f6616dc862afa27d22a4b1d007a7072eab0bb3d373ea96"} Dec 03 13:41:51 crc kubenswrapper[5052]: I1203 13:41:51.649185 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-operators/9704761d240e56fb98655ffd81084895b33a73ec711f4dcdef0450e590nfq89" Dec 03 13:41:51 crc kubenswrapper[5052]: I1203 13:41:51.825101 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/845f4c5b-f54f-4cf4-8b78-17c73bdccfdc-util\") pod \"845f4c5b-f54f-4cf4-8b78-17c73bdccfdc\" (UID: \"845f4c5b-f54f-4cf4-8b78-17c73bdccfdc\") " Dec 03 13:41:51 crc kubenswrapper[5052]: I1203 13:41:51.825143 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/845f4c5b-f54f-4cf4-8b78-17c73bdccfdc-bundle\") pod \"845f4c5b-f54f-4cf4-8b78-17c73bdccfdc\" (UID: \"845f4c5b-f54f-4cf4-8b78-17c73bdccfdc\") " Dec 03 13:41:51 crc kubenswrapper[5052]: I1203 13:41:51.825226 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tr7rg\" (UniqueName: \"kubernetes.io/projected/845f4c5b-f54f-4cf4-8b78-17c73bdccfdc-kube-api-access-tr7rg\") pod \"845f4c5b-f54f-4cf4-8b78-17c73bdccfdc\" (UID: \"845f4c5b-f54f-4cf4-8b78-17c73bdccfdc\") " Dec 03 13:41:51 crc kubenswrapper[5052]: I1203 13:41:51.825857 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/845f4c5b-f54f-4cf4-8b78-17c73bdccfdc-bundle" (OuterVolumeSpecName: "bundle") pod "845f4c5b-f54f-4cf4-8b78-17c73bdccfdc" (UID: "845f4c5b-f54f-4cf4-8b78-17c73bdccfdc"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 13:41:51 crc kubenswrapper[5052]: I1203 13:41:51.830221 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/845f4c5b-f54f-4cf4-8b78-17c73bdccfdc-kube-api-access-tr7rg" (OuterVolumeSpecName: "kube-api-access-tr7rg") pod "845f4c5b-f54f-4cf4-8b78-17c73bdccfdc" (UID: "845f4c5b-f54f-4cf4-8b78-17c73bdccfdc"). InnerVolumeSpecName "kube-api-access-tr7rg". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:41:51 crc kubenswrapper[5052]: I1203 13:41:51.840255 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/845f4c5b-f54f-4cf4-8b78-17c73bdccfdc-util" (OuterVolumeSpecName: "util") pod "845f4c5b-f54f-4cf4-8b78-17c73bdccfdc" (UID: "845f4c5b-f54f-4cf4-8b78-17c73bdccfdc"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 13:41:51 crc kubenswrapper[5052]: I1203 13:41:51.927282 5052 reconciler_common.go:293] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/845f4c5b-f54f-4cf4-8b78-17c73bdccfdc-util\") on node \"crc\" DevicePath \"\"" Dec 03 13:41:51 crc kubenswrapper[5052]: I1203 13:41:51.927318 5052 reconciler_common.go:293] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/845f4c5b-f54f-4cf4-8b78-17c73bdccfdc-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 13:41:51 crc kubenswrapper[5052]: I1203 13:41:51.927332 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-tr7rg\" (UniqueName: \"kubernetes.io/projected/845f4c5b-f54f-4cf4-8b78-17c73bdccfdc-kube-api-access-tr7rg\") on node \"crc\" DevicePath \"\"" Dec 03 13:41:52 crc kubenswrapper[5052]: I1203 13:41:52.327468 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/9704761d240e56fb98655ffd81084895b33a73ec711f4dcdef0450e590nfq89" event={"ID":"845f4c5b-f54f-4cf4-8b78-17c73bdccfdc","Type":"ContainerDied","Data":"cf2d31c5efdb47d10073fe39fd953f89b99a7fccf0ab9f754de6141e663057f2"} Dec 03 13:41:52 crc kubenswrapper[5052]: I1203 13:41:52.327548 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-operators/9704761d240e56fb98655ffd81084895b33a73ec711f4dcdef0450e590nfq89" Dec 03 13:41:52 crc kubenswrapper[5052]: I1203 13:41:52.327557 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cf2d31c5efdb47d10073fe39fd953f89b99a7fccf0ab9f754de6141e663057f2" Dec 03 13:41:56 crc kubenswrapper[5052]: I1203 13:41:56.491040 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-operators/rabbitmq-cluster-operator-index-spsf8" Dec 03 13:41:56 crc kubenswrapper[5052]: I1203 13:41:56.492107 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-operators/rabbitmq-cluster-operator-index-spsf8" Dec 03 13:41:56 crc kubenswrapper[5052]: I1203 13:41:56.512926 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-operators/rabbitmq-cluster-operator-index-spsf8" Dec 03 13:41:56 crc kubenswrapper[5052]: I1203 13:41:56.883403 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="horizon-kuttl-tests/memcached-0" Dec 03 13:41:57 crc kubenswrapper[5052]: I1203 13:41:57.383858 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-operators/rabbitmq-cluster-operator-index-spsf8" Dec 03 13:41:58 crc kubenswrapper[5052]: I1203 13:41:58.673365 5052 scope.go:117] "RemoveContainer" containerID="2fbd293d6edef0fa059e91c119312e77b262534107c322feb7b59dbea3464526" Dec 03 13:41:58 crc kubenswrapper[5052]: E1203 13:41:58.673622 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 13:42:01 crc kubenswrapper[5052]: I1203 13:42:01.083213 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-operators/rabbitmq-cluster-operator-779fc9694b-7kdqx"] Dec 03 13:42:01 crc kubenswrapper[5052]: E1203 13:42:01.083743 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="3cfb23b9-370f-493f-9595-229867b9c138" containerName="registry-server" Dec 03 13:42:01 crc kubenswrapper[5052]: I1203 13:42:01.083754 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="3cfb23b9-370f-493f-9595-229867b9c138" containerName="registry-server" Dec 03 13:42:01 crc kubenswrapper[5052]: E1203 13:42:01.083778 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="3cfb23b9-370f-493f-9595-229867b9c138" containerName="extract-utilities" Dec 03 13:42:01 crc kubenswrapper[5052]: I1203 13:42:01.083784 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="3cfb23b9-370f-493f-9595-229867b9c138" containerName="extract-utilities" Dec 03 13:42:01 crc kubenswrapper[5052]: E1203 13:42:01.083797 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="845f4c5b-f54f-4cf4-8b78-17c73bdccfdc" containerName="util" Dec 03 13:42:01 crc kubenswrapper[5052]: I1203 13:42:01.083804 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="845f4c5b-f54f-4cf4-8b78-17c73bdccfdc" containerName="util" Dec 03 13:42:01 crc kubenswrapper[5052]: E1203 13:42:01.083813 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="845f4c5b-f54f-4cf4-8b78-17c73bdccfdc" containerName="extract" Dec 03 13:42:01 crc kubenswrapper[5052]: I1203 13:42:01.083819 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="845f4c5b-f54f-4cf4-8b78-17c73bdccfdc" containerName="extract" Dec 03 13:42:01 crc kubenswrapper[5052]: E1203 13:42:01.083837 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="3cfb23b9-370f-493f-9595-229867b9c138" containerName="extract-content" Dec 03 13:42:01 crc kubenswrapper[5052]: I1203 13:42:01.083843 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="3cfb23b9-370f-493f-9595-229867b9c138" containerName="extract-content" Dec 03 13:42:01 crc kubenswrapper[5052]: E1203 13:42:01.083849 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="845f4c5b-f54f-4cf4-8b78-17c73bdccfdc" containerName="pull" Dec 03 13:42:01 crc kubenswrapper[5052]: I1203 13:42:01.083855 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="845f4c5b-f54f-4cf4-8b78-17c73bdccfdc" containerName="pull" Dec 03 13:42:01 crc kubenswrapper[5052]: I1203 13:42:01.083985 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="845f4c5b-f54f-4cf4-8b78-17c73bdccfdc" containerName="extract" Dec 03 13:42:01 crc kubenswrapper[5052]: I1203 13:42:01.083998 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="3cfb23b9-370f-493f-9595-229867b9c138" containerName="registry-server" Dec 03 13:42:01 crc kubenswrapper[5052]: I1203 13:42:01.084485 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-operators/rabbitmq-cluster-operator-779fc9694b-7kdqx" Dec 03 13:42:01 crc kubenswrapper[5052]: I1203 13:42:01.087609 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-operators"/"rabbitmq-cluster-operator-dockercfg-nfp8p" Dec 03 13:42:01 crc kubenswrapper[5052]: I1203 13:42:01.093106 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-operators/rabbitmq-cluster-operator-779fc9694b-7kdqx"] Dec 03 13:42:01 crc kubenswrapper[5052]: I1203 13:42:01.256137 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k9gzx\" (UniqueName: \"kubernetes.io/projected/202fe9d9-dcdf-43ae-bfc4-4efc7f9e7dba-kube-api-access-k9gzx\") pod \"rabbitmq-cluster-operator-779fc9694b-7kdqx\" (UID: \"202fe9d9-dcdf-43ae-bfc4-4efc7f9e7dba\") " pod="openstack-operators/rabbitmq-cluster-operator-779fc9694b-7kdqx" Dec 03 13:42:01 crc kubenswrapper[5052]: I1203 13:42:01.358032 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-k9gzx\" (UniqueName: \"kubernetes.io/projected/202fe9d9-dcdf-43ae-bfc4-4efc7f9e7dba-kube-api-access-k9gzx\") pod \"rabbitmq-cluster-operator-779fc9694b-7kdqx\" (UID: \"202fe9d9-dcdf-43ae-bfc4-4efc7f9e7dba\") " pod="openstack-operators/rabbitmq-cluster-operator-779fc9694b-7kdqx" Dec 03 13:42:01 crc kubenswrapper[5052]: I1203 13:42:01.381558 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-k9gzx\" (UniqueName: \"kubernetes.io/projected/202fe9d9-dcdf-43ae-bfc4-4efc7f9e7dba-kube-api-access-k9gzx\") pod \"rabbitmq-cluster-operator-779fc9694b-7kdqx\" (UID: \"202fe9d9-dcdf-43ae-bfc4-4efc7f9e7dba\") " pod="openstack-operators/rabbitmq-cluster-operator-779fc9694b-7kdqx" Dec 03 13:42:01 crc kubenswrapper[5052]: I1203 13:42:01.407305 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-operators/rabbitmq-cluster-operator-779fc9694b-7kdqx" Dec 03 13:42:01 crc kubenswrapper[5052]: I1203 13:42:01.897190 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-operators/rabbitmq-cluster-operator-779fc9694b-7kdqx"] Dec 03 13:42:01 crc kubenswrapper[5052]: W1203 13:42:01.902653 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod202fe9d9_dcdf_43ae_bfc4_4efc7f9e7dba.slice/crio-edbbebf2ec7e0cba6a6ca54c9e584a4ccf23e889d590cd2c044a08111204f856 WatchSource:0}: Error finding container edbbebf2ec7e0cba6a6ca54c9e584a4ccf23e889d590cd2c044a08111204f856: Status 404 returned error can't find the container with id edbbebf2ec7e0cba6a6ca54c9e584a4ccf23e889d590cd2c044a08111204f856 Dec 03 13:42:02 crc kubenswrapper[5052]: I1203 13:42:02.392448 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/rabbitmq-cluster-operator-779fc9694b-7kdqx" event={"ID":"202fe9d9-dcdf-43ae-bfc4-4efc7f9e7dba","Type":"ContainerStarted","Data":"e0e745f7b4d4193fc358279485d9af00093d854c8e76c0c9fccc12f7c2007e8d"} Dec 03 13:42:02 crc kubenswrapper[5052]: I1203 13:42:02.392486 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/rabbitmq-cluster-operator-779fc9694b-7kdqx" event={"ID":"202fe9d9-dcdf-43ae-bfc4-4efc7f9e7dba","Type":"ContainerStarted","Data":"edbbebf2ec7e0cba6a6ca54c9e584a4ccf23e889d590cd2c044a08111204f856"} Dec 03 13:42:02 crc kubenswrapper[5052]: I1203 13:42:02.410533 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-operators/rabbitmq-cluster-operator-779fc9694b-7kdqx" podStartSLOduration=1.410490616 podStartE2EDuration="1.410490616s" podCreationTimestamp="2025-12-03 13:42:01 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 13:42:02.405897664 +0000 UTC m=+8512.994139428" watchObservedRunningTime="2025-12-03 13:42:02.410490616 +0000 UTC m=+8512.998732370" Dec 03 13:42:09 crc kubenswrapper[5052]: I1203 13:42:09.695741 5052 scope.go:117] "RemoveContainer" containerID="2fbd293d6edef0fa059e91c119312e77b262534107c322feb7b59dbea3464526" Dec 03 13:42:09 crc kubenswrapper[5052]: E1203 13:42:09.699278 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 13:42:10 crc kubenswrapper[5052]: I1203 13:42:10.167177 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-operators/keystone-operator-index-dtmwp"] Dec 03 13:42:10 crc kubenswrapper[5052]: I1203 13:42:10.168070 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-operators/keystone-operator-index-dtmwp" Dec 03 13:42:10 crc kubenswrapper[5052]: I1203 13:42:10.175143 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-operators"/"keystone-operator-index-dockercfg-r282m" Dec 03 13:42:10 crc kubenswrapper[5052]: I1203 13:42:10.183718 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-operators/keystone-operator-index-dtmwp"] Dec 03 13:42:10 crc kubenswrapper[5052]: I1203 13:42:10.192665 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2pqnh\" (UniqueName: \"kubernetes.io/projected/2273c0a7-907e-4e0c-b6bf-f193263d9db7-kube-api-access-2pqnh\") pod \"keystone-operator-index-dtmwp\" (UID: \"2273c0a7-907e-4e0c-b6bf-f193263d9db7\") " pod="openstack-operators/keystone-operator-index-dtmwp" Dec 03 13:42:10 crc kubenswrapper[5052]: I1203 13:42:10.294262 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-2pqnh\" (UniqueName: \"kubernetes.io/projected/2273c0a7-907e-4e0c-b6bf-f193263d9db7-kube-api-access-2pqnh\") pod \"keystone-operator-index-dtmwp\" (UID: \"2273c0a7-907e-4e0c-b6bf-f193263d9db7\") " pod="openstack-operators/keystone-operator-index-dtmwp" Dec 03 13:42:10 crc kubenswrapper[5052]: I1203 13:42:10.332250 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-2pqnh\" (UniqueName: \"kubernetes.io/projected/2273c0a7-907e-4e0c-b6bf-f193263d9db7-kube-api-access-2pqnh\") pod \"keystone-operator-index-dtmwp\" (UID: \"2273c0a7-907e-4e0c-b6bf-f193263d9db7\") " pod="openstack-operators/keystone-operator-index-dtmwp" Dec 03 13:42:10 crc kubenswrapper[5052]: I1203 13:42:10.486730 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-operators/keystone-operator-index-dtmwp" Dec 03 13:42:10 crc kubenswrapper[5052]: I1203 13:42:10.904313 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-operators/keystone-operator-index-dtmwp"] Dec 03 13:42:10 crc kubenswrapper[5052]: W1203 13:42:10.909823 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2273c0a7_907e_4e0c_b6bf_f193263d9db7.slice/crio-e4e83015e6d431c38f68278b26cce99cab88dc458d7386fe65efcf1a06ef0620 WatchSource:0}: Error finding container e4e83015e6d431c38f68278b26cce99cab88dc458d7386fe65efcf1a06ef0620: Status 404 returned error can't find the container with id e4e83015e6d431c38f68278b26cce99cab88dc458d7386fe65efcf1a06ef0620 Dec 03 13:42:11 crc kubenswrapper[5052]: I1203 13:42:11.404992 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-operators/49c083020ae5dfe237b73a6c6b807501660a323f061d7879268c43a121w2872"] Dec 03 13:42:11 crc kubenswrapper[5052]: I1203 13:42:11.406880 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-operators/49c083020ae5dfe237b73a6c6b807501660a323f061d7879268c43a121w2872" Dec 03 13:42:11 crc kubenswrapper[5052]: I1203 13:42:11.409435 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-operators"/"default-dockercfg-7fklr" Dec 03 13:42:11 crc kubenswrapper[5052]: I1203 13:42:11.410151 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ec7ca777-d2e5-4b89-91ad-5084b2716811-util\") pod \"49c083020ae5dfe237b73a6c6b807501660a323f061d7879268c43a121w2872\" (UID: \"ec7ca777-d2e5-4b89-91ad-5084b2716811\") " pod="openstack-operators/49c083020ae5dfe237b73a6c6b807501660a323f061d7879268c43a121w2872" Dec 03 13:42:11 crc kubenswrapper[5052]: I1203 13:42:11.410196 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gnmpt\" (UniqueName: \"kubernetes.io/projected/ec7ca777-d2e5-4b89-91ad-5084b2716811-kube-api-access-gnmpt\") pod \"49c083020ae5dfe237b73a6c6b807501660a323f061d7879268c43a121w2872\" (UID: \"ec7ca777-d2e5-4b89-91ad-5084b2716811\") " pod="openstack-operators/49c083020ae5dfe237b73a6c6b807501660a323f061d7879268c43a121w2872" Dec 03 13:42:11 crc kubenswrapper[5052]: I1203 13:42:11.410259 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ec7ca777-d2e5-4b89-91ad-5084b2716811-bundle\") pod \"49c083020ae5dfe237b73a6c6b807501660a323f061d7879268c43a121w2872\" (UID: \"ec7ca777-d2e5-4b89-91ad-5084b2716811\") " pod="openstack-operators/49c083020ae5dfe237b73a6c6b807501660a323f061d7879268c43a121w2872" Dec 03 13:42:11 crc kubenswrapper[5052]: I1203 13:42:11.425016 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-operators/49c083020ae5dfe237b73a6c6b807501660a323f061d7879268c43a121w2872"] Dec 03 13:42:11 crc kubenswrapper[5052]: I1203 13:42:11.457554 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/keystone-operator-index-dtmwp" event={"ID":"2273c0a7-907e-4e0c-b6bf-f193263d9db7","Type":"ContainerStarted","Data":"e4e83015e6d431c38f68278b26cce99cab88dc458d7386fe65efcf1a06ef0620"} Dec 03 13:42:11 crc kubenswrapper[5052]: I1203 13:42:11.511006 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ec7ca777-d2e5-4b89-91ad-5084b2716811-util\") pod \"49c083020ae5dfe237b73a6c6b807501660a323f061d7879268c43a121w2872\" (UID: \"ec7ca777-d2e5-4b89-91ad-5084b2716811\") " pod="openstack-operators/49c083020ae5dfe237b73a6c6b807501660a323f061d7879268c43a121w2872" Dec 03 13:42:11 crc kubenswrapper[5052]: I1203 13:42:11.511430 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-gnmpt\" (UniqueName: \"kubernetes.io/projected/ec7ca777-d2e5-4b89-91ad-5084b2716811-kube-api-access-gnmpt\") pod \"49c083020ae5dfe237b73a6c6b807501660a323f061d7879268c43a121w2872\" (UID: \"ec7ca777-d2e5-4b89-91ad-5084b2716811\") " pod="openstack-operators/49c083020ae5dfe237b73a6c6b807501660a323f061d7879268c43a121w2872" Dec 03 13:42:11 crc kubenswrapper[5052]: I1203 13:42:11.511490 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ec7ca777-d2e5-4b89-91ad-5084b2716811-util\") pod \"49c083020ae5dfe237b73a6c6b807501660a323f061d7879268c43a121w2872\" (UID: \"ec7ca777-d2e5-4b89-91ad-5084b2716811\") " pod="openstack-operators/49c083020ae5dfe237b73a6c6b807501660a323f061d7879268c43a121w2872" Dec 03 13:42:11 crc kubenswrapper[5052]: I1203 13:42:11.511971 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ec7ca777-d2e5-4b89-91ad-5084b2716811-bundle\") pod \"49c083020ae5dfe237b73a6c6b807501660a323f061d7879268c43a121w2872\" (UID: \"ec7ca777-d2e5-4b89-91ad-5084b2716811\") " pod="openstack-operators/49c083020ae5dfe237b73a6c6b807501660a323f061d7879268c43a121w2872" Dec 03 13:42:11 crc kubenswrapper[5052]: I1203 13:42:11.512350 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ec7ca777-d2e5-4b89-91ad-5084b2716811-bundle\") pod \"49c083020ae5dfe237b73a6c6b807501660a323f061d7879268c43a121w2872\" (UID: \"ec7ca777-d2e5-4b89-91ad-5084b2716811\") " pod="openstack-operators/49c083020ae5dfe237b73a6c6b807501660a323f061d7879268c43a121w2872" Dec 03 13:42:11 crc kubenswrapper[5052]: I1203 13:42:11.531869 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-gnmpt\" (UniqueName: \"kubernetes.io/projected/ec7ca777-d2e5-4b89-91ad-5084b2716811-kube-api-access-gnmpt\") pod \"49c083020ae5dfe237b73a6c6b807501660a323f061d7879268c43a121w2872\" (UID: \"ec7ca777-d2e5-4b89-91ad-5084b2716811\") " pod="openstack-operators/49c083020ae5dfe237b73a6c6b807501660a323f061d7879268c43a121w2872" Dec 03 13:42:11 crc kubenswrapper[5052]: I1203 13:42:11.727491 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-operators/49c083020ae5dfe237b73a6c6b807501660a323f061d7879268c43a121w2872" Dec 03 13:42:12 crc kubenswrapper[5052]: I1203 13:42:12.132016 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-operators/49c083020ae5dfe237b73a6c6b807501660a323f061d7879268c43a121w2872"] Dec 03 13:42:12 crc kubenswrapper[5052]: W1203 13:42:12.137995 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec7ca777_d2e5_4b89_91ad_5084b2716811.slice/crio-e094773c81ce36f3545a1f1f6f31dae47db2220500ff97dace1daa35c16c3c72 WatchSource:0}: Error finding container e094773c81ce36f3545a1f1f6f31dae47db2220500ff97dace1daa35c16c3c72: Status 404 returned error can't find the container with id e094773c81ce36f3545a1f1f6f31dae47db2220500ff97dace1daa35c16c3c72 Dec 03 13:42:12 crc kubenswrapper[5052]: I1203 13:42:12.467210 5052 generic.go:334] "Generic (PLEG): container finished" podID="ec7ca777-d2e5-4b89-91ad-5084b2716811" containerID="92049bf81ac84c4ad82c46c284e670bf4586778ae001e3942eb6bb28c4b60821" exitCode=0 Dec 03 13:42:12 crc kubenswrapper[5052]: I1203 13:42:12.467304 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/49c083020ae5dfe237b73a6c6b807501660a323f061d7879268c43a121w2872" event={"ID":"ec7ca777-d2e5-4b89-91ad-5084b2716811","Type":"ContainerDied","Data":"92049bf81ac84c4ad82c46c284e670bf4586778ae001e3942eb6bb28c4b60821"} Dec 03 13:42:12 crc kubenswrapper[5052]: I1203 13:42:12.467643 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/49c083020ae5dfe237b73a6c6b807501660a323f061d7879268c43a121w2872" event={"ID":"ec7ca777-d2e5-4b89-91ad-5084b2716811","Type":"ContainerStarted","Data":"e094773c81ce36f3545a1f1f6f31dae47db2220500ff97dace1daa35c16c3c72"} Dec 03 13:42:12 crc kubenswrapper[5052]: I1203 13:42:12.469297 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/keystone-operator-index-dtmwp" event={"ID":"2273c0a7-907e-4e0c-b6bf-f193263d9db7","Type":"ContainerStarted","Data":"047dc47b555b7198544f9325a376136bd2cc325989f21b37c2d5d61d89f2b421"} Dec 03 13:42:12 crc kubenswrapper[5052]: I1203 13:42:12.508097 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-operators/keystone-operator-index-dtmwp" podStartSLOduration=1.9162663229999999 podStartE2EDuration="2.508074403s" podCreationTimestamp="2025-12-03 13:42:10 +0000 UTC" firstStartedPulling="2025-12-03 13:42:10.911805917 +0000 UTC m=+8521.500047671" lastFinishedPulling="2025-12-03 13:42:11.503613997 +0000 UTC m=+8522.091855751" observedRunningTime="2025-12-03 13:42:12.504890501 +0000 UTC m=+8523.093132275" watchObservedRunningTime="2025-12-03 13:42:12.508074403 +0000 UTC m=+8523.096316167" Dec 03 13:42:14 crc kubenswrapper[5052]: I1203 13:42:14.492577 5052 generic.go:334] "Generic (PLEG): container finished" podID="ec7ca777-d2e5-4b89-91ad-5084b2716811" containerID="5aaf8ba60f80237776fb309acc3491ff2ae50dd27b76fff4ff321dbcc6831e02" exitCode=0 Dec 03 13:42:14 crc kubenswrapper[5052]: I1203 13:42:14.492713 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/49c083020ae5dfe237b73a6c6b807501660a323f061d7879268c43a121w2872" event={"ID":"ec7ca777-d2e5-4b89-91ad-5084b2716811","Type":"ContainerDied","Data":"5aaf8ba60f80237776fb309acc3491ff2ae50dd27b76fff4ff321dbcc6831e02"} Dec 03 13:42:14 crc kubenswrapper[5052]: I1203 13:42:14.564030 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-operators/keystone-operator-index-dtmwp"] Dec 03 13:42:14 crc kubenswrapper[5052]: I1203 13:42:14.564231 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-operators/keystone-operator-index-dtmwp" podUID="2273c0a7-907e-4e0c-b6bf-f193263d9db7" containerName="registry-server" containerID="cri-o://047dc47b555b7198544f9325a376136bd2cc325989f21b37c2d5d61d89f2b421" gracePeriod=2 Dec 03 13:42:15 crc kubenswrapper[5052]: I1203 13:42:15.167324 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-operators/keystone-operator-index-5bhhd"] Dec 03 13:42:15 crc kubenswrapper[5052]: I1203 13:42:15.168764 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-operators/keystone-operator-index-5bhhd" Dec 03 13:42:15 crc kubenswrapper[5052]: I1203 13:42:15.175739 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-operators/keystone-operator-index-5bhhd"] Dec 03 13:42:15 crc kubenswrapper[5052]: I1203 13:42:15.367264 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9rhjt\" (UniqueName: \"kubernetes.io/projected/a38bb331-9bf3-48e9-bd95-7b5854cd4347-kube-api-access-9rhjt\") pod \"keystone-operator-index-5bhhd\" (UID: \"a38bb331-9bf3-48e9-bd95-7b5854cd4347\") " pod="openstack-operators/keystone-operator-index-5bhhd" Dec 03 13:42:15 crc kubenswrapper[5052]: I1203 13:42:15.469572 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-9rhjt\" (UniqueName: \"kubernetes.io/projected/a38bb331-9bf3-48e9-bd95-7b5854cd4347-kube-api-access-9rhjt\") pod \"keystone-operator-index-5bhhd\" (UID: \"a38bb331-9bf3-48e9-bd95-7b5854cd4347\") " pod="openstack-operators/keystone-operator-index-5bhhd" Dec 03 13:42:15 crc kubenswrapper[5052]: I1203 13:42:15.488426 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-9rhjt\" (UniqueName: \"kubernetes.io/projected/a38bb331-9bf3-48e9-bd95-7b5854cd4347-kube-api-access-9rhjt\") pod \"keystone-operator-index-5bhhd\" (UID: \"a38bb331-9bf3-48e9-bd95-7b5854cd4347\") " pod="openstack-operators/keystone-operator-index-5bhhd" Dec 03 13:42:15 crc kubenswrapper[5052]: I1203 13:42:15.504454 5052 generic.go:334] "Generic (PLEG): container finished" podID="ec7ca777-d2e5-4b89-91ad-5084b2716811" containerID="9989e568af0ffc2e95e8d5ddca5df126f8d6f9d4472fda82276b45c89dd4020f" exitCode=0 Dec 03 13:42:15 crc kubenswrapper[5052]: I1203 13:42:15.504526 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/49c083020ae5dfe237b73a6c6b807501660a323f061d7879268c43a121w2872" event={"ID":"ec7ca777-d2e5-4b89-91ad-5084b2716811","Type":"ContainerDied","Data":"9989e568af0ffc2e95e8d5ddca5df126f8d6f9d4472fda82276b45c89dd4020f"} Dec 03 13:42:15 crc kubenswrapper[5052]: I1203 13:42:15.506004 5052 generic.go:334] "Generic (PLEG): container finished" podID="2273c0a7-907e-4e0c-b6bf-f193263d9db7" containerID="047dc47b555b7198544f9325a376136bd2cc325989f21b37c2d5d61d89f2b421" exitCode=0 Dec 03 13:42:15 crc kubenswrapper[5052]: I1203 13:42:15.506024 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/keystone-operator-index-dtmwp" event={"ID":"2273c0a7-907e-4e0c-b6bf-f193263d9db7","Type":"ContainerDied","Data":"047dc47b555b7198544f9325a376136bd2cc325989f21b37c2d5d61d89f2b421"} Dec 03 13:42:15 crc kubenswrapper[5052]: I1203 13:42:15.506037 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/keystone-operator-index-dtmwp" event={"ID":"2273c0a7-907e-4e0c-b6bf-f193263d9db7","Type":"ContainerDied","Data":"e4e83015e6d431c38f68278b26cce99cab88dc458d7386fe65efcf1a06ef0620"} Dec 03 13:42:15 crc kubenswrapper[5052]: I1203 13:42:15.506046 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e4e83015e6d431c38f68278b26cce99cab88dc458d7386fe65efcf1a06ef0620" Dec 03 13:42:15 crc kubenswrapper[5052]: I1203 13:42:15.526491 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-operators/keystone-operator-index-dtmwp" Dec 03 13:42:15 crc kubenswrapper[5052]: I1203 13:42:15.570734 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2pqnh\" (UniqueName: \"kubernetes.io/projected/2273c0a7-907e-4e0c-b6bf-f193263d9db7-kube-api-access-2pqnh\") pod \"2273c0a7-907e-4e0c-b6bf-f193263d9db7\" (UID: \"2273c0a7-907e-4e0c-b6bf-f193263d9db7\") " Dec 03 13:42:15 crc kubenswrapper[5052]: I1203 13:42:15.574139 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2273c0a7-907e-4e0c-b6bf-f193263d9db7-kube-api-access-2pqnh" (OuterVolumeSpecName: "kube-api-access-2pqnh") pod "2273c0a7-907e-4e0c-b6bf-f193263d9db7" (UID: "2273c0a7-907e-4e0c-b6bf-f193263d9db7"). InnerVolumeSpecName "kube-api-access-2pqnh". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:42:15 crc kubenswrapper[5052]: I1203 13:42:15.673361 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-2pqnh\" (UniqueName: \"kubernetes.io/projected/2273c0a7-907e-4e0c-b6bf-f193263d9db7-kube-api-access-2pqnh\") on node \"crc\" DevicePath \"\"" Dec 03 13:42:15 crc kubenswrapper[5052]: I1203 13:42:15.784925 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-operators/keystone-operator-index-5bhhd" Dec 03 13:42:16 crc kubenswrapper[5052]: I1203 13:42:16.290693 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-operators/keystone-operator-index-5bhhd"] Dec 03 13:42:16 crc kubenswrapper[5052]: I1203 13:42:16.530415 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/keystone-operator-index-5bhhd" event={"ID":"a38bb331-9bf3-48e9-bd95-7b5854cd4347","Type":"ContainerStarted","Data":"d7c08e0b52997e0a5f907a57b87153c5f78b1f164f5e54ef61ca26f31ef11753"} Dec 03 13:42:16 crc kubenswrapper[5052]: I1203 13:42:16.530682 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-operators/keystone-operator-index-dtmwp" Dec 03 13:42:16 crc kubenswrapper[5052]: I1203 13:42:16.556919 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-operators/keystone-operator-index-dtmwp"] Dec 03 13:42:16 crc kubenswrapper[5052]: I1203 13:42:16.563855 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-operators/keystone-operator-index-dtmwp"] Dec 03 13:42:16 crc kubenswrapper[5052]: I1203 13:42:16.788899 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-operators/49c083020ae5dfe237b73a6c6b807501660a323f061d7879268c43a121w2872" Dec 03 13:42:16 crc kubenswrapper[5052]: I1203 13:42:16.916617 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ec7ca777-d2e5-4b89-91ad-5084b2716811-bundle\") pod \"ec7ca777-d2e5-4b89-91ad-5084b2716811\" (UID: \"ec7ca777-d2e5-4b89-91ad-5084b2716811\") " Dec 03 13:42:16 crc kubenswrapper[5052]: I1203 13:42:16.916714 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ec7ca777-d2e5-4b89-91ad-5084b2716811-util\") pod \"ec7ca777-d2e5-4b89-91ad-5084b2716811\" (UID: \"ec7ca777-d2e5-4b89-91ad-5084b2716811\") " Dec 03 13:42:16 crc kubenswrapper[5052]: I1203 13:42:16.916744 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gnmpt\" (UniqueName: \"kubernetes.io/projected/ec7ca777-d2e5-4b89-91ad-5084b2716811-kube-api-access-gnmpt\") pod \"ec7ca777-d2e5-4b89-91ad-5084b2716811\" (UID: \"ec7ca777-d2e5-4b89-91ad-5084b2716811\") " Dec 03 13:42:16 crc kubenswrapper[5052]: I1203 13:42:16.917586 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec7ca777-d2e5-4b89-91ad-5084b2716811-bundle" (OuterVolumeSpecName: "bundle") pod "ec7ca777-d2e5-4b89-91ad-5084b2716811" (UID: "ec7ca777-d2e5-4b89-91ad-5084b2716811"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 13:42:16 crc kubenswrapper[5052]: I1203 13:42:16.924434 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ec7ca777-d2e5-4b89-91ad-5084b2716811-kube-api-access-gnmpt" (OuterVolumeSpecName: "kube-api-access-gnmpt") pod "ec7ca777-d2e5-4b89-91ad-5084b2716811" (UID: "ec7ca777-d2e5-4b89-91ad-5084b2716811"). InnerVolumeSpecName "kube-api-access-gnmpt". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:42:16 crc kubenswrapper[5052]: I1203 13:42:16.936161 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec7ca777-d2e5-4b89-91ad-5084b2716811-util" (OuterVolumeSpecName: "util") pod "ec7ca777-d2e5-4b89-91ad-5084b2716811" (UID: "ec7ca777-d2e5-4b89-91ad-5084b2716811"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 13:42:17 crc kubenswrapper[5052]: I1203 13:42:17.018706 5052 reconciler_common.go:293] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ec7ca777-d2e5-4b89-91ad-5084b2716811-util\") on node \"crc\" DevicePath \"\"" Dec 03 13:42:17 crc kubenswrapper[5052]: I1203 13:42:17.018775 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-gnmpt\" (UniqueName: \"kubernetes.io/projected/ec7ca777-d2e5-4b89-91ad-5084b2716811-kube-api-access-gnmpt\") on node \"crc\" DevicePath \"\"" Dec 03 13:42:17 crc kubenswrapper[5052]: I1203 13:42:17.018800 5052 reconciler_common.go:293] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ec7ca777-d2e5-4b89-91ad-5084b2716811-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 13:42:17 crc kubenswrapper[5052]: I1203 13:42:17.540903 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/49c083020ae5dfe237b73a6c6b807501660a323f061d7879268c43a121w2872" event={"ID":"ec7ca777-d2e5-4b89-91ad-5084b2716811","Type":"ContainerDied","Data":"e094773c81ce36f3545a1f1f6f31dae47db2220500ff97dace1daa35c16c3c72"} Dec 03 13:42:17 crc kubenswrapper[5052]: I1203 13:42:17.540953 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e094773c81ce36f3545a1f1f6f31dae47db2220500ff97dace1daa35c16c3c72" Dec 03 13:42:17 crc kubenswrapper[5052]: I1203 13:42:17.541024 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-operators/49c083020ae5dfe237b73a6c6b807501660a323f061d7879268c43a121w2872" Dec 03 13:42:17 crc kubenswrapper[5052]: I1203 13:42:17.680360 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2273c0a7-907e-4e0c-b6bf-f193263d9db7" path="/var/lib/kubelet/pods/2273c0a7-907e-4e0c-b6bf-f193263d9db7/volumes" Dec 03 13:42:18 crc kubenswrapper[5052]: I1203 13:42:18.549292 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/keystone-operator-index-5bhhd" event={"ID":"a38bb331-9bf3-48e9-bd95-7b5854cd4347","Type":"ContainerStarted","Data":"7fff1ae9198e592a285110ae6f3947b827987f4d0dd31a100ca6a586e63dd7a3"} Dec 03 13:42:18 crc kubenswrapper[5052]: I1203 13:42:18.566537 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-operators/keystone-operator-index-5bhhd" podStartSLOduration=2.423357206 podStartE2EDuration="3.566499248s" podCreationTimestamp="2025-12-03 13:42:15 +0000 UTC" firstStartedPulling="2025-12-03 13:42:16.291049395 +0000 UTC m=+8526.879291159" lastFinishedPulling="2025-12-03 13:42:17.434191437 +0000 UTC m=+8528.022433201" observedRunningTime="2025-12-03 13:42:18.563453881 +0000 UTC m=+8529.151695645" watchObservedRunningTime="2025-12-03 13:42:18.566499248 +0000 UTC m=+8529.154741012" Dec 03 13:42:20 crc kubenswrapper[5052]: I1203 13:42:20.304292 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["horizon-kuttl-tests/rabbitmq-server-0"] Dec 03 13:42:20 crc kubenswrapper[5052]: E1203 13:42:20.304900 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="ec7ca777-d2e5-4b89-91ad-5084b2716811" containerName="extract" Dec 03 13:42:20 crc kubenswrapper[5052]: I1203 13:42:20.304913 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="ec7ca777-d2e5-4b89-91ad-5084b2716811" containerName="extract" Dec 03 13:42:20 crc kubenswrapper[5052]: E1203 13:42:20.304933 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="ec7ca777-d2e5-4b89-91ad-5084b2716811" containerName="pull" Dec 03 13:42:20 crc kubenswrapper[5052]: I1203 13:42:20.304941 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="ec7ca777-d2e5-4b89-91ad-5084b2716811" containerName="pull" Dec 03 13:42:20 crc kubenswrapper[5052]: E1203 13:42:20.304952 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="2273c0a7-907e-4e0c-b6bf-f193263d9db7" containerName="registry-server" Dec 03 13:42:20 crc kubenswrapper[5052]: I1203 13:42:20.304959 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="2273c0a7-907e-4e0c-b6bf-f193263d9db7" containerName="registry-server" Dec 03 13:42:20 crc kubenswrapper[5052]: E1203 13:42:20.305008 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="ec7ca777-d2e5-4b89-91ad-5084b2716811" containerName="util" Dec 03 13:42:20 crc kubenswrapper[5052]: I1203 13:42:20.305016 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="ec7ca777-d2e5-4b89-91ad-5084b2716811" containerName="util" Dec 03 13:42:20 crc kubenswrapper[5052]: I1203 13:42:20.305143 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="2273c0a7-907e-4e0c-b6bf-f193263d9db7" containerName="registry-server" Dec 03 13:42:20 crc kubenswrapper[5052]: I1203 13:42:20.305167 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="ec7ca777-d2e5-4b89-91ad-5084b2716811" containerName="extract" Dec 03 13:42:20 crc kubenswrapper[5052]: I1203 13:42:20.305966 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="horizon-kuttl-tests/rabbitmq-server-0" Dec 03 13:42:20 crc kubenswrapper[5052]: I1203 13:42:20.308229 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"horizon-kuttl-tests"/"rabbitmq-server-conf" Dec 03 13:42:20 crc kubenswrapper[5052]: I1203 13:42:20.308296 5052 reflector.go:368] Caches populated for *v1.Secret from object-"horizon-kuttl-tests"/"rabbitmq-default-user" Dec 03 13:42:20 crc kubenswrapper[5052]: I1203 13:42:20.308432 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"horizon-kuttl-tests"/"rabbitmq-plugins-conf" Dec 03 13:42:20 crc kubenswrapper[5052]: I1203 13:42:20.308537 5052 reflector.go:368] Caches populated for *v1.Secret from object-"horizon-kuttl-tests"/"rabbitmq-server-dockercfg-nvnjc" Dec 03 13:42:20 crc kubenswrapper[5052]: I1203 13:42:20.309684 5052 reflector.go:368] Caches populated for *v1.Secret from object-"horizon-kuttl-tests"/"rabbitmq-erlang-cookie" Dec 03 13:42:20 crc kubenswrapper[5052]: I1203 13:42:20.326418 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["horizon-kuttl-tests/rabbitmq-server-0"] Dec 03 13:42:20 crc kubenswrapper[5052]: I1203 13:42:20.471981 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lwvhx\" (UniqueName: \"kubernetes.io/projected/aa07e692-9f86-4c61-b674-beeb49376c0e-kube-api-access-lwvhx\") pod \"rabbitmq-server-0\" (UID: \"aa07e692-9f86-4c61-b674-beeb49376c0e\") " pod="horizon-kuttl-tests/rabbitmq-server-0" Dec 03 13:42:20 crc kubenswrapper[5052]: I1203 13:42:20.472037 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"erlang-cookie-secret\" (UniqueName: \"kubernetes.io/secret/aa07e692-9f86-4c61-b674-beeb49376c0e-erlang-cookie-secret\") pod \"rabbitmq-server-0\" (UID: \"aa07e692-9f86-4c61-b674-beeb49376c0e\") " pod="horizon-kuttl-tests/rabbitmq-server-0" Dec 03 13:42:20 crc kubenswrapper[5052]: I1203 13:42:20.472088 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"pvc-64073964-8794-4f6e-ae70-f5b8cada0665\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-64073964-8794-4f6e-ae70-f5b8cada0665\") pod \"rabbitmq-server-0\" (UID: \"aa07e692-9f86-4c61-b674-beeb49376c0e\") " pod="horizon-kuttl-tests/rabbitmq-server-0" Dec 03 13:42:20 crc kubenswrapper[5052]: I1203 13:42:20.472124 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"rabbitmq-erlang-cookie\" (UniqueName: \"kubernetes.io/empty-dir/aa07e692-9f86-4c61-b674-beeb49376c0e-rabbitmq-erlang-cookie\") pod \"rabbitmq-server-0\" (UID: \"aa07e692-9f86-4c61-b674-beeb49376c0e\") " pod="horizon-kuttl-tests/rabbitmq-server-0" Dec 03 13:42:20 crc kubenswrapper[5052]: I1203 13:42:20.472327 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"rabbitmq-confd\" (UniqueName: \"kubernetes.io/projected/aa07e692-9f86-4c61-b674-beeb49376c0e-rabbitmq-confd\") pod \"rabbitmq-server-0\" (UID: \"aa07e692-9f86-4c61-b674-beeb49376c0e\") " pod="horizon-kuttl-tests/rabbitmq-server-0" Dec 03 13:42:20 crc kubenswrapper[5052]: I1203 13:42:20.472564 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"pod-info\" (UniqueName: \"kubernetes.io/downward-api/aa07e692-9f86-4c61-b674-beeb49376c0e-pod-info\") pod \"rabbitmq-server-0\" (UID: \"aa07e692-9f86-4c61-b674-beeb49376c0e\") " pod="horizon-kuttl-tests/rabbitmq-server-0" Dec 03 13:42:20 crc kubenswrapper[5052]: I1203 13:42:20.472597 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"rabbitmq-plugins\" (UniqueName: \"kubernetes.io/empty-dir/aa07e692-9f86-4c61-b674-beeb49376c0e-rabbitmq-plugins\") pod \"rabbitmq-server-0\" (UID: \"aa07e692-9f86-4c61-b674-beeb49376c0e\") " pod="horizon-kuttl-tests/rabbitmq-server-0" Dec 03 13:42:20 crc kubenswrapper[5052]: I1203 13:42:20.472644 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-conf\" (UniqueName: \"kubernetes.io/configmap/aa07e692-9f86-4c61-b674-beeb49376c0e-plugins-conf\") pod \"rabbitmq-server-0\" (UID: \"aa07e692-9f86-4c61-b674-beeb49376c0e\") " pod="horizon-kuttl-tests/rabbitmq-server-0" Dec 03 13:42:20 crc kubenswrapper[5052]: I1203 13:42:20.573911 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"pod-info\" (UniqueName: \"kubernetes.io/downward-api/aa07e692-9f86-4c61-b674-beeb49376c0e-pod-info\") pod \"rabbitmq-server-0\" (UID: \"aa07e692-9f86-4c61-b674-beeb49376c0e\") " pod="horizon-kuttl-tests/rabbitmq-server-0" Dec 03 13:42:20 crc kubenswrapper[5052]: I1203 13:42:20.574004 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"rabbitmq-plugins\" (UniqueName: \"kubernetes.io/empty-dir/aa07e692-9f86-4c61-b674-beeb49376c0e-rabbitmq-plugins\") pod \"rabbitmq-server-0\" (UID: \"aa07e692-9f86-4c61-b674-beeb49376c0e\") " pod="horizon-kuttl-tests/rabbitmq-server-0" Dec 03 13:42:20 crc kubenswrapper[5052]: I1203 13:42:20.574050 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"plugins-conf\" (UniqueName: \"kubernetes.io/configmap/aa07e692-9f86-4c61-b674-beeb49376c0e-plugins-conf\") pod \"rabbitmq-server-0\" (UID: \"aa07e692-9f86-4c61-b674-beeb49376c0e\") " pod="horizon-kuttl-tests/rabbitmq-server-0" Dec 03 13:42:20 crc kubenswrapper[5052]: I1203 13:42:20.574104 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-lwvhx\" (UniqueName: \"kubernetes.io/projected/aa07e692-9f86-4c61-b674-beeb49376c0e-kube-api-access-lwvhx\") pod \"rabbitmq-server-0\" (UID: \"aa07e692-9f86-4c61-b674-beeb49376c0e\") " pod="horizon-kuttl-tests/rabbitmq-server-0" Dec 03 13:42:20 crc kubenswrapper[5052]: I1203 13:42:20.574129 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"erlang-cookie-secret\" (UniqueName: \"kubernetes.io/secret/aa07e692-9f86-4c61-b674-beeb49376c0e-erlang-cookie-secret\") pod \"rabbitmq-server-0\" (UID: \"aa07e692-9f86-4c61-b674-beeb49376c0e\") " pod="horizon-kuttl-tests/rabbitmq-server-0" Dec 03 13:42:20 crc kubenswrapper[5052]: I1203 13:42:20.574173 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"pvc-64073964-8794-4f6e-ae70-f5b8cada0665\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-64073964-8794-4f6e-ae70-f5b8cada0665\") pod \"rabbitmq-server-0\" (UID: \"aa07e692-9f86-4c61-b674-beeb49376c0e\") " pod="horizon-kuttl-tests/rabbitmq-server-0" Dec 03 13:42:20 crc kubenswrapper[5052]: I1203 13:42:20.574215 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"rabbitmq-erlang-cookie\" (UniqueName: \"kubernetes.io/empty-dir/aa07e692-9f86-4c61-b674-beeb49376c0e-rabbitmq-erlang-cookie\") pod \"rabbitmq-server-0\" (UID: \"aa07e692-9f86-4c61-b674-beeb49376c0e\") " pod="horizon-kuttl-tests/rabbitmq-server-0" Dec 03 13:42:20 crc kubenswrapper[5052]: I1203 13:42:20.574263 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"rabbitmq-confd\" (UniqueName: \"kubernetes.io/projected/aa07e692-9f86-4c61-b674-beeb49376c0e-rabbitmq-confd\") pod \"rabbitmq-server-0\" (UID: \"aa07e692-9f86-4c61-b674-beeb49376c0e\") " pod="horizon-kuttl-tests/rabbitmq-server-0" Dec 03 13:42:20 crc kubenswrapper[5052]: I1203 13:42:20.574712 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"rabbitmq-plugins\" (UniqueName: \"kubernetes.io/empty-dir/aa07e692-9f86-4c61-b674-beeb49376c0e-rabbitmq-plugins\") pod \"rabbitmq-server-0\" (UID: \"aa07e692-9f86-4c61-b674-beeb49376c0e\") " pod="horizon-kuttl-tests/rabbitmq-server-0" Dec 03 13:42:20 crc kubenswrapper[5052]: I1203 13:42:20.574985 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"rabbitmq-erlang-cookie\" (UniqueName: \"kubernetes.io/empty-dir/aa07e692-9f86-4c61-b674-beeb49376c0e-rabbitmq-erlang-cookie\") pod \"rabbitmq-server-0\" (UID: \"aa07e692-9f86-4c61-b674-beeb49376c0e\") " pod="horizon-kuttl-tests/rabbitmq-server-0" Dec 03 13:42:20 crc kubenswrapper[5052]: I1203 13:42:20.575623 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"plugins-conf\" (UniqueName: \"kubernetes.io/configmap/aa07e692-9f86-4c61-b674-beeb49376c0e-plugins-conf\") pod \"rabbitmq-server-0\" (UID: \"aa07e692-9f86-4c61-b674-beeb49376c0e\") " pod="horizon-kuttl-tests/rabbitmq-server-0" Dec 03 13:42:20 crc kubenswrapper[5052]: I1203 13:42:20.578108 5052 csi_attacher.go:380] kubernetes.io/csi: attacher.MountDevice STAGE_UNSTAGE_VOLUME capability not set. Skipping MountDevice... Dec 03 13:42:20 crc kubenswrapper[5052]: I1203 13:42:20.578160 5052 operation_generator.go:580] "MountVolume.MountDevice succeeded for volume \"pvc-64073964-8794-4f6e-ae70-f5b8cada0665\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-64073964-8794-4f6e-ae70-f5b8cada0665\") pod \"rabbitmq-server-0\" (UID: \"aa07e692-9f86-4c61-b674-beeb49376c0e\") device mount path \"/var/lib/kubelet/plugins/kubernetes.io/csi/kubevirt.io.hostpath-provisioner/7f4f4d43906f2e590f824bc1f5703b423c74bca7002a278f2fbd802b0e0393e0/globalmount\"" pod="horizon-kuttl-tests/rabbitmq-server-0" Dec 03 13:42:20 crc kubenswrapper[5052]: I1203 13:42:20.579798 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"pod-info\" (UniqueName: \"kubernetes.io/downward-api/aa07e692-9f86-4c61-b674-beeb49376c0e-pod-info\") pod \"rabbitmq-server-0\" (UID: \"aa07e692-9f86-4c61-b674-beeb49376c0e\") " pod="horizon-kuttl-tests/rabbitmq-server-0" Dec 03 13:42:20 crc kubenswrapper[5052]: I1203 13:42:20.579801 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"erlang-cookie-secret\" (UniqueName: \"kubernetes.io/secret/aa07e692-9f86-4c61-b674-beeb49376c0e-erlang-cookie-secret\") pod \"rabbitmq-server-0\" (UID: \"aa07e692-9f86-4c61-b674-beeb49376c0e\") " pod="horizon-kuttl-tests/rabbitmq-server-0" Dec 03 13:42:20 crc kubenswrapper[5052]: I1203 13:42:20.594613 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"rabbitmq-confd\" (UniqueName: \"kubernetes.io/projected/aa07e692-9f86-4c61-b674-beeb49376c0e-rabbitmq-confd\") pod \"rabbitmq-server-0\" (UID: \"aa07e692-9f86-4c61-b674-beeb49376c0e\") " pod="horizon-kuttl-tests/rabbitmq-server-0" Dec 03 13:42:20 crc kubenswrapper[5052]: I1203 13:42:20.598567 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-lwvhx\" (UniqueName: \"kubernetes.io/projected/aa07e692-9f86-4c61-b674-beeb49376c0e-kube-api-access-lwvhx\") pod \"rabbitmq-server-0\" (UID: \"aa07e692-9f86-4c61-b674-beeb49376c0e\") " pod="horizon-kuttl-tests/rabbitmq-server-0" Dec 03 13:42:20 crc kubenswrapper[5052]: I1203 13:42:20.608730 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"pvc-64073964-8794-4f6e-ae70-f5b8cada0665\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-64073964-8794-4f6e-ae70-f5b8cada0665\") pod \"rabbitmq-server-0\" (UID: \"aa07e692-9f86-4c61-b674-beeb49376c0e\") " pod="horizon-kuttl-tests/rabbitmq-server-0" Dec 03 13:42:20 crc kubenswrapper[5052]: I1203 13:42:20.625584 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="horizon-kuttl-tests/rabbitmq-server-0" Dec 03 13:42:21 crc kubenswrapper[5052]: I1203 13:42:21.057891 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["horizon-kuttl-tests/rabbitmq-server-0"] Dec 03 13:42:21 crc kubenswrapper[5052]: W1203 13:42:21.070269 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa07e692_9f86_4c61_b674_beeb49376c0e.slice/crio-eb8a2577efef8333de462a2cd8da81a0d97dbe335d9a29d4675d151e5cd977cc WatchSource:0}: Error finding container eb8a2577efef8333de462a2cd8da81a0d97dbe335d9a29d4675d151e5cd977cc: Status 404 returned error can't find the container with id eb8a2577efef8333de462a2cd8da81a0d97dbe335d9a29d4675d151e5cd977cc Dec 03 13:42:21 crc kubenswrapper[5052]: I1203 13:42:21.570349 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="horizon-kuttl-tests/rabbitmq-server-0" event={"ID":"aa07e692-9f86-4c61-b674-beeb49376c0e","Type":"ContainerStarted","Data":"eb8a2577efef8333de462a2cd8da81a0d97dbe335d9a29d4675d151e5cd977cc"} Dec 03 13:42:22 crc kubenswrapper[5052]: I1203 13:42:22.581462 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="horizon-kuttl-tests/rabbitmq-server-0" event={"ID":"aa07e692-9f86-4c61-b674-beeb49376c0e","Type":"ContainerStarted","Data":"29a5cf76d0275587ac82e2efe006e49acba2897de049585dae86807c72ae9434"} Dec 03 13:42:23 crc kubenswrapper[5052]: I1203 13:42:23.673680 5052 scope.go:117] "RemoveContainer" containerID="2fbd293d6edef0fa059e91c119312e77b262534107c322feb7b59dbea3464526" Dec 03 13:42:23 crc kubenswrapper[5052]: E1203 13:42:23.674409 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 13:42:25 crc kubenswrapper[5052]: I1203 13:42:25.786182 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-operators/keystone-operator-index-5bhhd" Dec 03 13:42:25 crc kubenswrapper[5052]: I1203 13:42:25.786506 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-operators/keystone-operator-index-5bhhd" Dec 03 13:42:25 crc kubenswrapper[5052]: I1203 13:42:25.860658 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-operators/keystone-operator-index-5bhhd" Dec 03 13:42:26 crc kubenswrapper[5052]: I1203 13:42:26.637329 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-operators/keystone-operator-index-5bhhd" Dec 03 13:42:28 crc kubenswrapper[5052]: I1203 13:42:28.808949 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-operators/keystone-operator-controller-manager-768697f4bc-42dw6"] Dec 03 13:42:28 crc kubenswrapper[5052]: I1203 13:42:28.810410 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-operators/keystone-operator-controller-manager-768697f4bc-42dw6" Dec 03 13:42:28 crc kubenswrapper[5052]: I1203 13:42:28.813641 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-operators"/"keystone-operator-controller-manager-service-cert" Dec 03 13:42:28 crc kubenswrapper[5052]: I1203 13:42:28.814159 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-operators"/"keystone-operator-controller-manager-dockercfg-xh2vc" Dec 03 13:42:28 crc kubenswrapper[5052]: I1203 13:42:28.824334 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-operators/keystone-operator-controller-manager-768697f4bc-42dw6"] Dec 03 13:42:28 crc kubenswrapper[5052]: I1203 13:42:28.995614 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2be98578-12f4-41ef-9fda-5db102aa4702-webhook-cert\") pod \"keystone-operator-controller-manager-768697f4bc-42dw6\" (UID: \"2be98578-12f4-41ef-9fda-5db102aa4702\") " pod="openstack-operators/keystone-operator-controller-manager-768697f4bc-42dw6" Dec 03 13:42:28 crc kubenswrapper[5052]: I1203 13:42:28.995864 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/2be98578-12f4-41ef-9fda-5db102aa4702-apiservice-cert\") pod \"keystone-operator-controller-manager-768697f4bc-42dw6\" (UID: \"2be98578-12f4-41ef-9fda-5db102aa4702\") " pod="openstack-operators/keystone-operator-controller-manager-768697f4bc-42dw6" Dec 03 13:42:28 crc kubenswrapper[5052]: I1203 13:42:28.996001 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-st72c\" (UniqueName: \"kubernetes.io/projected/2be98578-12f4-41ef-9fda-5db102aa4702-kube-api-access-st72c\") pod \"keystone-operator-controller-manager-768697f4bc-42dw6\" (UID: \"2be98578-12f4-41ef-9fda-5db102aa4702\") " pod="openstack-operators/keystone-operator-controller-manager-768697f4bc-42dw6" Dec 03 13:42:29 crc kubenswrapper[5052]: I1203 13:42:29.097791 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2be98578-12f4-41ef-9fda-5db102aa4702-webhook-cert\") pod \"keystone-operator-controller-manager-768697f4bc-42dw6\" (UID: \"2be98578-12f4-41ef-9fda-5db102aa4702\") " pod="openstack-operators/keystone-operator-controller-manager-768697f4bc-42dw6" Dec 03 13:42:29 crc kubenswrapper[5052]: I1203 13:42:29.098254 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/2be98578-12f4-41ef-9fda-5db102aa4702-apiservice-cert\") pod \"keystone-operator-controller-manager-768697f4bc-42dw6\" (UID: \"2be98578-12f4-41ef-9fda-5db102aa4702\") " pod="openstack-operators/keystone-operator-controller-manager-768697f4bc-42dw6" Dec 03 13:42:29 crc kubenswrapper[5052]: I1203 13:42:29.099113 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-st72c\" (UniqueName: \"kubernetes.io/projected/2be98578-12f4-41ef-9fda-5db102aa4702-kube-api-access-st72c\") pod \"keystone-operator-controller-manager-768697f4bc-42dw6\" (UID: \"2be98578-12f4-41ef-9fda-5db102aa4702\") " pod="openstack-operators/keystone-operator-controller-manager-768697f4bc-42dw6" Dec 03 13:42:29 crc kubenswrapper[5052]: I1203 13:42:29.104578 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/2be98578-12f4-41ef-9fda-5db102aa4702-apiservice-cert\") pod \"keystone-operator-controller-manager-768697f4bc-42dw6\" (UID: \"2be98578-12f4-41ef-9fda-5db102aa4702\") " pod="openstack-operators/keystone-operator-controller-manager-768697f4bc-42dw6" Dec 03 13:42:29 crc kubenswrapper[5052]: I1203 13:42:29.104629 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2be98578-12f4-41ef-9fda-5db102aa4702-webhook-cert\") pod \"keystone-operator-controller-manager-768697f4bc-42dw6\" (UID: \"2be98578-12f4-41ef-9fda-5db102aa4702\") " pod="openstack-operators/keystone-operator-controller-manager-768697f4bc-42dw6" Dec 03 13:42:29 crc kubenswrapper[5052]: I1203 13:42:29.116693 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-st72c\" (UniqueName: \"kubernetes.io/projected/2be98578-12f4-41ef-9fda-5db102aa4702-kube-api-access-st72c\") pod \"keystone-operator-controller-manager-768697f4bc-42dw6\" (UID: \"2be98578-12f4-41ef-9fda-5db102aa4702\") " pod="openstack-operators/keystone-operator-controller-manager-768697f4bc-42dw6" Dec 03 13:42:29 crc kubenswrapper[5052]: I1203 13:42:29.136012 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-operators/keystone-operator-controller-manager-768697f4bc-42dw6" Dec 03 13:42:29 crc kubenswrapper[5052]: I1203 13:42:29.590874 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-operators/keystone-operator-controller-manager-768697f4bc-42dw6"] Dec 03 13:42:29 crc kubenswrapper[5052]: I1203 13:42:29.630529 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/keystone-operator-controller-manager-768697f4bc-42dw6" event={"ID":"2be98578-12f4-41ef-9fda-5db102aa4702","Type":"ContainerStarted","Data":"91ddede55c4fcc50b2e9a70d51ebfaf3e4da5f134cdda6c3111c8e343746c7bf"} Dec 03 13:42:30 crc kubenswrapper[5052]: I1203 13:42:30.642299 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/keystone-operator-controller-manager-768697f4bc-42dw6" event={"ID":"2be98578-12f4-41ef-9fda-5db102aa4702","Type":"ContainerStarted","Data":"fab7384a38f41af6c4ea1d6ba58502b0c3b9a330de3959baccf6fda8802c67af"} Dec 03 13:42:30 crc kubenswrapper[5052]: I1203 13:42:30.642966 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-operators/keystone-operator-controller-manager-768697f4bc-42dw6" Dec 03 13:42:30 crc kubenswrapper[5052]: I1203 13:42:30.671911 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-operators/keystone-operator-controller-manager-768697f4bc-42dw6" podStartSLOduration=2.671873292 podStartE2EDuration="2.671873292s" podCreationTimestamp="2025-12-03 13:42:28 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 13:42:30.664270934 +0000 UTC m=+8541.252512698" watchObservedRunningTime="2025-12-03 13:42:30.671873292 +0000 UTC m=+8541.260115056" Dec 03 13:42:34 crc kubenswrapper[5052]: I1203 13:42:34.672834 5052 scope.go:117] "RemoveContainer" containerID="2fbd293d6edef0fa059e91c119312e77b262534107c322feb7b59dbea3464526" Dec 03 13:42:34 crc kubenswrapper[5052]: E1203 13:42:34.673331 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 13:42:39 crc kubenswrapper[5052]: I1203 13:42:39.143134 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-operators/keystone-operator-controller-manager-768697f4bc-42dw6" Dec 03 13:42:42 crc kubenswrapper[5052]: I1203 13:42:42.973202 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-operators/horizon-operator-index-d4wxq"] Dec 03 13:42:42 crc kubenswrapper[5052]: I1203 13:42:42.975037 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-operators/horizon-operator-index-d4wxq" Dec 03 13:42:42 crc kubenswrapper[5052]: I1203 13:42:42.977976 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-operators"/"horizon-operator-index-dockercfg-dqtmn" Dec 03 13:42:42 crc kubenswrapper[5052]: I1203 13:42:42.985629 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-operators/horizon-operator-index-d4wxq"] Dec 03 13:42:43 crc kubenswrapper[5052]: I1203 13:42:43.008621 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7xbks\" (UniqueName: \"kubernetes.io/projected/01966e73-42ef-439a-8549-1ab114dda929-kube-api-access-7xbks\") pod \"horizon-operator-index-d4wxq\" (UID: \"01966e73-42ef-439a-8549-1ab114dda929\") " pod="openstack-operators/horizon-operator-index-d4wxq" Dec 03 13:42:43 crc kubenswrapper[5052]: I1203 13:42:43.109830 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-7xbks\" (UniqueName: \"kubernetes.io/projected/01966e73-42ef-439a-8549-1ab114dda929-kube-api-access-7xbks\") pod \"horizon-operator-index-d4wxq\" (UID: \"01966e73-42ef-439a-8549-1ab114dda929\") " pod="openstack-operators/horizon-operator-index-d4wxq" Dec 03 13:42:43 crc kubenswrapper[5052]: I1203 13:42:43.128316 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-7xbks\" (UniqueName: \"kubernetes.io/projected/01966e73-42ef-439a-8549-1ab114dda929-kube-api-access-7xbks\") pod \"horizon-operator-index-d4wxq\" (UID: \"01966e73-42ef-439a-8549-1ab114dda929\") " pod="openstack-operators/horizon-operator-index-d4wxq" Dec 03 13:42:43 crc kubenswrapper[5052]: I1203 13:42:43.292864 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-operators/horizon-operator-index-d4wxq" Dec 03 13:42:43 crc kubenswrapper[5052]: I1203 13:42:43.724111 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-operators/horizon-operator-index-d4wxq"] Dec 03 13:42:43 crc kubenswrapper[5052]: I1203 13:42:43.746418 5052 provider.go:102] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Dec 03 13:42:44 crc kubenswrapper[5052]: I1203 13:42:44.755621 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/horizon-operator-index-d4wxq" event={"ID":"01966e73-42ef-439a-8549-1ab114dda929","Type":"ContainerStarted","Data":"0e9e4a44f4377c0a78a0f342e67d5b6edb55c39891e99dc150306bf6e551b87c"} Dec 03 13:42:44 crc kubenswrapper[5052]: I1203 13:42:44.756227 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/horizon-operator-index-d4wxq" event={"ID":"01966e73-42ef-439a-8549-1ab114dda929","Type":"ContainerStarted","Data":"7d49ee4bbba2a44c3372284f3bc77b3fef528d2735cb10d96ae26b43af36ef18"} Dec 03 13:42:44 crc kubenswrapper[5052]: I1203 13:42:44.772464 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-operators/horizon-operator-index-d4wxq" podStartSLOduration=1.950565114 podStartE2EDuration="2.772449603s" podCreationTimestamp="2025-12-03 13:42:42 +0000 UTC" firstStartedPulling="2025-12-03 13:42:43.74608956 +0000 UTC m=+8554.334331324" lastFinishedPulling="2025-12-03 13:42:44.567974049 +0000 UTC m=+8555.156215813" observedRunningTime="2025-12-03 13:42:44.7713 +0000 UTC m=+8555.359541754" watchObservedRunningTime="2025-12-03 13:42:44.772449603 +0000 UTC m=+8555.360691367" Dec 03 13:42:46 crc kubenswrapper[5052]: I1203 13:42:46.141945 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["horizon-kuttl-tests/keystone-db-create-c58sf"] Dec 03 13:42:46 crc kubenswrapper[5052]: I1203 13:42:46.143081 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="horizon-kuttl-tests/keystone-db-create-c58sf" Dec 03 13:42:46 crc kubenswrapper[5052]: I1203 13:42:46.152901 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tms7h\" (UniqueName: \"kubernetes.io/projected/e500bdc5-8e72-481a-8e8c-177ddfb97582-kube-api-access-tms7h\") pod \"keystone-db-create-c58sf\" (UID: \"e500bdc5-8e72-481a-8e8c-177ddfb97582\") " pod="horizon-kuttl-tests/keystone-db-create-c58sf" Dec 03 13:42:46 crc kubenswrapper[5052]: I1203 13:42:46.152960 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/e500bdc5-8e72-481a-8e8c-177ddfb97582-operator-scripts\") pod \"keystone-db-create-c58sf\" (UID: \"e500bdc5-8e72-481a-8e8c-177ddfb97582\") " pod="horizon-kuttl-tests/keystone-db-create-c58sf" Dec 03 13:42:46 crc kubenswrapper[5052]: I1203 13:42:46.168692 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["horizon-kuttl-tests/keystone-db-create-c58sf"] Dec 03 13:42:46 crc kubenswrapper[5052]: I1203 13:42:46.179077 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["horizon-kuttl-tests/keystone-f429-account-create-update-r6vvp"] Dec 03 13:42:46 crc kubenswrapper[5052]: I1203 13:42:46.180787 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="horizon-kuttl-tests/keystone-f429-account-create-update-r6vvp" Dec 03 13:42:46 crc kubenswrapper[5052]: I1203 13:42:46.183932 5052 reflector.go:368] Caches populated for *v1.Secret from object-"horizon-kuttl-tests"/"keystone-db-secret" Dec 03 13:42:46 crc kubenswrapper[5052]: I1203 13:42:46.209255 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["horizon-kuttl-tests/keystone-f429-account-create-update-r6vvp"] Dec 03 13:42:46 crc kubenswrapper[5052]: I1203 13:42:46.257414 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-tms7h\" (UniqueName: \"kubernetes.io/projected/e500bdc5-8e72-481a-8e8c-177ddfb97582-kube-api-access-tms7h\") pod \"keystone-db-create-c58sf\" (UID: \"e500bdc5-8e72-481a-8e8c-177ddfb97582\") " pod="horizon-kuttl-tests/keystone-db-create-c58sf" Dec 03 13:42:46 crc kubenswrapper[5052]: I1203 13:42:46.257485 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9zjjt\" (UniqueName: \"kubernetes.io/projected/12abe26f-468f-4c73-aa6a-11a42582ebc3-kube-api-access-9zjjt\") pod \"keystone-f429-account-create-update-r6vvp\" (UID: \"12abe26f-468f-4c73-aa6a-11a42582ebc3\") " pod="horizon-kuttl-tests/keystone-f429-account-create-update-r6vvp" Dec 03 13:42:46 crc kubenswrapper[5052]: I1203 13:42:46.257553 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/e500bdc5-8e72-481a-8e8c-177ddfb97582-operator-scripts\") pod \"keystone-db-create-c58sf\" (UID: \"e500bdc5-8e72-481a-8e8c-177ddfb97582\") " pod="horizon-kuttl-tests/keystone-db-create-c58sf" Dec 03 13:42:46 crc kubenswrapper[5052]: I1203 13:42:46.257595 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/12abe26f-468f-4c73-aa6a-11a42582ebc3-operator-scripts\") pod \"keystone-f429-account-create-update-r6vvp\" (UID: \"12abe26f-468f-4c73-aa6a-11a42582ebc3\") " pod="horizon-kuttl-tests/keystone-f429-account-create-update-r6vvp" Dec 03 13:42:46 crc kubenswrapper[5052]: I1203 13:42:46.259181 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/e500bdc5-8e72-481a-8e8c-177ddfb97582-operator-scripts\") pod \"keystone-db-create-c58sf\" (UID: \"e500bdc5-8e72-481a-8e8c-177ddfb97582\") " pod="horizon-kuttl-tests/keystone-db-create-c58sf" Dec 03 13:42:46 crc kubenswrapper[5052]: I1203 13:42:46.314717 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-tms7h\" (UniqueName: \"kubernetes.io/projected/e500bdc5-8e72-481a-8e8c-177ddfb97582-kube-api-access-tms7h\") pod \"keystone-db-create-c58sf\" (UID: \"e500bdc5-8e72-481a-8e8c-177ddfb97582\") " pod="horizon-kuttl-tests/keystone-db-create-c58sf" Dec 03 13:42:46 crc kubenswrapper[5052]: I1203 13:42:46.358897 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-9zjjt\" (UniqueName: \"kubernetes.io/projected/12abe26f-468f-4c73-aa6a-11a42582ebc3-kube-api-access-9zjjt\") pod \"keystone-f429-account-create-update-r6vvp\" (UID: \"12abe26f-468f-4c73-aa6a-11a42582ebc3\") " pod="horizon-kuttl-tests/keystone-f429-account-create-update-r6vvp" Dec 03 13:42:46 crc kubenswrapper[5052]: I1203 13:42:46.358971 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/12abe26f-468f-4c73-aa6a-11a42582ebc3-operator-scripts\") pod \"keystone-f429-account-create-update-r6vvp\" (UID: \"12abe26f-468f-4c73-aa6a-11a42582ebc3\") " pod="horizon-kuttl-tests/keystone-f429-account-create-update-r6vvp" Dec 03 13:42:46 crc kubenswrapper[5052]: I1203 13:42:46.359689 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/12abe26f-468f-4c73-aa6a-11a42582ebc3-operator-scripts\") pod \"keystone-f429-account-create-update-r6vvp\" (UID: \"12abe26f-468f-4c73-aa6a-11a42582ebc3\") " pod="horizon-kuttl-tests/keystone-f429-account-create-update-r6vvp" Dec 03 13:42:46 crc kubenswrapper[5052]: I1203 13:42:46.397177 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-9zjjt\" (UniqueName: \"kubernetes.io/projected/12abe26f-468f-4c73-aa6a-11a42582ebc3-kube-api-access-9zjjt\") pod \"keystone-f429-account-create-update-r6vvp\" (UID: \"12abe26f-468f-4c73-aa6a-11a42582ebc3\") " pod="horizon-kuttl-tests/keystone-f429-account-create-update-r6vvp" Dec 03 13:42:46 crc kubenswrapper[5052]: I1203 13:42:46.459980 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="horizon-kuttl-tests/keystone-db-create-c58sf" Dec 03 13:42:46 crc kubenswrapper[5052]: I1203 13:42:46.495589 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="horizon-kuttl-tests/keystone-f429-account-create-update-r6vvp" Dec 03 13:42:46 crc kubenswrapper[5052]: I1203 13:42:46.909464 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["horizon-kuttl-tests/keystone-db-create-c58sf"] Dec 03 13:42:46 crc kubenswrapper[5052]: W1203 13:42:46.914488 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode500bdc5_8e72_481a_8e8c_177ddfb97582.slice/crio-852c00364245a14c0a8a8f22305afe889646eff4e3bd862e66fd6e619d5afe34 WatchSource:0}: Error finding container 852c00364245a14c0a8a8f22305afe889646eff4e3bd862e66fd6e619d5afe34: Status 404 returned error can't find the container with id 852c00364245a14c0a8a8f22305afe889646eff4e3bd862e66fd6e619d5afe34 Dec 03 13:42:46 crc kubenswrapper[5052]: I1203 13:42:46.985089 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["horizon-kuttl-tests/keystone-f429-account-create-update-r6vvp"] Dec 03 13:42:46 crc kubenswrapper[5052]: W1203 13:42:46.992727 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod12abe26f_468f_4c73_aa6a_11a42582ebc3.slice/crio-521032dbce07947fda5d4fb0c821bc85588cc04449a9ca75211d3a4d43c51f41 WatchSource:0}: Error finding container 521032dbce07947fda5d4fb0c821bc85588cc04449a9ca75211d3a4d43c51f41: Status 404 returned error can't find the container with id 521032dbce07947fda5d4fb0c821bc85588cc04449a9ca75211d3a4d43c51f41 Dec 03 13:42:47 crc kubenswrapper[5052]: I1203 13:42:47.783041 5052 generic.go:334] "Generic (PLEG): container finished" podID="e500bdc5-8e72-481a-8e8c-177ddfb97582" containerID="33dc678fd0c9e2a495d5335a1ad34ac104752fa08706d562d239fe848a24978c" exitCode=0 Dec 03 13:42:47 crc kubenswrapper[5052]: I1203 13:42:47.783182 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="horizon-kuttl-tests/keystone-db-create-c58sf" event={"ID":"e500bdc5-8e72-481a-8e8c-177ddfb97582","Type":"ContainerDied","Data":"33dc678fd0c9e2a495d5335a1ad34ac104752fa08706d562d239fe848a24978c"} Dec 03 13:42:47 crc kubenswrapper[5052]: I1203 13:42:47.783544 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="horizon-kuttl-tests/keystone-db-create-c58sf" event={"ID":"e500bdc5-8e72-481a-8e8c-177ddfb97582","Type":"ContainerStarted","Data":"852c00364245a14c0a8a8f22305afe889646eff4e3bd862e66fd6e619d5afe34"} Dec 03 13:42:47 crc kubenswrapper[5052]: I1203 13:42:47.788069 5052 generic.go:334] "Generic (PLEG): container finished" podID="12abe26f-468f-4c73-aa6a-11a42582ebc3" containerID="3d757c16532fe237e6034dd78dfe2e5e41a58f7d22ea0da43944853c350904f9" exitCode=0 Dec 03 13:42:47 crc kubenswrapper[5052]: I1203 13:42:47.788105 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="horizon-kuttl-tests/keystone-f429-account-create-update-r6vvp" event={"ID":"12abe26f-468f-4c73-aa6a-11a42582ebc3","Type":"ContainerDied","Data":"3d757c16532fe237e6034dd78dfe2e5e41a58f7d22ea0da43944853c350904f9"} Dec 03 13:42:47 crc kubenswrapper[5052]: I1203 13:42:47.788127 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="horizon-kuttl-tests/keystone-f429-account-create-update-r6vvp" event={"ID":"12abe26f-468f-4c73-aa6a-11a42582ebc3","Type":"ContainerStarted","Data":"521032dbce07947fda5d4fb0c821bc85588cc04449a9ca75211d3a4d43c51f41"} Dec 03 13:42:48 crc kubenswrapper[5052]: I1203 13:42:48.161421 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-operators/horizon-operator-index-d4wxq"] Dec 03 13:42:48 crc kubenswrapper[5052]: I1203 13:42:48.161655 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-operators/horizon-operator-index-d4wxq" podUID="01966e73-42ef-439a-8549-1ab114dda929" containerName="registry-server" containerID="cri-o://0e9e4a44f4377c0a78a0f342e67d5b6edb55c39891e99dc150306bf6e551b87c" gracePeriod=2 Dec 03 13:42:48 crc kubenswrapper[5052]: I1203 13:42:48.639747 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-operators/horizon-operator-index-d4wxq" Dec 03 13:42:48 crc kubenswrapper[5052]: I1203 13:42:48.765402 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-operators/horizon-operator-index-gkbwv"] Dec 03 13:42:48 crc kubenswrapper[5052]: E1203 13:42:48.765740 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="01966e73-42ef-439a-8549-1ab114dda929" containerName="registry-server" Dec 03 13:42:48 crc kubenswrapper[5052]: I1203 13:42:48.765756 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="01966e73-42ef-439a-8549-1ab114dda929" containerName="registry-server" Dec 03 13:42:48 crc kubenswrapper[5052]: I1203 13:42:48.765909 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="01966e73-42ef-439a-8549-1ab114dda929" containerName="registry-server" Dec 03 13:42:48 crc kubenswrapper[5052]: I1203 13:42:48.766373 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-operators/horizon-operator-index-gkbwv" Dec 03 13:42:48 crc kubenswrapper[5052]: I1203 13:42:48.778897 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-operators/horizon-operator-index-gkbwv"] Dec 03 13:42:48 crc kubenswrapper[5052]: I1203 13:42:48.791724 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7xbks\" (UniqueName: \"kubernetes.io/projected/01966e73-42ef-439a-8549-1ab114dda929-kube-api-access-7xbks\") pod \"01966e73-42ef-439a-8549-1ab114dda929\" (UID: \"01966e73-42ef-439a-8549-1ab114dda929\") " Dec 03 13:42:48 crc kubenswrapper[5052]: I1203 13:42:48.796887 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/01966e73-42ef-439a-8549-1ab114dda929-kube-api-access-7xbks" (OuterVolumeSpecName: "kube-api-access-7xbks") pod "01966e73-42ef-439a-8549-1ab114dda929" (UID: "01966e73-42ef-439a-8549-1ab114dda929"). InnerVolumeSpecName "kube-api-access-7xbks". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:42:48 crc kubenswrapper[5052]: I1203 13:42:48.797814 5052 generic.go:334] "Generic (PLEG): container finished" podID="01966e73-42ef-439a-8549-1ab114dda929" containerID="0e9e4a44f4377c0a78a0f342e67d5b6edb55c39891e99dc150306bf6e551b87c" exitCode=0 Dec 03 13:42:48 crc kubenswrapper[5052]: I1203 13:42:48.797876 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-operators/horizon-operator-index-d4wxq" Dec 03 13:42:48 crc kubenswrapper[5052]: I1203 13:42:48.797910 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/horizon-operator-index-d4wxq" event={"ID":"01966e73-42ef-439a-8549-1ab114dda929","Type":"ContainerDied","Data":"0e9e4a44f4377c0a78a0f342e67d5b6edb55c39891e99dc150306bf6e551b87c"} Dec 03 13:42:48 crc kubenswrapper[5052]: I1203 13:42:48.797940 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/horizon-operator-index-d4wxq" event={"ID":"01966e73-42ef-439a-8549-1ab114dda929","Type":"ContainerDied","Data":"7d49ee4bbba2a44c3372284f3bc77b3fef528d2735cb10d96ae26b43af36ef18"} Dec 03 13:42:48 crc kubenswrapper[5052]: I1203 13:42:48.797961 5052 scope.go:117] "RemoveContainer" containerID="0e9e4a44f4377c0a78a0f342e67d5b6edb55c39891e99dc150306bf6e551b87c" Dec 03 13:42:48 crc kubenswrapper[5052]: I1203 13:42:48.839538 5052 scope.go:117] "RemoveContainer" containerID="0e9e4a44f4377c0a78a0f342e67d5b6edb55c39891e99dc150306bf6e551b87c" Dec 03 13:42:48 crc kubenswrapper[5052]: E1203 13:42:48.840334 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0e9e4a44f4377c0a78a0f342e67d5b6edb55c39891e99dc150306bf6e551b87c\": container with ID starting with 0e9e4a44f4377c0a78a0f342e67d5b6edb55c39891e99dc150306bf6e551b87c not found: ID does not exist" containerID="0e9e4a44f4377c0a78a0f342e67d5b6edb55c39891e99dc150306bf6e551b87c" Dec 03 13:42:48 crc kubenswrapper[5052]: I1203 13:42:48.840366 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0e9e4a44f4377c0a78a0f342e67d5b6edb55c39891e99dc150306bf6e551b87c"} err="failed to get container status \"0e9e4a44f4377c0a78a0f342e67d5b6edb55c39891e99dc150306bf6e551b87c\": rpc error: code = NotFound desc = could not find container \"0e9e4a44f4377c0a78a0f342e67d5b6edb55c39891e99dc150306bf6e551b87c\": container with ID starting with 0e9e4a44f4377c0a78a0f342e67d5b6edb55c39891e99dc150306bf6e551b87c not found: ID does not exist" Dec 03 13:42:48 crc kubenswrapper[5052]: I1203 13:42:48.842742 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-operators/horizon-operator-index-d4wxq"] Dec 03 13:42:48 crc kubenswrapper[5052]: I1203 13:42:48.851503 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-operators/horizon-operator-index-d4wxq"] Dec 03 13:42:48 crc kubenswrapper[5052]: I1203 13:42:48.893805 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-79989\" (UniqueName: \"kubernetes.io/projected/8496c1b5-ae6d-4357-9812-9d44818ee9d1-kube-api-access-79989\") pod \"horizon-operator-index-gkbwv\" (UID: \"8496c1b5-ae6d-4357-9812-9d44818ee9d1\") " pod="openstack-operators/horizon-operator-index-gkbwv" Dec 03 13:42:48 crc kubenswrapper[5052]: I1203 13:42:48.893911 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-7xbks\" (UniqueName: \"kubernetes.io/projected/01966e73-42ef-439a-8549-1ab114dda929-kube-api-access-7xbks\") on node \"crc\" DevicePath \"\"" Dec 03 13:42:48 crc kubenswrapper[5052]: I1203 13:42:48.995204 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-79989\" (UniqueName: \"kubernetes.io/projected/8496c1b5-ae6d-4357-9812-9d44818ee9d1-kube-api-access-79989\") pod \"horizon-operator-index-gkbwv\" (UID: \"8496c1b5-ae6d-4357-9812-9d44818ee9d1\") " pod="openstack-operators/horizon-operator-index-gkbwv" Dec 03 13:42:49 crc kubenswrapper[5052]: I1203 13:42:49.017382 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-79989\" (UniqueName: \"kubernetes.io/projected/8496c1b5-ae6d-4357-9812-9d44818ee9d1-kube-api-access-79989\") pod \"horizon-operator-index-gkbwv\" (UID: \"8496c1b5-ae6d-4357-9812-9d44818ee9d1\") " pod="openstack-operators/horizon-operator-index-gkbwv" Dec 03 13:42:49 crc kubenswrapper[5052]: I1203 13:42:49.081233 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-operators/horizon-operator-index-gkbwv" Dec 03 13:42:49 crc kubenswrapper[5052]: I1203 13:42:49.139086 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="horizon-kuttl-tests/keystone-db-create-c58sf" Dec 03 13:42:49 crc kubenswrapper[5052]: I1203 13:42:49.145170 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="horizon-kuttl-tests/keystone-f429-account-create-update-r6vvp" Dec 03 13:42:49 crc kubenswrapper[5052]: I1203 13:42:49.300443 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/e500bdc5-8e72-481a-8e8c-177ddfb97582-operator-scripts\") pod \"e500bdc5-8e72-481a-8e8c-177ddfb97582\" (UID: \"e500bdc5-8e72-481a-8e8c-177ddfb97582\") " Dec 03 13:42:49 crc kubenswrapper[5052]: I1203 13:42:49.300764 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/12abe26f-468f-4c73-aa6a-11a42582ebc3-operator-scripts\") pod \"12abe26f-468f-4c73-aa6a-11a42582ebc3\" (UID: \"12abe26f-468f-4c73-aa6a-11a42582ebc3\") " Dec 03 13:42:49 crc kubenswrapper[5052]: I1203 13:42:49.300817 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tms7h\" (UniqueName: \"kubernetes.io/projected/e500bdc5-8e72-481a-8e8c-177ddfb97582-kube-api-access-tms7h\") pod \"e500bdc5-8e72-481a-8e8c-177ddfb97582\" (UID: \"e500bdc5-8e72-481a-8e8c-177ddfb97582\") " Dec 03 13:42:49 crc kubenswrapper[5052]: I1203 13:42:49.300848 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9zjjt\" (UniqueName: \"kubernetes.io/projected/12abe26f-468f-4c73-aa6a-11a42582ebc3-kube-api-access-9zjjt\") pod \"12abe26f-468f-4c73-aa6a-11a42582ebc3\" (UID: \"12abe26f-468f-4c73-aa6a-11a42582ebc3\") " Dec 03 13:42:49 crc kubenswrapper[5052]: I1203 13:42:49.300875 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e500bdc5-8e72-481a-8e8c-177ddfb97582-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "e500bdc5-8e72-481a-8e8c-177ddfb97582" (UID: "e500bdc5-8e72-481a-8e8c-177ddfb97582"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:42:49 crc kubenswrapper[5052]: I1203 13:42:49.301096 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/e500bdc5-8e72-481a-8e8c-177ddfb97582-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 13:42:49 crc kubenswrapper[5052]: I1203 13:42:49.301186 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/12abe26f-468f-4c73-aa6a-11a42582ebc3-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "12abe26f-468f-4c73-aa6a-11a42582ebc3" (UID: "12abe26f-468f-4c73-aa6a-11a42582ebc3"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:42:49 crc kubenswrapper[5052]: I1203 13:42:49.305624 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e500bdc5-8e72-481a-8e8c-177ddfb97582-kube-api-access-tms7h" (OuterVolumeSpecName: "kube-api-access-tms7h") pod "e500bdc5-8e72-481a-8e8c-177ddfb97582" (UID: "e500bdc5-8e72-481a-8e8c-177ddfb97582"). InnerVolumeSpecName "kube-api-access-tms7h". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:42:49 crc kubenswrapper[5052]: I1203 13:42:49.306379 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/12abe26f-468f-4c73-aa6a-11a42582ebc3-kube-api-access-9zjjt" (OuterVolumeSpecName: "kube-api-access-9zjjt") pod "12abe26f-468f-4c73-aa6a-11a42582ebc3" (UID: "12abe26f-468f-4c73-aa6a-11a42582ebc3"). InnerVolumeSpecName "kube-api-access-9zjjt". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:42:49 crc kubenswrapper[5052]: I1203 13:42:49.405615 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/12abe26f-468f-4c73-aa6a-11a42582ebc3-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 13:42:49 crc kubenswrapper[5052]: I1203 13:42:49.405654 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-tms7h\" (UniqueName: \"kubernetes.io/projected/e500bdc5-8e72-481a-8e8c-177ddfb97582-kube-api-access-tms7h\") on node \"crc\" DevicePath \"\"" Dec 03 13:42:49 crc kubenswrapper[5052]: I1203 13:42:49.405669 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-9zjjt\" (UniqueName: \"kubernetes.io/projected/12abe26f-468f-4c73-aa6a-11a42582ebc3-kube-api-access-9zjjt\") on node \"crc\" DevicePath \"\"" Dec 03 13:42:49 crc kubenswrapper[5052]: I1203 13:42:49.534230 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-operators/horizon-operator-index-gkbwv"] Dec 03 13:42:49 crc kubenswrapper[5052]: W1203 13:42:49.536130 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8496c1b5_ae6d_4357_9812_9d44818ee9d1.slice/crio-6d8d15ac75b70a3c76f418044b4fbc4e0e48dabf92bf342d17042ffb20ffcf63 WatchSource:0}: Error finding container 6d8d15ac75b70a3c76f418044b4fbc4e0e48dabf92bf342d17042ffb20ffcf63: Status 404 returned error can't find the container with id 6d8d15ac75b70a3c76f418044b4fbc4e0e48dabf92bf342d17042ffb20ffcf63 Dec 03 13:42:49 crc kubenswrapper[5052]: I1203 13:42:49.680654 5052 scope.go:117] "RemoveContainer" containerID="2fbd293d6edef0fa059e91c119312e77b262534107c322feb7b59dbea3464526" Dec 03 13:42:49 crc kubenswrapper[5052]: E1203 13:42:49.681189 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 13:42:49 crc kubenswrapper[5052]: I1203 13:42:49.681242 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="01966e73-42ef-439a-8549-1ab114dda929" path="/var/lib/kubelet/pods/01966e73-42ef-439a-8549-1ab114dda929/volumes" Dec 03 13:42:49 crc kubenswrapper[5052]: I1203 13:42:49.804935 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/horizon-operator-index-gkbwv" event={"ID":"8496c1b5-ae6d-4357-9812-9d44818ee9d1","Type":"ContainerStarted","Data":"6d8d15ac75b70a3c76f418044b4fbc4e0e48dabf92bf342d17042ffb20ffcf63"} Dec 03 13:42:49 crc kubenswrapper[5052]: I1203 13:42:49.807196 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="horizon-kuttl-tests/keystone-db-create-c58sf" event={"ID":"e500bdc5-8e72-481a-8e8c-177ddfb97582","Type":"ContainerDied","Data":"852c00364245a14c0a8a8f22305afe889646eff4e3bd862e66fd6e619d5afe34"} Dec 03 13:42:49 crc kubenswrapper[5052]: I1203 13:42:49.807221 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="852c00364245a14c0a8a8f22305afe889646eff4e3bd862e66fd6e619d5afe34" Dec 03 13:42:49 crc kubenswrapper[5052]: I1203 13:42:49.807266 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="horizon-kuttl-tests/keystone-db-create-c58sf" Dec 03 13:42:49 crc kubenswrapper[5052]: I1203 13:42:49.809067 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="horizon-kuttl-tests/keystone-f429-account-create-update-r6vvp" event={"ID":"12abe26f-468f-4c73-aa6a-11a42582ebc3","Type":"ContainerDied","Data":"521032dbce07947fda5d4fb0c821bc85588cc04449a9ca75211d3a4d43c51f41"} Dec 03 13:42:49 crc kubenswrapper[5052]: I1203 13:42:49.809099 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="521032dbce07947fda5d4fb0c821bc85588cc04449a9ca75211d3a4d43c51f41" Dec 03 13:42:49 crc kubenswrapper[5052]: I1203 13:42:49.809140 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="horizon-kuttl-tests/keystone-f429-account-create-update-r6vvp" Dec 03 13:42:50 crc kubenswrapper[5052]: I1203 13:42:50.816996 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/horizon-operator-index-gkbwv" event={"ID":"8496c1b5-ae6d-4357-9812-9d44818ee9d1","Type":"ContainerStarted","Data":"58a8a984bcc25e97fc3cb0134e46c0fd093e75da4a13403e3a7dcd59d80b2901"} Dec 03 13:42:50 crc kubenswrapper[5052]: I1203 13:42:50.835697 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-operators/horizon-operator-index-gkbwv" podStartSLOduration=2.397534254 podStartE2EDuration="2.835679718s" podCreationTimestamp="2025-12-03 13:42:48 +0000 UTC" firstStartedPulling="2025-12-03 13:42:49.539615551 +0000 UTC m=+8560.127857305" lastFinishedPulling="2025-12-03 13:42:49.977761005 +0000 UTC m=+8560.566002769" observedRunningTime="2025-12-03 13:42:50.829351637 +0000 UTC m=+8561.417593401" watchObservedRunningTime="2025-12-03 13:42:50.835679718 +0000 UTC m=+8561.423921482" Dec 03 13:42:53 crc kubenswrapper[5052]: I1203 13:42:53.844397 5052 generic.go:334] "Generic (PLEG): container finished" podID="aa07e692-9f86-4c61-b674-beeb49376c0e" containerID="29a5cf76d0275587ac82e2efe006e49acba2897de049585dae86807c72ae9434" exitCode=0 Dec 03 13:42:53 crc kubenswrapper[5052]: I1203 13:42:53.844549 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="horizon-kuttl-tests/rabbitmq-server-0" event={"ID":"aa07e692-9f86-4c61-b674-beeb49376c0e","Type":"ContainerDied","Data":"29a5cf76d0275587ac82e2efe006e49acba2897de049585dae86807c72ae9434"} Dec 03 13:42:54 crc kubenswrapper[5052]: I1203 13:42:54.853442 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="horizon-kuttl-tests/rabbitmq-server-0" event={"ID":"aa07e692-9f86-4c61-b674-beeb49376c0e","Type":"ContainerStarted","Data":"574b3d04ceb3e56d6b1097c3df01bb5f5d8454557a84049929fc504a6dc656ea"} Dec 03 13:42:54 crc kubenswrapper[5052]: I1203 13:42:54.853676 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="horizon-kuttl-tests/rabbitmq-server-0" Dec 03 13:42:54 crc kubenswrapper[5052]: I1203 13:42:54.887155 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="horizon-kuttl-tests/rabbitmq-server-0" podStartSLOduration=35.887135551 podStartE2EDuration="35.887135551s" podCreationTimestamp="2025-12-03 13:42:19 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 13:42:54.883961 +0000 UTC m=+8565.472202764" watchObservedRunningTime="2025-12-03 13:42:54.887135551 +0000 UTC m=+8565.475377315" Dec 03 13:42:59 crc kubenswrapper[5052]: I1203 13:42:59.081708 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-operators/horizon-operator-index-gkbwv" Dec 03 13:42:59 crc kubenswrapper[5052]: I1203 13:42:59.082277 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openstack-operators/horizon-operator-index-gkbwv" Dec 03 13:42:59 crc kubenswrapper[5052]: I1203 13:42:59.110821 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openstack-operators/horizon-operator-index-gkbwv" Dec 03 13:42:59 crc kubenswrapper[5052]: I1203 13:42:59.920162 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-operators/horizon-operator-index-gkbwv" Dec 03 13:43:00 crc kubenswrapper[5052]: I1203 13:43:00.672966 5052 scope.go:117] "RemoveContainer" containerID="2fbd293d6edef0fa059e91c119312e77b262534107c322feb7b59dbea3464526" Dec 03 13:43:00 crc kubenswrapper[5052]: E1203 13:43:00.673495 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 13:43:03 crc kubenswrapper[5052]: I1203 13:43:03.013700 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-operators/9f0c59a3968beec894e04476dd5efd0a707bad85f482efd4940498368cx9f7f"] Dec 03 13:43:03 crc kubenswrapper[5052]: E1203 13:43:03.014006 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="12abe26f-468f-4c73-aa6a-11a42582ebc3" containerName="mariadb-account-create-update" Dec 03 13:43:03 crc kubenswrapper[5052]: I1203 13:43:03.014019 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="12abe26f-468f-4c73-aa6a-11a42582ebc3" containerName="mariadb-account-create-update" Dec 03 13:43:03 crc kubenswrapper[5052]: E1203 13:43:03.014030 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="e500bdc5-8e72-481a-8e8c-177ddfb97582" containerName="mariadb-database-create" Dec 03 13:43:03 crc kubenswrapper[5052]: I1203 13:43:03.014036 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="e500bdc5-8e72-481a-8e8c-177ddfb97582" containerName="mariadb-database-create" Dec 03 13:43:03 crc kubenswrapper[5052]: I1203 13:43:03.014154 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="12abe26f-468f-4c73-aa6a-11a42582ebc3" containerName="mariadb-account-create-update" Dec 03 13:43:03 crc kubenswrapper[5052]: I1203 13:43:03.014172 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="e500bdc5-8e72-481a-8e8c-177ddfb97582" containerName="mariadb-database-create" Dec 03 13:43:03 crc kubenswrapper[5052]: I1203 13:43:03.015236 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-operators/9f0c59a3968beec894e04476dd5efd0a707bad85f482efd4940498368cx9f7f" Dec 03 13:43:03 crc kubenswrapper[5052]: I1203 13:43:03.022026 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-operators"/"default-dockercfg-7fklr" Dec 03 13:43:03 crc kubenswrapper[5052]: I1203 13:43:03.025353 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-operators/9f0c59a3968beec894e04476dd5efd0a707bad85f482efd4940498368cx9f7f"] Dec 03 13:43:03 crc kubenswrapper[5052]: I1203 13:43:03.107030 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3f46060f-d03d-452f-b08a-baa387f2efbf-util\") pod \"9f0c59a3968beec894e04476dd5efd0a707bad85f482efd4940498368cx9f7f\" (UID: \"3f46060f-d03d-452f-b08a-baa387f2efbf\") " pod="openstack-operators/9f0c59a3968beec894e04476dd5efd0a707bad85f482efd4940498368cx9f7f" Dec 03 13:43:03 crc kubenswrapper[5052]: I1203 13:43:03.107410 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3f46060f-d03d-452f-b08a-baa387f2efbf-bundle\") pod \"9f0c59a3968beec894e04476dd5efd0a707bad85f482efd4940498368cx9f7f\" (UID: \"3f46060f-d03d-452f-b08a-baa387f2efbf\") " pod="openstack-operators/9f0c59a3968beec894e04476dd5efd0a707bad85f482efd4940498368cx9f7f" Dec 03 13:43:03 crc kubenswrapper[5052]: I1203 13:43:03.107569 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vnmfd\" (UniqueName: \"kubernetes.io/projected/3f46060f-d03d-452f-b08a-baa387f2efbf-kube-api-access-vnmfd\") pod \"9f0c59a3968beec894e04476dd5efd0a707bad85f482efd4940498368cx9f7f\" (UID: \"3f46060f-d03d-452f-b08a-baa387f2efbf\") " pod="openstack-operators/9f0c59a3968beec894e04476dd5efd0a707bad85f482efd4940498368cx9f7f" Dec 03 13:43:03 crc kubenswrapper[5052]: I1203 13:43:03.209238 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3f46060f-d03d-452f-b08a-baa387f2efbf-bundle\") pod \"9f0c59a3968beec894e04476dd5efd0a707bad85f482efd4940498368cx9f7f\" (UID: \"3f46060f-d03d-452f-b08a-baa387f2efbf\") " pod="openstack-operators/9f0c59a3968beec894e04476dd5efd0a707bad85f482efd4940498368cx9f7f" Dec 03 13:43:03 crc kubenswrapper[5052]: I1203 13:43:03.209533 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-vnmfd\" (UniqueName: \"kubernetes.io/projected/3f46060f-d03d-452f-b08a-baa387f2efbf-kube-api-access-vnmfd\") pod \"9f0c59a3968beec894e04476dd5efd0a707bad85f482efd4940498368cx9f7f\" (UID: \"3f46060f-d03d-452f-b08a-baa387f2efbf\") " pod="openstack-operators/9f0c59a3968beec894e04476dd5efd0a707bad85f482efd4940498368cx9f7f" Dec 03 13:43:03 crc kubenswrapper[5052]: I1203 13:43:03.209679 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3f46060f-d03d-452f-b08a-baa387f2efbf-util\") pod \"9f0c59a3968beec894e04476dd5efd0a707bad85f482efd4940498368cx9f7f\" (UID: \"3f46060f-d03d-452f-b08a-baa387f2efbf\") " pod="openstack-operators/9f0c59a3968beec894e04476dd5efd0a707bad85f482efd4940498368cx9f7f" Dec 03 13:43:03 crc kubenswrapper[5052]: I1203 13:43:03.209770 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3f46060f-d03d-452f-b08a-baa387f2efbf-bundle\") pod \"9f0c59a3968beec894e04476dd5efd0a707bad85f482efd4940498368cx9f7f\" (UID: \"3f46060f-d03d-452f-b08a-baa387f2efbf\") " pod="openstack-operators/9f0c59a3968beec894e04476dd5efd0a707bad85f482efd4940498368cx9f7f" Dec 03 13:43:03 crc kubenswrapper[5052]: I1203 13:43:03.210147 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3f46060f-d03d-452f-b08a-baa387f2efbf-util\") pod \"9f0c59a3968beec894e04476dd5efd0a707bad85f482efd4940498368cx9f7f\" (UID: \"3f46060f-d03d-452f-b08a-baa387f2efbf\") " pod="openstack-operators/9f0c59a3968beec894e04476dd5efd0a707bad85f482efd4940498368cx9f7f" Dec 03 13:43:03 crc kubenswrapper[5052]: I1203 13:43:03.228255 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-vnmfd\" (UniqueName: \"kubernetes.io/projected/3f46060f-d03d-452f-b08a-baa387f2efbf-kube-api-access-vnmfd\") pod \"9f0c59a3968beec894e04476dd5efd0a707bad85f482efd4940498368cx9f7f\" (UID: \"3f46060f-d03d-452f-b08a-baa387f2efbf\") " pod="openstack-operators/9f0c59a3968beec894e04476dd5efd0a707bad85f482efd4940498368cx9f7f" Dec 03 13:43:03 crc kubenswrapper[5052]: I1203 13:43:03.338737 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-operators/9f0c59a3968beec894e04476dd5efd0a707bad85f482efd4940498368cx9f7f" Dec 03 13:43:03 crc kubenswrapper[5052]: I1203 13:43:03.745915 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-operators/9f0c59a3968beec894e04476dd5efd0a707bad85f482efd4940498368cx9f7f"] Dec 03 13:43:03 crc kubenswrapper[5052]: I1203 13:43:03.916757 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/9f0c59a3968beec894e04476dd5efd0a707bad85f482efd4940498368cx9f7f" event={"ID":"3f46060f-d03d-452f-b08a-baa387f2efbf","Type":"ContainerStarted","Data":"c71d00af602e633f64ea6831c33edeffdc6184aaca0eacd47120abd41c85aa94"} Dec 03 13:43:03 crc kubenswrapper[5052]: I1203 13:43:03.916804 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/9f0c59a3968beec894e04476dd5efd0a707bad85f482efd4940498368cx9f7f" event={"ID":"3f46060f-d03d-452f-b08a-baa387f2efbf","Type":"ContainerStarted","Data":"658600d6f5469c5e63ad68c40278b80c2ba78b30d516c569eb2eeb13805b46ce"} Dec 03 13:43:04 crc kubenswrapper[5052]: I1203 13:43:04.924130 5052 generic.go:334] "Generic (PLEG): container finished" podID="3f46060f-d03d-452f-b08a-baa387f2efbf" containerID="c71d00af602e633f64ea6831c33edeffdc6184aaca0eacd47120abd41c85aa94" exitCode=0 Dec 03 13:43:04 crc kubenswrapper[5052]: I1203 13:43:04.924211 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/9f0c59a3968beec894e04476dd5efd0a707bad85f482efd4940498368cx9f7f" event={"ID":"3f46060f-d03d-452f-b08a-baa387f2efbf","Type":"ContainerDied","Data":"c71d00af602e633f64ea6831c33edeffdc6184aaca0eacd47120abd41c85aa94"} Dec 03 13:43:06 crc kubenswrapper[5052]: I1203 13:43:06.941803 5052 generic.go:334] "Generic (PLEG): container finished" podID="3f46060f-d03d-452f-b08a-baa387f2efbf" containerID="76b042da28713d09345ef0b170c843ae2fb4756281a64e7bda9fae75dc908335" exitCode=0 Dec 03 13:43:06 crc kubenswrapper[5052]: I1203 13:43:06.941955 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/9f0c59a3968beec894e04476dd5efd0a707bad85f482efd4940498368cx9f7f" event={"ID":"3f46060f-d03d-452f-b08a-baa387f2efbf","Type":"ContainerDied","Data":"76b042da28713d09345ef0b170c843ae2fb4756281a64e7bda9fae75dc908335"} Dec 03 13:43:07 crc kubenswrapper[5052]: I1203 13:43:07.954666 5052 generic.go:334] "Generic (PLEG): container finished" podID="3f46060f-d03d-452f-b08a-baa387f2efbf" containerID="d7f0c01793bafc3fd7fbc4d56442c47f7eb67bf7b561a0a12c62f9e020f177ad" exitCode=0 Dec 03 13:43:07 crc kubenswrapper[5052]: I1203 13:43:07.954724 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/9f0c59a3968beec894e04476dd5efd0a707bad85f482efd4940498368cx9f7f" event={"ID":"3f46060f-d03d-452f-b08a-baa387f2efbf","Type":"ContainerDied","Data":"d7f0c01793bafc3fd7fbc4d56442c47f7eb67bf7b561a0a12c62f9e020f177ad"} Dec 03 13:43:09 crc kubenswrapper[5052]: I1203 13:43:09.273946 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-operators/9f0c59a3968beec894e04476dd5efd0a707bad85f482efd4940498368cx9f7f" Dec 03 13:43:09 crc kubenswrapper[5052]: I1203 13:43:09.415434 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3f46060f-d03d-452f-b08a-baa387f2efbf-bundle\") pod \"3f46060f-d03d-452f-b08a-baa387f2efbf\" (UID: \"3f46060f-d03d-452f-b08a-baa387f2efbf\") " Dec 03 13:43:09 crc kubenswrapper[5052]: I1203 13:43:09.415586 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vnmfd\" (UniqueName: \"kubernetes.io/projected/3f46060f-d03d-452f-b08a-baa387f2efbf-kube-api-access-vnmfd\") pod \"3f46060f-d03d-452f-b08a-baa387f2efbf\" (UID: \"3f46060f-d03d-452f-b08a-baa387f2efbf\") " Dec 03 13:43:09 crc kubenswrapper[5052]: I1203 13:43:09.415641 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3f46060f-d03d-452f-b08a-baa387f2efbf-util\") pod \"3f46060f-d03d-452f-b08a-baa387f2efbf\" (UID: \"3f46060f-d03d-452f-b08a-baa387f2efbf\") " Dec 03 13:43:09 crc kubenswrapper[5052]: I1203 13:43:09.416281 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3f46060f-d03d-452f-b08a-baa387f2efbf-bundle" (OuterVolumeSpecName: "bundle") pod "3f46060f-d03d-452f-b08a-baa387f2efbf" (UID: "3f46060f-d03d-452f-b08a-baa387f2efbf"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 13:43:09 crc kubenswrapper[5052]: I1203 13:43:09.421747 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3f46060f-d03d-452f-b08a-baa387f2efbf-kube-api-access-vnmfd" (OuterVolumeSpecName: "kube-api-access-vnmfd") pod "3f46060f-d03d-452f-b08a-baa387f2efbf" (UID: "3f46060f-d03d-452f-b08a-baa387f2efbf"). InnerVolumeSpecName "kube-api-access-vnmfd". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:43:09 crc kubenswrapper[5052]: I1203 13:43:09.507571 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3f46060f-d03d-452f-b08a-baa387f2efbf-util" (OuterVolumeSpecName: "util") pod "3f46060f-d03d-452f-b08a-baa387f2efbf" (UID: "3f46060f-d03d-452f-b08a-baa387f2efbf"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 13:43:09 crc kubenswrapper[5052]: I1203 13:43:09.517744 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-vnmfd\" (UniqueName: \"kubernetes.io/projected/3f46060f-d03d-452f-b08a-baa387f2efbf-kube-api-access-vnmfd\") on node \"crc\" DevicePath \"\"" Dec 03 13:43:09 crc kubenswrapper[5052]: I1203 13:43:09.517939 5052 reconciler_common.go:293] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3f46060f-d03d-452f-b08a-baa387f2efbf-util\") on node \"crc\" DevicePath \"\"" Dec 03 13:43:09 crc kubenswrapper[5052]: I1203 13:43:09.517997 5052 reconciler_common.go:293] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3f46060f-d03d-452f-b08a-baa387f2efbf-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 13:43:09 crc kubenswrapper[5052]: I1203 13:43:09.972912 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/9f0c59a3968beec894e04476dd5efd0a707bad85f482efd4940498368cx9f7f" event={"ID":"3f46060f-d03d-452f-b08a-baa387f2efbf","Type":"ContainerDied","Data":"658600d6f5469c5e63ad68c40278b80c2ba78b30d516c569eb2eeb13805b46ce"} Dec 03 13:43:09 crc kubenswrapper[5052]: I1203 13:43:09.973264 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="658600d6f5469c5e63ad68c40278b80c2ba78b30d516c569eb2eeb13805b46ce" Dec 03 13:43:09 crc kubenswrapper[5052]: I1203 13:43:09.973111 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-operators/9f0c59a3968beec894e04476dd5efd0a707bad85f482efd4940498368cx9f7f" Dec 03 13:43:10 crc kubenswrapper[5052]: I1203 13:43:10.628383 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="horizon-kuttl-tests/rabbitmq-server-0" Dec 03 13:43:11 crc kubenswrapper[5052]: I1203 13:43:11.087188 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["horizon-kuttl-tests/keystone-db-sync-2qj57"] Dec 03 13:43:11 crc kubenswrapper[5052]: E1203 13:43:11.087455 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="3f46060f-d03d-452f-b08a-baa387f2efbf" containerName="pull" Dec 03 13:43:11 crc kubenswrapper[5052]: I1203 13:43:11.087468 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="3f46060f-d03d-452f-b08a-baa387f2efbf" containerName="pull" Dec 03 13:43:11 crc kubenswrapper[5052]: E1203 13:43:11.087480 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="3f46060f-d03d-452f-b08a-baa387f2efbf" containerName="util" Dec 03 13:43:11 crc kubenswrapper[5052]: I1203 13:43:11.087485 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="3f46060f-d03d-452f-b08a-baa387f2efbf" containerName="util" Dec 03 13:43:11 crc kubenswrapper[5052]: E1203 13:43:11.087529 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="3f46060f-d03d-452f-b08a-baa387f2efbf" containerName="extract" Dec 03 13:43:11 crc kubenswrapper[5052]: I1203 13:43:11.087536 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="3f46060f-d03d-452f-b08a-baa387f2efbf" containerName="extract" Dec 03 13:43:11 crc kubenswrapper[5052]: I1203 13:43:11.087644 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="3f46060f-d03d-452f-b08a-baa387f2efbf" containerName="extract" Dec 03 13:43:11 crc kubenswrapper[5052]: I1203 13:43:11.088085 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="horizon-kuttl-tests/keystone-db-sync-2qj57" Dec 03 13:43:11 crc kubenswrapper[5052]: I1203 13:43:11.090927 5052 reflector.go:368] Caches populated for *v1.Secret from object-"horizon-kuttl-tests"/"keystone-scripts" Dec 03 13:43:11 crc kubenswrapper[5052]: I1203 13:43:11.091219 5052 reflector.go:368] Caches populated for *v1.Secret from object-"horizon-kuttl-tests"/"keystone-keystone-dockercfg-kqrlh" Dec 03 13:43:11 crc kubenswrapper[5052]: I1203 13:43:11.091766 5052 reflector.go:368] Caches populated for *v1.Secret from object-"horizon-kuttl-tests"/"keystone" Dec 03 13:43:11 crc kubenswrapper[5052]: I1203 13:43:11.092049 5052 reflector.go:368] Caches populated for *v1.Secret from object-"horizon-kuttl-tests"/"keystone-config-data" Dec 03 13:43:11 crc kubenswrapper[5052]: I1203 13:43:11.103663 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["horizon-kuttl-tests/keystone-db-sync-2qj57"] Dec 03 13:43:11 crc kubenswrapper[5052]: I1203 13:43:11.244335 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tc4gv\" (UniqueName: \"kubernetes.io/projected/3cefc769-6ac1-4466-bcb4-bc1928dfce9e-kube-api-access-tc4gv\") pod \"keystone-db-sync-2qj57\" (UID: \"3cefc769-6ac1-4466-bcb4-bc1928dfce9e\") " pod="horizon-kuttl-tests/keystone-db-sync-2qj57" Dec 03 13:43:11 crc kubenswrapper[5052]: I1203 13:43:11.244637 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/3cefc769-6ac1-4466-bcb4-bc1928dfce9e-config-data\") pod \"keystone-db-sync-2qj57\" (UID: \"3cefc769-6ac1-4466-bcb4-bc1928dfce9e\") " pod="horizon-kuttl-tests/keystone-db-sync-2qj57" Dec 03 13:43:11 crc kubenswrapper[5052]: I1203 13:43:11.346096 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-tc4gv\" (UniqueName: \"kubernetes.io/projected/3cefc769-6ac1-4466-bcb4-bc1928dfce9e-kube-api-access-tc4gv\") pod \"keystone-db-sync-2qj57\" (UID: \"3cefc769-6ac1-4466-bcb4-bc1928dfce9e\") " pod="horizon-kuttl-tests/keystone-db-sync-2qj57" Dec 03 13:43:11 crc kubenswrapper[5052]: I1203 13:43:11.346174 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/3cefc769-6ac1-4466-bcb4-bc1928dfce9e-config-data\") pod \"keystone-db-sync-2qj57\" (UID: \"3cefc769-6ac1-4466-bcb4-bc1928dfce9e\") " pod="horizon-kuttl-tests/keystone-db-sync-2qj57" Dec 03 13:43:11 crc kubenswrapper[5052]: I1203 13:43:11.352444 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/3cefc769-6ac1-4466-bcb4-bc1928dfce9e-config-data\") pod \"keystone-db-sync-2qj57\" (UID: \"3cefc769-6ac1-4466-bcb4-bc1928dfce9e\") " pod="horizon-kuttl-tests/keystone-db-sync-2qj57" Dec 03 13:43:11 crc kubenswrapper[5052]: I1203 13:43:11.365119 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-tc4gv\" (UniqueName: \"kubernetes.io/projected/3cefc769-6ac1-4466-bcb4-bc1928dfce9e-kube-api-access-tc4gv\") pod \"keystone-db-sync-2qj57\" (UID: \"3cefc769-6ac1-4466-bcb4-bc1928dfce9e\") " pod="horizon-kuttl-tests/keystone-db-sync-2qj57" Dec 03 13:43:11 crc kubenswrapper[5052]: I1203 13:43:11.404798 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="horizon-kuttl-tests/keystone-db-sync-2qj57" Dec 03 13:43:11 crc kubenswrapper[5052]: I1203 13:43:11.648926 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["horizon-kuttl-tests/keystone-db-sync-2qj57"] Dec 03 13:43:11 crc kubenswrapper[5052]: I1203 13:43:11.986994 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="horizon-kuttl-tests/keystone-db-sync-2qj57" event={"ID":"3cefc769-6ac1-4466-bcb4-bc1928dfce9e","Type":"ContainerStarted","Data":"610354f9fb50c7dbfda327519d396e09b7928db7639a32bbd4ca5aa25fbe105f"} Dec 03 13:43:13 crc kubenswrapper[5052]: I1203 13:43:13.673278 5052 scope.go:117] "RemoveContainer" containerID="2fbd293d6edef0fa059e91c119312e77b262534107c322feb7b59dbea3464526" Dec 03 13:43:13 crc kubenswrapper[5052]: E1203 13:43:13.673683 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 13:43:15 crc kubenswrapper[5052]: I1203 13:43:15.010778 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="horizon-kuttl-tests/keystone-db-sync-2qj57" event={"ID":"3cefc769-6ac1-4466-bcb4-bc1928dfce9e","Type":"ContainerStarted","Data":"c264afb73103dca4a9c8be43892dd84ddc4910a887544320f95185671931cd3e"} Dec 03 13:43:15 crc kubenswrapper[5052]: I1203 13:43:15.026715 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="horizon-kuttl-tests/keystone-db-sync-2qj57" podStartSLOduration=4.026696281 podStartE2EDuration="4.026696281s" podCreationTimestamp="2025-12-03 13:43:11 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 13:43:15.023232432 +0000 UTC m=+8585.611474216" watchObservedRunningTime="2025-12-03 13:43:15.026696281 +0000 UTC m=+8585.614938045" Dec 03 13:43:21 crc kubenswrapper[5052]: I1203 13:43:21.065265 5052 generic.go:334] "Generic (PLEG): container finished" podID="3cefc769-6ac1-4466-bcb4-bc1928dfce9e" containerID="c264afb73103dca4a9c8be43892dd84ddc4910a887544320f95185671931cd3e" exitCode=0 Dec 03 13:43:21 crc kubenswrapper[5052]: I1203 13:43:21.066411 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="horizon-kuttl-tests/keystone-db-sync-2qj57" event={"ID":"3cefc769-6ac1-4466-bcb4-bc1928dfce9e","Type":"ContainerDied","Data":"c264afb73103dca4a9c8be43892dd84ddc4910a887544320f95185671931cd3e"} Dec 03 13:43:22 crc kubenswrapper[5052]: I1203 13:43:22.380504 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="horizon-kuttl-tests/keystone-db-sync-2qj57" Dec 03 13:43:22 crc kubenswrapper[5052]: I1203 13:43:22.515289 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tc4gv\" (UniqueName: \"kubernetes.io/projected/3cefc769-6ac1-4466-bcb4-bc1928dfce9e-kube-api-access-tc4gv\") pod \"3cefc769-6ac1-4466-bcb4-bc1928dfce9e\" (UID: \"3cefc769-6ac1-4466-bcb4-bc1928dfce9e\") " Dec 03 13:43:22 crc kubenswrapper[5052]: I1203 13:43:22.515386 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/3cefc769-6ac1-4466-bcb4-bc1928dfce9e-config-data\") pod \"3cefc769-6ac1-4466-bcb4-bc1928dfce9e\" (UID: \"3cefc769-6ac1-4466-bcb4-bc1928dfce9e\") " Dec 03 13:43:22 crc kubenswrapper[5052]: I1203 13:43:22.522755 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3cefc769-6ac1-4466-bcb4-bc1928dfce9e-kube-api-access-tc4gv" (OuterVolumeSpecName: "kube-api-access-tc4gv") pod "3cefc769-6ac1-4466-bcb4-bc1928dfce9e" (UID: "3cefc769-6ac1-4466-bcb4-bc1928dfce9e"). InnerVolumeSpecName "kube-api-access-tc4gv". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:43:22 crc kubenswrapper[5052]: I1203 13:43:22.556800 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3cefc769-6ac1-4466-bcb4-bc1928dfce9e-config-data" (OuterVolumeSpecName: "config-data") pod "3cefc769-6ac1-4466-bcb4-bc1928dfce9e" (UID: "3cefc769-6ac1-4466-bcb4-bc1928dfce9e"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:43:22 crc kubenswrapper[5052]: I1203 13:43:22.617696 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-tc4gv\" (UniqueName: \"kubernetes.io/projected/3cefc769-6ac1-4466-bcb4-bc1928dfce9e-kube-api-access-tc4gv\") on node \"crc\" DevicePath \"\"" Dec 03 13:43:22 crc kubenswrapper[5052]: I1203 13:43:22.617729 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/3cefc769-6ac1-4466-bcb4-bc1928dfce9e-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 13:43:23 crc kubenswrapper[5052]: I1203 13:43:23.083873 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="horizon-kuttl-tests/keystone-db-sync-2qj57" event={"ID":"3cefc769-6ac1-4466-bcb4-bc1928dfce9e","Type":"ContainerDied","Data":"610354f9fb50c7dbfda327519d396e09b7928db7639a32bbd4ca5aa25fbe105f"} Dec 03 13:43:23 crc kubenswrapper[5052]: I1203 13:43:23.083910 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="610354f9fb50c7dbfda327519d396e09b7928db7639a32bbd4ca5aa25fbe105f" Dec 03 13:43:23 crc kubenswrapper[5052]: I1203 13:43:23.083912 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="horizon-kuttl-tests/keystone-db-sync-2qj57" Dec 03 13:43:23 crc kubenswrapper[5052]: I1203 13:43:23.269365 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["horizon-kuttl-tests/keystone-bootstrap-k99zw"] Dec 03 13:43:23 crc kubenswrapper[5052]: E1203 13:43:23.269739 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="3cefc769-6ac1-4466-bcb4-bc1928dfce9e" containerName="keystone-db-sync" Dec 03 13:43:23 crc kubenswrapper[5052]: I1203 13:43:23.269760 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="3cefc769-6ac1-4466-bcb4-bc1928dfce9e" containerName="keystone-db-sync" Dec 03 13:43:23 crc kubenswrapper[5052]: I1203 13:43:23.269960 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="3cefc769-6ac1-4466-bcb4-bc1928dfce9e" containerName="keystone-db-sync" Dec 03 13:43:23 crc kubenswrapper[5052]: I1203 13:43:23.270532 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="horizon-kuttl-tests/keystone-bootstrap-k99zw" Dec 03 13:43:23 crc kubenswrapper[5052]: I1203 13:43:23.280103 5052 reflector.go:368] Caches populated for *v1.Secret from object-"horizon-kuttl-tests"/"keystone" Dec 03 13:43:23 crc kubenswrapper[5052]: I1203 13:43:23.280233 5052 reflector.go:368] Caches populated for *v1.Secret from object-"horizon-kuttl-tests"/"osp-secret" Dec 03 13:43:23 crc kubenswrapper[5052]: I1203 13:43:23.280556 5052 reflector.go:368] Caches populated for *v1.Secret from object-"horizon-kuttl-tests"/"keystone-scripts" Dec 03 13:43:23 crc kubenswrapper[5052]: I1203 13:43:23.281904 5052 reflector.go:368] Caches populated for *v1.Secret from object-"horizon-kuttl-tests"/"keystone-config-data" Dec 03 13:43:23 crc kubenswrapper[5052]: I1203 13:43:23.283859 5052 reflector.go:368] Caches populated for *v1.Secret from object-"horizon-kuttl-tests"/"keystone-keystone-dockercfg-kqrlh" Dec 03 13:43:23 crc kubenswrapper[5052]: I1203 13:43:23.289821 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["horizon-kuttl-tests/keystone-bootstrap-k99zw"] Dec 03 13:43:23 crc kubenswrapper[5052]: I1203 13:43:23.428546 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/eb70247b-750b-4da4-9f00-f9e6085163da-config-data\") pod \"keystone-bootstrap-k99zw\" (UID: \"eb70247b-750b-4da4-9f00-f9e6085163da\") " pod="horizon-kuttl-tests/keystone-bootstrap-k99zw" Dec 03 13:43:23 crc kubenswrapper[5052]: I1203 13:43:23.429299 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/eb70247b-750b-4da4-9f00-f9e6085163da-credential-keys\") pod \"keystone-bootstrap-k99zw\" (UID: \"eb70247b-750b-4da4-9f00-f9e6085163da\") " pod="horizon-kuttl-tests/keystone-bootstrap-k99zw" Dec 03 13:43:23 crc kubenswrapper[5052]: I1203 13:43:23.429377 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/eb70247b-750b-4da4-9f00-f9e6085163da-fernet-keys\") pod \"keystone-bootstrap-k99zw\" (UID: \"eb70247b-750b-4da4-9f00-f9e6085163da\") " pod="horizon-kuttl-tests/keystone-bootstrap-k99zw" Dec 03 13:43:23 crc kubenswrapper[5052]: I1203 13:43:23.429421 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/eb70247b-750b-4da4-9f00-f9e6085163da-scripts\") pod \"keystone-bootstrap-k99zw\" (UID: \"eb70247b-750b-4da4-9f00-f9e6085163da\") " pod="horizon-kuttl-tests/keystone-bootstrap-k99zw" Dec 03 13:43:23 crc kubenswrapper[5052]: I1203 13:43:23.429481 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f6896\" (UniqueName: \"kubernetes.io/projected/eb70247b-750b-4da4-9f00-f9e6085163da-kube-api-access-f6896\") pod \"keystone-bootstrap-k99zw\" (UID: \"eb70247b-750b-4da4-9f00-f9e6085163da\") " pod="horizon-kuttl-tests/keystone-bootstrap-k99zw" Dec 03 13:43:23 crc kubenswrapper[5052]: I1203 13:43:23.530316 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/eb70247b-750b-4da4-9f00-f9e6085163da-config-data\") pod \"keystone-bootstrap-k99zw\" (UID: \"eb70247b-750b-4da4-9f00-f9e6085163da\") " pod="horizon-kuttl-tests/keystone-bootstrap-k99zw" Dec 03 13:43:23 crc kubenswrapper[5052]: I1203 13:43:23.530578 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/eb70247b-750b-4da4-9f00-f9e6085163da-credential-keys\") pod \"keystone-bootstrap-k99zw\" (UID: \"eb70247b-750b-4da4-9f00-f9e6085163da\") " pod="horizon-kuttl-tests/keystone-bootstrap-k99zw" Dec 03 13:43:23 crc kubenswrapper[5052]: I1203 13:43:23.530686 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/eb70247b-750b-4da4-9f00-f9e6085163da-fernet-keys\") pod \"keystone-bootstrap-k99zw\" (UID: \"eb70247b-750b-4da4-9f00-f9e6085163da\") " pod="horizon-kuttl-tests/keystone-bootstrap-k99zw" Dec 03 13:43:23 crc kubenswrapper[5052]: I1203 13:43:23.530796 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/eb70247b-750b-4da4-9f00-f9e6085163da-scripts\") pod \"keystone-bootstrap-k99zw\" (UID: \"eb70247b-750b-4da4-9f00-f9e6085163da\") " pod="horizon-kuttl-tests/keystone-bootstrap-k99zw" Dec 03 13:43:23 crc kubenswrapper[5052]: I1203 13:43:23.530926 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-f6896\" (UniqueName: \"kubernetes.io/projected/eb70247b-750b-4da4-9f00-f9e6085163da-kube-api-access-f6896\") pod \"keystone-bootstrap-k99zw\" (UID: \"eb70247b-750b-4da4-9f00-f9e6085163da\") " pod="horizon-kuttl-tests/keystone-bootstrap-k99zw" Dec 03 13:43:23 crc kubenswrapper[5052]: I1203 13:43:23.535366 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/eb70247b-750b-4da4-9f00-f9e6085163da-scripts\") pod \"keystone-bootstrap-k99zw\" (UID: \"eb70247b-750b-4da4-9f00-f9e6085163da\") " pod="horizon-kuttl-tests/keystone-bootstrap-k99zw" Dec 03 13:43:23 crc kubenswrapper[5052]: I1203 13:43:23.535590 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/eb70247b-750b-4da4-9f00-f9e6085163da-fernet-keys\") pod \"keystone-bootstrap-k99zw\" (UID: \"eb70247b-750b-4da4-9f00-f9e6085163da\") " pod="horizon-kuttl-tests/keystone-bootstrap-k99zw" Dec 03 13:43:23 crc kubenswrapper[5052]: I1203 13:43:23.540033 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/eb70247b-750b-4da4-9f00-f9e6085163da-credential-keys\") pod \"keystone-bootstrap-k99zw\" (UID: \"eb70247b-750b-4da4-9f00-f9e6085163da\") " pod="horizon-kuttl-tests/keystone-bootstrap-k99zw" Dec 03 13:43:23 crc kubenswrapper[5052]: I1203 13:43:23.540626 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/eb70247b-750b-4da4-9f00-f9e6085163da-config-data\") pod \"keystone-bootstrap-k99zw\" (UID: \"eb70247b-750b-4da4-9f00-f9e6085163da\") " pod="horizon-kuttl-tests/keystone-bootstrap-k99zw" Dec 03 13:43:23 crc kubenswrapper[5052]: I1203 13:43:23.551111 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-f6896\" (UniqueName: \"kubernetes.io/projected/eb70247b-750b-4da4-9f00-f9e6085163da-kube-api-access-f6896\") pod \"keystone-bootstrap-k99zw\" (UID: \"eb70247b-750b-4da4-9f00-f9e6085163da\") " pod="horizon-kuttl-tests/keystone-bootstrap-k99zw" Dec 03 13:43:23 crc kubenswrapper[5052]: I1203 13:43:23.587443 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="horizon-kuttl-tests/keystone-bootstrap-k99zw" Dec 03 13:43:24 crc kubenswrapper[5052]: I1203 13:43:24.045045 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["horizon-kuttl-tests/keystone-bootstrap-k99zw"] Dec 03 13:43:24 crc kubenswrapper[5052]: I1203 13:43:24.091539 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="horizon-kuttl-tests/keystone-bootstrap-k99zw" event={"ID":"eb70247b-750b-4da4-9f00-f9e6085163da","Type":"ContainerStarted","Data":"d26663bb009c494bf375c2e46d3b5843d1ac0eb89330bf4b9c9bae9a29f56dcf"} Dec 03 13:43:25 crc kubenswrapper[5052]: I1203 13:43:25.100942 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="horizon-kuttl-tests/keystone-bootstrap-k99zw" event={"ID":"eb70247b-750b-4da4-9f00-f9e6085163da","Type":"ContainerStarted","Data":"44d0a04f74d3dfb319e297b561d5e3800514c62116cc94cb9090d80eb040c731"} Dec 03 13:43:25 crc kubenswrapper[5052]: I1203 13:43:25.145257 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="horizon-kuttl-tests/keystone-bootstrap-k99zw" podStartSLOduration=2.14523388 podStartE2EDuration="2.14523388s" podCreationTimestamp="2025-12-03 13:43:23 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 13:43:25.118410711 +0000 UTC m=+8595.706652475" watchObservedRunningTime="2025-12-03 13:43:25.14523388 +0000 UTC m=+8595.733475644" Dec 03 13:43:25 crc kubenswrapper[5052]: I1203 13:43:25.673568 5052 scope.go:117] "RemoveContainer" containerID="2fbd293d6edef0fa059e91c119312e77b262534107c322feb7b59dbea3464526" Dec 03 13:43:25 crc kubenswrapper[5052]: E1203 13:43:25.673811 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 13:43:27 crc kubenswrapper[5052]: I1203 13:43:27.116344 5052 generic.go:334] "Generic (PLEG): container finished" podID="eb70247b-750b-4da4-9f00-f9e6085163da" containerID="44d0a04f74d3dfb319e297b561d5e3800514c62116cc94cb9090d80eb040c731" exitCode=0 Dec 03 13:43:27 crc kubenswrapper[5052]: I1203 13:43:27.116429 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="horizon-kuttl-tests/keystone-bootstrap-k99zw" event={"ID":"eb70247b-750b-4da4-9f00-f9e6085163da","Type":"ContainerDied","Data":"44d0a04f74d3dfb319e297b561d5e3800514c62116cc94cb9090d80eb040c731"} Dec 03 13:43:28 crc kubenswrapper[5052]: I1203 13:43:28.506861 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="horizon-kuttl-tests/keystone-bootstrap-k99zw" Dec 03 13:43:28 crc kubenswrapper[5052]: I1203 13:43:28.616173 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/eb70247b-750b-4da4-9f00-f9e6085163da-fernet-keys\") pod \"eb70247b-750b-4da4-9f00-f9e6085163da\" (UID: \"eb70247b-750b-4da4-9f00-f9e6085163da\") " Dec 03 13:43:28 crc kubenswrapper[5052]: I1203 13:43:28.616257 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f6896\" (UniqueName: \"kubernetes.io/projected/eb70247b-750b-4da4-9f00-f9e6085163da-kube-api-access-f6896\") pod \"eb70247b-750b-4da4-9f00-f9e6085163da\" (UID: \"eb70247b-750b-4da4-9f00-f9e6085163da\") " Dec 03 13:43:28 crc kubenswrapper[5052]: I1203 13:43:28.616369 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/eb70247b-750b-4da4-9f00-f9e6085163da-config-data\") pod \"eb70247b-750b-4da4-9f00-f9e6085163da\" (UID: \"eb70247b-750b-4da4-9f00-f9e6085163da\") " Dec 03 13:43:28 crc kubenswrapper[5052]: I1203 13:43:28.616409 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/eb70247b-750b-4da4-9f00-f9e6085163da-credential-keys\") pod \"eb70247b-750b-4da4-9f00-f9e6085163da\" (UID: \"eb70247b-750b-4da4-9f00-f9e6085163da\") " Dec 03 13:43:28 crc kubenswrapper[5052]: I1203 13:43:28.616466 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/eb70247b-750b-4da4-9f00-f9e6085163da-scripts\") pod \"eb70247b-750b-4da4-9f00-f9e6085163da\" (UID: \"eb70247b-750b-4da4-9f00-f9e6085163da\") " Dec 03 13:43:28 crc kubenswrapper[5052]: I1203 13:43:28.622014 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eb70247b-750b-4da4-9f00-f9e6085163da-scripts" (OuterVolumeSpecName: "scripts") pod "eb70247b-750b-4da4-9f00-f9e6085163da" (UID: "eb70247b-750b-4da4-9f00-f9e6085163da"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:43:28 crc kubenswrapper[5052]: I1203 13:43:28.622432 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eb70247b-750b-4da4-9f00-f9e6085163da-kube-api-access-f6896" (OuterVolumeSpecName: "kube-api-access-f6896") pod "eb70247b-750b-4da4-9f00-f9e6085163da" (UID: "eb70247b-750b-4da4-9f00-f9e6085163da"). InnerVolumeSpecName "kube-api-access-f6896". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:43:28 crc kubenswrapper[5052]: I1203 13:43:28.623269 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eb70247b-750b-4da4-9f00-f9e6085163da-fernet-keys" (OuterVolumeSpecName: "fernet-keys") pod "eb70247b-750b-4da4-9f00-f9e6085163da" (UID: "eb70247b-750b-4da4-9f00-f9e6085163da"). InnerVolumeSpecName "fernet-keys". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:43:28 crc kubenswrapper[5052]: I1203 13:43:28.624048 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eb70247b-750b-4da4-9f00-f9e6085163da-credential-keys" (OuterVolumeSpecName: "credential-keys") pod "eb70247b-750b-4da4-9f00-f9e6085163da" (UID: "eb70247b-750b-4da4-9f00-f9e6085163da"). InnerVolumeSpecName "credential-keys". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:43:28 crc kubenswrapper[5052]: I1203 13:43:28.644600 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eb70247b-750b-4da4-9f00-f9e6085163da-config-data" (OuterVolumeSpecName: "config-data") pod "eb70247b-750b-4da4-9f00-f9e6085163da" (UID: "eb70247b-750b-4da4-9f00-f9e6085163da"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:43:28 crc kubenswrapper[5052]: I1203 13:43:28.718271 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-f6896\" (UniqueName: \"kubernetes.io/projected/eb70247b-750b-4da4-9f00-f9e6085163da-kube-api-access-f6896\") on node \"crc\" DevicePath \"\"" Dec 03 13:43:28 crc kubenswrapper[5052]: I1203 13:43:28.718320 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/eb70247b-750b-4da4-9f00-f9e6085163da-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 13:43:28 crc kubenswrapper[5052]: I1203 13:43:28.718336 5052 reconciler_common.go:293] "Volume detached for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/eb70247b-750b-4da4-9f00-f9e6085163da-credential-keys\") on node \"crc\" DevicePath \"\"" Dec 03 13:43:28 crc kubenswrapper[5052]: I1203 13:43:28.718347 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/eb70247b-750b-4da4-9f00-f9e6085163da-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 13:43:28 crc kubenswrapper[5052]: I1203 13:43:28.718360 5052 reconciler_common.go:293] "Volume detached for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/eb70247b-750b-4da4-9f00-f9e6085163da-fernet-keys\") on node \"crc\" DevicePath \"\"" Dec 03 13:43:29 crc kubenswrapper[5052]: I1203 13:43:29.131361 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="horizon-kuttl-tests/keystone-bootstrap-k99zw" event={"ID":"eb70247b-750b-4da4-9f00-f9e6085163da","Type":"ContainerDied","Data":"d26663bb009c494bf375c2e46d3b5843d1ac0eb89330bf4b9c9bae9a29f56dcf"} Dec 03 13:43:29 crc kubenswrapper[5052]: I1203 13:43:29.131418 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d26663bb009c494bf375c2e46d3b5843d1ac0eb89330bf4b9c9bae9a29f56dcf" Dec 03 13:43:29 crc kubenswrapper[5052]: I1203 13:43:29.131421 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="horizon-kuttl-tests/keystone-bootstrap-k99zw" Dec 03 13:43:29 crc kubenswrapper[5052]: I1203 13:43:29.696172 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["horizon-kuttl-tests/keystone-68bf6dbf4f-g4nbm"] Dec 03 13:43:29 crc kubenswrapper[5052]: E1203 13:43:29.701222 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="eb70247b-750b-4da4-9f00-f9e6085163da" containerName="keystone-bootstrap" Dec 03 13:43:29 crc kubenswrapper[5052]: I1203 13:43:29.701257 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="eb70247b-750b-4da4-9f00-f9e6085163da" containerName="keystone-bootstrap" Dec 03 13:43:29 crc kubenswrapper[5052]: I1203 13:43:29.701412 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="eb70247b-750b-4da4-9f00-f9e6085163da" containerName="keystone-bootstrap" Dec 03 13:43:29 crc kubenswrapper[5052]: I1203 13:43:29.702144 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="horizon-kuttl-tests/keystone-68bf6dbf4f-g4nbm" Dec 03 13:43:29 crc kubenswrapper[5052]: I1203 13:43:29.704720 5052 reflector.go:368] Caches populated for *v1.Secret from object-"horizon-kuttl-tests"/"keystone" Dec 03 13:43:29 crc kubenswrapper[5052]: I1203 13:43:29.704831 5052 reflector.go:368] Caches populated for *v1.Secret from object-"horizon-kuttl-tests"/"keystone-config-data" Dec 03 13:43:29 crc kubenswrapper[5052]: I1203 13:43:29.704993 5052 reflector.go:368] Caches populated for *v1.Secret from object-"horizon-kuttl-tests"/"keystone-scripts" Dec 03 13:43:29 crc kubenswrapper[5052]: I1203 13:43:29.705022 5052 reflector.go:368] Caches populated for *v1.Secret from object-"horizon-kuttl-tests"/"keystone-keystone-dockercfg-kqrlh" Dec 03 13:43:29 crc kubenswrapper[5052]: I1203 13:43:29.706280 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["horizon-kuttl-tests/keystone-68bf6dbf4f-g4nbm"] Dec 03 13:43:29 crc kubenswrapper[5052]: I1203 13:43:29.835844 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/d8df61e7-dad4-4072-a268-ddc6b55f0696-scripts\") pod \"keystone-68bf6dbf4f-g4nbm\" (UID: \"d8df61e7-dad4-4072-a268-ddc6b55f0696\") " pod="horizon-kuttl-tests/keystone-68bf6dbf4f-g4nbm" Dec 03 13:43:29 crc kubenswrapper[5052]: I1203 13:43:29.836089 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z46st\" (UniqueName: \"kubernetes.io/projected/d8df61e7-dad4-4072-a268-ddc6b55f0696-kube-api-access-z46st\") pod \"keystone-68bf6dbf4f-g4nbm\" (UID: \"d8df61e7-dad4-4072-a268-ddc6b55f0696\") " pod="horizon-kuttl-tests/keystone-68bf6dbf4f-g4nbm" Dec 03 13:43:29 crc kubenswrapper[5052]: I1203 13:43:29.836219 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/d8df61e7-dad4-4072-a268-ddc6b55f0696-fernet-keys\") pod \"keystone-68bf6dbf4f-g4nbm\" (UID: \"d8df61e7-dad4-4072-a268-ddc6b55f0696\") " pod="horizon-kuttl-tests/keystone-68bf6dbf4f-g4nbm" Dec 03 13:43:29 crc kubenswrapper[5052]: I1203 13:43:29.836303 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/d8df61e7-dad4-4072-a268-ddc6b55f0696-config-data\") pod \"keystone-68bf6dbf4f-g4nbm\" (UID: \"d8df61e7-dad4-4072-a268-ddc6b55f0696\") " pod="horizon-kuttl-tests/keystone-68bf6dbf4f-g4nbm" Dec 03 13:43:29 crc kubenswrapper[5052]: I1203 13:43:29.836370 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/d8df61e7-dad4-4072-a268-ddc6b55f0696-credential-keys\") pod \"keystone-68bf6dbf4f-g4nbm\" (UID: \"d8df61e7-dad4-4072-a268-ddc6b55f0696\") " pod="horizon-kuttl-tests/keystone-68bf6dbf4f-g4nbm" Dec 03 13:43:29 crc kubenswrapper[5052]: I1203 13:43:29.938079 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/d8df61e7-dad4-4072-a268-ddc6b55f0696-scripts\") pod \"keystone-68bf6dbf4f-g4nbm\" (UID: \"d8df61e7-dad4-4072-a268-ddc6b55f0696\") " pod="horizon-kuttl-tests/keystone-68bf6dbf4f-g4nbm" Dec 03 13:43:29 crc kubenswrapper[5052]: I1203 13:43:29.938167 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-z46st\" (UniqueName: \"kubernetes.io/projected/d8df61e7-dad4-4072-a268-ddc6b55f0696-kube-api-access-z46st\") pod \"keystone-68bf6dbf4f-g4nbm\" (UID: \"d8df61e7-dad4-4072-a268-ddc6b55f0696\") " pod="horizon-kuttl-tests/keystone-68bf6dbf4f-g4nbm" Dec 03 13:43:29 crc kubenswrapper[5052]: I1203 13:43:29.938200 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/d8df61e7-dad4-4072-a268-ddc6b55f0696-fernet-keys\") pod \"keystone-68bf6dbf4f-g4nbm\" (UID: \"d8df61e7-dad4-4072-a268-ddc6b55f0696\") " pod="horizon-kuttl-tests/keystone-68bf6dbf4f-g4nbm" Dec 03 13:43:29 crc kubenswrapper[5052]: I1203 13:43:29.938243 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/d8df61e7-dad4-4072-a268-ddc6b55f0696-config-data\") pod \"keystone-68bf6dbf4f-g4nbm\" (UID: \"d8df61e7-dad4-4072-a268-ddc6b55f0696\") " pod="horizon-kuttl-tests/keystone-68bf6dbf4f-g4nbm" Dec 03 13:43:29 crc kubenswrapper[5052]: I1203 13:43:29.938269 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/d8df61e7-dad4-4072-a268-ddc6b55f0696-credential-keys\") pod \"keystone-68bf6dbf4f-g4nbm\" (UID: \"d8df61e7-dad4-4072-a268-ddc6b55f0696\") " pod="horizon-kuttl-tests/keystone-68bf6dbf4f-g4nbm" Dec 03 13:43:29 crc kubenswrapper[5052]: I1203 13:43:29.942974 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/d8df61e7-dad4-4072-a268-ddc6b55f0696-credential-keys\") pod \"keystone-68bf6dbf4f-g4nbm\" (UID: \"d8df61e7-dad4-4072-a268-ddc6b55f0696\") " pod="horizon-kuttl-tests/keystone-68bf6dbf4f-g4nbm" Dec 03 13:43:29 crc kubenswrapper[5052]: I1203 13:43:29.944313 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/d8df61e7-dad4-4072-a268-ddc6b55f0696-scripts\") pod \"keystone-68bf6dbf4f-g4nbm\" (UID: \"d8df61e7-dad4-4072-a268-ddc6b55f0696\") " pod="horizon-kuttl-tests/keystone-68bf6dbf4f-g4nbm" Dec 03 13:43:29 crc kubenswrapper[5052]: I1203 13:43:29.944439 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/d8df61e7-dad4-4072-a268-ddc6b55f0696-fernet-keys\") pod \"keystone-68bf6dbf4f-g4nbm\" (UID: \"d8df61e7-dad4-4072-a268-ddc6b55f0696\") " pod="horizon-kuttl-tests/keystone-68bf6dbf4f-g4nbm" Dec 03 13:43:29 crc kubenswrapper[5052]: I1203 13:43:29.945845 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/d8df61e7-dad4-4072-a268-ddc6b55f0696-config-data\") pod \"keystone-68bf6dbf4f-g4nbm\" (UID: \"d8df61e7-dad4-4072-a268-ddc6b55f0696\") " pod="horizon-kuttl-tests/keystone-68bf6dbf4f-g4nbm" Dec 03 13:43:29 crc kubenswrapper[5052]: I1203 13:43:29.957809 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-z46st\" (UniqueName: \"kubernetes.io/projected/d8df61e7-dad4-4072-a268-ddc6b55f0696-kube-api-access-z46st\") pod \"keystone-68bf6dbf4f-g4nbm\" (UID: \"d8df61e7-dad4-4072-a268-ddc6b55f0696\") " pod="horizon-kuttl-tests/keystone-68bf6dbf4f-g4nbm" Dec 03 13:43:30 crc kubenswrapper[5052]: I1203 13:43:30.027865 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="horizon-kuttl-tests/keystone-68bf6dbf4f-g4nbm" Dec 03 13:43:30 crc kubenswrapper[5052]: I1203 13:43:30.473427 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["horizon-kuttl-tests/keystone-68bf6dbf4f-g4nbm"] Dec 03 13:43:31 crc kubenswrapper[5052]: I1203 13:43:31.150756 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="horizon-kuttl-tests/keystone-68bf6dbf4f-g4nbm" event={"ID":"d8df61e7-dad4-4072-a268-ddc6b55f0696","Type":"ContainerStarted","Data":"e698f2ff7e2689a1f1a877cb6db7518a6479f4b8d5bedb508f36e4f0ed3c80c0"} Dec 03 13:43:31 crc kubenswrapper[5052]: I1203 13:43:31.151250 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="horizon-kuttl-tests/keystone-68bf6dbf4f-g4nbm" Dec 03 13:43:31 crc kubenswrapper[5052]: I1203 13:43:31.151271 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="horizon-kuttl-tests/keystone-68bf6dbf4f-g4nbm" event={"ID":"d8df61e7-dad4-4072-a268-ddc6b55f0696","Type":"ContainerStarted","Data":"f9e6e6f32051263e5de536ea7744589066d3aca4ec9499a1d51c54be42f67fd9"} Dec 03 13:43:31 crc kubenswrapper[5052]: I1203 13:43:31.172544 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="horizon-kuttl-tests/keystone-68bf6dbf4f-g4nbm" podStartSLOduration=2.172503534 podStartE2EDuration="2.172503534s" podCreationTimestamp="2025-12-03 13:43:29 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 13:43:31.167887032 +0000 UTC m=+8601.756128796" watchObservedRunningTime="2025-12-03 13:43:31.172503534 +0000 UTC m=+8601.760745298" Dec 03 13:43:33 crc kubenswrapper[5052]: I1203 13:43:33.306909 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openstack-operators/horizon-operator-controller-manager-9b47db64f-zztn2"] Dec 03 13:43:33 crc kubenswrapper[5052]: I1203 13:43:33.308364 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-operators/horizon-operator-controller-manager-9b47db64f-zztn2" Dec 03 13:43:33 crc kubenswrapper[5052]: I1203 13:43:33.310216 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-operators"/"horizon-operator-controller-manager-dockercfg-25c2d" Dec 03 13:43:33 crc kubenswrapper[5052]: I1203 13:43:33.310697 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openstack-operators"/"horizon-operator-controller-manager-service-cert" Dec 03 13:43:33 crc kubenswrapper[5052]: I1203 13:43:33.328758 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-operators/horizon-operator-controller-manager-9b47db64f-zztn2"] Dec 03 13:43:33 crc kubenswrapper[5052]: I1203 13:43:33.388316 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pdtdk\" (UniqueName: \"kubernetes.io/projected/5b2f3c3a-2756-4269-99da-ba83e8097c9e-kube-api-access-pdtdk\") pod \"horizon-operator-controller-manager-9b47db64f-zztn2\" (UID: \"5b2f3c3a-2756-4269-99da-ba83e8097c9e\") " pod="openstack-operators/horizon-operator-controller-manager-9b47db64f-zztn2" Dec 03 13:43:33 crc kubenswrapper[5052]: I1203 13:43:33.388411 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5b2f3c3a-2756-4269-99da-ba83e8097c9e-webhook-cert\") pod \"horizon-operator-controller-manager-9b47db64f-zztn2\" (UID: \"5b2f3c3a-2756-4269-99da-ba83e8097c9e\") " pod="openstack-operators/horizon-operator-controller-manager-9b47db64f-zztn2" Dec 03 13:43:33 crc kubenswrapper[5052]: I1203 13:43:33.388494 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/5b2f3c3a-2756-4269-99da-ba83e8097c9e-apiservice-cert\") pod \"horizon-operator-controller-manager-9b47db64f-zztn2\" (UID: \"5b2f3c3a-2756-4269-99da-ba83e8097c9e\") " pod="openstack-operators/horizon-operator-controller-manager-9b47db64f-zztn2" Dec 03 13:43:33 crc kubenswrapper[5052]: I1203 13:43:33.489405 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5b2f3c3a-2756-4269-99da-ba83e8097c9e-webhook-cert\") pod \"horizon-operator-controller-manager-9b47db64f-zztn2\" (UID: \"5b2f3c3a-2756-4269-99da-ba83e8097c9e\") " pod="openstack-operators/horizon-operator-controller-manager-9b47db64f-zztn2" Dec 03 13:43:33 crc kubenswrapper[5052]: I1203 13:43:33.489480 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/5b2f3c3a-2756-4269-99da-ba83e8097c9e-apiservice-cert\") pod \"horizon-operator-controller-manager-9b47db64f-zztn2\" (UID: \"5b2f3c3a-2756-4269-99da-ba83e8097c9e\") " pod="openstack-operators/horizon-operator-controller-manager-9b47db64f-zztn2" Dec 03 13:43:33 crc kubenswrapper[5052]: I1203 13:43:33.489547 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-pdtdk\" (UniqueName: \"kubernetes.io/projected/5b2f3c3a-2756-4269-99da-ba83e8097c9e-kube-api-access-pdtdk\") pod \"horizon-operator-controller-manager-9b47db64f-zztn2\" (UID: \"5b2f3c3a-2756-4269-99da-ba83e8097c9e\") " pod="openstack-operators/horizon-operator-controller-manager-9b47db64f-zztn2" Dec 03 13:43:33 crc kubenswrapper[5052]: I1203 13:43:33.494647 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5b2f3c3a-2756-4269-99da-ba83e8097c9e-webhook-cert\") pod \"horizon-operator-controller-manager-9b47db64f-zztn2\" (UID: \"5b2f3c3a-2756-4269-99da-ba83e8097c9e\") " pod="openstack-operators/horizon-operator-controller-manager-9b47db64f-zztn2" Dec 03 13:43:33 crc kubenswrapper[5052]: I1203 13:43:33.494670 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/5b2f3c3a-2756-4269-99da-ba83e8097c9e-apiservice-cert\") pod \"horizon-operator-controller-manager-9b47db64f-zztn2\" (UID: \"5b2f3c3a-2756-4269-99da-ba83e8097c9e\") " pod="openstack-operators/horizon-operator-controller-manager-9b47db64f-zztn2" Dec 03 13:43:33 crc kubenswrapper[5052]: I1203 13:43:33.504981 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-pdtdk\" (UniqueName: \"kubernetes.io/projected/5b2f3c3a-2756-4269-99da-ba83e8097c9e-kube-api-access-pdtdk\") pod \"horizon-operator-controller-manager-9b47db64f-zztn2\" (UID: \"5b2f3c3a-2756-4269-99da-ba83e8097c9e\") " pod="openstack-operators/horizon-operator-controller-manager-9b47db64f-zztn2" Dec 03 13:43:33 crc kubenswrapper[5052]: I1203 13:43:33.629611 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openstack-operators/horizon-operator-controller-manager-9b47db64f-zztn2" Dec 03 13:43:34 crc kubenswrapper[5052]: I1203 13:43:34.044813 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openstack-operators/horizon-operator-controller-manager-9b47db64f-zztn2"] Dec 03 13:43:34 crc kubenswrapper[5052]: W1203 13:43:34.049640 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b2f3c3a_2756_4269_99da_ba83e8097c9e.slice/crio-04d3e96435d5a9f855434516b0e8b515b923608601014363b5cade655605fb91 WatchSource:0}: Error finding container 04d3e96435d5a9f855434516b0e8b515b923608601014363b5cade655605fb91: Status 404 returned error can't find the container with id 04d3e96435d5a9f855434516b0e8b515b923608601014363b5cade655605fb91 Dec 03 13:43:34 crc kubenswrapper[5052]: I1203 13:43:34.174841 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/horizon-operator-controller-manager-9b47db64f-zztn2" event={"ID":"5b2f3c3a-2756-4269-99da-ba83e8097c9e","Type":"ContainerStarted","Data":"04d3e96435d5a9f855434516b0e8b515b923608601014363b5cade655605fb91"} Dec 03 13:43:35 crc kubenswrapper[5052]: I1203 13:43:35.183618 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/horizon-operator-controller-manager-9b47db64f-zztn2" event={"ID":"5b2f3c3a-2756-4269-99da-ba83e8097c9e","Type":"ContainerStarted","Data":"6075c6b42637b01fc6c756047fec5638fa9a992df8271f644096237de65e87d9"} Dec 03 13:43:35 crc kubenswrapper[5052]: I1203 13:43:35.184057 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openstack-operators/horizon-operator-controller-manager-9b47db64f-zztn2" Dec 03 13:43:35 crc kubenswrapper[5052]: I1203 13:43:35.202568 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openstack-operators/horizon-operator-controller-manager-9b47db64f-zztn2" podStartSLOduration=2.202533201 podStartE2EDuration="2.202533201s" podCreationTimestamp="2025-12-03 13:43:33 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 13:43:35.198284599 +0000 UTC m=+8605.786526373" watchObservedRunningTime="2025-12-03 13:43:35.202533201 +0000 UTC m=+8605.790774965" Dec 03 13:43:36 crc kubenswrapper[5052]: I1203 13:43:36.673578 5052 scope.go:117] "RemoveContainer" containerID="2fbd293d6edef0fa059e91c119312e77b262534107c322feb7b59dbea3464526" Dec 03 13:43:36 crc kubenswrapper[5052]: E1203 13:43:36.673926 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 13:43:43 crc kubenswrapper[5052]: I1203 13:43:43.634055 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openstack-operators/horizon-operator-controller-manager-9b47db64f-zztn2" Dec 03 13:43:49 crc kubenswrapper[5052]: I1203 13:43:49.678610 5052 scope.go:117] "RemoveContainer" containerID="2fbd293d6edef0fa059e91c119312e77b262534107c322feb7b59dbea3464526" Dec 03 13:43:49 crc kubenswrapper[5052]: E1203 13:43:49.679246 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 13:44:01 crc kubenswrapper[5052]: I1203 13:44:01.602924 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="horizon-kuttl-tests/keystone-68bf6dbf4f-g4nbm" Dec 03 13:44:02 crc kubenswrapper[5052]: I1203 13:44:02.673371 5052 scope.go:117] "RemoveContainer" containerID="2fbd293d6edef0fa059e91c119312e77b262534107c322feb7b59dbea3464526" Dec 03 13:44:02 crc kubenswrapper[5052]: E1203 13:44:02.673646 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 13:44:07 crc kubenswrapper[5052]: I1203 13:44:07.837752 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["horizon-kuttl-tests/horizon-59cf497657-pm9sl"] Dec 03 13:44:07 crc kubenswrapper[5052]: I1203 13:44:07.842293 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="horizon-kuttl-tests/horizon-59cf497657-pm9sl" Dec 03 13:44:07 crc kubenswrapper[5052]: I1203 13:44:07.844318 5052 reflector.go:368] Caches populated for *v1.Secret from object-"horizon-kuttl-tests"/"horizon" Dec 03 13:44:07 crc kubenswrapper[5052]: I1203 13:44:07.846180 5052 reflector.go:368] Caches populated for *v1.Secret from object-"horizon-kuttl-tests"/"horizon-horizon-dockercfg-mkz5j" Dec 03 13:44:07 crc kubenswrapper[5052]: I1203 13:44:07.847733 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"horizon-kuttl-tests"/"horizon-config-data" Dec 03 13:44:07 crc kubenswrapper[5052]: I1203 13:44:07.857659 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["horizon-kuttl-tests/horizon-59cf497657-pm9sl"] Dec 03 13:44:07 crc kubenswrapper[5052]: I1203 13:44:07.858109 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"horizon-kuttl-tests"/"horizon-scripts" Dec 03 13:44:07 crc kubenswrapper[5052]: I1203 13:44:07.890686 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/8beb534e-1595-4943-98be-598d43afec0d-config-data\") pod \"horizon-59cf497657-pm9sl\" (UID: \"8beb534e-1595-4943-98be-598d43afec0d\") " pod="horizon-kuttl-tests/horizon-59cf497657-pm9sl" Dec 03 13:44:07 crc kubenswrapper[5052]: I1203 13:44:07.890757 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"horizon-secret-key\" (UniqueName: \"kubernetes.io/secret/8beb534e-1595-4943-98be-598d43afec0d-horizon-secret-key\") pod \"horizon-59cf497657-pm9sl\" (UID: \"8beb534e-1595-4943-98be-598d43afec0d\") " pod="horizon-kuttl-tests/horizon-59cf497657-pm9sl" Dec 03 13:44:07 crc kubenswrapper[5052]: I1203 13:44:07.890796 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/8beb534e-1595-4943-98be-598d43afec0d-logs\") pod \"horizon-59cf497657-pm9sl\" (UID: \"8beb534e-1595-4943-98be-598d43afec0d\") " pod="horizon-kuttl-tests/horizon-59cf497657-pm9sl" Dec 03 13:44:07 crc kubenswrapper[5052]: I1203 13:44:07.890848 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-std6q\" (UniqueName: \"kubernetes.io/projected/8beb534e-1595-4943-98be-598d43afec0d-kube-api-access-std6q\") pod \"horizon-59cf497657-pm9sl\" (UID: \"8beb534e-1595-4943-98be-598d43afec0d\") " pod="horizon-kuttl-tests/horizon-59cf497657-pm9sl" Dec 03 13:44:07 crc kubenswrapper[5052]: I1203 13:44:07.890873 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/8beb534e-1595-4943-98be-598d43afec0d-scripts\") pod \"horizon-59cf497657-pm9sl\" (UID: \"8beb534e-1595-4943-98be-598d43afec0d\") " pod="horizon-kuttl-tests/horizon-59cf497657-pm9sl" Dec 03 13:44:07 crc kubenswrapper[5052]: I1203 13:44:07.911759 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["horizon-kuttl-tests/horizon-577c75f77-gjq8d"] Dec 03 13:44:07 crc kubenswrapper[5052]: I1203 13:44:07.913747 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="horizon-kuttl-tests/horizon-577c75f77-gjq8d" Dec 03 13:44:07 crc kubenswrapper[5052]: I1203 13:44:07.928298 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["horizon-kuttl-tests/horizon-577c75f77-gjq8d"] Dec 03 13:44:07 crc kubenswrapper[5052]: I1203 13:44:07.992062 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/8beb534e-1595-4943-98be-598d43afec0d-config-data\") pod \"horizon-59cf497657-pm9sl\" (UID: \"8beb534e-1595-4943-98be-598d43afec0d\") " pod="horizon-kuttl-tests/horizon-59cf497657-pm9sl" Dec 03 13:44:07 crc kubenswrapper[5052]: I1203 13:44:07.992137 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"horizon-secret-key\" (UniqueName: \"kubernetes.io/secret/8beb534e-1595-4943-98be-598d43afec0d-horizon-secret-key\") pod \"horizon-59cf497657-pm9sl\" (UID: \"8beb534e-1595-4943-98be-598d43afec0d\") " pod="horizon-kuttl-tests/horizon-59cf497657-pm9sl" Dec 03 13:44:07 crc kubenswrapper[5052]: I1203 13:44:07.992538 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/8beb534e-1595-4943-98be-598d43afec0d-logs\") pod \"horizon-59cf497657-pm9sl\" (UID: \"8beb534e-1595-4943-98be-598d43afec0d\") " pod="horizon-kuttl-tests/horizon-59cf497657-pm9sl" Dec 03 13:44:07 crc kubenswrapper[5052]: I1203 13:44:07.992621 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-std6q\" (UniqueName: \"kubernetes.io/projected/8beb534e-1595-4943-98be-598d43afec0d-kube-api-access-std6q\") pod \"horizon-59cf497657-pm9sl\" (UID: \"8beb534e-1595-4943-98be-598d43afec0d\") " pod="horizon-kuttl-tests/horizon-59cf497657-pm9sl" Dec 03 13:44:07 crc kubenswrapper[5052]: I1203 13:44:07.992652 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/8beb534e-1595-4943-98be-598d43afec0d-scripts\") pod \"horizon-59cf497657-pm9sl\" (UID: \"8beb534e-1595-4943-98be-598d43afec0d\") " pod="horizon-kuttl-tests/horizon-59cf497657-pm9sl" Dec 03 13:44:07 crc kubenswrapper[5052]: I1203 13:44:07.992887 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/8beb534e-1595-4943-98be-598d43afec0d-logs\") pod \"horizon-59cf497657-pm9sl\" (UID: \"8beb534e-1595-4943-98be-598d43afec0d\") " pod="horizon-kuttl-tests/horizon-59cf497657-pm9sl" Dec 03 13:44:07 crc kubenswrapper[5052]: I1203 13:44:07.993626 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/8beb534e-1595-4943-98be-598d43afec0d-scripts\") pod \"horizon-59cf497657-pm9sl\" (UID: \"8beb534e-1595-4943-98be-598d43afec0d\") " pod="horizon-kuttl-tests/horizon-59cf497657-pm9sl" Dec 03 13:44:07 crc kubenswrapper[5052]: I1203 13:44:07.994278 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/8beb534e-1595-4943-98be-598d43afec0d-config-data\") pod \"horizon-59cf497657-pm9sl\" (UID: \"8beb534e-1595-4943-98be-598d43afec0d\") " pod="horizon-kuttl-tests/horizon-59cf497657-pm9sl" Dec 03 13:44:07 crc kubenswrapper[5052]: I1203 13:44:07.998378 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"horizon-secret-key\" (UniqueName: \"kubernetes.io/secret/8beb534e-1595-4943-98be-598d43afec0d-horizon-secret-key\") pod \"horizon-59cf497657-pm9sl\" (UID: \"8beb534e-1595-4943-98be-598d43afec0d\") " pod="horizon-kuttl-tests/horizon-59cf497657-pm9sl" Dec 03 13:44:08 crc kubenswrapper[5052]: I1203 13:44:08.008873 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-std6q\" (UniqueName: \"kubernetes.io/projected/8beb534e-1595-4943-98be-598d43afec0d-kube-api-access-std6q\") pod \"horizon-59cf497657-pm9sl\" (UID: \"8beb534e-1595-4943-98be-598d43afec0d\") " pod="horizon-kuttl-tests/horizon-59cf497657-pm9sl" Dec 03 13:44:08 crc kubenswrapper[5052]: I1203 13:44:08.094210 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/bbdb204c-cb9d-4299-b5fb-3c76fc5649ce-logs\") pod \"horizon-577c75f77-gjq8d\" (UID: \"bbdb204c-cb9d-4299-b5fb-3c76fc5649ce\") " pod="horizon-kuttl-tests/horizon-577c75f77-gjq8d" Dec 03 13:44:08 crc kubenswrapper[5052]: I1203 13:44:08.094295 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dz5jr\" (UniqueName: \"kubernetes.io/projected/bbdb204c-cb9d-4299-b5fb-3c76fc5649ce-kube-api-access-dz5jr\") pod \"horizon-577c75f77-gjq8d\" (UID: \"bbdb204c-cb9d-4299-b5fb-3c76fc5649ce\") " pod="horizon-kuttl-tests/horizon-577c75f77-gjq8d" Dec 03 13:44:08 crc kubenswrapper[5052]: I1203 13:44:08.094481 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"horizon-secret-key\" (UniqueName: \"kubernetes.io/secret/bbdb204c-cb9d-4299-b5fb-3c76fc5649ce-horizon-secret-key\") pod \"horizon-577c75f77-gjq8d\" (UID: \"bbdb204c-cb9d-4299-b5fb-3c76fc5649ce\") " pod="horizon-kuttl-tests/horizon-577c75f77-gjq8d" Dec 03 13:44:08 crc kubenswrapper[5052]: I1203 13:44:08.094546 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/bbdb204c-cb9d-4299-b5fb-3c76fc5649ce-config-data\") pod \"horizon-577c75f77-gjq8d\" (UID: \"bbdb204c-cb9d-4299-b5fb-3c76fc5649ce\") " pod="horizon-kuttl-tests/horizon-577c75f77-gjq8d" Dec 03 13:44:08 crc kubenswrapper[5052]: I1203 13:44:08.094645 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/bbdb204c-cb9d-4299-b5fb-3c76fc5649ce-scripts\") pod \"horizon-577c75f77-gjq8d\" (UID: \"bbdb204c-cb9d-4299-b5fb-3c76fc5649ce\") " pod="horizon-kuttl-tests/horizon-577c75f77-gjq8d" Dec 03 13:44:08 crc kubenswrapper[5052]: I1203 13:44:08.174869 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="horizon-kuttl-tests/horizon-59cf497657-pm9sl" Dec 03 13:44:08 crc kubenswrapper[5052]: I1203 13:44:08.195715 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-dz5jr\" (UniqueName: \"kubernetes.io/projected/bbdb204c-cb9d-4299-b5fb-3c76fc5649ce-kube-api-access-dz5jr\") pod \"horizon-577c75f77-gjq8d\" (UID: \"bbdb204c-cb9d-4299-b5fb-3c76fc5649ce\") " pod="horizon-kuttl-tests/horizon-577c75f77-gjq8d" Dec 03 13:44:08 crc kubenswrapper[5052]: I1203 13:44:08.196019 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"horizon-secret-key\" (UniqueName: \"kubernetes.io/secret/bbdb204c-cb9d-4299-b5fb-3c76fc5649ce-horizon-secret-key\") pod \"horizon-577c75f77-gjq8d\" (UID: \"bbdb204c-cb9d-4299-b5fb-3c76fc5649ce\") " pod="horizon-kuttl-tests/horizon-577c75f77-gjq8d" Dec 03 13:44:08 crc kubenswrapper[5052]: I1203 13:44:08.196042 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/bbdb204c-cb9d-4299-b5fb-3c76fc5649ce-config-data\") pod \"horizon-577c75f77-gjq8d\" (UID: \"bbdb204c-cb9d-4299-b5fb-3c76fc5649ce\") " pod="horizon-kuttl-tests/horizon-577c75f77-gjq8d" Dec 03 13:44:08 crc kubenswrapper[5052]: I1203 13:44:08.196074 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/bbdb204c-cb9d-4299-b5fb-3c76fc5649ce-scripts\") pod \"horizon-577c75f77-gjq8d\" (UID: \"bbdb204c-cb9d-4299-b5fb-3c76fc5649ce\") " pod="horizon-kuttl-tests/horizon-577c75f77-gjq8d" Dec 03 13:44:08 crc kubenswrapper[5052]: I1203 13:44:08.196149 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/bbdb204c-cb9d-4299-b5fb-3c76fc5649ce-logs\") pod \"horizon-577c75f77-gjq8d\" (UID: \"bbdb204c-cb9d-4299-b5fb-3c76fc5649ce\") " pod="horizon-kuttl-tests/horizon-577c75f77-gjq8d" Dec 03 13:44:08 crc kubenswrapper[5052]: I1203 13:44:08.196801 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/bbdb204c-cb9d-4299-b5fb-3c76fc5649ce-logs\") pod \"horizon-577c75f77-gjq8d\" (UID: \"bbdb204c-cb9d-4299-b5fb-3c76fc5649ce\") " pod="horizon-kuttl-tests/horizon-577c75f77-gjq8d" Dec 03 13:44:08 crc kubenswrapper[5052]: I1203 13:44:08.196916 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/bbdb204c-cb9d-4299-b5fb-3c76fc5649ce-scripts\") pod \"horizon-577c75f77-gjq8d\" (UID: \"bbdb204c-cb9d-4299-b5fb-3c76fc5649ce\") " pod="horizon-kuttl-tests/horizon-577c75f77-gjq8d" Dec 03 13:44:08 crc kubenswrapper[5052]: I1203 13:44:08.198572 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/bbdb204c-cb9d-4299-b5fb-3c76fc5649ce-config-data\") pod \"horizon-577c75f77-gjq8d\" (UID: \"bbdb204c-cb9d-4299-b5fb-3c76fc5649ce\") " pod="horizon-kuttl-tests/horizon-577c75f77-gjq8d" Dec 03 13:44:08 crc kubenswrapper[5052]: I1203 13:44:08.200629 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"horizon-secret-key\" (UniqueName: \"kubernetes.io/secret/bbdb204c-cb9d-4299-b5fb-3c76fc5649ce-horizon-secret-key\") pod \"horizon-577c75f77-gjq8d\" (UID: \"bbdb204c-cb9d-4299-b5fb-3c76fc5649ce\") " pod="horizon-kuttl-tests/horizon-577c75f77-gjq8d" Dec 03 13:44:08 crc kubenswrapper[5052]: I1203 13:44:08.218446 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-dz5jr\" (UniqueName: \"kubernetes.io/projected/bbdb204c-cb9d-4299-b5fb-3c76fc5649ce-kube-api-access-dz5jr\") pod \"horizon-577c75f77-gjq8d\" (UID: \"bbdb204c-cb9d-4299-b5fb-3c76fc5649ce\") " pod="horizon-kuttl-tests/horizon-577c75f77-gjq8d" Dec 03 13:44:08 crc kubenswrapper[5052]: I1203 13:44:08.231226 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="horizon-kuttl-tests/horizon-577c75f77-gjq8d" Dec 03 13:44:08 crc kubenswrapper[5052]: I1203 13:44:08.621785 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["horizon-kuttl-tests/horizon-59cf497657-pm9sl"] Dec 03 13:44:08 crc kubenswrapper[5052]: I1203 13:44:08.725809 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["horizon-kuttl-tests/horizon-577c75f77-gjq8d"] Dec 03 13:44:09 crc kubenswrapper[5052]: I1203 13:44:09.446013 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="horizon-kuttl-tests/horizon-577c75f77-gjq8d" event={"ID":"bbdb204c-cb9d-4299-b5fb-3c76fc5649ce","Type":"ContainerStarted","Data":"aba5be2d18d55a71ed507bb93dd44ecc0faa0187dd63b3d48e07980828db3d5d"} Dec 03 13:44:09 crc kubenswrapper[5052]: I1203 13:44:09.447753 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="horizon-kuttl-tests/horizon-59cf497657-pm9sl" event={"ID":"8beb534e-1595-4943-98be-598d43afec0d","Type":"ContainerStarted","Data":"88559fa26c90352ea25f76413a63d2c2df3c198cc2df2bffc32931a9b173792d"} Dec 03 13:44:13 crc kubenswrapper[5052]: I1203 13:44:13.673369 5052 scope.go:117] "RemoveContainer" containerID="2fbd293d6edef0fa059e91c119312e77b262534107c322feb7b59dbea3464526" Dec 03 13:44:13 crc kubenswrapper[5052]: E1203 13:44:13.674161 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 13:44:22 crc kubenswrapper[5052]: I1203 13:44:22.582907 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="horizon-kuttl-tests/horizon-577c75f77-gjq8d" event={"ID":"bbdb204c-cb9d-4299-b5fb-3c76fc5649ce","Type":"ContainerStarted","Data":"a13c586622babcda45ba98ecf179ccc5d357cbaa30c59aeb86edd53d3f383621"} Dec 03 13:44:22 crc kubenswrapper[5052]: I1203 13:44:22.583479 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="horizon-kuttl-tests/horizon-577c75f77-gjq8d" event={"ID":"bbdb204c-cb9d-4299-b5fb-3c76fc5649ce","Type":"ContainerStarted","Data":"b508037e0f85ac50f298e6fe9ef4374889e3706d798dd65ad3856045cefbd3e1"} Dec 03 13:44:22 crc kubenswrapper[5052]: I1203 13:44:22.584802 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="horizon-kuttl-tests/horizon-59cf497657-pm9sl" event={"ID":"8beb534e-1595-4943-98be-598d43afec0d","Type":"ContainerStarted","Data":"8aa294f51880d733dd4145d59518ff9926094a73fdcf32b47d9039b653bff497"} Dec 03 13:44:22 crc kubenswrapper[5052]: I1203 13:44:22.584854 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="horizon-kuttl-tests/horizon-59cf497657-pm9sl" event={"ID":"8beb534e-1595-4943-98be-598d43afec0d","Type":"ContainerStarted","Data":"5f2b3d40c83b81d9a2a083bd0efa68bbade657f18d624e2f09f489f9ccf1a658"} Dec 03 13:44:22 crc kubenswrapper[5052]: I1203 13:44:22.616181 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="horizon-kuttl-tests/horizon-577c75f77-gjq8d" podStartSLOduration=2.3426924319999998 podStartE2EDuration="15.616161183s" podCreationTimestamp="2025-12-03 13:44:07 +0000 UTC" firstStartedPulling="2025-12-03 13:44:08.740886658 +0000 UTC m=+8639.329128422" lastFinishedPulling="2025-12-03 13:44:22.014355409 +0000 UTC m=+8652.602597173" observedRunningTime="2025-12-03 13:44:22.59965758 +0000 UTC m=+8653.187899374" watchObservedRunningTime="2025-12-03 13:44:22.616161183 +0000 UTC m=+8653.204402947" Dec 03 13:44:22 crc kubenswrapper[5052]: I1203 13:44:22.631171 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="horizon-kuttl-tests/horizon-59cf497657-pm9sl" podStartSLOduration=2.277877342 podStartE2EDuration="15.631149523s" podCreationTimestamp="2025-12-03 13:44:07 +0000 UTC" firstStartedPulling="2025-12-03 13:44:08.637207313 +0000 UTC m=+8639.225449077" lastFinishedPulling="2025-12-03 13:44:21.990479494 +0000 UTC m=+8652.578721258" observedRunningTime="2025-12-03 13:44:22.628088566 +0000 UTC m=+8653.216330330" watchObservedRunningTime="2025-12-03 13:44:22.631149523 +0000 UTC m=+8653.219391287" Dec 03 13:44:25 crc kubenswrapper[5052]: I1203 13:44:25.672630 5052 scope.go:117] "RemoveContainer" containerID="2fbd293d6edef0fa059e91c119312e77b262534107c322feb7b59dbea3464526" Dec 03 13:44:25 crc kubenswrapper[5052]: E1203 13:44:25.673471 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 13:44:28 crc kubenswrapper[5052]: I1203 13:44:28.175695 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="horizon-kuttl-tests/horizon-59cf497657-pm9sl" Dec 03 13:44:28 crc kubenswrapper[5052]: I1203 13:44:28.176322 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="horizon-kuttl-tests/horizon-59cf497657-pm9sl" Dec 03 13:44:28 crc kubenswrapper[5052]: I1203 13:44:28.232122 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="horizon-kuttl-tests/horizon-577c75f77-gjq8d" Dec 03 13:44:28 crc kubenswrapper[5052]: I1203 13:44:28.232195 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="horizon-kuttl-tests/horizon-577c75f77-gjq8d" Dec 03 13:44:37 crc kubenswrapper[5052]: I1203 13:44:37.673530 5052 scope.go:117] "RemoveContainer" containerID="2fbd293d6edef0fa059e91c119312e77b262534107c322feb7b59dbea3464526" Dec 03 13:44:37 crc kubenswrapper[5052]: E1203 13:44:37.674366 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 13:44:38 crc kubenswrapper[5052]: I1203 13:44:38.177951 5052 prober.go:107] "Probe failed" probeType="Startup" pod="horizon-kuttl-tests/horizon-59cf497657-pm9sl" podUID="8beb534e-1595-4943-98be-598d43afec0d" containerName="horizon" probeResult="failure" output="Get \"http://10.217.0.71:8080/dashboard/auth/login/?next=/dashboard/\": dial tcp 10.217.0.71:8080: connect: connection refused" Dec 03 13:44:38 crc kubenswrapper[5052]: I1203 13:44:38.233938 5052 prober.go:107] "Probe failed" probeType="Startup" pod="horizon-kuttl-tests/horizon-577c75f77-gjq8d" podUID="bbdb204c-cb9d-4299-b5fb-3c76fc5649ce" containerName="horizon" probeResult="failure" output="Get \"http://10.217.0.72:8080/dashboard/auth/login/?next=/dashboard/\": dial tcp 10.217.0.72:8080: connect: connection refused" Dec 03 13:44:50 crc kubenswrapper[5052]: I1203 13:44:50.069366 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="horizon-kuttl-tests/horizon-59cf497657-pm9sl" Dec 03 13:44:50 crc kubenswrapper[5052]: I1203 13:44:50.148090 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="horizon-kuttl-tests/horizon-577c75f77-gjq8d" Dec 03 13:44:51 crc kubenswrapper[5052]: I1203 13:44:51.673433 5052 scope.go:117] "RemoveContainer" containerID="2fbd293d6edef0fa059e91c119312e77b262534107c322feb7b59dbea3464526" Dec 03 13:44:51 crc kubenswrapper[5052]: E1203 13:44:51.673986 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 13:44:51 crc kubenswrapper[5052]: I1203 13:44:51.877608 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="horizon-kuttl-tests/horizon-577c75f77-gjq8d" Dec 03 13:44:51 crc kubenswrapper[5052]: I1203 13:44:51.935917 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="horizon-kuttl-tests/horizon-59cf497657-pm9sl" Dec 03 13:44:51 crc kubenswrapper[5052]: I1203 13:44:51.940858 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["horizon-kuttl-tests/horizon-59cf497657-pm9sl"] Dec 03 13:44:52 crc kubenswrapper[5052]: I1203 13:44:52.818462 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="horizon-kuttl-tests/horizon-59cf497657-pm9sl" podUID="8beb534e-1595-4943-98be-598d43afec0d" containerName="horizon-log" containerID="cri-o://5f2b3d40c83b81d9a2a083bd0efa68bbade657f18d624e2f09f489f9ccf1a658" gracePeriod=30 Dec 03 13:44:52 crc kubenswrapper[5052]: I1203 13:44:52.818536 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="horizon-kuttl-tests/horizon-59cf497657-pm9sl" podUID="8beb534e-1595-4943-98be-598d43afec0d" containerName="horizon" containerID="cri-o://8aa294f51880d733dd4145d59518ff9926094a73fdcf32b47d9039b653bff497" gracePeriod=30 Dec 03 13:44:53 crc kubenswrapper[5052]: I1203 13:44:53.176383 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openshift-marketplace/redhat-operators-bppm7"] Dec 03 13:44:53 crc kubenswrapper[5052]: I1203 13:44:53.179083 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/redhat-operators-bppm7" Dec 03 13:44:53 crc kubenswrapper[5052]: I1203 13:44:53.188994 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/redhat-operators-bppm7"] Dec 03 13:44:53 crc kubenswrapper[5052]: I1203 13:44:53.268921 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/9ce53d74-f6a1-48b6-b39b-dd9b2a04a8db-utilities\") pod \"redhat-operators-bppm7\" (UID: \"9ce53d74-f6a1-48b6-b39b-dd9b2a04a8db\") " pod="openshift-marketplace/redhat-operators-bppm7" Dec 03 13:44:53 crc kubenswrapper[5052]: I1203 13:44:53.269024 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/9ce53d74-f6a1-48b6-b39b-dd9b2a04a8db-catalog-content\") pod \"redhat-operators-bppm7\" (UID: \"9ce53d74-f6a1-48b6-b39b-dd9b2a04a8db\") " pod="openshift-marketplace/redhat-operators-bppm7" Dec 03 13:44:53 crc kubenswrapper[5052]: I1203 13:44:53.269162 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dv7pn\" (UniqueName: \"kubernetes.io/projected/9ce53d74-f6a1-48b6-b39b-dd9b2a04a8db-kube-api-access-dv7pn\") pod \"redhat-operators-bppm7\" (UID: \"9ce53d74-f6a1-48b6-b39b-dd9b2a04a8db\") " pod="openshift-marketplace/redhat-operators-bppm7" Dec 03 13:44:53 crc kubenswrapper[5052]: I1203 13:44:53.370647 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-dv7pn\" (UniqueName: \"kubernetes.io/projected/9ce53d74-f6a1-48b6-b39b-dd9b2a04a8db-kube-api-access-dv7pn\") pod \"redhat-operators-bppm7\" (UID: \"9ce53d74-f6a1-48b6-b39b-dd9b2a04a8db\") " pod="openshift-marketplace/redhat-operators-bppm7" Dec 03 13:44:53 crc kubenswrapper[5052]: I1203 13:44:53.370774 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/9ce53d74-f6a1-48b6-b39b-dd9b2a04a8db-utilities\") pod \"redhat-operators-bppm7\" (UID: \"9ce53d74-f6a1-48b6-b39b-dd9b2a04a8db\") " pod="openshift-marketplace/redhat-operators-bppm7" Dec 03 13:44:53 crc kubenswrapper[5052]: I1203 13:44:53.370827 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/9ce53d74-f6a1-48b6-b39b-dd9b2a04a8db-catalog-content\") pod \"redhat-operators-bppm7\" (UID: \"9ce53d74-f6a1-48b6-b39b-dd9b2a04a8db\") " pod="openshift-marketplace/redhat-operators-bppm7" Dec 03 13:44:53 crc kubenswrapper[5052]: I1203 13:44:53.371356 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/9ce53d74-f6a1-48b6-b39b-dd9b2a04a8db-utilities\") pod \"redhat-operators-bppm7\" (UID: \"9ce53d74-f6a1-48b6-b39b-dd9b2a04a8db\") " pod="openshift-marketplace/redhat-operators-bppm7" Dec 03 13:44:53 crc kubenswrapper[5052]: I1203 13:44:53.371423 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/9ce53d74-f6a1-48b6-b39b-dd9b2a04a8db-catalog-content\") pod \"redhat-operators-bppm7\" (UID: \"9ce53d74-f6a1-48b6-b39b-dd9b2a04a8db\") " pod="openshift-marketplace/redhat-operators-bppm7" Dec 03 13:44:53 crc kubenswrapper[5052]: I1203 13:44:53.391378 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-dv7pn\" (UniqueName: \"kubernetes.io/projected/9ce53d74-f6a1-48b6-b39b-dd9b2a04a8db-kube-api-access-dv7pn\") pod \"redhat-operators-bppm7\" (UID: \"9ce53d74-f6a1-48b6-b39b-dd9b2a04a8db\") " pod="openshift-marketplace/redhat-operators-bppm7" Dec 03 13:44:53 crc kubenswrapper[5052]: I1203 13:44:53.497001 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/redhat-operators-bppm7" Dec 03 13:44:53 crc kubenswrapper[5052]: I1203 13:44:53.931185 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/redhat-operators-bppm7"] Dec 03 13:44:54 crc kubenswrapper[5052]: I1203 13:44:54.834553 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-operators-bppm7" event={"ID":"9ce53d74-f6a1-48b6-b39b-dd9b2a04a8db","Type":"ContainerStarted","Data":"522f4b8d0f62fb9f58c236d8beb2e04dc9ff1a83311f1ea8cc0c8ceeb82fefc6"} Dec 03 13:44:56 crc kubenswrapper[5052]: I1203 13:44:56.852473 5052 generic.go:334] "Generic (PLEG): container finished" podID="8beb534e-1595-4943-98be-598d43afec0d" containerID="8aa294f51880d733dd4145d59518ff9926094a73fdcf32b47d9039b653bff497" exitCode=0 Dec 03 13:44:56 crc kubenswrapper[5052]: I1203 13:44:56.852593 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="horizon-kuttl-tests/horizon-59cf497657-pm9sl" event={"ID":"8beb534e-1595-4943-98be-598d43afec0d","Type":"ContainerDied","Data":"8aa294f51880d733dd4145d59518ff9926094a73fdcf32b47d9039b653bff497"} Dec 03 13:44:56 crc kubenswrapper[5052]: I1203 13:44:56.856271 5052 generic.go:334] "Generic (PLEG): container finished" podID="9ce53d74-f6a1-48b6-b39b-dd9b2a04a8db" containerID="283137c4c8783be5b493aed31f1b56383b600fe5b95eb4668073e9d6c8275f0f" exitCode=0 Dec 03 13:44:56 crc kubenswrapper[5052]: I1203 13:44:56.856332 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-operators-bppm7" event={"ID":"9ce53d74-f6a1-48b6-b39b-dd9b2a04a8db","Type":"ContainerDied","Data":"283137c4c8783be5b493aed31f1b56383b600fe5b95eb4668073e9d6c8275f0f"} Dec 03 13:44:57 crc kubenswrapper[5052]: I1203 13:44:57.867009 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-operators-bppm7" event={"ID":"9ce53d74-f6a1-48b6-b39b-dd9b2a04a8db","Type":"ContainerStarted","Data":"ed30b7810dd296b4d5cdafb9fd446fe5d303fddd286e1b7c6990e60300d1180e"} Dec 03 13:44:57 crc kubenswrapper[5052]: I1203 13:44:57.884558 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["horizon-kuttl-tests/horizon-54f88854b8-75fgz"] Dec 03 13:44:57 crc kubenswrapper[5052]: I1203 13:44:57.885803 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="horizon-kuttl-tests/horizon-54f88854b8-75fgz" Dec 03 13:44:57 crc kubenswrapper[5052]: I1203 13:44:57.899879 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"horizon-kuttl-tests"/"horizon-policy" Dec 03 13:44:57 crc kubenswrapper[5052]: I1203 13:44:57.915817 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["horizon-kuttl-tests/horizon-54f88854b8-75fgz"] Dec 03 13:44:57 crc kubenswrapper[5052]: I1203 13:44:57.953058 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/9d264a5a-d4a9-43a0-a05e-5cbc1f5db79f-logs\") pod \"horizon-54f88854b8-75fgz\" (UID: \"9d264a5a-d4a9-43a0-a05e-5cbc1f5db79f\") " pod="horizon-kuttl-tests/horizon-54f88854b8-75fgz" Dec 03 13:44:57 crc kubenswrapper[5052]: I1203 13:44:57.953166 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z5wj8\" (UniqueName: \"kubernetes.io/projected/9d264a5a-d4a9-43a0-a05e-5cbc1f5db79f-kube-api-access-z5wj8\") pod \"horizon-54f88854b8-75fgz\" (UID: \"9d264a5a-d4a9-43a0-a05e-5cbc1f5db79f\") " pod="horizon-kuttl-tests/horizon-54f88854b8-75fgz" Dec 03 13:44:57 crc kubenswrapper[5052]: I1203 13:44:57.953229 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"horizon-secret-key\" (UniqueName: \"kubernetes.io/secret/9d264a5a-d4a9-43a0-a05e-5cbc1f5db79f-horizon-secret-key\") pod \"horizon-54f88854b8-75fgz\" (UID: \"9d264a5a-d4a9-43a0-a05e-5cbc1f5db79f\") " pod="horizon-kuttl-tests/horizon-54f88854b8-75fgz" Dec 03 13:44:57 crc kubenswrapper[5052]: I1203 13:44:57.953262 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"policy\" (UniqueName: \"kubernetes.io/configmap/9d264a5a-d4a9-43a0-a05e-5cbc1f5db79f-policy\") pod \"horizon-54f88854b8-75fgz\" (UID: \"9d264a5a-d4a9-43a0-a05e-5cbc1f5db79f\") " pod="horizon-kuttl-tests/horizon-54f88854b8-75fgz" Dec 03 13:44:57 crc kubenswrapper[5052]: I1203 13:44:57.953290 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/9d264a5a-d4a9-43a0-a05e-5cbc1f5db79f-scripts\") pod \"horizon-54f88854b8-75fgz\" (UID: \"9d264a5a-d4a9-43a0-a05e-5cbc1f5db79f\") " pod="horizon-kuttl-tests/horizon-54f88854b8-75fgz" Dec 03 13:44:57 crc kubenswrapper[5052]: I1203 13:44:57.953345 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/9d264a5a-d4a9-43a0-a05e-5cbc1f5db79f-config-data\") pod \"horizon-54f88854b8-75fgz\" (UID: \"9d264a5a-d4a9-43a0-a05e-5cbc1f5db79f\") " pod="horizon-kuttl-tests/horizon-54f88854b8-75fgz" Dec 03 13:44:57 crc kubenswrapper[5052]: I1203 13:44:57.980703 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["horizon-kuttl-tests/horizon-54f88854b8-75fgz"] Dec 03 13:44:57 crc kubenswrapper[5052]: E1203 13:44:57.981593 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[config-data horizon-secret-key kube-api-access-z5wj8 logs policy scripts], unattached volumes=[], failed to process volumes=[]: context canceled" pod="horizon-kuttl-tests/horizon-54f88854b8-75fgz" podUID="9d264a5a-d4a9-43a0-a05e-5cbc1f5db79f" Dec 03 13:44:57 crc kubenswrapper[5052]: I1203 13:44:57.989834 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["horizon-kuttl-tests/horizon-577c75f77-gjq8d"] Dec 03 13:44:57 crc kubenswrapper[5052]: I1203 13:44:57.990101 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="horizon-kuttl-tests/horizon-577c75f77-gjq8d" podUID="bbdb204c-cb9d-4299-b5fb-3c76fc5649ce" containerName="horizon-log" containerID="cri-o://b508037e0f85ac50f298e6fe9ef4374889e3706d798dd65ad3856045cefbd3e1" gracePeriod=30 Dec 03 13:44:57 crc kubenswrapper[5052]: I1203 13:44:57.990151 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="horizon-kuttl-tests/horizon-577c75f77-gjq8d" podUID="bbdb204c-cb9d-4299-b5fb-3c76fc5649ce" containerName="horizon" containerID="cri-o://a13c586622babcda45ba98ecf179ccc5d357cbaa30c59aeb86edd53d3f383621" gracePeriod=30 Dec 03 13:44:58 crc kubenswrapper[5052]: I1203 13:44:58.054008 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"horizon-secret-key\" (UniqueName: \"kubernetes.io/secret/9d264a5a-d4a9-43a0-a05e-5cbc1f5db79f-horizon-secret-key\") pod \"horizon-54f88854b8-75fgz\" (UID: \"9d264a5a-d4a9-43a0-a05e-5cbc1f5db79f\") " pod="horizon-kuttl-tests/horizon-54f88854b8-75fgz" Dec 03 13:44:58 crc kubenswrapper[5052]: I1203 13:44:58.054066 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"policy\" (UniqueName: \"kubernetes.io/configmap/9d264a5a-d4a9-43a0-a05e-5cbc1f5db79f-policy\") pod \"horizon-54f88854b8-75fgz\" (UID: \"9d264a5a-d4a9-43a0-a05e-5cbc1f5db79f\") " pod="horizon-kuttl-tests/horizon-54f88854b8-75fgz" Dec 03 13:44:58 crc kubenswrapper[5052]: I1203 13:44:58.054093 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/9d264a5a-d4a9-43a0-a05e-5cbc1f5db79f-scripts\") pod \"horizon-54f88854b8-75fgz\" (UID: \"9d264a5a-d4a9-43a0-a05e-5cbc1f5db79f\") " pod="horizon-kuttl-tests/horizon-54f88854b8-75fgz" Dec 03 13:44:58 crc kubenswrapper[5052]: I1203 13:44:58.054132 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/9d264a5a-d4a9-43a0-a05e-5cbc1f5db79f-config-data\") pod \"horizon-54f88854b8-75fgz\" (UID: \"9d264a5a-d4a9-43a0-a05e-5cbc1f5db79f\") " pod="horizon-kuttl-tests/horizon-54f88854b8-75fgz" Dec 03 13:44:58 crc kubenswrapper[5052]: I1203 13:44:58.054188 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/9d264a5a-d4a9-43a0-a05e-5cbc1f5db79f-logs\") pod \"horizon-54f88854b8-75fgz\" (UID: \"9d264a5a-d4a9-43a0-a05e-5cbc1f5db79f\") " pod="horizon-kuttl-tests/horizon-54f88854b8-75fgz" Dec 03 13:44:58 crc kubenswrapper[5052]: I1203 13:44:58.054221 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-z5wj8\" (UniqueName: \"kubernetes.io/projected/9d264a5a-d4a9-43a0-a05e-5cbc1f5db79f-kube-api-access-z5wj8\") pod \"horizon-54f88854b8-75fgz\" (UID: \"9d264a5a-d4a9-43a0-a05e-5cbc1f5db79f\") " pod="horizon-kuttl-tests/horizon-54f88854b8-75fgz" Dec 03 13:44:58 crc kubenswrapper[5052]: E1203 13:44:58.054225 5052 secret.go:188] Couldn't get secret horizon-kuttl-tests/horizon: secret "horizon" not found Dec 03 13:44:58 crc kubenswrapper[5052]: E1203 13:44:58.054323 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9d264a5a-d4a9-43a0-a05e-5cbc1f5db79f-horizon-secret-key podName:9d264a5a-d4a9-43a0-a05e-5cbc1f5db79f nodeName:}" failed. No retries permitted until 2025-12-03 13:44:58.554298663 +0000 UTC m=+8689.142540427 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "horizon-secret-key" (UniqueName: "kubernetes.io/secret/9d264a5a-d4a9-43a0-a05e-5cbc1f5db79f-horizon-secret-key") pod "horizon-54f88854b8-75fgz" (UID: "9d264a5a-d4a9-43a0-a05e-5cbc1f5db79f") : secret "horizon" not found Dec 03 13:44:58 crc kubenswrapper[5052]: E1203 13:44:58.054488 5052 configmap.go:193] Couldn't get configMap horizon-kuttl-tests/horizon-config-data: configmap "horizon-config-data" not found Dec 03 13:44:58 crc kubenswrapper[5052]: E1203 13:44:58.054544 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9d264a5a-d4a9-43a0-a05e-5cbc1f5db79f-config-data podName:9d264a5a-d4a9-43a0-a05e-5cbc1f5db79f nodeName:}" failed. No retries permitted until 2025-12-03 13:44:58.554528 +0000 UTC m=+8689.142769844 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "config-data" (UniqueName: "kubernetes.io/configmap/9d264a5a-d4a9-43a0-a05e-5cbc1f5db79f-config-data") pod "horizon-54f88854b8-75fgz" (UID: "9d264a5a-d4a9-43a0-a05e-5cbc1f5db79f") : configmap "horizon-config-data" not found Dec 03 13:44:58 crc kubenswrapper[5052]: E1203 13:44:58.054600 5052 configmap.go:193] Couldn't get configMap horizon-kuttl-tests/horizon-scripts: configmap "horizon-scripts" not found Dec 03 13:44:58 crc kubenswrapper[5052]: E1203 13:44:58.054627 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9d264a5a-d4a9-43a0-a05e-5cbc1f5db79f-scripts podName:9d264a5a-d4a9-43a0-a05e-5cbc1f5db79f nodeName:}" failed. No retries permitted until 2025-12-03 13:44:58.554617212 +0000 UTC m=+8689.142859066 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "scripts" (UniqueName: "kubernetes.io/configmap/9d264a5a-d4a9-43a0-a05e-5cbc1f5db79f-scripts") pod "horizon-54f88854b8-75fgz" (UID: "9d264a5a-d4a9-43a0-a05e-5cbc1f5db79f") : configmap "horizon-scripts" not found Dec 03 13:44:58 crc kubenswrapper[5052]: I1203 13:44:58.054944 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/9d264a5a-d4a9-43a0-a05e-5cbc1f5db79f-logs\") pod \"horizon-54f88854b8-75fgz\" (UID: \"9d264a5a-d4a9-43a0-a05e-5cbc1f5db79f\") " pod="horizon-kuttl-tests/horizon-54f88854b8-75fgz" Dec 03 13:44:58 crc kubenswrapper[5052]: I1203 13:44:58.055053 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"policy\" (UniqueName: \"kubernetes.io/configmap/9d264a5a-d4a9-43a0-a05e-5cbc1f5db79f-policy\") pod \"horizon-54f88854b8-75fgz\" (UID: \"9d264a5a-d4a9-43a0-a05e-5cbc1f5db79f\") " pod="horizon-kuttl-tests/horizon-54f88854b8-75fgz" Dec 03 13:44:58 crc kubenswrapper[5052]: E1203 13:44:58.058536 5052 projected.go:194] Error preparing data for projected volume kube-api-access-z5wj8 for pod horizon-kuttl-tests/horizon-54f88854b8-75fgz: failed to fetch token: serviceaccounts "horizon-horizon" not found Dec 03 13:44:58 crc kubenswrapper[5052]: E1203 13:44:58.058606 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9d264a5a-d4a9-43a0-a05e-5cbc1f5db79f-kube-api-access-z5wj8 podName:9d264a5a-d4a9-43a0-a05e-5cbc1f5db79f nodeName:}" failed. No retries permitted until 2025-12-03 13:44:58.558588916 +0000 UTC m=+8689.146830680 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-z5wj8" (UniqueName: "kubernetes.io/projected/9d264a5a-d4a9-43a0-a05e-5cbc1f5db79f-kube-api-access-z5wj8") pod "horizon-54f88854b8-75fgz" (UID: "9d264a5a-d4a9-43a0-a05e-5cbc1f5db79f") : failed to fetch token: serviceaccounts "horizon-horizon" not found Dec 03 13:44:58 crc kubenswrapper[5052]: I1203 13:44:58.176631 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="horizon-kuttl-tests/horizon-59cf497657-pm9sl" podUID="8beb534e-1595-4943-98be-598d43afec0d" containerName="horizon" probeResult="failure" output="Get \"http://10.217.0.71:8080/dashboard/auth/login/?next=/dashboard/\": dial tcp 10.217.0.71:8080: connect: connection refused" Dec 03 13:44:58 crc kubenswrapper[5052]: I1203 13:44:58.560316 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-z5wj8\" (UniqueName: \"kubernetes.io/projected/9d264a5a-d4a9-43a0-a05e-5cbc1f5db79f-kube-api-access-z5wj8\") pod \"horizon-54f88854b8-75fgz\" (UID: \"9d264a5a-d4a9-43a0-a05e-5cbc1f5db79f\") " pod="horizon-kuttl-tests/horizon-54f88854b8-75fgz" Dec 03 13:44:58 crc kubenswrapper[5052]: I1203 13:44:58.560401 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"horizon-secret-key\" (UniqueName: \"kubernetes.io/secret/9d264a5a-d4a9-43a0-a05e-5cbc1f5db79f-horizon-secret-key\") pod \"horizon-54f88854b8-75fgz\" (UID: \"9d264a5a-d4a9-43a0-a05e-5cbc1f5db79f\") " pod="horizon-kuttl-tests/horizon-54f88854b8-75fgz" Dec 03 13:44:58 crc kubenswrapper[5052]: I1203 13:44:58.560443 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/9d264a5a-d4a9-43a0-a05e-5cbc1f5db79f-scripts\") pod \"horizon-54f88854b8-75fgz\" (UID: \"9d264a5a-d4a9-43a0-a05e-5cbc1f5db79f\") " pod="horizon-kuttl-tests/horizon-54f88854b8-75fgz" Dec 03 13:44:58 crc kubenswrapper[5052]: I1203 13:44:58.560501 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/9d264a5a-d4a9-43a0-a05e-5cbc1f5db79f-config-data\") pod \"horizon-54f88854b8-75fgz\" (UID: \"9d264a5a-d4a9-43a0-a05e-5cbc1f5db79f\") " pod="horizon-kuttl-tests/horizon-54f88854b8-75fgz" Dec 03 13:44:58 crc kubenswrapper[5052]: E1203 13:44:58.560596 5052 configmap.go:193] Couldn't get configMap horizon-kuttl-tests/horizon-scripts: configmap "horizon-scripts" not found Dec 03 13:44:58 crc kubenswrapper[5052]: E1203 13:44:58.560633 5052 configmap.go:193] Couldn't get configMap horizon-kuttl-tests/horizon-config-data: configmap "horizon-config-data" not found Dec 03 13:44:58 crc kubenswrapper[5052]: E1203 13:44:58.560660 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9d264a5a-d4a9-43a0-a05e-5cbc1f5db79f-scripts podName:9d264a5a-d4a9-43a0-a05e-5cbc1f5db79f nodeName:}" failed. No retries permitted until 2025-12-03 13:44:59.560642509 +0000 UTC m=+8690.148884273 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "scripts" (UniqueName: "kubernetes.io/configmap/9d264a5a-d4a9-43a0-a05e-5cbc1f5db79f-scripts") pod "horizon-54f88854b8-75fgz" (UID: "9d264a5a-d4a9-43a0-a05e-5cbc1f5db79f") : configmap "horizon-scripts" not found Dec 03 13:44:58 crc kubenswrapper[5052]: E1203 13:44:58.560678 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9d264a5a-d4a9-43a0-a05e-5cbc1f5db79f-config-data podName:9d264a5a-d4a9-43a0-a05e-5cbc1f5db79f nodeName:}" failed. No retries permitted until 2025-12-03 13:44:59.56067076 +0000 UTC m=+8690.148912524 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "config-data" (UniqueName: "kubernetes.io/configmap/9d264a5a-d4a9-43a0-a05e-5cbc1f5db79f-config-data") pod "horizon-54f88854b8-75fgz" (UID: "9d264a5a-d4a9-43a0-a05e-5cbc1f5db79f") : configmap "horizon-config-data" not found Dec 03 13:44:58 crc kubenswrapper[5052]: E1203 13:44:58.560764 5052 secret.go:188] Couldn't get secret horizon-kuttl-tests/horizon: secret "horizon" not found Dec 03 13:44:58 crc kubenswrapper[5052]: E1203 13:44:58.560807 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9d264a5a-d4a9-43a0-a05e-5cbc1f5db79f-horizon-secret-key podName:9d264a5a-d4a9-43a0-a05e-5cbc1f5db79f nodeName:}" failed. No retries permitted until 2025-12-03 13:44:59.560795043 +0000 UTC m=+8690.149036807 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "horizon-secret-key" (UniqueName: "kubernetes.io/secret/9d264a5a-d4a9-43a0-a05e-5cbc1f5db79f-horizon-secret-key") pod "horizon-54f88854b8-75fgz" (UID: "9d264a5a-d4a9-43a0-a05e-5cbc1f5db79f") : secret "horizon" not found Dec 03 13:44:58 crc kubenswrapper[5052]: E1203 13:44:58.563351 5052 projected.go:194] Error preparing data for projected volume kube-api-access-z5wj8 for pod horizon-kuttl-tests/horizon-54f88854b8-75fgz: failed to fetch token: serviceaccounts "horizon-horizon" not found Dec 03 13:44:58 crc kubenswrapper[5052]: E1203 13:44:58.563443 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9d264a5a-d4a9-43a0-a05e-5cbc1f5db79f-kube-api-access-z5wj8 podName:9d264a5a-d4a9-43a0-a05e-5cbc1f5db79f nodeName:}" failed. No retries permitted until 2025-12-03 13:44:59.563423199 +0000 UTC m=+8690.151664963 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-z5wj8" (UniqueName: "kubernetes.io/projected/9d264a5a-d4a9-43a0-a05e-5cbc1f5db79f-kube-api-access-z5wj8") pod "horizon-54f88854b8-75fgz" (UID: "9d264a5a-d4a9-43a0-a05e-5cbc1f5db79f") : failed to fetch token: serviceaccounts "horizon-horizon" not found Dec 03 13:44:58 crc kubenswrapper[5052]: I1203 13:44:58.876061 5052 generic.go:334] "Generic (PLEG): container finished" podID="9ce53d74-f6a1-48b6-b39b-dd9b2a04a8db" containerID="ed30b7810dd296b4d5cdafb9fd446fe5d303fddd286e1b7c6990e60300d1180e" exitCode=0 Dec 03 13:44:58 crc kubenswrapper[5052]: I1203 13:44:58.876146 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="horizon-kuttl-tests/horizon-54f88854b8-75fgz" Dec 03 13:44:58 crc kubenswrapper[5052]: I1203 13:44:58.876133 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-operators-bppm7" event={"ID":"9ce53d74-f6a1-48b6-b39b-dd9b2a04a8db","Type":"ContainerDied","Data":"ed30b7810dd296b4d5cdafb9fd446fe5d303fddd286e1b7c6990e60300d1180e"} Dec 03 13:44:58 crc kubenswrapper[5052]: I1203 13:44:58.886780 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="horizon-kuttl-tests/horizon-54f88854b8-75fgz" Dec 03 13:44:59 crc kubenswrapper[5052]: I1203 13:44:59.069588 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"policy\" (UniqueName: \"kubernetes.io/configmap/9d264a5a-d4a9-43a0-a05e-5cbc1f5db79f-policy\") pod \"9d264a5a-d4a9-43a0-a05e-5cbc1f5db79f\" (UID: \"9d264a5a-d4a9-43a0-a05e-5cbc1f5db79f\") " Dec 03 13:44:59 crc kubenswrapper[5052]: I1203 13:44:59.069728 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/9d264a5a-d4a9-43a0-a05e-5cbc1f5db79f-logs\") pod \"9d264a5a-d4a9-43a0-a05e-5cbc1f5db79f\" (UID: \"9d264a5a-d4a9-43a0-a05e-5cbc1f5db79f\") " Dec 03 13:44:59 crc kubenswrapper[5052]: I1203 13:44:59.070147 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d264a5a-d4a9-43a0-a05e-5cbc1f5db79f-logs" (OuterVolumeSpecName: "logs") pod "9d264a5a-d4a9-43a0-a05e-5cbc1f5db79f" (UID: "9d264a5a-d4a9-43a0-a05e-5cbc1f5db79f"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 13:44:59 crc kubenswrapper[5052]: I1203 13:44:59.070370 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/9d264a5a-d4a9-43a0-a05e-5cbc1f5db79f-logs\") on node \"crc\" DevicePath \"\"" Dec 03 13:44:59 crc kubenswrapper[5052]: I1203 13:44:59.070362 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9d264a5a-d4a9-43a0-a05e-5cbc1f5db79f-policy" (OuterVolumeSpecName: "policy") pod "9d264a5a-d4a9-43a0-a05e-5cbc1f5db79f" (UID: "9d264a5a-d4a9-43a0-a05e-5cbc1f5db79f"). InnerVolumeSpecName "policy". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:44:59 crc kubenswrapper[5052]: I1203 13:44:59.171532 5052 reconciler_common.go:293] "Volume detached for volume \"policy\" (UniqueName: \"kubernetes.io/configmap/9d264a5a-d4a9-43a0-a05e-5cbc1f5db79f-policy\") on node \"crc\" DevicePath \"\"" Dec 03 13:44:59 crc kubenswrapper[5052]: I1203 13:44:59.577399 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-z5wj8\" (UniqueName: \"kubernetes.io/projected/9d264a5a-d4a9-43a0-a05e-5cbc1f5db79f-kube-api-access-z5wj8\") pod \"horizon-54f88854b8-75fgz\" (UID: \"9d264a5a-d4a9-43a0-a05e-5cbc1f5db79f\") " pod="horizon-kuttl-tests/horizon-54f88854b8-75fgz" Dec 03 13:44:59 crc kubenswrapper[5052]: I1203 13:44:59.577751 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"horizon-secret-key\" (UniqueName: \"kubernetes.io/secret/9d264a5a-d4a9-43a0-a05e-5cbc1f5db79f-horizon-secret-key\") pod \"horizon-54f88854b8-75fgz\" (UID: \"9d264a5a-d4a9-43a0-a05e-5cbc1f5db79f\") " pod="horizon-kuttl-tests/horizon-54f88854b8-75fgz" Dec 03 13:44:59 crc kubenswrapper[5052]: I1203 13:44:59.577788 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/9d264a5a-d4a9-43a0-a05e-5cbc1f5db79f-scripts\") pod \"horizon-54f88854b8-75fgz\" (UID: \"9d264a5a-d4a9-43a0-a05e-5cbc1f5db79f\") " pod="horizon-kuttl-tests/horizon-54f88854b8-75fgz" Dec 03 13:44:59 crc kubenswrapper[5052]: I1203 13:44:59.577844 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/9d264a5a-d4a9-43a0-a05e-5cbc1f5db79f-config-data\") pod \"horizon-54f88854b8-75fgz\" (UID: \"9d264a5a-d4a9-43a0-a05e-5cbc1f5db79f\") " pod="horizon-kuttl-tests/horizon-54f88854b8-75fgz" Dec 03 13:44:59 crc kubenswrapper[5052]: E1203 13:44:59.577966 5052 secret.go:188] Couldn't get secret horizon-kuttl-tests/horizon: secret "horizon" not found Dec 03 13:44:59 crc kubenswrapper[5052]: E1203 13:44:59.578047 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9d264a5a-d4a9-43a0-a05e-5cbc1f5db79f-horizon-secret-key podName:9d264a5a-d4a9-43a0-a05e-5cbc1f5db79f nodeName:}" failed. No retries permitted until 2025-12-03 13:45:01.578028086 +0000 UTC m=+8692.166269910 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "horizon-secret-key" (UniqueName: "kubernetes.io/secret/9d264a5a-d4a9-43a0-a05e-5cbc1f5db79f-horizon-secret-key") pod "horizon-54f88854b8-75fgz" (UID: "9d264a5a-d4a9-43a0-a05e-5cbc1f5db79f") : secret "horizon" not found Dec 03 13:44:59 crc kubenswrapper[5052]: E1203 13:44:59.577984 5052 configmap.go:193] Couldn't get configMap horizon-kuttl-tests/horizon-config-data: configmap "horizon-config-data" not found Dec 03 13:44:59 crc kubenswrapper[5052]: E1203 13:44:59.578136 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9d264a5a-d4a9-43a0-a05e-5cbc1f5db79f-config-data podName:9d264a5a-d4a9-43a0-a05e-5cbc1f5db79f nodeName:}" failed. No retries permitted until 2025-12-03 13:45:01.578117608 +0000 UTC m=+8692.166359442 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "config-data" (UniqueName: "kubernetes.io/configmap/9d264a5a-d4a9-43a0-a05e-5cbc1f5db79f-config-data") pod "horizon-54f88854b8-75fgz" (UID: "9d264a5a-d4a9-43a0-a05e-5cbc1f5db79f") : configmap "horizon-config-data" not found Dec 03 13:44:59 crc kubenswrapper[5052]: E1203 13:44:59.578237 5052 configmap.go:193] Couldn't get configMap horizon-kuttl-tests/horizon-scripts: configmap "horizon-scripts" not found Dec 03 13:44:59 crc kubenswrapper[5052]: E1203 13:44:59.578326 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9d264a5a-d4a9-43a0-a05e-5cbc1f5db79f-scripts podName:9d264a5a-d4a9-43a0-a05e-5cbc1f5db79f nodeName:}" failed. No retries permitted until 2025-12-03 13:45:01.578304774 +0000 UTC m=+8692.166546598 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "scripts" (UniqueName: "kubernetes.io/configmap/9d264a5a-d4a9-43a0-a05e-5cbc1f5db79f-scripts") pod "horizon-54f88854b8-75fgz" (UID: "9d264a5a-d4a9-43a0-a05e-5cbc1f5db79f") : configmap "horizon-scripts" not found Dec 03 13:44:59 crc kubenswrapper[5052]: E1203 13:44:59.581088 5052 projected.go:194] Error preparing data for projected volume kube-api-access-z5wj8 for pod horizon-kuttl-tests/horizon-54f88854b8-75fgz: failed to fetch token: serviceaccounts "horizon-horizon" not found Dec 03 13:44:59 crc kubenswrapper[5052]: E1203 13:44:59.581189 5052 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9d264a5a-d4a9-43a0-a05e-5cbc1f5db79f-kube-api-access-z5wj8 podName:9d264a5a-d4a9-43a0-a05e-5cbc1f5db79f nodeName:}" failed. No retries permitted until 2025-12-03 13:45:01.581164746 +0000 UTC m=+8692.169406570 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-z5wj8" (UniqueName: "kubernetes.io/projected/9d264a5a-d4a9-43a0-a05e-5cbc1f5db79f-kube-api-access-z5wj8") pod "horizon-54f88854b8-75fgz" (UID: "9d264a5a-d4a9-43a0-a05e-5cbc1f5db79f") : failed to fetch token: serviceaccounts "horizon-horizon" not found Dec 03 13:44:59 crc kubenswrapper[5052]: I1203 13:44:59.888193 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="horizon-kuttl-tests/horizon-54f88854b8-75fgz" Dec 03 13:44:59 crc kubenswrapper[5052]: I1203 13:44:59.888194 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-operators-bppm7" event={"ID":"9ce53d74-f6a1-48b6-b39b-dd9b2a04a8db","Type":"ContainerStarted","Data":"e69434e0746c3fc8d799dcd5bddb58a43fd330f3e7d4488ba426d7b4e122018f"} Dec 03 13:44:59 crc kubenswrapper[5052]: I1203 13:44:59.936055 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["horizon-kuttl-tests/horizon-54f88854b8-75fgz"] Dec 03 13:44:59 crc kubenswrapper[5052]: I1203 13:44:59.945218 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["horizon-kuttl-tests/horizon-54f88854b8-75fgz"] Dec 03 13:44:59 crc kubenswrapper[5052]: I1203 13:44:59.983830 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/9d264a5a-d4a9-43a0-a05e-5cbc1f5db79f-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 13:44:59 crc kubenswrapper[5052]: I1203 13:44:59.983874 5052 reconciler_common.go:293] "Volume detached for volume \"horizon-secret-key\" (UniqueName: \"kubernetes.io/secret/9d264a5a-d4a9-43a0-a05e-5cbc1f5db79f-horizon-secret-key\") on node \"crc\" DevicePath \"\"" Dec 03 13:44:59 crc kubenswrapper[5052]: I1203 13:44:59.983887 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/9d264a5a-d4a9-43a0-a05e-5cbc1f5db79f-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 13:44:59 crc kubenswrapper[5052]: I1203 13:44:59.983902 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-z5wj8\" (UniqueName: \"kubernetes.io/projected/9d264a5a-d4a9-43a0-a05e-5cbc1f5db79f-kube-api-access-z5wj8\") on node \"crc\" DevicePath \"\"" Dec 03 13:45:00 crc kubenswrapper[5052]: I1203 13:45:00.139852 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-marketplace/redhat-operators-bppm7" podStartSLOduration=4.7278095449999995 podStartE2EDuration="7.139829781s" podCreationTimestamp="2025-12-03 13:44:53 +0000 UTC" firstStartedPulling="2025-12-03 13:44:56.860270899 +0000 UTC m=+8687.448512663" lastFinishedPulling="2025-12-03 13:44:59.272291135 +0000 UTC m=+8689.860532899" observedRunningTime="2025-12-03 13:44:59.952139507 +0000 UTC m=+8690.540381271" watchObservedRunningTime="2025-12-03 13:45:00.139829781 +0000 UTC m=+8690.728071535" Dec 03 13:45:00 crc kubenswrapper[5052]: I1203 13:45:00.142158 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openshift-operator-lifecycle-manager/collect-profiles-29412825-ddwbg"] Dec 03 13:45:00 crc kubenswrapper[5052]: I1203 13:45:00.143321 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-operator-lifecycle-manager/collect-profiles-29412825-ddwbg" Dec 03 13:45:00 crc kubenswrapper[5052]: I1203 13:45:00.145189 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-operator-lifecycle-manager"/"collect-profiles-config" Dec 03 13:45:00 crc kubenswrapper[5052]: I1203 13:45:00.145277 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-operator-lifecycle-manager"/"collect-profiles-dockercfg-kzf4t" Dec 03 13:45:00 crc kubenswrapper[5052]: I1203 13:45:00.157662 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-operator-lifecycle-manager/collect-profiles-29412825-ddwbg"] Dec 03 13:45:00 crc kubenswrapper[5052]: I1203 13:45:00.186353 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/73544e7c-ad0a-4443-982d-2bde1474ec9b-config-volume\") pod \"collect-profiles-29412825-ddwbg\" (UID: \"73544e7c-ad0a-4443-982d-2bde1474ec9b\") " pod="openshift-operator-lifecycle-manager/collect-profiles-29412825-ddwbg" Dec 03 13:45:00 crc kubenswrapper[5052]: I1203 13:45:00.186406 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-47pbm\" (UniqueName: \"kubernetes.io/projected/73544e7c-ad0a-4443-982d-2bde1474ec9b-kube-api-access-47pbm\") pod \"collect-profiles-29412825-ddwbg\" (UID: \"73544e7c-ad0a-4443-982d-2bde1474ec9b\") " pod="openshift-operator-lifecycle-manager/collect-profiles-29412825-ddwbg" Dec 03 13:45:00 crc kubenswrapper[5052]: I1203 13:45:00.186456 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-volume\" (UniqueName: \"kubernetes.io/secret/73544e7c-ad0a-4443-982d-2bde1474ec9b-secret-volume\") pod \"collect-profiles-29412825-ddwbg\" (UID: \"73544e7c-ad0a-4443-982d-2bde1474ec9b\") " pod="openshift-operator-lifecycle-manager/collect-profiles-29412825-ddwbg" Dec 03 13:45:00 crc kubenswrapper[5052]: I1203 13:45:00.287315 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/73544e7c-ad0a-4443-982d-2bde1474ec9b-config-volume\") pod \"collect-profiles-29412825-ddwbg\" (UID: \"73544e7c-ad0a-4443-982d-2bde1474ec9b\") " pod="openshift-operator-lifecycle-manager/collect-profiles-29412825-ddwbg" Dec 03 13:45:00 crc kubenswrapper[5052]: I1203 13:45:00.287387 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-47pbm\" (UniqueName: \"kubernetes.io/projected/73544e7c-ad0a-4443-982d-2bde1474ec9b-kube-api-access-47pbm\") pod \"collect-profiles-29412825-ddwbg\" (UID: \"73544e7c-ad0a-4443-982d-2bde1474ec9b\") " pod="openshift-operator-lifecycle-manager/collect-profiles-29412825-ddwbg" Dec 03 13:45:00 crc kubenswrapper[5052]: I1203 13:45:00.287470 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"secret-volume\" (UniqueName: \"kubernetes.io/secret/73544e7c-ad0a-4443-982d-2bde1474ec9b-secret-volume\") pod \"collect-profiles-29412825-ddwbg\" (UID: \"73544e7c-ad0a-4443-982d-2bde1474ec9b\") " pod="openshift-operator-lifecycle-manager/collect-profiles-29412825-ddwbg" Dec 03 13:45:00 crc kubenswrapper[5052]: I1203 13:45:00.288350 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/73544e7c-ad0a-4443-982d-2bde1474ec9b-config-volume\") pod \"collect-profiles-29412825-ddwbg\" (UID: \"73544e7c-ad0a-4443-982d-2bde1474ec9b\") " pod="openshift-operator-lifecycle-manager/collect-profiles-29412825-ddwbg" Dec 03 13:45:00 crc kubenswrapper[5052]: I1203 13:45:00.295971 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"secret-volume\" (UniqueName: \"kubernetes.io/secret/73544e7c-ad0a-4443-982d-2bde1474ec9b-secret-volume\") pod \"collect-profiles-29412825-ddwbg\" (UID: \"73544e7c-ad0a-4443-982d-2bde1474ec9b\") " pod="openshift-operator-lifecycle-manager/collect-profiles-29412825-ddwbg" Dec 03 13:45:00 crc kubenswrapper[5052]: I1203 13:45:00.313451 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-47pbm\" (UniqueName: \"kubernetes.io/projected/73544e7c-ad0a-4443-982d-2bde1474ec9b-kube-api-access-47pbm\") pod \"collect-profiles-29412825-ddwbg\" (UID: \"73544e7c-ad0a-4443-982d-2bde1474ec9b\") " pod="openshift-operator-lifecycle-manager/collect-profiles-29412825-ddwbg" Dec 03 13:45:00 crc kubenswrapper[5052]: I1203 13:45:00.461394 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-operator-lifecycle-manager/collect-profiles-29412825-ddwbg" Dec 03 13:45:00 crc kubenswrapper[5052]: I1203 13:45:00.902581 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-operator-lifecycle-manager/collect-profiles-29412825-ddwbg"] Dec 03 13:45:01 crc kubenswrapper[5052]: I1203 13:45:01.141720 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="horizon-kuttl-tests/horizon-577c75f77-gjq8d" podUID="bbdb204c-cb9d-4299-b5fb-3c76fc5649ce" containerName="horizon" probeResult="failure" output="Get \"http://10.217.0.72:8080/dashboard/auth/login/?next=/dashboard/\": read tcp 10.217.0.2:60406->10.217.0.72:8080: read: connection reset by peer" Dec 03 13:45:01 crc kubenswrapper[5052]: I1203 13:45:01.681476 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9d264a5a-d4a9-43a0-a05e-5cbc1f5db79f" path="/var/lib/kubelet/pods/9d264a5a-d4a9-43a0-a05e-5cbc1f5db79f/volumes" Dec 03 13:45:01 crc kubenswrapper[5052]: I1203 13:45:01.906551 5052 generic.go:334] "Generic (PLEG): container finished" podID="bbdb204c-cb9d-4299-b5fb-3c76fc5649ce" containerID="a13c586622babcda45ba98ecf179ccc5d357cbaa30c59aeb86edd53d3f383621" exitCode=0 Dec 03 13:45:01 crc kubenswrapper[5052]: I1203 13:45:01.906605 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="horizon-kuttl-tests/horizon-577c75f77-gjq8d" event={"ID":"bbdb204c-cb9d-4299-b5fb-3c76fc5649ce","Type":"ContainerDied","Data":"a13c586622babcda45ba98ecf179ccc5d357cbaa30c59aeb86edd53d3f383621"} Dec 03 13:45:01 crc kubenswrapper[5052]: I1203 13:45:01.915479 5052 generic.go:334] "Generic (PLEG): container finished" podID="73544e7c-ad0a-4443-982d-2bde1474ec9b" containerID="04155d3f266f31098933e9f436fbd45e170c04350c2ef6c6d83c5fcaa0e0e8c9" exitCode=0 Dec 03 13:45:01 crc kubenswrapper[5052]: I1203 13:45:01.915558 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-operator-lifecycle-manager/collect-profiles-29412825-ddwbg" event={"ID":"73544e7c-ad0a-4443-982d-2bde1474ec9b","Type":"ContainerDied","Data":"04155d3f266f31098933e9f436fbd45e170c04350c2ef6c6d83c5fcaa0e0e8c9"} Dec 03 13:45:01 crc kubenswrapper[5052]: I1203 13:45:01.915620 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-operator-lifecycle-manager/collect-profiles-29412825-ddwbg" event={"ID":"73544e7c-ad0a-4443-982d-2bde1474ec9b","Type":"ContainerStarted","Data":"c98b78bdac6eb8daecb0ce849645f3b369126dd98264879c1dc5aa48f63d1455"} Dec 03 13:45:03 crc kubenswrapper[5052]: I1203 13:45:03.268851 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-operator-lifecycle-manager/collect-profiles-29412825-ddwbg" Dec 03 13:45:03 crc kubenswrapper[5052]: I1203 13:45:03.438750 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"secret-volume\" (UniqueName: \"kubernetes.io/secret/73544e7c-ad0a-4443-982d-2bde1474ec9b-secret-volume\") pod \"73544e7c-ad0a-4443-982d-2bde1474ec9b\" (UID: \"73544e7c-ad0a-4443-982d-2bde1474ec9b\") " Dec 03 13:45:03 crc kubenswrapper[5052]: I1203 13:45:03.438916 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-47pbm\" (UniqueName: \"kubernetes.io/projected/73544e7c-ad0a-4443-982d-2bde1474ec9b-kube-api-access-47pbm\") pod \"73544e7c-ad0a-4443-982d-2bde1474ec9b\" (UID: \"73544e7c-ad0a-4443-982d-2bde1474ec9b\") " Dec 03 13:45:03 crc kubenswrapper[5052]: I1203 13:45:03.438976 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/73544e7c-ad0a-4443-982d-2bde1474ec9b-config-volume\") pod \"73544e7c-ad0a-4443-982d-2bde1474ec9b\" (UID: \"73544e7c-ad0a-4443-982d-2bde1474ec9b\") " Dec 03 13:45:03 crc kubenswrapper[5052]: I1203 13:45:03.439895 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/73544e7c-ad0a-4443-982d-2bde1474ec9b-config-volume" (OuterVolumeSpecName: "config-volume") pod "73544e7c-ad0a-4443-982d-2bde1474ec9b" (UID: "73544e7c-ad0a-4443-982d-2bde1474ec9b"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:45:03 crc kubenswrapper[5052]: I1203 13:45:03.440758 5052 reconciler_common.go:293] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/73544e7c-ad0a-4443-982d-2bde1474ec9b-config-volume\") on node \"crc\" DevicePath \"\"" Dec 03 13:45:03 crc kubenswrapper[5052]: I1203 13:45:03.453879 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/73544e7c-ad0a-4443-982d-2bde1474ec9b-secret-volume" (OuterVolumeSpecName: "secret-volume") pod "73544e7c-ad0a-4443-982d-2bde1474ec9b" (UID: "73544e7c-ad0a-4443-982d-2bde1474ec9b"). InnerVolumeSpecName "secret-volume". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:45:03 crc kubenswrapper[5052]: I1203 13:45:03.453929 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/73544e7c-ad0a-4443-982d-2bde1474ec9b-kube-api-access-47pbm" (OuterVolumeSpecName: "kube-api-access-47pbm") pod "73544e7c-ad0a-4443-982d-2bde1474ec9b" (UID: "73544e7c-ad0a-4443-982d-2bde1474ec9b"). InnerVolumeSpecName "kube-api-access-47pbm". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:45:03 crc kubenswrapper[5052]: I1203 13:45:03.497569 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-marketplace/redhat-operators-bppm7" Dec 03 13:45:03 crc kubenswrapper[5052]: I1203 13:45:03.497642 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openshift-marketplace/redhat-operators-bppm7" Dec 03 13:45:03 crc kubenswrapper[5052]: I1203 13:45:03.541958 5052 reconciler_common.go:293] "Volume detached for volume \"secret-volume\" (UniqueName: \"kubernetes.io/secret/73544e7c-ad0a-4443-982d-2bde1474ec9b-secret-volume\") on node \"crc\" DevicePath \"\"" Dec 03 13:45:03 crc kubenswrapper[5052]: I1203 13:45:03.542026 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-47pbm\" (UniqueName: \"kubernetes.io/projected/73544e7c-ad0a-4443-982d-2bde1474ec9b-kube-api-access-47pbm\") on node \"crc\" DevicePath \"\"" Dec 03 13:45:03 crc kubenswrapper[5052]: E1203 13:45:03.839656 5052 cadvisor_stats_provider.go:516] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73544e7c_ad0a_4443_982d_2bde1474ec9b.slice\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73544e7c_ad0a_4443_982d_2bde1474ec9b.slice/crio-c98b78bdac6eb8daecb0ce849645f3b369126dd98264879c1dc5aa48f63d1455\": RecentStats: unable to find data in memory cache]" Dec 03 13:45:03 crc kubenswrapper[5052]: I1203 13:45:03.933140 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-operator-lifecycle-manager/collect-profiles-29412825-ddwbg" event={"ID":"73544e7c-ad0a-4443-982d-2bde1474ec9b","Type":"ContainerDied","Data":"c98b78bdac6eb8daecb0ce849645f3b369126dd98264879c1dc5aa48f63d1455"} Dec 03 13:45:03 crc kubenswrapper[5052]: I1203 13:45:03.933185 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-operator-lifecycle-manager/collect-profiles-29412825-ddwbg" Dec 03 13:45:03 crc kubenswrapper[5052]: I1203 13:45:03.933191 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c98b78bdac6eb8daecb0ce849645f3b369126dd98264879c1dc5aa48f63d1455" Dec 03 13:45:04 crc kubenswrapper[5052]: I1203 13:45:04.346297 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openshift-operator-lifecycle-manager/collect-profiles-29412780-k6h9t"] Dec 03 13:45:04 crc kubenswrapper[5052]: I1203 13:45:04.351499 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openshift-operator-lifecycle-manager/collect-profiles-29412780-k6h9t"] Dec 03 13:45:04 crc kubenswrapper[5052]: I1203 13:45:04.542933 5052 prober.go:107] "Probe failed" probeType="Startup" pod="openshift-marketplace/redhat-operators-bppm7" podUID="9ce53d74-f6a1-48b6-b39b-dd9b2a04a8db" containerName="registry-server" probeResult="failure" output=< Dec 03 13:45:04 crc kubenswrapper[5052]: timeout: failed to connect service ":50051" within 1s Dec 03 13:45:04 crc kubenswrapper[5052]: > Dec 03 13:45:05 crc kubenswrapper[5052]: I1203 13:45:05.682405 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f3de51af-ca99-42fa-a358-35184a5f1726" path="/var/lib/kubelet/pods/f3de51af-ca99-42fa-a358-35184a5f1726/volumes" Dec 03 13:45:07 crc kubenswrapper[5052]: I1203 13:45:06.673354 5052 scope.go:117] "RemoveContainer" containerID="2fbd293d6edef0fa059e91c119312e77b262534107c322feb7b59dbea3464526" Dec 03 13:45:07 crc kubenswrapper[5052]: E1203 13:45:06.673730 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 13:45:08 crc kubenswrapper[5052]: I1203 13:45:08.176386 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="horizon-kuttl-tests/horizon-59cf497657-pm9sl" podUID="8beb534e-1595-4943-98be-598d43afec0d" containerName="horizon" probeResult="failure" output="Get \"http://10.217.0.71:8080/dashboard/auth/login/?next=/dashboard/\": dial tcp 10.217.0.71:8080: connect: connection refused" Dec 03 13:45:08 crc kubenswrapper[5052]: I1203 13:45:08.233104 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="horizon-kuttl-tests/horizon-577c75f77-gjq8d" podUID="bbdb204c-cb9d-4299-b5fb-3c76fc5649ce" containerName="horizon" probeResult="failure" output="Get \"http://10.217.0.72:8080/dashboard/auth/login/?next=/dashboard/\": dial tcp 10.217.0.72:8080: connect: connection refused" Dec 03 13:45:13 crc kubenswrapper[5052]: I1203 13:45:13.573091 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-marketplace/redhat-operators-bppm7" Dec 03 13:45:13 crc kubenswrapper[5052]: I1203 13:45:13.730088 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-marketplace/redhat-operators-bppm7" Dec 03 13:45:13 crc kubenswrapper[5052]: I1203 13:45:13.818728 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openshift-marketplace/redhat-operators-bppm7"] Dec 03 13:45:15 crc kubenswrapper[5052]: I1203 13:45:15.017975 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openshift-marketplace/redhat-operators-bppm7" podUID="9ce53d74-f6a1-48b6-b39b-dd9b2a04a8db" containerName="registry-server" containerID="cri-o://e69434e0746c3fc8d799dcd5bddb58a43fd330f3e7d4488ba426d7b4e122018f" gracePeriod=2 Dec 03 13:45:15 crc kubenswrapper[5052]: I1203 13:45:15.421639 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/redhat-operators-bppm7" Dec 03 13:45:15 crc kubenswrapper[5052]: I1203 13:45:15.570408 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/9ce53d74-f6a1-48b6-b39b-dd9b2a04a8db-catalog-content\") pod \"9ce53d74-f6a1-48b6-b39b-dd9b2a04a8db\" (UID: \"9ce53d74-f6a1-48b6-b39b-dd9b2a04a8db\") " Dec 03 13:45:15 crc kubenswrapper[5052]: I1203 13:45:15.570556 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dv7pn\" (UniqueName: \"kubernetes.io/projected/9ce53d74-f6a1-48b6-b39b-dd9b2a04a8db-kube-api-access-dv7pn\") pod \"9ce53d74-f6a1-48b6-b39b-dd9b2a04a8db\" (UID: \"9ce53d74-f6a1-48b6-b39b-dd9b2a04a8db\") " Dec 03 13:45:15 crc kubenswrapper[5052]: I1203 13:45:15.570606 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/9ce53d74-f6a1-48b6-b39b-dd9b2a04a8db-utilities\") pod \"9ce53d74-f6a1-48b6-b39b-dd9b2a04a8db\" (UID: \"9ce53d74-f6a1-48b6-b39b-dd9b2a04a8db\") " Dec 03 13:45:15 crc kubenswrapper[5052]: I1203 13:45:15.571448 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ce53d74-f6a1-48b6-b39b-dd9b2a04a8db-utilities" (OuterVolumeSpecName: "utilities") pod "9ce53d74-f6a1-48b6-b39b-dd9b2a04a8db" (UID: "9ce53d74-f6a1-48b6-b39b-dd9b2a04a8db"). InnerVolumeSpecName "utilities". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 13:45:15 crc kubenswrapper[5052]: I1203 13:45:15.575820 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9ce53d74-f6a1-48b6-b39b-dd9b2a04a8db-kube-api-access-dv7pn" (OuterVolumeSpecName: "kube-api-access-dv7pn") pod "9ce53d74-f6a1-48b6-b39b-dd9b2a04a8db" (UID: "9ce53d74-f6a1-48b6-b39b-dd9b2a04a8db"). InnerVolumeSpecName "kube-api-access-dv7pn". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:45:15 crc kubenswrapper[5052]: I1203 13:45:15.672339 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-dv7pn\" (UniqueName: \"kubernetes.io/projected/9ce53d74-f6a1-48b6-b39b-dd9b2a04a8db-kube-api-access-dv7pn\") on node \"crc\" DevicePath \"\"" Dec 03 13:45:15 crc kubenswrapper[5052]: I1203 13:45:15.672387 5052 reconciler_common.go:293] "Volume detached for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/9ce53d74-f6a1-48b6-b39b-dd9b2a04a8db-utilities\") on node \"crc\" DevicePath \"\"" Dec 03 13:45:15 crc kubenswrapper[5052]: I1203 13:45:15.676879 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ce53d74-f6a1-48b6-b39b-dd9b2a04a8db-catalog-content" (OuterVolumeSpecName: "catalog-content") pod "9ce53d74-f6a1-48b6-b39b-dd9b2a04a8db" (UID: "9ce53d74-f6a1-48b6-b39b-dd9b2a04a8db"). InnerVolumeSpecName "catalog-content". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 13:45:15 crc kubenswrapper[5052]: I1203 13:45:15.775219 5052 reconciler_common.go:293] "Volume detached for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/9ce53d74-f6a1-48b6-b39b-dd9b2a04a8db-catalog-content\") on node \"crc\" DevicePath \"\"" Dec 03 13:45:16 crc kubenswrapper[5052]: I1203 13:45:16.027006 5052 generic.go:334] "Generic (PLEG): container finished" podID="9ce53d74-f6a1-48b6-b39b-dd9b2a04a8db" containerID="e69434e0746c3fc8d799dcd5bddb58a43fd330f3e7d4488ba426d7b4e122018f" exitCode=0 Dec 03 13:45:16 crc kubenswrapper[5052]: I1203 13:45:16.027057 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-operators-bppm7" event={"ID":"9ce53d74-f6a1-48b6-b39b-dd9b2a04a8db","Type":"ContainerDied","Data":"e69434e0746c3fc8d799dcd5bddb58a43fd330f3e7d4488ba426d7b4e122018f"} Dec 03 13:45:16 crc kubenswrapper[5052]: I1203 13:45:16.027091 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-operators-bppm7" event={"ID":"9ce53d74-f6a1-48b6-b39b-dd9b2a04a8db","Type":"ContainerDied","Data":"522f4b8d0f62fb9f58c236d8beb2e04dc9ff1a83311f1ea8cc0c8ceeb82fefc6"} Dec 03 13:45:16 crc kubenswrapper[5052]: I1203 13:45:16.027112 5052 scope.go:117] "RemoveContainer" containerID="e69434e0746c3fc8d799dcd5bddb58a43fd330f3e7d4488ba426d7b4e122018f" Dec 03 13:45:16 crc kubenswrapper[5052]: I1203 13:45:16.027941 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/redhat-operators-bppm7" Dec 03 13:45:16 crc kubenswrapper[5052]: I1203 13:45:16.049842 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openshift-marketplace/redhat-operators-bppm7"] Dec 03 13:45:16 crc kubenswrapper[5052]: I1203 13:45:16.051224 5052 scope.go:117] "RemoveContainer" containerID="ed30b7810dd296b4d5cdafb9fd446fe5d303fddd286e1b7c6990e60300d1180e" Dec 03 13:45:16 crc kubenswrapper[5052]: I1203 13:45:16.056814 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openshift-marketplace/redhat-operators-bppm7"] Dec 03 13:45:16 crc kubenswrapper[5052]: I1203 13:45:16.072341 5052 scope.go:117] "RemoveContainer" containerID="283137c4c8783be5b493aed31f1b56383b600fe5b95eb4668073e9d6c8275f0f" Dec 03 13:45:16 crc kubenswrapper[5052]: I1203 13:45:16.108993 5052 scope.go:117] "RemoveContainer" containerID="e69434e0746c3fc8d799dcd5bddb58a43fd330f3e7d4488ba426d7b4e122018f" Dec 03 13:45:16 crc kubenswrapper[5052]: E1203 13:45:16.109621 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e69434e0746c3fc8d799dcd5bddb58a43fd330f3e7d4488ba426d7b4e122018f\": container with ID starting with e69434e0746c3fc8d799dcd5bddb58a43fd330f3e7d4488ba426d7b4e122018f not found: ID does not exist" containerID="e69434e0746c3fc8d799dcd5bddb58a43fd330f3e7d4488ba426d7b4e122018f" Dec 03 13:45:16 crc kubenswrapper[5052]: I1203 13:45:16.109653 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e69434e0746c3fc8d799dcd5bddb58a43fd330f3e7d4488ba426d7b4e122018f"} err="failed to get container status \"e69434e0746c3fc8d799dcd5bddb58a43fd330f3e7d4488ba426d7b4e122018f\": rpc error: code = NotFound desc = could not find container \"e69434e0746c3fc8d799dcd5bddb58a43fd330f3e7d4488ba426d7b4e122018f\": container with ID starting with e69434e0746c3fc8d799dcd5bddb58a43fd330f3e7d4488ba426d7b4e122018f not found: ID does not exist" Dec 03 13:45:16 crc kubenswrapper[5052]: I1203 13:45:16.109678 5052 scope.go:117] "RemoveContainer" containerID="ed30b7810dd296b4d5cdafb9fd446fe5d303fddd286e1b7c6990e60300d1180e" Dec 03 13:45:16 crc kubenswrapper[5052]: E1203 13:45:16.109977 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ed30b7810dd296b4d5cdafb9fd446fe5d303fddd286e1b7c6990e60300d1180e\": container with ID starting with ed30b7810dd296b4d5cdafb9fd446fe5d303fddd286e1b7c6990e60300d1180e not found: ID does not exist" containerID="ed30b7810dd296b4d5cdafb9fd446fe5d303fddd286e1b7c6990e60300d1180e" Dec 03 13:45:16 crc kubenswrapper[5052]: I1203 13:45:16.110021 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ed30b7810dd296b4d5cdafb9fd446fe5d303fddd286e1b7c6990e60300d1180e"} err="failed to get container status \"ed30b7810dd296b4d5cdafb9fd446fe5d303fddd286e1b7c6990e60300d1180e\": rpc error: code = NotFound desc = could not find container \"ed30b7810dd296b4d5cdafb9fd446fe5d303fddd286e1b7c6990e60300d1180e\": container with ID starting with ed30b7810dd296b4d5cdafb9fd446fe5d303fddd286e1b7c6990e60300d1180e not found: ID does not exist" Dec 03 13:45:16 crc kubenswrapper[5052]: I1203 13:45:16.110048 5052 scope.go:117] "RemoveContainer" containerID="283137c4c8783be5b493aed31f1b56383b600fe5b95eb4668073e9d6c8275f0f" Dec 03 13:45:16 crc kubenswrapper[5052]: E1203 13:45:16.110466 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"283137c4c8783be5b493aed31f1b56383b600fe5b95eb4668073e9d6c8275f0f\": container with ID starting with 283137c4c8783be5b493aed31f1b56383b600fe5b95eb4668073e9d6c8275f0f not found: ID does not exist" containerID="283137c4c8783be5b493aed31f1b56383b600fe5b95eb4668073e9d6c8275f0f" Dec 03 13:45:16 crc kubenswrapper[5052]: I1203 13:45:16.110492 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"283137c4c8783be5b493aed31f1b56383b600fe5b95eb4668073e9d6c8275f0f"} err="failed to get container status \"283137c4c8783be5b493aed31f1b56383b600fe5b95eb4668073e9d6c8275f0f\": rpc error: code = NotFound desc = could not find container \"283137c4c8783be5b493aed31f1b56383b600fe5b95eb4668073e9d6c8275f0f\": container with ID starting with 283137c4c8783be5b493aed31f1b56383b600fe5b95eb4668073e9d6c8275f0f not found: ID does not exist" Dec 03 13:45:17 crc kubenswrapper[5052]: I1203 13:45:17.682419 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9ce53d74-f6a1-48b6-b39b-dd9b2a04a8db" path="/var/lib/kubelet/pods/9ce53d74-f6a1-48b6-b39b-dd9b2a04a8db/volumes" Dec 03 13:45:18 crc kubenswrapper[5052]: I1203 13:45:18.176402 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="horizon-kuttl-tests/horizon-59cf497657-pm9sl" podUID="8beb534e-1595-4943-98be-598d43afec0d" containerName="horizon" probeResult="failure" output="Get \"http://10.217.0.71:8080/dashboard/auth/login/?next=/dashboard/\": dial tcp 10.217.0.71:8080: connect: connection refused" Dec 03 13:45:18 crc kubenswrapper[5052]: I1203 13:45:18.176522 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="horizon-kuttl-tests/horizon-59cf497657-pm9sl" Dec 03 13:45:18 crc kubenswrapper[5052]: I1203 13:45:18.233092 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="horizon-kuttl-tests/horizon-577c75f77-gjq8d" podUID="bbdb204c-cb9d-4299-b5fb-3c76fc5649ce" containerName="horizon" probeResult="failure" output="Get \"http://10.217.0.72:8080/dashboard/auth/login/?next=/dashboard/\": dial tcp 10.217.0.72:8080: connect: connection refused" Dec 03 13:45:18 crc kubenswrapper[5052]: I1203 13:45:18.233209 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="horizon-kuttl-tests/horizon-577c75f77-gjq8d" Dec 03 13:45:21 crc kubenswrapper[5052]: I1203 13:45:21.672658 5052 scope.go:117] "RemoveContainer" containerID="2fbd293d6edef0fa059e91c119312e77b262534107c322feb7b59dbea3464526" Dec 03 13:45:21 crc kubenswrapper[5052]: E1203 13:45:21.673476 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 13:45:23 crc kubenswrapper[5052]: I1203 13:45:23.082743 5052 generic.go:334] "Generic (PLEG): container finished" podID="8beb534e-1595-4943-98be-598d43afec0d" containerID="5f2b3d40c83b81d9a2a083bd0efa68bbade657f18d624e2f09f489f9ccf1a658" exitCode=137 Dec 03 13:45:23 crc kubenswrapper[5052]: I1203 13:45:23.082783 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="horizon-kuttl-tests/horizon-59cf497657-pm9sl" event={"ID":"8beb534e-1595-4943-98be-598d43afec0d","Type":"ContainerDied","Data":"5f2b3d40c83b81d9a2a083bd0efa68bbade657f18d624e2f09f489f9ccf1a658"} Dec 03 13:45:23 crc kubenswrapper[5052]: I1203 13:45:23.649771 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="horizon-kuttl-tests/horizon-59cf497657-pm9sl" Dec 03 13:45:23 crc kubenswrapper[5052]: I1203 13:45:23.792876 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/8beb534e-1595-4943-98be-598d43afec0d-scripts\") pod \"8beb534e-1595-4943-98be-598d43afec0d\" (UID: \"8beb534e-1595-4943-98be-598d43afec0d\") " Dec 03 13:45:23 crc kubenswrapper[5052]: I1203 13:45:23.793141 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/8beb534e-1595-4943-98be-598d43afec0d-logs\") pod \"8beb534e-1595-4943-98be-598d43afec0d\" (UID: \"8beb534e-1595-4943-98be-598d43afec0d\") " Dec 03 13:45:23 crc kubenswrapper[5052]: I1203 13:45:23.793255 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-std6q\" (UniqueName: \"kubernetes.io/projected/8beb534e-1595-4943-98be-598d43afec0d-kube-api-access-std6q\") pod \"8beb534e-1595-4943-98be-598d43afec0d\" (UID: \"8beb534e-1595-4943-98be-598d43afec0d\") " Dec 03 13:45:23 crc kubenswrapper[5052]: I1203 13:45:23.793392 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"horizon-secret-key\" (UniqueName: \"kubernetes.io/secret/8beb534e-1595-4943-98be-598d43afec0d-horizon-secret-key\") pod \"8beb534e-1595-4943-98be-598d43afec0d\" (UID: \"8beb534e-1595-4943-98be-598d43afec0d\") " Dec 03 13:45:23 crc kubenswrapper[5052]: I1203 13:45:23.793440 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/8beb534e-1595-4943-98be-598d43afec0d-config-data\") pod \"8beb534e-1595-4943-98be-598d43afec0d\" (UID: \"8beb534e-1595-4943-98be-598d43afec0d\") " Dec 03 13:45:23 crc kubenswrapper[5052]: I1203 13:45:23.793802 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8beb534e-1595-4943-98be-598d43afec0d-logs" (OuterVolumeSpecName: "logs") pod "8beb534e-1595-4943-98be-598d43afec0d" (UID: "8beb534e-1595-4943-98be-598d43afec0d"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 13:45:23 crc kubenswrapper[5052]: I1203 13:45:23.793986 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/8beb534e-1595-4943-98be-598d43afec0d-logs\") on node \"crc\" DevicePath \"\"" Dec 03 13:45:23 crc kubenswrapper[5052]: I1203 13:45:23.798926 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8beb534e-1595-4943-98be-598d43afec0d-kube-api-access-std6q" (OuterVolumeSpecName: "kube-api-access-std6q") pod "8beb534e-1595-4943-98be-598d43afec0d" (UID: "8beb534e-1595-4943-98be-598d43afec0d"). InnerVolumeSpecName "kube-api-access-std6q". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:45:23 crc kubenswrapper[5052]: I1203 13:45:23.799782 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8beb534e-1595-4943-98be-598d43afec0d-horizon-secret-key" (OuterVolumeSpecName: "horizon-secret-key") pod "8beb534e-1595-4943-98be-598d43afec0d" (UID: "8beb534e-1595-4943-98be-598d43afec0d"). InnerVolumeSpecName "horizon-secret-key". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:45:23 crc kubenswrapper[5052]: I1203 13:45:23.816249 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8beb534e-1595-4943-98be-598d43afec0d-scripts" (OuterVolumeSpecName: "scripts") pod "8beb534e-1595-4943-98be-598d43afec0d" (UID: "8beb534e-1595-4943-98be-598d43afec0d"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:45:23 crc kubenswrapper[5052]: I1203 13:45:23.816315 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8beb534e-1595-4943-98be-598d43afec0d-config-data" (OuterVolumeSpecName: "config-data") pod "8beb534e-1595-4943-98be-598d43afec0d" (UID: "8beb534e-1595-4943-98be-598d43afec0d"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:45:23 crc kubenswrapper[5052]: I1203 13:45:23.895346 5052 reconciler_common.go:293] "Volume detached for volume \"horizon-secret-key\" (UniqueName: \"kubernetes.io/secret/8beb534e-1595-4943-98be-598d43afec0d-horizon-secret-key\") on node \"crc\" DevicePath \"\"" Dec 03 13:45:23 crc kubenswrapper[5052]: I1203 13:45:23.895380 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/8beb534e-1595-4943-98be-598d43afec0d-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 13:45:23 crc kubenswrapper[5052]: I1203 13:45:23.895391 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/8beb534e-1595-4943-98be-598d43afec0d-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 13:45:23 crc kubenswrapper[5052]: I1203 13:45:23.895405 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-std6q\" (UniqueName: \"kubernetes.io/projected/8beb534e-1595-4943-98be-598d43afec0d-kube-api-access-std6q\") on node \"crc\" DevicePath \"\"" Dec 03 13:45:24 crc kubenswrapper[5052]: I1203 13:45:24.092863 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="horizon-kuttl-tests/horizon-59cf497657-pm9sl" event={"ID":"8beb534e-1595-4943-98be-598d43afec0d","Type":"ContainerDied","Data":"88559fa26c90352ea25f76413a63d2c2df3c198cc2df2bffc32931a9b173792d"} Dec 03 13:45:24 crc kubenswrapper[5052]: I1203 13:45:24.092917 5052 scope.go:117] "RemoveContainer" containerID="8aa294f51880d733dd4145d59518ff9926094a73fdcf32b47d9039b653bff497" Dec 03 13:45:24 crc kubenswrapper[5052]: I1203 13:45:24.092928 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="horizon-kuttl-tests/horizon-59cf497657-pm9sl" Dec 03 13:45:24 crc kubenswrapper[5052]: I1203 13:45:24.127700 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["horizon-kuttl-tests/horizon-59cf497657-pm9sl"] Dec 03 13:45:24 crc kubenswrapper[5052]: I1203 13:45:24.133639 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["horizon-kuttl-tests/horizon-59cf497657-pm9sl"] Dec 03 13:45:24 crc kubenswrapper[5052]: E1203 13:45:24.208174 5052 cadvisor_stats_provider.go:516] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8beb534e_1595_4943_98be_598d43afec0d.slice\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8beb534e_1595_4943_98be_598d43afec0d.slice/crio-88559fa26c90352ea25f76413a63d2c2df3c198cc2df2bffc32931a9b173792d\": RecentStats: unable to find data in memory cache]" Dec 03 13:45:24 crc kubenswrapper[5052]: I1203 13:45:24.254049 5052 scope.go:117] "RemoveContainer" containerID="5f2b3d40c83b81d9a2a083bd0efa68bbade657f18d624e2f09f489f9ccf1a658" Dec 03 13:45:25 crc kubenswrapper[5052]: I1203 13:45:25.681447 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8beb534e-1595-4943-98be-598d43afec0d" path="/var/lib/kubelet/pods/8beb534e-1595-4943-98be-598d43afec0d/volumes" Dec 03 13:45:28 crc kubenswrapper[5052]: I1203 13:45:28.128608 5052 generic.go:334] "Generic (PLEG): container finished" podID="bbdb204c-cb9d-4299-b5fb-3c76fc5649ce" containerID="b508037e0f85ac50f298e6fe9ef4374889e3706d798dd65ad3856045cefbd3e1" exitCode=137 Dec 03 13:45:28 crc kubenswrapper[5052]: I1203 13:45:28.128660 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="horizon-kuttl-tests/horizon-577c75f77-gjq8d" event={"ID":"bbdb204c-cb9d-4299-b5fb-3c76fc5649ce","Type":"ContainerDied","Data":"b508037e0f85ac50f298e6fe9ef4374889e3706d798dd65ad3856045cefbd3e1"} Dec 03 13:45:28 crc kubenswrapper[5052]: I1203 13:45:28.342253 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="horizon-kuttl-tests/horizon-577c75f77-gjq8d" Dec 03 13:45:28 crc kubenswrapper[5052]: I1203 13:45:28.468082 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dz5jr\" (UniqueName: \"kubernetes.io/projected/bbdb204c-cb9d-4299-b5fb-3c76fc5649ce-kube-api-access-dz5jr\") pod \"bbdb204c-cb9d-4299-b5fb-3c76fc5649ce\" (UID: \"bbdb204c-cb9d-4299-b5fb-3c76fc5649ce\") " Dec 03 13:45:28 crc kubenswrapper[5052]: I1203 13:45:28.468198 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/bbdb204c-cb9d-4299-b5fb-3c76fc5649ce-config-data\") pod \"bbdb204c-cb9d-4299-b5fb-3c76fc5649ce\" (UID: \"bbdb204c-cb9d-4299-b5fb-3c76fc5649ce\") " Dec 03 13:45:28 crc kubenswrapper[5052]: I1203 13:45:28.468246 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/bbdb204c-cb9d-4299-b5fb-3c76fc5649ce-logs\") pod \"bbdb204c-cb9d-4299-b5fb-3c76fc5649ce\" (UID: \"bbdb204c-cb9d-4299-b5fb-3c76fc5649ce\") " Dec 03 13:45:28 crc kubenswrapper[5052]: I1203 13:45:28.468365 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"horizon-secret-key\" (UniqueName: \"kubernetes.io/secret/bbdb204c-cb9d-4299-b5fb-3c76fc5649ce-horizon-secret-key\") pod \"bbdb204c-cb9d-4299-b5fb-3c76fc5649ce\" (UID: \"bbdb204c-cb9d-4299-b5fb-3c76fc5649ce\") " Dec 03 13:45:28 crc kubenswrapper[5052]: I1203 13:45:28.468409 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/bbdb204c-cb9d-4299-b5fb-3c76fc5649ce-scripts\") pod \"bbdb204c-cb9d-4299-b5fb-3c76fc5649ce\" (UID: \"bbdb204c-cb9d-4299-b5fb-3c76fc5649ce\") " Dec 03 13:45:28 crc kubenswrapper[5052]: I1203 13:45:28.469295 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bbdb204c-cb9d-4299-b5fb-3c76fc5649ce-logs" (OuterVolumeSpecName: "logs") pod "bbdb204c-cb9d-4299-b5fb-3c76fc5649ce" (UID: "bbdb204c-cb9d-4299-b5fb-3c76fc5649ce"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 13:45:28 crc kubenswrapper[5052]: I1203 13:45:28.475041 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bbdb204c-cb9d-4299-b5fb-3c76fc5649ce-horizon-secret-key" (OuterVolumeSpecName: "horizon-secret-key") pod "bbdb204c-cb9d-4299-b5fb-3c76fc5649ce" (UID: "bbdb204c-cb9d-4299-b5fb-3c76fc5649ce"). InnerVolumeSpecName "horizon-secret-key". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:45:28 crc kubenswrapper[5052]: I1203 13:45:28.475961 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bbdb204c-cb9d-4299-b5fb-3c76fc5649ce-kube-api-access-dz5jr" (OuterVolumeSpecName: "kube-api-access-dz5jr") pod "bbdb204c-cb9d-4299-b5fb-3c76fc5649ce" (UID: "bbdb204c-cb9d-4299-b5fb-3c76fc5649ce"). InnerVolumeSpecName "kube-api-access-dz5jr". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:45:28 crc kubenswrapper[5052]: I1203 13:45:28.491402 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bbdb204c-cb9d-4299-b5fb-3c76fc5649ce-scripts" (OuterVolumeSpecName: "scripts") pod "bbdb204c-cb9d-4299-b5fb-3c76fc5649ce" (UID: "bbdb204c-cb9d-4299-b5fb-3c76fc5649ce"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:45:28 crc kubenswrapper[5052]: I1203 13:45:28.493021 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bbdb204c-cb9d-4299-b5fb-3c76fc5649ce-config-data" (OuterVolumeSpecName: "config-data") pod "bbdb204c-cb9d-4299-b5fb-3c76fc5649ce" (UID: "bbdb204c-cb9d-4299-b5fb-3c76fc5649ce"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:45:28 crc kubenswrapper[5052]: I1203 13:45:28.570749 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-dz5jr\" (UniqueName: \"kubernetes.io/projected/bbdb204c-cb9d-4299-b5fb-3c76fc5649ce-kube-api-access-dz5jr\") on node \"crc\" DevicePath \"\"" Dec 03 13:45:28 crc kubenswrapper[5052]: I1203 13:45:28.570783 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/bbdb204c-cb9d-4299-b5fb-3c76fc5649ce-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 13:45:28 crc kubenswrapper[5052]: I1203 13:45:28.570794 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/bbdb204c-cb9d-4299-b5fb-3c76fc5649ce-logs\") on node \"crc\" DevicePath \"\"" Dec 03 13:45:28 crc kubenswrapper[5052]: I1203 13:45:28.570803 5052 reconciler_common.go:293] "Volume detached for volume \"horizon-secret-key\" (UniqueName: \"kubernetes.io/secret/bbdb204c-cb9d-4299-b5fb-3c76fc5649ce-horizon-secret-key\") on node \"crc\" DevicePath \"\"" Dec 03 13:45:28 crc kubenswrapper[5052]: I1203 13:45:28.570812 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/bbdb204c-cb9d-4299-b5fb-3c76fc5649ce-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 13:45:29 crc kubenswrapper[5052]: I1203 13:45:29.140740 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="horizon-kuttl-tests/horizon-577c75f77-gjq8d" event={"ID":"bbdb204c-cb9d-4299-b5fb-3c76fc5649ce","Type":"ContainerDied","Data":"aba5be2d18d55a71ed507bb93dd44ecc0faa0187dd63b3d48e07980828db3d5d"} Dec 03 13:45:29 crc kubenswrapper[5052]: I1203 13:45:29.140819 5052 scope.go:117] "RemoveContainer" containerID="a13c586622babcda45ba98ecf179ccc5d357cbaa30c59aeb86edd53d3f383621" Dec 03 13:45:29 crc kubenswrapper[5052]: I1203 13:45:29.140973 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="horizon-kuttl-tests/horizon-577c75f77-gjq8d" Dec 03 13:45:29 crc kubenswrapper[5052]: I1203 13:45:29.173794 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["horizon-kuttl-tests/horizon-577c75f77-gjq8d"] Dec 03 13:45:29 crc kubenswrapper[5052]: I1203 13:45:29.184245 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["horizon-kuttl-tests/horizon-577c75f77-gjq8d"] Dec 03 13:45:29 crc kubenswrapper[5052]: I1203 13:45:29.290351 5052 scope.go:117] "RemoveContainer" containerID="b508037e0f85ac50f298e6fe9ef4374889e3706d798dd65ad3856045cefbd3e1" Dec 03 13:45:29 crc kubenswrapper[5052]: I1203 13:45:29.687356 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bbdb204c-cb9d-4299-b5fb-3c76fc5649ce" path="/var/lib/kubelet/pods/bbdb204c-cb9d-4299-b5fb-3c76fc5649ce/volumes" Dec 03 13:45:30 crc kubenswrapper[5052]: I1203 13:45:30.052776 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["horizon-kuttl-tests/horizon-85b965564b-7lzxg"] Dec 03 13:45:30 crc kubenswrapper[5052]: E1203 13:45:30.053132 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="9ce53d74-f6a1-48b6-b39b-dd9b2a04a8db" containerName="registry-server" Dec 03 13:45:30 crc kubenswrapper[5052]: I1203 13:45:30.053150 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ce53d74-f6a1-48b6-b39b-dd9b2a04a8db" containerName="registry-server" Dec 03 13:45:30 crc kubenswrapper[5052]: E1203 13:45:30.053164 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="8beb534e-1595-4943-98be-598d43afec0d" containerName="horizon" Dec 03 13:45:30 crc kubenswrapper[5052]: I1203 13:45:30.053172 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="8beb534e-1595-4943-98be-598d43afec0d" containerName="horizon" Dec 03 13:45:30 crc kubenswrapper[5052]: E1203 13:45:30.053187 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="bbdb204c-cb9d-4299-b5fb-3c76fc5649ce" containerName="horizon-log" Dec 03 13:45:30 crc kubenswrapper[5052]: I1203 13:45:30.053196 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="bbdb204c-cb9d-4299-b5fb-3c76fc5649ce" containerName="horizon-log" Dec 03 13:45:30 crc kubenswrapper[5052]: E1203 13:45:30.053214 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="9ce53d74-f6a1-48b6-b39b-dd9b2a04a8db" containerName="extract-content" Dec 03 13:45:30 crc kubenswrapper[5052]: I1203 13:45:30.053220 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ce53d74-f6a1-48b6-b39b-dd9b2a04a8db" containerName="extract-content" Dec 03 13:45:30 crc kubenswrapper[5052]: E1203 13:45:30.053231 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="bbdb204c-cb9d-4299-b5fb-3c76fc5649ce" containerName="horizon" Dec 03 13:45:30 crc kubenswrapper[5052]: I1203 13:45:30.053239 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="bbdb204c-cb9d-4299-b5fb-3c76fc5649ce" containerName="horizon" Dec 03 13:45:30 crc kubenswrapper[5052]: E1203 13:45:30.053257 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="9ce53d74-f6a1-48b6-b39b-dd9b2a04a8db" containerName="extract-utilities" Dec 03 13:45:30 crc kubenswrapper[5052]: I1203 13:45:30.053264 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ce53d74-f6a1-48b6-b39b-dd9b2a04a8db" containerName="extract-utilities" Dec 03 13:45:30 crc kubenswrapper[5052]: E1203 13:45:30.053280 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="73544e7c-ad0a-4443-982d-2bde1474ec9b" containerName="collect-profiles" Dec 03 13:45:30 crc kubenswrapper[5052]: I1203 13:45:30.053287 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="73544e7c-ad0a-4443-982d-2bde1474ec9b" containerName="collect-profiles" Dec 03 13:45:30 crc kubenswrapper[5052]: E1203 13:45:30.053300 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="8beb534e-1595-4943-98be-598d43afec0d" containerName="horizon-log" Dec 03 13:45:30 crc kubenswrapper[5052]: I1203 13:45:30.053307 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="8beb534e-1595-4943-98be-598d43afec0d" containerName="horizon-log" Dec 03 13:45:30 crc kubenswrapper[5052]: I1203 13:45:30.053473 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="73544e7c-ad0a-4443-982d-2bde1474ec9b" containerName="collect-profiles" Dec 03 13:45:30 crc kubenswrapper[5052]: I1203 13:45:30.053500 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="8beb534e-1595-4943-98be-598d43afec0d" containerName="horizon-log" Dec 03 13:45:30 crc kubenswrapper[5052]: I1203 13:45:30.053575 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="8beb534e-1595-4943-98be-598d43afec0d" containerName="horizon" Dec 03 13:45:30 crc kubenswrapper[5052]: I1203 13:45:30.053590 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="bbdb204c-cb9d-4299-b5fb-3c76fc5649ce" containerName="horizon-log" Dec 03 13:45:30 crc kubenswrapper[5052]: I1203 13:45:30.053598 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="9ce53d74-f6a1-48b6-b39b-dd9b2a04a8db" containerName="registry-server" Dec 03 13:45:30 crc kubenswrapper[5052]: I1203 13:45:30.053608 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="bbdb204c-cb9d-4299-b5fb-3c76fc5649ce" containerName="horizon" Dec 03 13:45:30 crc kubenswrapper[5052]: I1203 13:45:30.054481 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="horizon-kuttl-tests/horizon-85b965564b-7lzxg" Dec 03 13:45:30 crc kubenswrapper[5052]: I1203 13:45:30.057053 5052 reflector.go:368] Caches populated for *v1.Secret from object-"horizon-kuttl-tests"/"cert-horizon-svc" Dec 03 13:45:30 crc kubenswrapper[5052]: I1203 13:45:30.057106 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"horizon-kuttl-tests"/"horizon-scripts" Dec 03 13:45:30 crc kubenswrapper[5052]: I1203 13:45:30.057147 5052 reflector.go:368] Caches populated for *v1.Secret from object-"horizon-kuttl-tests"/"horizon" Dec 03 13:45:30 crc kubenswrapper[5052]: I1203 13:45:30.057295 5052 reflector.go:368] Caches populated for *v1.Secret from object-"horizon-kuttl-tests"/"horizon-horizon-dockercfg-dbntz" Dec 03 13:45:30 crc kubenswrapper[5052]: I1203 13:45:30.057310 5052 reflector.go:368] Caches populated for *v1.Secret from object-"horizon-kuttl-tests"/"combined-ca-bundle" Dec 03 13:45:30 crc kubenswrapper[5052]: I1203 13:45:30.058455 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"horizon-kuttl-tests"/"horizon-config-data" Dec 03 13:45:30 crc kubenswrapper[5052]: I1203 13:45:30.073955 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["horizon-kuttl-tests/horizon-85b965564b-7lzxg"] Dec 03 13:45:30 crc kubenswrapper[5052]: I1203 13:45:30.096437 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"horizon-tls-certs\" (UniqueName: \"kubernetes.io/secret/325d2988-e676-4f3f-8e31-8af823f54962-horizon-tls-certs\") pod \"horizon-85b965564b-7lzxg\" (UID: \"325d2988-e676-4f3f-8e31-8af823f54962\") " pod="horizon-kuttl-tests/horizon-85b965564b-7lzxg" Dec 03 13:45:30 crc kubenswrapper[5052]: I1203 13:45:30.096493 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/325d2988-e676-4f3f-8e31-8af823f54962-config-data\") pod \"horizon-85b965564b-7lzxg\" (UID: \"325d2988-e676-4f3f-8e31-8af823f54962\") " pod="horizon-kuttl-tests/horizon-85b965564b-7lzxg" Dec 03 13:45:30 crc kubenswrapper[5052]: I1203 13:45:30.096583 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"horizon-secret-key\" (UniqueName: \"kubernetes.io/secret/325d2988-e676-4f3f-8e31-8af823f54962-horizon-secret-key\") pod \"horizon-85b965564b-7lzxg\" (UID: \"325d2988-e676-4f3f-8e31-8af823f54962\") " pod="horizon-kuttl-tests/horizon-85b965564b-7lzxg" Dec 03 13:45:30 crc kubenswrapper[5052]: I1203 13:45:30.096637 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-88vpt\" (UniqueName: \"kubernetes.io/projected/325d2988-e676-4f3f-8e31-8af823f54962-kube-api-access-88vpt\") pod \"horizon-85b965564b-7lzxg\" (UID: \"325d2988-e676-4f3f-8e31-8af823f54962\") " pod="horizon-kuttl-tests/horizon-85b965564b-7lzxg" Dec 03 13:45:30 crc kubenswrapper[5052]: I1203 13:45:30.096668 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/325d2988-e676-4f3f-8e31-8af823f54962-logs\") pod \"horizon-85b965564b-7lzxg\" (UID: \"325d2988-e676-4f3f-8e31-8af823f54962\") " pod="horizon-kuttl-tests/horizon-85b965564b-7lzxg" Dec 03 13:45:30 crc kubenswrapper[5052]: I1203 13:45:30.096695 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/325d2988-e676-4f3f-8e31-8af823f54962-combined-ca-bundle\") pod \"horizon-85b965564b-7lzxg\" (UID: \"325d2988-e676-4f3f-8e31-8af823f54962\") " pod="horizon-kuttl-tests/horizon-85b965564b-7lzxg" Dec 03 13:45:30 crc kubenswrapper[5052]: I1203 13:45:30.096723 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/325d2988-e676-4f3f-8e31-8af823f54962-scripts\") pod \"horizon-85b965564b-7lzxg\" (UID: \"325d2988-e676-4f3f-8e31-8af823f54962\") " pod="horizon-kuttl-tests/horizon-85b965564b-7lzxg" Dec 03 13:45:30 crc kubenswrapper[5052]: I1203 13:45:30.137021 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["horizon-kuttl-tests/horizon-6b4d9768c6-rmfmj"] Dec 03 13:45:30 crc kubenswrapper[5052]: I1203 13:45:30.138276 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="horizon-kuttl-tests/horizon-6b4d9768c6-rmfmj" Dec 03 13:45:30 crc kubenswrapper[5052]: I1203 13:45:30.154070 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["horizon-kuttl-tests/horizon-6b4d9768c6-rmfmj"] Dec 03 13:45:30 crc kubenswrapper[5052]: I1203 13:45:30.199650 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-llps6\" (UniqueName: \"kubernetes.io/projected/9616ad60-9cfd-45bf-b9f7-19c14dc2e4e1-kube-api-access-llps6\") pod \"horizon-6b4d9768c6-rmfmj\" (UID: \"9616ad60-9cfd-45bf-b9f7-19c14dc2e4e1\") " pod="horizon-kuttl-tests/horizon-6b4d9768c6-rmfmj" Dec 03 13:45:30 crc kubenswrapper[5052]: I1203 13:45:30.199817 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/325d2988-e676-4f3f-8e31-8af823f54962-config-data\") pod \"horizon-85b965564b-7lzxg\" (UID: \"325d2988-e676-4f3f-8e31-8af823f54962\") " pod="horizon-kuttl-tests/horizon-85b965564b-7lzxg" Dec 03 13:45:30 crc kubenswrapper[5052]: I1203 13:45:30.199926 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/9616ad60-9cfd-45bf-b9f7-19c14dc2e4e1-logs\") pod \"horizon-6b4d9768c6-rmfmj\" (UID: \"9616ad60-9cfd-45bf-b9f7-19c14dc2e4e1\") " pod="horizon-kuttl-tests/horizon-6b4d9768c6-rmfmj" Dec 03 13:45:30 crc kubenswrapper[5052]: I1203 13:45:30.200023 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"horizon-secret-key\" (UniqueName: \"kubernetes.io/secret/9616ad60-9cfd-45bf-b9f7-19c14dc2e4e1-horizon-secret-key\") pod \"horizon-6b4d9768c6-rmfmj\" (UID: \"9616ad60-9cfd-45bf-b9f7-19c14dc2e4e1\") " pod="horizon-kuttl-tests/horizon-6b4d9768c6-rmfmj" Dec 03 13:45:30 crc kubenswrapper[5052]: I1203 13:45:30.200054 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"horizon-secret-key\" (UniqueName: \"kubernetes.io/secret/325d2988-e676-4f3f-8e31-8af823f54962-horizon-secret-key\") pod \"horizon-85b965564b-7lzxg\" (UID: \"325d2988-e676-4f3f-8e31-8af823f54962\") " pod="horizon-kuttl-tests/horizon-85b965564b-7lzxg" Dec 03 13:45:30 crc kubenswrapper[5052]: I1203 13:45:30.200115 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/9616ad60-9cfd-45bf-b9f7-19c14dc2e4e1-scripts\") pod \"horizon-6b4d9768c6-rmfmj\" (UID: \"9616ad60-9cfd-45bf-b9f7-19c14dc2e4e1\") " pod="horizon-kuttl-tests/horizon-6b4d9768c6-rmfmj" Dec 03 13:45:30 crc kubenswrapper[5052]: I1203 13:45:30.200147 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-88vpt\" (UniqueName: \"kubernetes.io/projected/325d2988-e676-4f3f-8e31-8af823f54962-kube-api-access-88vpt\") pod \"horizon-85b965564b-7lzxg\" (UID: \"325d2988-e676-4f3f-8e31-8af823f54962\") " pod="horizon-kuttl-tests/horizon-85b965564b-7lzxg" Dec 03 13:45:30 crc kubenswrapper[5052]: I1203 13:45:30.200193 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/325d2988-e676-4f3f-8e31-8af823f54962-logs\") pod \"horizon-85b965564b-7lzxg\" (UID: \"325d2988-e676-4f3f-8e31-8af823f54962\") " pod="horizon-kuttl-tests/horizon-85b965564b-7lzxg" Dec 03 13:45:30 crc kubenswrapper[5052]: I1203 13:45:30.200242 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/325d2988-e676-4f3f-8e31-8af823f54962-combined-ca-bundle\") pod \"horizon-85b965564b-7lzxg\" (UID: \"325d2988-e676-4f3f-8e31-8af823f54962\") " pod="horizon-kuttl-tests/horizon-85b965564b-7lzxg" Dec 03 13:45:30 crc kubenswrapper[5052]: I1203 13:45:30.200290 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/9616ad60-9cfd-45bf-b9f7-19c14dc2e4e1-config-data\") pod \"horizon-6b4d9768c6-rmfmj\" (UID: \"9616ad60-9cfd-45bf-b9f7-19c14dc2e4e1\") " pod="horizon-kuttl-tests/horizon-6b4d9768c6-rmfmj" Dec 03 13:45:30 crc kubenswrapper[5052]: I1203 13:45:30.200321 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/325d2988-e676-4f3f-8e31-8af823f54962-scripts\") pod \"horizon-85b965564b-7lzxg\" (UID: \"325d2988-e676-4f3f-8e31-8af823f54962\") " pod="horizon-kuttl-tests/horizon-85b965564b-7lzxg" Dec 03 13:45:30 crc kubenswrapper[5052]: I1203 13:45:30.200372 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"horizon-tls-certs\" (UniqueName: \"kubernetes.io/secret/9616ad60-9cfd-45bf-b9f7-19c14dc2e4e1-horizon-tls-certs\") pod \"horizon-6b4d9768c6-rmfmj\" (UID: \"9616ad60-9cfd-45bf-b9f7-19c14dc2e4e1\") " pod="horizon-kuttl-tests/horizon-6b4d9768c6-rmfmj" Dec 03 13:45:30 crc kubenswrapper[5052]: I1203 13:45:30.200424 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9616ad60-9cfd-45bf-b9f7-19c14dc2e4e1-combined-ca-bundle\") pod \"horizon-6b4d9768c6-rmfmj\" (UID: \"9616ad60-9cfd-45bf-b9f7-19c14dc2e4e1\") " pod="horizon-kuttl-tests/horizon-6b4d9768c6-rmfmj" Dec 03 13:45:30 crc kubenswrapper[5052]: I1203 13:45:30.200737 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"horizon-tls-certs\" (UniqueName: \"kubernetes.io/secret/325d2988-e676-4f3f-8e31-8af823f54962-horizon-tls-certs\") pod \"horizon-85b965564b-7lzxg\" (UID: \"325d2988-e676-4f3f-8e31-8af823f54962\") " pod="horizon-kuttl-tests/horizon-85b965564b-7lzxg" Dec 03 13:45:30 crc kubenswrapper[5052]: I1203 13:45:30.200913 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/325d2988-e676-4f3f-8e31-8af823f54962-logs\") pod \"horizon-85b965564b-7lzxg\" (UID: \"325d2988-e676-4f3f-8e31-8af823f54962\") " pod="horizon-kuttl-tests/horizon-85b965564b-7lzxg" Dec 03 13:45:30 crc kubenswrapper[5052]: I1203 13:45:30.201384 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/325d2988-e676-4f3f-8e31-8af823f54962-config-data\") pod \"horizon-85b965564b-7lzxg\" (UID: \"325d2988-e676-4f3f-8e31-8af823f54962\") " pod="horizon-kuttl-tests/horizon-85b965564b-7lzxg" Dec 03 13:45:30 crc kubenswrapper[5052]: I1203 13:45:30.202385 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/325d2988-e676-4f3f-8e31-8af823f54962-scripts\") pod \"horizon-85b965564b-7lzxg\" (UID: \"325d2988-e676-4f3f-8e31-8af823f54962\") " pod="horizon-kuttl-tests/horizon-85b965564b-7lzxg" Dec 03 13:45:30 crc kubenswrapper[5052]: I1203 13:45:30.206500 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"horizon-tls-certs\" (UniqueName: \"kubernetes.io/secret/325d2988-e676-4f3f-8e31-8af823f54962-horizon-tls-certs\") pod \"horizon-85b965564b-7lzxg\" (UID: \"325d2988-e676-4f3f-8e31-8af823f54962\") " pod="horizon-kuttl-tests/horizon-85b965564b-7lzxg" Dec 03 13:45:30 crc kubenswrapper[5052]: I1203 13:45:30.206498 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/325d2988-e676-4f3f-8e31-8af823f54962-combined-ca-bundle\") pod \"horizon-85b965564b-7lzxg\" (UID: \"325d2988-e676-4f3f-8e31-8af823f54962\") " pod="horizon-kuttl-tests/horizon-85b965564b-7lzxg" Dec 03 13:45:30 crc kubenswrapper[5052]: I1203 13:45:30.206976 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"horizon-secret-key\" (UniqueName: \"kubernetes.io/secret/325d2988-e676-4f3f-8e31-8af823f54962-horizon-secret-key\") pod \"horizon-85b965564b-7lzxg\" (UID: \"325d2988-e676-4f3f-8e31-8af823f54962\") " pod="horizon-kuttl-tests/horizon-85b965564b-7lzxg" Dec 03 13:45:30 crc kubenswrapper[5052]: I1203 13:45:30.219349 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-88vpt\" (UniqueName: \"kubernetes.io/projected/325d2988-e676-4f3f-8e31-8af823f54962-kube-api-access-88vpt\") pod \"horizon-85b965564b-7lzxg\" (UID: \"325d2988-e676-4f3f-8e31-8af823f54962\") " pod="horizon-kuttl-tests/horizon-85b965564b-7lzxg" Dec 03 13:45:30 crc kubenswrapper[5052]: I1203 13:45:30.301798 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-llps6\" (UniqueName: \"kubernetes.io/projected/9616ad60-9cfd-45bf-b9f7-19c14dc2e4e1-kube-api-access-llps6\") pod \"horizon-6b4d9768c6-rmfmj\" (UID: \"9616ad60-9cfd-45bf-b9f7-19c14dc2e4e1\") " pod="horizon-kuttl-tests/horizon-6b4d9768c6-rmfmj" Dec 03 13:45:30 crc kubenswrapper[5052]: I1203 13:45:30.301918 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/9616ad60-9cfd-45bf-b9f7-19c14dc2e4e1-logs\") pod \"horizon-6b4d9768c6-rmfmj\" (UID: \"9616ad60-9cfd-45bf-b9f7-19c14dc2e4e1\") " pod="horizon-kuttl-tests/horizon-6b4d9768c6-rmfmj" Dec 03 13:45:30 crc kubenswrapper[5052]: I1203 13:45:30.301960 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"horizon-secret-key\" (UniqueName: \"kubernetes.io/secret/9616ad60-9cfd-45bf-b9f7-19c14dc2e4e1-horizon-secret-key\") pod \"horizon-6b4d9768c6-rmfmj\" (UID: \"9616ad60-9cfd-45bf-b9f7-19c14dc2e4e1\") " pod="horizon-kuttl-tests/horizon-6b4d9768c6-rmfmj" Dec 03 13:45:30 crc kubenswrapper[5052]: I1203 13:45:30.301991 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/9616ad60-9cfd-45bf-b9f7-19c14dc2e4e1-scripts\") pod \"horizon-6b4d9768c6-rmfmj\" (UID: \"9616ad60-9cfd-45bf-b9f7-19c14dc2e4e1\") " pod="horizon-kuttl-tests/horizon-6b4d9768c6-rmfmj" Dec 03 13:45:30 crc kubenswrapper[5052]: I1203 13:45:30.302023 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/9616ad60-9cfd-45bf-b9f7-19c14dc2e4e1-config-data\") pod \"horizon-6b4d9768c6-rmfmj\" (UID: \"9616ad60-9cfd-45bf-b9f7-19c14dc2e4e1\") " pod="horizon-kuttl-tests/horizon-6b4d9768c6-rmfmj" Dec 03 13:45:30 crc kubenswrapper[5052]: I1203 13:45:30.302052 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"horizon-tls-certs\" (UniqueName: \"kubernetes.io/secret/9616ad60-9cfd-45bf-b9f7-19c14dc2e4e1-horizon-tls-certs\") pod \"horizon-6b4d9768c6-rmfmj\" (UID: \"9616ad60-9cfd-45bf-b9f7-19c14dc2e4e1\") " pod="horizon-kuttl-tests/horizon-6b4d9768c6-rmfmj" Dec 03 13:45:30 crc kubenswrapper[5052]: I1203 13:45:30.302077 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9616ad60-9cfd-45bf-b9f7-19c14dc2e4e1-combined-ca-bundle\") pod \"horizon-6b4d9768c6-rmfmj\" (UID: \"9616ad60-9cfd-45bf-b9f7-19c14dc2e4e1\") " pod="horizon-kuttl-tests/horizon-6b4d9768c6-rmfmj" Dec 03 13:45:30 crc kubenswrapper[5052]: I1203 13:45:30.302363 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/9616ad60-9cfd-45bf-b9f7-19c14dc2e4e1-logs\") pod \"horizon-6b4d9768c6-rmfmj\" (UID: \"9616ad60-9cfd-45bf-b9f7-19c14dc2e4e1\") " pod="horizon-kuttl-tests/horizon-6b4d9768c6-rmfmj" Dec 03 13:45:30 crc kubenswrapper[5052]: I1203 13:45:30.302797 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/9616ad60-9cfd-45bf-b9f7-19c14dc2e4e1-scripts\") pod \"horizon-6b4d9768c6-rmfmj\" (UID: \"9616ad60-9cfd-45bf-b9f7-19c14dc2e4e1\") " pod="horizon-kuttl-tests/horizon-6b4d9768c6-rmfmj" Dec 03 13:45:30 crc kubenswrapper[5052]: I1203 13:45:30.303726 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/9616ad60-9cfd-45bf-b9f7-19c14dc2e4e1-config-data\") pod \"horizon-6b4d9768c6-rmfmj\" (UID: \"9616ad60-9cfd-45bf-b9f7-19c14dc2e4e1\") " pod="horizon-kuttl-tests/horizon-6b4d9768c6-rmfmj" Dec 03 13:45:30 crc kubenswrapper[5052]: I1203 13:45:30.306209 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"horizon-secret-key\" (UniqueName: \"kubernetes.io/secret/9616ad60-9cfd-45bf-b9f7-19c14dc2e4e1-horizon-secret-key\") pod \"horizon-6b4d9768c6-rmfmj\" (UID: \"9616ad60-9cfd-45bf-b9f7-19c14dc2e4e1\") " pod="horizon-kuttl-tests/horizon-6b4d9768c6-rmfmj" Dec 03 13:45:30 crc kubenswrapper[5052]: I1203 13:45:30.306302 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"horizon-tls-certs\" (UniqueName: \"kubernetes.io/secret/9616ad60-9cfd-45bf-b9f7-19c14dc2e4e1-horizon-tls-certs\") pod \"horizon-6b4d9768c6-rmfmj\" (UID: \"9616ad60-9cfd-45bf-b9f7-19c14dc2e4e1\") " pod="horizon-kuttl-tests/horizon-6b4d9768c6-rmfmj" Dec 03 13:45:30 crc kubenswrapper[5052]: I1203 13:45:30.306640 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9616ad60-9cfd-45bf-b9f7-19c14dc2e4e1-combined-ca-bundle\") pod \"horizon-6b4d9768c6-rmfmj\" (UID: \"9616ad60-9cfd-45bf-b9f7-19c14dc2e4e1\") " pod="horizon-kuttl-tests/horizon-6b4d9768c6-rmfmj" Dec 03 13:45:30 crc kubenswrapper[5052]: I1203 13:45:30.316790 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-llps6\" (UniqueName: \"kubernetes.io/projected/9616ad60-9cfd-45bf-b9f7-19c14dc2e4e1-kube-api-access-llps6\") pod \"horizon-6b4d9768c6-rmfmj\" (UID: \"9616ad60-9cfd-45bf-b9f7-19c14dc2e4e1\") " pod="horizon-kuttl-tests/horizon-6b4d9768c6-rmfmj" Dec 03 13:45:30 crc kubenswrapper[5052]: I1203 13:45:30.376595 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="horizon-kuttl-tests/horizon-85b965564b-7lzxg" Dec 03 13:45:30 crc kubenswrapper[5052]: I1203 13:45:30.453469 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="horizon-kuttl-tests/horizon-6b4d9768c6-rmfmj" Dec 03 13:45:30 crc kubenswrapper[5052]: I1203 13:45:30.814121 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["horizon-kuttl-tests/horizon-85b965564b-7lzxg"] Dec 03 13:45:30 crc kubenswrapper[5052]: I1203 13:45:30.857549 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["horizon-kuttl-tests/horizon-6b4d9768c6-rmfmj"] Dec 03 13:45:30 crc kubenswrapper[5052]: W1203 13:45:30.862580 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9616ad60_9cfd_45bf_b9f7_19c14dc2e4e1.slice/crio-9bffb1566d98b6101a6427a3960df494c186d5728b1920df052d72a96bce2a07 WatchSource:0}: Error finding container 9bffb1566d98b6101a6427a3960df494c186d5728b1920df052d72a96bce2a07: Status 404 returned error can't find the container with id 9bffb1566d98b6101a6427a3960df494c186d5728b1920df052d72a96bce2a07 Dec 03 13:45:31 crc kubenswrapper[5052]: I1203 13:45:31.163827 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="horizon-kuttl-tests/horizon-85b965564b-7lzxg" event={"ID":"325d2988-e676-4f3f-8e31-8af823f54962","Type":"ContainerStarted","Data":"13783643ae78728d4ca2849a82962b25d95f266723e5431696c56dab53a770bc"} Dec 03 13:45:31 crc kubenswrapper[5052]: I1203 13:45:31.163886 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="horizon-kuttl-tests/horizon-85b965564b-7lzxg" event={"ID":"325d2988-e676-4f3f-8e31-8af823f54962","Type":"ContainerStarted","Data":"da3dc23ba250ea46313a9cd9250f558d7ebff28efe8d24ad700efbabfb5812fb"} Dec 03 13:45:31 crc kubenswrapper[5052]: I1203 13:45:31.163903 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="horizon-kuttl-tests/horizon-85b965564b-7lzxg" event={"ID":"325d2988-e676-4f3f-8e31-8af823f54962","Type":"ContainerStarted","Data":"01178513b4b05251eb08dcff783c20cefe3c41a3e58b0a21fae3353a85455731"} Dec 03 13:45:31 crc kubenswrapper[5052]: I1203 13:45:31.166870 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="horizon-kuttl-tests/horizon-6b4d9768c6-rmfmj" event={"ID":"9616ad60-9cfd-45bf-b9f7-19c14dc2e4e1","Type":"ContainerStarted","Data":"7fe593c35e2399494e5984380b482861b053fd02e044d4c4257b0f30bcb7694c"} Dec 03 13:45:31 crc kubenswrapper[5052]: I1203 13:45:31.166940 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="horizon-kuttl-tests/horizon-6b4d9768c6-rmfmj" event={"ID":"9616ad60-9cfd-45bf-b9f7-19c14dc2e4e1","Type":"ContainerStarted","Data":"75270f145a47b408181bcd64c713610f2299ffc32f92a623c175305f871066c1"} Dec 03 13:45:31 crc kubenswrapper[5052]: I1203 13:45:31.167004 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="horizon-kuttl-tests/horizon-6b4d9768c6-rmfmj" event={"ID":"9616ad60-9cfd-45bf-b9f7-19c14dc2e4e1","Type":"ContainerStarted","Data":"9bffb1566d98b6101a6427a3960df494c186d5728b1920df052d72a96bce2a07"} Dec 03 13:45:31 crc kubenswrapper[5052]: I1203 13:45:31.237707 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="horizon-kuttl-tests/horizon-6b4d9768c6-rmfmj" podStartSLOduration=1.237683048 podStartE2EDuration="1.237683048s" podCreationTimestamp="2025-12-03 13:45:30 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 13:45:31.233386265 +0000 UTC m=+8721.821628029" watchObservedRunningTime="2025-12-03 13:45:31.237683048 +0000 UTC m=+8721.825924812" Dec 03 13:45:31 crc kubenswrapper[5052]: I1203 13:45:31.239089 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="horizon-kuttl-tests/horizon-85b965564b-7lzxg" podStartSLOduration=1.239081238 podStartE2EDuration="1.239081238s" podCreationTimestamp="2025-12-03 13:45:30 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 13:45:31.198681379 +0000 UTC m=+8721.786923163" watchObservedRunningTime="2025-12-03 13:45:31.239081238 +0000 UTC m=+8721.827323012" Dec 03 13:45:33 crc kubenswrapper[5052]: I1203 13:45:33.234478 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="horizon-kuttl-tests/horizon-577c75f77-gjq8d" podUID="bbdb204c-cb9d-4299-b5fb-3c76fc5649ce" containerName="horizon" probeResult="failure" output="Get \"http://10.217.0.72:8080/dashboard/auth/login/?next=/dashboard/\": dial tcp 10.217.0.72:8080: i/o timeout (Client.Timeout exceeded while awaiting headers)" Dec 03 13:45:34 crc kubenswrapper[5052]: I1203 13:45:34.673282 5052 scope.go:117] "RemoveContainer" containerID="2fbd293d6edef0fa059e91c119312e77b262534107c322feb7b59dbea3464526" Dec 03 13:45:34 crc kubenswrapper[5052]: E1203 13:45:34.674480 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 13:45:40 crc kubenswrapper[5052]: I1203 13:45:40.377830 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="horizon-kuttl-tests/horizon-85b965564b-7lzxg" Dec 03 13:45:40 crc kubenswrapper[5052]: I1203 13:45:40.378537 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="horizon-kuttl-tests/horizon-85b965564b-7lzxg" Dec 03 13:45:40 crc kubenswrapper[5052]: I1203 13:45:40.454434 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="horizon-kuttl-tests/horizon-6b4d9768c6-rmfmj" Dec 03 13:45:40 crc kubenswrapper[5052]: I1203 13:45:40.454787 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="horizon-kuttl-tests/horizon-6b4d9768c6-rmfmj" Dec 03 13:45:49 crc kubenswrapper[5052]: I1203 13:45:49.680045 5052 scope.go:117] "RemoveContainer" containerID="2fbd293d6edef0fa059e91c119312e77b262534107c322feb7b59dbea3464526" Dec 03 13:45:49 crc kubenswrapper[5052]: E1203 13:45:49.680812 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 13:45:50 crc kubenswrapper[5052]: I1203 13:45:50.213393 5052 scope.go:117] "RemoveContainer" containerID="f2e21c2232a9ccb381f0e9b579b8dff5ffc898fae2db699e39b929ba65fa5db0" Dec 03 13:45:50 crc kubenswrapper[5052]: I1203 13:45:50.247843 5052 scope.go:117] "RemoveContainer" containerID="3ac66b90ddd94a93b20370b1a1cd343faca7462f8609d6c9ef6bc35c293eebb2" Dec 03 13:45:50 crc kubenswrapper[5052]: I1203 13:45:50.379724 5052 prober.go:107] "Probe failed" probeType="Startup" pod="horizon-kuttl-tests/horizon-85b965564b-7lzxg" podUID="325d2988-e676-4f3f-8e31-8af823f54962" containerName="horizon" probeResult="failure" output="Get \"https://10.217.0.76:8443/dashboard/auth/login/?next=/dashboard/\": dial tcp 10.217.0.76:8443: connect: connection refused" Dec 03 13:45:50 crc kubenswrapper[5052]: I1203 13:45:50.455621 5052 prober.go:107] "Probe failed" probeType="Startup" pod="horizon-kuttl-tests/horizon-6b4d9768c6-rmfmj" podUID="9616ad60-9cfd-45bf-b9f7-19c14dc2e4e1" containerName="horizon" probeResult="failure" output="Get \"https://10.217.0.77:8443/dashboard/auth/login/?next=/dashboard/\": dial tcp 10.217.0.77:8443: connect: connection refused" Dec 03 13:46:01 crc kubenswrapper[5052]: I1203 13:46:01.673592 5052 scope.go:117] "RemoveContainer" containerID="2fbd293d6edef0fa059e91c119312e77b262534107c322feb7b59dbea3464526" Dec 03 13:46:01 crc kubenswrapper[5052]: E1203 13:46:01.675641 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 13:46:02 crc kubenswrapper[5052]: I1203 13:46:02.232312 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="horizon-kuttl-tests/horizon-85b965564b-7lzxg" Dec 03 13:46:02 crc kubenswrapper[5052]: I1203 13:46:02.313441 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="horizon-kuttl-tests/horizon-6b4d9768c6-rmfmj" Dec 03 13:46:03 crc kubenswrapper[5052]: I1203 13:46:03.974110 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="horizon-kuttl-tests/horizon-6b4d9768c6-rmfmj" Dec 03 13:46:04 crc kubenswrapper[5052]: I1203 13:46:04.038976 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["horizon-kuttl-tests/horizon-85b965564b-7lzxg"] Dec 03 13:46:04 crc kubenswrapper[5052]: I1203 13:46:04.039411 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="horizon-kuttl-tests/horizon-85b965564b-7lzxg" podUID="325d2988-e676-4f3f-8e31-8af823f54962" containerName="horizon-log" containerID="cri-o://da3dc23ba250ea46313a9cd9250f558d7ebff28efe8d24ad700efbabfb5812fb" gracePeriod=30 Dec 03 13:46:04 crc kubenswrapper[5052]: I1203 13:46:04.039590 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="horizon-kuttl-tests/horizon-85b965564b-7lzxg" podUID="325d2988-e676-4f3f-8e31-8af823f54962" containerName="horizon" containerID="cri-o://13783643ae78728d4ca2849a82962b25d95f266723e5431696c56dab53a770bc" gracePeriod=30 Dec 03 13:46:04 crc kubenswrapper[5052]: I1203 13:46:04.046733 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="horizon-kuttl-tests/horizon-85b965564b-7lzxg" podUID="325d2988-e676-4f3f-8e31-8af823f54962" containerName="horizon" probeResult="failure" output="Get \"https://10.217.0.76:8443/dashboard/auth/login/?next=/dashboard/\": EOF" Dec 03 13:46:04 crc kubenswrapper[5052]: I1203 13:46:04.316842 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["horizon-kuttl-tests/horizon-6b4d9768c6-rmfmj"] Dec 03 13:46:04 crc kubenswrapper[5052]: I1203 13:46:04.433136 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="horizon-kuttl-tests/horizon-6b4d9768c6-rmfmj" podUID="9616ad60-9cfd-45bf-b9f7-19c14dc2e4e1" containerName="horizon-log" containerID="cri-o://75270f145a47b408181bcd64c713610f2299ffc32f92a623c175305f871066c1" gracePeriod=30 Dec 03 13:46:04 crc kubenswrapper[5052]: I1203 13:46:04.433875 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="horizon-kuttl-tests/horizon-6b4d9768c6-rmfmj" podUID="9616ad60-9cfd-45bf-b9f7-19c14dc2e4e1" containerName="horizon" containerID="cri-o://7fe593c35e2399494e5984380b482861b053fd02e044d4c4257b0f30bcb7694c" gracePeriod=30 Dec 03 13:46:07 crc kubenswrapper[5052]: I1203 13:46:07.426230 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="horizon-kuttl-tests/horizon-85b965564b-7lzxg" podUID="325d2988-e676-4f3f-8e31-8af823f54962" containerName="horizon" probeResult="failure" output="Get \"https://10.217.0.76:8443/dashboard/auth/login/?next=/dashboard/\": read tcp 10.217.0.2:51870->10.217.0.76:8443: read: connection reset by peer" Dec 03 13:46:08 crc kubenswrapper[5052]: I1203 13:46:08.463971 5052 generic.go:334] "Generic (PLEG): container finished" podID="325d2988-e676-4f3f-8e31-8af823f54962" containerID="13783643ae78728d4ca2849a82962b25d95f266723e5431696c56dab53a770bc" exitCode=0 Dec 03 13:46:08 crc kubenswrapper[5052]: I1203 13:46:08.464024 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="horizon-kuttl-tests/horizon-85b965564b-7lzxg" event={"ID":"325d2988-e676-4f3f-8e31-8af823f54962","Type":"ContainerDied","Data":"13783643ae78728d4ca2849a82962b25d95f266723e5431696c56dab53a770bc"} Dec 03 13:46:08 crc kubenswrapper[5052]: I1203 13:46:08.469101 5052 generic.go:334] "Generic (PLEG): container finished" podID="9616ad60-9cfd-45bf-b9f7-19c14dc2e4e1" containerID="7fe593c35e2399494e5984380b482861b053fd02e044d4c4257b0f30bcb7694c" exitCode=0 Dec 03 13:46:08 crc kubenswrapper[5052]: I1203 13:46:08.469141 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="horizon-kuttl-tests/horizon-6b4d9768c6-rmfmj" event={"ID":"9616ad60-9cfd-45bf-b9f7-19c14dc2e4e1","Type":"ContainerDied","Data":"7fe593c35e2399494e5984380b482861b053fd02e044d4c4257b0f30bcb7694c"} Dec 03 13:46:10 crc kubenswrapper[5052]: I1203 13:46:10.378273 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="horizon-kuttl-tests/horizon-85b965564b-7lzxg" podUID="325d2988-e676-4f3f-8e31-8af823f54962" containerName="horizon" probeResult="failure" output="Get \"https://10.217.0.76:8443/dashboard/auth/login/?next=/dashboard/\": dial tcp 10.217.0.76:8443: connect: connection refused" Dec 03 13:46:10 crc kubenswrapper[5052]: I1203 13:46:10.454479 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="horizon-kuttl-tests/horizon-6b4d9768c6-rmfmj" podUID="9616ad60-9cfd-45bf-b9f7-19c14dc2e4e1" containerName="horizon" probeResult="failure" output="Get \"https://10.217.0.77:8443/dashboard/auth/login/?next=/dashboard/\": dial tcp 10.217.0.77:8443: connect: connection refused" Dec 03 13:46:16 crc kubenswrapper[5052]: I1203 13:46:16.673657 5052 scope.go:117] "RemoveContainer" containerID="2fbd293d6edef0fa059e91c119312e77b262534107c322feb7b59dbea3464526" Dec 03 13:46:16 crc kubenswrapper[5052]: E1203 13:46:16.674389 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 13:46:20 crc kubenswrapper[5052]: I1203 13:46:20.378023 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="horizon-kuttl-tests/horizon-85b965564b-7lzxg" podUID="325d2988-e676-4f3f-8e31-8af823f54962" containerName="horizon" probeResult="failure" output="Get \"https://10.217.0.76:8443/dashboard/auth/login/?next=/dashboard/\": dial tcp 10.217.0.76:8443: connect: connection refused" Dec 03 13:46:20 crc kubenswrapper[5052]: I1203 13:46:20.454623 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="horizon-kuttl-tests/horizon-6b4d9768c6-rmfmj" podUID="9616ad60-9cfd-45bf-b9f7-19c14dc2e4e1" containerName="horizon" probeResult="failure" output="Get \"https://10.217.0.77:8443/dashboard/auth/login/?next=/dashboard/\": dial tcp 10.217.0.77:8443: connect: connection refused" Dec 03 13:46:30 crc kubenswrapper[5052]: I1203 13:46:30.378373 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="horizon-kuttl-tests/horizon-85b965564b-7lzxg" podUID="325d2988-e676-4f3f-8e31-8af823f54962" containerName="horizon" probeResult="failure" output="Get \"https://10.217.0.76:8443/dashboard/auth/login/?next=/dashboard/\": dial tcp 10.217.0.76:8443: connect: connection refused" Dec 03 13:46:30 crc kubenswrapper[5052]: I1203 13:46:30.454725 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="horizon-kuttl-tests/horizon-6b4d9768c6-rmfmj" podUID="9616ad60-9cfd-45bf-b9f7-19c14dc2e4e1" containerName="horizon" probeResult="failure" output="Get \"https://10.217.0.77:8443/dashboard/auth/login/?next=/dashboard/\": dial tcp 10.217.0.77:8443: connect: connection refused" Dec 03 13:46:30 crc kubenswrapper[5052]: I1203 13:46:30.454875 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="horizon-kuttl-tests/horizon-6b4d9768c6-rmfmj" Dec 03 13:46:30 crc kubenswrapper[5052]: I1203 13:46:30.672946 5052 scope.go:117] "RemoveContainer" containerID="2fbd293d6edef0fa059e91c119312e77b262534107c322feb7b59dbea3464526" Dec 03 13:46:31 crc kubenswrapper[5052]: I1203 13:46:31.638843 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" event={"ID":"10c237bc-72e7-4da3-9f2c-b1688395bf6e","Type":"ContainerStarted","Data":"af948543b565b59ee5a764bc4f80d394be230d8c49adc6125d06e305c9dffa71"} Dec 03 13:46:34 crc kubenswrapper[5052]: I1203 13:46:34.433966 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="horizon-kuttl-tests/horizon-85b965564b-7lzxg" Dec 03 13:46:34 crc kubenswrapper[5052]: I1203 13:46:34.526373 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/325d2988-e676-4f3f-8e31-8af823f54962-logs\") pod \"325d2988-e676-4f3f-8e31-8af823f54962\" (UID: \"325d2988-e676-4f3f-8e31-8af823f54962\") " Dec 03 13:46:34 crc kubenswrapper[5052]: I1203 13:46:34.526433 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"horizon-secret-key\" (UniqueName: \"kubernetes.io/secret/325d2988-e676-4f3f-8e31-8af823f54962-horizon-secret-key\") pod \"325d2988-e676-4f3f-8e31-8af823f54962\" (UID: \"325d2988-e676-4f3f-8e31-8af823f54962\") " Dec 03 13:46:34 crc kubenswrapper[5052]: I1203 13:46:34.526546 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/325d2988-e676-4f3f-8e31-8af823f54962-config-data\") pod \"325d2988-e676-4f3f-8e31-8af823f54962\" (UID: \"325d2988-e676-4f3f-8e31-8af823f54962\") " Dec 03 13:46:34 crc kubenswrapper[5052]: I1203 13:46:34.526590 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-88vpt\" (UniqueName: \"kubernetes.io/projected/325d2988-e676-4f3f-8e31-8af823f54962-kube-api-access-88vpt\") pod \"325d2988-e676-4f3f-8e31-8af823f54962\" (UID: \"325d2988-e676-4f3f-8e31-8af823f54962\") " Dec 03 13:46:34 crc kubenswrapper[5052]: I1203 13:46:34.526616 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"horizon-tls-certs\" (UniqueName: \"kubernetes.io/secret/325d2988-e676-4f3f-8e31-8af823f54962-horizon-tls-certs\") pod \"325d2988-e676-4f3f-8e31-8af823f54962\" (UID: \"325d2988-e676-4f3f-8e31-8af823f54962\") " Dec 03 13:46:34 crc kubenswrapper[5052]: I1203 13:46:34.526657 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/325d2988-e676-4f3f-8e31-8af823f54962-scripts\") pod \"325d2988-e676-4f3f-8e31-8af823f54962\" (UID: \"325d2988-e676-4f3f-8e31-8af823f54962\") " Dec 03 13:46:34 crc kubenswrapper[5052]: I1203 13:46:34.526685 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/325d2988-e676-4f3f-8e31-8af823f54962-combined-ca-bundle\") pod \"325d2988-e676-4f3f-8e31-8af823f54962\" (UID: \"325d2988-e676-4f3f-8e31-8af823f54962\") " Dec 03 13:46:34 crc kubenswrapper[5052]: I1203 13:46:34.526986 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/325d2988-e676-4f3f-8e31-8af823f54962-logs" (OuterVolumeSpecName: "logs") pod "325d2988-e676-4f3f-8e31-8af823f54962" (UID: "325d2988-e676-4f3f-8e31-8af823f54962"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 13:46:34 crc kubenswrapper[5052]: I1203 13:46:34.540877 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/325d2988-e676-4f3f-8e31-8af823f54962-horizon-secret-key" (OuterVolumeSpecName: "horizon-secret-key") pod "325d2988-e676-4f3f-8e31-8af823f54962" (UID: "325d2988-e676-4f3f-8e31-8af823f54962"). InnerVolumeSpecName "horizon-secret-key". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:46:34 crc kubenswrapper[5052]: I1203 13:46:34.540945 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/325d2988-e676-4f3f-8e31-8af823f54962-kube-api-access-88vpt" (OuterVolumeSpecName: "kube-api-access-88vpt") pod "325d2988-e676-4f3f-8e31-8af823f54962" (UID: "325d2988-e676-4f3f-8e31-8af823f54962"). InnerVolumeSpecName "kube-api-access-88vpt". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:46:34 crc kubenswrapper[5052]: I1203 13:46:34.552962 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/325d2988-e676-4f3f-8e31-8af823f54962-scripts" (OuterVolumeSpecName: "scripts") pod "325d2988-e676-4f3f-8e31-8af823f54962" (UID: "325d2988-e676-4f3f-8e31-8af823f54962"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:46:34 crc kubenswrapper[5052]: I1203 13:46:34.552999 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/325d2988-e676-4f3f-8e31-8af823f54962-config-data" (OuterVolumeSpecName: "config-data") pod "325d2988-e676-4f3f-8e31-8af823f54962" (UID: "325d2988-e676-4f3f-8e31-8af823f54962"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:46:34 crc kubenswrapper[5052]: I1203 13:46:34.558704 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/325d2988-e676-4f3f-8e31-8af823f54962-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "325d2988-e676-4f3f-8e31-8af823f54962" (UID: "325d2988-e676-4f3f-8e31-8af823f54962"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:46:34 crc kubenswrapper[5052]: I1203 13:46:34.571240 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/325d2988-e676-4f3f-8e31-8af823f54962-horizon-tls-certs" (OuterVolumeSpecName: "horizon-tls-certs") pod "325d2988-e676-4f3f-8e31-8af823f54962" (UID: "325d2988-e676-4f3f-8e31-8af823f54962"). InnerVolumeSpecName "horizon-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:46:34 crc kubenswrapper[5052]: I1203 13:46:34.627801 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/325d2988-e676-4f3f-8e31-8af823f54962-logs\") on node \"crc\" DevicePath \"\"" Dec 03 13:46:34 crc kubenswrapper[5052]: I1203 13:46:34.628124 5052 reconciler_common.go:293] "Volume detached for volume \"horizon-secret-key\" (UniqueName: \"kubernetes.io/secret/325d2988-e676-4f3f-8e31-8af823f54962-horizon-secret-key\") on node \"crc\" DevicePath \"\"" Dec 03 13:46:34 crc kubenswrapper[5052]: I1203 13:46:34.628213 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/325d2988-e676-4f3f-8e31-8af823f54962-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 13:46:34 crc kubenswrapper[5052]: I1203 13:46:34.628282 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-88vpt\" (UniqueName: \"kubernetes.io/projected/325d2988-e676-4f3f-8e31-8af823f54962-kube-api-access-88vpt\") on node \"crc\" DevicePath \"\"" Dec 03 13:46:34 crc kubenswrapper[5052]: I1203 13:46:34.628349 5052 reconciler_common.go:293] "Volume detached for volume \"horizon-tls-certs\" (UniqueName: \"kubernetes.io/secret/325d2988-e676-4f3f-8e31-8af823f54962-horizon-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 13:46:34 crc kubenswrapper[5052]: I1203 13:46:34.628404 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/325d2988-e676-4f3f-8e31-8af823f54962-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 13:46:34 crc kubenswrapper[5052]: I1203 13:46:34.628464 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/325d2988-e676-4f3f-8e31-8af823f54962-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 13:46:34 crc kubenswrapper[5052]: I1203 13:46:34.662809 5052 generic.go:334] "Generic (PLEG): container finished" podID="325d2988-e676-4f3f-8e31-8af823f54962" containerID="da3dc23ba250ea46313a9cd9250f558d7ebff28efe8d24ad700efbabfb5812fb" exitCode=137 Dec 03 13:46:34 crc kubenswrapper[5052]: I1203 13:46:34.662906 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="horizon-kuttl-tests/horizon-85b965564b-7lzxg" event={"ID":"325d2988-e676-4f3f-8e31-8af823f54962","Type":"ContainerDied","Data":"da3dc23ba250ea46313a9cd9250f558d7ebff28efe8d24ad700efbabfb5812fb"} Dec 03 13:46:34 crc kubenswrapper[5052]: I1203 13:46:34.662961 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="horizon-kuttl-tests/horizon-85b965564b-7lzxg" event={"ID":"325d2988-e676-4f3f-8e31-8af823f54962","Type":"ContainerDied","Data":"01178513b4b05251eb08dcff783c20cefe3c41a3e58b0a21fae3353a85455731"} Dec 03 13:46:34 crc kubenswrapper[5052]: I1203 13:46:34.662982 5052 scope.go:117] "RemoveContainer" containerID="13783643ae78728d4ca2849a82962b25d95f266723e5431696c56dab53a770bc" Dec 03 13:46:34 crc kubenswrapper[5052]: I1203 13:46:34.663209 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="horizon-kuttl-tests/horizon-85b965564b-7lzxg" Dec 03 13:46:34 crc kubenswrapper[5052]: I1203 13:46:34.673438 5052 generic.go:334] "Generic (PLEG): container finished" podID="9616ad60-9cfd-45bf-b9f7-19c14dc2e4e1" containerID="75270f145a47b408181bcd64c713610f2299ffc32f92a623c175305f871066c1" exitCode=137 Dec 03 13:46:34 crc kubenswrapper[5052]: I1203 13:46:34.673502 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="horizon-kuttl-tests/horizon-6b4d9768c6-rmfmj" event={"ID":"9616ad60-9cfd-45bf-b9f7-19c14dc2e4e1","Type":"ContainerDied","Data":"75270f145a47b408181bcd64c713610f2299ffc32f92a623c175305f871066c1"} Dec 03 13:46:34 crc kubenswrapper[5052]: I1203 13:46:34.708151 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["horizon-kuttl-tests/horizon-85b965564b-7lzxg"] Dec 03 13:46:34 crc kubenswrapper[5052]: I1203 13:46:34.717605 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["horizon-kuttl-tests/horizon-85b965564b-7lzxg"] Dec 03 13:46:34 crc kubenswrapper[5052]: I1203 13:46:34.859998 5052 scope.go:117] "RemoveContainer" containerID="da3dc23ba250ea46313a9cd9250f558d7ebff28efe8d24ad700efbabfb5812fb" Dec 03 13:46:34 crc kubenswrapper[5052]: I1203 13:46:34.874692 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="horizon-kuttl-tests/horizon-6b4d9768c6-rmfmj" Dec 03 13:46:34 crc kubenswrapper[5052]: I1203 13:46:34.879692 5052 scope.go:117] "RemoveContainer" containerID="13783643ae78728d4ca2849a82962b25d95f266723e5431696c56dab53a770bc" Dec 03 13:46:34 crc kubenswrapper[5052]: E1203 13:46:34.880057 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"13783643ae78728d4ca2849a82962b25d95f266723e5431696c56dab53a770bc\": container with ID starting with 13783643ae78728d4ca2849a82962b25d95f266723e5431696c56dab53a770bc not found: ID does not exist" containerID="13783643ae78728d4ca2849a82962b25d95f266723e5431696c56dab53a770bc" Dec 03 13:46:34 crc kubenswrapper[5052]: I1203 13:46:34.880094 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"13783643ae78728d4ca2849a82962b25d95f266723e5431696c56dab53a770bc"} err="failed to get container status \"13783643ae78728d4ca2849a82962b25d95f266723e5431696c56dab53a770bc\": rpc error: code = NotFound desc = could not find container \"13783643ae78728d4ca2849a82962b25d95f266723e5431696c56dab53a770bc\": container with ID starting with 13783643ae78728d4ca2849a82962b25d95f266723e5431696c56dab53a770bc not found: ID does not exist" Dec 03 13:46:34 crc kubenswrapper[5052]: I1203 13:46:34.880118 5052 scope.go:117] "RemoveContainer" containerID="da3dc23ba250ea46313a9cd9250f558d7ebff28efe8d24ad700efbabfb5812fb" Dec 03 13:46:34 crc kubenswrapper[5052]: E1203 13:46:34.880416 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"da3dc23ba250ea46313a9cd9250f558d7ebff28efe8d24ad700efbabfb5812fb\": container with ID starting with da3dc23ba250ea46313a9cd9250f558d7ebff28efe8d24ad700efbabfb5812fb not found: ID does not exist" containerID="da3dc23ba250ea46313a9cd9250f558d7ebff28efe8d24ad700efbabfb5812fb" Dec 03 13:46:34 crc kubenswrapper[5052]: I1203 13:46:34.880434 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"da3dc23ba250ea46313a9cd9250f558d7ebff28efe8d24ad700efbabfb5812fb"} err="failed to get container status \"da3dc23ba250ea46313a9cd9250f558d7ebff28efe8d24ad700efbabfb5812fb\": rpc error: code = NotFound desc = could not find container \"da3dc23ba250ea46313a9cd9250f558d7ebff28efe8d24ad700efbabfb5812fb\": container with ID starting with da3dc23ba250ea46313a9cd9250f558d7ebff28efe8d24ad700efbabfb5812fb not found: ID does not exist" Dec 03 13:46:34 crc kubenswrapper[5052]: I1203 13:46:34.932744 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/9616ad60-9cfd-45bf-b9f7-19c14dc2e4e1-logs\") pod \"9616ad60-9cfd-45bf-b9f7-19c14dc2e4e1\" (UID: \"9616ad60-9cfd-45bf-b9f7-19c14dc2e4e1\") " Dec 03 13:46:34 crc kubenswrapper[5052]: I1203 13:46:34.932788 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"horizon-secret-key\" (UniqueName: \"kubernetes.io/secret/9616ad60-9cfd-45bf-b9f7-19c14dc2e4e1-horizon-secret-key\") pod \"9616ad60-9cfd-45bf-b9f7-19c14dc2e4e1\" (UID: \"9616ad60-9cfd-45bf-b9f7-19c14dc2e4e1\") " Dec 03 13:46:34 crc kubenswrapper[5052]: I1203 13:46:34.932887 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/9616ad60-9cfd-45bf-b9f7-19c14dc2e4e1-config-data\") pod \"9616ad60-9cfd-45bf-b9f7-19c14dc2e4e1\" (UID: \"9616ad60-9cfd-45bf-b9f7-19c14dc2e4e1\") " Dec 03 13:46:34 crc kubenswrapper[5052]: I1203 13:46:34.932907 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/9616ad60-9cfd-45bf-b9f7-19c14dc2e4e1-scripts\") pod \"9616ad60-9cfd-45bf-b9f7-19c14dc2e4e1\" (UID: \"9616ad60-9cfd-45bf-b9f7-19c14dc2e4e1\") " Dec 03 13:46:34 crc kubenswrapper[5052]: I1203 13:46:34.932967 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-llps6\" (UniqueName: \"kubernetes.io/projected/9616ad60-9cfd-45bf-b9f7-19c14dc2e4e1-kube-api-access-llps6\") pod \"9616ad60-9cfd-45bf-b9f7-19c14dc2e4e1\" (UID: \"9616ad60-9cfd-45bf-b9f7-19c14dc2e4e1\") " Dec 03 13:46:34 crc kubenswrapper[5052]: I1203 13:46:34.933016 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"horizon-tls-certs\" (UniqueName: \"kubernetes.io/secret/9616ad60-9cfd-45bf-b9f7-19c14dc2e4e1-horizon-tls-certs\") pod \"9616ad60-9cfd-45bf-b9f7-19c14dc2e4e1\" (UID: \"9616ad60-9cfd-45bf-b9f7-19c14dc2e4e1\") " Dec 03 13:46:34 crc kubenswrapper[5052]: I1203 13:46:34.933052 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9616ad60-9cfd-45bf-b9f7-19c14dc2e4e1-combined-ca-bundle\") pod \"9616ad60-9cfd-45bf-b9f7-19c14dc2e4e1\" (UID: \"9616ad60-9cfd-45bf-b9f7-19c14dc2e4e1\") " Dec 03 13:46:34 crc kubenswrapper[5052]: I1203 13:46:34.933862 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9616ad60-9cfd-45bf-b9f7-19c14dc2e4e1-logs" (OuterVolumeSpecName: "logs") pod "9616ad60-9cfd-45bf-b9f7-19c14dc2e4e1" (UID: "9616ad60-9cfd-45bf-b9f7-19c14dc2e4e1"). InnerVolumeSpecName "logs". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 13:46:34 crc kubenswrapper[5052]: I1203 13:46:34.934142 5052 reconciler_common.go:293] "Volume detached for volume \"logs\" (UniqueName: \"kubernetes.io/empty-dir/9616ad60-9cfd-45bf-b9f7-19c14dc2e4e1-logs\") on node \"crc\" DevicePath \"\"" Dec 03 13:46:34 crc kubenswrapper[5052]: I1203 13:46:34.936801 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9616ad60-9cfd-45bf-b9f7-19c14dc2e4e1-kube-api-access-llps6" (OuterVolumeSpecName: "kube-api-access-llps6") pod "9616ad60-9cfd-45bf-b9f7-19c14dc2e4e1" (UID: "9616ad60-9cfd-45bf-b9f7-19c14dc2e4e1"). InnerVolumeSpecName "kube-api-access-llps6". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:46:34 crc kubenswrapper[5052]: I1203 13:46:34.937087 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9616ad60-9cfd-45bf-b9f7-19c14dc2e4e1-horizon-secret-key" (OuterVolumeSpecName: "horizon-secret-key") pod "9616ad60-9cfd-45bf-b9f7-19c14dc2e4e1" (UID: "9616ad60-9cfd-45bf-b9f7-19c14dc2e4e1"). InnerVolumeSpecName "horizon-secret-key". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:46:34 crc kubenswrapper[5052]: I1203 13:46:34.950071 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9616ad60-9cfd-45bf-b9f7-19c14dc2e4e1-scripts" (OuterVolumeSpecName: "scripts") pod "9616ad60-9cfd-45bf-b9f7-19c14dc2e4e1" (UID: "9616ad60-9cfd-45bf-b9f7-19c14dc2e4e1"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:46:34 crc kubenswrapper[5052]: I1203 13:46:34.951449 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9616ad60-9cfd-45bf-b9f7-19c14dc2e4e1-combined-ca-bundle" (OuterVolumeSpecName: "combined-ca-bundle") pod "9616ad60-9cfd-45bf-b9f7-19c14dc2e4e1" (UID: "9616ad60-9cfd-45bf-b9f7-19c14dc2e4e1"). InnerVolumeSpecName "combined-ca-bundle". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:46:34 crc kubenswrapper[5052]: I1203 13:46:34.951522 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9616ad60-9cfd-45bf-b9f7-19c14dc2e4e1-config-data" (OuterVolumeSpecName: "config-data") pod "9616ad60-9cfd-45bf-b9f7-19c14dc2e4e1" (UID: "9616ad60-9cfd-45bf-b9f7-19c14dc2e4e1"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:46:34 crc kubenswrapper[5052]: I1203 13:46:34.967983 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9616ad60-9cfd-45bf-b9f7-19c14dc2e4e1-horizon-tls-certs" (OuterVolumeSpecName: "horizon-tls-certs") pod "9616ad60-9cfd-45bf-b9f7-19c14dc2e4e1" (UID: "9616ad60-9cfd-45bf-b9f7-19c14dc2e4e1"). InnerVolumeSpecName "horizon-tls-certs". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:46:35 crc kubenswrapper[5052]: I1203 13:46:35.035612 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-llps6\" (UniqueName: \"kubernetes.io/projected/9616ad60-9cfd-45bf-b9f7-19c14dc2e4e1-kube-api-access-llps6\") on node \"crc\" DevicePath \"\"" Dec 03 13:46:35 crc kubenswrapper[5052]: I1203 13:46:35.035651 5052 reconciler_common.go:293] "Volume detached for volume \"horizon-tls-certs\" (UniqueName: \"kubernetes.io/secret/9616ad60-9cfd-45bf-b9f7-19c14dc2e4e1-horizon-tls-certs\") on node \"crc\" DevicePath \"\"" Dec 03 13:46:35 crc kubenswrapper[5052]: I1203 13:46:35.035662 5052 reconciler_common.go:293] "Volume detached for volume \"combined-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9616ad60-9cfd-45bf-b9f7-19c14dc2e4e1-combined-ca-bundle\") on node \"crc\" DevicePath \"\"" Dec 03 13:46:35 crc kubenswrapper[5052]: I1203 13:46:35.035671 5052 reconciler_common.go:293] "Volume detached for volume \"horizon-secret-key\" (UniqueName: \"kubernetes.io/secret/9616ad60-9cfd-45bf-b9f7-19c14dc2e4e1-horizon-secret-key\") on node \"crc\" DevicePath \"\"" Dec 03 13:46:35 crc kubenswrapper[5052]: I1203 13:46:35.035681 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/9616ad60-9cfd-45bf-b9f7-19c14dc2e4e1-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 13:46:35 crc kubenswrapper[5052]: I1203 13:46:35.035689 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/configmap/9616ad60-9cfd-45bf-b9f7-19c14dc2e4e1-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 13:46:35 crc kubenswrapper[5052]: I1203 13:46:35.682762 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="horizon-kuttl-tests/horizon-6b4d9768c6-rmfmj" Dec 03 13:46:35 crc kubenswrapper[5052]: I1203 13:46:35.694262 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="325d2988-e676-4f3f-8e31-8af823f54962" path="/var/lib/kubelet/pods/325d2988-e676-4f3f-8e31-8af823f54962/volumes" Dec 03 13:46:35 crc kubenswrapper[5052]: I1203 13:46:35.695298 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="horizon-kuttl-tests/horizon-6b4d9768c6-rmfmj" event={"ID":"9616ad60-9cfd-45bf-b9f7-19c14dc2e4e1","Type":"ContainerDied","Data":"9bffb1566d98b6101a6427a3960df494c186d5728b1920df052d72a96bce2a07"} Dec 03 13:46:35 crc kubenswrapper[5052]: I1203 13:46:35.695346 5052 scope.go:117] "RemoveContainer" containerID="7fe593c35e2399494e5984380b482861b053fd02e044d4c4257b0f30bcb7694c" Dec 03 13:46:35 crc kubenswrapper[5052]: I1203 13:46:35.739276 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["horizon-kuttl-tests/horizon-6b4d9768c6-rmfmj"] Dec 03 13:46:35 crc kubenswrapper[5052]: I1203 13:46:35.746801 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["horizon-kuttl-tests/horizon-6b4d9768c6-rmfmj"] Dec 03 13:46:35 crc kubenswrapper[5052]: I1203 13:46:35.879398 5052 scope.go:117] "RemoveContainer" containerID="75270f145a47b408181bcd64c713610f2299ffc32f92a623c175305f871066c1" Dec 03 13:46:37 crc kubenswrapper[5052]: I1203 13:46:37.682472 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9616ad60-9cfd-45bf-b9f7-19c14dc2e4e1" path="/var/lib/kubelet/pods/9616ad60-9cfd-45bf-b9f7-19c14dc2e4e1/volumes" Dec 03 13:46:44 crc kubenswrapper[5052]: I1203 13:46:44.736350 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["horizon-kuttl-tests/keystone-bootstrap-k99zw"] Dec 03 13:46:44 crc kubenswrapper[5052]: I1203 13:46:44.755397 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["horizon-kuttl-tests/keystone-bootstrap-k99zw"] Dec 03 13:46:44 crc kubenswrapper[5052]: I1203 13:46:44.783437 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["horizon-kuttl-tests/keystone-db-sync-2qj57"] Dec 03 13:46:44 crc kubenswrapper[5052]: I1203 13:46:44.793610 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["horizon-kuttl-tests/keystone-db-sync-2qj57"] Dec 03 13:46:44 crc kubenswrapper[5052]: I1203 13:46:44.806832 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["horizon-kuttl-tests/keystone-68bf6dbf4f-g4nbm"] Dec 03 13:46:44 crc kubenswrapper[5052]: I1203 13:46:44.807028 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="horizon-kuttl-tests/keystone-68bf6dbf4f-g4nbm" podUID="d8df61e7-dad4-4072-a268-ddc6b55f0696" containerName="keystone-api" containerID="cri-o://e698f2ff7e2689a1f1a877cb6db7518a6479f4b8d5bedb508f36e4f0ed3c80c0" gracePeriod=30 Dec 03 13:46:44 crc kubenswrapper[5052]: I1203 13:46:44.819672 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["horizon-kuttl-tests/keystonef429-account-delete-m9szg"] Dec 03 13:46:44 crc kubenswrapper[5052]: E1203 13:46:44.820044 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="9616ad60-9cfd-45bf-b9f7-19c14dc2e4e1" containerName="horizon" Dec 03 13:46:44 crc kubenswrapper[5052]: I1203 13:46:44.820065 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="9616ad60-9cfd-45bf-b9f7-19c14dc2e4e1" containerName="horizon" Dec 03 13:46:44 crc kubenswrapper[5052]: E1203 13:46:44.820087 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="9616ad60-9cfd-45bf-b9f7-19c14dc2e4e1" containerName="horizon-log" Dec 03 13:46:44 crc kubenswrapper[5052]: I1203 13:46:44.820096 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="9616ad60-9cfd-45bf-b9f7-19c14dc2e4e1" containerName="horizon-log" Dec 03 13:46:44 crc kubenswrapper[5052]: E1203 13:46:44.820113 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="325d2988-e676-4f3f-8e31-8af823f54962" containerName="horizon" Dec 03 13:46:44 crc kubenswrapper[5052]: I1203 13:46:44.820122 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="325d2988-e676-4f3f-8e31-8af823f54962" containerName="horizon" Dec 03 13:46:44 crc kubenswrapper[5052]: E1203 13:46:44.820158 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="325d2988-e676-4f3f-8e31-8af823f54962" containerName="horizon-log" Dec 03 13:46:44 crc kubenswrapper[5052]: I1203 13:46:44.820167 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="325d2988-e676-4f3f-8e31-8af823f54962" containerName="horizon-log" Dec 03 13:46:44 crc kubenswrapper[5052]: I1203 13:46:44.820327 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="325d2988-e676-4f3f-8e31-8af823f54962" containerName="horizon" Dec 03 13:46:44 crc kubenswrapper[5052]: I1203 13:46:44.820348 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="9616ad60-9cfd-45bf-b9f7-19c14dc2e4e1" containerName="horizon" Dec 03 13:46:44 crc kubenswrapper[5052]: I1203 13:46:44.820375 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="9616ad60-9cfd-45bf-b9f7-19c14dc2e4e1" containerName="horizon-log" Dec 03 13:46:44 crc kubenswrapper[5052]: I1203 13:46:44.820389 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="325d2988-e676-4f3f-8e31-8af823f54962" containerName="horizon-log" Dec 03 13:46:44 crc kubenswrapper[5052]: I1203 13:46:44.820998 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="horizon-kuttl-tests/keystonef429-account-delete-m9szg" Dec 03 13:46:44 crc kubenswrapper[5052]: I1203 13:46:44.848850 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["horizon-kuttl-tests/keystonef429-account-delete-m9szg"] Dec 03 13:46:44 crc kubenswrapper[5052]: I1203 13:46:44.892089 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/5179e964-3df1-4d07-b440-a01ba074ac05-operator-scripts\") pod \"keystonef429-account-delete-m9szg\" (UID: \"5179e964-3df1-4d07-b440-a01ba074ac05\") " pod="horizon-kuttl-tests/keystonef429-account-delete-m9szg" Dec 03 13:46:44 crc kubenswrapper[5052]: I1203 13:46:44.892151 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5xq28\" (UniqueName: \"kubernetes.io/projected/5179e964-3df1-4d07-b440-a01ba074ac05-kube-api-access-5xq28\") pod \"keystonef429-account-delete-m9szg\" (UID: \"5179e964-3df1-4d07-b440-a01ba074ac05\") " pod="horizon-kuttl-tests/keystonef429-account-delete-m9szg" Dec 03 13:46:44 crc kubenswrapper[5052]: I1203 13:46:44.993403 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/5179e964-3df1-4d07-b440-a01ba074ac05-operator-scripts\") pod \"keystonef429-account-delete-m9szg\" (UID: \"5179e964-3df1-4d07-b440-a01ba074ac05\") " pod="horizon-kuttl-tests/keystonef429-account-delete-m9szg" Dec 03 13:46:44 crc kubenswrapper[5052]: I1203 13:46:44.993764 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-5xq28\" (UniqueName: \"kubernetes.io/projected/5179e964-3df1-4d07-b440-a01ba074ac05-kube-api-access-5xq28\") pod \"keystonef429-account-delete-m9szg\" (UID: \"5179e964-3df1-4d07-b440-a01ba074ac05\") " pod="horizon-kuttl-tests/keystonef429-account-delete-m9szg" Dec 03 13:46:44 crc kubenswrapper[5052]: I1203 13:46:44.994848 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/5179e964-3df1-4d07-b440-a01ba074ac05-operator-scripts\") pod \"keystonef429-account-delete-m9szg\" (UID: \"5179e964-3df1-4d07-b440-a01ba074ac05\") " pod="horizon-kuttl-tests/keystonef429-account-delete-m9szg" Dec 03 13:46:45 crc kubenswrapper[5052]: I1203 13:46:45.014092 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-5xq28\" (UniqueName: \"kubernetes.io/projected/5179e964-3df1-4d07-b440-a01ba074ac05-kube-api-access-5xq28\") pod \"keystonef429-account-delete-m9szg\" (UID: \"5179e964-3df1-4d07-b440-a01ba074ac05\") " pod="horizon-kuttl-tests/keystonef429-account-delete-m9szg" Dec 03 13:46:45 crc kubenswrapper[5052]: I1203 13:46:45.143722 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="horizon-kuttl-tests/keystonef429-account-delete-m9szg" Dec 03 13:46:45 crc kubenswrapper[5052]: I1203 13:46:45.635784 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["horizon-kuttl-tests/keystonef429-account-delete-m9szg"] Dec 03 13:46:45 crc kubenswrapper[5052]: I1203 13:46:45.681489 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3cefc769-6ac1-4466-bcb4-bc1928dfce9e" path="/var/lib/kubelet/pods/3cefc769-6ac1-4466-bcb4-bc1928dfce9e/volumes" Dec 03 13:46:45 crc kubenswrapper[5052]: I1203 13:46:45.682184 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eb70247b-750b-4da4-9f00-f9e6085163da" path="/var/lib/kubelet/pods/eb70247b-750b-4da4-9f00-f9e6085163da/volumes" Dec 03 13:46:45 crc kubenswrapper[5052]: I1203 13:46:45.763068 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="horizon-kuttl-tests/keystonef429-account-delete-m9szg" event={"ID":"5179e964-3df1-4d07-b440-a01ba074ac05","Type":"ContainerStarted","Data":"9c7ab2751625b70bbc10179bc935d5e2089069140ad91315588ad4ee3ecdb34e"} Dec 03 13:46:46 crc kubenswrapper[5052]: I1203 13:46:46.772182 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="horizon-kuttl-tests/keystonef429-account-delete-m9szg" event={"ID":"5179e964-3df1-4d07-b440-a01ba074ac05","Type":"ContainerStarted","Data":"06eb853e582954e757cdbf2f774b8864440ed80f9893b4fdb23abe72ed90929f"} Dec 03 13:46:46 crc kubenswrapper[5052]: I1203 13:46:46.788887 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="horizon-kuttl-tests/keystonef429-account-delete-m9szg" podStartSLOduration=2.788867576 podStartE2EDuration="2.788867576s" podCreationTimestamp="2025-12-03 13:46:44 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 13:46:46.783687588 +0000 UTC m=+8797.371929352" watchObservedRunningTime="2025-12-03 13:46:46.788867576 +0000 UTC m=+8797.377109340" Dec 03 13:46:48 crc kubenswrapper[5052]: I1203 13:46:48.093021 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["horizon-kuttl-tests/openstack-galera-1"] Dec 03 13:46:48 crc kubenswrapper[5052]: I1203 13:46:48.109282 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["horizon-kuttl-tests/openstack-galera-0"] Dec 03 13:46:48 crc kubenswrapper[5052]: I1203 13:46:48.119333 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["horizon-kuttl-tests/openstack-galera-2"] Dec 03 13:46:48 crc kubenswrapper[5052]: I1203 13:46:48.264988 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="horizon-kuttl-tests/openstack-galera-2" podUID="a2db0de0-8930-4d21-ad37-58889377d33b" containerName="galera" containerID="cri-o://fbc326fc54a520afef74ee80957e5d836345c10d4d03b2e32cd807930864490e" gracePeriod=30 Dec 03 13:46:48 crc kubenswrapper[5052]: I1203 13:46:48.266337 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="horizon-kuttl-tests/keystone-68bf6dbf4f-g4nbm" Dec 03 13:46:48 crc kubenswrapper[5052]: I1203 13:46:48.461379 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/d8df61e7-dad4-4072-a268-ddc6b55f0696-fernet-keys\") pod \"d8df61e7-dad4-4072-a268-ddc6b55f0696\" (UID: \"d8df61e7-dad4-4072-a268-ddc6b55f0696\") " Dec 03 13:46:48 crc kubenswrapper[5052]: I1203 13:46:48.461444 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/d8df61e7-dad4-4072-a268-ddc6b55f0696-scripts\") pod \"d8df61e7-dad4-4072-a268-ddc6b55f0696\" (UID: \"d8df61e7-dad4-4072-a268-ddc6b55f0696\") " Dec 03 13:46:48 crc kubenswrapper[5052]: I1203 13:46:48.461485 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/d8df61e7-dad4-4072-a268-ddc6b55f0696-credential-keys\") pod \"d8df61e7-dad4-4072-a268-ddc6b55f0696\" (UID: \"d8df61e7-dad4-4072-a268-ddc6b55f0696\") " Dec 03 13:46:48 crc kubenswrapper[5052]: I1203 13:46:48.461548 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z46st\" (UniqueName: \"kubernetes.io/projected/d8df61e7-dad4-4072-a268-ddc6b55f0696-kube-api-access-z46st\") pod \"d8df61e7-dad4-4072-a268-ddc6b55f0696\" (UID: \"d8df61e7-dad4-4072-a268-ddc6b55f0696\") " Dec 03 13:46:48 crc kubenswrapper[5052]: I1203 13:46:48.461594 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/d8df61e7-dad4-4072-a268-ddc6b55f0696-config-data\") pod \"d8df61e7-dad4-4072-a268-ddc6b55f0696\" (UID: \"d8df61e7-dad4-4072-a268-ddc6b55f0696\") " Dec 03 13:46:48 crc kubenswrapper[5052]: I1203 13:46:48.472440 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d8df61e7-dad4-4072-a268-ddc6b55f0696-fernet-keys" (OuterVolumeSpecName: "fernet-keys") pod "d8df61e7-dad4-4072-a268-ddc6b55f0696" (UID: "d8df61e7-dad4-4072-a268-ddc6b55f0696"). InnerVolumeSpecName "fernet-keys". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:46:48 crc kubenswrapper[5052]: I1203 13:46:48.472468 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d8df61e7-dad4-4072-a268-ddc6b55f0696-kube-api-access-z46st" (OuterVolumeSpecName: "kube-api-access-z46st") pod "d8df61e7-dad4-4072-a268-ddc6b55f0696" (UID: "d8df61e7-dad4-4072-a268-ddc6b55f0696"). InnerVolumeSpecName "kube-api-access-z46st". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:46:48 crc kubenswrapper[5052]: I1203 13:46:48.472663 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d8df61e7-dad4-4072-a268-ddc6b55f0696-credential-keys" (OuterVolumeSpecName: "credential-keys") pod "d8df61e7-dad4-4072-a268-ddc6b55f0696" (UID: "d8df61e7-dad4-4072-a268-ddc6b55f0696"). InnerVolumeSpecName "credential-keys". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:46:48 crc kubenswrapper[5052]: I1203 13:46:48.472734 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d8df61e7-dad4-4072-a268-ddc6b55f0696-scripts" (OuterVolumeSpecName: "scripts") pod "d8df61e7-dad4-4072-a268-ddc6b55f0696" (UID: "d8df61e7-dad4-4072-a268-ddc6b55f0696"). InnerVolumeSpecName "scripts". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:46:48 crc kubenswrapper[5052]: I1203 13:46:48.494692 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d8df61e7-dad4-4072-a268-ddc6b55f0696-config-data" (OuterVolumeSpecName: "config-data") pod "d8df61e7-dad4-4072-a268-ddc6b55f0696" (UID: "d8df61e7-dad4-4072-a268-ddc6b55f0696"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:46:48 crc kubenswrapper[5052]: I1203 13:46:48.563145 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/secret/d8df61e7-dad4-4072-a268-ddc6b55f0696-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 13:46:48 crc kubenswrapper[5052]: I1203 13:46:48.563186 5052 reconciler_common.go:293] "Volume detached for volume \"fernet-keys\" (UniqueName: \"kubernetes.io/secret/d8df61e7-dad4-4072-a268-ddc6b55f0696-fernet-keys\") on node \"crc\" DevicePath \"\"" Dec 03 13:46:48 crc kubenswrapper[5052]: I1203 13:46:48.563197 5052 reconciler_common.go:293] "Volume detached for volume \"scripts\" (UniqueName: \"kubernetes.io/secret/d8df61e7-dad4-4072-a268-ddc6b55f0696-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 13:46:48 crc kubenswrapper[5052]: I1203 13:46:48.563207 5052 reconciler_common.go:293] "Volume detached for volume \"credential-keys\" (UniqueName: \"kubernetes.io/secret/d8df61e7-dad4-4072-a268-ddc6b55f0696-credential-keys\") on node \"crc\" DevicePath \"\"" Dec 03 13:46:48 crc kubenswrapper[5052]: I1203 13:46:48.563221 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-z46st\" (UniqueName: \"kubernetes.io/projected/d8df61e7-dad4-4072-a268-ddc6b55f0696-kube-api-access-z46st\") on node \"crc\" DevicePath \"\"" Dec 03 13:46:48 crc kubenswrapper[5052]: I1203 13:46:48.790572 5052 generic.go:334] "Generic (PLEG): container finished" podID="5179e964-3df1-4d07-b440-a01ba074ac05" containerID="06eb853e582954e757cdbf2f774b8864440ed80f9893b4fdb23abe72ed90929f" exitCode=0 Dec 03 13:46:48 crc kubenswrapper[5052]: I1203 13:46:48.790639 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="horizon-kuttl-tests/keystonef429-account-delete-m9szg" event={"ID":"5179e964-3df1-4d07-b440-a01ba074ac05","Type":"ContainerDied","Data":"06eb853e582954e757cdbf2f774b8864440ed80f9893b4fdb23abe72ed90929f"} Dec 03 13:46:48 crc kubenswrapper[5052]: I1203 13:46:48.792957 5052 generic.go:334] "Generic (PLEG): container finished" podID="d8df61e7-dad4-4072-a268-ddc6b55f0696" containerID="e698f2ff7e2689a1f1a877cb6db7518a6479f4b8d5bedb508f36e4f0ed3c80c0" exitCode=0 Dec 03 13:46:48 crc kubenswrapper[5052]: I1203 13:46:48.792997 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="horizon-kuttl-tests/keystone-68bf6dbf4f-g4nbm" event={"ID":"d8df61e7-dad4-4072-a268-ddc6b55f0696","Type":"ContainerDied","Data":"e698f2ff7e2689a1f1a877cb6db7518a6479f4b8d5bedb508f36e4f0ed3c80c0"} Dec 03 13:46:48 crc kubenswrapper[5052]: I1203 13:46:48.793027 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="horizon-kuttl-tests/keystone-68bf6dbf4f-g4nbm" Dec 03 13:46:48 crc kubenswrapper[5052]: I1203 13:46:48.793048 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="horizon-kuttl-tests/keystone-68bf6dbf4f-g4nbm" event={"ID":"d8df61e7-dad4-4072-a268-ddc6b55f0696","Type":"ContainerDied","Data":"f9e6e6f32051263e5de536ea7744589066d3aca4ec9499a1d51c54be42f67fd9"} Dec 03 13:46:48 crc kubenswrapper[5052]: I1203 13:46:48.793066 5052 scope.go:117] "RemoveContainer" containerID="e698f2ff7e2689a1f1a877cb6db7518a6479f4b8d5bedb508f36e4f0ed3c80c0" Dec 03 13:46:48 crc kubenswrapper[5052]: I1203 13:46:48.815115 5052 scope.go:117] "RemoveContainer" containerID="e698f2ff7e2689a1f1a877cb6db7518a6479f4b8d5bedb508f36e4f0ed3c80c0" Dec 03 13:46:48 crc kubenswrapper[5052]: E1203 13:46:48.816546 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e698f2ff7e2689a1f1a877cb6db7518a6479f4b8d5bedb508f36e4f0ed3c80c0\": container with ID starting with e698f2ff7e2689a1f1a877cb6db7518a6479f4b8d5bedb508f36e4f0ed3c80c0 not found: ID does not exist" containerID="e698f2ff7e2689a1f1a877cb6db7518a6479f4b8d5bedb508f36e4f0ed3c80c0" Dec 03 13:46:48 crc kubenswrapper[5052]: I1203 13:46:48.816594 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e698f2ff7e2689a1f1a877cb6db7518a6479f4b8d5bedb508f36e4f0ed3c80c0"} err="failed to get container status \"e698f2ff7e2689a1f1a877cb6db7518a6479f4b8d5bedb508f36e4f0ed3c80c0\": rpc error: code = NotFound desc = could not find container \"e698f2ff7e2689a1f1a877cb6db7518a6479f4b8d5bedb508f36e4f0ed3c80c0\": container with ID starting with e698f2ff7e2689a1f1a877cb6db7518a6479f4b8d5bedb508f36e4f0ed3c80c0 not found: ID does not exist" Dec 03 13:46:48 crc kubenswrapper[5052]: I1203 13:46:48.831480 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["horizon-kuttl-tests/keystone-68bf6dbf4f-g4nbm"] Dec 03 13:46:48 crc kubenswrapper[5052]: I1203 13:46:48.837856 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["horizon-kuttl-tests/keystone-68bf6dbf4f-g4nbm"] Dec 03 13:46:48 crc kubenswrapper[5052]: I1203 13:46:48.920689 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["horizon-kuttl-tests/memcached-0"] Dec 03 13:46:48 crc kubenswrapper[5052]: I1203 13:46:48.920935 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="horizon-kuttl-tests/memcached-0" podUID="9dfea40e-ca29-4e45-85ff-fde0ca025e75" containerName="memcached" containerID="cri-o://3da4c1b8183200dee64b40f8bf2a81811e6e3aa41058e945b5ba1277203b2410" gracePeriod=30 Dec 03 13:46:49 crc kubenswrapper[5052]: I1203 13:46:49.605922 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["horizon-kuttl-tests/rabbitmq-server-0"] Dec 03 13:46:49 crc kubenswrapper[5052]: I1203 13:46:49.680167 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d8df61e7-dad4-4072-a268-ddc6b55f0696" path="/var/lib/kubelet/pods/d8df61e7-dad4-4072-a268-ddc6b55f0696/volumes" Dec 03 13:46:49 crc kubenswrapper[5052]: I1203 13:46:49.801454 5052 generic.go:334] "Generic (PLEG): container finished" podID="a2db0de0-8930-4d21-ad37-58889377d33b" containerID="fbc326fc54a520afef74ee80957e5d836345c10d4d03b2e32cd807930864490e" exitCode=0 Dec 03 13:46:49 crc kubenswrapper[5052]: I1203 13:46:49.801531 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="horizon-kuttl-tests/openstack-galera-2" event={"ID":"a2db0de0-8930-4d21-ad37-58889377d33b","Type":"ContainerDied","Data":"fbc326fc54a520afef74ee80957e5d836345c10d4d03b2e32cd807930864490e"} Dec 03 13:46:49 crc kubenswrapper[5052]: I1203 13:46:49.836468 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["horizon-kuttl-tests/keystone-db-create-c58sf"] Dec 03 13:46:49 crc kubenswrapper[5052]: I1203 13:46:49.845613 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["horizon-kuttl-tests/keystone-db-create-c58sf"] Dec 03 13:46:49 crc kubenswrapper[5052]: I1203 13:46:49.852055 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["horizon-kuttl-tests/keystone-f429-account-create-update-r6vvp"] Dec 03 13:46:49 crc kubenswrapper[5052]: I1203 13:46:49.862832 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["horizon-kuttl-tests/keystone-f429-account-create-update-r6vvp"] Dec 03 13:46:49 crc kubenswrapper[5052]: I1203 13:46:49.868673 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["horizon-kuttl-tests/keystonef429-account-delete-m9szg"] Dec 03 13:46:50 crc kubenswrapper[5052]: I1203 13:46:50.002003 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["horizon-kuttl-tests/rabbitmq-server-0"] Dec 03 13:46:50 crc kubenswrapper[5052]: I1203 13:46:50.044475 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="horizon-kuttl-tests/rabbitmq-server-0" podUID="aa07e692-9f86-4c61-b674-beeb49376c0e" containerName="rabbitmq" containerID="cri-o://574b3d04ceb3e56d6b1097c3df01bb5f5d8454557a84049929fc504a6dc656ea" gracePeriod=604800 Dec 03 13:46:50 crc kubenswrapper[5052]: I1203 13:46:50.155543 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="horizon-kuttl-tests/keystonef429-account-delete-m9szg" Dec 03 13:46:50 crc kubenswrapper[5052]: I1203 13:46:50.271397 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="horizon-kuttl-tests/openstack-galera-1" podUID="2e3a3ea5-4ff6-4c9f-a093-38754c10012a" containerName="galera" containerID="cri-o://13cf7bd27ade2aba6b22d0a54ef5527f62fa3398c10e1e0f8714b3adc70d7e83" gracePeriod=28 Dec 03 13:46:50 crc kubenswrapper[5052]: I1203 13:46:50.301213 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/5179e964-3df1-4d07-b440-a01ba074ac05-operator-scripts\") pod \"5179e964-3df1-4d07-b440-a01ba074ac05\" (UID: \"5179e964-3df1-4d07-b440-a01ba074ac05\") " Dec 03 13:46:50 crc kubenswrapper[5052]: I1203 13:46:50.301304 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5xq28\" (UniqueName: \"kubernetes.io/projected/5179e964-3df1-4d07-b440-a01ba074ac05-kube-api-access-5xq28\") pod \"5179e964-3df1-4d07-b440-a01ba074ac05\" (UID: \"5179e964-3df1-4d07-b440-a01ba074ac05\") " Dec 03 13:46:50 crc kubenswrapper[5052]: I1203 13:46:50.302349 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5179e964-3df1-4d07-b440-a01ba074ac05-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "5179e964-3df1-4d07-b440-a01ba074ac05" (UID: "5179e964-3df1-4d07-b440-a01ba074ac05"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:46:50 crc kubenswrapper[5052]: I1203 13:46:50.307332 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5179e964-3df1-4d07-b440-a01ba074ac05-kube-api-access-5xq28" (OuterVolumeSpecName: "kube-api-access-5xq28") pod "5179e964-3df1-4d07-b440-a01ba074ac05" (UID: "5179e964-3df1-4d07-b440-a01ba074ac05"). InnerVolumeSpecName "kube-api-access-5xq28". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:46:50 crc kubenswrapper[5052]: I1203 13:46:50.403126 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/5179e964-3df1-4d07-b440-a01ba074ac05-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 13:46:50 crc kubenswrapper[5052]: I1203 13:46:50.403168 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-5xq28\" (UniqueName: \"kubernetes.io/projected/5179e964-3df1-4d07-b440-a01ba074ac05-kube-api-access-5xq28\") on node \"crc\" DevicePath \"\"" Dec 03 13:46:50 crc kubenswrapper[5052]: I1203 13:46:50.627031 5052 prober.go:107] "Probe failed" probeType="Readiness" pod="horizon-kuttl-tests/rabbitmq-server-0" podUID="aa07e692-9f86-4c61-b674-beeb49376c0e" containerName="rabbitmq" probeResult="failure" output="dial tcp 10.217.0.16:5672: connect: connection refused" Dec 03 13:46:50 crc kubenswrapper[5052]: I1203 13:46:50.696980 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="horizon-kuttl-tests/openstack-galera-2" Dec 03 13:46:50 crc kubenswrapper[5052]: I1203 13:46:50.810386 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"mysql-db\" (UniqueName: \"kubernetes.io/local-volume/local-storage06-crc\") pod \"a2db0de0-8930-4d21-ad37-58889377d33b\" (UID: \"a2db0de0-8930-4d21-ad37-58889377d33b\") " Dec 03 13:46:50 crc kubenswrapper[5052]: I1203 13:46:50.810493 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/a2db0de0-8930-4d21-ad37-58889377d33b-config-data-generated\") pod \"a2db0de0-8930-4d21-ad37-58889377d33b\" (UID: \"a2db0de0-8930-4d21-ad37-58889377d33b\") " Dec 03 13:46:50 crc kubenswrapper[5052]: I1203 13:46:50.810570 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/a2db0de0-8930-4d21-ad37-58889377d33b-kolla-config\") pod \"a2db0de0-8930-4d21-ad37-58889377d33b\" (UID: \"a2db0de0-8930-4d21-ad37-58889377d33b\") " Dec 03 13:46:50 crc kubenswrapper[5052]: I1203 13:46:50.810597 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bcjgn\" (UniqueName: \"kubernetes.io/projected/a2db0de0-8930-4d21-ad37-58889377d33b-kube-api-access-bcjgn\") pod \"a2db0de0-8930-4d21-ad37-58889377d33b\" (UID: \"a2db0de0-8930-4d21-ad37-58889377d33b\") " Dec 03 13:46:50 crc kubenswrapper[5052]: I1203 13:46:50.810629 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/a2db0de0-8930-4d21-ad37-58889377d33b-operator-scripts\") pod \"a2db0de0-8930-4d21-ad37-58889377d33b\" (UID: \"a2db0de0-8930-4d21-ad37-58889377d33b\") " Dec 03 13:46:50 crc kubenswrapper[5052]: I1203 13:46:50.810660 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/a2db0de0-8930-4d21-ad37-58889377d33b-config-data-default\") pod \"a2db0de0-8930-4d21-ad37-58889377d33b\" (UID: \"a2db0de0-8930-4d21-ad37-58889377d33b\") " Dec 03 13:46:50 crc kubenswrapper[5052]: I1203 13:46:50.810906 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2db0de0-8930-4d21-ad37-58889377d33b-config-data-generated" (OuterVolumeSpecName: "config-data-generated") pod "a2db0de0-8930-4d21-ad37-58889377d33b" (UID: "a2db0de0-8930-4d21-ad37-58889377d33b"). InnerVolumeSpecName "config-data-generated". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 13:46:50 crc kubenswrapper[5052]: I1203 13:46:50.811277 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/a2db0de0-8930-4d21-ad37-58889377d33b-config-data-generated\") on node \"crc\" DevicePath \"\"" Dec 03 13:46:50 crc kubenswrapper[5052]: I1203 13:46:50.811360 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a2db0de0-8930-4d21-ad37-58889377d33b-kolla-config" (OuterVolumeSpecName: "kolla-config") pod "a2db0de0-8930-4d21-ad37-58889377d33b" (UID: "a2db0de0-8930-4d21-ad37-58889377d33b"). InnerVolumeSpecName "kolla-config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:46:50 crc kubenswrapper[5052]: I1203 13:46:50.811448 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a2db0de0-8930-4d21-ad37-58889377d33b-config-data-default" (OuterVolumeSpecName: "config-data-default") pod "a2db0de0-8930-4d21-ad37-58889377d33b" (UID: "a2db0de0-8930-4d21-ad37-58889377d33b"). InnerVolumeSpecName "config-data-default". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:46:50 crc kubenswrapper[5052]: I1203 13:46:50.812005 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="horizon-kuttl-tests/keystonef429-account-delete-m9szg" event={"ID":"5179e964-3df1-4d07-b440-a01ba074ac05","Type":"ContainerDied","Data":"9c7ab2751625b70bbc10179bc935d5e2089069140ad91315588ad4ee3ecdb34e"} Dec 03 13:46:50 crc kubenswrapper[5052]: I1203 13:46:50.812071 5052 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9c7ab2751625b70bbc10179bc935d5e2089069140ad91315588ad4ee3ecdb34e" Dec 03 13:46:50 crc kubenswrapper[5052]: I1203 13:46:50.812111 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="horizon-kuttl-tests/keystonef429-account-delete-m9szg" Dec 03 13:46:50 crc kubenswrapper[5052]: I1203 13:46:50.812382 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a2db0de0-8930-4d21-ad37-58889377d33b-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "a2db0de0-8930-4d21-ad37-58889377d33b" (UID: "a2db0de0-8930-4d21-ad37-58889377d33b"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:46:50 crc kubenswrapper[5052]: I1203 13:46:50.814209 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="horizon-kuttl-tests/openstack-galera-2" event={"ID":"a2db0de0-8930-4d21-ad37-58889377d33b","Type":"ContainerDied","Data":"7ad7f58856967988188b660fbc86a7f695b89a66430918f2fe18d1f92c2282c0"} Dec 03 13:46:50 crc kubenswrapper[5052]: I1203 13:46:50.814252 5052 scope.go:117] "RemoveContainer" containerID="fbc326fc54a520afef74ee80957e5d836345c10d4d03b2e32cd807930864490e" Dec 03 13:46:50 crc kubenswrapper[5052]: I1203 13:46:50.814401 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="horizon-kuttl-tests/openstack-galera-2" Dec 03 13:46:50 crc kubenswrapper[5052]: I1203 13:46:50.816575 5052 generic.go:334] "Generic (PLEG): container finished" podID="9dfea40e-ca29-4e45-85ff-fde0ca025e75" containerID="3da4c1b8183200dee64b40f8bf2a81811e6e3aa41058e945b5ba1277203b2410" exitCode=0 Dec 03 13:46:50 crc kubenswrapper[5052]: I1203 13:46:50.816602 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="horizon-kuttl-tests/memcached-0" event={"ID":"9dfea40e-ca29-4e45-85ff-fde0ca025e75","Type":"ContainerDied","Data":"3da4c1b8183200dee64b40f8bf2a81811e6e3aa41058e945b5ba1277203b2410"} Dec 03 13:46:50 crc kubenswrapper[5052]: I1203 13:46:50.819441 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage06-crc" (OuterVolumeSpecName: "mysql-db") pod "a2db0de0-8930-4d21-ad37-58889377d33b" (UID: "a2db0de0-8930-4d21-ad37-58889377d33b"). InnerVolumeSpecName "local-storage06-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 13:46:50 crc kubenswrapper[5052]: I1203 13:46:50.832949 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a2db0de0-8930-4d21-ad37-58889377d33b-kube-api-access-bcjgn" (OuterVolumeSpecName: "kube-api-access-bcjgn") pod "a2db0de0-8930-4d21-ad37-58889377d33b" (UID: "a2db0de0-8930-4d21-ad37-58889377d33b"). InnerVolumeSpecName "kube-api-access-bcjgn". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:46:50 crc kubenswrapper[5052]: I1203 13:46:50.912118 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage06-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage06-crc\") on node \"crc\" " Dec 03 13:46:50 crc kubenswrapper[5052]: I1203 13:46:50.912145 5052 reconciler_common.go:293] "Volume detached for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/a2db0de0-8930-4d21-ad37-58889377d33b-kolla-config\") on node \"crc\" DevicePath \"\"" Dec 03 13:46:50 crc kubenswrapper[5052]: I1203 13:46:50.912159 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-bcjgn\" (UniqueName: \"kubernetes.io/projected/a2db0de0-8930-4d21-ad37-58889377d33b-kube-api-access-bcjgn\") on node \"crc\" DevicePath \"\"" Dec 03 13:46:50 crc kubenswrapper[5052]: I1203 13:46:50.912170 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/a2db0de0-8930-4d21-ad37-58889377d33b-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 13:46:50 crc kubenswrapper[5052]: I1203 13:46:50.912180 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/a2db0de0-8930-4d21-ad37-58889377d33b-config-data-default\") on node \"crc\" DevicePath \"\"" Dec 03 13:46:50 crc kubenswrapper[5052]: I1203 13:46:50.929729 5052 scope.go:117] "RemoveContainer" containerID="e632e67c1bed917f6c8760138200ea096bc9a6b7c545fd437d184fea50809f96" Dec 03 13:46:50 crc kubenswrapper[5052]: I1203 13:46:50.932580 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage06-crc" (UniqueName: "kubernetes.io/local-volume/local-storage06-crc") on node "crc" Dec 03 13:46:50 crc kubenswrapper[5052]: I1203 13:46:50.938054 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="horizon-kuttl-tests/memcached-0" Dec 03 13:46:50 crc kubenswrapper[5052]: I1203 13:46:50.943617 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["horizon-kuttl-tests/keystonef429-account-delete-m9szg"] Dec 03 13:46:50 crc kubenswrapper[5052]: I1203 13:46:50.969092 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["horizon-kuttl-tests/keystonef429-account-delete-m9szg"] Dec 03 13:46:51 crc kubenswrapper[5052]: I1203 13:46:51.015145 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage06-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage06-crc\") on node \"crc\" DevicePath \"\"" Dec 03 13:46:51 crc kubenswrapper[5052]: I1203 13:46:51.102328 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-operators/horizon-operator-controller-manager-9b47db64f-zztn2"] Dec 03 13:46:51 crc kubenswrapper[5052]: I1203 13:46:51.106192 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-operators/horizon-operator-controller-manager-9b47db64f-zztn2" podUID="5b2f3c3a-2756-4269-99da-ba83e8097c9e" containerName="manager" containerID="cri-o://6075c6b42637b01fc6c756047fec5638fa9a992df8271f644096237de65e87d9" gracePeriod=10 Dec 03 13:46:51 crc kubenswrapper[5052]: I1203 13:46:51.116418 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/9dfea40e-ca29-4e45-85ff-fde0ca025e75-config-data\") pod \"9dfea40e-ca29-4e45-85ff-fde0ca025e75\" (UID: \"9dfea40e-ca29-4e45-85ff-fde0ca025e75\") " Dec 03 13:46:51 crc kubenswrapper[5052]: I1203 13:46:51.116578 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/9dfea40e-ca29-4e45-85ff-fde0ca025e75-kolla-config\") pod \"9dfea40e-ca29-4e45-85ff-fde0ca025e75\" (UID: \"9dfea40e-ca29-4e45-85ff-fde0ca025e75\") " Dec 03 13:46:51 crc kubenswrapper[5052]: I1203 13:46:51.116647 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lj4j7\" (UniqueName: \"kubernetes.io/projected/9dfea40e-ca29-4e45-85ff-fde0ca025e75-kube-api-access-lj4j7\") pod \"9dfea40e-ca29-4e45-85ff-fde0ca025e75\" (UID: \"9dfea40e-ca29-4e45-85ff-fde0ca025e75\") " Dec 03 13:46:51 crc kubenswrapper[5052]: I1203 13:46:51.117179 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9dfea40e-ca29-4e45-85ff-fde0ca025e75-kolla-config" (OuterVolumeSpecName: "kolla-config") pod "9dfea40e-ca29-4e45-85ff-fde0ca025e75" (UID: "9dfea40e-ca29-4e45-85ff-fde0ca025e75"). InnerVolumeSpecName "kolla-config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:46:51 crc kubenswrapper[5052]: I1203 13:46:51.117197 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9dfea40e-ca29-4e45-85ff-fde0ca025e75-config-data" (OuterVolumeSpecName: "config-data") pod "9dfea40e-ca29-4e45-85ff-fde0ca025e75" (UID: "9dfea40e-ca29-4e45-85ff-fde0ca025e75"). InnerVolumeSpecName "config-data". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:46:51 crc kubenswrapper[5052]: I1203 13:46:51.124834 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9dfea40e-ca29-4e45-85ff-fde0ca025e75-kube-api-access-lj4j7" (OuterVolumeSpecName: "kube-api-access-lj4j7") pod "9dfea40e-ca29-4e45-85ff-fde0ca025e75" (UID: "9dfea40e-ca29-4e45-85ff-fde0ca025e75"). InnerVolumeSpecName "kube-api-access-lj4j7". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:46:51 crc kubenswrapper[5052]: I1203 13:46:51.162520 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["horizon-kuttl-tests/openstack-galera-2"] Dec 03 13:46:51 crc kubenswrapper[5052]: I1203 13:46:51.168318 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["horizon-kuttl-tests/openstack-galera-2"] Dec 03 13:46:51 crc kubenswrapper[5052]: I1203 13:46:51.218253 5052 reconciler_common.go:293] "Volume detached for volume \"config-data\" (UniqueName: \"kubernetes.io/configmap/9dfea40e-ca29-4e45-85ff-fde0ca025e75-config-data\") on node \"crc\" DevicePath \"\"" Dec 03 13:46:51 crc kubenswrapper[5052]: I1203 13:46:51.218290 5052 reconciler_common.go:293] "Volume detached for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/9dfea40e-ca29-4e45-85ff-fde0ca025e75-kolla-config\") on node \"crc\" DevicePath \"\"" Dec 03 13:46:51 crc kubenswrapper[5052]: I1203 13:46:51.218303 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-lj4j7\" (UniqueName: \"kubernetes.io/projected/9dfea40e-ca29-4e45-85ff-fde0ca025e75-kube-api-access-lj4j7\") on node \"crc\" DevicePath \"\"" Dec 03 13:46:51 crc kubenswrapper[5052]: I1203 13:46:51.356502 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-operators/horizon-operator-index-gkbwv"] Dec 03 13:46:51 crc kubenswrapper[5052]: I1203 13:46:51.356721 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-operators/horizon-operator-index-gkbwv" podUID="8496c1b5-ae6d-4357-9812-9d44818ee9d1" containerName="registry-server" containerID="cri-o://58a8a984bcc25e97fc3cb0134e46c0fd093e75da4a13403e3a7dcd59d80b2901" gracePeriod=30 Dec 03 13:46:51 crc kubenswrapper[5052]: I1203 13:46:51.392017 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-operators/9f0c59a3968beec894e04476dd5efd0a707bad85f482efd4940498368cx9f7f"] Dec 03 13:46:51 crc kubenswrapper[5052]: I1203 13:46:51.397499 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-operators/9f0c59a3968beec894e04476dd5efd0a707bad85f482efd4940498368cx9f7f"] Dec 03 13:46:51 crc kubenswrapper[5052]: I1203 13:46:51.683189 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="12abe26f-468f-4c73-aa6a-11a42582ebc3" path="/var/lib/kubelet/pods/12abe26f-468f-4c73-aa6a-11a42582ebc3/volumes" Dec 03 13:46:51 crc kubenswrapper[5052]: I1203 13:46:51.683726 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3f46060f-d03d-452f-b08a-baa387f2efbf" path="/var/lib/kubelet/pods/3f46060f-d03d-452f-b08a-baa387f2efbf/volumes" Dec 03 13:46:51 crc kubenswrapper[5052]: I1203 13:46:51.684302 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5179e964-3df1-4d07-b440-a01ba074ac05" path="/var/lib/kubelet/pods/5179e964-3df1-4d07-b440-a01ba074ac05/volumes" Dec 03 13:46:51 crc kubenswrapper[5052]: I1203 13:46:51.684841 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a2db0de0-8930-4d21-ad37-58889377d33b" path="/var/lib/kubelet/pods/a2db0de0-8930-4d21-ad37-58889377d33b/volumes" Dec 03 13:46:51 crc kubenswrapper[5052]: I1203 13:46:51.685420 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e500bdc5-8e72-481a-8e8c-177ddfb97582" path="/var/lib/kubelet/pods/e500bdc5-8e72-481a-8e8c-177ddfb97582/volumes" Dec 03 13:46:51 crc kubenswrapper[5052]: I1203 13:46:51.841170 5052 generic.go:334] "Generic (PLEG): container finished" podID="8496c1b5-ae6d-4357-9812-9d44818ee9d1" containerID="58a8a984bcc25e97fc3cb0134e46c0fd093e75da4a13403e3a7dcd59d80b2901" exitCode=0 Dec 03 13:46:51 crc kubenswrapper[5052]: I1203 13:46:51.841246 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/horizon-operator-index-gkbwv" event={"ID":"8496c1b5-ae6d-4357-9812-9d44818ee9d1","Type":"ContainerDied","Data":"58a8a984bcc25e97fc3cb0134e46c0fd093e75da4a13403e3a7dcd59d80b2901"} Dec 03 13:46:51 crc kubenswrapper[5052]: I1203 13:46:51.843093 5052 generic.go:334] "Generic (PLEG): container finished" podID="5b2f3c3a-2756-4269-99da-ba83e8097c9e" containerID="6075c6b42637b01fc6c756047fec5638fa9a992df8271f644096237de65e87d9" exitCode=0 Dec 03 13:46:51 crc kubenswrapper[5052]: I1203 13:46:51.843151 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/horizon-operator-controller-manager-9b47db64f-zztn2" event={"ID":"5b2f3c3a-2756-4269-99da-ba83e8097c9e","Type":"ContainerDied","Data":"6075c6b42637b01fc6c756047fec5638fa9a992df8271f644096237de65e87d9"} Dec 03 13:46:51 crc kubenswrapper[5052]: I1203 13:46:51.844315 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="horizon-kuttl-tests/memcached-0" event={"ID":"9dfea40e-ca29-4e45-85ff-fde0ca025e75","Type":"ContainerDied","Data":"079d371f2803e67ed128a139ef7126f51323a9fb550c4d00094fd7db8be77b7b"} Dec 03 13:46:51 crc kubenswrapper[5052]: I1203 13:46:51.844344 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="horizon-kuttl-tests/memcached-0" Dec 03 13:46:51 crc kubenswrapper[5052]: I1203 13:46:51.844347 5052 scope.go:117] "RemoveContainer" containerID="3da4c1b8183200dee64b40f8bf2a81811e6e3aa41058e945b5ba1277203b2410" Dec 03 13:46:51 crc kubenswrapper[5052]: I1203 13:46:51.846053 5052 generic.go:334] "Generic (PLEG): container finished" podID="aa07e692-9f86-4c61-b674-beeb49376c0e" containerID="574b3d04ceb3e56d6b1097c3df01bb5f5d8454557a84049929fc504a6dc656ea" exitCode=0 Dec 03 13:46:51 crc kubenswrapper[5052]: I1203 13:46:51.846100 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="horizon-kuttl-tests/rabbitmq-server-0" event={"ID":"aa07e692-9f86-4c61-b674-beeb49376c0e","Type":"ContainerDied","Data":"574b3d04ceb3e56d6b1097c3df01bb5f5d8454557a84049929fc504a6dc656ea"} Dec 03 13:46:51 crc kubenswrapper[5052]: I1203 13:46:51.872043 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["horizon-kuttl-tests/memcached-0"] Dec 03 13:46:51 crc kubenswrapper[5052]: I1203 13:46:51.881230 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["horizon-kuttl-tests/memcached-0"] Dec 03 13:46:52 crc kubenswrapper[5052]: I1203 13:46:52.069086 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-operators/horizon-operator-controller-manager-9b47db64f-zztn2" Dec 03 13:46:52 crc kubenswrapper[5052]: I1203 13:46:52.190871 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="horizon-kuttl-tests/rabbitmq-server-0" Dec 03 13:46:52 crc kubenswrapper[5052]: I1203 13:46:52.253064 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/5b2f3c3a-2756-4269-99da-ba83e8097c9e-apiservice-cert\") pod \"5b2f3c3a-2756-4269-99da-ba83e8097c9e\" (UID: \"5b2f3c3a-2756-4269-99da-ba83e8097c9e\") " Dec 03 13:46:52 crc kubenswrapper[5052]: I1203 13:46:52.253103 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5b2f3c3a-2756-4269-99da-ba83e8097c9e-webhook-cert\") pod \"5b2f3c3a-2756-4269-99da-ba83e8097c9e\" (UID: \"5b2f3c3a-2756-4269-99da-ba83e8097c9e\") " Dec 03 13:46:52 crc kubenswrapper[5052]: I1203 13:46:52.253157 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pdtdk\" (UniqueName: \"kubernetes.io/projected/5b2f3c3a-2756-4269-99da-ba83e8097c9e-kube-api-access-pdtdk\") pod \"5b2f3c3a-2756-4269-99da-ba83e8097c9e\" (UID: \"5b2f3c3a-2756-4269-99da-ba83e8097c9e\") " Dec 03 13:46:52 crc kubenswrapper[5052]: I1203 13:46:52.263658 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5b2f3c3a-2756-4269-99da-ba83e8097c9e-apiservice-cert" (OuterVolumeSpecName: "apiservice-cert") pod "5b2f3c3a-2756-4269-99da-ba83e8097c9e" (UID: "5b2f3c3a-2756-4269-99da-ba83e8097c9e"). InnerVolumeSpecName "apiservice-cert". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:46:52 crc kubenswrapper[5052]: I1203 13:46:52.269714 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5b2f3c3a-2756-4269-99da-ba83e8097c9e-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "5b2f3c3a-2756-4269-99da-ba83e8097c9e" (UID: "5b2f3c3a-2756-4269-99da-ba83e8097c9e"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:46:52 crc kubenswrapper[5052]: I1203 13:46:52.272925 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5b2f3c3a-2756-4269-99da-ba83e8097c9e-kube-api-access-pdtdk" (OuterVolumeSpecName: "kube-api-access-pdtdk") pod "5b2f3c3a-2756-4269-99da-ba83e8097c9e" (UID: "5b2f3c3a-2756-4269-99da-ba83e8097c9e"). InnerVolumeSpecName "kube-api-access-pdtdk". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:46:52 crc kubenswrapper[5052]: I1203 13:46:52.357959 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"rabbitmq-confd\" (UniqueName: \"kubernetes.io/projected/aa07e692-9f86-4c61-b674-beeb49376c0e-rabbitmq-confd\") pod \"aa07e692-9f86-4c61-b674-beeb49376c0e\" (UID: \"aa07e692-9f86-4c61-b674-beeb49376c0e\") " Dec 03 13:46:52 crc kubenswrapper[5052]: I1203 13:46:52.358038 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"plugins-conf\" (UniqueName: \"kubernetes.io/configmap/aa07e692-9f86-4c61-b674-beeb49376c0e-plugins-conf\") pod \"aa07e692-9f86-4c61-b674-beeb49376c0e\" (UID: \"aa07e692-9f86-4c61-b674-beeb49376c0e\") " Dec 03 13:46:52 crc kubenswrapper[5052]: I1203 13:46:52.358090 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"rabbitmq-erlang-cookie\" (UniqueName: \"kubernetes.io/empty-dir/aa07e692-9f86-4c61-b674-beeb49376c0e-rabbitmq-erlang-cookie\") pod \"aa07e692-9f86-4c61-b674-beeb49376c0e\" (UID: \"aa07e692-9f86-4c61-b674-beeb49376c0e\") " Dec 03 13:46:52 crc kubenswrapper[5052]: I1203 13:46:52.358145 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lwvhx\" (UniqueName: \"kubernetes.io/projected/aa07e692-9f86-4c61-b674-beeb49376c0e-kube-api-access-lwvhx\") pod \"aa07e692-9f86-4c61-b674-beeb49376c0e\" (UID: \"aa07e692-9f86-4c61-b674-beeb49376c0e\") " Dec 03 13:46:52 crc kubenswrapper[5052]: I1203 13:46:52.358203 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"pod-info\" (UniqueName: \"kubernetes.io/downward-api/aa07e692-9f86-4c61-b674-beeb49376c0e-pod-info\") pod \"aa07e692-9f86-4c61-b674-beeb49376c0e\" (UID: \"aa07e692-9f86-4c61-b674-beeb49376c0e\") " Dec 03 13:46:52 crc kubenswrapper[5052]: I1203 13:46:52.358227 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"rabbitmq-plugins\" (UniqueName: \"kubernetes.io/empty-dir/aa07e692-9f86-4c61-b674-beeb49376c0e-rabbitmq-plugins\") pod \"aa07e692-9f86-4c61-b674-beeb49376c0e\" (UID: \"aa07e692-9f86-4c61-b674-beeb49376c0e\") " Dec 03 13:46:52 crc kubenswrapper[5052]: I1203 13:46:52.358392 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"persistence\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-64073964-8794-4f6e-ae70-f5b8cada0665\") pod \"aa07e692-9f86-4c61-b674-beeb49376c0e\" (UID: \"aa07e692-9f86-4c61-b674-beeb49376c0e\") " Dec 03 13:46:52 crc kubenswrapper[5052]: I1203 13:46:52.358451 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"erlang-cookie-secret\" (UniqueName: \"kubernetes.io/secret/aa07e692-9f86-4c61-b674-beeb49376c0e-erlang-cookie-secret\") pod \"aa07e692-9f86-4c61-b674-beeb49376c0e\" (UID: \"aa07e692-9f86-4c61-b674-beeb49376c0e\") " Dec 03 13:46:52 crc kubenswrapper[5052]: I1203 13:46:52.358721 5052 reconciler_common.go:293] "Volume detached for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/5b2f3c3a-2756-4269-99da-ba83e8097c9e-apiservice-cert\") on node \"crc\" DevicePath \"\"" Dec 03 13:46:52 crc kubenswrapper[5052]: I1203 13:46:52.358737 5052 reconciler_common.go:293] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5b2f3c3a-2756-4269-99da-ba83e8097c9e-webhook-cert\") on node \"crc\" DevicePath \"\"" Dec 03 13:46:52 crc kubenswrapper[5052]: I1203 13:46:52.358748 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-pdtdk\" (UniqueName: \"kubernetes.io/projected/5b2f3c3a-2756-4269-99da-ba83e8097c9e-kube-api-access-pdtdk\") on node \"crc\" DevicePath \"\"" Dec 03 13:46:52 crc kubenswrapper[5052]: I1203 13:46:52.363045 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa07e692-9f86-4c61-b674-beeb49376c0e-rabbitmq-plugins" (OuterVolumeSpecName: "rabbitmq-plugins") pod "aa07e692-9f86-4c61-b674-beeb49376c0e" (UID: "aa07e692-9f86-4c61-b674-beeb49376c0e"). InnerVolumeSpecName "rabbitmq-plugins". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 13:46:52 crc kubenswrapper[5052]: I1203 13:46:52.366729 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aa07e692-9f86-4c61-b674-beeb49376c0e-plugins-conf" (OuterVolumeSpecName: "plugins-conf") pod "aa07e692-9f86-4c61-b674-beeb49376c0e" (UID: "aa07e692-9f86-4c61-b674-beeb49376c0e"). InnerVolumeSpecName "plugins-conf". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:46:52 crc kubenswrapper[5052]: I1203 13:46:52.368556 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aa07e692-9f86-4c61-b674-beeb49376c0e-kube-api-access-lwvhx" (OuterVolumeSpecName: "kube-api-access-lwvhx") pod "aa07e692-9f86-4c61-b674-beeb49376c0e" (UID: "aa07e692-9f86-4c61-b674-beeb49376c0e"). InnerVolumeSpecName "kube-api-access-lwvhx". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:46:52 crc kubenswrapper[5052]: I1203 13:46:52.368956 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa07e692-9f86-4c61-b674-beeb49376c0e-rabbitmq-erlang-cookie" (OuterVolumeSpecName: "rabbitmq-erlang-cookie") pod "aa07e692-9f86-4c61-b674-beeb49376c0e" (UID: "aa07e692-9f86-4c61-b674-beeb49376c0e"). InnerVolumeSpecName "rabbitmq-erlang-cookie". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 13:46:52 crc kubenswrapper[5052]: I1203 13:46:52.404841 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/aa07e692-9f86-4c61-b674-beeb49376c0e-pod-info" (OuterVolumeSpecName: "pod-info") pod "aa07e692-9f86-4c61-b674-beeb49376c0e" (UID: "aa07e692-9f86-4c61-b674-beeb49376c0e"). InnerVolumeSpecName "pod-info". PluginName "kubernetes.io/downward-api", VolumeGidValue "" Dec 03 13:46:52 crc kubenswrapper[5052]: I1203 13:46:52.404915 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aa07e692-9f86-4c61-b674-beeb49376c0e-erlang-cookie-secret" (OuterVolumeSpecName: "erlang-cookie-secret") pod "aa07e692-9f86-4c61-b674-beeb49376c0e" (UID: "aa07e692-9f86-4c61-b674-beeb49376c0e"). InnerVolumeSpecName "erlang-cookie-secret". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:46:52 crc kubenswrapper[5052]: I1203 13:46:52.459958 5052 reconciler_common.go:293] "Volume detached for volume \"pod-info\" (UniqueName: \"kubernetes.io/downward-api/aa07e692-9f86-4c61-b674-beeb49376c0e-pod-info\") on node \"crc\" DevicePath \"\"" Dec 03 13:46:52 crc kubenswrapper[5052]: I1203 13:46:52.459999 5052 reconciler_common.go:293] "Volume detached for volume \"rabbitmq-plugins\" (UniqueName: \"kubernetes.io/empty-dir/aa07e692-9f86-4c61-b674-beeb49376c0e-rabbitmq-plugins\") on node \"crc\" DevicePath \"\"" Dec 03 13:46:52 crc kubenswrapper[5052]: I1203 13:46:52.460019 5052 reconciler_common.go:293] "Volume detached for volume \"erlang-cookie-secret\" (UniqueName: \"kubernetes.io/secret/aa07e692-9f86-4c61-b674-beeb49376c0e-erlang-cookie-secret\") on node \"crc\" DevicePath \"\"" Dec 03 13:46:52 crc kubenswrapper[5052]: I1203 13:46:52.460032 5052 reconciler_common.go:293] "Volume detached for volume \"plugins-conf\" (UniqueName: \"kubernetes.io/configmap/aa07e692-9f86-4c61-b674-beeb49376c0e-plugins-conf\") on node \"crc\" DevicePath \"\"" Dec 03 13:46:52 crc kubenswrapper[5052]: I1203 13:46:52.460045 5052 reconciler_common.go:293] "Volume detached for volume \"rabbitmq-erlang-cookie\" (UniqueName: \"kubernetes.io/empty-dir/aa07e692-9f86-4c61-b674-beeb49376c0e-rabbitmq-erlang-cookie\") on node \"crc\" DevicePath \"\"" Dec 03 13:46:52 crc kubenswrapper[5052]: I1203 13:46:52.460061 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-lwvhx\" (UniqueName: \"kubernetes.io/projected/aa07e692-9f86-4c61-b674-beeb49376c0e-kube-api-access-lwvhx\") on node \"crc\" DevicePath \"\"" Dec 03 13:46:52 crc kubenswrapper[5052]: I1203 13:46:52.463959 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-64073964-8794-4f6e-ae70-f5b8cada0665" (OuterVolumeSpecName: "persistence") pod "aa07e692-9f86-4c61-b674-beeb49376c0e" (UID: "aa07e692-9f86-4c61-b674-beeb49376c0e"). InnerVolumeSpecName "pvc-64073964-8794-4f6e-ae70-f5b8cada0665". PluginName "kubernetes.io/csi", VolumeGidValue "" Dec 03 13:46:52 crc kubenswrapper[5052]: I1203 13:46:52.480754 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="horizon-kuttl-tests/openstack-galera-0" podUID="5096dd99-719d-412e-804b-fc1452ee8b95" containerName="galera" containerID="cri-o://a302cf322d3d960edabc879fda7af5d1764ecc2cb32a3acf71f26c4b65d418d4" gracePeriod=26 Dec 03 13:46:52 crc kubenswrapper[5052]: I1203 13:46:52.485025 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aa07e692-9f86-4c61-b674-beeb49376c0e-rabbitmq-confd" (OuterVolumeSpecName: "rabbitmq-confd") pod "aa07e692-9f86-4c61-b674-beeb49376c0e" (UID: "aa07e692-9f86-4c61-b674-beeb49376c0e"). InnerVolumeSpecName "rabbitmq-confd". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:46:52 crc kubenswrapper[5052]: I1203 13:46:52.521261 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-operators/horizon-operator-index-gkbwv" Dec 03 13:46:52 crc kubenswrapper[5052]: I1203 13:46:52.562033 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"pvc-64073964-8794-4f6e-ae70-f5b8cada0665\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-64073964-8794-4f6e-ae70-f5b8cada0665\") on node \"crc\" " Dec 03 13:46:52 crc kubenswrapper[5052]: I1203 13:46:52.562072 5052 reconciler_common.go:293] "Volume detached for volume \"rabbitmq-confd\" (UniqueName: \"kubernetes.io/projected/aa07e692-9f86-4c61-b674-beeb49376c0e-rabbitmq-confd\") on node \"crc\" DevicePath \"\"" Dec 03 13:46:52 crc kubenswrapper[5052]: I1203 13:46:52.578212 5052 csi_attacher.go:630] kubernetes.io/csi: attacher.UnmountDevice STAGE_UNSTAGE_VOLUME capability not set. Skipping UnmountDevice... Dec 03 13:46:52 crc kubenswrapper[5052]: I1203 13:46:52.578367 5052 operation_generator.go:917] UnmountDevice succeeded for volume "pvc-64073964-8794-4f6e-ae70-f5b8cada0665" (UniqueName: "kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-64073964-8794-4f6e-ae70-f5b8cada0665") on node "crc" Dec 03 13:46:52 crc kubenswrapper[5052]: I1203 13:46:52.662716 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-79989\" (UniqueName: \"kubernetes.io/projected/8496c1b5-ae6d-4357-9812-9d44818ee9d1-kube-api-access-79989\") pod \"8496c1b5-ae6d-4357-9812-9d44818ee9d1\" (UID: \"8496c1b5-ae6d-4357-9812-9d44818ee9d1\") " Dec 03 13:46:52 crc kubenswrapper[5052]: I1203 13:46:52.664268 5052 reconciler_common.go:293] "Volume detached for volume \"pvc-64073964-8794-4f6e-ae70-f5b8cada0665\" (UniqueName: \"kubernetes.io/csi/kubevirt.io.hostpath-provisioner^pvc-64073964-8794-4f6e-ae70-f5b8cada0665\") on node \"crc\" DevicePath \"\"" Dec 03 13:46:52 crc kubenswrapper[5052]: I1203 13:46:52.668979 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8496c1b5-ae6d-4357-9812-9d44818ee9d1-kube-api-access-79989" (OuterVolumeSpecName: "kube-api-access-79989") pod "8496c1b5-ae6d-4357-9812-9d44818ee9d1" (UID: "8496c1b5-ae6d-4357-9812-9d44818ee9d1"). InnerVolumeSpecName "kube-api-access-79989". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:46:52 crc kubenswrapper[5052]: I1203 13:46:52.757898 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="horizon-kuttl-tests/openstack-galera-1" Dec 03 13:46:52 crc kubenswrapper[5052]: I1203 13:46:52.767797 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-79989\" (UniqueName: \"kubernetes.io/projected/8496c1b5-ae6d-4357-9812-9d44818ee9d1-kube-api-access-79989\") on node \"crc\" DevicePath \"\"" Dec 03 13:46:52 crc kubenswrapper[5052]: I1203 13:46:52.867575 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="horizon-kuttl-tests/rabbitmq-server-0" event={"ID":"aa07e692-9f86-4c61-b674-beeb49376c0e","Type":"ContainerDied","Data":"eb8a2577efef8333de462a2cd8da81a0d97dbe335d9a29d4675d151e5cd977cc"} Dec 03 13:46:52 crc kubenswrapper[5052]: I1203 13:46:52.867628 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="horizon-kuttl-tests/rabbitmq-server-0" Dec 03 13:46:52 crc kubenswrapper[5052]: I1203 13:46:52.868265 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/2e3a3ea5-4ff6-4c9f-a093-38754c10012a-operator-scripts\") pod \"2e3a3ea5-4ff6-4c9f-a093-38754c10012a\" (UID: \"2e3a3ea5-4ff6-4c9f-a093-38754c10012a\") " Dec 03 13:46:52 crc kubenswrapper[5052]: I1203 13:46:52.868309 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"mysql-db\" (UniqueName: \"kubernetes.io/local-volume/local-storage18-crc\") pod \"2e3a3ea5-4ff6-4c9f-a093-38754c10012a\" (UID: \"2e3a3ea5-4ff6-4c9f-a093-38754c10012a\") " Dec 03 13:46:52 crc kubenswrapper[5052]: I1203 13:46:52.868330 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/2e3a3ea5-4ff6-4c9f-a093-38754c10012a-kolla-config\") pod \"2e3a3ea5-4ff6-4c9f-a093-38754c10012a\" (UID: \"2e3a3ea5-4ff6-4c9f-a093-38754c10012a\") " Dec 03 13:46:52 crc kubenswrapper[5052]: I1203 13:46:52.868383 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/2e3a3ea5-4ff6-4c9f-a093-38754c10012a-config-data-default\") pod \"2e3a3ea5-4ff6-4c9f-a093-38754c10012a\" (UID: \"2e3a3ea5-4ff6-4c9f-a093-38754c10012a\") " Dec 03 13:46:52 crc kubenswrapper[5052]: I1203 13:46:52.868414 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/2e3a3ea5-4ff6-4c9f-a093-38754c10012a-config-data-generated\") pod \"2e3a3ea5-4ff6-4c9f-a093-38754c10012a\" (UID: \"2e3a3ea5-4ff6-4c9f-a093-38754c10012a\") " Dec 03 13:46:52 crc kubenswrapper[5052]: I1203 13:46:52.868457 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qwt8c\" (UniqueName: \"kubernetes.io/projected/2e3a3ea5-4ff6-4c9f-a093-38754c10012a-kube-api-access-qwt8c\") pod \"2e3a3ea5-4ff6-4c9f-a093-38754c10012a\" (UID: \"2e3a3ea5-4ff6-4c9f-a093-38754c10012a\") " Dec 03 13:46:52 crc kubenswrapper[5052]: I1203 13:46:52.867850 5052 scope.go:117] "RemoveContainer" containerID="574b3d04ceb3e56d6b1097c3df01bb5f5d8454557a84049929fc504a6dc656ea" Dec 03 13:46:52 crc kubenswrapper[5052]: I1203 13:46:52.870555 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2e3a3ea5-4ff6-4c9f-a093-38754c10012a-kolla-config" (OuterVolumeSpecName: "kolla-config") pod "2e3a3ea5-4ff6-4c9f-a093-38754c10012a" (UID: "2e3a3ea5-4ff6-4c9f-a093-38754c10012a"). InnerVolumeSpecName "kolla-config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:46:52 crc kubenswrapper[5052]: I1203 13:46:52.871071 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2e3a3ea5-4ff6-4c9f-a093-38754c10012a-config-data-default" (OuterVolumeSpecName: "config-data-default") pod "2e3a3ea5-4ff6-4c9f-a093-38754c10012a" (UID: "2e3a3ea5-4ff6-4c9f-a093-38754c10012a"). InnerVolumeSpecName "config-data-default". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:46:52 crc kubenswrapper[5052]: I1203 13:46:52.871251 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2e3a3ea5-4ff6-4c9f-a093-38754c10012a-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "2e3a3ea5-4ff6-4c9f-a093-38754c10012a" (UID: "2e3a3ea5-4ff6-4c9f-a093-38754c10012a"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:46:52 crc kubenswrapper[5052]: I1203 13:46:52.872073 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e3a3ea5-4ff6-4c9f-a093-38754c10012a-config-data-generated" (OuterVolumeSpecName: "config-data-generated") pod "2e3a3ea5-4ff6-4c9f-a093-38754c10012a" (UID: "2e3a3ea5-4ff6-4c9f-a093-38754c10012a"). InnerVolumeSpecName "config-data-generated". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 13:46:52 crc kubenswrapper[5052]: I1203 13:46:52.872313 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/horizon-operator-index-gkbwv" event={"ID":"8496c1b5-ae6d-4357-9812-9d44818ee9d1","Type":"ContainerDied","Data":"6d8d15ac75b70a3c76f418044b4fbc4e0e48dabf92bf342d17042ffb20ffcf63"} Dec 03 13:46:52 crc kubenswrapper[5052]: I1203 13:46:52.872390 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-operators/horizon-operator-index-gkbwv" Dec 03 13:46:52 crc kubenswrapper[5052]: I1203 13:46:52.873110 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2e3a3ea5-4ff6-4c9f-a093-38754c10012a-kube-api-access-qwt8c" (OuterVolumeSpecName: "kube-api-access-qwt8c") pod "2e3a3ea5-4ff6-4c9f-a093-38754c10012a" (UID: "2e3a3ea5-4ff6-4c9f-a093-38754c10012a"). InnerVolumeSpecName "kube-api-access-qwt8c". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:46:52 crc kubenswrapper[5052]: I1203 13:46:52.878318 5052 generic.go:334] "Generic (PLEG): container finished" podID="2e3a3ea5-4ff6-4c9f-a093-38754c10012a" containerID="13cf7bd27ade2aba6b22d0a54ef5527f62fa3398c10e1e0f8714b3adc70d7e83" exitCode=0 Dec 03 13:46:52 crc kubenswrapper[5052]: I1203 13:46:52.878368 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="horizon-kuttl-tests/openstack-galera-1" event={"ID":"2e3a3ea5-4ff6-4c9f-a093-38754c10012a","Type":"ContainerDied","Data":"13cf7bd27ade2aba6b22d0a54ef5527f62fa3398c10e1e0f8714b3adc70d7e83"} Dec 03 13:46:52 crc kubenswrapper[5052]: I1203 13:46:52.878408 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="horizon-kuttl-tests/openstack-galera-1" event={"ID":"2e3a3ea5-4ff6-4c9f-a093-38754c10012a","Type":"ContainerDied","Data":"a89f203ce2e2eb97a77dbff1eeb25d1e0344ea80e27e5ab0ca177959b28fe31c"} Dec 03 13:46:52 crc kubenswrapper[5052]: I1203 13:46:52.878418 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="horizon-kuttl-tests/openstack-galera-1" Dec 03 13:46:52 crc kubenswrapper[5052]: I1203 13:46:52.880086 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/horizon-operator-controller-manager-9b47db64f-zztn2" event={"ID":"5b2f3c3a-2756-4269-99da-ba83e8097c9e","Type":"ContainerDied","Data":"04d3e96435d5a9f855434516b0e8b515b923608601014363b5cade655605fb91"} Dec 03 13:46:52 crc kubenswrapper[5052]: I1203 13:46:52.880194 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-operators/horizon-operator-controller-manager-9b47db64f-zztn2" Dec 03 13:46:52 crc kubenswrapper[5052]: I1203 13:46:52.888792 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage18-crc" (OuterVolumeSpecName: "mysql-db") pod "2e3a3ea5-4ff6-4c9f-a093-38754c10012a" (UID: "2e3a3ea5-4ff6-4c9f-a093-38754c10012a"). InnerVolumeSpecName "local-storage18-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 13:46:52 crc kubenswrapper[5052]: I1203 13:46:52.930343 5052 scope.go:117] "RemoveContainer" containerID="29a5cf76d0275587ac82e2efe006e49acba2897de049585dae86807c72ae9434" Dec 03 13:46:52 crc kubenswrapper[5052]: I1203 13:46:52.946622 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-operators/horizon-operator-controller-manager-9b47db64f-zztn2"] Dec 03 13:46:52 crc kubenswrapper[5052]: I1203 13:46:52.951791 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-operators/horizon-operator-controller-manager-9b47db64f-zztn2"] Dec 03 13:46:52 crc kubenswrapper[5052]: I1203 13:46:52.961870 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-operators/horizon-operator-index-gkbwv"] Dec 03 13:46:52 crc kubenswrapper[5052]: I1203 13:46:52.970603 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/2e3a3ea5-4ff6-4c9f-a093-38754c10012a-config-data-default\") on node \"crc\" DevicePath \"\"" Dec 03 13:46:52 crc kubenswrapper[5052]: I1203 13:46:52.970648 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/2e3a3ea5-4ff6-4c9f-a093-38754c10012a-config-data-generated\") on node \"crc\" DevicePath \"\"" Dec 03 13:46:52 crc kubenswrapper[5052]: I1203 13:46:52.970663 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-qwt8c\" (UniqueName: \"kubernetes.io/projected/2e3a3ea5-4ff6-4c9f-a093-38754c10012a-kube-api-access-qwt8c\") on node \"crc\" DevicePath \"\"" Dec 03 13:46:52 crc kubenswrapper[5052]: I1203 13:46:52.970678 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/2e3a3ea5-4ff6-4c9f-a093-38754c10012a-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 13:46:52 crc kubenswrapper[5052]: I1203 13:46:52.970721 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage18-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage18-crc\") on node \"crc\" " Dec 03 13:46:52 crc kubenswrapper[5052]: I1203 13:46:52.970735 5052 reconciler_common.go:293] "Volume detached for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/2e3a3ea5-4ff6-4c9f-a093-38754c10012a-kolla-config\") on node \"crc\" DevicePath \"\"" Dec 03 13:46:52 crc kubenswrapper[5052]: I1203 13:46:52.971764 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-operators/horizon-operator-index-gkbwv"] Dec 03 13:46:52 crc kubenswrapper[5052]: I1203 13:46:52.974337 5052 scope.go:117] "RemoveContainer" containerID="58a8a984bcc25e97fc3cb0134e46c0fd093e75da4a13403e3a7dcd59d80b2901" Dec 03 13:46:52 crc kubenswrapper[5052]: I1203 13:46:52.977741 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["horizon-kuttl-tests/rabbitmq-server-0"] Dec 03 13:46:52 crc kubenswrapper[5052]: I1203 13:46:52.984074 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["horizon-kuttl-tests/rabbitmq-server-0"] Dec 03 13:46:52 crc kubenswrapper[5052]: I1203 13:46:52.987505 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage18-crc" (UniqueName: "kubernetes.io/local-volume/local-storage18-crc") on node "crc" Dec 03 13:46:52 crc kubenswrapper[5052]: I1203 13:46:52.991982 5052 scope.go:117] "RemoveContainer" containerID="13cf7bd27ade2aba6b22d0a54ef5527f62fa3398c10e1e0f8714b3adc70d7e83" Dec 03 13:46:53 crc kubenswrapper[5052]: I1203 13:46:53.011088 5052 scope.go:117] "RemoveContainer" containerID="4a8f84ee4a2986347ca455ced4186fc482fec4ebfa9e3c68729d3ad41525166b" Dec 03 13:46:53 crc kubenswrapper[5052]: I1203 13:46:53.031444 5052 scope.go:117] "RemoveContainer" containerID="13cf7bd27ade2aba6b22d0a54ef5527f62fa3398c10e1e0f8714b3adc70d7e83" Dec 03 13:46:53 crc kubenswrapper[5052]: E1203 13:46:53.032001 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"13cf7bd27ade2aba6b22d0a54ef5527f62fa3398c10e1e0f8714b3adc70d7e83\": container with ID starting with 13cf7bd27ade2aba6b22d0a54ef5527f62fa3398c10e1e0f8714b3adc70d7e83 not found: ID does not exist" containerID="13cf7bd27ade2aba6b22d0a54ef5527f62fa3398c10e1e0f8714b3adc70d7e83" Dec 03 13:46:53 crc kubenswrapper[5052]: I1203 13:46:53.032048 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"13cf7bd27ade2aba6b22d0a54ef5527f62fa3398c10e1e0f8714b3adc70d7e83"} err="failed to get container status \"13cf7bd27ade2aba6b22d0a54ef5527f62fa3398c10e1e0f8714b3adc70d7e83\": rpc error: code = NotFound desc = could not find container \"13cf7bd27ade2aba6b22d0a54ef5527f62fa3398c10e1e0f8714b3adc70d7e83\": container with ID starting with 13cf7bd27ade2aba6b22d0a54ef5527f62fa3398c10e1e0f8714b3adc70d7e83 not found: ID does not exist" Dec 03 13:46:53 crc kubenswrapper[5052]: I1203 13:46:53.032089 5052 scope.go:117] "RemoveContainer" containerID="4a8f84ee4a2986347ca455ced4186fc482fec4ebfa9e3c68729d3ad41525166b" Dec 03 13:46:53 crc kubenswrapper[5052]: E1203 13:46:53.032417 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4a8f84ee4a2986347ca455ced4186fc482fec4ebfa9e3c68729d3ad41525166b\": container with ID starting with 4a8f84ee4a2986347ca455ced4186fc482fec4ebfa9e3c68729d3ad41525166b not found: ID does not exist" containerID="4a8f84ee4a2986347ca455ced4186fc482fec4ebfa9e3c68729d3ad41525166b" Dec 03 13:46:53 crc kubenswrapper[5052]: I1203 13:46:53.032443 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4a8f84ee4a2986347ca455ced4186fc482fec4ebfa9e3c68729d3ad41525166b"} err="failed to get container status \"4a8f84ee4a2986347ca455ced4186fc482fec4ebfa9e3c68729d3ad41525166b\": rpc error: code = NotFound desc = could not find container \"4a8f84ee4a2986347ca455ced4186fc482fec4ebfa9e3c68729d3ad41525166b\": container with ID starting with 4a8f84ee4a2986347ca455ced4186fc482fec4ebfa9e3c68729d3ad41525166b not found: ID does not exist" Dec 03 13:46:53 crc kubenswrapper[5052]: I1203 13:46:53.032459 5052 scope.go:117] "RemoveContainer" containerID="6075c6b42637b01fc6c756047fec5638fa9a992df8271f644096237de65e87d9" Dec 03 13:46:53 crc kubenswrapper[5052]: I1203 13:46:53.072495 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage18-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage18-crc\") on node \"crc\" DevicePath \"\"" Dec 03 13:46:53 crc kubenswrapper[5052]: I1203 13:46:53.208199 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["horizon-kuttl-tests/openstack-galera-1"] Dec 03 13:46:53 crc kubenswrapper[5052]: I1203 13:46:53.213752 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["horizon-kuttl-tests/openstack-galera-1"] Dec 03 13:46:53 crc kubenswrapper[5052]: I1203 13:46:53.681868 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2e3a3ea5-4ff6-4c9f-a093-38754c10012a" path="/var/lib/kubelet/pods/2e3a3ea5-4ff6-4c9f-a093-38754c10012a/volumes" Dec 03 13:46:53 crc kubenswrapper[5052]: I1203 13:46:53.682439 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5b2f3c3a-2756-4269-99da-ba83e8097c9e" path="/var/lib/kubelet/pods/5b2f3c3a-2756-4269-99da-ba83e8097c9e/volumes" Dec 03 13:46:53 crc kubenswrapper[5052]: I1203 13:46:53.682939 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8496c1b5-ae6d-4357-9812-9d44818ee9d1" path="/var/lib/kubelet/pods/8496c1b5-ae6d-4357-9812-9d44818ee9d1/volumes" Dec 03 13:46:53 crc kubenswrapper[5052]: I1203 13:46:53.683910 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9dfea40e-ca29-4e45-85ff-fde0ca025e75" path="/var/lib/kubelet/pods/9dfea40e-ca29-4e45-85ff-fde0ca025e75/volumes" Dec 03 13:46:53 crc kubenswrapper[5052]: I1203 13:46:53.684561 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aa07e692-9f86-4c61-b674-beeb49376c0e" path="/var/lib/kubelet/pods/aa07e692-9f86-4c61-b674-beeb49376c0e/volumes" Dec 03 13:46:53 crc kubenswrapper[5052]: I1203 13:46:53.912012 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="horizon-kuttl-tests/openstack-galera-0" event={"ID":"5096dd99-719d-412e-804b-fc1452ee8b95","Type":"ContainerDied","Data":"a302cf322d3d960edabc879fda7af5d1764ecc2cb32a3acf71f26c4b65d418d4"} Dec 03 13:46:53 crc kubenswrapper[5052]: I1203 13:46:53.911915 5052 generic.go:334] "Generic (PLEG): container finished" podID="5096dd99-719d-412e-804b-fc1452ee8b95" containerID="a302cf322d3d960edabc879fda7af5d1764ecc2cb32a3acf71f26c4b65d418d4" exitCode=0 Dec 03 13:46:54 crc kubenswrapper[5052]: I1203 13:46:54.114987 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="horizon-kuttl-tests/openstack-galera-0" Dec 03 13:46:54 crc kubenswrapper[5052]: I1203 13:46:54.186377 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-98wmd\" (UniqueName: \"kubernetes.io/projected/5096dd99-719d-412e-804b-fc1452ee8b95-kube-api-access-98wmd\") pod \"5096dd99-719d-412e-804b-fc1452ee8b95\" (UID: \"5096dd99-719d-412e-804b-fc1452ee8b95\") " Dec 03 13:46:54 crc kubenswrapper[5052]: I1203 13:46:54.186446 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"mysql-db\" (UniqueName: \"kubernetes.io/local-volume/local-storage02-crc\") pod \"5096dd99-719d-412e-804b-fc1452ee8b95\" (UID: \"5096dd99-719d-412e-804b-fc1452ee8b95\") " Dec 03 13:46:54 crc kubenswrapper[5052]: I1203 13:46:54.186476 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/5096dd99-719d-412e-804b-fc1452ee8b95-config-data-default\") pod \"5096dd99-719d-412e-804b-fc1452ee8b95\" (UID: \"5096dd99-719d-412e-804b-fc1452ee8b95\") " Dec 03 13:46:54 crc kubenswrapper[5052]: I1203 13:46:54.186502 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/5096dd99-719d-412e-804b-fc1452ee8b95-operator-scripts\") pod \"5096dd99-719d-412e-804b-fc1452ee8b95\" (UID: \"5096dd99-719d-412e-804b-fc1452ee8b95\") " Dec 03 13:46:54 crc kubenswrapper[5052]: I1203 13:46:54.186655 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/5096dd99-719d-412e-804b-fc1452ee8b95-config-data-generated\") pod \"5096dd99-719d-412e-804b-fc1452ee8b95\" (UID: \"5096dd99-719d-412e-804b-fc1452ee8b95\") " Dec 03 13:46:54 crc kubenswrapper[5052]: I1203 13:46:54.186725 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/5096dd99-719d-412e-804b-fc1452ee8b95-kolla-config\") pod \"5096dd99-719d-412e-804b-fc1452ee8b95\" (UID: \"5096dd99-719d-412e-804b-fc1452ee8b95\") " Dec 03 13:46:54 crc kubenswrapper[5052]: I1203 13:46:54.187408 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5096dd99-719d-412e-804b-fc1452ee8b95-kolla-config" (OuterVolumeSpecName: "kolla-config") pod "5096dd99-719d-412e-804b-fc1452ee8b95" (UID: "5096dd99-719d-412e-804b-fc1452ee8b95"). InnerVolumeSpecName "kolla-config". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:46:54 crc kubenswrapper[5052]: I1203 13:46:54.187628 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5096dd99-719d-412e-804b-fc1452ee8b95-config-data-generated" (OuterVolumeSpecName: "config-data-generated") pod "5096dd99-719d-412e-804b-fc1452ee8b95" (UID: "5096dd99-719d-412e-804b-fc1452ee8b95"). InnerVolumeSpecName "config-data-generated". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 13:46:54 crc kubenswrapper[5052]: I1203 13:46:54.187719 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5096dd99-719d-412e-804b-fc1452ee8b95-operator-scripts" (OuterVolumeSpecName: "operator-scripts") pod "5096dd99-719d-412e-804b-fc1452ee8b95" (UID: "5096dd99-719d-412e-804b-fc1452ee8b95"). InnerVolumeSpecName "operator-scripts". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:46:54 crc kubenswrapper[5052]: I1203 13:46:54.187962 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5096dd99-719d-412e-804b-fc1452ee8b95-config-data-default" (OuterVolumeSpecName: "config-data-default") pod "5096dd99-719d-412e-804b-fc1452ee8b95" (UID: "5096dd99-719d-412e-804b-fc1452ee8b95"). InnerVolumeSpecName "config-data-default". PluginName "kubernetes.io/configmap", VolumeGidValue "" Dec 03 13:46:54 crc kubenswrapper[5052]: I1203 13:46:54.191267 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5096dd99-719d-412e-804b-fc1452ee8b95-kube-api-access-98wmd" (OuterVolumeSpecName: "kube-api-access-98wmd") pod "5096dd99-719d-412e-804b-fc1452ee8b95" (UID: "5096dd99-719d-412e-804b-fc1452ee8b95"). InnerVolumeSpecName "kube-api-access-98wmd". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:46:54 crc kubenswrapper[5052]: I1203 13:46:54.195582 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/local-volume/local-storage02-crc" (OuterVolumeSpecName: "mysql-db") pod "5096dd99-719d-412e-804b-fc1452ee8b95" (UID: "5096dd99-719d-412e-804b-fc1452ee8b95"). InnerVolumeSpecName "local-storage02-crc". PluginName "kubernetes.io/local-volume", VolumeGidValue "" Dec 03 13:46:54 crc kubenswrapper[5052]: I1203 13:46:54.288586 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-generated\" (UniqueName: \"kubernetes.io/empty-dir/5096dd99-719d-412e-804b-fc1452ee8b95-config-data-generated\") on node \"crc\" DevicePath \"\"" Dec 03 13:46:54 crc kubenswrapper[5052]: I1203 13:46:54.288619 5052 reconciler_common.go:293] "Volume detached for volume \"kolla-config\" (UniqueName: \"kubernetes.io/configmap/5096dd99-719d-412e-804b-fc1452ee8b95-kolla-config\") on node \"crc\" DevicePath \"\"" Dec 03 13:46:54 crc kubenswrapper[5052]: I1203 13:46:54.288631 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-98wmd\" (UniqueName: \"kubernetes.io/projected/5096dd99-719d-412e-804b-fc1452ee8b95-kube-api-access-98wmd\") on node \"crc\" DevicePath \"\"" Dec 03 13:46:54 crc kubenswrapper[5052]: I1203 13:46:54.288667 5052 reconciler_common.go:286] "operationExecutor.UnmountDevice started for volume \"local-storage02-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage02-crc\") on node \"crc\" " Dec 03 13:46:54 crc kubenswrapper[5052]: I1203 13:46:54.288677 5052 reconciler_common.go:293] "Volume detached for volume \"config-data-default\" (UniqueName: \"kubernetes.io/configmap/5096dd99-719d-412e-804b-fc1452ee8b95-config-data-default\") on node \"crc\" DevicePath \"\"" Dec 03 13:46:54 crc kubenswrapper[5052]: I1203 13:46:54.288689 5052 reconciler_common.go:293] "Volume detached for volume \"operator-scripts\" (UniqueName: \"kubernetes.io/configmap/5096dd99-719d-412e-804b-fc1452ee8b95-operator-scripts\") on node \"crc\" DevicePath \"\"" Dec 03 13:46:54 crc kubenswrapper[5052]: I1203 13:46:54.297701 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-operators/keystone-operator-controller-manager-768697f4bc-42dw6"] Dec 03 13:46:54 crc kubenswrapper[5052]: I1203 13:46:54.297949 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-operators/keystone-operator-controller-manager-768697f4bc-42dw6" podUID="2be98578-12f4-41ef-9fda-5db102aa4702" containerName="manager" containerID="cri-o://fab7384a38f41af6c4ea1d6ba58502b0c3b9a330de3959baccf6fda8802c67af" gracePeriod=10 Dec 03 13:46:54 crc kubenswrapper[5052]: I1203 13:46:54.302881 5052 operation_generator.go:917] UnmountDevice succeeded for volume "local-storage02-crc" (UniqueName: "kubernetes.io/local-volume/local-storage02-crc") on node "crc" Dec 03 13:46:54 crc kubenswrapper[5052]: I1203 13:46:54.389993 5052 reconciler_common.go:293] "Volume detached for volume \"local-storage02-crc\" (UniqueName: \"kubernetes.io/local-volume/local-storage02-crc\") on node \"crc\" DevicePath \"\"" Dec 03 13:46:54 crc kubenswrapper[5052]: I1203 13:46:54.584892 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-operators/keystone-operator-index-5bhhd"] Dec 03 13:46:54 crc kubenswrapper[5052]: I1203 13:46:54.585443 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-operators/keystone-operator-index-5bhhd" podUID="a38bb331-9bf3-48e9-bd95-7b5854cd4347" containerName="registry-server" containerID="cri-o://7fff1ae9198e592a285110ae6f3947b827987f4d0dd31a100ca6a586e63dd7a3" gracePeriod=30 Dec 03 13:46:54 crc kubenswrapper[5052]: I1203 13:46:54.621845 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-operators/49c083020ae5dfe237b73a6c6b807501660a323f061d7879268c43a121w2872"] Dec 03 13:46:54 crc kubenswrapper[5052]: I1203 13:46:54.635837 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-operators/49c083020ae5dfe237b73a6c6b807501660a323f061d7879268c43a121w2872"] Dec 03 13:46:54 crc kubenswrapper[5052]: I1203 13:46:54.825139 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-operators/keystone-operator-controller-manager-768697f4bc-42dw6" Dec 03 13:46:54 crc kubenswrapper[5052]: I1203 13:46:54.899753 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-st72c\" (UniqueName: \"kubernetes.io/projected/2be98578-12f4-41ef-9fda-5db102aa4702-kube-api-access-st72c\") pod \"2be98578-12f4-41ef-9fda-5db102aa4702\" (UID: \"2be98578-12f4-41ef-9fda-5db102aa4702\") " Dec 03 13:46:54 crc kubenswrapper[5052]: I1203 13:46:54.899864 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/2be98578-12f4-41ef-9fda-5db102aa4702-apiservice-cert\") pod \"2be98578-12f4-41ef-9fda-5db102aa4702\" (UID: \"2be98578-12f4-41ef-9fda-5db102aa4702\") " Dec 03 13:46:54 crc kubenswrapper[5052]: I1203 13:46:54.900287 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2be98578-12f4-41ef-9fda-5db102aa4702-webhook-cert\") pod \"2be98578-12f4-41ef-9fda-5db102aa4702\" (UID: \"2be98578-12f4-41ef-9fda-5db102aa4702\") " Dec 03 13:46:54 crc kubenswrapper[5052]: I1203 13:46:54.903538 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2be98578-12f4-41ef-9fda-5db102aa4702-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "2be98578-12f4-41ef-9fda-5db102aa4702" (UID: "2be98578-12f4-41ef-9fda-5db102aa4702"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:46:54 crc kubenswrapper[5052]: I1203 13:46:54.903583 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2be98578-12f4-41ef-9fda-5db102aa4702-apiservice-cert" (OuterVolumeSpecName: "apiservice-cert") pod "2be98578-12f4-41ef-9fda-5db102aa4702" (UID: "2be98578-12f4-41ef-9fda-5db102aa4702"). InnerVolumeSpecName "apiservice-cert". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:46:54 crc kubenswrapper[5052]: I1203 13:46:54.928127 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2be98578-12f4-41ef-9fda-5db102aa4702-kube-api-access-st72c" (OuterVolumeSpecName: "kube-api-access-st72c") pod "2be98578-12f4-41ef-9fda-5db102aa4702" (UID: "2be98578-12f4-41ef-9fda-5db102aa4702"). InnerVolumeSpecName "kube-api-access-st72c". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:46:54 crc kubenswrapper[5052]: I1203 13:46:54.935889 5052 generic.go:334] "Generic (PLEG): container finished" podID="2be98578-12f4-41ef-9fda-5db102aa4702" containerID="fab7384a38f41af6c4ea1d6ba58502b0c3b9a330de3959baccf6fda8802c67af" exitCode=0 Dec 03 13:46:54 crc kubenswrapper[5052]: I1203 13:46:54.935951 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/keystone-operator-controller-manager-768697f4bc-42dw6" event={"ID":"2be98578-12f4-41ef-9fda-5db102aa4702","Type":"ContainerDied","Data":"fab7384a38f41af6c4ea1d6ba58502b0c3b9a330de3959baccf6fda8802c67af"} Dec 03 13:46:54 crc kubenswrapper[5052]: I1203 13:46:54.935976 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/keystone-operator-controller-manager-768697f4bc-42dw6" event={"ID":"2be98578-12f4-41ef-9fda-5db102aa4702","Type":"ContainerDied","Data":"91ddede55c4fcc50b2e9a70d51ebfaf3e4da5f134cdda6c3111c8e343746c7bf"} Dec 03 13:46:54 crc kubenswrapper[5052]: I1203 13:46:54.935993 5052 scope.go:117] "RemoveContainer" containerID="fab7384a38f41af6c4ea1d6ba58502b0c3b9a330de3959baccf6fda8802c67af" Dec 03 13:46:54 crc kubenswrapper[5052]: I1203 13:46:54.936141 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-operators/keystone-operator-controller-manager-768697f4bc-42dw6" Dec 03 13:46:54 crc kubenswrapper[5052]: I1203 13:46:54.951280 5052 generic.go:334] "Generic (PLEG): container finished" podID="a38bb331-9bf3-48e9-bd95-7b5854cd4347" containerID="7fff1ae9198e592a285110ae6f3947b827987f4d0dd31a100ca6a586e63dd7a3" exitCode=0 Dec 03 13:46:54 crc kubenswrapper[5052]: I1203 13:46:54.951346 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/keystone-operator-index-5bhhd" event={"ID":"a38bb331-9bf3-48e9-bd95-7b5854cd4347","Type":"ContainerDied","Data":"7fff1ae9198e592a285110ae6f3947b827987f4d0dd31a100ca6a586e63dd7a3"} Dec 03 13:46:54 crc kubenswrapper[5052]: I1203 13:46:54.954210 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="horizon-kuttl-tests/openstack-galera-0" event={"ID":"5096dd99-719d-412e-804b-fc1452ee8b95","Type":"ContainerDied","Data":"a44911a627c7c251eb81c830124d8c2d245e843c16fad9c055b0d77736b3f2b5"} Dec 03 13:46:54 crc kubenswrapper[5052]: I1203 13:46:54.954317 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="horizon-kuttl-tests/openstack-galera-0" Dec 03 13:46:54 crc kubenswrapper[5052]: I1203 13:46:54.974252 5052 scope.go:117] "RemoveContainer" containerID="fab7384a38f41af6c4ea1d6ba58502b0c3b9a330de3959baccf6fda8802c67af" Dec 03 13:46:54 crc kubenswrapper[5052]: E1203 13:46:54.975008 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fab7384a38f41af6c4ea1d6ba58502b0c3b9a330de3959baccf6fda8802c67af\": container with ID starting with fab7384a38f41af6c4ea1d6ba58502b0c3b9a330de3959baccf6fda8802c67af not found: ID does not exist" containerID="fab7384a38f41af6c4ea1d6ba58502b0c3b9a330de3959baccf6fda8802c67af" Dec 03 13:46:54 crc kubenswrapper[5052]: I1203 13:46:54.975049 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fab7384a38f41af6c4ea1d6ba58502b0c3b9a330de3959baccf6fda8802c67af"} err="failed to get container status \"fab7384a38f41af6c4ea1d6ba58502b0c3b9a330de3959baccf6fda8802c67af\": rpc error: code = NotFound desc = could not find container \"fab7384a38f41af6c4ea1d6ba58502b0c3b9a330de3959baccf6fda8802c67af\": container with ID starting with fab7384a38f41af6c4ea1d6ba58502b0c3b9a330de3959baccf6fda8802c67af not found: ID does not exist" Dec 03 13:46:54 crc kubenswrapper[5052]: I1203 13:46:54.975071 5052 scope.go:117] "RemoveContainer" containerID="a302cf322d3d960edabc879fda7af5d1764ecc2cb32a3acf71f26c4b65d418d4" Dec 03 13:46:54 crc kubenswrapper[5052]: I1203 13:46:54.991070 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-operators/keystone-operator-controller-manager-768697f4bc-42dw6"] Dec 03 13:46:55 crc kubenswrapper[5052]: I1203 13:46:55.004933 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-operators/keystone-operator-controller-manager-768697f4bc-42dw6"] Dec 03 13:46:55 crc kubenswrapper[5052]: I1203 13:46:55.006767 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-st72c\" (UniqueName: \"kubernetes.io/projected/2be98578-12f4-41ef-9fda-5db102aa4702-kube-api-access-st72c\") on node \"crc\" DevicePath \"\"" Dec 03 13:46:55 crc kubenswrapper[5052]: I1203 13:46:55.006883 5052 reconciler_common.go:293] "Volume detached for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/2be98578-12f4-41ef-9fda-5db102aa4702-apiservice-cert\") on node \"crc\" DevicePath \"\"" Dec 03 13:46:55 crc kubenswrapper[5052]: I1203 13:46:55.006959 5052 reconciler_common.go:293] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2be98578-12f4-41ef-9fda-5db102aa4702-webhook-cert\") on node \"crc\" DevicePath \"\"" Dec 03 13:46:55 crc kubenswrapper[5052]: I1203 13:46:55.009311 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["horizon-kuttl-tests/openstack-galera-0"] Dec 03 13:46:55 crc kubenswrapper[5052]: I1203 13:46:55.009487 5052 scope.go:117] "RemoveContainer" containerID="09dbf2188eb534590a55f1a626c0cdf554c64b93ef01ea9824fcf16cd298c2c7" Dec 03 13:46:55 crc kubenswrapper[5052]: I1203 13:46:55.009846 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-operators/keystone-operator-index-5bhhd" Dec 03 13:46:55 crc kubenswrapper[5052]: I1203 13:46:55.015277 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["horizon-kuttl-tests/openstack-galera-0"] Dec 03 13:46:55 crc kubenswrapper[5052]: I1203 13:46:55.108013 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9rhjt\" (UniqueName: \"kubernetes.io/projected/a38bb331-9bf3-48e9-bd95-7b5854cd4347-kube-api-access-9rhjt\") pod \"a38bb331-9bf3-48e9-bd95-7b5854cd4347\" (UID: \"a38bb331-9bf3-48e9-bd95-7b5854cd4347\") " Dec 03 13:46:55 crc kubenswrapper[5052]: I1203 13:46:55.112380 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a38bb331-9bf3-48e9-bd95-7b5854cd4347-kube-api-access-9rhjt" (OuterVolumeSpecName: "kube-api-access-9rhjt") pod "a38bb331-9bf3-48e9-bd95-7b5854cd4347" (UID: "a38bb331-9bf3-48e9-bd95-7b5854cd4347"). InnerVolumeSpecName "kube-api-access-9rhjt". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:46:55 crc kubenswrapper[5052]: I1203 13:46:55.209838 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-9rhjt\" (UniqueName: \"kubernetes.io/projected/a38bb331-9bf3-48e9-bd95-7b5854cd4347-kube-api-access-9rhjt\") on node \"crc\" DevicePath \"\"" Dec 03 13:46:55 crc kubenswrapper[5052]: I1203 13:46:55.681639 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2be98578-12f4-41ef-9fda-5db102aa4702" path="/var/lib/kubelet/pods/2be98578-12f4-41ef-9fda-5db102aa4702/volumes" Dec 03 13:46:55 crc kubenswrapper[5052]: I1203 13:46:55.682325 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5096dd99-719d-412e-804b-fc1452ee8b95" path="/var/lib/kubelet/pods/5096dd99-719d-412e-804b-fc1452ee8b95/volumes" Dec 03 13:46:55 crc kubenswrapper[5052]: I1203 13:46:55.683146 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ec7ca777-d2e5-4b89-91ad-5084b2716811" path="/var/lib/kubelet/pods/ec7ca777-d2e5-4b89-91ad-5084b2716811/volumes" Dec 03 13:46:55 crc kubenswrapper[5052]: I1203 13:46:55.962748 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/keystone-operator-index-5bhhd" event={"ID":"a38bb331-9bf3-48e9-bd95-7b5854cd4347","Type":"ContainerDied","Data":"d7c08e0b52997e0a5f907a57b87153c5f78b1f164f5e54ef61ca26f31ef11753"} Dec 03 13:46:55 crc kubenswrapper[5052]: I1203 13:46:55.962815 5052 scope.go:117] "RemoveContainer" containerID="7fff1ae9198e592a285110ae6f3947b827987f4d0dd31a100ca6a586e63dd7a3" Dec 03 13:46:55 crc kubenswrapper[5052]: I1203 13:46:55.962773 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-operators/keystone-operator-index-5bhhd" Dec 03 13:46:55 crc kubenswrapper[5052]: I1203 13:46:55.982622 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-operators/keystone-operator-index-5bhhd"] Dec 03 13:46:55 crc kubenswrapper[5052]: I1203 13:46:55.987912 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-operators/keystone-operator-index-5bhhd"] Dec 03 13:46:57 crc kubenswrapper[5052]: I1203 13:46:57.682244 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a38bb331-9bf3-48e9-bd95-7b5854cd4347" path="/var/lib/kubelet/pods/a38bb331-9bf3-48e9-bd95-7b5854cd4347/volumes" Dec 03 13:46:57 crc kubenswrapper[5052]: I1203 13:46:57.709261 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-operators/rabbitmq-cluster-operator-779fc9694b-7kdqx"] Dec 03 13:46:57 crc kubenswrapper[5052]: I1203 13:46:57.709546 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-operators/rabbitmq-cluster-operator-779fc9694b-7kdqx" podUID="202fe9d9-dcdf-43ae-bfc4-4efc7f9e7dba" containerName="operator" containerID="cri-o://e0e745f7b4d4193fc358279485d9af00093d854c8e76c0c9fccc12f7c2007e8d" gracePeriod=10 Dec 03 13:46:58 crc kubenswrapper[5052]: I1203 13:46:58.070016 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-operators/rabbitmq-cluster-operator-index-spsf8"] Dec 03 13:46:58 crc kubenswrapper[5052]: I1203 13:46:58.070585 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-operators/rabbitmq-cluster-operator-index-spsf8" podUID="b70bf54e-974c-4721-8384-2c6bceaee23f" containerName="registry-server" containerID="cri-o://1fbf3c65821d10127aefc7f2256bb0068a8e464cb01a82c3b91a365cc1011a71" gracePeriod=30 Dec 03 13:46:58 crc kubenswrapper[5052]: I1203 13:46:58.101386 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-operators/9704761d240e56fb98655ffd81084895b33a73ec711f4dcdef0450e590nfq89"] Dec 03 13:46:58 crc kubenswrapper[5052]: I1203 13:46:58.109270 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-operators/9704761d240e56fb98655ffd81084895b33a73ec711f4dcdef0450e590nfq89"] Dec 03 13:46:58 crc kubenswrapper[5052]: I1203 13:46:58.699705 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-operators/rabbitmq-cluster-operator-779fc9694b-7kdqx" Dec 03 13:46:58 crc kubenswrapper[5052]: I1203 13:46:58.871954 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k9gzx\" (UniqueName: \"kubernetes.io/projected/202fe9d9-dcdf-43ae-bfc4-4efc7f9e7dba-kube-api-access-k9gzx\") pod \"202fe9d9-dcdf-43ae-bfc4-4efc7f9e7dba\" (UID: \"202fe9d9-dcdf-43ae-bfc4-4efc7f9e7dba\") " Dec 03 13:46:58 crc kubenswrapper[5052]: I1203 13:46:58.877722 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/202fe9d9-dcdf-43ae-bfc4-4efc7f9e7dba-kube-api-access-k9gzx" (OuterVolumeSpecName: "kube-api-access-k9gzx") pod "202fe9d9-dcdf-43ae-bfc4-4efc7f9e7dba" (UID: "202fe9d9-dcdf-43ae-bfc4-4efc7f9e7dba"). InnerVolumeSpecName "kube-api-access-k9gzx". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:46:58 crc kubenswrapper[5052]: I1203 13:46:58.974593 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-k9gzx\" (UniqueName: \"kubernetes.io/projected/202fe9d9-dcdf-43ae-bfc4-4efc7f9e7dba-kube-api-access-k9gzx\") on node \"crc\" DevicePath \"\"" Dec 03 13:46:58 crc kubenswrapper[5052]: I1203 13:46:58.984816 5052 generic.go:334] "Generic (PLEG): container finished" podID="b70bf54e-974c-4721-8384-2c6bceaee23f" containerID="1fbf3c65821d10127aefc7f2256bb0068a8e464cb01a82c3b91a365cc1011a71" exitCode=0 Dec 03 13:46:58 crc kubenswrapper[5052]: I1203 13:46:58.984904 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/rabbitmq-cluster-operator-index-spsf8" event={"ID":"b70bf54e-974c-4721-8384-2c6bceaee23f","Type":"ContainerDied","Data":"1fbf3c65821d10127aefc7f2256bb0068a8e464cb01a82c3b91a365cc1011a71"} Dec 03 13:46:58 crc kubenswrapper[5052]: I1203 13:46:58.986473 5052 generic.go:334] "Generic (PLEG): container finished" podID="202fe9d9-dcdf-43ae-bfc4-4efc7f9e7dba" containerID="e0e745f7b4d4193fc358279485d9af00093d854c8e76c0c9fccc12f7c2007e8d" exitCode=0 Dec 03 13:46:58 crc kubenswrapper[5052]: I1203 13:46:58.986529 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-operators/rabbitmq-cluster-operator-779fc9694b-7kdqx" Dec 03 13:46:58 crc kubenswrapper[5052]: I1203 13:46:58.986525 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/rabbitmq-cluster-operator-779fc9694b-7kdqx" event={"ID":"202fe9d9-dcdf-43ae-bfc4-4efc7f9e7dba","Type":"ContainerDied","Data":"e0e745f7b4d4193fc358279485d9af00093d854c8e76c0c9fccc12f7c2007e8d"} Dec 03 13:46:58 crc kubenswrapper[5052]: I1203 13:46:58.986574 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/rabbitmq-cluster-operator-779fc9694b-7kdqx" event={"ID":"202fe9d9-dcdf-43ae-bfc4-4efc7f9e7dba","Type":"ContainerDied","Data":"edbbebf2ec7e0cba6a6ca54c9e584a4ccf23e889d590cd2c044a08111204f856"} Dec 03 13:46:58 crc kubenswrapper[5052]: I1203 13:46:58.986596 5052 scope.go:117] "RemoveContainer" containerID="e0e745f7b4d4193fc358279485d9af00093d854c8e76c0c9fccc12f7c2007e8d" Dec 03 13:46:59 crc kubenswrapper[5052]: I1203 13:46:59.006713 5052 scope.go:117] "RemoveContainer" containerID="e0e745f7b4d4193fc358279485d9af00093d854c8e76c0c9fccc12f7c2007e8d" Dec 03 13:46:59 crc kubenswrapper[5052]: E1203 13:46:59.007124 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e0e745f7b4d4193fc358279485d9af00093d854c8e76c0c9fccc12f7c2007e8d\": container with ID starting with e0e745f7b4d4193fc358279485d9af00093d854c8e76c0c9fccc12f7c2007e8d not found: ID does not exist" containerID="e0e745f7b4d4193fc358279485d9af00093d854c8e76c0c9fccc12f7c2007e8d" Dec 03 13:46:59 crc kubenswrapper[5052]: I1203 13:46:59.007160 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e0e745f7b4d4193fc358279485d9af00093d854c8e76c0c9fccc12f7c2007e8d"} err="failed to get container status \"e0e745f7b4d4193fc358279485d9af00093d854c8e76c0c9fccc12f7c2007e8d\": rpc error: code = NotFound desc = could not find container \"e0e745f7b4d4193fc358279485d9af00093d854c8e76c0c9fccc12f7c2007e8d\": container with ID starting with e0e745f7b4d4193fc358279485d9af00093d854c8e76c0c9fccc12f7c2007e8d not found: ID does not exist" Dec 03 13:46:59 crc kubenswrapper[5052]: I1203 13:46:59.019744 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-operators/rabbitmq-cluster-operator-779fc9694b-7kdqx"] Dec 03 13:46:59 crc kubenswrapper[5052]: I1203 13:46:59.025315 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-operators/rabbitmq-cluster-operator-779fc9694b-7kdqx"] Dec 03 13:46:59 crc kubenswrapper[5052]: I1203 13:46:59.331873 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-operators/rabbitmq-cluster-operator-index-spsf8" Dec 03 13:46:59 crc kubenswrapper[5052]: I1203 13:46:59.499039 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qgjq4\" (UniqueName: \"kubernetes.io/projected/b70bf54e-974c-4721-8384-2c6bceaee23f-kube-api-access-qgjq4\") pod \"b70bf54e-974c-4721-8384-2c6bceaee23f\" (UID: \"b70bf54e-974c-4721-8384-2c6bceaee23f\") " Dec 03 13:46:59 crc kubenswrapper[5052]: I1203 13:46:59.504723 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b70bf54e-974c-4721-8384-2c6bceaee23f-kube-api-access-qgjq4" (OuterVolumeSpecName: "kube-api-access-qgjq4") pod "b70bf54e-974c-4721-8384-2c6bceaee23f" (UID: "b70bf54e-974c-4721-8384-2c6bceaee23f"). InnerVolumeSpecName "kube-api-access-qgjq4". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:46:59 crc kubenswrapper[5052]: I1203 13:46:59.601431 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-qgjq4\" (UniqueName: \"kubernetes.io/projected/b70bf54e-974c-4721-8384-2c6bceaee23f-kube-api-access-qgjq4\") on node \"crc\" DevicePath \"\"" Dec 03 13:46:59 crc kubenswrapper[5052]: I1203 13:46:59.681308 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="202fe9d9-dcdf-43ae-bfc4-4efc7f9e7dba" path="/var/lib/kubelet/pods/202fe9d9-dcdf-43ae-bfc4-4efc7f9e7dba/volumes" Dec 03 13:46:59 crc kubenswrapper[5052]: I1203 13:46:59.681866 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="845f4c5b-f54f-4cf4-8b78-17c73bdccfdc" path="/var/lib/kubelet/pods/845f4c5b-f54f-4cf4-8b78-17c73bdccfdc/volumes" Dec 03 13:46:59 crc kubenswrapper[5052]: I1203 13:46:59.994432 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/rabbitmq-cluster-operator-index-spsf8" event={"ID":"b70bf54e-974c-4721-8384-2c6bceaee23f","Type":"ContainerDied","Data":"0afac7c1b1cfca5ac7751926324f2fa323ed32cb3638b4b6bd3bee2d0e23f7e6"} Dec 03 13:46:59 crc kubenswrapper[5052]: I1203 13:46:59.994473 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-operators/rabbitmq-cluster-operator-index-spsf8" Dec 03 13:46:59 crc kubenswrapper[5052]: I1203 13:46:59.994498 5052 scope.go:117] "RemoveContainer" containerID="1fbf3c65821d10127aefc7f2256bb0068a8e464cb01a82c3b91a365cc1011a71" Dec 03 13:47:00 crc kubenswrapper[5052]: I1203 13:47:00.018755 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-operators/rabbitmq-cluster-operator-index-spsf8"] Dec 03 13:47:00 crc kubenswrapper[5052]: I1203 13:47:00.025378 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-operators/rabbitmq-cluster-operator-index-spsf8"] Dec 03 13:47:01 crc kubenswrapper[5052]: I1203 13:47:01.681849 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b70bf54e-974c-4721-8384-2c6bceaee23f" path="/var/lib/kubelet/pods/b70bf54e-974c-4721-8384-2c6bceaee23f/volumes" Dec 03 13:47:02 crc kubenswrapper[5052]: I1203 13:47:02.097607 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-operators/infra-operator-controller-manager-598d599d5d-bp6rp"] Dec 03 13:47:02 crc kubenswrapper[5052]: I1203 13:47:02.097864 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-operators/infra-operator-controller-manager-598d599d5d-bp6rp" podUID="afe4ff83-55b7-48a0-902c-4162f0f68ac5" containerName="manager" containerID="cri-o://64f7ed9421435a11b729eee8e4a6e3db7a76f095f7066f7d1d2ba30a8b6da64f" gracePeriod=10 Dec 03 13:47:02 crc kubenswrapper[5052]: I1203 13:47:02.097951 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-operators/infra-operator-controller-manager-598d599d5d-bp6rp" podUID="afe4ff83-55b7-48a0-902c-4162f0f68ac5" containerName="kube-rbac-proxy" containerID="cri-o://54216bbfe5e666b89a444f7877f5ea60ca671124686b8f915233b4d21e590480" gracePeriod=10 Dec 03 13:47:02 crc kubenswrapper[5052]: I1203 13:47:02.410546 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-operators/infra-operator-index-5rfr5"] Dec 03 13:47:02 crc kubenswrapper[5052]: I1203 13:47:02.411046 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-operators/infra-operator-index-5rfr5" podUID="a8c2a4d6-1e91-40de-9edc-60b79b888d1f" containerName="registry-server" containerID="cri-o://bbe020afe41be5770ac45a3d0ff5e4a49b3a6871dca3b0cc331acc3c7df428b9" gracePeriod=30 Dec 03 13:47:02 crc kubenswrapper[5052]: I1203 13:47:02.435755 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-operators/5d473c3169f40b179d14921c90af2c8546b7b757fe551b7dba7d903f5dr8sfn"] Dec 03 13:47:02 crc kubenswrapper[5052]: I1203 13:47:02.447209 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-operators/5d473c3169f40b179d14921c90af2c8546b7b757fe551b7dba7d903f5dr8sfn"] Dec 03 13:47:02 crc kubenswrapper[5052]: I1203 13:47:02.695910 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-operators/infra-operator-controller-manager-598d599d5d-bp6rp" Dec 03 13:47:02 crc kubenswrapper[5052]: I1203 13:47:02.854319 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jvcdg\" (UniqueName: \"kubernetes.io/projected/afe4ff83-55b7-48a0-902c-4162f0f68ac5-kube-api-access-jvcdg\") pod \"afe4ff83-55b7-48a0-902c-4162f0f68ac5\" (UID: \"afe4ff83-55b7-48a0-902c-4162f0f68ac5\") " Dec 03 13:47:02 crc kubenswrapper[5052]: I1203 13:47:02.855648 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/afe4ff83-55b7-48a0-902c-4162f0f68ac5-apiservice-cert\") pod \"afe4ff83-55b7-48a0-902c-4162f0f68ac5\" (UID: \"afe4ff83-55b7-48a0-902c-4162f0f68ac5\") " Dec 03 13:47:02 crc kubenswrapper[5052]: I1203 13:47:02.855734 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/afe4ff83-55b7-48a0-902c-4162f0f68ac5-webhook-cert\") pod \"afe4ff83-55b7-48a0-902c-4162f0f68ac5\" (UID: \"afe4ff83-55b7-48a0-902c-4162f0f68ac5\") " Dec 03 13:47:02 crc kubenswrapper[5052]: I1203 13:47:02.860621 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/afe4ff83-55b7-48a0-902c-4162f0f68ac5-apiservice-cert" (OuterVolumeSpecName: "apiservice-cert") pod "afe4ff83-55b7-48a0-902c-4162f0f68ac5" (UID: "afe4ff83-55b7-48a0-902c-4162f0f68ac5"). InnerVolumeSpecName "apiservice-cert". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:47:02 crc kubenswrapper[5052]: I1203 13:47:02.860643 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/afe4ff83-55b7-48a0-902c-4162f0f68ac5-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "afe4ff83-55b7-48a0-902c-4162f0f68ac5" (UID: "afe4ff83-55b7-48a0-902c-4162f0f68ac5"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:47:02 crc kubenswrapper[5052]: I1203 13:47:02.862725 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/afe4ff83-55b7-48a0-902c-4162f0f68ac5-kube-api-access-jvcdg" (OuterVolumeSpecName: "kube-api-access-jvcdg") pod "afe4ff83-55b7-48a0-902c-4162f0f68ac5" (UID: "afe4ff83-55b7-48a0-902c-4162f0f68ac5"). InnerVolumeSpecName "kube-api-access-jvcdg". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:47:02 crc kubenswrapper[5052]: I1203 13:47:02.957837 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-jvcdg\" (UniqueName: \"kubernetes.io/projected/afe4ff83-55b7-48a0-902c-4162f0f68ac5-kube-api-access-jvcdg\") on node \"crc\" DevicePath \"\"" Dec 03 13:47:02 crc kubenswrapper[5052]: I1203 13:47:02.957877 5052 reconciler_common.go:293] "Volume detached for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/afe4ff83-55b7-48a0-902c-4162f0f68ac5-apiservice-cert\") on node \"crc\" DevicePath \"\"" Dec 03 13:47:02 crc kubenswrapper[5052]: I1203 13:47:02.957888 5052 reconciler_common.go:293] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/afe4ff83-55b7-48a0-902c-4162f0f68ac5-webhook-cert\") on node \"crc\" DevicePath \"\"" Dec 03 13:47:03 crc kubenswrapper[5052]: I1203 13:47:03.024301 5052 generic.go:334] "Generic (PLEG): container finished" podID="afe4ff83-55b7-48a0-902c-4162f0f68ac5" containerID="54216bbfe5e666b89a444f7877f5ea60ca671124686b8f915233b4d21e590480" exitCode=0 Dec 03 13:47:03 crc kubenswrapper[5052]: I1203 13:47:03.024671 5052 generic.go:334] "Generic (PLEG): container finished" podID="afe4ff83-55b7-48a0-902c-4162f0f68ac5" containerID="64f7ed9421435a11b729eee8e4a6e3db7a76f095f7066f7d1d2ba30a8b6da64f" exitCode=0 Dec 03 13:47:03 crc kubenswrapper[5052]: I1203 13:47:03.024372 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-operators/infra-operator-controller-manager-598d599d5d-bp6rp" Dec 03 13:47:03 crc kubenswrapper[5052]: I1203 13:47:03.024419 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/infra-operator-controller-manager-598d599d5d-bp6rp" event={"ID":"afe4ff83-55b7-48a0-902c-4162f0f68ac5","Type":"ContainerDied","Data":"54216bbfe5e666b89a444f7877f5ea60ca671124686b8f915233b4d21e590480"} Dec 03 13:47:03 crc kubenswrapper[5052]: I1203 13:47:03.025943 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/infra-operator-controller-manager-598d599d5d-bp6rp" event={"ID":"afe4ff83-55b7-48a0-902c-4162f0f68ac5","Type":"ContainerDied","Data":"64f7ed9421435a11b729eee8e4a6e3db7a76f095f7066f7d1d2ba30a8b6da64f"} Dec 03 13:47:03 crc kubenswrapper[5052]: I1203 13:47:03.025968 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/infra-operator-controller-manager-598d599d5d-bp6rp" event={"ID":"afe4ff83-55b7-48a0-902c-4162f0f68ac5","Type":"ContainerDied","Data":"134000982b2104e950f30f602bb879142ea99e91d212ec4378b7858698bcc651"} Dec 03 13:47:03 crc kubenswrapper[5052]: I1203 13:47:03.025988 5052 scope.go:117] "RemoveContainer" containerID="54216bbfe5e666b89a444f7877f5ea60ca671124686b8f915233b4d21e590480" Dec 03 13:47:03 crc kubenswrapper[5052]: I1203 13:47:03.032404 5052 generic.go:334] "Generic (PLEG): container finished" podID="a8c2a4d6-1e91-40de-9edc-60b79b888d1f" containerID="bbe020afe41be5770ac45a3d0ff5e4a49b3a6871dca3b0cc331acc3c7df428b9" exitCode=0 Dec 03 13:47:03 crc kubenswrapper[5052]: I1203 13:47:03.032441 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/infra-operator-index-5rfr5" event={"ID":"a8c2a4d6-1e91-40de-9edc-60b79b888d1f","Type":"ContainerDied","Data":"bbe020afe41be5770ac45a3d0ff5e4a49b3a6871dca3b0cc331acc3c7df428b9"} Dec 03 13:47:03 crc kubenswrapper[5052]: I1203 13:47:03.058205 5052 scope.go:117] "RemoveContainer" containerID="64f7ed9421435a11b729eee8e4a6e3db7a76f095f7066f7d1d2ba30a8b6da64f" Dec 03 13:47:03 crc kubenswrapper[5052]: I1203 13:47:03.062154 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-operators/infra-operator-controller-manager-598d599d5d-bp6rp"] Dec 03 13:47:03 crc kubenswrapper[5052]: I1203 13:47:03.071849 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-operators/infra-operator-controller-manager-598d599d5d-bp6rp"] Dec 03 13:47:03 crc kubenswrapper[5052]: I1203 13:47:03.094036 5052 scope.go:117] "RemoveContainer" containerID="54216bbfe5e666b89a444f7877f5ea60ca671124686b8f915233b4d21e590480" Dec 03 13:47:03 crc kubenswrapper[5052]: E1203 13:47:03.094389 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"54216bbfe5e666b89a444f7877f5ea60ca671124686b8f915233b4d21e590480\": container with ID starting with 54216bbfe5e666b89a444f7877f5ea60ca671124686b8f915233b4d21e590480 not found: ID does not exist" containerID="54216bbfe5e666b89a444f7877f5ea60ca671124686b8f915233b4d21e590480" Dec 03 13:47:03 crc kubenswrapper[5052]: I1203 13:47:03.094547 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"54216bbfe5e666b89a444f7877f5ea60ca671124686b8f915233b4d21e590480"} err="failed to get container status \"54216bbfe5e666b89a444f7877f5ea60ca671124686b8f915233b4d21e590480\": rpc error: code = NotFound desc = could not find container \"54216bbfe5e666b89a444f7877f5ea60ca671124686b8f915233b4d21e590480\": container with ID starting with 54216bbfe5e666b89a444f7877f5ea60ca671124686b8f915233b4d21e590480 not found: ID does not exist" Dec 03 13:47:03 crc kubenswrapper[5052]: I1203 13:47:03.094635 5052 scope.go:117] "RemoveContainer" containerID="64f7ed9421435a11b729eee8e4a6e3db7a76f095f7066f7d1d2ba30a8b6da64f" Dec 03 13:47:03 crc kubenswrapper[5052]: E1203 13:47:03.105400 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"64f7ed9421435a11b729eee8e4a6e3db7a76f095f7066f7d1d2ba30a8b6da64f\": container with ID starting with 64f7ed9421435a11b729eee8e4a6e3db7a76f095f7066f7d1d2ba30a8b6da64f not found: ID does not exist" containerID="64f7ed9421435a11b729eee8e4a6e3db7a76f095f7066f7d1d2ba30a8b6da64f" Dec 03 13:47:03 crc kubenswrapper[5052]: I1203 13:47:03.105451 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"64f7ed9421435a11b729eee8e4a6e3db7a76f095f7066f7d1d2ba30a8b6da64f"} err="failed to get container status \"64f7ed9421435a11b729eee8e4a6e3db7a76f095f7066f7d1d2ba30a8b6da64f\": rpc error: code = NotFound desc = could not find container \"64f7ed9421435a11b729eee8e4a6e3db7a76f095f7066f7d1d2ba30a8b6da64f\": container with ID starting with 64f7ed9421435a11b729eee8e4a6e3db7a76f095f7066f7d1d2ba30a8b6da64f not found: ID does not exist" Dec 03 13:47:03 crc kubenswrapper[5052]: I1203 13:47:03.105479 5052 scope.go:117] "RemoveContainer" containerID="54216bbfe5e666b89a444f7877f5ea60ca671124686b8f915233b4d21e590480" Dec 03 13:47:03 crc kubenswrapper[5052]: I1203 13:47:03.106161 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"54216bbfe5e666b89a444f7877f5ea60ca671124686b8f915233b4d21e590480"} err="failed to get container status \"54216bbfe5e666b89a444f7877f5ea60ca671124686b8f915233b4d21e590480\": rpc error: code = NotFound desc = could not find container \"54216bbfe5e666b89a444f7877f5ea60ca671124686b8f915233b4d21e590480\": container with ID starting with 54216bbfe5e666b89a444f7877f5ea60ca671124686b8f915233b4d21e590480 not found: ID does not exist" Dec 03 13:47:03 crc kubenswrapper[5052]: I1203 13:47:03.106216 5052 scope.go:117] "RemoveContainer" containerID="64f7ed9421435a11b729eee8e4a6e3db7a76f095f7066f7d1d2ba30a8b6da64f" Dec 03 13:47:03 crc kubenswrapper[5052]: I1203 13:47:03.106564 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"64f7ed9421435a11b729eee8e4a6e3db7a76f095f7066f7d1d2ba30a8b6da64f"} err="failed to get container status \"64f7ed9421435a11b729eee8e4a6e3db7a76f095f7066f7d1d2ba30a8b6da64f\": rpc error: code = NotFound desc = could not find container \"64f7ed9421435a11b729eee8e4a6e3db7a76f095f7066f7d1d2ba30a8b6da64f\": container with ID starting with 64f7ed9421435a11b729eee8e4a6e3db7a76f095f7066f7d1d2ba30a8b6da64f not found: ID does not exist" Dec 03 13:47:03 crc kubenswrapper[5052]: I1203 13:47:03.193486 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-operators/infra-operator-index-5rfr5" Dec 03 13:47:03 crc kubenswrapper[5052]: I1203 13:47:03.364560 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qxwhk\" (UniqueName: \"kubernetes.io/projected/a8c2a4d6-1e91-40de-9edc-60b79b888d1f-kube-api-access-qxwhk\") pod \"a8c2a4d6-1e91-40de-9edc-60b79b888d1f\" (UID: \"a8c2a4d6-1e91-40de-9edc-60b79b888d1f\") " Dec 03 13:47:03 crc kubenswrapper[5052]: I1203 13:47:03.368207 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a8c2a4d6-1e91-40de-9edc-60b79b888d1f-kube-api-access-qxwhk" (OuterVolumeSpecName: "kube-api-access-qxwhk") pod "a8c2a4d6-1e91-40de-9edc-60b79b888d1f" (UID: "a8c2a4d6-1e91-40de-9edc-60b79b888d1f"). InnerVolumeSpecName "kube-api-access-qxwhk". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:47:03 crc kubenswrapper[5052]: I1203 13:47:03.466952 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-qxwhk\" (UniqueName: \"kubernetes.io/projected/a8c2a4d6-1e91-40de-9edc-60b79b888d1f-kube-api-access-qxwhk\") on node \"crc\" DevicePath \"\"" Dec 03 13:47:03 crc kubenswrapper[5052]: I1203 13:47:03.662732 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-operators/mariadb-operator-controller-manager-5bdcd87d9d-b7mh5"] Dec 03 13:47:03 crc kubenswrapper[5052]: I1203 13:47:03.663019 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-operators/mariadb-operator-controller-manager-5bdcd87d9d-b7mh5" podUID="8400a794-64c3-4172-93fa-ceb4acef604d" containerName="manager" containerID="cri-o://f09e2aed5955802de7fb6d5d419275a5a96d96c3647698e76456aabab1a980c9" gracePeriod=10 Dec 03 13:47:03 crc kubenswrapper[5052]: I1203 13:47:03.682776 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2aaa3b0a-cd10-42b4-8c0d-fc9317d22547" path="/var/lib/kubelet/pods/2aaa3b0a-cd10-42b4-8c0d-fc9317d22547/volumes" Dec 03 13:47:03 crc kubenswrapper[5052]: I1203 13:47:03.683683 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="afe4ff83-55b7-48a0-902c-4162f0f68ac5" path="/var/lib/kubelet/pods/afe4ff83-55b7-48a0-902c-4162f0f68ac5/volumes" Dec 03 13:47:04 crc kubenswrapper[5052]: I1203 13:47:04.053769 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/infra-operator-index-5rfr5" event={"ID":"a8c2a4d6-1e91-40de-9edc-60b79b888d1f","Type":"ContainerDied","Data":"02e30a400a6bcd15b0cc33f7903897d8c54556d43d11a15a3b0b3a7f9f4db5f9"} Dec 03 13:47:04 crc kubenswrapper[5052]: I1203 13:47:04.053818 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-operators/infra-operator-index-5rfr5" Dec 03 13:47:04 crc kubenswrapper[5052]: I1203 13:47:04.053853 5052 scope.go:117] "RemoveContainer" containerID="bbe020afe41be5770ac45a3d0ff5e4a49b3a6871dca3b0cc331acc3c7df428b9" Dec 03 13:47:04 crc kubenswrapper[5052]: I1203 13:47:04.077826 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-operators/infra-operator-index-5rfr5"] Dec 03 13:47:04 crc kubenswrapper[5052]: I1203 13:47:04.084929 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-operators/infra-operator-index-5rfr5"] Dec 03 13:47:04 crc kubenswrapper[5052]: I1203 13:47:04.099645 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-operators/mariadb-operator-index-p2x5s"] Dec 03 13:47:04 crc kubenswrapper[5052]: I1203 13:47:04.099903 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openstack-operators/mariadb-operator-index-p2x5s" podUID="f82e13e4-6fd4-4bf8-8d78-c00e7b5a402e" containerName="registry-server" containerID="cri-o://29ad16f9f611514fc76240ac2c1cfa6a3a5cb4213b29155759c964ed5238361c" gracePeriod=30 Dec 03 13:47:04 crc kubenswrapper[5052]: I1203 13:47:04.131024 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-operators/55a3886cc1ed42812df4eab61c7a6033dc924d195539e8545c8f175f6194wgk"] Dec 03 13:47:04 crc kubenswrapper[5052]: I1203 13:47:04.135572 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-operators/55a3886cc1ed42812df4eab61c7a6033dc924d195539e8545c8f175f6194wgk"] Dec 03 13:47:04 crc kubenswrapper[5052]: I1203 13:47:04.768006 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-operators/mariadb-operator-index-p2x5s" Dec 03 13:47:04 crc kubenswrapper[5052]: I1203 13:47:04.772737 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-operators/mariadb-operator-controller-manager-5bdcd87d9d-b7mh5" Dec 03 13:47:04 crc kubenswrapper[5052]: I1203 13:47:04.805975 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8400a794-64c3-4172-93fa-ceb4acef604d-webhook-cert\") pod \"8400a794-64c3-4172-93fa-ceb4acef604d\" (UID: \"8400a794-64c3-4172-93fa-ceb4acef604d\") " Dec 03 13:47:04 crc kubenswrapper[5052]: I1203 13:47:04.806028 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hr2xr\" (UniqueName: \"kubernetes.io/projected/f82e13e4-6fd4-4bf8-8d78-c00e7b5a402e-kube-api-access-hr2xr\") pod \"f82e13e4-6fd4-4bf8-8d78-c00e7b5a402e\" (UID: \"f82e13e4-6fd4-4bf8-8d78-c00e7b5a402e\") " Dec 03 13:47:04 crc kubenswrapper[5052]: I1203 13:47:04.806080 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/8400a794-64c3-4172-93fa-ceb4acef604d-apiservice-cert\") pod \"8400a794-64c3-4172-93fa-ceb4acef604d\" (UID: \"8400a794-64c3-4172-93fa-ceb4acef604d\") " Dec 03 13:47:04 crc kubenswrapper[5052]: I1203 13:47:04.806202 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kkdx2\" (UniqueName: \"kubernetes.io/projected/8400a794-64c3-4172-93fa-ceb4acef604d-kube-api-access-kkdx2\") pod \"8400a794-64c3-4172-93fa-ceb4acef604d\" (UID: \"8400a794-64c3-4172-93fa-ceb4acef604d\") " Dec 03 13:47:04 crc kubenswrapper[5052]: I1203 13:47:04.810303 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8400a794-64c3-4172-93fa-ceb4acef604d-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "8400a794-64c3-4172-93fa-ceb4acef604d" (UID: "8400a794-64c3-4172-93fa-ceb4acef604d"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:47:04 crc kubenswrapper[5052]: I1203 13:47:04.810412 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8400a794-64c3-4172-93fa-ceb4acef604d-apiservice-cert" (OuterVolumeSpecName: "apiservice-cert") pod "8400a794-64c3-4172-93fa-ceb4acef604d" (UID: "8400a794-64c3-4172-93fa-ceb4acef604d"). InnerVolumeSpecName "apiservice-cert". PluginName "kubernetes.io/secret", VolumeGidValue "" Dec 03 13:47:04 crc kubenswrapper[5052]: I1203 13:47:04.811407 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f82e13e4-6fd4-4bf8-8d78-c00e7b5a402e-kube-api-access-hr2xr" (OuterVolumeSpecName: "kube-api-access-hr2xr") pod "f82e13e4-6fd4-4bf8-8d78-c00e7b5a402e" (UID: "f82e13e4-6fd4-4bf8-8d78-c00e7b5a402e"). InnerVolumeSpecName "kube-api-access-hr2xr". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:47:04 crc kubenswrapper[5052]: I1203 13:47:04.815688 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8400a794-64c3-4172-93fa-ceb4acef604d-kube-api-access-kkdx2" (OuterVolumeSpecName: "kube-api-access-kkdx2") pod "8400a794-64c3-4172-93fa-ceb4acef604d" (UID: "8400a794-64c3-4172-93fa-ceb4acef604d"). InnerVolumeSpecName "kube-api-access-kkdx2". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:47:04 crc kubenswrapper[5052]: I1203 13:47:04.907471 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-kkdx2\" (UniqueName: \"kubernetes.io/projected/8400a794-64c3-4172-93fa-ceb4acef604d-kube-api-access-kkdx2\") on node \"crc\" DevicePath \"\"" Dec 03 13:47:04 crc kubenswrapper[5052]: I1203 13:47:04.907558 5052 reconciler_common.go:293] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8400a794-64c3-4172-93fa-ceb4acef604d-webhook-cert\") on node \"crc\" DevicePath \"\"" Dec 03 13:47:04 crc kubenswrapper[5052]: I1203 13:47:04.907576 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-hr2xr\" (UniqueName: \"kubernetes.io/projected/f82e13e4-6fd4-4bf8-8d78-c00e7b5a402e-kube-api-access-hr2xr\") on node \"crc\" DevicePath \"\"" Dec 03 13:47:04 crc kubenswrapper[5052]: I1203 13:47:04.907587 5052 reconciler_common.go:293] "Volume detached for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/8400a794-64c3-4172-93fa-ceb4acef604d-apiservice-cert\") on node \"crc\" DevicePath \"\"" Dec 03 13:47:05 crc kubenswrapper[5052]: I1203 13:47:05.064966 5052 generic.go:334] "Generic (PLEG): container finished" podID="8400a794-64c3-4172-93fa-ceb4acef604d" containerID="f09e2aed5955802de7fb6d5d419275a5a96d96c3647698e76456aabab1a980c9" exitCode=0 Dec 03 13:47:05 crc kubenswrapper[5052]: I1203 13:47:05.065016 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/mariadb-operator-controller-manager-5bdcd87d9d-b7mh5" event={"ID":"8400a794-64c3-4172-93fa-ceb4acef604d","Type":"ContainerDied","Data":"f09e2aed5955802de7fb6d5d419275a5a96d96c3647698e76456aabab1a980c9"} Dec 03 13:47:05 crc kubenswrapper[5052]: I1203 13:47:05.065072 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/mariadb-operator-controller-manager-5bdcd87d9d-b7mh5" event={"ID":"8400a794-64c3-4172-93fa-ceb4acef604d","Type":"ContainerDied","Data":"ed7aff77cd517c1dbb79af44d954a67ad4672631c50588bbc8f241f8f4b1b277"} Dec 03 13:47:05 crc kubenswrapper[5052]: I1203 13:47:05.065081 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-operators/mariadb-operator-controller-manager-5bdcd87d9d-b7mh5" Dec 03 13:47:05 crc kubenswrapper[5052]: I1203 13:47:05.065098 5052 scope.go:117] "RemoveContainer" containerID="f09e2aed5955802de7fb6d5d419275a5a96d96c3647698e76456aabab1a980c9" Dec 03 13:47:05 crc kubenswrapper[5052]: I1203 13:47:05.066808 5052 generic.go:334] "Generic (PLEG): container finished" podID="f82e13e4-6fd4-4bf8-8d78-c00e7b5a402e" containerID="29ad16f9f611514fc76240ac2c1cfa6a3a5cb4213b29155759c964ed5238361c" exitCode=0 Dec 03 13:47:05 crc kubenswrapper[5052]: I1203 13:47:05.066853 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/mariadb-operator-index-p2x5s" event={"ID":"f82e13e4-6fd4-4bf8-8d78-c00e7b5a402e","Type":"ContainerDied","Data":"29ad16f9f611514fc76240ac2c1cfa6a3a5cb4213b29155759c964ed5238361c"} Dec 03 13:47:05 crc kubenswrapper[5052]: I1203 13:47:05.066880 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openstack-operators/mariadb-operator-index-p2x5s" Dec 03 13:47:05 crc kubenswrapper[5052]: I1203 13:47:05.066900 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openstack-operators/mariadb-operator-index-p2x5s" event={"ID":"f82e13e4-6fd4-4bf8-8d78-c00e7b5a402e","Type":"ContainerDied","Data":"1980c94fffa6f6a1807de1ec07b99d9b54965f9d063a33252ff595d7c1d8ea26"} Dec 03 13:47:05 crc kubenswrapper[5052]: I1203 13:47:05.087279 5052 scope.go:117] "RemoveContainer" containerID="f09e2aed5955802de7fb6d5d419275a5a96d96c3647698e76456aabab1a980c9" Dec 03 13:47:05 crc kubenswrapper[5052]: E1203 13:47:05.087726 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f09e2aed5955802de7fb6d5d419275a5a96d96c3647698e76456aabab1a980c9\": container with ID starting with f09e2aed5955802de7fb6d5d419275a5a96d96c3647698e76456aabab1a980c9 not found: ID does not exist" containerID="f09e2aed5955802de7fb6d5d419275a5a96d96c3647698e76456aabab1a980c9" Dec 03 13:47:05 crc kubenswrapper[5052]: I1203 13:47:05.087776 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f09e2aed5955802de7fb6d5d419275a5a96d96c3647698e76456aabab1a980c9"} err="failed to get container status \"f09e2aed5955802de7fb6d5d419275a5a96d96c3647698e76456aabab1a980c9\": rpc error: code = NotFound desc = could not find container \"f09e2aed5955802de7fb6d5d419275a5a96d96c3647698e76456aabab1a980c9\": container with ID starting with f09e2aed5955802de7fb6d5d419275a5a96d96c3647698e76456aabab1a980c9 not found: ID does not exist" Dec 03 13:47:05 crc kubenswrapper[5052]: I1203 13:47:05.087802 5052 scope.go:117] "RemoveContainer" containerID="29ad16f9f611514fc76240ac2c1cfa6a3a5cb4213b29155759c964ed5238361c" Dec 03 13:47:05 crc kubenswrapper[5052]: I1203 13:47:05.103062 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-operators/mariadb-operator-controller-manager-5bdcd87d9d-b7mh5"] Dec 03 13:47:05 crc kubenswrapper[5052]: I1203 13:47:05.106254 5052 scope.go:117] "RemoveContainer" containerID="29ad16f9f611514fc76240ac2c1cfa6a3a5cb4213b29155759c964ed5238361c" Dec 03 13:47:05 crc kubenswrapper[5052]: E1203 13:47:05.106843 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"29ad16f9f611514fc76240ac2c1cfa6a3a5cb4213b29155759c964ed5238361c\": container with ID starting with 29ad16f9f611514fc76240ac2c1cfa6a3a5cb4213b29155759c964ed5238361c not found: ID does not exist" containerID="29ad16f9f611514fc76240ac2c1cfa6a3a5cb4213b29155759c964ed5238361c" Dec 03 13:47:05 crc kubenswrapper[5052]: I1203 13:47:05.106905 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"29ad16f9f611514fc76240ac2c1cfa6a3a5cb4213b29155759c964ed5238361c"} err="failed to get container status \"29ad16f9f611514fc76240ac2c1cfa6a3a5cb4213b29155759c964ed5238361c\": rpc error: code = NotFound desc = could not find container \"29ad16f9f611514fc76240ac2c1cfa6a3a5cb4213b29155759c964ed5238361c\": container with ID starting with 29ad16f9f611514fc76240ac2c1cfa6a3a5cb4213b29155759c964ed5238361c not found: ID does not exist" Dec 03 13:47:05 crc kubenswrapper[5052]: I1203 13:47:05.109883 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-operators/mariadb-operator-controller-manager-5bdcd87d9d-b7mh5"] Dec 03 13:47:05 crc kubenswrapper[5052]: I1203 13:47:05.117685 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openstack-operators/mariadb-operator-index-p2x5s"] Dec 03 13:47:05 crc kubenswrapper[5052]: I1203 13:47:05.121314 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openstack-operators/mariadb-operator-index-p2x5s"] Dec 03 13:47:05 crc kubenswrapper[5052]: I1203 13:47:05.681905 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="11c2bcbf-8d83-494a-8019-e2f34b7b1125" path="/var/lib/kubelet/pods/11c2bcbf-8d83-494a-8019-e2f34b7b1125/volumes" Dec 03 13:47:05 crc kubenswrapper[5052]: I1203 13:47:05.682807 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8400a794-64c3-4172-93fa-ceb4acef604d" path="/var/lib/kubelet/pods/8400a794-64c3-4172-93fa-ceb4acef604d/volumes" Dec 03 13:47:05 crc kubenswrapper[5052]: I1203 13:47:05.683240 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a8c2a4d6-1e91-40de-9edc-60b79b888d1f" path="/var/lib/kubelet/pods/a8c2a4d6-1e91-40de-9edc-60b79b888d1f/volumes" Dec 03 13:47:05 crc kubenswrapper[5052]: I1203 13:47:05.684161 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f82e13e4-6fd4-4bf8-8d78-c00e7b5a402e" path="/var/lib/kubelet/pods/f82e13e4-6fd4-4bf8-8d78-c00e7b5a402e/volumes" Dec 03 13:47:09 crc kubenswrapper[5052]: I1203 13:47:09.775030 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openshift-must-gather-58j6j/must-gather-xj487"] Dec 03 13:47:09 crc kubenswrapper[5052]: E1203 13:47:09.775963 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="a2db0de0-8930-4d21-ad37-58889377d33b" containerName="galera" Dec 03 13:47:09 crc kubenswrapper[5052]: I1203 13:47:09.775982 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="a2db0de0-8930-4d21-ad37-58889377d33b" containerName="galera" Dec 03 13:47:09 crc kubenswrapper[5052]: E1203 13:47:09.775994 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="a38bb331-9bf3-48e9-bd95-7b5854cd4347" containerName="registry-server" Dec 03 13:47:09 crc kubenswrapper[5052]: I1203 13:47:09.776003 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="a38bb331-9bf3-48e9-bd95-7b5854cd4347" containerName="registry-server" Dec 03 13:47:09 crc kubenswrapper[5052]: E1203 13:47:09.776024 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="5096dd99-719d-412e-804b-fc1452ee8b95" containerName="mysql-bootstrap" Dec 03 13:47:09 crc kubenswrapper[5052]: I1203 13:47:09.776035 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="5096dd99-719d-412e-804b-fc1452ee8b95" containerName="mysql-bootstrap" Dec 03 13:47:09 crc kubenswrapper[5052]: E1203 13:47:09.776049 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="202fe9d9-dcdf-43ae-bfc4-4efc7f9e7dba" containerName="operator" Dec 03 13:47:09 crc kubenswrapper[5052]: I1203 13:47:09.776057 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="202fe9d9-dcdf-43ae-bfc4-4efc7f9e7dba" containerName="operator" Dec 03 13:47:09 crc kubenswrapper[5052]: E1203 13:47:09.776069 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="f82e13e4-6fd4-4bf8-8d78-c00e7b5a402e" containerName="registry-server" Dec 03 13:47:09 crc kubenswrapper[5052]: I1203 13:47:09.776077 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="f82e13e4-6fd4-4bf8-8d78-c00e7b5a402e" containerName="registry-server" Dec 03 13:47:09 crc kubenswrapper[5052]: E1203 13:47:09.776088 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="aa07e692-9f86-4c61-b674-beeb49376c0e" containerName="setup-container" Dec 03 13:47:09 crc kubenswrapper[5052]: I1203 13:47:09.776096 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa07e692-9f86-4c61-b674-beeb49376c0e" containerName="setup-container" Dec 03 13:47:09 crc kubenswrapper[5052]: E1203 13:47:09.776111 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="5179e964-3df1-4d07-b440-a01ba074ac05" containerName="mariadb-account-delete" Dec 03 13:47:09 crc kubenswrapper[5052]: I1203 13:47:09.776119 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="5179e964-3df1-4d07-b440-a01ba074ac05" containerName="mariadb-account-delete" Dec 03 13:47:09 crc kubenswrapper[5052]: E1203 13:47:09.776131 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="d8df61e7-dad4-4072-a268-ddc6b55f0696" containerName="keystone-api" Dec 03 13:47:09 crc kubenswrapper[5052]: I1203 13:47:09.776139 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="d8df61e7-dad4-4072-a268-ddc6b55f0696" containerName="keystone-api" Dec 03 13:47:09 crc kubenswrapper[5052]: E1203 13:47:09.776149 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="2e3a3ea5-4ff6-4c9f-a093-38754c10012a" containerName="galera" Dec 03 13:47:09 crc kubenswrapper[5052]: I1203 13:47:09.776157 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="2e3a3ea5-4ff6-4c9f-a093-38754c10012a" containerName="galera" Dec 03 13:47:09 crc kubenswrapper[5052]: E1203 13:47:09.776168 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="a2db0de0-8930-4d21-ad37-58889377d33b" containerName="mysql-bootstrap" Dec 03 13:47:09 crc kubenswrapper[5052]: I1203 13:47:09.776177 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="a2db0de0-8930-4d21-ad37-58889377d33b" containerName="mysql-bootstrap" Dec 03 13:47:09 crc kubenswrapper[5052]: E1203 13:47:09.776188 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="afe4ff83-55b7-48a0-902c-4162f0f68ac5" containerName="manager" Dec 03 13:47:09 crc kubenswrapper[5052]: I1203 13:47:09.776196 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="afe4ff83-55b7-48a0-902c-4162f0f68ac5" containerName="manager" Dec 03 13:47:09 crc kubenswrapper[5052]: E1203 13:47:09.776212 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="b70bf54e-974c-4721-8384-2c6bceaee23f" containerName="registry-server" Dec 03 13:47:09 crc kubenswrapper[5052]: I1203 13:47:09.776220 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="b70bf54e-974c-4721-8384-2c6bceaee23f" containerName="registry-server" Dec 03 13:47:09 crc kubenswrapper[5052]: E1203 13:47:09.776232 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="2be98578-12f4-41ef-9fda-5db102aa4702" containerName="manager" Dec 03 13:47:09 crc kubenswrapper[5052]: I1203 13:47:09.776241 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="2be98578-12f4-41ef-9fda-5db102aa4702" containerName="manager" Dec 03 13:47:09 crc kubenswrapper[5052]: E1203 13:47:09.776251 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="5b2f3c3a-2756-4269-99da-ba83e8097c9e" containerName="manager" Dec 03 13:47:09 crc kubenswrapper[5052]: I1203 13:47:09.776259 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="5b2f3c3a-2756-4269-99da-ba83e8097c9e" containerName="manager" Dec 03 13:47:09 crc kubenswrapper[5052]: E1203 13:47:09.776275 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="afe4ff83-55b7-48a0-902c-4162f0f68ac5" containerName="kube-rbac-proxy" Dec 03 13:47:09 crc kubenswrapper[5052]: I1203 13:47:09.776285 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="afe4ff83-55b7-48a0-902c-4162f0f68ac5" containerName="kube-rbac-proxy" Dec 03 13:47:09 crc kubenswrapper[5052]: E1203 13:47:09.776295 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="8496c1b5-ae6d-4357-9812-9d44818ee9d1" containerName="registry-server" Dec 03 13:47:09 crc kubenswrapper[5052]: I1203 13:47:09.776303 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="8496c1b5-ae6d-4357-9812-9d44818ee9d1" containerName="registry-server" Dec 03 13:47:09 crc kubenswrapper[5052]: E1203 13:47:09.776320 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="2e3a3ea5-4ff6-4c9f-a093-38754c10012a" containerName="mysql-bootstrap" Dec 03 13:47:09 crc kubenswrapper[5052]: I1203 13:47:09.776328 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="2e3a3ea5-4ff6-4c9f-a093-38754c10012a" containerName="mysql-bootstrap" Dec 03 13:47:09 crc kubenswrapper[5052]: E1203 13:47:09.776340 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="8400a794-64c3-4172-93fa-ceb4acef604d" containerName="manager" Dec 03 13:47:09 crc kubenswrapper[5052]: I1203 13:47:09.776348 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="8400a794-64c3-4172-93fa-ceb4acef604d" containerName="manager" Dec 03 13:47:09 crc kubenswrapper[5052]: E1203 13:47:09.776357 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="a8c2a4d6-1e91-40de-9edc-60b79b888d1f" containerName="registry-server" Dec 03 13:47:09 crc kubenswrapper[5052]: I1203 13:47:09.776365 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="a8c2a4d6-1e91-40de-9edc-60b79b888d1f" containerName="registry-server" Dec 03 13:47:09 crc kubenswrapper[5052]: E1203 13:47:09.776376 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="9dfea40e-ca29-4e45-85ff-fde0ca025e75" containerName="memcached" Dec 03 13:47:09 crc kubenswrapper[5052]: I1203 13:47:09.776385 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="9dfea40e-ca29-4e45-85ff-fde0ca025e75" containerName="memcached" Dec 03 13:47:09 crc kubenswrapper[5052]: E1203 13:47:09.776399 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="aa07e692-9f86-4c61-b674-beeb49376c0e" containerName="rabbitmq" Dec 03 13:47:09 crc kubenswrapper[5052]: I1203 13:47:09.776409 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa07e692-9f86-4c61-b674-beeb49376c0e" containerName="rabbitmq" Dec 03 13:47:09 crc kubenswrapper[5052]: E1203 13:47:09.776422 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="5096dd99-719d-412e-804b-fc1452ee8b95" containerName="galera" Dec 03 13:47:09 crc kubenswrapper[5052]: I1203 13:47:09.776429 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="5096dd99-719d-412e-804b-fc1452ee8b95" containerName="galera" Dec 03 13:47:09 crc kubenswrapper[5052]: I1203 13:47:09.776617 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="a38bb331-9bf3-48e9-bd95-7b5854cd4347" containerName="registry-server" Dec 03 13:47:09 crc kubenswrapper[5052]: I1203 13:47:09.776629 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="b70bf54e-974c-4721-8384-2c6bceaee23f" containerName="registry-server" Dec 03 13:47:09 crc kubenswrapper[5052]: I1203 13:47:09.776640 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="2e3a3ea5-4ff6-4c9f-a093-38754c10012a" containerName="galera" Dec 03 13:47:09 crc kubenswrapper[5052]: I1203 13:47:09.776656 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="5b2f3c3a-2756-4269-99da-ba83e8097c9e" containerName="manager" Dec 03 13:47:09 crc kubenswrapper[5052]: I1203 13:47:09.776671 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="5179e964-3df1-4d07-b440-a01ba074ac05" containerName="mariadb-account-delete" Dec 03 13:47:09 crc kubenswrapper[5052]: I1203 13:47:09.776683 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="afe4ff83-55b7-48a0-902c-4162f0f68ac5" containerName="kube-rbac-proxy" Dec 03 13:47:09 crc kubenswrapper[5052]: I1203 13:47:09.776696 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="f82e13e4-6fd4-4bf8-8d78-c00e7b5a402e" containerName="registry-server" Dec 03 13:47:09 crc kubenswrapper[5052]: I1203 13:47:09.776706 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="9dfea40e-ca29-4e45-85ff-fde0ca025e75" containerName="memcached" Dec 03 13:47:09 crc kubenswrapper[5052]: I1203 13:47:09.776717 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="8400a794-64c3-4172-93fa-ceb4acef604d" containerName="manager" Dec 03 13:47:09 crc kubenswrapper[5052]: I1203 13:47:09.776731 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="a8c2a4d6-1e91-40de-9edc-60b79b888d1f" containerName="registry-server" Dec 03 13:47:09 crc kubenswrapper[5052]: I1203 13:47:09.776743 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="a2db0de0-8930-4d21-ad37-58889377d33b" containerName="galera" Dec 03 13:47:09 crc kubenswrapper[5052]: I1203 13:47:09.776754 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="d8df61e7-dad4-4072-a268-ddc6b55f0696" containerName="keystone-api" Dec 03 13:47:09 crc kubenswrapper[5052]: I1203 13:47:09.776766 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="afe4ff83-55b7-48a0-902c-4162f0f68ac5" containerName="manager" Dec 03 13:47:09 crc kubenswrapper[5052]: I1203 13:47:09.776777 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="2be98578-12f4-41ef-9fda-5db102aa4702" containerName="manager" Dec 03 13:47:09 crc kubenswrapper[5052]: I1203 13:47:09.776786 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="aa07e692-9f86-4c61-b674-beeb49376c0e" containerName="rabbitmq" Dec 03 13:47:09 crc kubenswrapper[5052]: I1203 13:47:09.776798 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="5096dd99-719d-412e-804b-fc1452ee8b95" containerName="galera" Dec 03 13:47:09 crc kubenswrapper[5052]: I1203 13:47:09.776808 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="202fe9d9-dcdf-43ae-bfc4-4efc7f9e7dba" containerName="operator" Dec 03 13:47:09 crc kubenswrapper[5052]: I1203 13:47:09.776819 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="8496c1b5-ae6d-4357-9812-9d44818ee9d1" containerName="registry-server" Dec 03 13:47:09 crc kubenswrapper[5052]: I1203 13:47:09.777661 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-58j6j/must-gather-xj487" Dec 03 13:47:09 crc kubenswrapper[5052]: I1203 13:47:09.781565 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-must-gather-58j6j"/"openshift-service-ca.crt" Dec 03 13:47:09 crc kubenswrapper[5052]: I1203 13:47:09.781907 5052 reflector.go:368] Caches populated for *v1.Secret from object-"openshift-must-gather-58j6j"/"default-dockercfg-njj8b" Dec 03 13:47:09 crc kubenswrapper[5052]: I1203 13:47:09.781976 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-must-gather-58j6j"/"kube-root-ca.crt" Dec 03 13:47:09 crc kubenswrapper[5052]: I1203 13:47:09.791483 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-58j6j/must-gather-xj487"] Dec 03 13:47:09 crc kubenswrapper[5052]: I1203 13:47:09.871200 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/f3203993-dab9-40db-9d30-8d8058573a90-must-gather-output\") pod \"must-gather-xj487\" (UID: \"f3203993-dab9-40db-9d30-8d8058573a90\") " pod="openshift-must-gather-58j6j/must-gather-xj487" Dec 03 13:47:09 crc kubenswrapper[5052]: I1203 13:47:09.871263 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ltb2k\" (UniqueName: \"kubernetes.io/projected/f3203993-dab9-40db-9d30-8d8058573a90-kube-api-access-ltb2k\") pod \"must-gather-xj487\" (UID: \"f3203993-dab9-40db-9d30-8d8058573a90\") " pod="openshift-must-gather-58j6j/must-gather-xj487" Dec 03 13:47:09 crc kubenswrapper[5052]: I1203 13:47:09.972442 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/f3203993-dab9-40db-9d30-8d8058573a90-must-gather-output\") pod \"must-gather-xj487\" (UID: \"f3203993-dab9-40db-9d30-8d8058573a90\") " pod="openshift-must-gather-58j6j/must-gather-xj487" Dec 03 13:47:09 crc kubenswrapper[5052]: I1203 13:47:09.972564 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-ltb2k\" (UniqueName: \"kubernetes.io/projected/f3203993-dab9-40db-9d30-8d8058573a90-kube-api-access-ltb2k\") pod \"must-gather-xj487\" (UID: \"f3203993-dab9-40db-9d30-8d8058573a90\") " pod="openshift-must-gather-58j6j/must-gather-xj487" Dec 03 13:47:09 crc kubenswrapper[5052]: I1203 13:47:09.972989 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/f3203993-dab9-40db-9d30-8d8058573a90-must-gather-output\") pod \"must-gather-xj487\" (UID: \"f3203993-dab9-40db-9d30-8d8058573a90\") " pod="openshift-must-gather-58j6j/must-gather-xj487" Dec 03 13:47:10 crc kubenswrapper[5052]: I1203 13:47:09.993888 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-ltb2k\" (UniqueName: \"kubernetes.io/projected/f3203993-dab9-40db-9d30-8d8058573a90-kube-api-access-ltb2k\") pod \"must-gather-xj487\" (UID: \"f3203993-dab9-40db-9d30-8d8058573a90\") " pod="openshift-must-gather-58j6j/must-gather-xj487" Dec 03 13:47:10 crc kubenswrapper[5052]: I1203 13:47:10.096875 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-58j6j/must-gather-xj487" Dec 03 13:47:10 crc kubenswrapper[5052]: I1203 13:47:10.557585 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-58j6j/must-gather-xj487"] Dec 03 13:47:11 crc kubenswrapper[5052]: I1203 13:47:11.111477 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-58j6j/must-gather-xj487" event={"ID":"f3203993-dab9-40db-9d30-8d8058573a90","Type":"ContainerStarted","Data":"727fee71b50b24e4ef9dbe96636856d807d286f77dc33be5deb40907e214e39c"} Dec 03 13:47:15 crc kubenswrapper[5052]: I1203 13:47:15.142706 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-58j6j/must-gather-xj487" event={"ID":"f3203993-dab9-40db-9d30-8d8058573a90","Type":"ContainerStarted","Data":"99f6b08b0dfd77b55d7c46ab10e4cf4ded65ff3ba4e32445bac15d2c975c36c8"} Dec 03 13:47:15 crc kubenswrapper[5052]: I1203 13:47:15.143286 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-58j6j/must-gather-xj487" event={"ID":"f3203993-dab9-40db-9d30-8d8058573a90","Type":"ContainerStarted","Data":"3a45fdbf0aa9269d811a6da6d0e21822286efa4edc16fbc248e622729c89d274"} Dec 03 13:47:15 crc kubenswrapper[5052]: I1203 13:47:15.158805 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-must-gather-58j6j/must-gather-xj487" podStartSLOduration=2.575689314 podStartE2EDuration="6.158786894s" podCreationTimestamp="2025-12-03 13:47:09 +0000 UTC" firstStartedPulling="2025-12-03 13:47:10.565623487 +0000 UTC m=+8821.153865251" lastFinishedPulling="2025-12-03 13:47:14.148721057 +0000 UTC m=+8824.736962831" observedRunningTime="2025-12-03 13:47:15.157408694 +0000 UTC m=+8825.745650458" watchObservedRunningTime="2025-12-03 13:47:15.158786894 +0000 UTC m=+8825.747028658" Dec 03 13:47:28 crc kubenswrapper[5052]: I1203 13:47:28.875992 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openshift-must-gather-vj2x8/must-gather-s9pjl"] Dec 03 13:47:28 crc kubenswrapper[5052]: I1203 13:47:28.878820 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-vj2x8/must-gather-s9pjl" Dec 03 13:47:28 crc kubenswrapper[5052]: I1203 13:47:28.884470 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-must-gather-vj2x8"/"kube-root-ca.crt" Dec 03 13:47:28 crc kubenswrapper[5052]: I1203 13:47:28.884718 5052 reflector.go:368] Caches populated for *v1.ConfigMap from object-"openshift-must-gather-vj2x8"/"openshift-service-ca.crt" Dec 03 13:47:28 crc kubenswrapper[5052]: I1203 13:47:28.904468 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-vj2x8/must-gather-s9pjl"] Dec 03 13:47:29 crc kubenswrapper[5052]: I1203 13:47:29.014154 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/186b2ab5-1c1f-4a2c-90bf-c5dc971d4677-must-gather-output\") pod \"must-gather-s9pjl\" (UID: \"186b2ab5-1c1f-4a2c-90bf-c5dc971d4677\") " pod="openshift-must-gather-vj2x8/must-gather-s9pjl" Dec 03 13:47:29 crc kubenswrapper[5052]: I1203 13:47:29.014213 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8fp4q\" (UniqueName: \"kubernetes.io/projected/186b2ab5-1c1f-4a2c-90bf-c5dc971d4677-kube-api-access-8fp4q\") pod \"must-gather-s9pjl\" (UID: \"186b2ab5-1c1f-4a2c-90bf-c5dc971d4677\") " pod="openshift-must-gather-vj2x8/must-gather-s9pjl" Dec 03 13:47:29 crc kubenswrapper[5052]: I1203 13:47:29.115775 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/186b2ab5-1c1f-4a2c-90bf-c5dc971d4677-must-gather-output\") pod \"must-gather-s9pjl\" (UID: \"186b2ab5-1c1f-4a2c-90bf-c5dc971d4677\") " pod="openshift-must-gather-vj2x8/must-gather-s9pjl" Dec 03 13:47:29 crc kubenswrapper[5052]: I1203 13:47:29.116128 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-8fp4q\" (UniqueName: \"kubernetes.io/projected/186b2ab5-1c1f-4a2c-90bf-c5dc971d4677-kube-api-access-8fp4q\") pod \"must-gather-s9pjl\" (UID: \"186b2ab5-1c1f-4a2c-90bf-c5dc971d4677\") " pod="openshift-must-gather-vj2x8/must-gather-s9pjl" Dec 03 13:47:29 crc kubenswrapper[5052]: I1203 13:47:29.116289 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/186b2ab5-1c1f-4a2c-90bf-c5dc971d4677-must-gather-output\") pod \"must-gather-s9pjl\" (UID: \"186b2ab5-1c1f-4a2c-90bf-c5dc971d4677\") " pod="openshift-must-gather-vj2x8/must-gather-s9pjl" Dec 03 13:47:29 crc kubenswrapper[5052]: I1203 13:47:29.136214 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-8fp4q\" (UniqueName: \"kubernetes.io/projected/186b2ab5-1c1f-4a2c-90bf-c5dc971d4677-kube-api-access-8fp4q\") pod \"must-gather-s9pjl\" (UID: \"186b2ab5-1c1f-4a2c-90bf-c5dc971d4677\") " pod="openshift-must-gather-vj2x8/must-gather-s9pjl" Dec 03 13:47:29 crc kubenswrapper[5052]: I1203 13:47:29.197999 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-vj2x8/must-gather-s9pjl" Dec 03 13:47:29 crc kubenswrapper[5052]: I1203 13:47:29.640018 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-vj2x8/must-gather-s9pjl"] Dec 03 13:47:30 crc kubenswrapper[5052]: I1203 13:47:30.246819 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-vj2x8/must-gather-s9pjl" event={"ID":"186b2ab5-1c1f-4a2c-90bf-c5dc971d4677","Type":"ContainerStarted","Data":"8f1493a3c2276f873ac6c7a4554dc39d23d61c4c9d2681fd2d7a00c65ff9225f"} Dec 03 13:47:31 crc kubenswrapper[5052]: I1203 13:47:31.257576 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-vj2x8/must-gather-s9pjl" event={"ID":"186b2ab5-1c1f-4a2c-90bf-c5dc971d4677","Type":"ContainerStarted","Data":"38689613e122abf04f2f1334aa689722b063681e07893bf9d12ea7fce4c29e94"} Dec 03 13:47:31 crc kubenswrapper[5052]: I1203 13:47:31.258185 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-vj2x8/must-gather-s9pjl" event={"ID":"186b2ab5-1c1f-4a2c-90bf-c5dc971d4677","Type":"ContainerStarted","Data":"5bf5e212c38591b042130bd0a9422fc8bb9656717c05a83220f5ae0ca67264af"} Dec 03 13:47:31 crc kubenswrapper[5052]: I1203 13:47:31.285174 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-must-gather-vj2x8/must-gather-s9pjl" podStartSLOduration=3.285154586 podStartE2EDuration="3.285154586s" podCreationTimestamp="2025-12-03 13:47:28 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-12-03 13:47:31.275694165 +0000 UTC m=+8841.863935959" watchObservedRunningTime="2025-12-03 13:47:31.285154586 +0000 UTC m=+8841.873396360" Dec 03 13:47:38 crc kubenswrapper[5052]: I1203 13:47:38.298109 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openshift-marketplace/community-operators-ptbl7"] Dec 03 13:47:38 crc kubenswrapper[5052]: I1203 13:47:38.300210 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/community-operators-ptbl7" Dec 03 13:47:38 crc kubenswrapper[5052]: I1203 13:47:38.309658 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/community-operators-ptbl7"] Dec 03 13:47:38 crc kubenswrapper[5052]: I1203 13:47:38.444156 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/4cc70472-d066-4cab-bde9-9590e1bd74c2-catalog-content\") pod \"community-operators-ptbl7\" (UID: \"4cc70472-d066-4cab-bde9-9590e1bd74c2\") " pod="openshift-marketplace/community-operators-ptbl7" Dec 03 13:47:38 crc kubenswrapper[5052]: I1203 13:47:38.444355 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4jfkv\" (UniqueName: \"kubernetes.io/projected/4cc70472-d066-4cab-bde9-9590e1bd74c2-kube-api-access-4jfkv\") pod \"community-operators-ptbl7\" (UID: \"4cc70472-d066-4cab-bde9-9590e1bd74c2\") " pod="openshift-marketplace/community-operators-ptbl7" Dec 03 13:47:38 crc kubenswrapper[5052]: I1203 13:47:38.444387 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/4cc70472-d066-4cab-bde9-9590e1bd74c2-utilities\") pod \"community-operators-ptbl7\" (UID: \"4cc70472-d066-4cab-bde9-9590e1bd74c2\") " pod="openshift-marketplace/community-operators-ptbl7" Dec 03 13:47:38 crc kubenswrapper[5052]: I1203 13:47:38.499751 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openshift-marketplace/redhat-marketplace-lg6kc"] Dec 03 13:47:38 crc kubenswrapper[5052]: I1203 13:47:38.501253 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/redhat-marketplace-lg6kc" Dec 03 13:47:38 crc kubenswrapper[5052]: I1203 13:47:38.510546 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/redhat-marketplace-lg6kc"] Dec 03 13:47:38 crc kubenswrapper[5052]: I1203 13:47:38.545895 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-4jfkv\" (UniqueName: \"kubernetes.io/projected/4cc70472-d066-4cab-bde9-9590e1bd74c2-kube-api-access-4jfkv\") pod \"community-operators-ptbl7\" (UID: \"4cc70472-d066-4cab-bde9-9590e1bd74c2\") " pod="openshift-marketplace/community-operators-ptbl7" Dec 03 13:47:38 crc kubenswrapper[5052]: I1203 13:47:38.545946 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/4cc70472-d066-4cab-bde9-9590e1bd74c2-utilities\") pod \"community-operators-ptbl7\" (UID: \"4cc70472-d066-4cab-bde9-9590e1bd74c2\") " pod="openshift-marketplace/community-operators-ptbl7" Dec 03 13:47:38 crc kubenswrapper[5052]: I1203 13:47:38.545986 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/4cc70472-d066-4cab-bde9-9590e1bd74c2-catalog-content\") pod \"community-operators-ptbl7\" (UID: \"4cc70472-d066-4cab-bde9-9590e1bd74c2\") " pod="openshift-marketplace/community-operators-ptbl7" Dec 03 13:47:38 crc kubenswrapper[5052]: I1203 13:47:38.546575 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/4cc70472-d066-4cab-bde9-9590e1bd74c2-utilities\") pod \"community-operators-ptbl7\" (UID: \"4cc70472-d066-4cab-bde9-9590e1bd74c2\") " pod="openshift-marketplace/community-operators-ptbl7" Dec 03 13:47:38 crc kubenswrapper[5052]: I1203 13:47:38.546625 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/4cc70472-d066-4cab-bde9-9590e1bd74c2-catalog-content\") pod \"community-operators-ptbl7\" (UID: \"4cc70472-d066-4cab-bde9-9590e1bd74c2\") " pod="openshift-marketplace/community-operators-ptbl7" Dec 03 13:47:38 crc kubenswrapper[5052]: I1203 13:47:38.575742 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-4jfkv\" (UniqueName: \"kubernetes.io/projected/4cc70472-d066-4cab-bde9-9590e1bd74c2-kube-api-access-4jfkv\") pod \"community-operators-ptbl7\" (UID: \"4cc70472-d066-4cab-bde9-9590e1bd74c2\") " pod="openshift-marketplace/community-operators-ptbl7" Dec 03 13:47:38 crc kubenswrapper[5052]: I1203 13:47:38.617413 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/community-operators-ptbl7" Dec 03 13:47:38 crc kubenswrapper[5052]: I1203 13:47:38.646897 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g7l79\" (UniqueName: \"kubernetes.io/projected/e369fda6-f1c7-47eb-b690-6c268b3000b3-kube-api-access-g7l79\") pod \"redhat-marketplace-lg6kc\" (UID: \"e369fda6-f1c7-47eb-b690-6c268b3000b3\") " pod="openshift-marketplace/redhat-marketplace-lg6kc" Dec 03 13:47:38 crc kubenswrapper[5052]: I1203 13:47:38.646968 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/e369fda6-f1c7-47eb-b690-6c268b3000b3-catalog-content\") pod \"redhat-marketplace-lg6kc\" (UID: \"e369fda6-f1c7-47eb-b690-6c268b3000b3\") " pod="openshift-marketplace/redhat-marketplace-lg6kc" Dec 03 13:47:38 crc kubenswrapper[5052]: I1203 13:47:38.647011 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/e369fda6-f1c7-47eb-b690-6c268b3000b3-utilities\") pod \"redhat-marketplace-lg6kc\" (UID: \"e369fda6-f1c7-47eb-b690-6c268b3000b3\") " pod="openshift-marketplace/redhat-marketplace-lg6kc" Dec 03 13:47:38 crc kubenswrapper[5052]: I1203 13:47:38.748466 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-g7l79\" (UniqueName: \"kubernetes.io/projected/e369fda6-f1c7-47eb-b690-6c268b3000b3-kube-api-access-g7l79\") pod \"redhat-marketplace-lg6kc\" (UID: \"e369fda6-f1c7-47eb-b690-6c268b3000b3\") " pod="openshift-marketplace/redhat-marketplace-lg6kc" Dec 03 13:47:38 crc kubenswrapper[5052]: I1203 13:47:38.748565 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/e369fda6-f1c7-47eb-b690-6c268b3000b3-catalog-content\") pod \"redhat-marketplace-lg6kc\" (UID: \"e369fda6-f1c7-47eb-b690-6c268b3000b3\") " pod="openshift-marketplace/redhat-marketplace-lg6kc" Dec 03 13:47:38 crc kubenswrapper[5052]: I1203 13:47:38.748615 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/e369fda6-f1c7-47eb-b690-6c268b3000b3-utilities\") pod \"redhat-marketplace-lg6kc\" (UID: \"e369fda6-f1c7-47eb-b690-6c268b3000b3\") " pod="openshift-marketplace/redhat-marketplace-lg6kc" Dec 03 13:47:38 crc kubenswrapper[5052]: I1203 13:47:38.749088 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/e369fda6-f1c7-47eb-b690-6c268b3000b3-utilities\") pod \"redhat-marketplace-lg6kc\" (UID: \"e369fda6-f1c7-47eb-b690-6c268b3000b3\") " pod="openshift-marketplace/redhat-marketplace-lg6kc" Dec 03 13:47:38 crc kubenswrapper[5052]: I1203 13:47:38.749664 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/e369fda6-f1c7-47eb-b690-6c268b3000b3-catalog-content\") pod \"redhat-marketplace-lg6kc\" (UID: \"e369fda6-f1c7-47eb-b690-6c268b3000b3\") " pod="openshift-marketplace/redhat-marketplace-lg6kc" Dec 03 13:47:38 crc kubenswrapper[5052]: I1203 13:47:38.779260 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-g7l79\" (UniqueName: \"kubernetes.io/projected/e369fda6-f1c7-47eb-b690-6c268b3000b3-kube-api-access-g7l79\") pod \"redhat-marketplace-lg6kc\" (UID: \"e369fda6-f1c7-47eb-b690-6c268b3000b3\") " pod="openshift-marketplace/redhat-marketplace-lg6kc" Dec 03 13:47:38 crc kubenswrapper[5052]: I1203 13:47:38.820865 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/redhat-marketplace-lg6kc" Dec 03 13:47:39 crc kubenswrapper[5052]: I1203 13:47:39.146688 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/community-operators-ptbl7"] Dec 03 13:47:39 crc kubenswrapper[5052]: I1203 13:47:39.311557 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/community-operators-ptbl7" event={"ID":"4cc70472-d066-4cab-bde9-9590e1bd74c2","Type":"ContainerStarted","Data":"5a787ce7b5d223a6c3b1042160fa3e1371ee80b4e60472f01a540ad041b0f611"} Dec 03 13:47:39 crc kubenswrapper[5052]: I1203 13:47:39.345538 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/redhat-marketplace-lg6kc"] Dec 03 13:47:39 crc kubenswrapper[5052]: W1203 13:47:39.392844 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode369fda6_f1c7_47eb_b690_6c268b3000b3.slice/crio-df4e17c807d1afcdd97e12f0548a51c660f58f69c1b05c14a9054ba0a46491f6 WatchSource:0}: Error finding container df4e17c807d1afcdd97e12f0548a51c660f58f69c1b05c14a9054ba0a46491f6: Status 404 returned error can't find the container with id df4e17c807d1afcdd97e12f0548a51c660f58f69c1b05c14a9054ba0a46491f6 Dec 03 13:47:40 crc kubenswrapper[5052]: I1203 13:47:40.319446 5052 generic.go:334] "Generic (PLEG): container finished" podID="e369fda6-f1c7-47eb-b690-6c268b3000b3" containerID="c66159fdb0b1b367e7b63df8ddafb3d826e53e6e468ecbd3f3a73b27b6589896" exitCode=0 Dec 03 13:47:40 crc kubenswrapper[5052]: I1203 13:47:40.319594 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-marketplace-lg6kc" event={"ID":"e369fda6-f1c7-47eb-b690-6c268b3000b3","Type":"ContainerDied","Data":"c66159fdb0b1b367e7b63df8ddafb3d826e53e6e468ecbd3f3a73b27b6589896"} Dec 03 13:47:40 crc kubenswrapper[5052]: I1203 13:47:40.319843 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-marketplace-lg6kc" event={"ID":"e369fda6-f1c7-47eb-b690-6c268b3000b3","Type":"ContainerStarted","Data":"df4e17c807d1afcdd97e12f0548a51c660f58f69c1b05c14a9054ba0a46491f6"} Dec 03 13:47:40 crc kubenswrapper[5052]: I1203 13:47:40.321903 5052 generic.go:334] "Generic (PLEG): container finished" podID="4cc70472-d066-4cab-bde9-9590e1bd74c2" containerID="e2ec56e1185bb58184a0288f43d74d64e9313b046bf22b0417a6962c4d1765c6" exitCode=0 Dec 03 13:47:40 crc kubenswrapper[5052]: I1203 13:47:40.321941 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/community-operators-ptbl7" event={"ID":"4cc70472-d066-4cab-bde9-9590e1bd74c2","Type":"ContainerDied","Data":"e2ec56e1185bb58184a0288f43d74d64e9313b046bf22b0417a6962c4d1765c6"} Dec 03 13:47:42 crc kubenswrapper[5052]: I1203 13:47:42.341414 5052 generic.go:334] "Generic (PLEG): container finished" podID="e369fda6-f1c7-47eb-b690-6c268b3000b3" containerID="80ff4dec9c6815b0b9fbf76ad52e18bd76eeee746310c83a9502b90087a6c930" exitCode=0 Dec 03 13:47:42 crc kubenswrapper[5052]: I1203 13:47:42.341574 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-marketplace-lg6kc" event={"ID":"e369fda6-f1c7-47eb-b690-6c268b3000b3","Type":"ContainerDied","Data":"80ff4dec9c6815b0b9fbf76ad52e18bd76eeee746310c83a9502b90087a6c930"} Dec 03 13:47:43 crc kubenswrapper[5052]: I1203 13:47:43.351348 5052 generic.go:334] "Generic (PLEG): container finished" podID="4cc70472-d066-4cab-bde9-9590e1bd74c2" containerID="c31891366f4945c27f51762beb027b8c89261ebadae51b5ad7889a0133980513" exitCode=0 Dec 03 13:47:43 crc kubenswrapper[5052]: I1203 13:47:43.351421 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/community-operators-ptbl7" event={"ID":"4cc70472-d066-4cab-bde9-9590e1bd74c2","Type":"ContainerDied","Data":"c31891366f4945c27f51762beb027b8c89261ebadae51b5ad7889a0133980513"} Dec 03 13:47:43 crc kubenswrapper[5052]: I1203 13:47:43.354655 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-marketplace-lg6kc" event={"ID":"e369fda6-f1c7-47eb-b690-6c268b3000b3","Type":"ContainerStarted","Data":"56fa32fc48221eee53ae9f20d122eb2389491b1a38c17eef6563d076048813d8"} Dec 03 13:47:43 crc kubenswrapper[5052]: I1203 13:47:43.390595 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-marketplace/redhat-marketplace-lg6kc" podStartSLOduration=2.996130021 podStartE2EDuration="5.390579031s" podCreationTimestamp="2025-12-03 13:47:38 +0000 UTC" firstStartedPulling="2025-12-03 13:47:40.32125148 +0000 UTC m=+8850.909493254" lastFinishedPulling="2025-12-03 13:47:42.7157005 +0000 UTC m=+8853.303942264" observedRunningTime="2025-12-03 13:47:43.388003657 +0000 UTC m=+8853.976245441" watchObservedRunningTime="2025-12-03 13:47:43.390579031 +0000 UTC m=+8853.978820795" Dec 03 13:47:44 crc kubenswrapper[5052]: I1203 13:47:44.362797 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/community-operators-ptbl7" event={"ID":"4cc70472-d066-4cab-bde9-9590e1bd74c2","Type":"ContainerStarted","Data":"81a993e689fdeda9fc0809bed7abc1b5ba7b3f69f308f374a725fd3cc1208492"} Dec 03 13:47:44 crc kubenswrapper[5052]: I1203 13:47:44.379325 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-marketplace/community-operators-ptbl7" podStartSLOduration=2.99893549 podStartE2EDuration="6.379310234s" podCreationTimestamp="2025-12-03 13:47:38 +0000 UTC" firstStartedPulling="2025-12-03 13:47:40.323038121 +0000 UTC m=+8850.911279885" lastFinishedPulling="2025-12-03 13:47:43.703412865 +0000 UTC m=+8854.291654629" observedRunningTime="2025-12-03 13:47:44.378383008 +0000 UTC m=+8854.966624812" watchObservedRunningTime="2025-12-03 13:47:44.379310234 +0000 UTC m=+8854.967551998" Dec 03 13:47:48 crc kubenswrapper[5052]: I1203 13:47:48.618492 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openshift-marketplace/community-operators-ptbl7" Dec 03 13:47:48 crc kubenswrapper[5052]: I1203 13:47:48.618832 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-marketplace/community-operators-ptbl7" Dec 03 13:47:48 crc kubenswrapper[5052]: I1203 13:47:48.665126 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-marketplace/community-operators-ptbl7" Dec 03 13:47:48 crc kubenswrapper[5052]: I1203 13:47:48.821410 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openshift-marketplace/redhat-marketplace-lg6kc" Dec 03 13:47:48 crc kubenswrapper[5052]: I1203 13:47:48.821472 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-marketplace/redhat-marketplace-lg6kc" Dec 03 13:47:48 crc kubenswrapper[5052]: I1203 13:47:48.861796 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-marketplace/redhat-marketplace-lg6kc" Dec 03 13:47:49 crc kubenswrapper[5052]: I1203 13:47:49.435671 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-marketplace/community-operators-ptbl7" Dec 03 13:47:49 crc kubenswrapper[5052]: I1203 13:47:49.436677 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-marketplace/redhat-marketplace-lg6kc" Dec 03 13:47:50 crc kubenswrapper[5052]: I1203 13:47:50.413592 5052 scope.go:117] "RemoveContainer" containerID="389d7bcd41004f53ddb05546d5ccd89efddb3c3608f638ed464fec31dc8be00f" Dec 03 13:47:50 crc kubenswrapper[5052]: I1203 13:47:50.447172 5052 scope.go:117] "RemoveContainer" containerID="2b5c7cb691c57e605cca4635b3f50a70ac794369cf573145396a541e9dbf3452" Dec 03 13:47:50 crc kubenswrapper[5052]: I1203 13:47:50.470154 5052 scope.go:117] "RemoveContainer" containerID="f5f0bb0a00d163138a9bdba06050a73ef3becaa2960669339132343450137116" Dec 03 13:47:50 crc kubenswrapper[5052]: I1203 13:47:50.496714 5052 scope.go:117] "RemoveContainer" containerID="15c5bcdbe9f2bd77b8f6616dc862afa27d22a4b1d007a7072eab0bb3d373ea96" Dec 03 13:47:50 crc kubenswrapper[5052]: I1203 13:47:50.522145 5052 scope.go:117] "RemoveContainer" containerID="c3d33d9edee4d4da19e64abf662598c36eceea009e210193291bdb4169fa0983" Dec 03 13:47:50 crc kubenswrapper[5052]: I1203 13:47:50.562832 5052 scope.go:117] "RemoveContainer" containerID="e13a734f46760dde236b0ad6acc81c9bd68e9050cb89d5e1a84ab2008f0ccb9c" Dec 03 13:47:50 crc kubenswrapper[5052]: I1203 13:47:50.576529 5052 scope.go:117] "RemoveContainer" containerID="d16a0d5eefe4a10e9077ee9312a1eeaa8cbc2a0b41d3f9cece70e26a2bb2f117" Dec 03 13:47:50 crc kubenswrapper[5052]: I1203 13:47:50.591854 5052 scope.go:117] "RemoveContainer" containerID="bf666d027e71853bcf0130e438fe6020c28e6d75ee676b585d4aba6febc4bc3e" Dec 03 13:47:50 crc kubenswrapper[5052]: I1203 13:47:50.613367 5052 scope.go:117] "RemoveContainer" containerID="a0d9a069ce3a3dc919e542b0eff35e9e7d9b2ddf5b1ace2db265d276d2873513" Dec 03 13:47:50 crc kubenswrapper[5052]: I1203 13:47:50.687943 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openshift-marketplace/redhat-marketplace-lg6kc"] Dec 03 13:47:50 crc kubenswrapper[5052]: I1203 13:47:50.886397 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openshift-marketplace/community-operators-ptbl7"] Dec 03 13:47:51 crc kubenswrapper[5052]: I1203 13:47:51.410689 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openshift-marketplace/community-operators-ptbl7" podUID="4cc70472-d066-4cab-bde9-9590e1bd74c2" containerName="registry-server" containerID="cri-o://81a993e689fdeda9fc0809bed7abc1b5ba7b3f69f308f374a725fd3cc1208492" gracePeriod=2 Dec 03 13:47:51 crc kubenswrapper[5052]: I1203 13:47:51.411143 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openshift-marketplace/redhat-marketplace-lg6kc" podUID="e369fda6-f1c7-47eb-b690-6c268b3000b3" containerName="registry-server" containerID="cri-o://56fa32fc48221eee53ae9f20d122eb2389491b1a38c17eef6563d076048813d8" gracePeriod=2 Dec 03 13:47:52 crc kubenswrapper[5052]: I1203 13:47:52.420585 5052 generic.go:334] "Generic (PLEG): container finished" podID="4cc70472-d066-4cab-bde9-9590e1bd74c2" containerID="81a993e689fdeda9fc0809bed7abc1b5ba7b3f69f308f374a725fd3cc1208492" exitCode=0 Dec 03 13:47:52 crc kubenswrapper[5052]: I1203 13:47:52.420662 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/community-operators-ptbl7" event={"ID":"4cc70472-d066-4cab-bde9-9590e1bd74c2","Type":"ContainerDied","Data":"81a993e689fdeda9fc0809bed7abc1b5ba7b3f69f308f374a725fd3cc1208492"} Dec 03 13:47:52 crc kubenswrapper[5052]: I1203 13:47:52.425255 5052 generic.go:334] "Generic (PLEG): container finished" podID="e369fda6-f1c7-47eb-b690-6c268b3000b3" containerID="56fa32fc48221eee53ae9f20d122eb2389491b1a38c17eef6563d076048813d8" exitCode=0 Dec 03 13:47:52 crc kubenswrapper[5052]: I1203 13:47:52.425301 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-marketplace-lg6kc" event={"ID":"e369fda6-f1c7-47eb-b690-6c268b3000b3","Type":"ContainerDied","Data":"56fa32fc48221eee53ae9f20d122eb2389491b1a38c17eef6563d076048813d8"} Dec 03 13:47:53 crc kubenswrapper[5052]: I1203 13:47:53.009184 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/redhat-marketplace-lg6kc" Dec 03 13:47:53 crc kubenswrapper[5052]: I1203 13:47:53.014711 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/community-operators-ptbl7" Dec 03 13:47:53 crc kubenswrapper[5052]: I1203 13:47:53.148805 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4jfkv\" (UniqueName: \"kubernetes.io/projected/4cc70472-d066-4cab-bde9-9590e1bd74c2-kube-api-access-4jfkv\") pod \"4cc70472-d066-4cab-bde9-9590e1bd74c2\" (UID: \"4cc70472-d066-4cab-bde9-9590e1bd74c2\") " Dec 03 13:47:53 crc kubenswrapper[5052]: I1203 13:47:53.148935 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/e369fda6-f1c7-47eb-b690-6c268b3000b3-utilities\") pod \"e369fda6-f1c7-47eb-b690-6c268b3000b3\" (UID: \"e369fda6-f1c7-47eb-b690-6c268b3000b3\") " Dec 03 13:47:53 crc kubenswrapper[5052]: I1203 13:47:53.149008 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/4cc70472-d066-4cab-bde9-9590e1bd74c2-utilities\") pod \"4cc70472-d066-4cab-bde9-9590e1bd74c2\" (UID: \"4cc70472-d066-4cab-bde9-9590e1bd74c2\") " Dec 03 13:47:53 crc kubenswrapper[5052]: I1203 13:47:53.149059 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g7l79\" (UniqueName: \"kubernetes.io/projected/e369fda6-f1c7-47eb-b690-6c268b3000b3-kube-api-access-g7l79\") pod \"e369fda6-f1c7-47eb-b690-6c268b3000b3\" (UID: \"e369fda6-f1c7-47eb-b690-6c268b3000b3\") " Dec 03 13:47:53 crc kubenswrapper[5052]: I1203 13:47:53.149124 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/e369fda6-f1c7-47eb-b690-6c268b3000b3-catalog-content\") pod \"e369fda6-f1c7-47eb-b690-6c268b3000b3\" (UID: \"e369fda6-f1c7-47eb-b690-6c268b3000b3\") " Dec 03 13:47:53 crc kubenswrapper[5052]: I1203 13:47:53.149300 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/4cc70472-d066-4cab-bde9-9590e1bd74c2-catalog-content\") pod \"4cc70472-d066-4cab-bde9-9590e1bd74c2\" (UID: \"4cc70472-d066-4cab-bde9-9590e1bd74c2\") " Dec 03 13:47:53 crc kubenswrapper[5052]: I1203 13:47:53.150030 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e369fda6-f1c7-47eb-b690-6c268b3000b3-utilities" (OuterVolumeSpecName: "utilities") pod "e369fda6-f1c7-47eb-b690-6c268b3000b3" (UID: "e369fda6-f1c7-47eb-b690-6c268b3000b3"). InnerVolumeSpecName "utilities". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 13:47:53 crc kubenswrapper[5052]: I1203 13:47:53.150877 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4cc70472-d066-4cab-bde9-9590e1bd74c2-utilities" (OuterVolumeSpecName: "utilities") pod "4cc70472-d066-4cab-bde9-9590e1bd74c2" (UID: "4cc70472-d066-4cab-bde9-9590e1bd74c2"). InnerVolumeSpecName "utilities". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 13:47:53 crc kubenswrapper[5052]: I1203 13:47:53.159100 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4cc70472-d066-4cab-bde9-9590e1bd74c2-kube-api-access-4jfkv" (OuterVolumeSpecName: "kube-api-access-4jfkv") pod "4cc70472-d066-4cab-bde9-9590e1bd74c2" (UID: "4cc70472-d066-4cab-bde9-9590e1bd74c2"). InnerVolumeSpecName "kube-api-access-4jfkv". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:47:53 crc kubenswrapper[5052]: I1203 13:47:53.159462 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e369fda6-f1c7-47eb-b690-6c268b3000b3-kube-api-access-g7l79" (OuterVolumeSpecName: "kube-api-access-g7l79") pod "e369fda6-f1c7-47eb-b690-6c268b3000b3" (UID: "e369fda6-f1c7-47eb-b690-6c268b3000b3"). InnerVolumeSpecName "kube-api-access-g7l79". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:47:53 crc kubenswrapper[5052]: I1203 13:47:53.179780 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e369fda6-f1c7-47eb-b690-6c268b3000b3-catalog-content" (OuterVolumeSpecName: "catalog-content") pod "e369fda6-f1c7-47eb-b690-6c268b3000b3" (UID: "e369fda6-f1c7-47eb-b690-6c268b3000b3"). InnerVolumeSpecName "catalog-content". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 13:47:53 crc kubenswrapper[5052]: I1203 13:47:53.223868 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4cc70472-d066-4cab-bde9-9590e1bd74c2-catalog-content" (OuterVolumeSpecName: "catalog-content") pod "4cc70472-d066-4cab-bde9-9590e1bd74c2" (UID: "4cc70472-d066-4cab-bde9-9590e1bd74c2"). InnerVolumeSpecName "catalog-content". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 13:47:53 crc kubenswrapper[5052]: I1203 13:47:53.251180 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-4jfkv\" (UniqueName: \"kubernetes.io/projected/4cc70472-d066-4cab-bde9-9590e1bd74c2-kube-api-access-4jfkv\") on node \"crc\" DevicePath \"\"" Dec 03 13:47:53 crc kubenswrapper[5052]: I1203 13:47:53.251227 5052 reconciler_common.go:293] "Volume detached for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/e369fda6-f1c7-47eb-b690-6c268b3000b3-utilities\") on node \"crc\" DevicePath \"\"" Dec 03 13:47:53 crc kubenswrapper[5052]: I1203 13:47:53.251240 5052 reconciler_common.go:293] "Volume detached for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/4cc70472-d066-4cab-bde9-9590e1bd74c2-utilities\") on node \"crc\" DevicePath \"\"" Dec 03 13:47:53 crc kubenswrapper[5052]: I1203 13:47:53.251253 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-g7l79\" (UniqueName: \"kubernetes.io/projected/e369fda6-f1c7-47eb-b690-6c268b3000b3-kube-api-access-g7l79\") on node \"crc\" DevicePath \"\"" Dec 03 13:47:53 crc kubenswrapper[5052]: I1203 13:47:53.251267 5052 reconciler_common.go:293] "Volume detached for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/e369fda6-f1c7-47eb-b690-6c268b3000b3-catalog-content\") on node \"crc\" DevicePath \"\"" Dec 03 13:47:53 crc kubenswrapper[5052]: I1203 13:47:53.251280 5052 reconciler_common.go:293] "Volume detached for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/4cc70472-d066-4cab-bde9-9590e1bd74c2-catalog-content\") on node \"crc\" DevicePath \"\"" Dec 03 13:47:53 crc kubenswrapper[5052]: I1203 13:47:53.434484 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/community-operators-ptbl7" Dec 03 13:47:53 crc kubenswrapper[5052]: I1203 13:47:53.434478 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/community-operators-ptbl7" event={"ID":"4cc70472-d066-4cab-bde9-9590e1bd74c2","Type":"ContainerDied","Data":"5a787ce7b5d223a6c3b1042160fa3e1371ee80b4e60472f01a540ad041b0f611"} Dec 03 13:47:53 crc kubenswrapper[5052]: I1203 13:47:53.434640 5052 scope.go:117] "RemoveContainer" containerID="81a993e689fdeda9fc0809bed7abc1b5ba7b3f69f308f374a725fd3cc1208492" Dec 03 13:47:53 crc kubenswrapper[5052]: I1203 13:47:53.437294 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-marketplace-lg6kc" event={"ID":"e369fda6-f1c7-47eb-b690-6c268b3000b3","Type":"ContainerDied","Data":"df4e17c807d1afcdd97e12f0548a51c660f58f69c1b05c14a9054ba0a46491f6"} Dec 03 13:47:53 crc kubenswrapper[5052]: I1203 13:47:53.437398 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/redhat-marketplace-lg6kc" Dec 03 13:47:53 crc kubenswrapper[5052]: I1203 13:47:53.457750 5052 scope.go:117] "RemoveContainer" containerID="c31891366f4945c27f51762beb027b8c89261ebadae51b5ad7889a0133980513" Dec 03 13:47:53 crc kubenswrapper[5052]: I1203 13:47:53.477341 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openshift-marketplace/redhat-marketplace-lg6kc"] Dec 03 13:47:53 crc kubenswrapper[5052]: I1203 13:47:53.481949 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openshift-marketplace/redhat-marketplace-lg6kc"] Dec 03 13:47:53 crc kubenswrapper[5052]: I1203 13:47:53.487749 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openshift-marketplace/community-operators-ptbl7"] Dec 03 13:47:53 crc kubenswrapper[5052]: I1203 13:47:53.490998 5052 scope.go:117] "RemoveContainer" containerID="e2ec56e1185bb58184a0288f43d74d64e9313b046bf22b0417a6962c4d1765c6" Dec 03 13:47:53 crc kubenswrapper[5052]: I1203 13:47:53.493680 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openshift-marketplace/community-operators-ptbl7"] Dec 03 13:47:53 crc kubenswrapper[5052]: I1203 13:47:53.506896 5052 scope.go:117] "RemoveContainer" containerID="56fa32fc48221eee53ae9f20d122eb2389491b1a38c17eef6563d076048813d8" Dec 03 13:47:53 crc kubenswrapper[5052]: I1203 13:47:53.524150 5052 scope.go:117] "RemoveContainer" containerID="80ff4dec9c6815b0b9fbf76ad52e18bd76eeee746310c83a9502b90087a6c930" Dec 03 13:47:53 crc kubenswrapper[5052]: I1203 13:47:53.539110 5052 scope.go:117] "RemoveContainer" containerID="c66159fdb0b1b367e7b63df8ddafb3d826e53e6e468ecbd3f3a73b27b6589896" Dec 03 13:47:53 crc kubenswrapper[5052]: I1203 13:47:53.681632 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4cc70472-d066-4cab-bde9-9590e1bd74c2" path="/var/lib/kubelet/pods/4cc70472-d066-4cab-bde9-9590e1bd74c2/volumes" Dec 03 13:47:53 crc kubenswrapper[5052]: I1203 13:47:53.683651 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e369fda6-f1c7-47eb-b690-6c268b3000b3" path="/var/lib/kubelet/pods/e369fda6-f1c7-47eb-b690-6c268b3000b3/volumes" Dec 03 13:48:30 crc kubenswrapper[5052]: I1203 13:48:30.987990 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-machine-api_control-plane-machine-set-operator-78cbb6b69f-p4b4l_693b46e6-42c3-4d07-bfbd-bd83d6982199/control-plane-machine-set-operator/0.log" Dec 03 13:48:31 crc kubenswrapper[5052]: I1203 13:48:31.213219 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-machine-api_machine-api-operator-5694c8668f-l5lrg_3d483046-743f-43c6-b73d-31f4f1bcca8c/kube-rbac-proxy/0.log" Dec 03 13:48:31 crc kubenswrapper[5052]: I1203 13:48:31.240179 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-machine-api_machine-api-operator-5694c8668f-l5lrg_3d483046-743f-43c6-b73d-31f4f1bcca8c/machine-api-operator/0.log" Dec 03 13:48:44 crc kubenswrapper[5052]: I1203 13:48:44.161044 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/cert-manager_cert-manager-webhook-f4fb5df64-z8l9s_cca488df-be0b-41f9-bd69-edf11dd061be/cert-manager-webhook/0.log" Dec 03 13:48:44 crc kubenswrapper[5052]: I1203 13:48:44.163718 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/cert-manager_cert-manager-86cb77c54b-jdfq2_15312175-c609-4674-a597-a520517b3d0f/cert-manager-controller/0.log" Dec 03 13:48:44 crc kubenswrapper[5052]: I1203 13:48:44.207686 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/cert-manager_cert-manager-cainjector-855d9ccff4-5pthb_03732802-639c-4591-b3bf-07bc550527f3/cert-manager-cainjector/0.log" Dec 03 13:48:47 crc kubenswrapper[5052]: I1203 13:48:47.122875 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-machine-api_control-plane-machine-set-operator-78cbb6b69f-p4b4l_693b46e6-42c3-4d07-bfbd-bd83d6982199/control-plane-machine-set-operator/0.log" Dec 03 13:48:47 crc kubenswrapper[5052]: I1203 13:48:47.277917 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-machine-api_machine-api-operator-5694c8668f-l5lrg_3d483046-743f-43c6-b73d-31f4f1bcca8c/kube-rbac-proxy/0.log" Dec 03 13:48:47 crc kubenswrapper[5052]: I1203 13:48:47.290692 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-machine-api_machine-api-operator-5694c8668f-l5lrg_3d483046-743f-43c6-b73d-31f4f1bcca8c/machine-api-operator/0.log" Dec 03 13:48:50 crc kubenswrapper[5052]: I1203 13:48:50.850358 5052 scope.go:117] "RemoveContainer" containerID="5aaf8ba60f80237776fb309acc3491ff2ae50dd27b76fff4ff321dbcc6831e02" Dec 03 13:48:50 crc kubenswrapper[5052]: I1203 13:48:50.866982 5052 scope.go:117] "RemoveContainer" containerID="9989e568af0ffc2e95e8d5ddca5df126f8d6f9d4472fda82276b45c89dd4020f" Dec 03 13:48:50 crc kubenswrapper[5052]: I1203 13:48:50.897015 5052 scope.go:117] "RemoveContainer" containerID="3d757c16532fe237e6034dd78dfe2e5e41a58f7d22ea0da43944853c350904f9" Dec 03 13:48:50 crc kubenswrapper[5052]: I1203 13:48:50.917975 5052 scope.go:117] "RemoveContainer" containerID="047dc47b555b7198544f9325a376136bd2cc325989f21b37c2d5d61d89f2b421" Dec 03 13:48:50 crc kubenswrapper[5052]: I1203 13:48:50.938999 5052 scope.go:117] "RemoveContainer" containerID="92049bf81ac84c4ad82c46c284e670bf4586778ae001e3942eb6bb28c4b60821" Dec 03 13:48:50 crc kubenswrapper[5052]: I1203 13:48:50.964017 5052 scope.go:117] "RemoveContainer" containerID="33dc678fd0c9e2a495d5335a1ad34ac104752fa08706d562d239fe848a24978c" Dec 03 13:48:58 crc kubenswrapper[5052]: I1203 13:48:58.667281 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-nmstate_nmstate-console-plugin-7fbb5f6569-br7vf_04ad7b19-29ef-46b9-8668-b5437b6c88cc/nmstate-console-plugin/0.log" Dec 03 13:48:58 crc kubenswrapper[5052]: I1203 13:48:58.853604 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-nmstate_nmstate-handler-qggff_318304dc-2b11-480e-8683-7627f288faa8/nmstate-handler/0.log" Dec 03 13:48:59 crc kubenswrapper[5052]: I1203 13:48:59.005870 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-nmstate_nmstate-metrics-7f946cbc9-ffkk9_bdc64cdc-4e09-4eff-b1bc-23b9dd86ab8b/kube-rbac-proxy/0.log" Dec 03 13:48:59 crc kubenswrapper[5052]: I1203 13:48:59.040834 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-nmstate_nmstate-metrics-7f946cbc9-ffkk9_bdc64cdc-4e09-4eff-b1bc-23b9dd86ab8b/nmstate-metrics/0.log" Dec 03 13:48:59 crc kubenswrapper[5052]: I1203 13:48:59.141522 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-nmstate_nmstate-operator-5b5b58f5c8-8bhqp_11ca0a66-5f66-4ff4-a6d2-77048290159d/nmstate-operator/0.log" Dec 03 13:48:59 crc kubenswrapper[5052]: I1203 13:48:59.258708 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-nmstate_nmstate-webhook-5f6d4c5ccb-gbkms_f60b6277-113c-459c-801b-aa5c94de641a/nmstate-webhook/0.log" Dec 03 13:48:59 crc kubenswrapper[5052]: I1203 13:48:59.379393 5052 patch_prober.go:28] interesting pod/machine-config-daemon-74bj6 container/machine-config-daemon namespace/openshift-machine-config-operator: Liveness probe status=failure output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" start-of-body= Dec 03 13:48:59 crc kubenswrapper[5052]: I1203 13:48:59.379475 5052 prober.go:107] "Probe failed" probeType="Liveness" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" containerName="machine-config-daemon" probeResult="failure" output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" Dec 03 13:49:02 crc kubenswrapper[5052]: I1203 13:49:02.042850 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/cert-manager_cert-manager-86cb77c54b-jdfq2_15312175-c609-4674-a597-a520517b3d0f/cert-manager-controller/0.log" Dec 03 13:49:02 crc kubenswrapper[5052]: I1203 13:49:02.125553 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/cert-manager_cert-manager-webhook-f4fb5df64-z8l9s_cca488df-be0b-41f9-bd69-edf11dd061be/cert-manager-webhook/0.log" Dec 03 13:49:02 crc kubenswrapper[5052]: I1203 13:49:02.141430 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/cert-manager_cert-manager-cainjector-855d9ccff4-5pthb_03732802-639c-4591-b3bf-07bc550527f3/cert-manager-cainjector/0.log" Dec 03 13:49:16 crc kubenswrapper[5052]: I1203 13:49:16.152765 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/metallb-system_controller-f8648f98b-nc9sj_3d2671fe-8f61-40ab-8ebf-3c7d8fe99644/kube-rbac-proxy/0.log" Dec 03 13:49:16 crc kubenswrapper[5052]: I1203 13:49:16.394707 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/metallb-system_controller-f8648f98b-nc9sj_3d2671fe-8f61-40ab-8ebf-3c7d8fe99644/controller/0.log" Dec 03 13:49:16 crc kubenswrapper[5052]: I1203 13:49:16.496344 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/metallb-system_frr-k8s-47pp5_76bb2cd3-e75d-40f1-a9e6-63f7bd725eea/cp-frr-files/0.log" Dec 03 13:49:16 crc kubenswrapper[5052]: I1203 13:49:16.583572 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/metallb-system_frr-k8s-47pp5_76bb2cd3-e75d-40f1-a9e6-63f7bd725eea/cp-frr-files/0.log" Dec 03 13:49:16 crc kubenswrapper[5052]: I1203 13:49:16.633335 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/metallb-system_frr-k8s-47pp5_76bb2cd3-e75d-40f1-a9e6-63f7bd725eea/cp-reloader/0.log" Dec 03 13:49:16 crc kubenswrapper[5052]: I1203 13:49:16.807005 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/metallb-system_frr-k8s-47pp5_76bb2cd3-e75d-40f1-a9e6-63f7bd725eea/cp-metrics/0.log" Dec 03 13:49:16 crc kubenswrapper[5052]: I1203 13:49:16.822767 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/metallb-system_frr-k8s-47pp5_76bb2cd3-e75d-40f1-a9e6-63f7bd725eea/cp-reloader/0.log" Dec 03 13:49:17 crc kubenswrapper[5052]: I1203 13:49:17.031794 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-nmstate_nmstate-console-plugin-7fbb5f6569-br7vf_04ad7b19-29ef-46b9-8668-b5437b6c88cc/nmstate-console-plugin/0.log" Dec 03 13:49:17 crc kubenswrapper[5052]: I1203 13:49:17.080863 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/metallb-system_frr-k8s-47pp5_76bb2cd3-e75d-40f1-a9e6-63f7bd725eea/cp-reloader/0.log" Dec 03 13:49:17 crc kubenswrapper[5052]: I1203 13:49:17.114885 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/metallb-system_frr-k8s-47pp5_76bb2cd3-e75d-40f1-a9e6-63f7bd725eea/cp-frr-files/0.log" Dec 03 13:49:17 crc kubenswrapper[5052]: I1203 13:49:17.196132 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/metallb-system_frr-k8s-47pp5_76bb2cd3-e75d-40f1-a9e6-63f7bd725eea/cp-metrics/0.log" Dec 03 13:49:17 crc kubenswrapper[5052]: I1203 13:49:17.203226 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/metallb-system_frr-k8s-47pp5_76bb2cd3-e75d-40f1-a9e6-63f7bd725eea/cp-metrics/0.log" Dec 03 13:49:17 crc kubenswrapper[5052]: I1203 13:49:17.345742 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-nmstate_nmstate-handler-qggff_318304dc-2b11-480e-8683-7627f288faa8/nmstate-handler/0.log" Dec 03 13:49:17 crc kubenswrapper[5052]: I1203 13:49:17.450679 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-nmstate_nmstate-metrics-7f946cbc9-ffkk9_bdc64cdc-4e09-4eff-b1bc-23b9dd86ab8b/kube-rbac-proxy/0.log" Dec 03 13:49:17 crc kubenswrapper[5052]: I1203 13:49:17.463059 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/metallb-system_frr-k8s-47pp5_76bb2cd3-e75d-40f1-a9e6-63f7bd725eea/cp-frr-files/0.log" Dec 03 13:49:17 crc kubenswrapper[5052]: I1203 13:49:17.465127 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/metallb-system_frr-k8s-47pp5_76bb2cd3-e75d-40f1-a9e6-63f7bd725eea/cp-reloader/0.log" Dec 03 13:49:17 crc kubenswrapper[5052]: I1203 13:49:17.504846 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/metallb-system_frr-k8s-47pp5_76bb2cd3-e75d-40f1-a9e6-63f7bd725eea/cp-metrics/0.log" Dec 03 13:49:17 crc kubenswrapper[5052]: I1203 13:49:17.547087 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-nmstate_nmstate-metrics-7f946cbc9-ffkk9_bdc64cdc-4e09-4eff-b1bc-23b9dd86ab8b/nmstate-metrics/0.log" Dec 03 13:49:17 crc kubenswrapper[5052]: I1203 13:49:17.563807 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/metallb-system_frr-k8s-47pp5_76bb2cd3-e75d-40f1-a9e6-63f7bd725eea/controller/0.log" Dec 03 13:49:17 crc kubenswrapper[5052]: I1203 13:49:17.636712 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-nmstate_nmstate-operator-5b5b58f5c8-8bhqp_11ca0a66-5f66-4ff4-a6d2-77048290159d/nmstate-operator/0.log" Dec 03 13:49:17 crc kubenswrapper[5052]: I1203 13:49:17.747866 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/metallb-system_frr-k8s-47pp5_76bb2cd3-e75d-40f1-a9e6-63f7bd725eea/kube-rbac-proxy/0.log" Dec 03 13:49:17 crc kubenswrapper[5052]: I1203 13:49:17.838812 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/metallb-system_frr-k8s-47pp5_76bb2cd3-e75d-40f1-a9e6-63f7bd725eea/frr-metrics/0.log" Dec 03 13:49:17 crc kubenswrapper[5052]: I1203 13:49:17.919799 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-nmstate_nmstate-webhook-5f6d4c5ccb-gbkms_f60b6277-113c-459c-801b-aa5c94de641a/nmstate-webhook/0.log" Dec 03 13:49:17 crc kubenswrapper[5052]: I1203 13:49:17.946577 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/metallb-system_frr-k8s-47pp5_76bb2cd3-e75d-40f1-a9e6-63f7bd725eea/kube-rbac-proxy-frr/0.log" Dec 03 13:49:18 crc kubenswrapper[5052]: I1203 13:49:18.144546 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/metallb-system_frr-k8s-47pp5_76bb2cd3-e75d-40f1-a9e6-63f7bd725eea/reloader/0.log" Dec 03 13:49:18 crc kubenswrapper[5052]: I1203 13:49:18.260665 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/metallb-system_frr-k8s-webhook-server-7fcb986d4-66rvv_c89171df-ff1e-4bc6-8e4e-9c24dc8ee436/frr-k8s-webhook-server/0.log" Dec 03 13:49:18 crc kubenswrapper[5052]: I1203 13:49:18.437147 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/metallb-system_metallb-operator-controller-manager-b9dd667db-j4qd7_ba0d3ff2-b21c-4788-9dad-6fb7f16a6cb8/manager/0.log" Dec 03 13:49:18 crc kubenswrapper[5052]: I1203 13:49:18.684406 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/metallb-system_metallb-operator-webhook-server-5b4bcb5c5f-ttnfj_74df1e0f-e3ec-4206-9e30-a99b914dd282/webhook-server/0.log" Dec 03 13:49:18 crc kubenswrapper[5052]: I1203 13:49:18.912271 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/metallb-system_speaker-56dnr_92c3adf7-3dc8-4644-9fb7-f68994747973/kube-rbac-proxy/0.log" Dec 03 13:49:18 crc kubenswrapper[5052]: I1203 13:49:18.948784 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/metallb-system_frr-k8s-47pp5_76bb2cd3-e75d-40f1-a9e6-63f7bd725eea/frr/0.log" Dec 03 13:49:20 crc kubenswrapper[5052]: I1203 13:49:20.164121 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/metallb-system_speaker-56dnr_92c3adf7-3dc8-4644-9fb7-f68994747973/speaker/0.log" Dec 03 13:49:29 crc kubenswrapper[5052]: I1203 13:49:29.379264 5052 patch_prober.go:28] interesting pod/machine-config-daemon-74bj6 container/machine-config-daemon namespace/openshift-machine-config-operator: Liveness probe status=failure output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" start-of-body= Dec 03 13:49:29 crc kubenswrapper[5052]: I1203 13:49:29.379922 5052 prober.go:107] "Probe failed" probeType="Liveness" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" containerName="machine-config-daemon" probeResult="failure" output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" Dec 03 13:49:35 crc kubenswrapper[5052]: I1203 13:49:35.233136 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/metallb-system_controller-f8648f98b-nc9sj_3d2671fe-8f61-40ab-8ebf-3c7d8fe99644/kube-rbac-proxy/0.log" Dec 03 13:49:35 crc kubenswrapper[5052]: I1203 13:49:35.503080 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/metallb-system_frr-k8s-47pp5_76bb2cd3-e75d-40f1-a9e6-63f7bd725eea/cp-frr-files/0.log" Dec 03 13:49:35 crc kubenswrapper[5052]: I1203 13:49:35.503525 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/metallb-system_controller-f8648f98b-nc9sj_3d2671fe-8f61-40ab-8ebf-3c7d8fe99644/controller/0.log" Dec 03 13:49:35 crc kubenswrapper[5052]: I1203 13:49:35.721013 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/metallb-system_frr-k8s-47pp5_76bb2cd3-e75d-40f1-a9e6-63f7bd725eea/cp-reloader/0.log" Dec 03 13:49:35 crc kubenswrapper[5052]: I1203 13:49:35.721025 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/metallb-system_frr-k8s-47pp5_76bb2cd3-e75d-40f1-a9e6-63f7bd725eea/cp-frr-files/0.log" Dec 03 13:49:35 crc kubenswrapper[5052]: I1203 13:49:35.749109 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/metallb-system_frr-k8s-47pp5_76bb2cd3-e75d-40f1-a9e6-63f7bd725eea/cp-reloader/0.log" Dec 03 13:49:35 crc kubenswrapper[5052]: I1203 13:49:35.760599 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/metallb-system_frr-k8s-47pp5_76bb2cd3-e75d-40f1-a9e6-63f7bd725eea/cp-metrics/0.log" Dec 03 13:49:35 crc kubenswrapper[5052]: I1203 13:49:35.998461 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/metallb-system_frr-k8s-47pp5_76bb2cd3-e75d-40f1-a9e6-63f7bd725eea/cp-frr-files/0.log" Dec 03 13:49:36 crc kubenswrapper[5052]: I1203 13:49:36.002163 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/metallb-system_frr-k8s-47pp5_76bb2cd3-e75d-40f1-a9e6-63f7bd725eea/cp-metrics/0.log" Dec 03 13:49:36 crc kubenswrapper[5052]: I1203 13:49:36.036179 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/metallb-system_frr-k8s-47pp5_76bb2cd3-e75d-40f1-a9e6-63f7bd725eea/cp-reloader/0.log" Dec 03 13:49:36 crc kubenswrapper[5052]: I1203 13:49:36.054971 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/metallb-system_frr-k8s-47pp5_76bb2cd3-e75d-40f1-a9e6-63f7bd725eea/cp-metrics/0.log" Dec 03 13:49:36 crc kubenswrapper[5052]: I1203 13:49:36.265836 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/metallb-system_frr-k8s-47pp5_76bb2cd3-e75d-40f1-a9e6-63f7bd725eea/cp-reloader/0.log" Dec 03 13:49:36 crc kubenswrapper[5052]: I1203 13:49:36.299043 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/metallb-system_frr-k8s-47pp5_76bb2cd3-e75d-40f1-a9e6-63f7bd725eea/cp-metrics/0.log" Dec 03 13:49:36 crc kubenswrapper[5052]: I1203 13:49:36.304959 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/metallb-system_frr-k8s-47pp5_76bb2cd3-e75d-40f1-a9e6-63f7bd725eea/cp-frr-files/0.log" Dec 03 13:49:36 crc kubenswrapper[5052]: I1203 13:49:36.309991 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/metallb-system_frr-k8s-47pp5_76bb2cd3-e75d-40f1-a9e6-63f7bd725eea/controller/0.log" Dec 03 13:49:36 crc kubenswrapper[5052]: I1203 13:49:36.530442 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/metallb-system_frr-k8s-47pp5_76bb2cd3-e75d-40f1-a9e6-63f7bd725eea/kube-rbac-proxy-frr/0.log" Dec 03 13:49:36 crc kubenswrapper[5052]: I1203 13:49:36.551485 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/metallb-system_frr-k8s-47pp5_76bb2cd3-e75d-40f1-a9e6-63f7bd725eea/kube-rbac-proxy/0.log" Dec 03 13:49:36 crc kubenswrapper[5052]: I1203 13:49:36.557477 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/metallb-system_frr-k8s-47pp5_76bb2cd3-e75d-40f1-a9e6-63f7bd725eea/frr-metrics/0.log" Dec 03 13:49:36 crc kubenswrapper[5052]: I1203 13:49:36.773002 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/metallb-system_frr-k8s-47pp5_76bb2cd3-e75d-40f1-a9e6-63f7bd725eea/reloader/0.log" Dec 03 13:49:36 crc kubenswrapper[5052]: I1203 13:49:36.774796 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/metallb-system_frr-k8s-webhook-server-7fcb986d4-66rvv_c89171df-ff1e-4bc6-8e4e-9c24dc8ee436/frr-k8s-webhook-server/0.log" Dec 03 13:49:37 crc kubenswrapper[5052]: I1203 13:49:37.396257 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/metallb-system_metallb-operator-controller-manager-b9dd667db-j4qd7_ba0d3ff2-b21c-4788-9dad-6fb7f16a6cb8/manager/0.log" Dec 03 13:49:37 crc kubenswrapper[5052]: I1203 13:49:37.676323 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/metallb-system_speaker-56dnr_92c3adf7-3dc8-4644-9fb7-f68994747973/kube-rbac-proxy/0.log" Dec 03 13:49:37 crc kubenswrapper[5052]: I1203 13:49:37.690854 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/metallb-system_metallb-operator-webhook-server-5b4bcb5c5f-ttnfj_74df1e0f-e3ec-4206-9e30-a99b914dd282/webhook-server/0.log" Dec 03 13:49:37 crc kubenswrapper[5052]: I1203 13:49:37.877275 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/metallb-system_frr-k8s-47pp5_76bb2cd3-e75d-40f1-a9e6-63f7bd725eea/frr/0.log" Dec 03 13:49:38 crc kubenswrapper[5052]: I1203 13:49:38.966355 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/metallb-system_speaker-56dnr_92c3adf7-3dc8-4644-9fb7-f68994747973/speaker/0.log" Dec 03 13:49:51 crc kubenswrapper[5052]: I1203 13:49:51.054467 5052 scope.go:117] "RemoveContainer" containerID="c71d00af602e633f64ea6831c33edeffdc6184aaca0eacd47120abd41c85aa94" Dec 03 13:49:51 crc kubenswrapper[5052]: I1203 13:49:51.135462 5052 scope.go:117] "RemoveContainer" containerID="d7f0c01793bafc3fd7fbc4d56442c47f7eb67bf7b561a0a12c62f9e020f177ad" Dec 03 13:49:51 crc kubenswrapper[5052]: I1203 13:49:51.184727 5052 scope.go:117] "RemoveContainer" containerID="76b042da28713d09345ef0b170c843ae2fb4756281a64e7bda9fae75dc908335" Dec 03 13:49:51 crc kubenswrapper[5052]: I1203 13:49:51.210632 5052 scope.go:117] "RemoveContainer" containerID="44d0a04f74d3dfb319e297b561d5e3800514c62116cc94cb9090d80eb040c731" Dec 03 13:49:51 crc kubenswrapper[5052]: I1203 13:49:51.245184 5052 scope.go:117] "RemoveContainer" containerID="c264afb73103dca4a9c8be43892dd84ddc4910a887544320f95185671931cd3e" Dec 03 13:49:59 crc kubenswrapper[5052]: I1203 13:49:59.379432 5052 patch_prober.go:28] interesting pod/machine-config-daemon-74bj6 container/machine-config-daemon namespace/openshift-machine-config-operator: Liveness probe status=failure output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" start-of-body= Dec 03 13:49:59 crc kubenswrapper[5052]: I1203 13:49:59.380069 5052 prober.go:107] "Probe failed" probeType="Liveness" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" containerName="machine-config-daemon" probeResult="failure" output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" Dec 03 13:49:59 crc kubenswrapper[5052]: I1203 13:49:59.380125 5052 kubelet.go:2542] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" Dec 03 13:49:59 crc kubenswrapper[5052]: I1203 13:49:59.380846 5052 kuberuntime_manager.go:1027] "Message for Container of pod" containerName="machine-config-daemon" containerStatusID={"Type":"cri-o","ID":"af948543b565b59ee5a764bc4f80d394be230d8c49adc6125d06e305c9dffa71"} pod="openshift-machine-config-operator/machine-config-daemon-74bj6" containerMessage="Container machine-config-daemon failed liveness probe, will be restarted" Dec 03 13:49:59 crc kubenswrapper[5052]: I1203 13:49:59.380911 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" containerName="machine-config-daemon" containerID="cri-o://af948543b565b59ee5a764bc4f80d394be230d8c49adc6125d06e305c9dffa71" gracePeriod=600 Dec 03 13:50:00 crc kubenswrapper[5052]: I1203 13:50:00.263668 5052 generic.go:334] "Generic (PLEG): container finished" podID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" containerID="af948543b565b59ee5a764bc4f80d394be230d8c49adc6125d06e305c9dffa71" exitCode=0 Dec 03 13:50:00 crc kubenswrapper[5052]: I1203 13:50:00.263733 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" event={"ID":"10c237bc-72e7-4da3-9f2c-b1688395bf6e","Type":"ContainerDied","Data":"af948543b565b59ee5a764bc4f80d394be230d8c49adc6125d06e305c9dffa71"} Dec 03 13:50:00 crc kubenswrapper[5052]: I1203 13:50:00.264044 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" event={"ID":"10c237bc-72e7-4da3-9f2c-b1688395bf6e","Type":"ContainerStarted","Data":"a4c6deffa55d4faa49789e30d4c8b2762d7fc719390c909181ec730f718e0556"} Dec 03 13:50:00 crc kubenswrapper[5052]: I1203 13:50:00.264062 5052 scope.go:117] "RemoveContainer" containerID="2fbd293d6edef0fa059e91c119312e77b262534107c322feb7b59dbea3464526" Dec 03 13:50:16 crc kubenswrapper[5052]: I1203 13:50:16.544221 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_dnsmasq-dnsmasq-84b9f45d47-d8nh7_84b4fa9a-0498-4663-8ed2-2609ba3e2f88/init/0.log" Dec 03 13:50:16 crc kubenswrapper[5052]: I1203 13:50:16.858357 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_dnsmasq-dnsmasq-84b9f45d47-d8nh7_84b4fa9a-0498-4663-8ed2-2609ba3e2f88/init/0.log" Dec 03 13:50:16 crc kubenswrapper[5052]: I1203 13:50:16.875971 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_dnsmasq-dnsmasq-84b9f45d47-d8nh7_84b4fa9a-0498-4663-8ed2-2609ba3e2f88/dnsmasq-dns/0.log" Dec 03 13:50:32 crc kubenswrapper[5052]: I1203 13:50:32.613120 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_1f59f640c8a0eb1a7b0f26c81382bbdde784d03eb439a940bb8da3931aj27fz_c121fea4-2ada-4c07-8cbb-06d0ab09b325/util/0.log" Dec 03 13:50:32 crc kubenswrapper[5052]: I1203 13:50:32.732690 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_1f59f640c8a0eb1a7b0f26c81382bbdde784d03eb439a940bb8da3931aj27fz_c121fea4-2ada-4c07-8cbb-06d0ab09b325/util/0.log" Dec 03 13:50:32 crc kubenswrapper[5052]: I1203 13:50:32.799265 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_1f59f640c8a0eb1a7b0f26c81382bbdde784d03eb439a940bb8da3931aj27fz_c121fea4-2ada-4c07-8cbb-06d0ab09b325/pull/0.log" Dec 03 13:50:32 crc kubenswrapper[5052]: I1203 13:50:32.865982 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_1f59f640c8a0eb1a7b0f26c81382bbdde784d03eb439a940bb8da3931aj27fz_c121fea4-2ada-4c07-8cbb-06d0ab09b325/pull/0.log" Dec 03 13:50:33 crc kubenswrapper[5052]: I1203 13:50:33.038418 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_1f59f640c8a0eb1a7b0f26c81382bbdde784d03eb439a940bb8da3931aj27fz_c121fea4-2ada-4c07-8cbb-06d0ab09b325/pull/0.log" Dec 03 13:50:33 crc kubenswrapper[5052]: I1203 13:50:33.041300 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_1f59f640c8a0eb1a7b0f26c81382bbdde784d03eb439a940bb8da3931aj27fz_c121fea4-2ada-4c07-8cbb-06d0ab09b325/util/0.log" Dec 03 13:50:33 crc kubenswrapper[5052]: I1203 13:50:33.048265 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_1f59f640c8a0eb1a7b0f26c81382bbdde784d03eb439a940bb8da3931aj27fz_c121fea4-2ada-4c07-8cbb-06d0ab09b325/extract/0.log" Dec 03 13:50:33 crc kubenswrapper[5052]: I1203 13:50:33.188072 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_5064f9f8917b246f69f5d7fc025e7e6c34236c02bca31167615d38212frd48v_58e476bb-2b81-464d-8331-b31e848ed4b4/util/0.log" Dec 03 13:50:33 crc kubenswrapper[5052]: I1203 13:50:33.367366 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_5064f9f8917b246f69f5d7fc025e7e6c34236c02bca31167615d38212frd48v_58e476bb-2b81-464d-8331-b31e848ed4b4/util/0.log" Dec 03 13:50:33 crc kubenswrapper[5052]: I1203 13:50:33.374724 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_5064f9f8917b246f69f5d7fc025e7e6c34236c02bca31167615d38212frd48v_58e476bb-2b81-464d-8331-b31e848ed4b4/pull/0.log" Dec 03 13:50:33 crc kubenswrapper[5052]: I1203 13:50:33.426949 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_5064f9f8917b246f69f5d7fc025e7e6c34236c02bca31167615d38212frd48v_58e476bb-2b81-464d-8331-b31e848ed4b4/pull/0.log" Dec 03 13:50:33 crc kubenswrapper[5052]: I1203 13:50:33.603826 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_5064f9f8917b246f69f5d7fc025e7e6c34236c02bca31167615d38212frd48v_58e476bb-2b81-464d-8331-b31e848ed4b4/util/0.log" Dec 03 13:50:33 crc kubenswrapper[5052]: I1203 13:50:33.668088 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_5064f9f8917b246f69f5d7fc025e7e6c34236c02bca31167615d38212frd48v_58e476bb-2b81-464d-8331-b31e848ed4b4/extract/0.log" Dec 03 13:50:33 crc kubenswrapper[5052]: I1203 13:50:33.701763 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_5064f9f8917b246f69f5d7fc025e7e6c34236c02bca31167615d38212frd48v_58e476bb-2b81-464d-8331-b31e848ed4b4/pull/0.log" Dec 03 13:50:33 crc kubenswrapper[5052]: I1203 13:50:33.817628 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_6c372a8d094fad7255d3bbeabb4914bd2356af7b203a2d2176be1c9210msh2h_c2bb83c5-741f-4f8d-ac20-4bee739a6f43/util/0.log" Dec 03 13:50:34 crc kubenswrapper[5052]: I1203 13:50:34.018759 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_6c372a8d094fad7255d3bbeabb4914bd2356af7b203a2d2176be1c9210msh2h_c2bb83c5-741f-4f8d-ac20-4bee739a6f43/util/0.log" Dec 03 13:50:34 crc kubenswrapper[5052]: I1203 13:50:34.032127 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_6c372a8d094fad7255d3bbeabb4914bd2356af7b203a2d2176be1c9210msh2h_c2bb83c5-741f-4f8d-ac20-4bee739a6f43/pull/0.log" Dec 03 13:50:34 crc kubenswrapper[5052]: I1203 13:50:34.088213 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_6c372a8d094fad7255d3bbeabb4914bd2356af7b203a2d2176be1c9210msh2h_c2bb83c5-741f-4f8d-ac20-4bee739a6f43/pull/0.log" Dec 03 13:50:34 crc kubenswrapper[5052]: I1203 13:50:34.258294 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_6c372a8d094fad7255d3bbeabb4914bd2356af7b203a2d2176be1c9210msh2h_c2bb83c5-741f-4f8d-ac20-4bee739a6f43/util/0.log" Dec 03 13:50:34 crc kubenswrapper[5052]: I1203 13:50:34.267536 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_6c372a8d094fad7255d3bbeabb4914bd2356af7b203a2d2176be1c9210msh2h_c2bb83c5-741f-4f8d-ac20-4bee739a6f43/pull/0.log" Dec 03 13:50:34 crc kubenswrapper[5052]: I1203 13:50:34.398951 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_6c372a8d094fad7255d3bbeabb4914bd2356af7b203a2d2176be1c9210msh2h_c2bb83c5-741f-4f8d-ac20-4bee739a6f43/extract/0.log" Dec 03 13:50:34 crc kubenswrapper[5052]: I1203 13:50:34.517358 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_af69698b82fdf008f5ff9e195cf808a654240e16b13dcd924b74994f83dzz4m_aeaaf048-0037-4550-b1f5-54cfe153dcd2/util/0.log" Dec 03 13:50:34 crc kubenswrapper[5052]: I1203 13:50:34.633386 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_af69698b82fdf008f5ff9e195cf808a654240e16b13dcd924b74994f83dzz4m_aeaaf048-0037-4550-b1f5-54cfe153dcd2/util/0.log" Dec 03 13:50:34 crc kubenswrapper[5052]: I1203 13:50:34.644989 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_af69698b82fdf008f5ff9e195cf808a654240e16b13dcd924b74994f83dzz4m_aeaaf048-0037-4550-b1f5-54cfe153dcd2/pull/0.log" Dec 03 13:50:34 crc kubenswrapper[5052]: I1203 13:50:34.666907 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_af69698b82fdf008f5ff9e195cf808a654240e16b13dcd924b74994f83dzz4m_aeaaf048-0037-4550-b1f5-54cfe153dcd2/pull/0.log" Dec 03 13:50:34 crc kubenswrapper[5052]: I1203 13:50:34.986691 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_af69698b82fdf008f5ff9e195cf808a654240e16b13dcd924b74994f83dzz4m_aeaaf048-0037-4550-b1f5-54cfe153dcd2/pull/0.log" Dec 03 13:50:34 crc kubenswrapper[5052]: I1203 13:50:34.998190 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_af69698b82fdf008f5ff9e195cf808a654240e16b13dcd924b74994f83dzz4m_aeaaf048-0037-4550-b1f5-54cfe153dcd2/util/0.log" Dec 03 13:50:35 crc kubenswrapper[5052]: I1203 13:50:35.156489 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_af69698b82fdf008f5ff9e195cf808a654240e16b13dcd924b74994f83dzz4m_aeaaf048-0037-4550-b1f5-54cfe153dcd2/extract/0.log" Dec 03 13:50:35 crc kubenswrapper[5052]: I1203 13:50:35.212176 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_dnsmasq-dnsmasq-84b9f45d47-d8nh7_84b4fa9a-0498-4663-8ed2-2609ba3e2f88/init/0.log" Dec 03 13:50:35 crc kubenswrapper[5052]: I1203 13:50:35.291789 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_certified-operators-r2kbh_75e05b55-fc4c-4f5f-851a-b0ae877fa047/extract-utilities/0.log" Dec 03 13:50:35 crc kubenswrapper[5052]: I1203 13:50:35.625472 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_dnsmasq-dnsmasq-84b9f45d47-d8nh7_84b4fa9a-0498-4663-8ed2-2609ba3e2f88/dnsmasq-dns/0.log" Dec 03 13:50:35 crc kubenswrapper[5052]: I1203 13:50:35.671687 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openstack-kuttl-tests_dnsmasq-dnsmasq-84b9f45d47-d8nh7_84b4fa9a-0498-4663-8ed2-2609ba3e2f88/init/0.log" Dec 03 13:50:35 crc kubenswrapper[5052]: I1203 13:50:35.731121 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_certified-operators-r2kbh_75e05b55-fc4c-4f5f-851a-b0ae877fa047/extract-utilities/0.log" Dec 03 13:50:35 crc kubenswrapper[5052]: I1203 13:50:35.767008 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_certified-operators-r2kbh_75e05b55-fc4c-4f5f-851a-b0ae877fa047/extract-content/0.log" Dec 03 13:50:35 crc kubenswrapper[5052]: I1203 13:50:35.774881 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_certified-operators-r2kbh_75e05b55-fc4c-4f5f-851a-b0ae877fa047/extract-content/0.log" Dec 03 13:50:36 crc kubenswrapper[5052]: I1203 13:50:36.055799 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_certified-operators-r2kbh_75e05b55-fc4c-4f5f-851a-b0ae877fa047/extract-content/0.log" Dec 03 13:50:36 crc kubenswrapper[5052]: I1203 13:50:36.102623 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_certified-operators-r2kbh_75e05b55-fc4c-4f5f-851a-b0ae877fa047/extract-utilities/0.log" Dec 03 13:50:36 crc kubenswrapper[5052]: I1203 13:50:36.345942 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_community-operators-jxmjl_3e53b2f1-00e5-49d7-a6a6-5bcae4e9befb/extract-utilities/0.log" Dec 03 13:50:36 crc kubenswrapper[5052]: I1203 13:50:36.527775 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_community-operators-jxmjl_3e53b2f1-00e5-49d7-a6a6-5bcae4e9befb/extract-content/0.log" Dec 03 13:50:36 crc kubenswrapper[5052]: I1203 13:50:36.575182 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_community-operators-jxmjl_3e53b2f1-00e5-49d7-a6a6-5bcae4e9befb/extract-utilities/0.log" Dec 03 13:50:36 crc kubenswrapper[5052]: I1203 13:50:36.678891 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_community-operators-jxmjl_3e53b2f1-00e5-49d7-a6a6-5bcae4e9befb/extract-content/0.log" Dec 03 13:50:36 crc kubenswrapper[5052]: I1203 13:50:36.831931 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_community-operators-jxmjl_3e53b2f1-00e5-49d7-a6a6-5bcae4e9befb/extract-content/0.log" Dec 03 13:50:36 crc kubenswrapper[5052]: I1203 13:50:36.834790 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_community-operators-jxmjl_3e53b2f1-00e5-49d7-a6a6-5bcae4e9befb/extract-utilities/0.log" Dec 03 13:50:37 crc kubenswrapper[5052]: I1203 13:50:37.156634 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_marketplace-operator-79b997595-sbjmz_ce27ce03-4ec0-418f-95d8-c74102f26bbd/marketplace-operator/0.log" Dec 03 13:50:37 crc kubenswrapper[5052]: I1203 13:50:37.525719 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_redhat-marketplace-5ppmc_7074b488-71b1-4f8a-b798-b878a519cf7c/extract-utilities/0.log" Dec 03 13:50:37 crc kubenswrapper[5052]: I1203 13:50:37.829816 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_redhat-marketplace-5ppmc_7074b488-71b1-4f8a-b798-b878a519cf7c/extract-content/0.log" Dec 03 13:50:37 crc kubenswrapper[5052]: I1203 13:50:37.849844 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_redhat-marketplace-5ppmc_7074b488-71b1-4f8a-b798-b878a519cf7c/extract-utilities/0.log" Dec 03 13:50:37 crc kubenswrapper[5052]: I1203 13:50:37.947240 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_certified-operators-r2kbh_75e05b55-fc4c-4f5f-851a-b0ae877fa047/registry-server/0.log" Dec 03 13:50:37 crc kubenswrapper[5052]: I1203 13:50:37.951028 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_redhat-marketplace-5ppmc_7074b488-71b1-4f8a-b798-b878a519cf7c/extract-content/0.log" Dec 03 13:50:38 crc kubenswrapper[5052]: I1203 13:50:38.268388 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_redhat-marketplace-5ppmc_7074b488-71b1-4f8a-b798-b878a519cf7c/extract-utilities/0.log" Dec 03 13:50:38 crc kubenswrapper[5052]: I1203 13:50:38.298738 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_redhat-marketplace-5ppmc_7074b488-71b1-4f8a-b798-b878a519cf7c/extract-content/0.log" Dec 03 13:50:38 crc kubenswrapper[5052]: I1203 13:50:38.483832 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_redhat-operators-xtmc5_3045e915-e20b-4e46-a4bc-55783cb77448/extract-utilities/0.log" Dec 03 13:50:38 crc kubenswrapper[5052]: I1203 13:50:38.733497 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_redhat-operators-xtmc5_3045e915-e20b-4e46-a4bc-55783cb77448/extract-utilities/0.log" Dec 03 13:50:38 crc kubenswrapper[5052]: I1203 13:50:38.856380 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_redhat-operators-xtmc5_3045e915-e20b-4e46-a4bc-55783cb77448/extract-content/0.log" Dec 03 13:50:38 crc kubenswrapper[5052]: I1203 13:50:38.886925 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_redhat-operators-xtmc5_3045e915-e20b-4e46-a4bc-55783cb77448/extract-content/0.log" Dec 03 13:50:38 crc kubenswrapper[5052]: I1203 13:50:38.944137 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_redhat-marketplace-5ppmc_7074b488-71b1-4f8a-b798-b878a519cf7c/registry-server/0.log" Dec 03 13:50:39 crc kubenswrapper[5052]: I1203 13:50:39.024140 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_redhat-operators-xtmc5_3045e915-e20b-4e46-a4bc-55783cb77448/extract-utilities/0.log" Dec 03 13:50:39 crc kubenswrapper[5052]: I1203 13:50:39.144917 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_redhat-operators-xtmc5_3045e915-e20b-4e46-a4bc-55783cb77448/extract-content/0.log" Dec 03 13:50:39 crc kubenswrapper[5052]: I1203 13:50:39.152734 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_community-operators-jxmjl_3e53b2f1-00e5-49d7-a6a6-5bcae4e9befb/registry-server/0.log" Dec 03 13:50:40 crc kubenswrapper[5052]: I1203 13:50:40.379696 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_redhat-operators-xtmc5_3045e915-e20b-4e46-a4bc-55783cb77448/registry-server/0.log" Dec 03 13:50:50 crc kubenswrapper[5052]: I1203 13:50:50.142487 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_1f59f640c8a0eb1a7b0f26c81382bbdde784d03eb439a940bb8da3931aj27fz_c121fea4-2ada-4c07-8cbb-06d0ab09b325/util/0.log" Dec 03 13:50:50 crc kubenswrapper[5052]: I1203 13:50:50.325014 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_1f59f640c8a0eb1a7b0f26c81382bbdde784d03eb439a940bb8da3931aj27fz_c121fea4-2ada-4c07-8cbb-06d0ab09b325/pull/0.log" Dec 03 13:50:50 crc kubenswrapper[5052]: I1203 13:50:50.363801 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_1f59f640c8a0eb1a7b0f26c81382bbdde784d03eb439a940bb8da3931aj27fz_c121fea4-2ada-4c07-8cbb-06d0ab09b325/util/0.log" Dec 03 13:50:50 crc kubenswrapper[5052]: I1203 13:50:50.382550 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_1f59f640c8a0eb1a7b0f26c81382bbdde784d03eb439a940bb8da3931aj27fz_c121fea4-2ada-4c07-8cbb-06d0ab09b325/pull/0.log" Dec 03 13:50:50 crc kubenswrapper[5052]: I1203 13:50:50.563047 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_1f59f640c8a0eb1a7b0f26c81382bbdde784d03eb439a940bb8da3931aj27fz_c121fea4-2ada-4c07-8cbb-06d0ab09b325/util/0.log" Dec 03 13:50:50 crc kubenswrapper[5052]: I1203 13:50:50.586762 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_1f59f640c8a0eb1a7b0f26c81382bbdde784d03eb439a940bb8da3931aj27fz_c121fea4-2ada-4c07-8cbb-06d0ab09b325/extract/0.log" Dec 03 13:50:50 crc kubenswrapper[5052]: I1203 13:50:50.610305 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_1f59f640c8a0eb1a7b0f26c81382bbdde784d03eb439a940bb8da3931aj27fz_c121fea4-2ada-4c07-8cbb-06d0ab09b325/pull/0.log" Dec 03 13:50:50 crc kubenswrapper[5052]: I1203 13:50:50.816639 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_5064f9f8917b246f69f5d7fc025e7e6c34236c02bca31167615d38212frd48v_58e476bb-2b81-464d-8331-b31e848ed4b4/util/0.log" Dec 03 13:50:50 crc kubenswrapper[5052]: I1203 13:50:50.988837 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_5064f9f8917b246f69f5d7fc025e7e6c34236c02bca31167615d38212frd48v_58e476bb-2b81-464d-8331-b31e848ed4b4/util/0.log" Dec 03 13:50:50 crc kubenswrapper[5052]: I1203 13:50:50.994561 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_5064f9f8917b246f69f5d7fc025e7e6c34236c02bca31167615d38212frd48v_58e476bb-2b81-464d-8331-b31e848ed4b4/pull/0.log" Dec 03 13:50:51 crc kubenswrapper[5052]: I1203 13:50:51.017564 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_5064f9f8917b246f69f5d7fc025e7e6c34236c02bca31167615d38212frd48v_58e476bb-2b81-464d-8331-b31e848ed4b4/pull/0.log" Dec 03 13:50:51 crc kubenswrapper[5052]: I1203 13:50:51.270169 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_5064f9f8917b246f69f5d7fc025e7e6c34236c02bca31167615d38212frd48v_58e476bb-2b81-464d-8331-b31e848ed4b4/pull/0.log" Dec 03 13:50:51 crc kubenswrapper[5052]: I1203 13:50:51.296714 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_5064f9f8917b246f69f5d7fc025e7e6c34236c02bca31167615d38212frd48v_58e476bb-2b81-464d-8331-b31e848ed4b4/extract/0.log" Dec 03 13:50:51 crc kubenswrapper[5052]: I1203 13:50:51.351882 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_5064f9f8917b246f69f5d7fc025e7e6c34236c02bca31167615d38212frd48v_58e476bb-2b81-464d-8331-b31e848ed4b4/util/0.log" Dec 03 13:50:51 crc kubenswrapper[5052]: I1203 13:50:51.570803 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_6c372a8d094fad7255d3bbeabb4914bd2356af7b203a2d2176be1c9210msh2h_c2bb83c5-741f-4f8d-ac20-4bee739a6f43/util/0.log" Dec 03 13:50:51 crc kubenswrapper[5052]: I1203 13:50:51.798522 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-operators_obo-prometheus-operator-668cf9dfbb-mddsw_c0580db1-72b0-4b6b-9136-d45a283a060c/prometheus-operator/0.log" Dec 03 13:50:51 crc kubenswrapper[5052]: I1203 13:50:51.803726 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_6c372a8d094fad7255d3bbeabb4914bd2356af7b203a2d2176be1c9210msh2h_c2bb83c5-741f-4f8d-ac20-4bee739a6f43/pull/0.log" Dec 03 13:50:51 crc kubenswrapper[5052]: I1203 13:50:51.808910 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_6c372a8d094fad7255d3bbeabb4914bd2356af7b203a2d2176be1c9210msh2h_c2bb83c5-741f-4f8d-ac20-4bee739a6f43/util/0.log" Dec 03 13:50:51 crc kubenswrapper[5052]: I1203 13:50:51.889906 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_6c372a8d094fad7255d3bbeabb4914bd2356af7b203a2d2176be1c9210msh2h_c2bb83c5-741f-4f8d-ac20-4bee739a6f43/pull/0.log" Dec 03 13:50:52 crc kubenswrapper[5052]: I1203 13:50:52.039010 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_6c372a8d094fad7255d3bbeabb4914bd2356af7b203a2d2176be1c9210msh2h_c2bb83c5-741f-4f8d-ac20-4bee739a6f43/pull/0.log" Dec 03 13:50:52 crc kubenswrapper[5052]: I1203 13:50:52.039213 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_6c372a8d094fad7255d3bbeabb4914bd2356af7b203a2d2176be1c9210msh2h_c2bb83c5-741f-4f8d-ac20-4bee739a6f43/extract/0.log" Dec 03 13:50:52 crc kubenswrapper[5052]: I1203 13:50:52.044905 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-operators_obo-prometheus-operator-admission-webhook-7cd69dc689-zh2bj_6a2264eb-3e7a-4987-be04-9131bb690ebf/prometheus-operator-admission-webhook/0.log" Dec 03 13:50:52 crc kubenswrapper[5052]: I1203 13:50:52.100062 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_6c372a8d094fad7255d3bbeabb4914bd2356af7b203a2d2176be1c9210msh2h_c2bb83c5-741f-4f8d-ac20-4bee739a6f43/util/0.log" Dec 03 13:50:52 crc kubenswrapper[5052]: I1203 13:50:52.101636 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-operators_obo-prometheus-operator-admission-webhook-7cd69dc689-cwkf4_ff11d0bc-dedc-433d-9e0e-a3fd2232919f/prometheus-operator-admission-webhook/0.log" Dec 03 13:50:52 crc kubenswrapper[5052]: I1203 13:50:52.260174 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-operators_observability-operator-d8bb48f5d-swndl_0e9b7522-0b02-484a-b9e6-545e8dabcbc7/operator/0.log" Dec 03 13:50:52 crc kubenswrapper[5052]: I1203 13:50:52.293637 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_af69698b82fdf008f5ff9e195cf808a654240e16b13dcd924b74994f83dzz4m_aeaaf048-0037-4550-b1f5-54cfe153dcd2/util/0.log" Dec 03 13:50:52 crc kubenswrapper[5052]: I1203 13:50:52.352481 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-operators_perses-operator-5446b9c989-gqddj_8b649975-2e0f-4406-9494-d0e0a21fc168/perses-operator/0.log" Dec 03 13:50:52 crc kubenswrapper[5052]: I1203 13:50:52.513196 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_af69698b82fdf008f5ff9e195cf808a654240e16b13dcd924b74994f83dzz4m_aeaaf048-0037-4550-b1f5-54cfe153dcd2/pull/0.log" Dec 03 13:50:52 crc kubenswrapper[5052]: I1203 13:50:52.555383 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_af69698b82fdf008f5ff9e195cf808a654240e16b13dcd924b74994f83dzz4m_aeaaf048-0037-4550-b1f5-54cfe153dcd2/util/0.log" Dec 03 13:50:52 crc kubenswrapper[5052]: I1203 13:50:52.556001 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_af69698b82fdf008f5ff9e195cf808a654240e16b13dcd924b74994f83dzz4m_aeaaf048-0037-4550-b1f5-54cfe153dcd2/pull/0.log" Dec 03 13:50:52 crc kubenswrapper[5052]: I1203 13:50:52.808355 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_af69698b82fdf008f5ff9e195cf808a654240e16b13dcd924b74994f83dzz4m_aeaaf048-0037-4550-b1f5-54cfe153dcd2/util/0.log" Dec 03 13:50:52 crc kubenswrapper[5052]: I1203 13:50:52.819008 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_af69698b82fdf008f5ff9e195cf808a654240e16b13dcd924b74994f83dzz4m_aeaaf048-0037-4550-b1f5-54cfe153dcd2/extract/0.log" Dec 03 13:50:52 crc kubenswrapper[5052]: I1203 13:50:52.828090 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_af69698b82fdf008f5ff9e195cf808a654240e16b13dcd924b74994f83dzz4m_aeaaf048-0037-4550-b1f5-54cfe153dcd2/pull/0.log" Dec 03 13:50:53 crc kubenswrapper[5052]: I1203 13:50:53.007430 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_certified-operators-r2kbh_75e05b55-fc4c-4f5f-851a-b0ae877fa047/extract-utilities/0.log" Dec 03 13:50:53 crc kubenswrapper[5052]: I1203 13:50:53.251694 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_certified-operators-r2kbh_75e05b55-fc4c-4f5f-851a-b0ae877fa047/extract-utilities/0.log" Dec 03 13:50:53 crc kubenswrapper[5052]: I1203 13:50:53.283795 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_certified-operators-r2kbh_75e05b55-fc4c-4f5f-851a-b0ae877fa047/extract-content/0.log" Dec 03 13:50:53 crc kubenswrapper[5052]: I1203 13:50:53.369276 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_certified-operators-r2kbh_75e05b55-fc4c-4f5f-851a-b0ae877fa047/extract-content/0.log" Dec 03 13:50:53 crc kubenswrapper[5052]: I1203 13:50:53.523916 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_certified-operators-r2kbh_75e05b55-fc4c-4f5f-851a-b0ae877fa047/extract-utilities/0.log" Dec 03 13:50:53 crc kubenswrapper[5052]: I1203 13:50:53.564022 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_certified-operators-r2kbh_75e05b55-fc4c-4f5f-851a-b0ae877fa047/extract-content/0.log" Dec 03 13:50:53 crc kubenswrapper[5052]: I1203 13:50:53.841912 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_community-operators-jxmjl_3e53b2f1-00e5-49d7-a6a6-5bcae4e9befb/extract-utilities/0.log" Dec 03 13:50:54 crc kubenswrapper[5052]: I1203 13:50:54.168623 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_community-operators-jxmjl_3e53b2f1-00e5-49d7-a6a6-5bcae4e9befb/extract-content/0.log" Dec 03 13:50:54 crc kubenswrapper[5052]: I1203 13:50:54.300182 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_community-operators-jxmjl_3e53b2f1-00e5-49d7-a6a6-5bcae4e9befb/extract-utilities/0.log" Dec 03 13:50:54 crc kubenswrapper[5052]: I1203 13:50:54.359598 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_community-operators-jxmjl_3e53b2f1-00e5-49d7-a6a6-5bcae4e9befb/extract-content/0.log" Dec 03 13:50:54 crc kubenswrapper[5052]: I1203 13:50:54.504556 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_community-operators-jxmjl_3e53b2f1-00e5-49d7-a6a6-5bcae4e9befb/extract-content/0.log" Dec 03 13:50:54 crc kubenswrapper[5052]: I1203 13:50:54.511218 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_community-operators-jxmjl_3e53b2f1-00e5-49d7-a6a6-5bcae4e9befb/extract-utilities/0.log" Dec 03 13:50:54 crc kubenswrapper[5052]: I1203 13:50:54.682036 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_marketplace-operator-79b997595-sbjmz_ce27ce03-4ec0-418f-95d8-c74102f26bbd/marketplace-operator/0.log" Dec 03 13:50:54 crc kubenswrapper[5052]: I1203 13:50:54.956910 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_redhat-marketplace-5ppmc_7074b488-71b1-4f8a-b798-b878a519cf7c/extract-utilities/0.log" Dec 03 13:50:55 crc kubenswrapper[5052]: I1203 13:50:55.134715 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_certified-operators-r2kbh_75e05b55-fc4c-4f5f-851a-b0ae877fa047/registry-server/0.log" Dec 03 13:50:55 crc kubenswrapper[5052]: I1203 13:50:55.256668 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_redhat-marketplace-5ppmc_7074b488-71b1-4f8a-b798-b878a519cf7c/extract-utilities/0.log" Dec 03 13:50:55 crc kubenswrapper[5052]: I1203 13:50:55.278262 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_redhat-marketplace-5ppmc_7074b488-71b1-4f8a-b798-b878a519cf7c/extract-content/0.log" Dec 03 13:50:55 crc kubenswrapper[5052]: I1203 13:50:55.278400 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_redhat-marketplace-5ppmc_7074b488-71b1-4f8a-b798-b878a519cf7c/extract-content/0.log" Dec 03 13:50:55 crc kubenswrapper[5052]: I1203 13:50:55.548986 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_redhat-marketplace-5ppmc_7074b488-71b1-4f8a-b798-b878a519cf7c/extract-content/0.log" Dec 03 13:50:55 crc kubenswrapper[5052]: I1203 13:50:55.599567 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_redhat-marketplace-5ppmc_7074b488-71b1-4f8a-b798-b878a519cf7c/extract-utilities/0.log" Dec 03 13:50:55 crc kubenswrapper[5052]: I1203 13:50:55.772374 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_redhat-operators-xtmc5_3045e915-e20b-4e46-a4bc-55783cb77448/extract-utilities/0.log" Dec 03 13:50:56 crc kubenswrapper[5052]: I1203 13:50:56.020461 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_redhat-operators-xtmc5_3045e915-e20b-4e46-a4bc-55783cb77448/extract-utilities/0.log" Dec 03 13:50:56 crc kubenswrapper[5052]: I1203 13:50:56.020890 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_redhat-marketplace-5ppmc_7074b488-71b1-4f8a-b798-b878a519cf7c/registry-server/0.log" Dec 03 13:50:56 crc kubenswrapper[5052]: I1203 13:50:56.025113 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_redhat-operators-xtmc5_3045e915-e20b-4e46-a4bc-55783cb77448/extract-content/0.log" Dec 03 13:50:56 crc kubenswrapper[5052]: I1203 13:50:56.091709 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_redhat-operators-xtmc5_3045e915-e20b-4e46-a4bc-55783cb77448/extract-content/0.log" Dec 03 13:50:56 crc kubenswrapper[5052]: I1203 13:50:56.298475 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_redhat-operators-xtmc5_3045e915-e20b-4e46-a4bc-55783cb77448/extract-utilities/0.log" Dec 03 13:50:56 crc kubenswrapper[5052]: I1203 13:50:56.306216 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_redhat-operators-xtmc5_3045e915-e20b-4e46-a4bc-55783cb77448/extract-content/0.log" Dec 03 13:50:56 crc kubenswrapper[5052]: I1203 13:50:56.598144 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_community-operators-jxmjl_3e53b2f1-00e5-49d7-a6a6-5bcae4e9befb/registry-server/0.log" Dec 03 13:50:57 crc kubenswrapper[5052]: I1203 13:50:57.712678 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_redhat-operators-xtmc5_3045e915-e20b-4e46-a4bc-55783cb77448/registry-server/0.log" Dec 03 13:51:09 crc kubenswrapper[5052]: I1203 13:51:09.471217 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-operators_obo-prometheus-operator-668cf9dfbb-mddsw_c0580db1-72b0-4b6b-9136-d45a283a060c/prometheus-operator/0.log" Dec 03 13:51:09 crc kubenswrapper[5052]: I1203 13:51:09.677131 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-operators_obo-prometheus-operator-admission-webhook-7cd69dc689-cwkf4_ff11d0bc-dedc-433d-9e0e-a3fd2232919f/prometheus-operator-admission-webhook/0.log" Dec 03 13:51:09 crc kubenswrapper[5052]: I1203 13:51:09.732066 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-operators_obo-prometheus-operator-admission-webhook-7cd69dc689-zh2bj_6a2264eb-3e7a-4987-be04-9131bb690ebf/prometheus-operator-admission-webhook/0.log" Dec 03 13:51:09 crc kubenswrapper[5052]: I1203 13:51:09.883865 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-operators_observability-operator-d8bb48f5d-swndl_0e9b7522-0b02-484a-b9e6-545e8dabcbc7/operator/0.log" Dec 03 13:51:09 crc kubenswrapper[5052]: I1203 13:51:09.959066 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-operators_perses-operator-5446b9c989-gqddj_8b649975-2e0f-4406-9494-d0e0a21fc168/perses-operator/0.log" Dec 03 13:51:59 crc kubenswrapper[5052]: I1203 13:51:59.379658 5052 patch_prober.go:28] interesting pod/machine-config-daemon-74bj6 container/machine-config-daemon namespace/openshift-machine-config-operator: Liveness probe status=failure output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" start-of-body= Dec 03 13:51:59 crc kubenswrapper[5052]: I1203 13:51:59.380207 5052 prober.go:107] "Probe failed" probeType="Liveness" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" containerName="machine-config-daemon" probeResult="failure" output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" Dec 03 13:52:15 crc kubenswrapper[5052]: I1203 13:52:15.156153 5052 generic.go:334] "Generic (PLEG): container finished" podID="f3203993-dab9-40db-9d30-8d8058573a90" containerID="3a45fdbf0aa9269d811a6da6d0e21822286efa4edc16fbc248e622729c89d274" exitCode=0 Dec 03 13:52:15 crc kubenswrapper[5052]: I1203 13:52:15.156250 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-58j6j/must-gather-xj487" event={"ID":"f3203993-dab9-40db-9d30-8d8058573a90","Type":"ContainerDied","Data":"3a45fdbf0aa9269d811a6da6d0e21822286efa4edc16fbc248e622729c89d274"} Dec 03 13:52:15 crc kubenswrapper[5052]: I1203 13:52:15.157301 5052 scope.go:117] "RemoveContainer" containerID="3a45fdbf0aa9269d811a6da6d0e21822286efa4edc16fbc248e622729c89d274" Dec 03 13:52:15 crc kubenswrapper[5052]: I1203 13:52:15.777068 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-58j6j_must-gather-xj487_f3203993-dab9-40db-9d30-8d8058573a90/gather/0.log" Dec 03 13:52:23 crc kubenswrapper[5052]: I1203 13:52:23.565936 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openshift-must-gather-58j6j/must-gather-xj487"] Dec 03 13:52:23 crc kubenswrapper[5052]: I1203 13:52:23.566798 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openshift-must-gather-58j6j/must-gather-xj487" podUID="f3203993-dab9-40db-9d30-8d8058573a90" containerName="copy" containerID="cri-o://99f6b08b0dfd77b55d7c46ab10e4cf4ded65ff3ba4e32445bac15d2c975c36c8" gracePeriod=2 Dec 03 13:52:23 crc kubenswrapper[5052]: I1203 13:52:23.577449 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openshift-must-gather-58j6j/must-gather-xj487"] Dec 03 13:52:23 crc kubenswrapper[5052]: I1203 13:52:23.935689 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-58j6j_must-gather-xj487_f3203993-dab9-40db-9d30-8d8058573a90/copy/0.log" Dec 03 13:52:23 crc kubenswrapper[5052]: I1203 13:52:23.936873 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-58j6j/must-gather-xj487" Dec 03 13:52:24 crc kubenswrapper[5052]: I1203 13:52:24.084880 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/f3203993-dab9-40db-9d30-8d8058573a90-must-gather-output\") pod \"f3203993-dab9-40db-9d30-8d8058573a90\" (UID: \"f3203993-dab9-40db-9d30-8d8058573a90\") " Dec 03 13:52:24 crc kubenswrapper[5052]: I1203 13:52:24.084991 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ltb2k\" (UniqueName: \"kubernetes.io/projected/f3203993-dab9-40db-9d30-8d8058573a90-kube-api-access-ltb2k\") pod \"f3203993-dab9-40db-9d30-8d8058573a90\" (UID: \"f3203993-dab9-40db-9d30-8d8058573a90\") " Dec 03 13:52:24 crc kubenswrapper[5052]: I1203 13:52:24.106125 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f3203993-dab9-40db-9d30-8d8058573a90-kube-api-access-ltb2k" (OuterVolumeSpecName: "kube-api-access-ltb2k") pod "f3203993-dab9-40db-9d30-8d8058573a90" (UID: "f3203993-dab9-40db-9d30-8d8058573a90"). InnerVolumeSpecName "kube-api-access-ltb2k". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:52:24 crc kubenswrapper[5052]: I1203 13:52:24.187498 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-ltb2k\" (UniqueName: \"kubernetes.io/projected/f3203993-dab9-40db-9d30-8d8058573a90-kube-api-access-ltb2k\") on node \"crc\" DevicePath \"\"" Dec 03 13:52:24 crc kubenswrapper[5052]: I1203 13:52:24.204826 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f3203993-dab9-40db-9d30-8d8058573a90-must-gather-output" (OuterVolumeSpecName: "must-gather-output") pod "f3203993-dab9-40db-9d30-8d8058573a90" (UID: "f3203993-dab9-40db-9d30-8d8058573a90"). InnerVolumeSpecName "must-gather-output". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 13:52:24 crc kubenswrapper[5052]: I1203 13:52:24.222921 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-58j6j_must-gather-xj487_f3203993-dab9-40db-9d30-8d8058573a90/copy/0.log" Dec 03 13:52:24 crc kubenswrapper[5052]: I1203 13:52:24.223317 5052 generic.go:334] "Generic (PLEG): container finished" podID="f3203993-dab9-40db-9d30-8d8058573a90" containerID="99f6b08b0dfd77b55d7c46ab10e4cf4ded65ff3ba4e32445bac15d2c975c36c8" exitCode=143 Dec 03 13:52:24 crc kubenswrapper[5052]: I1203 13:52:24.223367 5052 scope.go:117] "RemoveContainer" containerID="99f6b08b0dfd77b55d7c46ab10e4cf4ded65ff3ba4e32445bac15d2c975c36c8" Dec 03 13:52:24 crc kubenswrapper[5052]: I1203 13:52:24.223530 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-58j6j/must-gather-xj487" Dec 03 13:52:24 crc kubenswrapper[5052]: I1203 13:52:24.243815 5052 scope.go:117] "RemoveContainer" containerID="3a45fdbf0aa9269d811a6da6d0e21822286efa4edc16fbc248e622729c89d274" Dec 03 13:52:24 crc kubenswrapper[5052]: I1203 13:52:24.291276 5052 reconciler_common.go:293] "Volume detached for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/f3203993-dab9-40db-9d30-8d8058573a90-must-gather-output\") on node \"crc\" DevicePath \"\"" Dec 03 13:52:24 crc kubenswrapper[5052]: I1203 13:52:24.307674 5052 scope.go:117] "RemoveContainer" containerID="99f6b08b0dfd77b55d7c46ab10e4cf4ded65ff3ba4e32445bac15d2c975c36c8" Dec 03 13:52:24 crc kubenswrapper[5052]: E1203 13:52:24.308142 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"99f6b08b0dfd77b55d7c46ab10e4cf4ded65ff3ba4e32445bac15d2c975c36c8\": container with ID starting with 99f6b08b0dfd77b55d7c46ab10e4cf4ded65ff3ba4e32445bac15d2c975c36c8 not found: ID does not exist" containerID="99f6b08b0dfd77b55d7c46ab10e4cf4ded65ff3ba4e32445bac15d2c975c36c8" Dec 03 13:52:24 crc kubenswrapper[5052]: I1203 13:52:24.308184 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"99f6b08b0dfd77b55d7c46ab10e4cf4ded65ff3ba4e32445bac15d2c975c36c8"} err="failed to get container status \"99f6b08b0dfd77b55d7c46ab10e4cf4ded65ff3ba4e32445bac15d2c975c36c8\": rpc error: code = NotFound desc = could not find container \"99f6b08b0dfd77b55d7c46ab10e4cf4ded65ff3ba4e32445bac15d2c975c36c8\": container with ID starting with 99f6b08b0dfd77b55d7c46ab10e4cf4ded65ff3ba4e32445bac15d2c975c36c8 not found: ID does not exist" Dec 03 13:52:24 crc kubenswrapper[5052]: I1203 13:52:24.308210 5052 scope.go:117] "RemoveContainer" containerID="3a45fdbf0aa9269d811a6da6d0e21822286efa4edc16fbc248e622729c89d274" Dec 03 13:52:24 crc kubenswrapper[5052]: E1203 13:52:24.309153 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3a45fdbf0aa9269d811a6da6d0e21822286efa4edc16fbc248e622729c89d274\": container with ID starting with 3a45fdbf0aa9269d811a6da6d0e21822286efa4edc16fbc248e622729c89d274 not found: ID does not exist" containerID="3a45fdbf0aa9269d811a6da6d0e21822286efa4edc16fbc248e622729c89d274" Dec 03 13:52:24 crc kubenswrapper[5052]: I1203 13:52:24.309182 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3a45fdbf0aa9269d811a6da6d0e21822286efa4edc16fbc248e622729c89d274"} err="failed to get container status \"3a45fdbf0aa9269d811a6da6d0e21822286efa4edc16fbc248e622729c89d274\": rpc error: code = NotFound desc = could not find container \"3a45fdbf0aa9269d811a6da6d0e21822286efa4edc16fbc248e622729c89d274\": container with ID starting with 3a45fdbf0aa9269d811a6da6d0e21822286efa4edc16fbc248e622729c89d274 not found: ID does not exist" Dec 03 13:52:25 crc kubenswrapper[5052]: I1203 13:52:25.681605 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f3203993-dab9-40db-9d30-8d8058573a90" path="/var/lib/kubelet/pods/f3203993-dab9-40db-9d30-8d8058573a90/volumes" Dec 03 13:52:29 crc kubenswrapper[5052]: I1203 13:52:29.378981 5052 patch_prober.go:28] interesting pod/machine-config-daemon-74bj6 container/machine-config-daemon namespace/openshift-machine-config-operator: Liveness probe status=failure output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" start-of-body= Dec 03 13:52:29 crc kubenswrapper[5052]: I1203 13:52:29.379646 5052 prober.go:107] "Probe failed" probeType="Liveness" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" containerName="machine-config-daemon" probeResult="failure" output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" Dec 03 13:52:30 crc kubenswrapper[5052]: I1203 13:52:30.263268 5052 generic.go:334] "Generic (PLEG): container finished" podID="186b2ab5-1c1f-4a2c-90bf-c5dc971d4677" containerID="5bf5e212c38591b042130bd0a9422fc8bb9656717c05a83220f5ae0ca67264af" exitCode=0 Dec 03 13:52:30 crc kubenswrapper[5052]: I1203 13:52:30.263375 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-vj2x8/must-gather-s9pjl" event={"ID":"186b2ab5-1c1f-4a2c-90bf-c5dc971d4677","Type":"ContainerDied","Data":"5bf5e212c38591b042130bd0a9422fc8bb9656717c05a83220f5ae0ca67264af"} Dec 03 13:52:30 crc kubenswrapper[5052]: I1203 13:52:30.264136 5052 scope.go:117] "RemoveContainer" containerID="5bf5e212c38591b042130bd0a9422fc8bb9656717c05a83220f5ae0ca67264af" Dec 03 13:52:30 crc kubenswrapper[5052]: I1203 13:52:30.299664 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-vj2x8_must-gather-s9pjl_186b2ab5-1c1f-4a2c-90bf-c5dc971d4677/gather/0.log" Dec 03 13:52:41 crc kubenswrapper[5052]: I1203 13:52:41.308552 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openshift-must-gather-vj2x8/must-gather-s9pjl"] Dec 03 13:52:41 crc kubenswrapper[5052]: I1203 13:52:41.309189 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openshift-must-gather-vj2x8/must-gather-s9pjl" podUID="186b2ab5-1c1f-4a2c-90bf-c5dc971d4677" containerName="copy" containerID="cri-o://38689613e122abf04f2f1334aa689722b063681e07893bf9d12ea7fce4c29e94" gracePeriod=2 Dec 03 13:52:41 crc kubenswrapper[5052]: I1203 13:52:41.314783 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openshift-must-gather-vj2x8/must-gather-s9pjl"] Dec 03 13:52:41 crc kubenswrapper[5052]: I1203 13:52:41.672773 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-vj2x8_must-gather-s9pjl_186b2ab5-1c1f-4a2c-90bf-c5dc971d4677/copy/0.log" Dec 03 13:52:41 crc kubenswrapper[5052]: I1203 13:52:41.673831 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-vj2x8/must-gather-s9pjl" Dec 03 13:52:41 crc kubenswrapper[5052]: I1203 13:52:41.862769 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/186b2ab5-1c1f-4a2c-90bf-c5dc971d4677-must-gather-output\") pod \"186b2ab5-1c1f-4a2c-90bf-c5dc971d4677\" (UID: \"186b2ab5-1c1f-4a2c-90bf-c5dc971d4677\") " Dec 03 13:52:41 crc kubenswrapper[5052]: I1203 13:52:41.862896 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8fp4q\" (UniqueName: \"kubernetes.io/projected/186b2ab5-1c1f-4a2c-90bf-c5dc971d4677-kube-api-access-8fp4q\") pod \"186b2ab5-1c1f-4a2c-90bf-c5dc971d4677\" (UID: \"186b2ab5-1c1f-4a2c-90bf-c5dc971d4677\") " Dec 03 13:52:41 crc kubenswrapper[5052]: I1203 13:52:41.869333 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/186b2ab5-1c1f-4a2c-90bf-c5dc971d4677-kube-api-access-8fp4q" (OuterVolumeSpecName: "kube-api-access-8fp4q") pod "186b2ab5-1c1f-4a2c-90bf-c5dc971d4677" (UID: "186b2ab5-1c1f-4a2c-90bf-c5dc971d4677"). InnerVolumeSpecName "kube-api-access-8fp4q". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:52:41 crc kubenswrapper[5052]: I1203 13:52:41.964928 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-8fp4q\" (UniqueName: \"kubernetes.io/projected/186b2ab5-1c1f-4a2c-90bf-c5dc971d4677-kube-api-access-8fp4q\") on node \"crc\" DevicePath \"\"" Dec 03 13:52:41 crc kubenswrapper[5052]: I1203 13:52:41.971495 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/186b2ab5-1c1f-4a2c-90bf-c5dc971d4677-must-gather-output" (OuterVolumeSpecName: "must-gather-output") pod "186b2ab5-1c1f-4a2c-90bf-c5dc971d4677" (UID: "186b2ab5-1c1f-4a2c-90bf-c5dc971d4677"). InnerVolumeSpecName "must-gather-output". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 13:52:42 crc kubenswrapper[5052]: I1203 13:52:42.066378 5052 reconciler_common.go:293] "Volume detached for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/186b2ab5-1c1f-4a2c-90bf-c5dc971d4677-must-gather-output\") on node \"crc\" DevicePath \"\"" Dec 03 13:52:42 crc kubenswrapper[5052]: I1203 13:52:42.340984 5052 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-vj2x8_must-gather-s9pjl_186b2ab5-1c1f-4a2c-90bf-c5dc971d4677/copy/0.log" Dec 03 13:52:42 crc kubenswrapper[5052]: I1203 13:52:42.341459 5052 generic.go:334] "Generic (PLEG): container finished" podID="186b2ab5-1c1f-4a2c-90bf-c5dc971d4677" containerID="38689613e122abf04f2f1334aa689722b063681e07893bf9d12ea7fce4c29e94" exitCode=143 Dec 03 13:52:42 crc kubenswrapper[5052]: I1203 13:52:42.341524 5052 scope.go:117] "RemoveContainer" containerID="38689613e122abf04f2f1334aa689722b063681e07893bf9d12ea7fce4c29e94" Dec 03 13:52:42 crc kubenswrapper[5052]: I1203 13:52:42.341543 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-vj2x8/must-gather-s9pjl" Dec 03 13:52:42 crc kubenswrapper[5052]: I1203 13:52:42.361575 5052 scope.go:117] "RemoveContainer" containerID="5bf5e212c38591b042130bd0a9422fc8bb9656717c05a83220f5ae0ca67264af" Dec 03 13:52:42 crc kubenswrapper[5052]: I1203 13:52:42.423971 5052 scope.go:117] "RemoveContainer" containerID="38689613e122abf04f2f1334aa689722b063681e07893bf9d12ea7fce4c29e94" Dec 03 13:52:42 crc kubenswrapper[5052]: E1203 13:52:42.424835 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"38689613e122abf04f2f1334aa689722b063681e07893bf9d12ea7fce4c29e94\": container with ID starting with 38689613e122abf04f2f1334aa689722b063681e07893bf9d12ea7fce4c29e94 not found: ID does not exist" containerID="38689613e122abf04f2f1334aa689722b063681e07893bf9d12ea7fce4c29e94" Dec 03 13:52:42 crc kubenswrapper[5052]: I1203 13:52:42.424997 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"38689613e122abf04f2f1334aa689722b063681e07893bf9d12ea7fce4c29e94"} err="failed to get container status \"38689613e122abf04f2f1334aa689722b063681e07893bf9d12ea7fce4c29e94\": rpc error: code = NotFound desc = could not find container \"38689613e122abf04f2f1334aa689722b063681e07893bf9d12ea7fce4c29e94\": container with ID starting with 38689613e122abf04f2f1334aa689722b063681e07893bf9d12ea7fce4c29e94 not found: ID does not exist" Dec 03 13:52:42 crc kubenswrapper[5052]: I1203 13:52:42.425127 5052 scope.go:117] "RemoveContainer" containerID="5bf5e212c38591b042130bd0a9422fc8bb9656717c05a83220f5ae0ca67264af" Dec 03 13:52:42 crc kubenswrapper[5052]: E1203 13:52:42.426448 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5bf5e212c38591b042130bd0a9422fc8bb9656717c05a83220f5ae0ca67264af\": container with ID starting with 5bf5e212c38591b042130bd0a9422fc8bb9656717c05a83220f5ae0ca67264af not found: ID does not exist" containerID="5bf5e212c38591b042130bd0a9422fc8bb9656717c05a83220f5ae0ca67264af" Dec 03 13:52:42 crc kubenswrapper[5052]: I1203 13:52:42.426552 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5bf5e212c38591b042130bd0a9422fc8bb9656717c05a83220f5ae0ca67264af"} err="failed to get container status \"5bf5e212c38591b042130bd0a9422fc8bb9656717c05a83220f5ae0ca67264af\": rpc error: code = NotFound desc = could not find container \"5bf5e212c38591b042130bd0a9422fc8bb9656717c05a83220f5ae0ca67264af\": container with ID starting with 5bf5e212c38591b042130bd0a9422fc8bb9656717c05a83220f5ae0ca67264af not found: ID does not exist" Dec 03 13:52:43 crc kubenswrapper[5052]: I1203 13:52:43.682356 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="186b2ab5-1c1f-4a2c-90bf-c5dc971d4677" path="/var/lib/kubelet/pods/186b2ab5-1c1f-4a2c-90bf-c5dc971d4677/volumes" Dec 03 13:52:51 crc kubenswrapper[5052]: I1203 13:52:51.341715 5052 scope.go:117] "RemoveContainer" containerID="06eb853e582954e757cdbf2f774b8864440ed80f9893b4fdb23abe72ed90929f" Dec 03 13:52:59 crc kubenswrapper[5052]: I1203 13:52:59.379580 5052 patch_prober.go:28] interesting pod/machine-config-daemon-74bj6 container/machine-config-daemon namespace/openshift-machine-config-operator: Liveness probe status=failure output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" start-of-body= Dec 03 13:52:59 crc kubenswrapper[5052]: I1203 13:52:59.380257 5052 prober.go:107] "Probe failed" probeType="Liveness" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" containerName="machine-config-daemon" probeResult="failure" output="Get \"http://127.0.0.1:8798/health\": dial tcp 127.0.0.1:8798: connect: connection refused" Dec 03 13:52:59 crc kubenswrapper[5052]: I1203 13:52:59.380311 5052 kubelet.go:2542] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" Dec 03 13:52:59 crc kubenswrapper[5052]: I1203 13:52:59.381054 5052 kuberuntime_manager.go:1027] "Message for Container of pod" containerName="machine-config-daemon" containerStatusID={"Type":"cri-o","ID":"a4c6deffa55d4faa49789e30d4c8b2762d7fc719390c909181ec730f718e0556"} pod="openshift-machine-config-operator/machine-config-daemon-74bj6" containerMessage="Container machine-config-daemon failed liveness probe, will be restarted" Dec 03 13:52:59 crc kubenswrapper[5052]: I1203 13:52:59.381127 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" containerName="machine-config-daemon" containerID="cri-o://a4c6deffa55d4faa49789e30d4c8b2762d7fc719390c909181ec730f718e0556" gracePeriod=600 Dec 03 13:53:00 crc kubenswrapper[5052]: E1203 13:53:00.001433 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 13:53:00 crc kubenswrapper[5052]: I1203 13:53:00.477266 5052 generic.go:334] "Generic (PLEG): container finished" podID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" containerID="a4c6deffa55d4faa49789e30d4c8b2762d7fc719390c909181ec730f718e0556" exitCode=0 Dec 03 13:53:00 crc kubenswrapper[5052]: I1203 13:53:00.477302 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" event={"ID":"10c237bc-72e7-4da3-9f2c-b1688395bf6e","Type":"ContainerDied","Data":"a4c6deffa55d4faa49789e30d4c8b2762d7fc719390c909181ec730f718e0556"} Dec 03 13:53:00 crc kubenswrapper[5052]: I1203 13:53:00.477333 5052 scope.go:117] "RemoveContainer" containerID="af948543b565b59ee5a764bc4f80d394be230d8c49adc6125d06e305c9dffa71" Dec 03 13:53:00 crc kubenswrapper[5052]: I1203 13:53:00.477742 5052 scope.go:117] "RemoveContainer" containerID="a4c6deffa55d4faa49789e30d4c8b2762d7fc719390c909181ec730f718e0556" Dec 03 13:53:00 crc kubenswrapper[5052]: E1203 13:53:00.477958 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 13:53:11 crc kubenswrapper[5052]: I1203 13:53:11.673237 5052 scope.go:117] "RemoveContainer" containerID="a4c6deffa55d4faa49789e30d4c8b2762d7fc719390c909181ec730f718e0556" Dec 03 13:53:11 crc kubenswrapper[5052]: E1203 13:53:11.674125 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 13:53:23 crc kubenswrapper[5052]: I1203 13:53:23.673922 5052 scope.go:117] "RemoveContainer" containerID="a4c6deffa55d4faa49789e30d4c8b2762d7fc719390c909181ec730f718e0556" Dec 03 13:53:23 crc kubenswrapper[5052]: E1203 13:53:23.675022 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 13:53:34 crc kubenswrapper[5052]: I1203 13:53:34.672962 5052 scope.go:117] "RemoveContainer" containerID="a4c6deffa55d4faa49789e30d4c8b2762d7fc719390c909181ec730f718e0556" Dec 03 13:53:34 crc kubenswrapper[5052]: E1203 13:53:34.673831 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 13:53:49 crc kubenswrapper[5052]: I1203 13:53:49.677784 5052 scope.go:117] "RemoveContainer" containerID="a4c6deffa55d4faa49789e30d4c8b2762d7fc719390c909181ec730f718e0556" Dec 03 13:53:49 crc kubenswrapper[5052]: E1203 13:53:49.678780 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 13:54:00 crc kubenswrapper[5052]: I1203 13:54:00.672984 5052 scope.go:117] "RemoveContainer" containerID="a4c6deffa55d4faa49789e30d4c8b2762d7fc719390c909181ec730f718e0556" Dec 03 13:54:00 crc kubenswrapper[5052]: E1203 13:54:00.675054 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 13:54:11 crc kubenswrapper[5052]: I1203 13:54:11.673247 5052 scope.go:117] "RemoveContainer" containerID="a4c6deffa55d4faa49789e30d4c8b2762d7fc719390c909181ec730f718e0556" Dec 03 13:54:11 crc kubenswrapper[5052]: E1203 13:54:11.674149 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 13:54:22 crc kubenswrapper[5052]: I1203 13:54:22.673210 5052 scope.go:117] "RemoveContainer" containerID="a4c6deffa55d4faa49789e30d4c8b2762d7fc719390c909181ec730f718e0556" Dec 03 13:54:22 crc kubenswrapper[5052]: E1203 13:54:22.674010 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 13:54:37 crc kubenswrapper[5052]: I1203 13:54:37.672965 5052 scope.go:117] "RemoveContainer" containerID="a4c6deffa55d4faa49789e30d4c8b2762d7fc719390c909181ec730f718e0556" Dec 03 13:54:37 crc kubenswrapper[5052]: E1203 13:54:37.673741 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 13:54:48 crc kubenswrapper[5052]: I1203 13:54:48.673166 5052 scope.go:117] "RemoveContainer" containerID="a4c6deffa55d4faa49789e30d4c8b2762d7fc719390c909181ec730f718e0556" Dec 03 13:54:48 crc kubenswrapper[5052]: E1203 13:54:48.673948 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 13:54:59 crc kubenswrapper[5052]: I1203 13:54:59.676129 5052 scope.go:117] "RemoveContainer" containerID="a4c6deffa55d4faa49789e30d4c8b2762d7fc719390c909181ec730f718e0556" Dec 03 13:54:59 crc kubenswrapper[5052]: E1203 13:54:59.677497 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 13:55:04 crc kubenswrapper[5052]: I1203 13:55:04.952214 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openshift-marketplace/redhat-operators-52bcc"] Dec 03 13:55:04 crc kubenswrapper[5052]: E1203 13:55:04.953105 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="186b2ab5-1c1f-4a2c-90bf-c5dc971d4677" containerName="copy" Dec 03 13:55:04 crc kubenswrapper[5052]: I1203 13:55:04.953121 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="186b2ab5-1c1f-4a2c-90bf-c5dc971d4677" containerName="copy" Dec 03 13:55:04 crc kubenswrapper[5052]: E1203 13:55:04.953136 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="e369fda6-f1c7-47eb-b690-6c268b3000b3" containerName="extract-utilities" Dec 03 13:55:04 crc kubenswrapper[5052]: I1203 13:55:04.953144 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="e369fda6-f1c7-47eb-b690-6c268b3000b3" containerName="extract-utilities" Dec 03 13:55:04 crc kubenswrapper[5052]: E1203 13:55:04.953160 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="f3203993-dab9-40db-9d30-8d8058573a90" containerName="copy" Dec 03 13:55:04 crc kubenswrapper[5052]: I1203 13:55:04.953168 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="f3203993-dab9-40db-9d30-8d8058573a90" containerName="copy" Dec 03 13:55:04 crc kubenswrapper[5052]: E1203 13:55:04.953182 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="e369fda6-f1c7-47eb-b690-6c268b3000b3" containerName="registry-server" Dec 03 13:55:04 crc kubenswrapper[5052]: I1203 13:55:04.953190 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="e369fda6-f1c7-47eb-b690-6c268b3000b3" containerName="registry-server" Dec 03 13:55:04 crc kubenswrapper[5052]: E1203 13:55:04.953217 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="f3203993-dab9-40db-9d30-8d8058573a90" containerName="gather" Dec 03 13:55:04 crc kubenswrapper[5052]: I1203 13:55:04.953224 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="f3203993-dab9-40db-9d30-8d8058573a90" containerName="gather" Dec 03 13:55:04 crc kubenswrapper[5052]: E1203 13:55:04.953243 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="4cc70472-d066-4cab-bde9-9590e1bd74c2" containerName="extract-utilities" Dec 03 13:55:04 crc kubenswrapper[5052]: I1203 13:55:04.953250 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="4cc70472-d066-4cab-bde9-9590e1bd74c2" containerName="extract-utilities" Dec 03 13:55:04 crc kubenswrapper[5052]: E1203 13:55:04.953266 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="186b2ab5-1c1f-4a2c-90bf-c5dc971d4677" containerName="gather" Dec 03 13:55:04 crc kubenswrapper[5052]: I1203 13:55:04.953273 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="186b2ab5-1c1f-4a2c-90bf-c5dc971d4677" containerName="gather" Dec 03 13:55:04 crc kubenswrapper[5052]: E1203 13:55:04.953288 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="4cc70472-d066-4cab-bde9-9590e1bd74c2" containerName="extract-content" Dec 03 13:55:04 crc kubenswrapper[5052]: I1203 13:55:04.953296 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="4cc70472-d066-4cab-bde9-9590e1bd74c2" containerName="extract-content" Dec 03 13:55:04 crc kubenswrapper[5052]: E1203 13:55:04.953307 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="4cc70472-d066-4cab-bde9-9590e1bd74c2" containerName="registry-server" Dec 03 13:55:04 crc kubenswrapper[5052]: I1203 13:55:04.953315 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="4cc70472-d066-4cab-bde9-9590e1bd74c2" containerName="registry-server" Dec 03 13:55:04 crc kubenswrapper[5052]: E1203 13:55:04.953331 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="e369fda6-f1c7-47eb-b690-6c268b3000b3" containerName="extract-content" Dec 03 13:55:04 crc kubenswrapper[5052]: I1203 13:55:04.953339 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="e369fda6-f1c7-47eb-b690-6c268b3000b3" containerName="extract-content" Dec 03 13:55:04 crc kubenswrapper[5052]: I1203 13:55:04.953468 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="e369fda6-f1c7-47eb-b690-6c268b3000b3" containerName="registry-server" Dec 03 13:55:04 crc kubenswrapper[5052]: I1203 13:55:04.953490 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="186b2ab5-1c1f-4a2c-90bf-c5dc971d4677" containerName="gather" Dec 03 13:55:04 crc kubenswrapper[5052]: I1203 13:55:04.953499 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="f3203993-dab9-40db-9d30-8d8058573a90" containerName="gather" Dec 03 13:55:04 crc kubenswrapper[5052]: I1203 13:55:04.953556 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="f3203993-dab9-40db-9d30-8d8058573a90" containerName="copy" Dec 03 13:55:04 crc kubenswrapper[5052]: I1203 13:55:04.953574 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="186b2ab5-1c1f-4a2c-90bf-c5dc971d4677" containerName="copy" Dec 03 13:55:04 crc kubenswrapper[5052]: I1203 13:55:04.953583 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="4cc70472-d066-4cab-bde9-9590e1bd74c2" containerName="registry-server" Dec 03 13:55:04 crc kubenswrapper[5052]: I1203 13:55:04.954571 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/redhat-operators-52bcc" Dec 03 13:55:04 crc kubenswrapper[5052]: I1203 13:55:04.973858 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/redhat-operators-52bcc"] Dec 03 13:55:05 crc kubenswrapper[5052]: I1203 13:55:05.021643 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-btjjv\" (UniqueName: \"kubernetes.io/projected/ab6a1b86-32fe-4e8d-ab14-2b45d134a928-kube-api-access-btjjv\") pod \"redhat-operators-52bcc\" (UID: \"ab6a1b86-32fe-4e8d-ab14-2b45d134a928\") " pod="openshift-marketplace/redhat-operators-52bcc" Dec 03 13:55:05 crc kubenswrapper[5052]: I1203 13:55:05.021705 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/ab6a1b86-32fe-4e8d-ab14-2b45d134a928-catalog-content\") pod \"redhat-operators-52bcc\" (UID: \"ab6a1b86-32fe-4e8d-ab14-2b45d134a928\") " pod="openshift-marketplace/redhat-operators-52bcc" Dec 03 13:55:05 crc kubenswrapper[5052]: I1203 13:55:05.021819 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/ab6a1b86-32fe-4e8d-ab14-2b45d134a928-utilities\") pod \"redhat-operators-52bcc\" (UID: \"ab6a1b86-32fe-4e8d-ab14-2b45d134a928\") " pod="openshift-marketplace/redhat-operators-52bcc" Dec 03 13:55:05 crc kubenswrapper[5052]: I1203 13:55:05.122754 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/ab6a1b86-32fe-4e8d-ab14-2b45d134a928-utilities\") pod \"redhat-operators-52bcc\" (UID: \"ab6a1b86-32fe-4e8d-ab14-2b45d134a928\") " pod="openshift-marketplace/redhat-operators-52bcc" Dec 03 13:55:05 crc kubenswrapper[5052]: I1203 13:55:05.122842 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-btjjv\" (UniqueName: \"kubernetes.io/projected/ab6a1b86-32fe-4e8d-ab14-2b45d134a928-kube-api-access-btjjv\") pod \"redhat-operators-52bcc\" (UID: \"ab6a1b86-32fe-4e8d-ab14-2b45d134a928\") " pod="openshift-marketplace/redhat-operators-52bcc" Dec 03 13:55:05 crc kubenswrapper[5052]: I1203 13:55:05.122867 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/ab6a1b86-32fe-4e8d-ab14-2b45d134a928-catalog-content\") pod \"redhat-operators-52bcc\" (UID: \"ab6a1b86-32fe-4e8d-ab14-2b45d134a928\") " pod="openshift-marketplace/redhat-operators-52bcc" Dec 03 13:55:05 crc kubenswrapper[5052]: I1203 13:55:05.123502 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/ab6a1b86-32fe-4e8d-ab14-2b45d134a928-utilities\") pod \"redhat-operators-52bcc\" (UID: \"ab6a1b86-32fe-4e8d-ab14-2b45d134a928\") " pod="openshift-marketplace/redhat-operators-52bcc" Dec 03 13:55:05 crc kubenswrapper[5052]: I1203 13:55:05.123537 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/ab6a1b86-32fe-4e8d-ab14-2b45d134a928-catalog-content\") pod \"redhat-operators-52bcc\" (UID: \"ab6a1b86-32fe-4e8d-ab14-2b45d134a928\") " pod="openshift-marketplace/redhat-operators-52bcc" Dec 03 13:55:05 crc kubenswrapper[5052]: I1203 13:55:05.142904 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-btjjv\" (UniqueName: \"kubernetes.io/projected/ab6a1b86-32fe-4e8d-ab14-2b45d134a928-kube-api-access-btjjv\") pod \"redhat-operators-52bcc\" (UID: \"ab6a1b86-32fe-4e8d-ab14-2b45d134a928\") " pod="openshift-marketplace/redhat-operators-52bcc" Dec 03 13:55:05 crc kubenswrapper[5052]: I1203 13:55:05.149280 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openshift-marketplace/certified-operators-442ll"] Dec 03 13:55:05 crc kubenswrapper[5052]: I1203 13:55:05.150484 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/certified-operators-442ll" Dec 03 13:55:05 crc kubenswrapper[5052]: I1203 13:55:05.163594 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/certified-operators-442ll"] Dec 03 13:55:05 crc kubenswrapper[5052]: I1203 13:55:05.224262 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lfpz9\" (UniqueName: \"kubernetes.io/projected/cf3076f1-790b-444d-ac11-7977417b97e6-kube-api-access-lfpz9\") pod \"certified-operators-442ll\" (UID: \"cf3076f1-790b-444d-ac11-7977417b97e6\") " pod="openshift-marketplace/certified-operators-442ll" Dec 03 13:55:05 crc kubenswrapper[5052]: I1203 13:55:05.224337 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/cf3076f1-790b-444d-ac11-7977417b97e6-utilities\") pod \"certified-operators-442ll\" (UID: \"cf3076f1-790b-444d-ac11-7977417b97e6\") " pod="openshift-marketplace/certified-operators-442ll" Dec 03 13:55:05 crc kubenswrapper[5052]: I1203 13:55:05.224400 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/cf3076f1-790b-444d-ac11-7977417b97e6-catalog-content\") pod \"certified-operators-442ll\" (UID: \"cf3076f1-790b-444d-ac11-7977417b97e6\") " pod="openshift-marketplace/certified-operators-442ll" Dec 03 13:55:05 crc kubenswrapper[5052]: I1203 13:55:05.273110 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/redhat-operators-52bcc" Dec 03 13:55:05 crc kubenswrapper[5052]: I1203 13:55:05.326322 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/cf3076f1-790b-444d-ac11-7977417b97e6-utilities\") pod \"certified-operators-442ll\" (UID: \"cf3076f1-790b-444d-ac11-7977417b97e6\") " pod="openshift-marketplace/certified-operators-442ll" Dec 03 13:55:05 crc kubenswrapper[5052]: I1203 13:55:05.326677 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/cf3076f1-790b-444d-ac11-7977417b97e6-catalog-content\") pod \"certified-operators-442ll\" (UID: \"cf3076f1-790b-444d-ac11-7977417b97e6\") " pod="openshift-marketplace/certified-operators-442ll" Dec 03 13:55:05 crc kubenswrapper[5052]: I1203 13:55:05.326763 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-lfpz9\" (UniqueName: \"kubernetes.io/projected/cf3076f1-790b-444d-ac11-7977417b97e6-kube-api-access-lfpz9\") pod \"certified-operators-442ll\" (UID: \"cf3076f1-790b-444d-ac11-7977417b97e6\") " pod="openshift-marketplace/certified-operators-442ll" Dec 03 13:55:05 crc kubenswrapper[5052]: I1203 13:55:05.326892 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/cf3076f1-790b-444d-ac11-7977417b97e6-utilities\") pod \"certified-operators-442ll\" (UID: \"cf3076f1-790b-444d-ac11-7977417b97e6\") " pod="openshift-marketplace/certified-operators-442ll" Dec 03 13:55:05 crc kubenswrapper[5052]: I1203 13:55:05.327102 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/cf3076f1-790b-444d-ac11-7977417b97e6-catalog-content\") pod \"certified-operators-442ll\" (UID: \"cf3076f1-790b-444d-ac11-7977417b97e6\") " pod="openshift-marketplace/certified-operators-442ll" Dec 03 13:55:05 crc kubenswrapper[5052]: I1203 13:55:05.352262 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-lfpz9\" (UniqueName: \"kubernetes.io/projected/cf3076f1-790b-444d-ac11-7977417b97e6-kube-api-access-lfpz9\") pod \"certified-operators-442ll\" (UID: \"cf3076f1-790b-444d-ac11-7977417b97e6\") " pod="openshift-marketplace/certified-operators-442ll" Dec 03 13:55:05 crc kubenswrapper[5052]: I1203 13:55:05.484881 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/certified-operators-442ll" Dec 03 13:55:05 crc kubenswrapper[5052]: I1203 13:55:05.558675 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/redhat-operators-52bcc"] Dec 03 13:55:06 crc kubenswrapper[5052]: I1203 13:55:06.004297 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/certified-operators-442ll"] Dec 03 13:55:06 crc kubenswrapper[5052]: W1203 13:55:06.007249 5052 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf3076f1_790b_444d_ac11_7977417b97e6.slice/crio-28ccde983f1229a3ba7c6eed974ce5196d5f6c5f8f503066a7298f278c323b23 WatchSource:0}: Error finding container 28ccde983f1229a3ba7c6eed974ce5196d5f6c5f8f503066a7298f278c323b23: Status 404 returned error can't find the container with id 28ccde983f1229a3ba7c6eed974ce5196d5f6c5f8f503066a7298f278c323b23 Dec 03 13:55:06 crc kubenswrapper[5052]: I1203 13:55:06.296559 5052 generic.go:334] "Generic (PLEG): container finished" podID="ab6a1b86-32fe-4e8d-ab14-2b45d134a928" containerID="fb0933c6a2140c110c79121f4c2f8840664dc910833f7170b1c20035aacfd84d" exitCode=0 Dec 03 13:55:06 crc kubenswrapper[5052]: I1203 13:55:06.296637 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-operators-52bcc" event={"ID":"ab6a1b86-32fe-4e8d-ab14-2b45d134a928","Type":"ContainerDied","Data":"fb0933c6a2140c110c79121f4c2f8840664dc910833f7170b1c20035aacfd84d"} Dec 03 13:55:06 crc kubenswrapper[5052]: I1203 13:55:06.296686 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-operators-52bcc" event={"ID":"ab6a1b86-32fe-4e8d-ab14-2b45d134a928","Type":"ContainerStarted","Data":"5c219fab77777589262771f9ca6ff95da7324b750f156d9555f8420df7eea2c0"} Dec 03 13:55:06 crc kubenswrapper[5052]: I1203 13:55:06.298181 5052 generic.go:334] "Generic (PLEG): container finished" podID="cf3076f1-790b-444d-ac11-7977417b97e6" containerID="1ae6f0bf4f450a71b8c7ef96cb75612b7e04f89387804d28ecbde2711c0ff38e" exitCode=0 Dec 03 13:55:06 crc kubenswrapper[5052]: I1203 13:55:06.298216 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/certified-operators-442ll" event={"ID":"cf3076f1-790b-444d-ac11-7977417b97e6","Type":"ContainerDied","Data":"1ae6f0bf4f450a71b8c7ef96cb75612b7e04f89387804d28ecbde2711c0ff38e"} Dec 03 13:55:06 crc kubenswrapper[5052]: I1203 13:55:06.298239 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/certified-operators-442ll" event={"ID":"cf3076f1-790b-444d-ac11-7977417b97e6","Type":"ContainerStarted","Data":"28ccde983f1229a3ba7c6eed974ce5196d5f6c5f8f503066a7298f278c323b23"} Dec 03 13:55:06 crc kubenswrapper[5052]: I1203 13:55:06.298315 5052 provider.go:102] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Dec 03 13:55:08 crc kubenswrapper[5052]: I1203 13:55:08.316756 5052 generic.go:334] "Generic (PLEG): container finished" podID="ab6a1b86-32fe-4e8d-ab14-2b45d134a928" containerID="7e474f482ae7751f0aa80b89eaa59163a14c7c15b642e83f71a2d440000b5a42" exitCode=0 Dec 03 13:55:08 crc kubenswrapper[5052]: I1203 13:55:08.317152 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-operators-52bcc" event={"ID":"ab6a1b86-32fe-4e8d-ab14-2b45d134a928","Type":"ContainerDied","Data":"7e474f482ae7751f0aa80b89eaa59163a14c7c15b642e83f71a2d440000b5a42"} Dec 03 13:55:08 crc kubenswrapper[5052]: I1203 13:55:08.320290 5052 generic.go:334] "Generic (PLEG): container finished" podID="cf3076f1-790b-444d-ac11-7977417b97e6" containerID="5d41aed28167854b7847df5406764586e887d80e33fdcdf3739a5d739a9fcd03" exitCode=0 Dec 03 13:55:08 crc kubenswrapper[5052]: I1203 13:55:08.320459 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/certified-operators-442ll" event={"ID":"cf3076f1-790b-444d-ac11-7977417b97e6","Type":"ContainerDied","Data":"5d41aed28167854b7847df5406764586e887d80e33fdcdf3739a5d739a9fcd03"} Dec 03 13:55:09 crc kubenswrapper[5052]: I1203 13:55:09.331224 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/certified-operators-442ll" event={"ID":"cf3076f1-790b-444d-ac11-7977417b97e6","Type":"ContainerStarted","Data":"0281205f5e784a326e497f2152cb5279cf08eddfa95258c79ad0971b52c4e5e7"} Dec 03 13:55:09 crc kubenswrapper[5052]: I1203 13:55:09.335177 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-operators-52bcc" event={"ID":"ab6a1b86-32fe-4e8d-ab14-2b45d134a928","Type":"ContainerStarted","Data":"c4729575387da3dc9bca3aa75fd25e434348080315c9f2fc03fcda713f175726"} Dec 03 13:55:09 crc kubenswrapper[5052]: I1203 13:55:09.354021 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-marketplace/certified-operators-442ll" podStartSLOduration=1.895443116 podStartE2EDuration="4.354002136s" podCreationTimestamp="2025-12-03 13:55:05 +0000 UTC" firstStartedPulling="2025-12-03 13:55:06.300005485 +0000 UTC m=+9296.888247259" lastFinishedPulling="2025-12-03 13:55:08.758564515 +0000 UTC m=+9299.346806279" observedRunningTime="2025-12-03 13:55:09.349181118 +0000 UTC m=+9299.937422882" watchObservedRunningTime="2025-12-03 13:55:09.354002136 +0000 UTC m=+9299.942243900" Dec 03 13:55:09 crc kubenswrapper[5052]: I1203 13:55:09.372867 5052 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-marketplace/redhat-operators-52bcc" podStartSLOduration=2.923595485 podStartE2EDuration="5.372835927s" podCreationTimestamp="2025-12-03 13:55:04 +0000 UTC" firstStartedPulling="2025-12-03 13:55:06.298044749 +0000 UTC m=+9296.886286513" lastFinishedPulling="2025-12-03 13:55:08.747285191 +0000 UTC m=+9299.335526955" observedRunningTime="2025-12-03 13:55:09.369040748 +0000 UTC m=+9299.957282532" watchObservedRunningTime="2025-12-03 13:55:09.372835927 +0000 UTC m=+9299.961077691" Dec 03 13:55:12 crc kubenswrapper[5052]: I1203 13:55:12.673289 5052 scope.go:117] "RemoveContainer" containerID="a4c6deffa55d4faa49789e30d4c8b2762d7fc719390c909181ec730f718e0556" Dec 03 13:55:12 crc kubenswrapper[5052]: E1203 13:55:12.674151 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 13:55:15 crc kubenswrapper[5052]: I1203 13:55:15.274204 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-marketplace/redhat-operators-52bcc" Dec 03 13:55:15 crc kubenswrapper[5052]: I1203 13:55:15.275751 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openshift-marketplace/redhat-operators-52bcc" Dec 03 13:55:15 crc kubenswrapper[5052]: I1203 13:55:15.321573 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-marketplace/redhat-operators-52bcc" Dec 03 13:55:15 crc kubenswrapper[5052]: I1203 13:55:15.421080 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-marketplace/redhat-operators-52bcc" Dec 03 13:55:15 crc kubenswrapper[5052]: I1203 13:55:15.485317 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="" pod="openshift-marketplace/certified-operators-442ll" Dec 03 13:55:15 crc kubenswrapper[5052]: I1203 13:55:15.485367 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-marketplace/certified-operators-442ll" Dec 03 13:55:15 crc kubenswrapper[5052]: I1203 13:55:15.523858 5052 kubelet.go:2542] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-marketplace/certified-operators-442ll" Dec 03 13:55:15 crc kubenswrapper[5052]: I1203 13:55:15.555456 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openshift-marketplace/redhat-operators-52bcc"] Dec 03 13:55:16 crc kubenswrapper[5052]: I1203 13:55:16.957105 5052 kubelet.go:2542] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-marketplace/certified-operators-442ll" Dec 03 13:55:17 crc kubenswrapper[5052]: I1203 13:55:17.387080 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openshift-marketplace/redhat-operators-52bcc" podUID="ab6a1b86-32fe-4e8d-ab14-2b45d134a928" containerName="registry-server" containerID="cri-o://c4729575387da3dc9bca3aa75fd25e434348080315c9f2fc03fcda713f175726" gracePeriod=2 Dec 03 13:55:17 crc kubenswrapper[5052]: I1203 13:55:17.757152 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openshift-marketplace/certified-operators-442ll"] Dec 03 13:55:17 crc kubenswrapper[5052]: I1203 13:55:17.759847 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/redhat-operators-52bcc" Dec 03 13:55:17 crc kubenswrapper[5052]: I1203 13:55:17.845659 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/ab6a1b86-32fe-4e8d-ab14-2b45d134a928-catalog-content\") pod \"ab6a1b86-32fe-4e8d-ab14-2b45d134a928\" (UID: \"ab6a1b86-32fe-4e8d-ab14-2b45d134a928\") " Dec 03 13:55:17 crc kubenswrapper[5052]: I1203 13:55:17.845726 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/ab6a1b86-32fe-4e8d-ab14-2b45d134a928-utilities\") pod \"ab6a1b86-32fe-4e8d-ab14-2b45d134a928\" (UID: \"ab6a1b86-32fe-4e8d-ab14-2b45d134a928\") " Dec 03 13:55:17 crc kubenswrapper[5052]: I1203 13:55:17.845773 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-btjjv\" (UniqueName: \"kubernetes.io/projected/ab6a1b86-32fe-4e8d-ab14-2b45d134a928-kube-api-access-btjjv\") pod \"ab6a1b86-32fe-4e8d-ab14-2b45d134a928\" (UID: \"ab6a1b86-32fe-4e8d-ab14-2b45d134a928\") " Dec 03 13:55:17 crc kubenswrapper[5052]: I1203 13:55:17.846571 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ab6a1b86-32fe-4e8d-ab14-2b45d134a928-utilities" (OuterVolumeSpecName: "utilities") pod "ab6a1b86-32fe-4e8d-ab14-2b45d134a928" (UID: "ab6a1b86-32fe-4e8d-ab14-2b45d134a928"). InnerVolumeSpecName "utilities". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 13:55:17 crc kubenswrapper[5052]: I1203 13:55:17.850592 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ab6a1b86-32fe-4e8d-ab14-2b45d134a928-kube-api-access-btjjv" (OuterVolumeSpecName: "kube-api-access-btjjv") pod "ab6a1b86-32fe-4e8d-ab14-2b45d134a928" (UID: "ab6a1b86-32fe-4e8d-ab14-2b45d134a928"). InnerVolumeSpecName "kube-api-access-btjjv". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:55:17 crc kubenswrapper[5052]: I1203 13:55:17.947271 5052 reconciler_common.go:293] "Volume detached for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/ab6a1b86-32fe-4e8d-ab14-2b45d134a928-utilities\") on node \"crc\" DevicePath \"\"" Dec 03 13:55:17 crc kubenswrapper[5052]: I1203 13:55:17.947333 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-btjjv\" (UniqueName: \"kubernetes.io/projected/ab6a1b86-32fe-4e8d-ab14-2b45d134a928-kube-api-access-btjjv\") on node \"crc\" DevicePath \"\"" Dec 03 13:55:18 crc kubenswrapper[5052]: I1203 13:55:18.147528 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ab6a1b86-32fe-4e8d-ab14-2b45d134a928-catalog-content" (OuterVolumeSpecName: "catalog-content") pod "ab6a1b86-32fe-4e8d-ab14-2b45d134a928" (UID: "ab6a1b86-32fe-4e8d-ab14-2b45d134a928"). InnerVolumeSpecName "catalog-content". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 13:55:18 crc kubenswrapper[5052]: I1203 13:55:18.149360 5052 reconciler_common.go:293] "Volume detached for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/ab6a1b86-32fe-4e8d-ab14-2b45d134a928-catalog-content\") on node \"crc\" DevicePath \"\"" Dec 03 13:55:18 crc kubenswrapper[5052]: I1203 13:55:18.394627 5052 generic.go:334] "Generic (PLEG): container finished" podID="ab6a1b86-32fe-4e8d-ab14-2b45d134a928" containerID="c4729575387da3dc9bca3aa75fd25e434348080315c9f2fc03fcda713f175726" exitCode=0 Dec 03 13:55:18 crc kubenswrapper[5052]: I1203 13:55:18.394964 5052 kuberuntime_container.go:808] "Killing container with a grace period" pod="openshift-marketplace/certified-operators-442ll" podUID="cf3076f1-790b-444d-ac11-7977417b97e6" containerName="registry-server" containerID="cri-o://0281205f5e784a326e497f2152cb5279cf08eddfa95258c79ad0971b52c4e5e7" gracePeriod=2 Dec 03 13:55:18 crc kubenswrapper[5052]: I1203 13:55:18.395359 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/redhat-operators-52bcc" Dec 03 13:55:18 crc kubenswrapper[5052]: I1203 13:55:18.395937 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-operators-52bcc" event={"ID":"ab6a1b86-32fe-4e8d-ab14-2b45d134a928","Type":"ContainerDied","Data":"c4729575387da3dc9bca3aa75fd25e434348080315c9f2fc03fcda713f175726"} Dec 03 13:55:18 crc kubenswrapper[5052]: I1203 13:55:18.396092 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/redhat-operators-52bcc" event={"ID":"ab6a1b86-32fe-4e8d-ab14-2b45d134a928","Type":"ContainerDied","Data":"5c219fab77777589262771f9ca6ff95da7324b750f156d9555f8420df7eea2c0"} Dec 03 13:55:18 crc kubenswrapper[5052]: I1203 13:55:18.396111 5052 scope.go:117] "RemoveContainer" containerID="c4729575387da3dc9bca3aa75fd25e434348080315c9f2fc03fcda713f175726" Dec 03 13:55:18 crc kubenswrapper[5052]: I1203 13:55:18.421206 5052 scope.go:117] "RemoveContainer" containerID="7e474f482ae7751f0aa80b89eaa59163a14c7c15b642e83f71a2d440000b5a42" Dec 03 13:55:18 crc kubenswrapper[5052]: I1203 13:55:18.436049 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openshift-marketplace/redhat-operators-52bcc"] Dec 03 13:55:18 crc kubenswrapper[5052]: I1203 13:55:18.442441 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openshift-marketplace/redhat-operators-52bcc"] Dec 03 13:55:18 crc kubenswrapper[5052]: I1203 13:55:18.540304 5052 scope.go:117] "RemoveContainer" containerID="fb0933c6a2140c110c79121f4c2f8840664dc910833f7170b1c20035aacfd84d" Dec 03 13:55:18 crc kubenswrapper[5052]: I1203 13:55:18.559463 5052 scope.go:117] "RemoveContainer" containerID="c4729575387da3dc9bca3aa75fd25e434348080315c9f2fc03fcda713f175726" Dec 03 13:55:18 crc kubenswrapper[5052]: E1203 13:55:18.560035 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c4729575387da3dc9bca3aa75fd25e434348080315c9f2fc03fcda713f175726\": container with ID starting with c4729575387da3dc9bca3aa75fd25e434348080315c9f2fc03fcda713f175726 not found: ID does not exist" containerID="c4729575387da3dc9bca3aa75fd25e434348080315c9f2fc03fcda713f175726" Dec 03 13:55:18 crc kubenswrapper[5052]: I1203 13:55:18.560074 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c4729575387da3dc9bca3aa75fd25e434348080315c9f2fc03fcda713f175726"} err="failed to get container status \"c4729575387da3dc9bca3aa75fd25e434348080315c9f2fc03fcda713f175726\": rpc error: code = NotFound desc = could not find container \"c4729575387da3dc9bca3aa75fd25e434348080315c9f2fc03fcda713f175726\": container with ID starting with c4729575387da3dc9bca3aa75fd25e434348080315c9f2fc03fcda713f175726 not found: ID does not exist" Dec 03 13:55:18 crc kubenswrapper[5052]: I1203 13:55:18.560099 5052 scope.go:117] "RemoveContainer" containerID="7e474f482ae7751f0aa80b89eaa59163a14c7c15b642e83f71a2d440000b5a42" Dec 03 13:55:18 crc kubenswrapper[5052]: E1203 13:55:18.560466 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7e474f482ae7751f0aa80b89eaa59163a14c7c15b642e83f71a2d440000b5a42\": container with ID starting with 7e474f482ae7751f0aa80b89eaa59163a14c7c15b642e83f71a2d440000b5a42 not found: ID does not exist" containerID="7e474f482ae7751f0aa80b89eaa59163a14c7c15b642e83f71a2d440000b5a42" Dec 03 13:55:18 crc kubenswrapper[5052]: I1203 13:55:18.560501 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7e474f482ae7751f0aa80b89eaa59163a14c7c15b642e83f71a2d440000b5a42"} err="failed to get container status \"7e474f482ae7751f0aa80b89eaa59163a14c7c15b642e83f71a2d440000b5a42\": rpc error: code = NotFound desc = could not find container \"7e474f482ae7751f0aa80b89eaa59163a14c7c15b642e83f71a2d440000b5a42\": container with ID starting with 7e474f482ae7751f0aa80b89eaa59163a14c7c15b642e83f71a2d440000b5a42 not found: ID does not exist" Dec 03 13:55:18 crc kubenswrapper[5052]: I1203 13:55:18.560545 5052 scope.go:117] "RemoveContainer" containerID="fb0933c6a2140c110c79121f4c2f8840664dc910833f7170b1c20035aacfd84d" Dec 03 13:55:18 crc kubenswrapper[5052]: E1203 13:55:18.560820 5052 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fb0933c6a2140c110c79121f4c2f8840664dc910833f7170b1c20035aacfd84d\": container with ID starting with fb0933c6a2140c110c79121f4c2f8840664dc910833f7170b1c20035aacfd84d not found: ID does not exist" containerID="fb0933c6a2140c110c79121f4c2f8840664dc910833f7170b1c20035aacfd84d" Dec 03 13:55:18 crc kubenswrapper[5052]: I1203 13:55:18.560852 5052 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fb0933c6a2140c110c79121f4c2f8840664dc910833f7170b1c20035aacfd84d"} err="failed to get container status \"fb0933c6a2140c110c79121f4c2f8840664dc910833f7170b1c20035aacfd84d\": rpc error: code = NotFound desc = could not find container \"fb0933c6a2140c110c79121f4c2f8840664dc910833f7170b1c20035aacfd84d\": container with ID starting with fb0933c6a2140c110c79121f4c2f8840664dc910833f7170b1c20035aacfd84d not found: ID does not exist" Dec 03 13:55:19 crc kubenswrapper[5052]: I1203 13:55:19.405071 5052 generic.go:334] "Generic (PLEG): container finished" podID="cf3076f1-790b-444d-ac11-7977417b97e6" containerID="0281205f5e784a326e497f2152cb5279cf08eddfa95258c79ad0971b52c4e5e7" exitCode=0 Dec 03 13:55:19 crc kubenswrapper[5052]: I1203 13:55:19.405128 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/certified-operators-442ll" event={"ID":"cf3076f1-790b-444d-ac11-7977417b97e6","Type":"ContainerDied","Data":"0281205f5e784a326e497f2152cb5279cf08eddfa95258c79ad0971b52c4e5e7"} Dec 03 13:55:19 crc kubenswrapper[5052]: I1203 13:55:19.686329 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ab6a1b86-32fe-4e8d-ab14-2b45d134a928" path="/var/lib/kubelet/pods/ab6a1b86-32fe-4e8d-ab14-2b45d134a928/volumes" Dec 03 13:55:20 crc kubenswrapper[5052]: I1203 13:55:20.056467 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/certified-operators-442ll" Dec 03 13:55:20 crc kubenswrapper[5052]: I1203 13:55:20.176399 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lfpz9\" (UniqueName: \"kubernetes.io/projected/cf3076f1-790b-444d-ac11-7977417b97e6-kube-api-access-lfpz9\") pod \"cf3076f1-790b-444d-ac11-7977417b97e6\" (UID: \"cf3076f1-790b-444d-ac11-7977417b97e6\") " Dec 03 13:55:20 crc kubenswrapper[5052]: I1203 13:55:20.176538 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/cf3076f1-790b-444d-ac11-7977417b97e6-utilities\") pod \"cf3076f1-790b-444d-ac11-7977417b97e6\" (UID: \"cf3076f1-790b-444d-ac11-7977417b97e6\") " Dec 03 13:55:20 crc kubenswrapper[5052]: I1203 13:55:20.176578 5052 reconciler_common.go:159] "operationExecutor.UnmountVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/cf3076f1-790b-444d-ac11-7977417b97e6-catalog-content\") pod \"cf3076f1-790b-444d-ac11-7977417b97e6\" (UID: \"cf3076f1-790b-444d-ac11-7977417b97e6\") " Dec 03 13:55:20 crc kubenswrapper[5052]: I1203 13:55:20.178181 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf3076f1-790b-444d-ac11-7977417b97e6-utilities" (OuterVolumeSpecName: "utilities") pod "cf3076f1-790b-444d-ac11-7977417b97e6" (UID: "cf3076f1-790b-444d-ac11-7977417b97e6"). InnerVolumeSpecName "utilities". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 13:55:20 crc kubenswrapper[5052]: I1203 13:55:20.181970 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cf3076f1-790b-444d-ac11-7977417b97e6-kube-api-access-lfpz9" (OuterVolumeSpecName: "kube-api-access-lfpz9") pod "cf3076f1-790b-444d-ac11-7977417b97e6" (UID: "cf3076f1-790b-444d-ac11-7977417b97e6"). InnerVolumeSpecName "kube-api-access-lfpz9". PluginName "kubernetes.io/projected", VolumeGidValue "" Dec 03 13:55:20 crc kubenswrapper[5052]: I1203 13:55:20.226628 5052 operation_generator.go:803] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf3076f1-790b-444d-ac11-7977417b97e6-catalog-content" (OuterVolumeSpecName: "catalog-content") pod "cf3076f1-790b-444d-ac11-7977417b97e6" (UID: "cf3076f1-790b-444d-ac11-7977417b97e6"). InnerVolumeSpecName "catalog-content". PluginName "kubernetes.io/empty-dir", VolumeGidValue "" Dec 03 13:55:20 crc kubenswrapper[5052]: I1203 13:55:20.278420 5052 reconciler_common.go:293] "Volume detached for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/cf3076f1-790b-444d-ac11-7977417b97e6-catalog-content\") on node \"crc\" DevicePath \"\"" Dec 03 13:55:20 crc kubenswrapper[5052]: I1203 13:55:20.278480 5052 reconciler_common.go:293] "Volume detached for volume \"kube-api-access-lfpz9\" (UniqueName: \"kubernetes.io/projected/cf3076f1-790b-444d-ac11-7977417b97e6-kube-api-access-lfpz9\") on node \"crc\" DevicePath \"\"" Dec 03 13:55:20 crc kubenswrapper[5052]: I1203 13:55:20.278496 5052 reconciler_common.go:293] "Volume detached for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/cf3076f1-790b-444d-ac11-7977417b97e6-utilities\") on node \"crc\" DevicePath \"\"" Dec 03 13:55:20 crc kubenswrapper[5052]: I1203 13:55:20.424316 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/certified-operators-442ll" event={"ID":"cf3076f1-790b-444d-ac11-7977417b97e6","Type":"ContainerDied","Data":"28ccde983f1229a3ba7c6eed974ce5196d5f6c5f8f503066a7298f278c323b23"} Dec 03 13:55:20 crc kubenswrapper[5052]: I1203 13:55:20.424401 5052 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/certified-operators-442ll" Dec 03 13:55:20 crc kubenswrapper[5052]: I1203 13:55:20.424431 5052 scope.go:117] "RemoveContainer" containerID="0281205f5e784a326e497f2152cb5279cf08eddfa95258c79ad0971b52c4e5e7" Dec 03 13:55:20 crc kubenswrapper[5052]: I1203 13:55:20.442691 5052 scope.go:117] "RemoveContainer" containerID="5d41aed28167854b7847df5406764586e887d80e33fdcdf3739a5d739a9fcd03" Dec 03 13:55:20 crc kubenswrapper[5052]: I1203 13:55:20.460730 5052 kubelet.go:2437] "SyncLoop DELETE" source="api" pods=["openshift-marketplace/certified-operators-442ll"] Dec 03 13:55:20 crc kubenswrapper[5052]: I1203 13:55:20.464344 5052 scope.go:117] "RemoveContainer" containerID="1ae6f0bf4f450a71b8c7ef96cb75612b7e04f89387804d28ecbde2711c0ff38e" Dec 03 13:55:20 crc kubenswrapper[5052]: I1203 13:55:20.467165 5052 kubelet.go:2431] "SyncLoop REMOVE" source="api" pods=["openshift-marketplace/certified-operators-442ll"] Dec 03 13:55:21 crc kubenswrapper[5052]: I1203 13:55:21.680333 5052 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cf3076f1-790b-444d-ac11-7977417b97e6" path="/var/lib/kubelet/pods/cf3076f1-790b-444d-ac11-7977417b97e6/volumes" Dec 03 13:55:23 crc kubenswrapper[5052]: I1203 13:55:23.673277 5052 scope.go:117] "RemoveContainer" containerID="a4c6deffa55d4faa49789e30d4c8b2762d7fc719390c909181ec730f718e0556" Dec 03 13:55:23 crc kubenswrapper[5052]: E1203 13:55:23.673951 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 13:55:36 crc kubenswrapper[5052]: I1203 13:55:36.673123 5052 scope.go:117] "RemoveContainer" containerID="a4c6deffa55d4faa49789e30d4c8b2762d7fc719390c909181ec730f718e0556" Dec 03 13:55:36 crc kubenswrapper[5052]: E1203 13:55:36.674072 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 13:55:50 crc kubenswrapper[5052]: I1203 13:55:50.672997 5052 scope.go:117] "RemoveContainer" containerID="a4c6deffa55d4faa49789e30d4c8b2762d7fc719390c909181ec730f718e0556" Dec 03 13:55:50 crc kubenswrapper[5052]: E1203 13:55:50.673836 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 13:56:02 crc kubenswrapper[5052]: I1203 13:56:02.673132 5052 scope.go:117] "RemoveContainer" containerID="a4c6deffa55d4faa49789e30d4c8b2762d7fc719390c909181ec730f718e0556" Dec 03 13:56:02 crc kubenswrapper[5052]: E1203 13:56:02.673891 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 13:56:15 crc kubenswrapper[5052]: I1203 13:56:15.673292 5052 scope.go:117] "RemoveContainer" containerID="a4c6deffa55d4faa49789e30d4c8b2762d7fc719390c909181ec730f718e0556" Dec 03 13:56:15 crc kubenswrapper[5052]: E1203 13:56:15.674042 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 13:56:30 crc kubenswrapper[5052]: I1203 13:56:30.673631 5052 scope.go:117] "RemoveContainer" containerID="a4c6deffa55d4faa49789e30d4c8b2762d7fc719390c909181ec730f718e0556" Dec 03 13:56:30 crc kubenswrapper[5052]: E1203 13:56:30.674486 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 13:56:43 crc kubenswrapper[5052]: I1203 13:56:43.673566 5052 scope.go:117] "RemoveContainer" containerID="a4c6deffa55d4faa49789e30d4c8b2762d7fc719390c909181ec730f718e0556" Dec 03 13:56:43 crc kubenswrapper[5052]: E1203 13:56:43.675564 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 13:56:57 crc kubenswrapper[5052]: I1203 13:56:57.672946 5052 scope.go:117] "RemoveContainer" containerID="a4c6deffa55d4faa49789e30d4c8b2762d7fc719390c909181ec730f718e0556" Dec 03 13:56:57 crc kubenswrapper[5052]: E1203 13:56:57.673641 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 13:57:10 crc kubenswrapper[5052]: I1203 13:57:10.672734 5052 scope.go:117] "RemoveContainer" containerID="a4c6deffa55d4faa49789e30d4c8b2762d7fc719390c909181ec730f718e0556" Dec 03 13:57:10 crc kubenswrapper[5052]: E1203 13:57:10.673542 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 13:57:25 crc kubenswrapper[5052]: I1203 13:57:25.673737 5052 scope.go:117] "RemoveContainer" containerID="a4c6deffa55d4faa49789e30d4c8b2762d7fc719390c909181ec730f718e0556" Dec 03 13:57:25 crc kubenswrapper[5052]: E1203 13:57:25.674953 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 13:57:39 crc kubenswrapper[5052]: I1203 13:57:39.677096 5052 scope.go:117] "RemoveContainer" containerID="a4c6deffa55d4faa49789e30d4c8b2762d7fc719390c909181ec730f718e0556" Dec 03 13:57:39 crc kubenswrapper[5052]: E1203 13:57:39.679281 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 13:57:51 crc kubenswrapper[5052]: I1203 13:57:51.673088 5052 scope.go:117] "RemoveContainer" containerID="a4c6deffa55d4faa49789e30d4c8b2762d7fc719390c909181ec730f718e0556" Dec 03 13:57:51 crc kubenswrapper[5052]: E1203 13:57:51.673867 5052 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"machine-config-daemon\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=machine-config-daemon pod=machine-config-daemon-74bj6_openshift-machine-config-operator(10c237bc-72e7-4da3-9f2c-b1688395bf6e)\"" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" podUID="10c237bc-72e7-4da3-9f2c-b1688395bf6e" Dec 03 13:58:02 crc kubenswrapper[5052]: I1203 13:58:02.672500 5052 scope.go:117] "RemoveContainer" containerID="a4c6deffa55d4faa49789e30d4c8b2762d7fc719390c909181ec730f718e0556" Dec 03 13:58:03 crc kubenswrapper[5052]: I1203 13:58:03.466306 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/machine-config-daemon-74bj6" event={"ID":"10c237bc-72e7-4da3-9f2c-b1688395bf6e","Type":"ContainerStarted","Data":"c2b19d10fb3716a32dcb4fa7b808a9b20e5c7da5ef4b129f091624905145bb20"} Dec 03 13:58:15 crc kubenswrapper[5052]: I1203 13:58:15.811257 5052 kubelet.go:2421] "SyncLoop ADD" source="api" pods=["openshift-marketplace/community-operators-8gqw8"] Dec 03 13:58:15 crc kubenswrapper[5052]: E1203 13:58:15.812948 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="cf3076f1-790b-444d-ac11-7977417b97e6" containerName="registry-server" Dec 03 13:58:15 crc kubenswrapper[5052]: I1203 13:58:15.812962 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="cf3076f1-790b-444d-ac11-7977417b97e6" containerName="registry-server" Dec 03 13:58:15 crc kubenswrapper[5052]: E1203 13:58:15.812983 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="cf3076f1-790b-444d-ac11-7977417b97e6" containerName="extract-utilities" Dec 03 13:58:15 crc kubenswrapper[5052]: I1203 13:58:15.812996 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="cf3076f1-790b-444d-ac11-7977417b97e6" containerName="extract-utilities" Dec 03 13:58:15 crc kubenswrapper[5052]: E1203 13:58:15.813014 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="ab6a1b86-32fe-4e8d-ab14-2b45d134a928" containerName="extract-content" Dec 03 13:58:15 crc kubenswrapper[5052]: I1203 13:58:15.813022 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="ab6a1b86-32fe-4e8d-ab14-2b45d134a928" containerName="extract-content" Dec 03 13:58:15 crc kubenswrapper[5052]: E1203 13:58:15.813039 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="ab6a1b86-32fe-4e8d-ab14-2b45d134a928" containerName="registry-server" Dec 03 13:58:15 crc kubenswrapper[5052]: I1203 13:58:15.813046 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="ab6a1b86-32fe-4e8d-ab14-2b45d134a928" containerName="registry-server" Dec 03 13:58:15 crc kubenswrapper[5052]: E1203 13:58:15.813055 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="cf3076f1-790b-444d-ac11-7977417b97e6" containerName="extract-content" Dec 03 13:58:15 crc kubenswrapper[5052]: I1203 13:58:15.813062 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="cf3076f1-790b-444d-ac11-7977417b97e6" containerName="extract-content" Dec 03 13:58:15 crc kubenswrapper[5052]: E1203 13:58:15.813075 5052 cpu_manager.go:410] "RemoveStaleState: removing container" podUID="ab6a1b86-32fe-4e8d-ab14-2b45d134a928" containerName="extract-utilities" Dec 03 13:58:15 crc kubenswrapper[5052]: I1203 13:58:15.813082 5052 state_mem.go:107] "Deleted CPUSet assignment" podUID="ab6a1b86-32fe-4e8d-ab14-2b45d134a928" containerName="extract-utilities" Dec 03 13:58:15 crc kubenswrapper[5052]: I1203 13:58:15.813201 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="ab6a1b86-32fe-4e8d-ab14-2b45d134a928" containerName="registry-server" Dec 03 13:58:15 crc kubenswrapper[5052]: I1203 13:58:15.813210 5052 memory_manager.go:354] "RemoveStaleState removing state" podUID="cf3076f1-790b-444d-ac11-7977417b97e6" containerName="registry-server" Dec 03 13:58:15 crc kubenswrapper[5052]: I1203 13:58:15.814789 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/community-operators-8gqw8" Dec 03 13:58:15 crc kubenswrapper[5052]: I1203 13:58:15.844666 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/e3ece61f-eaea-4b55-abb9-2326bd73d323-catalog-content\") pod \"community-operators-8gqw8\" (UID: \"e3ece61f-eaea-4b55-abb9-2326bd73d323\") " pod="openshift-marketplace/community-operators-8gqw8" Dec 03 13:58:15 crc kubenswrapper[5052]: I1203 13:58:15.844734 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qxvrh\" (UniqueName: \"kubernetes.io/projected/e3ece61f-eaea-4b55-abb9-2326bd73d323-kube-api-access-qxvrh\") pod \"community-operators-8gqw8\" (UID: \"e3ece61f-eaea-4b55-abb9-2326bd73d323\") " pod="openshift-marketplace/community-operators-8gqw8" Dec 03 13:58:15 crc kubenswrapper[5052]: I1203 13:58:15.844819 5052 reconciler_common.go:245] "operationExecutor.VerifyControllerAttachedVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/e3ece61f-eaea-4b55-abb9-2326bd73d323-utilities\") pod \"community-operators-8gqw8\" (UID: \"e3ece61f-eaea-4b55-abb9-2326bd73d323\") " pod="openshift-marketplace/community-operators-8gqw8" Dec 03 13:58:15 crc kubenswrapper[5052]: I1203 13:58:15.848014 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/community-operators-8gqw8"] Dec 03 13:58:15 crc kubenswrapper[5052]: I1203 13:58:15.946069 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/e3ece61f-eaea-4b55-abb9-2326bd73d323-utilities\") pod \"community-operators-8gqw8\" (UID: \"e3ece61f-eaea-4b55-abb9-2326bd73d323\") " pod="openshift-marketplace/community-operators-8gqw8" Dec 03 13:58:15 crc kubenswrapper[5052]: I1203 13:58:15.946188 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/e3ece61f-eaea-4b55-abb9-2326bd73d323-catalog-content\") pod \"community-operators-8gqw8\" (UID: \"e3ece61f-eaea-4b55-abb9-2326bd73d323\") " pod="openshift-marketplace/community-operators-8gqw8" Dec 03 13:58:15 crc kubenswrapper[5052]: I1203 13:58:15.946218 5052 reconciler_common.go:218] "operationExecutor.MountVolume started for volume \"kube-api-access-qxvrh\" (UniqueName: \"kubernetes.io/projected/e3ece61f-eaea-4b55-abb9-2326bd73d323-kube-api-access-qxvrh\") pod \"community-operators-8gqw8\" (UID: \"e3ece61f-eaea-4b55-abb9-2326bd73d323\") " pod="openshift-marketplace/community-operators-8gqw8" Dec 03 13:58:15 crc kubenswrapper[5052]: I1203 13:58:15.947186 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"utilities\" (UniqueName: \"kubernetes.io/empty-dir/e3ece61f-eaea-4b55-abb9-2326bd73d323-utilities\") pod \"community-operators-8gqw8\" (UID: \"e3ece61f-eaea-4b55-abb9-2326bd73d323\") " pod="openshift-marketplace/community-operators-8gqw8" Dec 03 13:58:15 crc kubenswrapper[5052]: I1203 13:58:15.947453 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"catalog-content\" (UniqueName: \"kubernetes.io/empty-dir/e3ece61f-eaea-4b55-abb9-2326bd73d323-catalog-content\") pod \"community-operators-8gqw8\" (UID: \"e3ece61f-eaea-4b55-abb9-2326bd73d323\") " pod="openshift-marketplace/community-operators-8gqw8" Dec 03 13:58:15 crc kubenswrapper[5052]: I1203 13:58:15.968706 5052 operation_generator.go:637] "MountVolume.SetUp succeeded for volume \"kube-api-access-qxvrh\" (UniqueName: \"kubernetes.io/projected/e3ece61f-eaea-4b55-abb9-2326bd73d323-kube-api-access-qxvrh\") pod \"community-operators-8gqw8\" (UID: \"e3ece61f-eaea-4b55-abb9-2326bd73d323\") " pod="openshift-marketplace/community-operators-8gqw8" Dec 03 13:58:16 crc kubenswrapper[5052]: I1203 13:58:16.142264 5052 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/community-operators-8gqw8" Dec 03 13:58:16 crc kubenswrapper[5052]: I1203 13:58:16.461034 5052 kubelet.go:2428] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/community-operators-8gqw8"] Dec 03 13:58:16 crc kubenswrapper[5052]: I1203 13:58:16.572926 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/community-operators-8gqw8" event={"ID":"e3ece61f-eaea-4b55-abb9-2326bd73d323","Type":"ContainerStarted","Data":"cb046ba9cb023c9d22fdd87ee8cbf7288affa6cb03c67e528d668ec7a07e5a03"} Dec 03 13:58:17 crc kubenswrapper[5052]: I1203 13:58:17.581454 5052 generic.go:334] "Generic (PLEG): container finished" podID="e3ece61f-eaea-4b55-abb9-2326bd73d323" containerID="6e50fec102ad3cd50e90da8e0cde29fef8a29b1ebede01dbd4a51989cb1cea5d" exitCode=0 Dec 03 13:58:17 crc kubenswrapper[5052]: I1203 13:58:17.581569 5052 kubelet.go:2453] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/community-operators-8gqw8" event={"ID":"e3ece61f-eaea-4b55-abb9-2326bd73d323","Type":"ContainerDied","Data":"6e50fec102ad3cd50e90da8e0cde29fef8a29b1ebede01dbd4a51989cb1cea5d"}